2023-07-18T02:31:03.283 INFO:root:teuthology version: 0.0.1.dev147+gff9991b 2023-07-18T02:31:03.284 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_rishabh@teuthology --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/dbench}} --name rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi --verbose --archive /home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342713 -- /home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342713/orig.config.yaml 2023-07-18T02:31:03.344 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-18T02:31:18.474 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342713 branch: wip-rishabh-2023Jul13-m-quota description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/dbench}} email: null first_in_suite: false job_id: '7342713' kernel: client: branch: testing flavor: default kdb: true sha1: a440c79eee0258ae07e4988d8690fe6187b8b7a3 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-rishabh-2023Jul13-m-quota ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 81cf785aa7781c4e08381c540f890cb9bc987ef9 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 81cf785aa7781c4e08381c540f890cb9bc987ef9 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-2023Jul13-m-quota sha1: 81cf785aa7781c4e08381c540f890cb9bc987ef9 owner: scheduled_rishabh@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 sha1: 81cf785aa7781c4e08381c540f890cb9bc987ef9 sleep_before_teardown: 0 suite: fs suite_branch: wip-rishabh-2023Jul13-m-quota suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 81cf785aa7781c4e08381c540f890cb9bc987ef9 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi101.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= smithi155.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi179.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1M - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6M3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b timestamp: 2023-07-18_02:11:50 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.23459 2023-07-18T02:31:18.475 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa; will attempt to use it 2023-07-18T02:31:18.476 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks 2023-07-18T02:31:18.476 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-07-18T02:31:18.479 INFO:teuthology.task.internal:Checking packages... 2023-07-18T02:31:18.512 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '81cf785aa7781c4e08381c540f890cb9bc987ef9' 2023-07-18T02:31:18.512 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-07-18T02:31:18.513 INFO:teuthology.packaging:ref: None 2023-07-18T02:31:18.513 INFO:teuthology.packaging:tag: None 2023-07-18T02:31:18.513 INFO:teuthology.packaging:branch: wip-rishabh-2023Jul13-m-quota 2023-07-18T02:31:18.513 INFO:teuthology.packaging:sha1: 81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-18T02:31:18.514 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-rishabh-2023Jul13-m-quota 2023-07-18T02:31:23.879 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-4746.g81cf785a 2023-07-18T02:31:23.879 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-07-18T02:31:23.885 INFO:teuthology.task.internal:no buildpackages task found 2023-07-18T02:31:23.886 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-07-18T02:31:23.891 INFO:teuthology.task.internal:Saving configuration 2023-07-18T02:31:23.911 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-07-18T02:31:23.916 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-18T02:31:28.887 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi101.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342713', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-18 02:24:47.806997', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-18T02:31:33.922 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi155.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342713', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-18 02:24:47.809202', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-18T02:31:38.955 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi179.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342713', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-18 02:24:47.810498', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-18T02:31:38.956 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-07-18T02:31:38.965 INFO:teuthology.task.internal:roles: ubuntu@smithi101.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-07-18T02:31:38.965 INFO:teuthology.task.internal:roles: ubuntu@smithi155.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-07-18T02:31:38.965 INFO:teuthology.task.internal:roles: ubuntu@smithi179.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-07-18T02:31:38.965 INFO:teuthology.run_tasks:Running task console_log... 2023-07-18T02:31:39.068 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fdf8503c5e0>, signals=[15]) 2023-07-18T02:31:39.069 INFO:teuthology.run_tasks:Running task internal.connect... 2023-07-18T02:31:39.082 INFO:teuthology.task.internal:Opening connections... 2023-07-18T02:31:39.083 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi101.front.sepia.ceph.com 2023-07-18T02:31:39.085 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi101.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-18T02:31:39.156 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi155.front.sepia.ceph.com 2023-07-18T02:31:39.157 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-18T02:31:39.224 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi179.front.sepia.ceph.com 2023-07-18T02:31:39.225 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-18T02:31:39.294 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-07-18T02:31:39.301 DEBUG:teuthology.orchestra.run.smithi101:> uname -m 2023-07-18T02:31:39.319 INFO:teuthology.orchestra.run.smithi101.stdout:x86_64 2023-07-18T02:31:39.320 DEBUG:teuthology.orchestra.run.smithi101:> cat /etc/os-release 2023-07-18T02:31:39.377 INFO:teuthology.orchestra.run.smithi101.stdout:NAME="Red Hat Enterprise Linux" 2023-07-18T02:31:39.377 INFO:teuthology.orchestra.run.smithi101.stdout:VERSION="8.6 (Ootpa)" 2023-07-18T02:31:39.378 INFO:teuthology.orchestra.run.smithi101.stdout:ID="rhel" 2023-07-18T02:31:39.378 INFO:teuthology.orchestra.run.smithi101.stdout:ID_LIKE="fedora" 2023-07-18T02:31:39.378 INFO:teuthology.orchestra.run.smithi101.stdout:VERSION_ID="8.6" 2023-07-18T02:31:39.378 INFO:teuthology.orchestra.run.smithi101.stdout:PLATFORM_ID="platform:el8" 2023-07-18T02:31:39.379 INFO:teuthology.orchestra.run.smithi101.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-18T02:31:39.379 INFO:teuthology.orchestra.run.smithi101.stdout:ANSI_COLOR="0;31" 2023-07-18T02:31:39.379 INFO:teuthology.orchestra.run.smithi101.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-18T02:31:39.380 INFO:teuthology.orchestra.run.smithi101.stdout:HOME_URL="https://www.redhat.com/" 2023-07-18T02:31:39.380 INFO:teuthology.orchestra.run.smithi101.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-18T02:31:39.380 INFO:teuthology.orchestra.run.smithi101.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-18T02:31:39.380 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T02:31:39.381 INFO:teuthology.orchestra.run.smithi101.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-18T02:31:39.381 INFO:teuthology.orchestra.run.smithi101.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-18T02:31:39.381 INFO:teuthology.orchestra.run.smithi101.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-18T02:31:39.381 INFO:teuthology.orchestra.run.smithi101.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-18T02:31:39.382 INFO:teuthology.lock.ops:Updating smithi101.front.sepia.ceph.com on lock server 2023-07-18T02:31:39.405 DEBUG:teuthology.orchestra.run.smithi155:> uname -m 2023-07-18T02:31:39.427 INFO:teuthology.orchestra.run.smithi155.stdout:x86_64 2023-07-18T02:31:39.427 DEBUG:teuthology.orchestra.run.smithi155:> cat /etc/os-release 2023-07-18T02:31:39.487 INFO:teuthology.orchestra.run.smithi155.stdout:NAME="Red Hat Enterprise Linux" 2023-07-18T02:31:39.487 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION="8.6 (Ootpa)" 2023-07-18T02:31:39.487 INFO:teuthology.orchestra.run.smithi155.stdout:ID="rhel" 2023-07-18T02:31:39.488 INFO:teuthology.orchestra.run.smithi155.stdout:ID_LIKE="fedora" 2023-07-18T02:31:39.488 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION_ID="8.6" 2023-07-18T02:31:39.488 INFO:teuthology.orchestra.run.smithi155.stdout:PLATFORM_ID="platform:el8" 2023-07-18T02:31:39.489 INFO:teuthology.orchestra.run.smithi155.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-18T02:31:39.489 INFO:teuthology.orchestra.run.smithi155.stdout:ANSI_COLOR="0;31" 2023-07-18T02:31:39.489 INFO:teuthology.orchestra.run.smithi155.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-18T02:31:39.490 INFO:teuthology.orchestra.run.smithi155.stdout:HOME_URL="https://www.redhat.com/" 2023-07-18T02:31:39.490 INFO:teuthology.orchestra.run.smithi155.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-18T02:31:39.490 INFO:teuthology.orchestra.run.smithi155.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-18T02:31:39.490 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-07-18T02:31:39.491 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-18T02:31:39.491 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-18T02:31:39.491 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-18T02:31:39.491 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-18T02:31:39.491 INFO:teuthology.lock.ops:Updating smithi155.front.sepia.ceph.com on lock server 2023-07-18T02:31:39.513 DEBUG:teuthology.orchestra.run.smithi179:> uname -m 2023-07-18T02:31:39.532 INFO:teuthology.orchestra.run.smithi179.stdout:x86_64 2023-07-18T02:31:39.532 DEBUG:teuthology.orchestra.run.smithi179:> cat /etc/os-release 2023-07-18T02:31:39.590 INFO:teuthology.orchestra.run.smithi179.stdout:NAME="Red Hat Enterprise Linux" 2023-07-18T02:31:39.590 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION="8.6 (Ootpa)" 2023-07-18T02:31:39.591 INFO:teuthology.orchestra.run.smithi179.stdout:ID="rhel" 2023-07-18T02:31:39.591 INFO:teuthology.orchestra.run.smithi179.stdout:ID_LIKE="fedora" 2023-07-18T02:31:39.591 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION_ID="8.6" 2023-07-18T02:31:39.591 INFO:teuthology.orchestra.run.smithi179.stdout:PLATFORM_ID="platform:el8" 2023-07-18T02:31:39.591 INFO:teuthology.orchestra.run.smithi179.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-18T02:31:39.591 INFO:teuthology.orchestra.run.smithi179.stdout:ANSI_COLOR="0;31" 2023-07-18T02:31:39.591 INFO:teuthology.orchestra.run.smithi179.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-18T02:31:39.592 INFO:teuthology.orchestra.run.smithi179.stdout:HOME_URL="https://www.redhat.com/" 2023-07-18T02:31:39.592 INFO:teuthology.orchestra.run.smithi179.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-18T02:31:39.592 INFO:teuthology.orchestra.run.smithi179.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-18T02:31:39.592 INFO:teuthology.orchestra.run.smithi179.stdout: 2023-07-18T02:31:39.592 INFO:teuthology.orchestra.run.smithi179.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-18T02:31:39.592 INFO:teuthology.orchestra.run.smithi179.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-18T02:31:39.593 INFO:teuthology.orchestra.run.smithi179.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-18T02:31:39.593 INFO:teuthology.orchestra.run.smithi179.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-18T02:31:39.593 INFO:teuthology.lock.ops:Updating smithi179.front.sepia.ceph.com on lock server 2023-07-18T02:31:39.613 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-07-18T02:31:39.639 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-07-18T02:31:39.647 INFO:teuthology.task.internal:Checking for old test directory... 2023-07-18T02:31:39.648 DEBUG:teuthology.orchestra.run.smithi101:> test '!' -e /home/ubuntu/cephtest 2023-07-18T02:31:39.651 DEBUG:teuthology.orchestra.run.smithi155:> test '!' -e /home/ubuntu/cephtest 2023-07-18T02:31:39.654 DEBUG:teuthology.orchestra.run.smithi179:> test '!' -e /home/ubuntu/cephtest 2023-07-18T02:31:39.671 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-07-18T02:31:39.680 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-07-18T02:31:39.681 DEBUG:teuthology.orchestra.run.smithi101:> test -z $(ls -A /var/lib/ceph) 2023-07-18T02:31:39.710 DEBUG:teuthology.orchestra.run.smithi155:> test -z $(ls -A /var/lib/ceph) 2023-07-18T02:31:39.712 DEBUG:teuthology.orchestra.run.smithi179:> test -z $(ls -A /var/lib/ceph) 2023-07-18T02:31:39.742 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-07-18T02:31:39.831 INFO:teuthology.run_tasks:Running task kernel... 2023-07-18T02:31:39.847 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': 'a440c79eee0258ae07e4988d8690fe6187b8b7a3'} 2023-07-18T02:31:39.848 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2023-07-18T02:31:39.848 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2023-07-18T02:31:39.849 DEBUG:teuthology.orchestra.run.smithi101:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-18T02:31:39.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-18T02:31:39.868 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-07-18T02:31:40.121 DEBUG:teuthology.task.kernel:sha1 for client.0 is a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-18T02:31:40.121 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "a440c79eee0258ae07e4988d8690fe6187b8b7a3"... 2023-07-18T02:31:40.122 DEBUG:teuthology.orchestra.run.smithi101:> uname -r 2023-07-18T02:31:40.141 INFO:teuthology.orchestra.run.smithi101.stdout:4.18.0-372.9.1.el8.x86_64 2023-07-18T02:31:40.141 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-18T02:31:40.142 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-07-18T02:31:40.142 DEBUG:teuthology.orchestra.run.smithi101:> sudo yum upgrade -y linux-firmware 2023-07-18T02:31:41.407 INFO:teuthology.orchestra.run.smithi101.stdout:Updating Subscription Management repositories. 2023-07-18T02:31:41.408 INFO:teuthology.orchestra.run.smithi101.stdout:Unable to read consumer identity 2023-07-18T02:31:43.877 INFO:teuthology.orchestra.run.smithi101.stdout:Copr repo for python3-asyncssh owned by ceph 2.9 kB/s | 3.5 kB 00:01 2023-07-18T02:31:48.335 INFO:teuthology.orchestra.run.smithi101.stdout:Extra Packages for Enterprise Linux 3.5 MB/s | 16 MB 00:04 2023-07-18T02:31:51.769 INFO:teuthology.orchestra.run.smithi101.stdout:lab-extras 610 kB/s | 24 kB 00:00 2023-07-18T02:31:52.529 INFO:teuthology.orchestra.run.smithi101.stdout:Dependencies resolved. 2023-07-18T02:31:52.530 INFO:teuthology.orchestra.run.smithi101.stdout:Nothing to do. 2023-07-18T02:31:52.530 INFO:teuthology.orchestra.run.smithi101.stdout:Complete! 2023-07-18T02:31:52.649 INFO:teuthology.task.kernel:Downloading kernel a440c79eee0258ae07e4988d8690fe6187b8b7a3 on client.0... 2023-07-18T02:31:52.650 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-18T02:31:57.897 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/a440c79eee0258ae07e4988d8690fe6187b8b7a3/centos/8/flavors/default/x86_64/ 2023-07-18T02:31:57.897 DEBUG:teuthology.orchestra.run.smithi101:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc1_ga440c79eee02-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/a440c79eee0258ae07e4988d8690fe6187b8b7a3/centos/8/flavors/default/x86_64/ --input-file=- 2023-07-18T02:31:57.900 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi101.front.sepia.ceph.com to complete... 2023-07-18T02:32:39.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18 02:32:39 URL:https://2.chacra.ceph.com/r/kernel/testing/a440c79eee0258ae07e4988d8690fe6187b8b7a3/centos/8/flavors/default/x86_64/kernel-6.5.0_rc1_ga440c79eee02-1.x86_64.rpm [380684764/380684764] -> "/tmp/kernel.x86_64.rpm" [1] 2023-07-18T02:32:39.475 INFO:teuthology.orchestra.run.smithi101.stderr:FINISHED --2023-07-18 02:32:39-- 2023-07-18T02:32:39.475 INFO:teuthology.orchestra.run.smithi101.stderr:Total wall clock time: 42s 2023-07-18T02:32:39.475 INFO:teuthology.orchestra.run.smithi101.stderr:Downloaded: 1 files, 363M in 41s (8.80 MB/s) 2023-07-18T02:32:39.476 INFO:teuthology.task.kernel:Installing kernel a440c79eee0258ae07e4988d8690fe6187b8b7a3 on client.0... 2023-07-18T02:32:39.477 DEBUG:teuthology.orchestra.run.smithi101:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-07-18T02:32:41.063 INFO:teuthology.orchestra.run.smithi101.stdout:Verifying... ######################################## 2023-07-18T02:32:41.115 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing... ######################################## 2023-07-18T02:32:41.204 INFO:teuthology.orchestra.run.smithi101.stdout:Updating / installing... 2023-07-18T02:33:19.403 INFO:teuthology.orchestra.run.smithi101.stdout:kernel-6.5.0_rc1_ga440c79eee02-1 ######################################## 2023-07-18T02:33:46.647 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi101.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-07-18T02:33:46.648 DEBUG:teuthology.orchestra.run.smithi101:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-07-18T02:33:46.697 INFO:teuthology.orchestra.run.smithi101.stdout:/boot/System.map-6.5.0-rc1-ga440c79eee02 2023-07-18T02:33:46.697 INFO:teuthology.orchestra.run.smithi101.stdout:/boot/config-6.5.0-rc1-ga440c79eee02 2023-07-18T02:33:46.697 INFO:teuthology.orchestra.run.smithi101.stdout:/boot/vmlinuz-6.5.0-rc1-ga440c79eee02 2023-07-18T02:33:46.698 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02 2023-07-18T02:33:46.698 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel 2023-07-18T02:33:46.698 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch 2023-07-18T02:33:46.698 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86 2023-07-18T02:33:46.698 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto 2023-07-18T02:33:46.698 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-07-18T02:33:46.699 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-07-18T02:33:46.699 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-07-18T02:33:46.699 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-07-18T02:33:46.699 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-07-18T02:33:46.699 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-07-18T02:33:46.700 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-07-18T02:33:46.700 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/crc32c-intel.ko 2023-07-18T02:33:46.700 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-07-18T02:33:46.700 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-07-18T02:33:46.700 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-07-18T02:33:46.700 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-07-18T02:33:46.701 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/serpent-avx2.ko 2023-07-18T02:33:46.701 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-07-18T02:33:46.701 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-07-18T02:33:46.701 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-07-18T02:33:46.701 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-07-18T02:33:46.701 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-07-18T02:33:46.701 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-07-18T02:33:46.701 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-07-18T02:33:46.702 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel 2023-07-18T02:33:46.702 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/cpu 2023-07-18T02:33:46.702 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/cpu/mce 2023-07-18T02:33:46.702 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-07-18T02:33:46.702 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm 2023-07-18T02:33:46.702 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm/kvm-amd.ko 2023-07-18T02:33:46.702 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm/kvm-intel.ko 2023-07-18T02:33:46.703 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm/kvm.ko 2023-07-18T02:33:46.703 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto 2023-07-18T02:33:46.703 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/ansi_cprng.ko 2023-07-18T02:33:46.703 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/anubis.ko 2023-07-18T02:33:46.703 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/arc4.ko 2023-07-18T02:33:46.703 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx 2023-07-18T02:33:46.703 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_memcpy.ko 2023-07-18T02:33:46.704 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_pq.ko 2023-07-18T02:33:46.704 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_raid6_recov.ko 2023-07-18T02:33:46.704 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_tx.ko 2023-07-18T02:33:46.704 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_xor.ko 2023-07-18T02:33:46.704 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/raid6test.ko 2023-07-18T02:33:46.705 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/authenc.ko 2023-07-18T02:33:46.705 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/authencesn.ko 2023-07-18T02:33:46.705 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/blake2b_generic.ko 2023-07-18T02:33:46.705 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/blowfish_common.ko 2023-07-18T02:33:46.705 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/blowfish_generic.ko 2023-07-18T02:33:46.705 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/camellia_generic.ko 2023-07-18T02:33:46.706 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cast5_generic.ko 2023-07-18T02:33:46.706 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cast6_generic.ko 2023-07-18T02:33:46.706 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cast_common.ko 2023-07-18T02:33:46.706 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/ccm.ko 2023-07-18T02:33:46.706 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cmac.ko 2023-07-18T02:33:46.706 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/crc32_generic.ko 2023-07-18T02:33:46.706 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/crypto_engine.ko 2023-07-18T02:33:46.706 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/crypto_user.ko 2023-07-18T02:33:46.707 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cts.ko 2023-07-18T02:33:46.707 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/des_generic.ko 2023-07-18T02:33:46.707 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/dh_generic.ko 2023-07-18T02:33:46.707 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/echainiv.ko 2023-07-18T02:33:46.707 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/essiv.ko 2023-07-18T02:33:46.707 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/fcrypt.ko 2023-07-18T02:33:46.707 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/khazad.ko 2023-07-18T02:33:46.708 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/lz4.ko 2023-07-18T02:33:46.708 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/lz4hc.ko 2023-07-18T02:33:46.708 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/md4.ko 2023-07-18T02:33:46.708 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/michael_mic.ko 2023-07-18T02:33:46.708 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/pcbc.ko 2023-07-18T02:33:46.708 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/pcrypt.ko 2023-07-18T02:33:46.709 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/rmd160.ko 2023-07-18T02:33:46.709 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/seed.ko 2023-07-18T02:33:46.709 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/serpent_generic.ko 2023-07-18T02:33:46.709 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/tcrypt.ko 2023-07-18T02:33:46.710 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/tea.ko 2023-07-18T02:33:46.710 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/twofish_common.ko 2023-07-18T02:33:46.710 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/twofish_generic.ko 2023-07-18T02:33:46.711 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/vmac.ko 2023-07-18T02:33:46.711 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/wp512.ko 2023-07-18T02:33:46.711 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/xcbc.ko 2023-07-18T02:33:46.711 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/xor.ko 2023-07-18T02:33:46.712 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/xxhash_generic.ko 2023-07-18T02:33:46.712 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers 2023-07-18T02:33:46.712 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi 2023-07-18T02:33:46.713 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/acpi_ipmi.ko 2023-07-18T02:33:46.713 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/acpi_pad.ko 2023-07-18T02:33:46.713 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/custom_method.ko 2023-07-18T02:33:46.713 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/ec_sys.ko 2023-07-18T02:33:46.714 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/platform_profile.ko 2023-07-18T02:33:46.715 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/sbs.ko 2023-07-18T02:33:46.715 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/sbshc.ko 2023-07-18T02:33:46.716 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/video.ko 2023-07-18T02:33:46.716 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata 2023-07-18T02:33:46.716 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/acard-ahci.ko 2023-07-18T02:33:46.716 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/ahci_platform.ko 2023-07-18T02:33:46.717 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/ata_generic.ko 2023-07-18T02:33:46.717 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/libahci_platform.ko 2023-07-18T02:33:46.717 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_acpi.ko 2023-07-18T02:33:46.717 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ali.ko 2023-07-18T02:33:46.717 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_amd.ko 2023-07-18T02:33:46.718 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_artop.ko 2023-07-18T02:33:46.718 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_atiixp.ko 2023-07-18T02:33:46.718 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_atp867x.ko 2023-07-18T02:33:46.719 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_cmd640.ko 2023-07-18T02:33:46.719 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_cmd64x.ko 2023-07-18T02:33:46.719 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_cypress.ko 2023-07-18T02:33:46.719 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_efar.ko 2023-07-18T02:33:46.720 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt366.ko 2023-07-18T02:33:46.720 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt37x.ko 2023-07-18T02:33:46.720 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt3x2n.ko 2023-07-18T02:33:46.720 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt3x3.ko 2023-07-18T02:33:46.720 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_it8213.ko 2023-07-18T02:33:46.720 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_it821x.ko 2023-07-18T02:33:46.720 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_jmicron.ko 2023-07-18T02:33:46.721 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_marvell.ko 2023-07-18T02:33:46.721 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_mpiix.ko 2023-07-18T02:33:46.721 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_netcell.ko 2023-07-18T02:33:46.721 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ninja32.ko 2023-07-18T02:33:46.721 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ns87410.ko 2023-07-18T02:33:46.721 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ns87415.ko 2023-07-18T02:33:46.721 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_oldpiix.ko 2023-07-18T02:33:46.722 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_opti.ko 2023-07-18T02:33:46.722 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_optidma.ko 2023-07-18T02:33:46.722 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_pcmcia.ko 2023-07-18T02:33:46.722 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_pdc2027x.ko 2023-07-18T02:33:46.722 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-07-18T02:33:46.722 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_piccolo.ko 2023-07-18T02:33:46.722 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_rdc.ko 2023-07-18T02:33:46.723 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sch.ko 2023-07-18T02:33:46.723 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_serverworks.ko 2023-07-18T02:33:46.723 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sil680.ko 2023-07-18T02:33:46.723 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sis.ko 2023-07-18T02:33:46.723 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sl82c105.ko 2023-07-18T02:33:46.723 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_triflex.ko 2023-07-18T02:33:46.724 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_via.ko 2023-07-18T02:33:46.724 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pdc_adma.ko 2023-07-18T02:33:46.724 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_inic162x.ko 2023-07-18T02:33:46.724 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_mv.ko 2023-07-18T02:33:46.724 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_nv.ko 2023-07-18T02:33:46.724 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_promise.ko 2023-07-18T02:33:46.724 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_qstor.ko 2023-07-18T02:33:46.725 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sil.ko 2023-07-18T02:33:46.725 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sil24.ko 2023-07-18T02:33:46.725 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sis.ko 2023-07-18T02:33:46.725 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_svw.ko 2023-07-18T02:33:46.725 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sx4.ko 2023-07-18T02:33:46.725 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_uli.ko 2023-07-18T02:33:46.725 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_via.ko 2023-07-18T02:33:46.725 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_vsc.ko 2023-07-18T02:33:46.726 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/auxdisplay 2023-07-18T02:33:46.726 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/auxdisplay/ks0108.ko 2023-07-18T02:33:46.726 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base 2023-07-18T02:33:46.726 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base/regmap 2023-07-18T02:33:46.726 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base/regmap/regmap-i2c.ko 2023-07-18T02:33:46.726 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base/regmap/regmap-mmio.ko 2023-07-18T02:33:46.726 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/bcma 2023-07-18T02:33:46.727 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/bcma/bcma.ko 2023-07-18T02:33:46.727 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block 2023-07-18T02:33:46.727 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/aoe 2023-07-18T02:33:46.727 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/aoe/aoe.ko 2023-07-18T02:33:46.727 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/brd.ko 2023-07-18T02:33:46.727 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/drbd 2023-07-18T02:33:46.727 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/drbd/drbd.ko 2023-07-18T02:33:46.727 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/floppy.ko 2023-07-18T02:33:46.728 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/loop.ko 2023-07-18T02:33:46.728 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/mtip32xx 2023-07-18T02:33:46.728 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-07-18T02:33:46.728 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/nbd.ko 2023-07-18T02:33:46.728 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/null_blk 2023-07-18T02:33:46.728 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/null_blk/null_blk.ko 2023-07-18T02:33:46.728 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/pktcdvd.ko 2023-07-18T02:33:46.729 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/rbd.ko 2023-07-18T02:33:46.729 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/virtio_blk.ko 2023-07-18T02:33:46.729 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/xen-blkback 2023-07-18T02:33:46.729 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-07-18T02:33:46.729 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/xen-blkfront.ko 2023-07-18T02:33:46.729 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/zram 2023-07-18T02:33:46.729 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/zram/zram.ko 2023-07-18T02:33:46.730 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char 2023-07-18T02:33:46.730 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hangcheck-timer.ko 2023-07-18T02:33:46.730 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random 2023-07-18T02:33:46.730 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/amd-rng.ko 2023-07-18T02:33:46.730 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/intel-rng.ko 2023-07-18T02:33:46.730 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-07-18T02:33:46.730 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/via-rng.ko 2023-07-18T02:33:46.730 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/virtio-rng.ko 2023-07-18T02:33:46.731 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi 2023-07-18T02:33:46.731 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-07-18T02:33:46.731 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-07-18T02:33:46.731 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-07-18T02:33:46.731 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_si.ko 2023-07-18T02:33:46.731 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-07-18T02:33:46.731 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-07-18T02:33:46.732 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/lp.ko 2023-07-18T02:33:46.732 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/mwave 2023-07-18T02:33:46.732 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/mwave/mwave.ko 2023-07-18T02:33:46.732 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ppdev.ko 2023-07-18T02:33:46.732 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tlclk.ko 2023-07-18T02:33:46.732 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm 2023-07-18T02:33:46.732 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm.ko 2023-07-18T02:33:46.733 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_atmel.ko 2023-07-18T02:33:46.733 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_infineon.ko 2023-07-18T02:33:46.733 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_nsc.ko 2023-07-18T02:33:46.733 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_tis.ko 2023-07-18T02:33:46.733 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-07-18T02:33:46.733 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/uv_mmtimer.ko 2023-07-18T02:33:46.733 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/virtio_console.ko 2023-07-18T02:33:46.733 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq 2023-07-18T02:33:46.734 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-07-18T02:33:46.734 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-07-18T02:33:46.734 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/p4-clockmod.ko 2023-07-18T02:33:46.734 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-07-18T02:33:46.734 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/powernow-k8.ko 2023-07-18T02:33:46.734 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/speedstep-lib.ko 2023-07-18T02:33:46.734 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto 2023-07-18T02:33:46.734 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/ccp 2023-07-18T02:33:46.735 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-07-18T02:33:46.735 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/ccp/ccp.ko 2023-07-18T02:33:46.735 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel 2023-07-18T02:33:46.735 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat 2023-07-18T02:33:46.735 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_common 2023-07-18T02:33:46.735 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-07-18T02:33:46.735 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-07-18T02:33:46.736 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-07-18T02:33:46.736 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/padlock-aes.ko 2023-07-18T02:33:46.736 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/padlock-sha.ko 2023-07-18T02:33:46.736 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/virtio 2023-07-18T02:33:46.736 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-07-18T02:33:46.736 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dca 2023-07-18T02:33:46.736 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dca/dca.ko 2023-07-18T02:33:46.737 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/devfreq 2023-07-18T02:33:46.737 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-07-18T02:33:46.737 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma 2023-07-18T02:33:46.737 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/dw 2023-07-18T02:33:46.737 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/dw/dw_dmac.ko 2023-07-18T02:33:46.737 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/ioat 2023-07-18T02:33:46.737 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/ioat/ioatdma.ko 2023-07-18T02:33:46.737 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac 2023-07-18T02:33:46.738 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/amd64_edac.ko 2023-07-18T02:33:46.738 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/e752x_edac.ko 2023-07-18T02:33:46.738 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/edac_mce_amd.ko 2023-07-18T02:33:46.738 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i3000_edac.ko 2023-07-18T02:33:46.738 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i3200_edac.ko 2023-07-18T02:33:46.739 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i5100_edac.ko 2023-07-18T02:33:46.739 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i5400_edac.ko 2023-07-18T02:33:46.739 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i7300_edac.ko 2023-07-18T02:33:46.739 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i7core_edac.ko 2023-07-18T02:33:46.739 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i82975x_edac.ko 2023-07-18T02:33:46.739 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/ie31200_edac.ko 2023-07-18T02:33:46.739 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/sb_edac.ko 2023-07-18T02:33:46.740 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/x38_edac.ko 2023-07-18T02:33:46.740 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/firmware 2023-07-18T02:33:46.740 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/firmware/edd.ko 2023-07-18T02:33:46.740 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/firmware/iscsi_ibft.ko 2023-07-18T02:33:46.740 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio 2023-07-18T02:33:46.740 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-ich.ko 2023-07-18T02:33:46.740 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-viperboard.ko 2023-07-18T02:33:46.741 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid 2023-07-18T02:33:46.741 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-appleir.ko 2023-07-18T02:33:46.741 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-aureal.ko 2023-07-18T02:33:46.741 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-axff.ko 2023-07-18T02:33:46.741 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-dr.ko 2023-07-18T02:33:46.741 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-elecom.ko 2023-07-18T02:33:46.741 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-elo.ko 2023-07-18T02:33:46.741 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-emsff.ko 2023-07-18T02:33:46.742 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-gaff.ko 2023-07-18T02:33:46.742 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-gt683r.ko 2023-07-18T02:33:46.742 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-gyration.ko 2023-07-18T02:33:46.742 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-holtek-kbd.ko 2023-07-18T02:33:46.742 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-holtek-mouse.ko 2023-07-18T02:33:46.742 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-holtekff.ko 2023-07-18T02:33:46.742 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-hyperv.ko 2023-07-18T02:33:46.742 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-icade.ko 2023-07-18T02:33:46.743 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-keytouch.ko 2023-07-18T02:33:46.743 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-kye.ko 2023-07-18T02:33:46.743 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-lcpower.ko 2023-07-18T02:33:46.743 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-led.ko 2023-07-18T02:33:46.743 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-lenovo.ko 2023-07-18T02:33:46.743 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-logitech-dj.ko 2023-07-18T02:33:46.743 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-07-18T02:33:46.744 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-multitouch.ko 2023-07-18T02:33:46.744 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-ortek.ko 2023-07-18T02:33:46.744 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-penmount.ko 2023-07-18T02:33:46.744 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-petalynx.ko 2023-07-18T02:33:46.744 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-picolcd.ko 2023-07-18T02:33:46.744 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-pl.ko 2023-07-18T02:33:46.744 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-plantronics.ko 2023-07-18T02:33:46.744 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-primax.ko 2023-07-18T02:33:46.745 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-rmi.ko 2023-07-18T02:33:46.745 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-arvo.ko 2023-07-18T02:33:46.745 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-common.ko 2023-07-18T02:33:46.745 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-isku.ko 2023-07-18T02:33:46.745 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-kone.ko 2023-07-18T02:33:46.745 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-07-18T02:33:46.745 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-konepure.ko 2023-07-18T02:33:46.746 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-07-18T02:33:46.746 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-lua.ko 2023-07-18T02:33:46.746 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-pyra.ko 2023-07-18T02:33:46.746 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-ryos.ko 2023-07-18T02:33:46.746 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-savu.ko 2023-07-18T02:33:46.746 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat.ko 2023-07-18T02:33:46.746 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-saitek.ko 2023-07-18T02:33:46.746 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-samsung.ko 2023-07-18T02:33:46.747 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sensor-hub.ko 2023-07-18T02:33:46.747 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sjoy.ko 2023-07-18T02:33:46.747 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sony.ko 2023-07-18T02:33:46.747 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-speedlink.ko 2023-07-18T02:33:46.747 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-steelseries.ko 2023-07-18T02:33:46.747 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sunplus.ko 2023-07-18T02:33:46.747 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-thrustmaster.ko 2023-07-18T02:33:46.747 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-tivo.ko 2023-07-18T02:33:46.748 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-tmff.ko 2023-07-18T02:33:46.748 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-topseed.ko 2023-07-18T02:33:46.748 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-twinhan.ko 2023-07-18T02:33:46.748 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-uclogic.ko 2023-07-18T02:33:46.748 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-waltop.ko 2023-07-18T02:33:46.748 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-wiimote.ko 2023-07-18T02:33:46.748 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-xinmo.ko 2023-07-18T02:33:46.749 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-zpff.ko 2023-07-18T02:33:46.749 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-zydacron.ko 2023-07-18T02:33:46.749 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/uhid.ko 2023-07-18T02:33:46.749 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/wacom.ko 2023-07-18T02:33:46.749 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv 2023-07-18T02:33:46.749 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv/hv_balloon.ko 2023-07-18T02:33:46.749 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv/hv_utils.ko 2023-07-18T02:33:46.750 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv/hv_vmbus.ko 2023-07-18T02:33:46.750 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon 2023-07-18T02:33:46.750 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/abituguru.ko 2023-07-18T02:33:46.750 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/abituguru3.ko 2023-07-18T02:33:46.750 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/acpi_power_meter.ko 2023-07-18T02:33:46.750 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ad7414.ko 2023-07-18T02:33:46.750 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ad7418.ko 2023-07-18T02:33:46.750 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adc128d818.ko 2023-07-18T02:33:46.751 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1025.ko 2023-07-18T02:33:46.751 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1026.ko 2023-07-18T02:33:46.751 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1029.ko 2023-07-18T02:33:46.751 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1031.ko 2023-07-18T02:33:46.751 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm9240.ko 2023-07-18T02:33:46.751 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ads7828.ko 2023-07-18T02:33:46.751 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7410.ko 2023-07-18T02:33:46.751 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7411.ko 2023-07-18T02:33:46.752 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7462.ko 2023-07-18T02:33:46.752 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7470.ko 2023-07-18T02:33:46.752 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7475.ko 2023-07-18T02:33:46.752 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7x10.ko 2023-07-18T02:33:46.752 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/amc6821.ko 2023-07-18T02:33:46.752 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/applesmc.ko 2023-07-18T02:33:46.752 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/asb100.ko 2023-07-18T02:33:46.753 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/asc7621.ko 2023-07-18T02:33:46.753 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/asus_atk0110.ko 2023-07-18T02:33:46.753 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/atxp1.ko 2023-07-18T02:33:46.753 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/coretemp.ko 2023-07-18T02:33:46.753 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-07-18T02:33:46.753 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/dme1737.ko 2023-07-18T02:33:46.753 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ds1621.ko 2023-07-18T02:33:46.754 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ds620.ko 2023-07-18T02:33:46.754 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/emc1403.ko 2023-07-18T02:33:46.754 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/emc6w201.ko 2023-07-18T02:33:46.754 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/f71805f.ko 2023-07-18T02:33:46.754 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/f71882fg.ko 2023-07-18T02:33:46.754 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/f75375s.ko 2023-07-18T02:33:46.754 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/fam15h_power.ko 2023-07-18T02:33:46.754 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/fschmd.ko 2023-07-18T02:33:46.755 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/g760a.ko 2023-07-18T02:33:46.755 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/g762.ko 2023-07-18T02:33:46.755 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/gl518sm.ko 2023-07-18T02:33:46.755 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/gl520sm.ko 2023-07-18T02:33:46.755 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/hwmon-vid.ko 2023-07-18T02:33:46.755 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/i5500_temp.ko 2023-07-18T02:33:46.755 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/i5k_amb.ko 2023-07-18T02:33:46.756 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ibmaem.ko 2023-07-18T02:33:46.756 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ibmpex.ko 2023-07-18T02:33:46.756 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ina209.ko 2023-07-18T02:33:46.756 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ina2xx.ko 2023-07-18T02:33:46.756 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/it87.ko 2023-07-18T02:33:46.756 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/k10temp.ko 2023-07-18T02:33:46.756 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/k8temp.ko 2023-07-18T02:33:46.756 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lineage-pem.ko 2023-07-18T02:33:46.757 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm63.ko 2023-07-18T02:33:46.757 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm73.ko 2023-07-18T02:33:46.757 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm75.ko 2023-07-18T02:33:46.757 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm77.ko 2023-07-18T02:33:46.757 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm78.ko 2023-07-18T02:33:46.757 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm80.ko 2023-07-18T02:33:46.757 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm83.ko 2023-07-18T02:33:46.758 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm85.ko 2023-07-18T02:33:46.758 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm87.ko 2023-07-18T02:33:46.758 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm90.ko 2023-07-18T02:33:46.758 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm92.ko 2023-07-18T02:33:46.758 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm93.ko 2023-07-18T02:33:46.758 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm95234.ko 2023-07-18T02:33:46.758 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm95241.ko 2023-07-18T02:33:46.758 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm95245.ko 2023-07-18T02:33:46.759 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc2945.ko 2023-07-18T02:33:46.759 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4151.ko 2023-07-18T02:33:46.759 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4215.ko 2023-07-18T02:33:46.759 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4222.ko 2023-07-18T02:33:46.759 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4245.ko 2023-07-18T02:33:46.759 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4260.ko 2023-07-18T02:33:46.759 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4261.ko 2023-07-18T02:33:46.759 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max16065.ko 2023-07-18T02:33:46.760 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max1619.ko 2023-07-18T02:33:46.760 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max1668.ko 2023-07-18T02:33:46.760 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max197.ko 2023-07-18T02:33:46.760 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max6639.ko 2023-07-18T02:33:46.760 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max6650.ko 2023-07-18T02:33:46.760 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max6697.ko 2023-07-18T02:33:46.760 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/mcp3021.ko 2023-07-18T02:33:46.761 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct6683.ko 2023-07-18T02:33:46.761 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct6775-core.ko 2023-07-18T02:33:46.761 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct6775.ko 2023-07-18T02:33:46.761 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct7802.ko 2023-07-18T02:33:46.761 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pc87360.ko 2023-07-18T02:33:46.761 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pc87427.ko 2023-07-18T02:33:46.761 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pcf8591.ko 2023-07-18T02:33:46.761 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus 2023-07-18T02:33:46.762 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-07-18T02:33:46.762 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-07-18T02:33:46.762 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-07-18T02:33:46.762 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/max16064.ko 2023-07-18T02:33:46.762 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/max34440.ko 2023-07-18T02:33:46.762 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/max8688.ko 2023-07-18T02:33:46.762 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-07-18T02:33:46.763 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-07-18T02:33:46.763 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-07-18T02:33:46.763 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-07-18T02:33:46.763 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-07-18T02:33:46.763 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-07-18T02:33:46.763 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/powr1220.ko 2023-07-18T02:33:46.763 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sch5627.ko 2023-07-18T02:33:46.763 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sch5636.ko 2023-07-18T02:33:46.764 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sch56xx-common.ko 2023-07-18T02:33:46.764 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sht15.ko 2023-07-18T02:33:46.764 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sht21.ko 2023-07-18T02:33:46.764 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/shtc1.ko 2023-07-18T02:33:46.764 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sis5595.ko 2023-07-18T02:33:46.764 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/smsc47b397.ko 2023-07-18T02:33:46.764 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/smsc47m1.ko 2023-07-18T02:33:46.765 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/smsc47m192.ko 2023-07-18T02:33:46.765 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/thmc50.ko 2023-07-18T02:33:46.765 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp102.ko 2023-07-18T02:33:46.765 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp103.ko 2023-07-18T02:33:46.765 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp401.ko 2023-07-18T02:33:46.765 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp421.ko 2023-07-18T02:33:46.765 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/via-cputemp.ko 2023-07-18T02:33:46.765 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/via686a.ko 2023-07-18T02:33:46.766 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/vt1211.ko 2023-07-18T02:33:46.766 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/vt8231.ko 2023-07-18T02:33:46.766 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83627ehf.ko 2023-07-18T02:33:46.766 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83627hf.ko 2023-07-18T02:33:46.766 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83781d.ko 2023-07-18T02:33:46.766 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83791d.ko 2023-07-18T02:33:46.766 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83792d.ko 2023-07-18T02:33:46.766 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83793.ko 2023-07-18T02:33:46.767 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83795.ko 2023-07-18T02:33:46.767 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83l785ts.ko 2023-07-18T02:33:46.767 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83l786ng.ko 2023-07-18T02:33:46.767 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c 2023-07-18T02:33:46.767 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/algos 2023-07-18T02:33:46.767 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-07-18T02:33:46.767 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-07-18T02:33:46.768 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses 2023-07-18T02:33:46.768 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-07-18T02:33:46.768 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-07-18T02:33:46.768 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-07-18T02:33:46.768 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-07-18T02:33:46.768 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-07-18T02:33:46.768 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-07-18T02:33:46.768 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-07-18T02:33:46.769 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-07-18T02:33:46.769 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-i801.ko 2023-07-18T02:33:46.769 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-isch.ko 2023-07-18T02:33:46.769 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-07-18T02:33:46.770 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-07-18T02:33:46.770 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-07-18T02:33:46.770 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-parport.ko 2023-07-18T02:33:46.770 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-07-18T02:33:46.770 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-07-18T02:33:46.770 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-07-18T02:33:46.770 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-07-18T02:33:46.771 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-07-18T02:33:46.771 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-07-18T02:33:46.771 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-via.ko 2023-07-18T02:33:46.771 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-07-18T02:33:46.771 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-07-18T02:33:46.771 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-dev.ko 2023-07-18T02:33:46.771 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-mux.ko 2023-07-18T02:33:46.771 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-07-18T02:33:46.772 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-smbus.ko 2023-07-18T02:33:46.772 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-stub.ko 2023-07-18T02:33:46.772 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband 2023-07-18T02:33:46.772 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core 2023-07-18T02:33:46.772 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_cm.ko 2023-07-18T02:33:46.772 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_core.ko 2023-07-18T02:33:46.772 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_umad.ko 2023-07-18T02:33:46.773 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-07-18T02:33:46.773 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/iw_cm.ko 2023-07-18T02:33:46.773 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/rdma_cm.ko 2023-07-18T02:33:46.773 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-07-18T02:33:46.773 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw 2023-07-18T02:33:46.774 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/cxgb4 2023-07-18T02:33:46.774 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-07-18T02:33:46.774 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx4 2023-07-18T02:33:46.774 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-07-18T02:33:46.775 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx5 2023-07-18T02:33:46.775 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-07-18T02:33:46.775 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mthca 2023-07-18T02:33:46.776 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-07-18T02:33:46.776 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp 2023-07-18T02:33:46.776 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/ipoib 2023-07-18T02:33:46.776 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-07-18T02:33:46.777 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/iser 2023-07-18T02:33:46.777 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-07-18T02:33:46.777 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/isert 2023-07-18T02:33:46.778 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-07-18T02:33:46.778 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srp 2023-07-18T02:33:46.778 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-07-18T02:33:46.779 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srpt 2023-07-18T02:33:46.779 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-07-18T02:33:46.779 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input 2023-07-18T02:33:46.780 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport 2023-07-18T02:33:46.780 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-07-18T02:33:46.780 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/fm801-gp.ko 2023-07-18T02:33:46.781 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/gameport.ko 2023-07-18T02:33:46.781 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/lightning.ko 2023-07-18T02:33:46.781 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/ns558.ko 2023-07-18T02:33:46.781 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/joydev.ko 2023-07-18T02:33:46.782 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard 2023-07-18T02:33:46.782 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/gpio_keys.ko 2023-07-18T02:33:46.782 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse 2023-07-18T02:33:46.782 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/appletouch.ko 2023-07-18T02:33:46.782 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/bcm5974.ko 2023-07-18T02:33:46.783 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/cyapatp.ko 2023-07-18T02:33:46.783 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/elan_i2c.ko 2023-07-18T02:33:46.783 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/sermouse.ko 2023-07-18T02:33:46.784 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-07-18T02:33:46.784 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/synaptics_usb.ko 2023-07-18T02:33:46.784 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/vsxxxaa.ko 2023-07-18T02:33:46.785 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/rmi4 2023-07-18T02:33:46.785 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/rmi4/rmi_core.ko 2023-07-18T02:33:46.785 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio 2023-07-18T02:33:46.785 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/altera_ps2.ko 2023-07-18T02:33:46.785 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/arc_ps2.ko 2023-07-18T02:33:46.785 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-07-18T02:33:46.786 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/serio_raw.ko 2023-07-18T02:33:46.786 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/sparse-keymap.ko 2023-07-18T02:33:46.786 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/iommu 2023-07-18T02:33:46.786 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/iommu/amd 2023-07-18T02:33:46.786 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/iommu/amd/iommu_v2.ko 2023-07-18T02:33:46.786 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds 2023-07-18T02:33:46.786 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-blinkm.ko 2023-07-18T02:33:46.787 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-lm3530.ko 2023-07-18T02:33:46.787 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-lp3944.ko 2023-07-18T02:33:46.787 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-ss4200.ko 2023-07-18T02:33:46.787 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger 2023-07-18T02:33:46.787 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-07-18T02:33:46.787 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-07-18T02:33:46.787 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-07-18T02:33:46.787 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-07-18T02:33:46.788 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-07-18T02:33:46.788 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-07-18T02:33:46.788 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-07-18T02:33:46.788 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-07-18T02:33:46.788 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md 2023-07-18T02:33:46.788 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/bcache 2023-07-18T02:33:46.789 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/bcache/bcache.ko 2023-07-18T02:33:46.789 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-bio-prison.ko 2023-07-18T02:33:46.789 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-cache-smq.ko 2023-07-18T02:33:46.789 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-cache.ko 2023-07-18T02:33:46.789 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-crypt.ko 2023-07-18T02:33:46.789 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-delay.ko 2023-07-18T02:33:46.789 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-flakey.ko 2023-07-18T02:33:46.789 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-log-userspace.ko 2023-07-18T02:33:46.790 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-multipath.ko 2023-07-18T02:33:46.790 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-queue-length.ko 2023-07-18T02:33:46.790 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-raid.ko 2023-07-18T02:33:46.790 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-round-robin.ko 2023-07-18T02:33:46.790 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-service-time.ko 2023-07-18T02:33:46.790 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-switch.ko 2023-07-18T02:33:46.790 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-thin-pool.ko 2023-07-18T02:33:46.791 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-verity.ko 2023-07-18T02:33:46.791 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/faulty.ko 2023-07-18T02:33:46.791 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/linear.ko 2023-07-18T02:33:46.791 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/multipath.ko 2023-07-18T02:33:46.791 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/persistent-data 2023-07-18T02:33:46.791 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-07-18T02:33:46.791 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid0.ko 2023-07-18T02:33:46.792 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid1.ko 2023-07-18T02:33:46.792 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid10.ko 2023-07-18T02:33:46.792 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid456.ko 2023-07-18T02:33:46.792 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message 2023-07-18T02:33:46.792 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion 2023-07-18T02:33:46.792 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptbase.ko 2023-07-18T02:33:46.792 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptctl.ko 2023-07-18T02:33:46.793 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptfc.ko 2023-07-18T02:33:46.793 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptlan.ko 2023-07-18T02:33:46.793 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptsas.ko 2023-07-18T02:33:46.793 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptscsih.ko 2023-07-18T02:33:46.793 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptspi.ko 2023-07-18T02:33:46.793 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd 2023-07-18T02:33:46.793 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/lpc_ich.ko 2023-07-18T02:33:46.793 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/lpc_sch.ko 2023-07-18T02:33:46.794 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/mfd-core.ko 2023-07-18T02:33:46.794 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/sm501.ko 2023-07-18T02:33:46.794 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/viperboard.ko 2023-07-18T02:33:46.794 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/vx855.ko 2023-07-18T02:33:46.794 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/wl1273-core.ko 2023-07-18T02:33:46.794 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc 2023-07-18T02:33:46.795 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/altera-stapl 2023-07-18T02:33:46.795 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-07-18T02:33:46.795 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/apds9802als.ko 2023-07-18T02:33:46.795 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/apds990x.ko 2023-07-18T02:33:46.795 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/bh1770glc.ko 2023-07-18T02:33:46.795 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/cb710 2023-07-18T02:33:46.795 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/cb710/cb710.ko 2023-07-18T02:33:46.795 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom 2023-07-18T02:33:46.796 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/at24.ko 2023-07-18T02:33:46.796 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/eeprom.ko 2023-07-18T02:33:46.796 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-07-18T02:33:46.796 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/max6875.ko 2023-07-18T02:33:46.796 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/enclosure.ko 2023-07-18T02:33:46.796 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/hpilo.ko 2023-07-18T02:33:46.796 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/isl29003.ko 2023-07-18T02:33:46.797 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/isl29020.ko 2023-07-18T02:33:46.797 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/lis3lv02d 2023-07-18T02:33:46.797 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-07-18T02:33:46.797 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-07-18T02:33:46.797 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei 2023-07-18T02:33:46.797 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei/mei-me.ko 2023-07-18T02:33:46.797 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei/mei-txe.ko 2023-07-18T02:33:46.797 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei/mei.ko 2023-07-18T02:33:46.798 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-gru 2023-07-18T02:33:46.798 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-gru/gru.ko 2023-07-18T02:33:46.798 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp 2023-07-18T02:33:46.798 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp/xp.ko 2023-07-18T02:33:46.798 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp/xpc.ko 2023-07-18T02:33:46.798 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-07-18T02:33:46.798 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/tifm_7xx1.ko 2023-07-18T02:33:46.799 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/tifm_core.ko 2023-07-18T02:33:46.799 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/tsl2550.ko 2023-07-18T02:33:46.799 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/vmw_balloon.ko 2023-07-18T02:33:46.799 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/vmw_vmci 2023-07-18T02:33:46.799 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-07-18T02:33:46.799 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net 2023-07-18T02:33:46.799 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/bonding 2023-07-18T02:33:46.799 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/bonding/bonding.ko 2023-07-18T02:33:46.800 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa 2023-07-18T02:33:46.800 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/b53 2023-07-18T02:33:46.800 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/b53/b53_common.ko 2023-07-18T02:33:46.800 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/bcm-sf2.ko 2023-07-18T02:33:46.800 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/mv88e6060.ko 2023-07-18T02:33:46.800 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/mv88e6xxx 2023-07-18T02:33:46.800 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-07-18T02:33:46.801 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dummy.ko 2023-07-18T02:33:46.801 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/eql.ko 2023-07-18T02:33:46.801 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet 2023-07-18T02:33:46.801 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com 2023-07-18T02:33:46.801 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-07-18T02:33:46.801 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-07-18T02:33:46.801 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-07-18T02:33:46.802 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-07-18T02:33:46.802 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390 2023-07-18T02:33:46.802 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/8390.ko 2023-07-18T02:33:46.802 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-07-18T02:33:46.802 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-07-18T02:33:46.802 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-07-18T02:33:46.802 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/adaptec 2023-07-18T02:33:46.802 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-07-18T02:33:46.803 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/agere 2023-07-18T02:33:46.803 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/agere/et131x.ko 2023-07-18T02:33:46.803 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/alteon 2023-07-18T02:33:46.803 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-07-18T02:33:46.803 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/altera 2023-07-18T02:33:46.803 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-07-18T02:33:46.803 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd 2023-07-18T02:33:46.803 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-07-18T02:33:46.804 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-07-18T02:33:46.804 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-07-18T02:33:46.804 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros 2023-07-18T02:33:46.804 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/alx 2023-07-18T02:33:46.804 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-07-18T02:33:46.804 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1c 2023-07-18T02:33:46.804 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-07-18T02:33:46.805 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1e 2023-07-18T02:33:46.805 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-07-18T02:33:46.805 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atlx 2023-07-18T02:33:46.805 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-07-18T02:33:46.805 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-07-18T02:33:46.805 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom 2023-07-18T02:33:46.805 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-07-18T02:33:46.806 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-07-18T02:33:46.806 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-07-18T02:33:46.806 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-07-18T02:33:46.806 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-07-18T02:33:46.806 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/genet 2023-07-18T02:33:46.806 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-07-18T02:33:46.806 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-07-18T02:33:46.806 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/brocade 2023-07-18T02:33:46.807 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/brocade/bna 2023-07-18T02:33:46.807 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-07-18T02:33:46.807 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio 2023-07-18T02:33:46.807 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb 2023-07-18T02:33:46.807 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-07-18T02:33:46.807 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-07-18T02:33:46.807 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-07-18T02:33:46.808 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-07-18T02:33:46.808 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-07-18T02:33:46.808 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-07-18T02:33:46.808 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-07-18T02:33:46.808 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-07-18T02:33:46.808 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-07-18T02:33:46.808 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/cisco 2023-07-18T02:33:46.809 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/cisco/enic 2023-07-18T02:33:46.809 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-07-18T02:33:46.809 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec 2023-07-18T02:33:46.809 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip 2023-07-18T02:33:46.809 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-07-18T02:33:46.810 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-07-18T02:33:46.810 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-07-18T02:33:46.810 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-07-18T02:33:46.810 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-07-18T02:33:46.811 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-07-18T02:33:46.811 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dlink 2023-07-18T02:33:46.811 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-07-18T02:33:46.812 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-07-18T02:33:46.812 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dnet.ko 2023-07-18T02:33:46.812 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/emulex 2023-07-18T02:33:46.812 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/emulex/benet 2023-07-18T02:33:46.813 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-07-18T02:33:46.813 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/ethoc.ko 2023-07-18T02:33:46.813 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/fealnx.ko 2023-07-18T02:33:46.813 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel 2023-07-18T02:33:46.814 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e100.ko 2023-07-18T02:33:46.814 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000 2023-07-18T02:33:46.814 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-07-18T02:33:46.815 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000e 2023-07-18T02:33:46.815 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-07-18T02:33:46.815 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/fm10k 2023-07-18T02:33:46.815 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-07-18T02:33:46.816 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/i40e 2023-07-18T02:33:46.816 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-07-18T02:33:46.816 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/iavf 2023-07-18T02:33:46.817 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-07-18T02:33:46.817 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igb 2023-07-18T02:33:46.817 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-07-18T02:33:46.818 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igbvf 2023-07-18T02:33:46.818 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-07-18T02:33:46.818 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbe 2023-07-18T02:33:46.818 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-07-18T02:33:46.818 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbevf 2023-07-18T02:33:46.819 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-07-18T02:33:46.819 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/jme.ko 2023-07-18T02:33:46.819 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell 2023-07-18T02:33:46.819 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-07-18T02:33:46.820 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell/skge.ko 2023-07-18T02:33:46.820 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-07-18T02:33:46.820 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox 2023-07-18T02:33:46.821 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx4 2023-07-18T02:33:46.821 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-07-18T02:33:46.821 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-07-18T02:33:46.821 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx5 2023-07-18T02:33:46.821 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-07-18T02:33:46.821 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-07-18T02:33:46.821 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/micrel 2023-07-18T02:33:46.822 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-07-18T02:33:46.822 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/myricom 2023-07-18T02:33:46.822 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/myricom/myri10ge 2023-07-18T02:33:46.822 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-07-18T02:33:46.822 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/natsemi 2023-07-18T02:33:46.822 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-07-18T02:33:46.822 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-07-18T02:33:46.822 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/neterion 2023-07-18T02:33:46.823 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-07-18T02:33:46.823 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/nvidia 2023-07-18T02:33:46.823 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-07-18T02:33:46.823 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/packetengines 2023-07-18T02:33:46.823 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-07-18T02:33:46.823 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-07-18T02:33:46.823 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic 2023-07-18T02:33:46.824 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/netxen 2023-07-18T02:33:46.824 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-07-18T02:33:46.824 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-07-18T02:33:46.824 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-07-18T02:33:46.824 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-07-18T02:33:46.824 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rdc 2023-07-18T02:33:46.824 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-07-18T02:33:46.825 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek 2023-07-18T02:33:46.825 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-07-18T02:33:46.825 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-07-18T02:33:46.825 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/atp.ko 2023-07-18T02:33:46.825 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-07-18T02:33:46.825 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rocker 2023-07-18T02:33:46.825 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-07-18T02:33:46.825 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sfc 2023-07-18T02:33:46.826 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-07-18T02:33:46.826 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/silan 2023-07-18T02:33:46.826 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-07-18T02:33:46.826 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sis 2023-07-18T02:33:46.826 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sis/sis190.ko 2023-07-18T02:33:46.826 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sis/sis900.ko 2023-07-18T02:33:46.826 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc 2023-07-18T02:33:46.827 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-07-18T02:33:46.827 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-07-18T02:33:46.827 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-07-18T02:33:46.827 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-07-18T02:33:46.827 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro 2023-07-18T02:33:46.827 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/stmmac 2023-07-18T02:33:46.827 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-07-18T02:33:46.827 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-07-18T02:33:46.828 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun 2023-07-18T02:33:46.828 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/cassini.ko 2023-07-18T02:33:46.828 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/niu.ko 2023-07-18T02:33:46.828 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/sungem.ko 2023-07-18T02:33:46.828 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-07-18T02:33:46.828 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/tehuti 2023-07-18T02:33:46.828 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-07-18T02:33:46.829 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/ti 2023-07-18T02:33:46.829 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/ti/tlan.ko 2023-07-18T02:33:46.829 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/via 2023-07-18T02:33:46.829 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-07-18T02:33:46.829 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-07-18T02:33:46.829 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/wiznet 2023-07-18T02:33:46.829 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-07-18T02:33:46.829 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-07-18T02:33:46.830 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/xircom 2023-07-18T02:33:46.830 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-07-18T02:33:46.830 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/geneve.ko 2023-07-18T02:33:46.830 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/hyperv 2023-07-18T02:33:46.830 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-07-18T02:33:46.830 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ieee802154 2023-07-18T02:33:46.830 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ieee802154/fakelb.ko 2023-07-18T02:33:46.831 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ifb.ko 2023-07-18T02:33:46.831 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ipvlan 2023-07-18T02:33:46.831 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ipvlan/ipvlan.ko 2023-07-18T02:33:46.831 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/macvlan.ko 2023-07-18T02:33:46.831 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/macvtap.ko 2023-07-18T02:33:46.831 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio 2023-07-18T02:33:46.831 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio.ko 2023-07-18T02:33:46.831 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-07-18T02:33:46.832 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-07-18T02:33:46.832 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio/mdio-regmap.ko 2023-07-18T02:33:46.832 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mii.ko 2023-07-18T02:33:46.832 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/net_failover.ko 2023-07-18T02:33:46.832 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/netconsole.ko 2023-07-18T02:33:46.832 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/nlmon.ko 2023-07-18T02:33:46.832 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/pcs 2023-07-18T02:33:46.833 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/pcs/pcs-lynx.ko 2023-07-18T02:33:46.833 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-07-18T02:33:46.833 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy 2023-07-18T02:33:46.833 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/amd.ko 2023-07-18T02:33:46.833 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-07-18T02:33:46.833 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-07-18T02:33:46.833 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm7xxx.ko 2023-07-18T02:33:46.834 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm87xx.ko 2023-07-18T02:33:46.834 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/broadcom.ko 2023-07-18T02:33:46.834 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/cicada.ko 2023-07-18T02:33:46.834 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/davicom.ko 2023-07-18T02:33:46.834 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/dp83640.ko 2023-07-18T02:33:46.834 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/et1011c.ko 2023-07-18T02:33:46.834 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/icplus.ko 2023-07-18T02:33:46.835 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/lxt.ko 2023-07-18T02:33:46.835 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/marvell.ko 2023-07-18T02:33:46.835 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/micrel.ko 2023-07-18T02:33:46.835 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/national.ko 2023-07-18T02:33:46.835 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/phylink.ko 2023-07-18T02:33:46.836 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/qsemi.ko 2023-07-18T02:33:46.836 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/realtek.ko 2023-07-18T02:33:46.836 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/smsc.ko 2023-07-18T02:33:46.836 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/ste10Xp.ko 2023-07-18T02:33:46.836 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/vitesse.ko 2023-07-18T02:33:46.836 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp 2023-07-18T02:33:46.836 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/bsd_comp.ko 2023-07-18T02:33:46.837 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_async.ko 2023-07-18T02:33:46.837 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_deflate.ko 2023-07-18T02:33:46.837 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_generic.ko 2023-07-18T02:33:46.837 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_mppe.ko 2023-07-18T02:33:46.837 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_synctty.ko 2023-07-18T02:33:46.837 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/pppoe.ko 2023-07-18T02:33:46.837 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/pppox.ko 2023-07-18T02:33:46.837 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/pptp.ko 2023-07-18T02:33:46.838 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/slip 2023-07-18T02:33:46.838 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/slip/slhc.ko 2023-07-18T02:33:46.838 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/slip/slip.ko 2023-07-18T02:33:46.838 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/sungem_phy.ko 2023-07-18T02:33:46.838 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/tap.ko 2023-07-18T02:33:46.838 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team 2023-07-18T02:33:46.838 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team.ko 2023-07-18T02:33:46.838 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_activebackup.ko 2023-07-18T02:33:46.839 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_broadcast.ko 2023-07-18T02:33:46.839 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-07-18T02:33:46.839 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_random.ko 2023-07-18T02:33:46.839 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-07-18T02:33:46.839 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/tun.ko 2023-07-18T02:33:46.839 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/veth.ko 2023-07-18T02:33:46.839 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/virtio_net.ko 2023-07-18T02:33:46.840 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vmxnet3 2023-07-18T02:33:46.840 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-07-18T02:33:46.840 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vxlan 2023-07-18T02:33:46.840 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vxlan/vxlan.ko 2023-07-18T02:33:46.840 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/xen-netback 2023-07-18T02:33:46.840 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/xen-netback/xen-netback.ko 2023-07-18T02:33:46.840 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/xen-netfront.ko 2023-07-18T02:33:46.841 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ntb 2023-07-18T02:33:46.841 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ntb/ntb.ko 2023-07-18T02:33:46.841 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme 2023-07-18T02:33:46.841 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme/host 2023-07-18T02:33:46.841 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme/host/nvme-core.ko 2023-07-18T02:33:46.841 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme/host/nvme.ko 2023-07-18T02:33:46.841 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport 2023-07-18T02:33:46.841 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport.ko 2023-07-18T02:33:46.842 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport_cs.ko 2023-07-18T02:33:46.842 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport_pc.ko 2023-07-18T02:33:46.842 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport_serial.ko 2023-07-18T02:33:46.842 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci 2023-07-18T02:33:46.842 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/hotplug 2023-07-18T02:33:46.842 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-07-18T02:33:46.842 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/pcie 2023-07-18T02:33:46.842 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/pcie/aer_inject.ko 2023-07-18T02:33:46.843 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/xen-pcifront.ko 2023-07-18T02:33:46.843 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia 2023-07-18T02:33:46.843 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/i82092.ko 2023-07-18T02:33:46.843 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/pd6729.ko 2023-07-18T02:33:46.843 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/yenta_socket.ko 2023-07-18T02:33:46.843 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pinctrl 2023-07-18T02:33:46.843 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pinctrl/intel 2023-07-18T02:33:46.844 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-07-18T02:33:46.844 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform 2023-07-18T02:33:46.844 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86 2023-07-18T02:33:46.844 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/acer-wmi.ko 2023-07-18T02:33:46.844 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/acerhdf.ko 2023-07-18T02:33:46.844 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/apple-gmux.ko 2023-07-18T02:33:46.844 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/asus-laptop.ko 2023-07-18T02:33:46.844 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-07-18T02:33:46.845 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/asus-wmi.ko 2023-07-18T02:33:46.845 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/classmate-laptop.ko 2023-07-18T02:33:46.845 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-07-18T02:33:46.845 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-07-18T02:33:46.845 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-07-18T02:33:46.845 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-07-18T02:33:46.845 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/hdaps.ko 2023-07-18T02:33:46.846 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/intel 2023-07-18T02:33:46.846 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-07-18T02:33:46.846 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/intel_ips.ko 2023-07-18T02:33:46.846 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/msi-wmi.ko 2023-07-18T02:33:46.846 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/mxm-wmi.ko 2023-07-18T02:33:46.846 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-07-18T02:33:46.846 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/samsung-laptop.ko 2023-07-18T02:33:46.846 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/samsung-q10.ko 2023-07-18T02:33:46.847 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/topstar-laptop.ko 2023-07-18T02:33:46.847 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-07-18T02:33:46.847 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/toshiba_haps.ko 2023-07-18T02:33:46.847 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/wmi-bmof.ko 2023-07-18T02:33:46.847 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/wmi.ko 2023-07-18T02:33:46.847 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/powercap 2023-07-18T02:33:46.847 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/powercap/intel_rapl_common.ko 2023-07-18T02:33:46.847 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/powercap/intel_rapl_msr.ko 2023-07-18T02:33:46.848 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps 2023-07-18T02:33:46.848 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients 2023-07-18T02:33:46.848 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients/pps-gpio.ko 2023-07-18T02:33:46.848 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients/pps-ldisc.ko 2023-07-18T02:33:46.848 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients/pps_parport.ko 2023-07-18T02:33:46.848 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/pps_core.ko 2023-07-18T02:33:46.849 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ptp 2023-07-18T02:33:46.849 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ptp/ptp.ko 2023-07-18T02:33:46.849 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ptp/ptp_kvm.ko 2023-07-18T02:33:46.849 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc 2023-07-18T02:33:46.850 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-bq32k.ko 2023-07-18T02:33:46.850 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-bq4802.ko 2023-07-18T02:33:46.850 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1286.ko 2023-07-18T02:33:46.851 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1307.ko 2023-07-18T02:33:46.851 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1374.ko 2023-07-18T02:33:46.851 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1511.ko 2023-07-18T02:33:46.851 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1553.ko 2023-07-18T02:33:46.852 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1672.ko 2023-07-18T02:33:46.852 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1742.ko 2023-07-18T02:33:46.852 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds2404.ko 2023-07-18T02:33:46.853 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds3232.ko 2023-07-18T02:33:46.853 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-em3027.ko 2023-07-18T02:33:46.853 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-fm3130.ko 2023-07-18T02:33:46.853 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-isl12022.ko 2023-07-18T02:33:46.854 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-isl1208.ko 2023-07-18T02:33:46.854 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-m41t80.ko 2023-07-18T02:33:46.854 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-m48t35.ko 2023-07-18T02:33:46.854 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-m48t59.ko 2023-07-18T02:33:46.855 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-max6900.ko 2023-07-18T02:33:46.855 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-msm6242.ko 2023-07-18T02:33:46.855 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf2127.ko 2023-07-18T02:33:46.856 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf85063.ko 2023-07-18T02:33:46.856 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf8523.ko 2023-07-18T02:33:46.856 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf8563.ko 2023-07-18T02:33:46.856 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf8583.ko 2023-07-18T02:33:46.857 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rp5c01.ko 2023-07-18T02:33:46.857 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rs5c372.ko 2023-07-18T02:33:46.857 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-07-18T02:33:46.857 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rx8025.ko 2023-07-18T02:33:46.857 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rx8581.ko 2023-07-18T02:33:46.858 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-07-18T02:33:46.858 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-x1205.ko 2023-07-18T02:33:46.858 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi 2023-07-18T02:33:46.859 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/3w-9xxx.ko 2023-07-18T02:33:46.859 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/3w-sas.ko 2023-07-18T02:33:46.859 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/3w-xxxx.ko 2023-07-18T02:33:46.859 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/BusLogic.ko 2023-07-18T02:33:46.859 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/a100u2w.ko 2023-07-18T02:33:46.859 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aacraid 2023-07-18T02:33:46.860 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aacraid/aacraid.ko 2023-07-18T02:33:46.860 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/advansys.ko 2023-07-18T02:33:46.860 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aic7xxx 2023-07-18T02:33:46.860 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-07-18T02:33:46.860 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-07-18T02:33:46.860 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/am53c974.ko 2023-07-18T02:33:46.860 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/arcmsr 2023-07-18T02:33:46.861 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-07-18T02:33:46.861 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/atp870u.ko 2023-07-18T02:33:46.861 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/be2iscsi 2023-07-18T02:33:46.861 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-07-18T02:33:46.861 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bfa 2023-07-18T02:33:46.861 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bfa/bfa.ko 2023-07-18T02:33:46.861 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2fc 2023-07-18T02:33:46.862 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-07-18T02:33:46.862 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2i 2023-07-18T02:33:46.862 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-07-18T02:33:46.862 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ch.ko 2023-07-18T02:33:46.862 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/csiostor 2023-07-18T02:33:46.862 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/csiostor/csiostor.ko 2023-07-18T02:33:46.862 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi 2023-07-18T02:33:46.862 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb3i 2023-07-18T02:33:46.863 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-07-18T02:33:46.863 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb4i 2023-07-18T02:33:46.863 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-07-18T02:33:46.863 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-07-18T02:33:46.863 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/dc395x.ko 2023-07-18T02:33:46.863 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler 2023-07-18T02:33:46.863 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-07-18T02:33:46.863 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-07-18T02:33:46.864 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-07-18T02:33:46.864 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-07-18T02:33:46.864 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/dmx3191d.ko 2023-07-18T02:33:46.864 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/esas2r 2023-07-18T02:33:46.864 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/esas2r/esas2r.ko 2023-07-18T02:33:46.864 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/esp_scsi.ko 2023-07-18T02:33:46.864 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fcoe 2023-07-18T02:33:46.865 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fcoe/fcoe.ko 2023-07-18T02:33:46.865 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-07-18T02:33:46.865 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fnic 2023-07-18T02:33:46.865 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fnic/fnic.ko 2023-07-18T02:33:46.865 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/hpsa.ko 2023-07-18T02:33:46.865 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/hptiop.ko 2023-07-18T02:33:46.865 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/hv_storvsc.ko 2023-07-18T02:33:46.865 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/initio.ko 2023-07-18T02:33:46.866 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ipr.ko 2023-07-18T02:33:46.866 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ips.ko 2023-07-18T02:33:46.866 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/isci 2023-07-18T02:33:46.866 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/isci/isci.ko 2023-07-18T02:33:46.866 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-07-18T02:33:46.866 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/iscsi_tcp.ko 2023-07-18T02:33:46.867 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libfc 2023-07-18T02:33:46.867 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libfc/libfc.ko 2023-07-18T02:33:46.867 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libiscsi.ko 2023-07-18T02:33:46.867 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libiscsi_tcp.ko 2023-07-18T02:33:46.867 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libsas 2023-07-18T02:33:46.868 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libsas/libsas.ko 2023-07-18T02:33:46.868 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/lpfc 2023-07-18T02:33:46.868 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/lpfc/lpfc.ko 2023-07-18T02:33:46.868 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid 2023-07-18T02:33:46.868 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid.ko 2023-07-18T02:33:46.868 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-07-18T02:33:46.869 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-07-18T02:33:46.869 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-07-18T02:33:46.869 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mpt3sas 2023-07-18T02:33:46.869 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-07-18T02:33:46.869 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mvsas 2023-07-18T02:33:46.869 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mvsas/mvsas.ko 2023-07-18T02:33:46.869 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mvumi.ko 2023-07-18T02:33:46.870 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pm8001 2023-07-18T02:33:46.870 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-07-18T02:33:46.870 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pmcraid.ko 2023-07-18T02:33:46.870 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla1280.ko 2023-07-18T02:33:46.870 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla2xxx 2023-07-18T02:33:46.870 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-07-18T02:33:46.870 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-07-18T02:33:46.870 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla4xxx 2023-07-18T02:33:46.871 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-07-18T02:33:46.871 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/raid_class.ko 2023-07-18T02:33:46.871 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_debug.ko 2023-07-18T02:33:46.871 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_fc.ko 2023-07-18T02:33:46.871 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-07-18T02:33:46.871 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_sas.ko 2023-07-18T02:33:46.871 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_spi.ko 2023-07-18T02:33:46.871 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_srp.ko 2023-07-18T02:33:46.872 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ses.ko 2023-07-18T02:33:46.872 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/st.ko 2023-07-18T02:33:46.872 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/stex.ko 2023-07-18T02:33:46.872 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/sym53c8xx_2 2023-07-18T02:33:46.872 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-07-18T02:33:46.872 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/virtio_scsi.ko 2023-07-18T02:33:46.872 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/vmw_pvscsi.ko 2023-07-18T02:33:46.873 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/wd719x.ko 2023-07-18T02:33:46.873 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ssb 2023-07-18T02:33:46.873 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ssb/ssb.ko 2023-07-18T02:33:46.873 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target 2023-07-18T02:33:46.873 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/iscsi 2023-07-18T02:33:46.873 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-07-18T02:33:46.873 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/loopback 2023-07-18T02:33:46.873 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/loopback/tcm_loop.ko 2023-07-18T02:33:46.874 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_file.ko 2023-07-18T02:33:46.874 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_iblock.ko 2023-07-18T02:33:46.874 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_mod.ko 2023-07-18T02:33:46.874 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_pscsi.ko 2023-07-18T02:33:46.874 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_user.ko 2023-07-18T02:33:46.874 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/tcm_fc 2023-07-18T02:33:46.874 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-07-18T02:33:46.875 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal 2023-07-18T02:33:46.875 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel 2023-07-18T02:33:46.875 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal 2023-07-18T02:33:46.875 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-07-18T02:33:46.875 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-07-18T02:33:46.875 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-07-18T02:33:46.875 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-07-18T02:33:46.875 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-07-18T02:33:46.876 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-07-18T02:33:46.876 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-07-18T02:33:46.876 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-07-18T02:33:46.876 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-07-18T02:33:46.876 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-07-18T02:33:46.876 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-07-18T02:33:46.876 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-07-18T02:33:46.877 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-07-18T02:33:46.877 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-07-18T02:33:46.877 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-07-18T02:33:46.877 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty 2023-07-18T02:33:46.877 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/ipwireless 2023-07-18T02:33:46.877 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-07-18T02:33:46.877 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/n_gsm.ko 2023-07-18T02:33:46.877 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/n_hdlc.ko 2023-07-18T02:33:46.878 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/nozomi.ko 2023-07-18T02:33:46.878 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial 2023-07-18T02:33:46.878 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/8250 2023-07-18T02:33:46.878 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-07-18T02:33:46.878 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/arc_uart.ko 2023-07-18T02:33:46.878 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/jsm 2023-07-18T02:33:46.878 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/jsm/jsm.ko 2023-07-18T02:33:46.879 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/synclink_gt.ko 2023-07-18T02:33:46.879 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs 2023-07-18T02:33:46.879 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/core 2023-07-18T02:33:46.879 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/core/ufshcd-core.ko 2023-07-18T02:33:46.879 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/host 2023-07-18T02:33:46.879 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-07-18T02:33:46.879 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio 2023-07-18T02:33:46.880 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio.ko 2023-07-18T02:33:46.880 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_aec.ko 2023-07-18T02:33:46.880 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_cif.ko 2023-07-18T02:33:46.881 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_pci_generic.ko 2023-07-18T02:33:46.881 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_sercos3.ko 2023-07-18T02:33:46.881 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb 2023-07-18T02:33:46.881 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class 2023-07-18T02:33:46.882 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/cdc-acm.ko 2023-07-18T02:33:46.882 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/cdc-wdm.ko 2023-07-18T02:33:46.882 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/usblp.ko 2023-07-18T02:33:46.883 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/usbtmc.ko 2023-07-18T02:33:46.883 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/host 2023-07-18T02:33:46.883 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/host/sl811-hcd.ko 2023-07-18T02:33:46.884 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/image 2023-07-18T02:33:46.884 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/image/mdc800.ko 2023-07-18T02:33:46.884 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/image/microtek.ko 2023-07-18T02:33:46.884 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc 2023-07-18T02:33:46.884 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/adutux.ko 2023-07-18T02:33:46.885 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/appledisplay.ko 2023-07-18T02:33:46.885 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/emi26.ko 2023-07-18T02:33:46.885 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/emi62.ko 2023-07-18T02:33:46.885 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/ezusb.ko 2023-07-18T02:33:46.885 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/idmouse.ko 2023-07-18T02:33:46.885 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/iowarrior.ko 2023-07-18T02:33:46.885 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/isight_firmware.ko 2023-07-18T02:33:46.885 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/ldusb.ko 2023-07-18T02:33:46.886 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/legousbtower.ko 2023-07-18T02:33:46.886 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/sisusbvga 2023-07-18T02:33:46.886 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-07-18T02:33:46.886 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/trancevibrator.ko 2023-07-18T02:33:46.886 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/usb3503.ko 2023-07-18T02:33:46.886 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/usblcd.ko 2023-07-18T02:33:46.886 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/usbsevseg.ko 2023-07-18T02:33:46.886 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/uss720.ko 2023-07-18T02:33:46.887 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/yurex.ko 2023-07-18T02:33:46.887 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/phy 2023-07-18T02:33:46.887 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/phy/phy-generic.ko 2023-07-18T02:33:46.887 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial 2023-07-18T02:33:46.887 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/aircable.ko 2023-07-18T02:33:46.887 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ark3116.ko 2023-07-18T02:33:46.887 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/belkin_sa.ko 2023-07-18T02:33:46.888 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ch341.ko 2023-07-18T02:33:46.888 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/cp210x.ko 2023-07-18T02:33:46.888 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/cyberjack.ko 2023-07-18T02:33:46.888 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/cypress_m8.ko 2023-07-18T02:33:46.888 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/digi_acceleport.ko 2023-07-18T02:33:46.888 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/empeg.ko 2023-07-18T02:33:46.888 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ftdi_sio.ko 2023-07-18T02:33:46.888 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/garmin_gps.ko 2023-07-18T02:33:46.889 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/io_edgeport.ko 2023-07-18T02:33:46.889 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/io_ti.ko 2023-07-18T02:33:46.889 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ipaq.ko 2023-07-18T02:33:46.889 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ipw.ko 2023-07-18T02:33:46.889 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ir-usb.ko 2023-07-18T02:33:46.889 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-07-18T02:33:46.889 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/keyspan.ko 2023-07-18T02:33:46.890 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/keyspan_pda.ko 2023-07-18T02:33:46.890 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/kl5kusb105.ko 2023-07-18T02:33:46.890 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/kobil_sct.ko 2023-07-18T02:33:46.890 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/mct_u232.ko 2023-07-18T02:33:46.890 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/mos7720.ko 2023-07-18T02:33:46.890 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/mos7840.ko 2023-07-18T02:33:46.890 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/navman.ko 2023-07-18T02:33:46.890 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/omninet.ko 2023-07-18T02:33:46.891 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/opticon.ko 2023-07-18T02:33:46.891 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/option.ko 2023-07-18T02:33:46.891 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/oti6858.ko 2023-07-18T02:33:46.891 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/pl2303.ko 2023-07-18T02:33:46.891 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/qcaux.ko 2023-07-18T02:33:46.891 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/qcserial.ko 2023-07-18T02:33:46.891 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/quatech2.ko 2023-07-18T02:33:46.891 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/safe_serial.ko 2023-07-18T02:33:46.892 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/sierra.ko 2023-07-18T02:33:46.892 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/spcp8x5.ko 2023-07-18T02:33:46.892 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ssu100.ko 2023-07-18T02:33:46.892 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/symbolserial.ko 2023-07-18T02:33:46.892 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-07-18T02:33:46.892 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-07-18T02:33:46.892 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/usb_debug.ko 2023-07-18T02:33:46.893 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/usb_wwan.ko 2023-07-18T02:33:46.893 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/visor.ko 2023-07-18T02:33:46.893 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/whiteheat.ko 2023-07-18T02:33:46.893 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/xsens_mt.ko 2023-07-18T02:33:46.893 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage 2023-07-18T02:33:46.893 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/uas.ko 2023-07-18T02:33:46.893 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-alauda.ko 2023-07-18T02:33:46.893 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-cypress.ko 2023-07-18T02:33:46.894 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-datafab.ko 2023-07-18T02:33:46.894 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-07-18T02:33:46.894 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-freecom.ko 2023-07-18T02:33:46.894 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-isd200.ko 2023-07-18T02:33:46.894 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-07-18T02:33:46.894 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-karma.ko 2023-07-18T02:33:46.895 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-onetouch.ko 2023-07-18T02:33:46.895 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-realtek.ko 2023-07-18T02:33:46.895 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-sddr09.ko 2023-07-18T02:33:46.895 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-sddr55.ko 2023-07-18T02:33:46.895 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-usbat.ko 2023-07-18T02:33:46.895 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/usb-storage.ko 2023-07-18T02:33:46.895 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip 2023-07-18T02:33:46.895 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip/usbip-core.ko 2023-07-18T02:33:46.896 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip/usbip-host.ko 2023-07-18T02:33:46.896 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-07-18T02:33:46.896 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio 2023-07-18T02:33:46.896 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/pci 2023-07-18T02:33:46.896 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-07-18T02:33:46.896 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/pci/vfio-pci.ko 2023-07-18T02:33:46.896 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/vfio.ko 2023-07-18T02:33:46.897 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-07-18T02:33:46.897 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost 2023-07-18T02:33:46.897 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost.ko 2023-07-18T02:33:46.897 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost_iotlb.ko 2023-07-18T02:33:46.897 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost_net.ko 2023-07-18T02:33:46.898 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost_scsi.ko 2023-07-18T02:33:46.898 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video 2023-07-18T02:33:46.898 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight 2023-07-18T02:33:46.898 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/apple_bl.ko 2023-07-18T02:33:46.899 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/lcd.ko 2023-07-18T02:33:46.899 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/platform_lcd.ko 2023-07-18T02:33:46.899 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio 2023-07-18T02:33:46.899 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio.ko 2023-07-18T02:33:46.900 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_balloon.ko 2023-07-18T02:33:46.900 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_mmio.ko 2023-07-18T02:33:46.900 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_pci.ko 2023-07-18T02:33:46.901 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-07-18T02:33:46.901 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-07-18T02:33:46.901 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_ring.ko 2023-07-18T02:33:46.902 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1 2023-07-18T02:33:46.902 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/masters 2023-07-18T02:33:46.902 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/masters/ds2482.ko 2023-07-18T02:33:46.902 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/masters/ds2490.ko 2023-07-18T02:33:46.902 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves 2023-07-18T02:33:46.903 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-07-18T02:33:46.903 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-07-18T02:33:46.903 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-07-18T02:33:46.903 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-07-18T02:33:46.903 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-07-18T02:33:46.903 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-07-18T02:33:46.903 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-07-18T02:33:46.903 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-07-18T02:33:46.904 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-07-18T02:33:46.904 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_smem.ko 2023-07-18T02:33:46.904 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_therm.ko 2023-07-18T02:33:46.904 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/wire.ko 2023-07-18T02:33:46.904 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog 2023-07-18T02:33:46.904 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/alim1535_wdt.ko 2023-07-18T02:33:46.904 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/alim7101_wdt.ko 2023-07-18T02:33:46.905 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/f71808e_wdt.ko 2023-07-18T02:33:46.905 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/hpwdt.ko 2023-07-18T02:33:46.905 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/i6300esb.ko 2023-07-18T02:33:46.905 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-07-18T02:33:46.905 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/iTCO_wdt.ko 2023-07-18T02:33:46.905 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/ib700wdt.ko 2023-07-18T02:33:46.905 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/ibmasr.ko 2023-07-18T02:33:46.905 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-07-18T02:33:46.906 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/it8712f_wdt.ko 2023-07-18T02:33:46.906 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/it87_wdt.ko 2023-07-18T02:33:46.906 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/machzwd.ko 2023-07-18T02:33:46.906 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/nv_tco.ko 2023-07-18T02:33:46.906 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/pcwd_pci.ko 2023-07-18T02:33:46.906 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/pcwd_usb.ko 2023-07-18T02:33:46.906 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-07-18T02:33:46.907 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sch311x_wdt.ko 2023-07-18T02:33:46.907 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/softdog.ko 2023-07-18T02:33:46.907 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sp5100_tco.ko 2023-07-18T02:33:46.907 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/via_wdt.ko 2023-07-18T02:33:46.907 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-07-18T02:33:46.907 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/w83877f_wdt.ko 2023-07-18T02:33:46.907 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/w83977f_wdt.ko 2023-07-18T02:33:46.907 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/wdt_pci.ko 2023-07-18T02:33:46.908 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/xen_wdt.ko 2023-07-18T02:33:46.908 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen 2023-07-18T02:33:46.908 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-acpi-processor.ko 2023-07-18T02:33:46.908 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-evtchn.ko 2023-07-18T02:33:46.908 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-gntalloc.ko 2023-07-18T02:33:46.908 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-gntdev.ko 2023-07-18T02:33:46.908 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-pciback 2023-07-18T02:33:46.909 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-07-18T02:33:46.909 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-privcmd.ko 2023-07-18T02:33:46.909 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xenfs 2023-07-18T02:33:46.909 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xenfs/xenfs.ko 2023-07-18T02:33:46.909 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs 2023-07-18T02:33:46.909 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/9p 2023-07-18T02:33:46.909 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/9p/9p.ko 2023-07-18T02:33:46.909 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/affs 2023-07-18T02:33:46.910 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/affs/affs.ko 2023-07-18T02:33:46.910 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/befs 2023-07-18T02:33:46.910 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/befs/befs.ko 2023-07-18T02:33:46.910 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/binfmt_misc.ko 2023-07-18T02:33:46.910 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/btrfs 2023-07-18T02:33:46.910 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/btrfs/btrfs.ko 2023-07-18T02:33:46.910 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cachefiles 2023-07-18T02:33:46.910 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cachefiles/cachefiles.ko 2023-07-18T02:33:46.911 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ceph 2023-07-18T02:33:46.911 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ceph/ceph.ko 2023-07-18T02:33:46.911 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/coda 2023-07-18T02:33:46.911 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/coda/coda.ko 2023-07-18T02:33:46.911 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cramfs 2023-07-18T02:33:46.911 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cramfs/cramfs.ko 2023-07-18T02:33:46.911 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/dlm 2023-07-18T02:33:46.912 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/dlm/dlm.ko 2023-07-18T02:33:46.912 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ecryptfs 2023-07-18T02:33:46.912 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ecryptfs/ecryptfs.ko 2023-07-18T02:33:46.912 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/f2fs 2023-07-18T02:33:46.912 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/f2fs/f2fs.ko 2023-07-18T02:33:46.912 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat 2023-07-18T02:33:46.912 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat/fat.ko 2023-07-18T02:33:46.912 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat/msdos.ko 2023-07-18T02:33:46.913 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat/vfat.ko 2023-07-18T02:33:46.913 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fscache 2023-07-18T02:33:46.913 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fscache/fscache.ko 2023-07-18T02:33:46.913 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fuse 2023-07-18T02:33:46.913 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fuse/cuse.ko 2023-07-18T02:33:46.913 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fuse/fuse.ko 2023-07-18T02:33:46.913 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/gfs2 2023-07-18T02:33:46.914 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/gfs2/gfs2.ko 2023-07-18T02:33:46.914 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfs 2023-07-18T02:33:46.914 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfs/hfs.ko 2023-07-18T02:33:46.914 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfsplus 2023-07-18T02:33:46.914 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfsplus/hfsplus.ko 2023-07-18T02:33:46.914 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/isofs 2023-07-18T02:33:46.914 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/isofs/isofs.ko 2023-07-18T02:33:46.914 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/jfs 2023-07-18T02:33:46.915 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/jfs/jfs.ko 2023-07-18T02:33:46.915 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/lockd 2023-07-18T02:33:46.915 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/lockd/lockd.ko 2023-07-18T02:33:46.915 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/minix 2023-07-18T02:33:46.915 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/minix/minix.ko 2023-07-18T02:33:46.915 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/netfs 2023-07-18T02:33:46.915 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/netfs/netfs.ko 2023-07-18T02:33:46.915 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs 2023-07-18T02:33:46.916 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/blocklayout 2023-07-18T02:33:46.916 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-07-18T02:33:46.916 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/filelayout 2023-07-18T02:33:46.916 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-07-18T02:33:46.916 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/flexfilelayout 2023-07-18T02:33:46.916 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-07-18T02:33:46.916 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/nfs.ko 2023-07-18T02:33:46.917 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/nfsv3.ko 2023-07-18T02:33:46.917 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/nfsv4.ko 2023-07-18T02:33:46.917 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs_common 2023-07-18T02:33:46.917 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs_common/grace.ko 2023-07-18T02:33:46.917 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs_common/nfs_acl.ko 2023-07-18T02:33:46.917 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfsd 2023-07-18T02:33:46.917 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfsd/nfsd.ko 2023-07-18T02:33:46.917 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nilfs2 2023-07-18T02:33:46.918 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nilfs2/nilfs2.ko 2023-07-18T02:33:46.918 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls 2023-07-18T02:33:46.918 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-celtic.ko 2023-07-18T02:33:46.918 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-centeuro.ko 2023-07-18T02:33:46.918 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-croatian.ko 2023-07-18T02:33:46.918 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-cyrillic.ko 2023-07-18T02:33:46.918 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-gaelic.ko 2023-07-18T02:33:46.919 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-greek.ko 2023-07-18T02:33:46.919 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-iceland.ko 2023-07-18T02:33:46.919 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-inuit.ko 2023-07-18T02:33:46.919 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-roman.ko 2023-07-18T02:33:46.919 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-romanian.ko 2023-07-18T02:33:46.920 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-turkish.ko 2023-07-18T02:33:46.920 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp1250.ko 2023-07-18T02:33:46.920 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp1251.ko 2023-07-18T02:33:46.920 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp1255.ko 2023-07-18T02:33:46.921 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp737.ko 2023-07-18T02:33:46.921 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp775.ko 2023-07-18T02:33:46.921 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp850.ko 2023-07-18T02:33:46.922 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp852.ko 2023-07-18T02:33:46.922 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp855.ko 2023-07-18T02:33:46.922 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp857.ko 2023-07-18T02:33:46.922 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp860.ko 2023-07-18T02:33:46.923 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp861.ko 2023-07-18T02:33:46.923 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp862.ko 2023-07-18T02:33:46.923 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp863.ko 2023-07-18T02:33:46.923 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp864.ko 2023-07-18T02:33:46.924 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp865.ko 2023-07-18T02:33:46.924 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp866.ko 2023-07-18T02:33:46.924 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp869.ko 2023-07-18T02:33:46.925 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp874.ko 2023-07-18T02:33:46.925 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp932.ko 2023-07-18T02:33:46.925 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp936.ko 2023-07-18T02:33:46.925 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp949.ko 2023-07-18T02:33:46.925 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp950.ko 2023-07-18T02:33:46.926 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_euc-jp.ko 2023-07-18T02:33:46.926 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-1.ko 2023-07-18T02:33:46.926 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-13.ko 2023-07-18T02:33:46.926 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-14.ko 2023-07-18T02:33:46.926 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-15.ko 2023-07-18T02:33:46.926 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-2.ko 2023-07-18T02:33:46.926 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-3.ko 2023-07-18T02:33:46.926 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-4.ko 2023-07-18T02:33:46.927 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-5.ko 2023-07-18T02:33:46.927 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-6.ko 2023-07-18T02:33:46.927 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-7.ko 2023-07-18T02:33:46.927 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-9.ko 2023-07-18T02:33:46.927 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_koi8-r.ko 2023-07-18T02:33:46.927 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_koi8-ru.ko 2023-07-18T02:33:46.927 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_koi8-u.ko 2023-07-18T02:33:46.928 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_utf8.ko 2023-07-18T02:33:46.928 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2 2023-07-18T02:33:46.928 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/cluster 2023-07-18T02:33:46.928 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-07-18T02:33:46.928 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlm 2023-07-18T02:33:46.928 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-07-18T02:33:46.928 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlmfs 2023-07-18T02:33:46.929 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-07-18T02:33:46.929 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2.ko 2023-07-18T02:33:46.929 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-07-18T02:33:46.929 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-07-18T02:33:46.929 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-07-18T02:33:46.929 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/overlayfs 2023-07-18T02:33:46.929 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/overlayfs/overlay.ko 2023-07-18T02:33:46.929 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/pstore 2023-07-18T02:33:46.930 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/pstore/ramoops.ko 2023-07-18T02:33:46.930 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/reiserfs 2023-07-18T02:33:46.930 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/reiserfs/reiserfs.ko 2023-07-18T02:33:46.930 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/romfs 2023-07-18T02:33:46.930 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/romfs/romfs.ko 2023-07-18T02:33:46.930 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb 2023-07-18T02:33:46.930 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/client 2023-07-18T02:33:46.930 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/client/cifs.ko 2023-07-18T02:33:46.931 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/common 2023-07-18T02:33:46.931 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/common/cifs_arc4.ko 2023-07-18T02:33:46.931 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/common/cifs_md4.ko 2023-07-18T02:33:46.931 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/squashfs 2023-07-18T02:33:46.931 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/squashfs/squashfs.ko 2023-07-18T02:33:46.931 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/sysv 2023-07-18T02:33:46.931 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/sysv/sysv.ko 2023-07-18T02:33:46.931 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/udf 2023-07-18T02:33:46.932 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/udf/udf.ko 2023-07-18T02:33:46.932 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ufs 2023-07-18T02:33:46.932 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ufs/ufs.ko 2023-07-18T02:33:46.932 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/xfs 2023-07-18T02:33:46.932 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/xfs/xfs.ko 2023-07-18T02:33:46.932 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel 2023-07-18T02:33:46.932 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/rcu 2023-07-18T02:33:46.933 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/rcu/rcutorture.ko 2023-07-18T02:33:46.933 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/torture.ko 2023-07-18T02:33:46.933 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/trace 2023-07-18T02:33:46.933 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-07-18T02:33:46.933 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib 2023-07-18T02:33:46.933 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/asn1_encoder.ko 2023-07-18T02:33:46.933 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crc-itu-t.ko 2023-07-18T02:33:46.934 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crc8.ko 2023-07-18T02:33:46.934 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crypto 2023-07-18T02:33:46.934 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crypto/libarc4.ko 2023-07-18T02:33:46.934 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crypto/libdes.ko 2023-07-18T02:33:46.934 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/libcrc32c.ko 2023-07-18T02:33:46.934 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lru_cache.ko 2023-07-18T02:33:46.934 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lz4 2023-07-18T02:33:46.934 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lz4/lz4_compress.ko 2023-07-18T02:33:46.935 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lz4/lz4hc_compress.ko 2023-07-18T02:33:46.935 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/math 2023-07-18T02:33:46.935 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/math/cordic.ko 2023-07-18T02:33:46.935 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/raid6 2023-07-18T02:33:46.935 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/raid6/raid6_pq.ko 2023-07-18T02:33:46.935 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/reed_solomon 2023-07-18T02:33:46.935 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/reed_solomon/reed_solomon.ko 2023-07-18T02:33:46.935 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/ts_bm.ko 2023-07-18T02:33:46.936 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/ts_fsm.ko 2023-07-18T02:33:46.936 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/ts_kmp.ko 2023-07-18T02:33:46.936 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/zstd 2023-07-18T02:33:46.936 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/zstd/zstd_compress.ko 2023-07-18T02:33:46.936 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/mm 2023-07-18T02:33:46.936 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/mm/hwpoison-inject.ko 2023-07-18T02:33:46.936 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net 2023-07-18T02:33:46.937 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan 2023-07-18T02:33:46.937 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/6lowpan.ko 2023-07-18T02:33:46.937 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_dest.ko 2023-07-18T02:33:46.937 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_fragment.ko 2023-07-18T02:33:46.937 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_hop.ko 2023-07-18T02:33:46.937 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_ipv6.ko 2023-07-18T02:33:46.937 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_mobility.ko 2023-07-18T02:33:46.937 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_routing.ko 2023-07-18T02:33:46.938 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_udp.ko 2023-07-18T02:33:46.938 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802 2023-07-18T02:33:46.938 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/garp.ko 2023-07-18T02:33:46.938 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/mrp.ko 2023-07-18T02:33:46.938 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/p8022.ko 2023-07-18T02:33:46.938 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/psnap.ko 2023-07-18T02:33:46.938 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/stp.ko 2023-07-18T02:33:46.938 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/8021q 2023-07-18T02:33:46.939 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/8021q/8021q.ko 2023-07-18T02:33:46.939 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p 2023-07-18T02:33:46.939 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet.ko 2023-07-18T02:33:46.939 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet_fd.ko 2023-07-18T02:33:46.939 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet_rdma.ko 2023-07-18T02:33:46.939 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet_virtio.ko 2023-07-18T02:33:46.939 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/batman-adv 2023-07-18T02:33:46.939 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/batman-adv/batman-adv.ko 2023-07-18T02:33:46.940 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge 2023-07-18T02:33:46.940 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/br_netfilter.ko 2023-07-18T02:33:46.940 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/bridge.ko 2023-07-18T02:33:46.940 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter 2023-07-18T02:33:46.940 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-07-18T02:33:46.940 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_among.ko 2023-07-18T02:33:46.940 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_arp.ko 2023-07-18T02:33:46.941 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-07-18T02:33:46.941 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-07-18T02:33:46.941 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_ip.ko 2023-07-18T02:33:46.941 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-07-18T02:33:46.941 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_limit.ko 2023-07-18T02:33:46.941 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_log.ko 2023-07-18T02:33:46.941 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_mark.ko 2023-07-18T02:33:46.941 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-07-18T02:33:46.942 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-07-18T02:33:46.942 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-07-18T02:33:46.942 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-07-18T02:33:46.942 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_snat.ko 2023-07-18T02:33:46.942 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_stp.ko 2023-07-18T02:33:46.942 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-07-18T02:33:46.942 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-07-18T02:33:46.943 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-07-18T02:33:46.943 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-07-18T02:33:46.943 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtables.ko 2023-07-18T02:33:46.943 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-07-18T02:33:46.943 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-07-18T02:33:46.943 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-07-18T02:33:46.943 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ceph 2023-07-18T02:33:46.943 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ceph/libceph.ko 2023-07-18T02:33:46.944 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/core 2023-07-18T02:33:46.944 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/core/failover.ko 2023-07-18T02:33:46.944 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/core/pktgen.ko 2023-07-18T02:33:46.944 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp 2023-07-18T02:33:46.944 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp.ko 2023-07-18T02:33:46.944 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp_diag.ko 2023-07-18T02:33:46.944 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp_ipv4.ko 2023-07-18T02:33:46.945 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp_ipv6.ko 2023-07-18T02:33:46.945 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dns_resolver 2023-07-18T02:33:46.945 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dns_resolver/dns_resolver.ko 2023-07-18T02:33:46.945 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa 2023-07-18T02:33:46.945 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/dsa_core.ko 2023-07-18T02:33:46.945 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_brcm.ko 2023-07-18T02:33:46.945 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_dsa.ko 2023-07-18T02:33:46.945 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_none.ko 2023-07-18T02:33:46.946 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_trailer.ko 2023-07-18T02:33:46.946 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154 2023-07-18T02:33:46.946 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/6lowpan 2023-07-18T02:33:46.946 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-07-18T02:33:46.946 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/ieee802154.ko 2023-07-18T02:33:46.946 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/ieee802154_socket.ko 2023-07-18T02:33:46.946 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4 2023-07-18T02:33:46.946 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ah4.ko 2023-07-18T02:33:46.947 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/esp4.ko 2023-07-18T02:33:46.947 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/fou.ko 2023-07-18T02:33:46.947 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/gre.ko 2023-07-18T02:33:46.947 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/inet_diag.ko 2023-07-18T02:33:46.947 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ip_gre.ko 2023-07-18T02:33:46.947 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ip_tunnel.ko 2023-07-18T02:33:46.947 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ip_vti.ko 2023-07-18T02:33:46.947 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ipcomp.ko 2023-07-18T02:33:46.948 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ipip.ko 2023-07-18T02:33:46.948 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter 2023-07-18T02:33:46.948 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/arp_tables.ko 2023-07-18T02:33:46.948 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-07-18T02:33:46.948 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-07-18T02:33:46.948 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-07-18T02:33:46.948 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-07-18T02:33:46.949 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-07-18T02:33:46.949 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-07-18T02:33:46.949 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-07-18T02:33:46.949 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-07-18T02:33:46.949 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-07-18T02:33:46.949 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_security.ko 2023-07-18T02:33:46.949 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-07-18T02:33:46.949 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-07-18T02:33:46.950 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-07-18T02:33:46.950 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-07-18T02:33:46.950 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-07-18T02:33:46.950 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-07-18T02:33:46.950 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-07-18T02:33:46.950 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-07-18T02:33:46.950 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-07-18T02:33:46.951 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-07-18T02:33:46.951 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_bic.ko 2023-07-18T02:33:46.951 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_dctcp.ko 2023-07-18T02:33:46.951 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_diag.ko 2023-07-18T02:33:46.951 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_highspeed.ko 2023-07-18T02:33:46.951 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_htcp.ko 2023-07-18T02:33:46.951 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_hybla.ko 2023-07-18T02:33:46.951 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_illinois.ko 2023-07-18T02:33:46.952 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_lp.ko 2023-07-18T02:33:46.952 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_scalable.ko 2023-07-18T02:33:46.952 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_vegas.ko 2023-07-18T02:33:46.952 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_veno.ko 2023-07-18T02:33:46.952 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_westwood.ko 2023-07-18T02:33:46.952 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_yeah.ko 2023-07-18T02:33:46.952 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tunnel4.ko 2023-07-18T02:33:46.953 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/udp_diag.ko 2023-07-18T02:33:46.953 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/udp_tunnel.ko 2023-07-18T02:33:46.953 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/xfrm4_tunnel.ko 2023-07-18T02:33:46.953 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6 2023-07-18T02:33:46.954 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ah6.ko 2023-07-18T02:33:46.954 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/esp6.ko 2023-07-18T02:33:46.954 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/fou6.ko 2023-07-18T02:33:46.954 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ip6_tunnel.ko 2023-07-18T02:33:46.955 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-07-18T02:33:46.955 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ip6_vti.ko 2023-07-18T02:33:46.955 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ipcomp6.ko 2023-07-18T02:33:46.955 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter 2023-07-18T02:33:46.956 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-07-18T02:33:46.956 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-07-18T02:33:46.956 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-07-18T02:33:46.957 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-07-18T02:33:46.957 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-07-18T02:33:46.957 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-07-18T02:33:46.957 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-07-18T02:33:46.958 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-07-18T02:33:46.958 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-07-18T02:33:46.958 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-07-18T02:33:46.959 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-07-18T02:33:46.959 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-07-18T02:33:46.959 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-07-18T02:33:46.960 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-07-18T02:33:46.960 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-07-18T02:33:46.960 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-07-18T02:33:46.960 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-07-18T02:33:46.961 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-07-18T02:33:46.961 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-07-18T02:33:46.961 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-07-18T02:33:46.961 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-07-18T02:33:46.961 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-07-18T02:33:46.962 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-07-18T02:33:46.962 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-07-18T02:33:46.962 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/sit.ko 2023-07-18T02:33:46.962 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/tunnel6.ko 2023-07-18T02:33:46.963 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/xfrm6_tunnel.ko 2023-07-18T02:33:46.963 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/key 2023-07-18T02:33:46.963 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/key/af_key.ko 2023-07-18T02:33:46.963 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp 2023-07-18T02:33:46.964 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_core.ko 2023-07-18T02:33:46.964 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_debugfs.ko 2023-07-18T02:33:46.964 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_eth.ko 2023-07-18T02:33:46.964 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_ip.ko 2023-07-18T02:33:46.964 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_ip6.ko 2023-07-18T02:33:46.964 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_netlink.ko 2023-07-18T02:33:46.964 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_ppp.ko 2023-07-18T02:33:46.965 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/llc 2023-07-18T02:33:46.965 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/llc/llc.ko 2023-07-18T02:33:46.965 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mac802154 2023-07-18T02:33:46.965 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mac802154/mac802154.ko 2023-07-18T02:33:46.965 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mpls 2023-07-18T02:33:46.965 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mpls/mpls_gso.ko 2023-07-18T02:33:46.965 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter 2023-07-18T02:33:46.965 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset 2023-07-18T02:33:46.966 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set.ko 2023-07-18T02:33:46.966 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-07-18T02:33:46.966 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-07-18T02:33:46.966 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-07-18T02:33:46.966 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-07-18T02:33:46.966 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-07-18T02:33:46.966 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-07-18T02:33:46.967 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-07-18T02:33:46.967 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-07-18T02:33:46.967 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-07-18T02:33:46.967 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-07-18T02:33:46.967 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-07-18T02:33:46.967 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-07-18T02:33:46.967 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-07-18T02:33:46.967 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-07-18T02:33:46.968 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-07-18T02:33:46.968 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs 2023-07-18T02:33:46.968 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs.ko 2023-07-18T02:33:46.968 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-07-18T02:33:46.968 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-07-18T02:33:46.968 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-07-18T02:33:46.968 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-07-18T02:33:46.969 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-07-18T02:33:46.969 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-07-18T02:33:46.969 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-07-18T02:33:46.969 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-07-18T02:33:46.969 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-07-18T02:33:46.969 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-07-18T02:33:46.969 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-07-18T02:33:46.969 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-07-18T02:33:46.970 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-07-18T02:33:46.970 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conncount.ko 2023-07-18T02:33:46.970 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack.ko 2023-07-18T02:33:46.970 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-07-18T02:33:46.970 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-07-18T02:33:46.970 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-07-18T02:33:46.971 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_h323.ko 2023-07-18T02:33:46.971 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_irc.ko 2023-07-18T02:33:46.971 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-07-18T02:33:46.971 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-07-18T02:33:46.971 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-07-18T02:33:46.972 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_sane.ko 2023-07-18T02:33:46.972 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_sip.ko 2023-07-18T02:33:46.972 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-07-18T02:33:46.972 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-07-18T02:33:46.972 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_dup_netdev.ko 2023-07-18T02:33:46.972 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_flow_table.ko 2023-07-18T02:33:46.972 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_flow_table_inet.ko 2023-07-18T02:33:46.973 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_log_syslog.ko 2023-07-18T02:33:46.973 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat.ko 2023-07-18T02:33:46.973 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_amanda.ko 2023-07-18T02:33:46.973 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_ftp.ko 2023-07-18T02:33:46.973 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_irc.ko 2023-07-18T02:33:46.973 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_sip.ko 2023-07-18T02:33:46.973 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_tftp.ko 2023-07-18T02:33:46.973 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_synproxy_core.ko 2023-07-18T02:33:46.974 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_tables.ko 2023-07-18T02:33:46.974 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink.ko 2023-07-18T02:33:46.974 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_acct.ko 2023-07-18T02:33:46.974 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_log.ko 2023-07-18T02:33:46.974 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_osf.ko 2023-07-18T02:33:46.974 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_queue.ko 2023-07-18T02:33:46.974 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_chain_nat.ko 2023-07-18T02:33:46.975 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_compat.ko 2023-07-18T02:33:46.975 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_ct.ko 2023-07-18T02:33:46.975 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_dup_netdev.ko 2023-07-18T02:33:46.975 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_fib.ko 2023-07-18T02:33:46.975 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_fwd_netdev.ko 2023-07-18T02:33:46.975 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_hash.ko 2023-07-18T02:33:46.975 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_limit.ko 2023-07-18T02:33:46.975 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_log.ko 2023-07-18T02:33:46.976 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_masq.ko 2023-07-18T02:33:46.976 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_nat.ko 2023-07-18T02:33:46.976 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_numgen.ko 2023-07-18T02:33:46.976 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_queue.ko 2023-07-18T02:33:46.976 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_quota.ko 2023-07-18T02:33:46.976 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_redir.ko 2023-07-18T02:33:46.976 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_reject.ko 2023-07-18T02:33:46.977 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_reject_inet.ko 2023-07-18T02:33:46.977 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_socket.ko 2023-07-18T02:33:46.977 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_tunnel.ko 2023-07-18T02:33:46.977 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_AUDIT.ko 2023-07-18T02:33:46.977 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CHECKSUM.ko 2023-07-18T02:33:46.977 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CLASSIFY.ko 2023-07-18T02:33:46.977 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-07-18T02:33:46.977 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CT.ko 2023-07-18T02:33:46.978 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_DSCP.ko 2023-07-18T02:33:46.978 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_HL.ko 2023-07-18T02:33:46.978 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_HMARK.ko 2023-07-18T02:33:46.978 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_IDLETIMER.ko 2023-07-18T02:33:46.978 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_LED.ko 2023-07-18T02:33:46.978 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_LOG.ko 2023-07-18T02:33:46.978 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_MASQUERADE.ko 2023-07-18T02:33:46.978 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_NETMAP.ko 2023-07-18T02:33:46.979 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_NFLOG.ko 2023-07-18T02:33:46.979 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_NFQUEUE.ko 2023-07-18T02:33:46.979 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_RATEEST.ko 2023-07-18T02:33:46.979 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_REDIRECT.ko 2023-07-18T02:33:46.979 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_SECMARK.ko 2023-07-18T02:33:46.979 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TCPMSS.ko 2023-07-18T02:33:46.979 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-07-18T02:33:46.979 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TEE.ko 2023-07-18T02:33:46.980 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TPROXY.ko 2023-07-18T02:33:46.980 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TRACE.ko 2023-07-18T02:33:46.980 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_addrtype.ko 2023-07-18T02:33:46.980 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_bpf.ko 2023-07-18T02:33:46.980 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_cgroup.ko 2023-07-18T02:33:46.980 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_cluster.ko 2023-07-18T02:33:46.980 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_comment.ko 2023-07-18T02:33:46.981 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connbytes.ko 2023-07-18T02:33:46.981 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connlabel.ko 2023-07-18T02:33:46.981 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connlimit.ko 2023-07-18T02:33:46.981 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connmark.ko 2023-07-18T02:33:46.981 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_conntrack.ko 2023-07-18T02:33:46.981 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_cpu.ko 2023-07-18T02:33:46.981 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_dccp.ko 2023-07-18T02:33:46.981 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_devgroup.ko 2023-07-18T02:33:46.982 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_dscp.ko 2023-07-18T02:33:46.982 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_ecn.ko 2023-07-18T02:33:46.982 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_esp.ko 2023-07-18T02:33:46.982 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_hashlimit.ko 2023-07-18T02:33:46.982 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_helper.ko 2023-07-18T02:33:46.982 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_hl.ko 2023-07-18T02:33:46.982 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_ipcomp.ko 2023-07-18T02:33:46.983 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_iprange.ko 2023-07-18T02:33:46.983 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_ipvs.ko 2023-07-18T02:33:46.983 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_l2tp.ko 2023-07-18T02:33:46.983 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_length.ko 2023-07-18T02:33:46.983 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_limit.ko 2023-07-18T02:33:46.983 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_mac.ko 2023-07-18T02:33:46.983 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_mark.ko 2023-07-18T02:33:46.983 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_multiport.ko 2023-07-18T02:33:46.984 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_nat.ko 2023-07-18T02:33:46.984 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_nfacct.ko 2023-07-18T02:33:46.984 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_osf.ko 2023-07-18T02:33:46.984 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_owner.ko 2023-07-18T02:33:46.984 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_physdev.ko 2023-07-18T02:33:46.984 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_pkttype.ko 2023-07-18T02:33:46.984 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_policy.ko 2023-07-18T02:33:46.985 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_quota.ko 2023-07-18T02:33:46.985 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_rateest.ko 2023-07-18T02:33:46.985 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_realm.ko 2023-07-18T02:33:46.985 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_recent.ko 2023-07-18T02:33:46.985 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_sctp.ko 2023-07-18T02:33:46.985 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_set.ko 2023-07-18T02:33:46.985 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_socket.ko 2023-07-18T02:33:46.985 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_state.ko 2023-07-18T02:33:46.986 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_statistic.ko 2023-07-18T02:33:46.986 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_string.ko 2023-07-18T02:33:46.986 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_tcpmss.ko 2023-07-18T02:33:46.986 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_time.ko 2023-07-18T02:33:46.986 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_u32.ko 2023-07-18T02:33:46.986 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netlink 2023-07-18T02:33:46.986 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netlink/netlink_diag.ko 2023-07-18T02:33:46.986 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/nsh 2023-07-18T02:33:46.987 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/nsh/nsh.ko 2023-07-18T02:33:46.987 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch 2023-07-18T02:33:46.987 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/openvswitch.ko 2023-07-18T02:33:46.987 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/vport-geneve.ko 2023-07-18T02:33:46.987 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/vport-gre.ko 2023-07-18T02:33:46.987 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/vport-vxlan.ko 2023-07-18T02:33:46.987 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/packet 2023-07-18T02:33:46.988 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/packet/af_packet_diag.ko 2023-07-18T02:33:46.988 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds 2023-07-18T02:33:46.988 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds/rds.ko 2023-07-18T02:33:46.988 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds/rds_rdma.ko 2023-07-18T02:33:46.988 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds/rds_tcp.ko 2023-07-18T02:33:46.988 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched 2023-07-18T02:33:46.988 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_csum.ko 2023-07-18T02:33:46.988 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_gact.ko 2023-07-18T02:33:46.989 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_ipt.ko 2023-07-18T02:33:46.989 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_mirred.ko 2023-07-18T02:33:46.989 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_nat.ko 2023-07-18T02:33:46.989 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_pedit.ko 2023-07-18T02:33:46.989 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_police.ko 2023-07-18T02:33:46.989 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_simple.ko 2023-07-18T02:33:46.989 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_skbedit.ko 2023-07-18T02:33:46.990 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_vlan.ko 2023-07-18T02:33:46.990 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_basic.ko 2023-07-18T02:33:46.990 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_bpf.ko 2023-07-18T02:33:46.990 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_flow.ko 2023-07-18T02:33:46.990 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_fw.ko 2023-07-18T02:33:46.990 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_route.ko 2023-07-18T02:33:46.991 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_u32.ko 2023-07-18T02:33:46.991 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_cmp.ko 2023-07-18T02:33:46.991 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_ipset.ko 2023-07-18T02:33:46.991 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_meta.ko 2023-07-18T02:33:46.992 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_nbyte.ko 2023-07-18T02:33:46.992 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_text.ko 2023-07-18T02:33:46.992 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_u32.ko 2023-07-18T02:33:46.992 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_choke.ko 2023-07-18T02:33:46.992 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_codel.ko 2023-07-18T02:33:46.992 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_drr.ko 2023-07-18T02:33:46.992 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_fq.ko 2023-07-18T02:33:46.993 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_gred.ko 2023-07-18T02:33:46.993 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_hfsc.ko 2023-07-18T02:33:46.993 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_hhf.ko 2023-07-18T02:33:46.993 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_htb.ko 2023-07-18T02:33:46.993 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_ingress.ko 2023-07-18T02:33:46.993 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_mqprio.ko 2023-07-18T02:33:46.993 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_mqprio_lib.ko 2023-07-18T02:33:46.993 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_multiq.ko 2023-07-18T02:33:46.994 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_netem.ko 2023-07-18T02:33:46.994 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_pie.ko 2023-07-18T02:33:46.994 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_plug.ko 2023-07-18T02:33:46.994 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_prio.ko 2023-07-18T02:33:46.994 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_qfq.ko 2023-07-18T02:33:46.994 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_red.ko 2023-07-18T02:33:46.995 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_sfb.ko 2023-07-18T02:33:46.995 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_sfq.ko 2023-07-18T02:33:46.995 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_tbf.ko 2023-07-18T02:33:46.995 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_teql.ko 2023-07-18T02:33:46.995 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sctp 2023-07-18T02:33:46.995 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sctp/sctp.ko 2023-07-18T02:33:46.995 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sctp/sctp_diag.ko 2023-07-18T02:33:46.995 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc 2023-07-18T02:33:46.996 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/auth_gss 2023-07-18T02:33:46.996 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-07-18T02:33:46.996 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-07-18T02:33:46.996 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/sunrpc.ko 2023-07-18T02:33:46.996 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/xprtrdma 2023-07-18T02:33:46.996 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-07-18T02:33:46.996 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/tipc 2023-07-18T02:33:46.997 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/tipc/diag.ko 2023-07-18T02:33:46.997 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/tipc/tipc.ko 2023-07-18T02:33:46.997 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/unix 2023-07-18T02:33:46.997 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/unix/unix_diag.ko 2023-07-18T02:33:46.997 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock 2023-07-18T02:33:46.997 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-07-18T02:33:46.997 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-07-18T02:33:46.997 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vsock.ko 2023-07-18T02:33:46.998 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vsock_diag.ko 2023-07-18T02:33:46.998 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vsock_loopback.ko 2023-07-18T02:33:46.998 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/xfrm 2023-07-18T02:33:46.998 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/xfrm/xfrm_ipcomp.ko 2023-07-18T02:33:46.998 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security 2023-07-18T02:33:46.998 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys 2023-07-18T02:33:46.998 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/encrypted-keys 2023-07-18T02:33:46.998 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-07-18T02:33:46.999 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/trusted-keys 2023-07-18T02:33:46.999 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/trusted-keys/trusted.ko 2023-07-18T02:33:46.999 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/virt 2023-07-18T02:33:46.999 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/virt/lib 2023-07-18T02:33:46.999 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/virt/lib/irqbypass.ko 2023-07-18T02:33:46.999 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.alias 2023-07-18T02:33:46.999 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.alias.bin 2023-07-18T02:33:47.000 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.builtin 2023-07-18T02:33:47.000 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.builtin.bin 2023-07-18T02:33:47.000 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.builtin.modinfo 2023-07-18T02:33:47.000 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.dep 2023-07-18T02:33:47.000 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.dep.bin 2023-07-18T02:33:47.000 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.devname 2023-07-18T02:33:47.000 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.order 2023-07-18T02:33:47.000 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.softdep 2023-07-18T02:33:47.001 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.symbols 2023-07-18T02:33:47.001 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.symbols.bin 2023-07-18T02:33:47.001 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc1-ga440c79eee02 2023-07-18T02:33:47.002 DEBUG:teuthology.orchestra.run.smithi101:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-07-18T02:33:47.029 INFO:teuthology.orchestra.run.smithi101.stdout:postinstall scriptlet (using /bin/sh): 2023-07-18T02:33:47.030 INFO:teuthology.orchestra.run.smithi101.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc1-ga440c79eee02 -a -r /boot/System.map-6.5.0-rc1-ga440c79eee02 ]; then 2023-07-18T02:33:47.030 INFO:teuthology.orchestra.run.smithi101.stdout:cp /boot/vmlinuz-6.5.0-rc1-ga440c79eee02 /boot/.vmlinuz-6.5.0-rc1-ga440c79eee02-rpm 2023-07-18T02:33:47.030 INFO:teuthology.orchestra.run.smithi101.stdout:cp /boot/System.map-6.5.0-rc1-ga440c79eee02 /boot/.System.map-6.5.0-rc1-ga440c79eee02-rpm 2023-07-18T02:33:47.031 INFO:teuthology.orchestra.run.smithi101.stdout:rm -f /boot/vmlinuz-6.5.0-rc1-ga440c79eee02 /boot/System.map-6.5.0-rc1-ga440c79eee02 2023-07-18T02:33:47.031 INFO:teuthology.orchestra.run.smithi101.stdout:/sbin/installkernel 6.5.0-rc1-ga440c79eee02 /boot/.vmlinuz-6.5.0-rc1-ga440c79eee02-rpm /boot/.System.map-6.5.0-rc1-ga440c79eee02-rpm 2023-07-18T02:33:47.031 INFO:teuthology.orchestra.run.smithi101.stdout:rm -f /boot/.vmlinuz-6.5.0-rc1-ga440c79eee02-rpm /boot/.System.map-6.5.0-rc1-ga440c79eee02-rpm 2023-07-18T02:33:47.031 INFO:teuthology.orchestra.run.smithi101.stdout:fi 2023-07-18T02:33:47.031 INFO:teuthology.orchestra.run.smithi101.stdout:preuninstall scriptlet (using /bin/sh): 2023-07-18T02:33:47.032 INFO:teuthology.orchestra.run.smithi101.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-07-18T02:33:47.032 INFO:teuthology.orchestra.run.smithi101.stdout:new-kernel-pkg --remove 6.5.0-rc1-ga440c79eee02 --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc1-ga440c79eee02.img 2023-07-18T02:33:47.032 INFO:teuthology.orchestra.run.smithi101.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-07-18T02:33:47.032 INFO:teuthology.orchestra.run.smithi101.stdout:kernel-install remove 6.5.0-rc1-ga440c79eee02 2023-07-18T02:33:47.033 INFO:teuthology.orchestra.run.smithi101.stdout:fi 2023-07-18T02:33:47.033 INFO:teuthology.orchestra.run.smithi101.stdout:postuninstall scriptlet (using /bin/sh): 2023-07-18T02:33:47.033 INFO:teuthology.orchestra.run.smithi101.stdout:if [ -x /sbin/update-bootloader ]; then 2023-07-18T02:33:47.033 INFO:teuthology.orchestra.run.smithi101.stdout:/sbin/update-bootloader --remove 6.5.0-rc1-ga440c79eee02 2023-07-18T02:33:47.033 INFO:teuthology.orchestra.run.smithi101.stdout:fi 2023-07-18T02:33:47.034 DEBUG:teuthology.orchestra.run.smithi101:> sudo rpm -qi grub2-tools 2023-07-18T02:33:47.114 INFO:teuthology.orchestra.run.smithi101.stdout:Name : grub2-tools 2023-07-18T02:33:47.114 INFO:teuthology.orchestra.run.smithi101.stdout:Epoch : 1 2023-07-18T02:33:47.114 INFO:teuthology.orchestra.run.smithi101.stdout:Version : 2.02 2023-07-18T02:33:47.114 INFO:teuthology.orchestra.run.smithi101.stdout:Release : 123.el8 2023-07-18T02:33:47.115 INFO:teuthology.orchestra.run.smithi101.stdout:Architecture: x86_64 2023-07-18T02:33:47.115 INFO:teuthology.orchestra.run.smithi101.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-07-18T02:33:47.115 INFO:teuthology.orchestra.run.smithi101.stdout:Group : System Environment/Base 2023-07-18T02:33:47.115 INFO:teuthology.orchestra.run.smithi101.stdout:Size : 9265450 2023-07-18T02:33:47.115 INFO:teuthology.orchestra.run.smithi101.stdout:License : GPLv3+ 2023-07-18T02:33:47.116 INFO:teuthology.orchestra.run.smithi101.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-07-18T02:33:47.116 INFO:teuthology.orchestra.run.smithi101.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-07-18T02:33:47.116 INFO:teuthology.orchestra.run.smithi101.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-07-18T02:33:47.116 INFO:teuthology.orchestra.run.smithi101.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-07-18T02:33:47.116 INFO:teuthology.orchestra.run.smithi101.stdout:Relocations : (not relocatable) 2023-07-18T02:33:47.116 INFO:teuthology.orchestra.run.smithi101.stdout:Packager : Red Hat, Inc. 2023-07-18T02:33:47.116 INFO:teuthology.orchestra.run.smithi101.stdout:Vendor : Red Hat, Inc. 2023-07-18T02:33:47.117 INFO:teuthology.orchestra.run.smithi101.stdout:URL : http://www.gnu.org/software/grub/ 2023-07-18T02:33:47.117 INFO:teuthology.orchestra.run.smithi101.stdout:Summary : Support tools for GRUB. 2023-07-18T02:33:47.117 INFO:teuthology.orchestra.run.smithi101.stdout:Description : 2023-07-18T02:33:47.117 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T02:33:47.117 INFO:teuthology.orchestra.run.smithi101.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-07-18T02:33:47.117 INFO:teuthology.orchestra.run.smithi101.stdout:customizable bootloader with modular architecture. It supports a rich 2023-07-18T02:33:47.117 INFO:teuthology.orchestra.run.smithi101.stdout:variety of kernel formats, file systems, computer architectures and 2023-07-18T02:33:47.118 INFO:teuthology.orchestra.run.smithi101.stdout:hardware devices. 2023-07-18T02:33:47.118 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T02:33:47.118 INFO:teuthology.orchestra.run.smithi101.stdout:This subpackage provides tools for support of all platforms. 2023-07-18T02:33:47.119 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-07-18T02:33:47.119 INFO:teuthology.task.kernel:Updating grub on smithi101 to boot 6.5.0-rc1-ga440c79eee02 2023-07-18T02:33:47.119 DEBUG:teuthology.orchestra.run.smithi101:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-07-18T02:33:49.369 INFO:teuthology.orchestra.run.smithi101.stderr:Generating grub configuration file ... 2023-07-18T02:33:51.311 INFO:teuthology.orchestra.run.smithi101.stderr:done 2023-07-18T02:33:51.313 DEBUG:teuthology.orchestra.run.smithi101:> mktemp 2023-07-18T02:33:51.337 INFO:teuthology.orchestra.run.smithi101.stdout:/tmp/tmp.NDCCxEUbXt 2023-07-18T02:33:51.337 DEBUG:teuthology.orchestra.run.smithi101:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.NDCCxEUbXt 2023-07-18T02:33:51.415 DEBUG:teuthology.orchestra.run.smithi101:> sudo chmod 0666 /tmp/tmp.NDCCxEUbXt 2023-07-18T02:33:51.571 DEBUG:teuthology.orchestra.remote:smithi101:/tmp/tmp.NDCCxEUbXt is 6KB 2023-07-18T02:33:51.618 DEBUG:teuthology.orchestra.run.smithi101:> rm -fr /tmp/tmp.NDCCxEUbXt 2023-07-18T02:33:51.634 DEBUG:teuthology.orchestra.run.smithi101:> sudo /bin/ls /boot/loader/entries || true 2023-07-18T02:33:51.709 INFO:teuthology.orchestra.run.smithi101.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-07-18T02:33:51.710 INFO:teuthology.orchestra.run.smithi101.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-07-18T02:33:51.710 INFO:teuthology.orchestra.run.smithi101.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc1-ga440c79eee02.conf 2023-07-18T02:33:51.711 DEBUG:teuthology.orchestra.run.smithi101:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc1-ga440c79eee02 2023-07-18T02:33:51.860 DEBUG:teuthology.orchestra.run.smithi101:> sudo shutdown -r now 2023-07-18T02:34:21.863 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-07-18T02:34:21.864 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com' 2023-07-18T02:34:21.864 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi101.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-18T02:35:26.870 DEBUG:teuthology.orchestra.remote:timed out 2023-07-18T02:35:35.873 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com' 2023-07-18T02:35:35.874 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi101.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-18T02:35:41.221 DEBUG:teuthology.orchestra.run.smithi101:> true 2023-07-18T02:35:41.613 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi101.front.sepia.ceph.com' 2023-07-18T02:35:41.613 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.5.0-rc1-ga440c79eee02"... 2023-07-18T02:35:41.614 DEBUG:teuthology.orchestra.run.smithi101:> uname -r 2023-07-18T02:35:41.656 INFO:teuthology.orchestra.run.smithi101.stdout:6.5.0-rc1-ga440c79eee02 2023-07-18T02:35:41.656 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc1-ga440c79eee02 vs 6.5.0-rc1-ga440c79eee02 2023-07-18T02:35:41.656 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-07-18T02:35:41.657 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-07-18T02:35:42.658 DEBUG:teuthology.parallel:result is None 2023-07-18T02:35:42.658 INFO:teuthology.run_tasks:Running task internal.base... 2023-07-18T02:35:42.666 INFO:teuthology.task.internal:Creating test directory... 2023-07-18T02:35:42.666 DEBUG:teuthology.orchestra.run.smithi101:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-18T02:35:42.671 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-18T02:35:42.673 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-18T02:35:42.714 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-07-18T02:35:42.721 INFO:teuthology.run_tasks:Running task internal.archive... 2023-07-18T02:35:42.731 INFO:teuthology.task.internal:Creating archive directory... 2023-07-18T02:35:42.731 DEBUG:teuthology.orchestra.run.smithi101:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-18T02:35:42.736 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-18T02:35:42.738 DEBUG:teuthology.orchestra.run.smithi179:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-18T02:35:42.795 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-07-18T02:35:42.802 INFO:teuthology.task.internal:Enabling coredump saving... 2023-07-18T02:35:42.802 DEBUG:teuthology.orchestra.run.smithi155:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-18T02:35:42.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-18T02:35:42.822 DEBUG:teuthology.orchestra.run.smithi179:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-18T02:35:42.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-18T02:35:42.841 DEBUG:teuthology.orchestra.run.smithi101:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-18T02:35:42.845 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-18T02:35:42.864 DEBUG:teuthology.orchestra.run.smithi179:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-18T02:35:42.921 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-18T02:35:42.933 INFO:teuthology.orchestra.run.smithi179.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-18T02:35:42.938 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-18T02:35:42.952 INFO:teuthology.orchestra.run.smithi179.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-18T02:35:42.981 INFO:teuthology.orchestra.run.smithi101.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-18T02:35:43.012 INFO:teuthology.orchestra.run.smithi101.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-18T02:35:43.015 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-07-18T02:35:43.023 INFO:teuthology.task.internal:Configuring sudo... 2023-07-18T02:35:43.023 DEBUG:teuthology.orchestra.run.smithi101:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-18T02:35:43.028 DEBUG:teuthology.orchestra.run.smithi155:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-18T02:35:43.030 DEBUG:teuthology.orchestra.run.smithi179:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-18T02:35:43.092 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-07-18T02:35:43.102 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-07-18T02:35:43.102 DEBUG:teuthology.orchestra.run.smithi101:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-18T02:35:43.107 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-18T02:35:43.109 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-18T02:35:43.145 DEBUG:teuthology.orchestra.run.smithi101:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-18T02:35:44.022 DEBUG:teuthology.orchestra.run.smithi101:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-18T02:35:44.097 DEBUG:teuthology.orchestra.run.smithi101:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-18T02:35:44.168 DEBUG:teuthology.orchestra.run.smithi101:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-18T02:35:44.236 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-07-18T02:35:44.237 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-18T02:35:44.303 DEBUG:teuthology.orchestra.run.smithi155:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-18T02:35:44.350 DEBUG:teuthology.orchestra.run.smithi155:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-18T02:35:44.406 DEBUG:teuthology.orchestra.run.smithi155:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-18T02:35:44.496 DEBUG:teuthology.orchestra.run.smithi155:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-18T02:35:44.537 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-07-18T02:35:44.537 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-18T02:35:44.606 DEBUG:teuthology.orchestra.run.smithi179:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-18T02:35:44.656 DEBUG:teuthology.orchestra.run.smithi179:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-18T02:35:44.711 DEBUG:teuthology.orchestra.run.smithi179:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-18T02:35:44.801 DEBUG:teuthology.orchestra.run.smithi179:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-18T02:35:44.842 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2023-07-18T02:35:44.842 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-18T02:35:44.912 DEBUG:teuthology.orchestra.run.smithi101:> sudo service rsyslog restart 2023-07-18T02:35:44.916 DEBUG:teuthology.orchestra.run.smithi155:> sudo service rsyslog restart 2023-07-18T02:35:44.919 DEBUG:teuthology.orchestra.run.smithi179:> sudo service rsyslog restart 2023-07-18T02:35:44.991 INFO:teuthology.orchestra.run.smithi155.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-18T02:35:45.016 INFO:teuthology.orchestra.run.smithi179.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-18T02:35:45.029 INFO:teuthology.orchestra.run.smithi101.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-18T02:35:45.503 INFO:teuthology.run_tasks:Running task internal.timer... 2023-07-18T02:35:45.511 INFO:teuthology.task.internal:Starting timer... 2023-07-18T02:35:45.511 INFO:teuthology.run_tasks:Running task pcp... 2023-07-18T02:35:45.532 INFO:teuthology.run_tasks:Running task selinux... 2023-07-18T02:35:45.541 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-07-18T02:35:45.542 DEBUG:teuthology.orchestra.run.smithi101:> sudo service auditd rotate 2023-07-18T02:35:45.694 INFO:teuthology.orchestra.run.smithi101.stdout:Rotating logs: 2023-07-18T02:35:45.697 DEBUG:teuthology.orchestra.run.smithi155:> sudo service auditd rotate 2023-07-18T02:35:45.771 INFO:teuthology.orchestra.run.smithi155.stdout:Rotating logs: 2023-07-18T02:35:45.772 DEBUG:teuthology.orchestra.run.smithi179:> sudo service auditd rotate 2023-07-18T02:35:45.846 INFO:teuthology.orchestra.run.smithi179.stdout:Rotating logs: 2023-07-18T02:35:45.848 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-07-18T02:35:45.849 DEBUG:teuthology.orchestra.run.smithi101:> /usr/sbin/getenforce 2023-07-18T02:35:45.902 INFO:teuthology.orchestra.run.smithi101.stdout:Permissive 2023-07-18T02:35:45.903 DEBUG:teuthology.orchestra.run.smithi155:> /usr/sbin/getenforce 2023-07-18T02:35:45.925 INFO:teuthology.orchestra.run.smithi155.stdout:Permissive 2023-07-18T02:35:45.925 DEBUG:teuthology.orchestra.run.smithi179:> /usr/sbin/getenforce 2023-07-18T02:35:45.953 INFO:teuthology.orchestra.run.smithi179.stdout:Permissive 2023-07-18T02:35:45.953 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi101.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi155.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi179.front.sepia.ceph.com': 'permissive'} 2023-07-18T02:35:45.954 DEBUG:teuthology.orchestra.run.smithi101:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-18T02:35:46.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-18T02:35:46.023 DEBUG:teuthology.orchestra.run.smithi155:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-18T02:35:46.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-18T02:35:46.051 DEBUG:teuthology.orchestra.run.smithi179:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-18T02:35:46.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-18T02:35:46.080 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-07-18T02:35:46.080 DEBUG:teuthology.orchestra.run.smithi101:> sudo /usr/sbin/setenforce permissive 2023-07-18T02:35:46.147 DEBUG:teuthology.orchestra.run.smithi155:> sudo /usr/sbin/setenforce permissive 2023-07-18T02:35:46.195 DEBUG:teuthology.orchestra.run.smithi179:> sudo /usr/sbin/setenforce permissive 2023-07-18T02:35:46.222 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-07-18T02:35:46.232 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-07-18T02:35:46.238 INFO:teuthology.repo_utils:Fetching main from origin 2023-07-18T02:35:51.386 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-07-18T02:35:51.405 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-07-18T02:35:51.407 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi101.front.sepia.ceph.com,smithi155.front.sepia.ceph.com,smithi179.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-07-18T02:51:58.783 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi101.front.sepia.ceph.com'), Remote(name='ubuntu@smithi155.front.sepia.ceph.com'), Remote(name='ubuntu@smithi179.front.sepia.ceph.com')] 2023-07-18T02:51:58.785 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com' 2023-07-18T02:51:58.786 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi101.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-18T02:52:03.886 DEBUG:teuthology.orchestra.run.smithi101:> true 2023-07-18T02:52:04.005 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi101.front.sepia.ceph.com' 2023-07-18T02:52:04.006 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2023-07-18T02:52:04.007 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-18T02:52:09.084 DEBUG:teuthology.orchestra.run.smithi155:> true 2023-07-18T02:52:09.112 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi155.front.sepia.ceph.com' 2023-07-18T02:52:09.112 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com' 2023-07-18T02:52:09.113 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-18T02:52:14.187 DEBUG:teuthology.orchestra.run.smithi179:> true 2023-07-18T02:52:14.256 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi179.front.sepia.ceph.com' 2023-07-18T02:52:14.256 INFO:teuthology.run_tasks:Running task clock... 2023-07-18T02:52:14.268 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-07-18T02:52:14.269 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-18T02:52:14.270 DEBUG:teuthology.orchestra.run.smithi101:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-18T02:52:14.274 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-18T02:52:14.275 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-18T02:52:14.277 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-18T02:52:14.278 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-18T02:52:14.312 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-18T02:52:14.329 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-18T02:52:14.332 INFO:teuthology.orchestra.run.smithi179.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-18T02:52:14.348 INFO:teuthology.orchestra.run.smithi179.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-18T02:52:14.350 INFO:teuthology.orchestra.run.smithi101.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-18T02:52:14.385 INFO:teuthology.orchestra.run.smithi101.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-18T02:52:14.400 INFO:teuthology.orchestra.run.smithi155.stderr:sudo: ntpd: command not found 2023-07-18T02:52:14.413 INFO:teuthology.orchestra.run.smithi155.stdout:506 Cannot talk to daemon 2023-07-18T02:52:14.426 INFO:teuthology.orchestra.run.smithi179.stderr:sudo: ntpd: command not found 2023-07-18T02:52:14.428 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-18T02:52:14.438 INFO:teuthology.orchestra.run.smithi179.stdout:506 Cannot talk to daemon 2023-07-18T02:52:14.443 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-18T02:52:14.453 INFO:teuthology.orchestra.run.smithi179.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-18T02:52:14.467 INFO:teuthology.orchestra.run.smithi179.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-18T02:52:14.478 INFO:teuthology.orchestra.run.smithi101.stderr:sudo: ntpd: command not found 2023-07-18T02:52:14.507 INFO:teuthology.orchestra.run.smithi101.stdout:506 Cannot talk to daemon 2023-07-18T02:52:14.517 INFO:teuthology.orchestra.run.smithi155.stderr:bash: ntpq: command not found 2023-07-18T02:52:14.520 INFO:teuthology.orchestra.run.smithi155.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-18T02:52:14.520 INFO:teuthology.orchestra.run.smithi155.stdout:=============================================================================== 2023-07-18T02:52:14.520 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-18T02:52:14.521 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-18T02:52:14.521 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-18T02:52:14.521 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-18T02:52:14.543 INFO:teuthology.orchestra.run.smithi101.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-18T02:52:14.546 INFO:teuthology.orchestra.run.smithi179.stderr:bash: ntpq: command not found 2023-07-18T02:52:14.549 INFO:teuthology.orchestra.run.smithi179.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-18T02:52:14.550 INFO:teuthology.orchestra.run.smithi179.stdout:=============================================================================== 2023-07-18T02:52:14.550 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-18T02:52:14.550 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-18T02:52:14.550 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-18T02:52:14.550 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-18T02:52:14.575 INFO:teuthology.orchestra.run.smithi101.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-18T02:52:14.663 INFO:teuthology.orchestra.run.smithi101.stderr:bash: ntpq: command not found 2023-07-18T02:52:14.670 INFO:teuthology.orchestra.run.smithi101.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-18T02:52:14.670 INFO:teuthology.orchestra.run.smithi101.stdout:=============================================================================== 2023-07-18T02:52:14.670 INFO:teuthology.orchestra.run.smithi101.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-18T02:52:14.670 INFO:teuthology.orchestra.run.smithi101.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-18T02:52:14.671 INFO:teuthology.orchestra.run.smithi101.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-18T02:52:14.671 INFO:teuthology.orchestra.run.smithi101.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-18T02:52:14.672 INFO:teuthology.run_tasks:Running task pexec... 2023-07-18T02:52:14.683 INFO:teuthology.task.pexec:Executing custom commands... 2023-07-18T02:52:14.684 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi101.front.sepia.ceph.com 2023-07-18T02:52:14.684 DEBUG:teuthology.orchestra.run.smithi101:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-07-18T02:52:14.685 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi155.front.sepia.ceph.com 2023-07-18T02:52:14.685 DEBUG:teuthology.orchestra.run.smithi155:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-07-18T02:52:14.685 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi179.front.sepia.ceph.com 2023-07-18T02:52:14.686 DEBUG:teuthology.orchestra.run.smithi179:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-07-18T02:52:15.060 INFO:teuthology.orchestra.run.smithi179.stdout:Updating Subscription Management repositories. 2023-07-18T02:52:15.060 INFO:teuthology.orchestra.run.smithi155.stdout:Updating Subscription Management repositories. 2023-07-18T02:52:15.166 INFO:teuthology.orchestra.run.smithi101.stdout:Updating Subscription Management repositories. 2023-07-18T02:52:16.627 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:02:11 ago on Tue 18 Jul 2023 02:50:05 AM UTC. 2023-07-18T02:52:16.679 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:08:42 ago on Tue 18 Jul 2023 02:43:34 AM UTC. 2023-07-18T02:52:17.020 INFO:teuthology.orchestra.run.smithi101.stdout:Last metadata expiration check: 0:06:22 ago on Tue 18 Jul 2023 02:45:54 AM UTC. 2023-07-18T02:52:18.366 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2023-07-18T02:52:18.367 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2023-07-18T02:52:18.367 INFO:teuthology.orchestra.run.smithi179.stdout: Package Architecture Version Repository Size 2023-07-18T02:52:18.367 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2023-07-18T02:52:18.367 INFO:teuthology.orchestra.run.smithi179.stdout:Resetting modules: 2023-07-18T02:52:18.367 INFO:teuthology.orchestra.run.smithi179.stdout: container-tools 2023-07-18T02:52:18.367 INFO:teuthology.orchestra.run.smithi179.stdout: 2023-07-18T02:52:18.367 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2023-07-18T02:52:18.368 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2023-07-18T02:52:18.368 INFO:teuthology.orchestra.run.smithi179.stdout: 2023-07-18T02:52:18.433 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2023-07-18T02:52:18.434 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2023-07-18T02:52:18.434 INFO:teuthology.orchestra.run.smithi155.stdout: Package Architecture Version Repository Size 2023-07-18T02:52:18.434 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2023-07-18T02:52:18.435 INFO:teuthology.orchestra.run.smithi155.stdout:Resetting modules: 2023-07-18T02:52:18.435 INFO:teuthology.orchestra.run.smithi155.stdout: container-tools 2023-07-18T02:52:18.435 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-07-18T02:52:18.435 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2023-07-18T02:52:18.435 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2023-07-18T02:52:18.435 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-07-18T02:52:18.787 INFO:teuthology.orchestra.run.smithi101.stdout:Dependencies resolved. 2023-07-18T02:52:18.788 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-07-18T02:52:18.789 INFO:teuthology.orchestra.run.smithi101.stdout: Package Architecture Version Repository Size 2023-07-18T02:52:18.789 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-07-18T02:52:18.789 INFO:teuthology.orchestra.run.smithi101.stdout:Resetting modules: 2023-07-18T02:52:18.789 INFO:teuthology.orchestra.run.smithi101.stdout: container-tools 2023-07-18T02:52:18.789 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T02:52:18.789 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction Summary 2023-07-18T02:52:18.789 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-07-18T02:52:18.790 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T02:52:19.323 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:02:13 ago on Tue 18 Jul 2023 02:50:05 AM UTC. 2023-07-18T02:52:19.392 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:08:44 ago on Tue 18 Jul 2023 02:43:34 AM UTC. 2023-07-18T02:52:20.021 INFO:teuthology.orchestra.run.smithi101.stdout:Last metadata expiration check: 0:06:25 ago on Tue 18 Jul 2023 02:45:54 AM UTC. 2023-07-18T02:52:25.324 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2023-07-18T02:52:26.083 INFO:teuthology.orchestra.run.smithi101.stdout:Complete! 2023-07-18T02:52:26.124 INFO:teuthology.orchestra.run.smithi179.stdout:Updating Subscription Management repositories. 2023-07-18T02:52:26.660 INFO:teuthology.orchestra.run.smithi101.stdout:Updating Subscription Management repositories. 2023-07-18T02:52:26.960 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2023-07-18T02:52:27.420 INFO:teuthology.orchestra.run.smithi155.stdout:Updating Subscription Management repositories. 2023-07-18T02:52:27.589 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:02:22 ago on Tue 18 Jul 2023 02:50:05 AM UTC. 2023-07-18T02:52:28.325 INFO:teuthology.orchestra.run.smithi101.stdout:Last metadata expiration check: 0:06:33 ago on Tue 18 Jul 2023 02:45:54 AM UTC. 2023-07-18T02:52:28.870 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:08:54 ago on Tue 18 Jul 2023 02:43:34 AM UTC. 2023-07-18T02:52:29.440 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2023-07-18T02:52:29.536 INFO:teuthology.orchestra.run.smithi179.stdout:============================================================================================================ 2023-07-18T02:52:29.536 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2023-07-18T02:52:29.536 INFO:teuthology.orchestra.run.smithi179.stdout:============================================================================================================ 2023-07-18T02:52:29.537 INFO:teuthology.orchestra.run.smithi179.stdout:Installing group/module packages: 2023-07-18T02:52:29.537 INFO:teuthology.orchestra.run.smithi179.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-07-18T02:52:29.537 INFO:teuthology.orchestra.run.smithi179.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-07-18T02:52:29.537 INFO:teuthology.orchestra.run.smithi179.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-07-18T02:52:29.537 INFO:teuthology.orchestra.run.smithi179.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-07-18T02:52:29.538 INFO:teuthology.orchestra.run.smithi179.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-07-18T02:52:29.538 INFO:teuthology.orchestra.run.smithi179.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-07-18T02:52:29.538 INFO:teuthology.orchestra.run.smithi179.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-07-18T02:52:29.538 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2023-07-18T02:52:29.538 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-07-18T02:52:29.538 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-07-18T02:52:29.539 INFO:teuthology.orchestra.run.smithi179.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-07-18T02:52:29.539 INFO:teuthology.orchestra.run.smithi179.stdout:Installing module profiles: 2023-07-18T02:52:29.539 INFO:teuthology.orchestra.run.smithi179.stdout: container-tools/common 2023-07-18T02:52:29.539 INFO:teuthology.orchestra.run.smithi179.stdout: 2023-07-18T02:52:29.539 INFO:teuthology.orchestra.run.smithi179.stdout:Enabling module streams: 2023-07-18T02:52:29.540 INFO:teuthology.orchestra.run.smithi179.stdout: container-tools rhel8 2023-07-18T02:52:29.540 INFO:teuthology.orchestra.run.smithi179.stdout: 2023-07-18T02:52:29.540 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2023-07-18T02:52:29.540 INFO:teuthology.orchestra.run.smithi179.stdout:============================================================================================================ 2023-07-18T02:52:29.540 INFO:teuthology.orchestra.run.smithi179.stdout:Install 10 Packages 2023-07-18T02:52:29.540 INFO:teuthology.orchestra.run.smithi179.stdout: 2023-07-18T02:52:29.541 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 18 M 2023-07-18T02:52:29.541 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 65 M 2023-07-18T02:52:29.541 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2023-07-18T02:52:29.681 INFO:teuthology.orchestra.run.smithi179.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 660 kB/s | 94 kB 00:00 2023-07-18T02:52:29.731 INFO:teuthology.orchestra.run.smithi179.stdout:(2/10): yajl-2.1.0-10.el8.x86_64.rpm 212 kB/s | 41 kB 00:00 2023-07-18T02:52:29.756 INFO:teuthology.orchestra.run.smithi179.stdout:(3/10): python3-pytoml-0.1.14-5.git7dea353.el8. 114 kB/s | 25 kB 00:00 2023-07-18T02:52:29.823 INFO:teuthology.orchestra.run.smithi179.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 347 kB/s | 49 kB 00:00 2023-07-18T02:52:29.907 INFO:teuthology.orchestra.run.smithi179.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 2.9 MB/s | 524 kB 00:00 2023-07-18T02:52:29.974 INFO:teuthology.orchestra.run.smithi179.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 993 kB/s | 149 kB 00:00 2023-07-18T02:52:30.092 INFO:teuthology.orchestra.run.smithi101.stdout:Dependencies resolved. 2023-07-18T02:52:30.141 INFO:teuthology.orchestra.run.smithi179.stdout:(7/10): buildah-1.26.2-1.module+el8.6.0+16771+2 21 MB/s | 8.0 MB 00:00 2023-07-18T02:52:30.166 INFO:teuthology.orchestra.run.smithi179.stdout:(8/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 822 kB/s | 213 kB 00:00 2023-07-18T02:52:30.190 INFO:teuthology.orchestra.run.smithi101.stdout:============================================================================================================ 2023-07-18T02:52:30.191 INFO:teuthology.orchestra.run.smithi101.stdout: Package Arch Version Repository Size 2023-07-18T02:52:30.191 INFO:teuthology.orchestra.run.smithi101.stdout:============================================================================================================ 2023-07-18T02:52:30.191 INFO:teuthology.orchestra.run.smithi101.stdout:Installing group/module packages: 2023-07-18T02:52:30.191 INFO:teuthology.orchestra.run.smithi101.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-07-18T02:52:30.191 INFO:teuthology.orchestra.run.smithi101.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-07-18T02:52:30.191 INFO:teuthology.orchestra.run.smithi101.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-07-18T02:52:30.192 INFO:teuthology.orchestra.run.smithi101.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-07-18T02:52:30.192 INFO:teuthology.orchestra.run.smithi101.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-07-18T02:52:30.192 INFO:teuthology.orchestra.run.smithi101.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-07-18T02:52:30.192 INFO:teuthology.orchestra.run.smithi101.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-07-18T02:52:30.192 INFO:teuthology.orchestra.run.smithi101.stdout:Installing dependencies: 2023-07-18T02:52:30.193 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-07-18T02:52:30.193 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-07-18T02:52:30.193 INFO:teuthology.orchestra.run.smithi101.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-07-18T02:52:30.193 INFO:teuthology.orchestra.run.smithi101.stdout:Installing module profiles: 2023-07-18T02:52:30.193 INFO:teuthology.orchestra.run.smithi101.stdout: container-tools/common 2023-07-18T02:52:30.193 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T02:52:30.194 INFO:teuthology.orchestra.run.smithi101.stdout:Enabling module streams: 2023-07-18T02:52:30.195 INFO:teuthology.orchestra.run.smithi101.stdout: container-tools rhel8 2023-07-18T02:52:30.195 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T02:52:30.195 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction Summary 2023-07-18T02:52:30.195 INFO:teuthology.orchestra.run.smithi101.stdout:============================================================================================================ 2023-07-18T02:52:30.195 INFO:teuthology.orchestra.run.smithi101.stdout:Install 10 Packages 2023-07-18T02:52:30.195 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T02:52:30.196 INFO:teuthology.orchestra.run.smithi101.stdout:Total download size: 18 M 2023-07-18T02:52:30.196 INFO:teuthology.orchestra.run.smithi101.stdout:Installed size: 65 M 2023-07-18T02:52:30.196 INFO:teuthology.orchestra.run.smithi101.stdout:Downloading Packages: 2023-07-18T02:52:30.308 INFO:teuthology.orchestra.run.smithi179.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 21 MB/s | 7.0 MB 00:00 2023-07-18T02:52:30.359 INFO:teuthology.orchestra.run.smithi179.stdout:(10/10): toolbox-0.0.99.3-0.6.module+el8.6.0+16 10 MB/s | 2.2 MB 00:00 2023-07-18T02:52:30.359 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2023-07-18T02:52:30.360 INFO:teuthology.orchestra.run.smithi179.stdout:Total 22 MB/s | 18 MB 00:00 2023-07-18T02:52:30.374 INFO:teuthology.orchestra.run.smithi101.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 527 kB/s | 94 kB 00:00 2023-07-18T02:52:30.397 INFO:teuthology.orchestra.run.smithi101.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 122 kB/s | 25 kB 00:00 2023-07-18T02:52:30.423 INFO:teuthology.orchestra.run.smithi101.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 178 kB/s | 41 kB 00:00 2023-07-18T02:52:30.481 INFO:teuthology.orchestra.run.smithi101.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 452 kB/s | 49 kB 00:00 2023-07-18T02:52:30.523 INFO:teuthology.orchestra.run.smithi101.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 4.1 MB/s | 524 kB 00:00 2023-07-18T02:52:30.549 INFO:teuthology.orchestra.run.smithi101.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.2 MB/s | 149 kB 00:00 2023-07-18T02:52:30.560 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2023-07-18T02:52:30.566 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2023-07-18T02:52:30.578 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2023-07-18T02:52:30.579 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2023-07-18T02:52:30.658 INFO:teuthology.orchestra.run.smithi155.stdout:============================================================================================================ 2023-07-18T02:52:30.659 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2023-07-18T02:52:30.659 INFO:teuthology.orchestra.run.smithi155.stdout:============================================================================================================ 2023-07-18T02:52:30.659 INFO:teuthology.orchestra.run.smithi155.stdout:Installing group/module packages: 2023-07-18T02:52:30.659 INFO:teuthology.orchestra.run.smithi155.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-07-18T02:52:30.659 INFO:teuthology.orchestra.run.smithi155.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-07-18T02:52:30.659 INFO:teuthology.orchestra.run.smithi155.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-07-18T02:52:30.660 INFO:teuthology.orchestra.run.smithi155.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-07-18T02:52:30.660 INFO:teuthology.orchestra.run.smithi155.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-07-18T02:52:30.660 INFO:teuthology.orchestra.run.smithi155.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-07-18T02:52:30.660 INFO:teuthology.orchestra.run.smithi155.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-07-18T02:52:30.660 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2023-07-18T02:52:30.660 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-07-18T02:52:30.660 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-07-18T02:52:30.661 INFO:teuthology.orchestra.run.smithi155.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-07-18T02:52:30.661 INFO:teuthology.orchestra.run.smithi155.stdout:Installing module profiles: 2023-07-18T02:52:30.661 INFO:teuthology.orchestra.run.smithi155.stdout: container-tools/common 2023-07-18T02:52:30.661 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-07-18T02:52:30.661 INFO:teuthology.orchestra.run.smithi155.stdout:Enabling module streams: 2023-07-18T02:52:30.661 INFO:teuthology.orchestra.run.smithi155.stdout: container-tools rhel8 2023-07-18T02:52:30.661 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-07-18T02:52:30.661 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2023-07-18T02:52:30.662 INFO:teuthology.orchestra.run.smithi155.stdout:============================================================================================================ 2023-07-18T02:52:30.662 INFO:teuthology.orchestra.run.smithi155.stdout:Install 10 Packages 2023-07-18T02:52:30.662 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-07-18T02:52:30.662 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 18 M 2023-07-18T02:52:30.663 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 65 M 2023-07-18T02:52:30.663 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2023-07-18T02:52:30.683 INFO:teuthology.orchestra.run.smithi101.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 1.3 MB/s | 213 kB 00:00 2023-07-18T02:52:30.761 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2023-07-18T02:52:30.765 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2023-07-18T02:52:30.823 INFO:teuthology.orchestra.run.smithi155.stdout:(1/10): yajl-2.1.0-10.el8.x86_64.rpm 252 kB/s | 41 kB 00:00 2023-07-18T02:52:30.833 INFO:teuthology.orchestra.run.smithi101.stdout:(8/10): buildah-1.26.2-1.module+el8.6.0+16771+2 19 MB/s | 8.0 MB 00:00 2023-07-18T02:52:30.848 INFO:teuthology.orchestra.run.smithi155.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 132 kB/s | 25 kB 00:00 2023-07-18T02:52:30.874 INFO:teuthology.orchestra.run.smithi155.stdout:(3/10): python3-pyxdg-0.25-16.el8.noarch.rpm 441 kB/s | 94 kB 00:00 2023-07-18T02:52:30.917 INFO:teuthology.orchestra.run.smithi101.stdout:(9/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 9.4 MB/s | 2.2 MB 00:00 2023-07-18T02:52:30.966 INFO:teuthology.orchestra.run.smithi155.stdout:(4/10): cockpit-podman-49.1-1.module+el8.6.0+16 4.4 MB/s | 524 kB 00:00 2023-07-18T02:52:30.999 INFO:teuthology.orchestra.run.smithi155.stdout:(5/10): udica-0.2.6-4.module+el8.6.0+16771+28df 281 kB/s | 49 kB 00:00 2023-07-18T02:52:31.008 INFO:teuthology.orchestra.run.smithi101.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 15 MB/s | 7.0 MB 00:00 2023-07-18T02:52:31.009 INFO:teuthology.orchestra.run.smithi101.stdout:-------------------------------------------------------------------------------- 2023-07-18T02:52:31.009 INFO:teuthology.orchestra.run.smithi101.stdout:Total 22 MB/s | 18 MB 00:00 2023-07-18T02:52:31.100 INFO:teuthology.orchestra.run.smithi155.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 1.1 MB/s | 149 kB 00:00 2023-07-18T02:52:31.141 INFO:teuthology.orchestra.run.smithi155.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 1.5 MB/s | 213 kB 00:00 2023-07-18T02:52:31.259 INFO:teuthology.orchestra.run.smithi155.stdout:(8/10): buildah-1.26.2-1.module+el8.6.0+16771+2 21 MB/s | 8.0 MB 00:00 2023-07-18T02:52:31.289 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2023-07-18T02:52:31.340 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction check 2023-07-18T02:52:31.342 INFO:teuthology.orchestra.run.smithi155.stdout:(9/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 11 MB/s | 2.2 MB 00:00 2023-07-18T02:52:31.363 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction check succeeded. 2023-07-18T02:52:31.363 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction test 2023-07-18T02:52:31.507 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-07-18T02:52:31.525 INFO:teuthology.orchestra.run.smithi155.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 16 MB/s | 7.0 MB 00:00 2023-07-18T02:52:31.526 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2023-07-18T02:52:31.526 INFO:teuthology.orchestra.run.smithi155.stdout:Total 21 MB/s | 18 MB 00:00 2023-07-18T02:52:31.666 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-18T02:52:31.727 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2023-07-18T02:52:31.746 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2023-07-18T02:52:31.746 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2023-07-18T02:52:31.852 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-07-18T02:52:31.891 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction test succeeded. 2023-07-18T02:52:31.895 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction 2023-07-18T02:52:31.928 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2023-07-18T02:52:31.933 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2023-07-18T02:52:32.095 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-07-18T02:52:32.461 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-07-18T02:52:32.472 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2023-07-18T02:52:32.484 INFO:teuthology.orchestra.run.smithi101.stdout: Preparing : 1/1 2023-07-18T02:52:32.695 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-07-18T02:52:32.730 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-07-18T02:52:32.879 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-18T02:52:32.952 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-18T02:52:33.144 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-07-18T02:52:33.149 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-07-18T02:52:33.288 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-07-18T02:52:33.407 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-07-18T02:52:33.434 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-07-18T02:52:33.797 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-07-18T02:52:33.819 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-07-18T02:52:34.171 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-07-18T02:52:34.399 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-07-18T02:52:34.600 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-07-18T02:52:34.606 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-07-18T02:52:34.675 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-07-18T02:52:34.808 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-18T02:52:35.111 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-18T02:52:35.111 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-07-18T02:52:35.111 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-18T02:52:35.112 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-07-18T02:52:35.112 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-07-18T02:52:35.112 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-07-18T02:52:35.112 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-07-18T02:52:35.113 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-07-18T02:52:35.113 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-07-18T02:52:35.113 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-07-18T02:52:35.581 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-07-18T02:52:35.632 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-07-18T02:52:35.872 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-07-18T02:52:35.896 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-07-18T02:52:36.109 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-07-18T02:52:36.110 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-07-18T02:52:36.257 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-07-18T02:52:36.257 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:02:30 ago on Tue 18 Jul 2023 02:50:05 AM UTC. 2023-07-18T02:52:36.291 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-18T02:52:36.375 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-18T02:52:36.598 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-18T02:52:36.598 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-07-18T02:52:36.599 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-18T02:52:36.599 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-07-18T02:52:36.599 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-07-18T02:52:36.599 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-07-18T02:52:36.599 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-07-18T02:52:36.599 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-07-18T02:52:36.599 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-07-18T02:52:36.600 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-07-18T02:52:36.759 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-18T02:52:36.759 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-07-18T02:52:36.760 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-18T02:52:36.760 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-07-18T02:52:36.760 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-07-18T02:52:36.760 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-07-18T02:52:36.760 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-07-18T02:52:36.760 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-07-18T02:52:36.760 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-07-18T02:52:36.761 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-07-18T02:52:37.860 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-07-18T02:52:37.861 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:09:03 ago on Tue 18 Jul 2023 02:43:34 AM UTC. 2023-07-18T02:52:38.227 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-07-18T02:52:38.228 INFO:teuthology.orchestra.run.smithi101.stdout:Last metadata expiration check: 0:06:43 ago on Tue 18 Jul 2023 02:45:54 AM UTC. 2023-07-18T02:52:42.963 INFO:teuthology.orchestra.run.smithi179.stdout:Installed products updated. 2023-07-18T02:52:43.164 INFO:teuthology.orchestra.run.smithi179.stdout: 2023-07-18T02:52:43.165 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2023-07-18T02:52:43.165 INFO:teuthology.orchestra.run.smithi179.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-18T02:52:43.165 INFO:teuthology.orchestra.run.smithi179.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-07-18T02:52:43.165 INFO:teuthology.orchestra.run.smithi179.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-18T02:52:43.165 INFO:teuthology.orchestra.run.smithi179.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-07-18T02:52:43.165 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-07-18T02:52:43.166 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-07-18T02:52:43.166 INFO:teuthology.orchestra.run.smithi179.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-18T02:52:43.166 INFO:teuthology.orchestra.run.smithi179.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-07-18T02:52:43.166 INFO:teuthology.orchestra.run.smithi179.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-07-18T02:52:43.166 INFO:teuthology.orchestra.run.smithi179.stdout: yajl-2.1.0-10.el8.x86_64 2023-07-18T02:52:43.166 INFO:teuthology.orchestra.run.smithi179.stdout: 2023-07-18T02:52:43.166 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2023-07-18T02:52:43.317 DEBUG:teuthology.parallel:result is None 2023-07-18T02:52:46.242 INFO:teuthology.orchestra.run.smithi101.stdout:Installed products updated. 2023-07-18T02:52:46.467 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T02:52:46.468 INFO:teuthology.orchestra.run.smithi101.stdout:Installed: 2023-07-18T02:52:46.468 INFO:teuthology.orchestra.run.smithi101.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-18T02:52:46.468 INFO:teuthology.orchestra.run.smithi101.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-07-18T02:52:46.469 INFO:teuthology.orchestra.run.smithi101.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-18T02:52:46.469 INFO:teuthology.orchestra.run.smithi101.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-07-18T02:52:46.469 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-07-18T02:52:46.469 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-07-18T02:52:46.470 INFO:teuthology.orchestra.run.smithi101.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-18T02:52:46.470 INFO:teuthology.orchestra.run.smithi101.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-07-18T02:52:46.470 INFO:teuthology.orchestra.run.smithi101.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-07-18T02:52:46.470 INFO:teuthology.orchestra.run.smithi101.stdout: yajl-2.1.0-10.el8.x86_64 2023-07-18T02:52:46.471 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T02:52:46.471 INFO:teuthology.orchestra.run.smithi101.stdout:Complete! 2023-07-18T02:52:46.717 DEBUG:teuthology.parallel:result is None 2023-07-18T02:52:47.282 INFO:teuthology.orchestra.run.smithi155.stdout:Installed products updated. 2023-07-18T02:52:47.520 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-07-18T02:52:47.521 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2023-07-18T02:52:47.521 INFO:teuthology.orchestra.run.smithi155.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-18T02:52:47.521 INFO:teuthology.orchestra.run.smithi155.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-07-18T02:52:47.521 INFO:teuthology.orchestra.run.smithi155.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-18T02:52:47.521 INFO:teuthology.orchestra.run.smithi155.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-07-18T02:52:47.521 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-07-18T02:52:47.522 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-07-18T02:52:47.522 INFO:teuthology.orchestra.run.smithi155.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-18T02:52:47.522 INFO:teuthology.orchestra.run.smithi155.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-07-18T02:52:47.522 INFO:teuthology.orchestra.run.smithi155.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-07-18T02:52:47.522 INFO:teuthology.orchestra.run.smithi155.stdout: yajl-2.1.0-10.el8.x86_64 2023-07-18T02:52:47.522 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-07-18T02:52:47.522 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2023-07-18T02:52:48.057 DEBUG:teuthology.parallel:result is None 2023-07-18T02:52:48.057 INFO:teuthology.run_tasks:Running task install... 2023-07-18T02:52:48.067 DEBUG:teuthology.task.install:project ceph 2023-07-18T02:52:48.068 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '81cf785aa7781c4e08381c540f890cb9bc987ef9'}} 2023-07-18T02:52:48.068 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '81cf785aa7781c4e08381c540f890cb9bc987ef9'} 2023-07-18T02:52:48.068 INFO:teuthology.task.install:Using flavor: default 2023-07-18T02:52:48.074 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-07-18T02:52:48.075 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-07-18T02:52:48.075 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '81cf785aa7781c4e08381c540f890cb9bc987ef9', 'tag': None, 'wait_for_package': False} 2023-07-18T02:52:48.076 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-18T02:52:48.078 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '81cf785aa7781c4e08381c540f890cb9bc987ef9', 'tag': None, 'wait_for_package': False} 2023-07-18T02:52:48.079 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-18T02:52:48.081 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '81cf785aa7781c4e08381c540f890cb9bc987ef9', 'tag': None, 'wait_for_package': False} 2023-07-18T02:52:48.081 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-18T02:53:03.378 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/ 2023-07-18T02:53:03.378 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4746.g81cf785a 2023-07-18T02:53:08.391 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/ 2023-07-18T02:53:08.392 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4746.g81cf785a 2023-07-18T02:53:08.393 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/ 2023-07-18T02:53:08.394 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4746.g81cf785a 2023-07-18T02:53:13.598 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-18T02:53:13.599 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-07-18T02:53:13.599 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-18T02:53:13.637 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-18T02:53:13.637 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2023-07-18T02:53:13.637 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-18T02:53:13.659 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-18T02:53:13.660 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-07-18T02:53:13.660 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-18T02:53:13.670 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-18T02:53:13.670 DEBUG:teuthology.orchestra.run.smithi101:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/81cf785aa7781c4e08381c540f890cb9bc987ef9/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-18T02:53:13.673 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-18T02:53:13.673 DEBUG:teuthology.orchestra.run.smithi179:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/81cf785aa7781c4e08381c540f890cb9bc987ef9/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-18T02:53:13.695 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-18T02:53:13.695 DEBUG:teuthology.orchestra.run.smithi155:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/81cf785aa7781c4e08381c540f890cb9bc987ef9/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-18T02:53:13.745 DEBUG:teuthology.orchestra.run.smithi179:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-18T02:53:13.766 DEBUG:teuthology.orchestra.run.smithi155:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-18T02:53:13.780 DEBUG:teuthology.orchestra.run.smithi101:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-18T02:53:13.826 DEBUG:teuthology.orchestra.run.smithi179:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-18T02:53:13.847 DEBUG:teuthology.orchestra.run.smithi155:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-18T02:53:13.876 DEBUG:teuthology.orchestra.run.smithi101:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-18T02:53:13.894 INFO:teuthology.orchestra.run.smithi179.stdout:check_obsoletes = 1 2023-07-18T02:53:13.896 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum clean all 2023-07-18T02:53:13.915 INFO:teuthology.orchestra.run.smithi155.stdout:check_obsoletes = 1 2023-07-18T02:53:13.917 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum clean all 2023-07-18T02:53:13.944 INFO:teuthology.orchestra.run.smithi101.stdout:check_obsoletes = 1 2023-07-18T02:53:13.948 DEBUG:teuthology.orchestra.run.smithi101:> sudo yum clean all 2023-07-18T02:53:14.271 INFO:teuthology.orchestra.run.smithi179.stdout:Updating Subscription Management repositories. 2023-07-18T02:53:14.293 INFO:teuthology.orchestra.run.smithi155.stdout:Updating Subscription Management repositories. 2023-07-18T02:53:14.408 INFO:teuthology.orchestra.run.smithi101.stdout:Updating Subscription Management repositories. 2023-07-18T02:53:14.513 INFO:teuthology.orchestra.run.smithi179.stdout:44 files removed 2023-07-18T02:53:14.529 INFO:teuthology.orchestra.run.smithi155.stdout:44 files removed 2023-07-18T02:53:14.559 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-18T02:53:14.576 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-18T02:53:14.709 INFO:teuthology.orchestra.run.smithi101.stdout:44 files removed 2023-07-18T02:53:14.772 DEBUG:teuthology.orchestra.run.smithi101:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-18T02:53:14.894 INFO:teuthology.orchestra.run.smithi179.stdout:Updating Subscription Management repositories. 2023-07-18T02:53:14.910 INFO:teuthology.orchestra.run.smithi155.stdout:Updating Subscription Management repositories. 2023-07-18T02:53:15.232 INFO:teuthology.orchestra.run.smithi101.stdout:Updating Subscription Management repositories. 2023-07-18T02:53:15.856 INFO:teuthology.orchestra.run.smithi179.stdout:ceph packages for x86_64 133 kB/s | 81 kB 00:00 2023-07-18T02:53:16.266 INFO:teuthology.orchestra.run.smithi101.stdout:ceph packages for x86_64 131 kB/s | 81 kB 00:00 2023-07-18T02:53:16.349 INFO:teuthology.orchestra.run.smithi179.stdout:ceph noarch packages 20 kB/s | 9.4 kB 00:00 2023-07-18T02:53:16.766 INFO:teuthology.orchestra.run.smithi101.stdout:ceph noarch packages 20 kB/s | 9.4 kB 00:00 2023-07-18T02:53:16.800 INFO:teuthology.orchestra.run.smithi179.stdout:ceph source packages 3.5 kB/s | 1.5 kB 00:00 2023-07-18T02:53:16.992 INFO:teuthology.orchestra.run.smithi179.stdout:Copr repo for python3-asyncssh owned by ceph 20 kB/s | 3.5 kB 00:00 2023-07-18T02:53:17.226 INFO:teuthology.orchestra.run.smithi101.stdout:ceph source packages 3.4 kB/s | 1.5 kB 00:00 2023-07-18T02:53:17.401 INFO:teuthology.orchestra.run.smithi101.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2023-07-18T02:53:18.836 INFO:teuthology.orchestra.run.smithi155.stdout:ceph packages for x86_64 23 kB/s | 81 kB 00:03 2023-07-18T02:53:19.425 INFO:teuthology.orchestra.run.smithi155.stdout:ceph noarch packages 17 kB/s | 9.4 kB 00:00 2023-07-18T02:53:19.971 INFO:teuthology.orchestra.run.smithi155.stdout:ceph source packages 2.8 kB/s | 1.5 kB 00:00 2023-07-18T02:53:20.162 INFO:teuthology.orchestra.run.smithi179.stdout:Extra Packages for Enterprise Linux 5.0 MB/s | 16 MB 00:03 2023-07-18T02:53:20.171 INFO:teuthology.orchestra.run.smithi155.stdout:Copr repo for python3-asyncssh owned by ceph 19 kB/s | 3.5 kB 00:00 2023-07-18T02:53:20.544 INFO:teuthology.orchestra.run.smithi101.stdout:Extra Packages for Enterprise Linux 5.0 MB/s | 16 MB 00:03 2023-07-18T02:53:22.364 INFO:teuthology.orchestra.run.smithi155.stdout:Extra Packages for Enterprise Linux 7.2 MB/s | 16 MB 00:02 2023-07-18T02:53:23.626 INFO:teuthology.orchestra.run.smithi179.stdout:lab-extras 413 kB/s | 24 kB 00:00 2023-07-18T02:53:24.496 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 59 MB/s | 47 MB 00:00 2023-07-18T02:53:24.519 INFO:teuthology.orchestra.run.smithi101.stdout:lab-extras 359 kB/s | 24 kB 00:00 2023-07-18T02:53:25.672 INFO:teuthology.orchestra.run.smithi101.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 44 MB/s | 47 MB 00:01 2023-07-18T02:53:25.839 INFO:teuthology.orchestra.run.smithi155.stdout:lab-extras 345 kB/s | 24 kB 00:00 2023-07-18T02:53:26.807 INFO:teuthology.orchestra.run.smithi155.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 53 MB/s | 47 MB 00:00 2023-07-18T02:53:32.397 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 64 MB/s | 53 MB 00:00 2023-07-18T02:53:34.297 INFO:teuthology.orchestra.run.smithi101.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 49 MB/s | 53 MB 00:01 2023-07-18T02:53:34.641 INFO:teuthology.orchestra.run.smithi155.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 60 MB/s | 53 MB 00:00 2023-07-18T02:53:43.494 INFO:teuthology.orchestra.run.smithi179.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-18T02:53:43.495 INFO:teuthology.orchestra.run.smithi179.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-18T02:53:43.510 INFO:teuthology.orchestra.run.smithi179.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-18T02:53:43.510 INFO:teuthology.orchestra.run.smithi179.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-18T02:53:43.511 INFO:teuthology.orchestra.run.smithi179.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-18T02:53:43.512 INFO:teuthology.orchestra.run.smithi179.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-18T02:53:43.514 INFO:teuthology.orchestra.run.smithi179.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-18T02:53:43.517 INFO:teuthology.orchestra.run.smithi179.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-18T02:53:43.518 INFO:teuthology.orchestra.run.smithi179.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-18T02:53:43.650 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2023-07-18T02:53:43.662 INFO:teuthology.orchestra.run.smithi179.stdout:=========================================================================================================================== 2023-07-18T02:53:43.663 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2023-07-18T02:53:43.663 INFO:teuthology.orchestra.run.smithi179.stdout:=========================================================================================================================== 2023-07-18T02:53:43.663 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2023-07-18T02:53:43.663 INFO:teuthology.orchestra.run.smithi179.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-18T02:53:43.663 INFO:teuthology.orchestra.run.smithi179.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-18T02:53:43.663 INFO:teuthology.orchestra.run.smithi179.stdout: ceph x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 6.4 k 2023-07-18T02:53:43.664 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-base x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 5.4 M 2023-07-18T02:53:43.664 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-fuse x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 848 k 2023-07-18T02:53:43.664 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 165 k 2023-07-18T02:53:43.664 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 1.5 M 2023-07-18T02:53:43.664 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 129 k 2023-07-18T02:53:43.664 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 1.7 M 2023-07-18T02:53:43.664 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 7.4 M 2023-07-18T02:53:43.665 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-rook noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 47 k 2023-07-18T02:53:43.665 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-radosgw x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 13 M 2023-07-18T02:53:43.665 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-test x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 48 M 2023-07-18T02:53:43.665 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-volume noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 258 k 2023-07-18T02:53:43.665 INFO:teuthology.orchestra.run.smithi179.stdout: cephadm noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 166 k 2023-07-18T02:53:43.665 INFO:teuthology.orchestra.run.smithi179.stdout: cephfs-mirror x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 258 k 2023-07-18T02:53:43.665 INFO:teuthology.orchestra.run.smithi179.stdout: cephfs-top noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 18 k 2023-07-18T02:53:43.665 INFO:teuthology.orchestra.run.smithi179.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-18T02:53:43.666 INFO:teuthology.orchestra.run.smithi179.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-18T02:53:43.666 INFO:teuthology.orchestra.run.smithi179.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-18T02:53:43.666 INFO:teuthology.orchestra.run.smithi179.stdout: libcephfs-devel x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 31 k 2023-07-18T02:53:43.666 INFO:teuthology.orchestra.run.smithi179.stdout: libcephfs2 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 772 k 2023-07-18T02:53:43.666 INFO:teuthology.orchestra.run.smithi179.stdout: librados-devel x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 139 k 2023-07-18T02:53:43.666 INFO:teuthology.orchestra.run.smithi179.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-18T02:53:43.666 INFO:teuthology.orchestra.run.smithi179.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-18T02:53:43.667 INFO:teuthology.orchestra.run.smithi179.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-18T02:53:43.667 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-18T02:53:43.667 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-18T02:53:43.667 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-18T02:53:43.667 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cephfs x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 197 k 2023-07-18T02:53:43.667 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rados x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 370 k 2023-07-18T02:53:43.667 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rbd x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 364 k 2023-07-18T02:53:43.667 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rgw x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 107 k 2023-07-18T02:53:43.668 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-fuse x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 94 k 2023-07-18T02:53:43.668 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-mirror x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 3.4 M 2023-07-18T02:53:43.668 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-nbd x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 179 k 2023-07-18T02:53:43.668 INFO:teuthology.orchestra.run.smithi179.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-18T02:53:43.668 INFO:teuthology.orchestra.run.smithi179.stdout:Upgrading: 2023-07-18T02:53:43.668 INFO:teuthology.orchestra.run.smithi179.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-18T02:53:43.668 INFO:teuthology.orchestra.run.smithi179.stdout: librados2 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 3.7 M 2023-07-18T02:53:43.669 INFO:teuthology.orchestra.run.smithi179.stdout: librbd1 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 3.6 M 2023-07-18T02:53:43.669 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2023-07-18T02:53:43.669 INFO:teuthology.orchestra.run.smithi179.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-18T02:53:43.669 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-common x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 22 M 2023-07-18T02:53:43.669 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 23 k 2023-07-18T02:53:43.669 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mds x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 2.3 M 2023-07-18T02:53:43.669 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 240 k 2023-07-18T02:53:43.670 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mon x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 4.6 M 2023-07-18T02:53:43.670 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-osd x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 18 M 2023-07-18T02:53:43.670 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 15 k 2023-07-18T02:53:43.670 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-selinux x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 24 k 2023-07-18T02:53:43.670 INFO:teuthology.orchestra.run.smithi179.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-18T02:53:43.670 INFO:teuthology.orchestra.run.smithi179.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-18T02:53:43.670 INFO:teuthology.orchestra.run.smithi179.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-18T02:53:43.671 INFO:teuthology.orchestra.run.smithi179.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-18T02:53:43.671 INFO:teuthology.orchestra.run.smithi179.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-18T02:53:43.671 INFO:teuthology.orchestra.run.smithi179.stdout: libcephsqlite x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 137 k 2023-07-18T02:53:43.671 INFO:teuthology.orchestra.run.smithi179.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-18T02:53:43.671 INFO:teuthology.orchestra.run.smithi179.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-18T02:53:43.671 INFO:teuthology.orchestra.run.smithi179.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-18T02:53:43.671 INFO:teuthology.orchestra.run.smithi179.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-18T02:53:43.671 INFO:teuthology.orchestra.run.smithi179.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-18T02:53:43.672 INFO:teuthology.orchestra.run.smithi179.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-18T02:53:43.672 INFO:teuthology.orchestra.run.smithi179.stdout: libradosstriper1 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 473 k 2023-07-18T02:53:43.672 INFO:teuthology.orchestra.run.smithi179.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-18T02:53:43.672 INFO:teuthology.orchestra.run.smithi179.stdout: librgw2 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 6.7 M 2023-07-18T02:53:43.672 INFO:teuthology.orchestra.run.smithi179.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-18T02:53:43.672 INFO:teuthology.orchestra.run.smithi179.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-18T02:53:43.672 INFO:teuthology.orchestra.run.smithi179.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-18T02:53:43.673 INFO:teuthology.orchestra.run.smithi179.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-18T02:53:43.673 INFO:teuthology.orchestra.run.smithi179.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-18T02:53:43.673 INFO:teuthology.orchestra.run.smithi179.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-18T02:53:43.673 INFO:teuthology.orchestra.run.smithi179.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-18T02:53:43.673 INFO:teuthology.orchestra.run.smithi179.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-18T02:53:43.673 INFO:teuthology.orchestra.run.smithi179.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-18T02:53:43.673 INFO:teuthology.orchestra.run.smithi179.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-18T02:53:43.674 INFO:teuthology.orchestra.run.smithi179.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-18T02:53:43.674 INFO:teuthology.orchestra.run.smithi179.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-18T02:53:43.674 INFO:teuthology.orchestra.run.smithi179.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-18T02:53:43.674 INFO:teuthology.orchestra.run.smithi179.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-18T02:53:43.674 INFO:teuthology.orchestra.run.smithi179.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-18T02:53:43.674 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-18T02:53:43.674 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-argparse x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 45 k 2023-07-18T02:53:43.674 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-common x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 119 k 2023-07-18T02:53:43.675 INFO:teuthology.orchestra.run.smithi179.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-18T02:53:43.675 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-18T02:53:43.675 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-18T02:53:43.675 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-18T02:53:43.675 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-18T02:53:43.675 INFO:teuthology.orchestra.run.smithi179.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-18T02:53:43.675 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-18T02:53:43.675 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-18T02:53:43.676 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-18T02:53:43.676 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-18T02:53:43.676 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-18T02:53:43.676 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-18T02:53:43.676 INFO:teuthology.orchestra.run.smithi179.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-18T02:53:43.676 INFO:teuthology.orchestra.run.smithi179.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-18T02:53:43.676 INFO:teuthology.orchestra.run.smithi179.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-18T02:53:43.677 INFO:teuthology.orchestra.run.smithi179.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-18T02:53:43.677 INFO:teuthology.orchestra.run.smithi179.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-18T02:53:43.677 INFO:teuthology.orchestra.run.smithi179.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-18T02:53:43.677 INFO:teuthology.orchestra.run.smithi179.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-18T02:53:43.677 INFO:teuthology.orchestra.run.smithi179.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-18T02:53:43.677 INFO:teuthology.orchestra.run.smithi179.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-18T02:53:43.677 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-18T02:53:43.678 INFO:teuthology.orchestra.run.smithi179.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-18T02:53:43.678 INFO:teuthology.orchestra.run.smithi179.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-18T02:53:43.678 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-18T02:53:43.678 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-18T02:53:43.678 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-18T02:53:43.678 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-18T02:53:43.678 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-18T02:53:43.678 INFO:teuthology.orchestra.run.smithi179.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-18T02:53:43.679 INFO:teuthology.orchestra.run.smithi179.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-18T02:53:43.679 INFO:teuthology.orchestra.run.smithi179.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-18T02:53:43.679 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-18T02:53:43.679 INFO:teuthology.orchestra.run.smithi179.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-18T02:53:43.679 INFO:teuthology.orchestra.run.smithi179.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-18T02:53:43.679 INFO:teuthology.orchestra.run.smithi179.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-18T02:53:43.679 INFO:teuthology.orchestra.run.smithi179.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-18T02:53:43.679 INFO:teuthology.orchestra.run.smithi179.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-18T02:53:43.680 INFO:teuthology.orchestra.run.smithi179.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-18T02:53:43.680 INFO:teuthology.orchestra.run.smithi179.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-18T02:53:43.680 INFO:teuthology.orchestra.run.smithi179.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-18T02:53:43.680 INFO:teuthology.orchestra.run.smithi179.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-18T02:53:43.680 INFO:teuthology.orchestra.run.smithi179.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-18T02:53:43.680 INFO:teuthology.orchestra.run.smithi179.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-18T02:53:43.680 INFO:teuthology.orchestra.run.smithi179.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-18T02:53:43.681 INFO:teuthology.orchestra.run.smithi179.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-18T02:53:43.681 INFO:teuthology.orchestra.run.smithi179.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-18T02:53:43.681 INFO:teuthology.orchestra.run.smithi179.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-18T02:53:43.681 INFO:teuthology.orchestra.run.smithi179.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-18T02:53:43.681 INFO:teuthology.orchestra.run.smithi179.stdout:Enabling module streams: 2023-07-18T02:53:43.681 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql 10 2023-07-18T02:53:43.681 INFO:teuthology.orchestra.run.smithi179.stdout: 2023-07-18T02:53:43.682 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2023-07-18T02:53:43.682 INFO:teuthology.orchestra.run.smithi179.stdout:=========================================================================================================================== 2023-07-18T02:53:43.682 INFO:teuthology.orchestra.run.smithi179.stdout:Install 128 Packages 2023-07-18T02:53:43.682 INFO:teuthology.orchestra.run.smithi179.stdout:Upgrade 3 Packages 2023-07-18T02:53:43.682 INFO:teuthology.orchestra.run.smithi179.stdout: 2023-07-18T02:53:43.683 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 205 M 2023-07-18T02:53:43.683 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2023-07-18T02:53:43.989 INFO:teuthology.orchestra.run.smithi179.stdout:(1/131): ceph-18.0.0-4746.g81cf785a.el8.x86_64. 23 kB/s | 6.4 kB 00:00 2023-07-18T02:53:44.731 INFO:teuthology.orchestra.run.smithi179.stdout:(2/131): ceph-fuse-18.0.0-4746.g81cf785a.el8.x8 1.1 MB/s | 848 kB 00:00 2023-07-18T02:53:44.956 INFO:teuthology.orchestra.run.smithi179.stdout:(3/131): ceph-immutable-object-cache-18.0.0-474 736 kB/s | 165 kB 00:00 2023-07-18T02:53:45.780 INFO:teuthology.orchestra.run.smithi155.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-18T02:53:45.780 INFO:teuthology.orchestra.run.smithi155.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-18T02:53:45.792 INFO:teuthology.orchestra.run.smithi155.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-18T02:53:45.793 INFO:teuthology.orchestra.run.smithi155.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-18T02:53:45.794 INFO:teuthology.orchestra.run.smithi155.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-18T02:53:45.795 INFO:teuthology.orchestra.run.smithi155.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-18T02:53:45.797 INFO:teuthology.orchestra.run.smithi155.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-18T02:53:45.800 INFO:teuthology.orchestra.run.smithi155.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-18T02:53:45.801 INFO:teuthology.orchestra.run.smithi155.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-18T02:53:45.823 INFO:teuthology.orchestra.run.smithi101.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-18T02:53:45.824 INFO:teuthology.orchestra.run.smithi101.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-18T02:53:45.839 INFO:teuthology.orchestra.run.smithi101.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-18T02:53:45.840 INFO:teuthology.orchestra.run.smithi101.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-18T02:53:45.842 INFO:teuthology.orchestra.run.smithi101.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-18T02:53:45.843 INFO:teuthology.orchestra.run.smithi101.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-18T02:53:45.845 INFO:teuthology.orchestra.run.smithi101.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-18T02:53:45.849 INFO:teuthology.orchestra.run.smithi101.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-18T02:53:45.850 INFO:teuthology.orchestra.run.smithi101.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-18T02:53:45.933 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2023-07-18T02:53:45.945 INFO:teuthology.orchestra.run.smithi155.stdout:=========================================================================================================================== 2023-07-18T02:53:45.945 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2023-07-18T02:53:45.946 INFO:teuthology.orchestra.run.smithi155.stdout:=========================================================================================================================== 2023-07-18T02:53:45.946 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2023-07-18T02:53:45.946 INFO:teuthology.orchestra.run.smithi155.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-18T02:53:45.946 INFO:teuthology.orchestra.run.smithi155.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-18T02:53:45.946 INFO:teuthology.orchestra.run.smithi155.stdout: ceph x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 6.4 k 2023-07-18T02:53:45.946 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-base x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 5.4 M 2023-07-18T02:53:45.946 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-fuse x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 848 k 2023-07-18T02:53:45.947 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 165 k 2023-07-18T02:53:45.947 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 1.5 M 2023-07-18T02:53:45.947 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 129 k 2023-07-18T02:53:45.947 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 1.7 M 2023-07-18T02:53:45.947 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 7.4 M 2023-07-18T02:53:45.948 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-rook noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 47 k 2023-07-18T02:53:45.948 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-radosgw x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 13 M 2023-07-18T02:53:45.948 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-test x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 48 M 2023-07-18T02:53:45.948 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-volume noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 258 k 2023-07-18T02:53:45.949 INFO:teuthology.orchestra.run.smithi155.stdout: cephadm noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 166 k 2023-07-18T02:53:45.949 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-mirror x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 258 k 2023-07-18T02:53:45.949 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-top noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 18 k 2023-07-18T02:53:45.949 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-18T02:53:45.949 INFO:teuthology.orchestra.run.smithi155.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-18T02:53:45.949 INFO:teuthology.orchestra.run.smithi155.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-18T02:53:45.949 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs-devel x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 31 k 2023-07-18T02:53:45.950 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs2 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 772 k 2023-07-18T02:53:45.950 INFO:teuthology.orchestra.run.smithi155.stdout: librados-devel x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 139 k 2023-07-18T02:53:45.950 INFO:teuthology.orchestra.run.smithi155.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-18T02:53:45.950 INFO:teuthology.orchestra.run.smithi155.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-18T02:53:45.950 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-18T02:53:45.950 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-18T02:53:45.951 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-18T02:53:45.951 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-18T02:53:45.951 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cephfs x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 197 k 2023-07-18T02:53:45.951 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rados x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 370 k 2023-07-18T02:53:45.951 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rbd x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 364 k 2023-07-18T02:53:45.952 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rgw x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 107 k 2023-07-18T02:53:45.952 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-fuse x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 94 k 2023-07-18T02:53:45.952 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-mirror x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 3.4 M 2023-07-18T02:53:45.952 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-nbd x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 179 k 2023-07-18T02:53:45.953 INFO:teuthology.orchestra.run.smithi155.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-18T02:53:45.953 INFO:teuthology.orchestra.run.smithi155.stdout:Upgrading: 2023-07-18T02:53:45.953 INFO:teuthology.orchestra.run.smithi155.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-18T02:53:45.953 INFO:teuthology.orchestra.run.smithi155.stdout: librados2 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 3.7 M 2023-07-18T02:53:45.953 INFO:teuthology.orchestra.run.smithi155.stdout: librbd1 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 3.6 M 2023-07-18T02:53:45.953 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2023-07-18T02:53:45.954 INFO:teuthology.orchestra.run.smithi155.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-18T02:53:45.954 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-common x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 22 M 2023-07-18T02:53:45.954 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 23 k 2023-07-18T02:53:45.954 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mds x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 2.3 M 2023-07-18T02:53:45.954 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 240 k 2023-07-18T02:53:45.955 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mon x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 4.6 M 2023-07-18T02:53:45.955 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-osd x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 18 M 2023-07-18T02:53:45.955 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 15 k 2023-07-18T02:53:45.955 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-selinux x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 24 k 2023-07-18T02:53:45.955 INFO:teuthology.orchestra.run.smithi155.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-18T02:53:45.955 INFO:teuthology.orchestra.run.smithi155.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-18T02:53:45.955 INFO:teuthology.orchestra.run.smithi155.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-18T02:53:45.956 INFO:teuthology.orchestra.run.smithi155.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-18T02:53:45.956 INFO:teuthology.orchestra.run.smithi155.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-18T02:53:45.956 INFO:teuthology.orchestra.run.smithi155.stdout: libcephsqlite x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 137 k 2023-07-18T02:53:45.956 INFO:teuthology.orchestra.run.smithi155.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-18T02:53:45.956 INFO:teuthology.orchestra.run.smithi155.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-18T02:53:45.957 INFO:teuthology.orchestra.run.smithi155.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-18T02:53:45.957 INFO:teuthology.orchestra.run.smithi155.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-18T02:53:45.957 INFO:teuthology.orchestra.run.smithi155.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-18T02:53:45.957 INFO:teuthology.orchestra.run.smithi155.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-18T02:53:45.957 INFO:teuthology.orchestra.run.smithi155.stdout: libradosstriper1 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 473 k 2023-07-18T02:53:45.957 INFO:teuthology.orchestra.run.smithi155.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-18T02:53:45.958 INFO:teuthology.orchestra.run.smithi155.stdout: librgw2 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 6.7 M 2023-07-18T02:53:45.958 INFO:teuthology.orchestra.run.smithi155.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-18T02:53:45.958 INFO:teuthology.orchestra.run.smithi155.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-18T02:53:45.958 INFO:teuthology.orchestra.run.smithi155.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-18T02:53:45.958 INFO:teuthology.orchestra.run.smithi155.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-18T02:53:45.959 INFO:teuthology.orchestra.run.smithi155.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-18T02:53:45.959 INFO:teuthology.orchestra.run.smithi155.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-18T02:53:45.959 INFO:teuthology.orchestra.run.smithi155.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-18T02:53:45.959 INFO:teuthology.orchestra.run.smithi155.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-18T02:53:45.959 INFO:teuthology.orchestra.run.smithi155.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-18T02:53:45.960 INFO:teuthology.orchestra.run.smithi155.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-18T02:53:45.960 INFO:teuthology.orchestra.run.smithi155.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-18T02:53:45.960 INFO:teuthology.orchestra.run.smithi155.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-18T02:53:45.960 INFO:teuthology.orchestra.run.smithi155.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-18T02:53:45.960 INFO:teuthology.orchestra.run.smithi155.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-18T02:53:45.960 INFO:teuthology.orchestra.run.smithi155.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-18T02:53:45.960 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-18T02:53:45.961 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-argparse x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 45 k 2023-07-18T02:53:45.961 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-common x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 119 k 2023-07-18T02:53:45.961 INFO:teuthology.orchestra.run.smithi155.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-18T02:53:45.961 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-18T02:53:45.961 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-18T02:53:45.961 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-18T02:53:45.962 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-18T02:53:45.962 INFO:teuthology.orchestra.run.smithi155.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-18T02:53:45.962 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-18T02:53:45.962 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-18T02:53:45.962 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-18T02:53:45.962 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-18T02:53:45.962 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-18T02:53:45.963 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-18T02:53:45.963 INFO:teuthology.orchestra.run.smithi155.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-18T02:53:45.963 INFO:teuthology.orchestra.run.smithi155.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-18T02:53:45.963 INFO:teuthology.orchestra.run.smithi155.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-18T02:53:45.963 INFO:teuthology.orchestra.run.smithi155.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-18T02:53:45.963 INFO:teuthology.orchestra.run.smithi155.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-18T02:53:45.963 INFO:teuthology.orchestra.run.smithi155.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-18T02:53:45.964 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-18T02:53:45.964 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-18T02:53:45.964 INFO:teuthology.orchestra.run.smithi155.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-18T02:53:45.964 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-18T02:53:45.964 INFO:teuthology.orchestra.run.smithi155.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-18T02:53:45.964 INFO:teuthology.orchestra.run.smithi155.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-18T02:53:45.964 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-18T02:53:45.965 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-18T02:53:45.965 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-18T02:53:45.965 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-18T02:53:45.965 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-18T02:53:45.965 INFO:teuthology.orchestra.run.smithi155.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-18T02:53:45.965 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-18T02:53:45.966 INFO:teuthology.orchestra.run.smithi155.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-18T02:53:45.966 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-18T02:53:45.966 INFO:teuthology.orchestra.run.smithi155.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-18T02:53:45.966 INFO:teuthology.orchestra.run.smithi155.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-18T02:53:45.966 INFO:teuthology.orchestra.run.smithi155.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-18T02:53:45.966 INFO:teuthology.orchestra.run.smithi155.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-18T02:53:45.967 INFO:teuthology.orchestra.run.smithi155.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-18T02:53:45.967 INFO:teuthology.orchestra.run.smithi155.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-18T02:53:45.967 INFO:teuthology.orchestra.run.smithi155.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-18T02:53:45.967 INFO:teuthology.orchestra.run.smithi155.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-18T02:53:45.967 INFO:teuthology.orchestra.run.smithi155.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-18T02:53:45.967 INFO:teuthology.orchestra.run.smithi155.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-18T02:53:45.968 INFO:teuthology.orchestra.run.smithi155.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-18T02:53:45.968 INFO:teuthology.orchestra.run.smithi155.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-18T02:53:45.968 INFO:teuthology.orchestra.run.smithi155.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-18T02:53:45.968 INFO:teuthology.orchestra.run.smithi155.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-18T02:53:45.968 INFO:teuthology.orchestra.run.smithi155.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-18T02:53:45.968 INFO:teuthology.orchestra.run.smithi155.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-18T02:53:45.969 INFO:teuthology.orchestra.run.smithi155.stdout:Enabling module streams: 2023-07-18T02:53:45.969 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql 10 2023-07-18T02:53:45.969 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-07-18T02:53:45.969 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2023-07-18T02:53:45.969 INFO:teuthology.orchestra.run.smithi155.stdout:=========================================================================================================================== 2023-07-18T02:53:45.969 INFO:teuthology.orchestra.run.smithi155.stdout:Install 128 Packages 2023-07-18T02:53:45.969 INFO:teuthology.orchestra.run.smithi155.stdout:Upgrade 3 Packages 2023-07-18T02:53:45.970 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-07-18T02:53:45.970 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 205 M 2023-07-18T02:53:45.970 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2023-07-18T02:53:45.991 INFO:teuthology.orchestra.run.smithi101.stdout:Dependencies resolved. 2023-07-18T02:53:46.004 INFO:teuthology.orchestra.run.smithi101.stdout:=========================================================================================================================== 2023-07-18T02:53:46.004 INFO:teuthology.orchestra.run.smithi101.stdout: Package Arch Version Repository Size 2023-07-18T02:53:46.005 INFO:teuthology.orchestra.run.smithi101.stdout:=========================================================================================================================== 2023-07-18T02:53:46.005 INFO:teuthology.orchestra.run.smithi101.stdout:Installing: 2023-07-18T02:53:46.005 INFO:teuthology.orchestra.run.smithi101.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-18T02:53:46.005 INFO:teuthology.orchestra.run.smithi101.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-18T02:53:46.005 INFO:teuthology.orchestra.run.smithi101.stdout: ceph x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 6.4 k 2023-07-18T02:53:46.005 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-base x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 5.4 M 2023-07-18T02:53:46.005 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-fuse x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 848 k 2023-07-18T02:53:46.006 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 165 k 2023-07-18T02:53:46.006 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 1.5 M 2023-07-18T02:53:46.006 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 129 k 2023-07-18T02:53:46.006 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 1.7 M 2023-07-18T02:53:46.006 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 7.4 M 2023-07-18T02:53:46.006 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-rook noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 47 k 2023-07-18T02:53:46.006 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-radosgw x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 13 M 2023-07-18T02:53:46.007 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-test x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 48 M 2023-07-18T02:53:46.007 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-volume noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 258 k 2023-07-18T02:53:46.007 INFO:teuthology.orchestra.run.smithi101.stdout: cephadm noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 166 k 2023-07-18T02:53:46.007 INFO:teuthology.orchestra.run.smithi101.stdout: cephfs-mirror x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 258 k 2023-07-18T02:53:46.007 INFO:teuthology.orchestra.run.smithi101.stdout: cephfs-top noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 18 k 2023-07-18T02:53:46.007 INFO:teuthology.orchestra.run.smithi101.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-18T02:53:46.007 INFO:teuthology.orchestra.run.smithi101.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-18T02:53:46.007 INFO:teuthology.orchestra.run.smithi101.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-18T02:53:46.008 INFO:teuthology.orchestra.run.smithi101.stdout: libcephfs-devel x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 31 k 2023-07-18T02:53:46.008 INFO:teuthology.orchestra.run.smithi101.stdout: libcephfs2 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 772 k 2023-07-18T02:53:46.008 INFO:teuthology.orchestra.run.smithi101.stdout: librados-devel x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 139 k 2023-07-18T02:53:46.008 INFO:teuthology.orchestra.run.smithi101.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-18T02:53:46.008 INFO:teuthology.orchestra.run.smithi101.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-18T02:53:46.008 INFO:teuthology.orchestra.run.smithi101.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-18T02:53:46.008 INFO:teuthology.orchestra.run.smithi101.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-18T02:53:46.009 INFO:teuthology.orchestra.run.smithi101.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-18T02:53:46.009 INFO:teuthology.orchestra.run.smithi101.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-18T02:53:46.009 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cephfs x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 197 k 2023-07-18T02:53:46.009 INFO:teuthology.orchestra.run.smithi101.stdout: python3-rados x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 370 k 2023-07-18T02:53:46.009 INFO:teuthology.orchestra.run.smithi101.stdout: python3-rbd x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 364 k 2023-07-18T02:53:46.009 INFO:teuthology.orchestra.run.smithi101.stdout: python3-rgw x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 107 k 2023-07-18T02:53:46.009 INFO:teuthology.orchestra.run.smithi101.stdout: rbd-fuse x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 94 k 2023-07-18T02:53:46.009 INFO:teuthology.orchestra.run.smithi101.stdout: rbd-mirror x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 3.4 M 2023-07-18T02:53:46.010 INFO:teuthology.orchestra.run.smithi101.stdout: rbd-nbd x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 179 k 2023-07-18T02:53:46.010 INFO:teuthology.orchestra.run.smithi101.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-18T02:53:46.010 INFO:teuthology.orchestra.run.smithi101.stdout:Upgrading: 2023-07-18T02:53:46.010 INFO:teuthology.orchestra.run.smithi101.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-18T02:53:46.010 INFO:teuthology.orchestra.run.smithi101.stdout: librados2 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 3.7 M 2023-07-18T02:53:46.010 INFO:teuthology.orchestra.run.smithi101.stdout: librbd1 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 3.6 M 2023-07-18T02:53:46.010 INFO:teuthology.orchestra.run.smithi101.stdout:Installing dependencies: 2023-07-18T02:53:46.011 INFO:teuthology.orchestra.run.smithi101.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-18T02:53:46.011 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-common x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 22 M 2023-07-18T02:53:46.011 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 23 k 2023-07-18T02:53:46.011 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mds x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 2.3 M 2023-07-18T02:53:46.011 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 240 k 2023-07-18T02:53:46.012 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mon x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 4.6 M 2023-07-18T02:53:46.012 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-osd x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 18 M 2023-07-18T02:53:46.012 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 15 k 2023-07-18T02:53:46.012 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-selinux x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 24 k 2023-07-18T02:53:46.012 INFO:teuthology.orchestra.run.smithi101.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-18T02:53:46.012 INFO:teuthology.orchestra.run.smithi101.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-18T02:53:46.013 INFO:teuthology.orchestra.run.smithi101.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-18T02:53:46.013 INFO:teuthology.orchestra.run.smithi101.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-18T02:53:46.013 INFO:teuthology.orchestra.run.smithi101.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-18T02:53:46.013 INFO:teuthology.orchestra.run.smithi101.stdout: libcephsqlite x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 137 k 2023-07-18T02:53:46.013 INFO:teuthology.orchestra.run.smithi101.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-18T02:53:46.013 INFO:teuthology.orchestra.run.smithi101.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-18T02:53:46.013 INFO:teuthology.orchestra.run.smithi101.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-18T02:53:46.015 INFO:teuthology.orchestra.run.smithi101.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-18T02:53:46.015 INFO:teuthology.orchestra.run.smithi101.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-18T02:53:46.015 INFO:teuthology.orchestra.run.smithi101.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-18T02:53:46.016 INFO:teuthology.orchestra.run.smithi101.stdout: libradosstriper1 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 473 k 2023-07-18T02:53:46.016 INFO:teuthology.orchestra.run.smithi101.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-18T02:53:46.016 INFO:teuthology.orchestra.run.smithi101.stdout: librgw2 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 6.7 M 2023-07-18T02:53:46.016 INFO:teuthology.orchestra.run.smithi101.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-18T02:53:46.016 INFO:teuthology.orchestra.run.smithi101.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-18T02:53:46.016 INFO:teuthology.orchestra.run.smithi101.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-18T02:53:46.017 INFO:teuthology.orchestra.run.smithi101.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-18T02:53:46.017 INFO:teuthology.orchestra.run.smithi101.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-18T02:53:46.017 INFO:teuthology.orchestra.run.smithi101.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-18T02:53:46.017 INFO:teuthology.orchestra.run.smithi101.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-18T02:53:46.017 INFO:teuthology.orchestra.run.smithi101.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-18T02:53:46.017 INFO:teuthology.orchestra.run.smithi101.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-18T02:53:46.017 INFO:teuthology.orchestra.run.smithi101.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-18T02:53:46.018 INFO:teuthology.orchestra.run.smithi101.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-18T02:53:46.018 INFO:teuthology.orchestra.run.smithi101.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-18T02:53:46.018 INFO:teuthology.orchestra.run.smithi101.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-18T02:53:46.018 INFO:teuthology.orchestra.run.smithi101.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-18T02:53:46.018 INFO:teuthology.orchestra.run.smithi101.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-18T02:53:46.018 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-18T02:53:46.018 INFO:teuthology.orchestra.run.smithi101.stdout: python3-ceph-argparse x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 45 k 2023-07-18T02:53:46.018 INFO:teuthology.orchestra.run.smithi101.stdout: python3-ceph-common x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 119 k 2023-07-18T02:53:46.019 INFO:teuthology.orchestra.run.smithi101.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-18T02:53:46.019 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-18T02:53:46.019 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-18T02:53:46.019 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-18T02:53:46.019 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-18T02:53:46.019 INFO:teuthology.orchestra.run.smithi101.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-18T02:53:46.019 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-18T02:53:46.020 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-18T02:53:46.020 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-18T02:53:46.020 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-18T02:53:46.020 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-18T02:53:46.020 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-18T02:53:46.020 INFO:teuthology.orchestra.run.smithi101.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-18T02:53:46.020 INFO:teuthology.orchestra.run.smithi101.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-18T02:53:46.021 INFO:teuthology.orchestra.run.smithi101.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-18T02:53:46.021 INFO:teuthology.orchestra.run.smithi101.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-18T02:53:46.021 INFO:teuthology.orchestra.run.smithi101.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-18T02:53:46.021 INFO:teuthology.orchestra.run.smithi101.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-18T02:53:46.021 INFO:teuthology.orchestra.run.smithi101.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-18T02:53:46.021 INFO:teuthology.orchestra.run.smithi101.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-18T02:53:46.021 INFO:teuthology.orchestra.run.smithi101.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-18T02:53:46.021 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-18T02:53:46.022 INFO:teuthology.orchestra.run.smithi101.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-18T02:53:46.022 INFO:teuthology.orchestra.run.smithi101.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-18T02:53:46.022 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-18T02:53:46.022 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-18T02:53:46.022 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-18T02:53:46.022 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-18T02:53:46.022 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-18T02:53:46.022 INFO:teuthology.orchestra.run.smithi101.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-18T02:53:46.023 INFO:teuthology.orchestra.run.smithi101.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-18T02:53:46.023 INFO:teuthology.orchestra.run.smithi101.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-18T02:53:46.023 INFO:teuthology.orchestra.run.smithi101.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-18T02:53:46.023 INFO:teuthology.orchestra.run.smithi101.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-18T02:53:46.023 INFO:teuthology.orchestra.run.smithi101.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-18T02:53:46.023 INFO:teuthology.orchestra.run.smithi101.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-18T02:53:46.023 INFO:teuthology.orchestra.run.smithi101.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-18T02:53:46.024 INFO:teuthology.orchestra.run.smithi101.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-18T02:53:46.024 INFO:teuthology.orchestra.run.smithi101.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-18T02:53:46.024 INFO:teuthology.orchestra.run.smithi101.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-18T02:53:46.024 INFO:teuthology.orchestra.run.smithi101.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-18T02:53:46.024 INFO:teuthology.orchestra.run.smithi101.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-18T02:53:46.024 INFO:teuthology.orchestra.run.smithi101.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-18T02:53:46.024 INFO:teuthology.orchestra.run.smithi101.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-18T02:53:46.024 INFO:teuthology.orchestra.run.smithi101.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-18T02:53:46.025 INFO:teuthology.orchestra.run.smithi101.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-18T02:53:46.025 INFO:teuthology.orchestra.run.smithi101.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-18T02:53:46.025 INFO:teuthology.orchestra.run.smithi101.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-18T02:53:46.025 INFO:teuthology.orchestra.run.smithi101.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-18T02:53:46.025 INFO:teuthology.orchestra.run.smithi101.stdout:Enabling module streams: 2023-07-18T02:53:46.025 INFO:teuthology.orchestra.run.smithi101.stdout: postgresql 10 2023-07-18T02:53:46.025 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T02:53:46.026 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction Summary 2023-07-18T02:53:46.026 INFO:teuthology.orchestra.run.smithi101.stdout:=========================================================================================================================== 2023-07-18T02:53:46.026 INFO:teuthology.orchestra.run.smithi101.stdout:Install 128 Packages 2023-07-18T02:53:46.026 INFO:teuthology.orchestra.run.smithi101.stdout:Upgrade 3 Packages 2023-07-18T02:53:46.026 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T02:53:46.027 INFO:teuthology.orchestra.run.smithi101.stdout:Total download size: 205 M 2023-07-18T02:53:46.027 INFO:teuthology.orchestra.run.smithi101.stdout:Downloading Packages: 2023-07-18T02:53:46.216 INFO:teuthology.orchestra.run.smithi155.stdout:(1/131): ceph-18.0.0-4746.g81cf785a.el8.x86_64. 29 kB/s | 6.4 kB 00:00 2023-07-18T02:53:46.323 INFO:teuthology.orchestra.run.smithi101.stdout:(1/131): ceph-18.0.0-4746.g81cf785a.el8.x86_64. 24 kB/s | 6.4 kB 00:00 2023-07-18T02:53:46.831 INFO:teuthology.orchestra.run.smithi179.stdout:(4/131): ceph-mds-18.0.0-4746.g81cf785a.el8.x86 1.2 MB/s | 2.3 MB 00:01 2023-07-18T02:53:47.000 INFO:teuthology.orchestra.run.smithi155.stdout:(2/131): ceph-fuse-18.0.0-4746.g81cf785a.el8.x8 1.1 MB/s | 848 kB 00:00 2023-07-18T02:53:47.217 INFO:teuthology.orchestra.run.smithi155.stdout:(3/131): ceph-immutable-object-cache-18.0.0-474 764 kB/s | 165 kB 00:00 2023-07-18T02:53:47.218 INFO:teuthology.orchestra.run.smithi101.stdout:(2/131): ceph-fuse-18.0.0-4746.g81cf785a.el8.x8 948 kB/s | 848 kB 00:00 2023-07-18T02:53:47.449 INFO:teuthology.orchestra.run.smithi101.stdout:(3/131): ceph-immutable-object-cache-18.0.0-474 717 kB/s | 165 kB 00:00 2023-07-18T02:53:48.139 INFO:teuthology.orchestra.run.smithi179.stdout:(5/131): ceph-mgr-18.0.0-4746.g81cf785a.el8.x86 1.2 MB/s | 1.5 MB 00:01 2023-07-18T02:53:49.991 INFO:teuthology.orchestra.run.smithi101.stdout:(4/131): ceph-mds-18.0.0-4746.g81cf785a.el8.x86 923 kB/s | 2.3 MB 00:02 2023-07-18T02:53:50.741 INFO:teuthology.orchestra.run.smithi155.stdout:(4/131): ceph-mds-18.0.0-4746.g81cf785a.el8.x86 666 kB/s | 2.3 MB 00:03 2023-07-18T02:53:51.299 INFO:teuthology.orchestra.run.smithi101.stdout:(5/131): ceph-mgr-18.0.0-4746.g81cf785a.el8.x86 1.2 MB/s | 1.5 MB 00:01 2023-07-18T02:53:52.400 INFO:teuthology.orchestra.run.smithi155.stdout:(5/131): ceph-mgr-18.0.0-4746.g81cf785a.el8.x86 956 kB/s | 1.5 MB 00:01 2023-07-18T02:53:53.569 INFO:teuthology.orchestra.run.smithi179.stdout:(6/131): ceph-base-18.0.0-4746.g81cf785a.el8.x8 559 kB/s | 5.4 MB 00:09 2023-07-18T02:53:54.523 INFO:teuthology.orchestra.run.smithi179.stdout:(7/131): ceph-mon-18.0.0-4746.g81cf785a.el8.x86 741 kB/s | 4.6 MB 00:06 2023-07-18T02:53:55.367 INFO:teuthology.orchestra.run.smithi155.stdout:(6/131): ceph-base-18.0.0-4746.g81cf785a.el8.x8 588 kB/s | 5.4 MB 00:09 2023-07-18T02:53:55.691 INFO:teuthology.orchestra.run.smithi101.stdout:(6/131): ceph-base-18.0.0-4746.g81cf785a.el8.x8 572 kB/s | 5.4 MB 00:09 2023-07-18T02:53:56.981 INFO:teuthology.orchestra.run.smithi101.stdout:(7/131): ceph-mon-18.0.0-4746.g81cf785a.el8.x86 832 kB/s | 4.6 MB 00:05 2023-07-18T02:53:58.458 INFO:teuthology.orchestra.run.smithi155.stdout:(7/131): ceph-mon-18.0.0-4746.g81cf785a.el8.x86 781 kB/s | 4.6 MB 00:06 2023-07-18T02:54:11.223 INFO:teuthology.orchestra.run.smithi101.stdout:(8/131): ceph-common-18.0.0-4746.g81cf785a.el8. 898 kB/s | 22 MB 00:25 2023-07-18T02:54:11.307 INFO:teuthology.orchestra.run.smithi101.stdout:(9/131): ceph-selinux-18.0.0-4746.g81cf785a.el8 293 kB/s | 24 kB 00:00 2023-07-18T02:54:12.790 INFO:teuthology.orchestra.run.smithi155.stdout:(8/131): ceph-common-18.0.0-4746.g81cf785a.el8. 843 kB/s | 22 MB 00:26 2023-07-18T02:54:12.857 INFO:teuthology.orchestra.run.smithi155.stdout:(9/131): ceph-selinux-18.0.0-4746.g81cf785a.el8 366 kB/s | 24 kB 00:00 2023-07-18T02:54:13.163 INFO:teuthology.orchestra.run.smithi179.stdout:(8/131): ceph-radosgw-18.0.0-4746.g81cf785a.el8 741 kB/s | 13 MB 00:18 2023-07-18T02:54:13.238 INFO:teuthology.orchestra.run.smithi179.stdout:(9/131): ceph-selinux-18.0.0-4746.g81cf785a.el8 325 kB/s | 24 kB 00:00 2023-07-18T02:54:14.039 INFO:teuthology.orchestra.run.smithi179.stdout:(10/131): ceph-common-18.0.0-4746.g81cf785a.el8 745 kB/s | 22 MB 00:30 2023-07-18T02:54:14.082 INFO:teuthology.orchestra.run.smithi101.stdout:(10/131): ceph-radosgw-18.0.0-4746.g81cf785a.el 808 kB/s | 13 MB 00:17 2023-07-18T02:54:14.314 INFO:teuthology.orchestra.run.smithi179.stdout:(11/131): cephfs-mirror-18.0.0-4746.g81cf785a.e 938 kB/s | 258 kB 00:00 2023-07-18T02:54:14.365 INFO:teuthology.orchestra.run.smithi101.stdout:(11/131): cephfs-mirror-18.0.0-4746.g81cf785a.e 911 kB/s | 258 kB 00:00 2023-07-18T02:54:14.397 INFO:teuthology.orchestra.run.smithi179.stdout:(12/131): libcephfs-devel-18.0.0-4746.g81cf785a 372 kB/s | 31 kB 00:00 2023-07-18T02:54:14.449 INFO:teuthology.orchestra.run.smithi101.stdout:(12/131): libcephfs-devel-18.0.0-4746.g81cf785a 373 kB/s | 31 kB 00:00 2023-07-18T02:54:14.490 INFO:teuthology.orchestra.run.smithi155.stdout:(10/131): ceph-radosgw-18.0.0-4746.g81cf785a.el 861 kB/s | 13 MB 00:16 2023-07-18T02:54:14.716 INFO:teuthology.orchestra.run.smithi155.stdout:(11/131): cephfs-mirror-18.0.0-4746.g81cf785a.e 1.1 MB/s | 258 kB 00:00 2023-07-18T02:54:14.833 INFO:teuthology.orchestra.run.smithi101.stdout:(13/131): ceph-osd-18.0.0-4746.g81cf785a.el8.x8 948 kB/s | 18 MB 00:19 2023-07-18T02:54:14.834 INFO:teuthology.orchestra.run.smithi155.stdout:(12/131): libcephfs-devel-18.0.0-4746.g81cf785a 262 kB/s | 31 kB 00:00 2023-07-18T02:54:15.000 INFO:teuthology.orchestra.run.smithi101.stdout:(14/131): libcephsqlite-18.0.0-4746.g81cf785a.e 826 kB/s | 137 kB 00:00 2023-07-18T02:54:15.090 INFO:teuthology.orchestra.run.smithi179.stdout:(13/131): libcephfs2-18.0.0-4746.g81cf785a.el8. 1.1 MB/s | 772 kB 00:00 2023-07-18T02:54:15.167 INFO:teuthology.orchestra.run.smithi101.stdout:(15/131): librados-devel-18.0.0-4746.g81cf785a. 835 kB/s | 139 kB 00:00 2023-07-18T02:54:15.240 INFO:teuthology.orchestra.run.smithi179.stdout:(14/131): libcephsqlite-18.0.0-4746.g81cf785a.e 915 kB/s | 137 kB 00:00 2023-07-18T02:54:15.258 INFO:teuthology.orchestra.run.smithi101.stdout:(16/131): libcephfs2-18.0.0-4746.g81cf785a.el8. 954 kB/s | 772 kB 00:00 2023-07-18T02:54:15.391 INFO:teuthology.orchestra.run.smithi155.stdout:(13/131): libcephfs2-18.0.0-4746.g81cf785a.el8. 1.4 MB/s | 772 kB 00:00 2023-07-18T02:54:15.533 INFO:teuthology.orchestra.run.smithi155.stdout:(14/131): libcephsqlite-18.0.0-4746.g81cf785a.e 969 kB/s | 137 kB 00:00 2023-07-18T02:54:15.542 INFO:teuthology.orchestra.run.smithi101.stdout:(17/131): libradosstriper1-18.0.0-4746.g81cf785 1.2 MB/s | 473 kB 00:00 2023-07-18T02:54:15.565 INFO:teuthology.orchestra.run.smithi179.stdout:(15/131): ceph-osd-18.0.0-4746.g81cf785a.el8.x8 825 kB/s | 18 MB 00:21 2023-07-18T02:54:15.591 INFO:teuthology.orchestra.run.smithi179.stdout:(16/131): librados-devel-18.0.0-4746.g81cf785a. 397 kB/s | 139 kB 00:00 2023-07-18T02:54:15.634 INFO:teuthology.orchestra.run.smithi101.stdout:(18/131): python3-ceph-argparse-18.0.0-4746.g81 489 kB/s | 45 kB 00:00 2023-07-18T02:54:15.658 INFO:teuthology.orchestra.run.smithi155.stdout:(15/131): librados-devel-18.0.0-4746.g81cf785a. 1.1 MB/s | 139 kB 00:00 2023-07-18T02:54:15.827 INFO:teuthology.orchestra.run.smithi101.stdout:(19/131): python3-ceph-common-18.0.0-4746.g81cf 617 kB/s | 119 kB 00:00 2023-07-18T02:54:15.958 INFO:teuthology.orchestra.run.smithi179.stdout:(17/131): libradosstriper1-18.0.0-4746.g81cf785 1.2 MB/s | 473 kB 00:00 2023-07-18T02:54:15.985 INFO:teuthology.orchestra.run.smithi101.stdout:(20/131): python3-cephfs-18.0.0-4746.g81cf785a. 1.2 MB/s | 197 kB 00:00 2023-07-18T02:54:16.020 INFO:teuthology.orchestra.run.smithi155.stdout:(16/131): libradosstriper1-18.0.0-4746.g81cf785 1.3 MB/s | 473 kB 00:00 2023-07-18T02:54:16.083 INFO:teuthology.orchestra.run.smithi179.stdout:(18/131): python3-ceph-argparse-18.0.0-4746.g81 358 kB/s | 45 kB 00:00 2023-07-18T02:54:16.217 INFO:teuthology.orchestra.run.smithi179.stdout:(19/131): python3-ceph-common-18.0.0-4746.g81cf 891 kB/s | 119 kB 00:00 2023-07-18T02:54:16.243 INFO:teuthology.orchestra.run.smithi101.stdout:(21/131): python3-rados-18.0.0-4746.g81cf785a.e 1.4 MB/s | 370 kB 00:00 2023-07-18T02:54:16.417 INFO:teuthology.orchestra.run.smithi179.stdout:(20/131): python3-cephfs-18.0.0-4746.g81cf785a. 986 kB/s | 197 kB 00:00 2023-07-18T02:54:16.451 INFO:teuthology.orchestra.run.smithi155.stdout:(17/131): ceph-osd-18.0.0-4746.g81cf785a.el8.x8 861 kB/s | 18 MB 00:21 2023-07-18T02:54:16.502 INFO:teuthology.orchestra.run.smithi101.stdout:(22/131): python3-rbd-18.0.0-4746.g81cf785a.el8 1.4 MB/s | 364 kB 00:00 2023-07-18T02:54:16.526 INFO:teuthology.orchestra.run.smithi155.stdout:(18/131): python3-ceph-argparse-18.0.0-4746.g81 599 kB/s | 45 kB 00:00 2023-07-18T02:54:16.602 INFO:teuthology.orchestra.run.smithi101.stdout:(23/131): python3-rgw-18.0.0-4746.g81cf785a.el8 1.0 MB/s | 107 kB 00:00 2023-07-18T02:54:16.642 INFO:teuthology.orchestra.run.smithi179.stdout:(21/131): python3-rados-18.0.0-4746.g81cf785a.e 1.6 MB/s | 370 kB 00:00 2023-07-18T02:54:16.688 INFO:teuthology.orchestra.run.smithi155.stdout:(19/131): python3-ceph-common-18.0.0-4746.g81cf 734 kB/s | 119 kB 00:00 2023-07-18T02:54:16.694 INFO:teuthology.orchestra.run.smithi101.stdout:(24/131): rbd-fuse-18.0.0-4746.g81cf785a.el8.x8 1.0 MB/s | 94 kB 00:00 2023-07-18T02:54:16.819 INFO:teuthology.orchestra.run.smithi155.stdout:(20/131): python3-cephfs-18.0.0-4746.g81cf785a. 1.5 MB/s | 197 kB 00:00 2023-07-18T02:54:16.868 INFO:teuthology.orchestra.run.smithi179.stdout:(22/131): python3-rbd-18.0.0-4746.g81cf785a.el8 1.6 MB/s | 364 kB 00:00 2023-07-18T02:54:17.001 INFO:teuthology.orchestra.run.smithi179.stdout:(23/131): python3-rgw-18.0.0-4746.g81cf785a.el8 801 kB/s | 107 kB 00:00 2023-07-18T02:54:17.076 INFO:teuthology.orchestra.run.smithi179.stdout:(24/131): rbd-fuse-18.0.0-4746.g81cf785a.el8.x8 1.2 MB/s | 94 kB 00:00 2023-07-18T02:54:17.077 INFO:teuthology.orchestra.run.smithi155.stdout:(21/131): python3-rados-18.0.0-4746.g81cf785a.e 1.4 MB/s | 370 kB 00:00 2023-07-18T02:54:17.356 INFO:teuthology.orchestra.run.smithi155.stdout:(22/131): python3-rbd-18.0.0-4746.g81cf785a.el8 1.3 MB/s | 364 kB 00:00 2023-07-18T02:54:17.436 INFO:teuthology.orchestra.run.smithi155.stdout:(23/131): python3-rgw-18.0.0-4746.g81cf785a.el8 1.3 MB/s | 107 kB 00:00 2023-07-18T02:54:17.527 INFO:teuthology.orchestra.run.smithi155.stdout:(24/131): rbd-fuse-18.0.0-4746.g81cf785a.el8.x8 1.0 MB/s | 94 kB 00:00 2023-07-18T02:54:18.561 INFO:teuthology.orchestra.run.smithi101.stdout:(25/131): rbd-mirror-18.0.0-4746.g81cf785a.el8. 1.8 MB/s | 3.4 MB 00:01 2023-07-18T02:54:18.711 INFO:teuthology.orchestra.run.smithi101.stdout:(26/131): rbd-nbd-18.0.0-4746.g81cf785a.el8.x86 1.2 MB/s | 179 kB 00:00 2023-07-18T02:54:18.840 INFO:teuthology.orchestra.run.smithi101.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4746.g 180 kB/s | 23 kB 00:00 2023-07-18T02:54:19.171 INFO:teuthology.orchestra.run.smithi101.stdout:(28/131): ceph-mgr-cephadm-18.0.0-4746.g81cf785 390 kB/s | 129 kB 00:00 2023-07-18T02:54:19.251 INFO:teuthology.orchestra.run.smithi179.stdout:(25/131): rbd-mirror-18.0.0-4746.g81cf785a.el8. 1.6 MB/s | 3.4 MB 00:02 2023-07-18T02:54:19.427 INFO:teuthology.orchestra.run.smithi179.stdout:(26/131): rbd-nbd-18.0.0-4746.g81cf785a.el8.x86 1.0 MB/s | 179 kB 00:00 2023-07-18T02:54:19.510 INFO:teuthology.orchestra.run.smithi179.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4746.g 277 kB/s | 23 kB 00:00 2023-07-18T02:54:19.944 INFO:teuthology.orchestra.run.smithi179.stdout:(28/131): ceph-mgr-cephadm-18.0.0-4746.g81cf785 297 kB/s | 129 kB 00:00 2023-07-18T02:54:20.021 INFO:teuthology.orchestra.run.smithi101.stdout:(29/131): librgw2-18.0.0-4746.g81cf785a.el8.x86 1.4 MB/s | 6.7 MB 00:04 2023-07-18T02:54:20.119 INFO:teuthology.orchestra.run.smithi155.stdout:(25/131): rbd-mirror-18.0.0-4746.g81cf785a.el8. 1.3 MB/s | 3.4 MB 00:02 2023-07-18T02:54:20.303 INFO:teuthology.orchestra.run.smithi179.stdout:(29/131): librgw2-18.0.0-4746.g81cf785a.el8.x86 1.4 MB/s | 6.7 MB 00:04 2023-07-18T02:54:20.370 INFO:teuthology.orchestra.run.smithi155.stdout:(26/131): rbd-nbd-18.0.0-4746.g81cf785a.el8.x86 718 kB/s | 179 kB 00:00 2023-07-18T02:54:20.470 INFO:teuthology.orchestra.run.smithi155.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4746.g 231 kB/s | 23 kB 00:00 2023-07-18T02:54:20.653 INFO:teuthology.orchestra.run.smithi155.stdout:(28/131): ceph-mgr-cephadm-18.0.0-4746.g81cf785 703 kB/s | 129 kB 00:00 2023-07-18T02:54:20.829 INFO:teuthology.orchestra.run.smithi101.stdout:(30/131): ceph-mgr-dashboard-18.0.0-4746.g81cf7 1.0 MB/s | 1.7 MB 00:01 2023-07-18T02:54:20.920 INFO:teuthology.orchestra.run.smithi155.stdout:(29/131): librgw2-18.0.0-4746.g81cf785a.el8.x86 1.4 MB/s | 6.7 MB 00:04 2023-07-18T02:54:21.103 INFO:teuthology.orchestra.run.smithi101.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4746.g81 880 kB/s | 240 kB 00:00 2023-07-18T02:54:21.213 INFO:teuthology.orchestra.run.smithi101.stdout:(32/131): ceph-mgr-rook-18.0.0-4746.g81cf785a.e 432 kB/s | 47 kB 00:00 2023-07-18T02:54:21.330 INFO:teuthology.orchestra.run.smithi101.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4746.g8 125 kB/s | 15 kB 00:00 2023-07-18T02:54:21.639 INFO:teuthology.orchestra.run.smithi101.stdout:(34/131): ceph-volume-18.0.0-4746.g81cf785a.el8 838 kB/s | 258 kB 00:00 2023-07-18T02:54:21.686 INFO:teuthology.orchestra.run.smithi179.stdout:(30/131): ceph-mgr-dashboard-18.0.0-4746.g81cf7 1.0 MB/s | 1.7 MB 00:01 2023-07-18T02:54:21.806 INFO:teuthology.orchestra.run.smithi101.stdout:(35/131): cephadm-18.0.0-4746.g81cf785a.el8.noa 1.0 MB/s | 166 kB 00:00 2023-07-18T02:54:21.898 INFO:teuthology.orchestra.run.smithi101.stdout:(36/131): cephfs-top-18.0.0-4746.g81cf785a.el8. 196 kB/s | 18 kB 00:00 2023-07-18T02:54:21.903 INFO:teuthology.orchestra.run.smithi179.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4746.g81 1.1 MB/s | 240 kB 00:00 2023-07-18T02:54:21.987 INFO:teuthology.orchestra.run.smithi179.stdout:(32/131): ceph-mgr-rook-18.0.0-4746.g81cf785a.e 568 kB/s | 47 kB 00:00 2023-07-18T02:54:22.079 INFO:teuthology.orchestra.run.smithi179.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4746.g8 159 kB/s | 15 kB 00:00 2023-07-18T02:54:22.220 INFO:teuthology.orchestra.run.smithi155.stdout:(30/131): ceph-mgr-dashboard-18.0.0-4746.g81cf7 1.1 MB/s | 1.7 MB 00:01 2023-07-18T02:54:22.321 INFO:teuthology.orchestra.run.smithi179.stdout:(34/131): ceph-volume-18.0.0-4746.g81cf785a.el8 1.0 MB/s | 258 kB 00:00 2023-07-18T02:54:22.379 INFO:teuthology.orchestra.run.smithi155.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4746.g81 1.5 MB/s | 240 kB 00:00 2023-07-18T02:54:22.429 INFO:teuthology.orchestra.run.smithi179.stdout:(35/131): cephadm-18.0.0-4746.g81cf785a.el8.noa 1.5 MB/s | 166 kB 00:00 2023-07-18T02:54:22.463 INFO:teuthology.orchestra.run.smithi155.stdout:(32/131): ceph-mgr-rook-18.0.0-4746.g81cf785a.e 570 kB/s | 47 kB 00:00 2023-07-18T02:54:22.465 INFO:teuthology.orchestra.run.smithi101.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 540 kB/s | 306 kB 00:00 2023-07-18T02:54:22.521 INFO:teuthology.orchestra.run.smithi179.stdout:(36/131): cephfs-top-18.0.0-4746.g81cf785a.el8. 196 kB/s | 18 kB 00:00 2023-07-18T02:54:22.530 INFO:teuthology.orchestra.run.smithi155.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4746.g8 218 kB/s | 15 kB 00:00 2023-07-18T02:54:22.557 INFO:teuthology.orchestra.run.smithi101.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 644 kB/s | 59 kB 00:00 2023-07-18T02:54:22.657 INFO:teuthology.orchestra.run.smithi101.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 751 kB/s | 75 kB 00:00 2023-07-18T02:54:22.713 INFO:teuthology.orchestra.run.smithi179.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.6 MB/s | 306 kB 00:00 2023-07-18T02:54:22.730 INFO:teuthology.orchestra.run.smithi155.stdout:(34/131): ceph-volume-18.0.0-4746.g81cf785a.el8 1.3 MB/s | 258 kB 00:00 2023-07-18T02:54:22.772 INFO:teuthology.orchestra.run.smithi179.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 1.0 MB/s | 59 kB 00:00 2023-07-18T02:54:22.831 INFO:teuthology.orchestra.run.smithi179.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 1.3 MB/s | 75 kB 00:00 2023-07-18T02:54:22.839 INFO:teuthology.orchestra.run.smithi155.stdout:(35/131): cephadm-18.0.0-4746.g81cf785a.el8.noa 1.5 MB/s | 166 kB 00:00 2023-07-18T02:54:22.849 INFO:teuthology.orchestra.run.smithi101.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 2.4 MB/s | 476 kB 00:00 2023-07-18T02:54:22.914 INFO:teuthology.orchestra.run.smithi155.stdout:(36/131): cephfs-top-18.0.0-4746.g81cf785a.el8. 240 kB/s | 18 kB 00:00 2023-07-18T02:54:22.973 INFO:teuthology.orchestra.run.smithi179.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 3.3 MB/s | 476 kB 00:00 2023-07-18T02:54:22.980 INFO:teuthology.orchestra.run.smithi101.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 335 kB/s | 44 kB 00:00 2023-07-18T02:54:23.024 INFO:teuthology.orchestra.run.smithi179.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 871 kB/s | 44 kB 00:00 2023-07-18T02:54:23.091 INFO:teuthology.orchestra.run.smithi101.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.6 MB/s | 185 kB 00:00 2023-07-18T02:54:23.115 INFO:teuthology.orchestra.run.smithi179.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 2.0 MB/s | 185 kB 00:00 2023-07-18T02:54:23.166 INFO:teuthology.orchestra.run.smithi179.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 659 kB/s | 33 kB 00:00 2023-07-18T02:54:23.200 INFO:teuthology.orchestra.run.smithi101.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 305 kB/s | 33 kB 00:00 2023-07-18T02:54:23.225 INFO:teuthology.orchestra.run.smithi179.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 278 kB/s | 16 kB 00:00 2023-07-18T02:54:23.239 INFO:teuthology.orchestra.run.smithi155.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 942 kB/s | 306 kB 00:00 2023-07-18T02:54:23.323 INFO:teuthology.orchestra.run.smithi155.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 708 kB/s | 59 kB 00:00 2023-07-18T02:54:23.341 INFO:teuthology.orchestra.run.smithi179.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.4 MB/s | 173 kB 00:00 2023-07-18T02:54:23.375 INFO:teuthology.orchestra.run.smithi101.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 93 kB/s | 16 kB 00:00 2023-07-18T02:54:23.423 INFO:teuthology.orchestra.run.smithi155.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 751 kB/s | 75 kB 00:00 2023-07-18T02:54:23.501 INFO:teuthology.orchestra.run.smithi101.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.3 MB/s | 173 kB 00:00 2023-07-18T02:54:23.517 INFO:teuthology.orchestra.run.smithi179.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 2.1 MB/s | 384 kB 00:00 2023-07-18T02:54:23.584 INFO:teuthology.orchestra.run.smithi179.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 1.2 MB/s | 81 kB 00:00 2023-07-18T02:54:23.598 INFO:teuthology.orchestra.run.smithi155.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 2.7 MB/s | 476 kB 00:00 2023-07-18T02:54:23.618 INFO:teuthology.orchestra.run.smithi101.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 3.2 MB/s | 384 kB 00:00 2023-07-18T02:54:23.649 INFO:teuthology.orchestra.run.smithi155.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 870 kB/s | 44 kB 00:00 2023-07-18T02:54:23.668 INFO:teuthology.orchestra.run.smithi179.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 133 kB/s | 11 kB 00:00 2023-07-18T02:54:23.718 INFO:teuthology.orchestra.run.smithi101.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 815 kB/s | 81 kB 00:00 2023-07-18T02:54:23.726 INFO:teuthology.orchestra.run.smithi179.stdout:(49/131): python3-jaraco-functools-2.0-4.el8.no 302 kB/s | 18 kB 00:00 2023-07-18T02:54:23.766 INFO:teuthology.orchestra.run.smithi155.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.6 MB/s | 185 kB 00:00 2023-07-18T02:54:23.816 INFO:teuthology.orchestra.run.smithi155.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 663 kB/s | 33 kB 00:00 2023-07-18T02:54:23.827 INFO:teuthology.orchestra.run.smithi101.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 102 kB/s | 11 kB 00:00 2023-07-18T02:54:23.866 INFO:teuthology.orchestra.run.smithi155.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 326 kB/s | 16 kB 00:00 2023-07-18T02:54:23.927 INFO:teuthology.orchestra.run.smithi101.stdout:(49/131): python3-jaraco-functools-2.0-4.el8.no 176 kB/s | 18 kB 00:00 2023-07-18T02:54:23.935 INFO:teuthology.orchestra.run.smithi179.stdout:(50/131): python3-kubernetes-11.0.0-6.el8.noarc 5.6 MB/s | 1.2 MB 00:00 2023-07-18T02:54:23.950 INFO:teuthology.orchestra.run.smithi155.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.0 MB/s | 173 kB 00:00 2023-07-18T02:54:23.993 INFO:teuthology.orchestra.run.smithi179.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 837 kB/s | 49 kB 00:00 2023-07-18T02:54:24.052 INFO:teuthology.orchestra.run.smithi179.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 1.0 MB/s | 59 kB 00:00 2023-07-18T02:54:24.058 INFO:teuthology.orchestra.run.smithi155.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 3.5 MB/s | 384 kB 00:00 2023-07-18T02:54:24.111 INFO:teuthology.orchestra.run.smithi179.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.0 MB/s | 60 kB 00:00 2023-07-18T02:54:24.134 INFO:teuthology.orchestra.run.smithi155.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 1.1 MB/s | 81 kB 00:00 2023-07-18T02:54:24.153 INFO:teuthology.orchestra.run.smithi101.stdout:(50/131): python3-kubernetes-11.0.0-6.el8.noarc 5.2 MB/s | 1.2 MB 00:00 2023-07-18T02:54:24.186 INFO:teuthology.orchestra.run.smithi179.stdout:(54/131): python3-pecan-1.3.2-9.el8.noarch.rpm 3.7 MB/s | 283 kB 00:00 2023-07-18T02:54:24.193 INFO:teuthology.orchestra.run.smithi155.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 191 kB/s | 11 kB 00:00 2023-07-18T02:54:24.245 INFO:teuthology.orchestra.run.smithi179.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 267 kB/s | 16 kB 00:00 2023-07-18T02:54:24.261 INFO:teuthology.orchestra.run.smithi101.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 450 kB/s | 49 kB 00:00 2023-07-18T02:54:24.276 INFO:teuthology.orchestra.run.smithi155.stdout:(49/131): python3-jaraco-functools-2.0-4.el8.no 211 kB/s | 18 kB 00:00 2023-07-18T02:54:24.287 INFO:teuthology.orchestra.run.smithi179.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 808 kB/s | 34 kB 00:00 2023-07-18T02:54:24.346 INFO:teuthology.orchestra.run.smithi179.stdout:(57/131): python3-routes-2.4.1-12.el8.noarch.rp 3.3 MB/s | 196 kB 00:00 2023-07-18T02:54:24.370 INFO:teuthology.orchestra.run.smithi101.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 543 kB/s | 59 kB 00:00 2023-07-18T02:54:24.413 INFO:teuthology.orchestra.run.smithi179.stdout:(58/131): python3-rsa-4.8-1.el8.noarch.rpm 923 kB/s | 61 kB 00:00 2023-07-18T02:54:24.451 INFO:teuthology.orchestra.run.smithi155.stdout:(50/131): python3-kubernetes-11.0.0-6.el8.noarc 6.7 MB/s | 1.2 MB 00:00 2023-07-18T02:54:24.480 INFO:teuthology.orchestra.run.smithi179.stdout:(59/131): python3-simplegeneric-0.8.1-17.el8.no 281 kB/s | 19 kB 00:00 2023-07-18T02:54:24.520 INFO:teuthology.orchestra.run.smithi101.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 404 kB/s | 60 kB 00:00 2023-07-18T02:54:24.530 INFO:teuthology.orchestra.run.smithi179.stdout:(60/131): python3-singledispatch-3.4.0.3-18.el8 480 kB/s | 24 kB 00:00 2023-07-18T02:54:24.535 INFO:teuthology.orchestra.run.smithi155.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 585 kB/s | 49 kB 00:00 2023-07-18T02:54:24.589 INFO:teuthology.orchestra.run.smithi179.stdout:(61/131): python3-tempora-1.14.1-5.el8.noarch.r 500 kB/s | 29 kB 00:00 2023-07-18T02:54:24.602 INFO:teuthology.orchestra.run.smithi155.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 882 kB/s | 59 kB 00:00 2023-07-18T02:54:24.647 INFO:teuthology.orchestra.run.smithi179.stdout:(62/131): python3-trustme-0.6.0-4.el8.noarch.rp 472 kB/s | 27 kB 00:00 2023-07-18T02:54:24.654 INFO:teuthology.orchestra.run.smithi101.stdout:(54/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.1 MB/s | 283 kB 00:00 2023-07-18T02:54:24.660 INFO:teuthology.orchestra.run.smithi155.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.0 MB/s | 60 kB 00:00 2023-07-18T02:54:24.748 INFO:teuthology.orchestra.run.smithi179.stdout:(63/131): python3-waitress-1.4.3-1.el8.noarch.r 2.5 MB/s | 255 kB 00:00 2023-07-18T02:54:24.754 INFO:teuthology.orchestra.run.smithi101.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 156 kB/s | 16 kB 00:00 2023-07-18T02:54:24.777 INFO:teuthology.orchestra.run.smithi155.stdout:(54/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.4 MB/s | 283 kB 00:00 2023-07-18T02:54:24.840 INFO:teuthology.orchestra.run.smithi179.stdout:(64/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.7 MB/s | 251 kB 00:00 2023-07-18T02:54:24.853 INFO:teuthology.orchestra.run.smithi155.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 207 kB/s | 16 kB 00:00 2023-07-18T02:54:24.890 INFO:teuthology.orchestra.run.smithi101.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 249 kB/s | 34 kB 00:00 2023-07-18T02:54:24.899 INFO:teuthology.orchestra.run.smithi179.stdout:(65/131): python3-websocket-client-0.56.0-5.el8 1.0 MB/s | 61 kB 00:00 2023-07-18T02:54:24.920 INFO:teuthology.orchestra.run.smithi155.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 506 kB/s | 34 kB 00:00 2023-07-18T02:54:24.957 INFO:teuthology.orchestra.run.smithi179.stdout:(66/131): python3-webtest-2.0.33-1.el8.noarch.r 1.4 MB/s | 85 kB 00:00 2023-07-18T02:54:24.995 INFO:teuthology.orchestra.run.smithi155.stdout:(57/131): python3-routes-2.4.1-12.el8.noarch.rp 2.6 MB/s | 196 kB 00:00 2023-07-18T02:54:25.005 INFO:teuthology.orchestra.run.smithi101.stdout:(57/131): python3-routes-2.4.1-12.el8.noarch.rp 1.7 MB/s | 196 kB 00:00 2023-07-18T02:54:25.025 INFO:teuthology.orchestra.run.smithi179.stdout:(67/131): python3-zc-lockfile-2.0-2.el8.noarch. 345 kB/s | 23 kB 00:00 2023-07-18T02:54:25.079 INFO:teuthology.orchestra.run.smithi155.stdout:(58/131): python3-rsa-4.8-1.el8.noarch.rpm 736 kB/s | 61 kB 00:00 2023-07-18T02:54:25.122 INFO:teuthology.orchestra.run.smithi101.stdout:(58/131): python3-rsa-4.8-1.el8.noarch.rpm 528 kB/s | 61 kB 00:00 2023-07-18T02:54:25.146 INFO:teuthology.orchestra.run.smithi155.stdout:(59/131): python3-simplegeneric-0.8.1-17.el8.no 281 kB/s | 19 kB 00:00 2023-07-18T02:54:25.221 INFO:teuthology.orchestra.run.smithi155.stdout:(60/131): python3-singledispatch-3.4.0.3-18.el8 320 kB/s | 24 kB 00:00 2023-07-18T02:54:25.281 INFO:teuthology.orchestra.run.smithi101.stdout:(59/131): python3-simplegeneric-0.8.1-17.el8.no 118 kB/s | 19 kB 00:00 2023-07-18T02:54:25.324 INFO:teuthology.orchestra.run.smithi155.stdout:(61/131): python3-tempora-1.14.1-5.el8.noarch.r 293 kB/s | 29 kB 00:00 2023-07-18T02:54:25.373 INFO:teuthology.orchestra.run.smithi101.stdout:(60/131): ceph-mgr-diskprediction-local-18.0.0- 1.4 MB/s | 7.4 MB 00:05 2023-07-18T02:54:25.375 INFO:teuthology.orchestra.run.smithi179.stdout:(68/131): thrift-0.13.0-2.el8.x86_64.rpm 5.0 MB/s | 1.7 MB 00:00 2023-07-18T02:54:25.388 INFO:teuthology.orchestra.run.smithi155.stdout:(62/131): python3-trustme-0.6.0-4.el8.noarch.rp 413 kB/s | 27 kB 00:00 2023-07-18T02:54:25.398 INFO:teuthology.orchestra.run.smithi101.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 204 kB/s | 24 kB 00:00 2023-07-18T02:54:25.400 INFO:teuthology.orchestra.run.smithi179.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 70 kB 00:00 2023-07-18T02:54:25.480 INFO:teuthology.orchestra.run.smithi155.stdout:(63/131): python3-waitress-1.4.3-1.el8.noarch.r 2.7 MB/s | 255 kB 00:00 2023-07-18T02:54:25.523 INFO:teuthology.orchestra.run.smithi101.stdout:(62/131): python3-trustme-0.6.0-4.el8.noarch.rp 220 kB/s | 27 kB 00:00 2023-07-18T02:54:25.580 INFO:teuthology.orchestra.run.smithi155.stdout:(64/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.5 MB/s | 251 kB 00:00 2023-07-18T02:54:25.665 INFO:teuthology.orchestra.run.smithi101.stdout:(63/131): python3-waitress-1.4.3-1.el8.noarch.r 1.8 MB/s | 255 kB 00:00 2023-07-18T02:54:25.672 INFO:teuthology.orchestra.run.smithi155.stdout:(65/131): python3-websocket-client-0.56.0-5.el8 669 kB/s | 61 kB 00:00 2023-07-18T02:54:25.699 INFO:teuthology.orchestra.run.smithi101.stdout:(64/131): python3-tempora-1.14.1-5.el8.noarch.r 90 kB/s | 29 kB 00:00 2023-07-18T02:54:25.700 INFO:teuthology.orchestra.run.smithi179.stdout:(70/131): python3-scipy-1.0.0-21.module+el8.5.0 46 MB/s | 14 MB 00:00 2023-07-18T02:54:25.742 INFO:teuthology.orchestra.run.smithi179.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-07-18T02:54:25.764 INFO:teuthology.orchestra.run.smithi155.stdout:(66/131): python3-webtest-2.0.33-1.el8.noarch.r 932 kB/s | 85 kB 00:00 2023-07-18T02:54:25.784 INFO:teuthology.orchestra.run.smithi179.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-07-18T02:54:25.816 INFO:teuthology.orchestra.run.smithi101.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.6 MB/s | 251 kB 00:00 2023-07-18T02:54:25.835 INFO:teuthology.orchestra.run.smithi179.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.1 MB/s | 259 kB 00:00 2023-07-18T02:54:25.840 INFO:teuthology.orchestra.run.smithi155.stdout:(67/131): python3-zc-lockfile-2.0-2.el8.noarch. 306 kB/s | 23 kB 00:00 2023-07-18T02:54:25.877 INFO:teuthology.orchestra.run.smithi179.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 7.5 MB/s | 323 kB 00:00 2023-07-18T02:54:25.916 INFO:teuthology.orchestra.run.smithi101.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 282 kB/s | 61 kB 00:00 2023-07-18T02:54:25.919 INFO:teuthology.orchestra.run.smithi179.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 933 kB/s | 39 kB 00:00 2023-07-18T02:54:25.942 INFO:teuthology.orchestra.run.smithi101.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 683 kB/s | 85 kB 00:00 2023-07-18T02:54:25.977 INFO:teuthology.orchestra.run.smithi179.stdout:(76/131): bison-3.0.4-10.el8.x86_64.rpm 11 MB/s | 688 kB 00:00 2023-07-18T02:54:26.028 INFO:teuthology.orchestra.run.smithi179.stdout:(77/131): python3-jinja2-2.10.1-3.el8.noarch.rp 10 MB/s | 538 kB 00:00 2023-07-18T02:54:26.059 INFO:teuthology.orchestra.run.smithi101.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 162 kB/s | 23 kB 00:00 2023-07-18T02:54:26.070 INFO:teuthology.orchestra.run.smithi179.stdout:(78/131): python3-mako-1.0.6-13.el8.noarch.rpm 3.7 MB/s | 157 kB 00:00 2023-07-18T02:54:26.084 INFO:teuthology.orchestra.run.smithi101.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 70 kB 00:00 2023-07-18T02:54:26.103 INFO:teuthology.orchestra.run.smithi179.stdout:(79/131): oniguruma-6.8.2-2.el8.x86_64.rpm 5.5 MB/s | 187 kB 00:00 2023-07-18T02:54:26.115 INFO:teuthology.orchestra.run.smithi155.stdout:(68/131): thrift-0.13.0-2.el8.x86_64.rpm 6.3 MB/s | 1.7 MB 00:00 2023-07-18T02:54:26.140 INFO:teuthology.orchestra.run.smithi155.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 70 kB 00:00 2023-07-18T02:54:26.162 INFO:teuthology.orchestra.run.smithi179.stdout:(80/131): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2023-07-18T02:54:26.251 INFO:teuthology.orchestra.run.smithi101.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 5.6 MB/s | 1.7 MB 00:00 2023-07-18T02:54:26.304 INFO:teuthology.orchestra.run.smithi179.stdout:(81/131): python3-babel-2.5.1-7.el8.noarch.rpm 34 MB/s | 4.8 MB 00:00 2023-07-18T02:54:26.309 INFO:teuthology.orchestra.run.smithi101.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.9 MB/s | 110 kB 00:00 2023-07-18T02:54:26.352 INFO:teuthology.orchestra.run.smithi101.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-07-18T02:54:26.396 INFO:teuthology.orchestra.run.smithi179.stdout:(82/131): ceph-mgr-diskprediction-local-18.0.0- 1.2 MB/s | 7.4 MB 00:06 2023-07-18T02:54:26.413 INFO:teuthology.orchestra.run.smithi179.stdout:(83/131): python3-jsonpatch-1.21-2.el8.noarch.r 245 kB/s | 27 kB 00:00 2023-07-18T02:54:26.427 INFO:teuthology.orchestra.run.smithi101.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.4 MB/s | 259 kB 00:00 2023-07-18T02:54:26.457 INFO:teuthology.orchestra.run.smithi155.stdout:(70/131): python3-scipy-1.0.0-21.module+el8.5.0 43 MB/s | 14 MB 00:00 2023-07-18T02:54:26.463 INFO:teuthology.orchestra.run.smithi179.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 988 kB/s | 44 kB 00:00 2023-07-18T02:54:26.492 INFO:teuthology.orchestra.run.smithi179.stdout:(85/131): flex-2.6.1-9.el8.x86_64.rpm 3.4 MB/s | 320 kB 00:00 2023-07-18T02:54:26.496 INFO:teuthology.orchestra.run.smithi101.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 4.6 MB/s | 323 kB 00:00 2023-07-18T02:54:26.499 INFO:teuthology.orchestra.run.smithi155.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-07-18T02:54:26.548 INFO:teuthology.orchestra.run.smithi179.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 7.6 MB/s | 457 kB 00:00 2023-07-18T02:54:26.558 INFO:teuthology.orchestra.run.smithi155.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 928 kB/s | 54 kB 00:00 2023-07-18T02:54:26.606 INFO:teuthology.orchestra.run.smithi179.stdout:(87/131): python3-numpy-1.14.3-10.el8.x86_64.rp 26 MB/s | 3.7 MB 00:00 2023-07-18T02:54:26.625 INFO:teuthology.orchestra.run.smithi155.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.8 MB/s | 259 kB 00:00 2023-07-18T02:54:26.632 INFO:teuthology.orchestra.run.smithi179.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 2.6 MB/s | 225 kB 00:00 2023-07-18T02:54:26.657 INFO:teuthology.orchestra.run.smithi179.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2023-07-18T02:54:26.675 INFO:teuthology.orchestra.run.smithi155.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 6.3 MB/s | 323 kB 00:00 2023-07-18T02:54:26.682 INFO:teuthology.orchestra.run.smithi179.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2023-07-18T02:54:26.688 INFO:teuthology.orchestra.run.smithi101.stdout:(75/131): python3-scipy-1.0.0-21.module+el8.5.0 23 MB/s | 14 MB 00:00 2023-07-18T02:54:26.708 INFO:teuthology.orchestra.run.smithi179.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 397 kB/s | 20 kB 00:00 2023-07-18T02:54:26.713 INFO:teuthology.orchestra.run.smithi101.stdout:(76/131): python3-markupsafe-0.23-19.el8.x86_64 179 kB/s | 39 kB 00:00 2023-07-18T02:54:26.717 INFO:teuthology.orchestra.run.smithi155.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 931 kB/s | 39 kB 00:00 2023-07-18T02:54:26.733 INFO:teuthology.orchestra.run.smithi179.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.0 MB/s | 103 kB 00:00 2023-07-18T02:54:26.775 INFO:teuthology.orchestra.run.smithi179.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 10 MB/s | 709 kB 00:00 2023-07-18T02:54:26.783 INFO:teuthology.orchestra.run.smithi101.stdout:(77/131): bison-3.0.4-10.el8.x86_64.rpm 7.1 MB/s | 688 kB 00:00 2023-07-18T02:54:26.784 INFO:teuthology.orchestra.run.smithi155.stdout:(76/131): bison-3.0.4-10.el8.x86_64.rpm 10 MB/s | 688 kB 00:00 2023-07-18T02:54:26.823 INFO:teuthology.orchestra.run.smithi101.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 4.8 MB/s | 538 kB 00:00 2023-07-18T02:54:26.825 INFO:teuthology.orchestra.run.smithi179.stdout:(94/131): libpq-13.5-1.el8.x86_64.rpm 3.9 MB/s | 198 kB 00:00 2023-07-18T02:54:26.848 INFO:teuthology.orchestra.run.smithi101.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.4 MB/s | 157 kB 00:00 2023-07-18T02:54:26.873 INFO:teuthology.orchestra.run.smithi101.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.6 MB/s | 187 kB 00:00 2023-07-18T02:54:26.884 INFO:teuthology.orchestra.run.smithi155.stdout:(77/131): python3-jinja2-2.10.1-3.el8.noarch.rp 5.2 MB/s | 538 kB 00:00 2023-07-18T02:54:26.892 INFO:teuthology.orchestra.run.smithi179.stdout:(95/131): openblas-threads-0.3.15-3.el8.x86_64. 31 MB/s | 4.9 MB 00:00 2023-07-18T02:54:26.915 INFO:teuthology.orchestra.run.smithi101.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 10 MB/s | 713 kB 00:00 2023-07-18T02:54:26.926 INFO:teuthology.orchestra.run.smithi155.stdout:(78/131): python3-mako-1.0.6-13.el8.noarch.rpm 3.7 MB/s | 157 kB 00:00 2023-07-18T02:54:26.951 INFO:teuthology.orchestra.run.smithi179.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 3.4 MB/s | 202 kB 00:00 2023-07-18T02:54:26.957 INFO:teuthology.orchestra.run.smithi101.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 637 kB/s | 27 kB 00:00 2023-07-18T02:54:26.976 INFO:teuthology.orchestra.run.smithi155.stdout:(79/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2023-07-18T02:54:27.007 INFO:teuthology.orchestra.run.smithi101.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 6.3 MB/s | 320 kB 00:00 2023-07-18T02:54:27.026 INFO:teuthology.orchestra.run.smithi179.stdout:(97/131): openblas-0.3.15-3.el8.x86_64.rpm 24 MB/s | 4.7 MB 00:00 2023-07-18T02:54:27.027 INFO:teuthology.orchestra.run.smithi155.stdout:(80/131): automake-1.16.1-7.el8.noarch.rpm 14 MB/s | 713 kB 00:00 2023-07-18T02:54:27.049 INFO:teuthology.orchestra.run.smithi101.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 1.0 MB/s | 44 kB 00:00 2023-07-18T02:54:27.051 INFO:teuthology.orchestra.run.smithi179.stdout:(98/131): librdkafka-0.11.4-3.el8.x86_64.rpm 3.4 MB/s | 354 kB 00:00 2023-07-18T02:54:27.116 INFO:teuthology.orchestra.run.smithi101.stdout:(85/131): python3-babel-2.5.1-7.el8.noarch.rpm 20 MB/s | 4.8 MB 00:00 2023-07-18T02:54:27.135 INFO:teuthology.orchestra.run.smithi179.stdout:(99/131): postgresql-contrib-10.21-2.module+el8 9.5 MB/s | 808 kB 00:00 2023-07-18T02:54:27.169 INFO:teuthology.orchestra.run.smithi155.stdout:(81/131): python3-babel-2.5.1-7.el8.noarch.rpm 34 MB/s | 4.8 MB 00:00 2023-07-18T02:54:27.183 INFO:teuthology.orchestra.run.smithi101.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 6.7 MB/s | 457 kB 00:00 2023-07-18T02:54:27.204 INFO:teuthology.orchestra.run.smithi155.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 803 kB/s | 27 kB 00:00 2023-07-18T02:54:27.219 INFO:teuthology.orchestra.run.smithi179.stdout:(100/131): postgresql-server-10.21-2.module+el8 26 MB/s | 5.1 MB 00:00 2023-07-18T02:54:27.233 INFO:teuthology.orchestra.run.smithi101.stdout:(87/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 4.4 MB/s | 225 kB 00:00 2023-07-18T02:54:27.244 INFO:teuthology.orchestra.run.smithi155.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-07-18T02:54:27.269 INFO:teuthology.orchestra.run.smithi179.stdout:(101/131): postgresql-10.21-2.module+el8.6.0+15 11 MB/s | 1.5 MB 00:00 2023-07-18T02:54:27.278 INFO:teuthology.orchestra.run.smithi155.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 1.3 MB/s | 44 kB 00:00 2023-07-18T02:54:27.295 INFO:teuthology.orchestra.run.smithi179.stdout:(102/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 620 kB/s | 47 kB 00:00 2023-07-18T02:54:27.301 INFO:teuthology.orchestra.run.smithi101.stdout:(88/131): python3-numpy-1.14.3-10.el8.x86_64.rp 15 MB/s | 3.7 MB 00:00 2023-07-18T02:54:27.320 INFO:teuthology.orchestra.run.smithi179.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 3.9 MB/s | 200 kB 00:00 2023-07-18T02:54:27.326 INFO:teuthology.orchestra.run.smithi101.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 695 kB/s | 64 kB 00:00 2023-07-18T02:54:27.346 INFO:teuthology.orchestra.run.smithi179.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 848 kB/s | 43 kB 00:00 2023-07-18T02:54:27.351 INFO:teuthology.orchestra.run.smithi101.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2023-07-18T02:54:27.371 INFO:teuthology.orchestra.run.smithi179.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-07-18T02:54:27.377 INFO:teuthology.orchestra.run.smithi101.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 396 kB/s | 20 kB 00:00 2023-07-18T02:54:27.387 INFO:teuthology.orchestra.run.smithi179.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 1.1 MB/s | 48 kB 00:00 2023-07-18T02:54:27.402 INFO:teuthology.orchestra.run.smithi101.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.0 MB/s | 103 kB 00:00 2023-07-18T02:54:27.404 INFO:teuthology.orchestra.run.smithi179.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 2.5 MB/s | 87 kB 00:00 2023-07-18T02:54:27.428 INFO:teuthology.orchestra.run.smithi155.stdout:(85/131): python3-numpy-1.14.3-10.el8.x86_64.rp 25 MB/s | 3.7 MB 00:00 2023-07-18T02:54:27.430 INFO:teuthology.orchestra.run.smithi179.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 1.0 MB/s | 43 kB 00:00 2023-07-18T02:54:27.444 INFO:teuthology.orchestra.run.smithi101.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 10 MB/s | 709 kB 00:00 2023-07-18T02:54:27.455 INFO:teuthology.orchestra.run.smithi179.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.1 MB/s | 109 kB 00:00 2023-07-18T02:54:27.479 INFO:teuthology.orchestra.run.smithi155.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 8.9 MB/s | 457 kB 00:00 2023-07-18T02:54:27.481 INFO:teuthology.orchestra.run.smithi179.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.4 MB/s | 229 kB 00:00 2023-07-18T02:54:27.486 INFO:teuthology.orchestra.run.smithi101.stdout:(94/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-07-18T02:54:27.515 INFO:teuthology.orchestra.run.smithi179.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.3 MB/s | 560 kB 00:00 2023-07-18T02:54:27.520 INFO:teuthology.orchestra.run.smithi155.stdout:(87/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 5.3 MB/s | 225 kB 00:00 2023-07-18T02:54:27.532 INFO:teuthology.orchestra.run.smithi179.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 371 kB/s | 19 kB 00:00 2023-07-18T02:54:27.557 INFO:teuthology.orchestra.run.smithi179.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 3.6 MB/s | 155 kB 00:00 2023-07-18T02:54:27.562 INFO:teuthology.orchestra.run.smithi155.stdout:(88/131): uuid-1.6.2-43.el8.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2023-07-18T02:54:27.591 INFO:teuthology.orchestra.run.smithi179.stdout:(114/131): python3-cryptography-3.2.1-5.el8.x86 9.3 MB/s | 559 kB 00:00 2023-07-18T02:54:27.611 INFO:teuthology.orchestra.run.smithi101.stdout:(95/131): openblas-threads-0.3.15-3.el8.x86_64. 23 MB/s | 4.9 MB 00:00 2023-07-18T02:54:27.616 INFO:teuthology.orchestra.run.smithi179.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 304 kB/s | 18 kB 00:00 2023-07-18T02:54:27.630 INFO:teuthology.orchestra.run.smithi155.stdout:(89/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.8 MB/s | 126 kB 00:00 2023-07-18T02:54:27.650 INFO:teuthology.orchestra.run.smithi179.stdout:(116/131): pcre2-devel-10.32-2.el8.x86_64.rpm 10 MB/s | 605 kB 00:00 2023-07-18T02:54:27.653 INFO:teuthology.orchestra.run.smithi101.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-07-18T02:54:27.671 INFO:teuthology.orchestra.run.smithi155.stdout:(90/131): python3-jsonpointer-1.10-11.el8.noarc 491 kB/s | 20 kB 00:00 2023-07-18T02:54:27.677 INFO:teuthology.orchestra.run.smithi179.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.9 MB/s | 238 kB 00:00 2023-07-18T02:54:27.714 INFO:teuthology.orchestra.run.smithi155.stdout:(91/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.4 MB/s | 103 kB 00:00 2023-07-18T02:54:27.717 INFO:teuthology.orchestra.run.smithi179.stdout:(118/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.7 MB/s | 72 kB 00:00 2023-07-18T02:54:27.745 INFO:teuthology.orchestra.run.smithi101.stdout:(97/131): openblas-0.3.15-3.el8.x86_64.rpm 18 MB/s | 4.7 MB 00:00 2023-07-18T02:54:27.764 INFO:teuthology.orchestra.run.smithi155.stdout:(92/131): libtool-2.4.6-25.el8.x86_64.rpm 14 MB/s | 709 kB 00:00 2023-07-18T02:54:27.776 INFO:teuthology.orchestra.run.smithi179.stdout:(119/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 18 MB/s | 2.3 MB 00:00 2023-07-18T02:54:27.787 INFO:teuthology.orchestra.run.smithi101.stdout:(98/131): librdkafka-0.11.4-3.el8.x86_64.rpm 2.6 MB/s | 354 kB 00:00 2023-07-18T02:54:27.802 INFO:teuthology.orchestra.run.smithi179.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 987 kB/s | 83 kB 00:00 2023-07-18T02:54:27.827 INFO:teuthology.orchestra.run.smithi179.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.2 MB/s | 60 kB 00:00 2023-07-18T02:54:27.843 INFO:teuthology.orchestra.run.smithi179.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 2.3 MB/s | 98 kB 00:00 2023-07-18T02:54:27.854 INFO:teuthology.orchestra.run.smithi101.stdout:(99/131): postgresql-contrib-10.21-2.module+el8 12 MB/s | 808 kB 00:00 2023-07-18T02:54:27.869 INFO:teuthology.orchestra.run.smithi179.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.3 MB/s | 58 kB 00:00 2023-07-18T02:54:27.894 INFO:teuthology.orchestra.run.smithi179.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 764 kB/s | 39 kB 00:00 2023-07-18T02:54:27.928 INFO:teuthology.orchestra.run.smithi179.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 11 MB/s | 644 kB 00:00 2023-07-18T02:54:27.947 INFO:teuthology.orchestra.run.smithi101.stdout:(100/131): postgresql-10.21-2.module+el8.6.0+15 16 MB/s | 1.5 MB 00:00 2023-07-18T02:54:27.947 INFO:teuthology.orchestra.run.smithi155.stdout:(93/131): openblas-threads-0.3.15-3.el8.x86_64. 27 MB/s | 4.9 MB 00:00 2023-07-18T02:54:27.953 INFO:teuthology.orchestra.run.smithi179.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.8 MB/s | 171 kB 00:00 2023-07-18T02:54:27.979 INFO:teuthology.orchestra.run.smithi179.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-07-18T02:54:27.989 INFO:teuthology.orchestra.run.smithi155.stdout:(94/131): libpq-13.5-1.el8.x86_64.rpm 4.7 MB/s | 198 kB 00:00 2023-07-18T02:54:28.021 INFO:teuthology.orchestra.run.smithi101.stdout:(101/131): postgresql-server-10.21-2.module+el8 18 MB/s | 5.1 MB 00:00 2023-07-18T02:54:28.047 INFO:teuthology.orchestra.run.smithi101.stdout:(102/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 468 kB/s | 47 kB 00:00 2023-07-18T02:54:28.073 INFO:teuthology.orchestra.run.smithi101.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 3.9 MB/s | 200 kB 00:00 2023-07-18T02:54:28.097 INFO:teuthology.orchestra.run.smithi101.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 848 kB/s | 43 kB 00:00 2023-07-18T02:54:28.116 INFO:teuthology.orchestra.run.smithi155.stdout:(95/131): openblas-0.3.15-3.el8.x86_64.rpm 38 MB/s | 4.7 MB 00:00 2023-07-18T02:54:28.123 INFO:teuthology.orchestra.run.smithi101.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-07-18T02:54:28.149 INFO:teuthology.orchestra.run.smithi101.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 953 kB/s | 48 kB 00:00 2023-07-18T02:54:28.174 INFO:teuthology.orchestra.run.smithi101.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-07-18T02:54:28.191 INFO:teuthology.orchestra.run.smithi101.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 1.0 MB/s | 43 kB 00:00 2023-07-18T02:54:28.206 INFO:teuthology.orchestra.run.smithi155.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 2.2 MB/s | 202 kB 00:00 2023-07-18T02:54:28.216 INFO:teuthology.orchestra.run.smithi101.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.5 MB/s | 109 kB 00:00 2023-07-18T02:54:28.249 INFO:teuthology.orchestra.run.smithi155.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-07-18T02:54:28.264 INFO:teuthology.orchestra.run.smithi101.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.1 MB/s | 229 kB 00:00 2023-07-18T02:54:28.308 INFO:teuthology.orchestra.run.smithi101.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 5.9 MB/s | 560 kB 00:00 2023-07-18T02:54:28.325 INFO:teuthology.orchestra.run.smithi101.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 305 kB/s | 19 kB 00:00 2023-07-18T02:54:28.359 INFO:teuthology.orchestra.run.smithi101.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 3.0 MB/s | 155 kB 00:00 2023-07-18T02:54:28.374 INFO:teuthology.orchestra.run.smithi155.stdout:(98/131): postgresql-server-10.21-2.module+el8. 41 MB/s | 5.1 MB 00:00 2023-07-18T02:54:28.392 INFO:teuthology.orchestra.run.smithi101.stdout:(114/131): python3-cryptography-3.2.1-5.el8.x86 8.2 MB/s | 559 kB 00:00 2023-07-18T02:54:28.418 INFO:teuthology.orchestra.run.smithi101.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 305 kB/s | 18 kB 00:00 2023-07-18T02:54:28.451 INFO:teuthology.orchestra.run.smithi101.stdout:(116/131): pcre2-devel-10.32-2.el8.x86_64.rpm 10 MB/s | 605 kB 00:00 2023-07-18T02:54:28.457 INFO:teuthology.orchestra.run.smithi155.stdout:(99/131): postgresql-contrib-10.21-2.module+el8 9.5 MB/s | 808 kB 00:00 2023-07-18T02:54:28.485 INFO:teuthology.orchestra.run.smithi101.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.5 MB/s | 238 kB 00:00 2023-07-18T02:54:28.519 INFO:teuthology.orchestra.run.smithi101.stdout:(118/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 2.1 MB/s | 72 kB 00:00 2023-07-18T02:54:28.541 INFO:teuthology.orchestra.run.smithi155.stdout:(100/131): ceph-mgr-diskprediction-local-18.0.0 995 kB/s | 7.4 MB 00:07 2023-07-18T02:54:28.577 INFO:teuthology.orchestra.run.smithi101.stdout:(119/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 19 MB/s | 2.3 MB 00:00 2023-07-18T02:54:28.591 INFO:teuthology.orchestra.run.smithi155.stdout:(101/131): postgresql-10.21-2.module+el8.6.0+15 11 MB/s | 1.5 MB 00:00 2023-07-18T02:54:28.603 INFO:teuthology.orchestra.run.smithi101.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 987 kB/s | 83 kB 00:00 2023-07-18T02:54:28.641 INFO:teuthology.orchestra.run.smithi155.stdout:(102/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 468 kB/s | 47 kB 00:00 2023-07-18T02:54:28.661 INFO:teuthology.orchestra.run.smithi101.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 716 kB/s | 60 kB 00:00 2023-07-18T02:54:28.687 INFO:teuthology.orchestra.run.smithi101.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.1 MB/s | 98 kB 00:00 2023-07-18T02:54:28.700 INFO:teuthology.orchestra.run.smithi155.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 1.9 MB/s | 200 kB 00:00 2023-07-18T02:54:28.712 INFO:teuthology.orchestra.run.smithi101.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-07-18T02:54:28.726 INFO:teuthology.orchestra.run.smithi155.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 508 kB/s | 43 kB 00:00 2023-07-18T02:54:28.737 INFO:teuthology.orchestra.run.smithi101.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 768 kB/s | 39 kB 00:00 2023-07-18T02:54:28.751 INFO:teuthology.orchestra.run.smithi155.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-07-18T02:54:28.776 INFO:teuthology.orchestra.run.smithi155.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 961 kB/s | 48 kB 00:00 2023-07-18T02:54:28.779 INFO:teuthology.orchestra.run.smithi101.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 9.4 MB/s | 644 kB 00:00 2023-07-18T02:54:28.801 INFO:teuthology.orchestra.run.smithi155.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-07-18T02:54:28.805 INFO:teuthology.orchestra.run.smithi101.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.5 MB/s | 171 kB 00:00 2023-07-18T02:54:28.827 INFO:teuthology.orchestra.run.smithi155.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 849 kB/s | 43 kB 00:00 2023-07-18T02:54:28.830 INFO:teuthology.orchestra.run.smithi101.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-07-18T02:54:28.844 INFO:teuthology.orchestra.run.smithi155.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.5 MB/s | 109 kB 00:00 2023-07-18T02:54:28.869 INFO:teuthology.orchestra.run.smithi155.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 5.3 MB/s | 229 kB 00:00 2023-07-18T02:54:28.902 INFO:teuthology.orchestra.run.smithi155.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.3 MB/s | 560 kB 00:00 2023-07-18T02:54:28.961 INFO:teuthology.orchestra.run.smithi155.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 203 kB/s | 19 kB 00:00 2023-07-18T02:54:28.987 INFO:teuthology.orchestra.run.smithi155.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 1.8 MB/s | 155 kB 00:00 2023-07-18T02:54:29.029 INFO:teuthology.orchestra.run.smithi155.stdout:(114/131): python3-cryptography-3.2.1-5.el8.x86 8.1 MB/s | 559 kB 00:00 2023-07-18T02:54:29.046 INFO:teuthology.orchestra.run.smithi155.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 304 kB/s | 18 kB 00:00 2023-07-18T02:54:29.079 INFO:teuthology.orchestra.run.smithi155.stdout:(116/131): pcre2-devel-10.32-2.el8.x86_64.rpm 12 MB/s | 605 kB 00:00 2023-07-18T02:54:29.104 INFO:teuthology.orchestra.run.smithi155.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-07-18T02:54:29.146 INFO:teuthology.orchestra.run.smithi155.stdout:(118/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.7 MB/s | 72 kB 00:00 2023-07-18T02:54:29.246 INFO:teuthology.orchestra.run.smithi155.stdout:(119/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 14 MB/s | 2.3 MB 00:00 2023-07-18T02:54:29.272 INFO:teuthology.orchestra.run.smithi155.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 660 kB/s | 83 kB 00:00 2023-07-18T02:54:29.297 INFO:teuthology.orchestra.run.smithi155.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.2 MB/s | 60 kB 00:00 2023-07-18T02:54:29.322 INFO:teuthology.orchestra.run.smithi155.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-07-18T02:54:29.348 INFO:teuthology.orchestra.run.smithi155.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-07-18T02:54:29.373 INFO:teuthology.orchestra.run.smithi155.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 767 kB/s | 39 kB 00:00 2023-07-18T02:54:29.407 INFO:teuthology.orchestra.run.smithi155.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 11 MB/s | 644 kB 00:00 2023-07-18T02:54:29.432 INFO:teuthology.orchestra.run.smithi155.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.9 MB/s | 171 kB 00:00 2023-07-18T02:54:29.499 INFO:teuthology.orchestra.run.smithi155.stdout:(127/131): NetworkManager-initscripts-updown-1. 1.5 MB/s | 138 kB 00:00 2023-07-18T02:54:30.404 INFO:teuthology.orchestra.run.smithi179.stdout:(128/131): librbd1-18.0.0-4746.g81cf785a.el8.x8 1.5 MB/s | 3.6 MB 00:02 2023-07-18T02:54:30.462 INFO:teuthology.orchestra.run.smithi179.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-07-18T02:54:30.621 INFO:teuthology.orchestra.run.smithi179.stdout:(130/131): librados2-18.0.0-4746.g81cf785a.el8. 1.4 MB/s | 3.7 MB 00:02 2023-07-18T02:54:31.097 INFO:teuthology.orchestra.run.smithi101.stdout:(128/131): librados2-18.0.0-4746.g81cf785a.el8. 1.6 MB/s | 3.7 MB 00:02 2023-07-18T02:54:31.147 INFO:teuthology.orchestra.run.smithi101.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-07-18T02:54:31.277 INFO:teuthology.orchestra.run.smithi101.stdout:(130/131): librbd1-18.0.0-4746.g81cf785a.el8.x8 1.5 MB/s | 3.6 MB 00:02 2023-07-18T02:54:31.832 INFO:teuthology.orchestra.run.smithi155.stdout:(128/131): librbd1-18.0.0-4746.g81cf785a.el8.x8 1.5 MB/s | 3.6 MB 00:02 2023-07-18T02:54:31.891 INFO:teuthology.orchestra.run.smithi155.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-07-18T02:54:31.991 INFO:teuthology.orchestra.run.smithi155.stdout:(130/131): librados2-18.0.0-4746.g81cf785a.el8. 1.4 MB/s | 3.7 MB 00:02 2023-07-18T02:54:36.349 INFO:teuthology.orchestra.run.smithi155.stdout:(131/131): ceph-test-18.0.0-4746.g81cf785a.el8. 2.0 MB/s | 48 MB 00:23 2023-07-18T02:54:36.352 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2023-07-18T02:54:36.353 INFO:teuthology.orchestra.run.smithi155.stdout:Total 4.1 MB/s | 205 MB 00:50 2023-07-18T02:54:37.145 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2023-07-18T02:54:37.264 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2023-07-18T02:54:37.264 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2023-07-18T02:54:38.391 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2023-07-18T02:54:38.395 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2023-07-18T02:54:39.314 INFO:teuthology.orchestra.run.smithi101.stdout:(131/131): ceph-test-18.0.0-4746.g81cf785a.el8. 1.7 MB/s | 48 MB 00:28 2023-07-18T02:54:39.317 INFO:teuthology.orchestra.run.smithi101.stdout:-------------------------------------------------------------------------------- 2023-07-18T02:54:39.317 INFO:teuthology.orchestra.run.smithi101.stdout:Total 3.8 MB/s | 205 MB 00:53 2023-07-18T02:54:40.943 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction check 2023-07-18T02:54:41.077 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction check succeeded. 2023-07-18T02:54:41.078 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction test 2023-07-18T02:54:41.145 INFO:teuthology.orchestra.run.smithi179.stdout:(131/131): ceph-test-18.0.0-4746.g81cf785a.el8. 1.7 MB/s | 48 MB 00:27 2023-07-18T02:54:41.148 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2023-07-18T02:54:41.148 INFO:teuthology.orchestra.run.smithi179.stdout:Total 3.6 MB/s | 205 MB 00:57 2023-07-18T02:54:41.979 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2023-07-18T02:54:42.099 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2023-07-18T02:54:42.099 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2023-07-18T02:54:42.669 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction test succeeded. 2023-07-18T02:54:42.674 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction 2023-07-18T02:54:43.216 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2023-07-18T02:54:43.221 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2023-07-18T02:54:43.675 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2023-07-18T02:54:43.888 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-18T02:54:44.127 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-18T02:54:44.381 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-07-18T02:54:44.598 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-18T02:54:44.709 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-18T02:54:44.982 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-18T02:54:45.433 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-18T02:54:45.681 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 6/134 2023-07-18T02:54:46.066 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 6/134 2023-07-18T02:54:46.290 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 7/134 2023-07-18T02:54:46.399 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 7/134 2023-07-18T02:54:46.581 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 8/134 2023-07-18T02:54:46.704 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 8/134 2023-07-18T02:54:46.949 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rados-2:18.0.0-4746.g81cf785a.el8.x86_64 9/134 2023-07-18T02:54:47.227 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-18T02:54:47.501 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-18T02:54:47.765 INFO:teuthology.orchestra.run.smithi101.stdout: Preparing : 1/1 2023-07-18T02:54:47.965 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-18T02:54:48.007 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-18T02:54:48.045 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2023-07-18T02:54:48.197 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-18T02:54:48.228 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-18T02:54:48.268 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-18T02:54:48.362 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-18T02:54:48.406 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 14/134 2023-07-18T02:54:48.510 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-07-18T02:54:48.542 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 14/134 2023-07-18T02:54:48.597 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-07-18T02:54:48.753 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86 15/134 2023-07-18T02:54:48.758 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-18T02:54:48.766 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-18T02:54:48.840 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86 15/134 2023-07-18T02:54:48.864 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-18T02:54:48.908 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-18T02:54:49.009 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-18T02:54:49.158 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-18T02:54:49.166 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-18T02:54:49.208 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-18T02:54:49.372 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-18T02:54:49.559 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-18T02:54:49.677 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-18T02:54:49.783 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 6/134 2023-07-18T02:54:49.975 INFO:teuthology.orchestra.run.smithi101.stdout: Upgrading : librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 6/134 2023-07-18T02:54:49.991 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-18T02:54:50.163 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 6/134 2023-07-18T02:54:50.231 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-18T02:54:50.408 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 7/134 2023-07-18T02:54:50.433 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 6/134 2023-07-18T02:54:50.522 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 7/134 2023-07-18T02:54:50.677 INFO:teuthology.orchestra.run.smithi101.stdout: Upgrading : librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 7/134 2023-07-18T02:54:50.732 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 8/134 2023-07-18T02:54:50.813 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 7/134 2023-07-18T02:54:50.827 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 8/134 2023-07-18T02:54:50.875 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-18T02:54:51.036 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-rados-2:18.0.0-4746.g81cf785a.el8.x86_64 9/134 2023-07-18T02:54:51.075 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 8/134 2023-07-18T02:54:51.130 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-07-18T02:54:51.176 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 8/134 2023-07-18T02:54:51.294 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-18T02:54:51.455 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-rados-2:18.0.0-4746.g81cf785a.el8.x86_64 9/134 2023-07-18T02:54:51.527 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-18T02:54:51.748 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-18T02:54:51.837 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-07-18T02:54:51.915 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-18T02:54:51.997 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-18T02:54:52.134 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-07-18T02:54:52.169 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-18T02:54:52.316 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 14/134 2023-07-18T02:54:52.373 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-07-18T02:54:52.417 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 14/134 2023-07-18T02:54:52.453 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-18T02:54:52.648 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86 15/134 2023-07-18T02:54:52.653 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-07-18T02:54:52.712 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-18T02:54:52.734 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86 15/134 2023-07-18T02:54:52.865 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-18T02:54:52.869 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-07-18T02:54:52.941 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 14/134 2023-07-18T02:54:53.004 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-18T02:54:53.066 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 14/134 2023-07-18T02:54:53.174 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-18T02:54:53.316 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86 15/134 2023-07-18T02:54:53.403 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86 15/134 2023-07-18T02:54:53.438 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-07-18T02:54:53.557 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-18T02:54:53.700 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-18T02:54:53.759 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-18T02:54:53.790 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-18T02:54:53.903 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-ceph-common-2:18.0.0-4746.g81cf785a.el8. 26/134 2023-07-18T02:54:53.999 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-18T02:54:53.999 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-18T02:54:54.092 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-ceph-argparse-2:18.0.0-4746.g81cf785a.el 27/134 2023-07-18T02:54:54.581 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-18T02:54:54.700 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-18T02:54:54.782 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-07-18T02:54:54.907 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-18T02:54:55.066 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cephfs-2:18.0.0-4746.g81cf785a.el8.x86_6 28/134 2023-07-18T02:54:55.420 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-07-18T02:54:55.461 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-07-18T02:54:55.574 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-18T02:54:55.730 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-07-18T02:54:55.741 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-07-18T02:54:55.799 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-07-18T02:54:55.966 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-07-18T02:54:56.059 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-07-18T02:54:56.235 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-07-18T02:54:56.313 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-18T02:54:56.429 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-07-18T02:54:56.655 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-07-18T02:54:56.664 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-07-18T02:54:56.908 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-18T02:54:56.979 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-07-18T02:54:57.105 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-ceph-common-2:18.0.0-4746.g81cf785a.el8. 26/134 2023-07-18T02:54:57.242 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-ceph-argparse-2:18.0.0-4746.g81cf785a.el 27/134 2023-07-18T02:54:57.276 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-07-18T02:54:57.539 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-07-18T02:54:57.799 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-07-18T02:54:58.072 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-18T02:54:58.093 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-07-18T02:54:58.203 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cephfs-2:18.0.0-4746.g81cf785a.el8.x86_6 28/134 2023-07-18T02:54:58.428 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-18T02:54:58.504 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-07-18T02:54:58.519 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-07-18T02:54:58.665 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-ceph-common-2:18.0.0-4746.g81cf785a.el8. 26/134 2023-07-18T02:54:58.784 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-18T02:54:58.824 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-07-18T02:54:58.838 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-ceph-argparse-2:18.0.0-4746.g81cf785a.el 27/134 2023-07-18T02:54:58.982 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-18T02:54:59.093 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-07-18T02:54:59.202 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rbd-2:18.0.0-4746.g81cf785a.el8.x86_64 36/134 2023-07-18T02:54:59.323 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-18T02:54:59.450 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librados-devel-2:18.0.0-4746.g81cf785a.el8.x86_6 37/134 2023-07-18T02:54:59.687 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-07-18T02:54:59.915 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-07-18T02:54:59.959 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-cephfs-2:18.0.0-4746.g81cf785a.el8.x86_6 28/134 2023-07-18T02:55:00.157 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-07-18T02:55:00.308 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-07-18T02:55:00.360 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-07-18T02:55:00.587 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-07-18T02:55:00.679 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-07-18T02:55:00.810 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-07-18T02:55:01.032 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-07-18T02:55:01.075 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-18T02:55:01.245 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-07-18T02:55:01.298 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-18T02:55:01.407 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-07-18T02:55:01.565 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-07-18T02:55:01.642 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-18T02:55:01.823 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-07-18T02:55:01.840 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-18T02:55:02.099 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-07-18T02:55:02.103 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-rbd-2:18.0.0-4746.g81cf785a.el8.x86_64 36/134 2023-07-18T02:55:02.292 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : librados-devel-2:18.0.0-4746.g81cf785a.el8.x86_6 37/134 2023-07-18T02:55:02.419 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-07-18T02:55:02.454 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-07-18T02:55:02.632 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-07-18T02:55:02.808 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-07-18T02:55:02.813 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-07-18T02:55:02.976 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-07-18T02:55:03.087 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-07-18T02:55:03.238 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-07-18T02:55:03.355 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-18T02:55:03.382 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-07-18T02:55:03.453 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-07-18T02:55:03.637 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-07-18T02:55:03.765 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-07-18T02:55:03.843 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-07-18T02:55:03.868 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-07-18T02:55:04.058 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-07-18T02:55:04.068 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-18T02:55:04.108 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-07-18T02:55:04.304 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-18T02:55:04.323 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-07-18T02:55:04.386 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-07-18T02:55:04.531 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-rbd-2:18.0.0-4746.g81cf785a.el8.x86_64 36/134 2023-07-18T02:55:04.556 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-07-18T02:55:04.703 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-07-18T02:55:04.803 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : librados-devel-2:18.0.0-4746.g81cf785a.el8.x86_6 37/134 2023-07-18T02:55:04.826 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-07-18T02:55:04.991 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-07-18T02:55:05.049 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-07-18T02:55:05.179 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-07-18T02:55:05.261 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-07-18T02:55:05.262 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-07-18T02:55:05.413 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-07-18T02:55:05.469 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-07-18T02:55:05.534 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-07-18T02:55:05.622 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-07-18T02:55:05.643 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-07-18T02:55:05.780 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-07-18T02:55:05.845 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-07-18T02:55:05.869 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-07-18T02:55:05.946 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-07-18T02:55:06.055 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-07-18T02:55:06.060 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-07-18T02:55:06.209 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-07-18T02:55:06.217 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-07-18T02:55:06.346 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-07-18T02:55:06.463 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-07-18T02:55:06.525 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-07-18T02:55:06.653 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-18T02:55:06.766 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-18T02:55:06.819 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-07-18T02:55:06.820 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-07-18T02:55:07.041 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-07-18T02:55:07.093 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-07-18T02:55:07.171 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-07-18T02:55:07.271 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-07-18T02:55:07.345 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-07-18T02:55:07.452 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-18T02:55:07.454 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-07-18T02:55:07.520 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-07-18T02:55:07.681 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-07-18T02:55:07.697 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-18T02:55:07.774 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-07-18T02:55:07.888 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-07-18T02:55:07.954 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-07-18T02:55:08.004 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-18T02:55:08.124 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-07-18T02:55:08.233 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-07-18T02:55:08.310 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-18T02:55:08.441 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-07-18T02:55:08.467 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-18T02:55:08.516 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-07-18T02:55:08.596 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-18T02:55:08.672 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-18T02:55:08.675 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-18T02:55:08.757 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-07-18T02:55:08.943 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-07-18T02:55:08.984 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-07-18T02:55:08.988 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-18T02:55:09.139 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-07-18T02:55:09.266 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-18T02:55:09.282 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-07-18T02:55:09.322 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-18T02:55:09.507 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-18T02:55:09.510 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-18T02:55:09.521 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-07-18T02:55:09.715 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-18T02:55:09.744 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-07-18T02:55:09.787 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-18T02:55:09.906 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-18T02:55:10.071 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-18T02:55:10.126 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-07-18T02:55:10.166 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-18T02:55:10.243 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-18T02:55:10.424 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-18T02:55:10.457 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-18T02:55:10.501 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-07-18T02:55:10.631 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-18T02:55:10.650 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-18T02:55:10.853 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-07-18T02:55:10.875 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-18T02:55:10.958 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-18T02:55:11.058 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-18T02:55:11.147 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-07-18T02:55:11.235 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-18T02:55:11.274 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-18T02:55:11.356 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-07-18T02:55:11.436 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-18T02:55:11.601 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-07-18T02:55:11.711 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-18T02:55:11.785 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-18T02:55:11.838 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-07-18T02:55:11.942 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-07-18T02:55:12.035 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-18T02:55:12.217 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-07-18T02:55:12.283 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-18T02:55:12.469 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-07-18T02:55:12.527 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-18T02:55:12.697 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-18T02:55:12.784 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-18T02:55:12.797 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-18T02:55:12.869 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-18T02:55:13.028 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-18T02:55:13.082 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-07-18T02:55:13.206 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-18T02:55:13.219 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-18T02:55:13.307 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-07-18T02:55:13.471 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-18T02:55:13.519 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-18T02:55:13.531 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-18T02:55:13.785 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-18T02:55:13.896 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-18T02:55:14.114 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-18T02:55:14.325 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-18T02:55:14.596 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-18T02:55:14.615 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-18T02:55:14.756 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-18T02:55:14.916 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-18T02:55:14.978 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-18T02:55:15.177 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-18T02:55:15.178 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-18T02:55:15.462 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-18T02:55:15.465 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-18T02:55:15.521 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-18T02:55:15.660 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 92/134 2023-07-18T02:55:15.698 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-18T02:55:15.718 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 92/134 2023-07-18T02:55:15.806 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-18T02:55:15.944 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rgw-2:18.0.0-4746.g81cf785a.el8.x86_64 93/134 2023-07-18T02:55:15.972 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-18T02:55:16.080 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-18T02:55:16.175 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-18T02:55:16.329 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-18T02:55:16.589 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-18T02:55:16.828 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-18T02:55:16.894 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-18T02:55:17.070 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-18T02:55:17.136 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 92/134 2023-07-18T02:55:17.191 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 92/134 2023-07-18T02:55:17.395 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-rgw-2:18.0.0-4746.g81cf785a.el8.x86_64 93/134 2023-07-18T02:55:17.396 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-18T02:55:17.647 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-18T02:55:17.840 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-18T02:55:18.123 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-18T02:55:19.482 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-18T02:55:19.790 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-18T02:55:19.836 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 94/134 2023-07-18T02:55:20.126 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 94/134 2023-07-18T02:55:20.296 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-18T02:55:20.687 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-18T02:55:20.721 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 94/134 2023-07-18T02:55:20.972 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-18T02:55:21.022 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 94/134 2023-07-18T02:55:21.070 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 95/134 2023-07-18T02:55:21.156 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 95/134 2023-07-18T02:55:21.207 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-18T02:55:21.295 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 94/134 2023-07-18T02:55:21.359 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 96/134 2023-07-18T02:55:21.875 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 94/134 2023-07-18T02:55:22.106 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-18T02:55:22.328 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 92/134 2023-07-18T02:55:22.418 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 92/134 2023-07-18T02:55:22.646 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-rgw-2:18.0.0-4746.g81cf785a.el8.x86_64 93/134 2023-07-18T02:55:22.778 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 95/134 2023-07-18T02:55:22.882 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 95/134 2023-07-18T02:55:23.036 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 96/134 2023-07-18T02:55:27.449 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 94/134 2023-07-18T02:55:27.728 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 94/134 2023-07-18T02:55:28.422 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 94/134 2023-07-18T02:55:28.753 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 95/134 2023-07-18T02:55:28.888 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 95/134 2023-07-18T02:55:29.044 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 96/134 2023-07-18T02:55:40.095 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 96/134 2023-07-18T02:55:40.095 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /sys 2023-07-18T02:55:40.095 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /proc 2023-07-18T02:55:40.095 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /dev 2023-07-18T02:55:40.095 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /run 2023-07-18T02:55:40.096 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /mnt 2023-07-18T02:55:40.096 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /var/tmp 2023-07-18T02:55:40.096 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /home 2023-07-18T02:55:40.096 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /tmp 2023-07-18T02:55:40.096 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /dev 2023-07-18T02:55:40.096 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-07-18T02:55:40.498 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 97/134 2023-07-18T02:55:42.083 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 96/134 2023-07-18T02:55:42.083 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /sys 2023-07-18T02:55:42.083 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /proc 2023-07-18T02:55:42.083 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /dev 2023-07-18T02:55:42.083 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /run 2023-07-18T02:55:42.083 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /mnt 2023-07-18T02:55:42.084 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /var/tmp 2023-07-18T02:55:42.084 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /home 2023-07-18T02:55:42.084 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /tmp 2023-07-18T02:55:42.084 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /dev 2023-07-18T02:55:42.084 INFO:teuthology.orchestra.run.smithi179.stdout: 2023-07-18T02:55:42.085 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 97/134 2023-07-18T02:55:42.315 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 98/134 2023-07-18T02:55:42.450 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 97/134 2023-07-18T02:55:42.534 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 98/134 2023-07-18T02:55:42.764 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 99/134 2023-07-18T02:55:43.263 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 99/134 2023-07-18T02:55:43.422 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 100/134 2023-07-18T02:55:43.439 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 100/134 2023-07-18T02:55:44.022 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 97/134 2023-07-18T02:55:44.234 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 98/134 2023-07-18T02:55:44.452 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 98/134 2023-07-18T02:55:44.683 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 99/134 2023-07-18T02:55:45.139 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 99/134 2023-07-18T02:55:45.215 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 101/134 2023-07-18T02:55:45.309 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 100/134 2023-07-18T02:55:45.325 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 100/134 2023-07-18T02:55:45.458 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 101/134 2023-07-18T02:55:45.586 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4746.g81cf785a.e 102/134 2023-07-18T02:55:45.844 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4746.g81cf785a. 103/134 2023-07-18T02:55:46.082 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noa 104/134 2023-07-18T02:55:46.293 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noa 104/134 2023-07-18T02:55:46.539 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.n 105/134 2023-07-18T02:55:47.050 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 101/134 2023-07-18T02:55:47.269 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 101/134 2023-07-18T02:55:47.413 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4746.g81cf785a.e 102/134 2023-07-18T02:55:47.500 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.n 105/134 2023-07-18T02:55:47.621 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4746.g81cf785a. 103/134 2023-07-18T02:55:47.713 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4746.g81c 106/134 2023-07-18T02:55:47.804 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4746.g81c 106/134 2023-07-18T02:55:47.832 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noa 104/134 2023-07-18T02:55:48.047 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noa 104/134 2023-07-18T02:55:48.233 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4746.g81cf785a.el 107/134 2023-07-18T02:55:48.315 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.n 105/134 2023-07-18T02:55:48.451 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 108/134 2023-07-18T02:55:48.486 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 108/134 2023-07-18T02:55:48.646 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 109/134 2023-07-18T02:55:48.657 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 109/134 2023-07-18T02:55:49.275 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.n 105/134 2023-07-18T02:55:49.432 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4746.g81c 106/134 2023-07-18T02:55:49.521 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4746.g81c 106/134 2023-07-18T02:55:49.696 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 96/134 2023-07-18T02:55:49.697 INFO:teuthology.orchestra.run.smithi101.stdout:skipping the directory /sys 2023-07-18T02:55:49.697 INFO:teuthology.orchestra.run.smithi101.stdout:skipping the directory /proc 2023-07-18T02:55:49.697 INFO:teuthology.orchestra.run.smithi101.stdout:skipping the directory /dev 2023-07-18T02:55:49.697 INFO:teuthology.orchestra.run.smithi101.stdout:skipping the directory /run 2023-07-18T02:55:49.697 INFO:teuthology.orchestra.run.smithi101.stdout:skipping the directory /mnt 2023-07-18T02:55:49.698 INFO:teuthology.orchestra.run.smithi101.stdout:skipping the directory /var/tmp 2023-07-18T02:55:49.698 INFO:teuthology.orchestra.run.smithi101.stdout:skipping the directory /home 2023-07-18T02:55:49.698 INFO:teuthology.orchestra.run.smithi101.stdout:skipping the directory /tmp 2023-07-18T02:55:49.698 INFO:teuthology.orchestra.run.smithi101.stdout:skipping the directory /dev 2023-07-18T02:55:49.698 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T02:55:49.893 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4746.g81cf785a.el 107/134 2023-07-18T02:55:50.004 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-2:18.0.0-4746.g81cf785a.el8.x86_64 110/134 2023-07-18T02:55:50.125 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 97/134 2023-07-18T02:55:50.155 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 108/134 2023-07-18T02:55:50.190 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 108/134 2023-07-18T02:55:50.252 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 111/134 2023-07-18T02:55:50.291 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 111/134 2023-07-18T02:55:50.374 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 109/134 2023-07-18T02:55:50.385 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 109/134 2023-07-18T02:55:50.488 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4746.g81cf7 112/134 2023-07-18T02:55:50.529 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4746.g81cf7 112/134 2023-07-18T02:55:50.721 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 113/134 2023-07-18T02:55:51.067 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 113/134 2023-07-18T02:55:51.288 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 114/134 2023-07-18T02:55:51.697 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-2:18.0.0-4746.g81cf785a.el8.x86_64 110/134 2023-07-18T02:55:51.864 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 97/134 2023-07-18T02:55:51.918 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 111/134 2023-07-18T02:55:51.961 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 111/134 2023-07-18T02:55:52.074 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 98/134 2023-07-18T02:55:52.148 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4746.g81cf7 112/134 2023-07-18T02:55:52.190 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4746.g81cf7 112/134 2023-07-18T02:55:52.342 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 98/134 2023-07-18T02:55:52.365 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 113/134 2023-07-18T02:55:52.600 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 99/134 2023-07-18T02:55:52.710 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 113/134 2023-07-18T02:55:52.882 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 114/134 2023-07-18T02:55:53.134 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 99/134 2023-07-18T02:55:53.408 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 100/134 2023-07-18T02:55:53.449 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 100/134 2023-07-18T02:55:55.844 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 114/134 2023-07-18T02:55:56.129 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 101/134 2023-07-18T02:55:56.179 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-test-2:18.0.0-4746.g81cf785a.el8.x86_64 115/134 2023-07-18T02:55:56.377 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 101/134 2023-07-18T02:55:56.395 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-07-18T02:55:56.515 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4746.g81cf785a.e 102/134 2023-07-18T02:55:56.586 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-07-18T02:55:56.771 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4746.g81cf785a. 103/134 2023-07-18T02:55:56.982 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noa 104/134 2023-07-18T02:55:57.241 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noa 104/134 2023-07-18T02:55:57.416 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 114/134 2023-07-18T02:55:57.490 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.n 105/134 2023-07-18T02:55:57.542 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-07-18T02:55:57.719 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-test-2:18.0.0-4746.g81cf785a.el8.x86_64 115/134 2023-07-18T02:55:57.880 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-07-18T02:55:57.973 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-07-18T02:55:58.070 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-07-18T02:55:58.198 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 119/134 2023-07-18T02:55:58.449 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-18T02:55:58.608 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.n 105/134 2023-07-18T02:55:58.624 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephfs-devel-2:18.0.0-4746.g81cf785a.el8.x86_ 121/134 2023-07-18T02:55:58.782 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4746.g81c 106/134 2023-07-18T02:55:58.846 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-18T02:55:58.909 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4746.g81c 106/134 2023-07-18T02:55:58.982 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-07-18T02:55:59.312 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-07-18T02:55:59.331 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4746.g81cf785a.el 107/134 2023-07-18T02:55:59.525 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 119/134 2023-07-18T02:55:59.562 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 108/134 2023-07-18T02:55:59.626 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 108/134 2023-07-18T02:55:59.709 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-18T02:55:59.873 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 109/134 2023-07-18T02:55:59.901 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libcephfs-devel-2:18.0.0-4746.g81cf785a.el8.x86_ 121/134 2023-07-18T02:55:59.901 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 109/134 2023-07-18T02:56:00.073 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-18T02:56:01.070 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-18T02:56:01.325 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-2:18.0.0-4746.g81cf785a.el8.x86_64 110/134 2023-07-18T02:56:01.386 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-18T02:56:01.403 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-18T02:56:01.607 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 111/134 2023-07-18T02:56:01.636 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cephfs-top-2:18.0.0-4746.g81cf785a.el8.noarch 124/134 2023-07-18T02:56:01.675 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 111/134 2023-07-18T02:56:01.857 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4746.g81cf7 112/134 2023-07-18T02:56:01.927 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4746.g81cf7 112/134 2023-07-18T02:56:01.969 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 125/134 2023-07-18T02:56:02.173 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 113/134 2023-07-18T02:56:02.275 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-18T02:56:02.341 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-nbd-2:18.0.0-4746.g81cf785a.el8.x86_64 126/134 2023-07-18T02:56:02.539 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-18T02:56:02.556 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-18T02:56:02.581 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 113/134 2023-07-18T02:56:02.730 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : cephfs-top-2:18.0.0-4746.g81cf785a.el8.noarch 124/134 2023-07-18T02:56:02.746 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-18T02:56:02.790 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 114/134 2023-07-18T02:56:02.886 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-18T02:56:02.894 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-18T02:56:03.054 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : rbd-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 125/134 2023-07-18T02:56:03.133 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-07-18T02:56:03.369 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : rbd-nbd-2:18.0.0-4746.g81cf785a.el8.x86_64 126/134 2023-07-18T02:56:03.377 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-07-18T02:56:03.469 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-07-18T02:56:03.715 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-18T02:56:03.786 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-07-18T02:56:03.799 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-07-18T02:56:03.799 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-18T02:56:03.855 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-18T02:56:03.863 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-18T02:56:04.081 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-18T02:56:04.081 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-18T02:56:04.084 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-07-18T02:56:04.271 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-07-18T02:56:04.352 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-07-18T02:56:04.392 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-18T02:56:04.530 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-07-18T02:56:04.542 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-07-18T02:56:04.543 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-18T02:56:04.676 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-18T02:56:04.749 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-18T02:56:04.750 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-18T02:56:05.010 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-18T02:56:05.190 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-18T02:56:05.190 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-2:18.0.0-4746.g81cf785a.el8.x86_64 1/134 2023-07-18T02:56:05.190 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 2/134 2023-07-18T02:56:05.190 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 3/134 2023-07-18T02:56:05.190 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 4/134 2023-07-18T02:56:05.190 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4746.g81cf7 5/134 2023-07-18T02:56:05.191 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 6/134 2023-07-18T02:56:05.191 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 7/134 2023-07-18T02:56:05.191 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 8/134 2023-07-18T02:56:05.191 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 9/134 2023-07-18T02:56:05.191 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 10/134 2023-07-18T02:56:05.191 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 11/134 2023-07-18T02:56:05.191 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-test-2:18.0.0-4746.g81cf785a.el8.x86_64 12/134 2023-07-18T02:56:05.192 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 13/134 2023-07-18T02:56:05.192 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephfs-devel-2:18.0.0-4746.g81cf785a.el8.x86_ 14/134 2023-07-18T02:56:05.192 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 15/134 2023-07-18T02:56:05.192 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 16/134 2023-07-18T02:56:05.192 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados-devel-2:18.0.0-4746.g81cf785a.el8.x86_6 17/134 2023-07-18T02:56:05.192 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86 18/134 2023-07-18T02:56:05.192 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 19/134 2023-07-18T02:56:05.193 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4746.g81cf785a.el 20/134 2023-07-18T02:56:05.193 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-ceph-common-2:18.0.0-4746.g81cf785a.el8. 21/134 2023-07-18T02:56:05.193 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cephfs-2:18.0.0-4746.g81cf785a.el8.x86_6 22/134 2023-07-18T02:56:05.193 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rados-2:18.0.0-4746.g81cf785a.el8.x86_64 23/134 2023-07-18T02:56:05.193 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rbd-2:18.0.0-4746.g81cf785a.el8.x86_64 24/134 2023-07-18T02:56:05.193 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rgw-2:18.0.0-4746.g81cf785a.el8.x86_64 25/134 2023-07-18T02:56:05.193 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 26/134 2023-07-18T02:56:05.193 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 27/134 2023-07-18T02:56:05.194 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-nbd-2:18.0.0-4746.g81cf785a.el8.x86_64 28/134 2023-07-18T02:56:05.194 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4746.g81cf785a. 29/134 2023-07-18T02:56:05.194 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noa 30/134 2023-07-18T02:56:05.194 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.n 31/134 2023-07-18T02:56:05.194 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4746.g81c 32/134 2023-07-18T02:56:05.194 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4746.g81cf785a.el 33/134 2023-07-18T02:56:05.194 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 34/134 2023-07-18T02:56:05.195 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4746.g81cf785a.e 35/134 2023-07-18T02:56:05.195 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 36/134 2023-07-18T02:56:05.195 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 37/134 2023-07-18T02:56:05.195 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cephfs-top-2:18.0.0-4746.g81cf785a.el8.noarch 38/134 2023-07-18T02:56:05.195 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-18T02:56:05.195 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-18T02:56:05.195 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-18T02:56:05.196 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-18T02:56:05.196 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-18T02:56:05.196 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-18T02:56:05.196 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-18T02:56:05.196 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-18T02:56:05.196 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-18T02:56:05.196 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-18T02:56:05.196 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-18T02:56:05.197 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-18T02:56:05.197 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-18T02:56:05.197 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-18T02:56:05.197 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-18T02:56:05.197 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-18T02:56:05.198 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-18T02:56:05.198 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-18T02:56:05.198 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-18T02:56:05.198 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-18T02:56:05.198 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-18T02:56:05.198 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-18T02:56:05.199 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-18T02:56:05.199 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-18T02:56:05.199 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-18T02:56:05.199 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-18T02:56:05.199 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-18T02:56:05.199 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-18T02:56:05.199 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-18T02:56:05.200 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-18T02:56:05.200 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-18T02:56:05.200 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-18T02:56:05.200 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-18T02:56:05.200 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-07-18T02:56:05.200 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-07-18T02:56:05.200 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-07-18T02:56:05.201 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-07-18T02:56:05.201 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-07-18T02:56:05.201 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-07-18T02:56:05.201 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-07-18T02:56:05.201 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-07-18T02:56:05.201 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-07-18T02:56:05.201 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-07-18T02:56:05.201 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-07-18T02:56:05.202 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-07-18T02:56:05.202 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-07-18T02:56:05.202 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-07-18T02:56:05.202 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-07-18T02:56:05.202 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-07-18T02:56:05.202 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-07-18T02:56:05.202 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-07-18T02:56:05.203 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-07-18T02:56:05.203 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-07-18T02:56:05.203 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-07-18T02:56:05.203 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-07-18T02:56:05.203 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-07-18T02:56:05.203 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-07-18T02:56:05.203 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-07-18T02:56:05.203 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-07-18T02:56:05.204 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-07-18T02:56:05.204 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-07-18T02:56:05.204 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-07-18T02:56:05.204 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-07-18T02:56:05.204 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-07-18T02:56:05.204 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-07-18T02:56:05.204 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-07-18T02:56:05.205 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-07-18T02:56:05.205 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-07-18T02:56:05.205 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-07-18T02:56:05.205 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-07-18T02:56:05.205 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-07-18T02:56:05.205 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-07-18T02:56:05.205 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-07-18T02:56:05.205 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-07-18T02:56:05.206 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-07-18T02:56:05.206 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-07-18T02:56:05.206 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-07-18T02:56:05.206 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-07-18T02:56:05.206 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-07-18T02:56:05.206 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-07-18T02:56:05.206 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-07-18T02:56:05.207 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-18T02:56:05.207 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-07-18T02:56:05.207 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-18T02:56:05.207 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-07-18T02:56:05.207 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-07-18T02:56:05.207 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-07-18T02:56:05.207 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-07-18T02:56:05.207 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-07-18T02:56:05.208 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-18T02:56:05.208 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 129/134 2023-07-18T02:56:05.208 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-18T02:56:05.208 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 131/134 2023-07-18T02:56:05.208 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-18T02:56:05.208 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-18T02:56:05.270 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-18T02:56:05.787 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-18T02:56:05.787 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-2:18.0.0-4746.g81cf785a.el8.x86_64 1/134 2023-07-18T02:56:05.787 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 2/134 2023-07-18T02:56:05.787 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 3/134 2023-07-18T02:56:05.787 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 4/134 2023-07-18T02:56:05.787 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4746.g81cf7 5/134 2023-07-18T02:56:05.788 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 6/134 2023-07-18T02:56:05.788 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 7/134 2023-07-18T02:56:05.788 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 8/134 2023-07-18T02:56:05.788 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 9/134 2023-07-18T02:56:05.788 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 10/134 2023-07-18T02:56:05.788 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 11/134 2023-07-18T02:56:05.788 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-test-2:18.0.0-4746.g81cf785a.el8.x86_64 12/134 2023-07-18T02:56:05.789 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 13/134 2023-07-18T02:56:05.789 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libcephfs-devel-2:18.0.0-4746.g81cf785a.el8.x86_ 14/134 2023-07-18T02:56:05.789 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 15/134 2023-07-18T02:56:05.789 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 16/134 2023-07-18T02:56:05.789 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librados-devel-2:18.0.0-4746.g81cf785a.el8.x86_6 17/134 2023-07-18T02:56:05.789 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86 18/134 2023-07-18T02:56:05.789 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 19/134 2023-07-18T02:56:05.790 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4746.g81cf785a.el 20/134 2023-07-18T02:56:05.790 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-ceph-common-2:18.0.0-4746.g81cf785a.el8. 21/134 2023-07-18T02:56:05.790 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cephfs-2:18.0.0-4746.g81cf785a.el8.x86_6 22/134 2023-07-18T02:56:05.790 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-rados-2:18.0.0-4746.g81cf785a.el8.x86_64 23/134 2023-07-18T02:56:05.790 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-rbd-2:18.0.0-4746.g81cf785a.el8.x86_64 24/134 2023-07-18T02:56:05.790 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-rgw-2:18.0.0-4746.g81cf785a.el8.x86_64 25/134 2023-07-18T02:56:05.790 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : rbd-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 26/134 2023-07-18T02:56:05.790 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 27/134 2023-07-18T02:56:05.791 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : rbd-nbd-2:18.0.0-4746.g81cf785a.el8.x86_64 28/134 2023-07-18T02:56:05.791 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4746.g81cf785a. 29/134 2023-07-18T02:56:05.798 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noa 30/134 2023-07-18T02:56:05.798 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.n 31/134 2023-07-18T02:56:05.799 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4746.g81c 32/134 2023-07-18T02:56:05.799 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4746.g81cf785a.el 33/134 2023-07-18T02:56:05.799 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 34/134 2023-07-18T02:56:05.799 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4746.g81cf785a.e 35/134 2023-07-18T02:56:05.799 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 36/134 2023-07-18T02:56:05.799 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 37/134 2023-07-18T02:56:05.799 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : cephfs-top-2:18.0.0-4746.g81cf785a.el8.noarch 38/134 2023-07-18T02:56:05.800 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-18T02:56:05.800 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-18T02:56:05.800 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-18T02:56:05.800 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-18T02:56:05.800 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-18T02:56:05.800 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-18T02:56:05.800 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-18T02:56:05.800 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-18T02:56:05.801 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-18T02:56:05.801 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-18T02:56:05.801 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-18T02:56:05.801 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-18T02:56:05.801 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-18T02:56:05.801 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-18T02:56:05.801 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-18T02:56:05.801 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-18T02:56:05.802 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-18T02:56:05.802 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-18T02:56:05.802 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-18T02:56:05.802 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-18T02:56:05.802 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-18T02:56:05.802 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-18T02:56:05.802 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-18T02:56:05.803 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-18T02:56:05.803 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-18T02:56:05.803 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-18T02:56:05.803 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-18T02:56:05.803 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-18T02:56:05.803 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-18T02:56:05.804 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-18T02:56:05.804 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-18T02:56:05.804 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-18T02:56:05.804 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-18T02:56:05.804 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-07-18T02:56:05.804 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-07-18T02:56:05.805 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-07-18T02:56:05.805 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-07-18T02:56:05.805 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-07-18T02:56:05.805 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-07-18T02:56:05.805 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-07-18T02:56:05.805 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-07-18T02:56:05.805 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-07-18T02:56:05.806 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-07-18T02:56:05.806 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-07-18T02:56:05.806 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-07-18T02:56:05.806 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-07-18T02:56:05.806 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-07-18T02:56:05.806 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-07-18T02:56:05.806 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-07-18T02:56:05.806 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-07-18T02:56:05.807 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-07-18T02:56:05.807 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-07-18T02:56:05.807 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-07-18T02:56:05.807 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-07-18T02:56:05.807 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-07-18T02:56:05.807 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-07-18T02:56:05.807 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-07-18T02:56:05.808 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-07-18T02:56:05.808 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-07-18T02:56:05.808 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-07-18T02:56:05.808 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-07-18T02:56:05.808 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-07-18T02:56:05.808 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-07-18T02:56:05.808 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-07-18T02:56:05.808 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-07-18T02:56:05.809 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-07-18T02:56:05.809 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-07-18T02:56:05.809 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-07-18T02:56:05.809 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-07-18T02:56:05.809 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-07-18T02:56:05.809 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-07-18T02:56:05.809 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-07-18T02:56:05.810 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-07-18T02:56:05.810 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-07-18T02:56:05.810 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-07-18T02:56:05.810 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-07-18T02:56:05.810 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-07-18T02:56:05.810 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-07-18T02:56:05.810 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-07-18T02:56:05.810 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-07-18T02:56:05.811 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-07-18T02:56:05.811 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-18T02:56:05.811 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-07-18T02:56:05.811 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-18T02:56:05.811 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-07-18T02:56:05.811 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-07-18T02:56:05.811 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-07-18T02:56:05.812 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-07-18T02:56:05.812 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-07-18T02:56:05.812 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-18T02:56:05.812 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 129/134 2023-07-18T02:56:05.812 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-18T02:56:05.812 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 131/134 2023-07-18T02:56:05.812 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-18T02:56:05.813 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-18T02:56:07.769 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 114/134 2023-07-18T02:56:07.903 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-18T02:56:07.903 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:02:33 ago on Tue 18 Jul 2023 02:53:34 AM UTC. 2023-07-18T02:56:08.130 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-test-2:18.0.0-4746.g81cf785a.el8.x86_64 115/134 2023-07-18T02:56:08.259 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-18T02:56:08.259 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:02:36 ago on Tue 18 Jul 2023 02:53:32 AM UTC. 2023-07-18T02:56:08.289 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-07-18T02:56:08.543 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-07-18T02:56:09.800 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-07-18T02:56:10.160 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-07-18T02:56:10.386 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 119/134 2023-07-18T02:56:10.648 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-18T02:56:10.844 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libcephfs-devel-2:18.0.0-4746.g81cf785a.el8.x86_ 121/134 2023-07-18T02:56:11.057 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-18T02:56:14.076 INFO:teuthology.orchestra.run.smithi155.stdout:Installed products updated. 2023-07-18T02:56:14.269 INFO:teuthology.orchestra.run.smithi179.stdout:Installed products updated. 2023-07-18T02:56:14.290 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-07-18T02:56:14.291 INFO:teuthology.orchestra.run.smithi155.stdout:Upgraded: 2023-07-18T02:56:14.291 INFO:teuthology.orchestra.run.smithi155.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-18T02:56:14.291 INFO:teuthology.orchestra.run.smithi155.stdout: librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.291 INFO:teuthology.orchestra.run.smithi155.stdout: librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.291 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2023-07-18T02:56:14.291 INFO:teuthology.orchestra.run.smithi155.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-18T02:56:14.292 INFO:teuthology.orchestra.run.smithi155.stdout: automake-1.16.1-7.el8.noarch 2023-07-18T02:56:14.292 INFO:teuthology.orchestra.run.smithi155.stdout: bison-3.0.4-10.el8.x86_64 2023-07-18T02:56:14.292 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.292 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.292 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.292 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.293 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-grafana-dashboards-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T02:56:14.293 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-immutable-object-cache-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.293 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.293 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.293 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T02:56:14.293 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T02:56:14.293 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T02:56:14.293 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-modules-core-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T02:56:14.294 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T02:56:14.294 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.294 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.294 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-prometheus-alerts-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T02:56:14.294 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.294 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.295 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-test-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.295 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T02:56:14.295 INFO:teuthology.orchestra.run.smithi155.stdout: cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T02:56:14.295 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.295 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-top-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T02:56:14.295 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-18T02:56:14.295 INFO:teuthology.orchestra.run.smithi155.stdout: flex-2.6.1-9.el8.x86_64 2023-07-18T02:56:14.296 INFO:teuthology.orchestra.run.smithi155.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-18T02:56:14.296 INFO:teuthology.orchestra.run.smithi155.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-18T02:56:14.296 INFO:teuthology.orchestra.run.smithi155.stdout: jq-1.6-3.el8.x86_64 2023-07-18T02:56:14.296 INFO:teuthology.orchestra.run.smithi155.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-18T02:56:14.296 INFO:teuthology.orchestra.run.smithi155.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-18T02:56:14.296 INFO:teuthology.orchestra.run.smithi155.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-18T02:56:14.296 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs-devel-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.297 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.297 INFO:teuthology.orchestra.run.smithi155.stdout: libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.297 INFO:teuthology.orchestra.run.smithi155.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-18T02:56:14.297 INFO:teuthology.orchestra.run.smithi155.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-18T02:56:14.297 INFO:teuthology.orchestra.run.smithi155.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-18T02:56:14.297 INFO:teuthology.orchestra.run.smithi155.stdout: libpq-13.5-1.el8.x86_64 2023-07-18T02:56:14.297 INFO:teuthology.orchestra.run.smithi155.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-18T02:56:14.298 INFO:teuthology.orchestra.run.smithi155.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-18T02:56:14.298 INFO:teuthology.orchestra.run.smithi155.stdout: librados-devel-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.298 INFO:teuthology.orchestra.run.smithi155.stdout: libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.298 INFO:teuthology.orchestra.run.smithi155.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-18T02:56:14.298 INFO:teuthology.orchestra.run.smithi155.stdout: librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.298 INFO:teuthology.orchestra.run.smithi155.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-18T02:56:14.299 INFO:teuthology.orchestra.run.smithi155.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-18T02:56:14.299 INFO:teuthology.orchestra.run.smithi155.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-18T02:56:14.299 INFO:teuthology.orchestra.run.smithi155.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-18T02:56:14.299 INFO:teuthology.orchestra.run.smithi155.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-18T02:56:14.299 INFO:teuthology.orchestra.run.smithi155.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-18T02:56:14.299 INFO:teuthology.orchestra.run.smithi155.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-18T02:56:14.299 INFO:teuthology.orchestra.run.smithi155.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-18T02:56:14.300 INFO:teuthology.orchestra.run.smithi155.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-18T02:56:14.300 INFO:teuthology.orchestra.run.smithi155.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-18T02:56:14.300 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-18T02:56:14.300 INFO:teuthology.orchestra.run.smithi155.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-18T02:56:14.301 INFO:teuthology.orchestra.run.smithi155.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-18T02:56:14.301 INFO:teuthology.orchestra.run.smithi155.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-18T02:56:14.301 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-18T02:56:14.302 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-18T02:56:14.302 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-18T02:56:14.302 INFO:teuthology.orchestra.run.smithi155.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-18T02:56:14.302 INFO:teuthology.orchestra.run.smithi155.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-18T02:56:14.303 INFO:teuthology.orchestra.run.smithi155.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-18T02:56:14.303 INFO:teuthology.orchestra.run.smithi155.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-18T02:56:14.303 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-18T02:56:14.304 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-argparse-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.304 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.304 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cephfs-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.305 INFO:teuthology.orchestra.run.smithi155.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-18T02:56:14.305 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-18T02:56:14.305 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-18T02:56:14.305 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-18T02:56:14.306 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-18T02:56:14.306 INFO:teuthology.orchestra.run.smithi155.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-18T02:56:14.306 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-18T02:56:14.307 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-18T02:56:14.307 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-18T02:56:14.307 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-18T02:56:14.307 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-18T02:56:14.308 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-18T02:56:14.308 INFO:teuthology.orchestra.run.smithi155.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-18T02:56:14.308 INFO:teuthology.orchestra.run.smithi155.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-18T02:56:14.309 INFO:teuthology.orchestra.run.smithi155.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-18T02:56:14.309 INFO:teuthology.orchestra.run.smithi155.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-18T02:56:14.309 INFO:teuthology.orchestra.run.smithi155.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-18T02:56:14.309 INFO:teuthology.orchestra.run.smithi155.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-18T02:56:14.309 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-18T02:56:14.310 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-18T02:56:14.310 INFO:teuthology.orchestra.run.smithi155.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-18T02:56:14.310 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-18T02:56:14.310 INFO:teuthology.orchestra.run.smithi155.stdout: python3-portend-2.6-1.el8.noarch 2023-07-18T02:56:14.310 INFO:teuthology.orchestra.run.smithi155.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-18T02:56:14.310 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-18T02:56:14.311 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-18T02:56:14.311 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-18T02:56:14.311 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-18T02:56:14.311 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-18T02:56:14.311 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rados-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.311 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rbd-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.311 INFO:teuthology.orchestra.run.smithi155.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-18T02:56:14.312 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-18T02:56:14.312 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rgw-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.312 INFO:teuthology.orchestra.run.smithi155.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-18T02:56:14.312 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-18T02:56:14.312 INFO:teuthology.orchestra.run.smithi155.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-18T02:56:14.312 INFO:teuthology.orchestra.run.smithi155.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-18T02:56:14.312 INFO:teuthology.orchestra.run.smithi155.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-18T02:56:14.313 INFO:teuthology.orchestra.run.smithi155.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-18T02:56:14.313 INFO:teuthology.orchestra.run.smithi155.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-18T02:56:14.313 INFO:teuthology.orchestra.run.smithi155.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-18T02:56:14.313 INFO:teuthology.orchestra.run.smithi155.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-18T02:56:14.313 INFO:teuthology.orchestra.run.smithi155.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-18T02:56:14.313 INFO:teuthology.orchestra.run.smithi155.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-18T02:56:14.313 INFO:teuthology.orchestra.run.smithi155.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-18T02:56:14.314 INFO:teuthology.orchestra.run.smithi155.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-18T02:56:14.314 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.314 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.314 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-nbd-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.314 INFO:teuthology.orchestra.run.smithi155.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-18T02:56:14.314 INFO:teuthology.orchestra.run.smithi155.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-18T02:56:14.314 INFO:teuthology.orchestra.run.smithi155.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-18T02:56:14.315 INFO:teuthology.orchestra.run.smithi155.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-18T02:56:14.315 INFO:teuthology.orchestra.run.smithi155.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-18T02:56:14.315 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-18T02:56:14.316 INFO:teuthology.orchestra.run.smithi155.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-18T02:56:14.316 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-07-18T02:56:14.316 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2023-07-18T02:56:14.479 INFO:teuthology.orchestra.run.smithi179.stdout: 2023-07-18T02:56:14.480 INFO:teuthology.orchestra.run.smithi179.stdout:Upgraded: 2023-07-18T02:56:14.480 INFO:teuthology.orchestra.run.smithi179.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-18T02:56:14.480 INFO:teuthology.orchestra.run.smithi179.stdout: librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.480 INFO:teuthology.orchestra.run.smithi179.stdout: librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.480 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2023-07-18T02:56:14.481 INFO:teuthology.orchestra.run.smithi179.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-18T02:56:14.481 INFO:teuthology.orchestra.run.smithi179.stdout: automake-1.16.1-7.el8.noarch 2023-07-18T02:56:14.481 INFO:teuthology.orchestra.run.smithi179.stdout: bison-3.0.4-10.el8.x86_64 2023-07-18T02:56:14.481 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.481 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.481 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.481 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.482 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-grafana-dashboards-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T02:56:14.482 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-immutable-object-cache-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.482 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.482 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.482 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T02:56:14.482 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T02:56:14.482 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T02:56:14.483 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-modules-core-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T02:56:14.483 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T02:56:14.483 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.483 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.483 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-prometheus-alerts-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T02:56:14.484 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.484 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.484 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-test-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.484 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T02:56:14.485 INFO:teuthology.orchestra.run.smithi179.stdout: cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T02:56:14.485 INFO:teuthology.orchestra.run.smithi179.stdout: cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.485 INFO:teuthology.orchestra.run.smithi179.stdout: cephfs-top-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T02:56:14.485 INFO:teuthology.orchestra.run.smithi179.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-18T02:56:14.486 INFO:teuthology.orchestra.run.smithi179.stdout: flex-2.6.1-9.el8.x86_64 2023-07-18T02:56:14.486 INFO:teuthology.orchestra.run.smithi179.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-18T02:56:14.486 INFO:teuthology.orchestra.run.smithi179.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-18T02:56:14.486 INFO:teuthology.orchestra.run.smithi179.stdout: jq-1.6-3.el8.x86_64 2023-07-18T02:56:14.486 INFO:teuthology.orchestra.run.smithi179.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-18T02:56:14.486 INFO:teuthology.orchestra.run.smithi179.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-18T02:56:14.486 INFO:teuthology.orchestra.run.smithi179.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-18T02:56:14.487 INFO:teuthology.orchestra.run.smithi179.stdout: libcephfs-devel-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.487 INFO:teuthology.orchestra.run.smithi179.stdout: libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.487 INFO:teuthology.orchestra.run.smithi179.stdout: libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.487 INFO:teuthology.orchestra.run.smithi179.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-18T02:56:14.487 INFO:teuthology.orchestra.run.smithi179.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-18T02:56:14.487 INFO:teuthology.orchestra.run.smithi179.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-18T02:56:14.488 INFO:teuthology.orchestra.run.smithi179.stdout: libpq-13.5-1.el8.x86_64 2023-07-18T02:56:14.488 INFO:teuthology.orchestra.run.smithi179.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-18T02:56:14.488 INFO:teuthology.orchestra.run.smithi179.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-18T02:56:14.488 INFO:teuthology.orchestra.run.smithi179.stdout: librados-devel-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.488 INFO:teuthology.orchestra.run.smithi179.stdout: libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.488 INFO:teuthology.orchestra.run.smithi179.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-18T02:56:14.488 INFO:teuthology.orchestra.run.smithi179.stdout: librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.489 INFO:teuthology.orchestra.run.smithi179.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-18T02:56:14.489 INFO:teuthology.orchestra.run.smithi179.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-18T02:56:14.489 INFO:teuthology.orchestra.run.smithi179.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-18T02:56:14.489 INFO:teuthology.orchestra.run.smithi179.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-18T02:56:14.489 INFO:teuthology.orchestra.run.smithi179.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-18T02:56:14.489 INFO:teuthology.orchestra.run.smithi179.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-18T02:56:14.489 INFO:teuthology.orchestra.run.smithi179.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-18T02:56:14.490 INFO:teuthology.orchestra.run.smithi179.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-18T02:56:14.490 INFO:teuthology.orchestra.run.smithi179.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-18T02:56:14.490 INFO:teuthology.orchestra.run.smithi179.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-18T02:56:14.490 INFO:teuthology.orchestra.run.smithi179.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-18T02:56:14.490 INFO:teuthology.orchestra.run.smithi179.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-18T02:56:14.490 INFO:teuthology.orchestra.run.smithi179.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-18T02:56:14.491 INFO:teuthology.orchestra.run.smithi179.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-18T02:56:14.491 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-18T02:56:14.491 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-18T02:56:14.491 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-18T02:56:14.491 INFO:teuthology.orchestra.run.smithi179.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-18T02:56:14.491 INFO:teuthology.orchestra.run.smithi179.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-18T02:56:14.491 INFO:teuthology.orchestra.run.smithi179.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-18T02:56:14.492 INFO:teuthology.orchestra.run.smithi179.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-18T02:56:14.492 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-18T02:56:14.492 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-argparse-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.492 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.492 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cephfs-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.492 INFO:teuthology.orchestra.run.smithi179.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-18T02:56:14.492 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-18T02:56:14.493 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-18T02:56:14.493 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-18T02:56:14.493 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-18T02:56:14.493 INFO:teuthology.orchestra.run.smithi179.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-18T02:56:14.493 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-18T02:56:14.493 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-18T02:56:14.493 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-18T02:56:14.494 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-18T02:56:14.494 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-18T02:56:14.494 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-18T02:56:14.494 INFO:teuthology.orchestra.run.smithi179.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-18T02:56:14.494 INFO:teuthology.orchestra.run.smithi179.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-18T02:56:14.495 INFO:teuthology.orchestra.run.smithi179.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-18T02:56:14.495 INFO:teuthology.orchestra.run.smithi179.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-18T02:56:14.495 INFO:teuthology.orchestra.run.smithi179.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-18T02:56:14.495 INFO:teuthology.orchestra.run.smithi179.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-18T02:56:14.495 INFO:teuthology.orchestra.run.smithi179.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-18T02:56:14.495 INFO:teuthology.orchestra.run.smithi179.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-18T02:56:14.496 INFO:teuthology.orchestra.run.smithi179.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-18T02:56:14.496 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-18T02:56:14.496 INFO:teuthology.orchestra.run.smithi179.stdout: python3-portend-2.6-1.el8.noarch 2023-07-18T02:56:14.496 INFO:teuthology.orchestra.run.smithi179.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-18T02:56:14.496 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-18T02:56:14.496 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-18T02:56:14.497 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-18T02:56:14.497 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-18T02:56:14.497 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-18T02:56:14.497 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rados-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.497 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rbd-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.497 INFO:teuthology.orchestra.run.smithi179.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-18T02:56:14.497 INFO:teuthology.orchestra.run.smithi179.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-18T02:56:14.498 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rgw-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.498 INFO:teuthology.orchestra.run.smithi179.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-18T02:56:14.498 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-18T02:56:14.498 INFO:teuthology.orchestra.run.smithi179.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-18T02:56:14.498 INFO:teuthology.orchestra.run.smithi179.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-18T02:56:14.498 INFO:teuthology.orchestra.run.smithi179.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-18T02:56:14.498 INFO:teuthology.orchestra.run.smithi179.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-18T02:56:14.499 INFO:teuthology.orchestra.run.smithi179.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-18T02:56:14.499 INFO:teuthology.orchestra.run.smithi179.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-18T02:56:14.499 INFO:teuthology.orchestra.run.smithi179.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-18T02:56:14.499 INFO:teuthology.orchestra.run.smithi179.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-18T02:56:14.499 INFO:teuthology.orchestra.run.smithi179.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-18T02:56:14.499 INFO:teuthology.orchestra.run.smithi179.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-18T02:56:14.499 INFO:teuthology.orchestra.run.smithi179.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-18T02:56:14.500 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.500 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.500 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-nbd-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:14.500 INFO:teuthology.orchestra.run.smithi179.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-18T02:56:14.500 INFO:teuthology.orchestra.run.smithi179.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-18T02:56:14.500 INFO:teuthology.orchestra.run.smithi179.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-18T02:56:14.501 INFO:teuthology.orchestra.run.smithi179.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-18T02:56:14.501 INFO:teuthology.orchestra.run.smithi179.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-18T02:56:14.501 INFO:teuthology.orchestra.run.smithi179.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-18T02:56:14.501 INFO:teuthology.orchestra.run.smithi179.stdout: 2023-07-18T02:56:14.502 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2023-07-18T02:56:14.630 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-18T02:56:14.667 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-18T02:56:14.672 DEBUG:teuthology.parallel:result is None 2023-07-18T02:56:14.922 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : cephfs-top-2:18.0.0-4746.g81cf785a.el8.noarch 124/134 2023-07-18T02:56:15.178 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : rbd-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 125/134 2023-07-18T02:56:15.522 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : rbd-nbd-2:18.0.0-4746.g81cf785a.el8.x86_64 126/134 2023-07-18T02:56:15.840 INFO:teuthology.orchestra.run.smithi101.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-18T02:56:16.063 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-18T02:56:16.085 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-18T02:56:16.126 DEBUG:teuthology.parallel:result is None 2023-07-18T02:56:16.371 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-07-18T02:56:16.554 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-07-18T02:56:16.690 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-07-18T02:56:16.904 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-07-18T02:56:16.932 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-07-18T02:56:16.933 INFO:teuthology.orchestra.run.smithi101.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-18T02:56:17.186 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-18T02:56:17.186 INFO:teuthology.orchestra.run.smithi101.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-18T02:56:17.487 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-18T02:56:17.679 INFO:teuthology.orchestra.run.smithi101.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-18T02:56:17.953 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-18T02:56:17.953 INFO:teuthology.orchestra.run.smithi101.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-07-18T02:56:17.953 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T02:56:18.587 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-2:18.0.0-4746.g81cf785a.el8.x86_64 1/134 2023-07-18T02:56:18.588 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 2/134 2023-07-18T02:56:18.588 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 3/134 2023-07-18T02:56:18.588 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 4/134 2023-07-18T02:56:18.588 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4746.g81cf7 5/134 2023-07-18T02:56:18.588 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 6/134 2023-07-18T02:56:18.588 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 7/134 2023-07-18T02:56:18.588 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 8/134 2023-07-18T02:56:18.589 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 9/134 2023-07-18T02:56:18.589 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 10/134 2023-07-18T02:56:18.589 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 11/134 2023-07-18T02:56:18.589 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-test-2:18.0.0-4746.g81cf785a.el8.x86_64 12/134 2023-07-18T02:56:18.589 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 13/134 2023-07-18T02:56:18.589 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libcephfs-devel-2:18.0.0-4746.g81cf785a.el8.x86_ 14/134 2023-07-18T02:56:18.589 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 15/134 2023-07-18T02:56:18.589 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 16/134 2023-07-18T02:56:18.590 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : librados-devel-2:18.0.0-4746.g81cf785a.el8.x86_6 17/134 2023-07-18T02:56:18.590 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86 18/134 2023-07-18T02:56:18.591 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 19/134 2023-07-18T02:56:18.591 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4746.g81cf785a.el 20/134 2023-07-18T02:56:18.591 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-ceph-common-2:18.0.0-4746.g81cf785a.el8. 21/134 2023-07-18T02:56:18.591 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-cephfs-2:18.0.0-4746.g81cf785a.el8.x86_6 22/134 2023-07-18T02:56:18.591 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-rados-2:18.0.0-4746.g81cf785a.el8.x86_64 23/134 2023-07-18T02:56:18.592 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-rbd-2:18.0.0-4746.g81cf785a.el8.x86_64 24/134 2023-07-18T02:56:18.592 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-rgw-2:18.0.0-4746.g81cf785a.el8.x86_64 25/134 2023-07-18T02:56:18.592 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : rbd-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 26/134 2023-07-18T02:56:18.592 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 27/134 2023-07-18T02:56:18.592 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : rbd-nbd-2:18.0.0-4746.g81cf785a.el8.x86_64 28/134 2023-07-18T02:56:18.592 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4746.g81cf785a. 29/134 2023-07-18T02:56:18.599 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noa 30/134 2023-07-18T02:56:18.599 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.n 31/134 2023-07-18T02:56:18.599 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4746.g81c 32/134 2023-07-18T02:56:18.599 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4746.g81cf785a.el 33/134 2023-07-18T02:56:18.599 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 34/134 2023-07-18T02:56:18.600 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4746.g81cf785a.e 35/134 2023-07-18T02:56:18.600 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 36/134 2023-07-18T02:56:18.600 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 37/134 2023-07-18T02:56:18.600 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : cephfs-top-2:18.0.0-4746.g81cf785a.el8.noarch 38/134 2023-07-18T02:56:18.600 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-18T02:56:18.600 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-18T02:56:18.600 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-18T02:56:18.601 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-18T02:56:18.601 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-18T02:56:18.601 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-18T02:56:18.601 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-18T02:56:18.601 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-18T02:56:18.601 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-18T02:56:18.601 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-18T02:56:18.602 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-18T02:56:18.602 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-18T02:56:18.602 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-18T02:56:18.602 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-18T02:56:18.602 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-18T02:56:18.602 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-18T02:56:18.602 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-18T02:56:18.602 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-18T02:56:18.603 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-18T02:56:18.603 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-18T02:56:18.603 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-18T02:56:18.603 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-18T02:56:18.603 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-18T02:56:18.603 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-18T02:56:18.603 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-18T02:56:18.604 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-18T02:56:18.604 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-18T02:56:18.604 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-18T02:56:18.604 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-18T02:56:18.604 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-18T02:56:18.604 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-18T02:56:18.604 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-18T02:56:18.604 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-18T02:56:18.605 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-07-18T02:56:18.605 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-07-18T02:56:18.605 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-07-18T02:56:18.605 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-07-18T02:56:18.605 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-07-18T02:56:18.605 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-07-18T02:56:18.605 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-07-18T02:56:18.605 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-07-18T02:56:18.606 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-07-18T02:56:18.606 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-07-18T02:56:18.606 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-07-18T02:56:18.606 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-07-18T02:56:18.606 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-07-18T02:56:18.606 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-07-18T02:56:18.606 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-07-18T02:56:18.607 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-07-18T02:56:18.607 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-07-18T02:56:18.607 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-07-18T02:56:18.607 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-07-18T02:56:18.607 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-07-18T02:56:18.607 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-07-18T02:56:18.607 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-07-18T02:56:18.607 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-07-18T02:56:18.608 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-07-18T02:56:18.608 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-07-18T02:56:18.608 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-07-18T02:56:18.608 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-07-18T02:56:18.608 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-07-18T02:56:18.608 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-07-18T02:56:18.608 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-07-18T02:56:18.609 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-07-18T02:56:18.609 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-07-18T02:56:18.609 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-07-18T02:56:18.609 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-07-18T02:56:18.609 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-07-18T02:56:18.609 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-07-18T02:56:18.609 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-07-18T02:56:18.609 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-07-18T02:56:18.610 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-07-18T02:56:18.610 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-07-18T02:56:18.610 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-07-18T02:56:18.610 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-07-18T02:56:18.610 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-07-18T02:56:18.610 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-07-18T02:56:18.610 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-07-18T02:56:18.611 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-07-18T02:56:18.611 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-07-18T02:56:18.611 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-07-18T02:56:18.611 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-18T02:56:18.611 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-07-18T02:56:18.611 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-18T02:56:18.611 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-07-18T02:56:18.611 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-07-18T02:56:18.612 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-07-18T02:56:18.612 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-07-18T02:56:18.612 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-07-18T02:56:18.612 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-18T02:56:18.612 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 129/134 2023-07-18T02:56:18.612 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-18T02:56:18.612 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 131/134 2023-07-18T02:56:18.612 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-18T02:56:18.613 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-18T02:56:21.393 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-18T02:56:21.394 INFO:teuthology.orchestra.run.smithi101.stdout:Last metadata expiration check: 0:02:48 ago on Tue 18 Jul 2023 02:53:33 AM UTC. 2023-07-18T02:56:26.118 INFO:teuthology.orchestra.run.smithi101.stdout:Installed products updated. 2023-07-18T02:56:26.358 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T02:56:26.358 INFO:teuthology.orchestra.run.smithi101.stdout:Upgraded: 2023-07-18T02:56:26.358 INFO:teuthology.orchestra.run.smithi101.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-18T02:56:26.358 INFO:teuthology.orchestra.run.smithi101.stdout: librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:26.359 INFO:teuthology.orchestra.run.smithi101.stdout: librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:26.359 INFO:teuthology.orchestra.run.smithi101.stdout:Installed: 2023-07-18T02:56:26.359 INFO:teuthology.orchestra.run.smithi101.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-18T02:56:26.359 INFO:teuthology.orchestra.run.smithi101.stdout: automake-1.16.1-7.el8.noarch 2023-07-18T02:56:26.359 INFO:teuthology.orchestra.run.smithi101.stdout: bison-3.0.4-10.el8.x86_64 2023-07-18T02:56:26.359 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:26.359 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:26.360 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:26.360 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:26.360 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-grafana-dashboards-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T02:56:26.360 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-immutable-object-cache-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:26.360 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:26.360 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:26.360 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T02:56:26.361 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T02:56:26.361 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T02:56:26.361 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-modules-core-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T02:56:26.361 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T02:56:26.361 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:26.361 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:26.361 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-prometheus-alerts-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T02:56:26.362 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:26.362 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:26.362 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-test-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:26.362 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T02:56:26.362 INFO:teuthology.orchestra.run.smithi101.stdout: cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T02:56:26.362 INFO:teuthology.orchestra.run.smithi101.stdout: cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:26.362 INFO:teuthology.orchestra.run.smithi101.stdout: cephfs-top-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T02:56:26.362 INFO:teuthology.orchestra.run.smithi101.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-18T02:56:26.363 INFO:teuthology.orchestra.run.smithi101.stdout: flex-2.6.1-9.el8.x86_64 2023-07-18T02:56:26.363 INFO:teuthology.orchestra.run.smithi101.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-18T02:56:26.363 INFO:teuthology.orchestra.run.smithi101.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-18T02:56:26.363 INFO:teuthology.orchestra.run.smithi101.stdout: jq-1.6-3.el8.x86_64 2023-07-18T02:56:26.363 INFO:teuthology.orchestra.run.smithi101.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-18T02:56:26.363 INFO:teuthology.orchestra.run.smithi101.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-18T02:56:26.363 INFO:teuthology.orchestra.run.smithi101.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-18T02:56:26.364 INFO:teuthology.orchestra.run.smithi101.stdout: libcephfs-devel-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:26.364 INFO:teuthology.orchestra.run.smithi101.stdout: libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:26.364 INFO:teuthology.orchestra.run.smithi101.stdout: libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:26.364 INFO:teuthology.orchestra.run.smithi101.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-18T02:56:26.364 INFO:teuthology.orchestra.run.smithi101.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-18T02:56:26.364 INFO:teuthology.orchestra.run.smithi101.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-18T02:56:26.364 INFO:teuthology.orchestra.run.smithi101.stdout: libpq-13.5-1.el8.x86_64 2023-07-18T02:56:26.365 INFO:teuthology.orchestra.run.smithi101.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-18T02:56:26.365 INFO:teuthology.orchestra.run.smithi101.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-18T02:56:26.365 INFO:teuthology.orchestra.run.smithi101.stdout: librados-devel-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:26.365 INFO:teuthology.orchestra.run.smithi101.stdout: libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:26.365 INFO:teuthology.orchestra.run.smithi101.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-18T02:56:26.365 INFO:teuthology.orchestra.run.smithi101.stdout: librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:26.365 INFO:teuthology.orchestra.run.smithi101.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-18T02:56:26.365 INFO:teuthology.orchestra.run.smithi101.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-18T02:56:26.366 INFO:teuthology.orchestra.run.smithi101.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-18T02:56:26.366 INFO:teuthology.orchestra.run.smithi101.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-18T02:56:26.366 INFO:teuthology.orchestra.run.smithi101.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-18T02:56:26.366 INFO:teuthology.orchestra.run.smithi101.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-18T02:56:26.366 INFO:teuthology.orchestra.run.smithi101.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-18T02:56:26.366 INFO:teuthology.orchestra.run.smithi101.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-18T02:56:26.366 INFO:teuthology.orchestra.run.smithi101.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-18T02:56:26.367 INFO:teuthology.orchestra.run.smithi101.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-18T02:56:26.367 INFO:teuthology.orchestra.run.smithi101.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-18T02:56:26.367 INFO:teuthology.orchestra.run.smithi101.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-18T02:56:26.367 INFO:teuthology.orchestra.run.smithi101.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-18T02:56:26.367 INFO:teuthology.orchestra.run.smithi101.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-18T02:56:26.367 INFO:teuthology.orchestra.run.smithi101.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-18T02:56:26.367 INFO:teuthology.orchestra.run.smithi101.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-18T02:56:26.367 INFO:teuthology.orchestra.run.smithi101.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-18T02:56:26.368 INFO:teuthology.orchestra.run.smithi101.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-18T02:56:26.368 INFO:teuthology.orchestra.run.smithi101.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-18T02:56:26.368 INFO:teuthology.orchestra.run.smithi101.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-18T02:56:26.368 INFO:teuthology.orchestra.run.smithi101.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-18T02:56:26.368 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-18T02:56:26.368 INFO:teuthology.orchestra.run.smithi101.stdout: python3-ceph-argparse-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:26.368 INFO:teuthology.orchestra.run.smithi101.stdout: python3-ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:26.369 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cephfs-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:26.369 INFO:teuthology.orchestra.run.smithi101.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-18T02:56:26.369 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-18T02:56:26.369 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-18T02:56:26.369 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-18T02:56:26.369 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-18T02:56:26.369 INFO:teuthology.orchestra.run.smithi101.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-18T02:56:26.369 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-18T02:56:26.370 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-18T02:56:26.370 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-18T02:56:26.370 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-18T02:56:26.370 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-18T02:56:26.370 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-18T02:56:26.370 INFO:teuthology.orchestra.run.smithi101.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-18T02:56:26.370 INFO:teuthology.orchestra.run.smithi101.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-18T02:56:26.370 INFO:teuthology.orchestra.run.smithi101.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-18T02:56:26.371 INFO:teuthology.orchestra.run.smithi101.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-18T02:56:26.371 INFO:teuthology.orchestra.run.smithi101.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-18T02:56:26.371 INFO:teuthology.orchestra.run.smithi101.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-18T02:56:26.371 INFO:teuthology.orchestra.run.smithi101.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-18T02:56:26.371 INFO:teuthology.orchestra.run.smithi101.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-18T02:56:26.371 INFO:teuthology.orchestra.run.smithi101.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-18T02:56:26.371 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-18T02:56:26.372 INFO:teuthology.orchestra.run.smithi101.stdout: python3-portend-2.6-1.el8.noarch 2023-07-18T02:56:26.372 INFO:teuthology.orchestra.run.smithi101.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-18T02:56:26.372 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-18T02:56:26.372 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-18T02:56:26.372 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-18T02:56:26.372 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-18T02:56:26.372 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-18T02:56:26.372 INFO:teuthology.orchestra.run.smithi101.stdout: python3-rados-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:26.373 INFO:teuthology.orchestra.run.smithi101.stdout: python3-rbd-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:26.373 INFO:teuthology.orchestra.run.smithi101.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-18T02:56:26.373 INFO:teuthology.orchestra.run.smithi101.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-18T02:56:26.373 INFO:teuthology.orchestra.run.smithi101.stdout: python3-rgw-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:26.373 INFO:teuthology.orchestra.run.smithi101.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-18T02:56:26.373 INFO:teuthology.orchestra.run.smithi101.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-18T02:56:26.373 INFO:teuthology.orchestra.run.smithi101.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-18T02:56:26.374 INFO:teuthology.orchestra.run.smithi101.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-18T02:56:26.374 INFO:teuthology.orchestra.run.smithi101.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-18T02:56:26.374 INFO:teuthology.orchestra.run.smithi101.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-18T02:56:26.374 INFO:teuthology.orchestra.run.smithi101.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-18T02:56:26.374 INFO:teuthology.orchestra.run.smithi101.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-18T02:56:26.374 INFO:teuthology.orchestra.run.smithi101.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-18T02:56:26.374 INFO:teuthology.orchestra.run.smithi101.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-18T02:56:26.374 INFO:teuthology.orchestra.run.smithi101.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-18T02:56:26.375 INFO:teuthology.orchestra.run.smithi101.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-18T02:56:26.375 INFO:teuthology.orchestra.run.smithi101.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-18T02:56:26.375 INFO:teuthology.orchestra.run.smithi101.stdout: rbd-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:26.375 INFO:teuthology.orchestra.run.smithi101.stdout: rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:26.375 INFO:teuthology.orchestra.run.smithi101.stdout: rbd-nbd-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T02:56:26.375 INFO:teuthology.orchestra.run.smithi101.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-18T02:56:26.375 INFO:teuthology.orchestra.run.smithi101.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-18T02:56:26.375 INFO:teuthology.orchestra.run.smithi101.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-18T02:56:26.376 INFO:teuthology.orchestra.run.smithi101.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-18T02:56:26.376 INFO:teuthology.orchestra.run.smithi101.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-18T02:56:26.376 INFO:teuthology.orchestra.run.smithi101.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-18T02:56:26.376 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T02:56:26.377 INFO:teuthology.orchestra.run.smithi101.stdout:Complete! 2023-07-18T02:56:26.720 DEBUG:teuthology.parallel:result is None 2023-07-18T02:56:26.721 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-18T02:56:27.075 DEBUG:teuthology.orchestra.run.smithi101:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-18T02:56:27.130 INFO:teuthology.orchestra.run.smithi101.stdout:18.0.0-4746.g81cf785a.el8 2023-07-18T02:56:27.131 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4746.g81cf785a.el8 2023-07-18T02:56:27.131 INFO:teuthology.task.install:The correct ceph version 18.0.0-4746.g81cf785a is installed. 2023-07-18T02:56:27.132 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-18T02:56:27.454 DEBUG:teuthology.orchestra.run.smithi155:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-18T02:56:27.482 INFO:teuthology.orchestra.run.smithi155.stdout:18.0.0-4746.g81cf785a.el8 2023-07-18T02:56:27.483 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4746.g81cf785a.el8 2023-07-18T02:56:27.483 INFO:teuthology.task.install:The correct ceph version 18.0.0-4746.g81cf785a is installed. 2023-07-18T02:56:27.483 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-18T02:56:27.743 DEBUG:teuthology.orchestra.run.smithi179:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-18T02:56:27.772 INFO:teuthology.orchestra.run.smithi179.stdout:18.0.0-4746.g81cf785a.el8 2023-07-18T02:56:27.772 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4746.g81cf785a.el8 2023-07-18T02:56:27.772 INFO:teuthology.task.install:The correct ceph version 18.0.0-4746.g81cf785a is installed. 2023-07-18T02:56:27.773 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-07-18T02:56:27.773 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-07-18T02:56:27.774 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-18T02:56:27.862 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-07-18T02:56:27.862 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-18T02:56:27.909 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2023-07-18T02:56:27.910 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-18T02:56:27.957 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-07-18T02:56:27.958 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-07-18T02:56:27.958 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd of=/usr/bin/daemon-helper 2023-07-18T02:56:28.026 DEBUG:teuthology.orchestra.run.smithi101:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-18T02:56:28.133 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-07-18T02:56:28.133 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/daemon-helper 2023-07-18T02:56:28.165 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-18T02:56:28.235 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2023-07-18T02:56:28.235 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/usr/bin/daemon-helper 2023-07-18T02:56:28.267 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-18T02:56:28.340 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-07-18T02:56:28.340 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-07-18T02:56:28.341 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-18T02:56:28.409 DEBUG:teuthology.orchestra.run.smithi101:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-18T02:56:28.515 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-07-18T02:56:28.516 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-18T02:56:28.547 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-18T02:56:28.617 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2023-07-18T02:56:28.617 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-18T02:56:28.649 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-18T02:56:28.719 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-07-18T02:56:28.719 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-07-18T02:56:28.720 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd of=/usr/bin/stdin-killer 2023-07-18T02:56:28.788 DEBUG:teuthology.orchestra.run.smithi101:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-18T02:56:28.894 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-07-18T02:56:28.894 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/stdin-killer 2023-07-18T02:56:28.926 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-18T02:56:28.997 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2023-07-18T02:56:28.997 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/usr/bin/stdin-killer 2023-07-18T02:56:29.029 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-18T02:56:29.100 INFO:teuthology.run_tasks:Running task cephadm... 2023-07-18T02:56:29.191 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '81cf785aa7781c4e08381c540f890cb9bc987ef9', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2023-07-18T02:56:29.191 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-18T02:56:29.201 INFO:tasks.cephadm:Cluster fsid is b1782daa-2516-11ee-9b34-001a4aab830c 2023-07-18T02:56:29.202 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-07-18T02:56:29.202 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.101', 'mon.b': '172.21.15.155', 'mon.c': '172.21.15.179'} 2023-07-18T02:56:29.203 INFO:tasks.cephadm:First mon is mon.a on smithi101 2023-07-18T02:56:29.203 INFO:tasks.cephadm:First mgr is x 2023-07-18T02:56:29.203 INFO:tasks.cephadm:Normalizing hostnames... 2023-07-18T02:56:29.204 DEBUG:teuthology.orchestra.run.smithi101:> sudo hostname $(hostname -s) 2023-07-18T02:56:29.276 DEBUG:teuthology.orchestra.run.smithi155:> sudo hostname $(hostname -s) 2023-07-18T02:56:29.309 DEBUG:teuthology.orchestra.run.smithi179:> sudo hostname $(hostname -s) 2023-07-18T02:56:29.342 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-07-18T02:56:29.343 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-18T02:56:29.595 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '81cf785aa7781c4e08381c540f890cb9bc987ef9', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/71509/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-4746-g81cf785a', 'node_name': '172.21.2.5+braggi05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-4746.g81cf785a'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-07-18 00:38:12.274423', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-2023Jul13-m-quota', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2023-07-18T02:56:29.830 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-rishabh-2023Jul13-m-quota, sha1 81cf785aa7781c4e08381c540f890cb9bc987ef9 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-18T02:56:29.830 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/x86_64/flavors/default/cephadm 2023-07-18T02:56:29.831 DEBUG:teuthology.orchestra.run.smithi101:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-07-18T02:56:30.727 INFO:teuthology.orchestra.run.smithi101.stdout:-rw-rw-r--. 1 ubuntu ubuntu 663791 Jul 18 02:56 /home/ubuntu/cephtest/cephadm 2023-07-18T02:56:30.727 DEBUG:teuthology.orchestra.run.smithi155:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-07-18T02:56:31.633 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-rw-r--. 1 ubuntu ubuntu 663791 Jul 18 02:56 /home/ubuntu/cephtest/cephadm 2023-07-18T02:56:31.633 DEBUG:teuthology.orchestra.run.smithi179:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-07-18T02:56:32.506 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-rw-r--. 1 ubuntu ubuntu 663791 Jul 18 02:56 /home/ubuntu/cephtest/cephadm 2023-07-18T02:56:32.506 DEBUG:teuthology.orchestra.run.smithi101:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-07-18T02:56:32.557 DEBUG:teuthology.orchestra.run.smithi155:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-07-18T02:56:32.578 DEBUG:teuthology.orchestra.run.smithi179:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-07-18T02:56:32.613 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 on all hosts... 2023-07-18T02:56:32.613 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 pull 2023-07-18T02:56:32.618 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 pull 2023-07-18T02:56:32.620 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 pull 2023-07-18T02:56:34.401 INFO:teuthology.orchestra.run.smithi179.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9... 2023-07-18T02:56:34.715 INFO:teuthology.orchestra.run.smithi155.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9... 2023-07-18T02:56:36.913 INFO:teuthology.orchestra.run.smithi101.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9... 2023-07-18T02:57:10.696 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2023-07-18T02:57:10.697 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_version": "ceph version 18.0.0-4746-g81cf785a (81cf785aa7781c4e08381c540f890cb9bc987ef9) reef (dev)", 2023-07-18T02:57:10.697 INFO:teuthology.orchestra.run.smithi179.stdout: "image_id": "66a4f7b015b53bd7c8d67b561c2d8f6525dccea2a3d3e0d20f2a1cd17a765421", 2023-07-18T02:57:10.697 INFO:teuthology.orchestra.run.smithi179.stdout: "repo_digests": [ 2023-07-18T02:57:10.697 INFO:teuthology.orchestra.run.smithi179.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b4b7dd677a0f5b8e56a57eadd762824d684980cb80d319543e86b72e549ef9c" 2023-07-18T02:57:10.697 INFO:teuthology.orchestra.run.smithi179.stdout: ] 2023-07-18T02:57:10.697 INFO:teuthology.orchestra.run.smithi179.stdout:} 2023-07-18T02:57:11.090 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T02:57:11.091 INFO:teuthology.orchestra.run.smithi101.stdout: "ceph_version": "ceph version 18.0.0-4746-g81cf785a (81cf785aa7781c4e08381c540f890cb9bc987ef9) reef (dev)", 2023-07-18T02:57:11.091 INFO:teuthology.orchestra.run.smithi101.stdout: "image_id": "66a4f7b015b53bd7c8d67b561c2d8f6525dccea2a3d3e0d20f2a1cd17a765421", 2023-07-18T02:57:11.091 INFO:teuthology.orchestra.run.smithi101.stdout: "repo_digests": [ 2023-07-18T02:57:11.091 INFO:teuthology.orchestra.run.smithi101.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b4b7dd677a0f5b8e56a57eadd762824d684980cb80d319543e86b72e549ef9c" 2023-07-18T02:57:11.091 INFO:teuthology.orchestra.run.smithi101.stdout: ] 2023-07-18T02:57:11.091 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T02:57:11.103 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2023-07-18T02:57:11.103 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_version": "ceph version 18.0.0-4746-g81cf785a (81cf785aa7781c4e08381c540f890cb9bc987ef9) reef (dev)", 2023-07-18T02:57:11.104 INFO:teuthology.orchestra.run.smithi155.stdout: "image_id": "66a4f7b015b53bd7c8d67b561c2d8f6525dccea2a3d3e0d20f2a1cd17a765421", 2023-07-18T02:57:11.104 INFO:teuthology.orchestra.run.smithi155.stdout: "repo_digests": [ 2023-07-18T02:57:11.104 INFO:teuthology.orchestra.run.smithi155.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b4b7dd677a0f5b8e56a57eadd762824d684980cb80d319543e86b72e549ef9c" 2023-07-18T02:57:11.104 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2023-07-18T02:57:11.104 INFO:teuthology.orchestra.run.smithi155.stdout:} 2023-07-18T02:57:11.123 DEBUG:teuthology.orchestra.run.smithi101:> sudo mkdir -p /etc/ceph 2023-07-18T02:57:11.194 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /etc/ceph 2023-07-18T02:57:11.224 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /etc/ceph 2023-07-18T02:57:11.257 DEBUG:teuthology.orchestra.run.smithi101:> sudo chmod 777 /etc/ceph 2023-07-18T02:57:11.323 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 777 /etc/ceph 2023-07-18T02:57:11.351 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 777 /etc/ceph 2023-07-18T02:57:11.379 INFO:tasks.cephadm:Writing seed config... 2023-07-18T02:57:11.380 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-07-18T02:57:11.381 INFO:tasks.cephadm: override: [client] debug client = 20 2023-07-18T02:57:11.381 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-07-18T02:57:11.381 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-07-18T02:57:11.382 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-07-18T02:57:11.382 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-07-18T02:57:11.382 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-07-18T02:57:11.382 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2023-07-18T02:57:11.382 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-07-18T02:57:11.383 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-07-18T02:57:11.383 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-07-18T02:57:11.383 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-07-18T02:57:11.383 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-07-18T02:57:11.383 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-07-18T02:57:11.384 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-07-18T02:57:11.384 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-07-18T02:57:11.384 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-07-18T02:57:11.384 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-07-18T02:57:11.384 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-07-18T02:57:11.385 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-07-18T02:57:11.385 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-07-18T02:57:11.385 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2023-07-18T02:57:11.385 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-07-18T02:57:11.385 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-07-18T02:57:11.386 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-07-18T02:57:11.386 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-07-18T02:57:11.386 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-07-18T02:57:11.386 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-07-18T02:57:11.387 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-07-18T02:57:11.387 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2023-07-18T02:57:11.387 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-07-18T02:57:11.388 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2023-07-18T02:57:11.388 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2023-07-18T02:57:11.388 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-07-18T02:57:11.389 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-07-18T02:57:11.389 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2023-07-18T02:57:11.389 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-07-18T02:57:11.389 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-07-18T02:57:11.389 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-07-18T02:57:11.390 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-07-18T02:57:11.390 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-07-18T02:57:11.390 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-07-18T02:57:11.390 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-07-18T02:57:11.390 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2023-07-18T02:57:11.391 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-07-18T02:57:11.391 DEBUG:teuthology.orchestra.run.smithi101:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-07-18T02:57:11.433 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = b1782daa-2516-11ee-9b34-001a4aab830c ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-07-18T02:57:11.434 DEBUG:teuthology.orchestra.run.smithi101:mon.a> sudo journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mon.a.service 2023-07-18T02:57:11.478 DEBUG:teuthology.orchestra.run.smithi101:mgr.x> sudo journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mgr.x.service 2023-07-18T02:57:11.523 INFO:tasks.cephadm:Bootstrapping... 2023-07-18T02:57:11.524 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 -v bootstrap --fsid b1782daa-2516-11ee-9b34-001a4aab830c --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.101 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-07-18T02:57:11.544 INFO:journalctl@ceph.mon.a.smithi101.stdout:-- Logs begin at Tue 2023-07-18 02:35:16 UTC. -- 2023-07-18T02:57:11.592 INFO:journalctl@ceph.mgr.x.smithi101.stdout:-- Logs begin at Tue 2023-07-18 02:35:16 UTC. -- 2023-07-18T02:57:11.743 INFO:teuthology.orchestra.run.smithi101.stdout:-------------------------------------------------------------------------------- 2023-07-18T02:57:11.743 INFO:teuthology.orchestra.run.smithi101.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9', '-v', 'bootstrap', '--fsid', 'b1782daa-2516-11ee-9b34-001a4aab830c', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.101', '--skip-admin-label'] 2023-07-18T02:57:11.914 INFO:teuthology.orchestra.run.smithi101.stdout:/bin/podman: stdout 4.1.1 2023-07-18T02:57:11.915 INFO:teuthology.orchestra.run.smithi101.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-07-18T02:57:11.915 INFO:teuthology.orchestra.run.smithi101.stdout:Verifying podman|docker is present... 2023-07-18T02:57:12.087 INFO:teuthology.orchestra.run.smithi101.stdout:/bin/podman: stdout 4.1.1 2023-07-18T02:57:12.087 INFO:teuthology.orchestra.run.smithi101.stdout:Verifying lvm2 is present... 2023-07-18T02:57:12.088 INFO:teuthology.orchestra.run.smithi101.stdout:Verifying time synchronization is in place... 2023-07-18T02:57:12.101 INFO:teuthology.orchestra.run.smithi101.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-07-18T02:57:12.102 INFO:teuthology.orchestra.run.smithi101.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-07-18T02:57:12.114 INFO:teuthology.orchestra.run.smithi101.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-07-18T02:57:12.114 INFO:teuthology.orchestra.run.smithi101.stdout:systemctl: stdout inactive 2023-07-18T02:57:12.127 INFO:teuthology.orchestra.run.smithi101.stdout:systemctl: stdout enabled 2023-07-18T02:57:12.140 INFO:teuthology.orchestra.run.smithi101.stdout:systemctl: stdout active 2023-07-18T02:57:12.141 INFO:teuthology.orchestra.run.smithi101.stdout:Unit chronyd.service is enabled and running 2023-07-18T02:57:12.141 INFO:teuthology.orchestra.run.smithi101.stdout:Repeating the final host check... 2023-07-18T02:57:12.309 INFO:teuthology.orchestra.run.smithi101.stdout:/bin/podman: stdout 4.1.1 2023-07-18T02:57:12.309 INFO:teuthology.orchestra.run.smithi101.stdout:podman (/bin/podman) version 4.1.1 is present 2023-07-18T02:57:12.309 INFO:teuthology.orchestra.run.smithi101.stdout:systemctl is present 2023-07-18T02:57:12.309 INFO:teuthology.orchestra.run.smithi101.stdout:lvcreate is present 2023-07-18T02:57:12.322 INFO:teuthology.orchestra.run.smithi101.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-07-18T02:57:12.322 INFO:teuthology.orchestra.run.smithi101.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-07-18T02:57:12.335 INFO:teuthology.orchestra.run.smithi101.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-07-18T02:57:12.335 INFO:teuthology.orchestra.run.smithi101.stdout:systemctl: stdout inactive 2023-07-18T02:57:12.348 INFO:teuthology.orchestra.run.smithi101.stdout:systemctl: stdout enabled 2023-07-18T02:57:12.361 INFO:teuthology.orchestra.run.smithi101.stdout:systemctl: stdout active 2023-07-18T02:57:12.361 INFO:teuthology.orchestra.run.smithi101.stdout:Unit chronyd.service is enabled and running 2023-07-18T02:57:12.361 INFO:teuthology.orchestra.run.smithi101.stdout:Host looks OK 2023-07-18T02:57:12.362 INFO:teuthology.orchestra.run.smithi101.stdout:Cluster fsid: b1782daa-2516-11ee-9b34-001a4aab830c 2023-07-18T02:57:12.362 INFO:teuthology.orchestra.run.smithi101.stdout:Acquiring lock 140129242101744 on /run/cephadm/b1782daa-2516-11ee-9b34-001a4aab830c.lock 2023-07-18T02:57:12.362 INFO:teuthology.orchestra.run.smithi101.stdout:Lock 140129242101744 acquired on /run/cephadm/b1782daa-2516-11ee-9b34-001a4aab830c.lock 2023-07-18T02:57:12.363 INFO:teuthology.orchestra.run.smithi101.stdout:Verifying IP 172.21.15.101 port 3300 ... 2023-07-18T02:57:12.363 INFO:teuthology.orchestra.run.smithi101.stdout:Verifying IP 172.21.15.101 port 6789 ... 2023-07-18T02:57:12.364 INFO:teuthology.orchestra.run.smithi101.stdout:Base mon IP(s) is [172.21.15.101:3300, 172.21.15.101:6789], mon addrv is [v2:172.21.15.101:3300,v1:172.21.15.101:6789] 2023-07-18T02:57:12.370 INFO:teuthology.orchestra.run.smithi101.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp metric 100 2023-07-18T02:57:12.370 INFO:teuthology.orchestra.run.smithi101.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.101 metric 100 2023-07-18T02:57:12.376 INFO:teuthology.orchestra.run.smithi101.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-07-18T02:57:12.377 INFO:teuthology.orchestra.run.smithi101.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2023-07-18T02:57:12.377 INFO:teuthology.orchestra.run.smithi101.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2023-07-18T02:57:12.385 INFO:teuthology.orchestra.run.smithi101.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-07-18T02:57:12.385 INFO:teuthology.orchestra.run.smithi101.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-07-18T02:57:12.386 INFO:teuthology.orchestra.run.smithi101.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-07-18T02:57:12.386 INFO:teuthology.orchestra.run.smithi101.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2023-07-18T02:57:12.386 INFO:teuthology.orchestra.run.smithi101.stdout:/sbin/ip: stdout inet6 fe80::ac5b:f81b:4a3:d684/64 scope link noprefixroute 2023-07-18T02:57:12.386 INFO:teuthology.orchestra.run.smithi101.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-07-18T02:57:12.386 INFO:teuthology.orchestra.run.smithi101.stdout:Mon IP `172.21.15.101` is in CIDR network `172.21.0.0/20` 2023-07-18T02:57:12.386 INFO:teuthology.orchestra.run.smithi101.stdout:Mon IP `172.21.15.101` is in CIDR network `172.21.0.0/20` 2023-07-18T02:57:12.387 INFO:teuthology.orchestra.run.smithi101.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-07-18T02:57:12.387 INFO:teuthology.orchestra.run.smithi101.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-07-18T02:57:12.387 INFO:teuthology.orchestra.run.smithi101.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9... 2023-07-18T02:57:13.159 INFO:teuthology.orchestra.run.smithi101.stdout:/bin/podman: stdout 66a4f7b015b53bd7c8d67b561c2d8f6525dccea2a3d3e0d20f2a1cd17a765421 2023-07-18T02:57:13.160 INFO:teuthology.orchestra.run.smithi101.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9... 2023-07-18T02:57:13.160 INFO:teuthology.orchestra.run.smithi101.stdout:/bin/podman: stderr Getting image source signatures 2023-07-18T02:57:13.160 INFO:teuthology.orchestra.run.smithi101.stdout:/bin/podman: stderr Copying blob sha256:711183bdaeabd1b922e81a2784cd69d11b05a304985585795e1ae029bf69712a 2023-07-18T02:57:13.160 INFO:teuthology.orchestra.run.smithi101.stdout:/bin/podman: stderr Copying blob sha256:6022c611d474f2b306ddd1edfc8360123ece5266119096f25fde843d7967e636 2023-07-18T02:57:13.160 INFO:teuthology.orchestra.run.smithi101.stdout:/bin/podman: stderr Copying config sha256:66a4f7b015b53bd7c8d67b561c2d8f6525dccea2a3d3e0d20f2a1cd17a765421 2023-07-18T02:57:13.160 INFO:teuthology.orchestra.run.smithi101.stdout:/bin/podman: stderr Writing manifest to image destination 2023-07-18T02:57:13.161 INFO:teuthology.orchestra.run.smithi101.stdout:/bin/podman: stderr Storing signatures 2023-07-18T02:57:14.133 INFO:teuthology.orchestra.run.smithi101.stdout:ceph: stdout ceph version 18.0.0-4746-g81cf785a (81cf785aa7781c4e08381c540f890cb9bc987ef9) reef (dev) 2023-07-18T02:57:14.134 INFO:teuthology.orchestra.run.smithi101.stdout:Ceph version: ceph version 18.0.0-4746-g81cf785a (81cf785aa7781c4e08381c540f890cb9bc987ef9) reef (dev) 2023-07-18T02:57:14.134 INFO:teuthology.orchestra.run.smithi101.stdout:Extracting ceph user uid/gid from container image... 2023-07-18T02:57:15.005 INFO:teuthology.orchestra.run.smithi101.stdout:stat: stdout 167 167 2023-07-18T02:57:15.006 INFO:teuthology.orchestra.run.smithi101.stdout:Creating initial keys... 2023-07-18T02:57:15.880 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph-authtool: stdout AQCL/7Vk/ECcGxAACkW/zqjkWQtDCdfMBs676Q== 2023-07-18T02:57:16.853 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph-authtool: stdout AQCM/7Vkpf4kEhAAOstInNC4KKSOk0OpQiAgvA== 2023-07-18T02:57:19.135 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph-authtool: stdout AQCN/7VkhaUfEhAAfh1Cz5tda5TAqP3fBT8zWg== 2023-07-18T02:57:19.137 INFO:teuthology.orchestra.run.smithi101.stdout:Creating initial monmap... 2023-07-18T02:57:19.916 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-07-18T02:57:19.916 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-07-18T02:57:19.917 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to b1782daa-2516-11ee-9b34-001a4aab830c 2023-07-18T02:57:19.917 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-07-18T02:57:19.917 INFO:teuthology.orchestra.run.smithi101.stdout:monmaptool for a [v2:172.21.15.101:3300,v1:172.21.15.101:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-07-18T02:57:19.918 INFO:teuthology.orchestra.run.smithi101.stdout:setting min_mon_release = pacific 2023-07-18T02:57:19.918 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/monmaptool: set fsid to b1782daa-2516-11ee-9b34-001a4aab830c 2023-07-18T02:57:19.918 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-07-18T02:57:19.918 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T02:57:19.918 INFO:teuthology.orchestra.run.smithi101.stdout:Creating mon... 2023-07-18T02:57:20.889 INFO:teuthology.orchestra.run.smithi101.stdout:create mon.a on 2023-07-18T02:57:21.154 INFO:teuthology.orchestra.run.smithi101.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-07-18T02:57:21.355 INFO:teuthology.orchestra.run.smithi101.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-07-18T02:57:21.601 INFO:teuthology.orchestra.run.smithi101.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-b1782daa-2516-11ee-9b34-001a4aab830c.target → /etc/systemd/system/ceph-b1782daa-2516-11ee-9b34-001a4aab830c.target. 2023-07-18T02:57:21.602 INFO:teuthology.orchestra.run.smithi101.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-b1782daa-2516-11ee-9b34-001a4aab830c.target → /etc/systemd/system/ceph-b1782daa-2516-11ee-9b34-001a4aab830c.target. 2023-07-18T02:57:22.008 INFO:teuthology.orchestra.run.smithi101.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mon.a 2023-07-18T02:57:22.008 INFO:teuthology.orchestra.run.smithi101.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mon.a.service: Unit ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mon.a.service not loaded. 2023-07-18T02:57:22.201 INFO:teuthology.orchestra.run.smithi101.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b1782daa-2516-11ee-9b34-001a4aab830c.target.wants/ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mon.a.service → /etc/systemd/system/ceph-b1782daa-2516-11ee-9b34-001a4aab830c@.service. 2023-07-18T02:57:22.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:22 smithi101 systemd[1]: Starting Ceph mon.a for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T02:57:22.947 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:22 smithi101 podman[125824]: 2023-07-18T02:57:22.948 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:22 smithi101 bash[125824]: 6dcae2cee73857411ff1d13ed467e97b9d273c543ab54610f9b75337bfb7c53d 2023-07-18T02:57:23.014 INFO:teuthology.orchestra.run.smithi101.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-18T02:57:23.014 INFO:teuthology.orchestra.run.smithi101.stdout:systemctl: stdout disabled 2023-07-18T02:57:23.029 INFO:teuthology.orchestra.run.smithi101.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-18T02:57:23.030 INFO:teuthology.orchestra.run.smithi101.stdout:systemctl: stdout inactive 2023-07-18T02:57:23.030 INFO:teuthology.orchestra.run.smithi101.stdout:firewalld.service is not enabled 2023-07-18T02:57:23.030 INFO:teuthology.orchestra.run.smithi101.stdout:Not possible to enable service . firewalld.service is not available 2023-07-18T02:57:23.030 INFO:teuthology.orchestra.run.smithi101.stdout:Waiting for mon to start... 2023-07-18T02:57:23.031 INFO:teuthology.orchestra.run.smithi101.stdout:Waiting for mon... 2023-07-18T02:57:23.217 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:22 smithi101 ceph-mon[125865]: mkfs b1782daa-2516-11ee-9b34-001a4aab830c 2023-07-18T02:57:23.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:22 smithi101 ceph-mon[125865]: mon.a is new leader, mons a in quorum (ranks 0) 2023-07-18T02:57:23.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:23 smithi101 systemd[1]: Started Ceph mon.a for b1782daa-2516-11ee-9b34-001a4aab830c. 2023-07-18T02:57:24.115 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout cluster: 2023-07-18T02:57:24.115 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout id: b1782daa-2516-11ee-9b34-001a4aab830c 2023-07-18T02:57:24.115 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-07-18T02:57:24.116 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout 2023-07-18T02:57:24.116 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout services: 2023-07-18T02:57:24.116 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.734358s) 2023-07-18T02:57:24.116 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-07-18T02:57:24.116 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-07-18T02:57:24.116 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout 2023-07-18T02:57:24.116 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout data: 2023-07-18T02:57:24.117 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-07-18T02:57:24.117 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-07-18T02:57:24.117 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-07-18T02:57:24.117 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout pgs: 2023-07-18T02:57:24.117 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout 2023-07-18T02:57:24.117 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:23.624+0000 7f9fa8b7a700 1 Processor -- start 2023-07-18T02:57:24.117 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:23.624+0000 7f9fa8b7a700 1 -- start start 2023-07-18T02:57:24.117 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:23.624+0000 7f9fa8b7a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9fa4105320 0x7f9fa4105730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:57:24.118 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:23.624+0000 7f9fa8b7a700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa4105c70 con 0x7f9fa4105320 2023-07-18T02:57:24.118 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:23.625+0000 7f9fa259c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9fa4105320 0x7f9fa4105730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:57:24.118 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:23.625+0000 7f9fa259c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9fa4105320 0x7f9fa4105730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:56320/0 (socket says 172.21.15.101:56320) 2023-07-18T02:57:24.118 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:23.625+0000 7f9fa259c700 1 -- 172.21.15.101:0/329797285 learned_addr learned my addr 172.21.15.101:0/329797285 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:57:24.118 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:23.625+0000 7f9fa259c700 1 -- 172.21.15.101:0/329797285 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa4105db0 con 0x7f9fa4105320 2023-07-18T02:57:24.118 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:23.625+0000 7f9fa259c700 1 --2- 172.21.15.101:0/329797285 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9fa4105320 0x7f9fa4105730 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9f940098a0 tx=0x7f9f94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ecbe1f9fb940a92 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:57:24.118 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:23.626+0000 7f9fa159a700 1 -- 172.21.15.101:0/329797285 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9f94004160 con 0x7f9fa4105320 2023-07-18T02:57:24.119 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:23.626+0000 7f9fa159a700 1 -- 172.21.15.101:0/329797285 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f940042c0 con 0x7f9fa4105320 2023-07-18T02:57:24.119 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:23.626+0000 7f9fa159a700 1 -- 172.21.15.101:0/329797285 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9f940045a0 con 0x7f9fa4105320 2023-07-18T02:57:24.119 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:23.626+0000 7f9fa8b7a700 1 -- 172.21.15.101:0/329797285 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9fa4105320 msgr2=0x7f9fa4105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:57:24.119 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:23.626+0000 7f9fa8b7a700 1 --2- 172.21.15.101:0/329797285 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9fa4105320 0x7f9fa4105730 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9f940098a0 tx=0x7f9f94009bb0 comp rx=0 tx=0).stop 2023-07-18T02:57:24.119 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:23.626+0000 7f9fa8b7a700 1 -- 172.21.15.101:0/329797285 shutdown_connections 2023-07-18T02:57:24.119 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:23.626+0000 7f9fa8b7a700 1 --2- 172.21.15.101:0/329797285 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9fa4105320 0x7f9fa4105730 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:57:24.119 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:23.626+0000 7f9fa8b7a700 1 -- 172.21.15.101:0/329797285 >> 172.21.15.101:0/329797285 conn(0x7f9fa4100970 msgr2=0x7f9fa4102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:57:24.120 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:23.626+0000 7f9fa8b7a700 1 -- 172.21.15.101:0/329797285 shutdown_connections 2023-07-18T02:57:24.120 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:23.626+0000 7f9fa8b7a700 1 -- 172.21.15.101:0/329797285 wait complete. 2023-07-18T02:57:24.120 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:23.627+0000 7f9fa8b7a700 1 Processor -- start 2023-07-18T02:57:24.120 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:23.627+0000 7f9fa8b7a700 1 -- start start 2023-07-18T02:57:24.120 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:23.628+0000 7f9fa8b7a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9fa4105320 0x7f9fa41988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:57:24.120 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:23.628+0000 7f9fa8b7a700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa4198e00 con 0x7f9fa4105320 2023-07-18T02:57:24.120 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:23.628+0000 7f9fa259c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9fa4105320 0x7f9fa41988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:57:24.121 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:23.628+0000 7f9fa259c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9fa4105320 0x7f9fa41988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:56334/0 (socket says 172.21.15.101:56334) 2023-07-18T02:57:24.121 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:23.628+0000 7f9fa259c700 1 -- 172.21.15.101:0/2749340754 learned_addr learned my addr 172.21.15.101:0/2749340754 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:57:24.121 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:23.628+0000 7f9fa259c700 1 -- 172.21.15.101:0/2749340754 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f94009550 con 0x7f9fa4105320 2023-07-18T02:57:24.121 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:23.629+0000 7f9fa259c700 1 --2- 172.21.15.101:0/2749340754 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9fa4105320 0x7f9fa41988c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4106050 tx=0x7f9f94004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:57:24.121 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:23.629+0000 7f9f937fe700 1 -- 172.21.15.101:0/2749340754 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9f94011630 con 0x7f9fa4105320 2023-07-18T02:57:24.121 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:23.629+0000 7f9f937fe700 1 -- 172.21.15.101:0/2749340754 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f94011790 con 0x7f9fa4105320 2023-07-18T02:57:24.121 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:23.629+0000 7f9f937fe700 1 -- 172.21.15.101:0/2749340754 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9f94011a30 con 0x7f9fa4105320 2023-07-18T02:57:24.121 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:23.629+0000 7f9fa8b7a700 1 -- 172.21.15.101:0/2749340754 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fa4199000 con 0x7f9fa4105320 2023-07-18T02:57:24.122 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:23.629+0000 7f9fa8b7a700 1 -- 172.21.15.101:0/2749340754 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fa4199420 con 0x7f9fa4105320 2023-07-18T02:57:24.122 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:23.629+0000 7f9f937fe700 1 -- 172.21.15.101:0/2749340754 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9f94022070 con 0x7f9fa4105320 2023-07-18T02:57:24.122 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:23.629+0000 7f9f937fe700 1 -- 172.21.15.101:0/2749340754 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9f9401c450 con 0x7f9fa4105320 2023-07-18T02:57:24.122 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:23.630+0000 7f9fa8b7a700 1 -- 172.21.15.101:0/2749340754 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fa41920d0 con 0x7f9fa4105320 2023-07-18T02:57:24.122 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:23.633+0000 7f9f937fe700 1 -- 172.21.15.101:0/2749340754 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f9f9401b450 con 0x7f9fa4105320 2023-07-18T02:57:24.122 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:23.704+0000 7f9fa8b7a700 1 -- 172.21.15.101:0/2749340754 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f9fa402c850 con 0x7f9fa4105320 2023-07-18T02:57:24.122 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:23.704+0000 7f9f937fe700 1 -- 172.21.15.101:0/2749340754 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f9f9401b5f0 con 0x7f9fa4105320 2023-07-18T02:57:24.123 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:23.705+0000 7f9fa8b7a700 1 -- 172.21.15.101:0/2749340754 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9fa4105320 msgr2=0x7f9fa41988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:57:24.124 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:23.705+0000 7f9fa8b7a700 1 --2- 172.21.15.101:0/2749340754 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9fa4105320 0x7f9fa41988c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4106050 tx=0x7f9f94004750 comp rx=0 tx=0).stop 2023-07-18T02:57:24.124 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:23.705+0000 7f9fa8b7a700 1 -- 172.21.15.101:0/2749340754 shutdown_connections 2023-07-18T02:57:24.124 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:23.705+0000 7f9fa8b7a700 1 --2- 172.21.15.101:0/2749340754 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9fa4105320 0x7f9fa41988c0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:57:24.124 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:23.705+0000 7f9fa8b7a700 1 -- 172.21.15.101:0/2749340754 >> 172.21.15.101:0/2749340754 conn(0x7f9fa4100970 msgr2=0x7f9fa418cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:57:24.124 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:23.705+0000 7f9fa8b7a700 1 -- 172.21.15.101:0/2749340754 shutdown_connections 2023-07-18T02:57:24.124 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:23.705+0000 7f9fa8b7a700 1 -- 172.21.15.101:0/2749340754 wait complete. 2023-07-18T02:57:24.124 INFO:teuthology.orchestra.run.smithi101.stdout:mon is available 2023-07-18T02:57:24.125 INFO:teuthology.orchestra.run.smithi101.stdout:Assimilating anything we can from ceph.conf... 2023-07-18T02:57:24.217 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:23 smithi101 ceph-mon[125865]: mon.a is new leader, mons a in quorum (ranks 0) 2023-07-18T02:57:24.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:23 smithi101 ceph-mon[125865]: monmap e1: 1 mons at {a=[v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0]} removed_ranks: {} 2023-07-18T02:57:24.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:23 smithi101 ceph-mon[125865]: fsmap 2023-07-18T02:57:24.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:23 smithi101 ceph-mon[125865]: osdmap e1: 0 total, 0 up, 0 in 2023-07-18T02:57:24.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:23 smithi101 ceph-mon[125865]: mgrmap e1: no daemons active 2023-07-18T02:57:24.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:23 smithi101 ceph-mon[125865]: from='client.? 172.21.15.101:0/2749340754' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-07-18T02:57:25.187 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:24 smithi101 ceph-mon[125865]: from='client.? 172.21.15.101:0/487291256' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-07-18T02:57:25.187 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:24 smithi101 ceph-mon[125865]: from='client.? 172.21.15.101:0/487291256' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-07-18T02:57:25.195 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout 2023-07-18T02:57:25.196 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout [global] 2023-07-18T02:57:25.196 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout fsid = b1782daa-2516-11ee-9b34-001a4aab830c 2023-07-18T02:57:25.196 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.101:3300,v1:172.21.15.101:6789] 2023-07-18T02:57:25.196 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-07-18T02:57:25.196 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-07-18T02:57:25.196 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-07-18T02:57:25.197 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-07-18T02:57:25.197 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout 2023-07-18T02:57:25.197 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout [mgr] 2023-07-18T02:57:25.197 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-07-18T02:57:25.197 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout 2023-07-18T02:57:25.197 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout [osd] 2023-07-18T02:57:25.197 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-07-18T02:57:25.198 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-07-18T02:57:25.198 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-07-18T02:57:25.198 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-07-18T02:57:25.198 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-07-18T02:57:25.198 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.713+0000 7f816600e700 1 Processor -- start 2023-07-18T02:57:25.198 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.714+0000 7f816600e700 1 -- start start 2023-07-18T02:57:25.198 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.714+0000 7f816600e700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8160105320 0x7f8160105730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:57:25.198 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.714+0000 7f816600e700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8160105c70 con 0x7f8160105320 2023-07-18T02:57:25.199 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.715+0000 7f815f7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8160105320 0x7f8160105730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:57:25.199 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.715+0000 7f815f7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8160105320 0x7f8160105730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:56350/0 (socket says 172.21.15.101:56350) 2023-07-18T02:57:25.199 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.715+0000 7f815f7fe700 1 -- 172.21.15.101:0/1532138464 learned_addr learned my addr 172.21.15.101:0/1532138464 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:57:25.199 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.715+0000 7f815f7fe700 1 -- 172.21.15.101:0/1532138464 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8160105db0 con 0x7f8160105320 2023-07-18T02:57:25.199 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.715+0000 7f815f7fe700 1 --2- 172.21.15.101:0/1532138464 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8160105320 0x7f8160105730 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f81500098a0 tx=0x7f8150009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a84e7864ce6343d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:57:25.199 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.716+0000 7f815e7fc700 1 -- 172.21.15.101:0/1532138464 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8150004160 con 0x7f8160105320 2023-07-18T02:57:25.199 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.716+0000 7f815e7fc700 1 -- 172.21.15.101:0/1532138464 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81500042c0 con 0x7f8160105320 2023-07-18T02:57:25.200 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.716+0000 7f815e7fc700 1 -- 172.21.15.101:0/1532138464 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f81500045a0 con 0x7f8160105320 2023-07-18T02:57:25.200 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.716+0000 7f816600e700 1 -- 172.21.15.101:0/1532138464 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8160105320 msgr2=0x7f8160105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:57:25.200 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.716+0000 7f816600e700 1 --2- 172.21.15.101:0/1532138464 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8160105320 0x7f8160105730 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f81500098a0 tx=0x7f8150009bb0 comp rx=0 tx=0).stop 2023-07-18T02:57:25.200 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.716+0000 7f816600e700 1 -- 172.21.15.101:0/1532138464 shutdown_connections 2023-07-18T02:57:25.200 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.716+0000 7f816600e700 1 --2- 172.21.15.101:0/1532138464 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8160105320 0x7f8160105730 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:57:25.200 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.716+0000 7f816600e700 1 -- 172.21.15.101:0/1532138464 >> 172.21.15.101:0/1532138464 conn(0x7f8160100970 msgr2=0x7f8160102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:57:25.201 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.716+0000 7f816600e700 1 -- 172.21.15.101:0/1532138464 shutdown_connections 2023-07-18T02:57:25.201 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.716+0000 7f816600e700 1 -- 172.21.15.101:0/1532138464 wait complete. 2023-07-18T02:57:25.201 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.717+0000 7f816600e700 1 Processor -- start 2023-07-18T02:57:25.201 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.717+0000 7f816600e700 1 -- start start 2023-07-18T02:57:25.201 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.718+0000 7f816600e700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8160105320 0x7f816019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:57:25.201 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.718+0000 7f816600e700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816019b000 con 0x7f8160105320 2023-07-18T02:57:25.201 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.718+0000 7f815f7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8160105320 0x7f816019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:57:25.201 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.718+0000 7f815f7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8160105320 0x7f816019aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:56366/0 (socket says 172.21.15.101:56366) 2023-07-18T02:57:25.202 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.718+0000 7f815f7fe700 1 -- 172.21.15.101:0/487291256 learned_addr learned my addr 172.21.15.101:0/487291256 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:57:25.202 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.718+0000 7f815f7fe700 1 -- 172.21.15.101:0/487291256 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8150009550 con 0x7f8160105320 2023-07-18T02:57:25.202 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.718+0000 7f815f7fe700 1 --2- 172.21.15.101:0/487291256 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8160105320 0x7f816019aac0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f8160106050 tx=0x7f8150004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:57:25.202 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.719+0000 7f815cff9700 1 -- 172.21.15.101:0/487291256 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8150011630 con 0x7f8160105320 2023-07-18T02:57:25.202 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.719+0000 7f815cff9700 1 -- 172.21.15.101:0/487291256 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8150011790 con 0x7f8160105320 2023-07-18T02:57:25.202 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.719+0000 7f816600e700 1 -- 172.21.15.101:0/487291256 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f816019b200 con 0x7f8160105320 2023-07-18T02:57:25.202 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.719+0000 7f815cff9700 1 -- 172.21.15.101:0/487291256 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8150011a30 con 0x7f8160105320 2023-07-18T02:57:25.203 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.719+0000 7f816600e700 1 -- 172.21.15.101:0/487291256 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f816019b620 con 0x7f8160105320 2023-07-18T02:57:25.203 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.719+0000 7f815cff9700 1 -- 172.21.15.101:0/487291256 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8150022070 con 0x7f8160105320 2023-07-18T02:57:25.203 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.719+0000 7f815cff9700 1 -- 172.21.15.101:0/487291256 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f815001c790 con 0x7f8160105320 2023-07-18T02:57:25.203 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.720+0000 7f816600e700 1 -- 172.21.15.101:0/487291256 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81601944d0 con 0x7f8160105320 2023-07-18T02:57:25.203 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.723+0000 7f815cff9700 1 -- 172.21.15.101:0/487291256 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f815001bcf0 con 0x7f8160105320 2023-07-18T02:57:25.203 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.793+0000 7f816600e700 1 -- 172.21.15.101:0/487291256 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f816002cad0 con 0x7f8160105320 2023-07-18T02:57:25.203 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.797+0000 7f815cff9700 1 -- 172.21.15.101:0/487291256 <== mon.0 v2:172.21.15.101:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f815001be90 con 0x7f8160105320 2023-07-18T02:57:25.203 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.797+0000 7f815cff9700 1 -- 172.21.15.101:0/487291256 <== mon.0 v2:172.21.15.101:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+431 (secure 0 0 0) 0x7f81500245c0 con 0x7f8160105320 2023-07-18T02:57:25.204 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.798+0000 7f816600e700 1 -- 172.21.15.101:0/487291256 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8160105320 msgr2=0x7f816019aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:57:25.204 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.798+0000 7f816600e700 1 --2- 172.21.15.101:0/487291256 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8160105320 0x7f816019aac0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f8160106050 tx=0x7f8150004750 comp rx=0 tx=0).stop 2023-07-18T02:57:25.204 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.798+0000 7f816600e700 1 -- 172.21.15.101:0/487291256 shutdown_connections 2023-07-18T02:57:25.204 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.798+0000 7f816600e700 1 --2- 172.21.15.101:0/487291256 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8160105320 0x7f816019aac0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:57:25.204 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.798+0000 7f816600e700 1 -- 172.21.15.101:0/487291256 >> 172.21.15.101:0/487291256 conn(0x7f8160100970 msgr2=0x7f816018cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:57:25.204 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.799+0000 7f816600e700 1 -- 172.21.15.101:0/487291256 shutdown_connections 2023-07-18T02:57:25.204 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:24.799+0000 7f816600e700 1 -- 172.21.15.101:0/487291256 wait complete. 2023-07-18T02:57:25.205 INFO:teuthology.orchestra.run.smithi101.stdout:Generating new minimal ceph.conf... 2023-07-18T02:57:26.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:25 smithi101 ceph-mon[125865]: from='client.? 172.21.15.101:0/2007544788' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:57:26.270 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:25.774+0000 7fe6e0cc8700 1 Processor -- start 2023-07-18T02:57:26.274 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:25.775+0000 7fe6e0cc8700 1 -- start start 2023-07-18T02:57:26.274 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:25.775+0000 7fe6e0cc8700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe6dc07be00 0x7fe6dc07a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:57:26.274 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:25.775+0000 7fe6e0cc8700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6dc07a840 con 0x7fe6dc07be00 2023-07-18T02:57:26.274 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:25.775+0000 7fe6da59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe6dc07be00 0x7fe6dc07a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:57:26.275 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:25.775+0000 7fe6da59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe6dc07be00 0x7fe6dc07a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:56376/0 (socket says 172.21.15.101:56376) 2023-07-18T02:57:26.275 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:25.775+0000 7fe6da59c700 1 -- 172.21.15.101:0/1465231019 learned_addr learned my addr 172.21.15.101:0/1465231019 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:57:26.275 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:25.776+0000 7fe6da59c700 1 -- 172.21.15.101:0/1465231019 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6dc07a980 con 0x7fe6dc07be00 2023-07-18T02:57:26.275 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:25.776+0000 7fe6da59c700 1 --2- 172.21.15.101:0/1465231019 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe6dc07be00 0x7fe6dc07a300 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc0098a0 tx=0x7fe6cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ea8f14890591fa server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:57:26.275 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:25.776+0000 7fe6d959a700 1 -- 172.21.15.101:0/1465231019 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe6cc004160 con 0x7fe6dc07be00 2023-07-18T02:57:26.275 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:25.776+0000 7fe6d959a700 1 -- 172.21.15.101:0/1465231019 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe6cc0042c0 con 0x7fe6dc07be00 2023-07-18T02:57:26.275 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:25.776+0000 7fe6d959a700 1 -- 172.21.15.101:0/1465231019 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe6cc011a00 con 0x7fe6dc07be00 2023-07-18T02:57:26.275 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:25.777+0000 7fe6e0cc8700 1 -- 172.21.15.101:0/1465231019 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe6dc07be00 msgr2=0x7fe6dc07a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:57:26.276 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:25.777+0000 7fe6e0cc8700 1 --2- 172.21.15.101:0/1465231019 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe6dc07be00 0x7fe6dc07a300 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc0098a0 tx=0x7fe6cc009bb0 comp rx=0 tx=0).stop 2023-07-18T02:57:26.276 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:25.777+0000 7fe6e0cc8700 1 -- 172.21.15.101:0/1465231019 shutdown_connections 2023-07-18T02:57:26.276 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:25.777+0000 7fe6e0cc8700 1 --2- 172.21.15.101:0/1465231019 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe6dc07be00 0x7fe6dc07a300 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:57:26.276 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:25.777+0000 7fe6e0cc8700 1 -- 172.21.15.101:0/1465231019 >> 172.21.15.101:0/1465231019 conn(0x7fe6dc100970 msgr2=0x7fe6dc102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:57:26.276 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:25.777+0000 7fe6e0cc8700 1 -- 172.21.15.101:0/1465231019 shutdown_connections 2023-07-18T02:57:26.276 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:25.777+0000 7fe6e0cc8700 1 -- 172.21.15.101:0/1465231019 wait complete. 2023-07-18T02:57:26.276 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:25.778+0000 7fe6e0cc8700 1 Processor -- start 2023-07-18T02:57:26.277 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:25.778+0000 7fe6e0cc8700 1 -- start start 2023-07-18T02:57:26.277 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:25.778+0000 7fe6e0cc8700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe6dc07be00 0x7fe6dc19b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:57:26.277 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:25.778+0000 7fe6e0cc8700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6dc19b880 con 0x7fe6dc07be00 2023-07-18T02:57:26.277 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:25.779+0000 7fe6da59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe6dc07be00 0x7fe6dc19b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:57:26.277 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:25.779+0000 7fe6da59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe6dc07be00 0x7fe6dc19b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:56392/0 (socket says 172.21.15.101:56392) 2023-07-18T02:57:26.277 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:25.779+0000 7fe6da59c700 1 -- 172.21.15.101:0/2007544788 learned_addr learned my addr 172.21.15.101:0/2007544788 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:57:26.278 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:25.779+0000 7fe6da59c700 1 -- 172.21.15.101:0/2007544788 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6cc009550 con 0x7fe6dc07be00 2023-07-18T02:57:26.278 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:25.779+0000 7fe6da59c700 1 --2- 172.21.15.101:0/2007544788 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe6dc07be00 0x7fe6dc19b340 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc009130 tx=0x7fe6cc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:57:26.278 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:25.779+0000 7fe6cb7fe700 1 -- 172.21.15.101:0/2007544788 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe6cc003ab0 con 0x7fe6dc07be00 2023-07-18T02:57:26.278 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:25.779+0000 7fe6cb7fe700 1 -- 172.21.15.101:0/2007544788 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe6cc003c10 con 0x7fe6dc07be00 2023-07-18T02:57:26.278 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:25.779+0000 7fe6e0cc8700 1 -- 172.21.15.101:0/2007544788 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6dc19ba80 con 0x7fe6dc07be00 2023-07-18T02:57:26.278 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:25.779+0000 7fe6cb7fe700 1 -- 172.21.15.101:0/2007544788 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe6cc0247d0 con 0x7fe6dc07be00 2023-07-18T02:57:26.278 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:25.780+0000 7fe6e0cc8700 1 -- 172.21.15.101:0/2007544788 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6dc19bea0 con 0x7fe6dc07be00 2023-07-18T02:57:26.279 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:25.780+0000 7fe6cb7fe700 1 -- 172.21.15.101:0/2007544788 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe6cc024d50 con 0x7fe6dc07be00 2023-07-18T02:57:26.279 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:25.780+0000 7fe6cb7fe700 1 -- 172.21.15.101:0/2007544788 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe6cc019490 con 0x7fe6dc07be00 2023-07-18T02:57:26.279 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:25.781+0000 7fe6e0cc8700 1 -- 172.21.15.101:0/2007544788 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6dc1952f0 con 0x7fe6dc07be00 2023-07-18T02:57:26.279 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:25.783+0000 7fe6cb7fe700 1 -- 172.21.15.101:0/2007544788 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7fe6cc01abb0 con 0x7fe6dc07be00 2023-07-18T02:57:26.279 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:25.853+0000 7fe6e0cc8700 1 -- 172.21.15.101:0/2007544788 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fe6dc02c850 con 0x7fe6dc07be00 2023-07-18T02:57:26.279 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:25.853+0000 7fe6cb7fe700 1 -- 172.21.15.101:0/2007544788 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+177 (secure 0 0 0) 0x7fe6cc01ad50 con 0x7fe6dc07be00 2023-07-18T02:57:26.279 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:25.854+0000 7fe6e0cc8700 1 -- 172.21.15.101:0/2007544788 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe6dc07be00 msgr2=0x7fe6dc19b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:57:26.279 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:25.854+0000 7fe6e0cc8700 1 --2- 172.21.15.101:0/2007544788 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe6dc07be00 0x7fe6dc19b340 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc009130 tx=0x7fe6cc011ad0 comp rx=0 tx=0).stop 2023-07-18T02:57:26.280 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:25.854+0000 7fe6e0cc8700 1 -- 172.21.15.101:0/2007544788 shutdown_connections 2023-07-18T02:57:26.280 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:25.854+0000 7fe6e0cc8700 1 --2- 172.21.15.101:0/2007544788 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe6dc07be00 0x7fe6dc19b340 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:57:26.280 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:25.854+0000 7fe6e0cc8700 1 -- 172.21.15.101:0/2007544788 >> 172.21.15.101:0/2007544788 conn(0x7fe6dc100970 msgr2=0x7fe6dc1020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:57:26.280 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:25.854+0000 7fe6e0cc8700 1 -- 172.21.15.101:0/2007544788 shutdown_connections 2023-07-18T02:57:26.280 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:25.854+0000 7fe6e0cc8700 1 -- 172.21.15.101:0/2007544788 wait complete. 2023-07-18T02:57:26.280 INFO:teuthology.orchestra.run.smithi101.stdout:Restarting the monitor... 2023-07-18T02:57:26.718 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:26 smithi101 systemd[1]: Stopping Ceph mon.a for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T02:57:26.991 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:26 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mon-a[125840]: 2023-07-18T02:57:26.762+0000 7fee3a685700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T02:57:26.992 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:26 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mon-a[125840]: 2023-07-18T02:57:26.762+0000 7fee3a685700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-07-18T02:57:27.294 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:27 smithi101 bash[126211]: ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mon-a 2023-07-18T02:57:27.717 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:27 smithi101 systemd[1]: ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mon.a.service: Succeeded. 2023-07-18T02:57:27.718 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:27 smithi101 systemd[1]: Stopped Ceph mon.a for b1782daa-2516-11ee-9b34-001a4aab830c. 2023-07-18T02:57:27.718 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:27 smithi101 systemd[1]: Starting Ceph mon.a for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T02:57:28.139 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:27 smithi101 podman[126338]: 2023-07-18T02:57:28.139 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 bash[126338]: 19e14fc4c800ad1ff85abb841cb642940f85d2b4db00fb92b4ee26d1aaec4634 2023-07-18T02:57:28.139 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: set uid:gid to 167:167 (ceph:ceph) 2023-07-18T02:57:28.190 INFO:teuthology.orchestra.run.smithi101.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2023-07-18T02:57:28.446 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: ceph version 18.0.0-4746-g81cf785a (81cf785aa7781c4e08381c540f890cb9bc987ef9) reef (dev), process ceph-mon, pid 7 2023-07-18T02:57:28.447 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: pidfile_write: ignore empty --pid-file 2023-07-18T02:57:28.447 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: load: jerasure load: lrc 2023-07-18T02:57:28.447 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: RocksDB version: 7.9.2 2023-07-18T02:57:28.447 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Git sha 0 2023-07-18T02:57:28.447 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Compile date 2023-07-17 23:00:14 2023-07-18T02:57:28.447 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: DB SUMMARY 2023-07-18T02:57:28.448 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: DB Session ID: NQ2WCL6NHEOIYFLR9Q4D 2023-07-18T02:57:28.448 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: CURRENT file: CURRENT 2023-07-18T02:57:28.448 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: IDENTITY file: IDENTITY 2023-07-18T02:57:28.448 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-07-18T02:57:28.448 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-07-18T02:57:28.448 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 98765 ; 2023-07-18T02:57:28.448 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.error_if_exists: 0 2023-07-18T02:57:28.449 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.create_if_missing: 0 2023-07-18T02:57:28.449 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.paranoid_checks: 1 2023-07-18T02:57:28.449 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.flush_verify_memtable_count: 1 2023-07-18T02:57:28.449 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-07-18T02:57:28.449 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-07-18T02:57:28.449 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.env: 0x5594440c4720 2023-07-18T02:57:28.449 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.fs: PosixFileSystem 2023-07-18T02:57:28.450 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.info_log: 0x5594465c3280 2023-07-18T02:57:28.450 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.max_file_opening_threads: 16 2023-07-18T02:57:28.450 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.statistics: (nil) 2023-07-18T02:57:28.450 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.use_fsync: 0 2023-07-18T02:57:28.450 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.max_log_file_size: 0 2023-07-18T02:57:28.450 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-07-18T02:57:28.450 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.log_file_time_to_roll: 0 2023-07-18T02:57:28.450 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.keep_log_file_num: 1000 2023-07-18T02:57:28.451 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.recycle_log_file_num: 0 2023-07-18T02:57:28.451 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.allow_fallocate: 1 2023-07-18T02:57:28.451 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.allow_mmap_reads: 0 2023-07-18T02:57:28.451 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.allow_mmap_writes: 0 2023-07-18T02:57:28.451 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.use_direct_reads: 0 2023-07-18T02:57:28.451 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-07-18T02:57:28.451 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.create_missing_column_families: 0 2023-07-18T02:57:28.452 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.db_log_dir: 2023-07-18T02:57:28.452 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.wal_dir: 2023-07-18T02:57:28.452 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.table_cache_numshardbits: 6 2023-07-18T02:57:28.452 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.WAL_ttl_seconds: 0 2023-07-18T02:57:28.452 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.WAL_size_limit_MB: 0 2023-07-18T02:57:28.452 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-07-18T02:57:28.452 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-07-18T02:57:28.452 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.is_fd_close_on_exec: 1 2023-07-18T02:57:28.453 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.advise_random_on_open: 1 2023-07-18T02:57:28.453 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.db_write_buffer_size: 0 2023-07-18T02:57:28.453 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.write_buffer_manager: 0x5594464e43c0 2023-07-18T02:57:28.453 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-07-18T02:57:28.453 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-07-18T02:57:28.453 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.use_adaptive_mutex: 0 2023-07-18T02:57:28.453 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.rate_limiter: (nil) 2023-07-18T02:57:28.454 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-07-18T02:57:28.454 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.wal_recovery_mode: 2 2023-07-18T02:57:28.454 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.enable_thread_tracking: 0 2023-07-18T02:57:28.454 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.enable_pipelined_write: 0 2023-07-18T02:57:28.454 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.unordered_write: 0 2023-07-18T02:57:28.454 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-07-18T02:57:28.454 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-07-18T02:57:28.454 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-07-18T02:57:28.455 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-07-18T02:57:28.455 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.row_cache: None 2023-07-18T02:57:28.455 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.wal_filter: None 2023-07-18T02:57:28.455 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-07-18T02:57:28.455 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.allow_ingest_behind: 0 2023-07-18T02:57:28.455 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.two_write_queues: 0 2023-07-18T02:57:28.455 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.manual_wal_flush: 0 2023-07-18T02:57:28.455 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.wal_compression: 0 2023-07-18T02:57:28.456 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.atomic_flush: 0 2023-07-18T02:57:28.456 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-07-18T02:57:28.456 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.persist_stats_to_disk: 0 2023-07-18T02:57:28.456 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.write_dbid_to_manifest: 0 2023-07-18T02:57:28.456 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.log_readahead_size: 0 2023-07-18T02:57:28.456 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-07-18T02:57:28.456 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.best_efforts_recovery: 0 2023-07-18T02:57:28.456 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-07-18T02:57:28.457 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-07-18T02:57:28.457 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.allow_data_in_errors: 0 2023-07-18T02:57:28.457 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.db_host_id: __hostname__ 2023-07-18T02:57:28.457 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.enforce_single_del_contracts: true 2023-07-18T02:57:28.457 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.max_background_jobs: 2 2023-07-18T02:57:28.457 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.max_background_compactions: -1 2023-07-18T02:57:28.457 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.max_subcompactions: 1 2023-07-18T02:57:28.458 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-07-18T02:57:28.458 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-07-18T02:57:28.458 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.delayed_write_rate : 16777216 2023-07-18T02:57:28.458 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.max_total_wal_size: 0 2023-07-18T02:57:28.458 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-07-18T02:57:28.458 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.stats_dump_period_sec: 600 2023-07-18T02:57:28.458 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.stats_persist_period_sec: 600 2023-07-18T02:57:28.458 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-07-18T02:57:28.459 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.max_open_files: -1 2023-07-18T02:57:28.459 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.bytes_per_sync: 0 2023-07-18T02:57:28.459 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.wal_bytes_per_sync: 0 2023-07-18T02:57:28.459 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.strict_bytes_per_sync: 0 2023-07-18T02:57:28.459 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.compaction_readahead_size: 0 2023-07-18T02:57:28.459 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.max_background_flushes: -1 2023-07-18T02:57:28.459 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Compression algorithms supported: 2023-07-18T02:57:28.460 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-07-18T02:57:28.460 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: kZSTD supported: 0 2023-07-18T02:57:28.460 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: kXpressCompression supported: 0 2023-07-18T02:57:28.460 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: kLZ4HCCompression supported: 1 2023-07-18T02:57:28.460 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: kZlibCompression supported: 1 2023-07-18T02:57:28.460 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: kSnappyCompression supported: 1 2023-07-18T02:57:28.460 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: kLZ4Compression supported: 1 2023-07-18T02:57:28.460 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: kBZip2Compression supported: 0 2023-07-18T02:57:28.461 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Fast CRC32 supported: Supported on x86 2023-07-18T02:57:28.461 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: DMutex implementation: pthread_mutex_t 2023-07-18T02:57:28.461 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-07-18T02:57:28.461 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-07-18T02:57:28.461 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-07-18T02:57:28.461 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.merge_operator: 2023-07-18T02:57:28.461 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.compaction_filter: None 2023-07-18T02:57:28.461 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.compaction_filter_factory: None 2023-07-18T02:57:28.462 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.sst_partitioner_factory: None 2023-07-18T02:57:28.462 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.memtable_factory: SkipListFactory 2023-07-18T02:57:28.462 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.table_factory: BlockBasedTable 2023-07-18T02:57:28.462 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5594465c3380) 2023-07-18T02:57:28.462 INFO:journalctl@ceph.mon.a.smithi101.stdout: cache_index_and_filter_blocks: 1 2023-07-18T02:57:28.462 INFO:journalctl@ceph.mon.a.smithi101.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-07-18T02:57:28.462 INFO:journalctl@ceph.mon.a.smithi101.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-07-18T02:57:28.463 INFO:journalctl@ceph.mon.a.smithi101.stdout: pin_top_level_index_and_filter: 1 2023-07-18T02:57:28.463 INFO:journalctl@ceph.mon.a.smithi101.stdout: index_type: 0 2023-07-18T02:57:28.463 INFO:journalctl@ceph.mon.a.smithi101.stdout: data_block_index_type: 0 2023-07-18T02:57:28.463 INFO:journalctl@ceph.mon.a.smithi101.stdout: index_shortening: 1 2023-07-18T02:57:28.463 INFO:journalctl@ceph.mon.a.smithi101.stdout: data_block_hash_table_util_ratio: 0.750000 2023-07-18T02:57:28.463 INFO:journalctl@ceph.mon.a.smithi101.stdout: checksum: 4 2023-07-18T02:57:28.463 INFO:journalctl@ceph.mon.a.smithi101.stdout: no_block_cache: 0 2023-07-18T02:57:28.463 INFO:journalctl@ceph.mon.a.smithi101.stdout: block_cache: 0x5594465831f0 2023-07-18T02:57:28.464 INFO:journalctl@ceph.mon.a.smithi101.stdout: block_cache_name: BinnedLRUCache 2023-07-18T02:57:28.464 INFO:journalctl@ceph.mon.a.smithi101.stdout: block_cache_options: 2023-07-18T02:57:28.464 INFO:journalctl@ceph.mon.a.smithi101.stdout: capacity : 536870912 2023-07-18T02:57:28.464 INFO:journalctl@ceph.mon.a.smithi101.stdout: num_shard_bits : 4 2023-07-18T02:57:28.464 INFO:journalctl@ceph.mon.a.smithi101.stdout: strict_capacity_limit : 0 2023-07-18T02:57:28.464 INFO:journalctl@ceph.mon.a.smithi101.stdout: high_pri_pool_ratio: 0.000 2023-07-18T02:57:28.464 INFO:journalctl@ceph.mon.a.smithi101.stdout: block_cache_compressed: (nil) 2023-07-18T02:57:28.464 INFO:journalctl@ceph.mon.a.smithi101.stdout: persistent_cache: (nil) 2023-07-18T02:57:28.465 INFO:journalctl@ceph.mon.a.smithi101.stdout: block_size: 4096 2023-07-18T02:57:28.465 INFO:journalctl@ceph.mon.a.smithi101.stdout: block_size_deviation: 10 2023-07-18T02:57:28.465 INFO:journalctl@ceph.mon.a.smithi101.stdout: block_restart_interval: 16 2023-07-18T02:57:28.465 INFO:journalctl@ceph.mon.a.smithi101.stdout: index_block_restart_interval: 1 2023-07-18T02:57:28.465 INFO:journalctl@ceph.mon.a.smithi101.stdout: metadata_block_size: 4096 2023-07-18T02:57:28.465 INFO:journalctl@ceph.mon.a.smithi101.stdout: partition_filters: 0 2023-07-18T02:57:28.465 INFO:journalctl@ceph.mon.a.smithi101.stdout: use_delta_encoding: 1 2023-07-18T02:57:28.466 INFO:journalctl@ceph.mon.a.smithi101.stdout: filter_policy: bloomfilter 2023-07-18T02:57:28.466 INFO:journalctl@ceph.mon.a.smithi101.stdout: whole_key_filtering: 1 2023-07-18T02:57:28.466 INFO:journalctl@ceph.mon.a.smithi101.stdout: verify_compression: 0 2023-07-18T02:57:28.466 INFO:journalctl@ceph.mon.a.smithi101.stdout: read_amp_bytes_per_bit: 0 2023-07-18T02:57:28.466 INFO:journalctl@ceph.mon.a.smithi101.stdout: format_version: 5 2023-07-18T02:57:28.466 INFO:journalctl@ceph.mon.a.smithi101.stdout: enable_index_compression: 1 2023-07-18T02:57:28.466 INFO:journalctl@ceph.mon.a.smithi101.stdout: block_align: 0 2023-07-18T02:57:28.466 INFO:journalctl@ceph.mon.a.smithi101.stdout: max_auto_readahead_size: 262144 2023-07-18T02:57:28.467 INFO:journalctl@ceph.mon.a.smithi101.stdout: prepopulate_block_cache: 0 2023-07-18T02:57:28.467 INFO:journalctl@ceph.mon.a.smithi101.stdout: initial_auto_readahead_size: 8192 2023-07-18T02:57:28.467 INFO:journalctl@ceph.mon.a.smithi101.stdout: num_file_reads_for_auto_readahead: 2 2023-07-18T02:57:28.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.write_buffer_size: 33554432 2023-07-18T02:57:28.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.max_write_buffer_number: 2 2023-07-18T02:57:28.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.compression: NoCompression 2023-07-18T02:57:28.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.bottommost_compression: Disabled 2023-07-18T02:57:28.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.prefix_extractor: nullptr 2023-07-18T02:57:28.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-07-18T02:57:28.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.num_levels: 7 2023-07-18T02:57:28.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-07-18T02:57:28.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-07-18T02:57:28.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-07-18T02:57:28.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-07-18T02:57:28.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-07-18T02:57:28.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-07-18T02:57:28.469 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-07-18T02:57:28.470 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-07-18T02:57:28.470 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-07-18T02:57:28.470 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-07-18T02:57:28.470 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-07-18T02:57:28.470 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-07-18T02:57:28.470 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.compression_opts.window_bits: -14 2023-07-18T02:57:28.470 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.compression_opts.level: 32767 2023-07-18T02:57:28.471 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.compression_opts.strategy: 0 2023-07-18T02:57:28.471 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-07-18T02:57:28.471 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-07-18T02:57:28.471 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-07-18T02:57:28.471 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-07-18T02:57:28.471 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.compression_opts.enabled: false 2023-07-18T02:57:28.471 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-07-18T02:57:28.471 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-07-18T02:57:28.472 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-07-18T02:57:28.472 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-07-18T02:57:28.472 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.target_file_size_base: 67108864 2023-07-18T02:57:28.472 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.target_file_size_multiplier: 1 2023-07-18T02:57:28.472 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-07-18T02:57:28.472 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-07-18T02:57:28.472 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-07-18T02:57:28.473 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-07-18T02:57:28.473 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-07-18T02:57:28.473 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-07-18T02:57:28.473 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-07-18T02:57:28.473 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-07-18T02:57:28.473 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-07-18T02:57:28.473 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-07-18T02:57:28.473 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-07-18T02:57:28.474 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-07-18T02:57:28.474 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-07-18T02:57:28.474 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.arena_block_size: 1048576 2023-07-18T02:57:28.474 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-07-18T02:57:28.474 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-07-18T02:57:28.474 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.disable_auto_compactions: 0 2023-07-18T02:57:28.474 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-07-18T02:57:28.475 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-07-18T02:57:28.475 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-07-18T02:57:28.475 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-07-18T02:57:28.475 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-07-18T02:57:28.475 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-07-18T02:57:28.475 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-07-18T02:57:28.475 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-07-18T02:57:28.475 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-07-18T02:57:28.476 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-07-18T02:57:28.476 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.table_properties_collectors: 2023-07-18T02:57:28.476 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.inplace_update_support: 0 2023-07-18T02:57:28.476 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.inplace_update_num_locks: 10000 2023-07-18T02:57:28.476 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-07-18T02:57:28.476 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-07-18T02:57:28.476 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.memtable_huge_page_size: 0 2023-07-18T02:57:28.476 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.bloom_locality: 0 2023-07-18T02:57:28.477 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.max_successive_merges: 0 2023-07-18T02:57:28.477 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.optimize_filters_for_hits: 0 2023-07-18T02:57:28.477 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.paranoid_file_checks: 0 2023-07-18T02:57:28.477 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.force_consistency_checks: 1 2023-07-18T02:57:28.477 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.report_bg_io_stats: 0 2023-07-18T02:57:28.477 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.ttl: 2592000 2023-07-18T02:57:28.477 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.periodic_compaction_seconds: 0 2023-07-18T02:57:28.478 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-07-18T02:57:28.478 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-07-18T02:57:28.478 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.enable_blob_files: false 2023-07-18T02:57:28.478 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.min_blob_size: 0 2023-07-18T02:57:28.478 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.blob_file_size: 268435456 2023-07-18T02:57:28.478 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.blob_compression_type: NoCompression 2023-07-18T02:57:28.478 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.enable_blob_garbage_collection: false 2023-07-18T02:57:28.478 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-07-18T02:57:28.479 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-07-18T02:57:28.479 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-07-18T02:57:28.479 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.blob_file_starting_level: 0 2023-07-18T02:57:28.479 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-07-18T02:57:28.479 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-07-18T02:57:28.479 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-07-18T02:57:28.479 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: bf61586b-fa2f-4ab6-9acb-f0546baaa321 2023-07-18T02:57:28.480 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689649048175282, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-07-18T02:57:28.480 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-07-18T02:57:28.480 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689649048177098, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 94006, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 314, "table_properties": {"data_size": 92011, "index_size": 277, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13640, "raw_average_key_size": 49, "raw_value_size": 84576, "raw_average_value_size": 309, "num_data_blocks": 11, "num_entries": 273, "num_filter_entries": 273, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1689649048, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "bf61586b-fa2f-4ab6-9acb-f0546baaa321", "db_session_id": "NQ2WCL6NHEOIYFLR9Q4D", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-07-18T02:57:28.480 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689649048177260, "job": 1, "event": "recovery_finished"} 2023-07-18T02:57:28.480 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-07-18T02:57:28.480 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-07-18T02:57:28.480 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55944661c000 2023-07-18T02:57:28.480 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: DB pointer 0x559446608000 2023-07-18T02:57:28.481 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-07-18T02:57:28.481 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-07-18T02:57:28.481 INFO:journalctl@ceph.mon.a.smithi101.stdout: ** DB Stats ** 2023-07-18T02:57:28.481 INFO:journalctl@ceph.mon.a.smithi101.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-18T02:57:28.481 INFO:journalctl@ceph.mon.a.smithi101.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-07-18T02:57:28.481 INFO:journalctl@ceph.mon.a.smithi101.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-18T02:57:28.481 INFO:journalctl@ceph.mon.a.smithi101.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-18T02:57:28.482 INFO:journalctl@ceph.mon.a.smithi101.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-07-18T02:57:28.482 INFO:journalctl@ceph.mon.a.smithi101.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-18T02:57:28.482 INFO:journalctl@ceph.mon.a.smithi101.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-18T02:57:28.482 INFO:journalctl@ceph.mon.a.smithi101.stdout: 2023-07-18T02:57:28.482 INFO:journalctl@ceph.mon.a.smithi101.stdout: ** Compaction Stats [default] ** 2023-07-18T02:57:28.482 INFO:journalctl@ceph.mon.a.smithi101.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-18T02:57:28.482 INFO:journalctl@ceph.mon.a.smithi101.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-07-18T02:57:28.482 INFO:journalctl@ceph.mon.a.smithi101.stdout: L0 2/0 93.64 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 86.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-18T02:57:28.483 INFO:journalctl@ceph.mon.a.smithi101.stdout: Sum 2/0 93.64 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 86.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-18T02:57:28.483 INFO:journalctl@ceph.mon.a.smithi101.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 86.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-18T02:57:28.483 INFO:journalctl@ceph.mon.a.smithi101.stdout: 2023-07-18T02:57:28.483 INFO:journalctl@ceph.mon.a.smithi101.stdout: ** Compaction Stats [default] ** 2023-07-18T02:57:28.483 INFO:journalctl@ceph.mon.a.smithi101.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-18T02:57:28.483 INFO:journalctl@ceph.mon.a.smithi101.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-07-18T02:57:28.483 INFO:journalctl@ceph.mon.a.smithi101.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 86.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-18T02:57:28.484 INFO:journalctl@ceph.mon.a.smithi101.stdout: 2023-07-18T02:57:28.484 INFO:journalctl@ceph.mon.a.smithi101.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-07-18T02:57:28.484 INFO:journalctl@ceph.mon.a.smithi101.stdout: 2023-07-18T02:57:28.484 INFO:journalctl@ceph.mon.a.smithi101.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-18T02:57:28.484 INFO:journalctl@ceph.mon.a.smithi101.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-07-18T02:57:28.484 INFO:journalctl@ceph.mon.a.smithi101.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-07-18T02:57:28.484 INFO:journalctl@ceph.mon.a.smithi101.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-07-18T02:57:28.484 INFO:journalctl@ceph.mon.a.smithi101.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-07-18T02:57:28.485 INFO:journalctl@ceph.mon.a.smithi101.stdout: AddFile(Keys): cumulative 0, interval 0 2023-07-18T02:57:28.485 INFO:journalctl@ceph.mon.a.smithi101.stdout: Cumulative compaction: 0.00 GB write, 14.51 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-18T02:57:28.485 INFO:journalctl@ceph.mon.a.smithi101.stdout: Interval compaction: 0.00 GB write, 14.51 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-18T02:57:28.485 INFO:journalctl@ceph.mon.a.smithi101.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-07-18T02:57:28.485 INFO:journalctl@ceph.mon.a.smithi101.stdout: Block cache BinnedLRUCache@0x5594465831f0#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2023-07-18T02:57:28.485 INFO:journalctl@ceph.mon.a.smithi101.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2023-07-18T02:57:28.485 INFO:journalctl@ceph.mon.a.smithi101.stdout: 2023-07-18T02:57:28.486 INFO:journalctl@ceph.mon.a.smithi101.stdout: ** File Read Latency Histogram By Level [default] ** 2023-07-18T02:57:28.486 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: starting mon.a rank 0 at public addrs [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] at bind addrs [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid b1782daa-2516-11ee-9b34-001a4aab830c 2023-07-18T02:57:28.486 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: mon.a@-1(???) e1 preinit fsid b1782daa-2516-11ee-9b34-001a4aab830c 2023-07-18T02:57:28.486 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: mon.a@-1(???).mds e1 new map 2023-07-18T02:57:28.486 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: mon.a@-1(???).mds e1 print_map 2023-07-18T02:57:28.486 INFO:journalctl@ceph.mon.a.smithi101.stdout: e1 2023-07-18T02:57:28.486 INFO:journalctl@ceph.mon.a.smithi101.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-07-18T02:57:28.487 INFO:journalctl@ceph.mon.a.smithi101.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-18T02:57:28.487 INFO:journalctl@ceph.mon.a.smithi101.stdout: legacy client fscid: -1 2023-07-18T02:57:28.487 INFO:journalctl@ceph.mon.a.smithi101.stdout: 2023-07-18T02:57:28.487 INFO:journalctl@ceph.mon.a.smithi101.stdout: No filesystems configured 2023-07-18T02:57:28.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-07-18T02:57:28.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-07-18T02:57:28.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-07-18T02:57:28.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-07-18T02:57:28.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-07-18T02:57:28.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: expand_channel_meta expand map: {default=false} 2023-07-18T02:57:28.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: expand_channel_meta from 'false' to 'false' 2023-07-18T02:57:28.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: expand_channel_meta expanded map: {default=false} 2023-07-18T02:57:28.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: expand_channel_meta expand map: {default=info} 2023-07-18T02:57:28.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: expand_channel_meta from 'info' to 'info' 2023-07-18T02:57:28.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: expand_channel_meta expanded map: {default=info} 2023-07-18T02:57:28.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: expand_channel_meta expand map: {default=daemon} 2023-07-18T02:57:28.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: expand_channel_meta from 'daemon' to 'daemon' 2023-07-18T02:57:28.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: expand_channel_meta expanded map: {default=daemon} 2023-07-18T02:57:28.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: expand_channel_meta expand map: {default=debug} 2023-07-18T02:57:28.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: expand_channel_meta from 'debug' to 'debug' 2023-07-18T02:57:28.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: expand_channel_meta expanded map: {default=debug} 2023-07-18T02:57:28.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 systemd[1]: Started Ceph mon.a for b1782daa-2516-11ee-9b34-001a4aab830c. 2023-07-18T02:57:28.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: mon.a is new leader, mons a in quorum (ranks 0) 2023-07-18T02:57:28.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: monmap e1: 1 mons at {a=[v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0]} removed_ranks: {} 2023-07-18T02:57:28.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: fsmap 2023-07-18T02:57:28.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: osdmap e1: 0 total, 0 up, 0 in 2023-07-18T02:57:28.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:28 smithi101 ceph-mon[126379]: mgrmap e1: no daemons active 2023-07-18T02:57:29.177 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:28.728+0000 7faf1529b700 1 Processor -- start 2023-07-18T02:57:29.178 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:28.728+0000 7faf1529b700 1 -- start start 2023-07-18T02:57:29.178 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:28.729+0000 7faf1529b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf1007be20 0x7faf1007a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:57:29.178 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:28.729+0000 7faf1529b700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf1007a860 con 0x7faf1007be20 2023-07-18T02:57:29.178 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:28.729+0000 7faf0effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf1007be20 0x7faf1007a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:57:29.178 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:28.729+0000 7faf0effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf1007be20 0x7faf1007a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:58626/0 (socket says 172.21.15.101:58626) 2023-07-18T02:57:29.178 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:28.729+0000 7faf0effd700 1 -- 172.21.15.101:0/2743162411 learned_addr learned my addr 172.21.15.101:0/2743162411 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:57:29.179 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:28.730+0000 7faf0effd700 1 -- 172.21.15.101:0/2743162411 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf1007a9a0 con 0x7faf1007be20 2023-07-18T02:57:29.179 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:28.730+0000 7faf0effd700 1 --2- 172.21.15.101:0/2743162411 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf1007be20 0x7faf1007a320 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7faf000098a0 tx=0x7faf00009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47210ed77a27e05a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:57:29.179 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:28.730+0000 7faf0dffb700 1 -- 172.21.15.101:0/2743162411 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf00004160 con 0x7faf1007be20 2023-07-18T02:57:29.179 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:28.730+0000 7faf0dffb700 1 -- 172.21.15.101:0/2743162411 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7faf000042c0 con 0x7faf1007be20 2023-07-18T02:57:29.179 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:28.730+0000 7faf0dffb700 1 -- 172.21.15.101:0/2743162411 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf00011a00 con 0x7faf1007be20 2023-07-18T02:57:29.179 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:28.731+0000 7faf1529b700 1 -- 172.21.15.101:0/2743162411 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf1007be20 msgr2=0x7faf1007a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:57:29.179 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:28.731+0000 7faf1529b700 1 --2- 172.21.15.101:0/2743162411 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf1007be20 0x7faf1007a320 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7faf000098a0 tx=0x7faf00009bb0 comp rx=0 tx=0).stop 2023-07-18T02:57:29.180 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:28.731+0000 7faf1529b700 1 -- 172.21.15.101:0/2743162411 shutdown_connections 2023-07-18T02:57:29.180 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:28.731+0000 7faf1529b700 1 --2- 172.21.15.101:0/2743162411 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf1007be20 0x7faf1007a320 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:57:29.180 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:28.731+0000 7faf1529b700 1 -- 172.21.15.101:0/2743162411 >> 172.21.15.101:0/2743162411 conn(0x7faf10100970 msgr2=0x7faf10102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:57:29.180 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:28.731+0000 7faf1529b700 1 -- 172.21.15.101:0/2743162411 shutdown_connections 2023-07-18T02:57:29.180 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:28.731+0000 7faf1529b700 1 -- 172.21.15.101:0/2743162411 wait complete. 2023-07-18T02:57:29.180 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:28.732+0000 7faf1529b700 1 Processor -- start 2023-07-18T02:57:29.180 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:28.733+0000 7faf1529b700 1 -- start start 2023-07-18T02:57:29.181 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:28.733+0000 7faf1529b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf1007be20 0x7faf1019b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:57:29.181 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:28.733+0000 7faf1529b700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf1019b880 con 0x7faf1007be20 2023-07-18T02:57:29.181 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:28.733+0000 7faf0effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf1007be20 0x7faf1019b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:57:29.183 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:28.733+0000 7faf0effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf1007be20 0x7faf1019b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:58638/0 (socket says 172.21.15.101:58638) 2023-07-18T02:57:29.183 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:28.733+0000 7faf0effd700 1 -- 172.21.15.101:0/3040495495 learned_addr learned my addr 172.21.15.101:0/3040495495 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:57:29.183 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:28.734+0000 7faf0effd700 1 -- 172.21.15.101:0/3040495495 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf00009550 con 0x7faf1007be20 2023-07-18T02:57:29.183 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:28.734+0000 7faf0effd700 1 --2- 172.21.15.101:0/3040495495 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf1007be20 0x7faf1019b340 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7faf00003f50 tx=0x7faf00011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:57:29.183 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:28.734+0000 7faef7fff700 1 -- 172.21.15.101:0/3040495495 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf00003aa0 con 0x7faf1007be20 2023-07-18T02:57:29.183 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:28.734+0000 7faef7fff700 1 -- 172.21.15.101:0/3040495495 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7faf00003c00 con 0x7faf1007be20 2023-07-18T02:57:29.184 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:28.734+0000 7faf1529b700 1 -- 172.21.15.101:0/3040495495 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf1019ba80 con 0x7faf1007be20 2023-07-18T02:57:29.184 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:28.734+0000 7faef7fff700 1 -- 172.21.15.101:0/3040495495 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf000247c0 con 0x7faf1007be20 2023-07-18T02:57:29.184 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:28.734+0000 7faf1529b700 1 -- 172.21.15.101:0/3040495495 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf1019bea0 con 0x7faf1007be20 2023-07-18T02:57:29.184 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:28.735+0000 7faef7fff700 1 -- 172.21.15.101:0/3040495495 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7faf00024c20 con 0x7faf1007be20 2023-07-18T02:57:29.184 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:28.735+0000 7faef7fff700 1 -- 172.21.15.101:0/3040495495 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7faf000193c0 con 0x7faf1007be20 2023-07-18T02:57:29.184 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:28.735+0000 7faf1529b700 1 -- 172.21.15.101:0/3040495495 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf10195320 con 0x7faf1007be20 2023-07-18T02:57:29.184 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:28.738+0000 7faef7fff700 1 -- 172.21.15.101:0/3040495495 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7faf0002de40 con 0x7faf1007be20 2023-07-18T02:57:29.184 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:28.807+0000 7faf1529b700 1 -- 172.21.15.101:0/3040495495 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7faf1002c850 con 0x7faf1007be20 2023-07-18T02:57:29.185 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:28.810+0000 7faef7fff700 1 -- 172.21.15.101:0/3040495495 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7faf0001ab60 con 0x7faf1007be20 2023-07-18T02:57:29.185 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:28.810+0000 7faf1529b700 1 -- 172.21.15.101:0/3040495495 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf1007be20 msgr2=0x7faf1019b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:57:29.185 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:28.811+0000 7faf1529b700 1 --2- 172.21.15.101:0/3040495495 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf1007be20 0x7faf1019b340 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7faf00003f50 tx=0x7faf00011ad0 comp rx=0 tx=0).stop 2023-07-18T02:57:29.185 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:28.811+0000 7faf1529b700 1 -- 172.21.15.101:0/3040495495 shutdown_connections 2023-07-18T02:57:29.185 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:28.811+0000 7faf1529b700 1 --2- 172.21.15.101:0/3040495495 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf1007be20 0x7faf1019b340 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:57:29.185 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:28.811+0000 7faf1529b700 1 -- 172.21.15.101:0/3040495495 >> 172.21.15.101:0/3040495495 conn(0x7faf10100970 msgr2=0x7faf10102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:57:29.185 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:28.811+0000 7faf1529b700 1 -- 172.21.15.101:0/3040495495 shutdown_connections 2023-07-18T02:57:29.186 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:28.811+0000 7faf1529b700 1 -- 172.21.15.101:0/3040495495 wait complete. 2023-07-18T02:57:29.186 INFO:teuthology.orchestra.run.smithi101.stdout:Wrote config to /etc/ceph/ceph.conf 2023-07-18T02:57:29.250 INFO:teuthology.orchestra.run.smithi101.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-07-18T02:57:29.251 INFO:teuthology.orchestra.run.smithi101.stdout:Creating mgr... 2023-07-18T02:57:29.251 INFO:teuthology.orchestra.run.smithi101.stdout:Verifying port 9283 ... 2023-07-18T02:57:29.251 INFO:teuthology.orchestra.run.smithi101.stdout:Verifying port 8765 ... 2023-07-18T02:57:29.614 INFO:teuthology.orchestra.run.smithi101.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mgr.x 2023-07-18T02:57:29.615 INFO:teuthology.orchestra.run.smithi101.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mgr.x.service: Unit ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mgr.x.service not loaded. 2023-07-18T02:57:29.820 INFO:teuthology.orchestra.run.smithi101.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b1782daa-2516-11ee-9b34-001a4aab830c.target.wants/ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mgr.x.service → /etc/systemd/system/ceph-b1782daa-2516-11ee-9b34-001a4aab830c@.service. 2023-07-18T02:57:29.967 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:29 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3040495495' entity='client.admin' 2023-07-18T02:57:29.968 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:29 smithi101 systemd[1]: Starting Ceph mgr.x for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T02:57:30.604 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:30 smithi101 podman[126610]: 2023-07-18T02:57:30.604 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:30 smithi101 bash[126610]: c8d9e9e2ba2db94dea4825b4923d635ab9e6045acaff08baca84f79827a23b16 2023-07-18T02:57:30.631 INFO:teuthology.orchestra.run.smithi101.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-18T02:57:30.631 INFO:teuthology.orchestra.run.smithi101.stdout:systemctl: stdout disabled 2023-07-18T02:57:30.650 INFO:teuthology.orchestra.run.smithi101.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-18T02:57:30.650 INFO:teuthology.orchestra.run.smithi101.stdout:systemctl: stdout inactive 2023-07-18T02:57:30.650 INFO:teuthology.orchestra.run.smithi101.stdout:firewalld.service is not enabled 2023-07-18T02:57:30.651 INFO:teuthology.orchestra.run.smithi101.stdout:Not possible to enable service . firewalld.service is not available 2023-07-18T02:57:30.670 INFO:teuthology.orchestra.run.smithi101.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-18T02:57:30.671 INFO:teuthology.orchestra.run.smithi101.stdout:systemctl: stdout disabled 2023-07-18T02:57:30.689 INFO:teuthology.orchestra.run.smithi101.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-18T02:57:30.689 INFO:teuthology.orchestra.run.smithi101.stdout:systemctl: stdout inactive 2023-07-18T02:57:30.690 INFO:teuthology.orchestra.run.smithi101.stdout:firewalld.service is not enabled 2023-07-18T02:57:30.690 INFO:teuthology.orchestra.run.smithi101.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-07-18T02:57:30.690 INFO:teuthology.orchestra.run.smithi101.stdout:Waiting for mgr to start... 2023-07-18T02:57:30.690 INFO:teuthology.orchestra.run.smithi101.stdout:Waiting for mgr... 2023-07-18T02:57:30.863 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:30 smithi101 systemd[1]: Started Ceph mgr.x for b1782daa-2516-11ee-9b34-001a4aab830c. 2023-07-18T02:57:30.863 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:30 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:30.779+0000 7f7632b37200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-18T02:57:31.218 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:30 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:30.890+0000 7f7632b37200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-18T02:57:31.218 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:31 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:31.007+0000 7f7632b37200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-18T02:57:31.218 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:31 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:31.133+0000 7f7632b37200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-18T02:57:31.717 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:31 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/974609039' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-18T02:57:31.718 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:31 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:31.361+0000 7f7632b37200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-18T02:57:31.773 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout 2023-07-18T02:57:31.773 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout { 2023-07-18T02:57:31.774 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "fsid": "b1782daa-2516-11ee-9b34-001a4aab830c", 2023-07-18T02:57:31.774 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "health": { 2023-07-18T02:57:31.774 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-18T02:57:31.776 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-18T02:57:31.776 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-18T02:57:31.776 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout }, 2023-07-18T02:57:31.776 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-18T02:57:31.777 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-18T02:57:31.777 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout 0 2023-07-18T02:57:31.777 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout ], 2023-07-18T02:57:31.777 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-18T02:57:31.777 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "a" 2023-07-18T02:57:31.777 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout ], 2023-07-18T02:57:31.778 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2023-07-18T02:57:31.778 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-18T02:57:31.778 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-18T02:57:31.778 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-18T02:57:31.778 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-18T02:57:31.778 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout }, 2023-07-18T02:57:31.778 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-18T02:57:31.779 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-18T02:57:31.779 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-18T02:57:31.779 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-18T02:57:31.779 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-18T02:57:31.785 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-18T02:57:31.785 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-18T02:57:31.786 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-18T02:57:31.786 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout }, 2023-07-18T02:57:31.786 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-18T02:57:31.786 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-18T02:57:31.786 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-18T02:57:31.786 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-18T02:57:31.786 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-18T02:57:31.786 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-18T02:57:31.787 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-18T02:57:31.787 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-18T02:57:31.787 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-18T02:57:31.787 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout }, 2023-07-18T02:57:31.787 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-18T02:57:31.787 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-18T02:57:31.787 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-18T02:57:31.788 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-18T02:57:31.788 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout }, 2023-07-18T02:57:31.788 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-18T02:57:31.788 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "available": false, 2023-07-18T02:57:31.788 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-18T02:57:31.788 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-18T02:57:31.788 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-18T02:57:31.788 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-18T02:57:31.789 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "restful" 2023-07-18T02:57:31.789 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout ], 2023-07-18T02:57:31.789 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-18T02:57:31.789 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout }, 2023-07-18T02:57:31.789 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-18T02:57:31.789 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-18T02:57:31.789 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "modified": "2023-07-18T02:57:22.972928+0000", 2023-07-18T02:57:31.790 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-18T02:57:31.790 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout }, 2023-07-18T02:57:31.790 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-18T02:57:31.790 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout } 2023-07-18T02:57:31.790 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:31.295+0000 7f63bb859700 1 Processor -- start 2023-07-18T02:57:31.790 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:31.296+0000 7f63bb859700 1 -- start start 2023-07-18T02:57:31.790 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:31.296+0000 7f63bb859700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63b4079d20 0x7f63b4107ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:57:31.790 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:31.296+0000 7f63bb859700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b4108400 con 0x7f63b4079d20 2023-07-18T02:57:31.791 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:31.297+0000 7f63b95f5700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63b4079d20 0x7f63b4107ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:57:31.791 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:31.297+0000 7f63b95f5700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63b4079d20 0x7f63b4107ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:58656/0 (socket says 172.21.15.101:58656) 2023-07-18T02:57:31.791 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:31.297+0000 7f63b95f5700 1 -- 172.21.15.101:0/403428867 learned_addr learned my addr 172.21.15.101:0/403428867 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:57:31.791 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:31.297+0000 7f63b95f5700 1 -- 172.21.15.101:0/403428867 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b4108540 con 0x7f63b4079d20 2023-07-18T02:57:31.791 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:31.297+0000 7f63b95f5700 1 --2- 172.21.15.101:0/403428867 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63b4079d20 0x7f63b4107ec0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f63b00098a0 tx=0x7f63b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd280f406d65dd7c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:57:31.791 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:31.297+0000 7f63abfff700 1 -- 172.21.15.101:0/403428867 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f63b0004160 con 0x7f63b4079d20 2023-07-18T02:57:31.791 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:31.297+0000 7f63abfff700 1 -- 172.21.15.101:0/403428867 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f63b00042c0 con 0x7f63b4079d20 2023-07-18T02:57:31.792 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:31.297+0000 7f63abfff700 1 -- 172.21.15.101:0/403428867 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f63b0011a00 con 0x7f63b4079d20 2023-07-18T02:57:31.792 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:31.298+0000 7f63bb859700 1 -- 172.21.15.101:0/403428867 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63b4079d20 msgr2=0x7f63b4107ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:57:31.792 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:31.298+0000 7f63bb859700 1 --2- 172.21.15.101:0/403428867 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63b4079d20 0x7f63b4107ec0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f63b00098a0 tx=0x7f63b0009bb0 comp rx=0 tx=0).stop 2023-07-18T02:57:31.792 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:31.298+0000 7f63bb859700 1 -- 172.21.15.101:0/403428867 shutdown_connections 2023-07-18T02:57:31.792 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:31.298+0000 7f63bb859700 1 --2- 172.21.15.101:0/403428867 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63b4079d20 0x7f63b4107ec0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:57:31.792 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:31.298+0000 7f63bb859700 1 -- 172.21.15.101:0/403428867 >> 172.21.15.101:0/403428867 conn(0x7f63b40ff890 msgr2=0x7f63b4101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:57:31.792 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:31.298+0000 7f63bb859700 1 -- 172.21.15.101:0/403428867 shutdown_connections 2023-07-18T02:57:31.793 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:31.298+0000 7f63bb859700 1 -- 172.21.15.101:0/403428867 wait complete. 2023-07-18T02:57:31.793 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:31.299+0000 7f63bb859700 1 Processor -- start 2023-07-18T02:57:31.793 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:31.299+0000 7f63bb859700 1 -- start start 2023-07-18T02:57:31.793 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:31.300+0000 7f63bb859700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63b419b2d0 0x7f63b419b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:57:31.793 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:31.300+0000 7f63bb859700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b419bc20 con 0x7f63b419b2d0 2023-07-18T02:57:31.793 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:31.300+0000 7f63b95f5700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63b419b2d0 0x7f63b419b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:57:31.793 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:31.300+0000 7f63b95f5700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63b419b2d0 0x7f63b419b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:58658/0 (socket says 172.21.15.101:58658) 2023-07-18T02:57:31.793 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:31.300+0000 7f63b95f5700 1 -- 172.21.15.101:0/974609039 learned_addr learned my addr 172.21.15.101:0/974609039 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:57:31.794 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:31.300+0000 7f63b95f5700 1 -- 172.21.15.101:0/974609039 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b0009550 con 0x7f63b419b2d0 2023-07-18T02:57:31.794 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:31.301+0000 7f63b95f5700 1 --2- 172.21.15.101:0/974609039 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63b419b2d0 0x7f63b419b6e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f63b0009fb0 tx=0x7f63b0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:57:31.794 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:31.301+0000 7f63aa7fc700 1 -- 172.21.15.101:0/974609039 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f63b0003a40 con 0x7f63b419b2d0 2023-07-18T02:57:31.794 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:31.301+0000 7f63aa7fc700 1 -- 172.21.15.101:0/974609039 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f63b0003ba0 con 0x7f63b419b2d0 2023-07-18T02:57:31.794 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:31.301+0000 7f63bb859700 1 -- 172.21.15.101:0/974609039 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63b419be20 con 0x7f63b419b2d0 2023-07-18T02:57:31.794 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:31.301+0000 7f63aa7fc700 1 -- 172.21.15.101:0/974609039 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f63b001a750 con 0x7f63b419b2d0 2023-07-18T02:57:31.794 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:31.301+0000 7f63bb859700 1 -- 172.21.15.101:0/974609039 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63b419ea80 con 0x7f63b419b2d0 2023-07-18T02:57:31.795 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:31.301+0000 7f63aa7fc700 1 -- 172.21.15.101:0/974609039 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f63b001ac80 con 0x7f63b419b2d0 2023-07-18T02:57:31.795 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:31.301+0000 7f63aa7fc700 1 -- 172.21.15.101:0/974609039 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f63b00193c0 con 0x7f63b419b2d0 2023-07-18T02:57:31.795 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:31.302+0000 7f63bb859700 1 -- 172.21.15.101:0/974609039 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63b4061b40 con 0x7f63b419b2d0 2023-07-18T02:57:31.795 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:31.305+0000 7f63aa7fc700 1 -- 172.21.15.101:0/974609039 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f63b0024b60 con 0x7f63b419b2d0 2023-07-18T02:57:31.795 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:31.380+0000 7f63bb859700 1 -- 172.21.15.101:0/974609039 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f63b4079d20 con 0x7f63b419b2d0 2023-07-18T02:57:31.795 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:31.381+0000 7f63aa7fc700 1 -- 172.21.15.101:0/974609039 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f63b0024d00 con 0x7f63b419b2d0 2023-07-18T02:57:31.795 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:31.382+0000 7f63bb859700 1 -- 172.21.15.101:0/974609039 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63b419b2d0 msgr2=0x7f63b419b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:57:31.795 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:31.382+0000 7f63bb859700 1 --2- 172.21.15.101:0/974609039 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63b419b2d0 0x7f63b419b6e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f63b0009fb0 tx=0x7f63b0011ad0 comp rx=0 tx=0).stop 2023-07-18T02:57:31.796 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:31.382+0000 7f63bb859700 1 -- 172.21.15.101:0/974609039 shutdown_connections 2023-07-18T02:57:31.796 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:31.382+0000 7f63bb859700 1 --2- 172.21.15.101:0/974609039 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63b419b2d0 0x7f63b419b6e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:57:31.796 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:31.382+0000 7f63bb859700 1 -- 172.21.15.101:0/974609039 >> 172.21.15.101:0/974609039 conn(0x7f63b40ff890 msgr2=0x7f63b4100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:57:31.796 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:31.382+0000 7f63bb859700 1 -- 172.21.15.101:0/974609039 shutdown_connections 2023-07-18T02:57:31.796 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:31.382+0000 7f63bb859700 1 -- 172.21.15.101:0/974609039 wait complete. 2023-07-18T02:57:31.796 INFO:teuthology.orchestra.run.smithi101.stdout:mgr not available, waiting (1/15)... 2023-07-18T02:57:32.032 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:31 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:31.781+0000 7f7632b37200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-18T02:57:32.033 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:31 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:31.901+0000 7f7632b37200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-18T02:57:32.467 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:32 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:32.059+0000 7f7632b37200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-18T02:57:32.468 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:32 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:32.180+0000 7f7632b37200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-18T02:57:32.967 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:32 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:32.721+0000 7f7632b37200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-18T02:57:33.967 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:33 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:33.497+0000 7f7632b37200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-18T02:57:33.968 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:33 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:33.703+0000 7f7632b37200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-18T02:57:34.885 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:34 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:34.661+0000 7f7632b37200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-18T02:57:34.885 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:34 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:34.780+0000 7f7632b37200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-18T02:57:35.217 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:35 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:35.010+0000 7f7632b37200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-18T02:57:35.619 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:35 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:35.371+0000 7f7632b37200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-18T02:57:35.967 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:35 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:35.646+0000 7f7632b37200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-18T02:57:35.968 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:35 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:35.836+0000 7f7632b37200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-18T02:57:36.467 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:36 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:36.177+0000 7f7632b37200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-18T02:57:36.967 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:36 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3183352585' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-18T02:57:37.467 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:37 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:37.105+0000 7f7632b37200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-18T02:57:37.468 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:37 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:37.278+0000 7f7632b37200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-18T02:57:37.919 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:37 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:37.523+0000 7f7632b37200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-18T02:57:37.919 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:37 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:37.671+0000 7f7632b37200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-18T02:57:38.217 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:37 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:37.946+0000 7f7632b37200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-18T02:57:39.467 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:39 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:39.011+0000 7f7632b37200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-18T02:57:39.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:39 smithi101 ceph-mon[126379]: Activating manager daemon x 2023-07-18T02:57:39.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:39 smithi101 ceph-mon[126379]: mgrmap e2: x(active, starting, since 0.00459242s) 2023-07-18T02:57:39.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:39 smithi101 ceph-mon[126379]: from='mgr.14100 172.21.15.101:0/3267960796' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-18T02:57:39.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:39 smithi101 ceph-mon[126379]: from='mgr.14100 172.21.15.101:0/3267960796' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-18T02:57:39.469 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:39 smithi101 ceph-mon[126379]: from='mgr.14100 172.21.15.101:0/3267960796' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-18T02:57:39.469 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:39 smithi101 ceph-mon[126379]: from='mgr.14100 172.21.15.101:0/3267960796' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-18T02:57:39.469 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:39 smithi101 ceph-mon[126379]: from='mgr.14100 172.21.15.101:0/3267960796' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-18T02:57:39.469 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:39 smithi101 ceph-mon[126379]: Manager daemon x is now available 2023-07-18T02:57:39.469 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:39 smithi101 ceph-mon[126379]: from='mgr.14100 172.21.15.101:0/3267960796' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-18T02:57:39.469 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:39 smithi101 ceph-mon[126379]: from='mgr.14100 172.21.15.101:0/3267960796' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-18T02:57:39.469 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:39 smithi101 ceph-mon[126379]: from='mgr.14100 172.21.15.101:0/3267960796' entity='mgr.x' 2023-07-18T02:57:39.469 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:39 smithi101 ceph-mon[126379]: from='mgr.14100 172.21.15.101:0/3267960796' entity='mgr.x' 2023-07-18T02:57:39.470 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:39 smithi101 ceph-mon[126379]: from='mgr.14100 172.21.15.101:0/3267960796' entity='mgr.x' 2023-07-18T02:57:41.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:41 smithi101 ceph-mon[126379]: mgrmap e3: x(active, since 1.01s) 2023-07-18T02:57:42.331 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:42 smithi101 ceph-mon[126379]: mgrmap e4: x(active, since 2s) 2023-07-18T02:57:42.381 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout 2023-07-18T02:57:42.383 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout { 2023-07-18T02:57:42.383 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "fsid": "b1782daa-2516-11ee-9b34-001a4aab830c", 2023-07-18T02:57:42.383 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "health": { 2023-07-18T02:57:42.383 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-18T02:57:42.385 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-18T02:57:42.385 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-18T02:57:42.385 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout }, 2023-07-18T02:57:42.385 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-18T02:57:42.385 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-18T02:57:42.385 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout 0 2023-07-18T02:57:42.386 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout ], 2023-07-18T02:57:42.386 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-18T02:57:42.386 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "a" 2023-07-18T02:57:42.386 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout ], 2023-07-18T02:57:42.386 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2023-07-18T02:57:42.386 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-18T02:57:42.386 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-18T02:57:42.387 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-18T02:57:42.387 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-18T02:57:42.387 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout }, 2023-07-18T02:57:42.387 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-18T02:57:42.387 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-18T02:57:42.387 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-18T02:57:42.387 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-18T02:57:42.393 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-18T02:57:42.393 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-18T02:57:42.393 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-18T02:57:42.393 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-18T02:57:42.394 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout }, 2023-07-18T02:57:42.394 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-18T02:57:42.394 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-18T02:57:42.394 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-18T02:57:42.394 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-18T02:57:42.394 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-18T02:57:42.394 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-18T02:57:42.394 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-18T02:57:42.395 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-18T02:57:42.395 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-18T02:57:42.395 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout }, 2023-07-18T02:57:42.395 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-18T02:57:42.395 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-18T02:57:42.395 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-18T02:57:42.395 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-18T02:57:42.396 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout }, 2023-07-18T02:57:42.396 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-18T02:57:42.396 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "available": false, 2023-07-18T02:57:42.396 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-18T02:57:42.396 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-18T02:57:42.396 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-18T02:57:42.396 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-18T02:57:42.396 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "restful" 2023-07-18T02:57:42.397 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout ], 2023-07-18T02:57:42.397 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-18T02:57:42.397 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout }, 2023-07-18T02:57:42.397 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-18T02:57:42.397 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-18T02:57:42.397 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "modified": "2023-07-18T02:57:22.972928+0000", 2023-07-18T02:57:42.397 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-18T02:57:42.398 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout }, 2023-07-18T02:57:42.398 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-18T02:57:42.398 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout } 2023-07-18T02:57:42.398 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:36.413+0000 7f2446f29700 1 Processor -- start 2023-07-18T02:57:42.398 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:36.413+0000 7f2446f29700 1 -- start start 2023-07-18T02:57:42.398 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:36.413+0000 7f2446f29700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f244007be00 0x7f244007a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:57:42.399 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:36.413+0000 7f2446f29700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244007a840 con 0x7f244007be00 2023-07-18T02:57:42.399 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:36.414+0000 7f2444cc5700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f244007be00 0x7f244007a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:57:42.399 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:36.414+0000 7f2444cc5700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f244007be00 0x7f244007a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:33058/0 (socket says 172.21.15.101:33058) 2023-07-18T02:57:42.399 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:36.414+0000 7f2444cc5700 1 -- 172.21.15.101:0/4061386372 learned_addr learned my addr 172.21.15.101:0/4061386372 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:57:42.399 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:36.414+0000 7f2444cc5700 1 -- 172.21.15.101:0/4061386372 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f244007a980 con 0x7f244007be00 2023-07-18T02:57:42.399 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:36.414+0000 7f2444cc5700 1 --2- 172.21.15.101:0/4061386372 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f244007be00 0x7f244007a300 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f24340098a0 tx=0x7f2434009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b45e03183ee7d36 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:57:42.399 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:36.415+0000 7f243f7fe700 1 -- 172.21.15.101:0/4061386372 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2434004160 con 0x7f244007be00 2023-07-18T02:57:42.400 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:36.415+0000 7f243f7fe700 1 -- 172.21.15.101:0/4061386372 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f24340042c0 con 0x7f244007be00 2023-07-18T02:57:42.400 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:36.415+0000 7f243f7fe700 1 -- 172.21.15.101:0/4061386372 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2434011a00 con 0x7f244007be00 2023-07-18T02:57:42.400 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:36.415+0000 7f2446f29700 1 -- 172.21.15.101:0/4061386372 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f244007be00 msgr2=0x7f244007a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:57:42.400 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:36.415+0000 7f2446f29700 1 --2- 172.21.15.101:0/4061386372 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f244007be00 0x7f244007a300 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f24340098a0 tx=0x7f2434009bb0 comp rx=0 tx=0).stop 2023-07-18T02:57:42.400 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:36.415+0000 7f2446f29700 1 -- 172.21.15.101:0/4061386372 shutdown_connections 2023-07-18T02:57:42.400 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:36.415+0000 7f2446f29700 1 --2- 172.21.15.101:0/4061386372 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f244007be00 0x7f244007a300 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:57:42.401 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:36.415+0000 7f2446f29700 1 -- 172.21.15.101:0/4061386372 >> 172.21.15.101:0/4061386372 conn(0x7f2440100970 msgr2=0x7f2440102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:57:42.401 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:36.416+0000 7f2446f29700 1 -- 172.21.15.101:0/4061386372 shutdown_connections 2023-07-18T02:57:42.401 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:36.416+0000 7f2446f29700 1 -- 172.21.15.101:0/4061386372 wait complete. 2023-07-18T02:57:42.401 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:36.416+0000 7f2446f29700 1 Processor -- start 2023-07-18T02:57:42.401 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:36.417+0000 7f2446f29700 1 -- start start 2023-07-18T02:57:42.401 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:36.417+0000 7f2446f29700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f244007be00 0x7f244019b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:57:42.402 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:36.417+0000 7f2446f29700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244019b880 con 0x7f244007be00 2023-07-18T02:57:42.402 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:36.417+0000 7f2444cc5700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f244007be00 0x7f244019b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:57:42.402 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:36.417+0000 7f2444cc5700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f244007be00 0x7f244019b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:33060/0 (socket says 172.21.15.101:33060) 2023-07-18T02:57:42.402 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:36.417+0000 7f2444cc5700 1 -- 172.21.15.101:0/3183352585 learned_addr learned my addr 172.21.15.101:0/3183352585 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:57:42.402 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:36.417+0000 7f2444cc5700 1 -- 172.21.15.101:0/3183352585 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2434009550 con 0x7f244007be00 2023-07-18T02:57:42.402 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:36.418+0000 7f2444cc5700 1 --2- 172.21.15.101:0/3183352585 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f244007be00 0x7f244019b340 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2434009130 tx=0x7f2434011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:57:42.402 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:36.418+0000 7f243dffb700 1 -- 172.21.15.101:0/3183352585 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2434003ab0 con 0x7f244007be00 2023-07-18T02:57:42.402 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:36.418+0000 7f243dffb700 1 -- 172.21.15.101:0/3183352585 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2434003c10 con 0x7f244007be00 2023-07-18T02:57:42.403 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:36.418+0000 7f243dffb700 1 -- 172.21.15.101:0/3183352585 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24340247d0 con 0x7f244007be00 2023-07-18T02:57:42.403 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:36.418+0000 7f2446f29700 1 -- 172.21.15.101:0/3183352585 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f244019ba80 con 0x7f244007be00 2023-07-18T02:57:42.403 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:36.418+0000 7f2446f29700 1 -- 172.21.15.101:0/3183352585 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f244019bea0 con 0x7f244007be00 2023-07-18T02:57:42.403 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:36.418+0000 7f243dffb700 1 -- 172.21.15.101:0/3183352585 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2434024d00 con 0x7f244007be00 2023-07-18T02:57:42.403 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:36.418+0000 7f243dffb700 1 -- 172.21.15.101:0/3183352585 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f2434019450 con 0x7f244007be00 2023-07-18T02:57:42.403 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:36.419+0000 7f2446f29700 1 -- 172.21.15.101:0/3183352585 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24401952f0 con 0x7f244007be00 2023-07-18T02:57:42.403 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:36.422+0000 7f243dffb700 1 -- 172.21.15.101:0/3183352585 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f243401abb0 con 0x7f244007be00 2023-07-18T02:57:42.404 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:36.497+0000 7f2446f29700 1 -- 172.21.15.101:0/3183352585 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f244002c850 con 0x7f244007be00 2023-07-18T02:57:42.404 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:36.497+0000 7f243dffb700 1 -- 172.21.15.101:0/3183352585 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f243401ad50 con 0x7f244007be00 2023-07-18T02:57:42.404 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:36.498+0000 7f2446f29700 1 -- 172.21.15.101:0/3183352585 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f244007be00 msgr2=0x7f244019b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:57:42.404 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:36.498+0000 7f2446f29700 1 --2- 172.21.15.101:0/3183352585 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f244007be00 0x7f244019b340 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2434009130 tx=0x7f2434011ad0 comp rx=0 tx=0).stop 2023-07-18T02:57:42.404 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:36.498+0000 7f2446f29700 1 -- 172.21.15.101:0/3183352585 shutdown_connections 2023-07-18T02:57:42.404 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:36.498+0000 7f2446f29700 1 --2- 172.21.15.101:0/3183352585 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f244007be00 0x7f244019b340 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:57:42.404 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:36.498+0000 7f2446f29700 1 -- 172.21.15.101:0/3183352585 >> 172.21.15.101:0/3183352585 conn(0x7f2440100970 msgr2=0x7f24401020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:57:42.405 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:36.498+0000 7f2446f29700 1 -- 172.21.15.101:0/3183352585 shutdown_connections 2023-07-18T02:57:42.405 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:36.498+0000 7f2446f29700 1 -- 172.21.15.101:0/3183352585 wait complete. 2023-07-18T02:57:42.405 INFO:teuthology.orchestra.run.smithi101.stdout:mgr not available, waiting (2/15)... 2023-07-18T02:57:45.648 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:45 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3277722299' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-18T02:57:45.696 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout 2023-07-18T02:57:45.696 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout { 2023-07-18T02:57:45.697 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "fsid": "b1782daa-2516-11ee-9b34-001a4aab830c", 2023-07-18T02:57:45.697 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "health": { 2023-07-18T02:57:45.697 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-18T02:57:45.697 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-18T02:57:45.698 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-18T02:57:45.698 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout }, 2023-07-18T02:57:45.698 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-18T02:57:45.698 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-18T02:57:45.698 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout 0 2023-07-18T02:57:45.698 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout ], 2023-07-18T02:57:45.698 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-18T02:57:45.698 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "a" 2023-07-18T02:57:45.699 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout ], 2023-07-18T02:57:45.699 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "quorum_age": 17, 2023-07-18T02:57:45.699 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-18T02:57:45.699 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-18T02:57:45.699 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-18T02:57:45.699 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-18T02:57:45.699 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout }, 2023-07-18T02:57:45.700 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-18T02:57:45.700 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-18T02:57:45.700 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-18T02:57:45.700 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-18T02:57:45.700 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-18T02:57:45.700 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-18T02:57:45.700 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-18T02:57:45.701 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-18T02:57:45.701 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout }, 2023-07-18T02:57:45.701 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-18T02:57:45.701 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-18T02:57:45.701 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-18T02:57:45.701 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-18T02:57:45.702 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-18T02:57:45.702 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-18T02:57:45.702 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-18T02:57:45.702 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-18T02:57:45.702 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-18T02:57:45.702 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout }, 2023-07-18T02:57:45.702 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-18T02:57:45.703 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-18T02:57:45.703 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-18T02:57:45.703 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-18T02:57:45.703 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout }, 2023-07-18T02:57:45.703 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-18T02:57:45.703 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "available": true, 2023-07-18T02:57:45.703 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-18T02:57:45.703 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-18T02:57:45.704 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-18T02:57:45.704 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-18T02:57:45.704 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "restful" 2023-07-18T02:57:45.704 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout ], 2023-07-18T02:57:45.704 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-18T02:57:45.704 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout }, 2023-07-18T02:57:45.704 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-18T02:57:45.705 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-18T02:57:45.705 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "modified": "2023-07-18T02:57:22.972928+0000", 2023-07-18T02:57:45.705 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-18T02:57:45.705 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout }, 2023-07-18T02:57:45.705 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-18T02:57:45.705 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout } 2023-07-18T02:57:45.705 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:44.961+0000 7f5f22eaa700 1 Processor -- start 2023-07-18T02:57:45.705 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:44.961+0000 7f5f22eaa700 1 -- start start 2023-07-18T02:57:45.706 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:44.962+0000 7f5f22eaa700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5f1c105320 0x7f5f1c105730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:57:45.706 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:44.962+0000 7f5f22eaa700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c105c70 con 0x7f5f1c105320 2023-07-18T02:57:45.706 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:44.962+0000 7f5f20c46700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5f1c105320 0x7f5f1c105730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:57:45.706 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:44.962+0000 7f5f20c46700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5f1c105320 0x7f5f1c105730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:33130/0 (socket says 172.21.15.101:33130) 2023-07-18T02:57:45.706 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:44.962+0000 7f5f20c46700 1 -- 172.21.15.101:0/618541402 learned_addr learned my addr 172.21.15.101:0/618541402 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:57:45.706 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:44.962+0000 7f5f20c46700 1 -- 172.21.15.101:0/618541402 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f1c105db0 con 0x7f5f1c105320 2023-07-18T02:57:45.706 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:44.963+0000 7f5f20c46700 1 --2- 172.21.15.101:0/618541402 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5f1c105320 0x7f5f1c105730 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5f100098a0 tx=0x7f5f10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b387d818e8dc0792 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:57:45.707 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:44.963+0000 7f5f1b7fe700 1 -- 172.21.15.101:0/618541402 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f10004160 con 0x7f5f1c105320 2023-07-18T02:57:45.707 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:44.963+0000 7f5f1b7fe700 1 -- 172.21.15.101:0/618541402 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5f100042c0 con 0x7f5f1c105320 2023-07-18T02:57:45.707 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:44.963+0000 7f5f1b7fe700 1 -- 172.21.15.101:0/618541402 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f10011a00 con 0x7f5f1c105320 2023-07-18T02:57:45.707 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:44.963+0000 7f5f22eaa700 1 -- 172.21.15.101:0/618541402 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5f1c105320 msgr2=0x7f5f1c105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:57:45.707 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:44.963+0000 7f5f22eaa700 1 --2- 172.21.15.101:0/618541402 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5f1c105320 0x7f5f1c105730 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5f100098a0 tx=0x7f5f10009bb0 comp rx=0 tx=0).stop 2023-07-18T02:57:45.707 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:44.964+0000 7f5f22eaa700 1 -- 172.21.15.101:0/618541402 shutdown_connections 2023-07-18T02:57:45.707 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:44.964+0000 7f5f22eaa700 1 --2- 172.21.15.101:0/618541402 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5f1c105320 0x7f5f1c105730 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:57:45.708 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:44.964+0000 7f5f22eaa700 1 -- 172.21.15.101:0/618541402 >> 172.21.15.101:0/618541402 conn(0x7f5f1c100970 msgr2=0x7f5f1c102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:57:45.708 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:44.964+0000 7f5f22eaa700 1 -- 172.21.15.101:0/618541402 shutdown_connections 2023-07-18T02:57:45.708 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:44.964+0000 7f5f22eaa700 1 -- 172.21.15.101:0/618541402 wait complete. 2023-07-18T02:57:45.708 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:44.965+0000 7f5f22eaa700 1 Processor -- start 2023-07-18T02:57:45.708 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:44.965+0000 7f5f22eaa700 1 -- start start 2023-07-18T02:57:45.708 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:44.965+0000 7f5f22eaa700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5f1c105320 0x7f5f1c19b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:57:45.709 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:44.965+0000 7f5f22eaa700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c19b820 con 0x7f5f1c105320 2023-07-18T02:57:45.709 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:44.965+0000 7f5f20c46700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5f1c105320 0x7f5f1c19b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:57:45.709 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:44.965+0000 7f5f20c46700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5f1c105320 0x7f5f1c19b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:33138/0 (socket says 172.21.15.101:33138) 2023-07-18T02:57:45.709 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:44.965+0000 7f5f20c46700 1 -- 172.21.15.101:0/3277722299 learned_addr learned my addr 172.21.15.101:0/3277722299 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:57:45.709 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:44.966+0000 7f5f20c46700 1 -- 172.21.15.101:0/3277722299 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f10009550 con 0x7f5f1c105320 2023-07-18T02:57:45.709 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:44.966+0000 7f5f20c46700 1 --2- 172.21.15.101:0/3277722299 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5f1c105320 0x7f5f1c19b2e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5f10009130 tx=0x7f5f10011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:57:45.709 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:44.966+0000 7f5f19ffb700 1 -- 172.21.15.101:0/3277722299 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f10003b70 con 0x7f5f1c105320 2023-07-18T02:57:45.709 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:44.966+0000 7f5f19ffb700 1 -- 172.21.15.101:0/3277722299 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5f10003cd0 con 0x7f5f1c105320 2023-07-18T02:57:45.710 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:44.966+0000 7f5f22eaa700 1 -- 172.21.15.101:0/3277722299 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f1c19ba20 con 0x7f5f1c105320 2023-07-18T02:57:45.710 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:44.966+0000 7f5f19ffb700 1 -- 172.21.15.101:0/3277722299 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f100197d0 con 0x7f5f1c105320 2023-07-18T02:57:45.710 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:44.966+0000 7f5f22eaa700 1 -- 172.21.15.101:0/3277722299 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f1c19be40 con 0x7f5f1c105320 2023-07-18T02:57:45.710 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:44.967+0000 7f5f22eaa700 1 -- 172.21.15.101:0/3277722299 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f1c1097b0 con 0x7f5f1c105320 2023-07-18T02:57:45.710 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:44.967+0000 7f5f19ffb700 1 -- 172.21.15.101:0/3277722299 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7f5f1001abb0 con 0x7f5f1c105320 2023-07-18T02:57:45.710 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:44.968+0000 7f5f19ffb700 1 --2- 172.21.15.101:0/3277722299 >> [v2:172.21.15.101:6800/3497914514,v1:172.21.15.101:6801/3497914514] conn(0x7f5f0c038710 0x7f5f0c03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:57:45.710 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:44.968+0000 7f5f19ffb700 1 -- 172.21.15.101:0/3277722299 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5f1004c840 con 0x7f5f1c105320 2023-07-18T02:57:45.711 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:44.968+0000 7f5f1bfff700 1 --2- 172.21.15.101:0/3277722299 >> [v2:172.21.15.101:6800/3497914514,v1:172.21.15.101:6801/3497914514] conn(0x7f5f0c038710 0x7f5f0c03abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:57:45.711 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:44.969+0000 7f5f1bfff700 1 --2- 172.21.15.101:0/3277722299 >> [v2:172.21.15.101:6800/3497914514,v1:172.21.15.101:6801/3497914514] conn(0x7f5f0c038710 0x7f5f0c03abc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5f08006fd0 tx=0x7f5f08006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:57:45.711 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:44.973+0000 7f5f19ffb700 1 -- 172.21.15.101:0/3277722299 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5f1001a430 con 0x7f5f1c105320 2023-07-18T02:57:45.711 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:45.241+0000 7f5f22eaa700 1 -- 172.21.15.101:0/3277722299 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f5f1c061b40 con 0x7f5f1c105320 2023-07-18T02:57:45.711 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:45.242+0000 7f5f19ffb700 1 -- 172.21.15.101:0/3277722299 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f5f1002db00 con 0x7f5f1c105320 2023-07-18T02:57:45.711 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:45.244+0000 7f5f22eaa700 1 -- 172.21.15.101:0/3277722299 >> [v2:172.21.15.101:6800/3497914514,v1:172.21.15.101:6801/3497914514] conn(0x7f5f0c038710 msgr2=0x7f5f0c03abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:57:45.711 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:45.244+0000 7f5f22eaa700 1 --2- 172.21.15.101:0/3277722299 >> [v2:172.21.15.101:6800/3497914514,v1:172.21.15.101:6801/3497914514] conn(0x7f5f0c038710 0x7f5f0c03abc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5f08006fd0 tx=0x7f5f08006e40 comp rx=0 tx=0).stop 2023-07-18T02:57:45.712 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:45.244+0000 7f5f22eaa700 1 -- 172.21.15.101:0/3277722299 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5f1c105320 msgr2=0x7f5f1c19b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:57:45.712 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:45.244+0000 7f5f22eaa700 1 --2- 172.21.15.101:0/3277722299 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5f1c105320 0x7f5f1c19b2e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5f10009130 tx=0x7f5f10011ad0 comp rx=0 tx=0).stop 2023-07-18T02:57:45.712 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:45.244+0000 7f5f22eaa700 1 -- 172.21.15.101:0/3277722299 shutdown_connections 2023-07-18T02:57:45.712 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:45.244+0000 7f5f22eaa700 1 --2- 172.21.15.101:0/3277722299 >> [v2:172.21.15.101:6800/3497914514,v1:172.21.15.101:6801/3497914514] conn(0x7f5f0c038710 0x7f5f0c03abc0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:57:45.712 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:45.244+0000 7f5f22eaa700 1 --2- 172.21.15.101:0/3277722299 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5f1c105320 0x7f5f1c19b2e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:57:45.712 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:45.244+0000 7f5f22eaa700 1 -- 172.21.15.101:0/3277722299 >> 172.21.15.101:0/3277722299 conn(0x7f5f1c100970 msgr2=0x7f5f1c102040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:57:45.712 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:45.245+0000 7f5f22eaa700 1 -- 172.21.15.101:0/3277722299 shutdown_connections 2023-07-18T02:57:45.713 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:45.245+0000 7f5f22eaa700 1 -- 172.21.15.101:0/3277722299 wait complete. 2023-07-18T02:57:45.713 INFO:teuthology.orchestra.run.smithi101.stdout:mgr is available 2023-07-18T02:57:46.717 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:46 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/269644380' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-07-18T02:57:46.864 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout 2023-07-18T02:57:46.864 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout [global] 2023-07-18T02:57:46.864 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout fsid = b1782daa-2516-11ee-9b34-001a4aab830c 2023-07-18T02:57:46.864 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-07-18T02:57:46.864 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-07-18T02:57:46.865 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-07-18T02:57:46.865 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-07-18T02:57:46.865 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout 2023-07-18T02:57:46.865 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout [mgr] 2023-07-18T02:57:46.865 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-07-18T02:57:46.865 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout 2023-07-18T02:57:46.865 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout [osd] 2023-07-18T02:57:46.865 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-07-18T02:57:46.866 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-07-18T02:57:46.866 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-07-18T02:57:46.866 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-07-18T02:57:46.866 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-07-18T02:57:46.866 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.245+0000 7fe711db9700 1 Processor -- start 2023-07-18T02:57:46.866 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.245+0000 7fe711db9700 1 -- start start 2023-07-18T02:57:46.866 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.245+0000 7fe711db9700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe70c105320 0x7fe70c105730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:57:46.867 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.245+0000 7fe711db9700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70c105c70 con 0x7fe70c105320 2023-07-18T02:57:46.867 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.246+0000 7fe70b7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe70c105320 0x7fe70c105730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:57:46.867 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.246+0000 7fe70b7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe70c105320 0x7fe70c105730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:56790/0 (socket says 172.21.15.101:56790) 2023-07-18T02:57:46.867 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.246+0000 7fe70b7fe700 1 -- 172.21.15.101:0/3655410197 learned_addr learned my addr 172.21.15.101:0/3655410197 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:57:46.867 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.246+0000 7fe70b7fe700 1 -- 172.21.15.101:0/3655410197 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe70c105db0 con 0x7fe70c105320 2023-07-18T02:57:46.867 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.246+0000 7fe70b7fe700 1 --2- 172.21.15.101:0/3655410197 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe70c105320 0x7fe70c105730 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc0098a0 tx=0x7fe6fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6610a5fc85c1e16d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:57:46.867 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.246+0000 7fe70a7fc700 1 -- 172.21.15.101:0/3655410197 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe6fc004160 con 0x7fe70c105320 2023-07-18T02:57:46.868 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.246+0000 7fe70a7fc700 1 -- 172.21.15.101:0/3655410197 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe6fc0042c0 con 0x7fe70c105320 2023-07-18T02:57:46.868 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.246+0000 7fe70a7fc700 1 -- 172.21.15.101:0/3655410197 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe6fc011a00 con 0x7fe70c105320 2023-07-18T02:57:46.868 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.247+0000 7fe711db9700 1 -- 172.21.15.101:0/3655410197 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe70c105320 msgr2=0x7fe70c105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:57:46.868 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.247+0000 7fe711db9700 1 --2- 172.21.15.101:0/3655410197 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe70c105320 0x7fe70c105730 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc0098a0 tx=0x7fe6fc009bb0 comp rx=0 tx=0).stop 2023-07-18T02:57:46.868 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.247+0000 7fe711db9700 1 -- 172.21.15.101:0/3655410197 shutdown_connections 2023-07-18T02:57:46.868 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.247+0000 7fe711db9700 1 --2- 172.21.15.101:0/3655410197 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe70c105320 0x7fe70c105730 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:57:46.868 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.247+0000 7fe711db9700 1 -- 172.21.15.101:0/3655410197 >> 172.21.15.101:0/3655410197 conn(0x7fe70c100970 msgr2=0x7fe70c102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:57:46.869 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.247+0000 7fe711db9700 1 -- 172.21.15.101:0/3655410197 shutdown_connections 2023-07-18T02:57:46.869 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.247+0000 7fe711db9700 1 -- 172.21.15.101:0/3655410197 wait complete. 2023-07-18T02:57:46.869 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.248+0000 7fe711db9700 1 Processor -- start 2023-07-18T02:57:46.869 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.248+0000 7fe711db9700 1 -- start start 2023-07-18T02:57:46.869 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.249+0000 7fe711db9700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe70c105320 0x7fe70c19b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:57:46.869 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.249+0000 7fe711db9700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70c19b820 con 0x7fe70c105320 2023-07-18T02:57:46.869 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.249+0000 7fe70b7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe70c105320 0x7fe70c19b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:57:46.870 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.249+0000 7fe70b7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe70c105320 0x7fe70c19b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:56798/0 (socket says 172.21.15.101:56798) 2023-07-18T02:57:46.870 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.249+0000 7fe70b7fe700 1 -- 172.21.15.101:0/269644380 learned_addr learned my addr 172.21.15.101:0/269644380 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:57:46.870 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.249+0000 7fe70b7fe700 1 -- 172.21.15.101:0/269644380 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6fc009550 con 0x7fe70c105320 2023-07-18T02:57:46.870 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.249+0000 7fe70b7fe700 1 --2- 172.21.15.101:0/269644380 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe70c105320 0x7fe70c19b2e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc009130 tx=0x7fe6fc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:57:46.870 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.249+0000 7fe708ff9700 1 -- 172.21.15.101:0/269644380 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe6fc003b70 con 0x7fe70c105320 2023-07-18T02:57:46.870 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.250+0000 7fe711db9700 1 -- 172.21.15.101:0/269644380 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe70c19ba20 con 0x7fe70c105320 2023-07-18T02:57:46.870 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.250+0000 7fe708ff9700 1 -- 172.21.15.101:0/269644380 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe6fc003cd0 con 0x7fe70c105320 2023-07-18T02:57:46.870 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.250+0000 7fe708ff9700 1 -- 172.21.15.101:0/269644380 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe6fc0197d0 con 0x7fe70c105320 2023-07-18T02:57:46.871 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.250+0000 7fe711db9700 1 -- 172.21.15.101:0/269644380 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe70c19be40 con 0x7fe70c105320 2023-07-18T02:57:46.871 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.251+0000 7fe708ff9700 1 -- 172.21.15.101:0/269644380 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7fe6fc01a430 con 0x7fe70c105320 2023-07-18T02:57:46.871 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.251+0000 7fe711db9700 1 -- 172.21.15.101:0/269644380 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe70c1097b0 con 0x7fe70c105320 2023-07-18T02:57:46.871 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.251+0000 7fe708ff9700 1 --2- 172.21.15.101:0/269644380 >> [v2:172.21.15.101:6800/3497914514,v1:172.21.15.101:6801/3497914514] conn(0x7fe6f40386c0 0x7fe6f403ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:57:46.871 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.251+0000 7fe708ff9700 1 -- 172.21.15.101:0/269644380 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe6fc04c890 con 0x7fe70c105320 2023-07-18T02:57:46.871 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.251+0000 7fe70affd700 1 --2- 172.21.15.101:0/269644380 >> [v2:172.21.15.101:6800/3497914514,v1:172.21.15.101:6801/3497914514] conn(0x7fe6f40386c0 0x7fe6f403ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:57:46.871 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.252+0000 7fe70affd700 1 --2- 172.21.15.101:0/269644380 >> [v2:172.21.15.101:6800/3497914514,v1:172.21.15.101:6801/3497914514] conn(0x7fe6f40386c0 0x7fe6f403ab70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe700006fd0 tx=0x7fe700006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:57:46.872 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.256+0000 7fe708ff9700 1 -- 172.21.15.101:0/269644380 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe6fc02dd90 con 0x7fe70c105320 2023-07-18T02:57:46.872 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.447+0000 7fe711db9700 1 -- 172.21.15.101:0/269644380 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fe70c061b40 con 0x7fe70c105320 2023-07-18T02:57:46.872 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.448+0000 7fe708ff9700 1 -- 172.21.15.101:0/269644380 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fe6fc02d350 con 0x7fe70c105320 2023-07-18T02:57:46.872 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.450+0000 7fe711db9700 1 -- 172.21.15.101:0/269644380 >> [v2:172.21.15.101:6800/3497914514,v1:172.21.15.101:6801/3497914514] conn(0x7fe6f40386c0 msgr2=0x7fe6f403ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:57:46.872 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.450+0000 7fe711db9700 1 --2- 172.21.15.101:0/269644380 >> [v2:172.21.15.101:6800/3497914514,v1:172.21.15.101:6801/3497914514] conn(0x7fe6f40386c0 0x7fe6f403ab70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe700006fd0 tx=0x7fe700006e40 comp rx=0 tx=0).stop 2023-07-18T02:57:46.872 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.450+0000 7fe711db9700 1 -- 172.21.15.101:0/269644380 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe70c105320 msgr2=0x7fe70c19b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:57:46.872 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.450+0000 7fe711db9700 1 --2- 172.21.15.101:0/269644380 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe70c105320 0x7fe70c19b2e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc009130 tx=0x7fe6fc011ad0 comp rx=0 tx=0).stop 2023-07-18T02:57:46.873 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.451+0000 7fe711db9700 1 -- 172.21.15.101:0/269644380 shutdown_connections 2023-07-18T02:57:46.873 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.451+0000 7fe711db9700 1 --2- 172.21.15.101:0/269644380 >> [v2:172.21.15.101:6800/3497914514,v1:172.21.15.101:6801/3497914514] conn(0x7fe6f40386c0 0x7fe6f403ab70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:57:46.873 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.451+0000 7fe711db9700 1 --2- 172.21.15.101:0/269644380 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe70c105320 0x7fe70c19b2e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:57:46.873 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.451+0000 7fe711db9700 1 -- 172.21.15.101:0/269644380 >> 172.21.15.101:0/269644380 conn(0x7fe70c100970 msgr2=0x7fe70c102080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:57:46.873 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.451+0000 7fe711db9700 1 -- 172.21.15.101:0/269644380 shutdown_connections 2023-07-18T02:57:46.873 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:46.451+0000 7fe711db9700 1 -- 172.21.15.101:0/269644380 wait complete. 2023-07-18T02:57:46.873 INFO:teuthology.orchestra.run.smithi101.stdout:Enabling cephadm module... 2023-07-18T02:57:48.145 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:47 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3306514257' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-07-18T02:57:48.145 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:48 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: ignoring --setuser ceph since I am not root 2023-07-18T02:57:48.145 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:48 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: ignoring --setgroup ceph since I am not root 2023-07-18T02:57:48.344 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.587+0000 7f21bc169700 1 Processor -- start 2023-07-18T02:57:48.344 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.587+0000 7f21bc169700 1 -- start start 2023-07-18T02:57:48.344 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.587+0000 7f21bc169700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f21b407be40 0x7f21b407a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:57:48.345 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.587+0000 7f21bc169700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b407a880 con 0x7f21b407be40 2023-07-18T02:57:48.345 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.588+0000 7f21b9f05700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f21b407be40 0x7f21b407a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:57:48.345 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.588+0000 7f21b9f05700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f21b407be40 0x7f21b407a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:56812/0 (socket says 172.21.15.101:56812) 2023-07-18T02:57:48.345 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.588+0000 7f21b9f05700 1 -- 172.21.15.101:0/793313893 learned_addr learned my addr 172.21.15.101:0/793313893 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:57:48.345 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.588+0000 7f21b9f05700 1 -- 172.21.15.101:0/793313893 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b407a9c0 con 0x7f21b407be40 2023-07-18T02:57:48.345 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.588+0000 7f21b9f05700 1 --2- 172.21.15.101:0/793313893 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f21b407be40 0x7f21b407a340 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f21b00098a0 tx=0x7f21b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6c5ae10fd3ad32d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:57:48.345 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.588+0000 7f21b8f03700 1 -- 172.21.15.101:0/793313893 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f21b0004160 con 0x7f21b407be40 2023-07-18T02:57:48.346 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.589+0000 7f21b8f03700 1 -- 172.21.15.101:0/793313893 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f21b00042c0 con 0x7f21b407be40 2023-07-18T02:57:48.346 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.589+0000 7f21b8f03700 1 -- 172.21.15.101:0/793313893 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f21b0011a00 con 0x7f21b407be40 2023-07-18T02:57:48.346 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.589+0000 7f21bc169700 1 -- 172.21.15.101:0/793313893 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f21b407be40 msgr2=0x7f21b407a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:57:48.346 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.589+0000 7f21bc169700 1 --2- 172.21.15.101:0/793313893 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f21b407be40 0x7f21b407a340 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f21b00098a0 tx=0x7f21b0009bb0 comp rx=0 tx=0).stop 2023-07-18T02:57:48.346 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.589+0000 7f21bc169700 1 -- 172.21.15.101:0/793313893 shutdown_connections 2023-07-18T02:57:48.346 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.589+0000 7f21bc169700 1 --2- 172.21.15.101:0/793313893 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f21b407be40 0x7f21b407a340 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:57:48.346 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.589+0000 7f21bc169700 1 -- 172.21.15.101:0/793313893 >> 172.21.15.101:0/793313893 conn(0x7f21b4100970 msgr2=0x7f21b4102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:57:48.347 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.589+0000 7f21bc169700 1 -- 172.21.15.101:0/793313893 shutdown_connections 2023-07-18T02:57:48.347 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.589+0000 7f21bc169700 1 -- 172.21.15.101:0/793313893 wait complete. 2023-07-18T02:57:48.347 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.590+0000 7f21bc169700 1 Processor -- start 2023-07-18T02:57:48.347 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.590+0000 7f21bc169700 1 -- start start 2023-07-18T02:57:48.347 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.591+0000 7f21bc169700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f21b407be40 0x7f21b419b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:57:48.347 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.591+0000 7f21bc169700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b419b860 con 0x7f21b407be40 2023-07-18T02:57:48.347 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.591+0000 7f21b9f05700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f21b407be40 0x7f21b419b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:57:48.348 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.591+0000 7f21b9f05700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f21b407be40 0x7f21b419b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:56814/0 (socket says 172.21.15.101:56814) 2023-07-18T02:57:48.348 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.591+0000 7f21b9f05700 1 -- 172.21.15.101:0/3306514257 learned_addr learned my addr 172.21.15.101:0/3306514257 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:57:48.348 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.591+0000 7f21b9f05700 1 -- 172.21.15.101:0/3306514257 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b0009550 con 0x7f21b407be40 2023-07-18T02:57:48.348 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.591+0000 7f21b9f05700 1 --2- 172.21.15.101:0/3306514257 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f21b407be40 0x7f21b419b320 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f21b0009130 tx=0x7f21b0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:57:48.348 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.592+0000 7f21aaffd700 1 -- 172.21.15.101:0/3306514257 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f21b0003ab0 con 0x7f21b407be40 2023-07-18T02:57:48.348 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.592+0000 7f21aaffd700 1 -- 172.21.15.101:0/3306514257 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f21b0003c10 con 0x7f21b407be40 2023-07-18T02:57:48.348 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.592+0000 7f21aaffd700 1 -- 172.21.15.101:0/3306514257 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f21b00247d0 con 0x7f21b407be40 2023-07-18T02:57:48.349 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.592+0000 7f21bc169700 1 -- 172.21.15.101:0/3306514257 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21b419ba60 con 0x7f21b407be40 2023-07-18T02:57:48.349 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.592+0000 7f21bc169700 1 -- 172.21.15.101:0/3306514257 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21b419be80 con 0x7f21b407be40 2023-07-18T02:57:48.349 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.593+0000 7f21aaffd700 1 -- 172.21.15.101:0/3306514257 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7f21b001abb0 con 0x7f21b407be40 2023-07-18T02:57:48.349 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.593+0000 7f21bc169700 1 -- 172.21.15.101:0/3306514257 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21b4195300 con 0x7f21b407be40 2023-07-18T02:57:48.349 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.593+0000 7f21aaffd700 1 --2- 172.21.15.101:0/3306514257 >> [v2:172.21.15.101:6800/3497914514,v1:172.21.15.101:6801/3497914514] conn(0x7f21a0038710 0x7f21a003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:57:48.349 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.593+0000 7f21aaffd700 1 -- 172.21.15.101:0/3306514257 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f21b004c430 con 0x7f21b407be40 2023-07-18T02:57:48.349 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.593+0000 7f21b9704700 1 --2- 172.21.15.101:0/3306514257 >> [v2:172.21.15.101:6800/3497914514,v1:172.21.15.101:6801/3497914514] conn(0x7f21a0038710 0x7f21a003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:57:48.349 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.594+0000 7f21b9704700 1 --2- 172.21.15.101:0/3306514257 >> [v2:172.21.15.101:6800/3497914514,v1:172.21.15.101:6801/3497914514] conn(0x7f21a0038710 0x7f21a003abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f21a4006fd0 tx=0x7f21a4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:57:48.350 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.598+0000 7f21aaffd700 1 -- 172.21.15.101:0/3306514257 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f21b0018710 con 0x7f21b407be40 2023-07-18T02:57:48.350 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.843+0000 7f21bc169700 1 -- 172.21.15.101:0/3306514257 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f21b402c850 con 0x7f21b407be40 2023-07-18T02:57:48.350 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.896+0000 7f21aaffd700 1 -- 172.21.15.101:0/3306514257 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f21b002d770 con 0x7f21b407be40 2023-07-18T02:57:48.350 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.898+0000 7f21aaffd700 1 -- 172.21.15.101:0/3306514257 <== mon.0 v2:172.21.15.101:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f21b004c190 con 0x7f21b407be40 2023-07-18T02:57:48.350 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.900+0000 7f21bc169700 1 -- 172.21.15.101:0/3306514257 >> [v2:172.21.15.101:6800/3497914514,v1:172.21.15.101:6801/3497914514] conn(0x7f21a0038710 msgr2=0x7f21a003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:57:48.350 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.900+0000 7f21bc169700 1 --2- 172.21.15.101:0/3306514257 >> [v2:172.21.15.101:6800/3497914514,v1:172.21.15.101:6801/3497914514] conn(0x7f21a0038710 0x7f21a003abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f21a4006fd0 tx=0x7f21a4006e40 comp rx=0 tx=0).stop 2023-07-18T02:57:48.350 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.900+0000 7f21bc169700 1 -- 172.21.15.101:0/3306514257 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f21b407be40 msgr2=0x7f21b419b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:57:48.351 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.900+0000 7f21bc169700 1 --2- 172.21.15.101:0/3306514257 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f21b407be40 0x7f21b419b320 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f21b0009130 tx=0x7f21b0011ad0 comp rx=0 tx=0).stop 2023-07-18T02:57:48.351 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.901+0000 7f21bc169700 1 -- 172.21.15.101:0/3306514257 shutdown_connections 2023-07-18T02:57:48.351 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.901+0000 7f21bc169700 1 --2- 172.21.15.101:0/3306514257 >> [v2:172.21.15.101:6800/3497914514,v1:172.21.15.101:6801/3497914514] conn(0x7f21a0038710 0x7f21a003abc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:57:48.351 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.901+0000 7f21bc169700 1 --2- 172.21.15.101:0/3306514257 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f21b407be40 0x7f21b419b320 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:57:48.351 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.901+0000 7f21bc169700 1 -- 172.21.15.101:0/3306514257 >> 172.21.15.101:0/3306514257 conn(0x7f21b4100970 msgr2=0x7f21b4102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:57:48.351 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.901+0000 7f21bc169700 1 -- 172.21.15.101:0/3306514257 shutdown_connections 2023-07-18T02:57:48.351 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:47.901+0000 7f21bc169700 1 -- 172.21.15.101:0/3306514257 wait complete. 2023-07-18T02:57:48.467 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:48 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:48.171+0000 7fb94f2cb200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-18T02:57:48.468 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:48 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:48.275+0000 7fb94f2cb200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-18T02:57:48.468 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:48 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:48.392+0000 7fb94f2cb200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-18T02:57:48.873 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:48 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:48.516+0000 7fb94f2cb200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-18T02:57:48.873 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:48 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:48.745+0000 7fb94f2cb200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-18T02:57:49.123 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:48 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3306514257' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-07-18T02:57:49.124 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:48 smithi101 ceph-mon[126379]: mgrmap e5: x(active, since 8s) 2023-07-18T02:57:49.419 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:49 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:49.149+0000 7fb94f2cb200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-18T02:57:49.419 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:49 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:49.277+0000 7fb94f2cb200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-18T02:57:49.622 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout { 2023-07-18T02:57:49.622 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-07-18T02:57:49.622 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "available": true, 2023-07-18T02:57:49.623 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-07-18T02:57:49.623 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-07-18T02:57:49.623 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout } 2023-07-18T02:57:49.623 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:48.929+0000 7f6e65162700 1 Processor -- start 2023-07-18T02:57:49.623 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:48.929+0000 7f6e65162700 1 -- start start 2023-07-18T02:57:49.623 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:48.929+0000 7f6e65162700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6e6007be20 0x7f6e6007a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:57:49.623 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:48.929+0000 7f6e65162700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6007a860 con 0x7f6e6007be20 2023-07-18T02:57:49.624 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:48.930+0000 7f6e5ed9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6e6007be20 0x7f6e6007a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:57:49.624 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:48.930+0000 7f6e5ed9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6e6007be20 0x7f6e6007a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:56838/0 (socket says 172.21.15.101:56838) 2023-07-18T02:57:49.624 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:48.930+0000 7f6e5ed9d700 1 -- 172.21.15.101:0/2641569809 learned_addr learned my addr 172.21.15.101:0/2641569809 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:57:49.624 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:48.930+0000 7f6e5ed9d700 1 -- 172.21.15.101:0/2641569809 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e6007a9a0 con 0x7f6e6007be20 2023-07-18T02:57:49.624 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:48.930+0000 7f6e5ed9d700 1 --2- 172.21.15.101:0/2641569809 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6e6007be20 0x7f6e6007a320 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6e500098a0 tx=0x7f6e50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a10db56491cb8ef1 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:57:49.624 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:48.931+0000 7f6e5dd9b700 1 -- 172.21.15.101:0/2641569809 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e50004160 con 0x7f6e6007be20 2023-07-18T02:57:49.624 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:48.931+0000 7f6e5dd9b700 1 -- 172.21.15.101:0/2641569809 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6e500042c0 con 0x7f6e6007be20 2023-07-18T02:57:49.625 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:48.931+0000 7f6e5dd9b700 1 -- 172.21.15.101:0/2641569809 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e50011a00 con 0x7f6e6007be20 2023-07-18T02:57:49.625 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:48.931+0000 7f6e65162700 1 -- 172.21.15.101:0/2641569809 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6e6007be20 msgr2=0x7f6e6007a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:57:49.625 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:48.931+0000 7f6e65162700 1 --2- 172.21.15.101:0/2641569809 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6e6007be20 0x7f6e6007a320 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6e500098a0 tx=0x7f6e50009bb0 comp rx=0 tx=0).stop 2023-07-18T02:57:49.625 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:48.931+0000 7f6e65162700 1 -- 172.21.15.101:0/2641569809 shutdown_connections 2023-07-18T02:57:49.625 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:48.931+0000 7f6e65162700 1 --2- 172.21.15.101:0/2641569809 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6e6007be20 0x7f6e6007a320 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:57:49.626 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:48.931+0000 7f6e65162700 1 -- 172.21.15.101:0/2641569809 >> 172.21.15.101:0/2641569809 conn(0x7f6e60100970 msgr2=0x7f6e60102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:57:49.626 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:48.931+0000 7f6e65162700 1 -- 172.21.15.101:0/2641569809 shutdown_connections 2023-07-18T02:57:49.626 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:48.931+0000 7f6e65162700 1 -- 172.21.15.101:0/2641569809 wait complete. 2023-07-18T02:57:49.626 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:48.933+0000 7f6e65162700 1 Processor -- start 2023-07-18T02:57:49.626 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:48.933+0000 7f6e65162700 1 -- start start 2023-07-18T02:57:49.626 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:48.933+0000 7f6e65162700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6e6007be20 0x7f6e6019b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:57:49.627 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:48.934+0000 7f6e65162700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6019b880 con 0x7f6e6007be20 2023-07-18T02:57:49.627 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:48.934+0000 7f6e5ed9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6e6007be20 0x7f6e6019b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:57:49.627 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:48.934+0000 7f6e5ed9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6e6007be20 0x7f6e6019b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:56842/0 (socket says 172.21.15.101:56842) 2023-07-18T02:57:49.627 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:48.934+0000 7f6e5ed9d700 1 -- 172.21.15.101:0/2927667503 learned_addr learned my addr 172.21.15.101:0/2927667503 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:57:49.627 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:48.934+0000 7f6e5ed9d700 1 -- 172.21.15.101:0/2927667503 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e50009550 con 0x7f6e6007be20 2023-07-18T02:57:49.627 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:48.934+0000 7f6e5ed9d700 1 --2- 172.21.15.101:0/2927667503 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6e6007be20 0x7f6e6019b340 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6e50003f50 tx=0x7f6e50011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:57:49.627 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:48.935+0000 7f6e47fff700 1 -- 172.21.15.101:0/2927667503 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e50003aa0 con 0x7f6e6007be20 2023-07-18T02:57:49.627 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:48.935+0000 7f6e47fff700 1 -- 172.21.15.101:0/2927667503 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6e50003c00 con 0x7f6e6007be20 2023-07-18T02:57:49.630 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:48.935+0000 7f6e47fff700 1 -- 172.21.15.101:0/2927667503 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e500247c0 con 0x7f6e6007be20 2023-07-18T02:57:49.630 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:48.935+0000 7f6e65162700 1 -- 172.21.15.101:0/2927667503 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e6019ba80 con 0x7f6e6007be20 2023-07-18T02:57:49.630 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:48.935+0000 7f6e65162700 1 -- 172.21.15.101:0/2927667503 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e6019bea0 con 0x7f6e6007be20 2023-07-18T02:57:49.630 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:48.936+0000 7f6e65162700 1 -- 172.21.15.101:0/2927667503 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e60195320 con 0x7f6e6007be20 2023-07-18T02:57:49.630 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:48.936+0000 7f6e47fff700 1 -- 172.21.15.101:0/2927667503 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f6e5001ab60 con 0x7f6e6007be20 2023-07-18T02:57:49.630 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:48.937+0000 7f6e47fff700 1 --2- 172.21.15.101:0/2927667503 >> [v2:172.21.15.101:6800/3497914514,v1:172.21.15.101:6801/3497914514] conn(0x7f6e48038760 0x7f6e4803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:57:49.630 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:48.937+0000 7f6e47fff700 1 -- 172.21.15.101:0/2927667503 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6e5004c2e0 con 0x7f6e6007be20 2023-07-18T02:57:49.631 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:48.937+0000 7f6e5e59c700 1 -- 172.21.15.101:0/2927667503 >> [v2:172.21.15.101:6800/3497914514,v1:172.21.15.101:6801/3497914514] conn(0x7f6e48038760 msgr2=0x7f6e4803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.101:6800/3497914514 2023-07-18T02:57:49.631 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:48.937+0000 7f6e5e59c700 1 --2- 172.21.15.101:0/2927667503 >> [v2:172.21.15.101:6800/3497914514,v1:172.21.15.101:6801/3497914514] conn(0x7f6e48038760 0x7f6e4803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-18T02:57:49.631 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:48.945+0000 7f6e47fff700 1 -- 172.21.15.101:0/2927667503 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6e50024920 con 0x7f6e6007be20 2023-07-18T02:57:49.631 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:49.138+0000 7f6e5e59c700 1 -- 172.21.15.101:0/2927667503 >> [v2:172.21.15.101:6800/3497914514,v1:172.21.15.101:6801/3497914514] conn(0x7f6e48038760 msgr2=0x7f6e4803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.101:6800/3497914514 2023-07-18T02:57:49.631 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:49.138+0000 7f6e5e59c700 1 --2- 172.21.15.101:0/2927667503 >> [v2:172.21.15.101:6800/3497914514,v1:172.21.15.101:6801/3497914514] conn(0x7f6e48038760 0x7f6e4803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-18T02:57:49.631 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:49.203+0000 7f6e65162700 1 -- 172.21.15.101:0/2927667503 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f6e6002c850 con 0x7f6e6007be20 2023-07-18T02:57:49.631 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:49.204+0000 7f6e47fff700 1 -- 172.21.15.101:0/2927667503 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f6e5002cb40 con 0x7f6e6007be20 2023-07-18T02:57:49.632 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:49.206+0000 7f6e65162700 1 -- 172.21.15.101:0/2927667503 >> [v2:172.21.15.101:6800/3497914514,v1:172.21.15.101:6801/3497914514] conn(0x7f6e48038760 msgr2=0x7f6e4803ac10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-18T02:57:49.632 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:49.206+0000 7f6e65162700 1 --2- 172.21.15.101:0/2927667503 >> [v2:172.21.15.101:6800/3497914514,v1:172.21.15.101:6801/3497914514] conn(0x7f6e48038760 0x7f6e4803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:57:49.632 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:49.206+0000 7f6e65162700 1 -- 172.21.15.101:0/2927667503 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6e6007be20 msgr2=0x7f6e6019b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:57:49.632 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:49.206+0000 7f6e65162700 1 --2- 172.21.15.101:0/2927667503 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6e6007be20 0x7f6e6019b340 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6e50003f50 tx=0x7f6e50011ad0 comp rx=0 tx=0).stop 2023-07-18T02:57:49.632 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:49.206+0000 7f6e65162700 1 -- 172.21.15.101:0/2927667503 shutdown_connections 2023-07-18T02:57:49.632 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:49.207+0000 7f6e65162700 1 --2- 172.21.15.101:0/2927667503 >> [v2:172.21.15.101:6800/3497914514,v1:172.21.15.101:6801/3497914514] conn(0x7f6e48038760 0x7f6e4803ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:57:49.633 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:49.207+0000 7f6e65162700 1 --2- 172.21.15.101:0/2927667503 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6e6007be20 0x7f6e6019b340 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:57:49.633 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:49.207+0000 7f6e65162700 1 -- 172.21.15.101:0/2927667503 >> 172.21.15.101:0/2927667503 conn(0x7f6e60100970 msgr2=0x7f6e60102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:57:49.633 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:49.207+0000 7f6e65162700 1 -- 172.21.15.101:0/2927667503 shutdown_connections 2023-07-18T02:57:49.633 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:49.207+0000 7f6e65162700 1 -- 172.21.15.101:0/2927667503 wait complete. 2023-07-18T02:57:49.633 INFO:teuthology.orchestra.run.smithi101.stdout:Waiting for the mgr to restart... 2023-07-18T02:57:49.633 INFO:teuthology.orchestra.run.smithi101.stdout:Waiting for mgr epoch 5... 2023-07-18T02:57:49.719 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:49 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:49.445+0000 7fb94f2cb200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-18T02:57:49.719 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:49 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:49.570+0000 7fb94f2cb200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-18T02:57:50.125 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:49 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2927667503' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-07-18T02:57:50.467 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:50 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:50.151+0000 7fb94f2cb200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-18T02:57:51.217 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:50 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:50.879+0000 7fb94f2cb200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-18T02:57:51.218 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:51 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:51.082+0000 7fb94f2cb200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-18T02:57:52.313 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:52 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:52.006+0000 7fb94f2cb200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-18T02:57:52.313 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:52 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:52.123+0000 7fb94f2cb200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-18T02:57:52.934 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:52 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:52.339+0000 7fb94f2cb200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-18T02:57:52.935 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:52 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:52.690+0000 7fb94f2cb200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-18T02:57:53.217 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:52 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:52.961+0000 7fb94f2cb200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-18T02:57:53.218 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:53 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:53.148+0000 7fb94f2cb200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-18T02:57:53.717 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:53 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:53.485+0000 7fb94f2cb200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-18T02:57:54.717 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:54 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:54.388+0000 7fb94f2cb200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-18T02:57:54.717 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:54 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:54.554+0000 7fb94f2cb200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-18T02:57:55.175 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:54 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:54.794+0000 7fb94f2cb200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-18T02:57:55.176 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:54 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:54.937+0000 7fb94f2cb200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-18T02:57:55.176 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:55 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:55.201+0000 7fb94f2cb200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-18T02:57:56.467 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:57:56 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:57:56.240+0000 7fb94f2cb200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-18T02:57:56.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:56 smithi101 ceph-mon[126379]: Active manager daemon x restarted 2023-07-18T02:57:56.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:56 smithi101 ceph-mon[126379]: Activating manager daemon x 2023-07-18T02:57:56.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:56 smithi101 ceph-mon[126379]: osdmap e2: 0 total, 0 up, 0 in 2023-07-18T02:57:56.469 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:56 smithi101 ceph-mon[126379]: mgrmap e6: x(active, starting, since 0.00412685s) 2023-07-18T02:57:56.469 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:56 smithi101 ceph-mon[126379]: from='mgr.14118 172.21.15.101:0/4056684411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-18T02:57:56.469 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:56 smithi101 ceph-mon[126379]: from='mgr.14118 172.21.15.101:0/4056684411' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-18T02:57:56.469 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:56 smithi101 ceph-mon[126379]: from='mgr.14118 172.21.15.101:0/4056684411' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-18T02:57:56.469 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:56 smithi101 ceph-mon[126379]: from='mgr.14118 172.21.15.101:0/4056684411' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-18T02:57:56.469 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:56 smithi101 ceph-mon[126379]: from='mgr.14118 172.21.15.101:0/4056684411' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-18T02:57:56.469 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:56 smithi101 ceph-mon[126379]: Manager daemon x is now available 2023-07-18T02:57:56.470 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:56 smithi101 ceph-mon[126379]: from='mgr.14118 172.21.15.101:0/4056684411' entity='mgr.x' 2023-07-18T02:57:56.470 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:56 smithi101 ceph-mon[126379]: from='mgr.14118 172.21.15.101:0/4056684411' entity='mgr.x' 2023-07-18T02:57:56.470 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:56 smithi101 ceph-mon[126379]: from='mgr.14118 172.21.15.101:0/4056684411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T02:57:56.470 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:56 smithi101 ceph-mon[126379]: from='mgr.14118 172.21.15.101:0/4056684411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T02:57:57.717 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout { 2023-07-18T02:57:57.718 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-07-18T02:57:57.718 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "initialized": true 2023-07-18T02:57:57.718 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout } 2023-07-18T02:57:57.718 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:50.190+0000 7ffba1fc2700 1 Processor -- start 2023-07-18T02:57:57.718 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:50.190+0000 7ffba1fc2700 1 -- start start 2023-07-18T02:57:57.718 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:50.191+0000 7ffba1fc2700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffb9c07be40 0x7ffb9c07a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:57:57.718 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:50.191+0000 7ffba1fc2700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9c07a880 con 0x7ffb9c07be40 2023-07-18T02:57:57.719 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:50.191+0000 7ffb9b7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffb9c07be40 0x7ffb9c07a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:57:57.719 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:50.191+0000 7ffb9b7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffb9c07be40 0x7ffb9c07a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:56850/0 (socket says 172.21.15.101:56850) 2023-07-18T02:57:57.719 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:57 smithi101 ceph-mon[126379]: Found migration_current of "None". Setting to last migration. 2023-07-18T02:57:57.719 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:57 smithi101 ceph-mon[126379]: from='mgr.14118 172.21.15.101:0/4056684411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T02:57:57.719 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:57 smithi101 ceph-mon[126379]: from='mgr.14118 172.21.15.101:0/4056684411' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-18T02:57:57.720 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:57 smithi101 ceph-mon[126379]: from='mgr.14118 172.21.15.101:0/4056684411' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-18T02:57:57.720 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:57 smithi101 ceph-mon[126379]: from='mgr.14118 172.21.15.101:0/4056684411' entity='mgr.x' 2023-07-18T02:57:57.720 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:57 smithi101 ceph-mon[126379]: from='mgr.14118 172.21.15.101:0/4056684411' entity='mgr.x' 2023-07-18T02:57:57.720 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:57 smithi101 ceph-mon[126379]: mgrmap e7: x(active, since 1.00776s) 2023-07-18T02:57:57.724 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:50.191+0000 7ffb9b7fe700 1 -- 172.21.15.101:0/1071120271 learned_addr learned my addr 172.21.15.101:0/1071120271 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:57:57.725 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:50.191+0000 7ffb9b7fe700 1 -- 172.21.15.101:0/1071120271 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb9c07a9c0 con 0x7ffb9c07be40 2023-07-18T02:57:57.725 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:50.192+0000 7ffb9b7fe700 1 --2- 172.21.15.101:0/1071120271 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffb9c07be40 0x7ffb9c07a340 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c0098a0 tx=0x7ffb8c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c50e0631a33c796d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:57:57.725 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:50.192+0000 7ffb9a7fc700 1 -- 172.21.15.101:0/1071120271 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb8c004160 con 0x7ffb9c07be40 2023-07-18T02:57:57.725 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:50.192+0000 7ffb9a7fc700 1 -- 172.21.15.101:0/1071120271 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ffb8c0042c0 con 0x7ffb9c07be40 2023-07-18T02:57:57.725 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:50.192+0000 7ffb9a7fc700 1 -- 172.21.15.101:0/1071120271 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb8c011a00 con 0x7ffb9c07be40 2023-07-18T02:57:57.725 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:50.192+0000 7ffba1fc2700 1 -- 172.21.15.101:0/1071120271 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffb9c07be40 msgr2=0x7ffb9c07a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:57:57.725 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:50.192+0000 7ffba1fc2700 1 --2- 172.21.15.101:0/1071120271 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffb9c07be40 0x7ffb9c07a340 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c0098a0 tx=0x7ffb8c009bb0 comp rx=0 tx=0).stop 2023-07-18T02:57:57.726 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:50.192+0000 7ffba1fc2700 1 -- 172.21.15.101:0/1071120271 shutdown_connections 2023-07-18T02:57:57.726 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:50.193+0000 7ffba1fc2700 1 --2- 172.21.15.101:0/1071120271 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffb9c07be40 0x7ffb9c07a340 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:57:57.726 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:50.193+0000 7ffba1fc2700 1 -- 172.21.15.101:0/1071120271 >> 172.21.15.101:0/1071120271 conn(0x7ffb9c100970 msgr2=0x7ffb9c102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:57:57.726 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:50.193+0000 7ffba1fc2700 1 -- 172.21.15.101:0/1071120271 shutdown_connections 2023-07-18T02:57:57.726 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:50.193+0000 7ffba1fc2700 1 -- 172.21.15.101:0/1071120271 wait complete. 2023-07-18T02:57:57.726 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:50.194+0000 7ffba1fc2700 1 Processor -- start 2023-07-18T02:57:57.726 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:50.194+0000 7ffba1fc2700 1 -- start start 2023-07-18T02:57:57.727 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:50.194+0000 7ffba1fc2700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffb9c07be40 0x7ffb9c19b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:57:57.727 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:50.194+0000 7ffba1fc2700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9c19b860 con 0x7ffb9c07be40 2023-07-18T02:57:57.727 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:50.195+0000 7ffb9b7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffb9c07be40 0x7ffb9c19b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:57:57.727 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:50.195+0000 7ffb9b7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffb9c07be40 0x7ffb9c19b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:56858/0 (socket says 172.21.15.101:56858) 2023-07-18T02:57:57.727 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:50.195+0000 7ffb9b7fe700 1 -- 172.21.15.101:0/4080459014 learned_addr learned my addr 172.21.15.101:0/4080459014 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:57:57.727 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:50.195+0000 7ffb9b7fe700 1 -- 172.21.15.101:0/4080459014 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb8c009550 con 0x7ffb9c07be40 2023-07-18T02:57:57.727 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:50.195+0000 7ffb9b7fe700 1 --2- 172.21.15.101:0/4080459014 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffb9c07be40 0x7ffb9c19b320 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c009130 tx=0x7ffb8c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:57:57.728 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:50.195+0000 7ffb98ff9700 1 -- 172.21.15.101:0/4080459014 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb8c003ab0 con 0x7ffb9c07be40 2023-07-18T02:57:57.728 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:50.195+0000 7ffb98ff9700 1 -- 172.21.15.101:0/4080459014 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ffb8c003c10 con 0x7ffb9c07be40 2023-07-18T02:57:57.728 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:50.195+0000 7ffba1fc2700 1 -- 172.21.15.101:0/4080459014 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb9c19ba60 con 0x7ffb9c07be40 2023-07-18T02:57:57.728 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:50.196+0000 7ffb98ff9700 1 -- 172.21.15.101:0/4080459014 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb8c0247d0 con 0x7ffb9c07be40 2023-07-18T02:57:57.728 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:50.196+0000 7ffba1fc2700 1 -- 172.21.15.101:0/4080459014 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb9c19be80 con 0x7ffb9c07be40 2023-07-18T02:57:57.728 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:50.197+0000 7ffb98ff9700 1 -- 172.21.15.101:0/4080459014 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7ffb8c01a430 con 0x7ffb9c07be40 2023-07-18T02:57:57.728 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:50.197+0000 7ffb98ff9700 1 --2- 172.21.15.101:0/4080459014 >> [v2:172.21.15.101:6800/3497914514,v1:172.21.15.101:6801/3497914514] conn(0x7ffb84038710 0x7ffb8403abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:57:57.728 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:50.197+0000 7ffb98ff9700 1 -- 172.21.15.101:0/4080459014 --> [v2:172.21.15.101:6800/3497914514,v1:172.21.15.101:6801/3497914514] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffb8403b2d0 con 0x7ffb84038710 2023-07-18T02:57:57.729 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:50.197+0000 7ffb98ff9700 1 -- 172.21.15.101:0/4080459014 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ffb8c04c4d0 con 0x7ffb9c07be40 2023-07-18T02:57:57.729 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:50.198+0000 7ffb9affd700 1 -- 172.21.15.101:0/4080459014 >> [v2:172.21.15.101:6800/3497914514,v1:172.21.15.101:6801/3497914514] conn(0x7ffb84038710 msgr2=0x7ffb8403abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.101:6800/3497914514 2023-07-18T02:57:57.729 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:50.198+0000 7ffb9affd700 1 --2- 172.21.15.101:0/4080459014 >> [v2:172.21.15.101:6800/3497914514,v1:172.21.15.101:6801/3497914514] conn(0x7ffb84038710 0x7ffb8403abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-18T02:57:57.729 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:50.399+0000 7ffb9affd700 1 -- 172.21.15.101:0/4080459014 >> [v2:172.21.15.101:6800/3497914514,v1:172.21.15.101:6801/3497914514] conn(0x7ffb84038710 msgr2=0x7ffb8403abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.101:6800/3497914514 2023-07-18T02:57:57.729 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:50.399+0000 7ffb9affd700 1 --2- 172.21.15.101:0/4080459014 >> [v2:172.21.15.101:6800/3497914514,v1:172.21.15.101:6801/3497914514] conn(0x7ffb84038710 0x7ffb8403abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-18T02:57:57.729 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:50.800+0000 7ffb9affd700 1 -- 172.21.15.101:0/4080459014 >> [v2:172.21.15.101:6800/3497914514,v1:172.21.15.101:6801/3497914514] conn(0x7ffb84038710 msgr2=0x7ffb8403abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.101:6800/3497914514 2023-07-18T02:57:57.729 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:50.800+0000 7ffb9affd700 1 --2- 172.21.15.101:0/4080459014 >> [v2:172.21.15.101:6800/3497914514,v1:172.21.15.101:6801/3497914514] conn(0x7ffb84038710 0x7ffb8403abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-07-18T02:57:57.730 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:51.601+0000 7ffb9affd700 1 -- 172.21.15.101:0/4080459014 >> [v2:172.21.15.101:6800/3497914514,v1:172.21.15.101:6801/3497914514] conn(0x7ffb84038710 msgr2=0x7ffb8403abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.101:6800/3497914514 2023-07-18T02:57:57.730 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:51.601+0000 7ffb9affd700 1 --2- 172.21.15.101:0/4080459014 >> [v2:172.21.15.101:6800/3497914514,v1:172.21.15.101:6801/3497914514] conn(0x7ffb84038710 0x7ffb8403abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-07-18T02:57:57.730 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:53.202+0000 7ffb9affd700 1 -- 172.21.15.101:0/4080459014 >> [v2:172.21.15.101:6800/3497914514,v1:172.21.15.101:6801/3497914514] conn(0x7ffb84038710 msgr2=0x7ffb8403abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.101:6800/3497914514 2023-07-18T02:57:57.730 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:53.203+0000 7ffb9affd700 1 --2- 172.21.15.101:0/4080459014 >> [v2:172.21.15.101:6800/3497914514,v1:172.21.15.101:6801/3497914514] conn(0x7ffb84038710 0x7ffb8403abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-07-18T02:57:57.730 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:56.245+0000 7ffb98ff9700 1 -- 172.21.15.101:0/4080459014 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mgrmap(e 6) v1 ==== 45250+0+0 (secure 0 0 0) 0x7ffb8c02d4e0 con 0x7ffb9c07be40 2023-07-18T02:57:57.730 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:56.245+0000 7ffb98ff9700 1 -- 172.21.15.101:0/4080459014 >> [v2:172.21.15.101:6800/3497914514,v1:172.21.15.101:6801/3497914514] conn(0x7ffb84038710 msgr2=0x7ffb8403abc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-18T02:57:57.730 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:56.245+0000 7ffb98ff9700 1 --2- 172.21.15.101:0/4080459014 >> [v2:172.21.15.101:6800/3497914514,v1:172.21.15.101:6801/3497914514] conn(0x7ffb84038710 0x7ffb8403abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:57:57.731 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:57.247+0000 7ffb98ff9700 1 -- 172.21.15.101:0/4080459014 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mgrmap(e 7) v1 ==== 45377+0+0 (secure 0 0 0) 0x7ffb8c021d00 con 0x7ffb9c07be40 2023-07-18T02:57:57.731 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:57.247+0000 7ffb98ff9700 1 --2- 172.21.15.101:0/4080459014 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7ffb8403bfe0 0x7ffb8403e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:57:57.731 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:57.247+0000 7ffb98ff9700 1 -- 172.21.15.101:0/4080459014 --> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffb8403b2d0 con 0x7ffb8403bfe0 2023-07-18T02:57:57.731 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:57.248+0000 7ffb9affd700 1 --2- 172.21.15.101:0/4080459014 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7ffb8403bfe0 0x7ffb8403e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:57:57.731 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:57.249+0000 7ffb9affd700 1 --2- 172.21.15.101:0/4080459014 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7ffb8403bfe0 0x7ffb8403e3c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ffb90003a10 tx=0x7ffb90009290 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:57:57.731 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:57.250+0000 7ffb98ff9700 1 -- 172.21.15.101:0/4080459014 <== mgr.14118 v2:172.21.15.101:6800/2461037266 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7ffb8403b2d0 con 0x7ffb8403bfe0 2023-07-18T02:57:57.731 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:57.261+0000 7ffba1fc2700 1 -- 172.21.15.101:0/4080459014 --> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7ffb9c19c660 con 0x7ffb8403bfe0 2023-07-18T02:57:57.731 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:57.261+0000 7ffb98ff9700 1 -- 172.21.15.101:0/4080459014 <== mgr.14118 v2:172.21.15.101:6800/2461037266 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7ffb9c19c660 con 0x7ffb8403bfe0 2023-07-18T02:57:57.732 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:57.261+0000 7ffba1fc2700 1 -- 172.21.15.101:0/4080459014 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7ffb8403bfe0 msgr2=0x7ffb8403e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:57:57.732 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:57.261+0000 7ffba1fc2700 1 --2- 172.21.15.101:0/4080459014 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7ffb8403bfe0 0x7ffb8403e3c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ffb90003a10 tx=0x7ffb90009290 comp rx=0 tx=0).stop 2023-07-18T02:57:57.732 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:57.261+0000 7ffba1fc2700 1 -- 172.21.15.101:0/4080459014 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffb9c07be40 msgr2=0x7ffb9c19b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:57:57.732 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:57.261+0000 7ffba1fc2700 1 --2- 172.21.15.101:0/4080459014 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffb9c07be40 0x7ffb9c19b320 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c009130 tx=0x7ffb8c011ad0 comp rx=0 tx=0).stop 2023-07-18T02:57:57.732 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:57.261+0000 7ffba1fc2700 1 -- 172.21.15.101:0/4080459014 shutdown_connections 2023-07-18T02:57:57.732 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:57.261+0000 7ffba1fc2700 1 --2- 172.21.15.101:0/4080459014 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7ffb8403bfe0 0x7ffb8403e3c0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:57:57.732 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:57.261+0000 7ffba1fc2700 1 --2- 172.21.15.101:0/4080459014 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffb9c07be40 0x7ffb9c19b320 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:57:57.733 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:57.261+0000 7ffba1fc2700 1 --2- 172.21.15.101:0/4080459014 >> [v2:172.21.15.101:6800/3497914514,v1:172.21.15.101:6801/3497914514] conn(0x7ffb84038710 0x7ffb8403abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:57:57.733 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:57.261+0000 7ffba1fc2700 1 -- 172.21.15.101:0/4080459014 >> 172.21.15.101:0/4080459014 conn(0x7ffb9c100970 msgr2=0x7ffb9c102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:57:57.733 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:57.262+0000 7ffba1fc2700 1 -- 172.21.15.101:0/4080459014 shutdown_connections 2023-07-18T02:57:57.733 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:57.262+0000 7ffba1fc2700 1 -- 172.21.15.101:0/4080459014 wait complete. 2023-07-18T02:57:57.733 INFO:teuthology.orchestra.run.smithi101.stdout:mgr epoch 5 is available 2023-07-18T02:57:57.733 INFO:teuthology.orchestra.run.smithi101.stdout:Setting orchestrator backend to cephadm... 2023-07-18T02:57:58.536 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:58 smithi101 ceph-mon[126379]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-07-18T02:57:58.536 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:58 smithi101 ceph-mon[126379]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-07-18T02:57:58.901 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.306+0000 7fbfe470e700 1 Processor -- start 2023-07-18T02:57:58.902 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.307+0000 7fbfe470e700 1 -- start start 2023-07-18T02:57:58.902 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.307+0000 7fbfe470e700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbfdc07be40 0x7fbfdc07a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:57:58.902 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.307+0000 7fbfe470e700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc07a880 con 0x7fbfdc07be40 2023-07-18T02:57:58.902 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.307+0000 7fbfe24aa700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbfdc07be40 0x7fbfdc07a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:57:58.902 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.307+0000 7fbfe24aa700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbfdc07be40 0x7fbfdc07a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:33310/0 (socket says 172.21.15.101:33310) 2023-07-18T02:57:58.902 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.307+0000 7fbfe24aa700 1 -- 172.21.15.101:0/1822957905 learned_addr learned my addr 172.21.15.101:0/1822957905 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:57:58.903 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.308+0000 7fbfe24aa700 1 -- 172.21.15.101:0/1822957905 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfdc07a9c0 con 0x7fbfdc07be40 2023-07-18T02:57:58.903 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.308+0000 7fbfe24aa700 1 --2- 172.21.15.101:0/1822957905 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbfdc07be40 0x7fbfdc07a340 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbfd80098a0 tx=0x7fbfd8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f54a804c8734095b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:57:58.903 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.308+0000 7fbfe14a8700 1 -- 172.21.15.101:0/1822957905 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbfd8004160 con 0x7fbfdc07be40 2023-07-18T02:57:58.903 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.308+0000 7fbfe14a8700 1 -- 172.21.15.101:0/1822957905 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbfd80042c0 con 0x7fbfdc07be40 2023-07-18T02:57:58.903 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.308+0000 7fbfe14a8700 1 -- 172.21.15.101:0/1822957905 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbfd8011a00 con 0x7fbfdc07be40 2023-07-18T02:57:58.903 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.309+0000 7fbfe470e700 1 -- 172.21.15.101:0/1822957905 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbfdc07be40 msgr2=0x7fbfdc07a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:57:58.903 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.309+0000 7fbfe470e700 1 --2- 172.21.15.101:0/1822957905 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbfdc07be40 0x7fbfdc07a340 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbfd80098a0 tx=0x7fbfd8009bb0 comp rx=0 tx=0).stop 2023-07-18T02:57:58.903 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.309+0000 7fbfe470e700 1 -- 172.21.15.101:0/1822957905 shutdown_connections 2023-07-18T02:57:58.904 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.309+0000 7fbfe470e700 1 --2- 172.21.15.101:0/1822957905 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbfdc07be40 0x7fbfdc07a340 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:57:58.904 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.309+0000 7fbfe470e700 1 -- 172.21.15.101:0/1822957905 >> 172.21.15.101:0/1822957905 conn(0x7fbfdc100970 msgr2=0x7fbfdc102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:57:58.904 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.309+0000 7fbfe470e700 1 -- 172.21.15.101:0/1822957905 shutdown_connections 2023-07-18T02:57:58.904 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.309+0000 7fbfe470e700 1 -- 172.21.15.101:0/1822957905 wait complete. 2023-07-18T02:57:58.904 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.310+0000 7fbfe470e700 1 Processor -- start 2023-07-18T02:57:58.904 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.310+0000 7fbfe470e700 1 -- start start 2023-07-18T02:57:58.904 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.310+0000 7fbfe470e700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbfdc07be40 0x7fbfdc19b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:57:58.905 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.310+0000 7fbfe470e700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc19b860 con 0x7fbfdc07be40 2023-07-18T02:57:58.905 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.311+0000 7fbfe24aa700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbfdc07be40 0x7fbfdc19b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:57:58.905 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.311+0000 7fbfe24aa700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbfdc07be40 0x7fbfdc19b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:33320/0 (socket says 172.21.15.101:33320) 2023-07-18T02:57:58.905 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.311+0000 7fbfe24aa700 1 -- 172.21.15.101:0/1812152427 learned_addr learned my addr 172.21.15.101:0/1812152427 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:57:58.905 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.311+0000 7fbfe24aa700 1 -- 172.21.15.101:0/1812152427 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd8009550 con 0x7fbfdc07be40 2023-07-18T02:57:58.905 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.311+0000 7fbfe24aa700 1 --2- 172.21.15.101:0/1812152427 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbfdc07be40 0x7fbfdc19b320 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8009130 tx=0x7fbfd8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:57:58.905 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.311+0000 7fbfd37fe700 1 -- 172.21.15.101:0/1812152427 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbfd8003ab0 con 0x7fbfdc07be40 2023-07-18T02:57:58.906 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.311+0000 7fbfd37fe700 1 -- 172.21.15.101:0/1812152427 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbfd8003c10 con 0x7fbfdc07be40 2023-07-18T02:57:58.906 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.311+0000 7fbfe470e700 1 -- 172.21.15.101:0/1812152427 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfdc19ba60 con 0x7fbfdc07be40 2023-07-18T02:57:58.906 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.311+0000 7fbfd37fe700 1 -- 172.21.15.101:0/1812152427 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbfd80247d0 con 0x7fbfdc07be40 2023-07-18T02:57:58.906 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.312+0000 7fbfe470e700 1 -- 172.21.15.101:0/1812152427 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfdc19be80 con 0x7fbfdc07be40 2023-07-18T02:57:58.906 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.313+0000 7fbfe470e700 1 -- 172.21.15.101:0/1812152427 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfdc195300 con 0x7fbfdc07be40 2023-07-18T02:57:58.906 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.313+0000 7fbfd37fe700 1 -- 172.21.15.101:0/1812152427 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 7) v1 ==== 45377+0+0 (secure 0 0 0) 0x7fbfd801abb0 con 0x7fbfdc07be40 2023-07-18T02:57:58.906 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.314+0000 7fbfd37fe700 1 --2- 172.21.15.101:0/1812152427 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7fbfc80385e0 0x7fbfc803aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:57:58.907 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.314+0000 7fbfd37fe700 1 -- 172.21.15.101:0/1812152427 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fbfd8055760 con 0x7fbfdc07be40 2023-07-18T02:57:58.907 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.314+0000 7fbfe1ca9700 1 --2- 172.21.15.101:0/1812152427 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7fbfc80385e0 0x7fbfc803aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:57:58.907 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.314+0000 7fbfe1ca9700 1 --2- 172.21.15.101:0/1812152427 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7fbfc80385e0 0x7fbfc803aa90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc006fd0 tx=0x7fbfcc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:57:58.907 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.319+0000 7fbfd37fe700 1 -- 172.21.15.101:0/1812152427 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fbfd8052060 con 0x7fbfdc07be40 2023-07-18T02:57:58.907 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.532+0000 7fbfe470e700 1 -- 172.21.15.101:0/1812152427 --> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fbfdc060950 con 0x7fbfc80385e0 2023-07-18T02:57:58.907 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.539+0000 7fbfd37fe700 1 -- 172.21.15.101:0/1812152427 <== mgr.14118 v2:172.21.15.101:6800/2461037266 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fbfdc060950 con 0x7fbfc80385e0 2023-07-18T02:57:58.907 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.541+0000 7fbfe470e700 1 -- 172.21.15.101:0/1812152427 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7fbfc80385e0 msgr2=0x7fbfc803aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:57:58.907 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.541+0000 7fbfe470e700 1 --2- 172.21.15.101:0/1812152427 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7fbfc80385e0 0x7fbfc803aa90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc006fd0 tx=0x7fbfcc006e40 comp rx=0 tx=0).stop 2023-07-18T02:57:58.908 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.541+0000 7fbfe470e700 1 -- 172.21.15.101:0/1812152427 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbfdc07be40 msgr2=0x7fbfdc19b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:57:58.908 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.541+0000 7fbfe470e700 1 --2- 172.21.15.101:0/1812152427 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbfdc07be40 0x7fbfdc19b320 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8009130 tx=0x7fbfd8011ad0 comp rx=0 tx=0).stop 2023-07-18T02:57:58.908 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.541+0000 7fbfe470e700 1 -- 172.21.15.101:0/1812152427 shutdown_connections 2023-07-18T02:57:58.908 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.541+0000 7fbfe470e700 1 --2- 172.21.15.101:0/1812152427 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7fbfc80385e0 0x7fbfc803aa90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:57:58.908 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.541+0000 7fbfe470e700 1 --2- 172.21.15.101:0/1812152427 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbfdc07be40 0x7fbfdc19b320 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:57:58.908 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.541+0000 7fbfe470e700 1 -- 172.21.15.101:0/1812152427 >> 172.21.15.101:0/1812152427 conn(0x7fbfdc100970 msgr2=0x7fbfdc102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:57:58.908 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.542+0000 7fbfe470e700 1 -- 172.21.15.101:0/1812152427 shutdown_connections 2023-07-18T02:57:58.909 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:58.542+0000 7fbfe470e700 1 -- 172.21.15.101:0/1812152427 wait complete. 2023-07-18T02:57:59.800 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:59 smithi101 ceph-mon[126379]: [18/Jul/2023:02:57:58] ENGINE Bus STARTING 2023-07-18T02:57:59.800 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:59 smithi101 ceph-mon[126379]: [18/Jul/2023:02:57:58] ENGINE Serving on https://172.21.15.101:7150 2023-07-18T02:57:59.800 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:59 smithi101 ceph-mon[126379]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T02:57:59.801 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:59 smithi101 ceph-mon[126379]: from='mgr.14118 172.21.15.101:0/4056684411' entity='mgr.x' 2023-07-18T02:57:59.801 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:59 smithi101 ceph-mon[126379]: from='mgr.14118 172.21.15.101:0/4056684411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T02:57:59.801 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:59 smithi101 ceph-mon[126379]: [18/Jul/2023:02:57:58] ENGINE Serving on http://172.21.15.101:8765 2023-07-18T02:57:59.801 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:59 smithi101 ceph-mon[126379]: [18/Jul/2023:02:57:58] ENGINE Bus STARTED 2023-07-18T02:57:59.801 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:59 smithi101 ceph-mon[126379]: from='mgr.14118 172.21.15.101:0/4056684411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T02:57:59.801 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:57:59 smithi101 ceph-mon[126379]: mgrmap e8: x(active, since 2s) 2023-07-18T02:58:00.080 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout value unchanged 2023-07-18T02:58:00.080 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.444+0000 7f7426cbc700 1 Processor -- start 2023-07-18T02:58:00.080 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.444+0000 7f7426cbc700 1 -- start start 2023-07-18T02:58:00.080 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.444+0000 7f7426cbc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f742007be40 0x7f742007a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:00.081 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.444+0000 7f7426cbc700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742007a880 con 0x7f742007be40 2023-07-18T02:58:00.081 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.445+0000 7f7424a58700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f742007be40 0x7f742007a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:00.081 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.445+0000 7f7424a58700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f742007be40 0x7f742007a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:33332/0 (socket says 172.21.15.101:33332) 2023-07-18T02:58:00.081 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.445+0000 7f7424a58700 1 -- 172.21.15.101:0/2575149964 learned_addr learned my addr 172.21.15.101:0/2575149964 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:58:00.081 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.445+0000 7f7424a58700 1 -- 172.21.15.101:0/2575149964 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f742007a9c0 con 0x7f742007be40 2023-07-18T02:58:00.081 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.445+0000 7f7424a58700 1 --2- 172.21.15.101:0/2575149964 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f742007be40 0x7f742007a340 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f74140098a0 tx=0x7f7414009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af80344e55d60b67 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:00.081 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.445+0000 7f741f7fe700 1 -- 172.21.15.101:0/2575149964 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7414004160 con 0x7f742007be40 2023-07-18T02:58:00.082 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.446+0000 7f741f7fe700 1 -- 172.21.15.101:0/2575149964 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f74140042c0 con 0x7f742007be40 2023-07-18T02:58:00.082 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.446+0000 7f741f7fe700 1 -- 172.21.15.101:0/2575149964 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7414011a00 con 0x7f742007be40 2023-07-18T02:58:00.082 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.446+0000 7f7426cbc700 1 -- 172.21.15.101:0/2575149964 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f742007be40 msgr2=0x7f742007a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:00.082 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.446+0000 7f7426cbc700 1 --2- 172.21.15.101:0/2575149964 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f742007be40 0x7f742007a340 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f74140098a0 tx=0x7f7414009bb0 comp rx=0 tx=0).stop 2023-07-18T02:58:00.082 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.446+0000 7f7426cbc700 1 -- 172.21.15.101:0/2575149964 shutdown_connections 2023-07-18T02:58:00.082 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.446+0000 7f7426cbc700 1 --2- 172.21.15.101:0/2575149964 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f742007be40 0x7f742007a340 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:00.082 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.446+0000 7f7426cbc700 1 -- 172.21.15.101:0/2575149964 >> 172.21.15.101:0/2575149964 conn(0x7f7420100970 msgr2=0x7f7420102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:58:00.083 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.446+0000 7f7426cbc700 1 -- 172.21.15.101:0/2575149964 shutdown_connections 2023-07-18T02:58:00.083 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.446+0000 7f7426cbc700 1 -- 172.21.15.101:0/2575149964 wait complete. 2023-07-18T02:58:00.083 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.447+0000 7f7426cbc700 1 Processor -- start 2023-07-18T02:58:00.083 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.447+0000 7f7426cbc700 1 -- start start 2023-07-18T02:58:00.083 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.448+0000 7f7426cbc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f742007be40 0x7f742019b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:00.083 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.448+0000 7f7426cbc700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742019b860 con 0x7f742007be40 2023-07-18T02:58:00.083 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.448+0000 7f7424a58700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f742007be40 0x7f742019b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:00.084 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.448+0000 7f7424a58700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f742007be40 0x7f742019b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:33344/0 (socket says 172.21.15.101:33344) 2023-07-18T02:58:00.084 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.448+0000 7f7424a58700 1 -- 172.21.15.101:0/4256167098 learned_addr learned my addr 172.21.15.101:0/4256167098 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:58:00.084 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.448+0000 7f7424a58700 1 -- 172.21.15.101:0/4256167098 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7414009550 con 0x7f742007be40 2023-07-18T02:58:00.084 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.448+0000 7f7424a58700 1 --2- 172.21.15.101:0/4256167098 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f742007be40 0x7f742019b320 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7414009130 tx=0x7f7414011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:00.084 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.449+0000 7f741dffb700 1 -- 172.21.15.101:0/4256167098 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7414003ab0 con 0x7f742007be40 2023-07-18T02:58:00.084 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.449+0000 7f741dffb700 1 -- 172.21.15.101:0/4256167098 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7414003c10 con 0x7f742007be40 2023-07-18T02:58:00.084 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.449+0000 7f7426cbc700 1 -- 172.21.15.101:0/4256167098 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f742019ba60 con 0x7f742007be40 2023-07-18T02:58:00.084 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.449+0000 7f741dffb700 1 -- 172.21.15.101:0/4256167098 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74140247d0 con 0x7f742007be40 2023-07-18T02:58:00.085 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.449+0000 7f7426cbc700 1 -- 172.21.15.101:0/4256167098 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f742019be80 con 0x7f742007be40 2023-07-18T02:58:00.085 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.450+0000 7f7426cbc700 1 -- 172.21.15.101:0/4256167098 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7420195300 con 0x7f742007be40 2023-07-18T02:58:00.085 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.450+0000 7f741dffb700 1 -- 172.21.15.101:0/4256167098 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f741401abb0 con 0x7f742007be40 2023-07-18T02:58:00.085 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.450+0000 7f741dffb700 1 --2- 172.21.15.101:0/4256167098 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f7410038760 0x7f741003ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:00.085 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.451+0000 7f741dffb700 1 -- 172.21.15.101:0/4256167098 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f741404c850 con 0x7f742007be40 2023-07-18T02:58:00.085 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.451+0000 7f741ffff700 1 --2- 172.21.15.101:0/4256167098 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f7410038760 0x7f741003ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:00.085 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.451+0000 7f741ffff700 1 --2- 172.21.15.101:0/4256167098 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f7410038760 0x7f741003ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f740c006fd0 tx=0x7f740c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:00.086 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.455+0000 7f741dffb700 1 -- 172.21.15.101:0/4256167098 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7414018530 con 0x7f742007be40 2023-07-18T02:58:00.086 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.651+0000 7f7426cbc700 1 -- 172.21.15.101:0/4256167098 --> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f7420060950 con 0x7f7410038760 2023-07-18T02:58:00.086 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.652+0000 7f741dffb700 1 -- 172.21.15.101:0/4256167098 <== mgr.14118 v2:172.21.15.101:6800/2461037266 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f7420060950 con 0x7f7410038760 2023-07-18T02:58:00.086 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.655+0000 7f7426cbc700 1 -- 172.21.15.101:0/4256167098 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f7410038760 msgr2=0x7f741003ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:00.086 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.655+0000 7f7426cbc700 1 --2- 172.21.15.101:0/4256167098 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f7410038760 0x7f741003ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f740c006fd0 tx=0x7f740c006e40 comp rx=0 tx=0).stop 2023-07-18T02:58:00.086 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.655+0000 7f7426cbc700 1 -- 172.21.15.101:0/4256167098 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f742007be40 msgr2=0x7f742019b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:00.086 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.655+0000 7f7426cbc700 1 --2- 172.21.15.101:0/4256167098 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f742007be40 0x7f742019b320 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7414009130 tx=0x7f7414011ad0 comp rx=0 tx=0).stop 2023-07-18T02:58:00.087 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.655+0000 7f7426cbc700 1 -- 172.21.15.101:0/4256167098 shutdown_connections 2023-07-18T02:58:00.087 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.655+0000 7f7426cbc700 1 --2- 172.21.15.101:0/4256167098 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f7410038760 0x7f741003ac10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:00.087 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.655+0000 7f7426cbc700 1 --2- 172.21.15.101:0/4256167098 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f742007be40 0x7f742019b320 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:00.087 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.655+0000 7f7426cbc700 1 -- 172.21.15.101:0/4256167098 >> 172.21.15.101:0/4256167098 conn(0x7f7420100970 msgr2=0x7f7420102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:58:00.087 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.655+0000 7f7426cbc700 1 -- 172.21.15.101:0/4256167098 shutdown_connections 2023-07-18T02:58:00.087 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:57:59.655+0000 7f7426cbc700 1 -- 172.21.15.101:0/4256167098 wait complete. 2023-07-18T02:58:00.087 INFO:teuthology.orchestra.run.smithi101.stdout:Generating ssh key... 2023-07-18T02:58:00.717 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:00 smithi101 ceph-mon[126379]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T02:58:01.467 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:58:01 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: Generating public/private rsa key pair. 2023-07-18T02:58:01.468 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:58:01 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: Your identification has been saved in /tmp/tmpgskelerk/key. 2023-07-18T02:58:01.468 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:58:01 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: Your public key has been saved in /tmp/tmpgskelerk/key.pub. 2023-07-18T02:58:01.468 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:58:01 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: The key fingerprint is: 2023-07-18T02:58:01.468 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:58:01 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: SHA256:ZIbGHQiK5xYHcqE5wLumrDhGPBALCB8S65S5OxosnCI ceph-b1782daa-2516-11ee-9b34-001a4aab830c 2023-07-18T02:58:01.469 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:58:01 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: The key's randomart image is: 2023-07-18T02:58:01.469 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:58:01 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: +---[RSA 3072]----+ 2023-07-18T02:58:01.469 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:58:01 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: |X.=... .. | 2023-07-18T02:58:01.469 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:58:01 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: |=@o+ ..o . | 2023-07-18T02:58:01.469 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:58:01 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: |B** . + = | 2023-07-18T02:58:01.469 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:58:01 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: |==.o . + | 2023-07-18T02:58:01.470 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:58:01 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: |oo+ S | 2023-07-18T02:58:01.470 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:58:01 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: |oBo | 2023-07-18T02:58:01.470 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:58:01 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: |E*. | 2023-07-18T02:58:01.470 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:58:01 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: |O+. | 2023-07-18T02:58:01.470 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:58:01 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: |*. | 2023-07-18T02:58:01.471 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:58:01 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: +----[SHA256]-----+ 2023-07-18T02:58:01.562 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:00.677+0000 7fcdb2832700 1 Processor -- start 2023-07-18T02:58:01.562 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:00.677+0000 7fcdb2832700 1 -- start start 2023-07-18T02:58:01.562 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:00.678+0000 7fcdb2832700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcdac07be00 0x7fcdac07a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:01.562 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:00.678+0000 7fcdb2832700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac07a840 con 0x7fcdac07be00 2023-07-18T02:58:01.562 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:00.678+0000 7fcdabfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcdac07be00 0x7fcdac07a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:01.563 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:00.678+0000 7fcdabfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcdac07be00 0x7fcdac07a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:33356/0 (socket says 172.21.15.101:33356) 2023-07-18T02:58:01.563 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:00.678+0000 7fcdabfff700 1 -- 172.21.15.101:0/30494870 learned_addr learned my addr 172.21.15.101:0/30494870 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:58:01.563 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:00.678+0000 7fcdabfff700 1 -- 172.21.15.101:0/30494870 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdac07a980 con 0x7fcdac07be00 2023-07-18T02:58:01.564 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:00.679+0000 7fcdabfff700 1 --2- 172.21.15.101:0/30494870 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcdac07be00 0x7fcdac07a300 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c0098a0 tx=0x7fcd9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55a7992905d39f4f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:01.564 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:00.679+0000 7fcdaaffd700 1 -- 172.21.15.101:0/30494870 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcd9c004160 con 0x7fcdac07be00 2023-07-18T02:58:01.564 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:00.679+0000 7fcdaaffd700 1 -- 172.21.15.101:0/30494870 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcd9c0042c0 con 0x7fcdac07be00 2023-07-18T02:58:01.564 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:00.679+0000 7fcdaaffd700 1 -- 172.21.15.101:0/30494870 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcd9c011a00 con 0x7fcdac07be00 2023-07-18T02:58:01.564 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:00.679+0000 7fcdb2832700 1 -- 172.21.15.101:0/30494870 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcdac07be00 msgr2=0x7fcdac07a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:01.565 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:00.679+0000 7fcdb2832700 1 --2- 172.21.15.101:0/30494870 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcdac07be00 0x7fcdac07a300 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c0098a0 tx=0x7fcd9c009bb0 comp rx=0 tx=0).stop 2023-07-18T02:58:01.565 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:00.679+0000 7fcdb2832700 1 -- 172.21.15.101:0/30494870 shutdown_connections 2023-07-18T02:58:01.565 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:00.679+0000 7fcdb2832700 1 --2- 172.21.15.101:0/30494870 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcdac07be00 0x7fcdac07a300 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:01.565 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:00.679+0000 7fcdb2832700 1 -- 172.21.15.101:0/30494870 >> 172.21.15.101:0/30494870 conn(0x7fcdac100970 msgr2=0x7fcdac102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:58:01.565 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:00.680+0000 7fcdb2832700 1 -- 172.21.15.101:0/30494870 shutdown_connections 2023-07-18T02:58:01.565 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:00.680+0000 7fcdb2832700 1 -- 172.21.15.101:0/30494870 wait complete. 2023-07-18T02:58:01.565 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:00.681+0000 7fcdb2832700 1 Processor -- start 2023-07-18T02:58:01.566 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:00.681+0000 7fcdb2832700 1 -- start start 2023-07-18T02:58:01.566 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:00.681+0000 7fcdb2832700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcdac07be00 0x7fcdac19b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:01.566 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:00.681+0000 7fcdb2832700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac19b880 con 0x7fcdac07be00 2023-07-18T02:58:01.566 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:00.681+0000 7fcdabfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcdac07be00 0x7fcdac19b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:01.566 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:00.681+0000 7fcdabfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcdac07be00 0x7fcdac19b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:33368/0 (socket says 172.21.15.101:33368) 2023-07-18T02:58:01.566 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:00.681+0000 7fcdabfff700 1 -- 172.21.15.101:0/1155375666 learned_addr learned my addr 172.21.15.101:0/1155375666 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:58:01.566 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:00.681+0000 7fcdabfff700 1 -- 172.21.15.101:0/1155375666 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd9c009550 con 0x7fcdac07be00 2023-07-18T02:58:01.567 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:00.682+0000 7fcdabfff700 1 --2- 172.21.15.101:0/1155375666 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcdac07be00 0x7fcdac19b340 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c009fb0 tx=0x7fcd9c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:01.567 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:00.682+0000 7fcda97fa700 1 -- 172.21.15.101:0/1155375666 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcd9c003a40 con 0x7fcdac07be00 2023-07-18T02:58:01.567 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:00.682+0000 7fcda97fa700 1 -- 172.21.15.101:0/1155375666 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcd9c003ba0 con 0x7fcdac07be00 2023-07-18T02:58:01.567 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:00.682+0000 7fcdb2832700 1 -- 172.21.15.101:0/1155375666 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdac19ba80 con 0x7fcdac07be00 2023-07-18T02:58:01.567 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:00.682+0000 7fcda97fa700 1 -- 172.21.15.101:0/1155375666 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcd9c01a750 con 0x7fcdac07be00 2023-07-18T02:58:01.567 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:00.682+0000 7fcdb2832700 1 -- 172.21.15.101:0/1155375666 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdac19bea0 con 0x7fcdac07be00 2023-07-18T02:58:01.567 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:00.683+0000 7fcda97fa700 1 -- 172.21.15.101:0/1155375666 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fcd9c024430 con 0x7fcdac07be00 2023-07-18T02:58:01.568 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:00.683+0000 7fcdb2832700 1 -- 172.21.15.101:0/1155375666 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdac1952f0 con 0x7fcdac07be00 2023-07-18T02:58:01.568 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:00.684+0000 7fcda97fa700 1 --2- 172.21.15.101:0/1155375666 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7fcd94038710 0x7fcd9403abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:01.568 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:00.684+0000 7fcda97fa700 1 -- 172.21.15.101:0/1155375666 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fcd9c04c790 con 0x7fcdac07be00 2023-07-18T02:58:01.568 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:00.684+0000 7fcdab7fe700 1 --2- 172.21.15.101:0/1155375666 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7fcd94038710 0x7fcd9403abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:01.568 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:00.684+0000 7fcdab7fe700 1 --2- 172.21.15.101:0/1155375666 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7fcd94038710 0x7fcd9403abc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcda0006fd0 tx=0x7fcda0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:01.568 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:00.688+0000 7fcda97fa700 1 -- 172.21.15.101:0/1155375666 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fcd9c0246e0 con 0x7fcdac07be00 2023-07-18T02:58:01.569 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:00.882+0000 7fcdb2832700 1 -- 172.21.15.101:0/1155375666 --> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fcdac060950 con 0x7fcd94038710 2023-07-18T02:58:01.569 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:01.135+0000 7fcda97fa700 1 -- 172.21.15.101:0/1155375666 <== mgr.14118 v2:172.21.15.101:6800/2461037266 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fcdac060950 con 0x7fcd94038710 2023-07-18T02:58:01.569 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:01.138+0000 7fcdb2832700 1 -- 172.21.15.101:0/1155375666 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7fcd94038710 msgr2=0x7fcd9403abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:01.569 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:01.138+0000 7fcdb2832700 1 --2- 172.21.15.101:0/1155375666 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7fcd94038710 0x7fcd9403abc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcda0006fd0 tx=0x7fcda0006e40 comp rx=0 tx=0).stop 2023-07-18T02:58:01.569 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:01.138+0000 7fcdb2832700 1 -- 172.21.15.101:0/1155375666 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcdac07be00 msgr2=0x7fcdac19b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:01.569 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:01.138+0000 7fcdb2832700 1 --2- 172.21.15.101:0/1155375666 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcdac07be00 0x7fcdac19b340 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c009fb0 tx=0x7fcd9c011ad0 comp rx=0 tx=0).stop 2023-07-18T02:58:01.569 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:01.138+0000 7fcdb2832700 1 -- 172.21.15.101:0/1155375666 shutdown_connections 2023-07-18T02:58:01.570 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:01.138+0000 7fcdb2832700 1 --2- 172.21.15.101:0/1155375666 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7fcd94038710 0x7fcd9403abc0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:01.570 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:01.138+0000 7fcdb2832700 1 --2- 172.21.15.101:0/1155375666 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcdac07be00 0x7fcdac19b340 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:01.570 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:01.138+0000 7fcdb2832700 1 -- 172.21.15.101:0/1155375666 >> 172.21.15.101:0/1155375666 conn(0x7fcdac100970 msgr2=0x7fcdac1020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:58:01.570 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:01.138+0000 7fcdb2832700 1 -- 172.21.15.101:0/1155375666 shutdown_connections 2023-07-18T02:58:01.570 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:01.138+0000 7fcdb2832700 1 -- 172.21.15.101:0/1155375666 wait complete. 2023-07-18T02:58:01.809 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:01 smithi101 ceph-mon[126379]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T02:58:01.815 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:01 smithi101 ceph-mon[126379]: Generating ssh key... 2023-07-18T02:58:01.815 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:01 smithi101 ceph-mon[126379]: from='mgr.14118 172.21.15.101:0/4056684411' entity='mgr.x' 2023-07-18T02:58:01.816 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:01 smithi101 ceph-mon[126379]: from='mgr.14118 172.21.15.101:0/4056684411' entity='mgr.x' 2023-07-18T02:58:02.736 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCgq7jqdjrsbHBK+iLEAXJN4+1FDgm+QP/Srl6wgTyPcBpTSrXf5QYB3mEx7VFHISYTkQRM51hs5ek5q96AAfKqE/0yWrDHOGrjTdYMLQbrl6lG9hJQG+TgJvPGPltNL7lLXFmNhQfkpp3sJXOfWPDja0g9KzPnwRaySBKSOpTsjrc/MpO3UnzG/QQKfN5KtuGpqPvpYib+nZmO/TkGsBAiPm/nEUa6yJOCJPad16oZe6AGCCt/GlyB1ej+KOh3VfS7RpeDAP82hmGuQY1xBoOG1oQYFPwlMP3pzuyJVTMn0OTE6P3MTIYwddrQj4jZoM8K6S9oCIXvrlGGLWht6dsT5Gsn4d8sO7Lj/60O/Kx9fRxrg1gs1IuodLxhsc8LAQ2VagxlO8yt6V1953AuEA8MubIF49i8ldgfPRng/Mq8rzm7IZ9TfLa8tW/H84N24vFaj3YxiJ3OZ6YYk5ssXYLz8MNaCVsUrRMNq/GyMr+m3Qg+Toucp4LHvLcqbIcFZ9s= ceph-b1782daa-2516-11ee-9b34-001a4aab830c 2023-07-18T02:58:02.736 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.078+0000 7f1993eee700 1 Processor -- start 2023-07-18T02:58:02.736 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.078+0000 7f1993eee700 1 -- start start 2023-07-18T02:58:02.737 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.078+0000 7f1993eee700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f198c07be00 0x7f198c07a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:02.737 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.078+0000 7f1993eee700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198c07a840 con 0x7f198c07be00 2023-07-18T02:58:02.737 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.079+0000 7f1991c8a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f198c07be00 0x7f198c07a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:02.737 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.079+0000 7f1991c8a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f198c07be00 0x7f198c07a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:33382/0 (socket says 172.21.15.101:33382) 2023-07-18T02:58:02.737 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.079+0000 7f1991c8a700 1 -- 172.21.15.101:0/3880017691 learned_addr learned my addr 172.21.15.101:0/3880017691 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:58:02.737 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.079+0000 7f1991c8a700 1 -- 172.21.15.101:0/3880017691 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f198c07a980 con 0x7f198c07be00 2023-07-18T02:58:02.737 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.079+0000 7f1991c8a700 1 --2- 172.21.15.101:0/3880017691 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f198c07be00 0x7f198c07a300 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f19880098a0 tx=0x7f1988009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4017fca4cd75c76 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:02.738 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.080+0000 7f1990c88700 1 -- 172.21.15.101:0/3880017691 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1988004160 con 0x7f198c07be00 2023-07-18T02:58:02.738 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.080+0000 7f1990c88700 1 -- 172.21.15.101:0/3880017691 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f19880042c0 con 0x7f198c07be00 2023-07-18T02:58:02.738 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.080+0000 7f1990c88700 1 -- 172.21.15.101:0/3880017691 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1988011a00 con 0x7f198c07be00 2023-07-18T02:58:02.738 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.080+0000 7f1993eee700 1 -- 172.21.15.101:0/3880017691 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f198c07be00 msgr2=0x7f198c07a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:02.738 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.080+0000 7f1993eee700 1 --2- 172.21.15.101:0/3880017691 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f198c07be00 0x7f198c07a300 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f19880098a0 tx=0x7f1988009bb0 comp rx=0 tx=0).stop 2023-07-18T02:58:02.738 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.080+0000 7f1993eee700 1 -- 172.21.15.101:0/3880017691 shutdown_connections 2023-07-18T02:58:02.738 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.080+0000 7f1993eee700 1 --2- 172.21.15.101:0/3880017691 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f198c07be00 0x7f198c07a300 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:02.739 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.080+0000 7f1993eee700 1 -- 172.21.15.101:0/3880017691 >> 172.21.15.101:0/3880017691 conn(0x7f198c100970 msgr2=0x7f198c102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:58:02.739 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.080+0000 7f1993eee700 1 -- 172.21.15.101:0/3880017691 shutdown_connections 2023-07-18T02:58:02.739 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.080+0000 7f1993eee700 1 -- 172.21.15.101:0/3880017691 wait complete. 2023-07-18T02:58:02.739 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.081+0000 7f1993eee700 1 Processor -- start 2023-07-18T02:58:02.739 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.082+0000 7f1993eee700 1 -- start start 2023-07-18T02:58:02.739 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.082+0000 7f1993eee700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f198c07be00 0x7f198c19b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:02.739 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.082+0000 7f1993eee700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198c19b880 con 0x7f198c07be00 2023-07-18T02:58:02.739 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.082+0000 7f1991c8a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f198c07be00 0x7f198c19b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:02.740 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.082+0000 7f1991c8a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f198c07be00 0x7f198c19b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:33396/0 (socket says 172.21.15.101:33396) 2023-07-18T02:58:02.740 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.082+0000 7f1991c8a700 1 -- 172.21.15.101:0/3186809954 learned_addr learned my addr 172.21.15.101:0/3186809954 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:58:02.740 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.082+0000 7f1991c8a700 1 -- 172.21.15.101:0/3186809954 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1988009550 con 0x7f198c07be00 2023-07-18T02:58:02.740 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.083+0000 7f1991c8a700 1 --2- 172.21.15.101:0/3186809954 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f198c07be00 0x7f198c19b340 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1988009130 tx=0x7f1988011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:02.740 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.083+0000 7f1982ffd700 1 -- 172.21.15.101:0/3186809954 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1988003ab0 con 0x7f198c07be00 2023-07-18T02:58:02.740 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.083+0000 7f1982ffd700 1 -- 172.21.15.101:0/3186809954 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1988003c10 con 0x7f198c07be00 2023-07-18T02:58:02.740 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.083+0000 7f1982ffd700 1 -- 172.21.15.101:0/3186809954 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f19880247d0 con 0x7f198c07be00 2023-07-18T02:58:02.741 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.083+0000 7f1993eee700 1 -- 172.21.15.101:0/3186809954 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f198c19ba80 con 0x7f198c07be00 2023-07-18T02:58:02.741 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.083+0000 7f1993eee700 1 -- 172.21.15.101:0/3186809954 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f198c19bea0 con 0x7f198c07be00 2023-07-18T02:58:02.741 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.084+0000 7f1982ffd700 1 -- 172.21.15.101:0/3186809954 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f198801abb0 con 0x7f198c07be00 2023-07-18T02:58:02.741 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.084+0000 7f1993eee700 1 -- 172.21.15.101:0/3186809954 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f198c1952f0 con 0x7f198c07be00 2023-07-18T02:58:02.741 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.085+0000 7f1982ffd700 1 --2- 172.21.15.101:0/3186809954 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f1978038760 0x7f197803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:02.741 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.085+0000 7f1982ffd700 1 -- 172.21.15.101:0/3186809954 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f198804c850 con 0x7f198c07be00 2023-07-18T02:58:02.741 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.085+0000 7f1991489700 1 --2- 172.21.15.101:0/3186809954 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f1978038760 0x7f197803ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:02.742 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.085+0000 7f1991489700 1 --2- 172.21.15.101:0/3186809954 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f1978038760 0x7f197803ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f197c006fd0 tx=0x7f197c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:02.742 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.089+0000 7f1982ffd700 1 -- 172.21.15.101:0/3186809954 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1988018530 con 0x7f198c07be00 2023-07-18T02:58:02.742 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.285+0000 7f1993eee700 1 -- 172.21.15.101:0/3186809954 --> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f198c060950 con 0x7f1978038760 2023-07-18T02:58:02.742 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.285+0000 7f1982ffd700 1 -- 172.21.15.101:0/3186809954 <== mgr.14118 v2:172.21.15.101:6800/2461037266 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f198c060950 con 0x7f1978038760 2023-07-18T02:58:02.742 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.288+0000 7f1993eee700 1 -- 172.21.15.101:0/3186809954 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f1978038760 msgr2=0x7f197803ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:02.742 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.288+0000 7f1993eee700 1 --2- 172.21.15.101:0/3186809954 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f1978038760 0x7f197803ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f197c006fd0 tx=0x7f197c006e40 comp rx=0 tx=0).stop 2023-07-18T02:58:02.742 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.288+0000 7f1993eee700 1 -- 172.21.15.101:0/3186809954 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f198c07be00 msgr2=0x7f198c19b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:02.743 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.288+0000 7f1993eee700 1 --2- 172.21.15.101:0/3186809954 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f198c07be00 0x7f198c19b340 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1988009130 tx=0x7f1988011ad0 comp rx=0 tx=0).stop 2023-07-18T02:58:02.743 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.288+0000 7f1993eee700 1 -- 172.21.15.101:0/3186809954 shutdown_connections 2023-07-18T02:58:02.743 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.288+0000 7f1993eee700 1 --2- 172.21.15.101:0/3186809954 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f1978038760 0x7f197803ac10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:02.743 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.288+0000 7f1993eee700 1 --2- 172.21.15.101:0/3186809954 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f198c07be00 0x7f198c19b340 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:02.743 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.288+0000 7f1993eee700 1 -- 172.21.15.101:0/3186809954 >> 172.21.15.101:0/3186809954 conn(0x7f198c100970 msgr2=0x7f198c1020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:58:02.743 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.288+0000 7f1993eee700 1 -- 172.21.15.101:0/3186809954 shutdown_connections 2023-07-18T02:58:02.743 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:02.288+0000 7f1993eee700 1 -- 172.21.15.101:0/3186809954 wait complete. 2023-07-18T02:58:02.744 INFO:teuthology.orchestra.run.smithi101.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-07-18T02:58:02.744 INFO:teuthology.orchestra.run.smithi101.stdout:Adding key to root@localhost authorized_keys... 2023-07-18T02:58:02.744 INFO:teuthology.orchestra.run.smithi101.stdout:Adding host smithi101... 2023-07-18T02:58:03.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:03 smithi101 ceph-mon[126379]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T02:58:04.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:04 smithi101 ceph-mon[126379]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi101", "addr": "172.21.15.101", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T02:58:05.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:05 smithi101 ceph-mon[126379]: Deploying cephadm binary to smithi101 2023-07-18T02:58:07.024 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout Added host 'smithi101' with addr '172.21.15.101' 2023-07-18T02:58:07.024 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:03.271+0000 7f1cb95cc700 1 Processor -- start 2023-07-18T02:58:07.024 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:03.271+0000 7f1cb95cc700 1 -- start start 2023-07-18T02:58:07.024 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:03.272+0000 7f1cb95cc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1cb407be20 0x7f1cb407a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:07.025 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:03.272+0000 7f1cb95cc700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb407a860 con 0x7f1cb407be20 2023-07-18T02:58:07.025 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:03.272+0000 7f1cb2ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1cb407be20 0x7f1cb407a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:07.025 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:03.272+0000 7f1cb2ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1cb407be20 0x7f1cb407a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:33412/0 (socket says 172.21.15.101:33412) 2023-07-18T02:58:07.025 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:03.272+0000 7f1cb2ffd700 1 -- 172.21.15.101:0/1943120845 learned_addr learned my addr 172.21.15.101:0/1943120845 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:58:07.025 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:03.272+0000 7f1cb2ffd700 1 -- 172.21.15.101:0/1943120845 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb407a9a0 con 0x7f1cb407be20 2023-07-18T02:58:07.025 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:03.273+0000 7f1cb2ffd700 1 --2- 172.21.15.101:0/1943120845 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1cb407be20 0x7f1cb407a320 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1ca40098a0 tx=0x7f1ca4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fb8865cb818e07c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:07.025 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:03.273+0000 7f1cb1ffb700 1 -- 172.21.15.101:0/1943120845 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1ca4004160 con 0x7f1cb407be20 2023-07-18T02:58:07.026 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:03.273+0000 7f1cb1ffb700 1 -- 172.21.15.101:0/1943120845 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1ca40042c0 con 0x7f1cb407be20 2023-07-18T02:58:07.026 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:03.273+0000 7f1cb1ffb700 1 -- 172.21.15.101:0/1943120845 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1ca4011a00 con 0x7f1cb407be20 2023-07-18T02:58:07.026 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:03.273+0000 7f1cb95cc700 1 -- 172.21.15.101:0/1943120845 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1cb407be20 msgr2=0x7f1cb407a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:07.026 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:03.273+0000 7f1cb95cc700 1 --2- 172.21.15.101:0/1943120845 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1cb407be20 0x7f1cb407a320 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1ca40098a0 tx=0x7f1ca4009bb0 comp rx=0 tx=0).stop 2023-07-18T02:58:07.026 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:03.274+0000 7f1cb95cc700 1 -- 172.21.15.101:0/1943120845 shutdown_connections 2023-07-18T02:58:07.026 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:03.274+0000 7f1cb95cc700 1 --2- 172.21.15.101:0/1943120845 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1cb407be20 0x7f1cb407a320 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:07.026 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:03.274+0000 7f1cb95cc700 1 -- 172.21.15.101:0/1943120845 >> 172.21.15.101:0/1943120845 conn(0x7f1cb4100970 msgr2=0x7f1cb4102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:58:07.027 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:03.274+0000 7f1cb95cc700 1 -- 172.21.15.101:0/1943120845 shutdown_connections 2023-07-18T02:58:07.027 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:03.274+0000 7f1cb95cc700 1 -- 172.21.15.101:0/1943120845 wait complete. 2023-07-18T02:58:07.027 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:03.275+0000 7f1cb95cc700 1 Processor -- start 2023-07-18T02:58:07.027 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:03.275+0000 7f1cb95cc700 1 -- start start 2023-07-18T02:58:07.027 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:03.275+0000 7f1cb95cc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1cb407be20 0x7f1cb419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:07.027 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:03.275+0000 7f1cb95cc700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb419b880 con 0x7f1cb407be20 2023-07-18T02:58:07.027 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:03.275+0000 7f1cb2ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1cb407be20 0x7f1cb419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:07.028 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:03.275+0000 7f1cb2ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1cb407be20 0x7f1cb419b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:33414/0 (socket says 172.21.15.101:33414) 2023-07-18T02:58:07.028 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:03.275+0000 7f1cb2ffd700 1 -- 172.21.15.101:0/4044343612 learned_addr learned my addr 172.21.15.101:0/4044343612 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:58:07.028 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:03.276+0000 7f1cb2ffd700 1 -- 172.21.15.101:0/4044343612 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca4009550 con 0x7f1cb407be20 2023-07-18T02:58:07.028 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:03.276+0000 7f1cb2ffd700 1 --2- 172.21.15.101:0/4044343612 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1cb407be20 0x7f1cb419b340 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4003f50 tx=0x7f1ca4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:07.028 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:03.276+0000 7f1c9bfff700 1 -- 172.21.15.101:0/4044343612 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1ca4003aa0 con 0x7f1cb407be20 2023-07-18T02:58:07.029 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:03.276+0000 7f1c9bfff700 1 -- 172.21.15.101:0/4044343612 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1ca4003c00 con 0x7f1cb407be20 2023-07-18T02:58:07.029 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:03.276+0000 7f1cb95cc700 1 -- 172.21.15.101:0/4044343612 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cb419ba80 con 0x7f1cb407be20 2023-07-18T02:58:07.029 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:03.276+0000 7f1c9bfff700 1 -- 172.21.15.101:0/4044343612 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1ca40247c0 con 0x7f1cb407be20 2023-07-18T02:58:07.029 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:03.276+0000 7f1cb95cc700 1 -- 172.21.15.101:0/4044343612 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cb419bea0 con 0x7f1cb407be20 2023-07-18T02:58:07.030 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:03.277+0000 7f1c9bfff700 1 -- 172.21.15.101:0/4044343612 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f1ca401ab60 con 0x7f1cb407be20 2023-07-18T02:58:07.030 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:03.277+0000 7f1cb95cc700 1 -- 172.21.15.101:0/4044343612 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cb4195320 con 0x7f1cb407be20 2023-07-18T02:58:07.030 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:03.278+0000 7f1c9bfff700 1 --2- 172.21.15.101:0/4044343612 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f1c9c038710 0x7f1c9c03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:07.031 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:03.278+0000 7f1c9bfff700 1 -- 172.21.15.101:0/4044343612 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f1ca404c770 con 0x7f1cb407be20 2023-07-18T02:58:07.031 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:03.278+0000 7f1cb27fc700 1 --2- 172.21.15.101:0/4044343612 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f1c9c038710 0x7f1c9c03abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:07.031 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:03.278+0000 7f1cb27fc700 1 --2- 172.21.15.101:0/4044343612 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f1c9c038710 0x7f1c9c03abc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8006fd0 tx=0x7f1ca8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:07.032 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:03.283+0000 7f1c9bfff700 1 -- 172.21.15.101:0/4044343612 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1ca4024920 con 0x7f1cb407be20 2023-07-18T02:58:07.032 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:03.481+0000 7f1cb95cc700 1 -- 172.21.15.101:0/4044343612 --> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi101", "addr": "172.21.15.101", "target": ["mon-mgr", ""]}) v1 -- 0x7f1cb4060950 con 0x7f1c9c038710 2023-07-18T02:58:07.032 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:06.600+0000 7f1c9bfff700 1 -- 172.21.15.101:0/4044343612 <== mgr.14118 v2:172.21.15.101:6800/2461037266 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f1cb4060950 con 0x7f1c9c038710 2023-07-18T02:58:07.032 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:06.603+0000 7f1cb95cc700 1 -- 172.21.15.101:0/4044343612 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f1c9c038710 msgr2=0x7f1c9c03abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:07.033 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:06.603+0000 7f1cb95cc700 1 --2- 172.21.15.101:0/4044343612 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f1c9c038710 0x7f1c9c03abc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8006fd0 tx=0x7f1ca8006e40 comp rx=0 tx=0).stop 2023-07-18T02:58:07.033 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:06.603+0000 7f1cb95cc700 1 -- 172.21.15.101:0/4044343612 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1cb407be20 msgr2=0x7f1cb419b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:07.033 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:06.603+0000 7f1cb95cc700 1 --2- 172.21.15.101:0/4044343612 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1cb407be20 0x7f1cb419b340 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4003f50 tx=0x7f1ca4011ad0 comp rx=0 tx=0).stop 2023-07-18T02:58:07.034 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:06.604+0000 7f1cb95cc700 1 -- 172.21.15.101:0/4044343612 shutdown_connections 2023-07-18T02:58:07.034 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:06.604+0000 7f1cb95cc700 1 --2- 172.21.15.101:0/4044343612 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f1c9c038710 0x7f1c9c03abc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:07.034 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:06.604+0000 7f1cb95cc700 1 --2- 172.21.15.101:0/4044343612 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1cb407be20 0x7f1cb419b340 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:07.035 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:06.604+0000 7f1cb95cc700 1 -- 172.21.15.101:0/4044343612 >> 172.21.15.101:0/4044343612 conn(0x7f1cb4100970 msgr2=0x7f1cb4102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:58:07.035 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:06.604+0000 7f1cb95cc700 1 -- 172.21.15.101:0/4044343612 shutdown_connections 2023-07-18T02:58:07.035 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:06.604+0000 7f1cb95cc700 1 -- 172.21.15.101:0/4044343612 wait complete. 2023-07-18T02:58:07.035 INFO:teuthology.orchestra.run.smithi101.stdout:Deploying unmanaged mon service... 2023-07-18T02:58:07.717 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:07 smithi101 ceph-mon[126379]: from='mgr.14118 172.21.15.101:0/4056684411' entity='mgr.x' 2023-07-18T02:58:07.717 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:07 smithi101 ceph-mon[126379]: Added host smithi101 2023-07-18T02:58:07.717 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:07 smithi101 ceph-mon[126379]: from='mgr.14118 172.21.15.101:0/4056684411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T02:58:08.496 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-07-18T02:58:08.496 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.626+0000 7fbe2da28700 1 Processor -- start 2023-07-18T02:58:08.496 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.626+0000 7fbe2da28700 1 -- start start 2023-07-18T02:58:08.496 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.627+0000 7fbe2da28700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbe2807be00 0x7fbe2807a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:08.497 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.627+0000 7fbe2da28700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2807a840 con 0x7fbe2807be00 2023-07-18T02:58:08.500 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.627+0000 7fbe26ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbe2807be00 0x7fbe2807a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:08.501 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.627+0000 7fbe26ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbe2807be00 0x7fbe2807a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:51356/0 (socket says 172.21.15.101:51356) 2023-07-18T02:58:08.501 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.627+0000 7fbe26ffd700 1 -- 172.21.15.101:0/168221381 learned_addr learned my addr 172.21.15.101:0/168221381 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:58:08.501 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.627+0000 7fbe26ffd700 1 -- 172.21.15.101:0/168221381 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe2807a980 con 0x7fbe2807be00 2023-07-18T02:58:08.501 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.628+0000 7fbe26ffd700 1 --2- 172.21.15.101:0/168221381 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbe2807be00 0x7fbe2807a300 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbe180098a0 tx=0x7fbe18009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29214e4aeeb1b02b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:08.501 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.628+0000 7fbe25ffb700 1 -- 172.21.15.101:0/168221381 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe18004160 con 0x7fbe2807be00 2023-07-18T02:58:08.501 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.628+0000 7fbe25ffb700 1 -- 172.21.15.101:0/168221381 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbe180042c0 con 0x7fbe2807be00 2023-07-18T02:58:08.502 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.628+0000 7fbe25ffb700 1 -- 172.21.15.101:0/168221381 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe18011a00 con 0x7fbe2807be00 2023-07-18T02:58:08.502 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.628+0000 7fbe2da28700 1 -- 172.21.15.101:0/168221381 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbe2807be00 msgr2=0x7fbe2807a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:08.502 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.628+0000 7fbe2da28700 1 --2- 172.21.15.101:0/168221381 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbe2807be00 0x7fbe2807a300 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbe180098a0 tx=0x7fbe18009bb0 comp rx=0 tx=0).stop 2023-07-18T02:58:08.502 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.628+0000 7fbe2da28700 1 -- 172.21.15.101:0/168221381 shutdown_connections 2023-07-18T02:58:08.502 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.628+0000 7fbe2da28700 1 --2- 172.21.15.101:0/168221381 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbe2807be00 0x7fbe2807a300 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:08.502 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.628+0000 7fbe2da28700 1 -- 172.21.15.101:0/168221381 >> 172.21.15.101:0/168221381 conn(0x7fbe28100970 msgr2=0x7fbe28102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:58:08.502 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.629+0000 7fbe2da28700 1 -- 172.21.15.101:0/168221381 shutdown_connections 2023-07-18T02:58:08.502 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.629+0000 7fbe2da28700 1 -- 172.21.15.101:0/168221381 wait complete. 2023-07-18T02:58:08.503 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.629+0000 7fbe2da28700 1 Processor -- start 2023-07-18T02:58:08.503 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.630+0000 7fbe2da28700 1 -- start start 2023-07-18T02:58:08.503 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.630+0000 7fbe2da28700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbe2807be00 0x7fbe2819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:08.503 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.630+0000 7fbe2da28700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2819b880 con 0x7fbe2807be00 2023-07-18T02:58:08.503 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.630+0000 7fbe26ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbe2807be00 0x7fbe2819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:08.503 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.630+0000 7fbe26ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbe2807be00 0x7fbe2819b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:51362/0 (socket says 172.21.15.101:51362) 2023-07-18T02:58:08.503 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.630+0000 7fbe26ffd700 1 -- 172.21.15.101:0/455532681 learned_addr learned my addr 172.21.15.101:0/455532681 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:58:08.504 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.630+0000 7fbe26ffd700 1 -- 172.21.15.101:0/455532681 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe18009550 con 0x7fbe2807be00 2023-07-18T02:58:08.504 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.631+0000 7fbe26ffd700 1 --2- 172.21.15.101:0/455532681 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbe2807be00 0x7fbe2819b340 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbe18009130 tx=0x7fbe18011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:08.504 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.631+0000 7fbe2ca26700 1 -- 172.21.15.101:0/455532681 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe18003ab0 con 0x7fbe2807be00 2023-07-18T02:58:08.504 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.631+0000 7fbe2ca26700 1 -- 172.21.15.101:0/455532681 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbe18003c10 con 0x7fbe2807be00 2023-07-18T02:58:08.504 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.631+0000 7fbe2ca26700 1 -- 172.21.15.101:0/455532681 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe180247d0 con 0x7fbe2807be00 2023-07-18T02:58:08.504 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.631+0000 7fbe2da28700 1 -- 172.21.15.101:0/455532681 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe2819ba80 con 0x7fbe2807be00 2023-07-18T02:58:08.505 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.631+0000 7fbe2da28700 1 -- 172.21.15.101:0/455532681 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe2819bea0 con 0x7fbe2807be00 2023-07-18T02:58:08.505 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.632+0000 7fbe2ca26700 1 -- 172.21.15.101:0/455532681 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fbe1801abb0 con 0x7fbe2807be00 2023-07-18T02:58:08.505 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.632+0000 7fbe2da28700 1 -- 172.21.15.101:0/455532681 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe281952f0 con 0x7fbe2807be00 2023-07-18T02:58:08.505 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.633+0000 7fbe2ca26700 1 --2- 172.21.15.101:0/455532681 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7fbe10038760 0x7fbe1003ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:08.505 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.633+0000 7fbe2ca26700 1 -- 172.21.15.101:0/455532681 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fbe1804c850 con 0x7fbe2807be00 2023-07-18T02:58:08.505 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.633+0000 7fbe267fc700 1 --2- 172.21.15.101:0/455532681 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7fbe10038760 0x7fbe1003ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:08.505 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.633+0000 7fbe267fc700 1 --2- 172.21.15.101:0/455532681 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7fbe10038760 0x7fbe1003ac10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c006fd0 tx=0x7fbe1c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:08.506 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.637+0000 7fbe2ca26700 1 -- 172.21.15.101:0/455532681 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fbe18018530 con 0x7fbe2807be00 2023-07-18T02:58:08.506 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.852+0000 7fbe2da28700 1 -- 172.21.15.101:0/455532681 --> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fbe28060950 con 0x7fbe10038760 2023-07-18T02:58:08.506 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.861+0000 7fbe2ca26700 1 -- 172.21.15.101:0/455532681 <== mgr.14118 v2:172.21.15.101:6800/2461037266 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fbe28060950 con 0x7fbe10038760 2023-07-18T02:58:08.506 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.863+0000 7fbe2da28700 1 -- 172.21.15.101:0/455532681 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7fbe10038760 msgr2=0x7fbe1003ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:08.506 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.863+0000 7fbe2da28700 1 --2- 172.21.15.101:0/455532681 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7fbe10038760 0x7fbe1003ac10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c006fd0 tx=0x7fbe1c006e40 comp rx=0 tx=0).stop 2023-07-18T02:58:08.506 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.863+0000 7fbe2da28700 1 -- 172.21.15.101:0/455532681 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbe2807be00 msgr2=0x7fbe2819b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:08.507 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.863+0000 7fbe2da28700 1 --2- 172.21.15.101:0/455532681 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbe2807be00 0x7fbe2819b340 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbe18009130 tx=0x7fbe18011ad0 comp rx=0 tx=0).stop 2023-07-18T02:58:08.507 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.865+0000 7fbe2da28700 1 -- 172.21.15.101:0/455532681 shutdown_connections 2023-07-18T02:58:08.507 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.865+0000 7fbe2da28700 1 --2- 172.21.15.101:0/455532681 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7fbe10038760 0x7fbe1003ac10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:08.507 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.865+0000 7fbe2da28700 1 --2- 172.21.15.101:0/455532681 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbe2807be00 0x7fbe2819b340 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:08.507 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.865+0000 7fbe2da28700 1 -- 172.21.15.101:0/455532681 >> 172.21.15.101:0/455532681 conn(0x7fbe28100970 msgr2=0x7fbe281020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:58:08.507 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.865+0000 7fbe2da28700 1 -- 172.21.15.101:0/455532681 shutdown_connections 2023-07-18T02:58:08.507 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:07.865+0000 7fbe2da28700 1 -- 172.21.15.101:0/455532681 wait complete. 2023-07-18T02:58:08.508 INFO:teuthology.orchestra.run.smithi101.stdout:Deploying unmanaged mgr service... 2023-07-18T02:58:08.967 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:08 smithi101 ceph-mon[126379]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T02:58:08.968 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:08 smithi101 ceph-mon[126379]: Saving service mon spec with placement count:5 2023-07-18T02:58:08.968 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:08 smithi101 ceph-mon[126379]: from='mgr.14118 172.21.15.101:0/4056684411' entity='mgr.x' 2023-07-18T02:58:09.788 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-07-18T02:58:09.789 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.063+0000 7ff92e94f700 1 Processor -- start 2023-07-18T02:58:09.789 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.063+0000 7ff92e94f700 1 -- start start 2023-07-18T02:58:09.789 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.063+0000 7ff92e94f700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff928105e90 0x7ff928108270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:09.790 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.063+0000 7ff92e94f700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff928079d40 con 0x7ff928105e90 2023-07-18T02:58:09.790 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.064+0000 7ff927fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff928105e90 0x7ff928108270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:09.790 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.064+0000 7ff927fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff928105e90 0x7ff928108270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:51378/0 (socket says 172.21.15.101:51378) 2023-07-18T02:58:09.790 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.064+0000 7ff927fff700 1 -- 172.21.15.101:0/3883924176 learned_addr learned my addr 172.21.15.101:0/3883924176 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:58:09.790 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.064+0000 7ff927fff700 1 -- 172.21.15.101:0/3883924176 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff928079e80 con 0x7ff928105e90 2023-07-18T02:58:09.790 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.065+0000 7ff927fff700 1 --2- 172.21.15.101:0/3883924176 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff928105e90 0x7ff928108270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff9180098a0 tx=0x7ff918009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bf35c1e7c7d90d2 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:09.791 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.065+0000 7ff926ffd700 1 -- 172.21.15.101:0/3883924176 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff918004160 con 0x7ff928105e90 2023-07-18T02:58:09.791 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.065+0000 7ff926ffd700 1 -- 172.21.15.101:0/3883924176 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff9180042c0 con 0x7ff928105e90 2023-07-18T02:58:09.791 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.065+0000 7ff926ffd700 1 -- 172.21.15.101:0/3883924176 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff918011a00 con 0x7ff928105e90 2023-07-18T02:58:09.791 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.065+0000 7ff92e94f700 1 -- 172.21.15.101:0/3883924176 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff928105e90 msgr2=0x7ff928108270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:09.791 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.065+0000 7ff92e94f700 1 --2- 172.21.15.101:0/3883924176 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff928105e90 0x7ff928108270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff9180098a0 tx=0x7ff918009bb0 comp rx=0 tx=0).stop 2023-07-18T02:58:09.791 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.066+0000 7ff92e94f700 1 -- 172.21.15.101:0/3883924176 shutdown_connections 2023-07-18T02:58:09.792 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.066+0000 7ff92e94f700 1 --2- 172.21.15.101:0/3883924176 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff928105e90 0x7ff928108270 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:09.792 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.066+0000 7ff92e94f700 1 -- 172.21.15.101:0/3883924176 >> 172.21.15.101:0/3883924176 conn(0x7ff9280ff890 msgr2=0x7ff928101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:58:09.792 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.066+0000 7ff92e94f700 1 -- 172.21.15.101:0/3883924176 shutdown_connections 2023-07-18T02:58:09.792 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.066+0000 7ff92e94f700 1 -- 172.21.15.101:0/3883924176 wait complete. 2023-07-18T02:58:09.792 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.067+0000 7ff92e94f700 1 Processor -- start 2023-07-18T02:58:09.793 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.067+0000 7ff92e94f700 1 -- start start 2023-07-18T02:58:09.793 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.067+0000 7ff92e94f700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff928105e90 0x7ff9281057f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:09.793 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.067+0000 7ff92e94f700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff928105d30 con 0x7ff928105e90 2023-07-18T02:58:09.793 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.067+0000 7ff927fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff928105e90 0x7ff9281057f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:09.793 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.067+0000 7ff927fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff928105e90 0x7ff9281057f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:51380/0 (socket says 172.21.15.101:51380) 2023-07-18T02:58:09.793 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.067+0000 7ff927fff700 1 -- 172.21.15.101:0/2931610782 learned_addr learned my addr 172.21.15.101:0/2931610782 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:58:09.794 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.068+0000 7ff927fff700 1 -- 172.21.15.101:0/2931610782 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff918009550 con 0x7ff928105e90 2023-07-18T02:58:09.794 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.068+0000 7ff927fff700 1 --2- 172.21.15.101:0/2931610782 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff928105e90 0x7ff9281057f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff918009130 tx=0x7ff918011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:09.794 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.068+0000 7ff9257fa700 1 -- 172.21.15.101:0/2931610782 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff918003ab0 con 0x7ff928105e90 2023-07-18T02:58:09.794 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.068+0000 7ff9257fa700 1 -- 172.21.15.101:0/2931610782 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff918003c10 con 0x7ff928105e90 2023-07-18T02:58:09.794 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.068+0000 7ff92e94f700 1 -- 172.21.15.101:0/2931610782 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff928103e40 con 0x7ff928105e90 2023-07-18T02:58:09.794 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.068+0000 7ff9257fa700 1 -- 172.21.15.101:0/2931610782 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff9180247d0 con 0x7ff928105e90 2023-07-18T02:58:09.794 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.068+0000 7ff92e94f700 1 -- 172.21.15.101:0/2931610782 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff928104260 con 0x7ff928105e90 2023-07-18T02:58:09.795 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.069+0000 7ff92e94f700 1 -- 172.21.15.101:0/2931610782 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff928061b40 con 0x7ff928105e90 2023-07-18T02:58:09.795 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.070+0000 7ff9257fa700 1 -- 172.21.15.101:0/2931610782 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7ff91801abb0 con 0x7ff928105e90 2023-07-18T02:58:09.795 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.071+0000 7ff9257fa700 1 --2- 172.21.15.101:0/2931610782 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7ff910038760 0x7ff91003ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:09.795 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.071+0000 7ff9257fa700 1 -- 172.21.15.101:0/2931610782 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff91804c850 con 0x7ff928105e90 2023-07-18T02:58:09.795 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.071+0000 7ff9277fe700 1 --2- 172.21.15.101:0/2931610782 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7ff910038760 0x7ff91003ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:09.795 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.072+0000 7ff9277fe700 1 --2- 172.21.15.101:0/2931610782 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7ff910038760 0x7ff91003ac10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff91c006fd0 tx=0x7ff91c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:09.796 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.076+0000 7ff9257fa700 1 -- 172.21.15.101:0/2931610782 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff918018530 con 0x7ff928105e90 2023-07-18T02:58:09.796 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.308+0000 7ff92e94f700 1 -- 172.21.15.101:0/2931610782 --> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7ff928101be0 con 0x7ff910038760 2023-07-18T02:58:09.796 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.315+0000 7ff9257fa700 1 -- 172.21.15.101:0/2931610782 <== mgr.14118 v2:172.21.15.101:6800/2461037266 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7ff928101be0 con 0x7ff910038760 2023-07-18T02:58:09.796 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.318+0000 7ff92e94f700 1 -- 172.21.15.101:0/2931610782 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7ff910038760 msgr2=0x7ff91003ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:09.796 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.318+0000 7ff92e94f700 1 --2- 172.21.15.101:0/2931610782 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7ff910038760 0x7ff91003ac10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff91c006fd0 tx=0x7ff91c006e40 comp rx=0 tx=0).stop 2023-07-18T02:58:09.796 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.318+0000 7ff92e94f700 1 -- 172.21.15.101:0/2931610782 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff928105e90 msgr2=0x7ff9281057f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:09.797 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.318+0000 7ff92e94f700 1 --2- 172.21.15.101:0/2931610782 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff928105e90 0x7ff9281057f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff918009130 tx=0x7ff918011ad0 comp rx=0 tx=0).stop 2023-07-18T02:58:09.797 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.318+0000 7ff92e94f700 1 -- 172.21.15.101:0/2931610782 shutdown_connections 2023-07-18T02:58:09.797 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.318+0000 7ff92e94f700 1 --2- 172.21.15.101:0/2931610782 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7ff910038760 0x7ff91003ac10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:09.797 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.318+0000 7ff92e94f700 1 --2- 172.21.15.101:0/2931610782 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff928105e90 0x7ff9281057f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:09.797 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.318+0000 7ff92e94f700 1 -- 172.21.15.101:0/2931610782 >> 172.21.15.101:0/2931610782 conn(0x7ff9280ff890 msgr2=0x7ff928100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:58:09.797 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.318+0000 7ff92e94f700 1 -- 172.21.15.101:0/2931610782 shutdown_connections 2023-07-18T02:58:09.798 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:09.318+0000 7ff92e94f700 1 -- 172.21.15.101:0/2931610782 wait complete. 2023-07-18T02:58:10.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:10 smithi101 ceph-mon[126379]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T02:58:10.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:10 smithi101 ceph-mon[126379]: Saving service mgr spec with placement count:2 2023-07-18T02:58:10.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:10 smithi101 ceph-mon[126379]: from='mgr.14118 172.21.15.101:0/4056684411' entity='mgr.x' 2023-07-18T02:58:10.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:10 smithi101 ceph-mon[126379]: from='mgr.14118 172.21.15.101:0/4056684411' entity='mgr.x' 2023-07-18T02:58:10.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:10 smithi101 ceph-mon[126379]: from='mgr.14118 172.21.15.101:0/4056684411' entity='mgr.x' 2023-07-18T02:58:10.970 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.401+0000 7f99f62ed700 1 Processor -- start 2023-07-18T02:58:10.970 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.401+0000 7f99f62ed700 1 -- start start 2023-07-18T02:58:10.970 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.401+0000 7f99f62ed700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f99f0105140 0x7f99f0105550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:10.971 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.401+0000 7f99f62ed700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f0105a90 con 0x7f99f0105140 2023-07-18T02:58:10.971 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.402+0000 7f99f52eb700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f99f0105140 0x7f99f0105550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:10.971 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.402+0000 7f99f52eb700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f99f0105140 0x7f99f0105550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:51382/0 (socket says 172.21.15.101:51382) 2023-07-18T02:58:10.971 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.402+0000 7f99f52eb700 1 -- 172.21.15.101:0/1936523214 learned_addr learned my addr 172.21.15.101:0/1936523214 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:58:10.971 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.402+0000 7f99f52eb700 1 -- 172.21.15.101:0/1936523214 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f0105bd0 con 0x7f99f0105140 2023-07-18T02:58:10.971 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.402+0000 7f99f52eb700 1 --2- 172.21.15.101:0/1936523214 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f99f0105140 0x7f99f0105550 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f99ec0098a0 tx=0x7f99ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f71c441ae7fc1 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:10.971 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.402+0000 7f99e7fff700 1 -- 172.21.15.101:0/1936523214 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f99ec004160 con 0x7f99f0105140 2023-07-18T02:58:10.971 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.402+0000 7f99e7fff700 1 -- 172.21.15.101:0/1936523214 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99ec0042c0 con 0x7f99f0105140 2023-07-18T02:58:10.972 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.402+0000 7f99e7fff700 1 -- 172.21.15.101:0/1936523214 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f99ec0119b0 con 0x7f99f0105140 2023-07-18T02:58:10.972 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.403+0000 7f99f62ed700 1 -- 172.21.15.101:0/1936523214 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f99f0105140 msgr2=0x7f99f0105550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:10.972 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.403+0000 7f99f62ed700 1 --2- 172.21.15.101:0/1936523214 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f99f0105140 0x7f99f0105550 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f99ec0098a0 tx=0x7f99ec009bb0 comp rx=0 tx=0).stop 2023-07-18T02:58:10.972 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.403+0000 7f99f62ed700 1 -- 172.21.15.101:0/1936523214 shutdown_connections 2023-07-18T02:58:10.972 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.403+0000 7f99f62ed700 1 --2- 172.21.15.101:0/1936523214 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f99f0105140 0x7f99f0105550 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:10.972 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.403+0000 7f99f62ed700 1 -- 172.21.15.101:0/1936523214 >> 172.21.15.101:0/1936523214 conn(0x7f99f01007f0 msgr2=0x7f99f0102c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:58:10.972 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.403+0000 7f99f62ed700 1 -- 172.21.15.101:0/1936523214 shutdown_connections 2023-07-18T02:58:10.973 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.403+0000 7f99f62ed700 1 -- 172.21.15.101:0/1936523214 wait complete. 2023-07-18T02:58:10.973 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.404+0000 7f99f62ed700 1 Processor -- start 2023-07-18T02:58:10.973 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.405+0000 7f99f62ed700 1 -- start start 2023-07-18T02:58:10.973 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.405+0000 7f99f62ed700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f99f0105140 0x7f99f019b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:10.973 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.405+0000 7f99f62ed700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f019b630 con 0x7f99f0105140 2023-07-18T02:58:10.973 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.405+0000 7f99f52eb700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f99f0105140 0x7f99f019b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:10.973 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.405+0000 7f99f52eb700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f99f0105140 0x7f99f019b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:51394/0 (socket says 172.21.15.101:51394) 2023-07-18T02:58:10.974 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.405+0000 7f99f52eb700 1 -- 172.21.15.101:0/3110788249 learned_addr learned my addr 172.21.15.101:0/3110788249 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:58:10.974 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.406+0000 7f99f52eb700 1 -- 172.21.15.101:0/3110788249 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99ec009550 con 0x7f99f0105140 2023-07-18T02:58:10.974 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.406+0000 7f99f52eb700 1 --2- 172.21.15.101:0/3110788249 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f99f0105140 0x7f99f019b0f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f99ec011980 tx=0x7f99ec011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:10.974 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.406+0000 7f99e67fc700 1 -- 172.21.15.101:0/3110788249 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f99ec003a60 con 0x7f99f0105140 2023-07-18T02:58:10.974 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.406+0000 7f99f62ed700 1 -- 172.21.15.101:0/3110788249 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99f019b830 con 0x7f99f0105140 2023-07-18T02:58:10.974 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.406+0000 7f99e67fc700 1 -- 172.21.15.101:0/3110788249 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99ec003bc0 con 0x7f99f0105140 2023-07-18T02:58:10.974 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.406+0000 7f99e67fc700 1 -- 172.21.15.101:0/3110788249 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f99ec01a670 con 0x7f99f0105140 2023-07-18T02:58:10.975 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.406+0000 7f99f62ed700 1 -- 172.21.15.101:0/3110788249 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99f019bc50 con 0x7f99f0105140 2023-07-18T02:58:10.975 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.407+0000 7f99e67fc700 1 -- 172.21.15.101:0/3110788249 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f99ec01a930 con 0x7f99f0105140 2023-07-18T02:58:10.975 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.407+0000 7f99f62ed700 1 -- 172.21.15.101:0/3110788249 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99f01095d0 con 0x7f99f0105140 2023-07-18T02:58:10.975 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.408+0000 7f99e67fc700 1 --2- 172.21.15.101:0/3110788249 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f99dc038710 0x7f99dc03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:10.975 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.408+0000 7f99e67fc700 1 -- 172.21.15.101:0/3110788249 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f99ec04c850 con 0x7f99f0105140 2023-07-18T02:58:10.975 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.408+0000 7f99f4aea700 1 --2- 172.21.15.101:0/3110788249 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f99dc038710 0x7f99dc03abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:10.975 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.409+0000 7f99f4aea700 1 --2- 172.21.15.101:0/3110788249 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f99dc038710 0x7f99dc03abc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f99e0006fd0 tx=0x7f99e0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:10.975 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.413+0000 7f99e67fc700 1 -- 172.21.15.101:0/3110788249 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f99ec018b80 con 0x7f99f0105140 2023-07-18T02:58:10.976 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.621+0000 7f99f62ed700 1 -- 172.21.15.101:0/3110788249 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f99f0061b40 con 0x7f99f0105140 2023-07-18T02:58:10.976 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.627+0000 7f99e67fc700 1 -- 172.21.15.101:0/3110788249 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f99ec018350 con 0x7f99f0105140 2023-07-18T02:58:10.976 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.629+0000 7f99f62ed700 1 -- 172.21.15.101:0/3110788249 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f99dc038710 msgr2=0x7f99dc03abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:10.976 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.629+0000 7f99f62ed700 1 --2- 172.21.15.101:0/3110788249 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f99dc038710 0x7f99dc03abc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f99e0006fd0 tx=0x7f99e0006e40 comp rx=0 tx=0).stop 2023-07-18T02:58:10.977 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.629+0000 7f99f62ed700 1 -- 172.21.15.101:0/3110788249 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f99f0105140 msgr2=0x7f99f019b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:10.977 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.629+0000 7f99f62ed700 1 --2- 172.21.15.101:0/3110788249 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f99f0105140 0x7f99f019b0f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f99ec011980 tx=0x7f99ec011a60 comp rx=0 tx=0).stop 2023-07-18T02:58:10.977 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.630+0000 7f99f62ed700 1 -- 172.21.15.101:0/3110788249 shutdown_connections 2023-07-18T02:58:10.977 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.630+0000 7f99f62ed700 1 --2- 172.21.15.101:0/3110788249 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f99dc038710 0x7f99dc03abc0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:10.977 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.630+0000 7f99f62ed700 1 --2- 172.21.15.101:0/3110788249 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f99f0105140 0x7f99f019b0f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:10.977 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.630+0000 7f99f62ed700 1 -- 172.21.15.101:0/3110788249 >> 172.21.15.101:0/3110788249 conn(0x7f99f01007f0 msgr2=0x7f99f0101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:58:10.978 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.630+0000 7f99f62ed700 1 -- 172.21.15.101:0/3110788249 shutdown_connections 2023-07-18T02:58:10.978 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:10.630+0000 7f99f62ed700 1 -- 172.21.15.101:0/3110788249 wait complete. 2023-07-18T02:58:11.952 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:11 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3110788249' entity='client.admin' 2023-07-18T02:58:12.450 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.525+0000 7fde2d0b7700 1 Processor -- start 2023-07-18T02:58:12.451 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.525+0000 7fde2d0b7700 1 -- start start 2023-07-18T02:58:12.451 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.525+0000 7fde2d0b7700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fde2807be00 0x7fde2807a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:12.451 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.525+0000 7fde2d0b7700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde2807a840 con 0x7fde2807be00 2023-07-18T02:58:12.451 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.525+0000 7fde26d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fde2807be00 0x7fde2807a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:12.451 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.525+0000 7fde26d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fde2807be00 0x7fde2807a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:51410/0 (socket says 172.21.15.101:51410) 2023-07-18T02:58:12.451 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.525+0000 7fde26d9d700 1 -- 172.21.15.101:0/3645491194 learned_addr learned my addr 172.21.15.101:0/3645491194 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:58:12.452 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.526+0000 7fde26d9d700 1 -- 172.21.15.101:0/3645491194 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde2807a980 con 0x7fde2807be00 2023-07-18T02:58:12.452 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.526+0000 7fde26d9d700 1 --2- 172.21.15.101:0/3645491194 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fde2807be00 0x7fde2807a300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fde180098a0 tx=0x7fde18009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d15a926add1e7b2a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:12.452 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.526+0000 7fde25d9b700 1 -- 172.21.15.101:0/3645491194 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fde18004160 con 0x7fde2807be00 2023-07-18T02:58:12.452 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.526+0000 7fde25d9b700 1 -- 172.21.15.101:0/3645491194 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde180042c0 con 0x7fde2807be00 2023-07-18T02:58:12.452 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.526+0000 7fde25d9b700 1 -- 172.21.15.101:0/3645491194 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fde180119b0 con 0x7fde2807be00 2023-07-18T02:58:12.452 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.527+0000 7fde2d0b7700 1 -- 172.21.15.101:0/3645491194 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fde2807be00 msgr2=0x7fde2807a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:12.452 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.527+0000 7fde2d0b7700 1 --2- 172.21.15.101:0/3645491194 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fde2807be00 0x7fde2807a300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fde180098a0 tx=0x7fde18009bb0 comp rx=0 tx=0).stop 2023-07-18T02:58:12.453 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.527+0000 7fde2d0b7700 1 -- 172.21.15.101:0/3645491194 shutdown_connections 2023-07-18T02:58:12.453 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.527+0000 7fde2d0b7700 1 --2- 172.21.15.101:0/3645491194 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fde2807be00 0x7fde2807a300 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:12.453 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.527+0000 7fde2d0b7700 1 -- 172.21.15.101:0/3645491194 >> 172.21.15.101:0/3645491194 conn(0x7fde28100970 msgr2=0x7fde28102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:58:12.453 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.527+0000 7fde2d0b7700 1 -- 172.21.15.101:0/3645491194 shutdown_connections 2023-07-18T02:58:12.453 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.527+0000 7fde2d0b7700 1 -- 172.21.15.101:0/3645491194 wait complete. 2023-07-18T02:58:12.453 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.528+0000 7fde2d0b7700 1 Processor -- start 2023-07-18T02:58:12.453 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.528+0000 7fde2d0b7700 1 -- start start 2023-07-18T02:58:12.454 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.528+0000 7fde2d0b7700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fde2807be00 0x7fde2819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:12.454 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.528+0000 7fde2d0b7700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde2819b880 con 0x7fde2807be00 2023-07-18T02:58:12.454 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.529+0000 7fde26d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fde2807be00 0x7fde2819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:12.454 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.529+0000 7fde26d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fde2807be00 0x7fde2819b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:51418/0 (socket says 172.21.15.101:51418) 2023-07-18T02:58:12.454 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.529+0000 7fde26d9d700 1 -- 172.21.15.101:0/2387660864 learned_addr learned my addr 172.21.15.101:0/2387660864 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:58:12.454 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.529+0000 7fde26d9d700 1 -- 172.21.15.101:0/2387660864 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde18009550 con 0x7fde2807be00 2023-07-18T02:58:12.454 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.529+0000 7fde26d9d700 1 --2- 172.21.15.101:0/2387660864 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fde2807be00 0x7fde2819b340 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fde18003e60 tx=0x7fde18011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:12.454 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.529+0000 7fde0ffff700 1 -- 172.21.15.101:0/2387660864 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fde18003a60 con 0x7fde2807be00 2023-07-18T02:58:12.455 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.529+0000 7fde0ffff700 1 -- 172.21.15.101:0/2387660864 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde18003bc0 con 0x7fde2807be00 2023-07-18T02:58:12.455 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.529+0000 7fde0ffff700 1 -- 172.21.15.101:0/2387660864 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fde180196c0 con 0x7fde2807be00 2023-07-18T02:58:12.455 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.529+0000 7fde2d0b7700 1 -- 172.21.15.101:0/2387660864 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde2819ba80 con 0x7fde2807be00 2023-07-18T02:58:12.455 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.530+0000 7fde2d0b7700 1 -- 172.21.15.101:0/2387660864 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde2819bea0 con 0x7fde2807be00 2023-07-18T02:58:12.455 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.530+0000 7fde0ffff700 1 -- 172.21.15.101:0/2387660864 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fde1801ab60 con 0x7fde2807be00 2023-07-18T02:58:12.455 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.531+0000 7fde2d0b7700 1 -- 172.21.15.101:0/2387660864 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde28061b40 con 0x7fde2807be00 2023-07-18T02:58:12.455 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.531+0000 7fde0ffff700 1 --2- 172.21.15.101:0/2387660864 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7fde10038740 0x7fde1003abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:12.456 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.531+0000 7fde0ffff700 1 -- 172.21.15.101:0/2387660864 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fde1804c8f0 con 0x7fde2807be00 2023-07-18T02:58:12.456 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.531+0000 7fde2659c700 1 --2- 172.21.15.101:0/2387660864 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7fde10038740 0x7fde1003abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:12.456 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.532+0000 7fde2659c700 1 --2- 172.21.15.101:0/2387660864 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7fde10038740 0x7fde1003abf0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fde1c006fd0 tx=0x7fde1c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:12.456 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.536+0000 7fde0ffff700 1 -- 172.21.15.101:0/2387660864 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fde18018b00 con 0x7fde2807be00 2023-07-18T02:58:12.456 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.741+0000 7fde2d0b7700 1 -- 172.21.15.101:0/2387660864 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fde2819e8e0 con 0x7fde2807be00 2023-07-18T02:58:12.456 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.747+0000 7fde0ffff700 1 -- 172.21.15.101:0/2387660864 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7fde18018b00 con 0x7fde2807be00 2023-07-18T02:58:12.456 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.750+0000 7fde2d0b7700 1 -- 172.21.15.101:0/2387660864 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7fde10038740 msgr2=0x7fde1003abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:12.457 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.750+0000 7fde2d0b7700 1 --2- 172.21.15.101:0/2387660864 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7fde10038740 0x7fde1003abf0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fde1c006fd0 tx=0x7fde1c006e40 comp rx=0 tx=0).stop 2023-07-18T02:58:12.457 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.750+0000 7fde2d0b7700 1 -- 172.21.15.101:0/2387660864 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fde2807be00 msgr2=0x7fde2819b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:12.457 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.750+0000 7fde2d0b7700 1 --2- 172.21.15.101:0/2387660864 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fde2807be00 0x7fde2819b340 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fde18003e60 tx=0x7fde18011a30 comp rx=0 tx=0).stop 2023-07-18T02:58:12.457 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.750+0000 7fde2d0b7700 1 -- 172.21.15.101:0/2387660864 shutdown_connections 2023-07-18T02:58:12.457 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.750+0000 7fde2d0b7700 1 --2- 172.21.15.101:0/2387660864 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7fde10038740 0x7fde1003abf0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:12.457 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.750+0000 7fde2d0b7700 1 --2- 172.21.15.101:0/2387660864 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fde2807be00 0x7fde2819b340 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:12.457 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.750+0000 7fde2d0b7700 1 -- 172.21.15.101:0/2387660864 >> 172.21.15.101:0/2387660864 conn(0x7fde28100970 msgr2=0x7fde281020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:58:12.458 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.751+0000 7fde2d0b7700 1 -- 172.21.15.101:0/2387660864 shutdown_connections 2023-07-18T02:58:12.458 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:11.751+0000 7fde2d0b7700 1 -- 172.21.15.101:0/2387660864 wait complete. 2023-07-18T02:58:12.458 INFO:teuthology.orchestra.run.smithi101.stdout:Enabling the dashboard module... 2023-07-18T02:58:13.045 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:12 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2387660864' entity='client.admin' 2023-07-18T02:58:13.045 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:12 smithi101 ceph-mon[126379]: from='mgr.14118 172.21.15.101:0/4056684411' entity='mgr.x' 2023-07-18T02:58:14.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:14 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2965748193' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-07-18T02:58:14.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:14 smithi101 ceph-mon[126379]: from='mgr.14118 172.21.15.101:0/4056684411' entity='mgr.x' 2023-07-18T02:58:14.468 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:58:14 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: ignoring --setuser ceph since I am not root 2023-07-18T02:58:14.468 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:58:14 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: ignoring --setgroup ceph since I am not root 2023-07-18T02:58:14.629 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:13.030+0000 7f30837d8700 1 Processor -- start 2023-07-18T02:58:14.630 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:13.031+0000 7f30837d8700 1 -- start start 2023-07-18T02:58:14.630 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:13.031+0000 7f30837d8700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f307c104220 0x7f307c104630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:14.630 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:13.031+0000 7f30837d8700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307c104b70 con 0x7f307c104220 2023-07-18T02:58:14.630 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:13.031+0000 7f3081574700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f307c104220 0x7f307c104630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:14.630 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:13.031+0000 7f3081574700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f307c104220 0x7f307c104630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:51424/0 (socket says 172.21.15.101:51424) 2023-07-18T02:58:14.631 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:13.031+0000 7f3081574700 1 -- 172.21.15.101:0/2003879269 learned_addr learned my addr 172.21.15.101:0/2003879269 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:58:14.631 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:13.032+0000 7f3081574700 1 -- 172.21.15.101:0/2003879269 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f307c104cb0 con 0x7f307c104220 2023-07-18T02:58:14.631 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:13.032+0000 7f3081574700 1 --2- 172.21.15.101:0/2003879269 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f307c104220 0x7f307c104630 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f30780098a0 tx=0x7f3078009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e4706d03906217c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:14.631 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:13.032+0000 7f3073fff700 1 -- 172.21.15.101:0/2003879269 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3078004160 con 0x7f307c104220 2023-07-18T02:58:14.631 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:13.032+0000 7f3073fff700 1 -- 172.21.15.101:0/2003879269 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30780042c0 con 0x7f307c104220 2023-07-18T02:58:14.631 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:13.032+0000 7f3073fff700 1 -- 172.21.15.101:0/2003879269 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30780119b0 con 0x7f307c104220 2023-07-18T02:58:14.632 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:13.033+0000 7f30837d8700 1 -- 172.21.15.101:0/2003879269 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f307c104220 msgr2=0x7f307c104630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:14.632 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:13.033+0000 7f30837d8700 1 --2- 172.21.15.101:0/2003879269 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f307c104220 0x7f307c104630 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f30780098a0 tx=0x7f3078009bb0 comp rx=0 tx=0).stop 2023-07-18T02:58:14.632 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:13.033+0000 7f30837d8700 1 -- 172.21.15.101:0/2003879269 shutdown_connections 2023-07-18T02:58:14.632 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:13.033+0000 7f30837d8700 1 --2- 172.21.15.101:0/2003879269 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f307c104220 0x7f307c104630 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:14.632 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:13.033+0000 7f30837d8700 1 -- 172.21.15.101:0/2003879269 >> 172.21.15.101:0/2003879269 conn(0x7f307c0ff890 msgr2=0x7f307c101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:58:14.632 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:13.033+0000 7f30837d8700 1 -- 172.21.15.101:0/2003879269 shutdown_connections 2023-07-18T02:58:14.632 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:13.033+0000 7f30837d8700 1 -- 172.21.15.101:0/2003879269 wait complete. 2023-07-18T02:58:14.633 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:13.034+0000 7f30837d8700 1 Processor -- start 2023-07-18T02:58:14.633 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:13.034+0000 7f30837d8700 1 -- start start 2023-07-18T02:58:14.633 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:13.034+0000 7f30837d8700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f307c104220 0x7f307c1990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:14.633 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:13.034+0000 7f30837d8700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307c199600 con 0x7f307c104220 2023-07-18T02:58:14.633 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:13.035+0000 7f3081574700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f307c104220 0x7f307c1990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:14.633 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:13.035+0000 7f3081574700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f307c104220 0x7f307c1990c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:51436/0 (socket says 172.21.15.101:51436) 2023-07-18T02:58:14.633 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:13.035+0000 7f3081574700 1 -- 172.21.15.101:0/2965748193 learned_addr learned my addr 172.21.15.101:0/2965748193 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:58:14.633 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:13.035+0000 7f3081574700 1 -- 172.21.15.101:0/2965748193 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3078009550 con 0x7f307c104220 2023-07-18T02:58:14.634 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:13.035+0000 7f3081574700 1 --2- 172.21.15.101:0/2965748193 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f307c104220 0x7f307c1990c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3078006af0 tx=0x7f3078011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:14.634 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:13.035+0000 7f30727fc700 1 -- 172.21.15.101:0/2965748193 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3078003a40 con 0x7f307c104220 2023-07-18T02:58:14.634 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:13.035+0000 7f30727fc700 1 -- 172.21.15.101:0/2965748193 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3078003ba0 con 0x7f307c104220 2023-07-18T02:58:14.634 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:13.035+0000 7f30837d8700 1 -- 172.21.15.101:0/2965748193 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f307c199800 con 0x7f307c104220 2023-07-18T02:58:14.634 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:13.035+0000 7f30727fc700 1 -- 172.21.15.101:0/2965748193 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30780196c0 con 0x7f307c104220 2023-07-18T02:58:14.634 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:13.036+0000 7f30837d8700 1 -- 172.21.15.101:0/2965748193 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f307c199c20 con 0x7f307c104220 2023-07-18T02:58:14.634 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:13.037+0000 7f30727fc700 1 -- 172.21.15.101:0/2965748193 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f307801aa80 con 0x7f307c104220 2023-07-18T02:58:14.635 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:13.037+0000 7f30837d8700 1 -- 172.21.15.101:0/2965748193 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f307c061b40 con 0x7f307c104220 2023-07-18T02:58:14.635 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:13.037+0000 7f30727fc700 1 --2- 172.21.15.101:0/2965748193 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f30680386c0 0x7f306803ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:14.635 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:13.037+0000 7f30727fc700 1 -- 172.21.15.101:0/2965748193 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f307804c7d0 con 0x7f307c104220 2023-07-18T02:58:14.635 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:13.037+0000 7f3080d73700 1 --2- 172.21.15.101:0/2965748193 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f30680386c0 0x7f306803ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:14.635 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:13.038+0000 7f3080d73700 1 --2- 172.21.15.101:0/2965748193 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f30680386c0 0x7f306803ab70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f306c006fd0 tx=0x7f306c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:14.635 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:13.042+0000 7f30727fc700 1 -- 172.21.15.101:0/2965748193 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f307801a430 con 0x7f307c104220 2023-07-18T02:58:14.635 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:13.305+0000 7f30837d8700 1 -- 172.21.15.101:0/2965748193 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f307c19cca0 con 0x7f307c104220 2023-07-18T02:58:14.636 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:14.203+0000 7f30727fc700 1 -- 172.21.15.101:0/2965748193 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f307802d430 con 0x7f307c104220 2023-07-18T02:58:14.636 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:14.205+0000 7f30727fc700 1 -- 172.21.15.101:0/2965748193 <== mon.0 v2:172.21.15.101:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f307804c530 con 0x7f307c104220 2023-07-18T02:58:14.636 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:14.208+0000 7f30837d8700 1 -- 172.21.15.101:0/2965748193 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f30680386c0 msgr2=0x7f306803ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:14.636 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:14.208+0000 7f30837d8700 1 --2- 172.21.15.101:0/2965748193 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f30680386c0 0x7f306803ab70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f306c006fd0 tx=0x7f306c006e40 comp rx=0 tx=0).stop 2023-07-18T02:58:14.636 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:14.208+0000 7f30837d8700 1 -- 172.21.15.101:0/2965748193 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f307c104220 msgr2=0x7f307c1990c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:14.636 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:14.208+0000 7f30837d8700 1 --2- 172.21.15.101:0/2965748193 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f307c104220 0x7f307c1990c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3078006af0 tx=0x7f3078011a10 comp rx=0 tx=0).stop 2023-07-18T02:58:14.636 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:14.208+0000 7f30837d8700 1 -- 172.21.15.101:0/2965748193 shutdown_connections 2023-07-18T02:58:14.637 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:14.208+0000 7f30837d8700 1 --2- 172.21.15.101:0/2965748193 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f30680386c0 0x7f306803ab70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:14.637 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:14.208+0000 7f30837d8700 1 --2- 172.21.15.101:0/2965748193 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f307c104220 0x7f307c1990c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:14.637 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:14.208+0000 7f30837d8700 1 -- 172.21.15.101:0/2965748193 >> 172.21.15.101:0/2965748193 conn(0x7f307c0ff890 msgr2=0x7f307c079900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:58:14.637 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:14.208+0000 7f30837d8700 1 -- 172.21.15.101:0/2965748193 shutdown_connections 2023-07-18T02:58:14.637 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:14.209+0000 7f30837d8700 1 -- 172.21.15.101:0/2965748193 wait complete. 2023-07-18T02:58:14.831 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:58:14 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:58:14.503+0000 7f2a833f7200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-18T02:58:14.831 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:58:14 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:58:14.611+0000 7f2a833f7200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-18T02:58:14.831 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:58:14 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:58:14.734+0000 7f2a833f7200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-18T02:58:15.085 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:58:14 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:58:14.858+0000 7f2a833f7200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-18T02:58:15.352 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:15 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2965748193' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-07-18T02:58:15.353 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:15 smithi101 ceph-mon[126379]: mgrmap e9: x(active, since 17s) 2023-07-18T02:58:15.353 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:58:15 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:58:15.113+0000 7f2a833f7200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-18T02:58:15.635 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:58:15 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:58:15.562+0000 7f2a833f7200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-18T02:58:15.899 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:58:15 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:58:15.702+0000 7f2a833f7200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-18T02:58:15.900 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:58:15 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:58:15.866+0000 7f2a833f7200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-18T02:58:16.163 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:58:15 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:58:15.991+0000 7f2a833f7200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-18T02:58:16.212 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout { 2023-07-18T02:58:16.214 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-07-18T02:58:16.214 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "available": true, 2023-07-18T02:58:16.214 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-07-18T02:58:16.214 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-07-18T02:58:16.214 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout } 2023-07-18T02:58:16.215 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.277+0000 7f419431e700 1 Processor -- start 2023-07-18T02:58:16.215 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.278+0000 7f419431e700 1 -- start start 2023-07-18T02:58:16.216 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.278+0000 7f419431e700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f418c07be00 0x7f418c07a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:16.216 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.278+0000 7f419431e700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f418c07a840 con 0x7f418c07be00 2023-07-18T02:58:16.216 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.278+0000 7f41920ba700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f418c07be00 0x7f418c07a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:16.216 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.278+0000 7f41920ba700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f418c07be00 0x7f418c07a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:51462/0 (socket says 172.21.15.101:51462) 2023-07-18T02:58:16.220 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.278+0000 7f41920ba700 1 -- 172.21.15.101:0/2421390141 learned_addr learned my addr 172.21.15.101:0/2421390141 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:58:16.220 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.279+0000 7f41920ba700 1 -- 172.21.15.101:0/2421390141 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f418c07a980 con 0x7f418c07be00 2023-07-18T02:58:16.220 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.279+0000 7f41920ba700 1 --2- 172.21.15.101:0/2421390141 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f418c07be00 0x7f418c07a300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f41880098a0 tx=0x7f4188009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75bd7e57006ecb2a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:16.220 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.279+0000 7f41910b8700 1 -- 172.21.15.101:0/2421390141 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4188004160 con 0x7f418c07be00 2023-07-18T02:58:16.220 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.279+0000 7f41910b8700 1 -- 172.21.15.101:0/2421390141 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41880042c0 con 0x7f418c07be00 2023-07-18T02:58:16.220 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.279+0000 7f41910b8700 1 -- 172.21.15.101:0/2421390141 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f41880119b0 con 0x7f418c07be00 2023-07-18T02:58:16.221 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.280+0000 7f419431e700 1 -- 172.21.15.101:0/2421390141 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f418c07be00 msgr2=0x7f418c07a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:16.221 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.280+0000 7f419431e700 1 --2- 172.21.15.101:0/2421390141 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f418c07be00 0x7f418c07a300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f41880098a0 tx=0x7f4188009bb0 comp rx=0 tx=0).stop 2023-07-18T02:58:16.221 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.280+0000 7f419431e700 1 -- 172.21.15.101:0/2421390141 shutdown_connections 2023-07-18T02:58:16.221 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.280+0000 7f419431e700 1 --2- 172.21.15.101:0/2421390141 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f418c07be00 0x7f418c07a300 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:16.221 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.280+0000 7f419431e700 1 -- 172.21.15.101:0/2421390141 >> 172.21.15.101:0/2421390141 conn(0x7f418c100970 msgr2=0x7f418c102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:58:16.221 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.280+0000 7f419431e700 1 -- 172.21.15.101:0/2421390141 shutdown_connections 2023-07-18T02:58:16.221 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.280+0000 7f419431e700 1 -- 172.21.15.101:0/2421390141 wait complete. 2023-07-18T02:58:16.222 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.282+0000 7f419431e700 1 Processor -- start 2023-07-18T02:58:16.222 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.282+0000 7f419431e700 1 -- start start 2023-07-18T02:58:16.222 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.282+0000 7f419431e700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f418c07be00 0x7f418c19b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:16.222 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.283+0000 7f419431e700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f418c19b810 con 0x7f418c07be00 2023-07-18T02:58:16.222 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.283+0000 7f41920ba700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f418c07be00 0x7f418c19b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:16.222 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.283+0000 7f41920ba700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f418c07be00 0x7f418c19b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:51474/0 (socket says 172.21.15.101:51474) 2023-07-18T02:58:16.222 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.283+0000 7f41920ba700 1 -- 172.21.15.101:0/270849966 learned_addr learned my addr 172.21.15.101:0/270849966 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:58:16.223 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.283+0000 7f41920ba700 1 -- 172.21.15.101:0/270849966 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4188009550 con 0x7f418c07be00 2023-07-18T02:58:16.223 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.283+0000 7f41920ba700 1 --2- 172.21.15.101:0/270849966 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f418c07be00 0x7f418c19b2d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4188003e60 tx=0x7f4188011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:16.223 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.284+0000 7f41837fe700 1 -- 172.21.15.101:0/270849966 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4188003a60 con 0x7f418c07be00 2023-07-18T02:58:16.223 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.284+0000 7f41837fe700 1 -- 172.21.15.101:0/270849966 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4188003bc0 con 0x7f418c07be00 2023-07-18T02:58:16.223 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.284+0000 7f419431e700 1 -- 172.21.15.101:0/270849966 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f418c19ba10 con 0x7f418c07be00 2023-07-18T02:58:16.223 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.284+0000 7f41837fe700 1 -- 172.21.15.101:0/270849966 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f41880196c0 con 0x7f418c07be00 2023-07-18T02:58:16.223 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.284+0000 7f419431e700 1 -- 172.21.15.101:0/270849966 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f418c19be30 con 0x7f418c07be00 2023-07-18T02:58:16.223 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.285+0000 7f41837fe700 1 -- 172.21.15.101:0/270849966 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f418801a430 con 0x7f418c07be00 2023-07-18T02:58:16.224 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.285+0000 7f419431e700 1 -- 172.21.15.101:0/270849966 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f418c061b40 con 0x7f418c07be00 2023-07-18T02:58:16.224 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.286+0000 7f41837fe700 1 --2- 172.21.15.101:0/270849966 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f4178038790 0x7f417803ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:16.224 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.286+0000 7f41837fe700 1 -- 172.21.15.101:0/270849966 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f418804ca10 con 0x7f418c07be00 2023-07-18T02:58:16.224 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.286+0000 7f41918b9700 1 -- 172.21.15.101:0/270849966 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f4178038790 msgr2=0x7f417803ac40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.101:6800/2461037266 2023-07-18T02:58:16.224 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.286+0000 7f41918b9700 1 --2- 172.21.15.101:0/270849966 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f4178038790 0x7f417803ac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-18T02:58:16.224 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.294+0000 7f41837fe700 1 -- 172.21.15.101:0/270849966 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4188010380 con 0x7f418c07be00 2023-07-18T02:58:16.224 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.487+0000 7f41918b9700 1 -- 172.21.15.101:0/270849966 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f4178038790 msgr2=0x7f417803ac40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.101:6800/2461037266 2023-07-18T02:58:16.225 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.487+0000 7f41918b9700 1 --2- 172.21.15.101:0/270849966 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f4178038790 0x7f417803ac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-18T02:58:16.225 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.565+0000 7f419431e700 1 -- 172.21.15.101:0/270849966 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f418c19e960 con 0x7f418c07be00 2023-07-18T02:58:16.225 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.566+0000 7f41837fe700 1 -- 172.21.15.101:0/270849966 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f4188010380 con 0x7f418c07be00 2023-07-18T02:58:16.225 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.568+0000 7f419431e700 1 -- 172.21.15.101:0/270849966 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f4178038790 msgr2=0x7f417803ac40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-18T02:58:16.225 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.568+0000 7f419431e700 1 --2- 172.21.15.101:0/270849966 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f4178038790 0x7f417803ac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:16.225 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.568+0000 7f419431e700 1 -- 172.21.15.101:0/270849966 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f418c07be00 msgr2=0x7f418c19b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:16.225 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.568+0000 7f419431e700 1 --2- 172.21.15.101:0/270849966 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f418c07be00 0x7f418c19b2d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4188003e60 tx=0x7f4188011a30 comp rx=0 tx=0).stop 2023-07-18T02:58:16.226 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.568+0000 7f419431e700 1 -- 172.21.15.101:0/270849966 shutdown_connections 2023-07-18T02:58:16.226 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.568+0000 7f419431e700 1 --2- 172.21.15.101:0/270849966 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f4178038790 0x7f417803ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:16.226 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.568+0000 7f419431e700 1 --2- 172.21.15.101:0/270849966 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f418c07be00 0x7f418c19b2d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:16.226 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.569+0000 7f419431e700 1 -- 172.21.15.101:0/270849966 >> 172.21.15.101:0/270849966 conn(0x7f418c100970 msgr2=0x7f418c1020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:58:16.226 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.569+0000 7f419431e700 1 -- 172.21.15.101:0/270849966 shutdown_connections 2023-07-18T02:58:16.226 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:15.569+0000 7f419431e700 1 -- 172.21.15.101:0/270849966 wait complete. 2023-07-18T02:58:16.226 INFO:teuthology.orchestra.run.smithi101.stdout:Waiting for the mgr to restart... 2023-07-18T02:58:16.226 INFO:teuthology.orchestra.run.smithi101.stdout:Waiting for mgr epoch 9... 2023-07-18T02:58:16.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:16 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/270849966' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-07-18T02:58:16.967 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:58:16 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:58:16.544+0000 7f2a833f7200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-18T02:58:17.717 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:58:17 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:58:17.297+0000 7f2a833f7200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-18T02:58:17.717 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:58:17 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:58:17.505+0000 7f2a833f7200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-18T02:58:18.717 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:58:18 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:58:18.442+0000 7f2a833f7200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-18T02:58:18.717 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:58:18 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:58:18.561+0000 7f2a833f7200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-18T02:58:19.112 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:58:18 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:58:18.781+0000 7f2a833f7200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-18T02:58:19.387 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:58:19 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:58:19.139+0000 7f2a833f7200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-18T02:58:19.717 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:58:19 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:58:19.414+0000 7f2a833f7200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-18T02:58:19.718 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:58:19 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:58:19.604+0000 7f2a833f7200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-18T02:58:20.217 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:58:19 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:58:19.945+0000 7f2a833f7200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-18T02:58:21.217 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:58:20 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:58:20.859+0000 7f2a833f7200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-18T02:58:21.217 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:58:21 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:58:21.031+0000 7f2a833f7200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-18T02:58:21.668 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:58:21 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:58:21.276+0000 7f2a833f7200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-18T02:58:21.668 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:58:21 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:58:21.422+0000 7f2a833f7200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-18T02:58:21.668 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:58:21 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:58:21.694+0000 7f2a833f7200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-18T02:58:23.217 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:58:22 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:58:22.759+0000 7f2a833f7200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-18T02:58:23.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:22 smithi101 ceph-mon[126379]: Active manager daemon x restarted 2023-07-18T02:58:23.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:22 smithi101 ceph-mon[126379]: Activating manager daemon x 2023-07-18T02:58:23.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:22 smithi101 ceph-mon[126379]: osdmap e3: 0 total, 0 up, 0 in 2023-07-18T02:58:23.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:22 smithi101 ceph-mon[126379]: mgrmap e10: x(active, starting, since 0.00618636s) 2023-07-18T02:58:23.219 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:22 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-18T02:58:23.219 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:22 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-18T02:58:23.219 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:22 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-18T02:58:23.219 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:22 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-18T02:58:23.219 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:22 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-18T02:58:23.219 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:22 smithi101 ceph-mon[126379]: Manager daemon x is now available 2023-07-18T02:58:23.219 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:22 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T02:58:23.220 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:22 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T02:58:24.182 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:23 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-18T02:58:24.182 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:23 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-18T02:58:24.183 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:23 smithi101 ceph-mon[126379]: [18/Jul/2023:02:58:23] ENGINE Bus STARTING 2023-07-18T02:58:24.183 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:23 smithi101 ceph-mon[126379]: [18/Jul/2023:02:58:23] ENGINE Serving on http://172.21.15.101:8765 2023-07-18T02:58:24.183 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:23 smithi101 ceph-mon[126379]: mgrmap e11: x(active, since 1.00778s) 2023-07-18T02:58:24.218 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout { 2023-07-18T02:58:24.219 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-07-18T02:58:24.219 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout "initialized": true 2023-07-18T02:58:24.219 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout } 2023-07-18T02:58:24.219 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:16.815+0000 7f6dc819f700 1 Processor -- start 2023-07-18T02:58:24.219 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:16.815+0000 7f6dc819f700 1 -- start start 2023-07-18T02:58:24.220 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:16.815+0000 7f6dc819f700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6dc007be40 0x7f6dc007a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:24.220 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:16.815+0000 7f6dc819f700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc007a880 con 0x7f6dc007be40 2023-07-18T02:58:24.220 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:16.816+0000 7f6dc5f3b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6dc007be40 0x7f6dc007a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:24.220 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:16.816+0000 7f6dc5f3b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6dc007be40 0x7f6dc007a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:56968/0 (socket says 172.21.15.101:56968) 2023-07-18T02:58:24.220 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:16.816+0000 7f6dc5f3b700 1 -- 172.21.15.101:0/1620414564 learned_addr learned my addr 172.21.15.101:0/1620414564 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:58:24.221 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:16.816+0000 7f6dc5f3b700 1 -- 172.21.15.101:0/1620414564 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc007a9c0 con 0x7f6dc007be40 2023-07-18T02:58:24.221 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:16.816+0000 7f6dc5f3b700 1 --2- 172.21.15.101:0/1620414564 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6dc007be40 0x7f6dc007a340 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc0098a0 tx=0x7f6dbc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=979cfeb1a9b91c84 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:24.221 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:16.816+0000 7f6dc4f39700 1 -- 172.21.15.101:0/1620414564 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6dbc004160 con 0x7f6dc007be40 2023-07-18T02:58:24.221 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:16.817+0000 7f6dc4f39700 1 -- 172.21.15.101:0/1620414564 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6dbc0042c0 con 0x7f6dc007be40 2023-07-18T02:58:24.221 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:16.817+0000 7f6dc4f39700 1 -- 172.21.15.101:0/1620414564 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6dbc0119b0 con 0x7f6dc007be40 2023-07-18T02:58:24.221 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:16.817+0000 7f6dc819f700 1 -- 172.21.15.101:0/1620414564 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6dc007be40 msgr2=0x7f6dc007a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:24.222 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:16.817+0000 7f6dc819f700 1 --2- 172.21.15.101:0/1620414564 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6dc007be40 0x7f6dc007a340 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc0098a0 tx=0x7f6dbc009bb0 comp rx=0 tx=0).stop 2023-07-18T02:58:24.222 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:16.817+0000 7f6dc819f700 1 -- 172.21.15.101:0/1620414564 shutdown_connections 2023-07-18T02:58:24.222 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:16.817+0000 7f6dc819f700 1 --2- 172.21.15.101:0/1620414564 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6dc007be40 0x7f6dc007a340 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:24.222 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:16.817+0000 7f6dc819f700 1 -- 172.21.15.101:0/1620414564 >> 172.21.15.101:0/1620414564 conn(0x7f6dc0100970 msgr2=0x7f6dc0102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:58:24.222 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:16.817+0000 7f6dc819f700 1 -- 172.21.15.101:0/1620414564 shutdown_connections 2023-07-18T02:58:24.222 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:16.817+0000 7f6dc819f700 1 -- 172.21.15.101:0/1620414564 wait complete. 2023-07-18T02:58:24.222 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:16.818+0000 7f6dc819f700 1 Processor -- start 2023-07-18T02:58:24.223 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:16.818+0000 7f6dc819f700 1 -- start start 2023-07-18T02:58:24.223 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:16.819+0000 7f6dc819f700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6dc007be40 0x7f6dc019b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:24.223 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:16.819+0000 7f6dc819f700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc019b870 con 0x7f6dc007be40 2023-07-18T02:58:24.223 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:16.819+0000 7f6dc5f3b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6dc007be40 0x7f6dc019b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:24.223 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:16.819+0000 7f6dc5f3b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6dc007be40 0x7f6dc019b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:56982/0 (socket says 172.21.15.101:56982) 2023-07-18T02:58:24.223 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:16.819+0000 7f6dc5f3b700 1 -- 172.21.15.101:0/401820731 learned_addr learned my addr 172.21.15.101:0/401820731 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:58:24.223 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:16.819+0000 7f6dc5f3b700 1 -- 172.21.15.101:0/401820731 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dbc009550 con 0x7f6dc007be40 2023-07-18T02:58:24.224 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:16.820+0000 7f6dc5f3b700 1 --2- 172.21.15.101:0/401820731 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6dc007be40 0x7f6dc019b330 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc006af0 tx=0x7f6dbc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:24.224 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:16.820+0000 7f6db6ffd700 1 -- 172.21.15.101:0/401820731 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6dbc003a80 con 0x7f6dc007be40 2023-07-18T02:58:24.224 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:16.820+0000 7f6db6ffd700 1 -- 172.21.15.101:0/401820731 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6dbc003be0 con 0x7f6dc007be40 2023-07-18T02:58:24.224 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:16.820+0000 7f6dc819f700 1 -- 172.21.15.101:0/401820731 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dc019ba70 con 0x7f6dc007be40 2023-07-18T02:58:24.224 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:16.820+0000 7f6db6ffd700 1 -- 172.21.15.101:0/401820731 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6dbc01a760 con 0x7f6dc007be40 2023-07-18T02:58:24.224 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:16.820+0000 7f6dc819f700 1 -- 172.21.15.101:0/401820731 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dc019be90 con 0x7f6dc007be40 2023-07-18T02:58:24.225 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:16.821+0000 7f6db6ffd700 1 -- 172.21.15.101:0/401820731 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f6dbc024430 con 0x7f6dc007be40 2023-07-18T02:58:24.225 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:16.822+0000 7f6db6ffd700 1 --2- 172.21.15.101:0/401820731 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f6dac038760 0x7f6dac03ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:24.225 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:16.822+0000 7f6db6ffd700 1 -- 172.21.15.101:0/401820731 --> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6dac03b320 con 0x7f6dac038760 2023-07-18T02:58:24.225 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:16.822+0000 7f6db6ffd700 1 -- 172.21.15.101:0/401820731 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f6dbc04c8b0 con 0x7f6dc007be40 2023-07-18T02:58:24.225 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:16.822+0000 7f6dc573a700 1 -- 172.21.15.101:0/401820731 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f6dac038760 msgr2=0x7f6dac03ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.101:6800/2461037266 2023-07-18T02:58:24.225 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:16.822+0000 7f6dc573a700 1 --2- 172.21.15.101:0/401820731 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f6dac038760 0x7f6dac03ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-18T02:58:24.225 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:17.023+0000 7f6dc573a700 1 -- 172.21.15.101:0/401820731 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f6dac038760 msgr2=0x7f6dac03ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.101:6800/2461037266 2023-07-18T02:58:24.225 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:17.023+0000 7f6dc573a700 1 --2- 172.21.15.101:0/401820731 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f6dac038760 0x7f6dac03ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-18T02:58:24.226 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:17.424+0000 7f6dc573a700 1 -- 172.21.15.101:0/401820731 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f6dac038760 msgr2=0x7f6dac03ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.101:6800/2461037266 2023-07-18T02:58:24.226 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:17.424+0000 7f6dc573a700 1 --2- 172.21.15.101:0/401820731 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f6dac038760 0x7f6dac03ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-07-18T02:58:24.226 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:18.225+0000 7f6dc573a700 1 -- 172.21.15.101:0/401820731 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f6dac038760 msgr2=0x7f6dac03ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.101:6800/2461037266 2023-07-18T02:58:24.226 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:18.225+0000 7f6dc573a700 1 --2- 172.21.15.101:0/401820731 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f6dac038760 0x7f6dac03ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-07-18T02:58:24.226 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:19.826+0000 7f6dc573a700 1 -- 172.21.15.101:0/401820731 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f6dac038760 msgr2=0x7f6dac03ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.101:6800/2461037266 2023-07-18T02:58:24.226 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:19.826+0000 7f6dc573a700 1 --2- 172.21.15.101:0/401820731 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f6dac038760 0x7f6dac03ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-07-18T02:58:24.227 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:22.766+0000 7f6db6ffd700 1 -- 172.21.15.101:0/401820731 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mgrmap(e 10) v1 ==== 45263+0+0 (secure 0 0 0) 0x7f6dbc024b80 con 0x7f6dc007be40 2023-07-18T02:58:24.227 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:22.766+0000 7f6db6ffd700 1 -- 172.21.15.101:0/401820731 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f6dac038760 msgr2=0x7f6dac03ac10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-18T02:58:24.227 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:22.766+0000 7f6db6ffd700 1 --2- 172.21.15.101:0/401820731 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f6dac038760 0x7f6dac03ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:24.228 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:23.766+0000 7f6db6ffd700 1 -- 172.21.15.101:0/401820731 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mgrmap(e 11) v1 ==== 45390+0+0 (secure 0 0 0) 0x7f6dbc04e270 con 0x7f6dc007be40 2023-07-18T02:58:24.228 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:23.766+0000 7f6db6ffd700 1 --2- 172.21.15.101:0/401820731 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f6dac03bff0 0x7f6dac03e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:24.228 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:23.766+0000 7f6db6ffd700 1 -- 172.21.15.101:0/401820731 --> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6dac03b320 con 0x7f6dac03bff0 2023-07-18T02:58:24.228 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:23.767+0000 7f6dc573a700 1 --2- 172.21.15.101:0/401820731 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f6dac03bff0 0x7f6dac03e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:24.228 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:23.768+0000 7f6dc573a700 1 --2- 172.21.15.101:0/401820731 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f6dac03bff0 0x7f6dac03e3d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f6db0003a10 tx=0x7f6db0009270 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:24.228 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:23.769+0000 7f6db6ffd700 1 -- 172.21.15.101:0/401820731 <== mgr.14150 v2:172.21.15.101:6800/3963742842 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f6dac03b320 con 0x7f6dac03bff0 2023-07-18T02:58:24.229 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:23.780+0000 7f6dc819f700 1 -- 172.21.15.101:0/401820731 --> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f6dc019c670 con 0x7f6dac03bff0 2023-07-18T02:58:24.229 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:23.780+0000 7f6db6ffd700 1 -- 172.21.15.101:0/401820731 <== mgr.14150 v2:172.21.15.101:6800/3963742842 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f6dc019c670 con 0x7f6dac03bff0 2023-07-18T02:58:24.229 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:23.780+0000 7f6dc819f700 1 -- 172.21.15.101:0/401820731 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f6dac03bff0 msgr2=0x7f6dac03e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:24.229 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:23.780+0000 7f6dc819f700 1 --2- 172.21.15.101:0/401820731 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f6dac03bff0 0x7f6dac03e3d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f6db0003a10 tx=0x7f6db0009270 comp rx=0 tx=0).stop 2023-07-18T02:58:24.229 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:23.780+0000 7f6dc819f700 1 -- 172.21.15.101:0/401820731 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6dc007be40 msgr2=0x7f6dc019b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:24.229 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:23.780+0000 7f6dc819f700 1 --2- 172.21.15.101:0/401820731 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6dc007be40 0x7f6dc019b330 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc006af0 tx=0x7f6dbc011aa0 comp rx=0 tx=0).stop 2023-07-18T02:58:24.229 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:23.781+0000 7f6dc819f700 1 -- 172.21.15.101:0/401820731 shutdown_connections 2023-07-18T02:58:24.230 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:23.781+0000 7f6dc819f700 1 --2- 172.21.15.101:0/401820731 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f6dac03bff0 0x7f6dac03e3d0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:24.230 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:23.781+0000 7f6dc819f700 1 --2- 172.21.15.101:0/401820731 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6dc007be40 0x7f6dc019b330 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:24.230 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:23.781+0000 7f6dc819f700 1 --2- 172.21.15.101:0/401820731 >> [v2:172.21.15.101:6800/2461037266,v1:172.21.15.101:6801/2461037266] conn(0x7f6dac038760 0x7f6dac03ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:24.231 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:23.781+0000 7f6dc819f700 1 -- 172.21.15.101:0/401820731 >> 172.21.15.101:0/401820731 conn(0x7f6dc0100970 msgr2=0x7f6dc0102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:58:24.231 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:23.781+0000 7f6dc819f700 1 -- 172.21.15.101:0/401820731 shutdown_connections 2023-07-18T02:58:24.231 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:23.781+0000 7f6dc819f700 1 -- 172.21.15.101:0/401820731 wait complete. 2023-07-18T02:58:24.231 INFO:teuthology.orchestra.run.smithi101.stdout:mgr epoch 9 is available 2023-07-18T02:58:24.231 INFO:teuthology.orchestra.run.smithi101.stdout:Generating a dashboard self-signed certificate... 2023-07-18T02:58:25.217 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:24 smithi101 ceph-mon[126379]: [18/Jul/2023:02:58:23] ENGINE Serving on https://172.21.15.101:7150 2023-07-18T02:58:25.217 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:24 smithi101 ceph-mon[126379]: [18/Jul/2023:02:58:23] ENGINE Bus STARTED 2023-07-18T02:58:25.217 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:24 smithi101 ceph-mon[126379]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-07-18T02:58:25.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:24 smithi101 ceph-mon[126379]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-07-18T02:58:25.805 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-07-18T02:58:25.805 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.099+0000 7f99ec633700 1 Processor -- start 2023-07-18T02:58:25.805 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.099+0000 7f99ec633700 1 -- start start 2023-07-18T02:58:25.806 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.100+0000 7f99ec633700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f99e407be00 0x7f99e407a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:25.806 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.100+0000 7f99ec633700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e407a840 con 0x7f99e407be00 2023-07-18T02:58:25.806 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.100+0000 7f99ea3cf700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f99e407be00 0x7f99e407a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:25.806 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.100+0000 7f99ea3cf700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f99e407be00 0x7f99e407a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:57066/0 (socket says 172.21.15.101:57066) 2023-07-18T02:58:25.806 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.100+0000 7f99ea3cf700 1 -- 172.21.15.101:0/3988615918 learned_addr learned my addr 172.21.15.101:0/3988615918 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:58:25.806 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.101+0000 7f99ea3cf700 1 -- 172.21.15.101:0/3988615918 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e407a980 con 0x7f99e407be00 2023-07-18T02:58:25.806 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.101+0000 7f99ea3cf700 1 --2- 172.21.15.101:0/3988615918 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f99e407be00 0x7f99e407a300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f99e00098a0 tx=0x7f99e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95426beca26f9c82 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:25.807 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.101+0000 7f99e93cd700 1 -- 172.21.15.101:0/3988615918 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f99e0004160 con 0x7f99e407be00 2023-07-18T02:58:25.807 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.101+0000 7f99e93cd700 1 -- 172.21.15.101:0/3988615918 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99e00042c0 con 0x7f99e407be00 2023-07-18T02:58:25.807 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.101+0000 7f99e93cd700 1 -- 172.21.15.101:0/3988615918 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f99e00119b0 con 0x7f99e407be00 2023-07-18T02:58:25.807 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.102+0000 7f99ec633700 1 -- 172.21.15.101:0/3988615918 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f99e407be00 msgr2=0x7f99e407a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:25.807 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.102+0000 7f99ec633700 1 --2- 172.21.15.101:0/3988615918 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f99e407be00 0x7f99e407a300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f99e00098a0 tx=0x7f99e0009bb0 comp rx=0 tx=0).stop 2023-07-18T02:58:25.807 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.102+0000 7f99ec633700 1 -- 172.21.15.101:0/3988615918 shutdown_connections 2023-07-18T02:58:25.807 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.102+0000 7f99ec633700 1 --2- 172.21.15.101:0/3988615918 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f99e407be00 0x7f99e407a300 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:25.807 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.102+0000 7f99ec633700 1 -- 172.21.15.101:0/3988615918 >> 172.21.15.101:0/3988615918 conn(0x7f99e4100970 msgr2=0x7f99e4102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:58:25.808 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.102+0000 7f99ec633700 1 -- 172.21.15.101:0/3988615918 shutdown_connections 2023-07-18T02:58:25.808 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.102+0000 7f99ec633700 1 -- 172.21.15.101:0/3988615918 wait complete. 2023-07-18T02:58:25.808 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.104+0000 7f99ec633700 1 Processor -- start 2023-07-18T02:58:25.808 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.104+0000 7f99ec633700 1 -- start start 2023-07-18T02:58:25.808 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.104+0000 7f99ec633700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f99e407be00 0x7f99e419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:25.808 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.104+0000 7f99ec633700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e419b880 con 0x7f99e407be00 2023-07-18T02:58:25.808 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.104+0000 7f99ea3cf700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f99e407be00 0x7f99e419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:25.809 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.105+0000 7f99ea3cf700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f99e407be00 0x7f99e419b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:57080/0 (socket says 172.21.15.101:57080) 2023-07-18T02:58:25.809 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.105+0000 7f99ea3cf700 1 -- 172.21.15.101:0/3839240465 learned_addr learned my addr 172.21.15.101:0/3839240465 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:58:25.809 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.105+0000 7f99ea3cf700 1 -- 172.21.15.101:0/3839240465 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e0009550 con 0x7f99e407be00 2023-07-18T02:58:25.809 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.105+0000 7f99ea3cf700 1 --2- 172.21.15.101:0/3839240465 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f99e407be00 0x7f99e419b340 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f99e0003e60 tx=0x7f99e0011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:25.809 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.105+0000 7f99db7fe700 1 -- 172.21.15.101:0/3839240465 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f99e0003a60 con 0x7f99e407be00 2023-07-18T02:58:25.809 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.105+0000 7f99db7fe700 1 -- 172.21.15.101:0/3839240465 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99e0003bc0 con 0x7f99e407be00 2023-07-18T02:58:25.809 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.105+0000 7f99ec633700 1 -- 172.21.15.101:0/3839240465 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99e419ba80 con 0x7f99e407be00 2023-07-18T02:58:25.809 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.105+0000 7f99db7fe700 1 -- 172.21.15.101:0/3839240465 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f99e00196c0 con 0x7f99e407be00 2023-07-18T02:58:25.810 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.106+0000 7f99ec633700 1 -- 172.21.15.101:0/3839240465 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99e419bea0 con 0x7f99e407be00 2023-07-18T02:58:25.810 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.107+0000 7f99ec633700 1 -- 172.21.15.101:0/3839240465 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99e410b6d0 con 0x7f99e407be00 2023-07-18T02:58:25.810 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.107+0000 7f99db7fe700 1 -- 172.21.15.101:0/3839240465 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f99e001a430 con 0x7f99e407be00 2023-07-18T02:58:25.810 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.107+0000 7f99db7fe700 1 --2- 172.21.15.101:0/3839240465 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f99d0038740 0x7f99d003abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:25.810 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.107+0000 7f99db7fe700 1 -- 172.21.15.101:0/3839240465 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f99e004ca60 con 0x7f99e407be00 2023-07-18T02:58:25.810 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.108+0000 7f99e9bce700 1 --2- 172.21.15.101:0/3839240465 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f99d0038740 0x7f99d003abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:25.810 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.108+0000 7f99e9bce700 1 --2- 172.21.15.101:0/3839240465 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f99d0038740 0x7f99d003abf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f99d4006fd0 tx=0x7f99d4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:25.811 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.112+0000 7f99db7fe700 1 -- 172.21.15.101:0/3839240465 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f99e0010560 con 0x7f99e407be00 2023-07-18T02:58:25.811 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.330+0000 7f99ec633700 1 -- 172.21.15.101:0/3839240465 --> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f99e4102ce0 con 0x7f99d0038740 2023-07-18T02:58:25.811 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.387+0000 7f99db7fe700 1 -- 172.21.15.101:0/3839240465 <== mgr.14150 v2:172.21.15.101:6800/3963742842 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f99e4102ce0 con 0x7f99d0038740 2023-07-18T02:58:25.811 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.389+0000 7f99ec633700 1 -- 172.21.15.101:0/3839240465 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f99d0038740 msgr2=0x7f99d003abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:25.811 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.389+0000 7f99ec633700 1 --2- 172.21.15.101:0/3839240465 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f99d0038740 0x7f99d003abf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f99d4006fd0 tx=0x7f99d4006e40 comp rx=0 tx=0).stop 2023-07-18T02:58:25.811 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.389+0000 7f99ec633700 1 -- 172.21.15.101:0/3839240465 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f99e407be00 msgr2=0x7f99e419b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:25.811 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.389+0000 7f99ec633700 1 --2- 172.21.15.101:0/3839240465 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f99e407be00 0x7f99e419b340 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f99e0003e60 tx=0x7f99e0011a30 comp rx=0 tx=0).stop 2023-07-18T02:58:25.811 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.389+0000 7f99ec633700 1 -- 172.21.15.101:0/3839240465 shutdown_connections 2023-07-18T02:58:25.812 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.389+0000 7f99ec633700 1 --2- 172.21.15.101:0/3839240465 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f99d0038740 0x7f99d003abf0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:25.812 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.389+0000 7f99ec633700 1 --2- 172.21.15.101:0/3839240465 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f99e407be00 0x7f99e419b340 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:25.812 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.389+0000 7f99ec633700 1 -- 172.21.15.101:0/3839240465 >> 172.21.15.101:0/3839240465 conn(0x7f99e4100970 msgr2=0x7f99e41020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:58:25.812 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.389+0000 7f99ec633700 1 -- 172.21.15.101:0/3839240465 shutdown_connections 2023-07-18T02:58:25.812 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:25.389+0000 7f99ec633700 1 -- 172.21.15.101:0/3839240465 wait complete. 2023-07-18T02:58:25.812 INFO:teuthology.orchestra.run.smithi101.stdout:Creating initial admin user... 2023-07-18T02:58:26.065 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:25 smithi101 ceph-mon[126379]: mgrmap e12: x(active, since 2s) 2023-07-18T02:58:26.065 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:25 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:58:26.065 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:25 smithi101 ceph-mon[126379]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T02:58:26.065 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:25 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:58:26.066 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:25 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:58:27.391 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$AZOYukPkaS3QMqbxYL9C3O50kOxvt1k0sytpOg7Wh.jl2tGtsswGC", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1689649106, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-07-18T02:58:27.391 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.367+0000 7f18075f5700 1 Processor -- start 2023-07-18T02:58:27.391 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.367+0000 7f18075f5700 1 -- start start 2023-07-18T02:58:27.392 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.367+0000 7f18075f5700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f180010cf50 0x7f180010d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:27.392 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.367+0000 7f18075f5700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180010d8a0 con 0x7f180010cf50 2023-07-18T02:58:27.392 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.368+0000 7f1805391700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f180010cf50 0x7f180010d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:27.392 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.368+0000 7f1805391700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f180010cf50 0x7f180010d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:58610/0 (socket says 172.21.15.101:58610) 2023-07-18T02:58:27.392 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.368+0000 7f1805391700 1 -- 172.21.15.101:0/2213790539 learned_addr learned my addr 172.21.15.101:0/2213790539 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:58:27.392 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.368+0000 7f1805391700 1 -- 172.21.15.101:0/2213790539 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f180010d9e0 con 0x7f180010cf50 2023-07-18T02:58:27.392 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.369+0000 7f1805391700 1 --2- 172.21.15.101:0/2213790539 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f180010cf50 0x7f180010d360 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f17fc0098a0 tx=0x7f17fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea24cf13049e56cd server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:27.393 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.370+0000 7f17f7fff700 1 -- 172.21.15.101:0/2213790539 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f17fc004160 con 0x7f180010cf50 2023-07-18T02:58:27.393 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.370+0000 7f17f7fff700 1 -- 172.21.15.101:0/2213790539 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17fc0042c0 con 0x7f180010cf50 2023-07-18T02:58:27.393 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.370+0000 7f17f7fff700 1 -- 172.21.15.101:0/2213790539 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f17fc0119b0 con 0x7f180010cf50 2023-07-18T02:58:27.393 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.371+0000 7f18075f5700 1 -- 172.21.15.101:0/2213790539 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f180010cf50 msgr2=0x7f180010d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:27.393 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.371+0000 7f18075f5700 1 --2- 172.21.15.101:0/2213790539 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f180010cf50 0x7f180010d360 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f17fc0098a0 tx=0x7f17fc009bb0 comp rx=0 tx=0).stop 2023-07-18T02:58:27.393 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.371+0000 7f18075f5700 1 -- 172.21.15.101:0/2213790539 shutdown_connections 2023-07-18T02:58:27.393 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.371+0000 7f18075f5700 1 --2- 172.21.15.101:0/2213790539 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f180010cf50 0x7f180010d360 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:27.393 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.371+0000 7f18075f5700 1 -- 172.21.15.101:0/2213790539 >> 172.21.15.101:0/2213790539 conn(0x7f180007a9c0 msgr2=0x7f180007cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:58:27.394 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.371+0000 7f18075f5700 1 -- 172.21.15.101:0/2213790539 shutdown_connections 2023-07-18T02:58:27.394 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.371+0000 7f18075f5700 1 -- 172.21.15.101:0/2213790539 wait complete. 2023-07-18T02:58:27.394 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.372+0000 7f18075f5700 1 Processor -- start 2023-07-18T02:58:27.394 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.372+0000 7f18075f5700 1 -- start start 2023-07-18T02:58:27.394 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.372+0000 7f18075f5700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f180010cf50 0x7f180019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:27.394 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.372+0000 7f18075f5700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180019fc10 con 0x7f180010cf50 2023-07-18T02:58:27.394 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.373+0000 7f1805391700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f180010cf50 0x7f180019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:27.395 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.373+0000 7f1805391700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f180010cf50 0x7f180019f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:58624/0 (socket says 172.21.15.101:58624) 2023-07-18T02:58:27.395 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.373+0000 7f1805391700 1 -- 172.21.15.101:0/4256743447 learned_addr learned my addr 172.21.15.101:0/4256743447 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:58:27.395 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.373+0000 7f1805391700 1 -- 172.21.15.101:0/4256743447 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17fc009550 con 0x7f180010cf50 2023-07-18T02:58:27.395 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.373+0000 7f1805391700 1 --2- 172.21.15.101:0/4256743447 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f180010cf50 0x7f180019f6d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f17fc006af0 tx=0x7f17fc011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:27.395 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.374+0000 7f17f67fc700 1 -- 172.21.15.101:0/4256743447 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f17fc003a40 con 0x7f180010cf50 2023-07-18T02:58:27.395 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.374+0000 7f17f67fc700 1 -- 172.21.15.101:0/4256743447 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17fc003ba0 con 0x7f180010cf50 2023-07-18T02:58:27.395 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.374+0000 7f17f67fc700 1 -- 172.21.15.101:0/4256743447 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f17fc0196c0 con 0x7f180010cf50 2023-07-18T02:58:27.396 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.374+0000 7f18075f5700 1 -- 172.21.15.101:0/4256743447 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f180019fe10 con 0x7f180010cf50 2023-07-18T02:58:27.396 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.374+0000 7f18075f5700 1 -- 172.21.15.101:0/4256743447 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18001a0230 con 0x7f180010cf50 2023-07-18T02:58:27.396 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.375+0000 7f18075f5700 1 -- 172.21.15.101:0/4256743447 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1800199710 con 0x7f180010cf50 2023-07-18T02:58:27.396 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.375+0000 7f17f67fc700 1 -- 172.21.15.101:0/4256743447 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f17fc01a430 con 0x7f180010cf50 2023-07-18T02:58:27.396 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.376+0000 7f17f67fc700 1 --2- 172.21.15.101:0/4256743447 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f17ec038760 0x7f17ec03ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:27.396 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.376+0000 7f17f67fc700 1 -- 172.21.15.101:0/4256743447 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f17fc04cb30 con 0x7f180010cf50 2023-07-18T02:58:27.396 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.376+0000 7f1804b90700 1 --2- 172.21.15.101:0/4256743447 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f17ec038760 0x7f17ec03ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:27.397 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.376+0000 7f1804b90700 1 --2- 172.21.15.101:0/4256743447 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f17ec038760 0x7f17ec03ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f17f0006fd0 tx=0x7f17f0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:27.397 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.380+0000 7f17f67fc700 1 -- 172.21.15.101:0/4256743447 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f17fc01ac60 con 0x7f180010cf50 2023-07-18T02:58:27.397 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.603+0000 7f18075f5700 1 -- 172.21.15.101:0/4256743447 --> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f1800061960 con 0x7f17ec038760 2023-07-18T02:58:27.397 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.829+0000 7f17f67fc700 1 -- 172.21.15.101:0/4256743447 <== mgr.14150 v2:172.21.15.101:6800/3963742842 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f1800061960 con 0x7f17ec038760 2023-07-18T02:58:27.397 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.832+0000 7f18075f5700 1 -- 172.21.15.101:0/4256743447 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f17ec038760 msgr2=0x7f17ec03ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:27.397 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.832+0000 7f18075f5700 1 --2- 172.21.15.101:0/4256743447 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f17ec038760 0x7f17ec03ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f17f0006fd0 tx=0x7f17f0006e40 comp rx=0 tx=0).stop 2023-07-18T02:58:27.397 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.832+0000 7f18075f5700 1 -- 172.21.15.101:0/4256743447 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f180010cf50 msgr2=0x7f180019f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:27.398 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.832+0000 7f18075f5700 1 --2- 172.21.15.101:0/4256743447 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f180010cf50 0x7f180019f6d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f17fc006af0 tx=0x7f17fc011a10 comp rx=0 tx=0).stop 2023-07-18T02:58:27.398 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.832+0000 7f18075f5700 1 -- 172.21.15.101:0/4256743447 shutdown_connections 2023-07-18T02:58:27.398 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.832+0000 7f18075f5700 1 --2- 172.21.15.101:0/4256743447 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f17ec038760 0x7f17ec03ac10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:27.398 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.832+0000 7f18075f5700 1 --2- 172.21.15.101:0/4256743447 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f180010cf50 0x7f180019f6d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:27.398 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.832+0000 7f18075f5700 1 -- 172.21.15.101:0/4256743447 >> 172.21.15.101:0/4256743447 conn(0x7f180007a9c0 msgr2=0x7f180007c130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:58:27.398 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.832+0000 7f18075f5700 1 -- 172.21.15.101:0/4256743447 shutdown_connections 2023-07-18T02:58:27.398 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:26.832+0000 7f18075f5700 1 -- 172.21.15.101:0/4256743447 wait complete. 2023-07-18T02:58:27.399 INFO:teuthology.orchestra.run.smithi101.stdout:Fetching dashboard port number... 2023-07-18T02:58:27.697 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:27 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:58:27.697 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:27 smithi101 ceph-mon[126379]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T02:58:27.697 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:27 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:58:28.770 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stdout 8443 2023-07-18T02:58:28.771 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.020+0000 7f127f4a6700 1 Processor -- start 2023-07-18T02:58:28.771 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.020+0000 7f127f4a6700 1 -- start start 2023-07-18T02:58:28.771 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.021+0000 7f127f4a6700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1278105320 0x7f1278105730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:28.771 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.021+0000 7f127f4a6700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1278105c70 con 0x7f1278105320 2023-07-18T02:58:28.771 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.021+0000 7f127d242700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1278105320 0x7f1278105730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:28.771 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.021+0000 7f127d242700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1278105320 0x7f1278105730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:58632/0 (socket says 172.21.15.101:58632) 2023-07-18T02:58:28.772 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.021+0000 7f127d242700 1 -- 172.21.15.101:0/2704166293 learned_addr learned my addr 172.21.15.101:0/2704166293 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:58:28.772 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.021+0000 7f127d242700 1 -- 172.21.15.101:0/2704166293 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1278105db0 con 0x7f1278105320 2023-07-18T02:58:28.772 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.022+0000 7f127d242700 1 --2- 172.21.15.101:0/2704166293 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1278105320 0x7f1278105730 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f12740098a0 tx=0x7f1274009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d02388625919a33 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:28.772 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.022+0000 7f126ffff700 1 -- 172.21.15.101:0/2704166293 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1274004160 con 0x7f1278105320 2023-07-18T02:58:28.772 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.022+0000 7f126ffff700 1 -- 172.21.15.101:0/2704166293 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12740042c0 con 0x7f1278105320 2023-07-18T02:58:28.772 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.022+0000 7f126ffff700 1 -- 172.21.15.101:0/2704166293 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f12740119b0 con 0x7f1278105320 2023-07-18T02:58:28.773 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.023+0000 7f127f4a6700 1 -- 172.21.15.101:0/2704166293 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1278105320 msgr2=0x7f1278105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:28.773 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.023+0000 7f127f4a6700 1 --2- 172.21.15.101:0/2704166293 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1278105320 0x7f1278105730 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f12740098a0 tx=0x7f1274009bb0 comp rx=0 tx=0).stop 2023-07-18T02:58:28.773 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.023+0000 7f127f4a6700 1 -- 172.21.15.101:0/2704166293 shutdown_connections 2023-07-18T02:58:28.773 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.023+0000 7f127f4a6700 1 --2- 172.21.15.101:0/2704166293 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1278105320 0x7f1278105730 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:28.773 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.023+0000 7f127f4a6700 1 -- 172.21.15.101:0/2704166293 >> 172.21.15.101:0/2704166293 conn(0x7f1278100970 msgr2=0x7f1278102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:58:28.773 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.023+0000 7f127f4a6700 1 -- 172.21.15.101:0/2704166293 shutdown_connections 2023-07-18T02:58:28.773 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.023+0000 7f127f4a6700 1 -- 172.21.15.101:0/2704166293 wait complete. 2023-07-18T02:58:28.774 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.024+0000 7f127f4a6700 1 Processor -- start 2023-07-18T02:58:28.774 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.024+0000 7f127f4a6700 1 -- start start 2023-07-18T02:58:28.774 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.025+0000 7f127f4a6700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1278105320 0x7f127807bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:28.774 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.025+0000 7f127f4a6700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f127807c1b0 con 0x7f1278105320 2023-07-18T02:58:28.774 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.025+0000 7f127d242700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1278105320 0x7f127807bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:28.774 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.025+0000 7f127d242700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1278105320 0x7f127807bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:58644/0 (socket says 172.21.15.101:58644) 2023-07-18T02:58:28.775 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.025+0000 7f127d242700 1 -- 172.21.15.101:0/1245546926 learned_addr learned my addr 172.21.15.101:0/1245546926 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:58:28.775 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.025+0000 7f127d242700 1 -- 172.21.15.101:0/1245546926 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1274009550 con 0x7f1278105320 2023-07-18T02:58:28.775 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.026+0000 7f127d242700 1 --2- 172.21.15.101:0/1245546926 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1278105320 0x7f127807bc70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1274003f00 tx=0x7f1274011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:28.775 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.026+0000 7f126e7fc700 1 -- 172.21.15.101:0/1245546926 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1274003b20 con 0x7f1278105320 2023-07-18T02:58:28.775 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.026+0000 7f127f4a6700 1 -- 172.21.15.101:0/1245546926 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f127807a2c0 con 0x7f1278105320 2023-07-18T02:58:28.775 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.026+0000 7f126e7fc700 1 -- 172.21.15.101:0/1245546926 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1274003c80 con 0x7f1278105320 2023-07-18T02:58:28.775 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.026+0000 7f126e7fc700 1 -- 172.21.15.101:0/1245546926 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1274019770 con 0x7f1278105320 2023-07-18T02:58:28.776 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.026+0000 7f127f4a6700 1 -- 172.21.15.101:0/1245546926 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f127807a6e0 con 0x7f1278105320 2023-07-18T02:58:28.776 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.027+0000 7f126e7fc700 1 -- 172.21.15.101:0/1245546926 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f1274019a00 con 0x7f1278105320 2023-07-18T02:58:28.776 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.027+0000 7f127f4a6700 1 -- 172.21.15.101:0/1245546926 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12781097b0 con 0x7f1278105320 2023-07-18T02:58:28.776 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.028+0000 7f126e7fc700 1 --2- 172.21.15.101:0/1245546926 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f12640387b0 0x7f126403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:28.776 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.028+0000 7f126e7fc700 1 -- 172.21.15.101:0/1245546926 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f127404cc90 con 0x7f1278105320 2023-07-18T02:58:28.776 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.029+0000 7f127ca41700 1 --2- 172.21.15.101:0/1245546926 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f12640387b0 0x7f126403ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:28.777 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.029+0000 7f127ca41700 1 --2- 172.21.15.101:0/1245546926 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f12640387b0 0x7f126403ac60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1268006fd0 tx=0x7f1268006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:28.777 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.033+0000 7f126e7fc700 1 -- 172.21.15.101:0/1245546926 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1274018b80 con 0x7f1278105320 2023-07-18T02:58:28.777 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.247+0000 7f127f4a6700 1 -- 172.21.15.101:0/1245546926 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f127807ae30 con 0x7f1278105320 2023-07-18T02:58:28.777 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.248+0000 7f126e7fc700 1 -- 172.21.15.101:0/1245546926 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f1274023e60 con 0x7f1278105320 2023-07-18T02:58:28.777 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.250+0000 7f127f4a6700 1 -- 172.21.15.101:0/1245546926 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f12640387b0 msgr2=0x7f126403ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:28.777 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.250+0000 7f127f4a6700 1 --2- 172.21.15.101:0/1245546926 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f12640387b0 0x7f126403ac60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1268006fd0 tx=0x7f1268006e40 comp rx=0 tx=0).stop 2023-07-18T02:58:28.778 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.250+0000 7f127f4a6700 1 -- 172.21.15.101:0/1245546926 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1278105320 msgr2=0x7f127807bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:28.778 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.250+0000 7f127f4a6700 1 --2- 172.21.15.101:0/1245546926 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1278105320 0x7f127807bc70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1274003f00 tx=0x7f1274011a30 comp rx=0 tx=0).stop 2023-07-18T02:58:28.778 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.250+0000 7f127f4a6700 1 -- 172.21.15.101:0/1245546926 shutdown_connections 2023-07-18T02:58:28.778 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.250+0000 7f127f4a6700 1 --2- 172.21.15.101:0/1245546926 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f12640387b0 0x7f126403ac60 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:28.778 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.250+0000 7f127f4a6700 1 --2- 172.21.15.101:0/1245546926 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1278105320 0x7f127807bc70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:28.778 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.250+0000 7f127f4a6700 1 -- 172.21.15.101:0/1245546926 >> 172.21.15.101:0/1245546926 conn(0x7f1278100970 msgr2=0x7f12781020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:58:28.778 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.250+0000 7f127f4a6700 1 -- 172.21.15.101:0/1245546926 shutdown_connections 2023-07-18T02:58:28.779 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:28.250+0000 7f127f4a6700 1 -- 172.21.15.101:0/1245546926 wait complete. 2023-07-18T02:58:28.789 INFO:teuthology.orchestra.run.smithi101.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-18T02:58:28.789 INFO:teuthology.orchestra.run.smithi101.stdout:systemctl: stdout disabled 2023-07-18T02:58:28.808 INFO:teuthology.orchestra.run.smithi101.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-18T02:58:28.809 INFO:teuthology.orchestra.run.smithi101.stdout:systemctl: stdout inactive 2023-07-18T02:58:28.809 INFO:teuthology.orchestra.run.smithi101.stdout:firewalld.service is not enabled 2023-07-18T02:58:28.809 INFO:teuthology.orchestra.run.smithi101.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-07-18T02:58:28.811 INFO:teuthology.orchestra.run.smithi101.stdout:Ceph Dashboard is now available at: 2023-07-18T02:58:28.811 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T02:58:28.811 INFO:teuthology.orchestra.run.smithi101.stdout: URL: https://smithi101.front.sepia.ceph.com:8443/ 2023-07-18T02:58:28.811 INFO:teuthology.orchestra.run.smithi101.stdout: User: admin 2023-07-18T02:58:28.812 INFO:teuthology.orchestra.run.smithi101.stdout: Password: 9k7ilboqs5 2023-07-18T02:58:28.812 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T02:58:28.812 INFO:teuthology.orchestra.run.smithi101.stdout:Saving cluster configuration to /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/config directory 2023-07-18T02:58:28.812 INFO:teuthology.orchestra.run.smithi101.stdout:Enabling autotune for osd_memory_target 2023-07-18T02:58:29.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:29 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1245546926' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-07-18T02:58:29.995 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.367+0000 7f17a7ff0700 1 Processor -- start 2023-07-18T02:58:29.995 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.367+0000 7f17a7ff0700 1 -- start start 2023-07-18T02:58:29.995 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.368+0000 7f17a7ff0700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f17a007be00 0x7f17a007a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:29.995 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.368+0000 7f17a7ff0700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a007a840 con 0x7f17a007be00 2023-07-18T02:58:29.995 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.368+0000 7f17a5d8c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f17a007be00 0x7f17a007a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:29.996 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.368+0000 7f17a5d8c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f17a007be00 0x7f17a007a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:58648/0 (socket says 172.21.15.101:58648) 2023-07-18T02:58:29.996 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.368+0000 7f17a5d8c700 1 -- 172.21.15.101:0/3392777678 learned_addr learned my addr 172.21.15.101:0/3392777678 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:58:29.996 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.368+0000 7f17a5d8c700 1 -- 172.21.15.101:0/3392777678 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a007a980 con 0x7f17a007be00 2023-07-18T02:58:29.996 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.369+0000 7f17a5d8c700 1 --2- 172.21.15.101:0/3392777678 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f17a007be00 0x7f17a007a300 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f179c0098a0 tx=0x7f179c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=931991941ae53336 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:29.996 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.369+0000 7f17a4d8a700 1 -- 172.21.15.101:0/3392777678 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f179c004160 con 0x7f17a007be00 2023-07-18T02:58:29.996 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.369+0000 7f17a4d8a700 1 -- 172.21.15.101:0/3392777678 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f179c0042c0 con 0x7f17a007be00 2023-07-18T02:58:29.996 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.369+0000 7f17a4d8a700 1 -- 172.21.15.101:0/3392777678 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f179c0119b0 con 0x7f17a007be00 2023-07-18T02:58:29.997 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.369+0000 7f17a7ff0700 1 -- 172.21.15.101:0/3392777678 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f17a007be00 msgr2=0x7f17a007a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:29.997 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.369+0000 7f17a7ff0700 1 --2- 172.21.15.101:0/3392777678 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f17a007be00 0x7f17a007a300 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f179c0098a0 tx=0x7f179c009bb0 comp rx=0 tx=0).stop 2023-07-18T02:58:29.997 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.370+0000 7f17a7ff0700 1 -- 172.21.15.101:0/3392777678 shutdown_connections 2023-07-18T02:58:29.997 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.370+0000 7f17a7ff0700 1 --2- 172.21.15.101:0/3392777678 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f17a007be00 0x7f17a007a300 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:29.997 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.370+0000 7f17a7ff0700 1 -- 172.21.15.101:0/3392777678 >> 172.21.15.101:0/3392777678 conn(0x7f17a0100970 msgr2=0x7f17a0102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:58:29.997 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.370+0000 7f17a7ff0700 1 -- 172.21.15.101:0/3392777678 shutdown_connections 2023-07-18T02:58:29.997 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.370+0000 7f17a7ff0700 1 -- 172.21.15.101:0/3392777678 wait complete. 2023-07-18T02:58:29.998 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.371+0000 7f17a7ff0700 1 Processor -- start 2023-07-18T02:58:29.998 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.371+0000 7f17a7ff0700 1 -- start start 2023-07-18T02:58:29.998 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.371+0000 7f17a7ff0700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f17a007be00 0x7f17a019b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:29.998 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.371+0000 7f17a7ff0700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a019b880 con 0x7f17a007be00 2023-07-18T02:58:29.998 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.371+0000 7f17a5d8c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f17a007be00 0x7f17a019b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:29.998 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.371+0000 7f17a5d8c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f17a007be00 0x7f17a019b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:58650/0 (socket says 172.21.15.101:58650) 2023-07-18T02:58:29.998 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.371+0000 7f17a5d8c700 1 -- 172.21.15.101:0/2002235503 learned_addr learned my addr 172.21.15.101:0/2002235503 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:58:29.999 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.372+0000 7f17a5d8c700 1 -- 172.21.15.101:0/2002235503 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f179c009550 con 0x7f17a007be00 2023-07-18T02:58:29.999 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.372+0000 7f17a5d8c700 1 --2- 172.21.15.101:0/2002235503 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f17a007be00 0x7f17a019b340 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f179c003e60 tx=0x7f179c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:29.999 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.372+0000 7f1796ffd700 1 -- 172.21.15.101:0/2002235503 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f179c003a60 con 0x7f17a007be00 2023-07-18T02:58:29.999 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.372+0000 7f1796ffd700 1 -- 172.21.15.101:0/2002235503 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f179c003bc0 con 0x7f17a007be00 2023-07-18T02:58:29.999 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.372+0000 7f17a7ff0700 1 -- 172.21.15.101:0/2002235503 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17a019ba80 con 0x7f17a007be00 2023-07-18T02:58:29.999 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.372+0000 7f1796ffd700 1 -- 172.21.15.101:0/2002235503 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f179c0196c0 con 0x7f17a007be00 2023-07-18T02:58:29.999 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.372+0000 7f17a7ff0700 1 -- 172.21.15.101:0/2002235503 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17a019bea0 con 0x7f17a007be00 2023-07-18T02:58:29.999 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.373+0000 7f1796ffd700 1 -- 172.21.15.101:0/2002235503 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f179c01ab60 con 0x7f17a007be00 2023-07-18T02:58:30.000 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.373+0000 7f17a7ff0700 1 -- 172.21.15.101:0/2002235503 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17a010b6d0 con 0x7f17a007be00 2023-07-18T02:58:30.000 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.374+0000 7f1796ffd700 1 --2- 172.21.15.101:0/2002235503 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f178c038830 0x7f178c03ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:30.000 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.374+0000 7f1796ffd700 1 -- 172.21.15.101:0/2002235503 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f179c04cb00 con 0x7f17a007be00 2023-07-18T02:58:30.000 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.374+0000 7f17a558b700 1 --2- 172.21.15.101:0/2002235503 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f178c038830 0x7f178c03ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:30.000 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.375+0000 7f17a558b700 1 --2- 172.21.15.101:0/2002235503 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f178c038830 0x7f178c03ace0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1790006fd0 tx=0x7f1790006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:30.000 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.379+0000 7f1796ffd700 1 -- 172.21.15.101:0/2002235503 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f179c019930 con 0x7f17a007be00 2023-07-18T02:58:30.001 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.573+0000 7f17a7ff0700 1 -- 172.21.15.101:0/2002235503 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f17a010b8c0 con 0x7f17a007be00 2023-07-18T02:58:30.001 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.573+0000 7f1796ffd700 1 -- 172.21.15.101:0/2002235503 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f179c018350 con 0x7f17a007be00 2023-07-18T02:58:30.001 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.576+0000 7f17a7ff0700 1 -- 172.21.15.101:0/2002235503 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f178c038830 msgr2=0x7f178c03ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:30.001 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.576+0000 7f17a7ff0700 1 --2- 172.21.15.101:0/2002235503 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f178c038830 0x7f178c03ace0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1790006fd0 tx=0x7f1790006e40 comp rx=0 tx=0).stop 2023-07-18T02:58:30.001 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.576+0000 7f17a7ff0700 1 -- 172.21.15.101:0/2002235503 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f17a007be00 msgr2=0x7f17a019b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:30.001 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.576+0000 7f17a7ff0700 1 --2- 172.21.15.101:0/2002235503 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f17a007be00 0x7f17a019b340 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f179c003e60 tx=0x7f179c011a30 comp rx=0 tx=0).stop 2023-07-18T02:58:30.002 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.576+0000 7f17a7ff0700 1 -- 172.21.15.101:0/2002235503 shutdown_connections 2023-07-18T02:58:30.002 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.576+0000 7f17a7ff0700 1 --2- 172.21.15.101:0/2002235503 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f178c038830 0x7f178c03ace0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:30.002 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.576+0000 7f17a7ff0700 1 --2- 172.21.15.101:0/2002235503 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f17a007be00 0x7f17a019b340 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:30.002 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.576+0000 7f17a7ff0700 1 -- 172.21.15.101:0/2002235503 >> 172.21.15.101:0/2002235503 conn(0x7f17a0100970 msgr2=0x7f17a01020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:58:30.002 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.576+0000 7f17a7ff0700 1 -- 172.21.15.101:0/2002235503 shutdown_connections 2023-07-18T02:58:30.002 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:29.576+0000 7f17a7ff0700 1 -- 172.21.15.101:0/2002235503 wait complete. 2023-07-18T02:58:30.247 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:30 smithi101 ceph-mon[126379]: mgrmap e13: x(active, since 6s) 2023-07-18T02:58:31.270 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.525+0000 7f6ae6c75700 1 Processor -- start 2023-07-18T02:58:31.270 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.526+0000 7f6ae6c75700 1 -- start start 2023-07-18T02:58:31.270 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.526+0000 7f6ae6c75700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6ae007be00 0x7f6ae007a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:31.270 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.526+0000 7f6ae6c75700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae007a840 con 0x7f6ae007be00 2023-07-18T02:58:31.271 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.526+0000 7f6ae4a11700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6ae007be00 0x7f6ae007a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:31.271 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.526+0000 7f6ae4a11700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6ae007be00 0x7f6ae007a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:58664/0 (socket says 172.21.15.101:58664) 2023-07-18T02:58:31.271 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.526+0000 7f6ae4a11700 1 -- 172.21.15.101:0/1629378916 learned_addr learned my addr 172.21.15.101:0/1629378916 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:58:31.271 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.527+0000 7f6ae4a11700 1 -- 172.21.15.101:0/1629378916 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae007a980 con 0x7f6ae007be00 2023-07-18T02:58:31.271 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.527+0000 7f6ae4a11700 1 --2- 172.21.15.101:0/1629378916 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6ae007be00 0x7f6ae007a300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6ad40098a0 tx=0x7f6ad4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6655f14d0f53d879 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:31.271 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.527+0000 7f6adf7fe700 1 -- 172.21.15.101:0/1629378916 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6ad4004160 con 0x7f6ae007be00 2023-07-18T02:58:31.272 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.527+0000 7f6adf7fe700 1 -- 172.21.15.101:0/1629378916 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ad40042c0 con 0x7f6ae007be00 2023-07-18T02:58:31.272 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.527+0000 7f6adf7fe700 1 -- 172.21.15.101:0/1629378916 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6ad40119b0 con 0x7f6ae007be00 2023-07-18T02:58:31.272 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.528+0000 7f6ae6c75700 1 -- 172.21.15.101:0/1629378916 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6ae007be00 msgr2=0x7f6ae007a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:31.272 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.528+0000 7f6ae6c75700 1 --2- 172.21.15.101:0/1629378916 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6ae007be00 0x7f6ae007a300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6ad40098a0 tx=0x7f6ad4009bb0 comp rx=0 tx=0).stop 2023-07-18T02:58:31.272 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.528+0000 7f6ae6c75700 1 -- 172.21.15.101:0/1629378916 shutdown_connections 2023-07-18T02:58:31.273 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.528+0000 7f6ae6c75700 1 --2- 172.21.15.101:0/1629378916 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6ae007be00 0x7f6ae007a300 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:31.273 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.528+0000 7f6ae6c75700 1 -- 172.21.15.101:0/1629378916 >> 172.21.15.101:0/1629378916 conn(0x7f6ae0100970 msgr2=0x7f6ae0102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:58:31.273 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.528+0000 7f6ae6c75700 1 -- 172.21.15.101:0/1629378916 shutdown_connections 2023-07-18T02:58:31.273 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.528+0000 7f6ae6c75700 1 -- 172.21.15.101:0/1629378916 wait complete. 2023-07-18T02:58:31.273 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.529+0000 7f6ae6c75700 1 Processor -- start 2023-07-18T02:58:31.274 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.529+0000 7f6ae6c75700 1 -- start start 2023-07-18T02:58:31.274 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.529+0000 7f6ae6c75700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6ae007be00 0x7f6ae019b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:31.274 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.529+0000 7f6ae6c75700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae019b880 con 0x7f6ae007be00 2023-07-18T02:58:31.275 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.530+0000 7f6ae4a11700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6ae007be00 0x7f6ae019b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:31.275 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.530+0000 7f6ae4a11700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6ae007be00 0x7f6ae019b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:58672/0 (socket says 172.21.15.101:58672) 2023-07-18T02:58:31.275 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.530+0000 7f6ae4a11700 1 -- 172.21.15.101:0/2627445114 learned_addr learned my addr 172.21.15.101:0/2627445114 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:58:31.275 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.530+0000 7f6ae4a11700 1 -- 172.21.15.101:0/2627445114 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad4009550 con 0x7f6ae007be00 2023-07-18T02:58:31.275 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.530+0000 7f6ae4a11700 1 --2- 172.21.15.101:0/2627445114 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6ae007be00 0x7f6ae019b340 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4003e60 tx=0x7f6ad4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:31.275 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.530+0000 7f6addffb700 1 -- 172.21.15.101:0/2627445114 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6ad4003a60 con 0x7f6ae007be00 2023-07-18T02:58:31.276 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.530+0000 7f6addffb700 1 -- 172.21.15.101:0/2627445114 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ad4003bc0 con 0x7f6ae007be00 2023-07-18T02:58:31.276 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.530+0000 7f6addffb700 1 -- 172.21.15.101:0/2627445114 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6ad40196c0 con 0x7f6ae007be00 2023-07-18T02:58:31.276 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.530+0000 7f6ae6c75700 1 -- 172.21.15.101:0/2627445114 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ae019ba80 con 0x7f6ae007be00 2023-07-18T02:58:31.276 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.531+0000 7f6ae6c75700 1 -- 172.21.15.101:0/2627445114 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ae019bea0 con 0x7f6ae007be00 2023-07-18T02:58:31.276 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.531+0000 7f6ae6c75700 1 -- 172.21.15.101:0/2627445114 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ae0061b40 con 0x7f6ae007be00 2023-07-18T02:58:31.276 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.532+0000 7f6addffb700 1 -- 172.21.15.101:0/2627445114 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f6ad401ab60 con 0x7f6ae007be00 2023-07-18T02:58:31.276 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.532+0000 7f6addffb700 1 --2- 172.21.15.101:0/2627445114 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f6ad0038830 0x7f6ad003ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:31.277 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.532+0000 7f6addffb700 1 -- 172.21.15.101:0/2627445114 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6ad404cbe0 con 0x7f6ae007be00 2023-07-18T02:58:31.277 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.532+0000 7f6adffff700 1 --2- 172.21.15.101:0/2627445114 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f6ad0038830 0x7f6ad003ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:31.277 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.533+0000 7f6adffff700 1 --2- 172.21.15.101:0/2627445114 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f6ad0038830 0x7f6ad003ace0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6acc006fd0 tx=0x7f6acc006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:31.277 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.537+0000 7f6addffb700 1 -- 172.21.15.101:0/2627445114 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6ad4018b00 con 0x7f6ae007be00 2023-07-18T02:58:31.277 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.840+0000 7f6ae6c75700 1 -- 172.21.15.101:0/2627445114 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f6ae019e8e0 con 0x7f6ae007be00 2023-07-18T02:58:31.277 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.843+0000 7f6addffb700 1 -- 172.21.15.101:0/2627445114 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f6ad4018b00 con 0x7f6ae007be00 2023-07-18T02:58:31.277 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.845+0000 7f6ae6c75700 1 -- 172.21.15.101:0/2627445114 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f6ad0038830 msgr2=0x7f6ad003ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:31.278 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.845+0000 7f6ae6c75700 1 --2- 172.21.15.101:0/2627445114 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f6ad0038830 0x7f6ad003ace0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6acc006fd0 tx=0x7f6acc006e40 comp rx=0 tx=0).stop 2023-07-18T02:58:31.278 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.845+0000 7f6ae6c75700 1 -- 172.21.15.101:0/2627445114 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6ae007be00 msgr2=0x7f6ae019b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:31.278 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.846+0000 7f6ae6c75700 1 --2- 172.21.15.101:0/2627445114 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6ae007be00 0x7f6ae019b340 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4003e60 tx=0x7f6ad4011a30 comp rx=0 tx=0).stop 2023-07-18T02:58:31.278 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.846+0000 7f6ae6c75700 1 -- 172.21.15.101:0/2627445114 shutdown_connections 2023-07-18T02:58:31.278 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.846+0000 7f6ae6c75700 1 --2- 172.21.15.101:0/2627445114 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f6ad0038830 0x7f6ad003ace0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:31.278 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.846+0000 7f6ae6c75700 1 --2- 172.21.15.101:0/2627445114 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6ae007be00 0x7f6ae019b340 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:31.278 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.846+0000 7f6ae6c75700 1 -- 172.21.15.101:0/2627445114 >> 172.21.15.101:0/2627445114 conn(0x7f6ae0100970 msgr2=0x7f6ae01020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:58:31.278 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.846+0000 7f6ae6c75700 1 -- 172.21.15.101:0/2627445114 shutdown_connections 2023-07-18T02:58:31.279 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr 2023-07-18T02:58:30.846+0000 7f6ae6c75700 1 -- 172.21.15.101:0/2627445114 wait complete. 2023-07-18T02:58:31.279 INFO:teuthology.orchestra.run.smithi101.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-07-18T02:58:31.279 INFO:teuthology.orchestra.run.smithi101.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-07-18T02:58:31.279 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T02:58:31.279 INFO:teuthology.orchestra.run.smithi101.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-07-18T02:58:31.279 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T02:58:31.279 INFO:teuthology.orchestra.run.smithi101.stdout:Or, if you are only running a single cluster on this host: 2023-07-18T02:58:31.280 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T02:58:31.280 INFO:teuthology.orchestra.run.smithi101.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-07-18T02:58:31.280 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T02:58:31.280 INFO:teuthology.orchestra.run.smithi101.stdout:Please consider enabling telemetry to help improve Ceph: 2023-07-18T02:58:31.280 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T02:58:31.280 INFO:teuthology.orchestra.run.smithi101.stdout: ceph telemetry on 2023-07-18T02:58:31.280 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T02:58:31.280 INFO:teuthology.orchestra.run.smithi101.stdout:For more information see: 2023-07-18T02:58:31.281 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T02:58:31.281 INFO:teuthology.orchestra.run.smithi101.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-07-18T02:58:31.281 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T02:58:31.281 INFO:teuthology.orchestra.run.smithi101.stdout:Bootstrap complete. 2023-07-18T02:58:31.327 INFO:tasks.cephadm:Fetching config... 2023-07-18T02:58:31.327 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-07-18T02:58:31.328 DEBUG:teuthology.orchestra.run.smithi101:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-07-18T02:58:31.371 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-07-18T02:58:31.372 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-07-18T02:58:31.372 DEBUG:teuthology.orchestra.run.smithi101:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-07-18T02:58:31.460 INFO:tasks.cephadm:Fetching mon keyring... 2023-07-18T02:58:31.460 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-07-18T02:58:31.460 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd if=/var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/keyring of=/dev/stdout 2023-07-18T02:58:31.570 INFO:tasks.cephadm:Fetching pub ssh key... 2023-07-18T02:58:31.571 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-07-18T02:58:31.571 DEBUG:teuthology.orchestra.run.smithi101:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-07-18T02:58:31.655 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-07-18T02:58:31.656 DEBUG:teuthology.orchestra.run.smithi101:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCgq7jqdjrsbHBK+iLEAXJN4+1FDgm+QP/Srl6wgTyPcBpTSrXf5QYB3mEx7VFHISYTkQRM51hs5ek5q96AAfKqE/0yWrDHOGrjTdYMLQbrl6lG9hJQG+TgJvPGPltNL7lLXFmNhQfkpp3sJXOfWPDja0g9KzPnwRaySBKSOpTsjrc/MpO3UnzG/QQKfN5KtuGpqPvpYib+nZmO/TkGsBAiPm/nEUa6yJOCJPad16oZe6AGCCt/GlyB1ej+KOh3VfS7RpeDAP82hmGuQY1xBoOG1oQYFPwlMP3pzuyJVTMn0OTE6P3MTIYwddrQj4jZoM8K6S9oCIXvrlGGLWht6dsT5Gsn4d8sO7Lj/60O/Kx9fRxrg1gs1IuodLxhsc8LAQ2VagxlO8yt6V1953AuEA8MubIF49i8ldgfPRng/Mq8rzm7IZ9TfLa8tW/H84N24vFaj3YxiJ3OZ6YYk5ssXYLz8MNaCVsUrRMNq/GyMr+m3Qg+Toucp4LHvLcqbIcFZ9s= ceph-b1782daa-2516-11ee-9b34-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-07-18T02:58:31.796 INFO:teuthology.orchestra.run.smithi101.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCgq7jqdjrsbHBK+iLEAXJN4+1FDgm+QP/Srl6wgTyPcBpTSrXf5QYB3mEx7VFHISYTkQRM51hs5ek5q96AAfKqE/0yWrDHOGrjTdYMLQbrl6lG9hJQG+TgJvPGPltNL7lLXFmNhQfkpp3sJXOfWPDja0g9KzPnwRaySBKSOpTsjrc/MpO3UnzG/QQKfN5KtuGpqPvpYib+nZmO/TkGsBAiPm/nEUa6yJOCJPad16oZe6AGCCt/GlyB1ej+KOh3VfS7RpeDAP82hmGuQY1xBoOG1oQYFPwlMP3pzuyJVTMn0OTE6P3MTIYwddrQj4jZoM8K6S9oCIXvrlGGLWht6dsT5Gsn4d8sO7Lj/60O/Kx9fRxrg1gs1IuodLxhsc8LAQ2VagxlO8yt6V1953AuEA8MubIF49i8ldgfPRng/Mq8rzm7IZ9TfLa8tW/H84N24vFaj3YxiJ3OZ6YYk5ssXYLz8MNaCVsUrRMNq/GyMr+m3Qg+Toucp4LHvLcqbIcFZ9s= ceph-b1782daa-2516-11ee-9b34-001a4aab830c 2023-07-18T02:58:31.828 DEBUG:teuthology.orchestra.run.smithi155:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCgq7jqdjrsbHBK+iLEAXJN4+1FDgm+QP/Srl6wgTyPcBpTSrXf5QYB3mEx7VFHISYTkQRM51hs5ek5q96AAfKqE/0yWrDHOGrjTdYMLQbrl6lG9hJQG+TgJvPGPltNL7lLXFmNhQfkpp3sJXOfWPDja0g9KzPnwRaySBKSOpTsjrc/MpO3UnzG/QQKfN5KtuGpqPvpYib+nZmO/TkGsBAiPm/nEUa6yJOCJPad16oZe6AGCCt/GlyB1ej+KOh3VfS7RpeDAP82hmGuQY1xBoOG1oQYFPwlMP3pzuyJVTMn0OTE6P3MTIYwddrQj4jZoM8K6S9oCIXvrlGGLWht6dsT5Gsn4d8sO7Lj/60O/Kx9fRxrg1gs1IuodLxhsc8LAQ2VagxlO8yt6V1953AuEA8MubIF49i8ldgfPRng/Mq8rzm7IZ9TfLa8tW/H84N24vFaj3YxiJ3OZ6YYk5ssXYLz8MNaCVsUrRMNq/GyMr+m3Qg+Toucp4LHvLcqbIcFZ9s= ceph-b1782daa-2516-11ee-9b34-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-07-18T02:58:31.875 INFO:teuthology.orchestra.run.smithi155.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCgq7jqdjrsbHBK+iLEAXJN4+1FDgm+QP/Srl6wgTyPcBpTSrXf5QYB3mEx7VFHISYTkQRM51hs5ek5q96AAfKqE/0yWrDHOGrjTdYMLQbrl6lG9hJQG+TgJvPGPltNL7lLXFmNhQfkpp3sJXOfWPDja0g9KzPnwRaySBKSOpTsjrc/MpO3UnzG/QQKfN5KtuGpqPvpYib+nZmO/TkGsBAiPm/nEUa6yJOCJPad16oZe6AGCCt/GlyB1ej+KOh3VfS7RpeDAP82hmGuQY1xBoOG1oQYFPwlMP3pzuyJVTMn0OTE6P3MTIYwddrQj4jZoM8K6S9oCIXvrlGGLWht6dsT5Gsn4d8sO7Lj/60O/Kx9fRxrg1gs1IuodLxhsc8LAQ2VagxlO8yt6V1953AuEA8MubIF49i8ldgfPRng/Mq8rzm7IZ9TfLa8tW/H84N24vFaj3YxiJ3OZ6YYk5ssXYLz8MNaCVsUrRMNq/GyMr+m3Qg+Toucp4LHvLcqbIcFZ9s= ceph-b1782daa-2516-11ee-9b34-001a4aab830c 2023-07-18T02:58:31.888 DEBUG:teuthology.orchestra.run.smithi179:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCgq7jqdjrsbHBK+iLEAXJN4+1FDgm+QP/Srl6wgTyPcBpTSrXf5QYB3mEx7VFHISYTkQRM51hs5ek5q96AAfKqE/0yWrDHOGrjTdYMLQbrl6lG9hJQG+TgJvPGPltNL7lLXFmNhQfkpp3sJXOfWPDja0g9KzPnwRaySBKSOpTsjrc/MpO3UnzG/QQKfN5KtuGpqPvpYib+nZmO/TkGsBAiPm/nEUa6yJOCJPad16oZe6AGCCt/GlyB1ej+KOh3VfS7RpeDAP82hmGuQY1xBoOG1oQYFPwlMP3pzuyJVTMn0OTE6P3MTIYwddrQj4jZoM8K6S9oCIXvrlGGLWht6dsT5Gsn4d8sO7Lj/60O/Kx9fRxrg1gs1IuodLxhsc8LAQ2VagxlO8yt6V1953AuEA8MubIF49i8ldgfPRng/Mq8rzm7IZ9TfLa8tW/H84N24vFaj3YxiJ3OZ6YYk5ssXYLz8MNaCVsUrRMNq/GyMr+m3Qg+Toucp4LHvLcqbIcFZ9s= ceph-b1782daa-2516-11ee-9b34-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-07-18T02:58:31.932 INFO:teuthology.orchestra.run.smithi179.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCgq7jqdjrsbHBK+iLEAXJN4+1FDgm+QP/Srl6wgTyPcBpTSrXf5QYB3mEx7VFHISYTkQRM51hs5ek5q96AAfKqE/0yWrDHOGrjTdYMLQbrl6lG9hJQG+TgJvPGPltNL7lLXFmNhQfkpp3sJXOfWPDja0g9KzPnwRaySBKSOpTsjrc/MpO3UnzG/QQKfN5KtuGpqPvpYib+nZmO/TkGsBAiPm/nEUa6yJOCJPad16oZe6AGCCt/GlyB1ej+KOh3VfS7RpeDAP82hmGuQY1xBoOG1oQYFPwlMP3pzuyJVTMn0OTE6P3MTIYwddrQj4jZoM8K6S9oCIXvrlGGLWht6dsT5Gsn4d8sO7Lj/60O/Kx9fRxrg1gs1IuodLxhsc8LAQ2VagxlO8yt6V1953AuEA8MubIF49i8ldgfPRng/Mq8rzm7IZ9TfLa8tW/H84N24vFaj3YxiJ3OZ6YYk5ssXYLz8MNaCVsUrRMNq/GyMr+m3Qg+Toucp4LHvLcqbIcFZ9s= ceph-b1782daa-2516-11ee-9b34-001a4aab830c 2023-07-18T02:58:31.945 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-07-18T02:58:32.217 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:31 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2627445114' entity='client.admin' 2023-07-18T02:58:32.777 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T02:58:34.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:34.633+0000 7fe244c0c700 1 -- 172.21.15.101:0/1003017753 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe2400fde70 msgr2=0x7fe240100290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:34.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:34.633+0000 7fe244c0c700 1 --2- 172.21.15.101:0/1003017753 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe2400fde70 0x7fe240100290 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe230009910 tx=0x7fe230009c20 comp rx=0 tx=0).stop 2023-07-18T02:58:34.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:34.633+0000 7fe244c0c700 1 -- 172.21.15.101:0/1003017753 shutdown_connections 2023-07-18T02:58:34.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:34.633+0000 7fe244c0c700 1 --2- 172.21.15.101:0/1003017753 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe2400fde70 0x7fe240100290 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:34.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:34.633+0000 7fe244c0c700 1 -- 172.21.15.101:0/1003017753 >> 172.21.15.101:0/1003017753 conn(0x7fe2400f9aa0 msgr2=0x7fe2400fbed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:58:34.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:34.633+0000 7fe244c0c700 1 -- 172.21.15.101:0/1003017753 shutdown_connections 2023-07-18T02:58:34.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:34.633+0000 7fe244c0c700 1 -- 172.21.15.101:0/1003017753 wait complete. 2023-07-18T02:58:34.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:34.635+0000 7fe244c0c700 1 Processor -- start 2023-07-18T02:58:34.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:34.635+0000 7fe244c0c700 1 -- start start 2023-07-18T02:58:34.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:34.635+0000 7fe244c0c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe2400fde70 0x7fe24019ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:34.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:34.635+0000 7fe244c0c700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24019b240 con 0x7fe2400fde70 2023-07-18T02:58:34.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:34.635+0000 7fe23e59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe2400fde70 0x7fe24019ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:34.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:34.636+0000 7fe23e59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe2400fde70 0x7fe24019ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:58690/0 (socket says 172.21.15.101:58690) 2023-07-18T02:58:34.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:34.636+0000 7fe23e59c700 1 -- 172.21.15.101:0/2739409667 learned_addr learned my addr 172.21.15.101:0/2739409667 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:58:34.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:34.636+0000 7fe23e59c700 1 -- 172.21.15.101:0/2739409667 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2300095f0 con 0x7fe2400fde70 2023-07-18T02:58:34.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:34.636+0000 7fe23e59c700 1 --2- 172.21.15.101:0/2739409667 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe2400fde70 0x7fe24019ad00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe230005f50 tx=0x7fe2300057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:34.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:34.636+0000 7fe22f7fe700 1 -- 172.21.15.101:0/2739409667 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe23001d070 con 0x7fe2400fde70 2023-07-18T02:58:34.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:34.636+0000 7fe22f7fe700 1 -- 172.21.15.101:0/2739409667 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe230022470 con 0x7fe2400fde70 2023-07-18T02:58:34.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:34.636+0000 7fe22f7fe700 1 -- 172.21.15.101:0/2739409667 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe23000f5f0 con 0x7fe2400fde70 2023-07-18T02:58:34.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:34.636+0000 7fe244c0c700 1 -- 172.21.15.101:0/2739409667 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe24019b440 con 0x7fe2400fde70 2023-07-18T02:58:34.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:34.636+0000 7fe244c0c700 1 -- 172.21.15.101:0/2739409667 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe24019b8e0 con 0x7fe2400fde70 2023-07-18T02:58:34.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:34.638+0000 7fe244c0c700 1 -- 172.21.15.101:0/2739409667 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe240195160 con 0x7fe2400fde70 2023-07-18T02:58:34.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:34.638+0000 7fe22f7fe700 1 -- 172.21.15.101:0/2739409667 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7fe230022850 con 0x7fe2400fde70 2023-07-18T02:58:34.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:34.638+0000 7fe22f7fe700 1 --2- 172.21.15.101:0/2739409667 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe228038800 0x7fe22803acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:34.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:34.638+0000 7fe22f7fe700 1 -- 172.21.15.101:0/2739409667 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe23004cb90 con 0x7fe2400fde70 2023-07-18T02:58:34.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:34.639+0000 7fe23dd9b700 1 --2- 172.21.15.101:0/2739409667 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe228038800 0x7fe22803acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:34.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:34.639+0000 7fe23dd9b700 1 --2- 172.21.15.101:0/2739409667 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe228038800 0x7fe22803acb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe234006fd0 tx=0x7fe234006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:34.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:34.643+0000 7fe22f7fe700 1 -- 172.21.15.101:0/2739409667 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe23002bba0 con 0x7fe2400fde70 2023-07-18T02:58:34.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:34.857+0000 7fe244c0c700 1 -- 172.21.15.101:0/2739409667 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7fe24002c850 con 0x7fe2400fde70 2023-07-18T02:58:34.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:34.861+0000 7fe22f7fe700 1 -- 172.21.15.101:0/2739409667 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7fe230027030 con 0x7fe2400fde70 2023-07-18T02:58:34.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:34.863+0000 7fe244c0c700 1 -- 172.21.15.101:0/2739409667 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe228038800 msgr2=0x7fe22803acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:34.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:34.863+0000 7fe244c0c700 1 --2- 172.21.15.101:0/2739409667 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe228038800 0x7fe22803acb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe234006fd0 tx=0x7fe234006e40 comp rx=0 tx=0).stop 2023-07-18T02:58:34.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:34.863+0000 7fe244c0c700 1 -- 172.21.15.101:0/2739409667 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe2400fde70 msgr2=0x7fe24019ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:34.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:34.863+0000 7fe244c0c700 1 --2- 172.21.15.101:0/2739409667 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe2400fde70 0x7fe24019ad00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe230005f50 tx=0x7fe2300057c0 comp rx=0 tx=0).stop 2023-07-18T02:58:34.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:34.863+0000 7fe244c0c700 1 -- 172.21.15.101:0/2739409667 shutdown_connections 2023-07-18T02:58:34.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:34.863+0000 7fe244c0c700 1 --2- 172.21.15.101:0/2739409667 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe228038800 0x7fe22803acb0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:34.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:34.863+0000 7fe244c0c700 1 --2- 172.21.15.101:0/2739409667 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe2400fde70 0x7fe24019ad00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:34.839 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:34.863+0000 7fe244c0c700 1 -- 172.21.15.101:0/2739409667 >> 172.21.15.101:0/2739409667 conn(0x7fe2400f9aa0 msgr2=0x7fe2400fa750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:58:34.839 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:34.864+0000 7fe244c0c700 1 -- 172.21.15.101:0/2739409667 shutdown_connections 2023-07-18T02:58:34.839 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:34.864+0000 7fe244c0c700 1 -- 172.21.15.101:0/2739409667 wait complete. 2023-07-18T02:58:35.432 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-07-18T02:58:35.433 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-07-18T02:58:36.171 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T02:58:36.217 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:35 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2739409667' entity='client.admin' 2023-07-18T02:58:36.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:35 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:58:36.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:35 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:58:36.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:35 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi101", "name": "osd_memory_target"}]: dispatch 2023-07-18T02:58:36.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:35 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:58:36.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:35 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T02:58:36.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:35 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:58:37.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:37.596+0000 7fcf17fff700 1 -- 172.21.15.101:0/4223186844 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcf180ff430 msgr2=0x7fcf180ff840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:37.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:37.596+0000 7fcf17fff700 1 --2- 172.21.15.101:0/4223186844 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcf180ff430 0x7fcf180ff840 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcf08009910 tx=0x7fcf08009c20 comp rx=0 tx=0).stop 2023-07-18T02:58:37.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:37.596+0000 7fcf17fff700 1 -- 172.21.15.101:0/4223186844 shutdown_connections 2023-07-18T02:58:37.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:37.596+0000 7fcf17fff700 1 --2- 172.21.15.101:0/4223186844 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcf180ff430 0x7fcf180ff840 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:37.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:37.596+0000 7fcf17fff700 1 -- 172.21.15.101:0/4223186844 >> 172.21.15.101:0/4223186844 conn(0x7fcf180fa9e0 msgr2=0x7fcf180fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:58:37.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:37.596+0000 7fcf17fff700 1 -- 172.21.15.101:0/4223186844 shutdown_connections 2023-07-18T02:58:37.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:37.596+0000 7fcf17fff700 1 -- 172.21.15.101:0/4223186844 wait complete. 2023-07-18T02:58:37.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:37.597+0000 7fcf17fff700 1 Processor -- start 2023-07-18T02:58:37.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:37.598+0000 7fcf17fff700 1 -- start start 2023-07-18T02:58:37.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:37.598+0000 7fcf17fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcf180ff430 0x7fcf181967d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:37.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:37.598+0000 7fcf17fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf18196d10 con 0x7fcf180ff430 2023-07-18T02:58:37.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:37.598+0000 7fcf16ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcf180ff430 0x7fcf181967d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:37.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:37.598+0000 7fcf16ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcf180ff430 0x7fcf181967d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:42596/0 (socket says 172.21.15.101:42596) 2023-07-18T02:58:37.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:37.598+0000 7fcf16ffd700 1 -- 172.21.15.101:0/3995952474 learned_addr learned my addr 172.21.15.101:0/3995952474 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:58:37.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:37.598+0000 7fcf16ffd700 1 -- 172.21.15.101:0/3995952474 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf080095f0 con 0x7fcf180ff430 2023-07-18T02:58:37.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:37.599+0000 7fcf16ffd700 1 --2- 172.21.15.101:0/3995952474 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcf180ff430 0x7fcf181967d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcf08004f40 tx=0x7fcf080057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:37.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:37.599+0000 7fcf1c930700 1 -- 172.21.15.101:0/3995952474 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcf0801d070 con 0x7fcf180ff430 2023-07-18T02:58:37.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:37.599+0000 7fcf1c930700 1 -- 172.21.15.101:0/3995952474 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf0800bc10 con 0x7fcf180ff430 2023-07-18T02:58:37.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:37.599+0000 7fcf17fff700 1 -- 172.21.15.101:0/3995952474 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf18196f10 con 0x7fcf180ff430 2023-07-18T02:58:37.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:37.599+0000 7fcf1c930700 1 -- 172.21.15.101:0/3995952474 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcf08003a70 con 0x7fcf180ff430 2023-07-18T02:58:37.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:37.599+0000 7fcf17fff700 1 -- 172.21.15.101:0/3995952474 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf181973b0 con 0x7fcf180ff430 2023-07-18T02:58:37.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:37.600+0000 7fcf1c930700 1 -- 172.21.15.101:0/3995952474 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7fcf08003d30 con 0x7fcf180ff430 2023-07-18T02:58:37.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:37.600+0000 7fcf17fff700 1 -- 172.21.15.101:0/3995952474 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf18190400 con 0x7fcf180ff430 2023-07-18T02:58:37.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:37.601+0000 7fcf1c930700 1 --2- 172.21.15.101:0/3995952474 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fcf000387b0 0x7fcf0003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:37.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:37.601+0000 7fcf1c930700 1 -- 172.21.15.101:0/3995952474 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fcf0804c4e0 con 0x7fcf180ff430 2023-07-18T02:58:37.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:37.601+0000 7fcf167fc700 1 --2- 172.21.15.101:0/3995952474 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fcf000387b0 0x7fcf0003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:37.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:37.601+0000 7fcf167fc700 1 --2- 172.21.15.101:0/3995952474 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fcf000387b0 0x7fcf0003ac60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c006fd0 tx=0x7fcf0c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:37.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:37.605+0000 7fcf1c930700 1 -- 172.21.15.101:0/3995952474 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fcf0802ae20 con 0x7fcf180ff430 2023-07-18T02:58:37.777 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:37.803+0000 7fcf17fff700 1 -- 172.21.15.101:0/3995952474 --> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fcf180609e0 con 0x7fcf000387b0 2023-07-18T02:58:37.780 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:37.807+0000 7fcf1c930700 1 -- 172.21.15.101:0/3995952474 <== mgr.14150 v2:172.21.15.101:6800/3963742842 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fcf180609e0 con 0x7fcf000387b0 2023-07-18T02:58:37.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:37.809+0000 7fcf17fff700 1 -- 172.21.15.101:0/3995952474 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fcf000387b0 msgr2=0x7fcf0003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:37.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:37.809+0000 7fcf17fff700 1 --2- 172.21.15.101:0/3995952474 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fcf000387b0 0x7fcf0003ac60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c006fd0 tx=0x7fcf0c006e40 comp rx=0 tx=0).stop 2023-07-18T02:58:37.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:37.809+0000 7fcf17fff700 1 -- 172.21.15.101:0/3995952474 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcf180ff430 msgr2=0x7fcf181967d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:37.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:37.809+0000 7fcf17fff700 1 --2- 172.21.15.101:0/3995952474 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcf180ff430 0x7fcf181967d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcf08004f40 tx=0x7fcf080057c0 comp rx=0 tx=0).stop 2023-07-18T02:58:37.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:37.809+0000 7fcf17fff700 1 -- 172.21.15.101:0/3995952474 shutdown_connections 2023-07-18T02:58:37.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:37.809+0000 7fcf17fff700 1 --2- 172.21.15.101:0/3995952474 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fcf000387b0 0x7fcf0003ac60 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:37.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:37.809+0000 7fcf17fff700 1 --2- 172.21.15.101:0/3995952474 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcf180ff430 0x7fcf181967d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:37.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:37.809+0000 7fcf17fff700 1 -- 172.21.15.101:0/3995952474 >> 172.21.15.101:0/3995952474 conn(0x7fcf180fa9e0 msgr2=0x7fcf180fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:58:37.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:37.810+0000 7fcf17fff700 1 -- 172.21.15.101:0/3995952474 shutdown_connections 2023-07-18T02:58:37.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:37.810+0000 7fcf17fff700 1 -- 172.21.15.101:0/3995952474 wait complete. 2023-07-18T02:58:38.249 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi155 2023-07-18T02:58:38.250 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-07-18T02:58:38.250 DEBUG:teuthology.orchestra.run.smithi155:> dd of=/etc/ceph/ceph.conf 2023-07-18T02:58:38.270 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-07-18T02:58:38.271 DEBUG:teuthology.orchestra.run.smithi155:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-18T02:58:38.329 INFO:tasks.cephadm:Adding host smithi155 to orchestrator... 2023-07-18T02:58:38.330 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph orch host add smithi155 2023-07-18T02:58:38.969 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T02:58:39.217 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:38 smithi101 ceph-mon[126379]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T02:58:39.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:38 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:58:39.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:38 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T02:58:39.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:38 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:58:39.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:38 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T02:58:39.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:38 smithi101 ceph-mon[126379]: Updating smithi101:/etc/ceph/ceph.conf 2023-07-18T02:58:39.848 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:39 smithi101 ceph-mon[126379]: Updating smithi101:/var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-18T02:58:39.848 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:39 smithi101 ceph-mon[126379]: Updating smithi101:/etc/ceph/ceph.client.admin.keyring 2023-07-18T02:58:40.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:40.519+0000 7f5c2780b700 1 -- 172.21.15.101:0/3726867727 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5c20105650 msgr2=0x7f5c20105a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:40.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:40.519+0000 7f5c2780b700 1 --2- 172.21.15.101:0/3726867727 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5c20105650 0x7f5c20105a20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c009910 tx=0x7f5c1c009c20 comp rx=0 tx=0).stop 2023-07-18T02:58:40.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:40.519+0000 7f5c2780b700 1 -- 172.21.15.101:0/3726867727 shutdown_connections 2023-07-18T02:58:40.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:40.519+0000 7f5c2780b700 1 --2- 172.21.15.101:0/3726867727 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5c20105650 0x7f5c20105a20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:40.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:40.519+0000 7f5c2780b700 1 -- 172.21.15.101:0/3726867727 >> 172.21.15.101:0/3726867727 conn(0x7f5c200fb150 msgr2=0x7f5c200fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:58:40.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:40.519+0000 7f5c2780b700 1 -- 172.21.15.101:0/3726867727 shutdown_connections 2023-07-18T02:58:40.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:40.519+0000 7f5c2780b700 1 -- 172.21.15.101:0/3726867727 wait complete. 2023-07-18T02:58:40.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:40.520+0000 7f5c2780b700 1 Processor -- start 2023-07-18T02:58:40.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:40.521+0000 7f5c2780b700 1 -- start start 2023-07-18T02:58:40.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:40.521+0000 7f5c2780b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5c20105650 0x7f5c20194980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:40.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:40.521+0000 7f5c2780b700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c20194ec0 con 0x7f5c20105650 2023-07-18T02:58:40.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:40.521+0000 7f5c255a7700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5c20105650 0x7f5c20194980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:40.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:40.521+0000 7f5c255a7700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5c20105650 0x7f5c20194980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:42612/0 (socket says 172.21.15.101:42612) 2023-07-18T02:58:40.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:40.521+0000 7f5c255a7700 1 -- 172.21.15.101:0/2944823217 learned_addr learned my addr 172.21.15.101:0/2944823217 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:58:40.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:40.522+0000 7f5c255a7700 1 -- 172.21.15.101:0/2944823217 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c1c0095f0 con 0x7f5c20105650 2023-07-18T02:58:40.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:40.522+0000 7f5c255a7700 1 --2- 172.21.15.101:0/2944823217 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5c20105650 0x7f5c20194980 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c004f40 tx=0x7f5c1c005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:40.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:40.522+0000 7f5c167fc700 1 -- 172.21.15.101:0/2944823217 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c1c01d070 con 0x7f5c20105650 2023-07-18T02:58:40.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:40.522+0000 7f5c167fc700 1 -- 172.21.15.101:0/2944823217 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c1c004370 con 0x7f5c20105650 2023-07-18T02:58:40.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:40.522+0000 7f5c2780b700 1 -- 172.21.15.101:0/2944823217 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c20195120 con 0x7f5c20105650 2023-07-18T02:58:40.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:40.522+0000 7f5c167fc700 1 -- 172.21.15.101:0/2944823217 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c1c0039b0 con 0x7f5c20105650 2023-07-18T02:58:40.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:40.522+0000 7f5c2780b700 1 -- 172.21.15.101:0/2944823217 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c20195620 con 0x7f5c20105650 2023-07-18T02:58:40.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:40.523+0000 7f5c2780b700 1 -- 172.21.15.101:0/2944823217 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c20061b40 con 0x7f5c20105650 2023-07-18T02:58:40.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:40.523+0000 7f5c167fc700 1 -- 172.21.15.101:0/2944823217 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f5c1c003c20 con 0x7f5c20105650 2023-07-18T02:58:40.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:40.524+0000 7f5c167fc700 1 --2- 172.21.15.101:0/2944823217 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f5c0c038800 0x7f5c0c03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:40.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:40.524+0000 7f5c167fc700 1 -- 172.21.15.101:0/2944823217 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5c1c04c3f0 con 0x7f5c20105650 2023-07-18T02:58:40.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:40.524+0000 7f5c24da6700 1 --2- 172.21.15.101:0/2944823217 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f5c0c038800 0x7f5c0c03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:40.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:40.525+0000 7f5c24da6700 1 --2- 172.21.15.101:0/2944823217 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f5c0c038800 0x7f5c0c03acb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5c10006fd0 tx=0x7f5c10006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:40.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:40.529+0000 7f5c167fc700 1 -- 172.21.15.101:0/2944823217 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5c1c00b6f0 con 0x7f5c20105650 2023-07-18T02:58:40.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:40.727+0000 7f5c2780b700 1 -- 172.21.15.101:0/2944823217 --> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi155", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c20195620 con 0x7f5c0c038800 2023-07-18T02:58:41.217 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:40 smithi101 ceph-mon[126379]: Updating smithi101:/var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-18T02:58:41.217 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:40 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:58:41.217 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:40 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:58:41.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:40 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:58:42.217 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:41 smithi101 ceph-mon[126379]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi155", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T02:58:42.217 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:41 smithi101 ceph-mon[126379]: Deploying cephadm binary to smithi155 2023-07-18T02:58:43.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:43.032+0000 7f5c167fc700 1 -- 172.21.15.101:0/2944823217 <== mgr.14150 v2:172.21.15.101:6800/3963742842 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f5c20195620 con 0x7f5c0c038800 2023-07-18T02:58:43.006 INFO:teuthology.orchestra.run.smithi101.stdout:Added host 'smithi155' with addr '172.21.15.155' 2023-07-18T02:58:43.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:43.035+0000 7f5c2780b700 1 -- 172.21.15.101:0/2944823217 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f5c0c038800 msgr2=0x7f5c0c03acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:43.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:43.035+0000 7f5c2780b700 1 --2- 172.21.15.101:0/2944823217 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f5c0c038800 0x7f5c0c03acb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5c10006fd0 tx=0x7f5c10006e40 comp rx=0 tx=0).stop 2023-07-18T02:58:43.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:43.035+0000 7f5c2780b700 1 -- 172.21.15.101:0/2944823217 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5c20105650 msgr2=0x7f5c20194980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:43.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:43.035+0000 7f5c2780b700 1 --2- 172.21.15.101:0/2944823217 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5c20105650 0x7f5c20194980 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c004f40 tx=0x7f5c1c005020 comp rx=0 tx=0).stop 2023-07-18T02:58:43.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:43.035+0000 7f5c2780b700 1 -- 172.21.15.101:0/2944823217 shutdown_connections 2023-07-18T02:58:43.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:43.035+0000 7f5c2780b700 1 --2- 172.21.15.101:0/2944823217 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f5c0c038800 0x7f5c0c03acb0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:43.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:43.035+0000 7f5c2780b700 1 --2- 172.21.15.101:0/2944823217 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5c20105650 0x7f5c20194980 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:43.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:43.035+0000 7f5c2780b700 1 -- 172.21.15.101:0/2944823217 >> 172.21.15.101:0/2944823217 conn(0x7f5c200fb150 msgr2=0x7f5c200fbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:58:43.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:43.035+0000 7f5c2780b700 1 -- 172.21.15.101:0/2944823217 shutdown_connections 2023-07-18T02:58:43.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:43.035+0000 7f5c2780b700 1 -- 172.21.15.101:0/2944823217 wait complete. 2023-07-18T02:58:43.457 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph orch host ls --format=json 2023-07-18T02:58:44.173 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T02:58:44.459 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:44 smithi101 ceph-mon[126379]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:58:44.459 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:44 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:58:44.459 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:44 smithi101 ceph-mon[126379]: Added host smithi155 2023-07-18T02:58:44.460 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:44 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T02:58:44.460 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:44 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:58:45.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:45 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:58:45.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:45.808+0000 7f982ffeb700 1 -- 172.21.15.101:0/505525911 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9828105650 msgr2=0x7f9828105a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:45.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:45.808+0000 7f982ffeb700 1 --2- 172.21.15.101:0/505525911 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9828105650 0x7f9828105a20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9824009910 tx=0x7f9824009c20 comp rx=0 tx=0).stop 2023-07-18T02:58:45.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:45.808+0000 7f982ffeb700 1 -- 172.21.15.101:0/505525911 shutdown_connections 2023-07-18T02:58:45.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:45.808+0000 7f982ffeb700 1 --2- 172.21.15.101:0/505525911 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9828105650 0x7f9828105a20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:45.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:45.808+0000 7f982ffeb700 1 -- 172.21.15.101:0/505525911 >> 172.21.15.101:0/505525911 conn(0x7f98280fb150 msgr2=0x7f98280fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:58:45.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:45.808+0000 7f982ffeb700 1 -- 172.21.15.101:0/505525911 shutdown_connections 2023-07-18T02:58:45.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:45.808+0000 7f982ffeb700 1 -- 172.21.15.101:0/505525911 wait complete. 2023-07-18T02:58:45.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:45.809+0000 7f982ffeb700 1 Processor -- start 2023-07-18T02:58:45.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:45.809+0000 7f982ffeb700 1 -- start start 2023-07-18T02:58:45.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:45.810+0000 7f982ffeb700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9828105650 0x7f982819c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:45.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:45.810+0000 7f982ffeb700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f982819ce60 con 0x7f9828105650 2023-07-18T02:58:45.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:45.810+0000 7f982dd87700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9828105650 0x7f982819c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:45.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:45.810+0000 7f982dd87700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9828105650 0x7f982819c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:42630/0 (socket says 172.21.15.101:42630) 2023-07-18T02:58:45.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:45.810+0000 7f982dd87700 1 -- 172.21.15.101:0/602622969 learned_addr learned my addr 172.21.15.101:0/602622969 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:58:45.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:45.810+0000 7f982dd87700 1 -- 172.21.15.101:0/602622969 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98240095f0 con 0x7f9828105650 2023-07-18T02:58:45.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:45.810+0000 7f982dd87700 1 --2- 172.21.15.101:0/602622969 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9828105650 0x7f982819c920 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9824006010 tx=0x7f9824004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:45.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:45.810+0000 7f981effd700 1 -- 172.21.15.101:0/602622969 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f982401d070 con 0x7f9828105650 2023-07-18T02:58:45.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:45.811+0000 7f981effd700 1 -- 172.21.15.101:0/602622969 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f982400bc10 con 0x7f9828105650 2023-07-18T02:58:45.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:45.811+0000 7f982ffeb700 1 -- 172.21.15.101:0/602622969 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f982819d0c0 con 0x7f9828105650 2023-07-18T02:58:45.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:45.811+0000 7f981effd700 1 -- 172.21.15.101:0/602622969 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9824003910 con 0x7f9828105650 2023-07-18T02:58:45.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:45.811+0000 7f982ffeb700 1 -- 172.21.15.101:0/602622969 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98281969a0 con 0x7f9828105650 2023-07-18T02:58:45.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:45.812+0000 7f981effd700 1 -- 172.21.15.101:0/602622969 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f982400bd80 con 0x7f9828105650 2023-07-18T02:58:45.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:45.812+0000 7f982ffeb700 1 -- 172.21.15.101:0/602622969 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9828061b40 con 0x7f9828105650 2023-07-18T02:58:45.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:45.812+0000 7f981effd700 1 --2- 172.21.15.101:0/602622969 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f9814038800 0x7f981403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:45.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:45.812+0000 7f981effd700 1 -- 172.21.15.101:0/602622969 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f982404c560 con 0x7f9828105650 2023-07-18T02:58:45.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:45.813+0000 7f982d586700 1 --2- 172.21.15.101:0/602622969 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f9814038800 0x7f981403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:45.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:45.813+0000 7f982d586700 1 --2- 172.21.15.101:0/602622969 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f9814038800 0x7f981403acb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9818006fd0 tx=0x7f9818006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:45.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:45.817+0000 7f981effd700 1 -- 172.21.15.101:0/602622969 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f982402ad50 con 0x7f9828105650 2023-07-18T02:58:45.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:46.018+0000 7f982ffeb700 1 -- 172.21.15.101:0/602622969 --> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9828197460 con 0x7f9814038800 2023-07-18T02:58:45.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:46.019+0000 7f981effd700 1 -- 172.21.15.101:0/602622969 <== mgr.14150 v2:172.21.15.101:6800/3963742842 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (secure 0 0 0) 0x7f9828197460 con 0x7f9814038800 2023-07-18T02:58:45.993 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T02:58:45.993 INFO:teuthology.orchestra.run.smithi101.stdout:[{"addr": "172.21.15.101", "hostname": "smithi101", "labels": [], "status": ""}, {"addr": "172.21.15.155", "hostname": "smithi155", "labels": [], "status": ""}] 2023-07-18T02:58:45.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:46.022+0000 7f982ffeb700 1 -- 172.21.15.101:0/602622969 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f9814038800 msgr2=0x7f981403acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:45.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:46.022+0000 7f982ffeb700 1 --2- 172.21.15.101:0/602622969 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f9814038800 0x7f981403acb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9818006fd0 tx=0x7f9818006e40 comp rx=0 tx=0).stop 2023-07-18T02:58:45.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:46.022+0000 7f982ffeb700 1 -- 172.21.15.101:0/602622969 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9828105650 msgr2=0x7f982819c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:45.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:46.022+0000 7f982ffeb700 1 --2- 172.21.15.101:0/602622969 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9828105650 0x7f982819c920 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9824006010 tx=0x7f9824004ff0 comp rx=0 tx=0).stop 2023-07-18T02:58:45.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:46.022+0000 7f982ffeb700 1 -- 172.21.15.101:0/602622969 shutdown_connections 2023-07-18T02:58:45.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:46.022+0000 7f982ffeb700 1 --2- 172.21.15.101:0/602622969 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f9814038800 0x7f981403acb0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:45.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:46.022+0000 7f982ffeb700 1 --2- 172.21.15.101:0/602622969 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9828105650 0x7f982819c920 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:45.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:46.022+0000 7f982ffeb700 1 -- 172.21.15.101:0/602622969 >> 172.21.15.101:0/602622969 conn(0x7f98280fb150 msgr2=0x7f9828074200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:58:45.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:46.022+0000 7f982ffeb700 1 -- 172.21.15.101:0/602622969 shutdown_connections 2023-07-18T02:58:45.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:46.022+0000 7f982ffeb700 1 -- 172.21.15.101:0/602622969 wait complete. 2023-07-18T02:58:46.417 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:46 smithi101 ceph-mon[126379]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:58:46.418 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:46 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:58:46.446 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi179 2023-07-18T02:58:46.446 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2023-07-18T02:58:46.446 DEBUG:teuthology.orchestra.run.smithi179:> dd of=/etc/ceph/ceph.conf 2023-07-18T02:58:46.469 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2023-07-18T02:58:46.469 DEBUG:teuthology.orchestra.run.smithi179:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-18T02:58:46.528 INFO:tasks.cephadm:Adding host smithi179 to orchestrator... 2023-07-18T02:58:46.529 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph orch host add smithi179 2023-07-18T02:58:47.243 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T02:58:47.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:47 smithi101 ceph-mon[126379]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T02:58:48.410 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:48 smithi101 ceph-mon[126379]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:58:48.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:48.676+0000 7f9913aae700 1 -- 172.21.15.101:0/2355546218 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f990c105650 msgr2=0x7f990c105a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:48.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:48.676+0000 7f9913aae700 1 --2- 172.21.15.101:0/2355546218 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f990c105650 0x7f990c105a20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9908009910 tx=0x7f9908009c20 comp rx=0 tx=0).stop 2023-07-18T02:58:48.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:48.676+0000 7f9913aae700 1 -- 172.21.15.101:0/2355546218 shutdown_connections 2023-07-18T02:58:48.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:48.676+0000 7f9913aae700 1 --2- 172.21.15.101:0/2355546218 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f990c105650 0x7f990c105a20 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:48.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:48.676+0000 7f9913aae700 1 -- 172.21.15.101:0/2355546218 >> 172.21.15.101:0/2355546218 conn(0x7f990c0fb150 msgr2=0x7f990c0fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:58:48.651 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:48.676+0000 7f9913aae700 1 -- 172.21.15.101:0/2355546218 shutdown_connections 2023-07-18T02:58:48.651 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:48.676+0000 7f9913aae700 1 -- 172.21.15.101:0/2355546218 wait complete. 2023-07-18T02:58:48.654 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:48.677+0000 7f9913aae700 1 Processor -- start 2023-07-18T02:58:48.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:48.677+0000 7f9913aae700 1 -- start start 2023-07-18T02:58:48.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:48.678+0000 7f9913aae700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f990c105650 0x7f990c072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:48.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:48.678+0000 7f9913aae700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990c074790 con 0x7f990c105650 2023-07-18T02:58:48.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:48.678+0000 7f991184a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f990c105650 0x7f990c072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:48.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:48.678+0000 7f991184a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f990c105650 0x7f990c072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:35462/0 (socket says 172.21.15.101:35462) 2023-07-18T02:58:48.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:48.678+0000 7f991184a700 1 -- 172.21.15.101:0/3094800446 learned_addr learned my addr 172.21.15.101:0/3094800446 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:58:48.657 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:48.678+0000 7f991184a700 1 -- 172.21.15.101:0/3094800446 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99080095f0 con 0x7f990c105650 2023-07-18T02:58:48.657 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:48.678+0000 7f991184a700 1 --2- 172.21.15.101:0/3094800446 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f990c105650 0x7f990c072810 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9908006010 tx=0x7f9908004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:48.657 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:48.679+0000 7f9902ffd700 1 -- 172.21.15.101:0/3094800446 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f990801d070 con 0x7f990c105650 2023-07-18T02:58:48.658 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:48.679+0000 7f9902ffd700 1 -- 172.21.15.101:0/3094800446 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f990800bc10 con 0x7f990c105650 2023-07-18T02:58:48.658 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:48.679+0000 7f9913aae700 1 -- 172.21.15.101:0/3094800446 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f990c072d50 con 0x7f990c105650 2023-07-18T02:58:48.658 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:48.679+0000 7f9902ffd700 1 -- 172.21.15.101:0/3094800446 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9908003910 con 0x7f990c105650 2023-07-18T02:58:48.659 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:48.679+0000 7f9913aae700 1 -- 172.21.15.101:0/3094800446 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f990c073250 con 0x7f990c105650 2023-07-18T02:58:48.659 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:48.680+0000 7f9902ffd700 1 -- 172.21.15.101:0/3094800446 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f990800bd80 con 0x7f990c105650 2023-07-18T02:58:48.659 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:48.680+0000 7f9913aae700 1 -- 172.21.15.101:0/3094800446 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f990c1048f0 con 0x7f990c105650 2023-07-18T02:58:48.659 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:48.680+0000 7f9902ffd700 1 --2- 172.21.15.101:0/3094800446 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f98f80387b0 0x7f98f803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:48.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:48.680+0000 7f9902ffd700 1 -- 172.21.15.101:0/3094800446 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f990804c510 con 0x7f990c105650 2023-07-18T02:58:48.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:48.681+0000 7f9911049700 1 --2- 172.21.15.101:0/3094800446 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f98f80387b0 0x7f98f803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:48.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:48.681+0000 7f9911049700 1 --2- 172.21.15.101:0/3094800446 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f98f80387b0 0x7f98f803ac60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f98fc006fd0 tx=0x7f98fc006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:48.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:48.685+0000 7f9902ffd700 1 -- 172.21.15.101:0/3094800446 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f990802a3b0 con 0x7f990c105650 2023-07-18T02:58:48.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:48.898+0000 7f9913aae700 1 -- 172.21.15.101:0/3094800446 --> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi179", "target": ["mon-mgr", ""]}) v1 -- 0x7f990c0008d0 con 0x7f98f80387b0 2023-07-18T02:58:49.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:49 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:58:49.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:49 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:58:49.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:49 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:58:49.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:49 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:58:49.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:49 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi155", "name": "osd_memory_target"}]: dispatch 2023-07-18T02:58:49.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:49 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:58:49.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:49 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T02:58:49.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:49 smithi101 ceph-mon[126379]: Updating smithi155:/etc/ceph/ceph.conf 2023-07-18T02:58:49.469 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:49 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:58:49.469 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:49 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:58:49.469 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:49 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:58:50.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:50 smithi101 ceph-mon[126379]: Updating smithi155:/var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-18T02:58:50.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:50 smithi101 ceph-mon[126379]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2023-07-18T02:58:50.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:50 smithi101 ceph-mon[126379]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:58:50.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:50 smithi101 ceph-mon[126379]: Updating smithi155:/var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-18T02:58:50.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:50 smithi101 ceph-mon[126379]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi179", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T02:58:51.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:51.162+0000 7f9902ffd700 1 -- 172.21.15.101:0/3094800446 <== mgr.14150 v2:172.21.15.101:6800/3963742842 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f990c0008d0 con 0x7f98f80387b0 2023-07-18T02:58:51.135 INFO:teuthology.orchestra.run.smithi101.stdout:Added host 'smithi179' with addr '172.21.15.179' 2023-07-18T02:58:51.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:51.165+0000 7f9913aae700 1 -- 172.21.15.101:0/3094800446 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f98f80387b0 msgr2=0x7f98f803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:51.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:51.165+0000 7f9913aae700 1 --2- 172.21.15.101:0/3094800446 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f98f80387b0 0x7f98f803ac60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f98fc006fd0 tx=0x7f98fc006e40 comp rx=0 tx=0).stop 2023-07-18T02:58:51.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:51.165+0000 7f9913aae700 1 -- 172.21.15.101:0/3094800446 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f990c105650 msgr2=0x7f990c072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:51.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:51.165+0000 7f9913aae700 1 --2- 172.21.15.101:0/3094800446 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f990c105650 0x7f990c072810 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9908006010 tx=0x7f9908004ff0 comp rx=0 tx=0).stop 2023-07-18T02:58:51.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:51.165+0000 7f9913aae700 1 -- 172.21.15.101:0/3094800446 shutdown_connections 2023-07-18T02:58:51.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:51.165+0000 7f9913aae700 1 --2- 172.21.15.101:0/3094800446 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f98f80387b0 0x7f98f803ac60 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:51.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:51.165+0000 7f9913aae700 1 --2- 172.21.15.101:0/3094800446 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f990c105650 0x7f990c072810 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:51.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:51.165+0000 7f9913aae700 1 -- 172.21.15.101:0/3094800446 >> 172.21.15.101:0/3094800446 conn(0x7f990c0fb150 msgr2=0x7f990c0fbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:58:51.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:51.165+0000 7f9913aae700 1 -- 172.21.15.101:0/3094800446 shutdown_connections 2023-07-18T02:58:51.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:51.165+0000 7f9913aae700 1 -- 172.21.15.101:0/3094800446 wait complete. 2023-07-18T02:58:51.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:51 smithi101 ceph-mon[126379]: Deploying cephadm binary to smithi179 2023-07-18T02:58:51.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:51 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:58:51.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:51 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T02:58:51.617 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph orch host ls --format=json 2023-07-18T02:58:52.334 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T02:58:52.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:52 smithi101 ceph-mon[126379]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:58:52.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:52 smithi101 ceph-mon[126379]: Added host smithi179 2023-07-18T02:58:52.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:52 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:58:52.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:52 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:58:53.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:53.924+0000 7f8550a8e700 1 -- 172.21.15.101:0/280374606 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f854c0728b0 msgr2=0x7f854c072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:53.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:53.924+0000 7f8550a8e700 1 --2- 172.21.15.101:0/280374606 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f854c0728b0 0x7f854c072c80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f853c009910 tx=0x7f853c009c20 comp rx=0 tx=0).stop 2023-07-18T02:58:53.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:53.924+0000 7f8550a8e700 1 -- 172.21.15.101:0/280374606 shutdown_connections 2023-07-18T02:58:53.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:53.924+0000 7f8550a8e700 1 --2- 172.21.15.101:0/280374606 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f854c0728b0 0x7f854c072c80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:53.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:53.924+0000 7f8550a8e700 1 -- 172.21.15.101:0/280374606 >> 172.21.15.101:0/280374606 conn(0x7f854c0fb150 msgr2=0x7f854c0fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:58:53.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:53.924+0000 7f8550a8e700 1 -- 172.21.15.101:0/280374606 shutdown_connections 2023-07-18T02:58:53.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:53.924+0000 7f8550a8e700 1 -- 172.21.15.101:0/280374606 wait complete. 2023-07-18T02:58:53.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:53.925+0000 7f8550a8e700 1 Processor -- start 2023-07-18T02:58:53.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:53.926+0000 7f8550a8e700 1 -- start start 2023-07-18T02:58:53.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:53.926+0000 7f8550a8e700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f854c0728b0 0x7f854c19ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:53.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:53.926+0000 7f8550a8e700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f854c19cf90 con 0x7f854c0728b0 2023-07-18T02:58:53.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:53.926+0000 7f854a59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f854c0728b0 0x7f854c19ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:53.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:53.926+0000 7f854a59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f854c0728b0 0x7f854c19ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:35482/0 (socket says 172.21.15.101:35482) 2023-07-18T02:58:53.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:53.926+0000 7f854a59c700 1 -- 172.21.15.101:0/2556100300 learned_addr learned my addr 172.21.15.101:0/2556100300 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:58:53.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:53.926+0000 7f854a59c700 1 -- 172.21.15.101:0/2556100300 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f853c0095f0 con 0x7f854c0728b0 2023-07-18T02:58:53.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:53.927+0000 7f854a59c700 1 --2- 172.21.15.101:0/2556100300 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f854c0728b0 0x7f854c19ca50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f853c006010 tx=0x7f853c004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:53.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:53.927+0000 7f853b7fe700 1 -- 172.21.15.101:0/2556100300 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f853c01d070 con 0x7f854c0728b0 2023-07-18T02:58:53.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:53.927+0000 7f853b7fe700 1 -- 172.21.15.101:0/2556100300 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f853c0044a0 con 0x7f854c0728b0 2023-07-18T02:58:53.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:53.927+0000 7f853b7fe700 1 -- 172.21.15.101:0/2556100300 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f853c003a70 con 0x7f854c0728b0 2023-07-18T02:58:53.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:53.927+0000 7f8550a8e700 1 -- 172.21.15.101:0/2556100300 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f854c19d190 con 0x7f854c0728b0 2023-07-18T02:58:53.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:53.927+0000 7f8550a8e700 1 -- 172.21.15.101:0/2556100300 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f854c19d5b0 con 0x7f854c0728b0 2023-07-18T02:58:53.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:53.928+0000 7f853b7fe700 1 -- 172.21.15.101:0/2556100300 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f853c003c60 con 0x7f854c0728b0 2023-07-18T02:58:53.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:53.928+0000 7f8550a8e700 1 -- 172.21.15.101:0/2556100300 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f854c061b40 con 0x7f854c0728b0 2023-07-18T02:58:53.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:53.929+0000 7f853b7fe700 1 --2- 172.21.15.101:0/2556100300 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8534038800 0x7f853403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:53.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:53.929+0000 7f853b7fe700 1 -- 172.21.15.101:0/2556100300 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f853c04c660 con 0x7f854c0728b0 2023-07-18T02:58:53.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:53.929+0000 7f8549d9b700 1 --2- 172.21.15.101:0/2556100300 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8534038800 0x7f853403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:53.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:53.929+0000 7f8549d9b700 1 --2- 172.21.15.101:0/2556100300 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8534038800 0x7f853403acb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8540006fd0 tx=0x7f8540006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:53.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:53.934+0000 7f853b7fe700 1 -- 172.21.15.101:0/2556100300 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f853c005c00 con 0x7f854c0728b0 2023-07-18T02:58:54.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:54.132+0000 7f8550a8e700 1 -- 172.21.15.101:0/2556100300 --> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f854c108c70 con 0x7f8534038800 2023-07-18T02:58:54.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:54.134+0000 7f853b7fe700 1 -- 172.21.15.101:0/2556100300 <== mgr.14150 v2:172.21.15.101:6800/3963742842 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+241 (secure 0 0 0) 0x7f854c108c70 con 0x7f8534038800 2023-07-18T02:58:54.106 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T02:58:54.107 INFO:teuthology.orchestra.run.smithi101.stdout:[{"addr": "172.21.15.101", "hostname": "smithi101", "labels": [], "status": ""}, {"addr": "172.21.15.155", "hostname": "smithi155", "labels": [], "status": ""}, {"addr": "172.21.15.179", "hostname": "smithi179", "labels": [], "status": ""}] 2023-07-18T02:58:54.108 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:54.136+0000 7f8550a8e700 1 -- 172.21.15.101:0/2556100300 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8534038800 msgr2=0x7f853403acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:54.108 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:54.136+0000 7f8550a8e700 1 --2- 172.21.15.101:0/2556100300 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8534038800 0x7f853403acb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8540006fd0 tx=0x7f8540006e40 comp rx=0 tx=0).stop 2023-07-18T02:58:54.108 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:54.136+0000 7f8550a8e700 1 -- 172.21.15.101:0/2556100300 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f854c0728b0 msgr2=0x7f854c19ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:54.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:54.136+0000 7f8550a8e700 1 --2- 172.21.15.101:0/2556100300 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f854c0728b0 0x7f854c19ca50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f853c006010 tx=0x7f853c004ff0 comp rx=0 tx=0).stop 2023-07-18T02:58:54.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:54.136+0000 7f8550a8e700 1 -- 172.21.15.101:0/2556100300 shutdown_connections 2023-07-18T02:58:54.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:54.136+0000 7f8550a8e700 1 --2- 172.21.15.101:0/2556100300 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8534038800 0x7f853403acb0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:54.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:54.136+0000 7f8550a8e700 1 --2- 172.21.15.101:0/2556100300 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f854c0728b0 0x7f854c19ca50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:54.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:54.136+0000 7f8550a8e700 1 -- 172.21.15.101:0/2556100300 >> 172.21.15.101:0/2556100300 conn(0x7f854c0fb150 msgr2=0x7f854c106150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:58:54.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:54.136+0000 7f8550a8e700 1 -- 172.21.15.101:0/2556100300 shutdown_connections 2023-07-18T02:58:54.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:54.136+0000 7f8550a8e700 1 -- 172.21.15.101:0/2556100300 wait complete. 2023-07-18T02:58:54.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:54 smithi101 ceph-mon[126379]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:58:54.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:54 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:58:54.607 INFO:tasks.cephadm:Setting crush tunables to default 2023-07-18T02:58:54.607 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph osd crush tunables default 2023-07-18T02:58:55.324 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T02:58:55.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:55 smithi101 ceph-mon[126379]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T02:58:56.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:56 smithi101 ceph-mon[126379]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:58:56.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:56 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:58:56.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:56 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:58:56.469 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:56 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:58:56.469 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:56 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:58:56.469 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:56 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi179", "name": "osd_memory_target"}]: dispatch 2023-07-18T02:58:56.469 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:56 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:58:56.470 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:56 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T02:58:56.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:56.928+0000 7fd10bfe2700 1 -- 172.21.15.101:0/2935898406 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd104105650 msgr2=0x7fd104105a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:56.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:56.928+0000 7fd10bfe2700 1 --2- 172.21.15.101:0/2935898406 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd104105650 0x7fd104105a20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd100009910 tx=0x7fd100009c20 comp rx=0 tx=0).stop 2023-07-18T02:58:56.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:56.928+0000 7fd10bfe2700 1 -- 172.21.15.101:0/2935898406 shutdown_connections 2023-07-18T02:58:56.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:56.928+0000 7fd10bfe2700 1 --2- 172.21.15.101:0/2935898406 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd104105650 0x7fd104105a20 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:56.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:56.928+0000 7fd10bfe2700 1 -- 172.21.15.101:0/2935898406 >> 172.21.15.101:0/2935898406 conn(0x7fd1040fb150 msgr2=0x7fd1040fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:58:56.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:56.928+0000 7fd10bfe2700 1 -- 172.21.15.101:0/2935898406 shutdown_connections 2023-07-18T02:58:56.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:56.928+0000 7fd10bfe2700 1 -- 172.21.15.101:0/2935898406 wait complete. 2023-07-18T02:58:56.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:56.929+0000 7fd10bfe2700 1 Processor -- start 2023-07-18T02:58:56.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:56.929+0000 7fd10bfe2700 1 -- start start 2023-07-18T02:58:56.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:56.929+0000 7fd10bfe2700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd104105650 0x7fd10419c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:56.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:56.930+0000 7fd10bfe2700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10419ceb0 con 0x7fd104105650 2023-07-18T02:58:56.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:56.930+0000 7fd109d7e700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd104105650 0x7fd10419c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:56.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:56.930+0000 7fd109d7e700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd104105650 0x7fd10419c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:35442/0 (socket says 172.21.15.101:35442) 2023-07-18T02:58:56.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:56.930+0000 7fd109d7e700 1 -- 172.21.15.101:0/1472864393 learned_addr learned my addr 172.21.15.101:0/1472864393 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:58:56.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:56.930+0000 7fd109d7e700 1 -- 172.21.15.101:0/1472864393 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1000095f0 con 0x7fd104105650 2023-07-18T02:58:56.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:56.930+0000 7fd109d7e700 1 --2- 172.21.15.101:0/1472864393 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd104105650 0x7fd10419c970 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd100006010 tx=0x7fd100004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:56.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:56.931+0000 7fd0faffd700 1 -- 172.21.15.101:0/1472864393 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd10001d070 con 0x7fd104105650 2023-07-18T02:58:56.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:56.931+0000 7fd10bfe2700 1 -- 172.21.15.101:0/1472864393 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd10419d110 con 0x7fd104105650 2023-07-18T02:58:56.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:56.931+0000 7fd0faffd700 1 -- 172.21.15.101:0/1472864393 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd10000bc10 con 0x7fd104105650 2023-07-18T02:58:56.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:56.931+0000 7fd0faffd700 1 -- 172.21.15.101:0/1472864393 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd100003910 con 0x7fd104105650 2023-07-18T02:58:56.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:56.931+0000 7fd10bfe2700 1 -- 172.21.15.101:0/1472864393 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1041969f0 con 0x7fd104105650 2023-07-18T02:58:56.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:56.932+0000 7fd10bfe2700 1 -- 172.21.15.101:0/1472864393 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd104061b40 con 0x7fd104105650 2023-07-18T02:58:56.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:56.932+0000 7fd0faffd700 1 -- 172.21.15.101:0/1472864393 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7fd10000bd80 con 0x7fd104105650 2023-07-18T02:58:56.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:56.933+0000 7fd0faffd700 1 --2- 172.21.15.101:0/1472864393 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd0f0038800 0x7fd0f003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:58:56.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:56.933+0000 7fd0faffd700 1 -- 172.21.15.101:0/1472864393 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd10004c560 con 0x7fd104105650 2023-07-18T02:58:56.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:56.933+0000 7fd10957d700 1 --2- 172.21.15.101:0/1472864393 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd0f0038800 0x7fd0f003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:58:56.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:56.933+0000 7fd10957d700 1 --2- 172.21.15.101:0/1472864393 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd0f0038800 0x7fd0f003acb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4006fd0 tx=0x7fd0f4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:58:56.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:56.937+0000 7fd0faffd700 1 -- 172.21.15.101:0/1472864393 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd10002ad00 con 0x7fd104105650 2023-07-18T02:58:57.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:57.154+0000 7fd10bfe2700 1 -- 172.21.15.101:0/1472864393 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fd104197f00 con 0x7fd104105650 2023-07-18T02:58:57.717 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:57 smithi101 ceph-mon[126379]: Updating smithi179:/etc/ceph/ceph.conf 2023-07-18T02:58:57.717 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:57 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:58:57.717 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:57 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:58:57.718 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:57 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:58:57.718 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:57 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1472864393' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-07-18T02:58:58.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:58.035+0000 7fd0faffd700 1 -- 172.21.15.101:0/1472864393 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7fd100026020 con 0x7fd104105650 2023-07-18T02:58:58.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:58.038+0000 7fd10bfe2700 1 -- 172.21.15.101:0/1472864393 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd0f0038800 msgr2=0x7fd0f003acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:58.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:58.038+0000 7fd10bfe2700 1 --2- 172.21.15.101:0/1472864393 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd0f0038800 0x7fd0f003acb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4006fd0 tx=0x7fd0f4006e40 comp rx=0 tx=0).stop 2023-07-18T02:58:58.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:58.038+0000 7fd10bfe2700 1 -- 172.21.15.101:0/1472864393 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd104105650 msgr2=0x7fd10419c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:58:58.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:58.038+0000 7fd10bfe2700 1 --2- 172.21.15.101:0/1472864393 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd104105650 0x7fd10419c970 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd100006010 tx=0x7fd100004ff0 comp rx=0 tx=0).stop 2023-07-18T02:58:58.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:58.039+0000 7fd10bfe2700 1 -- 172.21.15.101:0/1472864393 shutdown_connections 2023-07-18T02:58:58.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:58.039+0000 7fd10bfe2700 1 --2- 172.21.15.101:0/1472864393 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd0f0038800 0x7fd0f003acb0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:58.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:58.039+0000 7fd10bfe2700 1 --2- 172.21.15.101:0/1472864393 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd104105650 0x7fd10419c970 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:58:58.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:58.039+0000 7fd10bfe2700 1 -- 172.21.15.101:0/1472864393 >> 172.21.15.101:0/1472864393 conn(0x7fd1040fb150 msgr2=0x7fd1040fbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:58:58.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:58.039+0000 7fd10bfe2700 1 -- 172.21.15.101:0/1472864393 shutdown_connections 2023-07-18T02:58:58.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:58:58.039+0000 7fd10bfe2700 1 -- 172.21.15.101:0/1472864393 wait complete. 2023-07-18T02:58:58.012 INFO:teuthology.orchestra.run.smithi101.stderr:adjusted tunables profile to default 2023-07-18T02:58:58.433 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:58 smithi101 ceph-mon[126379]: Updating smithi179:/var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-18T02:58:58.434 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:58 smithi101 ceph-mon[126379]: Updating smithi179:/etc/ceph/ceph.client.admin.keyring 2023-07-18T02:58:58.434 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:58 smithi101 ceph-mon[126379]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:58:58.434 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:58 smithi101 ceph-mon[126379]: Updating smithi179:/var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-18T02:58:58.434 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:58 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1472864393' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-07-18T02:58:58.434 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:58:58 smithi101 ceph-mon[126379]: osdmap e4: 0 total, 0 up, 0 in 2023-07-18T02:58:58.500 INFO:tasks.cephadm:Adding mon.a on smithi101 2023-07-18T02:58:58.501 INFO:tasks.cephadm:Adding mon.b on smithi155 2023-07-18T02:58:58.501 INFO:tasks.cephadm:Adding mon.c on smithi179 2023-07-18T02:58:58.501 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph orch apply mon '3;smithi101:172.21.15.101=a;smithi155:172.21.15.155=b;smithi179:172.21.15.179=c' 2023-07-18T02:58:59.086 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-18T02:59:00.253 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:00.279+0000 7f3330995700 1 -- 172.21.15.179:0/2137055081 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f332c0ff560 msgr2=0x7f332c0ff930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:59:00.253 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:00.279+0000 7f3330995700 1 --2- 172.21.15.179:0/2137055081 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f332c0ff560 0x7f332c0ff930 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f331c009910 tx=0x7f331c009c20 comp rx=0 tx=0).stop 2023-07-18T02:59:00.253 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:00.279+0000 7f3330995700 1 -- 172.21.15.179:0/2137055081 shutdown_connections 2023-07-18T02:59:00.253 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:00.279+0000 7f3330995700 1 --2- 172.21.15.179:0/2137055081 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f332c0ff560 0x7f332c0ff930 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:59:00.253 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:00.279+0000 7f3330995700 1 -- 172.21.15.179:0/2137055081 >> 172.21.15.179:0/2137055081 conn(0x7f332c074f40 msgr2=0x7f332c075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:59:00.254 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:00.279+0000 7f3330995700 1 -- 172.21.15.179:0/2137055081 shutdown_connections 2023-07-18T02:59:00.254 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:00.280+0000 7f3330995700 1 -- 172.21.15.179:0/2137055081 wait complete. 2023-07-18T02:59:00.255 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:00.280+0000 7f3330995700 1 Processor -- start 2023-07-18T02:59:00.255 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:00.280+0000 7f3330995700 1 -- start start 2023-07-18T02:59:00.255 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:00.280+0000 7f3330995700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f332c0ff560 0x7f332c195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:59:00.255 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:00.280+0000 7f3330995700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332c195eb0 con 0x7f332c0ff560 2023-07-18T02:59:00.255 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:00.281+0000 7f332a59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f332c0ff560 0x7f332c195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:59:00.255 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:00.281+0000 7f332a59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f332c0ff560 0x7f332c195970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.179:34392/0 (socket says 172.21.15.179:34392) 2023-07-18T02:59:00.256 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:00.281+0000 7f332a59c700 1 -- 172.21.15.179:0/1537339069 learned_addr learned my addr 172.21.15.179:0/1537339069 (peer_addr_for_me v2:172.21.15.179:0/0) 2023-07-18T02:59:00.256 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:00.281+0000 7f332a59c700 1 -- 172.21.15.179:0/1537339069 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f331c0095f0 con 0x7f332c0ff560 2023-07-18T02:59:00.256 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:00.281+0000 7f332a59c700 1 --2- 172.21.15.179:0/1537339069 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f332c0ff560 0x7f332c195970 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f331c0092e0 tx=0x7f331c004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:59:00.256 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:00.282+0000 7f33237fe700 1 -- 172.21.15.179:0/1537339069 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f331c01d070 con 0x7f332c0ff560 2023-07-18T02:59:00.256 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:00.282+0000 7f3330995700 1 -- 172.21.15.179:0/1537339069 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f332c196050 con 0x7f332c0ff560 2023-07-18T02:59:00.256 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:00.282+0000 7f33237fe700 1 -- 172.21.15.179:0/1537339069 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f331c00bc10 con 0x7f332c0ff560 2023-07-18T02:59:00.256 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:00.282+0000 7f33237fe700 1 -- 172.21.15.179:0/1537339069 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f331c003930 con 0x7f332c0ff560 2023-07-18T02:59:00.257 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:00.282+0000 7f3330995700 1 -- 172.21.15.179:0/1537339069 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f332c196470 con 0x7f332c0ff560 2023-07-18T02:59:00.257 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:00.282+0000 7f3330995700 1 -- 172.21.15.179:0/1537339069 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f332c103ac0 con 0x7f332c0ff560 2023-07-18T02:59:00.258 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:00.283+0000 7f33237fe700 1 -- 172.21.15.179:0/1537339069 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f331c003bb0 con 0x7f332c0ff560 2023-07-18T02:59:00.258 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:00.283+0000 7f33237fe700 1 --2- 172.21.15.179:0/1537339069 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f33180387b0 0x7f331803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:59:00.258 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:00.283+0000 7f33237fe700 1 -- 172.21.15.179:0/1537339069 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f331c04c460 con 0x7f332c0ff560 2023-07-18T02:59:00.258 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:00.283+0000 7f3329d9b700 1 --2- 172.21.15.179:0/1537339069 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f33180387b0 0x7f331803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:59:00.258 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:00.284+0000 7f3329d9b700 1 --2- 172.21.15.179:0/1537339069 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f33180387b0 0x7f331803ac60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3314006fd0 tx=0x7f3314006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:59:00.261 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:00.289+0000 7f33237fe700 1 -- 172.21.15.179:0/1537339069 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f331c00ba70 con 0x7f332c0ff560 2023-07-18T02:59:00.454 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:00.481+0000 7f3330995700 1 -- 172.21.15.179:0/1537339069 --> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi101:172.21.15.101=a;smithi155:172.21.15.155=b;smithi179:172.21.15.179=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f332c196720 con 0x7f33180387b0 2023-07-18T02:59:00.461 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:00.488+0000 7f33237fe700 1 -- 172.21.15.179:0/1537339069 <== mgr.14150 v2:172.21.15.101:6800/3963742842 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f332c196720 con 0x7f33180387b0 2023-07-18T02:59:00.461 INFO:teuthology.orchestra.run.smithi179.stdout:Scheduled mon update... 2023-07-18T02:59:00.463 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:00.490+0000 7f3330995700 1 -- 172.21.15.179:0/1537339069 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f33180387b0 msgr2=0x7f331803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:59:00.464 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:00.490+0000 7f3330995700 1 --2- 172.21.15.179:0/1537339069 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f33180387b0 0x7f331803ac60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3314006fd0 tx=0x7f3314006e40 comp rx=0 tx=0).stop 2023-07-18T02:59:00.464 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:00.490+0000 7f3330995700 1 -- 172.21.15.179:0/1537339069 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f332c0ff560 msgr2=0x7f332c195970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:59:00.464 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:00.490+0000 7f3330995700 1 --2- 172.21.15.179:0/1537339069 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f332c0ff560 0x7f332c195970 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f331c0092e0 tx=0x7f331c004ff0 comp rx=0 tx=0).stop 2023-07-18T02:59:00.464 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:00.490+0000 7f3330995700 1 -- 172.21.15.179:0/1537339069 shutdown_connections 2023-07-18T02:59:00.464 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:00.490+0000 7f3330995700 1 --2- 172.21.15.179:0/1537339069 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f33180387b0 0x7f331803ac60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:59:00.464 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:00.490+0000 7f3330995700 1 --2- 172.21.15.179:0/1537339069 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f332c0ff560 0x7f332c195970 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:59:00.464 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:00.490+0000 7f3330995700 1 -- 172.21.15.179:0/1537339069 >> 172.21.15.179:0/1537339069 conn(0x7f332c074f40 msgr2=0x7f332c0fbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:59:00.465 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:00.490+0000 7f3330995700 1 -- 172.21.15.179:0/1537339069 shutdown_connections 2023-07-18T02:59:00.465 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:00.490+0000 7f3330995700 1 -- 172.21.15.179:0/1537339069 wait complete. 2023-07-18T02:59:00.717 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:00 smithi101 ceph-mon[126379]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:00.859 DEBUG:teuthology.orchestra.run.smithi155:mon.b> sudo journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mon.b.service 2023-07-18T02:59:00.862 DEBUG:teuthology.orchestra.run.smithi179:mon.c> sudo journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mon.c.service 2023-07-18T02:59:00.865 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-07-18T02:59:00.866 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph mon dump -f json 2023-07-18T02:59:00.894 INFO:journalctl@ceph.mon.c.smithi179.stdout:-- Logs begin at Tue 2023-07-18 02:29:48 UTC. -- 2023-07-18T02:59:00.898 INFO:journalctl@ceph.mon.b.smithi155.stdout:-- Logs begin at Tue 2023-07-18 02:29:36 UTC. -- 2023-07-18T02:59:01.717 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:01 smithi101 ceph-mon[126379]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi101:172.21.15.101=a;smithi155:172.21.15.155=b;smithi179:172.21.15.179=c", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T02:59:01.717 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:01 smithi101 ceph-mon[126379]: Saving service mon spec with placement smithi101:172.21.15.101=a;smithi155:172.21.15.155=b;smithi179:172.21.15.179=c;count:3 2023-07-18T02:59:01.718 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:01 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:01.718 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:01 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T02:59:01.718 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:01 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:01.718 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:01 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T02:59:01.718 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:01 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:01.718 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:01 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-18T02:59:01.718 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:01 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:01.719 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:01 smithi101 ceph-mon[126379]: Deploying daemon mon.c on smithi179 2023-07-18T02:59:01.719 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:01 smithi101 ceph-mon[126379]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:01.800 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-18T02:59:03.443 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:03.469+0000 7f588e919700 1 -- 172.21.15.179:0/3365002576 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f58881006f0 msgr2=0x7f5888100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:59:03.443 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:03.469+0000 7f588e919700 1 --2- 172.21.15.179:0/3365002576 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f58881006f0 0x7f5888100ac0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5878009910 tx=0x7f5878009c20 comp rx=0 tx=0).stop 2023-07-18T02:59:03.444 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:03.469+0000 7f588e919700 1 -- 172.21.15.179:0/3365002576 shutdown_connections 2023-07-18T02:59:03.444 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:03.469+0000 7f588e919700 1 --2- 172.21.15.179:0/3365002576 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f58881006f0 0x7f5888100ac0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:59:03.444 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:03.469+0000 7f588e919700 1 -- 172.21.15.179:0/3365002576 >> 172.21.15.179:0/3365002576 conn(0x7f58880742b0 msgr2=0x7f58880746b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:59:03.444 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:03.469+0000 7f588e919700 1 -- 172.21.15.179:0/3365002576 shutdown_connections 2023-07-18T02:59:03.444 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:03.469+0000 7f588e919700 1 -- 172.21.15.179:0/3365002576 wait complete. 2023-07-18T02:59:03.444 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:03.470+0000 7f588e919700 1 Processor -- start 2023-07-18T02:59:03.445 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:03.470+0000 7f588e919700 1 -- start start 2023-07-18T02:59:03.445 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:03.470+0000 7f588e919700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f58881006f0 0x7f588819a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:59:03.445 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:03.470+0000 7f588e919700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588819add0 con 0x7f58881006f0 2023-07-18T02:59:03.446 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:03.471+0000 7f5887fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f58881006f0 0x7f588819a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:59:03.447 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:03.471+0000 7f5887fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f58881006f0 0x7f588819a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.179:34398/0 (socket says 172.21.15.179:34398) 2023-07-18T02:59:03.447 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:03.471+0000 7f5887fff700 1 -- 172.21.15.179:0/770475943 learned_addr learned my addr 172.21.15.179:0/770475943 (peer_addr_for_me v2:172.21.15.179:0/0) 2023-07-18T02:59:03.447 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:03.471+0000 7f5887fff700 1 -- 172.21.15.179:0/770475943 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58780095f0 con 0x7f58881006f0 2023-07-18T02:59:03.447 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:03.471+0000 7f5887fff700 1 --2- 172.21.15.179:0/770475943 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f58881006f0 0x7f588819a890 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5878006010 tx=0x7f5878004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:59:03.447 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:03.472+0000 7f58857fa700 1 -- 172.21.15.179:0/770475943 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f587801d070 con 0x7f58881006f0 2023-07-18T02:59:03.447 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:03.472+0000 7f58857fa700 1 -- 172.21.15.179:0/770475943 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58780044a0 con 0x7f58881006f0 2023-07-18T02:59:03.448 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:03.472+0000 7f588e919700 1 -- 172.21.15.179:0/770475943 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f588819afd0 con 0x7f58881006f0 2023-07-18T02:59:03.448 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:03.472+0000 7f58857fa700 1 -- 172.21.15.179:0/770475943 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5878003a70 con 0x7f58881006f0 2023-07-18T02:59:03.448 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:03.472+0000 7f588e919700 1 -- 172.21.15.179:0/770475943 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f588819b3f0 con 0x7f58881006f0 2023-07-18T02:59:03.448 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:03.473+0000 7f58857fa700 1 -- 172.21.15.179:0/770475943 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f5878003cb0 con 0x7f58881006f0 2023-07-18T02:59:03.448 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:03.473+0000 7f588e919700 1 -- 172.21.15.179:0/770475943 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5888061b40 con 0x7f58881006f0 2023-07-18T02:59:03.448 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:03.473+0000 7f58857fa700 1 --2- 172.21.15.179:0/770475943 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f58700387b0 0x7f587003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:59:03.449 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:03.473+0000 7f58857fa700 1 -- 172.21.15.179:0/770475943 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f587804d470 con 0x7f58881006f0 2023-07-18T02:59:03.449 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:03.473+0000 7f58877fe700 1 --2- 172.21.15.179:0/770475943 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f58700387b0 0x7f587003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:59:03.449 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:03.474+0000 7f58877fe700 1 --2- 172.21.15.179:0/770475943 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f58700387b0 0x7f587003ac60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f587c006fd0 tx=0x7f587c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:59:03.452 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:03.479+0000 7f58857fa700 1 -- 172.21.15.179:0/770475943 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5878029950 con 0x7f58881006f0 2023-07-18T02:59:03.740 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:03.767+0000 7f588e919700 1 -- 172.21.15.179:0/770475943 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f5888195e20 con 0x7f58881006f0 2023-07-18T02:59:03.740 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:03.767+0000 7f58857fa700 1 -- 172.21.15.179:0/770475943 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+740 (secure 0 0 0) 0x7f5878026070 con 0x7f58881006f0 2023-07-18T02:59:03.740 INFO:teuthology.orchestra.run.smithi179.stdout: 2023-07-18T02:59:03.741 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":1,"fsid":"b1782daa-2516-11ee-9b34-001a4aab830c","modified":"2023-07-18T02:57:19.585180Z","created":"2023-07-18T02:57:19.585180Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:3300","nonce":0},{"type":"v1","addr":"172.21.15.101:6789","nonce":0}]},"addr":"172.21.15.101:6789/0","public_addr":"172.21.15.101:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-07-18T02:59:03.742 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:03.769+0000 7f588e919700 1 -- 172.21.15.179:0/770475943 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f58700387b0 msgr2=0x7f587003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:59:03.743 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:03.769+0000 7f588e919700 1 --2- 172.21.15.179:0/770475943 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f58700387b0 0x7f587003ac60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f587c006fd0 tx=0x7f587c006e40 comp rx=0 tx=0).stop 2023-07-18T02:59:03.743 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:03.769+0000 7f588e919700 1 -- 172.21.15.179:0/770475943 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f58881006f0 msgr2=0x7f588819a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:59:03.743 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:03.769+0000 7f588e919700 1 --2- 172.21.15.179:0/770475943 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f58881006f0 0x7f588819a890 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5878006010 tx=0x7f5878004ff0 comp rx=0 tx=0).stop 2023-07-18T02:59:03.743 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:03.770+0000 7f588e919700 1 -- 172.21.15.179:0/770475943 shutdown_connections 2023-07-18T02:59:03.743 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:03.770+0000 7f588e919700 1 --2- 172.21.15.179:0/770475943 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f58700387b0 0x7f587003ac60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:59:03.743 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:03.770+0000 7f588e919700 1 --2- 172.21.15.179:0/770475943 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f58881006f0 0x7f588819a890 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:59:03.744 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:03.770+0000 7f588e919700 1 -- 172.21.15.179:0/770475943 >> 172.21.15.179:0/770475943 conn(0x7f58880742b0 msgr2=0x7f5888103f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:59:03.744 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:03.770+0000 7f588e919700 1 -- 172.21.15.179:0/770475943 shutdown_connections 2023-07-18T02:59:03.744 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:03.770+0000 7f588e919700 1 -- 172.21.15.179:0/770475943 wait complete. 2023-07-18T02:59:03.744 INFO:teuthology.orchestra.run.smithi179.stderr:dumped monmap epoch 1 2023-07-18T02:59:03.861 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-18T02:59:03.861 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-18T02:59:03.861 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-18T02:59:04.125 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:04.125 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:04.125 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:04.125 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:04.125 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:04.125 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi155", "name": "osd_memory_target"}]: dispatch 2023-07-18T02:59:04.126 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:04.126 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T02:59:04.126 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: Updating smithi155:/etc/ceph/ceph.conf 2023-07-18T02:59:04.126 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:04.126 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:04.126 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:04.126 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: Updating smithi155:/var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-18T02:59:04.127 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2023-07-18T02:59:04.127 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:04.127 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: Updating smithi155:/var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-18T02:59:04.127 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi179", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T02:59:04.127 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: Deploying cephadm binary to smithi179 2023-07-18T02:59:04.127 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:04.127 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T02:59:04.128 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:04.128 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: Added host smithi179 2023-07-18T02:59:04.128 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:04.128 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:04.128 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:04.128 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:04.128 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T02:59:04.129 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:04.129 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:04.129 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:04.129 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:04.129 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:04.129 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi179", "name": "osd_memory_target"}]: dispatch 2023-07-18T02:59:04.129 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:04.129 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T02:59:04.130 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: Updating smithi179:/etc/ceph/ceph.conf 2023-07-18T02:59:04.130 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:04.130 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:04.130 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:04.130 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1472864393' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-07-18T02:59:04.130 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: Updating smithi179:/var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-18T02:59:04.130 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: Updating smithi179:/etc/ceph/ceph.client.admin.keyring 2023-07-18T02:59:04.131 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:04.131 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: Updating smithi179:/var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-18T02:59:04.131 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1472864393' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-07-18T02:59:04.131 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: osdmap e4: 0 total, 0 up, 0 in 2023-07-18T02:59:04.131 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:04.131 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi101:172.21.15.101=a;smithi155:172.21.15.155=b;smithi179:172.21.15.179=c", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T02:59:04.131 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: Saving service mon spec with placement smithi101:172.21.15.101=a;smithi155:172.21.15.155=b;smithi179:172.21.15.179=c;count:3 2023-07-18T02:59:04.131 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:04.132 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T02:59:04.132 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:04.132 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T02:59:04.132 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:04.132 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-18T02:59:04.132 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:04.132 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: Deploying daemon mon.c on smithi179 2023-07-18T02:59:04.133 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:04.133 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-07-18T02:59:04.133 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: expand_channel_meta expand map: {default=false} 2023-07-18T02:59:04.133 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: expand_channel_meta from 'false' to 'false' 2023-07-18T02:59:04.133 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: expand_channel_meta expanded map: {default=false} 2023-07-18T02:59:04.133 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: expand_channel_meta expand map: {default=info} 2023-07-18T02:59:04.133 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: expand_channel_meta from 'info' to 'info' 2023-07-18T02:59:04.133 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: expand_channel_meta expanded map: {default=info} 2023-07-18T02:59:04.134 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: expand_channel_meta expand map: {default=daemon} 2023-07-18T02:59:04.134 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: expand_channel_meta from 'daemon' to 'daemon' 2023-07-18T02:59:04.134 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: expand_channel_meta expanded map: {default=daemon} 2023-07-18T02:59:04.134 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: expand_channel_meta expand map: {default=debug} 2023-07-18T02:59:04.134 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: expand_channel_meta from 'debug' to 'debug' 2023-07-18T02:59:04.134 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: expand_channel_meta expanded map: {default=debug} 2023-07-18T02:59:04.134 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:03 smithi179 ceph-mon[129229]: mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2023-07-18T02:59:04.135 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:04 smithi179 bash[129185]: 14d3b3560bbbf6c02777a88089ae3b9040aabe57d312f737797376af3b291dde 2023-07-18T02:59:04.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:04 smithi179 systemd[1]: Started Ceph mon.c for b1782daa-2516-11ee-9b34-001a4aab830c. 2023-07-18T02:59:05.175 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-07-18T02:59:05.176 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph mon dump -f json 2023-07-18T02:59:05.737 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.c/config 2023-07-18T02:59:09.217 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:08 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-18T02:59:09.217 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:08 smithi101 ceph-mon[126379]: mon.a calling monitor election 2023-07-18T02:59:09.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:08 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T02:59:09.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:08 smithi101 ceph-mon[126379]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:09.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:08 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T02:59:09.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:08 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T02:59:09.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:08 smithi101 ceph-mon[126379]: mon.c calling monitor election 2023-07-18T02:59:09.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:08 smithi101 ceph-mon[126379]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:09.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:08 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T02:59:09.219 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:08 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T02:59:09.219 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:08 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T02:59:09.219 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:08 smithi101 ceph-mon[126379]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-18T02:59:09.219 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:08 smithi101 ceph-mon[126379]: monmap e2: 2 mons at {a=[v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0],c=[v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0]} removed_ranks: {} 2023-07-18T02:59:09.219 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:08 smithi101 ceph-mon[126379]: fsmap 2023-07-18T02:59:09.219 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:08 smithi101 ceph-mon[126379]: osdmap e4: 0 total, 0 up, 0 in 2023-07-18T02:59:09.219 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:08 smithi101 ceph-mon[126379]: mgrmap e13: x(active, since 46s) 2023-07-18T02:59:09.220 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:08 smithi101 ceph-mon[126379]: overall HEALTH_OK 2023-07-18T02:59:09.220 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:08 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:09.220 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:08 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:09.220 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:08 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:09.220 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:08 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-18T02:59:09.220 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:08 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:09.429 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:08 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-18T02:59:09.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:08 smithi179 ceph-mon[129229]: mon.a calling monitor election 2023-07-18T02:59:09.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:08 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T02:59:09.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:08 smithi179 ceph-mon[129229]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:09.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:08 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T02:59:09.431 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:08 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T02:59:09.431 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:08 smithi179 ceph-mon[129229]: mon.c calling monitor election 2023-07-18T02:59:09.431 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:08 smithi179 ceph-mon[129229]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:09.431 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:08 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T02:59:09.431 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:08 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T02:59:09.431 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:08 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T02:59:09.431 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:08 smithi179 ceph-mon[129229]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-18T02:59:09.432 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:08 smithi179 ceph-mon[129229]: monmap e2: 2 mons at {a=[v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0],c=[v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0]} removed_ranks: {} 2023-07-18T02:59:09.432 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:08 smithi179 ceph-mon[129229]: fsmap 2023-07-18T02:59:09.432 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:08 smithi179 ceph-mon[129229]: osdmap e4: 0 total, 0 up, 0 in 2023-07-18T02:59:09.432 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:08 smithi179 ceph-mon[129229]: mgrmap e13: x(active, since 46s) 2023-07-18T02:59:09.432 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:08 smithi179 ceph-mon[129229]: overall HEALTH_OK 2023-07-18T02:59:09.432 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:08 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:09.432 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:08 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:09.433 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:08 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:09.433 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:08 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-18T02:59:09.433 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:08 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:09.943 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:09.969+0000 7f2ee389b700 1 -- 172.21.15.179:0/761152282 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2ec80057d0 msgr2=0x7f2ec8007c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:59:09.943 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:09.969+0000 7f2ee389b700 1 --2- 172.21.15.179:0/761152282 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2ec80057d0 0x7f2ec8007c50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2edc0fe570 tx=0x7f2ecc00feb0 comp rx=0 tx=0).stop 2023-07-18T02:59:09.943 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:09.969+0000 7f2ee389b700 1 -- 172.21.15.179:0/761152282 shutdown_connections 2023-07-18T02:59:09.943 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:09.969+0000 7f2ee389b700 1 --2- 172.21.15.179:0/761152282 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2ec80057d0 0x7f2ec8007c50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:59:09.943 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:09.969+0000 7f2ee389b700 1 -- 172.21.15.179:0/761152282 >> 172.21.15.179:0/761152282 conn(0x7f2edc0fa160 msgr2=0x7f2edc0fc570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:59:09.944 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:09.970+0000 7f2ee389b700 1 -- 172.21.15.179:0/761152282 shutdown_connections 2023-07-18T02:59:09.944 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:09.970+0000 7f2ee389b700 1 -- 172.21.15.179:0/761152282 wait complete. 2023-07-18T02:59:09.945 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:09.970+0000 7f2ee389b700 1 Processor -- start 2023-07-18T02:59:09.945 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:09.970+0000 7f2ee389b700 1 -- start start 2023-07-18T02:59:09.945 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:09.971+0000 7f2ee389b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2ec80057d0 0x7f2edc192950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:59:09.946 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:09.971+0000 7f2ee389b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2edc192e90 0x7f2edc197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:59:09.946 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:09.971+0000 7f2ee389b700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2edc193410 con 0x7f2ec80057d0 2023-07-18T02:59:09.946 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:09.971+0000 7f2ee389b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2edc193550 con 0x7f2edc192e90 2023-07-18T02:59:09.946 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:09.971+0000 7f2ee0e36700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2edc192e90 0x7f2edc197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:59:09.946 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:09.971+0000 7f2ee0e36700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2edc192e90 0x7f2edc197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48124/0 (socket says 172.21.15.179:48124) 2023-07-18T02:59:09.946 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:09.971+0000 7f2ee0e36700 1 -- 172.21.15.179:0/3273816216 learned_addr learned my addr 172.21.15.179:0/3273816216 (peer_addr_for_me v2:172.21.15.179:0/0) 2023-07-18T02:59:09.946 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:09.971+0000 7f2ee0e36700 1 -- 172.21.15.179:0/3273816216 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2edc192e90 msgr2=0x7f2edc197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-07-18T02:59:09.947 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:09.971+0000 7f2ee0e36700 1 -- 172.21.15.179:0/3273816216 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2edc192e90 msgr2=0x7f2edc197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-07-18T02:59:09.947 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:09.971+0000 7f2ee0e36700 1 --2- 172.21.15.179:0/3273816216 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2edc192e90 0x7f2edc197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-18T02:59:09.947 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:09.971+0000 7f2ee1637700 1 --2- 172.21.15.179:0/3273816216 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2ec80057d0 0x7f2edc192950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:59:09.947 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:09.971+0000 7f2ee0e36700 1 --2- 172.21.15.179:0/3273816216 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2edc192e90 0x7f2edc197330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-18T02:59:09.947 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:09.971+0000 7f2ee1637700 1 -- 172.21.15.179:0/3273816216 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2edc192e90 msgr2=0x7f2edc197330 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-18T02:59:09.947 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:09.971+0000 7f2ee1637700 1 --2- 172.21.15.179:0/3273816216 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2edc192e90 0x7f2edc197330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:59:09.947 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:09.971+0000 7f2ee1637700 1 -- 172.21.15.179:0/3273816216 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ecc00f900 con 0x7f2ec80057d0 2023-07-18T02:59:09.948 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:09.972+0000 7f2ee1637700 1 --2- 172.21.15.179:0/3273816216 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2ec80057d0 0x7f2edc192950 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc000c00 tx=0x7f2ecc0047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:59:09.948 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:09.972+0000 7f2ed27fc700 1 -- 172.21.15.179:0/3273816216 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f2ecc013b00 con 0x7f2ec80057d0 2023-07-18T02:59:09.948 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:09.972+0000 7f2ed27fc700 1 -- 172.21.15.179:0/3273816216 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ecc0044f0 con 0x7f2ec80057d0 2023-07-18T02:59:09.948 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:09.972+0000 7f2ee389b700 1 -- 172.21.15.179:0/3273816216 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2edc197870 con 0x7f2ec80057d0 2023-07-18T02:59:09.948 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:09.972+0000 7f2ed27fc700 1 -- 172.21.15.179:0/3273816216 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f2ecc003be0 con 0x7f2ec80057d0 2023-07-18T02:59:09.948 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:09.972+0000 7f2ee389b700 1 -- 172.21.15.179:0/3273816216 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2edc197e40 con 0x7f2ec80057d0 2023-07-18T02:59:09.949 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:09.973+0000 7f2ed27fc700 1 -- 172.21.15.179:0/3273816216 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f2ecc011540 con 0x7f2ec80057d0 2023-07-18T02:59:09.949 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:09.973+0000 7f2ee389b700 1 -- 172.21.15.179:0/3273816216 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2edc061b40 con 0x7f2ec80057d0 2023-07-18T02:59:09.949 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:09.973+0000 7f2ed27fc700 1 --2- 172.21.15.179:0/3273816216 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f2ec00385f0 0x7f2ec003aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:59:09.949 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:09.973+0000 7f2ed27fc700 1 -- 172.21.15.179:0/3273816216 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2ecc04dc30 con 0x7f2ec80057d0 2023-07-18T02:59:09.950 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:09.974+0000 7f2ee0e36700 1 --2- 172.21.15.179:0/3273816216 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f2ec00385f0 0x7f2ec003aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:59:09.950 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:09.974+0000 7f2ee0e36700 1 --2- 172.21.15.179:0/3273816216 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f2ec00385f0 0x7f2ec003aaa0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8009180 tx=0x7f2ed8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:59:09.952 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:09.979+0000 7f2ed27fc700 1 -- 172.21.15.179:0/3273816216 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2ecc024400 con 0x7f2ec80057d0 2023-07-18T02:59:10.216 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:10.243+0000 7f2ee389b700 1 -- 172.21.15.179:0/3273816216 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2edc193df0 con 0x7f2ec80057d0 2023-07-18T02:59:10.216 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:10.243+0000 7f2ed27fc700 1 -- 172.21.15.179:0/3273816216 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1010 (secure 0 0 0) 0x7f2ecc023360 con 0x7f2ec80057d0 2023-07-18T02:59:10.217 INFO:teuthology.orchestra.run.smithi179.stdout: 2023-07-18T02:59:10.217 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":2,"fsid":"b1782daa-2516-11ee-9b34-001a4aab830c","modified":"2023-07-18T02:59:03.897470Z","created":"2023-07-18T02:57:19.585180Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:3300","nonce":0},{"type":"v1","addr":"172.21.15.101:6789","nonce":0}]},"addr":"172.21.15.101:6789/0","public_addr":"172.21.15.101:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-07-18T02:59:10.217 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:09 smithi101 ceph-mon[126379]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:10.217 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:09 smithi101 ceph-mon[126379]: Deploying daemon mon.b on smithi155 2023-07-18T02:59:10.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:09 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T02:59:10.218 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:10.245+0000 7f2ee389b700 1 -- 172.21.15.179:0/3273816216 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f2ec00385f0 msgr2=0x7f2ec003aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:59:10.218 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:10.245+0000 7f2ee389b700 1 --2- 172.21.15.179:0/3273816216 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f2ec00385f0 0x7f2ec003aaa0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8009180 tx=0x7f2ed8008040 comp rx=0 tx=0).stop 2023-07-18T02:59:10.219 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:10.245+0000 7f2ee389b700 1 -- 172.21.15.179:0/3273816216 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2ec80057d0 msgr2=0x7f2edc192950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:59:10.219 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:10.245+0000 7f2ee389b700 1 --2- 172.21.15.179:0/3273816216 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2ec80057d0 0x7f2edc192950 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc000c00 tx=0x7f2ecc0047a0 comp rx=0 tx=0).stop 2023-07-18T02:59:10.219 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:10.245+0000 7f2ee389b700 1 -- 172.21.15.179:0/3273816216 shutdown_connections 2023-07-18T02:59:10.219 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:10.245+0000 7f2ee389b700 1 --2- 172.21.15.179:0/3273816216 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f2ec00385f0 0x7f2ec003aaa0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:59:10.219 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:10.245+0000 7f2ee389b700 1 --2- 172.21.15.179:0/3273816216 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2ec80057d0 0x7f2edc192950 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:59:10.219 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:10.245+0000 7f2ee389b700 1 --2- 172.21.15.179:0/3273816216 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2edc192e90 0x7f2edc197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:59:10.219 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:10.245+0000 7f2ee389b700 1 -- 172.21.15.179:0/3273816216 >> 172.21.15.179:0/3273816216 conn(0x7f2edc0fa160 msgr2=0x7f2edc18b560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:59:10.219 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:10.245+0000 7f2ee389b700 1 -- 172.21.15.179:0/3273816216 shutdown_connections 2023-07-18T02:59:10.220 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:10.245+0000 7f2ee389b700 1 -- 172.21.15.179:0/3273816216 wait complete. 2023-07-18T02:59:10.220 INFO:teuthology.orchestra.run.smithi179.stderr:dumped monmap epoch 2 2023-07-18T02:59:10.241 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:09 smithi179 ceph-mon[129229]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:10.242 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:09 smithi179 ceph-mon[129229]: Deploying daemon mon.b on smithi155 2023-07-18T02:59:10.242 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:09 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T02:59:11.217 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:10 smithi101 ceph-mon[126379]: from='client.? 172.21.15.179:0/3273816216' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-18T02:59:11.217 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:59:10 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:59:10.896+0000 7f2a3cd80700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-07-18T02:59:11.429 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:10 smithi179 ceph-mon[129229]: from='client.? 172.21.15.179:0/3273816216' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-18T02:59:11.597 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-07-18T02:59:11.598 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph mon dump -f json 2023-07-18T02:59:11.775 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:11 smithi155 systemd[1]: Starting Ceph mon.b for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T02:59:12.127 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:11 smithi155 podman[128845]: 2023-07-18T02:59:12.127 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: set uid:gid to 167:167 (ceph:ceph) 2023-07-18T02:59:12.128 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: ceph version 18.0.0-4746-g81cf785a (81cf785aa7781c4e08381c540f890cb9bc987ef9) reef (dev), process ceph-mon, pid 7 2023-07-18T02:59:12.128 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: pidfile_write: ignore empty --pid-file 2023-07-18T02:59:12.128 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 bash[128845]: 797e34671e86b9f400a3b3774109ff88ce8188c3af4ab3c791a9631b6a5158b3 2023-07-18T02:59:12.128 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: load: jerasure load: lrc 2023-07-18T02:59:12.162 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.c/config 2023-07-18T02:59:12.217 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:11 smithi101 ceph-mon[126379]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:12.395 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:11 smithi179 ceph-mon[129229]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:12.525 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: RocksDB version: 7.9.2 2023-07-18T02:59:12.525 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Git sha 0 2023-07-18T02:59:12.525 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Compile date 2023-07-17 23:00:14 2023-07-18T02:59:12.525 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: DB SUMMARY 2023-07-18T02:59:12.525 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: DB Session ID: RM69DM3A92QQNHX42G9P 2023-07-18T02:59:12.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: CURRENT file: CURRENT 2023-07-18T02:59:12.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: IDENTITY file: IDENTITY 2023-07-18T02:59:12.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-07-18T02:59:12.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-07-18T02:59:12.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-07-18T02:59:12.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.error_if_exists: 0 2023-07-18T02:59:12.527 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.create_if_missing: 0 2023-07-18T02:59:12.527 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.paranoid_checks: 1 2023-07-18T02:59:12.527 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.flush_verify_memtable_count: 1 2023-07-18T02:59:12.527 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-07-18T02:59:12.527 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-07-18T02:59:12.527 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.env: 0x56171e5eb720 2023-07-18T02:59:12.527 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.fs: PosixFileSystem 2023-07-18T02:59:12.527 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.info_log: 0x56171f65f280 2023-07-18T02:59:12.528 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.max_file_opening_threads: 16 2023-07-18T02:59:12.528 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.statistics: (nil) 2023-07-18T02:59:12.528 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.use_fsync: 0 2023-07-18T02:59:12.528 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.max_log_file_size: 0 2023-07-18T02:59:12.528 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-07-18T02:59:12.528 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.log_file_time_to_roll: 0 2023-07-18T02:59:12.528 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.keep_log_file_num: 1000 2023-07-18T02:59:12.529 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.recycle_log_file_num: 0 2023-07-18T02:59:12.529 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.allow_fallocate: 1 2023-07-18T02:59:12.529 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.allow_mmap_reads: 0 2023-07-18T02:59:12.529 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.allow_mmap_writes: 0 2023-07-18T02:59:12.529 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.use_direct_reads: 0 2023-07-18T02:59:12.529 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-07-18T02:59:12.529 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.create_missing_column_families: 0 2023-07-18T02:59:12.530 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.db_log_dir: 2023-07-18T02:59:12.530 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.wal_dir: 2023-07-18T02:59:12.530 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.table_cache_numshardbits: 6 2023-07-18T02:59:12.530 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.WAL_ttl_seconds: 0 2023-07-18T02:59:12.530 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.WAL_size_limit_MB: 0 2023-07-18T02:59:12.530 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-07-18T02:59:12.530 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-07-18T02:59:12.530 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.is_fd_close_on_exec: 1 2023-07-18T02:59:12.531 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.advise_random_on_open: 1 2023-07-18T02:59:12.531 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.db_write_buffer_size: 0 2023-07-18T02:59:12.531 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.write_buffer_manager: 0x56171f5803c0 2023-07-18T02:59:12.531 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-07-18T02:59:12.531 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-07-18T02:59:12.531 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.use_adaptive_mutex: 0 2023-07-18T02:59:12.531 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.rate_limiter: (nil) 2023-07-18T02:59:12.532 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-07-18T02:59:12.532 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.wal_recovery_mode: 2 2023-07-18T02:59:12.532 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.enable_thread_tracking: 0 2023-07-18T02:59:12.532 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.enable_pipelined_write: 0 2023-07-18T02:59:12.532 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.unordered_write: 0 2023-07-18T02:59:12.532 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-07-18T02:59:12.532 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-07-18T02:59:12.533 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-07-18T02:59:12.533 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-07-18T02:59:12.535 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.row_cache: None 2023-07-18T02:59:12.535 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.wal_filter: None 2023-07-18T02:59:12.535 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-07-18T02:59:12.535 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.allow_ingest_behind: 0 2023-07-18T02:59:12.535 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.two_write_queues: 0 2023-07-18T02:59:12.536 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.manual_wal_flush: 0 2023-07-18T02:59:12.536 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.wal_compression: 0 2023-07-18T02:59:12.536 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.atomic_flush: 0 2023-07-18T02:59:12.536 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-07-18T02:59:12.536 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.persist_stats_to_disk: 0 2023-07-18T02:59:12.536 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.write_dbid_to_manifest: 0 2023-07-18T02:59:12.536 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.log_readahead_size: 0 2023-07-18T02:59:12.536 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-07-18T02:59:12.537 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.best_efforts_recovery: 0 2023-07-18T02:59:12.537 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-07-18T02:59:12.537 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-07-18T02:59:12.537 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.allow_data_in_errors: 0 2023-07-18T02:59:12.537 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.db_host_id: __hostname__ 2023-07-18T02:59:12.537 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.enforce_single_del_contracts: true 2023-07-18T02:59:12.537 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.max_background_jobs: 2 2023-07-18T02:59:12.538 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.max_background_compactions: -1 2023-07-18T02:59:12.538 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.max_subcompactions: 1 2023-07-18T02:59:12.539 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-07-18T02:59:12.539 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-07-18T02:59:12.539 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.delayed_write_rate : 16777216 2023-07-18T02:59:12.540 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.max_total_wal_size: 0 2023-07-18T02:59:12.540 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-07-18T02:59:12.540 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.stats_dump_period_sec: 600 2023-07-18T02:59:12.540 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.stats_persist_period_sec: 600 2023-07-18T02:59:12.540 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-07-18T02:59:12.540 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.max_open_files: -1 2023-07-18T02:59:12.540 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.bytes_per_sync: 0 2023-07-18T02:59:12.541 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.wal_bytes_per_sync: 0 2023-07-18T02:59:12.541 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.strict_bytes_per_sync: 0 2023-07-18T02:59:12.541 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.compaction_readahead_size: 0 2023-07-18T02:59:12.541 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.max_background_flushes: -1 2023-07-18T02:59:12.541 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Compression algorithms supported: 2023-07-18T02:59:12.541 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-07-18T02:59:12.541 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: kZSTD supported: 0 2023-07-18T02:59:12.542 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: kXpressCompression supported: 0 2023-07-18T02:59:12.542 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: kLZ4HCCompression supported: 1 2023-07-18T02:59:12.542 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: kZlibCompression supported: 1 2023-07-18T02:59:12.542 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: kSnappyCompression supported: 1 2023-07-18T02:59:12.542 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: kLZ4Compression supported: 1 2023-07-18T02:59:12.542 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: kBZip2Compression supported: 0 2023-07-18T02:59:12.542 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Fast CRC32 supported: Supported on x86 2023-07-18T02:59:12.542 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: DMutex implementation: pthread_mutex_t 2023-07-18T02:59:12.543 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-07-18T02:59:12.543 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-07-18T02:59:12.543 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-07-18T02:59:12.543 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.merge_operator: 2023-07-18T02:59:12.543 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.compaction_filter: None 2023-07-18T02:59:12.543 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.compaction_filter_factory: None 2023-07-18T02:59:12.543 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.sst_partitioner_factory: None 2023-07-18T02:59:12.544 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.memtable_factory: SkipListFactory 2023-07-18T02:59:12.544 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.table_factory: BlockBasedTable 2023-07-18T02:59:12.544 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56171f65f380) 2023-07-18T02:59:12.544 INFO:journalctl@ceph.mon.b.smithi155.stdout: cache_index_and_filter_blocks: 1 2023-07-18T02:59:12.544 INFO:journalctl@ceph.mon.b.smithi155.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-07-18T02:59:12.544 INFO:journalctl@ceph.mon.b.smithi155.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-07-18T02:59:12.544 INFO:journalctl@ceph.mon.b.smithi155.stdout: pin_top_level_index_and_filter: 1 2023-07-18T02:59:12.545 INFO:journalctl@ceph.mon.b.smithi155.stdout: index_type: 0 2023-07-18T02:59:12.545 INFO:journalctl@ceph.mon.b.smithi155.stdout: data_block_index_type: 0 2023-07-18T02:59:12.545 INFO:journalctl@ceph.mon.b.smithi155.stdout: index_shortening: 1 2023-07-18T02:59:12.545 INFO:journalctl@ceph.mon.b.smithi155.stdout: data_block_hash_table_util_ratio: 0.750000 2023-07-18T02:59:12.545 INFO:journalctl@ceph.mon.b.smithi155.stdout: checksum: 4 2023-07-18T02:59:12.545 INFO:journalctl@ceph.mon.b.smithi155.stdout: no_block_cache: 0 2023-07-18T02:59:12.546 INFO:journalctl@ceph.mon.b.smithi155.stdout: block_cache: 0x56171f61f1f0 2023-07-18T02:59:12.546 INFO:journalctl@ceph.mon.b.smithi155.stdout: block_cache_name: BinnedLRUCache 2023-07-18T02:59:12.546 INFO:journalctl@ceph.mon.b.smithi155.stdout: block_cache_options: 2023-07-18T02:59:12.546 INFO:journalctl@ceph.mon.b.smithi155.stdout: capacity : 536870912 2023-07-18T02:59:12.546 INFO:journalctl@ceph.mon.b.smithi155.stdout: num_shard_bits : 4 2023-07-18T02:59:12.546 INFO:journalctl@ceph.mon.b.smithi155.stdout: strict_capacity_limit : 0 2023-07-18T02:59:12.546 INFO:journalctl@ceph.mon.b.smithi155.stdout: high_pri_pool_ratio: 0.000 2023-07-18T02:59:12.547 INFO:journalctl@ceph.mon.b.smithi155.stdout: block_cache_compressed: (nil) 2023-07-18T02:59:12.547 INFO:journalctl@ceph.mon.b.smithi155.stdout: persistent_cache: (nil) 2023-07-18T02:59:12.547 INFO:journalctl@ceph.mon.b.smithi155.stdout: block_size: 4096 2023-07-18T02:59:12.547 INFO:journalctl@ceph.mon.b.smithi155.stdout: block_size_deviation: 10 2023-07-18T02:59:12.547 INFO:journalctl@ceph.mon.b.smithi155.stdout: block_restart_interval: 16 2023-07-18T02:59:12.547 INFO:journalctl@ceph.mon.b.smithi155.stdout: index_block_restart_interval: 1 2023-07-18T02:59:12.547 INFO:journalctl@ceph.mon.b.smithi155.stdout: metadata_block_size: 4096 2023-07-18T02:59:12.547 INFO:journalctl@ceph.mon.b.smithi155.stdout: partition_filters: 0 2023-07-18T02:59:12.548 INFO:journalctl@ceph.mon.b.smithi155.stdout: use_delta_encoding: 1 2023-07-18T02:59:12.548 INFO:journalctl@ceph.mon.b.smithi155.stdout: filter_policy: bloomfilter 2023-07-18T02:59:12.548 INFO:journalctl@ceph.mon.b.smithi155.stdout: whole_key_filtering: 1 2023-07-18T02:59:12.548 INFO:journalctl@ceph.mon.b.smithi155.stdout: verify_compression: 0 2023-07-18T02:59:12.548 INFO:journalctl@ceph.mon.b.smithi155.stdout: read_amp_bytes_per_bit: 0 2023-07-18T02:59:12.548 INFO:journalctl@ceph.mon.b.smithi155.stdout: format_version: 5 2023-07-18T02:59:12.548 INFO:journalctl@ceph.mon.b.smithi155.stdout: enable_index_compression: 1 2023-07-18T02:59:12.549 INFO:journalctl@ceph.mon.b.smithi155.stdout: block_align: 0 2023-07-18T02:59:12.549 INFO:journalctl@ceph.mon.b.smithi155.stdout: max_auto_readahead_size: 262144 2023-07-18T02:59:12.549 INFO:journalctl@ceph.mon.b.smithi155.stdout: prepopulate_block_cache: 0 2023-07-18T02:59:12.549 INFO:journalctl@ceph.mon.b.smithi155.stdout: initial_auto_readahead_size: 8192 2023-07-18T02:59:12.549 INFO:journalctl@ceph.mon.b.smithi155.stdout: num_file_reads_for_auto_readahead: 2 2023-07-18T02:59:12.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.write_buffer_size: 33554432 2023-07-18T02:59:12.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.max_write_buffer_number: 2 2023-07-18T02:59:12.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.compression: NoCompression 2023-07-18T02:59:12.550 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.bottommost_compression: Disabled 2023-07-18T02:59:12.550 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.prefix_extractor: nullptr 2023-07-18T02:59:12.550 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-07-18T02:59:12.550 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.num_levels: 7 2023-07-18T02:59:12.550 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-07-18T02:59:12.550 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-07-18T02:59:12.550 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-07-18T02:59:12.550 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-07-18T02:59:12.551 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-07-18T02:59:12.551 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-07-18T02:59:12.551 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-07-18T02:59:12.551 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-07-18T02:59:12.551 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-07-18T02:59:12.551 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-07-18T02:59:12.551 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-07-18T02:59:12.552 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-07-18T02:59:12.552 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.compression_opts.window_bits: -14 2023-07-18T02:59:12.552 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.compression_opts.level: 32767 2023-07-18T02:59:12.552 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.compression_opts.strategy: 0 2023-07-18T02:59:12.552 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-07-18T02:59:12.552 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-07-18T02:59:12.552 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-07-18T02:59:12.553 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-07-18T02:59:12.553 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.compression_opts.enabled: false 2023-07-18T02:59:12.553 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-07-18T02:59:12.553 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-07-18T02:59:12.553 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-07-18T02:59:12.553 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-07-18T02:59:12.553 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.target_file_size_base: 67108864 2023-07-18T02:59:12.553 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.target_file_size_multiplier: 1 2023-07-18T02:59:12.554 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-07-18T02:59:12.554 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-07-18T02:59:12.554 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-07-18T02:59:12.554 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-07-18T02:59:12.554 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-07-18T02:59:12.554 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-07-18T02:59:12.554 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-07-18T02:59:12.554 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-07-18T02:59:12.555 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-07-18T02:59:12.555 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-07-18T02:59:12.555 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-07-18T02:59:12.555 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-07-18T02:59:12.555 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-07-18T02:59:12.555 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.arena_block_size: 1048576 2023-07-18T02:59:12.555 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-07-18T02:59:12.556 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-07-18T02:59:12.556 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.disable_auto_compactions: 0 2023-07-18T02:59:12.556 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-07-18T02:59:12.556 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-07-18T02:59:12.556 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-07-18T02:59:12.556 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-07-18T02:59:12.556 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-07-18T02:59:12.556 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-07-18T02:59:12.557 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-07-18T02:59:12.557 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-07-18T02:59:12.557 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-07-18T02:59:12.557 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-07-18T02:59:12.557 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.table_properties_collectors: 2023-07-18T02:59:12.557 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.inplace_update_support: 0 2023-07-18T02:59:12.557 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.inplace_update_num_locks: 10000 2023-07-18T02:59:12.557 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-07-18T02:59:12.558 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-07-18T02:59:12.558 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.memtable_huge_page_size: 0 2023-07-18T02:59:12.558 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.bloom_locality: 0 2023-07-18T02:59:12.558 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.max_successive_merges: 0 2023-07-18T02:59:12.558 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.optimize_filters_for_hits: 0 2023-07-18T02:59:12.559 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.paranoid_file_checks: 0 2023-07-18T02:59:12.559 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.force_consistency_checks: 1 2023-07-18T02:59:12.559 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.report_bg_io_stats: 0 2023-07-18T02:59:12.559 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.ttl: 2592000 2023-07-18T02:59:12.559 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.periodic_compaction_seconds: 0 2023-07-18T02:59:12.559 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-07-18T02:59:12.559 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-07-18T02:59:12.560 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.enable_blob_files: false 2023-07-18T02:59:12.560 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.min_blob_size: 0 2023-07-18T02:59:12.560 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.blob_file_size: 268435456 2023-07-18T02:59:12.560 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.blob_compression_type: NoCompression 2023-07-18T02:59:12.560 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.enable_blob_garbage_collection: false 2023-07-18T02:59:12.560 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-07-18T02:59:12.560 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-07-18T02:59:12.560 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-07-18T02:59:12.561 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.blob_file_starting_level: 0 2023-07-18T02:59:12.561 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-07-18T02:59:12.561 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-07-18T02:59:12.561 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-07-18T02:59:12.561 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 81bde651-99d9-4f7e-9cb7-76ae26453418 2023-07-18T02:59:12.561 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689649152157779, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-07-18T02:59:12.561 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-07-18T02:59:12.562 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689649152158182, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1689649152, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "81bde651-99d9-4f7e-9cb7-76ae26453418", "db_session_id": "RM69DM3A92QQNHX42G9P", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-07-18T02:59:12.562 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689649152158248, "job": 1, "event": "recovery_finished"} 2023-07-18T02:59:12.562 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-07-18T02:59:12.562 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-07-18T02:59:12.562 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56171f6b8000 2023-07-18T02:59:12.562 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: DB pointer 0x56171f6a4000 2023-07-18T02:59:12.562 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-07-18T02:59:12.563 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-07-18T02:59:12.563 INFO:journalctl@ceph.mon.b.smithi155.stdout: ** DB Stats ** 2023-07-18T02:59:12.563 INFO:journalctl@ceph.mon.b.smithi155.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-18T02:59:12.563 INFO:journalctl@ceph.mon.b.smithi155.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-07-18T02:59:12.563 INFO:journalctl@ceph.mon.b.smithi155.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-18T02:59:12.563 INFO:journalctl@ceph.mon.b.smithi155.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-18T02:59:12.563 INFO:journalctl@ceph.mon.b.smithi155.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-07-18T02:59:12.563 INFO:journalctl@ceph.mon.b.smithi155.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-18T02:59:12.564 INFO:journalctl@ceph.mon.b.smithi155.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-18T02:59:12.564 INFO:journalctl@ceph.mon.b.smithi155.stdout: 2023-07-18T02:59:12.564 INFO:journalctl@ceph.mon.b.smithi155.stdout: ** Compaction Stats [default] ** 2023-07-18T02:59:12.564 INFO:journalctl@ceph.mon.b.smithi155.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-18T02:59:12.564 INFO:journalctl@ceph.mon.b.smithi155.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-07-18T02:59:12.564 INFO:journalctl@ceph.mon.b.smithi155.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-18T02:59:12.564 INFO:journalctl@ceph.mon.b.smithi155.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-18T02:59:12.565 INFO:journalctl@ceph.mon.b.smithi155.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-18T02:59:12.565 INFO:journalctl@ceph.mon.b.smithi155.stdout: 2023-07-18T02:59:12.565 INFO:journalctl@ceph.mon.b.smithi155.stdout: ** Compaction Stats [default] ** 2023-07-18T02:59:12.565 INFO:journalctl@ceph.mon.b.smithi155.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-18T02:59:12.565 INFO:journalctl@ceph.mon.b.smithi155.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-07-18T02:59:12.565 INFO:journalctl@ceph.mon.b.smithi155.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-18T02:59:12.565 INFO:journalctl@ceph.mon.b.smithi155.stdout: 2023-07-18T02:59:12.566 INFO:journalctl@ceph.mon.b.smithi155.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-07-18T02:59:12.566 INFO:journalctl@ceph.mon.b.smithi155.stdout: 2023-07-18T02:59:12.566 INFO:journalctl@ceph.mon.b.smithi155.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-18T02:59:12.566 INFO:journalctl@ceph.mon.b.smithi155.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-07-18T02:59:12.566 INFO:journalctl@ceph.mon.b.smithi155.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-07-18T02:59:12.566 INFO:journalctl@ceph.mon.b.smithi155.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-07-18T02:59:12.566 INFO:journalctl@ceph.mon.b.smithi155.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-07-18T02:59:12.566 INFO:journalctl@ceph.mon.b.smithi155.stdout: AddFile(Keys): cumulative 0, interval 0 2023-07-18T02:59:12.567 INFO:journalctl@ceph.mon.b.smithi155.stdout: Cumulative compaction: 0.00 GB write, 0.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-18T02:59:12.567 INFO:journalctl@ceph.mon.b.smithi155.stdout: Interval compaction: 0.00 GB write, 0.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-18T02:59:12.567 INFO:journalctl@ceph.mon.b.smithi155.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-07-18T02:59:12.567 INFO:journalctl@ceph.mon.b.smithi155.stdout: Block cache BinnedLRUCache@0x56171f61f1f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-07-18T02:59:12.567 INFO:journalctl@ceph.mon.b.smithi155.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-07-18T02:59:12.567 INFO:journalctl@ceph.mon.b.smithi155.stdout: 2023-07-18T02:59:12.567 INFO:journalctl@ceph.mon.b.smithi155.stdout: ** File Read Latency Histogram By Level [default] ** 2023-07-18T02:59:12.568 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-07-18T02:59:12.568 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: using public_addr v2:172.21.15.155:0/0 -> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] 2023-07-18T02:59:12.568 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: starting mon.b rank -1 at public addrs [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] at bind addrs [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid b1782daa-2516-11ee-9b34-001a4aab830c 2023-07-18T02:59:12.568 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: mon.b@-1(???) e0 preinit fsid b1782daa-2516-11ee-9b34-001a4aab830c 2023-07-18T02:59:12.568 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: mon.b@-1(synchronizing).mds e1 new map 2023-07-18T02:59:12.568 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: mon.b@-1(synchronizing).mds e1 print_map 2023-07-18T02:59:12.568 INFO:journalctl@ceph.mon.b.smithi155.stdout: e1 2023-07-18T02:59:12.569 INFO:journalctl@ceph.mon.b.smithi155.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-07-18T02:59:12.569 INFO:journalctl@ceph.mon.b.smithi155.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-18T02:59:12.569 INFO:journalctl@ceph.mon.b.smithi155.stdout: legacy client fscid: -1 2023-07-18T02:59:12.569 INFO:journalctl@ceph.mon.b.smithi155.stdout: 2023-07-18T02:59:12.569 INFO:journalctl@ceph.mon.b.smithi155.stdout: No filesystems configured 2023-07-18T02:59:12.569 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-07-18T02:59:12.569 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-07-18T02:59:12.570 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-07-18T02:59:12.570 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-07-18T02:59:12.570 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-07-18T02:59:12.570 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-07-18T02:59:12.570 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-07-18T02:59:12.570 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-18T02:59:12.570 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-18T02:59:12.570 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-18T02:59:12.571 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:12.571 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:12.571 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:12.571 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:12.571 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:12.571 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi155", "name": "osd_memory_target"}]: dispatch 2023-07-18T02:59:12.571 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:12.572 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T02:59:12.572 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: Updating smithi155:/etc/ceph/ceph.conf 2023-07-18T02:59:12.572 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:12.572 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:12.572 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:12.572 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: Updating smithi155:/var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-18T02:59:12.572 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2023-07-18T02:59:12.572 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:12.573 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: Updating smithi155:/var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-18T02:59:12.573 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi179", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T02:59:12.573 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: Deploying cephadm binary to smithi179 2023-07-18T02:59:12.573 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:12.573 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T02:59:12.573 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:12.573 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: Added host smithi179 2023-07-18T02:59:12.574 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:12.574 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:12.574 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:12.574 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:12.574 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T02:59:12.574 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:12.574 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:12.574 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:12.575 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:12.575 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:12.575 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi179", "name": "osd_memory_target"}]: dispatch 2023-07-18T02:59:12.575 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:12.575 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T02:59:12.575 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: Updating smithi179:/etc/ceph/ceph.conf 2023-07-18T02:59:12.575 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:12.575 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:12.576 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:12.576 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1472864393' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-07-18T02:59:12.576 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: Updating smithi179:/var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-18T02:59:12.576 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: Updating smithi179:/etc/ceph/ceph.client.admin.keyring 2023-07-18T02:59:12.576 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:12.576 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: Updating smithi179:/var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-18T02:59:12.576 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1472864393' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-07-18T02:59:12.577 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: osdmap e4: 0 total, 0 up, 0 in 2023-07-18T02:59:12.577 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:12.577 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi101:172.21.15.101=a;smithi155:172.21.15.155=b;smithi179:172.21.15.179=c", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T02:59:12.577 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: Saving service mon spec with placement smithi101:172.21.15.101=a;smithi155:172.21.15.155=b;smithi179:172.21.15.179=c;count:3 2023-07-18T02:59:12.577 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:12.577 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T02:59:12.577 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:12.578 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T02:59:12.578 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:12.578 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-18T02:59:12.578 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:12.578 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: Deploying daemon mon.c on smithi179 2023-07-18T02:59:12.578 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:12.578 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-18T02:59:12.578 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: mon.a calling monitor election 2023-07-18T02:59:12.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T02:59:12.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:12.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T02:59:12.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T02:59:12.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: mon.c calling monitor election 2023-07-18T02:59:12.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:12.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T02:59:12.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T02:59:12.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T02:59:12.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-18T02:59:12.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: monmap e2: 2 mons at {a=[v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0],c=[v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0]} removed_ranks: {} 2023-07-18T02:59:12.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: fsmap 2023-07-18T02:59:12.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: osdmap e4: 0 total, 0 up, 0 in 2023-07-18T02:59:12.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: mgrmap e13: x(active, since 46s) 2023-07-18T02:59:12.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: overall HEALTH_OK 2023-07-18T02:59:12.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:12.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:12.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:12.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-18T02:59:12.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:12.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:12.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: Deploying daemon mon.b on smithi155 2023-07-18T02:59:12.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T02:59:12.582 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: from='client.? 172.21.15.179:0/3273816216' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-18T02:59:12.582 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:12.582 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-07-18T02:59:12.582 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 ceph-mon[128885]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-07-18T02:59:12.582 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:12 smithi155 systemd[1]: Started Ceph mon.b for b1782daa-2516-11ee-9b34-001a4aab830c. 2023-07-18T02:59:17.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:17 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-18T02:59:17.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:17 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T02:59:17.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:17 smithi101 ceph-mon[126379]: mon.c calling monitor election 2023-07-18T02:59:17.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:17 smithi101 ceph-mon[126379]: mon.a calling monitor election 2023-07-18T02:59:17.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:17 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T02:59:17.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:17 smithi101 ceph-mon[126379]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:17.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:17 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T02:59:17.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:17 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T02:59:17.469 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:17 smithi101 ceph-mon[126379]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:17.469 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:17 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T02:59:17.469 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:17 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T02:59:17.469 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:17 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T02:59:17.469 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:17 smithi101 ceph-mon[126379]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-18T02:59:17.469 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:17 smithi101 ceph-mon[126379]: monmap e3: 3 mons at {a=[v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0],b=[v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0],c=[v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0]} removed_ranks: {} 2023-07-18T02:59:17.469 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:17 smithi101 ceph-mon[126379]: fsmap 2023-07-18T02:59:17.470 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:17 smithi101 ceph-mon[126379]: osdmap e4: 0 total, 0 up, 0 in 2023-07-18T02:59:17.470 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:17 smithi101 ceph-mon[126379]: mgrmap e13: x(active, since 54s) 2023-07-18T02:59:17.470 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:17 smithi101 ceph-mon[126379]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-07-18T02:59:17.470 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:17 smithi101 ceph-mon[126379]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-07-18T02:59:17.470 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:17 smithi101 ceph-mon[126379]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-07-18T02:59:17.470 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:17 smithi101 ceph-mon[126379]: mon.b (rank 2) addr [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] is down (out of quorum) 2023-07-18T02:59:17.470 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:17 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:17.470 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:17 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:17.680 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:17 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-18T02:59:17.680 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:17 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T02:59:17.680 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:17 smithi179 ceph-mon[129229]: mon.c calling monitor election 2023-07-18T02:59:17.681 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:17 smithi179 ceph-mon[129229]: mon.a calling monitor election 2023-07-18T02:59:17.681 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:17 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T02:59:17.681 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:17 smithi179 ceph-mon[129229]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:17.682 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:17 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T02:59:17.682 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:17 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T02:59:17.682 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:17 smithi179 ceph-mon[129229]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:17.682 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:17 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T02:59:17.683 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:17 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T02:59:17.683 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:17 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T02:59:17.683 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:17 smithi179 ceph-mon[129229]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-18T02:59:17.684 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:17 smithi179 ceph-mon[129229]: monmap e3: 3 mons at {a=[v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0],b=[v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0],c=[v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0]} removed_ranks: {} 2023-07-18T02:59:17.684 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:17 smithi179 ceph-mon[129229]: fsmap 2023-07-18T02:59:17.684 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:17 smithi179 ceph-mon[129229]: osdmap e4: 0 total, 0 up, 0 in 2023-07-18T02:59:17.684 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:17 smithi179 ceph-mon[129229]: mgrmap e13: x(active, since 54s) 2023-07-18T02:59:17.685 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:17 smithi179 ceph-mon[129229]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-07-18T02:59:17.685 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:17 smithi179 ceph-mon[129229]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-07-18T02:59:17.685 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:17 smithi179 ceph-mon[129229]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-07-18T02:59:17.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:17 smithi179 ceph-mon[129229]: mon.b (rank 2) addr [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] is down (out of quorum) 2023-07-18T02:59:17.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:17 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:17.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:17 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:17.861 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:17.887+0000 7f0237d41700 1 -- 172.21.15.179:0/3778331879 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f021c0035c0 msgr2=0x7f021c005a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:59:17.861 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:17.887+0000 7f0237d41700 1 --2- 172.21.15.179:0/3778331879 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f021c0035c0 0x7f021c005a40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f02300778e0 tx=0x7f022c00fe60 comp rx=0 tx=0).stop 2023-07-18T02:59:17.861 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:17.888+0000 7f0237d41700 1 -- 172.21.15.179:0/3778331879 shutdown_connections 2023-07-18T02:59:17.861 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:17.888+0000 7f0237d41700 1 --2- 172.21.15.179:0/3778331879 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f021c0035c0 0x7f021c005a40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:59:17.861 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:17.888+0000 7f0237d41700 1 -- 172.21.15.179:0/3778331879 >> 172.21.15.179:0/3778331879 conn(0x7f0230073d70 msgr2=0x7f0230074170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:59:17.862 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:17.888+0000 7f0237d41700 1 -- 172.21.15.179:0/3778331879 shutdown_connections 2023-07-18T02:59:17.862 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:17.888+0000 7f0237d41700 1 -- 172.21.15.179:0/3778331879 wait complete. 2023-07-18T02:59:17.864 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:17.888+0000 7f0237d41700 1 Processor -- start 2023-07-18T02:59:17.864 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:17.888+0000 7f0237d41700 1 -- start start 2023-07-18T02:59:17.864 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:17.889+0000 7f0237d41700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f021c0035c0 0x7f0230193cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:59:17.864 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:17.889+0000 7f0237d41700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02301941f0 0x7f02301975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:59:17.865 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:17.889+0000 7f0237d41700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0230197af0 0x7f0230197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:59:17.865 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:17.889+0000 7f0237d41700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0230194890 con 0x7f0230197af0 2023-07-18T02:59:17.865 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:17.889+0000 7f0237d41700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02301a0940 con 0x7f02301941f0 2023-07-18T02:59:17.865 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:17.889+0000 7f0237d41700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02301a0a80 con 0x7f021c0035c0 2023-07-18T02:59:17.865 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:17.889+0000 7f0235add700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f021c0035c0 0x7f0230193cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:59:17.865 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:17.889+0000 7f0235add700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f021c0035c0 0x7f0230193cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48146/0 (socket says 172.21.15.179:48146) 2023-07-18T02:59:17.865 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:17.889+0000 7f0235add700 1 -- 172.21.15.179:0/3882550763 learned_addr learned my addr 172.21.15.179:0/3882550763 (peer_addr_for_me v2:172.21.15.179:0/0) 2023-07-18T02:59:17.866 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:17.889+0000 7f02362de700 1 --2- 172.21.15.179:0/3882550763 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0230197af0 0x7f0230197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:59:17.866 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:17.889+0000 7f0235add700 1 -- 172.21.15.179:0/3882550763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f021c0035c0 msgr2=0x7f0230193cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-07-18T02:59:17.866 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:17.889+0000 7f0235add700 1 -- 172.21.15.179:0/3882550763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f021c0035c0 msgr2=0x7f0230193cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-07-18T02:59:17.866 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:17.889+0000 7f0235add700 1 --2- 172.21.15.179:0/3882550763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f021c0035c0 0x7f0230193cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-18T02:59:17.866 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:17.889+0000 7f0235add700 1 --2- 172.21.15.179:0/3882550763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f021c0035c0 0x7f0230193cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-18T02:59:17.866 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:17.889+0000 7f02352dc700 1 --2- 172.21.15.179:0/3882550763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02301941f0 0x7f02301975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:59:17.866 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:17.889+0000 7f02352dc700 1 -- 172.21.15.179:0/3882550763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02301941f0 msgr2=0x7f02301975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-07-18T02:59:17.867 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:17.889+0000 7f02352dc700 1 -- 172.21.15.179:0/3882550763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02301941f0 msgr2=0x7f02301975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-07-18T02:59:17.867 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:17.889+0000 7f02352dc700 1 --2- 172.21.15.179:0/3882550763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02301941f0 0x7f02301975b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-18T02:59:17.867 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:17.889+0000 7f02352dc700 1 --2- 172.21.15.179:0/3882550763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02301941f0 0x7f02301975b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-18T02:59:17.867 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:17.889+0000 7f02362de700 1 -- 172.21.15.179:0/3882550763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f021c0035c0 msgr2=0x7f0230193cb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-18T02:59:17.867 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:17.889+0000 7f02362de700 1 --2- 172.21.15.179:0/3882550763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f021c0035c0 0x7f0230193cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:59:17.867 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:17.889+0000 7f02362de700 1 -- 172.21.15.179:0/3882550763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02301941f0 msgr2=0x7f02301975b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-18T02:59:17.868 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:17.889+0000 7f02362de700 1 --2- 172.21.15.179:0/3882550763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02301941f0 0x7f02301975b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:59:17.868 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:17.889+0000 7f02362de700 1 -- 172.21.15.179:0/3882550763 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f022c00f8b0 con 0x7f0230197af0 2023-07-18T02:59:17.868 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:17.889+0000 7f02362de700 1 --2- 172.21.15.179:0/3882550763 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0230197af0 0x7f0230197f60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f022800bd50 tx=0x7f022800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:59:17.868 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:17.890+0000 7f0226ffd700 1 -- 172.21.15.179:0/3882550763 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022800cd10 con 0x7f0230197af0 2023-07-18T02:59:17.868 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:17.890+0000 7f0237d41700 1 -- 172.21.15.179:0/3882550763 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02301a0d10 con 0x7f0230197af0 2023-07-18T02:59:17.868 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:17.890+0000 7f0226ffd700 1 -- 172.21.15.179:0/3882550763 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f022800ce70 con 0x7f0230197af0 2023-07-18T02:59:17.868 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:17.890+0000 7f0226ffd700 1 -- 172.21.15.179:0/3882550763 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0228012840 con 0x7f0230197af0 2023-07-18T02:59:17.869 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:17.890+0000 7f0237d41700 1 -- 172.21.15.179:0/3882550763 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02301a11c0 con 0x7f0230197af0 2023-07-18T02:59:17.869 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:17.891+0000 7f0226ffd700 1 -- 172.21.15.179:0/3882550763 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f0228014440 con 0x7f0230197af0 2023-07-18T02:59:17.869 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:17.891+0000 7f0226ffd700 1 --2- 172.21.15.179:0/3882550763 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f0214038c30 0x7f021403b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:59:17.869 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:17.891+0000 7f0226ffd700 1 -- 172.21.15.179:0/3882550763 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f022804ad10 con 0x7f0230197af0 2023-07-18T02:59:17.869 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:17.891+0000 7f0237d41700 1 -- 172.21.15.179:0/3882550763 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f023018e160 con 0x7f0230197af0 2023-07-18T02:59:17.869 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:17.891+0000 7f0235add700 1 --2- 172.21.15.179:0/3882550763 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f0214038c30 0x7f021403b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:59:17.870 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:17.892+0000 7f0235add700 1 --2- 172.21.15.179:0/3882550763 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f0214038c30 0x7f021403b0e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f022c020040 tx=0x7f022c001e70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:59:17.870 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:17.896+0000 7f0226ffd700 1 -- 172.21.15.179:0/3882550763 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0228014b00 con 0x7f0230197af0 2023-07-18T02:59:18.158 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:18.185+0000 7f0237d41700 1 -- 172.21.15.179:0/3882550763 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0230061b40 con 0x7f0230197af0 2023-07-18T02:59:18.159 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:18.186+0000 7f0226ffd700 1 -- 172.21.15.179:0/3882550763 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1278 (secure 0 0 0) 0x7f022801b8d0 con 0x7f0230197af0 2023-07-18T02:59:18.159 INFO:teuthology.orchestra.run.smithi179.stdout: 2023-07-18T02:59:18.160 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":3,"fsid":"b1782daa-2516-11ee-9b34-001a4aab830c","modified":"2023-07-18T02:59:12.180898Z","created":"2023-07-18T02:57:19.585180Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:3300","nonce":0},{"type":"v1","addr":"172.21.15.101:6789","nonce":0}]},"addr":"172.21.15.101:6789/0","public_addr":"172.21.15.101:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:3300","nonce":0},{"type":"v1","addr":"172.21.15.155:6789","nonce":0}]},"addr":"172.21.15.155:6789/0","public_addr":"172.21.15.155:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-07-18T02:59:18.162 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:18.188+0000 7f0237d41700 1 -- 172.21.15.179:0/3882550763 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f0214038c30 msgr2=0x7f021403b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:59:18.162 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:18.188+0000 7f0237d41700 1 --2- 172.21.15.179:0/3882550763 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f0214038c30 0x7f021403b0e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f022c020040 tx=0x7f022c001e70 comp rx=0 tx=0).stop 2023-07-18T02:59:18.162 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:18.188+0000 7f0237d41700 1 -- 172.21.15.179:0/3882550763 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0230197af0 msgr2=0x7f0230197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:59:18.162 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:18.188+0000 7f0237d41700 1 --2- 172.21.15.179:0/3882550763 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0230197af0 0x7f0230197f60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f022800bd50 tx=0x7f022800be30 comp rx=0 tx=0).stop 2023-07-18T02:59:18.163 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:18.188+0000 7f0237d41700 1 -- 172.21.15.179:0/3882550763 shutdown_connections 2023-07-18T02:59:18.163 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:18.188+0000 7f0237d41700 1 --2- 172.21.15.179:0/3882550763 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f0214038c30 0x7f021403b0e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:59:18.163 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:18.188+0000 7f0237d41700 1 --2- 172.21.15.179:0/3882550763 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0230197af0 0x7f0230197f60 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:59:18.163 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:18.188+0000 7f0237d41700 1 --2- 172.21.15.179:0/3882550763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f021c0035c0 0x7f0230193cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:59:18.164 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:18.188+0000 7f0237d41700 1 --2- 172.21.15.179:0/3882550763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02301941f0 0x7f02301975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:59:18.164 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:18.188+0000 7f0237d41700 1 -- 172.21.15.179:0/3882550763 >> 172.21.15.179:0/3882550763 conn(0x7f0230073d70 msgr2=0x7f0230100a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:59:18.164 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:18.188+0000 7f0237d41700 1 -- 172.21.15.179:0/3882550763 shutdown_connections 2023-07-18T02:59:18.164 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:18.188+0000 7f0237d41700 1 -- 172.21.15.179:0/3882550763 wait complete. 2023-07-18T02:59:18.165 INFO:teuthology.orchestra.run.smithi179.stderr:dumped monmap epoch 3 2023-07-18T02:59:18.509 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:18 smithi179 ceph-mon[129229]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:18.510 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:18 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:18.510 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:18 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:18.510 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:18 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T02:59:18.510 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:18 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:18.510 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:18 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T02:59:18.510 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:18 smithi179 ceph-mon[129229]: from='client.? 172.21.15.179:0/3882550763' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-18T02:59:18.516 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-07-18T02:59:18.516 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph config generate-minimal-conf 2023-07-18T02:59:18.584 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:18 smithi101 ceph-mon[126379]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:18.585 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:18 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:18.585 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:18 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:18.585 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:18 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T02:59:18.585 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:18 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:18.585 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:18 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T02:59:18.585 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:18 smithi101 ceph-mon[126379]: from='client.? 172.21.15.179:0/3882550763' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-18T02:59:19.253 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T02:59:19.525 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-18T02:59:19.525 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T02:59:19.525 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: mon.c calling monitor election 2023-07-18T02:59:19.525 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: mon.a calling monitor election 2023-07-18T02:59:19.525 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T02:59:19.525 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:19.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T02:59:19.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T02:59:19.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:19.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T02:59:19.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T02:59:19.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T02:59:19.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-18T02:59:19.527 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: monmap e3: 3 mons at {a=[v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0],b=[v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0],c=[v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0]} removed_ranks: {} 2023-07-18T02:59:19.527 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: fsmap 2023-07-18T02:59:19.527 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: osdmap e4: 0 total, 0 up, 0 in 2023-07-18T02:59:19.527 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: mgrmap e13: x(active, since 54s) 2023-07-18T02:59:19.527 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-07-18T02:59:19.527 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-07-18T02:59:19.527 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-07-18T02:59:19.527 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: mon.b (rank 2) addr [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] is down (out of quorum) 2023-07-18T02:59:19.528 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:19.528 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:19.528 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:19.528 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:19.528 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:19.528 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T02:59:19.528 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:19.529 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T02:59:19.529 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: from='client.? 172.21.15.179:0/3882550763' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-18T02:59:20.217 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:19 smithi101 ceph-mon[126379]: mon.b calling monitor election 2023-07-18T02:59:20.217 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:19 smithi101 ceph-mon[126379]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:20.217 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:19 smithi101 ceph-mon[126379]: Updating smithi101:/etc/ceph/ceph.conf 2023-07-18T02:59:20.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:19 smithi101 ceph-mon[126379]: Updating smithi155:/etc/ceph/ceph.conf 2023-07-18T02:59:20.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:19 smithi101 ceph-mon[126379]: Updating smithi179:/etc/ceph/ceph.conf 2023-07-18T02:59:20.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:19 smithi101 ceph-mon[126379]: mon.b calling monitor election 2023-07-18T02:59:20.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:19 smithi101 ceph-mon[126379]: mon.c calling monitor election 2023-07-18T02:59:20.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:19 smithi101 ceph-mon[126379]: mon.a calling monitor election 2023-07-18T02:59:20.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:19 smithi101 ceph-mon[126379]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-07-18T02:59:20.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:19 smithi101 ceph-mon[126379]: monmap e3: 3 mons at {a=[v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0],b=[v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0],c=[v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0]} removed_ranks: {} 2023-07-18T02:59:20.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:19 smithi101 ceph-mon[126379]: fsmap 2023-07-18T02:59:20.219 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:19 smithi101 ceph-mon[126379]: osdmap e4: 0 total, 0 up, 0 in 2023-07-18T02:59:20.219 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:19 smithi101 ceph-mon[126379]: mgrmap e13: x(active, since 56s) 2023-07-18T02:59:20.219 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:19 smithi101 ceph-mon[126379]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-07-18T02:59:20.219 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:19 smithi101 ceph-mon[126379]: Cluster is now healthy 2023-07-18T02:59:20.219 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:19 smithi101 ceph-mon[126379]: overall HEALTH_OK 2023-07-18T02:59:20.219 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:19 smithi101 ceph-mon[126379]: Updating smithi155:/var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-18T02:59:20.219 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:19 smithi101 ceph-mon[126379]: Updating smithi179:/var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-18T02:59:20.220 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:19 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:20.220 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:19 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:20.220 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:19 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:20.220 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:19 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:20.274 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: mon.b calling monitor election 2023-07-18T02:59:20.275 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:20.275 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: Updating smithi101:/etc/ceph/ceph.conf 2023-07-18T02:59:20.275 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: Updating smithi155:/etc/ceph/ceph.conf 2023-07-18T02:59:20.275 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: Updating smithi179:/etc/ceph/ceph.conf 2023-07-18T02:59:20.275 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: mon.b calling monitor election 2023-07-18T02:59:20.275 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: mon.c calling monitor election 2023-07-18T02:59:20.275 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: mon.a calling monitor election 2023-07-18T02:59:20.276 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-07-18T02:59:20.276 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: monmap e3: 3 mons at {a=[v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0],b=[v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0],c=[v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0]} removed_ranks: {} 2023-07-18T02:59:20.276 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: fsmap 2023-07-18T02:59:20.276 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: osdmap e4: 0 total, 0 up, 0 in 2023-07-18T02:59:20.276 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: mgrmap e13: x(active, since 56s) 2023-07-18T02:59:20.276 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-07-18T02:59:20.276 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: Cluster is now healthy 2023-07-18T02:59:20.277 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: overall HEALTH_OK 2023-07-18T02:59:20.277 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: Updating smithi155:/var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-18T02:59:20.277 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: Updating smithi179:/var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-18T02:59:20.277 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:20.277 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:20.277 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:20.277 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:19 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:20.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:19 smithi179 ceph-mon[129229]: mon.b calling monitor election 2023-07-18T02:59:20.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:19 smithi179 ceph-mon[129229]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:20.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:19 smithi179 ceph-mon[129229]: Updating smithi101:/etc/ceph/ceph.conf 2023-07-18T02:59:20.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:19 smithi179 ceph-mon[129229]: Updating smithi155:/etc/ceph/ceph.conf 2023-07-18T02:59:20.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:19 smithi179 ceph-mon[129229]: Updating smithi179:/etc/ceph/ceph.conf 2023-07-18T02:59:20.431 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:19 smithi179 ceph-mon[129229]: mon.b calling monitor election 2023-07-18T02:59:20.431 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:19 smithi179 ceph-mon[129229]: mon.c calling monitor election 2023-07-18T02:59:20.431 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:19 smithi179 ceph-mon[129229]: mon.a calling monitor election 2023-07-18T02:59:20.431 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:19 smithi179 ceph-mon[129229]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-07-18T02:59:20.431 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:19 smithi179 ceph-mon[129229]: monmap e3: 3 mons at {a=[v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0],b=[v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0],c=[v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0]} removed_ranks: {} 2023-07-18T02:59:20.432 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:19 smithi179 ceph-mon[129229]: fsmap 2023-07-18T02:59:20.432 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:19 smithi179 ceph-mon[129229]: osdmap e4: 0 total, 0 up, 0 in 2023-07-18T02:59:20.432 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:19 smithi179 ceph-mon[129229]: mgrmap e13: x(active, since 56s) 2023-07-18T02:59:20.432 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:19 smithi179 ceph-mon[129229]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-07-18T02:59:20.432 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:19 smithi179 ceph-mon[129229]: Cluster is now healthy 2023-07-18T02:59:20.433 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:19 smithi179 ceph-mon[129229]: overall HEALTH_OK 2023-07-18T02:59:20.433 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:19 smithi179 ceph-mon[129229]: Updating smithi155:/var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-18T02:59:20.433 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:19 smithi179 ceph-mon[129229]: Updating smithi179:/var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-18T02:59:20.433 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:19 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:20.434 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:19 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:20.434 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:19 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:20.434 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:19 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:20.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:20.913+0000 7f951a270700 1 -- 172.21.15.101:0/718801676 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f95140ff6a0 msgr2=0x7f95140ffa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:59:20.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:20.913+0000 7f951a270700 1 --2- 172.21.15.101:0/718801676 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f95140ff6a0 0x7f95140ffa70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9510009910 tx=0x7f9510009c20 comp rx=0 tx=0).stop 2023-07-18T02:59:20.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:20.913+0000 7f951a270700 1 -- 172.21.15.101:0/718801676 shutdown_connections 2023-07-18T02:59:20.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:20.913+0000 7f951a270700 1 --2- 172.21.15.101:0/718801676 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f95140ff6a0 0x7f95140ffa70 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:59:20.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:20.913+0000 7f951a270700 1 -- 172.21.15.101:0/718801676 >> 172.21.15.101:0/718801676 conn(0x7f95140faf40 msgr2=0x7f95140fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:59:20.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:20.914+0000 7f951a270700 1 -- 172.21.15.101:0/718801676 shutdown_connections 2023-07-18T02:59:20.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:20.914+0000 7f951a270700 1 -- 172.21.15.101:0/718801676 wait complete. 2023-07-18T02:59:20.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:20.915+0000 7f951a270700 1 Processor -- start 2023-07-18T02:59:20.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:20.915+0000 7f951a270700 1 -- start start 2023-07-18T02:59:20.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:20.915+0000 7f951a270700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f95140ff6a0 0x7f9514072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:59:20.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:20.915+0000 7f951a270700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9514072d50 0x7f95140731c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:59:20.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:20.915+0000 7f951a270700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95141a0c70 0x7f95141a3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:59:20.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:20.915+0000 7f951a270700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95141a36a0 con 0x7f95140ff6a0 2023-07-18T02:59:20.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:20.915+0000 7f951a270700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95141a3810 con 0x7f9514072d50 2023-07-18T02:59:20.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:20.915+0000 7f951a270700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95141a39d0 con 0x7f95141a0c70 2023-07-18T02:59:20.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:20.916+0000 7f951926e700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f95140ff6a0 0x7f9514072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:59:20.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:20.916+0000 7f951926e700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f95140ff6a0 0x7f9514072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:40522/0 (socket says 172.21.15.101:40522) 2023-07-18T02:59:20.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:20.916+0000 7f951926e700 1 -- 172.21.15.101:0/1526894849 learned_addr learned my addr 172.21.15.101:0/1526894849 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:59:20.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:20.916+0000 7f9519a6f700 1 --2- 172.21.15.101:0/1526894849 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95141a0c70 0x7f95141a3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:59:20.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:20.916+0000 7f9518a6d700 1 --2- 172.21.15.101:0/1526894849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9514072d50 0x7f95140731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:59:20.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:20.916+0000 7f951926e700 1 -- 172.21.15.101:0/1526894849 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95141a0c70 msgr2=0x7f95141a3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:59:20.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:20.916+0000 7f951926e700 1 --2- 172.21.15.101:0/1526894849 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95141a0c70 0x7f95141a3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:59:20.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:20.916+0000 7f951926e700 1 -- 172.21.15.101:0/1526894849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9514072d50 msgr2=0x7f95140731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:59:20.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:20.916+0000 7f951926e700 1 --2- 172.21.15.101:0/1526894849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9514072d50 0x7f95140731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:59:20.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:20.916+0000 7f951926e700 1 -- 172.21.15.101:0/1526894849 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95100095f0 con 0x7f95140ff6a0 2023-07-18T02:59:20.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:20.916+0000 7f9518a6d700 1 --2- 172.21.15.101:0/1526894849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9514072d50 0x7f95140731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T02:59:20.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:20.916+0000 7f951926e700 1 --2- 172.21.15.101:0/1526894849 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f95140ff6a0 0x7f9514072810 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9510004c70 tx=0x7f9510004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:59:20.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:20.916+0000 7f950a7fc700 1 -- 172.21.15.101:0/1526894849 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951001d070 con 0x7f95140ff6a0 2023-07-18T02:59:20.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:20.916+0000 7f951a270700 1 -- 172.21.15.101:0/1526894849 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95141a3c50 con 0x7f95140ff6a0 2023-07-18T02:59:20.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:20.916+0000 7f950a7fc700 1 -- 172.21.15.101:0/1526894849 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f951000be20 con 0x7f95140ff6a0 2023-07-18T02:59:20.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:20.916+0000 7f950a7fc700 1 -- 172.21.15.101:0/1526894849 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9510022dd0 con 0x7f95140ff6a0 2023-07-18T02:59:20.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:20.917+0000 7f951a270700 1 -- 172.21.15.101:0/1526894849 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95141a4140 con 0x7f95140ff6a0 2023-07-18T02:59:20.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:20.918+0000 7f950a7fc700 1 -- 172.21.15.101:0/1526894849 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f951000f460 con 0x7f95140ff6a0 2023-07-18T02:59:20.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:20.918+0000 7f951a270700 1 -- 172.21.15.101:0/1526894849 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9514061b40 con 0x7f95140ff6a0 2023-07-18T02:59:20.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:20.918+0000 7f950a7fc700 1 --2- 172.21.15.101:0/1526894849 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f9500038c80 0x7f950003b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:59:20.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:20.918+0000 7f950a7fc700 1 -- 172.21.15.101:0/1526894849 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f951004ca40 con 0x7f95140ff6a0 2023-07-18T02:59:20.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:20.918+0000 7f9518a6d700 1 --2- 172.21.15.101:0/1526894849 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f9500038c80 0x7f950003b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:59:20.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:20.919+0000 7f9518a6d700 1 --2- 172.21.15.101:0/1526894849 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f9500038c80 0x7f950003b130 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f95040079b0 tx=0x7f9504008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:59:20.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:20.923+0000 7f950a7fc700 1 -- 172.21.15.101:0/1526894849 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f951000edb0 con 0x7f95140ff6a0 2023-07-18T02:59:21.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:21.132+0000 7f951a270700 1 -- 172.21.15.101:0/1526894849 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f95141a4390 con 0x7f95140ff6a0 2023-07-18T02:59:21.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:21.132+0000 7f950a7fc700 1 -- 172.21.15.101:0/1526894849 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+277 (secure 0 0 0) 0x7f9510027030 con 0x7f95140ff6a0 2023-07-18T02:59:21.106 INFO:teuthology.orchestra.run.smithi101.stdout:# minimal ceph.conf for b1782daa-2516-11ee-9b34-001a4aab830c 2023-07-18T02:59:21.106 INFO:teuthology.orchestra.run.smithi101.stdout:[global] 2023-07-18T02:59:21.106 INFO:teuthology.orchestra.run.smithi101.stdout: fsid = b1782daa-2516-11ee-9b34-001a4aab830c 2023-07-18T02:59:21.106 INFO:teuthology.orchestra.run.smithi101.stdout: mon_host = [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2023-07-18T02:59:21.108 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:21.135+0000 7f951a270700 1 -- 172.21.15.101:0/1526894849 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f9500038c80 msgr2=0x7f950003b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:59:21.108 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:21.135+0000 7f951a270700 1 --2- 172.21.15.101:0/1526894849 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f9500038c80 0x7f950003b130 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f95040079b0 tx=0x7f9504008040 comp rx=0 tx=0).stop 2023-07-18T02:59:21.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:21.135+0000 7f951a270700 1 -- 172.21.15.101:0/1526894849 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f95140ff6a0 msgr2=0x7f9514072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:59:21.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:21.135+0000 7f951a270700 1 --2- 172.21.15.101:0/1526894849 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f95140ff6a0 0x7f9514072810 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9510004c70 tx=0x7f9510004d50 comp rx=0 tx=0).stop 2023-07-18T02:59:21.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:21.136+0000 7f951a270700 1 -- 172.21.15.101:0/1526894849 shutdown_connections 2023-07-18T02:59:21.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:21.136+0000 7f951a270700 1 --2- 172.21.15.101:0/1526894849 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f9500038c80 0x7f950003b130 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:59:21.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:21.136+0000 7f951a270700 1 --2- 172.21.15.101:0/1526894849 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95141a0c70 0x7f95141a3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:59:21.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:21.136+0000 7f951a270700 1 --2- 172.21.15.101:0/1526894849 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f95140ff6a0 0x7f9514072810 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:59:21.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:21.136+0000 7f951a270700 1 --2- 172.21.15.101:0/1526894849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9514072d50 0x7f95140731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:59:21.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:21.136+0000 7f951a270700 1 -- 172.21.15.101:0/1526894849 >> 172.21.15.101:0/1526894849 conn(0x7f95140faf40 msgr2=0x7f9514105170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:59:21.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:21.136+0000 7f951a270700 1 -- 172.21.15.101:0/1526894849 shutdown_connections 2023-07-18T02:59:21.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:21.136+0000 7f951a270700 1 -- 172.21.15.101:0/1526894849 wait complete. 2023-07-18T02:59:21.305 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 02:59:21 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T02:59:21.180+0000 7f2a3cd80700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-07-18T02:59:21.305 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:21 smithi101 ceph-mon[126379]: Updating smithi101:/var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-18T02:59:21.305 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:21 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:21.305 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:21 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:21.306 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:21 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:21.306 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:21 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T02:59:21.306 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:21 smithi101 ceph-mon[126379]: Reconfiguring mon.a (unknown last config time)... 2023-07-18T02:59:21.306 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:21 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-18T02:59:21.306 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:21 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-18T02:59:21.306 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:21 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:21.306 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:21 smithi101 ceph-mon[126379]: Reconfiguring daemon mon.a on smithi101 2023-07-18T02:59:21.429 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:21 smithi179 ceph-mon[129229]: Updating smithi101:/var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-18T02:59:21.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:21 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:21.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:21 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:21.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:21 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:21.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:21 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T02:59:21.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:21 smithi179 ceph-mon[129229]: Reconfiguring mon.a (unknown last config time)... 2023-07-18T02:59:21.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:21 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-18T02:59:21.431 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:21 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-18T02:59:21.431 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:21 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:21.431 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:21 smithi179 ceph-mon[129229]: Reconfiguring daemon mon.a on smithi101 2023-07-18T02:59:21.524 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:21 smithi155 ceph-mon[128885]: Updating smithi101:/var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-18T02:59:21.525 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:21 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:21.525 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:21 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:21.525 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:21 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:21.525 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:21 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T02:59:21.525 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:21 smithi155 ceph-mon[128885]: Reconfiguring mon.a (unknown last config time)... 2023-07-18T02:59:21.525 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:21 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-18T02:59:21.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:21 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-18T02:59:21.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:21 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:21.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:21 smithi155 ceph-mon[128885]: Reconfiguring daemon mon.a on smithi101 2023-07-18T02:59:21.736 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-07-18T02:59:21.736 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-07-18T02:59:21.737 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd of=/etc/ceph/ceph.conf 2023-07-18T02:59:21.808 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-07-18T02:59:21.808 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-18T02:59:21.916 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-07-18T02:59:21.916 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/ceph/ceph.conf 2023-07-18T02:59:21.952 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-07-18T02:59:21.953 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-18T02:59:22.022 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2023-07-18T02:59:22.023 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/ceph/ceph.conf 2023-07-18T02:59:22.059 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2023-07-18T02:59:22.059 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-18T02:59:22.130 INFO:tasks.cephadm:Adding mgr.x on smithi101 2023-07-18T02:59:22.130 INFO:tasks.cephadm:Adding mgr.y on smithi155 2023-07-18T02:59:22.131 INFO:tasks.cephadm:Adding mgr.z on smithi179 2023-07-18T02:59:22.131 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph orch apply mgr '3;smithi101=x;smithi155=y;smithi179=z' 2023-07-18T02:59:22.217 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:22 smithi101 ceph-mon[126379]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:22.217 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:22 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1526894849' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:22.274 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:22 smithi155 ceph-mon[128885]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:22.274 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:22 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1526894849' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:22.429 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:22 smithi179 ceph-mon[129229]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:22.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:22 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1526894849' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:22.717 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.c/config 2023-07-18T02:59:23.179 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:23 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:23.180 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:23 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:23.180 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:23 smithi179 ceph-mon[129229]: Reconfiguring mon.b (monmap changed)... 2023-07-18T02:59:23.180 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:23 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-18T02:59:23.180 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:23 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-18T02:59:23.180 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:23 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:23.180 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:23 smithi179 ceph-mon[129229]: Reconfiguring daemon mon.b on smithi155 2023-07-18T02:59:23.343 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:23 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:23.343 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:23 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:23.343 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:23 smithi155 ceph-mon[128885]: Reconfiguring mon.b (monmap changed)... 2023-07-18T02:59:23.344 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:23 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-18T02:59:23.344 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:23 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-18T02:59:23.344 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:23 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:23.344 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:23 smithi155 ceph-mon[128885]: Reconfiguring daemon mon.b on smithi155 2023-07-18T02:59:23.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:23 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:23.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:23 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:23.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:23 smithi101 ceph-mon[126379]: Reconfiguring mon.b (monmap changed)... 2023-07-18T02:59:23.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:23 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-18T02:59:23.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:23 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-18T02:59:23.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:23 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:23.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:23 smithi101 ceph-mon[126379]: Reconfiguring daemon mon.b on smithi155 2023-07-18T02:59:24.034 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.059+0000 7f25b214a700 1 -- 172.21.15.179:0/274591856 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f25ac0fe600 msgr2=0x7f25ac0fea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:59:24.034 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.059+0000 7f25b214a700 1 --2- 172.21.15.179:0/274591856 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f25ac0fe600 0x7f25ac0fea10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f25a0009910 tx=0x7f25a0009c20 comp rx=0 tx=0).stop 2023-07-18T02:59:24.034 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.060+0000 7f25b214a700 1 -- 172.21.15.179:0/274591856 shutdown_connections 2023-07-18T02:59:24.034 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.060+0000 7f25b214a700 1 --2- 172.21.15.179:0/274591856 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f25ac0fe600 0x7f25ac0fea10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:59:24.035 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.060+0000 7f25b214a700 1 -- 172.21.15.179:0/274591856 >> 172.21.15.179:0/274591856 conn(0x7f25ac0f9bb0 msgr2=0x7f25ac0fbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:59:24.035 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.060+0000 7f25b214a700 1 -- 172.21.15.179:0/274591856 shutdown_connections 2023-07-18T02:59:24.035 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.060+0000 7f25b214a700 1 -- 172.21.15.179:0/274591856 wait complete. 2023-07-18T02:59:24.035 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.060+0000 7f25b214a700 1 Processor -- start 2023-07-18T02:59:24.035 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.060+0000 7f25b214a700 1 -- start start 2023-07-18T02:59:24.036 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.060+0000 7f25b214a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25ac0fe600 0x7f25ac069060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:59:24.036 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.061+0000 7f25b214a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25ac197730 0x7f25ac199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:59:24.036 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.061+0000 7f25b214a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f25ac19a0a0 0x7f25ac19c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:59:24.036 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.061+0000 7f25b214a700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac19ccc0 con 0x7f25ac19a0a0 2023-07-18T02:59:24.036 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.061+0000 7f25b214a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac19ce30 con 0x7f25ac0fe600 2023-07-18T02:59:24.036 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.061+0000 7f25b214a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac19cff0 con 0x7f25ac197730 2023-07-18T02:59:24.036 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.061+0000 7f25aaffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25ac197730 0x7f25ac199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:59:24.037 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.061+0000 7f25aaffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25ac197730 0x7f25ac199b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48158/0 (socket says 172.21.15.179:48158) 2023-07-18T02:59:24.037 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.061+0000 7f25aaffd700 1 -- 172.21.15.179:0/2821367290 learned_addr learned my addr 172.21.15.179:0/2821367290 (peer_addr_for_me v2:172.21.15.179:0/0) 2023-07-18T02:59:24.037 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.061+0000 7f25ab7fe700 1 --2- 172.21.15.179:0/2821367290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25ac0fe600 0x7f25ac069060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:59:24.037 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.061+0000 7f25abfff700 1 --2- 172.21.15.179:0/2821367290 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f25ac19a0a0 0x7f25ac19c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:59:24.037 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.061+0000 7f25aaffd700 1 -- 172.21.15.179:0/2821367290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25ac0fe600 msgr2=0x7f25ac069060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:59:24.038 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.061+0000 7f25aaffd700 1 --2- 172.21.15.179:0/2821367290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25ac0fe600 0x7f25ac069060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:59:24.038 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.061+0000 7f25aaffd700 1 -- 172.21.15.179:0/2821367290 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f25ac19a0a0 msgr2=0x7f25ac19c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:59:24.038 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.061+0000 7f25aaffd700 1 --2- 172.21.15.179:0/2821367290 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f25ac19a0a0 0x7f25ac19c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:59:24.038 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.061+0000 7f25aaffd700 1 -- 172.21.15.179:0/2821367290 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a00095f0 con 0x7f25ac197730 2023-07-18T02:59:24.038 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.061+0000 7f25aaffd700 1 --2- 172.21.15.179:0/2821367290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25ac197730 0x7f25ac199b10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f259c00b780 tx=0x7f259c00ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:59:24.039 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.061+0000 7f25a8ff9700 1 -- 172.21.15.179:0/2821367290 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259c004230 con 0x7f25ac197730 2023-07-18T02:59:24.039 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.061+0000 7f25a8ff9700 1 -- 172.21.15.179:0/2821367290 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f259c004390 con 0x7f25ac197730 2023-07-18T02:59:24.039 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.061+0000 7f25a8ff9700 1 -- 172.21.15.179:0/2821367290 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259c011b90 con 0x7f25ac197730 2023-07-18T02:59:24.040 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.061+0000 7f25b214a700 1 -- 172.21.15.179:0/2821367290 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25ac19d2d0 con 0x7f25ac197730 2023-07-18T02:59:24.040 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.062+0000 7f25b214a700 1 -- 172.21.15.179:0/2821367290 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25ac19d850 con 0x7f25ac197730 2023-07-18T02:59:24.040 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.062+0000 7f25a8ff9700 1 -- 172.21.15.179:0/2821367290 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f259c011cf0 con 0x7f25ac197730 2023-07-18T02:59:24.040 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.062+0000 7f25a8ff9700 1 --2- 172.21.15.179:0/2821367290 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f2594038c80 0x7f259403b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:59:24.040 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.062+0000 7f25a8ff9700 1 -- 172.21.15.179:0/2821367290 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f259c049f90 con 0x7f25ac197730 2023-07-18T02:59:24.041 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.063+0000 7f25b214a700 1 -- 172.21.15.179:0/2821367290 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25ac18d1d0 con 0x7f25ac197730 2023-07-18T02:59:24.041 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.063+0000 7f25ab7fe700 1 --2- 172.21.15.179:0/2821367290 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f2594038c80 0x7f259403b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:59:24.041 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.063+0000 7f25ab7fe700 1 --2- 172.21.15.179:0/2821367290 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f2594038c80 0x7f259403b130 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f25a0006010 tx=0x7f25a00058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:59:24.041 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.067+0000 7f25a8ff9700 1 -- 172.21.15.179:0/2821367290 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f259c018d20 con 0x7f25ac197730 2023-07-18T02:59:24.249 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.276+0000 7f25b214a700 1 -- 172.21.15.179:0/2821367290 --> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi101=x;smithi155=y;smithi179=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f25ac0609e0 con 0x7f2594038c80 2023-07-18T02:59:24.256 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.283+0000 7f25a8ff9700 1 -- 172.21.15.179:0/2821367290 <== mgr.14150 v2:172.21.15.101:6800/3963742842 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f25ac0609e0 con 0x7f2594038c80 2023-07-18T02:59:24.256 INFO:teuthology.orchestra.run.smithi179.stdout:Scheduled mgr update... 2023-07-18T02:59:24.258 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.285+0000 7f25b214a700 1 -- 172.21.15.179:0/2821367290 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f2594038c80 msgr2=0x7f259403b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:59:24.258 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.285+0000 7f25b214a700 1 --2- 172.21.15.179:0/2821367290 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f2594038c80 0x7f259403b130 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f25a0006010 tx=0x7f25a00058e0 comp rx=0 tx=0).stop 2023-07-18T02:59:24.258 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.285+0000 7f25b214a700 1 -- 172.21.15.179:0/2821367290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25ac197730 msgr2=0x7f25ac199b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:59:24.259 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.285+0000 7f25b214a700 1 --2- 172.21.15.179:0/2821367290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25ac197730 0x7f25ac199b10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f259c00b780 tx=0x7f259c00ba90 comp rx=0 tx=0).stop 2023-07-18T02:59:24.259 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.285+0000 7f25b214a700 1 -- 172.21.15.179:0/2821367290 shutdown_connections 2023-07-18T02:59:24.259 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.285+0000 7f25b214a700 1 --2- 172.21.15.179:0/2821367290 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f2594038c80 0x7f259403b130 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:59:24.259 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.285+0000 7f25b214a700 1 --2- 172.21.15.179:0/2821367290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25ac0fe600 0x7f25ac069060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:59:24.259 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.285+0000 7f25b214a700 1 --2- 172.21.15.179:0/2821367290 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f25ac19a0a0 0x7f25ac19c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:59:24.260 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.285+0000 7f25b214a700 1 --2- 172.21.15.179:0/2821367290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25ac197730 0x7f25ac199b10 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:59:24.260 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.285+0000 7f25b214a700 1 -- 172.21.15.179:0/2821367290 >> 172.21.15.179:0/2821367290 conn(0x7f25ac0f9bb0 msgr2=0x7f25ac0fa880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:59:24.260 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.285+0000 7f25b214a700 1 -- 172.21.15.179:0/2821367290 shutdown_connections 2023-07-18T02:59:24.260 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T02:59:24.285+0000 7f25b214a700 1 -- 172.21.15.179:0/2821367290 wait complete. 2023-07-18T02:59:24.405 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:24 smithi179 ceph-mon[129229]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:24.405 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:24 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:24.405 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:24 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:24.405 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:24 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-18T02:59:24.406 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:24 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-18T02:59:24.406 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:24 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:24.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:24 smithi101 ceph-mon[126379]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:24.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:24 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:24.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:24 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:24.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:24 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-18T02:59:24.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:24 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-18T02:59:24.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:24 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:24.524 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:24 smithi155 ceph-mon[128885]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:24.524 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:24 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:24.525 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:24 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:24.525 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:24 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-18T02:59:24.525 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:24 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-18T02:59:24.525 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:24 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:24.980 DEBUG:teuthology.orchestra.run.smithi155:mgr.y> sudo journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mgr.y.service 2023-07-18T02:59:24.983 DEBUG:teuthology.orchestra.run.smithi179:mgr.z> sudo journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mgr.z.service 2023-07-18T02:59:24.985 INFO:tasks.cephadm:Deploying OSDs... 2023-07-18T02:59:24.985 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-07-18T02:59:24.985 DEBUG:teuthology.orchestra.run.smithi101:> dd if=/scratch_devs of=/dev/stdout 2023-07-18T02:59:25.013 INFO:journalctl@ceph.mgr.z.smithi179.stdout:-- Logs begin at Tue 2023-07-18 02:29:48 UTC. -- 2023-07-18T02:59:25.017 INFO:journalctl@ceph.mgr.y.smithi155.stdout:-- Logs begin at Tue 2023-07-18 02:29:36 UTC. -- 2023-07-18T02:59:25.032 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-18T02:59:25.033 DEBUG:teuthology.orchestra.run.smithi101:> stat /dev/vg_nvme/lv_1 2023-07-18T02:59:25.121 INFO:teuthology.orchestra.run.smithi101.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-18T02:59:25.121 INFO:teuthology.orchestra.run.smithi101.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-18T02:59:25.121 INFO:teuthology.orchestra.run.smithi101.stdout:Device: 5h/5d Inode: 446 Links: 1 2023-07-18T02:59:25.121 INFO:teuthology.orchestra.run.smithi101.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-18T02:59:25.121 INFO:teuthology.orchestra.run.smithi101.stdout:Context: system_u:object_r:device_t:s0 2023-07-18T02:59:25.122 INFO:teuthology.orchestra.run.smithi101.stdout:Access: 2023-07-18 02:58:34.440675157 +0000 2023-07-18T02:59:25.122 INFO:teuthology.orchestra.run.smithi101.stdout:Modify: 2023-07-18 02:46:59.336446854 +0000 2023-07-18T02:59:25.122 INFO:teuthology.orchestra.run.smithi101.stdout:Change: 2023-07-18 02:46:59.336446854 +0000 2023-07-18T02:59:25.122 INFO:teuthology.orchestra.run.smithi101.stdout: Birth: 2023-07-18 02:46:59.307447526 +0000 2023-07-18T02:59:25.122 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-18T02:59:25.228 INFO:teuthology.orchestra.run.smithi101.stderr:1+0 records in 2023-07-18T02:59:25.228 INFO:teuthology.orchestra.run.smithi101.stderr:1+0 records out 2023-07-18T02:59:25.228 INFO:teuthology.orchestra.run.smithi101.stderr:512 bytes copied, 0.000332757 s, 1.5 MB/s 2023-07-18T02:59:25.230 DEBUG:teuthology.orchestra.run.smithi101:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-18T02:59:25.275 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:25 smithi155 ceph-mon[128885]: Reconfiguring mon.c (monmap changed)... 2023-07-18T02:59:25.275 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:25 smithi155 ceph-mon[128885]: Reconfiguring daemon mon.c on smithi179 2023-07-18T02:59:25.275 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:25 smithi155 ceph-mon[128885]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi101=x;smithi155=y;smithi179=z", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T02:59:25.275 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:25 smithi155 ceph-mon[128885]: Saving service mgr spec with placement smithi101=x;smithi155=y;smithi179=z;count:3 2023-07-18T02:59:25.275 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:25 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:25.276 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:25 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:25.276 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:25 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:25.276 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:25 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T02:59:25.276 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:25 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:25.276 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:25 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T02:59:25.276 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:25 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:25.277 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:25 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-18T02:59:25.277 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:25 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-18T02:59:25.277 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:25 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-18T02:59:25.277 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:25 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:25.278 DEBUG:teuthology.orchestra.run.smithi101:> stat /dev/vg_nvme/lv_2 2023-07-18T02:59:25.361 INFO:teuthology.orchestra.run.smithi101.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-18T02:59:25.361 INFO:teuthology.orchestra.run.smithi101.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-18T02:59:25.361 INFO:teuthology.orchestra.run.smithi101.stdout:Device: 5h/5d Inode: 452 Links: 1 2023-07-18T02:59:25.361 INFO:teuthology.orchestra.run.smithi101.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-18T02:59:25.361 INFO:teuthology.orchestra.run.smithi101.stdout:Context: system_u:object_r:device_t:s0 2023-07-18T02:59:25.362 INFO:teuthology.orchestra.run.smithi101.stdout:Access: 2023-07-18 02:58:34.440675157 +0000 2023-07-18T02:59:25.362 INFO:teuthology.orchestra.run.smithi101.stdout:Modify: 2023-07-18 02:47:00.161427730 +0000 2023-07-18T02:59:25.362 INFO:teuthology.orchestra.run.smithi101.stdout:Change: 2023-07-18 02:47:00.161427730 +0000 2023-07-18T02:59:25.362 INFO:teuthology.orchestra.run.smithi101.stdout: Birth: 2023-07-18 02:47:00.136428309 +0000 2023-07-18T02:59:25.362 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-18T02:59:25.466 INFO:teuthology.orchestra.run.smithi101.stderr:1+0 records in 2023-07-18T02:59:25.501 INFO:teuthology.orchestra.run.smithi101.stderr:1+0 records out 2023-07-18T02:59:25.501 INFO:teuthology.orchestra.run.smithi101.stderr:512 bytes copied, 0.000324611 s, 1.6 MB/s 2023-07-18T02:59:25.502 DEBUG:teuthology.orchestra.run.smithi101:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-18T02:59:25.542 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:25 smithi179 ceph-mon[129229]: Reconfiguring mon.c (monmap changed)... 2023-07-18T02:59:25.542 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:25 smithi179 ceph-mon[129229]: Reconfiguring daemon mon.c on smithi179 2023-07-18T02:59:25.542 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:25 smithi179 ceph-mon[129229]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi101=x;smithi155=y;smithi179=z", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T02:59:25.542 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:25 smithi179 ceph-mon[129229]: Saving service mgr spec with placement smithi101=x;smithi155=y;smithi179=z;count:3 2023-07-18T02:59:25.542 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:25 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:25.542 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:25 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:25.543 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:25 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:25.543 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:25 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T02:59:25.543 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:25 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:25.543 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:25 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T02:59:25.543 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:25 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:25.543 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:25 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-18T02:59:25.543 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:25 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-18T02:59:25.544 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:25 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-18T02:59:25.544 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:25 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:25.556 DEBUG:teuthology.orchestra.run.smithi101:> stat /dev/vg_nvme/lv_3 2023-07-18T02:59:25.646 INFO:teuthology.orchestra.run.smithi101.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-18T02:59:25.646 INFO:teuthology.orchestra.run.smithi101.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-18T02:59:25.646 INFO:teuthology.orchestra.run.smithi101.stdout:Device: 5h/5d Inode: 458 Links: 1 2023-07-18T02:59:25.646 INFO:teuthology.orchestra.run.smithi101.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-18T02:59:25.647 INFO:teuthology.orchestra.run.smithi101.stdout:Context: system_u:object_r:device_t:s0 2023-07-18T02:59:25.647 INFO:teuthology.orchestra.run.smithi101.stdout:Access: 2023-07-18 02:58:34.440675157 +0000 2023-07-18T02:59:25.647 INFO:teuthology.orchestra.run.smithi101.stdout:Modify: 2023-07-18 02:47:00.993408443 +0000 2023-07-18T02:59:25.647 INFO:teuthology.orchestra.run.smithi101.stdout:Change: 2023-07-18 02:47:00.993408443 +0000 2023-07-18T02:59:25.647 INFO:teuthology.orchestra.run.smithi101.stdout: Birth: 2023-07-18 02:47:00.966409069 +0000 2023-07-18T02:59:25.648 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-18T02:59:25.717 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:25 smithi101 ceph-mon[126379]: Reconfiguring mon.c (monmap changed)... 2023-07-18T02:59:25.717 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:25 smithi101 ceph-mon[126379]: Reconfiguring daemon mon.c on smithi179 2023-07-18T02:59:25.717 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:25 smithi101 ceph-mon[126379]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi101=x;smithi155=y;smithi179=z", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T02:59:25.717 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:25 smithi101 ceph-mon[126379]: Saving service mgr spec with placement smithi101=x;smithi155=y;smithi179=z;count:3 2023-07-18T02:59:25.717 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:25 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:25.718 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:25 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:25.718 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:25 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:25.718 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:25 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T02:59:25.718 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:25 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:25.718 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:25 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T02:59:25.718 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:25 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:25.718 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:25 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-18T02:59:25.719 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:25 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-18T02:59:25.719 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:25 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-18T02:59:25.719 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:25 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:25.761 INFO:teuthology.orchestra.run.smithi101.stderr:1+0 records in 2023-07-18T02:59:25.762 INFO:teuthology.orchestra.run.smithi101.stderr:1+0 records out 2023-07-18T02:59:25.762 INFO:teuthology.orchestra.run.smithi101.stderr:512 bytes copied, 0.000312976 s, 1.6 MB/s 2023-07-18T02:59:25.764 DEBUG:teuthology.orchestra.run.smithi101:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-18T02:59:25.853 DEBUG:teuthology.orchestra.run.smithi101:> stat /dev/vg_nvme/lv_4 2023-07-18T02:59:25.939 INFO:teuthology.orchestra.run.smithi101.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-18T02:59:25.940 INFO:teuthology.orchestra.run.smithi101.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-18T02:59:25.940 INFO:teuthology.orchestra.run.smithi101.stdout:Device: 5h/5d Inode: 464 Links: 1 2023-07-18T02:59:25.941 INFO:teuthology.orchestra.run.smithi101.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-18T02:59:25.941 INFO:teuthology.orchestra.run.smithi101.stdout:Context: system_u:object_r:device_t:s0 2023-07-18T02:59:25.941 INFO:teuthology.orchestra.run.smithi101.stdout:Access: 2023-07-18 02:58:34.440675157 +0000 2023-07-18T02:59:25.942 INFO:teuthology.orchestra.run.smithi101.stdout:Modify: 2023-07-18 02:47:01.826389134 +0000 2023-07-18T02:59:25.942 INFO:teuthology.orchestra.run.smithi101.stdout:Change: 2023-07-18 02:47:01.826389134 +0000 2023-07-18T02:59:25.942 INFO:teuthology.orchestra.run.smithi101.stdout: Birth: 2023-07-18 02:47:01.799389760 +0000 2023-07-18T02:59:25.943 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-18T02:59:26.045 INFO:teuthology.orchestra.run.smithi101.stderr:1+0 records in 2023-07-18T02:59:26.045 INFO:teuthology.orchestra.run.smithi101.stderr:1+0 records out 2023-07-18T02:59:26.045 INFO:teuthology.orchestra.run.smithi101.stderr:512 bytes copied, 0.000275816 s, 1.9 MB/s 2023-07-18T02:59:26.047 DEBUG:teuthology.orchestra.run.smithi101:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-18T02:59:26.092 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-07-18T02:59:26.093 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/scratch_devs of=/dev/stdout 2023-07-18T02:59:26.113 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-18T02:59:26.114 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_1 2023-07-18T02:59:26.173 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-18T02:59:26.173 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-18T02:59:26.173 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 6h/6d Inode: 376988 Links: 1 2023-07-18T02:59:26.173 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-18T02:59:26.174 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2023-07-18T02:59:26.174 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2023-07-18 02:59:11.583766116 +0000 2023-07-18T02:59:26.174 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2023-07-18 02:44:29.216511267 +0000 2023-07-18T02:59:26.174 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2023-07-18 02:44:29.216511267 +0000 2023-07-18T02:59:26.174 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: - 2023-07-18T02:59:26.174 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-18T02:59:26.243 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2023-07-18T02:59:26.244 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2023-07-18T02:59:26.244 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000332568 s, 1.5 MB/s 2023-07-18T02:59:26.245 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-18T02:59:26.305 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_2 2023-07-18T02:59:26.363 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-18T02:59:26.364 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-18T02:59:26.364 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 6h/6d Inode: 380680 Links: 1 2023-07-18T02:59:26.364 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-18T02:59:26.364 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2023-07-18T02:59:26.365 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2023-07-18 02:59:11.583766116 +0000 2023-07-18T02:59:26.365 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2023-07-18 02:44:29.753499910 +0000 2023-07-18T02:59:26.365 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2023-07-18 02:44:29.753499910 +0000 2023-07-18T02:59:26.365 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: - 2023-07-18T02:59:26.365 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-18T02:59:26.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:26 smithi179 ceph-mon[129229]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:26.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:26 smithi179 ceph-mon[129229]: Deploying daemon mgr.z on smithi179 2023-07-18T02:59:26.431 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 02:59:26 smithi179 systemd[1]: Starting Ceph mgr.z for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T02:59:26.431 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2023-07-18T02:59:26.431 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2023-07-18T02:59:26.431 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000244437 s, 2.1 MB/s 2023-07-18T02:59:26.432 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-18T02:59:26.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:26 smithi101 ceph-mon[126379]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:26.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:26 smithi101 ceph-mon[126379]: Deploying daemon mgr.z on smithi179 2023-07-18T02:59:26.490 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_3 2023-07-18T02:59:26.525 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:26 smithi155 ceph-mon[128885]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:26.525 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:26 smithi155 ceph-mon[128885]: Deploying daemon mgr.z on smithi179 2023-07-18T02:59:26.541 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-18T02:59:26.542 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-18T02:59:26.542 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 6h/6d Inode: 377039 Links: 1 2023-07-18T02:59:26.542 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-18T02:59:26.542 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2023-07-18T02:59:26.542 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2023-07-18 02:59:11.583766116 +0000 2023-07-18T02:59:26.542 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2023-07-18 02:44:30.308488198 +0000 2023-07-18T02:59:26.543 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2023-07-18 02:44:30.308488198 +0000 2023-07-18T02:59:26.543 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: - 2023-07-18T02:59:26.543 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-18T02:59:26.607 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2023-07-18T02:59:26.608 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2023-07-18T02:59:26.608 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000219993 s, 2.3 MB/s 2023-07-18T02:59:26.609 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-18T02:59:26.667 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_4 2023-07-18T02:59:26.724 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-18T02:59:26.724 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-18T02:59:26.724 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 6h/6d Inode: 373351 Links: 1 2023-07-18T02:59:26.724 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-18T02:59:26.724 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2023-07-18T02:59:26.725 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2023-07-18 02:59:11.583766116 +0000 2023-07-18T02:59:26.725 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2023-07-18 02:44:30.865476563 +0000 2023-07-18T02:59:26.725 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2023-07-18 02:44:30.865476563 +0000 2023-07-18T02:59:26.725 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: - 2023-07-18T02:59:26.725 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-18T02:59:26.768 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 02:59:26 smithi179 podman[131077]: 2023-07-18T02:59:26.789 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2023-07-18T02:59:26.789 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2023-07-18T02:59:26.789 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000174552 s, 2.9 MB/s 2023-07-18T02:59:26.790 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-18T02:59:26.848 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2023-07-18T02:59:26.848 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/scratch_devs of=/dev/stdout 2023-07-18T02:59:26.865 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-18T02:59:26.866 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_1 2023-07-18T02:59:26.923 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-18T02:59:26.923 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-18T02:59:26.923 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 6h/6d Inode: 378739 Links: 1 2023-07-18T02:59:26.924 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-18T02:59:26.924 INFO:teuthology.orchestra.run.smithi179.stdout:Context: system_u:object_r:device_t:s0 2023-07-18T02:59:26.924 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2023-07-18 02:59:26.207385414 +0000 2023-07-18T02:59:26.924 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2023-07-18 02:51:00.181116186 +0000 2023-07-18T02:59:26.924 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2023-07-18 02:51:00.181116186 +0000 2023-07-18T02:59:26.924 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: - 2023-07-18T02:59:26.925 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-18T02:59:26.993 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2023-07-18T02:59:26.993 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2023-07-18T02:59:26.993 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.00024968 s, 2.1 MB/s 2023-07-18T02:59:26.994 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-18T02:59:27.052 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_2 2023-07-18T02:59:27.065 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 02:59:26 smithi179 bash[131077]: d36ff5685165b39e66a23d05bd71c3ecb324d53ecfe2de0603a3c85c02baf0d2 2023-07-18T02:59:27.065 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 02:59:26 smithi179 systemd[1]: Started Ceph mgr.z for b1782daa-2516-11ee-9b34-001a4aab830c. 2023-07-18T02:59:27.066 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 02:59:26 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T02:59:26.915+0000 7fcaba485200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-18T02:59:27.066 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 02:59:26 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T02:59:26.998+0000 7fcaba485200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-18T02:59:27.066 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 02:59:27 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T02:59:27.092+0000 7fcaba485200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-18T02:59:27.082 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-18T02:59:27.083 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-18T02:59:27.083 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 6h/6d Inode: 374374 Links: 1 2023-07-18T02:59:27.083 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-18T02:59:27.083 INFO:teuthology.orchestra.run.smithi179.stdout:Context: system_u:object_r:device_t:s0 2023-07-18T02:59:27.083 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2023-07-18 02:59:26.207385414 +0000 2023-07-18T02:59:27.084 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2023-07-18 02:51:00.697104670 +0000 2023-07-18T02:59:27.084 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2023-07-18 02:51:00.697104670 +0000 2023-07-18T02:59:27.084 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: - 2023-07-18T02:59:27.084 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-18T02:59:27.148 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2023-07-18T02:59:27.148 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2023-07-18T02:59:27.148 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000246297 s, 2.1 MB/s 2023-07-18T02:59:27.149 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-18T02:59:27.206 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_3 2023-07-18T02:59:27.264 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-18T02:59:27.264 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-18T02:59:27.265 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 6h/6d Inode: 382230 Links: 1 2023-07-18T02:59:27.265 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-18T02:59:27.265 INFO:teuthology.orchestra.run.smithi179.stdout:Context: system_u:object_r:device_t:s0 2023-07-18T02:59:27.266 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2023-07-18 02:59:26.207385414 +0000 2023-07-18T02:59:27.266 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2023-07-18 02:51:01.204093355 +0000 2023-07-18T02:59:27.266 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2023-07-18 02:51:01.204093355 +0000 2023-07-18T02:59:27.266 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: - 2023-07-18T02:59:27.267 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-18T02:59:27.327 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 02:59:27 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T02:59:27.191+0000 7fcaba485200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-18T02:59:27.329 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2023-07-18T02:59:27.329 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2023-07-18T02:59:27.329 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000236534 s, 2.2 MB/s 2023-07-18T02:59:27.330 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-18T02:59:27.388 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_4 2023-07-18T02:59:27.445 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-18T02:59:27.446 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-18T02:59:27.446 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 6h/6d Inode: 381349 Links: 1 2023-07-18T02:59:27.446 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-18T02:59:27.446 INFO:teuthology.orchestra.run.smithi179.stdout:Context: system_u:object_r:device_t:s0 2023-07-18T02:59:27.446 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2023-07-18 02:59:26.207385414 +0000 2023-07-18T02:59:27.446 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2023-07-18 02:51:01.716081929 +0000 2023-07-18T02:59:27.447 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2023-07-18 02:51:01.716081929 +0000 2023-07-18T02:59:27.447 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: - 2023-07-18T02:59:27.447 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-18T02:59:27.510 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2023-07-18T02:59:27.511 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2023-07-18T02:59:27.511 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000190632 s, 2.7 MB/s 2023-07-18T02:59:27.512 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-18T02:59:27.569 INFO:tasks.cephadm:Deploying osd.0 on smithi101 with /dev/vg_nvme/lv_4... 2023-07-18T02:59:27.570 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-07-18T02:59:27.658 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 02:59:27 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T02:59:27.367+0000 7fcaba485200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-18T02:59:27.929 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 02:59:27 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T02:59:27.685+0000 7fcaba485200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-18T02:59:27.930 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 02:59:27 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T02:59:27.774+0000 7fcaba485200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-18T02:59:27.930 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 02:59:27 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T02:59:27.908+0000 7fcaba485200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-18T02:59:27.930 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:27 smithi179 ceph-mon[129229]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:27.931 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:27 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:27.931 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:27 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:27.931 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:27 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:27.931 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:27 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-18T02:59:27.931 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:27 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-18T02:59:27.931 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:27 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-18T02:59:27.932 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:27 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:27.932 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:27 smithi179 ceph-mon[129229]: Deploying daemon mgr.y on smithi155 2023-07-18T02:59:27.967 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:27 smithi101 ceph-mon[126379]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:27.967 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:27 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:27.967 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:27 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:27.967 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:27 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:27.967 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:27 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-18T02:59:27.968 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:27 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-18T02:59:27.968 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:27 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-18T02:59:27.968 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:27 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:27.968 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:27 smithi101 ceph-mon[126379]: Deploying daemon mgr.y on smithi155 2023-07-18T02:59:28.023 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:27 smithi155 ceph-mon[128885]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:28.024 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:27 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:28.024 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:27 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:28.024 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:27 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:28.024 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:27 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-18T02:59:28.024 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:27 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-18T02:59:28.024 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:27 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-18T02:59:28.024 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:27 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:28.025 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:27 smithi155 ceph-mon[128885]: Deploying daemon mgr.y on smithi155 2023-07-18T02:59:28.294 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T02:59:28.391 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 02:59:28 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T02:59:28.002+0000 7fcaba485200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-18T02:59:28.679 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 02:59:28 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T02:59:28.418+0000 7fcaba485200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-18T02:59:28.774 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 02:59:28 smithi155 systemd[1]: Starting Ceph mgr.y for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T02:59:29.140 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 02:59:28 smithi155 podman[130322]: 2023-07-18T02:59:29.429 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 02:59:28 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T02:59:28.985+0000 7fcaba485200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-18T02:59:29.429 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 02:59:29 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T02:59:29.155+0000 7fcaba485200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-18T02:59:29.430 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 02:59:29 smithi155 ceph-mgr[130362]: -- 172.21.15.155:0/2137121427 <== mon.2 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55afa423b1e0 con 0x55afa4211000 2023-07-18T02:59:29.430 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 02:59:29 smithi155 bash[130322]: 41afb61dbbcd6be8fe4e3c33a6871175212aa704bfd09125135cb44f8cb2eb0f 2023-07-18T02:59:29.430 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 02:59:29 smithi155 systemd[1]: Started Ceph mgr.y for b1782daa-2516-11ee-9b34-001a4aab830c. 2023-07-18T02:59:29.430 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 02:59:29 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T02:59:29.285+0000 7f4578b33200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-18T02:59:29.431 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 02:59:29 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T02:59:29.369+0000 7f4578b33200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-18T02:59:29.431 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 02:59:29 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T02:59:29.457+0000 7f4578b33200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-18T02:59:29.702 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 02:59:29 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T02:59:29.554+0000 7f4578b33200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-18T02:59:30.024 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 02:59:29 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T02:59:29.730+0000 7f4578b33200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-18T02:59:30.179 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 02:59:29 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T02:59:29.940+0000 7fcaba485200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-18T02:59:30.180 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 02:59:30 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T02:59:30.032+0000 7fcaba485200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-18T02:59:30.294 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 02:59:30 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T02:59:30.055+0000 7f4578b33200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-18T02:59:30.294 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 02:59:30 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T02:59:30.144+0000 7f4578b33200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-18T02:59:30.294 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 02:59:30 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T02:59:30.281+0000 7f4578b33200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-18T02:59:30.343 INFO:teuthology.orchestra.run.smithi101.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-07-18T02:59:30.343 INFO:teuthology.orchestra.run.smithi101.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-07-18T02:59:30.343 INFO:teuthology.orchestra.run.smithi101.stderr: stderr: 10+0 records in 2023-07-18T02:59:30.343 INFO:teuthology.orchestra.run.smithi101.stderr:10+0 records out 2023-07-18T02:59:30.343 INFO:teuthology.orchestra.run.smithi101.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0421356 s, 249 MB/s 2023-07-18T02:59:30.343 INFO:teuthology.orchestra.run.smithi101.stderr:--> Zapping successful for: 2023-07-18T02:59:30.478 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 02:59:30 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T02:59:30.207+0000 7fcaba485200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-18T02:59:30.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:30 smithi179 ceph-mon[129229]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:30.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:30 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:30.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:30 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:30.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:30 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:30.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:30 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:30.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:30 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T02:59:30.717 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:30 smithi101 ceph-mon[126379]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:30.718 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:30 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:30.718 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:30 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:30.718 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:30 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:30.718 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:30 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:30.718 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:30 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T02:59:30.774 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:30 smithi155 ceph-mon[128885]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:30.775 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:30 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:30.775 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:30 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:30.775 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:30 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:30.775 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:30 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:30.775 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:30 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T02:59:30.775 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 02:59:30 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T02:59:30.381+0000 7f4578b33200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-18T02:59:30.867 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi101:vg_nvme/lv_4 2023-07-18T02:59:30.868 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 02:59:30 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T02:59:30.505+0000 7fcaba485200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-18T02:59:30.868 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 02:59:30 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T02:59:30.744+0000 7fcaba485200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-18T02:59:30.868 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 02:59:30 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T02:59:30.893+0000 7fcaba485200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-18T02:59:31.274 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 02:59:30 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T02:59:30.835+0000 7f4578b33200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-18T02:59:31.430 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 02:59:31 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T02:59:31.164+0000 7fcaba485200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-18T02:59:31.591 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T02:59:31.774 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 02:59:31 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T02:59:31.436+0000 7f4578b33200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-18T02:59:31.774 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 02:59:31 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T02:59:31.605+0000 7f4578b33200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-18T02:59:32.153 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 02:59:31 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T02:59:31.861+0000 7fcaba485200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-18T02:59:32.153 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 02:59:31 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T02:59:31.990+0000 7fcaba485200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-18T02:59:32.154 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:32 smithi179 ceph-mon[129229]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:32.154 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:32 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:32.154 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:32 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:32.154 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:32 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:32.154 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:32 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:32.154 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:32 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:32.155 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:32 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T02:59:32.155 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:32 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:32.155 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:32 smithi179 ceph-mon[129229]: Reconfiguring mgr.x (unknown last config time)... 2023-07-18T02:59:32.155 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:32 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-18T02:59:32.155 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:32 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-18T02:59:32.155 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:32 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:32.155 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:32 smithi179 ceph-mon[129229]: Reconfiguring daemon mgr.x on smithi101 2023-07-18T02:59:32.217 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:32 smithi101 ceph-mon[126379]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:32.217 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:32 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:32.217 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:32 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:32.217 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:32 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:32.217 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:32 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:32.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:32 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:32.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:32 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T02:59:32.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:32 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:32.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:32 smithi101 ceph-mon[126379]: Reconfiguring mgr.x (unknown last config time)... 2023-07-18T02:59:32.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:32 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-18T02:59:32.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:32 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-18T02:59:32.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:32 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:32.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:32 smithi101 ceph-mon[126379]: Reconfiguring daemon mgr.x on smithi101 2023-07-18T02:59:32.334 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:32 smithi155 ceph-mon[128885]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:32.334 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:32 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:32.334 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:32 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:32.334 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:32 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:32.334 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:32 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:32.334 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:32 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:32.335 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:32 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T02:59:32.335 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:32 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:32.335 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:32 smithi155 ceph-mon[128885]: Reconfiguring mgr.x (unknown last config time)... 2023-07-18T02:59:32.335 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:32 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-18T02:59:32.335 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:32 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-18T02:59:32.335 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:32 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:32.336 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:32 smithi155 ceph-mon[128885]: Reconfiguring daemon mgr.x on smithi101 2023-07-18T02:59:32.429 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 02:59:32 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T02:59:32.181+0000 7fcaba485200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-18T02:59:32.429 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 02:59:32 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T02:59:32.294+0000 7fcaba485200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-18T02:59:32.590 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 02:59:32 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T02:59:32.361+0000 7f4578b33200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-18T02:59:32.590 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 02:59:32 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T02:59:32.451+0000 7f4578b33200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-18T02:59:32.591 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 02:59:32 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T02:59:32.618+0000 7f4578b33200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-18T02:59:32.929 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 02:59:32 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T02:59:32.507+0000 7fcaba485200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-18T02:59:33.231 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 02:59:32 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T02:59:32.889+0000 7f4578b33200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-18T02:59:33.231 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 02:59:33 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T02:59:33.114+0000 7f4578b33200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-18T02:59:33.231 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 02:59:33 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T02:59:33.259+0000 7f4578b33200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-18T02:59:33.679 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 02:59:33 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T02:59:33.357+0000 7fcaba485200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-18T02:59:33.774 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 02:59:33 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T02:59:33.515+0000 7f4578b33200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-18T02:59:33.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:33.822+0000 7f0924282700 1 -- 172.21.15.101:0/415683350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f091c107a50 msgr2=0x7f091c109ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:59:33.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:33.822+0000 7f0924282700 1 --2- 172.21.15.101:0/415683350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f091c107a50 0x7f091c109ed0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f091400b0a0 tx=0x7f091400b3b0 comp rx=0 tx=0).stop 2023-07-18T02:59:33.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:33.822+0000 7f0924282700 1 -- 172.21.15.101:0/415683350 shutdown_connections 2023-07-18T02:59:33.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:33.822+0000 7f0924282700 1 --2- 172.21.15.101:0/415683350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f091c0744c0 0x7f091c072920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:59:33.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:33.822+0000 7f0924282700 1 --2- 172.21.15.101:0/415683350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f091c107a50 0x7f091c109ed0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:59:33.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:33.822+0000 7f0924282700 1 --2- 172.21.15.101:0/415683350 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f091c072ef0 0x7f091c073360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:59:33.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:33.822+0000 7f0924282700 1 -- 172.21.15.101:0/415683350 >> 172.21.15.101:0/415683350 conn(0x7f091c0fb1a0 msgr2=0x7f091c0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:59:33.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:33.822+0000 7f0924282700 1 -- 172.21.15.101:0/415683350 shutdown_connections 2023-07-18T02:59:33.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:33.822+0000 7f0924282700 1 -- 172.21.15.101:0/415683350 wait complete. 2023-07-18T02:59:33.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:33.824+0000 7f0924282700 1 Processor -- start 2023-07-18T02:59:33.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:33.824+0000 7f0924282700 1 -- start start 2023-07-18T02:59:33.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:33.824+0000 7f0924282700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f091c072ef0 0x7f091c197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:59:33.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:33.824+0000 7f0924282700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f091c0744c0 0x7f091c198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:59:33.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:33.824+0000 7f0924282700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f091c107a50 0x7f091c19f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:59:33.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:33.824+0000 7f0924282700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091c19f790 con 0x7f091c072ef0 2023-07-18T02:59:33.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:33.824+0000 7f0924282700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091c19f8d0 con 0x7f091c107a50 2023-07-18T02:59:33.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:33.824+0000 7f0924282700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091c19fa90 con 0x7f091c0744c0 2023-07-18T02:59:33.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:33.824+0000 7f092181d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f091c0744c0 0x7f091c198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:59:33.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:33.824+0000 7f092201e700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f091c072ef0 0x7f091c197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:59:33.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:33.824+0000 7f092181d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f091c0744c0 0x7f091c198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:55700/0 (socket says 172.21.15.101:55700) 2023-07-18T02:59:33.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:33.824+0000 7f092181d700 1 -- 172.21.15.101:0/2306888856 learned_addr learned my addr 172.21.15.101:0/2306888856 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T02:59:33.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:33.824+0000 7f092281f700 1 --2- 172.21.15.101:0/2306888856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f091c107a50 0x7f091c19f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:59:33.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:33.825+0000 7f092181d700 1 -- 172.21.15.101:0/2306888856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f091c107a50 msgr2=0x7f091c19f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:59:33.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:33.825+0000 7f092181d700 1 --2- 172.21.15.101:0/2306888856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f091c107a50 0x7f091c19f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:59:33.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:33.825+0000 7f092181d700 1 -- 172.21.15.101:0/2306888856 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f091c072ef0 msgr2=0x7f091c197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:59:33.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:33.825+0000 7f092181d700 1 --2- 172.21.15.101:0/2306888856 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f091c072ef0 0x7f091c197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:59:33.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:33.825+0000 7f092181d700 1 -- 172.21.15.101:0/2306888856 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0914009d00 con 0x7f091c0744c0 2023-07-18T02:59:33.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:33.825+0000 7f092201e700 1 --2- 172.21.15.101:0/2306888856 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f091c072ef0 0x7f091c197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T02:59:33.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:33.825+0000 7f092281f700 1 --2- 172.21.15.101:0/2306888856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f091c107a50 0x7f091c19f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T02:59:33.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:33.825+0000 7f092181d700 1 --2- 172.21.15.101:0/2306888856 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f091c0744c0 0x7f091c198180 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f090c00e9a0 tx=0x7f090c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:59:33.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:33.825+0000 7f09137fe700 1 -- 172.21.15.101:0/2306888856 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090c009790 con 0x7f091c0744c0 2023-07-18T02:59:33.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:33.825+0000 7f09137fe700 1 -- 172.21.15.101:0/2306888856 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f090c010430 con 0x7f091c0744c0 2023-07-18T02:59:33.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:33.825+0000 7f0924282700 1 -- 172.21.15.101:0/2306888856 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f091c19fd70 con 0x7f091c0744c0 2023-07-18T02:59:33.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:33.825+0000 7f09137fe700 1 -- 172.21.15.101:0/2306888856 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090c018680 con 0x7f091c0744c0 2023-07-18T02:59:33.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:33.825+0000 7f0924282700 1 -- 172.21.15.101:0/2306888856 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f091c1a01c0 con 0x7f091c0744c0 2023-07-18T02:59:33.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:33.826+0000 7f09137fe700 1 -- 172.21.15.101:0/2306888856 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f090c00ca10 con 0x7f091c0744c0 2023-07-18T02:59:33.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:33.826+0000 7f0924282700 1 -- 172.21.15.101:0/2306888856 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f091c106190 con 0x7f091c0744c0 2023-07-18T02:59:33.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:33.827+0000 7f09137fe700 1 --2- 172.21.15.101:0/2306888856 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f09080389e0 0x7f090803ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T02:59:33.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:33.827+0000 7f09137fe700 1 -- 172.21.15.101:0/2306888856 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f090c014070 con 0x7f091c0744c0 2023-07-18T02:59:33.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:33.827+0000 7f092201e700 1 --2- 172.21.15.101:0/2306888856 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f09080389e0 0x7f090803ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T02:59:33.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:33.827+0000 7f092201e700 1 --2- 172.21.15.101:0/2306888856 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f09080389e0 0x7f090803ae90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f091c199120 tx=0x7f0918009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T02:59:33.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:33.833+0000 7f09137fe700 1 -- 172.21.15.101:0/2306888856 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f090c00cdb0 con 0x7f091c0744c0 2023-07-18T02:59:34.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:34.036+0000 7f0924282700 1 -- 172.21.15.101:0/2306888856 --> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f091c060950 con 0x7f09080389e0 2023-07-18T02:59:34.179 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:33 smithi179 ceph-mon[129229]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:34.180 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:33 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:34.180 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:33 smithi179 ceph-mon[129229]: Standby manager daemon z started 2023-07-18T02:59:34.180 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:33 smithi179 ceph-mon[129229]: from='mgr.? 172.21.15.179:0/860860242' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-18T02:59:34.180 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:33 smithi179 ceph-mon[129229]: from='mgr.? 172.21.15.179:0/860860242' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-18T02:59:34.180 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:33 smithi179 ceph-mon[129229]: from='mgr.? 172.21.15.179:0/860860242' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-18T02:59:34.180 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:33 smithi179 ceph-mon[129229]: from='mgr.? 172.21.15.179:0/860860242' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-18T02:59:34.181 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:33 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:34.181 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:33 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:34.181 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:33 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T02:59:34.181 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:33 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:34.181 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:33 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T02:59:34.181 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:33 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:34.186 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:33 smithi155 ceph-mon[128885]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:34.186 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:33 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:34.186 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:33 smithi155 ceph-mon[128885]: Standby manager daemon z started 2023-07-18T02:59:34.187 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:33 smithi155 ceph-mon[128885]: from='mgr.? 172.21.15.179:0/860860242' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-18T02:59:34.187 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:33 smithi155 ceph-mon[128885]: from='mgr.? 172.21.15.179:0/860860242' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-18T02:59:34.187 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:33 smithi155 ceph-mon[128885]: from='mgr.? 172.21.15.179:0/860860242' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-18T02:59:34.187 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:33 smithi155 ceph-mon[128885]: from='mgr.? 172.21.15.179:0/860860242' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-18T02:59:34.187 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:33 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:34.187 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:33 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:34.187 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:33 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T02:59:34.187 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:33 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:34.188 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:33 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T02:59:34.188 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:33 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:34.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:34.217+0000 7f09137fe700 1 -- 172.21.15.101:0/2306888856 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mgrmap(e 14) v1 ==== 90600+0+0 (secure 0 0 0) 0x7f090c00acd0 con 0x7f091c0744c0 2023-07-18T02:59:34.217 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:33 smithi101 ceph-mon[126379]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:34.217 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:33 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:34.217 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:33 smithi101 ceph-mon[126379]: Standby manager daemon z started 2023-07-18T02:59:34.217 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:33 smithi101 ceph-mon[126379]: from='mgr.? 172.21.15.179:0/860860242' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-18T02:59:34.217 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:33 smithi101 ceph-mon[126379]: from='mgr.? 172.21.15.179:0/860860242' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-18T02:59:34.217 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:33 smithi101 ceph-mon[126379]: from='mgr.? 172.21.15.179:0/860860242' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-18T02:59:34.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:33 smithi101 ceph-mon[126379]: from='mgr.? 172.21.15.179:0/860860242' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-18T02:59:34.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:33 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:34.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:33 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:34.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:33 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T02:59:34.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:33 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:34.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:33 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T02:59:34.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:33 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:34.502 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 02:59:34 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T02:59:34.214+0000 7f4578b33200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-18T02:59:34.502 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 02:59:34 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T02:59:34.343+0000 7f4578b33200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-18T02:59:34.774 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 02:59:34 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T02:59:34.530+0000 7f4578b33200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-18T02:59:34.774 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 02:59:34 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T02:59:34.642+0000 7f4578b33200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-18T02:59:35.070 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:34 smithi101 ceph-mon[126379]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T02:59:35.071 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:34 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T02:59:35.071 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:34 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T02:59:35.071 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:34 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:35.071 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:34 smithi101 ceph-mon[126379]: mgrmap e14: x(active, since 71s), standbys: z 2023-07-18T02:59:35.072 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:34 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-18T02:59:35.179 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:34 smithi179 ceph-mon[129229]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T02:59:35.180 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:34 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T02:59:35.180 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:34 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T02:59:35.180 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:34 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:35.180 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:34 smithi179 ceph-mon[129229]: mgrmap e14: x(active, since 71s), standbys: z 2023-07-18T02:59:35.180 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:34 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-18T02:59:35.274 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 02:59:34 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T02:59:34.856+0000 7f4578b33200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-18T02:59:35.274 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:34 smithi155 ceph-mon[128885]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T02:59:35.275 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:34 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T02:59:35.275 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:34 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T02:59:35.275 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:34 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:35.275 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:34 smithi155 ceph-mon[128885]: mgrmap e14: x(active, since 71s), standbys: z 2023-07-18T02:59:35.275 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:34 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-18T02:59:36.024 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 02:59:35 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T02:59:35.720+0000 7f4578b33200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-18T02:59:36.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:36.228+0000 7f09137fe700 1 -- 172.21.15.101:0/2306888856 <== mon.1 v2:172.21.15.179:3300/0 8 ==== mgrmap(e 15) v1 ==== 135660+0+0 (secure 0 0 0) 0x7f090c018830 con 0x7f091c0744c0 2023-07-18T02:59:36.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:36 smithi101 ceph-mon[126379]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:36.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:36 smithi101 ceph-mon[126379]: Standby manager daemon y started 2023-07-18T02:59:36.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:36 smithi101 ceph-mon[126379]: from='mgr.? 172.21.15.155:0/1031950608' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-18T02:59:36.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:36 smithi101 ceph-mon[126379]: from='mgr.? 172.21.15.155:0/1031950608' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-18T02:59:36.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:36 smithi101 ceph-mon[126379]: from='mgr.? 172.21.15.155:0/1031950608' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-18T02:59:36.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:36 smithi101 ceph-mon[126379]: from='mgr.? 172.21.15.155:0/1031950608' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-18T02:59:36.524 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:36 smithi155 ceph-mon[128885]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:36.524 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:36 smithi155 ceph-mon[128885]: Standby manager daemon y started 2023-07-18T02:59:36.525 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:36 smithi155 ceph-mon[128885]: from='mgr.? 172.21.15.155:0/1031950608' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-18T02:59:36.525 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:36 smithi155 ceph-mon[128885]: from='mgr.? 172.21.15.155:0/1031950608' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-18T02:59:36.525 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:36 smithi155 ceph-mon[128885]: from='mgr.? 172.21.15.155:0/1031950608' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-18T02:59:36.525 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:36 smithi155 ceph-mon[128885]: from='mgr.? 172.21.15.155:0/1031950608' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-18T02:59:36.679 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:36 smithi179 ceph-mon[129229]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:36.680 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:36 smithi179 ceph-mon[129229]: Standby manager daemon y started 2023-07-18T02:59:36.680 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:36 smithi179 ceph-mon[129229]: from='mgr.? 172.21.15.155:0/1031950608' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-18T02:59:36.680 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:36 smithi179 ceph-mon[129229]: from='mgr.? 172.21.15.155:0/1031950608' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-18T02:59:36.680 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:36 smithi179 ceph-mon[129229]: from='mgr.? 172.21.15.155:0/1031950608' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-18T02:59:36.680 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:36 smithi179 ceph-mon[129229]: from='mgr.? 172.21.15.155:0/1031950608' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-18T02:59:37.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:37 smithi101 ceph-mon[126379]: mgrmap e15: x(active, since 73s), standbys: z, y 2023-07-18T02:59:37.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:37 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-18T02:59:37.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:37 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/179318791' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc76b5d1-38a0-4f1a-9335-756bfc40a62e"}]: dispatch 2023-07-18T02:59:37.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:37 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/179318791' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dc76b5d1-38a0-4f1a-9335-756bfc40a62e"}]': finished 2023-07-18T02:59:37.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:37 smithi101 ceph-mon[126379]: osdmap e5: 1 total, 0 up, 1 in 2023-07-18T02:59:37.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:37 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-18T02:59:37.524 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:37 smithi155 ceph-mon[128885]: mgrmap e15: x(active, since 73s), standbys: z, y 2023-07-18T02:59:37.524 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:37 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-18T02:59:37.525 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:37 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/179318791' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc76b5d1-38a0-4f1a-9335-756bfc40a62e"}]: dispatch 2023-07-18T02:59:37.525 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:37 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/179318791' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dc76b5d1-38a0-4f1a-9335-756bfc40a62e"}]': finished 2023-07-18T02:59:37.525 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:37 smithi155 ceph-mon[128885]: osdmap e5: 1 total, 0 up, 1 in 2023-07-18T02:59:37.525 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:37 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-18T02:59:37.679 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:37 smithi179 ceph-mon[129229]: mgrmap e15: x(active, since 73s), standbys: z, y 2023-07-18T02:59:37.679 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:37 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-18T02:59:37.680 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:37 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/179318791' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc76b5d1-38a0-4f1a-9335-756bfc40a62e"}]: dispatch 2023-07-18T02:59:37.680 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:37 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/179318791' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dc76b5d1-38a0-4f1a-9335-756bfc40a62e"}]': finished 2023-07-18T02:59:37.680 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:37 smithi179 ceph-mon[129229]: osdmap e5: 1 total, 0 up, 1 in 2023-07-18T02:59:37.680 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:37 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-18T02:59:38.466 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:38 smithi101 ceph-mon[126379]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:38.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:38 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/568241860' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T02:59:38.524 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:38 smithi155 ceph-mon[128885]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:38.524 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:38 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/568241860' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T02:59:38.679 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:38 smithi179 ceph-mon[129229]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:38.680 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:38 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/568241860' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T02:59:40.466 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:40 smithi101 ceph-mon[126379]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:40.524 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:40 smithi155 ceph-mon[128885]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:40.679 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:40 smithi179 ceph-mon[129229]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:42.466 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:42 smithi101 ceph-mon[126379]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:42.524 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:42 smithi155 ceph-mon[128885]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:42.679 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:42 smithi179 ceph-mon[129229]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:44.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:44 smithi101 ceph-mon[126379]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:44.524 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:44 smithi155 ceph-mon[128885]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:44.679 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:44 smithi179 ceph-mon[129229]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:45.524 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:45 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-07-18T02:59:45.524 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:45 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:45.606 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:45 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-07-18T02:59:45.606 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:45 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:45.679 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:45 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-07-18T02:59:45.680 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:45 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:46.466 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:46 smithi101 ceph-mon[126379]: Deploying daemon osd.0 on smithi101 2023-07-18T02:59:46.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:46 smithi101 ceph-mon[126379]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:46.524 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:46 smithi155 ceph-mon[128885]: Deploying daemon osd.0 on smithi101 2023-07-18T02:59:46.524 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:46 smithi155 ceph-mon[128885]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:46.679 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:46 smithi179 ceph-mon[129229]: Deploying daemon osd.0 on smithi101 2023-07-18T02:59:46.680 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:46 smithi179 ceph-mon[129229]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:48.524 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:48 smithi155 ceph-mon[128885]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:48.679 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:48 smithi179 ceph-mon[129229]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:48.717 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:48 smithi101 ceph-mon[126379]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:50.524 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:50 smithi155 ceph-mon[128885]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:50.679 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:50 smithi179 ceph-mon[129229]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:50.718 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:50 smithi101 ceph-mon[126379]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:52.388 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:52 smithi101 ceph-mon[126379]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:52.389 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:52 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:52.389 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:52 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:52.389 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:52 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T02:59:52.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:52 smithi179 ceph-mon[129229]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:52.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:52 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:52.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:52 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:52.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:52 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T02:59:52.524 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:52 smithi155 ceph-mon[128885]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:52.525 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:52 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:52.525 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:52 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:52.525 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:52 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T02:59:54.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:54 smithi101 ceph-mon[126379]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:54.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:54 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:54.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:54 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:54.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:54 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:54.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:54 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T02:59:54.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:54 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:54.524 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:54 smithi155 ceph-mon[128885]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:54.525 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:54 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:54.525 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:54 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:54.525 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:54 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:54.525 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:54 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T02:59:54.525 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:54 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:54.680 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:54 smithi179 ceph-mon[129229]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:54.680 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:54 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:54.681 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:54 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:54.681 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:54 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:54.681 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:54 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T02:59:54.682 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:54 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:54.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:55.007+0000 7f09137fe700 1 -- 172.21.15.101:0/2306888856 <== mgr.14150 v2:172.21.15.101:6800/3963742842 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f091c060950 con 0x7f09080389e0 2023-07-18T02:59:54.981 INFO:teuthology.orchestra.run.smithi101.stdout:Created osd(s) 0 on host 'smithi101' 2023-07-18T02:59:54.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:55.009+0000 7f0924282700 1 -- 172.21.15.101:0/2306888856 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f09080389e0 msgr2=0x7f090803ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:59:54.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:55.009+0000 7f0924282700 1 --2- 172.21.15.101:0/2306888856 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f09080389e0 0x7f090803ae90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f091c199120 tx=0x7f0918009260 comp rx=0 tx=0).stop 2023-07-18T02:59:54.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:55.009+0000 7f0924282700 1 -- 172.21.15.101:0/2306888856 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f091c0744c0 msgr2=0x7f091c198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T02:59:54.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:55.009+0000 7f0924282700 1 --2- 172.21.15.101:0/2306888856 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f091c0744c0 0x7f091c198180 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f090c00e9a0 tx=0x7f090c00ecb0 comp rx=0 tx=0).stop 2023-07-18T02:59:54.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:55.009+0000 7f0924282700 1 -- 172.21.15.101:0/2306888856 shutdown_connections 2023-07-18T02:59:54.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:55.009+0000 7f0924282700 1 --2- 172.21.15.101:0/2306888856 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f09080389e0 0x7f090803ae90 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:59:54.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:55.009+0000 7f0924282700 1 --2- 172.21.15.101:0/2306888856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f091c107a50 0x7f091c19f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:59:54.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:55.009+0000 7f0924282700 1 --2- 172.21.15.101:0/2306888856 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f091c072ef0 0x7f091c197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:59:54.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:55.009+0000 7f0924282700 1 --2- 172.21.15.101:0/2306888856 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f091c0744c0 0x7f091c198180 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T02:59:54.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:55.009+0000 7f0924282700 1 -- 172.21.15.101:0/2306888856 >> 172.21.15.101:0/2306888856 conn(0x7f091c0fb1a0 msgr2=0x7f091c0fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T02:59:54.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:55.009+0000 7f0924282700 1 -- 172.21.15.101:0/2306888856 shutdown_connections 2023-07-18T02:59:54.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T02:59:55.010+0000 7f0924282700 1 -- 172.21.15.101:0/2306888856 wait complete. 2023-07-18T02:59:55.510 DEBUG:teuthology.orchestra.run.smithi101:osd.0> sudo journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@osd.0.service 2023-07-18T02:59:55.514 INFO:tasks.cephadm:Deploying osd.1 on smithi155 with /dev/vg_nvme/lv_4... 2023-07-18T02:59:55.515 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-07-18T02:59:55.586 INFO:journalctl@ceph.osd.0.smithi101.stdout:-- Logs begin at Tue 2023-07-18 02:35:16 UTC. -- 2023-07-18T02:59:56.075 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.b/config 2023-07-18T02:59:56.217 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:55 smithi101 ceph-mon[126379]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:56.217 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:55 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:56.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:55 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:56.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:55 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T02:59:56.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:55 smithi101 ceph-mon[126379]: from='osd.0 [v2:172.21.15.101:6802/3610719929,v1:172.21.15.101:6803/3610719929]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-18T02:59:56.274 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:55 smithi155 ceph-mon[128885]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:56.275 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:55 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:56.275 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:55 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:56.275 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:55 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T02:59:56.275 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:55 smithi155 ceph-mon[128885]: from='osd.0 [v2:172.21.15.101:6802/3610719929,v1:172.21.15.101:6803/3610719929]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-18T02:59:56.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:55 smithi179 ceph-mon[129229]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:56.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:55 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:56.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:55 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:56.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:55 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T02:59:56.431 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:55 smithi179 ceph-mon[129229]: from='osd.0 [v2:172.21.15.101:6802/3610719929,v1:172.21.15.101:6803/3610719929]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-18T02:59:57.209 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:56 smithi101 ceph-mon[126379]: from='osd.0 [v2:172.21.15.101:6802/3610719929,v1:172.21.15.101:6803/3610719929]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-07-18T02:59:57.209 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:56 smithi101 ceph-mon[126379]: osdmap e6: 1 total, 0 up, 1 in 2023-07-18T02:59:57.210 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:56 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-18T02:59:57.210 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:56 smithi101 ceph-mon[126379]: from='osd.0 [v2:172.21.15.101:6802/3610719929,v1:172.21.15.101:6803/3610719929]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-07-18T02:59:57.210 INFO:journalctl@ceph.osd.0.smithi101.stdout:Jul 18 02:59:56 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-0[139271]: 2023-07-18T02:59:56.988+0000 7f7b5f8cc700 -1 osd.0 0 waiting for initial osdmap 2023-07-18T02:59:57.210 INFO:journalctl@ceph.osd.0.smithi101.stdout:Jul 18 02:59:56 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-0[139271]: 2023-07-18T02:59:56.991+0000 7f7b5b6c1700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-18T02:59:57.220 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:56 smithi155 ceph-mon[128885]: from='osd.0 [v2:172.21.15.101:6802/3610719929,v1:172.21.15.101:6803/3610719929]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-07-18T02:59:57.220 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:56 smithi155 ceph-mon[128885]: osdmap e6: 1 total, 0 up, 1 in 2023-07-18T02:59:57.220 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:56 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-18T02:59:57.221 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:56 smithi155 ceph-mon[128885]: from='osd.0 [v2:172.21.15.101:6802/3610719929,v1:172.21.15.101:6803/3610719929]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-07-18T02:59:57.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:56 smithi179 ceph-mon[129229]: from='osd.0 [v2:172.21.15.101:6802/3610719929,v1:172.21.15.101:6803/3610719929]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-07-18T02:59:57.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:56 smithi179 ceph-mon[129229]: osdmap e6: 1 total, 0 up, 1 in 2023-07-18T02:59:57.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:56 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-18T02:59:57.431 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:56 smithi179 ceph-mon[129229]: from='osd.0 [v2:172.21.15.101:6802/3610719929,v1:172.21.15.101:6803/3610719929]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-07-18T02:59:57.702 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-07-18T02:59:57.703 INFO:teuthology.orchestra.run.smithi155.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-07-18T02:59:57.703 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10+0 records in 2023-07-18T02:59:57.703 INFO:teuthology.orchestra.run.smithi155.stderr:10+0 records out 2023-07-18T02:59:57.703 INFO:teuthology.orchestra.run.smithi155.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011032 s, 950 MB/s 2023-07-18T02:59:57.703 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 2023-07-18T02:59:57.703 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping successful for: 2023-07-18T02:59:58.025 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:57 smithi155 ceph-mon[128885]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:58.026 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:57 smithi155 ceph-mon[128885]: from='osd.0 [v2:172.21.15.101:6802/3610719929,v1:172.21.15.101:6803/3610719929]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]': finished 2023-07-18T02:59:58.026 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:57 smithi155 ceph-mon[128885]: osdmap e7: 1 total, 0 up, 1 in 2023-07-18T02:59:58.026 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:57 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-18T02:59:58.026 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:57 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-18T02:59:58.118 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi155:vg_nvme/lv_4 2023-07-18T02:59:58.217 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:57 smithi101 ceph-mon[126379]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:58.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:57 smithi101 ceph-mon[126379]: from='osd.0 [v2:172.21.15.101:6802/3610719929,v1:172.21.15.101:6803/3610719929]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]': finished 2023-07-18T02:59:58.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:57 smithi101 ceph-mon[126379]: osdmap e7: 1 total, 0 up, 1 in 2023-07-18T02:59:58.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:57 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-18T02:59:58.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:57 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-18T02:59:58.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:57 smithi179 ceph-mon[129229]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T02:59:58.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:57 smithi179 ceph-mon[129229]: from='osd.0 [v2:172.21.15.101:6802/3610719929,v1:172.21.15.101:6803/3610719929]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]': finished 2023-07-18T02:59:58.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:57 smithi179 ceph-mon[129229]: osdmap e7: 1 total, 0 up, 1 in 2023-07-18T02:59:58.431 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:57 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-18T02:59:58.431 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:57 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-18T02:59:58.672 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.b/config 2023-07-18T02:59:59.217 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:58 smithi101 ceph-mon[126379]: purged_snaps scrub starts 2023-07-18T02:59:59.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:58 smithi101 ceph-mon[126379]: purged_snaps scrub ok 2023-07-18T02:59:59.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:58 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-18T02:59:59.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:58 smithi101 ceph-mon[126379]: osd.0 [v2:172.21.15.101:6802/3610719929,v1:172.21.15.101:6803/3610719929] boot 2023-07-18T02:59:59.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:58 smithi101 ceph-mon[126379]: osdmap e8: 1 total, 1 up, 1 in 2023-07-18T02:59:59.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:58 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-18T02:59:59.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:58 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:59.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:58 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:59.219 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:58 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-18T02:59:59.219 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:58 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:59.219 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:58 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:59.219 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:58 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T02:59:59.219 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:58 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:59.275 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:58 smithi155 ceph-mon[128885]: purged_snaps scrub starts 2023-07-18T02:59:59.275 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:58 smithi155 ceph-mon[128885]: purged_snaps scrub ok 2023-07-18T02:59:59.275 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:58 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-18T02:59:59.275 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:58 smithi155 ceph-mon[128885]: osd.0 [v2:172.21.15.101:6802/3610719929,v1:172.21.15.101:6803/3610719929] boot 2023-07-18T02:59:59.275 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:58 smithi155 ceph-mon[128885]: osdmap e8: 1 total, 1 up, 1 in 2023-07-18T02:59:59.276 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:58 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-18T02:59:59.276 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:58 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:59.276 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:58 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:59.276 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:58 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-18T02:59:59.276 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:58 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:59.276 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:58 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:59.276 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:58 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T02:59:59.277 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:58 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:59.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:58 smithi179 ceph-mon[129229]: purged_snaps scrub starts 2023-07-18T02:59:59.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:58 smithi179 ceph-mon[129229]: purged_snaps scrub ok 2023-07-18T02:59:59.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:58 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-18T02:59:59.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:58 smithi179 ceph-mon[129229]: osd.0 [v2:172.21.15.101:6802/3610719929,v1:172.21.15.101:6803/3610719929] boot 2023-07-18T02:59:59.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:58 smithi179 ceph-mon[129229]: osdmap e8: 1 total, 1 up, 1 in 2023-07-18T02:59:59.431 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:58 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-18T02:59:59.431 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:58 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:59.431 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:58 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:59.431 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:58 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-18T02:59:59.431 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:58 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T02:59:59.431 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:58 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T02:59:59.431 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:58 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T02:59:59.432 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:58 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:00.132 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:00.156+0000 7f2fd76b6700 1 -- 172.21.15.155:0/20915985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fd0100610 msgr2=0x7f2fd0106ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:00:00.132 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:00.156+0000 7f2fd76b6700 1 --2- 172.21.15.155:0/20915985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fd0100610 0x7f2fd0106ed0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2fc800b0a0 tx=0x7f2fc800b3b0 comp rx=0 tx=0).stop 2023-07-18T03:00:00.132 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:00.156+0000 7f2fd76b6700 1 -- 172.21.15.155:0/20915985 shutdown_connections 2023-07-18T03:00:00.132 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:00.156+0000 7f2fd76b6700 1 --2- 172.21.15.155:0/20915985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fd0100610 0x7f2fd0106ed0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:00:00.132 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:00.156+0000 7f2fd76b6700 1 --2- 172.21.15.155:0/20915985 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2fd00fea80 0x7f2fd00fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:00:00.133 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:00.156+0000 7f2fd76b6700 1 --2- 172.21.15.155:0/20915985 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2fd00ffc80 0x7f2fd01000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:00:00.133 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:00.156+0000 7f2fd76b6700 1 -- 172.21.15.155:0/20915985 >> 172.21.15.155:0/20915985 conn(0x7f2fd00fa0b0 msgr2=0x7f2fd00fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:00:00.133 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:00.156+0000 7f2fd76b6700 1 -- 172.21.15.155:0/20915985 shutdown_connections 2023-07-18T03:00:00.133 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:00.156+0000 7f2fd76b6700 1 -- 172.21.15.155:0/20915985 wait complete. 2023-07-18T03:00:00.133 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:00.157+0000 7f2fd76b6700 1 Processor -- start 2023-07-18T03:00:00.133 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:00.157+0000 7f2fd76b6700 1 -- start start 2023-07-18T03:00:00.133 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:00.157+0000 7f2fd76b6700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2fd00fea80 0x7f2fd01948c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:00:00.134 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:00.157+0000 7f2fd76b6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2fd00ffc80 0x7f2fd0194e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:00:00.134 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:00.157+0000 7f2fd76b6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fd0100610 0x7f2fd019be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:00:00.134 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:00.157+0000 7f2fd76b6700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd019c450 con 0x7f2fd00fea80 2023-07-18T03:00:00.134 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:00.157+0000 7f2fd76b6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd019c590 con 0x7f2fd0100610 2023-07-18T03:00:00.134 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:00.157+0000 7f2fd76b6700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd019c720 con 0x7f2fd00ffc80 2023-07-18T03:00:00.134 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:00.157+0000 7f2fd5c53700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fd0100610 0x7f2fd019be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:00:00.134 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:00.157+0000 7f2fd5c53700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fd0100610 0x7f2fd019be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:49596/0 (socket says 172.21.15.155:49596) 2023-07-18T03:00:00.134 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:00.157+0000 7f2fd5c53700 1 -- 172.21.15.155:0/1331934406 learned_addr learned my addr 172.21.15.155:0/1331934406 (peer_addr_for_me v2:172.21.15.155:0/0) 2023-07-18T03:00:00.135 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:00.157+0000 7f2fd4c51700 1 --2- 172.21.15.155:0/1331934406 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2fd00ffc80 0x7f2fd0194e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:00:00.135 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:00.157+0000 7f2fd5452700 1 --2- 172.21.15.155:0/1331934406 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2fd00fea80 0x7f2fd01948c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:00:00.135 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:00.157+0000 7f2fd5c53700 1 -- 172.21.15.155:0/1331934406 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2fd00ffc80 msgr2=0x7f2fd0194e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:00:00.135 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:00.157+0000 7f2fd5c53700 1 --2- 172.21.15.155:0/1331934406 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2fd00ffc80 0x7f2fd0194e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:00:00.135 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:00.157+0000 7f2fd5c53700 1 -- 172.21.15.155:0/1331934406 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2fd00fea80 msgr2=0x7f2fd01948c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:00:00.135 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:00.157+0000 7f2fd5c53700 1 --2- 172.21.15.155:0/1331934406 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2fd00fea80 0x7f2fd01948c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:00:00.135 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:00.157+0000 7f2fd5c53700 1 -- 172.21.15.155:0/1331934406 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc8009d00 con 0x7f2fd0100610 2023-07-18T03:00:00.136 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:00.157+0000 7f2fd5452700 1 --2- 172.21.15.155:0/1331934406 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2fd00fea80 0x7f2fd01948c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:00:00.136 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:00.158+0000 7f2fd5c53700 1 --2- 172.21.15.155:0/1331934406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fd0100610 0x7f2fd019be80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2fc80062a0 tx=0x7f2fc8004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:00:00.136 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:00.158+0000 7f2fc27fc700 1 -- 172.21.15.155:0/1331934406 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc800e040 con 0x7f2fd0100610 2023-07-18T03:00:00.136 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:00.158+0000 7f2fc27fc700 1 -- 172.21.15.155:0/1331934406 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2fc800cec0 con 0x7f2fd0100610 2023-07-18T03:00:00.136 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:00.158+0000 7f2fd76b6700 1 -- 172.21.15.155:0/1331934406 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fd0067fd0 con 0x7f2fd0100610 2023-07-18T03:00:00.136 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:00.158+0000 7f2fc27fc700 1 -- 172.21.15.155:0/1331934406 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc8004280 con 0x7f2fd0100610 2023-07-18T03:00:00.136 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:00.158+0000 7f2fd76b6700 1 -- 172.21.15.155:0/1331934406 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd0068550 con 0x7f2fd0100610 2023-07-18T03:00:00.137 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:00.159+0000 7f2fd76b6700 1 -- 172.21.15.155:0/1331934406 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fd010a2e0 con 0x7f2fd0100610 2023-07-18T03:00:00.137 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:00.160+0000 7f2fc27fc700 1 -- 172.21.15.155:0/1331934406 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 15) v1 ==== 135660+0+0 (secure 0 0 0) 0x7f2fc8003a00 con 0x7f2fd0100610 2023-07-18T03:00:00.138 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:00.160+0000 7f2fc27fc700 1 --2- 172.21.15.155:0/1331934406 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f2fbc0a1140 0x7f2fbc0a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:00:00.138 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:00.160+0000 7f2fc27fc700 1 -- 172.21.15.155:0/1331934406 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f2fc80ce2d0 con 0x7f2fd0100610 2023-07-18T03:00:00.138 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:00.161+0000 7f2fd5452700 1 --2- 172.21.15.155:0/1331934406 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f2fbc0a1140 0x7f2fbc0a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:00:00.138 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:00.161+0000 7f2fd5452700 1 --2- 172.21.15.155:0/1331934406 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f2fbc0a1140 0x7f2fbc0a35f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2fd00ffae0 tx=0x7f2fc4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:00:00.138 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:00.164+0000 7f2fc27fc700 1 -- 172.21.15.155:0/1331934406 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2fc8099480 con 0x7f2fd0100610 2023-07-18T03:00:00.275 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:59 smithi155 ceph-mon[128885]: Detected new or changed devices on smithi101 2023-07-18T03:00:00.275 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:59 smithi155 ceph-mon[128885]: Adjusting osd_memory_target on smithi101 to 17267M 2023-07-18T03:00:00.275 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:59 smithi155 ceph-mon[128885]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:00:00.275 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 02:59:59 smithi155 ceph-mon[128885]: osdmap e9: 1 total, 1 up, 1 in 2023-07-18T03:00:00.330 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:00.357+0000 7f2fd76b6700 1 -- 172.21.15.155:0/1331934406 --> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f2fd0068770 con 0x7f2fbc0a1140 2023-07-18T03:00:00.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:59 smithi179 ceph-mon[129229]: Detected new or changed devices on smithi101 2023-07-18T03:00:00.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:59 smithi179 ceph-mon[129229]: Adjusting osd_memory_target on smithi101 to 17267M 2023-07-18T03:00:00.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:59 smithi179 ceph-mon[129229]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:00:00.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 02:59:59 smithi179 ceph-mon[129229]: osdmap e9: 1 total, 1 up, 1 in 2023-07-18T03:00:00.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:59 smithi101 ceph-mon[126379]: Detected new or changed devices on smithi101 2023-07-18T03:00:00.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:59 smithi101 ceph-mon[126379]: Adjusting osd_memory_target on smithi101 to 17267M 2023-07-18T03:00:00.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:59 smithi101 ceph-mon[126379]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:00:00.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 02:59:59 smithi101 ceph-mon[126379]: osdmap e9: 1 total, 1 up, 1 in 2023-07-18T03:00:01.275 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:01 smithi155 ceph-mon[128885]: overall HEALTH_OK 2023-07-18T03:00:01.275 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:01 smithi155 ceph-mon[128885]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:00:01.275 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:01 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:00:01.275 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:01 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:00:01.275 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:01 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:00:01.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:01 smithi179 ceph-mon[129229]: overall HEALTH_OK 2023-07-18T03:00:01.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:01 smithi179 ceph-mon[129229]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:00:01.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:01 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:00:01.431 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:01 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:00:01.431 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:01 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:00:01.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:01 smithi101 ceph-mon[126379]: overall HEALTH_OK 2023-07-18T03:00:01.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:01 smithi101 ceph-mon[126379]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:00:01.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:01 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:00:01.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:01 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:00:01.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:01 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:00:02.066 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:02 smithi155 ceph-mon[128885]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:00:02.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:02 smithi179 ceph-mon[129229]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:00:02.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:02 smithi101 ceph-mon[126379]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:00:03.275 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:03 smithi155 ceph-mon[128885]: from='client.? 172.21.15.155:0/4045136302' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2a299573-0b34-4af9-aaae-b523e5058ee5"}]: dispatch 2023-07-18T03:00:03.275 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:03 smithi155 ceph-mon[128885]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2a299573-0b34-4af9-aaae-b523e5058ee5"}]: dispatch 2023-07-18T03:00:03.275 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:03 smithi155 ceph-mon[128885]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2a299573-0b34-4af9-aaae-b523e5058ee5"}]': finished 2023-07-18T03:00:03.276 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:03 smithi155 ceph-mon[128885]: osdmap e10: 2 total, 1 up, 2 in 2023-07-18T03:00:03.276 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:03 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-18T03:00:03.430 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:03 smithi179 ceph-mon[129229]: from='client.? 172.21.15.155:0/4045136302' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2a299573-0b34-4af9-aaae-b523e5058ee5"}]: dispatch 2023-07-18T03:00:03.431 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:03 smithi179 ceph-mon[129229]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2a299573-0b34-4af9-aaae-b523e5058ee5"}]: dispatch 2023-07-18T03:00:03.431 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:03 smithi179 ceph-mon[129229]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2a299573-0b34-4af9-aaae-b523e5058ee5"}]': finished 2023-07-18T03:00:03.431 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:03 smithi179 ceph-mon[129229]: osdmap e10: 2 total, 1 up, 2 in 2023-07-18T03:00:03.431 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:03 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-18T03:00:03.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:03 smithi101 ceph-mon[126379]: from='client.? 172.21.15.155:0/4045136302' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2a299573-0b34-4af9-aaae-b523e5058ee5"}]: dispatch 2023-07-18T03:00:03.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:03 smithi101 ceph-mon[126379]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2a299573-0b34-4af9-aaae-b523e5058ee5"}]: dispatch 2023-07-18T03:00:03.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:03 smithi101 ceph-mon[126379]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2a299573-0b34-4af9-aaae-b523e5058ee5"}]': finished 2023-07-18T03:00:03.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:03 smithi101 ceph-mon[126379]: osdmap e10: 2 total, 1 up, 2 in 2023-07-18T03:00:03.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:03 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-18T03:00:04.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:04 smithi101 ceph-mon[126379]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:00:04.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:04 smithi101 ceph-mon[126379]: from='client.? 172.21.15.155:0/984632891' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:00:04.525 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:04 smithi155 ceph-mon[128885]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:00:04.525 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:04 smithi155 ceph-mon[128885]: from='client.? 172.21.15.155:0/984632891' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:00:04.680 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:04 smithi179 ceph-mon[129229]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:00:04.680 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:04 smithi179 ceph-mon[129229]: from='client.? 172.21.15.155:0/984632891' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:00:06.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:06 smithi101 ceph-mon[126379]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:00:06.525 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:06 smithi155 ceph-mon[128885]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:00:06.680 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:06 smithi179 ceph-mon[129229]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:00:08.438 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:08 smithi155 ceph-mon[128885]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:00:08.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:08 smithi101 ceph-mon[126379]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:00:08.680 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:08 smithi179 ceph-mon[129229]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:00:10.467 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:10 smithi101 ceph-mon[126379]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:00:10.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:10 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-07-18T03:00:10.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:10 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:00:10.525 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:10 smithi155 ceph-mon[128885]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:00:10.525 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:10 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-07-18T03:00:10.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:10 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:00:10.680 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:10 smithi179 ceph-mon[129229]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:00:10.681 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:10 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-07-18T03:00:10.681 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:10 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:00:11.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:11 smithi101 ceph-mon[126379]: Deploying daemon osd.1 on smithi155 2023-07-18T03:00:11.525 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:11 smithi155 ceph-mon[128885]: Deploying daemon osd.1 on smithi155 2023-07-18T03:00:11.680 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:11 smithi179 ceph-mon[129229]: Deploying daemon osd.1 on smithi155 2023-07-18T03:00:12.392 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:12 smithi155 ceph-mon[128885]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:00:12.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:12 smithi101 ceph-mon[126379]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:00:12.680 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:12 smithi179 ceph-mon[129229]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:00:14.352 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:14 smithi155 ceph-mon[128885]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:00:14.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:14 smithi101 ceph-mon[126379]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:00:14.680 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:14 smithi179 ceph-mon[129229]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:00:15.680 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:15 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:15.681 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:15 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:00:15.681 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:15 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:15.681 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:15 smithi179 ceph-mon[129229]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:00:15.718 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:15 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:15.718 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:15 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:00:15.718 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:15 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:15.718 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:15 smithi101 ceph-mon[126379]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:00:15.776 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:15 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:15.776 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:15 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:00:15.776 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:15 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:15.776 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:15 smithi155 ceph-mon[128885]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:00:17.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:17 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:17.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:17 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:17.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:17 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:00:17.527 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:17 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:00:17.527 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:17 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:17.527 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:17 smithi155 ceph-mon[128885]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:00:17.749 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:17.775+0000 7f2fc27fc700 1 -- 172.21.15.155:0/1331934406 <== mgr.14150 v2:172.21.15.101:6800/3963742842 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2fd0068770 con 0x7f2fbc0a1140 2023-07-18T03:00:17.750 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 1 on host 'smithi155' 2023-07-18T03:00:17.751 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:17.777+0000 7f2fd76b6700 1 -- 172.21.15.155:0/1331934406 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f2fbc0a1140 msgr2=0x7f2fbc0a35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:00:17.751 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:17.777+0000 7f2fd76b6700 1 --2- 172.21.15.155:0/1331934406 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f2fbc0a1140 0x7f2fbc0a35f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2fd00ffae0 tx=0x7f2fc4008040 comp rx=0 tx=0).stop 2023-07-18T03:00:17.751 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:17.777+0000 7f2fd76b6700 1 -- 172.21.15.155:0/1331934406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fd0100610 msgr2=0x7f2fd019be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:00:17.752 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:17.777+0000 7f2fd76b6700 1 --2- 172.21.15.155:0/1331934406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fd0100610 0x7f2fd019be80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2fc80062a0 tx=0x7f2fc8004d00 comp rx=0 tx=0).stop 2023-07-18T03:00:17.752 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:17.777+0000 7f2fd76b6700 1 -- 172.21.15.155:0/1331934406 shutdown_connections 2023-07-18T03:00:17.752 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:17.777+0000 7f2fd76b6700 1 --2- 172.21.15.155:0/1331934406 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f2fbc0a1140 0x7f2fbc0a35f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:00:17.752 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:17.777+0000 7f2fd76b6700 1 --2- 172.21.15.155:0/1331934406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fd0100610 0x7f2fd019be80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:00:17.752 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:17.777+0000 7f2fd76b6700 1 --2- 172.21.15.155:0/1331934406 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2fd00fea80 0x7f2fd01948c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:00:17.752 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:17.777+0000 7f2fd76b6700 1 --2- 172.21.15.155:0/1331934406 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2fd00ffc80 0x7f2fd0194e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:00:17.752 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:17.778+0000 7f2fd76b6700 1 -- 172.21.15.155:0/1331934406 >> 172.21.15.155:0/1331934406 conn(0x7f2fd00fa0b0 msgr2=0x7f2fd01030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:00:17.753 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:17.778+0000 7f2fd76b6700 1 -- 172.21.15.155:0/1331934406 shutdown_connections 2023-07-18T03:00:17.753 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:00:17.778+0000 7f2fd76b6700 1 -- 172.21.15.155:0/1331934406 wait complete. 2023-07-18T03:00:17.931 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:17 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:17.931 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:17 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:17.931 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:17 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:00:17.931 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:17 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:00:17.932 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:17 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:17.932 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:17 smithi179 ceph-mon[129229]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:00:17.968 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:17 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:17.968 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:17 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:17.969 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:17 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:00:17.969 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:17 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:00:17.969 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:17 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:17.969 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:17 smithi101 ceph-mon[126379]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:00:18.273 DEBUG:teuthology.orchestra.run.smithi155:osd.1> sudo journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@osd.1.service 2023-07-18T03:00:18.277 INFO:tasks.cephadm:Deploying osd.2 on smithi179 with /dev/vg_nvme/lv_4... 2023-07-18T03:00:18.278 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-07-18T03:00:18.306 INFO:journalctl@ceph.osd.1.smithi155.stdout:-- Logs begin at Tue 2023-07-18 02:29:36 UTC. -- 2023-07-18T03:00:18.525 INFO:journalctl@ceph.osd.1.smithi155.stdout:Jul 18 03:00:18 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-1[133984]: 2023-07-18T03:00:18.355+0000 7fc2e2a5f640 -1 osd.1 0 log_to_monitors true 2023-07-18T03:00:18.843 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.c/config 2023-07-18T03:00:18.971 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:18 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:18.972 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:18 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:18.972 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:18 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:00:18.973 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:18 smithi155 ceph-mon[128885]: from='osd.1 [v2:172.21.15.155:6800/1030229579,v1:172.21.15.155:6801/1030229579]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-18T03:00:18.973 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:18 smithi155 ceph-mon[128885]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-18T03:00:19.137 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:18 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:19.138 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:18 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:19.138 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:18 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:00:19.138 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:18 smithi179 ceph-mon[129229]: from='osd.1 [v2:172.21.15.155:6800/1030229579,v1:172.21.15.155:6801/1030229579]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-18T03:00:19.138 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:18 smithi179 ceph-mon[129229]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-18T03:00:19.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:18 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:19.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:18 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:19.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:18 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:00:19.219 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:18 smithi101 ceph-mon[126379]: from='osd.1 [v2:172.21.15.155:6800/1030229579,v1:172.21.15.155:6801/1030229579]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-18T03:00:19.219 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:18 smithi101 ceph-mon[126379]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-18T03:00:20.440 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-07-18T03:00:20.440 INFO:teuthology.orchestra.run.smithi179.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-07-18T03:00:20.440 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10+0 records in 2023-07-18T03:00:20.440 INFO:teuthology.orchestra.run.smithi179.stderr:10+0 records out 2023-07-18T03:00:20.440 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110168 s, 952 MB/s 2023-07-18T03:00:20.441 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping successful for: 2023-07-18T03:00:20.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:20 smithi101 ceph-mon[126379]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:00:20.469 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:20 smithi101 ceph-mon[126379]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-07-18T03:00:20.469 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:20 smithi101 ceph-mon[126379]: osdmap e11: 2 total, 1 up, 2 in 2023-07-18T03:00:20.470 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:20 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-18T03:00:20.470 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:20 smithi101 ceph-mon[126379]: from='osd.1 [v2:172.21.15.155:6800/1030229579,v1:172.21.15.155:6801/1030229579]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-18T03:00:20.470 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:20 smithi101 ceph-mon[126379]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-18T03:00:20.476 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:20 smithi179 ceph-mon[129229]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:00:20.477 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:20 smithi179 ceph-mon[129229]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-07-18T03:00:20.477 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:20 smithi179 ceph-mon[129229]: osdmap e11: 2 total, 1 up, 2 in 2023-07-18T03:00:20.477 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:20 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-18T03:00:20.477 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:20 smithi179 ceph-mon[129229]: from='osd.1 [v2:172.21.15.155:6800/1030229579,v1:172.21.15.155:6801/1030229579]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-18T03:00:20.477 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:20 smithi179 ceph-mon[129229]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-18T03:00:20.526 INFO:journalctl@ceph.osd.1.smithi155.stdout:Jul 18 03:00:20 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-1[133984]: 2023-07-18T03:00:20.234+0000 7fc2d80d7700 -1 osd.1 0 waiting for initial osdmap 2023-07-18T03:00:20.526 INFO:journalctl@ceph.osd.1.smithi155.stdout:Jul 18 03:00:20 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-1[133984]: 2023-07-18T03:00:20.238+0000 7fc2cfec4700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-18T03:00:20.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:20 smithi155 ceph-mon[128885]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:00:20.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:20 smithi155 ceph-mon[128885]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-07-18T03:00:20.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:20 smithi155 ceph-mon[128885]: osdmap e11: 2 total, 1 up, 2 in 2023-07-18T03:00:20.527 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:20 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-18T03:00:20.527 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:20 smithi155 ceph-mon[128885]: from='osd.1 [v2:172.21.15.155:6800/1030229579,v1:172.21.15.155:6801/1030229579]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-18T03:00:20.527 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:20 smithi155 ceph-mon[128885]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-18T03:00:20.886 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi179:vg_nvme/lv_4 2023-07-18T03:00:21.437 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.c/config 2023-07-18T03:00:21.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:21 smithi101 ceph-mon[126379]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2023-07-18T03:00:21.469 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:21 smithi101 ceph-mon[126379]: osdmap e12: 2 total, 1 up, 2 in 2023-07-18T03:00:21.469 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:21 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-18T03:00:21.469 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:21 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-18T03:00:21.469 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:21 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:21.469 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:21 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:21.469 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:21 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:00:21.470 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:21 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:21.470 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:21 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:00:21.470 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:21 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:00:21.470 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:21 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:21.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:21 smithi155 ceph-mon[128885]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2023-07-18T03:00:21.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:21 smithi155 ceph-mon[128885]: osdmap e12: 2 total, 1 up, 2 in 2023-07-18T03:00:21.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:21 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-18T03:00:21.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:21 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-18T03:00:21.527 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:21 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:21.527 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:21 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:21.527 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:21 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:00:21.527 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:21 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:21.527 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:21 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:00:21.527 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:21 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:00:21.528 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:21 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:21.654 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:21 smithi179 ceph-mon[129229]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2023-07-18T03:00:21.654 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:21 smithi179 ceph-mon[129229]: osdmap e12: 2 total, 1 up, 2 in 2023-07-18T03:00:21.654 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:21 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-18T03:00:21.654 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:21 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-18T03:00:21.655 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:21 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:21.655 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:21 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:21.655 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:21 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:00:21.655 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:21 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:21.655 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:21 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:00:21.655 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:21 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:00:21.655 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:21 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:22.468 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:22 smithi101 ceph-mon[126379]: purged_snaps scrub starts 2023-07-18T03:00:22.469 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:22 smithi101 ceph-mon[126379]: purged_snaps scrub ok 2023-07-18T03:00:22.469 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:22 smithi101 ceph-mon[126379]: Detected new or changed devices on smithi155 2023-07-18T03:00:22.469 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:22 smithi101 ceph-mon[126379]: Adjusting osd_memory_target on smithi155 to 17168M 2023-07-18T03:00:22.469 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:22 smithi101 ceph-mon[126379]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:00:22.469 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:22 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-18T03:00:22.469 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:22 smithi101 ceph-mon[126379]: osd.1 [v2:172.21.15.155:6800/1030229579,v1:172.21.15.155:6801/1030229579] boot 2023-07-18T03:00:22.469 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:22 smithi101 ceph-mon[126379]: osdmap e13: 2 total, 2 up, 2 in 2023-07-18T03:00:22.470 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:22 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-18T03:00:22.486 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:22 smithi179 ceph-mon[129229]: purged_snaps scrub starts 2023-07-18T03:00:22.486 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:22 smithi179 ceph-mon[129229]: purged_snaps scrub ok 2023-07-18T03:00:22.487 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:22 smithi179 ceph-mon[129229]: Detected new or changed devices on smithi155 2023-07-18T03:00:22.487 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:22 smithi179 ceph-mon[129229]: Adjusting osd_memory_target on smithi155 to 17168M 2023-07-18T03:00:22.487 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:22 smithi179 ceph-mon[129229]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:00:22.487 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:22 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-18T03:00:22.487 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:22 smithi179 ceph-mon[129229]: osd.1 [v2:172.21.15.155:6800/1030229579,v1:172.21.15.155:6801/1030229579] boot 2023-07-18T03:00:22.488 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:22 smithi179 ceph-mon[129229]: osdmap e13: 2 total, 2 up, 2 in 2023-07-18T03:00:22.488 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:22 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-18T03:00:22.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:22 smithi155 ceph-mon[128885]: purged_snaps scrub starts 2023-07-18T03:00:22.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:22 smithi155 ceph-mon[128885]: purged_snaps scrub ok 2023-07-18T03:00:22.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:22 smithi155 ceph-mon[128885]: Detected new or changed devices on smithi155 2023-07-18T03:00:22.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:22 smithi155 ceph-mon[128885]: Adjusting osd_memory_target on smithi155 to 17168M 2023-07-18T03:00:22.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:22 smithi155 ceph-mon[128885]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:00:22.527 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:22 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-18T03:00:22.527 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:22 smithi155 ceph-mon[128885]: osd.1 [v2:172.21.15.155:6800/1030229579,v1:172.21.15.155:6801/1030229579] boot 2023-07-18T03:00:22.527 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:22 smithi155 ceph-mon[128885]: osdmap e13: 2 total, 2 up, 2 in 2023-07-18T03:00:22.527 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:22 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-18T03:00:22.629 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:22.653+0000 7f2d128e5700 1 -- 172.21.15.179:0/143980763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d0c0ffbc0 msgr2=0x7f2d0c0fffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:00:22.630 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:22.653+0000 7f2d128e5700 1 --2- 172.21.15.179:0/143980763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d0c0ffbc0 0x7f2d0c0fffd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2d00009990 tx=0x7f2d00009ca0 comp rx=0 tx=0).stop 2023-07-18T03:00:22.630 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:22.653+0000 7f2d128e5700 1 -- 172.21.15.179:0/143980763 shutdown_connections 2023-07-18T03:00:22.630 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:22.653+0000 7f2d128e5700 1 --2- 172.21.15.179:0/143980763 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2d0c101750 0x7f2d0c108010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:00:22.631 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:22.653+0000 7f2d128e5700 1 --2- 172.21.15.179:0/143980763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d0c0ffbc0 0x7f2d0c0fffd0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:00:22.631 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:22.653+0000 7f2d128e5700 1 --2- 172.21.15.179:0/143980763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d0c100dc0 0x7f2d0c101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:00:22.631 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:22.653+0000 7f2d128e5700 1 -- 172.21.15.179:0/143980763 >> 172.21.15.179:0/143980763 conn(0x7f2d0c0fb230 msgr2=0x7f2d0c0fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:00:22.631 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:22.653+0000 7f2d128e5700 1 -- 172.21.15.179:0/143980763 shutdown_connections 2023-07-18T03:00:22.632 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:22.653+0000 7f2d128e5700 1 -- 172.21.15.179:0/143980763 wait complete. 2023-07-18T03:00:22.632 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:22.654+0000 7f2d128e5700 1 Processor -- start 2023-07-18T03:00:22.633 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:22.654+0000 7f2d128e5700 1 -- start start 2023-07-18T03:00:22.633 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:22.654+0000 7f2d128e5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d0c0ffbc0 0x7f2d0c197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:00:22.633 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:22.654+0000 7f2d128e5700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2d0c100dc0 0x7f2d0c197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:00:22.634 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:22.654+0000 7f2d128e5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d0c101750 0x7f2d0c19f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:00:22.634 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:22.654+0000 7f2d128e5700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c19f630 con 0x7f2d0c100dc0 2023-07-18T03:00:22.634 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:22.654+0000 7f2d128e5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c19f770 con 0x7f2d0c0ffbc0 2023-07-18T03:00:22.634 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:22.654+0000 7f2d128e5700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c19f900 con 0x7f2d0c101750 2023-07-18T03:00:22.635 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:22.654+0000 7f2d10e82700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d0c101750 0x7f2d0c19f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:00:22.635 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:22.654+0000 7f2d10e82700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d0c101750 0x7f2d0c19f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48192/0 (socket says 172.21.15.179:48192) 2023-07-18T03:00:22.635 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:22.654+0000 7f2d10e82700 1 -- 172.21.15.179:0/1901091998 learned_addr learned my addr 172.21.15.179:0/1901091998 (peer_addr_for_me v2:172.21.15.179:0/0) 2023-07-18T03:00:22.635 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:22.654+0000 7f2d0bfff700 1 --2- 172.21.15.179:0/1901091998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d0c0ffbc0 0x7f2d0c197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:00:22.636 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:22.655+0000 7f2d10e82700 1 -- 172.21.15.179:0/1901091998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d0c0ffbc0 msgr2=0x7f2d0c197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:00:22.636 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:22.655+0000 7f2d10e82700 1 --2- 172.21.15.179:0/1901091998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d0c0ffbc0 0x7f2d0c197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:00:22.636 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:22.655+0000 7f2d0b7fe700 1 --2- 172.21.15.179:0/1901091998 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2d0c100dc0 0x7f2d0c197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:00:22.636 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:22.655+0000 7f2d10e82700 1 -- 172.21.15.179:0/1901091998 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2d0c100dc0 msgr2=0x7f2d0c197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:00:22.637 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:22.655+0000 7f2d10e82700 1 --2- 172.21.15.179:0/1901091998 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2d0c100dc0 0x7f2d0c197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:00:22.637 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:22.655+0000 7f2d10e82700 1 -- 172.21.15.179:0/1901091998 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d000095f0 con 0x7f2d0c101750 2023-07-18T03:00:22.637 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:22.655+0000 7f2d0b7fe700 1 --2- 172.21.15.179:0/1901091998 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2d0c100dc0 0x7f2d0c197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:00:22.637 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:22.655+0000 7f2d10e82700 1 --2- 172.21.15.179:0/1901091998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d0c101750 0x7f2d0c19f060 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2d04007d40 tx=0x7f2d0400a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:00:22.638 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:22.655+0000 7f2d097fa700 1 -- 172.21.15.179:0/1901091998 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0400ad60 con 0x7f2d0c101750 2023-07-18T03:00:22.638 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:22.655+0000 7f2d097fa700 1 -- 172.21.15.179:0/1901091998 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d04007e90 con 0x7f2d0c101750 2023-07-18T03:00:22.638 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:22.655+0000 7f2d097fa700 1 -- 172.21.15.179:0/1901091998 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d040116b0 con 0x7f2d0c101750 2023-07-18T03:00:22.638 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:22.655+0000 7f2d128e5700 1 -- 172.21.15.179:0/1901091998 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d0c19fbe0 con 0x7f2d0c101750 2023-07-18T03:00:22.639 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:22.655+0000 7f2d128e5700 1 -- 172.21.15.179:0/1901091998 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d0c1a0090 con 0x7f2d0c101750 2023-07-18T03:00:22.639 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:22.656+0000 7f2d128e5700 1 -- 172.21.15.179:0/1901091998 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d0c1922c0 con 0x7f2d0c101750 2023-07-18T03:00:22.640 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:22.657+0000 7f2d097fa700 1 -- 172.21.15.179:0/1901091998 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 15) v1 ==== 135660+0+0 (secure 0 0 0) 0x7f2d04012420 con 0x7f2d0c101750 2023-07-18T03:00:22.641 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:22.658+0000 7f2d097fa700 1 --2- 172.21.15.179:0/1901091998 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f2cf40a10c0 0x7f2cf40a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:00:22.641 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:22.658+0000 7f2d097fa700 1 -- 172.21.15.179:0/1901091998 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f2d040cb310 con 0x7f2d0c101750 2023-07-18T03:00:22.641 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:22.658+0000 7f2d0bfff700 1 --2- 172.21.15.179:0/1901091998 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f2cf40a10c0 0x7f2cf40a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:00:22.642 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:22.658+0000 7f2d0bfff700 1 --2- 172.21.15.179:0/1901091998 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f2cf40a10c0 0x7f2cf40a3570 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2d0000b5c0 tx=0x7f2d00005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:00:22.642 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:22.662+0000 7f2d097fa700 1 -- 172.21.15.179:0/1901091998 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2d040962b0 con 0x7f2d0c101750 2023-07-18T03:00:22.828 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:22.855+0000 7f2d128e5700 1 -- 172.21.15.179:0/1901091998 --> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d0c060950 con 0x7f2cf40a10c0 2023-07-18T03:00:23.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:23 smithi155 ceph-mon[128885]: osdmap e14: 2 total, 2 up, 2 in 2023-07-18T03:00:23.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:23 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:00:23.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:23 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:00:23.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:23 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:00:23.644 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:23 smithi179 ceph-mon[129229]: osdmap e14: 2 total, 2 up, 2 in 2023-07-18T03:00:23.645 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:23 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:00:23.645 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:23 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:00:23.645 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:23 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:00:23.718 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:23 smithi101 ceph-mon[126379]: osdmap e14: 2 total, 2 up, 2 in 2023-07-18T03:00:23.719 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:23 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:00:23.719 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:23 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:00:23.719 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:23 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:00:24.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:24 smithi155 ceph-mon[128885]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:00:24.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:24 smithi155 ceph-mon[128885]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:00:24.681 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:24 smithi179 ceph-mon[129229]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:00:24.682 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:24 smithi179 ceph-mon[129229]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:00:24.718 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:24 smithi101 ceph-mon[126379]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:00:24.719 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:24 smithi101 ceph-mon[126379]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:00:25.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:25 smithi155 ceph-mon[128885]: from='client.? 172.21.15.179:0/1648613141' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "123b3749-c636-48fa-9e33-6229b5575dde"}]: dispatch 2023-07-18T03:00:25.527 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:25 smithi155 ceph-mon[128885]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "123b3749-c636-48fa-9e33-6229b5575dde"}]: dispatch 2023-07-18T03:00:25.527 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:25 smithi155 ceph-mon[128885]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "123b3749-c636-48fa-9e33-6229b5575dde"}]': finished 2023-07-18T03:00:25.527 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:25 smithi155 ceph-mon[128885]: osdmap e15: 3 total, 2 up, 3 in 2023-07-18T03:00:25.527 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:25 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-18T03:00:25.681 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:25 smithi179 ceph-mon[129229]: from='client.? 172.21.15.179:0/1648613141' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "123b3749-c636-48fa-9e33-6229b5575dde"}]: dispatch 2023-07-18T03:00:25.681 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:25 smithi179 ceph-mon[129229]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "123b3749-c636-48fa-9e33-6229b5575dde"}]: dispatch 2023-07-18T03:00:25.682 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:25 smithi179 ceph-mon[129229]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "123b3749-c636-48fa-9e33-6229b5575dde"}]': finished 2023-07-18T03:00:25.682 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:25 smithi179 ceph-mon[129229]: osdmap e15: 3 total, 2 up, 3 in 2023-07-18T03:00:25.682 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:25 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-18T03:00:25.719 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:25 smithi101 ceph-mon[126379]: from='client.? 172.21.15.179:0/1648613141' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "123b3749-c636-48fa-9e33-6229b5575dde"}]: dispatch 2023-07-18T03:00:25.720 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:25 smithi101 ceph-mon[126379]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "123b3749-c636-48fa-9e33-6229b5575dde"}]: dispatch 2023-07-18T03:00:25.720 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:25 smithi101 ceph-mon[126379]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "123b3749-c636-48fa-9e33-6229b5575dde"}]': finished 2023-07-18T03:00:25.720 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:25 smithi101 ceph-mon[126379]: osdmap e15: 3 total, 2 up, 3 in 2023-07-18T03:00:25.720 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:25 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-18T03:00:26.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:26 smithi155 ceph-mon[128885]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:00:26.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:26 smithi155 ceph-mon[128885]: from='client.? 172.21.15.179:0/2600384676' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:00:26.681 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:26 smithi179 ceph-mon[129229]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:00:26.681 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:26 smithi179 ceph-mon[129229]: from='client.? 172.21.15.179:0/2600384676' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:00:26.718 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:26 smithi101 ceph-mon[126379]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:00:26.719 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:26 smithi101 ceph-mon[126379]: from='client.? 172.21.15.179:0/2600384676' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:00:28.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:28 smithi155 ceph-mon[128885]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:00:28.681 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:28 smithi179 ceph-mon[129229]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:00:28.719 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:28 smithi101 ceph-mon[126379]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:00:30.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:30 smithi155 ceph-mon[128885]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:00:30.681 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:30 smithi179 ceph-mon[129229]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:00:30.718 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:30 smithi101 ceph-mon[126379]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:00:32.432 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:32 smithi179 ceph-mon[129229]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:00:32.432 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:32 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-07-18T03:00:32.432 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:32 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:00:32.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:32 smithi155 ceph-mon[128885]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:00:32.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:32 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-07-18T03:00:32.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:32 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:00:32.719 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:32 smithi101 ceph-mon[126379]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:00:32.719 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:32 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-07-18T03:00:32.719 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:32 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:00:33.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:33 smithi179 ceph-mon[129229]: Deploying daemon osd.2 on smithi179 2023-07-18T03:00:33.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:33 smithi155 ceph-mon[128885]: Deploying daemon osd.2 on smithi179 2023-07-18T03:00:33.719 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:33 smithi101 ceph-mon[126379]: Deploying daemon osd.2 on smithi179 2023-07-18T03:00:34.431 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:34 smithi179 ceph-mon[129229]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:00:34.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:34 smithi155 ceph-mon[128885]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:00:34.719 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:34 smithi101 ceph-mon[126379]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:00:36.431 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:36 smithi179 ceph-mon[129229]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:00:36.432 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:36 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:36.432 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:36 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:36.432 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:36 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:00:36.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:36 smithi155 ceph-mon[128885]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:00:36.527 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:36 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:36.527 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:36 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:36.527 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:36 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:00:36.565 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:36 smithi101 ceph-mon[126379]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:00:36.565 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:36 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:36.566 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:36 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:36.566 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:36 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:00:38.431 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:38 smithi179 ceph-mon[129229]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:00:38.526 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:38 smithi155 ceph-mon[128885]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:00:38.721 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:38 smithi101 ceph-mon[126379]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:00:39.093 INFO:teuthology.orchestra.run.smithi179.stdout:Created osd(s) 2 on host 'smithi179' 2023-07-18T03:00:39.093 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:39.117+0000 7f2d097fa700 1 -- 172.21.15.179:0/1901091998 <== mgr.14150 v2:172.21.15.101:6800/3963742842 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2d0c060950 con 0x7f2cf40a10c0 2023-07-18T03:00:39.095 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:39.120+0000 7f2d128e5700 1 -- 172.21.15.179:0/1901091998 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f2cf40a10c0 msgr2=0x7f2cf40a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:00:39.095 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:39.120+0000 7f2d128e5700 1 --2- 172.21.15.179:0/1901091998 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f2cf40a10c0 0x7f2cf40a3570 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2d0000b5c0 tx=0x7f2d00005fd0 comp rx=0 tx=0).stop 2023-07-18T03:00:39.095 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:39.120+0000 7f2d128e5700 1 -- 172.21.15.179:0/1901091998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d0c101750 msgr2=0x7f2d0c19f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:00:39.095 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:39.120+0000 7f2d128e5700 1 --2- 172.21.15.179:0/1901091998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d0c101750 0x7f2d0c19f060 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2d04007d40 tx=0x7f2d0400a340 comp rx=0 tx=0).stop 2023-07-18T03:00:39.096 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:39.120+0000 7f2d128e5700 1 -- 172.21.15.179:0/1901091998 shutdown_connections 2023-07-18T03:00:39.096 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:39.120+0000 7f2d128e5700 1 --2- 172.21.15.179:0/1901091998 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f2cf40a10c0 0x7f2cf40a3570 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:00:39.096 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:39.120+0000 7f2d128e5700 1 --2- 172.21.15.179:0/1901091998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d0c0ffbc0 0x7f2d0c197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:00:39.096 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:39.120+0000 7f2d128e5700 1 --2- 172.21.15.179:0/1901091998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d0c101750 0x7f2d0c19f060 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:00:39.096 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:39.120+0000 7f2d128e5700 1 --2- 172.21.15.179:0/1901091998 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2d0c100dc0 0x7f2d0c197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:00:39.097 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:39.120+0000 7f2d128e5700 1 -- 172.21.15.179:0/1901091998 >> 172.21.15.179:0/1901091998 conn(0x7f2d0c0fb230 msgr2=0x7f2d0c1066b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:00:39.097 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:39.120+0000 7f2d128e5700 1 -- 172.21.15.179:0/1901091998 shutdown_connections 2023-07-18T03:00:39.097 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:00:39.120+0000 7f2d128e5700 1 -- 172.21.15.179:0/1901091998 wait complete. 2023-07-18T03:00:39.522 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:39 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:39.522 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:39 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:39.522 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:39 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:00:39.522 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:39 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:00:39.523 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:39 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:39.523 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:39 smithi179 ceph-mon[129229]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:00:39.523 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:39 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:39.523 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:39 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:39.523 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:39 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:00:39.524 DEBUG:teuthology.orchestra.run.smithi179:osd.2> sudo journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@osd.2.service 2023-07-18T03:00:39.526 INFO:tasks.cephadm:Deploying osd.3 on smithi101 with /dev/vg_nvme/lv_3... 2023-07-18T03:00:39.527 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-07-18T03:00:39.554 INFO:journalctl@ceph.osd.2.smithi179.stdout:-- Logs begin at Tue 2023-07-18 02:29:48 UTC. -- 2023-07-18T03:00:39.719 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:39 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:39.719 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:39 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:39.720 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:39 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:00:39.720 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:39 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:00:39.720 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:39 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:39.720 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:39 smithi101 ceph-mon[126379]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:00:39.720 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:39 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:39.720 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:39 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:39.720 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:39 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:00:39.776 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:39 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:39.777 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:39 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:39.777 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:39 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:00:39.777 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:39 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:00:39.777 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:39 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:39.777 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:39 smithi155 ceph-mon[128885]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:00:39.778 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:39 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:39.778 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:39 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:39.778 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:39 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:00:40.255 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:00:40.279 INFO:journalctl@ceph.osd.2.smithi179.stdout:Jul 18 03:00:40 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-2[135082]: 2023-07-18T03:00:40.140+0000 7f6b3b139640 -1 osd.2 0 log_to_monitors true 2023-07-18T03:00:40.682 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:40 smithi179 ceph-mon[129229]: from='osd.2 [v2:172.21.15.179:6800/1349487507,v1:172.21.15.179:6801/1349487507]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-18T03:00:40.682 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:40 smithi179 ceph-mon[129229]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-18T03:00:40.700 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:40 smithi101 ceph-mon[126379]: from='osd.2 [v2:172.21.15.179:6800/1349487507,v1:172.21.15.179:6801/1349487507]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-18T03:00:40.700 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:40 smithi101 ceph-mon[126379]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-18T03:00:40.777 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:40 smithi155 ceph-mon[128885]: from='osd.2 [v2:172.21.15.179:6800/1349487507,v1:172.21.15.179:6801/1349487507]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-18T03:00:40.777 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:40 smithi155 ceph-mon[128885]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-18T03:00:41.469 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:41 smithi101 ceph-mon[126379]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-07-18T03:00:41.470 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:41 smithi101 ceph-mon[126379]: osdmap e16: 3 total, 2 up, 3 in 2023-07-18T03:00:41.470 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:41 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-18T03:00:41.470 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:41 smithi101 ceph-mon[126379]: from='osd.2 [v2:172.21.15.179:6800/1349487507,v1:172.21.15.179:6801/1349487507]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2023-07-18T03:00:41.470 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:41 smithi101 ceph-mon[126379]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2023-07-18T03:00:41.470 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:41 smithi101 ceph-mon[126379]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:00:41.645 INFO:journalctl@ceph.osd.2.smithi179.stdout:Jul 18 03:00:41 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-2[135082]: 2023-07-18T03:00:41.465+0000 7f6b30fb2700 -1 osd.2 0 waiting for initial osdmap 2023-07-18T03:00:41.646 INFO:journalctl@ceph.osd.2.smithi179.stdout:Jul 18 03:00:41 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-2[135082]: 2023-07-18T03:00:41.469+0000 7f6b2c5a6700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-18T03:00:41.646 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:41 smithi179 ceph-mon[129229]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-07-18T03:00:41.646 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:41 smithi179 ceph-mon[129229]: osdmap e16: 3 total, 2 up, 3 in 2023-07-18T03:00:41.646 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:41 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-18T03:00:41.646 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:41 smithi179 ceph-mon[129229]: from='osd.2 [v2:172.21.15.179:6800/1349487507,v1:172.21.15.179:6801/1349487507]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2023-07-18T03:00:41.647 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:41 smithi179 ceph-mon[129229]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2023-07-18T03:00:41.647 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:41 smithi179 ceph-mon[129229]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:00:41.777 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:41 smithi155 ceph-mon[128885]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-07-18T03:00:41.777 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:41 smithi155 ceph-mon[128885]: osdmap e16: 3 total, 2 up, 3 in 2023-07-18T03:00:41.777 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:41 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-18T03:00:41.777 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:41 smithi155 ceph-mon[128885]: from='osd.2 [v2:172.21.15.179:6800/1349487507,v1:172.21.15.179:6801/1349487507]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2023-07-18T03:00:41.777 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:41 smithi155 ceph-mon[128885]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2023-07-18T03:00:41.778 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:41 smithi155 ceph-mon[128885]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:00:42.360 INFO:teuthology.orchestra.run.smithi101.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-07-18T03:00:42.360 INFO:teuthology.orchestra.run.smithi101.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-07-18T03:00:42.361 INFO:teuthology.orchestra.run.smithi101.stderr: stderr: 10+0 records in 2023-07-18T03:00:42.361 INFO:teuthology.orchestra.run.smithi101.stderr:10+0 records out 2023-07-18T03:00:42.361 INFO:teuthology.orchestra.run.smithi101.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0296221 s, 354 MB/s 2023-07-18T03:00:42.361 INFO:teuthology.orchestra.run.smithi101.stderr:--> Zapping successful for: 2023-07-18T03:00:42.719 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:42 smithi101 ceph-mon[126379]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2023-07-18T03:00:42.720 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:42 smithi101 ceph-mon[126379]: osdmap e17: 3 total, 2 up, 3 in 2023-07-18T03:00:42.720 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:42 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-18T03:00:42.720 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:42 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-18T03:00:42.720 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:42 smithi101 ceph-mon[126379]: Detected new or changed devices on smithi179 2023-07-18T03:00:42.721 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:42 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:42.721 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:42 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:42.721 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:42 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:00:42.721 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:42 smithi101 ceph-mon[126379]: Adjusting osd_memory_target on smithi179 to 17168M 2023-07-18T03:00:42.721 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:42 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:42.721 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:42 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:00:42.721 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:42 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:00:42.722 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:42 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:42.777 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:42 smithi155 ceph-mon[128885]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2023-07-18T03:00:42.777 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:42 smithi155 ceph-mon[128885]: osdmap e17: 3 total, 2 up, 3 in 2023-07-18T03:00:42.778 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:42 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-18T03:00:42.778 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:42 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-18T03:00:42.778 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:42 smithi155 ceph-mon[128885]: Detected new or changed devices on smithi179 2023-07-18T03:00:42.778 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:42 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:42.778 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:42 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:42.778 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:42 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:00:42.779 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:42 smithi155 ceph-mon[128885]: Adjusting osd_memory_target on smithi179 to 17168M 2023-07-18T03:00:42.779 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:42 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:42.779 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:42 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:00:42.779 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:42 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:00:42.779 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:42 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:42.932 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:42 smithi179 ceph-mon[129229]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2023-07-18T03:00:42.932 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:42 smithi179 ceph-mon[129229]: osdmap e17: 3 total, 2 up, 3 in 2023-07-18T03:00:42.932 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:42 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-18T03:00:42.932 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:42 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-18T03:00:42.933 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:42 smithi179 ceph-mon[129229]: Detected new or changed devices on smithi179 2023-07-18T03:00:42.933 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:42 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:42.933 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:42 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:42.933 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:42 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:00:42.933 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:42 smithi179 ceph-mon[129229]: Adjusting osd_memory_target on smithi179 to 17168M 2023-07-18T03:00:42.933 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:42 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:42.933 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:42 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:00:42.934 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:42 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:00:42.934 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:42 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:43.032 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi101:vg_nvme/lv_3 2023-07-18T03:00:43.719 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:43 smithi101 ceph-mon[126379]: purged_snaps scrub starts 2023-07-18T03:00:43.720 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:43 smithi101 ceph-mon[126379]: purged_snaps scrub ok 2023-07-18T03:00:43.720 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:43 smithi101 ceph-mon[126379]: osd.2 [v2:172.21.15.179:6800/1349487507,v1:172.21.15.179:6801/1349487507] boot 2023-07-18T03:00:43.720 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:43 smithi101 ceph-mon[126379]: osdmap e18: 3 total, 3 up, 3 in 2023-07-18T03:00:43.720 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:43 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-18T03:00:43.720 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:43 smithi101 ceph-mon[126379]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:00:43.720 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:43 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-07-18T03:00:43.753 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:00:43.777 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:43 smithi155 ceph-mon[128885]: purged_snaps scrub starts 2023-07-18T03:00:43.777 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:43 smithi155 ceph-mon[128885]: purged_snaps scrub ok 2023-07-18T03:00:43.777 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:43 smithi155 ceph-mon[128885]: osd.2 [v2:172.21.15.179:6800/1349487507,v1:172.21.15.179:6801/1349487507] boot 2023-07-18T03:00:43.778 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:43 smithi155 ceph-mon[128885]: osdmap e18: 3 total, 3 up, 3 in 2023-07-18T03:00:43.778 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:43 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-18T03:00:43.778 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:43 smithi155 ceph-mon[128885]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:00:43.778 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:43 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-07-18T03:00:43.932 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:43 smithi179 ceph-mon[129229]: purged_snaps scrub starts 2023-07-18T03:00:43.932 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:43 smithi179 ceph-mon[129229]: purged_snaps scrub ok 2023-07-18T03:00:43.933 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:43 smithi179 ceph-mon[129229]: osd.2 [v2:172.21.15.179:6800/1349487507,v1:172.21.15.179:6801/1349487507] boot 2023-07-18T03:00:43.933 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:43 smithi179 ceph-mon[129229]: osdmap e18: 3 total, 3 up, 3 in 2023-07-18T03:00:43.933 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:43 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-18T03:00:43.933 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:43 smithi179 ceph-mon[129229]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:00:43.933 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:43 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-07-18T03:00:44.719 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:44 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-07-18T03:00:44.720 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:44 smithi101 ceph-mon[126379]: osdmap e19: 3 total, 3 up, 3 in 2023-07-18T03:00:44.720 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:44 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-07-18T03:00:44.777 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:44 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-07-18T03:00:44.778 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:44 smithi155 ceph-mon[128885]: osdmap e19: 3 total, 3 up, 3 in 2023-07-18T03:00:44.778 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:44 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-07-18T03:00:44.932 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:44 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-07-18T03:00:44.932 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:44 smithi179 ceph-mon[129229]: osdmap e19: 3 total, 3 up, 3 in 2023-07-18T03:00:44.932 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:44 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-07-18T03:00:45.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:00:45.380+0000 7f3b04846700 1 -- 172.21.15.101:0/2325086912 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b00072f70 msgr2=0x7f3b000733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:00:45.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:00:45.380+0000 7f3b04846700 1 --2- 172.21.15.101:0/2325086912 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b00072f70 0x7f3b000733e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3af40099c0 tx=0x7f3af4009cd0 comp rx=0 tx=0).stop 2023-07-18T03:00:45.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:00:45.380+0000 7f3b04846700 1 -- 172.21.15.101:0/2325086912 shutdown_connections 2023-07-18T03:00:45.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:00:45.380+0000 7f3b04846700 1 --2- 172.21.15.101:0/2325086912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b00074540 0x7f3b000729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:00:45.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:00:45.380+0000 7f3b04846700 1 --2- 172.21.15.101:0/2325086912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b0010c170 0x7f3b0010e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:00:45.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:00:45.380+0000 7f3b04846700 1 --2- 172.21.15.101:0/2325086912 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b00072f70 0x7f3b000733e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:00:45.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:00:45.380+0000 7f3b04846700 1 -- 172.21.15.101:0/2325086912 >> 172.21.15.101:0/2325086912 conn(0x7f3b000fb1a0 msgr2=0x7f3b000fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:00:45.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:00:45.380+0000 7f3b04846700 1 -- 172.21.15.101:0/2325086912 shutdown_connections 2023-07-18T03:00:45.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:00:45.380+0000 7f3b04846700 1 -- 172.21.15.101:0/2325086912 wait complete. 2023-07-18T03:00:45.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:00:45.381+0000 7f3b04846700 1 Processor -- start 2023-07-18T03:00:45.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:00:45.381+0000 7f3b04846700 1 -- start start 2023-07-18T03:00:45.361 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:00:45.381+0000 7f3b04846700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b00072f70 0x7f3b0019bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:00:45.361 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:00:45.381+0000 7f3b04846700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b00074540 0x7f3b0019c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:00:45.361 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:00:45.381+0000 7f3b04846700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b0010c170 0x7f3b001a3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:00:45.361 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:00:45.381+0000 7f3b04846700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b001a3a50 con 0x7f3b0010c170 2023-07-18T03:00:45.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:00:45.381+0000 7f3b04846700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b001a3b90 con 0x7f3b00072f70 2023-07-18T03:00:45.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:00:45.381+0000 7f3b04846700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b001a3d20 con 0x7f3b00074540 2023-07-18T03:00:45.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:00:45.382+0000 7f3afed9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b0010c170 0x7f3b001a3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:00:45.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:00:45.382+0000 7f3afdd9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b00074540 0x7f3b0019c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:00:45.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:00:45.382+0000 7f3afed9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b0010c170 0x7f3b001a3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:47514/0 (socket says 172.21.15.101:47514) 2023-07-18T03:00:45.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:00:45.382+0000 7f3afed9d700 1 -- 172.21.15.101:0/2064926491 learned_addr learned my addr 172.21.15.101:0/2064926491 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:00:45.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:00:45.382+0000 7f3afe59c700 1 --2- 172.21.15.101:0/2064926491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b00072f70 0x7f3b0019bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:00:45.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:00:45.382+0000 7f3afed9d700 1 -- 172.21.15.101:0/2064926491 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b00074540 msgr2=0x7f3b0019c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:00:45.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:00:45.382+0000 7f3afed9d700 1 --2- 172.21.15.101:0/2064926491 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b00074540 0x7f3b0019c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:00:45.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:00:45.382+0000 7f3afed9d700 1 -- 172.21.15.101:0/2064926491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b00072f70 msgr2=0x7f3b0019bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:00:45.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:00:45.382+0000 7f3afed9d700 1 --2- 172.21.15.101:0/2064926491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b00072f70 0x7f3b0019bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:00:45.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:00:45.382+0000 7f3afed9d700 1 -- 172.21.15.101:0/2064926491 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af40095f0 con 0x7f3b0010c170 2023-07-18T03:00:45.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:00:45.382+0000 7f3afe59c700 1 --2- 172.21.15.101:0/2064926491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b00072f70 0x7f3b0019bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:00:45.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:00:45.382+0000 7f3afed9d700 1 --2- 172.21.15.101:0/2064926491 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b0010c170 0x7f3b001a3480 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3af800ef60 tx=0x7f3af800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:00:45.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:00:45.383+0000 7f3aef7fe700 1 -- 172.21.15.101:0/2064926491 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af8018070 con 0x7f3b0010c170 2023-07-18T03:00:45.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:00:45.383+0000 7f3aef7fe700 1 -- 172.21.15.101:0/2064926491 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3af800f040 con 0x7f3b0010c170 2023-07-18T03:00:45.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:00:45.383+0000 7f3b04846700 1 -- 172.21.15.101:0/2064926491 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b001a3fa0 con 0x7f3b0010c170 2023-07-18T03:00:45.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:00:45.383+0000 7f3aef7fe700 1 -- 172.21.15.101:0/2064926491 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af800ccc0 con 0x7f3b0010c170 2023-07-18T03:00:45.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:00:45.383+0000 7f3b04846700 1 -- 172.21.15.101:0/2064926491 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b001a43c0 con 0x7f3b0010c170 2023-07-18T03:00:45.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:00:45.384+0000 7f3b04846700 1 -- 172.21.15.101:0/2064926491 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b00101cc0 con 0x7f3b0010c170 2023-07-18T03:00:45.365 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:00:45.386+0000 7f3aef7fe700 1 -- 172.21.15.101:0/2064926491 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 15) v1 ==== 135660+0+0 (secure 0 0 0) 0x7f3af8003a00 con 0x7f3b0010c170 2023-07-18T03:00:45.365 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:00:45.387+0000 7f3aef7fe700 1 --2- 172.21.15.101:0/2064926491 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f3ae80a10c0 0x7f3ae80a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:00:45.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:00:45.387+0000 7f3aef7fe700 1 -- 172.21.15.101:0/2064926491 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3237+0+0 (secure 0 0 0) 0x7f3af80cb8a0 con 0x7f3b0010c170 2023-07-18T03:00:45.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:00:45.388+0000 7f3afe59c700 1 --2- 172.21.15.101:0/2064926491 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f3ae80a10c0 0x7f3ae80a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:00:45.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:00:45.388+0000 7f3afe59c700 1 --2- 172.21.15.101:0/2064926491 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f3ae80a10c0 0x7f3ae80a3570 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3af0011490 tx=0x7f3af0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:00:45.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:00:45.391+0000 7f3aef7fe700 1 -- 172.21.15.101:0/2064926491 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3af8096490 con 0x7f3b0010c170 2023-07-18T03:00:45.457 INFO:journalctl@ceph.osd.0.smithi101.stdout:Jul 18 03:00:45 smithi101 sudo[142236]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-18T03:00:45.457 INFO:journalctl@ceph.osd.0.smithi101.stdout:Jul 18 03:00:45 smithi101 sudo[142236]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-18T03:00:45.457 INFO:journalctl@ceph.osd.0.smithi101.stdout:Jul 18 03:00:45 smithi101 sudo[142236]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-18T03:00:45.457 INFO:journalctl@ceph.osd.0.smithi101.stdout:Jul 18 03:00:45 smithi101 sudo[142236]: pam_unix(sudo:session): session closed for user root 2023-07-18T03:00:45.472 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:00:45.497+0000 7f3aef7fe700 1 -- 172.21.15.101:0/2064926491 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f3af8003cb0 con 0x7f3b0010c170 2023-07-18T03:00:45.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:00:45.608+0000 7f3b04846700 1 -- 172.21.15.101:0/2064926491 --> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b00060950 con 0x7f3ae80a10c0 2023-07-18T03:00:45.719 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:45 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-07-18T03:00:45.719 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:45 smithi101 ceph-mon[126379]: osdmap e20: 3 total, 3 up, 3 in 2023-07-18T03:00:45.720 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:45 smithi101 ceph-mon[126379]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-18T03:00:45.720 INFO:journalctl@ceph.osd.0.smithi101.stdout:Jul 18 03:00:45 smithi101 sudo[142303]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-18T03:00:45.720 INFO:journalctl@ceph.osd.0.smithi101.stdout:Jul 18 03:00:45 smithi101 sudo[142303]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-18T03:00:45.720 INFO:journalctl@ceph.osd.0.smithi101.stdout:Jul 18 03:00:45 smithi101 sudo[142303]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-18T03:00:45.720 INFO:journalctl@ceph.osd.0.smithi101.stdout:Jul 18 03:00:45 smithi101 sudo[142303]: pam_unix(sudo:session): session closed for user root 2023-07-18T03:00:45.777 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:45 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-07-18T03:00:45.777 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:45 smithi155 ceph-mon[128885]: osdmap e20: 3 total, 3 up, 3 in 2023-07-18T03:00:45.777 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:45 smithi155 ceph-mon[128885]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-18T03:00:45.932 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:45 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-07-18T03:00:45.932 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:45 smithi179 ceph-mon[129229]: osdmap e20: 3 total, 3 up, 3 in 2023-07-18T03:00:45.932 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:45 smithi179 ceph-mon[129229]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-18T03:00:46.171 INFO:journalctl@ceph.osd.1.smithi155.stdout:Jul 18 03:00:45 smithi155 sudo[136204]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-18T03:00:46.171 INFO:journalctl@ceph.osd.1.smithi155.stdout:Jul 18 03:00:45 smithi155 sudo[136204]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-18T03:00:46.171 INFO:journalctl@ceph.osd.1.smithi155.stdout:Jul 18 03:00:45 smithi155 sudo[136204]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-18T03:00:46.171 INFO:journalctl@ceph.osd.1.smithi155.stdout:Jul 18 03:00:45 smithi155 sudo[136204]: pam_unix(sudo:session): session closed for user root 2023-07-18T03:00:46.466 INFO:journalctl@ceph.osd.1.smithi155.stdout:Jul 18 03:00:46 smithi155 sudo[136207]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-18T03:00:46.466 INFO:journalctl@ceph.osd.1.smithi155.stdout:Jul 18 03:00:46 smithi155 sudo[136207]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-18T03:00:46.466 INFO:journalctl@ceph.osd.1.smithi155.stdout:Jul 18 03:00:46 smithi155 sudo[136207]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-18T03:00:46.466 INFO:journalctl@ceph.osd.1.smithi155.stdout:Jul 18 03:00:46 smithi155 sudo[136207]: pam_unix(sudo:session): session closed for user root 2023-07-18T03:00:46.690 INFO:journalctl@ceph.osd.2.smithi179.stdout:Jul 18 03:00:46 smithi179 sudo[137208]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-18T03:00:46.690 INFO:journalctl@ceph.osd.2.smithi179.stdout:Jul 18 03:00:46 smithi179 sudo[137208]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-18T03:00:46.690 INFO:journalctl@ceph.osd.2.smithi179.stdout:Jul 18 03:00:46 smithi179 sudo[137208]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-18T03:00:46.690 INFO:journalctl@ceph.osd.2.smithi179.stdout:Jul 18 03:00:46 smithi179 sudo[137208]: pam_unix(sudo:session): session closed for user root 2023-07-18T03:00:46.691 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:46 smithi179 ceph-mon[129229]: osdmap e21: 3 total, 3 up, 3 in 2023-07-18T03:00:46.691 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:46 smithi179 ceph-mon[129229]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-07-18T03:00:46.691 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:46 smithi179 ceph-mon[129229]: from='client.14265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:00:46.691 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:46 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:00:46.691 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:46 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:00:46.691 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:46 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:00:46.692 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:46 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:46.692 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:46 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:46.719 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:46 smithi101 ceph-mon[126379]: osdmap e21: 3 total, 3 up, 3 in 2023-07-18T03:00:46.720 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:46 smithi101 ceph-mon[126379]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-07-18T03:00:46.720 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:46 smithi101 ceph-mon[126379]: from='client.14265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:00:46.720 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:46 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:00:46.720 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:46 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:00:46.720 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:46 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:00:46.720 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:46 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:46.720 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:46 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:46.777 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:46 smithi155 ceph-mon[128885]: osdmap e21: 3 total, 3 up, 3 in 2023-07-18T03:00:46.777 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:46 smithi155 ceph-mon[128885]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-07-18T03:00:46.777 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:46 smithi155 ceph-mon[128885]: from='client.14265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:00:46.778 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:46 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:00:46.778 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:46 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:00:46.778 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:46 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:00:46.778 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:46 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:46.778 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:46 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:47.182 INFO:journalctl@ceph.osd.2.smithi179.stdout:Jul 18 03:00:46 smithi179 sudo[137211]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-18T03:00:47.182 INFO:journalctl@ceph.osd.2.smithi179.stdout:Jul 18 03:00:46 smithi179 sudo[137211]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-18T03:00:47.182 INFO:journalctl@ceph.osd.2.smithi179.stdout:Jul 18 03:00:46 smithi179 sudo[137211]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-18T03:00:47.183 INFO:journalctl@ceph.osd.2.smithi179.stdout:Jul 18 03:00:46 smithi179 sudo[137211]: pam_unix(sudo:session): session closed for user root 2023-07-18T03:00:47.313 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:47 smithi101 sudo[142457]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-18T03:00:47.313 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:47 smithi101 sudo[142457]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-18T03:00:47.313 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:47 smithi101 sudo[142457]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-18T03:00:47.314 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:47 smithi101 sudo[142457]: pam_unix(sudo:session): session closed for user root 2023-07-18T03:00:47.969 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:47 smithi101 sudo[142505]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-18T03:00:47.970 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:47 smithi101 sudo[142505]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-18T03:00:47.970 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:47 smithi101 sudo[142505]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-18T03:00:47.970 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:47 smithi101 sudo[142505]: pam_unix(sudo:session): session closed for user root 2023-07-18T03:00:47.970 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:47 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:47.970 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:47 smithi101 ceph-mon[126379]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-18T03:00:48.080 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:47 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:48.080 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:47 smithi179 ceph-mon[129229]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-18T03:00:48.081 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:48 smithi179 sudo[137214]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-18T03:00:48.081 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:48 smithi179 sudo[137214]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-18T03:00:48.081 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:48 smithi179 sudo[137214]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-18T03:00:48.081 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:48 smithi179 sudo[137214]: pam_unix(sudo:session): session closed for user root 2023-07-18T03:00:48.277 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:47 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:00:48.277 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:47 smithi155 ceph-mon[128885]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-18T03:00:48.432 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:48 smithi179 sudo[137217]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-18T03:00:48.432 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:48 smithi179 sudo[137217]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-18T03:00:48.433 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:48 smithi179 sudo[137217]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-18T03:00:48.433 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:48 smithi179 sudo[137217]: pam_unix(sudo:session): session closed for user root 2023-07-18T03:00:48.811 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:48 smithi155 sudo[136210]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-18T03:00:48.812 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:48 smithi155 sudo[136210]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-18T03:00:48.812 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:48 smithi155 sudo[136210]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-18T03:00:48.812 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:48 smithi155 sudo[136210]: pam_unix(sudo:session): session closed for user root 2023-07-18T03:00:49.182 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:48 smithi179 ceph-mon[129229]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-18T03:00:49.182 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:48 smithi179 ceph-mon[129229]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-18T03:00:49.183 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:48 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-18T03:00:49.183 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:48 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T03:00:49.183 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:48 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T03:00:49.183 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:48 smithi179 ceph-mon[129229]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-18T03:00:49.183 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:48 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-18T03:00:49.183 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:48 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T03:00:49.183 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:48 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T03:00:49.184 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:48 smithi179 ceph-mon[129229]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-18T03:00:49.184 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:48 smithi179 ceph-mon[129229]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-18T03:00:49.184 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:48 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-18T03:00:49.184 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:48 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T03:00:49.184 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:48 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T03:00:49.184 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:48 smithi179 ceph-mon[129229]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-18T03:00:49.221 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:48 smithi101 ceph-mon[126379]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-18T03:00:49.222 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:48 smithi101 ceph-mon[126379]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-18T03:00:49.222 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:48 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-18T03:00:49.222 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:48 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T03:00:49.222 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:48 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T03:00:49.222 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:48 smithi101 ceph-mon[126379]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-18T03:00:49.222 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:48 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-18T03:00:49.223 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:48 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T03:00:49.223 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:48 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T03:00:49.223 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:48 smithi101 ceph-mon[126379]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-18T03:00:49.223 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:48 smithi101 ceph-mon[126379]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-18T03:00:49.223 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:48 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-18T03:00:49.223 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:48 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T03:00:49.223 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:48 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T03:00:49.224 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:48 smithi101 ceph-mon[126379]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-18T03:00:49.277 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:48 smithi155 sudo[136213]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-18T03:00:49.277 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:48 smithi155 sudo[136213]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-18T03:00:49.277 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:48 smithi155 sudo[136213]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-18T03:00:49.278 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:48 smithi155 sudo[136213]: pam_unix(sudo:session): session closed for user root 2023-07-18T03:00:49.278 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:48 smithi155 ceph-mon[128885]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-18T03:00:49.278 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:48 smithi155 ceph-mon[128885]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-18T03:00:49.278 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:48 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-18T03:00:49.278 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:48 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T03:00:49.279 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:48 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T03:00:49.279 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:48 smithi155 ceph-mon[128885]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-18T03:00:49.279 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:48 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-18T03:00:49.279 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:48 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T03:00:49.279 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:48 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T03:00:49.279 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:48 smithi155 ceph-mon[128885]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-18T03:00:49.279 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:48 smithi155 ceph-mon[128885]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-18T03:00:49.280 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:48 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-18T03:00:49.280 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:48 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T03:00:49.280 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:48 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T03:00:49.280 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:48 smithi155 ceph-mon[128885]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-18T03:00:50.182 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:49 smithi179 ceph-mon[129229]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-18T03:00:50.183 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:49 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/3757188489' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b0c409d-2954-4127-833a-b196bf226d5f"}]: dispatch 2023-07-18T03:00:50.183 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:49 smithi179 ceph-mon[129229]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b0c409d-2954-4127-833a-b196bf226d5f"}]: dispatch 2023-07-18T03:00:50.183 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:49 smithi179 ceph-mon[129229]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b0c409d-2954-4127-833a-b196bf226d5f"}]': finished 2023-07-18T03:00:50.183 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:49 smithi179 ceph-mon[129229]: osdmap e22: 4 total, 3 up, 4 in 2023-07-18T03:00:50.183 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:49 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-18T03:00:50.183 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:49 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1421998997' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:00:50.219 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:49 smithi101 ceph-mon[126379]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-18T03:00:50.220 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:49 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3757188489' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b0c409d-2954-4127-833a-b196bf226d5f"}]: dispatch 2023-07-18T03:00:50.220 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:49 smithi101 ceph-mon[126379]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b0c409d-2954-4127-833a-b196bf226d5f"}]: dispatch 2023-07-18T03:00:50.220 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:49 smithi101 ceph-mon[126379]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b0c409d-2954-4127-833a-b196bf226d5f"}]': finished 2023-07-18T03:00:50.220 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:49 smithi101 ceph-mon[126379]: osdmap e22: 4 total, 3 up, 4 in 2023-07-18T03:00:50.220 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:49 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-18T03:00:50.221 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:49 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1421998997' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:00:50.277 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:49 smithi155 ceph-mon[128885]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-18T03:00:50.277 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:49 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/3757188489' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b0c409d-2954-4127-833a-b196bf226d5f"}]: dispatch 2023-07-18T03:00:50.277 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:49 smithi155 ceph-mon[128885]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b0c409d-2954-4127-833a-b196bf226d5f"}]: dispatch 2023-07-18T03:00:50.277 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:49 smithi155 ceph-mon[128885]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b0c409d-2954-4127-833a-b196bf226d5f"}]': finished 2023-07-18T03:00:50.278 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:49 smithi155 ceph-mon[128885]: osdmap e22: 4 total, 3 up, 4 in 2023-07-18T03:00:50.278 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:49 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-18T03:00:50.278 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:49 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1421998997' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:00:52.182 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:51 smithi179 ceph-mon[129229]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-18T03:00:52.219 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:51 smithi101 ceph-mon[126379]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-18T03:00:52.277 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:51 smithi155 ceph-mon[128885]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-18T03:00:54.682 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:54 smithi101 ceph-mon[126379]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2023-07-18T03:00:54.683 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:54 smithi179 ceph-mon[129229]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2023-07-18T03:00:54.777 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:54 smithi155 ceph-mon[128885]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2023-07-18T03:00:56.470 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:56 smithi101 ceph-mon[126379]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2023-07-18T03:00:56.682 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:56 smithi179 ceph-mon[129229]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2023-07-18T03:00:56.777 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:56 smithi155 ceph-mon[128885]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2023-07-18T03:00:57.637 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:57 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-07-18T03:00:57.638 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:57 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:00:57.682 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:57 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-07-18T03:00:57.683 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:57 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:00:57.777 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:57 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-07-18T03:00:57.778 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:57 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:00:58.682 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:58 smithi179 ceph-mon[129229]: Deploying daemon osd.3 on smithi101 2023-07-18T03:00:58.683 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:00:58 smithi179 ceph-mon[129229]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2023-07-18T03:00:58.720 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:58 smithi101 ceph-mon[126379]: Deploying daemon osd.3 on smithi101 2023-07-18T03:00:58.720 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:00:58 smithi101 ceph-mon[126379]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2023-07-18T03:00:58.777 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:58 smithi155 ceph-mon[128885]: Deploying daemon osd.3 on smithi101 2023-07-18T03:00:58.778 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:00:58 smithi155 ceph-mon[128885]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2023-07-18T03:01:00.682 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:00 smithi179 ceph-mon[129229]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2023-07-18T03:01:00.720 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:00 smithi101 ceph-mon[126379]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2023-07-18T03:01:00.777 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:00 smithi155 ceph-mon[128885]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2023-07-18T03:01:02.683 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:02 smithi179 ceph-mon[129229]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 872 B/s rd, 22 KiB/s wr, 2 op/s 2023-07-18T03:01:02.720 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:02 smithi101 ceph-mon[126379]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 872 B/s rd, 22 KiB/s wr, 2 op/s 2023-07-18T03:01:02.777 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:02 smithi155 ceph-mon[128885]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 872 B/s rd, 22 KiB/s wr, 2 op/s 2023-07-18T03:01:04.433 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:04 smithi179 ceph-mon[129229]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-07-18T03:01:04.433 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:04 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:04.433 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:04 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:04.433 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:04 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:01:04.470 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:04 smithi101 ceph-mon[126379]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-07-18T03:01:04.470 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:04 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:04.470 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:04 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:04.471 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:04 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:01:04.528 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:04 smithi155 ceph-mon[128885]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-07-18T03:01:04.528 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:04 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:04.528 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:04 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:04.528 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:04 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:01:06.470 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:06 smithi101 ceph-mon[126379]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-18T03:01:06.471 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:06 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:06.471 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:06 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:06.471 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:06 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:01:06.472 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:06 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:01:06.472 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:06 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:06.528 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:06 smithi155 ceph-mon[128885]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-18T03:01:06.528 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:06 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:06.528 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:06 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:06.528 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:06 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:01:06.528 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:06 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:01:06.529 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:06 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:06.683 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:06 smithi179 ceph-mon[129229]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-18T03:01:06.683 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:06 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:06.684 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:06 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:06.684 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:06 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:01:06.684 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:06 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:01:06.684 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:06 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:07.205 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:07.229+0000 7f3aef7fe700 1 -- 172.21.15.101:0/2064926491 <== mgr.14150 v2:172.21.15.101:6800/3963742842 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3b00060950 con 0x7f3ae80a10c0 2023-07-18T03:01:07.205 INFO:teuthology.orchestra.run.smithi101.stdout:Created osd(s) 3 on host 'smithi101' 2023-07-18T03:01:07.208 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:07.232+0000 7f3b04846700 1 -- 172.21.15.101:0/2064926491 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f3ae80a10c0 msgr2=0x7f3ae80a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:01:07.208 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:07.232+0000 7f3b04846700 1 --2- 172.21.15.101:0/2064926491 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f3ae80a10c0 0x7f3ae80a3570 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3af0011490 tx=0x7f3af0009260 comp rx=0 tx=0).stop 2023-07-18T03:01:07.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:07.232+0000 7f3b04846700 1 -- 172.21.15.101:0/2064926491 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b0010c170 msgr2=0x7f3b001a3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:01:07.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:07.232+0000 7f3b04846700 1 --2- 172.21.15.101:0/2064926491 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b0010c170 0x7f3b001a3480 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3af800ef60 tx=0x7f3af800ef90 comp rx=0 tx=0).stop 2023-07-18T03:01:07.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:07.233+0000 7f3b04846700 1 -- 172.21.15.101:0/2064926491 shutdown_connections 2023-07-18T03:01:07.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:07.233+0000 7f3b04846700 1 --2- 172.21.15.101:0/2064926491 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f3ae80a10c0 0x7f3ae80a3570 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:01:07.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:07.233+0000 7f3b04846700 1 --2- 172.21.15.101:0/2064926491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b00072f70 0x7f3b0019bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:01:07.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:07.233+0000 7f3b04846700 1 --2- 172.21.15.101:0/2064926491 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b0010c170 0x7f3b001a3480 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:01:07.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:07.233+0000 7f3b04846700 1 --2- 172.21.15.101:0/2064926491 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b00074540 0x7f3b0019c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:01:07.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:07.233+0000 7f3b04846700 1 -- 172.21.15.101:0/2064926491 >> 172.21.15.101:0/2064926491 conn(0x7f3b000fb1a0 msgr2=0x7f3b000fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:01:07.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:07.233+0000 7f3b04846700 1 -- 172.21.15.101:0/2064926491 shutdown_connections 2023-07-18T03:01:07.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:07.233+0000 7f3b04846700 1 -- 172.21.15.101:0/2064926491 wait complete. 2023-07-18T03:01:07.694 DEBUG:teuthology.orchestra.run.smithi101:osd.3> sudo journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@osd.3.service 2023-07-18T03:01:07.699 INFO:tasks.cephadm:Deploying osd.4 on smithi155 with /dev/vg_nvme/lv_3... 2023-07-18T03:01:07.699 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-07-18T03:01:07.771 INFO:journalctl@ceph.osd.3.smithi101.stdout:-- Logs begin at Tue 2023-07-18 02:35:16 UTC. -- 2023-07-18T03:01:08.261 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.b/config 2023-07-18T03:01:08.433 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:08 smithi179 ceph-mon[129229]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-18T03:01:08.433 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:08 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:08.433 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:08 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:08.433 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:08 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:01:08.434 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:08 smithi179 ceph-mon[129229]: from='osd.3 [v2:172.21.15.101:6810/4226577463,v1:172.21.15.101:6811/4226577463]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-07-18T03:01:08.434 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:08 smithi179 ceph-mon[129229]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-07-18T03:01:08.448 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:08 smithi101 ceph-mon[126379]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-18T03:01:08.448 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:08 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:08.449 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:08 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:08.449 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:08 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:01:08.449 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:08 smithi101 ceph-mon[126379]: from='osd.3 [v2:172.21.15.101:6810/4226577463,v1:172.21.15.101:6811/4226577463]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-07-18T03:01:08.449 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:08 smithi101 ceph-mon[126379]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-07-18T03:01:08.476 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:08 smithi155 ceph-mon[128885]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-18T03:01:08.476 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:08 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:08.476 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:08 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:08.477 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:08 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:01:08.477 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:08 smithi155 ceph-mon[128885]: from='osd.3 [v2:172.21.15.101:6810/4226577463,v1:172.21.15.101:6811/4226577463]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-07-18T03:01:08.477 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:08 smithi155 ceph-mon[128885]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-07-18T03:01:09.471 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:09 smithi101 ceph-mon[126379]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-07-18T03:01:09.471 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:09 smithi101 ceph-mon[126379]: osdmap e23: 4 total, 3 up, 4 in 2023-07-18T03:01:09.471 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:09 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-18T03:01:09.471 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:09 smithi101 ceph-mon[126379]: from='osd.3 [v2:172.21.15.101:6810/4226577463,v1:172.21.15.101:6811/4226577463]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-07-18T03:01:09.471 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:09 smithi101 ceph-mon[126379]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-07-18T03:01:09.472 INFO:journalctl@ceph.osd.3.smithi101.stdout:Jul 18 03:01:09 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-3[144817]: 2023-07-18T03:01:09.223+0000 7fa39aa61700 -1 osd.3 0 waiting for initial osdmap 2023-07-18T03:01:09.472 INFO:journalctl@ceph.osd.3.smithi101.stdout:Jul 18 03:01:09 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-3[144817]: 2023-07-18T03:01:09.228+0000 7fa394852700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-18T03:01:09.472 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:09 smithi155 ceph-mon[128885]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-07-18T03:01:09.473 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:09 smithi155 ceph-mon[128885]: osdmap e23: 4 total, 3 up, 4 in 2023-07-18T03:01:09.473 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:09 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-18T03:01:09.473 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:09 smithi155 ceph-mon[128885]: from='osd.3 [v2:172.21.15.101:6810/4226577463,v1:172.21.15.101:6811/4226577463]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-07-18T03:01:09.473 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:09 smithi155 ceph-mon[128885]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-07-18T03:01:09.683 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:09 smithi179 ceph-mon[129229]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-07-18T03:01:09.683 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:09 smithi179 ceph-mon[129229]: osdmap e23: 4 total, 3 up, 4 in 2023-07-18T03:01:09.683 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:09 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-18T03:01:09.683 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:09 smithi179 ceph-mon[129229]: from='osd.3 [v2:172.21.15.101:6810/4226577463,v1:172.21.15.101:6811/4226577463]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-07-18T03:01:09.684 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:09 smithi179 ceph-mon[129229]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-07-18T03:01:09.903 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-07-18T03:01:09.903 INFO:teuthology.orchestra.run.smithi155.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-07-18T03:01:09.903 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10+0 records in 2023-07-18T03:01:09.903 INFO:teuthology.orchestra.run.smithi155.stderr:10+0 records out 2023-07-18T03:01:09.904 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125818 s, 833 MB/s 2023-07-18T03:01:09.904 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping successful for: 2023-07-18T03:01:10.253 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:10 smithi101 ceph-mon[126379]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-18T03:01:10.253 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:10 smithi101 ceph-mon[126379]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]': finished 2023-07-18T03:01:10.254 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:10 smithi101 ceph-mon[126379]: osdmap e24: 4 total, 3 up, 4 in 2023-07-18T03:01:10.254 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:10 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-18T03:01:10.254 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:10 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-18T03:01:10.303 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi155:vg_nvme/lv_3 2023-07-18T03:01:10.528 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:10 smithi155 ceph-mon[128885]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-18T03:01:10.528 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:10 smithi155 ceph-mon[128885]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]': finished 2023-07-18T03:01:10.528 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:10 smithi155 ceph-mon[128885]: osdmap e24: 4 total, 3 up, 4 in 2023-07-18T03:01:10.529 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:10 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-18T03:01:10.529 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:10 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-18T03:01:10.683 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:10 smithi179 ceph-mon[129229]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-18T03:01:10.684 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:10 smithi179 ceph-mon[129229]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]': finished 2023-07-18T03:01:10.684 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:10 smithi179 ceph-mon[129229]: osdmap e24: 4 total, 3 up, 4 in 2023-07-18T03:01:10.684 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:10 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-18T03:01:10.684 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:10 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-18T03:01:10.856 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.b/config 2023-07-18T03:01:11.353 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:11 smithi155 ceph-mon[128885]: purged_snaps scrub starts 2023-07-18T03:01:11.353 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:11 smithi155 ceph-mon[128885]: purged_snaps scrub ok 2023-07-18T03:01:11.353 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:11 smithi155 ceph-mon[128885]: osd.3 [v2:172.21.15.101:6810/4226577463,v1:172.21.15.101:6811/4226577463] boot 2023-07-18T03:01:11.354 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:11 smithi155 ceph-mon[128885]: osdmap e25: 4 total, 4 up, 4 in 2023-07-18T03:01:11.354 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:11 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-18T03:01:11.354 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:11 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:11.355 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:11 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:11.355 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:11 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:01:11.355 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:11 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:01:11.356 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:11 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:11.356 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:11 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:01:11.356 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:11 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:01:11.357 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:11 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:11.357 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:11 smithi155 ceph-mon[128885]: osdmap e26: 4 total, 4 up, 4 in 2023-07-18T03:01:11.471 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:11 smithi101 ceph-mon[126379]: purged_snaps scrub starts 2023-07-18T03:01:11.471 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:11 smithi101 ceph-mon[126379]: purged_snaps scrub ok 2023-07-18T03:01:11.471 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:11 smithi101 ceph-mon[126379]: osd.3 [v2:172.21.15.101:6810/4226577463,v1:172.21.15.101:6811/4226577463] boot 2023-07-18T03:01:11.471 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:11 smithi101 ceph-mon[126379]: osdmap e25: 4 total, 4 up, 4 in 2023-07-18T03:01:11.471 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:11 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-18T03:01:11.471 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:11 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:11.472 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:11 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:11.472 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:11 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:01:11.472 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:11 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:01:11.472 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:11 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:11.472 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:11 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:01:11.472 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:11 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:01:11.472 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:11 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:11.473 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:11 smithi101 ceph-mon[126379]: osdmap e26: 4 total, 4 up, 4 in 2023-07-18T03:01:11.683 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:11 smithi179 ceph-mon[129229]: purged_snaps scrub starts 2023-07-18T03:01:11.683 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:11 smithi179 ceph-mon[129229]: purged_snaps scrub ok 2023-07-18T03:01:11.684 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:11 smithi179 ceph-mon[129229]: osd.3 [v2:172.21.15.101:6810/4226577463,v1:172.21.15.101:6811/4226577463] boot 2023-07-18T03:01:11.684 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:11 smithi179 ceph-mon[129229]: osdmap e25: 4 total, 4 up, 4 in 2023-07-18T03:01:11.684 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:11 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-18T03:01:11.684 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:11 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:11.684 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:11 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:11.684 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:11 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:01:11.684 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:11 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:01:11.685 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:11 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:11.685 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:11 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:01:11.685 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:11 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:01:11.685 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:11 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:11.685 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:11 smithi179 ceph-mon[129229]: osdmap e26: 4 total, 4 up, 4 in 2023-07-18T03:01:12.087 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:12.110+0000 7f328ced6700 1 -- 172.21.15.155:0/1805724174 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32880fea80 msgr2=0x7f32880fee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:01:12.087 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:12.110+0000 7f328ced6700 1 --2- 172.21.15.155:0/1805724174 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32880fea80 0x7f32880fee90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3278009990 tx=0x7f3278009ca0 comp rx=0 tx=0).stop 2023-07-18T03:01:12.087 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:12.110+0000 7f328ced6700 1 -- 172.21.15.155:0/1805724174 shutdown_connections 2023-07-18T03:01:12.088 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:12.110+0000 7f328ced6700 1 --2- 172.21.15.155:0/1805724174 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32880fea80 0x7f32880fee90 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:01:12.088 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:12.110+0000 7f328ced6700 1 --2- 172.21.15.155:0/1805724174 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3288100610 0x7f3288106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:01:12.088 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:12.110+0000 7f328ced6700 1 --2- 172.21.15.155:0/1805724174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32880ffc80 0x7f32881000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:01:12.088 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:12.110+0000 7f328ced6700 1 -- 172.21.15.155:0/1805724174 >> 172.21.15.155:0/1805724174 conn(0x7f32880fa0b0 msgr2=0x7f32880fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:01:12.089 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:12.110+0000 7f328ced6700 1 -- 172.21.15.155:0/1805724174 shutdown_connections 2023-07-18T03:01:12.089 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:12.110+0000 7f328ced6700 1 -- 172.21.15.155:0/1805724174 wait complete. 2023-07-18T03:01:12.093 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:12.111+0000 7f328ced6700 1 Processor -- start 2023-07-18T03:01:12.094 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:12.111+0000 7f328ced6700 1 -- start start 2023-07-18T03:01:12.094 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:12.111+0000 7f328ced6700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f32880fea80 0x7f32881947f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:01:12.094 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:12.112+0000 7f328ced6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32880ffc80 0x7f3288194d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:01:12.096 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:12.112+0000 7f328ced6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3288100610 0x7f328819bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:01:12.096 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:12.112+0000 7f328ced6700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328819c380 con 0x7f32880fea80 2023-07-18T03:01:12.096 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:12.112+0000 7f328ced6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328819c4c0 con 0x7f32880ffc80 2023-07-18T03:01:12.097 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:12.112+0000 7f328ced6700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328819c650 con 0x7f3288100610 2023-07-18T03:01:12.097 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:12.112+0000 7f3285d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32880ffc80 0x7f3288194d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:01:12.097 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:12.112+0000 7f3285d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32880ffc80 0x7f3288194d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:49672/0 (socket says 172.21.15.155:49672) 2023-07-18T03:01:12.097 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:12.112+0000 7f3285d9b700 1 -- 172.21.15.155:0/2198624426 learned_addr learned my addr 172.21.15.155:0/2198624426 (peer_addr_for_me v2:172.21.15.155:0/0) 2023-07-18T03:01:12.097 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:12.112+0000 7f3286d9d700 1 --2- 172.21.15.155:0/2198624426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3288100610 0x7f328819bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:01:12.097 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:12.112+0000 7f328659c700 1 --2- 172.21.15.155:0/2198624426 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f32880fea80 0x7f32881947f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:01:12.098 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:12.112+0000 7f3285d9b700 1 -- 172.21.15.155:0/2198624426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3288100610 msgr2=0x7f328819bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:01:12.098 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:12.112+0000 7f3285d9b700 1 --2- 172.21.15.155:0/2198624426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3288100610 0x7f328819bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:01:12.098 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:12.112+0000 7f3285d9b700 1 -- 172.21.15.155:0/2198624426 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f32880fea80 msgr2=0x7f32881947f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:01:12.098 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:12.112+0000 7f3285d9b700 1 --2- 172.21.15.155:0/2198624426 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f32880fea80 0x7f32881947f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:01:12.098 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:12.112+0000 7f3285d9b700 1 -- 172.21.15.155:0/2198624426 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32780095f0 con 0x7f32880ffc80 2023-07-18T03:01:12.098 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:12.112+0000 7f3285d9b700 1 --2- 172.21.15.155:0/2198624426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32880ffc80 0x7f3288194d30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3270009970 tx=0x7f3270009c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:01:12.099 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:12.112+0000 7f327f7fe700 1 -- 172.21.15.155:0/2198624426 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3270012070 con 0x7f32880ffc80 2023-07-18T03:01:12.099 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:12.112+0000 7f328ced6700 1 -- 172.21.15.155:0/2198624426 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3288068030 con 0x7f32880ffc80 2023-07-18T03:01:12.099 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:12.112+0000 7f327f7fe700 1 -- 172.21.15.155:0/2198624426 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f327000d260 con 0x7f32880ffc80 2023-07-18T03:01:12.099 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:12.112+0000 7f327f7fe700 1 -- 172.21.15.155:0/2198624426 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3270017a60 con 0x7f32880ffc80 2023-07-18T03:01:12.099 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:12.112+0000 7f328ced6700 1 -- 172.21.15.155:0/2198624426 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3288068580 con 0x7f32880ffc80 2023-07-18T03:01:12.099 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:12.113+0000 7f328ced6700 1 -- 172.21.15.155:0/2198624426 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f328810a2c0 con 0x7f32880ffc80 2023-07-18T03:01:12.099 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:12.114+0000 7f327f7fe700 1 -- 172.21.15.155:0/2198624426 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f3270007c00 con 0x7f32880ffc80 2023-07-18T03:01:12.100 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:12.115+0000 7f327f7fe700 1 --2- 172.21.15.155:0/2198624426 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f32740a1150 0x7f32740a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:01:12.100 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:12.115+0000 7f327f7fe700 1 -- 172.21.15.155:0/2198624426 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 3698+0+0 (secure 0 0 0) 0x7f32700cca10 con 0x7f32880ffc80 2023-07-18T03:01:12.100 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:12.115+0000 7f328659c700 1 --2- 172.21.15.155:0/2198624426 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f32740a1150 0x7f32740a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:01:12.100 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:12.116+0000 7f328659c700 1 --2- 172.21.15.155:0/2198624426 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f32740a1150 0x7f32740a3600 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3278006010 tx=0x7f327800b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:01:12.101 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:12.120+0000 7f327f7fe700 1 -- 172.21.15.155:0/2198624426 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3270097360 con 0x7f32880ffc80 2023-07-18T03:01:12.291 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:12.315+0000 7f328ced6700 1 -- 172.21.15.155:0/2198624426 --> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f3288068800 con 0x7f32740a1150 2023-07-18T03:01:12.471 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:12 smithi101 ceph-mon[126379]: Detected new or changed devices on smithi101 2023-07-18T03:01:12.471 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:12 smithi101 ceph-mon[126379]: Adjusting osd_memory_target on smithi101 to 8633M 2023-07-18T03:01:12.471 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:12 smithi101 ceph-mon[126379]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-18T03:01:12.471 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:12 smithi101 ceph-mon[126379]: osdmap e27: 4 total, 4 up, 4 in 2023-07-18T03:01:12.528 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:12 smithi155 ceph-mon[128885]: Detected new or changed devices on smithi101 2023-07-18T03:01:12.528 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:12 smithi155 ceph-mon[128885]: Adjusting osd_memory_target on smithi101 to 8633M 2023-07-18T03:01:12.528 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:12 smithi155 ceph-mon[128885]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-18T03:01:12.529 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:12 smithi155 ceph-mon[128885]: osdmap e27: 4 total, 4 up, 4 in 2023-07-18T03:01:12.683 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:12 smithi179 ceph-mon[129229]: Detected new or changed devices on smithi101 2023-07-18T03:01:12.683 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:12 smithi179 ceph-mon[129229]: Adjusting osd_memory_target on smithi101 to 8633M 2023-07-18T03:01:12.683 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:12 smithi179 ceph-mon[129229]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-18T03:01:12.684 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:12 smithi179 ceph-mon[129229]: osdmap e27: 4 total, 4 up, 4 in 2023-07-18T03:01:13.361 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:13 smithi155 ceph-mon[128885]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:01:13.361 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:13 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:01:13.361 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:13 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:01:13.362 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:13 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:01:13.470 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:13 smithi101 ceph-mon[126379]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:01:13.471 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:13 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:01:13.471 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:13 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:01:13.471 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:13 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:01:13.683 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:13 smithi179 ceph-mon[129229]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:01:13.684 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:13 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:01:13.684 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:13 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:01:13.684 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:13 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:01:14.470 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:14 smithi101 ceph-mon[126379]: pgmap v102: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2023-07-18T03:01:14.471 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:14 smithi101 ceph-mon[126379]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2023-07-18T03:01:14.471 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:14 smithi101 ceph-mon[126379]: osdmap e28: 4 total, 4 up, 4 in 2023-07-18T03:01:14.528 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:14 smithi155 ceph-mon[128885]: pgmap v102: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2023-07-18T03:01:14.528 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:14 smithi155 ceph-mon[128885]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2023-07-18T03:01:14.528 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:14 smithi155 ceph-mon[128885]: osdmap e28: 4 total, 4 up, 4 in 2023-07-18T03:01:14.683 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:14 smithi179 ceph-mon[129229]: pgmap v102: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2023-07-18T03:01:14.683 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:14 smithi179 ceph-mon[129229]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2023-07-18T03:01:14.684 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:14 smithi179 ceph-mon[129229]: osdmap e28: 4 total, 4 up, 4 in 2023-07-18T03:01:15.470 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:15 smithi101 ceph-mon[126379]: osdmap e29: 4 total, 4 up, 4 in 2023-07-18T03:01:15.471 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:15 smithi101 ceph-mon[126379]: from='client.? 172.21.15.155:0/406429754' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2137f2df-984e-45ce-baf6-b4ac12c7e3c8"}]: dispatch 2023-07-18T03:01:15.471 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:15 smithi101 ceph-mon[126379]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2137f2df-984e-45ce-baf6-b4ac12c7e3c8"}]: dispatch 2023-07-18T03:01:15.471 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:15 smithi101 ceph-mon[126379]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2137f2df-984e-45ce-baf6-b4ac12c7e3c8"}]': finished 2023-07-18T03:01:15.471 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:15 smithi101 ceph-mon[126379]: osdmap e30: 5 total, 4 up, 5 in 2023-07-18T03:01:15.472 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:15 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-18T03:01:15.528 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:15 smithi155 ceph-mon[128885]: osdmap e29: 4 total, 4 up, 4 in 2023-07-18T03:01:15.529 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:15 smithi155 ceph-mon[128885]: from='client.? 172.21.15.155:0/406429754' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2137f2df-984e-45ce-baf6-b4ac12c7e3c8"}]: dispatch 2023-07-18T03:01:15.529 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:15 smithi155 ceph-mon[128885]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2137f2df-984e-45ce-baf6-b4ac12c7e3c8"}]: dispatch 2023-07-18T03:01:15.529 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:15 smithi155 ceph-mon[128885]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2137f2df-984e-45ce-baf6-b4ac12c7e3c8"}]': finished 2023-07-18T03:01:15.530 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:15 smithi155 ceph-mon[128885]: osdmap e30: 5 total, 4 up, 5 in 2023-07-18T03:01:15.530 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:15 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-18T03:01:15.683 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:15 smithi179 ceph-mon[129229]: osdmap e29: 4 total, 4 up, 4 in 2023-07-18T03:01:15.683 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:15 smithi179 ceph-mon[129229]: from='client.? 172.21.15.155:0/406429754' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2137f2df-984e-45ce-baf6-b4ac12c7e3c8"}]: dispatch 2023-07-18T03:01:15.684 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:15 smithi179 ceph-mon[129229]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2137f2df-984e-45ce-baf6-b4ac12c7e3c8"}]: dispatch 2023-07-18T03:01:15.684 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:15 smithi179 ceph-mon[129229]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2137f2df-984e-45ce-baf6-b4ac12c7e3c8"}]': finished 2023-07-18T03:01:15.684 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:15 smithi179 ceph-mon[129229]: osdmap e30: 5 total, 4 up, 5 in 2023-07-18T03:01:15.684 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:15 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-18T03:01:16.470 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:16 smithi101 ceph-mon[126379]: pgmap v106: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2023-07-18T03:01:16.471 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:16 smithi101 ceph-mon[126379]: from='client.? 172.21.15.155:0/3039394331' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:01:16.528 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:16 smithi155 ceph-mon[128885]: pgmap v106: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2023-07-18T03:01:16.528 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:16 smithi155 ceph-mon[128885]: from='client.? 172.21.15.155:0/3039394331' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:01:16.683 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:16 smithi179 ceph-mon[129229]: pgmap v106: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2023-07-18T03:01:16.683 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:16 smithi179 ceph-mon[129229]: from='client.? 172.21.15.155:0/3039394331' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:01:18.485 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:18 smithi155 ceph-mon[128885]: pgmap v107: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2023-07-18T03:01:18.683 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:18 smithi179 ceph-mon[129229]: pgmap v107: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2023-07-18T03:01:18.721 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:18 smithi101 ceph-mon[126379]: pgmap v107: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2023-07-18T03:01:19.683 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:19 smithi179 ceph-mon[129229]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-18T03:01:19.684 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:19 smithi179 ceph-mon[129229]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2023-07-18T03:01:19.684 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:19 smithi179 ceph-mon[129229]: Cluster is now healthy 2023-07-18T03:01:19.721 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:19 smithi101 ceph-mon[126379]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-18T03:01:19.721 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:19 smithi101 ceph-mon[126379]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2023-07-18T03:01:19.721 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:19 smithi101 ceph-mon[126379]: Cluster is now healthy 2023-07-18T03:01:19.778 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:19 smithi155 ceph-mon[128885]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-18T03:01:19.778 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:19 smithi155 ceph-mon[128885]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2023-07-18T03:01:19.779 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:19 smithi155 ceph-mon[128885]: Cluster is now healthy 2023-07-18T03:01:22.157 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:21 smithi155 ceph-mon[128885]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2023-07-18T03:01:22.158 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:21 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-07-18T03:01:22.158 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:21 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:01:22.183 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:21 smithi179 ceph-mon[129229]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2023-07-18T03:01:22.184 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:21 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-07-18T03:01:22.184 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:21 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:01:22.221 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:21 smithi101 ceph-mon[126379]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2023-07-18T03:01:22.221 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:21 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-07-18T03:01:22.221 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:21 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:01:23.028 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:22 smithi155 ceph-mon[128885]: Deploying daemon osd.4 on smithi155 2023-07-18T03:01:23.183 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:22 smithi179 ceph-mon[129229]: Deploying daemon osd.4 on smithi155 2023-07-18T03:01:23.221 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:22 smithi101 ceph-mon[126379]: Deploying daemon osd.4 on smithi155 2023-07-18T03:01:24.684 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:24 smithi179 ceph-mon[129229]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2023-07-18T03:01:24.721 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:24 smithi101 ceph-mon[126379]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2023-07-18T03:01:24.779 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:24 smithi155 ceph-mon[128885]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2023-07-18T03:01:26.479 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:26 smithi155 ceph-mon[128885]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2023-07-18T03:01:26.684 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:26 smithi179 ceph-mon[129229]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2023-07-18T03:01:26.721 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:26 smithi101 ceph-mon[126379]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2023-07-18T03:01:27.693 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:27 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:27.693 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:27 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:27.693 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:27 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:01:27.693 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:27 smithi155 ceph-mon[128885]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-18T03:01:27.934 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:27 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:27.934 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:27 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:27.934 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:27 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:01:27.934 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:27 smithi179 ceph-mon[129229]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-18T03:01:27.971 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:27 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:27.971 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:27 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:27.972 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:27 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:01:27.972 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:27 smithi101 ceph-mon[126379]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-18T03:01:29.558 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:29.581+0000 7f327f7fe700 1 -- 172.21.15.155:0/2198624426 <== mgr.14150 v2:172.21.15.101:6800/3963742842 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3288068800 con 0x7f32740a1150 2023-07-18T03:01:29.559 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 4 on host 'smithi155' 2023-07-18T03:01:29.561 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:29.583+0000 7f328ced6700 1 -- 172.21.15.155:0/2198624426 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f32740a1150 msgr2=0x7f32740a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:01:29.561 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:29.583+0000 7f328ced6700 1 --2- 172.21.15.155:0/2198624426 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f32740a1150 0x7f32740a3600 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3278006010 tx=0x7f327800b540 comp rx=0 tx=0).stop 2023-07-18T03:01:29.561 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:29.583+0000 7f328ced6700 1 -- 172.21.15.155:0/2198624426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32880ffc80 msgr2=0x7f3288194d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:01:29.561 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:29.583+0000 7f328ced6700 1 --2- 172.21.15.155:0/2198624426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32880ffc80 0x7f3288194d30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3270009970 tx=0x7f3270009c80 comp rx=0 tx=0).stop 2023-07-18T03:01:29.562 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:29.583+0000 7f328ced6700 1 -- 172.21.15.155:0/2198624426 shutdown_connections 2023-07-18T03:01:29.562 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:29.583+0000 7f328ced6700 1 --2- 172.21.15.155:0/2198624426 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f32740a1150 0x7f32740a3600 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:01:29.562 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:29.583+0000 7f328ced6700 1 --2- 172.21.15.155:0/2198624426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3288100610 0x7f328819bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:01:29.562 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:29.583+0000 7f328ced6700 1 --2- 172.21.15.155:0/2198624426 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f32880fea80 0x7f32881947f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:01:29.562 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:29.583+0000 7f328ced6700 1 --2- 172.21.15.155:0/2198624426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32880ffc80 0x7f3288194d30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:01:29.562 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:29.583+0000 7f328ced6700 1 -- 172.21.15.155:0/2198624426 >> 172.21.15.155:0/2198624426 conn(0x7f32880fa0b0 msgr2=0x7f32880fc380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:01:29.562 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:29.583+0000 7f328ced6700 1 -- 172.21.15.155:0/2198624426 shutdown_connections 2023-07-18T03:01:29.563 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:01:29.583+0000 7f328ced6700 1 -- 172.21.15.155:0/2198624426 wait complete. 2023-07-18T03:01:30.155 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:29 smithi155 ceph-mon[128885]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-18T03:01:30.155 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:29 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:30.155 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:29 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:30.155 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:29 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:01:30.156 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:29 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:01:30.156 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:29 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:30.156 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:29 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:30.156 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:29 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:30.156 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:29 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:01:30.157 DEBUG:teuthology.orchestra.run.smithi155:osd.4> sudo journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@osd.4.service 2023-07-18T03:01:30.159 INFO:tasks.cephadm:Deploying osd.5 on smithi179 with /dev/vg_nvme/lv_3... 2023-07-18T03:01:30.159 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-07-18T03:01:30.184 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:29 smithi179 ceph-mon[129229]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-18T03:01:30.184 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:29 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:30.184 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:29 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:30.185 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:29 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:01:30.185 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:29 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:01:30.185 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:29 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:30.185 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:29 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:30.185 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:29 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:30.185 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:29 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:01:30.204 INFO:journalctl@ceph.osd.4.smithi155.stdout:-- Logs begin at Tue 2023-07-18 02:29:36 UTC. -- 2023-07-18T03:01:30.221 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:29 smithi101 ceph-mon[126379]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-18T03:01:30.221 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:29 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:30.222 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:29 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:30.222 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:29 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:01:30.222 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:29 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:01:30.222 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:29 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:30.222 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:29 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:30.222 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:29 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:30.222 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:29 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:01:30.734 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.c/config 2023-07-18T03:01:30.779 INFO:journalctl@ceph.osd.4.smithi155.stdout:Jul 18 03:01:30 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-4[139452]: 2023-07-18T03:01:30.457+0000 7fbda7aaa640 -1 osd.4 0 log_to_monitors true 2023-07-18T03:01:31.184 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:30 smithi155 ceph-mon[128885]: from='osd.4 [v2:172.21.15.155:6808/1951135805,v1:172.21.15.155:6809/1951135805]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-18T03:01:31.184 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:30 smithi155 ceph-mon[128885]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-18T03:01:31.185 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:30 smithi179 ceph-mon[129229]: from='osd.4 [v2:172.21.15.155:6808/1951135805,v1:172.21.15.155:6809/1951135805]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-18T03:01:31.185 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:30 smithi179 ceph-mon[129229]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-18T03:01:31.221 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:30 smithi101 ceph-mon[126379]: from='osd.4 [v2:172.21.15.155:6808/1951135805,v1:172.21.15.155:6809/1951135805]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-18T03:01:31.222 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:30 smithi101 ceph-mon[126379]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-18T03:01:32.167 INFO:journalctl@ceph.osd.4.smithi155.stdout:Jul 18 03:01:31 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-4[139452]: 2023-07-18T03:01:31.904+0000 7fbd9d122700 -1 osd.4 0 waiting for initial osdmap 2023-07-18T03:01:32.168 INFO:journalctl@ceph.osd.4.smithi155.stdout:Jul 18 03:01:31 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-4[139452]: 2023-07-18T03:01:31.909+0000 7fbd97f15700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-18T03:01:32.168 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:31 smithi155 ceph-mon[128885]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-18T03:01:32.168 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:31 smithi155 ceph-mon[128885]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-07-18T03:01:32.168 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:31 smithi155 ceph-mon[128885]: osdmap e31: 5 total, 4 up, 5 in 2023-07-18T03:01:32.169 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:31 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-18T03:01:32.169 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:31 smithi155 ceph-mon[128885]: from='osd.4 [v2:172.21.15.155:6808/1951135805,v1:172.21.15.155:6809/1951135805]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-18T03:01:32.169 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:31 smithi155 ceph-mon[128885]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-18T03:01:32.184 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:31 smithi179 ceph-mon[129229]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-18T03:01:32.184 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:31 smithi179 ceph-mon[129229]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-07-18T03:01:32.184 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:31 smithi179 ceph-mon[129229]: osdmap e31: 5 total, 4 up, 5 in 2023-07-18T03:01:32.184 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:31 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-18T03:01:32.184 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:31 smithi179 ceph-mon[129229]: from='osd.4 [v2:172.21.15.155:6808/1951135805,v1:172.21.15.155:6809/1951135805]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-18T03:01:32.185 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:31 smithi179 ceph-mon[129229]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-18T03:01:32.221 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:31 smithi101 ceph-mon[126379]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-18T03:01:32.222 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:31 smithi101 ceph-mon[126379]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-07-18T03:01:32.222 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:31 smithi101 ceph-mon[126379]: osdmap e31: 5 total, 4 up, 5 in 2023-07-18T03:01:32.222 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:31 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-18T03:01:32.222 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:31 smithi101 ceph-mon[126379]: from='osd.4 [v2:172.21.15.155:6808/1951135805,v1:172.21.15.155:6809/1951135805]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-18T03:01:32.222 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:31 smithi101 ceph-mon[126379]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-18T03:01:32.226 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-07-18T03:01:32.227 INFO:teuthology.orchestra.run.smithi179.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-07-18T03:01:32.227 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10+0 records in 2023-07-18T03:01:32.227 INFO:teuthology.orchestra.run.smithi179.stderr:10+0 records out 2023-07-18T03:01:32.227 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110399 s, 950 MB/s 2023-07-18T03:01:32.227 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping successful for: 2023-07-18T03:01:32.677 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi179:vg_nvme/lv_3 2023-07-18T03:01:33.184 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:32 smithi179 ceph-mon[129229]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2023-07-18T03:01:33.184 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:32 smithi179 ceph-mon[129229]: osdmap e32: 5 total, 4 up, 5 in 2023-07-18T03:01:33.184 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:32 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-18T03:01:33.185 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:32 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-18T03:01:33.185 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:32 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:33.185 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:32 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:33.185 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:32 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:01:33.186 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:32 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:01:33.186 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:32 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:33.186 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:32 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:01:33.186 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:32 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:01:33.186 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:32 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:33.221 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:32 smithi101 ceph-mon[126379]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2023-07-18T03:01:33.222 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:32 smithi101 ceph-mon[126379]: osdmap e32: 5 total, 4 up, 5 in 2023-07-18T03:01:33.222 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:32 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-18T03:01:33.222 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:32 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-18T03:01:33.222 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:32 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:33.222 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:32 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:33.222 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:32 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:01:33.222 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:32 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:01:33.223 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:32 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:33.223 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:32 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:01:33.223 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:32 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:01:33.223 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:32 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:33.267 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.c/config 2023-07-18T03:01:33.279 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:32 smithi155 ceph-mon[128885]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2023-07-18T03:01:33.279 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:32 smithi155 ceph-mon[128885]: osdmap e32: 5 total, 4 up, 5 in 2023-07-18T03:01:33.279 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:32 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-18T03:01:33.279 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:32 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-18T03:01:33.280 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:32 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:33.280 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:32 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:33.280 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:32 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:01:33.280 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:32 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:01:33.280 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:32 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:33.280 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:32 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:01:33.280 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:32 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:01:33.281 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:32 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:34.012 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:33 smithi179 ceph-mon[129229]: purged_snaps scrub starts 2023-07-18T03:01:34.012 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:33 smithi179 ceph-mon[129229]: purged_snaps scrub ok 2023-07-18T03:01:34.012 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:33 smithi179 ceph-mon[129229]: Detected new or changed devices on smithi155 2023-07-18T03:01:34.013 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:33 smithi179 ceph-mon[129229]: Adjusting osd_memory_target on smithi155 to 8584M 2023-07-18T03:01:34.013 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:33 smithi179 ceph-mon[129229]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-18T03:01:34.013 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:33 smithi179 ceph-mon[129229]: osd.4 [v2:172.21.15.155:6808/1951135805,v1:172.21.15.155:6809/1951135805] boot 2023-07-18T03:01:34.013 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:33 smithi179 ceph-mon[129229]: osdmap e33: 5 total, 5 up, 5 in 2023-07-18T03:01:34.013 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:33 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-18T03:01:34.221 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:33 smithi101 ceph-mon[126379]: purged_snaps scrub starts 2023-07-18T03:01:34.222 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:33 smithi101 ceph-mon[126379]: purged_snaps scrub ok 2023-07-18T03:01:34.222 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:33 smithi101 ceph-mon[126379]: Detected new or changed devices on smithi155 2023-07-18T03:01:34.222 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:33 smithi101 ceph-mon[126379]: Adjusting osd_memory_target on smithi155 to 8584M 2023-07-18T03:01:34.222 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:33 smithi101 ceph-mon[126379]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-18T03:01:34.222 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:33 smithi101 ceph-mon[126379]: osd.4 [v2:172.21.15.155:6808/1951135805,v1:172.21.15.155:6809/1951135805] boot 2023-07-18T03:01:34.222 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:33 smithi101 ceph-mon[126379]: osdmap e33: 5 total, 5 up, 5 in 2023-07-18T03:01:34.223 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:33 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-18T03:01:34.279 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:33 smithi155 ceph-mon[128885]: purged_snaps scrub starts 2023-07-18T03:01:34.279 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:33 smithi155 ceph-mon[128885]: purged_snaps scrub ok 2023-07-18T03:01:34.279 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:33 smithi155 ceph-mon[128885]: Detected new or changed devices on smithi155 2023-07-18T03:01:34.279 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:33 smithi155 ceph-mon[128885]: Adjusting osd_memory_target on smithi155 to 8584M 2023-07-18T03:01:34.280 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:33 smithi155 ceph-mon[128885]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-18T03:01:34.280 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:33 smithi155 ceph-mon[128885]: osd.4 [v2:172.21.15.155:6808/1951135805,v1:172.21.15.155:6809/1951135805] boot 2023-07-18T03:01:34.280 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:33 smithi155 ceph-mon[128885]: osdmap e33: 5 total, 5 up, 5 in 2023-07-18T03:01:34.280 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:33 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-18T03:01:34.445 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:34.467+0000 7f82d4b19700 1 -- 172.21.15.179:0/942834814 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82d0100e30 msgr2=0x7f82d0101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:01:34.445 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:34.467+0000 7f82d4b19700 1 --2- 172.21.15.179:0/942834814 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82d0100e30 0x7f82d0101280 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f82c0009990 tx=0x7f82c0009ca0 comp rx=0 tx=0).stop 2023-07-18T03:01:34.445 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:34.467+0000 7f82d4b19700 1 -- 172.21.15.179:0/942834814 shutdown_connections 2023-07-18T03:01:34.446 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:34.467+0000 7f82d4b19700 1 --2- 172.21.15.179:0/942834814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82d00ffc30 0x7f82d0100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:01:34.446 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:34.467+0000 7f82d4b19700 1 --2- 172.21.15.179:0/942834814 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f82d01017c0 0x7f82d0108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:01:34.446 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:34.467+0000 7f82d4b19700 1 --2- 172.21.15.179:0/942834814 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82d0100e30 0x7f82d0101280 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:01:34.446 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:34.467+0000 7f82d4b19700 1 -- 172.21.15.179:0/942834814 >> 172.21.15.179:0/942834814 conn(0x7f82d00fb2a0 msgr2=0x7f82d00fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:01:34.446 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:34.467+0000 7f82d4b19700 1 -- 172.21.15.179:0/942834814 shutdown_connections 2023-07-18T03:01:34.446 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:34.467+0000 7f82d4b19700 1 -- 172.21.15.179:0/942834814 wait complete. 2023-07-18T03:01:34.447 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:34.468+0000 7f82d4b19700 1 Processor -- start 2023-07-18T03:01:34.447 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:34.468+0000 7f82d4b19700 1 -- start start 2023-07-18T03:01:34.448 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:34.468+0000 7f82d4b19700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f82d00ffc30 0x7f82d0197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:01:34.448 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:34.468+0000 7f82d4b19700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82d0100e30 0x7f82d0198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:01:34.448 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:34.468+0000 7f82d4b19700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82d01017c0 0x7f82d019f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:01:34.448 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:34.468+0000 7f82d4b19700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d019f670 con 0x7f82d00ffc30 2023-07-18T03:01:34.449 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:34.468+0000 7f82d4b19700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d019f7b0 con 0x7f82d0100e30 2023-07-18T03:01:34.449 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:34.468+0000 7f82d4b19700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d019f940 con 0x7f82d01017c0 2023-07-18T03:01:34.449 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:34.468+0000 7f82ced9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82d01017c0 0x7f82d019f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:01:34.449 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:34.468+0000 7f82ced9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82d01017c0 0x7f82d019f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48268/0 (socket says 172.21.15.179:48268) 2023-07-18T03:01:34.449 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:34.468+0000 7f82ced9d700 1 -- 172.21.15.179:0/305690558 learned_addr learned my addr 172.21.15.179:0/305690558 (peer_addr_for_me v2:172.21.15.179:0/0) 2023-07-18T03:01:34.449 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:34.468+0000 7f82cdd9b700 1 --2- 172.21.15.179:0/305690558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82d0100e30 0x7f82d0198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:01:34.449 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:34.468+0000 7f82ced9d700 1 -- 172.21.15.179:0/305690558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82d0100e30 msgr2=0x7f82d0198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:01:34.450 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:34.468+0000 7f82ced9d700 1 --2- 172.21.15.179:0/305690558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82d0100e30 0x7f82d0198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:01:34.450 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:34.468+0000 7f82ced9d700 1 -- 172.21.15.179:0/305690558 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f82d00ffc30 msgr2=0x7f82d0197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:01:34.450 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:34.468+0000 7f82ced9d700 1 --2- 172.21.15.179:0/305690558 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f82d00ffc30 0x7f82d0197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:01:34.450 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:34.468+0000 7f82ced9d700 1 -- 172.21.15.179:0/305690558 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c00095f0 con 0x7f82d01017c0 2023-07-18T03:01:34.450 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:34.469+0000 7f82ced9d700 1 --2- 172.21.15.179:0/305690558 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82d01017c0 0x7f82d019f0a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f82c800b9a0 tx=0x7f82c800bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:01:34.450 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:34.469+0000 7f82bf7fe700 1 -- 172.21.15.179:0/305690558 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c800c9b0 con 0x7f82d01017c0 2023-07-18T03:01:34.450 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:34.469+0000 7f82d4b19700 1 -- 172.21.15.179:0/305690558 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82d019fc20 con 0x7f82d01017c0 2023-07-18T03:01:34.450 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:34.469+0000 7f82bf7fe700 1 -- 172.21.15.179:0/305690558 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82c8014440 con 0x7f82d01017c0 2023-07-18T03:01:34.451 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:34.469+0000 7f82bf7fe700 1 -- 172.21.15.179:0/305690558 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c80075d0 con 0x7f82d01017c0 2023-07-18T03:01:34.451 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:34.469+0000 7f82d4b19700 1 -- 172.21.15.179:0/305690558 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82d01a00d0 con 0x7f82d01017c0 2023-07-18T03:01:34.451 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:34.470+0000 7f82d4b19700 1 -- 172.21.15.179:0/305690558 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82d010b470 con 0x7f82d01017c0 2023-07-18T03:01:34.452 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:34.471+0000 7f82bf7fe700 1 -- 172.21.15.179:0/305690558 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f82c80145b0 con 0x7f82d01017c0 2023-07-18T03:01:34.452 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:34.471+0000 7f82bf7fe700 1 --2- 172.21.15.179:0/305690558 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f82b80a1150 0x7f82b80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:01:34.452 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:34.471+0000 7f82bf7fe700 1 -- 172.21.15.179:0/305690558 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f82c8006e40 con 0x7f82d01017c0 2023-07-18T03:01:34.452 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:34.472+0000 7f82ce59c700 1 --2- 172.21.15.179:0/305690558 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f82b80a1150 0x7f82b80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:01:34.452 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:34.472+0000 7f82ce59c700 1 --2- 172.21.15.179:0/305690558 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f82b80a1150 0x7f82b80a3600 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f82d0100c90 tx=0x7f82c4005ca0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:01:34.453 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:34.475+0000 7f82bf7fe700 1 -- 172.21.15.179:0/305690558 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f82c8095d00 con 0x7f82d01017c0 2023-07-18T03:01:34.646 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:34.669+0000 7f82d4b19700 1 -- 172.21.15.179:0/305690558 --> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f82d0107790 con 0x7f82b80a1150 2023-07-18T03:01:35.764 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:35 smithi179 ceph-mon[129229]: osdmap e34: 5 total, 5 up, 5 in 2023-07-18T03:01:35.765 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:35 smithi179 ceph-mon[129229]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:01:35.765 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:35 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:01:35.765 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:35 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:01:35.765 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:35 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:01:35.765 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:35 smithi179 ceph-mon[129229]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:01:35.779 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:35 smithi155 ceph-mon[128885]: osdmap e34: 5 total, 5 up, 5 in 2023-07-18T03:01:35.779 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:35 smithi155 ceph-mon[128885]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:01:35.779 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:35 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:01:35.780 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:35 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:01:35.780 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:35 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:01:35.780 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:35 smithi155 ceph-mon[128885]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:01:35.972 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:35 smithi101 ceph-mon[126379]: osdmap e34: 5 total, 5 up, 5 in 2023-07-18T03:01:35.972 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:35 smithi101 ceph-mon[126379]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:01:35.972 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:35 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:01:35.972 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:35 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:01:35.972 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:35 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:01:35.973 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:35 smithi101 ceph-mon[126379]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:01:38.184 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:37 smithi179 ceph-mon[129229]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:01:38.185 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:37 smithi179 ceph-mon[129229]: from='client.? 172.21.15.179:0/3941127662' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1020bfc5-8c2f-482e-9588-66c7f98e46e3"}]: dispatch 2023-07-18T03:01:38.185 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:37 smithi179 ceph-mon[129229]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1020bfc5-8c2f-482e-9588-66c7f98e46e3"}]: dispatch 2023-07-18T03:01:38.185 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:37 smithi179 ceph-mon[129229]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1020bfc5-8c2f-482e-9588-66c7f98e46e3"}]': finished 2023-07-18T03:01:38.185 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:37 smithi179 ceph-mon[129229]: osdmap e35: 6 total, 5 up, 6 in 2023-07-18T03:01:38.185 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:37 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-18T03:01:38.185 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:37 smithi179 ceph-mon[129229]: from='client.? 172.21.15.179:0/1581813912' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:01:38.221 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:37 smithi101 ceph-mon[126379]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:01:38.222 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:37 smithi101 ceph-mon[126379]: from='client.? 172.21.15.179:0/3941127662' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1020bfc5-8c2f-482e-9588-66c7f98e46e3"}]: dispatch 2023-07-18T03:01:38.222 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:37 smithi101 ceph-mon[126379]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1020bfc5-8c2f-482e-9588-66c7f98e46e3"}]: dispatch 2023-07-18T03:01:38.222 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:37 smithi101 ceph-mon[126379]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1020bfc5-8c2f-482e-9588-66c7f98e46e3"}]': finished 2023-07-18T03:01:38.222 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:37 smithi101 ceph-mon[126379]: osdmap e35: 6 total, 5 up, 6 in 2023-07-18T03:01:38.222 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:37 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-18T03:01:38.222 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:37 smithi101 ceph-mon[126379]: from='client.? 172.21.15.179:0/1581813912' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:01:38.279 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:37 smithi155 ceph-mon[128885]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:01:38.279 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:37 smithi155 ceph-mon[128885]: from='client.? 172.21.15.179:0/3941127662' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1020bfc5-8c2f-482e-9588-66c7f98e46e3"}]: dispatch 2023-07-18T03:01:38.279 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:37 smithi155 ceph-mon[128885]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1020bfc5-8c2f-482e-9588-66c7f98e46e3"}]: dispatch 2023-07-18T03:01:38.279 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:37 smithi155 ceph-mon[128885]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1020bfc5-8c2f-482e-9588-66c7f98e46e3"}]': finished 2023-07-18T03:01:38.280 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:37 smithi155 ceph-mon[128885]: osdmap e35: 6 total, 5 up, 6 in 2023-07-18T03:01:38.280 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:37 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-18T03:01:38.280 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:37 smithi155 ceph-mon[128885]: from='client.? 172.21.15.179:0/1581813912' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:01:40.623 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:40 smithi179 ceph-mon[129229]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:01:40.721 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:40 smithi101 ceph-mon[126379]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:01:40.779 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:40 smithi155 ceph-mon[128885]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:01:42.590 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:42 smithi179 ceph-mon[129229]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:01:42.722 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:42 smithi101 ceph-mon[126379]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:01:42.779 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:42 smithi155 ceph-mon[128885]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:01:44.599 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:44 smithi179 ceph-mon[129229]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:01:44.599 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:44 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-07-18T03:01:44.599 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:44 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:01:44.722 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:44 smithi101 ceph-mon[126379]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:01:44.722 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:44 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-07-18T03:01:44.722 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:44 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:01:44.779 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:44 smithi155 ceph-mon[128885]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:01:44.779 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:44 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-07-18T03:01:44.780 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:44 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:01:45.547 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:45 smithi179 ceph-mon[129229]: Deploying daemon osd.5 on smithi179 2023-07-18T03:01:45.722 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:45 smithi101 ceph-mon[126379]: Deploying daemon osd.5 on smithi179 2023-07-18T03:01:45.779 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:45 smithi155 ceph-mon[128885]: Deploying daemon osd.5 on smithi179 2023-07-18T03:01:46.685 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:46 smithi179 ceph-mon[129229]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:01:46.722 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:46 smithi101 ceph-mon[126379]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:01:46.779 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:46 smithi155 ceph-mon[128885]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:01:47.649 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:47 smithi179 ceph-mon[129229]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:01:47.722 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:47 smithi101 ceph-mon[126379]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:01:47.779 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:47 smithi155 ceph-mon[128885]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:01:48.805 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:48 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:48.805 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:48 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:48.805 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:48 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:01:48.974 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:48 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:48.975 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:48 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:48.975 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:48 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:01:49.029 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:48 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:49.030 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:48 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:49.030 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:48 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:01:49.972 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:49 smithi101 ceph-mon[126379]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:01:49.975 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:49 smithi179 ceph-mon[129229]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:01:50.029 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:49 smithi155 ceph-mon[128885]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:01:50.363 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:50.386+0000 7f82bf7fe700 1 -- 172.21.15.179:0/305690558 <== mgr.14150 v2:172.21.15.101:6800/3963742842 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f82d0107790 con 0x7f82b80a1150 2023-07-18T03:01:50.363 INFO:teuthology.orchestra.run.smithi179.stdout:Created osd(s) 5 on host 'smithi179' 2023-07-18T03:01:50.365 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:50.388+0000 7f82d4b19700 1 -- 172.21.15.179:0/305690558 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f82b80a1150 msgr2=0x7f82b80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:01:50.365 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:50.388+0000 7f82d4b19700 1 --2- 172.21.15.179:0/305690558 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f82b80a1150 0x7f82b80a3600 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f82d0100c90 tx=0x7f82c4005ca0 comp rx=0 tx=0).stop 2023-07-18T03:01:50.366 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:50.388+0000 7f82d4b19700 1 -- 172.21.15.179:0/305690558 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82d01017c0 msgr2=0x7f82d019f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:01:50.366 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:50.388+0000 7f82d4b19700 1 --2- 172.21.15.179:0/305690558 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82d01017c0 0x7f82d019f0a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f82c800b9a0 tx=0x7f82c800bd60 comp rx=0 tx=0).stop 2023-07-18T03:01:50.366 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:50.388+0000 7f82d4b19700 1 -- 172.21.15.179:0/305690558 shutdown_connections 2023-07-18T03:01:50.366 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:50.388+0000 7f82d4b19700 1 --2- 172.21.15.179:0/305690558 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f82b80a1150 0x7f82b80a3600 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:01:50.366 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:50.388+0000 7f82d4b19700 1 --2- 172.21.15.179:0/305690558 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82d01017c0 0x7f82d019f0a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:01:50.366 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:50.388+0000 7f82d4b19700 1 --2- 172.21.15.179:0/305690558 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f82d00ffc30 0x7f82d0197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:01:50.366 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:50.388+0000 7f82d4b19700 1 --2- 172.21.15.179:0/305690558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82d0100e30 0x7f82d0198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:01:50.367 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:50.388+0000 7f82d4b19700 1 -- 172.21.15.179:0/305690558 >> 172.21.15.179:0/305690558 conn(0x7f82d00fb2a0 msgr2=0x7f82d00fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:01:50.367 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:50.388+0000 7f82d4b19700 1 -- 172.21.15.179:0/305690558 shutdown_connections 2023-07-18T03:01:50.367 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:01:50.388+0000 7f82d4b19700 1 -- 172.21.15.179:0/305690558 wait complete. 2023-07-18T03:01:50.748 DEBUG:teuthology.orchestra.run.smithi179:osd.5> sudo journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@osd.5.service 2023-07-18T03:01:50.751 INFO:tasks.cephadm:Deploying osd.6 on smithi101 with /dev/vg_nvme/lv_2... 2023-07-18T03:01:50.751 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-07-18T03:01:50.779 INFO:journalctl@ceph.osd.5.smithi179.stdout:-- Logs begin at Tue 2023-07-18 02:29:48 UTC. -- 2023-07-18T03:01:51.184 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:50 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:51.185 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:50 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:51.185 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:50 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:01:51.185 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:50 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:01:51.185 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:50 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:51.185 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:50 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:51.186 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:50 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:51.186 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:50 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:01:51.222 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:50 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:51.222 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:50 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:51.222 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:50 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:01:51.223 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:50 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:01:51.223 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:50 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:51.223 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:50 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:51.223 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:50 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:51.224 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:50 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:01:51.279 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:50 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:51.280 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:50 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:51.280 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:50 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:01:51.280 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:50 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:01:51.280 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:50 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:51.280 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:50 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:51.281 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:50 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:51.281 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:50 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:01:51.498 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:01:51.824 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jul 18 03:01:51 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-5[140498]: 2023-07-18T03:01:51.721+0000 7f635ff85640 -1 osd.5 0 log_to_monitors true 2023-07-18T03:01:52.472 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:52 smithi101 ceph-mon[126379]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:01:52.472 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:52 smithi101 ceph-mon[126379]: from='osd.5 [v2:172.21.15.179:6808/2746037021,v1:172.21.15.179:6809/2746037021]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-18T03:01:52.472 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:52 smithi101 ceph-mon[126379]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-18T03:01:52.685 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:52 smithi179 ceph-mon[129229]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:01:52.685 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:52 smithi179 ceph-mon[129229]: from='osd.5 [v2:172.21.15.179:6808/2746037021,v1:172.21.15.179:6809/2746037021]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-18T03:01:52.685 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:52 smithi179 ceph-mon[129229]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-18T03:01:52.779 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:52 smithi155 ceph-mon[128885]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:01:52.780 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:52 smithi155 ceph-mon[128885]: from='osd.5 [v2:172.21.15.179:6808/2746037021,v1:172.21.15.179:6809/2746037021]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-18T03:01:52.780 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:52 smithi155 ceph-mon[128885]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-18T03:01:53.614 INFO:teuthology.orchestra.run.smithi101.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-07-18T03:01:53.614 INFO:teuthology.orchestra.run.smithi101.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-07-18T03:01:53.614 INFO:teuthology.orchestra.run.smithi101.stderr: stderr: 10+0 records in 2023-07-18T03:01:53.614 INFO:teuthology.orchestra.run.smithi101.stderr:10+0 records out 2023-07-18T03:01:53.614 INFO:teuthology.orchestra.run.smithi101.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0393705 s, 266 MB/s 2023-07-18T03:01:53.615 INFO:teuthology.orchestra.run.smithi101.stderr:--> Zapping successful for: 2023-07-18T03:01:53.656 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:53 smithi101 ceph-mon[126379]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-07-18T03:01:53.656 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:53 smithi101 ceph-mon[126379]: osdmap e36: 6 total, 5 up, 6 in 2023-07-18T03:01:53.657 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:53 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-18T03:01:53.657 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:53 smithi101 ceph-mon[126379]: from='osd.5 [v2:172.21.15.179:6808/2746037021,v1:172.21.15.179:6809/2746037021]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2023-07-18T03:01:53.657 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:53 smithi101 ceph-mon[126379]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2023-07-18T03:01:53.657 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:53 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:53.657 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:53 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:53.657 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:53 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:01:53.658 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:53 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:01:53.658 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:53 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:53.658 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:53 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:01:53.659 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:53 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:01:53.659 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:53 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:53.685 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jul 18 03:01:53 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-5[140498]: 2023-07-18T03:01:53.382+0000 7f63555fd700 -1 osd.5 0 waiting for initial osdmap 2023-07-18T03:01:53.685 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jul 18 03:01:53 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-5[140498]: 2023-07-18T03:01:53.388+0000 7f63513f2700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-18T03:01:53.685 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:53 smithi179 ceph-mon[129229]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-07-18T03:01:53.685 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:53 smithi179 ceph-mon[129229]: osdmap e36: 6 total, 5 up, 6 in 2023-07-18T03:01:53.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:53 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-18T03:01:53.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:53 smithi179 ceph-mon[129229]: from='osd.5 [v2:172.21.15.179:6808/2746037021,v1:172.21.15.179:6809/2746037021]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2023-07-18T03:01:53.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:53 smithi179 ceph-mon[129229]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2023-07-18T03:01:53.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:53 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:53.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:53 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:53.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:53 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:01:53.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:53 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:01:53.687 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:53 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:53.687 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:53 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:01:53.687 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:53 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:01:53.687 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:53 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:53.780 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:53 smithi155 ceph-mon[128885]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-07-18T03:01:53.780 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:53 smithi155 ceph-mon[128885]: osdmap e36: 6 total, 5 up, 6 in 2023-07-18T03:01:53.780 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:53 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-18T03:01:53.780 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:53 smithi155 ceph-mon[128885]: from='osd.5 [v2:172.21.15.179:6808/2746037021,v1:172.21.15.179:6809/2746037021]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2023-07-18T03:01:53.780 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:53 smithi155 ceph-mon[128885]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2023-07-18T03:01:53.781 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:53 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:53.781 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:53 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:53.781 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:53 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:01:53.781 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:53 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:01:53.781 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:53 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:53.781 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:53 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:01:53.781 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:53 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:01:53.781 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:53 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:01:54.178 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi101:vg_nvme/lv_2 2023-07-18T03:01:54.472 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:54 smithi101 ceph-mon[126379]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:01:54.473 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:54 smithi101 ceph-mon[126379]: Detected new or changed devices on smithi179 2023-07-18T03:01:54.473 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:54 smithi101 ceph-mon[126379]: Adjusting osd_memory_target on smithi179 to 8584M 2023-07-18T03:01:54.473 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:54 smithi101 ceph-mon[126379]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2023-07-18T03:01:54.473 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:54 smithi101 ceph-mon[126379]: osdmap e37: 6 total, 5 up, 6 in 2023-07-18T03:01:54.473 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:54 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-18T03:01:54.473 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:54 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-18T03:01:54.685 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:54 smithi179 ceph-mon[129229]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:01:54.685 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:54 smithi179 ceph-mon[129229]: Detected new or changed devices on smithi179 2023-07-18T03:01:54.685 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:54 smithi179 ceph-mon[129229]: Adjusting osd_memory_target on smithi179 to 8584M 2023-07-18T03:01:54.685 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:54 smithi179 ceph-mon[129229]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2023-07-18T03:01:54.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:54 smithi179 ceph-mon[129229]: osdmap e37: 6 total, 5 up, 6 in 2023-07-18T03:01:54.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:54 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-18T03:01:54.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:54 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-18T03:01:54.780 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:54 smithi155 ceph-mon[128885]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:01:54.780 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:54 smithi155 ceph-mon[128885]: Detected new or changed devices on smithi179 2023-07-18T03:01:54.780 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:54 smithi155 ceph-mon[128885]: Adjusting osd_memory_target on smithi179 to 8584M 2023-07-18T03:01:54.780 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:54 smithi155 ceph-mon[128885]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2023-07-18T03:01:54.780 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:54 smithi155 ceph-mon[128885]: osdmap e37: 6 total, 5 up, 6 in 2023-07-18T03:01:54.781 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:54 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-18T03:01:54.781 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:54 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-18T03:01:54.904 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:01:55.472 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:55 smithi101 ceph-mon[126379]: purged_snaps scrub starts 2023-07-18T03:01:55.473 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:55 smithi101 ceph-mon[126379]: purged_snaps scrub ok 2023-07-18T03:01:55.473 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:55 smithi101 ceph-mon[126379]: osd.5 [v2:172.21.15.179:6808/2746037021,v1:172.21.15.179:6809/2746037021] boot 2023-07-18T03:01:55.473 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:55 smithi101 ceph-mon[126379]: osdmap e38: 6 total, 6 up, 6 in 2023-07-18T03:01:55.473 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:55 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-18T03:01:55.474 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:55 smithi101 ceph-mon[126379]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:01:55.474 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:55 smithi101 ceph-mon[126379]: osdmap e39: 6 total, 6 up, 6 in 2023-07-18T03:01:55.685 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:55 smithi179 ceph-mon[129229]: purged_snaps scrub starts 2023-07-18T03:01:55.685 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:55 smithi179 ceph-mon[129229]: purged_snaps scrub ok 2023-07-18T03:01:55.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:55 smithi179 ceph-mon[129229]: osd.5 [v2:172.21.15.179:6808/2746037021,v1:172.21.15.179:6809/2746037021] boot 2023-07-18T03:01:55.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:55 smithi179 ceph-mon[129229]: osdmap e38: 6 total, 6 up, 6 in 2023-07-18T03:01:55.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:55 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-18T03:01:55.687 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:55 smithi179 ceph-mon[129229]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:01:55.687 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:55 smithi179 ceph-mon[129229]: osdmap e39: 6 total, 6 up, 6 in 2023-07-18T03:01:55.780 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:55 smithi155 ceph-mon[128885]: purged_snaps scrub starts 2023-07-18T03:01:55.780 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:55 smithi155 ceph-mon[128885]: purged_snaps scrub ok 2023-07-18T03:01:55.780 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:55 smithi155 ceph-mon[128885]: osd.5 [v2:172.21.15.179:6808/2746037021,v1:172.21.15.179:6809/2746037021] boot 2023-07-18T03:01:55.781 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:55 smithi155 ceph-mon[128885]: osdmap e38: 6 total, 6 up, 6 in 2023-07-18T03:01:55.781 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:55 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-18T03:01:55.781 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:55 smithi155 ceph-mon[128885]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:01:55.781 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:55 smithi155 ceph-mon[128885]: osdmap e39: 6 total, 6 up, 6 in 2023-07-18T03:01:56.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:56.627+0000 7f8c83da4700 1 -- 172.21.15.101:0/1772901844 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c7c101720 msgr2=0x7f8c7c107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:01:56.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:56.627+0000 7f8c83da4700 1 --2- 172.21.15.101:0/1772901844 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c7c101720 0x7f8c7c107fe0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8c7400b0a0 tx=0x7f8c7400b3b0 comp rx=0 tx=0).stop 2023-07-18T03:01:56.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:56.627+0000 7f8c83da4700 1 -- 172.21.15.101:0/1772901844 shutdown_connections 2023-07-18T03:01:56.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:56.627+0000 7f8c83da4700 1 --2- 172.21.15.101:0/1772901844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c7c0ffb90 0x7f8c7c0fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:01:56.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:56.627+0000 7f8c83da4700 1 --2- 172.21.15.101:0/1772901844 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c7c101720 0x7f8c7c107fe0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:01:56.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:56.627+0000 7f8c83da4700 1 --2- 172.21.15.101:0/1772901844 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c7c100d90 0x7f8c7c1011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:01:56.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:56.627+0000 7f8c83da4700 1 -- 172.21.15.101:0/1772901844 >> 172.21.15.101:0/1772901844 conn(0x7f8c7c0fb1a0 msgr2=0x7f8c7c0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:01:56.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:56.627+0000 7f8c83da4700 1 -- 172.21.15.101:0/1772901844 shutdown_connections 2023-07-18T03:01:56.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:56.628+0000 7f8c83da4700 1 -- 172.21.15.101:0/1772901844 wait complete. 2023-07-18T03:01:56.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:56.628+0000 7f8c83da4700 1 Processor -- start 2023-07-18T03:01:56.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:56.629+0000 7f8c83da4700 1 -- start start 2023-07-18T03:01:56.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:56.629+0000 7f8c83da4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c7c0ffb90 0x7f8c7c197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:01:56.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:56.629+0000 7f8c83da4700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c7c100d90 0x7f8c7c198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:01:56.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:56.629+0000 7f8c83da4700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c7c101720 0x7f8c7c19f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:01:56.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:56.629+0000 7f8c83da4700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7c19f750 con 0x7f8c7c101720 2023-07-18T03:01:56.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:56.629+0000 7f8c83da4700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7c19f890 con 0x7f8c7c100d90 2023-07-18T03:01:56.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:56.629+0000 7f8c83da4700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7c19fa20 con 0x7f8c7c0ffb90 2023-07-18T03:01:56.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:56.629+0000 7f8c8133f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c7c100d90 0x7f8c7c198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:01:56.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:56.629+0000 7f8c82341700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c7c101720 0x7f8c7c19f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:01:56.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:56.629+0000 7f8c8133f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c7c100d90 0x7f8c7c198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:45274/0 (socket says 172.21.15.101:45274) 2023-07-18T03:01:56.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:56.629+0000 7f8c8133f700 1 -- 172.21.15.101:0/1935849002 learned_addr learned my addr 172.21.15.101:0/1935849002 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:01:56.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:56.629+0000 7f8c81b40700 1 --2- 172.21.15.101:0/1935849002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c7c0ffb90 0x7f8c7c197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:01:56.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:56.630+0000 7f8c8133f700 1 -- 172.21.15.101:0/1935849002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c7c0ffb90 msgr2=0x7f8c7c197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:01:56.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:56.630+0000 7f8c8133f700 1 --2- 172.21.15.101:0/1935849002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c7c0ffb90 0x7f8c7c197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:01:56.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:56.630+0000 7f8c8133f700 1 -- 172.21.15.101:0/1935849002 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c7c101720 msgr2=0x7f8c7c19f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:01:56.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:56.630+0000 7f8c8133f700 1 --2- 172.21.15.101:0/1935849002 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c7c101720 0x7f8c7c19f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:01:56.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:56.630+0000 7f8c8133f700 1 -- 172.21.15.101:0/1935849002 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c74009d00 con 0x7f8c7c100d90 2023-07-18T03:01:56.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:56.630+0000 7f8c82341700 1 --2- 172.21.15.101:0/1935849002 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c7c101720 0x7f8c7c19f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:01:56.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:56.630+0000 7f8c8133f700 1 --2- 172.21.15.101:0/1935849002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c7c100d90 0x7f8c7c198100 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c00d950 tx=0x7f8c6c00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:01:56.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:56.630+0000 7f8c72ffd700 1 -- 172.21.15.101:0/1935849002 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c00f890 con 0x7f8c7c100d90 2023-07-18T03:01:56.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:56.630+0000 7f8c83da4700 1 -- 172.21.15.101:0/1935849002 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c7c19fd00 con 0x7f8c7c100d90 2023-07-18T03:01:56.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:56.630+0000 7f8c72ffd700 1 -- 172.21.15.101:0/1935849002 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c6c010460 con 0x7f8c7c100d90 2023-07-18T03:01:56.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:56.630+0000 7f8c72ffd700 1 -- 172.21.15.101:0/1935849002 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c00e610 con 0x7f8c7c100d90 2023-07-18T03:01:56.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:56.630+0000 7f8c83da4700 1 -- 172.21.15.101:0/1935849002 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c7c1a01f0 con 0x7f8c7c100d90 2023-07-18T03:01:56.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:56.631+0000 7f8c83da4700 1 -- 172.21.15.101:0/1935849002 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c7c10b3b0 con 0x7f8c7c100d90 2023-07-18T03:01:56.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:56.633+0000 7f8c72ffd700 1 -- 172.21.15.101:0/1935849002 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f8c6c0105d0 con 0x7f8c7c100d90 2023-07-18T03:01:56.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:56.635+0000 7f8c72ffd700 1 --2- 172.21.15.101:0/1935849002 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8c680a1150 0x7f8c680a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:01:56.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:56.635+0000 7f8c72ffd700 1 -- 172.21.15.101:0/1935849002 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4562+0+0 (secure 0 0 0) 0x7f8c6c0cb830 con 0x7f8c7c100d90 2023-07-18T03:01:56.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:56.635+0000 7f8c81b40700 1 --2- 172.21.15.101:0/1935849002 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8c680a1150 0x7f8c680a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:01:56.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:56.636+0000 7f8c81b40700 1 --2- 172.21.15.101:0/1935849002 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8c680a1150 0x7f8c680a3600 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c100bf0 tx=0x7f8c7800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:01:56.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:56.637+0000 7f8c72ffd700 1 -- 172.21.15.101:0/1935849002 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8c6c095d80 con 0x7f8c7c100d90 2023-07-18T03:01:56.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:01:56.839+0000 7f8c83da4700 1 -- 172.21.15.101:0/1935849002 --> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c7c107d30 con 0x7f8c680a1150 2023-07-18T03:01:57.685 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:57 smithi179 ceph-mon[129229]: osdmap e40: 6 total, 6 up, 6 in 2023-07-18T03:01:57.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:57 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:01:57.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:57 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:01:57.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:57 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:01:57.722 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:57 smithi101 ceph-mon[126379]: osdmap e40: 6 total, 6 up, 6 in 2023-07-18T03:01:57.723 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:57 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:01:57.723 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:57 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:01:57.723 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:57 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:01:57.780 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:57 smithi155 ceph-mon[128885]: osdmap e40: 6 total, 6 up, 6 in 2023-07-18T03:01:57.780 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:57 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:01:57.780 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:57 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:01:57.781 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:57 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:01:58.472 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:58 smithi101 ceph-mon[126379]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-18T03:01:58.473 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:58 smithi101 ceph-mon[126379]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:01:58.473 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:58 smithi101 ceph-mon[126379]: osdmap e41: 6 total, 6 up, 6 in 2023-07-18T03:01:58.685 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:58 smithi179 ceph-mon[129229]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-18T03:01:58.685 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:58 smithi179 ceph-mon[129229]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:01:58.685 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:58 smithi179 ceph-mon[129229]: osdmap e41: 6 total, 6 up, 6 in 2023-07-18T03:01:58.780 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:58 smithi155 ceph-mon[128885]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-18T03:01:58.780 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:58 smithi155 ceph-mon[128885]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:01:58.780 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:58 smithi155 ceph-mon[128885]: osdmap e41: 6 total, 6 up, 6 in 2023-07-18T03:01:59.685 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:59 smithi179 ceph-mon[129229]: osdmap e42: 6 total, 6 up, 6 in 2023-07-18T03:01:59.685 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:01:59 smithi179 ceph-mon[129229]: pgmap v140: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-18T03:01:59.723 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:59 smithi101 ceph-mon[126379]: osdmap e42: 6 total, 6 up, 6 in 2023-07-18T03:01:59.723 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:01:59 smithi101 ceph-mon[126379]: pgmap v140: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-18T03:01:59.780 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:59 smithi155 ceph-mon[128885]: osdmap e42: 6 total, 6 up, 6 in 2023-07-18T03:01:59.780 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:01:59 smithi155 ceph-mon[128885]: pgmap v140: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-18T03:02:00.685 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:00 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/4102816302' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2cd99260-5c36-4603-9338-cf697970ce58"}]: dispatch 2023-07-18T03:02:00.685 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:00 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/4102816302' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2cd99260-5c36-4603-9338-cf697970ce58"}]': finished 2023-07-18T03:02:00.685 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:00 smithi179 ceph-mon[129229]: osdmap e43: 7 total, 6 up, 7 in 2023-07-18T03:02:00.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:00 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-18T03:02:00.722 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:00 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/4102816302' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2cd99260-5c36-4603-9338-cf697970ce58"}]: dispatch 2023-07-18T03:02:00.723 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:00 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/4102816302' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2cd99260-5c36-4603-9338-cf697970ce58"}]': finished 2023-07-18T03:02:00.723 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:00 smithi101 ceph-mon[126379]: osdmap e43: 7 total, 6 up, 7 in 2023-07-18T03:02:00.723 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:00 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-18T03:02:00.780 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:00 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/4102816302' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2cd99260-5c36-4603-9338-cf697970ce58"}]: dispatch 2023-07-18T03:02:00.780 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:00 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/4102816302' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2cd99260-5c36-4603-9338-cf697970ce58"}]': finished 2023-07-18T03:02:00.780 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:00 smithi155 ceph-mon[128885]: osdmap e43: 7 total, 6 up, 7 in 2023-07-18T03:02:00.780 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:00 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-18T03:02:01.685 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:01 smithi179 ceph-mon[129229]: pgmap v142: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 130 KiB/s, 0 objects/s recovering 2023-07-18T03:02:01.685 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:01 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/3870259806' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:02:01.722 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:01 smithi101 ceph-mon[126379]: pgmap v142: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 130 KiB/s, 0 objects/s recovering 2023-07-18T03:02:01.723 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:01 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3870259806' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:02:01.780 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:01 smithi155 ceph-mon[128885]: pgmap v142: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 130 KiB/s, 0 objects/s recovering 2023-07-18T03:02:01.780 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:01 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/3870259806' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:02:04.685 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:04 smithi179 ceph-mon[129229]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-18T03:02:04.723 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:04 smithi101 ceph-mon[126379]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-18T03:02:04.780 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:04 smithi155 ceph-mon[128885]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-18T03:02:06.473 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:06 smithi101 ceph-mon[126379]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-18T03:02:06.685 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:06 smithi179 ceph-mon[129229]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-18T03:02:06.780 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:06 smithi155 ceph-mon[128885]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-18T03:02:08.685 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:08 smithi179 ceph-mon[129229]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-18T03:02:08.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:08 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-07-18T03:02:08.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:08 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:02:08.718 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:08 smithi101 ceph-mon[126379]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-18T03:02:08.718 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:08 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-07-18T03:02:08.718 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:08 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:02:08.780 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:08 smithi155 ceph-mon[128885]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-18T03:02:08.780 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:08 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-07-18T03:02:08.780 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:08 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:02:09.649 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:09 smithi101 ceph-mon[126379]: Deploying daemon osd.6 on smithi101 2023-07-18T03:02:09.649 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:09 smithi101 ceph-mon[126379]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-18T03:02:09.685 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:09 smithi179 ceph-mon[129229]: Deploying daemon osd.6 on smithi101 2023-07-18T03:02:09.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:09 smithi179 ceph-mon[129229]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-18T03:02:09.780 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:09 smithi155 ceph-mon[128885]: Deploying daemon osd.6 on smithi101 2023-07-18T03:02:09.781 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:09 smithi155 ceph-mon[128885]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-18T03:02:12.143 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:11 smithi101 ceph-mon[126379]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-18T03:02:12.185 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:11 smithi179 ceph-mon[129229]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-18T03:02:12.280 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:11 smithi155 ceph-mon[128885]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-18T03:02:14.473 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:14 smithi101 ceph-mon[126379]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-18T03:02:14.473 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:14 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:14.473 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:14 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:02:14.474 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:14 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:14.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:14 smithi179 ceph-mon[129229]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-18T03:02:14.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:14 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:14.687 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:14 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:02:14.687 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:14 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:14.780 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:14 smithi155 ceph-mon[128885]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-18T03:02:14.781 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:14 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:14.781 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:14 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:02:14.781 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:14 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:16.685 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:16 smithi179 ceph-mon[129229]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-18T03:02:16.722 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:16 smithi101 ceph-mon[126379]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-18T03:02:16.780 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:16 smithi155 ceph-mon[128885]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-18T03:02:18.033 INFO:teuthology.orchestra.run.smithi101.stdout:Created osd(s) 6 on host 'smithi101' 2023-07-18T03:02:18.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:02:18.053+0000 7f8c72ffd700 1 -- 172.21.15.101:0/1935849002 <== mgr.14150 v2:172.21.15.101:6800/3963742842 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f8c7c107d30 con 0x7f8c680a1150 2023-07-18T03:02:18.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:02:18.056+0000 7f8c83da4700 1 -- 172.21.15.101:0/1935849002 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8c680a1150 msgr2=0x7f8c680a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:02:18.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:02:18.056+0000 7f8c83da4700 1 --2- 172.21.15.101:0/1935849002 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8c680a1150 0x7f8c680a3600 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c100bf0 tx=0x7f8c7800b410 comp rx=0 tx=0).stop 2023-07-18T03:02:18.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:02:18.056+0000 7f8c83da4700 1 -- 172.21.15.101:0/1935849002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c7c100d90 msgr2=0x7f8c7c198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:02:18.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:02:18.056+0000 7f8c83da4700 1 --2- 172.21.15.101:0/1935849002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c7c100d90 0x7f8c7c198100 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c00d950 tx=0x7f8c6c00dc60 comp rx=0 tx=0).stop 2023-07-18T03:02:18.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:02:18.056+0000 7f8c83da4700 1 -- 172.21.15.101:0/1935849002 shutdown_connections 2023-07-18T03:02:18.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:02:18.056+0000 7f8c83da4700 1 --2- 172.21.15.101:0/1935849002 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8c680a1150 0x7f8c680a3600 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:02:18.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:02:18.056+0000 7f8c83da4700 1 --2- 172.21.15.101:0/1935849002 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c7c101720 0x7f8c7c19f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:02:18.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:02:18.056+0000 7f8c83da4700 1 --2- 172.21.15.101:0/1935849002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c7c0ffb90 0x7f8c7c197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:02:18.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:02:18.056+0000 7f8c83da4700 1 --2- 172.21.15.101:0/1935849002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c7c100d90 0x7f8c7c198100 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:02:18.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:02:18.056+0000 7f8c83da4700 1 -- 172.21.15.101:0/1935849002 >> 172.21.15.101:0/1935849002 conn(0x7f8c7c0fb1a0 msgr2=0x7f8c7c106610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:02:18.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:02:18.056+0000 7f8c83da4700 1 -- 172.21.15.101:0/1935849002 shutdown_connections 2023-07-18T03:02:18.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:02:18.056+0000 7f8c83da4700 1 -- 172.21.15.101:0/1935849002 wait complete. 2023-07-18T03:02:18.513 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:18 smithi101 ceph-mon[126379]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-18T03:02:18.514 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:18 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:18.514 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:18 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:18.514 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:18 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:02:18.514 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:18 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:02:18.514 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:18 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:18.515 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:18 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:18.515 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:18 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:18.515 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:18 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:02:18.515 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:18 smithi101 ceph-mon[126379]: from='osd.6 [v2:172.21.15.101:6818/958044931,v1:172.21.15.101:6819/958044931]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-18T03:02:18.518 DEBUG:teuthology.orchestra.run.smithi101:osd.6> sudo journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@osd.6.service 2023-07-18T03:02:18.522 INFO:tasks.cephadm:Deploying osd.7 on smithi155 with /dev/vg_nvme/lv_2... 2023-07-18T03:02:18.523 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-07-18T03:02:18.596 INFO:journalctl@ceph.osd.6.smithi101.stdout:-- Logs begin at Tue 2023-07-18 02:35:16 UTC. -- 2023-07-18T03:02:18.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:18 smithi179 ceph-mon[129229]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-18T03:02:18.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:18 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:18.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:18 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:18.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:18 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:02:18.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:18 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:02:18.687 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:18 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:18.687 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:18 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:18.687 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:18 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:18.687 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:18 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:02:18.687 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:18 smithi179 ceph-mon[129229]: from='osd.6 [v2:172.21.15.101:6818/958044931,v1:172.21.15.101:6819/958044931]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-18T03:02:18.780 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:18 smithi155 ceph-mon[128885]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-18T03:02:18.781 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:18 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:18.781 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:18 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:18.781 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:18 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:02:18.781 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:18 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:02:18.781 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:18 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:18.782 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:18 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:18.782 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:18 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:18.782 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:18 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:02:18.782 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:18 smithi155 ceph-mon[128885]: from='osd.6 [v2:172.21.15.101:6818/958044931,v1:172.21.15.101:6819/958044931]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-18T03:02:19.083 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.b/config 2023-07-18T03:02:20.040 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:20 smithi101 ceph-mon[126379]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-18T03:02:20.040 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:20 smithi101 ceph-mon[126379]: from='osd.6 [v2:172.21.15.101:6818/958044931,v1:172.21.15.101:6819/958044931]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-07-18T03:02:20.040 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:20 smithi101 ceph-mon[126379]: osdmap e44: 7 total, 6 up, 7 in 2023-07-18T03:02:20.040 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:20 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-18T03:02:20.041 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:20 smithi101 ceph-mon[126379]: from='osd.6 [v2:172.21.15.101:6818/958044931,v1:172.21.15.101:6819/958044931]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-07-18T03:02:20.041 INFO:journalctl@ceph.osd.6.smithi101.stdout:Jul 18 03:02:20 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-6[150288]: 2023-07-18T03:02:20.049+0000 7f47cf573700 -1 osd.6 0 waiting for initial osdmap 2023-07-18T03:02:20.041 INFO:journalctl@ceph.osd.6.smithi101.stdout:Jul 18 03:02:20 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-6[150288]: 2023-07-18T03:02:20.060+0000 7f47c9364700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-18T03:02:20.280 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:20 smithi155 ceph-mon[128885]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-18T03:02:20.281 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:20 smithi155 ceph-mon[128885]: from='osd.6 [v2:172.21.15.101:6818/958044931,v1:172.21.15.101:6819/958044931]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-07-18T03:02:20.281 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:20 smithi155 ceph-mon[128885]: osdmap e44: 7 total, 6 up, 7 in 2023-07-18T03:02:20.281 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:20 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-18T03:02:20.281 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:20 smithi155 ceph-mon[128885]: from='osd.6 [v2:172.21.15.101:6818/958044931,v1:172.21.15.101:6819/958044931]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-07-18T03:02:20.436 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:20 smithi179 ceph-mon[129229]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-18T03:02:20.436 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:20 smithi179 ceph-mon[129229]: from='osd.6 [v2:172.21.15.101:6818/958044931,v1:172.21.15.101:6819/958044931]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-07-18T03:02:20.436 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:20 smithi179 ceph-mon[129229]: osdmap e44: 7 total, 6 up, 7 in 2023-07-18T03:02:20.437 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:20 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-18T03:02:20.437 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:20 smithi179 ceph-mon[129229]: from='osd.6 [v2:172.21.15.101:6818/958044931,v1:172.21.15.101:6819/958044931]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-07-18T03:02:20.540 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-07-18T03:02:20.540 INFO:teuthology.orchestra.run.smithi155.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-07-18T03:02:20.540 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10+0 records in 2023-07-18T03:02:20.540 INFO:teuthology.orchestra.run.smithi155.stderr:10+0 records out 2023-07-18T03:02:20.540 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125651 s, 835 MB/s 2023-07-18T03:02:20.541 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping successful for: 2023-07-18T03:02:20.940 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi155:vg_nvme/lv_2 2023-07-18T03:02:21.223 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:21 smithi101 ceph-mon[126379]: from='osd.6 [v2:172.21.15.101:6818/958044931,v1:172.21.15.101:6819/958044931]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]': finished 2023-07-18T03:02:21.224 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:21 smithi101 ceph-mon[126379]: osdmap e45: 7 total, 6 up, 7 in 2023-07-18T03:02:21.224 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:21 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-18T03:02:21.224 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:21 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-18T03:02:21.280 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:21 smithi155 ceph-mon[128885]: from='osd.6 [v2:172.21.15.101:6818/958044931,v1:172.21.15.101:6819/958044931]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]': finished 2023-07-18T03:02:21.281 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:21 smithi155 ceph-mon[128885]: osdmap e45: 7 total, 6 up, 7 in 2023-07-18T03:02:21.281 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:21 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-18T03:02:21.281 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:21 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-18T03:02:21.436 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:21 smithi179 ceph-mon[129229]: from='osd.6 [v2:172.21.15.101:6818/958044931,v1:172.21.15.101:6819/958044931]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]': finished 2023-07-18T03:02:21.436 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:21 smithi179 ceph-mon[129229]: osdmap e45: 7 total, 6 up, 7 in 2023-07-18T03:02:21.436 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:21 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-18T03:02:21.436 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:21 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-18T03:02:21.496 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.b/config 2023-07-18T03:02:22.281 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:22 smithi155 ceph-mon[128885]: purged_snaps scrub starts 2023-07-18T03:02:22.281 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:22 smithi155 ceph-mon[128885]: purged_snaps scrub ok 2023-07-18T03:02:22.281 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:22 smithi155 ceph-mon[128885]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-18T03:02:22.281 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:22 smithi155 ceph-mon[128885]: osd.6 [v2:172.21.15.101:6818/958044931,v1:172.21.15.101:6819/958044931] boot 2023-07-18T03:02:22.282 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:22 smithi155 ceph-mon[128885]: osdmap e46: 7 total, 7 up, 7 in 2023-07-18T03:02:22.282 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:22 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-18T03:02:22.282 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:22 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:22.282 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:22 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:22.282 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:22 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:02:22.282 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:22 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:02:22.282 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:22 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:02:22.282 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:22 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:22.283 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:22 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:02:22.283 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:22 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:02:22.283 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:22 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:22.436 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:22 smithi179 ceph-mon[129229]: purged_snaps scrub starts 2023-07-18T03:02:22.436 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:22 smithi179 ceph-mon[129229]: purged_snaps scrub ok 2023-07-18T03:02:22.436 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:22 smithi179 ceph-mon[129229]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-18T03:02:22.437 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:22 smithi179 ceph-mon[129229]: osd.6 [v2:172.21.15.101:6818/958044931,v1:172.21.15.101:6819/958044931] boot 2023-07-18T03:02:22.437 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:22 smithi179 ceph-mon[129229]: osdmap e46: 7 total, 7 up, 7 in 2023-07-18T03:02:22.437 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:22 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-18T03:02:22.437 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:22 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:22.437 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:22 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:22.437 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:22 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:02:22.437 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:22 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:02:22.437 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:22 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:02:22.438 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:22 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:22.438 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:22 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:02:22.438 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:22 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:02:22.438 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:22 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:22.473 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:22 smithi101 ceph-mon[126379]: purged_snaps scrub starts 2023-07-18T03:02:22.474 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:22 smithi101 ceph-mon[126379]: purged_snaps scrub ok 2023-07-18T03:02:22.474 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:22 smithi101 ceph-mon[126379]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-18T03:02:22.474 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:22 smithi101 ceph-mon[126379]: osd.6 [v2:172.21.15.101:6818/958044931,v1:172.21.15.101:6819/958044931] boot 2023-07-18T03:02:22.474 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:22 smithi101 ceph-mon[126379]: osdmap e46: 7 total, 7 up, 7 in 2023-07-18T03:02:22.475 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:22 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-18T03:02:22.475 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:22 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:22.475 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:22 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:22.475 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:22 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:02:22.476 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:22 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:02:22.476 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:22 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:02:22.476 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:22 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:22.477 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:22 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:02:22.477 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:22 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:02:22.477 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:22 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:22.687 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:22.707+0000 7f5f879ec700 1 -- 172.21.15.155:0/3463256518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f800ffc80 msgr2=0x7f5f801000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:02:22.687 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:22.707+0000 7f5f879ec700 1 --2- 172.21.15.155:0/3463256518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f800ffc80 0x7f5f801000d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c009990 tx=0x7f5f7c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:02:22.688 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:22.707+0000 7f5f879ec700 1 -- 172.21.15.155:0/3463256518 shutdown_connections 2023-07-18T03:02:22.688 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:22.707+0000 7f5f879ec700 1 --2- 172.21.15.155:0/3463256518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f80100610 0x7f5f80106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:02:22.688 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:22.707+0000 7f5f879ec700 1 --2- 172.21.15.155:0/3463256518 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5f800fea80 0x7f5f800fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:02:22.688 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:22.707+0000 7f5f879ec700 1 --2- 172.21.15.155:0/3463256518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f800ffc80 0x7f5f801000d0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:02:22.688 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:22.707+0000 7f5f879ec700 1 -- 172.21.15.155:0/3463256518 >> 172.21.15.155:0/3463256518 conn(0x7f5f800fa0b0 msgr2=0x7f5f800fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:02:22.688 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:22.707+0000 7f5f879ec700 1 -- 172.21.15.155:0/3463256518 shutdown_connections 2023-07-18T03:02:22.688 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:22.707+0000 7f5f879ec700 1 -- 172.21.15.155:0/3463256518 wait complete. 2023-07-18T03:02:22.689 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:22.708+0000 7f5f879ec700 1 Processor -- start 2023-07-18T03:02:22.689 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:22.708+0000 7f5f879ec700 1 -- start start 2023-07-18T03:02:22.690 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:22.708+0000 7f5f879ec700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f800fea80 0x7f5f80194860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:02:22.690 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:22.708+0000 7f5f879ec700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f800ffc80 0x7f5f80194da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:02:22.691 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:22.708+0000 7f5f879ec700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5f80100610 0x7f5f8019be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:02:22.691 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:22.708+0000 7f5f879ec700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8019c3f0 con 0x7f5f80100610 2023-07-18T03:02:22.691 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:22.708+0000 7f5f879ec700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8019c530 con 0x7f5f800ffc80 2023-07-18T03:02:22.691 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:22.708+0000 7f5f879ec700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8019c6c0 con 0x7f5f800fea80 2023-07-18T03:02:22.691 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:22.708+0000 7f5f84f87700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f800ffc80 0x7f5f80194da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:02:22.691 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:22.708+0000 7f5f84f87700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f800ffc80 0x7f5f80194da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:49778/0 (socket says 172.21.15.155:49778) 2023-07-18T03:02:22.691 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:22.708+0000 7f5f84f87700 1 -- 172.21.15.155:0/887940586 learned_addr learned my addr 172.21.15.155:0/887940586 (peer_addr_for_me v2:172.21.15.155:0/0) 2023-07-18T03:02:22.692 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:22.708+0000 7f5f85788700 1 --2- 172.21.15.155:0/887940586 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f800fea80 0x7f5f80194860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:02:22.692 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:22.708+0000 7f5f84f87700 1 -- 172.21.15.155:0/887940586 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f800fea80 msgr2=0x7f5f80194860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:02:22.692 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:22.708+0000 7f5f84f87700 1 --2- 172.21.15.155:0/887940586 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f800fea80 0x7f5f80194860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:02:22.692 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:22.708+0000 7f5f84f87700 1 -- 172.21.15.155:0/887940586 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5f80100610 msgr2=0x7f5f8019be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:02:22.692 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:22.708+0000 7f5f84f87700 1 --2- 172.21.15.155:0/887940586 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5f80100610 0x7f5f8019be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:02:22.692 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:22.708+0000 7f5f84f87700 1 -- 172.21.15.155:0/887940586 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f7c0095f0 con 0x7f5f800ffc80 2023-07-18T03:02:22.692 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:22.708+0000 7f5f84f87700 1 --2- 172.21.15.155:0/887940586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f800ffc80 0x7f5f80194da0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c000c00 tx=0x7f5f7c00f780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:02:22.693 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:22.709+0000 7f5f767fc700 1 -- 172.21.15.155:0/887940586 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c01d070 con 0x7f5f800ffc80 2023-07-18T03:02:22.693 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:22.709+0000 7f5f767fc700 1 -- 172.21.15.155:0/887940586 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f7c00fc30 con 0x7f5f800ffc80 2023-07-18T03:02:22.693 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:22.709+0000 7f5f879ec700 1 -- 172.21.15.155:0/887940586 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f8019c940 con 0x7f5f800ffc80 2023-07-18T03:02:22.693 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:22.709+0000 7f5f767fc700 1 -- 172.21.15.155:0/887940586 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c0176e0 con 0x7f5f800ffc80 2023-07-18T03:02:22.693 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:22.709+0000 7f5f879ec700 1 -- 172.21.15.155:0/887940586 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f8019ce60 con 0x7f5f800ffc80 2023-07-18T03:02:22.693 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:22.710+0000 7f5f879ec700 1 -- 172.21.15.155:0/887940586 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f80106010 con 0x7f5f800ffc80 2023-07-18T03:02:22.693 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:22.711+0000 7f5f767fc700 1 -- 172.21.15.155:0/887940586 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f5f7c022470 con 0x7f5f800ffc80 2023-07-18T03:02:22.694 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:22.711+0000 7f5f767fc700 1 --2- 172.21.15.155:0/887940586 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f5f6c0a1150 0x7f5f6c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:02:22.694 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:22.711+0000 7f5f767fc700 1 -- 172.21.15.155:0/887940586 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f5f7c0cd770 con 0x7f5f800ffc80 2023-07-18T03:02:22.694 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:22.712+0000 7f5f85788700 1 --2- 172.21.15.155:0/887940586 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f5f6c0a1150 0x7f5f6c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:02:22.695 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:22.712+0000 7f5f85788700 1 --2- 172.21.15.155:0/887940586 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f5f6c0a1150 0x7f5f6c0a3600 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5f800ffae0 tx=0x7f5f70009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:02:22.695 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:22.715+0000 7f5f767fc700 1 -- 172.21.15.155:0/887940586 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5f7c097c60 con 0x7f5f800ffc80 2023-07-18T03:02:22.887 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:22.907+0000 7f5f879ec700 1 -- 172.21.15.155:0/887940586 --> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f5f80060950 con 0x7f5f6c0a1150 2023-07-18T03:02:23.436 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:23 smithi179 ceph-mon[129229]: Detected new or changed devices on smithi101 2023-07-18T03:02:23.436 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:23 smithi179 ceph-mon[129229]: Adjusting osd_memory_target on smithi101 to 5755M 2023-07-18T03:02:23.436 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:23 smithi179 ceph-mon[129229]: osdmap e47: 7 total, 7 up, 7 in 2023-07-18T03:02:23.436 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:23 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:02:23.437 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:23 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:02:23.437 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:23 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:02:23.473 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:23 smithi101 ceph-mon[126379]: Detected new or changed devices on smithi101 2023-07-18T03:02:23.474 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:23 smithi101 ceph-mon[126379]: Adjusting osd_memory_target on smithi101 to 5755M 2023-07-18T03:02:23.474 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:23 smithi101 ceph-mon[126379]: osdmap e47: 7 total, 7 up, 7 in 2023-07-18T03:02:23.474 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:23 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:02:23.474 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:23 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:02:23.474 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:23 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:02:23.531 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:23 smithi155 ceph-mon[128885]: Detected new or changed devices on smithi101 2023-07-18T03:02:23.531 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:23 smithi155 ceph-mon[128885]: Adjusting osd_memory_target on smithi101 to 5755M 2023-07-18T03:02:23.531 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:23 smithi155 ceph-mon[128885]: osdmap e47: 7 total, 7 up, 7 in 2023-07-18T03:02:23.531 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:23 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:02:23.531 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:23 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:02:23.531 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:23 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:02:24.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:24 smithi179 ceph-mon[129229]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:02:24.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:24 smithi179 ceph-mon[129229]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:02:24.723 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:24 smithi101 ceph-mon[126379]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:02:24.724 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:24 smithi101 ceph-mon[126379]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:02:24.781 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:24 smithi155 ceph-mon[128885]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:02:24.781 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:24 smithi155 ceph-mon[128885]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:02:25.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:25 smithi179 ceph-mon[129229]: from='client.? 172.21.15.155:0/2480546838' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed23fd0a-678f-4e15-b42c-cfe873315714"}]: dispatch 2023-07-18T03:02:25.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:25 smithi179 ceph-mon[129229]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed23fd0a-678f-4e15-b42c-cfe873315714"}]: dispatch 2023-07-18T03:02:25.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:25 smithi179 ceph-mon[129229]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ed23fd0a-678f-4e15-b42c-cfe873315714"}]': finished 2023-07-18T03:02:25.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:25 smithi179 ceph-mon[129229]: osdmap e48: 8 total, 7 up, 8 in 2023-07-18T03:02:25.687 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:25 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-18T03:02:25.724 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:25 smithi101 ceph-mon[126379]: from='client.? 172.21.15.155:0/2480546838' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed23fd0a-678f-4e15-b42c-cfe873315714"}]: dispatch 2023-07-18T03:02:25.724 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:25 smithi101 ceph-mon[126379]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed23fd0a-678f-4e15-b42c-cfe873315714"}]: dispatch 2023-07-18T03:02:25.724 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:25 smithi101 ceph-mon[126379]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ed23fd0a-678f-4e15-b42c-cfe873315714"}]': finished 2023-07-18T03:02:25.724 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:25 smithi101 ceph-mon[126379]: osdmap e48: 8 total, 7 up, 8 in 2023-07-18T03:02:25.724 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:25 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-18T03:02:25.781 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:25 smithi155 ceph-mon[128885]: from='client.? 172.21.15.155:0/2480546838' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed23fd0a-678f-4e15-b42c-cfe873315714"}]: dispatch 2023-07-18T03:02:25.781 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:25 smithi155 ceph-mon[128885]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed23fd0a-678f-4e15-b42c-cfe873315714"}]: dispatch 2023-07-18T03:02:25.781 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:25 smithi155 ceph-mon[128885]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ed23fd0a-678f-4e15-b42c-cfe873315714"}]': finished 2023-07-18T03:02:25.781 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:25 smithi155 ceph-mon[128885]: osdmap e48: 8 total, 7 up, 8 in 2023-07-18T03:02:25.782 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:25 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-18T03:02:26.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:26 smithi179 ceph-mon[129229]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:02:26.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:26 smithi179 ceph-mon[129229]: from='client.? 172.21.15.155:0/2486941477' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:02:26.723 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:26 smithi101 ceph-mon[126379]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:02:26.723 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:26 smithi101 ceph-mon[126379]: from='client.? 172.21.15.155:0/2486941477' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:02:26.781 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:26 smithi155 ceph-mon[128885]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:02:26.781 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:26 smithi155 ceph-mon[128885]: from='client.? 172.21.15.155:0/2486941477' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:02:27.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:27 smithi179 ceph-mon[129229]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:02:27.723 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:27 smithi101 ceph-mon[126379]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:02:27.781 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:27 smithi155 ceph-mon[128885]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:02:30.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:30 smithi179 ceph-mon[129229]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:02:30.724 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:30 smithi101 ceph-mon[126379]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:02:30.781 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:30 smithi155 ceph-mon[128885]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:02:32.443 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:32 smithi155 ceph-mon[128885]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:02:32.443 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:32 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-07-18T03:02:32.443 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:32 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:02:32.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:32 smithi179 ceph-mon[129229]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:02:32.687 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:32 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-07-18T03:02:32.687 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:32 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:02:32.724 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:32 smithi101 ceph-mon[126379]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:02:32.724 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:32 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-07-18T03:02:32.724 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:32 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:02:33.531 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:33 smithi155 ceph-mon[128885]: Deploying daemon osd.7 on smithi155 2023-07-18T03:02:33.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:33 smithi179 ceph-mon[129229]: Deploying daemon osd.7 on smithi155 2023-07-18T03:02:33.724 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:33 smithi101 ceph-mon[126379]: Deploying daemon osd.7 on smithi155 2023-07-18T03:02:34.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:34 smithi179 ceph-mon[129229]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:02:34.723 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:34 smithi101 ceph-mon[126379]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:02:34.782 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:34 smithi155 ceph-mon[128885]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:02:35.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:35 smithi179 ceph-mon[129229]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:02:35.713 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:35 smithi155 ceph-mon[128885]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:02:35.723 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:35 smithi101 ceph-mon[126379]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:02:36.851 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:36 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:36.851 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:36 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:36.851 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:36 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:02:37.186 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:36 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:37.187 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:36 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:37.187 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:36 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:02:37.224 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:36 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:37.224 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:36 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:37.224 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:36 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:02:38.031 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:37 smithi155 ceph-mon[128885]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:02:38.186 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:37 smithi179 ceph-mon[129229]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:02:38.224 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:37 smithi101 ceph-mon[126379]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:02:38.499 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:38.519+0000 7f5f767fc700 1 -- 172.21.15.155:0/887940586 <== mgr.14150 v2:172.21.15.101:6800/3963742842 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5f80060950 con 0x7f5f6c0a1150 2023-07-18T03:02:38.500 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 7 on host 'smithi155' 2023-07-18T03:02:38.501 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:38.521+0000 7f5f879ec700 1 -- 172.21.15.155:0/887940586 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f5f6c0a1150 msgr2=0x7f5f6c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:02:38.501 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:38.521+0000 7f5f879ec700 1 --2- 172.21.15.155:0/887940586 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f5f6c0a1150 0x7f5f6c0a3600 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5f800ffae0 tx=0x7f5f70009190 comp rx=0 tx=0).stop 2023-07-18T03:02:38.501 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:38.521+0000 7f5f879ec700 1 -- 172.21.15.155:0/887940586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f800ffc80 msgr2=0x7f5f80194da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:02:38.501 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:38.521+0000 7f5f879ec700 1 --2- 172.21.15.155:0/887940586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f800ffc80 0x7f5f80194da0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c000c00 tx=0x7f5f7c00f780 comp rx=0 tx=0).stop 2023-07-18T03:02:38.501 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:38.522+0000 7f5f879ec700 1 -- 172.21.15.155:0/887940586 shutdown_connections 2023-07-18T03:02:38.502 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:38.522+0000 7f5f879ec700 1 --2- 172.21.15.155:0/887940586 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f5f6c0a1150 0x7f5f6c0a3600 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:02:38.502 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:38.522+0000 7f5f879ec700 1 --2- 172.21.15.155:0/887940586 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5f80100610 0x7f5f8019be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:02:38.502 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:38.522+0000 7f5f879ec700 1 --2- 172.21.15.155:0/887940586 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f800fea80 0x7f5f80194860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:02:38.502 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:38.522+0000 7f5f879ec700 1 --2- 172.21.15.155:0/887940586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f800ffc80 0x7f5f80194da0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:02:38.502 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:38.522+0000 7f5f879ec700 1 -- 172.21.15.155:0/887940586 >> 172.21.15.155:0/887940586 conn(0x7f5f800fa0b0 msgr2=0x7f5f80103150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:02:38.502 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:38.522+0000 7f5f879ec700 1 -- 172.21.15.155:0/887940586 shutdown_connections 2023-07-18T03:02:38.502 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:02:38.522+0000 7f5f879ec700 1 -- 172.21.15.155:0/887940586 wait complete. 2023-07-18T03:02:38.876 DEBUG:teuthology.orchestra.run.smithi155:osd.7> sudo journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@osd.7.service 2023-07-18T03:02:38.878 INFO:tasks.cephadm:Deploying osd.8 on smithi179 with /dev/vg_nvme/lv_2... 2023-07-18T03:02:38.879 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-07-18T03:02:38.910 INFO:journalctl@ceph.osd.7.smithi155.stdout:-- Logs begin at Tue 2023-07-18 02:29:36 UTC. -- 2023-07-18T03:02:39.272 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:39 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:39.272 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:39 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:39.272 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:39 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:02:39.273 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:39 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:02:39.273 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:39 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:39.273 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:39 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:39.443 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.c/config 2023-07-18T03:02:39.531 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:39 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:39.531 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:39 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:02:39.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:39 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:39.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:39 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:39.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:39 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:02:39.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:39 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:02:39.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:39 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:39.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:39 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:39.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:39 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:39.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:39 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:02:39.724 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:39 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:39.724 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:39 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:39.724 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:39 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:02:39.724 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:39 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:02:39.725 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:39 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:39.725 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:39 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:39.725 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:39 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:39.725 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:39 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:02:39.939 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:02:39 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:02:39.701+0000 7f237ae0e640 -1 osd.7 0 log_to_monitors true 2023-07-18T03:02:40.532 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:40 smithi155 ceph-mon[128885]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:02:40.532 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:40 smithi155 ceph-mon[128885]: from='osd.7 [v2:172.21.15.155:6816/3256322291,v1:172.21.15.155:6817/3256322291]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-18T03:02:40.532 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:40 smithi155 ceph-mon[128885]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-18T03:02:40.687 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:40 smithi179 ceph-mon[129229]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:02:40.687 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:40 smithi179 ceph-mon[129229]: from='osd.7 [v2:172.21.15.155:6816/3256322291,v1:172.21.15.155:6817/3256322291]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-18T03:02:40.687 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:40 smithi179 ceph-mon[129229]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-18T03:02:40.724 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:40 smithi101 ceph-mon[126379]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:02:40.724 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:40 smithi101 ceph-mon[126379]: from='osd.7 [v2:172.21.15.155:6816/3256322291,v1:172.21.15.155:6817/3256322291]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-18T03:02:40.724 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:40 smithi101 ceph-mon[126379]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-18T03:02:40.954 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-07-18T03:02:40.955 INFO:teuthology.orchestra.run.smithi179.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-07-18T03:02:40.955 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10+0 records in 2023-07-18T03:02:40.955 INFO:teuthology.orchestra.run.smithi179.stderr:10+0 records out 2023-07-18T03:02:40.956 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.014782 s, 709 MB/s 2023-07-18T03:02:40.956 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping successful for: 2023-07-18T03:02:41.382 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi179:vg_nvme/lv_2 2023-07-18T03:02:41.687 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:41 smithi179 ceph-mon[129229]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-07-18T03:02:41.687 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:41 smithi179 ceph-mon[129229]: osdmap e49: 8 total, 7 up, 8 in 2023-07-18T03:02:41.687 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:41 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-18T03:02:41.688 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:41 smithi179 ceph-mon[129229]: from='osd.7 [v2:172.21.15.155:6816/3256322291,v1:172.21.15.155:6817/3256322291]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-18T03:02:41.688 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:41 smithi179 ceph-mon[129229]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-18T03:02:41.688 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:41 smithi179 ceph-mon[129229]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:02:41.688 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:41 smithi179 ceph-mon[129229]: Detected new or changed devices on smithi155 2023-07-18T03:02:41.688 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:41 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:41.688 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:41 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:41.689 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:41 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:02:41.689 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:41 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:02:41.689 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:41 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:02:41.689 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:41 smithi179 ceph-mon[129229]: Adjusting osd_memory_target on smithi155 to 5722M 2023-07-18T03:02:41.689 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:41 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:41.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:41 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:02:41.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:41 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:02:41.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:41 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:41.781 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:02:41 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:02:41.521+0000 7f2370486700 -1 osd.7 0 waiting for initial osdmap 2023-07-18T03:02:41.782 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:02:41 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:02:41.533+0000 7f236b279700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-18T03:02:41.782 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:41 smithi155 ceph-mon[128885]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-07-18T03:02:41.782 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:41 smithi155 ceph-mon[128885]: osdmap e49: 8 total, 7 up, 8 in 2023-07-18T03:02:41.783 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:41 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-18T03:02:41.783 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:41 smithi155 ceph-mon[128885]: from='osd.7 [v2:172.21.15.155:6816/3256322291,v1:172.21.15.155:6817/3256322291]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-18T03:02:41.783 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:41 smithi155 ceph-mon[128885]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-18T03:02:41.783 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:41 smithi155 ceph-mon[128885]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:02:41.783 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:41 smithi155 ceph-mon[128885]: Detected new or changed devices on smithi155 2023-07-18T03:02:41.783 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:41 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:41.783 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:41 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:41.784 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:41 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:02:41.784 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:41 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:02:41.784 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:41 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:02:41.784 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:41 smithi155 ceph-mon[128885]: Adjusting osd_memory_target on smithi155 to 5722M 2023-07-18T03:02:41.784 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:41 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:41.784 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:41 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:02:41.784 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:41 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:02:41.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:41 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:41.934 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.c/config 2023-07-18T03:02:41.974 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:41 smithi101 ceph-mon[126379]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-07-18T03:02:41.975 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:41 smithi101 ceph-mon[126379]: osdmap e49: 8 total, 7 up, 8 in 2023-07-18T03:02:41.975 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:41 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-18T03:02:41.976 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:41 smithi101 ceph-mon[126379]: from='osd.7 [v2:172.21.15.155:6816/3256322291,v1:172.21.15.155:6817/3256322291]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-18T03:02:41.976 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:41 smithi101 ceph-mon[126379]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-18T03:02:41.977 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:41 smithi101 ceph-mon[126379]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:02:41.977 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:41 smithi101 ceph-mon[126379]: Detected new or changed devices on smithi155 2023-07-18T03:02:41.977 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:41 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:41.978 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:41 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:41.978 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:41 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:02:41.979 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:41 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:02:41.979 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:41 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:02:41.979 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:41 smithi101 ceph-mon[126379]: Adjusting osd_memory_target on smithi155 to 5722M 2023-07-18T03:02:41.980 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:41 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:41.980 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:41 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:02:41.980 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:41 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:02:41.981 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:41 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:42.781 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:42 smithi155 ceph-mon[128885]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2023-07-18T03:02:42.781 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:42 smithi155 ceph-mon[128885]: osdmap e50: 8 total, 7 up, 8 in 2023-07-18T03:02:42.782 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:42 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-18T03:02:42.782 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:42 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-18T03:02:42.937 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:42 smithi179 ceph-mon[129229]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2023-07-18T03:02:42.937 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:42 smithi179 ceph-mon[129229]: osdmap e50: 8 total, 7 up, 8 in 2023-07-18T03:02:42.937 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:42 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-18T03:02:42.937 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:42 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-18T03:02:42.974 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:42 smithi101 ceph-mon[126379]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2023-07-18T03:02:42.974 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:42 smithi101 ceph-mon[126379]: osdmap e50: 8 total, 7 up, 8 in 2023-07-18T03:02:42.975 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:42 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-18T03:02:42.975 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:42 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-18T03:02:43.128 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:43.149+0000 7f77ae79e700 1 -- 172.21.15.179:0/4055366237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f77a8100db0 msgr2=0x7f77a8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:02:43.128 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:43.149+0000 7f77ae79e700 1 --2- 172.21.15.179:0/4055366237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f77a8100db0 0x7f77a8101200 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7798009990 tx=0x7f7798009ca0 comp rx=0 tx=0).stop 2023-07-18T03:02:43.128 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:43.149+0000 7f77ae79e700 1 -- 172.21.15.179:0/4055366237 shutdown_connections 2023-07-18T03:02:43.129 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:43.149+0000 7f77ae79e700 1 --2- 172.21.15.179:0/4055366237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77a8101740 0x7f77a8108000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:02:43.129 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:43.149+0000 7f77ae79e700 1 --2- 172.21.15.179:0/4055366237 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f77a80ffbb0 0x7f77a80fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:02:43.129 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:43.149+0000 7f77ae79e700 1 --2- 172.21.15.179:0/4055366237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f77a8100db0 0x7f77a8101200 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:02:43.129 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:43.149+0000 7f77ae79e700 1 -- 172.21.15.179:0/4055366237 >> 172.21.15.179:0/4055366237 conn(0x7f77a80fb1c0 msgr2=0x7f77a80fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:02:43.129 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:43.149+0000 7f77ae79e700 1 -- 172.21.15.179:0/4055366237 shutdown_connections 2023-07-18T03:02:43.129 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:43.149+0000 7f77ae79e700 1 -- 172.21.15.179:0/4055366237 wait complete. 2023-07-18T03:02:43.131 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:43.149+0000 7f77ae79e700 1 Processor -- start 2023-07-18T03:02:43.131 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:43.149+0000 7f77ae79e700 1 -- start start 2023-07-18T03:02:43.131 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:43.150+0000 7f77ae79e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f77a80ffbb0 0x7f77a80742b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:02:43.131 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:43.150+0000 7f77ae79e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77a8100db0 0x7f77a8072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:02:43.131 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:43.150+0000 7f77ae79e700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f77a8101740 0x7f77a8072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:02:43.131 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:43.150+0000 7f77ae79e700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a80735c0 con 0x7f77a8101740 2023-07-18T03:02:43.132 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:43.150+0000 7f77ae79e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a8073730 con 0x7f77a8100db0 2023-07-18T03:02:43.132 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:43.150+0000 7f77ae79e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a819f9a0 con 0x7f77a80ffbb0 2023-07-18T03:02:43.132 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:43.150+0000 7f77a7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f77a80ffbb0 0x7f77a80742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:02:43.132 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:43.150+0000 7f77a7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f77a80ffbb0 0x7f77a80742b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48386/0 (socket says 172.21.15.179:48386) 2023-07-18T03:02:43.132 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:43.150+0000 7f77a7fff700 1 -- 172.21.15.179:0/2372589739 learned_addr learned my addr 172.21.15.179:0/2372589739 (peer_addr_for_me v2:172.21.15.179:0/0) 2023-07-18T03:02:43.132 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:43.150+0000 7f77a77fe700 1 --2- 172.21.15.179:0/2372589739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77a8100db0 0x7f77a8072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:02:43.133 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:43.150+0000 7f77a7fff700 1 -- 172.21.15.179:0/2372589739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77a8100db0 msgr2=0x7f77a8072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:02:43.133 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:43.150+0000 7f77a7fff700 1 --2- 172.21.15.179:0/2372589739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77a8100db0 0x7f77a8072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:02:43.133 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:43.150+0000 7f77a7fff700 1 -- 172.21.15.179:0/2372589739 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f77a8101740 msgr2=0x7f77a8072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:02:43.133 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:43.150+0000 7f77a7fff700 1 --2- 172.21.15.179:0/2372589739 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f77a8101740 0x7f77a8072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:02:43.133 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:43.150+0000 7f77a7fff700 1 -- 172.21.15.179:0/2372589739 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77980095f0 con 0x7f77a80ffbb0 2023-07-18T03:02:43.133 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:43.150+0000 7f77a7fff700 1 --2- 172.21.15.179:0/2372589739 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f77a80ffbb0 0x7f77a80742b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f779c00b500 tx=0x7f779c00b8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:02:43.133 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:43.150+0000 7f77a57fa700 1 -- 172.21.15.179:0/2372589739 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779c011830 con 0x7f77a80ffbb0 2023-07-18T03:02:43.134 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:43.151+0000 7f77a57fa700 1 -- 172.21.15.179:0/2372589739 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f779c011e50 con 0x7f77a80ffbb0 2023-07-18T03:02:43.134 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:43.151+0000 7f77ae79e700 1 -- 172.21.15.179:0/2372589739 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77a819fc80 con 0x7f77a80ffbb0 2023-07-18T03:02:43.134 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:43.151+0000 7f77a57fa700 1 -- 172.21.15.179:0/2372589739 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779c00f2e0 con 0x7f77a80ffbb0 2023-07-18T03:02:43.134 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:43.151+0000 7f77ae79e700 1 -- 172.21.15.179:0/2372589739 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77a81a0180 con 0x7f77a80ffbb0 2023-07-18T03:02:43.134 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:43.152+0000 7f77ae79e700 1 -- 172.21.15.179:0/2372589739 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77a8107200 con 0x7f77a80ffbb0 2023-07-18T03:02:43.135 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:43.153+0000 7f77a57fa700 1 -- 172.21.15.179:0/2372589739 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f779c010580 con 0x7f77a80ffbb0 2023-07-18T03:02:43.135 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:43.154+0000 7f77a57fa700 1 --2- 172.21.15.179:0/2372589739 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f77900a1150 0x7f77900a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:02:43.135 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:43.154+0000 7f77a57fa700 1 -- 172.21.15.179:0/2372589739 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5397+0+0 (secure 0 0 0) 0x7f779c0cbe50 con 0x7f77a80ffbb0 2023-07-18T03:02:43.135 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:43.154+0000 7f77a77fe700 1 --2- 172.21.15.179:0/2372589739 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f77900a1150 0x7f77900a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:02:43.135 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:43.155+0000 7f77a77fe700 1 --2- 172.21.15.179:0/2372589739 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f77900a1150 0x7f77900a3600 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7798006010 tx=0x7f779800b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:02:43.136 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:43.158+0000 7f77a57fa700 1 -- 172.21.15.179:0/2372589739 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f779c0960e0 con 0x7f77a80ffbb0 2023-07-18T03:02:43.328 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:43.349+0000 7f77ae79e700 1 -- 172.21.15.179:0/2372589739 --> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f77a8060950 con 0x7f77900a1150 2023-07-18T03:02:43.781 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:43 smithi155 ceph-mon[128885]: purged_snaps scrub starts 2023-07-18T03:02:43.782 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:43 smithi155 ceph-mon[128885]: purged_snaps scrub ok 2023-07-18T03:02:43.782 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:43 smithi155 ceph-mon[128885]: osd.7 [v2:172.21.15.155:6816/3256322291,v1:172.21.15.155:6817/3256322291] boot 2023-07-18T03:02:43.782 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:43 smithi155 ceph-mon[128885]: osdmap e51: 8 total, 8 up, 8 in 2023-07-18T03:02:43.782 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:43 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-18T03:02:43.782 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:43 smithi155 ceph-mon[128885]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:02:43.783 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:43 smithi155 ceph-mon[128885]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:02:43.783 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:43 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:02:43.783 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:43 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:02:43.783 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:43 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:02:43.783 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:43 smithi155 ceph-mon[128885]: osdmap e52: 8 total, 8 up, 8 in 2023-07-18T03:02:43.937 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:43 smithi179 ceph-mon[129229]: purged_snaps scrub starts 2023-07-18T03:02:43.937 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:43 smithi179 ceph-mon[129229]: purged_snaps scrub ok 2023-07-18T03:02:43.937 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:43 smithi179 ceph-mon[129229]: osd.7 [v2:172.21.15.155:6816/3256322291,v1:172.21.15.155:6817/3256322291] boot 2023-07-18T03:02:43.938 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:43 smithi179 ceph-mon[129229]: osdmap e51: 8 total, 8 up, 8 in 2023-07-18T03:02:43.938 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:43 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-18T03:02:43.938 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:43 smithi179 ceph-mon[129229]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:02:43.938 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:43 smithi179 ceph-mon[129229]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:02:43.938 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:43 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:02:43.938 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:43 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:02:43.938 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:43 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:02:43.939 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:43 smithi179 ceph-mon[129229]: osdmap e52: 8 total, 8 up, 8 in 2023-07-18T03:02:43.975 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:43 smithi101 ceph-mon[126379]: purged_snaps scrub starts 2023-07-18T03:02:43.975 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:43 smithi101 ceph-mon[126379]: purged_snaps scrub ok 2023-07-18T03:02:43.975 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:43 smithi101 ceph-mon[126379]: osd.7 [v2:172.21.15.155:6816/3256322291,v1:172.21.15.155:6817/3256322291] boot 2023-07-18T03:02:43.975 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:43 smithi101 ceph-mon[126379]: osdmap e51: 8 total, 8 up, 8 in 2023-07-18T03:02:43.975 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:43 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-18T03:02:43.976 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:43 smithi101 ceph-mon[126379]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:02:43.976 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:43 smithi101 ceph-mon[126379]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:02:43.976 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:43 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:02:43.976 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:43 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:02:43.976 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:43 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:02:43.976 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:43 smithi101 ceph-mon[126379]: osdmap e52: 8 total, 8 up, 8 in 2023-07-18T03:02:45.687 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:45 smithi179 ceph-mon[129229]: osdmap e53: 8 total, 8 up, 8 in 2023-07-18T03:02:45.687 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:45 smithi179 ceph-mon[129229]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 715 GiB / 715 GiB avail 2023-07-18T03:02:45.724 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:45 smithi101 ceph-mon[126379]: osdmap e53: 8 total, 8 up, 8 in 2023-07-18T03:02:45.724 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:45 smithi101 ceph-mon[126379]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 715 GiB / 715 GiB avail 2023-07-18T03:02:45.782 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:45 smithi155 ceph-mon[128885]: osdmap e53: 8 total, 8 up, 8 in 2023-07-18T03:02:45.782 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:45 smithi155 ceph-mon[128885]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 715 GiB / 715 GiB avail 2023-07-18T03:02:46.687 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:46 smithi179 ceph-mon[129229]: osdmap e54: 8 total, 8 up, 8 in 2023-07-18T03:02:46.687 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:46 smithi179 ceph-mon[129229]: from='client.? 172.21.15.179:0/1988882360' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eca3d8e6-7baf-47a5-b991-328001c7ebef"}]: dispatch 2023-07-18T03:02:46.688 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:46 smithi179 ceph-mon[129229]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eca3d8e6-7baf-47a5-b991-328001c7ebef"}]: dispatch 2023-07-18T03:02:46.688 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:46 smithi179 ceph-mon[129229]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eca3d8e6-7baf-47a5-b991-328001c7ebef"}]': finished 2023-07-18T03:02:46.688 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:46 smithi179 ceph-mon[129229]: osdmap e55: 9 total, 8 up, 9 in 2023-07-18T03:02:46.689 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:46 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-18T03:02:46.689 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:46 smithi179 ceph-mon[129229]: from='client.? 172.21.15.179:0/3480643591' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:02:46.724 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:46 smithi101 ceph-mon[126379]: osdmap e54: 8 total, 8 up, 8 in 2023-07-18T03:02:46.725 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:46 smithi101 ceph-mon[126379]: from='client.? 172.21.15.179:0/1988882360' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eca3d8e6-7baf-47a5-b991-328001c7ebef"}]: dispatch 2023-07-18T03:02:46.725 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:46 smithi101 ceph-mon[126379]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eca3d8e6-7baf-47a5-b991-328001c7ebef"}]: dispatch 2023-07-18T03:02:46.725 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:46 smithi101 ceph-mon[126379]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eca3d8e6-7baf-47a5-b991-328001c7ebef"}]': finished 2023-07-18T03:02:46.725 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:46 smithi101 ceph-mon[126379]: osdmap e55: 9 total, 8 up, 9 in 2023-07-18T03:02:46.726 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:46 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-18T03:02:46.726 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:46 smithi101 ceph-mon[126379]: from='client.? 172.21.15.179:0/3480643591' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:02:46.782 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:46 smithi155 ceph-mon[128885]: osdmap e54: 8 total, 8 up, 8 in 2023-07-18T03:02:46.782 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:46 smithi155 ceph-mon[128885]: from='client.? 172.21.15.179:0/1988882360' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eca3d8e6-7baf-47a5-b991-328001c7ebef"}]: dispatch 2023-07-18T03:02:46.782 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:46 smithi155 ceph-mon[128885]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eca3d8e6-7baf-47a5-b991-328001c7ebef"}]: dispatch 2023-07-18T03:02:46.783 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:46 smithi155 ceph-mon[128885]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eca3d8e6-7baf-47a5-b991-328001c7ebef"}]': finished 2023-07-18T03:02:46.783 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:46 smithi155 ceph-mon[128885]: osdmap e55: 9 total, 8 up, 9 in 2023-07-18T03:02:46.783 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:46 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-18T03:02:46.783 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:46 smithi155 ceph-mon[128885]: from='client.? 172.21.15.179:0/3480643591' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:02:47.687 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:47 smithi179 ceph-mon[129229]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-18T03:02:47.724 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:47 smithi101 ceph-mon[126379]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-18T03:02:47.781 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:47 smithi155 ceph-mon[128885]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-18T03:02:50.648 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:50 smithi179 ceph-mon[129229]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 106 KiB/s, 0 objects/s recovering 2023-07-18T03:02:50.724 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:50 smithi101 ceph-mon[126379]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 106 KiB/s, 0 objects/s recovering 2023-07-18T03:02:50.782 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:50 smithi155 ceph-mon[128885]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 106 KiB/s, 0 objects/s recovering 2023-07-18T03:02:51.687 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:51 smithi179 ceph-mon[129229]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 90 KiB/s, 0 objects/s recovering 2023-07-18T03:02:51.724 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:51 smithi101 ceph-mon[126379]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 90 KiB/s, 0 objects/s recovering 2023-07-18T03:02:51.782 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:51 smithi155 ceph-mon[128885]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 90 KiB/s, 0 objects/s recovering 2023-07-18T03:02:52.687 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:52 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-07-18T03:02:52.687 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:52 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:02:52.687 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:52 smithi179 ceph-mon[129229]: Deploying daemon osd.8 on smithi179 2023-07-18T03:02:52.724 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:52 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-07-18T03:02:52.725 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:52 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:02:52.725 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:52 smithi101 ceph-mon[126379]: Deploying daemon osd.8 on smithi179 2023-07-18T03:02:52.782 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:52 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-07-18T03:02:52.782 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:52 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:02:52.782 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:52 smithi155 ceph-mon[128885]: Deploying daemon osd.8 on smithi179 2023-07-18T03:02:54.687 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:54 smithi179 ceph-mon[129229]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-18T03:02:54.724 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:54 smithi101 ceph-mon[126379]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-18T03:02:54.782 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:54 smithi155 ceph-mon[128885]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-18T03:02:56.687 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:56 smithi179 ceph-mon[129229]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 61 KiB/s, 0 objects/s recovering 2023-07-18T03:02:56.724 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:56 smithi101 ceph-mon[126379]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 61 KiB/s, 0 objects/s recovering 2023-07-18T03:02:56.782 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:56 smithi155 ceph-mon[128885]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 61 KiB/s, 0 objects/s recovering 2023-07-18T03:02:57.648 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:57 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:57.649 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:57 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:57.649 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:57 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:02:57.649 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:57 smithi179 ceph-mon[129229]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 52 KiB/s, 0 objects/s recovering 2023-07-18T03:02:57.730 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:57 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:57.730 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:57 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:57.731 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:57 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:02:57.731 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:57 smithi101 ceph-mon[126379]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 52 KiB/s, 0 objects/s recovering 2023-07-18T03:02:57.782 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:57 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:57.783 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:57 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:57.783 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:57 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:02:57.783 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:57 smithi155 ceph-mon[128885]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 52 KiB/s, 0 objects/s recovering 2023-07-18T03:02:59.294 INFO:teuthology.orchestra.run.smithi179.stdout:Created osd(s) 8 on host 'smithi179' 2023-07-18T03:02:59.295 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:59.313+0000 7f77a57fa700 1 -- 172.21.15.179:0/2372589739 <== mgr.14150 v2:172.21.15.101:6800/3963742842 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f77a8060950 con 0x7f77900a1150 2023-07-18T03:02:59.297 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:59.316+0000 7f77ae79e700 1 -- 172.21.15.179:0/2372589739 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f77900a1150 msgr2=0x7f77900a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:02:59.297 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:59.316+0000 7f77ae79e700 1 --2- 172.21.15.179:0/2372589739 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f77900a1150 0x7f77900a3600 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7798006010 tx=0x7f779800b540 comp rx=0 tx=0).stop 2023-07-18T03:02:59.297 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:59.316+0000 7f77ae79e700 1 -- 172.21.15.179:0/2372589739 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f77a80ffbb0 msgr2=0x7f77a80742b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:02:59.297 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:59.316+0000 7f77ae79e700 1 --2- 172.21.15.179:0/2372589739 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f77a80ffbb0 0x7f77a80742b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f779c00b500 tx=0x7f779c00b8c0 comp rx=0 tx=0).stop 2023-07-18T03:02:59.298 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:59.316+0000 7f77ae79e700 1 -- 172.21.15.179:0/2372589739 shutdown_connections 2023-07-18T03:02:59.298 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:59.316+0000 7f77ae79e700 1 --2- 172.21.15.179:0/2372589739 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f77900a1150 0x7f77900a3600 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:02:59.298 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:59.316+0000 7f77ae79e700 1 --2- 172.21.15.179:0/2372589739 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f77a8101740 0x7f77a8072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:02:59.298 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:59.316+0000 7f77ae79e700 1 --2- 172.21.15.179:0/2372589739 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f77a80ffbb0 0x7f77a80742b0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:02:59.298 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:59.316+0000 7f77ae79e700 1 --2- 172.21.15.179:0/2372589739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77a8100db0 0x7f77a8072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:02:59.298 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:59.316+0000 7f77ae79e700 1 -- 172.21.15.179:0/2372589739 >> 172.21.15.179:0/2372589739 conn(0x7f77a80fb1c0 msgr2=0x7f77a8104340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:02:59.298 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:59.316+0000 7f77ae79e700 1 -- 172.21.15.179:0/2372589739 shutdown_connections 2023-07-18T03:02:59.299 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:02:59.316+0000 7f77ae79e700 1 -- 172.21.15.179:0/2372589739 wait complete. 2023-07-18T03:02:59.730 DEBUG:teuthology.orchestra.run.smithi179:osd.8> sudo journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@osd.8.service 2023-07-18T03:02:59.732 INFO:tasks.cephadm:Deploying osd.9 on smithi101 with /dev/vg_nvme/lv_1... 2023-07-18T03:02:59.733 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-07-18T03:02:59.763 INFO:journalctl@ceph.osd.8.smithi179.stdout:-- Logs begin at Tue 2023-07-18 02:29:48 UTC. -- 2023-07-18T03:02:59.937 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:59 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:59.938 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:59 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:59.938 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:59 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:02:59.938 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:59 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:02:59.938 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:59 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:59.938 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:59 smithi179 ceph-mon[129229]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-18T03:02:59.938 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:59 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:59.938 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:59 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:02:59.939 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:02:59 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:03:00.032 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:59 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:00.032 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:59 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:00.032 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:59 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:03:00.033 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:59 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:03:00.033 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:59 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:00.033 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:59 smithi155 ceph-mon[128885]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-18T03:03:00.033 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:59 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:00.033 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:59 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:00.033 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:02:59 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:03:00.225 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:59 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:00.225 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:59 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:00.225 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:59 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:03:00.225 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:59 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:03:00.225 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:59 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:00.226 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:59 smithi101 ceph-mon[126379]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-18T03:03:00.226 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:59 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:00.226 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:59 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:00.226 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:02:59 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:03:00.438 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:03:00 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:03:00.311+0000 7f046da92640 -1 osd.8 0 log_to_monitors true 2023-07-18T03:03:00.465 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:03:01.113 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:00 smithi101 ceph-mon[126379]: from='osd.8 [v2:172.21.15.179:6816/3520612559,v1:172.21.15.179:6817/3520612559]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-18T03:03:01.113 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:00 smithi101 ceph-mon[126379]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-18T03:03:01.187 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:00 smithi179 ceph-mon[129229]: from='osd.8 [v2:172.21.15.179:6816/3520612559,v1:172.21.15.179:6817/3520612559]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-18T03:03:01.188 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:00 smithi179 ceph-mon[129229]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-18T03:03:01.282 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:00 smithi155 ceph-mon[128885]: from='osd.8 [v2:172.21.15.179:6816/3520612559,v1:172.21.15.179:6817/3520612559]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-18T03:03:01.282 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:00 smithi155 ceph-mon[128885]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-18T03:03:02.187 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:03:01 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:03:01.835+0000 7f046310a700 -1 osd.8 0 waiting for initial osdmap 2023-07-18T03:03:02.188 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:03:01 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:03:01.847+0000 7f045c6fa700 -1 osd.8 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-18T03:03:02.188 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:01 smithi179 ceph-mon[129229]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-18T03:03:02.188 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:01 smithi179 ceph-mon[129229]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-07-18T03:03:02.188 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:01 smithi179 ceph-mon[129229]: osdmap e56: 9 total, 8 up, 9 in 2023-07-18T03:03:02.188 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:01 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-18T03:03:02.188 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:01 smithi179 ceph-mon[129229]: from='osd.8 [v2:172.21.15.179:6816/3520612559,v1:172.21.15.179:6817/3520612559]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2023-07-18T03:03:02.189 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:01 smithi179 ceph-mon[129229]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2023-07-18T03:03:02.224 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:01 smithi101 ceph-mon[126379]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-18T03:03:02.225 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:01 smithi101 ceph-mon[126379]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-07-18T03:03:02.225 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:01 smithi101 ceph-mon[126379]: osdmap e56: 9 total, 8 up, 9 in 2023-07-18T03:03:02.225 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:01 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-18T03:03:02.225 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:01 smithi101 ceph-mon[126379]: from='osd.8 [v2:172.21.15.179:6816/3520612559,v1:172.21.15.179:6817/3520612559]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2023-07-18T03:03:02.225 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:01 smithi101 ceph-mon[126379]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2023-07-18T03:03:02.282 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:01 smithi155 ceph-mon[128885]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-18T03:03:02.282 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:01 smithi155 ceph-mon[128885]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-07-18T03:03:02.282 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:01 smithi155 ceph-mon[128885]: osdmap e56: 9 total, 8 up, 9 in 2023-07-18T03:03:02.283 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:01 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-18T03:03:02.283 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:01 smithi155 ceph-mon[128885]: from='osd.8 [v2:172.21.15.179:6816/3520612559,v1:172.21.15.179:6817/3520612559]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2023-07-18T03:03:02.283 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:01 smithi155 ceph-mon[128885]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2023-07-18T03:03:02.566 INFO:teuthology.orchestra.run.smithi101.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-07-18T03:03:02.567 INFO:teuthology.orchestra.run.smithi101.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-07-18T03:03:02.567 INFO:teuthology.orchestra.run.smithi101.stderr: stderr: 10+0 records in 2023-07-18T03:03:02.567 INFO:teuthology.orchestra.run.smithi101.stderr:10+0 records out 2023-07-18T03:03:02.567 INFO:teuthology.orchestra.run.smithi101.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0387381 s, 271 MB/s 2023-07-18T03:03:02.567 INFO:teuthology.orchestra.run.smithi101.stderr:--> Zapping successful for: 2023-07-18T03:03:02.975 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:02 smithi101 ceph-mon[126379]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2023-07-18T03:03:02.976 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:02 smithi101 ceph-mon[126379]: osdmap e57: 9 total, 8 up, 9 in 2023-07-18T03:03:02.976 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:02 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-18T03:03:02.976 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:02 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-18T03:03:02.976 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:02 smithi101 ceph-mon[126379]: Detected new or changed devices on smithi179 2023-07-18T03:03:02.976 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:02 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:02.976 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:02 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:02.977 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:02 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:03:02.977 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:02 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:03:02.977 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:02 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:03:02.977 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:02 smithi101 ceph-mon[126379]: Adjusting osd_memory_target on smithi179 to 5722M 2023-07-18T03:03:02.977 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:02 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:02.977 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:02 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:03:02.977 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:02 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:03:02.978 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:02 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:03.042 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi101:vg_nvme/lv_1 2023-07-18T03:03:03.187 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:02 smithi179 ceph-mon[129229]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2023-07-18T03:03:03.188 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:02 smithi179 ceph-mon[129229]: osdmap e57: 9 total, 8 up, 9 in 2023-07-18T03:03:03.188 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:02 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-18T03:03:03.188 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:02 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-18T03:03:03.188 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:02 smithi179 ceph-mon[129229]: Detected new or changed devices on smithi179 2023-07-18T03:03:03.188 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:02 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:03.189 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:02 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:03.189 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:02 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:03:03.189 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:02 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:03:03.189 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:02 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:03:03.189 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:02 smithi179 ceph-mon[129229]: Adjusting osd_memory_target on smithi179 to 5722M 2023-07-18T03:03:03.190 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:02 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:03.190 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:02 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:03:03.190 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:02 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:03:03.190 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:02 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:03.282 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:02 smithi155 ceph-mon[128885]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2023-07-18T03:03:03.283 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:02 smithi155 ceph-mon[128885]: osdmap e57: 9 total, 8 up, 9 in 2023-07-18T03:03:03.283 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:02 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-18T03:03:03.283 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:02 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-18T03:03:03.284 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:02 smithi155 ceph-mon[128885]: Detected new or changed devices on smithi179 2023-07-18T03:03:03.284 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:02 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:03.284 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:02 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:03.285 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:02 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:03:03.285 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:02 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:03:03.285 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:02 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:03:03.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:02 smithi155 ceph-mon[128885]: Adjusting osd_memory_target on smithi179 to 5722M 2023-07-18T03:03:03.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:02 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:03.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:02 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:03:03.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:02 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:03:03.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:02 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:03.764 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:03:04.089 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:03 smithi101 ceph-mon[126379]: purged_snaps scrub starts 2023-07-18T03:03:04.090 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:03 smithi101 ceph-mon[126379]: purged_snaps scrub ok 2023-07-18T03:03:04.090 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:03 smithi101 ceph-mon[126379]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-18T03:03:04.090 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:03 smithi101 ceph-mon[126379]: osd.8 [v2:172.21.15.179:6816/3520612559,v1:172.21.15.179:6817/3520612559] boot 2023-07-18T03:03:04.090 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:03 smithi101 ceph-mon[126379]: osdmap e58: 9 total, 9 up, 9 in 2023-07-18T03:03:04.091 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:03 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-18T03:03:04.091 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:03 smithi101 ceph-mon[126379]: osdmap e59: 9 total, 9 up, 9 in 2023-07-18T03:03:04.187 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:03 smithi179 ceph-mon[129229]: purged_snaps scrub starts 2023-07-18T03:03:04.188 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:03 smithi179 ceph-mon[129229]: purged_snaps scrub ok 2023-07-18T03:03:04.188 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:03 smithi179 ceph-mon[129229]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-18T03:03:04.188 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:03 smithi179 ceph-mon[129229]: osd.8 [v2:172.21.15.179:6816/3520612559,v1:172.21.15.179:6817/3520612559] boot 2023-07-18T03:03:04.189 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:03 smithi179 ceph-mon[129229]: osdmap e58: 9 total, 9 up, 9 in 2023-07-18T03:03:04.189 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:03 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-18T03:03:04.189 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:03 smithi179 ceph-mon[129229]: osdmap e59: 9 total, 9 up, 9 in 2023-07-18T03:03:04.282 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:03 smithi155 ceph-mon[128885]: purged_snaps scrub starts 2023-07-18T03:03:04.283 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:03 smithi155 ceph-mon[128885]: purged_snaps scrub ok 2023-07-18T03:03:04.283 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:03 smithi155 ceph-mon[128885]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-18T03:03:04.283 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:03 smithi155 ceph-mon[128885]: osd.8 [v2:172.21.15.179:6816/3520612559,v1:172.21.15.179:6817/3520612559] boot 2023-07-18T03:03:04.283 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:03 smithi155 ceph-mon[128885]: osdmap e58: 9 total, 9 up, 9 in 2023-07-18T03:03:04.283 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:03 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-18T03:03:04.283 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:03 smithi155 ceph-mon[128885]: osdmap e59: 9 total, 9 up, 9 in 2023-07-18T03:03:05.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:05.271+0000 7f0226089700 1 -- 172.21.15.101:0/462744192 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0220100d90 msgr2=0x7f02201011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:03:05.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:05.271+0000 7f0226089700 1 --2- 172.21.15.101:0/462744192 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0220100d90 0x7f02201011e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0214009990 tx=0x7f0214009ca0 comp rx=0 tx=0).stop 2023-07-18T03:03:05.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:05.272+0000 7f0226089700 1 -- 172.21.15.101:0/462744192 shutdown_connections 2023-07-18T03:03:05.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:05.272+0000 7f0226089700 1 --2- 172.21.15.101:0/462744192 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0220101720 0x7f0220107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:03:05.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:05.272+0000 7f0226089700 1 --2- 172.21.15.101:0/462744192 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02200ffb90 0x7f02200fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:03:05.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:05.272+0000 7f0226089700 1 --2- 172.21.15.101:0/462744192 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0220100d90 0x7f02201011e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:03:05.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:05.272+0000 7f0226089700 1 -- 172.21.15.101:0/462744192 >> 172.21.15.101:0/462744192 conn(0x7f02200fb1a0 msgr2=0x7f02200fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:03:05.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:05.272+0000 7f0226089700 1 -- 172.21.15.101:0/462744192 shutdown_connections 2023-07-18T03:03:05.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:05.272+0000 7f0226089700 1 -- 172.21.15.101:0/462744192 wait complete. 2023-07-18T03:03:05.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:05.273+0000 7f0226089700 1 Processor -- start 2023-07-18T03:03:05.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:05.273+0000 7f0226089700 1 -- start start 2023-07-18T03:03:05.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:05.274+0000 7f0226089700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02200ffb90 0x7f02200742b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:03:05.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:05.274+0000 7f0226089700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0220100d90 0x7f0220072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:03:05.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:05.274+0000 7f0226089700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0220101720 0x7f0220072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:03:05.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:05.274+0000 7f0226089700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02200735c0 con 0x7f0220100d90 2023-07-18T03:03:05.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:05.274+0000 7f0226089700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0220073730 con 0x7f02200ffb90 2023-07-18T03:03:05.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:05.274+0000 7f0226089700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022019f970 con 0x7f0220101720 2023-07-18T03:03:05.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:05.274+0000 7f021f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02200ffb90 0x7f02200742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:03:05.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:05.274+0000 7f021effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0220100d90 0x7f0220072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:03:05.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:05.274+0000 7f021f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02200ffb90 0x7f02200742b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:40764/0 (socket says 172.21.15.101:40764) 2023-07-18T03:03:05.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:05.274+0000 7f021f7fe700 1 -- 172.21.15.101:0/3884492677 learned_addr learned my addr 172.21.15.101:0/3884492677 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:03:05.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:05.274+0000 7f021ffff700 1 --2- 172.21.15.101:0/3884492677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0220101720 0x7f0220072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:03:05.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:05.274+0000 7f021effd700 1 -- 172.21.15.101:0/3884492677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0220101720 msgr2=0x7f0220072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:03:05.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:05.274+0000 7f021effd700 1 --2- 172.21.15.101:0/3884492677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0220101720 0x7f0220072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:03:05.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:05.274+0000 7f021effd700 1 -- 172.21.15.101:0/3884492677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02200ffb90 msgr2=0x7f02200742b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:03:05.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:05.274+0000 7f021effd700 1 --2- 172.21.15.101:0/3884492677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02200ffb90 0x7f02200742b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:03:05.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:05.275+0000 7f021effd700 1 -- 172.21.15.101:0/3884492677 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02140095f0 con 0x7f0220100d90 2023-07-18T03:03:05.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:05.275+0000 7f021f7fe700 1 --2- 172.21.15.101:0/3884492677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02200ffb90 0x7f02200742b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:03:05.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:05.275+0000 7f021effd700 1 --2- 172.21.15.101:0/3884492677 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0220100d90 0x7f0220072900 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f021400b680 tx=0x7f0214005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:03:05.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:05.275+0000 7f021cff9700 1 -- 172.21.15.101:0/3884492677 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021401d070 con 0x7f0220100d90 2023-07-18T03:03:05.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:05.275+0000 7f021cff9700 1 -- 172.21.15.101:0/3884492677 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0214021410 con 0x7f0220100d90 2023-07-18T03:03:05.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:05.275+0000 7f0226089700 1 -- 172.21.15.101:0/3884492677 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f022019fbf0 con 0x7f0220100d90 2023-07-18T03:03:05.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:05.275+0000 7f021cff9700 1 -- 172.21.15.101:0/3884492677 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0214021ea0 con 0x7f0220100d90 2023-07-18T03:03:05.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:05.275+0000 7f0226089700 1 -- 172.21.15.101:0/3884492677 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02201a0000 con 0x7f0220100d90 2023-07-18T03:03:05.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:05.276+0000 7f0226089700 1 -- 172.21.15.101:0/3884492677 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0220107180 con 0x7f0220100d90 2023-07-18T03:03:05.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:05.278+0000 7f021cff9700 1 -- 172.21.15.101:0/3884492677 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f02140217f0 con 0x7f0220100d90 2023-07-18T03:03:05.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:05.279+0000 7f021cff9700 1 --2- 172.21.15.101:0/3884492677 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f02080a1150 0x7f02080a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:03:05.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:05.280+0000 7f021cff9700 1 -- 172.21.15.101:0/3884492677 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5858+0+0 (secure 0 0 0) 0x7f02140cd640 con 0x7f0220100d90 2023-07-18T03:03:05.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:05.280+0000 7f021f7fe700 1 --2- 172.21.15.101:0/3884492677 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f02080a1150 0x7f02080a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:03:05.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:05.280+0000 7f021f7fe700 1 --2- 172.21.15.101:0/3884492677 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f02080a1150 0x7f02080a3600 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0210005950 tx=0x7f0210009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:03:05.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:05.282+0000 7f021cff9700 1 -- 172.21.15.101:0/3884492677 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0214097710 con 0x7f0220100d90 2023-07-18T03:03:05.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:05.487+0000 7f0226089700 1 -- 172.21.15.101:0/3884492677 --> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f0220060950 con 0x7f02080a1150 2023-07-18T03:03:05.687 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:05 smithi179 ceph-mon[129229]: osdmap e60: 9 total, 9 up, 9 in 2023-07-18T03:03:05.688 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:05 smithi179 ceph-mon[129229]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-07-18T03:03:05.725 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:05 smithi101 ceph-mon[126379]: osdmap e60: 9 total, 9 up, 9 in 2023-07-18T03:03:05.725 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:05 smithi101 ceph-mon[126379]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-07-18T03:03:05.782 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:05 smithi155 ceph-mon[128885]: osdmap e60: 9 total, 9 up, 9 in 2023-07-18T03:03:05.783 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:05 smithi155 ceph-mon[128885]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-07-18T03:03:06.646 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:06 smithi101 ceph-mon[126379]: osdmap e61: 9 total, 9 up, 9 in 2023-07-18T03:03:06.646 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:06 smithi101 ceph-mon[126379]: from='client.14337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:03:06.646 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:06 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:03:06.647 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:06 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:03:06.647 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:06 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:03:06.687 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:06 smithi179 ceph-mon[129229]: osdmap e61: 9 total, 9 up, 9 in 2023-07-18T03:03:06.688 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:06 smithi179 ceph-mon[129229]: from='client.14337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:03:06.688 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:06 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:03:06.688 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:06 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:03:06.688 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:06 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:03:06.782 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:06 smithi155 ceph-mon[128885]: osdmap e61: 9 total, 9 up, 9 in 2023-07-18T03:03:06.782 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:06 smithi155 ceph-mon[128885]: from='client.14337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:03:06.783 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:06 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:03:06.783 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:06 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:03:06.783 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:06 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:03:07.676 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:07 smithi101 ceph-mon[126379]: osdmap e62: 9 total, 9 up, 9 in 2023-07-18T03:03:07.981 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:07 smithi101 ceph-mon[126379]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-07-18T03:03:07.982 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:07 smithi155 ceph-mon[128885]: osdmap e62: 9 total, 9 up, 9 in 2023-07-18T03:03:07.982 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:07 smithi155 ceph-mon[128885]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-07-18T03:03:07.982 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:07 smithi179 ceph-mon[129229]: osdmap e62: 9 total, 9 up, 9 in 2023-07-18T03:03:07.983 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:07 smithi179 ceph-mon[129229]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-07-18T03:03:09.687 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:09 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/4134644221' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "465a7ffb-67e4-49ef-812c-1b96feacfebf"}]: dispatch 2023-07-18T03:03:09.688 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:09 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/4134644221' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "465a7ffb-67e4-49ef-812c-1b96feacfebf"}]': finished 2023-07-18T03:03:09.688 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:09 smithi179 ceph-mon[129229]: osdmap e63: 10 total, 9 up, 10 in 2023-07-18T03:03:09.688 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:09 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-18T03:03:09.725 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:09 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/4134644221' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "465a7ffb-67e4-49ef-812c-1b96feacfebf"}]: dispatch 2023-07-18T03:03:09.725 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:09 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/4134644221' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "465a7ffb-67e4-49ef-812c-1b96feacfebf"}]': finished 2023-07-18T03:03:09.726 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:09 smithi101 ceph-mon[126379]: osdmap e63: 10 total, 9 up, 10 in 2023-07-18T03:03:09.726 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:09 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-18T03:03:09.782 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:09 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/4134644221' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "465a7ffb-67e4-49ef-812c-1b96feacfebf"}]: dispatch 2023-07-18T03:03:09.783 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:09 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/4134644221' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "465a7ffb-67e4-49ef-812c-1b96feacfebf"}]': finished 2023-07-18T03:03:09.783 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:09 smithi155 ceph-mon[128885]: osdmap e63: 10 total, 9 up, 10 in 2023-07-18T03:03:09.783 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:09 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-18T03:03:10.687 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:10 smithi179 ceph-mon[129229]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 130 KiB/s, 0 objects/s recovering 2023-07-18T03:03:10.688 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:10 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/3650013479' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:03:10.725 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:10 smithi101 ceph-mon[126379]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 130 KiB/s, 0 objects/s recovering 2023-07-18T03:03:10.725 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:10 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3650013479' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:03:10.782 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:10 smithi155 ceph-mon[128885]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 130 KiB/s, 0 objects/s recovering 2023-07-18T03:03:10.783 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:10 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/3650013479' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:03:11.687 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:11 smithi179 ceph-mon[129229]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-18T03:03:11.725 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:11 smithi101 ceph-mon[126379]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-18T03:03:11.782 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:11 smithi155 ceph-mon[128885]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-18T03:03:14.475 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:14 smithi101 ceph-mon[126379]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-07-18T03:03:14.687 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:14 smithi179 ceph-mon[129229]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-07-18T03:03:14.782 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:14 smithi155 ceph-mon[128885]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-07-18T03:03:15.688 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:15 smithi179 ceph-mon[129229]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-07-18T03:03:15.725 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:15 smithi101 ceph-mon[126379]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-07-18T03:03:15.782 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:15 smithi155 ceph-mon[128885]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-07-18T03:03:16.688 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:16 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-07-18T03:03:16.688 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:16 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:03:16.688 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:16 smithi179 ceph-mon[129229]: Deploying daemon osd.9 on smithi101 2023-07-18T03:03:16.725 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:16 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-07-18T03:03:16.725 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:16 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:03:16.726 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:16 smithi101 ceph-mon[126379]: Deploying daemon osd.9 on smithi101 2023-07-18T03:03:16.783 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:16 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-07-18T03:03:16.783 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:16 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:03:16.783 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:16 smithi155 ceph-mon[128885]: Deploying daemon osd.9 on smithi101 2023-07-18T03:03:17.569 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:17 smithi101 ceph-mon[126379]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-18T03:03:17.687 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:17 smithi179 ceph-mon[129229]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-18T03:03:17.782 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:17 smithi155 ceph-mon[128885]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-18T03:03:19.585 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:19 smithi101 ceph-mon[126379]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-18T03:03:19.688 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:19 smithi179 ceph-mon[129229]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-18T03:03:19.783 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:19 smithi155 ceph-mon[128885]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-18T03:03:22.188 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:21 smithi179 ceph-mon[129229]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-18T03:03:22.226 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:21 smithi101 ceph-mon[126379]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-18T03:03:22.283 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:21 smithi155 ceph-mon[128885]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-18T03:03:24.164 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:23 smithi101 ceph-mon[126379]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-18T03:03:24.165 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:23 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:24.165 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:23 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:24.165 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:23 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:03:24.188 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:23 smithi179 ceph-mon[129229]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-18T03:03:24.188 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:23 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:24.188 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:23 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:24.188 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:23 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:03:24.283 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:23 smithi155 ceph-mon[128885]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-18T03:03:24.283 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:23 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:24.283 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:23 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:24.283 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:23 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:03:26.161 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:25 smithi101 ceph-mon[126379]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-18T03:03:26.188 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:25 smithi179 ceph-mon[129229]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-18T03:03:26.283 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:25 smithi155 ceph-mon[128885]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-18T03:03:26.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:26.701+0000 7f021cff9700 1 -- 172.21.15.101:0/3884492677 <== mgr.14150 v2:172.21.15.101:6800/3963742842 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0220060950 con 0x7f02080a1150 2023-07-18T03:03:26.684 INFO:teuthology.orchestra.run.smithi101.stdout:Created osd(s) 9 on host 'smithi101' 2023-07-18T03:03:26.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:26.704+0000 7f0226089700 1 -- 172.21.15.101:0/3884492677 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f02080a1150 msgr2=0x7f02080a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:03:26.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:26.704+0000 7f0226089700 1 --2- 172.21.15.101:0/3884492677 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f02080a1150 0x7f02080a3600 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0210005950 tx=0x7f0210009260 comp rx=0 tx=0).stop 2023-07-18T03:03:26.690 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:26.704+0000 7f0226089700 1 -- 172.21.15.101:0/3884492677 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0220100d90 msgr2=0x7f0220072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:03:26.690 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:26.704+0000 7f0226089700 1 --2- 172.21.15.101:0/3884492677 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0220100d90 0x7f0220072900 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f021400b680 tx=0x7f0214005740 comp rx=0 tx=0).stop 2023-07-18T03:03:26.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:26.704+0000 7f0226089700 1 -- 172.21.15.101:0/3884492677 shutdown_connections 2023-07-18T03:03:26.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:26.704+0000 7f0226089700 1 --2- 172.21.15.101:0/3884492677 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f02080a1150 0x7f02080a3600 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:03:26.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:26.704+0000 7f0226089700 1 --2- 172.21.15.101:0/3884492677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02200ffb90 0x7f02200742b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:03:26.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:26.704+0000 7f0226089700 1 --2- 172.21.15.101:0/3884492677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0220101720 0x7f0220072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:03:26.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:26.704+0000 7f0226089700 1 --2- 172.21.15.101:0/3884492677 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0220100d90 0x7f0220072900 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:03:26.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:26.704+0000 7f0226089700 1 -- 172.21.15.101:0/3884492677 >> 172.21.15.101:0/3884492677 conn(0x7f02200fb1a0 msgr2=0x7f02201042c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:03:26.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:26.704+0000 7f0226089700 1 -- 172.21.15.101:0/3884492677 shutdown_connections 2023-07-18T03:03:26.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:03:26.705+0000 7f0226089700 1 -- 172.21.15.101:0/3884492677 wait complete. 2023-07-18T03:03:27.189 DEBUG:teuthology.orchestra.run.smithi101:osd.9> sudo journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@osd.9.service 2023-07-18T03:03:27.194 INFO:tasks.cephadm:Deploying osd.10 on smithi155 with /dev/vg_nvme/lv_1... 2023-07-18T03:03:27.195 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-07-18T03:03:27.271 INFO:journalctl@ceph.osd.9.smithi101.stdout:-- Logs begin at Tue 2023-07-18 02:35:16 UTC. -- 2023-07-18T03:03:27.533 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:27 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:27.533 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:27 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:27.534 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:27 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:03:27.534 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:27 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:03:27.534 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:27 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:27.534 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:27 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:27.534 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:27 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:27.534 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:27 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:03:27.534 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:27 smithi155 ceph-mon[128885]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-18T03:03:27.535 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:27 smithi155 ceph-mon[128885]: from='osd.9 [v2:172.21.15.101:6826/2166787667,v1:172.21.15.101:6827/2166787667]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-18T03:03:27.725 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:27 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:27.726 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:27 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:27.726 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:27 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:03:27.726 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:27 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:03:27.726 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:27 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:27.726 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:27 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:27.726 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:27 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:27.727 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:27 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:03:27.727 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:27 smithi101 ceph-mon[126379]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-18T03:03:27.727 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:27 smithi101 ceph-mon[126379]: from='osd.9 [v2:172.21.15.101:6826/2166787667,v1:172.21.15.101:6827/2166787667]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-18T03:03:27.758 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.b/config 2023-07-18T03:03:27.938 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:27 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:27.938 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:27 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:27.939 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:27 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:03:27.939 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:27 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:03:27.939 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:27 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:27.939 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:27 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:27.939 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:27 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:27.939 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:27 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:03:27.939 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:27 smithi179 ceph-mon[129229]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-18T03:03:27.940 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:27 smithi179 ceph-mon[129229]: from='osd.9 [v2:172.21.15.101:6826/2166787667,v1:172.21.15.101:6827/2166787667]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-18T03:03:28.876 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:28 smithi101 ceph-mon[126379]: from='osd.9 [v2:172.21.15.101:6826/2166787667,v1:172.21.15.101:6827/2166787667]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-07-18T03:03:28.876 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:28 smithi101 ceph-mon[126379]: osdmap e64: 10 total, 9 up, 10 in 2023-07-18T03:03:28.877 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:28 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-18T03:03:28.877 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:28 smithi101 ceph-mon[126379]: from='osd.9 [v2:172.21.15.101:6826/2166787667,v1:172.21.15.101:6827/2166787667]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-07-18T03:03:28.877 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:03:28 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:03:28.669+0000 7f3fcd91c700 -1 osd.9 0 waiting for initial osdmap 2023-07-18T03:03:28.877 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:03:28 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:03:28.686+0000 7f3fc770d700 -1 osd.9 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-18T03:03:28.938 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:28 smithi179 ceph-mon[129229]: from='osd.9 [v2:172.21.15.101:6826/2166787667,v1:172.21.15.101:6827/2166787667]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-07-18T03:03:28.938 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:28 smithi179 ceph-mon[129229]: osdmap e64: 10 total, 9 up, 10 in 2023-07-18T03:03:28.939 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:28 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-18T03:03:28.939 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:28 smithi179 ceph-mon[129229]: from='osd.9 [v2:172.21.15.101:6826/2166787667,v1:172.21.15.101:6827/2166787667]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-07-18T03:03:29.033 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:28 smithi155 ceph-mon[128885]: from='osd.9 [v2:172.21.15.101:6826/2166787667,v1:172.21.15.101:6827/2166787667]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-07-18T03:03:29.034 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:28 smithi155 ceph-mon[128885]: osdmap e64: 10 total, 9 up, 10 in 2023-07-18T03:03:29.034 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:28 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-18T03:03:29.034 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:28 smithi155 ceph-mon[128885]: from='osd.9 [v2:172.21.15.101:6826/2166787667,v1:172.21.15.101:6827/2166787667]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-07-18T03:03:29.263 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-07-18T03:03:29.263 INFO:teuthology.orchestra.run.smithi155.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-07-18T03:03:29.263 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10+0 records in 2023-07-18T03:03:29.264 INFO:teuthology.orchestra.run.smithi155.stderr:10+0 records out 2023-07-18T03:03:29.264 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.015221 s, 689 MB/s 2023-07-18T03:03:29.264 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping successful for: 2023-07-18T03:03:29.613 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi155:vg_nvme/lv_1 2023-07-18T03:03:29.938 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:29 smithi179 ceph-mon[129229]: from='osd.9 [v2:172.21.15.101:6826/2166787667,v1:172.21.15.101:6827/2166787667]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]': finished 2023-07-18T03:03:29.938 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:29 smithi179 ceph-mon[129229]: osdmap e65: 10 total, 9 up, 10 in 2023-07-18T03:03:29.938 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:29 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-18T03:03:29.939 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:29 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-18T03:03:29.939 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:29 smithi179 ceph-mon[129229]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-18T03:03:29.976 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:29 smithi101 ceph-mon[126379]: from='osd.9 [v2:172.21.15.101:6826/2166787667,v1:172.21.15.101:6827/2166787667]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]': finished 2023-07-18T03:03:29.976 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:29 smithi101 ceph-mon[126379]: osdmap e65: 10 total, 9 up, 10 in 2023-07-18T03:03:29.976 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:29 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-18T03:03:29.976 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:29 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-18T03:03:29.977 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:29 smithi101 ceph-mon[126379]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-18T03:03:30.033 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:29 smithi155 ceph-mon[128885]: from='osd.9 [v2:172.21.15.101:6826/2166787667,v1:172.21.15.101:6827/2166787667]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]': finished 2023-07-18T03:03:30.033 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:29 smithi155 ceph-mon[128885]: osdmap e65: 10 total, 9 up, 10 in 2023-07-18T03:03:30.033 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:29 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-18T03:03:30.033 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:29 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-18T03:03:30.034 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:29 smithi155 ceph-mon[128885]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-18T03:03:30.170 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.b/config 2023-07-18T03:03:30.938 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:30 smithi179 ceph-mon[129229]: purged_snaps scrub starts 2023-07-18T03:03:30.938 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:30 smithi179 ceph-mon[129229]: purged_snaps scrub ok 2023-07-18T03:03:30.939 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:30 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-18T03:03:30.939 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:30 smithi179 ceph-mon[129229]: osd.9 [v2:172.21.15.101:6826/2166787667,v1:172.21.15.101:6827/2166787667] boot 2023-07-18T03:03:30.939 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:30 smithi179 ceph-mon[129229]: osdmap e66: 10 total, 10 up, 10 in 2023-07-18T03:03:30.939 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:30 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-18T03:03:30.939 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:30 smithi179 ceph-mon[129229]: Detected new or changed devices on smithi101 2023-07-18T03:03:30.940 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:30 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:30.940 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:30 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:30.940 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:30 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:03:30.940 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:30 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:03:30.940 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:30 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:03:30.940 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:30 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:03:30.941 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:30 smithi179 ceph-mon[129229]: Adjusting osd_memory_target on smithi101 to 4316M 2023-07-18T03:03:30.941 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:30 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:30.941 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:30 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:03:30.941 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:30 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:03:30.941 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:30 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:30.976 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:30 smithi101 ceph-mon[126379]: purged_snaps scrub starts 2023-07-18T03:03:30.976 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:30 smithi101 ceph-mon[126379]: purged_snaps scrub ok 2023-07-18T03:03:30.977 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:30 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-18T03:03:30.977 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:30 smithi101 ceph-mon[126379]: osd.9 [v2:172.21.15.101:6826/2166787667,v1:172.21.15.101:6827/2166787667] boot 2023-07-18T03:03:30.977 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:30 smithi101 ceph-mon[126379]: osdmap e66: 10 total, 10 up, 10 in 2023-07-18T03:03:30.977 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:30 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-18T03:03:30.978 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:30 smithi101 ceph-mon[126379]: Detected new or changed devices on smithi101 2023-07-18T03:03:30.978 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:30 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:30.978 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:30 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:30.978 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:30 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:03:30.978 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:30 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:03:30.978 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:30 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:03:30.979 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:30 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:03:30.979 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:30 smithi101 ceph-mon[126379]: Adjusting osd_memory_target on smithi101 to 4316M 2023-07-18T03:03:30.979 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:30 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:30.979 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:30 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:03:30.980 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:30 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:03:30.980 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:30 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:31.033 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:30 smithi155 ceph-mon[128885]: purged_snaps scrub starts 2023-07-18T03:03:31.033 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:30 smithi155 ceph-mon[128885]: purged_snaps scrub ok 2023-07-18T03:03:31.034 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:30 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-18T03:03:31.034 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:30 smithi155 ceph-mon[128885]: osd.9 [v2:172.21.15.101:6826/2166787667,v1:172.21.15.101:6827/2166787667] boot 2023-07-18T03:03:31.034 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:30 smithi155 ceph-mon[128885]: osdmap e66: 10 total, 10 up, 10 in 2023-07-18T03:03:31.034 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:30 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-18T03:03:31.034 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:30 smithi155 ceph-mon[128885]: Detected new or changed devices on smithi101 2023-07-18T03:03:31.034 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:30 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:31.034 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:30 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:31.035 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:30 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:03:31.035 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:30 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:03:31.035 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:30 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:03:31.035 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:30 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:03:31.035 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:30 smithi155 ceph-mon[128885]: Adjusting osd_memory_target on smithi101 to 4316M 2023-07-18T03:03:31.035 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:30 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:31.035 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:30 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:03:31.036 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:30 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:03:31.036 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:30 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:31.357 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:31.376+0000 7f3165483700 1 -- 172.21.15.155:0/2954350331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31600ffc80 msgr2=0x7f31601000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:03:31.357 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:31.376+0000 7f3165483700 1 --2- 172.21.15.155:0/2954350331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31600ffc80 0x7f31601000d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3150009990 tx=0x7f3150009ca0 comp rx=0 tx=0).stop 2023-07-18T03:03:31.357 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:31.376+0000 7f3165483700 1 -- 172.21.15.155:0/2954350331 shutdown_connections 2023-07-18T03:03:31.358 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:31.376+0000 7f3165483700 1 --2- 172.21.15.155:0/2954350331 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3160100610 0x7f3160106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:03:31.358 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:31.376+0000 7f3165483700 1 --2- 172.21.15.155:0/2954350331 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f31600fea80 0x7f31600fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:03:31.358 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:31.376+0000 7f3165483700 1 --2- 172.21.15.155:0/2954350331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31600ffc80 0x7f31601000d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:03:31.358 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:31.376+0000 7f3165483700 1 -- 172.21.15.155:0/2954350331 >> 172.21.15.155:0/2954350331 conn(0x7f31600fa0b0 msgr2=0x7f31600fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:03:31.359 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:31.376+0000 7f3165483700 1 -- 172.21.15.155:0/2954350331 shutdown_connections 2023-07-18T03:03:31.359 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:31.376+0000 7f3165483700 1 -- 172.21.15.155:0/2954350331 wait complete. 2023-07-18T03:03:31.364 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:31.376+0000 7f3165483700 1 Processor -- start 2023-07-18T03:03:31.364 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:31.377+0000 7f3165483700 1 -- start start 2023-07-18T03:03:31.364 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:31.377+0000 7f3165483700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31600fea80 0x7f3160195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:03:31.365 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:31.377+0000 7f3165483700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f31600ffc80 0x7f3160195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:03:31.365 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:31.377+0000 7f3165483700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3160100610 0x7f316019cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:03:31.366 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:31.377+0000 7f3165483700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316019d500 con 0x7f3160100610 2023-07-18T03:03:31.366 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:31.377+0000 7f3165483700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316019d640 con 0x7f31600fea80 2023-07-18T03:03:31.366 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:31.377+0000 7f3165483700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316019d7d0 con 0x7f31600ffc80 2023-07-18T03:03:31.367 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:31.377+0000 7f315effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31600fea80 0x7f3160195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:03:31.367 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:31.377+0000 7f315effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31600fea80 0x7f3160195970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:49918/0 (socket says 172.21.15.155:49918) 2023-07-18T03:03:31.368 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:31.377+0000 7f315effd700 1 -- 172.21.15.155:0/2574030017 learned_addr learned my addr 172.21.15.155:0/2574030017 (peer_addr_for_me v2:172.21.15.155:0/0) 2023-07-18T03:03:31.368 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:31.377+0000 7f315e7fc700 1 --2- 172.21.15.155:0/2574030017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f31600ffc80 0x7f3160195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:03:31.368 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:31.377+0000 7f315f7fe700 1 --2- 172.21.15.155:0/2574030017 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3160100610 0x7f316019cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:03:31.368 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:31.377+0000 7f315effd700 1 -- 172.21.15.155:0/2574030017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f31600ffc80 msgr2=0x7f3160195eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:03:31.369 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:31.377+0000 7f315effd700 1 --2- 172.21.15.155:0/2574030017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f31600ffc80 0x7f3160195eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:03:31.369 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:31.377+0000 7f315effd700 1 -- 172.21.15.155:0/2574030017 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3160100610 msgr2=0x7f316019cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:03:31.369 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:31.377+0000 7f315effd700 1 --2- 172.21.15.155:0/2574030017 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3160100610 0x7f316019cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:03:31.369 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:31.377+0000 7f315effd700 1 -- 172.21.15.155:0/2574030017 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31500095f0 con 0x7f31600fea80 2023-07-18T03:03:31.369 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:31.377+0000 7f315effd700 1 --2- 172.21.15.155:0/2574030017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31600fea80 0x7f3160195970 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f315400b500 tx=0x7f315400b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:03:31.369 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:31.377+0000 7f3147fff700 1 -- 172.21.15.155:0/2574030017 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3154011830 con 0x7f31600fea80 2023-07-18T03:03:31.370 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:31.377+0000 7f3147fff700 1 -- 172.21.15.155:0/2574030017 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3154011e50 con 0x7f31600fea80 2023-07-18T03:03:31.370 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:31.377+0000 7f3165483700 1 -- 172.21.15.155:0/2574030017 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f316019dab0 con 0x7f31600fea80 2023-07-18T03:03:31.370 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:31.377+0000 7f3147fff700 1 -- 172.21.15.155:0/2574030017 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315400f2e0 con 0x7f31600fea80 2023-07-18T03:03:31.370 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:31.377+0000 7f3165483700 1 -- 172.21.15.155:0/2574030017 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f316019e000 con 0x7f31600fea80 2023-07-18T03:03:31.370 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:31.379+0000 7f3165483700 1 -- 172.21.15.155:0/2574030017 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f316010a2e0 con 0x7f31600fea80 2023-07-18T03:03:31.371 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:31.379+0000 7f3147fff700 1 -- 172.21.15.155:0/2574030017 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f3154010580 con 0x7f31600fea80 2023-07-18T03:03:31.371 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:31.380+0000 7f3147fff700 1 --2- 172.21.15.155:0/2574030017 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f31480a11d0 0x7f31480a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:03:31.371 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:31.380+0000 7f3147fff700 1 -- 172.21.15.155:0/2574030017 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f31540cbab0 con 0x7f31600fea80 2023-07-18T03:03:31.371 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:31.380+0000 7f315e7fc700 1 --2- 172.21.15.155:0/2574030017 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f31480a11d0 0x7f31480a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:03:31.371 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:31.381+0000 7f315e7fc700 1 --2- 172.21.15.155:0/2574030017 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f31480a11d0 0x7f31480a3680 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f315000b5c0 tx=0x7f3150005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:03:31.371 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:31.384+0000 7f3147fff700 1 -- 172.21.15.155:0/2574030017 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3154095a00 con 0x7f31600fea80 2023-07-18T03:03:31.558 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:31.577+0000 7f3165483700 1 -- 172.21.15.155:0/2574030017 --> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f3160060950 con 0x7f31480a11d0 2023-07-18T03:03:31.938 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:31 smithi179 ceph-mon[129229]: osdmap e67: 10 total, 10 up, 10 in 2023-07-18T03:03:31.938 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:31 smithi179 ceph-mon[129229]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:03:31.939 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:31 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:03:31.939 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:31 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:03:31.939 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:31 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:03:31.975 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:31 smithi101 ceph-mon[126379]: osdmap e67: 10 total, 10 up, 10 in 2023-07-18T03:03:31.976 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:31 smithi101 ceph-mon[126379]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:03:31.976 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:31 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:03:31.976 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:31 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:03:31.976 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:31 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:03:32.033 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:31 smithi155 ceph-mon[128885]: osdmap e67: 10 total, 10 up, 10 in 2023-07-18T03:03:32.033 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:31 smithi155 ceph-mon[128885]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:03:32.033 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:31 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:03:32.033 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:31 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:03:32.034 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:31 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:03:32.938 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:32 smithi179 ceph-mon[129229]: from='client.24368 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:03:32.975 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:32 smithi101 ceph-mon[126379]: from='client.24368 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:03:33.033 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:32 smithi155 ceph-mon[128885]: from='client.24368 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:03:33.938 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:33 smithi179 ceph-mon[129229]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:03:33.978 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:33 smithi101 ceph-mon[126379]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:03:34.033 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:33 smithi155 ceph-mon[128885]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:03:34.938 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:34 smithi179 ceph-mon[129229]: from='client.? 172.21.15.155:0/2843652522' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1897a027-f25d-434f-8df5-5fc2bd866578"}]: dispatch 2023-07-18T03:03:34.939 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:34 smithi179 ceph-mon[129229]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1897a027-f25d-434f-8df5-5fc2bd866578"}]: dispatch 2023-07-18T03:03:34.939 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:34 smithi179 ceph-mon[129229]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1897a027-f25d-434f-8df5-5fc2bd866578"}]': finished 2023-07-18T03:03:34.939 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:34 smithi179 ceph-mon[129229]: osdmap e68: 11 total, 10 up, 11 in 2023-07-18T03:03:34.939 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:34 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-18T03:03:34.939 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:34 smithi179 ceph-mon[129229]: from='client.? 172.21.15.155:0/3779189303' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:03:34.976 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:34 smithi101 ceph-mon[126379]: from='client.? 172.21.15.155:0/2843652522' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1897a027-f25d-434f-8df5-5fc2bd866578"}]: dispatch 2023-07-18T03:03:34.976 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:34 smithi101 ceph-mon[126379]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1897a027-f25d-434f-8df5-5fc2bd866578"}]: dispatch 2023-07-18T03:03:34.976 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:34 smithi101 ceph-mon[126379]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1897a027-f25d-434f-8df5-5fc2bd866578"}]': finished 2023-07-18T03:03:34.976 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:34 smithi101 ceph-mon[126379]: osdmap e68: 11 total, 10 up, 11 in 2023-07-18T03:03:34.977 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:34 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-18T03:03:34.977 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:34 smithi101 ceph-mon[126379]: from='client.? 172.21.15.155:0/3779189303' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:03:35.033 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:34 smithi155 ceph-mon[128885]: from='client.? 172.21.15.155:0/2843652522' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1897a027-f25d-434f-8df5-5fc2bd866578"}]: dispatch 2023-07-18T03:03:35.033 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:34 smithi155 ceph-mon[128885]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1897a027-f25d-434f-8df5-5fc2bd866578"}]: dispatch 2023-07-18T03:03:35.034 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:34 smithi155 ceph-mon[128885]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1897a027-f25d-434f-8df5-5fc2bd866578"}]': finished 2023-07-18T03:03:35.034 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:34 smithi155 ceph-mon[128885]: osdmap e68: 11 total, 10 up, 11 in 2023-07-18T03:03:35.034 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:34 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-18T03:03:35.034 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:34 smithi155 ceph-mon[128885]: from='client.? 172.21.15.155:0/3779189303' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:03:35.976 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:35 smithi101 ceph-mon[126379]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:03:36.033 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:35 smithi155 ceph-mon[128885]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:03:36.188 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:35 smithi179 ceph-mon[129229]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:03:38.188 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:37 smithi179 ceph-mon[129229]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:03:38.226 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:37 smithi101 ceph-mon[126379]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:03:38.284 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:37 smithi155 ceph-mon[128885]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:03:39.688 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:39 smithi179 ceph-mon[129229]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:03:39.726 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:39 smithi101 ceph-mon[126379]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:03:39.783 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:39 smithi155 ceph-mon[128885]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:03:40.533 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:40 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-07-18T03:03:40.534 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:40 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:03:40.534 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:40 smithi155 ceph-mon[128885]: Deploying daemon osd.10 on smithi155 2023-07-18T03:03:40.689 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:40 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-07-18T03:03:40.689 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:40 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:03:40.689 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:40 smithi179 ceph-mon[129229]: Deploying daemon osd.10 on smithi155 2023-07-18T03:03:40.726 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:40 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-07-18T03:03:40.727 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:40 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:03:40.727 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:40 smithi101 ceph-mon[126379]: Deploying daemon osd.10 on smithi155 2023-07-18T03:03:41.533 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:41 smithi155 ceph-mon[128885]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:03:41.688 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:41 smithi179 ceph-mon[129229]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:03:41.726 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:41 smithi101 ceph-mon[126379]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:03:44.158 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:43 smithi155 ceph-mon[128885]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:03:44.189 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:43 smithi179 ceph-mon[129229]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:03:44.226 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:43 smithi101 ceph-mon[126379]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:03:45.534 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:45 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:45.534 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:45 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:45.535 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:45 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:03:45.535 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:45 smithi155 ceph-mon[128885]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:03:45.688 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:45 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:45.689 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:45 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:45.689 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:45 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:03:45.689 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:45 smithi179 ceph-mon[129229]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:03:45.726 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:45 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:45.727 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:45 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:45.727 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:45 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:03:45.727 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:45 smithi101 ceph-mon[126379]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:03:47.071 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 10 on host 'smithi155' 2023-07-18T03:03:47.071 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:47.088+0000 7f3147fff700 1 -- 172.21.15.155:0/2574030017 <== mgr.14150 v2:172.21.15.101:6800/3963742842 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f3160060950 con 0x7f31480a11d0 2023-07-18T03:03:47.073 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:47.090+0000 7f3165483700 1 -- 172.21.15.155:0/2574030017 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f31480a11d0 msgr2=0x7f31480a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:03:47.073 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:47.090+0000 7f3165483700 1 --2- 172.21.15.155:0/2574030017 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f31480a11d0 0x7f31480a3680 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f315000b5c0 tx=0x7f3150005fd0 comp rx=0 tx=0).stop 2023-07-18T03:03:47.073 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:47.090+0000 7f3165483700 1 -- 172.21.15.155:0/2574030017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31600fea80 msgr2=0x7f3160195970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:03:47.073 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:47.090+0000 7f3165483700 1 --2- 172.21.15.155:0/2574030017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31600fea80 0x7f3160195970 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f315400b500 tx=0x7f315400b810 comp rx=0 tx=0).stop 2023-07-18T03:03:47.074 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:47.090+0000 7f3165483700 1 -- 172.21.15.155:0/2574030017 shutdown_connections 2023-07-18T03:03:47.074 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:47.090+0000 7f3165483700 1 --2- 172.21.15.155:0/2574030017 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f31480a11d0 0x7f31480a3680 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:03:47.074 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:47.090+0000 7f3165483700 1 --2- 172.21.15.155:0/2574030017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31600fea80 0x7f3160195970 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:03:47.074 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:47.090+0000 7f3165483700 1 --2- 172.21.15.155:0/2574030017 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3160100610 0x7f316019cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:03:47.074 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:47.090+0000 7f3165483700 1 --2- 172.21.15.155:0/2574030017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f31600ffc80 0x7f3160195eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:03:47.074 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:47.090+0000 7f3165483700 1 -- 172.21.15.155:0/2574030017 >> 172.21.15.155:0/2574030017 conn(0x7f31600fa0b0 msgr2=0x7f31600fc380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:03:47.074 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:47.091+0000 7f3165483700 1 -- 172.21.15.155:0/2574030017 shutdown_connections 2023-07-18T03:03:47.075 INFO:teuthology.orchestra.run.smithi155.stderr:2023-07-18T03:03:47.091+0000 7f3165483700 1 -- 172.21.15.155:0/2574030017 wait complete. 2023-07-18T03:03:47.457 DEBUG:teuthology.orchestra.run.smithi155:osd.10> sudo journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@osd.10.service 2023-07-18T03:03:47.460 INFO:tasks.cephadm:Deploying osd.11 on smithi179 with /dev/vg_nvme/lv_1... 2023-07-18T03:03:47.460 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-07-18T03:03:47.489 INFO:journalctl@ceph.osd.10.smithi155.stdout:-- Logs begin at Tue 2023-07-18 02:29:36 UTC. -- 2023-07-18T03:03:48.021 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.c/config 2023-07-18T03:03:48.187 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:03:48 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:03:48.154+0000 7fa5ac7ee640 -1 osd.10 0 log_to_monitors true 2023-07-18T03:03:48.187 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:47 smithi155 ceph-mon[128885]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:03:48.188 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:47 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:48.188 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:47 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:48.188 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:47 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:03:48.189 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:47 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:03:48.189 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:47 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:48.189 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:47 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:48.189 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:47 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:48.190 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:47 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:03:48.190 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:47 smithi179 ceph-mon[129229]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:03:48.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:47 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:48.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:47 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:48.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:47 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:03:48.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:47 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:03:48.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:47 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:48.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:47 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:48.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:47 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:48.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:47 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:03:48.226 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:47 smithi101 ceph-mon[126379]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:03:48.226 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:47 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:48.226 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:47 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:48.227 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:47 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:03:48.227 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:47 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:03:48.227 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:47 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:48.227 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:47 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:48.227 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:47 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:48.227 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:47 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:03:49.189 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:48 smithi179 ceph-mon[129229]: from='osd.10 [v2:172.21.15.155:6824/3682446349,v1:172.21.15.155:6825/3682446349]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-18T03:03:49.190 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:48 smithi179 ceph-mon[129229]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-18T03:03:49.226 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:48 smithi101 ceph-mon[126379]: from='osd.10 [v2:172.21.15.155:6824/3682446349,v1:172.21.15.155:6825/3682446349]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-18T03:03:49.227 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:48 smithi101 ceph-mon[126379]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-18T03:03:49.283 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:48 smithi155 ceph-mon[128885]: from='osd.10 [v2:172.21.15.155:6824/3682446349,v1:172.21.15.155:6825/3682446349]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-18T03:03:49.284 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:48 smithi155 ceph-mon[128885]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-18T03:03:49.514 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-07-18T03:03:49.514 INFO:teuthology.orchestra.run.smithi179.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-07-18T03:03:49.515 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10+0 records in 2023-07-18T03:03:49.515 INFO:teuthology.orchestra.run.smithi179.stderr:10+0 records out 2023-07-18T03:03:49.515 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0150898 s, 695 MB/s 2023-07-18T03:03:49.515 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping successful for: 2023-07-18T03:03:49.867 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi179:vg_nvme/lv_1 2023-07-18T03:03:50.189 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:49 smithi179 ceph-mon[129229]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:03:50.189 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:49 smithi179 ceph-mon[129229]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-07-18T03:03:50.189 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:49 smithi179 ceph-mon[129229]: osdmap e69: 11 total, 10 up, 11 in 2023-07-18T03:03:50.190 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:49 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-18T03:03:50.190 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:49 smithi179 ceph-mon[129229]: from='osd.10 [v2:172.21.15.155:6824/3682446349,v1:172.21.15.155:6825/3682446349]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-18T03:03:50.190 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:49 smithi179 ceph-mon[129229]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-18T03:03:50.190 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:49 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:50.190 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:49 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:50.190 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:49 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:03:50.190 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:49 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:03:50.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:49 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:03:50.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:49 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:03:50.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:49 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:50.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:49 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:03:50.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:49 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:03:50.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:49 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:50.226 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:49 smithi101 ceph-mon[126379]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:03:50.227 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:49 smithi101 ceph-mon[126379]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-07-18T03:03:50.227 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:49 smithi101 ceph-mon[126379]: osdmap e69: 11 total, 10 up, 11 in 2023-07-18T03:03:50.227 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:49 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-18T03:03:50.227 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:49 smithi101 ceph-mon[126379]: from='osd.10 [v2:172.21.15.155:6824/3682446349,v1:172.21.15.155:6825/3682446349]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-18T03:03:50.227 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:49 smithi101 ceph-mon[126379]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-18T03:03:50.227 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:49 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:50.227 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:49 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:50.228 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:49 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:03:50.228 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:49 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:03:50.228 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:49 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:03:50.228 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:49 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:03:50.228 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:49 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:50.228 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:49 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:03:50.228 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:49 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:03:50.228 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:49 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:50.283 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:03:49 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:03:49.936+0000 7fa5a1e66700 -1 osd.10 0 waiting for initial osdmap 2023-07-18T03:03:50.284 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:03:49 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:03:49.947+0000 7fa59bc57700 -1 osd.10 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-18T03:03:50.284 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:49 smithi155 ceph-mon[128885]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:03:50.284 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:49 smithi155 ceph-mon[128885]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-07-18T03:03:50.284 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:49 smithi155 ceph-mon[128885]: osdmap e69: 11 total, 10 up, 11 in 2023-07-18T03:03:50.285 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:49 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-18T03:03:50.285 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:49 smithi155 ceph-mon[128885]: from='osd.10 [v2:172.21.15.155:6824/3682446349,v1:172.21.15.155:6825/3682446349]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-18T03:03:50.285 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:49 smithi155 ceph-mon[128885]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2023-07-18T03:03:50.285 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:49 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:50.285 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:49 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:50.285 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:49 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:03:50.285 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:49 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:03:50.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:49 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:03:50.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:49 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:03:50.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:49 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:50.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:49 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:03:50.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:49 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:03:50.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:49 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:03:50.424 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.c/config 2023-07-18T03:03:51.165 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:50 smithi179 ceph-mon[129229]: Detected new or changed devices on smithi155 2023-07-18T03:03:51.165 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:50 smithi179 ceph-mon[129229]: Adjusting osd_memory_target on smithi155 to 4292M 2023-07-18T03:03:51.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:50 smithi179 ceph-mon[129229]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2023-07-18T03:03:51.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:50 smithi179 ceph-mon[129229]: osdmap e70: 11 total, 10 up, 11 in 2023-07-18T03:03:51.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:50 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-18T03:03:51.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:50 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-18T03:03:51.226 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:50 smithi101 ceph-mon[126379]: Detected new or changed devices on smithi155 2023-07-18T03:03:51.226 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:50 smithi101 ceph-mon[126379]: Adjusting osd_memory_target on smithi155 to 4292M 2023-07-18T03:03:51.226 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:50 smithi101 ceph-mon[126379]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2023-07-18T03:03:51.227 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:50 smithi101 ceph-mon[126379]: osdmap e70: 11 total, 10 up, 11 in 2023-07-18T03:03:51.227 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:50 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-18T03:03:51.227 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:50 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-18T03:03:51.284 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:50 smithi155 ceph-mon[128885]: Detected new or changed devices on smithi155 2023-07-18T03:03:51.284 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:50 smithi155 ceph-mon[128885]: Adjusting osd_memory_target on smithi155 to 4292M 2023-07-18T03:03:51.284 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:50 smithi155 ceph-mon[128885]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2023-07-18T03:03:51.284 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:50 smithi155 ceph-mon[128885]: osdmap e70: 11 total, 10 up, 11 in 2023-07-18T03:03:51.284 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:50 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-18T03:03:51.284 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:50 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-18T03:03:51.589 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:03:51.607+0000 7f968857e700 1 -- 172.21.15.179:0/656620206 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f96800ff6b0 msgr2=0x7f9680106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:03:51.590 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:03:51.607+0000 7f968857e700 1 --2- 172.21.15.179:0/656620206 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f96800ff6b0 0x7f9680106f60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f967800b0a0 tx=0x7f967800b3b0 comp rx=0 tx=0).stop 2023-07-18T03:03:51.590 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:03:51.607+0000 7f968857e700 1 -- 172.21.15.179:0/656620206 shutdown_connections 2023-07-18T03:03:51.590 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:03:51.607+0000 7f968857e700 1 --2- 172.21.15.179:0/656620206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96801000a0 0x7f96801004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:03:51.590 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:03:51.607+0000 7f968857e700 1 --2- 172.21.15.179:0/656620206 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f96800ff6b0 0x7f9680106f60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:03:51.590 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:03:51.607+0000 7f968857e700 1 --2- 172.21.15.179:0/656620206 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f96800fed00 0x7f96800ff170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:03:51.590 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:03:51.607+0000 7f968857e700 1 -- 172.21.15.179:0/656620206 >> 172.21.15.179:0/656620206 conn(0x7f96800f9860 msgr2=0x7f96800fbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:03:51.590 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:03:51.607+0000 7f968857e700 1 -- 172.21.15.179:0/656620206 shutdown_connections 2023-07-18T03:03:51.591 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:03:51.607+0000 7f968857e700 1 -- 172.21.15.179:0/656620206 wait complete. 2023-07-18T03:03:51.592 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:03:51.607+0000 7f968857e700 1 Processor -- start 2023-07-18T03:03:51.592 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:03:51.608+0000 7f968857e700 1 -- start start 2023-07-18T03:03:51.593 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:03:51.608+0000 7f968857e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96800fed00 0x7f9680197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:03:51.593 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:03:51.608+0000 7f968857e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f96800ff6b0 0x7f96801980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:03:51.593 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:03:51.608+0000 7f968857e700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f96801000a0 0x7f968019f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:03:51.593 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:03:51.608+0000 7f968857e700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968019f700 con 0x7f96801000a0 2023-07-18T03:03:51.593 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:03:51.608+0000 7f968857e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968019f840 con 0x7f96800fed00 2023-07-18T03:03:51.593 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:03:51.608+0000 7f968857e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968019f9d0 con 0x7f96800ff6b0 2023-07-18T03:03:51.593 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:03:51.608+0000 7f9685b19700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f96800ff6b0 0x7f96801980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:03:51.594 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:03:51.608+0000 7f9685b19700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f96800ff6b0 0x7f96801980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48540/0 (socket says 172.21.15.179:48540) 2023-07-18T03:03:51.594 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:03:51.608+0000 7f9685b19700 1 -- 172.21.15.179:0/3203036079 learned_addr learned my addr 172.21.15.179:0/3203036079 (peer_addr_for_me v2:172.21.15.179:0/0) 2023-07-18T03:03:51.594 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:03:51.608+0000 7f968631a700 1 --2- 172.21.15.179:0/3203036079 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96800fed00 0x7f9680197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:03:51.594 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:03:51.608+0000 7f9685b19700 1 -- 172.21.15.179:0/3203036079 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96800fed00 msgr2=0x7f9680197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:03:51.594 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:03:51.608+0000 7f9685b19700 1 --2- 172.21.15.179:0/3203036079 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96800fed00 0x7f9680197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:03:51.594 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:03:51.608+0000 7f9685b19700 1 -- 172.21.15.179:0/3203036079 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f96801000a0 msgr2=0x7f968019f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:03:51.594 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:03:51.608+0000 7f9685b19700 1 --2- 172.21.15.179:0/3203036079 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f96801000a0 0x7f968019f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:03:51.595 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:03:51.608+0000 7f9685b19700 1 -- 172.21.15.179:0/3203036079 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9678009d00 con 0x7f96800ff6b0 2023-07-18T03:03:51.595 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:03:51.608+0000 7f9685b19700 1 --2- 172.21.15.179:0/3203036079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f96800ff6b0 0x7f96801980b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f967c00c950 tx=0x7f967c00cd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:03:51.595 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:03:51.608+0000 7f96777fe700 1 -- 172.21.15.179:0/3203036079 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c007ac0 con 0x7f96800ff6b0 2023-07-18T03:03:51.595 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:03:51.608+0000 7f968857e700 1 -- 172.21.15.179:0/3203036079 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f968019fcb0 con 0x7f96800ff6b0 2023-07-18T03:03:51.595 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:03:51.608+0000 7f96777fe700 1 -- 172.21.15.179:0/3203036079 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f967c004cf0 con 0x7f96800ff6b0 2023-07-18T03:03:51.595 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:03:51.608+0000 7f96777fe700 1 -- 172.21.15.179:0/3203036079 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c005580 con 0x7f96800ff6b0 2023-07-18T03:03:51.595 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:03:51.608+0000 7f968857e700 1 -- 172.21.15.179:0/3203036079 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96801a01b0 con 0x7f96800ff6b0 2023-07-18T03:03:51.596 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:03:51.609+0000 7f968857e700 1 -- 172.21.15.179:0/3203036079 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f968010a250 con 0x7f96800ff6b0 2023-07-18T03:03:51.596 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:03:51.611+0000 7f96777fe700 1 -- 172.21.15.179:0/3203036079 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f967c007c20 con 0x7f96800ff6b0 2023-07-18T03:03:51.597 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:03:51.612+0000 7f96777fe700 1 --2- 172.21.15.179:0/3203036079 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f966c0a1150 0x7f966c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:03:51.597 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:03:51.612+0000 7f96777fe700 1 -- 172.21.15.179:0/3203036079 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6727+0+0 (secure 0 0 0) 0x7f967c0cc300 con 0x7f96800ff6b0 2023-07-18T03:03:51.597 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:03:51.612+0000 7f968631a700 1 --2- 172.21.15.179:0/3203036079 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f966c0a1150 0x7f966c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:03:51.597 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:03:51.613+0000 7f968631a700 1 --2- 172.21.15.179:0/3203036079 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f966c0a1150 0x7f966c0a3600 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9680067db0 tx=0x7f9670005cf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:03:51.597 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:03:51.615+0000 7f96777fe700 1 -- 172.21.15.179:0/3203036079 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f967c096080 con 0x7f96800ff6b0 2023-07-18T03:03:51.791 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:03:51.809+0000 7f968857e700 1 -- 172.21.15.179:0/3203036079 --> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f96800fe3a0 con 0x7f966c0a1150 2023-07-18T03:03:52.189 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:51 smithi179 ceph-mon[129229]: purged_snaps scrub starts 2023-07-18T03:03:52.189 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:51 smithi179 ceph-mon[129229]: purged_snaps scrub ok 2023-07-18T03:03:52.190 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:51 smithi179 ceph-mon[129229]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:03:52.190 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:51 smithi179 ceph-mon[129229]: osd.10 [v2:172.21.15.155:6824/3682446349,v1:172.21.15.155:6825/3682446349] boot 2023-07-18T03:03:52.190 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:51 smithi179 ceph-mon[129229]: osdmap e71: 11 total, 11 up, 11 in 2023-07-18T03:03:52.190 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:51 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-18T03:03:52.190 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:51 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:03:52.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:51 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:03:52.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:51 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:03:52.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:51 smithi179 ceph-mon[129229]: osdmap e72: 11 total, 11 up, 11 in 2023-07-18T03:03:52.226 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:51 smithi101 ceph-mon[126379]: purged_snaps scrub starts 2023-07-18T03:03:52.227 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:51 smithi101 ceph-mon[126379]: purged_snaps scrub ok 2023-07-18T03:03:52.227 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:51 smithi101 ceph-mon[126379]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:03:52.227 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:51 smithi101 ceph-mon[126379]: osd.10 [v2:172.21.15.155:6824/3682446349,v1:172.21.15.155:6825/3682446349] boot 2023-07-18T03:03:52.227 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:51 smithi101 ceph-mon[126379]: osdmap e71: 11 total, 11 up, 11 in 2023-07-18T03:03:52.227 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:51 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-18T03:03:52.227 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:51 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:03:52.228 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:51 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:03:52.228 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:51 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:03:52.228 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:51 smithi101 ceph-mon[126379]: osdmap e72: 11 total, 11 up, 11 in 2023-07-18T03:03:52.284 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:51 smithi155 ceph-mon[128885]: purged_snaps scrub starts 2023-07-18T03:03:52.284 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:51 smithi155 ceph-mon[128885]: purged_snaps scrub ok 2023-07-18T03:03:52.285 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:51 smithi155 ceph-mon[128885]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:03:52.285 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:51 smithi155 ceph-mon[128885]: osd.10 [v2:172.21.15.155:6824/3682446349,v1:172.21.15.155:6825/3682446349] boot 2023-07-18T03:03:52.285 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:51 smithi155 ceph-mon[128885]: osdmap e71: 11 total, 11 up, 11 in 2023-07-18T03:03:52.285 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:51 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-18T03:03:52.285 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:51 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:03:52.285 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:51 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:03:52.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:51 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:03:52.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:51 smithi155 ceph-mon[128885]: osdmap e72: 11 total, 11 up, 11 in 2023-07-18T03:03:53.127 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:52 smithi179 ceph-mon[129229]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:03:53.128 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:52 smithi179 ceph-mon[129229]: osdmap e73: 11 total, 11 up, 11 in 2023-07-18T03:03:53.226 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:52 smithi101 ceph-mon[126379]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:03:53.227 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:52 smithi101 ceph-mon[126379]: osdmap e73: 11 total, 11 up, 11 in 2023-07-18T03:03:53.284 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:52 smithi155 ceph-mon[128885]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:03:53.284 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:52 smithi155 ceph-mon[128885]: osdmap e73: 11 total, 11 up, 11 in 2023-07-18T03:03:54.226 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:53 smithi101 ceph-mon[126379]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail 2023-07-18T03:03:54.226 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:53 smithi101 ceph-mon[126379]: osdmap e74: 11 total, 11 up, 11 in 2023-07-18T03:03:54.284 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:53 smithi155 ceph-mon[128885]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail 2023-07-18T03:03:54.284 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:53 smithi155 ceph-mon[128885]: osdmap e74: 11 total, 11 up, 11 in 2023-07-18T03:03:54.439 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:53 smithi179 ceph-mon[129229]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail 2023-07-18T03:03:54.439 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:53 smithi179 ceph-mon[129229]: osdmap e74: 11 total, 11 up, 11 in 2023-07-18T03:03:55.226 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:54 smithi101 ceph-mon[126379]: from='client.? 172.21.15.179:0/3550604388' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b62effaf-45e2-4585-b8d0-db448f3474ea"}]: dispatch 2023-07-18T03:03:55.226 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:54 smithi101 ceph-mon[126379]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b62effaf-45e2-4585-b8d0-db448f3474ea"}]: dispatch 2023-07-18T03:03:55.227 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:54 smithi101 ceph-mon[126379]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b62effaf-45e2-4585-b8d0-db448f3474ea"}]': finished 2023-07-18T03:03:55.227 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:54 smithi101 ceph-mon[126379]: osdmap e75: 12 total, 11 up, 12 in 2023-07-18T03:03:55.227 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:54 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-18T03:03:55.227 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:54 smithi101 ceph-mon[126379]: from='client.? 172.21.15.179:0/2715272922' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:03:55.284 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:54 smithi155 ceph-mon[128885]: from='client.? 172.21.15.179:0/3550604388' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b62effaf-45e2-4585-b8d0-db448f3474ea"}]: dispatch 2023-07-18T03:03:55.285 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:54 smithi155 ceph-mon[128885]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b62effaf-45e2-4585-b8d0-db448f3474ea"}]: dispatch 2023-07-18T03:03:55.285 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:54 smithi155 ceph-mon[128885]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b62effaf-45e2-4585-b8d0-db448f3474ea"}]': finished 2023-07-18T03:03:55.285 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:54 smithi155 ceph-mon[128885]: osdmap e75: 12 total, 11 up, 12 in 2023-07-18T03:03:55.285 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:54 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-18T03:03:55.285 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:54 smithi155 ceph-mon[128885]: from='client.? 172.21.15.179:0/2715272922' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:03:55.439 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:54 smithi179 ceph-mon[129229]: from='client.? 172.21.15.179:0/3550604388' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b62effaf-45e2-4585-b8d0-db448f3474ea"}]: dispatch 2023-07-18T03:03:55.439 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:54 smithi179 ceph-mon[129229]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b62effaf-45e2-4585-b8d0-db448f3474ea"}]: dispatch 2023-07-18T03:03:55.440 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:54 smithi179 ceph-mon[129229]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b62effaf-45e2-4585-b8d0-db448f3474ea"}]': finished 2023-07-18T03:03:55.440 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:54 smithi179 ceph-mon[129229]: osdmap e75: 12 total, 11 up, 12 in 2023-07-18T03:03:55.440 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:54 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-18T03:03:55.440 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:54 smithi179 ceph-mon[129229]: from='client.? 172.21.15.179:0/2715272922' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:03:56.439 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:56 smithi179 ceph-mon[129229]: pgmap v231: 1 pgs: 1 peering; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail; 148 KiB/s, 0 objects/s recovering 2023-07-18T03:03:56.476 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:56 smithi101 ceph-mon[126379]: pgmap v231: 1 pgs: 1 peering; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail; 148 KiB/s, 0 objects/s recovering 2023-07-18T03:03:56.534 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:56 smithi155 ceph-mon[128885]: pgmap v231: 1 pgs: 1 peering; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail; 148 KiB/s, 0 objects/s recovering 2023-07-18T03:03:58.439 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:58 smithi179 ceph-mon[129229]: pgmap v232: 1 pgs: 1 peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 118 KiB/s, 0 objects/s recovering 2023-07-18T03:03:58.476 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:58 smithi101 ceph-mon[126379]: pgmap v232: 1 pgs: 1 peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 118 KiB/s, 0 objects/s recovering 2023-07-18T03:03:58.534 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:58 smithi155 ceph-mon[128885]: pgmap v232: 1 pgs: 1 peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 118 KiB/s, 0 objects/s recovering 2023-07-18T03:03:59.689 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:03:59 smithi179 ceph-mon[129229]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-18T03:03:59.726 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:03:59 smithi101 ceph-mon[126379]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-18T03:03:59.784 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:03:59 smithi155 ceph-mon[128885]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-18T03:04:00.940 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:00 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-07-18T03:04:00.941 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:00 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:00.977 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:00 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-07-18T03:04:00.977 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:00 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:01.034 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:00 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-07-18T03:04:01.034 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:00 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:01.939 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:01 smithi179 ceph-mon[129229]: Deploying daemon osd.11 on smithi179 2023-07-18T03:04:01.939 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:01 smithi179 ceph-mon[129229]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2023-07-18T03:04:02.034 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:01 smithi155 ceph-mon[128885]: Deploying daemon osd.11 on smithi179 2023-07-18T03:04:02.037 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:01 smithi155 ceph-mon[128885]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2023-07-18T03:04:02.227 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:01 smithi101 ceph-mon[126379]: Deploying daemon osd.11 on smithi179 2023-07-18T03:04:02.227 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:01 smithi101 ceph-mon[126379]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2023-07-18T03:04:03.940 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:03 smithi179 ceph-mon[129229]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 61 KiB/s, 0 objects/s recovering 2023-07-18T03:04:04.226 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:03 smithi101 ceph-mon[126379]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 61 KiB/s, 0 objects/s recovering 2023-07-18T03:04:04.284 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:03 smithi155 ceph-mon[128885]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 61 KiB/s, 0 objects/s recovering 2023-07-18T03:04:06.332 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:06 smithi179 ceph-mon[129229]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-18T03:04:06.332 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:06 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:06.332 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:06 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:06.332 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:06 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:04:06.476 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:06 smithi101 ceph-mon[126379]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-18T03:04:06.477 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:06 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:06.477 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:06 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:06.477 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:06 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:04:06.534 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:06 smithi155 ceph-mon[128885]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-18T03:04:06.534 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:06 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:06.534 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:06 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:06.535 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:06 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:04:07.759 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:07.777+0000 7f96777fe700 1 -- 172.21.15.179:0/3203036079 <== mgr.14150 v2:172.21.15.101:6800/3963742842 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f96800fe3a0 con 0x7f966c0a1150 2023-07-18T03:04:07.760 INFO:teuthology.orchestra.run.smithi179.stdout:Created osd(s) 11 on host 'smithi179' 2023-07-18T03:04:07.761 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:07.779+0000 7f968857e700 1 -- 172.21.15.179:0/3203036079 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f966c0a1150 msgr2=0x7f966c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:07.761 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:07.779+0000 7f968857e700 1 --2- 172.21.15.179:0/3203036079 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f966c0a1150 0x7f966c0a3600 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9680067db0 tx=0x7f9670005cf0 comp rx=0 tx=0).stop 2023-07-18T03:04:07.761 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:07.779+0000 7f968857e700 1 -- 172.21.15.179:0/3203036079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f96800ff6b0 msgr2=0x7f96801980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:07.762 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:07.779+0000 7f968857e700 1 --2- 172.21.15.179:0/3203036079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f96800ff6b0 0x7f96801980b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f967c00c950 tx=0x7f967c00cd10 comp rx=0 tx=0).stop 2023-07-18T03:04:07.762 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:07.780+0000 7f968857e700 1 -- 172.21.15.179:0/3203036079 shutdown_connections 2023-07-18T03:04:07.762 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:07.780+0000 7f968857e700 1 --2- 172.21.15.179:0/3203036079 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f966c0a1150 0x7f966c0a3600 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:07.762 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:07.780+0000 7f968857e700 1 --2- 172.21.15.179:0/3203036079 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96800fed00 0x7f9680197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:07.762 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:07.780+0000 7f968857e700 1 --2- 172.21.15.179:0/3203036079 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f96801000a0 0x7f968019f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:07.762 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:07.780+0000 7f968857e700 1 --2- 172.21.15.179:0/3203036079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f96800ff6b0 0x7f96801980b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:07.763 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:07.780+0000 7f968857e700 1 -- 172.21.15.179:0/3203036079 >> 172.21.15.179:0/3203036079 conn(0x7f96800f9860 msgr2=0x7f96800fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:07.763 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:07.780+0000 7f968857e700 1 -- 172.21.15.179:0/3203036079 shutdown_connections 2023-07-18T03:04:07.763 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:07.780+0000 7f968857e700 1 -- 172.21.15.179:0/3203036079 wait complete. 2023-07-18T03:04:08.102 DEBUG:teuthology.orchestra.run.smithi179:osd.11> sudo journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@osd.11.service 2023-07-18T03:04:08.104 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-07-18T03:04:08.104 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph osd stat -f json 2023-07-18T03:04:08.134 INFO:journalctl@ceph.osd.11.smithi179.stdout:-- Logs begin at Tue 2023-07-18 02:29:48 UTC. -- 2023-07-18T03:04:08.439 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:08 smithi179 ceph-mon[129229]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-18T03:04:08.440 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:08 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:08.440 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:08 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:08.440 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:08 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:08.440 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:08 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:08.476 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:08 smithi101 ceph-mon[126379]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-18T03:04:08.477 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:08 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:08.477 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:08 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:08.477 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:08 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:08.477 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:08 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:08.534 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:08 smithi155 ceph-mon[128885]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-18T03:04:08.535 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:08 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:08.535 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:08 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:08.535 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:08 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:08.535 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:08 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:08.834 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:04:09.189 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:04:09 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:04:09.073+0000 7f7499a3b640 -1 osd.11 0 log_to_monitors true 2023-07-18T03:04:09.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:09 smithi179 ceph-mon[129229]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-18T03:04:09.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:09 smithi179 ceph-mon[129229]: from='osd.11 [v2:172.21.15.179:6824/2602640858,v1:172.21.15.179:6825/2602640858]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-18T03:04:09.691 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:09 smithi179 ceph-mon[129229]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-18T03:04:09.727 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:09 smithi101 ceph-mon[126379]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-18T03:04:09.727 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:09 smithi101 ceph-mon[126379]: from='osd.11 [v2:172.21.15.179:6824/2602640858,v1:172.21.15.179:6825/2602640858]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-18T03:04:09.728 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:09 smithi101 ceph-mon[126379]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-18T03:04:09.784 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:09 smithi155 ceph-mon[128885]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-18T03:04:09.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:09 smithi155 ceph-mon[128885]: from='osd.11 [v2:172.21.15.179:6824/2602640858,v1:172.21.15.179:6825/2602640858]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-18T03:04:09.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:09 smithi155 ceph-mon[128885]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-18T03:04:10.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.478+0000 7fda58cdd700 1 -- 172.21.15.101:0/3340353669 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda54074540 msgr2=0x7fda540729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:10.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.478+0000 7fda58cdd700 1 --2- 172.21.15.101:0/3340353669 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda54074540 0x7fda540729a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fda44009990 tx=0x7fda44009ca0 comp rx=0 tx=0).stop 2023-07-18T03:04:10.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.478+0000 7fda58cdd700 1 -- 172.21.15.101:0/3340353669 shutdown_connections 2023-07-18T03:04:10.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.478+0000 7fda58cdd700 1 --2- 172.21.15.101:0/3340353669 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda54107a60 0x7fda54109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:10.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.478+0000 7fda58cdd700 1 --2- 172.21.15.101:0/3340353669 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda54074540 0x7fda540729a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:10.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.478+0000 7fda58cdd700 1 --2- 172.21.15.101:0/3340353669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda54072f70 0x7fda540733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:10.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.478+0000 7fda58cdd700 1 -- 172.21.15.101:0/3340353669 >> 172.21.15.101:0/3340353669 conn(0x7fda540fb1a0 msgr2=0x7fda540fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:10.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.478+0000 7fda58cdd700 1 -- 172.21.15.101:0/3340353669 shutdown_connections 2023-07-18T03:04:10.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.478+0000 7fda58cdd700 1 -- 172.21.15.101:0/3340353669 wait complete. 2023-07-18T03:04:10.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.479+0000 7fda58cdd700 1 Processor -- start 2023-07-18T03:04:10.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.479+0000 7fda58cdd700 1 -- start start 2023-07-18T03:04:10.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.479+0000 7fda58cdd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda54072f70 0x7fda54197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:10.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.479+0000 7fda58cdd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda54074540 0x7fda54198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:10.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.479+0000 7fda58cdd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda54107a60 0x7fda5419f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:10.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.479+0000 7fda58cdd700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5419f650 con 0x7fda54072f70 2023-07-18T03:04:10.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.479+0000 7fda58cdd700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5419f790 con 0x7fda54074540 2023-07-18T03:04:10.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.479+0000 7fda58cdd700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5419f920 con 0x7fda54107a60 2023-07-18T03:04:10.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.480+0000 7fda51d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda54074540 0x7fda54198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:10.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.480+0000 7fda52d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda54107a60 0x7fda5419f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:10.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.480+0000 7fda5259c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda54072f70 0x7fda54197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:10.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.480+0000 7fda51d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda54074540 0x7fda54198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:59352/0 (socket says 172.21.15.101:59352) 2023-07-18T03:04:10.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.480+0000 7fda51d9b700 1 -- 172.21.15.101:0/2668983972 learned_addr learned my addr 172.21.15.101:0/2668983972 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:04:10.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.480+0000 7fda5259c700 1 -- 172.21.15.101:0/2668983972 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda54107a60 msgr2=0x7fda5419f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:10.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.480+0000 7fda5259c700 1 --2- 172.21.15.101:0/2668983972 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda54107a60 0x7fda5419f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:10.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.480+0000 7fda5259c700 1 -- 172.21.15.101:0/2668983972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda54074540 msgr2=0x7fda54198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:10.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.480+0000 7fda5259c700 1 --2- 172.21.15.101:0/2668983972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda54074540 0x7fda54198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:10.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.480+0000 7fda5259c700 1 -- 172.21.15.101:0/2668983972 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda440095f0 con 0x7fda54072f70 2023-07-18T03:04:10.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.480+0000 7fda5259c700 1 --2- 172.21.15.101:0/2668983972 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda54072f70 0x7fda54197ac0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fda4400b680 tx=0x7fda44004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:10.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.481+0000 7fda437fe700 1 -- 172.21.15.101:0/2668983972 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4401d070 con 0x7fda54072f70 2023-07-18T03:04:10.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.481+0000 7fda437fe700 1 -- 172.21.15.101:0/2668983972 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda44005030 con 0x7fda54072f70 2023-07-18T03:04:10.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.481+0000 7fda437fe700 1 -- 172.21.15.101:0/2668983972 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda44004bc0 con 0x7fda54072f70 2023-07-18T03:04:10.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.481+0000 7fda58cdd700 1 -- 172.21.15.101:0/2668983972 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda540ff550 con 0x7fda54072f70 2023-07-18T03:04:10.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.481+0000 7fda58cdd700 1 -- 172.21.15.101:0/2668983972 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda540ff990 con 0x7fda54072f70 2023-07-18T03:04:10.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.482+0000 7fda58cdd700 1 -- 172.21.15.101:0/2668983972 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda54106110 con 0x7fda54072f70 2023-07-18T03:04:10.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.484+0000 7fda437fe700 1 -- 172.21.15.101:0/2668983972 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fda44004d40 con 0x7fda54072f70 2023-07-18T03:04:10.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.486+0000 7fda437fe700 1 --2- 172.21.15.101:0/2668983972 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fda3c0a11d0 0x7fda3c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:10.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.486+0000 7fda437fe700 1 -- 172.21.15.101:0/2668983972 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6847+0+0 (secure 0 0 0) 0x7fda440cd4d0 con 0x7fda54072f70 2023-07-18T03:04:10.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.486+0000 7fda51d9b700 1 --2- 172.21.15.101:0/2668983972 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fda3c0a11d0 0x7fda3c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:10.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.487+0000 7fda51d9b700 1 --2- 172.21.15.101:0/2668983972 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fda3c0a11d0 0x7fda3c0a3680 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fda48013fd0 tx=0x7fda48013d10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:10.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.490+0000 7fda437fe700 1 -- 172.21.15.101:0/2668983972 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fda440971d0 con 0x7fda54072f70 2023-07-18T03:04:10.477 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:10 smithi101 ceph-mon[126379]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-07-18T03:04:10.477 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:10 smithi101 ceph-mon[126379]: osdmap e76: 12 total, 11 up, 12 in 2023-07-18T03:04:10.477 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:10 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-18T03:04:10.477 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:10 smithi101 ceph-mon[126379]: from='osd.11 [v2:172.21.15.179:6824/2602640858,v1:172.21.15.179:6825/2602640858]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2023-07-18T03:04:10.477 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:10 smithi101 ceph-mon[126379]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2023-07-18T03:04:10.477 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:10 smithi101 ceph-mon[126379]: Detected new or changed devices on smithi179 2023-07-18T03:04:10.477 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:10 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:10.478 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:10 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:10.478 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:10 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:04:10.478 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:10 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:04:10.478 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:10 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:04:10.478 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:10 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:04:10.478 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:10 smithi101 ceph-mon[126379]: Adjusting osd_memory_target on smithi179 to 4292M 2023-07-18T03:04:10.478 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:10 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:10.479 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:10 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:10.479 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:10 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:04:10.479 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:10 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:10.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.700+0000 7fda58cdd700 1 -- 172.21.15.101:0/2668983972 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fda5402c850 con 0x7fda54072f70 2023-07-18T03:04:10.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.701+0000 7fda437fe700 1 -- 172.21.15.101:0/2668983972 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v77) v1 ==== 74+0+133 (secure 0 0 0) 0x7fda440051a0 con 0x7fda54072f70 2023-07-18T03:04:10.684 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:04:10.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.703+0000 7fda58cdd700 1 -- 172.21.15.101:0/2668983972 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fda3c0a11d0 msgr2=0x7fda3c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:10.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.703+0000 7fda58cdd700 1 --2- 172.21.15.101:0/2668983972 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fda3c0a11d0 0x7fda3c0a3680 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fda48013fd0 tx=0x7fda48013d10 comp rx=0 tx=0).stop 2023-07-18T03:04:10.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.703+0000 7fda58cdd700 1 -- 172.21.15.101:0/2668983972 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda54072f70 msgr2=0x7fda54197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:10.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.703+0000 7fda58cdd700 1 --2- 172.21.15.101:0/2668983972 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda54072f70 0x7fda54197ac0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fda4400b680 tx=0x7fda44004730 comp rx=0 tx=0).stop 2023-07-18T03:04:10.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.703+0000 7fda58cdd700 1 -- 172.21.15.101:0/2668983972 shutdown_connections 2023-07-18T03:04:10.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.703+0000 7fda58cdd700 1 --2- 172.21.15.101:0/2668983972 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fda3c0a11d0 0x7fda3c0a3680 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:10.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.703+0000 7fda58cdd700 1 --2- 172.21.15.101:0/2668983972 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda54107a60 0x7fda5419f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:10.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.703+0000 7fda58cdd700 1 --2- 172.21.15.101:0/2668983972 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda54072f70 0x7fda54197ac0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:10.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.703+0000 7fda58cdd700 1 --2- 172.21.15.101:0/2668983972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda54074540 0x7fda54198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:10.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.703+0000 7fda58cdd700 1 -- 172.21.15.101:0/2668983972 >> 172.21.15.101:0/2668983972 conn(0x7fda540fb1a0 msgr2=0x7fda540fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:10.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.703+0000 7fda58cdd700 1 -- 172.21.15.101:0/2668983972 shutdown_connections 2023-07-18T03:04:10.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:10.703+0000 7fda58cdd700 1 -- 172.21.15.101:0/2668983972 wait complete. 2023-07-18T03:04:10.689 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:04:10 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:04:10.436+0000 7f748f8b4700 -1 osd.11 0 waiting for initial osdmap 2023-07-18T03:04:10.689 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:04:10 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:04:10.448+0000 7f7488ea4700 -1 osd.11 77 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-18T03:04:10.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:10 smithi179 ceph-mon[129229]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-07-18T03:04:10.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:10 smithi179 ceph-mon[129229]: osdmap e76: 12 total, 11 up, 12 in 2023-07-18T03:04:10.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:10 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-18T03:04:10.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:10 smithi179 ceph-mon[129229]: from='osd.11 [v2:172.21.15.179:6824/2602640858,v1:172.21.15.179:6825/2602640858]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2023-07-18T03:04:10.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:10 smithi179 ceph-mon[129229]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2023-07-18T03:04:10.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:10 smithi179 ceph-mon[129229]: Detected new or changed devices on smithi179 2023-07-18T03:04:10.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:10 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:10.691 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:10 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:10.691 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:10 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:04:10.691 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:10 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:04:10.691 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:10 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:04:10.691 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:10 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:04:10.691 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:10 smithi179 ceph-mon[129229]: Adjusting osd_memory_target on smithi179 to 4292M 2023-07-18T03:04:10.692 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:10 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:10.692 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:10 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:10.692 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:10 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:04:10.692 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:10 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:10.784 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:10 smithi155 ceph-mon[128885]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-07-18T03:04:10.784 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:10 smithi155 ceph-mon[128885]: osdmap e76: 12 total, 11 up, 12 in 2023-07-18T03:04:10.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:10 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-18T03:04:10.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:10 smithi155 ceph-mon[128885]: from='osd.11 [v2:172.21.15.179:6824/2602640858,v1:172.21.15.179:6825/2602640858]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2023-07-18T03:04:10.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:10 smithi155 ceph-mon[128885]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2023-07-18T03:04:10.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:10 smithi155 ceph-mon[128885]: Detected new or changed devices on smithi179 2023-07-18T03:04:10.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:10 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:10.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:10 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:10.786 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:10 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:04:10.786 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:10 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:04:10.786 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:10 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:04:10.786 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:10 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:04:10.786 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:10 smithi155 ceph-mon[128885]: Adjusting osd_memory_target on smithi179 to 4292M 2023-07-18T03:04:10.786 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:10 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:10.786 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:10 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:10.786 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:10 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:04:10.787 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:10 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:11.205 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":77,"num_osds":12,"num_up_osds":11,"osd_up_since":1689649430,"num_in_osds":12,"osd_in_since":1689649434,"num_remapped_pgs":0} 2023-07-18T03:04:11.689 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:11 smithi179 ceph-mon[129229]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2023-07-18T03:04:11.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:11 smithi179 ceph-mon[129229]: osdmap e77: 12 total, 11 up, 12 in 2023-07-18T03:04:11.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:11 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-18T03:04:11.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:11 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-18T03:04:11.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:11 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:04:11.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:11 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:11.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:11 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:04:11.691 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:11 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:11.691 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:11 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/2668983972' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-18T03:04:11.691 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:11 smithi179 ceph-mon[129229]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-18T03:04:11.727 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:11 smithi101 ceph-mon[126379]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2023-07-18T03:04:11.727 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:11 smithi101 ceph-mon[126379]: osdmap e77: 12 total, 11 up, 12 in 2023-07-18T03:04:11.727 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:11 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-18T03:04:11.728 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:11 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-18T03:04:11.728 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:11 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:04:11.728 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:11 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:11.728 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:11 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:04:11.728 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:11 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:11.728 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:11 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2668983972' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-18T03:04:11.728 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:11 smithi101 ceph-mon[126379]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-18T03:04:11.784 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:11 smithi155 ceph-mon[128885]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2023-07-18T03:04:11.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:11 smithi155 ceph-mon[128885]: osdmap e77: 12 total, 11 up, 12 in 2023-07-18T03:04:11.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:11 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-18T03:04:11.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:11 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-18T03:04:11.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:11 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:04:11.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:11 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:11.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:11 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:04:11.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:11 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:11.786 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:11 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/2668983972' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-18T03:04:11.786 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:11 smithi155 ceph-mon[128885]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-18T03:04:12.206 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph osd stat -f json 2023-07-18T03:04:12.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:12 smithi179 ceph-mon[129229]: purged_snaps scrub starts 2023-07-18T03:04:12.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:12 smithi179 ceph-mon[129229]: purged_snaps scrub ok 2023-07-18T03:04:12.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:12 smithi179 ceph-mon[129229]: osd.11 [v2:172.21.15.179:6824/2602640858,v1:172.21.15.179:6825/2602640858] boot 2023-07-18T03:04:12.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:12 smithi179 ceph-mon[129229]: osdmap e78: 12 total, 12 up, 12 in 2023-07-18T03:04:12.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:12 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-18T03:04:12.727 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:12 smithi101 ceph-mon[126379]: purged_snaps scrub starts 2023-07-18T03:04:12.727 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:12 smithi101 ceph-mon[126379]: purged_snaps scrub ok 2023-07-18T03:04:12.727 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:12 smithi101 ceph-mon[126379]: osd.11 [v2:172.21.15.179:6824/2602640858,v1:172.21.15.179:6825/2602640858] boot 2023-07-18T03:04:12.728 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:12 smithi101 ceph-mon[126379]: osdmap e78: 12 total, 12 up, 12 in 2023-07-18T03:04:12.728 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:12 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-18T03:04:12.784 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:12 smithi155 ceph-mon[128885]: purged_snaps scrub starts 2023-07-18T03:04:12.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:12 smithi155 ceph-mon[128885]: purged_snaps scrub ok 2023-07-18T03:04:12.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:12 smithi155 ceph-mon[128885]: osd.11 [v2:172.21.15.179:6824/2602640858,v1:172.21.15.179:6825/2602640858] boot 2023-07-18T03:04:12.786 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:12 smithi155 ceph-mon[128885]: osdmap e78: 12 total, 12 up, 12 in 2023-07-18T03:04:12.786 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:12 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-18T03:04:12.935 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:04:13.784 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:13 smithi155 ceph-mon[128885]: osdmap e79: 12 total, 12 up, 12 in 2023-07-18T03:04:13.784 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:13 smithi155 ceph-mon[128885]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:13.887 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:13 smithi101 ceph-mon[126379]: osdmap e79: 12 total, 12 up, 12 in 2023-07-18T03:04:13.888 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:13 smithi101 ceph-mon[126379]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:13.939 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:13 smithi179 ceph-mon[129229]: osdmap e79: 12 total, 12 up, 12 in 2023-07-18T03:04:13.940 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:13 smithi179 ceph-mon[129229]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:14.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.581+0000 7ff60b728700 1 -- 172.21.15.101:0/679515614 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff604101720 msgr2=0x7ff604107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:14.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.581+0000 7ff60b728700 1 --2- 172.21.15.101:0/679515614 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff604101720 0x7ff604107fe0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc00b0a0 tx=0x7ff5fc00b3b0 comp rx=0 tx=0).stop 2023-07-18T03:04:14.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.581+0000 7ff60b728700 1 -- 172.21.15.101:0/679515614 shutdown_connections 2023-07-18T03:04:14.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.581+0000 7ff60b728700 1 --2- 172.21.15.101:0/679515614 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6040ffb90 0x7ff6040fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:14.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.581+0000 7ff60b728700 1 --2- 172.21.15.101:0/679515614 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff604101720 0x7ff604107fe0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:14.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.581+0000 7ff60b728700 1 --2- 172.21.15.101:0/679515614 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff604100d90 0x7ff6041011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:14.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.581+0000 7ff60b728700 1 -- 172.21.15.101:0/679515614 >> 172.21.15.101:0/679515614 conn(0x7ff6040fb1a0 msgr2=0x7ff6040fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:14.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.581+0000 7ff60b728700 1 -- 172.21.15.101:0/679515614 shutdown_connections 2023-07-18T03:04:14.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.581+0000 7ff60b728700 1 -- 172.21.15.101:0/679515614 wait complete. 2023-07-18T03:04:14.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.582+0000 7ff60b728700 1 Processor -- start 2023-07-18T03:04:14.567 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.582+0000 7ff60b728700 1 -- start start 2023-07-18T03:04:14.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.582+0000 7ff60b728700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6040ffb90 0x7ff604197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:14.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.582+0000 7ff60b728700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff604100d90 0x7ff6041980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:14.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.582+0000 7ff60b728700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff604101720 0x7ff60419f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:14.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.582+0000 7ff60b728700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60419f740 con 0x7ff604100d90 2023-07-18T03:04:14.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.582+0000 7ff60b728700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60419f880 con 0x7ff604101720 2023-07-18T03:04:14.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.582+0000 7ff60b728700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60419fa10 con 0x7ff6040ffb90 2023-07-18T03:04:14.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.583+0000 7ff608cc3700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff604100d90 0x7ff6041980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:14.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.583+0000 7ff6094c4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6040ffb90 0x7ff604197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:14.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.583+0000 7ff608cc3700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff604100d90 0x7ff6041980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:49754/0 (socket says 172.21.15.101:49754) 2023-07-18T03:04:14.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.583+0000 7ff608cc3700 1 -- 172.21.15.101:0/1686585892 learned_addr learned my addr 172.21.15.101:0/1686585892 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:04:14.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.583+0000 7ff6094c4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6040ffb90 0x7ff604197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:43918/0 (socket says 172.21.15.101:43918) 2023-07-18T03:04:14.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.583+0000 7ff609cc5700 1 --2- 172.21.15.101:0/1686585892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff604101720 0x7ff60419f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:14.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.583+0000 7ff608cc3700 1 -- 172.21.15.101:0/1686585892 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6040ffb90 msgr2=0x7ff604197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:14.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.583+0000 7ff608cc3700 1 --2- 172.21.15.101:0/1686585892 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6040ffb90 0x7ff604197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:14.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.583+0000 7ff608cc3700 1 -- 172.21.15.101:0/1686585892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff604101720 msgr2=0x7ff60419f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:14.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.583+0000 7ff608cc3700 1 --2- 172.21.15.101:0/1686585892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff604101720 0x7ff60419f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:14.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.583+0000 7ff608cc3700 1 -- 172.21.15.101:0/1686585892 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5fc009d00 con 0x7ff604100d90 2023-07-18T03:04:14.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.583+0000 7ff608cc3700 1 --2- 172.21.15.101:0/1686585892 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff604100d90 0x7ff6041980f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff5f400e9d0 tx=0x7ff5f400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:14.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.584+0000 7ff5fa7fc700 1 -- 172.21.15.101:0/1686585892 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4009790 con 0x7ff604100d90 2023-07-18T03:04:14.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.584+0000 7ff5fa7fc700 1 -- 172.21.15.101:0/1686585892 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5f400cbd0 con 0x7ff604100d90 2023-07-18T03:04:14.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.584+0000 7ff5fa7fc700 1 -- 172.21.15.101:0/1686585892 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4018680 con 0x7ff604100d90 2023-07-18T03:04:14.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.584+0000 7ff60b728700 1 -- 172.21.15.101:0/1686585892 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff60419fcf0 con 0x7ff604100d90 2023-07-18T03:04:14.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.584+0000 7ff60b728700 1 -- 172.21.15.101:0/1686585892 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6041a01a0 con 0x7ff604100d90 2023-07-18T03:04:14.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.585+0000 7ff60b728700 1 -- 172.21.15.101:0/1686585892 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff604192270 con 0x7ff604100d90 2023-07-18T03:04:14.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.587+0000 7ff5fa7fc700 1 -- 172.21.15.101:0/1686585892 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7ff5f400cd40 con 0x7ff604100d90 2023-07-18T03:04:14.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.588+0000 7ff5fa7fc700 1 --2- 172.21.15.101:0/1686585892 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7ff5f00a11d0 0x7ff5f00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:14.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.588+0000 7ff5fa7fc700 1 -- 172.21.15.101:0/1686585892 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff5f4014070 con 0x7ff604100d90 2023-07-18T03:04:14.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.588+0000 7ff6094c4700 1 --2- 172.21.15.101:0/1686585892 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7ff5f00a11d0 0x7ff5f00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:14.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.589+0000 7ff6094c4700 1 --2- 172.21.15.101:0/1686585892 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7ff5f00a11d0 0x7ff5f00a3680 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff604100bf0 tx=0x7ff600009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:14.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.592+0000 7ff5fa7fc700 1 -- 172.21.15.101:0/1686585892 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff5f4051790 con 0x7ff604100d90 2023-07-18T03:04:14.777 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.787+0000 7ff60b728700 1 -- 172.21.15.101:0/1686585892 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7ff60402c850 con 0x7ff604100d90 2023-07-18T03:04:14.777 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.787+0000 7ff5fa7fc700 1 -- 172.21.15.101:0/1686585892 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v79) v1 ==== 74+0+133 (secure 0 0 0) 0x7ff5f4096250 con 0x7ff604100d90 2023-07-18T03:04:14.778 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:04:14.779 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.790+0000 7ff60b728700 1 -- 172.21.15.101:0/1686585892 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7ff5f00a11d0 msgr2=0x7ff5f00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:14.779 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.790+0000 7ff60b728700 1 --2- 172.21.15.101:0/1686585892 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7ff5f00a11d0 0x7ff5f00a3680 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff604100bf0 tx=0x7ff600009260 comp rx=0 tx=0).stop 2023-07-18T03:04:14.780 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.790+0000 7ff60b728700 1 -- 172.21.15.101:0/1686585892 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff604100d90 msgr2=0x7ff6041980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:14.780 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.790+0000 7ff60b728700 1 --2- 172.21.15.101:0/1686585892 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff604100d90 0x7ff6041980f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff5f400e9d0 tx=0x7ff5f400ece0 comp rx=0 tx=0).stop 2023-07-18T03:04:14.780 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.790+0000 7ff60b728700 1 -- 172.21.15.101:0/1686585892 shutdown_connections 2023-07-18T03:04:14.780 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.790+0000 7ff60b728700 1 --2- 172.21.15.101:0/1686585892 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7ff5f00a11d0 0x7ff5f00a3680 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:14.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.790+0000 7ff60b728700 1 --2- 172.21.15.101:0/1686585892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff604101720 0x7ff60419f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:14.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.790+0000 7ff60b728700 1 --2- 172.21.15.101:0/1686585892 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6040ffb90 0x7ff604197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:14.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.790+0000 7ff60b728700 1 --2- 172.21.15.101:0/1686585892 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff604100d90 0x7ff6041980f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:14.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.790+0000 7ff60b728700 1 -- 172.21.15.101:0/1686585892 >> 172.21.15.101:0/1686585892 conn(0x7ff6040fb1a0 msgr2=0x7ff604106670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:14.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.790+0000 7ff60b728700 1 -- 172.21.15.101:0/1686585892 shutdown_connections 2023-07-18T03:04:14.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:14.790+0000 7ff60b728700 1 -- 172.21.15.101:0/1686585892 wait complete. 2023-07-18T03:04:15.190 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:14 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1686585892' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-18T03:04:15.227 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:14 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1686585892' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-18T03:04:15.285 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:14 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1686585892' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-18T03:04:15.314 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":79,"num_osds":12,"num_up_osds":12,"osd_up_since":1689649451,"num_in_osds":12,"osd_in_since":1689649434,"num_remapped_pgs":0} 2023-07-18T03:04:15.314 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph osd dump --format=json 2023-07-18T03:04:16.036 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:04:16.189 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:15 smithi179 ceph-mon[129229]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:16.228 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:15 smithi101 ceph-mon[126379]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:16.284 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:15 smithi155 ceph-mon[128885]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:17.632 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.648+0000 7f94090b5700 1 -- 172.21.15.101:0/263631161 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f94041007a0 msgr2=0x7f9404104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:17.632 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.648+0000 7f94090b5700 1 --2- 172.21.15.101:0/263631161 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f94041007a0 0x7f9404104770 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f93f80099c0 tx=0x7f93f8009cd0 comp rx=0 tx=0).stop 2023-07-18T03:04:17.632 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.648+0000 7f94090b5700 1 -- 172.21.15.101:0/263631161 shutdown_connections 2023-07-18T03:04:17.632 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.648+0000 7f94090b5700 1 --2- 172.21.15.101:0/263631161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94040ffe00 0x7f94041001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:17.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.648+0000 7f94090b5700 1 --2- 172.21.15.101:0/263631161 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9404104ea0 0x7f940410ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:17.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.648+0000 7f94090b5700 1 --2- 172.21.15.101:0/263631161 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f94041007a0 0x7f9404104770 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:17.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.648+0000 7f94090b5700 1 -- 172.21.15.101:0/263631161 >> 172.21.15.101:0/263631161 conn(0x7f94040fb660 msgr2=0x7f94040fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:17.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.648+0000 7f94090b5700 1 -- 172.21.15.101:0/263631161 shutdown_connections 2023-07-18T03:04:17.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.648+0000 7f94090b5700 1 -- 172.21.15.101:0/263631161 wait complete. 2023-07-18T03:04:17.634 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.649+0000 7f94090b5700 1 Processor -- start 2023-07-18T03:04:17.635 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.649+0000 7f94090b5700 1 -- start start 2023-07-18T03:04:17.635 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.650+0000 7f94090b5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94040ffe00 0x7f9404072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:17.635 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.650+0000 7f94090b5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94041007a0 0x7f9404072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:17.635 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.650+0000 7f94090b5700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9404104ea0 0x7f94040732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:17.635 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.650+0000 7f94090b5700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94041a4120 con 0x7f9404104ea0 2023-07-18T03:04:17.635 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.650+0000 7f94090b5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94041a4290 con 0x7f94040ffe00 2023-07-18T03:04:17.636 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.650+0000 7f94090b5700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94041a4450 con 0x7f94041007a0 2023-07-18T03:04:17.636 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.650+0000 7f9402d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94040ffe00 0x7f9404072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:17.636 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.650+0000 7f9402d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94040ffe00 0x7f9404072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:49860/0 (socket says 172.21.15.101:49860) 2023-07-18T03:04:17.636 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.650+0000 7f9402d9d700 1 -- 172.21.15.101:0/1700812330 learned_addr learned my addr 172.21.15.101:0/1700812330 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:04:17.636 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.650+0000 7f940359e700 1 --2- 172.21.15.101:0/1700812330 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9404104ea0 0x7f94040732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:17.636 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.650+0000 7f940259c700 1 --2- 172.21.15.101:0/1700812330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94041007a0 0x7f9404072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:17.636 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.651+0000 7f9402d9d700 1 -- 172.21.15.101:0/1700812330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94041007a0 msgr2=0x7f9404072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:17.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.651+0000 7f9402d9d700 1 --2- 172.21.15.101:0/1700812330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94041007a0 0x7f9404072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:17.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.651+0000 7f9402d9d700 1 -- 172.21.15.101:0/1700812330 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9404104ea0 msgr2=0x7f94040732d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:17.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.651+0000 7f9402d9d700 1 --2- 172.21.15.101:0/1700812330 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9404104ea0 0x7f94040732d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:17.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.651+0000 7f9402d9d700 1 -- 172.21.15.101:0/1700812330 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f80095f0 con 0x7f94040ffe00 2023-07-18T03:04:17.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.651+0000 7f940359e700 1 --2- 172.21.15.101:0/1700812330 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9404104ea0 0x7f94040732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:04:17.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.651+0000 7f9402d9d700 1 --2- 172.21.15.101:0/1700812330 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94040ffe00 0x7f9404072810 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f93f400d980 tx=0x7f93f400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:17.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.651+0000 7f93ebfff700 1 -- 172.21.15.101:0/1700812330 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f4004220 con 0x7f94040ffe00 2023-07-18T03:04:17.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.651+0000 7f93ebfff700 1 -- 172.21.15.101:0/1700812330 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93f4007640 con 0x7f94040ffe00 2023-07-18T03:04:17.638 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.651+0000 7f94090b5700 1 -- 172.21.15.101:0/1700812330 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94041a4730 con 0x7f94040ffe00 2023-07-18T03:04:17.638 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.651+0000 7f93ebfff700 1 -- 172.21.15.101:0/1700812330 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f4005690 con 0x7f94040ffe00 2023-07-18T03:04:17.638 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.651+0000 7f94090b5700 1 -- 172.21.15.101:0/1700812330 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94041a4c00 con 0x7f94040ffe00 2023-07-18T03:04:17.639 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.652+0000 7f94090b5700 1 -- 172.21.15.101:0/1700812330 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f940404e210 con 0x7f94040ffe00 2023-07-18T03:04:17.639 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.654+0000 7f93ebfff700 1 -- 172.21.15.101:0/1700812330 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f93f4004750 con 0x7f94040ffe00 2023-07-18T03:04:17.639 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.655+0000 7f93ebfff700 1 --2- 172.21.15.101:0/1700812330 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f93ec0a1150 0x7f93ec0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:17.639 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.656+0000 7f93ebfff700 1 -- 172.21.15.101:0/1700812330 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f93f40cc750 con 0x7f94040ffe00 2023-07-18T03:04:17.639 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.656+0000 7f940259c700 1 --2- 172.21.15.101:0/1700812330 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f93ec0a1150 0x7f93ec0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:17.640 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.656+0000 7f940259c700 1 --2- 172.21.15.101:0/1700812330 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f93ec0a1150 0x7f93ec0a3600 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f93f8006010 tx=0x7f93f8003680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:17.641 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.658+0000 7f93ebfff700 1 -- 172.21.15.101:0/1700812330 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f93f4096340 con 0x7f94040ffe00 2023-07-18T03:04:17.839 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.856+0000 7f94090b5700 1 -- 172.21.15.101:0/1700812330 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9404073d40 con 0x7f94040ffe00 2023-07-18T03:04:17.841 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.858+0000 7f93ebfff700 1 -- 172.21.15.101:0/1700812330 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18503 (secure 0 0 0) 0x7f93f4099960 con 0x7f94040ffe00 2023-07-18T03:04:17.841 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:04:17.841 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":79,"fsid":"b1782daa-2516-11ee-9b34-001a4aab830c","created":"2023-07-18T02:57:22.972483+0000","modified":"2023-07-18T03:04:12.522208+0000","last_up_change":"2023-07-18T03:04:11.429530+0000","last_in_change":"2023-07-18T03:03:54.158340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:00:43.353489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc76b5d1-38a0-4f1a-9335-756bfc40a62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6803","nonce":3610719929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6805","nonce":3610719929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6809","nonce":3610719929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6807","nonce":3610719929}]},"public_addr":"172.21.15.101:6803/3610719929","cluster_addr":"172.21.15.101:6805/3610719929","heartbeat_back_addr":"172.21.15.101:6809/3610719929","heartbeat_front_addr":"172.21.15.101:6807/3610719929","state":["exists","up"]},{"osd":1,"uuid":"2a299573-0b34-4af9-aaae-b523e5058ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6801","nonce":1030229579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6803","nonce":1030229579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6807","nonce":1030229579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6805","nonce":1030229579}]},"public_addr":"172.21.15.155:6801/1030229579","cluster_addr":"172.21.15.155:6803/1030229579","heartbeat_back_addr":"172.21.15.155:6807/1030229579","heartbeat_front_addr":"172.21.15.155:6805/1030229579","state":["exists","up"]},{"osd":2,"uuid":"123b3749-c636-48fa-9e33-6229b5575dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6801","nonce":1349487507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6803","nonce":1349487507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6807","nonce":1349487507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6805","nonce":1349487507}]},"public_addr":"172.21.15.179:6801/1349487507","cluster_addr":"172.21.15.179:6803/1349487507","heartbeat_back_addr":"172.21.15.179:6807/1349487507","heartbeat_front_addr":"172.21.15.179:6805/1349487507","state":["exists","up"]},{"osd":3,"uuid":"4b0c409d-2954-4127-833a-b196bf226d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6811","nonce":4226577463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6813","nonce":4226577463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6817","nonce":4226577463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6815","nonce":4226577463}]},"public_addr":"172.21.15.101:6811/4226577463","cluster_addr":"172.21.15.101:6813/4226577463","heartbeat_back_addr":"172.21.15.101:6817/4226577463","heartbeat_front_addr":"172.21.15.101:6815/4226577463","state":["exists","up"]},{"osd":4,"uuid":"2137f2df-984e-45ce-baf6-b4ac12c7e3c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6809","nonce":1951135805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6811","nonce":1951135805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6815","nonce":1951135805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6813","nonce":1951135805}]},"public_addr":"172.21.15.155:6809/1951135805","cluster_addr":"172.21.15.155:6811/1951135805","heartbeat_back_addr":"172.21.15.155:6815/1951135805","heartbeat_front_addr":"172.21.15.155:6813/1951135805","state":["exists","up"]},{"osd":5,"uuid":"1020bfc5-8c2f-482e-9588-66c7f98e46e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6809","nonce":2746037021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6811","nonce":2746037021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6815","nonce":2746037021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6813","nonce":2746037021}]},"public_addr":"172.21.15.179:6809/2746037021","cluster_addr":"172.21.15.179:6811/2746037021","heartbeat_back_addr":"172.21.15.179:6815/2746037021","heartbeat_front_addr":"172.21.15.179:6813/2746037021","state":["exists","up"]},{"osd":6,"uuid":"2cd99260-5c36-4603-9338-cf697970ce58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6819","nonce":958044931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6821","nonce":958044931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6825","nonce":958044931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6823","nonce":958044931}]},"public_addr":"172.21.15.101:6819/958044931","cluster_addr":"172.21.15.101:6821/958044931","heartbeat_back_addr":"172.21.15.101:6825/958044931","heartbeat_front_addr":"172.21.15.101:6823/958044931","state":["exists","up"]},{"osd":7,"uuid":"ed23fd0a-678f-4e15-b42c-cfe873315714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6817","nonce":3256322291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6819","nonce":3256322291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6823","nonce":3256322291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6821","nonce":3256322291}]},"public_addr":"172.21.15.155:6817/3256322291","cluster_addr":"172.21.15.155:6819/3256322291","heartbeat_back_addr":"172.21.15.155:6823/3256322291","heartbeat_front_addr":"172.21.15.155:6821/3256322291","state":["exists","up"]},{"osd":8,"uuid":"eca3d8e6-7baf-47a5-b991-328001c7ebef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6817","nonce":3520612559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6819","nonce":3520612559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6823","nonce":3520612559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6821","nonce":3520612559}]},"public_addr":"172.21.15.179:6817/3520612559","cluster_addr":"172.21.15.179:6819/3520612559","heartbeat_back_addr":"172.21.15.179:6823/3520612559","heartbeat_front_addr":"172.21.15.179:6821/3520612559","state":["exists","up"]},{"osd":9,"uuid":"465a7ffb-67e4-49ef-812c-1b96feacfebf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6827","nonce":2166787667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6829","nonce":2166787667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6833","nonce":2166787667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6831","nonce":2166787667}]},"public_addr":"172.21.15.101:6827/2166787667","cluster_addr":"172.21.15.101:6829/2166787667","heartbeat_back_addr":"172.21.15.101:6833/2166787667","heartbeat_front_addr":"172.21.15.101:6831/2166787667","state":["exists","up"]},{"osd":10,"uuid":"1897a027-f25d-434f-8df5-5fc2bd866578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6825","nonce":3682446349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6827","nonce":3682446349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6831","nonce":3682446349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6829","nonce":3682446349}]},"public_addr":"172.21.15.155:6825/3682446349","cluster_addr":"172.21.15.155:6827/3682446349","heartbeat_back_addr":"172.21.15.155:6831/3682446349","heartbeat_front_addr":"172.21.15.155:6829/3682446349","state":["exists","up"]},{"osd":11,"uuid":"b62effaf-45e2-4585-b8d0-db448f3474ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6825","nonce":2602640858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6827","nonce":2602640858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6831","nonce":2602640858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6829","nonce":2602640858}]},"public_addr":"172.21.15.179:6825/2602640858","cluster_addr":"172.21.15.179:6827/2602640858","heartbeat_back_addr":"172.21.15.179:6831/2602640858","heartbeat_front_addr":"172.21.15.179:6829/2602640858","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T02:59:56.074519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:19.358142+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:41.128966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:08.227054+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:31.442592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:52.751685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:19.251054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:40.698241+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:01.339150+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:27.955910+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:49.136980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:04:10.084982+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.101:6800/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:6801/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/604228291":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/4055187849":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/2844574324":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/1603248215":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6801/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6800/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/3307578182":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/373189792":"2023-07-19T02:57:56.243200+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:04:17.843 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.860+0000 7f94090b5700 1 -- 172.21.15.101:0/1700812330 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f93ec0a1150 msgr2=0x7f93ec0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:17.843 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.860+0000 7f94090b5700 1 --2- 172.21.15.101:0/1700812330 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f93ec0a1150 0x7f93ec0a3600 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f93f8006010 tx=0x7f93f8003680 comp rx=0 tx=0).stop 2023-07-18T03:04:17.843 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.860+0000 7f94090b5700 1 -- 172.21.15.101:0/1700812330 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94040ffe00 msgr2=0x7f9404072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:17.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.860+0000 7f94090b5700 1 --2- 172.21.15.101:0/1700812330 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94040ffe00 0x7f9404072810 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f93f400d980 tx=0x7f93f400dc90 comp rx=0 tx=0).stop 2023-07-18T03:04:17.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.860+0000 7f94090b5700 1 -- 172.21.15.101:0/1700812330 shutdown_connections 2023-07-18T03:04:17.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.860+0000 7f94090b5700 1 --2- 172.21.15.101:0/1700812330 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f93ec0a1150 0x7f93ec0a3600 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:17.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.860+0000 7f94090b5700 1 --2- 172.21.15.101:0/1700812330 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94040ffe00 0x7f9404072810 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:17.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.860+0000 7f94090b5700 1 --2- 172.21.15.101:0/1700812330 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9404104ea0 0x7f94040732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:17.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.860+0000 7f94090b5700 1 --2- 172.21.15.101:0/1700812330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94041007a0 0x7f9404072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:17.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.860+0000 7f94090b5700 1 -- 172.21.15.101:0/1700812330 >> 172.21.15.101:0/1700812330 conn(0x7f94040fb660 msgr2=0x7f94040fc150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:17.845 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.860+0000 7f94090b5700 1 -- 172.21.15.101:0/1700812330 shutdown_connections 2023-07-18T03:04:17.845 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:17.860+0000 7f94090b5700 1 -- 172.21.15.101:0/1700812330 wait complete. 2023-07-18T03:04:18.189 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:17 smithi179 ceph-mon[129229]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:18.190 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:17 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1700812330' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:04:18.217 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:17 smithi101 ceph-mon[126379]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:18.217 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:17 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1700812330' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:04:18.284 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:17 smithi155 ceph-mon[128885]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:18.285 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:17 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1700812330' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:04:18.310 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:00:43.353489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-18T03:04:18.311 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph osd pool get .mgr pg_num 2023-07-18T03:04:19.034 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:04:19.589 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:19 smithi101 ceph-mon[126379]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:19.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:19 smithi179 ceph-mon[129229]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:19.784 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:19 smithi155 ceph-mon[128885]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:20.567 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.583+0000 7f7eb72dd700 1 -- 172.21.15.101:0/1362949708 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7eb00ffe00 msgr2=0x7f7eb01001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:20.567 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.583+0000 7f7eb72dd700 1 --2- 172.21.15.101:0/1362949708 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7eb00ffe00 0x7f7eb01001d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7eac0099c0 tx=0x7f7eac009cd0 comp rx=0 tx=0).stop 2023-07-18T03:04:20.567 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.583+0000 7f7eb72dd700 1 -- 172.21.15.101:0/1362949708 shutdown_connections 2023-07-18T03:04:20.567 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.583+0000 7f7eb72dd700 1 --2- 172.21.15.101:0/1362949708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7eb0104ea0 0x7f7eb010ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:20.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.583+0000 7f7eb72dd700 1 --2- 172.21.15.101:0/1362949708 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7eb00ffe00 0x7f7eb01001d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:20.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.583+0000 7f7eb72dd700 1 --2- 172.21.15.101:0/1362949708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7eb01007a0 0x7f7eb0104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:20.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.583+0000 7f7eb72dd700 1 -- 172.21.15.101:0/1362949708 >> 172.21.15.101:0/1362949708 conn(0x7f7eb00fb660 msgr2=0x7f7eb00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:20.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.583+0000 7f7eb72dd700 1 -- 172.21.15.101:0/1362949708 shutdown_connections 2023-07-18T03:04:20.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.583+0000 7f7eb72dd700 1 -- 172.21.15.101:0/1362949708 wait complete. 2023-07-18T03:04:20.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.584+0000 7f7eb72dd700 1 Processor -- start 2023-07-18T03:04:20.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.584+0000 7f7eb72dd700 1 -- start start 2023-07-18T03:04:20.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.585+0000 7f7eb72dd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7eb00ffe00 0x7f7eb0197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:20.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.585+0000 7f7eb72dd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7eb01007a0 0x7f7eb0198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:20.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.585+0000 7f7eb72dd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7eb0104ea0 0x7f7eb019c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:20.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.585+0000 7f7eb72dd700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb019ce80 con 0x7f7eb01007a0 2023-07-18T03:04:20.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.585+0000 7f7eb72dd700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb019cff0 con 0x7f7eb0104ea0 2023-07-18T03:04:20.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.585+0000 7f7eb72dd700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb019d1b0 con 0x7f7eb00ffe00 2023-07-18T03:04:20.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.585+0000 7f7eb4878700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7eb01007a0 0x7f7eb0198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:20.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.585+0000 7f7eb4878700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7eb01007a0 0x7f7eb0198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:59872/0 (socket says 172.21.15.101:59872) 2023-07-18T03:04:20.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.585+0000 7f7eb587a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7eb0104ea0 0x7f7eb019c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:20.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.585+0000 7f7eb4878700 1 -- 172.21.15.101:0/3587354143 learned_addr learned my addr 172.21.15.101:0/3587354143 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:04:20.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.585+0000 7f7eb5079700 1 --2- 172.21.15.101:0/3587354143 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7eb00ffe00 0x7f7eb0197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:20.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.585+0000 7f7eb4878700 1 -- 172.21.15.101:0/3587354143 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7eb00ffe00 msgr2=0x7f7eb0197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:20.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.585+0000 7f7eb4878700 1 --2- 172.21.15.101:0/3587354143 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7eb00ffe00 0x7f7eb0197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:20.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.585+0000 7f7eb4878700 1 -- 172.21.15.101:0/3587354143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7eb0104ea0 msgr2=0x7f7eb019c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:20.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.585+0000 7f7eb4878700 1 --2- 172.21.15.101:0/3587354143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7eb0104ea0 0x7f7eb019c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:20.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.585+0000 7f7eb4878700 1 -- 172.21.15.101:0/3587354143 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eac0095f0 con 0x7f7eb01007a0 2023-07-18T03:04:20.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.586+0000 7f7eb4878700 1 --2- 172.21.15.101:0/3587354143 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7eb01007a0 0x7f7eb0198400 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7ea000d950 tx=0x7f7ea000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:20.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.586+0000 7f7ea67fc700 1 -- 172.21.15.101:0/3587354143 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0004220 con 0x7f7eb01007a0 2023-07-18T03:04:20.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.586+0000 7f7ea67fc700 1 -- 172.21.15.101:0/3587354143 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ea0007620 con 0x7f7eb01007a0 2023-07-18T03:04:20.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.586+0000 7f7eb72dd700 1 -- 172.21.15.101:0/3587354143 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eb019d490 con 0x7f7eb01007a0 2023-07-18T03:04:20.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.586+0000 7f7ea67fc700 1 -- 172.21.15.101:0/3587354143 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea00056c0 con 0x7f7eb01007a0 2023-07-18T03:04:20.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.586+0000 7f7eb72dd700 1 -- 172.21.15.101:0/3587354143 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eb01a4cf0 con 0x7f7eb01007a0 2023-07-18T03:04:20.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.587+0000 7f7eb72dd700 1 -- 172.21.15.101:0/3587354143 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eb004e210 con 0x7f7eb01007a0 2023-07-18T03:04:20.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.589+0000 7f7ea67fc700 1 -- 172.21.15.101:0/3587354143 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f7ea0004380 con 0x7f7eb01007a0 2023-07-18T03:04:20.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.591+0000 7f7ea67fc700 1 --2- 172.21.15.101:0/3587354143 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f7e9c0a11d0 0x7f7e9c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:20.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.591+0000 7f7ea67fc700 1 -- 172.21.15.101:0/3587354143 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7ea00cc180 con 0x7f7eb01007a0 2023-07-18T03:04:20.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.591+0000 7f7eb5079700 1 --2- 172.21.15.101:0/3587354143 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f7e9c0a11d0 0x7f7e9c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:20.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.591+0000 7f7eb5079700 1 --2- 172.21.15.101:0/3587354143 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f7e9c0a11d0 0x7f7e9c0a3680 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7eac009990 tx=0x7f7eac005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:20.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.595+0000 7f7ea67fc700 1 -- 172.21.15.101:0/3587354143 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7ea0095cd0 con 0x7f7eb01007a0 2023-07-18T03:04:20.775 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.791+0000 7f7eb72dd700 1 -- 172.21.15.101:0/3587354143 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7eb01a5010 con 0x7f7eb01007a0 2023-07-18T03:04:20.775 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.791+0000 7f7ea67fc700 1 -- 172.21.15.101:0/3587354143 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7ea00992f0 con 0x7f7eb01007a0 2023-07-18T03:04:20.775 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 1 2023-07-18T03:04:20.777 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.794+0000 7f7eb72dd700 1 -- 172.21.15.101:0/3587354143 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f7e9c0a11d0 msgr2=0x7f7e9c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:20.777 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.794+0000 7f7eb72dd700 1 --2- 172.21.15.101:0/3587354143 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f7e9c0a11d0 0x7f7e9c0a3680 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7eac009990 tx=0x7f7eac005fb0 comp rx=0 tx=0).stop 2023-07-18T03:04:20.777 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.794+0000 7f7eb72dd700 1 -- 172.21.15.101:0/3587354143 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7eb01007a0 msgr2=0x7f7eb0198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:20.778 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.794+0000 7f7eb72dd700 1 --2- 172.21.15.101:0/3587354143 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7eb01007a0 0x7f7eb0198400 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7ea000d950 tx=0x7f7ea000dc60 comp rx=0 tx=0).stop 2023-07-18T03:04:20.778 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.794+0000 7f7eb72dd700 1 -- 172.21.15.101:0/3587354143 shutdown_connections 2023-07-18T03:04:20.778 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.794+0000 7f7eb72dd700 1 --2- 172.21.15.101:0/3587354143 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f7e9c0a11d0 0x7f7e9c0a3680 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:20.778 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.794+0000 7f7eb72dd700 1 --2- 172.21.15.101:0/3587354143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7eb0104ea0 0x7f7eb019c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:20.778 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.794+0000 7f7eb72dd700 1 --2- 172.21.15.101:0/3587354143 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7eb00ffe00 0x7f7eb0197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:20.778 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.794+0000 7f7eb72dd700 1 --2- 172.21.15.101:0/3587354143 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7eb01007a0 0x7f7eb0198400 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:20.778 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.794+0000 7f7eb72dd700 1 -- 172.21.15.101:0/3587354143 >> 172.21.15.101:0/3587354143 conn(0x7f7eb00fb660 msgr2=0x7f7eb00fc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:20.779 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.794+0000 7f7eb72dd700 1 -- 172.21.15.101:0/3587354143 shutdown_connections 2023-07-18T03:04:20.779 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:20.794+0000 7f7eb72dd700 1 -- 172.21.15.101:0/3587354143 wait complete. 2023-07-18T03:04:21.190 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:20 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/3587354143' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:04:21.227 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:20 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3587354143' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:04:21.284 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:20 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/3587354143' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:04:21.303 INFO:tasks.cephadm:Adding mds.a on smithi101 2023-07-18T03:04:21.303 INFO:tasks.cephadm:Adding mds.d on smithi101 2023-07-18T03:04:21.304 INFO:tasks.cephadm:Adding mds.g on smithi101 2023-07-18T03:04:21.304 INFO:tasks.cephadm:Adding mds.j on smithi101 2023-07-18T03:04:21.304 INFO:tasks.cephadm:Adding mds.b on smithi155 2023-07-18T03:04:21.304 INFO:tasks.cephadm:Adding mds.e on smithi155 2023-07-18T03:04:21.305 INFO:tasks.cephadm:Adding mds.h on smithi155 2023-07-18T03:04:21.305 INFO:tasks.cephadm:Adding mds.k on smithi155 2023-07-18T03:04:21.305 INFO:tasks.cephadm:Adding mds.c on smithi179 2023-07-18T03:04:21.305 INFO:tasks.cephadm:Adding mds.f on smithi179 2023-07-18T03:04:21.305 INFO:tasks.cephadm:Adding mds.i on smithi179 2023-07-18T03:04:21.306 INFO:tasks.cephadm:Adding mds.l on smithi179 2023-07-18T03:04:21.306 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph orch apply mds all '12;smithi101=a;smithi101=d;smithi101=g;smithi101=j;smithi155=b;smithi155=e;smithi155=h;smithi155=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l' 2023-07-18T03:04:21.875 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.c/config 2023-07-18T03:04:22.190 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:21 smithi179 ceph-mon[129229]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:22.227 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:21 smithi101 ceph-mon[126379]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:22.284 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:21 smithi155 ceph-mon[128885]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:23.095 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.112+0000 7fe1e8909700 1 -- 172.21.15.179:0/1651017962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1e40ff740 msgr2=0x7fe1e4106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:23.095 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.112+0000 7fe1e8909700 1 --2- 172.21.15.179:0/1651017962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1e40ff740 0x7fe1e4106ff0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc00b0a0 tx=0x7fe1dc00b3b0 comp rx=0 tx=0).stop 2023-07-18T03:04:23.096 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.112+0000 7fe1e8909700 1 -- 172.21.15.179:0/1651017962 shutdown_connections 2023-07-18T03:04:23.096 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.112+0000 7fe1e8909700 1 --2- 172.21.15.179:0/1651017962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1e40ff740 0x7fe1e4106ff0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:23.096 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.112+0000 7fe1e8909700 1 --2- 172.21.15.179:0/1651017962 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe1e4100090 0x7fe1e41004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:23.096 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.112+0000 7fe1e8909700 1 --2- 172.21.15.179:0/1651017962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1e40fed90 0x7fe1e40ff200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:23.096 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.112+0000 7fe1e8909700 1 -- 172.21.15.179:0/1651017962 >> 172.21.15.179:0/1651017962 conn(0x7fe1e40f9890 msgr2=0x7fe1e40fbca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:23.096 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.112+0000 7fe1e8909700 1 -- 172.21.15.179:0/1651017962 shutdown_connections 2023-07-18T03:04:23.097 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.112+0000 7fe1e8909700 1 -- 172.21.15.179:0/1651017962 wait complete. 2023-07-18T03:04:23.098 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.113+0000 7fe1e8909700 1 Processor -- start 2023-07-18T03:04:23.098 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.113+0000 7fe1e8909700 1 -- start start 2023-07-18T03:04:23.098 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.113+0000 7fe1e8909700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe1e40fed90 0x7fe1e4102b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:23.098 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.113+0000 7fe1e8909700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1e40ff740 0x7fe1e4101180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:23.099 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.113+0000 7fe1e8909700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1e4100090 0x7fe1e41016c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:23.099 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.113+0000 7fe1e8909700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e4101e40 con 0x7fe1e40fed90 2023-07-18T03:04:23.099 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.113+0000 7fe1e8909700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e4101fb0 con 0x7fe1e40ff740 2023-07-18T03:04:23.099 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.113+0000 7fe1e8909700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e419fb70 con 0x7fe1e4100090 2023-07-18T03:04:23.099 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.113+0000 7fe1e2d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1e4100090 0x7fe1e41016c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:23.099 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.113+0000 7fe1e2d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1e4100090 0x7fe1e41016c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48676/0 (socket says 172.21.15.179:48676) 2023-07-18T03:04:23.099 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.113+0000 7fe1e2d9d700 1 -- 172.21.15.179:0/2848313276 learned_addr learned my addr 172.21.15.179:0/2848313276 (peer_addr_for_me v2:172.21.15.179:0/0) 2023-07-18T03:04:23.100 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.113+0000 7fe1e1d9b700 1 --2- 172.21.15.179:0/2848313276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1e40ff740 0x7fe1e4101180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:23.100 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.113+0000 7fe1e2d9d700 1 -- 172.21.15.179:0/2848313276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1e40ff740 msgr2=0x7fe1e4101180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:23.100 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.113+0000 7fe1e2d9d700 1 --2- 172.21.15.179:0/2848313276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1e40ff740 0x7fe1e4101180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:23.100 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.113+0000 7fe1e2d9d700 1 -- 172.21.15.179:0/2848313276 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe1e40fed90 msgr2=0x7fe1e4102b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:23.100 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.113+0000 7fe1e2d9d700 1 --2- 172.21.15.179:0/2848313276 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe1e40fed90 0x7fe1e4102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:23.100 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.113+0000 7fe1e2d9d700 1 -- 172.21.15.179:0/2848313276 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1dc009d00 con 0x7fe1e4100090 2023-07-18T03:04:23.100 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.113+0000 7fe1e1d9b700 1 --2- 172.21.15.179:0/2848313276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1e40ff740 0x7fe1e4101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:04:23.101 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.113+0000 7fe1e2d9d700 1 --2- 172.21.15.179:0/2848313276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1e4100090 0x7fe1e41016c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc00b070 tx=0x7fe1dc004db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:23.101 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.113+0000 7fe1d37fe700 1 -- 172.21.15.179:0/2848313276 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc00e040 con 0x7fe1e4100090 2023-07-18T03:04:23.101 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.114+0000 7fe1e8909700 1 -- 172.21.15.179:0/2848313276 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1e419fdf0 con 0x7fe1e4100090 2023-07-18T03:04:23.101 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.114+0000 7fe1d37fe700 1 -- 172.21.15.179:0/2848313276 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe1dc01bd20 con 0x7fe1e4100090 2023-07-18T03:04:23.101 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.114+0000 7fe1d37fe700 1 -- 172.21.15.179:0/2848313276 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc005910 con 0x7fe1e4100090 2023-07-18T03:04:23.101 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.114+0000 7fe1e8909700 1 -- 172.21.15.179:0/2848313276 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1e41a0280 con 0x7fe1e4100090 2023-07-18T03:04:23.101 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.115+0000 7fe1e8909700 1 -- 172.21.15.179:0/2848313276 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1e410a380 con 0x7fe1e4100090 2023-07-18T03:04:23.102 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.116+0000 7fe1d37fe700 1 -- 172.21.15.179:0/2848313276 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fe1dc005040 con 0x7fe1e4100090 2023-07-18T03:04:23.102 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.116+0000 7fe1d37fe700 1 --2- 172.21.15.179:0/2848313276 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe1cc0a1150 0x7fe1cc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:23.102 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.116+0000 7fe1d37fe700 1 -- 172.21.15.179:0/2848313276 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe1dc0ce9c0 con 0x7fe1e4100090 2023-07-18T03:04:23.103 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.117+0000 7fe1e259c700 1 --2- 172.21.15.179:0/2848313276 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe1cc0a1150 0x7fe1cc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:23.103 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.117+0000 7fe1e259c700 1 --2- 172.21.15.179:0/2848313276 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe1cc0a1150 0x7fe1cc0a3600 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe1d80097b0 tx=0x7fe1d8006d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:23.103 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.120+0000 7fe1d37fe700 1 -- 172.21.15.179:0/2848313276 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe1dc098640 con 0x7fe1e4100090 2023-07-18T03:04:23.298 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.314+0000 7fe1e8909700 1 -- 172.21.15.179:0/2848313276 --> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi101=a;smithi101=d;smithi101=g;smithi101=j;smithi155=b;smithi155=e;smithi155=h;smithi155=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l", "target": ["mon-mgr", ""]}) v1 -- 0x7fe1e40fe530 con 0x7fe1cc0a1150 2023-07-18T03:04:23.308 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.325+0000 7fe1d37fe700 1 -- 172.21.15.179:0/2848313276 <== mgr.14150 v2:172.21.15.101:6800/3963742842 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fe1e40fe530 con 0x7fe1cc0a1150 2023-07-18T03:04:23.308 INFO:teuthology.orchestra.run.smithi179.stdout:Scheduled mds.all update... 2023-07-18T03:04:23.310 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.327+0000 7fe1e8909700 1 -- 172.21.15.179:0/2848313276 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe1cc0a1150 msgr2=0x7fe1cc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:23.310 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.327+0000 7fe1e8909700 1 --2- 172.21.15.179:0/2848313276 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe1cc0a1150 0x7fe1cc0a3600 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe1d80097b0 tx=0x7fe1d8006d20 comp rx=0 tx=0).stop 2023-07-18T03:04:23.311 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.327+0000 7fe1e8909700 1 -- 172.21.15.179:0/2848313276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1e4100090 msgr2=0x7fe1e41016c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:23.311 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.327+0000 7fe1e8909700 1 --2- 172.21.15.179:0/2848313276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1e4100090 0x7fe1e41016c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc00b070 tx=0x7fe1dc004db0 comp rx=0 tx=0).stop 2023-07-18T03:04:23.311 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.327+0000 7fe1e8909700 1 -- 172.21.15.179:0/2848313276 shutdown_connections 2023-07-18T03:04:23.311 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.327+0000 7fe1e8909700 1 --2- 172.21.15.179:0/2848313276 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe1cc0a1150 0x7fe1cc0a3600 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:23.311 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.327+0000 7fe1e8909700 1 --2- 172.21.15.179:0/2848313276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1e4100090 0x7fe1e41016c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:23.311 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.327+0000 7fe1e8909700 1 --2- 172.21.15.179:0/2848313276 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe1e40fed90 0x7fe1e4102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:23.311 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.327+0000 7fe1e8909700 1 --2- 172.21.15.179:0/2848313276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1e40ff740 0x7fe1e4101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:23.312 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.327+0000 7fe1e8909700 1 -- 172.21.15.179:0/2848313276 >> 172.21.15.179:0/2848313276 conn(0x7fe1e40f9890 msgr2=0x7fe1e40fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:23.312 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.327+0000 7fe1e8909700 1 -- 172.21.15.179:0/2848313276 shutdown_connections 2023-07-18T03:04:23.312 INFO:teuthology.orchestra.run.smithi179.stderr:2023-07-18T03:04:23.327+0000 7fe1e8909700 1 -- 172.21.15.179:0/2848313276 wait complete. 2023-07-18T03:04:23.732 DEBUG:teuthology.orchestra.run.smithi101:mds.a> sudo journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mds.a.service 2023-07-18T03:04:23.738 DEBUG:teuthology.orchestra.run.smithi101:mds.d> sudo journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mds.d.service 2023-07-18T03:04:23.784 DEBUG:teuthology.orchestra.run.smithi101:mds.g> sudo journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mds.g.service 2023-07-18T03:04:23.820 INFO:journalctl@ceph.mds.a.smithi101.stdout:-- Logs begin at Tue 2023-07-18 02:35:16 UTC. -- 2023-07-18T03:04:23.823 DEBUG:teuthology.orchestra.run.smithi101:mds.j> sudo journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mds.j.service 2023-07-18T03:04:23.857 INFO:journalctl@ceph.mds.d.smithi101.stdout:-- Logs begin at Tue 2023-07-18 02:35:16 UTC. -- 2023-07-18T03:04:23.860 DEBUG:teuthology.orchestra.run.smithi155:mds.b> sudo journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mds.b.service 2023-07-18T03:04:23.863 DEBUG:teuthology.orchestra.run.smithi155:mds.e> sudo journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mds.e.service 2023-07-18T03:04:23.894 INFO:journalctl@ceph.mds.g.smithi101.stdout:-- Logs begin at Tue 2023-07-18 02:35:16 UTC. -- 2023-07-18T03:04:23.900 INFO:journalctl@ceph.mds.b.smithi155.stdout:-- Logs begin at Tue 2023-07-18 02:29:36 UTC. -- 2023-07-18T03:04:23.901 DEBUG:teuthology.orchestra.run.smithi155:mds.h> sudo journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mds.h.service 2023-07-18T03:04:23.929 INFO:journalctl@ceph.mds.j.smithi101.stdout:-- Logs begin at Tue 2023-07-18 02:35:16 UTC. -- 2023-07-18T03:04:23.930 INFO:journalctl@ceph.mds.e.smithi155.stdout:-- Logs begin at Tue 2023-07-18 02:29:36 UTC. -- 2023-07-18T03:04:23.931 DEBUG:teuthology.orchestra.run.smithi155:mds.k> sudo journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mds.k.service 2023-07-18T03:04:23.957 INFO:journalctl@ceph.mds.h.smithi155.stdout:-- Logs begin at Tue 2023-07-18 02:29:36 UTC. -- 2023-07-18T03:04:23.958 DEBUG:teuthology.orchestra.run.smithi179:mds.c> sudo journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mds.c.service 2023-07-18T03:04:23.961 DEBUG:teuthology.orchestra.run.smithi179:mds.f> sudo journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mds.f.service 2023-07-18T03:04:23.984 INFO:journalctl@ceph.mds.k.smithi155.stdout:-- Logs begin at Tue 2023-07-18 02:29:36 UTC. -- 2023-07-18T03:04:23.989 INFO:journalctl@ceph.mds.c.smithi179.stdout:-- Logs begin at Tue 2023-07-18 02:29:48 UTC. -- 2023-07-18T03:04:23.990 DEBUG:teuthology.orchestra.run.smithi179:mds.i> sudo journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mds.i.service 2023-07-18T03:04:24.016 INFO:journalctl@ceph.mds.f.smithi179.stdout:-- Logs begin at Tue 2023-07-18 02:29:48 UTC. -- 2023-07-18T03:04:24.017 DEBUG:teuthology.orchestra.run.smithi179:mds.l> sudo journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mds.l.service 2023-07-18T03:04:24.042 INFO:journalctl@ceph.mds.i.smithi179.stdout:-- Logs begin at Tue 2023-07-18 02:29:48 UTC. -- 2023-07-18T03:04:24.043 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-07-18T03:04:24.043 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:04:24.068 INFO:journalctl@ceph.mds.l.smithi179.stdout:-- Logs begin at Tue 2023-07-18 02:29:48 UTC. -- 2023-07-18T03:04:24.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.264+0000 7fa09c3ff700 1 -- 172.21.15.101:0/1925140545 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa094101680 msgr2=0x7fa094107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:24.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.264+0000 7fa09c3ff700 1 --2- 172.21.15.101:0/1925140545 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa094101680 0x7fa094107f40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa08c00b0a0 tx=0x7fa08c00b3b0 comp rx=0 tx=0).stop 2023-07-18T03:04:24.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.264+0000 7fa09c3ff700 1 -- 172.21.15.101:0/1925140545 shutdown_connections 2023-07-18T03:04:24.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.264+0000 7fa09c3ff700 1 --2- 172.21.15.101:0/1925140545 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa094101680 0x7fa094107f40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:24.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.264+0000 7fa09c3ff700 1 --2- 172.21.15.101:0/1925140545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0940ffb00 0x7fa0940fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:24.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.264+0000 7fa09c3ff700 1 --2- 172.21.15.101:0/1925140545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa094100cf0 0x7fa094101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:24.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.264+0000 7fa09c3ff700 1 -- 172.21.15.101:0/1925140545 >> 172.21.15.101:0/1925140545 conn(0x7fa0940fb0e0 msgr2=0x7fa0940fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:24.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.264+0000 7fa09c3ff700 1 -- 172.21.15.101:0/1925140545 shutdown_connections 2023-07-18T03:04:24.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.264+0000 7fa09c3ff700 1 -- 172.21.15.101:0/1925140545 wait complete. 2023-07-18T03:04:24.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.265+0000 7fa09c3ff700 1 Processor -- start 2023-07-18T03:04:24.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.265+0000 7fa09c3ff700 1 -- start start 2023-07-18T03:04:24.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.266+0000 7fa09c3ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0940ffb00 0x7fa094197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:24.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.266+0000 7fa09c3ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa094100cf0 0x7fa094198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:24.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.266+0000 7fa09c3ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa094101680 0x7fa09419f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:24.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.266+0000 7fa09c3ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09419f6c0 con 0x7fa094101680 2023-07-18T03:04:24.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.266+0000 7fa09c3ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09419f800 con 0x7fa094100cf0 2023-07-18T03:04:24.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.266+0000 7fa09c3ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09419f990 con 0x7fa0940ffb00 2023-07-18T03:04:24.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.266+0000 7fa09a19b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0940ffb00 0x7fa094197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:24.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.266+0000 7fa09a19b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0940ffb00 0x7fa094197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:44236/0 (socket says 172.21.15.101:44236) 2023-07-18T03:04:24.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.266+0000 7fa09a19b700 1 -- 172.21.15.101:0/947828490 learned_addr learned my addr 172.21.15.101:0/947828490 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:04:24.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.266+0000 7fa09a99c700 1 --2- 172.21.15.101:0/947828490 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa094101680 0x7fa09419f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:24.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.266+0000 7fa09999a700 1 --2- 172.21.15.101:0/947828490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa094100cf0 0x7fa094198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:24.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.266+0000 7fa09a19b700 1 -- 172.21.15.101:0/947828490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa094100cf0 msgr2=0x7fa094198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:24.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.266+0000 7fa09a19b700 1 --2- 172.21.15.101:0/947828490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa094100cf0 0x7fa094198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:24.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.266+0000 7fa09a19b700 1 -- 172.21.15.101:0/947828490 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa094101680 msgr2=0x7fa09419f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:24.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.266+0000 7fa09a19b700 1 --2- 172.21.15.101:0/947828490 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa094101680 0x7fa09419f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:24.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.266+0000 7fa09a19b700 1 -- 172.21.15.101:0/947828490 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa08c009d00 con 0x7fa0940ffb00 2023-07-18T03:04:24.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.266+0000 7fa09999a700 1 --2- 172.21.15.101:0/947828490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa094100cf0 0x7fa094198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:04:24.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.266+0000 7fa09a19b700 1 --2- 172.21.15.101:0/947828490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0940ffb00 0x7fa094197b40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa09000e970 tx=0x7fa09000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:24.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.267+0000 7fa08b7fe700 1 -- 172.21.15.101:0/947828490 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa090009790 con 0x7fa0940ffb00 2023-07-18T03:04:24.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.267+0000 7fa08b7fe700 1 -- 172.21.15.101:0/947828490 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa09000eec0 con 0x7fa0940ffb00 2023-07-18T03:04:24.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.267+0000 7fa08b7fe700 1 -- 172.21.15.101:0/947828490 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa090003cf0 con 0x7fa0940ffb00 2023-07-18T03:04:24.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.267+0000 7fa09c3ff700 1 -- 172.21.15.101:0/947828490 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa09419fc70 con 0x7fa0940ffb00 2023-07-18T03:04:24.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.267+0000 7fa09c3ff700 1 -- 172.21.15.101:0/947828490 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0941a0130 con 0x7fa0940ffb00 2023-07-18T03:04:24.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.268+0000 7fa09c3ff700 1 -- 172.21.15.101:0/947828490 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa09410b330 con 0x7fa0940ffb00 2023-07-18T03:04:24.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.270+0000 7fa08b7fe700 1 -- 172.21.15.101:0/947828490 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fa09000ccd0 con 0x7fa0940ffb00 2023-07-18T03:04:24.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.271+0000 7fa08b7fe700 1 --2- 172.21.15.101:0/947828490 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa0800a11a0 0x7fa0800a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:24.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.271+0000 7fa08b7fe700 1 -- 172.21.15.101:0/947828490 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa090014070 con 0x7fa0940ffb00 2023-07-18T03:04:24.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.271+0000 7fa09999a700 1 --2- 172.21.15.101:0/947828490 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa0800a11a0 0x7fa0800a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:24.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.272+0000 7fa09999a700 1 --2- 172.21.15.101:0/947828490 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa0800a11a0 0x7fa0800a3650 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa084005950 tx=0x7fa0840058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:24.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.273+0000 7fa08b7fe700 1 -- 172.21.15.101:0/947828490 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa090096880 con 0x7fa0940ffb00 2023-07-18T03:04:24.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.472+0000 7fa09c3ff700 1 -- 172.21.15.101:0/947828490 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa09410b520 con 0x7fa0940ffb00 2023-07-18T03:04:24.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.474+0000 7fa08b7fe700 1 -- 172.21.15.101:0/947828490 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18503 (secure 0 0 0) 0x7fa090099ea0 con 0x7fa0940ffb00 2023-07-18T03:04:24.457 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:04:24.457 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":79,"fsid":"b1782daa-2516-11ee-9b34-001a4aab830c","created":"2023-07-18T02:57:22.972483+0000","modified":"2023-07-18T03:04:12.522208+0000","last_up_change":"2023-07-18T03:04:11.429530+0000","last_in_change":"2023-07-18T03:03:54.158340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:00:43.353489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc76b5d1-38a0-4f1a-9335-756bfc40a62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6803","nonce":3610719929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6805","nonce":3610719929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6809","nonce":3610719929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6807","nonce":3610719929}]},"public_addr":"172.21.15.101:6803/3610719929","cluster_addr":"172.21.15.101:6805/3610719929","heartbeat_back_addr":"172.21.15.101:6809/3610719929","heartbeat_front_addr":"172.21.15.101:6807/3610719929","state":["exists","up"]},{"osd":1,"uuid":"2a299573-0b34-4af9-aaae-b523e5058ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6801","nonce":1030229579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6803","nonce":1030229579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6807","nonce":1030229579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6805","nonce":1030229579}]},"public_addr":"172.21.15.155:6801/1030229579","cluster_addr":"172.21.15.155:6803/1030229579","heartbeat_back_addr":"172.21.15.155:6807/1030229579","heartbeat_front_addr":"172.21.15.155:6805/1030229579","state":["exists","up"]},{"osd":2,"uuid":"123b3749-c636-48fa-9e33-6229b5575dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6801","nonce":1349487507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6803","nonce":1349487507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6807","nonce":1349487507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6805","nonce":1349487507}]},"public_addr":"172.21.15.179:6801/1349487507","cluster_addr":"172.21.15.179:6803/1349487507","heartbeat_back_addr":"172.21.15.179:6807/1349487507","heartbeat_front_addr":"172.21.15.179:6805/1349487507","state":["exists","up"]},{"osd":3,"uuid":"4b0c409d-2954-4127-833a-b196bf226d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6811","nonce":4226577463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6813","nonce":4226577463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6817","nonce":4226577463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6815","nonce":4226577463}]},"public_addr":"172.21.15.101:6811/4226577463","cluster_addr":"172.21.15.101:6813/4226577463","heartbeat_back_addr":"172.21.15.101:6817/4226577463","heartbeat_front_addr":"172.21.15.101:6815/4226577463","state":["exists","up"]},{"osd":4,"uuid":"2137f2df-984e-45ce-baf6-b4ac12c7e3c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6809","nonce":1951135805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6811","nonce":1951135805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6815","nonce":1951135805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6813","nonce":1951135805}]},"public_addr":"172.21.15.155:6809/1951135805","cluster_addr":"172.21.15.155:6811/1951135805","heartbeat_back_addr":"172.21.15.155:6815/1951135805","heartbeat_front_addr":"172.21.15.155:6813/1951135805","state":["exists","up"]},{"osd":5,"uuid":"1020bfc5-8c2f-482e-9588-66c7f98e46e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6809","nonce":2746037021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6811","nonce":2746037021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6815","nonce":2746037021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6813","nonce":2746037021}]},"public_addr":"172.21.15.179:6809/2746037021","cluster_addr":"172.21.15.179:6811/2746037021","heartbeat_back_addr":"172.21.15.179:6815/2746037021","heartbeat_front_addr":"172.21.15.179:6813/2746037021","state":["exists","up"]},{"osd":6,"uuid":"2cd99260-5c36-4603-9338-cf697970ce58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6819","nonce":958044931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6821","nonce":958044931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6825","nonce":958044931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6823","nonce":958044931}]},"public_addr":"172.21.15.101:6819/958044931","cluster_addr":"172.21.15.101:6821/958044931","heartbeat_back_addr":"172.21.15.101:6825/958044931","heartbeat_front_addr":"172.21.15.101:6823/958044931","state":["exists","up"]},{"osd":7,"uuid":"ed23fd0a-678f-4e15-b42c-cfe873315714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6817","nonce":3256322291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6819","nonce":3256322291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6823","nonce":3256322291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6821","nonce":3256322291}]},"public_addr":"172.21.15.155:6817/3256322291","cluster_addr":"172.21.15.155:6819/3256322291","heartbeat_back_addr":"172.21.15.155:6823/3256322291","heartbeat_front_addr":"172.21.15.155:6821/3256322291","state":["exists","up"]},{"osd":8,"uuid":"eca3d8e6-7baf-47a5-b991-328001c7ebef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6817","nonce":3520612559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6819","nonce":3520612559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6823","nonce":3520612559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6821","nonce":3520612559}]},"public_addr":"172.21.15.179:6817/3520612559","cluster_addr":"172.21.15.179:6819/3520612559","heartbeat_back_addr":"172.21.15.179:6823/3520612559","heartbeat_front_addr":"172.21.15.179:6821/3520612559","state":["exists","up"]},{"osd":9,"uuid":"465a7ffb-67e4-49ef-812c-1b96feacfebf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6827","nonce":2166787667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6829","nonce":2166787667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6833","nonce":2166787667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6831","nonce":2166787667}]},"public_addr":"172.21.15.101:6827/2166787667","cluster_addr":"172.21.15.101:6829/2166787667","heartbeat_back_addr":"172.21.15.101:6833/2166787667","heartbeat_front_addr":"172.21.15.101:6831/2166787667","state":["exists","up"]},{"osd":10,"uuid":"1897a027-f25d-434f-8df5-5fc2bd866578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6825","nonce":3682446349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6827","nonce":3682446349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6831","nonce":3682446349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6829","nonce":3682446349}]},"public_addr":"172.21.15.155:6825/3682446349","cluster_addr":"172.21.15.155:6827/3682446349","heartbeat_back_addr":"172.21.15.155:6831/3682446349","heartbeat_front_addr":"172.21.15.155:6829/3682446349","state":["exists","up"]},{"osd":11,"uuid":"b62effaf-45e2-4585-b8d0-db448f3474ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6825","nonce":2602640858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6827","nonce":2602640858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6831","nonce":2602640858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6829","nonce":2602640858}]},"public_addr":"172.21.15.179:6825/2602640858","cluster_addr":"172.21.15.179:6827/2602640858","heartbeat_back_addr":"172.21.15.179:6831/2602640858","heartbeat_front_addr":"172.21.15.179:6829/2602640858","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T02:59:56.074519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:19.358142+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:41.128966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:08.227054+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:31.442592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:52.751685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:19.251054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:40.698241+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:01.339150+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:27.955910+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:49.136980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:04:10.084982+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.101:6800/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:6801/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/604228291":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/4055187849":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/2844574324":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/1603248215":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6801/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6800/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/3307578182":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/373189792":"2023-07-19T02:57:56.243200+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:04:24.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.475+0000 7fa09c3ff700 1 -- 172.21.15.101:0/947828490 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa0800a11a0 msgr2=0x7fa0800a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:24.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.475+0000 7fa09c3ff700 1 --2- 172.21.15.101:0/947828490 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa0800a11a0 0x7fa0800a3650 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa084005950 tx=0x7fa0840058e0 comp rx=0 tx=0).stop 2023-07-18T03:04:24.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.475+0000 7fa09c3ff700 1 -- 172.21.15.101:0/947828490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0940ffb00 msgr2=0x7fa094197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:24.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.475+0000 7fa09c3ff700 1 --2- 172.21.15.101:0/947828490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0940ffb00 0x7fa094197b40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa09000e970 tx=0x7fa09000ec80 comp rx=0 tx=0).stop 2023-07-18T03:04:24.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.476+0000 7fa09c3ff700 1 -- 172.21.15.101:0/947828490 shutdown_connections 2023-07-18T03:04:24.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.476+0000 7fa09c3ff700 1 --2- 172.21.15.101:0/947828490 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa0800a11a0 0x7fa0800a3650 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:24.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.476+0000 7fa09c3ff700 1 --2- 172.21.15.101:0/947828490 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa094101680 0x7fa09419f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:24.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.476+0000 7fa09c3ff700 1 --2- 172.21.15.101:0/947828490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0940ffb00 0x7fa094197b40 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:24.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.476+0000 7fa09c3ff700 1 --2- 172.21.15.101:0/947828490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa094100cf0 0x7fa094198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:24.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.476+0000 7fa09c3ff700 1 -- 172.21.15.101:0/947828490 >> 172.21.15.101:0/947828490 conn(0x7fa0940fb0e0 msgr2=0x7fa094106530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:24.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.476+0000 7fa09c3ff700 1 -- 172.21.15.101:0/947828490 shutdown_connections 2023-07-18T03:04:24.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.476+0000 7fa09c3ff700 1 -- 172.21.15.101:0/947828490 wait complete. 2023-07-18T03:04:24.477 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:24 smithi101 ceph-mon[126379]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:24.477 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:24 smithi101 ceph-mon[126379]: from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi101=a;smithi101=d;smithi101=g;smithi101=j;smithi155=b;smithi155=e;smithi155=h;smithi155=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:04:24.477 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:24 smithi101 ceph-mon[126379]: Saving service mds.all spec with placement smithi101=a;smithi101=d;smithi101=g;smithi101=j;smithi155=b;smithi155=e;smithi155=h;smithi155=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l;count:12 2023-07-18T03:04:24.477 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:24 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:24.478 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:24 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:04:24.478 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:24 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:24.478 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:24 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:04:24.478 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:24 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:24.478 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:24 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:04:24.478 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:24 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:04:24.478 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:24 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:24.478 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:24 smithi101 ceph-mon[126379]: Deploying daemon mds.l on smithi179 2023-07-18T03:04:24.480 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:00:43.353489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-18T03:04:24.480 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-18T03:04:24.518 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:24 smithi179 ceph-mon[129229]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:24.519 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:24 smithi179 ceph-mon[129229]: from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi101=a;smithi101=d;smithi101=g;smithi101=j;smithi155=b;smithi155=e;smithi155=h;smithi155=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:04:24.519 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:24 smithi179 ceph-mon[129229]: Saving service mds.all spec with placement smithi101=a;smithi101=d;smithi101=g;smithi101=j;smithi155=b;smithi155=e;smithi155=h;smithi155=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l;count:12 2023-07-18T03:04:24.519 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:24 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:24.519 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:24 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:04:24.520 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:24 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:24.520 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:24 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:04:24.521 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:24 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:24.521 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:24 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:04:24.521 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:24 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:04:24.521 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:24 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:24.522 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:24 smithi179 ceph-mon[129229]: Deploying daemon mds.l on smithi179 2023-07-18T03:04:24.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.703+0000 7fc235dff700 1 -- 172.21.15.101:0/1743197902 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc2300ffaf0 msgr2=0x7fc2300fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:24.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.703+0000 7fc235dff700 1 --2- 172.21.15.101:0/1743197902 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc2300ffaf0 0x7fc2300fff00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc220009990 tx=0x7fc220009ca0 comp rx=0 tx=0).stop 2023-07-18T03:04:24.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.703+0000 7fc235dff700 1 -- 172.21.15.101:0/1743197902 shutdown_connections 2023-07-18T03:04:24.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.703+0000 7fc235dff700 1 --2- 172.21.15.101:0/1743197902 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc230101670 0x7fc230107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:24.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.703+0000 7fc235dff700 1 --2- 172.21.15.101:0/1743197902 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc2300ffaf0 0x7fc2300fff00 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:24.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.703+0000 7fc235dff700 1 --2- 172.21.15.101:0/1743197902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc230100ce0 0x7fc230101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:24.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.703+0000 7fc235dff700 1 -- 172.21.15.101:0/1743197902 >> 172.21.15.101:0/1743197902 conn(0x7fc2300fb130 msgr2=0x7fc2300fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:24.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.703+0000 7fc235dff700 1 -- 172.21.15.101:0/1743197902 shutdown_connections 2023-07-18T03:04:24.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.703+0000 7fc235dff700 1 -- 172.21.15.101:0/1743197902 wait complete. 2023-07-18T03:04:24.690 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.704+0000 7fc235dff700 1 Processor -- start 2023-07-18T03:04:24.690 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.704+0000 7fc235dff700 1 -- start start 2023-07-18T03:04:24.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.705+0000 7fc235dff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2300ffaf0 0x7fc230197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:24.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.705+0000 7fc235dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc230100ce0 0x7fc230197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:24.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.705+0000 7fc235dff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc230101670 0x7fc23019f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:24.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.705+0000 7fc235dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23019f5e0 con 0x7fc230100ce0 2023-07-18T03:04:24.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.705+0000 7fc235dff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23019f720 con 0x7fc230101670 2023-07-18T03:04:24.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.705+0000 7fc235dff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23019f8b0 con 0x7fc2300ffaf0 2023-07-18T03:04:24.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.705+0000 7fc22f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2300ffaf0 0x7fc230197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:24.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.705+0000 7fc22effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc230100ce0 0x7fc230197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:24.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.705+0000 7fc22f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2300ffaf0 0x7fc230197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:44260/0 (socket says 172.21.15.101:44260) 2023-07-18T03:04:24.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.705+0000 7fc22f7fe700 1 -- 172.21.15.101:0/1315096120 learned_addr learned my addr 172.21.15.101:0/1315096120 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:04:24.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.705+0000 7fc22ffff700 1 --2- 172.21.15.101:0/1315096120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc230101670 0x7fc23019f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:24.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.705+0000 7fc22f7fe700 1 -- 172.21.15.101:0/1315096120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc230101670 msgr2=0x7fc23019f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:24.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.705+0000 7fc22f7fe700 1 --2- 172.21.15.101:0/1315096120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc230101670 0x7fc23019f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:24.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.705+0000 7fc22f7fe700 1 -- 172.21.15.101:0/1315096120 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc230100ce0 msgr2=0x7fc230197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:24.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.705+0000 7fc22f7fe700 1 --2- 172.21.15.101:0/1315096120 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc230100ce0 0x7fc230197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:24.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.705+0000 7fc22f7fe700 1 -- 172.21.15.101:0/1315096120 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2200095f0 con 0x7fc2300ffaf0 2023-07-18T03:04:24.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.706+0000 7fc22f7fe700 1 --2- 172.21.15.101:0/1315096120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2300ffaf0 0x7fc230197a60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc22000b620 tx=0x7fc220005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:24.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.706+0000 7fc22cff9700 1 -- 172.21.15.101:0/1315096120 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc22001d070 con 0x7fc2300ffaf0 2023-07-18T03:04:24.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.706+0000 7fc22cff9700 1 -- 172.21.15.101:0/1315096120 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc220021410 con 0x7fc2300ffaf0 2023-07-18T03:04:24.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.706+0000 7fc235dff700 1 -- 172.21.15.101:0/1315096120 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc23019fb30 con 0x7fc2300ffaf0 2023-07-18T03:04:24.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.706+0000 7fc22cff9700 1 -- 172.21.15.101:0/1315096120 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc220017650 con 0x7fc2300ffaf0 2023-07-18T03:04:24.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.706+0000 7fc235dff700 1 -- 172.21.15.101:0/1315096120 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc23019ff50 con 0x7fc2300ffaf0 2023-07-18T03:04:24.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.707+0000 7fc235dff700 1 -- 172.21.15.101:0/1315096120 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2301921d0 con 0x7fc2300ffaf0 2023-07-18T03:04:24.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.709+0000 7fc22cff9700 1 -- 172.21.15.101:0/1315096120 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fc220021580 con 0x7fc2300ffaf0 2023-07-18T03:04:24.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.710+0000 7fc22cff9700 1 --2- 172.21.15.101:0/1315096120 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc2180a1150 0x7fc2180a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:24.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.710+0000 7fc22cff9700 1 -- 172.21.15.101:0/1315096120 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc2200cda30 con 0x7fc2300ffaf0 2023-07-18T03:04:24.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.711+0000 7fc22effd700 1 --2- 172.21.15.101:0/1315096120 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc2180a1150 0x7fc2180a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:24.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.711+0000 7fc22effd700 1 --2- 172.21.15.101:0/1315096120 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc2180a1150 0x7fc2180a3600 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc224005950 tx=0x7fc22400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:24.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.714+0000 7fc22cff9700 1 -- 172.21.15.101:0/1315096120 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc220097610 con 0x7fc2300ffaf0 2023-07-18T03:04:24.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:24 smithi155 ceph-mon[128885]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:24.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:24 smithi155 ceph-mon[128885]: from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi101=a;smithi101=d;smithi101=g;smithi101=j;smithi155=b;smithi155=e;smithi155=h;smithi155=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:04:24.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:24 smithi155 ceph-mon[128885]: Saving service mds.all spec with placement smithi101=a;smithi101=d;smithi101=g;smithi101=j;smithi155=b;smithi155=e;smithi155=h;smithi155=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l;count:12 2023-07-18T03:04:24.786 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:24 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:24.786 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:24 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:04:24.786 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:24 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:24.786 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:24 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:04:24.786 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:24 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:24.786 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:24 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:04:24.786 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:24 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:04:24.786 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:24 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:24.787 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:24 smithi155 ceph-mon[128885]: Deploying daemon mds.l on smithi179 2023-07-18T03:04:24.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.911+0000 7fc235dff700 1 -- 172.21.15.101:0/1315096120 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc230061120 con 0x7fc2300ffaf0 2023-07-18T03:04:24.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.911+0000 7fc22cff9700 1 -- 172.21.15.101:0/1315096120 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc22009ac30 con 0x7fc2300ffaf0 2023-07-18T03:04:24.895 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 1 2023-07-18T03:04:24.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.914+0000 7fc235dff700 1 -- 172.21.15.101:0/1315096120 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc2180a1150 msgr2=0x7fc2180a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:24.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.914+0000 7fc235dff700 1 --2- 172.21.15.101:0/1315096120 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc2180a1150 0x7fc2180a3600 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc224005950 tx=0x7fc22400b410 comp rx=0 tx=0).stop 2023-07-18T03:04:24.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.914+0000 7fc235dff700 1 -- 172.21.15.101:0/1315096120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2300ffaf0 msgr2=0x7fc230197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:24.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.914+0000 7fc235dff700 1 --2- 172.21.15.101:0/1315096120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2300ffaf0 0x7fc230197a60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc22000b620 tx=0x7fc220005740 comp rx=0 tx=0).stop 2023-07-18T03:04:24.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.914+0000 7fc235dff700 1 -- 172.21.15.101:0/1315096120 shutdown_connections 2023-07-18T03:04:24.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.914+0000 7fc235dff700 1 --2- 172.21.15.101:0/1315096120 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc2180a1150 0x7fc2180a3600 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:24.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.914+0000 7fc235dff700 1 --2- 172.21.15.101:0/1315096120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc230101670 0x7fc23019f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:24.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.914+0000 7fc235dff700 1 --2- 172.21.15.101:0/1315096120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2300ffaf0 0x7fc230197a60 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:24.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.914+0000 7fc235dff700 1 --2- 172.21.15.101:0/1315096120 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc230100ce0 0x7fc230197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:24.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.914+0000 7fc235dff700 1 -- 172.21.15.101:0/1315096120 >> 172.21.15.101:0/1315096120 conn(0x7fc2300fb130 msgr2=0x7fc2301065c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:24.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.914+0000 7fc235dff700 1 -- 172.21.15.101:0/1315096120 shutdown_connections 2023-07-18T03:04:24.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:24.914+0000 7fc235dff700 1 -- 172.21.15.101:0/1315096120 wait complete. 2023-07-18T03:04:24.917 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:04:24.940 INFO:journalctl@ceph.mds.l.smithi179.stdout:Jul 18 03:04:24 smithi179 systemd[1]: Starting Ceph mds.l for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T03:04:25.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.130+0000 7f27233ff700 1 -- 172.21.15.101:0/791874496 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f271c100cf0 msgr2=0x7f271c101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:25.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.130+0000 7f27233ff700 1 --2- 172.21.15.101:0/791874496 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f271c100cf0 0x7f271c101140 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f270c009990 tx=0x7f270c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:04:25.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.130+0000 7f27233ff700 1 -- 172.21.15.101:0/791874496 shutdown_connections 2023-07-18T03:04:25.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.130+0000 7f27233ff700 1 --2- 172.21.15.101:0/791874496 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f271c0ffb00 0x7f271c0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:25.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.130+0000 7f27233ff700 1 --2- 172.21.15.101:0/791874496 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f271c101680 0x7f271c107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:25.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.130+0000 7f27233ff700 1 --2- 172.21.15.101:0/791874496 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f271c100cf0 0x7f271c101140 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:25.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.130+0000 7f27233ff700 1 -- 172.21.15.101:0/791874496 >> 172.21.15.101:0/791874496 conn(0x7f271c0fb0e0 msgr2=0x7f271c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:25.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.130+0000 7f27233ff700 1 -- 172.21.15.101:0/791874496 shutdown_connections 2023-07-18T03:04:25.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.130+0000 7f27233ff700 1 -- 172.21.15.101:0/791874496 wait complete. 2023-07-18T03:04:25.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.131+0000 7f27233ff700 1 Processor -- start 2023-07-18T03:04:25.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.131+0000 7f27233ff700 1 -- start start 2023-07-18T03:04:25.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.132+0000 7f27233ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f271c0ffb00 0x7f271c0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:25.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.132+0000 7f27233ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f271c100cf0 0x7f271c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:25.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.132+0000 7f27233ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f271c101680 0x7f271c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:25.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.132+0000 7f27233ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271c0735e0 con 0x7f271c101680 2023-07-18T03:04:25.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.132+0000 7f27233ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271c073750 con 0x7f271c0ffb00 2023-07-18T03:04:25.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.132+0000 7f27233ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271c19f890 con 0x7f271c100cf0 2023-07-18T03:04:25.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.132+0000 7f272099a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f271c100cf0 0x7f271c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:25.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.132+0000 7f272099a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f271c100cf0 0x7f271c072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:44282/0 (socket says 172.21.15.101:44282) 2023-07-18T03:04:25.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.132+0000 7f272099a700 1 -- 172.21.15.101:0/1893443176 learned_addr learned my addr 172.21.15.101:0/1893443176 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:04:25.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.132+0000 7f272199c700 1 --2- 172.21.15.101:0/1893443176 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f271c101680 0x7f271c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:25.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.132+0000 7f272119b700 1 --2- 172.21.15.101:0/1893443176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f271c0ffb00 0x7f271c0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:25.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.132+0000 7f272099a700 1 -- 172.21.15.101:0/1893443176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f271c0ffb00 msgr2=0x7f271c0742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:25.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.132+0000 7f272099a700 1 --2- 172.21.15.101:0/1893443176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f271c0ffb00 0x7f271c0742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:25.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.132+0000 7f272099a700 1 -- 172.21.15.101:0/1893443176 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f271c101680 msgr2=0x7f271c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:25.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.132+0000 7f272099a700 1 --2- 172.21.15.101:0/1893443176 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f271c101680 0x7f271c072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:25.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.132+0000 7f272099a700 1 -- 172.21.15.101:0/1893443176 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f270c0095f0 con 0x7f271c100cf0 2023-07-18T03:04:25.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.133+0000 7f272199c700 1 --2- 172.21.15.101:0/1893443176 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f271c101680 0x7f271c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:04:25.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.133+0000 7f272099a700 1 --2- 172.21.15.101:0/1893443176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f271c100cf0 0x7f271c072900 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f270c00b620 tx=0x7f270c005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:25.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.133+0000 7f27127fc700 1 -- 172.21.15.101:0/1893443176 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270c01d070 con 0x7f271c100cf0 2023-07-18T03:04:25.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.133+0000 7f27127fc700 1 -- 172.21.15.101:0/1893443176 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f270c022470 con 0x7f271c100cf0 2023-07-18T03:04:25.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.133+0000 7f27233ff700 1 -- 172.21.15.101:0/1893443176 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f271c19fb10 con 0x7f271c100cf0 2023-07-18T03:04:25.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.133+0000 7f27127fc700 1 -- 172.21.15.101:0/1893443176 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270c017650 con 0x7f271c100cf0 2023-07-18T03:04:25.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.133+0000 7f27233ff700 1 -- 172.21.15.101:0/1893443176 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f271c19ff30 con 0x7f271c100cf0 2023-07-18T03:04:25.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.134+0000 7f27233ff700 1 -- 172.21.15.101:0/1893443176 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f271c10b320 con 0x7f271c100cf0 2023-07-18T03:04:25.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.136+0000 7f27127fc700 1 -- 172.21.15.101:0/1893443176 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f270c0225e0 con 0x7f271c100cf0 2023-07-18T03:04:25.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.137+0000 7f27127fc700 1 --2- 172.21.15.101:0/1893443176 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f27080a11a0 0x7f27080a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:25.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.138+0000 7f27127fc700 1 -- 172.21.15.101:0/1893443176 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f270c0cdab0 con 0x7f271c100cf0 2023-07-18T03:04:25.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.138+0000 7f272119b700 1 --2- 172.21.15.101:0/1893443176 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f27080a11a0 0x7f27080a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:25.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.138+0000 7f272119b700 1 --2- 172.21.15.101:0/1893443176 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f27080a11a0 0x7f27080a3650 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f271c100b50 tx=0x7f2718009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:25.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.139+0000 7f27127fc700 1 -- 172.21.15.101:0/1893443176 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f270c097690 con 0x7f271c100cf0 2023-07-18T03:04:25.315 INFO:journalctl@ceph.mds.l.smithi179.stdout:Jul 18 03:04:25 smithi179 podman[154274]: 2023-07-18T03:04:25.316 INFO:journalctl@ceph.mds.l.smithi179.stdout:Jul 18 03:04:25 smithi179 ceph-mds[154314]: -- 172.21.15.179:0/1705550088 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564f657fd1e0 con 0x564f657d2800 2023-07-18T03:04:25.316 INFO:journalctl@ceph.mds.l.smithi179.stdout:Jul 18 03:04:25 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-l[154290]: starting mds.l at 2023-07-18T03:04:25.316 INFO:journalctl@ceph.mds.l.smithi179.stdout:Jul 18 03:04:25 smithi179 bash[154274]: f4c50de6ca5496633de862794729657bca1af16acd7b0e1f8b1e7c4242c1b58e 2023-07-18T03:04:25.373 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.389+0000 7f27233ff700 1 -- 172.21.15.101:0/1893443176 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f271c106990 con 0x7f271c100cf0 2023-07-18T03:04:25.374 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.390+0000 7f27127fc700 1 -- 172.21.15.101:0/1893443176 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1260 (secure 0 0 0) 0x7f270c09acb0 con 0x7f271c100cf0 2023-07-18T03:04:25.374 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:04:25.374 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-07-18T03:04:25.376 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.393+0000 7f27233ff700 1 -- 172.21.15.101:0/1893443176 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f27080a11a0 msgr2=0x7f27080a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:25.376 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.393+0000 7f27233ff700 1 --2- 172.21.15.101:0/1893443176 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f27080a11a0 0x7f27080a3650 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f271c100b50 tx=0x7f2718009260 comp rx=0 tx=0).stop 2023-07-18T03:04:25.377 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.393+0000 7f27233ff700 1 -- 172.21.15.101:0/1893443176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f271c100cf0 msgr2=0x7f271c072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:25.377 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.393+0000 7f27233ff700 1 --2- 172.21.15.101:0/1893443176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f271c100cf0 0x7f271c072900 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f270c00b620 tx=0x7f270c005740 comp rx=0 tx=0).stop 2023-07-18T03:04:25.377 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.393+0000 7f27233ff700 1 -- 172.21.15.101:0/1893443176 shutdown_connections 2023-07-18T03:04:25.377 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.393+0000 7f27233ff700 1 --2- 172.21.15.101:0/1893443176 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f27080a11a0 0x7f27080a3650 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:25.377 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.393+0000 7f27233ff700 1 --2- 172.21.15.101:0/1893443176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f271c0ffb00 0x7f271c0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:25.377 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.393+0000 7f27233ff700 1 --2- 172.21.15.101:0/1893443176 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f271c101680 0x7f271c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:25.377 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.393+0000 7f27233ff700 1 --2- 172.21.15.101:0/1893443176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f271c100cf0 0x7f271c072900 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:25.378 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.393+0000 7f27233ff700 1 -- 172.21.15.101:0/1893443176 >> 172.21.15.101:0/1893443176 conn(0x7f271c0fb0e0 msgr2=0x7f271c0fd330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:25.378 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.393+0000 7f27233ff700 1 -- 172.21.15.101:0/1893443176 shutdown_connections 2023-07-18T03:04:25.378 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:25.393+0000 7f27233ff700 1 -- 172.21.15.101:0/1893443176 wait complete. 2023-07-18T03:04:25.378 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 2 2023-07-18T03:04:25.690 INFO:journalctl@ceph.mds.l.smithi179.stdout:Jul 18 03:04:25 smithi179 systemd[1]: Started Ceph mds.l for b1782daa-2516-11ee-9b34-001a4aab830c. 2023-07-18T03:04:25.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:25 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/947828490' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:04:25.691 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:25 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1315096120' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:04:25.691 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:25 smithi179 ceph-mon[129229]: mds.? [v2:172.21.15.179:6832/2688212803,v1:172.21.15.179:6833/2688212803] up:boot 2023-07-18T03:04:25.691 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:25 smithi179 ceph-mon[129229]: fsmap 1 up:standby 2023-07-18T03:04:25.691 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:25 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-18T03:04:25.727 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:25 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/947828490' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:04:25.728 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:25 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1315096120' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:04:25.728 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:25 smithi101 ceph-mon[126379]: mds.? [v2:172.21.15.179:6832/2688212803,v1:172.21.15.179:6833/2688212803] up:boot 2023-07-18T03:04:25.728 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:25 smithi101 ceph-mon[126379]: fsmap 1 up:standby 2023-07-18T03:04:25.728 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:25 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-18T03:04:25.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:25 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/947828490' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:04:25.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:25 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1315096120' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:04:25.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:25 smithi155 ceph-mon[128885]: mds.? [v2:172.21.15.179:6832/2688212803,v1:172.21.15.179:6833/2688212803] up:boot 2023-07-18T03:04:25.786 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:25 smithi155 ceph-mon[128885]: fsmap 1 up:standby 2023-07-18T03:04:25.786 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:25 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-18T03:04:26.388 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:26 smithi155 ceph-mon[128885]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:26.388 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:26 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1893443176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:04:26.388 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:26 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:26.388 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:26 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:26.388 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:26 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:26.389 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:26 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:04:26.389 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:26 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:04:26.389 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:26 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:26.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:26 smithi179 ceph-mon[129229]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:26.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:26 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1893443176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:04:26.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:26 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:26.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:26 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:26.691 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:26 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:26.691 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:26 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:04:26.691 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:26 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:04:26.691 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:26 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:26.727 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:26 smithi101 ceph-mon[126379]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:26.728 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:26 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1893443176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:04:26.728 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:26 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:26.728 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:26 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:26.728 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:26 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:26.728 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:26 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:04:26.728 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:26 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:04:26.729 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:26 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:27.035 INFO:journalctl@ceph.mds.k.smithi155.stdout:Jul 18 03:04:26 smithi155 systemd[1]: Starting Ceph mds.k for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T03:04:27.397 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:04:27.452 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:27 smithi155 ceph-mon[128885]: Deploying daemon mds.k on smithi155 2023-07-18T03:04:27.452 INFO:journalctl@ceph.mds.k.smithi155.stdout:Jul 18 03:04:27 smithi155 podman[153112]: 2023-07-18T03:04:27.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.629+0000 7f18899ff700 1 -- 172.21.15.101:0/1247352746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1884072f70 msgr2=0x7f18840733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:27.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.629+0000 7f18899ff700 1 --2- 172.21.15.101:0/1247352746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1884072f70 0x7f18840733e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f18780098f0 tx=0x7f1878009c00 comp rx=0 tx=0).stop 2023-07-18T03:04:27.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.629+0000 7f18899ff700 1 -- 172.21.15.101:0/1247352746 shutdown_connections 2023-07-18T03:04:27.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.629+0000 7f18899ff700 1 --2- 172.21.15.101:0/1247352746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f188410c0d0 0x7f188410e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:27.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.629+0000 7f18899ff700 1 --2- 172.21.15.101:0/1247352746 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1884074550 0x7f18840729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:27.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.629+0000 7f18899ff700 1 --2- 172.21.15.101:0/1247352746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1884072f70 0x7f18840733e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:27.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.629+0000 7f18899ff700 1 -- 172.21.15.101:0/1247352746 >> 172.21.15.101:0/1247352746 conn(0x7f18840fb0e0 msgr2=0x7f18840fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:27.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.629+0000 7f18899ff700 1 -- 172.21.15.101:0/1247352746 shutdown_connections 2023-07-18T03:04:27.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.629+0000 7f18899ff700 1 -- 172.21.15.101:0/1247352746 wait complete. 2023-07-18T03:04:27.616 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.630+0000 7f18899ff700 1 Processor -- start 2023-07-18T03:04:27.616 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.630+0000 7f18899ff700 1 -- start start 2023-07-18T03:04:27.616 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.631+0000 7f18899ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1884072f70 0x7f188419bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:27.616 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.631+0000 7f18899ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1884074550 0x7f188419c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:27.616 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.631+0000 7f18899ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f188410c0d0 0x7f18841a3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:27.616 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.631+0000 7f18899ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18841a3a50 con 0x7f1884072f70 2023-07-18T03:04:27.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.631+0000 7f18899ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18841a3b90 con 0x7f1884074550 2023-07-18T03:04:27.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.631+0000 7f18899ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18841a3d20 con 0x7f188410c0d0 2023-07-18T03:04:27.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.631+0000 7f1882ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1884072f70 0x7f188419bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:27.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.631+0000 7f1882ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1884072f70 0x7f188419bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:49860/0 (socket says 172.21.15.101:49860) 2023-07-18T03:04:27.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.631+0000 7f1882ffd700 1 -- 172.21.15.101:0/2572920798 learned_addr learned my addr 172.21.15.101:0/2572920798 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:04:27.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.631+0000 7f18827fc700 1 --2- 172.21.15.101:0/2572920798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1884074550 0x7f188419c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:27.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.631+0000 7f18837fe700 1 --2- 172.21.15.101:0/2572920798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f188410c0d0 0x7f18841a3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:27.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.631+0000 7f1882ffd700 1 -- 172.21.15.101:0/2572920798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f188410c0d0 msgr2=0x7f18841a3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:27.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.631+0000 7f1882ffd700 1 --2- 172.21.15.101:0/2572920798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f188410c0d0 0x7f18841a3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:27.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.631+0000 7f1882ffd700 1 -- 172.21.15.101:0/2572920798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1884074550 msgr2=0x7f188419c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:27.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.631+0000 7f1882ffd700 1 --2- 172.21.15.101:0/2572920798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1884074550 0x7f188419c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:27.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.631+0000 7f1882ffd700 1 -- 172.21.15.101:0/2572920798 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1878009520 con 0x7f1884072f70 2023-07-18T03:04:27.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.632+0000 7f1882ffd700 1 --2- 172.21.15.101:0/2572920798 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1884072f70 0x7f188419bed0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f187400e9f0 tx=0x7f187400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:27.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.632+0000 7f18889fd700 1 -- 172.21.15.101:0/2572920798 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1874009690 con 0x7f1884072f70 2023-07-18T03:04:27.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.632+0000 7f18889fd700 1 -- 172.21.15.101:0/2572920798 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f187400ce80 con 0x7f1884072f70 2023-07-18T03:04:27.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.632+0000 7f18889fd700 1 -- 172.21.15.101:0/2572920798 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1874018420 con 0x7f1884072f70 2023-07-18T03:04:27.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.632+0000 7f18899ff700 1 -- 172.21.15.101:0/2572920798 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18841a3fa0 con 0x7f1884072f70 2023-07-18T03:04:27.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.632+0000 7f18899ff700 1 -- 172.21.15.101:0/2572920798 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18841a4420 con 0x7f1884072f70 2023-07-18T03:04:27.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.633+0000 7f18899ff700 1 -- 172.21.15.101:0/2572920798 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1884101c20 con 0x7f1884072f70 2023-07-18T03:04:27.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.636+0000 7f18889fd700 1 -- 172.21.15.101:0/2572920798 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f187400c9c0 con 0x7f1884072f70 2023-07-18T03:04:27.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.638+0000 7f18889fd700 1 --2- 172.21.15.101:0/2572920798 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f186c0a1220 0x7f186c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:27.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.638+0000 7f18889fd700 1 -- 172.21.15.101:0/2572920798 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1874014070 con 0x7f1884072f70 2023-07-18T03:04:27.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.638+0000 7f18827fc700 1 --2- 172.21.15.101:0/2572920798 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f186c0a1220 0x7f186c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:27.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.639+0000 7f18827fc700 1 --2- 172.21.15.101:0/2572920798 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f186c0a1220 0x7f186c0a36d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1878009fd0 tx=0x7f1878019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:27.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.642+0000 7f18889fd700 1 -- 172.21.15.101:0/2572920798 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1874051790 con 0x7f1884072f70 2023-07-18T03:04:27.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:27 smithi179 ceph-mon[129229]: Deploying daemon mds.k on smithi155 2023-07-18T03:04:27.727 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:27 smithi101 ceph-mon[126379]: Deploying daemon mds.k on smithi155 2023-07-18T03:04:27.785 INFO:journalctl@ceph.mds.k.smithi155.stdout:Jul 18 03:04:27 smithi155 ceph-mds[153152]: -- 172.21.15.155:0/688508102 <== mon.2 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55880c7591e0 con 0x55880c72e800 2023-07-18T03:04:27.785 INFO:journalctl@ceph.mds.k.smithi155.stdout:Jul 18 03:04:27 smithi155 bash[153112]: 7e4ecb464278a50bbaa88194210da4aaa514ead0ce741877a753263dbdc44e2b 2023-07-18T03:04:27.785 INFO:journalctl@ceph.mds.k.smithi155.stdout:Jul 18 03:04:27 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-k[153128]: starting mds.k at 2023-07-18T03:04:27.785 INFO:journalctl@ceph.mds.k.smithi155.stdout:Jul 18 03:04:27 smithi155 systemd[1]: Started Ceph mds.k for b1782daa-2516-11ee-9b34-001a4aab830c. 2023-07-18T03:04:27.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.896+0000 7f18899ff700 1 -- 172.21.15.101:0/2572920798 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f188402c850 con 0x7f1884072f70 2023-07-18T03:04:27.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.897+0000 7f18889fd700 1 -- 172.21.15.101:0/2572920798 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1260 (secure 0 0 0) 0x7f1874099b30 con 0x7f1884072f70 2023-07-18T03:04:27.881 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:04:27.881 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-07-18T03:04:27.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.899+0000 7f18899ff700 1 -- 172.21.15.101:0/2572920798 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f186c0a1220 msgr2=0x7f186c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:27.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.899+0000 7f18899ff700 1 --2- 172.21.15.101:0/2572920798 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f186c0a1220 0x7f186c0a36d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1878009fd0 tx=0x7f1878019040 comp rx=0 tx=0).stop 2023-07-18T03:04:27.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.899+0000 7f18899ff700 1 -- 172.21.15.101:0/2572920798 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1884072f70 msgr2=0x7f188419bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:27.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.899+0000 7f18899ff700 1 --2- 172.21.15.101:0/2572920798 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1884072f70 0x7f188419bed0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f187400e9f0 tx=0x7f187400ed00 comp rx=0 tx=0).stop 2023-07-18T03:04:27.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.899+0000 7f18899ff700 1 -- 172.21.15.101:0/2572920798 shutdown_connections 2023-07-18T03:04:27.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.899+0000 7f18899ff700 1 --2- 172.21.15.101:0/2572920798 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f186c0a1220 0x7f186c0a36d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:27.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.899+0000 7f18899ff700 1 --2- 172.21.15.101:0/2572920798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f188410c0d0 0x7f18841a3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:27.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.899+0000 7f18899ff700 1 --2- 172.21.15.101:0/2572920798 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1884072f70 0x7f188419bed0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:27.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.899+0000 7f18899ff700 1 --2- 172.21.15.101:0/2572920798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1884074550 0x7f188419c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:27.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.899+0000 7f18899ff700 1 -- 172.21.15.101:0/2572920798 >> 172.21.15.101:0/2572920798 conn(0x7f18840fb0e0 msgr2=0x7f18840fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:27.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.900+0000 7f18899ff700 1 -- 172.21.15.101:0/2572920798 shutdown_connections 2023-07-18T03:04:27.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:27.900+0000 7f18899ff700 1 -- 172.21.15.101:0/2572920798 wait complete. 2023-07-18T03:04:27.885 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 2 2023-07-18T03:04:28.593 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:28 smithi101 ceph-mon[126379]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:28.594 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:28 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:28.594 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:28 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:28.594 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:28 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:28.594 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:28 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:04:28.594 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:28 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:04:28.594 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:28 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:28.594 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:28 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2572920798' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:04:28.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:28 smithi179 ceph-mon[129229]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:28.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:28 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:28.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:28 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:28.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:28 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:28.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:28 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:04:28.691 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:28 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:04:28.691 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:28 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:28.691 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:28 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/2572920798' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:04:28.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:28 smithi155 ceph-mon[128885]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:28.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:28 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:28.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:28 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:28.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:28 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:28.786 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:28 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:04:28.786 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:28 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:04:28.786 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:28 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:28.786 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:28 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/2572920798' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:04:29.681 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:29 smithi101 ceph-mon[126379]: Deploying daemon mds.j on smithi101 2023-07-18T03:04:29.681 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:29 smithi101 ceph-mon[126379]: mds.? [v2:172.21.15.155:6832/3388721497,v1:172.21.15.155:6833/3388721497] up:boot 2023-07-18T03:04:29.682 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:29 smithi101 ceph-mon[126379]: fsmap 2 up:standby 2023-07-18T03:04:29.682 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:29 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-18T03:04:29.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:29 smithi179 ceph-mon[129229]: Deploying daemon mds.j on smithi101 2023-07-18T03:04:29.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:29 smithi179 ceph-mon[129229]: mds.? [v2:172.21.15.155:6832/3388721497,v1:172.21.15.155:6833/3388721497] up:boot 2023-07-18T03:04:29.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:29 smithi179 ceph-mon[129229]: fsmap 2 up:standby 2023-07-18T03:04:29.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:29 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-18T03:04:29.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:29 smithi155 ceph-mon[128885]: Deploying daemon mds.j on smithi101 2023-07-18T03:04:29.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:29 smithi155 ceph-mon[128885]: mds.? [v2:172.21.15.155:6832/3388721497,v1:172.21.15.155:6833/3388721497] up:boot 2023-07-18T03:04:29.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:29 smithi155 ceph-mon[128885]: fsmap 2 up:standby 2023-07-18T03:04:29.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:29 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-18T03:04:29.904 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:04:29.939 INFO:journalctl@ceph.mds.j.smithi101.stdout:Jul 18 03:04:29 smithi101 systemd[1]: Starting Ceph mds.j for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T03:04:30.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.136+0000 7f53683ff700 1 -- 172.21.15.101:0/59963684 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5360100cf0 msgr2=0x7f5360101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:30.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.136+0000 7f53683ff700 1 --2- 172.21.15.101:0/59963684 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5360100cf0 0x7f5360101140 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5350009990 tx=0x7f5350009ca0 comp rx=0 tx=0).stop 2023-07-18T03:04:30.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.137+0000 7f53683ff700 1 -- 172.21.15.101:0/59963684 shutdown_connections 2023-07-18T03:04:30.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.137+0000 7f53683ff700 1 --2- 172.21.15.101:0/59963684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5360101680 0x7f5360107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:30.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.137+0000 7f53683ff700 1 --2- 172.21.15.101:0/59963684 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f53600ffb00 0x7f53600fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:30.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.137+0000 7f53683ff700 1 --2- 172.21.15.101:0/59963684 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5360100cf0 0x7f5360101140 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:30.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.137+0000 7f53683ff700 1 -- 172.21.15.101:0/59963684 >> 172.21.15.101:0/59963684 conn(0x7f53600fb0e0 msgr2=0x7f53600fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:30.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.137+0000 7f53683ff700 1 -- 172.21.15.101:0/59963684 shutdown_connections 2023-07-18T03:04:30.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.137+0000 7f53683ff700 1 -- 172.21.15.101:0/59963684 wait complete. 2023-07-18T03:04:30.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.138+0000 7f53683ff700 1 Processor -- start 2023-07-18T03:04:30.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.138+0000 7f53683ff700 1 -- start start 2023-07-18T03:04:30.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.138+0000 7f53683ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f53600ffb00 0x7f53600742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:30.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.139+0000 7f53683ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5360100cf0 0x7f5360072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:30.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.139+0000 7f53683ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5360101680 0x7f5360072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:30.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.139+0000 7f53683ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53600735e0 con 0x7f53600ffb00 2023-07-18T03:04:30.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.139+0000 7f53683ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5360073750 con 0x7f5360100cf0 2023-07-18T03:04:30.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.139+0000 7f53683ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536019f7f0 con 0x7f5360101680 2023-07-18T03:04:30.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.139+0000 7f536619b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f53600ffb00 0x7f53600742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:30.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.139+0000 7f536619b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f53600ffb00 0x7f53600742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:49874/0 (socket says 172.21.15.101:49874) 2023-07-18T03:04:30.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.139+0000 7f536619b700 1 -- 172.21.15.101:0/1582629002 learned_addr learned my addr 172.21.15.101:0/1582629002 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:04:30.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.139+0000 7f536599a700 1 --2- 172.21.15.101:0/1582629002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5360100cf0 0x7f5360072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:30.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.139+0000 7f536699c700 1 --2- 172.21.15.101:0/1582629002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5360101680 0x7f5360072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:30.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.139+0000 7f536619b700 1 -- 172.21.15.101:0/1582629002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5360101680 msgr2=0x7f5360072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:30.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.139+0000 7f536619b700 1 --2- 172.21.15.101:0/1582629002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5360101680 0x7f5360072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:30.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.139+0000 7f536619b700 1 -- 172.21.15.101:0/1582629002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5360100cf0 msgr2=0x7f5360072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:30.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.139+0000 7f536619b700 1 --2- 172.21.15.101:0/1582629002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5360100cf0 0x7f5360072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:30.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.139+0000 7f536619b700 1 -- 172.21.15.101:0/1582629002 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f535c009520 con 0x7f53600ffb00 2023-07-18T03:04:30.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.139+0000 7f536599a700 1 --2- 172.21.15.101:0/1582629002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5360100cf0 0x7f5360072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:04:30.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.140+0000 7f536619b700 1 --2- 172.21.15.101:0/1582629002 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f53600ffb00 0x7f53600742c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f535c00ea50 tx=0x7f535c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:30.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.140+0000 7f53577fe700 1 -- 172.21.15.101:0/1582629002 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535c00cbd0 con 0x7f53600ffb00 2023-07-18T03:04:30.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.140+0000 7f53577fe700 1 -- 172.21.15.101:0/1582629002 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f535c00cd30 con 0x7f53600ffb00 2023-07-18T03:04:30.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.140+0000 7f53577fe700 1 -- 172.21.15.101:0/1582629002 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535c018680 con 0x7f53600ffb00 2023-07-18T03:04:30.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.140+0000 7f53683ff700 1 -- 172.21.15.101:0/1582629002 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53500095f0 con 0x7f53600ffb00 2023-07-18T03:04:30.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.140+0000 7f53683ff700 1 -- 172.21.15.101:0/1582629002 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f536019fe00 con 0x7f53600ffb00 2023-07-18T03:04:30.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.141+0000 7f53683ff700 1 -- 172.21.15.101:0/1582629002 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5360106fe0 con 0x7f53600ffb00 2023-07-18T03:04:30.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.143+0000 7f53577fe700 1 -- 172.21.15.101:0/1582629002 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f535c010430 con 0x7f53600ffb00 2023-07-18T03:04:30.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.145+0000 7f53577fe700 1 --2- 172.21.15.101:0/1582629002 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f534c0a1220 0x7f534c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:30.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.145+0000 7f53577fe700 1 -- 172.21.15.101:0/1582629002 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f535c014070 con 0x7f53600ffb00 2023-07-18T03:04:30.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.145+0000 7f536599a700 1 --2- 172.21.15.101:0/1582629002 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f534c0a1220 0x7f534c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:30.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.146+0000 7f536599a700 1 --2- 172.21.15.101:0/1582629002 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f534c0a1220 0x7f534c0a36d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f535000b2e0 tx=0x7f5350005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:30.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.149+0000 7f53577fe700 1 -- 172.21.15.101:0/1582629002 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f535c096470 con 0x7f53600ffb00 2023-07-18T03:04:30.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.421+0000 7f53683ff700 1 -- 172.21.15.101:0/1582629002 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f536002c850 con 0x7f53600ffb00 2023-07-18T03:04:30.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.422+0000 7f53577fe700 1 -- 172.21.15.101:0/1582629002 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2020 (secure 0 0 0) 0x7f535c099a90 con 0x7f53600ffb00 2023-07-18T03:04:30.405 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:04:30.405 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-07-18T03:04:30.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.424+0000 7f53683ff700 1 -- 172.21.15.101:0/1582629002 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f534c0a1220 msgr2=0x7f534c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:30.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.424+0000 7f53683ff700 1 --2- 172.21.15.101:0/1582629002 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f534c0a1220 0x7f534c0a36d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f535000b2e0 tx=0x7f5350005fb0 comp rx=0 tx=0).stop 2023-07-18T03:04:30.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.424+0000 7f53683ff700 1 -- 172.21.15.101:0/1582629002 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f53600ffb00 msgr2=0x7f53600742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:30.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.424+0000 7f53683ff700 1 --2- 172.21.15.101:0/1582629002 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f53600ffb00 0x7f53600742c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f535c00ea50 tx=0x7f535c00ed60 comp rx=0 tx=0).stop 2023-07-18T03:04:30.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.424+0000 7f53683ff700 1 -- 172.21.15.101:0/1582629002 shutdown_connections 2023-07-18T03:04:30.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.424+0000 7f53683ff700 1 --2- 172.21.15.101:0/1582629002 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f534c0a1220 0x7f534c0a36d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:30.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.424+0000 7f53683ff700 1 --2- 172.21.15.101:0/1582629002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5360101680 0x7f5360072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:30.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.424+0000 7f53683ff700 1 --2- 172.21.15.101:0/1582629002 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f53600ffb00 0x7f53600742c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:30.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.424+0000 7f53683ff700 1 --2- 172.21.15.101:0/1582629002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5360100cf0 0x7f5360072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:30.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.424+0000 7f53683ff700 1 -- 172.21.15.101:0/1582629002 >> 172.21.15.101:0/1582629002 conn(0x7f53600fb0e0 msgr2=0x7f5360104120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:30.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.424+0000 7f53683ff700 1 -- 172.21.15.101:0/1582629002 shutdown_connections 2023-07-18T03:04:30.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:30.424+0000 7f53683ff700 1 -- 172.21.15.101:0/1582629002 wait complete. 2023-07-18T03:04:30.409 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 3 2023-07-18T03:04:30.623 INFO:journalctl@ceph.mds.j.smithi101.stdout:Jul 18 03:04:30 smithi101 podman[159810]: 2023-07-18T03:04:30.624 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:30 smithi101 ceph-mon[126379]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:30.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:30 smithi179 ceph-mon[129229]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:30.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:30 smithi155 ceph-mon[128885]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:31.227 INFO:journalctl@ceph.mds.j.smithi101.stdout:Jul 18 03:04:30 smithi101 bash[159810]: 604af1288867e4c2851af54984e485e95d406c7864733ad63fca6c563a9d2548 2023-07-18T03:04:31.228 INFO:journalctl@ceph.mds.j.smithi101.stdout:Jul 18 03:04:30 smithi101 ceph-mds[159852]: -- 172.21.15.101:0/2561399132 <== mon.0 v2:172.21.15.101:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564f980f11e0 con 0x564f980c7000 2023-07-18T03:04:31.228 INFO:journalctl@ceph.mds.j.smithi101.stdout:Jul 18 03:04:30 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-j[159828]: starting mds.j at 2023-07-18T03:04:31.228 INFO:journalctl@ceph.mds.j.smithi101.stdout:Jul 18 03:04:30 smithi101 systemd[1]: Started Ceph mds.j for b1782daa-2516-11ee-9b34-001a4aab830c. 2023-07-18T03:04:31.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:31 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1582629002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:04:31.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:31 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:31.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:31 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:31.691 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:31 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:31.691 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:31 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:04:31.691 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:31 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:04:31.691 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:31 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:31.727 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:31 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1582629002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:04:31.728 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:31 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:31.728 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:31 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:31.728 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:31 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:31.728 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:31 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:04:31.728 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:31 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:04:31.729 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:31 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:31.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:31 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1582629002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:04:31.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:31 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:31.786 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:31 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:31.786 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:31 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:31.786 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:31 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:04:31.787 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:31 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:04:31.787 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:31 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:32.426 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:04:32.595 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:32 smithi101 ceph-mon[126379]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:32.596 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:32 smithi101 ceph-mon[126379]: Deploying daemon mds.d on smithi101 2023-07-18T03:04:32.596 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:32 smithi101 ceph-mon[126379]: mds.? [v2:172.21.15.101:6834/296128976,v1:172.21.15.101:6835/296128976] up:boot 2023-07-18T03:04:32.596 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:32 smithi101 ceph-mon[126379]: fsmap 3 up:standby 2023-07-18T03:04:32.596 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:32 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-18T03:04:32.640 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.657+0000 7fa656bff700 1 -- 172.21.15.101:0/1897439466 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa650100cf0 msgr2=0x7fa650101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:32.640 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.657+0000 7fa656bff700 1 --2- 172.21.15.101:0/1897439466 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa650100cf0 0x7fa650101140 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa63c009990 tx=0x7fa63c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:04:32.640 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.657+0000 7fa656bff700 1 -- 172.21.15.101:0/1897439466 shutdown_connections 2023-07-18T03:04:32.640 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.657+0000 7fa656bff700 1 --2- 172.21.15.101:0/1897439466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6500ffb00 0x7fa6500fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:32.641 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.657+0000 7fa656bff700 1 --2- 172.21.15.101:0/1897439466 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa650101680 0x7fa650107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:32.641 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.657+0000 7fa656bff700 1 --2- 172.21.15.101:0/1897439466 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa650100cf0 0x7fa650101140 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:32.641 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.657+0000 7fa656bff700 1 -- 172.21.15.101:0/1897439466 >> 172.21.15.101:0/1897439466 conn(0x7fa6500fb0e0 msgr2=0x7fa6500fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:32.641 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.657+0000 7fa656bff700 1 -- 172.21.15.101:0/1897439466 shutdown_connections 2023-07-18T03:04:32.641 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.657+0000 7fa656bff700 1 -- 172.21.15.101:0/1897439466 wait complete. 2023-07-18T03:04:32.643 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.658+0000 7fa656bff700 1 Processor -- start 2023-07-18T03:04:32.643 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.658+0000 7fa656bff700 1 -- start start 2023-07-18T03:04:32.643 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.659+0000 7fa656bff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6500ffb00 0x7fa6500742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:32.643 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.659+0000 7fa656bff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa650100cf0 0x7fa650072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:32.644 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.659+0000 7fa656bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa650101680 0x7fa650072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:32.644 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.659+0000 7fa656bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6500735e0 con 0x7fa650101680 2023-07-18T03:04:32.644 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.659+0000 7fa656bff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa650073750 con 0x7fa650100cf0 2023-07-18T03:04:32.644 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.659+0000 7fa656bff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65019f8f0 con 0x7fa6500ffb00 2023-07-18T03:04:32.644 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.659+0000 7fa65499b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6500ffb00 0x7fa6500742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:32.644 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.659+0000 7fa65519c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa650101680 0x7fa650072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:32.644 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.659+0000 7fa65499b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6500ffb00 0x7fa6500742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:43794/0 (socket says 172.21.15.101:43794) 2023-07-18T03:04:32.645 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.659+0000 7fa65519c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa650101680 0x7fa650072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:49922/0 (socket says 172.21.15.101:49922) 2023-07-18T03:04:32.645 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.659+0000 7fa65499b700 1 -- 172.21.15.101:0/77719605 learned_addr learned my addr 172.21.15.101:0/77719605 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:04:32.645 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.659+0000 7fa64ffff700 1 --2- 172.21.15.101:0/77719605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa650100cf0 0x7fa650072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:32.645 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.659+0000 7fa65519c700 1 -- 172.21.15.101:0/77719605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6500ffb00 msgr2=0x7fa6500742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:32.645 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.659+0000 7fa65519c700 1 --2- 172.21.15.101:0/77719605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6500ffb00 0x7fa6500742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:32.645 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.659+0000 7fa65519c700 1 -- 172.21.15.101:0/77719605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa650100cf0 msgr2=0x7fa650072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:32.645 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.659+0000 7fa65519c700 1 --2- 172.21.15.101:0/77719605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa650100cf0 0x7fa650072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:32.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.659+0000 7fa65519c700 1 -- 172.21.15.101:0/77719605 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa63c0095f0 con 0x7fa650101680 2023-07-18T03:04:32.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.660+0000 7fa65519c700 1 --2- 172.21.15.101:0/77719605 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa650101680 0x7fa650072e40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa648007f10 tx=0x7fa64800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:32.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.660+0000 7fa64dffb700 1 -- 172.21.15.101:0/77719605 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa648004db0 con 0x7fa650101680 2023-07-18T03:04:32.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.660+0000 7fa64dffb700 1 -- 172.21.15.101:0/77719605 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa64800f040 con 0x7fa650101680 2023-07-18T03:04:32.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.660+0000 7fa656bff700 1 -- 172.21.15.101:0/77719605 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa65019fbd0 con 0x7fa650101680 2023-07-18T03:04:32.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.660+0000 7fa64dffb700 1 -- 172.21.15.101:0/77719605 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa648005990 con 0x7fa650101680 2023-07-18T03:04:32.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.660+0000 7fa656bff700 1 -- 172.21.15.101:0/77719605 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6501a00e0 con 0x7fa650101680 2023-07-18T03:04:32.647 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.661+0000 7fa656bff700 1 -- 172.21.15.101:0/77719605 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa650107050 con 0x7fa650101680 2023-07-18T03:04:32.647 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.663+0000 7fa64dffb700 1 -- 172.21.15.101:0/77719605 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fa64800db20 con 0x7fa650101680 2023-07-18T03:04:32.647 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.664+0000 7fa64dffb700 1 --2- 172.21.15.101:0/77719605 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa6400a11a0 0x7fa6400a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:32.648 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.664+0000 7fa64dffb700 1 -- 172.21.15.101:0/77719605 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa6480cc510 con 0x7fa650101680 2023-07-18T03:04:32.648 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.665+0000 7fa65499b700 1 --2- 172.21.15.101:0/77719605 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa6400a11a0 0x7fa6400a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:32.648 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.665+0000 7fa65499b700 1 --2- 172.21.15.101:0/77719605 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa6400a11a0 0x7fa6400a3650 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa644005950 tx=0x7fa644009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:32.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.666+0000 7fa64dffb700 1 -- 172.21.15.101:0/77719605 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa648096060 con 0x7fa650101680 2023-07-18T03:04:32.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:32 smithi179 ceph-mon[129229]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:32.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:32 smithi179 ceph-mon[129229]: Deploying daemon mds.d on smithi101 2023-07-18T03:04:32.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:32 smithi179 ceph-mon[129229]: mds.? [v2:172.21.15.101:6834/296128976,v1:172.21.15.101:6835/296128976] up:boot 2023-07-18T03:04:32.691 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:32 smithi179 ceph-mon[129229]: fsmap 3 up:standby 2023-07-18T03:04:32.691 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:32 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-18T03:04:32.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:32 smithi155 ceph-mon[128885]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:32.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:32 smithi155 ceph-mon[128885]: Deploying daemon mds.d on smithi101 2023-07-18T03:04:32.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:32 smithi155 ceph-mon[128885]: mds.? [v2:172.21.15.101:6834/296128976,v1:172.21.15.101:6835/296128976] up:boot 2023-07-18T03:04:32.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:32 smithi155 ceph-mon[128885]: fsmap 3 up:standby 2023-07-18T03:04:32.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:32 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-18T03:04:32.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.950+0000 7fa656bff700 1 -- 172.21.15.101:0/77719605 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa65002c850 con 0x7fa650101680 2023-07-18T03:04:32.934 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.951+0000 7fa64dffb700 1 -- 172.21.15.101:0/77719605 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2777 (secure 0 0 0) 0x7fa648023af0 con 0x7fa650101680 2023-07-18T03:04:32.934 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:04:32.934 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-07-18T03:04:32.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.954+0000 7fa656bff700 1 -- 172.21.15.101:0/77719605 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa6400a11a0 msgr2=0x7fa6400a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:32.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.954+0000 7fa656bff700 1 --2- 172.21.15.101:0/77719605 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa6400a11a0 0x7fa6400a3650 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa644005950 tx=0x7fa644009260 comp rx=0 tx=0).stop 2023-07-18T03:04:32.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.954+0000 7fa656bff700 1 -- 172.21.15.101:0/77719605 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa650101680 msgr2=0x7fa650072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:32.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.954+0000 7fa656bff700 1 --2- 172.21.15.101:0/77719605 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa650101680 0x7fa650072e40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa648007f10 tx=0x7fa64800d3b0 comp rx=0 tx=0).stop 2023-07-18T03:04:32.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.954+0000 7fa656bff700 1 -- 172.21.15.101:0/77719605 shutdown_connections 2023-07-18T03:04:32.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.954+0000 7fa656bff700 1 --2- 172.21.15.101:0/77719605 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa6400a11a0 0x7fa6400a3650 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:32.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.954+0000 7fa656bff700 1 --2- 172.21.15.101:0/77719605 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa650101680 0x7fa650072e40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:32.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.954+0000 7fa656bff700 1 --2- 172.21.15.101:0/77719605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6500ffb00 0x7fa6500742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:32.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.954+0000 7fa656bff700 1 --2- 172.21.15.101:0/77719605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa650100cf0 0x7fa650072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:32.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.954+0000 7fa656bff700 1 -- 172.21.15.101:0/77719605 >> 172.21.15.101:0/77719605 conn(0x7fa6500fb0e0 msgr2=0x7fa650104190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:32.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.954+0000 7fa656bff700 1 -- 172.21.15.101:0/77719605 shutdown_connections 2023-07-18T03:04:32.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:32.954+0000 7fa656bff700 1 -- 172.21.15.101:0/77719605 wait complete. 2023-07-18T03:04:32.939 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 4 2023-07-18T03:04:33.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:33 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/77719605' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:04:33.728 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:33 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/77719605' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:04:33.728 INFO:journalctl@ceph.mds.d.smithi101.stdout:Jul 18 03:04:33 smithi101 systemd[1]: Starting Ceph mds.d for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T03:04:33.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:33 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/77719605' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:04:34.128 INFO:journalctl@ceph.mds.d.smithi101.stdout:Jul 18 03:04:33 smithi101 podman[160197]: 2023-07-18T03:04:34.128 INFO:journalctl@ceph.mds.d.smithi101.stdout:Jul 18 03:04:34 smithi101 bash[160197]: 359fbbe10bc29a60c93a11d256a9c5fa5f6080c000927e913c6803c2e294b8b4 2023-07-18T03:04:34.128 INFO:journalctl@ceph.mds.d.smithi101.stdout:Jul 18 03:04:34 smithi101 ceph-mds[160237]: -- 172.21.15.101:0/3172507632 <== mon.0 v2:172.21.15.101:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55af719111e0 con 0x55af718e6c00 2023-07-18T03:04:34.128 INFO:journalctl@ceph.mds.d.smithi101.stdout:Jul 18 03:04:34 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-d[160213]: starting mds.d at 2023-07-18T03:04:34.477 INFO:journalctl@ceph.mds.d.smithi101.stdout:Jul 18 03:04:34 smithi101 systemd[1]: Started Ceph mds.d for b1782daa-2516-11ee-9b34-001a4aab830c. 2023-07-18T03:04:34.478 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:34 smithi101 ceph-mon[126379]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:34.478 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:34 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:34.478 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:34 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:34.478 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:34 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:34.478 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:34 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:04:34.479 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:34 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:04:34.479 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:34 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:34.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:34 smithi179 ceph-mon[129229]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:34.691 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:34 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:34.691 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:34 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:34.691 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:34 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:34.691 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:34 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:04:34.691 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:34 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:04:34.691 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:34 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:34.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:34 smithi155 ceph-mon[128885]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:34.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:34 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:34.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:34 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:34.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:34 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:34.786 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:34 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:04:34.786 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:34 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:04:34.786 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:34 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:34.957 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:04:35.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.179+0000 7f55c63ff700 1 -- 172.21.15.101:0/3416706633 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55c0100cf0 msgr2=0x7f55c0101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:35.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.179+0000 7f55c63ff700 1 --2- 172.21.15.101:0/3416706633 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55c0100cf0 0x7f55c0101140 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009990 tx=0x7f55b4009ca0 comp rx=0 tx=0).stop 2023-07-18T03:04:35.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.179+0000 7f55c63ff700 1 -- 172.21.15.101:0/3416706633 shutdown_connections 2023-07-18T03:04:35.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.179+0000 7f55c63ff700 1 --2- 172.21.15.101:0/3416706633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55c0101680 0x7f55c0107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:35.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.179+0000 7f55c63ff700 1 --2- 172.21.15.101:0/3416706633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55c00ffb00 0x7f55c00fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:35.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.179+0000 7f55c63ff700 1 --2- 172.21.15.101:0/3416706633 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55c0100cf0 0x7f55c0101140 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:35.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.179+0000 7f55c63ff700 1 -- 172.21.15.101:0/3416706633 >> 172.21.15.101:0/3416706633 conn(0x7f55c00fb0e0 msgr2=0x7f55c00fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:35.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.179+0000 7f55c63ff700 1 -- 172.21.15.101:0/3416706633 shutdown_connections 2023-07-18T03:04:35.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.179+0000 7f55c63ff700 1 -- 172.21.15.101:0/3416706633 wait complete. 2023-07-18T03:04:35.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.180+0000 7f55c63ff700 1 Processor -- start 2023-07-18T03:04:35.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.180+0000 7f55c63ff700 1 -- start start 2023-07-18T03:04:35.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.181+0000 7f55c63ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55c00ffb00 0x7f55c0197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:35.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.181+0000 7f55c63ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55c0100cf0 0x7f55c0197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:35.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.181+0000 7f55c63ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55c0101680 0x7f55c019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:35.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.181+0000 7f55c63ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c019f590 con 0x7f55c0101680 2023-07-18T03:04:35.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.181+0000 7f55c63ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c019f6d0 con 0x7f55c0100cf0 2023-07-18T03:04:35.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.181+0000 7f55c63ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c019f860 con 0x7f55c00ffb00 2023-07-18T03:04:35.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.181+0000 7f55c499c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55c0101680 0x7f55c019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:35.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.181+0000 7f55c499c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55c0101680 0x7f55c019efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:49962/0 (socket says 172.21.15.101:49962) 2023-07-18T03:04:35.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.181+0000 7f55c499c700 1 -- 172.21.15.101:0/443687906 learned_addr learned my addr 172.21.15.101:0/443687906 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:04:35.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.181+0000 7f55bf7fe700 1 --2- 172.21.15.101:0/443687906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55c0100cf0 0x7f55c0197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:35.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.181+0000 7f55bffff700 1 --2- 172.21.15.101:0/443687906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55c00ffb00 0x7f55c0197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:35.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.181+0000 7f55c499c700 1 -- 172.21.15.101:0/443687906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55c00ffb00 msgr2=0x7f55c0197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:35.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.181+0000 7f55c499c700 1 --2- 172.21.15.101:0/443687906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55c00ffb00 0x7f55c0197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:35.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.181+0000 7f55c499c700 1 -- 172.21.15.101:0/443687906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55c0100cf0 msgr2=0x7f55c0197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:35.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.181+0000 7f55c499c700 1 --2- 172.21.15.101:0/443687906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55c0100cf0 0x7f55c0197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:35.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.181+0000 7f55c499c700 1 -- 172.21.15.101:0/443687906 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b40095f0 con 0x7f55c0101680 2023-07-18T03:04:35.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.181+0000 7f55bf7fe700 1 --2- 172.21.15.101:0/443687906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55c0100cf0 0x7f55c0197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:04:35.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.182+0000 7f55c499c700 1 --2- 172.21.15.101:0/443687906 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55c0101680 0x7f55c019efc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f55b800ef30 tx=0x7f55b800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:35.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.182+0000 7f55bd7fa700 1 -- 172.21.15.101:0/443687906 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b8018070 con 0x7f55c0101680 2023-07-18T03:04:35.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.182+0000 7f55c63ff700 1 -- 172.21.15.101:0/443687906 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55c019fae0 con 0x7f55c0101680 2023-07-18T03:04:35.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.182+0000 7f55bd7fa700 1 -- 172.21.15.101:0/443687906 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55b800f040 con 0x7f55c0101680 2023-07-18T03:04:35.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.182+0000 7f55bd7fa700 1 -- 172.21.15.101:0/443687906 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b800cd00 con 0x7f55c0101680 2023-07-18T03:04:35.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.182+0000 7f55c63ff700 1 -- 172.21.15.101:0/443687906 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55c019fff0 con 0x7f55c0101680 2023-07-18T03:04:35.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.183+0000 7f55c63ff700 1 -- 172.21.15.101:0/443687906 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55c010b320 con 0x7f55c0101680 2023-07-18T03:04:35.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.185+0000 7f55bd7fa700 1 -- 172.21.15.101:0/443687906 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f55b8003a00 con 0x7f55c0101680 2023-07-18T03:04:35.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.186+0000 7f55bd7fa700 1 --2- 172.21.15.101:0/443687906 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f55a80a11a0 0x7f55a80a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:35.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.186+0000 7f55bd7fa700 1 -- 172.21.15.101:0/443687906 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f55b80cca10 con 0x7f55c0101680 2023-07-18T03:04:35.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.187+0000 7f55bffff700 1 --2- 172.21.15.101:0/443687906 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f55a80a11a0 0x7f55a80a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:35.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.187+0000 7f55bffff700 1 --2- 172.21.15.101:0/443687906 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f55a80a11a0 0x7f55a80a3650 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f55c0100b50 tx=0x7f55b0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:35.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.189+0000 7f55bd7fa700 1 -- 172.21.15.101:0/443687906 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f55b80965f0 con 0x7f55c0101680 2023-07-18T03:04:35.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.433+0000 7f55c63ff700 1 -- 172.21.15.101:0/443687906 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f55c010b510 con 0x7f55c0101680 2023-07-18T03:04:35.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.434+0000 7f55bd7fa700 1 -- 172.21.15.101:0/443687906 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3537 (secure 0 0 0) 0x7f55b8099c10 con 0x7f55c0101680 2023-07-18T03:04:35.417 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:04:35.417 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-07-18T03:04:35.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.436+0000 7f55c63ff700 1 -- 172.21.15.101:0/443687906 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f55a80a11a0 msgr2=0x7f55a80a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:35.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.436+0000 7f55c63ff700 1 --2- 172.21.15.101:0/443687906 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f55a80a11a0 0x7f55a80a3650 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f55c0100b50 tx=0x7f55b0009260 comp rx=0 tx=0).stop 2023-07-18T03:04:35.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.436+0000 7f55c63ff700 1 -- 172.21.15.101:0/443687906 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55c0101680 msgr2=0x7f55c019efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:35.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.436+0000 7f55c63ff700 1 --2- 172.21.15.101:0/443687906 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55c0101680 0x7f55c019efc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f55b800ef30 tx=0x7f55b800c3a0 comp rx=0 tx=0).stop 2023-07-18T03:04:35.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.436+0000 7f55c63ff700 1 -- 172.21.15.101:0/443687906 shutdown_connections 2023-07-18T03:04:35.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.436+0000 7f55c63ff700 1 --2- 172.21.15.101:0/443687906 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f55a80a11a0 0x7f55a80a3650 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:35.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.436+0000 7f55c63ff700 1 --2- 172.21.15.101:0/443687906 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55c0101680 0x7f55c019efc0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:35.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.436+0000 7f55c63ff700 1 --2- 172.21.15.101:0/443687906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55c00ffb00 0x7f55c0197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:35.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.436+0000 7f55c63ff700 1 --2- 172.21.15.101:0/443687906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55c0100cf0 0x7f55c0197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:35.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.436+0000 7f55c63ff700 1 -- 172.21.15.101:0/443687906 >> 172.21.15.101:0/443687906 conn(0x7f55c00fb0e0 msgr2=0x7f55c0106590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:35.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.436+0000 7f55c63ff700 1 -- 172.21.15.101:0/443687906 shutdown_connections 2023-07-18T03:04:35.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:35.437+0000 7f55c63ff700 1 -- 172.21.15.101:0/443687906 wait complete. 2023-07-18T03:04:35.421 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 5 2023-07-18T03:04:35.469 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:35 smithi155 ceph-mon[128885]: Deploying daemon mds.e on smithi155 2023-07-18T03:04:35.469 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:35 smithi155 ceph-mon[128885]: mds.? [v2:172.21.15.101:6836/2478893321,v1:172.21.15.101:6837/2478893321] up:boot 2023-07-18T03:04:35.469 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:35 smithi155 ceph-mon[128885]: fsmap 4 up:standby 2023-07-18T03:04:35.469 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:35 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-18T03:04:35.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:35 smithi179 ceph-mon[129229]: Deploying daemon mds.e on smithi155 2023-07-18T03:04:35.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:35 smithi179 ceph-mon[129229]: mds.? [v2:172.21.15.101:6836/2478893321,v1:172.21.15.101:6837/2478893321] up:boot 2023-07-18T03:04:35.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:35 smithi179 ceph-mon[129229]: fsmap 4 up:standby 2023-07-18T03:04:35.691 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:35 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-18T03:04:35.728 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:35 smithi101 ceph-mon[126379]: Deploying daemon mds.e on smithi155 2023-07-18T03:04:35.728 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:35 smithi101 ceph-mon[126379]: mds.? [v2:172.21.15.101:6836/2478893321,v1:172.21.15.101:6837/2478893321] up:boot 2023-07-18T03:04:35.728 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:35 smithi101 ceph-mon[126379]: fsmap 4 up:standby 2023-07-18T03:04:35.728 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:35 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-18T03:04:35.785 INFO:journalctl@ceph.mds.e.smithi155.stdout:Jul 18 03:04:35 smithi155 systemd[1]: Starting Ceph mds.e for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T03:04:36.243 INFO:journalctl@ceph.mds.e.smithi155.stdout:Jul 18 03:04:36 smithi155 podman[153452]: 2023-07-18T03:04:36.535 INFO:journalctl@ceph.mds.e.smithi155.stdout:Jul 18 03:04:36 smithi155 bash[153452]: 84703f70245da3f2c4d40a5f2d5c72dd934924c11026930bbaf7e8aa35d8a1fe 2023-07-18T03:04:36.535 INFO:journalctl@ceph.mds.e.smithi155.stdout:Jul 18 03:04:36 smithi155 ceph-mds[153493]: -- 172.21.15.155:0/3773087036 <== mon.2 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a32ec411e0 con 0x55a32ec17000 2023-07-18T03:04:36.535 INFO:journalctl@ceph.mds.e.smithi155.stdout:Jul 18 03:04:36 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-e[153469]: starting mds.e at 2023-07-18T03:04:36.535 INFO:journalctl@ceph.mds.e.smithi155.stdout:Jul 18 03:04:36 smithi155 systemd[1]: Started Ceph mds.e for b1782daa-2516-11ee-9b34-001a4aab830c. 2023-07-18T03:04:36.536 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:36 smithi155 ceph-mon[128885]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:36.536 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:36 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/443687906' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:04:36.536 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:36 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:36.536 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:36 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:36.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:36 smithi179 ceph-mon[129229]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:36.690 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:36 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/443687906' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:04:36.691 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:36 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:36.691 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:36 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:36.727 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:36 smithi101 ceph-mon[126379]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:36.728 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:36 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/443687906' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:04:36.728 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:36 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:36.728 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:36 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:37.438 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:04:37.645 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.662+0000 7f7347dff700 1 -- 172.21.15.101:0/3116990330 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7340100cf0 msgr2=0x7f7340101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:37.645 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.662+0000 7f7347dff700 1 --2- 172.21.15.101:0/3116990330 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7340100cf0 0x7f7340101140 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7330009990 tx=0x7f7330009ca0 comp rx=0 tx=0).stop 2023-07-18T03:04:37.645 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.662+0000 7f7347dff700 1 -- 172.21.15.101:0/3116990330 shutdown_connections 2023-07-18T03:04:37.645 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.662+0000 7f7347dff700 1 --2- 172.21.15.101:0/3116990330 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73400ffb00 0x7f73400fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:37.645 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.662+0000 7f7347dff700 1 --2- 172.21.15.101:0/3116990330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7340101680 0x7f7340107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:37.645 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.662+0000 7f7347dff700 1 --2- 172.21.15.101:0/3116990330 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7340100cf0 0x7f7340101140 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:37.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.662+0000 7f7347dff700 1 -- 172.21.15.101:0/3116990330 >> 172.21.15.101:0/3116990330 conn(0x7f73400fb0e0 msgr2=0x7f73400fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:37.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.662+0000 7f7347dff700 1 -- 172.21.15.101:0/3116990330 shutdown_connections 2023-07-18T03:04:37.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.662+0000 7f7347dff700 1 -- 172.21.15.101:0/3116990330 wait complete. 2023-07-18T03:04:37.647 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.663+0000 7f7347dff700 1 Processor -- start 2023-07-18T03:04:37.648 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.663+0000 7f7347dff700 1 -- start start 2023-07-18T03:04:37.648 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.663+0000 7f7347dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f73400ffb00 0x7f7340197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:37.648 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.663+0000 7f7347dff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7340100cf0 0x7f7340197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:37.648 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.663+0000 7f7347dff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7340101680 0x7f734019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:37.648 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.663+0000 7f7347dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734019f590 con 0x7f73400ffb00 2023-07-18T03:04:37.648 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.663+0000 7f7347dff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734019f6d0 con 0x7f7340100cf0 2023-07-18T03:04:37.648 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.663+0000 7f7347dff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734019f860 con 0x7f7340101680 2023-07-18T03:04:37.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.664+0000 7f7345b9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f73400ffb00 0x7f7340197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:37.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.664+0000 7f7345b9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f73400ffb00 0x7f7340197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:48498/0 (socket says 172.21.15.101:48498) 2023-07-18T03:04:37.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.664+0000 7f7345b9b700 1 -- 172.21.15.101:0/1049620845 learned_addr learned my addr 172.21.15.101:0/1049620845 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:04:37.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.664+0000 7f734639c700 1 --2- 172.21.15.101:0/1049620845 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7340101680 0x7f734019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:37.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.664+0000 7f734539a700 1 --2- 172.21.15.101:0/1049620845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7340100cf0 0x7f7340197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:37.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.664+0000 7f7345b9b700 1 -- 172.21.15.101:0/1049620845 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7340101680 msgr2=0x7f734019efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:37.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.664+0000 7f7345b9b700 1 --2- 172.21.15.101:0/1049620845 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7340101680 0x7f734019efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:37.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.664+0000 7f7345b9b700 1 -- 172.21.15.101:0/1049620845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7340100cf0 msgr2=0x7f7340197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:37.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.664+0000 7f7345b9b700 1 --2- 172.21.15.101:0/1049620845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7340100cf0 0x7f7340197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:37.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.664+0000 7f7345b9b700 1 -- 172.21.15.101:0/1049620845 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73300095f0 con 0x7f73400ffb00 2023-07-18T03:04:37.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.664+0000 7f734539a700 1 --2- 172.21.15.101:0/1049620845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7340100cf0 0x7f7340197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:04:37.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.664+0000 7f7345b9b700 1 --2- 172.21.15.101:0/1049620845 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f73400ffb00 0x7f7340197a10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f733c00e970 tx=0x7f733c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:37.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.664+0000 7f7336ffd700 1 -- 172.21.15.101:0/1049620845 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733c009790 con 0x7f73400ffb00 2023-07-18T03:04:37.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.664+0000 7f7347dff700 1 -- 172.21.15.101:0/1049620845 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f734019fb40 con 0x7f73400ffb00 2023-07-18T03:04:37.651 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.664+0000 7f7336ffd700 1 -- 172.21.15.101:0/1049620845 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f733c00eec0 con 0x7f73400ffb00 2023-07-18T03:04:37.651 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.664+0000 7f7336ffd700 1 -- 172.21.15.101:0/1049620845 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733c003cf0 con 0x7f73400ffb00 2023-07-18T03:04:37.651 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.665+0000 7f7347dff700 1 -- 172.21.15.101:0/1049620845 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73401a0050 con 0x7f73400ffb00 2023-07-18T03:04:37.651 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.666+0000 7f7347dff700 1 -- 172.21.15.101:0/1049620845 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f734010b320 con 0x7f73400ffb00 2023-07-18T03:04:37.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.667+0000 7f7336ffd700 1 -- 172.21.15.101:0/1049620845 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f733c00ccd0 con 0x7f73400ffb00 2023-07-18T03:04:37.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.669+0000 7f7336ffd700 1 --2- 172.21.15.101:0/1049620845 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f732c0a11a0 0x7f732c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:37.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.669+0000 7f7336ffd700 1 -- 172.21.15.101:0/1049620845 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f733c014070 con 0x7f73400ffb00 2023-07-18T03:04:37.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.669+0000 7f734539a700 1 --2- 172.21.15.101:0/1049620845 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f732c0a11a0 0x7f732c0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:37.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.669+0000 7f734539a700 1 --2- 172.21.15.101:0/1049620845 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f732c0a11a0 0x7f732c0a3650 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f733000b2e0 tx=0x7f7330005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:37.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.673+0000 7f7336ffd700 1 -- 172.21.15.101:0/1049620845 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f733c0962b0 con 0x7f73400ffb00 2023-07-18T03:04:37.658 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:37 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:37.658 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:37 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:04:37.658 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:37 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:04:37.658 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:37 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:37.659 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:37 smithi179 ceph-mon[129229]: Deploying daemon mds.f on smithi179 2023-07-18T03:04:37.659 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:37 smithi179 ceph-mon[129229]: mds.? [v2:172.21.15.155:6834/943645270,v1:172.21.15.155:6835/943645270] up:boot 2023-07-18T03:04:37.659 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:37 smithi179 ceph-mon[129229]: fsmap 5 up:standby 2023-07-18T03:04:37.659 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:37 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-18T03:04:37.659 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:37 smithi179 ceph-mon[129229]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:37.727 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:37 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:37.728 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:37 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:04:37.728 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:37 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:04:37.728 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:37 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:37.728 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:37 smithi101 ceph-mon[126379]: Deploying daemon mds.f on smithi179 2023-07-18T03:04:37.728 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:37 smithi101 ceph-mon[126379]: mds.? [v2:172.21.15.155:6834/943645270,v1:172.21.15.155:6835/943645270] up:boot 2023-07-18T03:04:37.728 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:37 smithi101 ceph-mon[126379]: fsmap 5 up:standby 2023-07-18T03:04:37.729 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:37 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-18T03:04:37.729 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:37 smithi101 ceph-mon[126379]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:37.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:37 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:37.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:37 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:04:37.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:37 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:04:37.786 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:37 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:37.786 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:37 smithi155 ceph-mon[128885]: Deploying daemon mds.f on smithi179 2023-07-18T03:04:37.786 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:37 smithi155 ceph-mon[128885]: mds.? [v2:172.21.15.155:6834/943645270,v1:172.21.15.155:6835/943645270] up:boot 2023-07-18T03:04:37.786 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:37 smithi155 ceph-mon[128885]: fsmap 5 up:standby 2023-07-18T03:04:37.786 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:37 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-18T03:04:37.786 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:37 smithi155 ceph-mon[128885]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:37.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.920+0000 7f7347dff700 1 -- 172.21.15.101:0/1049620845 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f73401a0600 con 0x7f73400ffb00 2023-07-18T03:04:37.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.921+0000 7f7336ffd700 1 -- 172.21.15.101:0/1049620845 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4294 (secure 0 0 0) 0x7f733c0998d0 con 0x7f73400ffb00 2023-07-18T03:04:37.904 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:04:37.904 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2023-07-18T03:04:37.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.923+0000 7f7347dff700 1 -- 172.21.15.101:0/1049620845 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f732c0a11a0 msgr2=0x7f732c0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:37.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.923+0000 7f7347dff700 1 --2- 172.21.15.101:0/1049620845 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f732c0a11a0 0x7f732c0a3650 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f733000b2e0 tx=0x7f7330005fd0 comp rx=0 tx=0).stop 2023-07-18T03:04:37.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.923+0000 7f7347dff700 1 -- 172.21.15.101:0/1049620845 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f73400ffb00 msgr2=0x7f7340197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:37.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.923+0000 7f7347dff700 1 --2- 172.21.15.101:0/1049620845 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f73400ffb00 0x7f7340197a10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f733c00e970 tx=0x7f733c00ec80 comp rx=0 tx=0).stop 2023-07-18T03:04:37.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.923+0000 7f7347dff700 1 -- 172.21.15.101:0/1049620845 shutdown_connections 2023-07-18T03:04:37.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.923+0000 7f7347dff700 1 --2- 172.21.15.101:0/1049620845 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f732c0a11a0 0x7f732c0a3650 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:37.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.923+0000 7f7347dff700 1 --2- 172.21.15.101:0/1049620845 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7340101680 0x7f734019efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:37.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.923+0000 7f7347dff700 1 --2- 172.21.15.101:0/1049620845 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f73400ffb00 0x7f7340197a10 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:37.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.923+0000 7f7347dff700 1 --2- 172.21.15.101:0/1049620845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7340100cf0 0x7f7340197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:37.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.923+0000 7f7347dff700 1 -- 172.21.15.101:0/1049620845 >> 172.21.15.101:0/1049620845 conn(0x7f73400fb0e0 msgr2=0x7f7340106590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:37.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.923+0000 7f7347dff700 1 -- 172.21.15.101:0/1049620845 shutdown_connections 2023-07-18T03:04:37.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:37.924+0000 7f7347dff700 1 -- 172.21.15.101:0/1049620845 wait complete. 2023-07-18T03:04:37.909 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 6 2023-07-18T03:04:37.940 INFO:journalctl@ceph.mds.f.smithi179.stdout:Jul 18 03:04:37 smithi179 systemd[1]: Starting Ceph mds.f for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T03:04:38.441 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:38 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1049620845' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:04:38.441 INFO:journalctl@ceph.mds.f.smithi179.stdout:Jul 18 03:04:38 smithi179 podman[154615]: 2023-07-18T03:04:38.728 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:38 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1049620845' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:04:38.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:38 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1049620845' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:04:38.940 INFO:journalctl@ceph.mds.f.smithi179.stdout:Jul 18 03:04:38 smithi179 ceph-mds[154655]: -- 172.21.15.179:0/2691117578 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55786a8331e0 con 0x55786a808800 2023-07-18T03:04:38.941 INFO:journalctl@ceph.mds.f.smithi179.stdout:Jul 18 03:04:38 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-f[154631]: starting mds.f at 2023-07-18T03:04:38.941 INFO:journalctl@ceph.mds.f.smithi179.stdout:Jul 18 03:04:38 smithi179 bash[154615]: 88de317e4c8fe983496d788b85e720a3251790289ed8dbd8b0fb2ba668713e1b 2023-07-18T03:04:38.941 INFO:journalctl@ceph.mds.f.smithi179.stdout:Jul 18 03:04:38 smithi179 systemd[1]: Started Ceph mds.f for b1782daa-2516-11ee-9b34-001a4aab830c. 2023-07-18T03:04:39.785 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:39 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:39.786 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:39 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:39.786 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:39 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:39.786 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:39 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:04:39.786 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:39 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:04:39.786 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:39 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:39.786 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:39 smithi155 ceph-mon[128885]: Deploying daemon mds.c on smithi179 2023-07-18T03:04:39.786 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:39 smithi155 ceph-mon[128885]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:39.787 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:39 smithi155 ceph-mon[128885]: mds.? [v2:172.21.15.179:6834/1268903554,v1:172.21.15.179:6835/1268903554] up:boot 2023-07-18T03:04:39.787 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:39 smithi155 ceph-mon[128885]: fsmap 6 up:standby 2023-07-18T03:04:39.787 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:39 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-18T03:04:39.834 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:39 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:39.834 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:39 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:39.834 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:39 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:39.834 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:39 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:04:39.835 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:39 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:04:39.835 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:39 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:39.835 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:39 smithi179 ceph-mon[129229]: Deploying daemon mds.c on smithi179 2023-07-18T03:04:39.835 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:39 smithi179 ceph-mon[129229]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:39.835 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:39 smithi179 ceph-mon[129229]: mds.? [v2:172.21.15.179:6834/1268903554,v1:172.21.15.179:6835/1268903554] up:boot 2023-07-18T03:04:39.835 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:39 smithi179 ceph-mon[129229]: fsmap 6 up:standby 2023-07-18T03:04:39.835 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:39 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-18T03:04:39.926 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:04:39.978 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:39 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:39.978 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:39 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:39.978 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:39 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:39.978 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:39 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:04:39.979 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:39 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:04:39.979 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:39 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:39.979 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:39 smithi101 ceph-mon[126379]: Deploying daemon mds.c on smithi179 2023-07-18T03:04:39.979 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:39 smithi101 ceph-mon[126379]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:39.979 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:39 smithi101 ceph-mon[126379]: mds.? [v2:172.21.15.179:6834/1268903554,v1:172.21.15.179:6835/1268903554] up:boot 2023-07-18T03:04:39.980 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:39 smithi101 ceph-mon[126379]: fsmap 6 up:standby 2023-07-18T03:04:39.980 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:39 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-18T03:04:40.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.149+0000 7f65687ff700 1 -- 172.21.15.101:0/427304549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6560100cf0 msgr2=0x7f6560101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:40.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.149+0000 7f65687ff700 1 --2- 172.21.15.101:0/427304549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6560100cf0 0x7f6560101140 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f65500098f0 tx=0x7f6550009c00 comp rx=0 tx=0).stop 2023-07-18T03:04:40.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.149+0000 7f65687ff700 1 -- 172.21.15.101:0/427304549 shutdown_connections 2023-07-18T03:04:40.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.149+0000 7f65687ff700 1 --2- 172.21.15.101:0/427304549 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65600ffb00 0x7f65600fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:40.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.149+0000 7f65687ff700 1 --2- 172.21.15.101:0/427304549 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6560101680 0x7f6560107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:40.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.149+0000 7f65687ff700 1 --2- 172.21.15.101:0/427304549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6560100cf0 0x7f6560101140 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:40.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.149+0000 7f65687ff700 1 -- 172.21.15.101:0/427304549 >> 172.21.15.101:0/427304549 conn(0x7f65600fb0e0 msgr2=0x7f65600fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:40.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.149+0000 7f65687ff700 1 -- 172.21.15.101:0/427304549 shutdown_connections 2023-07-18T03:04:40.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.149+0000 7f65687ff700 1 -- 172.21.15.101:0/427304549 wait complete. 2023-07-18T03:04:40.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.150+0000 7f65687ff700 1 Processor -- start 2023-07-18T03:04:40.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.150+0000 7f65687ff700 1 -- start start 2023-07-18T03:04:40.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.151+0000 7f65687ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65600ffb00 0x7f6560197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:40.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.151+0000 7f65687ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6560100cf0 0x7f6560197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:40.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.151+0000 7f65687ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6560101680 0x7f656019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:40.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.151+0000 7f65687ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656019f5a0 con 0x7f6560101680 2023-07-18T03:04:40.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.151+0000 7f65687ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656019f6e0 con 0x7f6560100cf0 2023-07-18T03:04:40.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.151+0000 7f65687ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656019f870 con 0x7f65600ffb00 2023-07-18T03:04:40.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.151+0000 7f656659b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65600ffb00 0x7f6560197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:40.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.151+0000 7f656659b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65600ffb00 0x7f6560197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41270/0 (socket says 172.21.15.101:41270) 2023-07-18T03:04:40.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.151+0000 7f656659b700 1 -- 172.21.15.101:0/3960118450 learned_addr learned my addr 172.21.15.101:0/3960118450 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:04:40.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.151+0000 7f6566d9c700 1 --2- 172.21.15.101:0/3960118450 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6560101680 0x7f656019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:40.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.151+0000 7f6565d9a700 1 --2- 172.21.15.101:0/3960118450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6560100cf0 0x7f6560197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:40.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.151+0000 7f656659b700 1 -- 172.21.15.101:0/3960118450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6560100cf0 msgr2=0x7f6560197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:40.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.151+0000 7f656659b700 1 --2- 172.21.15.101:0/3960118450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6560100cf0 0x7f6560197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:40.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.151+0000 7f656659b700 1 -- 172.21.15.101:0/3960118450 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6560101680 msgr2=0x7f656019efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:40.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.151+0000 7f656659b700 1 --2- 172.21.15.101:0/3960118450 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6560101680 0x7f656019efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:40.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.151+0000 7f656659b700 1 -- 172.21.15.101:0/3960118450 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6550009520 con 0x7f65600ffb00 2023-07-18T03:04:40.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.152+0000 7f656659b700 1 --2- 172.21.15.101:0/3960118450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65600ffb00 0x7f6560197a20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f655c00e970 tx=0x7f655c00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:40.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.152+0000 7f65577fe700 1 -- 172.21.15.101:0/3960118450 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c009790 con 0x7f65600ffb00 2023-07-18T03:04:40.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.152+0000 7f65577fe700 1 -- 172.21.15.101:0/3960118450 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f655c00c7e0 con 0x7f65600ffb00 2023-07-18T03:04:40.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.152+0000 7f65687ff700 1 -- 172.21.15.101:0/3960118450 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f656019fb50 con 0x7f65600ffb00 2023-07-18T03:04:40.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.152+0000 7f65577fe700 1 -- 172.21.15.101:0/3960118450 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c003c70 con 0x7f65600ffb00 2023-07-18T03:04:40.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.152+0000 7f65687ff700 1 -- 172.21.15.101:0/3960118450 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65601a0060 con 0x7f65600ffb00 2023-07-18T03:04:40.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.153+0000 7f65687ff700 1 -- 172.21.15.101:0/3960118450 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6560192180 con 0x7f65600ffb00 2023-07-18T03:04:40.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.155+0000 7f65577fe700 1 -- 172.21.15.101:0/3960118450 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f655c00cc50 con 0x7f65600ffb00 2023-07-18T03:04:40.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.156+0000 7f65577fe700 1 --2- 172.21.15.101:0/3960118450 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f654c0a11a0 0x7f654c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:40.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.156+0000 7f65577fe700 1 -- 172.21.15.101:0/3960118450 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f655c014070 con 0x7f65600ffb00 2023-07-18T03:04:40.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.156+0000 7f6565d9a700 1 --2- 172.21.15.101:0/3960118450 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f654c0a11a0 0x7f654c0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:40.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.157+0000 7f6565d9a700 1 --2- 172.21.15.101:0/3960118450 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f654c0a11a0 0x7f654c0a3650 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6550005e90 tx=0x7f6550019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:40.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.158+0000 7f65577fe700 1 -- 172.21.15.101:0/3960118450 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f655c0967f0 con 0x7f65600ffb00 2023-07-18T03:04:40.154 INFO:journalctl@ceph.mds.c.smithi179.stdout:Jul 18 03:04:40 smithi179 systemd[1]: Starting Ceph mds.c for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T03:04:40.379 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.397+0000 7f65687ff700 1 -- 172.21.15.101:0/3960118450 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f656002c850 con 0x7f65600ffb00 2023-07-18T03:04:40.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.398+0000 7f65577fe700 1 -- 172.21.15.101:0/3960118450 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5054 (secure 0 0 0) 0x7f655c003dd0 con 0x7f65600ffb00 2023-07-18T03:04:40.381 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:04:40.381 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24529,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[]} 2023-07-18T03:04:40.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.400+0000 7f65687ff700 1 -- 172.21.15.101:0/3960118450 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f654c0a11a0 msgr2=0x7f654c0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:40.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.400+0000 7f65687ff700 1 --2- 172.21.15.101:0/3960118450 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f654c0a11a0 0x7f654c0a3650 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6550005e90 tx=0x7f6550019040 comp rx=0 tx=0).stop 2023-07-18T03:04:40.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.400+0000 7f65687ff700 1 -- 172.21.15.101:0/3960118450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65600ffb00 msgr2=0x7f6560197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:40.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.400+0000 7f65687ff700 1 --2- 172.21.15.101:0/3960118450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65600ffb00 0x7f6560197a20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f655c00e970 tx=0x7f655c00ec80 comp rx=0 tx=0).stop 2023-07-18T03:04:40.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.400+0000 7f65687ff700 1 -- 172.21.15.101:0/3960118450 shutdown_connections 2023-07-18T03:04:40.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.400+0000 7f65687ff700 1 --2- 172.21.15.101:0/3960118450 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f654c0a11a0 0x7f654c0a3650 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:40.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.400+0000 7f65687ff700 1 --2- 172.21.15.101:0/3960118450 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6560101680 0x7f656019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:40.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.400+0000 7f65687ff700 1 --2- 172.21.15.101:0/3960118450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65600ffb00 0x7f6560197a20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:40.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.400+0000 7f65687ff700 1 --2- 172.21.15.101:0/3960118450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6560100cf0 0x7f6560197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:40.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.400+0000 7f65687ff700 1 -- 172.21.15.101:0/3960118450 >> 172.21.15.101:0/3960118450 conn(0x7f65600fb0e0 msgr2=0x7f65601065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:40.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.400+0000 7f65687ff700 1 -- 172.21.15.101:0/3960118450 shutdown_connections 2023-07-18T03:04:40.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:40.400+0000 7f65687ff700 1 -- 172.21.15.101:0/3960118450 wait complete. 2023-07-18T03:04:40.385 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 7 2023-07-18T03:04:40.717 INFO:journalctl@ceph.mds.c.smithi179.stdout:Jul 18 03:04:40 smithi179 podman[154955]: 2023-07-18T03:04:40.718 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:40 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/3960118450' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:04:40.727 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:40 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3960118450' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:04:41.035 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:40 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/3960118450' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:04:41.190 INFO:journalctl@ceph.mds.c.smithi179.stdout:Jul 18 03:04:40 smithi179 ceph-mds[154995]: -- 172.21.15.179:0/3078805213 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5568287411e0 con 0x556828716c00 2023-07-18T03:04:41.191 INFO:journalctl@ceph.mds.c.smithi179.stdout:Jul 18 03:04:40 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-c[154971]: starting mds.c at 2023-07-18T03:04:41.191 INFO:journalctl@ceph.mds.c.smithi179.stdout:Jul 18 03:04:40 smithi179 bash[154955]: aae380257197ffb1e230fe29575b0526eee9ba6bfee5272b04bf8928546b4a78 2023-07-18T03:04:41.191 INFO:journalctl@ceph.mds.c.smithi179.stdout:Jul 18 03:04:40 smithi179 systemd[1]: Started Ceph mds.c for b1782daa-2516-11ee-9b34-001a4aab830c. 2023-07-18T03:04:42.035 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:41 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:42.036 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:41 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:42.036 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:41 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:42.036 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:41 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:04:42.036 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:41 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:04:42.036 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:41 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:42.036 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:41 smithi155 ceph-mon[128885]: Deploying daemon mds.h on smithi155 2023-07-18T03:04:42.037 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:41 smithi155 ceph-mon[128885]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:42.037 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:41 smithi155 ceph-mon[128885]: mds.? [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] up:boot 2023-07-18T03:04:42.037 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:41 smithi155 ceph-mon[128885]: fsmap 7 up:standby 2023-07-18T03:04:42.037 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:41 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-18T03:04:42.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:41 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:42.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:41 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:42.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:41 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:42.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:41 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:04:42.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:41 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:04:42.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:41 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:42.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:41 smithi179 ceph-mon[129229]: Deploying daemon mds.h on smithi155 2023-07-18T03:04:42.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:41 smithi179 ceph-mon[129229]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:42.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:41 smithi179 ceph-mon[129229]: mds.? [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] up:boot 2023-07-18T03:04:42.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:41 smithi179 ceph-mon[129229]: fsmap 7 up:standby 2023-07-18T03:04:42.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:41 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-18T03:04:42.228 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:41 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:42.228 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:41 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:42.228 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:41 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:42.228 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:41 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:04:42.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:41 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:04:42.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:41 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:42.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:41 smithi101 ceph-mon[126379]: Deploying daemon mds.h on smithi155 2023-07-18T03:04:42.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:41 smithi101 ceph-mon[126379]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:42.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:41 smithi101 ceph-mon[126379]: mds.? [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] up:boot 2023-07-18T03:04:42.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:41 smithi101 ceph-mon[126379]: fsmap 7 up:standby 2023-07-18T03:04:42.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:41 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-18T03:04:42.402 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:04:42.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.624+0000 7fa758fff700 1 -- 172.21.15.101:0/3207591537 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa754100cf0 msgr2=0x7fa754101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:42.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.624+0000 7fa758fff700 1 --2- 172.21.15.101:0/3207591537 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa754100cf0 0x7fa754101140 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa73c009990 tx=0x7fa73c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:04:42.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.624+0000 7fa758fff700 1 -- 172.21.15.101:0/3207591537 shutdown_connections 2023-07-18T03:04:42.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.624+0000 7fa758fff700 1 --2- 172.21.15.101:0/3207591537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7540ffb00 0x7fa7540fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:42.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.624+0000 7fa758fff700 1 --2- 172.21.15.101:0/3207591537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa754101680 0x7fa754107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:42.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.624+0000 7fa758fff700 1 --2- 172.21.15.101:0/3207591537 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa754100cf0 0x7fa754101140 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:42.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.624+0000 7fa758fff700 1 -- 172.21.15.101:0/3207591537 >> 172.21.15.101:0/3207591537 conn(0x7fa7540fb0e0 msgr2=0x7fa7540fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:42.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.624+0000 7fa758fff700 1 -- 172.21.15.101:0/3207591537 shutdown_connections 2023-07-18T03:04:42.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.624+0000 7fa758fff700 1 -- 172.21.15.101:0/3207591537 wait complete. 2023-07-18T03:04:42.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.625+0000 7fa758fff700 1 Processor -- start 2023-07-18T03:04:42.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.625+0000 7fa758fff700 1 -- start start 2023-07-18T03:04:42.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.625+0000 7fa758fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa7540ffb00 0x7fa7540742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:42.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.625+0000 7fa758fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa754100cf0 0x7fa754072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:42.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.625+0000 7fa758fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa754101680 0x7fa754072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:42.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.625+0000 7fa758fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7540735e0 con 0x7fa7540ffb00 2023-07-18T03:04:42.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.625+0000 7fa758fff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa754073750 con 0x7fa754101680 2023-07-18T03:04:42.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.625+0000 7fa758fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75419f8f0 con 0x7fa754100cf0 2023-07-18T03:04:42.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.625+0000 7fa75259c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa7540ffb00 0x7fa7540742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:42.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.625+0000 7fa75259c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa7540ffb00 0x7fa7540742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:48542/0 (socket says 172.21.15.101:48542) 2023-07-18T03:04:42.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.626+0000 7fa75259c700 1 -- 172.21.15.101:0/3784994812 learned_addr learned my addr 172.21.15.101:0/3784994812 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:04:42.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.626+0000 7fa74bfff700 1 --2- 172.21.15.101:0/3784994812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa754100cf0 0x7fa754072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:42.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.626+0000 7fa752d9d700 1 --2- 172.21.15.101:0/3784994812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa754101680 0x7fa754072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:42.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.626+0000 7fa75259c700 1 -- 172.21.15.101:0/3784994812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa754100cf0 msgr2=0x7fa754072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:42.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.626+0000 7fa75259c700 1 --2- 172.21.15.101:0/3784994812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa754100cf0 0x7fa754072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:42.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.626+0000 7fa75259c700 1 -- 172.21.15.101:0/3784994812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa754101680 msgr2=0x7fa754072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:42.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.626+0000 7fa75259c700 1 --2- 172.21.15.101:0/3784994812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa754101680 0x7fa754072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:42.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.626+0000 7fa75259c700 1 -- 172.21.15.101:0/3784994812 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa744009520 con 0x7fa7540ffb00 2023-07-18T03:04:42.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.626+0000 7fa75259c700 1 --2- 172.21.15.101:0/3784994812 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa7540ffb00 0x7fa7540742c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa74400ea50 tx=0x7fa74400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:42.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.626+0000 7fa74b7fe700 1 -- 172.21.15.101:0/3784994812 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74400cbd0 con 0x7fa7540ffb00 2023-07-18T03:04:42.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.626+0000 7fa74b7fe700 1 -- 172.21.15.101:0/3784994812 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa74400cd30 con 0x7fa7540ffb00 2023-07-18T03:04:42.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.626+0000 7fa74b7fe700 1 -- 172.21.15.101:0/3784994812 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa744018680 con 0x7fa7540ffb00 2023-07-18T03:04:42.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.626+0000 7fa758fff700 1 -- 172.21.15.101:0/3784994812 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa73c0095f0 con 0x7fa7540ffb00 2023-07-18T03:04:42.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.627+0000 7fa758fff700 1 -- 172.21.15.101:0/3784994812 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa75419ff00 con 0x7fa7540ffb00 2023-07-18T03:04:42.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.628+0000 7fa758fff700 1 -- 172.21.15.101:0/3784994812 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa754107050 con 0x7fa7540ffb00 2023-07-18T03:04:42.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.629+0000 7fa74b7fe700 1 -- 172.21.15.101:0/3784994812 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fa744010430 con 0x7fa7540ffb00 2023-07-18T03:04:42.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.631+0000 7fa74b7fe700 1 --2- 172.21.15.101:0/3784994812 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa7400a1220 0x7fa7400a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:42.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.631+0000 7fa74b7fe700 1 -- 172.21.15.101:0/3784994812 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa744014070 con 0x7fa7540ffb00 2023-07-18T03:04:42.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.631+0000 7fa74bfff700 1 --2- 172.21.15.101:0/3784994812 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa7400a1220 0x7fa7400a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:42.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.632+0000 7fa74bfff700 1 --2- 172.21.15.101:0/3784994812 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa7400a1220 0x7fa7400a36d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa73c00b2e0 tx=0x7fa73c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:42.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.635+0000 7fa74b7fe700 1 -- 172.21.15.101:0/3784994812 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa744096470 con 0x7fa7540ffb00 2023-07-18T03:04:42.682 INFO:journalctl@ceph.mds.h.smithi155.stdout:Jul 18 03:04:42 smithi155 systemd[1]: Starting Ceph mds.h for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T03:04:42.860 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.877+0000 7fa758fff700 1 -- 172.21.15.101:0/3784994812 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa75410b470 con 0x7fa7540ffb00 2023-07-18T03:04:42.861 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.877+0000 7fa74b7fe700 1 -- 172.21.15.101:0/3784994812 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+5814 (secure 0 0 0) 0x7fa744099a90 con 0x7fa7540ffb00 2023-07-18T03:04:42.861 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:04:42.862 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24529,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24541,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2023-07-18T03:04:42.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.880+0000 7fa758fff700 1 -- 172.21.15.101:0/3784994812 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa7400a1220 msgr2=0x7fa7400a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:42.864 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.880+0000 7fa758fff700 1 --2- 172.21.15.101:0/3784994812 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa7400a1220 0x7fa7400a36d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa73c00b2e0 tx=0x7fa73c005fb0 comp rx=0 tx=0).stop 2023-07-18T03:04:42.864 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.880+0000 7fa758fff700 1 -- 172.21.15.101:0/3784994812 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa7540ffb00 msgr2=0x7fa7540742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:42.864 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.880+0000 7fa758fff700 1 --2- 172.21.15.101:0/3784994812 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa7540ffb00 0x7fa7540742c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa74400ea50 tx=0x7fa74400ed60 comp rx=0 tx=0).stop 2023-07-18T03:04:42.864 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.880+0000 7fa758fff700 1 -- 172.21.15.101:0/3784994812 shutdown_connections 2023-07-18T03:04:42.864 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.880+0000 7fa758fff700 1 --2- 172.21.15.101:0/3784994812 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa7400a1220 0x7fa7400a36d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:42.864 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.880+0000 7fa758fff700 1 --2- 172.21.15.101:0/3784994812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa754101680 0x7fa754072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:42.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.880+0000 7fa758fff700 1 --2- 172.21.15.101:0/3784994812 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa7540ffb00 0x7fa7540742c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:42.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.880+0000 7fa758fff700 1 --2- 172.21.15.101:0/3784994812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa754100cf0 0x7fa754072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:42.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.880+0000 7fa758fff700 1 -- 172.21.15.101:0/3784994812 >> 172.21.15.101:0/3784994812 conn(0x7fa7540fb0e0 msgr2=0x7fa754104190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:42.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.880+0000 7fa758fff700 1 -- 172.21.15.101:0/3784994812 shutdown_connections 2023-07-18T03:04:42.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:42.880+0000 7fa758fff700 1 -- 172.21.15.101:0/3784994812 wait complete. 2023-07-18T03:04:42.866 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 8 2023-07-18T03:04:42.933 INFO:journalctl@ceph.mds.h.smithi155.stdout:Jul 18 03:04:42 smithi155 podman[153793]: 2023-07-18T03:04:42.933 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:42 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/3784994812' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:04:43.190 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:42 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/3784994812' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:04:43.228 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:42 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3784994812' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:04:43.285 INFO:journalctl@ceph.mds.h.smithi155.stdout:Jul 18 03:04:42 smithi155 ceph-mds[153833]: -- 172.21.15.155:0/1848408704 <== mon.2 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5596e42971e0 con 0x5596e426cc00 2023-07-18T03:04:43.285 INFO:journalctl@ceph.mds.h.smithi155.stdout:Jul 18 03:04:42 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-h[153809]: starting mds.h at 2023-07-18T03:04:43.286 INFO:journalctl@ceph.mds.h.smithi155.stdout:Jul 18 03:04:42 smithi155 bash[153793]: 9bd7b1b79232102f495329cb99191689d06d8cd4132ca4ece191271536fc5353 2023-07-18T03:04:43.286 INFO:journalctl@ceph.mds.h.smithi155.stdout:Jul 18 03:04:42 smithi155 systemd[1]: Started Ceph mds.h for b1782daa-2516-11ee-9b34-001a4aab830c. 2023-07-18T03:04:44.285 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:44 smithi155 ceph-mon[128885]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:44.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:44 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:44.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:44 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:44.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:44 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:44.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:44 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:04:44.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:44 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:04:44.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:44 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:44.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:44 smithi155 ceph-mon[128885]: Deploying daemon mds.i on smithi179 2023-07-18T03:04:44.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:44 smithi155 ceph-mon[128885]: mds.? [v2:172.21.15.155:6836/48029036,v1:172.21.15.155:6837/48029036] up:boot 2023-07-18T03:04:44.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:44 smithi155 ceph-mon[128885]: fsmap 8 up:standby 2023-07-18T03:04:44.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:44 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-18T03:04:44.388 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:44 smithi179 ceph-mon[129229]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:44.389 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:44 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:44.389 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:44 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:44.389 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:44 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:44.389 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:44 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:04:44.389 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:44 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:04:44.389 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:44 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:44.389 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:44 smithi179 ceph-mon[129229]: Deploying daemon mds.i on smithi179 2023-07-18T03:04:44.390 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:44 smithi179 ceph-mon[129229]: mds.? [v2:172.21.15.155:6836/48029036,v1:172.21.15.155:6837/48029036] up:boot 2023-07-18T03:04:44.390 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:44 smithi179 ceph-mon[129229]: fsmap 8 up:standby 2023-07-18T03:04:44.390 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:44 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-18T03:04:44.478 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:44 smithi101 ceph-mon[126379]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:44.479 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:44 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:44.479 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:44 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:44.479 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:44 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:44.479 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:44 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:04:44.479 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:44 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:04:44.480 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:44 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:44.480 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:44 smithi101 ceph-mon[126379]: Deploying daemon mds.i on smithi179 2023-07-18T03:04:44.480 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:44 smithi101 ceph-mon[126379]: mds.? [v2:172.21.15.155:6836/48029036,v1:172.21.15.155:6837/48029036] up:boot 2023-07-18T03:04:44.480 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:44 smithi101 ceph-mon[126379]: fsmap 8 up:standby 2023-07-18T03:04:44.480 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:44 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-18T03:04:44.883 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:04:44.941 INFO:journalctl@ceph.mds.i.smithi179.stdout:Jul 18 03:04:44 smithi179 systemd[1]: Starting Ceph mds.i for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T03:04:45.094 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.107+0000 7f7372fff700 1 -- 172.21.15.101:0/457647989 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f736c1057d0 msgr2=0x7f736c107bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:45.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.107+0000 7f7372fff700 1 --2- 172.21.15.101:0/457647989 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f736c1057d0 0x7f736c107bf0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f736400b0a0 tx=0x7f736400b3b0 comp rx=0 tx=0).stop 2023-07-18T03:04:45.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.107+0000 7f7372fff700 1 -- 172.21.15.101:0/457647989 shutdown_connections 2023-07-18T03:04:45.096 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.107+0000 7f7372fff700 1 --2- 172.21.15.101:0/457647989 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f736c1057d0 0x7f736c107bf0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:45.096 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.107+0000 7f7372fff700 1 --2- 172.21.15.101:0/457647989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f736c100590 0x7f736c102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:45.096 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.107+0000 7f7372fff700 1 --2- 172.21.15.101:0/457647989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f736c102eb0 0x7f736c105290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:45.096 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.107+0000 7f7372fff700 1 -- 172.21.15.101:0/457647989 >> 172.21.15.101:0/457647989 conn(0x7f736c0f9fd0 msgr2=0x7f736c0fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:45.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.107+0000 7f7372fff700 1 -- 172.21.15.101:0/457647989 shutdown_connections 2023-07-18T03:04:45.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.107+0000 7f7372fff700 1 -- 172.21.15.101:0/457647989 wait complete. 2023-07-18T03:04:45.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.108+0000 7f7372fff700 1 Processor -- start 2023-07-18T03:04:45.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.108+0000 7f7372fff700 1 -- start start 2023-07-18T03:04:45.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.108+0000 7f7372fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f736c100590 0x7f736c197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:45.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.108+0000 7f7372fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f736c102eb0 0x7f736c197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:45.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.108+0000 7f7372fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f736c1057d0 0x7f736c19f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:45.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.108+0000 7f7372fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f736c19f630 con 0x7f736c100590 2023-07-18T03:04:45.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.108+0000 7f7372fff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f736c19f770 con 0x7f736c102eb0 2023-07-18T03:04:45.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.108+0000 7f7372fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f736c19f900 con 0x7f736c1057d0 2023-07-18T03:04:45.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.108+0000 7f7370d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f736c100590 0x7f736c197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:45.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.109+0000 7f7370d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f736c100590 0x7f736c197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:48562/0 (socket says 172.21.15.101:48562) 2023-07-18T03:04:45.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.109+0000 7f7370d9b700 1 -- 172.21.15.101:0/3536094020 learned_addr learned my addr 172.21.15.101:0/3536094020 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:04:45.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.109+0000 7f737159c700 1 --2- 172.21.15.101:0/3536094020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f736c1057d0 0x7f736c19f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:45.100 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.109+0000 7f736bfff700 1 --2- 172.21.15.101:0/3536094020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f736c102eb0 0x7f736c197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:45.100 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.109+0000 7f737159c700 1 -- 172.21.15.101:0/3536094020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f736c102eb0 msgr2=0x7f736c197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:45.100 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.109+0000 7f737159c700 1 --2- 172.21.15.101:0/3536094020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f736c102eb0 0x7f736c197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:45.100 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.109+0000 7f737159c700 1 -- 172.21.15.101:0/3536094020 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f736c100590 msgr2=0x7f736c197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:45.100 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.109+0000 7f737159c700 1 --2- 172.21.15.101:0/3536094020 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f736c100590 0x7f736c197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:45.101 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.109+0000 7f737159c700 1 -- 172.21.15.101:0/3536094020 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7364009d00 con 0x7f736c1057d0 2023-07-18T03:04:45.101 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.109+0000 7f7370d9b700 1 --2- 172.21.15.101:0/3536094020 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f736c100590 0x7f736c197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:04:45.101 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.109+0000 7f737159c700 1 --2- 172.21.15.101:0/3536094020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f736c1057d0 0x7f736c19f060 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7364009fd0 tx=0x7f7364007930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:45.101 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.109+0000 7f7369ffb700 1 -- 172.21.15.101:0/3536094020 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736400e040 con 0x7f736c1057d0 2023-07-18T03:04:45.101 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.109+0000 7f7369ffb700 1 -- 172.21.15.101:0/3536094020 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7364004c50 con 0x7f736c1057d0 2023-07-18T03:04:45.101 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.110+0000 7f7372fff700 1 -- 172.21.15.101:0/3536094020 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f736c19fb80 con 0x7f736c1057d0 2023-07-18T03:04:45.101 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.110+0000 7f7369ffb700 1 -- 172.21.15.101:0/3536094020 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73640052a0 con 0x7f736c1057d0 2023-07-18T03:04:45.102 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.110+0000 7f7372fff700 1 -- 172.21.15.101:0/3536094020 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f736c1a0000 con 0x7f736c1057d0 2023-07-18T03:04:45.102 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.110+0000 7f7372fff700 1 -- 172.21.15.101:0/3536094020 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f736c107340 con 0x7f736c1057d0 2023-07-18T03:04:45.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.112+0000 7f7369ffb700 1 -- 172.21.15.101:0/3536094020 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f7364005610 con 0x7f736c1057d0 2023-07-18T03:04:45.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.114+0000 7f7369ffb700 1 --2- 172.21.15.101:0/3536094020 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f735c0a1220 0x7f735c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:45.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.114+0000 7f7369ffb700 1 -- 172.21.15.101:0/3536094020 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f73640cdd90 con 0x7f736c1057d0 2023-07-18T03:04:45.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.114+0000 7f7370d9b700 1 --2- 172.21.15.101:0/3536094020 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f735c0a1220 0x7f735c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:45.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.115+0000 7f7370d9b700 1 --2- 172.21.15.101:0/3536094020 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f735c0a1220 0x7f735c0a36d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7360011550 tx=0x7f7360009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:45.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.116+0000 7f7369ffb700 1 -- 172.21.15.101:0/3536094020 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7364097970 con 0x7f736c1057d0 2023-07-18T03:04:45.225 INFO:journalctl@ceph.mds.i.smithi179.stdout:Jul 18 03:04:44 smithi179 podman[155293]: 2023-07-18T03:04:45.338 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.354+0000 7f7372fff700 1 -- 172.21.15.101:0/3536094020 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f736c02c850 con 0x7f736c1057d0 2023-07-18T03:04:45.338 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.355+0000 7f7369ffb700 1 -- 172.21.15.101:0/3536094020 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6568 (secure 0 0 0) 0x7f736409af90 con 0x7f736c1057d0 2023-07-18T03:04:45.339 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:04:45.339 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24515,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24529,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24541,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2023-07-18T03:04:45.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.357+0000 7f7372fff700 1 -- 172.21.15.101:0/3536094020 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f735c0a1220 msgr2=0x7f735c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:45.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.357+0000 7f7372fff700 1 --2- 172.21.15.101:0/3536094020 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f735c0a1220 0x7f735c0a36d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7360011550 tx=0x7f7360009260 comp rx=0 tx=0).stop 2023-07-18T03:04:45.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.357+0000 7f7372fff700 1 -- 172.21.15.101:0/3536094020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f736c1057d0 msgr2=0x7f736c19f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:45.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.357+0000 7f7372fff700 1 --2- 172.21.15.101:0/3536094020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f736c1057d0 0x7f736c19f060 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7364009fd0 tx=0x7f7364007930 comp rx=0 tx=0).stop 2023-07-18T03:04:45.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.357+0000 7f7372fff700 1 -- 172.21.15.101:0/3536094020 shutdown_connections 2023-07-18T03:04:45.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.357+0000 7f7372fff700 1 --2- 172.21.15.101:0/3536094020 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f735c0a1220 0x7f735c0a36d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:45.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.357+0000 7f7372fff700 1 --2- 172.21.15.101:0/3536094020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f736c1057d0 0x7f736c19f060 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:45.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.357+0000 7f7372fff700 1 --2- 172.21.15.101:0/3536094020 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f736c100590 0x7f736c197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:45.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.357+0000 7f7372fff700 1 --2- 172.21.15.101:0/3536094020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f736c102eb0 0x7f736c197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:45.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.357+0000 7f7372fff700 1 -- 172.21.15.101:0/3536094020 >> 172.21.15.101:0/3536094020 conn(0x7f736c0f9fd0 msgr2=0x7f736c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:45.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.357+0000 7f7372fff700 1 -- 172.21.15.101:0/3536094020 shutdown_connections 2023-07-18T03:04:45.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:45.357+0000 7f7372fff700 1 -- 172.21.15.101:0/3536094020 wait complete. 2023-07-18T03:04:45.343 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 9 2023-07-18T03:04:45.690 INFO:journalctl@ceph.mds.i.smithi179.stdout:Jul 18 03:04:45 smithi179 ceph-mds[155333]: -- 172.21.15.179:0/2851668989 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562a60a6d1e0 con 0x562a60a42c00 2023-07-18T03:04:45.690 INFO:journalctl@ceph.mds.i.smithi179.stdout:Jul 18 03:04:45 smithi179 bash[155293]: f6c5b1caf4b56148b6a3694a01073f1056f803724539a540b8beda5cf95c04d2 2023-07-18T03:04:45.691 INFO:journalctl@ceph.mds.i.smithi179.stdout:Jul 18 03:04:45 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-i[155309]: starting mds.i at 2023-07-18T03:04:45.691 INFO:journalctl@ceph.mds.i.smithi179.stdout:Jul 18 03:04:45 smithi179 systemd[1]: Started Ceph mds.i for b1782daa-2516-11ee-9b34-001a4aab830c. 2023-07-18T03:04:46.285 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:46 smithi155 ceph-mon[128885]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:46.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:46 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/3536094020' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:04:46.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:46 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:46.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:46 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:46.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:46 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:46.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:46 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:04:46.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:46 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:04:46.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:46 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:46.433 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:46 smithi101 ceph-mon[126379]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:46.434 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:46 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3536094020' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:04:46.434 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:46 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:46.434 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:46 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:46.434 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:46 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:46.434 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:46 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:04:46.434 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:46 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:04:46.435 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:46 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:46.440 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:46 smithi179 ceph-mon[129229]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:46.441 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:46 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/3536094020' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:04:46.441 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:46 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:46.441 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:46 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:46.441 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:46 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:46.441 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:46 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:04:46.441 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:46 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:04:46.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:46 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:47.361 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:04:47.425 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:47 smithi101 ceph-mon[126379]: Deploying daemon mds.g on smithi101 2023-07-18T03:04:47.426 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:47 smithi101 ceph-mon[126379]: mds.? [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] up:boot 2023-07-18T03:04:47.426 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:47 smithi101 ceph-mon[126379]: fsmap 9 up:standby 2023-07-18T03:04:47.426 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:47 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-18T03:04:47.440 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:47 smithi179 ceph-mon[129229]: Deploying daemon mds.g on smithi101 2023-07-18T03:04:47.441 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:47 smithi179 ceph-mon[129229]: mds.? [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] up:boot 2023-07-18T03:04:47.441 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:47 smithi179 ceph-mon[129229]: fsmap 9 up:standby 2023-07-18T03:04:47.441 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:47 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-18T03:04:47.535 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:47 smithi155 ceph-mon[128885]: Deploying daemon mds.g on smithi101 2023-07-18T03:04:47.535 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:47 smithi155 ceph-mon[128885]: mds.? [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] up:boot 2023-07-18T03:04:47.536 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:47 smithi155 ceph-mon[128885]: fsmap 9 up:standby 2023-07-18T03:04:47.536 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:47 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-18T03:04:47.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.604+0000 7f08b0bff700 1 -- 172.21.15.101:0/469623656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08ac072f70 msgr2=0x7f08ac0733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:47.589 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.604+0000 7f08b0bff700 1 --2- 172.21.15.101:0/469623656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08ac072f70 0x7f08ac0733e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f08a0009990 tx=0x7f08a0009ca0 comp rx=0 tx=0).stop 2023-07-18T03:04:47.589 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.604+0000 7f08b0bff700 1 -- 172.21.15.101:0/469623656 shutdown_connections 2023-07-18T03:04:47.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.604+0000 7f08b0bff700 1 --2- 172.21.15.101:0/469623656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08ac074550 0x7f08ac0729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:47.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.604+0000 7f08b0bff700 1 --2- 172.21.15.101:0/469623656 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f08ac10c0d0 0x7f08ac10e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:47.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.604+0000 7f08b0bff700 1 --2- 172.21.15.101:0/469623656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08ac072f70 0x7f08ac0733e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:47.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.604+0000 7f08b0bff700 1 -- 172.21.15.101:0/469623656 >> 172.21.15.101:0/469623656 conn(0x7f08ac0fb0e0 msgr2=0x7f08ac0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:47.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.604+0000 7f08b0bff700 1 -- 172.21.15.101:0/469623656 shutdown_connections 2023-07-18T03:04:47.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.604+0000 7f08b0bff700 1 -- 172.21.15.101:0/469623656 wait complete. 2023-07-18T03:04:47.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.606+0000 7f08b0bff700 1 Processor -- start 2023-07-18T03:04:47.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.606+0000 7f08b0bff700 1 -- start start 2023-07-18T03:04:47.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.606+0000 7f08b0bff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08ac072f70 0x7f08ac19beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:47.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.606+0000 7f08b0bff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08ac074550 0x7f08ac19c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:47.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.606+0000 7f08b0bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f08ac10c0d0 0x7f08ac1a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:47.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.606+0000 7f08b0bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac1a3a30 con 0x7f08ac10c0d0 2023-07-18T03:04:47.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.606+0000 7f08b0bff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac1a3b70 con 0x7f08ac072f70 2023-07-18T03:04:47.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.606+0000 7f08b0bff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac1a3d00 con 0x7f08ac074550 2023-07-18T03:04:47.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.607+0000 7f08aa59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08ac072f70 0x7f08ac19beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:47.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.607+0000 7f08aa59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08ac072f70 0x7f08ac19beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:56458/0 (socket says 172.21.15.101:56458) 2023-07-18T03:04:47.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.607+0000 7f08aa59c700 1 -- 172.21.15.101:0/266852427 learned_addr learned my addr 172.21.15.101:0/266852427 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:04:47.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.607+0000 7f08aad9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f08ac10c0d0 0x7f08ac1a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:47.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.607+0000 7f08a9d9b700 1 --2- 172.21.15.101:0/266852427 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08ac074550 0x7f08ac19c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:47.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.607+0000 7f08aa59c700 1 -- 172.21.15.101:0/266852427 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08ac074550 msgr2=0x7f08ac19c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:47.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.607+0000 7f08aa59c700 1 --2- 172.21.15.101:0/266852427 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08ac074550 0x7f08ac19c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:47.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.607+0000 7f08aa59c700 1 -- 172.21.15.101:0/266852427 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f08ac10c0d0 msgr2=0x7f08ac1a3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:47.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.607+0000 7f08aa59c700 1 --2- 172.21.15.101:0/266852427 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f08ac10c0d0 0x7f08ac1a3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:47.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.607+0000 7f08aa59c700 1 -- 172.21.15.101:0/266852427 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a4009c40 con 0x7f08ac072f70 2023-07-18T03:04:47.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.607+0000 7f08aad9d700 1 --2- 172.21.15.101:0/266852427 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f08ac10c0d0 0x7f08ac1a3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:04:47.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.607+0000 7f08aa59c700 1 --2- 172.21.15.101:0/266852427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08ac072f70 0x7f08ac19beb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f089c00e970 tx=0x7f089c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:47.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.608+0000 7f089b7fe700 1 -- 172.21.15.101:0/266852427 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089c009790 con 0x7f08ac072f70 2023-07-18T03:04:47.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.608+0000 7f08b0bff700 1 -- 172.21.15.101:0/266852427 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08a00095f0 con 0x7f08ac072f70 2023-07-18T03:04:47.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.608+0000 7f089b7fe700 1 -- 172.21.15.101:0/266852427 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f089c00c7c0 con 0x7f08ac072f70 2023-07-18T03:04:47.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.608+0000 7f089b7fe700 1 -- 172.21.15.101:0/266852427 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089c005460 con 0x7f08ac072f70 2023-07-18T03:04:47.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.608+0000 7f08b0bff700 1 -- 172.21.15.101:0/266852427 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08ac1a4280 con 0x7f08ac072f70 2023-07-18T03:04:47.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.609+0000 7f08b0bff700 1 -- 172.21.15.101:0/266852427 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08ac101c20 con 0x7f08ac072f70 2023-07-18T03:04:47.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.611+0000 7f089b7fe700 1 -- 172.21.15.101:0/266852427 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f089c00c930 con 0x7f08ac072f70 2023-07-18T03:04:47.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.612+0000 7f089b7fe700 1 --2- 172.21.15.101:0/266852427 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f08940a11a0 0x7f08940a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:47.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.612+0000 7f089b7fe700 1 -- 172.21.15.101:0/266852427 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f089c014070 con 0x7f08ac072f70 2023-07-18T03:04:47.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.612+0000 7f08a9d9b700 1 --2- 172.21.15.101:0/266852427 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f08940a11a0 0x7f08940a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:47.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.613+0000 7f08a9d9b700 1 --2- 172.21.15.101:0/266852427 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f08940a11a0 0x7f08940a3650 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f08a000b2e0 tx=0x7f08a0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:47.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.614+0000 7f089b7fe700 1 -- 172.21.15.101:0/266852427 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f089c096370 con 0x7f08ac072f70 2023-07-18T03:04:47.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.882+0000 7f08b0bff700 1 -- 172.21.15.101:0/266852427 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08ac02cac0 con 0x7f08ac072f70 2023-07-18T03:04:47.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.883+0000 7f089b7fe700 1 -- 172.21.15.101:0/266852427 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7330 (secure 0 0 0) 0x7f089c099990 con 0x7f08ac072f70 2023-07-18T03:04:47.867 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:04:47.868 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24515,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24529,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24541,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24562,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-07-18T03:04:47.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.886+0000 7f08b0bff700 1 -- 172.21.15.101:0/266852427 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f08940a11a0 msgr2=0x7f08940a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:47.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.886+0000 7f08b0bff700 1 --2- 172.21.15.101:0/266852427 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f08940a11a0 0x7f08940a3650 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f08a000b2e0 tx=0x7f08a0005fb0 comp rx=0 tx=0).stop 2023-07-18T03:04:47.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.886+0000 7f08b0bff700 1 -- 172.21.15.101:0/266852427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08ac072f70 msgr2=0x7f08ac19beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:47.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.886+0000 7f08b0bff700 1 --2- 172.21.15.101:0/266852427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08ac072f70 0x7f08ac19beb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f089c00e970 tx=0x7f089c00ec80 comp rx=0 tx=0).stop 2023-07-18T03:04:47.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.886+0000 7f08b0bff700 1 -- 172.21.15.101:0/266852427 shutdown_connections 2023-07-18T03:04:47.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.886+0000 7f08b0bff700 1 --2- 172.21.15.101:0/266852427 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f08940a11a0 0x7f08940a3650 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:47.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.886+0000 7f08b0bff700 1 --2- 172.21.15.101:0/266852427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08ac072f70 0x7f08ac19beb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:47.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.886+0000 7f08b0bff700 1 --2- 172.21.15.101:0/266852427 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f08ac10c0d0 0x7f08ac1a3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:47.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.886+0000 7f08b0bff700 1 --2- 172.21.15.101:0/266852427 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08ac074550 0x7f08ac19c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:47.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.886+0000 7f08b0bff700 1 -- 172.21.15.101:0/266852427 >> 172.21.15.101:0/266852427 conn(0x7f08ac0fb0e0 msgr2=0x7f08ac0fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:47.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.887+0000 7f08b0bff700 1 -- 172.21.15.101:0/266852427 shutdown_connections 2023-07-18T03:04:47.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:47.887+0000 7f08b0bff700 1 -- 172.21.15.101:0/266852427 wait complete. 2023-07-18T03:04:47.872 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 10 2023-07-18T03:04:48.010 INFO:journalctl@ceph.mds.g.smithi101.stdout:Jul 18 03:04:48 smithi101 systemd[1]: Starting Ceph mds.g for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T03:04:48.440 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:48 smithi179 ceph-mon[129229]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:48.441 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:48 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/266852427' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:04:48.478 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:48 smithi101 ceph-mon[126379]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:48.478 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:48 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/266852427' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:04:48.535 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:48 smithi155 ceph-mon[128885]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:48.536 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:48 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/266852427' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:04:48.840 INFO:journalctl@ceph.mds.g.smithi101.stdout:Jul 18 03:04:48 smithi101 podman[160813]: 2023-07-18T03:04:48.840 INFO:journalctl@ceph.mds.g.smithi101.stdout:Jul 18 03:04:48 smithi101 ceph-mds[160853]: -- 172.21.15.101:0/1369232667 <== mon.0 v2:172.21.15.101:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5632d76b71e0 con 0x5632d768c800 2023-07-18T03:04:48.840 INFO:journalctl@ceph.mds.g.smithi101.stdout:Jul 18 03:04:48 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-g[160829]: starting mds.g at 2023-07-18T03:04:49.228 INFO:journalctl@ceph.mds.g.smithi101.stdout:Jul 18 03:04:48 smithi101 bash[160813]: a23f0b4bd338d5be6f81de4e4e597de9ff51272c393e138507ac76eb0e9281c6 2023-07-18T03:04:49.228 INFO:journalctl@ceph.mds.g.smithi101.stdout:Jul 18 03:04:48 smithi101 systemd[1]: Started Ceph mds.g for b1782daa-2516-11ee-9b34-001a4aab830c. 2023-07-18T03:04:49.896 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:04:50.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.122+0000 7f8a28bff700 1 -- 172.21.15.101:0/2442199248 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8a24100cf0 msgr2=0x7f8a24101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:50.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.122+0000 7f8a28bff700 1 --2- 172.21.15.101:0/2442199248 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8a24100cf0 0x7f8a24101140 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c009990 tx=0x7f8a0c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:04:50.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.122+0000 7f8a28bff700 1 -- 172.21.15.101:0/2442199248 shutdown_connections 2023-07-18T03:04:50.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.122+0000 7f8a28bff700 1 --2- 172.21.15.101:0/2442199248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a240ffb00 0x7f8a240fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:50.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.122+0000 7f8a28bff700 1 --2- 172.21.15.101:0/2442199248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a24101680 0x7f8a24107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:50.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.122+0000 7f8a28bff700 1 --2- 172.21.15.101:0/2442199248 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8a24100cf0 0x7f8a24101140 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:50.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.122+0000 7f8a28bff700 1 -- 172.21.15.101:0/2442199248 >> 172.21.15.101:0/2442199248 conn(0x7f8a240fb0e0 msgr2=0x7f8a240fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:50.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.122+0000 7f8a28bff700 1 -- 172.21.15.101:0/2442199248 shutdown_connections 2023-07-18T03:04:50.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.122+0000 7f8a28bff700 1 -- 172.21.15.101:0/2442199248 wait complete. 2023-07-18T03:04:50.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.123+0000 7f8a28bff700 1 Processor -- start 2023-07-18T03:04:50.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.123+0000 7f8a28bff700 1 -- start start 2023-07-18T03:04:50.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.123+0000 7f8a28bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8a240ffb00 0x7f8a24197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:50.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.123+0000 7f8a28bff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a24100cf0 0x7f8a24197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:50.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.123+0000 7f8a28bff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a24101680 0x7f8a2419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:50.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.123+0000 7f8a28bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2419f590 con 0x7f8a240ffb00 2023-07-18T03:04:50.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.123+0000 7f8a28bff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2419f6d0 con 0x7f8a24100cf0 2023-07-18T03:04:50.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.123+0000 7f8a28bff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2419f860 con 0x7f8a24101680 2023-07-18T03:04:50.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.124+0000 7f8a2259c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8a240ffb00 0x7f8a24197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:50.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.124+0000 7f8a2259c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8a240ffb00 0x7f8a24197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:39196/0 (socket says 172.21.15.101:39196) 2023-07-18T03:04:50.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.124+0000 7f8a2259c700 1 -- 172.21.15.101:0/677376528 learned_addr learned my addr 172.21.15.101:0/677376528 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:04:50.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.124+0000 7f8a22d9d700 1 --2- 172.21.15.101:0/677376528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a24101680 0x7f8a2419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:50.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.124+0000 7f8a1bfff700 1 --2- 172.21.15.101:0/677376528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a24100cf0 0x7f8a24197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:50.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.124+0000 7f8a2259c700 1 -- 172.21.15.101:0/677376528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a24101680 msgr2=0x7f8a2419efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:50.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.124+0000 7f8a2259c700 1 --2- 172.21.15.101:0/677376528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a24101680 0x7f8a2419efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:50.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.124+0000 7f8a2259c700 1 -- 172.21.15.101:0/677376528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a24100cf0 msgr2=0x7f8a24197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:50.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.124+0000 7f8a2259c700 1 --2- 172.21.15.101:0/677376528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a24100cf0 0x7f8a24197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:50.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.124+0000 7f8a2259c700 1 -- 172.21.15.101:0/677376528 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a0c0095f0 con 0x7f8a240ffb00 2023-07-18T03:04:50.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.124+0000 7f8a22d9d700 1 --2- 172.21.15.101:0/677376528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a24101680 0x7f8a2419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:04:50.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.124+0000 7f8a2259c700 1 --2- 172.21.15.101:0/677376528 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8a240ffb00 0x7f8a24197a10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400e970 tx=0x7f8a1400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:50.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.125+0000 7f8a1b7fe700 1 -- 172.21.15.101:0/677376528 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a14009790 con 0x7f8a240ffb00 2023-07-18T03:04:50.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.125+0000 7f8a28bff700 1 -- 172.21.15.101:0/677376528 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a2419fb40 con 0x7f8a240ffb00 2023-07-18T03:04:50.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.125+0000 7f8a1b7fe700 1 -- 172.21.15.101:0/677376528 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a1400c7e0 con 0x7f8a240ffb00 2023-07-18T03:04:50.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.125+0000 7f8a1b7fe700 1 -- 172.21.15.101:0/677376528 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a14003c70 con 0x7f8a240ffb00 2023-07-18T03:04:50.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.125+0000 7f8a28bff700 1 -- 172.21.15.101:0/677376528 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a241a00a0 con 0x7f8a240ffb00 2023-07-18T03:04:50.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.126+0000 7f8a28bff700 1 -- 172.21.15.101:0/677376528 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a2410b320 con 0x7f8a240ffb00 2023-07-18T03:04:50.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.128+0000 7f8a1b7fe700 1 -- 172.21.15.101:0/677376528 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f8a1400cc50 con 0x7f8a240ffb00 2023-07-18T03:04:50.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.129+0000 7f8a1b7fe700 1 --2- 172.21.15.101:0/677376528 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8a100a11a0 0x7f8a100a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:50.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.129+0000 7f8a1b7fe700 1 -- 172.21.15.101:0/677376528 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8a14014070 con 0x7f8a240ffb00 2023-07-18T03:04:50.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.129+0000 7f8a1bfff700 1 --2- 172.21.15.101:0/677376528 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8a100a11a0 0x7f8a100a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:50.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.130+0000 7f8a1bfff700 1 --2- 172.21.15.101:0/677376528 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8a100a11a0 0x7f8a100a3650 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c00b2e0 tx=0x7f8a0c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:50.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.133+0000 7f8a1b7fe700 1 -- 172.21.15.101:0/677376528 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8a14096210 con 0x7f8a240ffb00 2023-07-18T03:04:50.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:49 smithi179 ceph-mon[129229]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:50.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:49 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:50.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:49 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:50.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:49 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:50.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:49 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:04:50.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:49 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:04:50.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:49 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:50.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:49 smithi179 ceph-mon[129229]: Deploying daemon mds.a on smithi101 2023-07-18T03:04:50.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:49 smithi179 ceph-mon[129229]: mds.? [v2:172.21.15.101:6838/4192846769,v1:172.21.15.101:6839/4192846769] up:boot 2023-07-18T03:04:50.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:49 smithi179 ceph-mon[129229]: fsmap 10 up:standby 2023-07-18T03:04:50.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:49 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-18T03:04:50.228 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:49 smithi101 ceph-mon[126379]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:50.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:49 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:50.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:49 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:50.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:49 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:50.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:49 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:04:50.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:49 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:04:50.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:49 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:50.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:49 smithi101 ceph-mon[126379]: Deploying daemon mds.a on smithi101 2023-07-18T03:04:50.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:49 smithi101 ceph-mon[126379]: mds.? [v2:172.21.15.101:6838/4192846769,v1:172.21.15.101:6839/4192846769] up:boot 2023-07-18T03:04:50.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:49 smithi101 ceph-mon[126379]: fsmap 10 up:standby 2023-07-18T03:04:50.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:49 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-18T03:04:50.285 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:49 smithi155 ceph-mon[128885]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:50.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:49 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:50.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:49 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:50.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:49 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:50.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:49 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:04:50.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:49 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:04:50.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:49 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:50.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:49 smithi155 ceph-mon[128885]: Deploying daemon mds.a on smithi101 2023-07-18T03:04:50.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:49 smithi155 ceph-mon[128885]: mds.? [v2:172.21.15.101:6838/4192846769,v1:172.21.15.101:6839/4192846769] up:boot 2023-07-18T03:04:50.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:49 smithi155 ceph-mon[128885]: fsmap 10 up:standby 2023-07-18T03:04:50.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:49 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-18T03:04:50.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.405+0000 7f8a28bff700 1 -- 172.21.15.101:0/677376528 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a2410b510 con 0x7f8a240ffb00 2023-07-18T03:04:50.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.405+0000 7f8a1b7fe700 1 -- 172.21.15.101:0/677376528 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8091 (secure 0 0 0) 0x7f8a14003dd0 con 0x7f8a240ffb00 2023-07-18T03:04:50.390 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:04:50.390 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/4192846769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":4192846769},{"type":"v1","addr":"172.21.15.101:6839","nonce":4192846769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24515,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24529,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24541,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24562,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-07-18T03:04:50.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.408+0000 7f8a28bff700 1 -- 172.21.15.101:0/677376528 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8a100a11a0 msgr2=0x7f8a100a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:50.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.408+0000 7f8a28bff700 1 --2- 172.21.15.101:0/677376528 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8a100a11a0 0x7f8a100a3650 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c00b2e0 tx=0x7f8a0c005fb0 comp rx=0 tx=0).stop 2023-07-18T03:04:50.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.408+0000 7f8a28bff700 1 -- 172.21.15.101:0/677376528 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8a240ffb00 msgr2=0x7f8a24197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:50.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.408+0000 7f8a28bff700 1 --2- 172.21.15.101:0/677376528 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8a240ffb00 0x7f8a24197a10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400e970 tx=0x7f8a1400ec80 comp rx=0 tx=0).stop 2023-07-18T03:04:50.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.408+0000 7f8a28bff700 1 -- 172.21.15.101:0/677376528 shutdown_connections 2023-07-18T03:04:50.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.408+0000 7f8a28bff700 1 --2- 172.21.15.101:0/677376528 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8a100a11a0 0x7f8a100a3650 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:50.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.408+0000 7f8a28bff700 1 --2- 172.21.15.101:0/677376528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a24101680 0x7f8a2419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:50.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.408+0000 7f8a28bff700 1 --2- 172.21.15.101:0/677376528 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8a240ffb00 0x7f8a24197a10 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:50.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.408+0000 7f8a28bff700 1 --2- 172.21.15.101:0/677376528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a24100cf0 0x7f8a24197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:50.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.408+0000 7f8a28bff700 1 -- 172.21.15.101:0/677376528 >> 172.21.15.101:0/677376528 conn(0x7f8a240fb0e0 msgr2=0x7f8a240fd330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:50.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.408+0000 7f8a28bff700 1 -- 172.21.15.101:0/677376528 shutdown_connections 2023-07-18T03:04:50.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:50.408+0000 7f8a28bff700 1 -- 172.21.15.101:0/677376528 wait complete. 2023-07-18T03:04:50.394 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 11 2023-07-18T03:04:51.190 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:50 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/677376528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:04:51.228 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:50 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/677376528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:04:51.285 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:50 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/677376528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:04:51.729 INFO:journalctl@ceph.mds.a.smithi101.stdout:Jul 18 03:04:51 smithi101 systemd[1]: Starting Ceph mds.a for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T03:04:52.083 INFO:journalctl@ceph.mds.a.smithi101.stdout:Jul 18 03:04:51 smithi101 podman[161199]: 2023-07-18T03:04:52.083 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:51 smithi101 ceph-mon[126379]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:52.190 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:51 smithi179 ceph-mon[129229]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:52.285 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:51 smithi155 ceph-mon[128885]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:52.411 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:04:52.475 INFO:journalctl@ceph.mds.a.smithi101.stdout:Jul 18 03:04:52 smithi101 bash[161199]: fc15a5260222ccb19ecbc7c5dfc5dd7dc4cac3a87e5852e4584b8601a265e50f 2023-07-18T03:04:52.475 INFO:journalctl@ceph.mds.a.smithi101.stdout:Jul 18 03:04:52 smithi101 ceph-mds[161239]: -- 172.21.15.101:0/3889302405 <== mon.0 v2:172.21.15.101:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55741ed071e0 con 0x55741ecdc800 2023-07-18T03:04:52.475 INFO:journalctl@ceph.mds.a.smithi101.stdout:Jul 18 03:04:52 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-a[161215]: starting mds.a at 2023-07-18T03:04:52.475 INFO:journalctl@ceph.mds.a.smithi101.stdout:Jul 18 03:04:52 smithi101 systemd[1]: Started Ceph mds.a for b1782daa-2516-11ee-9b34-001a4aab830c. 2023-07-18T03:04:52.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.628+0000 7f8fb7fff700 1 -- 172.21.15.101:0/1373304781 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8fb0101680 msgr2=0x7f8fb0107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:52.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.628+0000 7f8fb7fff700 1 --2- 172.21.15.101:0/1373304781 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8fb0101680 0x7f8fb0107f40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8fa800b0a0 tx=0x7f8fa800b3b0 comp rx=0 tx=0).stop 2023-07-18T03:04:52.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.629+0000 7f8fb7fff700 1 -- 172.21.15.101:0/1373304781 shutdown_connections 2023-07-18T03:04:52.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.629+0000 7f8fb7fff700 1 --2- 172.21.15.101:0/1373304781 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8fb00ffb00 0x7f8fb00fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:52.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.629+0000 7f8fb7fff700 1 --2- 172.21.15.101:0/1373304781 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8fb0101680 0x7f8fb0107f40 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:52.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.629+0000 7f8fb7fff700 1 --2- 172.21.15.101:0/1373304781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8fb0100cf0 0x7f8fb0101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:52.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.629+0000 7f8fb7fff700 1 -- 172.21.15.101:0/1373304781 >> 172.21.15.101:0/1373304781 conn(0x7f8fb00fb0e0 msgr2=0x7f8fb00fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:52.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.629+0000 7f8fb7fff700 1 -- 172.21.15.101:0/1373304781 shutdown_connections 2023-07-18T03:04:52.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.629+0000 7f8fb7fff700 1 -- 172.21.15.101:0/1373304781 wait complete. 2023-07-18T03:04:52.616 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.630+0000 7f8fb7fff700 1 Processor -- start 2023-07-18T03:04:52.616 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.630+0000 7f8fb7fff700 1 -- start start 2023-07-18T03:04:52.616 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.630+0000 7f8fb7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8fb00ffb00 0x7f8fb00742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:52.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.630+0000 7f8fb7fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8fb0100cf0 0x7f8fb0072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:52.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.630+0000 7f8fb7fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8fb0101680 0x7f8fb0072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:52.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.630+0000 7f8fb7fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb00735e0 con 0x7f8fb0100cf0 2023-07-18T03:04:52.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.630+0000 7f8fb7fff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb0073750 con 0x7f8fb0101680 2023-07-18T03:04:52.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.630+0000 7f8fb7fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb019f960 con 0x7f8fb00ffb00 2023-07-18T03:04:52.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.630+0000 7f8fb559a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8fb0100cf0 0x7f8fb0072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:52.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.630+0000 7f8fb559a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8fb0100cf0 0x7f8fb0072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:39216/0 (socket says 172.21.15.101:39216) 2023-07-18T03:04:52.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.630+0000 7f8fb559a700 1 -- 172.21.15.101:0/2690085302 learned_addr learned my addr 172.21.15.101:0/2690085302 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:04:52.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.630+0000 7f8fb659c700 1 --2- 172.21.15.101:0/2690085302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8fb0101680 0x7f8fb0072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:52.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.631+0000 7f8fb5d9b700 1 --2- 172.21.15.101:0/2690085302 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8fb00ffb00 0x7f8fb00742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:52.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.631+0000 7f8fb559a700 1 -- 172.21.15.101:0/2690085302 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8fb00ffb00 msgr2=0x7f8fb00742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:52.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.631+0000 7f8fb559a700 1 --2- 172.21.15.101:0/2690085302 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8fb00ffb00 0x7f8fb00742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:52.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.631+0000 7f8fb559a700 1 -- 172.21.15.101:0/2690085302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8fb0101680 msgr2=0x7f8fb0072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:52.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.631+0000 7f8fb559a700 1 --2- 172.21.15.101:0/2690085302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8fb0101680 0x7f8fb0072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:52.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.631+0000 7f8fb559a700 1 -- 172.21.15.101:0/2690085302 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa8009d00 con 0x7f8fb0100cf0 2023-07-18T03:04:52.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.631+0000 7f8fb659c700 1 --2- 172.21.15.101:0/2690085302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8fb0101680 0x7f8fb0072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:04:52.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.631+0000 7f8fb559a700 1 --2- 172.21.15.101:0/2690085302 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8fb0100cf0 0x7f8fb0072900 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8fa000e9a0 tx=0x7f8fa000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:52.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.631+0000 7f8fa6ffd700 1 -- 172.21.15.101:0/2690085302 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0009790 con 0x7f8fb0100cf0 2023-07-18T03:04:52.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.631+0000 7f8fa6ffd700 1 -- 172.21.15.101:0/2690085302 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fa0010430 con 0x7f8fb0100cf0 2023-07-18T03:04:52.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.631+0000 7f8fb7fff700 1 -- 172.21.15.101:0/2690085302 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fb019fc40 con 0x7f8fb0100cf0 2023-07-18T03:04:52.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.631+0000 7f8fa6ffd700 1 -- 172.21.15.101:0/2690085302 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0018680 con 0x7f8fb0100cf0 2023-07-18T03:04:52.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.631+0000 7f8fb7fff700 1 -- 172.21.15.101:0/2690085302 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fb01a0120 con 0x7f8fb0100cf0 2023-07-18T03:04:52.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.632+0000 7f8fb7fff700 1 -- 172.21.15.101:0/2690085302 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fb010b330 con 0x7f8fb0100cf0 2023-07-18T03:04:52.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.634+0000 7f8fa6ffd700 1 -- 172.21.15.101:0/2690085302 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f8fa00105a0 con 0x7f8fb0100cf0 2023-07-18T03:04:52.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.636+0000 7f8fa6ffd700 1 --2- 172.21.15.101:0/2690085302 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8f9c0a11a0 0x7f8f9c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:52.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.636+0000 7f8fa6ffd700 1 -- 172.21.15.101:0/2690085302 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8fa0014070 con 0x7f8fb0100cf0 2023-07-18T03:04:52.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.636+0000 7f8fb5d9b700 1 --2- 172.21.15.101:0/2690085302 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8f9c0a11a0 0x7f8f9c0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:52.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.636+0000 7f8fb5d9b700 1 --2- 172.21.15.101:0/2690085302 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8f9c0a11a0 0x7f8f9c0a3650 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0100b50 tx=0x7f8fac009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:52.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.637+0000 7f8fa6ffd700 1 -- 172.21.15.101:0/2690085302 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8fa0096400 con 0x7f8fb0100cf0 2023-07-18T03:04:52.864 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.880+0000 7f8fb7fff700 1 -- 172.21.15.101:0/2690085302 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8fb0106930 con 0x7f8fb0100cf0 2023-07-18T03:04:52.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.880+0000 7f8fa6ffd700 1 -- 172.21.15.101:0/2690085302 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8091 (secure 0 0 0) 0x7f8fa0099a20 con 0x7f8fb0100cf0 2023-07-18T03:04:52.865 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:04:52.865 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/4192846769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":4192846769},{"type":"v1","addr":"172.21.15.101:6839","nonce":4192846769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24515,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24529,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24541,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24562,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-07-18T03:04:52.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.883+0000 7f8fb7fff700 1 -- 172.21.15.101:0/2690085302 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8f9c0a11a0 msgr2=0x7f8f9c0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:52.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.883+0000 7f8fb7fff700 1 --2- 172.21.15.101:0/2690085302 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8f9c0a11a0 0x7f8f9c0a3650 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0100b50 tx=0x7f8fac009260 comp rx=0 tx=0).stop 2023-07-18T03:04:52.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.883+0000 7f8fb7fff700 1 -- 172.21.15.101:0/2690085302 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8fb0100cf0 msgr2=0x7f8fb0072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:52.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.883+0000 7f8fb7fff700 1 --2- 172.21.15.101:0/2690085302 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8fb0100cf0 0x7f8fb0072900 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8fa000e9a0 tx=0x7f8fa000ecb0 comp rx=0 tx=0).stop 2023-07-18T03:04:52.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.883+0000 7f8fb7fff700 1 -- 172.21.15.101:0/2690085302 shutdown_connections 2023-07-18T03:04:52.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.883+0000 7f8fb7fff700 1 --2- 172.21.15.101:0/2690085302 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8f9c0a11a0 0x7f8f9c0a3650 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:52.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.883+0000 7f8fb7fff700 1 --2- 172.21.15.101:0/2690085302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8fb0101680 0x7f8fb0072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:52.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.883+0000 7f8fb7fff700 1 --2- 172.21.15.101:0/2690085302 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8fb00ffb00 0x7f8fb00742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:52.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.883+0000 7f8fb7fff700 1 --2- 172.21.15.101:0/2690085302 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8fb0100cf0 0x7f8fb0072900 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:52.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.883+0000 7f8fb7fff700 1 -- 172.21.15.101:0/2690085302 >> 172.21.15.101:0/2690085302 conn(0x7f8fb00fb0e0 msgr2=0x7f8fb00fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:52.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.883+0000 7f8fb7fff700 1 -- 172.21.15.101:0/2690085302 shutdown_connections 2023-07-18T03:04:52.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:52.883+0000 7f8fb7fff700 1 -- 172.21.15.101:0/2690085302 wait complete. 2023-07-18T03:04:52.869 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 11 2023-07-18T03:04:53.434 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:53 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:53.435 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:53 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:53.435 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:53 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:53.435 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:53 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:04:53.435 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:53 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:04:53.435 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:53 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:53.436 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:53 smithi155 ceph-mon[128885]: Deploying daemon mds.b on smithi155 2023-07-18T03:04:53.436 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:53 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/2690085302' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:04:53.436 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:53 smithi155 ceph-mon[128885]: mds.? [v2:172.21.15.101:6840/3824340175,v1:172.21.15.101:6841/3824340175] up:boot 2023-07-18T03:04:53.436 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:53 smithi155 ceph-mon[128885]: fsmap 11 up:standby 2023-07-18T03:04:53.436 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:53 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-18T03:04:53.441 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:53 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:53.441 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:53 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:53.441 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:53 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:53.441 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:53 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:04:53.441 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:53 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:04:53.441 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:53 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:53.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:53 smithi179 ceph-mon[129229]: Deploying daemon mds.b on smithi155 2023-07-18T03:04:53.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:53 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/2690085302' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:04:53.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:53 smithi179 ceph-mon[129229]: mds.? [v2:172.21.15.101:6840/3824340175,v1:172.21.15.101:6841/3824340175] up:boot 2023-07-18T03:04:53.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:53 smithi179 ceph-mon[129229]: fsmap 11 up:standby 2023-07-18T03:04:53.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:53 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-18T03:04:53.478 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:53 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:53.478 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:53 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:53.479 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:53 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:53.479 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:53 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:04:53.479 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:53 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:04:53.479 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:53 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:04:53.479 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:53 smithi101 ceph-mon[126379]: Deploying daemon mds.b on smithi155 2023-07-18T03:04:53.479 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:53 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2690085302' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:04:53.479 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:53 smithi101 ceph-mon[126379]: mds.? [v2:172.21.15.101:6840/3824340175,v1:172.21.15.101:6841/3824340175] up:boot 2023-07-18T03:04:53.480 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:53 smithi101 ceph-mon[126379]: fsmap 11 up:standby 2023-07-18T03:04:53.480 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:53 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-18T03:04:54.036 INFO:journalctl@ceph.mds.b.smithi155.stdout:Jul 18 03:04:53 smithi155 systemd[1]: Starting Ceph mds.b for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T03:04:54.383 INFO:journalctl@ceph.mds.b.smithi155.stdout:Jul 18 03:04:54 smithi155 podman[154141]: 2023-07-18T03:04:54.383 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:54 smithi155 ceph-mon[128885]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:54.691 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:54 smithi179 ceph-mon[129229]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:54.728 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:54 smithi101 ceph-mon[126379]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:54.785 INFO:journalctl@ceph.mds.b.smithi155.stdout:Jul 18 03:04:54 smithi155 ceph-mds[154181]: -- 172.21.15.155:0/1335067814 <== mon.2 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5598956b91e0 con 0x55989568ec00 2023-07-18T03:04:54.786 INFO:journalctl@ceph.mds.b.smithi155.stdout:Jul 18 03:04:54 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-b[154157]: starting mds.b at 2023-07-18T03:04:54.786 INFO:journalctl@ceph.mds.b.smithi155.stdout:Jul 18 03:04:54 smithi155 bash[154141]: d55ec25a20761ea741e58b64fca5e67d27959315f1e111e0d8b80dc51afdc3aa 2023-07-18T03:04:54.786 INFO:journalctl@ceph.mds.b.smithi155.stdout:Jul 18 03:04:54 smithi155 systemd[1]: Started Ceph mds.b for b1782daa-2516-11ee-9b34-001a4aab830c. 2023-07-18T03:04:54.886 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:04:55.088 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.104+0000 7f119e1ff700 1 -- 172.21.15.101:0/4167241238 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1198100560 msgr2=0x7f1198102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:55.089 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.104+0000 7f119e1ff700 1 --2- 172.21.15.101:0/4167241238 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1198100560 0x7f1198102940 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1188009990 tx=0x7f1188009ca0 comp rx=0 tx=0).stop 2023-07-18T03:04:55.089 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.104+0000 7f119e1ff700 1 -- 172.21.15.101:0/4167241238 shutdown_connections 2023-07-18T03:04:55.089 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.104+0000 7f119e1ff700 1 --2- 172.21.15.101:0/4167241238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11981057a0 0x7f1198107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:55.089 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.104+0000 7f119e1ff700 1 --2- 172.21.15.101:0/4167241238 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1198100560 0x7f1198102940 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:55.089 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.104+0000 7f119e1ff700 1 --2- 172.21.15.101:0/4167241238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1198102e80 0x7f1198105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:55.089 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.104+0000 7f119e1ff700 1 -- 172.21.15.101:0/4167241238 >> 172.21.15.101:0/4167241238 conn(0x7f11980fa000 msgr2=0x7f11980fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:55.089 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.104+0000 7f119e1ff700 1 -- 172.21.15.101:0/4167241238 shutdown_connections 2023-07-18T03:04:55.090 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.104+0000 7f119e1ff700 1 -- 172.21.15.101:0/4167241238 wait complete. 2023-07-18T03:04:55.093 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.105+0000 7f119e1ff700 1 Processor -- start 2023-07-18T03:04:55.093 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.105+0000 7f119e1ff700 1 -- start start 2023-07-18T03:04:55.093 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.105+0000 7f119e1ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1198100560 0x7f1198197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:55.093 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.105+0000 7f119e1ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1198102e80 0x7f1198197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:55.094 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.105+0000 7f119e1ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11981057a0 0x7f119819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:55.094 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.105+0000 7f119e1ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119819f590 con 0x7f1198100560 2023-07-18T03:04:55.094 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.105+0000 7f119e1ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119819f6d0 con 0x7f11981057a0 2023-07-18T03:04:55.094 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.105+0000 7f119e1ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119819f860 con 0x7f1198102e80 2023-07-18T03:04:55.094 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.105+0000 7f11977fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1198100560 0x7f1198197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:55.094 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.105+0000 7f11977fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1198100560 0x7f1198197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:39234/0 (socket says 172.21.15.101:39234) 2023-07-18T03:04:55.094 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.105+0000 7f11977fe700 1 -- 172.21.15.101:0/2226088373 learned_addr learned my addr 172.21.15.101:0/2226088373 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:04:55.094 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.106+0000 7f1196ffd700 1 --2- 172.21.15.101:0/2226088373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1198102e80 0x7f1198197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:55.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.106+0000 7f1197fff700 1 --2- 172.21.15.101:0/2226088373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11981057a0 0x7f119819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:55.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.106+0000 7f11977fe700 1 -- 172.21.15.101:0/2226088373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1198102e80 msgr2=0x7f1198197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:55.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.106+0000 7f11977fe700 1 --2- 172.21.15.101:0/2226088373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1198102e80 0x7f1198197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:55.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.106+0000 7f11977fe700 1 -- 172.21.15.101:0/2226088373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11981057a0 msgr2=0x7f119819efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:55.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.106+0000 7f11977fe700 1 --2- 172.21.15.101:0/2226088373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11981057a0 0x7f119819efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:55.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.106+0000 7f11977fe700 1 -- 172.21.15.101:0/2226088373 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11880095f0 con 0x7f1198100560 2023-07-18T03:04:55.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.106+0000 7f11977fe700 1 --2- 172.21.15.101:0/2226088373 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1198100560 0x7f1198197a10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1188005910 tx=0x7f118800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:55.096 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.106+0000 7f1194ff9700 1 -- 172.21.15.101:0/2226088373 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118801d070 con 0x7f1198100560 2023-07-18T03:04:55.096 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.106+0000 7f119e1ff700 1 -- 172.21.15.101:0/2226088373 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f119819fae0 con 0x7f1198100560 2023-07-18T03:04:55.096 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.106+0000 7f1194ff9700 1 -- 172.21.15.101:0/2226088373 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1188022470 con 0x7f1198100560 2023-07-18T03:04:55.096 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.106+0000 7f1194ff9700 1 -- 172.21.15.101:0/2226088373 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1188017650 con 0x7f1198100560 2023-07-18T03:04:55.096 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.107+0000 7f119e1ff700 1 -- 172.21.15.101:0/2226088373 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f119819ff60 con 0x7f1198100560 2023-07-18T03:04:55.096 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.108+0000 7f119e1ff700 1 -- 172.21.15.101:0/2226088373 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1198107360 con 0x7f1198100560 2023-07-18T03:04:55.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.111+0000 7f1194ff9700 1 -- 172.21.15.101:0/2226088373 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f118800fad0 con 0x7f1198100560 2023-07-18T03:04:55.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.112+0000 7f1194ff9700 1 --2- 172.21.15.101:0/2226088373 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f11800a11a0 0x7f11800a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:55.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.112+0000 7f1194ff9700 1 -- 172.21.15.101:0/2226088373 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f11880cd9f0 con 0x7f1198100560 2023-07-18T03:04:55.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.113+0000 7f1196ffd700 1 --2- 172.21.15.101:0/2226088373 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f11800a11a0 0x7f11800a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:55.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.113+0000 7f1196ffd700 1 --2- 172.21.15.101:0/2226088373 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f11800a11a0 0x7f11800a3650 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f118c005950 tx=0x7f118c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:55.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.115+0000 7f1194ff9700 1 -- 172.21.15.101:0/2226088373 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f11880975d0 con 0x7f1198100560 2023-07-18T03:04:55.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.380+0000 7f119e1ff700 1 -- 172.21.15.101:0/2226088373 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f119802c850 con 0x7f1198100560 2023-07-18T03:04:55.365 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.381+0000 7f1194ff9700 1 -- 172.21.15.101:0/2226088373 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9613 (secure 0 0 0) 0x7f118809abf0 con 0x7f1198100560 2023-07-18T03:04:55.365 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:04:55.365 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/4192846769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":4192846769},{"type":"v1","addr":"172.21.15.101:6839","nonce":4192846769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6841/3824340175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6840","nonce":3824340175},{"type":"v1","addr":"172.21.15.101:6841","nonce":3824340175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24515,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24529,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24541,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6839/2055462483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2055462483},{"type":"v1","addr":"172.21.15.155:6839","nonce":2055462483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24562,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-07-18T03:04:55.367 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.383+0000 7f119e1ff700 1 -- 172.21.15.101:0/2226088373 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f11800a11a0 msgr2=0x7f11800a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:55.367 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.383+0000 7f119e1ff700 1 --2- 172.21.15.101:0/2226088373 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f11800a11a0 0x7f11800a3650 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f118c005950 tx=0x7f118c00b410 comp rx=0 tx=0).stop 2023-07-18T03:04:55.368 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.383+0000 7f119e1ff700 1 -- 172.21.15.101:0/2226088373 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1198100560 msgr2=0x7f1198197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:55.368 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.383+0000 7f119e1ff700 1 --2- 172.21.15.101:0/2226088373 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1198100560 0x7f1198197a10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1188005910 tx=0x7f118800f690 comp rx=0 tx=0).stop 2023-07-18T03:04:55.368 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.383+0000 7f119e1ff700 1 -- 172.21.15.101:0/2226088373 shutdown_connections 2023-07-18T03:04:55.368 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.383+0000 7f119e1ff700 1 --2- 172.21.15.101:0/2226088373 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f11800a11a0 0x7f11800a3650 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:55.368 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.383+0000 7f119e1ff700 1 --2- 172.21.15.101:0/2226088373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11981057a0 0x7f119819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:55.368 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.383+0000 7f119e1ff700 1 --2- 172.21.15.101:0/2226088373 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1198100560 0x7f1198197a10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:55.368 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.383+0000 7f119e1ff700 1 --2- 172.21.15.101:0/2226088373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1198102e80 0x7f1198197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:55.369 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.383+0000 7f119e1ff700 1 -- 172.21.15.101:0/2226088373 >> 172.21.15.101:0/2226088373 conn(0x7f11980fa000 msgr2=0x7f11980fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:55.369 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.383+0000 7f119e1ff700 1 -- 172.21.15.101:0/2226088373 shutdown_connections 2023-07-18T03:04:55.369 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.383+0000 7f119e1ff700 1 -- 172.21.15.101:0/2226088373 wait complete. 2023-07-18T03:04:55.369 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 13 2023-07-18T03:04:55.387 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:04:55.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.611+0000 7f19b9dff700 1 -- 172.21.15.101:0/185840107 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f19b4100cf0 msgr2=0x7f19b4101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:55.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.611+0000 7f19b9dff700 1 --2- 172.21.15.101:0/185840107 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f19b4100cf0 0x7f19b4101140 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f199c0098f0 tx=0x7f199c009c00 comp rx=0 tx=0).stop 2023-07-18T03:04:55.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.611+0000 7f19b9dff700 1 -- 172.21.15.101:0/185840107 shutdown_connections 2023-07-18T03:04:55.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.611+0000 7f19b9dff700 1 --2- 172.21.15.101:0/185840107 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19b40ffb00 0x7f19b40fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:55.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.611+0000 7f19b9dff700 1 --2- 172.21.15.101:0/185840107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f19b4101680 0x7f19b4107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:55.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.611+0000 7f19b9dff700 1 --2- 172.21.15.101:0/185840107 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f19b4100cf0 0x7f19b4101140 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:55.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.611+0000 7f19b9dff700 1 -- 172.21.15.101:0/185840107 >> 172.21.15.101:0/185840107 conn(0x7f19b40fb0e0 msgr2=0x7f19b40fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:55.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.611+0000 7f19b9dff700 1 -- 172.21.15.101:0/185840107 shutdown_connections 2023-07-18T03:04:55.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.611+0000 7f19b9dff700 1 -- 172.21.15.101:0/185840107 wait complete. 2023-07-18T03:04:55.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.612+0000 7f19b9dff700 1 Processor -- start 2023-07-18T03:04:55.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.612+0000 7f19b9dff700 1 -- start start 2023-07-18T03:04:55.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.612+0000 7f19b9dff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19b40ffb00 0x7f19b4197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:55.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.612+0000 7f19b9dff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f19b4100cf0 0x7f19b4198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:55.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.612+0000 7f19b9dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f19b4101680 0x7f19b419f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:55.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.613+0000 7f19b9dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b419f670 con 0x7f19b4101680 2023-07-18T03:04:55.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.613+0000 7f19b9dff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b419f7b0 con 0x7f19b40ffb00 2023-07-18T03:04:55.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.613+0000 7f19b9dff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b419f940 con 0x7f19b4100cf0 2023-07-18T03:04:55.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.613+0000 7f19abfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f19b4100cf0 0x7f19b4198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:55.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.613+0000 7f19abfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f19b4100cf0 0x7f19b4198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:39832/0 (socket says 172.21.15.101:39832) 2023-07-18T03:04:55.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.613+0000 7f19abfff700 1 -- 172.21.15.101:0/2514465269 learned_addr learned my addr 172.21.15.101:0/2514465269 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:04:55.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.613+0000 7f19b3fff700 1 --2- 172.21.15.101:0/2514465269 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f19b4101680 0x7f19b419f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:55.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.613+0000 7f19b37fe700 1 --2- 172.21.15.101:0/2514465269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19b40ffb00 0x7f19b4197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:55.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.613+0000 7f19abfff700 1 -- 172.21.15.101:0/2514465269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19b40ffb00 msgr2=0x7f19b4197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:55.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.613+0000 7f19abfff700 1 --2- 172.21.15.101:0/2514465269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19b40ffb00 0x7f19b4197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:55.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.613+0000 7f19abfff700 1 -- 172.21.15.101:0/2514465269 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f19b4101680 msgr2=0x7f19b419f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:55.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.613+0000 7f19abfff700 1 --2- 172.21.15.101:0/2514465269 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f19b4101680 0x7f19b419f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:55.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.613+0000 7f19abfff700 1 -- 172.21.15.101:0/2514465269 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a4009520 con 0x7f19b4100cf0 2023-07-18T03:04:55.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.613+0000 7f19abfff700 1 --2- 172.21.15.101:0/2514465269 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f19b4100cf0 0x7f19b4198030 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f199c005e90 tx=0x7f199c00b730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:55.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.613+0000 7f19b17fa700 1 -- 172.21.15.101:0/2514465269 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199c01c070 con 0x7f19b4100cf0 2023-07-18T03:04:55.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.614+0000 7f19b17fa700 1 -- 172.21.15.101:0/2514465269 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f199c00f7f0 con 0x7f19b4100cf0 2023-07-18T03:04:55.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.614+0000 7f19b9dff700 1 -- 172.21.15.101:0/2514465269 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19ac009d00 con 0x7f19b4100cf0 2023-07-18T03:04:55.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.614+0000 7f19b17fa700 1 -- 172.21.15.101:0/2514465269 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199c0049c0 con 0x7f19b4100cf0 2023-07-18T03:04:55.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.614+0000 7f19b9dff700 1 -- 172.21.15.101:0/2514465269 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f199c009520 con 0x7f19b4100cf0 2023-07-18T03:04:55.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.615+0000 7f19b9dff700 1 -- 172.21.15.101:0/2514465269 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19b41921e0 con 0x7f19b4100cf0 2023-07-18T03:04:55.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.616+0000 7f19b17fa700 1 -- 172.21.15.101:0/2514465269 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f199c020070 con 0x7f19b4100cf0 2023-07-18T03:04:55.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.618+0000 7f19b17fa700 1 --2- 172.21.15.101:0/2514465269 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f19a00a1220 0x7f19a00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:55.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.618+0000 7f19b17fa700 1 -- 172.21.15.101:0/2514465269 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f199c0cd330 con 0x7f19b4100cf0 2023-07-18T03:04:55.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.618+0000 7f19b37fe700 1 --2- 172.21.15.101:0/2514465269 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f19a00a1220 0x7f19a00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:55.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.619+0000 7f19b37fe700 1 --2- 172.21.15.101:0/2514465269 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f19a00a1220 0x7f19a00a36d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f19b4100b50 tx=0x7f19a4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:55.604 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.620+0000 7f19b17fa700 1 -- 172.21.15.101:0/2514465269 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f199c096f10 con 0x7f19b4100cf0 2023-07-18T03:04:55.728 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:55 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:55.729 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:55 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:55.729 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:55 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:55.729 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:55 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:55.729 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:55 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:55.730 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:55 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:04:55.730 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:55 smithi101 ceph-mon[126379]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:55.730 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:55 smithi101 ceph-mon[126379]: mds.? [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] up:boot 2023-07-18T03:04:55.730 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:55 smithi101 ceph-mon[126379]: fsmap 12 up:standby 2023-07-18T03:04:55.731 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:55 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-18T03:04:55.731 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:55 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2226088373' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:04:55.813 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.829+0000 7f19b9dff700 1 -- 172.21.15.101:0/2514465269 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f19b402c850 con 0x7f19b4100cf0 2023-07-18T03:04:55.815 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:04:55.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.830+0000 7f19b17fa700 1 -- 172.21.15.101:0/2514465269 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18503 (secure 0 0 0) 0x7f199c0ba030 con 0x7f19b4100cf0 2023-07-18T03:04:55.816 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":79,"fsid":"b1782daa-2516-11ee-9b34-001a4aab830c","created":"2023-07-18T02:57:22.972483+0000","modified":"2023-07-18T03:04:12.522208+0000","last_up_change":"2023-07-18T03:04:11.429530+0000","last_in_change":"2023-07-18T03:03:54.158340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:00:43.353489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc76b5d1-38a0-4f1a-9335-756bfc40a62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6803","nonce":3610719929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6805","nonce":3610719929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6809","nonce":3610719929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6807","nonce":3610719929}]},"public_addr":"172.21.15.101:6803/3610719929","cluster_addr":"172.21.15.101:6805/3610719929","heartbeat_back_addr":"172.21.15.101:6809/3610719929","heartbeat_front_addr":"172.21.15.101:6807/3610719929","state":["exists","up"]},{"osd":1,"uuid":"2a299573-0b34-4af9-aaae-b523e5058ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6801","nonce":1030229579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6803","nonce":1030229579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6807","nonce":1030229579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6805","nonce":1030229579}]},"public_addr":"172.21.15.155:6801/1030229579","cluster_addr":"172.21.15.155:6803/1030229579","heartbeat_back_addr":"172.21.15.155:6807/1030229579","heartbeat_front_addr":"172.21.15.155:6805/1030229579","state":["exists","up"]},{"osd":2,"uuid":"123b3749-c636-48fa-9e33-6229b5575dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6801","nonce":1349487507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6803","nonce":1349487507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6807","nonce":1349487507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6805","nonce":1349487507}]},"public_addr":"172.21.15.179:6801/1349487507","cluster_addr":"172.21.15.179:6803/1349487507","heartbeat_back_addr":"172.21.15.179:6807/1349487507","heartbeat_front_addr":"172.21.15.179:6805/1349487507","state":["exists","up"]},{"osd":3,"uuid":"4b0c409d-2954-4127-833a-b196bf226d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6811","nonce":4226577463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6813","nonce":4226577463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6817","nonce":4226577463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6815","nonce":4226577463}]},"public_addr":"172.21.15.101:6811/4226577463","cluster_addr":"172.21.15.101:6813/4226577463","heartbeat_back_addr":"172.21.15.101:6817/4226577463","heartbeat_front_addr":"172.21.15.101:6815/4226577463","state":["exists","up"]},{"osd":4,"uuid":"2137f2df-984e-45ce-baf6-b4ac12c7e3c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6809","nonce":1951135805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6811","nonce":1951135805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6815","nonce":1951135805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6813","nonce":1951135805}]},"public_addr":"172.21.15.155:6809/1951135805","cluster_addr":"172.21.15.155:6811/1951135805","heartbeat_back_addr":"172.21.15.155:6815/1951135805","heartbeat_front_addr":"172.21.15.155:6813/1951135805","state":["exists","up"]},{"osd":5,"uuid":"1020bfc5-8c2f-482e-9588-66c7f98e46e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6809","nonce":2746037021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6811","nonce":2746037021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6815","nonce":2746037021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6813","nonce":2746037021}]},"public_addr":"172.21.15.179:6809/2746037021","cluster_addr":"172.21.15.179:6811/2746037021","heartbeat_back_addr":"172.21.15.179:6815/2746037021","heartbeat_front_addr":"172.21.15.179:6813/2746037021","state":["exists","up"]},{"osd":6,"uuid":"2cd99260-5c36-4603-9338-cf697970ce58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6819","nonce":958044931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6821","nonce":958044931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6825","nonce":958044931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6823","nonce":958044931}]},"public_addr":"172.21.15.101:6819/958044931","cluster_addr":"172.21.15.101:6821/958044931","heartbeat_back_addr":"172.21.15.101:6825/958044931","heartbeat_front_addr":"172.21.15.101:6823/958044931","state":["exists","up"]},{"osd":7,"uuid":"ed23fd0a-678f-4e15-b42c-cfe873315714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6817","nonce":3256322291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6819","nonce":3256322291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6823","nonce":3256322291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6821","nonce":3256322291}]},"public_addr":"172.21.15.155:6817/3256322291","cluster_addr":"172.21.15.155:6819/3256322291","heartbeat_back_addr":"172.21.15.155:6823/3256322291","heartbeat_front_addr":"172.21.15.155:6821/3256322291","state":["exists","up"]},{"osd":8,"uuid":"eca3d8e6-7baf-47a5-b991-328001c7ebef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6817","nonce":3520612559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6819","nonce":3520612559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6823","nonce":3520612559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6821","nonce":3520612559}]},"public_addr":"172.21.15.179:6817/3520612559","cluster_addr":"172.21.15.179:6819/3520612559","heartbeat_back_addr":"172.21.15.179:6823/3520612559","heartbeat_front_addr":"172.21.15.179:6821/3520612559","state":["exists","up"]},{"osd":9,"uuid":"465a7ffb-67e4-49ef-812c-1b96feacfebf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6827","nonce":2166787667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6829","nonce":2166787667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6833","nonce":2166787667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6831","nonce":2166787667}]},"public_addr":"172.21.15.101:6827/2166787667","cluster_addr":"172.21.15.101:6829/2166787667","heartbeat_back_addr":"172.21.15.101:6833/2166787667","heartbeat_front_addr":"172.21.15.101:6831/2166787667","state":["exists","up"]},{"osd":10,"uuid":"1897a027-f25d-434f-8df5-5fc2bd866578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6825","nonce":3682446349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6827","nonce":3682446349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6831","nonce":3682446349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6829","nonce":3682446349}]},"public_addr":"172.21.15.155:6825/3682446349","cluster_addr":"172.21.15.155:6827/3682446349","heartbeat_back_addr":"172.21.15.155:6831/3682446349","heartbeat_front_addr":"172.21.15.155:6829/3682446349","state":["exists","up"]},{"osd":11,"uuid":"b62effaf-45e2-4585-b8d0-db448f3474ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6825","nonce":2602640858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6827","nonce":2602640858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6831","nonce":2602640858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6829","nonce":2602640858}]},"public_addr":"172.21.15.179:6825/2602640858","cluster_addr":"172.21.15.179:6827/2602640858","heartbeat_back_addr":"172.21.15.179:6831/2602640858","heartbeat_front_addr":"172.21.15.179:6829/2602640858","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T02:59:56.074519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:19.358142+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:41.128966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:08.227054+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:31.442592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:52.751685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:19.251054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:40.698241+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:01.339150+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:27.955910+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:49.136980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:04:10.084982+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.101:6800/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:6801/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/604228291":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/4055187849":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/2844574324":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/1603248215":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6801/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6800/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/3307578182":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/373189792":"2023-07-19T02:57:56.243200+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:04:55.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.833+0000 7f19b9dff700 1 -- 172.21.15.101:0/2514465269 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f19a00a1220 msgr2=0x7f19a00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:55.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.833+0000 7f19b9dff700 1 --2- 172.21.15.101:0/2514465269 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f19a00a1220 0x7f19a00a36d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f19b4100b50 tx=0x7f19a4009260 comp rx=0 tx=0).stop 2023-07-18T03:04:55.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.833+0000 7f19b9dff700 1 -- 172.21.15.101:0/2514465269 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f19b4100cf0 msgr2=0x7f19b4198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:55.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.833+0000 7f19b9dff700 1 --2- 172.21.15.101:0/2514465269 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f19b4100cf0 0x7f19b4198030 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f199c005e90 tx=0x7f199c00b730 comp rx=0 tx=0).stop 2023-07-18T03:04:55.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.833+0000 7f19b9dff700 1 -- 172.21.15.101:0/2514465269 shutdown_connections 2023-07-18T03:04:55.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.833+0000 7f19b9dff700 1 --2- 172.21.15.101:0/2514465269 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f19a00a1220 0x7f19a00a36d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:55.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.833+0000 7f19b9dff700 1 --2- 172.21.15.101:0/2514465269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19b40ffb00 0x7f19b4197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:55.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.833+0000 7f19b9dff700 1 --2- 172.21.15.101:0/2514465269 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f19b4101680 0x7f19b419f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:55.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.833+0000 7f19b9dff700 1 --2- 172.21.15.101:0/2514465269 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f19b4100cf0 0x7f19b4198030 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:55.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.833+0000 7f19b9dff700 1 -- 172.21.15.101:0/2514465269 >> 172.21.15.101:0/2514465269 conn(0x7f19b40fb0e0 msgr2=0x7f19b41065e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:55.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.833+0000 7f19b9dff700 1 -- 172.21.15.101:0/2514465269 shutdown_connections 2023-07-18T03:04:55.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:55.833+0000 7f19b9dff700 1 -- 172.21.15.101:0/2514465269 wait complete. 2023-07-18T03:04:55.836 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:00:43.353489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-18T03:04:55.837 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-18T03:04:55.941 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:55 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:55.941 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:55 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:55.941 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:55 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:55.941 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:55 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:55.942 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:55 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:55.942 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:55 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:04:55.942 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:55 smithi179 ceph-mon[129229]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:55.942 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:55 smithi179 ceph-mon[129229]: mds.? [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] up:boot 2023-07-18T03:04:55.942 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:55 smithi179 ceph-mon[129229]: fsmap 12 up:standby 2023-07-18T03:04:55.942 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:55 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-18T03:04:55.943 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:55 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/2226088373' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:04:56.035 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:55 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:56.036 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:55 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:56.036 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:55 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:56.036 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:55 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:56.036 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:55 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:56.037 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:55 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:04:56.037 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:55 smithi155 ceph-mon[128885]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:56.037 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:55 smithi155 ceph-mon[128885]: mds.? [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] up:boot 2023-07-18T03:04:56.037 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:55 smithi155 ceph-mon[128885]: fsmap 12 up:standby 2023-07-18T03:04:56.037 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:55 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-18T03:04:56.037 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:55 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/2226088373' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:04:56.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.060+0000 7ff5813ff700 1 -- 172.21.15.101:0/416030684 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff57c0ffae0 msgr2=0x7ff57c0ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:56.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.060+0000 7ff5813ff700 1 --2- 172.21.15.101:0/416030684 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff57c0ffae0 0x7ff57c0ffef0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff56c0098f0 tx=0x7ff56c009c00 comp rx=0 tx=0).stop 2023-07-18T03:04:56.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.060+0000 7ff5813ff700 1 -- 172.21.15.101:0/416030684 shutdown_connections 2023-07-18T03:04:56.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.060+0000 7ff5813ff700 1 --2- 172.21.15.101:0/416030684 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff57c101660 0x7ff57c107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:56.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.060+0000 7ff5813ff700 1 --2- 172.21.15.101:0/416030684 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff57c0ffae0 0x7ff57c0ffef0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:56.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.060+0000 7ff5813ff700 1 --2- 172.21.15.101:0/416030684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff57c100cd0 0x7ff57c101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:56.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.060+0000 7ff5813ff700 1 -- 172.21.15.101:0/416030684 >> 172.21.15.101:0/416030684 conn(0x7ff57c0fb0e0 msgr2=0x7ff57c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:56.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.060+0000 7ff5813ff700 1 -- 172.21.15.101:0/416030684 shutdown_connections 2023-07-18T03:04:56.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.061+0000 7ff5813ff700 1 -- 172.21.15.101:0/416030684 wait complete. 2023-07-18T03:04:56.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.061+0000 7ff5813ff700 1 Processor -- start 2023-07-18T03:04:56.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.062+0000 7ff5813ff700 1 -- start start 2023-07-18T03:04:56.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.062+0000 7ff5813ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff57c0ffae0 0x7ff57c197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:56.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.062+0000 7ff5813ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff57c100cd0 0x7ff57c198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:56.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.062+0000 7ff5813ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff57c101660 0x7ff57c19f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:56.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.062+0000 7ff5813ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff57c19f670 con 0x7ff57c101660 2023-07-18T03:04:56.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.062+0000 7ff5813ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff57c19f7b0 con 0x7ff57c0ffae0 2023-07-18T03:04:56.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.062+0000 7ff5813ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff57c19f940 con 0x7ff57c100cd0 2023-07-18T03:04:56.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.062+0000 7ff57affd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff57c0ffae0 0x7ff57c197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:56.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.062+0000 7ff57affd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff57c0ffae0 0x7ff57c197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:58488/0 (socket says 172.21.15.101:58488) 2023-07-18T03:04:56.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.062+0000 7ff57affd700 1 -- 172.21.15.101:0/2681806464 learned_addr learned my addr 172.21.15.101:0/2681806464 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:04:56.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.062+0000 7ff57b7fe700 1 --2- 172.21.15.101:0/2681806464 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff57c101660 0x7ff57c19f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:56.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.062+0000 7ff57a7fc700 1 --2- 172.21.15.101:0/2681806464 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff57c100cd0 0x7ff57c198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:56.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.063+0000 7ff57affd700 1 -- 172.21.15.101:0/2681806464 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff57c100cd0 msgr2=0x7ff57c198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:56.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.063+0000 7ff57affd700 1 --2- 172.21.15.101:0/2681806464 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff57c100cd0 0x7ff57c198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:56.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.063+0000 7ff57affd700 1 -- 172.21.15.101:0/2681806464 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff57c101660 msgr2=0x7ff57c19f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:56.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.063+0000 7ff57affd700 1 --2- 172.21.15.101:0/2681806464 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff57c101660 0x7ff57c19f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:56.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.063+0000 7ff57affd700 1 -- 172.21.15.101:0/2681806464 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5700095f0 con 0x7ff57c0ffae0 2023-07-18T03:04:56.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.063+0000 7ff57a7fc700 1 --2- 172.21.15.101:0/2681806464 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff57c100cd0 0x7ff57c198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:04:56.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.063+0000 7ff57b7fe700 1 --2- 172.21.15.101:0/2681806464 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff57c101660 0x7ff57c19f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:04:56.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.063+0000 7ff57affd700 1 --2- 172.21.15.101:0/2681806464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff57c0ffae0 0x7ff57c197af0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff56c005e90 tx=0x7ff56c00b7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:56.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.063+0000 7ff563fff700 1 -- 172.21.15.101:0/2681806464 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56c01c070 con 0x7ff57c0ffae0 2023-07-18T03:04:56.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.063+0000 7ff563fff700 1 -- 172.21.15.101:0/2681806464 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff56c00f7f0 con 0x7ff57c0ffae0 2023-07-18T03:04:56.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.063+0000 7ff5813ff700 1 -- 172.21.15.101:0/2681806464 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff56c009520 con 0x7ff57c0ffae0 2023-07-18T03:04:56.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.063+0000 7ff563fff700 1 -- 172.21.15.101:0/2681806464 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56c003ba0 con 0x7ff57c0ffae0 2023-07-18T03:04:56.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.063+0000 7ff5813ff700 1 -- 172.21.15.101:0/2681806464 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff57c19fe50 con 0x7ff57c0ffae0 2023-07-18T03:04:56.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.064+0000 7ff5813ff700 1 -- 172.21.15.101:0/2681806464 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff57c1921c0 con 0x7ff57c0ffae0 2023-07-18T03:04:56.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.066+0000 7ff563fff700 1 -- 172.21.15.101:0/2681806464 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7ff56c020030 con 0x7ff57c0ffae0 2023-07-18T03:04:56.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.068+0000 7ff563fff700 1 --2- 172.21.15.101:0/2681806464 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7ff5640a1150 0x7ff5640a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:56.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.068+0000 7ff563fff700 1 -- 172.21.15.101:0/2681806464 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff56c0cd520 con 0x7ff57c0ffae0 2023-07-18T03:04:56.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.068+0000 7ff57a7fc700 1 --2- 172.21.15.101:0/2681806464 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7ff5640a1150 0x7ff5640a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:56.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.068+0000 7ff57a7fc700 1 --2- 172.21.15.101:0/2681806464 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7ff5640a1150 0x7ff5640a3600 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff570005fd0 tx=0x7ff570009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:56.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.072+0000 7ff563fff700 1 -- 172.21.15.101:0/2681806464 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff56c0971a0 con 0x7ff57c0ffae0 2023-07-18T03:04:56.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.281+0000 7ff5813ff700 1 -- 172.21.15.101:0/2681806464 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff57c02ca70 con 0x7ff57c0ffae0 2023-07-18T03:04:56.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.281+0000 7ff563fff700 1 -- 172.21.15.101:0/2681806464 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff56c09a7c0 con 0x7ff57c0ffae0 2023-07-18T03:04:56.266 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 1 2023-07-18T03:04:56.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.284+0000 7ff5813ff700 1 -- 172.21.15.101:0/2681806464 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7ff5640a1150 msgr2=0x7ff5640a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:56.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.284+0000 7ff5813ff700 1 --2- 172.21.15.101:0/2681806464 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7ff5640a1150 0x7ff5640a3600 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff570005fd0 tx=0x7ff570009310 comp rx=0 tx=0).stop 2023-07-18T03:04:56.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.284+0000 7ff5813ff700 1 -- 172.21.15.101:0/2681806464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff57c0ffae0 msgr2=0x7ff57c197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:56.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.284+0000 7ff5813ff700 1 --2- 172.21.15.101:0/2681806464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff57c0ffae0 0x7ff57c197af0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff56c005e90 tx=0x7ff56c00b7e0 comp rx=0 tx=0).stop 2023-07-18T03:04:56.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.284+0000 7ff5813ff700 1 -- 172.21.15.101:0/2681806464 shutdown_connections 2023-07-18T03:04:56.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.284+0000 7ff5813ff700 1 --2- 172.21.15.101:0/2681806464 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7ff5640a1150 0x7ff5640a3600 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:56.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.284+0000 7ff5813ff700 1 --2- 172.21.15.101:0/2681806464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff57c0ffae0 0x7ff57c197af0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:56.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.284+0000 7ff5813ff700 1 --2- 172.21.15.101:0/2681806464 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff57c101660 0x7ff57c19f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:56.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.284+0000 7ff5813ff700 1 --2- 172.21.15.101:0/2681806464 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff57c100cd0 0x7ff57c198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:56.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.284+0000 7ff5813ff700 1 -- 172.21.15.101:0/2681806464 >> 172.21.15.101:0/2681806464 conn(0x7ff57c0fb0e0 msgr2=0x7ff57c106600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:56.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.284+0000 7ff5813ff700 1 -- 172.21.15.101:0/2681806464 shutdown_connections 2023-07-18T03:04:56.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.284+0000 7ff5813ff700 1 -- 172.21.15.101:0/2681806464 wait complete. 2023-07-18T03:04:56.289 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-18T03:04:56.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.517+0000 7f78abdff700 1 -- 172.21.15.101:0/1276090201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78a40ffb00 msgr2=0x7f78a40fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:56.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.517+0000 7f78abdff700 1 --2- 172.21.15.101:0/1276090201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78a40ffb00 0x7f78a40fff10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f78a0009990 tx=0x7f78a0009ca0 comp rx=0 tx=0).stop 2023-07-18T03:04:56.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.517+0000 7f78abdff700 1 -- 172.21.15.101:0/1276090201 shutdown_connections 2023-07-18T03:04:56.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.517+0000 7f78abdff700 1 --2- 172.21.15.101:0/1276090201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78a40ffb00 0x7f78a40fff10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:56.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.517+0000 7f78abdff700 1 --2- 172.21.15.101:0/1276090201 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78a4101680 0x7f78a4107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:56.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.517+0000 7f78abdff700 1 --2- 172.21.15.101:0/1276090201 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f78a4100cf0 0x7f78a4101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:56.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.517+0000 7f78abdff700 1 -- 172.21.15.101:0/1276090201 >> 172.21.15.101:0/1276090201 conn(0x7f78a40fb0e0 msgr2=0x7f78a40fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:56.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.517+0000 7f78abdff700 1 -- 172.21.15.101:0/1276090201 shutdown_connections 2023-07-18T03:04:56.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.517+0000 7f78abdff700 1 -- 172.21.15.101:0/1276090201 wait complete. 2023-07-18T03:04:56.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.519+0000 7f78abdff700 1 Processor -- start 2023-07-18T03:04:56.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.519+0000 7f78abdff700 1 -- start start 2023-07-18T03:04:56.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.519+0000 7f78abdff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78a40ffb00 0x7f78a4197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:56.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.519+0000 7f78abdff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78a4100cf0 0x7f78a4197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:56.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.519+0000 7f78abdff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f78a4101680 0x7f78a419f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:56.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.519+0000 7f78abdff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a419f630 con 0x7f78a4101680 2023-07-18T03:04:56.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.519+0000 7f78abdff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a419f770 con 0x7f78a4100cf0 2023-07-18T03:04:56.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.519+0000 7f78abdff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a419f900 con 0x7f78a40ffb00 2023-07-18T03:04:56.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.519+0000 7f78a939a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78a4100cf0 0x7f78a4197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:56.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.519+0000 7f78a939a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78a4100cf0 0x7f78a4197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:58506/0 (socket says 172.21.15.101:58506) 2023-07-18T03:04:56.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.519+0000 7f78a939a700 1 -- 172.21.15.101:0/2501541798 learned_addr learned my addr 172.21.15.101:0/2501541798 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:04:56.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.519+0000 7f78a9b9b700 1 --2- 172.21.15.101:0/2501541798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78a40ffb00 0x7f78a4197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:56.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.519+0000 7f78aa39c700 1 --2- 172.21.15.101:0/2501541798 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f78a4101680 0x7f78a419f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:56.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.520+0000 7f78a939a700 1 -- 172.21.15.101:0/2501541798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78a40ffb00 msgr2=0x7f78a4197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:56.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.520+0000 7f78a939a700 1 --2- 172.21.15.101:0/2501541798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78a40ffb00 0x7f78a4197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:56.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.520+0000 7f78a939a700 1 -- 172.21.15.101:0/2501541798 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f78a4101680 msgr2=0x7f78a419f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:56.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.520+0000 7f78a939a700 1 --2- 172.21.15.101:0/2501541798 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f78a4101680 0x7f78a419f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:56.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.520+0000 7f78a939a700 1 -- 172.21.15.101:0/2501541798 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a00095f0 con 0x7f78a4100cf0 2023-07-18T03:04:56.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.520+0000 7f78a939a700 1 --2- 172.21.15.101:0/2501541798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78a4100cf0 0x7f78a4197ff0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f789400e9a0 tx=0x7f789400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:56.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.520+0000 7f789affd700 1 -- 172.21.15.101:0/2501541798 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7894009790 con 0x7f78a4100cf0 2023-07-18T03:04:56.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.520+0000 7f789affd700 1 -- 172.21.15.101:0/2501541798 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7894010430 con 0x7f78a4100cf0 2023-07-18T03:04:56.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.520+0000 7f78abdff700 1 -- 172.21.15.101:0/2501541798 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78a419fbe0 con 0x7f78a4100cf0 2023-07-18T03:04:56.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.520+0000 7f789affd700 1 -- 172.21.15.101:0/2501541798 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7894018680 con 0x7f78a4100cf0 2023-07-18T03:04:56.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.520+0000 7f78abdff700 1 -- 172.21.15.101:0/2501541798 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78a41a0140 con 0x7f78a4100cf0 2023-07-18T03:04:56.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.521+0000 7f78abdff700 1 -- 172.21.15.101:0/2501541798 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78a41921e0 con 0x7f78a4100cf0 2023-07-18T03:04:56.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.523+0000 7f789affd700 1 -- 172.21.15.101:0/2501541798 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f78940105a0 con 0x7f78a4100cf0 2023-07-18T03:04:56.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.525+0000 7f789affd700 1 --2- 172.21.15.101:0/2501541798 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f78900a1220 0x7f78900a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:56.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.525+0000 7f789affd700 1 -- 172.21.15.101:0/2501541798 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7894014070 con 0x7f78a4100cf0 2023-07-18T03:04:56.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.525+0000 7f78a9b9b700 1 --2- 172.21.15.101:0/2501541798 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f78900a1220 0x7f78900a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:56.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.526+0000 7f78a9b9b700 1 --2- 172.21.15.101:0/2501541798 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f78900a1220 0x7f78900a36d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f78a000b2e0 tx=0x7f78a0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:56.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.526+0000 7f789affd700 1 -- 172.21.15.101:0/2501541798 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f78940961c0 con 0x7f78a4100cf0 2023-07-18T03:04:56.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.758+0000 7f78abdff700 1 -- 172.21.15.101:0/2501541798 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f78a402c850 con 0x7f78a4100cf0 2023-07-18T03:04:56.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.759+0000 7f789affd700 1 -- 172.21.15.101:0/2501541798 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) v1 ==== 84+0+65 (secure 0 0 0) 0x7f78940997e0 con 0x7f78a4100cf0 2023-07-18T03:04:56.743 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:04:56.743 INFO:teuthology.orchestra.run.smithi101.stdout:[ 2023-07-18T03:04:56.743 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:04:56.744 INFO:teuthology.orchestra.run.smithi101.stdout: "poolnum": 1, 2023-07-18T03:04:56.744 INFO:teuthology.orchestra.run.smithi101.stdout: "poolname": ".mgr" 2023-07-18T03:04:56.744 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:04:56.744 INFO:teuthology.orchestra.run.smithi101.stdout:] 2023-07-18T03:04:56.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.762+0000 7f78abdff700 1 -- 172.21.15.101:0/2501541798 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f78900a1220 msgr2=0x7f78900a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:56.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.762+0000 7f78abdff700 1 --2- 172.21.15.101:0/2501541798 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f78900a1220 0x7f78900a36d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f78a000b2e0 tx=0x7f78a0005fb0 comp rx=0 tx=0).stop 2023-07-18T03:04:56.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.762+0000 7f78abdff700 1 -- 172.21.15.101:0/2501541798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78a4100cf0 msgr2=0x7f78a4197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:56.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.762+0000 7f78abdff700 1 --2- 172.21.15.101:0/2501541798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78a4100cf0 0x7f78a4197ff0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f789400e9a0 tx=0x7f789400ecb0 comp rx=0 tx=0).stop 2023-07-18T03:04:56.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.762+0000 7f78abdff700 1 -- 172.21.15.101:0/2501541798 shutdown_connections 2023-07-18T03:04:56.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.762+0000 7f78abdff700 1 --2- 172.21.15.101:0/2501541798 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f78900a1220 0x7f78900a36d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:56.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.762+0000 7f78abdff700 1 --2- 172.21.15.101:0/2501541798 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f78a4101680 0x7f78a419f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:56.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.762+0000 7f78abdff700 1 --2- 172.21.15.101:0/2501541798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78a40ffb00 0x7f78a4197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:56.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.762+0000 7f78abdff700 1 --2- 172.21.15.101:0/2501541798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78a4100cf0 0x7f78a4197ff0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:56.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.762+0000 7f78abdff700 1 -- 172.21.15.101:0/2501541798 >> 172.21.15.101:0/2501541798 conn(0x7f78a40fb0e0 msgr2=0x7f78a41065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:56.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.762+0000 7f78abdff700 1 -- 172.21.15.101:0/2501541798 shutdown_connections 2023-07-18T03:04:56.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.763+0000 7f78abdff700 1 -- 172.21.15.101:0/2501541798 wait complete. 2023-07-18T03:04:56.773 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-18T03:04:56.773 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-18T03:04:56.837 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:56 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2514465269' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:04:56.837 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:56 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2681806464' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:04:56.941 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:56 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/2514465269' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:04:56.941 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:56 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/2681806464' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:04:56.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.999+0000 7f19c71ff700 1 -- 172.21.15.101:0/160285289 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f19c0074550 msgr2=0x7f19c00729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:56.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.999+0000 7f19c71ff700 1 --2- 172.21.15.101:0/160285289 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f19c0074550 0x7f19c00729a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f19b4009990 tx=0x7f19b4009ca0 comp rx=0 tx=0).stop 2023-07-18T03:04:56.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.999+0000 7f19c71ff700 1 -- 172.21.15.101:0/160285289 shutdown_connections 2023-07-18T03:04:56.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.999+0000 7f19c71ff700 1 --2- 172.21.15.101:0/160285289 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f19c01079b0 0x7f19c0109e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:56.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.999+0000 7f19c71ff700 1 --2- 172.21.15.101:0/160285289 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f19c0074550 0x7f19c00729a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:56.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.999+0000 7f19c71ff700 1 --2- 172.21.15.101:0/160285289 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19c0072f70 0x7f19c00733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:56.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.999+0000 7f19c71ff700 1 -- 172.21.15.101:0/160285289 >> 172.21.15.101:0/160285289 conn(0x7f19c00fb130 msgr2=0x7f19c00fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:56.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:56.999+0000 7f19c71ff700 1 -- 172.21.15.101:0/160285289 shutdown_connections 2023-07-18T03:04:56.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:57.000+0000 7f19c71ff700 1 -- 172.21.15.101:0/160285289 wait complete. 2023-07-18T03:04:56.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:57.000+0000 7f19c71ff700 1 Processor -- start 2023-07-18T03:04:56.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:57.001+0000 7f19c71ff700 1 -- start start 2023-07-18T03:04:56.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:57.001+0000 7f19c71ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19c0072f70 0x7f19c0197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:56.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:57.001+0000 7f19c71ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f19c0074550 0x7f19c0197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:56.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:57.001+0000 7f19c71ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f19c01079b0 0x7f19c019f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:56.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:57.001+0000 7f19c71ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c019f610 con 0x7f19c0074550 2023-07-18T03:04:56.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:57.001+0000 7f19c71ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c019f750 con 0x7f19c0072f70 2023-07-18T03:04:56.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:57.001+0000 7f19c71ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c019f8e0 con 0x7f19c01079b0 2023-07-18T03:04:56.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:57.001+0000 7f19c4f9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19c0072f70 0x7f19c0197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:56.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:57.001+0000 7f19bffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f19c0074550 0x7f19c0197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:56.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:57.001+0000 7f19c4f9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19c0072f70 0x7f19c0197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:58538/0 (socket says 172.21.15.101:58538) 2023-07-18T03:04:56.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:57.002+0000 7f19c4f9b700 1 -- 172.21.15.101:0/3468400470 learned_addr learned my addr 172.21.15.101:0/3468400470 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:04:56.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:57.002+0000 7f19c579c700 1 --2- 172.21.15.101:0/3468400470 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f19c01079b0 0x7f19c019f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:56.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:57.002+0000 7f19c4f9b700 1 -- 172.21.15.101:0/3468400470 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f19c01079b0 msgr2=0x7f19c019f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:56.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:57.002+0000 7f19c4f9b700 1 --2- 172.21.15.101:0/3468400470 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f19c01079b0 0x7f19c019f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:56.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:57.002+0000 7f19c4f9b700 1 -- 172.21.15.101:0/3468400470 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f19c0074550 msgr2=0x7f19c0197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:56.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:57.002+0000 7f19c4f9b700 1 --2- 172.21.15.101:0/3468400470 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f19c0074550 0x7f19c0197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:56.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:57.002+0000 7f19c4f9b700 1 -- 172.21.15.101:0/3468400470 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b40095f0 con 0x7f19c0072f70 2023-07-18T03:04:56.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:57.002+0000 7f19c4f9b700 1 --2- 172.21.15.101:0/3468400470 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19c0072f70 0x7f19c0197a90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f19b4005720 tx=0x7f19b4009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:56.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:57.002+0000 7f19bdffb700 1 -- 172.21.15.101:0/3468400470 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b401d070 con 0x7f19c0072f70 2023-07-18T03:04:56.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:57.002+0000 7f19c71ff700 1 -- 172.21.15.101:0/3468400470 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19c019fb60 con 0x7f19c0072f70 2023-07-18T03:04:56.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:57.002+0000 7f19bdffb700 1 -- 172.21.15.101:0/3468400470 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19b4021410 con 0x7f19c0072f70 2023-07-18T03:04:56.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:57.003+0000 7f19bdffb700 1 -- 172.21.15.101:0/3468400470 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b4017650 con 0x7f19c0072f70 2023-07-18T03:04:56.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:57.003+0000 7f19c71ff700 1 -- 172.21.15.101:0/3468400470 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19c019ffe0 con 0x7f19c0072f70 2023-07-18T03:04:56.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:57.004+0000 7f19c71ff700 1 -- 172.21.15.101:0/3468400470 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19c0106060 con 0x7f19c0072f70 2023-07-18T03:04:56.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:57.006+0000 7f19bdffb700 1 -- 172.21.15.101:0/3468400470 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f19b4021580 con 0x7f19c0072f70 2023-07-18T03:04:56.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:57.008+0000 7f19bdffb700 1 --2- 172.21.15.101:0/3468400470 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f19b00a11d0 0x7f19b00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:56.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:57.008+0000 7f19bdffb700 1 -- 172.21.15.101:0/3468400470 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f19b40cd650 con 0x7f19c0072f70 2023-07-18T03:04:56.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:57.008+0000 7f19bffff700 1 --2- 172.21.15.101:0/3468400470 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f19b00a11d0 0x7f19b00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:56.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:57.009+0000 7f19bffff700 1 --2- 172.21.15.101:0/3468400470 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f19b00a11d0 0x7f19b00a3680 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f19c0074270 tx=0x7f19ac009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:56.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:57.009+0000 7f19bdffb700 1 -- 172.21.15.101:0/3468400470 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f19b4097230 con 0x7f19c0072f70 2023-07-18T03:04:57.035 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:56 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/2514465269' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:04:57.036 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:56 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/2681806464' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:04:57.206 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:57.222+0000 7f19c71ff700 1 -- 172.21.15.101:0/3468400470 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f19c002c850 con 0x7f19c0072f70 2023-07-18T03:04:57.813 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:57.829+0000 7f19bdffb700 1 -- 172.21.15.101:0/3468400470 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v80) v1 ==== 138+0+0 (secure 0 0 0) 0x7f19b409a850 con 0x7f19c0072f70 2023-07-18T03:04:57.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:57.834+0000 7f19c71ff700 1 -- 172.21.15.101:0/3468400470 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f19b00a11d0 msgr2=0x7f19b00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:57.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:57.834+0000 7f19c71ff700 1 --2- 172.21.15.101:0/3468400470 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f19b00a11d0 0x7f19b00a3680 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f19c0074270 tx=0x7f19ac009260 comp rx=0 tx=0).stop 2023-07-18T03:04:57.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:57.834+0000 7f19c71ff700 1 -- 172.21.15.101:0/3468400470 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19c0072f70 msgr2=0x7f19c0197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:57.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:57.834+0000 7f19c71ff700 1 --2- 172.21.15.101:0/3468400470 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19c0072f70 0x7f19c0197a90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f19b4005720 tx=0x7f19b4009580 comp rx=0 tx=0).stop 2023-07-18T03:04:57.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:57.834+0000 7f19c71ff700 1 -- 172.21.15.101:0/3468400470 shutdown_connections 2023-07-18T03:04:57.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:57.834+0000 7f19c71ff700 1 --2- 172.21.15.101:0/3468400470 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f19b00a11d0 0x7f19b00a3680 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:57.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:57.834+0000 7f19c71ff700 1 --2- 172.21.15.101:0/3468400470 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19c0072f70 0x7f19c0197a90 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:57.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:57.834+0000 7f19c71ff700 1 --2- 172.21.15.101:0/3468400470 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f19c01079b0 0x7f19c019f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:57.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:57.834+0000 7f19c71ff700 1 --2- 172.21.15.101:0/3468400470 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f19c0074550 0x7f19c0197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:57.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:57.834+0000 7f19c71ff700 1 -- 172.21.15.101:0/3468400470 >> 172.21.15.101:0/3468400470 conn(0x7f19c00fb130 msgr2=0x7f19c00fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:57.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:57.834+0000 7f19c71ff700 1 -- 172.21.15.101:0/3468400470 shutdown_connections 2023-07-18T03:04:57.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:57.834+0000 7f19c71ff700 1 -- 172.21.15.101:0/3468400470 wait complete. 2023-07-18T03:04:57.820 INFO:teuthology.orchestra.run.smithi101.stderr:pool 'cephfs_metadata' created 2023-07-18T03:04:57.839 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-07-18T03:04:58.035 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:57 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/2501541798' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-07-18T03:04:58.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.067+0000 7fe9979ff700 1 -- 172.21.15.101:0/3190793273 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe990100d00 msgr2=0x7fe990101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:58.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.067+0000 7fe9979ff700 1 --2- 172.21.15.101:0/3190793273 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe990100d00 0x7fe990101150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe980009990 tx=0x7fe980009ca0 comp rx=0 tx=0).stop 2023-07-18T03:04:58.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.067+0000 7fe9979ff700 1 -- 172.21.15.101:0/3190793273 shutdown_connections 2023-07-18T03:04:58.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.067+0000 7fe9979ff700 1 --2- 172.21.15.101:0/3190793273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe990101690 0x7fe990107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:58.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.067+0000 7fe9979ff700 1 --2- 172.21.15.101:0/3190793273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe9900ffb10 0x7fe9900fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:58.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.067+0000 7fe9979ff700 1 --2- 172.21.15.101:0/3190793273 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe990100d00 0x7fe990101150 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:58.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.067+0000 7fe9979ff700 1 -- 172.21.15.101:0/3190793273 >> 172.21.15.101:0/3190793273 conn(0x7fe9900fb130 msgr2=0x7fe9900fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:58.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.067+0000 7fe9979ff700 1 -- 172.21.15.101:0/3190793273 shutdown_connections 2023-07-18T03:04:58.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.067+0000 7fe9979ff700 1 -- 172.21.15.101:0/3190793273 wait complete. 2023-07-18T03:04:58.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.068+0000 7fe9979ff700 1 Processor -- start 2023-07-18T03:04:58.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.068+0000 7fe9979ff700 1 -- start start 2023-07-18T03:04:58.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.068+0000 7fe9979ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9900ffb10 0x7fe990197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:58.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.068+0000 7fe9979ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe990100d00 0x7fe990197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:58.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.068+0000 7fe9979ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe990101690 0x7fe99019f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:58.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.069+0000 7fe9979ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99019f630 con 0x7fe990100d00 2023-07-18T03:04:58.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.069+0000 7fe9979ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99019f770 con 0x7fe9900ffb10 2023-07-18T03:04:58.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.069+0000 7fe9979ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99019f900 con 0x7fe990101690 2023-07-18T03:04:58.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.069+0000 7fe99579b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9900ffb10 0x7fe990197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:58.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.069+0000 7fe99579b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9900ffb10 0x7fe990197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:58558/0 (socket says 172.21.15.101:58558) 2023-07-18T03:04:58.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.069+0000 7fe99579b700 1 -- 172.21.15.101:0/3321848812 learned_addr learned my addr 172.21.15.101:0/3321848812 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:04:58.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.069+0000 7fe994f9a700 1 --2- 172.21.15.101:0/3321848812 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe990100d00 0x7fe990197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:58.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.069+0000 7fe995f9c700 1 --2- 172.21.15.101:0/3321848812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe990101690 0x7fe99019f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:58.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.069+0000 7fe99579b700 1 -- 172.21.15.101:0/3321848812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe990101690 msgr2=0x7fe99019f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:58.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.069+0000 7fe99579b700 1 --2- 172.21.15.101:0/3321848812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe990101690 0x7fe99019f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:58.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.069+0000 7fe99579b700 1 -- 172.21.15.101:0/3321848812 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe990100d00 msgr2=0x7fe990197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:58.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.069+0000 7fe99579b700 1 --2- 172.21.15.101:0/3321848812 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe990100d00 0x7fe990197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:58.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.069+0000 7fe99579b700 1 -- 172.21.15.101:0/3321848812 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9800095f0 con 0x7fe9900ffb10 2023-07-18T03:04:58.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.069+0000 7fe99579b700 1 --2- 172.21.15.101:0/3321848812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9900ffb10 0x7fe990197ab0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe98c00efc0 tx=0x7fe98c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:58.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.070+0000 7fe9867fc700 1 -- 172.21.15.101:0/3321848812 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98c018070 con 0x7fe9900ffb10 2023-07-18T03:04:58.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.070+0000 7fe9867fc700 1 -- 172.21.15.101:0/3321848812 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe98c00f040 con 0x7fe9900ffb10 2023-07-18T03:04:58.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.070+0000 7fe9867fc700 1 -- 172.21.15.101:0/3321848812 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98c004280 con 0x7fe9900ffb10 2023-07-18T03:04:58.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.070+0000 7fe9979ff700 1 -- 172.21.15.101:0/3321848812 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe99019fb80 con 0x7fe9900ffb10 2023-07-18T03:04:58.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.070+0000 7fe9979ff700 1 -- 172.21.15.101:0/3321848812 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9901a0040 con 0x7fe9900ffb10 2023-07-18T03:04:58.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.071+0000 7fe9979ff700 1 -- 172.21.15.101:0/3321848812 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe99010b330 con 0x7fe9900ffb10 2023-07-18T03:04:58.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.073+0000 7fe9867fc700 1 -- 172.21.15.101:0/3321848812 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fe98c003a00 con 0x7fe9900ffb10 2023-07-18T03:04:58.059 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.074+0000 7fe9867fc700 1 --2- 172.21.15.101:0/3321848812 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe97c0a1150 0x7fe97c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:58.059 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.074+0000 7fe9867fc700 1 -- 172.21.15.101:0/3321848812 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7497+0+0 (secure 0 0 0) 0x7fe98c0cc480 con 0x7fe9900ffb10 2023-07-18T03:04:58.059 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.074+0000 7fe994f9a700 1 --2- 172.21.15.101:0/3321848812 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe97c0a1150 0x7fe97c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:58.059 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.075+0000 7fe994f9a700 1 --2- 172.21.15.101:0/3321848812 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe97c0a1150 0x7fe97c0a3600 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe98000b2e0 tx=0x7fe980005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:58.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.078+0000 7fe9867fc700 1 -- 172.21.15.101:0/3321848812 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe98c095f90 con 0x7fe9900ffb10 2023-07-18T03:04:58.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:57 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/2501541798' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-07-18T03:04:58.228 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:57 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2501541798' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-07-18T03:04:58.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.288+0000 7fe9979ff700 1 -- 172.21.15.101:0/3321848812 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fe9901a07c0 con 0x7fe9900ffb10 2023-07-18T03:04:58.823 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.838+0000 7fe9867fc700 1 -- 172.21.15.101:0/3321848812 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v81) v1 ==== 171+0+0 (secure 0 0 0) 0x7fe98c0995b0 con 0x7fe9900ffb10 2023-07-18T03:04:58.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.853+0000 7fe9979ff700 1 -- 172.21.15.101:0/3321848812 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe97c0a1150 msgr2=0x7fe97c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:58.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.853+0000 7fe9979ff700 1 --2- 172.21.15.101:0/3321848812 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe97c0a1150 0x7fe97c0a3600 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe98000b2e0 tx=0x7fe980005fb0 comp rx=0 tx=0).stop 2023-07-18T03:04:58.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.853+0000 7fe9979ff700 1 -- 172.21.15.101:0/3321848812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9900ffb10 msgr2=0x7fe990197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:58.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.853+0000 7fe9979ff700 1 --2- 172.21.15.101:0/3321848812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9900ffb10 0x7fe990197ab0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe98c00efc0 tx=0x7fe98c00c3a0 comp rx=0 tx=0).stop 2023-07-18T03:04:58.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.853+0000 7fe9979ff700 1 -- 172.21.15.101:0/3321848812 shutdown_connections 2023-07-18T03:04:58.839 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.853+0000 7fe9979ff700 1 --2- 172.21.15.101:0/3321848812 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe97c0a1150 0x7fe97c0a3600 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:58.839 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.853+0000 7fe9979ff700 1 --2- 172.21.15.101:0/3321848812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9900ffb10 0x7fe990197ab0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:58.839 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.853+0000 7fe9979ff700 1 --2- 172.21.15.101:0/3321848812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe990101690 0x7fe99019f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:58.840 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.853+0000 7fe9979ff700 1 --2- 172.21.15.101:0/3321848812 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe990100d00 0x7fe990197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:58.840 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.853+0000 7fe9979ff700 1 -- 172.21.15.101:0/3321848812 >> 172.21.15.101:0/3321848812 conn(0x7fe9900fb130 msgr2=0x7fe9900fd320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:58.841 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.853+0000 7fe9979ff700 1 -- 172.21.15.101:0/3321848812 shutdown_connections 2023-07-18T03:04:58.841 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:58.853+0000 7fe9979ff700 1 -- 172.21.15.101:0/3321848812 wait complete. 2023-07-18T03:04:58.842 INFO:teuthology.orchestra.run.smithi101.stderr:pool 'cephfs_data' created 2023-07-18T03:04:58.861 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-18T03:04:59.036 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:58 smithi155 ceph-mon[128885]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:59.036 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:58 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/3468400470' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-18T03:04:59.036 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:58 smithi155 ceph-mon[128885]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-18T03:04:59.037 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:58 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:59.037 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:58 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:59.037 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:58 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:59.037 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:58 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:59.038 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:58 smithi155 ceph-mon[128885]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-07-18T03:04:59.038 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:58 smithi155 ceph-mon[128885]: osdmap e80: 12 total, 12 up, 12 in 2023-07-18T03:04:59.038 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:58 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:59.038 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:58 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/3321848812' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-07-18T03:04:59.039 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:58 smithi155 ceph-mon[128885]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-07-18T03:04:59.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.083+0000 7fb0161ff700 1 -- 172.21.15.101:0/3683039197 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0100ffae0 msgr2=0x7fb0100ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:59.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.083+0000 7fb0161ff700 1 --2- 172.21.15.101:0/3683039197 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0100ffae0 0x7fb0100ffef0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb000009990 tx=0x7fb000009ca0 comp rx=0 tx=0).stop 2023-07-18T03:04:59.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.083+0000 7fb0161ff700 1 -- 172.21.15.101:0/3683039197 shutdown_connections 2023-07-18T03:04:59.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.083+0000 7fb0161ff700 1 --2- 172.21.15.101:0/3683039197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb010101660 0x7fb010107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:59.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.083+0000 7fb0161ff700 1 --2- 172.21.15.101:0/3683039197 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0100ffae0 0x7fb0100ffef0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:59.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.083+0000 7fb0161ff700 1 --2- 172.21.15.101:0/3683039197 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb010100cd0 0x7fb010101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:59.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.083+0000 7fb0161ff700 1 -- 172.21.15.101:0/3683039197 >> 172.21.15.101:0/3683039197 conn(0x7fb0100fb0e0 msgr2=0x7fb0100fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:59.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.083+0000 7fb0161ff700 1 -- 172.21.15.101:0/3683039197 shutdown_connections 2023-07-18T03:04:59.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.083+0000 7fb0161ff700 1 -- 172.21.15.101:0/3683039197 wait complete. 2023-07-18T03:04:59.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.084+0000 7fb0161ff700 1 Processor -- start 2023-07-18T03:04:59.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.084+0000 7fb0161ff700 1 -- start start 2023-07-18T03:04:59.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.085+0000 7fb0161ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0100ffae0 0x7fb010197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:59.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.085+0000 7fb0161ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb010100cd0 0x7fb010197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:59.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.085+0000 7fb0161ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb010101660 0x7fb01019f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:59.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.085+0000 7fb0161ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01019f5d0 con 0x7fb010101660 2023-07-18T03:04:59.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.085+0000 7fb0161ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01019f710 con 0x7fb0100ffae0 2023-07-18T03:04:59.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.085+0000 7fb0161ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01019f8a0 con 0x7fb010100cd0 2023-07-18T03:04:59.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.085+0000 7fb00ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb010101660 0x7fb01019f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:59.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.085+0000 7fb007fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb010100cd0 0x7fb010197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:59.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.085+0000 7fb00f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0100ffae0 0x7fb010197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:59.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.085+0000 7fb007fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb010100cd0 0x7fb010197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:38656/0 (socket says 172.21.15.101:38656) 2023-07-18T03:04:59.075 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.085+0000 7fb007fff700 1 -- 172.21.15.101:0/35296689 learned_addr learned my addr 172.21.15.101:0/35296689 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:04:59.075 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.085+0000 7fb007fff700 1 -- 172.21.15.101:0/35296689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0100ffae0 msgr2=0x7fb010197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:59.075 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.085+0000 7fb007fff700 1 --2- 172.21.15.101:0/35296689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0100ffae0 0x7fb010197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:59.075 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.085+0000 7fb007fff700 1 -- 172.21.15.101:0/35296689 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb010101660 msgr2=0x7fb01019f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:59.076 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.085+0000 7fb007fff700 1 --2- 172.21.15.101:0/35296689 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb010101660 0x7fb01019f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:59.076 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.085+0000 7fb007fff700 1 -- 172.21.15.101:0/35296689 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0000095f0 con 0x7fb010100cd0 2023-07-18T03:04:59.076 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.086+0000 7fb007fff700 1 --2- 172.21.15.101:0/35296689 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb010100cd0 0x7fb010197f90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faff800d950 tx=0x7faff800dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:59.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.086+0000 7fb00d7fa700 1 -- 172.21.15.101:0/35296689 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff800f890 con 0x7fb010100cd0 2023-07-18T03:04:59.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.086+0000 7fb00d7fa700 1 -- 172.21.15.101:0/35296689 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faff8010460 con 0x7fb010100cd0 2023-07-18T03:04:59.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.086+0000 7fb0161ff700 1 -- 172.21.15.101:0/35296689 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb01019fb80 con 0x7fb010100cd0 2023-07-18T03:04:59.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.086+0000 7fb00d7fa700 1 -- 172.21.15.101:0/35296689 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff800e610 con 0x7fb010100cd0 2023-07-18T03:04:59.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.086+0000 7fb0161ff700 1 -- 172.21.15.101:0/35296689 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0101a00e0 con 0x7fb010100cd0 2023-07-18T03:04:59.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.087+0000 7fb0161ff700 1 -- 172.21.15.101:0/35296689 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb010192160 con 0x7fb010100cd0 2023-07-18T03:04:59.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.089+0000 7fb00d7fa700 1 -- 172.21.15.101:0/35296689 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7faff80108d0 con 0x7fb010100cd0 2023-07-18T03:04:59.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.090+0000 7fb00d7fa700 1 --2- 172.21.15.101:0/35296689 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7faffc0a11d0 0x7faffc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:04:59.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.091+0000 7fb00d7fa700 1 -- 172.21.15.101:0/35296689 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7879+0+0 (secure 0 0 0) 0x7faff80cc4d0 con 0x7fb010100cd0 2023-07-18T03:04:59.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.091+0000 7fb00f7fe700 1 --2- 172.21.15.101:0/35296689 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7faffc0a11d0 0x7faffc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:04:59.081 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.091+0000 7fb00f7fe700 1 --2- 172.21.15.101:0/35296689 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7faffc0a11d0 0x7faffc0a3680 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb00000b5c0 tx=0x7fb000005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:04:59.081 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.094+0000 7fb00d7fa700 1 -- 172.21.15.101:0/35296689 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7faff8095db0 con 0x7fb010100cd0 2023-07-18T03:04:59.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:58 smithi179 ceph-mon[129229]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:59.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:58 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/3468400470' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-18T03:04:59.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:58 smithi179 ceph-mon[129229]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-18T03:04:59.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:58 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:59.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:58 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:59.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:58 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:59.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:58 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:59.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:58 smithi179 ceph-mon[129229]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-07-18T03:04:59.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:58 smithi179 ceph-mon[129229]: osdmap e80: 12 total, 12 up, 12 in 2023-07-18T03:04:59.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:58 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:59.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:58 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/3321848812' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-07-18T03:04:59.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:58 smithi179 ceph-mon[129229]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-07-18T03:04:59.228 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:58 smithi101 ceph-mon[126379]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:04:59.228 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:58 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3468400470' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-18T03:04:59.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:58 smithi101 ceph-mon[126379]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-18T03:04:59.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:58 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:59.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:58 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:59.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:58 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:59.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:58 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:59.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:58 smithi101 ceph-mon[126379]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-07-18T03:04:59.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:58 smithi101 ceph-mon[126379]: osdmap e80: 12 total, 12 up, 12 in 2023-07-18T03:04:59.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:58 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:04:59.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:58 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3321848812' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-07-18T03:04:59.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:58 smithi101 ceph-mon[126379]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-07-18T03:04:59.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.358+0000 7fb0161ff700 1 -- 172.21.15.101:0/35296689 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb01002ca70 con 0x7fb010100cd0 2023-07-18T03:04:59.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.833+0000 7fb00d7fa700 1 -- 172.21.15.101:0/35296689 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7faff80993d0 con 0x7fb010100cd0 2023-07-18T03:04:59.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.836+0000 7fb0161ff700 1 -- 172.21.15.101:0/35296689 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7faffc0a11d0 msgr2=0x7faffc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:59.822 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.836+0000 7fb0161ff700 1 --2- 172.21.15.101:0/35296689 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7faffc0a11d0 0x7faffc0a3680 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb00000b5c0 tx=0x7fb000005fd0 comp rx=0 tx=0).stop 2023-07-18T03:04:59.822 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.836+0000 7fb0161ff700 1 -- 172.21.15.101:0/35296689 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb010100cd0 msgr2=0x7fb010197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:04:59.822 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.836+0000 7fb0161ff700 1 --2- 172.21.15.101:0/35296689 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb010100cd0 0x7fb010197f90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faff800d950 tx=0x7faff800dc60 comp rx=0 tx=0).stop 2023-07-18T03:04:59.822 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.837+0000 7fb0161ff700 1 -- 172.21.15.101:0/35296689 shutdown_connections 2023-07-18T03:04:59.822 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.837+0000 7fb0161ff700 1 --2- 172.21.15.101:0/35296689 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7faffc0a11d0 0x7faffc0a3680 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:59.822 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.837+0000 7fb0161ff700 1 --2- 172.21.15.101:0/35296689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0100ffae0 0x7fb010197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:59.822 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.837+0000 7fb0161ff700 1 --2- 172.21.15.101:0/35296689 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb010101660 0x7fb01019f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:59.823 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.837+0000 7fb0161ff700 1 --2- 172.21.15.101:0/35296689 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb010100cd0 0x7fb010197f90 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:04:59.823 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.837+0000 7fb0161ff700 1 -- 172.21.15.101:0/35296689 >> 172.21.15.101:0/35296689 conn(0x7fb0100fb0e0 msgr2=0x7fb0101065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:04:59.823 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.837+0000 7fb0161ff700 1 -- 172.21.15.101:0/35296689 shutdown_connections 2023-07-18T03:04:59.823 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:04:59.838+0000 7fb0161ff700 1 -- 172.21.15.101:0/35296689 wait complete. 2023-07-18T03:04:59.824 INFO:teuthology.orchestra.run.smithi101.stderr:new fs with metadata pool 2 and data pool 3 2023-07-18T03:04:59.842 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-07-18T03:04:59.842 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-07-18T03:05:00.036 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:59 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:05:00.036 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:59 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:05:00.036 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:59 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:05:00.036 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:59 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:05:00.037 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:59 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:05:00.037 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:59 smithi155 ceph-mon[128885]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-07-18T03:05:00.037 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:59 smithi155 ceph-mon[128885]: osdmap e81: 12 total, 12 up, 12 in 2023-07-18T03:05:00.037 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:59 smithi155 ceph-mon[128885]: pgmap v269: 97 pgs: 12 creating+peering, 84 unknown, 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:05:00.037 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:59 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:05:00.037 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:59 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/35296689' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-18T03:05:00.037 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:59 smithi155 ceph-mon[128885]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-18T03:05:00.038 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:04:59 smithi155 ceph-mon[128885]: daemon mds.i assigned to filesystem cephfs as rank 0 2023-07-18T03:05:00.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.070+0000 7faf713ff700 1 -- 172.21.15.101:0/3674375875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf6c074550 msgr2=0x7faf6c0729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:00.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.070+0000 7faf713ff700 1 --2- 172.21.15.101:0/3674375875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf6c074550 0x7faf6c0729a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7faf5c0098f0 tx=0x7faf5c009c00 comp rx=0 tx=0).stop 2023-07-18T03:05:00.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.070+0000 7faf713ff700 1 -- 172.21.15.101:0/3674375875 shutdown_connections 2023-07-18T03:05:00.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.070+0000 7faf713ff700 1 --2- 172.21.15.101:0/3674375875 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf6c1079b0 0x7faf6c109e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:00.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.070+0000 7faf713ff700 1 --2- 172.21.15.101:0/3674375875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf6c074550 0x7faf6c0729a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:00.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.070+0000 7faf713ff700 1 --2- 172.21.15.101:0/3674375875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf6c072f70 0x7faf6c0733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:00.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.070+0000 7faf713ff700 1 -- 172.21.15.101:0/3674375875 >> 172.21.15.101:0/3674375875 conn(0x7faf6c0fb130 msgr2=0x7faf6c0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:00.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.070+0000 7faf713ff700 1 -- 172.21.15.101:0/3674375875 shutdown_connections 2023-07-18T03:05:00.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.070+0000 7faf713ff700 1 -- 172.21.15.101:0/3674375875 wait complete. 2023-07-18T03:05:00.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.071+0000 7faf713ff700 1 Processor -- start 2023-07-18T03:05:00.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.071+0000 7faf713ff700 1 -- start start 2023-07-18T03:05:00.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.072+0000 7faf713ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf6c072f70 0x7faf6c197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:00.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.072+0000 7faf713ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf6c074550 0x7faf6c198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:00.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.072+0000 7faf713ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf6c1079b0 0x7faf6c19f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:00.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.072+0000 7faf713ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6c19f650 con 0x7faf6c074550 2023-07-18T03:05:00.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.072+0000 7faf713ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6c19f790 con 0x7faf6c072f70 2023-07-18T03:05:00.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.072+0000 7faf713ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6c19f920 con 0x7faf6c1079b0 2023-07-18T03:05:00.059 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.072+0000 7faf6affd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf6c072f70 0x7faf6c197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:00.059 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.072+0000 7faf6a7fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf6c074550 0x7faf6c198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:00.059 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.072+0000 7faf6affd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf6c072f70 0x7faf6c197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:58588/0 (socket says 172.21.15.101:58588) 2023-07-18T03:05:00.059 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.072+0000 7faf6b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf6c1079b0 0x7faf6c19f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:00.059 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.072+0000 7faf6affd700 1 -- 172.21.15.101:0/3876953127 learned_addr learned my addr 172.21.15.101:0/3876953127 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:00.059 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.072+0000 7faf6affd700 1 -- 172.21.15.101:0/3876953127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf6c1079b0 msgr2=0x7faf6c19f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:00.059 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.072+0000 7faf6affd700 1 --2- 172.21.15.101:0/3876953127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf6c1079b0 0x7faf6c19f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:00.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.072+0000 7faf6affd700 1 -- 172.21.15.101:0/3876953127 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf6c074550 msgr2=0x7faf6c198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:00.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.072+0000 7faf6affd700 1 --2- 172.21.15.101:0/3876953127 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf6c074550 0x7faf6c198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:00.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.072+0000 7faf6affd700 1 -- 172.21.15.101:0/3876953127 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf600095f0 con 0x7faf6c072f70 2023-07-18T03:05:00.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.073+0000 7faf6affd700 1 --2- 172.21.15.101:0/3876953127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf6c072f70 0x7faf6c197ad0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faf5c009fd0 tx=0x7faf5c00b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:00.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.073+0000 7faf53fff700 1 -- 172.21.15.101:0/3876953127 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5c01c070 con 0x7faf6c072f70 2023-07-18T03:05:00.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.073+0000 7faf53fff700 1 -- 172.21.15.101:0/3876953127 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf5c00f7d0 con 0x7faf6c072f70 2023-07-18T03:05:00.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.073+0000 7faf713ff700 1 -- 172.21.15.101:0/3876953127 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf5c009520 con 0x7faf6c072f70 2023-07-18T03:05:00.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.073+0000 7faf53fff700 1 -- 172.21.15.101:0/3876953127 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5c003a40 con 0x7faf6c072f70 2023-07-18T03:05:00.061 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.073+0000 7faf713ff700 1 -- 172.21.15.101:0/3876953127 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf6c19fe30 con 0x7faf6c072f70 2023-07-18T03:05:00.061 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.074+0000 7faf713ff700 1 -- 172.21.15.101:0/3876953127 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf6c106060 con 0x7faf6c072f70 2023-07-18T03:05:00.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.076+0000 7faf53fff700 1 -- 172.21.15.101:0/3876953127 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7faf5c020030 con 0x7faf6c072f70 2023-07-18T03:05:00.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.077+0000 7faf53fff700 1 --2- 172.21.15.101:0/3876953127 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7faf540a1150 0x7faf540a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:00.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.078+0000 7faf53fff700 1 -- 172.21.15.101:0/3876953127 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7faf5c0cdd20 con 0x7faf6c072f70 2023-07-18T03:05:00.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.078+0000 7faf6a7fc700 1 --2- 172.21.15.101:0/3876953127 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7faf540a1150 0x7faf540a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:00.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.078+0000 7faf6a7fc700 1 --2- 172.21.15.101:0/3876953127 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7faf540a1150 0x7faf540a3600 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7faf6c074270 tx=0x7faf60009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:00.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.080+0000 7faf53fff700 1 -- 172.21.15.101:0/3876953127 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7faf5c0975b0 con 0x7faf6c072f70 2023-07-18T03:05:00.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:59 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:05:00.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:59 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:05:00.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:59 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:05:00.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:59 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:05:00.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:59 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:05:00.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:59 smithi179 ceph-mon[129229]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-07-18T03:05:00.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:59 smithi179 ceph-mon[129229]: osdmap e81: 12 total, 12 up, 12 in 2023-07-18T03:05:00.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:59 smithi179 ceph-mon[129229]: pgmap v269: 97 pgs: 12 creating+peering, 84 unknown, 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:05:00.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:59 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:05:00.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:59 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/35296689' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-18T03:05:00.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:59 smithi179 ceph-mon[129229]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-18T03:05:00.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:04:59 smithi179 ceph-mon[129229]: daemon mds.i assigned to filesystem cephfs as rank 0 2023-07-18T03:05:00.228 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:59 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:05:00.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:59 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:05:00.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:59 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:05:00.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:59 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:05:00.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:59 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:05:00.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:59 smithi101 ceph-mon[126379]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-07-18T03:05:00.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:59 smithi101 ceph-mon[126379]: osdmap e81: 12 total, 12 up, 12 in 2023-07-18T03:05:00.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:59 smithi101 ceph-mon[126379]: pgmap v269: 97 pgs: 12 creating+peering, 84 unknown, 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:05:00.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:59 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:05:00.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:59 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/35296689' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-18T03:05:00.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:59 smithi101 ceph-mon[126379]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-18T03:05:00.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:04:59 smithi101 ceph-mon[126379]: daemon mds.i assigned to filesystem cephfs as rank 0 2023-07-18T03:05:00.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.316+0000 7faf713ff700 1 -- 172.21.15.101:0/3876953127 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7faf6c02c850 con 0x7faf6c072f70 2023-07-18T03:05:00.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.797+0000 7faf53fff700 1 -- 172.21.15.101:0/3876953127 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v83) v1 ==== 157+0+0 (secure 0 0 0) 0x7faf5c09abd0 con 0x7faf6c072f70 2023-07-18T03:05:00.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.801+0000 7faf713ff700 1 -- 172.21.15.101:0/3876953127 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7faf540a1150 msgr2=0x7faf540a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:00.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.801+0000 7faf713ff700 1 --2- 172.21.15.101:0/3876953127 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7faf540a1150 0x7faf540a3600 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7faf6c074270 tx=0x7faf60009310 comp rx=0 tx=0).stop 2023-07-18T03:05:00.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.801+0000 7faf713ff700 1 -- 172.21.15.101:0/3876953127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf6c072f70 msgr2=0x7faf6c197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:00.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.801+0000 7faf713ff700 1 --2- 172.21.15.101:0/3876953127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf6c072f70 0x7faf6c197ad0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faf5c009fd0 tx=0x7faf5c00b790 comp rx=0 tx=0).stop 2023-07-18T03:05:00.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.801+0000 7faf713ff700 1 -- 172.21.15.101:0/3876953127 shutdown_connections 2023-07-18T03:05:00.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.801+0000 7faf713ff700 1 --2- 172.21.15.101:0/3876953127 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7faf540a1150 0x7faf540a3600 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:00.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.801+0000 7faf713ff700 1 --2- 172.21.15.101:0/3876953127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf6c072f70 0x7faf6c197ad0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:00.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.801+0000 7faf713ff700 1 --2- 172.21.15.101:0/3876953127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf6c1079b0 0x7faf6c19f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:00.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.801+0000 7faf713ff700 1 --2- 172.21.15.101:0/3876953127 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf6c074550 0x7faf6c198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:00.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.801+0000 7faf713ff700 1 -- 172.21.15.101:0/3876953127 >> 172.21.15.101:0/3876953127 conn(0x7faf6c0fb130 msgr2=0x7faf6c0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:00.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.801+0000 7faf713ff700 1 -- 172.21.15.101:0/3876953127 shutdown_connections 2023-07-18T03:05:00.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:00.801+0000 7faf713ff700 1 -- 172.21.15.101:0/3876953127 wait complete. 2023-07-18T03:05:00.807 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-07-18T03:05:01.035 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:00 smithi155 ceph-mon[128885]: osdmap e82: 12 total, 12 up, 12 in 2023-07-18T03:05:01.036 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:00 smithi155 ceph-mon[128885]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-07-18T03:05:01.036 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:00 smithi155 ceph-mon[128885]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2023-07-18T03:05:01.036 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:00 smithi155 ceph-mon[128885]: daemon mds.i is now active in filesystem cephfs as rank 0 2023-07-18T03:05:01.036 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:00 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/3876953127' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-07-18T03:05:01.036 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:00 smithi155 ceph-mon[128885]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-07-18T03:05:01.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:01.054+0000 7f88065ff700 1 -- 172.21.15.101:0/2757445670 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8800101680 msgr2=0x7f8800107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:01.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:01.054+0000 7f88065ff700 1 --2- 172.21.15.101:0/2757445670 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8800101680 0x7f8800107f40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f87f800b0a0 tx=0x7f87f800b3b0 comp rx=0 tx=0).stop 2023-07-18T03:05:01.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:01.054+0000 7f88065ff700 1 -- 172.21.15.101:0/2757445670 shutdown_connections 2023-07-18T03:05:01.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:01.054+0000 7f88065ff700 1 --2- 172.21.15.101:0/2757445670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88000ffb00 0x7f88000fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:01.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:01.054+0000 7f88065ff700 1 --2- 172.21.15.101:0/2757445670 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8800101680 0x7f8800107f40 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:01.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:01.054+0000 7f88065ff700 1 --2- 172.21.15.101:0/2757445670 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8800100cf0 0x7f8800101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:01.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:01.054+0000 7f88065ff700 1 -- 172.21.15.101:0/2757445670 >> 172.21.15.101:0/2757445670 conn(0x7f88000fb0e0 msgr2=0x7f88000fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:01.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:01.054+0000 7f88065ff700 1 -- 172.21.15.101:0/2757445670 shutdown_connections 2023-07-18T03:05:01.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:01.054+0000 7f88065ff700 1 -- 172.21.15.101:0/2757445670 wait complete. 2023-07-18T03:05:01.040 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:00 smithi179 ceph-mon[129229]: osdmap e82: 12 total, 12 up, 12 in 2023-07-18T03:05:01.040 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:00 smithi179 ceph-mon[129229]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-07-18T03:05:01.040 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:00 smithi179 ceph-mon[129229]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2023-07-18T03:05:01.040 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:00 smithi179 ceph-mon[129229]: daemon mds.i is now active in filesystem cephfs as rank 0 2023-07-18T03:05:01.041 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:00 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/3876953127' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-07-18T03:05:01.041 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:00 smithi179 ceph-mon[129229]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-07-18T03:05:01.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:01.055+0000 7f88065ff700 1 Processor -- start 2023-07-18T03:05:01.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:01.055+0000 7f88065ff700 1 -- start start 2023-07-18T03:05:01.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:01.055+0000 7f88065ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88000ffb00 0x7f8800195870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:01.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:01.055+0000 7f88065ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8800100cf0 0x7f8800195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:01.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:01.055+0000 7f88065ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8800101680 0x7f880019ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:01.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:01.055+0000 7f88065ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880019d3f0 con 0x7f8800101680 2023-07-18T03:05:01.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:01.055+0000 7f88065ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880019d530 con 0x7f88000ffb00 2023-07-18T03:05:01.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:01.055+0000 7f88065ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880019d6c0 con 0x7f8800100cf0 2023-07-18T03:05:01.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:01.055+0000 7f87fffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88000ffb00 0x7f8800195870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:01.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:01.055+0000 7f87fffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88000ffb00 0x7f8800195870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:58612/0 (socket says 172.21.15.101:58612) 2023-07-18T03:05:01.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:01.055+0000 7f87fffff700 1 -- 172.21.15.101:0/2967859629 learned_addr learned my addr 172.21.15.101:0/2967859629 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:01.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:01.055+0000 7f87ff7fe700 1 --2- 172.21.15.101:0/2967859629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8800100cf0 0x7f8800195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:01.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:01.056+0000 7f8804b9c700 1 --2- 172.21.15.101:0/2967859629 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8800101680 0x7f880019ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:01.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:01.056+0000 7f87fffff700 1 -- 172.21.15.101:0/2967859629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8800100cf0 msgr2=0x7f8800195db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:01.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:01.056+0000 7f87fffff700 1 --2- 172.21.15.101:0/2967859629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8800100cf0 0x7f8800195db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:01.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:01.056+0000 7f87fffff700 1 -- 172.21.15.101:0/2967859629 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8800101680 msgr2=0x7f880019ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:01.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:01.056+0000 7f87fffff700 1 --2- 172.21.15.101:0/2967859629 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8800101680 0x7f880019ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:01.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:01.056+0000 7f87fffff700 1 -- 172.21.15.101:0/2967859629 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f8009d00 con 0x7f88000ffb00 2023-07-18T03:05:01.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:01.056+0000 7f87ff7fe700 1 --2- 172.21.15.101:0/2967859629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8800100cf0 0x7f8800195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:05:01.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:01.056+0000 7f87fffff700 1 --2- 172.21.15.101:0/2967859629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88000ffb00 0x7f8800195870 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f87f000e970 tx=0x7f87f000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:01.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:01.056+0000 7f87fd7fa700 1 -- 172.21.15.101:0/2967859629 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f0009790 con 0x7f88000ffb00 2023-07-18T03:05:01.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:01.056+0000 7f87fd7fa700 1 -- 172.21.15.101:0/2967859629 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87f000c7e0 con 0x7f88000ffb00 2023-07-18T03:05:01.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:01.056+0000 7f88065ff700 1 -- 172.21.15.101:0/2967859629 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f880019d9a0 con 0x7f88000ffb00 2023-07-18T03:05:01.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:01.056+0000 7f87fd7fa700 1 -- 172.21.15.101:0/2967859629 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f0003c70 con 0x7f88000ffb00 2023-07-18T03:05:01.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:01.056+0000 7f88065ff700 1 -- 172.21.15.101:0/2967859629 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f880019deb0 con 0x7f88000ffb00 2023-07-18T03:05:01.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:01.057+0000 7f88065ff700 1 -- 172.21.15.101:0/2967859629 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f880010b330 con 0x7f88000ffb00 2023-07-18T03:05:01.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:01.059+0000 7f87fd7fa700 1 -- 172.21.15.101:0/2967859629 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f87f000cc50 con 0x7f88000ffb00 2023-07-18T03:05:01.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:01.061+0000 7f87fd7fa700 1 --2- 172.21.15.101:0/2967859629 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f87e80a11f0 0x7f87e80a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:01.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:01.061+0000 7f87fd7fa700 1 -- 172.21.15.101:0/2967859629 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8205+0+0 (secure 0 0 0) 0x7f87f0014070 con 0x7f88000ffb00 2023-07-18T03:05:01.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:01.061+0000 7f87ff7fe700 1 --2- 172.21.15.101:0/2967859629 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f87e80a11f0 0x7f87e80a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:01.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:01.062+0000 7f87ff7fe700 1 --2- 172.21.15.101:0/2967859629 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f87e80a11f0 0x7f87e80a36a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f87f4011490 tx=0x7f87f4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:01.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:01.063+0000 7f87fd7fa700 1 -- 172.21.15.101:0/2967859629 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f87f00967f0 con 0x7f88000ffb00 2023-07-18T03:05:01.228 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:00 smithi101 ceph-mon[126379]: osdmap e82: 12 total, 12 up, 12 in 2023-07-18T03:05:01.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:00 smithi101 ceph-mon[126379]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-07-18T03:05:01.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:00 smithi101 ceph-mon[126379]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2023-07-18T03:05:01.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:00 smithi101 ceph-mon[126379]: daemon mds.i is now active in filesystem cephfs as rank 0 2023-07-18T03:05:01.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:00 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3876953127' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-07-18T03:05:01.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:00 smithi101 ceph-mon[126379]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-07-18T03:05:01.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:01.272+0000 7f88065ff700 1 -- 172.21.15.101:0/2967859629 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f8800106850 con 0x7f88000ffb00 2023-07-18T03:05:01.440 INFO:journalctl@ceph.mds.i.smithi179.stdout:Jul 18 03:05:01 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-i[155309]: 2023-07-18T03:05:01.053+0000 7f20e6b1c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-18T03:05:02.036 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:01 smithi155 ceph-mon[128885]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-07-18T03:05:02.037 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:01 smithi155 ceph-mon[128885]: osdmap e83: 12 total, 12 up, 12 in 2023-07-18T03:05:02.037 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:01 smithi155 ceph-mon[128885]: mds.? [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] up:active 2023-07-18T03:05:02.037 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:01 smithi155 ceph-mon[128885]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-07-18T03:05:02.037 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:01 smithi155 ceph-mon[128885]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-07-18T03:05:02.037 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:01 smithi155 ceph-mon[128885]: pgmap v272: 97 pgs: 22 creating+peering, 46 unknown, 29 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 2 op/s 2023-07-18T03:05:02.038 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:01 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/2967859629' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-18T03:05:02.038 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:01 smithi155 ceph-mon[128885]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-18T03:05:02.038 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:01 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:05:02.038 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:01 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:05:02.038 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:01 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:05:02.038 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:01 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:05:02.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:01 smithi179 ceph-mon[129229]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-07-18T03:05:02.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:01 smithi179 ceph-mon[129229]: osdmap e83: 12 total, 12 up, 12 in 2023-07-18T03:05:02.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:01 smithi179 ceph-mon[129229]: mds.? [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] up:active 2023-07-18T03:05:02.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:01 smithi179 ceph-mon[129229]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-07-18T03:05:02.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:01 smithi179 ceph-mon[129229]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-07-18T03:05:02.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:01 smithi179 ceph-mon[129229]: pgmap v272: 97 pgs: 22 creating+peering, 46 unknown, 29 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 2 op/s 2023-07-18T03:05:02.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:01 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/2967859629' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-18T03:05:02.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:01 smithi179 ceph-mon[129229]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-18T03:05:02.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:01 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:05:02.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:01 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:05:02.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:01 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:05:02.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:01 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:05:02.228 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:01 smithi101 ceph-mon[126379]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-07-18T03:05:02.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:01 smithi101 ceph-mon[126379]: osdmap e83: 12 total, 12 up, 12 in 2023-07-18T03:05:02.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:01 smithi101 ceph-mon[126379]: mds.? [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] up:active 2023-07-18T03:05:02.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:01 smithi101 ceph-mon[126379]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-07-18T03:05:02.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:01 smithi101 ceph-mon[126379]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-07-18T03:05:02.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:01 smithi101 ceph-mon[126379]: pgmap v272: 97 pgs: 22 creating+peering, 46 unknown, 29 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 2 op/s 2023-07-18T03:05:02.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:01 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2967859629' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-18T03:05:02.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:01 smithi101 ceph-mon[126379]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-18T03:05:02.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:01 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:05:02.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:01 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:05:02.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:01 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:05:02.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:01 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:05:02.840 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:02.856+0000 7f87fd7fa700 1 -- 172.21.15.101:0/2967859629 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v85) v1 ==== 228+0+0 (secure 0 0 0) 0x7f87f0003dd0 con 0x7f88000ffb00 2023-07-18T03:05:02.847 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:02.863+0000 7f88065ff700 1 -- 172.21.15.101:0/2967859629 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f87e80a11f0 msgr2=0x7f87e80a36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:02.848 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:02.863+0000 7f88065ff700 1 --2- 172.21.15.101:0/2967859629 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f87e80a11f0 0x7f87e80a36a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f87f4011490 tx=0x7f87f4009260 comp rx=0 tx=0).stop 2023-07-18T03:05:02.848 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:02.863+0000 7f88065ff700 1 -- 172.21.15.101:0/2967859629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88000ffb00 msgr2=0x7f8800195870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:02.848 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:02.863+0000 7f88065ff700 1 --2- 172.21.15.101:0/2967859629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88000ffb00 0x7f8800195870 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f87f000e970 tx=0x7f87f000ec80 comp rx=0 tx=0).stop 2023-07-18T03:05:02.848 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:02.863+0000 7f88065ff700 1 -- 172.21.15.101:0/2967859629 shutdown_connections 2023-07-18T03:05:02.848 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:02.863+0000 7f88065ff700 1 --2- 172.21.15.101:0/2967859629 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f87e80a11f0 0x7f87e80a36a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:02.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:02.863+0000 7f88065ff700 1 --2- 172.21.15.101:0/2967859629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88000ffb00 0x7f8800195870 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:02.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:02.863+0000 7f88065ff700 1 --2- 172.21.15.101:0/2967859629 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8800101680 0x7f880019ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:02.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:02.863+0000 7f88065ff700 1 --2- 172.21.15.101:0/2967859629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8800100cf0 0x7f8800195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:02.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:02.863+0000 7f88065ff700 1 -- 172.21.15.101:0/2967859629 >> 172.21.15.101:0/2967859629 conn(0x7f88000fb0e0 msgr2=0x7f88000fd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:02.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:02.863+0000 7f88065ff700 1 -- 172.21.15.101:0/2967859629 shutdown_connections 2023-07-18T03:05:02.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:02.863+0000 7f88065ff700 1 -- 172.21.15.101:0/2967859629 wait complete. 2023-07-18T03:05:02.850 INFO:teuthology.orchestra.run.smithi101.stderr:pool 'cephfs_data_ec' created 2023-07-18T03:05:02.866 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-07-18T03:05:03.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.083+0000 7fa9b2bff700 1 -- 172.21.15.101:0/1607891701 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa9ac101670 msgr2=0x7fa9ac107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:03.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.083+0000 7fa9b2bff700 1 --2- 172.21.15.101:0/1607891701 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa9ac101670 0x7fa9ac107f30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa9a400b0a0 tx=0x7fa9a400b3b0 comp rx=0 tx=0).stop 2023-07-18T03:05:03.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.083+0000 7fa9b2bff700 1 -- 172.21.15.101:0/1607891701 shutdown_connections 2023-07-18T03:05:03.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.083+0000 7fa9b2bff700 1 --2- 172.21.15.101:0/1607891701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9ac0ffaf0 0x7fa9ac0fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:03.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.083+0000 7fa9b2bff700 1 --2- 172.21.15.101:0/1607891701 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa9ac101670 0x7fa9ac107f30 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:03.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.083+0000 7fa9b2bff700 1 --2- 172.21.15.101:0/1607891701 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9ac100ce0 0x7fa9ac101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:03.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.083+0000 7fa9b2bff700 1 -- 172.21.15.101:0/1607891701 >> 172.21.15.101:0/1607891701 conn(0x7fa9ac0fb130 msgr2=0x7fa9ac0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:03.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.083+0000 7fa9b2bff700 1 -- 172.21.15.101:0/1607891701 shutdown_connections 2023-07-18T03:05:03.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.083+0000 7fa9b2bff700 1 -- 172.21.15.101:0/1607891701 wait complete. 2023-07-18T03:05:03.070 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.084+0000 7fa9b2bff700 1 Processor -- start 2023-07-18T03:05:03.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.084+0000 7fa9b2bff700 1 -- start start 2023-07-18T03:05:03.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.084+0000 7fa9b2bff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9ac0ffaf0 0x7fa9ac0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:03.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.084+0000 7fa9b2bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa9ac100ce0 0x7fa9ac072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:03.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.085+0000 7fa9b2bff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9ac101670 0x7fa9ac072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:03.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.085+0000 7fa9b2bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ac0735e0 con 0x7fa9ac100ce0 2023-07-18T03:05:03.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.085+0000 7fa9b2bff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ac073750 con 0x7fa9ac101670 2023-07-18T03:05:03.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.085+0000 7fa9b2bff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ac19f9f0 con 0x7fa9ac0ffaf0 2023-07-18T03:05:03.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.085+0000 7fa9b099b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9ac0ffaf0 0x7fa9ac0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:03.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.085+0000 7fa9abfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa9ac100ce0 0x7fa9ac072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:03.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.085+0000 7fa9abfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa9ac100ce0 0x7fa9ac072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:56574/0 (socket says 172.21.15.101:56574) 2023-07-18T03:05:03.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.085+0000 7fa9b099b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9ac0ffaf0 0x7fa9ac0742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:38700/0 (socket says 172.21.15.101:38700) 2023-07-18T03:05:03.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.085+0000 7fa9abfff700 1 -- 172.21.15.101:0/3124041535 learned_addr learned my addr 172.21.15.101:0/3124041535 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:03.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.085+0000 7fa9b119c700 1 --2- 172.21.15.101:0/3124041535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9ac101670 0x7fa9ac072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:03.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.085+0000 7fa9abfff700 1 -- 172.21.15.101:0/3124041535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9ac0ffaf0 msgr2=0x7fa9ac0742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:03.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.085+0000 7fa9abfff700 1 --2- 172.21.15.101:0/3124041535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9ac0ffaf0 0x7fa9ac0742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:03.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.085+0000 7fa9abfff700 1 -- 172.21.15.101:0/3124041535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9ac101670 msgr2=0x7fa9ac072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:03.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.085+0000 7fa9abfff700 1 --2- 172.21.15.101:0/3124041535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9ac101670 0x7fa9ac072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:03.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.085+0000 7fa9abfff700 1 -- 172.21.15.101:0/3124041535 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a4009d00 con 0x7fa9ac100ce0 2023-07-18T03:05:03.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.085+0000 7fa9abfff700 1 --2- 172.21.15.101:0/3124041535 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa9ac100ce0 0x7fa9ac072900 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa99800d980 tx=0x7fa99800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:03.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.086+0000 7fa9a9ffb700 1 -- 172.21.15.101:0/3124041535 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa998004220 con 0x7fa9ac100ce0 2023-07-18T03:05:03.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.086+0000 7fa9a9ffb700 1 -- 172.21.15.101:0/3124041535 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa998007640 con 0x7fa9ac100ce0 2023-07-18T03:05:03.075 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.086+0000 7fa9b2bff700 1 -- 172.21.15.101:0/3124041535 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9ac19fcd0 con 0x7fa9ac100ce0 2023-07-18T03:05:03.075 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.086+0000 7fa9a9ffb700 1 -- 172.21.15.101:0/3124041535 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa998005690 con 0x7fa9ac100ce0 2023-07-18T03:05:03.075 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.086+0000 7fa9b2bff700 1 -- 172.21.15.101:0/3124041535 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9ac1a01b0 con 0x7fa9ac100ce0 2023-07-18T03:05:03.075 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.087+0000 7fa9b2bff700 1 -- 172.21.15.101:0/3124041535 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9ac106fd0 con 0x7fa9ac100ce0 2023-07-18T03:05:03.076 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.090+0000 7fa9a9ffb700 1 -- 172.21.15.101:0/3124041535 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fa998004750 con 0x7fa9ac100ce0 2023-07-18T03:05:03.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.092+0000 7fa9a9ffb700 1 --2- 172.21.15.101:0/3124041535 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa99c0a1150 0x7fa99c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:03.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.092+0000 7fa9a9ffb700 1 -- 172.21.15.101:0/3124041535 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8698+0+0 (secure 0 0 0) 0x7fa9980ccd60 con 0x7fa9ac100ce0 2023-07-18T03:05:03.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.092+0000 7fa9b099b700 1 --2- 172.21.15.101:0/3124041535 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa99c0a1150 0x7fa99c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:03.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.093+0000 7fa9b099b700 1 --2- 172.21.15.101:0/3124041535 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa99c0a1150 0x7fa99c0a3600 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac100b40 tx=0x7fa9a0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:03.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.094+0000 7fa9a9ffb700 1 -- 172.21.15.101:0/3124041535 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa9980962e0 con 0x7fa9ac100ce0 2023-07-18T03:05:03.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:02 smithi179 ceph-mon[129229]: osdmap e84: 12 total, 12 up, 12 in 2023-07-18T03:05:03.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:02 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:05:03.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:02 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:05:03.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:02 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:05:03.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:02 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:05:03.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:02 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:05:03.228 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:02 smithi101 ceph-mon[126379]: osdmap e84: 12 total, 12 up, 12 in 2023-07-18T03:05:03.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:02 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:05:03.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:02 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:05:03.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:02 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:05:03.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:02 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:05:03.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:02 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:05:03.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.293+0000 7fa9b2bff700 1 -- 172.21.15.101:0/3124041535 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fa9ac02caa0 con 0x7fa9ac100ce0 2023-07-18T03:05:03.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:02 smithi155 ceph-mon[128885]: osdmap e84: 12 total, 12 up, 12 in 2023-07-18T03:05:03.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:02 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:05:03.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:02 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:05:03.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:02 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:05:03.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:02 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:05:03.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:02 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:05:03.841 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.856+0000 7fa9a9ffb700 1 -- 172.21.15.101:0/3124041535 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v86) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa998016020 con 0x7fa9ac100ce0 2023-07-18T03:05:03.848 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.863+0000 7fa9b2bff700 1 -- 172.21.15.101:0/3124041535 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa99c0a1150 msgr2=0x7fa99c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:03.848 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.863+0000 7fa9b2bff700 1 --2- 172.21.15.101:0/3124041535 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa99c0a1150 0x7fa99c0a3600 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac100b40 tx=0x7fa9a0009260 comp rx=0 tx=0).stop 2023-07-18T03:05:03.848 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.863+0000 7fa9b2bff700 1 -- 172.21.15.101:0/3124041535 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa9ac100ce0 msgr2=0x7fa9ac072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:03.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.863+0000 7fa9b2bff700 1 --2- 172.21.15.101:0/3124041535 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa9ac100ce0 0x7fa9ac072900 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa99800d980 tx=0x7fa99800dc90 comp rx=0 tx=0).stop 2023-07-18T03:05:03.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.864+0000 7fa9b2bff700 1 -- 172.21.15.101:0/3124041535 shutdown_connections 2023-07-18T03:05:03.850 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.864+0000 7fa9b2bff700 1 --2- 172.21.15.101:0/3124041535 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa99c0a1150 0x7fa99c0a3600 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:03.850 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.864+0000 7fa9b2bff700 1 --2- 172.21.15.101:0/3124041535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9ac101670 0x7fa9ac072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:03.850 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.864+0000 7fa9b2bff700 1 --2- 172.21.15.101:0/3124041535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9ac0ffaf0 0x7fa9ac0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:03.850 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.864+0000 7fa9b2bff700 1 --2- 172.21.15.101:0/3124041535 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa9ac100ce0 0x7fa9ac072900 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:03.850 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.864+0000 7fa9b2bff700 1 -- 172.21.15.101:0/3124041535 >> 172.21.15.101:0/3124041535 conn(0x7fa9ac0fb130 msgr2=0x7fa9ac104110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:03.850 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.864+0000 7fa9b2bff700 1 -- 172.21.15.101:0/3124041535 shutdown_connections 2023-07-18T03:05:03.851 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:03.865+0000 7fa9b2bff700 1 -- 172.21.15.101:0/3124041535 wait complete. 2023-07-18T03:05:03.854 INFO:teuthology.orchestra.run.smithi101.stderr:set pool 4 allow_ec_overwrites to true 2023-07-18T03:05:03.879 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-07-18T03:05:04.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.093+0000 7fe13b1ff700 1 -- 172.21.15.101:0/1317158576 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe1340ffaf0 msgr2=0x7fe1340fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:04.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.093+0000 7fe13b1ff700 1 --2- 172.21.15.101:0/1317158576 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe1340ffaf0 0x7fe1340fff00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe128009990 tx=0x7fe128009ca0 comp rx=0 tx=0).stop 2023-07-18T03:05:04.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.093+0000 7fe13b1ff700 1 -- 172.21.15.101:0/1317158576 shutdown_connections 2023-07-18T03:05:04.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.093+0000 7fe13b1ff700 1 --2- 172.21.15.101:0/1317158576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe134101670 0x7fe134107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:04.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.093+0000 7fe13b1ff700 1 --2- 172.21.15.101:0/1317158576 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe1340ffaf0 0x7fe1340fff00 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:04.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.093+0000 7fe13b1ff700 1 --2- 172.21.15.101:0/1317158576 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe134100ce0 0x7fe134101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:04.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.093+0000 7fe13b1ff700 1 -- 172.21.15.101:0/1317158576 >> 172.21.15.101:0/1317158576 conn(0x7fe1340fb130 msgr2=0x7fe1340fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:04.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.093+0000 7fe13b1ff700 1 -- 172.21.15.101:0/1317158576 shutdown_connections 2023-07-18T03:05:04.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.093+0000 7fe13b1ff700 1 -- 172.21.15.101:0/1317158576 wait complete. 2023-07-18T03:05:04.081 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.094+0000 7fe13b1ff700 1 Processor -- start 2023-07-18T03:05:04.081 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.094+0000 7fe13b1ff700 1 -- start start 2023-07-18T03:05:04.081 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.095+0000 7fe13b1ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1340ffaf0 0x7fe134195890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:04.081 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.095+0000 7fe13b1ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe134100ce0 0x7fe134195dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:04.081 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.095+0000 7fe13b1ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe134101670 0x7fe13419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:04.081 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.095+0000 7fe13b1ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13419d410 con 0x7fe134101670 2023-07-18T03:05:04.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.095+0000 7fe13b1ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13419d550 con 0x7fe134100ce0 2023-07-18T03:05:04.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.095+0000 7fe13b1ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13419d6e0 con 0x7fe1340ffaf0 2023-07-18T03:05:04.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.095+0000 7fe13979c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe134101670 0x7fe13419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:04.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.095+0000 7fe13979c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe134101670 0x7fe13419ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:56590/0 (socket says 172.21.15.101:56590) 2023-07-18T03:05:04.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.095+0000 7fe13979c700 1 -- 172.21.15.101:0/1383791099 learned_addr learned my addr 172.21.15.101:0/1383791099 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:04.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.095+0000 7fe133fff700 1 --2- 172.21.15.101:0/1383791099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe134100ce0 0x7fe134195dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:04.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.095+0000 7fe138f9b700 1 --2- 172.21.15.101:0/1383791099 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1340ffaf0 0x7fe134195890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:04.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.095+0000 7fe13979c700 1 -- 172.21.15.101:0/1383791099 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1340ffaf0 msgr2=0x7fe134195890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:04.083 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.095+0000 7fe13979c700 1 --2- 172.21.15.101:0/1383791099 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1340ffaf0 0x7fe134195890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:04.083 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.095+0000 7fe13979c700 1 -- 172.21.15.101:0/1383791099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe134100ce0 msgr2=0x7fe134195dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:04.083 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.095+0000 7fe13979c700 1 --2- 172.21.15.101:0/1383791099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe134100ce0 0x7fe134195dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:04.083 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.095+0000 7fe13979c700 1 -- 172.21.15.101:0/1383791099 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1280095f0 con 0x7fe134101670 2023-07-18T03:05:04.083 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.096+0000 7fe13979c700 1 --2- 172.21.15.101:0/1383791099 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe134101670 0x7fe13419ce40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe12c007ee0 tx=0x7fe12c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:04.083 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.096+0000 7fe131ffb700 1 -- 172.21.15.101:0/1383791099 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12c00dd20 con 0x7fe134101670 2023-07-18T03:05:04.083 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.096+0000 7fe131ffb700 1 -- 172.21.15.101:0/1383791099 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe12c00f040 con 0x7fe134101670 2023-07-18T03:05:04.084 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.096+0000 7fe13b1ff700 1 -- 172.21.15.101:0/1383791099 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe13419d9c0 con 0x7fe134101670 2023-07-18T03:05:04.084 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.096+0000 7fe131ffb700 1 -- 172.21.15.101:0/1383791099 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12c0048a0 con 0x7fe134101670 2023-07-18T03:05:04.084 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.096+0000 7fe13b1ff700 1 -- 172.21.15.101:0/1383791099 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe13419de80 con 0x7fe134101670 2023-07-18T03:05:04.085 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.097+0000 7fe13b1ff700 1 -- 172.21.15.101:0/1383791099 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe13410b310 con 0x7fe134101670 2023-07-18T03:05:04.085 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.099+0000 7fe131ffb700 1 -- 172.21.15.101:0/1383791099 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fe12c003a00 con 0x7fe134101670 2023-07-18T03:05:04.085 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.100+0000 7fe131ffb700 1 --2- 172.21.15.101:0/1383791099 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe1240a1150 0x7fe1240a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:04.085 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.100+0000 7fe131ffb700 1 -- 172.21.15.101:0/1383791099 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8698+0+0 (secure 0 0 0) 0x7fe12c0ccbe0 con 0x7fe134101670 2023-07-18T03:05:04.085 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.101+0000 7fe138f9b700 1 --2- 172.21.15.101:0/1383791099 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe1240a1150 0x7fe1240a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:04.086 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.101+0000 7fe138f9b700 1 --2- 172.21.15.101:0/1383791099 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe1240a1150 0x7fe1240a3600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe12800b2e0 tx=0x7fe128005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:04.086 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.102+0000 7fe131ffb700 1 -- 172.21.15.101:0/1383791099 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe12c096100 con 0x7fe134101670 2023-07-18T03:05:04.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:03 smithi179 ceph-mon[129229]: pgmap v274: 97 pgs: 22 creating+peering, 16 unknown, 59 active+clean; 579 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2023-07-18T03:05:04.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:03 smithi179 ceph-mon[129229]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-07-18T03:05:04.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:03 smithi179 ceph-mon[129229]: osdmap e85: 12 total, 12 up, 12 in 2023-07-18T03:05:04.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:03 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/3124041535' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-07-18T03:05:04.228 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:03 smithi101 ceph-mon[126379]: pgmap v274: 97 pgs: 22 creating+peering, 16 unknown, 59 active+clean; 579 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2023-07-18T03:05:04.228 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:03 smithi101 ceph-mon[126379]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-07-18T03:05:04.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:03 smithi101 ceph-mon[126379]: osdmap e85: 12 total, 12 up, 12 in 2023-07-18T03:05:04.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:03 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3124041535' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-07-18T03:05:04.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:03 smithi155 ceph-mon[128885]: pgmap v274: 97 pgs: 22 creating+peering, 16 unknown, 59 active+clean; 579 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2023-07-18T03:05:04.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:03 smithi155 ceph-mon[128885]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-07-18T03:05:04.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:03 smithi155 ceph-mon[128885]: osdmap e85: 12 total, 12 up, 12 in 2023-07-18T03:05:04.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:03 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/3124041535' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-07-18T03:05:04.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.351+0000 7fe13b1ff700 1 -- 172.21.15.101:0/1383791099 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7fe1341069f0 con 0x7fe134101670 2023-07-18T03:05:04.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.883+0000 7fe131ffb700 1 -- 172.21.15.101:0/1383791099 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7fe12c0230b0 con 0x7fe134101670 2023-07-18T03:05:04.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.885+0000 7fe13b1ff700 1 -- 172.21.15.101:0/1383791099 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe1240a1150 msgr2=0x7fe1240a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:04.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.885+0000 7fe13b1ff700 1 --2- 172.21.15.101:0/1383791099 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe1240a1150 0x7fe1240a3600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe12800b2e0 tx=0x7fe128005fb0 comp rx=0 tx=0).stop 2023-07-18T03:05:04.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.885+0000 7fe13b1ff700 1 -- 172.21.15.101:0/1383791099 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe134101670 msgr2=0x7fe13419ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:04.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.885+0000 7fe13b1ff700 1 --2- 172.21.15.101:0/1383791099 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe134101670 0x7fe13419ce40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe12c007ee0 tx=0x7fe12c00d3b0 comp rx=0 tx=0).stop 2023-07-18T03:05:04.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.885+0000 7fe13b1ff700 1 -- 172.21.15.101:0/1383791099 shutdown_connections 2023-07-18T03:05:04.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.885+0000 7fe13b1ff700 1 --2- 172.21.15.101:0/1383791099 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe1240a1150 0x7fe1240a3600 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:04.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.885+0000 7fe13b1ff700 1 --2- 172.21.15.101:0/1383791099 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe134101670 0x7fe13419ce40 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:04.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.885+0000 7fe13b1ff700 1 --2- 172.21.15.101:0/1383791099 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1340ffaf0 0x7fe134195890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:04.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.885+0000 7fe13b1ff700 1 --2- 172.21.15.101:0/1383791099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe134100ce0 0x7fe134195dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:04.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.885+0000 7fe13b1ff700 1 -- 172.21.15.101:0/1383791099 >> 172.21.15.101:0/1383791099 conn(0x7fe1340fb130 msgr2=0x7fe1340fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:04.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.885+0000 7fe13b1ff700 1 -- 172.21.15.101:0/1383791099 shutdown_connections 2023-07-18T03:05:04.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:04.886+0000 7fe13b1ff700 1 -- 172.21.15.101:0/1383791099 wait complete. 2023-07-18T03:05:04.872 INFO:teuthology.orchestra.run.smithi101.stderr:added data pool 4 to fsmap 2023-07-18T03:05:04.890 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:05:05.087 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.103+0000 7f4a7a9ff700 1 -- 172.21.15.101:0/3919085486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a74100550 msgr2=0x7f4a74102930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:05.088 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.103+0000 7f4a7a9ff700 1 --2- 172.21.15.101:0/3919085486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a74100550 0x7f4a74102930 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4a64009990 tx=0x7f4a64009ca0 comp rx=0 tx=0).stop 2023-07-18T03:05:05.088 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.103+0000 7f4a7a9ff700 1 -- 172.21.15.101:0/3919085486 shutdown_connections 2023-07-18T03:05:05.088 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.103+0000 7f4a7a9ff700 1 --2- 172.21.15.101:0/3919085486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a74100550 0x7f4a74102930 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:05.088 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.103+0000 7f4a7a9ff700 1 --2- 172.21.15.101:0/3919085486 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4a74105790 0x7f4a74107bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:05.088 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.103+0000 7f4a7a9ff700 1 --2- 172.21.15.101:0/3919085486 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a74102e70 0x7f4a74105250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:05.088 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.103+0000 7f4a7a9ff700 1 -- 172.21.15.101:0/3919085486 >> 172.21.15.101:0/3919085486 conn(0x7f4a740f9fd0 msgr2=0x7f4a740fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:05.089 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.103+0000 7f4a7a9ff700 1 -- 172.21.15.101:0/3919085486 shutdown_connections 2023-07-18T03:05:05.089 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.103+0000 7f4a7a9ff700 1 -- 172.21.15.101:0/3919085486 wait complete. 2023-07-18T03:05:05.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.104+0000 7f4a7a9ff700 1 Processor -- start 2023-07-18T03:05:05.096 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.104+0000 7f4a7a9ff700 1 -- start start 2023-07-18T03:05:05.096 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.104+0000 7f4a7a9ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a74100550 0x7f4a741979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:05.096 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.104+0000 7f4a7a9ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a74102e70 0x7f4a74197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:05.096 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.104+0000 7f4a7a9ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4a74105790 0x7f4a7419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:05.096 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.104+0000 7f4a7a9ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7419f570 con 0x7f4a74105790 2023-07-18T03:05:05.096 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.104+0000 7f4a7a9ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7419f6b0 con 0x7f4a74100550 2023-07-18T03:05:05.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.105+0000 7f4a7a9ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7419f840 con 0x7f4a74102e70 2023-07-18T03:05:05.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.105+0000 7f4a73fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a74100550 0x7f4a741979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:05.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.105+0000 7f4a73fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a74100550 0x7f4a741979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:58676/0 (socket says 172.21.15.101:58676) 2023-07-18T03:05:05.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.105+0000 7f4a73fff700 1 -- 172.21.15.101:0/3823452627 learned_addr learned my addr 172.21.15.101:0/3823452627 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:05.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.105+0000 7f4a78f9c700 1 --2- 172.21.15.101:0/3823452627 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4a74105790 0x7f4a7419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:05.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.105+0000 7f4a6bfff700 1 --2- 172.21.15.101:0/3823452627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a74102e70 0x7f4a74197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:05.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.105+0000 7f4a73fff700 1 -- 172.21.15.101:0/3823452627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a74102e70 msgr2=0x7f4a74197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:05.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.105+0000 7f4a73fff700 1 --2- 172.21.15.101:0/3823452627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a74102e70 0x7f4a74197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:05.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.105+0000 7f4a73fff700 1 -- 172.21.15.101:0/3823452627 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4a74105790 msgr2=0x7f4a7419efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:05.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.105+0000 7f4a73fff700 1 --2- 172.21.15.101:0/3823452627 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4a74105790 0x7f4a7419efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:05.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.105+0000 7f4a73fff700 1 -- 172.21.15.101:0/3823452627 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a640095f0 con 0x7f4a74100550 2023-07-18T03:05:05.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.105+0000 7f4a73fff700 1 --2- 172.21.15.101:0/3823452627 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a74100550 0x7f4a741979f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4a64004740 tx=0x7f4a64009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:05.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.106+0000 7f4a71ffb700 1 -- 172.21.15.101:0/3823452627 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6401d070 con 0x7f4a74100550 2023-07-18T03:05:05.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.106+0000 7f4a71ffb700 1 -- 172.21.15.101:0/3823452627 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a64004830 con 0x7f4a74100550 2023-07-18T03:05:05.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.106+0000 7f4a7a9ff700 1 -- 172.21.15.101:0/3823452627 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a7419fac0 con 0x7f4a74100550 2023-07-18T03:05:05.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.106+0000 7f4a71ffb700 1 -- 172.21.15.101:0/3823452627 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a640053c0 con 0x7f4a74100550 2023-07-18T03:05:05.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.106+0000 7f4a7a9ff700 1 -- 172.21.15.101:0/3823452627 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a7419fe80 con 0x7f4a74100550 2023-07-18T03:05:05.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.107+0000 7f4a7a9ff700 1 -- 172.21.15.101:0/3823452627 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a74107350 con 0x7f4a74100550 2023-07-18T03:05:05.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.109+0000 7f4a71ffb700 1 -- 172.21.15.101:0/3823452627 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f4a64005580 con 0x7f4a74100550 2023-07-18T03:05:05.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.110+0000 7f4a71ffb700 1 --2- 172.21.15.101:0/3823452627 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f4a600a11a0 0x7f4a600a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:05.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.110+0000 7f4a71ffb700 1 -- 172.21.15.101:0/3823452627 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4a640cdcc0 con 0x7f4a74100550 2023-07-18T03:05:05.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.111+0000 7f4a6bfff700 1 --2- 172.21.15.101:0/3823452627 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f4a600a11a0 0x7f4a600a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:05.100 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.111+0000 7f4a6bfff700 1 --2- 172.21.15.101:0/3823452627 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f4a600a11a0 0x7f4a600a3650 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c009c60 tx=0x7f4a5c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:05.100 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.112+0000 7f4a71ffb700 1 -- 172.21.15.101:0/3823452627 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4a64097250 con 0x7f4a74100550 2023-07-18T03:05:05.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:04 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/3124041535' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-07-18T03:05:05.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:04 smithi179 ceph-mon[129229]: osdmap e86: 12 total, 12 up, 12 in 2023-07-18T03:05:05.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:04 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1383791099' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-07-18T03:05:05.228 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:04 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3124041535' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-07-18T03:05:05.228 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:04 smithi101 ceph-mon[126379]: osdmap e86: 12 total, 12 up, 12 in 2023-07-18T03:05:05.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:04 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1383791099' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-07-18T03:05:05.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:04 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/3124041535' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-07-18T03:05:05.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:04 smithi155 ceph-mon[128885]: osdmap e86: 12 total, 12 up, 12 in 2023-07-18T03:05:05.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:04 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1383791099' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-07-18T03:05:05.338 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.354+0000 7f4a7a9ff700 1 -- 172.21.15.101:0/3823452627 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a7402c850 con 0x7f4a74100550 2023-07-18T03:05:05.339 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.355+0000 7f4a71ffb700 1 -- 172.21.15.101:0/3823452627 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+10797 (secure 0 0 0) 0x7f4a6409a870 con 0x7f4a74100550 2023-07-18T03:05:05.340 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:05:05.340 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/4192846769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":4192846769},{"type":"v1","addr":"172.21.15.101:6839","nonce":4192846769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6841/3824340175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6840","nonce":3824340175},{"type":"v1","addr":"172.21.15.101:6841","nonce":3824340175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24515,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24529,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24541,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6839/2055462483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2055462483},{"type":"v1","addr":"172.21.15.155:6839","nonce":2055462483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-18T03:04:59.361185+0000","modified":"2023-07-18T03:05:04.873332+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24562":{"gid":24562,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-07-18T03:05:05.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.357+0000 7f4a7a9ff700 1 -- 172.21.15.101:0/3823452627 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f4a600a11a0 msgr2=0x7f4a600a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:05.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.357+0000 7f4a7a9ff700 1 --2- 172.21.15.101:0/3823452627 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f4a600a11a0 0x7f4a600a3650 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c009c60 tx=0x7f4a5c009260 comp rx=0 tx=0).stop 2023-07-18T03:05:05.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.357+0000 7f4a7a9ff700 1 -- 172.21.15.101:0/3823452627 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a74100550 msgr2=0x7f4a741979f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:05.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.357+0000 7f4a7a9ff700 1 --2- 172.21.15.101:0/3823452627 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a74100550 0x7f4a741979f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4a64004740 tx=0x7f4a64009580 comp rx=0 tx=0).stop 2023-07-18T03:05:05.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.357+0000 7f4a7a9ff700 1 -- 172.21.15.101:0/3823452627 shutdown_connections 2023-07-18T03:05:05.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.357+0000 7f4a7a9ff700 1 --2- 172.21.15.101:0/3823452627 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f4a600a11a0 0x7f4a600a3650 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:05.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.357+0000 7f4a7a9ff700 1 --2- 172.21.15.101:0/3823452627 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a74100550 0x7f4a741979f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:05.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.357+0000 7f4a7a9ff700 1 --2- 172.21.15.101:0/3823452627 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4a74105790 0x7f4a7419efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:05.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.357+0000 7f4a7a9ff700 1 --2- 172.21.15.101:0/3823452627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a74102e70 0x7f4a74197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:05.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.357+0000 7f4a7a9ff700 1 -- 172.21.15.101:0/3823452627 >> 172.21.15.101:0/3823452627 conn(0x7f4a740f9fd0 msgr2=0x7f4a740febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:05.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.358+0000 7f4a7a9ff700 1 -- 172.21.15.101:0/3823452627 shutdown_connections 2023-07-18T03:05:05.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.358+0000 7f4a7a9ff700 1 -- 172.21.15.101:0/3823452627 wait complete. 2023-07-18T03:05:05.344 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 17 2023-07-18T03:05:05.361 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:05:05.557 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.573+0000 7fc3f8dff700 1 -- 172.21.15.101:0/2769080856 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3f4072f70 msgr2=0x7fc3f40733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:05.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.573+0000 7fc3f8dff700 1 --2- 172.21.15.101:0/2769080856 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3f4072f70 0x7fc3f40733e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc009990 tx=0x7fc3dc009ca0 comp rx=0 tx=0).stop 2023-07-18T03:05:05.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.573+0000 7fc3f8dff700 1 -- 172.21.15.101:0/2769080856 shutdown_connections 2023-07-18T03:05:05.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.573+0000 7fc3f8dff700 1 --2- 172.21.15.101:0/2769080856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3f4074550 0x7fc3f40729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:05.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.573+0000 7fc3f8dff700 1 --2- 172.21.15.101:0/2769080856 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3f410c0d0 0x7fc3f410e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:05.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.573+0000 7fc3f8dff700 1 --2- 172.21.15.101:0/2769080856 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3f4072f70 0x7fc3f40733e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:05.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.573+0000 7fc3f8dff700 1 -- 172.21.15.101:0/2769080856 >> 172.21.15.101:0/2769080856 conn(0x7fc3f40fb0e0 msgr2=0x7fc3f40fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:05.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.573+0000 7fc3f8dff700 1 -- 172.21.15.101:0/2769080856 shutdown_connections 2023-07-18T03:05:05.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.573+0000 7fc3f8dff700 1 -- 172.21.15.101:0/2769080856 wait complete. 2023-07-18T03:05:05.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.574+0000 7fc3f8dff700 1 Processor -- start 2023-07-18T03:05:05.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.574+0000 7fc3f8dff700 1 -- start start 2023-07-18T03:05:05.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.574+0000 7fc3f8dff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3f4072f70 0x7fc3f419be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:05.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.574+0000 7fc3f8dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3f4074550 0x7fc3f419c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:05.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.574+0000 7fc3f8dff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3f410c0d0 0x7fc3f41a3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:05.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.574+0000 7fc3f8dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f41a39f0 con 0x7fc3f4074550 2023-07-18T03:05:05.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.574+0000 7fc3f8dff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f41a3b30 con 0x7fc3f4072f70 2023-07-18T03:05:05.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.574+0000 7fc3f8dff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f41a3cc0 con 0x7fc3f410c0d0 2023-07-18T03:05:05.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.575+0000 7fc3ebfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3f4074550 0x7fc3f419c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:05.562 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.575+0000 7fc3ebfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3f4074550 0x7fc3f419c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:56636/0 (socket says 172.21.15.101:56636) 2023-07-18T03:05:05.562 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.575+0000 7fc3ebfff700 1 -- 172.21.15.101:0/3872995129 learned_addr learned my addr 172.21.15.101:0/3872995129 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:05.562 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.575+0000 7fc3f2d9d700 1 --2- 172.21.15.101:0/3872995129 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3f410c0d0 0x7fc3f41a3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:05.562 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.575+0000 7fc3f259c700 1 --2- 172.21.15.101:0/3872995129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3f4072f70 0x7fc3f419be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:05.562 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.575+0000 7fc3ebfff700 1 -- 172.21.15.101:0/3872995129 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3f410c0d0 msgr2=0x7fc3f41a3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:05.562 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.575+0000 7fc3ebfff700 1 --2- 172.21.15.101:0/3872995129 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3f410c0d0 0x7fc3f41a3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:05.562 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.575+0000 7fc3ebfff700 1 -- 172.21.15.101:0/3872995129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3f4072f70 msgr2=0x7fc3f419be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:05.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.575+0000 7fc3ebfff700 1 --2- 172.21.15.101:0/3872995129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3f4072f70 0x7fc3f419be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:05.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.575+0000 7fc3ebfff700 1 -- 172.21.15.101:0/3872995129 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3dc0095f0 con 0x7fc3f4074550 2023-07-18T03:05:05.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.575+0000 7fc3ebfff700 1 --2- 172.21.15.101:0/3872995129 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3f4074550 0x7fc3f419c3b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc00b680 tx=0x7fc3dc005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:05.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.575+0000 7fc3eb7fe700 1 -- 172.21.15.101:0/3872995129 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc01d070 con 0x7fc3f4074550 2023-07-18T03:05:05.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.575+0000 7fc3eb7fe700 1 -- 172.21.15.101:0/3872995129 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3dc021410 con 0x7fc3f4074550 2023-07-18T03:05:05.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.575+0000 7fc3f8dff700 1 -- 172.21.15.101:0/3872995129 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3f41a3f40 con 0x7fc3f4074550 2023-07-18T03:05:05.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.575+0000 7fc3eb7fe700 1 -- 172.21.15.101:0/3872995129 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc017650 con 0x7fc3f4074550 2023-07-18T03:05:05.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.576+0000 7fc3f8dff700 1 -- 172.21.15.101:0/3872995129 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3f41a42b0 con 0x7fc3f4074550 2023-07-18T03:05:05.564 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.576+0000 7fc3f8dff700 1 -- 172.21.15.101:0/3872995129 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3f4101c20 con 0x7fc3f4074550 2023-07-18T03:05:05.564 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.578+0000 7fc3eb7fe700 1 -- 172.21.15.101:0/3872995129 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fc3dc021580 con 0x7fc3f4074550 2023-07-18T03:05:05.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.580+0000 7fc3eb7fe700 1 --2- 172.21.15.101:0/3872995129 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc3e00a11a0 0x7fc3e00a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:05.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.580+0000 7fc3eb7fe700 1 -- 172.21.15.101:0/3872995129 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fc3dc0cdfe0 con 0x7fc3f4074550 2023-07-18T03:05:05.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.580+0000 7fc3f259c700 1 --2- 172.21.15.101:0/3872995129 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc3e00a11a0 0x7fc3e00a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:05.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.580+0000 7fc3f259c700 1 --2- 172.21.15.101:0/3872995129 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc3e00a11a0 0x7fc3e00a3650 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4074920 tx=0x7fc3e4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:05.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.584+0000 7fc3eb7fe700 1 -- 172.21.15.101:0/3872995129 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc3dc097570 con 0x7fc3f4074550 2023-07-18T03:05:05.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.783+0000 7fc3f8dff700 1 -- 172.21.15.101:0/3872995129 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3f4111950 con 0x7fc3f4074550 2023-07-18T03:05:05.769 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.785+0000 7fc3eb7fe700 1 -- 172.21.15.101:0/3872995129 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24042 (secure 0 0 0) 0x7fc3dc09ab90 con 0x7fc3f4074550 2023-07-18T03:05:05.770 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:05:05.770 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":87,"fsid":"b1782daa-2516-11ee-9b34-001a4aab830c","created":"2023-07-18T02:57:22.972483+0000","modified":"2023-07-18T03:05:04.353661+0000","last_up_change":"2023-07-18T03:04:11.429530+0000","last_in_change":"2023-07-18T03:03:54.158340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:00:43.353489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:04:57.771616+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-18T03:04:58.769590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-18T03:05:02.358830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc76b5d1-38a0-4f1a-9335-756bfc40a62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6803","nonce":3610719929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6805","nonce":3610719929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6809","nonce":3610719929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6807","nonce":3610719929}]},"public_addr":"172.21.15.101:6803/3610719929","cluster_addr":"172.21.15.101:6805/3610719929","heartbeat_back_addr":"172.21.15.101:6809/3610719929","heartbeat_front_addr":"172.21.15.101:6807/3610719929","state":["exists","up"]},{"osd":1,"uuid":"2a299573-0b34-4af9-aaae-b523e5058ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6801","nonce":1030229579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6803","nonce":1030229579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6807","nonce":1030229579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6805","nonce":1030229579}]},"public_addr":"172.21.15.155:6801/1030229579","cluster_addr":"172.21.15.155:6803/1030229579","heartbeat_back_addr":"172.21.15.155:6807/1030229579","heartbeat_front_addr":"172.21.15.155:6805/1030229579","state":["exists","up"]},{"osd":2,"uuid":"123b3749-c636-48fa-9e33-6229b5575dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6801","nonce":1349487507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6803","nonce":1349487507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6807","nonce":1349487507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6805","nonce":1349487507}]},"public_addr":"172.21.15.179:6801/1349487507","cluster_addr":"172.21.15.179:6803/1349487507","heartbeat_back_addr":"172.21.15.179:6807/1349487507","heartbeat_front_addr":"172.21.15.179:6805/1349487507","state":["exists","up"]},{"osd":3,"uuid":"4b0c409d-2954-4127-833a-b196bf226d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6811","nonce":4226577463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6813","nonce":4226577463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6817","nonce":4226577463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6815","nonce":4226577463}]},"public_addr":"172.21.15.101:6811/4226577463","cluster_addr":"172.21.15.101:6813/4226577463","heartbeat_back_addr":"172.21.15.101:6817/4226577463","heartbeat_front_addr":"172.21.15.101:6815/4226577463","state":["exists","up"]},{"osd":4,"uuid":"2137f2df-984e-45ce-baf6-b4ac12c7e3c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6809","nonce":1951135805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6811","nonce":1951135805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6815","nonce":1951135805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6813","nonce":1951135805}]},"public_addr":"172.21.15.155:6809/1951135805","cluster_addr":"172.21.15.155:6811/1951135805","heartbeat_back_addr":"172.21.15.155:6815/1951135805","heartbeat_front_addr":"172.21.15.155:6813/1951135805","state":["exists","up"]},{"osd":5,"uuid":"1020bfc5-8c2f-482e-9588-66c7f98e46e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6809","nonce":2746037021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6811","nonce":2746037021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6815","nonce":2746037021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6813","nonce":2746037021}]},"public_addr":"172.21.15.179:6809/2746037021","cluster_addr":"172.21.15.179:6811/2746037021","heartbeat_back_addr":"172.21.15.179:6815/2746037021","heartbeat_front_addr":"172.21.15.179:6813/2746037021","state":["exists","up"]},{"osd":6,"uuid":"2cd99260-5c36-4603-9338-cf697970ce58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6819","nonce":958044931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6821","nonce":958044931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6825","nonce":958044931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6823","nonce":958044931}]},"public_addr":"172.21.15.101:6819/958044931","cluster_addr":"172.21.15.101:6821/958044931","heartbeat_back_addr":"172.21.15.101:6825/958044931","heartbeat_front_addr":"172.21.15.101:6823/958044931","state":["exists","up"]},{"osd":7,"uuid":"ed23fd0a-678f-4e15-b42c-cfe873315714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6817","nonce":3256322291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6819","nonce":3256322291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6823","nonce":3256322291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6821","nonce":3256322291}]},"public_addr":"172.21.15.155:6817/3256322291","cluster_addr":"172.21.15.155:6819/3256322291","heartbeat_back_addr":"172.21.15.155:6823/3256322291","heartbeat_front_addr":"172.21.15.155:6821/3256322291","state":["exists","up"]},{"osd":8,"uuid":"eca3d8e6-7baf-47a5-b991-328001c7ebef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6817","nonce":3520612559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6819","nonce":3520612559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6823","nonce":3520612559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6821","nonce":3520612559}]},"public_addr":"172.21.15.179:6817/3520612559","cluster_addr":"172.21.15.179:6819/3520612559","heartbeat_back_addr":"172.21.15.179:6823/3520612559","heartbeat_front_addr":"172.21.15.179:6821/3520612559","state":["exists","up"]},{"osd":9,"uuid":"465a7ffb-67e4-49ef-812c-1b96feacfebf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6827","nonce":2166787667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6829","nonce":2166787667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6833","nonce":2166787667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6831","nonce":2166787667}]},"public_addr":"172.21.15.101:6827/2166787667","cluster_addr":"172.21.15.101:6829/2166787667","heartbeat_back_addr":"172.21.15.101:6833/2166787667","heartbeat_front_addr":"172.21.15.101:6831/2166787667","state":["exists","up"]},{"osd":10,"uuid":"1897a027-f25d-434f-8df5-5fc2bd866578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6825","nonce":3682446349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6827","nonce":3682446349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6831","nonce":3682446349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6829","nonce":3682446349}]},"public_addr":"172.21.15.155:6825/3682446349","cluster_addr":"172.21.15.155:6827/3682446349","heartbeat_back_addr":"172.21.15.155:6831/3682446349","heartbeat_front_addr":"172.21.15.155:6829/3682446349","state":["exists","up"]},{"osd":11,"uuid":"b62effaf-45e2-4585-b8d0-db448f3474ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6825","nonce":2602640858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6827","nonce":2602640858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6831","nonce":2602640858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6829","nonce":2602640858}]},"public_addr":"172.21.15.179:6825/2602640858","cluster_addr":"172.21.15.179:6827/2602640858","heartbeat_back_addr":"172.21.15.179:6831/2602640858","heartbeat_front_addr":"172.21.15.179:6829/2602640858","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T02:59:56.074519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:19.358142+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:41.128966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:08.227054+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:31.442592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:52.751685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:19.251054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:40.698241+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:01.339150+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:27.955910+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:49.136980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:04:10.084982+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.101:6800/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:6801/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/604228291":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/4055187849":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/2844574324":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/1603248215":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6801/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6800/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/3307578182":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/373189792":"2023-07-19T02:57:56.243200+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:05:05.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.787+0000 7fc3f8dff700 1 -- 172.21.15.101:0/3872995129 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc3e00a11a0 msgr2=0x7fc3e00a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:05.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.787+0000 7fc3f8dff700 1 --2- 172.21.15.101:0/3872995129 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc3e00a11a0 0x7fc3e00a3650 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4074920 tx=0x7fc3e4009260 comp rx=0 tx=0).stop 2023-07-18T03:05:05.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.787+0000 7fc3f8dff700 1 -- 172.21.15.101:0/3872995129 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3f4074550 msgr2=0x7fc3f419c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:05.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.787+0000 7fc3f8dff700 1 --2- 172.21.15.101:0/3872995129 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3f4074550 0x7fc3f419c3b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc00b680 tx=0x7fc3dc005740 comp rx=0 tx=0).stop 2023-07-18T03:05:05.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.788+0000 7fc3f8dff700 1 -- 172.21.15.101:0/3872995129 shutdown_connections 2023-07-18T03:05:05.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.788+0000 7fc3f8dff700 1 --2- 172.21.15.101:0/3872995129 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc3e00a11a0 0x7fc3e00a3650 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:05.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.788+0000 7fc3f8dff700 1 --2- 172.21.15.101:0/3872995129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3f4072f70 0x7fc3f419be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:05.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.788+0000 7fc3f8dff700 1 --2- 172.21.15.101:0/3872995129 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3f410c0d0 0x7fc3f41a3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:05.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.788+0000 7fc3f8dff700 1 --2- 172.21.15.101:0/3872995129 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3f4074550 0x7fc3f419c3b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:05.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.788+0000 7fc3f8dff700 1 -- 172.21.15.101:0/3872995129 >> 172.21.15.101:0/3872995129 conn(0x7fc3f40fb0e0 msgr2=0x7fc3f40fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:05.774 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.788+0000 7fc3f8dff700 1 -- 172.21.15.101:0/3872995129 shutdown_connections 2023-07-18T03:05:05.774 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:05.788+0000 7fc3f8dff700 1 -- 172.21.15.101:0/3872995129 wait complete. 2023-07-18T03:05:05.791 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:05:05.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.003+0000 7f18ecdff700 1 -- 172.21.15.101:0/1708520956 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18e80ffb00 msgr2=0x7f18e80fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:05.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.003+0000 7f18ecdff700 1 --2- 172.21.15.101:0/1708520956 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18e80ffb00 0x7f18e80fff10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f18d8009990 tx=0x7f18d8009ca0 comp rx=0 tx=0).stop 2023-07-18T03:05:05.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.003+0000 7f18ecdff700 1 -- 172.21.15.101:0/1708520956 shutdown_connections 2023-07-18T03:05:05.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.003+0000 7f18ecdff700 1 --2- 172.21.15.101:0/1708520956 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18e8101680 0x7f18e8107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:05.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.003+0000 7f18ecdff700 1 --2- 172.21.15.101:0/1708520956 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18e80ffb00 0x7f18e80fff10 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:05.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.003+0000 7f18ecdff700 1 --2- 172.21.15.101:0/1708520956 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18e8100cf0 0x7f18e8101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:05.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.003+0000 7f18ecdff700 1 -- 172.21.15.101:0/1708520956 >> 172.21.15.101:0/1708520956 conn(0x7f18e80fb0e0 msgr2=0x7f18e80fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:05.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.003+0000 7f18ecdff700 1 -- 172.21.15.101:0/1708520956 shutdown_connections 2023-07-18T03:05:05.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.003+0000 7f18ecdff700 1 -- 172.21.15.101:0/1708520956 wait complete. 2023-07-18T03:05:05.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.004+0000 7f18ecdff700 1 Processor -- start 2023-07-18T03:05:05.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.004+0000 7f18ecdff700 1 -- start start 2023-07-18T03:05:05.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.005+0000 7f18ecdff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18e80ffb00 0x7f18e80742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:05.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.005+0000 7f18ecdff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18e8100cf0 0x7f18e8072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:05.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.005+0000 7f18ecdff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18e8101680 0x7f18e8072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:05.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.005+0000 7f18ecdff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e80735e0 con 0x7f18e8101680 2023-07-18T03:05:05.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.005+0000 7f18ecdff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e8073750 con 0x7f18e80ffb00 2023-07-18T03:05:05.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.005+0000 7f18ecdff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e819f8b0 con 0x7f18e8100cf0 2023-07-18T03:05:05.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.005+0000 7f18e6d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18e8101680 0x7f18e8072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:05.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.005+0000 7f18e659c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18e80ffb00 0x7f18e80742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:05.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.005+0000 7f18e6d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18e8101680 0x7f18e8072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:52428/0 (socket says 172.21.15.101:52428) 2023-07-18T03:05:05.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.005+0000 7f18e659c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18e80ffb00 0x7f18e80742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47978/0 (socket says 172.21.15.101:47978) 2023-07-18T03:05:05.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.005+0000 7f18e6d9d700 1 -- 172.21.15.101:0/1936416630 learned_addr learned my addr 172.21.15.101:0/1936416630 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:05.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.005+0000 7f18e5d9b700 1 --2- 172.21.15.101:0/1936416630 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18e8100cf0 0x7f18e8072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:05.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.005+0000 7f18e659c700 1 -- 172.21.15.101:0/1936416630 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18e8100cf0 msgr2=0x7f18e8072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:05.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.005+0000 7f18e659c700 1 --2- 172.21.15.101:0/1936416630 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18e8100cf0 0x7f18e8072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:05.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.005+0000 7f18e659c700 1 -- 172.21.15.101:0/1936416630 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18e8101680 msgr2=0x7f18e8072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:05.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.005+0000 7f18e659c700 1 --2- 172.21.15.101:0/1936416630 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18e8101680 0x7f18e8072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:05.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.005+0000 7f18e659c700 1 -- 172.21.15.101:0/1936416630 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d80095f0 con 0x7f18e80ffb00 2023-07-18T03:05:05.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.005+0000 7f18e6d9d700 1 --2- 172.21.15.101:0/1936416630 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18e8101680 0x7f18e8072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:05:05.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.006+0000 7f18e659c700 1 --2- 172.21.15.101:0/1936416630 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18e80ffb00 0x7f18e80742c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f18d800b620 tx=0x7f18d8005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:05.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.006+0000 7f18d77fe700 1 -- 172.21.15.101:0/1936416630 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d801d070 con 0x7f18e80ffb00 2023-07-18T03:05:05.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.006+0000 7f18ecdff700 1 -- 172.21.15.101:0/1936416630 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18e819fb30 con 0x7f18e80ffb00 2023-07-18T03:05:05.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.006+0000 7f18d77fe700 1 -- 172.21.15.101:0/1936416630 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18d8021410 con 0x7f18e80ffb00 2023-07-18T03:05:05.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.006+0000 7f18d77fe700 1 -- 172.21.15.101:0/1936416630 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d8017650 con 0x7f18e80ffb00 2023-07-18T03:05:05.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.006+0000 7f18ecdff700 1 -- 172.21.15.101:0/1936416630 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18e819ffb0 con 0x7f18e80ffb00 2023-07-18T03:05:05.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.007+0000 7f18ecdff700 1 -- 172.21.15.101:0/1936416630 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18e810b320 con 0x7f18e80ffb00 2023-07-18T03:05:05.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.009+0000 7f18d77fe700 1 -- 172.21.15.101:0/1936416630 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f18d8021580 con 0x7f18e80ffb00 2023-07-18T03:05:05.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.010+0000 7f18d77fe700 1 --2- 172.21.15.101:0/1936416630 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f18d00a11a0 0x7f18d00a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:06.000 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.011+0000 7f18d77fe700 1 -- 172.21.15.101:0/1936416630 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f18d80ce120 con 0x7f18e80ffb00 2023-07-18T03:05:06.000 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.011+0000 7f18e5d9b700 1 --2- 172.21.15.101:0/1936416630 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f18d00a11a0 0x7f18d00a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:06.000 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.011+0000 7f18e5d9b700 1 --2- 172.21.15.101:0/1936416630 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f18d00a11a0 0x7f18d00a3650 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f18dc00bbd0 tx=0x7f18dc00b5b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:06.000 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.013+0000 7f18d77fe700 1 -- 172.21.15.101:0/1936416630 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f18d80976b0 con 0x7f18e80ffb00 2023-07-18T03:05:06.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:05 smithi179 ceph-mon[129229]: pgmap v277: 129 pgs: 5 creating+peering, 18 unknown, 106 active+clean; 579 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 8 op/s 2023-07-18T03:05:06.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:05 smithi179 ceph-mon[129229]: osdmap e87: 12 total, 12 up, 12 in 2023-07-18T03:05:06.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:05 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1383791099' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-07-18T03:05:06.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:05 smithi179 ceph-mon[129229]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-07-18T03:05:06.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:05 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/3823452627' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:05:06.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:05 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/3872995129' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:05:06.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.212+0000 7f18ecdff700 1 -- 172.21.15.101:0/1936416630 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18e81a0560 con 0x7f18e80ffb00 2023-07-18T03:05:06.198 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.214+0000 7f18d77fe700 1 -- 172.21.15.101:0/1936416630 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24042 (secure 0 0 0) 0x7f18d809acd0 con 0x7f18e80ffb00 2023-07-18T03:05:06.199 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:05:06.199 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":87,"fsid":"b1782daa-2516-11ee-9b34-001a4aab830c","created":"2023-07-18T02:57:22.972483+0000","modified":"2023-07-18T03:05:04.353661+0000","last_up_change":"2023-07-18T03:04:11.429530+0000","last_in_change":"2023-07-18T03:03:54.158340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:00:43.353489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:04:57.771616+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-18T03:04:58.769590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-18T03:05:02.358830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc76b5d1-38a0-4f1a-9335-756bfc40a62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6803","nonce":3610719929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6805","nonce":3610719929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6809","nonce":3610719929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6807","nonce":3610719929}]},"public_addr":"172.21.15.101:6803/3610719929","cluster_addr":"172.21.15.101:6805/3610719929","heartbeat_back_addr":"172.21.15.101:6809/3610719929","heartbeat_front_addr":"172.21.15.101:6807/3610719929","state":["exists","up"]},{"osd":1,"uuid":"2a299573-0b34-4af9-aaae-b523e5058ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6801","nonce":1030229579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6803","nonce":1030229579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6807","nonce":1030229579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6805","nonce":1030229579}]},"public_addr":"172.21.15.155:6801/1030229579","cluster_addr":"172.21.15.155:6803/1030229579","heartbeat_back_addr":"172.21.15.155:6807/1030229579","heartbeat_front_addr":"172.21.15.155:6805/1030229579","state":["exists","up"]},{"osd":2,"uuid":"123b3749-c636-48fa-9e33-6229b5575dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6801","nonce":1349487507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6803","nonce":1349487507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6807","nonce":1349487507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6805","nonce":1349487507}]},"public_addr":"172.21.15.179:6801/1349487507","cluster_addr":"172.21.15.179:6803/1349487507","heartbeat_back_addr":"172.21.15.179:6807/1349487507","heartbeat_front_addr":"172.21.15.179:6805/1349487507","state":["exists","up"]},{"osd":3,"uuid":"4b0c409d-2954-4127-833a-b196bf226d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6811","nonce":4226577463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6813","nonce":4226577463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6817","nonce":4226577463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6815","nonce":4226577463}]},"public_addr":"172.21.15.101:6811/4226577463","cluster_addr":"172.21.15.101:6813/4226577463","heartbeat_back_addr":"172.21.15.101:6817/4226577463","heartbeat_front_addr":"172.21.15.101:6815/4226577463","state":["exists","up"]},{"osd":4,"uuid":"2137f2df-984e-45ce-baf6-b4ac12c7e3c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6809","nonce":1951135805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6811","nonce":1951135805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6815","nonce":1951135805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6813","nonce":1951135805}]},"public_addr":"172.21.15.155:6809/1951135805","cluster_addr":"172.21.15.155:6811/1951135805","heartbeat_back_addr":"172.21.15.155:6815/1951135805","heartbeat_front_addr":"172.21.15.155:6813/1951135805","state":["exists","up"]},{"osd":5,"uuid":"1020bfc5-8c2f-482e-9588-66c7f98e46e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6809","nonce":2746037021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6811","nonce":2746037021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6815","nonce":2746037021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6813","nonce":2746037021}]},"public_addr":"172.21.15.179:6809/2746037021","cluster_addr":"172.21.15.179:6811/2746037021","heartbeat_back_addr":"172.21.15.179:6815/2746037021","heartbeat_front_addr":"172.21.15.179:6813/2746037021","state":["exists","up"]},{"osd":6,"uuid":"2cd99260-5c36-4603-9338-cf697970ce58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6819","nonce":958044931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6821","nonce":958044931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6825","nonce":958044931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6823","nonce":958044931}]},"public_addr":"172.21.15.101:6819/958044931","cluster_addr":"172.21.15.101:6821/958044931","heartbeat_back_addr":"172.21.15.101:6825/958044931","heartbeat_front_addr":"172.21.15.101:6823/958044931","state":["exists","up"]},{"osd":7,"uuid":"ed23fd0a-678f-4e15-b42c-cfe873315714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6817","nonce":3256322291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6819","nonce":3256322291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6823","nonce":3256322291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6821","nonce":3256322291}]},"public_addr":"172.21.15.155:6817/3256322291","cluster_addr":"172.21.15.155:6819/3256322291","heartbeat_back_addr":"172.21.15.155:6823/3256322291","heartbeat_front_addr":"172.21.15.155:6821/3256322291","state":["exists","up"]},{"osd":8,"uuid":"eca3d8e6-7baf-47a5-b991-328001c7ebef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6817","nonce":3520612559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6819","nonce":3520612559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6823","nonce":3520612559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6821","nonce":3520612559}]},"public_addr":"172.21.15.179:6817/3520612559","cluster_addr":"172.21.15.179:6819/3520612559","heartbeat_back_addr":"172.21.15.179:6823/3520612559","heartbeat_front_addr":"172.21.15.179:6821/3520612559","state":["exists","up"]},{"osd":9,"uuid":"465a7ffb-67e4-49ef-812c-1b96feacfebf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6827","nonce":2166787667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6829","nonce":2166787667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6833","nonce":2166787667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6831","nonce":2166787667}]},"public_addr":"172.21.15.101:6827/2166787667","cluster_addr":"172.21.15.101:6829/2166787667","heartbeat_back_addr":"172.21.15.101:6833/2166787667","heartbeat_front_addr":"172.21.15.101:6831/2166787667","state":["exists","up"]},{"osd":10,"uuid":"1897a027-f25d-434f-8df5-5fc2bd866578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6825","nonce":3682446349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6827","nonce":3682446349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6831","nonce":3682446349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6829","nonce":3682446349}]},"public_addr":"172.21.15.155:6825/3682446349","cluster_addr":"172.21.15.155:6827/3682446349","heartbeat_back_addr":"172.21.15.155:6831/3682446349","heartbeat_front_addr":"172.21.15.155:6829/3682446349","state":["exists","up"]},{"osd":11,"uuid":"b62effaf-45e2-4585-b8d0-db448f3474ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6825","nonce":2602640858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6827","nonce":2602640858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6831","nonce":2602640858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6829","nonce":2602640858}]},"public_addr":"172.21.15.179:6825/2602640858","cluster_addr":"172.21.15.179:6827/2602640858","heartbeat_back_addr":"172.21.15.179:6831/2602640858","heartbeat_front_addr":"172.21.15.179:6829/2602640858","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T02:59:56.074519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:19.358142+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:41.128966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:08.227054+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:31.442592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:52.751685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:19.251054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:40.698241+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:01.339150+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:27.955910+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:49.136980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:04:10.084982+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.101:6800/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:6801/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/604228291":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/4055187849":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/2844574324":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/1603248215":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6801/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6800/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/3307578182":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/373189792":"2023-07-19T02:57:56.243200+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:05:06.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.217+0000 7f18ecdff700 1 -- 172.21.15.101:0/1936416630 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f18d00a11a0 msgr2=0x7f18d00a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:06.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.217+0000 7f18ecdff700 1 --2- 172.21.15.101:0/1936416630 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f18d00a11a0 0x7f18d00a3650 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f18dc00bbd0 tx=0x7f18dc00b5b0 comp rx=0 tx=0).stop 2023-07-18T03:05:06.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.217+0000 7f18ecdff700 1 -- 172.21.15.101:0/1936416630 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18e80ffb00 msgr2=0x7f18e80742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:06.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.217+0000 7f18ecdff700 1 --2- 172.21.15.101:0/1936416630 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18e80ffb00 0x7f18e80742c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f18d800b620 tx=0x7f18d8005740 comp rx=0 tx=0).stop 2023-07-18T03:05:06.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.217+0000 7f18ecdff700 1 -- 172.21.15.101:0/1936416630 shutdown_connections 2023-07-18T03:05:06.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.217+0000 7f18ecdff700 1 --2- 172.21.15.101:0/1936416630 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f18d00a11a0 0x7f18d00a3650 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:06.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.217+0000 7f18ecdff700 1 --2- 172.21.15.101:0/1936416630 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18e80ffb00 0x7f18e80742c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:06.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.217+0000 7f18ecdff700 1 --2- 172.21.15.101:0/1936416630 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18e8101680 0x7f18e8072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:06.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.217+0000 7f18ecdff700 1 --2- 172.21.15.101:0/1936416630 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18e8100cf0 0x7f18e8072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:06.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.217+0000 7f18ecdff700 1 -- 172.21.15.101:0/1936416630 >> 172.21.15.101:0/1936416630 conn(0x7f18e80fb0e0 msgr2=0x7f18e80fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:06.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.217+0000 7f18ecdff700 1 -- 172.21.15.101:0/1936416630 shutdown_connections 2023-07-18T03:05:06.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.217+0000 7f18ecdff700 1 -- 172.21.15.101:0/1936416630 wait complete. 2023-07-18T03:05:06.217 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:05 smithi101 ceph-mon[126379]: pgmap v277: 129 pgs: 5 creating+peering, 18 unknown, 106 active+clean; 579 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 8 op/s 2023-07-18T03:05:06.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:05 smithi101 ceph-mon[126379]: osdmap e87: 12 total, 12 up, 12 in 2023-07-18T03:05:06.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:05 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1383791099' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-07-18T03:05:06.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:05 smithi101 ceph-mon[126379]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-07-18T03:05:06.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:05 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3823452627' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:05:06.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:05 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3872995129' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:05:06.237 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:05:06.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:05 smithi155 ceph-mon[128885]: pgmap v277: 129 pgs: 5 creating+peering, 18 unknown, 106 active+clean; 579 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 8 op/s 2023-07-18T03:05:06.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:05 smithi155 ceph-mon[128885]: osdmap e87: 12 total, 12 up, 12 in 2023-07-18T03:05:06.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:05 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1383791099' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-07-18T03:05:06.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:05 smithi155 ceph-mon[128885]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-07-18T03:05:06.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:05 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/3823452627' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:05:06.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:05 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/3872995129' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:05:06.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.449+0000 7f042c1ff700 1 -- 172.21.15.101:0/1289626534 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0424074550 msgr2=0x7f04240729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:06.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.449+0000 7f042c1ff700 1 --2- 172.21.15.101:0/1289626534 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0424074550 0x7f04240729a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0420009990 tx=0x7f0420009ca0 comp rx=0 tx=0).stop 2023-07-18T03:05:06.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.449+0000 7f042c1ff700 1 -- 172.21.15.101:0/1289626534 shutdown_connections 2023-07-18T03:05:06.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.449+0000 7f042c1ff700 1 --2- 172.21.15.101:0/1289626534 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f042410c0d0 0x7f042410e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:06.435 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.449+0000 7f042c1ff700 1 --2- 172.21.15.101:0/1289626534 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0424074550 0x7f04240729a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:06.435 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.449+0000 7f042c1ff700 1 --2- 172.21.15.101:0/1289626534 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0424072f70 0x7f04240733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:06.435 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.449+0000 7f042c1ff700 1 -- 172.21.15.101:0/1289626534 >> 172.21.15.101:0/1289626534 conn(0x7f04240fb0e0 msgr2=0x7f04240fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:06.435 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.449+0000 7f042c1ff700 1 -- 172.21.15.101:0/1289626534 shutdown_connections 2023-07-18T03:05:06.435 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.449+0000 7f042c1ff700 1 -- 172.21.15.101:0/1289626534 wait complete. 2023-07-18T03:05:06.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.450+0000 7f042c1ff700 1 Processor -- start 2023-07-18T03:05:06.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.450+0000 7f042c1ff700 1 -- start start 2023-07-18T03:05:06.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.451+0000 7f042c1ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0424072f70 0x7f042419be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:06.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.451+0000 7f042c1ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0424074550 0x7f042419c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:06.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.451+0000 7f042c1ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f042410c0d0 0x7f04241a3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:06.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.451+0000 7f042c1ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04241a3a00 con 0x7f0424072f70 2023-07-18T03:05:06.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.451+0000 7f042c1ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04241a3b40 con 0x7f042410c0d0 2023-07-18T03:05:06.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.451+0000 7f042c1ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04241a3cd0 con 0x7f0424074550 2023-07-18T03:05:06.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.451+0000 7f0429f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0424072f70 0x7f042419be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:06.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.451+0000 7f0429f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0424072f70 0x7f042419be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:52442/0 (socket says 172.21.15.101:52442) 2023-07-18T03:05:06.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.451+0000 7f0429f9b700 1 -- 172.21.15.101:0/3151470931 learned_addr learned my addr 172.21.15.101:0/3151470931 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:06.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.451+0000 7f042979a700 1 --2- 172.21.15.101:0/3151470931 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0424074550 0x7f042419c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:06.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.451+0000 7f042a79c700 1 --2- 172.21.15.101:0/3151470931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f042410c0d0 0x7f04241a3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:06.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.451+0000 7f0429f9b700 1 -- 172.21.15.101:0/3151470931 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0424074550 msgr2=0x7f042419c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:06.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.451+0000 7f0429f9b700 1 --2- 172.21.15.101:0/3151470931 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0424074550 0x7f042419c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:06.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.451+0000 7f0429f9b700 1 -- 172.21.15.101:0/3151470931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f042410c0d0 msgr2=0x7f04241a3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:06.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.451+0000 7f0429f9b700 1 --2- 172.21.15.101:0/3151470931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f042410c0d0 0x7f04241a3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:06.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.451+0000 7f0429f9b700 1 -- 172.21.15.101:0/3151470931 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04200095f0 con 0x7f0424072f70 2023-07-18T03:05:06.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.451+0000 7f0429f9b700 1 --2- 172.21.15.101:0/3151470931 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0424072f70 0x7f042419be80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f04200095c0 tx=0x7f042000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:06.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.452+0000 7f041affd700 1 -- 172.21.15.101:0/3151470931 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042001d070 con 0x7f0424072f70 2023-07-18T03:05:06.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.452+0000 7f041affd700 1 -- 172.21.15.101:0/3151470931 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0420004cf0 con 0x7f0424072f70 2023-07-18T03:05:06.440 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.452+0000 7f042c1ff700 1 -- 172.21.15.101:0/3151470931 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04241a3f50 con 0x7f0424072f70 2023-07-18T03:05:06.440 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.452+0000 7f041affd700 1 -- 172.21.15.101:0/3151470931 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0420005520 con 0x7f0424072f70 2023-07-18T03:05:06.440 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.452+0000 7f042c1ff700 1 -- 172.21.15.101:0/3151470931 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04241a42c0 con 0x7f0424072f70 2023-07-18T03:05:06.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.453+0000 7f042c1ff700 1 -- 172.21.15.101:0/3151470931 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0424101c20 con 0x7f0424072f70 2023-07-18T03:05:06.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.455+0000 7f041affd700 1 -- 172.21.15.101:0/3151470931 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f0420004830 con 0x7f0424072f70 2023-07-18T03:05:06.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.456+0000 7f041affd700 1 --2- 172.21.15.101:0/3151470931 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f04100a1220 0x7f04100a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:06.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.456+0000 7f041affd700 1 -- 172.21.15.101:0/3151470931 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f04200cdd60 con 0x7f0424072f70 2023-07-18T03:05:06.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.456+0000 7f042979a700 1 --2- 172.21.15.101:0/3151470931 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f04100a1220 0x7f04100a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:06.442 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.457+0000 7f042979a700 1 --2- 172.21.15.101:0/3151470931 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f04100a1220 0x7f04100a36d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0414005950 tx=0x7f04140058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:06.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.459+0000 7f041affd700 1 -- 172.21.15.101:0/3151470931 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f042005b790 con 0x7f0424072f70 2023-07-18T03:05:06.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.700+0000 7f042c1ff700 1 -- 172.21.15.101:0/3151470931 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f042402cc70 con 0x7f0424072f70 2023-07-18T03:05:06.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.701+0000 7f041affd700 1 -- 172.21.15.101:0/3151470931 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+10797 (secure 0 0 0) 0x7f04200972f0 con 0x7f0424072f70 2023-07-18T03:05:06.686 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:05:06.686 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/4192846769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":4192846769},{"type":"v1","addr":"172.21.15.101:6839","nonce":4192846769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6841/3824340175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6840","nonce":3824340175},{"type":"v1","addr":"172.21.15.101:6841","nonce":3824340175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24515,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24529,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24541,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6839/2055462483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2055462483},{"type":"v1","addr":"172.21.15.155:6839","nonce":2055462483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-18T03:04:59.361185+0000","modified":"2023-07-18T03:05:04.873332+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24562":{"gid":24562,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-07-18T03:05:06.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.703+0000 7f042c1ff700 1 -- 172.21.15.101:0/3151470931 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f04100a1220 msgr2=0x7f04100a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:06.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.703+0000 7f042c1ff700 1 --2- 172.21.15.101:0/3151470931 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f04100a1220 0x7f04100a36d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0414005950 tx=0x7f04140058e0 comp rx=0 tx=0).stop 2023-07-18T03:05:06.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.703+0000 7f042c1ff700 1 -- 172.21.15.101:0/3151470931 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0424072f70 msgr2=0x7f042419be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:06.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.703+0000 7f042c1ff700 1 --2- 172.21.15.101:0/3151470931 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0424072f70 0x7f042419be80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f04200095c0 tx=0x7f042000f690 comp rx=0 tx=0).stop 2023-07-18T03:05:06.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.703+0000 7f042c1ff700 1 -- 172.21.15.101:0/3151470931 shutdown_connections 2023-07-18T03:05:06.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.703+0000 7f042c1ff700 1 --2- 172.21.15.101:0/3151470931 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f04100a1220 0x7f04100a36d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:06.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.703+0000 7f042c1ff700 1 --2- 172.21.15.101:0/3151470931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f042410c0d0 0x7f04241a3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:06.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.703+0000 7f042c1ff700 1 --2- 172.21.15.101:0/3151470931 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0424072f70 0x7f042419be80 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:06.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.703+0000 7f042c1ff700 1 --2- 172.21.15.101:0/3151470931 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0424074550 0x7f042419c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:06.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.703+0000 7f042c1ff700 1 -- 172.21.15.101:0/3151470931 >> 172.21.15.101:0/3151470931 conn(0x7f04240fb0e0 msgr2=0x7f04240fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:06.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.704+0000 7f042c1ff700 1 -- 172.21.15.101:0/3151470931 shutdown_connections 2023-07-18T03:05:06.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.704+0000 7f042c1ff700 1 -- 172.21.15.101:0/3151470931 wait complete. 2023-07-18T03:05:06.690 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 17 2023-07-18T03:05:06.706 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:05:06.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.922+0000 7fd24fbff700 1 -- 172.21.15.101:0/2079891645 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd248101680 msgr2=0x7fd248107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:06.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.922+0000 7fd24fbff700 1 --2- 172.21.15.101:0/2079891645 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd248101680 0x7fd248107f40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd240009f90 tx=0x7fd24000b330 comp rx=0 tx=0).stop 2023-07-18T03:05:06.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.922+0000 7fd24fbff700 1 -- 172.21.15.101:0/2079891645 shutdown_connections 2023-07-18T03:05:06.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.922+0000 7fd24fbff700 1 --2- 172.21.15.101:0/2079891645 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd248101680 0x7fd248107f40 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:06.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.922+0000 7fd24fbff700 1 --2- 172.21.15.101:0/2079891645 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd2480ffb00 0x7fd2480fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:06.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.922+0000 7fd24fbff700 1 --2- 172.21.15.101:0/2079891645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd248100cf0 0x7fd248101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:06.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.922+0000 7fd24fbff700 1 -- 172.21.15.101:0/2079891645 >> 172.21.15.101:0/2079891645 conn(0x7fd2480fb0e0 msgr2=0x7fd2480fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:06.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.922+0000 7fd24fbff700 1 -- 172.21.15.101:0/2079891645 shutdown_connections 2023-07-18T03:05:06.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.922+0000 7fd24fbff700 1 -- 172.21.15.101:0/2079891645 wait complete. 2023-07-18T03:05:06.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.923+0000 7fd24fbff700 1 Processor -- start 2023-07-18T03:05:06.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.923+0000 7fd24fbff700 1 -- start start 2023-07-18T03:05:06.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.923+0000 7fd24fbff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2480ffb00 0x7fd2481978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:06.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.923+0000 7fd24fbff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd248100cf0 0x7fd248197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:06.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.923+0000 7fd24fbff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd248101680 0x7fd24819ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:06.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.923+0000 7fd24fbff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24819f450 con 0x7fd248100cf0 2023-07-18T03:05:06.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.923+0000 7fd24fbff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24819f590 con 0x7fd2480ffb00 2023-07-18T03:05:06.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.923+0000 7fd24fbff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24819f720 con 0x7fd248101680 2023-07-18T03:05:06.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.924+0000 7fd24d19a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd248100cf0 0x7fd248197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:06.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.924+0000 7fd24d19a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd248100cf0 0x7fd248197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:52464/0 (socket says 172.21.15.101:52464) 2023-07-18T03:05:06.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.924+0000 7fd24d19a700 1 -- 172.21.15.101:0/465795300 learned_addr learned my addr 172.21.15.101:0/465795300 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:06.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.924+0000 7fd24e19c700 1 --2- 172.21.15.101:0/465795300 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd248101680 0x7fd24819ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:06.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.924+0000 7fd24d99b700 1 --2- 172.21.15.101:0/465795300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2480ffb00 0x7fd2481978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:06.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.924+0000 7fd24d19a700 1 -- 172.21.15.101:0/465795300 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd248101680 msgr2=0x7fd24819ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:06.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.924+0000 7fd24d19a700 1 --2- 172.21.15.101:0/465795300 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd248101680 0x7fd24819ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:06.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.924+0000 7fd24d19a700 1 -- 172.21.15.101:0/465795300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2480ffb00 msgr2=0x7fd2481978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:06.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.924+0000 7fd24d19a700 1 --2- 172.21.15.101:0/465795300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2480ffb00 0x7fd2481978d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:06.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.924+0000 7fd24d19a700 1 -- 172.21.15.101:0/465795300 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2440095f0 con 0x7fd248100cf0 2023-07-18T03:05:06.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.924+0000 7fd24d19a700 1 --2- 172.21.15.101:0/465795300 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd248100cf0 0x7fd248197e10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd23800e9a0 tx=0x7fd23800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:06.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.924+0000 7fd23effd700 1 -- 172.21.15.101:0/465795300 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd238009790 con 0x7fd248100cf0 2023-07-18T03:05:06.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.924+0000 7fd24fbff700 1 -- 172.21.15.101:0/465795300 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd240009c40 con 0x7fd248100cf0 2023-07-18T03:05:06.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.924+0000 7fd23effd700 1 -- 172.21.15.101:0/465795300 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd23800cbd0 con 0x7fd248100cf0 2023-07-18T03:05:06.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.924+0000 7fd23effd700 1 -- 172.21.15.101:0/465795300 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd238018680 con 0x7fd248100cf0 2023-07-18T03:05:06.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.924+0000 7fd24fbff700 1 -- 172.21.15.101:0/465795300 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd24819fdd0 con 0x7fd248100cf0 2023-07-18T03:05:06.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.925+0000 7fd24fbff700 1 -- 172.21.15.101:0/465795300 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd24810b270 con 0x7fd248100cf0 2023-07-18T03:05:06.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.927+0000 7fd23effd700 1 -- 172.21.15.101:0/465795300 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fd23800cd40 con 0x7fd248100cf0 2023-07-18T03:05:06.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.929+0000 7fd23effd700 1 --2- 172.21.15.101:0/465795300 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd2340a11a0 0x7fd2340a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:06.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.929+0000 7fd23effd700 1 -- 172.21.15.101:0/465795300 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd238014070 con 0x7fd248100cf0 2023-07-18T03:05:06.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.929+0000 7fd24d99b700 1 --2- 172.21.15.101:0/465795300 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd2340a11a0 0x7fd2340a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:06.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.929+0000 7fd24d99b700 1 --2- 172.21.15.101:0/465795300 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd2340a11a0 0x7fd2340a3650 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd248100b50 tx=0x7fd244009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:06.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:06.930+0000 7fd23effd700 1 -- 172.21.15.101:0/465795300 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd2380965c0 con 0x7fd248100cf0 2023-07-18T03:05:06.978 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:06 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1936416630' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:05:06.978 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:06 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3151470931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:05:07.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.132+0000 7fd24fbff700 1 -- 172.21.15.101:0/465795300 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd24810b490 con 0x7fd248100cf0 2023-07-18T03:05:07.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.134+0000 7fd23effd700 1 -- 172.21.15.101:0/465795300 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24042 (secure 0 0 0) 0x7fd238099be0 con 0x7fd248100cf0 2023-07-18T03:05:07.119 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:05:07.120 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":87,"fsid":"b1782daa-2516-11ee-9b34-001a4aab830c","created":"2023-07-18T02:57:22.972483+0000","modified":"2023-07-18T03:05:04.353661+0000","last_up_change":"2023-07-18T03:04:11.429530+0000","last_in_change":"2023-07-18T03:03:54.158340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:00:43.353489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:04:57.771616+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-18T03:04:58.769590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-18T03:05:02.358830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc76b5d1-38a0-4f1a-9335-756bfc40a62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6803","nonce":3610719929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6805","nonce":3610719929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6809","nonce":3610719929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6807","nonce":3610719929}]},"public_addr":"172.21.15.101:6803/3610719929","cluster_addr":"172.21.15.101:6805/3610719929","heartbeat_back_addr":"172.21.15.101:6809/3610719929","heartbeat_front_addr":"172.21.15.101:6807/3610719929","state":["exists","up"]},{"osd":1,"uuid":"2a299573-0b34-4af9-aaae-b523e5058ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6801","nonce":1030229579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6803","nonce":1030229579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6807","nonce":1030229579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6805","nonce":1030229579}]},"public_addr":"172.21.15.155:6801/1030229579","cluster_addr":"172.21.15.155:6803/1030229579","heartbeat_back_addr":"172.21.15.155:6807/1030229579","heartbeat_front_addr":"172.21.15.155:6805/1030229579","state":["exists","up"]},{"osd":2,"uuid":"123b3749-c636-48fa-9e33-6229b5575dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6801","nonce":1349487507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6803","nonce":1349487507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6807","nonce":1349487507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6805","nonce":1349487507}]},"public_addr":"172.21.15.179:6801/1349487507","cluster_addr":"172.21.15.179:6803/1349487507","heartbeat_back_addr":"172.21.15.179:6807/1349487507","heartbeat_front_addr":"172.21.15.179:6805/1349487507","state":["exists","up"]},{"osd":3,"uuid":"4b0c409d-2954-4127-833a-b196bf226d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6811","nonce":4226577463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6813","nonce":4226577463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6817","nonce":4226577463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6815","nonce":4226577463}]},"public_addr":"172.21.15.101:6811/4226577463","cluster_addr":"172.21.15.101:6813/4226577463","heartbeat_back_addr":"172.21.15.101:6817/4226577463","heartbeat_front_addr":"172.21.15.101:6815/4226577463","state":["exists","up"]},{"osd":4,"uuid":"2137f2df-984e-45ce-baf6-b4ac12c7e3c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6809","nonce":1951135805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6811","nonce":1951135805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6815","nonce":1951135805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6813","nonce":1951135805}]},"public_addr":"172.21.15.155:6809/1951135805","cluster_addr":"172.21.15.155:6811/1951135805","heartbeat_back_addr":"172.21.15.155:6815/1951135805","heartbeat_front_addr":"172.21.15.155:6813/1951135805","state":["exists","up"]},{"osd":5,"uuid":"1020bfc5-8c2f-482e-9588-66c7f98e46e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6809","nonce":2746037021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6811","nonce":2746037021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6815","nonce":2746037021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6813","nonce":2746037021}]},"public_addr":"172.21.15.179:6809/2746037021","cluster_addr":"172.21.15.179:6811/2746037021","heartbeat_back_addr":"172.21.15.179:6815/2746037021","heartbeat_front_addr":"172.21.15.179:6813/2746037021","state":["exists","up"]},{"osd":6,"uuid":"2cd99260-5c36-4603-9338-cf697970ce58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6819","nonce":958044931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6821","nonce":958044931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6825","nonce":958044931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6823","nonce":958044931}]},"public_addr":"172.21.15.101:6819/958044931","cluster_addr":"172.21.15.101:6821/958044931","heartbeat_back_addr":"172.21.15.101:6825/958044931","heartbeat_front_addr":"172.21.15.101:6823/958044931","state":["exists","up"]},{"osd":7,"uuid":"ed23fd0a-678f-4e15-b42c-cfe873315714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6817","nonce":3256322291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6819","nonce":3256322291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6823","nonce":3256322291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6821","nonce":3256322291}]},"public_addr":"172.21.15.155:6817/3256322291","cluster_addr":"172.21.15.155:6819/3256322291","heartbeat_back_addr":"172.21.15.155:6823/3256322291","heartbeat_front_addr":"172.21.15.155:6821/3256322291","state":["exists","up"]},{"osd":8,"uuid":"eca3d8e6-7baf-47a5-b991-328001c7ebef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6817","nonce":3520612559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6819","nonce":3520612559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6823","nonce":3520612559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6821","nonce":3520612559}]},"public_addr":"172.21.15.179:6817/3520612559","cluster_addr":"172.21.15.179:6819/3520612559","heartbeat_back_addr":"172.21.15.179:6823/3520612559","heartbeat_front_addr":"172.21.15.179:6821/3520612559","state":["exists","up"]},{"osd":9,"uuid":"465a7ffb-67e4-49ef-812c-1b96feacfebf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6827","nonce":2166787667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6829","nonce":2166787667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6833","nonce":2166787667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6831","nonce":2166787667}]},"public_addr":"172.21.15.101:6827/2166787667","cluster_addr":"172.21.15.101:6829/2166787667","heartbeat_back_addr":"172.21.15.101:6833/2166787667","heartbeat_front_addr":"172.21.15.101:6831/2166787667","state":["exists","up"]},{"osd":10,"uuid":"1897a027-f25d-434f-8df5-5fc2bd866578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6825","nonce":3682446349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6827","nonce":3682446349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6831","nonce":3682446349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6829","nonce":3682446349}]},"public_addr":"172.21.15.155:6825/3682446349","cluster_addr":"172.21.15.155:6827/3682446349","heartbeat_back_addr":"172.21.15.155:6831/3682446349","heartbeat_front_addr":"172.21.15.155:6829/3682446349","state":["exists","up"]},{"osd":11,"uuid":"b62effaf-45e2-4585-b8d0-db448f3474ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6825","nonce":2602640858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6827","nonce":2602640858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6831","nonce":2602640858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6829","nonce":2602640858}]},"public_addr":"172.21.15.179:6825/2602640858","cluster_addr":"172.21.15.179:6827/2602640858","heartbeat_back_addr":"172.21.15.179:6831/2602640858","heartbeat_front_addr":"172.21.15.179:6829/2602640858","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T02:59:56.074519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:19.358142+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:41.128966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:08.227054+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:31.442592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:52.751685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:19.251054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:40.698241+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:01.339150+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:27.955910+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:49.136980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:04:10.084982+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.101:6800/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:6801/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/604228291":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/4055187849":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/2844574324":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/1603248215":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6801/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6800/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/3307578182":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/373189792":"2023-07-19T02:57:56.243200+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:05:07.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.136+0000 7fd24fbff700 1 -- 172.21.15.101:0/465795300 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd2340a11a0 msgr2=0x7fd2340a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:07.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.136+0000 7fd24fbff700 1 --2- 172.21.15.101:0/465795300 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd2340a11a0 0x7fd2340a3650 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd248100b50 tx=0x7fd244009310 comp rx=0 tx=0).stop 2023-07-18T03:05:07.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.136+0000 7fd24fbff700 1 -- 172.21.15.101:0/465795300 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd248100cf0 msgr2=0x7fd248197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:07.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.136+0000 7fd24fbff700 1 --2- 172.21.15.101:0/465795300 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd248100cf0 0x7fd248197e10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd23800e9a0 tx=0x7fd23800ecb0 comp rx=0 tx=0).stop 2023-07-18T03:05:07.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.136+0000 7fd24fbff700 1 -- 172.21.15.101:0/465795300 shutdown_connections 2023-07-18T03:05:07.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.136+0000 7fd24fbff700 1 --2- 172.21.15.101:0/465795300 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd2340a11a0 0x7fd2340a3650 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:07.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.137+0000 7fd24fbff700 1 --2- 172.21.15.101:0/465795300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2480ffb00 0x7fd2481978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:07.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.137+0000 7fd24fbff700 1 --2- 172.21.15.101:0/465795300 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd248101680 0x7fd24819ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:07.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.137+0000 7fd24fbff700 1 --2- 172.21.15.101:0/465795300 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd248100cf0 0x7fd248197e10 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:07.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.137+0000 7fd24fbff700 1 -- 172.21.15.101:0/465795300 >> 172.21.15.101:0/465795300 conn(0x7fd2480fb0e0 msgr2=0x7fd248106500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:07.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.137+0000 7fd24fbff700 1 -- 172.21.15.101:0/465795300 shutdown_connections 2023-07-18T03:05:07.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.137+0000 7fd24fbff700 1 -- 172.21.15.101:0/465795300 wait complete. 2023-07-18T03:05:07.140 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-07-18T03:04:59.361185+0000', 'modified': '2023-07-18T03:05:04.873332+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24562}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24562': {'gid': 24562, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.179:6839/2061828474', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6838', 'nonce': 2061828474}, {'type': 'v1', 'addr': '172.21.15.179:6839', 'nonce': 2061828474}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-07-18T03:05:07.140 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-07-18T03:05:07.141 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-07-18T03:05:07.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:06 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1936416630' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:05:07.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:06 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/3151470931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:05:07.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:06 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1936416630' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:05:07.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:06 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/3151470931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:05:07.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.356+0000 7efeba7ff700 1 -- 172.21.15.101:0/500606910 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efeb4100cd0 msgr2=0x7efeb4101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:07.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.356+0000 7efeba7ff700 1 --2- 172.21.15.101:0/500606910 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efeb4100cd0 0x7efeb4101120 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efea8009990 tx=0x7efea8009ca0 comp rx=0 tx=0).stop 2023-07-18T03:05:07.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.356+0000 7efeba7ff700 1 -- 172.21.15.101:0/500606910 shutdown_connections 2023-07-18T03:05:07.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.356+0000 7efeba7ff700 1 --2- 172.21.15.101:0/500606910 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efeb40ffae0 0x7efeb40ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:07.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.356+0000 7efeba7ff700 1 --2- 172.21.15.101:0/500606910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efeb4101660 0x7efeb4107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:07.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.356+0000 7efeba7ff700 1 --2- 172.21.15.101:0/500606910 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efeb4100cd0 0x7efeb4101120 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:07.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.356+0000 7efeba7ff700 1 -- 172.21.15.101:0/500606910 >> 172.21.15.101:0/500606910 conn(0x7efeb40fb0e0 msgr2=0x7efeb40fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:07.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.356+0000 7efeba7ff700 1 -- 172.21.15.101:0/500606910 shutdown_connections 2023-07-18T03:05:07.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.356+0000 7efeba7ff700 1 -- 172.21.15.101:0/500606910 wait complete. 2023-07-18T03:05:07.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.357+0000 7efeba7ff700 1 Processor -- start 2023-07-18T03:05:07.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.358+0000 7efeba7ff700 1 -- start start 2023-07-18T03:05:07.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.358+0000 7efeba7ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efeb40ffae0 0x7efeb4197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:07.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.358+0000 7efeba7ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efeb4100cd0 0x7efeb4197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:07.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.358+0000 7efeba7ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efeb4101660 0x7efeb419f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:07.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.358+0000 7efeba7ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb419f610 con 0x7efeb4100cd0 2023-07-18T03:05:07.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.358+0000 7efeba7ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb419f750 con 0x7efeb40ffae0 2023-07-18T03:05:07.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.358+0000 7efeba7ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb419f8e0 con 0x7efeb4101660 2023-07-18T03:05:07.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.358+0000 7efeb37fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efeb4100cd0 0x7efeb4197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:07.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.358+0000 7efeb37fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efeb4100cd0 0x7efeb4197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:52484/0 (socket says 172.21.15.101:52484) 2023-07-18T03:05:07.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.358+0000 7efeb37fe700 1 -- 172.21.15.101:0/4225296016 learned_addr learned my addr 172.21.15.101:0/4225296016 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:07.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.358+0000 7efeb8d9c700 1 --2- 172.21.15.101:0/4225296016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efeb4101660 0x7efeb419f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:07.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.358+0000 7efeb3fff700 1 --2- 172.21.15.101:0/4225296016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efeb40ffae0 0x7efeb4197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:07.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.359+0000 7efeb37fe700 1 -- 172.21.15.101:0/4225296016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efeb4101660 msgr2=0x7efeb419f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:07.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.359+0000 7efeb37fe700 1 --2- 172.21.15.101:0/4225296016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efeb4101660 0x7efeb419f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:07.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.359+0000 7efeb37fe700 1 -- 172.21.15.101:0/4225296016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efeb40ffae0 msgr2=0x7efeb4197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:07.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.359+0000 7efeb37fe700 1 --2- 172.21.15.101:0/4225296016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efeb40ffae0 0x7efeb4197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:07.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.359+0000 7efeb37fe700 1 -- 172.21.15.101:0/4225296016 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea80095f0 con 0x7efeb4100cd0 2023-07-18T03:05:07.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.359+0000 7efeb8d9c700 1 --2- 172.21.15.101:0/4225296016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efeb4101660 0x7efeb419f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:05:07.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.359+0000 7efeb3fff700 1 --2- 172.21.15.101:0/4225296016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efeb40ffae0 0x7efeb4197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:05:07.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.359+0000 7efeb37fe700 1 --2- 172.21.15.101:0/4225296016 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efeb4100cd0 0x7efeb4197fd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7efea800b620 tx=0x7efea8005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:07.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.359+0000 7efeb17fa700 1 -- 172.21.15.101:0/4225296016 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea801d070 con 0x7efeb4100cd0 2023-07-18T03:05:07.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.359+0000 7efeb17fa700 1 -- 172.21.15.101:0/4225296016 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efea8022470 con 0x7efeb4100cd0 2023-07-18T03:05:07.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.359+0000 7efeba7ff700 1 -- 172.21.15.101:0/4225296016 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeb419fb60 con 0x7efeb4100cd0 2023-07-18T03:05:07.348 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.359+0000 7efeb17fa700 1 -- 172.21.15.101:0/4225296016 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea8017650 con 0x7efeb4100cd0 2023-07-18T03:05:07.348 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.359+0000 7efeba7ff700 1 -- 172.21.15.101:0/4225296016 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb419ffe0 con 0x7efeb4100cd0 2023-07-18T03:05:07.349 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.363+0000 7efeb17fa700 1 -- 172.21.15.101:0/4225296016 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7efea80225e0 con 0x7efeb4100cd0 2023-07-18T03:05:07.349 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.364+0000 7efece0a7b80 1 Processor -- start 2023-07-18T03:05:07.349 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.364+0000 7efeb17fa700 1 --2- 172.21.15.101:0/4225296016 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7efe9c0a11d0 0x7efe9c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:07.349 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.364+0000 7efece0a7b80 1 -- start start 2023-07-18T03:05:07.349 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.364+0000 7efeb17fa700 1 -- 172.21.15.101:0/4225296016 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7efea80cdf50 con 0x7efeb4100cd0 2023-07-18T03:05:07.349 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.364+0000 7efeb8d9c700 1 --2- 172.21.15.101:0/4225296016 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7efe9c0a11d0 0x7efe9c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:07.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.365+0000 7efece0a7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56098bde1480 0x56098bde1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:07.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.365+0000 7efece0a7b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56098bde1dd0 0x56098bde2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:07.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.365+0000 7efece0a7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56098bde2c20 0x56098bde3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:07.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.365+0000 7efece0a7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56098bce6c60 con 0x56098bde1480 2023-07-18T03:05:07.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.365+0000 7efece0a7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56098bd399f0 con 0x56098bde2c20 2023-07-18T03:05:07.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.365+0000 7efece0a7b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56098bd32e80 con 0x56098bde1dd0 2023-07-18T03:05:07.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.365+0000 7efeb3fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56098bde1480 0x56098bde1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:07.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.365+0000 7efeb3fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56098bde1480 0x56098bde1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:52490/0 (socket says 172.21.15.101:52490) 2023-07-18T03:05:07.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.365+0000 7efeb8d9c700 1 --2- 172.21.15.101:0/4225296016 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7efe9c0a11d0 0x7efe9c0a3680 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efeac003e30 tx=0x7efeac00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:07.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.365+0000 7efeb3fff700 1 -- 172.21.15.101:0/2395054775 learned_addr learned my addr 172.21.15.101:0/2395054775 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:07.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.365+0000 7efeb37fe700 1 --2- 172.21.15.101:0/2395054775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56098bde1dd0 0x56098bde2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:07.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.365+0000 7efeb8d9c700 1 --2- 172.21.15.101:0/2395054775 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56098bde2c20 0x56098bde3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:07.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.365+0000 7efeb3fff700 1 -- 172.21.15.101:0/2395054775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56098bde1dd0 msgr2=0x56098bde2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:05:07.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.365+0000 7efeb3fff700 1 --2- 172.21.15.101:0/2395054775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56098bde1dd0 0x56098bde2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:07.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.365+0000 7efeb3fff700 1 -- 172.21.15.101:0/2395054775 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56098bde2c20 msgr2=0x56098bde3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:05:07.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.365+0000 7efeb3fff700 1 --2- 172.21.15.101:0/2395054775 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56098bde2c20 0x56098bde3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:07.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.365+0000 7efeb3fff700 1 -- 172.21.15.101:0/2395054775 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56098bb60d90 con 0x56098bde1480 2023-07-18T03:05:07.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.365+0000 7efeb37fe700 1 --2- 172.21.15.101:0/2395054775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56098bde1dd0 0x56098bde2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:05:07.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.366+0000 7efeb3fff700 1 --2- 172.21.15.101:0/2395054775 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56098bde1480 0x56098bde1890 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efea40115c0 tx=0x7efea40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46430b549e2ed298 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:07.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.366+0000 7efe9a7fc700 1 -- 172.21.15.101:0/2395054775 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea4004220 con 0x56098bde1480 2023-07-18T03:05:07.355 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.366+0000 7efe9a7fc700 1 -- 172.21.15.101:0/2395054775 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efea4004cf0 con 0x56098bde1480 2023-07-18T03:05:07.355 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.366+0000 7efe9a7fc700 1 -- 172.21.15.101:0/2395054775 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea4005650 con 0x56098bde1480 2023-07-18T03:05:07.355 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.366+0000 7efece0a7b80 1 -- 172.21.15.101:0/2395054775 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56098bde1480 msgr2=0x56098bde1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:07.355 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.366+0000 7efece0a7b80 1 --2- 172.21.15.101:0/2395054775 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56098bde1480 0x56098bde1890 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efea40115c0 tx=0x7efea40118d0 comp rx=0 tx=0).stop 2023-07-18T03:05:07.355 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.367+0000 7efece0a7b80 1 -- 172.21.15.101:0/2395054775 shutdown_connections 2023-07-18T03:05:07.355 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.367+0000 7efece0a7b80 1 --2- 172.21.15.101:0/2395054775 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56098bde2c20 0x56098bde3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:07.355 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.367+0000 7efece0a7b80 1 --2- 172.21.15.101:0/2395054775 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56098bde1480 0x56098bde1890 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:07.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.367+0000 7efece0a7b80 1 --2- 172.21.15.101:0/2395054775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56098bde1dd0 0x56098bde2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:07.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.367+0000 7efece0a7b80 1 -- 172.21.15.101:0/2395054775 >> 172.21.15.101:0/2395054775 conn(0x56098bdd6d50 msgr2=0x56098bddc180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:07.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.367+0000 7efece0a7b80 1 -- 172.21.15.101:0/2395054775 shutdown_connections 2023-07-18T03:05:07.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.367+0000 7efece0a7b80 1 -- 172.21.15.101:0/2395054775 wait complete. 2023-07-18T03:05:07.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.367+0000 7efece0a7b80 1 -- start start 2023-07-18T03:05:07.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.367+0000 7efece0a7b80 1 Processor -- start 2023-07-18T03:05:07.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.368+0000 7efece0a7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56098bde1480 0x56098bdeb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:07.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.368+0000 7efece0a7b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56098bde1dd0 0x56098bdeb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:07.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.368+0000 7efece0a7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56098bde2c20 0x56098bdf2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:07.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.368+0000 7efece0a7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56098bcd2a70 con 0x56098bde2c20 2023-07-18T03:05:07.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.368+0000 7efece0a7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56098bcd2770 con 0x56098bde1480 2023-07-18T03:05:07.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.368+0000 7efece0a7b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56098bbb68d0 con 0x56098bde1dd0 2023-07-18T03:05:07.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.368+0000 7efece0a7b80 10 client.0 _ll_register_callbacks cb 0x56098bcd0710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:05:07.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.368+0000 7efeb8d9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56098bde2c20 0x56098bdf2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:07.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.368+0000 7efeb8d9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56098bde2c20 0x56098bdf2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:52496/0 (socket says 172.21.15.101:52496) 2023-07-18T03:05:07.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.368+0000 7efeb8d9c700 1 -- 172.21.15.101:0/2010186384 learned_addr learned my addr 172.21.15.101:0/2010186384 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:07.361 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.368+0000 7efeb3fff700 1 --2- 172.21.15.101:0/2010186384 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56098bde1480 0x56098bdeb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:07.361 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.368+0000 7efeb37fe700 1 --2- 172.21.15.101:0/2010186384 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56098bde1dd0 0x56098bdeb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:07.361 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.368+0000 7efeb8d9c700 1 -- 172.21.15.101:0/2010186384 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56098bde1dd0 msgr2=0x56098bdeb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:05:07.361 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.368+0000 7efeb8d9c700 1 --2- 172.21.15.101:0/2010186384 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56098bde1dd0 0x56098bdeb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:07.361 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.368+0000 7efeb8d9c700 1 -- 172.21.15.101:0/2010186384 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56098bde1480 msgr2=0x56098bdeb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:05:07.361 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.368+0000 7efeb8d9c700 1 --2- 172.21.15.101:0/2010186384 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56098bde1480 0x56098bdeb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:07.361 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.368+0000 7efeb8d9c700 1 -- 172.21.15.101:0/2010186384 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb419ffe0 con 0x56098bde2c20 2023-07-18T03:05:07.361 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.369+0000 7efeb8d9c700 1 --2- 172.21.15.101:0/2010186384 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56098bde2c20 0x56098bdf2660 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efeac00eef0 tx=0x7efeac004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc67964cf7deae4c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:07.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.369+0000 7efe937fe700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:05:07.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.369+0000 7efe937fe700 1 -- 172.21.15.101:0/2010186384 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeac017d40 con 0x56098bde2c20 2023-07-18T03:05:07.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.369+0000 7efece0a7b80 1 -- 172.21.15.101:0/2010186384 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56098bc8c4a0 con 0x56098bde2c20 2023-07-18T03:05:07.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.369+0000 7efe937fe700 1 -- 172.21.15.101:0/2010186384 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7efeac01c040 con 0x56098bde2c20 2023-07-18T03:05:07.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.369+0000 7efe937fe700 1 -- 172.21.15.101:0/2010186384 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7efeac017ea0 con 0x56098bde2c20 2023-07-18T03:05:07.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.369+0000 7efece0a7b80 10 client.14628 fetch_fsmap learned FSMap version 17 2023-07-18T03:05:07.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.369+0000 7efece0a7b80 1 -- 172.21.15.101:0/2010186384 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x7efea4009d50 con 0x56098bde2c20 2023-07-18T03:05:07.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.369+0000 7efe937fe700 1 -- 172.21.15.101:0/2010186384 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56098bc8c4a0 con 0x56098bde2c20 2023-07-18T03:05:07.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.370+0000 7efe937fe700 1 -- 172.21.15.101:0/2010186384 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 17) v1 ==== 7482+0+0 (secure 0 0 0) 0x7efeac008cb0 con 0x56098bde2c20 2023-07-18T03:05:07.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.370+0000 7efece0a7b80 10 client.14628 fetch_fsmap finished waiting for FSMap version 17 2023-07-18T03:05:07.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.370+0000 7efece0a7b80 10 client.14628 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-18T03:05:07.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.370+0000 7efece0a7b80 20 client.14628 populate_metadata read hostname 'smithi101' 2023-07-18T03:05:07.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.370+0000 7efece0a7b80 1 --2- 172.21.15.101:0/2010186384 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56098bdf6410 0x56098bdf57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:07.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.370+0000 7efece0a7b80 4 client.14628 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:05:07.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.370+0000 7efece0a7b80 1 -- 172.21.15.101:0/2010186384 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56098bd2be60 con 0x56098bdf6410 2023-07-18T03:05:07.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.370+0000 7efeb3fff700 1 --2- 172.21.15.101:0/2010186384 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56098bdf6410 0x56098bdf57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:07.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.371+0000 7efe937fe700 10 client.14628 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:05:07.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.371+0000 7efeb3fff700 1 --2- 172.21.15.101:0/2010186384 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56098bdf6410 0x56098bdf57c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8571f24761ecb1f2 server_cookie=fea0a518ceef5b8c in_seq=0 out_seq=0 2023-07-18T03:05:07.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.371+0000 7efe937fe700 1 -- 172.21.15.101:0/2010186384 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56098bd2be60 con 0x56098bdf6410 2023-07-18T03:05:07.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.371+0000 7efe937fe700 10 client.14628 handle_command_reply: tid=0 2023-07-18T03:05:07.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.371+0000 7efece0a7b80 1 client.14628 shutdown 2023-07-18T03:05:07.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.372+0000 7efece0a7b80 1 -- 172.21.15.101:0/2010186384 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56098bde2c20 msgr2=0x56098bdf2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:07.365 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.372+0000 7efece0a7b80 1 --2- 172.21.15.101:0/2010186384 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56098bde2c20 0x56098bdf2660 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efeac00eef0 tx=0x7efeac004ad0 comp rx=0 tx=0).stop 2023-07-18T03:05:07.365 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.372+0000 7efece0a7b80 1 -- 172.21.15.101:0/2010186384 shutdown_connections 2023-07-18T03:05:07.365 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.372+0000 7efece0a7b80 1 --2- 172.21.15.101:0/2010186384 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56098bdf6410 0x56098bdf57c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:07.365 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.372+0000 7efece0a7b80 1 --2- 172.21.15.101:0/2010186384 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56098bde1480 0x56098bdeb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:07.365 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.372+0000 7efece0a7b80 1 --2- 172.21.15.101:0/2010186384 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56098bde2c20 0x56098bdf2660 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:07.365 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.372+0000 7efece0a7b80 1 --2- 172.21.15.101:0/2010186384 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56098bde1dd0 0x56098bdeb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:07.365 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.372+0000 7efe937fe700 0 client.14628 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:05:07.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.372+0000 7efece0a7b80 1 -- 172.21.15.101:0/2010186384 >> 172.21.15.101:0/2010186384 conn(0x56098bdd6d50 msgr2=0x56098bddb860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:07.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.372+0000 7efece0a7b80 1 -- 172.21.15.101:0/2010186384 shutdown_connections 2023-07-18T03:05:07.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.372+0000 7efece0a7b80 1 -- 172.21.15.101:0/2010186384 wait complete. 2023-07-18T03:05:07.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.372+0000 7efece0a7b80 20 client.14628 trim_cache size 0 max 16384 2023-07-18T03:05:07.374 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.390+0000 7efece0a7b80 1 Processor -- start 2023-07-18T03:05:07.374 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.390+0000 7efece0a7b80 1 -- start start 2023-07-18T03:05:07.375 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.390+0000 7efece0a7b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56098bde1480 0x56098bddb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:07.375 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.390+0000 7efece0a7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56098bde1dd0 0x56098bdf2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:07.375 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.390+0000 7efece0a7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56098bde2c20 0x56098bde9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:07.375 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.390+0000 7efece0a7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56098bdeacb0 con 0x56098bde1dd0 2023-07-18T03:05:07.375 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.390+0000 7efece0a7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56098bde9b30 con 0x56098bde2c20 2023-07-18T03:05:07.375 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.390+0000 7efece0a7b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56098bdec2d0 con 0x56098bde1480 2023-07-18T03:05:07.377 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.390+0000 7efeb37fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56098bde1dd0 0x56098bdf2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:07.377 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.390+0000 7efeb37fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56098bde1dd0 0x56098bdf2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:52498/0 (socket says 172.21.15.101:52498) 2023-07-18T03:05:07.377 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.390+0000 7efeb37fe700 1 -- 172.21.15.101:0/3288913302 learned_addr learned my addr 172.21.15.101:0/3288913302 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:07.377 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.390+0000 7efeb8d9c700 1 --2- 172.21.15.101:0/3288913302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56098bde2c20 0x56098bde9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:07.377 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.390+0000 7efeb3fff700 1 --2- 172.21.15.101:0/3288913302 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56098bde1480 0x56098bddb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:07.378 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.391+0000 7efeb37fe700 1 -- 172.21.15.101:0/3288913302 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56098bde1480 msgr2=0x56098bddb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:05:07.378 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.391+0000 7efeb37fe700 1 --2- 172.21.15.101:0/3288913302 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56098bde1480 0x56098bddb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:07.378 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.391+0000 7efeb37fe700 1 -- 172.21.15.101:0/3288913302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56098bde2c20 msgr2=0x56098bde9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:05:07.378 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.391+0000 7efeb37fe700 1 --2- 172.21.15.101:0/3288913302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56098bde2c20 0x56098bde9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:07.378 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.391+0000 7efeb37fe700 1 -- 172.21.15.101:0/3288913302 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56098bde32c0 con 0x56098bde1dd0 2023-07-18T03:05:07.378 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.391+0000 7efeb3fff700 1 --2- 172.21.15.101:0/3288913302 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56098bde1480 0x56098bddb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:05:07.378 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.391+0000 7efeb37fe700 1 --2- 172.21.15.101:0/3288913302 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56098bde1dd0 0x56098bdf2580 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7efea8022890 tx=0x7efea801a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbb2777314ba7634 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:07.378 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.391+0000 7efe9affd700 1 -- 172.21.15.101:0/3288913302 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea809b1a0 con 0x56098bde1dd0 2023-07-18T03:05:07.379 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.391+0000 7efe9affd700 1 -- 172.21.15.101:0/3288913302 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efea809f1d0 con 0x56098bde1dd0 2023-07-18T03:05:07.379 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.391+0000 7efe9affd700 1 -- 172.21.15.101:0/3288913302 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea809b400 con 0x56098bde1dd0 2023-07-18T03:05:07.379 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.392+0000 7efece0a7b80 1 -- 172.21.15.101:0/3288913302 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56098bde1dd0 msgr2=0x56098bdf2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:07.379 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.392+0000 7efece0a7b80 1 --2- 172.21.15.101:0/3288913302 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56098bde1dd0 0x56098bdf2580 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7efea8022890 tx=0x7efea801a040 comp rx=0 tx=0).stop 2023-07-18T03:05:07.379 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.392+0000 7efece0a7b80 1 -- 172.21.15.101:0/3288913302 shutdown_connections 2023-07-18T03:05:07.379 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.392+0000 7efece0a7b80 1 --2- 172.21.15.101:0/3288913302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56098bde2c20 0x56098bde9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:07.379 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.392+0000 7efece0a7b80 1 --2- 172.21.15.101:0/3288913302 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56098bde1480 0x56098bddb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:07.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.392+0000 7efece0a7b80 1 --2- 172.21.15.101:0/3288913302 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56098bde1dd0 0x56098bdf2580 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:07.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.392+0000 7efece0a7b80 1 -- 172.21.15.101:0/3288913302 >> 172.21.15.101:0/3288913302 conn(0x56098bdd6d50 msgr2=0x56098bde9e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:07.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.392+0000 7efece0a7b80 1 -- 172.21.15.101:0/3288913302 shutdown_connections 2023-07-18T03:05:07.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.392+0000 7efece0a7b80 1 -- 172.21.15.101:0/3288913302 wait complete. 2023-07-18T03:05:07.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.392+0000 7efece0a7b80 1 -- start start 2023-07-18T03:05:07.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.392+0000 7efece0a7b80 1 Processor -- start 2023-07-18T03:05:07.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.393+0000 7efece0a7b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56098bde1480 0x56098bded7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:07.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.393+0000 7efece0a7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56098bde1dd0 0x56098bdf2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:07.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.393+0000 7efece0a7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56098bde2c20 0x56098bdeddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:07.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.393+0000 7efece0a7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56098be4a2b0 con 0x56098bde2c20 2023-07-18T03:05:07.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.393+0000 7efece0a7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56098be4a3f0 con 0x56098bde1dd0 2023-07-18T03:05:07.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.393+0000 7efece0a7b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56098be4a530 con 0x56098bde1480 2023-07-18T03:05:07.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.393+0000 7efece0a7b80 10 client.0 _ll_register_callbacks cb 0x7efeac00e890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:05:07.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.393+0000 7efeb37fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56098bde1dd0 0x56098bdf2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:07.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.393+0000 7efeb8d9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56098bde2c20 0x56098bdeddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:07.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.393+0000 7efeb37fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56098bde1dd0 0x56098bdf2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:48062/0 (socket says 172.21.15.101:48062) 2023-07-18T03:05:07.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.393+0000 7efeb37fe700 1 -- 172.21.15.101:0/11311565 learned_addr learned my addr 172.21.15.101:0/11311565 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:07.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.393+0000 7efeb3fff700 1 --2- 172.21.15.101:0/11311565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56098bde1480 0x56098bded7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:07.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.393+0000 7efeb37fe700 1 -- 172.21.15.101:0/11311565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56098bde1480 msgr2=0x56098bded7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:05:07.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.393+0000 7efeb37fe700 1 --2- 172.21.15.101:0/11311565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56098bde1480 0x56098bded7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:07.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.393+0000 7efeb37fe700 1 -- 172.21.15.101:0/11311565 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56098bde2c20 msgr2=0x56098bdeddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:05:07.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.393+0000 7efeb37fe700 1 --2- 172.21.15.101:0/11311565 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56098bde2c20 0x56098bdeddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:07.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.393+0000 7efeb37fe700 1 -- 172.21.15.101:0/11311565 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb419fb60 con 0x56098bde1dd0 2023-07-18T03:05:07.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.394+0000 7efeb37fe700 1 --2- 172.21.15.101:0/11311565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56098bde1dd0 0x56098bdf2660 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efea80975b0 tx=0x7efea8005ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd1ce536b3a8da32 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:07.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.394+0000 7efe937fe700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:05:07.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.394+0000 7efe937fe700 1 -- 172.21.15.101:0/11311565 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea80c9070 con 0x56098bde1dd0 2023-07-18T03:05:07.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.394+0000 7efece0a7b80 1 -- 172.21.15.101:0/11311565 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efea4004cf0 con 0x56098bde1dd0 2023-07-18T03:05:07.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.394+0000 7efe937fe700 1 -- 172.21.15.101:0/11311565 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7efea809b1a0 con 0x56098bde1dd0 2023-07-18T03:05:07.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.394+0000 7efe937fe700 1 -- 172.21.15.101:0/11311565 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7efea80cb910 con 0x56098bde1dd0 2023-07-18T03:05:07.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.394+0000 7efece0a7b80 10 client.24671 fetch_fsmap learned FSMap version 17 2023-07-18T03:05:07.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.394+0000 7efece0a7b80 1 -- 172.21.15.101:0/11311565 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x56098bb49c10 con 0x56098bde1dd0 2023-07-18T03:05:07.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.394+0000 7efe937fe700 1 -- 172.21.15.101:0/11311565 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efea4004cf0 con 0x56098bde1dd0 2023-07-18T03:05:07.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.395+0000 7efe937fe700 1 -- 172.21.15.101:0/11311565 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 17) v1 ==== 7482+0+0 (secure 0 0 0) 0x7efea80cb2b0 con 0x56098bde1dd0 2023-07-18T03:05:07.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.395+0000 7efece0a7b80 10 client.24671 fetch_fsmap finished waiting for FSMap version 17 2023-07-18T03:05:07.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.395+0000 7efece0a7b80 10 client.24671 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-18T03:05:07.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.395+0000 7efece0a7b80 20 client.24671 populate_metadata read hostname 'smithi101' 2023-07-18T03:05:07.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.395+0000 7efece0a7b80 1 --2- 172.21.15.101:0/11311565 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56098be4edf0 0x56098be4b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:07.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.395+0000 7efece0a7b80 4 client.24671 mds_command: new command op to 24562 tid=0[{"prefix": "status"}] 2023-07-18T03:05:07.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.395+0000 7efece0a7b80 1 -- 172.21.15.101:0/11311565 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56098bcee000 con 0x56098be4edf0 2023-07-18T03:05:07.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.395+0000 7efeb3fff700 1 --2- 172.21.15.101:0/11311565 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56098be4edf0 0x56098be4b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:07.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.395+0000 7efe937fe700 10 client.24671 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:05:07.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.395+0000 7efeb3fff700 1 --2- 172.21.15.101:0/11311565 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56098be4edf0 0x56098be4b7b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46cedb79d21becd9 server_cookie=fe4fd22edba4a595 in_seq=0 out_seq=0 2023-07-18T03:05:07.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.396+0000 7efe937fe700 1 -- 172.21.15.101:0/11311565 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x56098bcee000 con 0x56098be4edf0 2023-07-18T03:05:07.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.396+0000 7efe937fe700 10 client.24671 handle_command_reply: tid=0 2023-07-18T03:05:07.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.396+0000 7efece0a7b80 1 client.24671 shutdown 2023-07-18T03:05:07.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.396+0000 7efece0a7b80 1 -- 172.21.15.101:0/11311565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56098bde1dd0 msgr2=0x56098bdf2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:07.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.396+0000 7efece0a7b80 1 --2- 172.21.15.101:0/11311565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56098bde1dd0 0x56098bdf2660 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efea80975b0 tx=0x7efea8005ad0 comp rx=0 tx=0).stop 2023-07-18T03:05:07.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.396+0000 7efece0a7b80 1 -- 172.21.15.101:0/11311565 shutdown_connections 2023-07-18T03:05:07.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.396+0000 7efece0a7b80 1 --2- 172.21.15.101:0/11311565 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56098be4edf0 0x56098be4b7b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:07.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.396+0000 7efece0a7b80 1 --2- 172.21.15.101:0/11311565 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56098bde2c20 0x56098bdeddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:07.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.396+0000 7efe937fe700 0 client.24671 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:05:07.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.396+0000 7efece0a7b80 1 --2- 172.21.15.101:0/11311565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56098bde1480 0x56098bded7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:07.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.396+0000 7efece0a7b80 1 --2- 172.21.15.101:0/11311565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56098bde1dd0 0x56098bdf2660 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:07.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.396+0000 7efece0a7b80 1 -- 172.21.15.101:0/11311565 >> 172.21.15.101:0/11311565 conn(0x56098bdd6d50 msgr2=0x56098bde9030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:07.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.396+0000 7efece0a7b80 1 -- 172.21.15.101:0/11311565 shutdown_connections 2023-07-18T03:05:07.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.396+0000 7efece0a7b80 1 -- 172.21.15.101:0/11311565 wait complete. 2023-07-18T03:05:07.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.396+0000 7efece0a7b80 20 client.24671 trim_cache size 0 max 16384 2023-07-18T03:05:07.393 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:05:07.394 INFO:teuthology.orchestra.run.smithi101.stdout: "cluster_fsid": "b1782daa-2516-11ee-9b34-001a4aab830c", 2023-07-18T03:05:07.394 INFO:teuthology.orchestra.run.smithi101.stdout: "whoami": 0, 2023-07-18T03:05:07.394 INFO:teuthology.orchestra.run.smithi101.stdout: "id": 24562, 2023-07-18T03:05:07.394 INFO:teuthology.orchestra.run.smithi101.stdout: "want_state": "up:active", 2023-07-18T03:05:07.394 INFO:teuthology.orchestra.run.smithi101.stdout: "state": "up:active", 2023-07-18T03:05:07.394 INFO:teuthology.orchestra.run.smithi101.stdout: "fs_name": "cephfs", 2023-07-18T03:05:07.394 INFO:teuthology.orchestra.run.smithi101.stdout: "rank_uptime": 7.5647547680000002, 2023-07-18T03:05:07.394 INFO:teuthology.orchestra.run.smithi101.stdout: "mdsmap_epoch": 17, 2023-07-18T03:05:07.395 INFO:teuthology.orchestra.run.smithi101.stdout: "osdmap_epoch": 87, 2023-07-18T03:05:07.395 INFO:teuthology.orchestra.run.smithi101.stdout: "osdmap_epoch_barrier": 87, 2023-07-18T03:05:07.395 INFO:teuthology.orchestra.run.smithi101.stdout: "uptime": 22.151750479 2023-07-18T03:05:07.395 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:05:07.396 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.397+0000 7efeba7ff700 1 -- 172.21.15.101:0/4225296016 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7efe9c0a11d0 msgr2=0x7efe9c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:07.396 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.397+0000 7efeba7ff700 1 --2- 172.21.15.101:0/4225296016 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7efe9c0a11d0 0x7efe9c0a3680 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efeac003e30 tx=0x7efeac00b040 comp rx=0 tx=0).stop 2023-07-18T03:05:07.396 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.397+0000 7efeba7ff700 1 -- 172.21.15.101:0/4225296016 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efeb4100cd0 msgr2=0x7efeb4197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:07.396 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.397+0000 7efeba7ff700 1 --2- 172.21.15.101:0/4225296016 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efeb4100cd0 0x7efeb4197fd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7efea800b620 tx=0x7efea8005740 comp rx=0 tx=0).stop 2023-07-18T03:05:07.396 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.397+0000 7efeba7ff700 1 -- 172.21.15.101:0/4225296016 shutdown_connections 2023-07-18T03:05:07.396 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.397+0000 7efeba7ff700 1 --2- 172.21.15.101:0/4225296016 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7efe9c0a11d0 0x7efe9c0a3680 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:07.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.397+0000 7efeba7ff700 1 --2- 172.21.15.101:0/4225296016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efeb40ffae0 0x7efeb4197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:07.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.397+0000 7efeba7ff700 1 --2- 172.21.15.101:0/4225296016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efeb4101660 0x7efeb419f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:07.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.397+0000 7efeba7ff700 1 --2- 172.21.15.101:0/4225296016 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efeb4100cd0 0x7efeb4197fd0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:07.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.397+0000 7efeba7ff700 1 -- 172.21.15.101:0/4225296016 >> 172.21.15.101:0/4225296016 conn(0x7efeb40fb0e0 msgr2=0x7efeb40fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:07.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.397+0000 7efeba7ff700 1 -- 172.21.15.101:0/4225296016 shutdown_connections 2023-07-18T03:05:07.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:07.397+0000 7efeba7ff700 1 -- 172.21.15.101:0/4225296016 wait complete. 2023-07-18T03:05:07.399 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-18T03:05:07.400 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-07-18T03:05:07.400 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph fs ls 2023-07-18T03:05:08.117 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:05:08.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:07 smithi179 ceph-mon[129229]: pgmap v279: 129 pgs: 5 creating+peering, 5 unknown, 119 active+clean; 579 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2023-07-18T03:05:08.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:07 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/465795300' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:05:08.228 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:07 smithi101 ceph-mon[126379]: pgmap v279: 129 pgs: 5 creating+peering, 5 unknown, 119 active+clean; 579 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2023-07-18T03:05:08.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:07 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/465795300' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:05:08.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:07 smithi155 ceph-mon[128885]: pgmap v279: 129 pgs: 5 creating+peering, 5 unknown, 119 active+clean; 579 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2023-07-18T03:05:08.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:07 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/465795300' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:05:09.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.675+0000 7f086d10c700 1 -- 172.21.15.101:0/4204364647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08680ffe00 msgr2=0x7f08681001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:09.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.675+0000 7f086d10c700 1 --2- 172.21.15.101:0/4204364647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08680ffe00 0x7f08681001d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0858009990 tx=0x7f0858009ca0 comp rx=0 tx=0).stop 2023-07-18T03:05:09.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.675+0000 7f086d10c700 1 -- 172.21.15.101:0/4204364647 shutdown_connections 2023-07-18T03:05:09.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.675+0000 7f086d10c700 1 --2- 172.21.15.101:0/4204364647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08680ffe00 0x7f08681001d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:09.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.675+0000 7f086d10c700 1 --2- 172.21.15.101:0/4204364647 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0868104ea0 0x7f086810ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:09.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.675+0000 7f086d10c700 1 --2- 172.21.15.101:0/4204364647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08681007a0 0x7f0868104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:09.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.675+0000 7f086d10c700 1 -- 172.21.15.101:0/4204364647 >> 172.21.15.101:0/4204364647 conn(0x7f08680fb660 msgr2=0x7f08680fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:09.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.675+0000 7f086d10c700 1 -- 172.21.15.101:0/4204364647 shutdown_connections 2023-07-18T03:05:09.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.675+0000 7f086d10c700 1 -- 172.21.15.101:0/4204364647 wait complete. 2023-07-18T03:05:09.662 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.676+0000 7f086d10c700 1 Processor -- start 2023-07-18T03:05:09.662 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.676+0000 7f086d10c700 1 -- start start 2023-07-18T03:05:09.663 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.677+0000 7f086d10c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f08680ffe00 0x7f0868072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:09.663 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.677+0000 7f086d10c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08681007a0 0x7f0868072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:09.663 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.677+0000 7f086d10c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0868104ea0 0x7f08680732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:09.663 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.677+0000 7f086d10c700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08681a40d0 con 0x7f08680ffe00 2023-07-18T03:05:09.663 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.677+0000 7f086d10c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08681a4240 con 0x7f0868104ea0 2023-07-18T03:05:09.663 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.677+0000 7f086d10c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08681a4400 con 0x7f08681007a0 2023-07-18T03:05:09.663 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.677+0000 7f0866d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f08680ffe00 0x7f0868072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:09.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.677+0000 7f0866d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f08680ffe00 0x7f0868072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:52520/0 (socket says 172.21.15.101:52520) 2023-07-18T03:05:09.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.677+0000 7f0866d9d700 1 -- 172.21.15.101:0/977161275 learned_addr learned my addr 172.21.15.101:0/977161275 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:09.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.677+0000 7f086759e700 1 --2- 172.21.15.101:0/977161275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0868104ea0 0x7f08680732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:09.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.677+0000 7f086659c700 1 --2- 172.21.15.101:0/977161275 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08681007a0 0x7f0868072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:09.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.678+0000 7f0866d9d700 1 -- 172.21.15.101:0/977161275 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08681007a0 msgr2=0x7f0868072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:09.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.678+0000 7f0866d9d700 1 --2- 172.21.15.101:0/977161275 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08681007a0 0x7f0868072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:09.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.678+0000 7f0866d9d700 1 -- 172.21.15.101:0/977161275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0868104ea0 msgr2=0x7f08680732d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:09.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.678+0000 7f0866d9d700 1 --2- 172.21.15.101:0/977161275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0868104ea0 0x7f08680732d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:09.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.678+0000 7f0866d9d700 1 -- 172.21.15.101:0/977161275 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08580095f0 con 0x7f08680ffe00 2023-07-18T03:05:09.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.678+0000 7f0866d9d700 1 --2- 172.21.15.101:0/977161275 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f08680ffe00 0x7f0868072810 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0858000c00 tx=0x7f0858005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:09.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.678+0000 7f084ffff700 1 -- 172.21.15.101:0/977161275 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085801d070 con 0x7f08680ffe00 2023-07-18T03:05:09.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.678+0000 7f086d10c700 1 -- 172.21.15.101:0/977161275 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08681a4680 con 0x7f08680ffe00 2023-07-18T03:05:09.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.678+0000 7f084ffff700 1 -- 172.21.15.101:0/977161275 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0858022470 con 0x7f08680ffe00 2023-07-18T03:05:09.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.678+0000 7f084ffff700 1 -- 172.21.15.101:0/977161275 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0858017650 con 0x7f08680ffe00 2023-07-18T03:05:09.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.678+0000 7f086d10c700 1 -- 172.21.15.101:0/977161275 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08681a4af0 con 0x7f08680ffe00 2023-07-18T03:05:09.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.679+0000 7f086d10c700 1 -- 172.21.15.101:0/977161275 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f086804e210 con 0x7f08680ffe00 2023-07-18T03:05:09.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.681+0000 7f084ffff700 1 -- 172.21.15.101:0/977161275 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f08580225e0 con 0x7f08680ffe00 2023-07-18T03:05:09.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.682+0000 7f084ffff700 1 --2- 172.21.15.101:0/977161275 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f08500a1150 0x7f08500a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:09.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.683+0000 7f084ffff700 1 -- 172.21.15.101:0/977161275 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f08580ce5b0 con 0x7f08680ffe00 2023-07-18T03:05:09.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.683+0000 7f086659c700 1 --2- 172.21.15.101:0/977161275 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f08500a1150 0x7f08500a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:09.668 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.683+0000 7f086659c700 1 --2- 172.21.15.101:0/977161275 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f08500a1150 0x7f08500a3600 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0868074000 tx=0x7f085c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:09.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.685+0000 7f084ffff700 1 -- 172.21.15.101:0/977161275 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0858097b40 con 0x7f08680ffe00 2023-07-18T03:05:09.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.929+0000 7f086d10c700 1 -- 172.21.15.101:0/977161275 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f08681a4d10 con 0x7f08680ffe00 2023-07-18T03:05:09.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.929+0000 7f084ffff700 1 -- 172.21.15.101:0/977161275 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7f085809b160 con 0x7f08680ffe00 2023-07-18T03:05:09.915 INFO:teuthology.orchestra.run.smithi101.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-07-18T03:05:09.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.932+0000 7f086d10c700 1 -- 172.21.15.101:0/977161275 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f08500a1150 msgr2=0x7f08500a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:09.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.932+0000 7f086d10c700 1 --2- 172.21.15.101:0/977161275 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f08500a1150 0x7f08500a3600 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0868074000 tx=0x7f085c009260 comp rx=0 tx=0).stop 2023-07-18T03:05:09.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.932+0000 7f086d10c700 1 -- 172.21.15.101:0/977161275 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f08680ffe00 msgr2=0x7f0868072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:09.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.932+0000 7f086d10c700 1 --2- 172.21.15.101:0/977161275 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f08680ffe00 0x7f0868072810 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0858000c00 tx=0x7f0858005740 comp rx=0 tx=0).stop 2023-07-18T03:05:09.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.932+0000 7f086d10c700 1 -- 172.21.15.101:0/977161275 shutdown_connections 2023-07-18T03:05:09.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.932+0000 7f086d10c700 1 --2- 172.21.15.101:0/977161275 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f08500a1150 0x7f08500a3600 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:09.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.932+0000 7f086d10c700 1 --2- 172.21.15.101:0/977161275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0868104ea0 0x7f08680732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:09.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.932+0000 7f086d10c700 1 --2- 172.21.15.101:0/977161275 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f08680ffe00 0x7f0868072810 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:09.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.932+0000 7f086d10c700 1 --2- 172.21.15.101:0/977161275 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08681007a0 0x7f0868072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:09.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.932+0000 7f086d10c700 1 -- 172.21.15.101:0/977161275 >> 172.21.15.101:0/977161275 conn(0x7f08680fb660 msgr2=0x7f0868103750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:09.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.932+0000 7f086d10c700 1 -- 172.21.15.101:0/977161275 shutdown_connections 2023-07-18T03:05:09.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:09.932+0000 7f086d10c700 1 -- 172.21.15.101:0/977161275 wait complete. 2023-07-18T03:05:10.036 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:09 smithi155 ceph-mon[128885]: pgmap v280: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2023-07-18T03:05:10.074 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:09 smithi101 ceph-mon[126379]: pgmap v280: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2023-07-18T03:05:10.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:09 smithi179 ceph-mon[129229]: pgmap v280: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2023-07-18T03:05:10.494 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-07-18T03:05:10.495 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-18T03:05:10.495 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi101.front.sepia.ceph.com 2023-07-18T03:05:10.495 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-07-18T03:05:10.496 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-07-18T03:05:10.496 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-07-18T03:05:10.497 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-18T03:05:10.497 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-18T03:05:10.497 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-07-18T03:05:10.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:05:10.498 DEBUG:teuthology.orchestra.run.smithi101:> ip addr 2023-07-18T03:05:10.544 INFO:teuthology.orchestra.run.smithi101.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-18T03:05:10.544 INFO:teuthology.orchestra.run.smithi101.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-18T03:05:10.544 INFO:teuthology.orchestra.run.smithi101.stdout: inet 127.0.0.1/8 scope host lo 2023-07-18T03:05:10.544 INFO:teuthology.orchestra.run.smithi101.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:05:10.545 INFO:teuthology.orchestra.run.smithi101.stdout: inet6 ::1/128 scope host 2023-07-18T03:05:10.545 INFO:teuthology.orchestra.run.smithi101.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:05:10.545 INFO:teuthology.orchestra.run.smithi101.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-18T03:05:10.545 INFO:teuthology.orchestra.run.smithi101.stdout: link/ether 0c:c4:7a:d8:a1:f2 brd ff:ff:ff:ff:ff:ff 2023-07-18T03:05:10.545 INFO:teuthology.orchestra.run.smithi101.stdout: altname enp7s0f0 2023-07-18T03:05:10.545 INFO:teuthology.orchestra.run.smithi101.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-18T03:05:10.545 INFO:teuthology.orchestra.run.smithi101.stdout: link/ether 0c:c4:7a:d8:a1:f3 brd ff:ff:ff:ff:ff:ff 2023-07-18T03:05:10.546 INFO:teuthology.orchestra.run.smithi101.stdout: altname enp7s0f1 2023-07-18T03:05:10.546 INFO:teuthology.orchestra.run.smithi101.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-18T03:05:10.546 INFO:teuthology.orchestra.run.smithi101.stdout: link/ether 0c:c4:7a:8f:d7:22 brd ff:ff:ff:ff:ff:ff 2023-07-18T03:05:10.546 INFO:teuthology.orchestra.run.smithi101.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-18T03:05:10.546 INFO:teuthology.orchestra.run.smithi101.stdout: link/ether 0c:c4:7a:8f:d7:23 brd ff:ff:ff:ff:ff:ff 2023-07-18T03:05:10.546 INFO:teuthology.orchestra.run.smithi101.stdout: inet 172.21.15.101/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-18T03:05:10.546 INFO:teuthology.orchestra.run.smithi101.stdout: valid_lft 41422sec preferred_lft 41422sec 2023-07-18T03:05:10.547 INFO:teuthology.orchestra.run.smithi101.stdout: inet6 fe80::ac5b:f81b:4a3:d684/64 scope link noprefixroute 2023-07-18T03:05:10.547 INFO:teuthology.orchestra.run.smithi101.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:05:10.547 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-07-18T03:05:10.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:05:10.548 DEBUG:teuthology.orchestra.run.smithi101:> (cd / && exec bash -c ' 2023-07-18T03:05:10.548 DEBUG:teuthology.orchestra.run.smithi101:> set -e 2023-07-18T03:05:10.549 DEBUG:teuthology.orchestra.run.smithi101:> sudo ip link add name ceph-brx type bridge 2023-07-18T03:05:10.549 DEBUG:teuthology.orchestra.run.smithi101:> sudo ip addr flush dev ceph-brx 2023-07-18T03:05:10.549 DEBUG:teuthology.orchestra.run.smithi101:> sudo ip link set ceph-brx up 2023-07-18T03:05:10.549 DEBUG:teuthology.orchestra.run.smithi101:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-07-18T03:05:10.550 DEBUG:teuthology.orchestra.run.smithi101:> ') 2023-07-18T03:05:11.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:05:11.017 DEBUG:teuthology.orchestra.run.smithi101:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-18T03:05:11.019 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:10 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/977161275' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-18T03:05:11.036 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:10 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/977161275' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-18T03:05:11.088 INFO:teuthology.orchestra.run.smithi101.stdout:1 2023-07-18T03:05:11.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:05:11.092 DEBUG:teuthology.orchestra.run.smithi101:> route 2023-07-18T03:05:11.153 INFO:teuthology.orchestra.run.smithi101.stdout:Kernel IP routing table 2023-07-18T03:05:11.153 INFO:teuthology.orchestra.run.smithi101.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-18T03:05:11.153 INFO:teuthology.orchestra.run.smithi101.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-18T03:05:11.153 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-18T03:05:11.154 INFO:teuthology.orchestra.run.smithi101.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-18T03:05:11.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:05:11.155 DEBUG:teuthology.orchestra.run.smithi101:> (cd / && exec bash -c ' 2023-07-18T03:05:11.155 DEBUG:teuthology.orchestra.run.smithi101:> set -e 2023-07-18T03:05:11.156 DEBUG:teuthology.orchestra.run.smithi101:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-18T03:05:11.156 DEBUG:teuthology.orchestra.run.smithi101:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-18T03:05:11.156 DEBUG:teuthology.orchestra.run.smithi101:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-18T03:05:11.157 DEBUG:teuthology.orchestra.run.smithi101:> ') 2023-07-18T03:05:11.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:10 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/977161275' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-18T03:05:11.920 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:11 smithi101 ceph-mon[126379]: pgmap v281: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 896 B/s wr, 0 op/s 2023-07-18T03:05:11.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:05:11.923 DEBUG:teuthology.orchestra.run.smithi101:> ip netns list 2023-07-18T03:05:11.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:05:11.968 DEBUG:teuthology.orchestra.run.smithi101:> ip netns list-id 2023-07-18T03:05:12.036 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:11 smithi155 ceph-mon[128885]: pgmap v281: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 896 B/s wr, 0 op/s 2023-07-18T03:05:12.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:05:12.051 DEBUG:teuthology.orchestra.run.smithi101:> (cd / && exec bash -c ' 2023-07-18T03:05:12.052 DEBUG:teuthology.orchestra.run.smithi101:> set -e 2023-07-18T03:05:12.052 DEBUG:teuthology.orchestra.run.smithi101:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-07-18T03:05:12.052 DEBUG:teuthology.orchestra.run.smithi101:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-07-18T03:05:12.052 DEBUG:teuthology.orchestra.run.smithi101:> ') 2023-07-18T03:05:12.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:11 smithi179 ceph-mon[129229]: pgmap v281: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 896 B/s wr, 0 op/s 2023-07-18T03:05:12.236 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-07-18T03:05:12.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:05:12.237 DEBUG:teuthology.orchestra.run.smithi101:> (cd / && exec bash -c ' 2023-07-18T03:05:12.237 DEBUG:teuthology.orchestra.run.smithi101:> set -e 2023-07-18T03:05:12.238 DEBUG:teuthology.orchestra.run.smithi101:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-07-18T03:05:12.238 DEBUG:teuthology.orchestra.run.smithi101:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-07-18T03:05:12.238 DEBUG:teuthology.orchestra.run.smithi101:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-07-18T03:05:12.238 DEBUG:teuthology.orchestra.run.smithi101:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-07-18T03:05:12.239 DEBUG:teuthology.orchestra.run.smithi101:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-07-18T03:05:12.239 DEBUG:teuthology.orchestra.run.smithi101:> ') 2023-07-18T03:05:12.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:05:12.506 DEBUG:teuthology.orchestra.run.smithi101:> (cd / && exec bash -c ' 2023-07-18T03:05:12.506 DEBUG:teuthology.orchestra.run.smithi101:> set -e 2023-07-18T03:05:12.506 DEBUG:teuthology.orchestra.run.smithi101:> sudo ip link set brx.0 up 2023-07-18T03:05:12.507 DEBUG:teuthology.orchestra.run.smithi101:> sudo ip link set dev brx.0 master ceph-brx 2023-07-18T03:05:12.507 DEBUG:teuthology.orchestra.run.smithi101:> ') 2023-07-18T03:05:12.680 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-07-18T03:05:12.680 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-18T03:05:12.681 DEBUG:teuthology.orchestra.run.smithi101:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-07-18T03:05:12.724 INFO:teuthology.orchestra.run.smithi101.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-07-18T03:05:12.724 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-18T03:05:12.725 DEBUG:teuthology.orchestra.run.smithi101:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-07-18T03:05:12.807 DEBUG:teuthology.orchestra.run.smithi101:> sudo modprobe fuse 2023-07-18T03:05:12.917 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-18T03:05:12.918 DEBUG:teuthology.orchestra.run.smithi101:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-18T03:05:12.985 INFO:teuthology.orchestra.run.smithi101.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-18T03:05:12.988 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-18T03:05:12.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:05:12.989 DEBUG:teuthology.orchestra.run.smithi101:> ls /sys/fs/fuse/connections 2023-07-18T03:05:13.032 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-18T03:05:13.033 DEBUG:teuthology.orchestra.run.smithi101:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-07-18T03:05:13.077 DEBUG:teuthology.orchestra.run.smithi101:> sudo modprobe fuse 2023-07-18T03:05:13.150 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-18T03:05:13.150 DEBUG:teuthology.orchestra.run.smithi101:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-18T03:05:13.220 INFO:teuthology.orchestra.run.smithi101.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-18T03:05:13.223 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-18T03:05:13.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:05:13.224 DEBUG:teuthology.orchestra.run.smithi101:> ls /sys/fs/fuse/connections 2023-07-18T03:05:13.290 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi101.stderr:2023-07-18T03:05:13.306+0000 7fda85c9d5c0 -1 init, newargv = 0x55ca29832e00 newargc=15 2023-07-18T03:05:13.290 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi101.stderr:ceph-fuse[163692]: starting ceph client 2023-07-18T03:05:13.300 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi101.stderr:ceph-fuse[163692]: starting fuse 2023-07-18T03:05:14.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:13 smithi179 ceph-mon[129229]: pgmap v282: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-07-18T03:05:14.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:13 smithi101 ceph-mon[126379]: pgmap v282: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-07-18T03:05:14.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:13 smithi155 ceph-mon[128885]: pgmap v282: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-07-18T03:05:14.369 DEBUG:teuthology.orchestra.run.smithi101:> sudo modprobe fuse 2023-07-18T03:05:14.447 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-18T03:05:14.448 DEBUG:teuthology.orchestra.run.smithi101:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-18T03:05:14.517 INFO:teuthology.orchestra.run.smithi101.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-18T03:05:14.520 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-18T03:05:14.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:05:14.522 DEBUG:teuthology.orchestra.run.smithi101:> ls /sys/fs/fuse/connections 2023-07-18T03:05:14.567 INFO:teuthology.orchestra.run.smithi101.stdout:121 2023-07-18T03:05:14.567 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [121] 2023-07-18T03:05:14.568 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-18T03:05:14.568 DEBUG:teuthology.orchestra.run.smithi101:> import glob 2023-07-18T03:05:14.568 DEBUG:teuthology.orchestra.run.smithi101:> import re 2023-07-18T03:05:14.569 DEBUG:teuthology.orchestra.run.smithi101:> import os 2023-07-18T03:05:14.569 DEBUG:teuthology.orchestra.run.smithi101:> import subprocess 2023-07-18T03:05:14.570 DEBUG:teuthology.orchestra.run.smithi101:> 2023-07-18T03:05:14.570 DEBUG:teuthology.orchestra.run.smithi101:> def _find_admin_socket(client_name): 2023-07-18T03:05:14.571 DEBUG:teuthology.orchestra.run.smithi101:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-07-18T03:05:14.572 DEBUG:teuthology.orchestra.run.smithi101:> files = glob.glob(asok_path) 2023-07-18T03:05:14.573 DEBUG:teuthology.orchestra.run.smithi101:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-07-18T03:05:14.574 DEBUG:teuthology.orchestra.run.smithi101:> 2023-07-18T03:05:14.574 DEBUG:teuthology.orchestra.run.smithi101:> # Given a non-glob path, it better be there 2023-07-18T03:05:14.575 DEBUG:teuthology.orchestra.run.smithi101:> if "*" not in asok_path: 2023-07-18T03:05:14.575 DEBUG:teuthology.orchestra.run.smithi101:> assert(len(files) == 1) 2023-07-18T03:05:14.576 DEBUG:teuthology.orchestra.run.smithi101:> return files[0] 2023-07-18T03:05:14.576 DEBUG:teuthology.orchestra.run.smithi101:> 2023-07-18T03:05:14.577 DEBUG:teuthology.orchestra.run.smithi101:> for f in files: 2023-07-18T03:05:14.577 DEBUG:teuthology.orchestra.run.smithi101:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-18T03:05:14.578 DEBUG:teuthology.orchestra.run.smithi101:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-18T03:05:14.578 DEBUG:teuthology.orchestra.run.smithi101:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-18T03:05:14.579 DEBUG:teuthology.orchestra.run.smithi101:> contents = proc_f.read() 2023-07-18T03:05:14.580 DEBUG:teuthology.orchestra.run.smithi101:> if mountpoint in contents: 2023-07-18T03:05:14.580 DEBUG:teuthology.orchestra.run.smithi101:> return f 2023-07-18T03:05:14.581 DEBUG:teuthology.orchestra.run.smithi101:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-18T03:05:14.581 DEBUG:teuthology.orchestra.run.smithi101:> 2023-07-18T03:05:14.582 DEBUG:teuthology.orchestra.run.smithi101:> print(_find_admin_socket("client.admin")) 2023-07-18T03:05:14.582 DEBUG:teuthology.orchestra.run.smithi101:> ' 2023-07-18T03:05:14.751 INFO:teuthology.orchestra.run.smithi101.stdout:/var/run/ceph/ceph-client.admin.163692.asok 2023-07-18T03:05:14.928 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.163692.asok 2023-07-18T03:05:14.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:05:14.929 DEBUG:teuthology.orchestra.run.smithi101:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.163692.asok status 2023-07-18T03:05:15.155 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:05:15.156 INFO:teuthology.orchestra.run.smithi101.stdout: "metadata": { 2023-07-18T03:05:15.156 INFO:teuthology.orchestra.run.smithi101.stdout: "ceph_sha1": "81cf785aa7781c4e08381c540f890cb9bc987ef9", 2023-07-18T03:05:15.156 INFO:teuthology.orchestra.run.smithi101.stdout: "ceph_version": "ceph version 18.0.0-4746-g81cf785a (81cf785aa7781c4e08381c540f890cb9bc987ef9) reef (dev)", 2023-07-18T03:05:15.156 INFO:teuthology.orchestra.run.smithi101.stdout: "entity_id": "admin", 2023-07-18T03:05:15.156 INFO:teuthology.orchestra.run.smithi101.stdout: "hostname": "smithi101", 2023-07-18T03:05:15.156 INFO:teuthology.orchestra.run.smithi101.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-07-18T03:05:15.157 INFO:teuthology.orchestra.run.smithi101.stdout: "pid": "163692", 2023-07-18T03:05:15.157 INFO:teuthology.orchestra.run.smithi101.stdout: "root": "/" 2023-07-18T03:05:15.157 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:05:15.157 INFO:teuthology.orchestra.run.smithi101.stdout: "dentry_count": 0, 2023-07-18T03:05:15.157 INFO:teuthology.orchestra.run.smithi101.stdout: "dentry_pinned_count": 0, 2023-07-18T03:05:15.157 INFO:teuthology.orchestra.run.smithi101.stdout: "id": 14646, 2023-07-18T03:05:15.157 INFO:teuthology.orchestra.run.smithi101.stdout: "inst": { 2023-07-18T03:05:15.158 INFO:teuthology.orchestra.run.smithi101.stdout: "name": { 2023-07-18T03:05:15.158 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "client", 2023-07-18T03:05:15.158 INFO:teuthology.orchestra.run.smithi101.stdout: "num": 14646 2023-07-18T03:05:15.158 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:05:15.158 INFO:teuthology.orchestra.run.smithi101.stdout: "addr": { 2023-07-18T03:05:15.158 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "v1", 2023-07-18T03:05:15.158 INFO:teuthology.orchestra.run.smithi101.stdout: "addr": "192.168.0.1:0", 2023-07-18T03:05:15.158 INFO:teuthology.orchestra.run.smithi101.stdout: "nonce": 2652620693 2023-07-18T03:05:15.159 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:05:15.159 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:05:15.159 INFO:teuthology.orchestra.run.smithi101.stdout: "addr": { 2023-07-18T03:05:15.159 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "v1", 2023-07-18T03:05:15.159 INFO:teuthology.orchestra.run.smithi101.stdout: "addr": "192.168.0.1:0", 2023-07-18T03:05:15.159 INFO:teuthology.orchestra.run.smithi101.stdout: "nonce": 2652620693 2023-07-18T03:05:15.159 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:05:15.159 INFO:teuthology.orchestra.run.smithi101.stdout: "inst_str": "client.14646 192.168.0.1:0/2652620693", 2023-07-18T03:05:15.160 INFO:teuthology.orchestra.run.smithi101.stdout: "addr_str": "192.168.0.1:0/2652620693", 2023-07-18T03:05:15.160 INFO:teuthology.orchestra.run.smithi101.stdout: "inode_count": 1, 2023-07-18T03:05:15.160 INFO:teuthology.orchestra.run.smithi101.stdout: "mds_epoch": 17, 2023-07-18T03:05:15.160 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_epoch": 87, 2023-07-18T03:05:15.160 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_epoch_barrier": 0, 2023-07-18T03:05:15.160 INFO:teuthology.orchestra.run.smithi101.stdout: "blocklisted": false, 2023-07-18T03:05:15.160 INFO:teuthology.orchestra.run.smithi101.stdout: "fs_name": "cephfs" 2023-07-18T03:05:15.161 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:05:15.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:05:15.170 DEBUG:teuthology.orchestra.run.smithi101:> stat --file-system '--printf=%T 2023-07-18T03:05:15.170 DEBUG:teuthology.orchestra.run.smithi101:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-07-18T03:05:15.214 INFO:teuthology.orchestra.run.smithi101.stdout:fuseblk 2023-07-18T03:05:15.215 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-07-18T03:05:15.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:05:15.215 DEBUG:teuthology.orchestra.run.smithi101:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-07-18T03:05:15.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:05:15.325 DEBUG:teuthology.orchestra.run.smithi101:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-07-18T03:05:15.462 INFO:teuthology.orchestra.run.smithi101.stdout:# file: . 2023-07-18T03:05:15.462 INFO:teuthology.orchestra.run.smithi101.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-07-18T03:05:15.462 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:05:15.463 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-07-18T03:05:15.464 DEBUG:teuthology.orchestra.run.smithi101:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:05:15.507 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:05:15.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.760+0000 7fcf585ff700 1 -- 172.21.15.101:0/1179452502 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcf50101680 msgr2=0x7fcf50107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:15.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.760+0000 7fcf585ff700 1 --2- 172.21.15.101:0/1179452502 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcf50101680 0x7fcf50107f40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcf4800b0a0 tx=0x7fcf4800b3b0 comp rx=0 tx=0).stop 2023-07-18T03:05:15.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.760+0000 7fcf585ff700 1 -- 172.21.15.101:0/1179452502 shutdown_connections 2023-07-18T03:05:15.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.760+0000 7fcf585ff700 1 --2- 172.21.15.101:0/1179452502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf500ffb00 0x7fcf500fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:15.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.760+0000 7fcf585ff700 1 --2- 172.21.15.101:0/1179452502 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcf50101680 0x7fcf50107f40 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:15.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.760+0000 7fcf585ff700 1 --2- 172.21.15.101:0/1179452502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf50100cf0 0x7fcf50101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:15.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.760+0000 7fcf585ff700 1 -- 172.21.15.101:0/1179452502 >> 172.21.15.101:0/1179452502 conn(0x7fcf500fb0e0 msgr2=0x7fcf500fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:15.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.760+0000 7fcf585ff700 1 -- 172.21.15.101:0/1179452502 shutdown_connections 2023-07-18T03:05:15.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.760+0000 7fcf585ff700 1 -- 172.21.15.101:0/1179452502 wait complete. 2023-07-18T03:05:15.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.761+0000 7fcf585ff700 1 Processor -- start 2023-07-18T03:05:15.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.761+0000 7fcf585ff700 1 -- start start 2023-07-18T03:05:15.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.762+0000 7fcf585ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf500ffb00 0x7fcf50195900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:15.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.762+0000 7fcf585ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcf50100cf0 0x7fcf50195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:15.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.762+0000 7fcf585ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf50101680 0x7fcf5019ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:15.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.762+0000 7fcf585ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5019d480 con 0x7fcf50100cf0 2023-07-18T03:05:15.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.762+0000 7fcf585ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5019d5c0 con 0x7fcf50101680 2023-07-18T03:05:15.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.762+0000 7fcf585ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5019d750 con 0x7fcf500ffb00 2023-07-18T03:05:15.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.762+0000 7fcf55b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcf50100cf0 0x7fcf50195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:15.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.762+0000 7fcf55b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcf50100cf0 0x7fcf50195e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:52528/0 (socket says 172.21.15.101:52528) 2023-07-18T03:05:15.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.762+0000 7fcf55b9a700 1 -- 172.21.15.101:0/620561317 learned_addr learned my addr 172.21.15.101:0/620561317 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:15.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.762+0000 7fcf5639b700 1 --2- 172.21.15.101:0/620561317 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf500ffb00 0x7fcf50195900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:15.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.762+0000 7fcf56b9c700 1 --2- 172.21.15.101:0/620561317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf50101680 0x7fcf5019ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:15.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.762+0000 7fcf55b9a700 1 -- 172.21.15.101:0/620561317 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf500ffb00 msgr2=0x7fcf50195900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:15.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.762+0000 7fcf55b9a700 1 --2- 172.21.15.101:0/620561317 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf500ffb00 0x7fcf50195900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:15.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.762+0000 7fcf55b9a700 1 -- 172.21.15.101:0/620561317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf50101680 msgr2=0x7fcf5019ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:15.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.762+0000 7fcf55b9a700 1 --2- 172.21.15.101:0/620561317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf50101680 0x7fcf5019ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:15.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.762+0000 7fcf55b9a700 1 -- 172.21.15.101:0/620561317 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf40009520 con 0x7fcf50100cf0 2023-07-18T03:05:15.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.763+0000 7fcf55b9a700 1 --2- 172.21.15.101:0/620561317 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcf50100cf0 0x7fcf50195e40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcf4000eb30 tx=0x7fcf4000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:15.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.763+0000 7fcf477fe700 1 -- 172.21.15.101:0/620561317 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf40009880 con 0x7fcf50100cf0 2023-07-18T03:05:15.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.763+0000 7fcf477fe700 1 -- 172.21.15.101:0/620561317 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf400044e0 con 0x7fcf50100cf0 2023-07-18T03:05:15.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.763+0000 7fcf585ff700 1 -- 172.21.15.101:0/620561317 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf48009d00 con 0x7fcf50100cf0 2023-07-18T03:05:15.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.763+0000 7fcf477fe700 1 -- 172.21.15.101:0/620561317 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf40003d80 con 0x7fcf50100cf0 2023-07-18T03:05:15.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.763+0000 7fcf585ff700 1 -- 172.21.15.101:0/620561317 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf5019de00 con 0x7fcf50100cf0 2023-07-18T03:05:15.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.764+0000 7fcf585ff700 1 -- 172.21.15.101:0/620561317 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf50106fc0 con 0x7fcf50100cf0 2023-07-18T03:05:15.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.766+0000 7fcf477fe700 1 -- 172.21.15.101:0/620561317 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fcf40003f20 con 0x7fcf50100cf0 2023-07-18T03:05:15.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.767+0000 7fcf477fe700 1 --2- 172.21.15.101:0/620561317 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fcf3c0a1220 0x7fcf3c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:15.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.767+0000 7fcf477fe700 1 -- 172.21.15.101:0/620561317 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fcf40014070 con 0x7fcf50100cf0 2023-07-18T03:05:15.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.768+0000 7fcf5639b700 1 --2- 172.21.15.101:0/620561317 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fcf3c0a1220 0x7fcf3c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:15.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.768+0000 7fcf5639b700 1 --2- 172.21.15.101:0/620561317 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fcf3c0a1220 0x7fcf3c0a36d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcf50100b50 tx=0x7fcf4c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:15.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.771+0000 7fcf477fe700 1 -- 172.21.15.101:0/620561317 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fcf40096530 con 0x7fcf50100cf0 2023-07-18T03:05:15.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.977+0000 7fcf585ff700 1 -- 172.21.15.101:0/620561317 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcf5002c850 con 0x7fcf50100cf0 2023-07-18T03:05:15.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.979+0000 7fcf477fe700 1 -- 172.21.15.101:0/620561317 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24042 (secure 0 0 0) 0x7fcf40099b50 con 0x7fcf50100cf0 2023-07-18T03:05:15.963 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:05:15.963 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":87,"fsid":"b1782daa-2516-11ee-9b34-001a4aab830c","created":"2023-07-18T02:57:22.972483+0000","modified":"2023-07-18T03:05:04.353661+0000","last_up_change":"2023-07-18T03:04:11.429530+0000","last_in_change":"2023-07-18T03:03:54.158340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:00:43.353489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:04:57.771616+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-18T03:04:58.769590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-18T03:05:02.358830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc76b5d1-38a0-4f1a-9335-756bfc40a62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6803","nonce":3610719929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6805","nonce":3610719929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6809","nonce":3610719929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6807","nonce":3610719929}]},"public_addr":"172.21.15.101:6803/3610719929","cluster_addr":"172.21.15.101:6805/3610719929","heartbeat_back_addr":"172.21.15.101:6809/3610719929","heartbeat_front_addr":"172.21.15.101:6807/3610719929","state":["exists","up"]},{"osd":1,"uuid":"2a299573-0b34-4af9-aaae-b523e5058ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6801","nonce":1030229579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6803","nonce":1030229579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6807","nonce":1030229579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6805","nonce":1030229579}]},"public_addr":"172.21.15.155:6801/1030229579","cluster_addr":"172.21.15.155:6803/1030229579","heartbeat_back_addr":"172.21.15.155:6807/1030229579","heartbeat_front_addr":"172.21.15.155:6805/1030229579","state":["exists","up"]},{"osd":2,"uuid":"123b3749-c636-48fa-9e33-6229b5575dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6801","nonce":1349487507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6803","nonce":1349487507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6807","nonce":1349487507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6805","nonce":1349487507}]},"public_addr":"172.21.15.179:6801/1349487507","cluster_addr":"172.21.15.179:6803/1349487507","heartbeat_back_addr":"172.21.15.179:6807/1349487507","heartbeat_front_addr":"172.21.15.179:6805/1349487507","state":["exists","up"]},{"osd":3,"uuid":"4b0c409d-2954-4127-833a-b196bf226d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6811","nonce":4226577463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6813","nonce":4226577463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6817","nonce":4226577463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6815","nonce":4226577463}]},"public_addr":"172.21.15.101:6811/4226577463","cluster_addr":"172.21.15.101:6813/4226577463","heartbeat_back_addr":"172.21.15.101:6817/4226577463","heartbeat_front_addr":"172.21.15.101:6815/4226577463","state":["exists","up"]},{"osd":4,"uuid":"2137f2df-984e-45ce-baf6-b4ac12c7e3c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6809","nonce":1951135805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6811","nonce":1951135805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6815","nonce":1951135805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6813","nonce":1951135805}]},"public_addr":"172.21.15.155:6809/1951135805","cluster_addr":"172.21.15.155:6811/1951135805","heartbeat_back_addr":"172.21.15.155:6815/1951135805","heartbeat_front_addr":"172.21.15.155:6813/1951135805","state":["exists","up"]},{"osd":5,"uuid":"1020bfc5-8c2f-482e-9588-66c7f98e46e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6809","nonce":2746037021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6811","nonce":2746037021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6815","nonce":2746037021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6813","nonce":2746037021}]},"public_addr":"172.21.15.179:6809/2746037021","cluster_addr":"172.21.15.179:6811/2746037021","heartbeat_back_addr":"172.21.15.179:6815/2746037021","heartbeat_front_addr":"172.21.15.179:6813/2746037021","state":["exists","up"]},{"osd":6,"uuid":"2cd99260-5c36-4603-9338-cf697970ce58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6819","nonce":958044931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6821","nonce":958044931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6825","nonce":958044931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6823","nonce":958044931}]},"public_addr":"172.21.15.101:6819/958044931","cluster_addr":"172.21.15.101:6821/958044931","heartbeat_back_addr":"172.21.15.101:6825/958044931","heartbeat_front_addr":"172.21.15.101:6823/958044931","state":["exists","up"]},{"osd":7,"uuid":"ed23fd0a-678f-4e15-b42c-cfe873315714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6817","nonce":3256322291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6819","nonce":3256322291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6823","nonce":3256322291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6821","nonce":3256322291}]},"public_addr":"172.21.15.155:6817/3256322291","cluster_addr":"172.21.15.155:6819/3256322291","heartbeat_back_addr":"172.21.15.155:6823/3256322291","heartbeat_front_addr":"172.21.15.155:6821/3256322291","state":["exists","up"]},{"osd":8,"uuid":"eca3d8e6-7baf-47a5-b991-328001c7ebef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6817","nonce":3520612559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6819","nonce":3520612559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6823","nonce":3520612559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6821","nonce":3520612559}]},"public_addr":"172.21.15.179:6817/3520612559","cluster_addr":"172.21.15.179:6819/3520612559","heartbeat_back_addr":"172.21.15.179:6823/3520612559","heartbeat_front_addr":"172.21.15.179:6821/3520612559","state":["exists","up"]},{"osd":9,"uuid":"465a7ffb-67e4-49ef-812c-1b96feacfebf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6827","nonce":2166787667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6829","nonce":2166787667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6833","nonce":2166787667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6831","nonce":2166787667}]},"public_addr":"172.21.15.101:6827/2166787667","cluster_addr":"172.21.15.101:6829/2166787667","heartbeat_back_addr":"172.21.15.101:6833/2166787667","heartbeat_front_addr":"172.21.15.101:6831/2166787667","state":["exists","up"]},{"osd":10,"uuid":"1897a027-f25d-434f-8df5-5fc2bd866578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6825","nonce":3682446349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6827","nonce":3682446349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6831","nonce":3682446349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6829","nonce":3682446349}]},"public_addr":"172.21.15.155:6825/3682446349","cluster_addr":"172.21.15.155:6827/3682446349","heartbeat_back_addr":"172.21.15.155:6831/3682446349","heartbeat_front_addr":"172.21.15.155:6829/3682446349","state":["exists","up"]},{"osd":11,"uuid":"b62effaf-45e2-4585-b8d0-db448f3474ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6825","nonce":2602640858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6827","nonce":2602640858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6831","nonce":2602640858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6829","nonce":2602640858}]},"public_addr":"172.21.15.179:6825/2602640858","cluster_addr":"172.21.15.179:6827/2602640858","heartbeat_back_addr":"172.21.15.179:6831/2602640858","heartbeat_front_addr":"172.21.15.179:6829/2602640858","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T02:59:56.074519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:19.358142+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:41.128966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:08.227054+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:31.442592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:52.751685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:19.251054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:40.698241+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:01.339150+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:27.955910+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:49.136980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:04:10.084982+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.101:6800/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:6801/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/604228291":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/4055187849":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/2844574324":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/1603248215":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6801/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6800/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/3307578182":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/373189792":"2023-07-19T02:57:56.243200+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:05:15.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.981+0000 7fcf585ff700 1 -- 172.21.15.101:0/620561317 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fcf3c0a1220 msgr2=0x7fcf3c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:15.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.981+0000 7fcf585ff700 1 --2- 172.21.15.101:0/620561317 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fcf3c0a1220 0x7fcf3c0a36d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcf50100b50 tx=0x7fcf4c009260 comp rx=0 tx=0).stop 2023-07-18T03:05:15.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.981+0000 7fcf585ff700 1 -- 172.21.15.101:0/620561317 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcf50100cf0 msgr2=0x7fcf50195e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:15.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.981+0000 7fcf585ff700 1 --2- 172.21.15.101:0/620561317 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcf50100cf0 0x7fcf50195e40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcf4000eb30 tx=0x7fcf4000ee40 comp rx=0 tx=0).stop 2023-07-18T03:05:15.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.981+0000 7fcf585ff700 1 -- 172.21.15.101:0/620561317 shutdown_connections 2023-07-18T03:05:15.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.981+0000 7fcf585ff700 1 --2- 172.21.15.101:0/620561317 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fcf3c0a1220 0x7fcf3c0a36d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:15.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.981+0000 7fcf585ff700 1 --2- 172.21.15.101:0/620561317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf50101680 0x7fcf5019ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:15.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.981+0000 7fcf585ff700 1 --2- 172.21.15.101:0/620561317 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf500ffb00 0x7fcf50195900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:15.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.981+0000 7fcf585ff700 1 --2- 172.21.15.101:0/620561317 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcf50100cf0 0x7fcf50195e40 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:15.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.981+0000 7fcf585ff700 1 -- 172.21.15.101:0/620561317 >> 172.21.15.101:0/620561317 conn(0x7fcf500fb0e0 msgr2=0x7fcf50104100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:15.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.982+0000 7fcf585ff700 1 -- 172.21.15.101:0/620561317 shutdown_connections 2023-07-18T03:05:15.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:15.982+0000 7fcf585ff700 1 -- 172.21.15.101:0/620561317 wait complete. 2023-07-18T03:05:15.984 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:00:43.353489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-18T03:04:57.771616+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-18T03:04:58.769590+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-18T03:05:02.358830+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-18T03:05:15.985 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-18T03:05:16.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.201+0000 7fc5231ff700 1 -- 172.21.15.101:0/116201724 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc51c100ce0 msgr2=0x7fc51c101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:16.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.201+0000 7fc5231ff700 1 --2- 172.21.15.101:0/116201724 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc51c100ce0 0x7fc51c101130 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc508009990 tx=0x7fc508009ca0 comp rx=0 tx=0).stop 2023-07-18T03:05:16.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.201+0000 7fc5231ff700 1 -- 172.21.15.101:0/116201724 shutdown_connections 2023-07-18T03:05:16.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.201+0000 7fc5231ff700 1 --2- 172.21.15.101:0/116201724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc51c101670 0x7fc51c107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:16.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.201+0000 7fc5231ff700 1 --2- 172.21.15.101:0/116201724 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc51c0ffaf0 0x7fc51c0fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:16.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.201+0000 7fc5231ff700 1 --2- 172.21.15.101:0/116201724 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc51c100ce0 0x7fc51c101130 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:16.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.201+0000 7fc5231ff700 1 -- 172.21.15.101:0/116201724 >> 172.21.15.101:0/116201724 conn(0x7fc51c0fb130 msgr2=0x7fc51c0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:16.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.201+0000 7fc5231ff700 1 -- 172.21.15.101:0/116201724 shutdown_connections 2023-07-18T03:05:16.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.201+0000 7fc5231ff700 1 -- 172.21.15.101:0/116201724 wait complete. 2023-07-18T03:05:16.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.202+0000 7fc5231ff700 1 Processor -- start 2023-07-18T03:05:16.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.203+0000 7fc5231ff700 1 -- start start 2023-07-18T03:05:16.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.203+0000 7fc5231ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc51c0ffaf0 0x7fc51c195840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:16.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.203+0000 7fc5231ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc51c100ce0 0x7fc51c195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:16.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.203+0000 7fc5231ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc51c101670 0x7fc51c19cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:16.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.203+0000 7fc5231ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51c19d3c0 con 0x7fc51c0ffaf0 2023-07-18T03:05:16.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.203+0000 7fc5231ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51c19d500 con 0x7fc51c100ce0 2023-07-18T03:05:16.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.203+0000 7fc5231ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51c19d690 con 0x7fc51c101670 2023-07-18T03:05:16.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.203+0000 7fc520f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc51c0ffaf0 0x7fc51c195840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:16.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.203+0000 7fc520f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc51c0ffaf0 0x7fc51c195840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:37434/0 (socket says 172.21.15.101:37434) 2023-07-18T03:05:16.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.203+0000 7fc520f9b700 1 -- 172.21.15.101:0/1270081276 learned_addr learned my addr 172.21.15.101:0/1270081276 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:16.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.203+0000 7fc51bfff700 1 --2- 172.21.15.101:0/1270081276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc51c100ce0 0x7fc51c195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:16.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.204+0000 7fc52179c700 1 --2- 172.21.15.101:0/1270081276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc51c101670 0x7fc51c19cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:16.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.204+0000 7fc520f9b700 1 -- 172.21.15.101:0/1270081276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc51c101670 msgr2=0x7fc51c19cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:16.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.204+0000 7fc520f9b700 1 --2- 172.21.15.101:0/1270081276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc51c101670 0x7fc51c19cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:16.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.204+0000 7fc520f9b700 1 -- 172.21.15.101:0/1270081276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc51c100ce0 msgr2=0x7fc51c195d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:16.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.204+0000 7fc520f9b700 1 --2- 172.21.15.101:0/1270081276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc51c100ce0 0x7fc51c195d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:16.192 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.204+0000 7fc520f9b700 1 -- 172.21.15.101:0/1270081276 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5080095f0 con 0x7fc51c0ffaf0 2023-07-18T03:05:16.192 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.204+0000 7fc51bfff700 1 --2- 172.21.15.101:0/1270081276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc51c100ce0 0x7fc51c195d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:05:16.192 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.204+0000 7fc52179c700 1 --2- 172.21.15.101:0/1270081276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc51c101670 0x7fc51c19cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:05:16.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.204+0000 7fc520f9b700 1 --2- 172.21.15.101:0/1270081276 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc51c0ffaf0 0x7fc51c195840 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc51000e970 tx=0x7fc51000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:16.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.204+0000 7fc519ffb700 1 -- 172.21.15.101:0/1270081276 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc510009790 con 0x7fc51c0ffaf0 2023-07-18T03:05:16.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.204+0000 7fc519ffb700 1 -- 172.21.15.101:0/1270081276 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5100044e0 con 0x7fc51c0ffaf0 2023-07-18T03:05:16.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.204+0000 7fc5231ff700 1 -- 172.21.15.101:0/1270081276 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc51c19d970 con 0x7fc51c0ffaf0 2023-07-18T03:05:16.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.204+0000 7fc519ffb700 1 -- 172.21.15.101:0/1270081276 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc510003e30 con 0x7fc51c0ffaf0 2023-07-18T03:05:16.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.204+0000 7fc5231ff700 1 -- 172.21.15.101:0/1270081276 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc51c19ded0 con 0x7fc51c0ffaf0 2023-07-18T03:05:16.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:15 smithi179 ceph-mon[129229]: pgmap v283: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-07-18T03:05:16.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.205+0000 7fc5231ff700 1 -- 172.21.15.101:0/1270081276 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc51c10b310 con 0x7fc51c0ffaf0 2023-07-18T03:05:16.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.207+0000 7fc519ffb700 1 -- 172.21.15.101:0/1270081276 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fc510003f90 con 0x7fc51c0ffaf0 2023-07-18T03:05:16.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.208+0000 7fc519ffb700 1 --2- 172.21.15.101:0/1270081276 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc50c0a11d0 0x7fc50c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:16.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.208+0000 7fc519ffb700 1 -- 172.21.15.101:0/1270081276 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fc510014070 con 0x7fc51c0ffaf0 2023-07-18T03:05:16.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.209+0000 7fc51bfff700 1 --2- 172.21.15.101:0/1270081276 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc50c0a11d0 0x7fc50c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:16.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.209+0000 7fc51bfff700 1 --2- 172.21.15.101:0/1270081276 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc50c0a11d0 0x7fc50c0a3680 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc50800b2e0 tx=0x7fc508005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:16.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.212+0000 7fc519ffb700 1 -- 172.21.15.101:0/1270081276 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc510096300 con 0x7fc51c0ffaf0 2023-07-18T03:05:16.228 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:15 smithi101 ceph-mon[126379]: pgmap v283: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-07-18T03:05:16.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:15 smithi155 ceph-mon[128885]: pgmap v283: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-07-18T03:05:16.396 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.412+0000 7fc5231ff700 1 -- 172.21.15.101:0/1270081276 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc51c19db00 con 0x7fc51c0ffaf0 2023-07-18T03:05:16.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.413+0000 7fc519ffb700 1 -- 172.21.15.101:0/1270081276 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc51c19db00 con 0x7fc51c0ffaf0 2023-07-18T03:05:16.397 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 1 2023-07-18T03:05:16.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.415+0000 7fc5231ff700 1 -- 172.21.15.101:0/1270081276 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc50c0a11d0 msgr2=0x7fc50c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:16.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.415+0000 7fc5231ff700 1 --2- 172.21.15.101:0/1270081276 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc50c0a11d0 0x7fc50c0a3680 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc50800b2e0 tx=0x7fc508005fd0 comp rx=0 tx=0).stop 2023-07-18T03:05:16.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.415+0000 7fc5231ff700 1 -- 172.21.15.101:0/1270081276 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc51c0ffaf0 msgr2=0x7fc51c195840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:16.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.415+0000 7fc5231ff700 1 --2- 172.21.15.101:0/1270081276 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc51c0ffaf0 0x7fc51c195840 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc51000e970 tx=0x7fc51000ed30 comp rx=0 tx=0).stop 2023-07-18T03:05:16.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.415+0000 7fc5231ff700 1 -- 172.21.15.101:0/1270081276 shutdown_connections 2023-07-18T03:05:16.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.415+0000 7fc5231ff700 1 --2- 172.21.15.101:0/1270081276 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc50c0a11d0 0x7fc50c0a3680 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:16.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.415+0000 7fc5231ff700 1 --2- 172.21.15.101:0/1270081276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc51c101670 0x7fc51c19cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:16.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.415+0000 7fc5231ff700 1 --2- 172.21.15.101:0/1270081276 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc51c0ffaf0 0x7fc51c195840 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:16.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.415+0000 7fc5231ff700 1 --2- 172.21.15.101:0/1270081276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc51c100ce0 0x7fc51c195d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:16.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.415+0000 7fc5231ff700 1 -- 172.21.15.101:0/1270081276 >> 172.21.15.101:0/1270081276 conn(0x7fc51c0fb130 msgr2=0x7fc51c0fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:16.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.416+0000 7fc5231ff700 1 -- 172.21.15.101:0/1270081276 shutdown_connections 2023-07-18T03:05:16.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.416+0000 7fc5231ff700 1 -- 172.21.15.101:0/1270081276 wait complete. 2023-07-18T03:05:16.417 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-18T03:05:16.616 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.632+0000 7f19dc3ff700 1 -- 172.21.15.101:0/2779665171 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f19d4074550 msgr2=0x7f19d40729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:16.616 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.632+0000 7f19dc3ff700 1 --2- 172.21.15.101:0/2779665171 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f19d4074550 0x7f19d40729a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f19d0009990 tx=0x7f19d0009ca0 comp rx=0 tx=0).stop 2023-07-18T03:05:16.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.632+0000 7f19dc3ff700 1 -- 172.21.15.101:0/2779665171 shutdown_connections 2023-07-18T03:05:16.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.632+0000 7f19dc3ff700 1 --2- 172.21.15.101:0/2779665171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f19d41079b0 0x7f19d4109e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:16.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.632+0000 7f19dc3ff700 1 --2- 172.21.15.101:0/2779665171 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f19d4074550 0x7f19d40729a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:16.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.632+0000 7f19dc3ff700 1 --2- 172.21.15.101:0/2779665171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19d4072f70 0x7f19d40733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:16.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.632+0000 7f19dc3ff700 1 -- 172.21.15.101:0/2779665171 >> 172.21.15.101:0/2779665171 conn(0x7f19d40fb130 msgr2=0x7f19d40fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:16.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.632+0000 7f19dc3ff700 1 -- 172.21.15.101:0/2779665171 shutdown_connections 2023-07-18T03:05:16.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.632+0000 7f19dc3ff700 1 -- 172.21.15.101:0/2779665171 wait complete. 2023-07-18T03:05:16.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.633+0000 7f19dc3ff700 1 Processor -- start 2023-07-18T03:05:16.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.633+0000 7f19dc3ff700 1 -- start start 2023-07-18T03:05:16.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.634+0000 7f19dc3ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f19d4072f70 0x7f19d4197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:16.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.634+0000 7f19dc3ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19d4074550 0x7f19d4198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:16.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.634+0000 7f19dc3ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f19d41079b0 0x7f19d419f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:16.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.634+0000 7f19dc3ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d419f640 con 0x7f19d4072f70 2023-07-18T03:05:16.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.634+0000 7f19dc3ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d419f780 con 0x7f19d4074550 2023-07-18T03:05:16.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.634+0000 7f19dc3ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d419f910 con 0x7f19d41079b0 2023-07-18T03:05:16.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.634+0000 7f19da19b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f19d4072f70 0x7f19d4197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:16.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.634+0000 7f19da19b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f19d4072f70 0x7f19d4197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:37456/0 (socket says 172.21.15.101:37456) 2023-07-18T03:05:16.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.634+0000 7f19da19b700 1 -- 172.21.15.101:0/3900063498 learned_addr learned my addr 172.21.15.101:0/3900063498 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:16.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.634+0000 7f19da99c700 1 --2- 172.21.15.101:0/3900063498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f19d41079b0 0x7f19d419f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:16.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.634+0000 7f19d999a700 1 --2- 172.21.15.101:0/3900063498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19d4074550 0x7f19d4198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:16.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.635+0000 7f19da19b700 1 -- 172.21.15.101:0/3900063498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f19d41079b0 msgr2=0x7f19d419f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:16.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.635+0000 7f19da19b700 1 --2- 172.21.15.101:0/3900063498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f19d41079b0 0x7f19d419f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:16.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.635+0000 7f19da19b700 1 -- 172.21.15.101:0/3900063498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19d4074550 msgr2=0x7f19d4198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:16.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.635+0000 7f19da19b700 1 --2- 172.21.15.101:0/3900063498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19d4074550 0x7f19d4198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:16.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.635+0000 7f19da19b700 1 -- 172.21.15.101:0/3900063498 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d00095f0 con 0x7f19d4072f70 2023-07-18T03:05:16.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.635+0000 7f19da19b700 1 --2- 172.21.15.101:0/3900063498 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f19d4072f70 0x7f19d4197ac0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f19d000b680 tx=0x7f19d0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:16.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.635+0000 7f19cb7fe700 1 -- 172.21.15.101:0/3900063498 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d001d070 con 0x7f19d4072f70 2023-07-18T03:05:16.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.635+0000 7f19cb7fe700 1 -- 172.21.15.101:0/3900063498 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19d0021410 con 0x7f19d4072f70 2023-07-18T03:05:16.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.635+0000 7f19cb7fe700 1 -- 172.21.15.101:0/3900063498 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d0017650 con 0x7f19d4072f70 2023-07-18T03:05:16.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.635+0000 7f19dc3ff700 1 -- 172.21.15.101:0/3900063498 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19d419fb90 con 0x7f19d4072f70 2023-07-18T03:05:16.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.635+0000 7f19dc3ff700 1 -- 172.21.15.101:0/3900063498 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19d419ff60 con 0x7f19d4072f70 2023-07-18T03:05:16.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.636+0000 7f19dc3ff700 1 -- 172.21.15.101:0/3900063498 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19d4106060 con 0x7f19d4072f70 2023-07-18T03:05:16.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.638+0000 7f19cb7fe700 1 -- 172.21.15.101:0/3900063498 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f19d0021580 con 0x7f19d4072f70 2023-07-18T03:05:16.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.639+0000 7f19cb7fe700 1 --2- 172.21.15.101:0/3900063498 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f19c00a1150 0x7f19c00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:16.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.639+0000 7f19cb7fe700 1 -- 172.21.15.101:0/3900063498 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f19d00cdfb0 con 0x7f19d4072f70 2023-07-18T03:05:16.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.640+0000 7f19d999a700 1 --2- 172.21.15.101:0/3900063498 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f19c00a1150 0x7f19c00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:16.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.640+0000 7f19d999a700 1 --2- 172.21.15.101:0/3900063498 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f19c00a1150 0x7f19c00a3600 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f19d4074270 tx=0x7f19c4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:16.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.643+0000 7f19cb7fe700 1 -- 172.21.15.101:0/3900063498 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f19d0097540 con 0x7f19d4072f70 2023-07-18T03:05:16.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.843+0000 7f19dc3ff700 1 -- 172.21.15.101:0/3900063498 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f19d40611c0 con 0x7f19d4072f70 2023-07-18T03:05:16.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.844+0000 7f19cb7fe700 1 -- 172.21.15.101:0/3900063498 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f19d009ab60 con 0x7f19d4072f70 2023-07-18T03:05:16.828 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 32 2023-07-18T03:05:16.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.846+0000 7f19dc3ff700 1 -- 172.21.15.101:0/3900063498 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f19c00a1150 msgr2=0x7f19c00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:16.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.846+0000 7f19dc3ff700 1 --2- 172.21.15.101:0/3900063498 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f19c00a1150 0x7f19c00a3600 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f19d4074270 tx=0x7f19c4009260 comp rx=0 tx=0).stop 2023-07-18T03:05:16.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.846+0000 7f19dc3ff700 1 -- 172.21.15.101:0/3900063498 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f19d4072f70 msgr2=0x7f19d4197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:16.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.846+0000 7f19dc3ff700 1 --2- 172.21.15.101:0/3900063498 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f19d4072f70 0x7f19d4197ac0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f19d000b680 tx=0x7f19d0005740 comp rx=0 tx=0).stop 2023-07-18T03:05:16.832 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.846+0000 7f19dc3ff700 1 -- 172.21.15.101:0/3900063498 shutdown_connections 2023-07-18T03:05:16.832 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.846+0000 7f19dc3ff700 1 --2- 172.21.15.101:0/3900063498 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f19c00a1150 0x7f19c00a3600 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:16.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.846+0000 7f19dc3ff700 1 --2- 172.21.15.101:0/3900063498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f19d41079b0 0x7f19d419f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:16.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.846+0000 7f19dc3ff700 1 --2- 172.21.15.101:0/3900063498 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f19d4072f70 0x7f19d4197ac0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:16.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.846+0000 7f19dc3ff700 1 --2- 172.21.15.101:0/3900063498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19d4074550 0x7f19d4198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:16.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.846+0000 7f19dc3ff700 1 -- 172.21.15.101:0/3900063498 >> 172.21.15.101:0/3900063498 conn(0x7f19d40fb130 msgr2=0x7f19d40fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:16.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.846+0000 7f19dc3ff700 1 -- 172.21.15.101:0/3900063498 shutdown_connections 2023-07-18T03:05:16.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:16.847+0000 7f19dc3ff700 1 -- 172.21.15.101:0/3900063498 wait complete. 2023-07-18T03:05:16.849 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-18T03:05:17.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.066+0000 7f65dbdff700 1 -- 172.21.15.101:0/1552225818 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f65d4101670 msgr2=0x7f65d4107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:17.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.066+0000 7f65dbdff700 1 --2- 172.21.15.101:0/1552225818 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f65d4101670 0x7f65d4107f30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f65cc00b0a0 tx=0x7f65cc00b3b0 comp rx=0 tx=0).stop 2023-07-18T03:05:17.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.066+0000 7f65dbdff700 1 -- 172.21.15.101:0/1552225818 shutdown_connections 2023-07-18T03:05:17.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.066+0000 7f65dbdff700 1 --2- 172.21.15.101:0/1552225818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65d40ffaf0 0x7f65d40fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:17.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.066+0000 7f65dbdff700 1 --2- 172.21.15.101:0/1552225818 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f65d4101670 0x7f65d4107f30 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:17.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.066+0000 7f65dbdff700 1 --2- 172.21.15.101:0/1552225818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65d4100ce0 0x7f65d4101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:17.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.066+0000 7f65dbdff700 1 -- 172.21.15.101:0/1552225818 >> 172.21.15.101:0/1552225818 conn(0x7f65d40fb130 msgr2=0x7f65d40fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:17.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.066+0000 7f65dbdff700 1 -- 172.21.15.101:0/1552225818 shutdown_connections 2023-07-18T03:05:17.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.066+0000 7f65dbdff700 1 -- 172.21.15.101:0/1552225818 wait complete. 2023-07-18T03:05:17.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.067+0000 7f65dbdff700 1 Processor -- start 2023-07-18T03:05:17.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.068+0000 7f65dbdff700 1 -- start start 2023-07-18T03:05:17.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.068+0000 7f65dbdff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65d40ffaf0 0x7f65d40742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:17.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.068+0000 7f65dbdff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65d4100ce0 0x7f65d4072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:17.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.068+0000 7f65dbdff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f65d4101670 0x7f65d4072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:17.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.068+0000 7f65dbdff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d40735e0 con 0x7f65d4101670 2023-07-18T03:05:17.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.068+0000 7f65dbdff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d4073750 con 0x7f65d40ffaf0 2023-07-18T03:05:17.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.068+0000 7f65dbdff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d419f950 con 0x7f65d4100ce0 2023-07-18T03:05:17.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.068+0000 7f65da39c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f65d4101670 0x7f65d4072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:17.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.068+0000 7f65da39c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f65d4101670 0x7f65d4072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:37474/0 (socket says 172.21.15.101:37474) 2023-07-18T03:05:17.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.068+0000 7f65da39c700 1 -- 172.21.15.101:0/2185452166 learned_addr learned my addr 172.21.15.101:0/2185452166 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:17.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.068+0000 7f65d9b9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65d40ffaf0 0x7f65d40742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:17.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.068+0000 7f65d939a700 1 --2- 172.21.15.101:0/2185452166 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65d4100ce0 0x7f65d4072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:17.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.069+0000 7f65da39c700 1 -- 172.21.15.101:0/2185452166 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65d4100ce0 msgr2=0x7f65d4072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:17.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.069+0000 7f65da39c700 1 --2- 172.21.15.101:0/2185452166 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65d4100ce0 0x7f65d4072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:17.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.069+0000 7f65da39c700 1 -- 172.21.15.101:0/2185452166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65d40ffaf0 msgr2=0x7f65d40742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:17.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.069+0000 7f65da39c700 1 --2- 172.21.15.101:0/2185452166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65d40ffaf0 0x7f65d40742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:17.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.069+0000 7f65da39c700 1 -- 172.21.15.101:0/2185452166 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c4009520 con 0x7f65d4101670 2023-07-18T03:05:17.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.069+0000 7f65da39c700 1 --2- 172.21.15.101:0/2185452166 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f65d4101670 0x7f65d4072e40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f65cc00b780 tx=0x7f65cc00bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:17.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.069+0000 7f65caffd700 1 -- 172.21.15.101:0/2185452166 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65cc00e040 con 0x7f65d4101670 2023-07-18T03:05:17.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.069+0000 7f65caffd700 1 -- 172.21.15.101:0/2185452166 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f65cc004c60 con 0x7f65d4101670 2023-07-18T03:05:17.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.069+0000 7f65dbdff700 1 -- 172.21.15.101:0/2185452166 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65cc009d00 con 0x7f65d4101670 2023-07-18T03:05:17.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.069+0000 7f65caffd700 1 -- 172.21.15.101:0/2185452166 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65cc01b6b0 con 0x7f65d4101670 2023-07-18T03:05:17.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.069+0000 7f65dbdff700 1 -- 172.21.15.101:0/2185452166 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65d419fec0 con 0x7f65d4101670 2023-07-18T03:05:17.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.070+0000 7f65dbdff700 1 -- 172.21.15.101:0/2185452166 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65d410b2d0 con 0x7f65d4101670 2023-07-18T03:05:17.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.072+0000 7f65caffd700 1 -- 172.21.15.101:0/2185452166 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f65cc01d480 con 0x7f65d4101670 2023-07-18T03:05:17.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.073+0000 7f65caffd700 1 --2- 172.21.15.101:0/2185452166 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f65c00a1150 0x7f65c00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:17.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.074+0000 7f65caffd700 1 -- 172.21.15.101:0/2185452166 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f65cc0ce280 con 0x7f65d4101670 2023-07-18T03:05:17.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.074+0000 7f65d9b9b700 1 --2- 172.21.15.101:0/2185452166 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f65c00a1150 0x7f65c00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:17.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.074+0000 7f65d9b9b700 1 --2- 172.21.15.101:0/2185452166 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f65c00a1150 0x7f65c00a3600 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f65d4100b40 tx=0x7f65d0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:17.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.077+0000 7f65caffd700 1 -- 172.21.15.101:0/2185452166 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f65cc097810 con 0x7f65d4101670 2023-07-18T03:05:17.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:16 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/620561317' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:05:17.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:16 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1270081276' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:05:17.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:16 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/3900063498' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:05:17.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:16 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/620561317' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:05:17.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:16 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1270081276' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:05:17.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:16 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3900063498' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:05:17.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.276+0000 7f65dbdff700 1 -- 172.21.15.101:0/2185452166 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f65d4106970 con 0x7f65d4101670 2023-07-18T03:05:17.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.276+0000 7f65caffd700 1 -- 172.21.15.101:0/2185452166 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f65cc09ae30 con 0x7f65d4101670 2023-07-18T03:05:17.260 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 64 2023-07-18T03:05:17.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.278+0000 7f65dbdff700 1 -- 172.21.15.101:0/2185452166 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f65c00a1150 msgr2=0x7f65c00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:17.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.278+0000 7f65dbdff700 1 --2- 172.21.15.101:0/2185452166 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f65c00a1150 0x7f65c00a3600 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f65d4100b40 tx=0x7f65d0009260 comp rx=0 tx=0).stop 2023-07-18T03:05:17.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.278+0000 7f65dbdff700 1 -- 172.21.15.101:0/2185452166 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f65d4101670 msgr2=0x7f65d4072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:17.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.278+0000 7f65dbdff700 1 --2- 172.21.15.101:0/2185452166 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f65d4101670 0x7f65d4072e40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f65cc00b780 tx=0x7f65cc00bd00 comp rx=0 tx=0).stop 2023-07-18T03:05:17.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.278+0000 7f65dbdff700 1 -- 172.21.15.101:0/2185452166 shutdown_connections 2023-07-18T03:05:17.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.278+0000 7f65dbdff700 1 --2- 172.21.15.101:0/2185452166 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f65c00a1150 0x7f65c00a3600 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:17.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.278+0000 7f65dbdff700 1 --2- 172.21.15.101:0/2185452166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65d40ffaf0 0x7f65d40742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:17.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.278+0000 7f65dbdff700 1 --2- 172.21.15.101:0/2185452166 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f65d4101670 0x7f65d4072e40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:17.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.278+0000 7f65dbdff700 1 --2- 172.21.15.101:0/2185452166 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65d4100ce0 0x7f65d4072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:17.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.278+0000 7f65dbdff700 1 -- 172.21.15.101:0/2185452166 >> 172.21.15.101:0/2185452166 conn(0x7f65d40fb130 msgr2=0x7f65d40fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:17.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.279+0000 7f65dbdff700 1 -- 172.21.15.101:0/2185452166 shutdown_connections 2023-07-18T03:05:17.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.279+0000 7f65dbdff700 1 -- 172.21.15.101:0/2185452166 wait complete. 2023-07-18T03:05:17.281 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-18T03:05:17.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:16 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/620561317' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:05:17.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:16 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1270081276' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:05:17.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:16 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/3900063498' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:05:17.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.496+0000 7fd244fff700 1 -- 172.21.15.101:0/179275908 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd2400ffae0 msgr2=0x7fd2400ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:17.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.496+0000 7fd244fff700 1 --2- 172.21.15.101:0/179275908 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd2400ffae0 0x7fd2400ffef0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd230009990 tx=0x7fd230009ca0 comp rx=0 tx=0).stop 2023-07-18T03:05:17.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.497+0000 7fd244fff700 1 -- 172.21.15.101:0/179275908 shutdown_connections 2023-07-18T03:05:17.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.497+0000 7fd244fff700 1 --2- 172.21.15.101:0/179275908 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd240101660 0x7fd240107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:17.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.497+0000 7fd244fff700 1 --2- 172.21.15.101:0/179275908 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd2400ffae0 0x7fd2400ffef0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:17.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.497+0000 7fd244fff700 1 --2- 172.21.15.101:0/179275908 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd240100cd0 0x7fd240101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:17.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.497+0000 7fd244fff700 1 -- 172.21.15.101:0/179275908 >> 172.21.15.101:0/179275908 conn(0x7fd2400fb0e0 msgr2=0x7fd2400fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:17.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.497+0000 7fd244fff700 1 -- 172.21.15.101:0/179275908 shutdown_connections 2023-07-18T03:05:17.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.497+0000 7fd244fff700 1 -- 172.21.15.101:0/179275908 wait complete. 2023-07-18T03:05:17.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.498+0000 7fd244fff700 1 Processor -- start 2023-07-18T03:05:17.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.498+0000 7fd244fff700 1 -- start start 2023-07-18T03:05:17.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.498+0000 7fd244fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd2400ffae0 0x7fd2400742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:17.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.498+0000 7fd244fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd240100cd0 0x7fd240072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:17.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.498+0000 7fd244fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd240101660 0x7fd240072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:17.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.498+0000 7fd244fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2400735e0 con 0x7fd2400ffae0 2023-07-18T03:05:17.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.498+0000 7fd244fff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd240073750 con 0x7fd240100cd0 2023-07-18T03:05:17.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.498+0000 7fd244fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24019f8f0 con 0x7fd240101660 2023-07-18T03:05:17.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.498+0000 7fd23e59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd2400ffae0 0x7fd2400742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:17.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.498+0000 7fd23e59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd2400ffae0 0x7fd2400742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:37490/0 (socket says 172.21.15.101:37490) 2023-07-18T03:05:17.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.498+0000 7fd23e59c700 1 -- 172.21.15.101:0/1439015734 learned_addr learned my addr 172.21.15.101:0/1439015734 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:17.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.499+0000 7fd237fff700 1 --2- 172.21.15.101:0/1439015734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd240100cd0 0x7fd240072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:17.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.499+0000 7fd23ed9d700 1 --2- 172.21.15.101:0/1439015734 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd240101660 0x7fd240072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:17.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.499+0000 7fd23e59c700 1 -- 172.21.15.101:0/1439015734 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd240101660 msgr2=0x7fd240072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:17.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.499+0000 7fd23e59c700 1 --2- 172.21.15.101:0/1439015734 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd240101660 0x7fd240072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:17.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.499+0000 7fd23e59c700 1 -- 172.21.15.101:0/1439015734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd240100cd0 msgr2=0x7fd240072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:17.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.499+0000 7fd23e59c700 1 --2- 172.21.15.101:0/1439015734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd240100cd0 0x7fd240072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:17.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.499+0000 7fd23e59c700 1 -- 172.21.15.101:0/1439015734 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd238009c40 con 0x7fd2400ffae0 2023-07-18T03:05:17.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.499+0000 7fd23e59c700 1 --2- 172.21.15.101:0/1439015734 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd2400ffae0 0x7fd2400742c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd230005740 tx=0x7fd230009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:17.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.499+0000 7fd2377fe700 1 -- 172.21.15.101:0/1439015734 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23001d070 con 0x7fd2400ffae0 2023-07-18T03:05:17.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.499+0000 7fd2377fe700 1 -- 172.21.15.101:0/1439015734 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd230021410 con 0x7fd2400ffae0 2023-07-18T03:05:17.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.499+0000 7fd244fff700 1 -- 172.21.15.101:0/1439015734 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2300095f0 con 0x7fd2400ffae0 2023-07-18T03:05:17.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.499+0000 7fd2377fe700 1 -- 172.21.15.101:0/1439015734 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd230017650 con 0x7fd2400ffae0 2023-07-18T03:05:17.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.499+0000 7fd244fff700 1 -- 172.21.15.101:0/1439015734 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd24019fe00 con 0x7fd2400ffae0 2023-07-18T03:05:17.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.500+0000 7fd244fff700 1 -- 172.21.15.101:0/1439015734 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd240106a60 con 0x7fd2400ffae0 2023-07-18T03:05:17.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.502+0000 7fd2377fe700 1 -- 172.21.15.101:0/1439015734 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fd230021810 con 0x7fd2400ffae0 2023-07-18T03:05:17.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.504+0000 7fd2377fe700 1 --2- 172.21.15.101:0/1439015734 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd22c0a1150 0x7fd22c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:17.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.505+0000 7fd2377fe700 1 -- 172.21.15.101:0/1439015734 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd2300cdff0 con 0x7fd2400ffae0 2023-07-18T03:05:17.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.505+0000 7fd237fff700 1 --2- 172.21.15.101:0/1439015734 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd22c0a1150 0x7fd22c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:17.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.505+0000 7fd237fff700 1 --2- 172.21.15.101:0/1439015734 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd22c0a1150 0x7fd22c0a3600 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd228009bf0 tx=0x7fd228009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:17.492 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.508+0000 7fd2377fe700 1 -- 172.21.15.101:0/1439015734 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd2300975e0 con 0x7fd2400ffae0 2023-07-18T03:05:17.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.707+0000 7fd244fff700 1 -- 172.21.15.101:0/1439015734 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd2400611c0 con 0x7fd2400ffae0 2023-07-18T03:05:17.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.708+0000 7fd2377fe700 1 -- 172.21.15.101:0/1439015734 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd23009ac00 con 0x7fd2400ffae0 2023-07-18T03:05:17.692 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 32 2023-07-18T03:05:17.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.710+0000 7fd244fff700 1 -- 172.21.15.101:0/1439015734 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd22c0a1150 msgr2=0x7fd22c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:17.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.710+0000 7fd244fff700 1 --2- 172.21.15.101:0/1439015734 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd22c0a1150 0x7fd22c0a3600 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd228009bf0 tx=0x7fd228009260 comp rx=0 tx=0).stop 2023-07-18T03:05:17.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.710+0000 7fd244fff700 1 -- 172.21.15.101:0/1439015734 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd2400ffae0 msgr2=0x7fd2400742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:17.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.710+0000 7fd244fff700 1 --2- 172.21.15.101:0/1439015734 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd2400ffae0 0x7fd2400742c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd230005740 tx=0x7fd230009510 comp rx=0 tx=0).stop 2023-07-18T03:05:17.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.710+0000 7fd244fff700 1 -- 172.21.15.101:0/1439015734 shutdown_connections 2023-07-18T03:05:17.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.710+0000 7fd244fff700 1 --2- 172.21.15.101:0/1439015734 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd22c0a1150 0x7fd22c0a3600 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:17.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.710+0000 7fd244fff700 1 --2- 172.21.15.101:0/1439015734 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd240101660 0x7fd240072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:17.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.710+0000 7fd244fff700 1 --2- 172.21.15.101:0/1439015734 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd2400ffae0 0x7fd2400742c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:17.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.710+0000 7fd244fff700 1 --2- 172.21.15.101:0/1439015734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd240100cd0 0x7fd240072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:17.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.710+0000 7fd244fff700 1 -- 172.21.15.101:0/1439015734 >> 172.21.15.101:0/1439015734 conn(0x7fd2400fb0e0 msgr2=0x7fd2400fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:17.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.710+0000 7fd244fff700 1 -- 172.21.15.101:0/1439015734 shutdown_connections 2023-07-18T03:05:17.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.710+0000 7fd244fff700 1 -- 172.21.15.101:0/1439015734 wait complete. 2023-07-18T03:05:17.712 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-18T03:05:17.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.927+0000 7f3b03fff700 1 -- 172.21.15.101:0/280332211 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3afc0ffaf0 msgr2=0x7f3afc0fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:17.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.927+0000 7f3b03fff700 1 --2- 172.21.15.101:0/280332211 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3afc0ffaf0 0x7f3afc0fff00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3af8009990 tx=0x7f3af8009ca0 comp rx=0 tx=0).stop 2023-07-18T03:05:17.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.927+0000 7f3b03fff700 1 -- 172.21.15.101:0/280332211 shutdown_connections 2023-07-18T03:05:17.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.927+0000 7f3b03fff700 1 --2- 172.21.15.101:0/280332211 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3afc101670 0x7f3afc107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:17.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.927+0000 7f3b03fff700 1 --2- 172.21.15.101:0/280332211 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3afc0ffaf0 0x7f3afc0fff00 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:17.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.927+0000 7f3b03fff700 1 --2- 172.21.15.101:0/280332211 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3afc100ce0 0x7f3afc101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:17.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.927+0000 7f3b03fff700 1 -- 172.21.15.101:0/280332211 >> 172.21.15.101:0/280332211 conn(0x7f3afc0fb130 msgr2=0x7f3afc0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:17.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.927+0000 7f3b03fff700 1 -- 172.21.15.101:0/280332211 shutdown_connections 2023-07-18T03:05:17.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.927+0000 7f3b03fff700 1 -- 172.21.15.101:0/280332211 wait complete. 2023-07-18T03:05:17.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.928+0000 7f3b03fff700 1 Processor -- start 2023-07-18T03:05:17.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.928+0000 7f3b03fff700 1 -- start start 2023-07-18T03:05:17.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.929+0000 7f3b03fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3afc0ffaf0 0x7f3afc197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:17.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.929+0000 7f3b03fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3afc100ce0 0x7f3afc197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:17.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.929+0000 7f3b03fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3afc101670 0x7f3afc19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:17.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.929+0000 7f3b03fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3afc19f5e0 con 0x7f3afc101670 2023-07-18T03:05:17.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.929+0000 7f3b03fff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3afc19f720 con 0x7f3afc100ce0 2023-07-18T03:05:17.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.929+0000 7f3b03fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3afc19f8b0 con 0x7f3afc0ffaf0 2023-07-18T03:05:17.916 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.929+0000 7f3b01d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3afc0ffaf0 0x7f3afc197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:17.916 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.929+0000 7f3b01d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3afc0ffaf0 0x7f3afc197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:44498/0 (socket says 172.21.15.101:44498) 2023-07-18T03:05:17.916 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.929+0000 7f3b01d9b700 1 -- 172.21.15.101:0/1747736311 learned_addr learned my addr 172.21.15.101:0/1747736311 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:17.916 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.929+0000 7f3b0159a700 1 --2- 172.21.15.101:0/1747736311 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3afc100ce0 0x7f3afc197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:17.916 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.929+0000 7f3b0259c700 1 --2- 172.21.15.101:0/1747736311 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3afc101670 0x7f3afc19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:17.916 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.929+0000 7f3b01d9b700 1 -- 172.21.15.101:0/1747736311 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3afc100ce0 msgr2=0x7f3afc197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:17.916 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.929+0000 7f3b01d9b700 1 --2- 172.21.15.101:0/1747736311 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3afc100ce0 0x7f3afc197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:17.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.929+0000 7f3b01d9b700 1 -- 172.21.15.101:0/1747736311 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3afc101670 msgr2=0x7f3afc19f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:17.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.929+0000 7f3b01d9b700 1 --2- 172.21.15.101:0/1747736311 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3afc101670 0x7f3afc19f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:17.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.929+0000 7f3b01d9b700 1 -- 172.21.15.101:0/1747736311 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af80095f0 con 0x7f3afc0ffaf0 2023-07-18T03:05:17.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.929+0000 7f3b0259c700 1 --2- 172.21.15.101:0/1747736311 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3afc101670 0x7f3afc19f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:05:17.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.930+0000 7f3b01d9b700 1 --2- 172.21.15.101:0/1747736311 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3afc0ffaf0 0x7f3afc197a60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3af800b620 tx=0x7f3af8005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:17.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.930+0000 7f3af2ffd700 1 -- 172.21.15.101:0/1747736311 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af801d070 con 0x7f3afc0ffaf0 2023-07-18T03:05:17.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.930+0000 7f3af2ffd700 1 -- 172.21.15.101:0/1747736311 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3af8021410 con 0x7f3afc0ffaf0 2023-07-18T03:05:17.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.930+0000 7f3b03fff700 1 -- 172.21.15.101:0/1747736311 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3afc19fb30 con 0x7f3afc0ffaf0 2023-07-18T03:05:17.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.930+0000 7f3af2ffd700 1 -- 172.21.15.101:0/1747736311 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af8017650 con 0x7f3afc0ffaf0 2023-07-18T03:05:17.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.930+0000 7f3b03fff700 1 -- 172.21.15.101:0/1747736311 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3afc19ffb0 con 0x7f3afc0ffaf0 2023-07-18T03:05:17.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.931+0000 7f3b03fff700 1 -- 172.21.15.101:0/1747736311 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3afc1921d0 con 0x7f3afc0ffaf0 2023-07-18T03:05:17.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.933+0000 7f3af2ffd700 1 -- 172.21.15.101:0/1747736311 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f3af8021580 con 0x7f3afc0ffaf0 2023-07-18T03:05:17.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.934+0000 7f3af2ffd700 1 --2- 172.21.15.101:0/1747736311 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f3ae80a11d0 0x7f3ae80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:17.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.934+0000 7f3af2ffd700 1 -- 172.21.15.101:0/1747736311 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f3af80cdff0 con 0x7f3afc0ffaf0 2023-07-18T03:05:17.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.934+0000 7f3b0159a700 1 --2- 172.21.15.101:0/1747736311 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f3ae80a11d0 0x7f3ae80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:17.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.935+0000 7f3b0159a700 1 --2- 172.21.15.101:0/1747736311 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f3ae80a11d0 0x7f3ae80a3680 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3aec009c60 tx=0x7f3aec009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:17.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:17.936+0000 7f3af2ffd700 1 -- 172.21.15.101:0/1747736311 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3af80975e0 con 0x7f3afc0ffaf0 2023-07-18T03:05:17.979 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:17 smithi101 ceph-mon[126379]: pgmap v284: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-07-18T03:05:17.979 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:17 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2185452166' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:05:17.979 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:17 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1439015734' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-18T03:05:18.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.152+0000 7f3b03fff700 1 -- 172.21.15.101:0/1747736311 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3afc02c850 con 0x7f3afc0ffaf0 2023-07-18T03:05:18.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.152+0000 7f3af2ffd700 1 -- 172.21.15.101:0/1747736311 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) v1 ==== 81+0+600 (secure 0 0 0) 0x7f3af809ac00 con 0x7f3afc0ffaf0 2023-07-18T03:05:18.136 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.15.101:6800/2461037266 2023-07-19T02:58:22.762480+0000 2023-07-18T03:05:18.136 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.15.101:6801/2461037266 2023-07-19T02:58:22.762480+0000 2023-07-18T03:05:18.137 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.15.101:0/604228291 2023-07-19T02:58:22.762480+0000 2023-07-18T03:05:18.137 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.15.101:0/4055187849 2023-07-19T02:58:22.762480+0000 2023-07-18T03:05:18.137 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.15.101:0/2844574324 2023-07-19T02:57:56.243200+0000 2023-07-18T03:05:18.137 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.15.101:0/1603248215 2023-07-19T02:57:56.243200+0000 2023-07-18T03:05:18.137 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.15.101:6801/3497914514 2023-07-19T02:57:56.243200+0000 2023-07-18T03:05:18.137 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.15.101:6800/3497914514 2023-07-19T02:57:56.243200+0000 2023-07-18T03:05:18.137 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.15.101:0/3307578182 2023-07-19T02:58:22.762480+0000 2023-07-18T03:05:18.138 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.15.101:0/373189792 2023-07-19T02:57:56.243200+0000 2023-07-18T03:05:18.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.155+0000 7f3b03fff700 1 -- 172.21.15.101:0/1747736311 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f3ae80a11d0 msgr2=0x7f3ae80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:18.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.155+0000 7f3b03fff700 1 --2- 172.21.15.101:0/1747736311 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f3ae80a11d0 0x7f3ae80a3680 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3aec009c60 tx=0x7f3aec009260 comp rx=0 tx=0).stop 2023-07-18T03:05:18.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.155+0000 7f3b03fff700 1 -- 172.21.15.101:0/1747736311 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3afc0ffaf0 msgr2=0x7f3afc197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:18.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.155+0000 7f3b03fff700 1 --2- 172.21.15.101:0/1747736311 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3afc0ffaf0 0x7f3afc197a60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3af800b620 tx=0x7f3af8005740 comp rx=0 tx=0).stop 2023-07-18T03:05:18.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.155+0000 7f3b03fff700 1 -- 172.21.15.101:0/1747736311 shutdown_connections 2023-07-18T03:05:18.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.155+0000 7f3b03fff700 1 --2- 172.21.15.101:0/1747736311 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f3ae80a11d0 0x7f3ae80a3680 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:18.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.155+0000 7f3b03fff700 1 --2- 172.21.15.101:0/1747736311 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3afc101670 0x7f3afc19f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:18.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.155+0000 7f3b03fff700 1 --2- 172.21.15.101:0/1747736311 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3afc0ffaf0 0x7f3afc197a60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:18.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.155+0000 7f3b03fff700 1 --2- 172.21.15.101:0/1747736311 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3afc100ce0 0x7f3afc197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:18.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.155+0000 7f3b03fff700 1 -- 172.21.15.101:0/1747736311 >> 172.21.15.101:0/1747736311 conn(0x7f3afc0fb130 msgr2=0x7f3afc1065e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:18.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.155+0000 7f3b03fff700 1 -- 172.21.15.101:0/1747736311 shutdown_connections 2023-07-18T03:05:18.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.155+0000 7f3b03fff700 1 -- 172.21.15.101:0/1747736311 wait complete. 2023-07-18T03:05:18.141 INFO:teuthology.orchestra.run.smithi101.stderr:listed 10 entries 2023-07-18T03:05:18.156 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-07-18T03:05:18.157 DEBUG:teuthology.orchestra.run.smithi101:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:05:18.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:17 smithi179 ceph-mon[129229]: pgmap v284: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-07-18T03:05:18.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:17 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/2185452166' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:05:18.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:17 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1439015734' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-18T03:05:18.200 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:05:18.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:17 smithi155 ceph-mon[128885]: pgmap v284: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-07-18T03:05:18.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:17 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/2185452166' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:05:18.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:17 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1439015734' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-18T03:05:18.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.454+0000 7f1beb5ff700 1 -- 172.21.15.101:0/1769997597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1be4101680 msgr2=0x7f1be4107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:18.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.454+0000 7f1beb5ff700 1 --2- 172.21.15.101:0/1769997597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1be4101680 0x7f1be4107f40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc00b0a0 tx=0x7f1bdc00b3b0 comp rx=0 tx=0).stop 2023-07-18T03:05:18.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.454+0000 7f1beb5ff700 1 -- 172.21.15.101:0/1769997597 shutdown_connections 2023-07-18T03:05:18.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.454+0000 7f1beb5ff700 1 --2- 172.21.15.101:0/1769997597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1be40ffb00 0x7f1be40fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:18.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.454+0000 7f1beb5ff700 1 --2- 172.21.15.101:0/1769997597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1be4101680 0x7f1be4107f40 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:18.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.454+0000 7f1beb5ff700 1 --2- 172.21.15.101:0/1769997597 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1be4100cf0 0x7f1be4101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:18.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.454+0000 7f1beb5ff700 1 -- 172.21.15.101:0/1769997597 >> 172.21.15.101:0/1769997597 conn(0x7f1be40fb0e0 msgr2=0x7f1be40fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:18.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.454+0000 7f1beb5ff700 1 -- 172.21.15.101:0/1769997597 shutdown_connections 2023-07-18T03:05:18.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.454+0000 7f1beb5ff700 1 -- 172.21.15.101:0/1769997597 wait complete. 2023-07-18T03:05:18.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.455+0000 7f1beb5ff700 1 Processor -- start 2023-07-18T03:05:18.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.455+0000 7f1beb5ff700 1 -- start start 2023-07-18T03:05:18.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.456+0000 7f1beb5ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1be40ffb00 0x7f1be4195750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:18.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.456+0000 7f1beb5ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1be4100cf0 0x7f1be4195c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:18.442 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.456+0000 7f1beb5ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1be4101680 0x7f1be419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:18.442 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.456+0000 7f1beb5ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be419d2d0 con 0x7f1be4101680 2023-07-18T03:05:18.442 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.456+0000 7f1beb5ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be419d410 con 0x7f1be4100cf0 2023-07-18T03:05:18.442 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.456+0000 7f1beb5ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be419d5a0 con 0x7f1be40ffb00 2023-07-18T03:05:18.442 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.456+0000 7f1be9b9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1be4101680 0x7f1be419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:18.442 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.456+0000 7f1be939b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1be40ffb00 0x7f1be4195750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:18.442 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.456+0000 7f1be939b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1be40ffb00 0x7f1be4195750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:44524/0 (socket says 172.21.15.101:44524) 2023-07-18T03:05:18.442 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.456+0000 7f1be939b700 1 -- 172.21.15.101:0/593526519 learned_addr learned my addr 172.21.15.101:0/593526519 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:18.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.456+0000 7f1be8b9a700 1 --2- 172.21.15.101:0/593526519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1be4100cf0 0x7f1be4195c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:18.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.456+0000 7f1be939b700 1 -- 172.21.15.101:0/593526519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1be4100cf0 msgr2=0x7f1be4195c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:18.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.456+0000 7f1be939b700 1 --2- 172.21.15.101:0/593526519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1be4100cf0 0x7f1be4195c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:18.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.456+0000 7f1be939b700 1 -- 172.21.15.101:0/593526519 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1be4101680 msgr2=0x7f1be419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:18.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.456+0000 7f1be939b700 1 --2- 172.21.15.101:0/593526519 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1be4101680 0x7f1be419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:18.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.456+0000 7f1be939b700 1 -- 172.21.15.101:0/593526519 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd4009520 con 0x7f1be40ffb00 2023-07-18T03:05:18.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.456+0000 7f1be9b9c700 1 --2- 172.21.15.101:0/593526519 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1be4101680 0x7f1be419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:05:18.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.457+0000 7f1be939b700 1 --2- 172.21.15.101:0/593526519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1be40ffb00 0x7f1be4195750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1be000e970 tx=0x7f1be000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:18.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.457+0000 7f1bda7fc700 1 -- 172.21.15.101:0/593526519 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be0009790 con 0x7f1be40ffb00 2023-07-18T03:05:18.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.457+0000 7f1bda7fc700 1 -- 172.21.15.101:0/593526519 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1be000c7e0 con 0x7f1be40ffb00 2023-07-18T03:05:18.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.457+0000 7f1beb5ff700 1 -- 172.21.15.101:0/593526519 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bdc009d00 con 0x7f1be40ffb00 2023-07-18T03:05:18.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.457+0000 7f1bda7fc700 1 -- 172.21.15.101:0/593526519 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be0003c70 con 0x7f1be40ffb00 2023-07-18T03:05:18.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.457+0000 7f1beb5ff700 1 -- 172.21.15.101:0/593526519 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1be419dc00 con 0x7f1be40ffb00 2023-07-18T03:05:18.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.458+0000 7f1beb5ff700 1 -- 172.21.15.101:0/593526519 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1be410b270 con 0x7f1be40ffb00 2023-07-18T03:05:18.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.460+0000 7f1bda7fc700 1 -- 172.21.15.101:0/593526519 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f1be000cc50 con 0x7f1be40ffb00 2023-07-18T03:05:18.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.461+0000 7f1bda7fc700 1 --2- 172.21.15.101:0/593526519 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f1bd00a11a0 0x7f1bd00a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:18.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.461+0000 7f1bda7fc700 1 -- 172.21.15.101:0/593526519 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1be0014070 con 0x7f1be40ffb00 2023-07-18T03:05:18.446 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.462+0000 7f1be8b9a700 1 --2- 172.21.15.101:0/593526519 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f1bd00a11a0 0x7f1bd00a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:18.446 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.462+0000 7f1be8b9a700 1 --2- 172.21.15.101:0/593526519 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f1bd00a11a0 0x7f1bd00a3650 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4009d70 tx=0x7f1bd4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:18.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.463+0000 7f1bda7fc700 1 -- 172.21.15.101:0/593526519 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1be0096350 con 0x7f1be40ffb00 2023-07-18T03:05:18.647 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.664+0000 7f1beb5ff700 1 -- 172.21.15.101:0/593526519 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1be4106960 con 0x7f1be40ffb00 2023-07-18T03:05:18.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.666+0000 7f1bda7fc700 1 -- 172.21.15.101:0/593526519 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24042 (secure 0 0 0) 0x7f1be0003dd0 con 0x7f1be40ffb00 2023-07-18T03:05:18.650 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:05:18.650 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":87,"fsid":"b1782daa-2516-11ee-9b34-001a4aab830c","created":"2023-07-18T02:57:22.972483+0000","modified":"2023-07-18T03:05:04.353661+0000","last_up_change":"2023-07-18T03:04:11.429530+0000","last_in_change":"2023-07-18T03:03:54.158340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:00:43.353489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:04:57.771616+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-18T03:04:58.769590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-18T03:05:02.358830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc76b5d1-38a0-4f1a-9335-756bfc40a62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6803","nonce":3610719929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6805","nonce":3610719929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6809","nonce":3610719929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6807","nonce":3610719929}]},"public_addr":"172.21.15.101:6803/3610719929","cluster_addr":"172.21.15.101:6805/3610719929","heartbeat_back_addr":"172.21.15.101:6809/3610719929","heartbeat_front_addr":"172.21.15.101:6807/3610719929","state":["exists","up"]},{"osd":1,"uuid":"2a299573-0b34-4af9-aaae-b523e5058ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6801","nonce":1030229579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6803","nonce":1030229579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6807","nonce":1030229579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6805","nonce":1030229579}]},"public_addr":"172.21.15.155:6801/1030229579","cluster_addr":"172.21.15.155:6803/1030229579","heartbeat_back_addr":"172.21.15.155:6807/1030229579","heartbeat_front_addr":"172.21.15.155:6805/1030229579","state":["exists","up"]},{"osd":2,"uuid":"123b3749-c636-48fa-9e33-6229b5575dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6801","nonce":1349487507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6803","nonce":1349487507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6807","nonce":1349487507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6805","nonce":1349487507}]},"public_addr":"172.21.15.179:6801/1349487507","cluster_addr":"172.21.15.179:6803/1349487507","heartbeat_back_addr":"172.21.15.179:6807/1349487507","heartbeat_front_addr":"172.21.15.179:6805/1349487507","state":["exists","up"]},{"osd":3,"uuid":"4b0c409d-2954-4127-833a-b196bf226d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6811","nonce":4226577463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6813","nonce":4226577463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6817","nonce":4226577463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6815","nonce":4226577463}]},"public_addr":"172.21.15.101:6811/4226577463","cluster_addr":"172.21.15.101:6813/4226577463","heartbeat_back_addr":"172.21.15.101:6817/4226577463","heartbeat_front_addr":"172.21.15.101:6815/4226577463","state":["exists","up"]},{"osd":4,"uuid":"2137f2df-984e-45ce-baf6-b4ac12c7e3c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6809","nonce":1951135805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6811","nonce":1951135805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6815","nonce":1951135805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6813","nonce":1951135805}]},"public_addr":"172.21.15.155:6809/1951135805","cluster_addr":"172.21.15.155:6811/1951135805","heartbeat_back_addr":"172.21.15.155:6815/1951135805","heartbeat_front_addr":"172.21.15.155:6813/1951135805","state":["exists","up"]},{"osd":5,"uuid":"1020bfc5-8c2f-482e-9588-66c7f98e46e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6809","nonce":2746037021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6811","nonce":2746037021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6815","nonce":2746037021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6813","nonce":2746037021}]},"public_addr":"172.21.15.179:6809/2746037021","cluster_addr":"172.21.15.179:6811/2746037021","heartbeat_back_addr":"172.21.15.179:6815/2746037021","heartbeat_front_addr":"172.21.15.179:6813/2746037021","state":["exists","up"]},{"osd":6,"uuid":"2cd99260-5c36-4603-9338-cf697970ce58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6819","nonce":958044931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6821","nonce":958044931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6825","nonce":958044931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6823","nonce":958044931}]},"public_addr":"172.21.15.101:6819/958044931","cluster_addr":"172.21.15.101:6821/958044931","heartbeat_back_addr":"172.21.15.101:6825/958044931","heartbeat_front_addr":"172.21.15.101:6823/958044931","state":["exists","up"]},{"osd":7,"uuid":"ed23fd0a-678f-4e15-b42c-cfe873315714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6817","nonce":3256322291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6819","nonce":3256322291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6823","nonce":3256322291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6821","nonce":3256322291}]},"public_addr":"172.21.15.155:6817/3256322291","cluster_addr":"172.21.15.155:6819/3256322291","heartbeat_back_addr":"172.21.15.155:6823/3256322291","heartbeat_front_addr":"172.21.15.155:6821/3256322291","state":["exists","up"]},{"osd":8,"uuid":"eca3d8e6-7baf-47a5-b991-328001c7ebef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6817","nonce":3520612559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6819","nonce":3520612559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6823","nonce":3520612559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6821","nonce":3520612559}]},"public_addr":"172.21.15.179:6817/3520612559","cluster_addr":"172.21.15.179:6819/3520612559","heartbeat_back_addr":"172.21.15.179:6823/3520612559","heartbeat_front_addr":"172.21.15.179:6821/3520612559","state":["exists","up"]},{"osd":9,"uuid":"465a7ffb-67e4-49ef-812c-1b96feacfebf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6827","nonce":2166787667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6829","nonce":2166787667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6833","nonce":2166787667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6831","nonce":2166787667}]},"public_addr":"172.21.15.101:6827/2166787667","cluster_addr":"172.21.15.101:6829/2166787667","heartbeat_back_addr":"172.21.15.101:6833/2166787667","heartbeat_front_addr":"172.21.15.101:6831/2166787667","state":["exists","up"]},{"osd":10,"uuid":"1897a027-f25d-434f-8df5-5fc2bd866578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6825","nonce":3682446349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6827","nonce":3682446349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6831","nonce":3682446349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6829","nonce":3682446349}]},"public_addr":"172.21.15.155:6825/3682446349","cluster_addr":"172.21.15.155:6827/3682446349","heartbeat_back_addr":"172.21.15.155:6831/3682446349","heartbeat_front_addr":"172.21.15.155:6829/3682446349","state":["exists","up"]},{"osd":11,"uuid":"b62effaf-45e2-4585-b8d0-db448f3474ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6825","nonce":2602640858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6827","nonce":2602640858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6831","nonce":2602640858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6829","nonce":2602640858}]},"public_addr":"172.21.15.179:6825/2602640858","cluster_addr":"172.21.15.179:6827/2602640858","heartbeat_back_addr":"172.21.15.179:6831/2602640858","heartbeat_front_addr":"172.21.15.179:6829/2602640858","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T02:59:56.074519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:19.358142+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:41.128966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:08.227054+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:31.442592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:52.751685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:19.251054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:40.698241+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:01.339150+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:27.955910+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:49.136980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:04:10.084982+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.101:6800/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:6801/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/604228291":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/4055187849":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/2844574324":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/1603248215":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6801/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6800/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/3307578182":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/373189792":"2023-07-19T02:57:56.243200+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:05:18.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.668+0000 7f1beb5ff700 1 -- 172.21.15.101:0/593526519 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f1bd00a11a0 msgr2=0x7f1bd00a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:18.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.668+0000 7f1beb5ff700 1 --2- 172.21.15.101:0/593526519 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f1bd00a11a0 0x7f1bd00a3650 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4009d70 tx=0x7f1bd4009260 comp rx=0 tx=0).stop 2023-07-18T03:05:18.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.668+0000 7f1beb5ff700 1 -- 172.21.15.101:0/593526519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1be40ffb00 msgr2=0x7f1be4195750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:18.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.668+0000 7f1beb5ff700 1 --2- 172.21.15.101:0/593526519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1be40ffb00 0x7f1be4195750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1be000e970 tx=0x7f1be000ec80 comp rx=0 tx=0).stop 2023-07-18T03:05:18.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.668+0000 7f1beb5ff700 1 -- 172.21.15.101:0/593526519 shutdown_connections 2023-07-18T03:05:18.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.668+0000 7f1beb5ff700 1 --2- 172.21.15.101:0/593526519 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f1bd00a11a0 0x7f1bd00a3650 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:18.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.668+0000 7f1beb5ff700 1 --2- 172.21.15.101:0/593526519 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1be4101680 0x7f1be419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:18.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.668+0000 7f1beb5ff700 1 --2- 172.21.15.101:0/593526519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1be40ffb00 0x7f1be4195750 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:18.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.668+0000 7f1beb5ff700 1 --2- 172.21.15.101:0/593526519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1be4100cf0 0x7f1be4195c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:18.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.668+0000 7f1beb5ff700 1 -- 172.21.15.101:0/593526519 >> 172.21.15.101:0/593526519 conn(0x7f1be40fb0e0 msgr2=0x7f1be40fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:18.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.668+0000 7f1beb5ff700 1 -- 172.21.15.101:0/593526519 shutdown_connections 2023-07-18T03:05:18.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.668+0000 7f1beb5ff700 1 -- 172.21.15.101:0/593526519 wait complete. 2023-07-18T03:05:18.670 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:00:43.353489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-18T03:04:57.771616+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-18T03:04:58.769590+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-18T03:05:02.358830+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-18T03:05:18.671 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-18T03:05:18.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.888+0000 7f1f915ff700 1 -- 172.21.15.101:0/1619397501 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1f8c0ffaf0 msgr2=0x7f1f8c0fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:18.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.888+0000 7f1f915ff700 1 --2- 172.21.15.101:0/1619397501 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1f8c0ffaf0 0x7f1f8c0fff00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c009990 tx=0x7f1f7c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:05:18.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.888+0000 7f1f915ff700 1 -- 172.21.15.101:0/1619397501 shutdown_connections 2023-07-18T03:05:18.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.888+0000 7f1f915ff700 1 --2- 172.21.15.101:0/1619397501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f8c101670 0x7f1f8c107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:18.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.888+0000 7f1f915ff700 1 --2- 172.21.15.101:0/1619397501 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1f8c0ffaf0 0x7f1f8c0fff00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:18.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.888+0000 7f1f915ff700 1 --2- 172.21.15.101:0/1619397501 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f8c100ce0 0x7f1f8c101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:18.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.888+0000 7f1f915ff700 1 -- 172.21.15.101:0/1619397501 >> 172.21.15.101:0/1619397501 conn(0x7f1f8c0fb130 msgr2=0x7f1f8c0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:18.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.888+0000 7f1f915ff700 1 -- 172.21.15.101:0/1619397501 shutdown_connections 2023-07-18T03:05:18.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.888+0000 7f1f915ff700 1 -- 172.21.15.101:0/1619397501 wait complete. 2023-07-18T03:05:18.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.889+0000 7f1f915ff700 1 Processor -- start 2023-07-18T03:05:18.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.889+0000 7f1f915ff700 1 -- start start 2023-07-18T03:05:18.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.890+0000 7f1f915ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f8c0ffaf0 0x7f1f8c197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:18.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.890+0000 7f1f915ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f8c100ce0 0x7f1f8c197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:18.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.890+0000 7f1f915ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1f8c101670 0x7f1f8c19f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:18.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.890+0000 7f1f915ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8c19f620 con 0x7f1f8c101670 2023-07-18T03:05:18.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.890+0000 7f1f915ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8c19f760 con 0x7f1f8c100ce0 2023-07-18T03:05:18.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.890+0000 7f1f915ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8c19f8f0 con 0x7f1f8c0ffaf0 2023-07-18T03:05:18.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.890+0000 7f1f8b7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1f8c101670 0x7f1f8c19f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:18.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.890+0000 7f1f8b7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1f8c101670 0x7f1f8c19f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:37572/0 (socket says 172.21.15.101:37572) 2023-07-18T03:05:18.877 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.890+0000 7f1f8b7fe700 1 -- 172.21.15.101:0/2085860986 learned_addr learned my addr 172.21.15.101:0/2085860986 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:18.877 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.890+0000 7f1f8affd700 1 --2- 172.21.15.101:0/2085860986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f8c0ffaf0 0x7f1f8c197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:18.877 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.890+0000 7f1f8a7fc700 1 --2- 172.21.15.101:0/2085860986 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f8c100ce0 0x7f1f8c197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:18.877 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.890+0000 7f1f8b7fe700 1 -- 172.21.15.101:0/2085860986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f8c0ffaf0 msgr2=0x7f1f8c197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:18.877 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.890+0000 7f1f8b7fe700 1 --2- 172.21.15.101:0/2085860986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f8c0ffaf0 0x7f1f8c197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:18.877 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.890+0000 7f1f8b7fe700 1 -- 172.21.15.101:0/2085860986 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f8c100ce0 msgr2=0x7f1f8c197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:18.877 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.890+0000 7f1f8b7fe700 1 --2- 172.21.15.101:0/2085860986 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f8c100ce0 0x7f1f8c197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:18.877 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.890+0000 7f1f8b7fe700 1 -- 172.21.15.101:0/2085860986 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f7c0095f0 con 0x7f1f8c101670 2023-07-18T03:05:18.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.891+0000 7f1f8b7fe700 1 --2- 172.21.15.101:0/2085860986 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1f8c101670 0x7f1f8c19f050 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1f84007ee0 tx=0x7f1f8400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:18.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.891+0000 7f1f73fff700 1 -- 172.21.15.101:0/2085860986 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8400dd20 con 0x7f1f8c101670 2023-07-18T03:05:18.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.891+0000 7f1f73fff700 1 -- 172.21.15.101:0/2085860986 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f8400f040 con 0x7f1f8c101670 2023-07-18T03:05:18.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.891+0000 7f1f915ff700 1 -- 172.21.15.101:0/2085860986 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f8c19fbd0 con 0x7f1f8c101670 2023-07-18T03:05:18.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.891+0000 7f1f73fff700 1 -- 172.21.15.101:0/2085860986 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f840048a0 con 0x7f1f8c101670 2023-07-18T03:05:18.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.891+0000 7f1f915ff700 1 -- 172.21.15.101:0/2085860986 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f8c1a0130 con 0x7f1f8c101670 2023-07-18T03:05:18.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.892+0000 7f1f915ff700 1 -- 172.21.15.101:0/2085860986 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f8c10b310 con 0x7f1f8c101670 2023-07-18T03:05:18.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.894+0000 7f1f73fff700 1 -- 172.21.15.101:0/2085860986 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f1f84003a00 con 0x7f1f8c101670 2023-07-18T03:05:18.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.895+0000 7f1f73fff700 1 --2- 172.21.15.101:0/2085860986 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f1f740a11d0 0x7f1f740a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:18.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.895+0000 7f1f73fff700 1 -- 172.21.15.101:0/2085860986 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1f840cc2f0 con 0x7f1f8c101670 2023-07-18T03:05:18.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.895+0000 7f1f8affd700 1 --2- 172.21.15.101:0/2085860986 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f1f740a11d0 0x7f1f740a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:18.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.896+0000 7f1f8affd700 1 --2- 172.21.15.101:0/2085860986 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f1f740a11d0 0x7f1f740a3680 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c00b5c0 tx=0x7f1f7c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:18.882 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:18.899+0000 7f1f73fff700 1 -- 172.21.15.101:0/2085860986 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1f8404e790 con 0x7f1f8c101670 2023-07-18T03:05:18.978 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:18 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1747736311' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-18T03:05:18.979 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:18 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/593526519' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:05:19.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.098+0000 7f1f915ff700 1 -- 172.21.15.101:0/2085860986 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1f8c1a06e0 con 0x7f1f8c101670 2023-07-18T03:05:19.083 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.098+0000 7f1f73fff700 1 -- 172.21.15.101:0/2085860986 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1f840957f0 con 0x7f1f8c101670 2023-07-18T03:05:19.083 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 1 2023-07-18T03:05:19.085 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.101+0000 7f1f915ff700 1 -- 172.21.15.101:0/2085860986 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f1f740a11d0 msgr2=0x7f1f740a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:19.085 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.101+0000 7f1f915ff700 1 --2- 172.21.15.101:0/2085860986 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f1f740a11d0 0x7f1f740a3680 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c00b5c0 tx=0x7f1f7c005fd0 comp rx=0 tx=0).stop 2023-07-18T03:05:19.085 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.101+0000 7f1f915ff700 1 -- 172.21.15.101:0/2085860986 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1f8c101670 msgr2=0x7f1f8c19f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:19.085 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.101+0000 7f1f915ff700 1 --2- 172.21.15.101:0/2085860986 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1f8c101670 0x7f1f8c19f050 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1f84007ee0 tx=0x7f1f8400d3b0 comp rx=0 tx=0).stop 2023-07-18T03:05:19.085 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.101+0000 7f1f915ff700 1 -- 172.21.15.101:0/2085860986 shutdown_connections 2023-07-18T03:05:19.085 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.101+0000 7f1f915ff700 1 --2- 172.21.15.101:0/2085860986 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f1f740a11d0 0x7f1f740a3680 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:19.086 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.101+0000 7f1f915ff700 1 --2- 172.21.15.101:0/2085860986 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1f8c101670 0x7f1f8c19f050 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:19.086 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.101+0000 7f1f915ff700 1 --2- 172.21.15.101:0/2085860986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f8c0ffaf0 0x7f1f8c197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:19.086 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.101+0000 7f1f915ff700 1 --2- 172.21.15.101:0/2085860986 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f8c100ce0 0x7f1f8c197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:19.086 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.101+0000 7f1f915ff700 1 -- 172.21.15.101:0/2085860986 >> 172.21.15.101:0/2085860986 conn(0x7f1f8c0fb130 msgr2=0x7f1f8c0fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:19.086 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.101+0000 7f1f915ff700 1 -- 172.21.15.101:0/2085860986 shutdown_connections 2023-07-18T03:05:19.086 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.101+0000 7f1f915ff700 1 -- 172.21.15.101:0/2085860986 wait complete. 2023-07-18T03:05:19.102 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-18T03:05:19.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:18 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1747736311' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-18T03:05:19.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:18 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/593526519' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:05:19.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:18 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1747736311' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-18T03:05:19.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:18 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/593526519' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:05:19.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.317+0000 7f79f59ff700 1 -- 172.21.15.101:0/1570344753 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f79f0100ce0 msgr2=0x7f79f0101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:19.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.317+0000 7f79f59ff700 1 --2- 172.21.15.101:0/1570344753 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f79f0100ce0 0x7f79f0101130 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f79d8009990 tx=0x7f79d8009ca0 comp rx=0 tx=0).stop 2023-07-18T03:05:19.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.318+0000 7f79f59ff700 1 -- 172.21.15.101:0/1570344753 shutdown_connections 2023-07-18T03:05:19.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.318+0000 7f79f59ff700 1 --2- 172.21.15.101:0/1570344753 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79f00ffaf0 0x7f79f00fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:19.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.318+0000 7f79f59ff700 1 --2- 172.21.15.101:0/1570344753 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79f0101670 0x7f79f0107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:19.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.318+0000 7f79f59ff700 1 --2- 172.21.15.101:0/1570344753 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f79f0100ce0 0x7f79f0101130 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:19.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.318+0000 7f79f59ff700 1 -- 172.21.15.101:0/1570344753 >> 172.21.15.101:0/1570344753 conn(0x7f79f00fb130 msgr2=0x7f79f00fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:19.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.318+0000 7f79f59ff700 1 -- 172.21.15.101:0/1570344753 shutdown_connections 2023-07-18T03:05:19.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.318+0000 7f79f59ff700 1 -- 172.21.15.101:0/1570344753 wait complete. 2023-07-18T03:05:19.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.319+0000 7f79f59ff700 1 Processor -- start 2023-07-18T03:05:19.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.319+0000 7f79f59ff700 1 -- start start 2023-07-18T03:05:19.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.319+0000 7f79f59ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79f00ffaf0 0x7f79f0197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:19.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.319+0000 7f79f59ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f79f0100ce0 0x7f79f0197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:19.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.319+0000 7f79f59ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79f0101670 0x7f79f019f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:19.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.319+0000 7f79f59ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f019f5f0 con 0x7f79f0100ce0 2023-07-18T03:05:19.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.319+0000 7f79f59ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f019f730 con 0x7f79f0101670 2023-07-18T03:05:19.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.319+0000 7f79f59ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f019f8c0 con 0x7f79f00ffaf0 2023-07-18T03:05:19.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.319+0000 7f79e67fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f79f0100ce0 0x7f79f0197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:19.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.320+0000 7f79eeffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79f00ffaf0 0x7f79f0197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:19.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.320+0000 7f79eeffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79f00ffaf0 0x7f79f0197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:44564/0 (socket says 172.21.15.101:44564) 2023-07-18T03:05:19.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.320+0000 7f79eeffd700 1 -- 172.21.15.101:0/97925476 learned_addr learned my addr 172.21.15.101:0/97925476 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:19.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.320+0000 7f79e67fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f79f0100ce0 0x7f79f0197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:37588/0 (socket says 172.21.15.101:37588) 2023-07-18T03:05:19.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.320+0000 7f79ef7fe700 1 --2- 172.21.15.101:0/97925476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79f0101670 0x7f79f019f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:19.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.320+0000 7f79e67fc700 1 -- 172.21.15.101:0/97925476 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79f00ffaf0 msgr2=0x7f79f0197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:19.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.320+0000 7f79e67fc700 1 --2- 172.21.15.101:0/97925476 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79f00ffaf0 0x7f79f0197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:19.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.320+0000 7f79e67fc700 1 -- 172.21.15.101:0/97925476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79f0101670 msgr2=0x7f79f019f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:19.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.320+0000 7f79e67fc700 1 --2- 172.21.15.101:0/97925476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79f0101670 0x7f79f019f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:19.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.320+0000 7f79e67fc700 1 -- 172.21.15.101:0/97925476 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d80095f0 con 0x7f79f0100ce0 2023-07-18T03:05:19.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.320+0000 7f79e67fc700 1 --2- 172.21.15.101:0/97925476 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f79f0100ce0 0x7f79f0197fb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f79d8000c00 tx=0x7f79d8005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:19.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.320+0000 7f79f49fd700 1 -- 172.21.15.101:0/97925476 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d801d070 con 0x7f79f0100ce0 2023-07-18T03:05:19.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.320+0000 7f79f49fd700 1 -- 172.21.15.101:0/97925476 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79d8003f60 con 0x7f79f0100ce0 2023-07-18T03:05:19.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.320+0000 7f79f59ff700 1 -- 172.21.15.101:0/97925476 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79f019fb40 con 0x7f79f0100ce0 2023-07-18T03:05:19.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.320+0000 7f79f49fd700 1 -- 172.21.15.101:0/97925476 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d8022b40 con 0x7f79f0100ce0 2023-07-18T03:05:19.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.321+0000 7f79f59ff700 1 -- 172.21.15.101:0/97925476 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f019ffc0 con 0x7f79f0100ce0 2023-07-18T03:05:19.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.322+0000 7f79f59ff700 1 -- 172.21.15.101:0/97925476 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79f010b310 con 0x7f79f0100ce0 2023-07-18T03:05:19.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.324+0000 7f79f49fd700 1 -- 172.21.15.101:0/97925476 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f79d8003aa0 con 0x7f79f0100ce0 2023-07-18T03:05:19.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.325+0000 7f79f49fd700 1 --2- 172.21.15.101:0/97925476 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f79dc0a11d0 0x7f79dc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:19.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.325+0000 7f79f49fd700 1 -- 172.21.15.101:0/97925476 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f79d80ce1c0 con 0x7f79f0100ce0 2023-07-18T03:05:19.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.325+0000 7f79eeffd700 1 --2- 172.21.15.101:0/97925476 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f79dc0a11d0 0x7f79dc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:19.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.326+0000 7f79eeffd700 1 --2- 172.21.15.101:0/97925476 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f79dc0a11d0 0x7f79dc0a3680 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f79e0007cf0 tx=0x7f79e0007480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:19.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.328+0000 7f79f49fd700 1 -- 172.21.15.101:0/97925476 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f79d8097870 con 0x7f79f0100ce0 2023-07-18T03:05:19.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.527+0000 7f79f59ff700 1 -- 172.21.15.101:0/97925476 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f79f01a0570 con 0x7f79f0100ce0 2023-07-18T03:05:19.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.528+0000 7f79f49fd700 1 -- 172.21.15.101:0/97925476 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f79d809ae90 con 0x7f79f0100ce0 2023-07-18T03:05:19.512 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 32 2023-07-18T03:05:19.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.530+0000 7f79f59ff700 1 -- 172.21.15.101:0/97925476 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f79dc0a11d0 msgr2=0x7f79dc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:19.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.530+0000 7f79f59ff700 1 --2- 172.21.15.101:0/97925476 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f79dc0a11d0 0x7f79dc0a3680 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f79e0007cf0 tx=0x7f79e0007480 comp rx=0 tx=0).stop 2023-07-18T03:05:19.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.530+0000 7f79f59ff700 1 -- 172.21.15.101:0/97925476 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f79f0100ce0 msgr2=0x7f79f0197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:19.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.530+0000 7f79f59ff700 1 --2- 172.21.15.101:0/97925476 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f79f0100ce0 0x7f79f0197fb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f79d8000c00 tx=0x7f79d8005740 comp rx=0 tx=0).stop 2023-07-18T03:05:19.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.530+0000 7f79f59ff700 1 -- 172.21.15.101:0/97925476 shutdown_connections 2023-07-18T03:05:19.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.530+0000 7f79f59ff700 1 --2- 172.21.15.101:0/97925476 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f79dc0a11d0 0x7f79dc0a3680 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:19.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.530+0000 7f79f59ff700 1 --2- 172.21.15.101:0/97925476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79f0101670 0x7f79f019f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:19.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.530+0000 7f79f59ff700 1 --2- 172.21.15.101:0/97925476 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79f00ffaf0 0x7f79f0197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:19.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.530+0000 7f79f59ff700 1 --2- 172.21.15.101:0/97925476 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f79f0100ce0 0x7f79f0197fb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:19.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.530+0000 7f79f59ff700 1 -- 172.21.15.101:0/97925476 >> 172.21.15.101:0/97925476 conn(0x7f79f00fb130 msgr2=0x7f79f0106570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:19.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.530+0000 7f79f59ff700 1 -- 172.21.15.101:0/97925476 shutdown_connections 2023-07-18T03:05:19.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.530+0000 7f79f59ff700 1 -- 172.21.15.101:0/97925476 wait complete. 2023-07-18T03:05:19.532 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-18T03:05:19.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.750+0000 7fd3525ff700 1 -- 172.21.15.101:0/1385258764 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd34c0ffaf0 msgr2=0x7fd34c0fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:19.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.750+0000 7fd3525ff700 1 --2- 172.21.15.101:0/1385258764 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd34c0ffaf0 0x7fd34c0fff00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd33c0098f0 tx=0x7fd33c009c00 comp rx=0 tx=0).stop 2023-07-18T03:05:19.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.750+0000 7fd3525ff700 1 -- 172.21.15.101:0/1385258764 shutdown_connections 2023-07-18T03:05:19.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.750+0000 7fd3525ff700 1 --2- 172.21.15.101:0/1385258764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd34c101670 0x7fd34c107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:19.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.750+0000 7fd3525ff700 1 --2- 172.21.15.101:0/1385258764 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd34c0ffaf0 0x7fd34c0fff00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:19.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.750+0000 7fd3525ff700 1 --2- 172.21.15.101:0/1385258764 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd34c100ce0 0x7fd34c101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:19.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.750+0000 7fd3525ff700 1 -- 172.21.15.101:0/1385258764 >> 172.21.15.101:0/1385258764 conn(0x7fd34c0fb130 msgr2=0x7fd34c0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:19.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.750+0000 7fd3525ff700 1 -- 172.21.15.101:0/1385258764 shutdown_connections 2023-07-18T03:05:19.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.750+0000 7fd3525ff700 1 -- 172.21.15.101:0/1385258764 wait complete. 2023-07-18T03:05:19.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.751+0000 7fd3525ff700 1 Processor -- start 2023-07-18T03:05:19.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.751+0000 7fd3525ff700 1 -- start start 2023-07-18T03:05:19.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.751+0000 7fd3525ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd34c0ffaf0 0x7fd34c195880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:19.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.751+0000 7fd3525ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd34c100ce0 0x7fd34c195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:19.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.751+0000 7fd3525ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd34c101670 0x7fd34c19ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:19.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.751+0000 7fd3525ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd34c19d400 con 0x7fd34c0ffaf0 2023-07-18T03:05:19.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.751+0000 7fd3525ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd34c19d540 con 0x7fd34c101670 2023-07-18T03:05:19.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.751+0000 7fd3525ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd34c19d6d0 con 0x7fd34c100ce0 2023-07-18T03:05:19.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.752+0000 7fd34bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd34c0ffaf0 0x7fd34c195880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:19.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.752+0000 7fd34bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd34c0ffaf0 0x7fd34c195880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:37610/0 (socket says 172.21.15.101:37610) 2023-07-18T03:05:19.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.752+0000 7fd34bfff700 1 -- 172.21.15.101:0/4166350579 learned_addr learned my addr 172.21.15.101:0/4166350579 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:19.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.752+0000 7fd350b9c700 1 --2- 172.21.15.101:0/4166350579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd34c101670 0x7fd34c19ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:19.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.752+0000 7fd343fff700 1 --2- 172.21.15.101:0/4166350579 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd34c100ce0 0x7fd34c195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:19.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.752+0000 7fd34bfff700 1 -- 172.21.15.101:0/4166350579 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd34c100ce0 msgr2=0x7fd34c195dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:19.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.752+0000 7fd34bfff700 1 --2- 172.21.15.101:0/4166350579 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd34c100ce0 0x7fd34c195dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:19.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.752+0000 7fd34bfff700 1 -- 172.21.15.101:0/4166350579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd34c101670 msgr2=0x7fd34c19ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:19.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.752+0000 7fd34bfff700 1 --2- 172.21.15.101:0/4166350579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd34c101670 0x7fd34c19ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:19.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.752+0000 7fd34bfff700 1 -- 172.21.15.101:0/4166350579 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd344009d00 con 0x7fd34c0ffaf0 2023-07-18T03:05:19.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.752+0000 7fd343fff700 1 --2- 172.21.15.101:0/4166350579 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd34c100ce0 0x7fd34c195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:05:19.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.752+0000 7fd34bfff700 1 --2- 172.21.15.101:0/4166350579 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd34c0ffaf0 0x7fd34c195880 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd33c000c00 tx=0x7fd33c00b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:19.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.752+0000 7fd349ffb700 1 -- 172.21.15.101:0/4166350579 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33c01c070 con 0x7fd34c0ffaf0 2023-07-18T03:05:19.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.753+0000 7fd3525ff700 1 -- 172.21.15.101:0/4166350579 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd33c009520 con 0x7fd34c0ffaf0 2023-07-18T03:05:19.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.753+0000 7fd349ffb700 1 -- 172.21.15.101:0/4166350579 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd33c00f7b0 con 0x7fd34c0ffaf0 2023-07-18T03:05:19.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.753+0000 7fd349ffb700 1 -- 172.21.15.101:0/4166350579 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33c003a00 con 0x7fd34c0ffaf0 2023-07-18T03:05:19.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.753+0000 7fd3525ff700 1 -- 172.21.15.101:0/4166350579 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd34c19dbe0 con 0x7fd34c0ffaf0 2023-07-18T03:05:19.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.754+0000 7fd3525ff700 1 -- 172.21.15.101:0/4166350579 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd34c1070c0 con 0x7fd34c0ffaf0 2023-07-18T03:05:19.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.755+0000 7fd349ffb700 1 -- 172.21.15.101:0/4166350579 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fd33c020030 con 0x7fd34c0ffaf0 2023-07-18T03:05:19.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.757+0000 7fd349ffb700 1 --2- 172.21.15.101:0/4166350579 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd3380a1150 0x7fd3380a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:19.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.757+0000 7fd349ffb700 1 -- 172.21.15.101:0/4166350579 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd33c0cdba0 con 0x7fd34c0ffaf0 2023-07-18T03:05:19.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.757+0000 7fd343fff700 1 --2- 172.21.15.101:0/4166350579 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd3380a1150 0x7fd3380a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:19.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.758+0000 7fd343fff700 1 --2- 172.21.15.101:0/4166350579 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd3380a1150 0x7fd3380a3600 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd334009cb0 tx=0x7fd334009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:19.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.761+0000 7fd349ffb700 1 -- 172.21.15.101:0/4166350579 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd33c097130 con 0x7fd34c0ffaf0 2023-07-18T03:05:19.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.966+0000 7fd3525ff700 1 -- 172.21.15.101:0/4166350579 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd34c061120 con 0x7fd34c0ffaf0 2023-07-18T03:05:19.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.966+0000 7fd349ffb700 1 -- 172.21.15.101:0/4166350579 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd33c09a750 con 0x7fd34c0ffaf0 2023-07-18T03:05:19.950 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 64 2023-07-18T03:05:19.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.968+0000 7fd3525ff700 1 -- 172.21.15.101:0/4166350579 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd3380a1150 msgr2=0x7fd3380a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:19.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.968+0000 7fd3525ff700 1 --2- 172.21.15.101:0/4166350579 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd3380a1150 0x7fd3380a3600 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd334009cb0 tx=0x7fd334009260 comp rx=0 tx=0).stop 2023-07-18T03:05:19.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.968+0000 7fd3525ff700 1 -- 172.21.15.101:0/4166350579 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd34c0ffaf0 msgr2=0x7fd34c195880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:19.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.968+0000 7fd3525ff700 1 --2- 172.21.15.101:0/4166350579 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd34c0ffaf0 0x7fd34c195880 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd33c000c00 tx=0x7fd33c00b730 comp rx=0 tx=0).stop 2023-07-18T03:05:19.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.968+0000 7fd3525ff700 1 -- 172.21.15.101:0/4166350579 shutdown_connections 2023-07-18T03:05:19.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.968+0000 7fd3525ff700 1 --2- 172.21.15.101:0/4166350579 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd3380a1150 0x7fd3380a3600 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:19.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.968+0000 7fd3525ff700 1 --2- 172.21.15.101:0/4166350579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd34c101670 0x7fd34c19ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:19.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.968+0000 7fd3525ff700 1 --2- 172.21.15.101:0/4166350579 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd34c0ffaf0 0x7fd34c195880 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:19.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.968+0000 7fd3525ff700 1 --2- 172.21.15.101:0/4166350579 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd34c100ce0 0x7fd34c195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:19.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.968+0000 7fd3525ff700 1 -- 172.21.15.101:0/4166350579 >> 172.21.15.101:0/4166350579 conn(0x7fd34c0fb130 msgr2=0x7fd34c104200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:19.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.968+0000 7fd3525ff700 1 -- 172.21.15.101:0/4166350579 shutdown_connections 2023-07-18T03:05:19.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:19.968+0000 7fd3525ff700 1 -- 172.21.15.101:0/4166350579 wait complete. 2023-07-18T03:05:19.971 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-18T03:05:20.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.187+0000 7f5bb81ff700 1 -- 172.21.15.101:0/1843946051 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5bb00ffaf0 msgr2=0x7f5bb00fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:20.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.187+0000 7f5bb81ff700 1 --2- 172.21.15.101:0/1843946051 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5bb00ffaf0 0x7f5bb00fff00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5bac009990 tx=0x7f5bac009ca0 comp rx=0 tx=0).stop 2023-07-18T03:05:20.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.187+0000 7f5bb81ff700 1 -- 172.21.15.101:0/1843946051 shutdown_connections 2023-07-18T03:05:20.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.187+0000 7f5bb81ff700 1 --2- 172.21.15.101:0/1843946051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5bb0101670 0x7f5bb0107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:20.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.187+0000 7f5bb81ff700 1 --2- 172.21.15.101:0/1843946051 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5bb00ffaf0 0x7f5bb00fff00 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:20.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.187+0000 7f5bb81ff700 1 --2- 172.21.15.101:0/1843946051 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5bb0100ce0 0x7f5bb0101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:20.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.187+0000 7f5bb81ff700 1 -- 172.21.15.101:0/1843946051 >> 172.21.15.101:0/1843946051 conn(0x7f5bb00fb130 msgr2=0x7f5bb00fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:20.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.187+0000 7f5bb81ff700 1 -- 172.21.15.101:0/1843946051 shutdown_connections 2023-07-18T03:05:20.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.187+0000 7f5bb81ff700 1 -- 172.21.15.101:0/1843946051 wait complete. 2023-07-18T03:05:20.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.188+0000 7f5bb81ff700 1 Processor -- start 2023-07-18T03:05:20.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.188+0000 7f5bb81ff700 1 -- start start 2023-07-18T03:05:20.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.188+0000 7f5bb81ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5bb00ffaf0 0x7f5bb0197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:20.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.188+0000 7f5bb81ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5bb0100ce0 0x7f5bb0197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:20.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.189+0000 7f5bb81ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5bb0101670 0x7f5bb019f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:20.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.189+0000 7f5bb81ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb019f5f0 con 0x7f5bb0101670 2023-07-18T03:05:20.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.189+0000 7f5bb81ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb019f730 con 0x7f5bb0100ce0 2023-07-18T03:05:20.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.189+0000 7f5bb81ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb019f8c0 con 0x7f5bb00ffaf0 2023-07-18T03:05:20.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.189+0000 7f5bb5f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5bb00ffaf0 0x7f5bb0197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:20.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.189+0000 7f5bb679c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5bb0101670 0x7f5bb019f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:20.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.189+0000 7f5bb5f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5bb00ffaf0 0x7f5bb0197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:44594/0 (socket says 172.21.15.101:44594) 2023-07-18T03:05:20.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.189+0000 7f5bb5f9b700 1 -- 172.21.15.101:0/853924670 learned_addr learned my addr 172.21.15.101:0/853924670 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:20.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.189+0000 7f5bb579a700 1 --2- 172.21.15.101:0/853924670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5bb0100ce0 0x7f5bb0197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:20.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.189+0000 7f5bb5f9b700 1 -- 172.21.15.101:0/853924670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5bb0100ce0 msgr2=0x7f5bb0197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:20.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.189+0000 7f5bb5f9b700 1 --2- 172.21.15.101:0/853924670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5bb0100ce0 0x7f5bb0197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:20.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.189+0000 7f5bb5f9b700 1 -- 172.21.15.101:0/853924670 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5bb0101670 msgr2=0x7f5bb019f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:20.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.189+0000 7f5bb5f9b700 1 --2- 172.21.15.101:0/853924670 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5bb0101670 0x7f5bb019f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:20.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.189+0000 7f5bb5f9b700 1 -- 172.21.15.101:0/853924670 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bac0095f0 con 0x7f5bb00ffaf0 2023-07-18T03:05:20.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.189+0000 7f5bb679c700 1 --2- 172.21.15.101:0/853924670 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5bb0101670 0x7f5bb019f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:05:20.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.190+0000 7f5bb5f9b700 1 --2- 172.21.15.101:0/853924670 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5bb00ffaf0 0x7f5bb0197a70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5bac004730 tx=0x7f5bac004760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:20.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.190+0000 7f5ba6ffd700 1 -- 172.21.15.101:0/853924670 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bac01d070 con 0x7f5bb00ffaf0 2023-07-18T03:05:20.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.190+0000 7f5ba6ffd700 1 -- 172.21.15.101:0/853924670 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5bac021410 con 0x7f5bb00ffaf0 2023-07-18T03:05:20.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.190+0000 7f5bb81ff700 1 -- 172.21.15.101:0/853924670 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bb019fb40 con 0x7f5bb00ffaf0 2023-07-18T03:05:20.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.190+0000 7f5ba6ffd700 1 -- 172.21.15.101:0/853924670 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bac017650 con 0x7f5bb00ffaf0 2023-07-18T03:05:20.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.190+0000 7f5bb81ff700 1 -- 172.21.15.101:0/853924670 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bb019ffc0 con 0x7f5bb00ffaf0 2023-07-18T03:05:20.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.191+0000 7f5bb81ff700 1 -- 172.21.15.101:0/853924670 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bb01921d0 con 0x7f5bb00ffaf0 2023-07-18T03:05:20.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.193+0000 7f5ba6ffd700 1 -- 172.21.15.101:0/853924670 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f5bac021580 con 0x7f5bb00ffaf0 2023-07-18T03:05:20.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.194+0000 7f5ba6ffd700 1 --2- 172.21.15.101:0/853924670 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f5b9c0a11d0 0x7f5b9c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:20.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.194+0000 7f5ba6ffd700 1 -- 172.21.15.101:0/853924670 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f5bac0cdfe0 con 0x7f5bb00ffaf0 2023-07-18T03:05:20.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.194+0000 7f5bb579a700 1 --2- 172.21.15.101:0/853924670 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f5b9c0a11d0 0x7f5b9c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:20.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.195+0000 7f5bb579a700 1 --2- 172.21.15.101:0/853924670 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f5b9c0a11d0 0x7f5b9c0a3680 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0009c60 tx=0x7f5ba0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:20.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.196+0000 7f5ba6ffd700 1 -- 172.21.15.101:0/853924670 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5bac097590 con 0x7f5bb00ffaf0 2023-07-18T03:05:20.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:19 smithi179 ceph-mon[129229]: pgmap v285: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-07-18T03:05:20.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:19 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/2085860986' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:05:20.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:19 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/97925476' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:05:20.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:19 smithi101 ceph-mon[126379]: pgmap v285: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-07-18T03:05:20.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:19 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2085860986' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:05:20.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:19 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/97925476' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:05:20.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:19 smithi155 ceph-mon[128885]: pgmap v285: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-07-18T03:05:20.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:19 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/2085860986' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:05:20.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:19 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/97925476' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:05:20.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.396+0000 7f5bb81ff700 1 -- 172.21.15.101:0/853924670 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5bb002c9e0 con 0x7f5bb00ffaf0 2023-07-18T03:05:20.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.396+0000 7f5ba6ffd700 1 -- 172.21.15.101:0/853924670 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5bac095e60 con 0x7f5bb00ffaf0 2023-07-18T03:05:20.381 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 32 2023-07-18T03:05:20.382 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.399+0000 7f5bb81ff700 1 -- 172.21.15.101:0/853924670 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f5b9c0a11d0 msgr2=0x7f5b9c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:20.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.399+0000 7f5bb81ff700 1 --2- 172.21.15.101:0/853924670 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f5b9c0a11d0 0x7f5b9c0a3680 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0009c60 tx=0x7f5ba0009260 comp rx=0 tx=0).stop 2023-07-18T03:05:20.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.399+0000 7f5bb81ff700 1 -- 172.21.15.101:0/853924670 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5bb00ffaf0 msgr2=0x7f5bb0197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:20.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.399+0000 7f5bb81ff700 1 --2- 172.21.15.101:0/853924670 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5bb00ffaf0 0x7f5bb0197a70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5bac004730 tx=0x7f5bac004760 comp rx=0 tx=0).stop 2023-07-18T03:05:20.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.399+0000 7f5bb81ff700 1 -- 172.21.15.101:0/853924670 shutdown_connections 2023-07-18T03:05:20.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.399+0000 7f5bb81ff700 1 --2- 172.21.15.101:0/853924670 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f5b9c0a11d0 0x7f5b9c0a3680 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:20.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.399+0000 7f5bb81ff700 1 --2- 172.21.15.101:0/853924670 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5bb0101670 0x7f5bb019f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:20.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.399+0000 7f5bb81ff700 1 --2- 172.21.15.101:0/853924670 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5bb00ffaf0 0x7f5bb0197a70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:20.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.399+0000 7f5bb81ff700 1 --2- 172.21.15.101:0/853924670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5bb0100ce0 0x7f5bb0197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:20.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.399+0000 7f5bb81ff700 1 -- 172.21.15.101:0/853924670 >> 172.21.15.101:0/853924670 conn(0x7f5bb00fb130 msgr2=0x7f5bb01065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:20.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.399+0000 7f5bb81ff700 1 -- 172.21.15.101:0/853924670 shutdown_connections 2023-07-18T03:05:20.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.399+0000 7f5bb81ff700 1 -- 172.21.15.101:0/853924670 wait complete. 2023-07-18T03:05:20.401 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-18T03:05:20.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.614+0000 7fed71dff700 1 -- 172.21.15.101:0/2742365326 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fed6c074550 msgr2=0x7fed6c0729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:20.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.614+0000 7fed71dff700 1 --2- 172.21.15.101:0/2742365326 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fed6c074550 0x7fed6c0729a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fed5c009990 tx=0x7fed5c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:05:20.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.615+0000 7fed71dff700 1 -- 172.21.15.101:0/2742365326 shutdown_connections 2023-07-18T03:05:20.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.615+0000 7fed71dff700 1 --2- 172.21.15.101:0/2742365326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed6c1079a0 0x7fed6c109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:20.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.615+0000 7fed71dff700 1 --2- 172.21.15.101:0/2742365326 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fed6c074550 0x7fed6c0729a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:20.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.615+0000 7fed71dff700 1 --2- 172.21.15.101:0/2742365326 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed6c072f70 0x7fed6c0733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:20.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.615+0000 7fed71dff700 1 -- 172.21.15.101:0/2742365326 >> 172.21.15.101:0/2742365326 conn(0x7fed6c0fb0e0 msgr2=0x7fed6c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:20.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.615+0000 7fed71dff700 1 -- 172.21.15.101:0/2742365326 shutdown_connections 2023-07-18T03:05:20.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.615+0000 7fed71dff700 1 -- 172.21.15.101:0/2742365326 wait complete. 2023-07-18T03:05:20.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.616+0000 7fed71dff700 1 Processor -- start 2023-07-18T03:05:20.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.616+0000 7fed71dff700 1 -- start start 2023-07-18T03:05:20.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.616+0000 7fed71dff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed6c072f70 0x7fed6c197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:20.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.616+0000 7fed71dff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed6c074550 0x7fed6c197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:20.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.616+0000 7fed71dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fed6c1079a0 0x7fed6c19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:20.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.616+0000 7fed71dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c19f5e0 con 0x7fed6c1079a0 2023-07-18T03:05:20.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.616+0000 7fed71dff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c19f720 con 0x7fed6c072f70 2023-07-18T03:05:20.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.616+0000 7fed71dff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c19f8b0 con 0x7fed6c074550 2023-07-18T03:05:20.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.616+0000 7fed6bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fed6c1079a0 0x7fed6c19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:20.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.616+0000 7fed6bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fed6c1079a0 0x7fed6c19f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:37646/0 (socket says 172.21.15.101:37646) 2023-07-18T03:05:20.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.616+0000 7fed6b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed6c072f70 0x7fed6c197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:20.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.616+0000 7fed6bfff700 1 -- 172.21.15.101:0/943776287 learned_addr learned my addr 172.21.15.101:0/943776287 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:20.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.617+0000 7fed63fff700 1 --2- 172.21.15.101:0/943776287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed6c074550 0x7fed6c197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:20.604 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.617+0000 7fed6bfff700 1 -- 172.21.15.101:0/943776287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed6c074550 msgr2=0x7fed6c197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:20.604 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.617+0000 7fed6bfff700 1 --2- 172.21.15.101:0/943776287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed6c074550 0x7fed6c197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:20.604 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.617+0000 7fed6bfff700 1 -- 172.21.15.101:0/943776287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed6c072f70 msgr2=0x7fed6c197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:20.604 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.617+0000 7fed6bfff700 1 --2- 172.21.15.101:0/943776287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed6c072f70 0x7fed6c197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:20.604 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.617+0000 7fed6bfff700 1 -- 172.21.15.101:0/943776287 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed5c0095f0 con 0x7fed6c1079a0 2023-07-18T03:05:20.604 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.617+0000 7fed6b7fe700 1 --2- 172.21.15.101:0/943776287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed6c072f70 0x7fed6c197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:05:20.604 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.617+0000 7fed6bfff700 1 --2- 172.21.15.101:0/943776287 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fed6c1079a0 0x7fed6c19f010 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fed6400ef30 tx=0x7fed6400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:20.604 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.617+0000 7fed697fa700 1 -- 172.21.15.101:0/943776287 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed64018070 con 0x7fed6c1079a0 2023-07-18T03:05:20.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.617+0000 7fed697fa700 1 -- 172.21.15.101:0/943776287 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed6400f040 con 0x7fed6c1079a0 2023-07-18T03:05:20.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.617+0000 7fed71dff700 1 -- 172.21.15.101:0/943776287 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed6c19fb30 con 0x7fed6c1079a0 2023-07-18T03:05:20.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.617+0000 7fed697fa700 1 -- 172.21.15.101:0/943776287 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6400cd00 con 0x7fed6c1079a0 2023-07-18T03:05:20.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.617+0000 7fed71dff700 1 -- 172.21.15.101:0/943776287 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed6c1a0090 con 0x7fed6c1079a0 2023-07-18T03:05:20.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.618+0000 7fed71dff700 1 -- 172.21.15.101:0/943776287 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed6c106050 con 0x7fed6c1079a0 2023-07-18T03:05:20.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.620+0000 7fed697fa700 1 -- 172.21.15.101:0/943776287 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fed64003a00 con 0x7fed6c1079a0 2023-07-18T03:05:20.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.622+0000 7fed697fa700 1 --2- 172.21.15.101:0/943776287 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fed580a11d0 0x7fed580a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:20.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.622+0000 7fed697fa700 1 -- 172.21.15.101:0/943776287 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fed640cc5c0 con 0x7fed6c1079a0 2023-07-18T03:05:20.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.622+0000 7fed6b7fe700 1 --2- 172.21.15.101:0/943776287 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fed580a11d0 0x7fed580a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:20.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.622+0000 7fed6b7fe700 1 --2- 172.21.15.101:0/943776287 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fed580a11d0 0x7fed580a3680 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fed5c00b2e0 tx=0x7fed5c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:20.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.625+0000 7fed697fa700 1 -- 172.21.15.101:0/943776287 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fed64095b50 con 0x7fed6c1079a0 2023-07-18T03:05:20.824 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.840+0000 7fed71dff700 1 -- 172.21.15.101:0/943776287 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fed6c02c850 con 0x7fed6c1079a0 2023-07-18T03:05:20.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.841+0000 7fed697fa700 1 -- 172.21.15.101:0/943776287 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) v1 ==== 81+0+600 (secure 0 0 0) 0x7fed64099170 con 0x7fed6c1079a0 2023-07-18T03:05:20.825 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.15.101:6800/2461037266 2023-07-19T02:58:22.762480+0000 2023-07-18T03:05:20.825 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.15.101:6801/2461037266 2023-07-19T02:58:22.762480+0000 2023-07-18T03:05:20.825 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.15.101:0/604228291 2023-07-19T02:58:22.762480+0000 2023-07-18T03:05:20.825 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.15.101:0/4055187849 2023-07-19T02:58:22.762480+0000 2023-07-18T03:05:20.826 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.15.101:0/2844574324 2023-07-19T02:57:56.243200+0000 2023-07-18T03:05:20.826 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.15.101:0/1603248215 2023-07-19T02:57:56.243200+0000 2023-07-18T03:05:20.826 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.15.101:6801/3497914514 2023-07-19T02:57:56.243200+0000 2023-07-18T03:05:20.826 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.15.101:6800/3497914514 2023-07-19T02:57:56.243200+0000 2023-07-18T03:05:20.826 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.15.101:0/3307578182 2023-07-19T02:58:22.762480+0000 2023-07-18T03:05:20.826 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.15.101:0/373189792 2023-07-19T02:57:56.243200+0000 2023-07-18T03:05:20.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.843+0000 7fed71dff700 1 -- 172.21.15.101:0/943776287 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fed580a11d0 msgr2=0x7fed580a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:20.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.843+0000 7fed71dff700 1 --2- 172.21.15.101:0/943776287 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fed580a11d0 0x7fed580a3680 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fed5c00b2e0 tx=0x7fed5c005fb0 comp rx=0 tx=0).stop 2023-07-18T03:05:20.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.843+0000 7fed71dff700 1 -- 172.21.15.101:0/943776287 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fed6c1079a0 msgr2=0x7fed6c19f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:20.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.843+0000 7fed71dff700 1 --2- 172.21.15.101:0/943776287 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fed6c1079a0 0x7fed6c19f010 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fed6400ef30 tx=0x7fed6400c3a0 comp rx=0 tx=0).stop 2023-07-18T03:05:20.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.843+0000 7fed71dff700 1 -- 172.21.15.101:0/943776287 shutdown_connections 2023-07-18T03:05:20.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.843+0000 7fed71dff700 1 --2- 172.21.15.101:0/943776287 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fed580a11d0 0x7fed580a3680 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:20.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.843+0000 7fed71dff700 1 --2- 172.21.15.101:0/943776287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed6c072f70 0x7fed6c197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:20.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.843+0000 7fed71dff700 1 --2- 172.21.15.101:0/943776287 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fed6c1079a0 0x7fed6c19f010 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:20.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.843+0000 7fed71dff700 1 --2- 172.21.15.101:0/943776287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed6c074550 0x7fed6c197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:20.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.843+0000 7fed71dff700 1 -- 172.21.15.101:0/943776287 >> 172.21.15.101:0/943776287 conn(0x7fed6c0fb0e0 msgr2=0x7fed6c0fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:20.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.843+0000 7fed71dff700 1 -- 172.21.15.101:0/943776287 shutdown_connections 2023-07-18T03:05:20.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:20.843+0000 7fed71dff700 1 -- 172.21.15.101:0/943776287 wait complete. 2023-07-18T03:05:20.829 INFO:teuthology.orchestra.run.smithi101.stderr:listed 10 entries 2023-07-18T03:05:20.845 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi101.front.sepia.ceph.com... 2023-07-18T03:05:20.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:05:20.846 DEBUG:teuthology.orchestra.run.smithi101:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-07-18T03:05:20.913 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi101.stderr:ceph-fuse[163692]: fuse finished with error 0 and tester_r 0 2023-07-18T03:05:20.917 INFO:teuthology.orchestra.run:waiting for 300 2023-07-18T03:05:21.100 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:20 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/4166350579' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:05:21.101 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:20 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/853924670' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-18T03:05:21.101 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:20 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/943776287' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-18T03:05:21.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:20 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/4166350579' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:05:21.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:20 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/853924670' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-18T03:05:21.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:20 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/943776287' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-18T03:05:21.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:20 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/4166350579' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:05:21.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:20 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/853924670' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-18T03:05:21.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:20 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/943776287' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-18T03:05:22.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:21 smithi179 ceph-mon[129229]: pgmap v286: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-07-18T03:05:22.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:21 smithi101 ceph-mon[126379]: pgmap v286: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-07-18T03:05:22.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:21 smithi155 ceph-mon[128885]: pgmap v286: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-07-18T03:05:23.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:22 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2023-07-18T03:05:23.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:22 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]: dispatch 2023-07-18T03:05:23.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:22 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2023-07-18T03:05:23.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:22 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6, 5, 7]}]: dispatch 2023-07-18T03:05:23.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:22 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2023-07-18T03:05:23.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:22 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2023-07-18T03:05:23.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:22 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]: dispatch 2023-07-18T03:05:23.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:22 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2023-07-18T03:05:23.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:22 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]: dispatch 2023-07-18T03:05:23.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:22 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2023-07-18T03:05:23.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:22 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]: dispatch 2023-07-18T03:05:23.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:22 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2023-07-18T03:05:23.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:22 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6, 5, 7]}]: dispatch 2023-07-18T03:05:23.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:22 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2023-07-18T03:05:23.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:22 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2023-07-18T03:05:23.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:22 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]: dispatch 2023-07-18T03:05:23.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:22 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2023-07-18T03:05:23.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:22 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]: dispatch 2023-07-18T03:05:23.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:22 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2023-07-18T03:05:23.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:22 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]: dispatch 2023-07-18T03:05:23.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:22 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2023-07-18T03:05:23.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:22 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6, 5, 7]}]: dispatch 2023-07-18T03:05:23.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:22 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2023-07-18T03:05:23.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:22 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2023-07-18T03:05:23.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:22 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]: dispatch 2023-07-18T03:05:23.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:22 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2023-07-18T03:05:23.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:22 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]: dispatch 2023-07-18T03:05:24.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:23 smithi179 ceph-mon[129229]: pgmap v287: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-07-18T03:05:24.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:23 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2023-07-18T03:05:24.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:23 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]': finished 2023-07-18T03:05:24.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:23 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2023-07-18T03:05:24.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:23 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6, 5, 7]}]': finished 2023-07-18T03:05:24.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:23 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2023-07-18T03:05:24.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:23 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2023-07-18T03:05:24.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:23 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]': finished 2023-07-18T03:05:24.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:23 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2023-07-18T03:05:24.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:23 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]': finished 2023-07-18T03:05:24.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:23 smithi179 ceph-mon[129229]: osdmap e88: 12 total, 12 up, 12 in 2023-07-18T03:05:24.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:23 smithi101 ceph-mon[126379]: pgmap v287: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-07-18T03:05:24.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:23 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2023-07-18T03:05:24.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:23 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]': finished 2023-07-18T03:05:24.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:23 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2023-07-18T03:05:24.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:23 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6, 5, 7]}]': finished 2023-07-18T03:05:24.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:23 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2023-07-18T03:05:24.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:23 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2023-07-18T03:05:24.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:23 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]': finished 2023-07-18T03:05:24.232 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:23 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2023-07-18T03:05:24.232 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:23 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]': finished 2023-07-18T03:05:24.232 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:23 smithi101 ceph-mon[126379]: osdmap e88: 12 total, 12 up, 12 in 2023-07-18T03:05:24.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:23 smithi155 ceph-mon[128885]: pgmap v287: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-07-18T03:05:24.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:23 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2023-07-18T03:05:24.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:23 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]': finished 2023-07-18T03:05:24.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:23 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2023-07-18T03:05:24.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:23 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6, 5, 7]}]': finished 2023-07-18T03:05:24.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:23 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2023-07-18T03:05:24.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:23 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2023-07-18T03:05:24.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:23 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]': finished 2023-07-18T03:05:24.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:23 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2023-07-18T03:05:24.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:23 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]': finished 2023-07-18T03:05:24.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:23 smithi155 ceph-mon[128885]: osdmap e88: 12 total, 12 up, 12 in 2023-07-18T03:05:25.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:24 smithi179 ceph-mon[129229]: osdmap e89: 12 total, 12 up, 12 in 2023-07-18T03:05:25.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:24 smithi101 ceph-mon[126379]: osdmap e89: 12 total, 12 up, 12 in 2023-07-18T03:05:25.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:24 smithi155 ceph-mon[128885]: osdmap e89: 12 total, 12 up, 12 in 2023-07-18T03:05:26.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:25 smithi179 ceph-mon[129229]: pgmap v290: 129 pgs: 1 activating, 1 peering, 127 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s 2023-07-18T03:05:26.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:25 smithi101 ceph-mon[126379]: pgmap v290: 129 pgs: 1 activating, 1 peering, 127 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s 2023-07-18T03:05:26.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:25 smithi155 ceph-mon[128885]: pgmap v290: 129 pgs: 1 activating, 1 peering, 127 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s 2023-07-18T03:05:27.019 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi101.front.sepia.ceph.com 2023-07-18T03:05:27.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:05:27.020 DEBUG:teuthology.orchestra.run.smithi101:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-07-18T03:05:27.086 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:05:27.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.304+0000 7f91fa5ff700 1 -- 172.21.15.101:0/2326362840 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f91f4102eb0 msgr2=0x7f91f4105290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:27.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.304+0000 7f91fa5ff700 1 --2- 172.21.15.101:0/2326362840 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f91f4102eb0 0x7f91f4105290 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f91dc009990 tx=0x7f91dc009ca0 comp rx=0 tx=0).stop 2023-07-18T03:05:27.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.304+0000 7f91fa5ff700 1 -- 172.21.15.101:0/2326362840 shutdown_connections 2023-07-18T03:05:27.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.304+0000 7f91fa5ff700 1 --2- 172.21.15.101:0/2326362840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91f41057d0 0x7f91f4107bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:27.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.304+0000 7f91fa5ff700 1 --2- 172.21.15.101:0/2326362840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91f4100590 0x7f91f4102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:27.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.304+0000 7f91fa5ff700 1 --2- 172.21.15.101:0/2326362840 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f91f4102eb0 0x7f91f4105290 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:27.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.304+0000 7f91fa5ff700 1 -- 172.21.15.101:0/2326362840 >> 172.21.15.101:0/2326362840 conn(0x7f91f40f9fd0 msgr2=0x7f91f40fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:27.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.304+0000 7f91fa5ff700 1 -- 172.21.15.101:0/2326362840 shutdown_connections 2023-07-18T03:05:27.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.304+0000 7f91fa5ff700 1 -- 172.21.15.101:0/2326362840 wait complete. 2023-07-18T03:05:27.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.305+0000 7f91fa5ff700 1 Processor -- start 2023-07-18T03:05:27.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.305+0000 7f91fa5ff700 1 -- start start 2023-07-18T03:05:27.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.305+0000 7f91fa5ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91f4100590 0x7f91f41957e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:27.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.305+0000 7f91fa5ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f91f4102eb0 0x7f91f4195d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:27.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.306+0000 7f91fa5ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91f41057d0 0x7f91f419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:27.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.306+0000 7f91fa5ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f419d360 con 0x7f91f4102eb0 2023-07-18T03:05:27.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.306+0000 7f91fa5ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f419d4a0 con 0x7f91f41057d0 2023-07-18T03:05:27.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.306+0000 7f91fa5ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f419d630 con 0x7f91f4100590 2023-07-18T03:05:27.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.306+0000 7f91f3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91f4100590 0x7f91f41957e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:27.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.306+0000 7f91f3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91f4100590 0x7f91f41957e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:43522/0 (socket says 172.21.15.101:43522) 2023-07-18T03:05:27.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.306+0000 7f91ebfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f91f4102eb0 0x7f91f4195d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:27.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.306+0000 7f91f3fff700 1 -- 172.21.15.101:0/4233796821 learned_addr learned my addr 172.21.15.101:0/4233796821 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:27.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.306+0000 7f91f8b9c700 1 --2- 172.21.15.101:0/4233796821 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91f41057d0 0x7f91f419cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:27.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.306+0000 7f91ebfff700 1 -- 172.21.15.101:0/4233796821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91f4100590 msgr2=0x7f91f41957e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:27.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.306+0000 7f91ebfff700 1 --2- 172.21.15.101:0/4233796821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91f4100590 0x7f91f41957e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:27.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.306+0000 7f91ebfff700 1 -- 172.21.15.101:0/4233796821 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91f41057d0 msgr2=0x7f91f419cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:27.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.306+0000 7f91ebfff700 1 --2- 172.21.15.101:0/4233796821 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91f41057d0 0x7f91f419cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:27.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.306+0000 7f91ebfff700 1 -- 172.21.15.101:0/4233796821 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91dc0095f0 con 0x7f91f4102eb0 2023-07-18T03:05:27.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.306+0000 7f91f3fff700 1 --2- 172.21.15.101:0/4233796821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91f4100590 0x7f91f41957e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:05:27.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.306+0000 7f91ebfff700 1 --2- 172.21.15.101:0/4233796821 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f91f4102eb0 0x7f91f4195d20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f91dc00b680 tx=0x7f91dc005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:27.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.307+0000 7f91f1ffb700 1 -- 172.21.15.101:0/4233796821 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91dc01d070 con 0x7f91f4102eb0 2023-07-18T03:05:27.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.307+0000 7f91f1ffb700 1 -- 172.21.15.101:0/4233796821 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91dc021410 con 0x7f91f4102eb0 2023-07-18T03:05:27.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.307+0000 7f91f1ffb700 1 -- 172.21.15.101:0/4233796821 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91dc021ea0 con 0x7f91f4102eb0 2023-07-18T03:05:27.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.307+0000 7f91fa5ff700 1 -- 172.21.15.101:0/4233796821 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91f419d8b0 con 0x7f91f4102eb0 2023-07-18T03:05:27.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.307+0000 7f91fa5ff700 1 -- 172.21.15.101:0/4233796821 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91f419dc80 con 0x7f91f4102eb0 2023-07-18T03:05:27.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.308+0000 7f91fa5ff700 1 -- 172.21.15.101:0/4233796821 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91f40fbb40 con 0x7f91f4102eb0 2023-07-18T03:05:27.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.310+0000 7f91f1ffb700 1 -- 172.21.15.101:0/4233796821 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f91dc0217f0 con 0x7f91f4102eb0 2023-07-18T03:05:27.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.311+0000 7f91f1ffb700 1 --2- 172.21.15.101:0/4233796821 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f91e00a1220 0x7f91e00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:27.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.311+0000 7f91f1ffb700 1 -- 172.21.15.101:0/4233796821 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f91dc0ce230 con 0x7f91f4102eb0 2023-07-18T03:05:27.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.311+0000 7f91f3fff700 1 --2- 172.21.15.101:0/4233796821 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f91e00a1220 0x7f91e00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:27.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.312+0000 7f91f3fff700 1 --2- 172.21.15.101:0/4233796821 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f91e00a1220 0x7f91e00a36d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f91e4005950 tx=0x7f91e400b500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:27.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.313+0000 7f91f1ffb700 1 -- 172.21.15.101:0/4233796821 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f91dc0976b0 con 0x7f91f4102eb0 2023-07-18T03:05:27.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.515+0000 7f91fa5ff700 1 -- 172.21.15.101:0/4233796821 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91f402caa0 con 0x7f91f4102eb0 2023-07-18T03:05:27.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.516+0000 7f91f1ffb700 1 -- 172.21.15.101:0/4233796821 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24482 (secure 0 0 0) 0x7f91dc09acd0 con 0x7f91f4102eb0 2023-07-18T03:05:27.502 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:05:27.502 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":89,"fsid":"b1782daa-2516-11ee-9b34-001a4aab830c","created":"2023-07-18T02:57:22.972483+0000","modified":"2023-07-18T03:05:23.929113+0000","last_up_change":"2023-07-18T03:04:11.429530+0000","last_in_change":"2023-07-18T03:03:54.158340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:00:43.353489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:04:57.771616+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-18T03:04:58.769590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-18T03:05:02.358830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc76b5d1-38a0-4f1a-9335-756bfc40a62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6803","nonce":3610719929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6805","nonce":3610719929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6809","nonce":3610719929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6807","nonce":3610719929}]},"public_addr":"172.21.15.101:6803/3610719929","cluster_addr":"172.21.15.101:6805/3610719929","heartbeat_back_addr":"172.21.15.101:6809/3610719929","heartbeat_front_addr":"172.21.15.101:6807/3610719929","state":["exists","up"]},{"osd":1,"uuid":"2a299573-0b34-4af9-aaae-b523e5058ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6801","nonce":1030229579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6803","nonce":1030229579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6807","nonce":1030229579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6805","nonce":1030229579}]},"public_addr":"172.21.15.155:6801/1030229579","cluster_addr":"172.21.15.155:6803/1030229579","heartbeat_back_addr":"172.21.15.155:6807/1030229579","heartbeat_front_addr":"172.21.15.155:6805/1030229579","state":["exists","up"]},{"osd":2,"uuid":"123b3749-c636-48fa-9e33-6229b5575dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6801","nonce":1349487507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6803","nonce":1349487507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6807","nonce":1349487507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6805","nonce":1349487507}]},"public_addr":"172.21.15.179:6801/1349487507","cluster_addr":"172.21.15.179:6803/1349487507","heartbeat_back_addr":"172.21.15.179:6807/1349487507","heartbeat_front_addr":"172.21.15.179:6805/1349487507","state":["exists","up"]},{"osd":3,"uuid":"4b0c409d-2954-4127-833a-b196bf226d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6811","nonce":4226577463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6813","nonce":4226577463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6817","nonce":4226577463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6815","nonce":4226577463}]},"public_addr":"172.21.15.101:6811/4226577463","cluster_addr":"172.21.15.101:6813/4226577463","heartbeat_back_addr":"172.21.15.101:6817/4226577463","heartbeat_front_addr":"172.21.15.101:6815/4226577463","state":["exists","up"]},{"osd":4,"uuid":"2137f2df-984e-45ce-baf6-b4ac12c7e3c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6809","nonce":1951135805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6811","nonce":1951135805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6815","nonce":1951135805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6813","nonce":1951135805}]},"public_addr":"172.21.15.155:6809/1951135805","cluster_addr":"172.21.15.155:6811/1951135805","heartbeat_back_addr":"172.21.15.155:6815/1951135805","heartbeat_front_addr":"172.21.15.155:6813/1951135805","state":["exists","up"]},{"osd":5,"uuid":"1020bfc5-8c2f-482e-9588-66c7f98e46e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6809","nonce":2746037021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6811","nonce":2746037021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6815","nonce":2746037021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6813","nonce":2746037021}]},"public_addr":"172.21.15.179:6809/2746037021","cluster_addr":"172.21.15.179:6811/2746037021","heartbeat_back_addr":"172.21.15.179:6815/2746037021","heartbeat_front_addr":"172.21.15.179:6813/2746037021","state":["exists","up"]},{"osd":6,"uuid":"2cd99260-5c36-4603-9338-cf697970ce58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6819","nonce":958044931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6821","nonce":958044931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6825","nonce":958044931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6823","nonce":958044931}]},"public_addr":"172.21.15.101:6819/958044931","cluster_addr":"172.21.15.101:6821/958044931","heartbeat_back_addr":"172.21.15.101:6825/958044931","heartbeat_front_addr":"172.21.15.101:6823/958044931","state":["exists","up"]},{"osd":7,"uuid":"ed23fd0a-678f-4e15-b42c-cfe873315714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6817","nonce":3256322291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6819","nonce":3256322291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6823","nonce":3256322291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6821","nonce":3256322291}]},"public_addr":"172.21.15.155:6817/3256322291","cluster_addr":"172.21.15.155:6819/3256322291","heartbeat_back_addr":"172.21.15.155:6823/3256322291","heartbeat_front_addr":"172.21.15.155:6821/3256322291","state":["exists","up"]},{"osd":8,"uuid":"eca3d8e6-7baf-47a5-b991-328001c7ebef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6817","nonce":3520612559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6819","nonce":3520612559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6823","nonce":3520612559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6821","nonce":3520612559}]},"public_addr":"172.21.15.179:6817/3520612559","cluster_addr":"172.21.15.179:6819/3520612559","heartbeat_back_addr":"172.21.15.179:6823/3520612559","heartbeat_front_addr":"172.21.15.179:6821/3520612559","state":["exists","up"]},{"osd":9,"uuid":"465a7ffb-67e4-49ef-812c-1b96feacfebf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6827","nonce":2166787667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6829","nonce":2166787667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6833","nonce":2166787667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6831","nonce":2166787667}]},"public_addr":"172.21.15.101:6827/2166787667","cluster_addr":"172.21.15.101:6829/2166787667","heartbeat_back_addr":"172.21.15.101:6833/2166787667","heartbeat_front_addr":"172.21.15.101:6831/2166787667","state":["exists","up"]},{"osd":10,"uuid":"1897a027-f25d-434f-8df5-5fc2bd866578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6825","nonce":3682446349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6827","nonce":3682446349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6831","nonce":3682446349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6829","nonce":3682446349}]},"public_addr":"172.21.15.155:6825/3682446349","cluster_addr":"172.21.15.155:6827/3682446349","heartbeat_back_addr":"172.21.15.155:6831/3682446349","heartbeat_front_addr":"172.21.15.155:6829/3682446349","state":["exists","up"]},{"osd":11,"uuid":"b62effaf-45e2-4585-b8d0-db448f3474ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6825","nonce":2602640858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6827","nonce":2602640858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6831","nonce":2602640858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6829","nonce":2602640858}]},"public_addr":"172.21.15.179:6825/2602640858","cluster_addr":"172.21.15.179:6827/2602640858","heartbeat_back_addr":"172.21.15.179:6831/2602640858","heartbeat_front_addr":"172.21.15.179:6829/2602640858","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T02:59:56.074519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:19.358142+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:41.128966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:08.227054+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:31.442592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:52.751685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:19.251054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:40.698241+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:01.339150+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:27.955910+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:49.136980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:04:10.084982+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.101:6800/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:6801/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/604228291":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/4055187849":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/2844574324":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/1603248215":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6801/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6800/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/3307578182":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/373189792":"2023-07-19T02:57:56.243200+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:05:27.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.519+0000 7f91fa5ff700 1 -- 172.21.15.101:0/4233796821 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f91e00a1220 msgr2=0x7f91e00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:27.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.519+0000 7f91fa5ff700 1 --2- 172.21.15.101:0/4233796821 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f91e00a1220 0x7f91e00a36d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f91e4005950 tx=0x7f91e400b500 comp rx=0 tx=0).stop 2023-07-18T03:05:27.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.519+0000 7f91fa5ff700 1 -- 172.21.15.101:0/4233796821 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f91f4102eb0 msgr2=0x7f91f4195d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:27.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.519+0000 7f91fa5ff700 1 --2- 172.21.15.101:0/4233796821 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f91f4102eb0 0x7f91f4195d20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f91dc00b680 tx=0x7f91dc005740 comp rx=0 tx=0).stop 2023-07-18T03:05:27.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.519+0000 7f91fa5ff700 1 -- 172.21.15.101:0/4233796821 shutdown_connections 2023-07-18T03:05:27.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.519+0000 7f91fa5ff700 1 --2- 172.21.15.101:0/4233796821 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f91e00a1220 0x7f91e00a36d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:27.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.519+0000 7f91fa5ff700 1 --2- 172.21.15.101:0/4233796821 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91f41057d0 0x7f91f419cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:27.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.519+0000 7f91fa5ff700 1 --2- 172.21.15.101:0/4233796821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91f4100590 0x7f91f41957e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:27.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.519+0000 7f91fa5ff700 1 --2- 172.21.15.101:0/4233796821 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f91f4102eb0 0x7f91f4195d20 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:27.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.519+0000 7f91fa5ff700 1 -- 172.21.15.101:0/4233796821 >> 172.21.15.101:0/4233796821 conn(0x7f91f40f9fd0 msgr2=0x7f91f41011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:27.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.519+0000 7f91fa5ff700 1 -- 172.21.15.101:0/4233796821 shutdown_connections 2023-07-18T03:05:27.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.519+0000 7f91fa5ff700 1 -- 172.21.15.101:0/4233796821 wait complete. 2023-07-18T03:05:27.523 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:05:27.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.735+0000 7fbbdedff700 1 -- 172.21.15.101:0/1424011529 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbbd8100cf0 msgr2=0x7fbbd8101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:27.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.735+0000 7fbbdedff700 1 --2- 172.21.15.101:0/1424011529 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbbd8100cf0 0x7fbbd8101140 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbbc4009990 tx=0x7fbbc4009ca0 comp rx=0 tx=0).stop 2023-07-18T03:05:27.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.735+0000 7fbbdedff700 1 -- 172.21.15.101:0/1424011529 shutdown_connections 2023-07-18T03:05:27.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.735+0000 7fbbdedff700 1 --2- 172.21.15.101:0/1424011529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbbd80ffb00 0x7fbbd80fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:27.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.735+0000 7fbbdedff700 1 --2- 172.21.15.101:0/1424011529 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbbd8101680 0x7fbbd8107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:27.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.735+0000 7fbbdedff700 1 --2- 172.21.15.101:0/1424011529 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbbd8100cf0 0x7fbbd8101140 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:27.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.735+0000 7fbbdedff700 1 -- 172.21.15.101:0/1424011529 >> 172.21.15.101:0/1424011529 conn(0x7fbbd80fb0e0 msgr2=0x7fbbd80fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:27.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.735+0000 7fbbdedff700 1 -- 172.21.15.101:0/1424011529 shutdown_connections 2023-07-18T03:05:27.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.735+0000 7fbbdedff700 1 -- 172.21.15.101:0/1424011529 wait complete. 2023-07-18T03:05:27.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.736+0000 7fbbdedff700 1 Processor -- start 2023-07-18T03:05:27.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.736+0000 7fbbdedff700 1 -- start start 2023-07-18T03:05:27.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.737+0000 7fbbdedff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbbd80ffb00 0x7fbbd81979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:27.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.737+0000 7fbbdedff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbbd8100cf0 0x7fbbd8197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:27.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.737+0000 7fbbdedff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbbd8101680 0x7fbbd819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:27.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.737+0000 7fbbdedff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd819f550 con 0x7fbbd8100cf0 2023-07-18T03:05:27.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.737+0000 7fbbdedff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd819f690 con 0x7fbbd8101680 2023-07-18T03:05:27.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.737+0000 7fbbdedff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd819f820 con 0x7fbbd80ffb00 2023-07-18T03:05:27.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.737+0000 7fbbd7fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbbd8100cf0 0x7fbbd8197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:27.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.737+0000 7fbbd7fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbbd8100cf0 0x7fbbd8197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:53304/0 (socket says 172.21.15.101:53304) 2023-07-18T03:05:27.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.737+0000 7fbbd7fff700 1 -- 172.21.15.101:0/1911796560 learned_addr learned my addr 172.21.15.101:0/1911796560 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:27.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.737+0000 7fbbdcb9b700 1 --2- 172.21.15.101:0/1911796560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbbd80ffb00 0x7fbbd81979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:27.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.737+0000 7fbbdd39c700 1 --2- 172.21.15.101:0/1911796560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbbd8101680 0x7fbbd819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:27.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.737+0000 7fbbd7fff700 1 -- 172.21.15.101:0/1911796560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbbd80ffb00 msgr2=0x7fbbd81979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:27.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.737+0000 7fbbd7fff700 1 --2- 172.21.15.101:0/1911796560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbbd80ffb00 0x7fbbd81979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:27.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.737+0000 7fbbd7fff700 1 -- 172.21.15.101:0/1911796560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbbd8101680 msgr2=0x7fbbd819ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:27.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.737+0000 7fbbd7fff700 1 --2- 172.21.15.101:0/1911796560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbbd8101680 0x7fbbd819ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:27.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.737+0000 7fbbd7fff700 1 -- 172.21.15.101:0/1911796560 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbcc009c40 con 0x7fbbd8100cf0 2023-07-18T03:05:27.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.738+0000 7fbbd7fff700 1 --2- 172.21.15.101:0/1911796560 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbbd8100cf0 0x7fbbd8197f10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbbc4004760 tx=0x7fbbc4004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:27.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.738+0000 7fbbd5ffb700 1 -- 172.21.15.101:0/1911796560 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc401d070 con 0x7fbbd8100cf0 2023-07-18T03:05:27.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.738+0000 7fbbd5ffb700 1 -- 172.21.15.101:0/1911796560 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbc4004830 con 0x7fbbd8100cf0 2023-07-18T03:05:27.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.738+0000 7fbbdedff700 1 -- 172.21.15.101:0/1911796560 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbc40095f0 con 0x7fbbd8100cf0 2023-07-18T03:05:27.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.738+0000 7fbbd5ffb700 1 -- 172.21.15.101:0/1911796560 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc4005350 con 0x7fbbd8100cf0 2023-07-18T03:05:27.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.738+0000 7fbbdedff700 1 -- 172.21.15.101:0/1911796560 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbd819fd30 con 0x7fbbd8100cf0 2023-07-18T03:05:27.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.739+0000 7fbbdedff700 1 -- 172.21.15.101:0/1911796560 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbd81921e0 con 0x7fbbd8100cf0 2023-07-18T03:05:27.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.741+0000 7fbbd5ffb700 1 -- 172.21.15.101:0/1911796560 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fbbc4005530 con 0x7fbbd8100cf0 2023-07-18T03:05:27.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.743+0000 7fbbd5ffb700 1 --2- 172.21.15.101:0/1911796560 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fbbc80a1220 0x7fbbc80a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:27.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.743+0000 7fbbd5ffb700 1 -- 172.21.15.101:0/1911796560 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fbbc40cddc0 con 0x7fbbd8100cf0 2023-07-18T03:05:27.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.743+0000 7fbbdcb9b700 1 --2- 172.21.15.101:0/1911796560 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fbbc80a1220 0x7fbbc80a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:27.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.744+0000 7fbbdcb9b700 1 --2- 172.21.15.101:0/1911796560 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fbbc80a1220 0x7fbbc80a36d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8100b50 tx=0x7fbbcc00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:27.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.747+0000 7fbbd5ffb700 1 -- 172.21.15.101:0/1911796560 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fbbc405a790 con 0x7fbbd8100cf0 2023-07-18T03:05:27.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.953+0000 7fbbdedff700 1 -- 172.21.15.101:0/1911796560 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbbd802c850 con 0x7fbbd8100cf0 2023-07-18T03:05:27.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.954+0000 7fbbd5ffb700 1 -- 172.21.15.101:0/1911796560 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24482 (secure 0 0 0) 0x7fbbc4097240 con 0x7fbbd8100cf0 2023-07-18T03:05:27.940 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:05:27.940 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":89,"fsid":"b1782daa-2516-11ee-9b34-001a4aab830c","created":"2023-07-18T02:57:22.972483+0000","modified":"2023-07-18T03:05:23.929113+0000","last_up_change":"2023-07-18T03:04:11.429530+0000","last_in_change":"2023-07-18T03:03:54.158340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:00:43.353489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:04:57.771616+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-18T03:04:58.769590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-18T03:05:02.358830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc76b5d1-38a0-4f1a-9335-756bfc40a62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6803","nonce":3610719929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6805","nonce":3610719929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6809","nonce":3610719929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6807","nonce":3610719929}]},"public_addr":"172.21.15.101:6803/3610719929","cluster_addr":"172.21.15.101:6805/3610719929","heartbeat_back_addr":"172.21.15.101:6809/3610719929","heartbeat_front_addr":"172.21.15.101:6807/3610719929","state":["exists","up"]},{"osd":1,"uuid":"2a299573-0b34-4af9-aaae-b523e5058ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6801","nonce":1030229579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6803","nonce":1030229579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6807","nonce":1030229579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6805","nonce":1030229579}]},"public_addr":"172.21.15.155:6801/1030229579","cluster_addr":"172.21.15.155:6803/1030229579","heartbeat_back_addr":"172.21.15.155:6807/1030229579","heartbeat_front_addr":"172.21.15.155:6805/1030229579","state":["exists","up"]},{"osd":2,"uuid":"123b3749-c636-48fa-9e33-6229b5575dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6801","nonce":1349487507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6803","nonce":1349487507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6807","nonce":1349487507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6805","nonce":1349487507}]},"public_addr":"172.21.15.179:6801/1349487507","cluster_addr":"172.21.15.179:6803/1349487507","heartbeat_back_addr":"172.21.15.179:6807/1349487507","heartbeat_front_addr":"172.21.15.179:6805/1349487507","state":["exists","up"]},{"osd":3,"uuid":"4b0c409d-2954-4127-833a-b196bf226d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6811","nonce":4226577463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6813","nonce":4226577463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6817","nonce":4226577463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6815","nonce":4226577463}]},"public_addr":"172.21.15.101:6811/4226577463","cluster_addr":"172.21.15.101:6813/4226577463","heartbeat_back_addr":"172.21.15.101:6817/4226577463","heartbeat_front_addr":"172.21.15.101:6815/4226577463","state":["exists","up"]},{"osd":4,"uuid":"2137f2df-984e-45ce-baf6-b4ac12c7e3c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6809","nonce":1951135805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6811","nonce":1951135805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6815","nonce":1951135805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6813","nonce":1951135805}]},"public_addr":"172.21.15.155:6809/1951135805","cluster_addr":"172.21.15.155:6811/1951135805","heartbeat_back_addr":"172.21.15.155:6815/1951135805","heartbeat_front_addr":"172.21.15.155:6813/1951135805","state":["exists","up"]},{"osd":5,"uuid":"1020bfc5-8c2f-482e-9588-66c7f98e46e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6809","nonce":2746037021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6811","nonce":2746037021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6815","nonce":2746037021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6813","nonce":2746037021}]},"public_addr":"172.21.15.179:6809/2746037021","cluster_addr":"172.21.15.179:6811/2746037021","heartbeat_back_addr":"172.21.15.179:6815/2746037021","heartbeat_front_addr":"172.21.15.179:6813/2746037021","state":["exists","up"]},{"osd":6,"uuid":"2cd99260-5c36-4603-9338-cf697970ce58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6819","nonce":958044931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6821","nonce":958044931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6825","nonce":958044931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6823","nonce":958044931}]},"public_addr":"172.21.15.101:6819/958044931","cluster_addr":"172.21.15.101:6821/958044931","heartbeat_back_addr":"172.21.15.101:6825/958044931","heartbeat_front_addr":"172.21.15.101:6823/958044931","state":["exists","up"]},{"osd":7,"uuid":"ed23fd0a-678f-4e15-b42c-cfe873315714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6817","nonce":3256322291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6819","nonce":3256322291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6823","nonce":3256322291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6821","nonce":3256322291}]},"public_addr":"172.21.15.155:6817/3256322291","cluster_addr":"172.21.15.155:6819/3256322291","heartbeat_back_addr":"172.21.15.155:6823/3256322291","heartbeat_front_addr":"172.21.15.155:6821/3256322291","state":["exists","up"]},{"osd":8,"uuid":"eca3d8e6-7baf-47a5-b991-328001c7ebef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6817","nonce":3520612559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6819","nonce":3520612559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6823","nonce":3520612559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6821","nonce":3520612559}]},"public_addr":"172.21.15.179:6817/3520612559","cluster_addr":"172.21.15.179:6819/3520612559","heartbeat_back_addr":"172.21.15.179:6823/3520612559","heartbeat_front_addr":"172.21.15.179:6821/3520612559","state":["exists","up"]},{"osd":9,"uuid":"465a7ffb-67e4-49ef-812c-1b96feacfebf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6827","nonce":2166787667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6829","nonce":2166787667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6833","nonce":2166787667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6831","nonce":2166787667}]},"public_addr":"172.21.15.101:6827/2166787667","cluster_addr":"172.21.15.101:6829/2166787667","heartbeat_back_addr":"172.21.15.101:6833/2166787667","heartbeat_front_addr":"172.21.15.101:6831/2166787667","state":["exists","up"]},{"osd":10,"uuid":"1897a027-f25d-434f-8df5-5fc2bd866578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6825","nonce":3682446349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6827","nonce":3682446349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6831","nonce":3682446349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6829","nonce":3682446349}]},"public_addr":"172.21.15.155:6825/3682446349","cluster_addr":"172.21.15.155:6827/3682446349","heartbeat_back_addr":"172.21.15.155:6831/3682446349","heartbeat_front_addr":"172.21.15.155:6829/3682446349","state":["exists","up"]},{"osd":11,"uuid":"b62effaf-45e2-4585-b8d0-db448f3474ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6825","nonce":2602640858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6827","nonce":2602640858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6831","nonce":2602640858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6829","nonce":2602640858}]},"public_addr":"172.21.15.179:6825/2602640858","cluster_addr":"172.21.15.179:6827/2602640858","heartbeat_back_addr":"172.21.15.179:6831/2602640858","heartbeat_front_addr":"172.21.15.179:6829/2602640858","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T02:59:56.074519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:19.358142+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:41.128966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:08.227054+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:31.442592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:52.751685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:19.251054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:40.698241+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:01.339150+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:27.955910+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:49.136980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:04:10.084982+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.101:6800/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:6801/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/604228291":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/4055187849":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/2844574324":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/1603248215":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6801/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6800/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/3307578182":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/373189792":"2023-07-19T02:57:56.243200+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:05:27.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.957+0000 7fbbdedff700 1 -- 172.21.15.101:0/1911796560 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fbbc80a1220 msgr2=0x7fbbc80a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:27.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.957+0000 7fbbdedff700 1 --2- 172.21.15.101:0/1911796560 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fbbc80a1220 0x7fbbc80a36d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8100b50 tx=0x7fbbcc00b040 comp rx=0 tx=0).stop 2023-07-18T03:05:27.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.957+0000 7fbbdedff700 1 -- 172.21.15.101:0/1911796560 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbbd8100cf0 msgr2=0x7fbbd8197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:27.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.957+0000 7fbbdedff700 1 --2- 172.21.15.101:0/1911796560 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbbd8100cf0 0x7fbbd8197f10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbbc4004760 tx=0x7fbbc4004790 comp rx=0 tx=0).stop 2023-07-18T03:05:27.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.957+0000 7fbbdedff700 1 -- 172.21.15.101:0/1911796560 shutdown_connections 2023-07-18T03:05:27.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.957+0000 7fbbdedff700 1 --2- 172.21.15.101:0/1911796560 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fbbc80a1220 0x7fbbc80a36d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:27.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.957+0000 7fbbdedff700 1 --2- 172.21.15.101:0/1911796560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbbd8101680 0x7fbbd819ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:27.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.957+0000 7fbbdedff700 1 --2- 172.21.15.101:0/1911796560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbbd80ffb00 0x7fbbd81979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:27.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.957+0000 7fbbdedff700 1 --2- 172.21.15.101:0/1911796560 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbbd8100cf0 0x7fbbd8197f10 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:27.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.957+0000 7fbbdedff700 1 -- 172.21.15.101:0/1911796560 >> 172.21.15.101:0/1911796560 conn(0x7fbbd80fb0e0 msgr2=0x7fbbd81065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:27.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.957+0000 7fbbdedff700 1 -- 172.21.15.101:0/1911796560 shutdown_connections 2023-07-18T03:05:27.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:27.957+0000 7fbbdedff700 1 -- 172.21.15.101:0/1911796560 wait complete. 2023-07-18T03:05:27.961 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-18T03:05:28.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.177+0000 7f8ae2bff700 1 -- 172.21.15.101:0/3805942071 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8adc100ce0 msgr2=0x7f8adc101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:28.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.177+0000 7f8ae2bff700 1 --2- 172.21.15.101:0/3805942071 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8adc100ce0 0x7f8adc101130 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8009990 tx=0x7f8ac8009ca0 comp rx=0 tx=0).stop 2023-07-18T03:05:28.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.177+0000 7f8ae2bff700 1 -- 172.21.15.101:0/3805942071 shutdown_connections 2023-07-18T03:05:28.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.177+0000 7f8ae2bff700 1 --2- 172.21.15.101:0/3805942071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8adc0ffaf0 0x7f8adc0fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:28.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.177+0000 7f8ae2bff700 1 --2- 172.21.15.101:0/3805942071 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8adc101670 0x7f8adc107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:28.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.177+0000 7f8ae2bff700 1 --2- 172.21.15.101:0/3805942071 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8adc100ce0 0x7f8adc101130 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:28.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.177+0000 7f8ae2bff700 1 -- 172.21.15.101:0/3805942071 >> 172.21.15.101:0/3805942071 conn(0x7f8adc0fb130 msgr2=0x7f8adc0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:28.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.177+0000 7f8ae2bff700 1 -- 172.21.15.101:0/3805942071 shutdown_connections 2023-07-18T03:05:28.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.177+0000 7f8ae2bff700 1 -- 172.21.15.101:0/3805942071 wait complete. 2023-07-18T03:05:28.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.178+0000 7f8ae2bff700 1 Processor -- start 2023-07-18T03:05:28.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.178+0000 7f8ae2bff700 1 -- start start 2023-07-18T03:05:28.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.179+0000 7f8ae2bff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8adc0ffaf0 0x7f8adc197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:28.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.179+0000 7f8ae2bff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8adc100ce0 0x7f8adc197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:28.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.179+0000 7f8ae2bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8adc101670 0x7f8adc19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:28.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.179+0000 7f8ae2bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc19f4f0 con 0x7f8adc101670 2023-07-18T03:05:28.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.179+0000 7f8ae2bff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc19f630 con 0x7f8adc100ce0 2023-07-18T03:05:28.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.179+0000 7f8ae2bff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc19f7c0 con 0x7f8adc0ffaf0 2023-07-18T03:05:28.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.179+0000 7f8ae119c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8adc101670 0x7f8adc19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:28.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.179+0000 7f8ae119c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8adc101670 0x7f8adc19ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:53328/0 (socket says 172.21.15.101:53328) 2023-07-18T03:05:28.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.179+0000 7f8ae099b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8adc0ffaf0 0x7f8adc197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:28.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.179+0000 7f8ae119c700 1 -- 172.21.15.101:0/3554691267 learned_addr learned my addr 172.21.15.101:0/3554691267 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:28.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.179+0000 7f8adbfff700 1 --2- 172.21.15.101:0/3554691267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8adc100ce0 0x7f8adc197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:28.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.179+0000 7f8ae119c700 1 -- 172.21.15.101:0/3554691267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8adc0ffaf0 msgr2=0x7f8adc197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:28.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.179+0000 7f8ae119c700 1 --2- 172.21.15.101:0/3554691267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8adc0ffaf0 0x7f8adc197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:28.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.179+0000 7f8ae119c700 1 -- 172.21.15.101:0/3554691267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8adc100ce0 msgr2=0x7f8adc197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:28.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.179+0000 7f8ae119c700 1 --2- 172.21.15.101:0/3554691267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8adc100ce0 0x7f8adc197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:28.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.179+0000 7f8ae119c700 1 -- 172.21.15.101:0/3554691267 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac80095f0 con 0x7f8adc101670 2023-07-18T03:05:28.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.180+0000 7f8ae119c700 1 --2- 172.21.15.101:0/3554691267 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8adc101670 0x7f8adc19ef20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8ad400ef30 tx=0x7f8ad400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:28.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.180+0000 7f8ad9ffb700 1 -- 172.21.15.101:0/3554691267 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4018070 con 0x7f8adc101670 2023-07-18T03:05:28.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.180+0000 7f8ae2bff700 1 -- 172.21.15.101:0/3554691267 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8adc19fa40 con 0x7f8adc101670 2023-07-18T03:05:28.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.180+0000 7f8ad9ffb700 1 -- 172.21.15.101:0/3554691267 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ad400f040 con 0x7f8adc101670 2023-07-18T03:05:28.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.180+0000 7f8ad9ffb700 1 -- 172.21.15.101:0/3554691267 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad400cd00 con 0x7f8adc101670 2023-07-18T03:05:28.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.180+0000 7f8ae2bff700 1 -- 172.21.15.101:0/3554691267 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8adc19ff50 con 0x7f8adc101670 2023-07-18T03:05:28.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.181+0000 7f8ae2bff700 1 -- 172.21.15.101:0/3554691267 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8adc106720 con 0x7f8adc101670 2023-07-18T03:05:28.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.183+0000 7f8ad9ffb700 1 -- 172.21.15.101:0/3554691267 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f8ad4003a00 con 0x7f8adc101670 2023-07-18T03:05:28.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.184+0000 7f8ad9ffb700 1 --2- 172.21.15.101:0/3554691267 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8acc0a1150 0x7f8acc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:28.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.184+0000 7f8ad9ffb700 1 -- 172.21.15.101:0/3554691267 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f8ad40cc760 con 0x7f8adc101670 2023-07-18T03:05:28.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.184+0000 7f8ae099b700 1 --2- 172.21.15.101:0/3554691267 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8acc0a1150 0x7f8acc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:28.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.185+0000 7f8ae099b700 1 --2- 172.21.15.101:0/3554691267 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8acc0a1150 0x7f8acc0a3600 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8adc100b40 tx=0x7f8ad0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:28.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.188+0000 7f8ad9ffb700 1 -- 172.21.15.101:0/3554691267 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8ad4095c40 con 0x7f8adc101670 2023-07-18T03:05:28.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:27 smithi179 ceph-mon[129229]: pgmap v291: 129 pgs: 1 activating, 1 peering, 127 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2023-07-18T03:05:28.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:27 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/4233796821' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:05:28.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:27 smithi101 ceph-mon[126379]: pgmap v291: 129 pgs: 1 activating, 1 peering, 127 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2023-07-18T03:05:28.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:27 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/4233796821' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:05:28.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:27 smithi155 ceph-mon[128885]: pgmap v291: 129 pgs: 1 activating, 1 peering, 127 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2023-07-18T03:05:28.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:27 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/4233796821' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:05:28.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.431+0000 7f8ae2bff700 1 -- 172.21.15.101:0/3554691267 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8adc02c850 con 0x7f8adc101670 2023-07-18T03:05:28.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.960+0000 7f8ad9ffb700 1 -- 172.21.15.101:0/3554691267 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8ad4099260 con 0x7f8adc101670 2023-07-18T03:05:28.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.963+0000 7f8ae2bff700 1 -- 172.21.15.101:0/3554691267 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8acc0a1150 msgr2=0x7f8acc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:28.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.963+0000 7f8ae2bff700 1 --2- 172.21.15.101:0/3554691267 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8acc0a1150 0x7f8acc0a3600 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8adc100b40 tx=0x7f8ad0009260 comp rx=0 tx=0).stop 2023-07-18T03:05:28.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.963+0000 7f8ae2bff700 1 -- 172.21.15.101:0/3554691267 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8adc101670 msgr2=0x7f8adc19ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:28.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.963+0000 7f8ae2bff700 1 --2- 172.21.15.101:0/3554691267 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8adc101670 0x7f8adc19ef20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8ad400ef30 tx=0x7f8ad400c3a0 comp rx=0 tx=0).stop 2023-07-18T03:05:28.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.963+0000 7f8ae2bff700 1 -- 172.21.15.101:0/3554691267 shutdown_connections 2023-07-18T03:05:28.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.963+0000 7f8ae2bff700 1 --2- 172.21.15.101:0/3554691267 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8acc0a1150 0x7f8acc0a3600 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:28.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.963+0000 7f8ae2bff700 1 --2- 172.21.15.101:0/3554691267 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8adc101670 0x7f8adc19ef20 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:28.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.963+0000 7f8ae2bff700 1 --2- 172.21.15.101:0/3554691267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8adc0ffaf0 0x7f8adc197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:28.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.963+0000 7f8ae2bff700 1 --2- 172.21.15.101:0/3554691267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8adc100ce0 0x7f8adc197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:28.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.963+0000 7f8ae2bff700 1 -- 172.21.15.101:0/3554691267 >> 172.21.15.101:0/3554691267 conn(0x7f8adc0fb130 msgr2=0x7f8adc0fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:28.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.963+0000 7f8ae2bff700 1 -- 172.21.15.101:0/3554691267 shutdown_connections 2023-07-18T03:05:28.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:28.964+0000 7f8ae2bff700 1 -- 172.21.15.101:0/3554691267 wait complete. 2023-07-18T03:05:28.968 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2023-07-18T03:05:28.969 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2023-07-18T03:05:29.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.183+0000 7fc3fcbff700 1 -- 172.21.15.101:0/987639618 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3f8072f70 msgr2=0x7fc3f80733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:29.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.183+0000 7fc3fcbff700 1 --2- 172.21.15.101:0/987639618 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3f8072f70 0x7fc3f80733e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec009990 tx=0x7fc3ec009ca0 comp rx=0 tx=0).stop 2023-07-18T03:05:29.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.183+0000 7fc3fcbff700 1 -- 172.21.15.101:0/987639618 shutdown_connections 2023-07-18T03:05:29.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.183+0000 7fc3fcbff700 1 --2- 172.21.15.101:0/987639618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3f8074550 0x7fc3f80729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:29.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.183+0000 7fc3fcbff700 1 --2- 172.21.15.101:0/987639618 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3f81079a0 0x7fc3f8109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:29.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.183+0000 7fc3fcbff700 1 --2- 172.21.15.101:0/987639618 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3f8072f70 0x7fc3f80733e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:29.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.183+0000 7fc3fcbff700 1 -- 172.21.15.101:0/987639618 >> 172.21.15.101:0/987639618 conn(0x7fc3f80fb0e0 msgr2=0x7fc3f80fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:29.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.183+0000 7fc3fcbff700 1 -- 172.21.15.101:0/987639618 shutdown_connections 2023-07-18T03:05:29.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.183+0000 7fc3fcbff700 1 -- 172.21.15.101:0/987639618 wait complete. 2023-07-18T03:05:29.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.184+0000 7fc3fcbff700 1 Processor -- start 2023-07-18T03:05:29.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.185+0000 7fc3fcbff700 1 -- start start 2023-07-18T03:05:29.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.185+0000 7fc3fcbff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3f8072f70 0x7fc3f8197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:29.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.185+0000 7fc3fcbff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3f8074550 0x7fc3f8197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:29.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.185+0000 7fc3fcbff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3f81079a0 0x7fc3f819f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:29.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.185+0000 7fc3fcbff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f819f600 con 0x7fc3f8074550 2023-07-18T03:05:29.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.185+0000 7fc3fcbff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f819f740 con 0x7fc3f81079a0 2023-07-18T03:05:29.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.185+0000 7fc3fcbff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f819f8d0 con 0x7fc3f8072f70 2023-07-18T03:05:29.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.185+0000 7fc3f5d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3f8074550 0x7fc3f8197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:29.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.185+0000 7fc3f5d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3f8074550 0x7fc3f8197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:53346/0 (socket says 172.21.15.101:53346) 2023-07-18T03:05:29.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.185+0000 7fc3f5d9b700 1 -- 172.21.15.101:0/2667965850 learned_addr learned my addr 172.21.15.101:0/2667965850 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:29.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.185+0000 7fc3f6d9d700 1 --2- 172.21.15.101:0/2667965850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3f81079a0 0x7fc3f819f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:29.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.185+0000 7fc3f659c700 1 --2- 172.21.15.101:0/2667965850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3f8072f70 0x7fc3f8197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:29.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.186+0000 7fc3f5d9b700 1 -- 172.21.15.101:0/2667965850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3f8072f70 msgr2=0x7fc3f8197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:29.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.186+0000 7fc3f5d9b700 1 --2- 172.21.15.101:0/2667965850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3f8072f70 0x7fc3f8197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:29.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.186+0000 7fc3f5d9b700 1 -- 172.21.15.101:0/2667965850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3f81079a0 msgr2=0x7fc3f819f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:29.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.186+0000 7fc3f5d9b700 1 --2- 172.21.15.101:0/2667965850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3f81079a0 0x7fc3f819f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:29.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.186+0000 7fc3f5d9b700 1 -- 172.21.15.101:0/2667965850 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3ec0095f0 con 0x7fc3f8074550 2023-07-18T03:05:29.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.186+0000 7fc3f659c700 1 --2- 172.21.15.101:0/2667965850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3f8072f70 0x7fc3f8197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:05:29.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.186+0000 7fc3f5d9b700 1 --2- 172.21.15.101:0/2667965850 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3f8074550 0x7fc3f8197fc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec000c00 tx=0x7fc3ec005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:29.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.186+0000 7fc3e77fe700 1 -- 172.21.15.101:0/2667965850 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec01d070 con 0x7fc3f8074550 2023-07-18T03:05:29.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.186+0000 7fc3e77fe700 1 -- 172.21.15.101:0/2667965850 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3ec022470 con 0x7fc3f8074550 2023-07-18T03:05:29.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.186+0000 7fc3fcbff700 1 -- 172.21.15.101:0/2667965850 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3f819fb50 con 0x7fc3f8074550 2023-07-18T03:05:29.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.186+0000 7fc3e77fe700 1 -- 172.21.15.101:0/2667965850 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec017650 con 0x7fc3f8074550 2023-07-18T03:05:29.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.186+0000 7fc3fcbff700 1 -- 172.21.15.101:0/2667965850 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3f819ff70 con 0x7fc3f8074550 2023-07-18T03:05:29.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.187+0000 7fc3fcbff700 1 -- 172.21.15.101:0/2667965850 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3f8106050 con 0x7fc3f8074550 2023-07-18T03:05:29.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.189+0000 7fc3e77fe700 1 -- 172.21.15.101:0/2667965850 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fc3ec0225e0 con 0x7fc3f8074550 2023-07-18T03:05:29.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.190+0000 7fc3e77fe700 1 --2- 172.21.15.101:0/2667965850 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc3e00a1150 0x7fc3e00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:29.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.191+0000 7fc3e77fe700 1 -- 172.21.15.101:0/2667965850 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fc3ec0ce450 con 0x7fc3f8074550 2023-07-18T03:05:29.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.191+0000 7fc3f659c700 1 --2- 172.21.15.101:0/2667965850 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc3e00a1150 0x7fc3e00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:29.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.191+0000 7fc3f659c700 1 --2- 172.21.15.101:0/2667965850 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc3e00a1150 0x7fc3e00a3600 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8074920 tx=0x7fc3e8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:29.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.193+0000 7fc3e77fe700 1 -- 172.21.15.101:0/2667965850 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc3ec0978d0 con 0x7fc3f8074550 2023-07-18T03:05:29.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:28 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1911796560' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:05:29.191 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:28 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/3554691267' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-18T03:05:29.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:28 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1911796560' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:05:29.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:28 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3554691267' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-18T03:05:29.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:28 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1911796560' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:05:29.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:28 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/3554691267' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-18T03:05:29.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.438+0000 7fc3fcbff700 1 -- 172.21.15.101:0/2667965850 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7fc3f802cce0 con 0x7fc3f8074550 2023-07-18T03:05:29.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.971+0000 7fc3e77fe700 1 -- 172.21.15.101:0/2667965850 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7fc3ec09aef0 con 0x7fc3f8074550 2023-07-18T03:05:29.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.974+0000 7fc3fcbff700 1 -- 172.21.15.101:0/2667965850 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc3e00a1150 msgr2=0x7fc3e00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:29.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.974+0000 7fc3fcbff700 1 --2- 172.21.15.101:0/2667965850 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc3e00a1150 0x7fc3e00a3600 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8074920 tx=0x7fc3e8009260 comp rx=0 tx=0).stop 2023-07-18T03:05:29.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.974+0000 7fc3fcbff700 1 -- 172.21.15.101:0/2667965850 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3f8074550 msgr2=0x7fc3f8197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:29.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.974+0000 7fc3fcbff700 1 --2- 172.21.15.101:0/2667965850 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3f8074550 0x7fc3f8197fc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec000c00 tx=0x7fc3ec005720 comp rx=0 tx=0).stop 2023-07-18T03:05:29.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.975+0000 7fc3fcbff700 1 -- 172.21.15.101:0/2667965850 shutdown_connections 2023-07-18T03:05:29.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.975+0000 7fc3fcbff700 1 --2- 172.21.15.101:0/2667965850 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc3e00a1150 0x7fc3e00a3600 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:29.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.975+0000 7fc3fcbff700 1 --2- 172.21.15.101:0/2667965850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3f81079a0 0x7fc3f819f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:29.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.975+0000 7fc3fcbff700 1 --2- 172.21.15.101:0/2667965850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3f8072f70 0x7fc3f8197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:29.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.975+0000 7fc3fcbff700 1 --2- 172.21.15.101:0/2667965850 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3f8074550 0x7fc3f8197fc0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:29.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.975+0000 7fc3fcbff700 1 -- 172.21.15.101:0/2667965850 >> 172.21.15.101:0/2667965850 conn(0x7fc3f80fb0e0 msgr2=0x7fc3f80fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:29.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.975+0000 7fc3fcbff700 1 -- 172.21.15.101:0/2667965850 shutdown_connections 2023-07-18T03:05:29.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:29.975+0000 7fc3fcbff700 1 -- 172.21.15.101:0/2667965850 wait complete. 2023-07-18T03:05:29.984 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-07-18T03:05:30.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.203+0000 7f4464dff700 1 -- 172.21.15.101:0/3876471132 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f44600fe3b0 msgr2=0x7f44600fe7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:30.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.203+0000 7f4464dff700 1 --2- 172.21.15.101:0/3876471132 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f44600fe3b0 0x7f44600fe7c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4450009990 tx=0x7f4450009ca0 comp rx=0 tx=0).stop 2023-07-18T03:05:30.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.203+0000 7f4464dff700 1 -- 172.21.15.101:0/3876471132 shutdown_connections 2023-07-18T03:05:30.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.203+0000 7f4464dff700 1 --2- 172.21.15.101:0/3876471132 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44600ff6b0 0x7f44600ffb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:30.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.203+0000 7f4464dff700 1 --2- 172.21.15.101:0/3876471132 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f44600fe3b0 0x7f44600fe7c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:30.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.203+0000 7f4464dff700 1 --2- 172.21.15.101:0/3876471132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44600fed00 0x7f44600ff170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:30.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.203+0000 7f4464dff700 1 -- 172.21.15.101:0/3876471132 >> 172.21.15.101:0/3876471132 conn(0x7f44600fa000 msgr2=0x7f44600fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:30.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.203+0000 7f4464dff700 1 -- 172.21.15.101:0/3876471132 shutdown_connections 2023-07-18T03:05:30.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.203+0000 7f4464dff700 1 -- 172.21.15.101:0/3876471132 wait complete. 2023-07-18T03:05:30.192 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.204+0000 7f4464dff700 1 Processor -- start 2023-07-18T03:05:30.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.204+0000 7f4464dff700 1 -- start start 2023-07-18T03:05:30.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.204+0000 7f4464dff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44600fe3b0 0x7f4460197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:30.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.204+0000 7f4464dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f44600fed00 0x7f4460197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:30.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.204+0000 7f4464dff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44600ff6b0 0x7f446019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:30.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.205+0000 7f4464dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446019f5b0 con 0x7f44600fed00 2023-07-18T03:05:30.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.205+0000 7f4464dff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446019f6f0 con 0x7f44600fe3b0 2023-07-18T03:05:30.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.205+0000 7f4464dff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446019f880 con 0x7f44600ff6b0 2023-07-18T03:05:30.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.205+0000 7f445e59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44600fe3b0 0x7f4460197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:30.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.205+0000 7f4457fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f44600fed00 0x7f4460197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:30.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.205+0000 7f445e59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44600fe3b0 0x7f4460197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:51660/0 (socket says 172.21.15.101:51660) 2023-07-18T03:05:30.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.205+0000 7f445e59c700 1 -- 172.21.15.101:0/779421972 learned_addr learned my addr 172.21.15.101:0/779421972 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:30.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.205+0000 7f445ed9d700 1 --2- 172.21.15.101:0/779421972 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44600ff6b0 0x7f446019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:30.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.205+0000 7f445e59c700 1 -- 172.21.15.101:0/779421972 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44600ff6b0 msgr2=0x7f446019efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:30.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.205+0000 7f445e59c700 1 --2- 172.21.15.101:0/779421972 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44600ff6b0 0x7f446019efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:30.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.205+0000 7f445e59c700 1 -- 172.21.15.101:0/779421972 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f44600fed00 msgr2=0x7f4460197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:30.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.205+0000 7f445e59c700 1 --2- 172.21.15.101:0/779421972 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f44600fed00 0x7f4460197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:30.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.205+0000 7f445e59c700 1 -- 172.21.15.101:0/779421972 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44500095f0 con 0x7f44600fe3b0 2023-07-18T03:05:30.198 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.205+0000 7f4457fff700 1 --2- 172.21.15.101:0/779421972 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f44600fed00 0x7f4460197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:05:30.198 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.205+0000 7f445e59c700 1 --2- 172.21.15.101:0/779421972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44600fe3b0 0x7f4460197a30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4450000c00 tx=0x7f445000f460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:30.198 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.206+0000 7f44577fe700 1 -- 172.21.15.101:0/779421972 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445001d070 con 0x7f44600fe3b0 2023-07-18T03:05:30.198 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.206+0000 7f4464dff700 1 -- 172.21.15.101:0/779421972 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f446019fb00 con 0x7f44600fe3b0 2023-07-18T03:05:30.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.206+0000 7f44577fe700 1 -- 172.21.15.101:0/779421972 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f445000fc00 con 0x7f44600fe3b0 2023-07-18T03:05:30.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.206+0000 7f44577fe700 1 -- 172.21.15.101:0/779421972 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4450017650 con 0x7f44600fe3b0 2023-07-18T03:05:30.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.206+0000 7f4464dff700 1 -- 172.21.15.101:0/779421972 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f446019ffb0 con 0x7f44600fe3b0 2023-07-18T03:05:30.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.207+0000 7f4464dff700 1 -- 172.21.15.101:0/779421972 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4460109190 con 0x7f44600fe3b0 2023-07-18T03:05:30.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.209+0000 7f44577fe700 1 -- 172.21.15.101:0/779421972 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f4450021410 con 0x7f44600fe3b0 2023-07-18T03:05:30.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.210+0000 7f44577fe700 1 --2- 172.21.15.101:0/779421972 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f444c0a1150 0x7f444c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:30.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.211+0000 7f44577fe700 1 -- 172.21.15.101:0/779421972 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f44500ce8b0 con 0x7f44600fe3b0 2023-07-18T03:05:30.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.211+0000 7f4457fff700 1 --2- 172.21.15.101:0/779421972 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f444c0a1150 0x7f444c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:30.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.211+0000 7f4457fff700 1 --2- 172.21.15.101:0/779421972 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f444c0a1150 0x7f444c0a3600 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4448011490 tx=0x7f4448009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:30.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.212+0000 7f44577fe700 1 -- 172.21.15.101:0/779421972 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4450097d30 con 0x7f44600fe3b0 2023-07-18T03:05:30.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:29 smithi101 ceph-mon[126379]: pgmap v292: 129 pgs: 1 activating, 128 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2023-07-18T03:05:30.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:29 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3554691267' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-07-18T03:05:30.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:29 smithi101 ceph-mon[126379]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-07-18T03:05:30.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:29 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2667965850' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-07-18T03:05:30.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:29 smithi155 ceph-mon[128885]: pgmap v292: 129 pgs: 1 activating, 128 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2023-07-18T03:05:30.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:29 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/3554691267' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-07-18T03:05:30.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:29 smithi155 ceph-mon[128885]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-07-18T03:05:30.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:29 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/2667965850' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-07-18T03:05:30.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.456+0000 7f4464dff700 1 -- 172.21.15.101:0/779421972 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f446002c850 con 0x7f44600fe3b0 2023-07-18T03:05:30.441 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:29 smithi179 ceph-mon[129229]: pgmap v292: 129 pgs: 1 activating, 128 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2023-07-18T03:05:30.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:29 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/3554691267' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-07-18T03:05:30.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:29 smithi179 ceph-mon[129229]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-07-18T03:05:30.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:29 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/2667965850' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-07-18T03:05:30.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.977+0000 7f44577fe700 1 -- 172.21.15.101:0/779421972 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) v1 ==== 121+0+0 (secure 0 0 0) 0x7f445009b350 con 0x7f44600fe3b0 2023-07-18T03:05:30.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.979+0000 7f4464dff700 1 -- 172.21.15.101:0/779421972 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f444c0a1150 msgr2=0x7f444c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:30.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.979+0000 7f4464dff700 1 --2- 172.21.15.101:0/779421972 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f444c0a1150 0x7f444c0a3600 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4448011490 tx=0x7f4448009260 comp rx=0 tx=0).stop 2023-07-18T03:05:30.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.979+0000 7f4464dff700 1 -- 172.21.15.101:0/779421972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44600fe3b0 msgr2=0x7f4460197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:30.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.979+0000 7f4464dff700 1 --2- 172.21.15.101:0/779421972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44600fe3b0 0x7f4460197a30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4450000c00 tx=0x7f445000f460 comp rx=0 tx=0).stop 2023-07-18T03:05:30.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.979+0000 7f4464dff700 1 -- 172.21.15.101:0/779421972 shutdown_connections 2023-07-18T03:05:30.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.979+0000 7f4464dff700 1 --2- 172.21.15.101:0/779421972 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f444c0a1150 0x7f444c0a3600 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:30.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.979+0000 7f4464dff700 1 --2- 172.21.15.101:0/779421972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44600fe3b0 0x7f4460197a30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:30.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.979+0000 7f4464dff700 1 --2- 172.21.15.101:0/779421972 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44600ff6b0 0x7f446019efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:30.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.979+0000 7f4464dff700 1 --2- 172.21.15.101:0/779421972 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f44600fed00 0x7f4460197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:30.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.979+0000 7f4464dff700 1 -- 172.21.15.101:0/779421972 >> 172.21.15.101:0/779421972 conn(0x7f44600fa000 msgr2=0x7f44600fbd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:30.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.979+0000 7f4464dff700 1 -- 172.21.15.101:0/779421972 shutdown_connections 2023-07-18T03:05:30.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:30.980+0000 7f4464dff700 1 -- 172.21.15.101:0/779421972 wait complete. 2023-07-18T03:05:30.988 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-07-18T03:05:31.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.204+0000 7f7d821ff700 1 -- 172.21.15.101:0/2663512341 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7d7c101660 msgr2=0x7f7d7c107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:31.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.204+0000 7f7d821ff700 1 --2- 172.21.15.101:0/2663512341 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7d7c101660 0x7f7d7c107f20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7d7400b0a0 tx=0x7f7d7400b3b0 comp rx=0 tx=0).stop 2023-07-18T03:05:31.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.204+0000 7f7d821ff700 1 -- 172.21.15.101:0/2663512341 shutdown_connections 2023-07-18T03:05:31.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.204+0000 7f7d821ff700 1 --2- 172.21.15.101:0/2663512341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d7c0ffae0 0x7f7d7c0ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:31.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.204+0000 7f7d821ff700 1 --2- 172.21.15.101:0/2663512341 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7d7c101660 0x7f7d7c107f20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:31.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.204+0000 7f7d821ff700 1 --2- 172.21.15.101:0/2663512341 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d7c100cd0 0x7f7d7c101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:31.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.204+0000 7f7d821ff700 1 -- 172.21.15.101:0/2663512341 >> 172.21.15.101:0/2663512341 conn(0x7f7d7c0fb0e0 msgr2=0x7f7d7c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:31.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.204+0000 7f7d821ff700 1 -- 172.21.15.101:0/2663512341 shutdown_connections 2023-07-18T03:05:31.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.204+0000 7f7d821ff700 1 -- 172.21.15.101:0/2663512341 wait complete. 2023-07-18T03:05:31.192 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.205+0000 7f7d821ff700 1 Processor -- start 2023-07-18T03:05:31.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.205+0000 7f7d821ff700 1 -- start start 2023-07-18T03:05:31.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.206+0000 7f7d821ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d7c0ffae0 0x7f7d7c0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:31.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.206+0000 7f7d821ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7d7c100cd0 0x7f7d7c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:31.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.206+0000 7f7d821ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d7c101660 0x7f7d7c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:31.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.206+0000 7f7d821ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c0735e0 con 0x7f7d7c100cd0 2023-07-18T03:05:31.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.206+0000 7f7d821ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c073750 con 0x7f7d7c101660 2023-07-18T03:05:31.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.206+0000 7f7d821ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c19f9b0 con 0x7f7d7c0ffae0 2023-07-18T03:05:31.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.206+0000 7f7d7affd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7d7c100cd0 0x7f7d7c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:31.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.206+0000 7f7d7affd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7d7c100cd0 0x7f7d7c072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:53368/0 (socket says 172.21.15.101:53368) 2023-07-18T03:05:31.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.206+0000 7f7d7affd700 1 -- 172.21.15.101:0/2865994685 learned_addr learned my addr 172.21.15.101:0/2865994685 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:31.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.206+0000 7f7d7b7fe700 1 --2- 172.21.15.101:0/2865994685 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d7c0ffae0 0x7f7d7c0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:31.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.206+0000 7f7d7bfff700 1 --2- 172.21.15.101:0/2865994685 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d7c101660 0x7f7d7c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:31.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.206+0000 7f7d7affd700 1 -- 172.21.15.101:0/2865994685 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d7c0ffae0 msgr2=0x7f7d7c0742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:31.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.206+0000 7f7d7affd700 1 --2- 172.21.15.101:0/2865994685 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d7c0ffae0 0x7f7d7c0742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:31.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.206+0000 7f7d7affd700 1 -- 172.21.15.101:0/2865994685 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d7c101660 msgr2=0x7f7d7c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:31.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.206+0000 7f7d7affd700 1 --2- 172.21.15.101:0/2865994685 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d7c101660 0x7f7d7c072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:31.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.206+0000 7f7d7affd700 1 -- 172.21.15.101:0/2865994685 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d74009d00 con 0x7f7d7c100cd0 2023-07-18T03:05:31.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.206+0000 7f7d7b7fe700 1 --2- 172.21.15.101:0/2865994685 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d7c0ffae0 0x7f7d7c0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:05:31.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.206+0000 7f7d7bfff700 1 --2- 172.21.15.101:0/2865994685 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d7c101660 0x7f7d7c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:05:31.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.207+0000 7f7d7affd700 1 --2- 172.21.15.101:0/2865994685 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7d7c100cd0 0x7f7d7c072900 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7d7000e9a0 tx=0x7f7d7000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:31.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.207+0000 7f7d78ff9700 1 -- 172.21.15.101:0/2865994685 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d70009790 con 0x7f7d7c100cd0 2023-07-18T03:05:31.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.207+0000 7f7d78ff9700 1 -- 172.21.15.101:0/2865994685 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d70010430 con 0x7f7d7c100cd0 2023-07-18T03:05:31.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.207+0000 7f7d821ff700 1 -- 172.21.15.101:0/2865994685 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d7c19fc90 con 0x7f7d7c100cd0 2023-07-18T03:05:31.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.207+0000 7f7d78ff9700 1 -- 172.21.15.101:0/2865994685 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d70018680 con 0x7f7d7c100cd0 2023-07-18T03:05:31.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.207+0000 7f7d821ff700 1 -- 172.21.15.101:0/2865994685 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d7c1a0170 con 0x7f7d7c100cd0 2023-07-18T03:05:31.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.208+0000 7f7d821ff700 1 -- 172.21.15.101:0/2865994685 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d7c107000 con 0x7f7d7c100cd0 2023-07-18T03:05:31.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.210+0000 7f7d78ff9700 1 -- 172.21.15.101:0/2865994685 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f7d700105a0 con 0x7f7d7c100cd0 2023-07-18T03:05:31.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.211+0000 7f7d78ff9700 1 --2- 172.21.15.101:0/2865994685 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f7d640a1150 0x7f7d640a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:31.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.211+0000 7f7d78ff9700 1 -- 172.21.15.101:0/2865994685 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f7d70014070 con 0x7f7d7c100cd0 2023-07-18T03:05:31.198 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.211+0000 7f7d7b7fe700 1 --2- 172.21.15.101:0/2865994685 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f7d640a1150 0x7f7d640a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:31.198 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.212+0000 7f7d7b7fe700 1 --2- 172.21.15.101:0/2865994685 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f7d640a1150 0x7f7d640a3600 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c100b30 tx=0x7f7d6c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:31.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.215+0000 7f7d78ff9700 1 -- 172.21.15.101:0/2865994685 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7d70051790 con 0x7f7d7c100cd0 2023-07-18T03:05:31.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:30 smithi101 ceph-mon[126379]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-07-18T03:05:31.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:30 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2667965850' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-07-18T03:05:31.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:30 smithi101 ceph-mon[126379]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-07-18T03:05:31.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:30 smithi101 ceph-mon[126379]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-07-18T03:05:31.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:30 smithi101 ceph-mon[126379]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2023-07-18T03:05:31.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:30 smithi101 ceph-mon[126379]: daemon mds.b is now active in filesystem cephfs as rank 1 2023-07-18T03:05:31.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:30 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/779421972' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-07-18T03:05:31.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:30 smithi101 ceph-mon[126379]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-07-18T03:05:31.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:30 smithi155 ceph-mon[128885]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-07-18T03:05:31.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:30 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/2667965850' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-07-18T03:05:31.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:30 smithi155 ceph-mon[128885]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-07-18T03:05:31.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:30 smithi155 ceph-mon[128885]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-07-18T03:05:31.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:30 smithi155 ceph-mon[128885]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2023-07-18T03:05:31.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:30 smithi155 ceph-mon[128885]: daemon mds.b is now active in filesystem cephfs as rank 1 2023-07-18T03:05:31.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:30 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/779421972' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-07-18T03:05:31.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:30 smithi155 ceph-mon[128885]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-07-18T03:05:31.441 INFO:journalctl@ceph.mds.i.smithi179.stdout:Jul 18 03:05:31 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-i[155309]: 2023-07-18T03:05:31.055+0000 7f20e6b1c700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-07-18T03:05:31.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:30 smithi179 ceph-mon[129229]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-07-18T03:05:31.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:30 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/2667965850' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-07-18T03:05:31.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:30 smithi179 ceph-mon[129229]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-07-18T03:05:31.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:30 smithi179 ceph-mon[129229]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-07-18T03:05:31.443 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:30 smithi179 ceph-mon[129229]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2023-07-18T03:05:31.443 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:30 smithi179 ceph-mon[129229]: daemon mds.b is now active in filesystem cephfs as rank 1 2023-07-18T03:05:31.443 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:30 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/779421972' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-07-18T03:05:31.443 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:30 smithi179 ceph-mon[129229]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-07-18T03:05:31.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.458+0000 7f7d821ff700 1 -- 172.21.15.101:0/2865994685 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f7d7c02c850 con 0x7f7d7c100cd0 2023-07-18T03:05:31.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.984+0000 7f7d78ff9700 1 -- 172.21.15.101:0/2865994685 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) v1 ==== 115+0+0 (secure 0 0 0) 0x7f7d70096210 con 0x7f7d7c100cd0 2023-07-18T03:05:31.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.987+0000 7f7d821ff700 1 -- 172.21.15.101:0/2865994685 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f7d640a1150 msgr2=0x7f7d640a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:31.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.987+0000 7f7d821ff700 1 --2- 172.21.15.101:0/2865994685 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f7d640a1150 0x7f7d640a3600 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c100b30 tx=0x7f7d6c009260 comp rx=0 tx=0).stop 2023-07-18T03:05:31.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.987+0000 7f7d821ff700 1 -- 172.21.15.101:0/2865994685 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7d7c100cd0 msgr2=0x7f7d7c072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:31.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.987+0000 7f7d821ff700 1 --2- 172.21.15.101:0/2865994685 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7d7c100cd0 0x7f7d7c072900 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7d7000e9a0 tx=0x7f7d7000ecb0 comp rx=0 tx=0).stop 2023-07-18T03:05:31.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.987+0000 7f7d821ff700 1 -- 172.21.15.101:0/2865994685 shutdown_connections 2023-07-18T03:05:31.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.987+0000 7f7d821ff700 1 --2- 172.21.15.101:0/2865994685 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f7d640a1150 0x7f7d640a3600 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:31.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.987+0000 7f7d821ff700 1 --2- 172.21.15.101:0/2865994685 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d7c101660 0x7f7d7c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:31.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.987+0000 7f7d821ff700 1 --2- 172.21.15.101:0/2865994685 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d7c0ffae0 0x7f7d7c0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:31.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.987+0000 7f7d821ff700 1 --2- 172.21.15.101:0/2865994685 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7d7c100cd0 0x7f7d7c072900 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:31.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.987+0000 7f7d821ff700 1 -- 172.21.15.101:0/2865994685 >> 172.21.15.101:0/2865994685 conn(0x7f7d7c0fb0e0 msgr2=0x7f7d7c104140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:31.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.987+0000 7f7d821ff700 1 -- 172.21.15.101:0/2865994685 shutdown_connections 2023-07-18T03:05:31.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:31.987+0000 7f7d821ff700 1 -- 172.21.15.101:0/2865994685 wait complete. 2023-07-18T03:05:31.992 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2023-07-18T03:05:31.993 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-07-18T03:05:32.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.210+0000 7f102b3ff700 1 -- 172.21.15.101:0/1816996410 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f10240ffaf0 msgr2=0x7f10240fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:32.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.210+0000 7f102b3ff700 1 --2- 172.21.15.101:0/1816996410 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f10240ffaf0 0x7f10240fff00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1020009990 tx=0x7f1020009ca0 comp rx=0 tx=0).stop 2023-07-18T03:05:32.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.210+0000 7f102b3ff700 1 -- 172.21.15.101:0/1816996410 shutdown_connections 2023-07-18T03:05:32.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.210+0000 7f102b3ff700 1 --2- 172.21.15.101:0/1816996410 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1024101670 0x7f1024107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:32.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.210+0000 7f102b3ff700 1 --2- 172.21.15.101:0/1816996410 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f10240ffaf0 0x7f10240fff00 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:32.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.210+0000 7f102b3ff700 1 --2- 172.21.15.101:0/1816996410 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1024100ce0 0x7f1024101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:32.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.210+0000 7f102b3ff700 1 -- 172.21.15.101:0/1816996410 >> 172.21.15.101:0/1816996410 conn(0x7f10240fb130 msgr2=0x7f10240fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:32.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.210+0000 7f102b3ff700 1 -- 172.21.15.101:0/1816996410 shutdown_connections 2023-07-18T03:05:32.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.210+0000 7f102b3ff700 1 -- 172.21.15.101:0/1816996410 wait complete. 2023-07-18T03:05:32.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.211+0000 7f102b3ff700 1 Processor -- start 2023-07-18T03:05:32.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.211+0000 7f102b3ff700 1 -- start start 2023-07-18T03:05:32.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.212+0000 7f102b3ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10240ffaf0 0x7f1024195880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:32.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.212+0000 7f102b3ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1024100ce0 0x7f1024195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:32.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.212+0000 7f102b3ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1024101670 0x7f102419ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:32.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.212+0000 7f102b3ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102419d400 con 0x7f1024101670 2023-07-18T03:05:32.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.212+0000 7f102b3ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102419d540 con 0x7f10240ffaf0 2023-07-18T03:05:32.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.212+0000 7f102b3ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102419d6d0 con 0x7f1024100ce0 2023-07-18T03:05:32.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.212+0000 7f102999c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1024101670 0x7f102419ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:32.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.212+0000 7f102999c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1024101670 0x7f102419ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:53384/0 (socket says 172.21.15.101:53384) 2023-07-18T03:05:32.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.212+0000 7f102999c700 1 -- 172.21.15.101:0/2634520460 learned_addr learned my addr 172.21.15.101:0/2634520460 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:32.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.212+0000 7f102919b700 1 --2- 172.21.15.101:0/2634520460 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10240ffaf0 0x7f1024195880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:32.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.212+0000 7f102899a700 1 --2- 172.21.15.101:0/2634520460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1024100ce0 0x7f1024195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:32.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.212+0000 7f102999c700 1 -- 172.21.15.101:0/2634520460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1024100ce0 msgr2=0x7f1024195dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:32.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.212+0000 7f102999c700 1 --2- 172.21.15.101:0/2634520460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1024100ce0 0x7f1024195dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:32.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.212+0000 7f102999c700 1 -- 172.21.15.101:0/2634520460 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10240ffaf0 msgr2=0x7f1024195880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:32.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.212+0000 7f102999c700 1 --2- 172.21.15.101:0/2634520460 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10240ffaf0 0x7f1024195880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:32.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.212+0000 7f102999c700 1 -- 172.21.15.101:0/2634520460 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10200095f0 con 0x7f1024101670 2023-07-18T03:05:32.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.212+0000 7f102899a700 1 --2- 172.21.15.101:0/2634520460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1024100ce0 0x7f1024195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:05:32.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.213+0000 7f102999c700 1 --2- 172.21.15.101:0/2634520460 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1024101670 0x7f102419ce30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f101c00ef30 tx=0x7f101c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:32.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.213+0000 7f101a7fc700 1 -- 172.21.15.101:0/2634520460 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101c018070 con 0x7f1024101670 2023-07-18T03:05:32.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.213+0000 7f101a7fc700 1 -- 172.21.15.101:0/2634520460 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f101c00f040 con 0x7f1024101670 2023-07-18T03:05:32.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.213+0000 7f101a7fc700 1 -- 172.21.15.101:0/2634520460 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101c00cdb0 con 0x7f1024101670 2023-07-18T03:05:32.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.213+0000 7f102b3ff700 1 -- 172.21.15.101:0/2634520460 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f102419d950 con 0x7f1024101670 2023-07-18T03:05:32.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.213+0000 7f102b3ff700 1 -- 172.21.15.101:0/2634520460 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f102419de10 con 0x7f1024101670 2023-07-18T03:05:32.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.214+0000 7f102b3ff700 1 -- 172.21.15.101:0/2634520460 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1024106a70 con 0x7f1024101670 2023-07-18T03:05:32.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.216+0000 7f101a7fc700 1 -- 172.21.15.101:0/2634520460 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f101c003a00 con 0x7f1024101670 2023-07-18T03:05:32.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.219+0000 7f101a7fc700 1 --2- 172.21.15.101:0/2634520460 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f10100a11d0 0x7f10100a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:32.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.219+0000 7f101a7fc700 1 -- 172.21.15.101:0/2634520460 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f101c0cc7a0 con 0x7f1024101670 2023-07-18T03:05:32.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.219+0000 7f102919b700 1 --2- 172.21.15.101:0/2634520460 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f10100a11d0 0x7f10100a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:32.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.219+0000 7f102919b700 1 --2- 172.21.15.101:0/2634520460 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f10100a11d0 0x7f10100a3680 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f102000b2e0 tx=0x7f1020005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:32.207 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.222+0000 7f101a7fc700 1 -- 172.21.15.101:0/2634520460 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f101c095c20 con 0x7f1024101670 2023-07-18T03:05:32.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:31 smithi101 ceph-mon[126379]: pgmap v293: 129 pgs: 129 active+clean; 582 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2023-07-18T03:05:32.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:31 smithi101 ceph-mon[126379]: mds.? [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] up:active 2023-07-18T03:05:32.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:31 smithi101 ceph-mon[126379]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-07-18T03:05:32.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:31 smithi101 ceph-mon[126379]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-07-18T03:05:32.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:31 smithi101 ceph-mon[126379]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-07-18T03:05:32.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:31 smithi101 ceph-mon[126379]: Cluster is now healthy 2023-07-18T03:05:32.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:31 smithi101 ceph-mon[126379]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2023-07-18T03:05:32.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:31 smithi101 ceph-mon[126379]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2023-07-18T03:05:32.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:31 smithi101 ceph-mon[126379]: daemon mds.c is now active in filesystem cephfs as rank 2 2023-07-18T03:05:32.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:31 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2865994685' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-07-18T03:05:32.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:31 smithi155 ceph-mon[128885]: pgmap v293: 129 pgs: 129 active+clean; 582 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2023-07-18T03:05:32.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:31 smithi155 ceph-mon[128885]: mds.? [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] up:active 2023-07-18T03:05:32.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:31 smithi155 ceph-mon[128885]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-07-18T03:05:32.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:31 smithi155 ceph-mon[128885]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-07-18T03:05:32.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:31 smithi155 ceph-mon[128885]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-07-18T03:05:32.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:31 smithi155 ceph-mon[128885]: Cluster is now healthy 2023-07-18T03:05:32.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:31 smithi155 ceph-mon[128885]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2023-07-18T03:05:32.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:31 smithi155 ceph-mon[128885]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2023-07-18T03:05:32.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:31 smithi155 ceph-mon[128885]: daemon mds.c is now active in filesystem cephfs as rank 2 2023-07-18T03:05:32.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:31 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/2865994685' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-07-18T03:05:32.440 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.455+0000 7f102b3ff700 1 -- 172.21.15.101:0/2634520460 --> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f10240609e0 con 0x7f10100a11d0 2023-07-18T03:05:32.441 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:31 smithi179 ceph-mon[129229]: pgmap v293: 129 pgs: 129 active+clean; 582 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2023-07-18T03:05:32.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:31 smithi179 ceph-mon[129229]: mds.? [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] up:active 2023-07-18T03:05:32.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:31 smithi179 ceph-mon[129229]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-07-18T03:05:32.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:31 smithi179 ceph-mon[129229]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-07-18T03:05:32.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:31 smithi179 ceph-mon[129229]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-07-18T03:05:32.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:31 smithi179 ceph-mon[129229]: Cluster is now healthy 2023-07-18T03:05:32.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:31 smithi179 ceph-mon[129229]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2023-07-18T03:05:32.443 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:31 smithi179 ceph-mon[129229]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2023-07-18T03:05:32.443 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:31 smithi179 ceph-mon[129229]: daemon mds.c is now active in filesystem cephfs as rank 2 2023-07-18T03:05:32.443 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:31 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/2865994685' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-07-18T03:05:32.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.503+0000 7f101a7fc700 1 -- 172.21.15.101:0/2634520460 <== mgr.14150 v2:172.21.15.101:6800/3963742842 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f10240609e0 con 0x7f10100a11d0 2023-07-18T03:05:32.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.505+0000 7f102b3ff700 1 -- 172.21.15.101:0/2634520460 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f10100a11d0 msgr2=0x7f10100a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:32.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.505+0000 7f102b3ff700 1 --2- 172.21.15.101:0/2634520460 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f10100a11d0 0x7f10100a3680 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f102000b2e0 tx=0x7f1020005fb0 comp rx=0 tx=0).stop 2023-07-18T03:05:32.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.505+0000 7f102b3ff700 1 -- 172.21.15.101:0/2634520460 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1024101670 msgr2=0x7f102419ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:32.492 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.505+0000 7f102b3ff700 1 --2- 172.21.15.101:0/2634520460 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1024101670 0x7f102419ce30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f101c00ef30 tx=0x7f101c00c3a0 comp rx=0 tx=0).stop 2023-07-18T03:05:32.492 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.505+0000 7f102b3ff700 1 -- 172.21.15.101:0/2634520460 shutdown_connections 2023-07-18T03:05:32.492 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.505+0000 7f102b3ff700 1 --2- 172.21.15.101:0/2634520460 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f10100a11d0 0x7f10100a3680 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:32.492 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.505+0000 7f102b3ff700 1 --2- 172.21.15.101:0/2634520460 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10240ffaf0 0x7f1024195880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:32.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.505+0000 7f102b3ff700 1 --2- 172.21.15.101:0/2634520460 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1024101670 0x7f102419ce30 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:32.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.505+0000 7f102b3ff700 1 --2- 172.21.15.101:0/2634520460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1024100ce0 0x7f1024195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:32.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.505+0000 7f102b3ff700 1 -- 172.21.15.101:0/2634520460 >> 172.21.15.101:0/2634520460 conn(0x7f10240fb130 msgr2=0x7f10240fd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:32.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.505+0000 7f102b3ff700 1 -- 172.21.15.101:0/2634520460 shutdown_connections 2023-07-18T03:05:32.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.505+0000 7f102b3ff700 1 -- 172.21.15.101:0/2634520460 wait complete. 2023-07-18T03:05:32.509 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-07-18T03:05:32.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.724+0000 7f4784dff700 1 -- 172.21.15.101:0/1385820811 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4780101670 msgr2=0x7f4780107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:32.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.724+0000 7f4784dff700 1 --2- 172.21.15.101:0/1385820811 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4780101670 0x7f4780107f30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f477800b0a0 tx=0x7f477800b3b0 comp rx=0 tx=0).stop 2023-07-18T03:05:32.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.724+0000 7f4784dff700 1 -- 172.21.15.101:0/1385820811 shutdown_connections 2023-07-18T03:05:32.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.724+0000 7f4784dff700 1 --2- 172.21.15.101:0/1385820811 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4780101670 0x7f4780107f30 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:32.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.724+0000 7f4784dff700 1 --2- 172.21.15.101:0/1385820811 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47800ffaf0 0x7f47800fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:32.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.724+0000 7f4784dff700 1 --2- 172.21.15.101:0/1385820811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4780100ce0 0x7f4780101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:32.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.724+0000 7f4784dff700 1 -- 172.21.15.101:0/1385820811 >> 172.21.15.101:0/1385820811 conn(0x7f47800fb130 msgr2=0x7f47800fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:32.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.724+0000 7f4784dff700 1 -- 172.21.15.101:0/1385820811 shutdown_connections 2023-07-18T03:05:32.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.724+0000 7f4784dff700 1 -- 172.21.15.101:0/1385820811 wait complete. 2023-07-18T03:05:32.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.725+0000 7f4784dff700 1 Processor -- start 2023-07-18T03:05:32.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.726+0000 7f4784dff700 1 -- start start 2023-07-18T03:05:32.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.726+0000 7f4784dff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47800ffaf0 0x7f4780197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:32.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.726+0000 7f4784dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4780100ce0 0x7f47801980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:32.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.726+0000 7f4784dff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4780101670 0x7f478019f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:32.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.726+0000 7f4784dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478019f6e0 con 0x7f4780100ce0 2023-07-18T03:05:32.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.726+0000 7f4784dff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478019f820 con 0x7f47800ffaf0 2023-07-18T03:05:32.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.726+0000 7f4784dff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478019f9b0 con 0x7f4780101670 2023-07-18T03:05:32.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.726+0000 7f477dd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4780100ce0 0x7f47801980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:32.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.726+0000 7f477dd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4780100ce0 0x7f47801980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:53414/0 (socket says 172.21.15.101:53414) 2023-07-18T03:05:32.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.726+0000 7f477dd9b700 1 -- 172.21.15.101:0/4207408843 learned_addr learned my addr 172.21.15.101:0/4207408843 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:32.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.726+0000 7f477e59c700 1 --2- 172.21.15.101:0/4207408843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47800ffaf0 0x7f4780197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:32.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.726+0000 7f477ed9d700 1 --2- 172.21.15.101:0/4207408843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4780101670 0x7f478019f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:32.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.727+0000 7f477e59c700 1 -- 172.21.15.101:0/4207408843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4780101670 msgr2=0x7f478019f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:32.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.727+0000 7f477e59c700 1 --2- 172.21.15.101:0/4207408843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4780101670 0x7f478019f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:32.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.727+0000 7f477e59c700 1 -- 172.21.15.101:0/4207408843 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4780100ce0 msgr2=0x7f47801980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:32.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.727+0000 7f477e59c700 1 --2- 172.21.15.101:0/4207408843 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4780100ce0 0x7f47801980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:32.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.727+0000 7f477e59c700 1 -- 172.21.15.101:0/4207408843 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4778009d00 con 0x7f47800ffaf0 2023-07-18T03:05:32.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.727+0000 7f477e59c700 1 --2- 172.21.15.101:0/4207408843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47800ffaf0 0x7f4780197b60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f477000e9c0 tx=0x7f477000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:32.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.727+0000 7f476f7fe700 1 -- 172.21.15.101:0/4207408843 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4770004220 con 0x7f47800ffaf0 2023-07-18T03:05:32.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.727+0000 7f4784dff700 1 -- 172.21.15.101:0/4207408843 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f478019fc90 con 0x7f47800ffaf0 2023-07-18T03:05:32.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.727+0000 7f476f7fe700 1 -- 172.21.15.101:0/4207408843 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4770004380 con 0x7f47800ffaf0 2023-07-18T03:05:32.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.727+0000 7f476f7fe700 1 -- 172.21.15.101:0/4207408843 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4770003be0 con 0x7f47800ffaf0 2023-07-18T03:05:32.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.727+0000 7f4784dff700 1 -- 172.21.15.101:0/4207408843 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47801a01a0 con 0x7f47800ffaf0 2023-07-18T03:05:32.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.728+0000 7f4784dff700 1 -- 172.21.15.101:0/4207408843 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f478010b320 con 0x7f47800ffaf0 2023-07-18T03:05:32.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.730+0000 7f476f7fe700 1 -- 172.21.15.101:0/4207408843 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f4770003d40 con 0x7f47800ffaf0 2023-07-18T03:05:32.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.732+0000 7f476f7fe700 1 --2- 172.21.15.101:0/4207408843 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f47680a1150 0x7f47680a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:32.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.732+0000 7f476f7fe700 1 -- 172.21.15.101:0/4207408843 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f4770014070 con 0x7f47800ffaf0 2023-07-18T03:05:32.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.732+0000 7f477dd9b700 1 --2- 172.21.15.101:0/4207408843 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f47680a1150 0x7f47680a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:32.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.732+0000 7f477dd9b700 1 --2- 172.21.15.101:0/4207408843 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f47680a1150 0x7f47680a3600 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4774009d20 tx=0x7f4774009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:32.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.734+0000 7f476f7fe700 1 -- 172.21.15.101:0/4207408843 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4770096490 con 0x7f47800ffaf0 2023-07-18T03:05:32.729 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:05:32 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:05:32.462+0000 7f29e635d700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-18T03:05:32.729 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:05:32 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:05:32.462+0000 7f29e635d700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-18T03:05:32.729 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:05:32 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:05:32.462+0000 7f29e635d700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-18T03:05:32.729 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:05:32 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:05:32.462+0000 7f29e635d700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-18T03:05:32.729 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:05:32 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:05:32.462+0000 7f29e635d700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-18T03:05:32.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.968+0000 7f4784dff700 1 -- 172.21.15.101:0/4207408843 --> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f4780107580 con 0x7f47680a1150 2023-07-18T03:05:32.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.972+0000 7f476f7fe700 1 -- 172.21.15.101:0/4207408843 <== mgr.14150 v2:172.21.15.101:6800/3963742842 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f4780107580 con 0x7f47680a1150 2023-07-18T03:05:32.958 INFO:teuthology.orchestra.run.smithi101.stdout:/volumes/_nogroup/sv_0/3f4beb84-2bb7-45bd-8998-e0dbf85336fa 2023-07-18T03:05:32.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.974+0000 7f4784dff700 1 -- 172.21.15.101:0/4207408843 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f47680a1150 msgr2=0x7f47680a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:32.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.974+0000 7f4784dff700 1 --2- 172.21.15.101:0/4207408843 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f47680a1150 0x7f47680a3600 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4774009d20 tx=0x7f4774009260 comp rx=0 tx=0).stop 2023-07-18T03:05:32.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.974+0000 7f4784dff700 1 -- 172.21.15.101:0/4207408843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47800ffaf0 msgr2=0x7f4780197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:32.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.974+0000 7f4784dff700 1 --2- 172.21.15.101:0/4207408843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47800ffaf0 0x7f4780197b60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f477000e9c0 tx=0x7f477000ecd0 comp rx=0 tx=0).stop 2023-07-18T03:05:32.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.974+0000 7f4784dff700 1 -- 172.21.15.101:0/4207408843 shutdown_connections 2023-07-18T03:05:32.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.974+0000 7f4784dff700 1 --2- 172.21.15.101:0/4207408843 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f47680a1150 0x7f47680a3600 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:32.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.974+0000 7f4784dff700 1 --2- 172.21.15.101:0/4207408843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47800ffaf0 0x7f4780197b60 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:32.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.974+0000 7f4784dff700 1 --2- 172.21.15.101:0/4207408843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4780101670 0x7f478019f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:32.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.974+0000 7f4784dff700 1 --2- 172.21.15.101:0/4207408843 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4780100ce0 0x7f47801980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:32.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.974+0000 7f4784dff700 1 -- 172.21.15.101:0/4207408843 >> 172.21.15.101:0/4207408843 conn(0x7f47800fb130 msgr2=0x7f47800fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:32.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.975+0000 7f4784dff700 1 -- 172.21.15.101:0/4207408843 shutdown_connections 2023-07-18T03:05:32.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:32.975+0000 7f4784dff700 1 -- 172.21.15.101:0/4207408843 wait complete. 2023-07-18T03:05:32.979 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-07-18T03:05:33.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.199+0000 7fafa07ff700 1 -- 172.21.15.101:0/3460781586 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf98100ce0 msgr2=0x7faf98101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:33.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.199+0000 7fafa07ff700 1 --2- 172.21.15.101:0/3460781586 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf98100ce0 0x7faf98101130 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7faf88009990 tx=0x7faf88009ca0 comp rx=0 tx=0).stop 2023-07-18T03:05:33.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.199+0000 7fafa07ff700 1 -- 172.21.15.101:0/3460781586 shutdown_connections 2023-07-18T03:05:33.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.199+0000 7fafa07ff700 1 --2- 172.21.15.101:0/3460781586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf980ffaf0 0x7faf980fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:33.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.199+0000 7fafa07ff700 1 --2- 172.21.15.101:0/3460781586 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf98101670 0x7faf98107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:33.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.199+0000 7fafa07ff700 1 --2- 172.21.15.101:0/3460781586 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf98100ce0 0x7faf98101130 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:33.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.199+0000 7fafa07ff700 1 -- 172.21.15.101:0/3460781586 >> 172.21.15.101:0/3460781586 conn(0x7faf980fb130 msgr2=0x7faf980fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:33.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.199+0000 7fafa07ff700 1 -- 172.21.15.101:0/3460781586 shutdown_connections 2023-07-18T03:05:33.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.199+0000 7fafa07ff700 1 -- 172.21.15.101:0/3460781586 wait complete. 2023-07-18T03:05:33.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.200+0000 7fafa07ff700 1 Processor -- start 2023-07-18T03:05:33.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.200+0000 7fafa07ff700 1 -- start start 2023-07-18T03:05:33.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.201+0000 7fafa07ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf980ffaf0 0x7faf981979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:33.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.201+0000 7fafa07ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf98100ce0 0x7faf98197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:33.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.201+0000 7fafa07ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf98101670 0x7faf9819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:33.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.201+0000 7fafa07ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9819f570 con 0x7faf980ffaf0 2023-07-18T03:05:33.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.201+0000 7fafa07ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9819f6b0 con 0x7faf98101670 2023-07-18T03:05:33.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.201+0000 7fafa07ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9819f840 con 0x7faf98100ce0 2023-07-18T03:05:33.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.201+0000 7faf9e59b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf980ffaf0 0x7faf981979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:33.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.201+0000 7faf9e59b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf980ffaf0 0x7faf981979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:53446/0 (socket says 172.21.15.101:53446) 2023-07-18T03:05:33.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.201+0000 7faf9e59b700 1 -- 172.21.15.101:0/4294292826 learned_addr learned my addr 172.21.15.101:0/4294292826 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:33.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.201+0000 7faf9ed9c700 1 --2- 172.21.15.101:0/4294292826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf98101670 0x7faf9819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:33.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.201+0000 7faf9dd9a700 1 --2- 172.21.15.101:0/4294292826 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf98100ce0 0x7faf98197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:33.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.201+0000 7faf9e59b700 1 -- 172.21.15.101:0/4294292826 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf98100ce0 msgr2=0x7faf98197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:33.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.201+0000 7faf9e59b700 1 --2- 172.21.15.101:0/4294292826 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf98100ce0 0x7faf98197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:33.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.201+0000 7faf9e59b700 1 -- 172.21.15.101:0/4294292826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf98101670 msgr2=0x7faf9819efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:33.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.201+0000 7faf9e59b700 1 --2- 172.21.15.101:0/4294292826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf98101670 0x7faf9819efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:33.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.201+0000 7faf9e59b700 1 -- 172.21.15.101:0/4294292826 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf880095f0 con 0x7faf980ffaf0 2023-07-18T03:05:33.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.202+0000 7faf9e59b700 1 --2- 172.21.15.101:0/4294292826 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf980ffaf0 0x7faf981979f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7faf9400e970 tx=0x7faf9400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:33.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.202+0000 7faf8f7fe700 1 -- 172.21.15.101:0/4294292826 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf94009790 con 0x7faf980ffaf0 2023-07-18T03:05:33.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.202+0000 7faf8f7fe700 1 -- 172.21.15.101:0/4294292826 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf9400c7e0 con 0x7faf980ffaf0 2023-07-18T03:05:33.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.202+0000 7fafa07ff700 1 -- 172.21.15.101:0/4294292826 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf9819fb20 con 0x7faf980ffaf0 2023-07-18T03:05:33.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.202+0000 7faf8f7fe700 1 -- 172.21.15.101:0/4294292826 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf94003c70 con 0x7faf980ffaf0 2023-07-18T03:05:33.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.202+0000 7fafa07ff700 1 -- 172.21.15.101:0/4294292826 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf981a0080 con 0x7faf980ffaf0 2023-07-18T03:05:33.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.203+0000 7fafa07ff700 1 -- 172.21.15.101:0/4294292826 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf9810b2c0 con 0x7faf980ffaf0 2023-07-18T03:05:33.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.205+0000 7faf8f7fe700 1 -- 172.21.15.101:0/4294292826 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7faf9400cc50 con 0x7faf980ffaf0 2023-07-18T03:05:33.192 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.207+0000 7faf8f7fe700 1 --2- 172.21.15.101:0/4294292826 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7faf840a1260 0x7faf840a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:33.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.207+0000 7faf8f7fe700 1 -- 172.21.15.101:0/4294292826 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7faf94014070 con 0x7faf980ffaf0 2023-07-18T03:05:33.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.207+0000 7faf9dd9a700 1 --2- 172.21.15.101:0/4294292826 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7faf840a1260 0x7faf840a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:33.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.208+0000 7faf9dd9a700 1 --2- 172.21.15.101:0/4294292826 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7faf840a1260 0x7faf840a3710 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7faf8800b2e0 tx=0x7faf88005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:33.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.212+0000 7faf8f7fe700 1 -- 172.21.15.101:0/4294292826 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7faf940961d0 con 0x7faf980ffaf0 2023-07-18T03:05:33.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:32 smithi101 ceph-mon[126379]: mds.? [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] up:active 2023-07-18T03:05:33.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:32 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2865994685' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-07-18T03:05:33.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:32 smithi101 ceph-mon[126379]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2023-07-18T03:05:33.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:32 smithi101 ceph-mon[126379]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:05:33.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:32 smithi155 ceph-mon[128885]: mds.? [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] up:active 2023-07-18T03:05:33.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:32 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/2865994685' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-07-18T03:05:33.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:32 smithi155 ceph-mon[128885]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2023-07-18T03:05:33.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:32 smithi155 ceph-mon[128885]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:05:33.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.443+0000 7fafa07ff700 1 -- 172.21.15.101:0/4294292826 --> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7faf98107630 con 0x7faf840a1260 2023-07-18T03:05:33.441 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:32 smithi179 ceph-mon[129229]: mds.? [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] up:active 2023-07-18T03:05:33.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:32 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/2865994685' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-07-18T03:05:33.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:32 smithi179 ceph-mon[129229]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2023-07-18T03:05:33.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:32 smithi179 ceph-mon[129229]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:05:33.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.469+0000 7faf8f7fe700 1 -- 172.21.15.101:0/4294292826 <== mgr.14150 v2:172.21.15.101:6800/3963742842 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7faf98107630 con 0x7faf840a1260 2023-07-18T03:05:33.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.471+0000 7fafa07ff700 1 -- 172.21.15.101:0/4294292826 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7faf840a1260 msgr2=0x7faf840a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:33.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.471+0000 7fafa07ff700 1 --2- 172.21.15.101:0/4294292826 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7faf840a1260 0x7faf840a3710 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7faf8800b2e0 tx=0x7faf88005960 comp rx=0 tx=0).stop 2023-07-18T03:05:33.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.471+0000 7fafa07ff700 1 -- 172.21.15.101:0/4294292826 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf980ffaf0 msgr2=0x7faf981979f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:33.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.471+0000 7fafa07ff700 1 --2- 172.21.15.101:0/4294292826 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf980ffaf0 0x7faf981979f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7faf9400e970 tx=0x7faf9400ec80 comp rx=0 tx=0).stop 2023-07-18T03:05:33.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.471+0000 7fafa07ff700 1 -- 172.21.15.101:0/4294292826 shutdown_connections 2023-07-18T03:05:33.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.471+0000 7fafa07ff700 1 --2- 172.21.15.101:0/4294292826 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7faf840a1260 0x7faf840a3710 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:33.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.471+0000 7fafa07ff700 1 --2- 172.21.15.101:0/4294292826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf98101670 0x7faf9819efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:33.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.471+0000 7fafa07ff700 1 --2- 172.21.15.101:0/4294292826 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf980ffaf0 0x7faf981979f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:33.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.471+0000 7fafa07ff700 1 --2- 172.21.15.101:0/4294292826 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf98100ce0 0x7faf98197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:33.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.471+0000 7fafa07ff700 1 -- 172.21.15.101:0/4294292826 >> 172.21.15.101:0/4294292826 conn(0x7faf980fb130 msgr2=0x7faf980fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:33.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.471+0000 7fafa07ff700 1 -- 172.21.15.101:0/4294292826 shutdown_connections 2023-07-18T03:05:33.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.471+0000 7fafa07ff700 1 -- 172.21.15.101:0/4294292826 wait complete. 2023-07-18T03:05:33.475 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-07-18T03:05:33.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.689+0000 7fd5389ff700 1 -- 172.21.15.101:0/3260074147 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd5340ffaf0 msgr2=0x7fd5340fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:33.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.689+0000 7fd5389ff700 1 --2- 172.21.15.101:0/3260074147 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd5340ffaf0 0x7fd5340fff00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd524009990 tx=0x7fd524009ca0 comp rx=0 tx=0).stop 2023-07-18T03:05:33.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.689+0000 7fd5389ff700 1 -- 172.21.15.101:0/3260074147 shutdown_connections 2023-07-18T03:05:33.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.689+0000 7fd5389ff700 1 --2- 172.21.15.101:0/3260074147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd534101670 0x7fd534107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:33.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.689+0000 7fd5389ff700 1 --2- 172.21.15.101:0/3260074147 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd5340ffaf0 0x7fd5340fff00 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:33.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.689+0000 7fd5389ff700 1 --2- 172.21.15.101:0/3260074147 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd534100ce0 0x7fd534101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:33.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.689+0000 7fd5389ff700 1 -- 172.21.15.101:0/3260074147 >> 172.21.15.101:0/3260074147 conn(0x7fd5340fb130 msgr2=0x7fd5340fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:33.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.689+0000 7fd5389ff700 1 -- 172.21.15.101:0/3260074147 shutdown_connections 2023-07-18T03:05:33.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.689+0000 7fd5389ff700 1 -- 172.21.15.101:0/3260074147 wait complete. 2023-07-18T03:05:33.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.690+0000 7fd5389ff700 1 Processor -- start 2023-07-18T03:05:33.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.690+0000 7fd5389ff700 1 -- start start 2023-07-18T03:05:33.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.691+0000 7fd5389ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5340ffaf0 0x7fd534197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:33.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.691+0000 7fd5389ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd534100ce0 0x7fd534197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:33.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.691+0000 7fd5389ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd534101670 0x7fd53419f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:33.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.691+0000 7fd5389ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53419f620 con 0x7fd534101670 2023-07-18T03:05:33.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.691+0000 7fd5389ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53419f760 con 0x7fd5340ffaf0 2023-07-18T03:05:33.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.691+0000 7fd5389ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53419f8f0 con 0x7fd534100ce0 2023-07-18T03:05:33.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.691+0000 7fd53259c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5340ffaf0 0x7fd534197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:33.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.691+0000 7fd532d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd534101670 0x7fd53419f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:33.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.691+0000 7fd53259c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5340ffaf0 0x7fd534197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:51750/0 (socket says 172.21.15.101:51750) 2023-07-18T03:05:33.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.691+0000 7fd53259c700 1 -- 172.21.15.101:0/2435134708 learned_addr learned my addr 172.21.15.101:0/2435134708 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:33.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.691+0000 7fd52bfff700 1 --2- 172.21.15.101:0/2435134708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd534100ce0 0x7fd534197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:33.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.691+0000 7fd532d9d700 1 -- 172.21.15.101:0/2435134708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd534100ce0 msgr2=0x7fd534197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:33.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.691+0000 7fd532d9d700 1 --2- 172.21.15.101:0/2435134708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd534100ce0 0x7fd534197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:33.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.691+0000 7fd532d9d700 1 -- 172.21.15.101:0/2435134708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5340ffaf0 msgr2=0x7fd534197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:33.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.691+0000 7fd532d9d700 1 --2- 172.21.15.101:0/2435134708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5340ffaf0 0x7fd534197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:33.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.692+0000 7fd532d9d700 1 -- 172.21.15.101:0/2435134708 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5240095f0 con 0x7fd534101670 2023-07-18T03:05:33.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.692+0000 7fd53259c700 1 --2- 172.21.15.101:0/2435134708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5340ffaf0 0x7fd534197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:05:33.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.692+0000 7fd532d9d700 1 --2- 172.21.15.101:0/2435134708 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd534101670 0x7fd53419f050 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd52c007f10 tx=0x7fd52c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:33.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.692+0000 7fd52b7fe700 1 -- 172.21.15.101:0/2435134708 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52c009ea0 con 0x7fd534101670 2023-07-18T03:05:33.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.692+0000 7fd5389ff700 1 -- 172.21.15.101:0/2435134708 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd53419fbd0 con 0x7fd534101670 2023-07-18T03:05:33.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.692+0000 7fd52b7fe700 1 -- 172.21.15.101:0/2435134708 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd52c00f040 con 0x7fd534101670 2023-07-18T03:05:33.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.692+0000 7fd52b7fe700 1 -- 172.21.15.101:0/2435134708 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52c004400 con 0x7fd534101670 2023-07-18T03:05:33.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.692+0000 7fd5389ff700 1 -- 172.21.15.101:0/2435134708 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5341a0130 con 0x7fd534101670 2023-07-18T03:05:33.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.693+0000 7fd5389ff700 1 -- 172.21.15.101:0/2435134708 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd53410b310 con 0x7fd534101670 2023-07-18T03:05:33.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.695+0000 7fd52b7fe700 1 -- 172.21.15.101:0/2435134708 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fd52c004ad0 con 0x7fd534101670 2023-07-18T03:05:33.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.697+0000 7fd52b7fe700 1 --2- 172.21.15.101:0/2435134708 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd5200a11e0 0x7fd5200a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:33.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.698+0000 7fd52b7fe700 1 -- 172.21.15.101:0/2435134708 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fd52c0cc670 con 0x7fd534101670 2023-07-18T03:05:33.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.698+0000 7fd53259c700 1 --2- 172.21.15.101:0/2435134708 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd5200a11e0 0x7fd5200a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:33.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.698+0000 7fd53259c700 1 --2- 172.21.15.101:0/2435134708 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd5200a11e0 0x7fd5200a3690 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd52400b5c0 tx=0x7fd524005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:33.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.701+0000 7fd52b7fe700 1 -- 172.21.15.101:0/2435134708 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd52c095900 con 0x7fd534101670 2023-07-18T03:05:33.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.935+0000 7fd5389ff700 1 -- 172.21.15.101:0/2435134708 --> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd534107630 con 0x7fd5200a11e0 2023-07-18T03:05:33.925 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.940+0000 7fd52b7fe700 1 -- 172.21.15.101:0/2435134708 <== mgr.14150 v2:172.21.15.101:6800/3963742842 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7fd534107630 con 0x7fd5200a11e0 2023-07-18T03:05:33.925 INFO:teuthology.orchestra.run.smithi101.stdout:/volumes/_nogroup/sv_1/e0ae3bb7-7d8a-4636-a330-38d2ea2c61af 2023-07-18T03:05:33.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.942+0000 7fd5389ff700 1 -- 172.21.15.101:0/2435134708 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd5200a11e0 msgr2=0x7fd5200a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:33.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.942+0000 7fd5389ff700 1 --2- 172.21.15.101:0/2435134708 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd5200a11e0 0x7fd5200a3690 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd52400b5c0 tx=0x7fd524005fd0 comp rx=0 tx=0).stop 2023-07-18T03:05:33.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.942+0000 7fd5389ff700 1 -- 172.21.15.101:0/2435134708 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd534101670 msgr2=0x7fd53419f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:33.928 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.942+0000 7fd5389ff700 1 --2- 172.21.15.101:0/2435134708 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd534101670 0x7fd53419f050 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd52c007f10 tx=0x7fd52c00d3b0 comp rx=0 tx=0).stop 2023-07-18T03:05:33.929 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.942+0000 7fd5389ff700 1 -- 172.21.15.101:0/2435134708 shutdown_connections 2023-07-18T03:05:33.930 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.942+0000 7fd5389ff700 1 --2- 172.21.15.101:0/2435134708 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd5200a11e0 0x7fd5200a3690 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:33.930 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.942+0000 7fd5389ff700 1 --2- 172.21.15.101:0/2435134708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5340ffaf0 0x7fd534197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:33.930 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.942+0000 7fd5389ff700 1 --2- 172.21.15.101:0/2435134708 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd534101670 0x7fd53419f050 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:33.930 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.942+0000 7fd5389ff700 1 --2- 172.21.15.101:0/2435134708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd534100ce0 0x7fd534197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:33.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.942+0000 7fd5389ff700 1 -- 172.21.15.101:0/2435134708 >> 172.21.15.101:0/2435134708 conn(0x7fd5340fb130 msgr2=0x7fd5340fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:33.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.942+0000 7fd5389ff700 1 -- 172.21.15.101:0/2435134708 shutdown_connections 2023-07-18T03:05:33.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:33.942+0000 7fd5389ff700 1 -- 172.21.15.101:0/2435134708 wait complete. 2023-07-18T03:05:33.946 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:05:34.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.159+0000 7f1d6c7ff700 1 -- 172.21.15.101:0/59570058 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1d64101680 msgr2=0x7f1d64107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:34.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.159+0000 7f1d6c7ff700 1 --2- 172.21.15.101:0/59570058 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1d64101680 0x7f1d64107f40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c00b0a0 tx=0x7f1d5c00b3b0 comp rx=0 tx=0).stop 2023-07-18T03:05:34.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.159+0000 7f1d6c7ff700 1 -- 172.21.15.101:0/59570058 shutdown_connections 2023-07-18T03:05:34.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.159+0000 7f1d6c7ff700 1 --2- 172.21.15.101:0/59570058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d640ffb00 0x7f1d640fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:34.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.159+0000 7f1d6c7ff700 1 --2- 172.21.15.101:0/59570058 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1d64101680 0x7f1d64107f40 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:34.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.159+0000 7f1d6c7ff700 1 --2- 172.21.15.101:0/59570058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d64100cf0 0x7f1d64101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:34.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.159+0000 7f1d6c7ff700 1 -- 172.21.15.101:0/59570058 >> 172.21.15.101:0/59570058 conn(0x7f1d640fb0e0 msgr2=0x7f1d640fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:34.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.159+0000 7f1d6c7ff700 1 -- 172.21.15.101:0/59570058 shutdown_connections 2023-07-18T03:05:34.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.159+0000 7f1d6c7ff700 1 -- 172.21.15.101:0/59570058 wait complete. 2023-07-18T03:05:34.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.160+0000 7f1d6c7ff700 1 Processor -- start 2023-07-18T03:05:34.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.160+0000 7f1d6c7ff700 1 -- start start 2023-07-18T03:05:34.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.160+0000 7f1d6c7ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1d640ffb00 0x7f1d64197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:34.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.160+0000 7f1d6c7ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d64100cf0 0x7f1d64197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:34.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.160+0000 7f1d6c7ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d64101680 0x7f1d6419f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:34.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.160+0000 7f1d6c7ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6419f630 con 0x7f1d640ffb00 2023-07-18T03:05:34.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.160+0000 7f1d6c7ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6419f770 con 0x7f1d64101680 2023-07-18T03:05:34.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.160+0000 7f1d6c7ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6419f900 con 0x7f1d64100cf0 2023-07-18T03:05:34.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.161+0000 7f1d6a59b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1d640ffb00 0x7f1d64197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:34.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.161+0000 7f1d69d9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d64100cf0 0x7f1d64197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:34.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.161+0000 7f1d6a59b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1d640ffb00 0x7f1d64197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:53484/0 (socket says 172.21.15.101:53484) 2023-07-18T03:05:34.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.161+0000 7f1d6a59b700 1 -- 172.21.15.101:0/2866106597 learned_addr learned my addr 172.21.15.101:0/2866106597 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:34.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.161+0000 7f1d6ad9c700 1 --2- 172.21.15.101:0/2866106597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d64101680 0x7f1d6419f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:34.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.161+0000 7f1d6a59b700 1 -- 172.21.15.101:0/2866106597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d64100cf0 msgr2=0x7f1d64197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:34.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.161+0000 7f1d6a59b700 1 --2- 172.21.15.101:0/2866106597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d64100cf0 0x7f1d64197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:34.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.161+0000 7f1d6a59b700 1 -- 172.21.15.101:0/2866106597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d64101680 msgr2=0x7f1d6419f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:34.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.161+0000 7f1d6a59b700 1 --2- 172.21.15.101:0/2866106597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d64101680 0x7f1d6419f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:34.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.161+0000 7f1d6a59b700 1 -- 172.21.15.101:0/2866106597 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d5c009d00 con 0x7f1d640ffb00 2023-07-18T03:05:34.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.161+0000 7f1d6a59b700 1 --2- 172.21.15.101:0/2866106597 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1d640ffb00 0x7f1d64197ab0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1d6000e970 tx=0x7f1d6000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:34.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.162+0000 7f1d5b7fe700 1 -- 172.21.15.101:0/2866106597 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d60009790 con 0x7f1d640ffb00 2023-07-18T03:05:34.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.162+0000 7f1d5b7fe700 1 -- 172.21.15.101:0/2866106597 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d6000c7e0 con 0x7f1d640ffb00 2023-07-18T03:05:34.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.162+0000 7f1d6c7ff700 1 -- 172.21.15.101:0/2866106597 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d6419fbe0 con 0x7f1d640ffb00 2023-07-18T03:05:34.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.162+0000 7f1d5b7fe700 1 -- 172.21.15.101:0/2866106597 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d60003c70 con 0x7f1d640ffb00 2023-07-18T03:05:34.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.162+0000 7f1d6c7ff700 1 -- 172.21.15.101:0/2866106597 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d641a00f0 con 0x7f1d640ffb00 2023-07-18T03:05:34.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.163+0000 7f1d6c7ff700 1 -- 172.21.15.101:0/2866106597 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d6410b2e0 con 0x7f1d640ffb00 2023-07-18T03:05:34.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.165+0000 7f1d5b7fe700 1 -- 172.21.15.101:0/2866106597 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f1d6000cc50 con 0x7f1d640ffb00 2023-07-18T03:05:34.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.166+0000 7f1d5b7fe700 1 --2- 172.21.15.101:0/2866106597 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f1d500a1220 0x7f1d500a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:34.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.166+0000 7f1d5b7fe700 1 -- 172.21.15.101:0/2866106597 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f1d60014070 con 0x7f1d640ffb00 2023-07-18T03:05:34.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.166+0000 7f1d69d9a700 1 --2- 172.21.15.101:0/2866106597 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f1d500a1220 0x7f1d500a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:34.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.167+0000 7f1d69d9a700 1 --2- 172.21.15.101:0/2866106597 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f1d500a1220 0x7f1d500a36d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1d54009c60 tx=0x7f1d54009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:34.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.170+0000 7f1d5b7fe700 1 -- 172.21.15.101:0/2866106597 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1d60096220 con 0x7f1d640ffb00 2023-07-18T03:05:34.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:33 smithi101 ceph-mon[126379]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:05:34.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:33 smithi101 ceph-mon[126379]: pgmap v294: 129 pgs: 129 active+clean; 583 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 515 B/s rd, 1.5 KiB/s wr, 4 op/s 2023-07-18T03:05:34.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:33 smithi101 ceph-mon[126379]: from='client.24809 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:05:34.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:33 smithi101 ceph-mon[126379]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-07-18T03:05:34.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:33 smithi155 ceph-mon[128885]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:05:34.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:33 smithi155 ceph-mon[128885]: pgmap v294: 129 pgs: 129 active+clean; 583 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 515 B/s rd, 1.5 KiB/s wr, 4 op/s 2023-07-18T03:05:34.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:33 smithi155 ceph-mon[128885]: from='client.24809 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:05:34.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:33 smithi155 ceph-mon[128885]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-07-18T03:05:34.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.413+0000 7f1d6c7ff700 1 -- 172.21.15.101:0/2866106597 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d641a06a0 con 0x7f1d640ffb00 2023-07-18T03:05:34.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.414+0000 7f1d5b7fe700 1 -- 172.21.15.101:0/2866106597 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10858 (secure 0 0 0) 0x7f1d60003dd0 con 0x7f1d640ffb00 2023-07-18T03:05:34.400 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:05:34.400 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/4192846769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":4192846769},{"type":"v1","addr":"172.21.15.101:6839","nonce":4192846769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6841/3824340175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6840","nonce":3824340175},{"type":"v1","addr":"172.21.15.101:6841","nonce":3824340175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:04:59.361185+0000","modified":"2023-07-18T03:05:31.985959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.155:6839/2055462483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2055462483},{"type":"v1","addr":"172.21.15.155:6839","nonce":2055462483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:05:34.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.416+0000 7f1d6c7ff700 1 -- 172.21.15.101:0/2866106597 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f1d500a1220 msgr2=0x7f1d500a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:34.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.416+0000 7f1d6c7ff700 1 --2- 172.21.15.101:0/2866106597 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f1d500a1220 0x7f1d500a36d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1d54009c60 tx=0x7f1d54009260 comp rx=0 tx=0).stop 2023-07-18T03:05:34.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.416+0000 7f1d6c7ff700 1 -- 172.21.15.101:0/2866106597 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1d640ffb00 msgr2=0x7f1d64197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:34.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.416+0000 7f1d6c7ff700 1 --2- 172.21.15.101:0/2866106597 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1d640ffb00 0x7f1d64197ab0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1d6000e970 tx=0x7f1d6000ec80 comp rx=0 tx=0).stop 2023-07-18T03:05:34.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.417+0000 7f1d6c7ff700 1 -- 172.21.15.101:0/2866106597 shutdown_connections 2023-07-18T03:05:34.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.417+0000 7f1d6c7ff700 1 --2- 172.21.15.101:0/2866106597 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f1d500a1220 0x7f1d500a36d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:34.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.417+0000 7f1d6c7ff700 1 --2- 172.21.15.101:0/2866106597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d64101680 0x7f1d6419f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:34.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.417+0000 7f1d6c7ff700 1 --2- 172.21.15.101:0/2866106597 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1d640ffb00 0x7f1d64197ab0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:34.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.417+0000 7f1d6c7ff700 1 --2- 172.21.15.101:0/2866106597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d64100cf0 0x7f1d64197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:34.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.417+0000 7f1d6c7ff700 1 -- 172.21.15.101:0/2866106597 >> 172.21.15.101:0/2866106597 conn(0x7f1d640fb0e0 msgr2=0x7f1d640fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:34.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.417+0000 7f1d6c7ff700 1 -- 172.21.15.101:0/2866106597 shutdown_connections 2023-07-18T03:05:34.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.417+0000 7f1d6c7ff700 1 -- 172.21.15.101:0/2866106597 wait complete. 2023-07-18T03:05:34.404 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 24 2023-07-18T03:05:34.420 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:05:34.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:33 smithi179 ceph-mon[129229]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:05:34.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:33 smithi179 ceph-mon[129229]: pgmap v294: 129 pgs: 129 active+clean; 583 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 515 B/s rd, 1.5 KiB/s wr, 4 op/s 2023-07-18T03:05:34.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:33 smithi179 ceph-mon[129229]: from='client.24809 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:05:34.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:33 smithi179 ceph-mon[129229]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-07-18T03:05:34.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.633+0000 7f6deb9ff700 1 -- 172.21.15.101:0/3405031395 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6de4100550 msgr2=0x7f6de4102930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:34.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.633+0000 7f6deb9ff700 1 --2- 172.21.15.101:0/3405031395 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6de4100550 0x7f6de4102930 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6de0009990 tx=0x7f6de0009ca0 comp rx=0 tx=0).stop 2023-07-18T03:05:34.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.633+0000 7f6deb9ff700 1 -- 172.21.15.101:0/3405031395 shutdown_connections 2023-07-18T03:05:34.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.633+0000 7f6deb9ff700 1 --2- 172.21.15.101:0/3405031395 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6de4105790 0x7f6de4107bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:34.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.633+0000 7f6deb9ff700 1 --2- 172.21.15.101:0/3405031395 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6de4100550 0x7f6de4102930 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:34.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.633+0000 7f6deb9ff700 1 --2- 172.21.15.101:0/3405031395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6de4102e70 0x7f6de4105250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:34.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.633+0000 7f6deb9ff700 1 -- 172.21.15.101:0/3405031395 >> 172.21.15.101:0/3405031395 conn(0x7f6de40f9fd0 msgr2=0x7f6de40fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:34.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.633+0000 7f6deb9ff700 1 -- 172.21.15.101:0/3405031395 shutdown_connections 2023-07-18T03:05:34.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.633+0000 7f6deb9ff700 1 -- 172.21.15.101:0/3405031395 wait complete. 2023-07-18T03:05:34.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.634+0000 7f6deb9ff700 1 Processor -- start 2023-07-18T03:05:34.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.634+0000 7f6deb9ff700 1 -- start start 2023-07-18T03:05:34.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.634+0000 7f6deb9ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6de4100550 0x7f6de41979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:34.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.634+0000 7f6deb9ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6de4102e70 0x7f6de4197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:34.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.634+0000 7f6deb9ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6de4105790 0x7f6de419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:34.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.635+0000 7f6deb9ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de419f550 con 0x7f6de4100550 2023-07-18T03:05:34.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.635+0000 7f6deb9ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de419f690 con 0x7f6de4102e70 2023-07-18T03:05:34.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.635+0000 7f6deb9ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de419f820 con 0x7f6de4105790 2023-07-18T03:05:34.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.635+0000 7f6de979b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6de4100550 0x7f6de41979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:34.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.635+0000 7f6de9f9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6de4105790 0x7f6de419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:34.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.635+0000 7f6de9f9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6de4105790 0x7f6de419ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:43750/0 (socket says 172.21.15.101:43750) 2023-07-18T03:05:34.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.635+0000 7f6de9f9c700 1 -- 172.21.15.101:0/3800072218 learned_addr learned my addr 172.21.15.101:0/3800072218 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:34.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.635+0000 7f6de8f9a700 1 --2- 172.21.15.101:0/3800072218 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6de4102e70 0x7f6de4197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:34.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.635+0000 7f6de9f9c700 1 -- 172.21.15.101:0/3800072218 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6de4102e70 msgr2=0x7f6de4197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:34.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.635+0000 7f6de9f9c700 1 --2- 172.21.15.101:0/3800072218 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6de4102e70 0x7f6de4197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:34.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.635+0000 7f6de9f9c700 1 -- 172.21.15.101:0/3800072218 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6de4100550 msgr2=0x7f6de41979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:34.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.635+0000 7f6de9f9c700 1 --2- 172.21.15.101:0/3800072218 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6de4100550 0x7f6de41979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:34.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.635+0000 7f6de9f9c700 1 -- 172.21.15.101:0/3800072218 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de00095f0 con 0x7f6de4105790 2023-07-18T03:05:34.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.635+0000 7f6de9f9c700 1 --2- 172.21.15.101:0/3800072218 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6de4105790 0x7f6de419ef80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc00ef30 tx=0x7f6ddc00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:34.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.636+0000 7f6dda7fc700 1 -- 172.21.15.101:0/3800072218 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc018070 con 0x7f6de4105790 2023-07-18T03:05:34.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.636+0000 7f6dda7fc700 1 -- 172.21.15.101:0/3800072218 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ddc00f040 con 0x7f6de4105790 2023-07-18T03:05:34.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.636+0000 7f6dda7fc700 1 -- 172.21.15.101:0/3800072218 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc00cd00 con 0x7f6de4105790 2023-07-18T03:05:34.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.636+0000 7f6deb9ff700 1 -- 172.21.15.101:0/3800072218 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6de419faa0 con 0x7f6de4105790 2023-07-18T03:05:34.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.636+0000 7f6deb9ff700 1 -- 172.21.15.101:0/3800072218 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de419ff60 con 0x7f6de4105790 2023-07-18T03:05:34.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.637+0000 7f6deb9ff700 1 -- 172.21.15.101:0/3800072218 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6de4107350 con 0x7f6de4105790 2023-07-18T03:05:34.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.639+0000 7f6dda7fc700 1 -- 172.21.15.101:0/3800072218 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f6ddc003a00 con 0x7f6de4105790 2023-07-18T03:05:34.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.640+0000 7f6dda7fc700 1 --2- 172.21.15.101:0/3800072218 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f6dd00a1220 0x7f6dd00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:34.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.640+0000 7f6dda7fc700 1 -- 172.21.15.101:0/3800072218 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f6ddc0cd300 con 0x7f6de4105790 2023-07-18T03:05:34.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.641+0000 7f6de979b700 1 --2- 172.21.15.101:0/3800072218 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f6dd00a1220 0x7f6dd00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:34.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.641+0000 7f6de979b700 1 --2- 172.21.15.101:0/3800072218 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f6dd00a1220 0x7f6dd00a36d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6de000b2e0 tx=0x7f6de0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:34.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.642+0000 7f6dda7fc700 1 -- 172.21.15.101:0/3800072218 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6ddc096620 con 0x7f6de4105790 2023-07-18T03:05:34.826 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.841+0000 7f6deb9ff700 1 -- 172.21.15.101:0/3800072218 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6de402c850 con 0x7f6de4105790 2023-07-18T03:05:34.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.843+0000 7f6dda7fc700 1 -- 172.21.15.101:0/3800072218 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24482 (secure 0 0 0) 0x7f6ddc099c40 con 0x7f6de4105790 2023-07-18T03:05:34.829 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:05:34.829 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":89,"fsid":"b1782daa-2516-11ee-9b34-001a4aab830c","created":"2023-07-18T02:57:22.972483+0000","modified":"2023-07-18T03:05:23.929113+0000","last_up_change":"2023-07-18T03:04:11.429530+0000","last_in_change":"2023-07-18T03:03:54.158340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:00:43.353489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:04:57.771616+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-18T03:04:58.769590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-18T03:05:02.358830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc76b5d1-38a0-4f1a-9335-756bfc40a62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6803","nonce":3610719929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6805","nonce":3610719929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6809","nonce":3610719929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6807","nonce":3610719929}]},"public_addr":"172.21.15.101:6803/3610719929","cluster_addr":"172.21.15.101:6805/3610719929","heartbeat_back_addr":"172.21.15.101:6809/3610719929","heartbeat_front_addr":"172.21.15.101:6807/3610719929","state":["exists","up"]},{"osd":1,"uuid":"2a299573-0b34-4af9-aaae-b523e5058ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6801","nonce":1030229579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6803","nonce":1030229579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6807","nonce":1030229579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6805","nonce":1030229579}]},"public_addr":"172.21.15.155:6801/1030229579","cluster_addr":"172.21.15.155:6803/1030229579","heartbeat_back_addr":"172.21.15.155:6807/1030229579","heartbeat_front_addr":"172.21.15.155:6805/1030229579","state":["exists","up"]},{"osd":2,"uuid":"123b3749-c636-48fa-9e33-6229b5575dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6801","nonce":1349487507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6803","nonce":1349487507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6807","nonce":1349487507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6805","nonce":1349487507}]},"public_addr":"172.21.15.179:6801/1349487507","cluster_addr":"172.21.15.179:6803/1349487507","heartbeat_back_addr":"172.21.15.179:6807/1349487507","heartbeat_front_addr":"172.21.15.179:6805/1349487507","state":["exists","up"]},{"osd":3,"uuid":"4b0c409d-2954-4127-833a-b196bf226d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6811","nonce":4226577463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6813","nonce":4226577463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6817","nonce":4226577463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6815","nonce":4226577463}]},"public_addr":"172.21.15.101:6811/4226577463","cluster_addr":"172.21.15.101:6813/4226577463","heartbeat_back_addr":"172.21.15.101:6817/4226577463","heartbeat_front_addr":"172.21.15.101:6815/4226577463","state":["exists","up"]},{"osd":4,"uuid":"2137f2df-984e-45ce-baf6-b4ac12c7e3c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6809","nonce":1951135805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6811","nonce":1951135805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6815","nonce":1951135805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6813","nonce":1951135805}]},"public_addr":"172.21.15.155:6809/1951135805","cluster_addr":"172.21.15.155:6811/1951135805","heartbeat_back_addr":"172.21.15.155:6815/1951135805","heartbeat_front_addr":"172.21.15.155:6813/1951135805","state":["exists","up"]},{"osd":5,"uuid":"1020bfc5-8c2f-482e-9588-66c7f98e46e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6809","nonce":2746037021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6811","nonce":2746037021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6815","nonce":2746037021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6813","nonce":2746037021}]},"public_addr":"172.21.15.179:6809/2746037021","cluster_addr":"172.21.15.179:6811/2746037021","heartbeat_back_addr":"172.21.15.179:6815/2746037021","heartbeat_front_addr":"172.21.15.179:6813/2746037021","state":["exists","up"]},{"osd":6,"uuid":"2cd99260-5c36-4603-9338-cf697970ce58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6819","nonce":958044931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6821","nonce":958044931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6825","nonce":958044931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6823","nonce":958044931}]},"public_addr":"172.21.15.101:6819/958044931","cluster_addr":"172.21.15.101:6821/958044931","heartbeat_back_addr":"172.21.15.101:6825/958044931","heartbeat_front_addr":"172.21.15.101:6823/958044931","state":["exists","up"]},{"osd":7,"uuid":"ed23fd0a-678f-4e15-b42c-cfe873315714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6817","nonce":3256322291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6819","nonce":3256322291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6823","nonce":3256322291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6821","nonce":3256322291}]},"public_addr":"172.21.15.155:6817/3256322291","cluster_addr":"172.21.15.155:6819/3256322291","heartbeat_back_addr":"172.21.15.155:6823/3256322291","heartbeat_front_addr":"172.21.15.155:6821/3256322291","state":["exists","up"]},{"osd":8,"uuid":"eca3d8e6-7baf-47a5-b991-328001c7ebef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6817","nonce":3520612559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6819","nonce":3520612559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6823","nonce":3520612559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6821","nonce":3520612559}]},"public_addr":"172.21.15.179:6817/3520612559","cluster_addr":"172.21.15.179:6819/3520612559","heartbeat_back_addr":"172.21.15.179:6823/3520612559","heartbeat_front_addr":"172.21.15.179:6821/3520612559","state":["exists","up"]},{"osd":9,"uuid":"465a7ffb-67e4-49ef-812c-1b96feacfebf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6827","nonce":2166787667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6829","nonce":2166787667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6833","nonce":2166787667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6831","nonce":2166787667}]},"public_addr":"172.21.15.101:6827/2166787667","cluster_addr":"172.21.15.101:6829/2166787667","heartbeat_back_addr":"172.21.15.101:6833/2166787667","heartbeat_front_addr":"172.21.15.101:6831/2166787667","state":["exists","up"]},{"osd":10,"uuid":"1897a027-f25d-434f-8df5-5fc2bd866578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6825","nonce":3682446349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6827","nonce":3682446349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6831","nonce":3682446349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6829","nonce":3682446349}]},"public_addr":"172.21.15.155:6825/3682446349","cluster_addr":"172.21.15.155:6827/3682446349","heartbeat_back_addr":"172.21.15.155:6831/3682446349","heartbeat_front_addr":"172.21.15.155:6829/3682446349","state":["exists","up"]},{"osd":11,"uuid":"b62effaf-45e2-4585-b8d0-db448f3474ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6825","nonce":2602640858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6827","nonce":2602640858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6831","nonce":2602640858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6829","nonce":2602640858}]},"public_addr":"172.21.15.179:6825/2602640858","cluster_addr":"172.21.15.179:6827/2602640858","heartbeat_back_addr":"172.21.15.179:6831/2602640858","heartbeat_front_addr":"172.21.15.179:6829/2602640858","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T02:59:56.074519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:19.358142+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:41.128966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:08.227054+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:31.442592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:52.751685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:19.251054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:40.698241+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:01.339150+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:27.955910+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:49.136980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:04:10.084982+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.101:6800/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:6801/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/604228291":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/4055187849":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/2844574324":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/1603248215":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6801/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6800/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/3307578182":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/373189792":"2023-07-19T02:57:56.243200+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:05:34.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.845+0000 7f6deb9ff700 1 -- 172.21.15.101:0/3800072218 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f6dd00a1220 msgr2=0x7f6dd00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:34.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.845+0000 7f6deb9ff700 1 --2- 172.21.15.101:0/3800072218 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f6dd00a1220 0x7f6dd00a36d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6de000b2e0 tx=0x7f6de0005fb0 comp rx=0 tx=0).stop 2023-07-18T03:05:34.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.845+0000 7f6deb9ff700 1 -- 172.21.15.101:0/3800072218 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6de4105790 msgr2=0x7f6de419ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:34.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.845+0000 7f6deb9ff700 1 --2- 172.21.15.101:0/3800072218 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6de4105790 0x7f6de419ef80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc00ef30 tx=0x7f6ddc00c3a0 comp rx=0 tx=0).stop 2023-07-18T03:05:34.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.845+0000 7f6deb9ff700 1 -- 172.21.15.101:0/3800072218 shutdown_connections 2023-07-18T03:05:34.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.845+0000 7f6deb9ff700 1 --2- 172.21.15.101:0/3800072218 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f6dd00a1220 0x7f6dd00a36d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:34.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.845+0000 7f6deb9ff700 1 --2- 172.21.15.101:0/3800072218 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6de4105790 0x7f6de419ef80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:34.832 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.845+0000 7f6deb9ff700 1 --2- 172.21.15.101:0/3800072218 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6de4100550 0x7f6de41979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:34.832 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.845+0000 7f6deb9ff700 1 --2- 172.21.15.101:0/3800072218 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6de4102e70 0x7f6de4197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:34.832 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.845+0000 7f6deb9ff700 1 -- 172.21.15.101:0/3800072218 >> 172.21.15.101:0/3800072218 conn(0x7f6de40f9fd0 msgr2=0x7f6de40febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:34.832 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.845+0000 7f6deb9ff700 1 -- 172.21.15.101:0/3800072218 shutdown_connections 2023-07-18T03:05:34.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:34.846+0000 7f6deb9ff700 1 -- 172.21.15.101:0/3800072218 wait complete. 2023-07-18T03:05:34.849 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-18T03:05:34.849 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-18T03:05:35.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.062+0000 7fbfbc5ff700 1 -- 172.21.15.101:0/2064227502 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbfb4101680 msgr2=0x7fbfb4107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:35.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.062+0000 7fbfbc5ff700 1 --2- 172.21.15.101:0/2064227502 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbfb4101680 0x7fbfb4107f40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbfac00b0a0 tx=0x7fbfac00b3b0 comp rx=0 tx=0).stop 2023-07-18T03:05:35.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.062+0000 7fbfbc5ff700 1 -- 172.21.15.101:0/2064227502 shutdown_connections 2023-07-18T03:05:35.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.062+0000 7fbfbc5ff700 1 --2- 172.21.15.101:0/2064227502 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbfb4101680 0x7fbfb4107f40 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:35.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.062+0000 7fbfbc5ff700 1 --2- 172.21.15.101:0/2064227502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbfb40ffb00 0x7fbfb40fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:35.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.062+0000 7fbfbc5ff700 1 --2- 172.21.15.101:0/2064227502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfb4100cf0 0x7fbfb4101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:35.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.062+0000 7fbfbc5ff700 1 -- 172.21.15.101:0/2064227502 >> 172.21.15.101:0/2064227502 conn(0x7fbfb40fb0e0 msgr2=0x7fbfb40fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:35.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.062+0000 7fbfbc5ff700 1 -- 172.21.15.101:0/2064227502 shutdown_connections 2023-07-18T03:05:35.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.062+0000 7fbfbc5ff700 1 -- 172.21.15.101:0/2064227502 wait complete. 2023-07-18T03:05:35.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.063+0000 7fbfbc5ff700 1 Processor -- start 2023-07-18T03:05:35.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.063+0000 7fbfbc5ff700 1 -- start start 2023-07-18T03:05:35.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.063+0000 7fbfbc5ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbfb40ffb00 0x7fbfb4197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:35.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.063+0000 7fbfbc5ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbfb4100cf0 0x7fbfb4197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:35.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.063+0000 7fbfbc5ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfb4101680 0x7fbfb419f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:35.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.063+0000 7fbfbc5ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb419f630 con 0x7fbfb4100cf0 2023-07-18T03:05:35.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.063+0000 7fbfbc5ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb419f770 con 0x7fbfb4101680 2023-07-18T03:05:35.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.063+0000 7fbfbc5ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb419f900 con 0x7fbfb40ffb00 2023-07-18T03:05:35.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.064+0000 7fbfba39b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbfb40ffb00 0x7fbfb4197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:35.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.064+0000 7fbfba39b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbfb40ffb00 0x7fbfb4197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:43764/0 (socket says 172.21.15.101:43764) 2023-07-18T03:05:35.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.064+0000 7fbfba39b700 1 -- 172.21.15.101:0/903875987 learned_addr learned my addr 172.21.15.101:0/903875987 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:35.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.064+0000 7fbfb9b9a700 1 --2- 172.21.15.101:0/903875987 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbfb4100cf0 0x7fbfb4197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:35.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.064+0000 7fbfbab9c700 1 --2- 172.21.15.101:0/903875987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfb4101680 0x7fbfb419f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:35.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.064+0000 7fbfba39b700 1 -- 172.21.15.101:0/903875987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfb4101680 msgr2=0x7fbfb419f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:35.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.064+0000 7fbfba39b700 1 --2- 172.21.15.101:0/903875987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfb4101680 0x7fbfb419f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:35.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.064+0000 7fbfba39b700 1 -- 172.21.15.101:0/903875987 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbfb4100cf0 msgr2=0x7fbfb4197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:35.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.064+0000 7fbfba39b700 1 --2- 172.21.15.101:0/903875987 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbfb4100cf0 0x7fbfb4197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:35.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.064+0000 7fbfba39b700 1 -- 172.21.15.101:0/903875987 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfac009d00 con 0x7fbfb40ffb00 2023-07-18T03:05:35.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.064+0000 7fbfba39b700 1 --2- 172.21.15.101:0/903875987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbfb40ffb00 0x7fbfb4197ab0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbfb000e970 tx=0x7fbfb000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:35.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.064+0000 7fbfab7fe700 1 -- 172.21.15.101:0/903875987 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0009790 con 0x7fbfb40ffb00 2023-07-18T03:05:35.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.064+0000 7fbfab7fe700 1 -- 172.21.15.101:0/903875987 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfb000c7e0 con 0x7fbfb40ffb00 2023-07-18T03:05:35.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.064+0000 7fbfbc5ff700 1 -- 172.21.15.101:0/903875987 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfb419fbe0 con 0x7fbfb40ffb00 2023-07-18T03:05:35.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.064+0000 7fbfab7fe700 1 -- 172.21.15.101:0/903875987 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0003c70 con 0x7fbfb40ffb00 2023-07-18T03:05:35.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.065+0000 7fbfbc5ff700 1 -- 172.21.15.101:0/903875987 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfb41a0140 con 0x7fbfb40ffb00 2023-07-18T03:05:35.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.066+0000 7fbfbc5ff700 1 -- 172.21.15.101:0/903875987 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfb410b2e0 con 0x7fbfb40ffb00 2023-07-18T03:05:35.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.068+0000 7fbfab7fe700 1 -- 172.21.15.101:0/903875987 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fbfb000cc50 con 0x7fbfb40ffb00 2023-07-18T03:05:35.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.069+0000 7fbfab7fe700 1 --2- 172.21.15.101:0/903875987 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fbfa00a12a0 0x7fbfa00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:35.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.069+0000 7fbfab7fe700 1 -- 172.21.15.101:0/903875987 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fbfb0014070 con 0x7fbfb40ffb00 2023-07-18T03:05:35.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.069+0000 7fbfb9b9a700 1 --2- 172.21.15.101:0/903875987 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fbfa00a12a0 0x7fbfa00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:35.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.070+0000 7fbfb9b9a700 1 --2- 172.21.15.101:0/903875987 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fbfa00a12a0 0x7fbfa00a3750 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4009c60 tx=0x7fbfa4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:35.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.071+0000 7fbfab7fe700 1 -- 172.21.15.101:0/903875987 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fbfb0096820 con 0x7fbfb40ffb00 2023-07-18T03:05:35.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:35 smithi101 ceph-mon[126379]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:05:35.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:35 smithi101 ceph-mon[126379]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:05:35.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:35 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2866106597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:05:35.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:35 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3800072218' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:05:35.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.267+0000 7fbfbc5ff700 1 -- 172.21.15.101:0/903875987 --> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbfb4107c20 con 0x7fbfa00a12a0 2023-07-18T03:05:35.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.282+0000 7fbfab7fe700 1 -- 172.21.15.101:0/903875987 <== mgr.14150 v2:172.21.15.101:6800/3963742842 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422830 (secure 0 0 0) 0x7fbfb4107c20 con 0x7fbfa00a12a0 2023-07-18T03:05:35.272 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:05:35.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.287+0000 7fbfbc5ff700 1 -- 172.21.15.101:0/903875987 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fbfa00a12a0 msgr2=0x7fbfa00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:35.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.287+0000 7fbfbc5ff700 1 --2- 172.21.15.101:0/903875987 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fbfa00a12a0 0x7fbfa00a3750 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4009c60 tx=0x7fbfa4009260 comp rx=0 tx=0).stop 2023-07-18T03:05:35.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.287+0000 7fbfbc5ff700 1 -- 172.21.15.101:0/903875987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbfb40ffb00 msgr2=0x7fbfb4197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:35.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.287+0000 7fbfbc5ff700 1 --2- 172.21.15.101:0/903875987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbfb40ffb00 0x7fbfb4197ab0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbfb000e970 tx=0x7fbfb000ec80 comp rx=0 tx=0).stop 2023-07-18T03:05:35.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.287+0000 7fbfbc5ff700 1 -- 172.21.15.101:0/903875987 shutdown_connections 2023-07-18T03:05:35.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.287+0000 7fbfbc5ff700 1 --2- 172.21.15.101:0/903875987 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fbfa00a12a0 0x7fbfa00a3750 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:35.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.287+0000 7fbfbc5ff700 1 --2- 172.21.15.101:0/903875987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfb4101680 0x7fbfb419f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:35.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.287+0000 7fbfbc5ff700 1 --2- 172.21.15.101:0/903875987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbfb40ffb00 0x7fbfb4197ab0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:35.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.287+0000 7fbfbc5ff700 1 --2- 172.21.15.101:0/903875987 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbfb4100cf0 0x7fbfb4197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:35.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.287+0000 7fbfbc5ff700 1 -- 172.21.15.101:0/903875987 >> 172.21.15.101:0/903875987 conn(0x7fbfb40fb0e0 msgr2=0x7fbfb4106500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:35.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.287+0000 7fbfbc5ff700 1 -- 172.21.15.101:0/903875987 shutdown_connections 2023-07-18T03:05:35.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.287+0000 7fbfbc5ff700 1 -- 172.21.15.101:0/903875987 wait complete. 2023-07-18T03:05:35.280 INFO:teuthology.orchestra.run.smithi101.stderr:dumped all 2023-07-18T03:05:35.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:35 smithi155 ceph-mon[128885]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:05:35.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:35 smithi155 ceph-mon[128885]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:05:35.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:35 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/2866106597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:05:35.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:35 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/3800072218' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:05:35.291 INFO:teuthology.orchestra.run.smithi101.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2023-07-18T03:05:34.855176+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597922,"num_objects":59,"num_object_clones":0,"num_object_copies":179,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":151,"num_read_kb":132,"num_write":301,"num_write_kb":2983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":242,"ondisk_log_size":242,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335092,"kb_used_data":11668,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606732,"statfs":{"total":1151940427776,"available":1151597293568,"internally_reserved":0,"allocated":11948032,"data_stored":7615145,"data_compressed":318153,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2962,"num_objects":35,"num_object_clones":0,"num_object_copies":107,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":15,"num_read_kb":13,"num_write":71,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.923051"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.942306+0000","last_change":"2023-07-18T03:04:59.805689+0000","last_active":"2023-07-18T03:05:23.942306+0000","last_peered":"2023-07-18T03:05:23.942306+0000","last_clean":"2023-07-18T03:05:23.942306+0000","last_became_active":"2023-07-18T03:04:59.805224+0000","last_became_peered":"2023-07-18T03:04:59.805224+0000","last_unstale":"2023-07-18T03:05:23.942306+0000","last_undegraded":"2023-07-18T03:05:23.942306+0000","last_fullsized":"2023-07-18T03:05:23.942306+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:55:30.570149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.938022+0000","last_change":"2023-07-18T03:04:59.805013+0000","last_active":"2023-07-18T03:05:23.938022+0000","last_peered":"2023-07-18T03:05:23.938022+0000","last_clean":"2023-07-18T03:05:23.938022+0000","last_became_active":"2023-07-18T03:04:59.804702+0000","last_became_peered":"2023-07-18T03:04:59.804702+0000","last_unstale":"2023-07-18T03:05:23.938022+0000","last_undegraded":"2023-07-18T03:05:23.938022+0000","last_fullsized":"2023-07-18T03:05:23.938022+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:11:13.364060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.938179+0000","last_change":"2023-07-18T03:04:59.809035+0000","last_active":"2023-07-18T03:05:23.938179+0000","last_peered":"2023-07-18T03:05:23.938179+0000","last_clean":"2023-07-18T03:05:23.938179+0000","last_became_active":"2023-07-18T03:04:59.808630+0000","last_became_peered":"2023-07-18T03:04:59.808630+0000","last_unstale":"2023-07-18T03:05:23.938179+0000","last_undegraded":"2023-07-18T03:05:23.938179+0000","last_fullsized":"2023-07-18T03:05:23.938179+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:18:02.359613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089479+0000","last_change":"2023-07-18T03:04:59.803510+0000","last_active":"2023-07-18T03:05:24.089479+0000","last_peered":"2023-07-18T03:05:24.089479+0000","last_clean":"2023-07-18T03:05:24.089479+0000","last_became_active":"2023-07-18T03:04:59.803060+0000","last_became_peered":"2023-07-18T03:04:59.803060+0000","last_unstale":"2023-07-18T03:05:24.089479+0000","last_undegraded":"2023-07-18T03:05:24.089479+0000","last_fullsized":"2023-07-18T03:05:24.089479+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:15:54.456383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.936577+0000","last_change":"2023-07-18T03:04:59.806911+0000","last_active":"2023-07-18T03:05:23.936577+0000","last_peered":"2023-07-18T03:05:23.936577+0000","last_clean":"2023-07-18T03:05:23.936577+0000","last_became_active":"2023-07-18T03:04:59.806597+0000","last_became_peered":"2023-07-18T03:04:59.806597+0000","last_unstale":"2023-07-18T03:05:23.936577+0000","last_undegraded":"2023-07-18T03:05:23.936577+0000","last_fullsized":"2023-07-18T03:05:23.936577+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:06:55.956671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.936608+0000","last_change":"2023-07-18T03:04:59.809263+0000","last_active":"2023-07-18T03:05:23.936608+0000","last_peered":"2023-07-18T03:05:23.936608+0000","last_clean":"2023-07-18T03:05:23.936608+0000","last_became_active":"2023-07-18T03:04:59.808956+0000","last_became_peered":"2023-07-18T03:04:59.808956+0000","last_unstale":"2023-07-18T03:05:23.936608+0000","last_undegraded":"2023-07-18T03:05:23.936608+0000","last_fullsized":"2023-07-18T03:05:23.936608+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:49:07.735738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.933471+0000","last_change":"2023-07-18T03:04:59.804946+0000","last_active":"2023-07-18T03:05:23.933471+0000","last_peered":"2023-07-18T03:05:23.933471+0000","last_clean":"2023-07-18T03:05:23.933471+0000","last_became_active":"2023-07-18T03:04:59.804610+0000","last_became_peered":"2023-07-18T03:04:59.804610+0000","last_unstale":"2023-07-18T03:05:23.933471+0000","last_undegraded":"2023-07-18T03:05:23.933471+0000","last_fullsized":"2023-07-18T03:05:23.933471+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:00:45.505896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089839+0000","last_change":"2023-07-18T03:04:59.801796+0000","last_active":"2023-07-18T03:05:24.089839+0000","last_peered":"2023-07-18T03:05:24.089839+0000","last_clean":"2023-07-18T03:05:24.089839+0000","last_became_active":"2023-07-18T03:04:59.801432+0000","last_became_peered":"2023-07-18T03:04:59.801432+0000","last_unstale":"2023-07-18T03:05:24.089839+0000","last_undegraded":"2023-07-18T03:05:24.089839+0000","last_fullsized":"2023-07-18T03:05:24.089839+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:25:50.242446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089997+0000","last_change":"2023-07-18T03:04:59.798191+0000","last_active":"2023-07-18T03:05:24.089997+0000","last_peered":"2023-07-18T03:05:24.089997+0000","last_clean":"2023-07-18T03:05:24.089997+0000","last_became_active":"2023-07-18T03:04:59.797863+0000","last_became_peered":"2023-07-18T03:04:59.797863+0000","last_unstale":"2023-07-18T03:05:24.089997+0000","last_undegraded":"2023-07-18T03:05:24.089997+0000","last_fullsized":"2023-07-18T03:05:24.089997+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:23:59.798693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.936307+0000","last_change":"2023-07-18T03:04:59.809867+0000","last_active":"2023-07-18T03:05:23.936307+0000","last_peered":"2023-07-18T03:05:23.936307+0000","last_clean":"2023-07-18T03:05:23.936307+0000","last_became_active":"2023-07-18T03:04:59.809568+0000","last_became_peered":"2023-07-18T03:04:59.809568+0000","last_unstale":"2023-07-18T03:05:23.936307+0000","last_undegraded":"2023-07-18T03:05:23.936307+0000","last_fullsized":"2023-07-18T03:05:23.936307+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:51:02.596324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.934225+0000","last_change":"2023-07-18T03:04:59.805920+0000","last_active":"2023-07-18T03:05:23.934225+0000","last_peered":"2023-07-18T03:05:23.934225+0000","last_clean":"2023-07-18T03:05:23.934225+0000","last_became_active":"2023-07-18T03:04:59.805559+0000","last_became_peered":"2023-07-18T03:04:59.805559+0000","last_unstale":"2023-07-18T03:05:23.934225+0000","last_undegraded":"2023-07-18T03:05:23.934225+0000","last_fullsized":"2023-07-18T03:05:23.934225+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:57:01.069959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089324+0000","last_change":"2023-07-18T03:04:59.797940+0000","last_active":"2023-07-18T03:05:24.089324+0000","last_peered":"2023-07-18T03:05:24.089324+0000","last_clean":"2023-07-18T03:05:24.089324+0000","last_became_active":"2023-07-18T03:04:59.797575+0000","last_became_peered":"2023-07-18T03:04:59.797575+0000","last_unstale":"2023-07-18T03:05:24.089324+0000","last_undegraded":"2023-07-18T03:05:24.089324+0000","last_fullsized":"2023-07-18T03:05:24.089324+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:25:37.959233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.091650+0000","last_change":"2023-07-18T03:04:59.798026+0000","last_active":"2023-07-18T03:05:24.091650+0000","last_peered":"2023-07-18T03:05:24.091650+0000","last_clean":"2023-07-18T03:05:24.091650+0000","last_became_active":"2023-07-18T03:04:59.797734+0000","last_became_peered":"2023-07-18T03:04:59.797734+0000","last_unstale":"2023-07-18T03:05:24.091650+0000","last_undegraded":"2023-07-18T03:05:24.091650+0000","last_fullsized":"2023-07-18T03:05:24.091650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:33:14.476227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.938064+0000","last_change":"2023-07-18T03:04:59.804964+0000","last_active":"2023-07-18T03:05:23.938064+0000","last_peered":"2023-07-18T03:05:23.938064+0000","last_clean":"2023-07-18T03:05:23.938064+0000","last_became_active":"2023-07-18T03:04:59.804660+0000","last_became_peered":"2023-07-18T03:04:59.804660+0000","last_unstale":"2023-07-18T03:05:23.938064+0000","last_undegraded":"2023-07-18T03:05:23.938064+0000","last_fullsized":"2023-07-18T03:05:23.938064+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:31:37.327250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.938212+0000","last_change":"2023-07-18T03:04:59.809384+0000","last_active":"2023-07-18T03:05:23.938212+0000","last_peered":"2023-07-18T03:05:23.938212+0000","last_clean":"2023-07-18T03:05:23.938212+0000","last_became_active":"2023-07-18T03:04:59.808969+0000","last_became_peered":"2023-07-18T03:04:59.808969+0000","last_unstale":"2023-07-18T03:05:23.938212+0000","last_undegraded":"2023-07-18T03:05:23.938212+0000","last_fullsized":"2023-07-18T03:05:23.938212+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:41:11.572546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089702+0000","last_change":"2023-07-18T03:04:59.808344+0000","last_active":"2023-07-18T03:05:24.089702+0000","last_peered":"2023-07-18T03:05:24.089702+0000","last_clean":"2023-07-18T03:05:24.089702+0000","last_became_active":"2023-07-18T03:04:59.807995+0000","last_became_peered":"2023-07-18T03:04:59.807995+0000","last_unstale":"2023-07-18T03:05:24.089702+0000","last_undegraded":"2023-07-18T03:05:24.089702+0000","last_fullsized":"2023-07-18T03:05:24.089702+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:20:11.542223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.942105+0000","last_change":"2023-07-18T03:04:59.806584+0000","last_active":"2023-07-18T03:05:23.942105+0000","last_peered":"2023-07-18T03:05:23.942105+0000","last_clean":"2023-07-18T03:05:23.942105+0000","last_became_active":"2023-07-18T03:04:59.804118+0000","last_became_peered":"2023-07-18T03:04:59.804118+0000","last_unstale":"2023-07-18T03:05:23.942105+0000","last_undegraded":"2023-07-18T03:05:23.942105+0000","last_fullsized":"2023-07-18T03:05:23.942105+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:10:34.684069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.934563+0000","last_change":"2023-07-18T03:04:59.812610+0000","last_active":"2023-07-18T03:05:23.934563+0000","last_peered":"2023-07-18T03:05:23.934563+0000","last_clean":"2023-07-18T03:05:23.934563+0000","last_became_active":"2023-07-18T03:04:59.812230+0000","last_became_peered":"2023-07-18T03:04:59.812230+0000","last_unstale":"2023-07-18T03:05:23.934563+0000","last_undegraded":"2023-07-18T03:05:23.934563+0000","last_fullsized":"2023-07-18T03:05:23.934563+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:05:54.390233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.939088+0000","last_change":"2023-07-18T03:04:59.811880+0000","last_active":"2023-07-18T03:05:23.939088+0000","last_peered":"2023-07-18T03:05:23.939088+0000","last_clean":"2023-07-18T03:05:23.939088+0000","last_became_active":"2023-07-18T03:04:59.811561+0000","last_became_peered":"2023-07-18T03:04:59.811561+0000","last_unstale":"2023-07-18T03:05:23.939088+0000","last_undegraded":"2023-07-18T03:05:23.939088+0000","last_fullsized":"2023-07-18T03:05:23.939088+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:49:17.068386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.938056+0000","last_change":"2023-07-18T03:05:03.880843+0000","last_active":"2023-07-18T03:05:23.938056+0000","last_peered":"2023-07-18T03:05:23.938056+0000","last_clean":"2023-07-18T03:05:23.938056+0000","last_became_active":"2023-07-18T03:05:03.880391+0000","last_became_peered":"2023-07-18T03:05:03.880391+0000","last_unstale":"2023-07-18T03:05:23.938056+0000","last_undegraded":"2023-07-18T03:05:23.938056+0000","last_fullsized":"2023-07-18T03:05:23.938056+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:57:04.981691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'4","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.934651+0000","last_change":"2023-07-18T03:04:58.848771+0000","last_active":"2023-07-18T03:05:23.934651+0000","last_peered":"2023-07-18T03:05:23.934651+0000","last_clean":"2023-07-18T03:05:23.934651+0000","last_became_active":"2023-07-18T03:04:58.848405+0000","last_became_peered":"2023-07-18T03:04:58.848405+0000","last_unstale":"2023-07-18T03:05:23.934651+0000","last_undegraded":"2023-07-18T03:05:23.934651+0000","last_fullsized":"2023-07-18T03:05:23.934651+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:21:40.139702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.090125+0000","last_change":"2023-07-18T03:04:59.805749+0000","last_active":"2023-07-18T03:05:24.090125+0000","last_peered":"2023-07-18T03:05:24.090125+0000","last_clean":"2023-07-18T03:05:24.090125+0000","last_became_active":"2023-07-18T03:04:59.805388+0000","last_became_peered":"2023-07-18T03:04:59.805388+0000","last_unstale":"2023-07-18T03:05:24.090125+0000","last_undegraded":"2023-07-18T03:05:24.090125+0000","last_fullsized":"2023-07-18T03:05:24.090125+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:12:30.269632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089071+0000","last_change":"2023-07-18T03:05:03.868819+0000","last_active":"2023-07-18T03:05:24.089071+0000","last_peered":"2023-07-18T03:05:24.089071+0000","last_clean":"2023-07-18T03:05:24.089071+0000","last_became_active":"2023-07-18T03:05:03.868387+0000","last_became_peered":"2023-07-18T03:05:03.868387+0000","last_unstale":"2023-07-18T03:05:24.089071+0000","last_undegraded":"2023-07-18T03:05:24.089071+0000","last_fullsized":"2023-07-18T03:05:24.089071+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:07:41.670443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"82'1","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.090369+0000","last_change":"2023-07-18T03:04:58.856084+0000","last_active":"2023-07-18T03:05:24.090369+0000","last_peered":"2023-07-18T03:05:24.090369+0000","last_clean":"2023-07-18T03:05:24.090369+0000","last_became_active":"2023-07-18T03:04:58.855768+0000","last_became_peered":"2023-07-18T03:04:58.855768+0000","last_unstale":"2023-07-18T03:05:24.090369+0000","last_undegraded":"2023-07-18T03:05:24.090369+0000","last_fullsized":"2023-07-18T03:05:24.090369+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:28:05.975256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.938087+0000","last_change":"2023-07-18T03:04:59.809512+0000","last_active":"2023-07-18T03:05:23.938087+0000","last_peered":"2023-07-18T03:05:23.938087+0000","last_clean":"2023-07-18T03:05:23.938087+0000","last_became_active":"2023-07-18T03:04:59.808112+0000","last_became_peered":"2023-07-18T03:04:59.808112+0000","last_unstale":"2023-07-18T03:05:23.938087+0000","last_undegraded":"2023-07-18T03:05:23.938087+0000","last_fullsized":"2023-07-18T03:05:23.938087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:32:42.763158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.090171+0000","last_change":"2023-07-18T03:05:03.874728+0000","last_active":"2023-07-18T03:05:24.090171+0000","last_peered":"2023-07-18T03:05:24.090171+0000","last_clean":"2023-07-18T03:05:24.090171+0000","last_became_active":"2023-07-18T03:05:03.874262+0000","last_became_peered":"2023-07-18T03:05:03.874262+0000","last_unstale":"2023-07-18T03:05:24.090171+0000","last_undegraded":"2023-07-18T03:05:24.090171+0000","last_fullsized":"2023-07-18T03:05:24.090171+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:52:09.021724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:30.990070+0000","last_change":"2023-07-18T03:04:58.849140+0000","last_active":"2023-07-18T03:05:30.990070+0000","last_peered":"2023-07-18T03:05:30.990070+0000","last_clean":"2023-07-18T03:05:30.990070+0000","last_became_active":"2023-07-18T03:04:58.848693+0000","last_became_peered":"2023-07-18T03:04:58.848693+0000","last_unstale":"2023-07-18T03:05:30.990070+0000","last_undegraded":"2023-07-18T03:05:30.990070+0000","last_fullsized":"2023-07-18T03:05:30.990070+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:43:52.238305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089728+0000","last_change":"2023-07-18T03:04:59.807248+0000","last_active":"2023-07-18T03:05:24.089728+0000","last_peered":"2023-07-18T03:05:24.089728+0000","last_clean":"2023-07-18T03:05:24.089728+0000","last_became_active":"2023-07-18T03:04:59.806807+0000","last_became_peered":"2023-07-18T03:04:59.806807+0000","last_unstale":"2023-07-18T03:05:24.089728+0000","last_undegraded":"2023-07-18T03:05:24.089728+0000","last_fullsized":"2023-07-18T03:05:24.089728+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:35:50.471833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.088762+0000","last_change":"2023-07-18T03:05:03.869075+0000","last_active":"2023-07-18T03:05:24.088762+0000","last_peered":"2023-07-18T03:05:24.088762+0000","last_clean":"2023-07-18T03:05:24.088762+0000","last_became_active":"2023-07-18T03:05:03.868653+0000","last_became_peered":"2023-07-18T03:05:03.868653+0000","last_unstale":"2023-07-18T03:05:24.088762+0000","last_undegraded":"2023-07-18T03:05:24.088762+0000","last_fullsized":"2023-07-18T03:05:24.088762+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:44:09.437492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.937411+0000","last_change":"2023-07-18T03:04:58.843879+0000","last_active":"2023-07-18T03:05:23.937411+0000","last_peered":"2023-07-18T03:05:23.937411+0000","last_clean":"2023-07-18T03:05:23.937411+0000","last_became_active":"2023-07-18T03:04:58.843525+0000","last_became_peered":"2023-07-18T03:04:58.843525+0000","last_unstale":"2023-07-18T03:05:23.937411+0000","last_undegraded":"2023-07-18T03:05:23.937411+0000","last_fullsized":"2023-07-18T03:05:23.937411+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:42:05.200711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089568+0000","last_change":"2023-07-18T03:04:59.803597+0000","last_active":"2023-07-18T03:05:24.089568+0000","last_peered":"2023-07-18T03:05:24.089568+0000","last_clean":"2023-07-18T03:05:24.089568+0000","last_became_active":"2023-07-18T03:04:59.803293+0000","last_became_peered":"2023-07-18T03:04:59.803293+0000","last_unstale":"2023-07-18T03:05:24.089568+0000","last_undegraded":"2023-07-18T03:05:24.089568+0000","last_fullsized":"2023-07-18T03:05:24.089568+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:21:34.711198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.944721+0000","last_change":"2023-07-18T03:05:23.944721+0000","last_active":"2023-07-18T03:05:23.944721+0000","last_peered":"2023-07-18T03:05:23.944721+0000","last_clean":"2023-07-18T03:05:23.944721+0000","last_became_active":"2023-07-18T03:05:23.944101+0000","last_became_peered":"2023-07-18T03:05:23.944101+0000","last_unstale":"2023-07-18T03:05:23.944721+0000","last_undegraded":"2023-07-18T03:05:23.944721+0000","last_fullsized":"2023-07-18T03:05:23.944721+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:28:03.237639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:29.985509+0000","last_change":"2023-07-18T03:04:58.847990+0000","last_active":"2023-07-18T03:05:29.985509+0000","last_peered":"2023-07-18T03:05:29.985509+0000","last_clean":"2023-07-18T03:05:29.985509+0000","last_became_active":"2023-07-18T03:04:58.847634+0000","last_became_peered":"2023-07-18T03:04:58.847634+0000","last_unstale":"2023-07-18T03:05:29.985509+0000","last_undegraded":"2023-07-18T03:05:29.985509+0000","last_fullsized":"2023-07-18T03:05:29.985509+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:03:43.456118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.938391+0000","last_change":"2023-07-18T03:04:59.797076+0000","last_active":"2023-07-18T03:05:23.938391+0000","last_peered":"2023-07-18T03:05:23.938391+0000","last_clean":"2023-07-18T03:05:23.938391+0000","last_became_active":"2023-07-18T03:04:59.796753+0000","last_became_peered":"2023-07-18T03:04:59.796753+0000","last_unstale":"2023-07-18T03:05:23.938391+0000","last_undegraded":"2023-07-18T03:05:23.938391+0000","last_fullsized":"2023-07-18T03:05:23.938391+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:28:30.559043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.937833+0000","last_change":"2023-07-18T03:05:03.881793+0000","last_active":"2023-07-18T03:05:23.937833+0000","last_peered":"2023-07-18T03:05:23.937833+0000","last_clean":"2023-07-18T03:05:23.937833+0000","last_became_active":"2023-07-18T03:05:03.881341+0000","last_became_peered":"2023-07-18T03:05:03.881341+0000","last_unstale":"2023-07-18T03:05:23.937833+0000","last_undegraded":"2023-07-18T03:05:23.937833+0000","last_fullsized":"2023-07-18T03:05:23.937833+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:05:01.526051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:30.990979+0000","last_change":"2023-07-18T03:04:58.846732+0000","last_active":"2023-07-18T03:05:30.990979+0000","last_peered":"2023-07-18T03:05:30.990979+0000","last_clean":"2023-07-18T03:05:30.990979+0000","last_became_active":"2023-07-18T03:04:58.846321+0000","last_became_peered":"2023-07-18T03:04:58.846321+0000","last_unstale":"2023-07-18T03:05:30.990979+0000","last_undegraded":"2023-07-18T03:05:30.990979+0000","last_fullsized":"2023-07-18T03:05:30.990979+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:23:54.252356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.094233+0000","last_change":"2023-07-18T03:05:24.093277+0000","last_active":"2023-07-18T03:05:24.094233+0000","last_peered":"2023-07-18T03:05:24.094233+0000","last_clean":"2023-07-18T03:05:24.094233+0000","last_became_active":"2023-07-18T03:05:24.092842+0000","last_became_peered":"2023-07-18T03:05:24.092842+0000","last_unstale":"2023-07-18T03:05:24.094233+0000","last_undegraded":"2023-07-18T03:05:24.094233+0000","last_fullsized":"2023-07-18T03:05:24.094233+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:39:40.953742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.942246+0000","last_change":"2023-07-18T03:05:03.880782+0000","last_active":"2023-07-18T03:05:23.942246+0000","last_peered":"2023-07-18T03:05:23.942246+0000","last_clean":"2023-07-18T03:05:23.942246+0000","last_became_active":"2023-07-18T03:05:03.880362+0000","last_became_peered":"2023-07-18T03:05:03.880362+0000","last_unstale":"2023-07-18T03:05:23.942246+0000","last_undegraded":"2023-07-18T03:05:23.942246+0000","last_fullsized":"2023-07-18T03:05:23.942246+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:19:49.336092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:30.991863+0000","last_change":"2023-07-18T03:04:58.846095+0000","last_active":"2023-07-18T03:05:30.991863+0000","last_peered":"2023-07-18T03:05:30.991863+0000","last_clean":"2023-07-18T03:05:30.991863+0000","last_became_active":"2023-07-18T03:04:58.845242+0000","last_became_peered":"2023-07-18T03:04:58.845242+0000","last_unstale":"2023-07-18T03:05:30.991863+0000","last_undegraded":"2023-07-18T03:05:30.991863+0000","last_fullsized":"2023-07-18T03:05:30.991863+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:51:23.745763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089587+0000","last_change":"2023-07-18T03:04:59.803024+0000","last_active":"2023-07-18T03:05:24.089587+0000","last_peered":"2023-07-18T03:05:24.089587+0000","last_clean":"2023-07-18T03:05:24.089587+0000","last_became_active":"2023-07-18T03:04:59.802687+0000","last_became_peered":"2023-07-18T03:04:59.802687+0000","last_unstale":"2023-07-18T03:05:24.089587+0000","last_undegraded":"2023-07-18T03:05:24.089587+0000","last_fullsized":"2023-07-18T03:05:24.089587+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:14:46.616130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.937966+0000","last_change":"2023-07-18T03:05:03.872690+0000","last_active":"2023-07-18T03:05:23.937966+0000","last_peered":"2023-07-18T03:05:23.937966+0000","last_clean":"2023-07-18T03:05:23.937966+0000","last_became_active":"2023-07-18T03:05:03.872187+0000","last_became_peered":"2023-07-18T03:05:03.872187+0000","last_unstale":"2023-07-18T03:05:23.937966+0000","last_undegraded":"2023-07-18T03:05:23.937966+0000","last_fullsized":"2023-07-18T03:05:23.937966+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:05:04.373571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089675+0000","last_change":"2023-07-18T03:04:58.842877+0000","last_active":"2023-07-18T03:05:24.089675+0000","last_peered":"2023-07-18T03:05:24.089675+0000","last_clean":"2023-07-18T03:05:24.089675+0000","last_became_active":"2023-07-18T03:04:58.842516+0000","last_became_peered":"2023-07-18T03:04:58.842516+0000","last_unstale":"2023-07-18T03:05:24.089675+0000","last_undegraded":"2023-07-18T03:05:24.089675+0000","last_fullsized":"2023-07-18T03:05:24.089675+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:55:08.128143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.933854+0000","last_change":"2023-07-18T03:04:59.807592+0000","last_active":"2023-07-18T03:05:23.933854+0000","last_peered":"2023-07-18T03:05:23.933854+0000","last_clean":"2023-07-18T03:05:23.933854+0000","last_became_active":"2023-07-18T03:04:59.807276+0000","last_became_peered":"2023-07-18T03:04:59.807276+0000","last_unstale":"2023-07-18T03:05:23.933854+0000","last_undegraded":"2023-07-18T03:05:23.933854+0000","last_fullsized":"2023-07-18T03:05:23.933854+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:58:29.880849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089882+0000","last_change":"2023-07-18T03:05:03.876355+0000","last_active":"2023-07-18T03:05:24.089882+0000","last_peered":"2023-07-18T03:05:24.089882+0000","last_clean":"2023-07-18T03:05:24.089882+0000","last_became_active":"2023-07-18T03:05:03.875845+0000","last_became_peered":"2023-07-18T03:05:03.875845+0000","last_unstale":"2023-07-18T03:05:24.089882+0000","last_undegraded":"2023-07-18T03:05:24.089882+0000","last_fullsized":"2023-07-18T03:05:24.089882+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:01:54.675024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:29.985713+0000","last_change":"2023-07-18T03:04:58.840389+0000","last_active":"2023-07-18T03:05:29.985713+0000","last_peered":"2023-07-18T03:05:29.985713+0000","last_clean":"2023-07-18T03:05:29.985713+0000","last_became_active":"2023-07-18T03:04:58.839974+0000","last_became_peered":"2023-07-18T03:04:58.839974+0000","last_unstale":"2023-07-18T03:05:29.985713+0000","last_undegraded":"2023-07-18T03:05:29.985713+0000","last_fullsized":"2023-07-18T03:05:29.985713+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:21:20.010136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.090769+0000","last_change":"2023-07-18T03:04:59.807681+0000","last_active":"2023-07-18T03:05:24.090769+0000","last_peered":"2023-07-18T03:05:24.090769+0000","last_clean":"2023-07-18T03:05:24.090769+0000","last_became_active":"2023-07-18T03:04:59.807365+0000","last_became_peered":"2023-07-18T03:04:59.807365+0000","last_unstale":"2023-07-18T03:05:24.090769+0000","last_undegraded":"2023-07-18T03:05:24.090769+0000","last_fullsized":"2023-07-18T03:05:24.090769+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:35:09.523428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.092631+0000","last_change":"2023-07-18T03:05:24.091413+0000","last_active":"2023-07-18T03:05:24.092631+0000","last_peered":"2023-07-18T03:05:24.092631+0000","last_clean":"2023-07-18T03:05:24.092631+0000","last_became_active":"2023-07-18T03:05:24.090815+0000","last_became_peered":"2023-07-18T03:05:24.090815+0000","last_unstale":"2023-07-18T03:05:24.092631+0000","last_undegraded":"2023-07-18T03:05:24.092631+0000","last_fullsized":"2023-07-18T03:05:24.092631+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:18:39.070780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:29.984164+0000","last_change":"2023-07-18T03:04:58.846707+0000","last_active":"2023-07-18T03:05:29.984164+0000","last_peered":"2023-07-18T03:05:29.984164+0000","last_clean":"2023-07-18T03:05:29.984164+0000","last_became_active":"2023-07-18T03:04:58.846366+0000","last_became_peered":"2023-07-18T03:04:58.846366+0000","last_unstale":"2023-07-18T03:05:29.984164+0000","last_undegraded":"2023-07-18T03:05:29.984164+0000","last_fullsized":"2023-07-18T03:05:29.984164+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:54:55.036916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.092870+0000","last_change":"2023-07-18T03:05:24.092870+0000","last_active":"2023-07-18T03:05:24.092870+0000","last_peered":"2023-07-18T03:05:24.092870+0000","last_clean":"2023-07-18T03:05:24.092870+0000","last_became_active":"2023-07-18T03:05:24.092474+0000","last_became_peered":"2023-07-18T03:05:24.092474+0000","last_unstale":"2023-07-18T03:05:24.092870+0000","last_undegraded":"2023-07-18T03:05:24.092870+0000","last_fullsized":"2023-07-18T03:05:24.092870+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:22:48.062183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089952+0000","last_change":"2023-07-18T03:05:03.874444+0000","last_active":"2023-07-18T03:05:24.089952+0000","last_peered":"2023-07-18T03:05:24.089952+0000","last_clean":"2023-07-18T03:05:24.089952+0000","last_became_active":"2023-07-18T03:05:03.873737+0000","last_became_peered":"2023-07-18T03:05:03.873737+0000","last_unstale":"2023-07-18T03:05:24.089952+0000","last_undegraded":"2023-07-18T03:05:24.089952+0000","last_fullsized":"2023-07-18T03:05:24.089952+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:33:41.739156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089471+0000","last_change":"2023-07-18T03:04:58.845207+0000","last_active":"2023-07-18T03:05:24.089471+0000","last_peered":"2023-07-18T03:05:24.089471+0000","last_clean":"2023-07-18T03:05:24.089471+0000","last_became_active":"2023-07-18T03:04:58.844895+0000","last_became_peered":"2023-07-18T03:04:58.844895+0000","last_unstale":"2023-07-18T03:05:24.089471+0000","last_undegraded":"2023-07-18T03:05:24.089471+0000","last_fullsized":"2023-07-18T03:05:24.089471+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:16:23.615174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.090081+0000","last_change":"2023-07-18T03:04:59.807075+0000","last_active":"2023-07-18T03:05:24.090081+0000","last_peered":"2023-07-18T03:05:24.090081+0000","last_clean":"2023-07-18T03:05:24.090081+0000","last_became_active":"2023-07-18T03:04:59.806345+0000","last_became_peered":"2023-07-18T03:04:59.806345+0000","last_unstale":"2023-07-18T03:05:24.090081+0000","last_undegraded":"2023-07-18T03:05:24.090081+0000","last_fullsized":"2023-07-18T03:05:24.090081+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:38:38.275381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.091509+0000","last_change":"2023-07-18T03:05:24.091132+0000","last_active":"2023-07-18T03:05:24.091509+0000","last_peered":"2023-07-18T03:05:24.091509+0000","last_clean":"2023-07-18T03:05:24.091509+0000","last_became_active":"2023-07-18T03:05:24.090653+0000","last_became_peered":"2023-07-18T03:05:24.090653+0000","last_unstale":"2023-07-18T03:05:24.091509+0000","last_undegraded":"2023-07-18T03:05:24.091509+0000","last_fullsized":"2023-07-18T03:05:24.091509+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:37:46.585074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"89'12","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:29.985596+0000","last_change":"2023-07-18T03:04:58.844865+0000","last_active":"2023-07-18T03:05:29.985596+0000","last_peered":"2023-07-18T03:05:29.985596+0000","last_clean":"2023-07-18T03:05:29.985596+0000","last_became_active":"2023-07-18T03:04:58.844547+0000","last_became_peered":"2023-07-18T03:04:58.844547+0000","last_unstale":"2023-07-18T03:05:29.985596+0000","last_undegraded":"2023-07-18T03:05:29.985596+0000","last_fullsized":"2023-07-18T03:05:29.985596+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:01:48.220367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3794,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.937865+0000","last_change":"2023-07-18T03:04:59.803024+0000","last_active":"2023-07-18T03:05:23.937865+0000","last_peered":"2023-07-18T03:05:23.937865+0000","last_clean":"2023-07-18T03:05:23.937865+0000","last_became_active":"2023-07-18T03:04:59.802705+0000","last_became_peered":"2023-07-18T03:04:59.802705+0000","last_unstale":"2023-07-18T03:05:23.937865+0000","last_undegraded":"2023-07-18T03:05:23.937865+0000","last_fullsized":"2023-07-18T03:05:23.937865+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:51:13.019557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.090087+0000","last_change":"2023-07-18T03:05:03.876009+0000","last_active":"2023-07-18T03:05:24.090087+0000","last_peered":"2023-07-18T03:05:24.090087+0000","last_clean":"2023-07-18T03:05:24.090087+0000","last_became_active":"2023-07-18T03:05:03.875328+0000","last_became_peered":"2023-07-18T03:05:03.875328+0000","last_unstale":"2023-07-18T03:05:24.090087+0000","last_undegraded":"2023-07-18T03:05:24.090087+0000","last_fullsized":"2023-07-18T03:05:24.090087+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:29:03.036222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:30.992291+0000","last_change":"2023-07-18T03:04:58.846573+0000","last_active":"2023-07-18T03:05:30.992291+0000","last_peered":"2023-07-18T03:05:30.992291+0000","last_clean":"2023-07-18T03:05:30.992291+0000","last_became_active":"2023-07-18T03:04:58.846271+0000","last_became_peered":"2023-07-18T03:04:58.846271+0000","last_unstale":"2023-07-18T03:05:30.992291+0000","last_undegraded":"2023-07-18T03:05:30.992291+0000","last_fullsized":"2023-07-18T03:05:30.992291+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:07:28.151884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.091068+0000","last_change":"2023-07-18T03:04:59.801552+0000","last_active":"2023-07-18T03:05:24.091068+0000","last_peered":"2023-07-18T03:05:24.091068+0000","last_clean":"2023-07-18T03:05:24.091068+0000","last_became_active":"2023-07-18T03:04:59.801224+0000","last_became_peered":"2023-07-18T03:04:59.801224+0000","last_unstale":"2023-07-18T03:05:24.091068+0000","last_undegraded":"2023-07-18T03:05:24.091068+0000","last_fullsized":"2023-07-18T03:05:24.091068+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:43:54.484133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.941638+0000","last_change":"2023-07-18T03:05:03.881125+0000","last_active":"2023-07-18T03:05:23.941638+0000","last_peered":"2023-07-18T03:05:23.941638+0000","last_clean":"2023-07-18T03:05:23.941638+0000","last_became_active":"2023-07-18T03:05:03.880723+0000","last_became_peered":"2023-07-18T03:05:03.880723+0000","last_unstale":"2023-07-18T03:05:23.941638+0000","last_undegraded":"2023-07-18T03:05:23.941638+0000","last_fullsized":"2023-07-18T03:05:23.941638+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:10:01.286537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:29.984442+0000","last_change":"2023-07-18T03:04:58.847118+0000","last_active":"2023-07-18T03:05:29.984442+0000","last_peered":"2023-07-18T03:05:29.984442+0000","last_clean":"2023-07-18T03:05:29.984442+0000","last_became_active":"2023-07-18T03:04:58.846805+0000","last_became_peered":"2023-07-18T03:04:58.846805+0000","last_unstale":"2023-07-18T03:05:29.984442+0000","last_undegraded":"2023-07-18T03:05:29.984442+0000","last_fullsized":"2023-07-18T03:05:29.984442+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:43:05.061239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.933840+0000","last_change":"2023-07-18T03:04:59.803914+0000","last_active":"2023-07-18T03:05:23.933840+0000","last_peered":"2023-07-18T03:05:23.933840+0000","last_clean":"2023-07-18T03:05:23.933840+0000","last_became_active":"2023-07-18T03:04:59.803589+0000","last_became_peered":"2023-07-18T03:04:59.803589+0000","last_unstale":"2023-07-18T03:05:23.933840+0000","last_undegraded":"2023-07-18T03:05:23.933840+0000","last_fullsized":"2023-07-18T03:05:23.933840+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:56:07.440054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.934482+0000","last_change":"2023-07-18T03:05:03.877787+0000","last_active":"2023-07-18T03:05:23.934482+0000","last_peered":"2023-07-18T03:05:23.934482+0000","last_clean":"2023-07-18T03:05:23.934482+0000","last_became_active":"2023-07-18T03:05:03.877449+0000","last_became_peered":"2023-07-18T03:05:03.877449+0000","last_unstale":"2023-07-18T03:05:23.934482+0000","last_undegraded":"2023-07-18T03:05:23.934482+0000","last_fullsized":"2023-07-18T03:05:23.934482+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:25:39.108283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:31.996545+0000","last_change":"2023-07-18T03:04:58.844252+0000","last_active":"2023-07-18T03:05:31.996545+0000","last_peered":"2023-07-18T03:05:31.996545+0000","last_clean":"2023-07-18T03:05:31.996545+0000","last_became_active":"2023-07-18T03:04:58.843798+0000","last_became_peered":"2023-07-18T03:04:58.843798+0000","last_unstale":"2023-07-18T03:05:31.996545+0000","last_undegraded":"2023-07-18T03:05:31.996545+0000","last_fullsized":"2023-07-18T03:05:31.996545+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:54:33.788777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.933536+0000","last_change":"2023-07-18T03:04:59.806274+0000","last_active":"2023-07-18T03:05:23.933536+0000","last_peered":"2023-07-18T03:05:23.933536+0000","last_clean":"2023-07-18T03:05:23.933536+0000","last_became_active":"2023-07-18T03:04:59.805876+0000","last_became_peered":"2023-07-18T03:04:59.805876+0000","last_unstale":"2023-07-18T03:05:23.933536+0000","last_undegraded":"2023-07-18T03:05:23.933536+0000","last_fullsized":"2023-07-18T03:05:23.933536+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:38:39.397977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.091503+0000","last_change":"2023-07-18T03:05:03.876783+0000","last_active":"2023-07-18T03:05:24.091503+0000","last_peered":"2023-07-18T03:05:24.091503+0000","last_clean":"2023-07-18T03:05:24.091503+0000","last_became_active":"2023-07-18T03:05:03.876385+0000","last_became_peered":"2023-07-18T03:05:03.876385+0000","last_unstale":"2023-07-18T03:05:24.091503+0000","last_undegraded":"2023-07-18T03:05:24.091503+0000","last_fullsized":"2023-07-18T03:05:24.091503+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:20:54.077330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:30.992923+0000","last_change":"2023-07-18T03:04:58.851107+0000","last_active":"2023-07-18T03:05:30.992923+0000","last_peered":"2023-07-18T03:05:30.992923+0000","last_clean":"2023-07-18T03:05:30.992923+0000","last_became_active":"2023-07-18T03:04:58.850848+0000","last_became_peered":"2023-07-18T03:04:58.850848+0000","last_unstale":"2023-07-18T03:05:30.992923+0000","last_undegraded":"2023-07-18T03:05:30.992923+0000","last_fullsized":"2023-07-18T03:05:30.992923+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:17:37.985831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089918+0000","last_change":"2023-07-18T03:04:59.803683+0000","last_active":"2023-07-18T03:05:24.089918+0000","last_peered":"2023-07-18T03:05:24.089918+0000","last_clean":"2023-07-18T03:05:24.089918+0000","last_became_active":"2023-07-18T03:04:59.803365+0000","last_became_peered":"2023-07-18T03:04:59.803365+0000","last_unstale":"2023-07-18T03:05:24.089918+0000","last_undegraded":"2023-07-18T03:05:24.089918+0000","last_fullsized":"2023-07-18T03:05:24.089918+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:36:27.161561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089944+0000","last_change":"2023-07-18T03:05:03.875856+0000","last_active":"2023-07-18T03:05:24.089944+0000","last_peered":"2023-07-18T03:05:24.089944+0000","last_clean":"2023-07-18T03:05:24.089944+0000","last_became_active":"2023-07-18T03:05:03.875426+0000","last_became_peered":"2023-07-18T03:05:03.875426+0000","last_unstale":"2023-07-18T03:05:24.089944+0000","last_undegraded":"2023-07-18T03:05:24.089944+0000","last_fullsized":"2023-07-18T03:05:24.089944+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:00:07.610976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'1","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.938091+0000","last_change":"2023-07-18T03:04:58.844102+0000","last_active":"2023-07-18T03:05:23.938091+0000","last_peered":"2023-07-18T03:05:23.938091+0000","last_clean":"2023-07-18T03:05:23.938091+0000","last_became_active":"2023-07-18T03:04:58.843740+0000","last_became_peered":"2023-07-18T03:04:58.843740+0000","last_unstale":"2023-07-18T03:05:23.938091+0000","last_undegraded":"2023-07-18T03:05:23.938091+0000","last_fullsized":"2023-07-18T03:05:23.938091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:28:24.928761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.091161+0000","last_change":"2023-07-18T03:05:24.091161+0000","last_active":"2023-07-18T03:05:24.091161+0000","last_peered":"2023-07-18T03:05:24.091161+0000","last_clean":"2023-07-18T03:05:24.091161+0000","last_became_active":"2023-07-18T03:05:24.090631+0000","last_became_peered":"2023-07-18T03:05:24.090631+0000","last_unstale":"2023-07-18T03:05:24.091161+0000","last_undegraded":"2023-07-18T03:05:24.091161+0000","last_fullsized":"2023-07-18T03:05:24.091161+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:35:29.133177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.092328+0000","last_change":"2023-07-18T03:05:24.092328+0000","last_active":"2023-07-18T03:05:24.092328+0000","last_peered":"2023-07-18T03:05:24.092328+0000","last_clean":"2023-07-18T03:05:24.092328+0000","last_became_active":"2023-07-18T03:05:24.091839+0000","last_became_peered":"2023-07-18T03:05:24.091839+0000","last_unstale":"2023-07-18T03:05:24.092328+0000","last_undegraded":"2023-07-18T03:05:24.092328+0000","last_fullsized":"2023-07-18T03:05:24.092328+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:26:09.308228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.934182+0000","last_change":"2023-07-18T03:04:58.852395+0000","last_active":"2023-07-18T03:05:23.934182+0000","last_peered":"2023-07-18T03:05:23.934182+0000","last_clean":"2023-07-18T03:05:23.934182+0000","last_became_active":"2023-07-18T03:04:58.852063+0000","last_became_peered":"2023-07-18T03:04:58.852063+0000","last_unstale":"2023-07-18T03:05:23.934182+0000","last_undegraded":"2023-07-18T03:05:23.934182+0000","last_fullsized":"2023-07-18T03:05:23.934182+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:39:12.747039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.092486+0000","last_change":"2023-07-18T03:05:24.092486+0000","last_active":"2023-07-18T03:05:24.092486+0000","last_peered":"2023-07-18T03:05:24.092486+0000","last_clean":"2023-07-18T03:05:24.092486+0000","last_became_active":"2023-07-18T03:05:24.092189+0000","last_became_peered":"2023-07-18T03:05:24.092189+0000","last_unstale":"2023-07-18T03:05:24.092486+0000","last_undegraded":"2023-07-18T03:05:24.092486+0000","last_fullsized":"2023-07-18T03:05:24.092486+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:04:51.159197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.091135+0000","last_change":"2023-07-18T03:05:03.868851+0000","last_active":"2023-07-18T03:05:24.091135+0000","last_peered":"2023-07-18T03:05:24.091135+0000","last_clean":"2023-07-18T03:05:24.091135+0000","last_became_active":"2023-07-18T03:05:03.868433+0000","last_became_peered":"2023-07-18T03:05:03.868433+0000","last_unstale":"2023-07-18T03:05:24.091135+0000","last_undegraded":"2023-07-18T03:05:24.091135+0000","last_fullsized":"2023-07-18T03:05:24.091135+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:43:44.650042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.933677+0000","last_change":"2023-07-18T03:04:58.848059+0000","last_active":"2023-07-18T03:05:23.933677+0000","last_peered":"2023-07-18T03:05:23.933677+0000","last_clean":"2023-07-18T03:05:23.933677+0000","last_became_active":"2023-07-18T03:04:58.847700+0000","last_became_peered":"2023-07-18T03:04:58.847700+0000","last_unstale":"2023-07-18T03:05:23.933677+0000","last_undegraded":"2023-07-18T03:05:23.933677+0000","last_fullsized":"2023-07-18T03:05:23.933677+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:56:10.523441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.939939+0000","last_change":"2023-07-18T03:05:23.939939+0000","last_active":"2023-07-18T03:05:23.939939+0000","last_peered":"2023-07-18T03:05:23.939939+0000","last_clean":"2023-07-18T03:05:23.939939+0000","last_became_active":"2023-07-18T03:05:23.939607+0000","last_became_peered":"2023-07-18T03:05:23.939607+0000","last_unstale":"2023-07-18T03:05:23.939939+0000","last_undegraded":"2023-07-18T03:05:23.939939+0000","last_fullsized":"2023-07-18T03:05:23.939939+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:05:30.317897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.091495+0000","last_change":"2023-07-18T03:05:03.886724+0000","last_active":"2023-07-18T03:05:24.091495+0000","last_peered":"2023-07-18T03:05:24.091495+0000","last_clean":"2023-07-18T03:05:24.091495+0000","last_became_active":"2023-07-18T03:05:03.886172+0000","last_became_peered":"2023-07-18T03:05:03.886172+0000","last_unstale":"2023-07-18T03:05:24.091495+0000","last_undegraded":"2023-07-18T03:05:24.091495+0000","last_fullsized":"2023-07-18T03:05:24.091495+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:42:53.119172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089781+0000","last_change":"2023-07-18T03:04:59.804273+0000","last_active":"2023-07-18T03:05:24.089781+0000","last_peered":"2023-07-18T03:05:24.089781+0000","last_clean":"2023-07-18T03:05:24.089781+0000","last_became_active":"2023-07-18T03:04:59.803987+0000","last_became_peered":"2023-07-18T03:04:59.803987+0000","last_unstale":"2023-07-18T03:05:24.089781+0000","last_undegraded":"2023-07-18T03:05:24.089781+0000","last_fullsized":"2023-07-18T03:05:24.089781+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:47:40.524108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:30.992483+0000","last_change":"2023-07-18T03:04:58.846381+0000","last_active":"2023-07-18T03:05:30.992483+0000","last_peered":"2023-07-18T03:05:30.992483+0000","last_clean":"2023-07-18T03:05:30.992483+0000","last_became_active":"2023-07-18T03:04:58.845757+0000","last_became_peered":"2023-07-18T03:04:58.845757+0000","last_unstale":"2023-07-18T03:05:30.992483+0000","last_undegraded":"2023-07-18T03:05:30.992483+0000","last_fullsized":"2023-07-18T03:05:30.992483+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:41:53.865792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.933974+0000","last_change":"2023-07-18T03:05:03.873626+0000","last_active":"2023-07-18T03:05:23.933974+0000","last_peered":"2023-07-18T03:05:23.933974+0000","last_clean":"2023-07-18T03:05:23.933974+0000","last_became_active":"2023-07-18T03:05:03.872965+0000","last_became_peered":"2023-07-18T03:05:03.872965+0000","last_unstale":"2023-07-18T03:05:23.933974+0000","last_undegraded":"2023-07-18T03:05:23.933974+0000","last_fullsized":"2023-07-18T03:05:23.933974+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:11:15.362171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089960+0000","last_change":"2023-07-18T03:04:59.801153+0000","last_active":"2023-07-18T03:05:24.089960+0000","last_peered":"2023-07-18T03:05:24.089960+0000","last_clean":"2023-07-18T03:05:24.089960+0000","last_became_active":"2023-07-18T03:04:59.800798+0000","last_became_peered":"2023-07-18T03:04:59.800798+0000","last_unstale":"2023-07-18T03:05:24.089960+0000","last_undegraded":"2023-07-18T03:05:24.089960+0000","last_fullsized":"2023-07-18T03:05:24.089960+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:55:12.782843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:31.992249+0000","last_change":"2023-07-18T03:04:58.846047+0000","last_active":"2023-07-18T03:05:31.992249+0000","last_peered":"2023-07-18T03:05:31.992249+0000","last_clean":"2023-07-18T03:05:31.992249+0000","last_became_active":"2023-07-18T03:04:58.845678+0000","last_became_peered":"2023-07-18T03:04:58.845678+0000","last_unstale":"2023-07-18T03:05:31.992249+0000","last_undegraded":"2023-07-18T03:05:31.992249+0000","last_fullsized":"2023-07-18T03:05:31.992249+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:18:59.010410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":380,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.933385+0000","last_change":"2023-07-18T03:03:54.168837+0000","last_active":"2023-07-18T03:05:23.933385+0000","last_peered":"2023-07-18T03:05:23.933385+0000","last_clean":"2023-07-18T03:05:23.933385+0000","last_became_active":"2023-07-18T03:03:54.168298+0000","last_became_peered":"2023-07-18T03:03:54.168298+0000","last_unstale":"2023-07-18T03:05:23.933385+0000","last_undegraded":"2023-07-18T03:05:23.933385+0000","last_fullsized":"2023-07-18T03:05:23.933385+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:00:43.469168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:00:43.469168+0000","last_clean_scrub_stamp":"2023-07-18T03:00:43.469168+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:48:15.910084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089157+0000","last_change":"2023-07-18T03:05:03.886611+0000","last_active":"2023-07-18T03:05:24.089157+0000","last_peered":"2023-07-18T03:05:24.089157+0000","last_clean":"2023-07-18T03:05:24.089157+0000","last_became_active":"2023-07-18T03:05:03.886099+0000","last_became_peered":"2023-07-18T03:05:03.886099+0000","last_unstale":"2023-07-18T03:05:24.089157+0000","last_undegraded":"2023-07-18T03:05:24.089157+0000","last_fullsized":"2023-07-18T03:05:24.089157+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:22:16.652890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.090718+0000","last_change":"2023-07-18T03:04:59.804478+0000","last_active":"2023-07-18T03:05:24.090718+0000","last_peered":"2023-07-18T03:05:24.090718+0000","last_clean":"2023-07-18T03:05:24.090718+0000","last_became_active":"2023-07-18T03:04:59.804162+0000","last_became_peered":"2023-07-18T03:04:59.804162+0000","last_unstale":"2023-07-18T03:05:24.090718+0000","last_undegraded":"2023-07-18T03:05:24.090718+0000","last_fullsized":"2023-07-18T03:05:24.090718+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:30:38.735612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:30.990930+0000","last_change":"2023-07-18T03:04:58.847681+0000","last_active":"2023-07-18T03:05:30.990930+0000","last_peered":"2023-07-18T03:05:30.990930+0000","last_clean":"2023-07-18T03:05:30.990930+0000","last_became_active":"2023-07-18T03:04:58.847385+0000","last_became_peered":"2023-07-18T03:04:58.847385+0000","last_unstale":"2023-07-18T03:05:30.990930+0000","last_undegraded":"2023-07-18T03:05:30.990930+0000","last_fullsized":"2023-07-18T03:05:30.990930+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:16:57.492194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.941642+0000","last_change":"2023-07-18T03:05:03.872788+0000","last_active":"2023-07-18T03:05:23.941642+0000","last_peered":"2023-07-18T03:05:23.941642+0000","last_clean":"2023-07-18T03:05:23.941642+0000","last_became_active":"2023-07-18T03:05:03.872343+0000","last_became_peered":"2023-07-18T03:05:03.872343+0000","last_unstale":"2023-07-18T03:05:23.941642+0000","last_undegraded":"2023-07-18T03:05:23.941642+0000","last_fullsized":"2023-07-18T03:05:23.941642+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:05:33.222286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:29.985672+0000","last_change":"2023-07-18T03:04:58.855261+0000","last_active":"2023-07-18T03:05:29.985672+0000","last_peered":"2023-07-18T03:05:29.985672+0000","last_clean":"2023-07-18T03:05:29.985672+0000","last_became_active":"2023-07-18T03:04:58.854921+0000","last_became_peered":"2023-07-18T03:04:58.854921+0000","last_unstale":"2023-07-18T03:05:29.985672+0000","last_undegraded":"2023-07-18T03:05:29.985672+0000","last_fullsized":"2023-07-18T03:05:29.985672+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:08:26.349218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.091711+0000","last_change":"2023-07-18T03:04:59.804706+0000","last_active":"2023-07-18T03:05:24.091711+0000","last_peered":"2023-07-18T03:05:24.091711+0000","last_clean":"2023-07-18T03:05:24.091711+0000","last_became_active":"2023-07-18T03:04:59.804445+0000","last_became_peered":"2023-07-18T03:04:59.804445+0000","last_unstale":"2023-07-18T03:05:24.091711+0000","last_undegraded":"2023-07-18T03:05:24.091711+0000","last_fullsized":"2023-07-18T03:05:24.091711+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:06:24.461712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089557+0000","last_change":"2023-07-18T03:05:03.886065+0000","last_active":"2023-07-18T03:05:24.089557+0000","last_peered":"2023-07-18T03:05:24.089557+0000","last_clean":"2023-07-18T03:05:24.089557+0000","last_became_active":"2023-07-18T03:05:03.885625+0000","last_became_peered":"2023-07-18T03:05:03.885625+0000","last_unstale":"2023-07-18T03:05:24.089557+0000","last_undegraded":"2023-07-18T03:05:24.089557+0000","last_fullsized":"2023-07-18T03:05:24.089557+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:35:20.557825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'1","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.934410+0000","last_change":"2023-07-18T03:04:58.849246+0000","last_active":"2023-07-18T03:05:23.934410+0000","last_peered":"2023-07-18T03:05:23.934410+0000","last_clean":"2023-07-18T03:05:23.934410+0000","last_became_active":"2023-07-18T03:04:58.848888+0000","last_became_peered":"2023-07-18T03:04:58.848888+0000","last_unstale":"2023-07-18T03:05:23.934410+0000","last_undegraded":"2023-07-18T03:05:23.934410+0000","last_fullsized":"2023-07-18T03:05:23.934410+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:54:11.818147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.090505+0000","last_change":"2023-07-18T03:04:59.804561+0000","last_active":"2023-07-18T03:05:24.090505+0000","last_peered":"2023-07-18T03:05:24.090505+0000","last_clean":"2023-07-18T03:05:24.090505+0000","last_became_active":"2023-07-18T03:04:59.803659+0000","last_became_peered":"2023-07-18T03:04:59.803659+0000","last_unstale":"2023-07-18T03:05:24.090505+0000","last_undegraded":"2023-07-18T03:05:24.090505+0000","last_fullsized":"2023-07-18T03:05:24.090505+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:59:59.312017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.937606+0000","last_change":"2023-07-18T03:05:03.871418+0000","last_active":"2023-07-18T03:05:23.937606+0000","last_peered":"2023-07-18T03:05:23.937606+0000","last_clean":"2023-07-18T03:05:23.937606+0000","last_became_active":"2023-07-18T03:05:03.871054+0000","last_became_peered":"2023-07-18T03:05:03.871054+0000","last_unstale":"2023-07-18T03:05:23.937606+0000","last_undegraded":"2023-07-18T03:05:23.937606+0000","last_fullsized":"2023-07-18T03:05:23.937606+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:13:12.030292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:29.986067+0000","last_change":"2023-07-18T03:04:58.850505+0000","last_active":"2023-07-18T03:05:29.986067+0000","last_peered":"2023-07-18T03:05:29.986067+0000","last_clean":"2023-07-18T03:05:29.986067+0000","last_became_active":"2023-07-18T03:04:58.850234+0000","last_became_peered":"2023-07-18T03:04:58.850234+0000","last_unstale":"2023-07-18T03:05:29.986067+0000","last_undegraded":"2023-07-18T03:05:29.986067+0000","last_fullsized":"2023-07-18T03:05:29.986067+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:55:02.804220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.938190+0000","last_change":"2023-07-18T03:04:59.807348+0000","last_active":"2023-07-18T03:05:23.938190+0000","last_peered":"2023-07-18T03:05:23.938190+0000","last_clean":"2023-07-18T03:05:23.938190+0000","last_became_active":"2023-07-18T03:04:59.807013+0000","last_became_peered":"2023-07-18T03:04:59.807013+0000","last_unstale":"2023-07-18T03:05:23.938190+0000","last_undegraded":"2023-07-18T03:05:23.938190+0000","last_fullsized":"2023-07-18T03:05:23.938190+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:39:56.776375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089321+0000","last_change":"2023-07-18T03:05:03.886773+0000","last_active":"2023-07-18T03:05:24.089321+0000","last_peered":"2023-07-18T03:05:24.089321+0000","last_clean":"2023-07-18T03:05:24.089321+0000","last_became_active":"2023-07-18T03:05:03.886259+0000","last_became_peered":"2023-07-18T03:05:03.886259+0000","last_unstale":"2023-07-18T03:05:24.089321+0000","last_undegraded":"2023-07-18T03:05:24.089321+0000","last_fullsized":"2023-07-18T03:05:24.089321+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:04:49.928789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:29.985267+0000","last_change":"2023-07-18T03:04:58.855746+0000","last_active":"2023-07-18T03:05:29.985267+0000","last_peered":"2023-07-18T03:05:29.985267+0000","last_clean":"2023-07-18T03:05:29.985267+0000","last_became_active":"2023-07-18T03:04:58.855366+0000","last_became_peered":"2023-07-18T03:04:58.855366+0000","last_unstale":"2023-07-18T03:05:29.985267+0000","last_undegraded":"2023-07-18T03:05:29.985267+0000","last_fullsized":"2023-07-18T03:05:29.985267+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:52:56.120702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.936549+0000","last_change":"2023-07-18T03:04:59.800321+0000","last_active":"2023-07-18T03:05:23.936549+0000","last_peered":"2023-07-18T03:05:23.936549+0000","last_clean":"2023-07-18T03:05:23.936549+0000","last_became_active":"2023-07-18T03:04:59.800018+0000","last_became_peered":"2023-07-18T03:04:59.800018+0000","last_unstale":"2023-07-18T03:05:23.936549+0000","last_undegraded":"2023-07-18T03:05:23.936549+0000","last_fullsized":"2023-07-18T03:05:23.936549+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:12:47.433577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.934504+0000","last_change":"2023-07-18T03:05:03.877541+0000","last_active":"2023-07-18T03:05:23.934504+0000","last_peered":"2023-07-18T03:05:23.934504+0000","last_clean":"2023-07-18T03:05:23.934504+0000","last_became_active":"2023-07-18T03:05:03.877164+0000","last_became_peered":"2023-07-18T03:05:03.877164+0000","last_unstale":"2023-07-18T03:05:23.934504+0000","last_undegraded":"2023-07-18T03:05:23.934504+0000","last_fullsized":"2023-07-18T03:05:23.934504+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:52:54.205495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:29.985557+0000","last_change":"2023-07-18T03:04:58.845643+0000","last_active":"2023-07-18T03:05:29.985557+0000","last_peered":"2023-07-18T03:05:29.985557+0000","last_clean":"2023-07-18T03:05:29.985557+0000","last_became_active":"2023-07-18T03:04:58.845254+0000","last_became_peered":"2023-07-18T03:04:58.845254+0000","last_unstale":"2023-07-18T03:05:29.985557+0000","last_undegraded":"2023-07-18T03:05:29.985557+0000","last_fullsized":"2023-07-18T03:05:29.985557+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:42:32.740369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.941481+0000","last_change":"2023-07-18T03:04:59.803679+0000","last_active":"2023-07-18T03:05:23.941481+0000","last_peered":"2023-07-18T03:05:23.941481+0000","last_clean":"2023-07-18T03:05:23.941481+0000","last_became_active":"2023-07-18T03:04:59.802841+0000","last_became_peered":"2023-07-18T03:04:59.802841+0000","last_unstale":"2023-07-18T03:05:23.941481+0000","last_undegraded":"2023-07-18T03:05:23.941481+0000","last_fullsized":"2023-07-18T03:05:23.941481+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:14:53.042557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089419+0000","last_change":"2023-07-18T03:05:03.874093+0000","last_active":"2023-07-18T03:05:24.089419+0000","last_peered":"2023-07-18T03:05:24.089419+0000","last_clean":"2023-07-18T03:05:24.089419+0000","last_became_active":"2023-07-18T03:05:03.873710+0000","last_became_peered":"2023-07-18T03:05:03.873710+0000","last_unstale":"2023-07-18T03:05:24.089419+0000","last_undegraded":"2023-07-18T03:05:24.089419+0000","last_fullsized":"2023-07-18T03:05:24.089419+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:19:17.794343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.937140+0000","last_change":"2023-07-18T03:04:58.846686+0000","last_active":"2023-07-18T03:05:23.937140+0000","last_peered":"2023-07-18T03:05:23.937140+0000","last_clean":"2023-07-18T03:05:23.937140+0000","last_became_active":"2023-07-18T03:04:58.846371+0000","last_became_peered":"2023-07-18T03:04:58.846371+0000","last_unstale":"2023-07-18T03:05:23.937140+0000","last_undegraded":"2023-07-18T03:05:23.937140+0000","last_fullsized":"2023-07-18T03:05:23.937140+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:43:54.054638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089999+0000","last_change":"2023-07-18T03:04:59.811917+0000","last_active":"2023-07-18T03:05:24.089999+0000","last_peered":"2023-07-18T03:05:24.089999+0000","last_clean":"2023-07-18T03:05:24.089999+0000","last_became_active":"2023-07-18T03:04:59.811551+0000","last_became_peered":"2023-07-18T03:04:59.811551+0000","last_unstale":"2023-07-18T03:05:24.089999+0000","last_undegraded":"2023-07-18T03:05:24.089999+0000","last_fullsized":"2023-07-18T03:05:24.089999+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:32:42.153516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089397+0000","last_change":"2023-07-18T03:05:03.886072+0000","last_active":"2023-07-18T03:05:24.089397+0000","last_peered":"2023-07-18T03:05:24.089397+0000","last_clean":"2023-07-18T03:05:24.089397+0000","last_became_active":"2023-07-18T03:05:03.885652+0000","last_became_peered":"2023-07-18T03:05:03.885652+0000","last_unstale":"2023-07-18T03:05:24.089397+0000","last_undegraded":"2023-07-18T03:05:24.089397+0000","last_fullsized":"2023-07-18T03:05:24.089397+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:26:47.552178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:29.985186+0000","last_change":"2023-07-18T03:04:58.849069+0000","last_active":"2023-07-18T03:05:29.985186+0000","last_peered":"2023-07-18T03:05:29.985186+0000","last_clean":"2023-07-18T03:05:29.985186+0000","last_became_active":"2023-07-18T03:04:58.848744+0000","last_became_peered":"2023-07-18T03:04:58.848744+0000","last_unstale":"2023-07-18T03:05:29.985186+0000","last_undegraded":"2023-07-18T03:05:29.985186+0000","last_fullsized":"2023-07-18T03:05:29.985186+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:06:14.618475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.933710+0000","last_change":"2023-07-18T03:04:59.806976+0000","last_active":"2023-07-18T03:05:23.933710+0000","last_peered":"2023-07-18T03:05:23.933710+0000","last_clean":"2023-07-18T03:05:23.933710+0000","last_became_active":"2023-07-18T03:04:59.805750+0000","last_became_peered":"2023-07-18T03:04:59.805750+0000","last_unstale":"2023-07-18T03:05:23.933710+0000","last_undegraded":"2023-07-18T03:05:23.933710+0000","last_fullsized":"2023-07-18T03:05:23.933710+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:17:24.414508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.090382+0000","last_change":"2023-07-18T03:05:03.871210+0000","last_active":"2023-07-18T03:05:24.090382+0000","last_peered":"2023-07-18T03:05:24.090382+0000","last_clean":"2023-07-18T03:05:24.090382+0000","last_became_active":"2023-07-18T03:05:03.870826+0000","last_became_peered":"2023-07-18T03:05:03.870826+0000","last_unstale":"2023-07-18T03:05:24.090382+0000","last_undegraded":"2023-07-18T03:05:24.090382+0000","last_fullsized":"2023-07-18T03:05:24.090382+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:54:35.402542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:30.990936+0000","last_change":"2023-07-18T03:04:58.855673+0000","last_active":"2023-07-18T03:05:30.990936+0000","last_peered":"2023-07-18T03:05:30.990936+0000","last_clean":"2023-07-18T03:05:30.990936+0000","last_became_active":"2023-07-18T03:04:58.855338+0000","last_became_peered":"2023-07-18T03:04:58.855338+0000","last_unstale":"2023-07-18T03:05:30.990936+0000","last_undegraded":"2023-07-18T03:05:30.990936+0000","last_fullsized":"2023-07-18T03:05:30.990936+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:22:42.768582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.934248+0000","last_change":"2023-07-18T03:04:59.802488+0000","last_active":"2023-07-18T03:05:23.934248+0000","last_peered":"2023-07-18T03:05:23.934248+0000","last_clean":"2023-07-18T03:05:23.934248+0000","last_became_active":"2023-07-18T03:04:59.802184+0000","last_became_peered":"2023-07-18T03:04:59.802184+0000","last_unstale":"2023-07-18T03:05:23.934248+0000","last_undegraded":"2023-07-18T03:05:23.934248+0000","last_fullsized":"2023-07-18T03:05:23.934248+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:48:27.222393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.937967+0000","last_change":"2023-07-18T03:05:03.881585+0000","last_active":"2023-07-18T03:05:23.937967+0000","last_peered":"2023-07-18T03:05:23.937967+0000","last_clean":"2023-07-18T03:05:23.937967+0000","last_became_active":"2023-07-18T03:05:03.880917+0000","last_became_peered":"2023-07-18T03:05:03.880917+0000","last_unstale":"2023-07-18T03:05:23.937967+0000","last_undegraded":"2023-07-18T03:05:23.937967+0000","last_fullsized":"2023-07-18T03:05:23.937967+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:21:55.495075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089704+0000","last_change":"2023-07-18T03:04:58.852387+0000","last_active":"2023-07-18T03:05:24.089704+0000","last_peered":"2023-07-18T03:05:24.089704+0000","last_clean":"2023-07-18T03:05:24.089704+0000","last_became_active":"2023-07-18T03:04:58.852113+0000","last_became_peered":"2023-07-18T03:04:58.852113+0000","last_unstale":"2023-07-18T03:05:24.089704+0000","last_undegraded":"2023-07-18T03:05:24.089704+0000","last_fullsized":"2023-07-18T03:05:24.089704+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:47:27.148793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.933886+0000","last_change":"2023-07-18T03:04:59.811569+0000","last_active":"2023-07-18T03:05:23.933886+0000","last_peered":"2023-07-18T03:05:23.933886+0000","last_clean":"2023-07-18T03:05:23.933886+0000","last_became_active":"2023-07-18T03:04:59.811229+0000","last_became_peered":"2023-07-18T03:04:59.811229+0000","last_unstale":"2023-07-18T03:05:23.933886+0000","last_undegraded":"2023-07-18T03:05:23.933886+0000","last_fullsized":"2023-07-18T03:05:23.933886+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:53:58.785873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.088825+0000","last_change":"2023-07-18T03:05:03.872530+0000","last_active":"2023-07-18T03:05:24.088825+0000","last_peered":"2023-07-18T03:05:24.088825+0000","last_clean":"2023-07-18T03:05:24.088825+0000","last_became_active":"2023-07-18T03:05:03.871607+0000","last_became_peered":"2023-07-18T03:05:03.871607+0000","last_unstale":"2023-07-18T03:05:24.088825+0000","last_undegraded":"2023-07-18T03:05:24.088825+0000","last_fullsized":"2023-07-18T03:05:24.088825+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:34:34.583319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.091627+0000","last_change":"2023-07-18T03:04:58.847933+0000","last_active":"2023-07-18T03:05:24.091627+0000","last_peered":"2023-07-18T03:05:24.091627+0000","last_clean":"2023-07-18T03:05:24.091627+0000","last_became_active":"2023-07-18T03:04:58.847630+0000","last_became_peered":"2023-07-18T03:04:58.847630+0000","last_unstale":"2023-07-18T03:05:24.091627+0000","last_undegraded":"2023-07-18T03:05:24.091627+0000","last_fullsized":"2023-07-18T03:05:24.091627+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:35:14.343077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.934420+0000","last_change":"2023-07-18T03:04:59.803981+0000","last_active":"2023-07-18T03:05:23.934420+0000","last_peered":"2023-07-18T03:05:23.934420+0000","last_clean":"2023-07-18T03:05:23.934420+0000","last_became_active":"2023-07-18T03:04:59.803692+0000","last_became_peered":"2023-07-18T03:04:59.803692+0000","last_unstale":"2023-07-18T03:05:23.934420+0000","last_undegraded":"2023-07-18T03:05:23.934420+0000","last_fullsized":"2023-07-18T03:05:23.934420+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:52:39.970316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.941648+0000","last_change":"2023-07-18T03:04:59.803770+0000","last_active":"2023-07-18T03:05:23.941648+0000","last_peered":"2023-07-18T03:05:23.941648+0000","last_clean":"2023-07-18T03:05:23.941648+0000","last_became_active":"2023-07-18T03:04:59.803355+0000","last_became_peered":"2023-07-18T03:04:59.803355+0000","last_unstale":"2023-07-18T03:05:23.941648+0000","last_undegraded":"2023-07-18T03:05:23.941648+0000","last_fullsized":"2023-07-18T03:05:23.941648+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:58:39.773442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.937874+0000","last_change":"2023-07-18T03:04:59.808946+0000","last_active":"2023-07-18T03:05:23.937874+0000","last_peered":"2023-07-18T03:05:23.937874+0000","last_clean":"2023-07-18T03:05:23.937874+0000","last_became_active":"2023-07-18T03:04:59.808646+0000","last_became_peered":"2023-07-18T03:04:59.808646+0000","last_unstale":"2023-07-18T03:05:23.937874+0000","last_undegraded":"2023-07-18T03:05:23.937874+0000","last_fullsized":"2023-07-18T03:05:23.937874+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:30:10.702978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.936997+0000","last_change":"2023-07-18T03:04:59.811933+0000","last_active":"2023-07-18T03:05:23.936997+0000","last_peered":"2023-07-18T03:05:23.936997+0000","last_clean":"2023-07-18T03:05:23.936997+0000","last_became_active":"2023-07-18T03:04:59.811620+0000","last_became_peered":"2023-07-18T03:04:59.811620+0000","last_unstale":"2023-07-18T03:05:23.936997+0000","last_undegraded":"2023-07-18T03:05:23.936997+0000","last_fullsized":"2023-07-18T03:05:23.936997+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:14:30.959426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.938599+0000","last_change":"2023-07-18T03:04:59.803942+0000","last_active":"2023-07-18T03:05:23.938599+0000","last_peered":"2023-07-18T03:05:23.938599+0000","last_clean":"2023-07-18T03:05:23.938599+0000","last_became_active":"2023-07-18T03:04:59.803576+0000","last_became_peered":"2023-07-18T03:04:59.803576+0000","last_unstale":"2023-07-18T03:05:23.938599+0000","last_undegraded":"2023-07-18T03:05:23.938599+0000","last_fullsized":"2023-07-18T03:05:23.938599+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:24:31.974566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.091559+0000","last_change":"2023-07-18T03:04:59.802594+0000","last_active":"2023-07-18T03:05:24.091559+0000","last_peered":"2023-07-18T03:05:24.091559+0000","last_clean":"2023-07-18T03:05:24.091559+0000","last_became_active":"2023-07-18T03:04:59.802215+0000","last_became_peered":"2023-07-18T03:04:59.802215+0000","last_unstale":"2023-07-18T03:05:24.091559+0000","last_undegraded":"2023-07-18T03:05:24.091559+0000","last_fullsized":"2023-07-18T03:05:24.091559+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:18:26.515762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.942105+0000","last_change":"2023-07-18T03:04:59.803519+0000","last_active":"2023-07-18T03:05:23.942105+0000","last_peered":"2023-07-18T03:05:23.942105+0000","last_clean":"2023-07-18T03:05:23.942105+0000","last_became_active":"2023-07-18T03:04:59.803170+0000","last_became_peered":"2023-07-18T03:04:59.803170+0000","last_unstale":"2023-07-18T03:05:23.942105+0000","last_undegraded":"2023-07-18T03:05:23.942105+0000","last_fullsized":"2023-07-18T03:05:23.942105+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:57:22.069668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.938577+0000","last_change":"2023-07-18T03:04:59.813494+0000","last_active":"2023-07-18T03:05:23.938577+0000","last_peered":"2023-07-18T03:05:23.938577+0000","last_clean":"2023-07-18T03:05:23.938577+0000","last_became_active":"2023-07-18T03:04:59.813175+0000","last_became_peered":"2023-07-18T03:04:59.813175+0000","last_unstale":"2023-07-18T03:05:23.938577+0000","last_undegraded":"2023-07-18T03:05:23.938577+0000","last_fullsized":"2023-07-18T03:05:23.938577+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:13:17.098239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.090682+0000","last_change":"2023-07-18T03:04:59.806597+0000","last_active":"2023-07-18T03:05:24.090682+0000","last_peered":"2023-07-18T03:05:24.090682+0000","last_clean":"2023-07-18T03:05:24.090682+0000","last_became_active":"2023-07-18T03:04:59.806220+0000","last_became_peered":"2023-07-18T03:04:59.806220+0000","last_unstale":"2023-07-18T03:05:24.090682+0000","last_undegraded":"2023-07-18T03:05:24.090682+0000","last_fullsized":"2023-07-18T03:05:24.090682+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:33:51.675056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089408+0000","last_change":"2023-07-18T03:04:59.809051+0000","last_active":"2023-07-18T03:05:24.089408+0000","last_peered":"2023-07-18T03:05:24.089408+0000","last_clean":"2023-07-18T03:05:24.089408+0000","last_became_active":"2023-07-18T03:04:59.808783+0000","last_became_peered":"2023-07-18T03:04:59.808783+0000","last_unstale":"2023-07-18T03:05:24.089408+0000","last_undegraded":"2023-07-18T03:05:24.089408+0000","last_fullsized":"2023-07-18T03:05:24.089408+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:11:53.631126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.090344+0000","last_change":"2023-07-18T03:04:59.805741+0000","last_active":"2023-07-18T03:05:24.090344+0000","last_peered":"2023-07-18T03:05:24.090344+0000","last_clean":"2023-07-18T03:05:24.090344+0000","last_became_active":"2023-07-18T03:04:59.805412+0000","last_became_peered":"2023-07-18T03:04:59.805412+0000","last_unstale":"2023-07-18T03:05:24.090344+0000","last_undegraded":"2023-07-18T03:05:24.090344+0000","last_fullsized":"2023-07-18T03:05:24.090344+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:36:12.237809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.090649+0000","last_change":"2023-07-18T03:04:59.812052+0000","last_active":"2023-07-18T03:05:24.090649+0000","last_peered":"2023-07-18T03:05:24.090649+0000","last_clean":"2023-07-18T03:05:24.090649+0000","last_became_active":"2023-07-18T03:04:59.811756+0000","last_became_peered":"2023-07-18T03:04:59.811756+0000","last_unstale":"2023-07-18T03:05:24.090649+0000","last_undegraded":"2023-07-18T03:05:24.090649+0000","last_fullsized":"2023-07-18T03:05:24.090649+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:28:03.874575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.091423+0000","last_change":"2023-07-18T03:04:59.803009+0000","last_active":"2023-07-18T03:05:24.091423+0000","last_peered":"2023-07-18T03:05:24.091423+0000","last_clean":"2023-07-18T03:05:24.091423+0000","last_became_active":"2023-07-18T03:04:59.802687+0000","last_became_peered":"2023-07-18T03:04:59.802687+0000","last_unstale":"2023-07-18T03:05:24.091423+0000","last_undegraded":"2023-07-18T03:05:24.091423+0000","last_fullsized":"2023-07-18T03:05:24.091423+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:28:34.815336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.942144+0000","last_change":"2023-07-18T03:04:59.795562+0000","last_active":"2023-07-18T03:05:23.942144+0000","last_peered":"2023-07-18T03:05:23.942144+0000","last_clean":"2023-07-18T03:05:23.942144+0000","last_became_active":"2023-07-18T03:04:59.795241+0000","last_became_peered":"2023-07-18T03:04:59.795241+0000","last_unstale":"2023-07-18T03:05:23.942144+0000","last_undegraded":"2023-07-18T03:05:23.942144+0000","last_fullsized":"2023-07-18T03:05:23.942144+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:43:39.736724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":12288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281146,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7308,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":15,"num_read_kb":13,"num_write":118,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":274432,"data_stored":57301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":103,"ondisk_log_size":103,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449106,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":483597,"data_compressed":17573,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:05:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.70299999999999996}]},{"osd":1,"last update":"Tue Jul 18 03:05:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.58699999999999997}]},{"osd":2,"last update":"Tue Jul 18 03:05:13 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.73499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":4,"last update":"Tue Jul 18 03:05:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.75900000000000001}]},{"osd":5,"last update":"Tue Jul 18 03:05:13 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.86199999999999999}]},{"osd":6,"last update":"Tue Jul 18 03:05:13 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.79800000000000004}]},{"osd":7,"last update":"Tue Jul 18 03:05:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.89100000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]},{"osd":9,"last update":"Tue Jul 18 03:05:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.877},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.90300000000000002}]},{"osd":10,"last update":"Tue Jul 18 03:05:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.94999999999999996}]}]},{"osd":10,"up_from":71,"seq":304942678039,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":1032,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":1056768,"data_stored":1078707,"data_compressed":53332,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":1,"last update":"Tue Jul 18 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.42899999999999999}]},{"osd":2,"last update":"Tue Jul 18 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.59499999999999997}]},{"osd":3,"last update":"Tue Jul 18 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.52300000000000002}]},{"osd":4,"last update":"Tue Jul 18 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.47099999999999997}]},{"osd":5,"last update":"Tue Jul 18 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.78300000000000003}]},{"osd":6,"last update":"Tue Jul 18 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.65600000000000003}]},{"osd":7,"last update":"Tue Jul 18 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.55800000000000005}]},{"osd":8,"last update":"Tue Jul 18 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.72299999999999998}]},{"osd":9,"last update":"Tue Jul 18 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.755}]},{"osd":11,"last update":"Tue Jul 18 03:05:19 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.73099999999999998}]}]},{"osd":9,"up_from":66,"seq":283467841562,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":483597,"data_compressed":17573,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:04:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.37},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.45000000000000001}]},{"osd":1,"last update":"Tue Jul 18 03:04:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.55100000000000005}]},{"osd":2,"last update":"Tue Jul 18 03:04:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.48499999999999999}]},{"osd":3,"last update":"Tue Jul 18 03:04:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.38700000000000001}]},{"osd":4,"last update":"Tue Jul 18 03:04:30 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.66100000000000003}]},{"osd":5,"last update":"Tue Jul 18 03:04:30 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.499}]},{"osd":6,"last update":"Tue Jul 18 03:04:30 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.64300000000000002}]},{"osd":7,"last update":"Tue Jul 18 03:04:30 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":0.70899999999999996}]},{"osd":8,"last update":"Tue Jul 18 03:04:30 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.69099999999999995}]},{"osd":10,"last update":"Tue Jul 18 03:04:51 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.8899999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.72999999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]}]},{"osd":8,"up_from":58,"seq":249108103199,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":983040,"data_stored":483753,"data_compressed":17573,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:05:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.626,"15min":0.61799999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":2.0990000000000002,"5min":2.0990000000000002,"15min":2.0990000000000002},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.58099999999999996,"15min":0.56999999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":2.883,"5min":2.883,"15min":2.883},"last":0.56799999999999995}]},{"osd":1,"last update":"Tue Jul 18 03:05:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.63900000000000001,"15min":0.627},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":3.3340000000000001,"5min":3.3340000000000001,"15min":3.3340000000000001},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.61099999999999999,"15min":0.59599999999999997},"min":{"1min":0.37,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":2.0510000000000002,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":0.81000000000000005}]},{"osd":2,"last update":"Tue Jul 18 03:05:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.48199999999999998,"15min":0.46000000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":3.2850000000000001,"5min":3.2850000000000001,"15min":3.2850000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.502,"15min":0.49199999999999999},"min":{"1min":0.309,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":0.54200000000000004}]},{"osd":3,"last update":"Tue Jul 18 03:05:10 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.628,"15min":0.61699999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":2.9100000000000001,"5min":2.9100000000000001,"15min":2.9100000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.66700000000000004,"15min":0.64900000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":3.6059999999999999,"5min":3.6059999999999999,"15min":3.6059999999999999},"last":0.65500000000000003}]},{"osd":4,"last update":"Tue Jul 18 03:05:10 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.64500000000000002,"15min":0.627},"min":{"1min":0.437,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.7879999999999998,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.70299999999999996,"15min":0.68999999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":3.3650000000000002,"5min":3.3650000000000002,"15min":3.3650000000000002},"last":0.71699999999999997}]},{"osd":5,"last update":"Tue Jul 18 03:05:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.60699999999999998,"15min":0.58999999999999997},"min":{"1min":0.39200000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":2.4279999999999999,"5min":2.4279999999999999,"15min":2.4279999999999999},"last":0.747},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.61199999999999999,"15min":0.59799999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":2.7410000000000001,"5min":2.7410000000000001,"15min":2.7410000000000001},"last":0.82899999999999996}]},{"osd":6,"last update":"Tue Jul 18 03:05:10 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.73899999999999999,"15min":0.71799999999999997},"min":{"1min":0.54200000000000004,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":3.5779999999999998,"5min":3.5779999999999998,"15min":3.5779999999999998},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.746,"15min":0.73499999999999999},"min":{"1min":0.502,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.1429999999999998,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":0.84699999999999998}]},{"osd":7,"last update":"Tue Jul 18 03:05:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.73199999999999998,"15min":0.71999999999999997},"min":{"1min":0.48999999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.73499999999999999,"15min":0.71899999999999997},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":2.8290000000000002,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":0.96299999999999997}]},{"osd":9,"last update":"Tue Jul 18 03:04:33 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.97599999999999998}]},{"osd":10,"last update":"Tue Jul 18 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":2.8570000000000002,"5min":2.8570000000000002,"15min":2.8570000000000002},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":3.3879999999999999,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":1.02}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]}]},{"osd":7,"up_from":51,"seq":219043332131,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":1032192,"data_stored":1079565,"data_compressed":53332,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.55500000000000005,"15min":0.54300000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.495},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59999999999999998,"15min":0.60299999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82599999999999996,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.44500000000000001}]},{"osd":1,"last update":"Tue Jul 18 03:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.42099999999999999,"15min":0.41899999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.436,"15min":0.432},"min":{"1min":0.31900000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.63500000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.374}]},{"osd":2,"last update":"Tue Jul 18 03:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.57799999999999996,"15min":0.56799999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.59199999999999997,"15min":0.58199999999999996},"min":{"1min":0.45500000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.63200000000000001}]},{"osd":3,"last update":"Tue Jul 18 03:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.60399999999999998,"15min":0.59599999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.435,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.91500000000000004,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.57899999999999996}]},{"osd":4,"last update":"Tue Jul 18 03:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.48399999999999999,"15min":0.48199999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.71599999999999997,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.55100000000000005,"15min":0.54900000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.45800000000000002}]},{"osd":5,"last update":"Tue Jul 18 03:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.66000000000000003,"15min":0.64700000000000002},"min":{"1min":0.53300000000000003,"5min":0.501,"15min":0.501},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.66500000000000004,"15min":0.65500000000000003},"min":{"1min":0.52000000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.60199999999999998}]},{"osd":6,"last update":"Tue Jul 18 03:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.67500000000000004,"15min":0.67400000000000004},"min":{"1min":0.46999999999999997,"5min":0.435,"15min":0.435},"max":{"1min":0.89600000000000002,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.752},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.70999999999999996,"15min":0.70099999999999996},"min":{"1min":0.497,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.55300000000000005}]},{"osd":8,"last update":"Tue Jul 18 03:05:06 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.69899999999999995,"15min":0.68899999999999995},"min":{"1min":0.48999999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.65400000000000003,"15min":0.64500000000000002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.71199999999999997}]},{"osd":9,"last update":"Tue Jul 18 03:05:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.76600000000000001,"15min":0.76400000000000001},"min":{"1min":0.56499999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.70099999999999996}]},{"osd":10,"last update":"Tue Jul 18 03:04:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.64700000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]}]},{"osd":6,"up_from":46,"seq":197568495656,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":962560,"data_stored":483015,"data_compressed":17573,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.53400000000000003,"15min":0.56299999999999994},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.58199999999999996,"5min":3.1579999999999999,"15min":3.1579999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.53400000000000003,"15min":0.55000000000000004},"min":{"1min":0.35699999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.64000000000000001,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.45600000000000002}]},{"osd":1,"last update":"Tue Jul 18 03:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.627,"15min":0.60299999999999998},"min":{"1min":0.38800000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.88600000000000001,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.72799999999999998,"15min":0.74099999999999999},"min":{"1min":0.495,"5min":0.44,"15min":0.44},"max":{"1min":0.90500000000000003,"5min":3.0369999999999999,"15min":3.0369999999999999},"last":0.878}]},{"osd":2,"last update":"Tue Jul 18 03:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.65700000000000003,"15min":0.65600000000000003},"min":{"1min":0.48299999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.64700000000000002,"15min":0.65300000000000002},"min":{"1min":0.48399999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.94299999999999995,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.70699999999999996}]},{"osd":3,"last update":"Tue Jul 18 03:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.56499999999999995,"15min":0.57299999999999995},"min":{"1min":0.38800000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.69899999999999995,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.71199999999999997,"15min":0.748},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.748,"5min":3.226,"15min":3.226},"last":0.68999999999999995}]},{"osd":4,"last update":"Tue Jul 18 03:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.44600000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.92500000000000004,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.70199999999999996,"15min":0.69899999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.92700000000000005,"5min":1.01,"15min":1.01},"last":0.84499999999999997}]},{"osd":5,"last update":"Tue Jul 18 03:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.81200000000000006,"15min":0.84499999999999997},"min":{"1min":0.59199999999999997,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.88600000000000001,"5min":3.125,"15min":3.125},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.76600000000000001,"15min":0.78200000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.98699999999999999,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.63200000000000001}]},{"osd":7,"last update":"Tue Jul 18 03:04:45 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.84099999999999997,"15min":0.84199999999999997},"min":{"1min":0.54500000000000004,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.2070000000000001,"5min":3.0870000000000002,"15min":3.0870000000000002},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.74099999999999999,"15min":0.72999999999999998},"min":{"1min":0.60099999999999998,"5min":0.505,"15min":0.505},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.90500000000000003}]},{"osd":8,"last update":"Tue Jul 18 03:05:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.871,"15min":0.88300000000000001},"min":{"1min":0.59599999999999997,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.103,"5min":3.1909999999999998,"15min":3.1909999999999998},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.54800000000000004,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.66700000000000004}]},{"osd":9,"last update":"Tue Jul 18 03:04:30 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.94299999999999995}]},{"osd":10,"last update":"Tue Jul 18 03:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.93200000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":974848,"data_stored":486809,"data_compressed":17573,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.55100000000000005,"15min":0.52400000000000002},"min":{"1min":0.37,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.96299999999999997,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.54400000000000004,"15min":0.52400000000000002},"min":{"1min":0.35299999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.93100000000000005,"5min":1.421,"15min":1.421},"last":0.442}]},{"osd":2,"last update":"Tue Jul 18 03:04:53 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.52400000000000002,"15min":0.505},"min":{"1min":0.38500000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.53900000000000003,"15min":0.53400000000000003},"min":{"1min":0.45700000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.56899999999999995}]},{"osd":3,"last update":"Tue Jul 18 03:05:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.61799999999999999,"15min":0.59799999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.393,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.62,"15min":0.625},"min":{"1min":0.432,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82899999999999996,"5min":1.504,"15min":1.504},"last":0.59399999999999997}]},{"osd":4,"last update":"Tue Jul 18 03:04:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.41799999999999998,"15min":0.39300000000000002},"min":{"1min":0.30099999999999999,"5min":0.215,"15min":0.215},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.441,"15min":0.41299999999999998},"min":{"1min":0.38,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.624}]},{"osd":5,"last update":"Tue Jul 18 03:05:00 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.56699999999999995,"15min":0.51600000000000001},"min":{"1min":0.40799999999999997,"5min":0.375,"15min":0.375},"max":{"1min":1.5129999999999999,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.58199999999999996,"15min":0.55400000000000005},"min":{"1min":0.46100000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.66300000000000003}]},{"osd":6,"last update":"Tue Jul 18 03:05:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.64900000000000002,"15min":0.60499999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.4299999999999999,"5min":2.6600000000000001,"15min":2.6600000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.65200000000000002,"15min":0.63800000000000001},"min":{"1min":0.46400000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.94799999999999995,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.70499999999999996}]},{"osd":7,"last update":"Tue Jul 18 03:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.57299999999999995,"15min":0.55600000000000005},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.87,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.65500000000000003,"15min":0.64900000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.89700000000000002,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":0.84099999999999997}]},{"osd":8,"last update":"Tue Jul 18 03:05:09 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.77800000000000002,"15min":0.78300000000000003},"min":{"1min":0.55800000000000005,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.486,"5min":2.6419999999999999,"15min":2.6419999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.69199999999999995,"15min":0.67200000000000004},"min":{"1min":0.52000000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.698,"5min":1.698,"15min":1.698},"last":0.78000000000000003}]},{"osd":9,"last update":"Tue Jul 18 03:04:34 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.6789999999999998,"5min":2.6789999999999998,"15min":2.6789999999999998},"last":0.871},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.754}]},{"osd":10,"last update":"Tue Jul 18 03:04:53 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.80600000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":991232,"data_stored":484493,"data_compressed":17573,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Jul 18 03:05:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.59599999999999997,"15min":0.55200000000000005},"min":{"1min":0.45900000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.0149999999999999,"5min":1.075,"15min":1.075},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.61099999999999999,"15min":0.56100000000000005},"min":{"1min":0.38100000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.97699999999999998,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.61299999999999999}]},{"osd":2,"last update":"Tue Jul 18 03:04:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.57899999999999996,"15min":0.54400000000000004},"min":{"1min":0.41499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.85799999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.61199999999999999,"15min":0.55800000000000005},"min":{"1min":0.39000000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.016,"5min":1.204,"15min":1.204},"last":0.56200000000000006}]},{"osd":3,"last update":"Tue Jul 18 03:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.47499999999999998,"15min":0.45700000000000002},"min":{"1min":0.313,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.498,"15min":0.47999999999999998},"min":{"1min":0.34699999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.35899999999999999}]},{"osd":4,"last update":"Tue Jul 18 03:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.63400000000000001,"15min":0.60599999999999998},"min":{"1min":0.47399999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.61099999999999999,"15min":0.59399999999999997},"min":{"1min":0.52800000000000002,"5min":0.433,"15min":0.433},"max":{"1min":1.0680000000000001,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.65800000000000003}]},{"osd":5,"last update":"Tue Jul 18 03:05:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.64700000000000002,"15min":0.63500000000000001},"min":{"1min":0.46899999999999997,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.028,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.68200000000000005,"15min":0.65900000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.998,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.79600000000000004}]},{"osd":6,"last update":"Tue Jul 18 03:05:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59899999999999998,"15min":0.58999999999999997},"min":{"1min":0.439,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.91900000000000004,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.624,"15min":0.59999999999999998},"min":{"1min":0.42099999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.94699999999999995,"5min":1.216,"15min":1.216},"last":0.46300000000000002}]},{"osd":7,"last update":"Tue Jul 18 03:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.77800000000000002,"15min":0.76900000000000002},"min":{"1min":0.52200000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.177,"5min":1.349,"15min":1.349},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.81200000000000006,"15min":0.81000000000000005},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.1539999999999999,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.74099999999999999}]},{"osd":8,"last update":"Tue Jul 18 03:05:04 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.747,"15min":0.746},"min":{"1min":0.63300000000000001,"5min":0.498,"15min":0.498},"max":{"1min":1.0629999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.622},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.82799999999999996,"15min":0.83299999999999996},"min":{"1min":0.56899999999999995,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.181,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.52000000000000002}]},{"osd":9,"last update":"Tue Jul 18 03:05:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.79000000000000004,"15min":0.79100000000000004},"min":{"1min":0.51400000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.107,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.871,"15min":0.877},"min":{"1min":0.52200000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.218,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.76100000000000001}]},{"osd":10,"last update":"Tue Jul 18 03:04:51 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.67300000000000004}]},{"osd":11,"last update":"Tue Jul 18 03:05:14 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.68400000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411389,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":1019904,"data_stored":505882,"data_compressed":17573,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.54100000000000004,"15min":0.53100000000000003},"min":{"1min":0.34899999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55900000000000005,"15min":0.54700000000000004},"min":{"1min":0.373,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.94799999999999995,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.40400000000000003}]},{"osd":1,"last update":"Tue Jul 18 03:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53600000000000003,"15min":0.501},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.80900000000000005,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.52900000000000003,"15min":0.495},"min":{"1min":0.34300000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.72899999999999998,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.42699999999999999}]},{"osd":3,"last update":"Tue Jul 18 03:05:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57999999999999996,"15min":0.55900000000000005},"min":{"1min":0.40999999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.621,"5min":0.63300000000000001,"15min":0.58899999999999997},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.587,"5min":3.234,"15min":3.234},"last":0.55100000000000005}]},{"osd":4,"last update":"Tue Jul 18 03:04:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.57499999999999996,"15min":0.57899999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.86899999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78900000000000003,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.60399999999999998}]},{"osd":5,"last update":"Tue Jul 18 03:05:00 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.45200000000000001,"15min":0.442},"min":{"1min":0.312,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.498},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.46300000000000002,"15min":0.44700000000000001},"min":{"1min":0.32200000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.64000000000000001}]},{"osd":6,"last update":"Tue Jul 18 03:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.65800000000000003,"15min":0.64000000000000001},"min":{"1min":0.46600000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.70799999999999996,"15min":0.68799999999999994},"min":{"1min":0.53800000000000003,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":0.72799999999999998}]},{"osd":7,"last update":"Tue Jul 18 03:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.65700000000000003,"15min":0.65800000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.93700000000000006,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.85199999999999998,"5min":0.874,"15min":0.874},"last":0.75700000000000001}]},{"osd":8,"last update":"Tue Jul 18 03:05:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.63,"15min":0.61699999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.66200000000000003,"15min":0.65100000000000002},"min":{"1min":0.41099999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.69599999999999995}]},{"osd":9,"last update":"Tue Jul 18 03:04:33 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.80400000000000005}]},{"osd":10,"last update":"Tue Jul 18 03:04:51 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.745}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]}]},{"osd":3,"up_from":25,"seq":107374182454,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":974848,"data_stored":483209,"data_compressed":17573,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:05:24 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.42599999999999999,"15min":0.40300000000000002},"min":{"1min":0.32300000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.44700000000000001,"15min":0.41399999999999998},"min":{"1min":0.30299999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.86699999999999999,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.56699999999999995}]},{"osd":1,"last update":"Tue Jul 18 03:05:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59699999999999998,"15min":0.56899999999999995},"min":{"1min":0.432,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.87,"5min":1.115,"15min":1.115},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.62,"15min":0.57999999999999996},"min":{"1min":0.432,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.64700000000000002}]},{"osd":2,"last update":"Tue Jul 18 03:05:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.64700000000000002,"15min":0.60299999999999998},"min":{"1min":0.46400000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.1140000000000001,"5min":1.26,"15min":1.26},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61799999999999999,"15min":0.59599999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.036,"5min":1.212,"15min":1.212},"last":0.51100000000000001}]},{"osd":4,"last update":"Tue Jul 18 03:04:43 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.64100000000000001,"15min":0.61299999999999999},"min":{"1min":0.55800000000000005,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.64300000000000002,"15min":0.60999999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0249999999999999,"5min":1.143,"15min":1.143},"last":0.69699999999999995}]},{"osd":5,"last update":"Tue Jul 18 03:04:56 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.65900000000000003,"15min":0.629},"min":{"1min":0.48699999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.70099999999999996,"15min":0.67800000000000005},"min":{"1min":0.53500000000000003,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.79900000000000004}]},{"osd":6,"last update":"Tue Jul 18 03:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.66900000000000004,"15min":0.64300000000000002},"min":{"1min":0.48099999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0740000000000001,"5min":1.385,"15min":1.385},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.60999999999999999,"15min":0.59999999999999998},"min":{"1min":0.434,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85999999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.55200000000000005}]},{"osd":7,"last update":"Tue Jul 18 03:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.73699999999999999,"15min":0.72299999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.75900000000000001,"15min":0.75},"min":{"1min":0.504,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.84799999999999998}]},{"osd":8,"last update":"Tue Jul 18 03:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.76200000000000001}]},{"osd":9,"last update":"Tue Jul 18 03:04:30 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.73999999999999999}]},{"osd":10,"last update":"Tue Jul 18 03:04:51 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.96199999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]}]},{"osd":4,"up_from":33,"seq":141733920818,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":978944,"data_stored":487707,"data_compressed":17573,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.56200000000000006,"15min":0.54600000000000004},"min":{"1min":0.39600000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.439},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.56899999999999995,"15min":0.54500000000000004},"min":{"1min":0.42799999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.48999999999999999}]},{"osd":1,"last update":"Tue Jul 18 03:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.378,"15min":0.35799999999999998},"min":{"1min":0.30399999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.39200000000000002,"15min":0.375},"min":{"1min":0.32300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.46600000000000003}]},{"osd":2,"last update":"Tue Jul 18 03:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.58199999999999996,"15min":0.56999999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.58299999999999996,"15min":0.56000000000000005},"min":{"1min":0.435,"5min":0.378,"15min":0.378},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.58099999999999996}]},{"osd":3,"last update":"Tue Jul 18 03:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.60799999999999998,"15min":0.58199999999999996},"min":{"1min":0.45700000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.96199999999999997,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.59599999999999997,"15min":0.58099999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.54100000000000004}]},{"osd":5,"last update":"Tue Jul 18 03:05:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.629,"15min":0.60299999999999998},"min":{"1min":0.48099999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.96299999999999997,"5min":1.214,"15min":1.214},"last":0.751},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.627,"15min":0.60499999999999998},"min":{"1min":0.51500000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.97999999999999998,"5min":1.306,"15min":1.306},"last":0.78200000000000003}]},{"osd":6,"last update":"Tue Jul 18 03:05:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.71699999999999997,"15min":0.72599999999999998},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.42399999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.61199999999999999}]},{"osd":7,"last update":"Tue Jul 18 03:04:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.60799999999999998,"15min":0.60199999999999998},"min":{"1min":0.372,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.626},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.65200000000000002,"15min":0.64600000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.873}]},{"osd":8,"last update":"Tue Jul 18 03:05:07 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.73999999999999999,"15min":0.73499999999999999},"min":{"1min":0.59799999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.98399999999999999,"5min":1.155,"15min":1.155},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.752,"15min":0.74199999999999999},"min":{"1min":0.56299999999999994,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0920000000000001,"5min":1.135,"15min":1.135},"last":0.90400000000000003}]},{"osd":9,"last update":"Tue Jul 18 03:05:33 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.72399999999999998,"15min":0.72199999999999998},"min":{"1min":0.54000000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.82299999999999995,"15min":0.81100000000000005},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":3.415,"5min":3.415,"15min":3.415},"last":0.71399999999999997}]},{"osd":10,"last update":"Tue Jul 18 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.66500000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91700000000000004}]}]},{"osd":5,"up_from":38,"seq":163208757293,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":1016,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":1040384,"data_stored":1074811,"data_compressed":53332,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:05:02 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54100000000000004,"15min":0.52900000000000003},"min":{"1min":0.373,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.79200000000000004,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54700000000000004,"15min":0.53700000000000003},"min":{"1min":0.34200000000000003,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.63800000000000001,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.54100000000000004}]},{"osd":1,"last update":"Tue Jul 18 03:05:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.55800000000000005,"15min":0.53700000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79800000000000004,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55300000000000005,"15min":0.54300000000000004},"min":{"1min":0.39000000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.75600000000000001}]},{"osd":2,"last update":"Tue Jul 18 03:05:02 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.38600000000000001,"15min":0.373},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.64500000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.432,"5min":0.41099999999999998,"15min":0.39500000000000002},"min":{"1min":0.27500000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.59299999999999997,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.51800000000000002}]},{"osd":3,"last update":"Tue Jul 18 03:05:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58799999999999997,"15min":0.56799999999999995},"min":{"1min":0.45000000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58599999999999997,"15min":0.56899999999999995},"min":{"1min":0.41599999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73299999999999998,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.63800000000000001}]},{"osd":4,"last update":"Tue Jul 18 03:05:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58199999999999996,"15min":0.56799999999999995},"min":{"1min":0.42399999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.80700000000000005,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.58899999999999997,"15min":0.56499999999999995},"min":{"1min":0.44700000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.82199999999999995}]},{"osd":6,"last update":"Tue Jul 18 03:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.61599999999999999,"15min":0.60299999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.63800000000000001,"15min":0.622},"min":{"1min":0.54300000000000004,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.85799999999999998}]},{"osd":7,"last update":"Tue Jul 18 03:04:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.67500000000000004,"15min":0.67600000000000005},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.875},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.68400000000000005,"15min":0.68799999999999994},"min":{"1min":0.41399999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.89100000000000001}]},{"osd":8,"last update":"Tue Jul 18 03:05:06 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.56299999999999994,"15min":0.57199999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.71199999999999997,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.81299999999999994,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.73999999999999999}]},{"osd":9,"last update":"Tue Jul 18 03:04:32 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.91200000000000003}]},{"osd":10,"last update":"Tue Jul 18 03:04:56 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.93300000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34117,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34117,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34117,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35759,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35759,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35759,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35759,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35759,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":40207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-18T03:05:35.302 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-18T03:05:35.441 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:35 smithi179 ceph-mon[129229]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:05:35.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:35 smithi179 ceph-mon[129229]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:05:35.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:35 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/2866106597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:05:35.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:35 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/3800072218' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:05:35.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.543+0000 7f41d4f9d700 1 -- 172.21.15.101:0/2652267538 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f41d00ff9f0 msgr2=0x7f41d00ffe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:35.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.543+0000 7f41d4f9d700 1 --2- 172.21.15.101:0/2652267538 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f41d00ff9f0 0x7f41d00ffe00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f41c0009990 tx=0x7f41c0009ca0 comp rx=0 tx=0).stop 2023-07-18T03:05:35.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.543+0000 7f41d4f9d700 1 -- 172.21.15.101:0/2652267538 shutdown_connections 2023-07-18T03:05:35.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.543+0000 7f41d4f9d700 1 --2- 172.21.15.101:0/2652267538 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41d0101570 0x7f41d0107e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:35.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.543+0000 7f41d4f9d700 1 --2- 172.21.15.101:0/2652267538 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f41d00ff9f0 0x7f41d00ffe00 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:35.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.543+0000 7f41d4f9d700 1 --2- 172.21.15.101:0/2652267538 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41d0100be0 0x7f41d0101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:35.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.543+0000 7f41d4f9d700 1 -- 172.21.15.101:0/2652267538 >> 172.21.15.101:0/2652267538 conn(0x7f41d00fb010 msgr2=0x7f41d00fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:35.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.543+0000 7f41d4f9d700 1 -- 172.21.15.101:0/2652267538 shutdown_connections 2023-07-18T03:05:35.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.543+0000 7f41d4f9d700 1 -- 172.21.15.101:0/2652267538 wait complete. 2023-07-18T03:05:35.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.544+0000 7f41d4f9d700 1 Processor -- start 2023-07-18T03:05:35.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.545+0000 7f41d4f9d700 1 -- start start 2023-07-18T03:05:35.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.545+0000 7f41d4f9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41d00ff9f0 0x7f41d0195580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:35.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.545+0000 7f41d4f9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41d0100be0 0x7f41d0195ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:35.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.545+0000 7f41d4f9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f41d0101570 0x7f41d019cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:35.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.545+0000 7f41d4f9d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d019d100 con 0x7f41d0101570 2023-07-18T03:05:35.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.545+0000 7f41d4f9d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d019d240 con 0x7f41d00ff9f0 2023-07-18T03:05:35.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.545+0000 7f41d4f9d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d019d3d0 con 0x7f41d0100be0 2023-07-18T03:05:35.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.546+0000 7f41ceffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41d0100be0 0x7f41d0195ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:35.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.546+0000 7f41cffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f41d0101570 0x7f41d019cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:35.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.546+0000 7f41ceffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41d0100be0 0x7f41d0195ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:43794/0 (socket says 172.21.15.101:43794) 2023-07-18T03:05:35.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.546+0000 7f41ceffd700 1 -- 172.21.15.101:0/2714070805 learned_addr learned my addr 172.21.15.101:0/2714070805 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:35.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.546+0000 7f41cf7fe700 1 --2- 172.21.15.101:0/2714070805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41d00ff9f0 0x7f41d0195580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:35.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.546+0000 7f41ceffd700 1 -- 172.21.15.101:0/2714070805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41d00ff9f0 msgr2=0x7f41d0195580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:35.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.546+0000 7f41ceffd700 1 --2- 172.21.15.101:0/2714070805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41d00ff9f0 0x7f41d0195580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:35.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.546+0000 7f41ceffd700 1 -- 172.21.15.101:0/2714070805 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f41d0101570 msgr2=0x7f41d019cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:35.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.546+0000 7f41ceffd700 1 --2- 172.21.15.101:0/2714070805 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f41d0101570 0x7f41d019cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:35.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.546+0000 7f41ceffd700 1 -- 172.21.15.101:0/2714070805 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41c00095f0 con 0x7f41d0100be0 2023-07-18T03:05:35.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.546+0000 7f41cffff700 1 --2- 172.21.15.101:0/2714070805 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f41d0101570 0x7f41d019cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:05:35.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.546+0000 7f41ceffd700 1 --2- 172.21.15.101:0/2714070805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41d0100be0 0x7f41d0195ac0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f41c400e9a0 tx=0x7f41c400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:35.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.546+0000 7f41ccff9700 1 -- 172.21.15.101:0/2714070805 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c4009790 con 0x7f41d0100be0 2023-07-18T03:05:35.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.547+0000 7f41d4f9d700 1 -- 172.21.15.101:0/2714070805 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41d019d6b0 con 0x7f41d0100be0 2023-07-18T03:05:35.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.547+0000 7f41ccff9700 1 -- 172.21.15.101:0/2714070805 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41c4010430 con 0x7f41d0100be0 2023-07-18T03:05:35.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.547+0000 7f41ccff9700 1 -- 172.21.15.101:0/2714070805 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c4018680 con 0x7f41d0100be0 2023-07-18T03:05:35.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.547+0000 7f41d4f9d700 1 -- 172.21.15.101:0/2714070805 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41d019dc10 con 0x7f41d0100be0 2023-07-18T03:05:35.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.548+0000 7f41d4f9d700 1 -- 172.21.15.101:0/2714070805 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41d0106e90 con 0x7f41d0100be0 2023-07-18T03:05:35.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.550+0000 7f41ccff9700 1 -- 172.21.15.101:0/2714070805 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f41c40105a0 con 0x7f41d0100be0 2023-07-18T03:05:35.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.551+0000 7f41ccff9700 1 --2- 172.21.15.101:0/2714070805 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f41b80a12e0 0x7f41b80a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:35.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.551+0000 7f41ccff9700 1 -- 172.21.15.101:0/2714070805 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f41c4014070 con 0x7f41d0100be0 2023-07-18T03:05:35.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.551+0000 7f41cf7fe700 1 --2- 172.21.15.101:0/2714070805 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f41b80a12e0 0x7f41b80a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:35.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.552+0000 7f41cf7fe700 1 --2- 172.21.15.101:0/2714070805 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f41b80a12e0 0x7f41b80a3790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f41c000b2e0 tx=0x7f41c0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:35.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.553+0000 7f41ccff9700 1 -- 172.21.15.101:0/2714070805 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f41c4096440 con 0x7f41d0100be0 2023-07-18T03:05:35.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.749+0000 7f41d4f9d700 1 -- 172.21.15.101:0/2714070805 --> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f41d0060950 con 0x7f41b80a12e0 2023-07-18T03:05:35.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.762+0000 7f41ccff9700 1 -- 172.21.15.101:0/2714070805 <== mgr.14150 v2:172.21.15.101:6800/3963742842 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422830 (secure 0 0 0) 0x7f41d0060950 con 0x7f41b80a12e0 2023-07-18T03:05:35.751 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:05:35.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.767+0000 7f41d4f9d700 1 -- 172.21.15.101:0/2714070805 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f41b80a12e0 msgr2=0x7f41b80a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:35.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.767+0000 7f41d4f9d700 1 --2- 172.21.15.101:0/2714070805 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f41b80a12e0 0x7f41b80a3790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f41c000b2e0 tx=0x7f41c0005fb0 comp rx=0 tx=0).stop 2023-07-18T03:05:35.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.767+0000 7f41d4f9d700 1 -- 172.21.15.101:0/2714070805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41d0100be0 msgr2=0x7f41d0195ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:35.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.767+0000 7f41d4f9d700 1 --2- 172.21.15.101:0/2714070805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41d0100be0 0x7f41d0195ac0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f41c400e9a0 tx=0x7f41c400ecb0 comp rx=0 tx=0).stop 2023-07-18T03:05:35.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.767+0000 7f41d4f9d700 1 -- 172.21.15.101:0/2714070805 shutdown_connections 2023-07-18T03:05:35.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.767+0000 7f41d4f9d700 1 --2- 172.21.15.101:0/2714070805 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f41b80a12e0 0x7f41b80a3790 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:35.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.767+0000 7f41d4f9d700 1 --2- 172.21.15.101:0/2714070805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41d00ff9f0 0x7f41d0195580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:35.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.767+0000 7f41d4f9d700 1 --2- 172.21.15.101:0/2714070805 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f41d0101570 0x7f41d019cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:35.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.767+0000 7f41d4f9d700 1 --2- 172.21.15.101:0/2714070805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41d0100be0 0x7f41d0195ac0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:35.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.767+0000 7f41d4f9d700 1 -- 172.21.15.101:0/2714070805 >> 172.21.15.101:0/2714070805 conn(0x7f41d00fb010 msgr2=0x7f41d0103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:35.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.767+0000 7f41d4f9d700 1 -- 172.21.15.101:0/2714070805 shutdown_connections 2023-07-18T03:05:35.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:35.767+0000 7f41d4f9d700 1 -- 172.21.15.101:0/2714070805 wait complete. 2023-07-18T03:05:35.755 INFO:teuthology.orchestra.run.smithi101.stderr:dumped all 2023-07-18T03:05:35.771 INFO:teuthology.orchestra.run.smithi101.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2023-07-18T03:05:34.855176+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597922,"num_objects":59,"num_object_clones":0,"num_object_copies":179,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":151,"num_read_kb":132,"num_write":301,"num_write_kb":2983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":242,"ondisk_log_size":242,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335092,"kb_used_data":11668,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606732,"statfs":{"total":1151940427776,"available":1151597293568,"internally_reserved":0,"allocated":11948032,"data_stored":7615145,"data_compressed":318153,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2962,"num_objects":35,"num_object_clones":0,"num_object_copies":107,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":15,"num_read_kb":13,"num_write":71,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.923051"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.942306+0000","last_change":"2023-07-18T03:04:59.805689+0000","last_active":"2023-07-18T03:05:23.942306+0000","last_peered":"2023-07-18T03:05:23.942306+0000","last_clean":"2023-07-18T03:05:23.942306+0000","last_became_active":"2023-07-18T03:04:59.805224+0000","last_became_peered":"2023-07-18T03:04:59.805224+0000","last_unstale":"2023-07-18T03:05:23.942306+0000","last_undegraded":"2023-07-18T03:05:23.942306+0000","last_fullsized":"2023-07-18T03:05:23.942306+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:55:30.570149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.938022+0000","last_change":"2023-07-18T03:04:59.805013+0000","last_active":"2023-07-18T03:05:23.938022+0000","last_peered":"2023-07-18T03:05:23.938022+0000","last_clean":"2023-07-18T03:05:23.938022+0000","last_became_active":"2023-07-18T03:04:59.804702+0000","last_became_peered":"2023-07-18T03:04:59.804702+0000","last_unstale":"2023-07-18T03:05:23.938022+0000","last_undegraded":"2023-07-18T03:05:23.938022+0000","last_fullsized":"2023-07-18T03:05:23.938022+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:11:13.364060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.938179+0000","last_change":"2023-07-18T03:04:59.809035+0000","last_active":"2023-07-18T03:05:23.938179+0000","last_peered":"2023-07-18T03:05:23.938179+0000","last_clean":"2023-07-18T03:05:23.938179+0000","last_became_active":"2023-07-18T03:04:59.808630+0000","last_became_peered":"2023-07-18T03:04:59.808630+0000","last_unstale":"2023-07-18T03:05:23.938179+0000","last_undegraded":"2023-07-18T03:05:23.938179+0000","last_fullsized":"2023-07-18T03:05:23.938179+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:18:02.359613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089479+0000","last_change":"2023-07-18T03:04:59.803510+0000","last_active":"2023-07-18T03:05:24.089479+0000","last_peered":"2023-07-18T03:05:24.089479+0000","last_clean":"2023-07-18T03:05:24.089479+0000","last_became_active":"2023-07-18T03:04:59.803060+0000","last_became_peered":"2023-07-18T03:04:59.803060+0000","last_unstale":"2023-07-18T03:05:24.089479+0000","last_undegraded":"2023-07-18T03:05:24.089479+0000","last_fullsized":"2023-07-18T03:05:24.089479+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:15:54.456383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.936577+0000","last_change":"2023-07-18T03:04:59.806911+0000","last_active":"2023-07-18T03:05:23.936577+0000","last_peered":"2023-07-18T03:05:23.936577+0000","last_clean":"2023-07-18T03:05:23.936577+0000","last_became_active":"2023-07-18T03:04:59.806597+0000","last_became_peered":"2023-07-18T03:04:59.806597+0000","last_unstale":"2023-07-18T03:05:23.936577+0000","last_undegraded":"2023-07-18T03:05:23.936577+0000","last_fullsized":"2023-07-18T03:05:23.936577+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:06:55.956671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.936608+0000","last_change":"2023-07-18T03:04:59.809263+0000","last_active":"2023-07-18T03:05:23.936608+0000","last_peered":"2023-07-18T03:05:23.936608+0000","last_clean":"2023-07-18T03:05:23.936608+0000","last_became_active":"2023-07-18T03:04:59.808956+0000","last_became_peered":"2023-07-18T03:04:59.808956+0000","last_unstale":"2023-07-18T03:05:23.936608+0000","last_undegraded":"2023-07-18T03:05:23.936608+0000","last_fullsized":"2023-07-18T03:05:23.936608+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:49:07.735738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.933471+0000","last_change":"2023-07-18T03:04:59.804946+0000","last_active":"2023-07-18T03:05:23.933471+0000","last_peered":"2023-07-18T03:05:23.933471+0000","last_clean":"2023-07-18T03:05:23.933471+0000","last_became_active":"2023-07-18T03:04:59.804610+0000","last_became_peered":"2023-07-18T03:04:59.804610+0000","last_unstale":"2023-07-18T03:05:23.933471+0000","last_undegraded":"2023-07-18T03:05:23.933471+0000","last_fullsized":"2023-07-18T03:05:23.933471+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:00:45.505896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089839+0000","last_change":"2023-07-18T03:04:59.801796+0000","last_active":"2023-07-18T03:05:24.089839+0000","last_peered":"2023-07-18T03:05:24.089839+0000","last_clean":"2023-07-18T03:05:24.089839+0000","last_became_active":"2023-07-18T03:04:59.801432+0000","last_became_peered":"2023-07-18T03:04:59.801432+0000","last_unstale":"2023-07-18T03:05:24.089839+0000","last_undegraded":"2023-07-18T03:05:24.089839+0000","last_fullsized":"2023-07-18T03:05:24.089839+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:25:50.242446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089997+0000","last_change":"2023-07-18T03:04:59.798191+0000","last_active":"2023-07-18T03:05:24.089997+0000","last_peered":"2023-07-18T03:05:24.089997+0000","last_clean":"2023-07-18T03:05:24.089997+0000","last_became_active":"2023-07-18T03:04:59.797863+0000","last_became_peered":"2023-07-18T03:04:59.797863+0000","last_unstale":"2023-07-18T03:05:24.089997+0000","last_undegraded":"2023-07-18T03:05:24.089997+0000","last_fullsized":"2023-07-18T03:05:24.089997+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:23:59.798693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.936307+0000","last_change":"2023-07-18T03:04:59.809867+0000","last_active":"2023-07-18T03:05:23.936307+0000","last_peered":"2023-07-18T03:05:23.936307+0000","last_clean":"2023-07-18T03:05:23.936307+0000","last_became_active":"2023-07-18T03:04:59.809568+0000","last_became_peered":"2023-07-18T03:04:59.809568+0000","last_unstale":"2023-07-18T03:05:23.936307+0000","last_undegraded":"2023-07-18T03:05:23.936307+0000","last_fullsized":"2023-07-18T03:05:23.936307+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:51:02.596324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.934225+0000","last_change":"2023-07-18T03:04:59.805920+0000","last_active":"2023-07-18T03:05:23.934225+0000","last_peered":"2023-07-18T03:05:23.934225+0000","last_clean":"2023-07-18T03:05:23.934225+0000","last_became_active":"2023-07-18T03:04:59.805559+0000","last_became_peered":"2023-07-18T03:04:59.805559+0000","last_unstale":"2023-07-18T03:05:23.934225+0000","last_undegraded":"2023-07-18T03:05:23.934225+0000","last_fullsized":"2023-07-18T03:05:23.934225+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:57:01.069959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089324+0000","last_change":"2023-07-18T03:04:59.797940+0000","last_active":"2023-07-18T03:05:24.089324+0000","last_peered":"2023-07-18T03:05:24.089324+0000","last_clean":"2023-07-18T03:05:24.089324+0000","last_became_active":"2023-07-18T03:04:59.797575+0000","last_became_peered":"2023-07-18T03:04:59.797575+0000","last_unstale":"2023-07-18T03:05:24.089324+0000","last_undegraded":"2023-07-18T03:05:24.089324+0000","last_fullsized":"2023-07-18T03:05:24.089324+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:25:37.959233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.091650+0000","last_change":"2023-07-18T03:04:59.798026+0000","last_active":"2023-07-18T03:05:24.091650+0000","last_peered":"2023-07-18T03:05:24.091650+0000","last_clean":"2023-07-18T03:05:24.091650+0000","last_became_active":"2023-07-18T03:04:59.797734+0000","last_became_peered":"2023-07-18T03:04:59.797734+0000","last_unstale":"2023-07-18T03:05:24.091650+0000","last_undegraded":"2023-07-18T03:05:24.091650+0000","last_fullsized":"2023-07-18T03:05:24.091650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:33:14.476227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.938064+0000","last_change":"2023-07-18T03:04:59.804964+0000","last_active":"2023-07-18T03:05:23.938064+0000","last_peered":"2023-07-18T03:05:23.938064+0000","last_clean":"2023-07-18T03:05:23.938064+0000","last_became_active":"2023-07-18T03:04:59.804660+0000","last_became_peered":"2023-07-18T03:04:59.804660+0000","last_unstale":"2023-07-18T03:05:23.938064+0000","last_undegraded":"2023-07-18T03:05:23.938064+0000","last_fullsized":"2023-07-18T03:05:23.938064+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:31:37.327250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.938212+0000","last_change":"2023-07-18T03:04:59.809384+0000","last_active":"2023-07-18T03:05:23.938212+0000","last_peered":"2023-07-18T03:05:23.938212+0000","last_clean":"2023-07-18T03:05:23.938212+0000","last_became_active":"2023-07-18T03:04:59.808969+0000","last_became_peered":"2023-07-18T03:04:59.808969+0000","last_unstale":"2023-07-18T03:05:23.938212+0000","last_undegraded":"2023-07-18T03:05:23.938212+0000","last_fullsized":"2023-07-18T03:05:23.938212+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:41:11.572546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089702+0000","last_change":"2023-07-18T03:04:59.808344+0000","last_active":"2023-07-18T03:05:24.089702+0000","last_peered":"2023-07-18T03:05:24.089702+0000","last_clean":"2023-07-18T03:05:24.089702+0000","last_became_active":"2023-07-18T03:04:59.807995+0000","last_became_peered":"2023-07-18T03:04:59.807995+0000","last_unstale":"2023-07-18T03:05:24.089702+0000","last_undegraded":"2023-07-18T03:05:24.089702+0000","last_fullsized":"2023-07-18T03:05:24.089702+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:20:11.542223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.942105+0000","last_change":"2023-07-18T03:04:59.806584+0000","last_active":"2023-07-18T03:05:23.942105+0000","last_peered":"2023-07-18T03:05:23.942105+0000","last_clean":"2023-07-18T03:05:23.942105+0000","last_became_active":"2023-07-18T03:04:59.804118+0000","last_became_peered":"2023-07-18T03:04:59.804118+0000","last_unstale":"2023-07-18T03:05:23.942105+0000","last_undegraded":"2023-07-18T03:05:23.942105+0000","last_fullsized":"2023-07-18T03:05:23.942105+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:10:34.684069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.934563+0000","last_change":"2023-07-18T03:04:59.812610+0000","last_active":"2023-07-18T03:05:23.934563+0000","last_peered":"2023-07-18T03:05:23.934563+0000","last_clean":"2023-07-18T03:05:23.934563+0000","last_became_active":"2023-07-18T03:04:59.812230+0000","last_became_peered":"2023-07-18T03:04:59.812230+0000","last_unstale":"2023-07-18T03:05:23.934563+0000","last_undegraded":"2023-07-18T03:05:23.934563+0000","last_fullsized":"2023-07-18T03:05:23.934563+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:05:54.390233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.939088+0000","last_change":"2023-07-18T03:04:59.811880+0000","last_active":"2023-07-18T03:05:23.939088+0000","last_peered":"2023-07-18T03:05:23.939088+0000","last_clean":"2023-07-18T03:05:23.939088+0000","last_became_active":"2023-07-18T03:04:59.811561+0000","last_became_peered":"2023-07-18T03:04:59.811561+0000","last_unstale":"2023-07-18T03:05:23.939088+0000","last_undegraded":"2023-07-18T03:05:23.939088+0000","last_fullsized":"2023-07-18T03:05:23.939088+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:49:17.068386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.938056+0000","last_change":"2023-07-18T03:05:03.880843+0000","last_active":"2023-07-18T03:05:23.938056+0000","last_peered":"2023-07-18T03:05:23.938056+0000","last_clean":"2023-07-18T03:05:23.938056+0000","last_became_active":"2023-07-18T03:05:03.880391+0000","last_became_peered":"2023-07-18T03:05:03.880391+0000","last_unstale":"2023-07-18T03:05:23.938056+0000","last_undegraded":"2023-07-18T03:05:23.938056+0000","last_fullsized":"2023-07-18T03:05:23.938056+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:57:04.981691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'4","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.934651+0000","last_change":"2023-07-18T03:04:58.848771+0000","last_active":"2023-07-18T03:05:23.934651+0000","last_peered":"2023-07-18T03:05:23.934651+0000","last_clean":"2023-07-18T03:05:23.934651+0000","last_became_active":"2023-07-18T03:04:58.848405+0000","last_became_peered":"2023-07-18T03:04:58.848405+0000","last_unstale":"2023-07-18T03:05:23.934651+0000","last_undegraded":"2023-07-18T03:05:23.934651+0000","last_fullsized":"2023-07-18T03:05:23.934651+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:21:40.139702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.090125+0000","last_change":"2023-07-18T03:04:59.805749+0000","last_active":"2023-07-18T03:05:24.090125+0000","last_peered":"2023-07-18T03:05:24.090125+0000","last_clean":"2023-07-18T03:05:24.090125+0000","last_became_active":"2023-07-18T03:04:59.805388+0000","last_became_peered":"2023-07-18T03:04:59.805388+0000","last_unstale":"2023-07-18T03:05:24.090125+0000","last_undegraded":"2023-07-18T03:05:24.090125+0000","last_fullsized":"2023-07-18T03:05:24.090125+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:12:30.269632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089071+0000","last_change":"2023-07-18T03:05:03.868819+0000","last_active":"2023-07-18T03:05:24.089071+0000","last_peered":"2023-07-18T03:05:24.089071+0000","last_clean":"2023-07-18T03:05:24.089071+0000","last_became_active":"2023-07-18T03:05:03.868387+0000","last_became_peered":"2023-07-18T03:05:03.868387+0000","last_unstale":"2023-07-18T03:05:24.089071+0000","last_undegraded":"2023-07-18T03:05:24.089071+0000","last_fullsized":"2023-07-18T03:05:24.089071+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:07:41.670443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"82'1","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.090369+0000","last_change":"2023-07-18T03:04:58.856084+0000","last_active":"2023-07-18T03:05:24.090369+0000","last_peered":"2023-07-18T03:05:24.090369+0000","last_clean":"2023-07-18T03:05:24.090369+0000","last_became_active":"2023-07-18T03:04:58.855768+0000","last_became_peered":"2023-07-18T03:04:58.855768+0000","last_unstale":"2023-07-18T03:05:24.090369+0000","last_undegraded":"2023-07-18T03:05:24.090369+0000","last_fullsized":"2023-07-18T03:05:24.090369+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:28:05.975256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.938087+0000","last_change":"2023-07-18T03:04:59.809512+0000","last_active":"2023-07-18T03:05:23.938087+0000","last_peered":"2023-07-18T03:05:23.938087+0000","last_clean":"2023-07-18T03:05:23.938087+0000","last_became_active":"2023-07-18T03:04:59.808112+0000","last_became_peered":"2023-07-18T03:04:59.808112+0000","last_unstale":"2023-07-18T03:05:23.938087+0000","last_undegraded":"2023-07-18T03:05:23.938087+0000","last_fullsized":"2023-07-18T03:05:23.938087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:32:42.763158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.090171+0000","last_change":"2023-07-18T03:05:03.874728+0000","last_active":"2023-07-18T03:05:24.090171+0000","last_peered":"2023-07-18T03:05:24.090171+0000","last_clean":"2023-07-18T03:05:24.090171+0000","last_became_active":"2023-07-18T03:05:03.874262+0000","last_became_peered":"2023-07-18T03:05:03.874262+0000","last_unstale":"2023-07-18T03:05:24.090171+0000","last_undegraded":"2023-07-18T03:05:24.090171+0000","last_fullsized":"2023-07-18T03:05:24.090171+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:52:09.021724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:30.990070+0000","last_change":"2023-07-18T03:04:58.849140+0000","last_active":"2023-07-18T03:05:30.990070+0000","last_peered":"2023-07-18T03:05:30.990070+0000","last_clean":"2023-07-18T03:05:30.990070+0000","last_became_active":"2023-07-18T03:04:58.848693+0000","last_became_peered":"2023-07-18T03:04:58.848693+0000","last_unstale":"2023-07-18T03:05:30.990070+0000","last_undegraded":"2023-07-18T03:05:30.990070+0000","last_fullsized":"2023-07-18T03:05:30.990070+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:43:52.238305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089728+0000","last_change":"2023-07-18T03:04:59.807248+0000","last_active":"2023-07-18T03:05:24.089728+0000","last_peered":"2023-07-18T03:05:24.089728+0000","last_clean":"2023-07-18T03:05:24.089728+0000","last_became_active":"2023-07-18T03:04:59.806807+0000","last_became_peered":"2023-07-18T03:04:59.806807+0000","last_unstale":"2023-07-18T03:05:24.089728+0000","last_undegraded":"2023-07-18T03:05:24.089728+0000","last_fullsized":"2023-07-18T03:05:24.089728+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:35:50.471833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.088762+0000","last_change":"2023-07-18T03:05:03.869075+0000","last_active":"2023-07-18T03:05:24.088762+0000","last_peered":"2023-07-18T03:05:24.088762+0000","last_clean":"2023-07-18T03:05:24.088762+0000","last_became_active":"2023-07-18T03:05:03.868653+0000","last_became_peered":"2023-07-18T03:05:03.868653+0000","last_unstale":"2023-07-18T03:05:24.088762+0000","last_undegraded":"2023-07-18T03:05:24.088762+0000","last_fullsized":"2023-07-18T03:05:24.088762+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:44:09.437492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.937411+0000","last_change":"2023-07-18T03:04:58.843879+0000","last_active":"2023-07-18T03:05:23.937411+0000","last_peered":"2023-07-18T03:05:23.937411+0000","last_clean":"2023-07-18T03:05:23.937411+0000","last_became_active":"2023-07-18T03:04:58.843525+0000","last_became_peered":"2023-07-18T03:04:58.843525+0000","last_unstale":"2023-07-18T03:05:23.937411+0000","last_undegraded":"2023-07-18T03:05:23.937411+0000","last_fullsized":"2023-07-18T03:05:23.937411+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:42:05.200711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089568+0000","last_change":"2023-07-18T03:04:59.803597+0000","last_active":"2023-07-18T03:05:24.089568+0000","last_peered":"2023-07-18T03:05:24.089568+0000","last_clean":"2023-07-18T03:05:24.089568+0000","last_became_active":"2023-07-18T03:04:59.803293+0000","last_became_peered":"2023-07-18T03:04:59.803293+0000","last_unstale":"2023-07-18T03:05:24.089568+0000","last_undegraded":"2023-07-18T03:05:24.089568+0000","last_fullsized":"2023-07-18T03:05:24.089568+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:21:34.711198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.944721+0000","last_change":"2023-07-18T03:05:23.944721+0000","last_active":"2023-07-18T03:05:23.944721+0000","last_peered":"2023-07-18T03:05:23.944721+0000","last_clean":"2023-07-18T03:05:23.944721+0000","last_became_active":"2023-07-18T03:05:23.944101+0000","last_became_peered":"2023-07-18T03:05:23.944101+0000","last_unstale":"2023-07-18T03:05:23.944721+0000","last_undegraded":"2023-07-18T03:05:23.944721+0000","last_fullsized":"2023-07-18T03:05:23.944721+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:28:03.237639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:29.985509+0000","last_change":"2023-07-18T03:04:58.847990+0000","last_active":"2023-07-18T03:05:29.985509+0000","last_peered":"2023-07-18T03:05:29.985509+0000","last_clean":"2023-07-18T03:05:29.985509+0000","last_became_active":"2023-07-18T03:04:58.847634+0000","last_became_peered":"2023-07-18T03:04:58.847634+0000","last_unstale":"2023-07-18T03:05:29.985509+0000","last_undegraded":"2023-07-18T03:05:29.985509+0000","last_fullsized":"2023-07-18T03:05:29.985509+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:03:43.456118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.938391+0000","last_change":"2023-07-18T03:04:59.797076+0000","last_active":"2023-07-18T03:05:23.938391+0000","last_peered":"2023-07-18T03:05:23.938391+0000","last_clean":"2023-07-18T03:05:23.938391+0000","last_became_active":"2023-07-18T03:04:59.796753+0000","last_became_peered":"2023-07-18T03:04:59.796753+0000","last_unstale":"2023-07-18T03:05:23.938391+0000","last_undegraded":"2023-07-18T03:05:23.938391+0000","last_fullsized":"2023-07-18T03:05:23.938391+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:28:30.559043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.937833+0000","last_change":"2023-07-18T03:05:03.881793+0000","last_active":"2023-07-18T03:05:23.937833+0000","last_peered":"2023-07-18T03:05:23.937833+0000","last_clean":"2023-07-18T03:05:23.937833+0000","last_became_active":"2023-07-18T03:05:03.881341+0000","last_became_peered":"2023-07-18T03:05:03.881341+0000","last_unstale":"2023-07-18T03:05:23.937833+0000","last_undegraded":"2023-07-18T03:05:23.937833+0000","last_fullsized":"2023-07-18T03:05:23.937833+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:05:01.526051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:30.990979+0000","last_change":"2023-07-18T03:04:58.846732+0000","last_active":"2023-07-18T03:05:30.990979+0000","last_peered":"2023-07-18T03:05:30.990979+0000","last_clean":"2023-07-18T03:05:30.990979+0000","last_became_active":"2023-07-18T03:04:58.846321+0000","last_became_peered":"2023-07-18T03:04:58.846321+0000","last_unstale":"2023-07-18T03:05:30.990979+0000","last_undegraded":"2023-07-18T03:05:30.990979+0000","last_fullsized":"2023-07-18T03:05:30.990979+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:23:54.252356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.094233+0000","last_change":"2023-07-18T03:05:24.093277+0000","last_active":"2023-07-18T03:05:24.094233+0000","last_peered":"2023-07-18T03:05:24.094233+0000","last_clean":"2023-07-18T03:05:24.094233+0000","last_became_active":"2023-07-18T03:05:24.092842+0000","last_became_peered":"2023-07-18T03:05:24.092842+0000","last_unstale":"2023-07-18T03:05:24.094233+0000","last_undegraded":"2023-07-18T03:05:24.094233+0000","last_fullsized":"2023-07-18T03:05:24.094233+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:39:40.953742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.942246+0000","last_change":"2023-07-18T03:05:03.880782+0000","last_active":"2023-07-18T03:05:23.942246+0000","last_peered":"2023-07-18T03:05:23.942246+0000","last_clean":"2023-07-18T03:05:23.942246+0000","last_became_active":"2023-07-18T03:05:03.880362+0000","last_became_peered":"2023-07-18T03:05:03.880362+0000","last_unstale":"2023-07-18T03:05:23.942246+0000","last_undegraded":"2023-07-18T03:05:23.942246+0000","last_fullsized":"2023-07-18T03:05:23.942246+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:19:49.336092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:30.991863+0000","last_change":"2023-07-18T03:04:58.846095+0000","last_active":"2023-07-18T03:05:30.991863+0000","last_peered":"2023-07-18T03:05:30.991863+0000","last_clean":"2023-07-18T03:05:30.991863+0000","last_became_active":"2023-07-18T03:04:58.845242+0000","last_became_peered":"2023-07-18T03:04:58.845242+0000","last_unstale":"2023-07-18T03:05:30.991863+0000","last_undegraded":"2023-07-18T03:05:30.991863+0000","last_fullsized":"2023-07-18T03:05:30.991863+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:51:23.745763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089587+0000","last_change":"2023-07-18T03:04:59.803024+0000","last_active":"2023-07-18T03:05:24.089587+0000","last_peered":"2023-07-18T03:05:24.089587+0000","last_clean":"2023-07-18T03:05:24.089587+0000","last_became_active":"2023-07-18T03:04:59.802687+0000","last_became_peered":"2023-07-18T03:04:59.802687+0000","last_unstale":"2023-07-18T03:05:24.089587+0000","last_undegraded":"2023-07-18T03:05:24.089587+0000","last_fullsized":"2023-07-18T03:05:24.089587+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:14:46.616130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.937966+0000","last_change":"2023-07-18T03:05:03.872690+0000","last_active":"2023-07-18T03:05:23.937966+0000","last_peered":"2023-07-18T03:05:23.937966+0000","last_clean":"2023-07-18T03:05:23.937966+0000","last_became_active":"2023-07-18T03:05:03.872187+0000","last_became_peered":"2023-07-18T03:05:03.872187+0000","last_unstale":"2023-07-18T03:05:23.937966+0000","last_undegraded":"2023-07-18T03:05:23.937966+0000","last_fullsized":"2023-07-18T03:05:23.937966+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:05:04.373571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089675+0000","last_change":"2023-07-18T03:04:58.842877+0000","last_active":"2023-07-18T03:05:24.089675+0000","last_peered":"2023-07-18T03:05:24.089675+0000","last_clean":"2023-07-18T03:05:24.089675+0000","last_became_active":"2023-07-18T03:04:58.842516+0000","last_became_peered":"2023-07-18T03:04:58.842516+0000","last_unstale":"2023-07-18T03:05:24.089675+0000","last_undegraded":"2023-07-18T03:05:24.089675+0000","last_fullsized":"2023-07-18T03:05:24.089675+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:55:08.128143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.933854+0000","last_change":"2023-07-18T03:04:59.807592+0000","last_active":"2023-07-18T03:05:23.933854+0000","last_peered":"2023-07-18T03:05:23.933854+0000","last_clean":"2023-07-18T03:05:23.933854+0000","last_became_active":"2023-07-18T03:04:59.807276+0000","last_became_peered":"2023-07-18T03:04:59.807276+0000","last_unstale":"2023-07-18T03:05:23.933854+0000","last_undegraded":"2023-07-18T03:05:23.933854+0000","last_fullsized":"2023-07-18T03:05:23.933854+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:58:29.880849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089882+0000","last_change":"2023-07-18T03:05:03.876355+0000","last_active":"2023-07-18T03:05:24.089882+0000","last_peered":"2023-07-18T03:05:24.089882+0000","last_clean":"2023-07-18T03:05:24.089882+0000","last_became_active":"2023-07-18T03:05:03.875845+0000","last_became_peered":"2023-07-18T03:05:03.875845+0000","last_unstale":"2023-07-18T03:05:24.089882+0000","last_undegraded":"2023-07-18T03:05:24.089882+0000","last_fullsized":"2023-07-18T03:05:24.089882+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:01:54.675024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:29.985713+0000","last_change":"2023-07-18T03:04:58.840389+0000","last_active":"2023-07-18T03:05:29.985713+0000","last_peered":"2023-07-18T03:05:29.985713+0000","last_clean":"2023-07-18T03:05:29.985713+0000","last_became_active":"2023-07-18T03:04:58.839974+0000","last_became_peered":"2023-07-18T03:04:58.839974+0000","last_unstale":"2023-07-18T03:05:29.985713+0000","last_undegraded":"2023-07-18T03:05:29.985713+0000","last_fullsized":"2023-07-18T03:05:29.985713+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:21:20.010136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.090769+0000","last_change":"2023-07-18T03:04:59.807681+0000","last_active":"2023-07-18T03:05:24.090769+0000","last_peered":"2023-07-18T03:05:24.090769+0000","last_clean":"2023-07-18T03:05:24.090769+0000","last_became_active":"2023-07-18T03:04:59.807365+0000","last_became_peered":"2023-07-18T03:04:59.807365+0000","last_unstale":"2023-07-18T03:05:24.090769+0000","last_undegraded":"2023-07-18T03:05:24.090769+0000","last_fullsized":"2023-07-18T03:05:24.090769+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:35:09.523428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.092631+0000","last_change":"2023-07-18T03:05:24.091413+0000","last_active":"2023-07-18T03:05:24.092631+0000","last_peered":"2023-07-18T03:05:24.092631+0000","last_clean":"2023-07-18T03:05:24.092631+0000","last_became_active":"2023-07-18T03:05:24.090815+0000","last_became_peered":"2023-07-18T03:05:24.090815+0000","last_unstale":"2023-07-18T03:05:24.092631+0000","last_undegraded":"2023-07-18T03:05:24.092631+0000","last_fullsized":"2023-07-18T03:05:24.092631+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:18:39.070780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:29.984164+0000","last_change":"2023-07-18T03:04:58.846707+0000","last_active":"2023-07-18T03:05:29.984164+0000","last_peered":"2023-07-18T03:05:29.984164+0000","last_clean":"2023-07-18T03:05:29.984164+0000","last_became_active":"2023-07-18T03:04:58.846366+0000","last_became_peered":"2023-07-18T03:04:58.846366+0000","last_unstale":"2023-07-18T03:05:29.984164+0000","last_undegraded":"2023-07-18T03:05:29.984164+0000","last_fullsized":"2023-07-18T03:05:29.984164+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:54:55.036916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.092870+0000","last_change":"2023-07-18T03:05:24.092870+0000","last_active":"2023-07-18T03:05:24.092870+0000","last_peered":"2023-07-18T03:05:24.092870+0000","last_clean":"2023-07-18T03:05:24.092870+0000","last_became_active":"2023-07-18T03:05:24.092474+0000","last_became_peered":"2023-07-18T03:05:24.092474+0000","last_unstale":"2023-07-18T03:05:24.092870+0000","last_undegraded":"2023-07-18T03:05:24.092870+0000","last_fullsized":"2023-07-18T03:05:24.092870+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:22:48.062183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089952+0000","last_change":"2023-07-18T03:05:03.874444+0000","last_active":"2023-07-18T03:05:24.089952+0000","last_peered":"2023-07-18T03:05:24.089952+0000","last_clean":"2023-07-18T03:05:24.089952+0000","last_became_active":"2023-07-18T03:05:03.873737+0000","last_became_peered":"2023-07-18T03:05:03.873737+0000","last_unstale":"2023-07-18T03:05:24.089952+0000","last_undegraded":"2023-07-18T03:05:24.089952+0000","last_fullsized":"2023-07-18T03:05:24.089952+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:33:41.739156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089471+0000","last_change":"2023-07-18T03:04:58.845207+0000","last_active":"2023-07-18T03:05:24.089471+0000","last_peered":"2023-07-18T03:05:24.089471+0000","last_clean":"2023-07-18T03:05:24.089471+0000","last_became_active":"2023-07-18T03:04:58.844895+0000","last_became_peered":"2023-07-18T03:04:58.844895+0000","last_unstale":"2023-07-18T03:05:24.089471+0000","last_undegraded":"2023-07-18T03:05:24.089471+0000","last_fullsized":"2023-07-18T03:05:24.089471+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:16:23.615174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.090081+0000","last_change":"2023-07-18T03:04:59.807075+0000","last_active":"2023-07-18T03:05:24.090081+0000","last_peered":"2023-07-18T03:05:24.090081+0000","last_clean":"2023-07-18T03:05:24.090081+0000","last_became_active":"2023-07-18T03:04:59.806345+0000","last_became_peered":"2023-07-18T03:04:59.806345+0000","last_unstale":"2023-07-18T03:05:24.090081+0000","last_undegraded":"2023-07-18T03:05:24.090081+0000","last_fullsized":"2023-07-18T03:05:24.090081+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:38:38.275381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.091509+0000","last_change":"2023-07-18T03:05:24.091132+0000","last_active":"2023-07-18T03:05:24.091509+0000","last_peered":"2023-07-18T03:05:24.091509+0000","last_clean":"2023-07-18T03:05:24.091509+0000","last_became_active":"2023-07-18T03:05:24.090653+0000","last_became_peered":"2023-07-18T03:05:24.090653+0000","last_unstale":"2023-07-18T03:05:24.091509+0000","last_undegraded":"2023-07-18T03:05:24.091509+0000","last_fullsized":"2023-07-18T03:05:24.091509+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:37:46.585074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"89'12","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:29.985596+0000","last_change":"2023-07-18T03:04:58.844865+0000","last_active":"2023-07-18T03:05:29.985596+0000","last_peered":"2023-07-18T03:05:29.985596+0000","last_clean":"2023-07-18T03:05:29.985596+0000","last_became_active":"2023-07-18T03:04:58.844547+0000","last_became_peered":"2023-07-18T03:04:58.844547+0000","last_unstale":"2023-07-18T03:05:29.985596+0000","last_undegraded":"2023-07-18T03:05:29.985596+0000","last_fullsized":"2023-07-18T03:05:29.985596+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:01:48.220367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3794,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.937865+0000","last_change":"2023-07-18T03:04:59.803024+0000","last_active":"2023-07-18T03:05:23.937865+0000","last_peered":"2023-07-18T03:05:23.937865+0000","last_clean":"2023-07-18T03:05:23.937865+0000","last_became_active":"2023-07-18T03:04:59.802705+0000","last_became_peered":"2023-07-18T03:04:59.802705+0000","last_unstale":"2023-07-18T03:05:23.937865+0000","last_undegraded":"2023-07-18T03:05:23.937865+0000","last_fullsized":"2023-07-18T03:05:23.937865+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:51:13.019557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.090087+0000","last_change":"2023-07-18T03:05:03.876009+0000","last_active":"2023-07-18T03:05:24.090087+0000","last_peered":"2023-07-18T03:05:24.090087+0000","last_clean":"2023-07-18T03:05:24.090087+0000","last_became_active":"2023-07-18T03:05:03.875328+0000","last_became_peered":"2023-07-18T03:05:03.875328+0000","last_unstale":"2023-07-18T03:05:24.090087+0000","last_undegraded":"2023-07-18T03:05:24.090087+0000","last_fullsized":"2023-07-18T03:05:24.090087+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:29:03.036222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:30.992291+0000","last_change":"2023-07-18T03:04:58.846573+0000","last_active":"2023-07-18T03:05:30.992291+0000","last_peered":"2023-07-18T03:05:30.992291+0000","last_clean":"2023-07-18T03:05:30.992291+0000","last_became_active":"2023-07-18T03:04:58.846271+0000","last_became_peered":"2023-07-18T03:04:58.846271+0000","last_unstale":"2023-07-18T03:05:30.992291+0000","last_undegraded":"2023-07-18T03:05:30.992291+0000","last_fullsized":"2023-07-18T03:05:30.992291+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:07:28.151884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.091068+0000","last_change":"2023-07-18T03:04:59.801552+0000","last_active":"2023-07-18T03:05:24.091068+0000","last_peered":"2023-07-18T03:05:24.091068+0000","last_clean":"2023-07-18T03:05:24.091068+0000","last_became_active":"2023-07-18T03:04:59.801224+0000","last_became_peered":"2023-07-18T03:04:59.801224+0000","last_unstale":"2023-07-18T03:05:24.091068+0000","last_undegraded":"2023-07-18T03:05:24.091068+0000","last_fullsized":"2023-07-18T03:05:24.091068+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:43:54.484133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.941638+0000","last_change":"2023-07-18T03:05:03.881125+0000","last_active":"2023-07-18T03:05:23.941638+0000","last_peered":"2023-07-18T03:05:23.941638+0000","last_clean":"2023-07-18T03:05:23.941638+0000","last_became_active":"2023-07-18T03:05:03.880723+0000","last_became_peered":"2023-07-18T03:05:03.880723+0000","last_unstale":"2023-07-18T03:05:23.941638+0000","last_undegraded":"2023-07-18T03:05:23.941638+0000","last_fullsized":"2023-07-18T03:05:23.941638+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:10:01.286537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:29.984442+0000","last_change":"2023-07-18T03:04:58.847118+0000","last_active":"2023-07-18T03:05:29.984442+0000","last_peered":"2023-07-18T03:05:29.984442+0000","last_clean":"2023-07-18T03:05:29.984442+0000","last_became_active":"2023-07-18T03:04:58.846805+0000","last_became_peered":"2023-07-18T03:04:58.846805+0000","last_unstale":"2023-07-18T03:05:29.984442+0000","last_undegraded":"2023-07-18T03:05:29.984442+0000","last_fullsized":"2023-07-18T03:05:29.984442+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:43:05.061239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.933840+0000","last_change":"2023-07-18T03:04:59.803914+0000","last_active":"2023-07-18T03:05:23.933840+0000","last_peered":"2023-07-18T03:05:23.933840+0000","last_clean":"2023-07-18T03:05:23.933840+0000","last_became_active":"2023-07-18T03:04:59.803589+0000","last_became_peered":"2023-07-18T03:04:59.803589+0000","last_unstale":"2023-07-18T03:05:23.933840+0000","last_undegraded":"2023-07-18T03:05:23.933840+0000","last_fullsized":"2023-07-18T03:05:23.933840+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:56:07.440054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.934482+0000","last_change":"2023-07-18T03:05:03.877787+0000","last_active":"2023-07-18T03:05:23.934482+0000","last_peered":"2023-07-18T03:05:23.934482+0000","last_clean":"2023-07-18T03:05:23.934482+0000","last_became_active":"2023-07-18T03:05:03.877449+0000","last_became_peered":"2023-07-18T03:05:03.877449+0000","last_unstale":"2023-07-18T03:05:23.934482+0000","last_undegraded":"2023-07-18T03:05:23.934482+0000","last_fullsized":"2023-07-18T03:05:23.934482+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:25:39.108283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:31.996545+0000","last_change":"2023-07-18T03:04:58.844252+0000","last_active":"2023-07-18T03:05:31.996545+0000","last_peered":"2023-07-18T03:05:31.996545+0000","last_clean":"2023-07-18T03:05:31.996545+0000","last_became_active":"2023-07-18T03:04:58.843798+0000","last_became_peered":"2023-07-18T03:04:58.843798+0000","last_unstale":"2023-07-18T03:05:31.996545+0000","last_undegraded":"2023-07-18T03:05:31.996545+0000","last_fullsized":"2023-07-18T03:05:31.996545+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:54:33.788777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.933536+0000","last_change":"2023-07-18T03:04:59.806274+0000","last_active":"2023-07-18T03:05:23.933536+0000","last_peered":"2023-07-18T03:05:23.933536+0000","last_clean":"2023-07-18T03:05:23.933536+0000","last_became_active":"2023-07-18T03:04:59.805876+0000","last_became_peered":"2023-07-18T03:04:59.805876+0000","last_unstale":"2023-07-18T03:05:23.933536+0000","last_undegraded":"2023-07-18T03:05:23.933536+0000","last_fullsized":"2023-07-18T03:05:23.933536+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:38:39.397977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.091503+0000","last_change":"2023-07-18T03:05:03.876783+0000","last_active":"2023-07-18T03:05:24.091503+0000","last_peered":"2023-07-18T03:05:24.091503+0000","last_clean":"2023-07-18T03:05:24.091503+0000","last_became_active":"2023-07-18T03:05:03.876385+0000","last_became_peered":"2023-07-18T03:05:03.876385+0000","last_unstale":"2023-07-18T03:05:24.091503+0000","last_undegraded":"2023-07-18T03:05:24.091503+0000","last_fullsized":"2023-07-18T03:05:24.091503+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:20:54.077330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:30.992923+0000","last_change":"2023-07-18T03:04:58.851107+0000","last_active":"2023-07-18T03:05:30.992923+0000","last_peered":"2023-07-18T03:05:30.992923+0000","last_clean":"2023-07-18T03:05:30.992923+0000","last_became_active":"2023-07-18T03:04:58.850848+0000","last_became_peered":"2023-07-18T03:04:58.850848+0000","last_unstale":"2023-07-18T03:05:30.992923+0000","last_undegraded":"2023-07-18T03:05:30.992923+0000","last_fullsized":"2023-07-18T03:05:30.992923+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:17:37.985831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089918+0000","last_change":"2023-07-18T03:04:59.803683+0000","last_active":"2023-07-18T03:05:24.089918+0000","last_peered":"2023-07-18T03:05:24.089918+0000","last_clean":"2023-07-18T03:05:24.089918+0000","last_became_active":"2023-07-18T03:04:59.803365+0000","last_became_peered":"2023-07-18T03:04:59.803365+0000","last_unstale":"2023-07-18T03:05:24.089918+0000","last_undegraded":"2023-07-18T03:05:24.089918+0000","last_fullsized":"2023-07-18T03:05:24.089918+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:36:27.161561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089944+0000","last_change":"2023-07-18T03:05:03.875856+0000","last_active":"2023-07-18T03:05:24.089944+0000","last_peered":"2023-07-18T03:05:24.089944+0000","last_clean":"2023-07-18T03:05:24.089944+0000","last_became_active":"2023-07-18T03:05:03.875426+0000","last_became_peered":"2023-07-18T03:05:03.875426+0000","last_unstale":"2023-07-18T03:05:24.089944+0000","last_undegraded":"2023-07-18T03:05:24.089944+0000","last_fullsized":"2023-07-18T03:05:24.089944+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:00:07.610976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'1","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.938091+0000","last_change":"2023-07-18T03:04:58.844102+0000","last_active":"2023-07-18T03:05:23.938091+0000","last_peered":"2023-07-18T03:05:23.938091+0000","last_clean":"2023-07-18T03:05:23.938091+0000","last_became_active":"2023-07-18T03:04:58.843740+0000","last_became_peered":"2023-07-18T03:04:58.843740+0000","last_unstale":"2023-07-18T03:05:23.938091+0000","last_undegraded":"2023-07-18T03:05:23.938091+0000","last_fullsized":"2023-07-18T03:05:23.938091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:28:24.928761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.091161+0000","last_change":"2023-07-18T03:05:24.091161+0000","last_active":"2023-07-18T03:05:24.091161+0000","last_peered":"2023-07-18T03:05:24.091161+0000","last_clean":"2023-07-18T03:05:24.091161+0000","last_became_active":"2023-07-18T03:05:24.090631+0000","last_became_peered":"2023-07-18T03:05:24.090631+0000","last_unstale":"2023-07-18T03:05:24.091161+0000","last_undegraded":"2023-07-18T03:05:24.091161+0000","last_fullsized":"2023-07-18T03:05:24.091161+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:35:29.133177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.092328+0000","last_change":"2023-07-18T03:05:24.092328+0000","last_active":"2023-07-18T03:05:24.092328+0000","last_peered":"2023-07-18T03:05:24.092328+0000","last_clean":"2023-07-18T03:05:24.092328+0000","last_became_active":"2023-07-18T03:05:24.091839+0000","last_became_peered":"2023-07-18T03:05:24.091839+0000","last_unstale":"2023-07-18T03:05:24.092328+0000","last_undegraded":"2023-07-18T03:05:24.092328+0000","last_fullsized":"2023-07-18T03:05:24.092328+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:26:09.308228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.934182+0000","last_change":"2023-07-18T03:04:58.852395+0000","last_active":"2023-07-18T03:05:23.934182+0000","last_peered":"2023-07-18T03:05:23.934182+0000","last_clean":"2023-07-18T03:05:23.934182+0000","last_became_active":"2023-07-18T03:04:58.852063+0000","last_became_peered":"2023-07-18T03:04:58.852063+0000","last_unstale":"2023-07-18T03:05:23.934182+0000","last_undegraded":"2023-07-18T03:05:23.934182+0000","last_fullsized":"2023-07-18T03:05:23.934182+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:39:12.747039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.092486+0000","last_change":"2023-07-18T03:05:24.092486+0000","last_active":"2023-07-18T03:05:24.092486+0000","last_peered":"2023-07-18T03:05:24.092486+0000","last_clean":"2023-07-18T03:05:24.092486+0000","last_became_active":"2023-07-18T03:05:24.092189+0000","last_became_peered":"2023-07-18T03:05:24.092189+0000","last_unstale":"2023-07-18T03:05:24.092486+0000","last_undegraded":"2023-07-18T03:05:24.092486+0000","last_fullsized":"2023-07-18T03:05:24.092486+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:04:51.159197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.091135+0000","last_change":"2023-07-18T03:05:03.868851+0000","last_active":"2023-07-18T03:05:24.091135+0000","last_peered":"2023-07-18T03:05:24.091135+0000","last_clean":"2023-07-18T03:05:24.091135+0000","last_became_active":"2023-07-18T03:05:03.868433+0000","last_became_peered":"2023-07-18T03:05:03.868433+0000","last_unstale":"2023-07-18T03:05:24.091135+0000","last_undegraded":"2023-07-18T03:05:24.091135+0000","last_fullsized":"2023-07-18T03:05:24.091135+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:43:44.650042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.933677+0000","last_change":"2023-07-18T03:04:58.848059+0000","last_active":"2023-07-18T03:05:23.933677+0000","last_peered":"2023-07-18T03:05:23.933677+0000","last_clean":"2023-07-18T03:05:23.933677+0000","last_became_active":"2023-07-18T03:04:58.847700+0000","last_became_peered":"2023-07-18T03:04:58.847700+0000","last_unstale":"2023-07-18T03:05:23.933677+0000","last_undegraded":"2023-07-18T03:05:23.933677+0000","last_fullsized":"2023-07-18T03:05:23.933677+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:56:10.523441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.939939+0000","last_change":"2023-07-18T03:05:23.939939+0000","last_active":"2023-07-18T03:05:23.939939+0000","last_peered":"2023-07-18T03:05:23.939939+0000","last_clean":"2023-07-18T03:05:23.939939+0000","last_became_active":"2023-07-18T03:05:23.939607+0000","last_became_peered":"2023-07-18T03:05:23.939607+0000","last_unstale":"2023-07-18T03:05:23.939939+0000","last_undegraded":"2023-07-18T03:05:23.939939+0000","last_fullsized":"2023-07-18T03:05:23.939939+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:05:30.317897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.091495+0000","last_change":"2023-07-18T03:05:03.886724+0000","last_active":"2023-07-18T03:05:24.091495+0000","last_peered":"2023-07-18T03:05:24.091495+0000","last_clean":"2023-07-18T03:05:24.091495+0000","last_became_active":"2023-07-18T03:05:03.886172+0000","last_became_peered":"2023-07-18T03:05:03.886172+0000","last_unstale":"2023-07-18T03:05:24.091495+0000","last_undegraded":"2023-07-18T03:05:24.091495+0000","last_fullsized":"2023-07-18T03:05:24.091495+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:42:53.119172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089781+0000","last_change":"2023-07-18T03:04:59.804273+0000","last_active":"2023-07-18T03:05:24.089781+0000","last_peered":"2023-07-18T03:05:24.089781+0000","last_clean":"2023-07-18T03:05:24.089781+0000","last_became_active":"2023-07-18T03:04:59.803987+0000","last_became_peered":"2023-07-18T03:04:59.803987+0000","last_unstale":"2023-07-18T03:05:24.089781+0000","last_undegraded":"2023-07-18T03:05:24.089781+0000","last_fullsized":"2023-07-18T03:05:24.089781+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:47:40.524108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:30.992483+0000","last_change":"2023-07-18T03:04:58.846381+0000","last_active":"2023-07-18T03:05:30.992483+0000","last_peered":"2023-07-18T03:05:30.992483+0000","last_clean":"2023-07-18T03:05:30.992483+0000","last_became_active":"2023-07-18T03:04:58.845757+0000","last_became_peered":"2023-07-18T03:04:58.845757+0000","last_unstale":"2023-07-18T03:05:30.992483+0000","last_undegraded":"2023-07-18T03:05:30.992483+0000","last_fullsized":"2023-07-18T03:05:30.992483+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:41:53.865792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.933974+0000","last_change":"2023-07-18T03:05:03.873626+0000","last_active":"2023-07-18T03:05:23.933974+0000","last_peered":"2023-07-18T03:05:23.933974+0000","last_clean":"2023-07-18T03:05:23.933974+0000","last_became_active":"2023-07-18T03:05:03.872965+0000","last_became_peered":"2023-07-18T03:05:03.872965+0000","last_unstale":"2023-07-18T03:05:23.933974+0000","last_undegraded":"2023-07-18T03:05:23.933974+0000","last_fullsized":"2023-07-18T03:05:23.933974+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:11:15.362171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089960+0000","last_change":"2023-07-18T03:04:59.801153+0000","last_active":"2023-07-18T03:05:24.089960+0000","last_peered":"2023-07-18T03:05:24.089960+0000","last_clean":"2023-07-18T03:05:24.089960+0000","last_became_active":"2023-07-18T03:04:59.800798+0000","last_became_peered":"2023-07-18T03:04:59.800798+0000","last_unstale":"2023-07-18T03:05:24.089960+0000","last_undegraded":"2023-07-18T03:05:24.089960+0000","last_fullsized":"2023-07-18T03:05:24.089960+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:55:12.782843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:31.992249+0000","last_change":"2023-07-18T03:04:58.846047+0000","last_active":"2023-07-18T03:05:31.992249+0000","last_peered":"2023-07-18T03:05:31.992249+0000","last_clean":"2023-07-18T03:05:31.992249+0000","last_became_active":"2023-07-18T03:04:58.845678+0000","last_became_peered":"2023-07-18T03:04:58.845678+0000","last_unstale":"2023-07-18T03:05:31.992249+0000","last_undegraded":"2023-07-18T03:05:31.992249+0000","last_fullsized":"2023-07-18T03:05:31.992249+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:18:59.010410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":380,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.933385+0000","last_change":"2023-07-18T03:03:54.168837+0000","last_active":"2023-07-18T03:05:23.933385+0000","last_peered":"2023-07-18T03:05:23.933385+0000","last_clean":"2023-07-18T03:05:23.933385+0000","last_became_active":"2023-07-18T03:03:54.168298+0000","last_became_peered":"2023-07-18T03:03:54.168298+0000","last_unstale":"2023-07-18T03:05:23.933385+0000","last_undegraded":"2023-07-18T03:05:23.933385+0000","last_fullsized":"2023-07-18T03:05:23.933385+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:00:43.469168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:00:43.469168+0000","last_clean_scrub_stamp":"2023-07-18T03:00:43.469168+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:48:15.910084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089157+0000","last_change":"2023-07-18T03:05:03.886611+0000","last_active":"2023-07-18T03:05:24.089157+0000","last_peered":"2023-07-18T03:05:24.089157+0000","last_clean":"2023-07-18T03:05:24.089157+0000","last_became_active":"2023-07-18T03:05:03.886099+0000","last_became_peered":"2023-07-18T03:05:03.886099+0000","last_unstale":"2023-07-18T03:05:24.089157+0000","last_undegraded":"2023-07-18T03:05:24.089157+0000","last_fullsized":"2023-07-18T03:05:24.089157+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:22:16.652890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.090718+0000","last_change":"2023-07-18T03:04:59.804478+0000","last_active":"2023-07-18T03:05:24.090718+0000","last_peered":"2023-07-18T03:05:24.090718+0000","last_clean":"2023-07-18T03:05:24.090718+0000","last_became_active":"2023-07-18T03:04:59.804162+0000","last_became_peered":"2023-07-18T03:04:59.804162+0000","last_unstale":"2023-07-18T03:05:24.090718+0000","last_undegraded":"2023-07-18T03:05:24.090718+0000","last_fullsized":"2023-07-18T03:05:24.090718+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:30:38.735612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:30.990930+0000","last_change":"2023-07-18T03:04:58.847681+0000","last_active":"2023-07-18T03:05:30.990930+0000","last_peered":"2023-07-18T03:05:30.990930+0000","last_clean":"2023-07-18T03:05:30.990930+0000","last_became_active":"2023-07-18T03:04:58.847385+0000","last_became_peered":"2023-07-18T03:04:58.847385+0000","last_unstale":"2023-07-18T03:05:30.990930+0000","last_undegraded":"2023-07-18T03:05:30.990930+0000","last_fullsized":"2023-07-18T03:05:30.990930+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:16:57.492194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.941642+0000","last_change":"2023-07-18T03:05:03.872788+0000","last_active":"2023-07-18T03:05:23.941642+0000","last_peered":"2023-07-18T03:05:23.941642+0000","last_clean":"2023-07-18T03:05:23.941642+0000","last_became_active":"2023-07-18T03:05:03.872343+0000","last_became_peered":"2023-07-18T03:05:03.872343+0000","last_unstale":"2023-07-18T03:05:23.941642+0000","last_undegraded":"2023-07-18T03:05:23.941642+0000","last_fullsized":"2023-07-18T03:05:23.941642+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:05:33.222286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:29.985672+0000","last_change":"2023-07-18T03:04:58.855261+0000","last_active":"2023-07-18T03:05:29.985672+0000","last_peered":"2023-07-18T03:05:29.985672+0000","last_clean":"2023-07-18T03:05:29.985672+0000","last_became_active":"2023-07-18T03:04:58.854921+0000","last_became_peered":"2023-07-18T03:04:58.854921+0000","last_unstale":"2023-07-18T03:05:29.985672+0000","last_undegraded":"2023-07-18T03:05:29.985672+0000","last_fullsized":"2023-07-18T03:05:29.985672+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:08:26.349218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.091711+0000","last_change":"2023-07-18T03:04:59.804706+0000","last_active":"2023-07-18T03:05:24.091711+0000","last_peered":"2023-07-18T03:05:24.091711+0000","last_clean":"2023-07-18T03:05:24.091711+0000","last_became_active":"2023-07-18T03:04:59.804445+0000","last_became_peered":"2023-07-18T03:04:59.804445+0000","last_unstale":"2023-07-18T03:05:24.091711+0000","last_undegraded":"2023-07-18T03:05:24.091711+0000","last_fullsized":"2023-07-18T03:05:24.091711+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:06:24.461712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089557+0000","last_change":"2023-07-18T03:05:03.886065+0000","last_active":"2023-07-18T03:05:24.089557+0000","last_peered":"2023-07-18T03:05:24.089557+0000","last_clean":"2023-07-18T03:05:24.089557+0000","last_became_active":"2023-07-18T03:05:03.885625+0000","last_became_peered":"2023-07-18T03:05:03.885625+0000","last_unstale":"2023-07-18T03:05:24.089557+0000","last_undegraded":"2023-07-18T03:05:24.089557+0000","last_fullsized":"2023-07-18T03:05:24.089557+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:35:20.557825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'1","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.934410+0000","last_change":"2023-07-18T03:04:58.849246+0000","last_active":"2023-07-18T03:05:23.934410+0000","last_peered":"2023-07-18T03:05:23.934410+0000","last_clean":"2023-07-18T03:05:23.934410+0000","last_became_active":"2023-07-18T03:04:58.848888+0000","last_became_peered":"2023-07-18T03:04:58.848888+0000","last_unstale":"2023-07-18T03:05:23.934410+0000","last_undegraded":"2023-07-18T03:05:23.934410+0000","last_fullsized":"2023-07-18T03:05:23.934410+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:54:11.818147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.090505+0000","last_change":"2023-07-18T03:04:59.804561+0000","last_active":"2023-07-18T03:05:24.090505+0000","last_peered":"2023-07-18T03:05:24.090505+0000","last_clean":"2023-07-18T03:05:24.090505+0000","last_became_active":"2023-07-18T03:04:59.803659+0000","last_became_peered":"2023-07-18T03:04:59.803659+0000","last_unstale":"2023-07-18T03:05:24.090505+0000","last_undegraded":"2023-07-18T03:05:24.090505+0000","last_fullsized":"2023-07-18T03:05:24.090505+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:59:59.312017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.937606+0000","last_change":"2023-07-18T03:05:03.871418+0000","last_active":"2023-07-18T03:05:23.937606+0000","last_peered":"2023-07-18T03:05:23.937606+0000","last_clean":"2023-07-18T03:05:23.937606+0000","last_became_active":"2023-07-18T03:05:03.871054+0000","last_became_peered":"2023-07-18T03:05:03.871054+0000","last_unstale":"2023-07-18T03:05:23.937606+0000","last_undegraded":"2023-07-18T03:05:23.937606+0000","last_fullsized":"2023-07-18T03:05:23.937606+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:13:12.030292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:29.986067+0000","last_change":"2023-07-18T03:04:58.850505+0000","last_active":"2023-07-18T03:05:29.986067+0000","last_peered":"2023-07-18T03:05:29.986067+0000","last_clean":"2023-07-18T03:05:29.986067+0000","last_became_active":"2023-07-18T03:04:58.850234+0000","last_became_peered":"2023-07-18T03:04:58.850234+0000","last_unstale":"2023-07-18T03:05:29.986067+0000","last_undegraded":"2023-07-18T03:05:29.986067+0000","last_fullsized":"2023-07-18T03:05:29.986067+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:55:02.804220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.938190+0000","last_change":"2023-07-18T03:04:59.807348+0000","last_active":"2023-07-18T03:05:23.938190+0000","last_peered":"2023-07-18T03:05:23.938190+0000","last_clean":"2023-07-18T03:05:23.938190+0000","last_became_active":"2023-07-18T03:04:59.807013+0000","last_became_peered":"2023-07-18T03:04:59.807013+0000","last_unstale":"2023-07-18T03:05:23.938190+0000","last_undegraded":"2023-07-18T03:05:23.938190+0000","last_fullsized":"2023-07-18T03:05:23.938190+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:39:56.776375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089321+0000","last_change":"2023-07-18T03:05:03.886773+0000","last_active":"2023-07-18T03:05:24.089321+0000","last_peered":"2023-07-18T03:05:24.089321+0000","last_clean":"2023-07-18T03:05:24.089321+0000","last_became_active":"2023-07-18T03:05:03.886259+0000","last_became_peered":"2023-07-18T03:05:03.886259+0000","last_unstale":"2023-07-18T03:05:24.089321+0000","last_undegraded":"2023-07-18T03:05:24.089321+0000","last_fullsized":"2023-07-18T03:05:24.089321+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:04:49.928789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:29.985267+0000","last_change":"2023-07-18T03:04:58.855746+0000","last_active":"2023-07-18T03:05:29.985267+0000","last_peered":"2023-07-18T03:05:29.985267+0000","last_clean":"2023-07-18T03:05:29.985267+0000","last_became_active":"2023-07-18T03:04:58.855366+0000","last_became_peered":"2023-07-18T03:04:58.855366+0000","last_unstale":"2023-07-18T03:05:29.985267+0000","last_undegraded":"2023-07-18T03:05:29.985267+0000","last_fullsized":"2023-07-18T03:05:29.985267+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:52:56.120702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.936549+0000","last_change":"2023-07-18T03:04:59.800321+0000","last_active":"2023-07-18T03:05:23.936549+0000","last_peered":"2023-07-18T03:05:23.936549+0000","last_clean":"2023-07-18T03:05:23.936549+0000","last_became_active":"2023-07-18T03:04:59.800018+0000","last_became_peered":"2023-07-18T03:04:59.800018+0000","last_unstale":"2023-07-18T03:05:23.936549+0000","last_undegraded":"2023-07-18T03:05:23.936549+0000","last_fullsized":"2023-07-18T03:05:23.936549+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:12:47.433577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.934504+0000","last_change":"2023-07-18T03:05:03.877541+0000","last_active":"2023-07-18T03:05:23.934504+0000","last_peered":"2023-07-18T03:05:23.934504+0000","last_clean":"2023-07-18T03:05:23.934504+0000","last_became_active":"2023-07-18T03:05:03.877164+0000","last_became_peered":"2023-07-18T03:05:03.877164+0000","last_unstale":"2023-07-18T03:05:23.934504+0000","last_undegraded":"2023-07-18T03:05:23.934504+0000","last_fullsized":"2023-07-18T03:05:23.934504+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:52:54.205495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:29.985557+0000","last_change":"2023-07-18T03:04:58.845643+0000","last_active":"2023-07-18T03:05:29.985557+0000","last_peered":"2023-07-18T03:05:29.985557+0000","last_clean":"2023-07-18T03:05:29.985557+0000","last_became_active":"2023-07-18T03:04:58.845254+0000","last_became_peered":"2023-07-18T03:04:58.845254+0000","last_unstale":"2023-07-18T03:05:29.985557+0000","last_undegraded":"2023-07-18T03:05:29.985557+0000","last_fullsized":"2023-07-18T03:05:29.985557+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:42:32.740369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.941481+0000","last_change":"2023-07-18T03:04:59.803679+0000","last_active":"2023-07-18T03:05:23.941481+0000","last_peered":"2023-07-18T03:05:23.941481+0000","last_clean":"2023-07-18T03:05:23.941481+0000","last_became_active":"2023-07-18T03:04:59.802841+0000","last_became_peered":"2023-07-18T03:04:59.802841+0000","last_unstale":"2023-07-18T03:05:23.941481+0000","last_undegraded":"2023-07-18T03:05:23.941481+0000","last_fullsized":"2023-07-18T03:05:23.941481+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:14:53.042557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089419+0000","last_change":"2023-07-18T03:05:03.874093+0000","last_active":"2023-07-18T03:05:24.089419+0000","last_peered":"2023-07-18T03:05:24.089419+0000","last_clean":"2023-07-18T03:05:24.089419+0000","last_became_active":"2023-07-18T03:05:03.873710+0000","last_became_peered":"2023-07-18T03:05:03.873710+0000","last_unstale":"2023-07-18T03:05:24.089419+0000","last_undegraded":"2023-07-18T03:05:24.089419+0000","last_fullsized":"2023-07-18T03:05:24.089419+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:19:17.794343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.937140+0000","last_change":"2023-07-18T03:04:58.846686+0000","last_active":"2023-07-18T03:05:23.937140+0000","last_peered":"2023-07-18T03:05:23.937140+0000","last_clean":"2023-07-18T03:05:23.937140+0000","last_became_active":"2023-07-18T03:04:58.846371+0000","last_became_peered":"2023-07-18T03:04:58.846371+0000","last_unstale":"2023-07-18T03:05:23.937140+0000","last_undegraded":"2023-07-18T03:05:23.937140+0000","last_fullsized":"2023-07-18T03:05:23.937140+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:43:54.054638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089999+0000","last_change":"2023-07-18T03:04:59.811917+0000","last_active":"2023-07-18T03:05:24.089999+0000","last_peered":"2023-07-18T03:05:24.089999+0000","last_clean":"2023-07-18T03:05:24.089999+0000","last_became_active":"2023-07-18T03:04:59.811551+0000","last_became_peered":"2023-07-18T03:04:59.811551+0000","last_unstale":"2023-07-18T03:05:24.089999+0000","last_undegraded":"2023-07-18T03:05:24.089999+0000","last_fullsized":"2023-07-18T03:05:24.089999+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:32:42.153516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089397+0000","last_change":"2023-07-18T03:05:03.886072+0000","last_active":"2023-07-18T03:05:24.089397+0000","last_peered":"2023-07-18T03:05:24.089397+0000","last_clean":"2023-07-18T03:05:24.089397+0000","last_became_active":"2023-07-18T03:05:03.885652+0000","last_became_peered":"2023-07-18T03:05:03.885652+0000","last_unstale":"2023-07-18T03:05:24.089397+0000","last_undegraded":"2023-07-18T03:05:24.089397+0000","last_fullsized":"2023-07-18T03:05:24.089397+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:26:47.552178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:29.985186+0000","last_change":"2023-07-18T03:04:58.849069+0000","last_active":"2023-07-18T03:05:29.985186+0000","last_peered":"2023-07-18T03:05:29.985186+0000","last_clean":"2023-07-18T03:05:29.985186+0000","last_became_active":"2023-07-18T03:04:58.848744+0000","last_became_peered":"2023-07-18T03:04:58.848744+0000","last_unstale":"2023-07-18T03:05:29.985186+0000","last_undegraded":"2023-07-18T03:05:29.985186+0000","last_fullsized":"2023-07-18T03:05:29.985186+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:06:14.618475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.933710+0000","last_change":"2023-07-18T03:04:59.806976+0000","last_active":"2023-07-18T03:05:23.933710+0000","last_peered":"2023-07-18T03:05:23.933710+0000","last_clean":"2023-07-18T03:05:23.933710+0000","last_became_active":"2023-07-18T03:04:59.805750+0000","last_became_peered":"2023-07-18T03:04:59.805750+0000","last_unstale":"2023-07-18T03:05:23.933710+0000","last_undegraded":"2023-07-18T03:05:23.933710+0000","last_fullsized":"2023-07-18T03:05:23.933710+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:17:24.414508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.090382+0000","last_change":"2023-07-18T03:05:03.871210+0000","last_active":"2023-07-18T03:05:24.090382+0000","last_peered":"2023-07-18T03:05:24.090382+0000","last_clean":"2023-07-18T03:05:24.090382+0000","last_became_active":"2023-07-18T03:05:03.870826+0000","last_became_peered":"2023-07-18T03:05:03.870826+0000","last_unstale":"2023-07-18T03:05:24.090382+0000","last_undegraded":"2023-07-18T03:05:24.090382+0000","last_fullsized":"2023-07-18T03:05:24.090382+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:54:35.402542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:30.990936+0000","last_change":"2023-07-18T03:04:58.855673+0000","last_active":"2023-07-18T03:05:30.990936+0000","last_peered":"2023-07-18T03:05:30.990936+0000","last_clean":"2023-07-18T03:05:30.990936+0000","last_became_active":"2023-07-18T03:04:58.855338+0000","last_became_peered":"2023-07-18T03:04:58.855338+0000","last_unstale":"2023-07-18T03:05:30.990936+0000","last_undegraded":"2023-07-18T03:05:30.990936+0000","last_fullsized":"2023-07-18T03:05:30.990936+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:22:42.768582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.934248+0000","last_change":"2023-07-18T03:04:59.802488+0000","last_active":"2023-07-18T03:05:23.934248+0000","last_peered":"2023-07-18T03:05:23.934248+0000","last_clean":"2023-07-18T03:05:23.934248+0000","last_became_active":"2023-07-18T03:04:59.802184+0000","last_became_peered":"2023-07-18T03:04:59.802184+0000","last_unstale":"2023-07-18T03:05:23.934248+0000","last_undegraded":"2023-07-18T03:05:23.934248+0000","last_fullsized":"2023-07-18T03:05:23.934248+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:48:27.222393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.937967+0000","last_change":"2023-07-18T03:05:03.881585+0000","last_active":"2023-07-18T03:05:23.937967+0000","last_peered":"2023-07-18T03:05:23.937967+0000","last_clean":"2023-07-18T03:05:23.937967+0000","last_became_active":"2023-07-18T03:05:03.880917+0000","last_became_peered":"2023-07-18T03:05:03.880917+0000","last_unstale":"2023-07-18T03:05:23.937967+0000","last_undegraded":"2023-07-18T03:05:23.937967+0000","last_fullsized":"2023-07-18T03:05:23.937967+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:21:55.495075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089704+0000","last_change":"2023-07-18T03:04:58.852387+0000","last_active":"2023-07-18T03:05:24.089704+0000","last_peered":"2023-07-18T03:05:24.089704+0000","last_clean":"2023-07-18T03:05:24.089704+0000","last_became_active":"2023-07-18T03:04:58.852113+0000","last_became_peered":"2023-07-18T03:04:58.852113+0000","last_unstale":"2023-07-18T03:05:24.089704+0000","last_undegraded":"2023-07-18T03:05:24.089704+0000","last_fullsized":"2023-07-18T03:05:24.089704+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:47:27.148793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.933886+0000","last_change":"2023-07-18T03:04:59.811569+0000","last_active":"2023-07-18T03:05:23.933886+0000","last_peered":"2023-07-18T03:05:23.933886+0000","last_clean":"2023-07-18T03:05:23.933886+0000","last_became_active":"2023-07-18T03:04:59.811229+0000","last_became_peered":"2023-07-18T03:04:59.811229+0000","last_unstale":"2023-07-18T03:05:23.933886+0000","last_undegraded":"2023-07-18T03:05:23.933886+0000","last_fullsized":"2023-07-18T03:05:23.933886+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:53:58.785873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.088825+0000","last_change":"2023-07-18T03:05:03.872530+0000","last_active":"2023-07-18T03:05:24.088825+0000","last_peered":"2023-07-18T03:05:24.088825+0000","last_clean":"2023-07-18T03:05:24.088825+0000","last_became_active":"2023-07-18T03:05:03.871607+0000","last_became_peered":"2023-07-18T03:05:03.871607+0000","last_unstale":"2023-07-18T03:05:24.088825+0000","last_undegraded":"2023-07-18T03:05:24.088825+0000","last_fullsized":"2023-07-18T03:05:24.088825+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:34:34.583319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.091627+0000","last_change":"2023-07-18T03:04:58.847933+0000","last_active":"2023-07-18T03:05:24.091627+0000","last_peered":"2023-07-18T03:05:24.091627+0000","last_clean":"2023-07-18T03:05:24.091627+0000","last_became_active":"2023-07-18T03:04:58.847630+0000","last_became_peered":"2023-07-18T03:04:58.847630+0000","last_unstale":"2023-07-18T03:05:24.091627+0000","last_undegraded":"2023-07-18T03:05:24.091627+0000","last_fullsized":"2023-07-18T03:05:24.091627+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:35:14.343077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.934420+0000","last_change":"2023-07-18T03:04:59.803981+0000","last_active":"2023-07-18T03:05:23.934420+0000","last_peered":"2023-07-18T03:05:23.934420+0000","last_clean":"2023-07-18T03:05:23.934420+0000","last_became_active":"2023-07-18T03:04:59.803692+0000","last_became_peered":"2023-07-18T03:04:59.803692+0000","last_unstale":"2023-07-18T03:05:23.934420+0000","last_undegraded":"2023-07-18T03:05:23.934420+0000","last_fullsized":"2023-07-18T03:05:23.934420+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:52:39.970316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.941648+0000","last_change":"2023-07-18T03:04:59.803770+0000","last_active":"2023-07-18T03:05:23.941648+0000","last_peered":"2023-07-18T03:05:23.941648+0000","last_clean":"2023-07-18T03:05:23.941648+0000","last_became_active":"2023-07-18T03:04:59.803355+0000","last_became_peered":"2023-07-18T03:04:59.803355+0000","last_unstale":"2023-07-18T03:05:23.941648+0000","last_undegraded":"2023-07-18T03:05:23.941648+0000","last_fullsized":"2023-07-18T03:05:23.941648+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:58:39.773442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.937874+0000","last_change":"2023-07-18T03:04:59.808946+0000","last_active":"2023-07-18T03:05:23.937874+0000","last_peered":"2023-07-18T03:05:23.937874+0000","last_clean":"2023-07-18T03:05:23.937874+0000","last_became_active":"2023-07-18T03:04:59.808646+0000","last_became_peered":"2023-07-18T03:04:59.808646+0000","last_unstale":"2023-07-18T03:05:23.937874+0000","last_undegraded":"2023-07-18T03:05:23.937874+0000","last_fullsized":"2023-07-18T03:05:23.937874+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:30:10.702978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.936997+0000","last_change":"2023-07-18T03:04:59.811933+0000","last_active":"2023-07-18T03:05:23.936997+0000","last_peered":"2023-07-18T03:05:23.936997+0000","last_clean":"2023-07-18T03:05:23.936997+0000","last_became_active":"2023-07-18T03:04:59.811620+0000","last_became_peered":"2023-07-18T03:04:59.811620+0000","last_unstale":"2023-07-18T03:05:23.936997+0000","last_undegraded":"2023-07-18T03:05:23.936997+0000","last_fullsized":"2023-07-18T03:05:23.936997+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:14:30.959426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.938599+0000","last_change":"2023-07-18T03:04:59.803942+0000","last_active":"2023-07-18T03:05:23.938599+0000","last_peered":"2023-07-18T03:05:23.938599+0000","last_clean":"2023-07-18T03:05:23.938599+0000","last_became_active":"2023-07-18T03:04:59.803576+0000","last_became_peered":"2023-07-18T03:04:59.803576+0000","last_unstale":"2023-07-18T03:05:23.938599+0000","last_undegraded":"2023-07-18T03:05:23.938599+0000","last_fullsized":"2023-07-18T03:05:23.938599+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:24:31.974566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.091559+0000","last_change":"2023-07-18T03:04:59.802594+0000","last_active":"2023-07-18T03:05:24.091559+0000","last_peered":"2023-07-18T03:05:24.091559+0000","last_clean":"2023-07-18T03:05:24.091559+0000","last_became_active":"2023-07-18T03:04:59.802215+0000","last_became_peered":"2023-07-18T03:04:59.802215+0000","last_unstale":"2023-07-18T03:05:24.091559+0000","last_undegraded":"2023-07-18T03:05:24.091559+0000","last_fullsized":"2023-07-18T03:05:24.091559+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:18:26.515762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.942105+0000","last_change":"2023-07-18T03:04:59.803519+0000","last_active":"2023-07-18T03:05:23.942105+0000","last_peered":"2023-07-18T03:05:23.942105+0000","last_clean":"2023-07-18T03:05:23.942105+0000","last_became_active":"2023-07-18T03:04:59.803170+0000","last_became_peered":"2023-07-18T03:04:59.803170+0000","last_unstale":"2023-07-18T03:05:23.942105+0000","last_undegraded":"2023-07-18T03:05:23.942105+0000","last_fullsized":"2023-07-18T03:05:23.942105+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:57:22.069668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.938577+0000","last_change":"2023-07-18T03:04:59.813494+0000","last_active":"2023-07-18T03:05:23.938577+0000","last_peered":"2023-07-18T03:05:23.938577+0000","last_clean":"2023-07-18T03:05:23.938577+0000","last_became_active":"2023-07-18T03:04:59.813175+0000","last_became_peered":"2023-07-18T03:04:59.813175+0000","last_unstale":"2023-07-18T03:05:23.938577+0000","last_undegraded":"2023-07-18T03:05:23.938577+0000","last_fullsized":"2023-07-18T03:05:23.938577+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:13:17.098239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.090682+0000","last_change":"2023-07-18T03:04:59.806597+0000","last_active":"2023-07-18T03:05:24.090682+0000","last_peered":"2023-07-18T03:05:24.090682+0000","last_clean":"2023-07-18T03:05:24.090682+0000","last_became_active":"2023-07-18T03:04:59.806220+0000","last_became_peered":"2023-07-18T03:04:59.806220+0000","last_unstale":"2023-07-18T03:05:24.090682+0000","last_undegraded":"2023-07-18T03:05:24.090682+0000","last_fullsized":"2023-07-18T03:05:24.090682+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:33:51.675056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089408+0000","last_change":"2023-07-18T03:04:59.809051+0000","last_active":"2023-07-18T03:05:24.089408+0000","last_peered":"2023-07-18T03:05:24.089408+0000","last_clean":"2023-07-18T03:05:24.089408+0000","last_became_active":"2023-07-18T03:04:59.808783+0000","last_became_peered":"2023-07-18T03:04:59.808783+0000","last_unstale":"2023-07-18T03:05:24.089408+0000","last_undegraded":"2023-07-18T03:05:24.089408+0000","last_fullsized":"2023-07-18T03:05:24.089408+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:11:53.631126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.090344+0000","last_change":"2023-07-18T03:04:59.805741+0000","last_active":"2023-07-18T03:05:24.090344+0000","last_peered":"2023-07-18T03:05:24.090344+0000","last_clean":"2023-07-18T03:05:24.090344+0000","last_became_active":"2023-07-18T03:04:59.805412+0000","last_became_peered":"2023-07-18T03:04:59.805412+0000","last_unstale":"2023-07-18T03:05:24.090344+0000","last_undegraded":"2023-07-18T03:05:24.090344+0000","last_fullsized":"2023-07-18T03:05:24.090344+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:36:12.237809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.090649+0000","last_change":"2023-07-18T03:04:59.812052+0000","last_active":"2023-07-18T03:05:24.090649+0000","last_peered":"2023-07-18T03:05:24.090649+0000","last_clean":"2023-07-18T03:05:24.090649+0000","last_became_active":"2023-07-18T03:04:59.811756+0000","last_became_peered":"2023-07-18T03:04:59.811756+0000","last_unstale":"2023-07-18T03:05:24.090649+0000","last_undegraded":"2023-07-18T03:05:24.090649+0000","last_fullsized":"2023-07-18T03:05:24.090649+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:28:03.874575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.091423+0000","last_change":"2023-07-18T03:04:59.803009+0000","last_active":"2023-07-18T03:05:24.091423+0000","last_peered":"2023-07-18T03:05:24.091423+0000","last_clean":"2023-07-18T03:05:24.091423+0000","last_became_active":"2023-07-18T03:04:59.802687+0000","last_became_peered":"2023-07-18T03:04:59.802687+0000","last_unstale":"2023-07-18T03:05:24.091423+0000","last_undegraded":"2023-07-18T03:05:24.091423+0000","last_fullsized":"2023-07-18T03:05:24.091423+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:28:34.815336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.942144+0000","last_change":"2023-07-18T03:04:59.795562+0000","last_active":"2023-07-18T03:05:23.942144+0000","last_peered":"2023-07-18T03:05:23.942144+0000","last_clean":"2023-07-18T03:05:23.942144+0000","last_became_active":"2023-07-18T03:04:59.795241+0000","last_became_peered":"2023-07-18T03:04:59.795241+0000","last_unstale":"2023-07-18T03:05:23.942144+0000","last_undegraded":"2023-07-18T03:05:23.942144+0000","last_fullsized":"2023-07-18T03:05:23.942144+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:43:39.736724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":12288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281146,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7308,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":15,"num_read_kb":13,"num_write":118,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":274432,"data_stored":57301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":103,"ondisk_log_size":103,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449106,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":483597,"data_compressed":17573,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:05:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.70299999999999996}]},{"osd":1,"last update":"Tue Jul 18 03:05:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.58699999999999997}]},{"osd":2,"last update":"Tue Jul 18 03:05:13 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.73499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":4,"last update":"Tue Jul 18 03:05:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.75900000000000001}]},{"osd":5,"last update":"Tue Jul 18 03:05:13 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.86199999999999999}]},{"osd":6,"last update":"Tue Jul 18 03:05:13 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.79800000000000004}]},{"osd":7,"last update":"Tue Jul 18 03:05:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.89100000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]},{"osd":9,"last update":"Tue Jul 18 03:05:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.877},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.90300000000000002}]},{"osd":10,"last update":"Tue Jul 18 03:05:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.94999999999999996}]}]},{"osd":10,"up_from":71,"seq":304942678039,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":1032,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":1056768,"data_stored":1078707,"data_compressed":53332,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":1,"last update":"Tue Jul 18 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.42899999999999999}]},{"osd":2,"last update":"Tue Jul 18 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.59499999999999997}]},{"osd":3,"last update":"Tue Jul 18 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.52300000000000002}]},{"osd":4,"last update":"Tue Jul 18 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.47099999999999997}]},{"osd":5,"last update":"Tue Jul 18 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.78300000000000003}]},{"osd":6,"last update":"Tue Jul 18 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.65600000000000003}]},{"osd":7,"last update":"Tue Jul 18 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.55800000000000005}]},{"osd":8,"last update":"Tue Jul 18 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.72299999999999998}]},{"osd":9,"last update":"Tue Jul 18 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.755}]},{"osd":11,"last update":"Tue Jul 18 03:05:19 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.73099999999999998}]}]},{"osd":9,"up_from":66,"seq":283467841562,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":483597,"data_compressed":17573,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:04:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.37},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.45000000000000001}]},{"osd":1,"last update":"Tue Jul 18 03:04:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.55100000000000005}]},{"osd":2,"last update":"Tue Jul 18 03:04:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.48499999999999999}]},{"osd":3,"last update":"Tue Jul 18 03:04:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.38700000000000001}]},{"osd":4,"last update":"Tue Jul 18 03:04:30 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.66100000000000003}]},{"osd":5,"last update":"Tue Jul 18 03:04:30 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.499}]},{"osd":6,"last update":"Tue Jul 18 03:04:30 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.64300000000000002}]},{"osd":7,"last update":"Tue Jul 18 03:04:30 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":0.70899999999999996}]},{"osd":8,"last update":"Tue Jul 18 03:04:30 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.69099999999999995}]},{"osd":10,"last update":"Tue Jul 18 03:04:51 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.8899999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.72999999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]}]},{"osd":8,"up_from":58,"seq":249108103199,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":983040,"data_stored":483753,"data_compressed":17573,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:05:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.626,"15min":0.61799999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":2.0990000000000002,"5min":2.0990000000000002,"15min":2.0990000000000002},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.58099999999999996,"15min":0.56999999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":2.883,"5min":2.883,"15min":2.883},"last":0.56799999999999995}]},{"osd":1,"last update":"Tue Jul 18 03:05:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.63900000000000001,"15min":0.627},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":3.3340000000000001,"5min":3.3340000000000001,"15min":3.3340000000000001},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.61099999999999999,"15min":0.59599999999999997},"min":{"1min":0.37,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":2.0510000000000002,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":0.81000000000000005}]},{"osd":2,"last update":"Tue Jul 18 03:05:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.48199999999999998,"15min":0.46000000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":3.2850000000000001,"5min":3.2850000000000001,"15min":3.2850000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.502,"15min":0.49199999999999999},"min":{"1min":0.309,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":0.54200000000000004}]},{"osd":3,"last update":"Tue Jul 18 03:05:10 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.628,"15min":0.61699999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":2.9100000000000001,"5min":2.9100000000000001,"15min":2.9100000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.66700000000000004,"15min":0.64900000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":3.6059999999999999,"5min":3.6059999999999999,"15min":3.6059999999999999},"last":0.65500000000000003}]},{"osd":4,"last update":"Tue Jul 18 03:05:10 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.64500000000000002,"15min":0.627},"min":{"1min":0.437,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.7879999999999998,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.70299999999999996,"15min":0.68999999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":3.3650000000000002,"5min":3.3650000000000002,"15min":3.3650000000000002},"last":0.71699999999999997}]},{"osd":5,"last update":"Tue Jul 18 03:05:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.60699999999999998,"15min":0.58999999999999997},"min":{"1min":0.39200000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":2.4279999999999999,"5min":2.4279999999999999,"15min":2.4279999999999999},"last":0.747},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.61199999999999999,"15min":0.59799999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":2.7410000000000001,"5min":2.7410000000000001,"15min":2.7410000000000001},"last":0.82899999999999996}]},{"osd":6,"last update":"Tue Jul 18 03:05:10 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.73899999999999999,"15min":0.71799999999999997},"min":{"1min":0.54200000000000004,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":3.5779999999999998,"5min":3.5779999999999998,"15min":3.5779999999999998},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.746,"15min":0.73499999999999999},"min":{"1min":0.502,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.1429999999999998,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":0.84699999999999998}]},{"osd":7,"last update":"Tue Jul 18 03:05:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.73199999999999998,"15min":0.71999999999999997},"min":{"1min":0.48999999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.73499999999999999,"15min":0.71899999999999997},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":2.8290000000000002,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":0.96299999999999997}]},{"osd":9,"last update":"Tue Jul 18 03:04:33 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.97599999999999998}]},{"osd":10,"last update":"Tue Jul 18 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":2.8570000000000002,"5min":2.8570000000000002,"15min":2.8570000000000002},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":3.3879999999999999,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":1.02}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]}]},{"osd":7,"up_from":51,"seq":219043332131,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":1032192,"data_stored":1079565,"data_compressed":53332,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.55500000000000005,"15min":0.54300000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.495},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59999999999999998,"15min":0.60299999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82599999999999996,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.44500000000000001}]},{"osd":1,"last update":"Tue Jul 18 03:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.42099999999999999,"15min":0.41899999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.436,"15min":0.432},"min":{"1min":0.31900000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.63500000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.374}]},{"osd":2,"last update":"Tue Jul 18 03:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.57799999999999996,"15min":0.56799999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.59199999999999997,"15min":0.58199999999999996},"min":{"1min":0.45500000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.63200000000000001}]},{"osd":3,"last update":"Tue Jul 18 03:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.60399999999999998,"15min":0.59599999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.435,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.91500000000000004,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.57899999999999996}]},{"osd":4,"last update":"Tue Jul 18 03:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.48399999999999999,"15min":0.48199999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.71599999999999997,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.55100000000000005,"15min":0.54900000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.45800000000000002}]},{"osd":5,"last update":"Tue Jul 18 03:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.66000000000000003,"15min":0.64700000000000002},"min":{"1min":0.53300000000000003,"5min":0.501,"15min":0.501},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.66500000000000004,"15min":0.65500000000000003},"min":{"1min":0.52000000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.60199999999999998}]},{"osd":6,"last update":"Tue Jul 18 03:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.67500000000000004,"15min":0.67400000000000004},"min":{"1min":0.46999999999999997,"5min":0.435,"15min":0.435},"max":{"1min":0.89600000000000002,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.752},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.70999999999999996,"15min":0.70099999999999996},"min":{"1min":0.497,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.55300000000000005}]},{"osd":8,"last update":"Tue Jul 18 03:05:06 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.69899999999999995,"15min":0.68899999999999995},"min":{"1min":0.48999999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.65400000000000003,"15min":0.64500000000000002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.71199999999999997}]},{"osd":9,"last update":"Tue Jul 18 03:05:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.76600000000000001,"15min":0.76400000000000001},"min":{"1min":0.56499999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.70099999999999996}]},{"osd":10,"last update":"Tue Jul 18 03:04:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.64700000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]}]},{"osd":6,"up_from":46,"seq":197568495656,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":962560,"data_stored":483015,"data_compressed":17573,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.53400000000000003,"15min":0.56299999999999994},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.58199999999999996,"5min":3.1579999999999999,"15min":3.1579999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.53400000000000003,"15min":0.55000000000000004},"min":{"1min":0.35699999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.64000000000000001,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.45600000000000002}]},{"osd":1,"last update":"Tue Jul 18 03:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.627,"15min":0.60299999999999998},"min":{"1min":0.38800000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.88600000000000001,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.72799999999999998,"15min":0.74099999999999999},"min":{"1min":0.495,"5min":0.44,"15min":0.44},"max":{"1min":0.90500000000000003,"5min":3.0369999999999999,"15min":3.0369999999999999},"last":0.878}]},{"osd":2,"last update":"Tue Jul 18 03:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.65700000000000003,"15min":0.65600000000000003},"min":{"1min":0.48299999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.64700000000000002,"15min":0.65300000000000002},"min":{"1min":0.48399999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.94299999999999995,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.70699999999999996}]},{"osd":3,"last update":"Tue Jul 18 03:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.56499999999999995,"15min":0.57299999999999995},"min":{"1min":0.38800000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.69899999999999995,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.71199999999999997,"15min":0.748},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.748,"5min":3.226,"15min":3.226},"last":0.68999999999999995}]},{"osd":4,"last update":"Tue Jul 18 03:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.44600000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.92500000000000004,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.70199999999999996,"15min":0.69899999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.92700000000000005,"5min":1.01,"15min":1.01},"last":0.84499999999999997}]},{"osd":5,"last update":"Tue Jul 18 03:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.81200000000000006,"15min":0.84499999999999997},"min":{"1min":0.59199999999999997,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.88600000000000001,"5min":3.125,"15min":3.125},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.76600000000000001,"15min":0.78200000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.98699999999999999,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.63200000000000001}]},{"osd":7,"last update":"Tue Jul 18 03:04:45 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.84099999999999997,"15min":0.84199999999999997},"min":{"1min":0.54500000000000004,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.2070000000000001,"5min":3.0870000000000002,"15min":3.0870000000000002},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.74099999999999999,"15min":0.72999999999999998},"min":{"1min":0.60099999999999998,"5min":0.505,"15min":0.505},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.90500000000000003}]},{"osd":8,"last update":"Tue Jul 18 03:05:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.871,"15min":0.88300000000000001},"min":{"1min":0.59599999999999997,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.103,"5min":3.1909999999999998,"15min":3.1909999999999998},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.54800000000000004,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.66700000000000004}]},{"osd":9,"last update":"Tue Jul 18 03:04:30 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.94299999999999995}]},{"osd":10,"last update":"Tue Jul 18 03:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.93200000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":974848,"data_stored":486809,"data_compressed":17573,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.55100000000000005,"15min":0.52400000000000002},"min":{"1min":0.37,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.96299999999999997,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.54400000000000004,"15min":0.52400000000000002},"min":{"1min":0.35299999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.93100000000000005,"5min":1.421,"15min":1.421},"last":0.442}]},{"osd":2,"last update":"Tue Jul 18 03:04:53 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.52400000000000002,"15min":0.505},"min":{"1min":0.38500000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.53900000000000003,"15min":0.53400000000000003},"min":{"1min":0.45700000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.56899999999999995}]},{"osd":3,"last update":"Tue Jul 18 03:05:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.61799999999999999,"15min":0.59799999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.393,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.62,"15min":0.625},"min":{"1min":0.432,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82899999999999996,"5min":1.504,"15min":1.504},"last":0.59399999999999997}]},{"osd":4,"last update":"Tue Jul 18 03:04:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.41799999999999998,"15min":0.39300000000000002},"min":{"1min":0.30099999999999999,"5min":0.215,"15min":0.215},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.441,"15min":0.41299999999999998},"min":{"1min":0.38,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.624}]},{"osd":5,"last update":"Tue Jul 18 03:05:00 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.56699999999999995,"15min":0.51600000000000001},"min":{"1min":0.40799999999999997,"5min":0.375,"15min":0.375},"max":{"1min":1.5129999999999999,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.58199999999999996,"15min":0.55400000000000005},"min":{"1min":0.46100000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.66300000000000003}]},{"osd":6,"last update":"Tue Jul 18 03:05:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.64900000000000002,"15min":0.60499999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.4299999999999999,"5min":2.6600000000000001,"15min":2.6600000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.65200000000000002,"15min":0.63800000000000001},"min":{"1min":0.46400000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.94799999999999995,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.70499999999999996}]},{"osd":7,"last update":"Tue Jul 18 03:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.57299999999999995,"15min":0.55600000000000005},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.87,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.65500000000000003,"15min":0.64900000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.89700000000000002,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":0.84099999999999997}]},{"osd":8,"last update":"Tue Jul 18 03:05:09 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.77800000000000002,"15min":0.78300000000000003},"min":{"1min":0.55800000000000005,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.486,"5min":2.6419999999999999,"15min":2.6419999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.69199999999999995,"15min":0.67200000000000004},"min":{"1min":0.52000000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.698,"5min":1.698,"15min":1.698},"last":0.78000000000000003}]},{"osd":9,"last update":"Tue Jul 18 03:04:34 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.6789999999999998,"5min":2.6789999999999998,"15min":2.6789999999999998},"last":0.871},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.754}]},{"osd":10,"last update":"Tue Jul 18 03:04:53 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.80600000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":991232,"data_stored":484493,"data_compressed":17573,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Jul 18 03:05:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.59599999999999997,"15min":0.55200000000000005},"min":{"1min":0.45900000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.0149999999999999,"5min":1.075,"15min":1.075},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.61099999999999999,"15min":0.56100000000000005},"min":{"1min":0.38100000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.97699999999999998,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.61299999999999999}]},{"osd":2,"last update":"Tue Jul 18 03:04:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.57899999999999996,"15min":0.54400000000000004},"min":{"1min":0.41499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.85799999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.61199999999999999,"15min":0.55800000000000005},"min":{"1min":0.39000000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.016,"5min":1.204,"15min":1.204},"last":0.56200000000000006}]},{"osd":3,"last update":"Tue Jul 18 03:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.47499999999999998,"15min":0.45700000000000002},"min":{"1min":0.313,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.498,"15min":0.47999999999999998},"min":{"1min":0.34699999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.35899999999999999}]},{"osd":4,"last update":"Tue Jul 18 03:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.63400000000000001,"15min":0.60599999999999998},"min":{"1min":0.47399999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.61099999999999999,"15min":0.59399999999999997},"min":{"1min":0.52800000000000002,"5min":0.433,"15min":0.433},"max":{"1min":1.0680000000000001,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.65800000000000003}]},{"osd":5,"last update":"Tue Jul 18 03:05:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.64700000000000002,"15min":0.63500000000000001},"min":{"1min":0.46899999999999997,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.028,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.68200000000000005,"15min":0.65900000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.998,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.79600000000000004}]},{"osd":6,"last update":"Tue Jul 18 03:05:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59899999999999998,"15min":0.58999999999999997},"min":{"1min":0.439,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.91900000000000004,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.624,"15min":0.59999999999999998},"min":{"1min":0.42099999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.94699999999999995,"5min":1.216,"15min":1.216},"last":0.46300000000000002}]},{"osd":7,"last update":"Tue Jul 18 03:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.77800000000000002,"15min":0.76900000000000002},"min":{"1min":0.52200000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.177,"5min":1.349,"15min":1.349},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.81200000000000006,"15min":0.81000000000000005},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.1539999999999999,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.74099999999999999}]},{"osd":8,"last update":"Tue Jul 18 03:05:04 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.747,"15min":0.746},"min":{"1min":0.63300000000000001,"5min":0.498,"15min":0.498},"max":{"1min":1.0629999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.622},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.82799999999999996,"15min":0.83299999999999996},"min":{"1min":0.56899999999999995,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.181,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.52000000000000002}]},{"osd":9,"last update":"Tue Jul 18 03:05:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.79000000000000004,"15min":0.79100000000000004},"min":{"1min":0.51400000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.107,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.871,"15min":0.877},"min":{"1min":0.52200000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.218,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.76100000000000001}]},{"osd":10,"last update":"Tue Jul 18 03:04:51 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.67300000000000004}]},{"osd":11,"last update":"Tue Jul 18 03:05:14 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.68400000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411389,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":1019904,"data_stored":505882,"data_compressed":17573,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.54100000000000004,"15min":0.53100000000000003},"min":{"1min":0.34899999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55900000000000005,"15min":0.54700000000000004},"min":{"1min":0.373,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.94799999999999995,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.40400000000000003}]},{"osd":1,"last update":"Tue Jul 18 03:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53600000000000003,"15min":0.501},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.80900000000000005,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.52900000000000003,"15min":0.495},"min":{"1min":0.34300000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.72899999999999998,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.42699999999999999}]},{"osd":3,"last update":"Tue Jul 18 03:05:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57999999999999996,"15min":0.55900000000000005},"min":{"1min":0.40999999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.621,"5min":0.63300000000000001,"15min":0.58899999999999997},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.587,"5min":3.234,"15min":3.234},"last":0.55100000000000005}]},{"osd":4,"last update":"Tue Jul 18 03:04:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.57499999999999996,"15min":0.57899999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.86899999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78900000000000003,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.60399999999999998}]},{"osd":5,"last update":"Tue Jul 18 03:05:00 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.45200000000000001,"15min":0.442},"min":{"1min":0.312,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.498},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.46300000000000002,"15min":0.44700000000000001},"min":{"1min":0.32200000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.64000000000000001}]},{"osd":6,"last update":"Tue Jul 18 03:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.65800000000000003,"15min":0.64000000000000001},"min":{"1min":0.46600000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.70799999999999996,"15min":0.68799999999999994},"min":{"1min":0.53800000000000003,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":0.72799999999999998}]},{"osd":7,"last update":"Tue Jul 18 03:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.65700000000000003,"15min":0.65800000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.93700000000000006,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.85199999999999998,"5min":0.874,"15min":0.874},"last":0.75700000000000001}]},{"osd":8,"last update":"Tue Jul 18 03:05:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.63,"15min":0.61699999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.66200000000000003,"15min":0.65100000000000002},"min":{"1min":0.41099999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.69599999999999995}]},{"osd":9,"last update":"Tue Jul 18 03:04:33 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.80400000000000005}]},{"osd":10,"last update":"Tue Jul 18 03:04:51 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.745}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]}]},{"osd":3,"up_from":25,"seq":107374182454,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":974848,"data_stored":483209,"data_compressed":17573,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:05:24 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.42599999999999999,"15min":0.40300000000000002},"min":{"1min":0.32300000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.44700000000000001,"15min":0.41399999999999998},"min":{"1min":0.30299999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.86699999999999999,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.56699999999999995}]},{"osd":1,"last update":"Tue Jul 18 03:05:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59699999999999998,"15min":0.56899999999999995},"min":{"1min":0.432,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.87,"5min":1.115,"15min":1.115},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.62,"15min":0.57999999999999996},"min":{"1min":0.432,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.64700000000000002}]},{"osd":2,"last update":"Tue Jul 18 03:05:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.64700000000000002,"15min":0.60299999999999998},"min":{"1min":0.46400000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.1140000000000001,"5min":1.26,"15min":1.26},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61799999999999999,"15min":0.59599999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.036,"5min":1.212,"15min":1.212},"last":0.51100000000000001}]},{"osd":4,"last update":"Tue Jul 18 03:04:43 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.64100000000000001,"15min":0.61299999999999999},"min":{"1min":0.55800000000000005,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.64300000000000002,"15min":0.60999999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0249999999999999,"5min":1.143,"15min":1.143},"last":0.69699999999999995}]},{"osd":5,"last update":"Tue Jul 18 03:04:56 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.65900000000000003,"15min":0.629},"min":{"1min":0.48699999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.70099999999999996,"15min":0.67800000000000005},"min":{"1min":0.53500000000000003,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.79900000000000004}]},{"osd":6,"last update":"Tue Jul 18 03:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.66900000000000004,"15min":0.64300000000000002},"min":{"1min":0.48099999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0740000000000001,"5min":1.385,"15min":1.385},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.60999999999999999,"15min":0.59999999999999998},"min":{"1min":0.434,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85999999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.55200000000000005}]},{"osd":7,"last update":"Tue Jul 18 03:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.73699999999999999,"15min":0.72299999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.75900000000000001,"15min":0.75},"min":{"1min":0.504,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.84799999999999998}]},{"osd":8,"last update":"Tue Jul 18 03:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.76200000000000001}]},{"osd":9,"last update":"Tue Jul 18 03:04:30 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.73999999999999999}]},{"osd":10,"last update":"Tue Jul 18 03:04:51 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.96199999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]}]},{"osd":4,"up_from":33,"seq":141733920818,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":978944,"data_stored":487707,"data_compressed":17573,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.56200000000000006,"15min":0.54600000000000004},"min":{"1min":0.39600000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.439},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.56899999999999995,"15min":0.54500000000000004},"min":{"1min":0.42799999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.48999999999999999}]},{"osd":1,"last update":"Tue Jul 18 03:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.378,"15min":0.35799999999999998},"min":{"1min":0.30399999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.39200000000000002,"15min":0.375},"min":{"1min":0.32300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.46600000000000003}]},{"osd":2,"last update":"Tue Jul 18 03:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.58199999999999996,"15min":0.56999999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.58299999999999996,"15min":0.56000000000000005},"min":{"1min":0.435,"5min":0.378,"15min":0.378},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.58099999999999996}]},{"osd":3,"last update":"Tue Jul 18 03:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.60799999999999998,"15min":0.58199999999999996},"min":{"1min":0.45700000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.96199999999999997,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.59599999999999997,"15min":0.58099999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.54100000000000004}]},{"osd":5,"last update":"Tue Jul 18 03:05:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.629,"15min":0.60299999999999998},"min":{"1min":0.48099999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.96299999999999997,"5min":1.214,"15min":1.214},"last":0.751},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.627,"15min":0.60499999999999998},"min":{"1min":0.51500000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.97999999999999998,"5min":1.306,"15min":1.306},"last":0.78200000000000003}]},{"osd":6,"last update":"Tue Jul 18 03:05:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.71699999999999997,"15min":0.72599999999999998},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.42399999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.61199999999999999}]},{"osd":7,"last update":"Tue Jul 18 03:04:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.60799999999999998,"15min":0.60199999999999998},"min":{"1min":0.372,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.626},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.65200000000000002,"15min":0.64600000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.873}]},{"osd":8,"last update":"Tue Jul 18 03:05:07 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.73999999999999999,"15min":0.73499999999999999},"min":{"1min":0.59799999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.98399999999999999,"5min":1.155,"15min":1.155},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.752,"15min":0.74199999999999999},"min":{"1min":0.56299999999999994,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0920000000000001,"5min":1.135,"15min":1.135},"last":0.90400000000000003}]},{"osd":9,"last update":"Tue Jul 18 03:05:33 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.72399999999999998,"15min":0.72199999999999998},"min":{"1min":0.54000000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.82299999999999995,"15min":0.81100000000000005},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":3.415,"5min":3.415,"15min":3.415},"last":0.71399999999999997}]},{"osd":10,"last update":"Tue Jul 18 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.66500000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91700000000000004}]}]},{"osd":5,"up_from":38,"seq":163208757293,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":1016,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":1040384,"data_stored":1074811,"data_compressed":53332,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:05:02 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54100000000000004,"15min":0.52900000000000003},"min":{"1min":0.373,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.79200000000000004,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54700000000000004,"15min":0.53700000000000003},"min":{"1min":0.34200000000000003,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.63800000000000001,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.54100000000000004}]},{"osd":1,"last update":"Tue Jul 18 03:05:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.55800000000000005,"15min":0.53700000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79800000000000004,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55300000000000005,"15min":0.54300000000000004},"min":{"1min":0.39000000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.75600000000000001}]},{"osd":2,"last update":"Tue Jul 18 03:05:02 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.38600000000000001,"15min":0.373},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.64500000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.432,"5min":0.41099999999999998,"15min":0.39500000000000002},"min":{"1min":0.27500000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.59299999999999997,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.51800000000000002}]},{"osd":3,"last update":"Tue Jul 18 03:05:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58799999999999997,"15min":0.56799999999999995},"min":{"1min":0.45000000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58599999999999997,"15min":0.56899999999999995},"min":{"1min":0.41599999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73299999999999998,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.63800000000000001}]},{"osd":4,"last update":"Tue Jul 18 03:05:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58199999999999996,"15min":0.56799999999999995},"min":{"1min":0.42399999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.80700000000000005,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.58899999999999997,"15min":0.56499999999999995},"min":{"1min":0.44700000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.82199999999999995}]},{"osd":6,"last update":"Tue Jul 18 03:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.61599999999999999,"15min":0.60299999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.63800000000000001,"15min":0.622},"min":{"1min":0.54300000000000004,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.85799999999999998}]},{"osd":7,"last update":"Tue Jul 18 03:04:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.67500000000000004,"15min":0.67600000000000005},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.875},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.68400000000000005,"15min":0.68799999999999994},"min":{"1min":0.41399999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.89100000000000001}]},{"osd":8,"last update":"Tue Jul 18 03:05:06 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.56299999999999994,"15min":0.57199999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.71199999999999997,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.81299999999999994,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.73999999999999999}]},{"osd":9,"last update":"Tue Jul 18 03:04:32 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.91200000000000003}]},{"osd":10,"last update":"Tue Jul 18 03:04:56 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.93300000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34117,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34117,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34117,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35759,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35759,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35759,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35759,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35759,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":40207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-18T03:05:35.782 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-18T03:05:35.782 INFO:tasks.cephadm:Setting up client nodes... 2023-07-18T03:05:35.783 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-07-18T03:05:36.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:36 smithi101 ceph-mon[126379]: pgmap v295: 129 pgs: 129 active+clean; 584 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.4 KiB/s wr, 7 op/s 2023-07-18T03:05:36.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:36 smithi101 ceph-mon[126379]: from='client.24871 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:05:36.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:36 smithi155 ceph-mon[128885]: pgmap v295: 129 pgs: 129 active+clean; 584 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.4 KiB/s wr, 7 op/s 2023-07-18T03:05:36.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:36 smithi155 ceph-mon[128885]: from='client.24871 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:05:36.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:36 smithi179 ceph-mon[129229]: pgmap v295: 129 pgs: 129 active+clean; 584 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.4 KiB/s wr, 7 op/s 2023-07-18T03:05:36.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:36 smithi179 ceph-mon[129229]: from='client.24871 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:05:36.538 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:05:37.222 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:37 smithi101 ceph-mon[126379]: from='client.24877 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:05:37.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:37 smithi155 ceph-mon[128885]: from='client.24877 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:05:37.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:37 smithi179 ceph-mon[129229]: from='client.24877 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:05:38.251 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:38 smithi101 ceph-mon[126379]: pgmap v296: 129 pgs: 129 active+clean; 584 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2023-07-18T03:05:38.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:38 smithi155 ceph-mon[128885]: pgmap v296: 129 pgs: 129 active+clean; 584 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2023-07-18T03:05:38.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:38 smithi179 ceph-mon[129229]: pgmap v296: 129 pgs: 129 active+clean; 584 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2023-07-18T03:05:38.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.462+0000 7f952ff0c700 1 -- 172.21.15.101:0/1767092270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9528100db0 msgr2=0x7f9528101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:38.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.462+0000 7f952ff0c700 1 --2- 172.21.15.101:0/1767092270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9528100db0 0x7f9528101200 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9518009990 tx=0x7f9518009ca0 comp rx=0 tx=0).stop 2023-07-18T03:05:38.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.463+0000 7f952ff0c700 1 -- 172.21.15.101:0/1767092270 shutdown_connections 2023-07-18T03:05:38.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.463+0000 7f952ff0c700 1 --2- 172.21.15.101:0/1767092270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9528101740 0x7f9528108000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:38.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.463+0000 7f952ff0c700 1 --2- 172.21.15.101:0/1767092270 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f95280ffbb0 0x7f95280fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:38.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.463+0000 7f952ff0c700 1 --2- 172.21.15.101:0/1767092270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9528100db0 0x7f9528101200 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:38.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.463+0000 7f952ff0c700 1 -- 172.21.15.101:0/1767092270 >> 172.21.15.101:0/1767092270 conn(0x7f95280fb200 msgr2=0x7f95280fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:38.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.463+0000 7f952ff0c700 1 -- 172.21.15.101:0/1767092270 shutdown_connections 2023-07-18T03:05:38.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.463+0000 7f952ff0c700 1 -- 172.21.15.101:0/1767092270 wait complete. 2023-07-18T03:05:38.450 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.464+0000 7f952ff0c700 1 Processor -- start 2023-07-18T03:05:38.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.464+0000 7f952ff0c700 1 -- start start 2023-07-18T03:05:38.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.464+0000 7f952ff0c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95280ffbb0 0x7f95280742b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:38.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.464+0000 7f952ff0c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9528100db0 0x7f9528072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:38.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.464+0000 7f952ff0c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9528101740 0x7f9528072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:38.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.464+0000 7f952ff0c700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95280735c0 con 0x7f9528100db0 2023-07-18T03:05:38.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.464+0000 7f952ff0c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9528073730 con 0x7f95280ffbb0 2023-07-18T03:05:38.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.464+0000 7f952ff0c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952819f920 con 0x7f9528101740 2023-07-18T03:05:38.452 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.464+0000 7f952d4a7700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9528100db0 0x7f9528072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:38.452 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.465+0000 7f952d4a7700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9528100db0 0x7f9528072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:60782/0 (socket says 172.21.15.101:60782) 2023-07-18T03:05:38.452 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.465+0000 7f952d4a7700 1 -- 172.21.15.101:0/1476658754 learned_addr learned my addr 172.21.15.101:0/1476658754 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:38.452 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.465+0000 7f952dca8700 1 --2- 172.21.15.101:0/1476658754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95280ffbb0 0x7f95280742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:38.452 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.465+0000 7f952e4a9700 1 --2- 172.21.15.101:0/1476658754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9528101740 0x7f9528072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:38.452 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.465+0000 7f952d4a7700 1 -- 172.21.15.101:0/1476658754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9528101740 msgr2=0x7f9528072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:38.452 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.465+0000 7f952d4a7700 1 --2- 172.21.15.101:0/1476658754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9528101740 0x7f9528072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:38.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.465+0000 7f952d4a7700 1 -- 172.21.15.101:0/1476658754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95280ffbb0 msgr2=0x7f95280742b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:38.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.465+0000 7f952d4a7700 1 --2- 172.21.15.101:0/1476658754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95280ffbb0 0x7f95280742b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:38.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.465+0000 7f952d4a7700 1 -- 172.21.15.101:0/1476658754 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95180095f0 con 0x7f9528100db0 2023-07-18T03:05:38.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.465+0000 7f952d4a7700 1 --2- 172.21.15.101:0/1476658754 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9528100db0 0x7f9528072900 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9518004730 tx=0x7f9518004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:38.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.466+0000 7f951effd700 1 -- 172.21.15.101:0/1476658754 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951801d070 con 0x7f9528100db0 2023-07-18T03:05:38.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.466+0000 7f952ff0c700 1 -- 172.21.15.101:0/1476658754 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f952819fba0 con 0x7f9528100db0 2023-07-18T03:05:38.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.466+0000 7f951effd700 1 -- 172.21.15.101:0/1476658754 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9518004c90 con 0x7f9528100db0 2023-07-18T03:05:38.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.466+0000 7f951effd700 1 -- 172.21.15.101:0/1476658754 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95180054b0 con 0x7f9528100db0 2023-07-18T03:05:38.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.466+0000 7f952ff0c700 1 -- 172.21.15.101:0/1476658754 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f952819ffb0 con 0x7f9528100db0 2023-07-18T03:05:38.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.467+0000 7f952ff0c700 1 -- 172.21.15.101:0/1476658754 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f952810b3c0 con 0x7f9528100db0 2023-07-18T03:05:38.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.469+0000 7f951effd700 1 -- 172.21.15.101:0/1476658754 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f951800fbc0 con 0x7f9528100db0 2023-07-18T03:05:38.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.470+0000 7f951effd700 1 --2- 172.21.15.101:0/1476658754 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f95140a1260 0x7f95140a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:38.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.470+0000 7f951effd700 1 -- 172.21.15.101:0/1476658754 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f95180cdfd0 con 0x7f9528100db0 2023-07-18T03:05:38.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.470+0000 7f952dca8700 1 --2- 172.21.15.101:0/1476658754 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f95140a1260 0x7f95140a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:38.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.471+0000 7f952dca8700 1 --2- 172.21.15.101:0/1476658754 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f95140a1260 0x7f95140a3710 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9528100c10 tx=0x7f9524007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:38.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.474+0000 7f951effd700 1 -- 172.21.15.101:0/1476658754 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f951805a790 con 0x7f9528100db0 2023-07-18T03:05:38.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.746+0000 7f952ff0c700 1 -- 172.21.15.101:0/1476658754 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f9528106af0 con 0x7f9528100db0 2023-07-18T03:05:38.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.750+0000 7f951effd700 1 -- 172.21.15.101:0/1476658754 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7f95180972f0 con 0x7f9528100db0 2023-07-18T03:05:38.736 INFO:teuthology.orchestra.run.smithi101.stdout:[client.0] 2023-07-18T03:05:38.736 INFO:teuthology.orchestra.run.smithi101.stdout: key = AQCCAbZkeiGSLBAALWqI3R32nZafvmKPQfYybg== 2023-07-18T03:05:38.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.753+0000 7f952ff0c700 1 -- 172.21.15.101:0/1476658754 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f95140a1260 msgr2=0x7f95140a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:38.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.753+0000 7f952ff0c700 1 --2- 172.21.15.101:0/1476658754 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f95140a1260 0x7f95140a3710 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9528100c10 tx=0x7f9524007400 comp rx=0 tx=0).stop 2023-07-18T03:05:38.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.753+0000 7f952ff0c700 1 -- 172.21.15.101:0/1476658754 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9528100db0 msgr2=0x7f9528072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:38.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.753+0000 7f952ff0c700 1 --2- 172.21.15.101:0/1476658754 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9528100db0 0x7f9528072900 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9518004730 tx=0x7f9518004760 comp rx=0 tx=0).stop 2023-07-18T03:05:38.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.753+0000 7f952ff0c700 1 -- 172.21.15.101:0/1476658754 shutdown_connections 2023-07-18T03:05:38.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.753+0000 7f952ff0c700 1 --2- 172.21.15.101:0/1476658754 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f95140a1260 0x7f95140a3710 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:38.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.753+0000 7f952ff0c700 1 --2- 172.21.15.101:0/1476658754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95280ffbb0 0x7f95280742b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:38.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.753+0000 7f952ff0c700 1 --2- 172.21.15.101:0/1476658754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9528101740 0x7f9528072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:38.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.753+0000 7f952ff0c700 1 --2- 172.21.15.101:0/1476658754 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9528100db0 0x7f9528072900 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:38.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.753+0000 7f952ff0c700 1 -- 172.21.15.101:0/1476658754 >> 172.21.15.101:0/1476658754 conn(0x7f95280fb200 msgr2=0x7f95280fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:38.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.753+0000 7f952ff0c700 1 -- 172.21.15.101:0/1476658754 shutdown_connections 2023-07-18T03:05:38.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:38.753+0000 7f952ff0c700 1 -- 172.21.15.101:0/1476658754 wait complete. 2023-07-18T03:05:39.213 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:39 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1476658754' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-18T03:05:39.213 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:39 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1476658754' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-07-18T03:05:39.228 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-07-18T03:05:39.229 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-07-18T03:05:39.229 DEBUG:teuthology.orchestra.run.smithi101:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-07-18T03:05:39.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:39 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1476658754' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-18T03:05:39.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:39 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1476658754' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-07-18T03:05:39.329 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-07-18T03:05:39.330 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-07-18T03:05:39.330 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph mgr dump --format=json 2023-07-18T03:05:39.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:39 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1476658754' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-18T03:05:39.444 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:39 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1476658754' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-07-18T03:05:40.088 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:05:40.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:40 smithi155 ceph-mon[128885]: pgmap v297: 129 pgs: 129 active+clean; 623 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.9 KiB/s wr, 12 op/s 2023-07-18T03:05:40.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:40 smithi179 ceph-mon[129229]: pgmap v297: 129 pgs: 129 active+clean; 623 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.9 KiB/s wr, 12 op/s 2023-07-18T03:05:40.479 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:40 smithi101 ceph-mon[126379]: pgmap v297: 129 pgs: 129 active+clean; 623 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.9 KiB/s wr, 12 op/s 2023-07-18T03:05:41.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:41.814+0000 7f3ab0beb700 1 -- 172.21.15.101:0/310138965 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3aac0ffe00 msgr2=0x7f3aac1001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:41.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:41.814+0000 7f3ab0beb700 1 --2- 172.21.15.101:0/310138965 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3aac0ffe00 0x7f3aac1001d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c0099c0 tx=0x7f3a9c009cd0 comp rx=0 tx=0).stop 2023-07-18T03:05:41.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:41.814+0000 7f3ab0beb700 1 -- 172.21.15.101:0/310138965 shutdown_connections 2023-07-18T03:05:41.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:41.814+0000 7f3ab0beb700 1 --2- 172.21.15.101:0/310138965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3aac104ea0 0x7f3aac10ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:41.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:41.814+0000 7f3ab0beb700 1 --2- 172.21.15.101:0/310138965 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3aac0ffe00 0x7f3aac1001d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:41.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:41.814+0000 7f3ab0beb700 1 --2- 172.21.15.101:0/310138965 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3aac1007a0 0x7f3aac104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:41.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:41.814+0000 7f3ab0beb700 1 -- 172.21.15.101:0/310138965 >> 172.21.15.101:0/310138965 conn(0x7f3aac0fb660 msgr2=0x7f3aac0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:41.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:41.814+0000 7f3ab0beb700 1 -- 172.21.15.101:0/310138965 shutdown_connections 2023-07-18T03:05:41.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:41.814+0000 7f3ab0beb700 1 -- 172.21.15.101:0/310138965 wait complete. 2023-07-18T03:05:41.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:41.815+0000 7f3ab0beb700 1 Processor -- start 2023-07-18T03:05:41.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:41.815+0000 7f3ab0beb700 1 -- start start 2023-07-18T03:05:41.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:41.816+0000 7f3ab0beb700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3aac0ffe00 0x7f3aac197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:41.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:41.816+0000 7f3ab0beb700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3aac1007a0 0x7f3aac1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:41.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:41.816+0000 7f3ab0beb700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3aac104ea0 0x7f3aac19c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:41.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:41.816+0000 7f3ab0beb700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac19ce90 con 0x7f3aac0ffe00 2023-07-18T03:05:41.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:41.816+0000 7f3ab0beb700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac19d000 con 0x7f3aac1007a0 2023-07-18T03:05:41.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:41.816+0000 7f3ab0beb700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac19d1c0 con 0x7f3aac104ea0 2023-07-18T03:05:41.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:41.816+0000 7f3aaa59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3aac0ffe00 0x7f3aac197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:41.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:41.816+0000 7f3aaa59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3aac0ffe00 0x7f3aac197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:60792/0 (socket says 172.21.15.101:60792) 2023-07-18T03:05:41.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:41.816+0000 7f3aaa59c700 1 -- 172.21.15.101:0/2234300401 learned_addr learned my addr 172.21.15.101:0/2234300401 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:41.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:41.816+0000 7f3aa9d9b700 1 --2- 172.21.15.101:0/2234300401 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3aac1007a0 0x7f3aac1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:41.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:41.816+0000 7f3aaad9d700 1 --2- 172.21.15.101:0/2234300401 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3aac104ea0 0x7f3aac19c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:41.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:41.817+0000 7f3aaa59c700 1 -- 172.21.15.101:0/2234300401 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3aac104ea0 msgr2=0x7f3aac19c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:41.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:41.817+0000 7f3aaa59c700 1 --2- 172.21.15.101:0/2234300401 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3aac104ea0 0x7f3aac19c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:41.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:41.817+0000 7f3aaa59c700 1 -- 172.21.15.101:0/2234300401 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3aac1007a0 msgr2=0x7f3aac1984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:41.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:41.817+0000 7f3aaa59c700 1 --2- 172.21.15.101:0/2234300401 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3aac1007a0 0x7f3aac1984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:41.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:41.817+0000 7f3aaa59c700 1 -- 172.21.15.101:0/2234300401 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9c0095f0 con 0x7f3aac0ffe00 2023-07-18T03:05:41.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:41.817+0000 7f3aa9d9b700 1 --2- 172.21.15.101:0/2234300401 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3aac1007a0 0x7f3aac1984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:05:41.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:41.817+0000 7f3aaa59c700 1 --2- 172.21.15.101:0/2234300401 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3aac0ffe00 0x7f3aac197f70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c00b580 tx=0x7f3a9c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:41.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:41.817+0000 7f3a9b7fe700 1 -- 172.21.15.101:0/2234300401 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c01d070 con 0x7f3aac0ffe00 2023-07-18T03:05:41.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:41.817+0000 7f3ab0beb700 1 -- 172.21.15.101:0/2234300401 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aac19d440 con 0x7f3aac0ffe00 2023-07-18T03:05:41.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:41.817+0000 7f3a9b7fe700 1 -- 172.21.15.101:0/2234300401 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a9c005050 con 0x7f3aac0ffe00 2023-07-18T03:05:41.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:41.817+0000 7f3a9b7fe700 1 -- 172.21.15.101:0/2234300401 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c004a40 con 0x7f3aac0ffe00 2023-07-18T03:05:41.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:41.817+0000 7f3ab0beb700 1 -- 172.21.15.101:0/2234300401 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aac1a4d00 con 0x7f3aac0ffe00 2023-07-18T03:05:41.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:41.818+0000 7f3ab0beb700 1 -- 172.21.15.101:0/2234300401 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aac04e210 con 0x7f3aac0ffe00 2023-07-18T03:05:41.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:41.820+0000 7f3a9b7fe700 1 -- 172.21.15.101:0/2234300401 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f3a9c005430 con 0x7f3aac0ffe00 2023-07-18T03:05:41.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:41.822+0000 7f3a9b7fe700 1 --2- 172.21.15.101:0/2234300401 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f3a940a11e0 0x7f3a940a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:41.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:41.822+0000 7f3a9b7fe700 1 -- 172.21.15.101:0/2234300401 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f3a9c0cdc20 con 0x7f3aac0ffe00 2023-07-18T03:05:41.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:41.822+0000 7f3aa9d9b700 1 --2- 172.21.15.101:0/2234300401 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f3a940a11e0 0x7f3a940a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:41.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:41.822+0000 7f3aa9d9b700 1 --2- 172.21.15.101:0/2234300401 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f3a940a11e0 0x7f3a940a3690 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3aac199590 tx=0x7f3aa0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:41.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:41.826+0000 7f3a9b7fe700 1 -- 172.21.15.101:0/2234300401 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3a9c096f40 con 0x7f3aac0ffe00 2023-07-18T03:05:42.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:42.076+0000 7f3ab0beb700 1 -- 172.21.15.101:0/2234300401 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f3aac1a4e90 con 0x7f3aac0ffe00 2023-07-18T03:05:42.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:42.082+0000 7f3a9b7fe700 1 -- 172.21.15.101:0/2234300401 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260134 (secure 0 0 0) 0x7f3a9c09a560 con 0x7f3aac0ffe00 2023-07-18T03:05:42.071 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:05:42.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:42.086+0000 7f3ab0beb700 1 -- 172.21.15.101:0/2234300401 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f3a940a11e0 msgr2=0x7f3a940a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:42.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:42.086+0000 7f3ab0beb700 1 --2- 172.21.15.101:0/2234300401 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f3a940a11e0 0x7f3a940a3690 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3aac199590 tx=0x7f3aa0009260 comp rx=0 tx=0).stop 2023-07-18T03:05:42.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:42.086+0000 7f3ab0beb700 1 -- 172.21.15.101:0/2234300401 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3aac0ffe00 msgr2=0x7f3aac197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:42.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:42.086+0000 7f3ab0beb700 1 --2- 172.21.15.101:0/2234300401 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3aac0ffe00 0x7f3aac197f70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c00b580 tx=0x7f3a9c00f690 comp rx=0 tx=0).stop 2023-07-18T03:05:42.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:42.086+0000 7f3ab0beb700 1 -- 172.21.15.101:0/2234300401 shutdown_connections 2023-07-18T03:05:42.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:42.086+0000 7f3ab0beb700 1 --2- 172.21.15.101:0/2234300401 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f3a940a11e0 0x7f3a940a3690 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:42.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:42.086+0000 7f3ab0beb700 1 --2- 172.21.15.101:0/2234300401 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3aac104ea0 0x7f3aac19c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:42.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:42.086+0000 7f3ab0beb700 1 --2- 172.21.15.101:0/2234300401 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3aac0ffe00 0x7f3aac197f70 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:42.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:42.086+0000 7f3ab0beb700 1 --2- 172.21.15.101:0/2234300401 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3aac1007a0 0x7f3aac1984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:42.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:42.086+0000 7f3ab0beb700 1 -- 172.21.15.101:0/2234300401 >> 172.21.15.101:0/2234300401 conn(0x7f3aac0fb660 msgr2=0x7f3aac0fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:42.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:42.086+0000 7f3ab0beb700 1 -- 172.21.15.101:0/2234300401 shutdown_connections 2023-07-18T03:05:42.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:42.086+0000 7f3ab0beb700 1 -- 172.21.15.101:0/2234300401 wait complete. 2023-07-18T03:05:42.286 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:42 smithi155 ceph-mon[128885]: pgmap v298: 129 pgs: 129 active+clean; 623 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.9 KiB/s wr, 13 op/s 2023-07-18T03:05:42.295 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:42 smithi101 ceph-mon[126379]: pgmap v298: 129 pgs: 129 active+clean; 623 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.9 KiB/s wr, 13 op/s 2023-07-18T03:05:42.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:42 smithi179 ceph-mon[129229]: pgmap v298: 129 pgs: 129 active+clean; 623 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.9 KiB/s wr, 13 op/s 2023-07-18T03:05:42.578 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":17,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6800","nonce":3963742842},{"type":"v1","addr":"172.21.15.101:6801","nonce":3963742842}]},"active_addr":"172.21.15.101:6801/3963742842","active_change":"2023-07-18T02:58:22.762590+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.101:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.101:0","nonce":2766365637}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.101:0","nonce":1806538035}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.101:0","nonce":523372970}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.101:0","nonce":1004430030}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.101:0","nonce":1648908075}]}]} 2023-07-18T03:05:42.585 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-07-18T03:05:42.586 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-07-18T03:05:42.586 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph osd dump --format=json 2023-07-18T03:05:43.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:43 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/2234300401' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-07-18T03:05:43.307 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:05:43.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:43 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/2234300401' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-07-18T03:05:43.479 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:43 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2234300401' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-07-18T03:05:44.229 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:44 smithi101 ceph-mon[126379]: pgmap v299: 129 pgs: 129 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.8 KiB/s wr, 14 op/s 2023-07-18T03:05:44.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:44 smithi155 ceph-mon[128885]: pgmap v299: 129 pgs: 129 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.8 KiB/s wr, 14 op/s 2023-07-18T03:05:44.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:44 smithi179 ceph-mon[129229]: pgmap v299: 129 pgs: 129 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.8 KiB/s wr, 14 op/s 2023-07-18T03:05:44.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:44.997+0000 7f49186e6700 1 -- 172.21.15.101:0/2734676395 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4910104ea0 msgr2=0x7f491010ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:44.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:44.997+0000 7f49186e6700 1 --2- 172.21.15.101:0/2734676395 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4910104ea0 0x7f491010ca20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4908009fc0 tx=0x7f490800b330 comp rx=0 tx=0).stop 2023-07-18T03:05:44.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:44.997+0000 7f49186e6700 1 -- 172.21.15.101:0/2734676395 shutdown_connections 2023-07-18T03:05:44.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:44.997+0000 7f49186e6700 1 --2- 172.21.15.101:0/2734676395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49100ffe00 0x7f49101001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:44.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:44.997+0000 7f49186e6700 1 --2- 172.21.15.101:0/2734676395 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4910104ea0 0x7f491010ca20 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:44.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:44.997+0000 7f49186e6700 1 --2- 172.21.15.101:0/2734676395 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49101007a0 0x7f4910104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:44.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:44.997+0000 7f49186e6700 1 -- 172.21.15.101:0/2734676395 >> 172.21.15.101:0/2734676395 conn(0x7f49100fb660 msgr2=0x7f49100fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:44.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:44.997+0000 7f49186e6700 1 -- 172.21.15.101:0/2734676395 shutdown_connections 2023-07-18T03:05:44.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:44.997+0000 7f49186e6700 1 -- 172.21.15.101:0/2734676395 wait complete. 2023-07-18T03:05:44.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:44.998+0000 7f49186e6700 1 Processor -- start 2023-07-18T03:05:44.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:44.998+0000 7f49186e6700 1 -- start start 2023-07-18T03:05:44.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:44.999+0000 7f49186e6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49100ffe00 0x7f4910072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:44.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:44.999+0000 7f49186e6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49101007a0 0x7f4910072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:44.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:44.999+0000 7f49186e6700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4910104ea0 0x7f49100732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:44.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:44.999+0000 7f49186e6700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49101a41d0 con 0x7f4910104ea0 2023-07-18T03:05:44.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:44.999+0000 7f49186e6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49101a4340 con 0x7f49100ffe00 2023-07-18T03:05:44.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:44.999+0000 7f49186e6700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49101a4500 con 0x7f49101007a0 2023-07-18T03:05:44.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:44.999+0000 7f4916482700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49100ffe00 0x7f4910072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:44.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:44.999+0000 7f4916482700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49100ffe00 0x7f4910072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:55570/0 (socket says 172.21.15.101:55570) 2023-07-18T03:05:44.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:44.999+0000 7f4916482700 1 -- 172.21.15.101:0/448925280 learned_addr learned my addr 172.21.15.101:0/448925280 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:44.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:44.999+0000 7f4916c83700 1 --2- 172.21.15.101:0/448925280 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4910104ea0 0x7f49100732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:44.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:44.999+0000 7f4915c81700 1 --2- 172.21.15.101:0/448925280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49101007a0 0x7f4910072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:44.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:45.000+0000 7f4916482700 1 -- 172.21.15.101:0/448925280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49101007a0 msgr2=0x7f4910072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:44.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:45.000+0000 7f4916482700 1 --2- 172.21.15.101:0/448925280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49101007a0 0x7f4910072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:44.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:45.000+0000 7f4916482700 1 -- 172.21.15.101:0/448925280 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4910104ea0 msgr2=0x7f49100732d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:44.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:45.000+0000 7f4916482700 1 --2- 172.21.15.101:0/448925280 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4910104ea0 0x7f49100732d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:44.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:45.000+0000 7f4916482700 1 -- 172.21.15.101:0/448925280 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4908009c40 con 0x7f49100ffe00 2023-07-18T03:05:44.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:45.000+0000 7f4916c83700 1 --2- 172.21.15.101:0/448925280 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4910104ea0 0x7f49100732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:05:44.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:45.000+0000 7f4916482700 1 --2- 172.21.15.101:0/448925280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49100ffe00 0x7f4910072810 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f490c00e9c0 tx=0x7f490c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:44.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:45.000+0000 7f49077fe700 1 -- 172.21.15.101:0/448925280 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c004a30 con 0x7f49100ffe00 2023-07-18T03:05:44.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:45.000+0000 7f49186e6700 1 -- 172.21.15.101:0/448925280 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49101a47e0 con 0x7f49100ffe00 2023-07-18T03:05:44.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:45.000+0000 7f49077fe700 1 -- 172.21.15.101:0/448925280 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f490c00cc80 con 0x7f49100ffe00 2023-07-18T03:05:44.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:45.000+0000 7f49077fe700 1 -- 172.21.15.101:0/448925280 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c0054e0 con 0x7f49100ffe00 2023-07-18T03:05:44.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:45.000+0000 7f49186e6700 1 -- 172.21.15.101:0/448925280 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49101a4d30 con 0x7f49100ffe00 2023-07-18T03:05:44.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:45.001+0000 7f49186e6700 1 -- 172.21.15.101:0/448925280 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f491004e210 con 0x7f49100ffe00 2023-07-18T03:05:44.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:45.003+0000 7f49077fe700 1 -- 172.21.15.101:0/448925280 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f490c004b90 con 0x7f49100ffe00 2023-07-18T03:05:44.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:45.005+0000 7f49077fe700 1 --2- 172.21.15.101:0/448925280 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f48fc0a1260 0x7f48fc0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:44.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:45.005+0000 7f49077fe700 1 -- 172.21.15.101:0/448925280 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f490c014070 con 0x7f49100ffe00 2023-07-18T03:05:44.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:45.005+0000 7f4915c81700 1 --2- 172.21.15.101:0/448925280 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f48fc0a1260 0x7f48fc0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:44.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:45.006+0000 7f4915c81700 1 --2- 172.21.15.101:0/448925280 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f48fc0a1260 0x7f48fc0a3710 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4910074000 tx=0x7f4900009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:44.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:45.007+0000 7f49077fe700 1 -- 172.21.15.101:0/448925280 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f490c0962c0 con 0x7f49100ffe00 2023-07-18T03:05:45.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:45.204+0000 7f49186e6700 1 -- 172.21.15.101:0/448925280 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49101a50e0 con 0x7f49100ffe00 2023-07-18T03:05:45.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:45.205+0000 7f49077fe700 1 -- 172.21.15.101:0/448925280 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24482 (secure 0 0 0) 0x7f490c0998e0 con 0x7f49100ffe00 2023-07-18T03:05:45.192 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:05:45.192 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":89,"fsid":"b1782daa-2516-11ee-9b34-001a4aab830c","created":"2023-07-18T02:57:22.972483+0000","modified":"2023-07-18T03:05:23.929113+0000","last_up_change":"2023-07-18T03:04:11.429530+0000","last_in_change":"2023-07-18T03:03:54.158340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:00:43.353489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:04:57.771616+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-18T03:04:58.769590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-18T03:05:02.358830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc76b5d1-38a0-4f1a-9335-756bfc40a62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6803","nonce":3610719929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6805","nonce":3610719929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6809","nonce":3610719929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6807","nonce":3610719929}]},"public_addr":"172.21.15.101:6803/3610719929","cluster_addr":"172.21.15.101:6805/3610719929","heartbeat_back_addr":"172.21.15.101:6809/3610719929","heartbeat_front_addr":"172.21.15.101:6807/3610719929","state":["exists","up"]},{"osd":1,"uuid":"2a299573-0b34-4af9-aaae-b523e5058ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6801","nonce":1030229579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6803","nonce":1030229579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6807","nonce":1030229579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6805","nonce":1030229579}]},"public_addr":"172.21.15.155:6801/1030229579","cluster_addr":"172.21.15.155:6803/1030229579","heartbeat_back_addr":"172.21.15.155:6807/1030229579","heartbeat_front_addr":"172.21.15.155:6805/1030229579","state":["exists","up"]},{"osd":2,"uuid":"123b3749-c636-48fa-9e33-6229b5575dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6801","nonce":1349487507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6803","nonce":1349487507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6807","nonce":1349487507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6805","nonce":1349487507}]},"public_addr":"172.21.15.179:6801/1349487507","cluster_addr":"172.21.15.179:6803/1349487507","heartbeat_back_addr":"172.21.15.179:6807/1349487507","heartbeat_front_addr":"172.21.15.179:6805/1349487507","state":["exists","up"]},{"osd":3,"uuid":"4b0c409d-2954-4127-833a-b196bf226d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6811","nonce":4226577463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6813","nonce":4226577463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6817","nonce":4226577463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6815","nonce":4226577463}]},"public_addr":"172.21.15.101:6811/4226577463","cluster_addr":"172.21.15.101:6813/4226577463","heartbeat_back_addr":"172.21.15.101:6817/4226577463","heartbeat_front_addr":"172.21.15.101:6815/4226577463","state":["exists","up"]},{"osd":4,"uuid":"2137f2df-984e-45ce-baf6-b4ac12c7e3c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6809","nonce":1951135805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6811","nonce":1951135805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6815","nonce":1951135805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6813","nonce":1951135805}]},"public_addr":"172.21.15.155:6809/1951135805","cluster_addr":"172.21.15.155:6811/1951135805","heartbeat_back_addr":"172.21.15.155:6815/1951135805","heartbeat_front_addr":"172.21.15.155:6813/1951135805","state":["exists","up"]},{"osd":5,"uuid":"1020bfc5-8c2f-482e-9588-66c7f98e46e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6809","nonce":2746037021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6811","nonce":2746037021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6815","nonce":2746037021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6813","nonce":2746037021}]},"public_addr":"172.21.15.179:6809/2746037021","cluster_addr":"172.21.15.179:6811/2746037021","heartbeat_back_addr":"172.21.15.179:6815/2746037021","heartbeat_front_addr":"172.21.15.179:6813/2746037021","state":["exists","up"]},{"osd":6,"uuid":"2cd99260-5c36-4603-9338-cf697970ce58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6819","nonce":958044931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6821","nonce":958044931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6825","nonce":958044931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6823","nonce":958044931}]},"public_addr":"172.21.15.101:6819/958044931","cluster_addr":"172.21.15.101:6821/958044931","heartbeat_back_addr":"172.21.15.101:6825/958044931","heartbeat_front_addr":"172.21.15.101:6823/958044931","state":["exists","up"]},{"osd":7,"uuid":"ed23fd0a-678f-4e15-b42c-cfe873315714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6817","nonce":3256322291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6819","nonce":3256322291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6823","nonce":3256322291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6821","nonce":3256322291}]},"public_addr":"172.21.15.155:6817/3256322291","cluster_addr":"172.21.15.155:6819/3256322291","heartbeat_back_addr":"172.21.15.155:6823/3256322291","heartbeat_front_addr":"172.21.15.155:6821/3256322291","state":["exists","up"]},{"osd":8,"uuid":"eca3d8e6-7baf-47a5-b991-328001c7ebef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6817","nonce":3520612559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6819","nonce":3520612559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6823","nonce":3520612559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6821","nonce":3520612559}]},"public_addr":"172.21.15.179:6817/3520612559","cluster_addr":"172.21.15.179:6819/3520612559","heartbeat_back_addr":"172.21.15.179:6823/3520612559","heartbeat_front_addr":"172.21.15.179:6821/3520612559","state":["exists","up"]},{"osd":9,"uuid":"465a7ffb-67e4-49ef-812c-1b96feacfebf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6827","nonce":2166787667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6829","nonce":2166787667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6833","nonce":2166787667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6831","nonce":2166787667}]},"public_addr":"172.21.15.101:6827/2166787667","cluster_addr":"172.21.15.101:6829/2166787667","heartbeat_back_addr":"172.21.15.101:6833/2166787667","heartbeat_front_addr":"172.21.15.101:6831/2166787667","state":["exists","up"]},{"osd":10,"uuid":"1897a027-f25d-434f-8df5-5fc2bd866578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6825","nonce":3682446349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6827","nonce":3682446349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6831","nonce":3682446349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6829","nonce":3682446349}]},"public_addr":"172.21.15.155:6825/3682446349","cluster_addr":"172.21.15.155:6827/3682446349","heartbeat_back_addr":"172.21.15.155:6831/3682446349","heartbeat_front_addr":"172.21.15.155:6829/3682446349","state":["exists","up"]},{"osd":11,"uuid":"b62effaf-45e2-4585-b8d0-db448f3474ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6825","nonce":2602640858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6827","nonce":2602640858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6831","nonce":2602640858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6829","nonce":2602640858}]},"public_addr":"172.21.15.179:6825/2602640858","cluster_addr":"172.21.15.179:6827/2602640858","heartbeat_back_addr":"172.21.15.179:6831/2602640858","heartbeat_front_addr":"172.21.15.179:6829/2602640858","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T02:59:56.074519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:19.358142+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:41.128966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:08.227054+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:31.442592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:52.751685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:19.251054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:40.698241+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:01.339150+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:27.955910+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:49.136980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:04:10.084982+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.101:6800/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:6801/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/604228291":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/4055187849":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/2844574324":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/1603248215":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6801/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6800/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/3307578182":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/373189792":"2023-07-19T02:57:56.243200+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:05:45.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:45.208+0000 7f49186e6700 1 -- 172.21.15.101:0/448925280 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f48fc0a1260 msgr2=0x7f48fc0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:45.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:45.208+0000 7f49186e6700 1 --2- 172.21.15.101:0/448925280 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f48fc0a1260 0x7f48fc0a3710 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4910074000 tx=0x7f4900009260 comp rx=0 tx=0).stop 2023-07-18T03:05:45.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:45.208+0000 7f49186e6700 1 -- 172.21.15.101:0/448925280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49100ffe00 msgr2=0x7f4910072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:45.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:45.208+0000 7f49186e6700 1 --2- 172.21.15.101:0/448925280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49100ffe00 0x7f4910072810 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f490c00e9c0 tx=0x7f490c00ecd0 comp rx=0 tx=0).stop 2023-07-18T03:05:45.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:45.208+0000 7f49186e6700 1 -- 172.21.15.101:0/448925280 shutdown_connections 2023-07-18T03:05:45.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:45.208+0000 7f49186e6700 1 --2- 172.21.15.101:0/448925280 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f48fc0a1260 0x7f48fc0a3710 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:45.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:45.208+0000 7f49186e6700 1 --2- 172.21.15.101:0/448925280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49100ffe00 0x7f4910072810 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:45.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:45.208+0000 7f49186e6700 1 --2- 172.21.15.101:0/448925280 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4910104ea0 0x7f49100732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:45.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:45.208+0000 7f49186e6700 1 --2- 172.21.15.101:0/448925280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49101007a0 0x7f4910072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:45.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:45.208+0000 7f49186e6700 1 -- 172.21.15.101:0/448925280 >> 172.21.15.101:0/448925280 conn(0x7f49100fb660 msgr2=0x7f49100fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:45.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:45.208+0000 7f49186e6700 1 -- 172.21.15.101:0/448925280 shutdown_connections 2023-07-18T03:05:45.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:45.208+0000 7f49186e6700 1 -- 172.21.15.101:0/448925280 wait complete. 2023-07-18T03:05:45.697 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-07-18T03:05:45.697 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph osd dump --format=json 2023-07-18T03:05:46.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:46 smithi155 ceph-mon[128885]: pgmap v300: 129 pgs: 129 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.9 KiB/s wr, 11 op/s 2023-07-18T03:05:46.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:46 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/448925280' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:05:46.417 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:05:46.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:46 smithi179 ceph-mon[129229]: pgmap v300: 129 pgs: 129 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.9 KiB/s wr, 11 op/s 2023-07-18T03:05:46.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:46 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/448925280' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:05:46.479 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:46 smithi101 ceph-mon[126379]: pgmap v300: 129 pgs: 129 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.9 KiB/s wr, 11 op/s 2023-07-18T03:05:46.479 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:46 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/448925280' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:05:48.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.028+0000 7fad60e89700 1 -- 172.21.15.101:0/1658496606 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fad5c0ffe00 msgr2=0x7fad5c1001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:48.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.028+0000 7fad60e89700 1 --2- 172.21.15.101:0/1658496606 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fad5c0ffe00 0x7fad5c1001d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fad4c0099c0 tx=0x7fad4c009cd0 comp rx=0 tx=0).stop 2023-07-18T03:05:48.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.028+0000 7fad60e89700 1 -- 172.21.15.101:0/1658496606 shutdown_connections 2023-07-18T03:05:48.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.028+0000 7fad60e89700 1 --2- 172.21.15.101:0/1658496606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad5c104ea0 0x7fad5c10ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:48.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.028+0000 7fad60e89700 1 --2- 172.21.15.101:0/1658496606 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fad5c0ffe00 0x7fad5c1001d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:48.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.028+0000 7fad60e89700 1 --2- 172.21.15.101:0/1658496606 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad5c1007a0 0x7fad5c104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:48.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.028+0000 7fad60e89700 1 -- 172.21.15.101:0/1658496606 >> 172.21.15.101:0/1658496606 conn(0x7fad5c0fb660 msgr2=0x7fad5c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:48.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.028+0000 7fad60e89700 1 -- 172.21.15.101:0/1658496606 shutdown_connections 2023-07-18T03:05:48.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.028+0000 7fad60e89700 1 -- 172.21.15.101:0/1658496606 wait complete. 2023-07-18T03:05:48.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.029+0000 7fad60e89700 1 Processor -- start 2023-07-18T03:05:48.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.029+0000 7fad60e89700 1 -- start start 2023-07-18T03:05:48.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.030+0000 7fad60e89700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad5c0ffe00 0x7fad5c197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:48.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.030+0000 7fad60e89700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fad5c1007a0 0x7fad5c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:48.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.030+0000 7fad60e89700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad5c104ea0 0x7fad5c19c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:48.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.030+0000 7fad60e89700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5c19ceb0 con 0x7fad5c1007a0 2023-07-18T03:05:48.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.030+0000 7fad60e89700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5c19d020 con 0x7fad5c0ffe00 2023-07-18T03:05:48.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.030+0000 7fad60e89700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5c19d1e0 con 0x7fad5c104ea0 2023-07-18T03:05:48.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.030+0000 7fad59d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fad5c1007a0 0x7fad5c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:48.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.030+0000 7fad5a59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad5c0ffe00 0x7fad5c197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:48.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.030+0000 7fad59d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fad5c1007a0 0x7fad5c198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:34336/0 (socket says 172.21.15.101:34336) 2023-07-18T03:05:48.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.030+0000 7fad59d9b700 1 -- 172.21.15.101:0/52270902 learned_addr learned my addr 172.21.15.101:0/52270902 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:48.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.030+0000 7fad5ad9d700 1 --2- 172.21.15.101:0/52270902 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad5c104ea0 0x7fad5c19c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:48.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.031+0000 7fad5a59c700 1 -- 172.21.15.101:0/52270902 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad5c104ea0 msgr2=0x7fad5c19c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:48.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.031+0000 7fad5a59c700 1 --2- 172.21.15.101:0/52270902 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad5c104ea0 0x7fad5c19c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:48.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.031+0000 7fad5a59c700 1 -- 172.21.15.101:0/52270902 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fad5c1007a0 msgr2=0x7fad5c198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:48.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.031+0000 7fad5a59c700 1 --2- 172.21.15.101:0/52270902 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fad5c1007a0 0x7fad5c198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:48.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.031+0000 7fad5a59c700 1 -- 172.21.15.101:0/52270902 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad4c0095f0 con 0x7fad5c0ffe00 2023-07-18T03:05:48.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.031+0000 7fad5ad9d700 1 --2- 172.21.15.101:0/52270902 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad5c104ea0 0x7fad5c19c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:05:48.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.031+0000 7fad59d9b700 1 --2- 172.21.15.101:0/52270902 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fad5c1007a0 0x7fad5c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:05:48.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.031+0000 7fad5a59c700 1 --2- 172.21.15.101:0/52270902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad5c0ffe00 0x7fad5c197ef0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fad4c0098e0 tx=0x7fad4c00f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:48.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.031+0000 7fad4b7fe700 1 -- 172.21.15.101:0/52270902 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4c01d070 con 0x7fad5c0ffe00 2023-07-18T03:05:48.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.031+0000 7fad60e89700 1 -- 172.21.15.101:0/52270902 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad5c19d460 con 0x7fad5c0ffe00 2023-07-18T03:05:48.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.031+0000 7fad4b7fe700 1 -- 172.21.15.101:0/52270902 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad4c00fb80 con 0x7fad5c0ffe00 2023-07-18T03:05:48.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.031+0000 7fad4b7fe700 1 -- 172.21.15.101:0/52270902 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4c0173f0 con 0x7fad5c0ffe00 2023-07-18T03:05:48.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.031+0000 7fad60e89700 1 -- 172.21.15.101:0/52270902 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad5c1a4d20 con 0x7fad5c0ffe00 2023-07-18T03:05:48.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.032+0000 7fad60e89700 1 -- 172.21.15.101:0/52270902 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad5c04e210 con 0x7fad5c0ffe00 2023-07-18T03:05:48.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.037+0000 7fad4b7fe700 1 -- 172.21.15.101:0/52270902 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fad4c022470 con 0x7fad5c0ffe00 2023-07-18T03:05:48.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.039+0000 7fad4b7fe700 1 --2- 172.21.15.101:0/52270902 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fad440a11e0 0x7fad440a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:48.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.039+0000 7fad4b7fe700 1 -- 172.21.15.101:0/52270902 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fad4c0cdea0 con 0x7fad5c0ffe00 2023-07-18T03:05:48.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.039+0000 7fad59d9b700 1 --2- 172.21.15.101:0/52270902 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fad440a11e0 0x7fad440a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:48.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.040+0000 7fad59d9b700 1 --2- 172.21.15.101:0/52270902 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fad440a11e0 0x7fad440a3690 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fad5c199510 tx=0x7fad50009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:48.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.040+0000 7fad4b7fe700 1 -- 172.21.15.101:0/52270902 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fad4c0971c0 con 0x7fad5c0ffe00 2023-07-18T03:05:48.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.238+0000 7fad60e89700 1 -- 172.21.15.101:0/52270902 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad5c1a4eb0 con 0x7fad5c0ffe00 2023-07-18T03:05:48.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.240+0000 7fad4b7fe700 1 -- 172.21.15.101:0/52270902 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24482 (secure 0 0 0) 0x7fad4c09a7e0 con 0x7fad5c0ffe00 2023-07-18T03:05:48.226 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:05:48.226 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":89,"fsid":"b1782daa-2516-11ee-9b34-001a4aab830c","created":"2023-07-18T02:57:22.972483+0000","modified":"2023-07-18T03:05:23.929113+0000","last_up_change":"2023-07-18T03:04:11.429530+0000","last_in_change":"2023-07-18T03:03:54.158340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:00:43.353489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:04:57.771616+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-18T03:04:58.769590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-18T03:05:02.358830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc76b5d1-38a0-4f1a-9335-756bfc40a62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6803","nonce":3610719929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6805","nonce":3610719929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6809","nonce":3610719929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6807","nonce":3610719929}]},"public_addr":"172.21.15.101:6803/3610719929","cluster_addr":"172.21.15.101:6805/3610719929","heartbeat_back_addr":"172.21.15.101:6809/3610719929","heartbeat_front_addr":"172.21.15.101:6807/3610719929","state":["exists","up"]},{"osd":1,"uuid":"2a299573-0b34-4af9-aaae-b523e5058ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6801","nonce":1030229579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6803","nonce":1030229579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6807","nonce":1030229579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6805","nonce":1030229579}]},"public_addr":"172.21.15.155:6801/1030229579","cluster_addr":"172.21.15.155:6803/1030229579","heartbeat_back_addr":"172.21.15.155:6807/1030229579","heartbeat_front_addr":"172.21.15.155:6805/1030229579","state":["exists","up"]},{"osd":2,"uuid":"123b3749-c636-48fa-9e33-6229b5575dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6801","nonce":1349487507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6803","nonce":1349487507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6807","nonce":1349487507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6805","nonce":1349487507}]},"public_addr":"172.21.15.179:6801/1349487507","cluster_addr":"172.21.15.179:6803/1349487507","heartbeat_back_addr":"172.21.15.179:6807/1349487507","heartbeat_front_addr":"172.21.15.179:6805/1349487507","state":["exists","up"]},{"osd":3,"uuid":"4b0c409d-2954-4127-833a-b196bf226d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6811","nonce":4226577463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6813","nonce":4226577463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6817","nonce":4226577463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6815","nonce":4226577463}]},"public_addr":"172.21.15.101:6811/4226577463","cluster_addr":"172.21.15.101:6813/4226577463","heartbeat_back_addr":"172.21.15.101:6817/4226577463","heartbeat_front_addr":"172.21.15.101:6815/4226577463","state":["exists","up"]},{"osd":4,"uuid":"2137f2df-984e-45ce-baf6-b4ac12c7e3c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6809","nonce":1951135805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6811","nonce":1951135805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6815","nonce":1951135805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6813","nonce":1951135805}]},"public_addr":"172.21.15.155:6809/1951135805","cluster_addr":"172.21.15.155:6811/1951135805","heartbeat_back_addr":"172.21.15.155:6815/1951135805","heartbeat_front_addr":"172.21.15.155:6813/1951135805","state":["exists","up"]},{"osd":5,"uuid":"1020bfc5-8c2f-482e-9588-66c7f98e46e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6809","nonce":2746037021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6811","nonce":2746037021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6815","nonce":2746037021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6813","nonce":2746037021}]},"public_addr":"172.21.15.179:6809/2746037021","cluster_addr":"172.21.15.179:6811/2746037021","heartbeat_back_addr":"172.21.15.179:6815/2746037021","heartbeat_front_addr":"172.21.15.179:6813/2746037021","state":["exists","up"]},{"osd":6,"uuid":"2cd99260-5c36-4603-9338-cf697970ce58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6819","nonce":958044931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6821","nonce":958044931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6825","nonce":958044931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6823","nonce":958044931}]},"public_addr":"172.21.15.101:6819/958044931","cluster_addr":"172.21.15.101:6821/958044931","heartbeat_back_addr":"172.21.15.101:6825/958044931","heartbeat_front_addr":"172.21.15.101:6823/958044931","state":["exists","up"]},{"osd":7,"uuid":"ed23fd0a-678f-4e15-b42c-cfe873315714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6817","nonce":3256322291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6819","nonce":3256322291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6823","nonce":3256322291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6821","nonce":3256322291}]},"public_addr":"172.21.15.155:6817/3256322291","cluster_addr":"172.21.15.155:6819/3256322291","heartbeat_back_addr":"172.21.15.155:6823/3256322291","heartbeat_front_addr":"172.21.15.155:6821/3256322291","state":["exists","up"]},{"osd":8,"uuid":"eca3d8e6-7baf-47a5-b991-328001c7ebef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6817","nonce":3520612559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6819","nonce":3520612559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6823","nonce":3520612559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6821","nonce":3520612559}]},"public_addr":"172.21.15.179:6817/3520612559","cluster_addr":"172.21.15.179:6819/3520612559","heartbeat_back_addr":"172.21.15.179:6823/3520612559","heartbeat_front_addr":"172.21.15.179:6821/3520612559","state":["exists","up"]},{"osd":9,"uuid":"465a7ffb-67e4-49ef-812c-1b96feacfebf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6827","nonce":2166787667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6829","nonce":2166787667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6833","nonce":2166787667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6831","nonce":2166787667}]},"public_addr":"172.21.15.101:6827/2166787667","cluster_addr":"172.21.15.101:6829/2166787667","heartbeat_back_addr":"172.21.15.101:6833/2166787667","heartbeat_front_addr":"172.21.15.101:6831/2166787667","state":["exists","up"]},{"osd":10,"uuid":"1897a027-f25d-434f-8df5-5fc2bd866578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6825","nonce":3682446349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6827","nonce":3682446349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6831","nonce":3682446349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6829","nonce":3682446349}]},"public_addr":"172.21.15.155:6825/3682446349","cluster_addr":"172.21.15.155:6827/3682446349","heartbeat_back_addr":"172.21.15.155:6831/3682446349","heartbeat_front_addr":"172.21.15.155:6829/3682446349","state":["exists","up"]},{"osd":11,"uuid":"b62effaf-45e2-4585-b8d0-db448f3474ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6825","nonce":2602640858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6827","nonce":2602640858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6831","nonce":2602640858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6829","nonce":2602640858}]},"public_addr":"172.21.15.179:6825/2602640858","cluster_addr":"172.21.15.179:6827/2602640858","heartbeat_back_addr":"172.21.15.179:6831/2602640858","heartbeat_front_addr":"172.21.15.179:6829/2602640858","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T02:59:56.074519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:19.358142+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:41.128966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:08.227054+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:31.442592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:52.751685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:19.251054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:40.698241+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:01.339150+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:27.955910+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:49.136980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:04:10.084982+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.101:6800/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:6801/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/604228291":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/4055187849":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/2844574324":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/1603248215":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6801/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6800/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/3307578182":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/373189792":"2023-07-19T02:57:56.243200+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:05:48.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.242+0000 7fad60e89700 1 -- 172.21.15.101:0/52270902 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fad440a11e0 msgr2=0x7fad440a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:48.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.242+0000 7fad60e89700 1 --2- 172.21.15.101:0/52270902 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fad440a11e0 0x7fad440a3690 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fad5c199510 tx=0x7fad50009260 comp rx=0 tx=0).stop 2023-07-18T03:05:48.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.242+0000 7fad60e89700 1 -- 172.21.15.101:0/52270902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad5c0ffe00 msgr2=0x7fad5c197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:48.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.242+0000 7fad60e89700 1 --2- 172.21.15.101:0/52270902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad5c0ffe00 0x7fad5c197ef0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fad4c0098e0 tx=0x7fad4c00f690 comp rx=0 tx=0).stop 2023-07-18T03:05:48.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.242+0000 7fad60e89700 1 -- 172.21.15.101:0/52270902 shutdown_connections 2023-07-18T03:05:48.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.242+0000 7fad60e89700 1 --2- 172.21.15.101:0/52270902 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fad440a11e0 0x7fad440a3690 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:48.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.242+0000 7fad60e89700 1 --2- 172.21.15.101:0/52270902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad5c0ffe00 0x7fad5c197ef0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:48.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.242+0000 7fad60e89700 1 --2- 172.21.15.101:0/52270902 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad5c104ea0 0x7fad5c19c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:48.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.242+0000 7fad60e89700 1 --2- 172.21.15.101:0/52270902 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fad5c1007a0 0x7fad5c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:48.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.242+0000 7fad60e89700 1 -- 172.21.15.101:0/52270902 >> 172.21.15.101:0/52270902 conn(0x7fad5c0fb660 msgr2=0x7fad5c10bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:48.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.242+0000 7fad60e89700 1 -- 172.21.15.101:0/52270902 shutdown_connections 2023-07-18T03:05:48.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:48.242+0000 7fad60e89700 1 -- 172.21.15.101:0/52270902 wait complete. 2023-07-18T03:05:48.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:48 smithi155 ceph-mon[128885]: pgmap v301: 129 pgs: 129 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.0 KiB/s wr, 9 op/s 2023-07-18T03:05:48.420 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:48 smithi101 ceph-mon[126379]: pgmap v301: 129 pgs: 129 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.0 KiB/s wr, 9 op/s 2023-07-18T03:05:48.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:48 smithi179 ceph-mon[129229]: pgmap v301: 129 pgs: 129 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.0 KiB/s wr, 9 op/s 2023-07-18T03:05:48.805 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph tell osd.0 flush_pg_stats 2023-07-18T03:05:48.806 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph tell osd.1 flush_pg_stats 2023-07-18T03:05:48.806 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph tell osd.2 flush_pg_stats 2023-07-18T03:05:48.807 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph tell osd.3 flush_pg_stats 2023-07-18T03:05:48.807 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph tell osd.4 flush_pg_stats 2023-07-18T03:05:48.808 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph tell osd.5 flush_pg_stats 2023-07-18T03:05:48.808 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph tell osd.6 flush_pg_stats 2023-07-18T03:05:48.809 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph tell osd.7 flush_pg_stats 2023-07-18T03:05:48.809 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph tell osd.8 flush_pg_stats 2023-07-18T03:05:48.810 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph tell osd.9 flush_pg_stats 2023-07-18T03:05:48.810 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph tell osd.10 flush_pg_stats 2023-07-18T03:05:48.811 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph tell osd.11 flush_pg_stats 2023-07-18T03:05:49.058 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:49 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/52270902' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:05:49.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:49 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/52270902' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:05:49.537 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:49 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/52270902' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:05:50.384 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:05:50.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:50 smithi179 ceph-mon[129229]: pgmap v302: 129 pgs: 129 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.8 KiB/s wr, 8 op/s 2023-07-18T03:05:50.479 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:50 smithi101 ceph-mon[126379]: pgmap v302: 129 pgs: 129 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.8 KiB/s wr, 8 op/s 2023-07-18T03:05:50.537 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:50 smithi155 ceph-mon[128885]: pgmap v302: 129 pgs: 129 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.8 KiB/s wr, 8 op/s 2023-07-18T03:05:50.623 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:05:50.875 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:05:50.926 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:05:51.681 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:05:51.785 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:05:52.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:52 smithi179 ceph-mon[129229]: pgmap v303: 129 pgs: 129 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-07-18T03:05:52.480 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:52 smithi101 ceph-mon[126379]: pgmap v303: 129 pgs: 129 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-07-18T03:05:52.537 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:52 smithi155 ceph-mon[128885]: pgmap v303: 129 pgs: 129 active+clean; 635 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-07-18T03:05:52.595 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:05:54.214 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:05:54.322 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:54 smithi101 ceph-mon[126379]: pgmap v304: 129 pgs: 129 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-07-18T03:05:54.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:54 smithi179 ceph-mon[129229]: pgmap v304: 129 pgs: 129 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-07-18T03:05:54.537 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:54 smithi155 ceph-mon[128885]: pgmap v304: 129 pgs: 129 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-07-18T03:05:54.539 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:05:54.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.691+0000 7f7631b10700 1 -- 172.21.15.101:0/2182446878 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f762c104ea0 msgr2=0x7f762c10ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:54.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.691+0000 7f7631b10700 1 --2- 172.21.15.101:0/2182446878 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f762c104ea0 0x7f762c10ca20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f762400b0d0 tx=0x7f762400b3e0 comp rx=0 tx=0).stop 2023-07-18T03:05:54.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.692+0000 7f7631b10700 1 -- 172.21.15.101:0/2182446878 shutdown_connections 2023-07-18T03:05:54.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.692+0000 7f7631b10700 1 --2- 172.21.15.101:0/2182446878 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f762c104ea0 0x7f762c10ca20 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:54.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.692+0000 7f7631b10700 1 --2- 172.21.15.101:0/2182446878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f762c0ffe00 0x7f762c1001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:54.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.692+0000 7f7631b10700 1 --2- 172.21.15.101:0/2182446878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f762c1007a0 0x7f762c104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:54.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.692+0000 7f7631b10700 1 -- 172.21.15.101:0/2182446878 >> 172.21.15.101:0/2182446878 conn(0x7f762c0fb660 msgr2=0x7f762c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:54.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.692+0000 7f7631b10700 1 -- 172.21.15.101:0/2182446878 shutdown_connections 2023-07-18T03:05:54.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.692+0000 7f7631b10700 1 -- 172.21.15.101:0/2182446878 wait complete. 2023-07-18T03:05:54.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.693+0000 7f7631b10700 1 Processor -- start 2023-07-18T03:05:54.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.693+0000 7f7631b10700 1 -- start start 2023-07-18T03:05:54.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.693+0000 7f7631b10700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f762c0ffe00 0x7f762c197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:54.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.693+0000 7f7631b10700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f762c1007a0 0x7f762c1984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:54.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.693+0000 7f7631b10700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f762c104ea0 0x7f762c19c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:54.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.693+0000 7f7631b10700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762c19cf40 con 0x7f762c104ea0 2023-07-18T03:05:54.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.693+0000 7f7631b10700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762c19d0b0 con 0x7f762c0ffe00 2023-07-18T03:05:54.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.693+0000 7f7631b10700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762c19d270 con 0x7f762c1007a0 2023-07-18T03:05:54.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.693+0000 7f762b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f762c0ffe00 0x7f762c197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:54.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.693+0000 7f762affd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f762c1007a0 0x7f762c1984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:54.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.693+0000 7f762b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f762c0ffe00 0x7f762c197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:53164/0 (socket says 172.21.15.101:53164) 2023-07-18T03:05:54.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.693+0000 7f762b7fe700 1 -- 172.21.15.101:0/2438461503 learned_addr learned my addr 172.21.15.101:0/2438461503 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:54.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.693+0000 7f762bfff700 1 --2- 172.21.15.101:0/2438461503 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f762c104ea0 0x7f762c19c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:54.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.693+0000 7f762b7fe700 1 -- 172.21.15.101:0/2438461503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f762c1007a0 msgr2=0x7f762c1984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:54.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.693+0000 7f762b7fe700 1 --2- 172.21.15.101:0/2438461503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f762c1007a0 0x7f762c1984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:54.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.693+0000 7f762b7fe700 1 -- 172.21.15.101:0/2438461503 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f762c104ea0 msgr2=0x7f762c19c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:54.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.693+0000 7f762b7fe700 1 --2- 172.21.15.101:0/2438461503 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f762c104ea0 0x7f762c19c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:54.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.693+0000 7f762b7fe700 1 -- 172.21.15.101:0/2438461503 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7624009d00 con 0x7f762c0ffe00 2023-07-18T03:05:54.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.693+0000 7f762affd700 1 --2- 172.21.15.101:0/2438461503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f762c1007a0 0x7f762c1984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:05:54.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.693+0000 7f762b7fe700 1 --2- 172.21.15.101:0/2438461503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f762c0ffe00 0x7f762c197f80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f761c00e9a0 tx=0x7f761c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:54.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.693+0000 7f7628ff9700 1 -- 172.21.15.101:0/2438461503 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761c009790 con 0x7f762c0ffe00 2023-07-18T03:05:54.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.693+0000 7f7631b10700 1 -- 172.21.15.101:0/2438461503 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f762c19d550 con 0x7f762c0ffe00 2023-07-18T03:05:54.813 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.693+0000 7f7628ff9700 1 -- 172.21.15.101:0/2438461503 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f761c010430 con 0x7f762c0ffe00 2023-07-18T03:05:54.813 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.693+0000 7f7628ff9700 1 -- 172.21.15.101:0/2438461503 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761c018680 con 0x7f762c0ffe00 2023-07-18T03:05:54.813 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.693+0000 7f7631b10700 1 -- 172.21.15.101:0/2438461503 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f762c1a4db0 con 0x7f762c0ffe00 2023-07-18T03:05:54.814 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.693+0000 7f7631b10700 1 -- 172.21.15.101:0/2438461503 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f762c10fd20 con 0x7f762c0ffe00 2023-07-18T03:05:54.814 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.698+0000 7f7628ff9700 1 -- 172.21.15.101:0/2438461503 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f761c0105a0 con 0x7f762c0ffe00 2023-07-18T03:05:54.814 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.700+0000 7f7628ff9700 1 --2- 172.21.15.101:0/2438461503 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f76140a1260 0x7f76140a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:54.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.700+0000 7f7628ff9700 1 -- 172.21.15.101:0/2438461503 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f761c014070 con 0x7f762c0ffe00 2023-07-18T03:05:54.816 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.700+0000 7f762affd700 1 --2- 172.21.15.101:0/2438461503 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f76140a1260 0x7f76140a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:54.816 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.700+0000 7f7628ff9700 1 --2- 172.21.15.101:0/2438461503 >> [v2:172.21.15.101:6818/958044931,v1:172.21.15.101:6819/958044931] conn(0x7f76140ac1f0 0x7f76140ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:54.816 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.700+0000 7f7628ff9700 1 -- 172.21.15.101:0/2438461503 --> [v2:172.21.15.101:6818/958044931,v1:172.21.15.101:6819/958044931] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f76140aed50 con 0x7f76140ac1f0 2023-07-18T03:05:54.816 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.700+0000 7f7628ff9700 1 -- 172.21.15.101:0/2438461503 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f761c0cd640 con 0x7f762c0ffe00 2023-07-18T03:05:54.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.700+0000 7f762bfff700 1 --2- 172.21.15.101:0/2438461503 >> [v2:172.21.15.101:6818/958044931,v1:172.21.15.101:6819/958044931] conn(0x7f76140ac1f0 0x7f76140ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:54.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.700+0000 7f762affd700 1 --2- 172.21.15.101:0/2438461503 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f76140a1260 0x7f76140a3710 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f762c1995a0 tx=0x7f7620009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:54.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.700+0000 7f762bfff700 1 --2- 172.21.15.101:0/2438461503 >> [v2:172.21.15.101:6818/958044931,v1:172.21.15.101:6819/958044931] conn(0x7f76140ac1f0 0x7f76140ae680 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:54.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.700+0000 7f7628ff9700 1 -- 172.21.15.101:0/2438461503 <== osd.6 v2:172.21.15.101:6818/958044931 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f76140aed50 con 0x7f76140ac1f0 2023-07-18T03:05:54.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.721+0000 7f7631b10700 1 -- 172.21.15.101:0/2438461503 --> [v2:172.21.15.101:6818/958044931,v1:172.21.15.101:6819/958044931] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f762c04e210 con 0x7f76140ac1f0 2023-07-18T03:05:54.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.721+0000 7f7628ff9700 1 -- 172.21.15.101:0/2438461503 <== osd.6 v2:172.21.15.101:6818/958044931 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f762c04e210 con 0x7f76140ac1f0 2023-07-18T03:05:54.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.721+0000 7f7631b10700 1 -- 172.21.15.101:0/2438461503 >> [v2:172.21.15.101:6818/958044931,v1:172.21.15.101:6819/958044931] conn(0x7f76140ac1f0 msgr2=0x7f76140ae680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:54.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.721+0000 7f7631b10700 1 --2- 172.21.15.101:0/2438461503 >> [v2:172.21.15.101:6818/958044931,v1:172.21.15.101:6819/958044931] conn(0x7f76140ac1f0 0x7f76140ae680 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:54.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.721+0000 7f7631b10700 1 -- 172.21.15.101:0/2438461503 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f76140a1260 msgr2=0x7f76140a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:54.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.721+0000 7f7631b10700 1 --2- 172.21.15.101:0/2438461503 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f76140a1260 0x7f76140a3710 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f762c1995a0 tx=0x7f7620009260 comp rx=0 tx=0).stop 2023-07-18T03:05:54.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.721+0000 7f7631b10700 1 -- 172.21.15.101:0/2438461503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f762c0ffe00 msgr2=0x7f762c197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:54.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.721+0000 7f7631b10700 1 --2- 172.21.15.101:0/2438461503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f762c0ffe00 0x7f762c197f80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f761c00e9a0 tx=0x7f761c00ecb0 comp rx=0 tx=0).stop 2023-07-18T03:05:54.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.721+0000 7f762bfff700 1 -- 172.21.15.101:0/2438461503 reap_dead start 2023-07-18T03:05:54.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.721+0000 7f7631b10700 1 -- 172.21.15.101:0/2438461503 shutdown_connections 2023-07-18T03:05:54.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.721+0000 7f7631b10700 1 -- 172.21.15.101:0/2438461503 >> 172.21.15.101:0/2438461503 conn(0x7f762c0fb660 msgr2=0x7f762c0fc0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:54.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.721+0000 7f7631b10700 1 -- 172.21.15.101:0/2438461503 shutdown_connections 2023-07-18T03:05:54.822 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:54.721+0000 7f7631b10700 1 -- 172.21.15.101:0/2438461503 wait complete. 2023-07-18T03:05:54.890 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:05:55.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.017+0000 7f1c4612a700 1 -- 172.21.15.101:0/335197258 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1c40104ea0 msgr2=0x7f1c4010ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:55.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.017+0000 7f1c4612a700 1 --2- 172.21.15.101:0/335197258 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1c40104ea0 0x7f1c4010ca20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1c3800b0d0 tx=0x7f1c3800b3e0 comp rx=0 tx=0).stop 2023-07-18T03:05:55.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.017+0000 7f1c4612a700 1 -- 172.21.15.101:0/335197258 shutdown_connections 2023-07-18T03:05:55.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.017+0000 7f1c4612a700 1 --2- 172.21.15.101:0/335197258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c400ffe00 0x7f1c401001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:55.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.017+0000 7f1c4612a700 1 --2- 172.21.15.101:0/335197258 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1c40104ea0 0x7f1c4010ca20 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:55.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.017+0000 7f1c4612a700 1 --2- 172.21.15.101:0/335197258 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c401007a0 0x7f1c40104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:55.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.017+0000 7f1c4612a700 1 -- 172.21.15.101:0/335197258 >> 172.21.15.101:0/335197258 conn(0x7f1c400fb660 msgr2=0x7f1c400fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:55.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.017+0000 7f1c4612a700 1 -- 172.21.15.101:0/335197258 shutdown_connections 2023-07-18T03:05:55.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.017+0000 7f1c4612a700 1 -- 172.21.15.101:0/335197258 wait complete. 2023-07-18T03:05:55.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.018+0000 7f1c4612a700 1 Processor -- start 2023-07-18T03:05:55.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.019+0000 7f1c4612a700 1 -- start start 2023-07-18T03:05:55.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.019+0000 7f1c4612a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c400ffe00 0x7f1c40195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:55.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.019+0000 7f1c4612a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1c401007a0 0x7f1c401962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:55.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.019+0000 7f1c4612a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c40104ea0 0x7f1c4019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:55.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.019+0000 7f1c4612a700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4019ad20 con 0x7f1c401007a0 2023-07-18T03:05:55.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.019+0000 7f1c4612a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4019ae90 con 0x7f1c40104ea0 2023-07-18T03:05:55.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.019+0000 7f1c4612a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4019b050 con 0x7f1c400ffe00 2023-07-18T03:05:55.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.019+0000 7f1c3effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1c401007a0 0x7f1c401962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:55.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.019+0000 7f1c3effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1c401007a0 0x7f1c401962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:34374/0 (socket says 172.21.15.101:34374) 2023-07-18T03:05:55.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.019+0000 7f1c3effd700 1 -- 172.21.15.101:0/707668412 learned_addr learned my addr 172.21.15.101:0/707668412 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:55.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.020+0000 7f1c3f7fe700 1 --2- 172.21.15.101:0/707668412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c400ffe00 0x7f1c40195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:55.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.020+0000 7f1c3ffff700 1 --2- 172.21.15.101:0/707668412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c40104ea0 0x7f1c4019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:55.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.020+0000 7f1c3effd700 1 -- 172.21.15.101:0/707668412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c400ffe00 msgr2=0x7f1c40195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:55.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.020+0000 7f1c3effd700 1 --2- 172.21.15.101:0/707668412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c400ffe00 0x7f1c40195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:55.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.020+0000 7f1c3effd700 1 -- 172.21.15.101:0/707668412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c40104ea0 msgr2=0x7f1c4019a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:55.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.020+0000 7f1c3effd700 1 --2- 172.21.15.101:0/707668412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c40104ea0 0x7f1c4019a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:55.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.020+0000 7f1c3effd700 1 -- 172.21.15.101:0/707668412 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c38009d00 con 0x7f1c401007a0 2023-07-18T03:05:55.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.020+0000 7f1c3effd700 1 --2- 172.21.15.101:0/707668412 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1c401007a0 0x7f1c401962a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1c3400d980 tx=0x7f1c3400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:55.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.020+0000 7f1c3cff9700 1 -- 172.21.15.101:0/707668412 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3400f890 con 0x7f1c401007a0 2023-07-18T03:05:55.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.020+0000 7f1c3cff9700 1 -- 172.21.15.101:0/707668412 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c34010460 con 0x7f1c401007a0 2023-07-18T03:05:55.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.020+0000 7f1c3cff9700 1 -- 172.21.15.101:0/707668412 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3400e610 con 0x7f1c401007a0 2023-07-18T03:05:55.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.020+0000 7f1c4612a700 1 -- 172.21.15.101:0/707668412 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c4019b330 con 0x7f1c401007a0 2023-07-18T03:05:55.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.021+0000 7f1c4612a700 1 -- 172.21.15.101:0/707668412 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c401a2b90 con 0x7f1c401007a0 2023-07-18T03:05:55.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.021+0000 7f1c4612a700 1 -- 172.21.15.101:0/707668412 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1c4010fd50 con 0x7f1c401007a0 2023-07-18T03:05:55.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.023+0000 7f1c3cff9700 1 -- 172.21.15.101:0/707668412 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f1c340105d0 con 0x7f1c401007a0 2023-07-18T03:05:55.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.025+0000 7f1c3cff9700 1 --2- 172.21.15.101:0/707668412 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f1c280a1260 0x7f1c280a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:55.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.025+0000 7f1c3cff9700 1 -- 172.21.15.101:0/707668412 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f1c340ccaf0 con 0x7f1c401007a0 2023-07-18T03:05:55.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.025+0000 7f1c3f7fe700 1 --2- 172.21.15.101:0/707668412 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f1c280a1260 0x7f1c280a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:55.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.025+0000 7f1c3cff9700 1 --2- 172.21.15.101:0/707668412 >> [v2:172.21.15.179:6824/2602640858,v1:172.21.15.179:6825/2602640858] conn(0x7f1c280ac1f0 0x7f1c280ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:55.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.025+0000 7f1c3cff9700 1 -- 172.21.15.101:0/707668412 --> [v2:172.21.15.179:6824/2602640858,v1:172.21.15.179:6825/2602640858] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1c280aed50 con 0x7f1c280ac1f0 2023-07-18T03:05:55.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.025+0000 7f1c3cff9700 1 -- 172.21.15.101:0/707668412 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c340cce60 con 0x7f1c401007a0 2023-07-18T03:05:55.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.026+0000 7f1c3ffff700 1 --2- 172.21.15.101:0/707668412 >> [v2:172.21.15.179:6824/2602640858,v1:172.21.15.179:6825/2602640858] conn(0x7f1c280ac1f0 0x7f1c280ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:55.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.026+0000 7f1c3f7fe700 1 --2- 172.21.15.101:0/707668412 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f1c280a1260 0x7f1c280a3710 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1c30009c60 tx=0x7f1c30009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:55.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.026+0000 7f1c3ffff700 1 --2- 172.21.15.101:0/707668412 >> [v2:172.21.15.179:6824/2602640858,v1:172.21.15.179:6825/2602640858] conn(0x7f1c280ac1f0 0x7f1c280ae680 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:55.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.027+0000 7f1c3cff9700 1 -- 172.21.15.101:0/707668412 <== osd.11 v2:172.21.15.179:6824/2602640858 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f1c280aed50 con 0x7f1c280ac1f0 2023-07-18T03:05:55.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.043+0000 7f1c4612a700 1 -- 172.21.15.101:0/707668412 --> [v2:172.21.15.179:6824/2602640858,v1:172.21.15.179:6825/2602640858] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1c4004e210 con 0x7f1c280ac1f0 2023-07-18T03:05:55.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.044+0000 7f1c3cff9700 1 -- 172.21.15.101:0/707668412 <== osd.11 v2:172.21.15.179:6824/2602640858 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1c4004e210 con 0x7f1c280ac1f0 2023-07-18T03:05:55.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.044+0000 7f1c4612a700 1 -- 172.21.15.101:0/707668412 >> [v2:172.21.15.179:6824/2602640858,v1:172.21.15.179:6825/2602640858] conn(0x7f1c280ac1f0 msgr2=0x7f1c280ae680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:55.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.044+0000 7f1c4612a700 1 --2- 172.21.15.101:0/707668412 >> [v2:172.21.15.179:6824/2602640858,v1:172.21.15.179:6825/2602640858] conn(0x7f1c280ac1f0 0x7f1c280ae680 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:55.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.044+0000 7f1c4612a700 1 -- 172.21.15.101:0/707668412 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f1c280a1260 msgr2=0x7f1c280a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:55.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.044+0000 7f1c4612a700 1 --2- 172.21.15.101:0/707668412 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f1c280a1260 0x7f1c280a3710 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1c30009c60 tx=0x7f1c30009260 comp rx=0 tx=0).stop 2023-07-18T03:05:55.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.044+0000 7f1c4612a700 1 -- 172.21.15.101:0/707668412 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1c401007a0 msgr2=0x7f1c401962a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:55.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.044+0000 7f1c4612a700 1 --2- 172.21.15.101:0/707668412 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1c401007a0 0x7f1c401962a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1c3400d980 tx=0x7f1c3400dc90 comp rx=0 tx=0).stop 2023-07-18T03:05:55.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.044+0000 7f1c3ffff700 1 -- 172.21.15.101:0/707668412 reap_dead start 2023-07-18T03:05:55.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.045+0000 7f1c4612a700 1 -- 172.21.15.101:0/707668412 shutdown_connections 2023-07-18T03:05:55.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.045+0000 7f1c4612a700 1 -- 172.21.15.101:0/707668412 >> 172.21.15.101:0/707668412 conn(0x7f1c400fb660 msgr2=0x7f1c400fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:55.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.045+0000 7f1c4612a700 1 -- 172.21.15.101:0/707668412 shutdown_connections 2023-07-18T03:05:55.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:55.045+0000 7f1c4612a700 1 -- 172.21.15.101:0/707668412 wait complete. 2023-07-18T03:05:55.037 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:05:55.037 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:05:56.335 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:56 smithi101 ceph-mon[126379]: pgmap v305: 129 pgs: 129 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 5 op/s 2023-07-18T03:05:56.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:56 smithi179 ceph-mon[129229]: pgmap v305: 129 pgs: 129 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 5 op/s 2023-07-18T03:05:56.537 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:56 smithi155 ceph-mon[128885]: pgmap v305: 129 pgs: 129 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 5 op/s 2023-07-18T03:05:57.133 INFO:teuthology.orchestra.run.smithi101.stdout:197568495661 2023-07-18T03:05:57.134 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.6 2023-07-18T03:05:57.548 INFO:teuthology.orchestra.run.smithi101.stdout:335007449111 2023-07-18T03:05:57.549 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.11 2023-07-18T03:05:57.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.759+0000 7f65046c2700 1 -- 172.21.15.101:0/716506420 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f64fc0ffe00 msgr2=0x7f64fc1001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:57.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.759+0000 7f65046c2700 1 --2- 172.21.15.101:0/716506420 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f64fc0ffe00 0x7f64fc1001d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f64f80099c0 tx=0x7f64f8009cd0 comp rx=0 tx=0).stop 2023-07-18T03:05:57.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.759+0000 7f65046c2700 1 -- 172.21.15.101:0/716506420 shutdown_connections 2023-07-18T03:05:57.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.759+0000 7f65046c2700 1 --2- 172.21.15.101:0/716506420 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64fc104ea0 0x7f64fc10ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:57.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.759+0000 7f65046c2700 1 --2- 172.21.15.101:0/716506420 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f64fc0ffe00 0x7f64fc1001d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:57.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.759+0000 7f65046c2700 1 --2- 172.21.15.101:0/716506420 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64fc1007a0 0x7f64fc104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:57.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.759+0000 7f65046c2700 1 -- 172.21.15.101:0/716506420 >> 172.21.15.101:0/716506420 conn(0x7f64fc0fb660 msgr2=0x7f64fc0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:57.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.759+0000 7f65046c2700 1 -- 172.21.15.101:0/716506420 shutdown_connections 2023-07-18T03:05:57.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.759+0000 7f65046c2700 1 -- 172.21.15.101:0/716506420 wait complete. 2023-07-18T03:05:57.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.761+0000 7f65046c2700 1 Processor -- start 2023-07-18T03:05:57.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.761+0000 7f65046c2700 1 -- start start 2023-07-18T03:05:57.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.762+0000 7f65046c2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64fc0ffe00 0x7f64fc199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:57.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.762+0000 7f65046c2700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f64fc1007a0 0x7f64fc199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:57.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.762+0000 7f65046c2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64fc104ea0 0x7f64fc193900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:57.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.762+0000 7f65046c2700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc193f70 con 0x7f64fc1007a0 2023-07-18T03:05:57.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.762+0000 7f65046c2700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc1940e0 con 0x7f64fc104ea0 2023-07-18T03:05:57.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.762+0000 7f65046c2700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc1942a0 con 0x7f64fc0ffe00 2023-07-18T03:05:57.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.762+0000 7f650245e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64fc0ffe00 0x7f64fc199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:57.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.762+0000 7f650245e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64fc0ffe00 0x7f64fc199880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:42108/0 (socket says 172.21.15.101:42108) 2023-07-18T03:05:57.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.762+0000 7f650245e700 1 -- 172.21.15.101:0/1633022350 learned_addr learned my addr 172.21.15.101:0/1633022350 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:57.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.762+0000 7f6501c5d700 1 --2- 172.21.15.101:0/1633022350 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f64fc1007a0 0x7f64fc199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:57.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.762+0000 7f6502c5f700 1 --2- 172.21.15.101:0/1633022350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64fc104ea0 0x7f64fc193900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:57.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.762+0000 7f6501c5d700 1 -- 172.21.15.101:0/1633022350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64fc0ffe00 msgr2=0x7f64fc199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:57.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.762+0000 7f6501c5d700 1 --2- 172.21.15.101:0/1633022350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64fc0ffe00 0x7f64fc199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:57.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.762+0000 7f6501c5d700 1 -- 172.21.15.101:0/1633022350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64fc104ea0 msgr2=0x7f64fc193900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:57.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.762+0000 7f6501c5d700 1 --2- 172.21.15.101:0/1633022350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64fc104ea0 0x7f64fc193900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:57.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.763+0000 7f6501c5d700 1 -- 172.21.15.101:0/1633022350 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f80095f0 con 0x7f64fc1007a0 2023-07-18T03:05:57.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.763+0000 7f6501c5d700 1 --2- 172.21.15.101:0/1633022350 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f64fc1007a0 0x7f64fc199dc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f64ec00e9c0 tx=0x7f64ec00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:57.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.763+0000 7f64f37fe700 1 -- 172.21.15.101:0/1633022350 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ec004220 con 0x7f64fc1007a0 2023-07-18T03:05:57.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.763+0000 7f65046c2700 1 -- 172.21.15.101:0/1633022350 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64fc194580 con 0x7f64fc1007a0 2023-07-18T03:05:57.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.763+0000 7f64f37fe700 1 -- 172.21.15.101:0/1633022350 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64ec004cf0 con 0x7f64fc1007a0 2023-07-18T03:05:57.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.763+0000 7f64f37fe700 1 -- 172.21.15.101:0/1633022350 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ec00ce40 con 0x7f64fc1007a0 2023-07-18T03:05:57.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.763+0000 7f65046c2700 1 -- 172.21.15.101:0/1633022350 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64fc1a0870 con 0x7f64fc1007a0 2023-07-18T03:05:57.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.765+0000 7f65046c2700 1 -- 172.21.15.101:0/1633022350 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f64fc04e210 con 0x7f64fc1007a0 2023-07-18T03:05:57.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.766+0000 7f64f37fe700 1 -- 172.21.15.101:0/1633022350 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f64ec004750 con 0x7f64fc1007a0 2023-07-18T03:05:57.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.768+0000 7f64f37fe700 1 --2- 172.21.15.101:0/1633022350 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f64e80a55f0 0x7f64e80a7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:57.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.768+0000 7f64f37fe700 1 -- 172.21.15.101:0/1633022350 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f64ec014070 con 0x7f64fc1007a0 2023-07-18T03:05:57.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.768+0000 7f64f37fe700 1 --2- 172.21.15.101:0/1633022350 >> [v2:172.21.15.155:6800/1030229579,v1:172.21.15.155:6801/1030229579] conn(0x7f64e80b0580 0x7f64e80b2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:57.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.768+0000 7f650245e700 1 --2- 172.21.15.101:0/1633022350 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f64e80a55f0 0x7f64e80a7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:57.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.768+0000 7f64f37fe700 1 -- 172.21.15.101:0/1633022350 --> [v2:172.21.15.155:6800/1030229579,v1:172.21.15.155:6801/1030229579] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f64e80b30e0 con 0x7f64e80b0580 2023-07-18T03:05:57.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.768+0000 7f64f37fe700 1 -- 172.21.15.101:0/1633022350 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64ec0d12a0 con 0x7f64fc1007a0 2023-07-18T03:05:57.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.769+0000 7f6502c5f700 1 --2- 172.21.15.101:0/1633022350 >> [v2:172.21.15.155:6800/1030229579,v1:172.21.15.155:6801/1030229579] conn(0x7f64e80b0580 0x7f64e80b2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:57.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.769+0000 7f650245e700 1 --2- 172.21.15.101:0/1633022350 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f64e80a55f0 0x7f64e80a7aa0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f64f80036c0 tx=0x7f64f8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:57.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.770+0000 7f6502c5f700 1 --2- 172.21.15.101:0/1633022350 >> [v2:172.21.15.155:6800/1030229579,v1:172.21.15.155:6801/1030229579] conn(0x7f64e80b0580 0x7f64e80b2a10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:57.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.770+0000 7f64f37fe700 1 -- 172.21.15.101:0/1633022350 <== osd.1 v2:172.21.15.155:6800/1030229579 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f64e80b30e0 con 0x7f64e80b0580 2023-07-18T03:05:57.774 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.789+0000 7f65046c2700 1 -- 172.21.15.101:0/1633022350 --> [v2:172.21.15.155:6800/1030229579,v1:172.21.15.155:6801/1030229579] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f64fc0fcb20 con 0x7f64e80b0580 2023-07-18T03:05:57.774 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.789+0000 7f64f37fe700 1 -- 172.21.15.101:0/1633022350 <== osd.1 v2:172.21.15.155:6800/1030229579 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f64fc0fcb20 con 0x7f64e80b0580 2023-07-18T03:05:57.775 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.790+0000 7f65046c2700 1 -- 172.21.15.101:0/1633022350 >> [v2:172.21.15.155:6800/1030229579,v1:172.21.15.155:6801/1030229579] conn(0x7f64e80b0580 msgr2=0x7f64e80b2a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:57.775 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.790+0000 7f65046c2700 1 --2- 172.21.15.101:0/1633022350 >> [v2:172.21.15.155:6800/1030229579,v1:172.21.15.155:6801/1030229579] conn(0x7f64e80b0580 0x7f64e80b2a10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:57.775 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.790+0000 7f65046c2700 1 -- 172.21.15.101:0/1633022350 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f64e80a55f0 msgr2=0x7f64e80a7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:57.775 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.790+0000 7f65046c2700 1 --2- 172.21.15.101:0/1633022350 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f64e80a55f0 0x7f64e80a7aa0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f64f80036c0 tx=0x7f64f8005fb0 comp rx=0 tx=0).stop 2023-07-18T03:05:57.776 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.790+0000 7f65046c2700 1 -- 172.21.15.101:0/1633022350 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f64fc1007a0 msgr2=0x7f64fc199dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:57.776 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.790+0000 7f65046c2700 1 --2- 172.21.15.101:0/1633022350 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f64fc1007a0 0x7f64fc199dc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f64ec00e9c0 tx=0x7f64ec00ecd0 comp rx=0 tx=0).stop 2023-07-18T03:05:57.776 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.790+0000 7f6502c5f700 1 -- 172.21.15.101:0/1633022350 reap_dead start 2023-07-18T03:05:57.776 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.790+0000 7f65046c2700 1 -- 172.21.15.101:0/1633022350 shutdown_connections 2023-07-18T03:05:57.776 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.790+0000 7f65046c2700 1 -- 172.21.15.101:0/1633022350 >> 172.21.15.101:0/1633022350 conn(0x7f64fc0fb660 msgr2=0x7f64fc0fc190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:57.776 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.790+0000 7f65046c2700 1 -- 172.21.15.101:0/1633022350 shutdown_connections 2023-07-18T03:05:57.776 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:57.790+0000 7f65046c2700 1 -- 172.21.15.101:0/1633022350 wait complete. 2023-07-18T03:05:58.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.098+0000 7efd32a88700 1 -- 172.21.15.101:0/502719781 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efd2c1007a0 msgr2=0x7efd2c104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:58.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.098+0000 7efd32a88700 1 --2- 172.21.15.101:0/502719781 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efd2c1007a0 0x7efd2c104770 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efd180099c0 tx=0x7efd18009cd0 comp rx=0 tx=0).stop 2023-07-18T03:05:58.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.098+0000 7efd32a88700 1 -- 172.21.15.101:0/502719781 shutdown_connections 2023-07-18T03:05:58.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.098+0000 7efd32a88700 1 --2- 172.21.15.101:0/502719781 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd2c104ea0 0x7efd2c10ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:58.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.098+0000 7efd32a88700 1 --2- 172.21.15.101:0/502719781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd2c0ffe00 0x7efd2c1001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:58.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.098+0000 7efd32a88700 1 --2- 172.21.15.101:0/502719781 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efd2c1007a0 0x7efd2c104770 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:58.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.098+0000 7efd32a88700 1 -- 172.21.15.101:0/502719781 >> 172.21.15.101:0/502719781 conn(0x7efd2c0fb660 msgr2=0x7efd2c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:58.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.098+0000 7efd32a88700 1 -- 172.21.15.101:0/502719781 shutdown_connections 2023-07-18T03:05:58.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.098+0000 7efd32a88700 1 -- 172.21.15.101:0/502719781 wait complete. 2023-07-18T03:05:58.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.099+0000 7efd32a88700 1 Processor -- start 2023-07-18T03:05:58.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.099+0000 7efd32a88700 1 -- start start 2023-07-18T03:05:58.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.100+0000 7efd32a88700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efd2c0ffe00 0x7efd2c072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:58.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.100+0000 7efd32a88700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd2c1007a0 0x7efd2c072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:58.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.100+0000 7efd32a88700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd2c104ea0 0x7efd2c0732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:58.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.100+0000 7efd32a88700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd2c1a4130 con 0x7efd2c0ffe00 2023-07-18T03:05:58.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.100+0000 7efd32a88700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd2c1a42a0 con 0x7efd2c1007a0 2023-07-18T03:05:58.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.100+0000 7efd32a88700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd2c1a4460 con 0x7efd2c104ea0 2023-07-18T03:05:58.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.100+0000 7efd30824700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efd2c0ffe00 0x7efd2c072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:58.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.100+0000 7efd2bfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd2c1007a0 0x7efd2c072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:58.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.100+0000 7efd2bfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd2c1007a0 0x7efd2c072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:33120/0 (socket says 172.21.15.101:33120) 2023-07-18T03:05:58.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.100+0000 7efd2bfff700 1 -- 172.21.15.101:0/3259551684 learned_addr learned my addr 172.21.15.101:0/3259551684 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:58.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.100+0000 7efd30824700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efd2c0ffe00 0x7efd2c072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:53806/0 (socket says 172.21.15.101:53806) 2023-07-18T03:05:58.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.100+0000 7efd31025700 1 --2- 172.21.15.101:0/3259551684 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd2c104ea0 0x7efd2c0732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:58.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.101+0000 7efd2bfff700 1 -- 172.21.15.101:0/3259551684 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd2c104ea0 msgr2=0x7efd2c0732d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:58.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.101+0000 7efd2bfff700 1 --2- 172.21.15.101:0/3259551684 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd2c104ea0 0x7efd2c0732d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:58.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.101+0000 7efd2bfff700 1 -- 172.21.15.101:0/3259551684 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efd2c0ffe00 msgr2=0x7efd2c072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:58.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.101+0000 7efd2bfff700 1 --2- 172.21.15.101:0/3259551684 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efd2c0ffe00 0x7efd2c072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:58.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.101+0000 7efd2bfff700 1 -- 172.21.15.101:0/3259551684 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd180095f0 con 0x7efd2c1007a0 2023-07-18T03:05:58.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.101+0000 7efd30824700 1 --2- 172.21.15.101:0/3259551684 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efd2c0ffe00 0x7efd2c072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:05:58.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.101+0000 7efd2bfff700 1 --2- 172.21.15.101:0/3259551684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd2c1007a0 0x7efd2c072d50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efd18009fd0 tx=0x7efd18009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:58.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.101+0000 7efd29ffb700 1 -- 172.21.15.101:0/3259551684 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1801d070 con 0x7efd2c1007a0 2023-07-18T03:05:58.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.101+0000 7efd32a88700 1 -- 172.21.15.101:0/3259551684 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd2c1a46e0 con 0x7efd2c1007a0 2023-07-18T03:05:58.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.101+0000 7efd29ffb700 1 -- 172.21.15.101:0/3259551684 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd18005030 con 0x7efd2c1007a0 2023-07-18T03:05:58.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.101+0000 7efd29ffb700 1 -- 172.21.15.101:0/3259551684 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd18004a90 con 0x7efd2c1007a0 2023-07-18T03:05:58.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.101+0000 7efd32a88700 1 -- 172.21.15.101:0/3259551684 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd2c1a4b50 con 0x7efd2c1007a0 2023-07-18T03:05:58.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.102+0000 7efd32a88700 1 -- 172.21.15.101:0/3259551684 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efd2c10fd20 con 0x7efd2c1007a0 2023-07-18T03:05:58.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.104+0000 7efd29ffb700 1 -- 172.21.15.101:0/3259551684 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7efd180051a0 con 0x7efd2c1007a0 2023-07-18T03:05:58.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.106+0000 7efd29ffb700 1 --2- 172.21.15.101:0/3259551684 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7efd1c0a11e0 0x7efd1c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:58.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.106+0000 7efd29ffb700 1 -- 172.21.15.101:0/3259551684 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7efd180cdc80 con 0x7efd2c1007a0 2023-07-18T03:05:58.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.106+0000 7efd30824700 1 --2- 172.21.15.101:0/3259551684 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7efd1c0a11e0 0x7efd1c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:58.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.106+0000 7efd29ffb700 1 --2- 172.21.15.101:0/3259551684 >> [v2:172.21.15.179:6816/3520612559,v1:172.21.15.179:6817/3520612559] conn(0x7efd1c0ac170 0x7efd1c0ae600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:58.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.106+0000 7efd29ffb700 1 -- 172.21.15.101:0/3259551684 --> [v2:172.21.15.179:6816/3520612559,v1:172.21.15.179:6817/3520612559] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efd1c0aecd0 con 0x7efd1c0ac170 2023-07-18T03:05:58.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.106+0000 7efd29ffb700 1 -- 172.21.15.101:0/3259551684 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd180cdff0 con 0x7efd2c1007a0 2023-07-18T03:05:58.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.106+0000 7efd31025700 1 --2- 172.21.15.101:0/3259551684 >> [v2:172.21.15.179:6816/3520612559,v1:172.21.15.179:6817/3520612559] conn(0x7efd1c0ac170 0x7efd1c0ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:58.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.107+0000 7efd30824700 1 --2- 172.21.15.101:0/3259551684 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7efd1c0a11e0 0x7efd1c0a3690 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7efd20011490 tx=0x7efd20009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:58.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.107+0000 7efd31025700 1 --2- 172.21.15.101:0/3259551684 >> [v2:172.21.15.179:6816/3520612559,v1:172.21.15.179:6817/3520612559] conn(0x7efd1c0ac170 0x7efd1c0ae600 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:58.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.108+0000 7efd29ffb700 1 -- 172.21.15.101:0/3259551684 <== osd.8 v2:172.21.15.179:6816/3520612559 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7efd1c0aecd0 con 0x7efd1c0ac170 2023-07-18T03:05:58.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.139+0000 7efd32a88700 1 -- 172.21.15.101:0/3259551684 --> [v2:172.21.15.179:6816/3520612559,v1:172.21.15.179:6817/3520612559] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7efd2c04e210 con 0x7efd1c0ac170 2023-07-18T03:05:58.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.140+0000 7efd29ffb700 1 -- 172.21.15.101:0/3259551684 <== osd.8 v2:172.21.15.179:6816/3520612559 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7efd2c04e210 con 0x7efd1c0ac170 2023-07-18T03:05:58.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.140+0000 7efd32a88700 1 -- 172.21.15.101:0/3259551684 >> [v2:172.21.15.179:6816/3520612559,v1:172.21.15.179:6817/3520612559] conn(0x7efd1c0ac170 msgr2=0x7efd1c0ae600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:58.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.140+0000 7efd32a88700 1 --2- 172.21.15.101:0/3259551684 >> [v2:172.21.15.179:6816/3520612559,v1:172.21.15.179:6817/3520612559] conn(0x7efd1c0ac170 0x7efd1c0ae600 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:58.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.140+0000 7efd32a88700 1 -- 172.21.15.101:0/3259551684 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7efd1c0a11e0 msgr2=0x7efd1c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:58.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.140+0000 7efd32a88700 1 --2- 172.21.15.101:0/3259551684 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7efd1c0a11e0 0x7efd1c0a3690 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7efd20011490 tx=0x7efd20009260 comp rx=0 tx=0).stop 2023-07-18T03:05:58.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.140+0000 7efd32a88700 1 -- 172.21.15.101:0/3259551684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd2c1007a0 msgr2=0x7efd2c072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:58.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.140+0000 7efd32a88700 1 --2- 172.21.15.101:0/3259551684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd2c1007a0 0x7efd2c072d50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efd18009fd0 tx=0x7efd18009580 comp rx=0 tx=0).stop 2023-07-18T03:05:58.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.141+0000 7efd31025700 1 -- 172.21.15.101:0/3259551684 reap_dead start 2023-07-18T03:05:58.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.141+0000 7efd32a88700 1 -- 172.21.15.101:0/3259551684 shutdown_connections 2023-07-18T03:05:58.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.141+0000 7efd32a88700 1 -- 172.21.15.101:0/3259551684 >> 172.21.15.101:0/3259551684 conn(0x7efd2c0fb660 msgr2=0x7efd2c0fc0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:58.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.141+0000 7efd32a88700 1 -- 172.21.15.101:0/3259551684 shutdown_connections 2023-07-18T03:05:58.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:58.141+0000 7efd32a88700 1 -- 172.21.15.101:0/3259551684 wait complete. 2023-07-18T03:05:58.442 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:58 smithi179 ceph-mon[129229]: pgmap v306: 129 pgs: 129 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2023-07-18T03:05:58.480 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:58 smithi101 ceph-mon[126379]: pgmap v306: 129 pgs: 129 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2023-07-18T03:05:58.537 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:58 smithi155 ceph-mon[128885]: pgmap v306: 129 pgs: 129 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2023-07-18T03:05:59.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.275+0000 7f7e5cca1700 1 -- 172.21.15.101:0/1703376541 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7e580ffe00 msgr2=0x7f7e581001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:59.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.275+0000 7f7e5cca1700 1 --2- 172.21.15.101:0/1703376541 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7e580ffe00 0x7f7e581001d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7e480099c0 tx=0x7f7e48009cd0 comp rx=0 tx=0).stop 2023-07-18T03:05:59.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.275+0000 7f7e5cca1700 1 -- 172.21.15.101:0/1703376541 shutdown_connections 2023-07-18T03:05:59.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.275+0000 7f7e5cca1700 1 --2- 172.21.15.101:0/1703376541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e58104ea0 0x7f7e5810ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:59.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.275+0000 7f7e5cca1700 1 --2- 172.21.15.101:0/1703376541 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7e580ffe00 0x7f7e581001d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:59.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.275+0000 7f7e5cca1700 1 --2- 172.21.15.101:0/1703376541 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e581007a0 0x7f7e58104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:59.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.275+0000 7f7e5cca1700 1 -- 172.21.15.101:0/1703376541 >> 172.21.15.101:0/1703376541 conn(0x7f7e580fb660 msgr2=0x7f7e580fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:59.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.275+0000 7f7e5cca1700 1 -- 172.21.15.101:0/1703376541 shutdown_connections 2023-07-18T03:05:59.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.275+0000 7f7e5cca1700 1 -- 172.21.15.101:0/1703376541 wait complete. 2023-07-18T03:05:59.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.276+0000 7f7e5cca1700 1 Processor -- start 2023-07-18T03:05:59.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.276+0000 7f7e5cca1700 1 -- start start 2023-07-18T03:05:59.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.277+0000 7f7e5cca1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e580ffe00 0x7f7e58197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:59.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.277+0000 7f7e5cca1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e581007a0 0x7f7e581983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:59.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.277+0000 7f7e5cca1700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7e58104ea0 0x7f7e5819c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:59.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.277+0000 7f7e5cca1700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5819ce30 con 0x7f7e58104ea0 2023-07-18T03:05:59.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.277+0000 7f7e5cca1700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5819cfa0 con 0x7f7e580ffe00 2023-07-18T03:05:59.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.277+0000 7f7e5cca1700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5819d160 con 0x7f7e581007a0 2023-07-18T03:05:59.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.277+0000 7f7e56d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7e58104ea0 0x7f7e5819c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:59.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.277+0000 7f7e56d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7e58104ea0 0x7f7e5819c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:53820/0 (socket says 172.21.15.101:53820) 2023-07-18T03:05:59.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.277+0000 7f7e56d9d700 1 -- 172.21.15.101:0/626841756 learned_addr learned my addr 172.21.15.101:0/626841756 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:05:59.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.277+0000 7f7e5659c700 1 --2- 172.21.15.101:0/626841756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e580ffe00 0x7f7e58197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:59.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.277+0000 7f7e55d9b700 1 --2- 172.21.15.101:0/626841756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e581007a0 0x7f7e581983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:59.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.278+0000 7f7e56d9d700 1 -- 172.21.15.101:0/626841756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e581007a0 msgr2=0x7f7e581983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:59.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.278+0000 7f7e56d9d700 1 --2- 172.21.15.101:0/626841756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e581007a0 0x7f7e581983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:59.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.278+0000 7f7e56d9d700 1 -- 172.21.15.101:0/626841756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e580ffe00 msgr2=0x7f7e58197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:59.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.278+0000 7f7e56d9d700 1 --2- 172.21.15.101:0/626841756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e580ffe00 0x7f7e58197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:59.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.278+0000 7f7e56d9d700 1 -- 172.21.15.101:0/626841756 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e480095f0 con 0x7f7e58104ea0 2023-07-18T03:05:59.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.278+0000 7f7e5659c700 1 --2- 172.21.15.101:0/626841756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e580ffe00 0x7f7e58197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:05:59.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.278+0000 7f7e56d9d700 1 --2- 172.21.15.101:0/626841756 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7e58104ea0 0x7f7e5819c820 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7e50007ee0 tx=0x7f7e5000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:59.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.278+0000 7f7e477fe700 1 -- 172.21.15.101:0/626841756 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5000dd20 con 0x7f7e58104ea0 2023-07-18T03:05:59.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.278+0000 7f7e5cca1700 1 -- 172.21.15.101:0/626841756 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e5819d440 con 0x7f7e58104ea0 2023-07-18T03:05:59.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.278+0000 7f7e477fe700 1 -- 172.21.15.101:0/626841756 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e5000f040 con 0x7f7e58104ea0 2023-07-18T03:05:59.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.278+0000 7f7e477fe700 1 -- 172.21.15.101:0/626841756 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e50015400 con 0x7f7e58104ea0 2023-07-18T03:05:59.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.278+0000 7f7e5cca1700 1 -- 172.21.15.101:0/626841756 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e581a4ca0 con 0x7f7e58104ea0 2023-07-18T03:05:59.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.279+0000 7f7e5cca1700 1 -- 172.21.15.101:0/626841756 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7e5810fd50 con 0x7f7e58104ea0 2023-07-18T03:05:59.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.281+0000 7f7e477fe700 1 -- 172.21.15.101:0/626841756 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f7e50003a00 con 0x7f7e58104ea0 2023-07-18T03:05:59.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.283+0000 7f7e477fe700 1 --2- 172.21.15.101:0/626841756 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f7e400a11e0 0x7f7e400a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:59.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.283+0000 7f7e477fe700 1 -- 172.21.15.101:0/626841756 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f7e500ccda0 con 0x7f7e58104ea0 2023-07-18T03:05:59.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.283+0000 7f7e5659c700 1 --2- 172.21.15.101:0/626841756 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f7e400a11e0 0x7f7e400a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:59.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.283+0000 7f7e477fe700 1 --2- 172.21.15.101:0/626841756 >> [v2:172.21.15.101:6810/4226577463,v1:172.21.15.101:6811/4226577463] conn(0x7f7e400ac170 0x7f7e400ae600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:05:59.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.283+0000 7f7e477fe700 1 -- 172.21.15.101:0/626841756 --> [v2:172.21.15.101:6810/4226577463,v1:172.21.15.101:6811/4226577463] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7e400aecd0 con 0x7f7e400ac170 2023-07-18T03:05:59.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.283+0000 7f7e477fe700 1 -- 172.21.15.101:0/626841756 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e500cd110 con 0x7f7e58104ea0 2023-07-18T03:05:59.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.283+0000 7f7e55d9b700 1 --2- 172.21.15.101:0/626841756 >> [v2:172.21.15.101:6810/4226577463,v1:172.21.15.101:6811/4226577463] conn(0x7f7e400ac170 0x7f7e400ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:05:59.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.284+0000 7f7e5659c700 1 --2- 172.21.15.101:0/626841756 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f7e400a11e0 0x7f7e400a3690 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7e48009990 tx=0x7f7e48000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:59.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.284+0000 7f7e55d9b700 1 --2- 172.21.15.101:0/626841756 >> [v2:172.21.15.101:6810/4226577463,v1:172.21.15.101:6811/4226577463] conn(0x7f7e400ac170 0x7f7e400ae600 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:05:59.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.285+0000 7f7e477fe700 1 -- 172.21.15.101:0/626841756 <== osd.3 v2:172.21.15.101:6810/4226577463 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f7e400aecd0 con 0x7f7e400ac170 2023-07-18T03:05:59.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.302+0000 7f7e5cca1700 1 -- 172.21.15.101:0/626841756 --> [v2:172.21.15.101:6810/4226577463,v1:172.21.15.101:6811/4226577463] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7e5804e210 con 0x7f7e400ac170 2023-07-18T03:05:59.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.302+0000 7f7e477fe700 1 -- 172.21.15.101:0/626841756 <== osd.3 v2:172.21.15.101:6810/4226577463 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f7e5804e210 con 0x7f7e400ac170 2023-07-18T03:05:59.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.303+0000 7f7e5cca1700 1 -- 172.21.15.101:0/626841756 >> [v2:172.21.15.101:6810/4226577463,v1:172.21.15.101:6811/4226577463] conn(0x7f7e400ac170 msgr2=0x7f7e400ae600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:59.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.303+0000 7f7e5cca1700 1 --2- 172.21.15.101:0/626841756 >> [v2:172.21.15.101:6810/4226577463,v1:172.21.15.101:6811/4226577463] conn(0x7f7e400ac170 0x7f7e400ae600 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:05:59.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.303+0000 7f7e5cca1700 1 -- 172.21.15.101:0/626841756 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f7e400a11e0 msgr2=0x7f7e400a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:59.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.303+0000 7f7e5cca1700 1 --2- 172.21.15.101:0/626841756 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f7e400a11e0 0x7f7e400a3690 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7e48009990 tx=0x7f7e48000bc0 comp rx=0 tx=0).stop 2023-07-18T03:05:59.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.303+0000 7f7e5cca1700 1 -- 172.21.15.101:0/626841756 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7e58104ea0 msgr2=0x7f7e5819c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:05:59.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.303+0000 7f7e5cca1700 1 --2- 172.21.15.101:0/626841756 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7e58104ea0 0x7f7e5819c820 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7e50007ee0 tx=0x7f7e5000d3b0 comp rx=0 tx=0).stop 2023-07-18T03:05:59.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.303+0000 7f7e56d9d700 1 -- 172.21.15.101:0/626841756 reap_dead start 2023-07-18T03:05:59.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.303+0000 7f7e5cca1700 1 -- 172.21.15.101:0/626841756 shutdown_connections 2023-07-18T03:05:59.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.303+0000 7f7e5cca1700 1 -- 172.21.15.101:0/626841756 >> 172.21.15.101:0/626841756 conn(0x7f7e580fb660 msgr2=0x7f7e580fc490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:05:59.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.303+0000 7f7e5cca1700 1 -- 172.21.15.101:0/626841756 shutdown_connections 2023-07-18T03:05:59.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:05:59.303+0000 7f7e5cca1700 1 -- 172.21.15.101:0/626841756 wait complete. 2023-07-18T03:06:00.037 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:05:59 smithi155 ceph-mon[128885]: pgmap v307: 129 pgs: 129 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2023-07-18T03:06:00.168 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:05:59 smithi101 ceph-mon[126379]: pgmap v307: 129 pgs: 129 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2023-07-18T03:06:00.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:05:59 smithi179 ceph-mon[129229]: pgmap v307: 129 pgs: 129 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2023-07-18T03:06:00.838 INFO:teuthology.orchestra.run.smithi101.stdout:107374182460 2023-07-18T03:06:00.839 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.3 2023-07-18T03:06:00.864 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.879+0000 7fede02af700 1 -- 172.21.15.101:0/2987237946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fedd810c530 msgr2=0x7fedd810e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:00.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.879+0000 7fede02af700 1 --2- 172.21.15.101:0/2987237946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fedd810c530 0x7fedd810e910 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fedd000b0a0 tx=0x7fedd000b3b0 comp rx=0 tx=0).stop 2023-07-18T03:06:00.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.879+0000 7fede02af700 1 -- 172.21.15.101:0/2987237946 shutdown_connections 2023-07-18T03:06:00.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.879+0000 7fede02af700 1 --2- 172.21.15.101:0/2987237946 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fedd80fe980 0x7fedd80fed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:00.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.879+0000 7fede02af700 1 --2- 172.21.15.101:0/2987237946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fedd810c530 0x7fedd810e910 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:00.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.879+0000 7fede02af700 1 --2- 172.21.15.101:0/2987237946 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fedd80ff290 0x7fedd810bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:00.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.879+0000 7fede02af700 1 -- 172.21.15.101:0/2987237946 >> 172.21.15.101:0/2987237946 conn(0x7fedd8075b70 msgr2=0x7fedd8075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:00.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.880+0000 7fede02af700 1 -- 172.21.15.101:0/2987237946 shutdown_connections 2023-07-18T03:06:00.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.880+0000 7fede02af700 1 -- 172.21.15.101:0/2987237946 wait complete. 2023-07-18T03:06:00.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.881+0000 7fede02af700 1 Processor -- start 2023-07-18T03:06:00.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.881+0000 7fede02af700 1 -- start start 2023-07-18T03:06:00.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.882+0000 7fede02af700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fedd80fe980 0x7fedd8198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:00.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.882+0000 7fede02af700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fedd80ff290 0x7fedd8198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:00.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.882+0000 7fede02af700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fedd810c530 0x7fedd819c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:00.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.882+0000 7fede02af700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd819cfc0 con 0x7fedd810c530 2023-07-18T03:06:00.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.882+0000 7fede02af700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd819d130 con 0x7fedd80ff290 2023-07-18T03:06:00.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.882+0000 7fede02af700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd819d2f0 con 0x7fedd80fe980 2023-07-18T03:06:00.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.882+0000 7fedde84c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fedd810c530 0x7fedd819c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:00.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.882+0000 7fedde84c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fedd810c530 0x7fedd819c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:53842/0 (socket says 172.21.15.101:53842) 2023-07-18T03:06:00.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.882+0000 7fedde84c700 1 -- 172.21.15.101:0/3421575833 learned_addr learned my addr 172.21.15.101:0/3421575833 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:00.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.883+0000 7fedde04b700 1 --2- 172.21.15.101:0/3421575833 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fedd80fe980 0x7fedd8198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:00.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.883+0000 7fedde84c700 1 -- 172.21.15.101:0/3421575833 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fedd80fe980 msgr2=0x7fedd8198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:00.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.883+0000 7fedde84c700 1 --2- 172.21.15.101:0/3421575833 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fedd80fe980 0x7fedd8198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:00.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.883+0000 7fedde84c700 1 -- 172.21.15.101:0/3421575833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fedd80ff290 msgr2=0x7fedd8198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:00.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.883+0000 7fedde84c700 1 --2- 172.21.15.101:0/3421575833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fedd80ff290 0x7fedd8198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:00.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.883+0000 7fedde84c700 1 -- 172.21.15.101:0/3421575833 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd0009d00 con 0x7fedd810c530 2023-07-18T03:06:00.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.883+0000 7fedde84c700 1 --2- 172.21.15.101:0/3421575833 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fedd810c530 0x7fedd819c9b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fedd000b070 tx=0x7fedd0007990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:00.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.884+0000 7fedcf7fe700 1 -- 172.21.15.101:0/3421575833 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd000e040 con 0x7fedd810c530 2023-07-18T03:06:00.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.884+0000 7fede02af700 1 -- 172.21.15.101:0/3421575833 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedd819d570 con 0x7fedd810c530 2023-07-18T03:06:00.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.884+0000 7fedcf7fe700 1 -- 172.21.15.101:0/3421575833 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fedd000be80 con 0x7fedd810c530 2023-07-18T03:06:00.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.884+0000 7fedcf7fe700 1 -- 172.21.15.101:0/3421575833 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd00052a0 con 0x7fedd810c530 2023-07-18T03:06:00.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.884+0000 7fede02af700 1 -- 172.21.15.101:0/3421575833 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedd81a4e30 con 0x7fedd810c530 2023-07-18T03:06:00.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.885+0000 7fede02af700 1 -- 172.21.15.101:0/3421575833 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fedd8111f00 con 0x7fedd810c530 2023-07-18T03:06:00.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.889+0000 7fedcf7fe700 1 -- 172.21.15.101:0/3421575833 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fedd001d470 con 0x7fedd810c530 2023-07-18T03:06:00.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.890+0000 7fedcf7fe700 1 --2- 172.21.15.101:0/3421575833 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fedc40a12e0 0x7fedc40a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:00.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.891+0000 7fedcf7fe700 1 -- 172.21.15.101:0/3421575833 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fedd00ce850 con 0x7fedd810c530 2023-07-18T03:06:00.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.891+0000 7fedde04b700 1 --2- 172.21.15.101:0/3421575833 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fedc40a12e0 0x7fedc40a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:00.877 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.891+0000 7fedcf7fe700 1 --2- 172.21.15.101:0/3421575833 >> [v2:172.21.15.155:6816/3256322291,v1:172.21.15.155:6817/3256322291] conn(0x7fedc40ac270 0x7fedc40ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:00.877 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.891+0000 7fedcf7fe700 1 -- 172.21.15.101:0/3421575833 --> [v2:172.21.15.155:6816/3256322291,v1:172.21.15.155:6817/3256322291] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fedc40aedd0 con 0x7fedc40ac270 2023-07-18T03:06:00.877 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.891+0000 7fedcf7fe700 1 -- 172.21.15.101:0/3421575833 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fedd00cec00 con 0x7fedd810c530 2023-07-18T03:06:00.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.892+0000 7feddd84a700 1 --2- 172.21.15.101:0/3421575833 >> [v2:172.21.15.155:6816/3256322291,v1:172.21.15.155:6817/3256322291] conn(0x7fedc40ac270 0x7fedc40ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:00.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.892+0000 7fedde04b700 1 --2- 172.21.15.101:0/3421575833 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fedc40a12e0 0x7fedc40a3790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fedd4005950 tx=0x7fedd40058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:00.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.893+0000 7feddd84a700 1 --2- 172.21.15.101:0/3421575833 >> [v2:172.21.15.155:6816/3256322291,v1:172.21.15.155:6817/3256322291] conn(0x7fedc40ac270 0x7fedc40ae700 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:00.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.894+0000 7fedcf7fe700 1 -- 172.21.15.101:0/3421575833 <== osd.7 v2:172.21.15.155:6816/3256322291 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fedc40aedd0 con 0x7fedc40ac270 2023-07-18T03:06:00.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.917+0000 7fede02af700 1 -- 172.21.15.101:0/3421575833 --> [v2:172.21.15.155:6816/3256322291,v1:172.21.15.155:6817/3256322291] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fedd804e210 con 0x7fedc40ac270 2023-07-18T03:06:00.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.917+0000 7fedcf7fe700 1 -- 172.21.15.101:0/3421575833 <== osd.7 v2:172.21.15.155:6816/3256322291 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fedd804e210 con 0x7fedc40ac270 2023-07-18T03:06:00.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.918+0000 7fede02af700 1 -- 172.21.15.101:0/3421575833 >> [v2:172.21.15.155:6816/3256322291,v1:172.21.15.155:6817/3256322291] conn(0x7fedc40ac270 msgr2=0x7fedc40ae700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:00.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.918+0000 7fede02af700 1 --2- 172.21.15.101:0/3421575833 >> [v2:172.21.15.155:6816/3256322291,v1:172.21.15.155:6817/3256322291] conn(0x7fedc40ac270 0x7fedc40ae700 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:00.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.918+0000 7fede02af700 1 -- 172.21.15.101:0/3421575833 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fedc40a12e0 msgr2=0x7fedc40a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:00.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.918+0000 7fede02af700 1 --2- 172.21.15.101:0/3421575833 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fedc40a12e0 0x7fedc40a3790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fedd4005950 tx=0x7fedd40058e0 comp rx=0 tx=0).stop 2023-07-18T03:06:00.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.918+0000 7fede02af700 1 -- 172.21.15.101:0/3421575833 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fedd810c530 msgr2=0x7fedd819c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:00.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.918+0000 7fede02af700 1 --2- 172.21.15.101:0/3421575833 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fedd810c530 0x7fedd819c9b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fedd000b070 tx=0x7fedd0007990 comp rx=0 tx=0).stop 2023-07-18T03:06:00.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.918+0000 7fedde84c700 1 -- 172.21.15.101:0/3421575833 reap_dead start 2023-07-18T03:06:00.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.919+0000 7fede02af700 1 -- 172.21.15.101:0/3421575833 shutdown_connections 2023-07-18T03:06:00.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.919+0000 7fede02af700 1 -- 172.21.15.101:0/3421575833 >> 172.21.15.101:0/3421575833 conn(0x7fedd8075b70 msgr2=0x7fedd80fc3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:00.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.919+0000 7fede02af700 1 -- 172.21.15.101:0/3421575833 shutdown_connections 2023-07-18T03:06:00.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:00.920+0000 7fede02af700 1 -- 172.21.15.101:0/3421575833 wait complete. 2023-07-18T03:06:02.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:01 smithi179 ceph-mon[129229]: pgmap v308: 129 pgs: 129 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 6 op/s 2023-07-18T03:06:02.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:01 smithi101 ceph-mon[126379]: pgmap v308: 129 pgs: 129 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 6 op/s 2023-07-18T03:06:02.275 INFO:teuthology.orchestra.run.smithi101.stdout:249108103205 2023-07-18T03:06:02.275 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.8 2023-07-18T03:06:02.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:01 smithi155 ceph-mon[128885]: pgmap v308: 129 pgs: 129 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 6 op/s 2023-07-18T03:06:02.667 INFO:teuthology.orchestra.run.smithi101.stdout:55834574919 2023-07-18T03:06:02.668 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.1 2023-07-18T03:06:03.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:02 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:06:03.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.821+0000 7f5663d0b700 1 -- 172.21.15.101:0/3981220924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f565c10a140 msgr2=0x7f565c10c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:03.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.821+0000 7f5663d0b700 1 --2- 172.21.15.101:0/3981220924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f565c10a140 0x7f565c10c580 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f565400b0a0 tx=0x7f565400b3b0 comp rx=0 tx=0).stop 2023-07-18T03:06:03.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.822+0000 7f5663d0b700 1 -- 172.21.15.101:0/3981220924 shutdown_connections 2023-07-18T03:06:03.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.822+0000 7f5663d0b700 1 --2- 172.21.15.101:0/3981220924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f565c10a140 0x7f565c10c580 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:03.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.822+0000 7f5663d0b700 1 --2- 172.21.15.101:0/3981220924 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f565c100df0 0x7f565c1011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:03.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.822+0000 7f5663d0b700 1 --2- 172.21.15.101:0/3981220924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f565c101700 0x7f565c109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:03.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.822+0000 7f5663d0b700 1 -- 172.21.15.101:0/3981220924 >> 172.21.15.101:0/3981220924 conn(0x7f565c075b70 msgr2=0x7f565c075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:03.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.822+0000 7f5663d0b700 1 -- 172.21.15.101:0/3981220924 shutdown_connections 2023-07-18T03:06:03.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.822+0000 7f5663d0b700 1 -- 172.21.15.101:0/3981220924 wait complete. 2023-07-18T03:06:03.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.823+0000 7f5663d0b700 1 Processor -- start 2023-07-18T03:06:03.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.824+0000 7f5663d0b700 1 -- start start 2023-07-18T03:06:03.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.824+0000 7f5663d0b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f565c100df0 0x7f565c195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:03.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.824+0000 7f5663d0b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f565c101700 0x7f565c196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:03.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.824+0000 7f5663d0b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f565c10a140 0x7f565c19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:03.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.824+0000 7f5663d0b700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565c19ad10 con 0x7f565c10a140 2023-07-18T03:06:03.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.824+0000 7f5663d0b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565c19ae80 con 0x7f565c100df0 2023-07-18T03:06:03.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.824+0000 7f5663d0b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565c19b040 con 0x7f565c101700 2023-07-18T03:06:03.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.825+0000 7f5661aa7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f565c100df0 0x7f565c195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:03.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.825+0000 7f5661aa7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f565c100df0 0x7f565c195d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:33170/0 (socket says 172.21.15.101:33170) 2023-07-18T03:06:03.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.825+0000 7f5661aa7700 1 -- 172.21.15.101:0/3119380362 learned_addr learned my addr 172.21.15.101:0/3119380362 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:03.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.825+0000 7f5661aa7700 1 -- 172.21.15.101:0/3119380362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f565c101700 msgr2=0x7f565c196290 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-18T03:06:03.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.825+0000 7f5661aa7700 1 --2- 172.21.15.101:0/3119380362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f565c101700 0x7f565c196290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:03.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.825+0000 7f5661aa7700 1 -- 172.21.15.101:0/3119380362 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f565c10a140 msgr2=0x7f565c19a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:03.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.825+0000 7f5661aa7700 1 --2- 172.21.15.101:0/3119380362 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f565c10a140 0x7f565c19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:03.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.825+0000 7f5661aa7700 1 -- 172.21.15.101:0/3119380362 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5654009d00 con 0x7f565c100df0 2023-07-18T03:06:03.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.825+0000 7f5661aa7700 1 --2- 172.21.15.101:0/3119380362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f565c100df0 0x7f565c195d50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f565800e9a0 tx=0x7f565800ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:03.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.826+0000 7f5652ffd700 1 -- 172.21.15.101:0/3119380362 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5658009790 con 0x7f565c100df0 2023-07-18T03:06:03.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.826+0000 7f5652ffd700 1 -- 172.21.15.101:0/3119380362 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f565800cbd0 con 0x7f565c100df0 2023-07-18T03:06:03.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.826+0000 7f5652ffd700 1 -- 172.21.15.101:0/3119380362 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5658018680 con 0x7f565c100df0 2023-07-18T03:06:03.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.826+0000 7f5663d0b700 1 -- 172.21.15.101:0/3119380362 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f565c19b320 con 0x7f565c100df0 2023-07-18T03:06:03.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.826+0000 7f5663d0b700 1 -- 172.21.15.101:0/3119380362 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f565c0feb50 con 0x7f565c100df0 2023-07-18T03:06:03.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.828+0000 7f5663d0b700 1 -- 172.21.15.101:0/3119380362 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f565c04e210 con 0x7f565c100df0 2023-07-18T03:06:03.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.831+0000 7f5652ffd700 1 -- 172.21.15.101:0/3119380362 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f565800cd40 con 0x7f565c100df0 2023-07-18T03:06:03.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.834+0000 7f5652ffd700 1 --2- 172.21.15.101:0/3119380362 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f56480a12e0 0x7f56480a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:03.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.834+0000 7f5652ffd700 1 -- 172.21.15.101:0/3119380362 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f5658014070 con 0x7f565c100df0 2023-07-18T03:06:03.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.834+0000 7f5652ffd700 1 --2- 172.21.15.101:0/3119380362 >> [v2:172.21.15.179:6808/2746037021,v1:172.21.15.179:6809/2746037021] conn(0x7f56480ac270 0x7f56480ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:03.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.834+0000 7f5652ffd700 1 -- 172.21.15.101:0/3119380362 --> [v2:172.21.15.179:6808/2746037021,v1:172.21.15.179:6809/2746037021] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f56480aedd0 con 0x7f56480ac270 2023-07-18T03:06:03.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.834+0000 7f5652ffd700 1 -- 172.21.15.101:0/3119380362 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56580cdad0 con 0x7f565c100df0 2023-07-18T03:06:03.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.835+0000 7f56622a8700 1 --2- 172.21.15.101:0/3119380362 >> [v2:172.21.15.179:6808/2746037021,v1:172.21.15.179:6809/2746037021] conn(0x7f56480ac270 0x7f56480ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:03.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.836+0000 7f56622a8700 1 --2- 172.21.15.101:0/3119380362 >> [v2:172.21.15.179:6808/2746037021,v1:172.21.15.179:6809/2746037021] conn(0x7f56480ac270 0x7f56480ae700 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:03.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.836+0000 7f56612a6700 1 --2- 172.21.15.101:0/3119380362 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f56480a12e0 0x7f56480a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:03.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.837+0000 7f5652ffd700 1 -- 172.21.15.101:0/3119380362 <== osd.5 v2:172.21.15.179:6808/2746037021 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f56480aedd0 con 0x7f56480ac270 2023-07-18T03:06:03.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.837+0000 7f56612a6700 1 --2- 172.21.15.101:0/3119380362 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f56480a12e0 0x7f56480a3790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f565c197370 tx=0x7f564c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:03.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.868+0000 7f5663d0b700 1 -- 172.21.15.101:0/3119380362 --> [v2:172.21.15.179:6808/2746037021,v1:172.21.15.179:6809/2746037021] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f565c10fd70 con 0x7f56480ac270 2023-07-18T03:06:03.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.868+0000 7f5652ffd700 1 -- 172.21.15.101:0/3119380362 <== osd.5 v2:172.21.15.179:6808/2746037021 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f5654009b80 con 0x7f56480ac270 2023-07-18T03:06:03.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.873+0000 7f5663d0b700 1 -- 172.21.15.101:0/3119380362 >> [v2:172.21.15.179:6808/2746037021,v1:172.21.15.179:6809/2746037021] conn(0x7f56480ac270 msgr2=0x7f56480ae700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:03.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.873+0000 7f5663d0b700 1 --2- 172.21.15.101:0/3119380362 >> [v2:172.21.15.179:6808/2746037021,v1:172.21.15.179:6809/2746037021] conn(0x7f56480ac270 0x7f56480ae700 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:03.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.873+0000 7f5663d0b700 1 -- 172.21.15.101:0/3119380362 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f56480a12e0 msgr2=0x7f56480a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:03.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.873+0000 7f5663d0b700 1 --2- 172.21.15.101:0/3119380362 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f56480a12e0 0x7f56480a3790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f565c197370 tx=0x7f564c009260 comp rx=0 tx=0).stop 2023-07-18T03:06:03.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.873+0000 7f5663d0b700 1 -- 172.21.15.101:0/3119380362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f565c100df0 msgr2=0x7f565c195d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:03.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.873+0000 7f5663d0b700 1 --2- 172.21.15.101:0/3119380362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f565c100df0 0x7f565c195d50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f565800e9a0 tx=0x7f565800ed60 comp rx=0 tx=0).stop 2023-07-18T03:06:03.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.874+0000 7f56622a8700 1 -- 172.21.15.101:0/3119380362 reap_dead start 2023-07-18T03:06:03.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.874+0000 7f5663d0b700 1 -- 172.21.15.101:0/3119380362 shutdown_connections 2023-07-18T03:06:03.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.874+0000 7f5663d0b700 1 -- 172.21.15.101:0/3119380362 >> 172.21.15.101:0/3119380362 conn(0x7f565c075b70 msgr2=0x7f565c0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:03.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.876+0000 7f5663d0b700 1 -- 172.21.15.101:0/3119380362 shutdown_connections 2023-07-18T03:06:03.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:02.877+0000 7f5663d0b700 1 -- 172.21.15.101:0/3119380362 wait complete. 2023-07-18T03:06:03.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:02 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:06:03.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:02 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:06:03.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.683+0000 7f72e3eff700 1 -- 172.21.15.101:0/479015906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72dc0731c0 msgr2=0x7f72dc110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:03.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.683+0000 7f72e3eff700 1 --2- 172.21.15.101:0/479015906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72dc0731c0 0x7f72dc110670 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f72cc0099c0 tx=0x7f72cc009cd0 comp rx=0 tx=0).stop 2023-07-18T03:06:03.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.684+0000 7f72e3eff700 1 -- 172.21.15.101:0/479015906 shutdown_connections 2023-07-18T03:06:03.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.684+0000 7f72e3eff700 1 --2- 172.21.15.101:0/479015906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72dc110bb0 0x7f72dc112f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:03.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.684+0000 7f72e3eff700 1 --2- 172.21.15.101:0/479015906 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f72dc0728b0 0x7f72dc072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:03.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.684+0000 7f72e3eff700 1 --2- 172.21.15.101:0/479015906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72dc0731c0 0x7f72dc110670 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:03.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.684+0000 7f72e3eff700 1 -- 172.21.15.101:0/479015906 >> 172.21.15.101:0/479015906 conn(0x7f72dc0fb660 msgr2=0x7f72dc0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:03.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.684+0000 7f72e3eff700 1 -- 172.21.15.101:0/479015906 shutdown_connections 2023-07-18T03:06:03.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.684+0000 7f72e3eff700 1 -- 172.21.15.101:0/479015906 wait complete. 2023-07-18T03:06:03.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.685+0000 7f72e3eff700 1 Processor -- start 2023-07-18T03:06:03.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.685+0000 7f72e3eff700 1 -- start start 2023-07-18T03:06:03.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.686+0000 7f72e3eff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72dc0728b0 0x7f72dc19c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:03.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.686+0000 7f72e3eff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72dc0731c0 0x7f72dc19c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:03.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.686+0000 7f72e3eff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f72dc110bb0 0x7f72dc1a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:03.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.686+0000 7f72e3eff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc1a1250 con 0x7f72dc110bb0 2023-07-18T03:06:03.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.686+0000 7f72e3eff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc1a13c0 con 0x7f72dc0731c0 2023-07-18T03:06:03.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.686+0000 7f72e3eff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc1a1580 con 0x7f72dc0728b0 2023-07-18T03:06:03.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.686+0000 7f72e1c9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72dc0728b0 0x7f72dc19c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:03.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.686+0000 7f72e249c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f72dc110bb0 0x7f72dc1a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:03.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.686+0000 7f72e1c9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72dc0728b0 0x7f72dc19c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:42224/0 (socket says 172.21.15.101:42224) 2023-07-18T03:06:03.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.686+0000 7f72e1c9b700 1 -- 172.21.15.101:0/3603514517 learned_addr learned my addr 172.21.15.101:0/3603514517 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:03.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.687+0000 7f72e1c9b700 1 -- 172.21.15.101:0/3603514517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72dc0731c0 msgr2=0x7f72dc19c7d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-18T03:06:03.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.687+0000 7f72e1c9b700 1 --2- 172.21.15.101:0/3603514517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72dc0731c0 0x7f72dc19c7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:03.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.687+0000 7f72e1c9b700 1 -- 172.21.15.101:0/3603514517 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f72dc110bb0 msgr2=0x7f72dc1a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:03.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.687+0000 7f72e1c9b700 1 --2- 172.21.15.101:0/3603514517 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f72dc110bb0 0x7f72dc1a0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:03.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.687+0000 7f72e1c9b700 1 -- 172.21.15.101:0/3603514517 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72cc0095f0 con 0x7f72dc0728b0 2023-07-18T03:06:03.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.687+0000 7f72e249c700 1 --2- 172.21.15.101:0/3603514517 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f72dc110bb0 0x7f72dc1a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:06:03.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.687+0000 7f72e1c9b700 1 --2- 172.21.15.101:0/3603514517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72dc0728b0 0x7f72dc19c290 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f72d800d980 tx=0x7f72d800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:03.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.687+0000 7f72d2ffd700 1 -- 172.21.15.101:0/3603514517 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d8004220 con 0x7f72dc0728b0 2023-07-18T03:06:03.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.687+0000 7f72e3eff700 1 -- 172.21.15.101:0/3603514517 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72dc1a1860 con 0x7f72dc0728b0 2023-07-18T03:06:03.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.687+0000 7f72d2ffd700 1 -- 172.21.15.101:0/3603514517 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72d8004cf0 con 0x7f72dc0728b0 2023-07-18T03:06:03.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.687+0000 7f72d2ffd700 1 -- 172.21.15.101:0/3603514517 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d8005720 con 0x7f72dc0728b0 2023-07-18T03:06:03.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.688+0000 7f72e3eff700 1 -- 172.21.15.101:0/3603514517 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72dc1a90c0 con 0x7f72dc0728b0 2023-07-18T03:06:03.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.688+0000 7f72e3eff700 1 -- 172.21.15.101:0/3603514517 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f72dc116330 con 0x7f72dc0728b0 2023-07-18T03:06:03.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.691+0000 7f72d2ffd700 1 -- 172.21.15.101:0/3603514517 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f72d8004410 con 0x7f72dc0728b0 2023-07-18T03:06:03.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.693+0000 7f72d2ffd700 1 --2- 172.21.15.101:0/3603514517 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f72c80a1260 0x7f72c80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:03.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.693+0000 7f72d2ffd700 1 -- 172.21.15.101:0/3603514517 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f72d80cd210 con 0x7f72dc0728b0 2023-07-18T03:06:03.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.694+0000 7f72d2ffd700 1 --2- 172.21.15.101:0/3603514517 >> [v2:172.21.15.101:6802/3610719929,v1:172.21.15.101:6803/3610719929] conn(0x7f72c80ac1f0 0x7f72c80ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:03.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.694+0000 7f72d2ffd700 1 -- 172.21.15.101:0/3603514517 --> [v2:172.21.15.101:6802/3610719929,v1:172.21.15.101:6803/3610719929] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f72c80aed50 con 0x7f72c80ac1f0 2023-07-18T03:06:03.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.694+0000 7f72d2ffd700 1 -- 172.21.15.101:0/3603514517 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72d80cd580 con 0x7f72dc0728b0 2023-07-18T03:06:03.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.694+0000 7f72e149a700 1 --2- 172.21.15.101:0/3603514517 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f72c80a1260 0x7f72c80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:03.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.694+0000 7f72e249c700 1 --2- 172.21.15.101:0/3603514517 >> [v2:172.21.15.101:6802/3610719929,v1:172.21.15.101:6803/3610719929] conn(0x7f72c80ac1f0 0x7f72c80ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:03.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.695+0000 7f72e149a700 1 --2- 172.21.15.101:0/3603514517 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f72c80a1260 0x7f72c80a3710 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f72cc009990 tx=0x7f72cc00b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:03.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.695+0000 7f72e249c700 1 --2- 172.21.15.101:0/3603514517 >> [v2:172.21.15.101:6802/3610719929,v1:172.21.15.101:6803/3610719929] conn(0x7f72c80ac1f0 0x7f72c80ae680 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:03.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.696+0000 7f72d2ffd700 1 -- 172.21.15.101:0/3603514517 <== osd.0 v2:172.21.15.101:6802/3610719929 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f72c80aed50 con 0x7f72c80ac1f0 2023-07-18T03:06:03.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.715+0000 7f72e3eff700 1 -- 172.21.15.101:0/3603514517 --> [v2:172.21.15.101:6802/3610719929,v1:172.21.15.101:6803/3610719929] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f72dc04e210 con 0x7f72c80ac1f0 2023-07-18T03:06:03.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.716+0000 7f72d2ffd700 1 -- 172.21.15.101:0/3603514517 <== osd.0 v2:172.21.15.101:6802/3610719929 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f72dc04e210 con 0x7f72c80ac1f0 2023-07-18T03:06:03.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.716+0000 7f72e3eff700 1 -- 172.21.15.101:0/3603514517 >> [v2:172.21.15.101:6802/3610719929,v1:172.21.15.101:6803/3610719929] conn(0x7f72c80ac1f0 msgr2=0x7f72c80ae680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:03.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.716+0000 7f72e3eff700 1 --2- 172.21.15.101:0/3603514517 >> [v2:172.21.15.101:6802/3610719929,v1:172.21.15.101:6803/3610719929] conn(0x7f72c80ac1f0 0x7f72c80ae680 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:03.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.716+0000 7f72e3eff700 1 -- 172.21.15.101:0/3603514517 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f72c80a1260 msgr2=0x7f72c80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:03.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.716+0000 7f72e3eff700 1 --2- 172.21.15.101:0/3603514517 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f72c80a1260 0x7f72c80a3710 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f72cc009990 tx=0x7f72cc00b540 comp rx=0 tx=0).stop 2023-07-18T03:06:03.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.716+0000 7f72e3eff700 1 -- 172.21.15.101:0/3603514517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72dc0728b0 msgr2=0x7f72dc19c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:03.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.716+0000 7f72e3eff700 1 --2- 172.21.15.101:0/3603514517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72dc0728b0 0x7f72dc19c290 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f72d800d980 tx=0x7f72d800dc90 comp rx=0 tx=0).stop 2023-07-18T03:06:03.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.717+0000 7f72e249c700 1 -- 172.21.15.101:0/3603514517 reap_dead start 2023-07-18T03:06:03.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.717+0000 7f72e3eff700 1 -- 172.21.15.101:0/3603514517 shutdown_connections 2023-07-18T03:06:03.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.717+0000 7f72e3eff700 1 -- 172.21.15.101:0/3603514517 >> 172.21.15.101:0/3603514517 conn(0x7f72dc0fb660 msgr2=0x7f72dc0fc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:03.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.717+0000 7f72e3eff700 1 -- 172.21.15.101:0/3603514517 shutdown_connections 2023-07-18T03:06:03.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.717+0000 7f72e3eff700 1 -- 172.21.15.101:0/3603514517 wait complete. 2023-07-18T03:06:03.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.667+0000 7f1cae194700 1 -- 172.21.15.101:0/2257645918 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1ca80ffe00 msgr2=0x7f1ca81001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:03.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.667+0000 7f1cae194700 1 --2- 172.21.15.101:0/2257645918 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1ca80ffe00 0x7f1ca81001d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1c98009920 tx=0x7f1c98009c30 comp rx=0 tx=0).stop 2023-07-18T03:06:03.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.667+0000 7f1cae194700 1 -- 172.21.15.101:0/2257645918 shutdown_connections 2023-07-18T03:06:03.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.667+0000 7f1cae194700 1 --2- 172.21.15.101:0/2257645918 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ca8104ea0 0x7f1ca810ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:03.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.667+0000 7f1cae194700 1 --2- 172.21.15.101:0/2257645918 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1ca80ffe00 0x7f1ca81001d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:03.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.667+0000 7f1cae194700 1 --2- 172.21.15.101:0/2257645918 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ca81007a0 0x7f1ca8104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:03.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.667+0000 7f1cae194700 1 -- 172.21.15.101:0/2257645918 >> 172.21.15.101:0/2257645918 conn(0x7f1ca80fb660 msgr2=0x7f1ca80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:03.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.667+0000 7f1cae194700 1 -- 172.21.15.101:0/2257645918 shutdown_connections 2023-07-18T03:06:03.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.667+0000 7f1cae194700 1 -- 172.21.15.101:0/2257645918 wait complete. 2023-07-18T03:06:03.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.669+0000 7f1cae194700 1 Processor -- start 2023-07-18T03:06:03.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.669+0000 7f1cae194700 1 -- start start 2023-07-18T03:06:03.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.669+0000 7f1cae194700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1ca80ffe00 0x7f1ca8072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:03.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.669+0000 7f1cae194700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ca81007a0 0x7f1ca8072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:03.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.669+0000 7f1cae194700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ca8104ea0 0x7f1ca80732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:03.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.669+0000 7f1cae194700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca81a41c0 con 0x7f1ca80ffe00 2023-07-18T03:06:03.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.669+0000 7f1cae194700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca81a4330 con 0x7f1ca81007a0 2023-07-18T03:06:03.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.669+0000 7f1cae194700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca81a44f0 con 0x7f1ca8104ea0 2023-07-18T03:06:03.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.670+0000 7f1ca7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ca8104ea0 0x7f1ca80732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:03.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.670+0000 7f1ca6ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ca81007a0 0x7f1ca8072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:03.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.670+0000 7f1ca7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ca8104ea0 0x7f1ca80732d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:42196/0 (socket says 172.21.15.101:42196) 2023-07-18T03:06:03.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.670+0000 7f1ca7fff700 1 -- 172.21.15.101:0/3638303423 learned_addr learned my addr 172.21.15.101:0/3638303423 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:03.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.670+0000 7f1ca77fe700 1 --2- 172.21.15.101:0/3638303423 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1ca80ffe00 0x7f1ca8072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:03.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.670+0000 7f1ca7fff700 1 -- 172.21.15.101:0/3638303423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ca81007a0 msgr2=0x7f1ca8072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:03.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.670+0000 7f1ca7fff700 1 --2- 172.21.15.101:0/3638303423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ca81007a0 0x7f1ca8072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:03.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.670+0000 7f1ca7fff700 1 -- 172.21.15.101:0/3638303423 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1ca80ffe00 msgr2=0x7f1ca8072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:03.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.670+0000 7f1ca7fff700 1 --2- 172.21.15.101:0/3638303423 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1ca80ffe00 0x7f1ca8072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:03.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.670+0000 7f1ca7fff700 1 -- 172.21.15.101:0/3638303423 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c98009520 con 0x7f1ca8104ea0 2023-07-18T03:06:03.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.671+0000 7f1ca7fff700 1 --2- 172.21.15.101:0/3638303423 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ca8104ea0 0x7f1ca80732d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0007ee0 tx=0x7f1ca000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:03.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.671+0000 7f1ca4ff9700 1 -- 172.21.15.101:0/3638303423 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca000ddd0 con 0x7f1ca8104ea0 2023-07-18T03:06:03.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.671+0000 7f1cae194700 1 -- 172.21.15.101:0/3638303423 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ca81a47d0 con 0x7f1ca8104ea0 2023-07-18T03:06:03.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.671+0000 7f1ca4ff9700 1 -- 172.21.15.101:0/3638303423 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ca000f040 con 0x7f1ca8104ea0 2023-07-18T03:06:03.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.671+0000 7f1ca4ff9700 1 -- 172.21.15.101:0/3638303423 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca00048a0 con 0x7f1ca8104ea0 2023-07-18T03:06:03.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.671+0000 7f1cae194700 1 -- 172.21.15.101:0/3638303423 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ca81a4cd0 con 0x7f1ca8104ea0 2023-07-18T03:06:03.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.672+0000 7f1cae194700 1 -- 172.21.15.101:0/3638303423 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1ca804e210 con 0x7f1ca8104ea0 2023-07-18T03:06:03.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.675+0000 7f1ca4ff9700 1 -- 172.21.15.101:0/3638303423 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f1ca0003a00 con 0x7f1ca8104ea0 2023-07-18T03:06:03.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.677+0000 7f1ca4ff9700 1 --2- 172.21.15.101:0/3638303423 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f1c900a12e0 0x7f1c900a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:03.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.677+0000 7f1ca4ff9700 1 -- 172.21.15.101:0/3638303423 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f1ca00cc740 con 0x7f1ca8104ea0 2023-07-18T03:06:03.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.677+0000 7f1ca4ff9700 1 --2- 172.21.15.101:0/3638303423 >> [v2:172.21.15.155:6824/3682446349,v1:172.21.15.155:6825/3682446349] conn(0x7f1c900ac270 0x7f1c900ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:03.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.677+0000 7f1ca4ff9700 1 -- 172.21.15.101:0/3638303423 --> [v2:172.21.15.155:6824/3682446349,v1:172.21.15.155:6825/3682446349] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1c900aedd0 con 0x7f1c900ac270 2023-07-18T03:06:03.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.677+0000 7f1ca4ff9700 1 -- 172.21.15.101:0/3638303423 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ca00ccaf0 con 0x7f1ca8104ea0 2023-07-18T03:06:03.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.678+0000 7f1ca6ffd700 1 --2- 172.21.15.101:0/3638303423 >> [v2:172.21.15.155:6824/3682446349,v1:172.21.15.155:6825/3682446349] conn(0x7f1c900ac270 0x7f1c900ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:03.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.678+0000 7f1ca77fe700 1 --2- 172.21.15.101:0/3638303423 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f1c900a12e0 0x7f1c900a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:03.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.678+0000 7f1ca6ffd700 1 --2- 172.21.15.101:0/3638303423 >> [v2:172.21.15.155:6824/3682446349,v1:172.21.15.155:6825/3682446349] conn(0x7f1c900ac270 0x7f1c900ae700 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:03.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.679+0000 7f1ca77fe700 1 --2- 172.21.15.101:0/3638303423 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f1c900a12e0 0x7f1c900a3790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1c9800c010 tx=0x7f1c98019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:03.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.679+0000 7f1ca4ff9700 1 -- 172.21.15.101:0/3638303423 <== osd.10 v2:172.21.15.155:6824/3682446349 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f1c900aedd0 con 0x7f1c900ac270 2023-07-18T03:06:03.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.701+0000 7f1cae194700 1 -- 172.21.15.101:0/3638303423 --> [v2:172.21.15.155:6824/3682446349,v1:172.21.15.155:6825/3682446349] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1ca810fdc0 con 0x7f1c900ac270 2023-07-18T03:06:03.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.701+0000 7f1ca4ff9700 1 -- 172.21.15.101:0/3638303423 <== osd.10 v2:172.21.15.155:6824/3682446349 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1c9c0098f0 con 0x7f1c900ac270 2023-07-18T03:06:03.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.702+0000 7f1cae194700 1 -- 172.21.15.101:0/3638303423 >> [v2:172.21.15.155:6824/3682446349,v1:172.21.15.155:6825/3682446349] conn(0x7f1c900ac270 msgr2=0x7f1c900ae700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:03.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.702+0000 7f1cae194700 1 --2- 172.21.15.101:0/3638303423 >> [v2:172.21.15.155:6824/3682446349,v1:172.21.15.155:6825/3682446349] conn(0x7f1c900ac270 0x7f1c900ae700 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:03.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.702+0000 7f1cae194700 1 -- 172.21.15.101:0/3638303423 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f1c900a12e0 msgr2=0x7f1c900a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:03.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.702+0000 7f1cae194700 1 --2- 172.21.15.101:0/3638303423 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f1c900a12e0 0x7f1c900a3790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1c9800c010 tx=0x7f1c98019040 comp rx=0 tx=0).stop 2023-07-18T03:06:03.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.702+0000 7f1cae194700 1 -- 172.21.15.101:0/3638303423 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ca8104ea0 msgr2=0x7f1ca80732d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:03.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.702+0000 7f1cae194700 1 --2- 172.21.15.101:0/3638303423 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ca8104ea0 0x7f1ca80732d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0007ee0 tx=0x7f1ca000d3b0 comp rx=0 tx=0).stop 2023-07-18T03:06:03.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.702+0000 7f1ca7fff700 1 -- 172.21.15.101:0/3638303423 reap_dead start 2023-07-18T03:06:03.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.702+0000 7f1cae194700 1 -- 172.21.15.101:0/3638303423 shutdown_connections 2023-07-18T03:06:03.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.702+0000 7f1cae194700 1 -- 172.21.15.101:0/3638303423 >> 172.21.15.101:0/3638303423 conn(0x7f1ca80fb660 msgr2=0x7f1ca80fc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:03.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.702+0000 7f1cae194700 1 -- 172.21.15.101:0/3638303423 shutdown_connections 2023-07-18T03:06:03.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.702+0000 7f1cae194700 1 -- 172.21.15.101:0/3638303423 wait complete. 2023-07-18T03:06:03.980 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:03 smithi101 ceph-mon[126379]: pgmap v309: 129 pgs: 129 active+clean; 640 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2023-07-18T03:06:04.004 INFO:teuthology.orchestra.run.smithi101.stdout:219043332137 2023-07-18T03:06:04.004 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.7 2023-07-18T03:06:04.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:03 smithi179 ceph-mon[129229]: pgmap v309: 129 pgs: 129 active+clean; 640 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2023-07-18T03:06:04.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:03 smithi155 ceph-mon[128885]: pgmap v309: 129 pgs: 129 active+clean; 640 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2023-07-18T03:06:04.355 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.714+0000 7f35f3ac0700 1 -- 172.21.15.101:0/3298675269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35ec104ea0 msgr2=0x7f35ec10ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:04.355 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.714+0000 7f35f3ac0700 1 --2- 172.21.15.101:0/3298675269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35ec104ea0 0x7f35ec10ca20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f35e4009f90 tx=0x7f35e400b330 comp rx=0 tx=0).stop 2023-07-18T03:06:04.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.714+0000 7f35f3ac0700 1 -- 172.21.15.101:0/3298675269 shutdown_connections 2023-07-18T03:06:04.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.714+0000 7f35f3ac0700 1 --2- 172.21.15.101:0/3298675269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35ec104ea0 0x7f35ec10ca20 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:04.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.714+0000 7f35f3ac0700 1 --2- 172.21.15.101:0/3298675269 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f35ec0ffe00 0x7f35ec1001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:04.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.714+0000 7f35f3ac0700 1 --2- 172.21.15.101:0/3298675269 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35ec1007a0 0x7f35ec104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:04.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.714+0000 7f35f3ac0700 1 -- 172.21.15.101:0/3298675269 >> 172.21.15.101:0/3298675269 conn(0x7f35ec0fb660 msgr2=0x7f35ec0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:04.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.714+0000 7f35f3ac0700 1 -- 172.21.15.101:0/3298675269 shutdown_connections 2023-07-18T03:06:04.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.714+0000 7f35f3ac0700 1 -- 172.21.15.101:0/3298675269 wait complete. 2023-07-18T03:06:04.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.715+0000 7f35f3ac0700 1 Processor -- start 2023-07-18T03:06:04.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.716+0000 7f35f3ac0700 1 -- start start 2023-07-18T03:06:04.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.716+0000 7f35f3ac0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35ec0ffe00 0x7f35ec072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:04.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.716+0000 7f35f3ac0700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f35ec1007a0 0x7f35ec072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:04.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.716+0000 7f35f3ac0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35ec104ea0 0x7f35ec0732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:04.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.716+0000 7f35f3ac0700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ec1a41c0 con 0x7f35ec1007a0 2023-07-18T03:06:04.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.716+0000 7f35f3ac0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ec1a4330 con 0x7f35ec104ea0 2023-07-18T03:06:04.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.716+0000 7f35f3ac0700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ec1a44f0 con 0x7f35ec0ffe00 2023-07-18T03:06:04.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.716+0000 7f35f205d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35ec104ea0 0x7f35ec0732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:04.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.716+0000 7f35f185c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35ec0ffe00 0x7f35ec072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:04.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.717+0000 7f35f205d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35ec104ea0 0x7f35ec0732d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:33230/0 (socket says 172.21.15.101:33230) 2023-07-18T03:06:04.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.717+0000 7f35f185c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35ec0ffe00 0x7f35ec072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:42252/0 (socket says 172.21.15.101:42252) 2023-07-18T03:06:04.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.717+0000 7f35f205d700 1 -- 172.21.15.101:0/820916510 learned_addr learned my addr 172.21.15.101:0/820916510 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:04.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.717+0000 7f35f105b700 1 --2- 172.21.15.101:0/820916510 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f35ec1007a0 0x7f35ec072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:04.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.717+0000 7f35f185c700 1 -- 172.21.15.101:0/820916510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35ec104ea0 msgr2=0x7f35ec0732d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:04.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.717+0000 7f35f185c700 1 --2- 172.21.15.101:0/820916510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35ec104ea0 0x7f35ec0732d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:04.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.717+0000 7f35f185c700 1 -- 172.21.15.101:0/820916510 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f35ec1007a0 msgr2=0x7f35ec072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:04.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.717+0000 7f35f185c700 1 --2- 172.21.15.101:0/820916510 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f35ec1007a0 0x7f35ec072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:04.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.717+0000 7f35f185c700 1 -- 172.21.15.101:0/820916510 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e80095f0 con 0x7f35ec0ffe00 2023-07-18T03:06:04.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.717+0000 7f35f205d700 1 --2- 172.21.15.101:0/820916510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35ec104ea0 0x7f35ec0732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:06:04.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.717+0000 7f35f185c700 1 --2- 172.21.15.101:0/820916510 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35ec0ffe00 0x7f35ec072810 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f35e800eb90 tx=0x7f35e800eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:04.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.717+0000 7f35e2ffd700 1 -- 172.21.15.101:0/820916510 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e8009970 con 0x7f35ec0ffe00 2023-07-18T03:06:04.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.718+0000 7f35e2ffd700 1 -- 172.21.15.101:0/820916510 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35e80044e0 con 0x7f35ec0ffe00 2023-07-18T03:06:04.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.718+0000 7f35e2ffd700 1 -- 172.21.15.101:0/820916510 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e8003e40 con 0x7f35ec0ffe00 2023-07-18T03:06:04.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.718+0000 7f35f3ac0700 1 -- 172.21.15.101:0/820916510 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35e4009c40 con 0x7f35ec0ffe00 2023-07-18T03:06:04.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.718+0000 7f35f3ac0700 1 -- 172.21.15.101:0/820916510 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35ec1a4af0 con 0x7f35ec0ffe00 2023-07-18T03:06:04.361 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.719+0000 7f35f3ac0700 1 -- 172.21.15.101:0/820916510 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f35ec04e210 con 0x7f35ec0ffe00 2023-07-18T03:06:04.361 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.722+0000 7f35e2ffd700 1 -- 172.21.15.101:0/820916510 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f35e8003fe0 con 0x7f35ec0ffe00 2023-07-18T03:06:04.361 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.725+0000 7f35e2ffd700 1 --2- 172.21.15.101:0/820916510 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f35d80a1260 0x7f35d80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:04.361 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.725+0000 7f35e2ffd700 1 -- 172.21.15.101:0/820916510 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f35e8014070 con 0x7f35ec0ffe00 2023-07-18T03:06:04.361 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.725+0000 7f35f105b700 1 --2- 172.21.15.101:0/820916510 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f35d80a1260 0x7f35d80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:04.361 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.725+0000 7f35e2ffd700 1 --2- 172.21.15.101:0/820916510 >> [v2:172.21.15.101:6826/2166787667,v1:172.21.15.101:6827/2166787667] conn(0x7f35d80ac1f0 0x7f35d80ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:04.361 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.725+0000 7f35e2ffd700 1 -- 172.21.15.101:0/820916510 --> [v2:172.21.15.101:6826/2166787667,v1:172.21.15.101:6827/2166787667] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f35d80aed50 con 0x7f35d80ac1f0 2023-07-18T03:06:04.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.725+0000 7f35e2ffd700 1 -- 172.21.15.101:0/820916510 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35e80cd660 con 0x7f35ec0ffe00 2023-07-18T03:06:04.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.725+0000 7f35f205d700 1 --2- 172.21.15.101:0/820916510 >> [v2:172.21.15.101:6826/2166787667,v1:172.21.15.101:6827/2166787667] conn(0x7f35d80ac1f0 0x7f35d80ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:04.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.726+0000 7f35f105b700 1 --2- 172.21.15.101:0/820916510 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f35d80a1260 0x7f35d80a3710 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f35ec074000 tx=0x7f35dc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:04.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.726+0000 7f35f205d700 1 --2- 172.21.15.101:0/820916510 >> [v2:172.21.15.101:6826/2166787667,v1:172.21.15.101:6827/2166787667] conn(0x7f35d80ac1f0 0x7f35d80ae680 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:04.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.727+0000 7f35e2ffd700 1 -- 172.21.15.101:0/820916510 <== osd.9 v2:172.21.15.101:6826/2166787667 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f35d80aed50 con 0x7f35d80ac1f0 2023-07-18T03:06:04.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.748+0000 7f35f3ac0700 1 -- 172.21.15.101:0/820916510 --> [v2:172.21.15.101:6826/2166787667,v1:172.21.15.101:6827/2166787667] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f35ec10fd90 con 0x7f35d80ac1f0 2023-07-18T03:06:04.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.748+0000 7f35e2ffd700 1 -- 172.21.15.101:0/820916510 <== osd.9 v2:172.21.15.101:6826/2166787667 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f35e4009ac0 con 0x7f35d80ac1f0 2023-07-18T03:06:04.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.749+0000 7f35f3ac0700 1 -- 172.21.15.101:0/820916510 >> [v2:172.21.15.101:6826/2166787667,v1:172.21.15.101:6827/2166787667] conn(0x7f35d80ac1f0 msgr2=0x7f35d80ae680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:04.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.749+0000 7f35f3ac0700 1 --2- 172.21.15.101:0/820916510 >> [v2:172.21.15.101:6826/2166787667,v1:172.21.15.101:6827/2166787667] conn(0x7f35d80ac1f0 0x7f35d80ae680 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:04.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.749+0000 7f35f3ac0700 1 -- 172.21.15.101:0/820916510 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f35d80a1260 msgr2=0x7f35d80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:04.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.749+0000 7f35f3ac0700 1 --2- 172.21.15.101:0/820916510 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f35d80a1260 0x7f35d80a3710 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f35ec074000 tx=0x7f35dc009260 comp rx=0 tx=0).stop 2023-07-18T03:06:04.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.749+0000 7f35f3ac0700 1 -- 172.21.15.101:0/820916510 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35ec0ffe00 msgr2=0x7f35ec072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:04.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.749+0000 7f35f3ac0700 1 --2- 172.21.15.101:0/820916510 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35ec0ffe00 0x7f35ec072810 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f35e800eb90 tx=0x7f35e800eea0 comp rx=0 tx=0).stop 2023-07-18T03:06:04.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.749+0000 7f35f205d700 1 -- 172.21.15.101:0/820916510 reap_dead start 2023-07-18T03:06:04.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.749+0000 7f35f3ac0700 1 -- 172.21.15.101:0/820916510 shutdown_connections 2023-07-18T03:06:04.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.749+0000 7f35f3ac0700 1 -- 172.21.15.101:0/820916510 >> 172.21.15.101:0/820916510 conn(0x7f35ec0fb660 msgr2=0x7f35ec0fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:04.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.749+0000 7f35f3ac0700 1 -- 172.21.15.101:0/820916510 shutdown_connections 2023-07-18T03:06:04.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:03.749+0000 7f35f3ac0700 1 -- 172.21.15.101:0/820916510 wait complete. 2023-07-18T03:06:04.686 INFO:teuthology.orchestra.run.smithi101.stdout:34359738443 2023-07-18T03:06:04.686 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.0 2023-07-18T03:06:05.007 INFO:teuthology.orchestra.run.smithi101.stdout:163208757300 2023-07-18T03:06:05.007 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.5 2023-07-18T03:06:05.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.235+0000 7fc3a6d03700 1 -- 172.21.15.101:0/398082285 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3a01007a0 msgr2=0x7fc3a0104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:05.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.235+0000 7fc3a6d03700 1 --2- 172.21.15.101:0/398082285 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3a01007a0 0x7fc3a0104770 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc38c0099c0 tx=0x7fc38c009cd0 comp rx=0 tx=0).stop 2023-07-18T03:06:05.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.235+0000 7fc3a6d03700 1 -- 172.21.15.101:0/398082285 shutdown_connections 2023-07-18T03:06:05.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.235+0000 7fc3a6d03700 1 --2- 172.21.15.101:0/398082285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3a0104ea0 0x7fc3a010ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:05.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.235+0000 7fc3a6d03700 1 --2- 172.21.15.101:0/398082285 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3a00ffe00 0x7fc3a01001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:05.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.235+0000 7fc3a6d03700 1 --2- 172.21.15.101:0/398082285 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3a01007a0 0x7fc3a0104770 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:05.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.235+0000 7fc3a6d03700 1 -- 172.21.15.101:0/398082285 >> 172.21.15.101:0/398082285 conn(0x7fc3a00fb660 msgr2=0x7fc3a00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:05.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.235+0000 7fc3a6d03700 1 -- 172.21.15.101:0/398082285 shutdown_connections 2023-07-18T03:06:05.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.235+0000 7fc3a6d03700 1 -- 172.21.15.101:0/398082285 wait complete. 2023-07-18T03:06:05.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.236+0000 7fc3a6d03700 1 Processor -- start 2023-07-18T03:06:05.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.237+0000 7fc3a6d03700 1 -- start start 2023-07-18T03:06:05.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.237+0000 7fc3a6d03700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3a00ffe00 0x7fc3a0199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:05.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.237+0000 7fc3a6d03700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3a01007a0 0x7fc3a0199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:05.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.237+0000 7fc3a6d03700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3a0104ea0 0x7fc3a0193990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:05.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.237+0000 7fc3a6d03700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a0194000 con 0x7fc3a00ffe00 2023-07-18T03:06:05.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.237+0000 7fc3a6d03700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a0194170 con 0x7fc3a01007a0 2023-07-18T03:06:05.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.237+0000 7fc3a6d03700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a0194330 con 0x7fc3a0104ea0 2023-07-18T03:06:05.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.237+0000 7fc39ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3a01007a0 0x7fc3a0199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:05.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.237+0000 7fc3a4a9f700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3a00ffe00 0x7fc3a0199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:05.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.237+0000 7fc3a4a9f700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3a00ffe00 0x7fc3a0199910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:53926/0 (socket says 172.21.15.101:53926) 2023-07-18T03:06:05.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.237+0000 7fc3a4a9f700 1 -- 172.21.15.101:0/1372425734 learned_addr learned my addr 172.21.15.101:0/1372425734 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:05.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.237+0000 7fc3a52a0700 1 --2- 172.21.15.101:0/1372425734 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3a0104ea0 0x7fc3a0193990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:05.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.238+0000 7fc39ffff700 1 -- 172.21.15.101:0/1372425734 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3a0104ea0 msgr2=0x7fc3a0193990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:05.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.238+0000 7fc39ffff700 1 --2- 172.21.15.101:0/1372425734 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3a0104ea0 0x7fc3a0193990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:05.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.238+0000 7fc39ffff700 1 -- 172.21.15.101:0/1372425734 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3a00ffe00 msgr2=0x7fc3a0199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:05.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.238+0000 7fc39ffff700 1 --2- 172.21.15.101:0/1372425734 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3a00ffe00 0x7fc3a0199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:05.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.238+0000 7fc39ffff700 1 -- 172.21.15.101:0/1372425734 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc38c0095f0 con 0x7fc3a01007a0 2023-07-18T03:06:05.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.238+0000 7fc3a4a9f700 1 --2- 172.21.15.101:0/1372425734 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3a00ffe00 0x7fc3a0199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:06:05.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.238+0000 7fc39ffff700 1 --2- 172.21.15.101:0/1372425734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3a01007a0 0x7fc3a0199e50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc38c009fd0 tx=0x7fc38c014ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:05.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.238+0000 7fc39dffb700 1 -- 172.21.15.101:0/1372425734 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38c01c630 con 0x7fc3a01007a0 2023-07-18T03:06:05.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.238+0000 7fc3a6d03700 1 -- 172.21.15.101:0/1372425734 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3a01945b0 con 0x7fc3a01007a0 2023-07-18T03:06:05.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.239+0000 7fc3a6d03700 1 -- 172.21.15.101:0/1372425734 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3a01a0900 con 0x7fc3a01007a0 2023-07-18T03:06:05.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.239+0000 7fc39dffb700 1 -- 172.21.15.101:0/1372425734 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc38c01cc50 con 0x7fc3a01007a0 2023-07-18T03:06:05.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.239+0000 7fc39dffb700 1 -- 172.21.15.101:0/1372425734 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38c026c40 con 0x7fc3a01007a0 2023-07-18T03:06:05.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.239+0000 7fc3a6d03700 1 -- 172.21.15.101:0/1372425734 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc384000ff0 con 0x7fc3a01007a0 2023-07-18T03:06:05.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.243+0000 7fc39dffb700 1 -- 172.21.15.101:0/1372425734 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fc38c026420 con 0x7fc3a01007a0 2023-07-18T03:06:05.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.245+0000 7fc39dffb700 1 --2- 172.21.15.101:0/1372425734 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc3900a11e0 0x7fc3900a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:05.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.245+0000 7fc39dffb700 1 -- 172.21.15.101:0/1372425734 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fc38c0d6c90 con 0x7fc3a01007a0 2023-07-18T03:06:05.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.245+0000 7fc3a4a9f700 1 --2- 172.21.15.101:0/1372425734 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc3900a11e0 0x7fc3900a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:05.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.246+0000 7fc39dffb700 1 --2- 172.21.15.101:0/1372425734 >> [v2:172.21.15.155:6808/1951135805,v1:172.21.15.155:6809/1951135805] conn(0x7fc3900ac170 0x7fc3900ae600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:05.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.246+0000 7fc39dffb700 1 -- 172.21.15.101:0/1372425734 --> [v2:172.21.15.155:6808/1951135805,v1:172.21.15.155:6809/1951135805] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc3900aecd0 con 0x7fc3900ac170 2023-07-18T03:06:05.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.246+0000 7fc39dffb700 1 -- 172.21.15.101:0/1372425734 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc38c0d70a0 con 0x7fc3a01007a0 2023-07-18T03:06:05.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.246+0000 7fc3a52a0700 1 --2- 172.21.15.101:0/1372425734 >> [v2:172.21.15.155:6808/1951135805,v1:172.21.15.155:6809/1951135805] conn(0x7fc3900ac170 0x7fc3900ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:05.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.246+0000 7fc3a4a9f700 1 --2- 172.21.15.101:0/1372425734 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc3900a11e0 0x7fc3900a3690 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc394011550 tx=0x7fc394009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:05.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.247+0000 7fc3a52a0700 1 --2- 172.21.15.101:0/1372425734 >> [v2:172.21.15.155:6808/1951135805,v1:172.21.15.155:6809/1951135805] conn(0x7fc3900ac170 0x7fc3900ae600 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:05.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.247+0000 7fc39dffb700 1 -- 172.21.15.101:0/1372425734 <== osd.4 v2:172.21.15.155:6808/1951135805 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fc3900aecd0 con 0x7fc3900ac170 2023-07-18T03:06:05.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.264+0000 7fc3a6d03700 1 -- 172.21.15.101:0/1372425734 --> [v2:172.21.15.155:6808/1951135805,v1:172.21.15.155:6809/1951135805] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc384002d70 con 0x7fc3900ac170 2023-07-18T03:06:05.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.265+0000 7fc39dffb700 1 -- 172.21.15.101:0/1372425734 <== osd.4 v2:172.21.15.155:6808/1951135805 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fc384002d70 con 0x7fc3900ac170 2023-07-18T03:06:05.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.265+0000 7fc3a6d03700 1 -- 172.21.15.101:0/1372425734 >> [v2:172.21.15.155:6808/1951135805,v1:172.21.15.155:6809/1951135805] conn(0x7fc3900ac170 msgr2=0x7fc3900ae600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:05.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.265+0000 7fc3a6d03700 1 --2- 172.21.15.101:0/1372425734 >> [v2:172.21.15.155:6808/1951135805,v1:172.21.15.155:6809/1951135805] conn(0x7fc3900ac170 0x7fc3900ae600 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:05.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.265+0000 7fc3a6d03700 1 -- 172.21.15.101:0/1372425734 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc3900a11e0 msgr2=0x7fc3900a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:05.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.265+0000 7fc3a6d03700 1 --2- 172.21.15.101:0/1372425734 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc3900a11e0 0x7fc3900a3690 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc394011550 tx=0x7fc394009260 comp rx=0 tx=0).stop 2023-07-18T03:06:05.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.265+0000 7fc3a6d03700 1 -- 172.21.15.101:0/1372425734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3a01007a0 msgr2=0x7fc3a0199e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:05.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.265+0000 7fc3a6d03700 1 --2- 172.21.15.101:0/1372425734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3a01007a0 0x7fc3a0199e50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc38c009fd0 tx=0x7fc38c014ae0 comp rx=0 tx=0).stop 2023-07-18T03:06:05.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.265+0000 7fc3a52a0700 1 -- 172.21.15.101:0/1372425734 reap_dead start 2023-07-18T03:06:05.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.265+0000 7fc3a6d03700 1 -- 172.21.15.101:0/1372425734 shutdown_connections 2023-07-18T03:06:05.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.265+0000 7fc3a6d03700 1 -- 172.21.15.101:0/1372425734 >> 172.21.15.101:0/1372425734 conn(0x7fc3a00fb660 msgr2=0x7fc3a00fc120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:05.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.266+0000 7fc3a6d03700 1 -- 172.21.15.101:0/1372425734 shutdown_connections 2023-07-18T03:06:05.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.266+0000 7fc3a6d03700 1 -- 172.21.15.101:0/1372425734 wait complete. 2023-07-18T03:06:05.684 INFO:teuthology.orchestra.run.smithi101.stdout:283467841569 2023-07-18T03:06:05.685 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.9 2023-07-18T03:06:05.769 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:06:05.783 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:06:05.842 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.854+0000 7f910bdcf700 1 -- 172.21.15.101:0/2429115926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9104107770 msgr2=0x7f9104107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:05.842 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.854+0000 7f910bdcf700 1 --2- 172.21.15.101:0/2429115926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9104107770 0x7f9104107b40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f90fc00c310 tx=0x7f90fc00c620 comp rx=0 tx=0).stop 2023-07-18T03:06:05.843 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.855+0000 7f910bdcf700 1 -- 172.21.15.101:0/2429115926 shutdown_connections 2023-07-18T03:06:05.843 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.855+0000 7f910bdcf700 1 --2- 172.21.15.101:0/2429115926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9104107770 0x7f9104107b40 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:05.843 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.855+0000 7f910bdcf700 1 --2- 172.21.15.101:0/2429115926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9104110bf0 0x7f9104112fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:05.843 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.855+0000 7f910bdcf700 1 --2- 172.21.15.101:0/2429115926 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9104108080 0x7f9104110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:05.843 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.855+0000 7f910bdcf700 1 -- 172.21.15.101:0/2429115926 >> 172.21.15.101:0/2429115926 conn(0x7f910406c370 msgr2=0x7f910406c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:05.843 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.856+0000 7f910bdcf700 1 -- 172.21.15.101:0/2429115926 shutdown_connections 2023-07-18T03:06:05.843 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.856+0000 7f910bdcf700 1 -- 172.21.15.101:0/2429115926 wait complete. 2023-07-18T03:06:05.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.858+0000 7f910bdcf700 1 Processor -- start 2023-07-18T03:06:05.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.858+0000 7f910bdcf700 1 -- start start 2023-07-18T03:06:05.845 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.859+0000 7f910bdcf700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9104107770 0x7f91041a4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:05.845 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.859+0000 7f910bdcf700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9104108080 0x7f91041a4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:05.845 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.859+0000 7f910bdcf700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9104110bf0 0x7f910419e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:05.845 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.859+0000 7f910bdcf700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910419eb00 con 0x7f9104110bf0 2023-07-18T03:06:05.846 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.859+0000 7f910bdcf700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910419ec70 con 0x7f9104108080 2023-07-18T03:06:05.846 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.859+0000 7f910bdcf700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910419ee30 con 0x7f9104107770 2023-07-18T03:06:05.846 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.860+0000 7f910936a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9104108080 0x7f91041a4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:05.846 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.860+0000 7f910a36c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9104110bf0 0x7f910419e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:05.846 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.860+0000 7f910936a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9104108080 0x7f91041a4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:55170/0 (socket says 172.21.15.101:55170) 2023-07-18T03:06:05.846 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.860+0000 7f910936a700 1 -- 172.21.15.101:0/865663225 learned_addr learned my addr 172.21.15.101:0/865663225 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:05.847 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.861+0000 7f910936a700 1 -- 172.21.15.101:0/865663225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9104107770 msgr2=0x7f91041a4540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-18T03:06:05.847 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.861+0000 7f910936a700 1 --2- 172.21.15.101:0/865663225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9104107770 0x7f91041a4540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:05.847 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.861+0000 7f910936a700 1 -- 172.21.15.101:0/865663225 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9104110bf0 msgr2=0x7f910419e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:05.847 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.861+0000 7f910936a700 1 --2- 172.21.15.101:0/865663225 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9104110bf0 0x7f910419e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:05.847 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.861+0000 7f910936a700 1 -- 172.21.15.101:0/865663225 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90fc00c050 con 0x7f9104108080 2023-07-18T03:06:05.848 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.861+0000 7f910936a700 1 --2- 172.21.15.101:0/865663225 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9104108080 0x7f91041a4a80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f90f400b500 tx=0x7f90f400b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:05.848 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.861+0000 7f90faffd700 1 -- 172.21.15.101:0/865663225 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f4011830 con 0x7f9104108080 2023-07-18T03:06:05.848 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.861+0000 7f90faffd700 1 -- 172.21.15.101:0/865663225 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90f4011e50 con 0x7f9104108080 2023-07-18T03:06:05.848 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.861+0000 7f90faffd700 1 -- 172.21.15.101:0/865663225 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f400f2e0 con 0x7f9104108080 2023-07-18T03:06:05.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.862+0000 7f910bdcf700 1 -- 172.21.15.101:0/865663225 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f910419f030 con 0x7f9104108080 2023-07-18T03:06:05.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.862+0000 7f910bdcf700 1 -- 172.21.15.101:0/865663225 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91041a9350 con 0x7f9104108080 2023-07-18T03:06:05.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.863+0000 7f910bdcf700 1 -- 172.21.15.101:0/865663225 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f91041163c0 con 0x7f9104108080 2023-07-18T03:06:05.853 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.868+0000 7f90faffd700 1 -- 172.21.15.101:0/865663225 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f90f4010580 con 0x7f9104108080 2023-07-18T03:06:05.856 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.870+0000 7f90faffd700 1 --2- 172.21.15.101:0/865663225 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f90f00a1330 0x7f90f00a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:05.856 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.870+0000 7f90faffd700 1 -- 172.21.15.101:0/865663225 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f90f40ccce0 con 0x7f9104108080 2023-07-18T03:06:05.856 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.871+0000 7f90faffd700 1 --2- 172.21.15.101:0/865663225 >> [v2:172.21.15.179:6800/1349487507,v1:172.21.15.179:6801/1349487507] conn(0x7f90f00ac2c0 0x7f90f00ae750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:05.856 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.871+0000 7f90faffd700 1 -- 172.21.15.101:0/865663225 --> [v2:172.21.15.179:6800/1349487507,v1:172.21.15.179:6801/1349487507] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f90f00aee20 con 0x7f90f00ac2c0 2023-07-18T03:06:05.856 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.871+0000 7f90faffd700 1 -- 172.21.15.101:0/865663225 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90f40cd050 con 0x7f9104108080 2023-07-18T03:06:05.857 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.871+0000 7f9109b6b700 1 --2- 172.21.15.101:0/865663225 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f90f00a1330 0x7f90f00a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:05.857 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.871+0000 7f910a36c700 1 --2- 172.21.15.101:0/865663225 >> [v2:172.21.15.179:6800/1349487507,v1:172.21.15.179:6801/1349487507] conn(0x7f90f00ac2c0 0x7f90f00ae750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:05.857 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.872+0000 7f9109b6b700 1 --2- 172.21.15.101:0/865663225 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f90f00a1330 0x7f90f00a37e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f90fc00e010 tx=0x7f90fc00ca20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:05.857 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.872+0000 7f910a36c700 1 --2- 172.21.15.101:0/865663225 >> [v2:172.21.15.179:6800/1349487507,v1:172.21.15.179:6801/1349487507] conn(0x7f90f00ac2c0 0x7f90f00ae750 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:05.858 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.873+0000 7f90faffd700 1 -- 172.21.15.101:0/865663225 <== osd.2 v2:172.21.15.179:6800/1349487507 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f90f00aee20 con 0x7f90f00ac2c0 2023-07-18T03:06:05.877 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.892+0000 7f910bdcf700 1 -- 172.21.15.101:0/865663225 --> [v2:172.21.15.179:6800/1349487507,v1:172.21.15.179:6801/1349487507] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f910404e210 con 0x7f90f00ac2c0 2023-07-18T03:06:05.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.893+0000 7f90faffd700 1 -- 172.21.15.101:0/865663225 <== osd.2 v2:172.21.15.179:6800/1349487507 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f910404e210 con 0x7f90f00ac2c0 2023-07-18T03:06:05.882 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.897+0000 7f910bdcf700 1 -- 172.21.15.101:0/865663225 >> [v2:172.21.15.179:6800/1349487507,v1:172.21.15.179:6801/1349487507] conn(0x7f90f00ac2c0 msgr2=0x7f90f00ae750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:05.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.897+0000 7f910bdcf700 1 --2- 172.21.15.101:0/865663225 >> [v2:172.21.15.179:6800/1349487507,v1:172.21.15.179:6801/1349487507] conn(0x7f90f00ac2c0 0x7f90f00ae750 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:05.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.898+0000 7f910bdcf700 1 -- 172.21.15.101:0/865663225 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f90f00a1330 msgr2=0x7f90f00a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:05.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.898+0000 7f910bdcf700 1 --2- 172.21.15.101:0/865663225 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f90f00a1330 0x7f90f00a37e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f90fc00e010 tx=0x7f90fc00ca20 comp rx=0 tx=0).stop 2023-07-18T03:06:05.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.898+0000 7f910bdcf700 1 -- 172.21.15.101:0/865663225 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9104108080 msgr2=0x7f91041a4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:05.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.898+0000 7f910bdcf700 1 --2- 172.21.15.101:0/865663225 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9104108080 0x7f91041a4a80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f90f400b500 tx=0x7f90f400b8c0 comp rx=0 tx=0).stop 2023-07-18T03:06:05.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.898+0000 7f910a36c700 1 -- 172.21.15.101:0/865663225 reap_dead start 2023-07-18T03:06:05.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.898+0000 7f910bdcf700 1 -- 172.21.15.101:0/865663225 shutdown_connections 2023-07-18T03:06:05.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.898+0000 7f910bdcf700 1 -- 172.21.15.101:0/865663225 >> 172.21.15.101:0/865663225 conn(0x7f910406c370 msgr2=0x7f91040700e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:05.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.898+0000 7f910bdcf700 1 -- 172.21.15.101:0/865663225 shutdown_connections 2023-07-18T03:06:05.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:05.898+0000 7f910bdcf700 1 -- 172.21.15.101:0/865663225 wait complete. 2023-07-18T03:06:06.108 INFO:teuthology.orchestra.run.smithi101.stdout:304942678046 2023-07-18T03:06:06.108 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.10 2023-07-18T03:06:06.168 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:05 smithi101 ceph-mon[126379]: pgmap v310: 129 pgs: 129 active+clean; 640 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2023-07-18T03:06:06.169 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:05 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:06:06.169 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:05 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:06:06.169 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:05 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:06:06.170 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:05 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:06:06.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:05 smithi179 ceph-mon[129229]: pgmap v310: 129 pgs: 129 active+clean; 640 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2023-07-18T03:06:06.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:05 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:06:06.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:05 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:06:06.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:05 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:06:06.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:05 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:06:06.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:05 smithi155 ceph-mon[128885]: pgmap v310: 129 pgs: 129 active+clean; 640 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2023-07-18T03:06:06.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:05 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:06:06.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:05 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:06:06.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:05 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:06:06.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:05 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:06:06.629 INFO:teuthology.orchestra.run.smithi101.stdout:141733920825 2023-07-18T03:06:06.629 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.4 2023-07-18T03:06:06.954 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:06:06.965 INFO:teuthology.orchestra.run.smithi101.stdout:77309411396 2023-07-18T03:06:06.966 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.2 2023-07-18T03:06:08.015 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:07 smithi101 ceph-mon[126379]: pgmap v311: 129 pgs: 129 active+clean; 640 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 170 B/s wr, 6 op/s 2023-07-18T03:06:08.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:07 smithi179 ceph-mon[129229]: pgmap v311: 129 pgs: 129 active+clean; 640 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 170 B/s wr, 6 op/s 2023-07-18T03:06:08.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:07 smithi155 ceph-mon[128885]: pgmap v311: 129 pgs: 129 active+clean; 640 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 170 B/s wr, 6 op/s 2023-07-18T03:06:08.495 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:06:09.214 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:06:09.463 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:06:09.468 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:06:09.795 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:06:10.037 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:09 smithi155 ceph-mon[128885]: pgmap v312: 129 pgs: 129 active+clean; 640 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2023-07-18T03:06:10.047 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:09 smithi101 ceph-mon[126379]: pgmap v312: 129 pgs: 129 active+clean; 640 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2023-07-18T03:06:10.190 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:06:10.190 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:06:10.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:09 smithi179 ceph-mon[129229]: pgmap v312: 129 pgs: 129 active+clean; 640 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2023-07-18T03:06:10.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.526+0000 7f6c2fd8d700 1 -- 172.21.15.101:0/2865419779 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c281007a0 msgr2=0x7f6c28104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:10.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.526+0000 7f6c2fd8d700 1 --2- 172.21.15.101:0/2865419779 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c281007a0 0x7f6c28104770 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6c180099c0 tx=0x7f6c18009cd0 comp rx=0 tx=0).stop 2023-07-18T03:06:10.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.526+0000 7f6c2fd8d700 1 -- 172.21.15.101:0/2865419779 shutdown_connections 2023-07-18T03:06:10.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.526+0000 7f6c2fd8d700 1 --2- 172.21.15.101:0/2865419779 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c280ffe00 0x7f6c281001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:10.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.526+0000 7f6c2fd8d700 1 --2- 172.21.15.101:0/2865419779 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c28104ea0 0x7f6c2810ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:10.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.526+0000 7f6c2fd8d700 1 --2- 172.21.15.101:0/2865419779 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c281007a0 0x7f6c28104770 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:10.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.526+0000 7f6c2fd8d700 1 -- 172.21.15.101:0/2865419779 >> 172.21.15.101:0/2865419779 conn(0x7f6c280fb660 msgr2=0x7f6c280fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:10.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.526+0000 7f6c2fd8d700 1 -- 172.21.15.101:0/2865419779 shutdown_connections 2023-07-18T03:06:10.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.526+0000 7f6c2fd8d700 1 -- 172.21.15.101:0/2865419779 wait complete. 2023-07-18T03:06:10.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.527+0000 7f6c2fd8d700 1 Processor -- start 2023-07-18T03:06:10.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.527+0000 7f6c2fd8d700 1 -- start start 2023-07-18T03:06:10.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.528+0000 7f6c2fd8d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c280ffe00 0x7f6c28072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:10.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.528+0000 7f6c2fd8d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c281007a0 0x7f6c28072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:10.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.528+0000 7f6c2fd8d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c28104ea0 0x7f6c280732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:10.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.528+0000 7f6c2fd8d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c281a41a0 con 0x7f6c281007a0 2023-07-18T03:06:10.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.528+0000 7f6c2fd8d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c281a4310 con 0x7f6c280ffe00 2023-07-18T03:06:10.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.528+0000 7f6c2fd8d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c281a44d0 con 0x7f6c28104ea0 2023-07-18T03:06:10.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.528+0000 7f6c2d328700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c281007a0 0x7f6c28072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:10.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.528+0000 7f6c2db29700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c280ffe00 0x7f6c28072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:10.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.528+0000 7f6c2d328700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c281007a0 0x7f6c28072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:54888/0 (socket says 172.21.15.101:54888) 2023-07-18T03:06:10.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.528+0000 7f6c2d328700 1 -- 172.21.15.101:0/147102511 learned_addr learned my addr 172.21.15.101:0/147102511 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:10.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.528+0000 7f6c2e32a700 1 --2- 172.21.15.101:0/147102511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c28104ea0 0x7f6c280732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:10.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.529+0000 7f6c2d328700 1 -- 172.21.15.101:0/147102511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c28104ea0 msgr2=0x7f6c280732d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:10.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.529+0000 7f6c2d328700 1 --2- 172.21.15.101:0/147102511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c28104ea0 0x7f6c280732d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:10.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.529+0000 7f6c2d328700 1 -- 172.21.15.101:0/147102511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c280ffe00 msgr2=0x7f6c28072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:10.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.529+0000 7f6c2d328700 1 --2- 172.21.15.101:0/147102511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c280ffe00 0x7f6c28072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:10.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.529+0000 7f6c2d328700 1 -- 172.21.15.101:0/147102511 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c180095f0 con 0x7f6c281007a0 2023-07-18T03:06:10.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.529+0000 7f6c2db29700 1 --2- 172.21.15.101:0/147102511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c280ffe00 0x7f6c28072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:06:10.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.529+0000 7f6c2d328700 1 --2- 172.21.15.101:0/147102511 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c281007a0 0x7f6c28072d50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6c180098e0 tx=0x7f6c1800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:10.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.529+0000 7f6c1effd700 1 -- 172.21.15.101:0/147102511 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1801d070 con 0x7f6c281007a0 2023-07-18T03:06:10.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.529+0000 7f6c2fd8d700 1 -- 172.21.15.101:0/147102511 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c281a4750 con 0x7f6c281007a0 2023-07-18T03:06:10.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.529+0000 7f6c1effd700 1 -- 172.21.15.101:0/147102511 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c18005050 con 0x7f6c281007a0 2023-07-18T03:06:10.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.529+0000 7f6c1effd700 1 -- 172.21.15.101:0/147102511 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c18004970 con 0x7f6c281007a0 2023-07-18T03:06:10.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.529+0000 7f6c2fd8d700 1 -- 172.21.15.101:0/147102511 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c281a4b70 con 0x7f6c281007a0 2023-07-18T03:06:10.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.530+0000 7f6c2fd8d700 1 -- 172.21.15.101:0/147102511 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c2804e210 con 0x7f6c281007a0 2023-07-18T03:06:10.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.534+0000 7f6c1effd700 1 -- 172.21.15.101:0/147102511 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f6c18005430 con 0x7f6c281007a0 2023-07-18T03:06:10.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.535+0000 7f6c1effd700 1 --2- 172.21.15.101:0/147102511 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f6c140a11e0 0x7f6c140a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:10.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.535+0000 7f6c1effd700 1 -- 172.21.15.101:0/147102511 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f6c180cdfb0 con 0x7f6c281007a0 2023-07-18T03:06:10.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.535+0000 7f6c2db29700 1 --2- 172.21.15.101:0/147102511 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f6c140a11e0 0x7f6c140a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:10.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.536+0000 7f6c2db29700 1 --2- 172.21.15.101:0/147102511 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f6c140a11e0 0x7f6c140a3690 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6c24011490 tx=0x7f6c24009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:10.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.537+0000 7f6c1effd700 1 -- 172.21.15.101:0/147102511 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6c180972d0 con 0x7f6c281007a0 2023-07-18T03:06:10.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.767+0000 7f6c2fd8d700 1 -- 172.21.15.101:0/147102511 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6c281a4e30 con 0x7f6c281007a0 2023-07-18T03:06:10.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.767+0000 7f6c1effd700 1 -- 172.21.15.101:0/147102511 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6c1809a8f0 con 0x7f6c281007a0 2023-07-18T03:06:10.754 INFO:teuthology.orchestra.run.smithi101.stdout:107374182462 2023-07-18T03:06:10.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.769+0000 7f6c2fd8d700 1 -- 172.21.15.101:0/147102511 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f6c140a11e0 msgr2=0x7f6c140a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:10.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.769+0000 7f6c2fd8d700 1 --2- 172.21.15.101:0/147102511 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f6c140a11e0 0x7f6c140a3690 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6c24011490 tx=0x7f6c24009260 comp rx=0 tx=0).stop 2023-07-18T03:06:10.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.769+0000 7f6c2fd8d700 1 -- 172.21.15.101:0/147102511 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c281007a0 msgr2=0x7f6c28072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:10.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.769+0000 7f6c2fd8d700 1 --2- 172.21.15.101:0/147102511 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c281007a0 0x7f6c28072d50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6c180098e0 tx=0x7f6c1800f690 comp rx=0 tx=0).stop 2023-07-18T03:06:10.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.770+0000 7f6c2fd8d700 1 -- 172.21.15.101:0/147102511 shutdown_connections 2023-07-18T03:06:10.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.770+0000 7f6c2fd8d700 1 --2- 172.21.15.101:0/147102511 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f6c140a11e0 0x7f6c140a3690 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:10.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.770+0000 7f6c2fd8d700 1 --2- 172.21.15.101:0/147102511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c280ffe00 0x7f6c28072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:10.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.770+0000 7f6c2fd8d700 1 --2- 172.21.15.101:0/147102511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c28104ea0 0x7f6c280732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:10.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.770+0000 7f6c2fd8d700 1 --2- 172.21.15.101:0/147102511 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c281007a0 0x7f6c28072d50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:10.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.770+0000 7f6c2fd8d700 1 -- 172.21.15.101:0/147102511 >> 172.21.15.101:0/147102511 conn(0x7f6c280fb660 msgr2=0x7f6c280fc170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:10.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.770+0000 7f6c2fd8d700 1 -- 172.21.15.101:0/147102511 shutdown_connections 2023-07-18T03:06:10.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:10.770+0000 7f6c2fd8d700 1 -- 172.21.15.101:0/147102511 wait complete. 2023-07-18T03:06:10.817 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:06:11.104 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:10 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/147102511' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-07-18T03:06:11.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:10 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/147102511' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-07-18T03:06:11.246 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:06:11.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:10 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/147102511' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-07-18T03:06:11.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.542+0000 7f59ed890700 1 -- 172.21.15.101:0/991459874 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f59e8104ea0 msgr2=0x7f59e810ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:11.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.542+0000 7f59ed890700 1 --2- 172.21.15.101:0/991459874 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f59e8104ea0 0x7f59e810ca20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f59e000b0d0 tx=0x7f59e000b3e0 comp rx=0 tx=0).stop 2023-07-18T03:06:11.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.542+0000 7f59ed890700 1 -- 172.21.15.101:0/991459874 shutdown_connections 2023-07-18T03:06:11.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.542+0000 7f59ed890700 1 --2- 172.21.15.101:0/991459874 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f59e8104ea0 0x7f59e810ca20 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:11.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.542+0000 7f59ed890700 1 --2- 172.21.15.101:0/991459874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59e80ffe00 0x7f59e81001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:11.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.542+0000 7f59ed890700 1 --2- 172.21.15.101:0/991459874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59e81007a0 0x7f59e8104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:11.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.542+0000 7f59ed890700 1 -- 172.21.15.101:0/991459874 >> 172.21.15.101:0/991459874 conn(0x7f59e80fb660 msgr2=0x7f59e80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:11.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.542+0000 7f59ed890700 1 -- 172.21.15.101:0/991459874 shutdown_connections 2023-07-18T03:06:11.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.542+0000 7f59ed890700 1 -- 172.21.15.101:0/991459874 wait complete. 2023-07-18T03:06:11.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.543+0000 7f59ed890700 1 Processor -- start 2023-07-18T03:06:11.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.543+0000 7f59ed890700 1 -- start start 2023-07-18T03:06:11.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.543+0000 7f59ed890700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f59e80ffe00 0x7f59e8106aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:11.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.543+0000 7f59ed890700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59e81007a0 0x7f59e8106fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:11.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.543+0000 7f59ed890700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59e8104ea0 0x7f59e8107520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:11.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.543+0000 7f59ed890700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e81a85e0 con 0x7f59e80ffe00 2023-07-18T03:06:11.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.543+0000 7f59ed890700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e81a8750 con 0x7f59e81007a0 2023-07-18T03:06:11.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.543+0000 7f59ed890700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e81a8910 con 0x7f59e8104ea0 2023-07-18T03:06:11.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.544+0000 7f59e6ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f59e80ffe00 0x7f59e8106aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:11.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.544+0000 7f59e6ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f59e80ffe00 0x7f59e8106aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:54906/0 (socket says 172.21.15.101:54906) 2023-07-18T03:06:11.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.544+0000 7f59e6ffd700 1 -- 172.21.15.101:0/2728165548 learned_addr learned my addr 172.21.15.101:0/2728165548 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:11.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.544+0000 7f59e67fc700 1 --2- 172.21.15.101:0/2728165548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59e81007a0 0x7f59e8106fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:11.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.544+0000 7f59e77fe700 1 --2- 172.21.15.101:0/2728165548 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59e8104ea0 0x7f59e8107520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:11.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.544+0000 7f59e6ffd700 1 -- 172.21.15.101:0/2728165548 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59e8104ea0 msgr2=0x7f59e8107520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:11.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.544+0000 7f59e6ffd700 1 --2- 172.21.15.101:0/2728165548 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59e8104ea0 0x7f59e8107520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:11.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.544+0000 7f59e6ffd700 1 -- 172.21.15.101:0/2728165548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59e81007a0 msgr2=0x7f59e8106fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:11.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.544+0000 7f59e6ffd700 1 --2- 172.21.15.101:0/2728165548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59e81007a0 0x7f59e8106fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:11.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.544+0000 7f59e6ffd700 1 -- 172.21.15.101:0/2728165548 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e0009d00 con 0x7f59e80ffe00 2023-07-18T03:06:11.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.545+0000 7f59e6ffd700 1 --2- 172.21.15.101:0/2728165548 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f59e80ffe00 0x7f59e8106aa0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f59d800e9a0 tx=0x7f59d800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:11.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.545+0000 7f59ec88e700 1 -- 172.21.15.101:0/2728165548 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d8009790 con 0x7f59e80ffe00 2023-07-18T03:06:11.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.545+0000 7f59ec88e700 1 -- 172.21.15.101:0/2728165548 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59d8010430 con 0x7f59e80ffe00 2023-07-18T03:06:11.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.545+0000 7f59ec88e700 1 -- 172.21.15.101:0/2728165548 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d8018680 con 0x7f59e80ffe00 2023-07-18T03:06:11.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.545+0000 7f59ed890700 1 -- 172.21.15.101:0/2728165548 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59e81a8bf0 con 0x7f59e80ffe00 2023-07-18T03:06:11.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.545+0000 7f59ed890700 1 -- 172.21.15.101:0/2728165548 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59e81a90a0 con 0x7f59e80ffe00 2023-07-18T03:06:11.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.546+0000 7f59ed890700 1 -- 172.21.15.101:0/2728165548 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59e804e210 con 0x7f59e80ffe00 2023-07-18T03:06:11.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.550+0000 7f59ec88e700 1 -- 172.21.15.101:0/2728165548 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f59d80105a0 con 0x7f59e80ffe00 2023-07-18T03:06:11.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.552+0000 7f59ec88e700 1 --2- 172.21.15.101:0/2728165548 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f59d00a11e0 0x7f59d00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:11.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.552+0000 7f59ec88e700 1 -- 172.21.15.101:0/2728165548 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f59d8014070 con 0x7f59e80ffe00 2023-07-18T03:06:11.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.552+0000 7f59e67fc700 1 --2- 172.21.15.101:0/2728165548 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f59d00a11e0 0x7f59d00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:11.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.553+0000 7f59e67fc700 1 --2- 172.21.15.101:0/2728165548 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f59d00a11e0 0x7f59d00a3690 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f59e8108340 tx=0x7f59dc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:11.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.555+0000 7f59ec88e700 1 -- 172.21.15.101:0/2728165548 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f59d8051790 con 0x7f59e80ffe00 2023-07-18T03:06:11.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.776+0000 7f59ed890700 1 -- 172.21.15.101:0/2728165548 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f59e8107fd0 con 0x7f59e80ffe00 2023-07-18T03:06:11.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.777+0000 7f59ec88e700 1 -- 172.21.15.101:0/2728165548 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f59d8096250 con 0x7f59e80ffe00 2023-07-18T03:06:11.763 INFO:teuthology.orchestra.run.smithi101.stdout:335007449114 2023-07-18T03:06:11.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.779+0000 7f59ed890700 1 -- 172.21.15.101:0/2728165548 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f59d00a11e0 msgr2=0x7f59d00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:11.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.779+0000 7f59ed890700 1 --2- 172.21.15.101:0/2728165548 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f59d00a11e0 0x7f59d00a3690 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f59e8108340 tx=0x7f59dc009260 comp rx=0 tx=0).stop 2023-07-18T03:06:11.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.779+0000 7f59ed890700 1 -- 172.21.15.101:0/2728165548 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f59e80ffe00 msgr2=0x7f59e8106aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:11.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.779+0000 7f59ed890700 1 --2- 172.21.15.101:0/2728165548 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f59e80ffe00 0x7f59e8106aa0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f59d800e9a0 tx=0x7f59d800ecb0 comp rx=0 tx=0).stop 2023-07-18T03:06:11.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.779+0000 7f59ed890700 1 -- 172.21.15.101:0/2728165548 shutdown_connections 2023-07-18T03:06:11.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.779+0000 7f59ed890700 1 --2- 172.21.15.101:0/2728165548 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f59d00a11e0 0x7f59d00a3690 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:11.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.779+0000 7f59ed890700 1 --2- 172.21.15.101:0/2728165548 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59e8104ea0 0x7f59e8107520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:11.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.779+0000 7f59ed890700 1 --2- 172.21.15.101:0/2728165548 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f59e80ffe00 0x7f59e8106aa0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:11.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.779+0000 7f59ed890700 1 --2- 172.21.15.101:0/2728165548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59e81007a0 0x7f59e8106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:11.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.779+0000 7f59ed890700 1 -- 172.21.15.101:0/2728165548 >> 172.21.15.101:0/2728165548 conn(0x7f59e80fb660 msgr2=0x7f59e80fc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:11.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.779+0000 7f59ed890700 1 -- 172.21.15.101:0/2728165548 shutdown_connections 2023-07-18T03:06:11.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:11.779+0000 7f59ed890700 1 -- 172.21.15.101:0/2728165548 wait complete. 2023-07-18T03:06:11.955 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:11 smithi101 ceph-mon[126379]: pgmap v313: 129 pgs: 129 active+clean; 640 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2023-07-18T03:06:11.956 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:11 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2728165548' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-07-18T03:06:12.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:11 smithi179 ceph-mon[129229]: pgmap v313: 129 pgs: 129 active+clean; 640 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2023-07-18T03:06:12.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:11 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/2728165548' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-07-18T03:06:12.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:11 smithi155 ceph-mon[128885]: pgmap v313: 129 pgs: 129 active+clean; 640 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2023-07-18T03:06:12.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:11 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/2728165548' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-07-18T03:06:13.412 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182460 got 107374182462 for osd.3 2023-07-18T03:06:13.412 DEBUG:teuthology.parallel:result is None 2023-07-18T03:06:14.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.726+0000 7f488e62a700 1 -- 172.21.15.101:0/3416019005 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f48881007a0 msgr2=0x7f4888104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:14.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.726+0000 7f488e62a700 1 --2- 172.21.15.101:0/3416019005 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f48881007a0 0x7f4888104770 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f487c0099c0 tx=0x7f487c009cd0 comp rx=0 tx=0).stop 2023-07-18T03:06:14.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.726+0000 7f488e62a700 1 -- 172.21.15.101:0/3416019005 shutdown_connections 2023-07-18T03:06:14.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.726+0000 7f488e62a700 1 --2- 172.21.15.101:0/3416019005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4888104ea0 0x7f488810ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:14.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.726+0000 7f488e62a700 1 --2- 172.21.15.101:0/3416019005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48880ffe00 0x7f48881001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:14.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.726+0000 7f488e62a700 1 --2- 172.21.15.101:0/3416019005 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f48881007a0 0x7f4888104770 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:14.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.726+0000 7f488e62a700 1 -- 172.21.15.101:0/3416019005 >> 172.21.15.101:0/3416019005 conn(0x7f48880fb660 msgr2=0x7f48880fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:14.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.726+0000 7f488e62a700 1 -- 172.21.15.101:0/3416019005 shutdown_connections 2023-07-18T03:06:14.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.726+0000 7f488e62a700 1 -- 172.21.15.101:0/3416019005 wait complete. 2023-07-18T03:06:14.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.728+0000 7f488e62a700 1 Processor -- start 2023-07-18T03:06:14.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.728+0000 7f488e62a700 1 -- start start 2023-07-18T03:06:14.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.728+0000 7f488e62a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f48880ffe00 0x7f4888197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:14.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.728+0000 7f488e62a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48881007a0 0x7f4888198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:14.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.728+0000 7f488e62a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4888104ea0 0x7f488819c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:14.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.728+0000 7f488e62a700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488819cde0 con 0x7f48880ffe00 2023-07-18T03:06:14.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.728+0000 7f488e62a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488819cf50 con 0x7f48881007a0 2023-07-18T03:06:14.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.728+0000 7f488e62a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488819d110 con 0x7f4888104ea0 2023-07-18T03:06:14.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.728+0000 7f4887fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f48880ffe00 0x7f4888197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:14.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.728+0000 7f4887fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f48880ffe00 0x7f4888197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:54918/0 (socket says 172.21.15.101:54918) 2023-07-18T03:06:14.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.728+0000 7f4887fff700 1 -- 172.21.15.101:0/1354129873 learned_addr learned my addr 172.21.15.101:0/1354129873 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:14.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.728+0000 7f48877fe700 1 --2- 172.21.15.101:0/1354129873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48881007a0 0x7f4888198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:14.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.729+0000 7f488cbc7700 1 --2- 172.21.15.101:0/1354129873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4888104ea0 0x7f488819c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:14.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.729+0000 7f4887fff700 1 -- 172.21.15.101:0/1354129873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4888104ea0 msgr2=0x7f488819c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:14.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.729+0000 7f4887fff700 1 --2- 172.21.15.101:0/1354129873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4888104ea0 0x7f488819c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:14.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.729+0000 7f4887fff700 1 -- 172.21.15.101:0/1354129873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48881007a0 msgr2=0x7f4888198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:14.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.729+0000 7f4887fff700 1 --2- 172.21.15.101:0/1354129873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48881007a0 0x7f4888198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:14.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.729+0000 7f4887fff700 1 -- 172.21.15.101:0/1354129873 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f487c0095f0 con 0x7f48880ffe00 2023-07-18T03:06:14.070 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.729+0000 7f4887fff700 1 --2- 172.21.15.101:0/1354129873 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f48880ffe00 0x7f4888197e20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f487800e9a0 tx=0x7f487800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:14.070 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.729+0000 7f48857fa700 1 -- 172.21.15.101:0/1354129873 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4878009790 con 0x7f48880ffe00 2023-07-18T03:06:14.070 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.729+0000 7f488e62a700 1 -- 172.21.15.101:0/1354129873 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f488819d3f0 con 0x7f48880ffe00 2023-07-18T03:06:14.070 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.729+0000 7f48857fa700 1 -- 172.21.15.101:0/1354129873 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f487800cbd0 con 0x7f48880ffe00 2023-07-18T03:06:14.070 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.729+0000 7f48857fa700 1 -- 172.21.15.101:0/1354129873 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4878018680 con 0x7f48880ffe00 2023-07-18T03:06:14.070 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.729+0000 7f488e62a700 1 -- 172.21.15.101:0/1354129873 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48881a4c50 con 0x7f48880ffe00 2023-07-18T03:06:14.070 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.731+0000 7f488e62a700 1 -- 172.21.15.101:0/1354129873 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f488804e210 con 0x7f48880ffe00 2023-07-18T03:06:14.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.733+0000 7f48857fa700 1 -- 172.21.15.101:0/1354129873 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f487800cd40 con 0x7f48880ffe00 2023-07-18T03:06:14.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.734+0000 7f48857fa700 1 --2- 172.21.15.101:0/1354129873 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f48700a11e0 0x7f48700a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:14.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.734+0000 7f48857fa700 1 -- 172.21.15.101:0/1354129873 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f4878014070 con 0x7f48880ffe00 2023-07-18T03:06:14.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.734+0000 7f48877fe700 1 --2- 172.21.15.101:0/1354129873 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f48700a11e0 0x7f48700a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:14.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.735+0000 7f48877fe700 1 --2- 172.21.15.101:0/1354129873 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f48700a11e0 0x7f48700a3690 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f487c009fd0 tx=0x7f487c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:14.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.736+0000 7f48857fa700 1 -- 172.21.15.101:0/1354129873 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f48780965c0 con 0x7f48880ffe00 2023-07-18T03:06:14.072 INFO:teuthology.orchestra.run.smithi101.stdout:197568495665 2023-07-18T03:06:14.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.950+0000 7f488e62a700 1 -- 172.21.15.101:0/1354129873 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f48881a4ee0 con 0x7f48880ffe00 2023-07-18T03:06:14.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.951+0000 7f48857fa700 1 -- 172.21.15.101:0/1354129873 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4878099be0 con 0x7f48880ffe00 2023-07-18T03:06:14.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.954+0000 7f488e62a700 1 -- 172.21.15.101:0/1354129873 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f48700a11e0 msgr2=0x7f48700a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:14.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.954+0000 7f488e62a700 1 --2- 172.21.15.101:0/1354129873 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f48700a11e0 0x7f48700a3690 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f487c009fd0 tx=0x7f487c005fb0 comp rx=0 tx=0).stop 2023-07-18T03:06:14.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.954+0000 7f488e62a700 1 -- 172.21.15.101:0/1354129873 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f48880ffe00 msgr2=0x7f4888197e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:14.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.954+0000 7f488e62a700 1 --2- 172.21.15.101:0/1354129873 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f48880ffe00 0x7f4888197e20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f487800e9a0 tx=0x7f487800ecb0 comp rx=0 tx=0).stop 2023-07-18T03:06:14.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.955+0000 7f488e62a700 1 -- 172.21.15.101:0/1354129873 shutdown_connections 2023-07-18T03:06:14.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.955+0000 7f488e62a700 1 --2- 172.21.15.101:0/1354129873 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f48700a11e0 0x7f48700a3690 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:14.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.955+0000 7f488e62a700 1 --2- 172.21.15.101:0/1354129873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4888104ea0 0x7f488819c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:14.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.955+0000 7f488e62a700 1 --2- 172.21.15.101:0/1354129873 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f48880ffe00 0x7f4888197e20 secure :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f487800e9a0 tx=0x7f487800ecb0 comp rx=0 tx=0).stop 2023-07-18T03:06:14.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.955+0000 7f488e62a700 1 --2- 172.21.15.101:0/1354129873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48881007a0 0x7f4888198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:14.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.955+0000 7f488e62a700 1 -- 172.21.15.101:0/1354129873 >> 172.21.15.101:0/1354129873 conn(0x7f48880fb660 msgr2=0x7f48880fc3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:14.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.957+0000 7f488e62a700 1 -- 172.21.15.101:0/1354129873 shutdown_connections 2023-07-18T03:06:14.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:13.957+0000 7f488e62a700 1 -- 172.21.15.101:0/1354129873 wait complete. 2023-07-18T03:06:14.174 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:13 smithi101 ceph-mon[126379]: pgmap v314: 129 pgs: 129 active+clean; 640 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 6 op/s 2023-07-18T03:06:14.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:13 smithi179 ceph-mon[129229]: pgmap v314: 129 pgs: 129 active+clean; 640 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 6 op/s 2023-07-18T03:06:14.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:13 smithi155 ceph-mon[128885]: pgmap v314: 129 pgs: 129 active+clean; 640 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 6 op/s 2023-07-18T03:06:15.147 INFO:tasks.cephadm.ceph_manager.ceph:need seq 335007449111 got 335007449114 for osd.11 2023-07-18T03:06:15.147 DEBUG:teuthology.parallel:result is None 2023-07-18T03:06:15.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:14 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1354129873' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-07-18T03:06:15.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:14 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1354129873' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-07-18T03:06:15.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:14 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1354129873' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-07-18T03:06:15.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:15.880+0000 7fda1d30b700 1 -- 172.21.15.101:0/2975021398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda180ff290 msgr2=0x7fda1810bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:15.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:15.880+0000 7fda1d30b700 1 --2- 172.21.15.101:0/2975021398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda180ff290 0x7fda1810bff0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fda08009990 tx=0x7fda08009ca0 comp rx=0 tx=0).stop 2023-07-18T03:06:15.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:15.880+0000 7fda1d30b700 1 -- 172.21.15.101:0/2975021398 shutdown_connections 2023-07-18T03:06:15.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:15.880+0000 7fda1d30b700 1 --2- 172.21.15.101:0/2975021398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda180fe980 0x7fda180fed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:15.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:15.880+0000 7fda1d30b700 1 --2- 172.21.15.101:0/2975021398 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda1810c530 0x7fda1810e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:15.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:15.880+0000 7fda1d30b700 1 --2- 172.21.15.101:0/2975021398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda180ff290 0x7fda1810bff0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:15.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:15.880+0000 7fda1d30b700 1 -- 172.21.15.101:0/2975021398 >> 172.21.15.101:0/2975021398 conn(0x7fda18075b70 msgr2=0x7fda18075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:15.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:15.880+0000 7fda1d30b700 1 -- 172.21.15.101:0/2975021398 shutdown_connections 2023-07-18T03:06:15.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:15.880+0000 7fda1d30b700 1 -- 172.21.15.101:0/2975021398 wait complete. 2023-07-18T03:06:15.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:15.881+0000 7fda1d30b700 1 Processor -- start 2023-07-18T03:06:15.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:15.881+0000 7fda1d30b700 1 -- start start 2023-07-18T03:06:15.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:15.882+0000 7fda1d30b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda180fe980 0x7fda18197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:15.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:15.882+0000 7fda1d30b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda180ff290 0x7fda181983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:15.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:15.882+0000 7fda1d30b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda1810c530 0x7fda1819c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:15.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:15.882+0000 7fda1d30b700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1819ce20 con 0x7fda1810c530 2023-07-18T03:06:15.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:15.882+0000 7fda1d30b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1819cf90 con 0x7fda180fe980 2023-07-18T03:06:15.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:15.882+0000 7fda1d30b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1819d150 con 0x7fda180ff290 2023-07-18T03:06:15.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:15.882+0000 7fda16ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda180fe980 0x7fda18197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:15.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:15.882+0000 7fda16ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda180fe980 0x7fda18197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:59944/0 (socket says 172.21.15.101:59944) 2023-07-18T03:06:15.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:15.882+0000 7fda16ffd700 1 -- 172.21.15.101:0/4044190658 learned_addr learned my addr 172.21.15.101:0/4044190658 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:15.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:15.882+0000 7fda167fc700 1 --2- 172.21.15.101:0/4044190658 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda180ff290 0x7fda181983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:15.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:15.882+0000 7fda177fe700 1 --2- 172.21.15.101:0/4044190658 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda1810c530 0x7fda1819c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:15.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:15.883+0000 7fda177fe700 1 -- 172.21.15.101:0/4044190658 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda180ff290 msgr2=0x7fda181983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:15.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:15.883+0000 7fda177fe700 1 --2- 172.21.15.101:0/4044190658 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda180ff290 0x7fda181983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:15.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:15.883+0000 7fda177fe700 1 -- 172.21.15.101:0/4044190658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda180fe980 msgr2=0x7fda18197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:15.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:15.883+0000 7fda177fe700 1 --2- 172.21.15.101:0/4044190658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda180fe980 0x7fda18197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:15.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:15.883+0000 7fda177fe700 1 -- 172.21.15.101:0/4044190658 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda10009c40 con 0x7fda1810c530 2023-07-18T03:06:15.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:15.883+0000 7fda16ffd700 1 --2- 172.21.15.101:0/4044190658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda180fe980 0x7fda18197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:06:15.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:15.883+0000 7fda167fc700 1 --2- 172.21.15.101:0/4044190658 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda180ff290 0x7fda181983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:06:15.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:15.883+0000 7fda177fe700 1 --2- 172.21.15.101:0/4044190658 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda1810c530 0x7fda1819c810 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fda18100b60 tx=0x7fda1000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:15.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:15.884+0000 7fd9fffff700 1 -- 172.21.15.101:0/4044190658 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda10018070 con 0x7fda1810c530 2023-07-18T03:06:15.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:15.884+0000 7fd9fffff700 1 -- 172.21.15.101:0/4044190658 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda1000f040 con 0x7fda1810c530 2023-07-18T03:06:15.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:15.884+0000 7fda1d30b700 1 -- 172.21.15.101:0/4044190658 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda080095f0 con 0x7fda1810c530 2023-07-18T03:06:15.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:15.884+0000 7fd9fffff700 1 -- 172.21.15.101:0/4044190658 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda10004280 con 0x7fda1810c530 2023-07-18T03:06:15.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:15.884+0000 7fda1d30b700 1 -- 172.21.15.101:0/4044190658 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda181a4ac0 con 0x7fda1810c530 2023-07-18T03:06:15.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:15.885+0000 7fda1d30b700 1 -- 172.21.15.101:0/4044190658 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda1804e210 con 0x7fda1810c530 2023-07-18T03:06:15.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:15.887+0000 7fd9fffff700 1 -- 172.21.15.101:0/4044190658 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fda10003a00 con 0x7fda1810c530 2023-07-18T03:06:15.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:15.888+0000 7fd9fffff700 1 --2- 172.21.15.101:0/4044190658 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fda000a11e0 0x7fda000a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:15.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:15.888+0000 7fd9fffff700 1 -- 172.21.15.101:0/4044190658 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fda100cc4d0 con 0x7fda1810c530 2023-07-18T03:06:15.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:15.889+0000 7fda16ffd700 1 --2- 172.21.15.101:0/4044190658 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fda000a11e0 0x7fda000a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:15.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:15.889+0000 7fda16ffd700 1 --2- 172.21.15.101:0/4044190658 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fda000a11e0 0x7fda000a3690 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fda0c011550 tx=0x7fda0c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:15.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:15.892+0000 7fd9fffff700 1 -- 172.21.15.101:0/4044190658 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fda10095850 con 0x7fda1810c530 2023-07-18T03:06:16.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.135+0000 7fda1d30b700 1 -- 172.21.15.101:0/4044190658 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fda181a4c50 con 0x7fda1810c530 2023-07-18T03:06:16.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.135+0000 7fd9fffff700 1 -- 172.21.15.101:0/4044190658 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fda1000ce60 con 0x7fda1810c530 2023-07-18T03:06:16.122 INFO:teuthology.orchestra.run.smithi101.stdout:219043332140 2023-07-18T03:06:16.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.138+0000 7fda1d30b700 1 -- 172.21.15.101:0/4044190658 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fda000a11e0 msgr2=0x7fda000a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:16.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.138+0000 7fda1d30b700 1 --2- 172.21.15.101:0/4044190658 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fda000a11e0 0x7fda000a3690 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fda0c011550 tx=0x7fda0c009260 comp rx=0 tx=0).stop 2023-07-18T03:06:16.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.138+0000 7fda1d30b700 1 -- 172.21.15.101:0/4044190658 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda1810c530 msgr2=0x7fda1819c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:16.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.138+0000 7fda1d30b700 1 --2- 172.21.15.101:0/4044190658 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda1810c530 0x7fda1819c810 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fda18100b60 tx=0x7fda1000ef90 comp rx=0 tx=0).stop 2023-07-18T03:06:16.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.138+0000 7fda1d30b700 1 -- 172.21.15.101:0/4044190658 shutdown_connections 2023-07-18T03:06:16.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.138+0000 7fda1d30b700 1 --2- 172.21.15.101:0/4044190658 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fda000a11e0 0x7fda000a3690 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:16.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.138+0000 7fda1d30b700 1 --2- 172.21.15.101:0/4044190658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda180fe980 0x7fda18197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:16.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.138+0000 7fda1d30b700 1 --2- 172.21.15.101:0/4044190658 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda1810c530 0x7fda1819c810 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:16.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.138+0000 7fda1d30b700 1 --2- 172.21.15.101:0/4044190658 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda180ff290 0x7fda181983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:16.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.138+0000 7fda1d30b700 1 -- 172.21.15.101:0/4044190658 >> 172.21.15.101:0/4044190658 conn(0x7fda18075b70 msgr2=0x7fda180fc3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:16.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.138+0000 7fda1d30b700 1 -- 172.21.15.101:0/4044190658 shutdown_connections 2023-07-18T03:06:16.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.138+0000 7fda1d30b700 1 -- 172.21.15.101:0/4044190658 wait complete. 2023-07-18T03:06:16.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:15 smithi179 ceph-mon[129229]: pgmap v315: 129 pgs: 129 active+clean; 640 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:06:16.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:15 smithi101 ceph-mon[126379]: pgmap v315: 129 pgs: 129 active+clean; 640 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:06:16.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:15 smithi155 ceph-mon[128885]: pgmap v315: 129 pgs: 129 active+clean; 640 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:06:16.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.914+0000 7f2033502700 1 -- 172.21.15.101:0/2081091219 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f202c06caa0 msgr2=0x7f202c06ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:16.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.914+0000 7f2033502700 1 --2- 172.21.15.101:0/2081091219 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f202c06caa0 0x7f202c06ce70 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2028005ce0 tx=0x7f2028009a60 comp rx=0 tx=0).stop 2023-07-18T03:06:16.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.914+0000 7f2033502700 1 -- 172.21.15.101:0/2081091219 shutdown_connections 2023-07-18T03:06:16.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.914+0000 7f2033502700 1 --2- 172.21.15.101:0/2081091219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f202c10b2e0 0x7f202c0787e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:16.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.914+0000 7f2033502700 1 --2- 172.21.15.101:0/2081091219 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f202c06caa0 0x7f202c06ce70 secure :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2028005ce0 tx=0x7f2028009a60 comp rx=0 tx=0).stop 2023-07-18T03:06:16.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.914+0000 7f2033502700 1 --2- 172.21.15.101:0/2081091219 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f202c06d440 0x7f202c10abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:16.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.915+0000 7f2033502700 1 -- 172.21.15.101:0/2081091219 >> 172.21.15.101:0/2081091219 conn(0x7f202c06bde0 msgr2=0x7f202c070dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:16.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.916+0000 7f2031cff700 1 --2- 172.21.15.101:0/2081091219 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f202c06d440 0x7f202c10abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:06:16.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.919+0000 7f2033502700 1 -- 172.21.15.101:0/2081091219 shutdown_connections 2023-07-18T03:06:16.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.919+0000 7f2033502700 1 -- 172.21.15.101:0/2081091219 wait complete. 2023-07-18T03:06:16.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.920+0000 7f2033502700 1 Processor -- start 2023-07-18T03:06:16.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.920+0000 7f2033502700 1 -- start start 2023-07-18T03:06:16.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.921+0000 7f2033502700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f202c10b2e0 0x7f202c082e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:16.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.921+0000 7f2033502700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f202c07ce60 0x7f202c07d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:16.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.921+0000 7f2033502700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f202c07d810 0x7f202c1b9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:16.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.921+0000 7f2033502700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202c083550 con 0x7f202c10b2e0 2023-07-18T03:06:16.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.921+0000 7f2033502700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202c1ba030 con 0x7f202c07d810 2023-07-18T03:06:16.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.921+0000 7f2033502700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202c1ba170 con 0x7f202c07ce60 2023-07-18T03:06:16.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.922+0000 7f2032500700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f202c10b2e0 0x7f202c082e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:16.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.922+0000 7f2032500700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f202c10b2e0 0x7f202c082e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:34760/0 (socket says 172.21.15.101:34760) 2023-07-18T03:06:16.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.922+0000 7f2032500700 1 -- 172.21.15.101:0/3684372458 learned_addr learned my addr 172.21.15.101:0/3684372458 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:16.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.922+0000 7f2032d01700 1 --2- 172.21.15.101:0/3684372458 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f202c07d810 0x7f202c1b9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:16.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.922+0000 7f2031cff700 1 --2- 172.21.15.101:0/3684372458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f202c07ce60 0x7f202c07d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:16.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.922+0000 7f2032500700 1 -- 172.21.15.101:0/3684372458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f202c07ce60 msgr2=0x7f202c07d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:16.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.922+0000 7f2032500700 1 --2- 172.21.15.101:0/3684372458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f202c07ce60 0x7f202c07d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:16.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.922+0000 7f2032500700 1 -- 172.21.15.101:0/3684372458 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f202c07d810 msgr2=0x7f202c1b9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:16.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.922+0000 7f2032500700 1 --2- 172.21.15.101:0/3684372458 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f202c07d810 0x7f202c1b9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:16.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.923+0000 7f2032500700 1 -- 172.21.15.101:0/3684372458 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20280095f0 con 0x7f202c10b2e0 2023-07-18T03:06:16.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.923+0000 7f2032500700 1 --2- 172.21.15.101:0/3684372458 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f202c10b2e0 0x7f202c082e70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2028009ba0 tx=0x7f2028005af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:16.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.924+0000 7f20237fe700 1 -- 172.21.15.101:0/3684372458 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2028005030 con 0x7f202c10b2e0 2023-07-18T03:06:16.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.924+0000 7f2033502700 1 -- 172.21.15.101:0/3684372458 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f202c1ba3c0 con 0x7f202c10b2e0 2023-07-18T03:06:16.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.924+0000 7f20237fe700 1 -- 172.21.15.101:0/3684372458 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2028005650 con 0x7f202c10b2e0 2023-07-18T03:06:16.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.924+0000 7f20237fe700 1 -- 172.21.15.101:0/3684372458 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2028003c50 con 0x7f202c10b2e0 2023-07-18T03:06:16.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.924+0000 7f2033502700 1 -- 172.21.15.101:0/3684372458 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f202c1ba780 con 0x7f202c10b2e0 2023-07-18T03:06:16.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.926+0000 7f2033502700 1 -- 172.21.15.101:0/3684372458 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f202c073ee0 con 0x7f202c10b2e0 2023-07-18T03:06:16.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.930+0000 7f20237fe700 1 -- 172.21.15.101:0/3684372458 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f2028004990 con 0x7f202c10b2e0 2023-07-18T03:06:16.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.932+0000 7f20237fe700 1 --2- 172.21.15.101:0/3684372458 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f20180a5680 0x7f20180a7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:16.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.934+0000 7f2031cff700 1 --2- 172.21.15.101:0/3684372458 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f20180a5680 0x7f20180a7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:16.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.935+0000 7f2031cff700 1 --2- 172.21.15.101:0/3684372458 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f20180a5680 0x7f20180a7b30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f201c005fd0 tx=0x7f201c005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:16.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.936+0000 7f20237fe700 1 -- 172.21.15.101:0/3684372458 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f2028012070 con 0x7f202c10b2e0 2023-07-18T03:06:16.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:16.941+0000 7f20237fe700 1 -- 172.21.15.101:0/3684372458 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2028096a30 con 0x7f202c10b2e0 2023-07-18T03:06:16.989 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495661 got 197568495665 for osd.6 2023-07-18T03:06:16.990 DEBUG:teuthology.parallel:result is None 2023-07-18T03:06:17.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.188+0000 7f2033502700 1 -- 172.21.15.101:0/3684372458 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f202c04ea60 con 0x7f202c10b2e0 2023-07-18T03:06:17.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.188+0000 7f20237fe700 1 -- 172.21.15.101:0/3684372458 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f202809a050 con 0x7f202c10b2e0 2023-07-18T03:06:17.175 INFO:teuthology.orchestra.run.smithi101.stdout:55834574922 2023-07-18T03:06:17.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.190+0000 7f2033502700 1 -- 172.21.15.101:0/3684372458 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f20180a5680 msgr2=0x7f20180a7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:17.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.190+0000 7f2033502700 1 --2- 172.21.15.101:0/3684372458 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f20180a5680 0x7f20180a7b30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f201c005fd0 tx=0x7f201c005da0 comp rx=0 tx=0).stop 2023-07-18T03:06:17.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.190+0000 7f2033502700 1 -- 172.21.15.101:0/3684372458 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f202c10b2e0 msgr2=0x7f202c082e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:17.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.190+0000 7f2033502700 1 --2- 172.21.15.101:0/3684372458 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f202c10b2e0 0x7f202c082e70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2028009ba0 tx=0x7f2028005af0 comp rx=0 tx=0).stop 2023-07-18T03:06:17.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.191+0000 7f2033502700 1 -- 172.21.15.101:0/3684372458 shutdown_connections 2023-07-18T03:06:17.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.191+0000 7f2033502700 1 --2- 172.21.15.101:0/3684372458 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f20180a5680 0x7f20180a7b30 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:17.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.191+0000 7f2033502700 1 --2- 172.21.15.101:0/3684372458 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f202c07d810 0x7f202c1b9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:17.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.191+0000 7f2033502700 1 --2- 172.21.15.101:0/3684372458 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f202c10b2e0 0x7f202c082e70 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:17.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.191+0000 7f2033502700 1 --2- 172.21.15.101:0/3684372458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f202c07ce60 0x7f202c07d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:17.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.191+0000 7f2033502700 1 -- 172.21.15.101:0/3684372458 >> 172.21.15.101:0/3684372458 conn(0x7f202c06bde0 msgr2=0x7f202c10a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:17.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.191+0000 7f2033502700 1 -- 172.21.15.101:0/3684372458 shutdown_connections 2023-07-18T03:06:17.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.191+0000 7f2033502700 1 -- 172.21.15.101:0/3684372458 wait complete. 2023-07-18T03:06:17.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:16 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/4044190658' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-07-18T03:06:17.199 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:16 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/4044190658' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-07-18T03:06:17.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:16 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/4044190658' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-07-18T03:06:17.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.237+0000 7f3b706f9700 1 -- 172.21.15.101:0/2595200446 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b68104ea0 msgr2=0x7f3b6810ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:17.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.237+0000 7f3b706f9700 1 --2- 172.21.15.101:0/2595200446 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b68104ea0 0x7f3b6810ca20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3b6000b0d0 tx=0x7f3b6000b3e0 comp rx=0 tx=0).stop 2023-07-18T03:06:17.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.237+0000 7f3b706f9700 1 -- 172.21.15.101:0/2595200446 shutdown_connections 2023-07-18T03:06:17.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.237+0000 7f3b706f9700 1 --2- 172.21.15.101:0/2595200446 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b68104ea0 0x7f3b6810ca20 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:17.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.237+0000 7f3b706f9700 1 --2- 172.21.15.101:0/2595200446 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b680ffe00 0x7f3b681001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:17.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.237+0000 7f3b706f9700 1 --2- 172.21.15.101:0/2595200446 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b681007a0 0x7f3b68104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:17.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.237+0000 7f3b706f9700 1 -- 172.21.15.101:0/2595200446 >> 172.21.15.101:0/2595200446 conn(0x7f3b680fb660 msgr2=0x7f3b680fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:17.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.237+0000 7f3b706f9700 1 -- 172.21.15.101:0/2595200446 shutdown_connections 2023-07-18T03:06:17.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.237+0000 7f3b706f9700 1 -- 172.21.15.101:0/2595200446 wait complete. 2023-07-18T03:06:17.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.238+0000 7f3b706f9700 1 Processor -- start 2023-07-18T03:06:17.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.239+0000 7f3b706f9700 1 -- start start 2023-07-18T03:06:17.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.239+0000 7f3b706f9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b680ffe00 0x7f3b68195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:17.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.239+0000 7f3b706f9700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b681007a0 0x7f3b68196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:17.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.239+0000 7f3b706f9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b68104ea0 0x7f3b6819a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:17.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.239+0000 7f3b706f9700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6819ad10 con 0x7f3b681007a0 2023-07-18T03:06:17.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.239+0000 7f3b706f9700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6819ae80 con 0x7f3b68104ea0 2023-07-18T03:06:17.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.239+0000 7f3b706f9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6819b040 con 0x7f3b680ffe00 2023-07-18T03:06:17.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.239+0000 7f3b6dc94700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b681007a0 0x7f3b68196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:17.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.239+0000 7f3b6dc94700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b681007a0 0x7f3b68196290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:34770/0 (socket says 172.21.15.101:34770) 2023-07-18T03:06:17.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.239+0000 7f3b6dc94700 1 -- 172.21.15.101:0/3970408190 learned_addr learned my addr 172.21.15.101:0/3970408190 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:17.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.239+0000 7f3b6e495700 1 --2- 172.21.15.101:0/3970408190 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b680ffe00 0x7f3b68195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:17.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.240+0000 7f3b6ec96700 1 --2- 172.21.15.101:0/3970408190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b68104ea0 0x7f3b6819a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:17.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.240+0000 7f3b6dc94700 1 -- 172.21.15.101:0/3970408190 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b680ffe00 msgr2=0x7f3b68195d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:17.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.240+0000 7f3b6dc94700 1 --2- 172.21.15.101:0/3970408190 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b680ffe00 0x7f3b68195d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:17.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.240+0000 7f3b6dc94700 1 -- 172.21.15.101:0/3970408190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b68104ea0 msgr2=0x7f3b6819a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:17.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.240+0000 7f3b6dc94700 1 --2- 172.21.15.101:0/3970408190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b68104ea0 0x7f3b6819a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:17.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.240+0000 7f3b6dc94700 1 -- 172.21.15.101:0/3970408190 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b60009d00 con 0x7f3b681007a0 2023-07-18T03:06:17.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.240+0000 7f3b6dc94700 1 --2- 172.21.15.101:0/3970408190 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b681007a0 0x7f3b68196290 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3b5800e970 tx=0x7f3b5800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:17.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.240+0000 7f3b5f7fe700 1 -- 172.21.15.101:0/3970408190 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b58009790 con 0x7f3b681007a0 2023-07-18T03:06:17.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.240+0000 7f3b5f7fe700 1 -- 172.21.15.101:0/3970408190 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b5800c7e0 con 0x7f3b681007a0 2023-07-18T03:06:17.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.240+0000 7f3b5f7fe700 1 -- 172.21.15.101:0/3970408190 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b580054e0 con 0x7f3b681007a0 2023-07-18T03:06:17.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.240+0000 7f3b706f9700 1 -- 172.21.15.101:0/3970408190 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b6819b320 con 0x7f3b681007a0 2023-07-18T03:06:17.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.241+0000 7f3b706f9700 1 -- 172.21.15.101:0/3970408190 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b681a2b80 con 0x7f3b681007a0 2023-07-18T03:06:17.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.242+0000 7f3b706f9700 1 -- 172.21.15.101:0/3970408190 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b6804e210 con 0x7f3b681007a0 2023-07-18T03:06:17.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.244+0000 7f3b5f7fe700 1 -- 172.21.15.101:0/3970408190 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f3b5800c950 con 0x7f3b681007a0 2023-07-18T03:06:17.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.245+0000 7f3b5f7fe700 1 --2- 172.21.15.101:0/3970408190 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f3b540a1260 0x7f3b540a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:17.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.245+0000 7f3b5f7fe700 1 -- 172.21.15.101:0/3970408190 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f3b58014070 con 0x7f3b681007a0 2023-07-18T03:06:17.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.245+0000 7f3b6e495700 1 --2- 172.21.15.101:0/3970408190 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f3b540a1260 0x7f3b540a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:17.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.246+0000 7f3b6e495700 1 --2- 172.21.15.101:0/3970408190 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f3b540a1260 0x7f3b540a3710 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3b64009c60 tx=0x7f3b64009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:17.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.248+0000 7f3b5f7fe700 1 -- 172.21.15.101:0/3970408190 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3b580961d0 con 0x7f3b681007a0 2023-07-18T03:06:17.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.474+0000 7f3b706f9700 1 -- 172.21.15.101:0/3970408190 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f3b681970b0 con 0x7f3b681007a0 2023-07-18T03:06:17.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.474+0000 7f3b5f7fe700 1 -- 172.21.15.101:0/3970408190 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3b580997f0 con 0x7f3b681007a0 2023-07-18T03:06:17.461 INFO:teuthology.orchestra.run.smithi101.stdout:163208757303 2023-07-18T03:06:17.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.477+0000 7f3b706f9700 1 -- 172.21.15.101:0/3970408190 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f3b540a1260 msgr2=0x7f3b540a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:17.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.477+0000 7f3b706f9700 1 --2- 172.21.15.101:0/3970408190 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f3b540a1260 0x7f3b540a3710 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3b64009c60 tx=0x7f3b64009260 comp rx=0 tx=0).stop 2023-07-18T03:06:17.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.477+0000 7f3b706f9700 1 -- 172.21.15.101:0/3970408190 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b681007a0 msgr2=0x7f3b68196290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:17.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.477+0000 7f3b706f9700 1 --2- 172.21.15.101:0/3970408190 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b681007a0 0x7f3b68196290 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3b5800e970 tx=0x7f3b5800ec80 comp rx=0 tx=0).stop 2023-07-18T03:06:17.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.477+0000 7f3b706f9700 1 -- 172.21.15.101:0/3970408190 shutdown_connections 2023-07-18T03:06:17.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.477+0000 7f3b706f9700 1 --2- 172.21.15.101:0/3970408190 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f3b540a1260 0x7f3b540a3710 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:17.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.477+0000 7f3b706f9700 1 --2- 172.21.15.101:0/3970408190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b68104ea0 0x7f3b6819a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:17.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.477+0000 7f3b706f9700 1 --2- 172.21.15.101:0/3970408190 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b680ffe00 0x7f3b68195d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:17.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.477+0000 7f3b706f9700 1 --2- 172.21.15.101:0/3970408190 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b681007a0 0x7f3b68196290 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:17.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.477+0000 7f3b706f9700 1 -- 172.21.15.101:0/3970408190 >> 172.21.15.101:0/3970408190 conn(0x7f3b680fb660 msgr2=0x7f3b680fc070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:17.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.477+0000 7f3b706f9700 1 -- 172.21.15.101:0/3970408190 shutdown_connections 2023-07-18T03:06:17.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:17.477+0000 7f3b706f9700 1 -- 172.21.15.101:0/3970408190 wait complete. 2023-07-18T03:06:18.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:17 smithi179 ceph-mon[129229]: pgmap v316: 129 pgs: 129 active+clean; 640 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2023-07-18T03:06:18.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:17 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/3684372458' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-07-18T03:06:18.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:17 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/3970408190' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-07-18T03:06:18.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:17 smithi101 ceph-mon[126379]: pgmap v316: 129 pgs: 129 active+clean; 640 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2023-07-18T03:06:18.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:17 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3684372458' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-07-18T03:06:18.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:17 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3970408190' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-07-18T03:06:18.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:17 smithi155 ceph-mon[128885]: pgmap v316: 129 pgs: 129 active+clean; 640 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2023-07-18T03:06:18.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:17 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/3684372458' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-07-18T03:06:18.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:17 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/3970408190' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-07-18T03:06:18.440 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.453+0000 7f6e8431e700 1 -- 172.21.15.101:0/2080221693 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6e7c0ffe00 msgr2=0x7f6e7c1001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:18.440 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.453+0000 7f6e8431e700 1 --2- 172.21.15.101:0/2080221693 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6e7c0ffe00 0x7f6e7c1001d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6e780099c0 tx=0x7f6e78009cd0 comp rx=0 tx=0).stop 2023-07-18T03:06:18.440 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.453+0000 7f6e8431e700 1 -- 172.21.15.101:0/2080221693 shutdown_connections 2023-07-18T03:06:18.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.453+0000 7f6e8431e700 1 --2- 172.21.15.101:0/2080221693 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e7c104ea0 0x7f6e7c10ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:18.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.453+0000 7f6e8431e700 1 --2- 172.21.15.101:0/2080221693 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6e7c0ffe00 0x7f6e7c1001d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:18.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.453+0000 7f6e8431e700 1 --2- 172.21.15.101:0/2080221693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e7c1007a0 0x7f6e7c104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:18.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.453+0000 7f6e8431e700 1 -- 172.21.15.101:0/2080221693 >> 172.21.15.101:0/2080221693 conn(0x7f6e7c0fb660 msgr2=0x7f6e7c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:18.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.454+0000 7f6e8431e700 1 -- 172.21.15.101:0/2080221693 shutdown_connections 2023-07-18T03:06:18.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.454+0000 7f6e8431e700 1 -- 172.21.15.101:0/2080221693 wait complete. 2023-07-18T03:06:18.442 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.455+0000 7f6e8431e700 1 Processor -- start 2023-07-18T03:06:18.442 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.455+0000 7f6e8431e700 1 -- start start 2023-07-18T03:06:18.442 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.456+0000 7f6e8431e700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6e7c0ffe00 0x7f6e7c197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:18.442 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.456+0000 7f6e8431e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e7c1007a0 0x7f6e7c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:18.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.456+0000 7f6e8431e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e7c104ea0 0x7f6e7c19c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:18.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.456+0000 7f6e8431e700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c19ceb0 con 0x7f6e7c0ffe00 2023-07-18T03:06:18.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.456+0000 7f6e8431e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c19d020 con 0x7f6e7c104ea0 2023-07-18T03:06:18.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.456+0000 7f6e8431e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c19d1e0 con 0x7f6e7c1007a0 2023-07-18T03:06:18.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.456+0000 7f6e820ba700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6e7c0ffe00 0x7f6e7c197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:18.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.456+0000 7f6e820ba700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6e7c0ffe00 0x7f6e7c197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:34786/0 (socket says 172.21.15.101:34786) 2023-07-18T03:06:18.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.456+0000 7f6e820ba700 1 -- 172.21.15.101:0/2287396190 learned_addr learned my addr 172.21.15.101:0/2287396190 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:18.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.456+0000 7f6e828bb700 1 --2- 172.21.15.101:0/2287396190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e7c104ea0 0x7f6e7c19c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:18.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.456+0000 7f6e818b9700 1 --2- 172.21.15.101:0/2287396190 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e7c1007a0 0x7f6e7c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:18.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.457+0000 7f6e820ba700 1 -- 172.21.15.101:0/2287396190 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e7c1007a0 msgr2=0x7f6e7c198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:18.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.457+0000 7f6e820ba700 1 --2- 172.21.15.101:0/2287396190 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e7c1007a0 0x7f6e7c198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:18.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.457+0000 7f6e820ba700 1 -- 172.21.15.101:0/2287396190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e7c104ea0 msgr2=0x7f6e7c19c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:18.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.457+0000 7f6e820ba700 1 --2- 172.21.15.101:0/2287396190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e7c104ea0 0x7f6e7c19c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:18.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.457+0000 7f6e820ba700 1 -- 172.21.15.101:0/2287396190 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e780095f0 con 0x7f6e7c0ffe00 2023-07-18T03:06:18.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.457+0000 7f6e828bb700 1 --2- 172.21.15.101:0/2287396190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e7c104ea0 0x7f6e7c19c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:06:18.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.457+0000 7f6e820ba700 1 --2- 172.21.15.101:0/2287396190 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6e7c0ffe00 0x7f6e7c197ef0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6e78003fd0 tx=0x7f6e78005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:18.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.457+0000 7f6e737fe700 1 -- 172.21.15.101:0/2287396190 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7801d070 con 0x7f6e7c0ffe00 2023-07-18T03:06:18.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.457+0000 7f6e8431e700 1 -- 172.21.15.101:0/2287396190 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e7c19d460 con 0x7f6e7c0ffe00 2023-07-18T03:06:18.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.457+0000 7f6e737fe700 1 -- 172.21.15.101:0/2287396190 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e780056f0 con 0x7f6e7c0ffe00 2023-07-18T03:06:18.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.457+0000 7f6e737fe700 1 -- 172.21.15.101:0/2287396190 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7800fae0 con 0x7f6e7c0ffe00 2023-07-18T03:06:18.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.457+0000 7f6e8431e700 1 -- 172.21.15.101:0/2287396190 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e7c1a4d20 con 0x7f6e7c0ffe00 2023-07-18T03:06:18.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.459+0000 7f6e8431e700 1 -- 172.21.15.101:0/2287396190 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e7c04e210 con 0x7f6e7c0ffe00 2023-07-18T03:06:18.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.461+0000 7f6e737fe700 1 -- 172.21.15.101:0/2287396190 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f6e7800b730 con 0x7f6e7c0ffe00 2023-07-18T03:06:18.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.462+0000 7f6e737fe700 1 --2- 172.21.15.101:0/2287396190 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f6e680a1260 0x7f6e680a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:18.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.462+0000 7f6e737fe700 1 -- 172.21.15.101:0/2287396190 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f6e780ce200 con 0x7f6e7c0ffe00 2023-07-18T03:06:18.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.462+0000 7f6e818b9700 1 --2- 172.21.15.101:0/2287396190 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f6e680a1260 0x7f6e680a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:18.450 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.463+0000 7f6e818b9700 1 --2- 172.21.15.101:0/2287396190 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f6e680a1260 0x7f6e680a3710 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c199510 tx=0x7f6e6c009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:18.450 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.464+0000 7f6e737fe700 1 -- 172.21.15.101:0/2287396190 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6e78097520 con 0x7f6e7c0ffe00 2023-07-18T03:06:18.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.686+0000 7f6e8431e700 1 -- 172.21.15.101:0/2287396190 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f6e7c1a4eb0 con 0x7f6e7c0ffe00 2023-07-18T03:06:18.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.687+0000 7f6e737fe700 1 -- 172.21.15.101:0/2287396190 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6e7809ab40 con 0x7f6e7c0ffe00 2023-07-18T03:06:18.673 INFO:teuthology.orchestra.run.smithi101.stdout:283467841572 2023-07-18T03:06:18.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.689+0000 7f6e8431e700 1 -- 172.21.15.101:0/2287396190 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f6e680a1260 msgr2=0x7f6e680a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:18.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.689+0000 7f6e8431e700 1 --2- 172.21.15.101:0/2287396190 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f6e680a1260 0x7f6e680a3710 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c199510 tx=0x7f6e6c009190 comp rx=0 tx=0).stop 2023-07-18T03:06:18.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.689+0000 7f6e8431e700 1 -- 172.21.15.101:0/2287396190 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6e7c0ffe00 msgr2=0x7f6e7c197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:18.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.689+0000 7f6e8431e700 1 --2- 172.21.15.101:0/2287396190 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6e7c0ffe00 0x7f6e7c197ef0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6e78003fd0 tx=0x7f6e78005280 comp rx=0 tx=0).stop 2023-07-18T03:06:18.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.689+0000 7f6e8431e700 1 -- 172.21.15.101:0/2287396190 shutdown_connections 2023-07-18T03:06:18.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.689+0000 7f6e8431e700 1 --2- 172.21.15.101:0/2287396190 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f6e680a1260 0x7f6e680a3710 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:18.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.689+0000 7f6e8431e700 1 --2- 172.21.15.101:0/2287396190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e7c104ea0 0x7f6e7c19c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:18.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.689+0000 7f6e8431e700 1 --2- 172.21.15.101:0/2287396190 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6e7c0ffe00 0x7f6e7c197ef0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:18.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.689+0000 7f6e8431e700 1 --2- 172.21.15.101:0/2287396190 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e7c1007a0 0x7f6e7c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:18.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.689+0000 7f6e8431e700 1 -- 172.21.15.101:0/2287396190 >> 172.21.15.101:0/2287396190 conn(0x7f6e7c0fb660 msgr2=0x7f6e7c0fc490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:18.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.689+0000 7f6e8431e700 1 -- 172.21.15.101:0/2287396190 shutdown_connections 2023-07-18T03:06:18.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.690+0000 7f6e8431e700 1 -- 172.21.15.101:0/2287396190 wait complete. 2023-07-18T03:06:18.848 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332137 got 219043332140 for osd.7 2023-07-18T03:06:18.848 DEBUG:teuthology.parallel:result is None 2023-07-18T03:06:18.886 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574919 got 55834574922 for osd.1 2023-07-18T03:06:18.887 DEBUG:teuthology.parallel:result is None 2023-07-18T03:06:18.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.576+0000 7f21f8615700 1 -- 172.21.15.101:0/1368628545 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f21f00ffe00 msgr2=0x7f21f01001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:18.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.576+0000 7f21f8615700 1 --2- 172.21.15.101:0/1368628545 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f21f00ffe00 0x7f21f01001d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f21ec0099c0 tx=0x7f21ec009cd0 comp rx=0 tx=0).stop 2023-07-18T03:06:18.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.576+0000 7f21f8615700 1 -- 172.21.15.101:0/1368628545 shutdown_connections 2023-07-18T03:06:18.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.576+0000 7f21f8615700 1 --2- 172.21.15.101:0/1368628545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f21f0104ea0 0x7f21f010ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:18.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.576+0000 7f21f8615700 1 --2- 172.21.15.101:0/1368628545 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f21f00ffe00 0x7f21f01001d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:18.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.576+0000 7f21f8615700 1 --2- 172.21.15.101:0/1368628545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21f01007a0 0x7f21f0104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:18.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.576+0000 7f21f8615700 1 -- 172.21.15.101:0/1368628545 >> 172.21.15.101:0/1368628545 conn(0x7f21f00fb660 msgr2=0x7f21f00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:18.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.576+0000 7f21f8615700 1 -- 172.21.15.101:0/1368628545 shutdown_connections 2023-07-18T03:06:18.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.576+0000 7f21f8615700 1 -- 172.21.15.101:0/1368628545 wait complete. 2023-07-18T03:06:18.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.577+0000 7f21f8615700 1 Processor -- start 2023-07-18T03:06:18.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.578+0000 7f21f8615700 1 -- start start 2023-07-18T03:06:18.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.578+0000 7f21f8615700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f21f00ffe00 0x7f21f0197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:18.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.578+0000 7f21f8615700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21f01007a0 0x7f21f0198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:18.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.578+0000 7f21f8615700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f21f0104ea0 0x7f21f019c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:18.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.578+0000 7f21f8615700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f019ce60 con 0x7f21f0104ea0 2023-07-18T03:06:18.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.578+0000 7f21f8615700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f019cfd0 con 0x7f21f01007a0 2023-07-18T03:06:18.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.578+0000 7f21f8615700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f019d190 con 0x7f21f00ffe00 2023-07-18T03:06:18.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.578+0000 7f21f5bb0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21f01007a0 0x7f21f0198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:18.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.578+0000 7f21f5bb0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21f01007a0 0x7f21f0198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60016/0 (socket says 172.21.15.101:60016) 2023-07-18T03:06:18.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.578+0000 7f21f63b1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f21f00ffe00 0x7f21f0197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:18.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.578+0000 7f21f5bb0700 1 -- 172.21.15.101:0/2930853508 learned_addr learned my addr 172.21.15.101:0/2930853508 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:18.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.578+0000 7f21f6bb2700 1 --2- 172.21.15.101:0/2930853508 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f21f0104ea0 0x7f21f019c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:18.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.579+0000 7f21f5bb0700 1 -- 172.21.15.101:0/2930853508 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f21f00ffe00 msgr2=0x7f21f0197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:18.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.579+0000 7f21f5bb0700 1 --2- 172.21.15.101:0/2930853508 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f21f00ffe00 0x7f21f0197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:18.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.579+0000 7f21f5bb0700 1 -- 172.21.15.101:0/2930853508 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f21f0104ea0 msgr2=0x7f21f019c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:18.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.579+0000 7f21f5bb0700 1 --2- 172.21.15.101:0/2930853508 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f21f0104ea0 0x7f21f019c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:18.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.579+0000 7f21f5bb0700 1 -- 172.21.15.101:0/2930853508 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21ec0095f0 con 0x7f21f01007a0 2023-07-18T03:06:18.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.579+0000 7f21f5bb0700 1 --2- 172.21.15.101:0/2930853508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21f01007a0 0x7f21f0198480 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f21e000e970 tx=0x7f21e000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:18.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.579+0000 7f21e77fe700 1 -- 172.21.15.101:0/2930853508 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e0009790 con 0x7f21f01007a0 2023-07-18T03:06:18.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.579+0000 7f21f8615700 1 -- 172.21.15.101:0/2930853508 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21f019d470 con 0x7f21f01007a0 2023-07-18T03:06:18.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.579+0000 7f21e77fe700 1 -- 172.21.15.101:0/2930853508 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f21e000c7e0 con 0x7f21f01007a0 2023-07-18T03:06:18.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.579+0000 7f21e77fe700 1 -- 172.21.15.101:0/2930853508 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e00054e0 con 0x7f21f01007a0 2023-07-18T03:06:18.962 INFO:teuthology.orchestra.run.smithi101.stdout:249108103208 2023-07-18T03:06:18.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.579+0000 7f21f8615700 1 -- 172.21.15.101:0/2930853508 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21f01a4cd0 con 0x7f21f01007a0 2023-07-18T03:06:18.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.581+0000 7f21f8615700 1 -- 172.21.15.101:0/2930853508 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21f004e210 con 0x7f21f01007a0 2023-07-18T03:06:18.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.583+0000 7f21e77fe700 1 -- 172.21.15.101:0/2930853508 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f21e000c950 con 0x7f21f01007a0 2023-07-18T03:06:18.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.584+0000 7f21e77fe700 1 --2- 172.21.15.101:0/2930853508 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f21dc0a11e0 0x7f21dc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:18.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.584+0000 7f21e77fe700 1 -- 172.21.15.101:0/2930853508 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f21e0014070 con 0x7f21f01007a0 2023-07-18T03:06:18.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.584+0000 7f21f63b1700 1 --2- 172.21.15.101:0/2930853508 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f21dc0a11e0 0x7f21dc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:18.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.585+0000 7f21f63b1700 1 --2- 172.21.15.101:0/2930853508 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f21dc0a11e0 0x7f21dc0a3690 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f21ec009990 tx=0x7f21ec005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:18.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.586+0000 7f21e77fe700 1 -- 172.21.15.101:0/2930853508 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f21e0096990 con 0x7f21f01007a0 2023-07-18T03:06:18.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.808+0000 7f21f8615700 1 -- 172.21.15.101:0/2930853508 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f21f01992a0 con 0x7f21f01007a0 2023-07-18T03:06:18.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.809+0000 7f21e77fe700 1 -- 172.21.15.101:0/2930853508 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f21e0099fb0 con 0x7f21f01007a0 2023-07-18T03:06:18.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.812+0000 7f21f8615700 1 -- 172.21.15.101:0/2930853508 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f21dc0a11e0 msgr2=0x7f21dc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:18.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.812+0000 7f21f8615700 1 --2- 172.21.15.101:0/2930853508 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f21dc0a11e0 0x7f21dc0a3690 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f21ec009990 tx=0x7f21ec005fb0 comp rx=0 tx=0).stop 2023-07-18T03:06:18.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.812+0000 7f21f8615700 1 -- 172.21.15.101:0/2930853508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21f01007a0 msgr2=0x7f21f0198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:18.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.812+0000 7f21f8615700 1 --2- 172.21.15.101:0/2930853508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21f01007a0 0x7f21f0198480 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f21e000e970 tx=0x7f21e000ec80 comp rx=0 tx=0).stop 2023-07-18T03:06:18.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.812+0000 7f21f8615700 1 -- 172.21.15.101:0/2930853508 shutdown_connections 2023-07-18T03:06:18.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.812+0000 7f21f8615700 1 --2- 172.21.15.101:0/2930853508 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f21dc0a11e0 0x7f21dc0a3690 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:18.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.812+0000 7f21f8615700 1 --2- 172.21.15.101:0/2930853508 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f21f0104ea0 0x7f21f019c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:18.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.812+0000 7f21f8615700 1 --2- 172.21.15.101:0/2930853508 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f21f00ffe00 0x7f21f0197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:18.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.812+0000 7f21f8615700 1 --2- 172.21.15.101:0/2930853508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21f01007a0 0x7f21f0198480 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:18.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.812+0000 7f21f8615700 1 -- 172.21.15.101:0/2930853508 >> 172.21.15.101:0/2930853508 conn(0x7f21f00fb660 msgr2=0x7f21f00fc440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:18.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.812+0000 7f21f8615700 1 -- 172.21.15.101:0/2930853508 shutdown_connections 2023-07-18T03:06:18.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:18.813+0000 7f21f8615700 1 -- 172.21.15.101:0/2930853508 wait complete. 2023-07-18T03:06:18.966 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:18 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2287396190' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-07-18T03:06:18.967 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:18 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2930853508' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-07-18T03:06:19.192 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:18 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/2287396190' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-07-18T03:06:19.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:18 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/2930853508' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-07-18T03:06:19.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:18 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/2287396190' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-07-18T03:06:19.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:18 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/2930853508' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-07-18T03:06:19.463 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757300 got 163208757303 for osd.5 2023-07-18T03:06:19.463 DEBUG:teuthology.parallel:result is None 2023-07-18T03:06:19.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.543+0000 7f6755012700 1 -- 172.21.15.101:0/1717376355 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f67501007a0 msgr2=0x7f6750104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:19.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.543+0000 7f6755012700 1 --2- 172.21.15.101:0/1717376355 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f67501007a0 0x7f6750104770 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f67440099c0 tx=0x7f6744009cd0 comp rx=0 tx=0).stop 2023-07-18T03:06:19.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.543+0000 7f6755012700 1 -- 172.21.15.101:0/1717376355 shutdown_connections 2023-07-18T03:06:19.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.543+0000 7f6755012700 1 --2- 172.21.15.101:0/1717376355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67500ffe00 0x7f67501001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:19.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.543+0000 7f6755012700 1 --2- 172.21.15.101:0/1717376355 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6750104ea0 0x7f675010ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:19.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.543+0000 7f6755012700 1 --2- 172.21.15.101:0/1717376355 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f67501007a0 0x7f6750104770 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:19.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.543+0000 7f6755012700 1 -- 172.21.15.101:0/1717376355 >> 172.21.15.101:0/1717376355 conn(0x7f67500fb660 msgr2=0x7f67500fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:19.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.543+0000 7f6755012700 1 -- 172.21.15.101:0/1717376355 shutdown_connections 2023-07-18T03:06:19.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.543+0000 7f6755012700 1 -- 172.21.15.101:0/1717376355 wait complete. 2023-07-18T03:06:19.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.544+0000 7f6755012700 1 Processor -- start 2023-07-18T03:06:19.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.545+0000 7f6755012700 1 -- start start 2023-07-18T03:06:19.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.545+0000 7f6755012700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67500ffe00 0x7f6750195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:19.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.545+0000 7f6755012700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f67501007a0 0x7f6750196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:19.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.545+0000 7f6755012700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6750104ea0 0x7f675019a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:19.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.545+0000 7f6755012700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675019acf0 con 0x7f67501007a0 2023-07-18T03:06:19.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.545+0000 7f6755012700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675019ae60 con 0x7f67500ffe00 2023-07-18T03:06:19.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.545+0000 7f6755012700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675019b020 con 0x7f6750104ea0 2023-07-18T03:06:19.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.545+0000 7f674ed9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67500ffe00 0x7f6750195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:19.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.545+0000 7f674ed9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67500ffe00 0x7f6750195d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60046/0 (socket says 172.21.15.101:60046) 2023-07-18T03:06:19.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.545+0000 7f674ed9d700 1 -- 172.21.15.101:0/1118753398 learned_addr learned my addr 172.21.15.101:0/1118753398 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:19.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.545+0000 7f674f59e700 1 --2- 172.21.15.101:0/1118753398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6750104ea0 0x7f675019a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:19.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.546+0000 7f674ed9d700 1 -- 172.21.15.101:0/1118753398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6750104ea0 msgr2=0x7f675019a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:19.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.546+0000 7f674ed9d700 1 --2- 172.21.15.101:0/1118753398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6750104ea0 0x7f675019a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:19.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.546+0000 7f674ed9d700 1 -- 172.21.15.101:0/1118753398 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f67501007a0 msgr2=0x7f6750196270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:19.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.546+0000 7f674ed9d700 1 --2- 172.21.15.101:0/1118753398 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f67501007a0 0x7f6750196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:19.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.546+0000 7f674ed9d700 1 -- 172.21.15.101:0/1118753398 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6740009520 con 0x7f67500ffe00 2023-07-18T03:06:19.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.546+0000 7f674ed9d700 1 --2- 172.21.15.101:0/1118753398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67500ffe00 0x7f6750195d30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f674000eb30 tx=0x7f674000ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:19.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.546+0000 7f6737fff700 1 -- 172.21.15.101:0/1118753398 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6740009880 con 0x7f67500ffe00 2023-07-18T03:06:19.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.547+0000 7f6755012700 1 -- 172.21.15.101:0/1118753398 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67440095f0 con 0x7f67500ffe00 2023-07-18T03:06:19.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.547+0000 7f6737fff700 1 -- 172.21.15.101:0/1118753398 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67400044e0 con 0x7f67500ffe00 2023-07-18T03:06:19.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.547+0000 7f6737fff700 1 -- 172.21.15.101:0/1118753398 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6740003e90 con 0x7f67500ffe00 2023-07-18T03:06:19.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.547+0000 7f6755012700 1 -- 172.21.15.101:0/1118753398 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67501a2990 con 0x7f67500ffe00 2023-07-18T03:06:19.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.548+0000 7f6755012700 1 -- 172.21.15.101:0/1118753398 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f675004e210 con 0x7f67500ffe00 2023-07-18T03:06:19.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.550+0000 7f6737fff700 1 -- 172.21.15.101:0/1118753398 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f6740004000 con 0x7f67500ffe00 2023-07-18T03:06:19.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.551+0000 7f6737fff700 1 --2- 172.21.15.101:0/1118753398 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f67380a11e0 0x7f67380a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:19.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.552+0000 7f6737fff700 1 -- 172.21.15.101:0/1118753398 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f6740014070 con 0x7f67500ffe00 2023-07-18T03:06:19.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.552+0000 7f674e59c700 1 --2- 172.21.15.101:0/1118753398 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f67380a11e0 0x7f67380a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:19.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.552+0000 7f674e59c700 1 --2- 172.21.15.101:0/1118753398 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f67380a11e0 0x7f67380a3690 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f67440058e0 tx=0x7f6744003680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:19.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.553+0000 7f6737fff700 1 -- 172.21.15.101:0/1118753398 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6740096680 con 0x7f67500ffe00 2023-07-18T03:06:19.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.763+0000 7f6755012700 1 -- 172.21.15.101:0/1118753398 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f67501a2c10 con 0x7f67500ffe00 2023-07-18T03:06:19.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.764+0000 7f6737fff700 1 -- 172.21.15.101:0/1118753398 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f6740099ca0 con 0x7f67500ffe00 2023-07-18T03:06:19.751 INFO:teuthology.orchestra.run.smithi101.stdout:304942678048 2023-07-18T03:06:19.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.767+0000 7f6755012700 1 -- 172.21.15.101:0/1118753398 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f67380a11e0 msgr2=0x7f67380a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:19.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.767+0000 7f6755012700 1 --2- 172.21.15.101:0/1118753398 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f67380a11e0 0x7f67380a3690 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f67440058e0 tx=0x7f6744003680 comp rx=0 tx=0).stop 2023-07-18T03:06:19.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.767+0000 7f6755012700 1 -- 172.21.15.101:0/1118753398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67500ffe00 msgr2=0x7f6750195d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:19.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.767+0000 7f6755012700 1 --2- 172.21.15.101:0/1118753398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67500ffe00 0x7f6750195d30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f674000eb30 tx=0x7f674000ee40 comp rx=0 tx=0).stop 2023-07-18T03:06:19.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.767+0000 7f6755012700 1 -- 172.21.15.101:0/1118753398 shutdown_connections 2023-07-18T03:06:19.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.767+0000 7f6755012700 1 --2- 172.21.15.101:0/1118753398 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f67380a11e0 0x7f67380a3690 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:19.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.767+0000 7f6755012700 1 --2- 172.21.15.101:0/1118753398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67500ffe00 0x7f6750195d30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:19.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.767+0000 7f6755012700 1 --2- 172.21.15.101:0/1118753398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6750104ea0 0x7f675019a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:19.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.767+0000 7f6755012700 1 --2- 172.21.15.101:0/1118753398 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f67501007a0 0x7f6750196270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:19.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.767+0000 7f6755012700 1 -- 172.21.15.101:0/1118753398 >> 172.21.15.101:0/1118753398 conn(0x7f67500fb660 msgr2=0x7f67500fc170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:19.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.767+0000 7f6755012700 1 -- 172.21.15.101:0/1118753398 shutdown_connections 2023-07-18T03:06:19.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.767+0000 7f6755012700 1 -- 172.21.15.101:0/1118753398 wait complete. 2023-07-18T03:06:20.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:19 smithi179 ceph-mon[129229]: pgmap v317: 129 pgs: 129 active+clean; 640 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-07-18T03:06:20.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:19 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/319187197' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-07-18T03:06:20.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:19 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1118753398' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-07-18T03:06:20.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:19 smithi101 ceph-mon[126379]: pgmap v317: 129 pgs: 129 active+clean; 640 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-07-18T03:06:20.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:19 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/319187197' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-07-18T03:06:20.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:19 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1118753398' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-07-18T03:06:20.287 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:19 smithi155 ceph-mon[128885]: pgmap v317: 129 pgs: 129 active+clean; 640 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-07-18T03:06:20.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:19 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/319187197' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-07-18T03:06:20.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:19 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1118753398' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-07-18T03:06:20.473 INFO:tasks.cephadm.ceph_manager.ceph:need seq 249108103205 got 249108103208 for osd.8 2023-07-18T03:06:20.474 DEBUG:teuthology.parallel:result is None 2023-07-18T03:06:20.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.178+0000 7f7cf58f5700 1 -- 172.21.15.101:0/3357806130 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7cf01029b0 msgr2=0x7f7cf0106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:20.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.178+0000 7f7cf58f5700 1 --2- 172.21.15.101:0/3357806130 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7cf01029b0 0x7f7cf0106980 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7ce40099c0 tx=0x7f7ce4009cd0 comp rx=0 tx=0).stop 2023-07-18T03:06:20.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.178+0000 7f7cf58f5700 1 -- 172.21.15.101:0/3357806130 shutdown_connections 2023-07-18T03:06:20.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.178+0000 7f7cf58f5700 1 --2- 172.21.15.101:0/3357806130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7cf01070b0 0x7f7cf010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:20.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.178+0000 7f7cf58f5700 1 --2- 172.21.15.101:0/3357806130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7cf0102010 0x7f7cf01023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:20.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.178+0000 7f7cf58f5700 1 --2- 172.21.15.101:0/3357806130 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7cf01029b0 0x7f7cf0106980 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:20.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.178+0000 7f7cf58f5700 1 -- 172.21.15.101:0/3357806130 >> 172.21.15.101:0/3357806130 conn(0x7f7cf00fd8b0 msgr2=0x7f7cf00ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:20.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.178+0000 7f7cf58f5700 1 -- 172.21.15.101:0/3357806130 shutdown_connections 2023-07-18T03:06:20.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.178+0000 7f7cf58f5700 1 -- 172.21.15.101:0/3357806130 wait complete. 2023-07-18T03:06:20.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.179+0000 7f7cf58f5700 1 Processor -- start 2023-07-18T03:06:20.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.179+0000 7f7cf58f5700 1 -- start start 2023-07-18T03:06:20.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.180+0000 7f7cf58f5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7cf0102010 0x7f7cf0197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:20.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.180+0000 7f7cf58f5700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7cf01029b0 0x7f7cf0198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:20.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.180+0000 7f7cf58f5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7cf01070b0 0x7f7cf019c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:20.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.180+0000 7f7cf58f5700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf019ce10 con 0x7f7cf01029b0 2023-07-18T03:06:20.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.180+0000 7f7cf58f5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf019cf80 con 0x7f7cf0102010 2023-07-18T03:06:20.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.180+0000 7f7cf58f5700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf019d140 con 0x7f7cf01070b0 2023-07-18T03:06:20.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.180+0000 7f7cee7fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7cf01029b0 0x7f7cf0198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:20.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.180+0000 7f7ceeffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7cf0102010 0x7f7cf0197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:20.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.180+0000 7f7cee7fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7cf01029b0 0x7f7cf0198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:34816/0 (socket says 172.21.15.101:34816) 2023-07-18T03:06:20.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.180+0000 7f7cee7fc700 1 -- 172.21.15.101:0/319187197 learned_addr learned my addr 172.21.15.101:0/319187197 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:20.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.180+0000 7f7cef7fe700 1 --2- 172.21.15.101:0/319187197 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7cf01070b0 0x7f7cf019c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:20.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.180+0000 7f7cee7fc700 1 -- 172.21.15.101:0/319187197 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7cf01070b0 msgr2=0x7f7cf019c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:20.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.180+0000 7f7cee7fc700 1 --2- 172.21.15.101:0/319187197 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7cf01070b0 0x7f7cf019c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:20.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.180+0000 7f7cee7fc700 1 -- 172.21.15.101:0/319187197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7cf0102010 msgr2=0x7f7cf0197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:20.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.180+0000 7f7cee7fc700 1 --2- 172.21.15.101:0/319187197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7cf0102010 0x7f7cf0197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:20.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.180+0000 7f7cee7fc700 1 -- 172.21.15.101:0/319187197 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce40095f0 con 0x7f7cf01029b0 2023-07-18T03:06:20.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.180+0000 7f7cef7fe700 1 --2- 172.21.15.101:0/319187197 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7cf01070b0 0x7f7cf019c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:06:20.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.181+0000 7f7cee7fc700 1 --2- 172.21.15.101:0/319187197 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7cf01029b0 0x7f7cf0198390 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7ce40098e0 tx=0x7f7ce400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:20.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.181+0000 7f7cf48f3700 1 -- 172.21.15.101:0/319187197 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce401d070 con 0x7f7cf01029b0 2023-07-18T03:06:20.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.181+0000 7f7cf48f3700 1 -- 172.21.15.101:0/319187197 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ce400fb80 con 0x7f7cf01029b0 2023-07-18T03:06:20.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.181+0000 7f7cf48f3700 1 -- 172.21.15.101:0/319187197 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce40173f0 con 0x7f7cf01029b0 2023-07-18T03:06:20.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.181+0000 7f7cf58f5700 1 -- 172.21.15.101:0/319187197 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cf019d3c0 con 0x7f7cf01029b0 2023-07-18T03:06:20.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.181+0000 7f7cf58f5700 1 -- 172.21.15.101:0/319187197 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cf01a4ab0 con 0x7f7cf01029b0 2023-07-18T03:06:20.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.182+0000 7f7cf58f5700 1 -- 172.21.15.101:0/319187197 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cf004e210 con 0x7f7cf01029b0 2023-07-18T03:06:20.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.184+0000 7f7cf48f3700 1 -- 172.21.15.101:0/319187197 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f7ce400fcf0 con 0x7f7cf01029b0 2023-07-18T03:06:20.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.185+0000 7f7cf48f3700 1 --2- 172.21.15.101:0/319187197 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f7cd80a11e0 0x7f7cd80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:20.697 INFO:teuthology.orchestra.run.smithi101.stdout:34359738446 2023-07-18T03:06:20.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.185+0000 7f7cf48f3700 1 -- 172.21.15.101:0/319187197 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f7ce40cda00 con 0x7f7cf01029b0 2023-07-18T03:06:20.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.185+0000 7f7ceeffd700 1 --2- 172.21.15.101:0/319187197 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f7cd80a11e0 0x7f7cd80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:20.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.186+0000 7f7ceeffd700 1 --2- 172.21.15.101:0/319187197 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f7cd80a11e0 0x7f7cd80a3690 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0009c60 tx=0x7f7ce0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:20.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.189+0000 7f7cf48f3700 1 -- 172.21.15.101:0/319187197 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7ce4096d20 con 0x7f7cf01029b0 2023-07-18T03:06:20.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.430+0000 7f7cf58f5700 1 -- 172.21.15.101:0/319187197 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7cf01991b0 con 0x7f7cf01029b0 2023-07-18T03:06:20.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.430+0000 7f7cf48f3700 1 -- 172.21.15.101:0/319187197 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7ce409a340 con 0x7f7cf01029b0 2023-07-18T03:06:20.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.432+0000 7f7cf58f5700 1 -- 172.21.15.101:0/319187197 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f7cd80a11e0 msgr2=0x7f7cd80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:20.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.432+0000 7f7cf58f5700 1 --2- 172.21.15.101:0/319187197 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f7cd80a11e0 0x7f7cd80a3690 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0009c60 tx=0x7f7ce0009260 comp rx=0 tx=0).stop 2023-07-18T03:06:20.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.432+0000 7f7cf58f5700 1 -- 172.21.15.101:0/319187197 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7cf01029b0 msgr2=0x7f7cf0198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:20.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.432+0000 7f7cf58f5700 1 --2- 172.21.15.101:0/319187197 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7cf01029b0 0x7f7cf0198390 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7ce40098e0 tx=0x7f7ce400f690 comp rx=0 tx=0).stop 2023-07-18T03:06:20.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.433+0000 7f7cf58f5700 1 -- 172.21.15.101:0/319187197 shutdown_connections 2023-07-18T03:06:20.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.433+0000 7f7cf58f5700 1 --2- 172.21.15.101:0/319187197 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f7cd80a11e0 0x7f7cd80a3690 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:20.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.433+0000 7f7cf58f5700 1 --2- 172.21.15.101:0/319187197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7cf0102010 0x7f7cf0197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:20.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.433+0000 7f7cf58f5700 1 --2- 172.21.15.101:0/319187197 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7cf01070b0 0x7f7cf019c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:20.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.433+0000 7f7cf58f5700 1 --2- 172.21.15.101:0/319187197 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7cf01029b0 0x7f7cf0198390 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:20.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.433+0000 7f7cf58f5700 1 -- 172.21.15.101:0/319187197 >> 172.21.15.101:0/319187197 conn(0x7f7cf00fd8b0 msgr2=0x7f7cf00fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:20.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.433+0000 7f7cf58f5700 1 -- 172.21.15.101:0/319187197 shutdown_connections 2023-07-18T03:06:20.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:19.433+0000 7f7cf58f5700 1 -- 172.21.15.101:0/319187197 wait complete. 2023-07-18T03:06:21.010 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678046 got 304942678048 for osd.10 2023-07-18T03:06:21.010 DEBUG:teuthology.parallel:result is None 2023-07-18T03:06:21.491 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738443 got 34359738446 for osd.0 2023-07-18T03:06:21.492 DEBUG:teuthology.parallel:result is None 2023-07-18T03:06:21.926 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841569 got 283467841572 for osd.9 2023-07-18T03:06:21.927 DEBUG:teuthology.parallel:result is None 2023-07-18T03:06:21.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:21.935+0000 7fa366c35700 1 -- 172.21.15.101:0/2823404869 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa3600ffe30 msgr2=0x7fa360100200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:21.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:21.935+0000 7fa366c35700 1 --2- 172.21.15.101:0/2823404869 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa3600ffe30 0x7fa360100200 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa354009920 tx=0x7fa354009c30 comp rx=0 tx=0).stop 2023-07-18T03:06:21.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:21.935+0000 7fa366c35700 1 -- 172.21.15.101:0/2823404869 shutdown_connections 2023-07-18T03:06:21.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:21.935+0000 7fa366c35700 1 --2- 172.21.15.101:0/2823404869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa360104ed0 0x7fa36010ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:21.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:21.935+0000 7fa366c35700 1 --2- 172.21.15.101:0/2823404869 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa3600ffe30 0x7fa360100200 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:21.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:21.935+0000 7fa366c35700 1 --2- 172.21.15.101:0/2823404869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3601007d0 0x7fa3601047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:21.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:21.935+0000 7fa366c35700 1 -- 172.21.15.101:0/2823404869 >> 172.21.15.101:0/2823404869 conn(0x7fa3600fb6d0 msgr2=0x7fa3600fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:21.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:21.935+0000 7fa366c35700 1 -- 172.21.15.101:0/2823404869 shutdown_connections 2023-07-18T03:06:21.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:21.935+0000 7fa366c35700 1 -- 172.21.15.101:0/2823404869 wait complete. 2023-07-18T03:06:21.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:21.937+0000 7fa366c35700 1 Processor -- start 2023-07-18T03:06:21.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:21.937+0000 7fa366c35700 1 -- start start 2023-07-18T03:06:21.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:21.938+0000 7fa366c35700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3600ffe30 0x7fa36010b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:21.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:21.938+0000 7fa366c35700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3601007d0 0x7fa36010be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:21.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:21.938+0000 7fa366c35700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa360104ed0 0x7fa3601a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:21.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:21.938+0000 7fa366c35700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3601a12c0 con 0x7fa360104ed0 2023-07-18T03:06:21.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:21.938+0000 7fa366c35700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3601a1430 con 0x7fa3600ffe30 2023-07-18T03:06:21.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:21.938+0000 7fa366c35700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3601a15f0 con 0x7fa3601007d0 2023-07-18T03:06:21.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:21.938+0000 7fa3649d1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3600ffe30 0x7fa36010b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:21.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:21.938+0000 7fa3649d1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3600ffe30 0x7fa36010b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60064/0 (socket says 172.21.15.101:60064) 2023-07-18T03:06:21.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:21.938+0000 7fa3649d1700 1 -- 172.21.15.101:0/416909582 learned_addr learned my addr 172.21.15.101:0/416909582 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:21.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:21.938+0000 7fa35ffff700 1 --2- 172.21.15.101:0/416909582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3601007d0 0x7fa36010be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:21.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:21.939+0000 7fa3651d2700 1 --2- 172.21.15.101:0/416909582 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa360104ed0 0x7fa3601a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:21.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:21.939+0000 7fa3649d1700 1 -- 172.21.15.101:0/416909582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3601007d0 msgr2=0x7fa36010be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:21.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:21.939+0000 7fa3649d1700 1 --2- 172.21.15.101:0/416909582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3601007d0 0x7fa36010be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:21.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:21.939+0000 7fa3649d1700 1 -- 172.21.15.101:0/416909582 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa360104ed0 msgr2=0x7fa3601a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:21.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:21.939+0000 7fa3649d1700 1 --2- 172.21.15.101:0/416909582 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa360104ed0 0x7fa3601a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:21.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:21.939+0000 7fa3649d1700 1 -- 172.21.15.101:0/416909582 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa354009520 con 0x7fa3600ffe30 2023-07-18T03:06:21.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:21.939+0000 7fa3651d2700 1 --2- 172.21.15.101:0/416909582 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa360104ed0 0x7fa3601a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:06:21.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:21.939+0000 7fa3649d1700 1 --2- 172.21.15.101:0/416909582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3600ffe30 0x7fa36010b8e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa354009230 tx=0x7fa35400bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:21.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:21.939+0000 7fa35dffb700 1 -- 172.21.15.101:0/416909582 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35401c070 con 0x7fa3600ffe30 2023-07-18T03:06:21.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:21.940+0000 7fa366c35700 1 -- 172.21.15.101:0/416909582 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3601a1870 con 0x7fa3600ffe30 2023-07-18T03:06:21.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:21.940+0000 7fa35dffb700 1 -- 172.21.15.101:0/416909582 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa354005040 con 0x7fa3600ffe30 2023-07-18T03:06:21.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:21.940+0000 7fa35dffb700 1 -- 172.21.15.101:0/416909582 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa354004a10 con 0x7fa3600ffe30 2023-07-18T03:06:21.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:21.940+0000 7fa366c35700 1 -- 172.21.15.101:0/416909582 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3601a9130 con 0x7fa3600ffe30 2023-07-18T03:06:21.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:21.941+0000 7fa366c35700 1 -- 172.21.15.101:0/416909582 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa36004e210 con 0x7fa3600ffe30 2023-07-18T03:06:21.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:21.943+0000 7fa35dffb700 1 -- 172.21.15.101:0/416909582 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fa354020070 con 0x7fa3600ffe30 2023-07-18T03:06:21.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:21.944+0000 7fa35dffb700 1 --2- 172.21.15.101:0/416909582 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa3500a1260 0x7fa3500a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:21.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:21.945+0000 7fa35dffb700 1 -- 172.21.15.101:0/416909582 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fa3540cdd40 con 0x7fa3600ffe30 2023-07-18T03:06:21.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:21.945+0000 7fa35ffff700 1 --2- 172.21.15.101:0/416909582 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa3500a1260 0x7fa3500a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:21.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:21.945+0000 7fa35ffff700 1 --2- 172.21.15.101:0/416909582 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa3500a1260 0x7fa3500a3710 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa36010cf00 tx=0x7fa34c009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:21.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:21.946+0000 7fa35dffb700 1 -- 172.21.15.101:0/416909582 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa3540970c0 con 0x7fa3600ffe30 2023-07-18T03:06:21.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.011+0000 7fd35bb29700 1 -- 172.21.15.101:0/530329466 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3540728b0 msgr2=0x7fd354072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:21.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.011+0000 7fd35bb29700 1 --2- 172.21.15.101:0/530329466 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3540728b0 0x7fd354072c80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd3500099c0 tx=0x7fd350009cd0 comp rx=0 tx=0).stop 2023-07-18T03:06:21.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.012+0000 7fd35bb29700 1 -- 172.21.15.101:0/530329466 shutdown_connections 2023-07-18T03:06:21.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.012+0000 7fd35bb29700 1 --2- 172.21.15.101:0/530329466 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd354110bb0 0x7fd354112f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:21.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.012+0000 7fd35bb29700 1 --2- 172.21.15.101:0/530329466 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3540728b0 0x7fd354072c80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:21.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.012+0000 7fd35bb29700 1 --2- 172.21.15.101:0/530329466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3540731c0 0x7fd354110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:21.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.012+0000 7fd35bb29700 1 -- 172.21.15.101:0/530329466 >> 172.21.15.101:0/530329466 conn(0x7fd3540fb660 msgr2=0x7fd3540fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:21.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.012+0000 7fd35bb29700 1 -- 172.21.15.101:0/530329466 shutdown_connections 2023-07-18T03:06:21.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.012+0000 7fd35bb29700 1 -- 172.21.15.101:0/530329466 wait complete. 2023-07-18T03:06:22.001 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.013+0000 7fd35bb29700 1 Processor -- start 2023-07-18T03:06:22.001 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.013+0000 7fd35bb29700 1 -- start start 2023-07-18T03:06:22.001 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.013+0000 7fd35bb29700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3540728b0 0x7fd35419c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:22.001 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.013+0000 7fd35bb29700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3540731c0 0x7fd35419c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:22.001 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.013+0000 7fd35bb29700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd354110bb0 0x7fd3541a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:22.001 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.013+0000 7fd35bb29700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3541a1290 con 0x7fd354110bb0 2023-07-18T03:06:22.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.013+0000 7fd35bb29700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3541a1400 con 0x7fd3540728b0 2023-07-18T03:06:22.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.013+0000 7fd35bb29700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3541a15c0 con 0x7fd3540731c0 2023-07-18T03:06:22.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.014+0000 7fd3598c5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3540728b0 0x7fd35419c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:22.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.014+0000 7fd3598c5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3540728b0 0x7fd35419c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60070/0 (socket says 172.21.15.101:60070) 2023-07-18T03:06:22.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.014+0000 7fd3598c5700 1 -- 172.21.15.101:0/3253951571 learned_addr learned my addr 172.21.15.101:0/3253951571 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:22.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.014+0000 7fd35a0c6700 1 --2- 172.21.15.101:0/3253951571 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd354110bb0 0x7fd3541a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:22.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.014+0000 7fd3590c4700 1 --2- 172.21.15.101:0/3253951571 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3540731c0 0x7fd35419c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:22.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.014+0000 7fd3598c5700 1 -- 172.21.15.101:0/3253951571 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3540731c0 msgr2=0x7fd35419c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:22.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.014+0000 7fd3598c5700 1 --2- 172.21.15.101:0/3253951571 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3540731c0 0x7fd35419c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:22.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.014+0000 7fd3598c5700 1 -- 172.21.15.101:0/3253951571 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd354110bb0 msgr2=0x7fd3541a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:22.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.014+0000 7fd3598c5700 1 --2- 172.21.15.101:0/3253951571 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd354110bb0 0x7fd3541a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:22.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.014+0000 7fd3598c5700 1 -- 172.21.15.101:0/3253951571 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd344009520 con 0x7fd3540728b0 2023-07-18T03:06:22.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.014+0000 7fd3590c4700 1 --2- 172.21.15.101:0/3253951571 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3540731c0 0x7fd35419c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:06:22.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.014+0000 7fd3598c5700 1 --2- 172.21.15.101:0/3253951571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3540728b0 0x7fd35419c2d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd350009990 tx=0x7fd35000f740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:22.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.015+0000 7fd34affd700 1 -- 172.21.15.101:0/3253951571 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35001d070 con 0x7fd3540728b0 2023-07-18T03:06:22.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.015+0000 7fd35bb29700 1 -- 172.21.15.101:0/3253951571 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3500095f0 con 0x7fd3540728b0 2023-07-18T03:06:22.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.015+0000 7fd35bb29700 1 -- 172.21.15.101:0/3253951571 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3541a8f30 con 0x7fd3540728b0 2023-07-18T03:06:22.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.015+0000 7fd34affd700 1 -- 172.21.15.101:0/3253951571 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3500054f0 con 0x7fd3540728b0 2023-07-18T03:06:22.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.015+0000 7fd34affd700 1 -- 172.21.15.101:0/3253951571 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd350004bb0 con 0x7fd3540728b0 2023-07-18T03:06:22.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.016+0000 7fd35bb29700 1 -- 172.21.15.101:0/3253951571 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd338005320 con 0x7fd3540728b0 2023-07-18T03:06:22.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.018+0000 7fd34affd700 1 -- 172.21.15.101:0/3253951571 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fd35000fc30 con 0x7fd3540728b0 2023-07-18T03:06:22.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.020+0000 7fd34affd700 1 --2- 172.21.15.101:0/3253951571 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd3400a1260 0x7fd3400a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:22.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.021+0000 7fd34affd700 1 -- 172.21.15.101:0/3253951571 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fd3500cda40 con 0x7fd3540728b0 2023-07-18T03:06:22.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.021+0000 7fd3590c4700 1 --2- 172.21.15.101:0/3253951571 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd3400a1260 0x7fd3400a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:22.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.021+0000 7fd3590c4700 1 --2- 172.21.15.101:0/3253951571 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd3400a1260 0x7fd3400a3710 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd35419d8f0 tx=0x7fd344009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:22.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.022+0000 7fd34affd700 1 -- 172.21.15.101:0/3253951571 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd350096d60 con 0x7fd3540728b0 2023-07-18T03:06:22.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.161+0000 7fa366c35700 1 -- 172.21.15.101:0/416909582 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fa3601a92c0 con 0x7fa3600ffe30 2023-07-18T03:06:22.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.162+0000 7fa35dffb700 1 -- 172.21.15.101:0/416909582 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa354027090 con 0x7fa3600ffe30 2023-07-18T03:06:22.149 INFO:teuthology.orchestra.run.smithi101.stdout:141733920828 2023-07-18T03:06:22.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.165+0000 7fa366c35700 1 -- 172.21.15.101:0/416909582 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa3500a1260 msgr2=0x7fa3500a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:22.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.165+0000 7fa366c35700 1 --2- 172.21.15.101:0/416909582 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa3500a1260 0x7fa3500a3710 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa36010cf00 tx=0x7fa34c009310 comp rx=0 tx=0).stop 2023-07-18T03:06:22.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.165+0000 7fa366c35700 1 -- 172.21.15.101:0/416909582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3600ffe30 msgr2=0x7fa36010b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:22.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.165+0000 7fa366c35700 1 --2- 172.21.15.101:0/416909582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3600ffe30 0x7fa36010b8e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa354009230 tx=0x7fa35400bf60 comp rx=0 tx=0).stop 2023-07-18T03:06:22.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.165+0000 7fa366c35700 1 -- 172.21.15.101:0/416909582 shutdown_connections 2023-07-18T03:06:22.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.165+0000 7fa366c35700 1 --2- 172.21.15.101:0/416909582 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa3500a1260 0x7fa3500a3710 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:22.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.165+0000 7fa366c35700 1 --2- 172.21.15.101:0/416909582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3600ffe30 0x7fa36010b8e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:22.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.165+0000 7fa366c35700 1 --2- 172.21.15.101:0/416909582 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa360104ed0 0x7fa3601a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:22.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.165+0000 7fa366c35700 1 --2- 172.21.15.101:0/416909582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3601007d0 0x7fa36010be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:22.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.165+0000 7fa366c35700 1 -- 172.21.15.101:0/416909582 >> 172.21.15.101:0/416909582 conn(0x7fa3600fb6d0 msgr2=0x7fa3600fc210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:22.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.165+0000 7fa366c35700 1 -- 172.21.15.101:0/416909582 shutdown_connections 2023-07-18T03:06:22.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.165+0000 7fa366c35700 1 -- 172.21.15.101:0/416909582 wait complete. 2023-07-18T03:06:22.175 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:22 smithi101 ceph-mon[126379]: pgmap v318: 129 pgs: 129 active+clean; 640 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:06:22.175 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:22 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:06:22.175 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:22 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:06:22.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.250+0000 7fd35bb29700 1 -- 172.21.15.101:0/3253951571 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd338005cc0 con 0x7fd3540728b0 2023-07-18T03:06:22.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.251+0000 7fd34affd700 1 -- 172.21.15.101:0/3253951571 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd35009a380 con 0x7fd3540728b0 2023-07-18T03:06:22.238 INFO:teuthology.orchestra.run.smithi101.stdout:77309411399 2023-07-18T03:06:22.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.254+0000 7fd35bb29700 1 -- 172.21.15.101:0/3253951571 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd3400a1260 msgr2=0x7fd3400a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:22.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.254+0000 7fd35bb29700 1 --2- 172.21.15.101:0/3253951571 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd3400a1260 0x7fd3400a3710 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd35419d8f0 tx=0x7fd344009260 comp rx=0 tx=0).stop 2023-07-18T03:06:22.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.254+0000 7fd35bb29700 1 -- 172.21.15.101:0/3253951571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3540728b0 msgr2=0x7fd35419c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:22.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.254+0000 7fd35bb29700 1 --2- 172.21.15.101:0/3253951571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3540728b0 0x7fd35419c2d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd350009990 tx=0x7fd35000f740 comp rx=0 tx=0).stop 2023-07-18T03:06:22.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.255+0000 7fd35bb29700 1 -- 172.21.15.101:0/3253951571 shutdown_connections 2023-07-18T03:06:22.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.255+0000 7fd35bb29700 1 --2- 172.21.15.101:0/3253951571 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd3400a1260 0x7fd3400a3710 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:22.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.255+0000 7fd35bb29700 1 --2- 172.21.15.101:0/3253951571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3540728b0 0x7fd35419c2d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:22.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.255+0000 7fd35bb29700 1 --2- 172.21.15.101:0/3253951571 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd354110bb0 0x7fd3541a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:22.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.255+0000 7fd35bb29700 1 --2- 172.21.15.101:0/3253951571 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3540731c0 0x7fd35419c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:22.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.255+0000 7fd35bb29700 1 -- 172.21.15.101:0/3253951571 >> 172.21.15.101:0/3253951571 conn(0x7fd3540fb660 msgr2=0x7fd3540fc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:22.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.255+0000 7fd35bb29700 1 -- 172.21.15.101:0/3253951571 shutdown_connections 2023-07-18T03:06:22.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:22.255+0000 7fd35bb29700 1 -- 172.21.15.101:0/3253951571 wait complete. 2023-07-18T03:06:22.443 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:22 smithi179 ceph-mon[129229]: pgmap v318: 129 pgs: 129 active+clean; 640 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:06:22.443 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:22 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:06:22.443 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:22 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:06:22.538 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:22 smithi155 ceph-mon[128885]: pgmap v318: 129 pgs: 129 active+clean; 640 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:06:22.538 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:22 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:06:22.538 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:22 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:06:23.082 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920825 got 141733920828 for osd.4 2023-07-18T03:06:23.083 DEBUG:teuthology.parallel:result is None 2023-07-18T03:06:23.143 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411396 got 77309411399 for osd.2 2023-07-18T03:06:23.144 DEBUG:teuthology.parallel:result is None 2023-07-18T03:06:23.144 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-07-18T03:06:23.145 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph pg dump --format=json 2023-07-18T03:06:23.443 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:23 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/416909582' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-07-18T03:06:23.443 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:23 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:06:23.443 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:23 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:06:23.443 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:23 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:06:23.444 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:23 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/3253951571' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-07-18T03:06:23.444 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:23 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2023-07-18T03:06:23.444 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:23 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2023-07-18T03:06:23.444 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:23 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2023-07-18T03:06:23.444 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:23 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2023-07-18T03:06:23.480 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:23 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/416909582' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-07-18T03:06:23.480 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:23 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:06:23.481 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:23 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:06:23.481 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:23 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:06:23.481 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:23 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3253951571' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-07-18T03:06:23.481 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:23 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2023-07-18T03:06:23.481 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:23 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2023-07-18T03:06:23.481 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:23 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2023-07-18T03:06:23.482 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:23 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2023-07-18T03:06:23.538 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:23 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/416909582' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-07-18T03:06:23.538 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:23 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:06:23.538 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:23 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:06:23.538 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:23 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' 2023-07-18T03:06:23.538 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:23 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/3253951571' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-07-18T03:06:23.538 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:23 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2023-07-18T03:06:23.539 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:23 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2023-07-18T03:06:23.539 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:23 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2023-07-18T03:06:23.539 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:23 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2023-07-18T03:06:23.870 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:06:24.481 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:24 smithi101 ceph-mon[126379]: pgmap v319: 129 pgs: 129 active+clean; 640 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-18T03:06:24.481 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:24 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2023-07-18T03:06:24.481 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:24 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2023-07-18T03:06:24.481 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:24 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2023-07-18T03:06:24.481 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:24 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2023-07-18T03:06:24.482 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:24 smithi101 ceph-mon[126379]: osdmap e90: 12 total, 12 up, 12 in 2023-07-18T03:06:24.538 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:24 smithi155 ceph-mon[128885]: pgmap v319: 129 pgs: 129 active+clean; 640 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-18T03:06:24.538 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:24 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2023-07-18T03:06:24.538 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:24 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2023-07-18T03:06:24.538 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:24 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2023-07-18T03:06:24.539 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:24 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2023-07-18T03:06:24.539 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:24 smithi155 ceph-mon[128885]: osdmap e90: 12 total, 12 up, 12 in 2023-07-18T03:06:24.693 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:24 smithi179 ceph-mon[129229]: pgmap v319: 129 pgs: 129 active+clean; 640 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-18T03:06:24.693 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:24 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2023-07-18T03:06:24.693 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:24 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2023-07-18T03:06:24.693 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:24 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2023-07-18T03:06:24.694 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:24 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2023-07-18T03:06:24.694 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:24 smithi179 ceph-mon[129229]: osdmap e90: 12 total, 12 up, 12 in 2023-07-18T03:06:25.538 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:25 smithi155 ceph-mon[128885]: osdmap e91: 12 total, 12 up, 12 in 2023-07-18T03:06:25.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.581+0000 7f434d136700 1 -- 172.21.15.101:0/3774778122 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f43480ffe00 msgr2=0x7f43481001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:25.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.581+0000 7f434d136700 1 --2- 172.21.15.101:0/3774778122 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f43480ffe00 0x7f43481001d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f43380099c0 tx=0x7f4338009cd0 comp rx=0 tx=0).stop 2023-07-18T03:06:25.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.581+0000 7f434d136700 1 -- 172.21.15.101:0/3774778122 shutdown_connections 2023-07-18T03:06:25.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.581+0000 7f434d136700 1 --2- 172.21.15.101:0/3774778122 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4348104ea0 0x7f434810ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:25.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.581+0000 7f434d136700 1 --2- 172.21.15.101:0/3774778122 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f43480ffe00 0x7f43481001d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:25.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.581+0000 7f434d136700 1 --2- 172.21.15.101:0/3774778122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43481007a0 0x7f4348104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:25.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.581+0000 7f434d136700 1 -- 172.21.15.101:0/3774778122 >> 172.21.15.101:0/3774778122 conn(0x7f43480fb660 msgr2=0x7f43480fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:25.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.581+0000 7f434d136700 1 -- 172.21.15.101:0/3774778122 shutdown_connections 2023-07-18T03:06:25.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.581+0000 7f434d136700 1 -- 172.21.15.101:0/3774778122 wait complete. 2023-07-18T03:06:25.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.582+0000 7f434d136700 1 Processor -- start 2023-07-18T03:06:25.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.582+0000 7f434d136700 1 -- start start 2023-07-18T03:06:25.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.582+0000 7f434d136700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f43480ffe00 0x7f4348072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:25.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.582+0000 7f434d136700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43481007a0 0x7f4348072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:25.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.583+0000 7f434d136700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4348104ea0 0x7f43480732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:25.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.583+0000 7f434d136700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43481a40f0 con 0x7f43480ffe00 2023-07-18T03:06:25.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.583+0000 7f434d136700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43481a4260 con 0x7f4348104ea0 2023-07-18T03:06:25.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.583+0000 7f434d136700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43481a4420 con 0x7f43481007a0 2023-07-18T03:06:25.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.583+0000 7f4346d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f43480ffe00 0x7f4348072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:25.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.583+0000 7f4346d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f43480ffe00 0x7f4348072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:34874/0 (socket says 172.21.15.101:34874) 2023-07-18T03:06:25.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.583+0000 7f4346d9d700 1 -- 172.21.15.101:0/3767744201 learned_addr learned my addr 172.21.15.101:0/3767744201 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:25.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.583+0000 7f434659c700 1 --2- 172.21.15.101:0/3767744201 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43481007a0 0x7f4348072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:25.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.583+0000 7f434759e700 1 --2- 172.21.15.101:0/3767744201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4348104ea0 0x7f43480732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:25.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.583+0000 7f4346d9d700 1 -- 172.21.15.101:0/3767744201 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43481007a0 msgr2=0x7f4348072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:25.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.583+0000 7f4346d9d700 1 --2- 172.21.15.101:0/3767744201 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43481007a0 0x7f4348072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:25.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.583+0000 7f4346d9d700 1 -- 172.21.15.101:0/3767744201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4348104ea0 msgr2=0x7f43480732d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:25.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.583+0000 7f4346d9d700 1 --2- 172.21.15.101:0/3767744201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4348104ea0 0x7f43480732d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:25.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.583+0000 7f4346d9d700 1 -- 172.21.15.101:0/3767744201 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43380095f0 con 0x7f43480ffe00 2023-07-18T03:06:25.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.583+0000 7f4346d9d700 1 --2- 172.21.15.101:0/3767744201 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f43480ffe00 0x7f4348072810 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4338000c00 tx=0x7f433800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:25.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.584+0000 7f432ffff700 1 -- 172.21.15.101:0/3767744201 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433801d070 con 0x7f43480ffe00 2023-07-18T03:06:25.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.584+0000 7f432ffff700 1 -- 172.21.15.101:0/3767744201 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4338005050 con 0x7f43480ffe00 2023-07-18T03:06:25.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.584+0000 7f434d136700 1 -- 172.21.15.101:0/3767744201 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43481a46a0 con 0x7f43480ffe00 2023-07-18T03:06:25.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.584+0000 7f432ffff700 1 -- 172.21.15.101:0/3767744201 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4338004ab0 con 0x7f43480ffe00 2023-07-18T03:06:25.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.584+0000 7f434d136700 1 -- 172.21.15.101:0/3767744201 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43481a4ab0 con 0x7f43480ffe00 2023-07-18T03:06:25.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.585+0000 7f434d136700 1 -- 172.21.15.101:0/3767744201 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f434804e210 con 0x7f43480ffe00 2023-07-18T03:06:25.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.587+0000 7f432ffff700 1 -- 172.21.15.101:0/3767744201 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f43380051c0 con 0x7f43480ffe00 2023-07-18T03:06:25.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.588+0000 7f432ffff700 1 --2- 172.21.15.101:0/3767744201 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f43300a11e0 0x7f43300a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:25.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.588+0000 7f432ffff700 1 -- 172.21.15.101:0/3767744201 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f43380cdf60 con 0x7f43480ffe00 2023-07-18T03:06:25.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.588+0000 7f434659c700 1 --2- 172.21.15.101:0/3767744201 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f43300a11e0 0x7f43300a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:25.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.589+0000 7f434659c700 1 --2- 172.21.15.101:0/3767744201 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f43300a11e0 0x7f43300a3690 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4348074000 tx=0x7f433c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:25.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.590+0000 7f432ffff700 1 -- 172.21.15.101:0/3767744201 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4338097210 con 0x7f43480ffe00 2023-07-18T03:06:25.693 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:25 smithi179 ceph-mon[129229]: osdmap e91: 12 total, 12 up, 12 in 2023-07-18T03:06:25.730 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:25 smithi101 ceph-mon[126379]: osdmap e91: 12 total, 12 up, 12 in 2023-07-18T03:06:25.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.782+0000 7f434d136700 1 -- 172.21.15.101:0/3767744201 --> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f43480fc860 con 0x7f43300a11e0 2023-07-18T03:06:25.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.795+0000 7f432ffff700 1 -- 172.21.15.101:0/3767744201 <== mgr.14150 v2:172.21.15.101:6800/3963742842 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426607 (secure 0 0 0) 0x7f43480fc860 con 0x7f43300a11e0 2023-07-18T03:06:25.790 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:06:25.793 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.801+0000 7f434d136700 1 -- 172.21.15.101:0/3767744201 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f43300a11e0 msgr2=0x7f43300a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:25.793 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.801+0000 7f434d136700 1 --2- 172.21.15.101:0/3767744201 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f43300a11e0 0x7f43300a3690 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4348074000 tx=0x7f433c009260 comp rx=0 tx=0).stop 2023-07-18T03:06:25.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.801+0000 7f434d136700 1 -- 172.21.15.101:0/3767744201 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f43480ffe00 msgr2=0x7f4348072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:25.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.801+0000 7f434d136700 1 --2- 172.21.15.101:0/3767744201 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f43480ffe00 0x7f4348072810 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4338000c00 tx=0x7f433800bfd0 comp rx=0 tx=0).stop 2023-07-18T03:06:25.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.801+0000 7f434d136700 1 -- 172.21.15.101:0/3767744201 shutdown_connections 2023-07-18T03:06:25.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.801+0000 7f434d136700 1 --2- 172.21.15.101:0/3767744201 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f43300a11e0 0x7f43300a3690 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:25.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.801+0000 7f434d136700 1 --2- 172.21.15.101:0/3767744201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4348104ea0 0x7f43480732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:25.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.801+0000 7f434d136700 1 --2- 172.21.15.101:0/3767744201 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f43480ffe00 0x7f4348072810 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:25.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.801+0000 7f434d136700 1 --2- 172.21.15.101:0/3767744201 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43481007a0 0x7f4348072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:25.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.801+0000 7f434d136700 1 -- 172.21.15.101:0/3767744201 >> 172.21.15.101:0/3767744201 conn(0x7f43480fb660 msgr2=0x7f43480fc150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:25.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.801+0000 7f434d136700 1 -- 172.21.15.101:0/3767744201 shutdown_connections 2023-07-18T03:06:25.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:25.801+0000 7f434d136700 1 -- 172.21.15.101:0/3767744201 wait complete. 2023-07-18T03:06:25.795 INFO:teuthology.orchestra.run.smithi101.stderr:dumped all 2023-07-18T03:06:26.259 INFO:teuthology.orchestra.run.smithi101.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2023-07-18T03:06:24.872064+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655355,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":464,"num_read_kb":351,"num_write":337,"num_write_kb":3050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335300,"kb_used_data":11876,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606524,"statfs":{"total":1151940427776,"available":1151597080576,"internally_reserved":0,"allocated":12161024,"data_stored":7803975,"data_compressed":321125,"data_compressed_allocated":434176,"data_compressed_original":2367488,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":50,"num_read_kb":25,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002090"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.942306+0000","last_change":"2023-07-18T03:04:59.805689+0000","last_active":"2023-07-18T03:05:23.942306+0000","last_peered":"2023-07-18T03:05:23.942306+0000","last_clean":"2023-07-18T03:05:23.942306+0000","last_became_active":"2023-07-18T03:04:59.805224+0000","last_became_peered":"2023-07-18T03:04:59.805224+0000","last_unstale":"2023-07-18T03:05:23.942306+0000","last_undegraded":"2023-07-18T03:05:23.942306+0000","last_fullsized":"2023-07-18T03:05:23.942306+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:55:30.570149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.264959+0000","last_change":"2023-07-18T03:04:59.805013+0000","last_active":"2023-07-18T03:06:23.264959+0000","last_peered":"2023-07-18T03:06:23.264959+0000","last_clean":"2023-07-18T03:06:23.264959+0000","last_became_active":"2023-07-18T03:04:59.804702+0000","last_became_peered":"2023-07-18T03:04:59.804702+0000","last_unstale":"2023-07-18T03:06:23.264959+0000","last_undegraded":"2023-07-18T03:06:23.264959+0000","last_fullsized":"2023-07-18T03:06:23.264959+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:24:23.239057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.265217+0000","last_change":"2023-07-18T03:04:59.809035+0000","last_active":"2023-07-18T03:06:23.265217+0000","last_peered":"2023-07-18T03:06:23.265217+0000","last_clean":"2023-07-18T03:06:23.265217+0000","last_became_active":"2023-07-18T03:04:59.808630+0000","last_became_peered":"2023-07-18T03:04:59.808630+0000","last_unstale":"2023-07-18T03:06:23.265217+0000","last_undegraded":"2023-07-18T03:06:23.265217+0000","last_fullsized":"2023-07-18T03:06:23.265217+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:53:21.301875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.259429+0000","last_change":"2023-07-18T03:04:59.803510+0000","last_active":"2023-07-18T03:06:23.259429+0000","last_peered":"2023-07-18T03:06:23.259429+0000","last_clean":"2023-07-18T03:06:23.259429+0000","last_became_active":"2023-07-18T03:04:59.803060+0000","last_became_peered":"2023-07-18T03:04:59.803060+0000","last_unstale":"2023-07-18T03:06:23.259429+0000","last_undegraded":"2023-07-18T03:06:23.259429+0000","last_fullsized":"2023-07-18T03:06:23.259429+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:28:38.229617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.936577+0000","last_change":"2023-07-18T03:04:59.806911+0000","last_active":"2023-07-18T03:05:23.936577+0000","last_peered":"2023-07-18T03:05:23.936577+0000","last_clean":"2023-07-18T03:05:23.936577+0000","last_became_active":"2023-07-18T03:04:59.806597+0000","last_became_peered":"2023-07-18T03:04:59.806597+0000","last_unstale":"2023-07-18T03:05:23.936577+0000","last_undegraded":"2023-07-18T03:05:23.936577+0000","last_fullsized":"2023-07-18T03:05:23.936577+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:06:55.956671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.936608+0000","last_change":"2023-07-18T03:04:59.809263+0000","last_active":"2023-07-18T03:05:23.936608+0000","last_peered":"2023-07-18T03:05:23.936608+0000","last_clean":"2023-07-18T03:05:23.936608+0000","last_became_active":"2023-07-18T03:04:59.808956+0000","last_became_peered":"2023-07-18T03:04:59.808956+0000","last_unstale":"2023-07-18T03:05:23.936608+0000","last_undegraded":"2023-07-18T03:05:23.936608+0000","last_fullsized":"2023-07-18T03:05:23.936608+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:49:07.735738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.256036+0000","last_change":"2023-07-18T03:04:59.804946+0000","last_active":"2023-07-18T03:06:23.256036+0000","last_peered":"2023-07-18T03:06:23.256036+0000","last_clean":"2023-07-18T03:06:23.256036+0000","last_became_active":"2023-07-18T03:04:59.804610+0000","last_became_peered":"2023-07-18T03:04:59.804610+0000","last_unstale":"2023-07-18T03:06:23.256036+0000","last_undegraded":"2023-07-18T03:06:23.256036+0000","last_fullsized":"2023-07-18T03:06:23.256036+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:41:01.522713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089839+0000","last_change":"2023-07-18T03:04:59.801796+0000","last_active":"2023-07-18T03:05:24.089839+0000","last_peered":"2023-07-18T03:05:24.089839+0000","last_clean":"2023-07-18T03:05:24.089839+0000","last_became_active":"2023-07-18T03:04:59.801432+0000","last_became_peered":"2023-07-18T03:04:59.801432+0000","last_unstale":"2023-07-18T03:05:24.089839+0000","last_undegraded":"2023-07-18T03:05:24.089839+0000","last_fullsized":"2023-07-18T03:05:24.089839+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:25:50.242446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089997+0000","last_change":"2023-07-18T03:04:59.798191+0000","last_active":"2023-07-18T03:05:24.089997+0000","last_peered":"2023-07-18T03:05:24.089997+0000","last_clean":"2023-07-18T03:05:24.089997+0000","last_became_active":"2023-07-18T03:04:59.797863+0000","last_became_peered":"2023-07-18T03:04:59.797863+0000","last_unstale":"2023-07-18T03:05:24.089997+0000","last_undegraded":"2023-07-18T03:05:24.089997+0000","last_fullsized":"2023-07-18T03:05:24.089997+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:23:59.798693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.936307+0000","last_change":"2023-07-18T03:04:59.809867+0000","last_active":"2023-07-18T03:05:23.936307+0000","last_peered":"2023-07-18T03:05:23.936307+0000","last_clean":"2023-07-18T03:05:23.936307+0000","last_became_active":"2023-07-18T03:04:59.809568+0000","last_became_peered":"2023-07-18T03:04:59.809568+0000","last_unstale":"2023-07-18T03:05:23.936307+0000","last_undegraded":"2023-07-18T03:05:23.936307+0000","last_fullsized":"2023-07-18T03:05:23.936307+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:51:02.596324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.256807+0000","last_change":"2023-07-18T03:04:59.805920+0000","last_active":"2023-07-18T03:06:23.256807+0000","last_peered":"2023-07-18T03:06:23.256807+0000","last_clean":"2023-07-18T03:06:23.256807+0000","last_became_active":"2023-07-18T03:04:59.805559+0000","last_became_peered":"2023-07-18T03:04:59.805559+0000","last_unstale":"2023-07-18T03:06:23.256807+0000","last_undegraded":"2023-07-18T03:06:23.256807+0000","last_fullsized":"2023-07-18T03:06:23.256807+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:18:07.455353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.259385+0000","last_change":"2023-07-18T03:04:59.797940+0000","last_active":"2023-07-18T03:06:23.259385+0000","last_peered":"2023-07-18T03:06:23.259385+0000","last_clean":"2023-07-18T03:06:23.259385+0000","last_became_active":"2023-07-18T03:04:59.797575+0000","last_became_peered":"2023-07-18T03:04:59.797575+0000","last_unstale":"2023-07-18T03:06:23.259385+0000","last_undegraded":"2023-07-18T03:06:23.259385+0000","last_fullsized":"2023-07-18T03:06:23.259385+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:28:11.054129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.091650+0000","last_change":"2023-07-18T03:04:59.798026+0000","last_active":"2023-07-18T03:05:24.091650+0000","last_peered":"2023-07-18T03:05:24.091650+0000","last_clean":"2023-07-18T03:05:24.091650+0000","last_became_active":"2023-07-18T03:04:59.797734+0000","last_became_peered":"2023-07-18T03:04:59.797734+0000","last_unstale":"2023-07-18T03:05:24.091650+0000","last_undegraded":"2023-07-18T03:05:24.091650+0000","last_fullsized":"2023-07-18T03:05:24.091650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:33:14.476227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.264921+0000","last_change":"2023-07-18T03:04:59.804964+0000","last_active":"2023-07-18T03:06:23.264921+0000","last_peered":"2023-07-18T03:06:23.264921+0000","last_clean":"2023-07-18T03:06:23.264921+0000","last_became_active":"2023-07-18T03:04:59.804660+0000","last_became_peered":"2023-07-18T03:04:59.804660+0000","last_unstale":"2023-07-18T03:06:23.264921+0000","last_undegraded":"2023-07-18T03:06:23.264921+0000","last_fullsized":"2023-07-18T03:06:23.264921+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:14:40.387622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.264800+0000","last_change":"2023-07-18T03:04:59.809384+0000","last_active":"2023-07-18T03:06:23.264800+0000","last_peered":"2023-07-18T03:06:23.264800+0000","last_clean":"2023-07-18T03:06:23.264800+0000","last_became_active":"2023-07-18T03:04:59.808969+0000","last_became_peered":"2023-07-18T03:04:59.808969+0000","last_unstale":"2023-07-18T03:06:23.264800+0000","last_undegraded":"2023-07-18T03:06:23.264800+0000","last_fullsized":"2023-07-18T03:06:23.264800+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:56:24.126487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089702+0000","last_change":"2023-07-18T03:04:59.808344+0000","last_active":"2023-07-18T03:05:24.089702+0000","last_peered":"2023-07-18T03:05:24.089702+0000","last_clean":"2023-07-18T03:05:24.089702+0000","last_became_active":"2023-07-18T03:04:59.807995+0000","last_became_peered":"2023-07-18T03:04:59.807995+0000","last_unstale":"2023-07-18T03:05:24.089702+0000","last_undegraded":"2023-07-18T03:05:24.089702+0000","last_fullsized":"2023-07-18T03:05:24.089702+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:20:11.542223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.942105+0000","last_change":"2023-07-18T03:04:59.806584+0000","last_active":"2023-07-18T03:05:23.942105+0000","last_peered":"2023-07-18T03:05:23.942105+0000","last_clean":"2023-07-18T03:05:23.942105+0000","last_became_active":"2023-07-18T03:04:59.804118+0000","last_became_peered":"2023-07-18T03:04:59.804118+0000","last_unstale":"2023-07-18T03:05:23.942105+0000","last_undegraded":"2023-07-18T03:05:23.942105+0000","last_fullsized":"2023-07-18T03:05:23.942105+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:10:34.684069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.934563+0000","last_change":"2023-07-18T03:04:59.812610+0000","last_active":"2023-07-18T03:05:23.934563+0000","last_peered":"2023-07-18T03:05:23.934563+0000","last_clean":"2023-07-18T03:05:23.934563+0000","last_became_active":"2023-07-18T03:04:59.812230+0000","last_became_peered":"2023-07-18T03:04:59.812230+0000","last_unstale":"2023-07-18T03:05:23.934563+0000","last_undegraded":"2023-07-18T03:05:23.934563+0000","last_fullsized":"2023-07-18T03:05:23.934563+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:05:54.390233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.260611+0000","last_change":"2023-07-18T03:04:59.811880+0000","last_active":"2023-07-18T03:06:23.260611+0000","last_peered":"2023-07-18T03:06:23.260611+0000","last_clean":"2023-07-18T03:06:23.260611+0000","last_became_active":"2023-07-18T03:04:59.811561+0000","last_became_peered":"2023-07-18T03:04:59.811561+0000","last_unstale":"2023-07-18T03:06:23.260611+0000","last_undegraded":"2023-07-18T03:06:23.260611+0000","last_fullsized":"2023-07-18T03:06:23.260611+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:22:13.152543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.259374+0000","last_change":"2023-07-18T03:05:03.880843+0000","last_active":"2023-07-18T03:06:23.259374+0000","last_peered":"2023-07-18T03:06:23.259374+0000","last_clean":"2023-07-18T03:06:23.259374+0000","last_became_active":"2023-07-18T03:05:03.880391+0000","last_became_peered":"2023-07-18T03:05:03.880391+0000","last_unstale":"2023-07-18T03:06:23.259374+0000","last_undegraded":"2023-07-18T03:06:23.259374+0000","last_fullsized":"2023-07-18T03:06:23.259374+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:11:20.452097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:30.992774+0000","last_change":"2023-07-18T03:04:58.848771+0000","last_active":"2023-07-18T03:05:30.992774+0000","last_peered":"2023-07-18T03:05:30.992774+0000","last_clean":"2023-07-18T03:05:30.992774+0000","last_became_active":"2023-07-18T03:04:58.848405+0000","last_became_peered":"2023-07-18T03:04:58.848405+0000","last_unstale":"2023-07-18T03:05:30.992774+0000","last_undegraded":"2023-07-18T03:05:30.992774+0000","last_fullsized":"2023-07-18T03:05:30.992774+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:21:40.139702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.090125+0000","last_change":"2023-07-18T03:04:59.805749+0000","last_active":"2023-07-18T03:05:24.090125+0000","last_peered":"2023-07-18T03:05:24.090125+0000","last_clean":"2023-07-18T03:05:24.090125+0000","last_became_active":"2023-07-18T03:04:59.805388+0000","last_became_peered":"2023-07-18T03:04:59.805388+0000","last_unstale":"2023-07-18T03:05:24.090125+0000","last_undegraded":"2023-07-18T03:05:24.090125+0000","last_fullsized":"2023-07-18T03:05:24.090125+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:12:30.269632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.258828+0000","last_change":"2023-07-18T03:05:03.868819+0000","last_active":"2023-07-18T03:06:23.258828+0000","last_peered":"2023-07-18T03:06:23.258828+0000","last_clean":"2023-07-18T03:06:23.258828+0000","last_became_active":"2023-07-18T03:05:03.868387+0000","last_became_peered":"2023-07-18T03:05:03.868387+0000","last_unstale":"2023-07-18T03:06:23.258828+0000","last_undegraded":"2023-07-18T03:06:23.258828+0000","last_fullsized":"2023-07-18T03:06:23.258828+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:53:41.719909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.090445+0000","last_change":"2023-07-18T03:04:58.856084+0000","last_active":"2023-07-18T03:05:24.090445+0000","last_peered":"2023-07-18T03:05:24.090445+0000","last_clean":"2023-07-18T03:05:24.090445+0000","last_became_active":"2023-07-18T03:04:58.855768+0000","last_became_peered":"2023-07-18T03:04:58.855768+0000","last_unstale":"2023-07-18T03:05:24.090445+0000","last_undegraded":"2023-07-18T03:05:24.090445+0000","last_fullsized":"2023-07-18T03:05:24.090445+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:28:05.975256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.265432+0000","last_change":"2023-07-18T03:04:59.809512+0000","last_active":"2023-07-18T03:06:23.265432+0000","last_peered":"2023-07-18T03:06:23.265432+0000","last_clean":"2023-07-18T03:06:23.265432+0000","last_became_active":"2023-07-18T03:04:59.808112+0000","last_became_peered":"2023-07-18T03:04:59.808112+0000","last_unstale":"2023-07-18T03:06:23.265432+0000","last_undegraded":"2023-07-18T03:06:23.265432+0000","last_fullsized":"2023-07-18T03:06:23.265432+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:26:56.291603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.259007+0000","last_change":"2023-07-18T03:05:03.874728+0000","last_active":"2023-07-18T03:06:23.259007+0000","last_peered":"2023-07-18T03:06:23.259007+0000","last_clean":"2023-07-18T03:06:23.259007+0000","last_became_active":"2023-07-18T03:05:03.874262+0000","last_became_peered":"2023-07-18T03:05:03.874262+0000","last_unstale":"2023-07-18T03:06:23.259007+0000","last_undegraded":"2023-07-18T03:06:23.259007+0000","last_fullsized":"2023-07-18T03:06:23.259007+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:22:05.085389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:30.990070+0000","last_change":"2023-07-18T03:04:58.849140+0000","last_active":"2023-07-18T03:05:30.990070+0000","last_peered":"2023-07-18T03:05:30.990070+0000","last_clean":"2023-07-18T03:05:30.990070+0000","last_became_active":"2023-07-18T03:04:58.848693+0000","last_became_peered":"2023-07-18T03:04:58.848693+0000","last_unstale":"2023-07-18T03:05:30.990070+0000","last_undegraded":"2023-07-18T03:05:30.990070+0000","last_fullsized":"2023-07-18T03:05:30.990070+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:43:52.238305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089728+0000","last_change":"2023-07-18T03:04:59.807248+0000","last_active":"2023-07-18T03:05:24.089728+0000","last_peered":"2023-07-18T03:05:24.089728+0000","last_clean":"2023-07-18T03:05:24.089728+0000","last_became_active":"2023-07-18T03:04:59.806807+0000","last_became_peered":"2023-07-18T03:04:59.806807+0000","last_unstale":"2023-07-18T03:05:24.089728+0000","last_undegraded":"2023-07-18T03:05:24.089728+0000","last_fullsized":"2023-07-18T03:05:24.089728+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:35:50.471833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.258830+0000","last_change":"2023-07-18T03:05:03.869075+0000","last_active":"2023-07-18T03:06:23.258830+0000","last_peered":"2023-07-18T03:06:23.258830+0000","last_clean":"2023-07-18T03:06:23.258830+0000","last_became_active":"2023-07-18T03:05:03.868653+0000","last_became_peered":"2023-07-18T03:05:03.868653+0000","last_unstale":"2023-07-18T03:06:23.258830+0000","last_undegraded":"2023-07-18T03:06:23.258830+0000","last_fullsized":"2023-07-18T03:06:23.258830+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:20:02.501750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.937411+0000","last_change":"2023-07-18T03:04:58.843879+0000","last_active":"2023-07-18T03:05:23.937411+0000","last_peered":"2023-07-18T03:05:23.937411+0000","last_clean":"2023-07-18T03:05:23.937411+0000","last_became_active":"2023-07-18T03:04:58.843525+0000","last_became_peered":"2023-07-18T03:04:58.843525+0000","last_unstale":"2023-07-18T03:05:23.937411+0000","last_undegraded":"2023-07-18T03:05:23.937411+0000","last_fullsized":"2023-07-18T03:05:23.937411+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:42:05.200711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089568+0000","last_change":"2023-07-18T03:04:59.803597+0000","last_active":"2023-07-18T03:05:24.089568+0000","last_peered":"2023-07-18T03:05:24.089568+0000","last_clean":"2023-07-18T03:05:24.089568+0000","last_became_active":"2023-07-18T03:04:59.803293+0000","last_became_peered":"2023-07-18T03:04:59.803293+0000","last_unstale":"2023-07-18T03:05:24.089568+0000","last_undegraded":"2023-07-18T03:05:24.089568+0000","last_fullsized":"2023-07-18T03:05:24.089568+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:21:34.711198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.260380+0000","last_change":"2023-07-18T03:05:23.944721+0000","last_active":"2023-07-18T03:06:23.260380+0000","last_peered":"2023-07-18T03:06:23.260380+0000","last_clean":"2023-07-18T03:06:23.260380+0000","last_became_active":"2023-07-18T03:05:23.944101+0000","last_became_peered":"2023-07-18T03:05:23.944101+0000","last_unstale":"2023-07-18T03:06:23.260380+0000","last_undegraded":"2023-07-18T03:06:23.260380+0000","last_fullsized":"2023-07-18T03:06:23.260380+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:24:08.674518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":84,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:06:19.064874+0000","last_change":"2023-07-18T03:04:58.847990+0000","last_active":"2023-07-18T03:06:19.064874+0000","last_peered":"2023-07-18T03:06:19.064874+0000","last_clean":"2023-07-18T03:06:19.064874+0000","last_became_active":"2023-07-18T03:04:58.847634+0000","last_became_peered":"2023-07-18T03:04:58.847634+0000","last_unstale":"2023-07-18T03:06:19.064874+0000","last_undegraded":"2023-07-18T03:06:19.064874+0000","last_fullsized":"2023-07-18T03:06:19.064874+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:03:43.456118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":52,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.938391+0000","last_change":"2023-07-18T03:04:59.797076+0000","last_active":"2023-07-18T03:05:23.938391+0000","last_peered":"2023-07-18T03:05:23.938391+0000","last_clean":"2023-07-18T03:05:23.938391+0000","last_became_active":"2023-07-18T03:04:59.796753+0000","last_became_peered":"2023-07-18T03:04:59.796753+0000","last_unstale":"2023-07-18T03:05:23.938391+0000","last_undegraded":"2023-07-18T03:05:23.938391+0000","last_fullsized":"2023-07-18T03:05:23.938391+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:28:30.559043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.260343+0000","last_change":"2023-07-18T03:05:03.881793+0000","last_active":"2023-07-18T03:06:23.260343+0000","last_peered":"2023-07-18T03:06:23.260343+0000","last_clean":"2023-07-18T03:06:23.260343+0000","last_became_active":"2023-07-18T03:05:03.881341+0000","last_became_peered":"2023-07-18T03:05:03.881341+0000","last_unstale":"2023-07-18T03:06:23.260343+0000","last_undegraded":"2023-07-18T03:06:23.260343+0000","last_fullsized":"2023-07-18T03:06:23.260343+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:24:17.604475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:30.990979+0000","last_change":"2023-07-18T03:04:58.846732+0000","last_active":"2023-07-18T03:05:30.990979+0000","last_peered":"2023-07-18T03:05:30.990979+0000","last_clean":"2023-07-18T03:05:30.990979+0000","last_became_active":"2023-07-18T03:04:58.846321+0000","last_became_peered":"2023-07-18T03:04:58.846321+0000","last_unstale":"2023-07-18T03:05:30.990979+0000","last_undegraded":"2023-07-18T03:05:30.990979+0000","last_fullsized":"2023-07-18T03:05:30.990979+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:23:54.252356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.094233+0000","last_change":"2023-07-18T03:05:24.093277+0000","last_active":"2023-07-18T03:05:24.094233+0000","last_peered":"2023-07-18T03:05:24.094233+0000","last_clean":"2023-07-18T03:05:24.094233+0000","last_became_active":"2023-07-18T03:05:24.092842+0000","last_became_peered":"2023-07-18T03:05:24.092842+0000","last_unstale":"2023-07-18T03:05:24.094233+0000","last_undegraded":"2023-07-18T03:05:24.094233+0000","last_fullsized":"2023-07-18T03:05:24.094233+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:39:40.953742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.942246+0000","last_change":"2023-07-18T03:05:03.880782+0000","last_active":"2023-07-18T03:05:23.942246+0000","last_peered":"2023-07-18T03:05:23.942246+0000","last_clean":"2023-07-18T03:05:23.942246+0000","last_became_active":"2023-07-18T03:05:03.880362+0000","last_became_peered":"2023-07-18T03:05:03.880362+0000","last_unstale":"2023-07-18T03:05:23.942246+0000","last_undegraded":"2023-07-18T03:05:23.942246+0000","last_fullsized":"2023-07-18T03:05:23.942246+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:19:49.336092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:30.991863+0000","last_change":"2023-07-18T03:04:58.846095+0000","last_active":"2023-07-18T03:05:30.991863+0000","last_peered":"2023-07-18T03:05:30.991863+0000","last_clean":"2023-07-18T03:05:30.991863+0000","last_became_active":"2023-07-18T03:04:58.845242+0000","last_became_peered":"2023-07-18T03:04:58.845242+0000","last_unstale":"2023-07-18T03:05:30.991863+0000","last_undegraded":"2023-07-18T03:05:30.991863+0000","last_fullsized":"2023-07-18T03:05:30.991863+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:51:23.745763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089587+0000","last_change":"2023-07-18T03:04:59.803024+0000","last_active":"2023-07-18T03:05:24.089587+0000","last_peered":"2023-07-18T03:05:24.089587+0000","last_clean":"2023-07-18T03:05:24.089587+0000","last_became_active":"2023-07-18T03:04:59.802687+0000","last_became_peered":"2023-07-18T03:04:59.802687+0000","last_unstale":"2023-07-18T03:05:24.089587+0000","last_undegraded":"2023-07-18T03:05:24.089587+0000","last_fullsized":"2023-07-18T03:05:24.089587+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:14:46.616130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.264826+0000","last_change":"2023-07-18T03:05:03.872690+0000","last_active":"2023-07-18T03:06:23.264826+0000","last_peered":"2023-07-18T03:06:23.264826+0000","last_clean":"2023-07-18T03:06:23.264826+0000","last_became_active":"2023-07-18T03:05:03.872187+0000","last_became_peered":"2023-07-18T03:05:03.872187+0000","last_unstale":"2023-07-18T03:06:23.264826+0000","last_undegraded":"2023-07-18T03:06:23.264826+0000","last_fullsized":"2023-07-18T03:06:23.264826+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:45:16.836607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089675+0000","last_change":"2023-07-18T03:04:58.842877+0000","last_active":"2023-07-18T03:05:24.089675+0000","last_peered":"2023-07-18T03:05:24.089675+0000","last_clean":"2023-07-18T03:05:24.089675+0000","last_became_active":"2023-07-18T03:04:58.842516+0000","last_became_peered":"2023-07-18T03:04:58.842516+0000","last_unstale":"2023-07-18T03:05:24.089675+0000","last_undegraded":"2023-07-18T03:05:24.089675+0000","last_fullsized":"2023-07-18T03:05:24.089675+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:55:08.128143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.933854+0000","last_change":"2023-07-18T03:04:59.807592+0000","last_active":"2023-07-18T03:05:23.933854+0000","last_peered":"2023-07-18T03:05:23.933854+0000","last_clean":"2023-07-18T03:05:23.933854+0000","last_became_active":"2023-07-18T03:04:59.807276+0000","last_became_peered":"2023-07-18T03:04:59.807276+0000","last_unstale":"2023-07-18T03:05:23.933854+0000","last_undegraded":"2023-07-18T03:05:23.933854+0000","last_fullsized":"2023-07-18T03:05:23.933854+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:58:29.880849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089882+0000","last_change":"2023-07-18T03:05:03.876355+0000","last_active":"2023-07-18T03:05:24.089882+0000","last_peered":"2023-07-18T03:05:24.089882+0000","last_clean":"2023-07-18T03:05:24.089882+0000","last_became_active":"2023-07-18T03:05:03.875845+0000","last_became_peered":"2023-07-18T03:05:03.875845+0000","last_unstale":"2023-07-18T03:05:24.089882+0000","last_undegraded":"2023-07-18T03:05:24.089882+0000","last_fullsized":"2023-07-18T03:05:24.089882+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:01:54.675024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":79,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:06:20.065510+0000","last_change":"2023-07-18T03:04:58.840389+0000","last_active":"2023-07-18T03:06:20.065510+0000","last_peered":"2023-07-18T03:06:20.065510+0000","last_clean":"2023-07-18T03:06:20.065510+0000","last_became_active":"2023-07-18T03:04:58.839974+0000","last_became_peered":"2023-07-18T03:04:58.839974+0000","last_unstale":"2023-07-18T03:06:20.065510+0000","last_undegraded":"2023-07-18T03:06:20.065510+0000","last_fullsized":"2023-07-18T03:06:20.065510+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:21:20.010136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":50,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.090769+0000","last_change":"2023-07-18T03:04:59.807681+0000","last_active":"2023-07-18T03:05:24.090769+0000","last_peered":"2023-07-18T03:05:24.090769+0000","last_clean":"2023-07-18T03:05:24.090769+0000","last_became_active":"2023-07-18T03:04:59.807365+0000","last_became_peered":"2023-07-18T03:04:59.807365+0000","last_unstale":"2023-07-18T03:05:24.090769+0000","last_undegraded":"2023-07-18T03:05:24.090769+0000","last_fullsized":"2023-07-18T03:05:24.090769+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:35:09.523428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.092631+0000","last_change":"2023-07-18T03:05:24.091413+0000","last_active":"2023-07-18T03:05:24.092631+0000","last_peered":"2023-07-18T03:05:24.092631+0000","last_clean":"2023-07-18T03:05:24.092631+0000","last_became_active":"2023-07-18T03:05:24.090815+0000","last_became_peered":"2023-07-18T03:05:24.090815+0000","last_unstale":"2023-07-18T03:05:24.092631+0000","last_undegraded":"2023-07-18T03:05:24.092631+0000","last_fullsized":"2023-07-18T03:05:24.092631+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:18:39.070780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":29,"reported_epoch":89,"state":"peering","last_fresh":"2023-07-18T03:05:29.985893+0000","last_change":"2023-07-18T03:06:23.256454+0000","last_active":"2023-07-18T03:06:23.256454+0000","last_peered":"2023-07-18T03:05:29.985893+0000","last_clean":"2023-07-18T03:05:29.985893+0000","last_became_active":"2023-07-18T03:04:58.846366+0000","last_became_peered":"2023-07-18T03:04:58.846366+0000","last_unstale":"2023-07-18T03:05:29.985893+0000","last_undegraded":"2023-07-18T03:05:29.985893+0000","last_fullsized":"2023-07-18T03:05:29.985893+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2,4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.265282+0000","last_change":"2023-07-18T03:05:24.092870+0000","last_active":"2023-07-18T03:06:23.265282+0000","last_peered":"2023-07-18T03:06:23.265282+0000","last_clean":"2023-07-18T03:06:23.265282+0000","last_became_active":"2023-07-18T03:05:24.092474+0000","last_became_peered":"2023-07-18T03:05:24.092474+0000","last_unstale":"2023-07-18T03:06:23.265282+0000","last_undegraded":"2023-07-18T03:06:23.265282+0000","last_fullsized":"2023-07-18T03:06:23.265282+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:48:14.130393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.259166+0000","last_change":"2023-07-18T03:05:03.874444+0000","last_active":"2023-07-18T03:06:23.259166+0000","last_peered":"2023-07-18T03:06:23.259166+0000","last_clean":"2023-07-18T03:06:23.259166+0000","last_became_active":"2023-07-18T03:05:03.873737+0000","last_became_peered":"2023-07-18T03:05:03.873737+0000","last_unstale":"2023-07-18T03:06:23.259166+0000","last_undegraded":"2023-07-18T03:06:23.259166+0000","last_fullsized":"2023-07-18T03:06:23.259166+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:30:06.718472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:29.985446+0000","last_change":"2023-07-18T03:04:58.845207+0000","last_active":"2023-07-18T03:05:29.985446+0000","last_peered":"2023-07-18T03:05:29.985446+0000","last_clean":"2023-07-18T03:05:29.985446+0000","last_became_active":"2023-07-18T03:04:58.844895+0000","last_became_peered":"2023-07-18T03:04:58.844895+0000","last_unstale":"2023-07-18T03:05:29.985446+0000","last_undegraded":"2023-07-18T03:05:29.985446+0000","last_fullsized":"2023-07-18T03:05:29.985446+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:16:23.615174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.090081+0000","last_change":"2023-07-18T03:04:59.807075+0000","last_active":"2023-07-18T03:05:24.090081+0000","last_peered":"2023-07-18T03:05:24.090081+0000","last_clean":"2023-07-18T03:05:24.090081+0000","last_became_active":"2023-07-18T03:04:59.806345+0000","last_became_peered":"2023-07-18T03:04:59.806345+0000","last_unstale":"2023-07-18T03:05:24.090081+0000","last_undegraded":"2023-07-18T03:05:24.090081+0000","last_fullsized":"2023-07-18T03:05:24.090081+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:38:38.275381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.091509+0000","last_change":"2023-07-18T03:05:24.091132+0000","last_active":"2023-07-18T03:05:24.091509+0000","last_peered":"2023-07-18T03:05:24.091509+0000","last_clean":"2023-07-18T03:05:24.091509+0000","last_became_active":"2023-07-18T03:05:24.090653+0000","last_became_peered":"2023-07-18T03:05:24.090653+0000","last_unstale":"2023-07-18T03:05:24.091509+0000","last_undegraded":"2023-07-18T03:05:24.091509+0000","last_fullsized":"2023-07-18T03:05:24.091509+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:37:46.585074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"89'32","reported_seq":166,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:06:21.079566+0000","last_change":"2023-07-18T03:04:58.844865+0000","last_active":"2023-07-18T03:06:21.079566+0000","last_peered":"2023-07-18T03:06:21.079566+0000","last_clean":"2023-07-18T03:06:21.079566+0000","last_became_active":"2023-07-18T03:04:58.844547+0000","last_became_peered":"2023-07-18T03:04:58.844547+0000","last_unstale":"2023-07-18T03:06:21.079566+0000","last_undegraded":"2023-07-18T03:06:21.079566+0000","last_fullsized":"2023-07-18T03:06:21.079566+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:01:48.220367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60837,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":108,"num_read_kb":113,"num_write":33,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.937865+0000","last_change":"2023-07-18T03:04:59.803024+0000","last_active":"2023-07-18T03:05:23.937865+0000","last_peered":"2023-07-18T03:05:23.937865+0000","last_clean":"2023-07-18T03:05:23.937865+0000","last_became_active":"2023-07-18T03:04:59.802705+0000","last_became_peered":"2023-07-18T03:04:59.802705+0000","last_unstale":"2023-07-18T03:05:23.937865+0000","last_undegraded":"2023-07-18T03:05:23.937865+0000","last_fullsized":"2023-07-18T03:05:23.937865+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:51:13.019557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.090087+0000","last_change":"2023-07-18T03:05:03.876009+0000","last_active":"2023-07-18T03:05:24.090087+0000","last_peered":"2023-07-18T03:05:24.090087+0000","last_clean":"2023-07-18T03:05:24.090087+0000","last_became_active":"2023-07-18T03:05:03.875328+0000","last_became_peered":"2023-07-18T03:05:03.875328+0000","last_unstale":"2023-07-18T03:05:24.090087+0000","last_undegraded":"2023-07-18T03:05:24.090087+0000","last_fullsized":"2023-07-18T03:05:24.090087+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:29:03.036222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.260105+0000","last_change":"2023-07-18T03:04:58.846573+0000","last_active":"2023-07-18T03:06:23.260105+0000","last_peered":"2023-07-18T03:06:23.260105+0000","last_clean":"2023-07-18T03:06:23.260105+0000","last_became_active":"2023-07-18T03:04:58.846271+0000","last_became_peered":"2023-07-18T03:04:58.846271+0000","last_unstale":"2023-07-18T03:06:23.260105+0000","last_undegraded":"2023-07-18T03:06:23.260105+0000","last_fullsized":"2023-07-18T03:06:23.260105+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:45:26.155515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.091068+0000","last_change":"2023-07-18T03:04:59.801552+0000","last_active":"2023-07-18T03:05:24.091068+0000","last_peered":"2023-07-18T03:05:24.091068+0000","last_clean":"2023-07-18T03:05:24.091068+0000","last_became_active":"2023-07-18T03:04:59.801224+0000","last_became_peered":"2023-07-18T03:04:59.801224+0000","last_unstale":"2023-07-18T03:05:24.091068+0000","last_undegraded":"2023-07-18T03:05:24.091068+0000","last_fullsized":"2023-07-18T03:05:24.091068+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:43:54.484133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.941638+0000","last_change":"2023-07-18T03:05:03.881125+0000","last_active":"2023-07-18T03:05:23.941638+0000","last_peered":"2023-07-18T03:05:23.941638+0000","last_clean":"2023-07-18T03:05:23.941638+0000","last_became_active":"2023-07-18T03:05:03.880723+0000","last_became_peered":"2023-07-18T03:05:03.880723+0000","last_unstale":"2023-07-18T03:05:23.941638+0000","last_undegraded":"2023-07-18T03:05:23.941638+0000","last_fullsized":"2023-07-18T03:05:23.941638+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:10:01.286537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:29.984442+0000","last_change":"2023-07-18T03:04:58.847118+0000","last_active":"2023-07-18T03:05:29.984442+0000","last_peered":"2023-07-18T03:05:29.984442+0000","last_clean":"2023-07-18T03:05:29.984442+0000","last_became_active":"2023-07-18T03:04:58.846805+0000","last_became_peered":"2023-07-18T03:04:58.846805+0000","last_unstale":"2023-07-18T03:05:29.984442+0000","last_undegraded":"2023-07-18T03:05:29.984442+0000","last_fullsized":"2023-07-18T03:05:29.984442+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:43:05.061239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.256529+0000","last_change":"2023-07-18T03:04:59.803914+0000","last_active":"2023-07-18T03:06:23.256529+0000","last_peered":"2023-07-18T03:06:23.256529+0000","last_clean":"2023-07-18T03:06:23.256529+0000","last_became_active":"2023-07-18T03:04:59.803589+0000","last_became_peered":"2023-07-18T03:04:59.803589+0000","last_unstale":"2023-07-18T03:06:23.256529+0000","last_undegraded":"2023-07-18T03:06:23.256529+0000","last_fullsized":"2023-07-18T03:06:23.256529+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:05:59.470709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.934482+0000","last_change":"2023-07-18T03:05:03.877787+0000","last_active":"2023-07-18T03:05:23.934482+0000","last_peered":"2023-07-18T03:05:23.934482+0000","last_clean":"2023-07-18T03:05:23.934482+0000","last_became_active":"2023-07-18T03:05:03.877449+0000","last_became_peered":"2023-07-18T03:05:03.877449+0000","last_unstale":"2023-07-18T03:05:23.934482+0000","last_undegraded":"2023-07-18T03:05:23.934482+0000","last_fullsized":"2023-07-18T03:05:23.934482+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:25:39.108283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":85,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.258959+0000","last_change":"2023-07-18T03:04:58.844252+0000","last_active":"2023-07-18T03:06:23.258959+0000","last_peered":"2023-07-18T03:06:23.258959+0000","last_clean":"2023-07-18T03:06:23.258959+0000","last_became_active":"2023-07-18T03:04:58.843798+0000","last_became_peered":"2023-07-18T03:04:58.843798+0000","last_unstale":"2023-07-18T03:06:23.258959+0000","last_undegraded":"2023-07-18T03:06:23.258959+0000","last_fullsized":"2023-07-18T03:06:23.258959+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:33:22.080789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.256239+0000","last_change":"2023-07-18T03:04:59.806274+0000","last_active":"2023-07-18T03:06:23.256239+0000","last_peered":"2023-07-18T03:06:23.256239+0000","last_clean":"2023-07-18T03:06:23.256239+0000","last_became_active":"2023-07-18T03:04:59.805876+0000","last_became_peered":"2023-07-18T03:04:59.805876+0000","last_unstale":"2023-07-18T03:06:23.256239+0000","last_undegraded":"2023-07-18T03:06:23.256239+0000","last_fullsized":"2023-07-18T03:06:23.256239+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:54:12.988681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.091503+0000","last_change":"2023-07-18T03:05:03.876783+0000","last_active":"2023-07-18T03:05:24.091503+0000","last_peered":"2023-07-18T03:05:24.091503+0000","last_clean":"2023-07-18T03:05:24.091503+0000","last_became_active":"2023-07-18T03:05:03.876385+0000","last_became_peered":"2023-07-18T03:05:03.876385+0000","last_unstale":"2023-07-18T03:05:24.091503+0000","last_undegraded":"2023-07-18T03:05:24.091503+0000","last_fullsized":"2023-07-18T03:05:24.091503+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:20:54.077330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:30.992923+0000","last_change":"2023-07-18T03:04:58.851107+0000","last_active":"2023-07-18T03:05:30.992923+0000","last_peered":"2023-07-18T03:05:30.992923+0000","last_clean":"2023-07-18T03:05:30.992923+0000","last_became_active":"2023-07-18T03:04:58.850848+0000","last_became_peered":"2023-07-18T03:04:58.850848+0000","last_unstale":"2023-07-18T03:05:30.992923+0000","last_undegraded":"2023-07-18T03:05:30.992923+0000","last_fullsized":"2023-07-18T03:05:30.992923+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:17:37.985831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089918+0000","last_change":"2023-07-18T03:04:59.803683+0000","last_active":"2023-07-18T03:05:24.089918+0000","last_peered":"2023-07-18T03:05:24.089918+0000","last_clean":"2023-07-18T03:05:24.089918+0000","last_became_active":"2023-07-18T03:04:59.803365+0000","last_became_peered":"2023-07-18T03:04:59.803365+0000","last_unstale":"2023-07-18T03:05:24.089918+0000","last_undegraded":"2023-07-18T03:05:24.089918+0000","last_fullsized":"2023-07-18T03:05:24.089918+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:36:27.161561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089944+0000","last_change":"2023-07-18T03:05:03.875856+0000","last_active":"2023-07-18T03:05:24.089944+0000","last_peered":"2023-07-18T03:05:24.089944+0000","last_clean":"2023-07-18T03:05:24.089944+0000","last_became_active":"2023-07-18T03:05:03.875426+0000","last_became_peered":"2023-07-18T03:05:03.875426+0000","last_unstale":"2023-07-18T03:05:24.089944+0000","last_undegraded":"2023-07-18T03:05:24.089944+0000","last_fullsized":"2023-07-18T03:05:24.089944+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:00:07.610976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:31.993869+0000","last_change":"2023-07-18T03:04:58.844102+0000","last_active":"2023-07-18T03:05:31.993869+0000","last_peered":"2023-07-18T03:05:31.993869+0000","last_clean":"2023-07-18T03:05:31.993869+0000","last_became_active":"2023-07-18T03:04:58.843740+0000","last_became_peered":"2023-07-18T03:04:58.843740+0000","last_unstale":"2023-07-18T03:05:31.993869+0000","last_undegraded":"2023-07-18T03:05:31.993869+0000","last_fullsized":"2023-07-18T03:05:31.993869+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:28:24.928761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.091161+0000","last_change":"2023-07-18T03:05:24.091161+0000","last_active":"2023-07-18T03:05:24.091161+0000","last_peered":"2023-07-18T03:05:24.091161+0000","last_clean":"2023-07-18T03:05:24.091161+0000","last_became_active":"2023-07-18T03:05:24.090631+0000","last_became_peered":"2023-07-18T03:05:24.090631+0000","last_unstale":"2023-07-18T03:05:24.091161+0000","last_undegraded":"2023-07-18T03:05:24.091161+0000","last_fullsized":"2023-07-18T03:05:24.091161+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:35:29.133177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.092328+0000","last_change":"2023-07-18T03:05:24.092328+0000","last_active":"2023-07-18T03:05:24.092328+0000","last_peered":"2023-07-18T03:05:24.092328+0000","last_clean":"2023-07-18T03:05:24.092328+0000","last_became_active":"2023-07-18T03:05:24.091839+0000","last_became_peered":"2023-07-18T03:05:24.091839+0000","last_unstale":"2023-07-18T03:05:24.092328+0000","last_undegraded":"2023-07-18T03:05:24.092328+0000","last_fullsized":"2023-07-18T03:05:24.092328+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:26:09.308228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.934182+0000","last_change":"2023-07-18T03:04:58.852395+0000","last_active":"2023-07-18T03:05:23.934182+0000","last_peered":"2023-07-18T03:05:23.934182+0000","last_clean":"2023-07-18T03:05:23.934182+0000","last_became_active":"2023-07-18T03:04:58.852063+0000","last_became_peered":"2023-07-18T03:04:58.852063+0000","last_unstale":"2023-07-18T03:05:23.934182+0000","last_undegraded":"2023-07-18T03:05:23.934182+0000","last_fullsized":"2023-07-18T03:05:23.934182+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:39:12.747039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.092486+0000","last_change":"2023-07-18T03:05:24.092486+0000","last_active":"2023-07-18T03:05:24.092486+0000","last_peered":"2023-07-18T03:05:24.092486+0000","last_clean":"2023-07-18T03:05:24.092486+0000","last_became_active":"2023-07-18T03:05:24.092189+0000","last_became_peered":"2023-07-18T03:05:24.092189+0000","last_unstale":"2023-07-18T03:05:24.092486+0000","last_undegraded":"2023-07-18T03:05:24.092486+0000","last_fullsized":"2023-07-18T03:05:24.092486+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:04:51.159197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.091135+0000","last_change":"2023-07-18T03:05:03.868851+0000","last_active":"2023-07-18T03:05:24.091135+0000","last_peered":"2023-07-18T03:05:24.091135+0000","last_clean":"2023-07-18T03:05:24.091135+0000","last_became_active":"2023-07-18T03:05:03.868433+0000","last_became_peered":"2023-07-18T03:05:03.868433+0000","last_unstale":"2023-07-18T03:05:24.091135+0000","last_undegraded":"2023-07-18T03:05:24.091135+0000","last_fullsized":"2023-07-18T03:05:24.091135+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:43:44.650042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.256217+0000","last_change":"2023-07-18T03:04:58.848059+0000","last_active":"2023-07-18T03:06:23.256217+0000","last_peered":"2023-07-18T03:06:23.256217+0000","last_clean":"2023-07-18T03:06:23.256217+0000","last_became_active":"2023-07-18T03:04:58.847700+0000","last_became_peered":"2023-07-18T03:04:58.847700+0000","last_unstale":"2023-07-18T03:06:23.256217+0000","last_undegraded":"2023-07-18T03:06:23.256217+0000","last_fullsized":"2023-07-18T03:06:23.256217+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:12:45.404497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.939939+0000","last_change":"2023-07-18T03:05:23.939939+0000","last_active":"2023-07-18T03:05:23.939939+0000","last_peered":"2023-07-18T03:05:23.939939+0000","last_clean":"2023-07-18T03:05:23.939939+0000","last_became_active":"2023-07-18T03:05:23.939607+0000","last_became_peered":"2023-07-18T03:05:23.939607+0000","last_unstale":"2023-07-18T03:05:23.939939+0000","last_undegraded":"2023-07-18T03:05:23.939939+0000","last_fullsized":"2023-07-18T03:05:23.939939+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:05:30.317897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.091495+0000","last_change":"2023-07-18T03:05:03.886724+0000","last_active":"2023-07-18T03:05:24.091495+0000","last_peered":"2023-07-18T03:05:24.091495+0000","last_clean":"2023-07-18T03:05:24.091495+0000","last_became_active":"2023-07-18T03:05:03.886172+0000","last_became_peered":"2023-07-18T03:05:03.886172+0000","last_unstale":"2023-07-18T03:05:24.091495+0000","last_undegraded":"2023-07-18T03:05:24.091495+0000","last_fullsized":"2023-07-18T03:05:24.091495+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:42:53.119172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089781+0000","last_change":"2023-07-18T03:04:59.804273+0000","last_active":"2023-07-18T03:05:24.089781+0000","last_peered":"2023-07-18T03:05:24.089781+0000","last_clean":"2023-07-18T03:05:24.089781+0000","last_became_active":"2023-07-18T03:04:59.803987+0000","last_became_peered":"2023-07-18T03:04:59.803987+0000","last_unstale":"2023-07-18T03:05:24.089781+0000","last_undegraded":"2023-07-18T03:05:24.089781+0000","last_fullsized":"2023-07-18T03:05:24.089781+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:47:40.524108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":80,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:06:20.065577+0000","last_change":"2023-07-18T03:04:58.846381+0000","last_active":"2023-07-18T03:06:20.065577+0000","last_peered":"2023-07-18T03:06:20.065577+0000","last_clean":"2023-07-18T03:06:20.065577+0000","last_became_active":"2023-07-18T03:04:58.845757+0000","last_became_peered":"2023-07-18T03:04:58.845757+0000","last_unstale":"2023-07-18T03:06:20.065577+0000","last_undegraded":"2023-07-18T03:06:20.065577+0000","last_fullsized":"2023-07-18T03:06:20.065577+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:41:53.865792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":50,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.256735+0000","last_change":"2023-07-18T03:05:03.873626+0000","last_active":"2023-07-18T03:06:23.256735+0000","last_peered":"2023-07-18T03:06:23.256735+0000","last_clean":"2023-07-18T03:06:23.256735+0000","last_became_active":"2023-07-18T03:05:03.872965+0000","last_became_peered":"2023-07-18T03:05:03.872965+0000","last_unstale":"2023-07-18T03:06:23.256735+0000","last_undegraded":"2023-07-18T03:06:23.256735+0000","last_fullsized":"2023-07-18T03:06:23.256735+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:39:06.793540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089960+0000","last_change":"2023-07-18T03:04:59.801153+0000","last_active":"2023-07-18T03:05:24.089960+0000","last_peered":"2023-07-18T03:05:24.089960+0000","last_clean":"2023-07-18T03:05:24.089960+0000","last_became_active":"2023-07-18T03:04:59.800798+0000","last_became_peered":"2023-07-18T03:04:59.800798+0000","last_unstale":"2023-07-18T03:05:24.089960+0000","last_undegraded":"2023-07-18T03:05:24.089960+0000","last_fullsized":"2023-07-18T03:05:24.089960+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:55:12.782843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.259089+0000","last_change":"2023-07-18T03:04:58.846047+0000","last_active":"2023-07-18T03:06:23.259089+0000","last_peered":"2023-07-18T03:06:23.259089+0000","last_clean":"2023-07-18T03:06:23.259089+0000","last_became_active":"2023-07-18T03:04:58.845678+0000","last_became_peered":"2023-07-18T03:04:58.845678+0000","last_unstale":"2023-07-18T03:06:23.259089+0000","last_undegraded":"2023-07-18T03:06:23.259089+0000","last_fullsized":"2023-07-18T03:06:23.259089+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:49:09.335016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":382,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.255938+0000","last_change":"2023-07-18T03:03:54.168837+0000","last_active":"2023-07-18T03:06:23.255938+0000","last_peered":"2023-07-18T03:06:23.255938+0000","last_clean":"2023-07-18T03:06:23.255938+0000","last_became_active":"2023-07-18T03:03:54.168298+0000","last_became_peered":"2023-07-18T03:03:54.168298+0000","last_unstale":"2023-07-18T03:06:23.255938+0000","last_undegraded":"2023-07-18T03:06:23.255938+0000","last_fullsized":"2023-07-18T03:06:23.255938+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:00:43.469168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:00:43.469168+0000","last_clean_scrub_stamp":"2023-07-18T03:00:43.469168+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:23:12.775517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089157+0000","last_change":"2023-07-18T03:05:03.886611+0000","last_active":"2023-07-18T03:05:24.089157+0000","last_peered":"2023-07-18T03:05:24.089157+0000","last_clean":"2023-07-18T03:05:24.089157+0000","last_became_active":"2023-07-18T03:05:03.886099+0000","last_became_peered":"2023-07-18T03:05:03.886099+0000","last_unstale":"2023-07-18T03:05:24.089157+0000","last_undegraded":"2023-07-18T03:05:24.089157+0000","last_fullsized":"2023-07-18T03:05:24.089157+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:22:16.652890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.090718+0000","last_change":"2023-07-18T03:04:59.804478+0000","last_active":"2023-07-18T03:05:24.090718+0000","last_peered":"2023-07-18T03:05:24.090718+0000","last_clean":"2023-07-18T03:05:24.090718+0000","last_became_active":"2023-07-18T03:04:59.804162+0000","last_became_peered":"2023-07-18T03:04:59.804162+0000","last_unstale":"2023-07-18T03:05:24.090718+0000","last_undegraded":"2023-07-18T03:05:24.090718+0000","last_fullsized":"2023-07-18T03:05:24.090718+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:30:38.735612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":30,"reported_epoch":89,"state":"peering","last_fresh":"2023-07-18T03:05:30.991249+0000","last_change":"2023-07-18T03:06:23.256542+0000","last_active":"2023-07-18T03:06:23.256542+0000","last_peered":"2023-07-18T03:05:30.991249+0000","last_clean":"2023-07-18T03:05:30.991249+0000","last_became_active":"2023-07-18T03:04:58.847385+0000","last_became_peered":"2023-07-18T03:04:58.847385+0000","last_unstale":"2023-07-18T03:05:30.991249+0000","last_undegraded":"2023-07-18T03:05:30.991249+0000","last_fullsized":"2023-07-18T03:05:30.991249+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,5,8],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.941642+0000","last_change":"2023-07-18T03:05:03.872788+0000","last_active":"2023-07-18T03:05:23.941642+0000","last_peered":"2023-07-18T03:05:23.941642+0000","last_clean":"2023-07-18T03:05:23.941642+0000","last_became_active":"2023-07-18T03:05:03.872343+0000","last_became_peered":"2023-07-18T03:05:03.872343+0000","last_unstale":"2023-07-18T03:05:23.941642+0000","last_undegraded":"2023-07-18T03:05:23.941642+0000","last_fullsized":"2023-07-18T03:05:23.941642+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:05:33.222286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:29.985672+0000","last_change":"2023-07-18T03:04:58.855261+0000","last_active":"2023-07-18T03:05:29.985672+0000","last_peered":"2023-07-18T03:05:29.985672+0000","last_clean":"2023-07-18T03:05:29.985672+0000","last_became_active":"2023-07-18T03:04:58.854921+0000","last_became_peered":"2023-07-18T03:04:58.854921+0000","last_unstale":"2023-07-18T03:05:29.985672+0000","last_undegraded":"2023-07-18T03:05:29.985672+0000","last_fullsized":"2023-07-18T03:05:29.985672+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:08:26.349218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.091711+0000","last_change":"2023-07-18T03:04:59.804706+0000","last_active":"2023-07-18T03:05:24.091711+0000","last_peered":"2023-07-18T03:05:24.091711+0000","last_clean":"2023-07-18T03:05:24.091711+0000","last_became_active":"2023-07-18T03:04:59.804445+0000","last_became_peered":"2023-07-18T03:04:59.804445+0000","last_unstale":"2023-07-18T03:05:24.091711+0000","last_undegraded":"2023-07-18T03:05:24.091711+0000","last_fullsized":"2023-07-18T03:05:24.091711+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:06:24.461712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089557+0000","last_change":"2023-07-18T03:05:03.886065+0000","last_active":"2023-07-18T03:05:24.089557+0000","last_peered":"2023-07-18T03:05:24.089557+0000","last_clean":"2023-07-18T03:05:24.089557+0000","last_became_active":"2023-07-18T03:05:03.885625+0000","last_became_peered":"2023-07-18T03:05:03.885625+0000","last_unstale":"2023-07-18T03:05:24.089557+0000","last_undegraded":"2023-07-18T03:05:24.089557+0000","last_fullsized":"2023-07-18T03:05:24.089557+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:35:20.557825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'5","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:35.250443+0000","last_change":"2023-07-18T03:04:58.849246+0000","last_active":"2023-07-18T03:05:35.250443+0000","last_peered":"2023-07-18T03:05:35.250443+0000","last_clean":"2023-07-18T03:05:35.250443+0000","last_became_active":"2023-07-18T03:04:58.848888+0000","last_became_peered":"2023-07-18T03:04:58.848888+0000","last_unstale":"2023-07-18T03:05:35.250443+0000","last_undegraded":"2023-07-18T03:05:35.250443+0000","last_fullsized":"2023-07-18T03:05:35.250443+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:54:11.818147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.090505+0000","last_change":"2023-07-18T03:04:59.804561+0000","last_active":"2023-07-18T03:05:24.090505+0000","last_peered":"2023-07-18T03:05:24.090505+0000","last_clean":"2023-07-18T03:05:24.090505+0000","last_became_active":"2023-07-18T03:04:59.803659+0000","last_became_peered":"2023-07-18T03:04:59.803659+0000","last_unstale":"2023-07-18T03:05:24.090505+0000","last_undegraded":"2023-07-18T03:05:24.090505+0000","last_fullsized":"2023-07-18T03:05:24.090505+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:59:59.312017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.258965+0000","last_change":"2023-07-18T03:05:03.871418+0000","last_active":"2023-07-18T03:06:23.258965+0000","last_peered":"2023-07-18T03:06:23.258965+0000","last_clean":"2023-07-18T03:06:23.258965+0000","last_became_active":"2023-07-18T03:05:03.871054+0000","last_became_peered":"2023-07-18T03:05:03.871054+0000","last_unstale":"2023-07-18T03:06:23.258965+0000","last_undegraded":"2023-07-18T03:06:23.258965+0000","last_fullsized":"2023-07-18T03:06:23.258965+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:46:35.095446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.259519+0000","last_change":"2023-07-18T03:04:58.850505+0000","last_active":"2023-07-18T03:06:23.259519+0000","last_peered":"2023-07-18T03:06:23.259519+0000","last_clean":"2023-07-18T03:06:23.259519+0000","last_became_active":"2023-07-18T03:04:58.850234+0000","last_became_peered":"2023-07-18T03:04:58.850234+0000","last_unstale":"2023-07-18T03:06:23.259519+0000","last_undegraded":"2023-07-18T03:06:23.259519+0000","last_fullsized":"2023-07-18T03:06:23.259519+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:03:00.163608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.265586+0000","last_change":"2023-07-18T03:04:59.807348+0000","last_active":"2023-07-18T03:06:23.265586+0000","last_peered":"2023-07-18T03:06:23.265586+0000","last_clean":"2023-07-18T03:06:23.265586+0000","last_became_active":"2023-07-18T03:04:59.807013+0000","last_became_peered":"2023-07-18T03:04:59.807013+0000","last_unstale":"2023-07-18T03:06:23.265586+0000","last_undegraded":"2023-07-18T03:06:23.265586+0000","last_fullsized":"2023-07-18T03:06:23.265586+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:10:16.107970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.259356+0000","last_change":"2023-07-18T03:05:03.886773+0000","last_active":"2023-07-18T03:06:23.259356+0000","last_peered":"2023-07-18T03:06:23.259356+0000","last_clean":"2023-07-18T03:06:23.259356+0000","last_became_active":"2023-07-18T03:05:03.886259+0000","last_became_peered":"2023-07-18T03:05:03.886259+0000","last_unstale":"2023-07-18T03:06:23.259356+0000","last_undegraded":"2023-07-18T03:06:23.259356+0000","last_fullsized":"2023-07-18T03:06:23.259356+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:15:59.986776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:29.985267+0000","last_change":"2023-07-18T03:04:58.855746+0000","last_active":"2023-07-18T03:05:29.985267+0000","last_peered":"2023-07-18T03:05:29.985267+0000","last_clean":"2023-07-18T03:05:29.985267+0000","last_became_active":"2023-07-18T03:04:58.855366+0000","last_became_peered":"2023-07-18T03:04:58.855366+0000","last_unstale":"2023-07-18T03:05:29.985267+0000","last_undegraded":"2023-07-18T03:05:29.985267+0000","last_fullsized":"2023-07-18T03:05:29.985267+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:52:56.120702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.936549+0000","last_change":"2023-07-18T03:04:59.800321+0000","last_active":"2023-07-18T03:05:23.936549+0000","last_peered":"2023-07-18T03:05:23.936549+0000","last_clean":"2023-07-18T03:05:23.936549+0000","last_became_active":"2023-07-18T03:04:59.800018+0000","last_became_peered":"2023-07-18T03:04:59.800018+0000","last_unstale":"2023-07-18T03:05:23.936549+0000","last_undegraded":"2023-07-18T03:05:23.936549+0000","last_fullsized":"2023-07-18T03:05:23.936549+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:12:47.433577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.934504+0000","last_change":"2023-07-18T03:05:03.877541+0000","last_active":"2023-07-18T03:05:23.934504+0000","last_peered":"2023-07-18T03:05:23.934504+0000","last_clean":"2023-07-18T03:05:23.934504+0000","last_became_active":"2023-07-18T03:05:03.877164+0000","last_became_peered":"2023-07-18T03:05:03.877164+0000","last_unstale":"2023-07-18T03:05:23.934504+0000","last_undegraded":"2023-07-18T03:05:23.934504+0000","last_fullsized":"2023-07-18T03:05:23.934504+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:52:54.205495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:29.985557+0000","last_change":"2023-07-18T03:04:58.845643+0000","last_active":"2023-07-18T03:05:29.985557+0000","last_peered":"2023-07-18T03:05:29.985557+0000","last_clean":"2023-07-18T03:05:29.985557+0000","last_became_active":"2023-07-18T03:04:58.845254+0000","last_became_peered":"2023-07-18T03:04:58.845254+0000","last_unstale":"2023-07-18T03:05:29.985557+0000","last_undegraded":"2023-07-18T03:05:29.985557+0000","last_fullsized":"2023-07-18T03:05:29.985557+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:42:32.740369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.941481+0000","last_change":"2023-07-18T03:04:59.803679+0000","last_active":"2023-07-18T03:05:23.941481+0000","last_peered":"2023-07-18T03:05:23.941481+0000","last_clean":"2023-07-18T03:05:23.941481+0000","last_became_active":"2023-07-18T03:04:59.802841+0000","last_became_peered":"2023-07-18T03:04:59.802841+0000","last_unstale":"2023-07-18T03:05:23.941481+0000","last_undegraded":"2023-07-18T03:05:23.941481+0000","last_fullsized":"2023-07-18T03:05:23.941481+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:14:53.042557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.259148+0000","last_change":"2023-07-18T03:05:03.874093+0000","last_active":"2023-07-18T03:06:23.259148+0000","last_peered":"2023-07-18T03:06:23.259148+0000","last_clean":"2023-07-18T03:06:23.259148+0000","last_became_active":"2023-07-18T03:05:03.873710+0000","last_became_peered":"2023-07-18T03:05:03.873710+0000","last_unstale":"2023-07-18T03:06:23.259148+0000","last_undegraded":"2023-07-18T03:06:23.259148+0000","last_fullsized":"2023-07-18T03:06:23.259148+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:47:55.346705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.259709+0000","last_change":"2023-07-18T03:04:58.846686+0000","last_active":"2023-07-18T03:06:23.259709+0000","last_peered":"2023-07-18T03:06:23.259709+0000","last_clean":"2023-07-18T03:06:23.259709+0000","last_became_active":"2023-07-18T03:04:58.846371+0000","last_became_peered":"2023-07-18T03:04:58.846371+0000","last_unstale":"2023-07-18T03:06:23.259709+0000","last_undegraded":"2023-07-18T03:06:23.259709+0000","last_fullsized":"2023-07-18T03:06:23.259709+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:37:39.352218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089999+0000","last_change":"2023-07-18T03:04:59.811917+0000","last_active":"2023-07-18T03:05:24.089999+0000","last_peered":"2023-07-18T03:05:24.089999+0000","last_clean":"2023-07-18T03:05:24.089999+0000","last_became_active":"2023-07-18T03:04:59.811551+0000","last_became_peered":"2023-07-18T03:04:59.811551+0000","last_unstale":"2023-07-18T03:05:24.089999+0000","last_undegraded":"2023-07-18T03:05:24.089999+0000","last_fullsized":"2023-07-18T03:05:24.089999+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:32:42.153516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.259384+0000","last_change":"2023-07-18T03:05:03.886072+0000","last_active":"2023-07-18T03:06:23.259384+0000","last_peered":"2023-07-18T03:06:23.259384+0000","last_clean":"2023-07-18T03:06:23.259384+0000","last_became_active":"2023-07-18T03:05:03.885652+0000","last_became_peered":"2023-07-18T03:05:03.885652+0000","last_unstale":"2023-07-18T03:06:23.259384+0000","last_undegraded":"2023-07-18T03:06:23.259384+0000","last_fullsized":"2023-07-18T03:06:23.259384+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:51:11.860036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:29.985186+0000","last_change":"2023-07-18T03:04:58.849069+0000","last_active":"2023-07-18T03:05:29.985186+0000","last_peered":"2023-07-18T03:05:29.985186+0000","last_clean":"2023-07-18T03:05:29.985186+0000","last_became_active":"2023-07-18T03:04:58.848744+0000","last_became_peered":"2023-07-18T03:04:58.848744+0000","last_unstale":"2023-07-18T03:05:29.985186+0000","last_undegraded":"2023-07-18T03:05:29.985186+0000","last_fullsized":"2023-07-18T03:05:29.985186+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:06:14.618475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.256479+0000","last_change":"2023-07-18T03:04:59.806976+0000","last_active":"2023-07-18T03:06:23.256479+0000","last_peered":"2023-07-18T03:06:23.256479+0000","last_clean":"2023-07-18T03:06:23.256479+0000","last_became_active":"2023-07-18T03:04:59.805750+0000","last_became_peered":"2023-07-18T03:04:59.805750+0000","last_unstale":"2023-07-18T03:06:23.256479+0000","last_undegraded":"2023-07-18T03:06:23.256479+0000","last_fullsized":"2023-07-18T03:06:23.256479+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:11:01.991604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.090382+0000","last_change":"2023-07-18T03:05:03.871210+0000","last_active":"2023-07-18T03:05:24.090382+0000","last_peered":"2023-07-18T03:05:24.090382+0000","last_clean":"2023-07-18T03:05:24.090382+0000","last_became_active":"2023-07-18T03:05:03.870826+0000","last_became_peered":"2023-07-18T03:05:03.870826+0000","last_unstale":"2023-07-18T03:05:24.090382+0000","last_undegraded":"2023-07-18T03:05:24.090382+0000","last_fullsized":"2023-07-18T03:05:24.090382+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:54:35.402542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.260686+0000","last_change":"2023-07-18T03:04:58.855673+0000","last_active":"2023-07-18T03:06:23.260686+0000","last_peered":"2023-07-18T03:06:23.260686+0000","last_clean":"2023-07-18T03:06:23.260686+0000","last_became_active":"2023-07-18T03:04:58.855338+0000","last_became_peered":"2023-07-18T03:04:58.855338+0000","last_unstale":"2023-07-18T03:06:23.260686+0000","last_undegraded":"2023-07-18T03:06:23.260686+0000","last_fullsized":"2023-07-18T03:06:23.260686+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:00:49.530635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.256926+0000","last_change":"2023-07-18T03:04:59.802488+0000","last_active":"2023-07-18T03:06:23.256926+0000","last_peered":"2023-07-18T03:06:23.256926+0000","last_clean":"2023-07-18T03:06:23.256926+0000","last_became_active":"2023-07-18T03:04:59.802184+0000","last_became_peered":"2023-07-18T03:04:59.802184+0000","last_unstale":"2023-07-18T03:06:23.256926+0000","last_undegraded":"2023-07-18T03:06:23.256926+0000","last_fullsized":"2023-07-18T03:06:23.256926+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:11:16.175238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.264915+0000","last_change":"2023-07-18T03:05:03.881585+0000","last_active":"2023-07-18T03:06:23.264915+0000","last_peered":"2023-07-18T03:06:23.264915+0000","last_clean":"2023-07-18T03:06:23.264915+0000","last_became_active":"2023-07-18T03:05:03.880917+0000","last_became_peered":"2023-07-18T03:05:03.880917+0000","last_unstale":"2023-07-18T03:06:23.264915+0000","last_undegraded":"2023-07-18T03:06:23.264915+0000","last_fullsized":"2023-07-18T03:06:23.264915+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:00:08.510960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089801+0000","last_change":"2023-07-18T03:04:58.852387+0000","last_active":"2023-07-18T03:05:24.089801+0000","last_peered":"2023-07-18T03:05:24.089801+0000","last_clean":"2023-07-18T03:05:24.089801+0000","last_became_active":"2023-07-18T03:04:58.852113+0000","last_became_peered":"2023-07-18T03:04:58.852113+0000","last_unstale":"2023-07-18T03:05:24.089801+0000","last_undegraded":"2023-07-18T03:05:24.089801+0000","last_fullsized":"2023-07-18T03:05:24.089801+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:47:27.148793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.256377+0000","last_change":"2023-07-18T03:04:59.811569+0000","last_active":"2023-07-18T03:06:23.256377+0000","last_peered":"2023-07-18T03:06:23.256377+0000","last_clean":"2023-07-18T03:06:23.256377+0000","last_became_active":"2023-07-18T03:04:59.811229+0000","last_became_peered":"2023-07-18T03:04:59.811229+0000","last_unstale":"2023-07-18T03:06:23.256377+0000","last_undegraded":"2023-07-18T03:06:23.256377+0000","last_fullsized":"2023-07-18T03:06:23.256377+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:45:17.874405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.088825+0000","last_change":"2023-07-18T03:05:03.872530+0000","last_active":"2023-07-18T03:05:24.088825+0000","last_peered":"2023-07-18T03:05:24.088825+0000","last_clean":"2023-07-18T03:05:24.088825+0000","last_became_active":"2023-07-18T03:05:03.871607+0000","last_became_peered":"2023-07-18T03:05:03.871607+0000","last_unstale":"2023-07-18T03:05:24.088825+0000","last_undegraded":"2023-07-18T03:05:24.088825+0000","last_fullsized":"2023-07-18T03:05:24.088825+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:34:34.583319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.091627+0000","last_change":"2023-07-18T03:04:58.847933+0000","last_active":"2023-07-18T03:05:24.091627+0000","last_peered":"2023-07-18T03:05:24.091627+0000","last_clean":"2023-07-18T03:05:24.091627+0000","last_became_active":"2023-07-18T03:04:58.847630+0000","last_became_peered":"2023-07-18T03:04:58.847630+0000","last_unstale":"2023-07-18T03:05:24.091627+0000","last_undegraded":"2023-07-18T03:05:24.091627+0000","last_fullsized":"2023-07-18T03:05:24.091627+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:35:14.343077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.256932+0000","last_change":"2023-07-18T03:04:59.803981+0000","last_active":"2023-07-18T03:06:23.256932+0000","last_peered":"2023-07-18T03:06:23.256932+0000","last_clean":"2023-07-18T03:06:23.256932+0000","last_became_active":"2023-07-18T03:04:59.803692+0000","last_became_peered":"2023-07-18T03:04:59.803692+0000","last_unstale":"2023-07-18T03:06:23.256932+0000","last_undegraded":"2023-07-18T03:06:23.256932+0000","last_fullsized":"2023-07-18T03:06:23.256932+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:53:56.192507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.941648+0000","last_change":"2023-07-18T03:04:59.803770+0000","last_active":"2023-07-18T03:05:23.941648+0000","last_peered":"2023-07-18T03:05:23.941648+0000","last_clean":"2023-07-18T03:05:23.941648+0000","last_became_active":"2023-07-18T03:04:59.803355+0000","last_became_peered":"2023-07-18T03:04:59.803355+0000","last_unstale":"2023-07-18T03:05:23.941648+0000","last_undegraded":"2023-07-18T03:05:23.941648+0000","last_fullsized":"2023-07-18T03:05:23.941648+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:58:39.773442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.259243+0000","last_change":"2023-07-18T03:04:59.808946+0000","last_active":"2023-07-18T03:06:23.259243+0000","last_peered":"2023-07-18T03:06:23.259243+0000","last_clean":"2023-07-18T03:06:23.259243+0000","last_became_active":"2023-07-18T03:04:59.808646+0000","last_became_peered":"2023-07-18T03:04:59.808646+0000","last_unstale":"2023-07-18T03:06:23.259243+0000","last_undegraded":"2023-07-18T03:06:23.259243+0000","last_fullsized":"2023-07-18T03:06:23.259243+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:51:55.300703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.259577+0000","last_change":"2023-07-18T03:04:59.811933+0000","last_active":"2023-07-18T03:06:23.259577+0000","last_peered":"2023-07-18T03:06:23.259577+0000","last_clean":"2023-07-18T03:06:23.259577+0000","last_became_active":"2023-07-18T03:04:59.811620+0000","last_became_peered":"2023-07-18T03:04:59.811620+0000","last_unstale":"2023-07-18T03:06:23.259577+0000","last_undegraded":"2023-07-18T03:06:23.259577+0000","last_fullsized":"2023-07-18T03:06:23.259577+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:25:22.077867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.265182+0000","last_change":"2023-07-18T03:04:59.803942+0000","last_active":"2023-07-18T03:06:23.265182+0000","last_peered":"2023-07-18T03:06:23.265182+0000","last_clean":"2023-07-18T03:06:23.265182+0000","last_became_active":"2023-07-18T03:04:59.803576+0000","last_became_peered":"2023-07-18T03:04:59.803576+0000","last_unstale":"2023-07-18T03:06:23.265182+0000","last_undegraded":"2023-07-18T03:06:23.265182+0000","last_fullsized":"2023-07-18T03:06:23.265182+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:53:38.163912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.091559+0000","last_change":"2023-07-18T03:04:59.802594+0000","last_active":"2023-07-18T03:05:24.091559+0000","last_peered":"2023-07-18T03:05:24.091559+0000","last_clean":"2023-07-18T03:05:24.091559+0000","last_became_active":"2023-07-18T03:04:59.802215+0000","last_became_peered":"2023-07-18T03:04:59.802215+0000","last_unstale":"2023-07-18T03:05:24.091559+0000","last_undegraded":"2023-07-18T03:05:24.091559+0000","last_fullsized":"2023-07-18T03:05:24.091559+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:18:26.515762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.942105+0000","last_change":"2023-07-18T03:04:59.803519+0000","last_active":"2023-07-18T03:05:23.942105+0000","last_peered":"2023-07-18T03:05:23.942105+0000","last_clean":"2023-07-18T03:05:23.942105+0000","last_became_active":"2023-07-18T03:04:59.803170+0000","last_became_peered":"2023-07-18T03:04:59.803170+0000","last_unstale":"2023-07-18T03:05:23.942105+0000","last_undegraded":"2023-07-18T03:05:23.942105+0000","last_fullsized":"2023-07-18T03:05:23.942105+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:57:22.069668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.261369+0000","last_change":"2023-07-18T03:04:59.813494+0000","last_active":"2023-07-18T03:06:23.261369+0000","last_peered":"2023-07-18T03:06:23.261369+0000","last_clean":"2023-07-18T03:06:23.261369+0000","last_became_active":"2023-07-18T03:04:59.813175+0000","last_became_peered":"2023-07-18T03:04:59.813175+0000","last_unstale":"2023-07-18T03:06:23.261369+0000","last_undegraded":"2023-07-18T03:06:23.261369+0000","last_fullsized":"2023-07-18T03:06:23.261369+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:19:26.469876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.090682+0000","last_change":"2023-07-18T03:04:59.806597+0000","last_active":"2023-07-18T03:05:24.090682+0000","last_peered":"2023-07-18T03:05:24.090682+0000","last_clean":"2023-07-18T03:05:24.090682+0000","last_became_active":"2023-07-18T03:04:59.806220+0000","last_became_peered":"2023-07-18T03:04:59.806220+0000","last_unstale":"2023-07-18T03:05:24.090682+0000","last_undegraded":"2023-07-18T03:05:24.090682+0000","last_fullsized":"2023-07-18T03:05:24.090682+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:33:51.675056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.089408+0000","last_change":"2023-07-18T03:04:59.809051+0000","last_active":"2023-07-18T03:05:24.089408+0000","last_peered":"2023-07-18T03:05:24.089408+0000","last_clean":"2023-07-18T03:05:24.089408+0000","last_became_active":"2023-07-18T03:04:59.808783+0000","last_became_peered":"2023-07-18T03:04:59.808783+0000","last_unstale":"2023-07-18T03:05:24.089408+0000","last_undegraded":"2023-07-18T03:05:24.089408+0000","last_fullsized":"2023-07-18T03:05:24.089408+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:11:53.631126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.090344+0000","last_change":"2023-07-18T03:04:59.805741+0000","last_active":"2023-07-18T03:05:24.090344+0000","last_peered":"2023-07-18T03:05:24.090344+0000","last_clean":"2023-07-18T03:05:24.090344+0000","last_became_active":"2023-07-18T03:04:59.805412+0000","last_became_peered":"2023-07-18T03:04:59.805412+0000","last_unstale":"2023-07-18T03:05:24.090344+0000","last_undegraded":"2023-07-18T03:05:24.090344+0000","last_fullsized":"2023-07-18T03:05:24.090344+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:36:12.237809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.090649+0000","last_change":"2023-07-18T03:04:59.812052+0000","last_active":"2023-07-18T03:05:24.090649+0000","last_peered":"2023-07-18T03:05:24.090649+0000","last_clean":"2023-07-18T03:05:24.090649+0000","last_became_active":"2023-07-18T03:04:59.811756+0000","last_became_peered":"2023-07-18T03:04:59.811756+0000","last_unstale":"2023-07-18T03:05:24.090649+0000","last_undegraded":"2023-07-18T03:05:24.090649+0000","last_fullsized":"2023-07-18T03:05:24.090649+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:28:03.874575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:24.091423+0000","last_change":"2023-07-18T03:04:59.803009+0000","last_active":"2023-07-18T03:05:24.091423+0000","last_peered":"2023-07-18T03:05:24.091423+0000","last_clean":"2023-07-18T03:05:24.091423+0000","last_became_active":"2023-07-18T03:04:59.802687+0000","last_became_peered":"2023-07-18T03:04:59.802687+0000","last_unstale":"2023-07-18T03:05:24.091423+0000","last_undegraded":"2023-07-18T03:05:24.091423+0000","last_fullsized":"2023-07-18T03:05:24.091423+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:28:34.815336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-18T03:05:23.942144+0000","last_change":"2023-07-18T03:04:59.795562+0000","last_active":"2023-07-18T03:05:23.942144+0000","last_peered":"2023-07-18T03:05:23.942144+0000","last_clean":"2023-07-18T03:05:23.942144+0000","last_became_active":"2023-07-18T03:04:59.795241+0000","last_became_peered":"2023-07-18T03:04:59.795241+0000","last_unstale":"2023-07-18T03:05:23.942144+0000","last_undegraded":"2023-07-18T03:05:23.942144+0000","last_fullsized":"2023-07-18T03:05:23.942144+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:43:39.736724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281146,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64741,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":328,"num_read_kb":232,"num_write":154,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":194223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449117,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":970752,"data_stored":487693,"data_compressed":17573,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.56000000000000005,"15min":0.55300000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.52200000000000002,"15min":0.51400000000000001},"min":{"1min":0.41099999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.50600000000000001}]},{"osd":1,"last update":"Tue Jul 18 03:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.64300000000000002,"15min":0.63500000000000001},"min":{"1min":0.47799999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.85299999999999998,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.624,"15min":0.621},"min":{"1min":0.47099999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.82299999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.60999999999999999}]},{"osd":2,"last update":"Tue Jul 18 03:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.48299999999999998,"15min":0.47399999999999998},"min":{"1min":0.35199999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.71399999999999997,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.48999999999999999,"15min":0.47599999999999998},"min":{"1min":0.36699999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.47899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":4,"last update":"Tue Jul 18 03:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.66500000000000004,"15min":0.65100000000000002},"min":{"1min":0.53200000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.64200000000000002,"15min":0.63500000000000001},"min":{"1min":0.48999999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.84799999999999998,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.66400000000000003}]},{"osd":5,"last update":"Tue Jul 18 03:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.622,"15min":0.621},"min":{"1min":0.45400000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.82099999999999995,"5min":1.375,"15min":1.375},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.61399999999999999,"15min":0.59299999999999997},"min":{"1min":0.378,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.63900000000000001}]},{"osd":6,"last update":"Tue Jul 18 03:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.68400000000000005,"15min":0.67000000000000004},"min":{"1min":0.52200000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.70499999999999996,"15min":0.69299999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.622}]},{"osd":7,"last update":"Tue Jul 18 03:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.79200000000000004,"15min":0.78000000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1319999999999999,"5min":1.248,"15min":1.248},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.76900000000000002,"15min":0.76300000000000001},"min":{"1min":0.56499999999999995,"5min":0.502,"15min":0.502},"max":{"1min":1.024,"5min":1.288,"15min":1.288},"last":0.70499999999999996}]},{"osd":8,"last update":"Tue Jul 18 03:06:00 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.78900000000000003}]},{"osd":9,"last update":"Tue Jul 18 03:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.78300000000000003,"15min":0.76800000000000002},"min":{"1min":0.66700000000000004,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.145,"5min":1.304,"15min":1.304},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.81899999999999995,"15min":0.80400000000000005},"min":{"1min":0.63400000000000001,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1659999999999999,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.81200000000000006}]},{"osd":10,"last update":"Tue Jul 18 03:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.78200000000000003,"15min":0.77200000000000002},"min":{"1min":0.57999999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.054,"5min":1.163,"15min":1.163},"last":0.746},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.79900000000000004,"15min":0.78900000000000003},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.026,"5min":1.496,"15min":1.496},"last":0.71699999999999997}]}]},{"osd":10,"up_from":71,"seq":304942678050,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1088,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1114112,"data_stored":1135750,"data_compressed":53332,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.58699999999999997}]},{"osd":1,"last update":"Tue Jul 18 03:05:56 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47999999999999998,"15min":0.48199999999999998},"min":{"1min":0.34399999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.51000000000000001,"15min":0.51600000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.58899999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.70999999999999996}]},{"osd":2,"last update":"Tue Jul 18 03:05:56 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.63600000000000001,"15min":0.63},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82599999999999996,"5min":0.87,"15min":0.87},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64300000000000002,"15min":0.64200000000000002},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.752,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.65500000000000003}]},{"osd":3,"last update":"Tue Jul 18 03:05:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.68300000000000005,"15min":0.68899999999999995},"min":{"1min":0.48699999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.62,"5min":0.65700000000000003,"15min":0.66300000000000003},"min":{"1min":0.442,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82799999999999996,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.75600000000000001}]},{"osd":4,"last update":"Tue Jul 18 03:05:56 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.60899999999999999,"15min":0.61699999999999999},"min":{"1min":0.37,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72099999999999997,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77400000000000002,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.81299999999999994}]},{"osd":5,"last update":"Tue Jul 18 03:05:56 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.71499999999999997,"15min":0.72099999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.80000000000000004,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.75,"15min":0.75},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.86599999999999999,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.89500000000000002}]},{"osd":6,"last update":"Tue Jul 18 03:05:56 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.75600000000000001,"15min":0.76200000000000001},"min":{"1min":0.58599999999999997,"5min":0.501,"15min":0.501},"max":{"1min":0.88800000000000001,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.878},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.80200000000000005,"15min":0.80400000000000005},"min":{"1min":0.53400000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.94999999999999996}]},{"osd":7,"last update":"Tue Jul 18 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.55800000000000005}]},{"osd":8,"last update":"Tue Jul 18 03:05:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.80100000000000005,"15min":0.80800000000000005},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.91400000000000003,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.81799999999999995,"15min":0.82499999999999996},"min":{"1min":0.60699999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":0.95999999999999996,"5min":1.032,"15min":1.032},"last":0.93400000000000005}]},{"osd":9,"last update":"Tue Jul 18 03:05:56 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.84399999999999997,"15min":0.84599999999999997},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.85499999999999998,"15min":0.85799999999999998},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.018,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.996}]},{"osd":11,"last update":"Tue Jul 18 03:06:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.82699999999999996,"15min":0.83199999999999996},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.96599999999999997,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.745,"5min":0.80000000000000004,"15min":0.80900000000000005},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.92800000000000005,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.92500000000000004}]}]},{"osd":9,"up_from":66,"seq":283467841573,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":991232,"data_stored":492025,"data_compressed":17573,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:05:34 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.54900000000000004,"15min":0.55800000000000005},"min":{"1min":0.35799999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.69699999999999995,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.36899999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.84399999999999997,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.38300000000000001}]},{"osd":1,"last update":"Tue Jul 18 03:05:34 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.71899999999999997,"15min":0.73099999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0169999999999999,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.72999999999999998,"15min":0.74199999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.91500000000000004,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.54600000000000004}]},{"osd":2,"last update":"Tue Jul 18 03:05:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.70999999999999996,"15min":0.71799999999999997},"min":{"1min":0.52000000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.85699999999999998,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.74299999999999999,"15min":0.75},"min":{"1min":0.48499999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.98099999999999998,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.49399999999999999}]},{"osd":3,"last update":"Tue Jul 18 03:05:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.67000000000000004,"15min":0.68799999999999994},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88300000000000001,"5min":1.038,"15min":1.038},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.64900000000000002,"15min":0.66300000000000003},"min":{"1min":0.38700000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.90100000000000002,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.36699999999999999}]},{"osd":4,"last update":"Tue Jul 18 03:05:34 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.81699999999999995,"15min":0.83299999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.97299999999999998,"5min":1.47,"15min":1.47},"last":0.437},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.80700000000000005,"15min":0.82699999999999996},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.95699999999999996,"5min":1.619,"15min":1.619},"last":0.47599999999999998}]},{"osd":5,"last update":"Tue Jul 18 03:05:34 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.81999999999999995,"15min":0.82999999999999996},"min":{"1min":0.53400000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.95499999999999996,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.628},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.80600000000000005,"15min":0.81699999999999995},"min":{"1min":0.499,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1080000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.53500000000000003}]},{"osd":6,"last update":"Tue Jul 18 03:04:30 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.64300000000000002}]},{"osd":7,"last update":"Tue Jul 18 03:05:34 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.89100000000000001,"15min":0.91000000000000003},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0600000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.94599999999999995,"15min":0.93999999999999995},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":4.4009999999999998,"5min":4.4009999999999998,"15min":4.4009999999999998},"last":0.60699999999999998}]},{"osd":8,"last update":"Tue Jul 18 03:05:34 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.90900000000000003,"15min":0.92700000000000005},"min":{"1min":0.63800000000000001,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.077,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.93700000000000006,"15min":0.95499999999999996},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.0880000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.65400000000000003}]},{"osd":10,"last update":"Tue Jul 18 03:05:51 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.88400000000000001,"15min":0.90200000000000002},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.96299999999999997,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.91300000000000003,"15min":0.93899999999999995},"min":{"1min":0.63700000000000001,"5min":0.63,"15min":0.63},"max":{"1min":0.94799999999999995,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.64200000000000002}]},{"osd":11,"last update":"Tue Jul 18 03:06:05 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.66300000000000003}]}]},{"osd":8,"up_from":58,"seq":249108103210,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":987136,"data_stored":487849,"data_compressed":17573,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:06:13 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.69499999999999995,"15min":0.64100000000000001},"min":{"1min":0.433,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":7.0359999999999996,"5min":7.0359999999999996,"15min":7.0359999999999996},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58499999999999996,"15min":0.57099999999999995},"min":{"1min":0.443,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91500000000000004,"5min":2.883,"15min":2.883},"last":0.47599999999999998}]},{"osd":1,"last update":"Tue Jul 18 03:06:13 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.65400000000000003,"15min":0.63200000000000001},"min":{"1min":0.45300000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.879,"5min":3.3340000000000001,"15min":3.3340000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.64100000000000001,"15min":0.60599999999999998},"min":{"1min":0.48999999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.95099999999999996,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":0.76600000000000001}]},{"osd":2,"last update":"Tue Jul 18 03:06:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.503,"15min":0.46700000000000003},"min":{"1min":0.38300000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.0900000000000001,"5min":3.2850000000000001,"15min":3.2850000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.52200000000000002,"15min":0.498},"min":{"1min":0.39700000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.78900000000000003,"5min":2.012,"15min":2.012},"last":0.57799999999999996}]},{"osd":3,"last update":"Tue Jul 18 03:06:13 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.64200000000000002,"15min":0.622},"min":{"1min":0.45800000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.91000000000000003,"5min":2.9100000000000001,"15min":2.9100000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.66800000000000004,"15min":0.64900000000000002},"min":{"1min":0.50800000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.82699999999999996,"5min":3.6059999999999999,"15min":3.6059999999999999},"last":0.623}]},{"osd":4,"last update":"Tue Jul 18 03:06:13 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.65800000000000003,"15min":0.63200000000000001},"min":{"1min":0.42899999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.006,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":0.75},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.71499999999999997,"15min":0.69399999999999995},"min":{"1min":0.50600000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.99099999999999999,"5min":3.3650000000000002,"15min":3.3650000000000002},"last":0.72999999999999998}]},{"osd":5,"last update":"Tue Jul 18 03:06:13 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.626,"15min":0.59599999999999997},"min":{"1min":0.40899999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.97799999999999998,"5min":2.4279999999999999,"15min":2.4279999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.63800000000000001,"15min":0.60699999999999998},"min":{"1min":0.46300000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0549999999999999,"5min":2.7410000000000001,"15min":2.7410000000000001},"last":0.60199999999999998}]},{"osd":6,"last update":"Tue Jul 18 03:06:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.753,"15min":0.72299999999999998},"min":{"1min":0.56499999999999995,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.038,"5min":3.5779999999999998,"15min":3.5779999999999998},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.76400000000000001,"15min":0.74099999999999999},"min":{"1min":0.628,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0249999999999999,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":0.80700000000000005}]},{"osd":7,"last update":"Tue Jul 18 03:06:13 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.76200000000000001,"15min":0.72999999999999998},"min":{"1min":0.57599999999999996,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.167,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.76000000000000001,"15min":0.72799999999999998},"min":{"1min":0.64200000000000002,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1100000000000001,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":0.78100000000000003}]},{"osd":9,"last update":"Tue Jul 18 03:05:36 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.81000000000000005,"15min":0.78200000000000003},"min":{"1min":0.60599999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":3.544,"5min":3.544,"15min":3.544},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.81000000000000005,"15min":0.79000000000000004},"min":{"1min":0.61899999999999999,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.1230000000000002,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":0.83099999999999996}]},{"osd":10,"last update":"Tue Jul 18 03:05:54 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.82299999999999995,"15min":0.81499999999999995},"min":{"1min":0.61499999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0249999999999999,"5min":2.8570000000000002,"15min":2.8570000000000002},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.871,"5min":0.879,"15min":0.88100000000000001},"min":{"1min":0.59099999999999997,"5min":0.502,"15min":0.502},"max":{"1min":1.02,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":0.92400000000000004}]},{"osd":11,"last update":"Tue Jul 18 03:06:03 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.86799999999999999}]}]},{"osd":7,"up_from":51,"seq":219043332142,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":1036288,"data_stored":1083661,"data_compressed":53332,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.55500000000000005,"15min":0.54300000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.495},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59999999999999998,"15min":0.60299999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82599999999999996,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.44500000000000001}]},{"osd":1,"last update":"Tue Jul 18 03:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42199999999999999,"15min":0.41999999999999998},"min":{"1min":0.32900000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.55800000000000005,"5min":0.745,"15min":0.745},"last":0.439},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.439,"15min":0.433},"min":{"1min":0.33300000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.628,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.60399999999999998}]},{"osd":2,"last update":"Tue Jul 18 03:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58399999999999996,"15min":0.56999999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.80300000000000005,"5min":1.111,"15min":1.111},"last":0.623},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59599999999999997,"15min":0.58299999999999996},"min":{"1min":0.42399999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.752,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.56200000000000006}]},{"osd":3,"last update":"Tue Jul 18 03:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.628,"15min":0.60399999999999998},"min":{"1min":0.41099999999999998,"5min":0.38,"15min":0.38},"max":{"1min":2.6669999999999998,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.63400000000000001,"15min":0.64500000000000002},"min":{"1min":0.42399999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.76800000000000002,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.749}]},{"osd":4,"last update":"Tue Jul 18 03:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.48899999999999999,"15min":0.48399999999999999},"min":{"1min":0.39000000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.62,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54900000000000004,"15min":0.54800000000000004},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.53100000000000003}]},{"osd":5,"last update":"Tue Jul 18 03:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.66800000000000004,"15min":0.65000000000000002},"min":{"1min":0.51400000000000001,"5min":0.501,"15min":0.501},"max":{"1min":0.82999999999999996,"5min":1.077,"15min":1.077},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.66200000000000003,"15min":0.65400000000000003},"min":{"1min":0.52000000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.80800000000000005,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.63500000000000001}]},{"osd":6,"last update":"Tue Jul 18 03:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.67100000000000004,"15min":0.67300000000000004},"min":{"1min":0.46999999999999997,"5min":0.435,"15min":0.435},"max":{"1min":0.83699999999999997,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.71699999999999997,"15min":0.70299999999999996},"min":{"1min":0.49399999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.76300000000000001}]},{"osd":8,"last update":"Tue Jul 18 03:06:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.70599999999999996,"15min":0.69199999999999995},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.95699999999999996,"5min":1.022,"15min":1.022},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.66700000000000004,"15min":0.64900000000000002},"min":{"1min":0.52700000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.83099999999999996,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.69799999999999995}]},{"osd":9,"last update":"Tue Jul 18 03:05:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.76600000000000001,"15min":0.76400000000000001},"min":{"1min":0.56499999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.78700000000000003}]},{"osd":10,"last update":"Tue Jul 18 03:05:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.72099999999999997,"15min":0.72399999999999998},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.94099999999999995,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.75600000000000001,"15min":0.754},"min":{"1min":0.61499999999999999,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.92900000000000005,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.73899999999999999}]},{"osd":11,"last update":"Tue Jul 18 03:06:04 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.83499999999999996}]}]},{"osd":6,"up_from":46,"seq":197568495667,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":483105,"data_compressed":17573,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.53400000000000003,"15min":0.56299999999999994},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.58199999999999996,"5min":3.1579999999999999,"15min":3.1579999999999999},"last":1.48},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.53400000000000003,"15min":0.55000000000000004},"min":{"1min":0.35699999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.64000000000000001,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":1.617}]},{"osd":1,"last update":"Tue Jul 18 03:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.627,"15min":0.60299999999999998},"min":{"1min":0.38800000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.88600000000000001,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":1.2909999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.72799999999999998,"15min":0.74099999999999999},"min":{"1min":0.495,"5min":0.44,"15min":0.44},"max":{"1min":0.90500000000000003,"5min":3.0369999999999999,"15min":3.0369999999999999},"last":1.226}]},{"osd":2,"last update":"Tue Jul 18 03:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.65700000000000003,"15min":0.65600000000000003},"min":{"1min":0.48299999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":1.4450000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.64700000000000002,"15min":0.65300000000000002},"min":{"1min":0.48399999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.94299999999999995,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":1.375}]},{"osd":3,"last update":"Tue Jul 18 03:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.56499999999999995,"15min":0.57299999999999995},"min":{"1min":0.38800000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.69899999999999995,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":1.464},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.71199999999999997,"15min":0.748},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.748,"5min":3.226,"15min":3.226},"last":1.546}]},{"osd":4,"last update":"Tue Jul 18 03:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.44600000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.92500000000000004,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":1.355},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.70199999999999996,"15min":0.69899999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.92700000000000005,"5min":1.01,"15min":1.01},"last":1.4259999999999999}]},{"osd":5,"last update":"Tue Jul 18 03:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.81200000000000006,"15min":0.84499999999999997},"min":{"1min":0.59199999999999997,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.88600000000000001,"5min":3.125,"15min":3.125},"last":1.1060000000000001},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.76600000000000001,"15min":0.78200000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.98699999999999999,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":1.4039999999999999}]},{"osd":7,"last update":"Tue Jul 18 03:05:46 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.83399999999999996,"15min":0.83999999999999997},"min":{"1min":0.61399999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.97599999999999998,"5min":3.0870000000000002,"15min":3.0870000000000002},"last":1.1619999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.752,"15min":0.73399999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.01,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":1.5309999999999999}]},{"osd":8,"last update":"Tue Jul 18 03:06:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.85099999999999998,"15min":0.876},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0640000000000001,"5min":3.1909999999999998,"15min":3.1909999999999998},"last":1.0569999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.99099999999999999,"5min":1.046,"15min":1.046},"last":1.498}]},{"osd":9,"last update":"Tue Jul 18 03:05:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.754,"15min":0.76400000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.97899999999999998,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":1.5629999999999999},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.84899999999999998,"15min":0.85599999999999998},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.008,"5min":1.137,"15min":1.137},"last":1.2749999999999999}]},{"osd":10,"last update":"Tue Jul 18 03:05:54 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.876,"15min":0.878},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.044,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":1.1930000000000001},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.82099999999999995,"15min":0.81899999999999995},"min":{"1min":0.66900000000000004,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0429999999999999,"5min":1.083,"15min":1.083},"last":1.579}]},{"osd":11,"last update":"Tue Jul 18 03:06:05 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":1.6000000000000001},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":1.516}]}]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":1032192,"data_stored":543852,"data_compressed":17573,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.55100000000000005,"15min":0.52400000000000002},"min":{"1min":0.37,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.96299999999999997,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.54400000000000004,"15min":0.52400000000000002},"min":{"1min":0.35299999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.93100000000000005,"5min":1.421,"15min":1.421},"last":0.52800000000000002}]},{"osd":2,"last update":"Tue Jul 18 03:05:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.56599999999999995,"15min":0.51900000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":0.62},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.56899999999999995,"15min":0.54400000000000004},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.70399999999999996}]},{"osd":3,"last update":"Tue Jul 18 03:06:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.622,"15min":0.59999999999999998},"min":{"1min":0.378,"5min":0.312,"15min":0.312},"max":{"1min":1.014,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.627,"15min":0.627},"min":{"1min":0.47999999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.026,"5min":1.504,"15min":1.504},"last":0.68500000000000005}]},{"osd":4,"last update":"Tue Jul 18 03:05:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.45900000000000002,"15min":0.40699999999999997},"min":{"1min":0.317,"5min":0.215,"15min":0.215},"max":{"1min":1.784,"5min":1.784,"15min":1.784},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.47699999999999998,"15min":0.42499999999999999},"min":{"1min":0.34599999999999997,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.76700000000000002,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.83599999999999997}]},{"osd":5,"last update":"Tue Jul 18 03:06:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.61199999999999999,"15min":0.53100000000000003},"min":{"1min":0.57999999999999996,"5min":0.375,"15min":0.375},"max":{"1min":0.85299999999999998,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.745,"5min":0.623,"15min":0.56699999999999995},"min":{"1min":0.59399999999999997,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.872,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.76200000000000001}]},{"osd":6,"last update":"Tue Jul 18 03:05:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.64900000000000002,"15min":0.60499999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.4299999999999999,"5min":2.6600000000000001,"15min":2.6600000000000001},"last":0.87},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.65200000000000002,"15min":0.63800000000000001},"min":{"1min":0.46400000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.94799999999999995,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.72499999999999998}]},{"osd":7,"last update":"Tue Jul 18 03:05:51 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.59599999999999997,"15min":0.56399999999999995},"min":{"1min":0.38100000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.67400000000000004,"15min":0.65600000000000003},"min":{"1min":0.5,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.5409999999999999,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":0.90000000000000002}]},{"osd":8,"last update":"Tue Jul 18 03:06:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77500000000000002,"15min":0.78200000000000003},"min":{"1min":0.52900000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.98399999999999999,"5min":2.6419999999999999,"15min":2.6419999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.72199999999999998,"15min":0.68200000000000005},"min":{"1min":0.54000000000000004,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.036,"5min":1.698,"15min":1.698},"last":0.80300000000000005}]},{"osd":9,"last update":"Tue Jul 18 03:05:40 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.85599999999999998,"15min":0.85799999999999998},"min":{"1min":0.59099999999999997,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.458,"5min":2.6789999999999998,"15min":2.6789999999999998},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.76200000000000001,"15min":0.746},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.5409999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.91200000000000003}]},{"osd":10,"last update":"Tue Jul 18 03:05:56 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.78300000000000003,"15min":0.76900000000000002},"min":{"1min":0.57999999999999996,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.77000000000000002,"15min":0.76200000000000001},"min":{"1min":0.56899999999999995,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":0.88500000000000001}]},{"osd":11,"last update":"Tue Jul 18 03:06:01 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.97799999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":991232,"data_stored":484493,"data_compressed":17573,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Jul 18 03:05:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.59599999999999997,"15min":0.55200000000000005},"min":{"1min":0.45900000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.0149999999999999,"5min":1.075,"15min":1.075},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.61099999999999999,"15min":0.56100000000000005},"min":{"1min":0.38100000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.97699999999999998,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.63300000000000001}]},{"osd":2,"last update":"Tue Jul 18 03:05:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.60399999999999998,"15min":0.55200000000000005},"min":{"1min":0.45200000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.873,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.64100000000000001,"15min":0.56799999999999995},"min":{"1min":0.42799999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.88,"5min":1.204,"15min":1.204},"last":0.77600000000000002}]},{"osd":3,"last update":"Tue Jul 18 03:06:00 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.442}]},{"osd":4,"last update":"Tue Jul 18 03:05:40 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.65300000000000002,"15min":0.61299999999999999},"min":{"1min":0.437,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.63500000000000001,"15min":0.60199999999999998},"min":{"1min":0.47399999999999998,"5min":0.433,"15min":0.433},"max":{"1min":0.93600000000000005,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.52600000000000002}]},{"osd":5,"last update":"Tue Jul 18 03:05:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.64700000000000002,"15min":0.63500000000000001},"min":{"1min":0.46899999999999997,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.028,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.68200000000000005,"15min":0.65900000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.998,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.79600000000000004}]},{"osd":6,"last update":"Tue Jul 18 03:05:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59899999999999998,"15min":0.58999999999999997},"min":{"1min":0.439,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.91900000000000004,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.624,"15min":0.59999999999999998},"min":{"1min":0.42099999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.94699999999999995,"5min":1.216,"15min":1.216},"last":0.48599999999999999}]},{"osd":7,"last update":"Tue Jul 18 03:06:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.74299999999999999}]},{"osd":8,"last update":"Tue Jul 18 03:06:05 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.746},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.95999999999999996,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":6.96},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.81999999999999995,"15min":0.82999999999999996},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.284,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":6.9909999999999997}]},{"osd":9,"last update":"Tue Jul 18 03:05:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.79000000000000004,"15min":0.79100000000000004},"min":{"1min":0.51400000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.107,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.871,"15min":0.877},"min":{"1min":0.52200000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.218,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.71399999999999997}]},{"osd":10,"last update":"Tue Jul 18 03:05:52 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.91300000000000003,"15min":0.92300000000000004},"min":{"1min":0.68000000000000005,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.054,"5min":1.282,"15min":1.282},"last":6.9359999999999999},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.78900000000000003,"15min":0.79200000000000004},"min":{"1min":0.57599999999999996,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.72599999999999998}]},{"osd":11,"last update":"Tue Jul 18 03:06:16 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.81299999999999994,"15min":0.80900000000000005},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.0760000000000001,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.88600000000000001,"15min":0.88200000000000001},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.3160000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.80300000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":1032,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":1056768,"data_stored":545544,"data_compressed":17573,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55200000000000005,"15min":0.53500000000000003},"min":{"1min":0.42999999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.498},{"interface":"front","average":{"1min":0.622,"5min":0.57499999999999996,"15min":0.55200000000000005},"min":{"1min":0.38500000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.726,"5min":1.726,"15min":1.726},"last":0.54300000000000004}]},{"osd":1,"last update":"Tue Jul 18 03:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.57399999999999995,"15min":0.51400000000000001},"min":{"1min":0.50800000000000001,"5min":0.311,"15min":0.311},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.57099999999999995,"15min":0.50900000000000001},"min":{"1min":0.42699999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.92300000000000004,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.63100000000000001}]},{"osd":3,"last update":"Tue Jul 18 03:06:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59299999999999997,"15min":0.56299999999999994},"min":{"1min":0.439,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93899999999999995,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.64400000000000002,"15min":0.59299999999999997},"min":{"1min":0.40899999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.127,"5min":3.234,"15min":3.234},"last":0.73199999999999998}]},{"osd":4,"last update":"Tue Jul 18 03:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.80700000000000005}]},{"osd":5,"last update":"Tue Jul 18 03:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.48399999999999999,"15min":0.45300000000000001},"min":{"1min":0.40300000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.502,"15min":0.46000000000000002},"min":{"1min":0.38400000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.76400000000000001}]},{"osd":6,"last update":"Tue Jul 18 03:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.65800000000000003,"15min":0.64000000000000001},"min":{"1min":0.46600000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.70799999999999996,"15min":0.68799999999999994},"min":{"1min":0.53800000000000003,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":0.90100000000000002}]},{"osd":7,"last update":"Tue Jul 18 03:06:03 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.85099999999999998}]},{"osd":8,"last update":"Tue Jul 18 03:06:08 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.66300000000000003,"15min":0.628},"min":{"1min":0.55100000000000005,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":1.006},{"interface":"front","average":{"1min":0.749,"5min":0.68200000000000005,"15min":0.65800000000000003},"min":{"1min":0.60199999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.96099999999999997,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.92300000000000004}]},{"osd":9,"last update":"Tue Jul 18 03:05:34 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.75,"15min":0.74099999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.825,"5min":1.825,"15min":1.825},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.80300000000000005,"15min":0.78900000000000003},"min":{"1min":0.61099999999999999,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.97699999999999998}]},{"osd":10,"last update":"Tue Jul 18 03:05:53 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.80700000000000005,"15min":0.78500000000000003},"min":{"1min":0.55100000000000005,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.76100000000000001,"15min":0.745},"min":{"1min":0.59499999999999997,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.95699999999999996}]},{"osd":11,"last update":"Tue Jul 18 03:05:58 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.76,"5min":1.76,"15min":1.76},"last":1.034},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.876}]}]},{"osd":3,"up_from":25,"seq":107374182465,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":978944,"data_stored":483299,"data_compressed":17573,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:05:24 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.42599999999999999,"15min":0.40300000000000002},"min":{"1min":0.32300000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":1.8440000000000001},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.44700000000000001,"15min":0.41399999999999998},"min":{"1min":0.30299999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.86699999999999999,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.67200000000000004}]},{"osd":1,"last update":"Tue Jul 18 03:05:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59699999999999998,"15min":0.56899999999999995},"min":{"1min":0.432,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.87,"5min":1.115,"15min":1.115},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.62,"15min":0.57999999999999996},"min":{"1min":0.432,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":1.694}]},{"osd":2,"last update":"Tue Jul 18 03:05:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.64700000000000002,"15min":0.60299999999999998},"min":{"1min":0.46400000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.1140000000000001,"5min":1.26,"15min":1.26},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61799999999999999,"15min":0.59599999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.036,"5min":1.212,"15min":1.212},"last":0.58399999999999996}]},{"osd":4,"last update":"Tue Jul 18 03:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.66000000000000003,"15min":0.61899999999999999},"min":{"1min":0.48299999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":1.6579999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.66700000000000004,"15min":0.61799999999999999},"min":{"1min":0.53300000000000003,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.046,"5min":1.143,"15min":1.143},"last":0.76900000000000002}]},{"osd":5,"last update":"Tue Jul 18 03:05:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.63200000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84599999999999997,"5min":1.218,"15min":1.218},"last":1.8109999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.71299999999999997,"15min":0.68200000000000005},"min":{"1min":0.495,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.1240000000000001,"5min":1.218,"15min":1.218},"last":0.80300000000000005}]},{"osd":6,"last update":"Tue Jul 18 03:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.66900000000000004,"15min":0.64300000000000002},"min":{"1min":0.48099999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0740000000000001,"5min":1.385,"15min":1.385},"last":1.5600000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.60999999999999999,"15min":0.59999999999999998},"min":{"1min":0.434,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85999999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":1.2150000000000001}]},{"osd":7,"last update":"Tue Jul 18 03:05:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.73399999999999999,"15min":0.72199999999999998},"min":{"1min":0.51800000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.85899999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":1.7330000000000001},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.76800000000000002,"15min":0.753},"min":{"1min":0.57199999999999995,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.153,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.876}]},{"osd":8,"last update":"Tue Jul 18 03:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.76200000000000001}]},{"osd":9,"last update":"Tue Jul 18 03:05:31 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.76400000000000001,"15min":0.78400000000000003},"min":{"1min":0.498,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.83299999999999996,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.85399999999999998,"15min":0.873},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.99299999999999999,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":1.6200000000000001}]},{"osd":10,"last update":"Tue Jul 18 03:05:52 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.88700000000000001,"15min":0.90500000000000003},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":0.93600000000000005,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":1.77},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.876,"15min":0.88600000000000001},"min":{"1min":0.626,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.1850000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.90900000000000003}]},{"osd":11,"last update":"Tue Jul 18 03:06:03 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.61699999999999999}]}]},{"osd":4,"up_from":33,"seq":141733920829,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":991232,"data_stored":497407,"data_compressed":20545,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.56200000000000006,"15min":0.54600000000000004},"min":{"1min":0.39600000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.439},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.56899999999999995,"15min":0.54500000000000004},"min":{"1min":0.42799999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.48999999999999999}]},{"osd":1,"last update":"Tue Jul 18 03:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.39700000000000002,"15min":0.36499999999999999},"min":{"1min":0.28999999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.73599999999999999,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.437,"5min":0.40600000000000003,"15min":0.38},"min":{"1min":0.30099999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.70499999999999996,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.52400000000000002}]},{"osd":2,"last update":"Tue Jul 18 03:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.59699999999999998,"15min":0.57499999999999996},"min":{"1min":0.39200000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.93000000000000005,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.60499999999999998,"15min":0.56699999999999995},"min":{"1min":0.438,"5min":0.378,"15min":0.378},"max":{"1min":0.84099999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.88100000000000001}]},{"osd":3,"last update":"Tue Jul 18 03:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.64900000000000002,"15min":0.59499999999999997},"min":{"1min":0.45000000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":3.7669999999999999,"5min":3.7669999999999999,"15min":3.7669999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59499999999999997,"15min":0.58099999999999996},"min":{"1min":0.39000000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83699999999999997,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.626}]},{"osd":5,"last update":"Tue Jul 18 03:06:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.65400000000000003,"15min":0.61099999999999999},"min":{"1min":0.39300000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.92600000000000005,"5min":1.214,"15min":1.214},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.63900000000000001,"15min":0.60899999999999999},"min":{"1min":0.49299999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.83399999999999996,"5min":1.306,"15min":1.306},"last":0.80600000000000005}]},{"osd":6,"last update":"Tue Jul 18 03:05:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.71699999999999997,"15min":0.72599999999999998},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.42399999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.69199999999999995}]},{"osd":7,"last update":"Tue Jul 18 03:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60799999999999998,"15min":0.60199999999999998},"min":{"1min":0.39000000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.85899999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.65900000000000003,"15min":0.64900000000000002},"min":{"1min":0.499,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.94299999999999995,"5min":1.034,"15min":1.034},"last":0.60399999999999998}]},{"osd":8,"last update":"Tue Jul 18 03:06:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73999999999999999,"15min":0.73499999999999999},"min":{"1min":0.53300000000000003,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.99399999999999999,"5min":1.155,"15min":1.155},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.75800000000000001,"15min":0.74399999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.008,"5min":1.135,"15min":1.135},"last":0.82599999999999996}]},{"osd":9,"last update":"Tue Jul 18 03:05:33 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.72399999999999998,"15min":0.72199999999999998},"min":{"1min":0.54000000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.82299999999999995,"15min":0.81100000000000005},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":3.415,"5min":3.415,"15min":3.415},"last":0.95799999999999996}]},{"osd":10,"last update":"Tue Jul 18 03:05:59 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.78700000000000003,"15min":0.79600000000000004},"min":{"1min":0.48999999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.90200000000000002,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.68799999999999994,"15min":0.69799999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.79900000000000004,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.77300000000000002}]},{"osd":11,"last update":"Tue Jul 18 03:05:59 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":1.0940000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.745}]}]},{"osd":5,"up_from":38,"seq":163208757304,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":1044480,"data_stored":1079297,"data_compressed":53332,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54900000000000004,"15min":0.53100000000000003},"min":{"1min":0.39400000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.79800000000000004,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.497,"5min":0.54000000000000004,"15min":0.53500000000000003},"min":{"1min":0.371,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.64700000000000002,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.442}]},{"osd":1,"last update":"Tue Jul 18 03:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.57799999999999996,"15min":0.54400000000000004},"min":{"1min":0.40100000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.74299999999999999,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.56599999999999995,"15min":0.54800000000000004},"min":{"1min":0.42299999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.50800000000000001}]},{"osd":2,"last update":"Tue Jul 18 03:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.40400000000000003,"15min":0.379},"min":{"1min":0.30599999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.61499999999999999,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.437,"5min":0.42099999999999999,"15min":0.39800000000000002},"min":{"1min":0.32100000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.53300000000000003,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.57799999999999996}]},{"osd":3,"last update":"Tue Jul 18 03:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.59199999999999997,"15min":0.56899999999999995},"min":{"1min":0.434,"5min":0.372,"15min":0.372},"max":{"1min":0.76400000000000001,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59599999999999997,"15min":0.57199999999999995},"min":{"1min":0.45700000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.79000000000000004,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.60399999999999998}]},{"osd":4,"last update":"Tue Jul 18 03:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59199999999999997,"15min":0.57099999999999995},"min":{"1min":0.41599999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83799999999999997,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.61099999999999999,"15min":0.57199999999999995},"min":{"1min":0.41899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84399999999999997,"5min":0.871,"15min":0.871},"last":0.65700000000000003}]},{"osd":6,"last update":"Tue Jul 18 03:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.61599999999999999,"15min":0.60299999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.94299999999999995,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.64200000000000002,"15min":0.623},"min":{"1min":0.437,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85799999999999998,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.72499999999999998}]},{"osd":7,"last update":"Tue Jul 18 03:05:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.67300000000000004,"15min":0.67500000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.875,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.69299999999999995,"15min":0.69099999999999995},"min":{"1min":0.5,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.90100000000000002,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.75}]},{"osd":8,"last update":"Tue Jul 18 03:06:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56200000000000006,"15min":0.57199999999999995},"min":{"1min":0.36399999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.75900000000000001,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.627,"15min":0.625},"min":{"1min":0.433,"5min":0.371,"15min":0.371},"max":{"1min":0.84699999999999998,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.76900000000000002}]},{"osd":9,"last update":"Tue Jul 18 03:05:34 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.76400000000000001,"15min":0.77100000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.90100000000000002,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.755,"15min":0.76500000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.82399999999999995}]},{"osd":10,"last update":"Tue Jul 18 03:05:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.73899999999999999,"15min":0.73299999999999998},"min":{"1min":0.57799999999999996,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.92400000000000004,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.74299999999999999,"15min":0.73899999999999999},"min":{"1min":0.54900000000000004,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.93300000000000005,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.78300000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34117,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34117,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34117,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35759,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35759,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35759,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35759,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35759,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-18T03:06:26.274 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph pg dump --format=json 2023-07-18T03:06:26.538 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:26 smithi155 ceph-mon[128885]: pgmap v322: 129 pgs: 2 peering, 127 active+clean; 640 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2023-07-18T03:06:26.693 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:26 smithi179 ceph-mon[129229]: pgmap v322: 129 pgs: 2 peering, 127 active+clean; 640 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2023-07-18T03:06:26.730 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:26 smithi101 ceph-mon[126379]: pgmap v322: 129 pgs: 2 peering, 127 active+clean; 640 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2023-07-18T03:06:27.000 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:06:27.509 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:27 smithi101 ceph-mon[126379]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:06:27.538 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:27 smithi155 ceph-mon[128885]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:06:27.693 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:27 smithi179 ceph-mon[129229]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:06:28.481 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:28 smithi101 ceph-mon[126379]: pgmap v323: 129 pgs: 2 peering, 127 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:06:28.538 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:28 smithi155 ceph-mon[128885]: pgmap v323: 129 pgs: 2 peering, 127 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:06:28.693 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:28 smithi179 ceph-mon[129229]: pgmap v323: 129 pgs: 2 peering, 127 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:06:28.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:28.816+0000 7fe5b82b8700 1 -- 172.21.15.101:0/2753520000 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe5b0104ea0 msgr2=0x7fe5b010ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:28.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:28.816+0000 7fe5b82b8700 1 --2- 172.21.15.101:0/2753520000 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe5b0104ea0 0x7fe5b010ca20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe5a800b0d0 tx=0x7fe5a800b3e0 comp rx=0 tx=0).stop 2023-07-18T03:06:28.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:28.816+0000 7fe5b82b8700 1 -- 172.21.15.101:0/2753520000 shutdown_connections 2023-07-18T03:06:28.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:28.816+0000 7fe5b82b8700 1 --2- 172.21.15.101:0/2753520000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5b00ffe00 0x7fe5b01001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:28.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:28.816+0000 7fe5b82b8700 1 --2- 172.21.15.101:0/2753520000 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe5b0104ea0 0x7fe5b010ca20 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:28.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:28.816+0000 7fe5b82b8700 1 --2- 172.21.15.101:0/2753520000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5b01007a0 0x7fe5b0104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:28.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:28.816+0000 7fe5b82b8700 1 -- 172.21.15.101:0/2753520000 >> 172.21.15.101:0/2753520000 conn(0x7fe5b00fb660 msgr2=0x7fe5b00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:28.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:28.816+0000 7fe5b82b8700 1 -- 172.21.15.101:0/2753520000 shutdown_connections 2023-07-18T03:06:28.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:28.816+0000 7fe5b82b8700 1 -- 172.21.15.101:0/2753520000 wait complete. 2023-07-18T03:06:28.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:28.817+0000 7fe5b82b8700 1 Processor -- start 2023-07-18T03:06:28.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:28.817+0000 7fe5b82b8700 1 -- start start 2023-07-18T03:06:28.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:28.818+0000 7fe5b82b8700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe5b00ffe00 0x7fe5b0072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:28.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:28.818+0000 7fe5b82b8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5b01007a0 0x7fe5b0072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:28.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:28.818+0000 7fe5b82b8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5b0104ea0 0x7fe5b00732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:28.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:28.818+0000 7fe5b82b8700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b01a41d0 con 0x7fe5b00ffe00 2023-07-18T03:06:28.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:28.818+0000 7fe5b82b8700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b01a4340 con 0x7fe5b01007a0 2023-07-18T03:06:28.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:28.818+0000 7fe5b82b8700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b01a4500 con 0x7fe5b0104ea0 2023-07-18T03:06:28.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:28.818+0000 7fe5b6054700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe5b00ffe00 0x7fe5b0072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:28.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:28.818+0000 7fe5b6054700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe5b00ffe00 0x7fe5b0072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:45950/0 (socket says 172.21.15.101:45950) 2023-07-18T03:06:28.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:28.818+0000 7fe5b6054700 1 -- 172.21.15.101:0/3366497660 learned_addr learned my addr 172.21.15.101:0/3366497660 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:28.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:28.818+0000 7fe5b5853700 1 --2- 172.21.15.101:0/3366497660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5b01007a0 0x7fe5b0072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:28.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:28.818+0000 7fe5b6855700 1 --2- 172.21.15.101:0/3366497660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5b0104ea0 0x7fe5b00732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:28.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:28.818+0000 7fe5b6054700 1 -- 172.21.15.101:0/3366497660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5b0104ea0 msgr2=0x7fe5b00732d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:28.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:28.818+0000 7fe5b6054700 1 --2- 172.21.15.101:0/3366497660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5b0104ea0 0x7fe5b00732d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:28.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:28.818+0000 7fe5b6054700 1 -- 172.21.15.101:0/3366497660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5b01007a0 msgr2=0x7fe5b0072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:28.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:28.818+0000 7fe5b6054700 1 --2- 172.21.15.101:0/3366497660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5b01007a0 0x7fe5b0072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:28.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:28.818+0000 7fe5b6054700 1 -- 172.21.15.101:0/3366497660 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a8009d00 con 0x7fe5b00ffe00 2023-07-18T03:06:28.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:28.819+0000 7fe5b6054700 1 --2- 172.21.15.101:0/3366497660 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe5b00ffe00 0x7fe5b0072810 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac00e9a0 tx=0x7fe5ac00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:28.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:28.819+0000 7fe5a77fe700 1 -- 172.21.15.101:0/3366497660 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac009790 con 0x7fe5b00ffe00 2023-07-18T03:06:28.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:28.819+0000 7fe5a77fe700 1 -- 172.21.15.101:0/3366497660 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5ac00cbd0 con 0x7fe5b00ffe00 2023-07-18T03:06:28.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:28.819+0000 7fe5b82b8700 1 -- 172.21.15.101:0/3366497660 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5b01a47e0 con 0x7fe5b00ffe00 2023-07-18T03:06:28.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:28.819+0000 7fe5a77fe700 1 -- 172.21.15.101:0/3366497660 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac018680 con 0x7fe5b00ffe00 2023-07-18T03:06:28.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:28.819+0000 7fe5b82b8700 1 -- 172.21.15.101:0/3366497660 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5b01a4d30 con 0x7fe5b00ffe00 2023-07-18T03:06:28.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:28.820+0000 7fe5b82b8700 1 -- 172.21.15.101:0/3366497660 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5b004e210 con 0x7fe5b00ffe00 2023-07-18T03:06:28.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:28.822+0000 7fe5a77fe700 1 -- 172.21.15.101:0/3366497660 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fe5ac00cd40 con 0x7fe5b00ffe00 2023-07-18T03:06:28.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:28.824+0000 7fe5a77fe700 1 --2- 172.21.15.101:0/3366497660 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe59c0a1260 0x7fe59c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:28.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:28.824+0000 7fe5a77fe700 1 -- 172.21.15.101:0/3366497660 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fe5ac014070 con 0x7fe5b00ffe00 2023-07-18T03:06:28.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:28.824+0000 7fe5b5853700 1 --2- 172.21.15.101:0/3366497660 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe59c0a1260 0x7fe59c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:28.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:28.824+0000 7fe5b5853700 1 --2- 172.21.15.101:0/3366497660 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe59c0a1260 0x7fe59c0a3710 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0074000 tx=0x7fe5a00073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:28.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:28.826+0000 7fe5a77fe700 1 -- 172.21.15.101:0/3366497660 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe5ac096520 con 0x7fe5b00ffe00 2023-07-18T03:06:29.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:29.019+0000 7fe5b82b8700 1 -- 172.21.15.101:0/3366497660 --> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe5b00fc790 con 0x7fe59c0a1260 2023-07-18T03:06:29.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:29.032+0000 7fe5a77fe700 1 -- 172.21.15.101:0/3366497660 <== mgr.14150 v2:172.21.15.101:6800/3963742842 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426721 (secure 0 0 0) 0x7fe5b00fc790 con 0x7fe59c0a1260 2023-07-18T03:06:29.024 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:06:29.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:29.037+0000 7fe5b82b8700 1 -- 172.21.15.101:0/3366497660 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe59c0a1260 msgr2=0x7fe59c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:29.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:29.037+0000 7fe5b82b8700 1 --2- 172.21.15.101:0/3366497660 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe59c0a1260 0x7fe59c0a3710 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0074000 tx=0x7fe5a00073d0 comp rx=0 tx=0).stop 2023-07-18T03:06:29.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:29.037+0000 7fe5b82b8700 1 -- 172.21.15.101:0/3366497660 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe5b00ffe00 msgr2=0x7fe5b0072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:29.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:29.037+0000 7fe5b82b8700 1 --2- 172.21.15.101:0/3366497660 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe5b00ffe00 0x7fe5b0072810 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac00e9a0 tx=0x7fe5ac00ecb0 comp rx=0 tx=0).stop 2023-07-18T03:06:29.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:29.037+0000 7fe5b82b8700 1 -- 172.21.15.101:0/3366497660 shutdown_connections 2023-07-18T03:06:29.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:29.037+0000 7fe5b82b8700 1 --2- 172.21.15.101:0/3366497660 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe59c0a1260 0x7fe59c0a3710 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:29.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:29.037+0000 7fe5b82b8700 1 --2- 172.21.15.101:0/3366497660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5b0104ea0 0x7fe5b00732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:29.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:29.037+0000 7fe5b82b8700 1 --2- 172.21.15.101:0/3366497660 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe5b00ffe00 0x7fe5b0072810 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:29.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:29.037+0000 7fe5b82b8700 1 --2- 172.21.15.101:0/3366497660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5b01007a0 0x7fe5b0072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:29.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:29.037+0000 7fe5b82b8700 1 -- 172.21.15.101:0/3366497660 >> 172.21.15.101:0/3366497660 conn(0x7fe5b00fb660 msgr2=0x7fe5b00fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:29.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:29.038+0000 7fe5b82b8700 1 -- 172.21.15.101:0/3366497660 shutdown_connections 2023-07-18T03:06:29.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:29.038+0000 7fe5b82b8700 1 -- 172.21.15.101:0/3366497660 wait complete. 2023-07-18T03:06:29.028 INFO:teuthology.orchestra.run.smithi101.stderr:dumped all 2023-07-18T03:06:29.486 INFO:teuthology.orchestra.run.smithi101.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2023-07-18T03:06:28.872899+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655355,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":488,"num_read_kb":365,"num_write":337,"num_write_kb":3050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":418,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335540,"kb_used_data":12116,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606284,"statfs":{"total":1151940427776,"available":1151596834816,"internally_reserved":0,"allocated":12406784,"data_stored":7998215,"data_compressed":380635,"data_compressed_allocated":516096,"data_compressed_original":2531328,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":29,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001934"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.257745+0000","last_change":"2023-07-18T03:04:59.805689+0000","last_active":"2023-07-18T03:06:24.257745+0000","last_peered":"2023-07-18T03:06:24.257745+0000","last_clean":"2023-07-18T03:06:24.257745+0000","last_became_active":"2023-07-18T03:04:59.805224+0000","last_became_peered":"2023-07-18T03:04:59.805224+0000","last_unstale":"2023-07-18T03:06:24.257745+0000","last_undegraded":"2023-07-18T03:06:24.257745+0000","last_fullsized":"2023-07-18T03:06:24.257745+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:06:51.978483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.264959+0000","last_change":"2023-07-18T03:04:59.805013+0000","last_active":"2023-07-18T03:06:23.264959+0000","last_peered":"2023-07-18T03:06:23.264959+0000","last_clean":"2023-07-18T03:06:23.264959+0000","last_became_active":"2023-07-18T03:04:59.804702+0000","last_became_peered":"2023-07-18T03:04:59.804702+0000","last_unstale":"2023-07-18T03:06:23.264959+0000","last_undegraded":"2023-07-18T03:06:23.264959+0000","last_fullsized":"2023-07-18T03:06:23.264959+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:24:23.239057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.265217+0000","last_change":"2023-07-18T03:04:59.809035+0000","last_active":"2023-07-18T03:06:23.265217+0000","last_peered":"2023-07-18T03:06:23.265217+0000","last_clean":"2023-07-18T03:06:23.265217+0000","last_became_active":"2023-07-18T03:04:59.808630+0000","last_became_peered":"2023-07-18T03:04:59.808630+0000","last_unstale":"2023-07-18T03:06:23.265217+0000","last_undegraded":"2023-07-18T03:06:23.265217+0000","last_fullsized":"2023-07-18T03:06:23.265217+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:53:21.301875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.259429+0000","last_change":"2023-07-18T03:04:59.803510+0000","last_active":"2023-07-18T03:06:23.259429+0000","last_peered":"2023-07-18T03:06:23.259429+0000","last_clean":"2023-07-18T03:06:23.259429+0000","last_became_active":"2023-07-18T03:04:59.803060+0000","last_became_peered":"2023-07-18T03:04:59.803060+0000","last_unstale":"2023-07-18T03:06:23.259429+0000","last_undegraded":"2023-07-18T03:06:23.259429+0000","last_fullsized":"2023-07-18T03:06:23.259429+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:28:38.229617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.260081+0000","last_change":"2023-07-18T03:04:59.806911+0000","last_active":"2023-07-18T03:06:24.260081+0000","last_peered":"2023-07-18T03:06:24.260081+0000","last_clean":"2023-07-18T03:06:24.260081+0000","last_became_active":"2023-07-18T03:04:59.806597+0000","last_became_peered":"2023-07-18T03:04:59.806597+0000","last_unstale":"2023-07-18T03:06:24.260081+0000","last_undegraded":"2023-07-18T03:06:24.260081+0000","last_fullsized":"2023-07-18T03:06:24.260081+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:23:04.609208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.375607+0000","last_change":"2023-07-18T03:04:59.809263+0000","last_active":"2023-07-18T03:06:24.375607+0000","last_peered":"2023-07-18T03:06:24.375607+0000","last_clean":"2023-07-18T03:06:24.375607+0000","last_became_active":"2023-07-18T03:04:59.808956+0000","last_became_peered":"2023-07-18T03:04:59.808956+0000","last_unstale":"2023-07-18T03:06:24.375607+0000","last_undegraded":"2023-07-18T03:06:24.375607+0000","last_fullsized":"2023-07-18T03:06:24.375607+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:22:13.425291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.257561+0000","last_change":"2023-07-18T03:04:59.804946+0000","last_active":"2023-07-18T03:06:24.257561+0000","last_peered":"2023-07-18T03:06:24.257561+0000","last_clean":"2023-07-18T03:06:24.257561+0000","last_became_active":"2023-07-18T03:04:59.804610+0000","last_became_peered":"2023-07-18T03:04:59.804610+0000","last_unstale":"2023-07-18T03:06:24.257561+0000","last_undegraded":"2023-07-18T03:06:24.257561+0000","last_fullsized":"2023-07-18T03:06:24.257561+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:35:53.147695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.374997+0000","last_change":"2023-07-18T03:04:59.801796+0000","last_active":"2023-07-18T03:06:24.374997+0000","last_peered":"2023-07-18T03:06:24.374997+0000","last_clean":"2023-07-18T03:06:24.374997+0000","last_became_active":"2023-07-18T03:04:59.801432+0000","last_became_peered":"2023-07-18T03:04:59.801432+0000","last_unstale":"2023-07-18T03:06:24.374997+0000","last_undegraded":"2023-07-18T03:06:24.374997+0000","last_fullsized":"2023-07-18T03:06:24.374997+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:32:07.392642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.262068+0000","last_change":"2023-07-18T03:04:59.798191+0000","last_active":"2023-07-18T03:06:24.262068+0000","last_peered":"2023-07-18T03:06:24.262068+0000","last_clean":"2023-07-18T03:06:24.262068+0000","last_became_active":"2023-07-18T03:04:59.797863+0000","last_became_peered":"2023-07-18T03:04:59.797863+0000","last_unstale":"2023-07-18T03:06:24.262068+0000","last_undegraded":"2023-07-18T03:06:24.262068+0000","last_fullsized":"2023-07-18T03:06:24.262068+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:43:29.304091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.375910+0000","last_change":"2023-07-18T03:04:59.809867+0000","last_active":"2023-07-18T03:06:24.375910+0000","last_peered":"2023-07-18T03:06:24.375910+0000","last_clean":"2023-07-18T03:06:24.375910+0000","last_became_active":"2023-07-18T03:04:59.809568+0000","last_became_peered":"2023-07-18T03:04:59.809568+0000","last_unstale":"2023-07-18T03:06:24.375910+0000","last_undegraded":"2023-07-18T03:06:24.375910+0000","last_fullsized":"2023-07-18T03:06:24.375910+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:49:15.008273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.258199+0000","last_change":"2023-07-18T03:04:59.805920+0000","last_active":"2023-07-18T03:06:24.258199+0000","last_peered":"2023-07-18T03:06:24.258199+0000","last_clean":"2023-07-18T03:06:24.258199+0000","last_became_active":"2023-07-18T03:04:59.805559+0000","last_became_peered":"2023-07-18T03:04:59.805559+0000","last_unstale":"2023-07-18T03:06:24.258199+0000","last_undegraded":"2023-07-18T03:06:24.258199+0000","last_fullsized":"2023-07-18T03:06:24.258199+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:27:52.167301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.259385+0000","last_change":"2023-07-18T03:04:59.797940+0000","last_active":"2023-07-18T03:06:23.259385+0000","last_peered":"2023-07-18T03:06:23.259385+0000","last_clean":"2023-07-18T03:06:23.259385+0000","last_became_active":"2023-07-18T03:04:59.797575+0000","last_became_peered":"2023-07-18T03:04:59.797575+0000","last_unstale":"2023-07-18T03:06:23.259385+0000","last_undegraded":"2023-07-18T03:06:23.259385+0000","last_fullsized":"2023-07-18T03:06:23.259385+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:28:11.054129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.751022+0000","last_change":"2023-07-18T03:04:59.798026+0000","last_active":"2023-07-18T03:06:24.751022+0000","last_peered":"2023-07-18T03:06:24.751022+0000","last_clean":"2023-07-18T03:06:24.751022+0000","last_became_active":"2023-07-18T03:04:59.797734+0000","last_became_peered":"2023-07-18T03:04:59.797734+0000","last_unstale":"2023-07-18T03:06:24.751022+0000","last_undegraded":"2023-07-18T03:06:24.751022+0000","last_fullsized":"2023-07-18T03:06:24.751022+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:33:40.010231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.264921+0000","last_change":"2023-07-18T03:04:59.804964+0000","last_active":"2023-07-18T03:06:23.264921+0000","last_peered":"2023-07-18T03:06:23.264921+0000","last_clean":"2023-07-18T03:06:23.264921+0000","last_became_active":"2023-07-18T03:04:59.804660+0000","last_became_peered":"2023-07-18T03:04:59.804660+0000","last_unstale":"2023-07-18T03:06:23.264921+0000","last_undegraded":"2023-07-18T03:06:23.264921+0000","last_fullsized":"2023-07-18T03:06:23.264921+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:14:40.387622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.264800+0000","last_change":"2023-07-18T03:04:59.809384+0000","last_active":"2023-07-18T03:06:23.264800+0000","last_peered":"2023-07-18T03:06:23.264800+0000","last_clean":"2023-07-18T03:06:23.264800+0000","last_became_active":"2023-07-18T03:04:59.808969+0000","last_became_peered":"2023-07-18T03:04:59.808969+0000","last_unstale":"2023-07-18T03:06:23.264800+0000","last_undegraded":"2023-07-18T03:06:23.264800+0000","last_fullsized":"2023-07-18T03:06:23.264800+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:56:24.126487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.261272+0000","last_change":"2023-07-18T03:04:59.808344+0000","last_active":"2023-07-18T03:06:24.261272+0000","last_peered":"2023-07-18T03:06:24.261272+0000","last_clean":"2023-07-18T03:06:24.261272+0000","last_became_active":"2023-07-18T03:04:59.807995+0000","last_became_peered":"2023-07-18T03:04:59.807995+0000","last_unstale":"2023-07-18T03:06:24.261272+0000","last_undegraded":"2023-07-18T03:06:24.261272+0000","last_fullsized":"2023-07-18T03:06:24.261272+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:15:44.462832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.257694+0000","last_change":"2023-07-18T03:04:59.806584+0000","last_active":"2023-07-18T03:06:24.257694+0000","last_peered":"2023-07-18T03:06:24.257694+0000","last_clean":"2023-07-18T03:06:24.257694+0000","last_became_active":"2023-07-18T03:04:59.804118+0000","last_became_peered":"2023-07-18T03:04:59.804118+0000","last_unstale":"2023-07-18T03:06:24.257694+0000","last_undegraded":"2023-07-18T03:06:24.257694+0000","last_fullsized":"2023-07-18T03:06:24.257694+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:42:18.047326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255362+0000","last_change":"2023-07-18T03:04:59.812610+0000","last_active":"2023-07-18T03:06:24.255362+0000","last_peered":"2023-07-18T03:06:24.255362+0000","last_clean":"2023-07-18T03:06:24.255362+0000","last_became_active":"2023-07-18T03:04:59.812230+0000","last_became_peered":"2023-07-18T03:04:59.812230+0000","last_unstale":"2023-07-18T03:06:24.255362+0000","last_undegraded":"2023-07-18T03:06:24.255362+0000","last_fullsized":"2023-07-18T03:06:24.255362+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:07:54.810466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.256127+0000","last_change":"2023-07-18T03:04:59.811880+0000","last_active":"2023-07-18T03:06:24.256127+0000","last_peered":"2023-07-18T03:06:24.256127+0000","last_clean":"2023-07-18T03:06:24.256127+0000","last_became_active":"2023-07-18T03:04:59.811561+0000","last_became_peered":"2023-07-18T03:04:59.811561+0000","last_unstale":"2023-07-18T03:06:24.256127+0000","last_undegraded":"2023-07-18T03:06:24.256127+0000","last_fullsized":"2023-07-18T03:06:24.256127+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:53:25.407425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.256306+0000","last_change":"2023-07-18T03:05:03.880843+0000","last_active":"2023-07-18T03:06:24.256306+0000","last_peered":"2023-07-18T03:06:24.256306+0000","last_clean":"2023-07-18T03:06:24.256306+0000","last_became_active":"2023-07-18T03:05:03.880391+0000","last_became_peered":"2023-07-18T03:05:03.880391+0000","last_unstale":"2023-07-18T03:06:24.256306+0000","last_undegraded":"2023-07-18T03:06:24.256306+0000","last_fullsized":"2023-07-18T03:06:24.256306+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:25:52.190341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255174+0000","last_change":"2023-07-18T03:04:58.848771+0000","last_active":"2023-07-18T03:06:24.255174+0000","last_peered":"2023-07-18T03:06:24.255174+0000","last_clean":"2023-07-18T03:06:24.255174+0000","last_became_active":"2023-07-18T03:04:58.848405+0000","last_became_peered":"2023-07-18T03:04:58.848405+0000","last_unstale":"2023-07-18T03:06:24.255174+0000","last_undegraded":"2023-07-18T03:06:24.255174+0000","last_fullsized":"2023-07-18T03:06:24.255174+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:09:13.965066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.375449+0000","last_change":"2023-07-18T03:04:59.805749+0000","last_active":"2023-07-18T03:06:24.375449+0000","last_peered":"2023-07-18T03:06:24.375449+0000","last_clean":"2023-07-18T03:06:24.375449+0000","last_became_active":"2023-07-18T03:04:59.805388+0000","last_became_peered":"2023-07-18T03:04:59.805388+0000","last_unstale":"2023-07-18T03:06:24.375449+0000","last_undegraded":"2023-07-18T03:06:24.375449+0000","last_fullsized":"2023-07-18T03:06:24.375449+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:36:19.960742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.258828+0000","last_change":"2023-07-18T03:05:03.868819+0000","last_active":"2023-07-18T03:06:23.258828+0000","last_peered":"2023-07-18T03:06:23.258828+0000","last_clean":"2023-07-18T03:06:23.258828+0000","last_became_active":"2023-07-18T03:05:03.868387+0000","last_became_peered":"2023-07-18T03:05:03.868387+0000","last_unstale":"2023-07-18T03:06:23.258828+0000","last_undegraded":"2023-07-18T03:06:23.258828+0000","last_fullsized":"2023-07-18T03:06:23.258828+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:53:41.719909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.375462+0000","last_change":"2023-07-18T03:04:58.856084+0000","last_active":"2023-07-18T03:06:24.375462+0000","last_peered":"2023-07-18T03:06:24.375462+0000","last_clean":"2023-07-18T03:06:24.375462+0000","last_became_active":"2023-07-18T03:04:58.855768+0000","last_became_peered":"2023-07-18T03:04:58.855768+0000","last_unstale":"2023-07-18T03:06:24.375462+0000","last_undegraded":"2023-07-18T03:06:24.375462+0000","last_fullsized":"2023-07-18T03:06:24.375462+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:58:35.159671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.265432+0000","last_change":"2023-07-18T03:04:59.809512+0000","last_active":"2023-07-18T03:06:23.265432+0000","last_peered":"2023-07-18T03:06:23.265432+0000","last_clean":"2023-07-18T03:06:23.265432+0000","last_became_active":"2023-07-18T03:04:59.808112+0000","last_became_peered":"2023-07-18T03:04:59.808112+0000","last_unstale":"2023-07-18T03:06:23.265432+0000","last_undegraded":"2023-07-18T03:06:23.265432+0000","last_fullsized":"2023-07-18T03:06:23.265432+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:26:56.291603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.259007+0000","last_change":"2023-07-18T03:05:03.874728+0000","last_active":"2023-07-18T03:06:23.259007+0000","last_peered":"2023-07-18T03:06:23.259007+0000","last_clean":"2023-07-18T03:06:23.259007+0000","last_became_active":"2023-07-18T03:05:03.874262+0000","last_became_peered":"2023-07-18T03:05:03.874262+0000","last_unstale":"2023-07-18T03:06:23.259007+0000","last_undegraded":"2023-07-18T03:06:23.259007+0000","last_fullsized":"2023-07-18T03:06:23.259007+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:22:05.085389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.261317+0000","last_change":"2023-07-18T03:04:58.849140+0000","last_active":"2023-07-18T03:06:24.261317+0000","last_peered":"2023-07-18T03:06:24.261317+0000","last_clean":"2023-07-18T03:06:24.261317+0000","last_became_active":"2023-07-18T03:04:58.848693+0000","last_became_peered":"2023-07-18T03:04:58.848693+0000","last_unstale":"2023-07-18T03:06:24.261317+0000","last_undegraded":"2023-07-18T03:06:24.261317+0000","last_fullsized":"2023-07-18T03:06:24.261317+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:07:50.859417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.261591+0000","last_change":"2023-07-18T03:04:59.807248+0000","last_active":"2023-07-18T03:06:24.261591+0000","last_peered":"2023-07-18T03:06:24.261591+0000","last_clean":"2023-07-18T03:06:24.261591+0000","last_became_active":"2023-07-18T03:04:59.806807+0000","last_became_peered":"2023-07-18T03:04:59.806807+0000","last_unstale":"2023-07-18T03:06:24.261591+0000","last_undegraded":"2023-07-18T03:06:24.261591+0000","last_fullsized":"2023-07-18T03:06:24.261591+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:07:33.904712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.258830+0000","last_change":"2023-07-18T03:05:03.869075+0000","last_active":"2023-07-18T03:06:23.258830+0000","last_peered":"2023-07-18T03:06:23.258830+0000","last_clean":"2023-07-18T03:06:23.258830+0000","last_became_active":"2023-07-18T03:05:03.868653+0000","last_became_peered":"2023-07-18T03:05:03.868653+0000","last_unstale":"2023-07-18T03:06:23.258830+0000","last_undegraded":"2023-07-18T03:06:23.258830+0000","last_fullsized":"2023-07-18T03:06:23.258830+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:20:02.501750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.259374+0000","last_change":"2023-07-18T03:04:58.843879+0000","last_active":"2023-07-18T03:06:24.259374+0000","last_peered":"2023-07-18T03:06:24.259374+0000","last_clean":"2023-07-18T03:06:24.259374+0000","last_became_active":"2023-07-18T03:04:58.843525+0000","last_became_peered":"2023-07-18T03:04:58.843525+0000","last_unstale":"2023-07-18T03:06:24.259374+0000","last_undegraded":"2023-07-18T03:06:24.259374+0000","last_fullsized":"2023-07-18T03:06:24.259374+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:32:03.232823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.261880+0000","last_change":"2023-07-18T03:04:59.803597+0000","last_active":"2023-07-18T03:06:24.261880+0000","last_peered":"2023-07-18T03:06:24.261880+0000","last_clean":"2023-07-18T03:06:24.261880+0000","last_became_active":"2023-07-18T03:04:59.803293+0000","last_became_peered":"2023-07-18T03:04:59.803293+0000","last_unstale":"2023-07-18T03:06:24.261880+0000","last_undegraded":"2023-07-18T03:06:24.261880+0000","last_fullsized":"2023-07-18T03:06:24.261880+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:31:55.196443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255711+0000","last_change":"2023-07-18T03:05:23.944721+0000","last_active":"2023-07-18T03:06:24.255711+0000","last_peered":"2023-07-18T03:06:24.255711+0000","last_clean":"2023-07-18T03:06:24.255711+0000","last_became_active":"2023-07-18T03:05:23.944101+0000","last_became_peered":"2023-07-18T03:05:23.944101+0000","last_unstale":"2023-07-18T03:06:24.255711+0000","last_undegraded":"2023-07-18T03:06:24.255711+0000","last_fullsized":"2023-07-18T03:06:24.255711+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:20:50.819953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":93,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.261505+0000","last_change":"2023-07-18T03:04:58.847990+0000","last_active":"2023-07-18T03:06:24.261505+0000","last_peered":"2023-07-18T03:06:24.261505+0000","last_clean":"2023-07-18T03:06:24.261505+0000","last_became_active":"2023-07-18T03:04:58.847634+0000","last_became_peered":"2023-07-18T03:04:58.847634+0000","last_unstale":"2023-07-18T03:06:24.261505+0000","last_undegraded":"2023-07-18T03:06:24.261505+0000","last_fullsized":"2023-07-18T03:06:24.261505+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:14:18.454279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.259915+0000","last_change":"2023-07-18T03:04:59.797076+0000","last_active":"2023-07-18T03:06:24.259915+0000","last_peered":"2023-07-18T03:06:24.259915+0000","last_clean":"2023-07-18T03:06:24.259915+0000","last_became_active":"2023-07-18T03:04:59.796753+0000","last_became_peered":"2023-07-18T03:04:59.796753+0000","last_unstale":"2023-07-18T03:06:24.259915+0000","last_undegraded":"2023-07-18T03:06:24.259915+0000","last_fullsized":"2023-07-18T03:06:24.259915+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:46:03.085633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255734+0000","last_change":"2023-07-18T03:05:03.881793+0000","last_active":"2023-07-18T03:06:24.255734+0000","last_peered":"2023-07-18T03:06:24.255734+0000","last_clean":"2023-07-18T03:06:24.255734+0000","last_became_active":"2023-07-18T03:05:03.881341+0000","last_became_peered":"2023-07-18T03:05:03.881341+0000","last_unstale":"2023-07-18T03:06:24.255734+0000","last_undegraded":"2023-07-18T03:06:24.255734+0000","last_fullsized":"2023-07-18T03:06:24.255734+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:40:23.370420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.375558+0000","last_change":"2023-07-18T03:04:58.846732+0000","last_active":"2023-07-18T03:06:24.375558+0000","last_peered":"2023-07-18T03:06:24.375558+0000","last_clean":"2023-07-18T03:06:24.375558+0000","last_became_active":"2023-07-18T03:04:58.846321+0000","last_became_peered":"2023-07-18T03:04:58.846321+0000","last_unstale":"2023-07-18T03:06:24.375558+0000","last_undegraded":"2023-07-18T03:06:24.375558+0000","last_fullsized":"2023-07-18T03:06:24.375558+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:50:00.381631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.254627+0000","last_change":"2023-07-18T03:05:24.093277+0000","last_active":"2023-07-18T03:06:24.254627+0000","last_peered":"2023-07-18T03:06:24.254627+0000","last_clean":"2023-07-18T03:06:24.254627+0000","last_became_active":"2023-07-18T03:05:24.092842+0000","last_became_peered":"2023-07-18T03:05:24.092842+0000","last_unstale":"2023-07-18T03:06:24.254627+0000","last_undegraded":"2023-07-18T03:06:24.254627+0000","last_fullsized":"2023-07-18T03:06:24.254627+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:22:01.737422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.257590+0000","last_change":"2023-07-18T03:05:03.880782+0000","last_active":"2023-07-18T03:06:24.257590+0000","last_peered":"2023-07-18T03:06:24.257590+0000","last_clean":"2023-07-18T03:06:24.257590+0000","last_became_active":"2023-07-18T03:05:03.880362+0000","last_became_peered":"2023-07-18T03:05:03.880362+0000","last_unstale":"2023-07-18T03:06:24.257590+0000","last_undegraded":"2023-07-18T03:06:24.257590+0000","last_fullsized":"2023-07-18T03:06:24.257590+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:10:14.772789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.750818+0000","last_change":"2023-07-18T03:04:58.846095+0000","last_active":"2023-07-18T03:06:24.750818+0000","last_peered":"2023-07-18T03:06:24.750818+0000","last_clean":"2023-07-18T03:06:24.750818+0000","last_became_active":"2023-07-18T03:04:58.845242+0000","last_became_peered":"2023-07-18T03:04:58.845242+0000","last_unstale":"2023-07-18T03:06:24.750818+0000","last_undegraded":"2023-07-18T03:06:24.750818+0000","last_fullsized":"2023-07-18T03:06:24.750818+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:36:00.462709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.374919+0000","last_change":"2023-07-18T03:04:59.803024+0000","last_active":"2023-07-18T03:06:24.374919+0000","last_peered":"2023-07-18T03:06:24.374919+0000","last_clean":"2023-07-18T03:06:24.374919+0000","last_became_active":"2023-07-18T03:04:59.802687+0000","last_became_peered":"2023-07-18T03:04:59.802687+0000","last_unstale":"2023-07-18T03:06:24.374919+0000","last_undegraded":"2023-07-18T03:06:24.374919+0000","last_fullsized":"2023-07-18T03:06:24.374919+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:59:27.108157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.264826+0000","last_change":"2023-07-18T03:05:03.872690+0000","last_active":"2023-07-18T03:06:23.264826+0000","last_peered":"2023-07-18T03:06:23.264826+0000","last_clean":"2023-07-18T03:06:23.264826+0000","last_became_active":"2023-07-18T03:05:03.872187+0000","last_became_peered":"2023-07-18T03:05:03.872187+0000","last_unstale":"2023-07-18T03:06:23.264826+0000","last_undegraded":"2023-07-18T03:06:23.264826+0000","last_fullsized":"2023-07-18T03:06:23.264826+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:45:16.836607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.261033+0000","last_change":"2023-07-18T03:04:58.842877+0000","last_active":"2023-07-18T03:06:24.261033+0000","last_peered":"2023-07-18T03:06:24.261033+0000","last_clean":"2023-07-18T03:06:24.261033+0000","last_became_active":"2023-07-18T03:04:58.842516+0000","last_became_peered":"2023-07-18T03:04:58.842516+0000","last_unstale":"2023-07-18T03:06:24.261033+0000","last_undegraded":"2023-07-18T03:06:24.261033+0000","last_fullsized":"2023-07-18T03:06:24.261033+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:24:49.503369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.254669+0000","last_change":"2023-07-18T03:04:59.807592+0000","last_active":"2023-07-18T03:06:24.254669+0000","last_peered":"2023-07-18T03:06:24.254669+0000","last_clean":"2023-07-18T03:06:24.254669+0000","last_became_active":"2023-07-18T03:04:59.807276+0000","last_became_peered":"2023-07-18T03:04:59.807276+0000","last_unstale":"2023-07-18T03:06:24.254669+0000","last_undegraded":"2023-07-18T03:06:24.254669+0000","last_fullsized":"2023-07-18T03:06:24.254669+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:16:05.749019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.374960+0000","last_change":"2023-07-18T03:05:03.876355+0000","last_active":"2023-07-18T03:06:24.374960+0000","last_peered":"2023-07-18T03:06:24.374960+0000","last_clean":"2023-07-18T03:06:24.374960+0000","last_became_active":"2023-07-18T03:05:03.875845+0000","last_became_peered":"2023-07-18T03:05:03.875845+0000","last_unstale":"2023-07-18T03:06:24.374960+0000","last_undegraded":"2023-07-18T03:06:24.374960+0000","last_fullsized":"2023-07-18T03:06:24.374960+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:13:06.076832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":88,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:25.072477+0000","last_change":"2023-07-18T03:04:58.840389+0000","last_active":"2023-07-18T03:06:25.072477+0000","last_peered":"2023-07-18T03:06:25.072477+0000","last_clean":"2023-07-18T03:06:25.072477+0000","last_became_active":"2023-07-18T03:04:58.839974+0000","last_became_peered":"2023-07-18T03:04:58.839974+0000","last_unstale":"2023-07-18T03:06:25.072477+0000","last_undegraded":"2023-07-18T03:06:25.072477+0000","last_fullsized":"2023-07-18T03:06:25.072477+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:40:06.944095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":55,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.262049+0000","last_change":"2023-07-18T03:04:59.807681+0000","last_active":"2023-07-18T03:06:24.262049+0000","last_peered":"2023-07-18T03:06:24.262049+0000","last_clean":"2023-07-18T03:06:24.262049+0000","last_became_active":"2023-07-18T03:04:59.807365+0000","last_became_peered":"2023-07-18T03:04:59.807365+0000","last_unstale":"2023-07-18T03:06:24.262049+0000","last_undegraded":"2023-07-18T03:06:24.262049+0000","last_fullsized":"2023-07-18T03:06:24.262049+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:02:22.715114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.258920+0000","last_change":"2023-07-18T03:05:24.091413+0000","last_active":"2023-07-18T03:06:24.258920+0000","last_peered":"2023-07-18T03:06:24.258920+0000","last_clean":"2023-07-18T03:06:24.258920+0000","last_became_active":"2023-07-18T03:05:24.090815+0000","last_became_peered":"2023-07-18T03:05:24.090815+0000","last_unstale":"2023-07-18T03:06:24.258920+0000","last_undegraded":"2023-07-18T03:06:24.258920+0000","last_fullsized":"2023-07-18T03:06:24.258920+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:25:43.324509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.268967+0000","last_change":"2023-07-18T03:06:24.268967+0000","last_active":"2023-07-18T03:06:24.268967+0000","last_peered":"2023-07-18T03:06:24.268967+0000","last_clean":"2023-07-18T03:06:24.268967+0000","last_became_active":"2023-07-18T03:06:24.263424+0000","last_became_peered":"2023-07-18T03:06:24.263424+0000","last_unstale":"2023-07-18T03:06:24.268967+0000","last_undegraded":"2023-07-18T03:06:24.268967+0000","last_fullsized":"2023-07-18T03:06:24.268967+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:13:59.331501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.265282+0000","last_change":"2023-07-18T03:05:24.092870+0000","last_active":"2023-07-18T03:06:23.265282+0000","last_peered":"2023-07-18T03:06:23.265282+0000","last_clean":"2023-07-18T03:06:23.265282+0000","last_became_active":"2023-07-18T03:05:24.092474+0000","last_became_peered":"2023-07-18T03:05:24.092474+0000","last_unstale":"2023-07-18T03:06:23.265282+0000","last_undegraded":"2023-07-18T03:06:23.265282+0000","last_fullsized":"2023-07-18T03:06:23.265282+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:48:14.130393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.259166+0000","last_change":"2023-07-18T03:05:03.874444+0000","last_active":"2023-07-18T03:06:23.259166+0000","last_peered":"2023-07-18T03:06:23.259166+0000","last_clean":"2023-07-18T03:06:23.259166+0000","last_became_active":"2023-07-18T03:05:03.873737+0000","last_became_peered":"2023-07-18T03:05:03.873737+0000","last_unstale":"2023-07-18T03:06:23.259166+0000","last_undegraded":"2023-07-18T03:06:23.259166+0000","last_fullsized":"2023-07-18T03:06:23.259166+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:30:06.718472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.261721+0000","last_change":"2023-07-18T03:04:58.845207+0000","last_active":"2023-07-18T03:06:24.261721+0000","last_peered":"2023-07-18T03:06:24.261721+0000","last_clean":"2023-07-18T03:06:24.261721+0000","last_became_active":"2023-07-18T03:04:58.844895+0000","last_became_peered":"2023-07-18T03:04:58.844895+0000","last_unstale":"2023-07-18T03:06:24.261721+0000","last_undegraded":"2023-07-18T03:06:24.261721+0000","last_fullsized":"2023-07-18T03:06:24.261721+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:59:56.250544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.261824+0000","last_change":"2023-07-18T03:04:59.807075+0000","last_active":"2023-07-18T03:06:24.261824+0000","last_peered":"2023-07-18T03:06:24.261824+0000","last_clean":"2023-07-18T03:06:24.261824+0000","last_became_active":"2023-07-18T03:04:59.806345+0000","last_became_peered":"2023-07-18T03:04:59.806345+0000","last_unstale":"2023-07-18T03:06:24.261824+0000","last_undegraded":"2023-07-18T03:06:24.261824+0000","last_fullsized":"2023-07-18T03:06:24.261824+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:50:55.760441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.375376+0000","last_change":"2023-07-18T03:05:24.091132+0000","last_active":"2023-07-18T03:06:24.375376+0000","last_peered":"2023-07-18T03:06:24.375376+0000","last_clean":"2023-07-18T03:06:24.375376+0000","last_became_active":"2023-07-18T03:05:24.090653+0000","last_became_peered":"2023-07-18T03:05:24.090653+0000","last_unstale":"2023-07-18T03:06:24.375376+0000","last_undegraded":"2023-07-18T03:06:24.375376+0000","last_fullsized":"2023-07-18T03:06:24.375376+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:14:06.294348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"89'32","reported_seq":180,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:26.087972+0000","last_change":"2023-07-18T03:04:58.844865+0000","last_active":"2023-07-18T03:06:26.087972+0000","last_peered":"2023-07-18T03:06:26.087972+0000","last_clean":"2023-07-18T03:06:26.087972+0000","last_became_active":"2023-07-18T03:04:58.844547+0000","last_became_peered":"2023-07-18T03:04:58.844547+0000","last_unstale":"2023-07-18T03:06:26.087972+0000","last_undegraded":"2023-07-18T03:06:26.087972+0000","last_fullsized":"2023-07-18T03:06:26.087972+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:09:40.602341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60837,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":33,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.261523+0000","last_change":"2023-07-18T03:04:59.803024+0000","last_active":"2023-07-18T03:06:24.261523+0000","last_peered":"2023-07-18T03:06:24.261523+0000","last_clean":"2023-07-18T03:06:24.261523+0000","last_became_active":"2023-07-18T03:04:59.802705+0000","last_became_peered":"2023-07-18T03:04:59.802705+0000","last_unstale":"2023-07-18T03:06:24.261523+0000","last_undegraded":"2023-07-18T03:06:24.261523+0000","last_fullsized":"2023-07-18T03:06:24.261523+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:46:31.950692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.262478+0000","last_change":"2023-07-18T03:05:03.876009+0000","last_active":"2023-07-18T03:06:24.262478+0000","last_peered":"2023-07-18T03:06:24.262478+0000","last_clean":"2023-07-18T03:06:24.262478+0000","last_became_active":"2023-07-18T03:05:03.875328+0000","last_became_peered":"2023-07-18T03:05:03.875328+0000","last_unstale":"2023-07-18T03:06:24.262478+0000","last_undegraded":"2023-07-18T03:06:24.262478+0000","last_fullsized":"2023-07-18T03:06:24.262478+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:55:54.930973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.256055+0000","last_change":"2023-07-18T03:04:58.846573+0000","last_active":"2023-07-18T03:06:24.256055+0000","last_peered":"2023-07-18T03:06:24.256055+0000","last_clean":"2023-07-18T03:06:24.256055+0000","last_became_active":"2023-07-18T03:04:58.846271+0000","last_became_peered":"2023-07-18T03:04:58.846271+0000","last_unstale":"2023-07-18T03:06:24.256055+0000","last_undegraded":"2023-07-18T03:06:24.256055+0000","last_fullsized":"2023-07-18T03:06:24.256055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:22:31.518590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.750829+0000","last_change":"2023-07-18T03:04:59.801552+0000","last_active":"2023-07-18T03:06:24.750829+0000","last_peered":"2023-07-18T03:06:24.750829+0000","last_clean":"2023-07-18T03:06:24.750829+0000","last_became_active":"2023-07-18T03:04:59.801224+0000","last_became_peered":"2023-07-18T03:04:59.801224+0000","last_unstale":"2023-07-18T03:06:24.750829+0000","last_undegraded":"2023-07-18T03:06:24.750829+0000","last_fullsized":"2023-07-18T03:06:24.750829+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:44:10.394026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.257271+0000","last_change":"2023-07-18T03:05:03.881125+0000","last_active":"2023-07-18T03:06:24.257271+0000","last_peered":"2023-07-18T03:06:24.257271+0000","last_clean":"2023-07-18T03:06:24.257271+0000","last_became_active":"2023-07-18T03:05:03.880723+0000","last_became_peered":"2023-07-18T03:05:03.880723+0000","last_unstale":"2023-07-18T03:06:24.257271+0000","last_undegraded":"2023-07-18T03:06:24.257271+0000","last_fullsized":"2023-07-18T03:06:24.257271+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:28:40.857218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.261012+0000","last_change":"2023-07-18T03:04:58.847118+0000","last_active":"2023-07-18T03:06:24.261012+0000","last_peered":"2023-07-18T03:06:24.261012+0000","last_clean":"2023-07-18T03:06:24.261012+0000","last_became_active":"2023-07-18T03:04:58.846805+0000","last_became_peered":"2023-07-18T03:04:58.846805+0000","last_unstale":"2023-07-18T03:06:24.261012+0000","last_undegraded":"2023-07-18T03:06:24.261012+0000","last_fullsized":"2023-07-18T03:06:24.261012+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:48:50.417535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.258219+0000","last_change":"2023-07-18T03:04:59.803914+0000","last_active":"2023-07-18T03:06:24.258219+0000","last_peered":"2023-07-18T03:06:24.258219+0000","last_clean":"2023-07-18T03:06:24.258219+0000","last_became_active":"2023-07-18T03:04:59.803589+0000","last_became_peered":"2023-07-18T03:04:59.803589+0000","last_unstale":"2023-07-18T03:06:24.258219+0000","last_undegraded":"2023-07-18T03:06:24.258219+0000","last_fullsized":"2023-07-18T03:06:24.258219+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:24:46.537710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255012+0000","last_change":"2023-07-18T03:05:03.877787+0000","last_active":"2023-07-18T03:06:24.255012+0000","last_peered":"2023-07-18T03:06:24.255012+0000","last_clean":"2023-07-18T03:06:24.255012+0000","last_became_active":"2023-07-18T03:05:03.877449+0000","last_became_peered":"2023-07-18T03:05:03.877449+0000","last_unstale":"2023-07-18T03:06:24.255012+0000","last_undegraded":"2023-07-18T03:06:24.255012+0000","last_fullsized":"2023-07-18T03:06:24.255012+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:54:47.075087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":85,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.258959+0000","last_change":"2023-07-18T03:04:58.844252+0000","last_active":"2023-07-18T03:06:23.258959+0000","last_peered":"2023-07-18T03:06:23.258959+0000","last_clean":"2023-07-18T03:06:23.258959+0000","last_became_active":"2023-07-18T03:04:58.843798+0000","last_became_peered":"2023-07-18T03:04:58.843798+0000","last_unstale":"2023-07-18T03:06:23.258959+0000","last_undegraded":"2023-07-18T03:06:23.258959+0000","last_fullsized":"2023-07-18T03:06:23.258959+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:33:22.080789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.257510+0000","last_change":"2023-07-18T03:04:59.806274+0000","last_active":"2023-07-18T03:06:24.257510+0000","last_peered":"2023-07-18T03:06:24.257510+0000","last_clean":"2023-07-18T03:06:24.257510+0000","last_became_active":"2023-07-18T03:04:59.805876+0000","last_became_peered":"2023-07-18T03:04:59.805876+0000","last_unstale":"2023-07-18T03:06:24.257510+0000","last_undegraded":"2023-07-18T03:06:24.257510+0000","last_fullsized":"2023-07-18T03:06:24.257510+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:00:53.965603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.751210+0000","last_change":"2023-07-18T03:05:03.876783+0000","last_active":"2023-07-18T03:06:24.751210+0000","last_peered":"2023-07-18T03:06:24.751210+0000","last_clean":"2023-07-18T03:06:24.751210+0000","last_became_active":"2023-07-18T03:05:03.876385+0000","last_became_peered":"2023-07-18T03:05:03.876385+0000","last_unstale":"2023-07-18T03:06:24.751210+0000","last_undegraded":"2023-07-18T03:06:24.751210+0000","last_fullsized":"2023-07-18T03:06:24.751210+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:12:22.516248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.259581+0000","last_change":"2023-07-18T03:04:58.851107+0000","last_active":"2023-07-18T03:06:24.259581+0000","last_peered":"2023-07-18T03:06:24.259581+0000","last_clean":"2023-07-18T03:06:24.259581+0000","last_became_active":"2023-07-18T03:04:58.850848+0000","last_became_peered":"2023-07-18T03:04:58.850848+0000","last_unstale":"2023-07-18T03:06:24.259581+0000","last_undegraded":"2023-07-18T03:06:24.259581+0000","last_fullsized":"2023-07-18T03:06:24.259581+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:39:44.591519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.260987+0000","last_change":"2023-07-18T03:04:59.803683+0000","last_active":"2023-07-18T03:06:24.260987+0000","last_peered":"2023-07-18T03:06:24.260987+0000","last_clean":"2023-07-18T03:06:24.260987+0000","last_became_active":"2023-07-18T03:04:59.803365+0000","last_became_peered":"2023-07-18T03:04:59.803365+0000","last_unstale":"2023-07-18T03:06:24.260987+0000","last_undegraded":"2023-07-18T03:06:24.260987+0000","last_fullsized":"2023-07-18T03:06:24.260987+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:30:10.236288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.260801+0000","last_change":"2023-07-18T03:05:03.875856+0000","last_active":"2023-07-18T03:06:24.260801+0000","last_peered":"2023-07-18T03:06:24.260801+0000","last_clean":"2023-07-18T03:06:24.260801+0000","last_became_active":"2023-07-18T03:05:03.875426+0000","last_became_peered":"2023-07-18T03:05:03.875426+0000","last_unstale":"2023-07-18T03:06:24.260801+0000","last_undegraded":"2023-07-18T03:06:24.260801+0000","last_fullsized":"2023-07-18T03:06:24.260801+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:48:39.948769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.261120+0000","last_change":"2023-07-18T03:04:58.844102+0000","last_active":"2023-07-18T03:06:24.261120+0000","last_peered":"2023-07-18T03:06:24.261120+0000","last_clean":"2023-07-18T03:06:24.261120+0000","last_became_active":"2023-07-18T03:04:58.843740+0000","last_became_peered":"2023-07-18T03:04:58.843740+0000","last_unstale":"2023-07-18T03:06:24.261120+0000","last_undegraded":"2023-07-18T03:06:24.261120+0000","last_fullsized":"2023-07-18T03:06:24.261120+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:19:46.540756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.257520+0000","last_change":"2023-07-18T03:05:24.091161+0000","last_active":"2023-07-18T03:06:24.257520+0000","last_peered":"2023-07-18T03:06:24.257520+0000","last_clean":"2023-07-18T03:06:24.257520+0000","last_became_active":"2023-07-18T03:05:24.090631+0000","last_became_peered":"2023-07-18T03:05:24.090631+0000","last_unstale":"2023-07-18T03:06:24.257520+0000","last_undegraded":"2023-07-18T03:06:24.257520+0000","last_fullsized":"2023-07-18T03:06:24.257520+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:21:10.676261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.259764+0000","last_change":"2023-07-18T03:05:24.092328+0000","last_active":"2023-07-18T03:06:24.259764+0000","last_peered":"2023-07-18T03:06:24.259764+0000","last_clean":"2023-07-18T03:06:24.259764+0000","last_became_active":"2023-07-18T03:05:24.091839+0000","last_became_peered":"2023-07-18T03:05:24.091839+0000","last_unstale":"2023-07-18T03:06:24.259764+0000","last_undegraded":"2023-07-18T03:06:24.259764+0000","last_fullsized":"2023-07-18T03:06:24.259764+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:42:18.354994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.261616+0000","last_change":"2023-07-18T03:06:24.261616+0000","last_active":"2023-07-18T03:06:24.261616+0000","last_peered":"2023-07-18T03:06:24.261616+0000","last_clean":"2023-07-18T03:06:24.261616+0000","last_became_active":"2023-07-18T03:06:24.261250+0000","last_became_peered":"2023-07-18T03:06:24.261250+0000","last_unstale":"2023-07-18T03:06:24.261616+0000","last_undegraded":"2023-07-18T03:06:24.261616+0000","last_fullsized":"2023-07-18T03:06:24.261616+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:19:24.589920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.375817+0000","last_change":"2023-07-18T03:05:24.092486+0000","last_active":"2023-07-18T03:06:24.375817+0000","last_peered":"2023-07-18T03:06:24.375817+0000","last_clean":"2023-07-18T03:06:24.375817+0000","last_became_active":"2023-07-18T03:05:24.092189+0000","last_became_peered":"2023-07-18T03:05:24.092189+0000","last_unstale":"2023-07-18T03:06:24.375817+0000","last_undegraded":"2023-07-18T03:06:24.375817+0000","last_fullsized":"2023-07-18T03:06:24.375817+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:09:16.496072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.750970+0000","last_change":"2023-07-18T03:05:03.868851+0000","last_active":"2023-07-18T03:06:24.750970+0000","last_peered":"2023-07-18T03:06:24.750970+0000","last_clean":"2023-07-18T03:06:24.750970+0000","last_became_active":"2023-07-18T03:05:03.868433+0000","last_became_peered":"2023-07-18T03:05:03.868433+0000","last_unstale":"2023-07-18T03:06:24.750970+0000","last_undegraded":"2023-07-18T03:06:24.750970+0000","last_fullsized":"2023-07-18T03:06:24.750970+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:36:46.180540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.257995+0000","last_change":"2023-07-18T03:04:58.848059+0000","last_active":"2023-07-18T03:06:24.257995+0000","last_peered":"2023-07-18T03:06:24.257995+0000","last_clean":"2023-07-18T03:06:24.257995+0000","last_became_active":"2023-07-18T03:04:58.847700+0000","last_became_peered":"2023-07-18T03:04:58.847700+0000","last_unstale":"2023-07-18T03:06:24.257995+0000","last_undegraded":"2023-07-18T03:06:24.257995+0000","last_fullsized":"2023-07-18T03:06:24.257995+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:22:42.461972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.375602+0000","last_change":"2023-07-18T03:05:23.939939+0000","last_active":"2023-07-18T03:06:24.375602+0000","last_peered":"2023-07-18T03:06:24.375602+0000","last_clean":"2023-07-18T03:06:24.375602+0000","last_became_active":"2023-07-18T03:05:23.939607+0000","last_became_peered":"2023-07-18T03:05:23.939607+0000","last_unstale":"2023-07-18T03:06:24.375602+0000","last_undegraded":"2023-07-18T03:06:24.375602+0000","last_fullsized":"2023-07-18T03:06:24.375602+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:47:48.796749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.751153+0000","last_change":"2023-07-18T03:05:03.886724+0000","last_active":"2023-07-18T03:06:24.751153+0000","last_peered":"2023-07-18T03:06:24.751153+0000","last_clean":"2023-07-18T03:06:24.751153+0000","last_became_active":"2023-07-18T03:05:03.886172+0000","last_became_peered":"2023-07-18T03:05:03.886172+0000","last_unstale":"2023-07-18T03:06:24.751153+0000","last_undegraded":"2023-07-18T03:06:24.751153+0000","last_fullsized":"2023-07-18T03:06:24.751153+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:35:26.182278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.261189+0000","last_change":"2023-07-18T03:04:59.804273+0000","last_active":"2023-07-18T03:06:24.261189+0000","last_peered":"2023-07-18T03:06:24.261189+0000","last_clean":"2023-07-18T03:06:24.261189+0000","last_became_active":"2023-07-18T03:04:59.803987+0000","last_became_peered":"2023-07-18T03:04:59.803987+0000","last_unstale":"2023-07-18T03:06:24.261189+0000","last_undegraded":"2023-07-18T03:06:24.261189+0000","last_fullsized":"2023-07-18T03:06:24.261189+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:53:36.335539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":89,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:25.072598+0000","last_change":"2023-07-18T03:04:58.846381+0000","last_active":"2023-07-18T03:06:25.072598+0000","last_peered":"2023-07-18T03:06:25.072598+0000","last_clean":"2023-07-18T03:06:25.072598+0000","last_became_active":"2023-07-18T03:04:58.845757+0000","last_became_peered":"2023-07-18T03:04:58.845757+0000","last_unstale":"2023-07-18T03:06:25.072598+0000","last_undegraded":"2023-07-18T03:06:25.072598+0000","last_fullsized":"2023-07-18T03:06:25.072598+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:00:14.653137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":55,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.258013+0000","last_change":"2023-07-18T03:05:03.873626+0000","last_active":"2023-07-18T03:06:24.258013+0000","last_peered":"2023-07-18T03:06:24.258013+0000","last_clean":"2023-07-18T03:06:24.258013+0000","last_became_active":"2023-07-18T03:05:03.872965+0000","last_became_peered":"2023-07-18T03:05:03.872965+0000","last_unstale":"2023-07-18T03:06:24.258013+0000","last_undegraded":"2023-07-18T03:06:24.258013+0000","last_fullsized":"2023-07-18T03:06:24.258013+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:37:38.450618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.374965+0000","last_change":"2023-07-18T03:04:59.801153+0000","last_active":"2023-07-18T03:06:24.374965+0000","last_peered":"2023-07-18T03:06:24.374965+0000","last_clean":"2023-07-18T03:06:24.374965+0000","last_became_active":"2023-07-18T03:04:59.800798+0000","last_became_peered":"2023-07-18T03:04:59.800798+0000","last_unstale":"2023-07-18T03:06:24.374965+0000","last_undegraded":"2023-07-18T03:06:24.374965+0000","last_fullsized":"2023-07-18T03:06:24.374965+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:49:57.147042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.259089+0000","last_change":"2023-07-18T03:04:58.846047+0000","last_active":"2023-07-18T03:06:23.259089+0000","last_peered":"2023-07-18T03:06:23.259089+0000","last_clean":"2023-07-18T03:06:23.259089+0000","last_became_active":"2023-07-18T03:04:58.845678+0000","last_became_peered":"2023-07-18T03:04:58.845678+0000","last_unstale":"2023-07-18T03:06:23.259089+0000","last_undegraded":"2023-07-18T03:06:23.259089+0000","last_fullsized":"2023-07-18T03:06:23.259089+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:49:09.335016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":384,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.257343+0000","last_change":"2023-07-18T03:03:54.168837+0000","last_active":"2023-07-18T03:06:24.257343+0000","last_peered":"2023-07-18T03:06:24.257343+0000","last_clean":"2023-07-18T03:06:24.257343+0000","last_became_active":"2023-07-18T03:03:54.168298+0000","last_became_peered":"2023-07-18T03:03:54.168298+0000","last_unstale":"2023-07-18T03:06:24.257343+0000","last_undegraded":"2023-07-18T03:06:24.257343+0000","last_fullsized":"2023-07-18T03:06:24.257343+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:00:43.469168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:00:43.469168+0000","last_clean_scrub_stamp":"2023-07-18T03:00:43.469168+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:03:45.014912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.261390+0000","last_change":"2023-07-18T03:05:03.886611+0000","last_active":"2023-07-18T03:06:24.261390+0000","last_peered":"2023-07-18T03:06:24.261390+0000","last_clean":"2023-07-18T03:06:24.261390+0000","last_became_active":"2023-07-18T03:05:03.886099+0000","last_became_peered":"2023-07-18T03:05:03.886099+0000","last_unstale":"2023-07-18T03:06:24.261390+0000","last_undegraded":"2023-07-18T03:06:24.261390+0000","last_fullsized":"2023-07-18T03:06:24.261390+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:26:40.808017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.750658+0000","last_change":"2023-07-18T03:04:59.804478+0000","last_active":"2023-07-18T03:06:24.750658+0000","last_peered":"2023-07-18T03:06:24.750658+0000","last_clean":"2023-07-18T03:06:24.750658+0000","last_became_active":"2023-07-18T03:04:59.804162+0000","last_became_peered":"2023-07-18T03:04:59.804162+0000","last_unstale":"2023-07-18T03:06:24.750658+0000","last_undegraded":"2023-07-18T03:06:24.750658+0000","last_fullsized":"2023-07-18T03:06:24.750658+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:41:57.294071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.266708+0000","last_change":"2023-07-18T03:06:24.266708+0000","last_active":"2023-07-18T03:06:24.266708+0000","last_peered":"2023-07-18T03:06:24.266708+0000","last_clean":"2023-07-18T03:06:24.266708+0000","last_became_active":"2023-07-18T03:06:24.262606+0000","last_became_peered":"2023-07-18T03:06:24.262606+0000","last_unstale":"2023-07-18T03:06:24.266708+0000","last_undegraded":"2023-07-18T03:06:24.266708+0000","last_fullsized":"2023-07-18T03:06:24.266708+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:20:59.875839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.257266+0000","last_change":"2023-07-18T03:05:03.872788+0000","last_active":"2023-07-18T03:06:24.257266+0000","last_peered":"2023-07-18T03:06:24.257266+0000","last_clean":"2023-07-18T03:06:24.257266+0000","last_became_active":"2023-07-18T03:05:03.872343+0000","last_became_peered":"2023-07-18T03:05:03.872343+0000","last_unstale":"2023-07-18T03:06:24.257266+0000","last_undegraded":"2023-07-18T03:06:24.257266+0000","last_fullsized":"2023-07-18T03:06:24.257266+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:20:43.912026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.266064+0000","last_change":"2023-07-18T03:06:24.265727+0000","last_active":"2023-07-18T03:06:24.266064+0000","last_peered":"2023-07-18T03:06:24.266064+0000","last_clean":"2023-07-18T03:06:24.266064+0000","last_became_active":"2023-07-18T03:06:24.262373+0000","last_became_peered":"2023-07-18T03:06:24.262373+0000","last_unstale":"2023-07-18T03:06:24.266064+0000","last_undegraded":"2023-07-18T03:06:24.266064+0000","last_fullsized":"2023-07-18T03:06:24.266064+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:38:45.610472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.751058+0000","last_change":"2023-07-18T03:04:59.804706+0000","last_active":"2023-07-18T03:06:24.751058+0000","last_peered":"2023-07-18T03:06:24.751058+0000","last_clean":"2023-07-18T03:06:24.751058+0000","last_became_active":"2023-07-18T03:04:59.804445+0000","last_became_peered":"2023-07-18T03:04:59.804445+0000","last_unstale":"2023-07-18T03:06:24.751058+0000","last_undegraded":"2023-07-18T03:06:24.751058+0000","last_fullsized":"2023-07-18T03:06:24.751058+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:54:52.824231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.374862+0000","last_change":"2023-07-18T03:05:03.886065+0000","last_active":"2023-07-18T03:06:24.374862+0000","last_peered":"2023-07-18T03:06:24.374862+0000","last_clean":"2023-07-18T03:06:24.374862+0000","last_became_active":"2023-07-18T03:05:03.885625+0000","last_became_peered":"2023-07-18T03:05:03.885625+0000","last_unstale":"2023-07-18T03:06:24.374862+0000","last_undegraded":"2023-07-18T03:06:24.374862+0000","last_fullsized":"2023-07-18T03:06:24.374862+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:32:31.156943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'5","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255204+0000","last_change":"2023-07-18T03:04:58.849246+0000","last_active":"2023-07-18T03:06:24.255204+0000","last_peered":"2023-07-18T03:06:24.255204+0000","last_clean":"2023-07-18T03:06:24.255204+0000","last_became_active":"2023-07-18T03:04:58.848888+0000","last_became_peered":"2023-07-18T03:04:58.848888+0000","last_unstale":"2023-07-18T03:06:24.255204+0000","last_undegraded":"2023-07-18T03:06:24.255204+0000","last_fullsized":"2023-07-18T03:06:24.255204+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:47:14.002970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.375509+0000","last_change":"2023-07-18T03:04:59.804561+0000","last_active":"2023-07-18T03:06:24.375509+0000","last_peered":"2023-07-18T03:06:24.375509+0000","last_clean":"2023-07-18T03:06:24.375509+0000","last_became_active":"2023-07-18T03:04:59.803659+0000","last_became_peered":"2023-07-18T03:04:59.803659+0000","last_unstale":"2023-07-18T03:06:24.375509+0000","last_undegraded":"2023-07-18T03:06:24.375509+0000","last_fullsized":"2023-07-18T03:06:24.375509+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:55:58.968273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255438+0000","last_change":"2023-07-18T03:05:03.871418+0000","last_active":"2023-07-18T03:06:24.255438+0000","last_peered":"2023-07-18T03:06:24.255438+0000","last_clean":"2023-07-18T03:06:24.255438+0000","last_became_active":"2023-07-18T03:05:03.871054+0000","last_became_peered":"2023-07-18T03:05:03.871054+0000","last_unstale":"2023-07-18T03:06:24.255438+0000","last_undegraded":"2023-07-18T03:06:24.255438+0000","last_fullsized":"2023-07-18T03:06:24.255438+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:59:35.196165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.259519+0000","last_change":"2023-07-18T03:04:58.850505+0000","last_active":"2023-07-18T03:06:23.259519+0000","last_peered":"2023-07-18T03:06:23.259519+0000","last_clean":"2023-07-18T03:06:23.259519+0000","last_became_active":"2023-07-18T03:04:58.850234+0000","last_became_peered":"2023-07-18T03:04:58.850234+0000","last_unstale":"2023-07-18T03:06:23.259519+0000","last_undegraded":"2023-07-18T03:06:23.259519+0000","last_fullsized":"2023-07-18T03:06:23.259519+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:03:00.163608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.265586+0000","last_change":"2023-07-18T03:04:59.807348+0000","last_active":"2023-07-18T03:06:23.265586+0000","last_peered":"2023-07-18T03:06:23.265586+0000","last_clean":"2023-07-18T03:06:23.265586+0000","last_became_active":"2023-07-18T03:04:59.807013+0000","last_became_peered":"2023-07-18T03:04:59.807013+0000","last_unstale":"2023-07-18T03:06:23.265586+0000","last_undegraded":"2023-07-18T03:06:23.265586+0000","last_fullsized":"2023-07-18T03:06:23.265586+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:10:16.107970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.259356+0000","last_change":"2023-07-18T03:05:03.886773+0000","last_active":"2023-07-18T03:06:23.259356+0000","last_peered":"2023-07-18T03:06:23.259356+0000","last_clean":"2023-07-18T03:06:23.259356+0000","last_became_active":"2023-07-18T03:05:03.886259+0000","last_became_peered":"2023-07-18T03:05:03.886259+0000","last_unstale":"2023-07-18T03:06:23.259356+0000","last_undegraded":"2023-07-18T03:06:23.259356+0000","last_fullsized":"2023-07-18T03:06:23.259356+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:15:59.986776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.261426+0000","last_change":"2023-07-18T03:04:58.855746+0000","last_active":"2023-07-18T03:06:24.261426+0000","last_peered":"2023-07-18T03:06:24.261426+0000","last_clean":"2023-07-18T03:06:24.261426+0000","last_became_active":"2023-07-18T03:04:58.855366+0000","last_became_peered":"2023-07-18T03:04:58.855366+0000","last_unstale":"2023-07-18T03:06:24.261426+0000","last_undegraded":"2023-07-18T03:06:24.261426+0000","last_fullsized":"2023-07-18T03:06:24.261426+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:04:33.740458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.259103+0000","last_change":"2023-07-18T03:04:59.800321+0000","last_active":"2023-07-18T03:06:24.259103+0000","last_peered":"2023-07-18T03:06:24.259103+0000","last_clean":"2023-07-18T03:06:24.259103+0000","last_became_active":"2023-07-18T03:04:59.800018+0000","last_became_peered":"2023-07-18T03:04:59.800018+0000","last_unstale":"2023-07-18T03:06:24.259103+0000","last_undegraded":"2023-07-18T03:06:24.259103+0000","last_fullsized":"2023-07-18T03:06:24.259103+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:13:00.845492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255150+0000","last_change":"2023-07-18T03:05:03.877541+0000","last_active":"2023-07-18T03:06:24.255150+0000","last_peered":"2023-07-18T03:06:24.255150+0000","last_clean":"2023-07-18T03:06:24.255150+0000","last_became_active":"2023-07-18T03:05:03.877164+0000","last_became_peered":"2023-07-18T03:05:03.877164+0000","last_unstale":"2023-07-18T03:06:24.255150+0000","last_undegraded":"2023-07-18T03:06:24.255150+0000","last_fullsized":"2023-07-18T03:06:24.255150+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:58:38.384423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.375045+0000","last_change":"2023-07-18T03:04:58.845643+0000","last_active":"2023-07-18T03:06:24.375045+0000","last_peered":"2023-07-18T03:06:24.375045+0000","last_clean":"2023-07-18T03:06:24.375045+0000","last_became_active":"2023-07-18T03:04:58.845254+0000","last_became_peered":"2023-07-18T03:04:58.845254+0000","last_unstale":"2023-07-18T03:06:24.375045+0000","last_undegraded":"2023-07-18T03:06:24.375045+0000","last_fullsized":"2023-07-18T03:06:24.375045+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:03:15.745367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.257374+0000","last_change":"2023-07-18T03:04:59.803679+0000","last_active":"2023-07-18T03:06:24.257374+0000","last_peered":"2023-07-18T03:06:24.257374+0000","last_clean":"2023-07-18T03:06:24.257374+0000","last_became_active":"2023-07-18T03:04:59.802841+0000","last_became_peered":"2023-07-18T03:04:59.802841+0000","last_unstale":"2023-07-18T03:06:24.257374+0000","last_undegraded":"2023-07-18T03:06:24.257374+0000","last_fullsized":"2023-07-18T03:06:24.257374+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:51:12.763232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.259148+0000","last_change":"2023-07-18T03:05:03.874093+0000","last_active":"2023-07-18T03:06:23.259148+0000","last_peered":"2023-07-18T03:06:23.259148+0000","last_clean":"2023-07-18T03:06:23.259148+0000","last_became_active":"2023-07-18T03:05:03.873710+0000","last_became_peered":"2023-07-18T03:05:03.873710+0000","last_unstale":"2023-07-18T03:06:23.259148+0000","last_undegraded":"2023-07-18T03:06:23.259148+0000","last_fullsized":"2023-07-18T03:06:23.259148+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:47:55.346705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255614+0000","last_change":"2023-07-18T03:04:58.846686+0000","last_active":"2023-07-18T03:06:24.255614+0000","last_peered":"2023-07-18T03:06:24.255614+0000","last_clean":"2023-07-18T03:06:24.255614+0000","last_became_active":"2023-07-18T03:04:58.846371+0000","last_became_peered":"2023-07-18T03:04:58.846371+0000","last_unstale":"2023-07-18T03:06:24.255614+0000","last_undegraded":"2023-07-18T03:06:24.255614+0000","last_fullsized":"2023-07-18T03:06:24.255614+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:25:47.154236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.375244+0000","last_change":"2023-07-18T03:04:59.811917+0000","last_active":"2023-07-18T03:06:24.375244+0000","last_peered":"2023-07-18T03:06:24.375244+0000","last_clean":"2023-07-18T03:06:24.375244+0000","last_became_active":"2023-07-18T03:04:59.811551+0000","last_became_peered":"2023-07-18T03:04:59.811551+0000","last_unstale":"2023-07-18T03:06:24.375244+0000","last_undegraded":"2023-07-18T03:06:24.375244+0000","last_fullsized":"2023-07-18T03:06:24.375244+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:28:00.926747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.259384+0000","last_change":"2023-07-18T03:05:03.886072+0000","last_active":"2023-07-18T03:06:23.259384+0000","last_peered":"2023-07-18T03:06:23.259384+0000","last_clean":"2023-07-18T03:06:23.259384+0000","last_became_active":"2023-07-18T03:05:03.885652+0000","last_became_peered":"2023-07-18T03:05:03.885652+0000","last_unstale":"2023-07-18T03:06:23.259384+0000","last_undegraded":"2023-07-18T03:06:23.259384+0000","last_fullsized":"2023-07-18T03:06:23.259384+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:51:11.860036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.259624+0000","last_change":"2023-07-18T03:04:58.849069+0000","last_active":"2023-07-18T03:06:24.259624+0000","last_peered":"2023-07-18T03:06:24.259624+0000","last_clean":"2023-07-18T03:06:24.259624+0000","last_became_active":"2023-07-18T03:04:58.848744+0000","last_became_peered":"2023-07-18T03:04:58.848744+0000","last_unstale":"2023-07-18T03:06:24.259624+0000","last_undegraded":"2023-07-18T03:06:24.259624+0000","last_fullsized":"2023-07-18T03:06:24.259624+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:19:13.922970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.257880+0000","last_change":"2023-07-18T03:04:59.806976+0000","last_active":"2023-07-18T03:06:24.257880+0000","last_peered":"2023-07-18T03:06:24.257880+0000","last_clean":"2023-07-18T03:06:24.257880+0000","last_became_active":"2023-07-18T03:04:59.805750+0000","last_became_peered":"2023-07-18T03:04:59.805750+0000","last_unstale":"2023-07-18T03:06:24.257880+0000","last_undegraded":"2023-07-18T03:06:24.257880+0000","last_fullsized":"2023-07-18T03:06:24.257880+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:59:17.717720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.261529+0000","last_change":"2023-07-18T03:05:03.871210+0000","last_active":"2023-07-18T03:06:24.261529+0000","last_peered":"2023-07-18T03:06:24.261529+0000","last_clean":"2023-07-18T03:06:24.261529+0000","last_became_active":"2023-07-18T03:05:03.870826+0000","last_became_peered":"2023-07-18T03:05:03.870826+0000","last_unstale":"2023-07-18T03:06:24.261529+0000","last_undegraded":"2023-07-18T03:06:24.261529+0000","last_fullsized":"2023-07-18T03:06:24.261529+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:49:03.743543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.256102+0000","last_change":"2023-07-18T03:04:58.855673+0000","last_active":"2023-07-18T03:06:24.256102+0000","last_peered":"2023-07-18T03:06:24.256102+0000","last_clean":"2023-07-18T03:06:24.256102+0000","last_became_active":"2023-07-18T03:04:58.855338+0000","last_became_peered":"2023-07-18T03:04:58.855338+0000","last_unstale":"2023-07-18T03:06:24.256102+0000","last_undegraded":"2023-07-18T03:06:24.256102+0000","last_fullsized":"2023-07-18T03:06:24.256102+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:07:40.935595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.258224+0000","last_change":"2023-07-18T03:04:59.802488+0000","last_active":"2023-07-18T03:06:24.258224+0000","last_peered":"2023-07-18T03:06:24.258224+0000","last_clean":"2023-07-18T03:06:24.258224+0000","last_became_active":"2023-07-18T03:04:59.802184+0000","last_became_peered":"2023-07-18T03:04:59.802184+0000","last_unstale":"2023-07-18T03:06:24.258224+0000","last_undegraded":"2023-07-18T03:06:24.258224+0000","last_fullsized":"2023-07-18T03:06:24.258224+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:00:59.367787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.264915+0000","last_change":"2023-07-18T03:05:03.881585+0000","last_active":"2023-07-18T03:06:23.264915+0000","last_peered":"2023-07-18T03:06:23.264915+0000","last_clean":"2023-07-18T03:06:23.264915+0000","last_became_active":"2023-07-18T03:05:03.880917+0000","last_became_peered":"2023-07-18T03:05:03.880917+0000","last_unstale":"2023-07-18T03:06:23.264915+0000","last_undegraded":"2023-07-18T03:06:23.264915+0000","last_fullsized":"2023-07-18T03:06:23.264915+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:00:08.510960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.261933+0000","last_change":"2023-07-18T03:04:58.852387+0000","last_active":"2023-07-18T03:06:24.261933+0000","last_peered":"2023-07-18T03:06:24.261933+0000","last_clean":"2023-07-18T03:06:24.261933+0000","last_became_active":"2023-07-18T03:04:58.852113+0000","last_became_peered":"2023-07-18T03:04:58.852113+0000","last_unstale":"2023-07-18T03:06:24.261933+0000","last_undegraded":"2023-07-18T03:06:24.261933+0000","last_fullsized":"2023-07-18T03:06:24.261933+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:12:15.450770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.258050+0000","last_change":"2023-07-18T03:04:59.811569+0000","last_active":"2023-07-18T03:06:24.258050+0000","last_peered":"2023-07-18T03:06:24.258050+0000","last_clean":"2023-07-18T03:06:24.258050+0000","last_became_active":"2023-07-18T03:04:59.811229+0000","last_became_peered":"2023-07-18T03:04:59.811229+0000","last_unstale":"2023-07-18T03:06:24.258050+0000","last_undegraded":"2023-07-18T03:06:24.258050+0000","last_fullsized":"2023-07-18T03:06:24.258050+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:20:39.366577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.260914+0000","last_change":"2023-07-18T03:05:03.872530+0000","last_active":"2023-07-18T03:06:24.260914+0000","last_peered":"2023-07-18T03:06:24.260914+0000","last_clean":"2023-07-18T03:06:24.260914+0000","last_became_active":"2023-07-18T03:05:03.871607+0000","last_became_peered":"2023-07-18T03:05:03.871607+0000","last_unstale":"2023-07-18T03:06:24.260914+0000","last_undegraded":"2023-07-18T03:06:24.260914+0000","last_fullsized":"2023-07-18T03:06:24.260914+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:13:21.796882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.751181+0000","last_change":"2023-07-18T03:04:58.847933+0000","last_active":"2023-07-18T03:06:24.751181+0000","last_peered":"2023-07-18T03:06:24.751181+0000","last_clean":"2023-07-18T03:06:24.751181+0000","last_became_active":"2023-07-18T03:04:58.847630+0000","last_became_peered":"2023-07-18T03:04:58.847630+0000","last_unstale":"2023-07-18T03:06:24.751181+0000","last_undegraded":"2023-07-18T03:06:24.751181+0000","last_fullsized":"2023-07-18T03:06:24.751181+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:34:44.655423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.258453+0000","last_change":"2023-07-18T03:04:59.803981+0000","last_active":"2023-07-18T03:06:24.258453+0000","last_peered":"2023-07-18T03:06:24.258453+0000","last_clean":"2023-07-18T03:06:24.258453+0000","last_became_active":"2023-07-18T03:04:59.803692+0000","last_became_peered":"2023-07-18T03:04:59.803692+0000","last_unstale":"2023-07-18T03:06:24.258453+0000","last_undegraded":"2023-07-18T03:06:24.258453+0000","last_fullsized":"2023-07-18T03:06:24.258453+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:58:45.567557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.257138+0000","last_change":"2023-07-18T03:04:59.803770+0000","last_active":"2023-07-18T03:06:24.257138+0000","last_peered":"2023-07-18T03:06:24.257138+0000","last_clean":"2023-07-18T03:06:24.257138+0000","last_became_active":"2023-07-18T03:04:59.803355+0000","last_became_peered":"2023-07-18T03:04:59.803355+0000","last_unstale":"2023-07-18T03:06:24.257138+0000","last_undegraded":"2023-07-18T03:06:24.257138+0000","last_fullsized":"2023-07-18T03:06:24.257138+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:07:11.955948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.256207+0000","last_change":"2023-07-18T03:04:59.808946+0000","last_active":"2023-07-18T03:06:24.256207+0000","last_peered":"2023-07-18T03:06:24.256207+0000","last_clean":"2023-07-18T03:06:24.256207+0000","last_became_active":"2023-07-18T03:04:59.808646+0000","last_became_peered":"2023-07-18T03:04:59.808646+0000","last_unstale":"2023-07-18T03:06:24.256207+0000","last_undegraded":"2023-07-18T03:06:24.256207+0000","last_fullsized":"2023-07-18T03:06:24.256207+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:10:55.288876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255438+0000","last_change":"2023-07-18T03:04:59.811933+0000","last_active":"2023-07-18T03:06:24.255438+0000","last_peered":"2023-07-18T03:06:24.255438+0000","last_clean":"2023-07-18T03:06:24.255438+0000","last_became_active":"2023-07-18T03:04:59.811620+0000","last_became_peered":"2023-07-18T03:04:59.811620+0000","last_unstale":"2023-07-18T03:06:24.255438+0000","last_undegraded":"2023-07-18T03:06:24.255438+0000","last_fullsized":"2023-07-18T03:06:24.255438+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:21:21.164245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-18T03:06:23.265182+0000","last_change":"2023-07-18T03:04:59.803942+0000","last_active":"2023-07-18T03:06:23.265182+0000","last_peered":"2023-07-18T03:06:23.265182+0000","last_clean":"2023-07-18T03:06:23.265182+0000","last_became_active":"2023-07-18T03:04:59.803576+0000","last_became_peered":"2023-07-18T03:04:59.803576+0000","last_unstale":"2023-07-18T03:06:23.265182+0000","last_undegraded":"2023-07-18T03:06:23.265182+0000","last_fullsized":"2023-07-18T03:06:23.265182+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:53:38.163912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.751223+0000","last_change":"2023-07-18T03:04:59.802594+0000","last_active":"2023-07-18T03:06:24.751223+0000","last_peered":"2023-07-18T03:06:24.751223+0000","last_clean":"2023-07-18T03:06:24.751223+0000","last_became_active":"2023-07-18T03:04:59.802215+0000","last_became_peered":"2023-07-18T03:04:59.802215+0000","last_unstale":"2023-07-18T03:06:24.751223+0000","last_undegraded":"2023-07-18T03:06:24.751223+0000","last_fullsized":"2023-07-18T03:06:24.751223+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:57:57.598480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.257805+0000","last_change":"2023-07-18T03:04:59.803519+0000","last_active":"2023-07-18T03:06:24.257805+0000","last_peered":"2023-07-18T03:06:24.257805+0000","last_clean":"2023-07-18T03:06:24.257805+0000","last_became_active":"2023-07-18T03:04:59.803170+0000","last_became_peered":"2023-07-18T03:04:59.803170+0000","last_unstale":"2023-07-18T03:06:24.257805+0000","last_undegraded":"2023-07-18T03:06:24.257805+0000","last_fullsized":"2023-07-18T03:06:24.257805+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:40:11.785249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255963+0000","last_change":"2023-07-18T03:04:59.813494+0000","last_active":"2023-07-18T03:06:24.255963+0000","last_peered":"2023-07-18T03:06:24.255963+0000","last_clean":"2023-07-18T03:06:24.255963+0000","last_became_active":"2023-07-18T03:04:59.813175+0000","last_became_peered":"2023-07-18T03:04:59.813175+0000","last_unstale":"2023-07-18T03:06:24.255963+0000","last_undegraded":"2023-07-18T03:06:24.255963+0000","last_fullsized":"2023-07-18T03:06:24.255963+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:58:52.954498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.375669+0000","last_change":"2023-07-18T03:04:59.806597+0000","last_active":"2023-07-18T03:06:24.375669+0000","last_peered":"2023-07-18T03:06:24.375669+0000","last_clean":"2023-07-18T03:06:24.375669+0000","last_became_active":"2023-07-18T03:04:59.806220+0000","last_became_peered":"2023-07-18T03:04:59.806220+0000","last_unstale":"2023-07-18T03:06:24.375669+0000","last_undegraded":"2023-07-18T03:06:24.375669+0000","last_fullsized":"2023-07-18T03:06:24.375669+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:07:15.022728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.261137+0000","last_change":"2023-07-18T03:04:59.809051+0000","last_active":"2023-07-18T03:06:24.261137+0000","last_peered":"2023-07-18T03:06:24.261137+0000","last_clean":"2023-07-18T03:06:24.261137+0000","last_became_active":"2023-07-18T03:04:59.808783+0000","last_became_peered":"2023-07-18T03:04:59.808783+0000","last_unstale":"2023-07-18T03:06:24.261137+0000","last_undegraded":"2023-07-18T03:06:24.261137+0000","last_fullsized":"2023-07-18T03:06:24.261137+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:55:28.682728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.375113+0000","last_change":"2023-07-18T03:04:59.805741+0000","last_active":"2023-07-18T03:06:24.375113+0000","last_peered":"2023-07-18T03:06:24.375113+0000","last_clean":"2023-07-18T03:06:24.375113+0000","last_became_active":"2023-07-18T03:04:59.805412+0000","last_became_peered":"2023-07-18T03:04:59.805412+0000","last_unstale":"2023-07-18T03:06:24.375113+0000","last_undegraded":"2023-07-18T03:06:24.375113+0000","last_fullsized":"2023-07-18T03:06:24.375113+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:56:58.924751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.261688+0000","last_change":"2023-07-18T03:04:59.812052+0000","last_active":"2023-07-18T03:06:24.261688+0000","last_peered":"2023-07-18T03:06:24.261688+0000","last_clean":"2023-07-18T03:06:24.261688+0000","last_became_active":"2023-07-18T03:04:59.811756+0000","last_became_peered":"2023-07-18T03:04:59.811756+0000","last_unstale":"2023-07-18T03:06:24.261688+0000","last_undegraded":"2023-07-18T03:06:24.261688+0000","last_fullsized":"2023-07-18T03:06:24.261688+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:41:23.926360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.750840+0000","last_change":"2023-07-18T03:04:59.803009+0000","last_active":"2023-07-18T03:06:24.750840+0000","last_peered":"2023-07-18T03:06:24.750840+0000","last_clean":"2023-07-18T03:06:24.750840+0000","last_became_active":"2023-07-18T03:04:59.802687+0000","last_became_peered":"2023-07-18T03:04:59.802687+0000","last_unstale":"2023-07-18T03:06:24.750840+0000","last_undegraded":"2023-07-18T03:06:24.750840+0000","last_fullsized":"2023-07-18T03:06:24.750840+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:31:26.686903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.257288+0000","last_change":"2023-07-18T03:04:59.795562+0000","last_active":"2023-07-18T03:06:24.257288+0000","last_peered":"2023-07-18T03:06:24.257288+0000","last_clean":"2023-07-18T03:06:24.257288+0000","last_became_active":"2023-07-18T03:04:59.795241+0000","last_became_peered":"2023-07-18T03:04:59.795241+0000","last_unstale":"2023-07-18T03:06:24.257288+0000","last_undegraded":"2023-07-18T03:06:24.257288+0000","last_fullsized":"2023-07-18T03:06:24.257288+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:44:43.907380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281146,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64741,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":352,"num_read_kb":246,"num_write":154,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":194223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449118,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":999424,"data_stored":507139,"data_compressed":23524,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.56000000000000005,"15min":0.55300000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.434},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.52200000000000002,"15min":0.51400000000000001},"min":{"1min":0.41099999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.45900000000000002}]},{"osd":1,"last update":"Tue Jul 18 03:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.64300000000000002,"15min":0.63500000000000001},"min":{"1min":0.47799999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.85299999999999998,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.746},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.624,"15min":0.621},"min":{"1min":0.47099999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.82299999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.60399999999999998}]},{"osd":2,"last update":"Tue Jul 18 03:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.48299999999999998,"15min":0.47399999999999998},"min":{"1min":0.35199999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.71399999999999997,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.48999999999999999,"15min":0.47599999999999998},"min":{"1min":0.36699999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.34799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":4,"last update":"Tue Jul 18 03:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.66500000000000004,"15min":0.65100000000000002},"min":{"1min":0.53200000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.495},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.64200000000000002,"15min":0.63500000000000001},"min":{"1min":0.48999999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.84799999999999998,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.66300000000000003}]},{"osd":5,"last update":"Tue Jul 18 03:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.622,"15min":0.621},"min":{"1min":0.45400000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.82099999999999995,"5min":1.375,"15min":1.375},"last":0.374},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.61399999999999999,"15min":0.59299999999999997},"min":{"1min":0.378,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.69799999999999995}]},{"osd":6,"last update":"Tue Jul 18 03:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.68400000000000005,"15min":0.67000000000000004},"min":{"1min":0.52200000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.70499999999999996,"15min":0.69299999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.57499999999999996}]},{"osd":7,"last update":"Tue Jul 18 03:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.79200000000000004,"15min":0.78000000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1319999999999999,"5min":1.248,"15min":1.248},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.76900000000000002,"15min":0.76300000000000001},"min":{"1min":0.56499999999999995,"5min":0.502,"15min":0.502},"max":{"1min":1.024,"5min":1.288,"15min":1.288},"last":0.71399999999999997}]},{"osd":8,"last update":"Tue Jul 18 03:06:00 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.622},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.64500000000000002}]},{"osd":9,"last update":"Tue Jul 18 03:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.78300000000000003,"15min":0.76800000000000002},"min":{"1min":0.66700000000000004,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.145,"5min":1.304,"15min":1.304},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.81899999999999995,"15min":0.80400000000000005},"min":{"1min":0.63400000000000001,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1659999999999999,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.77100000000000002}]},{"osd":10,"last update":"Tue Jul 18 03:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.78200000000000003,"15min":0.77200000000000002},"min":{"1min":0.57999999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.054,"5min":1.163,"15min":1.163},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.79900000000000004,"15min":0.78900000000000003},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.026,"5min":1.496,"15min":1.496},"last":0.73299999999999998}]}]},{"osd":10,"up_from":71,"seq":304942678051,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1112,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1138688,"data_stored":1155174,"data_compressed":59283,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.84699999999999998}]},{"osd":1,"last update":"Tue Jul 18 03:05:56 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47999999999999998,"15min":0.48199999999999998},"min":{"1min":0.34399999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.51000000000000001,"15min":0.51600000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.58899999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.498}]},{"osd":2,"last update":"Tue Jul 18 03:05:56 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.63600000000000001,"15min":0.63},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82599999999999996,"5min":0.87,"15min":0.87},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64300000000000002,"15min":0.64200000000000002},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.752,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.64600000000000002}]},{"osd":3,"last update":"Tue Jul 18 03:05:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.68300000000000005,"15min":0.68899999999999995},"min":{"1min":0.48699999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.65700000000000003,"15min":0.66300000000000003},"min":{"1min":0.442,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82799999999999996,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.71899999999999997}]},{"osd":4,"last update":"Tue Jul 18 03:05:56 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.60899999999999999,"15min":0.61699999999999999},"min":{"1min":0.37,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72099999999999997,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77400000000000002,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.61899999999999999}]},{"osd":5,"last update":"Tue Jul 18 03:05:56 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.71499999999999997,"15min":0.72099999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.80000000000000004,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.749,"5min":0.75,"15min":0.75},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.86599999999999999,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.57399999999999995}]},{"osd":6,"last update":"Tue Jul 18 03:05:56 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.75600000000000001,"15min":0.76200000000000001},"min":{"1min":0.58599999999999997,"5min":0.501,"15min":0.501},"max":{"1min":0.88800000000000001,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.80200000000000005,"15min":0.80400000000000005},"min":{"1min":0.53400000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.78500000000000003}]},{"osd":7,"last update":"Tue Jul 18 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.55800000000000005}]},{"osd":8,"last update":"Tue Jul 18 03:05:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.80100000000000005,"15min":0.80800000000000005},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.91400000000000003,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.81799999999999995,"15min":0.82499999999999996},"min":{"1min":0.60699999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":0.95999999999999996,"5min":1.032,"15min":1.032},"last":0.66900000000000004}]},{"osd":9,"last update":"Tue Jul 18 03:05:56 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.84399999999999997,"15min":0.84599999999999997},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.85499999999999998,"15min":0.85799999999999998},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.018,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.86199999999999999}]},{"osd":11,"last update":"Tue Jul 18 03:06:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.82699999999999996,"15min":0.83199999999999996},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.96599999999999997,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.748},{"interface":"front","average":{"1min":0.745,"5min":0.80000000000000004,"15min":0.80900000000000005},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.92800000000000005,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.81699999999999995}]}]},{"osd":9,"up_from":66,"seq":283467841574,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":1015808,"data_stored":511449,"data_compressed":23524,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:05:34 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.54900000000000004,"15min":0.55800000000000005},"min":{"1min":0.35799999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.69699999999999995,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.36899999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.84399999999999997,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.36099999999999999}]},{"osd":1,"last update":"Tue Jul 18 03:05:34 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.71899999999999997,"15min":0.73099999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0169999999999999,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.72999999999999998,"15min":0.74199999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.91500000000000004,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.52300000000000002}]},{"osd":2,"last update":"Tue Jul 18 03:05:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.70999999999999996,"15min":0.71799999999999997},"min":{"1min":0.52000000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.85699999999999998,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.74299999999999999,"15min":0.75},"min":{"1min":0.48499999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.98099999999999998,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.45000000000000001}]},{"osd":3,"last update":"Tue Jul 18 03:05:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.67000000000000004,"15min":0.68799999999999994},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88300000000000001,"5min":1.038,"15min":1.038},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.64900000000000002,"15min":0.66300000000000003},"min":{"1min":0.38700000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.90100000000000002,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.40200000000000002}]},{"osd":4,"last update":"Tue Jul 18 03:05:34 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.81699999999999995,"15min":0.83299999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.97299999999999998,"5min":1.47,"15min":1.47},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.80700000000000005,"15min":0.82699999999999996},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.95699999999999996,"5min":1.619,"15min":1.619},"last":0.80500000000000005}]},{"osd":5,"last update":"Tue Jul 18 03:05:34 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.81999999999999995,"15min":0.82999999999999996},"min":{"1min":0.53400000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.95499999999999996,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.80600000000000005,"15min":0.81699999999999995},"min":{"1min":0.499,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1080000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.54900000000000004}]},{"osd":6,"last update":"Tue Jul 18 03:04:30 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.64300000000000002}]},{"osd":7,"last update":"Tue Jul 18 03:05:34 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.89100000000000001,"15min":0.91000000000000003},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0600000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.627},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.94599999999999995,"15min":0.93999999999999995},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":4.4009999999999998,"5min":4.4009999999999998,"15min":4.4009999999999998},"last":0.73399999999999999}]},{"osd":8,"last update":"Tue Jul 18 03:05:34 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.90900000000000003,"15min":0.92700000000000005},"min":{"1min":0.63800000000000001,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.077,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.93700000000000006,"15min":0.95499999999999996},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.0880000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.75}]},{"osd":10,"last update":"Tue Jul 18 03:05:51 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.88400000000000001,"15min":0.90200000000000002},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.96299999999999997,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.872},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.91300000000000003,"15min":0.93899999999999995},"min":{"1min":0.63700000000000001,"5min":0.63,"15min":0.63},"max":{"1min":0.94799999999999995,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.82699999999999996}]},{"osd":11,"last update":"Tue Jul 18 03:06:05 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.67100000000000004}]}]},{"osd":8,"up_from":58,"seq":249108103211,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":1007616,"data_stored":507251,"data_compressed":23524,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:06:13 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.69499999999999995,"15min":0.64100000000000001},"min":{"1min":0.433,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":7.0359999999999996,"5min":7.0359999999999996,"15min":7.0359999999999996},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58499999999999996,"15min":0.57099999999999995},"min":{"1min":0.443,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91500000000000004,"5min":2.883,"15min":2.883},"last":0.65900000000000003}]},{"osd":1,"last update":"Tue Jul 18 03:06:13 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.65400000000000003,"15min":0.63200000000000001},"min":{"1min":0.45300000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.879,"5min":3.3340000000000001,"15min":3.3340000000000001},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.64100000000000001,"15min":0.60599999999999998},"min":{"1min":0.48999999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.95099999999999996,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":0.82499999999999996}]},{"osd":2,"last update":"Tue Jul 18 03:06:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.503,"15min":0.46700000000000003},"min":{"1min":0.38300000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.0900000000000001,"5min":3.2850000000000001,"15min":3.2850000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.52200000000000002,"15min":0.498},"min":{"1min":0.39700000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.78900000000000003,"5min":2.012,"15min":2.012},"last":0.64200000000000002}]},{"osd":3,"last update":"Tue Jul 18 03:06:13 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.64200000000000002,"15min":0.622},"min":{"1min":0.45800000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.91000000000000003,"5min":2.9100000000000001,"15min":2.9100000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.66800000000000004,"15min":0.64900000000000002},"min":{"1min":0.50800000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.82699999999999996,"5min":3.6059999999999999,"15min":3.6059999999999999},"last":0.69099999999999995}]},{"osd":4,"last update":"Tue Jul 18 03:06:13 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.65800000000000003,"15min":0.63200000000000001},"min":{"1min":0.42899999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.006,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":0.877},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.71499999999999997,"15min":0.69399999999999995},"min":{"1min":0.50600000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.99099999999999999,"5min":3.3650000000000002,"15min":3.3650000000000002},"last":1.05}]},{"osd":5,"last update":"Tue Jul 18 03:06:13 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.626,"15min":0.59599999999999997},"min":{"1min":0.40899999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.97799999999999998,"5min":2.4279999999999999,"15min":2.4279999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.63800000000000001,"15min":0.60699999999999998},"min":{"1min":0.46300000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0549999999999999,"5min":2.7410000000000001,"15min":2.7410000000000001},"last":0.79700000000000004}]},{"osd":6,"last update":"Tue Jul 18 03:06:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.753,"15min":0.72299999999999998},"min":{"1min":0.56499999999999995,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.038,"5min":3.5779999999999998,"15min":3.5779999999999998},"last":0.751},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.76400000000000001,"15min":0.74099999999999999},"min":{"1min":0.628,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0249999999999999,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":0.89400000000000002}]},{"osd":7,"last update":"Tue Jul 18 03:06:13 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.76200000000000001,"15min":0.72999999999999998},"min":{"1min":0.57599999999999996,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.167,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.76000000000000001,"15min":0.72799999999999998},"min":{"1min":0.64200000000000002,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1100000000000001,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":0.91600000000000004}]},{"osd":9,"last update":"Tue Jul 18 03:05:36 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.81000000000000005,"15min":0.78200000000000003},"min":{"1min":0.60599999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":3.544,"5min":3.544,"15min":3.544},"last":1.008},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.81000000000000005,"15min":0.79000000000000004},"min":{"1min":0.61899999999999999,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.1230000000000002,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":0.996}]},{"osd":10,"last update":"Tue Jul 18 03:05:54 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.82299999999999995,"15min":0.81499999999999995},"min":{"1min":0.61499999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0249999999999999,"5min":2.8570000000000002,"15min":2.8570000000000002},"last":1.0349999999999999},{"interface":"front","average":{"1min":0.871,"5min":0.879,"15min":0.88100000000000001},"min":{"1min":0.59099999999999997,"5min":0.502,"15min":0.502},"max":{"1min":1.02,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":1.0800000000000001}]},{"osd":11,"last update":"Tue Jul 18 03:06:03 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.97899999999999998}]}]},{"osd":7,"up_from":51,"seq":219043332143,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":1036,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":1060864,"data_stored":1103085,"data_compressed":59283,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.55500000000000005,"15min":0.54300000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.495},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59999999999999998,"15min":0.60299999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82599999999999996,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.44500000000000001}]},{"osd":1,"last update":"Tue Jul 18 03:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42199999999999999,"15min":0.41999999999999998},"min":{"1min":0.32900000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.55800000000000005,"5min":0.745,"15min":0.745},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.439,"15min":0.433},"min":{"1min":0.33300000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.628,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.64900000000000002}]},{"osd":2,"last update":"Tue Jul 18 03:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58399999999999996,"15min":0.56999999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.80300000000000005,"5min":1.111,"15min":1.111},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59599999999999997,"15min":0.58299999999999996},"min":{"1min":0.42399999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.752,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.68100000000000005}]},{"osd":3,"last update":"Tue Jul 18 03:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.628,"15min":0.60399999999999998},"min":{"1min":0.41099999999999998,"5min":0.38,"15min":0.38},"max":{"1min":2.6669999999999998,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.63400000000000001,"15min":0.64500000000000002},"min":{"1min":0.42399999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.76800000000000002,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.72299999999999998}]},{"osd":4,"last update":"Tue Jul 18 03:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.48899999999999999,"15min":0.48399999999999999},"min":{"1min":0.39000000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.62,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.749},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54900000000000004,"15min":0.54800000000000004},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.61199999999999999}]},{"osd":5,"last update":"Tue Jul 18 03:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.66800000000000004,"15min":0.65000000000000002},"min":{"1min":0.51400000000000001,"5min":0.501,"15min":0.501},"max":{"1min":0.82999999999999996,"5min":1.077,"15min":1.077},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.66200000000000003,"15min":0.65400000000000003},"min":{"1min":0.52000000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.80800000000000005,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.80000000000000004}]},{"osd":6,"last update":"Tue Jul 18 03:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.67100000000000004,"15min":0.67300000000000004},"min":{"1min":0.46999999999999997,"5min":0.435,"15min":0.435},"max":{"1min":0.83699999999999997,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.874},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.71699999999999997,"15min":0.70299999999999996},"min":{"1min":0.49399999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.95199999999999996}]},{"osd":8,"last update":"Tue Jul 18 03:06:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.70599999999999996,"15min":0.69199999999999995},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.95699999999999996,"5min":1.022,"15min":1.022},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.66700000000000004,"15min":0.64900000000000002},"min":{"1min":0.52700000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.83099999999999996,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.92600000000000005}]},{"osd":9,"last update":"Tue Jul 18 03:05:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.76600000000000001,"15min":0.76400000000000001},"min":{"1min":0.56499999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":1.0189999999999999}]},{"osd":10,"last update":"Tue Jul 18 03:05:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.72099999999999997,"15min":0.72399999999999998},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.94099999999999995,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.75600000000000001,"15min":0.754},"min":{"1min":0.61499999999999999,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.92900000000000005,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.99299999999999999}]},{"osd":11,"last update":"Tue Jul 18 03:06:04 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":1.0620000000000001}]}]},{"osd":6,"up_from":46,"seq":197568495668,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":991232,"data_stored":502529,"data_compressed":23524,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.53400000000000003,"15min":0.56299999999999994},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.58199999999999996,"5min":3.1579999999999999,"15min":3.1579999999999999},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.53400000000000003,"15min":0.55000000000000004},"min":{"1min":0.35699999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.64000000000000001,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.41599999999999998}]},{"osd":1,"last update":"Tue Jul 18 03:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.627,"15min":0.60299999999999998},"min":{"1min":0.38800000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.88600000000000001,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.72799999999999998,"15min":0.74099999999999999},"min":{"1min":0.495,"5min":0.44,"15min":0.44},"max":{"1min":0.90500000000000003,"5min":3.0369999999999999,"15min":3.0369999999999999},"last":0.53300000000000003}]},{"osd":2,"last update":"Tue Jul 18 03:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.65700000000000003,"15min":0.65600000000000003},"min":{"1min":0.48299999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.64700000000000002,"15min":0.65300000000000002},"min":{"1min":0.48399999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.94299999999999995,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.78000000000000003}]},{"osd":3,"last update":"Tue Jul 18 03:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.56499999999999995,"15min":0.57299999999999995},"min":{"1min":0.38800000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.69899999999999995,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.71199999999999997,"15min":0.748},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.748,"5min":3.226,"15min":3.226},"last":0.56599999999999995}]},{"osd":4,"last update":"Tue Jul 18 03:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.44600000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.92500000000000004,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.70199999999999996,"15min":0.69899999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.92700000000000005,"5min":1.01,"15min":1.01},"last":0.66900000000000004}]},{"osd":5,"last update":"Tue Jul 18 03:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.81200000000000006,"15min":0.84499999999999997},"min":{"1min":0.59199999999999997,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.88600000000000001,"5min":3.125,"15min":3.125},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.76600000000000001,"15min":0.78200000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.98699999999999999,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.71499999999999997}]},{"osd":7,"last update":"Tue Jul 18 03:05:46 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.83399999999999996,"15min":0.83999999999999997},"min":{"1min":0.61399999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.97599999999999998,"5min":3.0870000000000002,"15min":3.0870000000000002},"last":0.746},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.752,"15min":0.73399999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.01,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.83299999999999996}]},{"osd":8,"last update":"Tue Jul 18 03:06:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.85099999999999998,"15min":0.876},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0640000000000001,"5min":3.1909999999999998,"15min":3.1909999999999998},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.99099999999999999,"5min":1.046,"15min":1.046},"last":0.85399999999999998}]},{"osd":9,"last update":"Tue Jul 18 03:05:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.754,"15min":0.76400000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.97899999999999998,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.84899999999999998,"15min":0.85599999999999998},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.008,"5min":1.137,"15min":1.137},"last":0.63300000000000001}]},{"osd":10,"last update":"Tue Jul 18 03:05:54 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.876,"15min":0.878},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.044,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.82099999999999995,"15min":0.81899999999999995},"min":{"1min":0.66900000000000004,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0429999999999999,"5min":1.083,"15min":1.083},"last":0.79700000000000004}]},{"osd":11,"last update":"Tue Jul 18 03:06:05 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.873},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.84399999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":1032,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":1056768,"data_stored":563276,"data_compressed":23524,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.55100000000000005,"15min":0.52400000000000002},"min":{"1min":0.37,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.96299999999999997,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.54400000000000004,"15min":0.52400000000000002},"min":{"1min":0.35299999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.93100000000000005,"5min":1.421,"15min":1.421},"last":0.42199999999999999}]},{"osd":2,"last update":"Tue Jul 18 03:05:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.56599999999999995,"15min":0.51900000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.56899999999999995,"15min":0.54400000000000004},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.59499999999999997}]},{"osd":3,"last update":"Tue Jul 18 03:06:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.622,"15min":0.59999999999999998},"min":{"1min":0.378,"5min":0.312,"15min":0.312},"max":{"1min":1.014,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.627,"15min":0.627},"min":{"1min":0.47999999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.026,"5min":1.504,"15min":1.504},"last":0.54900000000000004}]},{"osd":4,"last update":"Tue Jul 18 03:05:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.45900000000000002,"15min":0.40699999999999997},"min":{"1min":0.317,"5min":0.215,"15min":0.215},"max":{"1min":1.784,"5min":1.784,"15min":1.784},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.47699999999999998,"15min":0.42499999999999999},"min":{"1min":0.34599999999999997,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.76700000000000002,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.501}]},{"osd":5,"last update":"Tue Jul 18 03:06:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.61199999999999999,"15min":0.53100000000000003},"min":{"1min":0.57999999999999996,"5min":0.375,"15min":0.375},"max":{"1min":0.85299999999999998,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.623,"15min":0.56699999999999995},"min":{"1min":0.59399999999999997,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.872,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.63400000000000001}]},{"osd":6,"last update":"Tue Jul 18 03:05:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.64900000000000002,"15min":0.60499999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.4299999999999999,"5min":2.6600000000000001,"15min":2.6600000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.65200000000000002,"15min":0.63800000000000001},"min":{"1min":0.46400000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.94799999999999995,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.75600000000000001}]},{"osd":7,"last update":"Tue Jul 18 03:05:51 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.59599999999999997,"15min":0.56399999999999995},"min":{"1min":0.38100000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.67400000000000004,"15min":0.65600000000000003},"min":{"1min":0.5,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.5409999999999999,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":0.67500000000000004}]},{"osd":8,"last update":"Tue Jul 18 03:06:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77500000000000002,"15min":0.78200000000000003},"min":{"1min":0.52900000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.98399999999999999,"5min":2.6419999999999999,"15min":2.6419999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.72199999999999998,"15min":0.68200000000000005},"min":{"1min":0.54000000000000004,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.036,"5min":1.698,"15min":1.698},"last":0.80200000000000005}]},{"osd":9,"last update":"Tue Jul 18 03:05:40 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.85599999999999998,"15min":0.85799999999999998},"min":{"1min":0.59099999999999997,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.458,"5min":2.6789999999999998,"15min":2.6789999999999998},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.76200000000000001,"15min":0.746},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.5409999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.83799999999999997}]},{"osd":10,"last update":"Tue Jul 18 03:05:56 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.78300000000000003,"15min":0.76900000000000002},"min":{"1min":0.57999999999999996,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.77000000000000002,"15min":0.76200000000000001},"min":{"1min":0.56899999999999995,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":0.71799999999999997}]},{"osd":11,"last update":"Tue Jul 18 03:06:01 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.77300000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":991232,"data_stored":484493,"data_compressed":17573,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Jul 18 03:05:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.59599999999999997,"15min":0.55200000000000005},"min":{"1min":0.45900000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.0149999999999999,"5min":1.075,"15min":1.075},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.61099999999999999,"15min":0.56100000000000005},"min":{"1min":0.38100000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.97699999999999998,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.63300000000000001}]},{"osd":2,"last update":"Tue Jul 18 03:05:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.60399999999999998,"15min":0.55200000000000005},"min":{"1min":0.45200000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.873,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.64100000000000001,"15min":0.56799999999999995},"min":{"1min":0.42799999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.88,"5min":1.204,"15min":1.204},"last":0.77600000000000002}]},{"osd":3,"last update":"Tue Jul 18 03:06:00 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.442}]},{"osd":4,"last update":"Tue Jul 18 03:05:40 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.65300000000000002,"15min":0.61299999999999999},"min":{"1min":0.437,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.63500000000000001,"15min":0.60199999999999998},"min":{"1min":0.47399999999999998,"5min":0.433,"15min":0.433},"max":{"1min":0.93600000000000005,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.52600000000000002}]},{"osd":5,"last update":"Tue Jul 18 03:05:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.64700000000000002,"15min":0.63500000000000001},"min":{"1min":0.46899999999999997,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.028,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.68200000000000005,"15min":0.65900000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.998,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.79600000000000004}]},{"osd":6,"last update":"Tue Jul 18 03:05:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59899999999999998,"15min":0.58999999999999997},"min":{"1min":0.439,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.91900000000000004,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.624,"15min":0.59999999999999998},"min":{"1min":0.42099999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.94699999999999995,"5min":1.216,"15min":1.216},"last":0.48599999999999999}]},{"osd":7,"last update":"Tue Jul 18 03:06:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.74299999999999999}]},{"osd":8,"last update":"Tue Jul 18 03:06:05 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.746},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.95999999999999996,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":6.96},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.81999999999999995,"15min":0.82999999999999996},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.284,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":6.9909999999999997}]},{"osd":9,"last update":"Tue Jul 18 03:05:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.79000000000000004,"15min":0.79100000000000004},"min":{"1min":0.51400000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.107,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.871,"15min":0.877},"min":{"1min":0.52200000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.218,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.71399999999999997}]},{"osd":10,"last update":"Tue Jul 18 03:05:52 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.91300000000000003,"15min":0.92300000000000004},"min":{"1min":0.68000000000000005,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.054,"5min":1.282,"15min":1.282},"last":6.9359999999999999},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.78900000000000003,"15min":0.79200000000000004},"min":{"1min":0.57599999999999996,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.72599999999999998}]},{"osd":11,"last update":"Tue Jul 18 03:06:16 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.81299999999999994,"15min":0.80900000000000005},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.0760000000000001,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.88600000000000001,"15min":0.88200000000000001},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.3160000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.80300000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":1056,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1081344,"data_stored":564968,"data_compressed":23524,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55200000000000005,"15min":0.53500000000000003},"min":{"1min":0.42999999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.622,"5min":0.57499999999999996,"15min":0.55200000000000005},"min":{"1min":0.38500000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.726,"5min":1.726,"15min":1.726},"last":0.68799999999999994}]},{"osd":1,"last update":"Tue Jul 18 03:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.57399999999999995,"15min":0.51400000000000001},"min":{"1min":0.50800000000000001,"5min":0.311,"15min":0.311},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.57099999999999995,"15min":0.50900000000000001},"min":{"1min":0.42699999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.92300000000000004,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.70799999999999996}]},{"osd":3,"last update":"Tue Jul 18 03:06:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59299999999999997,"15min":0.56299999999999994},"min":{"1min":0.439,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93899999999999995,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.64400000000000002,"15min":0.59299999999999997},"min":{"1min":0.40899999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.127,"5min":3.234,"15min":3.234},"last":0.622}]},{"osd":4,"last update":"Tue Jul 18 03:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.83299999999999996}]},{"osd":5,"last update":"Tue Jul 18 03:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.48399999999999999,"15min":0.45300000000000001},"min":{"1min":0.40300000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.502,"15min":0.46000000000000002},"min":{"1min":0.38400000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.49199999999999999}]},{"osd":6,"last update":"Tue Jul 18 03:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.65800000000000003,"15min":0.64000000000000001},"min":{"1min":0.46600000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.70799999999999996,"15min":0.68799999999999994},"min":{"1min":0.53800000000000003,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":0.72899999999999998}]},{"osd":7,"last update":"Tue Jul 18 03:06:03 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":0.999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.74299999999999999}]},{"osd":8,"last update":"Tue Jul 18 03:06:08 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.66300000000000003,"15min":0.628},"min":{"1min":0.55100000000000005,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.873},{"interface":"front","average":{"1min":0.749,"5min":0.68200000000000005,"15min":0.65800000000000003},"min":{"1min":0.60199999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.96099999999999997,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.76200000000000001}]},{"osd":9,"last update":"Tue Jul 18 03:05:34 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.75,"15min":0.74099999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.825,"5min":1.825,"15min":1.825},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.80300000000000005,"15min":0.78900000000000003},"min":{"1min":0.61099999999999999,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.92500000000000004}]},{"osd":10,"last update":"Tue Jul 18 03:05:53 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.80700000000000005,"15min":0.78500000000000003},"min":{"1min":0.55100000000000005,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.76100000000000001,"15min":0.745},"min":{"1min":0.59499999999999997,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.89000000000000001}]},{"osd":11,"last update":"Tue Jul 18 03:05:58 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.76,"5min":1.76,"15min":1.76},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.81000000000000005}]}]},{"osd":3,"up_from":25,"seq":107374182466,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":1003520,"data_stored":502723,"data_compressed":23524,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:06:25 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.499,"15min":0.42699999999999999},"min":{"1min":0.34999999999999998,"5min":0.312,"15min":0.312},"max":{"1min":1.8440000000000001,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.504,"15min":0.433},"min":{"1min":0.35899999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.42899999999999999}]},{"osd":1,"last update":"Tue Jul 18 03:06:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.60499999999999998,"15min":0.57199999999999995},"min":{"1min":0.45600000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.79300000000000004,"5min":1.115,"15min":1.115},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.66700000000000004,"15min":0.59499999999999997},"min":{"1min":0.44400000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":0.78100000000000003}]},{"osd":2,"last update":"Tue Jul 18 03:06:25 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.68100000000000005,"15min":0.61399999999999999},"min":{"1min":0.437,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0960000000000001,"5min":1.26,"15min":1.26},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.63700000000000001,"15min":0.60299999999999998},"min":{"1min":0.46600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.048,"5min":1.212,"15min":1.212},"last":0.81799999999999995}]},{"osd":4,"last update":"Tue Jul 18 03:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.66000000000000003,"15min":0.61899999999999999},"min":{"1min":0.48299999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.66700000000000004,"15min":0.61799999999999999},"min":{"1min":0.53300000000000003,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.046,"5min":1.143,"15min":1.143},"last":0.66100000000000003}]},{"osd":5,"last update":"Tue Jul 18 03:05:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.63200000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84599999999999997,"5min":1.218,"15min":1.218},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.71299999999999997,"15min":0.68200000000000005},"min":{"1min":0.495,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.1240000000000001,"5min":1.218,"15min":1.218},"last":0.70799999999999996}]},{"osd":6,"last update":"Tue Jul 18 03:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.66900000000000004,"15min":0.64300000000000002},"min":{"1min":0.48099999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0740000000000001,"5min":1.385,"15min":1.385},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.60999999999999999,"15min":0.59999999999999998},"min":{"1min":0.434,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85999999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.56399999999999995}]},{"osd":7,"last update":"Tue Jul 18 03:05:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.73399999999999999,"15min":0.72199999999999998},"min":{"1min":0.51800000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.85899999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.622},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.76800000000000002,"15min":0.753},"min":{"1min":0.57199999999999995,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.153,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.81000000000000005}]},{"osd":8,"last update":"Tue Jul 18 03:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.76200000000000001}]},{"osd":9,"last update":"Tue Jul 18 03:05:31 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.76400000000000001,"15min":0.78400000000000003},"min":{"1min":0.498,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.83299999999999996,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.85399999999999998,"15min":0.873},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.99299999999999999,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.71899999999999997}]},{"osd":10,"last update":"Tue Jul 18 03:05:52 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.88700000000000001,"15min":0.90500000000000003},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":0.93600000000000005,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.876,"15min":0.88600000000000001},"min":{"1min":0.626,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.1850000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.85899999999999999}]},{"osd":11,"last update":"Tue Jul 18 03:06:03 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.74199999999999999}]}]},{"osd":4,"up_from":33,"seq":141733920829,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":991232,"data_stored":497407,"data_compressed":20545,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.56200000000000006,"15min":0.54600000000000004},"min":{"1min":0.39600000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.439},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.56899999999999995,"15min":0.54500000000000004},"min":{"1min":0.42799999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.48999999999999999}]},{"osd":1,"last update":"Tue Jul 18 03:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.39700000000000002,"15min":0.36499999999999999},"min":{"1min":0.28999999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.73599999999999999,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.437,"5min":0.40600000000000003,"15min":0.38},"min":{"1min":0.30099999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.70499999999999996,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.52400000000000002}]},{"osd":2,"last update":"Tue Jul 18 03:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.59699999999999998,"15min":0.57499999999999996},"min":{"1min":0.39200000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.93000000000000005,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.60499999999999998,"15min":0.56699999999999995},"min":{"1min":0.438,"5min":0.378,"15min":0.378},"max":{"1min":0.84099999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.88100000000000001}]},{"osd":3,"last update":"Tue Jul 18 03:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.64900000000000002,"15min":0.59499999999999997},"min":{"1min":0.45000000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":3.7669999999999999,"5min":3.7669999999999999,"15min":3.7669999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59499999999999997,"15min":0.58099999999999996},"min":{"1min":0.39000000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83699999999999997,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.626}]},{"osd":5,"last update":"Tue Jul 18 03:06:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.65400000000000003,"15min":0.61099999999999999},"min":{"1min":0.39300000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.92600000000000005,"5min":1.214,"15min":1.214},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.63900000000000001,"15min":0.60899999999999999},"min":{"1min":0.49299999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.83399999999999996,"5min":1.306,"15min":1.306},"last":0.80600000000000005}]},{"osd":6,"last update":"Tue Jul 18 03:05:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.71699999999999997,"15min":0.72599999999999998},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.42399999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.69199999999999995}]},{"osd":7,"last update":"Tue Jul 18 03:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60799999999999998,"15min":0.60199999999999998},"min":{"1min":0.39000000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.85899999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.65900000000000003,"15min":0.64900000000000002},"min":{"1min":0.499,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.94299999999999995,"5min":1.034,"15min":1.034},"last":0.60399999999999998}]},{"osd":8,"last update":"Tue Jul 18 03:06:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73999999999999999,"15min":0.73499999999999999},"min":{"1min":0.53300000000000003,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.99399999999999999,"5min":1.155,"15min":1.155},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.75800000000000001,"15min":0.74399999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.008,"5min":1.135,"15min":1.135},"last":0.82599999999999996}]},{"osd":9,"last update":"Tue Jul 18 03:05:33 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.72399999999999998,"15min":0.72199999999999998},"min":{"1min":0.54000000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.82299999999999995,"15min":0.81100000000000005},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":3.415,"5min":3.415,"15min":3.415},"last":0.95799999999999996}]},{"osd":10,"last update":"Tue Jul 18 03:05:59 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.78700000000000003,"15min":0.79600000000000004},"min":{"1min":0.48999999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.90200000000000002,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.68799999999999994,"15min":0.69799999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.79900000000000004,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.77300000000000002}]},{"osd":11,"last update":"Tue Jul 18 03:05:59 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":1.0940000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.745}]}]},{"osd":5,"up_from":38,"seq":163208757305,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1069056,"data_stored":1098721,"data_compressed":59283,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54900000000000004,"15min":0.53100000000000003},"min":{"1min":0.39400000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.79800000000000004,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.497,"5min":0.54000000000000004,"15min":0.53500000000000003},"min":{"1min":0.371,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.64700000000000002,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.38100000000000001}]},{"osd":1,"last update":"Tue Jul 18 03:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.57799999999999996,"15min":0.54400000000000004},"min":{"1min":0.40100000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.74299999999999999,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.56599999999999995,"15min":0.54800000000000004},"min":{"1min":0.42299999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.60499999999999998}]},{"osd":2,"last update":"Tue Jul 18 03:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.40400000000000003,"15min":0.379},"min":{"1min":0.30599999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.61499999999999999,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.437,"5min":0.42099999999999999,"15min":0.39800000000000002},"min":{"1min":0.32100000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.53300000000000003,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.42599999999999999}]},{"osd":3,"last update":"Tue Jul 18 03:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.59199999999999997,"15min":0.56899999999999995},"min":{"1min":0.434,"5min":0.372,"15min":0.372},"max":{"1min":0.76400000000000001,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.497},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59599999999999997,"15min":0.57199999999999995},"min":{"1min":0.45700000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.79000000000000004,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.57999999999999996}]},{"osd":4,"last update":"Tue Jul 18 03:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59199999999999997,"15min":0.57099999999999995},"min":{"1min":0.41599999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83799999999999997,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.61099999999999999,"15min":0.57199999999999995},"min":{"1min":0.41899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84399999999999997,"5min":0.871,"15min":0.871},"last":0.55200000000000005}]},{"osd":6,"last update":"Tue Jul 18 03:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.61599999999999999,"15min":0.60299999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.94299999999999995,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.64200000000000002,"15min":0.623},"min":{"1min":0.437,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85799999999999998,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":1.3999999999999999}]},{"osd":7,"last update":"Tue Jul 18 03:05:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.67300000000000004,"15min":0.67500000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.875,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.624},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.69299999999999995,"15min":0.69099999999999995},"min":{"1min":0.5,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.90100000000000002,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.65200000000000002}]},{"osd":8,"last update":"Tue Jul 18 03:06:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56200000000000006,"15min":0.57199999999999995},"min":{"1min":0.36399999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.75900000000000001,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.627,"15min":0.625},"min":{"1min":0.433,"5min":0.371,"15min":0.371},"max":{"1min":0.84699999999999998,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.67200000000000004}]},{"osd":9,"last update":"Tue Jul 18 03:05:34 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.76400000000000001,"15min":0.77100000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.90100000000000002,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.755,"15min":0.76500000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.73199999999999998}]},{"osd":10,"last update":"Tue Jul 18 03:05:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.73899999999999999,"15min":0.73299999999999998},"min":{"1min":0.57799999999999996,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.92400000000000004,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.748},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.74299999999999999,"15min":0.73899999999999999},"min":{"1min":0.54900000000000004,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.93300000000000005,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.69599999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34117,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34117,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34117,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35759,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35759,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35759,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35759,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35759,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-18T03:06:29.495 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-07-18T03:06:29.496 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-07-18T03:06:29.496 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-07-18T03:06:29.496 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph health --format=json 2023-07-18T03:06:30.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:29 smithi179 ceph-mon[129229]: pgmap v324: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s; 2 B/s, 0 objects/s recovering 2023-07-18T03:06:30.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:29 smithi179 ceph-mon[129229]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:06:30.222 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:06:30.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:29 smithi101 ceph-mon[126379]: pgmap v324: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s; 2 B/s, 0 objects/s recovering 2023-07-18T03:06:30.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:29 smithi101 ceph-mon[126379]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:06:30.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:29 smithi155 ceph-mon[128885]: pgmap v324: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s; 2 B/s, 0 objects/s recovering 2023-07-18T03:06:30.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:29 smithi155 ceph-mon[128885]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:06:31.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:31.830+0000 7fcbc049a700 1 -- 172.21.15.101:0/782775570 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcbb80ffe00 msgr2=0x7fcbb81001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:31.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:31.830+0000 7fcbc049a700 1 --2- 172.21.15.101:0/782775570 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcbb80ffe00 0x7fcbb81001d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcbb40099c0 tx=0x7fcbb4009cd0 comp rx=0 tx=0).stop 2023-07-18T03:06:31.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:31.830+0000 7fcbc049a700 1 -- 172.21.15.101:0/782775570 shutdown_connections 2023-07-18T03:06:31.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:31.830+0000 7fcbc049a700 1 --2- 172.21.15.101:0/782775570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbb8104ea0 0x7fcbb810ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:31.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:31.830+0000 7fcbc049a700 1 --2- 172.21.15.101:0/782775570 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcbb80ffe00 0x7fcbb81001d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:31.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:31.830+0000 7fcbc049a700 1 --2- 172.21.15.101:0/782775570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbb81007a0 0x7fcbb8104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:31.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:31.830+0000 7fcbc049a700 1 -- 172.21.15.101:0/782775570 >> 172.21.15.101:0/782775570 conn(0x7fcbb80fb660 msgr2=0x7fcbb80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:31.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:31.830+0000 7fcbc049a700 1 -- 172.21.15.101:0/782775570 shutdown_connections 2023-07-18T03:06:31.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:31.830+0000 7fcbc049a700 1 -- 172.21.15.101:0/782775570 wait complete. 2023-07-18T03:06:31.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:31.831+0000 7fcbc049a700 1 Processor -- start 2023-07-18T03:06:31.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:31.831+0000 7fcbc049a700 1 -- start start 2023-07-18T03:06:31.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:31.831+0000 7fcbc049a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbb80ffe00 0x7fcbb8197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:31.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:31.832+0000 7fcbc049a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbb81007a0 0x7fcbb81983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:31.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:31.832+0000 7fcbc049a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcbb8104ea0 0x7fcbb819c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:31.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:31.832+0000 7fcbc049a700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb819ce60 con 0x7fcbb8104ea0 2023-07-18T03:06:31.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:31.832+0000 7fcbc049a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb819cfd0 con 0x7fcbb80ffe00 2023-07-18T03:06:31.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:31.832+0000 7fcbc049a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb819d190 con 0x7fcbb81007a0 2023-07-18T03:06:31.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:31.832+0000 7fcbbea37700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcbb8104ea0 0x7fcbb819c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:31.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:31.832+0000 7fcbbea37700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcbb8104ea0 0x7fcbb819c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:45974/0 (socket says 172.21.15.101:45974) 2023-07-18T03:06:31.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:31.832+0000 7fcbbea37700 1 -- 172.21.15.101:0/521020024 learned_addr learned my addr 172.21.15.101:0/521020024 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:31.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:31.832+0000 7fcbbda35700 1 --2- 172.21.15.101:0/521020024 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbb81007a0 0x7fcbb81983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:31.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:31.832+0000 7fcbbe236700 1 --2- 172.21.15.101:0/521020024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbb80ffe00 0x7fcbb8197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:31.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:31.832+0000 7fcbbea37700 1 -- 172.21.15.101:0/521020024 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbb81007a0 msgr2=0x7fcbb81983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:31.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:31.832+0000 7fcbbea37700 1 --2- 172.21.15.101:0/521020024 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbb81007a0 0x7fcbb81983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:31.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:31.832+0000 7fcbbea37700 1 -- 172.21.15.101:0/521020024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbb80ffe00 msgr2=0x7fcbb8197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:31.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:31.832+0000 7fcbbea37700 1 --2- 172.21.15.101:0/521020024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbb80ffe00 0x7fcbb8197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:31.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:31.832+0000 7fcbbea37700 1 -- 172.21.15.101:0/521020024 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb40095f0 con 0x7fcbb8104ea0 2023-07-18T03:06:31.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:31.833+0000 7fcbbea37700 1 --2- 172.21.15.101:0/521020024 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcbb8104ea0 0x7fcbb819c850 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0007ee0 tx=0x7fcbb000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:31.822 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:31.833+0000 7fcbaf7fe700 1 -- 172.21.15.101:0/521020024 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb0017070 con 0x7fcbb8104ea0 2023-07-18T03:06:31.822 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:31.833+0000 7fcbc049a700 1 -- 172.21.15.101:0/521020024 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbb819d470 con 0x7fcbb8104ea0 2023-07-18T03:06:31.823 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:31.833+0000 7fcbaf7fe700 1 -- 172.21.15.101:0/521020024 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcbb000f040 con 0x7fcbb8104ea0 2023-07-18T03:06:31.823 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:31.833+0000 7fcbaf7fe700 1 -- 172.21.15.101:0/521020024 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb0005580 con 0x7fcbb8104ea0 2023-07-18T03:06:31.823 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:31.833+0000 7fcbc049a700 1 -- 172.21.15.101:0/521020024 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbb81a4cd0 con 0x7fcbb8104ea0 2023-07-18T03:06:31.823 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:31.834+0000 7fcbc049a700 1 -- 172.21.15.101:0/521020024 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbb804e210 con 0x7fcbb8104ea0 2023-07-18T03:06:31.823 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:31.836+0000 7fcbaf7fe700 1 -- 172.21.15.101:0/521020024 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fcbb000db20 con 0x7fcbb8104ea0 2023-07-18T03:06:31.823 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:31.837+0000 7fcbaf7fe700 1 --2- 172.21.15.101:0/521020024 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fcba40a1260 0x7fcba40a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:31.824 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:31.837+0000 7fcbaf7fe700 1 -- 172.21.15.101:0/521020024 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fcbb00cc4e0 con 0x7fcbb8104ea0 2023-07-18T03:06:31.824 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:31.838+0000 7fcbbe236700 1 --2- 172.21.15.101:0/521020024 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fcba40a1260 0x7fcba40a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:31.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:31.838+0000 7fcbbe236700 1 --2- 172.21.15.101:0/521020024 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fcba40a1260 0x7fcba40a3710 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4009990 tx=0x7fcbb4003680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:31.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:31.841+0000 7fcbaf7fe700 1 -- 172.21.15.101:0/521020024 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fcbb0095700 con 0x7fcbb8104ea0 2023-07-18T03:06:32.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.082+0000 7fcbc049a700 1 -- 172.21.15.101:0/521020024 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fcbb8199200 con 0x7fcbb8104ea0 2023-07-18T03:06:32.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.082+0000 7fcbaf7fe700 1 -- 172.21.15.101:0/521020024 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fcbb0098d20 con 0x7fcbb8104ea0 2023-07-18T03:06:32.069 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:06:32.070 INFO:teuthology.orchestra.run.smithi101.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-18T03:06:32.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.085+0000 7fcbc049a700 1 -- 172.21.15.101:0/521020024 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fcba40a1260 msgr2=0x7fcba40a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:32.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.085+0000 7fcbc049a700 1 --2- 172.21.15.101:0/521020024 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fcba40a1260 0x7fcba40a3710 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4009990 tx=0x7fcbb4003680 comp rx=0 tx=0).stop 2023-07-18T03:06:32.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.085+0000 7fcbc049a700 1 -- 172.21.15.101:0/521020024 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcbb8104ea0 msgr2=0x7fcbb819c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:32.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.085+0000 7fcbc049a700 1 --2- 172.21.15.101:0/521020024 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcbb8104ea0 0x7fcbb819c850 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0007ee0 tx=0x7fcbb000d3b0 comp rx=0 tx=0).stop 2023-07-18T03:06:32.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.085+0000 7fcbc049a700 1 -- 172.21.15.101:0/521020024 shutdown_connections 2023-07-18T03:06:32.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.085+0000 7fcbc049a700 1 --2- 172.21.15.101:0/521020024 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fcba40a1260 0x7fcba40a3710 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:32.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.085+0000 7fcbc049a700 1 --2- 172.21.15.101:0/521020024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbb80ffe00 0x7fcbb8197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:32.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.085+0000 7fcbc049a700 1 --2- 172.21.15.101:0/521020024 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcbb8104ea0 0x7fcbb819c850 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:32.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.085+0000 7fcbc049a700 1 --2- 172.21.15.101:0/521020024 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbb81007a0 0x7fcbb81983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:32.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.085+0000 7fcbc049a700 1 -- 172.21.15.101:0/521020024 >> 172.21.15.101:0/521020024 conn(0x7fcbb80fb660 msgr2=0x7fcbb80fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:32.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.085+0000 7fcbc049a700 1 -- 172.21.15.101:0/521020024 shutdown_connections 2023-07-18T03:06:32.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.085+0000 7fcbc049a700 1 -- 172.21.15.101:0/521020024 wait complete. 2023-07-18T03:06:32.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:31 smithi179 ceph-mon[129229]: pgmap v325: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4 op/s; 2 B/s, 0 objects/s recovering 2023-07-18T03:06:32.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:31 smithi101 ceph-mon[126379]: pgmap v325: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4 op/s; 2 B/s, 0 objects/s recovering 2023-07-18T03:06:32.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:31 smithi155 ceph-mon[128885]: pgmap v325: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4 op/s; 2 B/s, 0 objects/s recovering 2023-07-18T03:06:32.612 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-07-18T03:06:32.613 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:06:32.814 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.827+0000 7f5637fff700 1 -- 172.21.15.101:0/1271132192 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5630101680 msgr2=0x7f5630107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:32.814 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.827+0000 7f5637fff700 1 --2- 172.21.15.101:0/1271132192 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5630101680 0x7f5630107f40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f562800b0a0 tx=0x7f562800b3b0 comp rx=0 tx=0).stop 2023-07-18T03:06:32.814 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.827+0000 7f5637fff700 1 -- 172.21.15.101:0/1271132192 shutdown_connections 2023-07-18T03:06:32.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.827+0000 7f5637fff700 1 --2- 172.21.15.101:0/1271132192 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56300ffb00 0x7f56300fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:32.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.827+0000 7f5637fff700 1 --2- 172.21.15.101:0/1271132192 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5630101680 0x7f5630107f40 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:32.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.827+0000 7f5637fff700 1 --2- 172.21.15.101:0/1271132192 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5630100cf0 0x7f5630101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:32.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.827+0000 7f5637fff700 1 -- 172.21.15.101:0/1271132192 >> 172.21.15.101:0/1271132192 conn(0x7f56300fb0e0 msgr2=0x7f56300fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:32.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.827+0000 7f5637fff700 1 -- 172.21.15.101:0/1271132192 shutdown_connections 2023-07-18T03:06:32.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.827+0000 7f5637fff700 1 -- 172.21.15.101:0/1271132192 wait complete. 2023-07-18T03:06:32.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.828+0000 7f5637fff700 1 Processor -- start 2023-07-18T03:06:32.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.828+0000 7f5637fff700 1 -- start start 2023-07-18T03:06:32.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.829+0000 7f5637fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56300ffb00 0x7f5630197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:32.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.829+0000 7f5637fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5630100cf0 0x7f5630198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:32.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.829+0000 7f5637fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5630101680 0x7f563019f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:32.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.829+0000 7f5637fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563019f660 con 0x7f5630100cf0 2023-07-18T03:06:32.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.829+0000 7f5637fff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563019f7a0 con 0x7f56300ffb00 2023-07-18T03:06:32.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.829+0000 7f5637fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563019f930 con 0x7f5630101680 2023-07-18T03:06:32.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.829+0000 7f563559a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5630100cf0 0x7f5630198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:32.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.829+0000 7f5635d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56300ffb00 0x7f5630197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:32.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.829+0000 7f563559a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5630100cf0 0x7f5630198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:45980/0 (socket says 172.21.15.101:45980) 2023-07-18T03:06:32.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.829+0000 7f563559a700 1 -- 172.21.15.101:0/476360998 learned_addr learned my addr 172.21.15.101:0/476360998 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:32.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.829+0000 7f563659c700 1 --2- 172.21.15.101:0/476360998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5630101680 0x7f563019f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:32.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.829+0000 7f5635d9b700 1 -- 172.21.15.101:0/476360998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5630101680 msgr2=0x7f563019f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:32.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.830+0000 7f5635d9b700 1 --2- 172.21.15.101:0/476360998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5630101680 0x7f563019f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:32.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.830+0000 7f5635d9b700 1 -- 172.21.15.101:0/476360998 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5630100cf0 msgr2=0x7f5630198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:32.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.830+0000 7f5635d9b700 1 --2- 172.21.15.101:0/476360998 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5630100cf0 0x7f5630198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:32.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.830+0000 7f5635d9b700 1 -- 172.21.15.101:0/476360998 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5628009d00 con 0x7f56300ffb00 2023-07-18T03:06:32.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.830+0000 7f563559a700 1 --2- 172.21.15.101:0/476360998 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5630100cf0 0x7f5630198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:06:32.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.830+0000 7f563659c700 1 --2- 172.21.15.101:0/476360998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5630101680 0x7f563019f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:06:32.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.830+0000 7f5635d9b700 1 --2- 172.21.15.101:0/476360998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56300ffb00 0x7f5630197ae0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f562c00e9c0 tx=0x7f562c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:32.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.830+0000 7f5626ffd700 1 -- 172.21.15.101:0/476360998 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562c004220 con 0x7f56300ffb00 2023-07-18T03:06:32.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.830+0000 7f5637fff700 1 -- 172.21.15.101:0/476360998 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f563019fc10 con 0x7f56300ffb00 2023-07-18T03:06:32.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.830+0000 7f5626ffd700 1 -- 172.21.15.101:0/476360998 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f562c004380 con 0x7f56300ffb00 2023-07-18T03:06:32.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.830+0000 7f5626ffd700 1 -- 172.21.15.101:0/476360998 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562c003be0 con 0x7f56300ffb00 2023-07-18T03:06:32.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.830+0000 7f5637fff700 1 -- 172.21.15.101:0/476360998 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56301a0170 con 0x7f56300ffb00 2023-07-18T03:06:32.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.831+0000 7f5637fff700 1 -- 172.21.15.101:0/476360998 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f563010b330 con 0x7f56300ffb00 2023-07-18T03:06:32.822 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.833+0000 7f5626ffd700 1 -- 172.21.15.101:0/476360998 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f562c003d40 con 0x7f56300ffb00 2023-07-18T03:06:32.822 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.835+0000 7f5626ffd700 1 --2- 172.21.15.101:0/476360998 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f561c0a1220 0x7f561c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:32.822 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.835+0000 7f5626ffd700 1 -- 172.21.15.101:0/476360998 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f562c014070 con 0x7f56300ffb00 2023-07-18T03:06:32.822 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.835+0000 7f563559a700 1 --2- 172.21.15.101:0/476360998 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f561c0a1220 0x7f561c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:32.823 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.835+0000 7f563559a700 1 --2- 172.21.15.101:0/476360998 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f561c0a1220 0x7f561c0a36d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5620011550 tx=0x7f5620009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:32.823 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:32.837+0000 7f5626ffd700 1 -- 172.21.15.101:0/476360998 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f562c0964c0 con 0x7f56300ffb00 2023-07-18T03:06:33.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.037+0000 7f5637fff700 1 -- 172.21.15.101:0/476360998 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f563010b4f0 con 0x7f56300ffb00 2023-07-18T03:06:33.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.039+0000 7f5626ffd700 1 -- 172.21.15.101:0/476360998 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24725 (secure 0 0 0) 0x7f562c01f340 con 0x7f56300ffb00 2023-07-18T03:06:33.026 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:06:33.027 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":91,"fsid":"b1782daa-2516-11ee-9b34-001a4aab830c","created":"2023-07-18T02:57:22.972483+0000","modified":"2023-07-18T03:06:24.250777+0000","last_up_change":"2023-07-18T03:04:11.429530+0000","last_in_change":"2023-07-18T03:03:54.158340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:00:43.353489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:04:57.771616+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-18T03:04:58.769590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-18T03:05:02.358830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc76b5d1-38a0-4f1a-9335-756bfc40a62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6803","nonce":3610719929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6805","nonce":3610719929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6809","nonce":3610719929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6807","nonce":3610719929}]},"public_addr":"172.21.15.101:6803/3610719929","cluster_addr":"172.21.15.101:6805/3610719929","heartbeat_back_addr":"172.21.15.101:6809/3610719929","heartbeat_front_addr":"172.21.15.101:6807/3610719929","state":["exists","up"]},{"osd":1,"uuid":"2a299573-0b34-4af9-aaae-b523e5058ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6801","nonce":1030229579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6803","nonce":1030229579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6807","nonce":1030229579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6805","nonce":1030229579}]},"public_addr":"172.21.15.155:6801/1030229579","cluster_addr":"172.21.15.155:6803/1030229579","heartbeat_back_addr":"172.21.15.155:6807/1030229579","heartbeat_front_addr":"172.21.15.155:6805/1030229579","state":["exists","up"]},{"osd":2,"uuid":"123b3749-c636-48fa-9e33-6229b5575dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6801","nonce":1349487507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6803","nonce":1349487507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6807","nonce":1349487507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6805","nonce":1349487507}]},"public_addr":"172.21.15.179:6801/1349487507","cluster_addr":"172.21.15.179:6803/1349487507","heartbeat_back_addr":"172.21.15.179:6807/1349487507","heartbeat_front_addr":"172.21.15.179:6805/1349487507","state":["exists","up"]},{"osd":3,"uuid":"4b0c409d-2954-4127-833a-b196bf226d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6811","nonce":4226577463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6813","nonce":4226577463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6817","nonce":4226577463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6815","nonce":4226577463}]},"public_addr":"172.21.15.101:6811/4226577463","cluster_addr":"172.21.15.101:6813/4226577463","heartbeat_back_addr":"172.21.15.101:6817/4226577463","heartbeat_front_addr":"172.21.15.101:6815/4226577463","state":["exists","up"]},{"osd":4,"uuid":"2137f2df-984e-45ce-baf6-b4ac12c7e3c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6809","nonce":1951135805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6811","nonce":1951135805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6815","nonce":1951135805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6813","nonce":1951135805}]},"public_addr":"172.21.15.155:6809/1951135805","cluster_addr":"172.21.15.155:6811/1951135805","heartbeat_back_addr":"172.21.15.155:6815/1951135805","heartbeat_front_addr":"172.21.15.155:6813/1951135805","state":["exists","up"]},{"osd":5,"uuid":"1020bfc5-8c2f-482e-9588-66c7f98e46e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6809","nonce":2746037021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6811","nonce":2746037021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6815","nonce":2746037021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6813","nonce":2746037021}]},"public_addr":"172.21.15.179:6809/2746037021","cluster_addr":"172.21.15.179:6811/2746037021","heartbeat_back_addr":"172.21.15.179:6815/2746037021","heartbeat_front_addr":"172.21.15.179:6813/2746037021","state":["exists","up"]},{"osd":6,"uuid":"2cd99260-5c36-4603-9338-cf697970ce58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6819","nonce":958044931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6821","nonce":958044931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6825","nonce":958044931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6823","nonce":958044931}]},"public_addr":"172.21.15.101:6819/958044931","cluster_addr":"172.21.15.101:6821/958044931","heartbeat_back_addr":"172.21.15.101:6825/958044931","heartbeat_front_addr":"172.21.15.101:6823/958044931","state":["exists","up"]},{"osd":7,"uuid":"ed23fd0a-678f-4e15-b42c-cfe873315714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6817","nonce":3256322291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6819","nonce":3256322291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6823","nonce":3256322291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6821","nonce":3256322291}]},"public_addr":"172.21.15.155:6817/3256322291","cluster_addr":"172.21.15.155:6819/3256322291","heartbeat_back_addr":"172.21.15.155:6823/3256322291","heartbeat_front_addr":"172.21.15.155:6821/3256322291","state":["exists","up"]},{"osd":8,"uuid":"eca3d8e6-7baf-47a5-b991-328001c7ebef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6817","nonce":3520612559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6819","nonce":3520612559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6823","nonce":3520612559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6821","nonce":3520612559}]},"public_addr":"172.21.15.179:6817/3520612559","cluster_addr":"172.21.15.179:6819/3520612559","heartbeat_back_addr":"172.21.15.179:6823/3520612559","heartbeat_front_addr":"172.21.15.179:6821/3520612559","state":["exists","up"]},{"osd":9,"uuid":"465a7ffb-67e4-49ef-812c-1b96feacfebf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6827","nonce":2166787667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6829","nonce":2166787667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6833","nonce":2166787667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6831","nonce":2166787667}]},"public_addr":"172.21.15.101:6827/2166787667","cluster_addr":"172.21.15.101:6829/2166787667","heartbeat_back_addr":"172.21.15.101:6833/2166787667","heartbeat_front_addr":"172.21.15.101:6831/2166787667","state":["exists","up"]},{"osd":10,"uuid":"1897a027-f25d-434f-8df5-5fc2bd866578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6825","nonce":3682446349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6827","nonce":3682446349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6831","nonce":3682446349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6829","nonce":3682446349}]},"public_addr":"172.21.15.155:6825/3682446349","cluster_addr":"172.21.15.155:6827/3682446349","heartbeat_back_addr":"172.21.15.155:6831/3682446349","heartbeat_front_addr":"172.21.15.155:6829/3682446349","state":["exists","up"]},{"osd":11,"uuid":"b62effaf-45e2-4585-b8d0-db448f3474ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6825","nonce":2602640858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6827","nonce":2602640858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6831","nonce":2602640858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6829","nonce":2602640858}]},"public_addr":"172.21.15.179:6825/2602640858","cluster_addr":"172.21.15.179:6827/2602640858","heartbeat_back_addr":"172.21.15.179:6831/2602640858","heartbeat_front_addr":"172.21.15.179:6829/2602640858","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T02:59:56.074519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:19.358142+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:41.128966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:08.227054+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:31.442592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:52.751685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:19.251054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:40.698241+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:01.339150+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:27.955910+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:49.136980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:04:10.084982+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.101:6800/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:6801/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/604228291":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/4055187849":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/2844574324":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/1603248215":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6801/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6800/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/3307578182":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/373189792":"2023-07-19T02:57:56.243200+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:06:33.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.042+0000 7f5637fff700 1 -- 172.21.15.101:0/476360998 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f561c0a1220 msgr2=0x7f561c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:33.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.042+0000 7f5637fff700 1 --2- 172.21.15.101:0/476360998 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f561c0a1220 0x7f561c0a36d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5620011550 tx=0x7f5620009260 comp rx=0 tx=0).stop 2023-07-18T03:06:33.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.042+0000 7f5637fff700 1 -- 172.21.15.101:0/476360998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56300ffb00 msgr2=0x7f5630197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:33.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.042+0000 7f5637fff700 1 --2- 172.21.15.101:0/476360998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56300ffb00 0x7f5630197ae0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f562c00e9c0 tx=0x7f562c00ecd0 comp rx=0 tx=0).stop 2023-07-18T03:06:33.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.042+0000 7f5637fff700 1 -- 172.21.15.101:0/476360998 shutdown_connections 2023-07-18T03:06:33.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.042+0000 7f5637fff700 1 --2- 172.21.15.101:0/476360998 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f561c0a1220 0x7f561c0a36d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:33.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.042+0000 7f5637fff700 1 --2- 172.21.15.101:0/476360998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56300ffb00 0x7f5630197ae0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:33.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.042+0000 7f5637fff700 1 --2- 172.21.15.101:0/476360998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5630101680 0x7f563019f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:33.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.042+0000 7f5637fff700 1 --2- 172.21.15.101:0/476360998 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5630100cf0 0x7f5630198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:33.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.042+0000 7f5637fff700 1 -- 172.21.15.101:0/476360998 >> 172.21.15.101:0/476360998 conn(0x7f56300fb0e0 msgr2=0x7f56301064f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:33.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.042+0000 7f5637fff700 1 -- 172.21.15.101:0/476360998 shutdown_connections 2023-07-18T03:06:33.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.042+0000 7f5637fff700 1 -- 172.21.15.101:0/476360998 wait complete. 2023-07-18T03:06:33.047 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:00:43.353489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-18T03:04:57.771616+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-18T03:04:58.769590+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-18T03:05:02.358830+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-18T03:06:33.048 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-18T03:06:33.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:32 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/521020024' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-07-18T03:06:33.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:32 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/521020024' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-07-18T03:06:33.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.261+0000 7f73229ff700 1 -- 172.21.15.101:0/2166311682 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f731c0ffae0 msgr2=0x7f731c0ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:33.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.261+0000 7f73229ff700 1 --2- 172.21.15.101:0/2166311682 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f731c0ffae0 0x7f731c0ffef0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f730c009990 tx=0x7f730c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:06:33.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.261+0000 7f73229ff700 1 -- 172.21.15.101:0/2166311682 shutdown_connections 2023-07-18T03:06:33.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.261+0000 7f73229ff700 1 --2- 172.21.15.101:0/2166311682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f731c101660 0x7f731c107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:33.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.261+0000 7f73229ff700 1 --2- 172.21.15.101:0/2166311682 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f731c0ffae0 0x7f731c0ffef0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:33.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.261+0000 7f73229ff700 1 --2- 172.21.15.101:0/2166311682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f731c100cd0 0x7f731c101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:33.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.261+0000 7f73229ff700 1 -- 172.21.15.101:0/2166311682 >> 172.21.15.101:0/2166311682 conn(0x7f731c0fb0e0 msgr2=0x7f731c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:33.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.261+0000 7f73229ff700 1 -- 172.21.15.101:0/2166311682 shutdown_connections 2023-07-18T03:06:33.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.261+0000 7f73229ff700 1 -- 172.21.15.101:0/2166311682 wait complete. 2023-07-18T03:06:33.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.262+0000 7f73229ff700 1 Processor -- start 2023-07-18T03:06:33.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.262+0000 7f73229ff700 1 -- start start 2023-07-18T03:06:33.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.263+0000 7f73229ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f731c0ffae0 0x7f731c0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:33.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.263+0000 7f73229ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f731c100cd0 0x7f731c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:33.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.263+0000 7f73229ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f731c101660 0x7f731c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:33.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.263+0000 7f73229ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f731c0735e0 con 0x7f731c101660 2023-07-18T03:06:33.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.263+0000 7f73229ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f731c073750 con 0x7f731c0ffae0 2023-07-18T03:06:33.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.263+0000 7f73229ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f731c19f8f0 con 0x7f731c100cd0 2023-07-18T03:06:33.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.263+0000 7f731b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f731c100cd0 0x7f731c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:33.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.263+0000 7f7320f9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f731c101660 0x7f731c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:33.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.263+0000 7f731b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f731c100cd0 0x7f731c072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:38948/0 (socket says 172.21.15.101:38948) 2023-07-18T03:06:33.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.263+0000 7f731b7fe700 1 -- 172.21.15.101:0/1925049742 learned_addr learned my addr 172.21.15.101:0/1925049742 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:33.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.263+0000 7f731bfff700 1 --2- 172.21.15.101:0/1925049742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f731c0ffae0 0x7f731c0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:33.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.263+0000 7f731b7fe700 1 -- 172.21.15.101:0/1925049742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f731c0ffae0 msgr2=0x7f731c0742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:33.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.263+0000 7f731b7fe700 1 --2- 172.21.15.101:0/1925049742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f731c0ffae0 0x7f731c0742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:33.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.263+0000 7f731b7fe700 1 -- 172.21.15.101:0/1925049742 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f731c101660 msgr2=0x7f731c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:33.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.263+0000 7f731b7fe700 1 --2- 172.21.15.101:0/1925049742 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f731c101660 0x7f731c072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:33.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.263+0000 7f731b7fe700 1 -- 172.21.15.101:0/1925049742 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f730c0095f0 con 0x7f731c100cd0 2023-07-18T03:06:33.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.263+0000 7f7320f9c700 1 --2- 172.21.15.101:0/1925049742 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f731c101660 0x7f731c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:06:33.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.264+0000 7f731b7fe700 1 --2- 172.21.15.101:0/1925049742 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f731c100cd0 0x7f731c072900 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f731000e9a0 tx=0x7f731000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:33.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.264+0000 7f73197fa700 1 -- 172.21.15.101:0/1925049742 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7310009790 con 0x7f731c100cd0 2023-07-18T03:06:33.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.264+0000 7f73197fa700 1 -- 172.21.15.101:0/1925049742 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7310010430 con 0x7f731c100cd0 2023-07-18T03:06:33.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.264+0000 7f73197fa700 1 -- 172.21.15.101:0/1925049742 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7310018680 con 0x7f731c100cd0 2023-07-18T03:06:33.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.264+0000 7f73229ff700 1 -- 172.21.15.101:0/1925049742 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f731c19fbd0 con 0x7f731c100cd0 2023-07-18T03:06:33.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.264+0000 7f73229ff700 1 -- 172.21.15.101:0/1925049742 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f731c1a0090 con 0x7f731c100cd0 2023-07-18T03:06:33.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.265+0000 7f73229ff700 1 -- 172.21.15.101:0/1925049742 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f731c106a60 con 0x7f731c100cd0 2023-07-18T03:06:33.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.267+0000 7f73197fa700 1 -- 172.21.15.101:0/1925049742 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f73100105a0 con 0x7f731c100cd0 2023-07-18T03:06:33.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.268+0000 7f73197fa700 1 --2- 172.21.15.101:0/1925049742 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f73040a11e0 0x7f73040a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:33.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.268+0000 7f73197fa700 1 -- 172.21.15.101:0/1925049742 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f7310014070 con 0x7f731c100cd0 2023-07-18T03:06:33.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.269+0000 7f731bfff700 1 --2- 172.21.15.101:0/1925049742 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f73040a11e0 0x7f73040a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:33.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.269+0000 7f731bfff700 1 --2- 172.21.15.101:0/1925049742 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f73040a11e0 0x7f73040a3690 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f730c00b620 tx=0x7f730c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:33.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.271+0000 7f73197fa700 1 -- 172.21.15.101:0/1925049742 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f73100961e0 con 0x7f731c100cd0 2023-07-18T03:06:33.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:32 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/521020024' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-07-18T03:06:33.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.467+0000 7f73229ff700 1 -- 172.21.15.101:0/1925049742 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f731c02caa0 con 0x7f731c100cd0 2023-07-18T03:06:33.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.468+0000 7f73197fa700 1 -- 172.21.15.101:0/1925049742 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7310099800 con 0x7f731c100cd0 2023-07-18T03:06:33.455 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 1 2023-07-18T03:06:33.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.470+0000 7f73229ff700 1 -- 172.21.15.101:0/1925049742 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f73040a11e0 msgr2=0x7f73040a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:33.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.470+0000 7f73229ff700 1 --2- 172.21.15.101:0/1925049742 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f73040a11e0 0x7f73040a3690 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f730c00b620 tx=0x7f730c005fd0 comp rx=0 tx=0).stop 2023-07-18T03:06:33.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.470+0000 7f73229ff700 1 -- 172.21.15.101:0/1925049742 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f731c100cd0 msgr2=0x7f731c072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:33.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.470+0000 7f73229ff700 1 --2- 172.21.15.101:0/1925049742 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f731c100cd0 0x7f731c072900 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f731000e9a0 tx=0x7f731000ecb0 comp rx=0 tx=0).stop 2023-07-18T03:06:33.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.470+0000 7f73229ff700 1 -- 172.21.15.101:0/1925049742 shutdown_connections 2023-07-18T03:06:33.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.470+0000 7f73229ff700 1 --2- 172.21.15.101:0/1925049742 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f73040a11e0 0x7f73040a3690 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:33.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.470+0000 7f73229ff700 1 --2- 172.21.15.101:0/1925049742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f731c0ffae0 0x7f731c0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:33.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.470+0000 7f73229ff700 1 --2- 172.21.15.101:0/1925049742 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f731c101660 0x7f731c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:33.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.470+0000 7f73229ff700 1 --2- 172.21.15.101:0/1925049742 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f731c100cd0 0x7f731c072900 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:33.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.470+0000 7f73229ff700 1 -- 172.21.15.101:0/1925049742 >> 172.21.15.101:0/1925049742 conn(0x7f731c0fb0e0 msgr2=0x7f731c0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:33.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.471+0000 7f73229ff700 1 -- 172.21.15.101:0/1925049742 shutdown_connections 2023-07-18T03:06:33.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.471+0000 7f73229ff700 1 -- 172.21.15.101:0/1925049742 wait complete. 2023-07-18T03:06:33.477 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-18T03:06:33.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.691+0000 7fa39cfff700 1 -- 172.21.15.101:0/1695039521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3980ffae0 msgr2=0x7fa3980ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:33.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.691+0000 7fa39cfff700 1 --2- 172.21.15.101:0/1695039521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3980ffae0 0x7fa3980ffef0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa388009990 tx=0x7fa388009ca0 comp rx=0 tx=0).stop 2023-07-18T03:06:33.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.691+0000 7fa39cfff700 1 -- 172.21.15.101:0/1695039521 shutdown_connections 2023-07-18T03:06:33.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.691+0000 7fa39cfff700 1 --2- 172.21.15.101:0/1695039521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3980ffae0 0x7fa3980ffef0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:33.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.691+0000 7fa39cfff700 1 --2- 172.21.15.101:0/1695039521 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa398101660 0x7fa398107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:33.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.691+0000 7fa39cfff700 1 --2- 172.21.15.101:0/1695039521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa398100cd0 0x7fa398101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:33.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.691+0000 7fa39cfff700 1 -- 172.21.15.101:0/1695039521 >> 172.21.15.101:0/1695039521 conn(0x7fa3980fb0e0 msgr2=0x7fa3980fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:33.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.691+0000 7fa39cfff700 1 -- 172.21.15.101:0/1695039521 shutdown_connections 2023-07-18T03:06:33.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.691+0000 7fa39cfff700 1 -- 172.21.15.101:0/1695039521 wait complete. 2023-07-18T03:06:33.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.692+0000 7fa39cfff700 1 Processor -- start 2023-07-18T03:06:33.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.692+0000 7fa39cfff700 1 -- start start 2023-07-18T03:06:33.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.693+0000 7fa39cfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3980ffae0 0x7fa3980742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:33.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.693+0000 7fa39cfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa398100cd0 0x7fa398072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:33.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.693+0000 7fa39cfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa398101660 0x7fa398072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:33.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.693+0000 7fa39cfff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3980735e0 con 0x7fa398101660 2023-07-18T03:06:33.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.693+0000 7fa39cfff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa398073750 con 0x7fa398100cd0 2023-07-18T03:06:33.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.693+0000 7fa39cfff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39819f8f0 con 0x7fa3980ffae0 2023-07-18T03:06:33.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.693+0000 7fa396d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa398101660 0x7fa398072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:33.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.693+0000 7fa396d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa398101660 0x7fa398072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:46006/0 (socket says 172.21.15.101:46006) 2023-07-18T03:06:33.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.693+0000 7fa396d9d700 1 -- 172.21.15.101:0/186533111 learned_addr learned my addr 172.21.15.101:0/186533111 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:33.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.693+0000 7fa395d9b700 1 --2- 172.21.15.101:0/186533111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa398100cd0 0x7fa398072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:33.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.693+0000 7fa39659c700 1 --2- 172.21.15.101:0/186533111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3980ffae0 0x7fa3980742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:33.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.693+0000 7fa396d9d700 1 -- 172.21.15.101:0/186533111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3980ffae0 msgr2=0x7fa3980742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:33.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.693+0000 7fa396d9d700 1 --2- 172.21.15.101:0/186533111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3980ffae0 0x7fa3980742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:33.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.693+0000 7fa396d9d700 1 -- 172.21.15.101:0/186533111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa398100cd0 msgr2=0x7fa398072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:33.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.693+0000 7fa396d9d700 1 --2- 172.21.15.101:0/186533111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa398100cd0 0x7fa398072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:33.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.693+0000 7fa396d9d700 1 -- 172.21.15.101:0/186533111 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3880095f0 con 0x7fa398101660 2023-07-18T03:06:33.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.693+0000 7fa395d9b700 1 --2- 172.21.15.101:0/186533111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa398100cd0 0x7fa398072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:06:33.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.694+0000 7fa396d9d700 1 --2- 172.21.15.101:0/186533111 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa398101660 0x7fa398072e40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa39000ef30 tx=0x7fa39000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:33.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.694+0000 7fa3877fe700 1 -- 172.21.15.101:0/186533111 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa390018070 con 0x7fa398101660 2023-07-18T03:06:33.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.694+0000 7fa39cfff700 1 -- 172.21.15.101:0/186533111 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa39819fb70 con 0x7fa398101660 2023-07-18T03:06:33.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.694+0000 7fa3877fe700 1 -- 172.21.15.101:0/186533111 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa39000f040 con 0x7fa398101660 2023-07-18T03:06:33.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.694+0000 7fa3877fe700 1 -- 172.21.15.101:0/186533111 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39000cd00 con 0x7fa398101660 2023-07-18T03:06:33.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.694+0000 7fa39cfff700 1 -- 172.21.15.101:0/186533111 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3981a00d0 con 0x7fa398101660 2023-07-18T03:06:33.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.695+0000 7fa39cfff700 1 -- 172.21.15.101:0/186533111 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa398106a60 con 0x7fa398101660 2023-07-18T03:06:33.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.697+0000 7fa3877fe700 1 -- 172.21.15.101:0/186533111 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fa390003a00 con 0x7fa398101660 2023-07-18T03:06:33.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.698+0000 7fa3877fe700 1 --2- 172.21.15.101:0/186533111 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa3800a1260 0x7fa3800a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:33.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.698+0000 7fa3877fe700 1 -- 172.21.15.101:0/186533111 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fa3900cd270 con 0x7fa398101660 2023-07-18T03:06:33.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.699+0000 7fa39659c700 1 --2- 172.21.15.101:0/186533111 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa3800a1260 0x7fa3800a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:33.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.699+0000 7fa39659c700 1 --2- 172.21.15.101:0/186533111 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa3800a1260 0x7fa3800a3710 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa38800b2e0 tx=0x7fa388005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:33.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.700+0000 7fa3877fe700 1 -- 172.21.15.101:0/186533111 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa390096520 con 0x7fa398101660 2023-07-18T03:06:33.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.900+0000 7fa39cfff700 1 -- 172.21.15.101:0/186533111 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa39802c9e0 con 0x7fa398101660 2023-07-18T03:06:33.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.900+0000 7fa3877fe700 1 -- 172.21.15.101:0/186533111 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa390099b40 con 0x7fa398101660 2023-07-18T03:06:33.887 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 32 2023-07-18T03:06:33.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.902+0000 7fa39cfff700 1 -- 172.21.15.101:0/186533111 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa3800a1260 msgr2=0x7fa3800a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:33.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.902+0000 7fa39cfff700 1 --2- 172.21.15.101:0/186533111 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa3800a1260 0x7fa3800a3710 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa38800b2e0 tx=0x7fa388005fb0 comp rx=0 tx=0).stop 2023-07-18T03:06:33.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.902+0000 7fa39cfff700 1 -- 172.21.15.101:0/186533111 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa398101660 msgr2=0x7fa398072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:33.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.902+0000 7fa39cfff700 1 --2- 172.21.15.101:0/186533111 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa398101660 0x7fa398072e40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa39000ef30 tx=0x7fa39000c3a0 comp rx=0 tx=0).stop 2023-07-18T03:06:33.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.902+0000 7fa39cfff700 1 -- 172.21.15.101:0/186533111 shutdown_connections 2023-07-18T03:06:33.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.902+0000 7fa39cfff700 1 --2- 172.21.15.101:0/186533111 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa3800a1260 0x7fa3800a3710 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:33.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.902+0000 7fa39cfff700 1 --2- 172.21.15.101:0/186533111 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa398101660 0x7fa398072e40 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:33.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.902+0000 7fa39cfff700 1 --2- 172.21.15.101:0/186533111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3980ffae0 0x7fa3980742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:33.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.902+0000 7fa39cfff700 1 --2- 172.21.15.101:0/186533111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa398100cd0 0x7fa398072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:33.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.902+0000 7fa39cfff700 1 -- 172.21.15.101:0/186533111 >> 172.21.15.101:0/186533111 conn(0x7fa3980fb0e0 msgr2=0x7fa3980fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:33.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.903+0000 7fa39cfff700 1 -- 172.21.15.101:0/186533111 shutdown_connections 2023-07-18T03:06:33.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:33.903+0000 7fa39cfff700 1 -- 172.21.15.101:0/186533111 wait complete. 2023-07-18T03:06:33.908 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-18T03:06:34.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.123+0000 7f97439ff700 1 -- 172.21.15.101:0/2081882003 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f973c101670 msgr2=0x7f973c107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:34.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.123+0000 7f97439ff700 1 --2- 172.21.15.101:0/2081882003 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f973c101670 0x7f973c107f30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f973400b0a0 tx=0x7f973400b3b0 comp rx=0 tx=0).stop 2023-07-18T03:06:34.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.123+0000 7f97439ff700 1 -- 172.21.15.101:0/2081882003 shutdown_connections 2023-07-18T03:06:34.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.123+0000 7f97439ff700 1 --2- 172.21.15.101:0/2081882003 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f973c0ffaf0 0x7f973c0fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:34.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.123+0000 7f97439ff700 1 --2- 172.21.15.101:0/2081882003 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f973c101670 0x7f973c107f30 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:34.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.123+0000 7f97439ff700 1 --2- 172.21.15.101:0/2081882003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f973c100ce0 0x7f973c101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:34.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.123+0000 7f97439ff700 1 -- 172.21.15.101:0/2081882003 >> 172.21.15.101:0/2081882003 conn(0x7f973c0fb130 msgr2=0x7f973c0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:34.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.123+0000 7f97439ff700 1 -- 172.21.15.101:0/2081882003 shutdown_connections 2023-07-18T03:06:34.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.124+0000 7f97439ff700 1 -- 172.21.15.101:0/2081882003 wait complete. 2023-07-18T03:06:34.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.124+0000 7f97439ff700 1 Processor -- start 2023-07-18T03:06:34.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.125+0000 7f97439ff700 1 -- start start 2023-07-18T03:06:34.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.125+0000 7f97439ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f973c0ffaf0 0x7f973c0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:34.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.125+0000 7f97439ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f973c100ce0 0x7f973c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:34.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.125+0000 7f97439ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f973c101670 0x7f973c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:34.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.125+0000 7f97439ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973c0735e0 con 0x7f973c0ffaf0 2023-07-18T03:06:34.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.125+0000 7f97439ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973c073750 con 0x7f973c101670 2023-07-18T03:06:34.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.125+0000 7f97439ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973c19f980 con 0x7f973c100ce0 2023-07-18T03:06:34.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.125+0000 7f974179b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f973c0ffaf0 0x7f973c0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:34.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.125+0000 7f974179b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f973c0ffaf0 0x7f973c0742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:46030/0 (socket says 172.21.15.101:46030) 2023-07-18T03:06:34.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.125+0000 7f974179b700 1 -- 172.21.15.101:0/2919759317 learned_addr learned my addr 172.21.15.101:0/2919759317 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:34.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.125+0000 7f9740f9a700 1 --2- 172.21.15.101:0/2919759317 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f973c100ce0 0x7f973c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:34.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.125+0000 7f9741f9c700 1 --2- 172.21.15.101:0/2919759317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f973c101670 0x7f973c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:34.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.126+0000 7f974179b700 1 -- 172.21.15.101:0/2919759317 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f973c100ce0 msgr2=0x7f973c072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:34.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.126+0000 7f974179b700 1 --2- 172.21.15.101:0/2919759317 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f973c100ce0 0x7f973c072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:34.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.126+0000 7f974179b700 1 -- 172.21.15.101:0/2919759317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f973c101670 msgr2=0x7f973c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:34.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.126+0000 7f974179b700 1 --2- 172.21.15.101:0/2919759317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f973c101670 0x7f973c072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:34.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.126+0000 7f974179b700 1 -- 172.21.15.101:0/2919759317 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9734009d00 con 0x7f973c0ffaf0 2023-07-18T03:06:34.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.126+0000 7f974179b700 1 --2- 172.21.15.101:0/2919759317 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f973c0ffaf0 0x7f973c0742c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f973800e970 tx=0x7f973800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:34.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.126+0000 7f97327fc700 1 -- 172.21.15.101:0/2919759317 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9738009790 con 0x7f973c0ffaf0 2023-07-18T03:06:34.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.126+0000 7f97439ff700 1 -- 172.21.15.101:0/2919759317 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f973c19fc60 con 0x7f973c0ffaf0 2023-07-18T03:06:34.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.126+0000 7f97327fc700 1 -- 172.21.15.101:0/2919759317 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f973800c7e0 con 0x7f973c0ffaf0 2023-07-18T03:06:34.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.126+0000 7f97327fc700 1 -- 172.21.15.101:0/2919759317 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9738003c70 con 0x7f973c0ffaf0 2023-07-18T03:06:34.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.126+0000 7f97439ff700 1 -- 172.21.15.101:0/2919759317 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f973c1a0190 con 0x7f973c0ffaf0 2023-07-18T03:06:34.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.127+0000 7f97439ff700 1 -- 172.21.15.101:0/2919759317 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f973c1069d0 con 0x7f973c0ffaf0 2023-07-18T03:06:34.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.129+0000 7f97327fc700 1 -- 172.21.15.101:0/2919759317 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f973800cc50 con 0x7f973c0ffaf0 2023-07-18T03:06:34.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.130+0000 7f97327fc700 1 --2- 172.21.15.101:0/2919759317 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f97280a11e0 0x7f97280a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:34.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.130+0000 7f97327fc700 1 -- 172.21.15.101:0/2919759317 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f9738014070 con 0x7f973c0ffaf0 2023-07-18T03:06:34.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.131+0000 7f9740f9a700 1 --2- 172.21.15.101:0/2919759317 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f97280a11e0 0x7f97280a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:34.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.131+0000 7f9740f9a700 1 --2- 172.21.15.101:0/2919759317 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f97280a11e0 0x7f97280a3690 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f972c009c60 tx=0x7f972c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:34.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.135+0000 7f97327fc700 1 -- 172.21.15.101:0/2919759317 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9738096240 con 0x7f973c0ffaf0 2023-07-18T03:06:34.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:33 smithi179 ceph-mon[129229]: pgmap v326: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 2 B/s, 0 objects/s recovering 2023-07-18T03:06:34.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:33 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/476360998' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:06:34.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:33 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1925049742' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:06:34.194 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:33 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/186533111' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:06:34.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:33 smithi101 ceph-mon[126379]: pgmap v326: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 2 B/s, 0 objects/s recovering 2023-07-18T03:06:34.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:33 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/476360998' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:06:34.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:33 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1925049742' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:06:34.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:33 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/186533111' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:06:34.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:33 smithi155 ceph-mon[128885]: pgmap v326: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 2 B/s, 0 objects/s recovering 2023-07-18T03:06:34.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:33 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/476360998' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:06:34.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:33 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1925049742' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:06:34.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:33 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/186533111' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:06:34.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.335+0000 7f97439ff700 1 -- 172.21.15.101:0/2919759317 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f973c02c850 con 0x7f973c0ffaf0 2023-07-18T03:06:34.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.336+0000 7f97327fc700 1 -- 172.21.15.101:0/2919759317 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f973c02c850 con 0x7f973c0ffaf0 2023-07-18T03:06:34.323 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 64 2023-07-18T03:06:34.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.338+0000 7f97439ff700 1 -- 172.21.15.101:0/2919759317 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f97280a11e0 msgr2=0x7f97280a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:34.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.338+0000 7f97439ff700 1 --2- 172.21.15.101:0/2919759317 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f97280a11e0 0x7f97280a3690 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f972c009c60 tx=0x7f972c009260 comp rx=0 tx=0).stop 2023-07-18T03:06:34.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.338+0000 7f97439ff700 1 -- 172.21.15.101:0/2919759317 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f973c0ffaf0 msgr2=0x7f973c0742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:34.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.338+0000 7f97439ff700 1 --2- 172.21.15.101:0/2919759317 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f973c0ffaf0 0x7f973c0742c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f973800e970 tx=0x7f973800ec80 comp rx=0 tx=0).stop 2023-07-18T03:06:34.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.338+0000 7f97439ff700 1 -- 172.21.15.101:0/2919759317 shutdown_connections 2023-07-18T03:06:34.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.338+0000 7f97439ff700 1 --2- 172.21.15.101:0/2919759317 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f97280a11e0 0x7f97280a3690 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:34.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.338+0000 7f97439ff700 1 --2- 172.21.15.101:0/2919759317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f973c101670 0x7f973c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:34.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.338+0000 7f97439ff700 1 --2- 172.21.15.101:0/2919759317 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f973c0ffaf0 0x7f973c0742c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:34.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.338+0000 7f97439ff700 1 --2- 172.21.15.101:0/2919759317 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f973c100ce0 0x7f973c072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:34.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.338+0000 7f97439ff700 1 -- 172.21.15.101:0/2919759317 >> 172.21.15.101:0/2919759317 conn(0x7f973c0fb130 msgr2=0x7f973c0fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:34.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.338+0000 7f97439ff700 1 -- 172.21.15.101:0/2919759317 shutdown_connections 2023-07-18T03:06:34.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.338+0000 7f97439ff700 1 -- 172.21.15.101:0/2919759317 wait complete. 2023-07-18T03:06:34.344 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-18T03:06:34.544 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.557+0000 7f5b599ff700 1 -- 172.21.15.101:0/2456361896 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5b540ffae0 msgr2=0x7f5b540ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:34.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.557+0000 7f5b599ff700 1 --2- 172.21.15.101:0/2456361896 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5b540ffae0 0x7f5b540ffef0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5b44009990 tx=0x7f5b44009ca0 comp rx=0 tx=0).stop 2023-07-18T03:06:34.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.557+0000 7f5b599ff700 1 -- 172.21.15.101:0/2456361896 shutdown_connections 2023-07-18T03:06:34.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.557+0000 7f5b599ff700 1 --2- 172.21.15.101:0/2456361896 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b54101660 0x7f5b54107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:34.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.557+0000 7f5b599ff700 1 --2- 172.21.15.101:0/2456361896 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5b540ffae0 0x7f5b540ffef0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:34.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.557+0000 7f5b599ff700 1 --2- 172.21.15.101:0/2456361896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b54100cd0 0x7f5b54101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:34.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.557+0000 7f5b599ff700 1 -- 172.21.15.101:0/2456361896 >> 172.21.15.101:0/2456361896 conn(0x7f5b540fb0e0 msgr2=0x7f5b540fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:34.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.557+0000 7f5b599ff700 1 -- 172.21.15.101:0/2456361896 shutdown_connections 2023-07-18T03:06:34.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.557+0000 7f5b599ff700 1 -- 172.21.15.101:0/2456361896 wait complete. 2023-07-18T03:06:34.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.558+0000 7f5b599ff700 1 Processor -- start 2023-07-18T03:06:34.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.558+0000 7f5b599ff700 1 -- start start 2023-07-18T03:06:34.549 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.558+0000 7f5b599ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b540ffae0 0x7f5b54197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:34.549 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.558+0000 7f5b599ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b54100cd0 0x7f5b54197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:34.549 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.558+0000 7f5b599ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5b54101660 0x7f5b5419f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:34.549 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.559+0000 7f5b599ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5419f600 con 0x7f5b54101660 2023-07-18T03:06:34.549 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.559+0000 7f5b599ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5419f740 con 0x7f5b540ffae0 2023-07-18T03:06:34.550 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.559+0000 7f5b599ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5419f8d0 con 0x7f5b54100cd0 2023-07-18T03:06:34.550 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.559+0000 7f5b52ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b540ffae0 0x7f5b54197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:34.550 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.559+0000 7f5b537fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5b54101660 0x7f5b5419f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:34.550 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.559+0000 7f5b52ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b540ffae0 0x7f5b54197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:35500/0 (socket says 172.21.15.101:35500) 2023-07-18T03:06:34.550 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.559+0000 7f5b52ffd700 1 -- 172.21.15.101:0/2788785570 learned_addr learned my addr 172.21.15.101:0/2788785570 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:34.550 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.559+0000 7f5b527fc700 1 --2- 172.21.15.101:0/2788785570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b54100cd0 0x7f5b54197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:34.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.559+0000 7f5b52ffd700 1 -- 172.21.15.101:0/2788785570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b54100cd0 msgr2=0x7f5b54197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:34.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.559+0000 7f5b52ffd700 1 --2- 172.21.15.101:0/2788785570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b54100cd0 0x7f5b54197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:34.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.559+0000 7f5b52ffd700 1 -- 172.21.15.101:0/2788785570 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5b54101660 msgr2=0x7f5b5419f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:34.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.559+0000 7f5b52ffd700 1 --2- 172.21.15.101:0/2788785570 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5b54101660 0x7f5b5419f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:34.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.559+0000 7f5b52ffd700 1 -- 172.21.15.101:0/2788785570 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b440095f0 con 0x7f5b540ffae0 2023-07-18T03:06:34.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.559+0000 7f5b527fc700 1 --2- 172.21.15.101:0/2788785570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b54100cd0 0x7f5b54197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:06:34.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.559+0000 7f5b537fe700 1 --2- 172.21.15.101:0/2788785570 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5b54101660 0x7f5b5419f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:06:34.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.559+0000 7f5b52ffd700 1 --2- 172.21.15.101:0/2788785570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b540ffae0 0x7f5b54197a80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5b4400b620 tx=0x7f5b44005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:34.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.560+0000 7f5b589fd700 1 -- 172.21.15.101:0/2788785570 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4401d070 con 0x7f5b540ffae0 2023-07-18T03:06:34.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.560+0000 7f5b599ff700 1 -- 172.21.15.101:0/2788785570 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b5419fb50 con 0x7f5b540ffae0 2023-07-18T03:06:34.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.560+0000 7f5b589fd700 1 -- 172.21.15.101:0/2788785570 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b44021410 con 0x7f5b540ffae0 2023-07-18T03:06:34.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.560+0000 7f5b599ff700 1 -- 172.21.15.101:0/2788785570 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b5419ffd0 con 0x7f5b540ffae0 2023-07-18T03:06:34.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.560+0000 7f5b589fd700 1 -- 172.21.15.101:0/2788785570 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b44017650 con 0x7f5b540ffae0 2023-07-18T03:06:34.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.561+0000 7f5b599ff700 1 -- 172.21.15.101:0/2788785570 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b5410b300 con 0x7f5b540ffae0 2023-07-18T03:06:34.553 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.564+0000 7f5b589fd700 1 -- 172.21.15.101:0/2788785570 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f5b44021840 con 0x7f5b540ffae0 2023-07-18T03:06:34.553 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.565+0000 7f5b589fd700 1 --2- 172.21.15.101:0/2788785570 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f5b3c0a11e0 0x7f5b3c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:34.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.565+0000 7f5b589fd700 1 -- 172.21.15.101:0/2788785570 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f5b440ce410 con 0x7f5b540ffae0 2023-07-18T03:06:34.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.565+0000 7f5b527fc700 1 --2- 172.21.15.101:0/2788785570 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f5b3c0a11e0 0x7f5b3c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:34.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.566+0000 7f5b527fc700 1 --2- 172.21.15.101:0/2788785570 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f5b3c0a11e0 0x7f5b3c0a3690 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5b48009cb0 tx=0x7f5b48009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:34.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.567+0000 7f5b589fd700 1 -- 172.21.15.101:0/2788785570 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5b44097720 con 0x7f5b540ffae0 2023-07-18T03:06:34.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.765+0000 7f5b599ff700 1 -- 172.21.15.101:0/2788785570 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5b541a0580 con 0x7f5b540ffae0 2023-07-18T03:06:34.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.765+0000 7f5b589fd700 1 -- 172.21.15.101:0/2788785570 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5b4409ad40 con 0x7f5b540ffae0 2023-07-18T03:06:34.752 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 32 2023-07-18T03:06:34.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.768+0000 7f5b599ff700 1 -- 172.21.15.101:0/2788785570 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f5b3c0a11e0 msgr2=0x7f5b3c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:34.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.768+0000 7f5b599ff700 1 --2- 172.21.15.101:0/2788785570 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f5b3c0a11e0 0x7f5b3c0a3690 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5b48009cb0 tx=0x7f5b48009260 comp rx=0 tx=0).stop 2023-07-18T03:06:34.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.768+0000 7f5b599ff700 1 -- 172.21.15.101:0/2788785570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b540ffae0 msgr2=0x7f5b54197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:34.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.768+0000 7f5b599ff700 1 --2- 172.21.15.101:0/2788785570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b540ffae0 0x7f5b54197a80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5b4400b620 tx=0x7f5b44005740 comp rx=0 tx=0).stop 2023-07-18T03:06:34.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.768+0000 7f5b599ff700 1 -- 172.21.15.101:0/2788785570 shutdown_connections 2023-07-18T03:06:34.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.768+0000 7f5b599ff700 1 --2- 172.21.15.101:0/2788785570 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f5b3c0a11e0 0x7f5b3c0a3690 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:34.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.768+0000 7f5b599ff700 1 --2- 172.21.15.101:0/2788785570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b540ffae0 0x7f5b54197a80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:34.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.768+0000 7f5b599ff700 1 --2- 172.21.15.101:0/2788785570 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5b54101660 0x7f5b5419f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:34.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.768+0000 7f5b599ff700 1 --2- 172.21.15.101:0/2788785570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b54100cd0 0x7f5b54197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:34.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.768+0000 7f5b599ff700 1 -- 172.21.15.101:0/2788785570 >> 172.21.15.101:0/2788785570 conn(0x7f5b540fb0e0 msgr2=0x7f5b540fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:34.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.768+0000 7f5b599ff700 1 -- 172.21.15.101:0/2788785570 shutdown_connections 2023-07-18T03:06:34.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.768+0000 7f5b599ff700 1 -- 172.21.15.101:0/2788785570 wait complete. 2023-07-18T03:06:34.774 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:06:34.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.987+0000 7fced07ff700 1 -- 172.21.15.101:0/2531334528 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcec80ffb00 msgr2=0x7fcec80fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:34.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.987+0000 7fced07ff700 1 --2- 172.21.15.101:0/2531334528 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcec80ffb00 0x7fcec80fff10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcec400b0a0 tx=0x7fcec400b3b0 comp rx=0 tx=0).stop 2023-07-18T03:06:34.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.988+0000 7fced07ff700 1 -- 172.21.15.101:0/2531334528 shutdown_connections 2023-07-18T03:06:34.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.988+0000 7fced07ff700 1 --2- 172.21.15.101:0/2531334528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcec8101680 0x7fcec8107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:34.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.988+0000 7fced07ff700 1 --2- 172.21.15.101:0/2531334528 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcec80ffb00 0x7fcec80fff10 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:34.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.988+0000 7fced07ff700 1 --2- 172.21.15.101:0/2531334528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcec8100cf0 0x7fcec8101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:34.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.988+0000 7fced07ff700 1 -- 172.21.15.101:0/2531334528 >> 172.21.15.101:0/2531334528 conn(0x7fcec80fb0e0 msgr2=0x7fcec80fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:34.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.988+0000 7fced07ff700 1 -- 172.21.15.101:0/2531334528 shutdown_connections 2023-07-18T03:06:34.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.988+0000 7fced07ff700 1 -- 172.21.15.101:0/2531334528 wait complete. 2023-07-18T03:06:34.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.989+0000 7fced07ff700 1 Processor -- start 2023-07-18T03:06:34.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.989+0000 7fced07ff700 1 -- start start 2023-07-18T03:06:34.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.989+0000 7fced07ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcec80ffb00 0x7fcec8195990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:34.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.989+0000 7fced07ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcec8100cf0 0x7fcec8195ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:34.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.989+0000 7fced07ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcec8101680 0x7fcec819cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:34.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.989+0000 7fced07ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec819d510 con 0x7fcec8100cf0 2023-07-18T03:06:34.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.989+0000 7fced07ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec819d650 con 0x7fcec8101680 2023-07-18T03:06:34.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.989+0000 7fced07ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec819d7e0 con 0x7fcec80ffb00 2023-07-18T03:06:34.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.989+0000 7fcecdd9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcec8100cf0 0x7fcec8195ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:34.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.989+0000 7fcecdd9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcec8100cf0 0x7fcec8195ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:46074/0 (socket says 172.21.15.101:46074) 2023-07-18T03:06:34.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.990+0000 7fcecdd9a700 1 -- 172.21.15.101:0/2890591145 learned_addr learned my addr 172.21.15.101:0/2890591145 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:34.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.990+0000 7fcece59b700 1 --2- 172.21.15.101:0/2890591145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcec80ffb00 0x7fcec8195990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:34.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.990+0000 7fceced9c700 1 --2- 172.21.15.101:0/2890591145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcec8101680 0x7fcec819cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:34.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.990+0000 7fcecdd9a700 1 -- 172.21.15.101:0/2890591145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcec80ffb00 msgr2=0x7fcec8195990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:34.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.990+0000 7fcecdd9a700 1 --2- 172.21.15.101:0/2890591145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcec80ffb00 0x7fcec8195990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:34.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.990+0000 7fcecdd9a700 1 -- 172.21.15.101:0/2890591145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcec8101680 msgr2=0x7fcec819cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:34.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.990+0000 7fcecdd9a700 1 --2- 172.21.15.101:0/2890591145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcec8101680 0x7fcec819cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:34.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.990+0000 7fcecdd9a700 1 -- 172.21.15.101:0/2890591145 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec4009d00 con 0x7fcec8100cf0 2023-07-18T03:06:34.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.990+0000 7fceced9c700 1 --2- 172.21.15.101:0/2890591145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcec8101680 0x7fcec819cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:06:34.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.990+0000 7fcecdd9a700 1 --2- 172.21.15.101:0/2890591145 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcec8100cf0 0x7fcec8195ed0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fceb800d950 tx=0x7fceb800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:34.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.990+0000 7fcebf7fe700 1 -- 172.21.15.101:0/2890591145 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb8009940 con 0x7fcec8100cf0 2023-07-18T03:06:34.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.990+0000 7fcebf7fe700 1 -- 172.21.15.101:0/2890591145 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fceb8010460 con 0x7fcec8100cf0 2023-07-18T03:06:34.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.990+0000 7fced07ff700 1 -- 172.21.15.101:0/2890591145 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcec819dac0 con 0x7fcec8100cf0 2023-07-18T03:06:34.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.990+0000 7fcebf7fe700 1 -- 172.21.15.101:0/2890591145 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb800f620 con 0x7fcec8100cf0 2023-07-18T03:06:34.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.991+0000 7fced07ff700 1 -- 172.21.15.101:0/2890591145 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcec819e020 con 0x7fcec8100cf0 2023-07-18T03:06:34.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.992+0000 7fced07ff700 1 -- 172.21.15.101:0/2890591145 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcec8107050 con 0x7fcec8100cf0 2023-07-18T03:06:34.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.993+0000 7fcebf7fe700 1 -- 172.21.15.101:0/2890591145 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fceb80108d0 con 0x7fcec8100cf0 2023-07-18T03:06:34.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.996+0000 7fcebf7fe700 1 --2- 172.21.15.101:0/2890591145 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fceb40a12a0 0x7fceb40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:34.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.996+0000 7fcebf7fe700 1 -- 172.21.15.101:0/2890591145 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fceb80ccc40 con 0x7fcec8100cf0 2023-07-18T03:06:34.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.996+0000 7fcece59b700 1 --2- 172.21.15.101:0/2890591145 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fceb40a12a0 0x7fceb40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:34.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:34.997+0000 7fcece59b700 1 --2- 172.21.15.101:0/2890591145 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fceb40a12a0 0x7fceb40a3750 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcec4007a60 tx=0x7fcec400b7f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:34.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.000+0000 7fcebf7fe700 1 -- 172.21.15.101:0/2890591145 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fceb8095e60 con 0x7fcec8100cf0 2023-07-18T03:06:35.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:34 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/2919759317' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:06:35.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:34 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/2788785570' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-18T03:06:35.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.243+0000 7fced07ff700 1 -- 172.21.15.101:0/2890591145 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcec802c850 con 0x7fcec8100cf0 2023-07-18T03:06:35.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:34 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2919759317' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:06:35.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:34 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2788785570' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-18T03:06:35.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.244+0000 7fcebf7fe700 1 -- 172.21.15.101:0/2890591145 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10858 (secure 0 0 0) 0x7fceb8016090 con 0x7fcec8100cf0 2023-07-18T03:06:35.232 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:06:35.232 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/4192846769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":4192846769},{"type":"v1","addr":"172.21.15.101:6839","nonce":4192846769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6841/3824340175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6840","nonce":3824340175},{"type":"v1","addr":"172.21.15.101:6841","nonce":3824340175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:04:59.361185+0000","modified":"2023-07-18T03:05:31.985959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.155:6839/2055462483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2055462483},{"type":"v1","addr":"172.21.15.155:6839","nonce":2055462483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:06:35.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.246+0000 7fced07ff700 1 -- 172.21.15.101:0/2890591145 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fceb40a12a0 msgr2=0x7fceb40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:35.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.246+0000 7fced07ff700 1 --2- 172.21.15.101:0/2890591145 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fceb40a12a0 0x7fceb40a3750 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcec4007a60 tx=0x7fcec400b7f0 comp rx=0 tx=0).stop 2023-07-18T03:06:35.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.246+0000 7fced07ff700 1 -- 172.21.15.101:0/2890591145 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcec8100cf0 msgr2=0x7fcec8195ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:35.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.246+0000 7fced07ff700 1 --2- 172.21.15.101:0/2890591145 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcec8100cf0 0x7fcec8195ed0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fceb800d950 tx=0x7fceb800dc60 comp rx=0 tx=0).stop 2023-07-18T03:06:35.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.246+0000 7fced07ff700 1 -- 172.21.15.101:0/2890591145 shutdown_connections 2023-07-18T03:06:35.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.246+0000 7fced07ff700 1 --2- 172.21.15.101:0/2890591145 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fceb40a12a0 0x7fceb40a3750 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:35.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.246+0000 7fced07ff700 1 --2- 172.21.15.101:0/2890591145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcec8101680 0x7fcec819cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:35.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.246+0000 7fced07ff700 1 --2- 172.21.15.101:0/2890591145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcec80ffb00 0x7fcec8195990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:35.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.246+0000 7fced07ff700 1 --2- 172.21.15.101:0/2890591145 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcec8100cf0 0x7fcec8195ed0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:35.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.246+0000 7fced07ff700 1 -- 172.21.15.101:0/2890591145 >> 172.21.15.101:0/2890591145 conn(0x7fcec80fb0e0 msgr2=0x7fcec8104190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:35.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.247+0000 7fced07ff700 1 -- 172.21.15.101:0/2890591145 shutdown_connections 2023-07-18T03:06:35.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.247+0000 7fced07ff700 1 -- 172.21.15.101:0/2890591145 wait complete. 2023-07-18T03:06:35.235 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 24 2023-07-18T03:06:35.253 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:06:35.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:34 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/2919759317' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:06:35.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:34 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/2788785570' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-18T03:06:35.450 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.463+0000 7f93163ff700 1 -- 172.21.15.101:0/2947068060 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f93100ffb10 msgr2=0x7f93100fff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:35.450 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.463+0000 7f93163ff700 1 --2- 172.21.15.101:0/2947068060 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f93100ffb10 0x7f93100fff20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9300009990 tx=0x7f9300009ca0 comp rx=0 tx=0).stop 2023-07-18T03:06:35.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.463+0000 7f93163ff700 1 -- 172.21.15.101:0/2947068060 shutdown_connections 2023-07-18T03:06:35.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.463+0000 7f93163ff700 1 --2- 172.21.15.101:0/2947068060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9310101690 0x7f9310107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:35.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.463+0000 7f93163ff700 1 --2- 172.21.15.101:0/2947068060 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f93100ffb10 0x7f93100fff20 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:35.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.463+0000 7f93163ff700 1 --2- 172.21.15.101:0/2947068060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9310100d00 0x7f9310101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:35.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.463+0000 7f93163ff700 1 -- 172.21.15.101:0/2947068060 >> 172.21.15.101:0/2947068060 conn(0x7f93100fb110 msgr2=0x7f93100fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:35.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.463+0000 7f93163ff700 1 -- 172.21.15.101:0/2947068060 shutdown_connections 2023-07-18T03:06:35.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.463+0000 7f93163ff700 1 -- 172.21.15.101:0/2947068060 wait complete. 2023-07-18T03:06:35.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.464+0000 7f93163ff700 1 Processor -- start 2023-07-18T03:06:35.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.464+0000 7f93163ff700 1 -- start start 2023-07-18T03:06:35.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.465+0000 7f93163ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93100ffb10 0x7f93100742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:35.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.465+0000 7f93163ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9310100d00 0x7f9310072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:35.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.465+0000 7f93163ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9310101690 0x7f9310072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:35.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.465+0000 7f93163ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93100735e0 con 0x7f9310100d00 2023-07-18T03:06:35.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.465+0000 7f93163ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9310073750 con 0x7f93100ffb10 2023-07-18T03:06:35.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.465+0000 7f93163ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931019f890 con 0x7f9310101690 2023-07-18T03:06:35.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.465+0000 7f930f7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9310100d00 0x7f9310072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:35.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.465+0000 7f930f7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9310100d00 0x7f9310072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:46102/0 (socket says 172.21.15.101:46102) 2023-07-18T03:06:35.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.465+0000 7f930ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93100ffb10 0x7f93100742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:35.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.465+0000 7f930f7fe700 1 -- 172.21.15.101:0/1493216196 learned_addr learned my addr 172.21.15.101:0/1493216196 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:35.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.465+0000 7f931499c700 1 --2- 172.21.15.101:0/1493216196 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9310101690 0x7f9310072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:35.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.466+0000 7f930f7fe700 1 -- 172.21.15.101:0/1493216196 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9310101690 msgr2=0x7f9310072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:35.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.466+0000 7f930f7fe700 1 --2- 172.21.15.101:0/1493216196 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9310101690 0x7f9310072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:35.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.466+0000 7f930f7fe700 1 -- 172.21.15.101:0/1493216196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93100ffb10 msgr2=0x7f93100742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:35.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.466+0000 7f930f7fe700 1 --2- 172.21.15.101:0/1493216196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93100ffb10 0x7f93100742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:35.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.466+0000 7f930f7fe700 1 -- 172.21.15.101:0/1493216196 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93000095f0 con 0x7f9310100d00 2023-07-18T03:06:35.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.466+0000 7f930f7fe700 1 --2- 172.21.15.101:0/1493216196 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9310100d00 0x7f9310072900 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f930400e9a0 tx=0x7f930400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:35.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.466+0000 7f930d7fa700 1 -- 172.21.15.101:0/1493216196 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9304009790 con 0x7f9310100d00 2023-07-18T03:06:35.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.466+0000 7f930d7fa700 1 -- 172.21.15.101:0/1493216196 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9304010430 con 0x7f9310100d00 2023-07-18T03:06:35.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.466+0000 7f93163ff700 1 -- 172.21.15.101:0/1493216196 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f931019fb70 con 0x7f9310100d00 2023-07-18T03:06:35.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.466+0000 7f930d7fa700 1 -- 172.21.15.101:0/1493216196 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9304018680 con 0x7f9310100d00 2023-07-18T03:06:35.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.466+0000 7f93163ff700 1 -- 172.21.15.101:0/1493216196 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93101a00d0 con 0x7f9310100d00 2023-07-18T03:06:35.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.467+0000 7f93163ff700 1 -- 172.21.15.101:0/1493216196 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9310107000 con 0x7f9310100d00 2023-07-18T03:06:35.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.469+0000 7f930d7fa700 1 -- 172.21.15.101:0/1493216196 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f93040105a0 con 0x7f9310100d00 2023-07-18T03:06:35.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.471+0000 7f930d7fa700 1 --2- 172.21.15.101:0/1493216196 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f92f80a12a0 0x7f92f80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:35.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.471+0000 7f930d7fa700 1 -- 172.21.15.101:0/1493216196 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f9304014070 con 0x7f9310100d00 2023-07-18T03:06:35.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.471+0000 7f930ffff700 1 --2- 172.21.15.101:0/1493216196 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f92f80a12a0 0x7f92f80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:35.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.472+0000 7f930ffff700 1 --2- 172.21.15.101:0/1493216196 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f92f80a12a0 0x7f92f80a3750 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f930000b2e0 tx=0x7f9300005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:35.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.473+0000 7f930d7fa700 1 -- 172.21.15.101:0/1493216196 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f93040963f0 con 0x7f9310100d00 2023-07-18T03:06:35.659 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.672+0000 7f93163ff700 1 -- 172.21.15.101:0/1493216196 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f931002c850 con 0x7f9310100d00 2023-07-18T03:06:35.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.674+0000 7f930d7fa700 1 -- 172.21.15.101:0/1493216196 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24725 (secure 0 0 0) 0x7f9304026020 con 0x7f9310100d00 2023-07-18T03:06:35.661 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:06:35.661 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":91,"fsid":"b1782daa-2516-11ee-9b34-001a4aab830c","created":"2023-07-18T02:57:22.972483+0000","modified":"2023-07-18T03:06:24.250777+0000","last_up_change":"2023-07-18T03:04:11.429530+0000","last_in_change":"2023-07-18T03:03:54.158340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:00:43.353489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:04:57.771616+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-18T03:04:58.769590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-18T03:05:02.358830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc76b5d1-38a0-4f1a-9335-756bfc40a62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6803","nonce":3610719929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6805","nonce":3610719929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6809","nonce":3610719929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6807","nonce":3610719929}]},"public_addr":"172.21.15.101:6803/3610719929","cluster_addr":"172.21.15.101:6805/3610719929","heartbeat_back_addr":"172.21.15.101:6809/3610719929","heartbeat_front_addr":"172.21.15.101:6807/3610719929","state":["exists","up"]},{"osd":1,"uuid":"2a299573-0b34-4af9-aaae-b523e5058ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6801","nonce":1030229579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6803","nonce":1030229579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6807","nonce":1030229579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6805","nonce":1030229579}]},"public_addr":"172.21.15.155:6801/1030229579","cluster_addr":"172.21.15.155:6803/1030229579","heartbeat_back_addr":"172.21.15.155:6807/1030229579","heartbeat_front_addr":"172.21.15.155:6805/1030229579","state":["exists","up"]},{"osd":2,"uuid":"123b3749-c636-48fa-9e33-6229b5575dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6801","nonce":1349487507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6803","nonce":1349487507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6807","nonce":1349487507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6805","nonce":1349487507}]},"public_addr":"172.21.15.179:6801/1349487507","cluster_addr":"172.21.15.179:6803/1349487507","heartbeat_back_addr":"172.21.15.179:6807/1349487507","heartbeat_front_addr":"172.21.15.179:6805/1349487507","state":["exists","up"]},{"osd":3,"uuid":"4b0c409d-2954-4127-833a-b196bf226d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6811","nonce":4226577463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6813","nonce":4226577463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6817","nonce":4226577463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6815","nonce":4226577463}]},"public_addr":"172.21.15.101:6811/4226577463","cluster_addr":"172.21.15.101:6813/4226577463","heartbeat_back_addr":"172.21.15.101:6817/4226577463","heartbeat_front_addr":"172.21.15.101:6815/4226577463","state":["exists","up"]},{"osd":4,"uuid":"2137f2df-984e-45ce-baf6-b4ac12c7e3c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6809","nonce":1951135805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6811","nonce":1951135805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6815","nonce":1951135805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6813","nonce":1951135805}]},"public_addr":"172.21.15.155:6809/1951135805","cluster_addr":"172.21.15.155:6811/1951135805","heartbeat_back_addr":"172.21.15.155:6815/1951135805","heartbeat_front_addr":"172.21.15.155:6813/1951135805","state":["exists","up"]},{"osd":5,"uuid":"1020bfc5-8c2f-482e-9588-66c7f98e46e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6809","nonce":2746037021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6811","nonce":2746037021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6815","nonce":2746037021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6813","nonce":2746037021}]},"public_addr":"172.21.15.179:6809/2746037021","cluster_addr":"172.21.15.179:6811/2746037021","heartbeat_back_addr":"172.21.15.179:6815/2746037021","heartbeat_front_addr":"172.21.15.179:6813/2746037021","state":["exists","up"]},{"osd":6,"uuid":"2cd99260-5c36-4603-9338-cf697970ce58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6819","nonce":958044931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6821","nonce":958044931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6825","nonce":958044931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6823","nonce":958044931}]},"public_addr":"172.21.15.101:6819/958044931","cluster_addr":"172.21.15.101:6821/958044931","heartbeat_back_addr":"172.21.15.101:6825/958044931","heartbeat_front_addr":"172.21.15.101:6823/958044931","state":["exists","up"]},{"osd":7,"uuid":"ed23fd0a-678f-4e15-b42c-cfe873315714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6817","nonce":3256322291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6819","nonce":3256322291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6823","nonce":3256322291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6821","nonce":3256322291}]},"public_addr":"172.21.15.155:6817/3256322291","cluster_addr":"172.21.15.155:6819/3256322291","heartbeat_back_addr":"172.21.15.155:6823/3256322291","heartbeat_front_addr":"172.21.15.155:6821/3256322291","state":["exists","up"]},{"osd":8,"uuid":"eca3d8e6-7baf-47a5-b991-328001c7ebef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6817","nonce":3520612559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6819","nonce":3520612559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6823","nonce":3520612559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6821","nonce":3520612559}]},"public_addr":"172.21.15.179:6817/3520612559","cluster_addr":"172.21.15.179:6819/3520612559","heartbeat_back_addr":"172.21.15.179:6823/3520612559","heartbeat_front_addr":"172.21.15.179:6821/3520612559","state":["exists","up"]},{"osd":9,"uuid":"465a7ffb-67e4-49ef-812c-1b96feacfebf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6827","nonce":2166787667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6829","nonce":2166787667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6833","nonce":2166787667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6831","nonce":2166787667}]},"public_addr":"172.21.15.101:6827/2166787667","cluster_addr":"172.21.15.101:6829/2166787667","heartbeat_back_addr":"172.21.15.101:6833/2166787667","heartbeat_front_addr":"172.21.15.101:6831/2166787667","state":["exists","up"]},{"osd":10,"uuid":"1897a027-f25d-434f-8df5-5fc2bd866578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6825","nonce":3682446349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6827","nonce":3682446349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6831","nonce":3682446349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6829","nonce":3682446349}]},"public_addr":"172.21.15.155:6825/3682446349","cluster_addr":"172.21.15.155:6827/3682446349","heartbeat_back_addr":"172.21.15.155:6831/3682446349","heartbeat_front_addr":"172.21.15.155:6829/3682446349","state":["exists","up"]},{"osd":11,"uuid":"b62effaf-45e2-4585-b8d0-db448f3474ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6825","nonce":2602640858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6827","nonce":2602640858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6831","nonce":2602640858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6829","nonce":2602640858}]},"public_addr":"172.21.15.179:6825/2602640858","cluster_addr":"172.21.15.179:6827/2602640858","heartbeat_back_addr":"172.21.15.179:6831/2602640858","heartbeat_front_addr":"172.21.15.179:6829/2602640858","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T02:59:56.074519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:19.358142+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:41.128966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:08.227054+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:31.442592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:52.751685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:19.251054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:40.698241+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:01.339150+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:27.955910+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:49.136980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:04:10.084982+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.101:6800/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:6801/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/604228291":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/4055187849":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/2844574324":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/1603248215":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6801/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6800/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/3307578182":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/373189792":"2023-07-19T02:57:56.243200+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:06:35.663 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.676+0000 7f93163ff700 1 -- 172.21.15.101:0/1493216196 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f92f80a12a0 msgr2=0x7f92f80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:35.663 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.676+0000 7f93163ff700 1 --2- 172.21.15.101:0/1493216196 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f92f80a12a0 0x7f92f80a3750 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f930000b2e0 tx=0x7f9300005fb0 comp rx=0 tx=0).stop 2023-07-18T03:06:35.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.676+0000 7f93163ff700 1 -- 172.21.15.101:0/1493216196 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9310100d00 msgr2=0x7f9310072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:35.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.676+0000 7f93163ff700 1 --2- 172.21.15.101:0/1493216196 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9310100d00 0x7f9310072900 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f930400e9a0 tx=0x7f930400ecb0 comp rx=0 tx=0).stop 2023-07-18T03:06:35.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.676+0000 7f93163ff700 1 -- 172.21.15.101:0/1493216196 shutdown_connections 2023-07-18T03:06:35.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.676+0000 7f93163ff700 1 --2- 172.21.15.101:0/1493216196 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f92f80a12a0 0x7f92f80a3750 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:35.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.676+0000 7f93163ff700 1 --2- 172.21.15.101:0/1493216196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93100ffb10 0x7f93100742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:35.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.676+0000 7f93163ff700 1 --2- 172.21.15.101:0/1493216196 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9310101690 0x7f9310072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:35.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.676+0000 7f93163ff700 1 --2- 172.21.15.101:0/1493216196 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9310100d00 0x7f9310072900 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:35.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.676+0000 7f93163ff700 1 -- 172.21.15.101:0/1493216196 >> 172.21.15.101:0/1493216196 conn(0x7f93100fb110 msgr2=0x7f9310104140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:35.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.676+0000 7f93163ff700 1 -- 172.21.15.101:0/1493216196 shutdown_connections 2023-07-18T03:06:35.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.676+0000 7f93163ff700 1 -- 172.21.15.101:0/1493216196 wait complete. 2023-07-18T03:06:35.682 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:00:43.353489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-18T03:04:57.771616+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-18T03:04:58.769590+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-18T03:05:02.358830+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-18T03:06:35.683 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-18T03:06:35.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.902+0000 7f1899dff700 1 -- 172.21.15.101:0/4215416822 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18940ffb10 msgr2=0x7f18940fff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:35.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.902+0000 7f1899dff700 1 --2- 172.21.15.101:0/4215416822 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18940ffb10 0x7f18940fff20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1884009990 tx=0x7f1884009ca0 comp rx=0 tx=0).stop 2023-07-18T03:06:35.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.902+0000 7f1899dff700 1 -- 172.21.15.101:0/4215416822 shutdown_connections 2023-07-18T03:06:35.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.902+0000 7f1899dff700 1 --2- 172.21.15.101:0/4215416822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1894101690 0x7f1894107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:35.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.902+0000 7f1899dff700 1 --2- 172.21.15.101:0/4215416822 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18940ffb10 0x7f18940fff20 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:35.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.902+0000 7f1899dff700 1 --2- 172.21.15.101:0/4215416822 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1894100d00 0x7f1894101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:35.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.902+0000 7f1899dff700 1 -- 172.21.15.101:0/4215416822 >> 172.21.15.101:0/4215416822 conn(0x7f18940fb130 msgr2=0x7f18940fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:35.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.902+0000 7f1899dff700 1 -- 172.21.15.101:0/4215416822 shutdown_connections 2023-07-18T03:06:35.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.902+0000 7f1899dff700 1 -- 172.21.15.101:0/4215416822 wait complete. 2023-07-18T03:06:35.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.903+0000 7f1899dff700 1 Processor -- start 2023-07-18T03:06:35.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.903+0000 7f1899dff700 1 -- start start 2023-07-18T03:06:35.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.904+0000 7f1899dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18940ffb10 0x7f18940742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:35.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.904+0000 7f1899dff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1894100d00 0x7f1894072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:35.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.904+0000 7f1899dff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1894101690 0x7f1894072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:35.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.904+0000 7f1899dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18940735e0 con 0x7f18940ffb10 2023-07-18T03:06:35.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.904+0000 7f1899dff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1894073750 con 0x7f1894101690 2023-07-18T03:06:35.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.904+0000 7f1899dff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189419f8c0 con 0x7f1894100d00 2023-07-18T03:06:35.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.904+0000 7f18937fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18940ffb10 0x7f18940742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:35.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.904+0000 7f18937fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18940ffb10 0x7f18940742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:43638/0 (socket says 172.21.15.101:43638) 2023-07-18T03:06:35.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.904+0000 7f18937fe700 1 -- 172.21.15.101:0/3665447636 learned_addr learned my addr 172.21.15.101:0/3665447636 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:35.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.904+0000 7f1892ffd700 1 --2- 172.21.15.101:0/3665447636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1894100d00 0x7f1894072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:35.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.904+0000 7f1893fff700 1 --2- 172.21.15.101:0/3665447636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1894101690 0x7f1894072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:35.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.904+0000 7f18937fe700 1 -- 172.21.15.101:0/3665447636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1894100d00 msgr2=0x7f1894072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:35.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.904+0000 7f18937fe700 1 --2- 172.21.15.101:0/3665447636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1894100d00 0x7f1894072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:35.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.904+0000 7f18937fe700 1 -- 172.21.15.101:0/3665447636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1894101690 msgr2=0x7f1894072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:35.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.904+0000 7f18937fe700 1 --2- 172.21.15.101:0/3665447636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1894101690 0x7f1894072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:35.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.904+0000 7f18937fe700 1 -- 172.21.15.101:0/3665447636 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18840095f0 con 0x7f18940ffb10 2023-07-18T03:06:35.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.905+0000 7f18937fe700 1 --2- 172.21.15.101:0/3665447636 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18940ffb10 0x7f18940742c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f188400b620 tx=0x7f1884004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:35.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.905+0000 7f1890ff9700 1 -- 172.21.15.101:0/3665447636 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188401d070 con 0x7f18940ffb10 2023-07-18T03:06:35.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.905+0000 7f1890ff9700 1 -- 172.21.15.101:0/3665447636 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1884005030 con 0x7f18940ffb10 2023-07-18T03:06:35.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.905+0000 7f1899dff700 1 -- 172.21.15.101:0/3665447636 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f189419fb40 con 0x7f18940ffb10 2023-07-18T03:06:35.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.905+0000 7f1890ff9700 1 -- 172.21.15.101:0/3665447636 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1884004b80 con 0x7f18940ffb10 2023-07-18T03:06:35.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.905+0000 7f1899dff700 1 -- 172.21.15.101:0/3665447636 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f189419ffc0 con 0x7f18940ffb10 2023-07-18T03:06:35.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.906+0000 7f1899dff700 1 -- 172.21.15.101:0/3665447636 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f189410b2e0 con 0x7f18940ffb10 2023-07-18T03:06:35.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.908+0000 7f1890ff9700 1 -- 172.21.15.101:0/3665447636 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f18840051a0 con 0x7f18940ffb10 2023-07-18T03:06:35.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.909+0000 7f1890ff9700 1 --2- 172.21.15.101:0/3665447636 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f187c0a1260 0x7f187c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:35.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.909+0000 7f1890ff9700 1 -- 172.21.15.101:0/3665447636 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f18840cdd70 con 0x7f18940ffb10 2023-07-18T03:06:35.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.909+0000 7f1892ffd700 1 --2- 172.21.15.101:0/3665447636 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f187c0a1260 0x7f187c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:35.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.910+0000 7f1892ffd700 1 --2- 172.21.15.101:0/3665447636 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f187c0a1260 0x7f187c0a3710 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1888009c60 tx=0x7f1888009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:35.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:35.913+0000 7f1890ff9700 1 -- 172.21.15.101:0/3665447636 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1884097080 con 0x7f18940ffb10 2023-07-18T03:06:35.981 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:35 smithi101 ceph-mon[126379]: pgmap v327: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 2 B/s, 0 objects/s recovering 2023-07-18T03:06:35.981 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:35 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2890591145' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:06:35.981 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:35 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1493216196' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:06:36.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.117+0000 7f1899dff700 1 -- 172.21.15.101:0/3665447636 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1894106a10 con 0x7f18940ffb10 2023-07-18T03:06:36.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.118+0000 7f1890ff9700 1 -- 172.21.15.101:0/3665447636 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1884004ce0 con 0x7f18940ffb10 2023-07-18T03:06:36.105 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 1 2023-07-18T03:06:36.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.120+0000 7f1899dff700 1 -- 172.21.15.101:0/3665447636 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f187c0a1260 msgr2=0x7f187c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:36.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.120+0000 7f1899dff700 1 --2- 172.21.15.101:0/3665447636 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f187c0a1260 0x7f187c0a3710 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1888009c60 tx=0x7f1888009260 comp rx=0 tx=0).stop 2023-07-18T03:06:36.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.120+0000 7f1899dff700 1 -- 172.21.15.101:0/3665447636 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18940ffb10 msgr2=0x7f18940742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:36.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.120+0000 7f1899dff700 1 --2- 172.21.15.101:0/3665447636 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18940ffb10 0x7f18940742c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f188400b620 tx=0x7f1884004730 comp rx=0 tx=0).stop 2023-07-18T03:06:36.108 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.120+0000 7f1899dff700 1 -- 172.21.15.101:0/3665447636 shutdown_connections 2023-07-18T03:06:36.108 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.120+0000 7f1899dff700 1 --2- 172.21.15.101:0/3665447636 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f187c0a1260 0x7f187c0a3710 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:36.108 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.120+0000 7f1899dff700 1 --2- 172.21.15.101:0/3665447636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1894101690 0x7f1894072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:36.108 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.120+0000 7f1899dff700 1 --2- 172.21.15.101:0/3665447636 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18940ffb10 0x7f18940742c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:36.108 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.120+0000 7f1899dff700 1 --2- 172.21.15.101:0/3665447636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1894100d00 0x7f1894072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:36.108 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.120+0000 7f1899dff700 1 -- 172.21.15.101:0/3665447636 >> 172.21.15.101:0/3665447636 conn(0x7f18940fb130 msgr2=0x7f18940fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:36.108 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.121+0000 7f1899dff700 1 -- 172.21.15.101:0/3665447636 shutdown_connections 2023-07-18T03:06:36.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.121+0000 7f1899dff700 1 -- 172.21.15.101:0/3665447636 wait complete. 2023-07-18T03:06:36.127 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-18T03:06:36.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:35 smithi179 ceph-mon[129229]: pgmap v327: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 2 B/s, 0 objects/s recovering 2023-07-18T03:06:36.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:35 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/2890591145' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:06:36.194 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:35 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1493216196' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:06:36.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:35 smithi155 ceph-mon[128885]: pgmap v327: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 2 B/s, 0 objects/s recovering 2023-07-18T03:06:36.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:35 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/2890591145' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:06:36.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:35 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1493216196' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:06:36.328 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.341+0000 7fd1abbff700 1 -- 172.21.15.101:0/1690929517 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd1a4101660 msgr2=0x7fd1a4107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:36.328 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.341+0000 7fd1abbff700 1 --2- 172.21.15.101:0/1690929517 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd1a4101660 0x7fd1a4107f20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd19c00b0a0 tx=0x7fd19c00b3b0 comp rx=0 tx=0).stop 2023-07-18T03:06:36.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.341+0000 7fd1abbff700 1 -- 172.21.15.101:0/1690929517 shutdown_connections 2023-07-18T03:06:36.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.341+0000 7fd1abbff700 1 --2- 172.21.15.101:0/1690929517 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd1a4101660 0x7fd1a4107f20 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:36.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.341+0000 7fd1abbff700 1 --2- 172.21.15.101:0/1690929517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1a40ffae0 0x7fd1a40ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:36.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.341+0000 7fd1abbff700 1 --2- 172.21.15.101:0/1690929517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1a4100cd0 0x7fd1a4101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:36.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.341+0000 7fd1abbff700 1 -- 172.21.15.101:0/1690929517 >> 172.21.15.101:0/1690929517 conn(0x7fd1a40fb0e0 msgr2=0x7fd1a40fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:36.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.341+0000 7fd1abbff700 1 -- 172.21.15.101:0/1690929517 shutdown_connections 2023-07-18T03:06:36.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.341+0000 7fd1abbff700 1 -- 172.21.15.101:0/1690929517 wait complete. 2023-07-18T03:06:36.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.342+0000 7fd1abbff700 1 Processor -- start 2023-07-18T03:06:36.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.343+0000 7fd1abbff700 1 -- start start 2023-07-18T03:06:36.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.343+0000 7fd1abbff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd1a40ffae0 0x7fd1a40742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:36.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.343+0000 7fd1abbff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1a4100cd0 0x7fd1a4072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:36.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.343+0000 7fd1abbff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1a4101660 0x7fd1a4072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:36.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.343+0000 7fd1abbff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a40735e0 con 0x7fd1a40ffae0 2023-07-18T03:06:36.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.343+0000 7fd1abbff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a4073750 con 0x7fd1a4101660 2023-07-18T03:06:36.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.343+0000 7fd1abbff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a419f9c0 con 0x7fd1a4100cd0 2023-07-18T03:06:36.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.343+0000 7fd1a999b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd1a40ffae0 0x7fd1a40742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:36.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.343+0000 7fd1aa19c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1a4101660 0x7fd1a4072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:36.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.343+0000 7fd1a999b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd1a40ffae0 0x7fd1a40742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:43652/0 (socket says 172.21.15.101:43652) 2023-07-18T03:06:36.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.343+0000 7fd1a999b700 1 -- 172.21.15.101:0/3750089699 learned_addr learned my addr 172.21.15.101:0/3750089699 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:36.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.343+0000 7fd1a919a700 1 --2- 172.21.15.101:0/3750089699 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1a4100cd0 0x7fd1a4072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:36.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.344+0000 7fd1aa19c700 1 -- 172.21.15.101:0/3750089699 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1a4100cd0 msgr2=0x7fd1a4072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:36.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.344+0000 7fd1aa19c700 1 --2- 172.21.15.101:0/3750089699 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1a4100cd0 0x7fd1a4072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:36.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.344+0000 7fd1aa19c700 1 -- 172.21.15.101:0/3750089699 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd1a40ffae0 msgr2=0x7fd1a40742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:36.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.344+0000 7fd1aa19c700 1 --2- 172.21.15.101:0/3750089699 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd1a40ffae0 0x7fd1a40742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:36.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.344+0000 7fd1aa19c700 1 -- 172.21.15.101:0/3750089699 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd19c009d00 con 0x7fd1a4101660 2023-07-18T03:06:36.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.344+0000 7fd1aa19c700 1 --2- 172.21.15.101:0/3750089699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1a4101660 0x7fd1a4072e40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd19c00b070 tx=0x7fd19c00bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:36.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.344+0000 7fd19affd700 1 -- 172.21.15.101:0/3750089699 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19c00e040 con 0x7fd1a4101660 2023-07-18T03:06:36.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.344+0000 7fd1abbff700 1 -- 172.21.15.101:0/3750089699 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1a419fc40 con 0x7fd1a4101660 2023-07-18T03:06:36.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.344+0000 7fd19affd700 1 -- 172.21.15.101:0/3750089699 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd19c009480 con 0x7fd1a4101660 2023-07-18T03:06:36.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.344+0000 7fd19affd700 1 -- 172.21.15.101:0/3750089699 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19c0052a0 con 0x7fd1a4101660 2023-07-18T03:06:36.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.344+0000 7fd1abbff700 1 -- 172.21.15.101:0/3750089699 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1a41a0060 con 0x7fd1a4101660 2023-07-18T03:06:36.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.345+0000 7fd1abbff700 1 -- 172.21.15.101:0/3750089699 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1a410b310 con 0x7fd1a4101660 2023-07-18T03:06:36.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.348+0000 7fd19affd700 1 -- 172.21.15.101:0/3750089699 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fd19c01d480 con 0x7fd1a4101660 2023-07-18T03:06:36.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.349+0000 7fd19affd700 1 --2- 172.21.15.101:0/3750089699 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd1900a11e0 0x7fd1900a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:36.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.349+0000 7fd19affd700 1 -- 172.21.15.101:0/3750089699 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fd19c0ce2f0 con 0x7fd1a4101660 2023-07-18T03:06:36.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.349+0000 7fd1a999b700 1 --2- 172.21.15.101:0/3750089699 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd1900a11e0 0x7fd1900a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:36.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.350+0000 7fd1a999b700 1 --2- 172.21.15.101:0/3750089699 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd1900a11e0 0x7fd1900a3690 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0005950 tx=0x7fd1a0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:36.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.351+0000 7fd19affd700 1 -- 172.21.15.101:0/3750089699 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd19c0975a0 con 0x7fd1a4101660 2023-07-18T03:06:36.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.548+0000 7fd1abbff700 1 -- 172.21.15.101:0/3750089699 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd1a4106940 con 0x7fd1a4101660 2023-07-18T03:06:36.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.549+0000 7fd19affd700 1 -- 172.21.15.101:0/3750089699 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd19c004030 con 0x7fd1a4101660 2023-07-18T03:06:36.536 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 32 2023-07-18T03:06:36.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.551+0000 7fd1abbff700 1 -- 172.21.15.101:0/3750089699 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd1900a11e0 msgr2=0x7fd1900a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:36.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.551+0000 7fd1abbff700 1 --2- 172.21.15.101:0/3750089699 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd1900a11e0 0x7fd1900a3690 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0005950 tx=0x7fd1a0009260 comp rx=0 tx=0).stop 2023-07-18T03:06:36.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.551+0000 7fd1abbff700 1 -- 172.21.15.101:0/3750089699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1a4101660 msgr2=0x7fd1a4072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:36.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.551+0000 7fd1abbff700 1 --2- 172.21.15.101:0/3750089699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1a4101660 0x7fd1a4072e40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd19c00b070 tx=0x7fd19c00bc80 comp rx=0 tx=0).stop 2023-07-18T03:06:36.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.551+0000 7fd1abbff700 1 -- 172.21.15.101:0/3750089699 shutdown_connections 2023-07-18T03:06:36.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.551+0000 7fd1abbff700 1 --2- 172.21.15.101:0/3750089699 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd1900a11e0 0x7fd1900a3690 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:36.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.551+0000 7fd1abbff700 1 --2- 172.21.15.101:0/3750089699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1a4101660 0x7fd1a4072e40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:36.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.551+0000 7fd1abbff700 1 --2- 172.21.15.101:0/3750089699 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd1a40ffae0 0x7fd1a40742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:36.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.551+0000 7fd1abbff700 1 --2- 172.21.15.101:0/3750089699 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1a4100cd0 0x7fd1a4072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:36.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.551+0000 7fd1abbff700 1 -- 172.21.15.101:0/3750089699 >> 172.21.15.101:0/3750089699 conn(0x7fd1a40fb0e0 msgr2=0x7fd1a40fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:36.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.551+0000 7fd1abbff700 1 -- 172.21.15.101:0/3750089699 shutdown_connections 2023-07-18T03:06:36.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.551+0000 7fd1abbff700 1 -- 172.21.15.101:0/3750089699 wait complete. 2023-07-18T03:06:36.557 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-18T03:06:36.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.769+0000 7f3e10fff700 1 -- 172.21.15.101:0/3959694679 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3e0c0ffae0 msgr2=0x7f3e0c0ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:36.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.769+0000 7f3e10fff700 1 --2- 172.21.15.101:0/3959694679 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3e0c0ffae0 0x7f3e0c0ffef0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc009990 tx=0x7f3dfc009ca0 comp rx=0 tx=0).stop 2023-07-18T03:06:36.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.769+0000 7f3e10fff700 1 -- 172.21.15.101:0/3959694679 shutdown_connections 2023-07-18T03:06:36.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.769+0000 7f3e10fff700 1 --2- 172.21.15.101:0/3959694679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e0c101660 0x7f3e0c107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:36.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.769+0000 7f3e10fff700 1 --2- 172.21.15.101:0/3959694679 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3e0c0ffae0 0x7f3e0c0ffef0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:36.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.769+0000 7f3e10fff700 1 --2- 172.21.15.101:0/3959694679 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e0c100cd0 0x7f3e0c101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:36.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.769+0000 7f3e10fff700 1 -- 172.21.15.101:0/3959694679 >> 172.21.15.101:0/3959694679 conn(0x7f3e0c0fb0e0 msgr2=0x7f3e0c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:36.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.769+0000 7f3e10fff700 1 -- 172.21.15.101:0/3959694679 shutdown_connections 2023-07-18T03:06:36.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.769+0000 7f3e10fff700 1 -- 172.21.15.101:0/3959694679 wait complete. 2023-07-18T03:06:36.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.771+0000 7f3e10fff700 1 Processor -- start 2023-07-18T03:06:36.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.771+0000 7f3e10fff700 1 -- start start 2023-07-18T03:06:36.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.771+0000 7f3e10fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3e0c0ffae0 0x7f3e0c0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:36.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.771+0000 7f3e10fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e0c100cd0 0x7f3e0c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:36.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.771+0000 7f3e10fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e0c101660 0x7f3e0c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:36.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.772+0000 7f3e10fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0c0735e0 con 0x7f3e0c0ffae0 2023-07-18T03:06:36.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.772+0000 7f3e10fff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0c073750 con 0x7f3e0c101660 2023-07-18T03:06:36.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.772+0000 7f3e10fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0c19f8f0 con 0x7f3e0c100cd0 2023-07-18T03:06:36.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.772+0000 7f3e0a59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3e0c0ffae0 0x7f3e0c0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:36.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.772+0000 7f3e0a59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3e0c0ffae0 0x7f3e0c0742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:43680/0 (socket says 172.21.15.101:43680) 2023-07-18T03:06:36.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.772+0000 7f3e0a59c700 1 -- 172.21.15.101:0/3644993543 learned_addr learned my addr 172.21.15.101:0/3644993543 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:36.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.772+0000 7f3e09d9b700 1 --2- 172.21.15.101:0/3644993543 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e0c100cd0 0x7f3e0c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:36.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.772+0000 7f3e0ad9d700 1 --2- 172.21.15.101:0/3644993543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e0c101660 0x7f3e0c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:36.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.772+0000 7f3e0a59c700 1 -- 172.21.15.101:0/3644993543 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e0c100cd0 msgr2=0x7f3e0c072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:36.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.772+0000 7f3e0a59c700 1 --2- 172.21.15.101:0/3644993543 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e0c100cd0 0x7f3e0c072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:36.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.772+0000 7f3e0a59c700 1 -- 172.21.15.101:0/3644993543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e0c101660 msgr2=0x7f3e0c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:36.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.772+0000 7f3e0a59c700 1 --2- 172.21.15.101:0/3644993543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e0c101660 0x7f3e0c072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:36.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.772+0000 7f3e0a59c700 1 -- 172.21.15.101:0/3644993543 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dfc0095f0 con 0x7f3e0c0ffae0 2023-07-18T03:06:36.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.772+0000 7f3e0ad9d700 1 --2- 172.21.15.101:0/3644993543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e0c101660 0x7f3e0c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:06:36.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.772+0000 7f3e0a59c700 1 --2- 172.21.15.101:0/3644993543 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3e0c0ffae0 0x7f3e0c0742c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc005740 tx=0x7f3dfc009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:36.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.773+0000 7f3dfb7fe700 1 -- 172.21.15.101:0/3644993543 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc01d070 con 0x7f3e0c0ffae0 2023-07-18T03:06:36.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.773+0000 7f3e10fff700 1 -- 172.21.15.101:0/3644993543 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e0c19fb70 con 0x7f3e0c0ffae0 2023-07-18T03:06:36.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.773+0000 7f3dfb7fe700 1 -- 172.21.15.101:0/3644993543 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3dfc021410 con 0x7f3e0c0ffae0 2023-07-18T03:06:36.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.773+0000 7f3dfb7fe700 1 -- 172.21.15.101:0/3644993543 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc017650 con 0x7f3e0c0ffae0 2023-07-18T03:06:36.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.773+0000 7f3e10fff700 1 -- 172.21.15.101:0/3644993543 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e0c19ff90 con 0x7f3e0c0ffae0 2023-07-18T03:06:36.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.774+0000 7f3e10fff700 1 -- 172.21.15.101:0/3644993543 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e0c10b300 con 0x7f3e0c0ffae0 2023-07-18T03:06:36.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.776+0000 7f3dfb7fe700 1 -- 172.21.15.101:0/3644993543 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f3dfc021810 con 0x7f3e0c0ffae0 2023-07-18T03:06:36.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.777+0000 7f3dfb7fe700 1 --2- 172.21.15.101:0/3644993543 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f3df40a11e0 0x7f3df40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:36.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.777+0000 7f3dfb7fe700 1 -- 172.21.15.101:0/3644993543 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f3dfc0ce3f0 con 0x7f3e0c0ffae0 2023-07-18T03:06:36.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.777+0000 7f3e09d9b700 1 --2- 172.21.15.101:0/3644993543 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f3df40a11e0 0x7f3df40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:36.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.778+0000 7f3e09d9b700 1 --2- 172.21.15.101:0/3644993543 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f3df40a11e0 0x7f3df40a3690 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3e00009bf0 tx=0x7f3e00009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:36.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.779+0000 7f3dfb7fe700 1 -- 172.21.15.101:0/3644993543 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3dfc097700 con 0x7f3e0c0ffae0 2023-07-18T03:06:36.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.985+0000 7f3e10fff700 1 -- 172.21.15.101:0/3644993543 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3e0c106a30 con 0x7f3e0c0ffae0 2023-07-18T03:06:36.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.986+0000 7f3dfb7fe700 1 -- 172.21.15.101:0/3644993543 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3dfc09ad20 con 0x7f3e0c0ffae0 2023-07-18T03:06:36.973 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 64 2023-07-18T03:06:36.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.988+0000 7f3e10fff700 1 -- 172.21.15.101:0/3644993543 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f3df40a11e0 msgr2=0x7f3df40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:36.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.988+0000 7f3e10fff700 1 --2- 172.21.15.101:0/3644993543 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f3df40a11e0 0x7f3df40a3690 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3e00009bf0 tx=0x7f3e00009260 comp rx=0 tx=0).stop 2023-07-18T03:06:36.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.988+0000 7f3e10fff700 1 -- 172.21.15.101:0/3644993543 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3e0c0ffae0 msgr2=0x7f3e0c0742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:36.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.988+0000 7f3e10fff700 1 --2- 172.21.15.101:0/3644993543 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3e0c0ffae0 0x7f3e0c0742c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc005740 tx=0x7f3dfc009510 comp rx=0 tx=0).stop 2023-07-18T03:06:36.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.988+0000 7f3e10fff700 1 -- 172.21.15.101:0/3644993543 shutdown_connections 2023-07-18T03:06:36.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.988+0000 7f3e10fff700 1 --2- 172.21.15.101:0/3644993543 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f3df40a11e0 0x7f3df40a3690 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:36.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.988+0000 7f3e10fff700 1 --2- 172.21.15.101:0/3644993543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e0c101660 0x7f3e0c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:36.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.988+0000 7f3e10fff700 1 --2- 172.21.15.101:0/3644993543 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3e0c0ffae0 0x7f3e0c0742c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:36.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.988+0000 7f3e10fff700 1 --2- 172.21.15.101:0/3644993543 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e0c100cd0 0x7f3e0c072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:36.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.988+0000 7f3e10fff700 1 -- 172.21.15.101:0/3644993543 >> 172.21.15.101:0/3644993543 conn(0x7f3e0c0fb0e0 msgr2=0x7f3e0c0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:36.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.988+0000 7f3e10fff700 1 -- 172.21.15.101:0/3644993543 shutdown_connections 2023-07-18T03:06:36.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:36.988+0000 7f3e10fff700 1 -- 172.21.15.101:0/3644993543 wait complete. 2023-07-18T03:06:36.993 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-18T03:06:37.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.207+0000 7ff0dffff700 1 -- 172.21.15.101:0/2573972220 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff0d8102f20 msgr2=0x7ff0d8105300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:37.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.207+0000 7ff0dffff700 1 --2- 172.21.15.101:0/2573972220 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff0d8102f20 0x7ff0d8105300 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8009990 tx=0x7ff0c8009ca0 comp rx=0 tx=0).stop 2023-07-18T03:06:37.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.207+0000 7ff0dffff700 1 -- 172.21.15.101:0/2573972220 shutdown_connections 2023-07-18T03:06:37.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.207+0000 7ff0dffff700 1 --2- 172.21.15.101:0/2573972220 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0d8100600 0x7ff0d81029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:37.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.207+0000 7ff0dffff700 1 --2- 172.21.15.101:0/2573972220 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0d8105840 0x7ff0d8107c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:37.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.207+0000 7ff0dffff700 1 --2- 172.21.15.101:0/2573972220 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff0d8102f20 0x7ff0d8105300 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:37.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.207+0000 7ff0dffff700 1 -- 172.21.15.101:0/2573972220 >> 172.21.15.101:0/2573972220 conn(0x7ff0d80fa000 msgr2=0x7ff0d80fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:37.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.207+0000 7ff0dffff700 1 -- 172.21.15.101:0/2573972220 shutdown_connections 2023-07-18T03:06:37.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.207+0000 7ff0dffff700 1 -- 172.21.15.101:0/2573972220 wait complete. 2023-07-18T03:06:37.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:36 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/3665447636' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:06:37.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:36 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/3750089699' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:06:37.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.208+0000 7ff0dffff700 1 Processor -- start 2023-07-18T03:06:37.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.208+0000 7ff0dffff700 1 -- start start 2023-07-18T03:06:37.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.208+0000 7ff0dffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0d8100600 0x7ff0d8197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:37.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.208+0000 7ff0dffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff0d8102f20 0x7ff0d8197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:37.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.208+0000 7ff0dffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0d8105840 0x7ff0d819efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:37.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.208+0000 7ff0dffff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d819f5b0 con 0x7ff0d8102f20 2023-07-18T03:06:37.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.208+0000 7ff0dffff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d819f6f0 con 0x7ff0d8105840 2023-07-18T03:06:37.198 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.208+0000 7ff0dffff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d819f880 con 0x7ff0d8100600 2023-07-18T03:06:37.198 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.209+0000 7ff0dd59a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff0d8102f20 0x7ff0d8197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:37.198 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.209+0000 7ff0dd59a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff0d8102f20 0x7ff0d8197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:43688/0 (socket says 172.21.15.101:43688) 2023-07-18T03:06:37.198 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.209+0000 7ff0dd59a700 1 -- 172.21.15.101:0/2634461119 learned_addr learned my addr 172.21.15.101:0/2634461119 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:37.198 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.209+0000 7ff0ddd9b700 1 --2- 172.21.15.101:0/2634461119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0d8100600 0x7ff0d8197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:37.198 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.209+0000 7ff0de59c700 1 --2- 172.21.15.101:0/2634461119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0d8105840 0x7ff0d819efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:37.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.209+0000 7ff0dd59a700 1 -- 172.21.15.101:0/2634461119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0d8100600 msgr2=0x7ff0d8197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:37.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.209+0000 7ff0dd59a700 1 --2- 172.21.15.101:0/2634461119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0d8100600 0x7ff0d8197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:37.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.209+0000 7ff0dd59a700 1 -- 172.21.15.101:0/2634461119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0d8105840 msgr2=0x7ff0d819efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:37.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.209+0000 7ff0dd59a700 1 --2- 172.21.15.101:0/2634461119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0d8105840 0x7ff0d819efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:37.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.209+0000 7ff0dd59a700 1 -- 172.21.15.101:0/2634461119 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c80095f0 con 0x7ff0d8102f20 2023-07-18T03:06:37.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.209+0000 7ff0dd59a700 1 --2- 172.21.15.101:0/2634461119 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff0d8102f20 0x7ff0d8197f70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8004730 tx=0x7ff0c8004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:37.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.209+0000 7ff0ceffd700 1 -- 172.21.15.101:0/2634461119 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c801d070 con 0x7ff0d8102f20 2023-07-18T03:06:37.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.210+0000 7ff0ceffd700 1 -- 172.21.15.101:0/2634461119 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0c8004c90 con 0x7ff0d8102f20 2023-07-18T03:06:37.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.210+0000 7ff0dffff700 1 -- 172.21.15.101:0/2634461119 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0d819fb00 con 0x7ff0d8102f20 2023-07-18T03:06:37.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.210+0000 7ff0ceffd700 1 -- 172.21.15.101:0/2634461119 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8005570 con 0x7ff0d8102f20 2023-07-18T03:06:37.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.210+0000 7ff0dffff700 1 -- 172.21.15.101:0/2634461119 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0d819ff20 con 0x7ff0d8102f20 2023-07-18T03:06:37.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.211+0000 7ff0dffff700 1 -- 172.21.15.101:0/2634461119 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0d81073b0 con 0x7ff0d8102f20 2023-07-18T03:06:37.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.212+0000 7ff0ceffd700 1 -- 172.21.15.101:0/2634461119 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7ff0c800fc70 con 0x7ff0d8102f20 2023-07-18T03:06:37.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.215+0000 7ff0ceffd700 1 --2- 172.21.15.101:0/2634461119 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7ff0c40a1260 0x7ff0c40a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:37.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.215+0000 7ff0ceffd700 1 -- 172.21.15.101:0/2634461119 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7ff0c80ce130 con 0x7ff0d8102f20 2023-07-18T03:06:37.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.215+0000 7ff0ddd9b700 1 --2- 172.21.15.101:0/2634461119 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7ff0c40a1260 0x7ff0c40a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:37.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.215+0000 7ff0ddd9b700 1 --2- 172.21.15.101:0/2634461119 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7ff0c40a1260 0x7ff0c40a3710 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4009c60 tx=0x7ff0d4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:37.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.219+0000 7ff0ceffd700 1 -- 172.21.15.101:0/2634461119 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff0c805a790 con 0x7ff0d8102f20 2023-07-18T03:06:37.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:36 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3665447636' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:06:37.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:36 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3750089699' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:06:37.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:36 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/3665447636' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:06:37.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:36 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/3750089699' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:06:37.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.419+0000 7ff0dffff700 1 -- 172.21.15.101:0/2634461119 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff0d802ca70 con 0x7ff0d8102f20 2023-07-18T03:06:37.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.419+0000 7ff0ceffd700 1 -- 172.21.15.101:0/2634461119 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff0c8097440 con 0x7ff0d8102f20 2023-07-18T03:06:37.406 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 32 2023-07-18T03:06:37.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.422+0000 7ff0dffff700 1 -- 172.21.15.101:0/2634461119 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7ff0c40a1260 msgr2=0x7ff0c40a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:37.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.422+0000 7ff0dffff700 1 --2- 172.21.15.101:0/2634461119 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7ff0c40a1260 0x7ff0c40a3710 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4009c60 tx=0x7ff0d4009260 comp rx=0 tx=0).stop 2023-07-18T03:06:37.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.422+0000 7ff0dffff700 1 -- 172.21.15.101:0/2634461119 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff0d8102f20 msgr2=0x7ff0d8197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:37.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.422+0000 7ff0dffff700 1 --2- 172.21.15.101:0/2634461119 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff0d8102f20 0x7ff0d8197f70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8004730 tx=0x7ff0c8004760 comp rx=0 tx=0).stop 2023-07-18T03:06:37.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.422+0000 7ff0dffff700 1 -- 172.21.15.101:0/2634461119 shutdown_connections 2023-07-18T03:06:37.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.422+0000 7ff0dffff700 1 --2- 172.21.15.101:0/2634461119 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7ff0c40a1260 0x7ff0c40a3710 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:37.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.422+0000 7ff0dffff700 1 --2- 172.21.15.101:0/2634461119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0d8105840 0x7ff0d819efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:37.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.422+0000 7ff0dffff700 1 --2- 172.21.15.101:0/2634461119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0d8100600 0x7ff0d8197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:37.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.422+0000 7ff0dffff700 1 --2- 172.21.15.101:0/2634461119 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff0d8102f20 0x7ff0d8197f70 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:37.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.422+0000 7ff0dffff700 1 -- 172.21.15.101:0/2634461119 >> 172.21.15.101:0/2634461119 conn(0x7ff0d80fa000 msgr2=0x7ff0d80fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:37.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.422+0000 7ff0dffff700 1 -- 172.21.15.101:0/2634461119 shutdown_connections 2023-07-18T03:06:37.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.422+0000 7ff0dffff700 1 -- 172.21.15.101:0/2634461119 wait complete. 2023-07-18T03:06:37.426 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:06:37.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.636+0000 7fb28e9ff700 1 -- 172.21.15.101:0/1144665960 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb288101680 msgr2=0x7fb288107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:37.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.636+0000 7fb28e9ff700 1 --2- 172.21.15.101:0/1144665960 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb288101680 0x7fb288107f40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb28000b0a0 tx=0x7fb28000b3b0 comp rx=0 tx=0).stop 2023-07-18T03:06:37.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.636+0000 7fb28e9ff700 1 -- 172.21.15.101:0/1144665960 shutdown_connections 2023-07-18T03:06:37.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.636+0000 7fb28e9ff700 1 --2- 172.21.15.101:0/1144665960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2880ffb00 0x7fb2880fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:37.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.636+0000 7fb28e9ff700 1 --2- 172.21.15.101:0/1144665960 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb288101680 0x7fb288107f40 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:37.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.636+0000 7fb28e9ff700 1 --2- 172.21.15.101:0/1144665960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb288100cf0 0x7fb288101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:37.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.636+0000 7fb28e9ff700 1 -- 172.21.15.101:0/1144665960 >> 172.21.15.101:0/1144665960 conn(0x7fb2880fb0e0 msgr2=0x7fb2880fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:37.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.636+0000 7fb28e9ff700 1 -- 172.21.15.101:0/1144665960 shutdown_connections 2023-07-18T03:06:37.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.636+0000 7fb28e9ff700 1 -- 172.21.15.101:0/1144665960 wait complete. 2023-07-18T03:06:37.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.637+0000 7fb28e9ff700 1 Processor -- start 2023-07-18T03:06:37.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.637+0000 7fb28e9ff700 1 -- start start 2023-07-18T03:06:37.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.637+0000 7fb28e9ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb2880ffb00 0x7fb288197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:37.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.637+0000 7fb28e9ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb288100cf0 0x7fb288198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:37.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.637+0000 7fb28e9ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb288101680 0x7fb28819f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:37.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.637+0000 7fb28e9ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb28819f6a0 con 0x7fb2880ffb00 2023-07-18T03:06:37.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.637+0000 7fb28e9ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb28819f7e0 con 0x7fb288100cf0 2023-07-18T03:06:37.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.637+0000 7fb28e9ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb28819f970 con 0x7fb288101680 2023-07-18T03:06:37.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.638+0000 7fb287fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb2880ffb00 0x7fb288197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:37.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.638+0000 7fb287fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb2880ffb00 0x7fb288197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:43696/0 (socket says 172.21.15.101:43696) 2023-07-18T03:06:37.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.638+0000 7fb287fff700 1 -- 172.21.15.101:0/1526944880 learned_addr learned my addr 172.21.15.101:0/1526944880 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:37.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.638+0000 7fb28cf9c700 1 --2- 172.21.15.101:0/1526944880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb288101680 0x7fb28819f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:37.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.638+0000 7fb27ffff700 1 --2- 172.21.15.101:0/1526944880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb288100cf0 0x7fb288198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:37.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.638+0000 7fb287fff700 1 -- 172.21.15.101:0/1526944880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb288101680 msgr2=0x7fb28819f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:37.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.638+0000 7fb287fff700 1 --2- 172.21.15.101:0/1526944880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb288101680 0x7fb28819f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:37.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.638+0000 7fb287fff700 1 -- 172.21.15.101:0/1526944880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb288100cf0 msgr2=0x7fb288198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:37.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.638+0000 7fb287fff700 1 --2- 172.21.15.101:0/1526944880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb288100cf0 0x7fb288198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:37.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.638+0000 7fb287fff700 1 -- 172.21.15.101:0/1526944880 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb280009d00 con 0x7fb2880ffb00 2023-07-18T03:06:37.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.638+0000 7fb28cf9c700 1 --2- 172.21.15.101:0/1526944880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb288101680 0x7fb28819f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:06:37.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.638+0000 7fb287fff700 1 --2- 172.21.15.101:0/1526944880 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb2880ffb00 0x7fb288197b20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb27800e970 tx=0x7fb27800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:37.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.638+0000 7fb285ffb700 1 -- 172.21.15.101:0/1526944880 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb278009790 con 0x7fb2880ffb00 2023-07-18T03:06:37.628 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.639+0000 7fb28e9ff700 1 -- 172.21.15.101:0/1526944880 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb28819fc50 con 0x7fb2880ffb00 2023-07-18T03:06:37.628 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.639+0000 7fb285ffb700 1 -- 172.21.15.101:0/1526944880 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb27800c7e0 con 0x7fb2880ffb00 2023-07-18T03:06:37.628 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.639+0000 7fb285ffb700 1 -- 172.21.15.101:0/1526944880 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb278003c70 con 0x7fb2880ffb00 2023-07-18T03:06:37.628 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.639+0000 7fb28e9ff700 1 -- 172.21.15.101:0/1526944880 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2881a0160 con 0x7fb2880ffb00 2023-07-18T03:06:37.629 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.640+0000 7fb28e9ff700 1 -- 172.21.15.101:0/1526944880 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb28810b330 con 0x7fb2880ffb00 2023-07-18T03:06:37.629 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.642+0000 7fb285ffb700 1 -- 172.21.15.101:0/1526944880 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fb27800cc50 con 0x7fb2880ffb00 2023-07-18T03:06:37.629 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.643+0000 7fb285ffb700 1 --2- 172.21.15.101:0/1526944880 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb2740a1220 0x7fb2740a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:37.629 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.643+0000 7fb285ffb700 1 -- 172.21.15.101:0/1526944880 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fb278014070 con 0x7fb2880ffb00 2023-07-18T03:06:37.629 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.643+0000 7fb27ffff700 1 --2- 172.21.15.101:0/1526944880 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb2740a1220 0x7fb2740a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:37.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.644+0000 7fb27ffff700 1 --2- 172.21.15.101:0/1526944880 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb2740a1220 0x7fb2740a36d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb270009c60 tx=0x7fb270009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:37.632 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.647+0000 7fb285ffb700 1 -- 172.21.15.101:0/1526944880 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb278096220 con 0x7fb2880ffb00 2023-07-18T03:06:37.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.889+0000 7fb28e9ff700 1 -- 172.21.15.101:0/1526944880 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb2881a0710 con 0x7fb2880ffb00 2023-07-18T03:06:37.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.890+0000 7fb285ffb700 1 -- 172.21.15.101:0/1526944880 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10858 (secure 0 0 0) 0x7fb278003dd0 con 0x7fb2880ffb00 2023-07-18T03:06:37.876 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:06:37.876 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/4192846769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":4192846769},{"type":"v1","addr":"172.21.15.101:6839","nonce":4192846769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6841/3824340175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6840","nonce":3824340175},{"type":"v1","addr":"172.21.15.101:6841","nonce":3824340175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:04:59.361185+0000","modified":"2023-07-18T03:05:31.985959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.155:6839/2055462483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2055462483},{"type":"v1","addr":"172.21.15.155:6839","nonce":2055462483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:06:37.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.892+0000 7fb28e9ff700 1 -- 172.21.15.101:0/1526944880 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb2740a1220 msgr2=0x7fb2740a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:37.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.892+0000 7fb28e9ff700 1 --2- 172.21.15.101:0/1526944880 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb2740a1220 0x7fb2740a36d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb270009c60 tx=0x7fb270009260 comp rx=0 tx=0).stop 2023-07-18T03:06:37.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.892+0000 7fb28e9ff700 1 -- 172.21.15.101:0/1526944880 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb2880ffb00 msgr2=0x7fb288197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:37.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.892+0000 7fb28e9ff700 1 --2- 172.21.15.101:0/1526944880 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb2880ffb00 0x7fb288197b20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb27800e970 tx=0x7fb27800ec80 comp rx=0 tx=0).stop 2023-07-18T03:06:37.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.892+0000 7fb28e9ff700 1 -- 172.21.15.101:0/1526944880 shutdown_connections 2023-07-18T03:06:37.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.892+0000 7fb28e9ff700 1 --2- 172.21.15.101:0/1526944880 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb2740a1220 0x7fb2740a36d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:37.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.892+0000 7fb28e9ff700 1 --2- 172.21.15.101:0/1526944880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb288101680 0x7fb28819f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:37.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.892+0000 7fb28e9ff700 1 --2- 172.21.15.101:0/1526944880 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb2880ffb00 0x7fb288197b20 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:37.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.892+0000 7fb28e9ff700 1 --2- 172.21.15.101:0/1526944880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb288100cf0 0x7fb288198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:37.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.892+0000 7fb28e9ff700 1 -- 172.21.15.101:0/1526944880 >> 172.21.15.101:0/1526944880 conn(0x7fb2880fb0e0 msgr2=0x7fb2880fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:37.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.893+0000 7fb28e9ff700 1 -- 172.21.15.101:0/1526944880 shutdown_connections 2023-07-18T03:06:37.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:37.893+0000 7fb28e9ff700 1 -- 172.21.15.101:0/1526944880 wait complete. 2023-07-18T03:06:37.880 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 24 2023-07-18T03:06:37.897 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:06:38.094 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.108+0000 7f0a5cfff700 1 -- 172.21.15.101:0/320609908 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a58101680 msgr2=0x7f0a58107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:38.094 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.108+0000 7f0a5cfff700 1 --2- 172.21.15.101:0/320609908 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a58101680 0x7f0a58107f40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0a5000b0a0 tx=0x7f0a5000b3b0 comp rx=0 tx=0).stop 2023-07-18T03:06:38.094 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.108+0000 7f0a5cfff700 1 -- 172.21.15.101:0/320609908 shutdown_connections 2023-07-18T03:06:38.094 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.108+0000 7f0a5cfff700 1 --2- 172.21.15.101:0/320609908 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a58101680 0x7f0a58107f40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:38.094 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.108+0000 7f0a5cfff700 1 --2- 172.21.15.101:0/320609908 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a580ffb00 0x7f0a580fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:38.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.108+0000 7f0a5cfff700 1 --2- 172.21.15.101:0/320609908 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0a58100cf0 0x7f0a58101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:38.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.108+0000 7f0a5cfff700 1 -- 172.21.15.101:0/320609908 >> 172.21.15.101:0/320609908 conn(0x7f0a580fb0e0 msgr2=0x7f0a580fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:38.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.108+0000 7f0a5cfff700 1 -- 172.21.15.101:0/320609908 shutdown_connections 2023-07-18T03:06:38.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.108+0000 7f0a5cfff700 1 -- 172.21.15.101:0/320609908 wait complete. 2023-07-18T03:06:38.096 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.109+0000 7f0a5cfff700 1 Processor -- start 2023-07-18T03:06:38.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.109+0000 7f0a5cfff700 1 -- start start 2023-07-18T03:06:38.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.109+0000 7f0a5cfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a580ffb00 0x7f0a581958b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:38.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.109+0000 7f0a5cfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a58100cf0 0x7f0a58195df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:38.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.110+0000 7f0a5cfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0a58101680 0x7f0a5819ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:38.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.110+0000 7f0a5cfff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5819d430 con 0x7f0a58101680 2023-07-18T03:06:38.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.110+0000 7f0a5cfff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5819d570 con 0x7f0a580ffb00 2023-07-18T03:06:38.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.110+0000 7f0a5cfff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5819d700 con 0x7f0a58100cf0 2023-07-18T03:06:38.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.110+0000 7f0a56d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0a58101680 0x7f0a5819ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:38.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.110+0000 7f0a4ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a58100cf0 0x7f0a58195df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:38.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.110+0000 7f0a56d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0a58101680 0x7f0a5819ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:43714/0 (socket says 172.21.15.101:43714) 2023-07-18T03:06:38.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.110+0000 7f0a56d9d700 1 -- 172.21.15.101:0/643446297 learned_addr learned my addr 172.21.15.101:0/643446297 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:38.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.110+0000 7f0a5659c700 1 --2- 172.21.15.101:0/643446297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a580ffb00 0x7f0a581958b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:38.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.110+0000 7f0a56d9d700 1 -- 172.21.15.101:0/643446297 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a58100cf0 msgr2=0x7f0a58195df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:38.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.110+0000 7f0a56d9d700 1 --2- 172.21.15.101:0/643446297 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a58100cf0 0x7f0a58195df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:38.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.110+0000 7f0a56d9d700 1 -- 172.21.15.101:0/643446297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a580ffb00 msgr2=0x7f0a581958b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:38.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.110+0000 7f0a56d9d700 1 --2- 172.21.15.101:0/643446297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a580ffb00 0x7f0a581958b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:38.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.110+0000 7f0a56d9d700 1 -- 172.21.15.101:0/643446297 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a40009520 con 0x7f0a58101680 2023-07-18T03:06:38.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.110+0000 7f0a56d9d700 1 --2- 172.21.15.101:0/643446297 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0a58101680 0x7f0a5819ce60 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0a5000b780 tx=0x7f0a5000bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:38.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.111+0000 7f0a4f7fe700 1 -- 172.21.15.101:0/643446297 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5000e040 con 0x7f0a58101680 2023-07-18T03:06:38.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.111+0000 7f0a4f7fe700 1 -- 172.21.15.101:0/643446297 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a50004c60 con 0x7f0a58101680 2023-07-18T03:06:38.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.111+0000 7f0a5cfff700 1 -- 172.21.15.101:0/643446297 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a50009d00 con 0x7f0a58101680 2023-07-18T03:06:38.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.111+0000 7f0a4f7fe700 1 -- 172.21.15.101:0/643446297 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5001b6b0 con 0x7f0a58101680 2023-07-18T03:06:38.100 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.111+0000 7f0a5cfff700 1 -- 172.21.15.101:0/643446297 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a5819dc70 con 0x7f0a58101680 2023-07-18T03:06:38.100 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.112+0000 7f0a5cfff700 1 -- 172.21.15.101:0/643446297 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a58067370 con 0x7f0a58101680 2023-07-18T03:06:38.100 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.114+0000 7f0a4f7fe700 1 -- 172.21.15.101:0/643446297 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f0a5001d480 con 0x7f0a58101680 2023-07-18T03:06:38.102 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.116+0000 7f0a4f7fe700 1 --2- 172.21.15.101:0/643446297 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f0a440a1220 0x7f0a440a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:38.102 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.116+0000 7f0a4f7fe700 1 -- 172.21.15.101:0/643446297 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f0a500ce590 con 0x7f0a58101680 2023-07-18T03:06:38.102 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.117+0000 7f0a5659c700 1 --2- 172.21.15.101:0/643446297 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f0a440a1220 0x7f0a440a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:38.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.117+0000 7f0a5659c700 1 --2- 172.21.15.101:0/643446297 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f0a440a1220 0x7f0a440a36d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0a58100b50 tx=0x7f0a48009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:38.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.120+0000 7f0a4f7fe700 1 -- 172.21.15.101:0/643446297 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0a58067370 con 0x7f0a58101680 2023-07-18T03:06:38.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:37 smithi179 ceph-mon[129229]: pgmap v328: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 1 B/s, 0 objects/s recovering 2023-07-18T03:06:38.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:37 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/3644993543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:06:38.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:37 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/2634461119' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-18T03:06:38.194 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:37 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1526944880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:06:38.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:37 smithi101 ceph-mon[126379]: pgmap v328: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 1 B/s, 0 objects/s recovering 2023-07-18T03:06:38.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:37 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3644993543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:06:38.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:37 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2634461119' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-18T03:06:38.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:37 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1526944880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:06:38.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:37 smithi155 ceph-mon[128885]: pgmap v328: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 1 B/s, 0 objects/s recovering 2023-07-18T03:06:38.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:37 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/3644993543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:06:38.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:37 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/2634461119' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-18T03:06:38.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:37 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1526944880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:06:38.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.321+0000 7f0a5cfff700 1 -- 172.21.15.101:0/643446297 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a58067300 con 0x7f0a58101680 2023-07-18T03:06:38.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.323+0000 7f0a4f7fe700 1 -- 172.21.15.101:0/643446297 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24725 (secure 0 0 0) 0x7f0a50097840 con 0x7f0a58101680 2023-07-18T03:06:38.309 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:06:38.309 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":91,"fsid":"b1782daa-2516-11ee-9b34-001a4aab830c","created":"2023-07-18T02:57:22.972483+0000","modified":"2023-07-18T03:06:24.250777+0000","last_up_change":"2023-07-18T03:04:11.429530+0000","last_in_change":"2023-07-18T03:03:54.158340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:00:43.353489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:04:57.771616+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-18T03:04:58.769590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-18T03:05:02.358830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc76b5d1-38a0-4f1a-9335-756bfc40a62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6803","nonce":3610719929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6805","nonce":3610719929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6809","nonce":3610719929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6807","nonce":3610719929}]},"public_addr":"172.21.15.101:6803/3610719929","cluster_addr":"172.21.15.101:6805/3610719929","heartbeat_back_addr":"172.21.15.101:6809/3610719929","heartbeat_front_addr":"172.21.15.101:6807/3610719929","state":["exists","up"]},{"osd":1,"uuid":"2a299573-0b34-4af9-aaae-b523e5058ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6801","nonce":1030229579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6803","nonce":1030229579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6807","nonce":1030229579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6805","nonce":1030229579}]},"public_addr":"172.21.15.155:6801/1030229579","cluster_addr":"172.21.15.155:6803/1030229579","heartbeat_back_addr":"172.21.15.155:6807/1030229579","heartbeat_front_addr":"172.21.15.155:6805/1030229579","state":["exists","up"]},{"osd":2,"uuid":"123b3749-c636-48fa-9e33-6229b5575dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6801","nonce":1349487507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6803","nonce":1349487507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6807","nonce":1349487507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6805","nonce":1349487507}]},"public_addr":"172.21.15.179:6801/1349487507","cluster_addr":"172.21.15.179:6803/1349487507","heartbeat_back_addr":"172.21.15.179:6807/1349487507","heartbeat_front_addr":"172.21.15.179:6805/1349487507","state":["exists","up"]},{"osd":3,"uuid":"4b0c409d-2954-4127-833a-b196bf226d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6811","nonce":4226577463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6813","nonce":4226577463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6817","nonce":4226577463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6815","nonce":4226577463}]},"public_addr":"172.21.15.101:6811/4226577463","cluster_addr":"172.21.15.101:6813/4226577463","heartbeat_back_addr":"172.21.15.101:6817/4226577463","heartbeat_front_addr":"172.21.15.101:6815/4226577463","state":["exists","up"]},{"osd":4,"uuid":"2137f2df-984e-45ce-baf6-b4ac12c7e3c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6809","nonce":1951135805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6811","nonce":1951135805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6815","nonce":1951135805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6813","nonce":1951135805}]},"public_addr":"172.21.15.155:6809/1951135805","cluster_addr":"172.21.15.155:6811/1951135805","heartbeat_back_addr":"172.21.15.155:6815/1951135805","heartbeat_front_addr":"172.21.15.155:6813/1951135805","state":["exists","up"]},{"osd":5,"uuid":"1020bfc5-8c2f-482e-9588-66c7f98e46e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6809","nonce":2746037021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6811","nonce":2746037021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6815","nonce":2746037021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6813","nonce":2746037021}]},"public_addr":"172.21.15.179:6809/2746037021","cluster_addr":"172.21.15.179:6811/2746037021","heartbeat_back_addr":"172.21.15.179:6815/2746037021","heartbeat_front_addr":"172.21.15.179:6813/2746037021","state":["exists","up"]},{"osd":6,"uuid":"2cd99260-5c36-4603-9338-cf697970ce58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6819","nonce":958044931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6821","nonce":958044931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6825","nonce":958044931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6823","nonce":958044931}]},"public_addr":"172.21.15.101:6819/958044931","cluster_addr":"172.21.15.101:6821/958044931","heartbeat_back_addr":"172.21.15.101:6825/958044931","heartbeat_front_addr":"172.21.15.101:6823/958044931","state":["exists","up"]},{"osd":7,"uuid":"ed23fd0a-678f-4e15-b42c-cfe873315714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6817","nonce":3256322291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6819","nonce":3256322291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6823","nonce":3256322291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6821","nonce":3256322291}]},"public_addr":"172.21.15.155:6817/3256322291","cluster_addr":"172.21.15.155:6819/3256322291","heartbeat_back_addr":"172.21.15.155:6823/3256322291","heartbeat_front_addr":"172.21.15.155:6821/3256322291","state":["exists","up"]},{"osd":8,"uuid":"eca3d8e6-7baf-47a5-b991-328001c7ebef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6817","nonce":3520612559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6819","nonce":3520612559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6823","nonce":3520612559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6821","nonce":3520612559}]},"public_addr":"172.21.15.179:6817/3520612559","cluster_addr":"172.21.15.179:6819/3520612559","heartbeat_back_addr":"172.21.15.179:6823/3520612559","heartbeat_front_addr":"172.21.15.179:6821/3520612559","state":["exists","up"]},{"osd":9,"uuid":"465a7ffb-67e4-49ef-812c-1b96feacfebf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6827","nonce":2166787667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6829","nonce":2166787667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6833","nonce":2166787667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6831","nonce":2166787667}]},"public_addr":"172.21.15.101:6827/2166787667","cluster_addr":"172.21.15.101:6829/2166787667","heartbeat_back_addr":"172.21.15.101:6833/2166787667","heartbeat_front_addr":"172.21.15.101:6831/2166787667","state":["exists","up"]},{"osd":10,"uuid":"1897a027-f25d-434f-8df5-5fc2bd866578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6825","nonce":3682446349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6827","nonce":3682446349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6831","nonce":3682446349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6829","nonce":3682446349}]},"public_addr":"172.21.15.155:6825/3682446349","cluster_addr":"172.21.15.155:6827/3682446349","heartbeat_back_addr":"172.21.15.155:6831/3682446349","heartbeat_front_addr":"172.21.15.155:6829/3682446349","state":["exists","up"]},{"osd":11,"uuid":"b62effaf-45e2-4585-b8d0-db448f3474ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6825","nonce":2602640858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6827","nonce":2602640858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6831","nonce":2602640858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6829","nonce":2602640858}]},"public_addr":"172.21.15.179:6825/2602640858","cluster_addr":"172.21.15.179:6827/2602640858","heartbeat_back_addr":"172.21.15.179:6831/2602640858","heartbeat_front_addr":"172.21.15.179:6829/2602640858","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T02:59:56.074519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:19.358142+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:41.128966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:08.227054+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:31.442592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:52.751685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:19.251054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:40.698241+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:01.339150+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:27.955910+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:49.136980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:04:10.084982+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.101:6800/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:6801/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/604228291":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/4055187849":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/2844574324":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/1603248215":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6801/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6800/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/3307578182":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/373189792":"2023-07-19T02:57:56.243200+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:06:38.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.325+0000 7f0a5cfff700 1 -- 172.21.15.101:0/643446297 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f0a440a1220 msgr2=0x7f0a440a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:38.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.325+0000 7f0a5cfff700 1 --2- 172.21.15.101:0/643446297 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f0a440a1220 0x7f0a440a36d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0a58100b50 tx=0x7f0a48009260 comp rx=0 tx=0).stop 2023-07-18T03:06:38.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.325+0000 7f0a5cfff700 1 -- 172.21.15.101:0/643446297 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0a58101680 msgr2=0x7f0a5819ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:38.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.325+0000 7f0a5cfff700 1 --2- 172.21.15.101:0/643446297 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0a58101680 0x7f0a5819ce60 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0a5000b780 tx=0x7f0a5000bd00 comp rx=0 tx=0).stop 2023-07-18T03:06:38.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.325+0000 7f0a5cfff700 1 -- 172.21.15.101:0/643446297 shutdown_connections 2023-07-18T03:06:38.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.325+0000 7f0a5cfff700 1 --2- 172.21.15.101:0/643446297 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f0a440a1220 0x7f0a440a36d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:38.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.325+0000 7f0a5cfff700 1 --2- 172.21.15.101:0/643446297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a580ffb00 0x7f0a581958b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:38.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.325+0000 7f0a5cfff700 1 --2- 172.21.15.101:0/643446297 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0a58101680 0x7f0a5819ce60 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:38.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.325+0000 7f0a5cfff700 1 --2- 172.21.15.101:0/643446297 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a58100cf0 0x7f0a58195df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:38.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.325+0000 7f0a5cfff700 1 -- 172.21.15.101:0/643446297 >> 172.21.15.101:0/643446297 conn(0x7f0a580fb0e0 msgr2=0x7f0a581040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:38.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.325+0000 7f0a5cfff700 1 -- 172.21.15.101:0/643446297 shutdown_connections 2023-07-18T03:06:38.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.326+0000 7f0a5cfff700 1 -- 172.21.15.101:0/643446297 wait complete. 2023-07-18T03:06:38.331 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:06:38.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.541+0000 7fba753ff700 1 -- 172.21.15.101:0/2527840775 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fba700ffb00 msgr2=0x7fba700fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:38.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.541+0000 7fba753ff700 1 --2- 172.21.15.101:0/2527840775 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fba700ffb00 0x7fba700fff10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fba60009990 tx=0x7fba60009ca0 comp rx=0 tx=0).stop 2023-07-18T03:06:38.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.542+0000 7fba753ff700 1 -- 172.21.15.101:0/2527840775 shutdown_connections 2023-07-18T03:06:38.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.542+0000 7fba753ff700 1 --2- 172.21.15.101:0/2527840775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fba70101680 0x7fba70107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:38.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.542+0000 7fba753ff700 1 --2- 172.21.15.101:0/2527840775 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fba700ffb00 0x7fba700fff10 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:38.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.542+0000 7fba753ff700 1 --2- 172.21.15.101:0/2527840775 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba70100cf0 0x7fba70101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:38.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.542+0000 7fba753ff700 1 -- 172.21.15.101:0/2527840775 >> 172.21.15.101:0/2527840775 conn(0x7fba700fb0e0 msgr2=0x7fba700fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:38.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.542+0000 7fba753ff700 1 -- 172.21.15.101:0/2527840775 shutdown_connections 2023-07-18T03:06:38.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.542+0000 7fba753ff700 1 -- 172.21.15.101:0/2527840775 wait complete. 2023-07-18T03:06:38.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.543+0000 7fba753ff700 1 Processor -- start 2023-07-18T03:06:38.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.543+0000 7fba753ff700 1 -- start start 2023-07-18T03:06:38.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.544+0000 7fba753ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fba700ffb00 0x7fba70195840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:38.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.544+0000 7fba753ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba70100cf0 0x7fba70195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:38.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.544+0000 7fba753ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fba70101680 0x7fba7019cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:38.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.544+0000 7fba753ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7019d3c0 con 0x7fba700ffb00 2023-07-18T03:06:38.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.544+0000 7fba753ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7019d500 con 0x7fba70100cf0 2023-07-18T03:06:38.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.544+0000 7fba753ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7019d690 con 0x7fba70101680 2023-07-18T03:06:38.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.544+0000 7fba6effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fba700ffb00 0x7fba70195840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:38.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.544+0000 7fba6effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fba700ffb00 0x7fba70195840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:43728/0 (socket says 172.21.15.101:43728) 2023-07-18T03:06:38.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.544+0000 7fba6effd700 1 -- 172.21.15.101:0/4262328835 learned_addr learned my addr 172.21.15.101:0/4262328835 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:38.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.544+0000 7fba6e7fc700 1 --2- 172.21.15.101:0/4262328835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba70100cf0 0x7fba70195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:38.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.544+0000 7fba6f7fe700 1 --2- 172.21.15.101:0/4262328835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fba70101680 0x7fba7019cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:38.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.544+0000 7fba6effd700 1 -- 172.21.15.101:0/4262328835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fba70101680 msgr2=0x7fba7019cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:38.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.544+0000 7fba6effd700 1 --2- 172.21.15.101:0/4262328835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fba70101680 0x7fba7019cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:38.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.544+0000 7fba6effd700 1 -- 172.21.15.101:0/4262328835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba70100cf0 msgr2=0x7fba70195d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:38.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.544+0000 7fba6effd700 1 --2- 172.21.15.101:0/4262328835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba70100cf0 0x7fba70195d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:38.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.544+0000 7fba6effd700 1 -- 172.21.15.101:0/4262328835 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba64009520 con 0x7fba700ffb00 2023-07-18T03:06:38.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.544+0000 7fba6f7fe700 1 --2- 172.21.15.101:0/4262328835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fba70101680 0x7fba7019cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:06:38.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.545+0000 7fba6effd700 1 --2- 172.21.15.101:0/4262328835 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fba700ffb00 0x7fba70195840 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fba60005740 tx=0x7fba60005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:38.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.545+0000 7fba57fff700 1 -- 172.21.15.101:0/4262328835 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6001d070 con 0x7fba700ffb00 2023-07-18T03:06:38.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.545+0000 7fba57fff700 1 -- 172.21.15.101:0/4262328835 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba60021410 con 0x7fba700ffb00 2023-07-18T03:06:38.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.545+0000 7fba753ff700 1 -- 172.21.15.101:0/4262328835 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba600095f0 con 0x7fba700ffb00 2023-07-18T03:06:38.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.545+0000 7fba57fff700 1 -- 172.21.15.101:0/4262328835 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba60017650 con 0x7fba700ffb00 2023-07-18T03:06:38.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.545+0000 7fba753ff700 1 -- 172.21.15.101:0/4262328835 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba7019dba0 con 0x7fba700ffb00 2023-07-18T03:06:38.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.546+0000 7fba753ff700 1 -- 172.21.15.101:0/4262328835 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba7010b320 con 0x7fba700ffb00 2023-07-18T03:06:38.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.548+0000 7fba57fff700 1 -- 172.21.15.101:0/4262328835 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fba60021580 con 0x7fba700ffb00 2023-07-18T03:06:38.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.549+0000 7fba57fff700 1 --2- 172.21.15.101:0/4262328835 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fba580a1220 0x7fba580a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:38.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.549+0000 7fba57fff700 1 -- 172.21.15.101:0/4262328835 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fba600ce1f0 con 0x7fba700ffb00 2023-07-18T03:06:38.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.550+0000 7fba6e7fc700 1 --2- 172.21.15.101:0/4262328835 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fba580a1220 0x7fba580a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:38.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.550+0000 7fba6e7fc700 1 --2- 172.21.15.101:0/4262328835 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fba580a1220 0x7fba580a36d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fba64009df0 tx=0x7fba64009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:38.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.553+0000 7fba57fff700 1 -- 172.21.15.101:0/4262328835 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fba60097530 con 0x7fba700ffb00 2023-07-18T03:06:38.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.795+0000 7fba753ff700 1 -- 172.21.15.101:0/4262328835 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fba7019e2a0 con 0x7fba700ffb00 2023-07-18T03:06:38.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.796+0000 7fba57fff700 1 -- 172.21.15.101:0/4262328835 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10858 (secure 0 0 0) 0x7fba600aee60 con 0x7fba700ffb00 2023-07-18T03:06:38.783 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:06:38.783 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/4192846769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":4192846769},{"type":"v1","addr":"172.21.15.101:6839","nonce":4192846769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6841/3824340175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6840","nonce":3824340175},{"type":"v1","addr":"172.21.15.101:6841","nonce":3824340175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:04:59.361185+0000","modified":"2023-07-18T03:05:31.985959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.155:6839/2055462483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2055462483},{"type":"v1","addr":"172.21.15.155:6839","nonce":2055462483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:06:38.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.798+0000 7fba753ff700 1 -- 172.21.15.101:0/4262328835 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fba580a1220 msgr2=0x7fba580a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:38.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.798+0000 7fba753ff700 1 --2- 172.21.15.101:0/4262328835 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fba580a1220 0x7fba580a36d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fba64009df0 tx=0x7fba64009260 comp rx=0 tx=0).stop 2023-07-18T03:06:38.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.798+0000 7fba753ff700 1 -- 172.21.15.101:0/4262328835 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fba700ffb00 msgr2=0x7fba70195840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:38.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.798+0000 7fba753ff700 1 --2- 172.21.15.101:0/4262328835 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fba700ffb00 0x7fba70195840 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fba60005740 tx=0x7fba60005770 comp rx=0 tx=0).stop 2023-07-18T03:06:38.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.799+0000 7fba753ff700 1 -- 172.21.15.101:0/4262328835 shutdown_connections 2023-07-18T03:06:38.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.799+0000 7fba753ff700 1 --2- 172.21.15.101:0/4262328835 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fba580a1220 0x7fba580a36d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:38.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.799+0000 7fba753ff700 1 --2- 172.21.15.101:0/4262328835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fba70101680 0x7fba7019cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:38.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.799+0000 7fba753ff700 1 --2- 172.21.15.101:0/4262328835 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fba700ffb00 0x7fba70195840 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:38.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.799+0000 7fba753ff700 1 --2- 172.21.15.101:0/4262328835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba70100cf0 0x7fba70195d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:38.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.799+0000 7fba753ff700 1 -- 172.21.15.101:0/4262328835 >> 172.21.15.101:0/4262328835 conn(0x7fba700fb0e0 msgr2=0x7fba700fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:38.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.799+0000 7fba753ff700 1 -- 172.21.15.101:0/4262328835 shutdown_connections 2023-07-18T03:06:38.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:38.799+0000 7fba753ff700 1 -- 172.21.15.101:0/4262328835 wait complete. 2023-07-18T03:06:38.789 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 24 2023-07-18T03:06:38.804 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-07-18T03:04:59.361185+0000', 'modified': '2023-07-18T03:05:31.985959+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24562, 'mds_1': 24557, 'mds_2': 24541}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24488': {'gid': 24488, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.155:6835/943645270', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 943645270}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 943645270}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24515': {'gid': 24515, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.155:6837/48029036', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6836', 'nonce': 48029036}, {'type': 'v1', 'addr': '172.21.15.155:6837', 'nonce': 48029036}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24529': {'gid': 24529, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.179:6835/1268903554', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6834', 'nonce': 1268903554}, {'type': 'v1', 'addr': '172.21.15.179:6835', 'nonce': 1268903554}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24541': {'gid': 24541, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.179:6837/3458593442', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6836', 'nonce': 3458593442}, {'type': 'v1', 'addr': '172.21.15.179:6837', 'nonce': 3458593442}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24557': {'gid': 24557, 'name': 'b', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.155:6839/2055462483', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6838', 'nonce': 2055462483}, {'type': 'v1', 'addr': '172.21.15.155:6839', 'nonce': 2055462483}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24562': {'gid': 24562, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.179:6839/2061828474', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6838', 'nonce': 2061828474}, {'type': 'v1', 'addr': '172.21.15.179:6839', 'nonce': 2061828474}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-18T03:06:38.805 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-07-18T03:06:38.805 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-07-18T03:06:39.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.019+0000 7fb6fc5ff700 1 -- 172.21.15.101:0/3310866749 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb6f4100ce0 msgr2=0x7fb6f4101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:39.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.019+0000 7fb6fc5ff700 1 --2- 172.21.15.101:0/3310866749 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb6f4100ce0 0x7fb6f4101130 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4009990 tx=0x7fb6e4009ca0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.019+0000 7fb6fc5ff700 1 -- 172.21.15.101:0/3310866749 shutdown_connections 2023-07-18T03:06:39.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.019+0000 7fb6fc5ff700 1 --2- 172.21.15.101:0/3310866749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6f40ffaf0 0x7fb6f40fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.019+0000 7fb6fc5ff700 1 --2- 172.21.15.101:0/3310866749 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6f4101670 0x7fb6f4107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.019+0000 7fb6fc5ff700 1 --2- 172.21.15.101:0/3310866749 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb6f4100ce0 0x7fb6f4101130 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.019+0000 7fb6fc5ff700 1 -- 172.21.15.101:0/3310866749 >> 172.21.15.101:0/3310866749 conn(0x7fb6f40fb130 msgr2=0x7fb6f40fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:39.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.019+0000 7fb6fc5ff700 1 -- 172.21.15.101:0/3310866749 shutdown_connections 2023-07-18T03:06:39.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.020+0000 7fb6fc5ff700 1 -- 172.21.15.101:0/3310866749 wait complete. 2023-07-18T03:06:39.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.020+0000 7fb6fc5ff700 1 Processor -- start 2023-07-18T03:06:39.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.021+0000 7fb6fc5ff700 1 -- start start 2023-07-18T03:06:39.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.021+0000 7fb6fc5ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb6f40ffaf0 0x7fb6f4195850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.021+0000 7fb6fc5ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6f4100ce0 0x7fb6f4195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.021+0000 7fb6fc5ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6f4101670 0x7fb6f419ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.021+0000 7fb6fc5ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f419d3d0 con 0x7fb6f40ffaf0 2023-07-18T03:06:39.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.021+0000 7fb6fc5ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f419d510 con 0x7fb6f4101670 2023-07-18T03:06:39.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.021+0000 7fb6fc5ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f4072740 con 0x7fb6f4100ce0 2023-07-18T03:06:39.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.021+0000 7fb6fa39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb6f40ffaf0 0x7fb6f4195850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.021+0000 7fb6fa39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb6f40ffaf0 0x7fb6f4195850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:43748/0 (socket says 172.21.15.101:43748) 2023-07-18T03:06:39.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.021+0000 7fb6fa39b700 1 -- 172.21.15.101:0/3437146187 learned_addr learned my addr 172.21.15.101:0/3437146187 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:39.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.021+0000 7fb6fab9c700 1 --2- 172.21.15.101:0/3437146187 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6f4101670 0x7fb6f419ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.021+0000 7fb6f9b9a700 1 --2- 172.21.15.101:0/3437146187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6f4100ce0 0x7fb6f4195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.022+0000 7fb6fa39b700 1 -- 172.21.15.101:0/3437146187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6f4100ce0 msgr2=0x7fb6f4195d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:39.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.022+0000 7fb6fa39b700 1 --2- 172.21.15.101:0/3437146187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6f4100ce0 0x7fb6f4195d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.022+0000 7fb6fa39b700 1 -- 172.21.15.101:0/3437146187 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6f4101670 msgr2=0x7fb6f419ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:39.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.022+0000 7fb6fa39b700 1 --2- 172.21.15.101:0/3437146187 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6f4101670 0x7fb6f419ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.022+0000 7fb6fa39b700 1 -- 172.21.15.101:0/3437146187 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e40095f0 con 0x7fb6f40ffaf0 2023-07-18T03:06:39.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.022+0000 7fb6f9b9a700 1 --2- 172.21.15.101:0/3437146187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6f4100ce0 0x7fb6f4195d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:06:39.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.022+0000 7fb6fa39b700 1 --2- 172.21.15.101:0/3437146187 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb6f40ffaf0 0x7fb6f4195850 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb6f000d980 tx=0x7fb6f000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:39.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.022+0000 7fb6eb7fe700 1 -- 172.21.15.101:0/3437146187 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f0004020 con 0x7fb6f40ffaf0 2023-07-18T03:06:39.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.022+0000 7fb6fc5ff700 1 -- 172.21.15.101:0/3437146187 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6f4072a20 con 0x7fb6f40ffaf0 2023-07-18T03:06:39.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.022+0000 7fb6eb7fe700 1 -- 172.21.15.101:0/3437146187 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6f0009730 con 0x7fb6f40ffaf0 2023-07-18T03:06:39.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.022+0000 7fb6eb7fe700 1 -- 172.21.15.101:0/3437146187 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f0003a30 con 0x7fb6f40ffaf0 2023-07-18T03:06:39.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.022+0000 7fb6fc5ff700 1 -- 172.21.15.101:0/3437146187 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6f4072f30 con 0x7fb6f40ffaf0 2023-07-18T03:06:39.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.025+0000 7fb6eb7fe700 1 -- 172.21.15.101:0/3437146187 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fb6f00098a0 con 0x7fb6f40ffaf0 2023-07-18T03:06:39.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.026+0000 7fb70febdb80 1 Processor -- start 2023-07-18T03:06:39.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.027+0000 7fb6eb7fe700 1 --2- 172.21.15.101:0/3437146187 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb6e00a11e0 0x7fb6e00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.027+0000 7fb70febdb80 1 -- start start 2023-07-18T03:06:39.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.027+0000 7fb6eb7fe700 1 -- 172.21.15.101:0/3437146187 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fb6f00ccb70 con 0x7fb6f40ffaf0 2023-07-18T03:06:39.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.027+0000 7fb70febdb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af7d189c90 0x55af7d18a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.027+0000 7fb6fab9c700 1 --2- 172.21.15.101:0/3437146187 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb6e00a11e0 0x7fb6e00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.027+0000 7fb70febdb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af7d18a5e0 0x55af7d18aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.027+0000 7fb70febdb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af7d18b430 0x55af7d18b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.027+0000 7fb70febdb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af7d086c60 con 0x55af7d189c90 2023-07-18T03:06:39.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.027+0000 7fb70febdb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af7d0d99f0 con 0x55af7d18a5e0 2023-07-18T03:06:39.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.027+0000 7fb70febdb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af7d0d2e80 con 0x55af7d18b430 2023-07-18T03:06:39.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.027+0000 7fb6fa39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af7d189c90 0x55af7d18a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.027+0000 7fb6fa39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af7d189c90 0x55af7d18a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:43762/0 (socket says 172.21.15.101:43762) 2023-07-18T03:06:39.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.027+0000 7fb6fa39b700 1 -- 172.21.15.101:0/786476463 learned_addr learned my addr 172.21.15.101:0/786476463 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:39.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.027+0000 7fb6f9b9a700 1 --2- 172.21.15.101:0/786476463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af7d18a5e0 0x55af7d18aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.028+0000 7fb6fa39b700 1 -- 172.21.15.101:0/786476463 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af7d18b430 msgr2=0x55af7d18b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:06:39.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.028+0000 7fb6fa39b700 1 --2- 172.21.15.101:0/786476463 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af7d18b430 0x55af7d18b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.028+0000 7fb6fa39b700 1 -- 172.21.15.101:0/786476463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af7d18a5e0 msgr2=0x55af7d18aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:06:39.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.028+0000 7fb6fa39b700 1 --2- 172.21.15.101:0/786476463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af7d18a5e0 0x55af7d18aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.028+0000 7fb6fa39b700 1 -- 172.21.15.101:0/786476463 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af7cf00d90 con 0x55af7d189c90 2023-07-18T03:06:39.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.028+0000 7fb6fa39b700 1 --2- 172.21.15.101:0/786476463 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af7d189c90 0x55af7d18a0a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb6f00a1970 tx=0x7fb6f0099950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27ae85a046d42c40 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:39.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.028+0000 7fb6fab9c700 1 --2- 172.21.15.101:0/3437146187 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb6e00a11e0 0x7fb6e00a3690 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec003f20 tx=0x7fb6ec009ba0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:39.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.028+0000 7fb6e8bfe700 1 -- 172.21.15.101:0/786476463 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f00b2210 con 0x55af7d189c90 2023-07-18T03:06:39.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.028+0000 7fb6e8bfe700 1 -- 172.21.15.101:0/786476463 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6f009da80 con 0x55af7d189c90 2023-07-18T03:06:39.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.028+0000 7fb6e8bfe700 1 -- 172.21.15.101:0/786476463 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f00991a0 con 0x55af7d189c90 2023-07-18T03:06:39.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.029+0000 7fb70febdb80 1 -- 172.21.15.101:0/786476463 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af7d189c90 msgr2=0x55af7d18a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:39.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.029+0000 7fb70febdb80 1 --2- 172.21.15.101:0/786476463 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af7d189c90 0x55af7d18a0a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb6f00a1970 tx=0x7fb6f0099950 comp rx=0 tx=0).stop 2023-07-18T03:06:39.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.029+0000 7fb70febdb80 1 -- 172.21.15.101:0/786476463 shutdown_connections 2023-07-18T03:06:39.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.029+0000 7fb70febdb80 1 --2- 172.21.15.101:0/786476463 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af7d18b430 0x55af7d18b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.029+0000 7fb70febdb80 1 --2- 172.21.15.101:0/786476463 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af7d189c90 0x55af7d18a0a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.029+0000 7fb70febdb80 1 --2- 172.21.15.101:0/786476463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af7d18a5e0 0x55af7d18aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.029+0000 7fb70febdb80 1 -- 172.21.15.101:0/786476463 >> 172.21.15.101:0/786476463 conn(0x55af7d176d50 msgr2=0x55af7d17c180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:39.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.029+0000 7fb70febdb80 1 -- 172.21.15.101:0/786476463 shutdown_connections 2023-07-18T03:06:39.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.029+0000 7fb70febdb80 1 -- 172.21.15.101:0/786476463 wait complete. 2023-07-18T03:06:39.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.030+0000 7fb70febdb80 1 -- start start 2023-07-18T03:06:39.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.030+0000 7fb70febdb80 1 Processor -- start 2023-07-18T03:06:39.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.030+0000 7fb70febdb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af7d189c90 0x55af7d193890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.030+0000 7fb70febdb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af7d18a5e0 0x55af7d193dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.030+0000 7fb70febdb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af7d18b430 0x55af7d19ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.030+0000 7fb70febdb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af7d072a70 con 0x55af7d18a5e0 2023-07-18T03:06:39.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.030+0000 7fb70febdb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af7d072770 con 0x55af7d189c90 2023-07-18T03:06:39.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.030+0000 7fb70febdb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af7cf568d0 con 0x55af7d18b430 2023-07-18T03:06:39.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.030+0000 7fb70febdb80 10 client.0 _ll_register_callbacks cb 0x55af7d070710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:06:39.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.030+0000 7fb6f9b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af7d18a5e0 0x55af7d193dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.030+0000 7fb6f9b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af7d18a5e0 0x55af7d193dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:43778/0 (socket says 172.21.15.101:43778) 2023-07-18T03:06:39.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.030+0000 7fb6f9b9a700 1 -- 172.21.15.101:0/4045649629 learned_addr learned my addr 172.21.15.101:0/4045649629 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:39.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.031+0000 7fb6fa39b700 1 --2- 172.21.15.101:0/4045649629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af7d189c90 0x55af7d193890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.031+0000 7fb6fab9c700 1 --2- 172.21.15.101:0/4045649629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af7d18b430 0x55af7d19ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.031+0000 7fb6f9b9a700 1 -- 172.21.15.101:0/4045649629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af7d18b430 msgr2=0x55af7d19ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:06:39.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.031+0000 7fb6f9b9a700 1 --2- 172.21.15.101:0/4045649629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af7d18b430 0x55af7d19ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.031+0000 7fb6f9b9a700 1 -- 172.21.15.101:0/4045649629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af7d189c90 msgr2=0x55af7d193890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:06:39.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.031+0000 7fb6f9b9a700 1 --2- 172.21.15.101:0/4045649629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af7d189c90 0x55af7d193890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.031+0000 7fb6f9b9a700 1 -- 172.21.15.101:0/4045649629 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6f4072f30 con 0x55af7d18a5e0 2023-07-18T03:06:39.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.031+0000 7fb6fa39b700 1 --2- 172.21.15.101:0/4045649629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af7d189c90 0x55af7d193890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:06:39.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.031+0000 7fb6f9b9a700 1 --2- 172.21.15.101:0/4045649629 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af7d18a5e0 0x55af7d193dd0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4009990 tx=0x7fb6e4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8524dc00593dec1d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:39.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.031+0000 7fb6ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:06:39.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.031+0000 7fb6ddffb700 1 -- 172.21.15.101:0/4045649629 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e400bd40 con 0x55af7d18a5e0 2023-07-18T03:06:39.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.031+0000 7fb70febdb80 1 -- 172.21.15.101:0/4045649629 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55af7d02c4a0 con 0x55af7d18a5e0 2023-07-18T03:06:39.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.031+0000 7fb6ddffb700 1 -- 172.21.15.101:0/4045649629 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fb6e4004500 con 0x55af7d18a5e0 2023-07-18T03:06:39.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.032+0000 7fb6ddffb700 1 -- 172.21.15.101:0/4045649629 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6e400bea0 con 0x55af7d18a5e0 2023-07-18T03:06:39.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.032+0000 7fb70febdb80 10 client.15081 fetch_fsmap learned FSMap version 24 2023-07-18T03:06:39.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.032+0000 7fb70febdb80 1 -- 172.21.15.101:0/4045649629 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55af7cee9c10 con 0x55af7d18a5e0 2023-07-18T03:06:39.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.032+0000 7fb6ddffb700 1 -- 172.21.15.101:0/4045649629 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55af7d02c4a0 con 0x55af7d18a5e0 2023-07-18T03:06:39.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.032+0000 7fb6ddffb700 1 -- 172.21.15.101:0/4045649629 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb6e4003c30 con 0x55af7d18a5e0 2023-07-18T03:06:39.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.032+0000 7fb70febdb80 10 client.15081 fetch_fsmap finished waiting for FSMap version 24 2023-07-18T03:06:39.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.032+0000 7fb70febdb80 10 client.15081 resolve_mds: resolved name 'c' to daemon mds.c 2023-07-18T03:06:39.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.032+0000 7fb70febdb80 20 client.15081 populate_metadata read hostname 'smithi101' 2023-07-18T03:06:39.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.032+0000 7fb70febdb80 1 --2- 172.21.15.101:0/4045649629 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55af7d19eb60 0x55af7d19df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.032+0000 7fb70febdb80 4 client.15081 mds_command: new command op to 24541 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:06:39.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.032+0000 7fb70febdb80 1 -- 172.21.15.101:0/4045649629 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55af7d0cbe60 con 0x55af7d19eb60 2023-07-18T03:06:39.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.033+0000 7fb6fa39b700 1 --2- 172.21.15.101:0/4045649629 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55af7d19eb60 0x55af7d19df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.033+0000 7fb6fa39b700 1 --2- 172.21.15.101:0/4045649629 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55af7d19eb60 0x55af7d19df10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e8ca46c3ccb23fee server_cookie=a806e29c17f60ea3 in_seq=0 out_seq=0 2023-07-18T03:06:39.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.033+0000 7fb6ddffb700 10 client.15081 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:06:39.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.034+0000 7fb6ddffb700 1 -- 172.21.15.101:0/4045649629 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55af7d0cbe60 con 0x55af7d19eb60 2023-07-18T03:06:39.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.034+0000 7fb6ddffb700 10 client.15081 handle_command_reply: tid=0 2023-07-18T03:06:39.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.034+0000 7fb70febdb80 1 client.15081 shutdown 2023-07-18T03:06:39.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.034+0000 7fb70febdb80 1 -- 172.21.15.101:0/4045649629 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af7d18a5e0 msgr2=0x55af7d193dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:39.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.034+0000 7fb70febdb80 1 --2- 172.21.15.101:0/4045649629 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af7d18a5e0 0x55af7d193dd0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4009990 tx=0x7fb6e4005800 comp rx=0 tx=0).stop 2023-07-18T03:06:39.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.034+0000 7fb70febdb80 1 -- 172.21.15.101:0/4045649629 shutdown_connections 2023-07-18T03:06:39.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.034+0000 7fb70febdb80 1 --2- 172.21.15.101:0/4045649629 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55af7d19eb60 0x55af7d19df10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.034+0000 7fb70febdb80 1 --2- 172.21.15.101:0/4045649629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af7d189c90 0x55af7d193890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.034+0000 7fb70febdb80 1 --2- 172.21.15.101:0/4045649629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af7d18b430 0x55af7d19ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.034+0000 7fb70febdb80 1 --2- 172.21.15.101:0/4045649629 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af7d18a5e0 0x55af7d193dd0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.034+0000 7fb6ddffb700 0 client.15081 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:06:39.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.034+0000 7fb70febdb80 1 -- 172.21.15.101:0/4045649629 >> 172.21.15.101:0/4045649629 conn(0x55af7d176d50 msgr2=0x55af7d17b860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:39.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.034+0000 7fb70febdb80 1 -- 172.21.15.101:0/4045649629 shutdown_connections 2023-07-18T03:06:39.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.034+0000 7fb70febdb80 1 -- 172.21.15.101:0/4045649629 wait complete. 2023-07-18T03:06:39.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.034+0000 7fb70febdb80 20 client.15081 trim_cache size 0 max 16384 2023-07-18T03:06:39.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.052+0000 7fb70febdb80 1 Processor -- start 2023-07-18T03:06:39.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.052+0000 7fb70febdb80 1 -- start start 2023-07-18T03:06:39.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.053+0000 7fb70febdb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af7d176d50 0x55af7d191780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.053+0000 7fb70febdb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af7d189c90 0x55af7d191cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.053+0000 7fb70febdb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af7d18a5e0 0x55af7d19add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.053+0000 7fb70febdb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af7d091480 con 0x55af7d18a5e0 2023-07-18T03:06:39.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.053+0000 7fb70febdb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af7d06ed50 con 0x55af7d176d50 2023-07-18T03:06:39.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.053+0000 7fb70febdb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af7d19fe50 con 0x55af7d189c90 2023-07-18T03:06:39.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.053+0000 7fb6fab9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af7d18a5e0 0x55af7d19add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.053+0000 7fb6fab9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af7d18a5e0 0x55af7d19add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:43790/0 (socket says 172.21.15.101:43790) 2023-07-18T03:06:39.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.053+0000 7fb6fab9c700 1 -- 172.21.15.101:0/2880844541 learned_addr learned my addr 172.21.15.101:0/2880844541 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:39.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.053+0000 7fb6f9b9a700 1 --2- 172.21.15.101:0/2880844541 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af7d189c90 0x55af7d191cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.053+0000 7fb6fa39b700 1 --2- 172.21.15.101:0/2880844541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af7d176d50 0x55af7d191780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.053+0000 7fb6fab9c700 1 -- 172.21.15.101:0/2880844541 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af7d189c90 msgr2=0x55af7d191cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:06:39.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.053+0000 7fb6fab9c700 1 --2- 172.21.15.101:0/2880844541 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af7d189c90 0x55af7d191cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.053+0000 7fb6fab9c700 1 -- 172.21.15.101:0/2880844541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af7d176d50 msgr2=0x55af7d191780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:06:39.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.053+0000 7fb6fab9c700 1 --2- 172.21.15.101:0/2880844541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af7d176d50 0x55af7d191780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.053+0000 7fb6fab9c700 1 -- 172.21.15.101:0/2880844541 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af7cf27200 con 0x55af7d18a5e0 2023-07-18T03:06:39.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.053+0000 7fb6f9b9a700 1 --2- 172.21.15.101:0/2880844541 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af7d189c90 0x55af7d191cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:06:39.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.053+0000 7fb6fa39b700 1 --2- 172.21.15.101:0/2880844541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af7d176d50 0x55af7d191780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:06:39.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.054+0000 7fb6fab9c700 1 --2- 172.21.15.101:0/2880844541 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af7d18a5e0 0x55af7d19add0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec004710 tx=0x7fb6ec015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e62d14839a9add0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:39.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.054+0000 7fb6e93ff700 1 -- 172.21.15.101:0/2880844541 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec007dd0 con 0x55af7d18a5e0 2023-07-18T03:06:39.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.054+0000 7fb6e93ff700 1 -- 172.21.15.101:0/2880844541 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6ec01f040 con 0x55af7d18a5e0 2023-07-18T03:06:39.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.054+0000 7fb6e93ff700 1 -- 172.21.15.101:0/2880844541 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec016780 con 0x55af7d18a5e0 2023-07-18T03:06:39.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.054+0000 7fb70febdb80 1 -- 172.21.15.101:0/2880844541 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af7d18a5e0 msgr2=0x55af7d19add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:39.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.054+0000 7fb70febdb80 1 --2- 172.21.15.101:0/2880844541 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af7d18a5e0 0x55af7d19add0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec004710 tx=0x7fb6ec015000 comp rx=0 tx=0).stop 2023-07-18T03:06:39.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.054+0000 7fb70febdb80 1 -- 172.21.15.101:0/2880844541 shutdown_connections 2023-07-18T03:06:39.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.054+0000 7fb70febdb80 1 --2- 172.21.15.101:0/2880844541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af7d176d50 0x55af7d191780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.054+0000 7fb70febdb80 1 --2- 172.21.15.101:0/2880844541 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af7d18a5e0 0x55af7d19add0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.054+0000 7fb70febdb80 1 --2- 172.21.15.101:0/2880844541 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af7d189c90 0x55af7d191cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.054+0000 7fb70febdb80 1 -- 172.21.15.101:0/2880844541 >> 172.21.15.101:0/2880844541 conn(0x55af7d178640 msgr2=0x55af7d17b860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:39.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.054+0000 7fb70febdb80 1 -- 172.21.15.101:0/2880844541 shutdown_connections 2023-07-18T03:06:39.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.055+0000 7fb70febdb80 1 -- 172.21.15.101:0/2880844541 wait complete. 2023-07-18T03:06:39.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.055+0000 7fb70febdb80 1 -- start start 2023-07-18T03:06:39.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.055+0000 7fb70febdb80 1 Processor -- start 2023-07-18T03:06:39.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.055+0000 7fb70febdb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af7d176d50 0x55af7d196b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.055+0000 7fb70febdb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af7d189c90 0x55af7d197050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.055+0000 7fb70febdb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af7d18a5e0 0x55af7d19a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.055+0000 7fb70febdb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af7d19a5b0 con 0x55af7d176d50 2023-07-18T03:06:39.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.055+0000 7fb70febdb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af7d19a720 con 0x55af7d189c90 2023-07-18T03:06:39.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.055+0000 7fb70febdb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af7d19a8e0 con 0x55af7d18a5e0 2023-07-18T03:06:39.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.055+0000 7fb70febdb80 10 client.0 _ll_register_callbacks cb 0x7fb6e4005350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:06:39.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.056+0000 7fb6fa39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af7d176d50 0x55af7d196b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.056+0000 7fb6fa39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af7d176d50 0x55af7d196b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:43796/0 (socket says 172.21.15.101:43796) 2023-07-18T03:06:39.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.056+0000 7fb6fa39b700 1 -- 172.21.15.101:0/658999074 learned_addr learned my addr 172.21.15.101:0/658999074 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:39.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.056+0000 7fb6fab9c700 1 --2- 172.21.15.101:0/658999074 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af7d18a5e0 0x55af7d19a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.056+0000 7fb6f9b9a700 1 --2- 172.21.15.101:0/658999074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af7d189c90 0x55af7d197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.056+0000 7fb6fa39b700 1 -- 172.21.15.101:0/658999074 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af7d18a5e0 msgr2=0x55af7d19a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:06:39.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.056+0000 7fb6fa39b700 1 --2- 172.21.15.101:0/658999074 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af7d18a5e0 0x55af7d19a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.056+0000 7fb6fa39b700 1 -- 172.21.15.101:0/658999074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af7d189c90 msgr2=0x55af7d197050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:06:39.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.056+0000 7fb6fa39b700 1 --2- 172.21.15.101:0/658999074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af7d189c90 0x55af7d197050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.056+0000 7fb6fa39b700 1 -- 172.21.15.101:0/658999074 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6ec004390 con 0x55af7d176d50 2023-07-18T03:06:39.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.056+0000 7fb6f9b9a700 1 --2- 172.21.15.101:0/658999074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af7d189c90 0x55af7d197050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:06:39.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.056+0000 7fb6fab9c700 1 --2- 172.21.15.101:0/658999074 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af7d18a5e0 0x55af7d19a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:06:39.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.056+0000 7fb6fa39b700 1 --2- 172.21.15.101:0/658999074 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af7d176d50 0x55af7d196b10 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0095f50 tx=0x7fb6f0095d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04c00c9da63611 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:39.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.056+0000 7fb6ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:06:39.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.056+0000 7fb6ddffb700 1 -- 172.21.15.101:0/658999074 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f00a1ea0 con 0x55af7d176d50 2023-07-18T03:06:39.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.056+0000 7fb6ddffb700 1 -- 172.21.15.101:0/658999074 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fb6f0098f40 con 0x55af7d176d50 2023-07-18T03:06:39.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.056+0000 7fb70febdb80 1 -- 172.21.15.101:0/658999074 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb6f009da80 con 0x55af7d176d50 2023-07-18T03:06:39.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.057+0000 7fb6ddffb700 1 -- 172.21.15.101:0/658999074 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6f0099100 con 0x55af7d176d50 2023-07-18T03:06:39.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.057+0000 7fb70febdb80 10 client.15087 fetch_fsmap learned FSMap version 24 2023-07-18T03:06:39.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.057+0000 7fb70febdb80 1 -- 172.21.15.101:0/658999074 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55af7ceff260 con 0x55af7d176d50 2023-07-18T03:06:39.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.057+0000 7fb6ddffb700 1 -- 172.21.15.101:0/658999074 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6f009da80 con 0x55af7d176d50 2023-07-18T03:06:39.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.057+0000 7fb6ddffb700 1 -- 172.21.15.101:0/658999074 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb6f0023bc0 con 0x55af7d176d50 2023-07-18T03:06:39.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.057+0000 7fb70febdb80 10 client.15087 fetch_fsmap finished waiting for FSMap version 24 2023-07-18T03:06:39.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.057+0000 7fb70febdb80 10 client.15087 resolve_mds: resolved name 'c' to daemon mds.c 2023-07-18T03:06:39.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.057+0000 7fb70febdb80 20 client.15087 populate_metadata read hostname 'smithi101' 2023-07-18T03:06:39.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.057+0000 7fb70febdb80 1 --2- 172.21.15.101:0/658999074 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55af7d18b430 0x55af7d1f5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.057+0000 7fb70febdb80 4 client.15087 mds_command: new command op to 24541 tid=0[{"prefix": "status"}] 2023-07-18T03:06:39.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.057+0000 7fb70febdb80 1 -- 172.21.15.101:0/658999074 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55af7d08e000 con 0x55af7d18b430 2023-07-18T03:06:39.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.057+0000 7fb6f9b9a700 1 --2- 172.21.15.101:0/658999074 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55af7d18b430 0x55af7d1f5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.058+0000 7fb6ddffb700 10 client.15087 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:06:39.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.058+0000 7fb6f9b9a700 1 --2- 172.21.15.101:0/658999074 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55af7d18b430 0x55af7d1f5410 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9b2c0c73e51d603e server_cookie=d6fe056a2236bf0 in_seq=0 out_seq=0 2023-07-18T03:06:39.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.058+0000 7fb6ddffb700 1 -- 172.21.15.101:0/658999074 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55af7d08e000 con 0x55af7d18b430 2023-07-18T03:06:39.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.058+0000 7fb6ddffb700 10 client.15087 handle_command_reply: tid=0 2023-07-18T03:06:39.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.058+0000 7fb70febdb80 1 client.15087 shutdown 2023-07-18T03:06:39.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.058+0000 7fb70febdb80 1 -- 172.21.15.101:0/658999074 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af7d176d50 msgr2=0x55af7d196b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:39.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.058+0000 7fb70febdb80 1 --2- 172.21.15.101:0/658999074 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af7d176d50 0x55af7d196b10 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0095f50 tx=0x7fb6f0095d90 comp rx=0 tx=0).stop 2023-07-18T03:06:39.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.058+0000 7fb70febdb80 1 -- 172.21.15.101:0/658999074 shutdown_connections 2023-07-18T03:06:39.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.058+0000 7fb70febdb80 1 --2- 172.21.15.101:0/658999074 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55af7d18b430 0x55af7d1f5410 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.058+0000 7fb70febdb80 1 --2- 172.21.15.101:0/658999074 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af7d18a5e0 0x55af7d19a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.058+0000 7fb6ddffb700 0 client.15087 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:06:39.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.058+0000 7fb70febdb80 1 --2- 172.21.15.101:0/658999074 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af7d176d50 0x55af7d196b10 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.058+0000 7fb70febdb80 1 --2- 172.21.15.101:0/658999074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af7d189c90 0x55af7d197050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.058+0000 7fb70febdb80 1 -- 172.21.15.101:0/658999074 >> 172.21.15.101:0/658999074 conn(0x55af7d178640 msgr2=0x55af7d19df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:39.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.058+0000 7fb70febdb80 1 -- 172.21.15.101:0/658999074 shutdown_connections 2023-07-18T03:06:39.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.059+0000 7fb70febdb80 1 -- 172.21.15.101:0/658999074 wait complete. 2023-07-18T03:06:39.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.059+0000 7fb70febdb80 20 client.15087 trim_cache size 0 max 16384 2023-07-18T03:06:39.058 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:06:39.058 INFO:teuthology.orchestra.run.smithi101.stdout: "cluster_fsid": "b1782daa-2516-11ee-9b34-001a4aab830c", 2023-07-18T03:06:39.058 INFO:teuthology.orchestra.run.smithi101.stdout: "whoami": 2, 2023-07-18T03:06:39.059 INFO:teuthology.orchestra.run.smithi101.stdout: "id": 24541, 2023-07-18T03:06:39.059 INFO:teuthology.orchestra.run.smithi101.stdout: "want_state": "up:active", 2023-07-18T03:06:39.059 INFO:teuthology.orchestra.run.smithi101.stdout: "state": "up:active", 2023-07-18T03:06:39.059 INFO:teuthology.orchestra.run.smithi101.stdout: "fs_name": "cephfs", 2023-07-18T03:06:39.059 INFO:teuthology.orchestra.run.smithi101.stdout: "rank_uptime": 68.076813102000003, 2023-07-18T03:06:39.059 INFO:teuthology.orchestra.run.smithi101.stdout: "mdsmap_epoch": 24, 2023-07-18T03:06:39.059 INFO:teuthology.orchestra.run.smithi101.stdout: "osdmap_epoch": 91, 2023-07-18T03:06:39.060 INFO:teuthology.orchestra.run.smithi101.stdout: "osdmap_epoch_barrier": 89, 2023-07-18T03:06:39.060 INFO:teuthology.orchestra.run.smithi101.stdout: "uptime": 118.320698209 2023-07-18T03:06:39.060 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:06:39.061 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.059+0000 7fb6fc5ff700 1 -- 172.21.15.101:0/3437146187 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb6e00a11e0 msgr2=0x7fb6e00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:39.061 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.059+0000 7fb6fc5ff700 1 --2- 172.21.15.101:0/3437146187 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb6e00a11e0 0x7fb6e00a3690 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec003f20 tx=0x7fb6ec009ba0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.061 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.059+0000 7fb6fc5ff700 1 -- 172.21.15.101:0/3437146187 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb6f40ffaf0 msgr2=0x7fb6f4195850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:39.061 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.059+0000 7fb6fc5ff700 1 --2- 172.21.15.101:0/3437146187 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb6f40ffaf0 0x7fb6f4195850 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb6f000d980 tx=0x7fb6f000dc90 comp rx=0 tx=0).stop 2023-07-18T03:06:39.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.059+0000 7fb6fc5ff700 1 -- 172.21.15.101:0/3437146187 shutdown_connections 2023-07-18T03:06:39.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.059+0000 7fb6fc5ff700 1 --2- 172.21.15.101:0/3437146187 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb6e00a11e0 0x7fb6e00a3690 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.059+0000 7fb6fc5ff700 1 --2- 172.21.15.101:0/3437146187 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6f4101670 0x7fb6f419ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.059+0000 7fb6fc5ff700 1 --2- 172.21.15.101:0/3437146187 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb6f40ffaf0 0x7fb6f4195850 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.059+0000 7fb6fc5ff700 1 --2- 172.21.15.101:0/3437146187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6f4100ce0 0x7fb6f4195d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.059+0000 7fb6fc5ff700 1 -- 172.21.15.101:0/3437146187 >> 172.21.15.101:0/3437146187 conn(0x7fb6f40fb130 msgr2=0x7fb6f40fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:39.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.059+0000 7fb6fc5ff700 1 -- 172.21.15.101:0/3437146187 shutdown_connections 2023-07-18T03:06:39.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.059+0000 7fb6fc5ff700 1 -- 172.21.15.101:0/3437146187 wait complete. 2023-07-18T03:06:39.063 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:38 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/643446297' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:06:39.063 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:38 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/4262328835' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:06:39.064 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-07-18T03:06:39.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:38 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/643446297' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:06:39.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:38 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/4262328835' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:06:39.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.276+0000 7f02e67ff700 1 -- 172.21.15.101:0/1475224409 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f02e0102f40 msgr2=0x7f02e0105320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:39.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.276+0000 7f02e67ff700 1 --2- 172.21.15.101:0/1475224409 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f02e0102f40 0x7f02e0105320 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f02d4009990 tx=0x7f02d4009ca0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.276+0000 7f02e67ff700 1 -- 172.21.15.101:0/1475224409 shutdown_connections 2023-07-18T03:06:39.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.276+0000 7f02e67ff700 1 --2- 172.21.15.101:0/1475224409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02e0100620 0x7f02e0102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.276+0000 7f02e67ff700 1 --2- 172.21.15.101:0/1475224409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02e0105860 0x7f02e0107c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.276+0000 7f02e67ff700 1 --2- 172.21.15.101:0/1475224409 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f02e0102f40 0x7f02e0105320 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.276+0000 7f02e67ff700 1 -- 172.21.15.101:0/1475224409 >> 172.21.15.101:0/1475224409 conn(0x7f02e00fa000 msgr2=0x7f02e00fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:39.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.276+0000 7f02e67ff700 1 -- 172.21.15.101:0/1475224409 shutdown_connections 2023-07-18T03:06:39.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.276+0000 7f02e67ff700 1 -- 172.21.15.101:0/1475224409 wait complete. 2023-07-18T03:06:39.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.277+0000 7f02e67ff700 1 Processor -- start 2023-07-18T03:06:39.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.278+0000 7f02e67ff700 1 -- start start 2023-07-18T03:06:39.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.278+0000 7f02e67ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02e0100620 0x7f02e0193430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.278+0000 7f02e67ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02e0102f40 0x7f02e0193970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.278+0000 7f02e67ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f02e0105860 0x7f02e01a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.278+0000 7f02e67ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e0194260 con 0x7f02e0105860 2023-07-18T03:06:39.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.278+0000 7f02e67ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e01a3b90 con 0x7f02e0100620 2023-07-18T03:06:39.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.278+0000 7f02e67ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e01a3d00 con 0x7f02e0102f40 2023-07-18T03:06:39.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.278+0000 7f02dffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02e0100620 0x7f02e0193430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.278+0000 7f02dffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02e0100620 0x7f02e0193430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:41130/0 (socket says 172.21.15.101:41130) 2023-07-18T03:06:39.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.278+0000 7f02e4d9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f02e0105860 0x7f02e01a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.278+0000 7f02dffff700 1 -- 172.21.15.101:0/3448013737 learned_addr learned my addr 172.21.15.101:0/3448013737 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:39.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.278+0000 7f02df7fe700 1 --2- 172.21.15.101:0/3448013737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02e0102f40 0x7f02e0193970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.279+0000 7f02dffff700 1 -- 172.21.15.101:0/3448013737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02e0102f40 msgr2=0x7f02e0193970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:39.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.279+0000 7f02dffff700 1 --2- 172.21.15.101:0/3448013737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02e0102f40 0x7f02e0193970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.279+0000 7f02dffff700 1 -- 172.21.15.101:0/3448013737 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f02e0105860 msgr2=0x7f02e01a3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:39.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.279+0000 7f02dffff700 1 --2- 172.21.15.101:0/3448013737 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f02e0105860 0x7f02e01a3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.279+0000 7f02dffff700 1 -- 172.21.15.101:0/3448013737 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d40095f0 con 0x7f02e0100620 2023-07-18T03:06:39.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.279+0000 7f02dffff700 1 --2- 172.21.15.101:0/3448013737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02e0100620 0x7f02e0193430 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f02d000e9a0 tx=0x7f02d000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:39.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.279+0000 7f02dd7fa700 1 -- 172.21.15.101:0/3448013737 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d0009790 con 0x7f02e0100620 2023-07-18T03:06:39.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.279+0000 7f02dd7fa700 1 -- 172.21.15.101:0/3448013737 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02d0010430 con 0x7f02e0100620 2023-07-18T03:06:39.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.279+0000 7f02e67ff700 1 -- 172.21.15.101:0/3448013737 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02e01a3fe0 con 0x7f02e0100620 2023-07-18T03:06:39.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.279+0000 7f02dd7fa700 1 -- 172.21.15.101:0/3448013737 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d0018680 con 0x7f02e0100620 2023-07-18T03:06:39.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.279+0000 7f02e67ff700 1 -- 172.21.15.101:0/3448013737 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02e01a4400 con 0x7f02e0100620 2023-07-18T03:06:39.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.282+0000 7f02dd7fa700 1 -- 172.21.15.101:0/3448013737 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f02d00105a0 con 0x7f02e0100620 2023-07-18T03:06:39.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.283+0000 7f02fa0bcb80 1 Processor -- start 2023-07-18T03:06:39.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.284+0000 7f02fa0bcb80 1 -- start start 2023-07-18T03:06:39.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.284+0000 7f02fa0bcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5644f1ad1480 0x5644f1ad1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.284+0000 7f02fa0bcb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644f1ad1dd0 0x5644f1ad2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.284+0000 7f02fa0bcb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644f1ad2c20 0x5644f1ad3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.284+0000 7f02fa0bcb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644f19d6c60 con 0x5644f1ad1dd0 2023-07-18T03:06:39.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.284+0000 7f02fa0bcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644f1a299f0 con 0x5644f1ad1480 2023-07-18T03:06:39.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.284+0000 7f02fa0bcb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644f1a22e80 con 0x5644f1ad2c20 2023-07-18T03:06:39.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.284+0000 7f02dd7fa700 1 --2- 172.21.15.101:0/3448013737 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f02c80a11e0 0x7f02c80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.284+0000 7f02dffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644f1ad1dd0 0x5644f1ad2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.284+0000 7f02e4d9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5644f1ad1480 0x5644f1ad1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.284+0000 7f02dffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644f1ad1dd0 0x5644f1ad2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:43822/0 (socket says 172.21.15.101:43822) 2023-07-18T03:06:39.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.284+0000 7f02dffff700 1 -- 172.21.15.101:0/4160319840 learned_addr learned my addr 172.21.15.101:0/4160319840 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:39.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.284+0000 7f02df7fe700 1 --2- 172.21.15.101:0/4160319840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644f1ad2c20 0x5644f1ad3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.284+0000 7f02e4d9c700 1 --2- 172.21.15.101:0/3448013737 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f02c80a11e0 0x7f02c80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.284+0000 7f02dd7fa700 1 -- 172.21.15.101:0/3448013737 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f02d0014070 con 0x7f02e0100620 2023-07-18T03:06:39.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.285+0000 7f02e4d9c700 1 -- 172.21.15.101:0/4160319840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644f1ad2c20 msgr2=0x5644f1ad3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:06:39.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.285+0000 7f02e4d9c700 1 --2- 172.21.15.101:0/4160319840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644f1ad2c20 0x5644f1ad3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.285+0000 7f02e4d9c700 1 -- 172.21.15.101:0/4160319840 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644f1ad1dd0 msgr2=0x5644f1ad2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:06:39.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.285+0000 7f02e4d9c700 1 --2- 172.21.15.101:0/4160319840 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644f1ad1dd0 0x5644f1ad2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.285+0000 7f02e4d9c700 1 -- 172.21.15.101:0/4160319840 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5644f1850d90 con 0x5644f1ad1480 2023-07-18T03:06:39.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.285+0000 7f02e4d9c700 1 --2- 172.21.15.101:0/4160319840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5644f1ad1480 0x5644f1ad1890 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f02d8008e70 tx=0x7f02d801a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50509482f34b4fd9 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:39.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.285+0000 7f02e4d9c700 1 --2- 172.21.15.101:0/3448013737 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f02c80a11e0 0x7f02c80a3690 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f02d8009550 tx=0x7f02d800e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:39.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.285+0000 7f02c67fc700 1 -- 172.21.15.101:0/4160319840 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d800e070 con 0x5644f1ad1480 2023-07-18T03:06:39.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.286+0000 7f02c67fc700 1 -- 172.21.15.101:0/4160319840 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02d800f040 con 0x5644f1ad1480 2023-07-18T03:06:39.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.286+0000 7f02c67fc700 1 -- 172.21.15.101:0/4160319840 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d8007500 con 0x5644f1ad1480 2023-07-18T03:06:39.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.286+0000 7f02fa0bcb80 1 -- 172.21.15.101:0/4160319840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5644f1ad1480 msgr2=0x5644f1ad1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:39.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.286+0000 7f02fa0bcb80 1 --2- 172.21.15.101:0/4160319840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5644f1ad1480 0x5644f1ad1890 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f02d8008e70 tx=0x7f02d801a6c0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.286+0000 7f02fa0bcb80 1 -- 172.21.15.101:0/4160319840 shutdown_connections 2023-07-18T03:06:39.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.286+0000 7f02fa0bcb80 1 --2- 172.21.15.101:0/4160319840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5644f1ad1480 0x5644f1ad1890 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.286+0000 7f02fa0bcb80 1 --2- 172.21.15.101:0/4160319840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644f1ad2c20 0x5644f1ad3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.286+0000 7f02fa0bcb80 1 --2- 172.21.15.101:0/4160319840 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644f1ad1dd0 0x5644f1ad2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.286+0000 7f02fa0bcb80 1 -- 172.21.15.101:0/4160319840 >> 172.21.15.101:0/4160319840 conn(0x5644f1ac6d50 msgr2=0x5644f1acc180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:39.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.286+0000 7f02fa0bcb80 1 -- 172.21.15.101:0/4160319840 shutdown_connections 2023-07-18T03:06:39.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.286+0000 7f02fa0bcb80 1 -- 172.21.15.101:0/4160319840 wait complete. 2023-07-18T03:06:39.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.287+0000 7f02fa0bcb80 1 -- start start 2023-07-18T03:06:39.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.287+0000 7f02fa0bcb80 1 Processor -- start 2023-07-18T03:06:39.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.288+0000 7f02fa0bcb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644f1ad1480 0x5644f1adb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.288+0000 7f02fa0bcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5644f1ad1dd0 0x5644f1adb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.288+0000 7f02fa0bcb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644f1ad2c20 0x5644f1ae2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.288+0000 7f02fa0bcb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644f19c2a70 con 0x5644f1ad1480 2023-07-18T03:06:39.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.288+0000 7f02fa0bcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644f19c2770 con 0x5644f1ad1dd0 2023-07-18T03:06:39.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.288+0000 7f02fa0bcb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644f18a68d0 con 0x5644f1ad2c20 2023-07-18T03:06:39.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.288+0000 7f02fa0bcb80 10 client.0 _ll_register_callbacks cb 0x5644f19c0710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:06:39.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.288+0000 7f02dffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644f1ad1480 0x5644f1adb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.288+0000 7f02dffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644f1ad1480 0x5644f1adb1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:43824/0 (socket says 172.21.15.101:43824) 2023-07-18T03:06:39.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.288+0000 7f02dffff700 1 -- 172.21.15.101:0/1725522989 learned_addr learned my addr 172.21.15.101:0/1725522989 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:39.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.288+0000 7f02df7fe700 1 --2- 172.21.15.101:0/1725522989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5644f1ad1dd0 0x5644f1adb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.288+0000 7f02e4d9c700 1 --2- 172.21.15.101:0/1725522989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644f1ad2c20 0x5644f1ae2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.289+0000 7f02dffff700 1 -- 172.21.15.101:0/1725522989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644f1ad2c20 msgr2=0x5644f1ae2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:06:39.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.289+0000 7f02dffff700 1 --2- 172.21.15.101:0/1725522989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644f1ad2c20 0x5644f1ae2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.289+0000 7f02dffff700 1 -- 172.21.15.101:0/1725522989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5644f1ad1dd0 msgr2=0x5644f1adb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:06:39.284 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.289+0000 7f02dffff700 1 --2- 172.21.15.101:0/1725522989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5644f1ad1dd0 0x5644f1adb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.284 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.289+0000 7f02dffff700 1 -- 172.21.15.101:0/1725522989 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02d8008a20 con 0x5644f1ad1480 2023-07-18T03:06:39.284 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.289+0000 7f02dffff700 1 --2- 172.21.15.101:0/1725522989 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644f1ad1480 0x5644f1adb1c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f02d00963b0 tx=0x7f02d00cab30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb4760da0935507d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:39.284 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.289+0000 7f02bf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:06:39.284 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.289+0000 7f02bf7fe700 1 -- 172.21.15.101:0/1725522989 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d00b8310 con 0x5644f1ad1480 2023-07-18T03:06:39.284 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.289+0000 7f02fa0bcb80 1 -- 172.21.15.101:0/1725522989 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5644f197c4a0 con 0x5644f1ad1480 2023-07-18T03:06:39.284 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.289+0000 7f02bf7fe700 1 -- 172.21.15.101:0/1725522989 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f02d009df30 con 0x5644f1ad1480 2023-07-18T03:06:39.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.289+0000 7f02bf7fe700 1 -- 172.21.15.101:0/1725522989 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02d00a2350 con 0x5644f1ad1480 2023-07-18T03:06:39.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.289+0000 7f02fa0bcb80 10 client.15099 fetch_fsmap learned FSMap version 24 2023-07-18T03:06:39.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.289+0000 7f02fa0bcb80 1 -- 172.21.15.101:0/1725522989 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5644f1839c10 con 0x5644f1ad1480 2023-07-18T03:06:39.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.289+0000 7f02bf7fe700 1 -- 172.21.15.101:0/1725522989 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5644f197c4a0 con 0x5644f1ad1480 2023-07-18T03:06:39.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.290+0000 7f02bf7fe700 1 -- 172.21.15.101:0/1725522989 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f02d0065410 con 0x5644f1ad1480 2023-07-18T03:06:39.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.290+0000 7f02fa0bcb80 10 client.15099 fetch_fsmap finished waiting for FSMap version 24 2023-07-18T03:06:39.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.290+0000 7f02fa0bcb80 10 client.15099 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-18T03:06:39.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.290+0000 7f02fa0bcb80 20 client.15099 populate_metadata read hostname 'smithi101' 2023-07-18T03:06:39.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.290+0000 7f02fa0bcb80 1 --2- 172.21.15.101:0/1725522989 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5644f1ae6400 0x5644f1ae57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.290+0000 7f02fa0bcb80 4 client.15099 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:06:39.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.290+0000 7f02fa0bcb80 1 -- 172.21.15.101:0/1725522989 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5644f1a1be60 con 0x5644f1ae6400 2023-07-18T03:06:39.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.290+0000 7f02df7fe700 1 --2- 172.21.15.101:0/1725522989 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5644f1ae6400 0x5644f1ae57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.291+0000 7f02bf7fe700 10 client.15099 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:06:39.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.291+0000 7f02df7fe700 1 --2- 172.21.15.101:0/1725522989 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5644f1ae6400 0x5644f1ae57f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=68f20ca0b26f7bc9 server_cookie=8667d425fb986fe in_seq=0 out_seq=0 2023-07-18T03:06:39.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.292+0000 7f02bf7fe700 1 -- 172.21.15.101:0/1725522989 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5644f1a1be60 con 0x5644f1ae6400 2023-07-18T03:06:39.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.292+0000 7f02bf7fe700 10 client.15099 handle_command_reply: tid=0 2023-07-18T03:06:39.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.292+0000 7f02fa0bcb80 1 client.15099 shutdown 2023-07-18T03:06:39.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.292+0000 7f02fa0bcb80 1 -- 172.21.15.101:0/1725522989 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644f1ad1480 msgr2=0x5644f1adb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:39.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.292+0000 7f02fa0bcb80 1 --2- 172.21.15.101:0/1725522989 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644f1ad1480 0x5644f1adb1c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f02d00963b0 tx=0x7f02d00cab30 comp rx=0 tx=0).stop 2023-07-18T03:06:39.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.292+0000 7f02fa0bcb80 1 -- 172.21.15.101:0/1725522989 shutdown_connections 2023-07-18T03:06:39.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.292+0000 7f02fa0bcb80 1 --2- 172.21.15.101:0/1725522989 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5644f1ae6400 0x5644f1ae57f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.292+0000 7f02fa0bcb80 1 --2- 172.21.15.101:0/1725522989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644f1ad2c20 0x5644f1ae2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.292+0000 7f02fa0bcb80 1 --2- 172.21.15.101:0/1725522989 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644f1ad1480 0x5644f1adb1c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.292+0000 7f02bf7fe700 0 client.15099 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:06:39.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.292+0000 7f02fa0bcb80 1 --2- 172.21.15.101:0/1725522989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5644f1ad1dd0 0x5644f1adb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.292+0000 7f02fa0bcb80 1 -- 172.21.15.101:0/1725522989 >> 172.21.15.101:0/1725522989 conn(0x5644f1ac6d50 msgr2=0x5644f1acb860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:39.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.292+0000 7f02fa0bcb80 1 -- 172.21.15.101:0/1725522989 shutdown_connections 2023-07-18T03:06:39.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.292+0000 7f02fa0bcb80 1 -- 172.21.15.101:0/1725522989 wait complete. 2023-07-18T03:06:39.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.292+0000 7f02fa0bcb80 20 client.15099 trim_cache size 0 max 16384 2023-07-18T03:06:39.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:38 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/643446297' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:06:39.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:38 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/4262328835' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:06:39.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.310+0000 7f02fa0bcb80 1 Processor -- start 2023-07-18T03:06:39.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.310+0000 7f02fa0bcb80 1 -- start start 2023-07-18T03:06:39.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.311+0000 7f02fa0bcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5644f1ad1480 0x5644f1ad9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.311+0000 7f02fa0bcb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644f1ad1dd0 0x5644f1ae2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.311+0000 7f02fa0bcb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644f1ad2c20 0x5644f1ad9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.311+0000 7f02fa0bcb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644f1adad90 con 0x5644f1ad1dd0 2023-07-18T03:06:39.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.311+0000 7f02fa0bcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644f197c4a0 con 0x5644f1ad1480 2023-07-18T03:06:39.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.311+0000 7f02fa0bcb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644f19f1790 con 0x5644f1ad2c20 2023-07-18T03:06:39.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.311+0000 7f02df7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644f1ad1dd0 0x5644f1ae2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.311+0000 7f02dffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5644f1ad1480 0x5644f1ad9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.311+0000 7f02df7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644f1ad1dd0 0x5644f1ae2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:43838/0 (socket says 172.21.15.101:43838) 2023-07-18T03:06:39.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.311+0000 7f02df7fe700 1 -- 172.21.15.101:0/1603361185 learned_addr learned my addr 172.21.15.101:0/1603361185 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:39.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.311+0000 7f02e4d9c700 1 --2- 172.21.15.101:0/1603361185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644f1ad2c20 0x5644f1ad9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.311+0000 7f02df7fe700 1 -- 172.21.15.101:0/1603361185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644f1ad2c20 msgr2=0x5644f1ad9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:06:39.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.311+0000 7f02df7fe700 1 --2- 172.21.15.101:0/1603361185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644f1ad2c20 0x5644f1ad9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.311+0000 7f02df7fe700 1 -- 172.21.15.101:0/1603361185 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5644f1ad1480 msgr2=0x5644f1ad9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:06:39.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.311+0000 7f02df7fe700 1 --2- 172.21.15.101:0/1603361185 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5644f1ad1480 0x5644f1ad9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.311+0000 7f02df7fe700 1 -- 172.21.15.101:0/1603361185 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5644f1877200 con 0x5644f1ad1dd0 2023-07-18T03:06:39.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.311+0000 7f02e4d9c700 1 --2- 172.21.15.101:0/1603361185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644f1ad2c20 0x5644f1ad9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:06:39.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.311+0000 7f02dffff700 1 --2- 172.21.15.101:0/1603361185 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5644f1ad1480 0x5644f1ad9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:06:39.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.312+0000 7f02df7fe700 1 --2- 172.21.15.101:0/1603361185 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644f1ad1dd0 0x5644f1ae2700 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f02d400b680 tx=0x7f02d4005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a93ed40012fa13 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:39.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.312+0000 7f02c6ffd700 1 -- 172.21.15.101:0/1603361185 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d401d040 con 0x5644f1ad1dd0 2023-07-18T03:06:39.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.312+0000 7f02c6ffd700 1 -- 172.21.15.101:0/1603361185 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02d401e040 con 0x5644f1ad1dd0 2023-07-18T03:06:39.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.312+0000 7f02c6ffd700 1 -- 172.21.15.101:0/1603361185 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d40052a0 con 0x5644f1ad1dd0 2023-07-18T03:06:39.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.312+0000 7f02fa0bcb80 1 -- 172.21.15.101:0/1603361185 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644f1ad1dd0 msgr2=0x5644f1ae2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:39.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.312+0000 7f02fa0bcb80 1 --2- 172.21.15.101:0/1603361185 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644f1ad1dd0 0x5644f1ae2700 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f02d400b680 tx=0x7f02d4005fb0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.313+0000 7f02fa0bcb80 1 -- 172.21.15.101:0/1603361185 shutdown_connections 2023-07-18T03:06:39.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.313+0000 7f02fa0bcb80 1 --2- 172.21.15.101:0/1603361185 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5644f1ad1480 0x5644f1ad9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.313+0000 7f02fa0bcb80 1 --2- 172.21.15.101:0/1603361185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644f1ad2c20 0x5644f1ad9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.313+0000 7f02fa0bcb80 1 --2- 172.21.15.101:0/1603361185 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644f1ad1dd0 0x5644f1ae2700 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.313+0000 7f02fa0bcb80 1 -- 172.21.15.101:0/1603361185 >> 172.21.15.101:0/1603361185 conn(0x5644f1ac6d50 msgr2=0x5644f1acb860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:39.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.313+0000 7f02fa0bcb80 1 -- 172.21.15.101:0/1603361185 shutdown_connections 2023-07-18T03:06:39.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.313+0000 7f02fa0bcb80 1 -- 172.21.15.101:0/1603361185 wait complete. 2023-07-18T03:06:39.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.313+0000 7f02fa0bcb80 1 -- start start 2023-07-18T03:06:39.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.313+0000 7f02fa0bcb80 1 Processor -- start 2023-07-18T03:06:39.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.314+0000 7f02fa0bcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5644f1ad1480 0x5644f1adda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.314+0000 7f02fa0bcb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644f1ad1dd0 0x5644f1addfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.314+0000 7f02fa0bcb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644f1ad2c20 0x5644f1ae0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.314+0000 7f02fa0bcb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644f1ae1520 con 0x5644f1ad1dd0 2023-07-18T03:06:39.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.314+0000 7f02fa0bcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644f1ae1690 con 0x5644f1ad1480 2023-07-18T03:06:39.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.314+0000 7f02fa0bcb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644f1ae1850 con 0x5644f1ad2c20 2023-07-18T03:06:39.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.314+0000 7f02fa0bcb80 10 client.0 _ll_register_callbacks cb 0x7f02d0099860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:06:39.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.314+0000 7f02df7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644f1ad1dd0 0x5644f1addfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.314+0000 7f02df7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644f1ad1dd0 0x5644f1addfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:43844/0 (socket says 172.21.15.101:43844) 2023-07-18T03:06:39.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.314+0000 7f02df7fe700 1 -- 172.21.15.101:0/2691448436 learned_addr learned my addr 172.21.15.101:0/2691448436 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:39.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.314+0000 7f02dffff700 1 --2- 172.21.15.101:0/2691448436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5644f1ad1480 0x5644f1adda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.314+0000 7f02e4d9c700 1 --2- 172.21.15.101:0/2691448436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644f1ad2c20 0x5644f1ae0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.314+0000 7f02dffff700 1 -- 172.21.15.101:0/2691448436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644f1ad2c20 msgr2=0x5644f1ae0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:06:39.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.314+0000 7f02dffff700 1 --2- 172.21.15.101:0/2691448436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644f1ad2c20 0x5644f1ae0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.314+0000 7f02dffff700 1 -- 172.21.15.101:0/2691448436 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644f1ad1dd0 msgr2=0x5644f1addfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:06:39.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.314+0000 7f02dffff700 1 --2- 172.21.15.101:0/2691448436 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644f1ad1dd0 0x5644f1addfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.314+0000 7f02dffff700 1 -- 172.21.15.101:0/2691448436 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5644f1839c10 con 0x5644f1ad1480 2023-07-18T03:06:39.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.315+0000 7f02dffff700 1 --2- 172.21.15.101:0/2691448436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5644f1ad1480 0x5644f1adda80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f02d009a0c0 tx=0x7f02d00984b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af8c6163146c90e0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:39.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.315+0000 7f02bf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:06:39.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.315+0000 7f02bf7fe700 1 -- 172.21.15.101:0/2691448436 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d009df30 con 0x5644f1ad1480 2023-07-18T03:06:39.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.315+0000 7f02fa0bcb80 1 -- 172.21.15.101:0/2691448436 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f02d800f040 con 0x5644f1ad1480 2023-07-18T03:06:39.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.315+0000 7f02bf7fe700 1 -- 172.21.15.101:0/2691448436 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f02d00a2350 con 0x5644f1ad1480 2023-07-18T03:06:39.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.315+0000 7f02bf7fe700 1 -- 172.21.15.101:0/2691448436 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02d0094b10 con 0x5644f1ad1480 2023-07-18T03:06:39.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.315+0000 7f02fa0bcb80 10 client.25133 fetch_fsmap learned FSMap version 24 2023-07-18T03:06:39.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.315+0000 7f02fa0bcb80 1 -- 172.21.15.101:0/2691448436 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f02e0108810 con 0x5644f1ad1480 2023-07-18T03:06:39.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.315+0000 7f02bf7fe700 1 -- 172.21.15.101:0/2691448436 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02d800f040 con 0x5644f1ad1480 2023-07-18T03:06:39.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.315+0000 7f02bf7fe700 1 -- 172.21.15.101:0/2691448436 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f02d0098e60 con 0x5644f1ad1480 2023-07-18T03:06:39.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.316+0000 7f02fa0bcb80 10 client.25133 fetch_fsmap finished waiting for FSMap version 24 2023-07-18T03:06:39.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.316+0000 7f02fa0bcb80 10 client.25133 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-18T03:06:39.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.316+0000 7f02fa0bcb80 20 client.25133 populate_metadata read hostname 'smithi101' 2023-07-18T03:06:39.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.316+0000 7f02fa0bcb80 1 --2- 172.21.15.101:0/2691448436 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5644f1b3f920 0x5644f1b3c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.316+0000 7f02fa0bcb80 4 client.25133 mds_command: new command op to 24557 tid=0[{"prefix": "status"}] 2023-07-18T03:06:39.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.316+0000 7f02fa0bcb80 1 -- 172.21.15.101:0/2691448436 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5644f19de000 con 0x5644f1b3f920 2023-07-18T03:06:39.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.316+0000 7f02df7fe700 1 --2- 172.21.15.101:0/2691448436 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5644f1b3f920 0x5644f1b3c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.316+0000 7f02df7fe700 1 --2- 172.21.15.101:0/2691448436 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5644f1b3f920 0x5644f1b3c2e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5ae369ca24e097dd server_cookie=2c471d9b30b59085 in_seq=0 out_seq=0 2023-07-18T03:06:39.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.316+0000 7f02bf7fe700 10 client.25133 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:06:39.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.317+0000 7f02bf7fe700 1 -- 172.21.15.101:0/2691448436 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5644f19de000 con 0x5644f1b3f920 2023-07-18T03:06:39.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.317+0000 7f02bf7fe700 10 client.25133 handle_command_reply: tid=0 2023-07-18T03:06:39.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.317+0000 7f02fa0bcb80 1 client.25133 shutdown 2023-07-18T03:06:39.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.317+0000 7f02fa0bcb80 1 -- 172.21.15.101:0/2691448436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5644f1ad1480 msgr2=0x5644f1adda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:39.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.317+0000 7f02fa0bcb80 1 --2- 172.21.15.101:0/2691448436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5644f1ad1480 0x5644f1adda80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f02d009a0c0 tx=0x7f02d00984b0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.317+0000 7f02fa0bcb80 1 -- 172.21.15.101:0/2691448436 shutdown_connections 2023-07-18T03:06:39.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.317+0000 7f02fa0bcb80 1 --2- 172.21.15.101:0/2691448436 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5644f1b3f920 0x5644f1b3c2e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.317+0000 7f02fa0bcb80 1 --2- 172.21.15.101:0/2691448436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5644f1ad1480 0x5644f1adda80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.317+0000 7f02fa0bcb80 1 --2- 172.21.15.101:0/2691448436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644f1ad2c20 0x5644f1ae0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.317+0000 7f02fa0bcb80 1 --2- 172.21.15.101:0/2691448436 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644f1ad1dd0 0x5644f1addfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.317+0000 7f02bf7fe700 0 client.25133 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:06:39.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.317+0000 7f02fa0bcb80 1 -- 172.21.15.101:0/2691448436 >> 172.21.15.101:0/2691448436 conn(0x5644f1ac6d50 msgr2=0x5644f1ad9120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:39.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.317+0000 7f02fa0bcb80 1 -- 172.21.15.101:0/2691448436 shutdown_connections 2023-07-18T03:06:39.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.317+0000 7f02fa0bcb80 1 -- 172.21.15.101:0/2691448436 wait complete. 2023-07-18T03:06:39.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.317+0000 7f02fa0bcb80 20 client.25133 trim_cache size 0 max 16384 2023-07-18T03:06:39.321 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:06:39.321 INFO:teuthology.orchestra.run.smithi101.stdout: "cluster_fsid": "b1782daa-2516-11ee-9b34-001a4aab830c", 2023-07-18T03:06:39.321 INFO:teuthology.orchestra.run.smithi101.stdout: "whoami": 1, 2023-07-18T03:06:39.321 INFO:teuthology.orchestra.run.smithi101.stdout: "id": 24557, 2023-07-18T03:06:39.321 INFO:teuthology.orchestra.run.smithi101.stdout: "want_state": "up:active", 2023-07-18T03:06:39.322 INFO:teuthology.orchestra.run.smithi101.stdout: "state": "up:active", 2023-07-18T03:06:39.322 INFO:teuthology.orchestra.run.smithi101.stdout: "fs_name": "cephfs", 2023-07-18T03:06:39.322 INFO:teuthology.orchestra.run.smithi101.stdout: "rank_uptime": 69.341737976999994, 2023-07-18T03:06:39.322 INFO:teuthology.orchestra.run.smithi101.stdout: "mdsmap_epoch": 24, 2023-07-18T03:06:39.322 INFO:teuthology.orchestra.run.smithi101.stdout: "osdmap_epoch": 91, 2023-07-18T03:06:39.322 INFO:teuthology.orchestra.run.smithi101.stdout: "osdmap_epoch_barrier": 89, 2023-07-18T03:06:39.322 INFO:teuthology.orchestra.run.smithi101.stdout: "uptime": 104.914352455 2023-07-18T03:06:39.323 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:06:39.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.318+0000 7f02e67ff700 1 -- 172.21.15.101:0/3448013737 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f02c80a11e0 msgr2=0x7f02c80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:39.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.318+0000 7f02e67ff700 1 --2- 172.21.15.101:0/3448013737 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f02c80a11e0 0x7f02c80a3690 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f02d8009550 tx=0x7f02d800e000 comp rx=0 tx=0).stop 2023-07-18T03:06:39.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.318+0000 7f02e67ff700 1 -- 172.21.15.101:0/3448013737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02e0100620 msgr2=0x7f02e0193430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:39.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.318+0000 7f02e67ff700 1 --2- 172.21.15.101:0/3448013737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02e0100620 0x7f02e0193430 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f02d000e9a0 tx=0x7f02d000ecb0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.318+0000 7f02e67ff700 1 -- 172.21.15.101:0/3448013737 shutdown_connections 2023-07-18T03:06:39.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.318+0000 7f02e67ff700 1 --2- 172.21.15.101:0/3448013737 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f02c80a11e0 0x7f02c80a3690 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.318+0000 7f02e67ff700 1 --2- 172.21.15.101:0/3448013737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02e0100620 0x7f02e0193430 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.318+0000 7f02e67ff700 1 --2- 172.21.15.101:0/3448013737 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f02e0105860 0x7f02e01a3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.318+0000 7f02e67ff700 1 --2- 172.21.15.101:0/3448013737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02e0102f40 0x7f02e0193970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.318+0000 7f02e67ff700 1 -- 172.21.15.101:0/3448013737 >> 172.21.15.101:0/3448013737 conn(0x7f02e00fa000 msgr2=0x7f02e00fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:39.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.318+0000 7f02e67ff700 1 -- 172.21.15.101:0/3448013737 shutdown_connections 2023-07-18T03:06:39.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.318+0000 7f02e67ff700 1 -- 172.21.15.101:0/3448013737 wait complete. 2023-07-18T03:06:39.326 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-07-18T03:06:39.524 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.538+0000 7fa79b5ff700 1 -- 172.21.15.101:0/4174765155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7940ffae0 msgr2=0x7fa7940ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:39.524 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.538+0000 7fa79b5ff700 1 --2- 172.21.15.101:0/4174765155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7940ffae0 0x7fa7940ffef0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa790009f90 tx=0x7fa79000b330 comp rx=0 tx=0).stop 2023-07-18T03:06:39.524 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.538+0000 7fa79b5ff700 1 -- 172.21.15.101:0/4174765155 shutdown_connections 2023-07-18T03:06:39.525 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.538+0000 7fa79b5ff700 1 --2- 172.21.15.101:0/4174765155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7940ffae0 0x7fa7940ffef0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.525 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.538+0000 7fa79b5ff700 1 --2- 172.21.15.101:0/4174765155 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa794101660 0x7fa794107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.525 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.538+0000 7fa79b5ff700 1 --2- 172.21.15.101:0/4174765155 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa794100cd0 0x7fa794101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.525 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.538+0000 7fa79b5ff700 1 -- 172.21.15.101:0/4174765155 >> 172.21.15.101:0/4174765155 conn(0x7fa7940fb0e0 msgr2=0x7fa7940fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:39.525 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.538+0000 7fa79b5ff700 1 -- 172.21.15.101:0/4174765155 shutdown_connections 2023-07-18T03:06:39.525 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.538+0000 7fa79b5ff700 1 -- 172.21.15.101:0/4174765155 wait complete. 2023-07-18T03:06:39.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.539+0000 7fa79b5ff700 1 Processor -- start 2023-07-18T03:06:39.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.539+0000 7fa79b5ff700 1 -- start start 2023-07-18T03:06:39.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.540+0000 7fa79b5ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7940ffae0 0x7fa794197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.540+0000 7fa79b5ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa794100cd0 0x7fa7941980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.540+0000 7fa79b5ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa794101660 0x7fa79419f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.540+0000 7fa79b5ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79419f730 con 0x7fa794101660 2023-07-18T03:06:39.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.540+0000 7fa79b5ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79419f870 con 0x7fa794100cd0 2023-07-18T03:06:39.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.540+0000 7fa79b5ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79419fa00 con 0x7fa7940ffae0 2023-07-18T03:06:39.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.540+0000 7fa79939b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7940ffae0 0x7fa794197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.540+0000 7fa799b9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa794101660 0x7fa79419f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.540+0000 7fa79939b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7940ffae0 0x7fa794197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:38348/0 (socket says 172.21.15.101:38348) 2023-07-18T03:06:39.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.540+0000 7fa79939b700 1 -- 172.21.15.101:0/1595871499 learned_addr learned my addr 172.21.15.101:0/1595871499 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:39.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.540+0000 7fa798b9a700 1 --2- 172.21.15.101:0/1595871499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa794100cd0 0x7fa7941980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.540+0000 7fa79939b700 1 -- 172.21.15.101:0/1595871499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa794100cd0 msgr2=0x7fa7941980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:39.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.540+0000 7fa79939b700 1 --2- 172.21.15.101:0/1595871499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa794100cd0 0x7fa7941980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.540+0000 7fa79939b700 1 -- 172.21.15.101:0/1595871499 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa794101660 msgr2=0x7fa79419f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:39.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.540+0000 7fa79939b700 1 --2- 172.21.15.101:0/1595871499 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa794101660 0x7fa79419f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.540+0000 7fa79939b700 1 -- 172.21.15.101:0/1595871499 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa790009c40 con 0x7fa7940ffae0 2023-07-18T03:06:39.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.541+0000 7fa79939b700 1 --2- 172.21.15.101:0/1595871499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7940ffae0 0x7fa794197bb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa790000f80 tx=0x7fa790004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:39.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.541+0000 7fa78a7fc700 1 -- 172.21.15.101:0/1595871499 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79000e040 con 0x7fa7940ffae0 2023-07-18T03:06:39.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.541+0000 7fa78a7fc700 1 -- 172.21.15.101:0/1595871499 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa79000bda0 con 0x7fa7940ffae0 2023-07-18T03:06:39.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.541+0000 7fa79b5ff700 1 -- 172.21.15.101:0/1595871499 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa79419fc80 con 0x7fa7940ffae0 2023-07-18T03:06:39.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.541+0000 7fa78a7fc700 1 -- 172.21.15.101:0/1595871499 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7900052a0 con 0x7fa7940ffae0 2023-07-18T03:06:39.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.541+0000 7fa79b5ff700 1 -- 172.21.15.101:0/1595871499 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7941a0100 con 0x7fa7940ffae0 2023-07-18T03:06:39.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.544+0000 7fa78a7fc700 1 -- 172.21.15.101:0/1595871499 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fa79001d470 con 0x7fa7940ffae0 2023-07-18T03:06:39.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.545+0000 7fa7aef1cb80 1 Processor -- start 2023-07-18T03:06:39.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.546+0000 7fa7aef1cb80 1 -- start start 2023-07-18T03:06:39.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.546+0000 7fa78a7fc700 1 --2- 172.21.15.101:0/1595871499 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa7800a11e0 0x7fa7800a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.546+0000 7fa78a7fc700 1 -- 172.21.15.101:0/1595871499 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fa7900ce400 con 0x7fa7940ffae0 2023-07-18T03:06:39.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.546+0000 7fa7aef1cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a7a3e6480 0x562a7a3e6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.546+0000 7fa799b9c700 1 --2- 172.21.15.101:0/1595871499 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa7800a11e0 0x7fa7800a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.546+0000 7fa7aef1cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a7a3e6dd0 0x562a7a3e7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.546+0000 7fa7aef1cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a7a3e7c20 0x562a7a3e8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.546+0000 7fa7aef1cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a7a2ebc60 con 0x562a7a3e7c20 2023-07-18T03:06:39.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.546+0000 7fa7aef1cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a7a33e9f0 con 0x562a7a3e6480 2023-07-18T03:06:39.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.546+0000 7fa7aef1cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a7a337e80 con 0x562a7a3e6dd0 2023-07-18T03:06:39.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.546+0000 7fa799b9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a7a3e7c20 0x562a7a3e8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.546+0000 7fa79939b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a7a3e6480 0x562a7a3e6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.546+0000 7fa79939b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a7a3e6480 0x562a7a3e6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:41212/0 (socket says 172.21.15.101:41212) 2023-07-18T03:06:39.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.546+0000 7fa79939b700 1 -- 172.21.15.101:0/1951407912 learned_addr learned my addr 172.21.15.101:0/1951407912 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:39.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.546+0000 7fa799b9c700 1 --2- 172.21.15.101:0/1951407912 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a7a3e7c20 0x562a7a3e8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:43868/0 (socket says 172.21.15.101:43868) 2023-07-18T03:06:39.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.547+0000 7fa798b9a700 1 --2- 172.21.15.101:0/1951407912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a7a3e6dd0 0x562a7a3e7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.547+0000 7fa79939b700 1 -- 172.21.15.101:0/1951407912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a7a3e6dd0 msgr2=0x562a7a3e7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:06:39.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.547+0000 7fa79939b700 1 --2- 172.21.15.101:0/1951407912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a7a3e6dd0 0x562a7a3e7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.547+0000 7fa79939b700 1 -- 172.21.15.101:0/1951407912 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a7a3e7c20 msgr2=0x562a7a3e8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:06:39.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.547+0000 7fa79939b700 1 --2- 172.21.15.101:0/1951407912 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a7a3e7c20 0x562a7a3e8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.547+0000 7fa79939b700 1 -- 172.21.15.101:0/1951407912 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a7a165d90 con 0x562a7a3e6480 2023-07-18T03:06:39.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.547+0000 7fa799b9c700 1 --2- 172.21.15.101:0/1595871499 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa7800a11e0 0x7fa7800a3690 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa78c005fd0 tx=0x7fa78c013840 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:39.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.547+0000 7fa79939b700 1 --2- 172.21.15.101:0/1951407912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a7a3e6480 0x562a7a3e6890 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa790004f50 tx=0x7fa790006020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fe2519c2730261a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:39.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.547+0000 7fa77f7fe700 1 -- 172.21.15.101:0/1951407912 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79002d2f0 con 0x562a7a3e6480 2023-07-18T03:06:39.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.547+0000 7fa77f7fe700 1 -- 172.21.15.101:0/1951407912 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa79009b370 con 0x562a7a3e6480 2023-07-18T03:06:39.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.547+0000 7fa77f7fe700 1 -- 172.21.15.101:0/1951407912 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7900cbc30 con 0x562a7a3e6480 2023-07-18T03:06:39.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.548+0000 7fa7aef1cb80 1 -- 172.21.15.101:0/1951407912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a7a3e6480 msgr2=0x562a7a3e6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:39.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.548+0000 7fa7aef1cb80 1 --2- 172.21.15.101:0/1951407912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a7a3e6480 0x562a7a3e6890 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa790004f50 tx=0x7fa790006020 comp rx=0 tx=0).stop 2023-07-18T03:06:39.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.548+0000 7fa7aef1cb80 1 -- 172.21.15.101:0/1951407912 shutdown_connections 2023-07-18T03:06:39.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.548+0000 7fa7aef1cb80 1 --2- 172.21.15.101:0/1951407912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a7a3e6480 0x562a7a3e6890 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.548+0000 7fa7aef1cb80 1 --2- 172.21.15.101:0/1951407912 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a7a3e7c20 0x562a7a3e8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.548+0000 7fa7aef1cb80 1 --2- 172.21.15.101:0/1951407912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a7a3e6dd0 0x562a7a3e7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.548+0000 7fa7aef1cb80 1 -- 172.21.15.101:0/1951407912 >> 172.21.15.101:0/1951407912 conn(0x562a7a3dbd50 msgr2=0x562a7a3e1180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:39.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.548+0000 7fa7aef1cb80 1 -- 172.21.15.101:0/1951407912 shutdown_connections 2023-07-18T03:06:39.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.548+0000 7fa7aef1cb80 1 -- 172.21.15.101:0/1951407912 wait complete. 2023-07-18T03:06:39.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.549+0000 7fa7aef1cb80 1 -- start start 2023-07-18T03:06:39.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.549+0000 7fa7aef1cb80 1 Processor -- start 2023-07-18T03:06:39.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.549+0000 7fa7aef1cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a7a3e6480 0x562a7a3eff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.549+0000 7fa7aef1cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a7a3e6dd0 0x562a7a3f0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.549+0000 7fa7aef1cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a7a3e7c20 0x562a7a3f7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.549+0000 7fa7aef1cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a7a2d7a70 con 0x562a7a3e7c20 2023-07-18T03:06:39.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.549+0000 7fa7aef1cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a7a2d7770 con 0x562a7a3e6dd0 2023-07-18T03:06:39.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.549+0000 7fa7aef1cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a7a1bb8d0 con 0x562a7a3e6480 2023-07-18T03:06:39.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.549+0000 7fa7aef1cb80 10 client.0 _ll_register_callbacks cb 0x562a7a2d5710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:06:39.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.549+0000 7fa799b9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a7a3e7c20 0x562a7a3f7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.550+0000 7fa799b9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a7a3e7c20 0x562a7a3f7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:43872/0 (socket says 172.21.15.101:43872) 2023-07-18T03:06:39.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.550+0000 7fa799b9c700 1 -- 172.21.15.101:0/3370446391 learned_addr learned my addr 172.21.15.101:0/3370446391 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:39.544 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.550+0000 7fa798b9a700 1 --2- 172.21.15.101:0/3370446391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a7a3e6dd0 0x562a7a3f0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.544 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.550+0000 7fa79939b700 1 --2- 172.21.15.101:0/3370446391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a7a3e6480 0x562a7a3eff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.544 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.550+0000 7fa799b9c700 1 -- 172.21.15.101:0/3370446391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a7a3e6480 msgr2=0x562a7a3eff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:06:39.544 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.550+0000 7fa799b9c700 1 --2- 172.21.15.101:0/3370446391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a7a3e6480 0x562a7a3eff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.550+0000 7fa799b9c700 1 -- 172.21.15.101:0/3370446391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a7a3e6dd0 msgr2=0x562a7a3f0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:06:39.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.550+0000 7fa799b9c700 1 --2- 172.21.15.101:0/3370446391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a7a3e6dd0 0x562a7a3f0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.550+0000 7fa799b9c700 1 -- 172.21.15.101:0/3370446391 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7941a0100 con 0x562a7a3e7c20 2023-07-18T03:06:39.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.550+0000 7fa798b9a700 1 --2- 172.21.15.101:0/3370446391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a7a3e6dd0 0x562a7a3f0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:06:39.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.550+0000 7fa799b9c700 1 --2- 172.21.15.101:0/3370446391 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a7a3e7c20 0x562a7a3f7480 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa78c009860 tx=0x7fa78c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cb58a16d2256767 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:39.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.550+0000 7fa77cff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:06:39.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.550+0000 7fa77cff9700 1 -- 172.21.15.101:0/3370446391 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78c018040 con 0x562a7a3e7c20 2023-07-18T03:06:39.547 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.550+0000 7fa77cff9700 1 -- 172.21.15.101:0/3370446391 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fa78c010630 con 0x562a7a3e7c20 2023-07-18T03:06:39.547 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.551+0000 7fa7aef1cb80 1 -- 172.21.15.101:0/3370446391 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562a7a2914a0 con 0x562a7a3e7c20 2023-07-18T03:06:39.547 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.551+0000 7fa77cff9700 1 -- 172.21.15.101:0/3370446391 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa78c0122e0 con 0x562a7a3e7c20 2023-07-18T03:06:39.547 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.551+0000 7fa7aef1cb80 10 client.15117 fetch_fsmap learned FSMap version 24 2023-07-18T03:06:39.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.551+0000 7fa7aef1cb80 1 -- 172.21.15.101:0/3370446391 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x562a7a14ec10 con 0x562a7a3e7c20 2023-07-18T03:06:39.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.551+0000 7fa77cff9700 1 -- 172.21.15.101:0/3370446391 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562a7a2914a0 con 0x562a7a3e7c20 2023-07-18T03:06:39.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.551+0000 7fa77cff9700 1 -- 172.21.15.101:0/3370446391 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa78c026070 con 0x562a7a3e7c20 2023-07-18T03:06:39.549 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.551+0000 7fa7aef1cb80 10 client.15117 fetch_fsmap finished waiting for FSMap version 24 2023-07-18T03:06:39.549 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.551+0000 7fa7aef1cb80 10 client.15117 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-18T03:06:39.549 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.551+0000 7fa7aef1cb80 20 client.15117 populate_metadata read hostname 'smithi101' 2023-07-18T03:06:39.550 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.551+0000 7fa7aef1cb80 1 --2- 172.21.15.101:0/3370446391 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x562a7a3fb1c0 0x562a7a3fa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.550 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.551+0000 7fa7aef1cb80 4 client.15117 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:06:39.550 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.551+0000 7fa7aef1cb80 1 -- 172.21.15.101:0/3370446391 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562a7a330e60 con 0x562a7a3fb1c0 2023-07-18T03:06:39.550 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.552+0000 7fa79939b700 1 --2- 172.21.15.101:0/3370446391 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x562a7a3fb1c0 0x562a7a3fa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.552+0000 7fa79939b700 1 --2- 172.21.15.101:0/3370446391 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x562a7a3fb1c0 0x562a7a3fa570 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a873c831f108e71 server_cookie=4340a118831f662a in_seq=0 out_seq=0 2023-07-18T03:06:39.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.552+0000 7fa77cff9700 10 client.15117 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:06:39.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.553+0000 7fa77cff9700 1 -- 172.21.15.101:0/3370446391 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562a7a330e60 con 0x562a7a3fb1c0 2023-07-18T03:06:39.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.553+0000 7fa77cff9700 10 client.15117 handle_command_reply: tid=0 2023-07-18T03:06:39.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.553+0000 7fa7aef1cb80 1 client.15117 shutdown 2023-07-18T03:06:39.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.553+0000 7fa7aef1cb80 1 -- 172.21.15.101:0/3370446391 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a7a3e7c20 msgr2=0x562a7a3f7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:39.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.553+0000 7fa7aef1cb80 1 --2- 172.21.15.101:0/3370446391 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a7a3e7c20 0x562a7a3f7480 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa78c009860 tx=0x7fa78c0047c0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.553+0000 7fa7aef1cb80 1 -- 172.21.15.101:0/3370446391 shutdown_connections 2023-07-18T03:06:39.553 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.553+0000 7fa7aef1cb80 1 --2- 172.21.15.101:0/3370446391 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x562a7a3fb1c0 0x562a7a3fa570 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.553 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.553+0000 7fa7aef1cb80 1 --2- 172.21.15.101:0/3370446391 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a7a3e7c20 0x562a7a3f7480 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.553 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.553+0000 7fa77cff9700 0 client.15117 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:06:39.553 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.553+0000 7fa7aef1cb80 1 --2- 172.21.15.101:0/3370446391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a7a3e6480 0x562a7a3eff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.553+0000 7fa7aef1cb80 1 --2- 172.21.15.101:0/3370446391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a7a3e6dd0 0x562a7a3f0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.553+0000 7fa7aef1cb80 1 -- 172.21.15.101:0/3370446391 >> 172.21.15.101:0/3370446391 conn(0x562a7a3dbd50 msgr2=0x562a7a3e0860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:39.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.553+0000 7fa7aef1cb80 1 -- 172.21.15.101:0/3370446391 shutdown_connections 2023-07-18T03:06:39.555 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.553+0000 7fa7aef1cb80 1 -- 172.21.15.101:0/3370446391 wait complete. 2023-07-18T03:06:39.555 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.554+0000 7fa7aef1cb80 20 client.15117 trim_cache size 0 max 16384 2023-07-18T03:06:39.557 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.571+0000 7fa7aef1cb80 1 Processor -- start 2023-07-18T03:06:39.557 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.571+0000 7fa7aef1cb80 1 -- start start 2023-07-18T03:06:39.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.572+0000 7fa7aef1cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a7a3e6480 0x562a7a3eebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.572+0000 7fa7aef1cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a7a3e6dd0 0x562a7a3f73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.572+0000 7fa7aef1cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a7a3e7c20 0x562a7a3ee010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.572+0000 7fa7aef1cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a7a3efad0 con 0x562a7a3e6dd0 2023-07-18T03:06:39.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.572+0000 7fa7aef1cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a7a3f10f0 con 0x562a7a3e6480 2023-07-18T03:06:39.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.572+0000 7fa7aef1cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a7a306790 con 0x562a7a3e7c20 2023-07-18T03:06:39.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.572+0000 7fa798b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a7a3e6dd0 0x562a7a3f73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.572+0000 7fa798b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a7a3e6dd0 0x562a7a3f73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:43876/0 (socket says 172.21.15.101:43876) 2023-07-18T03:06:39.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.572+0000 7fa798b9a700 1 -- 172.21.15.101:0/360346398 learned_addr learned my addr 172.21.15.101:0/360346398 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:39.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.572+0000 7fa79939b700 1 --2- 172.21.15.101:0/360346398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a7a3e6480 0x562a7a3eebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.572+0000 7fa799b9c700 1 --2- 172.21.15.101:0/360346398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a7a3e7c20 0x562a7a3ee010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.572+0000 7fa798b9a700 1 -- 172.21.15.101:0/360346398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a7a3e7c20 msgr2=0x562a7a3ee010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:06:39.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.572+0000 7fa798b9a700 1 --2- 172.21.15.101:0/360346398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a7a3e7c20 0x562a7a3ee010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.572+0000 7fa798b9a700 1 -- 172.21.15.101:0/360346398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a7a3e6480 msgr2=0x562a7a3eebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:06:39.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.572+0000 7fa798b9a700 1 --2- 172.21.15.101:0/360346398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a7a3e6480 0x562a7a3eebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.573+0000 7fa798b9a700 1 -- 172.21.15.101:0/360346398 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a7a3e82c0 con 0x562a7a3e6dd0 2023-07-18T03:06:39.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.573+0000 7fa79939b700 1 --2- 172.21.15.101:0/360346398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a7a3e6480 0x562a7a3eebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:06:39.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.573+0000 7fa798b9a700 1 --2- 172.21.15.101:0/360346398 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a7a3e6dd0 0x562a7a3f73a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa78400b940 tx=0x7fa78400bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e05e98834366d726 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:39.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.573+0000 7fa77ffff700 1 -- 172.21.15.101:0/360346398 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78401f070 con 0x562a7a3e6dd0 2023-07-18T03:06:39.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.573+0000 7fa77ffff700 1 -- 172.21.15.101:0/360346398 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa784017040 con 0x562a7a3e6dd0 2023-07-18T03:06:39.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.573+0000 7fa77ffff700 1 -- 172.21.15.101:0/360346398 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa784016070 con 0x562a7a3e6dd0 2023-07-18T03:06:39.564 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.574+0000 7fa7aef1cb80 1 -- 172.21.15.101:0/360346398 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a7a3e6dd0 msgr2=0x562a7a3f73a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:39.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.574+0000 7fa7aef1cb80 1 --2- 172.21.15.101:0/360346398 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a7a3e6dd0 0x562a7a3f73a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa78400b940 tx=0x7fa78400bc50 comp rx=0 tx=0).stop 2023-07-18T03:06:39.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.574+0000 7fa7aef1cb80 1 -- 172.21.15.101:0/360346398 shutdown_connections 2023-07-18T03:06:39.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.574+0000 7fa7aef1cb80 1 --2- 172.21.15.101:0/360346398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a7a3e6480 0x562a7a3eebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.574+0000 7fa7aef1cb80 1 --2- 172.21.15.101:0/360346398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a7a3e7c20 0x562a7a3ee010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.574+0000 7fa7aef1cb80 1 --2- 172.21.15.101:0/360346398 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a7a3e6dd0 0x562a7a3f73a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.574+0000 7fa7aef1cb80 1 -- 172.21.15.101:0/360346398 >> 172.21.15.101:0/360346398 conn(0x562a7a3dbd50 msgr2=0x562a7a3f7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:39.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.574+0000 7fa7aef1cb80 1 -- 172.21.15.101:0/360346398 shutdown_connections 2023-07-18T03:06:39.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.574+0000 7fa7aef1cb80 1 -- 172.21.15.101:0/360346398 wait complete. 2023-07-18T03:06:39.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.574+0000 7fa7aef1cb80 1 -- start start 2023-07-18T03:06:39.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.574+0000 7fa7aef1cb80 1 Processor -- start 2023-07-18T03:06:39.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.575+0000 7fa7aef1cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a7a3e6480 0x562a7a3ed200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.575+0000 7fa7aef1cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a7a3e6dd0 0x562a7a3f73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.575+0000 7fa7aef1cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a7a3e7c20 0x562a7a44f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.575+0000 7fa7aef1cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a7a3ed910 con 0x562a7a3e6480 2023-07-18T03:06:39.567 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.575+0000 7fa7aef1cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a7a44f580 con 0x562a7a3e6dd0 2023-07-18T03:06:39.567 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.575+0000 7fa7aef1cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a7a44f6c0 con 0x562a7a3e7c20 2023-07-18T03:06:39.567 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.575+0000 7fa7aef1cb80 10 client.0 _ll_register_callbacks cb 0x7fa78c004350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:06:39.567 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.575+0000 7fa79939b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a7a3e6480 0x562a7a3ed200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.567 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.575+0000 7fa79939b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a7a3e6480 0x562a7a3ed200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:43890/0 (socket says 172.21.15.101:43890) 2023-07-18T03:06:39.567 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.575+0000 7fa79939b700 1 -- 172.21.15.101:0/3401904379 learned_addr learned my addr 172.21.15.101:0/3401904379 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:39.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.575+0000 7fa798b9a700 1 --2- 172.21.15.101:0/3401904379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a7a3e6dd0 0x562a7a3f73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.575+0000 7fa799b9c700 1 --2- 172.21.15.101:0/3401904379 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a7a3e7c20 0x562a7a44f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.575+0000 7fa79939b700 1 -- 172.21.15.101:0/3401904379 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a7a3e7c20 msgr2=0x562a7a44f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:06:39.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.575+0000 7fa79939b700 1 --2- 172.21.15.101:0/3401904379 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a7a3e7c20 0x562a7a44f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.575+0000 7fa79939b700 1 -- 172.21.15.101:0/3401904379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a7a3e6dd0 msgr2=0x562a7a3f73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:06:39.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.575+0000 7fa79939b700 1 --2- 172.21.15.101:0/3401904379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a7a3e6dd0 0x562a7a3f73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.575+0000 7fa79939b700 1 -- 172.21.15.101:0/3401904379 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562a7a165d90 con 0x562a7a3e6480 2023-07-18T03:06:39.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.575+0000 7fa799b9c700 1 --2- 172.21.15.101:0/3401904379 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a7a3e7c20 0x562a7a44f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:06:39.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.576+0000 7fa79939b700 1 --2- 172.21.15.101:0/3401904379 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a7a3e6480 0x562a7a3ed200 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa790007930 tx=0x7fa79000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fef17b4343fb117 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:39.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.576+0000 7fa77cff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:06:39.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.576+0000 7fa77cff9700 1 -- 172.21.15.101:0/3401904379 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79002d2f0 con 0x562a7a3e6480 2023-07-18T03:06:39.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.576+0000 7fa77cff9700 1 -- 172.21.15.101:0/3401904379 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fa79009f3a0 con 0x562a7a3e6480 2023-07-18T03:06:39.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.576+0000 7fa7aef1cb80 1 -- 172.21.15.101:0/3401904379 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa79009b370 con 0x562a7a3e6480 2023-07-18T03:06:39.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.576+0000 7fa77cff9700 1 -- 172.21.15.101:0/3401904379 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7900098a0 con 0x562a7a3e6480 2023-07-18T03:06:39.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.576+0000 7fa7aef1cb80 10 client.15123 fetch_fsmap learned FSMap version 24 2023-07-18T03:06:39.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.576+0000 7fa7aef1cb80 1 -- 172.21.15.101:0/3401904379 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7fa784004420 con 0x562a7a3e6480 2023-07-18T03:06:39.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.576+0000 7fa77cff9700 1 -- 172.21.15.101:0/3401904379 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa79009b370 con 0x562a7a3e6480 2023-07-18T03:06:39.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.576+0000 7fa77cff9700 1 -- 172.21.15.101:0/3401904379 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa79009b5d0 con 0x562a7a3e6480 2023-07-18T03:06:39.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.576+0000 7fa7aef1cb80 10 client.15123 fetch_fsmap finished waiting for FSMap version 24 2023-07-18T03:06:39.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.576+0000 7fa7aef1cb80 10 client.15123 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-18T03:06:39.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.576+0000 7fa7aef1cb80 20 client.15123 populate_metadata read hostname 'smithi101' 2023-07-18T03:06:39.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.576+0000 7fa7aef1cb80 1 --2- 172.21.15.101:0/3401904379 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x562a7a454090 0x562a7a4509e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:39.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.576+0000 7fa7aef1cb80 4 client.15123 mds_command: new command op to 24562 tid=0[{"prefix": "status"}] 2023-07-18T03:06:39.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.576+0000 7fa7aef1cb80 1 -- 172.21.15.101:0/3401904379 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562a7a2f3000 con 0x562a7a454090 2023-07-18T03:06:39.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.577+0000 7fa798b9a700 1 --2- 172.21.15.101:0/3401904379 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x562a7a454090 0x562a7a4509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:39.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.577+0000 7fa798b9a700 1 --2- 172.21.15.101:0/3401904379 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x562a7a454090 0x562a7a4509e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16a3ad2bccedeb98 server_cookie=7ac8d77e4455cc42 in_seq=0 out_seq=0 2023-07-18T03:06:39.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.577+0000 7fa77cff9700 10 client.15123 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:06:39.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.577+0000 7fa77cff9700 1 -- 172.21.15.101:0/3401904379 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x562a7a2f3000 con 0x562a7a454090 2023-07-18T03:06:39.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.577+0000 7fa77cff9700 10 client.15123 handle_command_reply: tid=0 2023-07-18T03:06:39.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.577+0000 7fa7aef1cb80 1 client.15123 shutdown 2023-07-18T03:06:39.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.578+0000 7fa7aef1cb80 1 -- 172.21.15.101:0/3401904379 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a7a3e6480 msgr2=0x562a7a3ed200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:39.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.578+0000 7fa7aef1cb80 1 --2- 172.21.15.101:0/3401904379 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a7a3e6480 0x562a7a3ed200 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa790007930 tx=0x7fa79000bf10 comp rx=0 tx=0).stop 2023-07-18T03:06:39.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.578+0000 7fa7aef1cb80 1 -- 172.21.15.101:0/3401904379 shutdown_connections 2023-07-18T03:06:39.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.578+0000 7fa7aef1cb80 1 --2- 172.21.15.101:0/3401904379 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x562a7a454090 0x562a7a4509e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.578+0000 7fa7aef1cb80 1 --2- 172.21.15.101:0/3401904379 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a7a3e7c20 0x562a7a44f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.578+0000 7fa7aef1cb80 1 --2- 172.21.15.101:0/3401904379 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a7a3e6480 0x562a7a3ed200 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.578+0000 7fa77cff9700 0 client.15123 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:06:39.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.578+0000 7fa7aef1cb80 1 --2- 172.21.15.101:0/3401904379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a7a3e6dd0 0x562a7a3f73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.578+0000 7fa7aef1cb80 1 -- 172.21.15.101:0/3401904379 >> 172.21.15.101:0/3401904379 conn(0x562a7a3dbd50 msgr2=0x562a7a3f7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:39.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.578+0000 7fa7aef1cb80 1 -- 172.21.15.101:0/3401904379 shutdown_connections 2023-07-18T03:06:39.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.578+0000 7fa7aef1cb80 1 -- 172.21.15.101:0/3401904379 wait complete. 2023-07-18T03:06:39.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.578+0000 7fa7aef1cb80 20 client.15123 trim_cache size 0 max 16384 2023-07-18T03:06:39.574 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:06:39.574 INFO:teuthology.orchestra.run.smithi101.stdout: "cluster_fsid": "b1782daa-2516-11ee-9b34-001a4aab830c", 2023-07-18T03:06:39.574 INFO:teuthology.orchestra.run.smithi101.stdout: "whoami": 0, 2023-07-18T03:06:39.574 INFO:teuthology.orchestra.run.smithi101.stdout: "id": 24562, 2023-07-18T03:06:39.574 INFO:teuthology.orchestra.run.smithi101.stdout: "want_state": "up:active", 2023-07-18T03:06:39.574 INFO:teuthology.orchestra.run.smithi101.stdout: "state": "up:active", 2023-07-18T03:06:39.574 INFO:teuthology.orchestra.run.smithi101.stdout: "fs_name": "cephfs", 2023-07-18T03:06:39.575 INFO:teuthology.orchestra.run.smithi101.stdout: "rank_uptime": 99.745498882999996, 2023-07-18T03:06:39.575 INFO:teuthology.orchestra.run.smithi101.stdout: "mdsmap_epoch": 24, 2023-07-18T03:06:39.575 INFO:teuthology.orchestra.run.smithi101.stdout: "osdmap_epoch": 91, 2023-07-18T03:06:39.575 INFO:teuthology.orchestra.run.smithi101.stdout: "osdmap_epoch_barrier": 89, 2023-07-18T03:06:39.575 INFO:teuthology.orchestra.run.smithi101.stdout: "uptime": 114.33249458 2023-07-18T03:06:39.575 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:06:39.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.578+0000 7fa79b5ff700 1 -- 172.21.15.101:0/1595871499 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa7800a11e0 msgr2=0x7fa7800a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:39.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.578+0000 7fa79b5ff700 1 --2- 172.21.15.101:0/1595871499 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa7800a11e0 0x7fa7800a3690 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa78c005fd0 tx=0x7fa78c013840 comp rx=0 tx=0).stop 2023-07-18T03:06:39.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.578+0000 7fa79b5ff700 1 -- 172.21.15.101:0/1595871499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7940ffae0 msgr2=0x7fa794197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:39.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.578+0000 7fa79b5ff700 1 --2- 172.21.15.101:0/1595871499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7940ffae0 0x7fa794197bb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa790000f80 tx=0x7fa790004d00 comp rx=0 tx=0).stop 2023-07-18T03:06:39.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.579+0000 7fa79b5ff700 1 -- 172.21.15.101:0/1595871499 shutdown_connections 2023-07-18T03:06:39.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.579+0000 7fa79b5ff700 1 --2- 172.21.15.101:0/1595871499 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa7800a11e0 0x7fa7800a3690 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.579+0000 7fa79b5ff700 1 --2- 172.21.15.101:0/1595871499 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa794101660 0x7fa79419f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.579+0000 7fa79b5ff700 1 --2- 172.21.15.101:0/1595871499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7940ffae0 0x7fa794197bb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.579+0000 7fa79b5ff700 1 --2- 172.21.15.101:0/1595871499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa794100cd0 0x7fa7941980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:39.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.579+0000 7fa79b5ff700 1 -- 172.21.15.101:0/1595871499 >> 172.21.15.101:0/1595871499 conn(0x7fa7940fb0e0 msgr2=0x7fa794106610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:39.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.579+0000 7fa79b5ff700 1 -- 172.21.15.101:0/1595871499 shutdown_connections 2023-07-18T03:06:39.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:39.579+0000 7fa79b5ff700 1 -- 172.21.15.101:0/1595871499 wait complete. 2023-07-18T03:06:39.582 INFO:tasks.cephadm:Setup complete, yielding 2023-07-18T03:06:39.583 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-07-18T03:06:39.598 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi101.front.sepia.ceph.com 2023-07-18T03:06:39.599 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- bash -c 'ceph orch status' 2023-07-18T03:06:40.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:39 smithi179 ceph-mon[129229]: pgmap v329: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 1 B/s, 0 objects/s recovering 2023-07-18T03:06:40.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:39 smithi101 ceph-mon[126379]: pgmap v329: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 1 B/s, 0 objects/s recovering 2023-07-18T03:06:40.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:39 smithi155 ceph-mon[128885]: pgmap v329: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 1 B/s, 0 objects/s recovering 2023-07-18T03:06:40.316 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:06:42.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.027+0000 7fe5bc23e700 1 -- 172.21.15.101:0/1264892538 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe5b40ffb90 msgr2=0x7fe5b40fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:42.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.027+0000 7fe5bc23e700 1 --2- 172.21.15.101:0/1264892538 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe5b40ffb90 0x7fe5b40fffa0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0009990 tx=0x7fe5b0009ca0 comp rx=0 tx=0).stop 2023-07-18T03:06:42.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.027+0000 7fe5bc23e700 1 -- 172.21.15.101:0/1264892538 shutdown_connections 2023-07-18T03:06:42.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.027+0000 7fe5bc23e700 1 --2- 172.21.15.101:0/1264892538 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5b4101720 0x7fe5b4107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:42.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.027+0000 7fe5bc23e700 1 --2- 172.21.15.101:0/1264892538 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe5b40ffb90 0x7fe5b40fffa0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:42.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.027+0000 7fe5bc23e700 1 --2- 172.21.15.101:0/1264892538 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5b4100d90 0x7fe5b41011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:42.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.027+0000 7fe5bc23e700 1 -- 172.21.15.101:0/1264892538 >> 172.21.15.101:0/1264892538 conn(0x7fe5b40fb1a0 msgr2=0x7fe5b40fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:42.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.027+0000 7fe5bc23e700 1 -- 172.21.15.101:0/1264892538 shutdown_connections 2023-07-18T03:06:42.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.027+0000 7fe5bc23e700 1 -- 172.21.15.101:0/1264892538 wait complete. 2023-07-18T03:06:42.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.028+0000 7fe5bc23e700 1 Processor -- start 2023-07-18T03:06:42.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.028+0000 7fe5bc23e700 1 -- start start 2023-07-18T03:06:42.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.028+0000 7fe5bc23e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5b40ffb90 0x7fe5b41958b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:42.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.028+0000 7fe5bc23e700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe5b4100d90 0x7fe5b4195df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:42.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.028+0000 7fe5bc23e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5b4101720 0x7fe5b419ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:42.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.028+0000 7fe5bc23e700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b419d440 con 0x7fe5b4100d90 2023-07-18T03:06:42.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.028+0000 7fe5bc23e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b419d580 con 0x7fe5b4101720 2023-07-18T03:06:42.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.028+0000 7fe5bc23e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b419d710 con 0x7fe5b40ffb90 2023-07-18T03:06:42.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.029+0000 7fe5b97d9700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe5b4100d90 0x7fe5b4195df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:42.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.029+0000 7fe5b97d9700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe5b4100d90 0x7fe5b4195df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:43912/0 (socket says 172.21.15.101:43912) 2023-07-18T03:06:42.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.029+0000 7fe5b97d9700 1 -- 172.21.15.101:0/298733873 learned_addr learned my addr 172.21.15.101:0/298733873 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:42.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.029+0000 7fe5b9fda700 1 --2- 172.21.15.101:0/298733873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5b40ffb90 0x7fe5b41958b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:42.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.029+0000 7fe5ba7db700 1 --2- 172.21.15.101:0/298733873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5b4101720 0x7fe5b419ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:42.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.029+0000 7fe5b9fda700 1 -- 172.21.15.101:0/298733873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5b4101720 msgr2=0x7fe5b419ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:42.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.029+0000 7fe5b9fda700 1 --2- 172.21.15.101:0/298733873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5b4101720 0x7fe5b419ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:42.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.029+0000 7fe5b9fda700 1 -- 172.21.15.101:0/298733873 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe5b4100d90 msgr2=0x7fe5b4195df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:42.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.029+0000 7fe5b9fda700 1 --2- 172.21.15.101:0/298733873 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe5b4100d90 0x7fe5b4195df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:42.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.029+0000 7fe5b9fda700 1 -- 172.21.15.101:0/298733873 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a4009520 con 0x7fe5b40ffb90 2023-07-18T03:06:42.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.029+0000 7fe5b97d9700 1 --2- 172.21.15.101:0/298733873 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe5b4100d90 0x7fe5b4195df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:06:42.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.029+0000 7fe5b9fda700 1 --2- 172.21.15.101:0/298733873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5b40ffb90 0x7fe5b41958b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe5b00098e0 tx=0x7fe5b0005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:42.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.030+0000 7fe5aaffd700 1 -- 172.21.15.101:0/298733873 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b001d070 con 0x7fe5b40ffb90 2023-07-18T03:06:42.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.030+0000 7fe5bc23e700 1 -- 172.21.15.101:0/298733873 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5b00095f0 con 0x7fe5b40ffb90 2023-07-18T03:06:42.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.030+0000 7fe5aaffd700 1 -- 172.21.15.101:0/298733873 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5b0022470 con 0x7fe5b40ffb90 2023-07-18T03:06:42.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.030+0000 7fe5aaffd700 1 -- 172.21.15.101:0/298733873 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0017650 con 0x7fe5b40ffb90 2023-07-18T03:06:42.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.030+0000 7fe5bc23e700 1 -- 172.21.15.101:0/298733873 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5b419dc70 con 0x7fe5b40ffb90 2023-07-18T03:06:42.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.031+0000 7fe5bc23e700 1 -- 172.21.15.101:0/298733873 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5b4107170 con 0x7fe5b40ffb90 2023-07-18T03:06:42.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.033+0000 7fe5aaffd700 1 -- 172.21.15.101:0/298733873 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fe5b0022870 con 0x7fe5b40ffb90 2023-07-18T03:06:42.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.034+0000 7fe5aaffd700 1 --2- 172.21.15.101:0/298733873 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe5a00a11e0 0x7fe5a00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:42.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.034+0000 7fe5aaffd700 1 -- 172.21.15.101:0/298733873 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fe5b00ce150 con 0x7fe5b40ffb90 2023-07-18T03:06:42.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.034+0000 7fe5b97d9700 1 --2- 172.21.15.101:0/298733873 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe5a00a11e0 0x7fe5a00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:42.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.035+0000 7fe5b97d9700 1 --2- 172.21.15.101:0/298733873 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe5a00a11e0 0x7fe5a00a3690 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4009fd0 tx=0x7fe5a4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:42.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.038+0000 7fe5aaffd700 1 -- 172.21.15.101:0/298733873 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe5b0097400 con 0x7fe5b40ffb90 2023-07-18T03:06:42.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:41 smithi179 ceph-mon[129229]: pgmap v330: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:06:42.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:41 smithi101 ceph-mon[126379]: pgmap v330: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:06:42.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.249+0000 7fe5bc23e700 1 -- 172.21.15.101:0/298733873 --> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe5b4060950 con 0x7fe5a00a11e0 2023-07-18T03:06:42.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.250+0000 7fe5aaffd700 1 -- 172.21.15.101:0/298733873 <== mgr.14150 v2:172.21.15.101:6800/3963742842 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fe5b4060950 con 0x7fe5a00a11e0 2023-07-18T03:06:42.236 INFO:teuthology.orchestra.run.smithi101.stdout:Backend: cephadm 2023-07-18T03:06:42.237 INFO:teuthology.orchestra.run.smithi101.stdout:Available: Yes 2023-07-18T03:06:42.237 INFO:teuthology.orchestra.run.smithi101.stdout:Paused: No 2023-07-18T03:06:42.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.253+0000 7fe5bc23e700 1 -- 172.21.15.101:0/298733873 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe5a00a11e0 msgr2=0x7fe5a00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:42.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.253+0000 7fe5bc23e700 1 --2- 172.21.15.101:0/298733873 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe5a00a11e0 0x7fe5a00a3690 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4009fd0 tx=0x7fe5a4009260 comp rx=0 tx=0).stop 2023-07-18T03:06:42.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.253+0000 7fe5bc23e700 1 -- 172.21.15.101:0/298733873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5b40ffb90 msgr2=0x7fe5b41958b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:42.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.253+0000 7fe5bc23e700 1 --2- 172.21.15.101:0/298733873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5b40ffb90 0x7fe5b41958b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe5b00098e0 tx=0x7fe5b0005720 comp rx=0 tx=0).stop 2023-07-18T03:06:42.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.253+0000 7fe5bc23e700 1 -- 172.21.15.101:0/298733873 shutdown_connections 2023-07-18T03:06:42.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.253+0000 7fe5bc23e700 1 --2- 172.21.15.101:0/298733873 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe5a00a11e0 0x7fe5a00a3690 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:42.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.253+0000 7fe5bc23e700 1 --2- 172.21.15.101:0/298733873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5b4101720 0x7fe5b419ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:42.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.253+0000 7fe5bc23e700 1 --2- 172.21.15.101:0/298733873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5b40ffb90 0x7fe5b41958b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:42.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.253+0000 7fe5bc23e700 1 --2- 172.21.15.101:0/298733873 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe5b4100d90 0x7fe5b4195df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:42.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.253+0000 7fe5bc23e700 1 -- 172.21.15.101:0/298733873 >> 172.21.15.101:0/298733873 conn(0x7fe5b40fb1a0 msgr2=0x7fe5b41042b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:42.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.253+0000 7fe5bc23e700 1 -- 172.21.15.101:0/298733873 shutdown_connections 2023-07-18T03:06:42.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:42.253+0000 7fe5bc23e700 1 -- 172.21.15.101:0/298733873 wait complete. 2023-07-18T03:06:42.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:41 smithi155 ceph-mon[128885]: pgmap v330: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:06:42.705 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- bash -c 'ceph orch ps' 2023-07-18T03:06:42.981 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:42 smithi101 ceph-mon[126379]: from='client.25186 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:06:43.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:42 smithi155 ceph-mon[128885]: from='client.25186 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:06:43.430 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:06:43.443 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:42 smithi179 ceph-mon[129229]: from='client.25186 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:06:43.981 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:43 smithi101 ceph-mon[126379]: pgmap v331: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-18T03:06:44.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:43 smithi155 ceph-mon[128885]: pgmap v331: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-18T03:06:44.443 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:43 smithi179 ceph-mon[129229]: pgmap v331: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-18T03:06:45.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.158+0000 7f103a934700 1 -- 172.21.15.101:0/3870931950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1034101720 msgr2=0x7f1034107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:45.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.158+0000 7f103a934700 1 --2- 172.21.15.101:0/3870931950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1034101720 0x7f1034107fe0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f102c009f90 tx=0x7f102c00b330 comp rx=0 tx=0).stop 2023-07-18T03:06:45.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.158+0000 7f103a934700 1 -- 172.21.15.101:0/3870931950 shutdown_connections 2023-07-18T03:06:45.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.158+0000 7f103a934700 1 --2- 172.21.15.101:0/3870931950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10340ffb90 0x7f10340fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:45.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.158+0000 7f103a934700 1 --2- 172.21.15.101:0/3870931950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1034101720 0x7f1034107fe0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:45.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.158+0000 7f103a934700 1 --2- 172.21.15.101:0/3870931950 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1034100d90 0x7f10341011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:45.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.158+0000 7f103a934700 1 -- 172.21.15.101:0/3870931950 >> 172.21.15.101:0/3870931950 conn(0x7f10340fb1a0 msgr2=0x7f10340fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:45.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.159+0000 7f103a934700 1 -- 172.21.15.101:0/3870931950 shutdown_connections 2023-07-18T03:06:45.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.159+0000 7f103a934700 1 -- 172.21.15.101:0/3870931950 wait complete. 2023-07-18T03:06:45.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.160+0000 7f103a934700 1 Processor -- start 2023-07-18T03:06:45.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.160+0000 7f103a934700 1 -- start start 2023-07-18T03:06:45.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.160+0000 7f103a934700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10340ffb90 0x7f10341959b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:45.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.160+0000 7f103a934700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1034100d90 0x7f1034195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:45.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.160+0000 7f103a934700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1034101720 0x7f103419cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:45.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.160+0000 7f103a934700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103419d540 con 0x7f1034100d90 2023-07-18T03:06:45.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.160+0000 7f103a934700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103419d680 con 0x7f1034101720 2023-07-18T03:06:45.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.160+0000 7f103a934700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103419d810 con 0x7f10340ffb90 2023-07-18T03:06:45.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.160+0000 7f10337fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1034100d90 0x7f1034195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:45.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.160+0000 7f10337fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1034100d90 0x7f1034195ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:43930/0 (socket says 172.21.15.101:43930) 2023-07-18T03:06:45.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.160+0000 7f10337fe700 1 -- 172.21.15.101:0/397527350 learned_addr learned my addr 172.21.15.101:0/397527350 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:45.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.161+0000 7f1033fff700 1 --2- 172.21.15.101:0/397527350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10340ffb90 0x7f10341959b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:45.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.161+0000 7f1038ed1700 1 --2- 172.21.15.101:0/397527350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1034101720 0x7f103419cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:45.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.161+0000 7f10337fe700 1 -- 172.21.15.101:0/397527350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10340ffb90 msgr2=0x7f10341959b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:45.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.161+0000 7f10337fe700 1 --2- 172.21.15.101:0/397527350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10340ffb90 0x7f10341959b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:45.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.161+0000 7f10337fe700 1 -- 172.21.15.101:0/397527350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1034101720 msgr2=0x7f103419cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:45.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.161+0000 7f10337fe700 1 --2- 172.21.15.101:0/397527350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1034101720 0x7f103419cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:45.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.161+0000 7f10337fe700 1 -- 172.21.15.101:0/397527350 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1028009520 con 0x7f1034100d90 2023-07-18T03:06:45.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.161+0000 7f10337fe700 1 --2- 172.21.15.101:0/397527350 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1034100d90 0x7f1034195ef0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f102800eb60 tx=0x7f102800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:45.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.161+0000 7f10317fa700 1 -- 172.21.15.101:0/397527350 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102800cc80 con 0x7f1034100d90 2023-07-18T03:06:45.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.161+0000 7f103a934700 1 -- 172.21.15.101:0/397527350 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f102c009c40 con 0x7f1034100d90 2023-07-18T03:06:45.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.161+0000 7f10317fa700 1 -- 172.21.15.101:0/397527350 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10280044e0 con 0x7f1034100d90 2023-07-18T03:06:45.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.161+0000 7f10317fa700 1 -- 172.21.15.101:0/397527350 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1028010430 con 0x7f1034100d90 2023-07-18T03:06:45.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.162+0000 7f103a934700 1 -- 172.21.15.101:0/397527350 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f103419deb0 con 0x7f1034100d90 2023-07-18T03:06:45.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.163+0000 7f103a934700 1 -- 172.21.15.101:0/397527350 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f103410b360 con 0x7f1034100d90 2023-07-18T03:06:45.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.164+0000 7f10317fa700 1 -- 172.21.15.101:0/397527350 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f1028003f40 con 0x7f1034100d90 2023-07-18T03:06:45.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.166+0000 7f10317fa700 1 --2- 172.21.15.101:0/397527350 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f101c0a11e0 0x7f101c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:45.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.166+0000 7f10317fa700 1 -- 172.21.15.101:0/397527350 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f1028014070 con 0x7f1034100d90 2023-07-18T03:06:45.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.166+0000 7f1033fff700 1 --2- 172.21.15.101:0/397527350 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f101c0a11e0 0x7f101c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:45.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.167+0000 7f1033fff700 1 --2- 172.21.15.101:0/397527350 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f101c0a11e0 0x7f101c0a3690 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1034100bf0 tx=0x7f1024009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:45.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.170+0000 7f10317fa700 1 -- 172.21.15.101:0/397527350 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f10280964b0 con 0x7f1034100d90 2023-07-18T03:06:45.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.373+0000 7f103a934700 1 -- 172.21.15.101:0/397527350 --> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1034060950 con 0x7f101c0a11e0 2023-07-18T03:06:45.372 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.386+0000 7f10317fa700 1 -- 172.21.15.101:0/397527350 <== mgr.14150 v2:172.21.15.101:6800/3963742842 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f1034060950 con 0x7f101c0a11e0 2023-07-18T03:06:45.373 INFO:teuthology.orchestra.run.smithi101.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-07-18T03:06:45.373 INFO:teuthology.orchestra.run.smithi101.stdout:mds.a smithi101 running (113s) 24s ago 113s 13.7M - 18.0.0-4746-g81cf785a 66a4f7b015b5 fc15a5260222 2023-07-18T03:06:45.373 INFO:teuthology.orchestra.run.smithi101.stdout:mds.b smithi155 running (111s) 39s ago 110s 19.9M - 18.0.0-4746-g81cf785a 66a4f7b015b5 d55ec25a2076 2023-07-18T03:06:45.374 INFO:teuthology.orchestra.run.smithi101.stdout:mds.c smithi179 running (2m) 39s ago 2m 18.2M - 18.0.0-4746-g81cf785a 66a4f7b015b5 aae380257197 2023-07-18T03:06:45.374 INFO:teuthology.orchestra.run.smithi101.stdout:mds.d smithi101 running (2m) 24s ago 2m 13.7M - 18.0.0-4746-g81cf785a 66a4f7b015b5 359fbbe10bc2 2023-07-18T03:06:45.374 INFO:teuthology.orchestra.run.smithi101.stdout:mds.e smithi155 running (2m) 39s ago 2m 17.0M - 18.0.0-4746-g81cf785a 66a4f7b015b5 84703f70245d 2023-07-18T03:06:45.374 INFO:teuthology.orchestra.run.smithi101.stdout:mds.f smithi179 running (2m) 39s ago 2m 18.4M - 18.0.0-4746-g81cf785a 66a4f7b015b5 88de317e4c8f 2023-07-18T03:06:45.375 INFO:teuthology.orchestra.run.smithi101.stdout:mds.g smithi101 running (116s) 24s ago 116s 13.7M - 18.0.0-4746-g81cf785a 66a4f7b015b5 a23f0b4bd338 2023-07-18T03:06:45.375 INFO:teuthology.orchestra.run.smithi101.stdout:mds.h smithi155 running (2m) 39s ago 2m 18.6M - 18.0.0-4746-g81cf785a 66a4f7b015b5 9bd7b1b79232 2023-07-18T03:06:45.375 INFO:teuthology.orchestra.run.smithi101.stdout:mds.i smithi179 running (2m) 39s ago 2m 30.1M - 18.0.0-4746-g81cf785a 66a4f7b015b5 f6c5b1caf4b5 2023-07-18T03:06:45.376 INFO:teuthology.orchestra.run.smithi101.stdout:mds.j smithi101 running (2m) 24s ago 2m 13.8M - 18.0.0-4746-g81cf785a 66a4f7b015b5 604af1288867 2023-07-18T03:06:45.376 INFO:teuthology.orchestra.run.smithi101.stdout:mds.k smithi155 running (2m) 39s ago 2m 15.2M - 18.0.0-4746-g81cf785a 66a4f7b015b5 7e4ecb464278 2023-07-18T03:06:45.376 INFO:teuthology.orchestra.run.smithi101.stdout:mds.l smithi179 running (2m) 39s ago 2m 17.5M - 18.0.0-4746-g81cf785a 66a4f7b015b5 f4c50de6ca54 2023-07-18T03:06:45.376 INFO:teuthology.orchestra.run.smithi101.stdout:mgr.x smithi101 *:9283,8765 running (9m) 24s ago 9m 522M - 18.0.0-4746-g81cf785a 66a4f7b015b5 c8d9e9e2ba2d 2023-07-18T03:06:45.377 INFO:teuthology.orchestra.run.smithi101.stdout:mgr.y smithi155 *:8443,8765 running (7m) 39s ago 7m 446M - 18.0.0-4746-g81cf785a 66a4f7b015b5 41afb61dbbcd 2023-07-18T03:06:45.377 INFO:teuthology.orchestra.run.smithi101.stdout:mgr.z smithi179 *:8443,8765 running (7m) 39s ago 7m 446M - 18.0.0-4746-g81cf785a 66a4f7b015b5 d36ff5685165 2023-07-18T03:06:45.377 INFO:teuthology.orchestra.run.smithi101.stdout:mon.a smithi101 running (9m) 24s ago 9m 95.0M 2048M 18.0.0-4746-g81cf785a 66a4f7b015b5 19e14fc4c800 2023-07-18T03:06:45.378 INFO:teuthology.orchestra.run.smithi101.stdout:mon.b smithi155 running (7m) 39s ago 7m 65.4M 2048M 18.0.0-4746-g81cf785a 66a4f7b015b5 797e34671e86 2023-07-18T03:06:45.378 INFO:teuthology.orchestra.run.smithi101.stdout:mon.c smithi179 running (7m) 39s ago 7m 63.6M 2048M 18.0.0-4746-g81cf785a 66a4f7b015b5 14d3b3560bbb 2023-07-18T03:06:45.378 INFO:teuthology.orchestra.run.smithi101.stdout:osd.0 smithi101 running (6m) 24s ago 6m 72.4M 4316M 18.0.0-4746-g81cf785a 66a4f7b015b5 333cbcc98249 2023-07-18T03:06:45.378 INFO:teuthology.orchestra.run.smithi101.stdout:osd.1 smithi155 running (6m) 39s ago 6m 78.8M 4292M 18.0.0-4746-g81cf785a 66a4f7b015b5 9fdb6c9fd8eb 2023-07-18T03:06:45.379 INFO:teuthology.orchestra.run.smithi101.stdout:osd.2 smithi179 running (6m) 39s ago 6m 74.8M 4292M 18.0.0-4746-g81cf785a 66a4f7b015b5 e7db7f2d763e 2023-07-18T03:06:45.379 INFO:teuthology.orchestra.run.smithi101.stdout:osd.3 smithi101 running (5m) 24s ago 5m 66.4M 4316M 18.0.0-4746-g81cf785a 66a4f7b015b5 884284eaee8e 2023-07-18T03:06:45.379 INFO:teuthology.orchestra.run.smithi101.stdout:osd.4 smithi155 running (5m) 39s ago 5m 66.9M 4292M 18.0.0-4746-g81cf785a 66a4f7b015b5 8ecfd4731f39 2023-07-18T03:06:45.380 INFO:teuthology.orchestra.run.smithi101.stdout:osd.5 smithi179 running (4m) 39s ago 4m 67.8M 4292M 18.0.0-4746-g81cf785a 66a4f7b015b5 cea52715c1fd 2023-07-18T03:06:45.380 INFO:teuthology.orchestra.run.smithi101.stdout:osd.6 smithi101 running (4m) 24s ago 4m 60.4M 4316M 18.0.0-4746-g81cf785a 66a4f7b015b5 e6ee493f08b2 2023-07-18T03:06:45.380 INFO:teuthology.orchestra.run.smithi101.stdout:osd.7 smithi155 running (4m) 39s ago 4m 64.6M 4292M 18.0.0-4746-g81cf785a 66a4f7b015b5 24a73771c8b2 2023-07-18T03:06:45.381 INFO:teuthology.orchestra.run.smithi101.stdout:osd.8 smithi179 running (3m) 39s ago 3m 65.5M 4292M 18.0.0-4746-g81cf785a 66a4f7b015b5 ecd816df1251 2023-07-18T03:06:45.381 INFO:teuthology.orchestra.run.smithi101.stdout:osd.9 smithi101 running (3m) 24s ago 3m 59.5M 4316M 18.0.0-4746-g81cf785a 66a4f7b015b5 095ba139821d 2023-07-18T03:06:45.381 INFO:teuthology.orchestra.run.smithi101.stdout:osd.10 smithi155 running (3m) 39s ago 3m 61.7M 4292M 18.0.0-4746-g81cf785a 66a4f7b015b5 96b36c6a4f56 2023-07-18T03:06:45.381 INFO:teuthology.orchestra.run.smithi101.stdout:osd.11 smithi179 running (2m) 39s ago 2m 55.6M 4292M 18.0.0-4746-g81cf785a 66a4f7b015b5 9ce5e4f535a2 2023-07-18T03:06:45.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.388+0000 7f103a934700 1 -- 172.21.15.101:0/397527350 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f101c0a11e0 msgr2=0x7f101c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:45.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.388+0000 7f103a934700 1 --2- 172.21.15.101:0/397527350 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f101c0a11e0 0x7f101c0a3690 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1034100bf0 tx=0x7f1024009310 comp rx=0 tx=0).stop 2023-07-18T03:06:45.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.388+0000 7f103a934700 1 -- 172.21.15.101:0/397527350 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1034100d90 msgr2=0x7f1034195ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:45.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.388+0000 7f103a934700 1 --2- 172.21.15.101:0/397527350 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1034100d90 0x7f1034195ef0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f102800eb60 tx=0x7f102800ef20 comp rx=0 tx=0).stop 2023-07-18T03:06:45.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.389+0000 7f103a934700 1 -- 172.21.15.101:0/397527350 shutdown_connections 2023-07-18T03:06:45.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.389+0000 7f103a934700 1 --2- 172.21.15.101:0/397527350 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f101c0a11e0 0x7f101c0a3690 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:45.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.389+0000 7f103a934700 1 --2- 172.21.15.101:0/397527350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1034101720 0x7f103419cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:45.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.389+0000 7f103a934700 1 --2- 172.21.15.101:0/397527350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10340ffb90 0x7f10341959b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:45.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.389+0000 7f103a934700 1 --2- 172.21.15.101:0/397527350 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1034100d90 0x7f1034195ef0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:45.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.389+0000 7f103a934700 1 -- 172.21.15.101:0/397527350 >> 172.21.15.101:0/397527350 conn(0x7f10340fb1a0 msgr2=0x7f10340fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:45.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.389+0000 7f103a934700 1 -- 172.21.15.101:0/397527350 shutdown_connections 2023-07-18T03:06:45.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:45.389+0000 7f103a934700 1 -- 172.21.15.101:0/397527350 wait complete. 2023-07-18T03:06:45.919 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- bash -c 'ceph orch ls' 2023-07-18T03:06:46.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:45 smithi101 ceph-mon[126379]: pgmap v332: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:06:46.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:45 smithi101 ceph-mon[126379]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:06:46.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:45 smithi155 ceph-mon[128885]: pgmap v332: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:06:46.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:45 smithi155 ceph-mon[128885]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:06:46.443 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:45 smithi179 ceph-mon[129229]: pgmap v332: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:06:46.443 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:45 smithi179 ceph-mon[129229]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:06:46.640 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:06:48.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:47 smithi101 ceph-mon[126379]: pgmap v333: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:06:48.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:47 smithi155 ceph-mon[128885]: pgmap v333: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:06:48.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.398+0000 7fb6a6ae9700 1 -- 172.21.15.101:0/3765792639 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb6a00ffc80 msgr2=0x7fb6a01000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:48.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.398+0000 7fb6a6ae9700 1 --2- 172.21.15.101:0/3765792639 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb6a00ffc80 0x7fb6a01000d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb68c009990 tx=0x7fb68c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:06:48.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.398+0000 7fb6a6ae9700 1 -- 172.21.15.101:0/3765792639 shutdown_connections 2023-07-18T03:06:48.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.398+0000 7fb6a6ae9700 1 --2- 172.21.15.101:0/3765792639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6a00fea80 0x7fb6a00fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:48.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.398+0000 7fb6a6ae9700 1 --2- 172.21.15.101:0/3765792639 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6a0100610 0x7fb6a0106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:48.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.398+0000 7fb6a6ae9700 1 --2- 172.21.15.101:0/3765792639 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb6a00ffc80 0x7fb6a01000d0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:48.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.398+0000 7fb6a6ae9700 1 -- 172.21.15.101:0/3765792639 >> 172.21.15.101:0/3765792639 conn(0x7fb6a00fa0b0 msgr2=0x7fb6a00fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:48.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.398+0000 7fb6a6ae9700 1 -- 172.21.15.101:0/3765792639 shutdown_connections 2023-07-18T03:06:48.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.399+0000 7fb6a6ae9700 1 -- 172.21.15.101:0/3765792639 wait complete. 2023-07-18T03:06:48.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.400+0000 7fb6a6ae9700 1 Processor -- start 2023-07-18T03:06:48.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.400+0000 7fb6a6ae9700 1 -- start start 2023-07-18T03:06:48.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.400+0000 7fb6a6ae9700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb6a00fea80 0x7fb6a0197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:48.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.400+0000 7fb6a6ae9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6a00ffc80 0x7fb6a0197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:48.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.400+0000 7fb6a6ae9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6a0100610 0x7fb6a019f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:48.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.400+0000 7fb6a6ae9700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a019f620 con 0x7fb6a00fea80 2023-07-18T03:06:48.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.400+0000 7fb6a6ae9700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a019f760 con 0x7fb6a00ffc80 2023-07-18T03:06:48.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.400+0000 7fb6a6ae9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a019f8f0 con 0x7fb6a0100610 2023-07-18T03:06:48.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.400+0000 7fb6a4885700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb6a00fea80 0x7fb6a0197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:48.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.400+0000 7fb6a4885700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb6a00fea80 0x7fb6a0197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:42382/0 (socket says 172.21.15.101:42382) 2023-07-18T03:06:48.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.400+0000 7fb6a4885700 1 -- 172.21.15.101:0/2194818901 learned_addr learned my addr 172.21.15.101:0/2194818901 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:48.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.400+0000 7fb69ffff700 1 --2- 172.21.15.101:0/2194818901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6a00ffc80 0x7fb6a0197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:48.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.401+0000 7fb6a5086700 1 --2- 172.21.15.101:0/2194818901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6a0100610 0x7fb6a019f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:48.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.401+0000 7fb6a4885700 1 -- 172.21.15.101:0/2194818901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6a0100610 msgr2=0x7fb6a019f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:48.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.401+0000 7fb6a4885700 1 --2- 172.21.15.101:0/2194818901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6a0100610 0x7fb6a019f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:48.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.401+0000 7fb6a4885700 1 -- 172.21.15.101:0/2194818901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6a00ffc80 msgr2=0x7fb6a0197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:48.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.401+0000 7fb6a4885700 1 --2- 172.21.15.101:0/2194818901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6a00ffc80 0x7fb6a0197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:48.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.401+0000 7fb6a4885700 1 -- 172.21.15.101:0/2194818901 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb694009520 con 0x7fb6a00fea80 2023-07-18T03:06:48.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.401+0000 7fb6a5086700 1 --2- 172.21.15.101:0/2194818901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6a0100610 0x7fb6a019f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:06:48.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.401+0000 7fb6a4885700 1 --2- 172.21.15.101:0/2194818901 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb6a00fea80 0x7fb6a0197a90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb69400ea80 tx=0x7fb69400ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:48.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.401+0000 7fb69dffb700 1 -- 172.21.15.101:0/2194818901 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69400cbd0 con 0x7fb6a00fea80 2023-07-18T03:06:48.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.401+0000 7fb6a6ae9700 1 -- 172.21.15.101:0/2194818901 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb68c0095f0 con 0x7fb6a00fea80 2023-07-18T03:06:48.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.401+0000 7fb69dffb700 1 -- 172.21.15.101:0/2194818901 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb69400cd30 con 0x7fb6a00fea80 2023-07-18T03:06:48.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.401+0000 7fb69dffb700 1 -- 172.21.15.101:0/2194818901 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb694018680 con 0x7fb6a00fea80 2023-07-18T03:06:48.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.401+0000 7fb6a6ae9700 1 -- 172.21.15.101:0/2194818901 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6a019ff90 con 0x7fb6a00fea80 2023-07-18T03:06:48.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.403+0000 7fb6a6ae9700 1 -- 172.21.15.101:0/2194818901 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6a010a290 con 0x7fb6a00fea80 2023-07-18T03:06:48.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.404+0000 7fb69dffb700 1 -- 172.21.15.101:0/2194818901 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fb6940188a0 con 0x7fb6a00fea80 2023-07-18T03:06:48.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.406+0000 7fb69dffb700 1 --2- 172.21.15.101:0/2194818901 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb6900a11e0 0x7fb6900a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:48.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.406+0000 7fb69dffb700 1 -- 172.21.15.101:0/2194818901 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fb694014070 con 0x7fb6a00fea80 2023-07-18T03:06:48.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.406+0000 7fb69ffff700 1 --2- 172.21.15.101:0/2194818901 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb6900a11e0 0x7fb6900a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:48.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.407+0000 7fb69ffff700 1 --2- 172.21.15.101:0/2194818901 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb6900a11e0 0x7fb6900a3690 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb68c00b2e0 tx=0x7fb68c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:48.395 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.409+0000 7fb69dffb700 1 -- 172.21.15.101:0/2194818901 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb694096650 con 0x7fb6a00fea80 2023-07-18T03:06:48.443 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:47 smithi179 ceph-mon[129229]: pgmap v333: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:06:48.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.611+0000 7fb6a6ae9700 1 -- 172.21.15.101:0/2194818901 --> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fb6a01a05a0 con 0x7fb6900a11e0 2023-07-18T03:06:48.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.617+0000 7fb69dffb700 1 -- 172.21.15.101:0/2194818901 <== mgr.14150 v2:172.21.15.101:6800/3963742842 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7fb6a01a05a0 con 0x7fb6900a11e0 2023-07-18T03:06:48.603 INFO:teuthology.orchestra.run.smithi101.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-07-18T03:06:48.603 INFO:teuthology.orchestra.run.smithi101.stdout:mds.all 12/12 43s ago 2m smithi101=a;smithi101=d;smithi101=g;smithi101=j;smithi155=b;smithi155=e;smithi155=h;smithi155=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l;count:12 2023-07-18T03:06:48.603 INFO:teuthology.orchestra.run.smithi101.stdout:mgr 3/3 43s ago 7m smithi101=x;smithi155=y;smithi179=z;count:3 2023-07-18T03:06:48.604 INFO:teuthology.orchestra.run.smithi101.stdout:mon 3/3 43s ago 7m smithi101:172.21.15.101=a;smithi155:172.21.15.155=b;smithi179:172.21.15.179=c;count:3 2023-07-18T03:06:48.604 INFO:teuthology.orchestra.run.smithi101.stdout:osd 12 43s ago - 2023-07-18T03:06:48.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.619+0000 7fb6a6ae9700 1 -- 172.21.15.101:0/2194818901 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb6900a11e0 msgr2=0x7fb6900a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:48.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.619+0000 7fb6a6ae9700 1 --2- 172.21.15.101:0/2194818901 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb6900a11e0 0x7fb6900a3690 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb68c00b2e0 tx=0x7fb68c005fb0 comp rx=0 tx=0).stop 2023-07-18T03:06:48.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.619+0000 7fb6a6ae9700 1 -- 172.21.15.101:0/2194818901 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb6a00fea80 msgr2=0x7fb6a0197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:48.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.619+0000 7fb6a6ae9700 1 --2- 172.21.15.101:0/2194818901 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb6a00fea80 0x7fb6a0197a90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb69400ea80 tx=0x7fb69400ed90 comp rx=0 tx=0).stop 2023-07-18T03:06:48.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.619+0000 7fb6a6ae9700 1 -- 172.21.15.101:0/2194818901 shutdown_connections 2023-07-18T03:06:48.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.619+0000 7fb6a6ae9700 1 --2- 172.21.15.101:0/2194818901 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb6900a11e0 0x7fb6900a3690 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:48.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.619+0000 7fb6a6ae9700 1 --2- 172.21.15.101:0/2194818901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6a0100610 0x7fb6a019f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:48.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.619+0000 7fb6a6ae9700 1 --2- 172.21.15.101:0/2194818901 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb6a00fea80 0x7fb6a0197a90 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:48.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.619+0000 7fb6a6ae9700 1 --2- 172.21.15.101:0/2194818901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6a00ffc80 0x7fb6a0197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:48.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.619+0000 7fb6a6ae9700 1 -- 172.21.15.101:0/2194818901 >> 172.21.15.101:0/2194818901 conn(0x7fb6a00fa0b0 msgr2=0x7fb6a0068050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:48.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.619+0000 7fb6a6ae9700 1 -- 172.21.15.101:0/2194818901 shutdown_connections 2023-07-18T03:06:48.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:48.620+0000 7fb6a6ae9700 1 -- 172.21.15.101:0/2194818901 wait complete. 2023-07-18T03:06:49.232 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- bash -c 'ceph orch host ls' 2023-07-18T03:06:49.953 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:06:50.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:49 smithi179 ceph-mon[129229]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:06:50.194 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:49 smithi179 ceph-mon[129229]: pgmap v334: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-18T03:06:50.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:49 smithi101 ceph-mon[126379]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:06:50.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:49 smithi101 ceph-mon[126379]: pgmap v334: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-18T03:06:50.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:49 smithi155 ceph-mon[128885]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:06:50.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:49 smithi155 ceph-mon[128885]: pgmap v334: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-18T03:06:51.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.705+0000 7fe6157cf700 1 -- 172.21.15.101:0/2505493405 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe610101720 msgr2=0x7fe610107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:51.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.705+0000 7fe6157cf700 1 --2- 172.21.15.101:0/2505493405 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe610101720 0x7fe610107fe0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe60800b0a0 tx=0x7fe60800b3b0 comp rx=0 tx=0).stop 2023-07-18T03:06:51.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.705+0000 7fe6157cf700 1 -- 172.21.15.101:0/2505493405 shutdown_connections 2023-07-18T03:06:51.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.705+0000 7fe6157cf700 1 --2- 172.21.15.101:0/2505493405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6100ffb90 0x7fe6100fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:51.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.705+0000 7fe6157cf700 1 --2- 172.21.15.101:0/2505493405 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe610101720 0x7fe610107fe0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:51.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.705+0000 7fe6157cf700 1 --2- 172.21.15.101:0/2505493405 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe610100d90 0x7fe6101011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:51.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.705+0000 7fe6157cf700 1 -- 172.21.15.101:0/2505493405 >> 172.21.15.101:0/2505493405 conn(0x7fe6100fb1a0 msgr2=0x7fe6100fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:51.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.705+0000 7fe6157cf700 1 -- 172.21.15.101:0/2505493405 shutdown_connections 2023-07-18T03:06:51.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.705+0000 7fe6157cf700 1 -- 172.21.15.101:0/2505493405 wait complete. 2023-07-18T03:06:51.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.706+0000 7fe6157cf700 1 Processor -- start 2023-07-18T03:06:51.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.706+0000 7fe6157cf700 1 -- start start 2023-07-18T03:06:51.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.707+0000 7fe6157cf700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6100ffb90 0x7fe610197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:51.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.707+0000 7fe6157cf700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe610100d90 0x7fe610198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:51.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.707+0000 7fe6157cf700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe610101720 0x7fe61019f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:51.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.707+0000 7fe6157cf700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61019f750 con 0x7fe610101720 2023-07-18T03:06:51.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.707+0000 7fe6157cf700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61019f890 con 0x7fe610100d90 2023-07-18T03:06:51.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.707+0000 7fe6157cf700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61019fa20 con 0x7fe6100ffb90 2023-07-18T03:06:51.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.707+0000 7fe60f7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe610101720 0x7fe61019f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:51.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.707+0000 7fe60effd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6100ffb90 0x7fe610197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:51.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.707+0000 7fe60f7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe610101720 0x7fe61019f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:42394/0 (socket says 172.21.15.101:42394) 2023-07-18T03:06:51.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.707+0000 7fe60f7fe700 1 -- 172.21.15.101:0/2166647679 learned_addr learned my addr 172.21.15.101:0/2166647679 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:51.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.707+0000 7fe60e7fc700 1 --2- 172.21.15.101:0/2166647679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe610100d90 0x7fe610198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:51.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.707+0000 7fe60f7fe700 1 -- 172.21.15.101:0/2166647679 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6100ffb90 msgr2=0x7fe610197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:51.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.707+0000 7fe60f7fe700 1 --2- 172.21.15.101:0/2166647679 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6100ffb90 0x7fe610197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:51.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.707+0000 7fe60f7fe700 1 -- 172.21.15.101:0/2166647679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe610100d90 msgr2=0x7fe610198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:51.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.707+0000 7fe60f7fe700 1 --2- 172.21.15.101:0/2166647679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe610100d90 0x7fe610198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:51.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.707+0000 7fe60f7fe700 1 -- 172.21.15.101:0/2166647679 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe608009d00 con 0x7fe610101720 2023-07-18T03:06:51.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.708+0000 7fe60f7fe700 1 --2- 172.21.15.101:0/2166647679 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe610101720 0x7fe61019f180 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe60800b070 tx=0x7fe60800bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:51.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.708+0000 7fe5f7fff700 1 -- 172.21.15.101:0/2166647679 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60800e040 con 0x7fe610101720 2023-07-18T03:06:51.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.708+0000 7fe5f7fff700 1 -- 172.21.15.101:0/2166647679 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6080094a0 con 0x7fe610101720 2023-07-18T03:06:51.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.708+0000 7fe6157cf700 1 -- 172.21.15.101:0/2166647679 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe61019fca0 con 0x7fe610101720 2023-07-18T03:06:51.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.708+0000 7fe5f7fff700 1 -- 172.21.15.101:0/2166647679 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60801b6c0 con 0x7fe610101720 2023-07-18T03:06:51.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.708+0000 7fe6157cf700 1 -- 172.21.15.101:0/2166647679 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6101a0060 con 0x7fe610101720 2023-07-18T03:06:51.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.709+0000 7fe6157cf700 1 -- 172.21.15.101:0/2166647679 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe61010b420 con 0x7fe610101720 2023-07-18T03:06:51.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.711+0000 7fe5f7fff700 1 -- 172.21.15.101:0/2166647679 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fe60801d470 con 0x7fe610101720 2023-07-18T03:06:51.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.712+0000 7fe5f7fff700 1 --2- 172.21.15.101:0/2166647679 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe5f80a11e0 0x7fe5f80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:51.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.713+0000 7fe5f7fff700 1 -- 172.21.15.101:0/2166647679 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fe6080ce070 con 0x7fe610101720 2023-07-18T03:06:51.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.713+0000 7fe60effd700 1 --2- 172.21.15.101:0/2166647679 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe5f80a11e0 0x7fe5f80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:51.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.713+0000 7fe60effd700 1 --2- 172.21.15.101:0/2166647679 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe5f80a11e0 0x7fe5f80a3690 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe610100bf0 tx=0x7fe600009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:51.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.717+0000 7fe5f7fff700 1 -- 172.21.15.101:0/2166647679 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe608097320 con 0x7fe610101720 2023-07-18T03:06:51.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.921+0000 7fe6157cf700 1 -- 172.21.15.101:0/2166647679 --> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fe610107d30 con 0x7fe5f80a11e0 2023-07-18T03:06:51.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.923+0000 7fe5f7fff700 1 -- 172.21.15.101:0/2166647679 <== mgr.14150 v2:172.21.15.101:6800/3963742842 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7fe610107d30 con 0x7fe5f80a11e0 2023-07-18T03:06:51.910 INFO:teuthology.orchestra.run.smithi101.stdout:HOST ADDR LABELS STATUS 2023-07-18T03:06:51.910 INFO:teuthology.orchestra.run.smithi101.stdout:smithi101 172.21.15.101 2023-07-18T03:06:51.910 INFO:teuthology.orchestra.run.smithi101.stdout:smithi155 172.21.15.155 2023-07-18T03:06:51.910 INFO:teuthology.orchestra.run.smithi101.stdout:smithi179 172.21.15.179 2023-07-18T03:06:51.910 INFO:teuthology.orchestra.run.smithi101.stdout:3 hosts in cluster 2023-07-18T03:06:51.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.925+0000 7fe6157cf700 1 -- 172.21.15.101:0/2166647679 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe5f80a11e0 msgr2=0x7fe5f80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:51.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.925+0000 7fe6157cf700 1 --2- 172.21.15.101:0/2166647679 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe5f80a11e0 0x7fe5f80a3690 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe610100bf0 tx=0x7fe600009260 comp rx=0 tx=0).stop 2023-07-18T03:06:51.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.925+0000 7fe6157cf700 1 -- 172.21.15.101:0/2166647679 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe610101720 msgr2=0x7fe61019f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:51.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.925+0000 7fe6157cf700 1 --2- 172.21.15.101:0/2166647679 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe610101720 0x7fe61019f180 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe60800b070 tx=0x7fe60800bce0 comp rx=0 tx=0).stop 2023-07-18T03:06:51.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.925+0000 7fe6157cf700 1 -- 172.21.15.101:0/2166647679 shutdown_connections 2023-07-18T03:06:51.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.925+0000 7fe6157cf700 1 --2- 172.21.15.101:0/2166647679 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe5f80a11e0 0x7fe5f80a3690 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:51.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.925+0000 7fe6157cf700 1 --2- 172.21.15.101:0/2166647679 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe610101720 0x7fe61019f180 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:51.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.925+0000 7fe6157cf700 1 --2- 172.21.15.101:0/2166647679 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6100ffb90 0x7fe610197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:51.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.925+0000 7fe6157cf700 1 --2- 172.21.15.101:0/2166647679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe610100d90 0x7fe610198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:51.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.925+0000 7fe6157cf700 1 -- 172.21.15.101:0/2166647679 >> 172.21.15.101:0/2166647679 conn(0x7fe6100fb1a0 msgr2=0x7fe610106610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:51.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.925+0000 7fe6157cf700 1 -- 172.21.15.101:0/2166647679 shutdown_connections 2023-07-18T03:06:51.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:51.925+0000 7fe6157cf700 1 -- 172.21.15.101:0/2166647679 wait complete. 2023-07-18T03:06:52.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:51 smithi179 ceph-mon[129229]: pgmap v335: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:06:52.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:51 smithi101 ceph-mon[126379]: pgmap v335: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:06:52.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:51 smithi155 ceph-mon[128885]: pgmap v335: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:06:52.444 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- bash -c 'ceph orch device ls' 2023-07-18T03:06:53.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:52 smithi179 ceph-mon[129229]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:06:53.209 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:06:53.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:52 smithi101 ceph-mon[126379]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:06:53.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:52 smithi155 ceph-mon[128885]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:06:54.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:53 smithi179 ceph-mon[129229]: pgmap v336: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-18T03:06:54.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:53 smithi101 ceph-mon[126379]: pgmap v336: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-18T03:06:54.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:53 smithi155 ceph-mon[128885]: pgmap v336: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-18T03:06:54.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:54.953+0000 7f322b16f700 1 -- 172.21.15.101:0/1970704360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3224102fc0 msgr2=0x7f3224103410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:54.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:54.953+0000 7f322b16f700 1 --2- 172.21.15.101:0/1970704360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3224102fc0 0x7f3224103410 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3210009990 tx=0x7f3210009ca0 comp rx=0 tx=0).stop 2023-07-18T03:06:54.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:54.953+0000 7f322b16f700 1 -- 172.21.15.101:0/1970704360 shutdown_connections 2023-07-18T03:06:54.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:54.953+0000 7f322b16f700 1 --2- 172.21.15.101:0/1970704360 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3224103950 0x7f322410a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:54.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:54.953+0000 7f322b16f700 1 --2- 172.21.15.101:0/1970704360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3224101dc0 0x7f32241021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:54.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:54.953+0000 7f322b16f700 1 --2- 172.21.15.101:0/1970704360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3224102fc0 0x7f3224103410 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:54.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:54.953+0000 7f322b16f700 1 -- 172.21.15.101:0/1970704360 >> 172.21.15.101:0/1970704360 conn(0x7f32240fd410 msgr2=0x7f32240ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:54.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:54.953+0000 7f322b16f700 1 -- 172.21.15.101:0/1970704360 shutdown_connections 2023-07-18T03:06:54.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:54.953+0000 7f322b16f700 1 -- 172.21.15.101:0/1970704360 wait complete. 2023-07-18T03:06:54.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:54.954+0000 7f322b16f700 1 Processor -- start 2023-07-18T03:06:54.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:54.954+0000 7f322b16f700 1 -- start start 2023-07-18T03:06:54.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:54.955+0000 7f322b16f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3224101dc0 0x7f3224197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:54.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:54.955+0000 7f322b16f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3224102fc0 0x7f3224197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:54.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:54.955+0000 7f322b16f700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3224103950 0x7f322419f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:54.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:54.955+0000 7f322b16f700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322419f610 con 0x7f3224103950 2023-07-18T03:06:54.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:54.955+0000 7f322b16f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322419f750 con 0x7f3224101dc0 2023-07-18T03:06:54.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:54.955+0000 7f322b16f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322419f8e0 con 0x7f3224102fc0 2023-07-18T03:06:54.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:54.955+0000 7f322970c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3224103950 0x7f322419f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:54.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:54.955+0000 7f3223fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3224102fc0 0x7f3224197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:54.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:54.955+0000 7f322970c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3224103950 0x7f322419f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:42426/0 (socket says 172.21.15.101:42426) 2023-07-18T03:06:54.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:54.955+0000 7f3223fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3224102fc0 0x7f3224197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:39818/0 (socket says 172.21.15.101:39818) 2023-07-18T03:06:54.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:54.955+0000 7f322970c700 1 -- 172.21.15.101:0/3339072771 learned_addr learned my addr 172.21.15.101:0/3339072771 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:54.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:54.955+0000 7f3228f0b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3224101dc0 0x7f3224197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:54.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:54.955+0000 7f322970c700 1 -- 172.21.15.101:0/3339072771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3224102fc0 msgr2=0x7f3224197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:54.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:54.955+0000 7f322970c700 1 --2- 172.21.15.101:0/3339072771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3224102fc0 0x7f3224197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:54.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:54.955+0000 7f322970c700 1 -- 172.21.15.101:0/3339072771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3224101dc0 msgr2=0x7f3224197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:54.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:54.955+0000 7f322970c700 1 --2- 172.21.15.101:0/3339072771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3224101dc0 0x7f3224197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:54.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:54.956+0000 7f322970c700 1 -- 172.21.15.101:0/3339072771 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32100095f0 con 0x7f3224103950 2023-07-18T03:06:54.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:54.956+0000 7f3228f0b700 1 --2- 172.21.15.101:0/3339072771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3224101dc0 0x7f3224197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:06:54.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:54.956+0000 7f322970c700 1 --2- 172.21.15.101:0/3339072771 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3224103950 0x7f322419f040 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f321c00ef60 tx=0x7f321c00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:54.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:54.956+0000 7f3221ffb700 1 -- 172.21.15.101:0/3339072771 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c018070 con 0x7f3224103950 2023-07-18T03:06:54.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:54.956+0000 7f322b16f700 1 -- 172.21.15.101:0/3339072771 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f322419fb60 con 0x7f3224103950 2023-07-18T03:06:54.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:54.956+0000 7f3221ffb700 1 -- 172.21.15.101:0/3339072771 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f321c00f040 con 0x7f3224103950 2023-07-18T03:06:54.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:54.956+0000 7f3221ffb700 1 -- 172.21.15.101:0/3339072771 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c00cc70 con 0x7f3224103950 2023-07-18T03:06:54.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:54.956+0000 7f322b16f700 1 -- 172.21.15.101:0/3339072771 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32241a00b0 con 0x7f3224103950 2023-07-18T03:06:54.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:54.957+0000 7f322b16f700 1 -- 172.21.15.101:0/3339072771 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3224109350 con 0x7f3224103950 2023-07-18T03:06:54.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:54.960+0000 7f3221ffb700 1 -- 172.21.15.101:0/3339072771 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f321c003a00 con 0x7f3224103950 2023-07-18T03:06:54.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:54.961+0000 7f3221ffb700 1 --2- 172.21.15.101:0/3339072771 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f32140a1150 0x7f32140a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:54.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:54.961+0000 7f3221ffb700 1 -- 172.21.15.101:0/3339072771 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f321c0cc750 con 0x7f3224103950 2023-07-18T03:06:54.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:54.961+0000 7f3228f0b700 1 --2- 172.21.15.101:0/3339072771 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f32140a1150 0x7f32140a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:54.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:54.962+0000 7f3228f0b700 1 --2- 172.21.15.101:0/3339072771 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f32140a1150 0x7f32140a3600 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3224102e20 tx=0x7f3218009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:54.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:54.965+0000 7f3221ffb700 1 -- 172.21.15.101:0/3339072771 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f321c095b60 con 0x7f3224103950 2023-07-18T03:06:55.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:55.170+0000 7f322b16f700 1 -- 172.21.15.101:0/3339072771 --> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f3224060950 con 0x7f32140a1150 2023-07-18T03:06:55.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:55.171+0000 7f3221ffb700 1 -- 172.21.15.101:0/3339072771 <== mgr.14150 v2:172.21.15.101:6800/3963742842 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+492 (secure 0 0 0) 0x7f3224060950 con 0x7f32140a1150 2023-07-18T03:06:55.159 INFO:teuthology.orchestra.run.smithi101.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-07-18T03:06:55.159 INFO:teuthology.orchestra.run.smithi101.stdout:smithi101 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204017B400BGN 372G No 3m ago LVM detected, locked 2023-07-18T03:06:55.159 INFO:teuthology.orchestra.run.smithi101.stdout:smithi155 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GW400BGN 372G No 3m ago LVM detected, locked 2023-07-18T03:06:55.159 INFO:teuthology.orchestra.run.smithi101.stdout:smithi179 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6303009J400BGN 372G No 2m ago LVM detected, locked 2023-07-18T03:06:55.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:55.174+0000 7f322b16f700 1 -- 172.21.15.101:0/3339072771 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f32140a1150 msgr2=0x7f32140a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:55.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:55.174+0000 7f322b16f700 1 --2- 172.21.15.101:0/3339072771 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f32140a1150 0x7f32140a3600 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3224102e20 tx=0x7f3218009260 comp rx=0 tx=0).stop 2023-07-18T03:06:55.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:55.174+0000 7f322b16f700 1 -- 172.21.15.101:0/3339072771 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3224103950 msgr2=0x7f322419f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:55.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:55.174+0000 7f322b16f700 1 --2- 172.21.15.101:0/3339072771 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3224103950 0x7f322419f040 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f321c00ef60 tx=0x7f321c00ef90 comp rx=0 tx=0).stop 2023-07-18T03:06:55.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:55.174+0000 7f322b16f700 1 -- 172.21.15.101:0/3339072771 shutdown_connections 2023-07-18T03:06:55.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:55.174+0000 7f322b16f700 1 --2- 172.21.15.101:0/3339072771 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f32140a1150 0x7f32140a3600 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:55.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:55.174+0000 7f322b16f700 1 --2- 172.21.15.101:0/3339072771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3224101dc0 0x7f3224197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:55.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:55.174+0000 7f322b16f700 1 --2- 172.21.15.101:0/3339072771 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3224103950 0x7f322419f040 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:55.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:55.174+0000 7f322b16f700 1 --2- 172.21.15.101:0/3339072771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3224102fc0 0x7f3224197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:55.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:55.174+0000 7f322b16f700 1 -- 172.21.15.101:0/3339072771 >> 172.21.15.101:0/3339072771 conn(0x7f32240fd410 msgr2=0x7f3224106490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:55.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:55.174+0000 7f322b16f700 1 -- 172.21.15.101:0/3339072771 shutdown_connections 2023-07-18T03:06:55.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:55.174+0000 7f322b16f700 1 -- 172.21.15.101:0/3339072771 wait complete. 2023-07-18T03:06:55.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:54 smithi179 ceph-mon[129229]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-07-18T03:06:55.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:54 smithi101 ceph-mon[126379]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-07-18T03:06:55.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:54 smithi155 ceph-mon[128885]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-07-18T03:06:55.704 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-07-18T03:06:55.714 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi101.front.sepia.ceph.com 2023-07-18T03:06:55.714 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- bash -c 'ceph fs dump' 2023-07-18T03:06:56.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:55 smithi179 ceph-mon[129229]: pgmap v337: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:06:56.194 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:55 smithi179 ceph-mon[129229]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:06:56.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:55 smithi101 ceph-mon[126379]: pgmap v337: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:06:56.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:55 smithi101 ceph-mon[126379]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:06:56.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:55 smithi155 ceph-mon[128885]: pgmap v337: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:06:56.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:55 smithi155 ceph-mon[128885]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:06:56.439 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:06:58.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.038+0000 7fa469f51700 1 -- 172.21.15.101:0/1301616081 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa464100d90 msgr2=0x7fa4641011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:58.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.038+0000 7fa469f51700 1 --2- 172.21.15.101:0/1301616081 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa464100d90 0x7fa4641011e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa458009990 tx=0x7fa458009ca0 comp rx=0 tx=0).stop 2023-07-18T03:06:58.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.038+0000 7fa469f51700 1 -- 172.21.15.101:0/1301616081 shutdown_connections 2023-07-18T03:06:58.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.038+0000 7fa469f51700 1 --2- 172.21.15.101:0/1301616081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa464101720 0x7fa464107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:58.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.038+0000 7fa469f51700 1 --2- 172.21.15.101:0/1301616081 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4640ffb90 0x7fa4640fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:58.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.038+0000 7fa469f51700 1 --2- 172.21.15.101:0/1301616081 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa464100d90 0x7fa4641011e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:58.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.038+0000 7fa469f51700 1 -- 172.21.15.101:0/1301616081 >> 172.21.15.101:0/1301616081 conn(0x7fa4640fb1a0 msgr2=0x7fa4640fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:58.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.038+0000 7fa469f51700 1 -- 172.21.15.101:0/1301616081 shutdown_connections 2023-07-18T03:06:58.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.038+0000 7fa469f51700 1 -- 172.21.15.101:0/1301616081 wait complete. 2023-07-18T03:06:58.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.039+0000 7fa469f51700 1 Processor -- start 2023-07-18T03:06:58.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.039+0000 7fa469f51700 1 -- start start 2023-07-18T03:06:58.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.040+0000 7fa469f51700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4640ffb90 0x7fa464197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:58.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.040+0000 7fa469f51700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa464100d90 0x7fa464197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:58.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.040+0000 7fa469f51700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa464101720 0x7fa46419f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:58.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.040+0000 7fa469f51700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46419f610 con 0x7fa464101720 2023-07-18T03:06:58.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.040+0000 7fa469f51700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46419f750 con 0x7fa464100d90 2023-07-18T03:06:58.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.040+0000 7fa469f51700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46419f8e0 con 0x7fa4640ffb90 2023-07-18T03:06:58.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.040+0000 7fa463fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa464101720 0x7fa46419f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:58.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.040+0000 7fa4637fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4640ffb90 0x7fa464197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:58.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.040+0000 7fa463fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa464101720 0x7fa46419f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:39792/0 (socket says 172.21.15.101:39792) 2023-07-18T03:06:58.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.040+0000 7fa463fff700 1 -- 172.21.15.101:0/3226886563 learned_addr learned my addr 172.21.15.101:0/3226886563 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:06:58.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.040+0000 7fa462ffd700 1 --2- 172.21.15.101:0/3226886563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa464100d90 0x7fa464197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:58.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.040+0000 7fa463fff700 1 -- 172.21.15.101:0/3226886563 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4640ffb90 msgr2=0x7fa464197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:58.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.040+0000 7fa463fff700 1 --2- 172.21.15.101:0/3226886563 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4640ffb90 0x7fa464197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:58.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.040+0000 7fa463fff700 1 -- 172.21.15.101:0/3226886563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa464100d90 msgr2=0x7fa464197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:58.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.040+0000 7fa463fff700 1 --2- 172.21.15.101:0/3226886563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa464100d90 0x7fa464197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:58.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.040+0000 7fa463fff700 1 -- 172.21.15.101:0/3226886563 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4580095f0 con 0x7fa464101720 2023-07-18T03:06:58.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.041+0000 7fa463fff700 1 --2- 172.21.15.101:0/3226886563 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa464101720 0x7fa46419f040 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa45c00ef60 tx=0x7fa45c00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:58.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.041+0000 7fa460ff9700 1 -- 172.21.15.101:0/3226886563 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45c018070 con 0x7fa464101720 2023-07-18T03:06:58.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.041+0000 7fa460ff9700 1 -- 172.21.15.101:0/3226886563 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa45c00f040 con 0x7fa464101720 2023-07-18T03:06:58.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.041+0000 7fa469f51700 1 -- 172.21.15.101:0/3226886563 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa46419fb60 con 0x7fa464101720 2023-07-18T03:06:58.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.041+0000 7fa460ff9700 1 -- 172.21.15.101:0/3226886563 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45c00ccc0 con 0x7fa464101720 2023-07-18T03:06:58.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.041+0000 7fa469f51700 1 -- 172.21.15.101:0/3226886563 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4641a0060 con 0x7fa464101720 2023-07-18T03:06:58.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.042+0000 7fa469f51700 1 -- 172.21.15.101:0/3226886563 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa46410b350 con 0x7fa464101720 2023-07-18T03:06:58.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.044+0000 7fa460ff9700 1 -- 172.21.15.101:0/3226886563 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fa45c003a00 con 0x7fa464101720 2023-07-18T03:06:58.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.046+0000 7fa460ff9700 1 --2- 172.21.15.101:0/3226886563 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa44c0a1150 0x7fa44c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:06:58.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.046+0000 7fa460ff9700 1 -- 172.21.15.101:0/3226886563 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fa45c0cc740 con 0x7fa464101720 2023-07-18T03:06:58.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.046+0000 7fa4637fe700 1 --2- 172.21.15.101:0/3226886563 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa44c0a1150 0x7fa44c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:06:58.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.047+0000 7fa4637fe700 1 --2- 172.21.15.101:0/3226886563 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa44c0a1150 0x7fa44c0a3600 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa464100bf0 tx=0x7fa454009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:06:58.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.050+0000 7fa460ff9700 1 -- 172.21.15.101:0/3226886563 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa45c095b50 con 0x7fa464101720 2023-07-18T03:06:58.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:57 smithi179 ceph-mon[129229]: pgmap v338: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 6 op/s 2023-07-18T03:06:58.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:57 smithi101 ceph-mon[126379]: pgmap v338: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 6 op/s 2023-07-18T03:06:58.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.290+0000 7fa469f51700 1 -- 172.21.15.101:0/3226886563 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa4641a0610 con 0x7fa464101720 2023-07-18T03:06:58.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.290+0000 7fa460ff9700 1 -- 172.21.15.101:0/3226886563 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) v1 ==== 76+0+2909 (secure 0 0 0) 0x7fa45c099170 con 0x7fa464101720 2023-07-18T03:06:58.278 INFO:teuthology.orchestra.run.smithi101.stdout:e24 2023-07-18T03:06:58.278 INFO:teuthology.orchestra.run.smithi101.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-07-18T03:06:58.278 INFO:teuthology.orchestra.run.smithi101.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-18T03:06:58.278 INFO:teuthology.orchestra.run.smithi101.stdout:legacy client fscid: 1 2023-07-18T03:06:58.278 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:06:58.279 INFO:teuthology.orchestra.run.smithi101.stdout:Filesystem 'cephfs' (1) 2023-07-18T03:06:58.279 INFO:teuthology.orchestra.run.smithi101.stdout:fs_name cephfs 2023-07-18T03:06:58.279 INFO:teuthology.orchestra.run.smithi101.stdout:epoch 24 2023-07-18T03:06:58.279 INFO:teuthology.orchestra.run.smithi101.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-07-18T03:06:58.279 INFO:teuthology.orchestra.run.smithi101.stdout:created 2023-07-18T03:04:59.361185+0000 2023-07-18T03:06:58.279 INFO:teuthology.orchestra.run.smithi101.stdout:modified 2023-07-18T03:05:31.985959+0000 2023-07-18T03:06:58.279 INFO:teuthology.orchestra.run.smithi101.stdout:tableserver 0 2023-07-18T03:06:58.279 INFO:teuthology.orchestra.run.smithi101.stdout:root 0 2023-07-18T03:06:58.280 INFO:teuthology.orchestra.run.smithi101.stdout:session_timeout 300 2023-07-18T03:06:58.280 INFO:teuthology.orchestra.run.smithi101.stdout:session_autoclose 300 2023-07-18T03:06:58.280 INFO:teuthology.orchestra.run.smithi101.stdout:max_file_size 1099511627776 2023-07-18T03:06:58.280 INFO:teuthology.orchestra.run.smithi101.stdout:max_xattr_size 65536 2023-07-18T03:06:58.280 INFO:teuthology.orchestra.run.smithi101.stdout:required_client_features {} 2023-07-18T03:06:58.280 INFO:teuthology.orchestra.run.smithi101.stdout:last_failure 0 2023-07-18T03:06:58.280 INFO:teuthology.orchestra.run.smithi101.stdout:last_failure_osd_epoch 0 2023-07-18T03:06:58.281 INFO:teuthology.orchestra.run.smithi101.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-18T03:06:58.281 INFO:teuthology.orchestra.run.smithi101.stdout:max_mds 3 2023-07-18T03:06:58.281 INFO:teuthology.orchestra.run.smithi101.stdout:in 0,1,2 2023-07-18T03:06:58.281 INFO:teuthology.orchestra.run.smithi101.stdout:up {0=24562,1=24557,2=24541} 2023-07-18T03:06:58.281 INFO:teuthology.orchestra.run.smithi101.stdout:failed 2023-07-18T03:06:58.281 INFO:teuthology.orchestra.run.smithi101.stdout:damaged 2023-07-18T03:06:58.281 INFO:teuthology.orchestra.run.smithi101.stdout:stopped 2023-07-18T03:06:58.281 INFO:teuthology.orchestra.run.smithi101.stdout:data_pools [3,4] 2023-07-18T03:06:58.282 INFO:teuthology.orchestra.run.smithi101.stdout:metadata_pool 2 2023-07-18T03:06:58.282 INFO:teuthology.orchestra.run.smithi101.stdout:inline_data disabled 2023-07-18T03:06:58.282 INFO:teuthology.orchestra.run.smithi101.stdout:balancer 2023-07-18T03:06:58.282 INFO:teuthology.orchestra.run.smithi101.stdout:bal_rank_mask -1 2023-07-18T03:06:58.282 INFO:teuthology.orchestra.run.smithi101.stdout:standby_count_wanted 0 2023-07-18T03:06:58.282 INFO:teuthology.orchestra.run.smithi101.stdout:[mds.i{0:24562} state up:active seq 5 addr [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] compat {c=[1],r=[1],i=[7ff]}] 2023-07-18T03:06:58.282 INFO:teuthology.orchestra.run.smithi101.stdout:[mds.f{0:24529} state up:standby-replay seq 1 addr [v2:172.21.15.179:6834/1268903554,v1:172.21.15.179:6835/1268903554] compat {c=[1],r=[1],i=[7ff]}] 2023-07-18T03:06:58.283 INFO:teuthology.orchestra.run.smithi101.stdout:[mds.b{1:24557} state up:active seq 10 addr [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] compat {c=[1],r=[1],i=[7ff]}] 2023-07-18T03:06:58.283 INFO:teuthology.orchestra.run.smithi101.stdout:[mds.h{1:24515} state up:standby-replay seq 1 addr [v2:172.21.15.155:6836/48029036,v1:172.21.15.155:6837/48029036] compat {c=[1],r=[1],i=[7ff]}] 2023-07-18T03:06:58.283 INFO:teuthology.orchestra.run.smithi101.stdout:[mds.c{2:24541} state up:active seq 14 addr [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] compat {c=[1],r=[1],i=[7ff]}] 2023-07-18T03:06:58.283 INFO:teuthology.orchestra.run.smithi101.stdout:[mds.e{2:24488} state up:standby-replay seq 1 addr [v2:172.21.15.155:6834/943645270,v1:172.21.15.155:6835/943645270] compat {c=[1],r=[1],i=[7ff]}] 2023-07-18T03:06:58.283 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:06:58.283 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:06:58.283 INFO:teuthology.orchestra.run.smithi101.stdout:Standby daemons: 2023-07-18T03:06:58.283 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:06:58.284 INFO:teuthology.orchestra.run.smithi101.stdout:[mds.l{-1:14415} state up:standby seq 1 addr [v2:172.21.15.179:6832/2688212803,v1:172.21.15.179:6833/2688212803] compat {c=[1],r=[1],i=[7ff]}] 2023-07-18T03:06:58.284 INFO:teuthology.orchestra.run.smithi101.stdout:[mds.j{-1:14436} state up:standby seq 1 addr [v2:172.21.15.101:6834/296128976,v1:172.21.15.101:6835/296128976] compat {c=[1],r=[1],i=[7ff]}] 2023-07-18T03:06:58.284 INFO:teuthology.orchestra.run.smithi101.stdout:[mds.d{-1:14448} state up:standby seq 1 addr [v2:172.21.15.101:6836/2478893321,v1:172.21.15.101:6837/2478893321] compat {c=[1],r=[1],i=[7ff]}] 2023-07-18T03:06:58.284 INFO:teuthology.orchestra.run.smithi101.stdout:[mds.g{-1:14502} state up:standby seq 1 addr [v2:172.21.15.101:6838/4192846769,v1:172.21.15.101:6839/4192846769] compat {c=[1],r=[1],i=[7ff]}] 2023-07-18T03:06:58.284 INFO:teuthology.orchestra.run.smithi101.stdout:[mds.a{-1:14514} state up:standby seq 1 addr [v2:172.21.15.101:6840/3824340175,v1:172.21.15.101:6841/3824340175] compat {c=[1],r=[1],i=[7ff]}] 2023-07-18T03:06:58.284 INFO:teuthology.orchestra.run.smithi101.stdout:[mds.k{-1:24449} state up:standby seq 1 addr [v2:172.21.15.155:6832/3388721497,v1:172.21.15.155:6833/3388721497] compat {c=[1],r=[1],i=[7ff]}] 2023-07-18T03:06:58.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.293+0000 7fa469f51700 1 -- 172.21.15.101:0/3226886563 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa44c0a1150 msgr2=0x7fa44c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:58.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.293+0000 7fa469f51700 1 --2- 172.21.15.101:0/3226886563 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa44c0a1150 0x7fa44c0a3600 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa464100bf0 tx=0x7fa454009260 comp rx=0 tx=0).stop 2023-07-18T03:06:58.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.293+0000 7fa469f51700 1 -- 172.21.15.101:0/3226886563 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa464101720 msgr2=0x7fa46419f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:06:58.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.293+0000 7fa469f51700 1 --2- 172.21.15.101:0/3226886563 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa464101720 0x7fa46419f040 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa45c00ef60 tx=0x7fa45c00ef90 comp rx=0 tx=0).stop 2023-07-18T03:06:58.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.293+0000 7fa469f51700 1 -- 172.21.15.101:0/3226886563 shutdown_connections 2023-07-18T03:06:58.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.293+0000 7fa469f51700 1 --2- 172.21.15.101:0/3226886563 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa44c0a1150 0x7fa44c0a3600 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:58.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.293+0000 7fa469f51700 1 --2- 172.21.15.101:0/3226886563 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa464101720 0x7fa46419f040 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:58.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.293+0000 7fa469f51700 1 --2- 172.21.15.101:0/3226886563 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4640ffb90 0x7fa464197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:58.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.293+0000 7fa469f51700 1 --2- 172.21.15.101:0/3226886563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa464100d90 0x7fa464197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:06:58.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.293+0000 7fa469f51700 1 -- 172.21.15.101:0/3226886563 >> 172.21.15.101:0/3226886563 conn(0x7fa4640fb1a0 msgr2=0x7fa4640fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:06:58.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.293+0000 7fa469f51700 1 -- 172.21.15.101:0/3226886563 shutdown_connections 2023-07-18T03:06:58.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:06:58.293+0000 7fa469f51700 1 -- 172.21.15.101:0/3226886563 wait complete. 2023-07-18T03:06:58.287 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 24 2023-07-18T03:06:58.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:57 smithi155 ceph-mon[128885]: pgmap v338: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 6 op/s 2023-07-18T03:06:58.829 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- bash -c 'ceph osd dump' 2023-07-18T03:06:59.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:58 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/3226886563' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-07-18T03:06:59.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:58 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3226886563' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-07-18T03:06:59.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:58 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/3226886563' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-07-18T03:06:59.556 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:07:00.205 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:06:59 smithi101 ceph-mon[126379]: pgmap v339: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2023-07-18T03:07:00.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:06:59 smithi155 ceph-mon[128885]: pgmap v339: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2023-07-18T03:07:00.443 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:06:59 smithi179 ceph-mon[129229]: pgmap v339: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2023-07-18T03:07:01.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.294+0000 7f62d7c6c700 1 -- 172.21.15.101:0/293756417 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f62d0107a60 msgr2=0x7f62d0109ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:01.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.294+0000 7f62d7c6c700 1 --2- 172.21.15.101:0/293756417 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f62d0107a60 0x7f62d0109ea0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f62c800b0a0 tx=0x7f62c800b3b0 comp rx=0 tx=0).stop 2023-07-18T03:07:01.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.294+0000 7f62d7c6c700 1 -- 172.21.15.101:0/293756417 shutdown_connections 2023-07-18T03:07:01.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.294+0000 7f62d7c6c700 1 --2- 172.21.15.101:0/293756417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62d0074540 0x7f62d00729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:01.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.294+0000 7f62d7c6c700 1 --2- 172.21.15.101:0/293756417 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f62d0107a60 0x7f62d0109ea0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:01.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.294+0000 7f62d7c6c700 1 --2- 172.21.15.101:0/293756417 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62d0072f70 0x7f62d00733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:01.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.294+0000 7f62d7c6c700 1 -- 172.21.15.101:0/293756417 >> 172.21.15.101:0/293756417 conn(0x7f62d00fb1a0 msgr2=0x7f62d00fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:01.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.295+0000 7f62d7c6c700 1 -- 172.21.15.101:0/293756417 shutdown_connections 2023-07-18T03:07:01.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.295+0000 7f62d7c6c700 1 -- 172.21.15.101:0/293756417 wait complete. 2023-07-18T03:07:01.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.296+0000 7f62d7c6c700 1 Processor -- start 2023-07-18T03:07:01.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.296+0000 7f62d7c6c700 1 -- start start 2023-07-18T03:07:01.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.296+0000 7f62d7c6c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62d0072f70 0x7f62d0197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:01.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.296+0000 7f62d7c6c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62d0074540 0x7f62d01980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:01.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.296+0000 7f62d7c6c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f62d0107a60 0x7f62d019f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:01.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.296+0000 7f62d7c6c700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d019f700 con 0x7f62d0107a60 2023-07-18T03:07:01.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.296+0000 7f62d7c6c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d019f840 con 0x7f62d0074540 2023-07-18T03:07:01.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.296+0000 7f62d7c6c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d019f9d0 con 0x7f62d0072f70 2023-07-18T03:07:01.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.296+0000 7f62d6209700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f62d0107a60 0x7f62d019f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:01.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.296+0000 7f62d6209700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f62d0107a60 0x7f62d019f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:39800/0 (socket says 172.21.15.101:39800) 2023-07-18T03:07:01.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.296+0000 7f62d6209700 1 -- 172.21.15.101:0/3822657696 learned_addr learned my addr 172.21.15.101:0/3822657696 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:01.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.296+0000 7f62d5a08700 1 --2- 172.21.15.101:0/3822657696 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62d0072f70 0x7f62d0197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:01.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.297+0000 7f62d5207700 1 --2- 172.21.15.101:0/3822657696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62d0074540 0x7f62d01980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:01.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.297+0000 7f62d6209700 1 -- 172.21.15.101:0/3822657696 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62d0072f70 msgr2=0x7f62d0197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:01.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.297+0000 7f62d6209700 1 --2- 172.21.15.101:0/3822657696 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62d0072f70 0x7f62d0197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:01.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.297+0000 7f62d6209700 1 -- 172.21.15.101:0/3822657696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62d0074540 msgr2=0x7f62d01980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:01.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.297+0000 7f62d6209700 1 --2- 172.21.15.101:0/3822657696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62d0074540 0x7f62d01980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:01.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.297+0000 7f62d6209700 1 -- 172.21.15.101:0/3822657696 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c8009d00 con 0x7f62d0107a60 2023-07-18T03:07:01.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.297+0000 7f62d6209700 1 --2- 172.21.15.101:0/3822657696 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f62d0107a60 0x7f62d019f130 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f62c8007a60 tx=0x7f62c800bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:01.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.297+0000 7f62c6ffd700 1 -- 172.21.15.101:0/3822657696 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c800e040 con 0x7f62d0107a60 2023-07-18T03:07:01.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.297+0000 7f62d7c6c700 1 -- 172.21.15.101:0/3822657696 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62d019fc50 con 0x7f62d0107a60 2023-07-18T03:07:01.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.297+0000 7f62c6ffd700 1 -- 172.21.15.101:0/3822657696 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62c8004c50 con 0x7f62d0107a60 2023-07-18T03:07:01.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.297+0000 7f62c6ffd700 1 -- 172.21.15.101:0/3822657696 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c80052a0 con 0x7f62d0107a60 2023-07-18T03:07:01.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.297+0000 7f62d7c6c700 1 -- 172.21.15.101:0/3822657696 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d01a00c0 con 0x7f62d0107a60 2023-07-18T03:07:01.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.299+0000 7f62d7c6c700 1 -- 172.21.15.101:0/3822657696 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62d0106110 con 0x7f62d0107a60 2023-07-18T03:07:01.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.300+0000 7f62c6ffd700 1 -- 172.21.15.101:0/3822657696 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f62c801d470 con 0x7f62d0107a60 2023-07-18T03:07:01.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.302+0000 7f62c6ffd700 1 --2- 172.21.15.101:0/3822657696 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f62bc0a1150 0x7f62bc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:01.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.302+0000 7f62c6ffd700 1 -- 172.21.15.101:0/3822657696 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f62c80ce180 con 0x7f62d0107a60 2023-07-18T03:07:01.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.302+0000 7f62d5a08700 1 --2- 172.21.15.101:0/3822657696 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f62bc0a1150 0x7f62bc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:01.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.302+0000 7f62d5a08700 1 --2- 172.21.15.101:0/3822657696 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f62bc0a1150 0x7f62bc0a3600 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f62cc009c60 tx=0x7f62cc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:01.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.306+0000 7f62c6ffd700 1 -- 172.21.15.101:0/3822657696 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f62c8097590 con 0x7f62d0107a60 2023-07-18T03:07:01.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.501+0000 7f62d7c6c700 1 -- 172.21.15.101:0/3822657696 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f62d002c850 con 0x7f62d0107a60 2023-07-18T03:07:01.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.502+0000 7f62c6ffd700 1 -- 172.21.15.101:0/3822657696 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v91) v1 ==== 56+0+5714 (secure 0 0 0) 0x7f62c8095e60 con 0x7f62d0107a60 2023-07-18T03:07:01.490 INFO:teuthology.orchestra.run.smithi101.stdout:epoch 91 2023-07-18T03:07:01.490 INFO:teuthology.orchestra.run.smithi101.stdout:fsid b1782daa-2516-11ee-9b34-001a4aab830c 2023-07-18T03:07:01.490 INFO:teuthology.orchestra.run.smithi101.stdout:created 2023-07-18T02:57:22.972483+0000 2023-07-18T03:07:01.490 INFO:teuthology.orchestra.run.smithi101.stdout:modified 2023-07-18T03:06:24.250777+0000 2023-07-18T03:07:01.490 INFO:teuthology.orchestra.run.smithi101.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-07-18T03:07:01.490 INFO:teuthology.orchestra.run.smithi101.stdout:crush_version 27 2023-07-18T03:07:01.491 INFO:teuthology.orchestra.run.smithi101.stdout:full_ratio 0.95 2023-07-18T03:07:01.491 INFO:teuthology.orchestra.run.smithi101.stdout:backfillfull_ratio 0.9 2023-07-18T03:07:01.491 INFO:teuthology.orchestra.run.smithi101.stdout:nearfull_ratio 0.85 2023-07-18T03:07:01.491 INFO:teuthology.orchestra.run.smithi101.stdout:require_min_compat_client luminous 2023-07-18T03:07:01.491 INFO:teuthology.orchestra.run.smithi101.stdout:min_compat_client luminous 2023-07-18T03:07:01.491 INFO:teuthology.orchestra.run.smithi101.stdout:require_osd_release reef 2023-07-18T03:07:01.491 INFO:teuthology.orchestra.run.smithi101.stdout:stretch_mode_enabled false 2023-07-18T03:07:01.491 INFO:teuthology.orchestra.run.smithi101.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-07-18T03:07:01.492 INFO:teuthology.orchestra.run.smithi101.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2023-07-18T03:07:01.492 INFO:teuthology.orchestra.run.smithi101.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 83 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2023-07-18T03:07:01.492 INFO:teuthology.orchestra.run.smithi101.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 87 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2023-07-18T03:07:01.492 INFO:teuthology.orchestra.run.smithi101.stdout:max_osd 12 2023-07-18T03:07:01.492 INFO:teuthology.orchestra.run.smithi101.stdout:osd.0 up in weight 1 up_from 8 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.101:6802/3610719929,v1:172.21.15.101:6803/3610719929] [v2:172.21.15.101:6804/3610719929,v1:172.21.15.101:6805/3610719929] exists,up dc76b5d1-38a0-4f1a-9335-756bfc40a62e 2023-07-18T03:07:01.492 INFO:teuthology.orchestra.run.smithi101.stdout:osd.1 up in weight 1 up_from 13 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.155:6800/1030229579,v1:172.21.15.155:6801/1030229579] [v2:172.21.15.155:6802/1030229579,v1:172.21.15.155:6803/1030229579] exists,up 2a299573-0b34-4af9-aaae-b523e5058ee5 2023-07-18T03:07:01.492 INFO:teuthology.orchestra.run.smithi101.stdout:osd.2 up in weight 1 up_from 18 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.179:6800/1349487507,v1:172.21.15.179:6801/1349487507] [v2:172.21.15.179:6802/1349487507,v1:172.21.15.179:6803/1349487507] exists,up 123b3749-c636-48fa-9e33-6229b5575dde 2023-07-18T03:07:01.493 INFO:teuthology.orchestra.run.smithi101.stdout:osd.3 up in weight 1 up_from 25 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.101:6810/4226577463,v1:172.21.15.101:6811/4226577463] [v2:172.21.15.101:6812/4226577463,v1:172.21.15.101:6813/4226577463] exists,up 4b0c409d-2954-4127-833a-b196bf226d5f 2023-07-18T03:07:01.493 INFO:teuthology.orchestra.run.smithi101.stdout:osd.4 up in weight 1 up_from 33 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.155:6808/1951135805,v1:172.21.15.155:6809/1951135805] [v2:172.21.15.155:6810/1951135805,v1:172.21.15.155:6811/1951135805] exists,up 2137f2df-984e-45ce-baf6-b4ac12c7e3c8 2023-07-18T03:07:01.493 INFO:teuthology.orchestra.run.smithi101.stdout:osd.5 up in weight 1 up_from 38 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.179:6808/2746037021,v1:172.21.15.179:6809/2746037021] [v2:172.21.15.179:6810/2746037021,v1:172.21.15.179:6811/2746037021] exists,up 1020bfc5-8c2f-482e-9588-66c7f98e46e3 2023-07-18T03:07:01.493 INFO:teuthology.orchestra.run.smithi101.stdout:osd.6 up in weight 1 up_from 46 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.101:6818/958044931,v1:172.21.15.101:6819/958044931] [v2:172.21.15.101:6820/958044931,v1:172.21.15.101:6821/958044931] exists,up 2cd99260-5c36-4603-9338-cf697970ce58 2023-07-18T03:07:01.493 INFO:teuthology.orchestra.run.smithi101.stdout:osd.7 up in weight 1 up_from 51 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.155:6816/3256322291,v1:172.21.15.155:6817/3256322291] [v2:172.21.15.155:6818/3256322291,v1:172.21.15.155:6819/3256322291] exists,up ed23fd0a-678f-4e15-b42c-cfe873315714 2023-07-18T03:07:01.493 INFO:teuthology.orchestra.run.smithi101.stdout:osd.8 up in weight 1 up_from 58 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.179:6816/3520612559,v1:172.21.15.179:6817/3520612559] [v2:172.21.15.179:6818/3520612559,v1:172.21.15.179:6819/3520612559] exists,up eca3d8e6-7baf-47a5-b991-328001c7ebef 2023-07-18T03:07:01.493 INFO:teuthology.orchestra.run.smithi101.stdout:osd.9 up in weight 1 up_from 66 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.101:6826/2166787667,v1:172.21.15.101:6827/2166787667] [v2:172.21.15.101:6828/2166787667,v1:172.21.15.101:6829/2166787667] exists,up 465a7ffb-67e4-49ef-812c-1b96feacfebf 2023-07-18T03:07:01.494 INFO:teuthology.orchestra.run.smithi101.stdout:osd.10 up in weight 1 up_from 71 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.155:6824/3682446349,v1:172.21.15.155:6825/3682446349] [v2:172.21.15.155:6826/3682446349,v1:172.21.15.155:6827/3682446349] exists,up 1897a027-f25d-434f-8df5-5fc2bd866578 2023-07-18T03:07:01.494 INFO:teuthology.orchestra.run.smithi101.stdout:osd.11 up in weight 1 up_from 78 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.179:6824/2602640858,v1:172.21.15.179:6825/2602640858] [v2:172.21.15.179:6826/2602640858,v1:172.21.15.179:6827/2602640858] exists,up b62effaf-45e2-4585-b8d0-db448f3474ea 2023-07-18T03:07:01.494 INFO:teuthology.orchestra.run.smithi101.stdout:pg_upmap_items 2.0 [8,4] 2023-07-18T03:07:01.494 INFO:teuthology.orchestra.run.smithi101.stdout:pg_upmap_items 2.4 [2,11] 2023-07-18T03:07:01.494 INFO:teuthology.orchestra.run.smithi101.stdout:pg_upmap_items 2.5 [8,11] 2023-07-18T03:07:01.494 INFO:teuthology.orchestra.run.smithi101.stdout:pg_upmap_items 2.16 [2,4] 2023-07-18T03:07:01.494 INFO:teuthology.orchestra.run.smithi101.stdout:pg_upmap_items 3.3 [4,9] 2023-07-18T03:07:01.494 INFO:teuthology.orchestra.run.smithi101.stdout:pg_upmap_items 3.5 [3,9] 2023-07-18T03:07:01.495 INFO:teuthology.orchestra.run.smithi101.stdout:pg_upmap_items 3.e [4,8] 2023-07-18T03:07:01.495 INFO:teuthology.orchestra.run.smithi101.stdout:pg_upmap_items 3.17 [4,6,5,7] 2023-07-18T03:07:01.495 INFO:teuthology.orchestra.run.smithi101.stdout:pg_upmap_items 3.1b [5,9] 2023-07-18T03:07:01.495 INFO:teuthology.orchestra.run.smithi101.stdout:pg_upmap_items 4.2 [9,1] 2023-07-18T03:07:01.495 INFO:teuthology.orchestra.run.smithi101.stdout:pg_upmap_items 4.10 [9,8] 2023-07-18T03:07:01.495 INFO:teuthology.orchestra.run.smithi101.stdout:pg_upmap_items 4.12 [9,10] 2023-07-18T03:07:01.495 INFO:teuthology.orchestra.run.smithi101.stdout:pg_upmap_items 4.1d [5,8] 2023-07-18T03:07:01.495 INFO:teuthology.orchestra.run.smithi101.stdout:blocklist 172.21.15.101:6800/2461037266 expires 2023-07-19T02:58:22.762480+0000 2023-07-18T03:07:01.496 INFO:teuthology.orchestra.run.smithi101.stdout:blocklist 172.21.15.101:6801/2461037266 expires 2023-07-19T02:58:22.762480+0000 2023-07-18T03:07:01.496 INFO:teuthology.orchestra.run.smithi101.stdout:blocklist 172.21.15.101:0/604228291 expires 2023-07-19T02:58:22.762480+0000 2023-07-18T03:07:01.496 INFO:teuthology.orchestra.run.smithi101.stdout:blocklist 172.21.15.101:0/4055187849 expires 2023-07-19T02:58:22.762480+0000 2023-07-18T03:07:01.496 INFO:teuthology.orchestra.run.smithi101.stdout:blocklist 172.21.15.101:0/2844574324 expires 2023-07-19T02:57:56.243200+0000 2023-07-18T03:07:01.496 INFO:teuthology.orchestra.run.smithi101.stdout:blocklist 172.21.15.101:0/1603248215 expires 2023-07-19T02:57:56.243200+0000 2023-07-18T03:07:01.496 INFO:teuthology.orchestra.run.smithi101.stdout:blocklist 172.21.15.101:6801/3497914514 expires 2023-07-19T02:57:56.243200+0000 2023-07-18T03:07:01.496 INFO:teuthology.orchestra.run.smithi101.stdout:blocklist 172.21.15.101:6800/3497914514 expires 2023-07-19T02:57:56.243200+0000 2023-07-18T03:07:01.496 INFO:teuthology.orchestra.run.smithi101.stdout:blocklist 172.21.15.101:0/3307578182 expires 2023-07-19T02:58:22.762480+0000 2023-07-18T03:07:01.497 INFO:teuthology.orchestra.run.smithi101.stdout:blocklist 172.21.15.101:0/373189792 expires 2023-07-19T02:57:56.243200+0000 2023-07-18T03:07:01.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.504+0000 7f62d7c6c700 1 -- 172.21.15.101:0/3822657696 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f62bc0a1150 msgr2=0x7f62bc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:01.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.504+0000 7f62d7c6c700 1 --2- 172.21.15.101:0/3822657696 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f62bc0a1150 0x7f62bc0a3600 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f62cc009c60 tx=0x7f62cc009260 comp rx=0 tx=0).stop 2023-07-18T03:07:01.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.504+0000 7f62d7c6c700 1 -- 172.21.15.101:0/3822657696 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f62d0107a60 msgr2=0x7f62d019f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:01.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.504+0000 7f62d7c6c700 1 --2- 172.21.15.101:0/3822657696 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f62d0107a60 0x7f62d019f130 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f62c8007a60 tx=0x7f62c800bab0 comp rx=0 tx=0).stop 2023-07-18T03:07:01.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.505+0000 7f62d7c6c700 1 -- 172.21.15.101:0/3822657696 shutdown_connections 2023-07-18T03:07:01.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.505+0000 7f62d7c6c700 1 --2- 172.21.15.101:0/3822657696 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f62bc0a1150 0x7f62bc0a3600 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:01.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.505+0000 7f62d7c6c700 1 --2- 172.21.15.101:0/3822657696 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f62d0107a60 0x7f62d019f130 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:01.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.505+0000 7f62d7c6c700 1 --2- 172.21.15.101:0/3822657696 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62d0072f70 0x7f62d0197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:01.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.505+0000 7f62d7c6c700 1 --2- 172.21.15.101:0/3822657696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62d0074540 0x7f62d01980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:01.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.505+0000 7f62d7c6c700 1 -- 172.21.15.101:0/3822657696 >> 172.21.15.101:0/3822657696 conn(0x7f62d00fb1a0 msgr2=0x7f62d00fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:01.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.505+0000 7f62d7c6c700 1 -- 172.21.15.101:0/3822657696 shutdown_connections 2023-07-18T03:07:01.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:01.505+0000 7f62d7c6c700 1 -- 172.21.15.101:0/3822657696 wait complete. 2023-07-18T03:07:02.045 INFO:teuthology.run_tasks:Running task fs.ready... 2023-07-18T03:07:02.056 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:07:02.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:01 smithi101 ceph-mon[126379]: pgmap v340: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2023-07-18T03:07:02.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:01 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3822657696' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-07-18T03:07:02.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.270+0000 7f01837ff700 1 -- 172.21.15.101:0/4129867493 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f017c101680 msgr2=0x7f017c107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:02.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.270+0000 7f01837ff700 1 --2- 172.21.15.101:0/4129867493 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f017c101680 0x7f017c107f40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f017400b0a0 tx=0x7f017400b3b0 comp rx=0 tx=0).stop 2023-07-18T03:07:02.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.270+0000 7f01837ff700 1 -- 172.21.15.101:0/4129867493 shutdown_connections 2023-07-18T03:07:02.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.270+0000 7f01837ff700 1 --2- 172.21.15.101:0/4129867493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f017c0ffb00 0x7f017c0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:02.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.270+0000 7f01837ff700 1 --2- 172.21.15.101:0/4129867493 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f017c101680 0x7f017c107f40 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:02.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.270+0000 7f01837ff700 1 --2- 172.21.15.101:0/4129867493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f017c100cf0 0x7f017c101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:02.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.270+0000 7f01837ff700 1 -- 172.21.15.101:0/4129867493 >> 172.21.15.101:0/4129867493 conn(0x7f017c0fb0e0 msgr2=0x7f017c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:02.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.270+0000 7f01837ff700 1 -- 172.21.15.101:0/4129867493 shutdown_connections 2023-07-18T03:07:02.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.270+0000 7f01837ff700 1 -- 172.21.15.101:0/4129867493 wait complete. 2023-07-18T03:07:02.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.271+0000 7f01837ff700 1 Processor -- start 2023-07-18T03:07:02.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.271+0000 7f01837ff700 1 -- start start 2023-07-18T03:07:02.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.272+0000 7f01837ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f017c0ffb00 0x7f017c197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:02.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.272+0000 7f01837ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f017c100cf0 0x7f017c198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:02.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.272+0000 7f01837ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f017c101680 0x7f017c19f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:02.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.272+0000 7f01837ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017c19f660 con 0x7f017c101680 2023-07-18T03:07:02.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.272+0000 7f01837ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017c19f7a0 con 0x7f017c100cf0 2023-07-18T03:07:02.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.272+0000 7f01837ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017c19f930 con 0x7f017c0ffb00 2023-07-18T03:07:02.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.272+0000 7f0181d9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f017c101680 0x7f017c19f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:02.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.272+0000 7f0181d9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f017c101680 0x7f017c19f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:39822/0 (socket says 172.21.15.101:39822) 2023-07-18T03:07:02.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.272+0000 7f0181d9c700 1 -- 172.21.15.101:0/1344789818 learned_addr learned my addr 172.21.15.101:0/1344789818 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:02.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.272+0000 7f0180d9a700 1 --2- 172.21.15.101:0/1344789818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f017c100cf0 0x7f017c198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:02.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.272+0000 7f018159b700 1 --2- 172.21.15.101:0/1344789818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f017c0ffb00 0x7f017c197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:02.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.272+0000 7f0181d9c700 1 -- 172.21.15.101:0/1344789818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f017c0ffb00 msgr2=0x7f017c197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:02.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.272+0000 7f0181d9c700 1 --2- 172.21.15.101:0/1344789818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f017c0ffb00 0x7f017c197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:02.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.272+0000 7f0181d9c700 1 -- 172.21.15.101:0/1344789818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f017c100cf0 msgr2=0x7f017c198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:02.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.272+0000 7f0181d9c700 1 --2- 172.21.15.101:0/1344789818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f017c100cf0 0x7f017c198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:02.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.272+0000 7f0181d9c700 1 -- 172.21.15.101:0/1344789818 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0174009d00 con 0x7f017c101680 2023-07-18T03:07:02.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.272+0000 7f018159b700 1 --2- 172.21.15.101:0/1344789818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f017c0ffb00 0x7f017c197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:07:02.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.273+0000 7f0181d9c700 1 --2- 172.21.15.101:0/1344789818 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f017c101680 0x7f017c19f090 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f017400d010 tx=0x7f017400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:02.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.273+0000 7f01727fc700 1 -- 172.21.15.101:0/1344789818 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017400e040 con 0x7f017c101680 2023-07-18T03:07:02.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.273+0000 7f01727fc700 1 -- 172.21.15.101:0/1344789818 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0174003e10 con 0x7f017c101680 2023-07-18T03:07:02.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.273+0000 7f01837ff700 1 -- 172.21.15.101:0/1344789818 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f017c19fbb0 con 0x7f017c101680 2023-07-18T03:07:02.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.273+0000 7f01727fc700 1 -- 172.21.15.101:0/1344789818 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01740052a0 con 0x7f017c101680 2023-07-18T03:07:02.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.273+0000 7f01837ff700 1 -- 172.21.15.101:0/1344789818 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f017c19ffd0 con 0x7f017c101680 2023-07-18T03:07:02.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.274+0000 7f01837ff700 1 -- 172.21.15.101:0/1344789818 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f017c10b3a0 con 0x7f017c101680 2023-07-18T03:07:02.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.276+0000 7f01727fc700 1 -- 172.21.15.101:0/1344789818 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f0174005400 con 0x7f017c101680 2023-07-18T03:07:02.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.277+0000 7f01727fc700 1 --2- 172.21.15.101:0/1344789818 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f01680a1220 0x7f01680a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:02.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.277+0000 7f01727fc700 1 -- 172.21.15.101:0/1344789818 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f01740ce090 con 0x7f017c101680 2023-07-18T03:07:02.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.277+0000 7f018159b700 1 --2- 172.21.15.101:0/1344789818 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f01680a1220 0x7f01680a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:02.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.278+0000 7f018159b700 1 --2- 172.21.15.101:0/1344789818 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f01680a1220 0x7f01680a36d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f017c100b50 tx=0x7f0178009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:02.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.281+0000 7f01727fc700 1 -- 172.21.15.101:0/1344789818 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f01740974a0 con 0x7f017c101680 2023-07-18T03:07:02.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:01 smithi155 ceph-mon[128885]: pgmap v340: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2023-07-18T03:07:02.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:01 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/3822657696' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-07-18T03:07:02.444 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:01 smithi179 ceph-mon[129229]: pgmap v340: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2023-07-18T03:07:02.444 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:01 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/3822657696' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-07-18T03:07:02.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.480+0000 7f01837ff700 1 -- 172.21.15.101:0/1344789818 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f017c1a0580 con 0x7f017c101680 2023-07-18T03:07:02.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.482+0000 7f01727fc700 1 -- 172.21.15.101:0/1344789818 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24725 (secure 0 0 0) 0x7f017409aac0 con 0x7f017c101680 2023-07-18T03:07:02.470 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:07:02.471 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":91,"fsid":"b1782daa-2516-11ee-9b34-001a4aab830c","created":"2023-07-18T02:57:22.972483+0000","modified":"2023-07-18T03:06:24.250777+0000","last_up_change":"2023-07-18T03:04:11.429530+0000","last_in_change":"2023-07-18T03:03:54.158340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:00:43.353489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:04:57.771616+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-18T03:04:58.769590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-18T03:05:02.358830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc76b5d1-38a0-4f1a-9335-756bfc40a62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6803","nonce":3610719929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6805","nonce":3610719929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6809","nonce":3610719929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6807","nonce":3610719929}]},"public_addr":"172.21.15.101:6803/3610719929","cluster_addr":"172.21.15.101:6805/3610719929","heartbeat_back_addr":"172.21.15.101:6809/3610719929","heartbeat_front_addr":"172.21.15.101:6807/3610719929","state":["exists","up"]},{"osd":1,"uuid":"2a299573-0b34-4af9-aaae-b523e5058ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6801","nonce":1030229579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6803","nonce":1030229579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6807","nonce":1030229579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6805","nonce":1030229579}]},"public_addr":"172.21.15.155:6801/1030229579","cluster_addr":"172.21.15.155:6803/1030229579","heartbeat_back_addr":"172.21.15.155:6807/1030229579","heartbeat_front_addr":"172.21.15.155:6805/1030229579","state":["exists","up"]},{"osd":2,"uuid":"123b3749-c636-48fa-9e33-6229b5575dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6801","nonce":1349487507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6803","nonce":1349487507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6807","nonce":1349487507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6805","nonce":1349487507}]},"public_addr":"172.21.15.179:6801/1349487507","cluster_addr":"172.21.15.179:6803/1349487507","heartbeat_back_addr":"172.21.15.179:6807/1349487507","heartbeat_front_addr":"172.21.15.179:6805/1349487507","state":["exists","up"]},{"osd":3,"uuid":"4b0c409d-2954-4127-833a-b196bf226d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6811","nonce":4226577463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6813","nonce":4226577463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6817","nonce":4226577463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6815","nonce":4226577463}]},"public_addr":"172.21.15.101:6811/4226577463","cluster_addr":"172.21.15.101:6813/4226577463","heartbeat_back_addr":"172.21.15.101:6817/4226577463","heartbeat_front_addr":"172.21.15.101:6815/4226577463","state":["exists","up"]},{"osd":4,"uuid":"2137f2df-984e-45ce-baf6-b4ac12c7e3c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6809","nonce":1951135805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6811","nonce":1951135805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6815","nonce":1951135805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6813","nonce":1951135805}]},"public_addr":"172.21.15.155:6809/1951135805","cluster_addr":"172.21.15.155:6811/1951135805","heartbeat_back_addr":"172.21.15.155:6815/1951135805","heartbeat_front_addr":"172.21.15.155:6813/1951135805","state":["exists","up"]},{"osd":5,"uuid":"1020bfc5-8c2f-482e-9588-66c7f98e46e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6809","nonce":2746037021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6811","nonce":2746037021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6815","nonce":2746037021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6813","nonce":2746037021}]},"public_addr":"172.21.15.179:6809/2746037021","cluster_addr":"172.21.15.179:6811/2746037021","heartbeat_back_addr":"172.21.15.179:6815/2746037021","heartbeat_front_addr":"172.21.15.179:6813/2746037021","state":["exists","up"]},{"osd":6,"uuid":"2cd99260-5c36-4603-9338-cf697970ce58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6819","nonce":958044931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6821","nonce":958044931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6825","nonce":958044931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6823","nonce":958044931}]},"public_addr":"172.21.15.101:6819/958044931","cluster_addr":"172.21.15.101:6821/958044931","heartbeat_back_addr":"172.21.15.101:6825/958044931","heartbeat_front_addr":"172.21.15.101:6823/958044931","state":["exists","up"]},{"osd":7,"uuid":"ed23fd0a-678f-4e15-b42c-cfe873315714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6817","nonce":3256322291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6819","nonce":3256322291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6823","nonce":3256322291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6821","nonce":3256322291}]},"public_addr":"172.21.15.155:6817/3256322291","cluster_addr":"172.21.15.155:6819/3256322291","heartbeat_back_addr":"172.21.15.155:6823/3256322291","heartbeat_front_addr":"172.21.15.155:6821/3256322291","state":["exists","up"]},{"osd":8,"uuid":"eca3d8e6-7baf-47a5-b991-328001c7ebef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6817","nonce":3520612559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6819","nonce":3520612559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6823","nonce":3520612559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6821","nonce":3520612559}]},"public_addr":"172.21.15.179:6817/3520612559","cluster_addr":"172.21.15.179:6819/3520612559","heartbeat_back_addr":"172.21.15.179:6823/3520612559","heartbeat_front_addr":"172.21.15.179:6821/3520612559","state":["exists","up"]},{"osd":9,"uuid":"465a7ffb-67e4-49ef-812c-1b96feacfebf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6827","nonce":2166787667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6829","nonce":2166787667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6833","nonce":2166787667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6831","nonce":2166787667}]},"public_addr":"172.21.15.101:6827/2166787667","cluster_addr":"172.21.15.101:6829/2166787667","heartbeat_back_addr":"172.21.15.101:6833/2166787667","heartbeat_front_addr":"172.21.15.101:6831/2166787667","state":["exists","up"]},{"osd":10,"uuid":"1897a027-f25d-434f-8df5-5fc2bd866578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6825","nonce":3682446349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6827","nonce":3682446349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6831","nonce":3682446349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6829","nonce":3682446349}]},"public_addr":"172.21.15.155:6825/3682446349","cluster_addr":"172.21.15.155:6827/3682446349","heartbeat_back_addr":"172.21.15.155:6831/3682446349","heartbeat_front_addr":"172.21.15.155:6829/3682446349","state":["exists","up"]},{"osd":11,"uuid":"b62effaf-45e2-4585-b8d0-db448f3474ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6825","nonce":2602640858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6827","nonce":2602640858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6831","nonce":2602640858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6829","nonce":2602640858}]},"public_addr":"172.21.15.179:6825/2602640858","cluster_addr":"172.21.15.179:6827/2602640858","heartbeat_back_addr":"172.21.15.179:6831/2602640858","heartbeat_front_addr":"172.21.15.179:6829/2602640858","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T02:59:56.074519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:19.358142+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:41.128966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:08.227054+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:31.442592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:52.751685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:19.251054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:40.698241+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:01.339150+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:27.955910+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:49.136980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:04:10.084982+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.101:6800/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:6801/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/604228291":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/4055187849":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/2844574324":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/1603248215":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6801/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6800/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/3307578182":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/373189792":"2023-07-19T02:57:56.243200+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:07:02.472 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.484+0000 7f01837ff700 1 -- 172.21.15.101:0/1344789818 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f01680a1220 msgr2=0x7f01680a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:02.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.484+0000 7f01837ff700 1 --2- 172.21.15.101:0/1344789818 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f01680a1220 0x7f01680a36d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f017c100b50 tx=0x7f0178009260 comp rx=0 tx=0).stop 2023-07-18T03:07:02.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.484+0000 7f01837ff700 1 -- 172.21.15.101:0/1344789818 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f017c101680 msgr2=0x7f017c19f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:02.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.484+0000 7f01837ff700 1 --2- 172.21.15.101:0/1344789818 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f017c101680 0x7f017c19f090 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f017400d010 tx=0x7f017400bc80 comp rx=0 tx=0).stop 2023-07-18T03:07:02.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.485+0000 7f01837ff700 1 -- 172.21.15.101:0/1344789818 shutdown_connections 2023-07-18T03:07:02.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.485+0000 7f01837ff700 1 --2- 172.21.15.101:0/1344789818 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f01680a1220 0x7f01680a36d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:02.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.485+0000 7f01837ff700 1 --2- 172.21.15.101:0/1344789818 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f017c101680 0x7f017c19f090 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:02.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.485+0000 7f01837ff700 1 --2- 172.21.15.101:0/1344789818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f017c0ffb00 0x7f017c197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:02.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.485+0000 7f01837ff700 1 --2- 172.21.15.101:0/1344789818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f017c100cf0 0x7f017c198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:02.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.485+0000 7f01837ff700 1 -- 172.21.15.101:0/1344789818 >> 172.21.15.101:0/1344789818 conn(0x7f017c0fb0e0 msgr2=0x7f017c1064f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:02.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.485+0000 7f01837ff700 1 -- 172.21.15.101:0/1344789818 shutdown_connections 2023-07-18T03:07:02.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.485+0000 7f01837ff700 1 -- 172.21.15.101:0/1344789818 wait complete. 2023-07-18T03:07:02.491 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:00:43.353489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-18T03:04:57.771616+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-18T03:04:58.769590+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-18T03:05:02.358830+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-18T03:07:02.492 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-18T03:07:02.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.705+0000 7f8dfc3ff700 1 -- 172.21.15.101:0/4254489379 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8df4072f70 msgr2=0x7f8df40733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:02.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.705+0000 7f8dfc3ff700 1 --2- 172.21.15.101:0/4254489379 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8df4072f70 0x7f8df40733e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8de4009990 tx=0x7f8de4009ca0 comp rx=0 tx=0).stop 2023-07-18T03:07:02.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.705+0000 7f8dfc3ff700 1 -- 172.21.15.101:0/4254489379 shutdown_connections 2023-07-18T03:07:02.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.705+0000 7f8dfc3ff700 1 --2- 172.21.15.101:0/4254489379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8df4074550 0x7f8df40729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:02.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.705+0000 7f8dfc3ff700 1 --2- 172.21.15.101:0/4254489379 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8df41079a0 0x7f8df4109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:02.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.705+0000 7f8dfc3ff700 1 --2- 172.21.15.101:0/4254489379 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8df4072f70 0x7f8df40733e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:02.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.705+0000 7f8dfc3ff700 1 -- 172.21.15.101:0/4254489379 >> 172.21.15.101:0/4254489379 conn(0x7f8df40fb0e0 msgr2=0x7f8df40fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:02.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.705+0000 7f8dfc3ff700 1 -- 172.21.15.101:0/4254489379 shutdown_connections 2023-07-18T03:07:02.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.705+0000 7f8dfc3ff700 1 -- 172.21.15.101:0/4254489379 wait complete. 2023-07-18T03:07:02.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.706+0000 7f8dfc3ff700 1 Processor -- start 2023-07-18T03:07:02.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.706+0000 7f8dfc3ff700 1 -- start start 2023-07-18T03:07:02.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.706+0000 7f8dfc3ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8df4072f70 0x7f8df4197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:02.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.706+0000 7f8dfc3ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8df4074550 0x7f8df4197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:02.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.706+0000 7f8dfc3ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8df41079a0 0x7f8df419f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:02.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.706+0000 7f8dfc3ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df419f5e0 con 0x7f8df4072f70 2023-07-18T03:07:02.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.706+0000 7f8dfc3ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df419f720 con 0x7f8df4074550 2023-07-18T03:07:02.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.706+0000 7f8dfc3ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df419f8b0 con 0x7f8df41079a0 2023-07-18T03:07:02.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.707+0000 7f8dfa19b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8df4072f70 0x7f8df4197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:02.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.707+0000 7f8dfa19b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8df4072f70 0x7f8df4197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:39850/0 (socket says 172.21.15.101:39850) 2023-07-18T03:07:02.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.707+0000 7f8dfa19b700 1 -- 172.21.15.101:0/1562994145 learned_addr learned my addr 172.21.15.101:0/1562994145 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:02.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.707+0000 7f8dfa99c700 1 --2- 172.21.15.101:0/1562994145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8df41079a0 0x7f8df419f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:02.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.707+0000 7f8df999a700 1 --2- 172.21.15.101:0/1562994145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8df4074550 0x7f8df4197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:02.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.707+0000 7f8dfa19b700 1 -- 172.21.15.101:0/1562994145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8df41079a0 msgr2=0x7f8df419f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:02.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.707+0000 7f8dfa19b700 1 --2- 172.21.15.101:0/1562994145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8df41079a0 0x7f8df419f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:02.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.707+0000 7f8dfa19b700 1 -- 172.21.15.101:0/1562994145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8df4074550 msgr2=0x7f8df4197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:02.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.707+0000 7f8dfa19b700 1 --2- 172.21.15.101:0/1562994145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8df4074550 0x7f8df4197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:02.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.707+0000 7f8dfa19b700 1 -- 172.21.15.101:0/1562994145 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de40095f0 con 0x7f8df4072f70 2023-07-18T03:07:02.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.707+0000 7f8dfa19b700 1 --2- 172.21.15.101:0/1562994145 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8df4072f70 0x7f8df4197a60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8df000e970 tx=0x7f8df000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:02.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.707+0000 7f8deb7fe700 1 -- 172.21.15.101:0/1562994145 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df0009790 con 0x7f8df4072f70 2023-07-18T03:07:02.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.708+0000 7f8dfc3ff700 1 -- 172.21.15.101:0/1562994145 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8df419fb90 con 0x7f8df4072f70 2023-07-18T03:07:02.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.708+0000 7f8deb7fe700 1 -- 172.21.15.101:0/1562994145 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8df000c7e0 con 0x7f8df4072f70 2023-07-18T03:07:02.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.708+0000 7f8deb7fe700 1 -- 172.21.15.101:0/1562994145 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df00054e0 con 0x7f8df4072f70 2023-07-18T03:07:02.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.708+0000 7f8dfc3ff700 1 -- 172.21.15.101:0/1562994145 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df41a00f0 con 0x7f8df4072f70 2023-07-18T03:07:02.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.709+0000 7f8dfc3ff700 1 -- 172.21.15.101:0/1562994145 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8df4106050 con 0x7f8df4072f70 2023-07-18T03:07:02.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.710+0000 7f8deb7fe700 1 -- 172.21.15.101:0/1562994145 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f8df000c950 con 0x7f8df4072f70 2023-07-18T03:07:02.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.712+0000 7f8deb7fe700 1 --2- 172.21.15.101:0/1562994145 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8de00a11d0 0x7f8de00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:02.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.712+0000 7f8deb7fe700 1 -- 172.21.15.101:0/1562994145 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f8df0014070 con 0x7f8df4072f70 2023-07-18T03:07:02.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.712+0000 7f8df999a700 1 --2- 172.21.15.101:0/1562994145 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8de00a11d0 0x7f8de00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:02.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.713+0000 7f8df999a700 1 --2- 172.21.15.101:0/1562994145 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8de00a11d0 0x7f8de00a3680 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8de400b680 tx=0x7f8de4005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:02.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.715+0000 7f8deb7fe700 1 -- 172.21.15.101:0/1562994145 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8df0096240 con 0x7f8df4072f70 2023-07-18T03:07:02.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.915+0000 7f8dfc3ff700 1 -- 172.21.15.101:0/1562994145 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8df4061220 con 0x7f8df4072f70 2023-07-18T03:07:02.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.916+0000 7f8deb7fe700 1 -- 172.21.15.101:0/1562994145 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8df0099860 con 0x7f8df4072f70 2023-07-18T03:07:02.904 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 1 2023-07-18T03:07:02.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.918+0000 7f8dfc3ff700 1 -- 172.21.15.101:0/1562994145 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8de00a11d0 msgr2=0x7f8de00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:02.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.918+0000 7f8dfc3ff700 1 --2- 172.21.15.101:0/1562994145 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8de00a11d0 0x7f8de00a3680 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8de400b680 tx=0x7f8de4005fd0 comp rx=0 tx=0).stop 2023-07-18T03:07:02.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.918+0000 7f8dfc3ff700 1 -- 172.21.15.101:0/1562994145 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8df4072f70 msgr2=0x7f8df4197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:02.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.918+0000 7f8dfc3ff700 1 --2- 172.21.15.101:0/1562994145 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8df4072f70 0x7f8df4197a60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8df000e970 tx=0x7f8df000ec80 comp rx=0 tx=0).stop 2023-07-18T03:07:02.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.918+0000 7f8dfc3ff700 1 -- 172.21.15.101:0/1562994145 shutdown_connections 2023-07-18T03:07:02.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.918+0000 7f8dfc3ff700 1 --2- 172.21.15.101:0/1562994145 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8de00a11d0 0x7f8de00a3680 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:02.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.918+0000 7f8dfc3ff700 1 --2- 172.21.15.101:0/1562994145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8df41079a0 0x7f8df419f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:02.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.918+0000 7f8dfc3ff700 1 --2- 172.21.15.101:0/1562994145 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8df4072f70 0x7f8df4197a60 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:02.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.918+0000 7f8dfc3ff700 1 --2- 172.21.15.101:0/1562994145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8df4074550 0x7f8df4197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:02.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.918+0000 7f8dfc3ff700 1 -- 172.21.15.101:0/1562994145 >> 172.21.15.101:0/1562994145 conn(0x7f8df40fb0e0 msgr2=0x7f8df40fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:02.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.918+0000 7f8dfc3ff700 1 -- 172.21.15.101:0/1562994145 shutdown_connections 2023-07-18T03:07:02.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:02.919+0000 7f8dfc3ff700 1 -- 172.21.15.101:0/1562994145 wait complete. 2023-07-18T03:07:02.924 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-18T03:07:03.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.136+0000 7f29db1ff700 1 -- 172.21.15.101:0/3517853989 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f29d40ffb00 msgr2=0x7f29d40fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:03.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.136+0000 7f29db1ff700 1 --2- 172.21.15.101:0/3517853989 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f29d40ffb00 0x7f29d40fff10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f29c8009990 tx=0x7f29c8009ca0 comp rx=0 tx=0).stop 2023-07-18T03:07:03.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.136+0000 7f29db1ff700 1 -- 172.21.15.101:0/3517853989 shutdown_connections 2023-07-18T03:07:03.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.136+0000 7f29db1ff700 1 --2- 172.21.15.101:0/3517853989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29d4101680 0x7f29d4107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:03.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.136+0000 7f29db1ff700 1 --2- 172.21.15.101:0/3517853989 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f29d40ffb00 0x7f29d40fff10 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:03.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.136+0000 7f29db1ff700 1 --2- 172.21.15.101:0/3517853989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29d4100cf0 0x7f29d4101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:03.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.136+0000 7f29db1ff700 1 -- 172.21.15.101:0/3517853989 >> 172.21.15.101:0/3517853989 conn(0x7f29d40fb120 msgr2=0x7f29d40fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:03.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.136+0000 7f29db1ff700 1 -- 172.21.15.101:0/3517853989 shutdown_connections 2023-07-18T03:07:03.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.136+0000 7f29db1ff700 1 -- 172.21.15.101:0/3517853989 wait complete. 2023-07-18T03:07:03.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.137+0000 7f29db1ff700 1 Processor -- start 2023-07-18T03:07:03.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.138+0000 7f29db1ff700 1 -- start start 2023-07-18T03:07:03.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.138+0000 7f29db1ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29d40ffb00 0x7f29d4197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:03.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.138+0000 7f29db1ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29d4100cf0 0x7f29d4197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:03.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.138+0000 7f29db1ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f29d4101680 0x7f29d419f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:03.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.138+0000 7f29db1ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d419f600 con 0x7f29d4101680 2023-07-18T03:07:03.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.138+0000 7f29db1ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d419f740 con 0x7f29d40ffb00 2023-07-18T03:07:03.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.138+0000 7f29db1ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d419f8d0 con 0x7f29d4100cf0 2023-07-18T03:07:03.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.138+0000 7f29d8f9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29d40ffb00 0x7f29d4197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:03.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.138+0000 7f29d8f9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29d40ffb00 0x7f29d4197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:59710/0 (socket says 172.21.15.101:59710) 2023-07-18T03:07:03.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.138+0000 7f29d8f9b700 1 -- 172.21.15.101:0/1130890462 learned_addr learned my addr 172.21.15.101:0/1130890462 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:03.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.138+0000 7f29d979c700 1 --2- 172.21.15.101:0/1130890462 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f29d4101680 0x7f29d419f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:03.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.138+0000 7f29d3fff700 1 --2- 172.21.15.101:0/1130890462 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29d4100cf0 0x7f29d4197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:03.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.139+0000 7f29d8f9b700 1 -- 172.21.15.101:0/1130890462 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29d4100cf0 msgr2=0x7f29d4197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:03.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.139+0000 7f29d8f9b700 1 --2- 172.21.15.101:0/1130890462 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29d4100cf0 0x7f29d4197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:03.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.139+0000 7f29d8f9b700 1 -- 172.21.15.101:0/1130890462 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f29d4101680 msgr2=0x7f29d419f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:03.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.139+0000 7f29d8f9b700 1 --2- 172.21.15.101:0/1130890462 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f29d4101680 0x7f29d419f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:03.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.139+0000 7f29d8f9b700 1 -- 172.21.15.101:0/1130890462 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c80095f0 con 0x7f29d40ffb00 2023-07-18T03:07:03.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.139+0000 7f29d979c700 1 --2- 172.21.15.101:0/1130890462 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f29d4101680 0x7f29d419f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:07:03.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.139+0000 7f29d8f9b700 1 --2- 172.21.15.101:0/1130890462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29d40ffb00 0x7f29d4197a80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f29c8000c00 tx=0x7f29c8005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:03.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.139+0000 7f29d1ffb700 1 -- 172.21.15.101:0/1130890462 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c801d070 con 0x7f29d40ffb00 2023-07-18T03:07:03.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.139+0000 7f29db1ff700 1 -- 172.21.15.101:0/1130890462 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29d419fb50 con 0x7f29d40ffb00 2023-07-18T03:07:03.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.139+0000 7f29d1ffb700 1 -- 172.21.15.101:0/1130890462 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29c8003f60 con 0x7f29d40ffb00 2023-07-18T03:07:03.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.139+0000 7f29d1ffb700 1 -- 172.21.15.101:0/1130890462 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c8022b50 con 0x7f29d40ffb00 2023-07-18T03:07:03.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.139+0000 7f29db1ff700 1 -- 172.21.15.101:0/1130890462 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29d419ff80 con 0x7f29d40ffb00 2023-07-18T03:07:03.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.140+0000 7f29db1ff700 1 -- 172.21.15.101:0/1130890462 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29d41921e0 con 0x7f29d40ffb00 2023-07-18T03:07:03.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.142+0000 7f29d1ffb700 1 -- 172.21.15.101:0/1130890462 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f29c80040d0 con 0x7f29d40ffb00 2023-07-18T03:07:03.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.143+0000 7f29d1ffb700 1 --2- 172.21.15.101:0/1130890462 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f29c40a1150 0x7f29c40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:03.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.143+0000 7f29d1ffb700 1 -- 172.21.15.101:0/1130890462 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f29c80ce4a0 con 0x7f29d40ffb00 2023-07-18T03:07:03.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.144+0000 7f29d3fff700 1 --2- 172.21.15.101:0/1130890462 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f29c40a1150 0x7f29c40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:03.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.144+0000 7f29d3fff700 1 --2- 172.21.15.101:0/1130890462 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f29c40a1150 0x7f29c40a3600 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f29c0005950 tx=0x7f29c000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:03.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.146+0000 7f29d1ffb700 1 -- 172.21.15.101:0/1130890462 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f29c80978b0 con 0x7f29d40ffb00 2023-07-18T03:07:03.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:02 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1344789818' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:07:03.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:02 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1562994145' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:07:03.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:02 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1344789818' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:07:03.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:02 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1562994145' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:07:03.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.348+0000 7f29db1ff700 1 -- 172.21.15.101:0/1130890462 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f29d402c9e0 con 0x7f29d40ffb00 2023-07-18T03:07:03.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.348+0000 7f29d1ffb700 1 -- 172.21.15.101:0/1130890462 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f29c809aed0 con 0x7f29d40ffb00 2023-07-18T03:07:03.336 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 32 2023-07-18T03:07:03.338 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.351+0000 7f29db1ff700 1 -- 172.21.15.101:0/1130890462 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f29c40a1150 msgr2=0x7f29c40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:03.338 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.351+0000 7f29db1ff700 1 --2- 172.21.15.101:0/1130890462 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f29c40a1150 0x7f29c40a3600 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f29c0005950 tx=0x7f29c000b410 comp rx=0 tx=0).stop 2023-07-18T03:07:03.339 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.351+0000 7f29db1ff700 1 -- 172.21.15.101:0/1130890462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29d40ffb00 msgr2=0x7f29d4197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:03.339 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.351+0000 7f29db1ff700 1 --2- 172.21.15.101:0/1130890462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29d40ffb00 0x7f29d4197a80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f29c8000c00 tx=0x7f29c8005740 comp rx=0 tx=0).stop 2023-07-18T03:07:03.339 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.351+0000 7f29db1ff700 1 -- 172.21.15.101:0/1130890462 shutdown_connections 2023-07-18T03:07:03.339 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.351+0000 7f29db1ff700 1 --2- 172.21.15.101:0/1130890462 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f29c40a1150 0x7f29c40a3600 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:03.339 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.351+0000 7f29db1ff700 1 --2- 172.21.15.101:0/1130890462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29d40ffb00 0x7f29d4197a80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:03.339 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.351+0000 7f29db1ff700 1 --2- 172.21.15.101:0/1130890462 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f29d4101680 0x7f29d419f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:03.339 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.351+0000 7f29db1ff700 1 --2- 172.21.15.101:0/1130890462 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29d4100cf0 0x7f29d4197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:03.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.351+0000 7f29db1ff700 1 -- 172.21.15.101:0/1130890462 >> 172.21.15.101:0/1130890462 conn(0x7f29d40fb120 msgr2=0x7f29d41065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:03.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.351+0000 7f29db1ff700 1 -- 172.21.15.101:0/1130890462 shutdown_connections 2023-07-18T03:07:03.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.351+0000 7f29db1ff700 1 -- 172.21.15.101:0/1130890462 wait complete. 2023-07-18T03:07:03.357 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-18T03:07:03.444 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:02 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1344789818' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:07:03.444 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:02 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1562994145' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:07:03.555 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.568+0000 7f55835ff700 1 -- 172.21.15.101:0/3123469548 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f557c101670 msgr2=0x7f557c107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:03.555 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.568+0000 7f55835ff700 1 --2- 172.21.15.101:0/3123469548 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f557c101670 0x7f557c107f30 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f557400b0a0 tx=0x7f557400b3b0 comp rx=0 tx=0).stop 2023-07-18T03:07:03.555 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.568+0000 7f55835ff700 1 -- 172.21.15.101:0/3123469548 shutdown_connections 2023-07-18T03:07:03.556 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.568+0000 7f55835ff700 1 --2- 172.21.15.101:0/3123469548 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f557c101670 0x7f557c107f30 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:03.556 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.568+0000 7f55835ff700 1 --2- 172.21.15.101:0/3123469548 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f557c0ffaf0 0x7f557c0fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:03.556 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.568+0000 7f55835ff700 1 --2- 172.21.15.101:0/3123469548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f557c100ce0 0x7f557c101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:03.556 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.568+0000 7f55835ff700 1 -- 172.21.15.101:0/3123469548 >> 172.21.15.101:0/3123469548 conn(0x7f557c0fb130 msgr2=0x7f557c0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:03.556 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.568+0000 7f55835ff700 1 -- 172.21.15.101:0/3123469548 shutdown_connections 2023-07-18T03:07:03.556 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.568+0000 7f55835ff700 1 -- 172.21.15.101:0/3123469548 wait complete. 2023-07-18T03:07:03.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.569+0000 7f55835ff700 1 Processor -- start 2023-07-18T03:07:03.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.569+0000 7f55835ff700 1 -- start start 2023-07-18T03:07:03.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.569+0000 7f55835ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f557c0ffaf0 0x7f557c0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:03.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.569+0000 7f55835ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f557c100ce0 0x7f557c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:03.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.569+0000 7f55835ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f557c101670 0x7f557c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:03.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.569+0000 7f55835ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f557c0735e0 con 0x7f557c0ffaf0 2023-07-18T03:07:03.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.569+0000 7f55835ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f557c073750 con 0x7f557c101670 2023-07-18T03:07:03.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.569+0000 7f55835ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f557c19f950 con 0x7f557c100ce0 2023-07-18T03:07:03.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.570+0000 7f558139b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f557c0ffaf0 0x7f557c0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:03.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.570+0000 7f558139b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f557c0ffaf0 0x7f557c0742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:39870/0 (socket says 172.21.15.101:39870) 2023-07-18T03:07:03.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.570+0000 7f558139b700 1 -- 172.21.15.101:0/939792987 learned_addr learned my addr 172.21.15.101:0/939792987 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:03.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.570+0000 7f5580b9a700 1 --2- 172.21.15.101:0/939792987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f557c100ce0 0x7f557c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:03.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.570+0000 7f5581b9c700 1 --2- 172.21.15.101:0/939792987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f557c101670 0x7f557c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:03.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.570+0000 7f558139b700 1 -- 172.21.15.101:0/939792987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f557c100ce0 msgr2=0x7f557c072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:03.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.570+0000 7f558139b700 1 --2- 172.21.15.101:0/939792987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f557c100ce0 0x7f557c072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:03.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.570+0000 7f558139b700 1 -- 172.21.15.101:0/939792987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f557c101670 msgr2=0x7f557c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:03.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.570+0000 7f558139b700 1 --2- 172.21.15.101:0/939792987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f557c101670 0x7f557c072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:03.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.570+0000 7f558139b700 1 -- 172.21.15.101:0/939792987 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5574009d00 con 0x7f557c0ffaf0 2023-07-18T03:07:03.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.570+0000 7f558139b700 1 --2- 172.21.15.101:0/939792987 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f557c0ffaf0 0x7f557c0742c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f557800e970 tx=0x7f557800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:03.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.570+0000 7f55727fc700 1 -- 172.21.15.101:0/939792987 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5578009790 con 0x7f557c0ffaf0 2023-07-18T03:07:03.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.571+0000 7f55727fc700 1 -- 172.21.15.101:0/939792987 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f557800c7e0 con 0x7f557c0ffaf0 2023-07-18T03:07:03.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.571+0000 7f55727fc700 1 -- 172.21.15.101:0/939792987 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5578003c70 con 0x7f557c0ffaf0 2023-07-18T03:07:03.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.571+0000 7f55835ff700 1 -- 172.21.15.101:0/939792987 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f557c19fc30 con 0x7f557c0ffaf0 2023-07-18T03:07:03.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.571+0000 7f55835ff700 1 -- 172.21.15.101:0/939792987 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f557c1a0140 con 0x7f557c0ffaf0 2023-07-18T03:07:03.562 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.572+0000 7f55835ff700 1 -- 172.21.15.101:0/939792987 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f557c10b2d0 con 0x7f557c0ffaf0 2023-07-18T03:07:03.562 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.574+0000 7f55727fc700 1 -- 172.21.15.101:0/939792987 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f557800cc50 con 0x7f557c0ffaf0 2023-07-18T03:07:03.562 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.575+0000 7f55727fc700 1 --2- 172.21.15.101:0/939792987 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f55680a11d0 0x7f55680a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:03.562 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.575+0000 7f55727fc700 1 -- 172.21.15.101:0/939792987 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f5578014070 con 0x7f557c0ffaf0 2023-07-18T03:07:03.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.575+0000 7f5580b9a700 1 --2- 172.21.15.101:0/939792987 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f55680a11d0 0x7f55680a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:03.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.576+0000 7f5580b9a700 1 --2- 172.21.15.101:0/939792987 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f55680a11d0 0x7f55680a3680 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f556c007be0 tx=0x7f556c0073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:03.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.579+0000 7f55727fc700 1 -- 172.21.15.101:0/939792987 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5578096170 con 0x7f557c0ffaf0 2023-07-18T03:07:03.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.776+0000 7f55835ff700 1 -- 172.21.15.101:0/939792987 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f557c106970 con 0x7f557c0ffaf0 2023-07-18T03:07:03.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.777+0000 7f55727fc700 1 -- 172.21.15.101:0/939792987 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5578003dd0 con 0x7f557c0ffaf0 2023-07-18T03:07:03.764 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 64 2023-07-18T03:07:03.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.779+0000 7f55835ff700 1 -- 172.21.15.101:0/939792987 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f55680a11d0 msgr2=0x7f55680a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:03.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.779+0000 7f55835ff700 1 --2- 172.21.15.101:0/939792987 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f55680a11d0 0x7f55680a3680 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f556c007be0 tx=0x7f556c0073d0 comp rx=0 tx=0).stop 2023-07-18T03:07:03.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.779+0000 7f55835ff700 1 -- 172.21.15.101:0/939792987 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f557c0ffaf0 msgr2=0x7f557c0742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:03.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.779+0000 7f55835ff700 1 --2- 172.21.15.101:0/939792987 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f557c0ffaf0 0x7f557c0742c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f557800e970 tx=0x7f557800ec80 comp rx=0 tx=0).stop 2023-07-18T03:07:03.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.779+0000 7f55835ff700 1 -- 172.21.15.101:0/939792987 shutdown_connections 2023-07-18T03:07:03.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.779+0000 7f55835ff700 1 --2- 172.21.15.101:0/939792987 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f55680a11d0 0x7f55680a3680 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:03.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.779+0000 7f55835ff700 1 --2- 172.21.15.101:0/939792987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f557c101670 0x7f557c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:03.769 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.779+0000 7f55835ff700 1 --2- 172.21.15.101:0/939792987 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f557c0ffaf0 0x7f557c0742c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:03.769 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.779+0000 7f55835ff700 1 --2- 172.21.15.101:0/939792987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f557c100ce0 0x7f557c072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:03.769 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.780+0000 7f55835ff700 1 -- 172.21.15.101:0/939792987 >> 172.21.15.101:0/939792987 conn(0x7f557c0fb130 msgr2=0x7f557c0fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:03.769 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.780+0000 7f55835ff700 1 -- 172.21.15.101:0/939792987 shutdown_connections 2023-07-18T03:07:03.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:03.780+0000 7f55835ff700 1 -- 172.21.15.101:0/939792987 wait complete. 2023-07-18T03:07:03.786 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-18T03:07:03.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.002+0000 7f3b53bff700 1 -- 172.21.15.101:0/1377844667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b4c072f70 msgr2=0x7f3b4c0733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:03.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.002+0000 7f3b53bff700 1 --2- 172.21.15.101:0/1377844667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b4c072f70 0x7f3b4c0733e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c009990 tx=0x7f3b3c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:07:03.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.002+0000 7f3b53bff700 1 -- 172.21.15.101:0/1377844667 shutdown_connections 2023-07-18T03:07:03.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.002+0000 7f3b53bff700 1 --2- 172.21.15.101:0/1377844667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b4c10c0c0 0x7f3b4c10e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:03.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.002+0000 7f3b53bff700 1 --2- 172.21.15.101:0/1377844667 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b4c074550 0x7f3b4c0729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:03.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.002+0000 7f3b53bff700 1 --2- 172.21.15.101:0/1377844667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b4c072f70 0x7f3b4c0733e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:03.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.002+0000 7f3b53bff700 1 -- 172.21.15.101:0/1377844667 >> 172.21.15.101:0/1377844667 conn(0x7f3b4c0fb130 msgr2=0x7f3b4c0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:03.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.002+0000 7f3b53bff700 1 -- 172.21.15.101:0/1377844667 shutdown_connections 2023-07-18T03:07:03.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.002+0000 7f3b53bff700 1 -- 172.21.15.101:0/1377844667 wait complete. 2023-07-18T03:07:03.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.003+0000 7f3b53bff700 1 Processor -- start 2023-07-18T03:07:03.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.004+0000 7f3b53bff700 1 -- start start 2023-07-18T03:07:03.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.004+0000 7f3b53bff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b4c072f70 0x7f3b4c19be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:03.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.004+0000 7f3b53bff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b4c074550 0x7f3b4c19c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:03.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.004+0000 7f3b53bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b4c10c0c0 0x7f3b4c1a33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:03.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.004+0000 7f3b53bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4c1a39b0 con 0x7f3b4c10c0c0 2023-07-18T03:07:03.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.004+0000 7f3b53bff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4c1a3af0 con 0x7f3b4c072f70 2023-07-18T03:07:03.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.004+0000 7f3b53bff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4c1a3c80 con 0x7f3b4c074550 2023-07-18T03:07:03.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.004+0000 7f3b5199b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b4c072f70 0x7f3b4c19be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:03.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.004+0000 7f3b5219c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b4c10c0c0 0x7f3b4c1a33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:03.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.004+0000 7f3b5219c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b4c10c0c0 0x7f3b4c1a33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:39884/0 (socket says 172.21.15.101:39884) 2023-07-18T03:07:03.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.004+0000 7f3b5119a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b4c074550 0x7f3b4c19c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:03.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.004+0000 7f3b5219c700 1 -- 172.21.15.101:0/2344505131 learned_addr learned my addr 172.21.15.101:0/2344505131 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:03.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.004+0000 7f3b5199b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b4c072f70 0x7f3b4c19be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:59746/0 (socket says 172.21.15.101:59746) 2023-07-18T03:07:03.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.005+0000 7f3b5199b700 1 -- 172.21.15.101:0/2344505131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b4c074550 msgr2=0x7f3b4c19c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:03.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.005+0000 7f3b5199b700 1 --2- 172.21.15.101:0/2344505131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b4c074550 0x7f3b4c19c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:03.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.005+0000 7f3b5199b700 1 -- 172.21.15.101:0/2344505131 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b4c10c0c0 msgr2=0x7f3b4c1a33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:03.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.005+0000 7f3b5199b700 1 --2- 172.21.15.101:0/2344505131 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b4c10c0c0 0x7f3b4c1a33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:03.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.005+0000 7f3b5199b700 1 -- 172.21.15.101:0/2344505131 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b48009520 con 0x7f3b4c072f70 2023-07-18T03:07:03.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.005+0000 7f3b5219c700 1 --2- 172.21.15.101:0/2344505131 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b4c10c0c0 0x7f3b4c1a33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:07:03.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.005+0000 7f3b5199b700 1 --2- 172.21.15.101:0/2344505131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b4c072f70 0x7f3b4c19be30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3b4800e9c0 tx=0x7f3b4800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:03.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.005+0000 7f3b42ffd700 1 -- 172.21.15.101:0/2344505131 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4800cb70 con 0x7f3b4c072f70 2023-07-18T03:07:03.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.005+0000 7f3b53bff700 1 -- 172.21.15.101:0/2344505131 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b3c0095f0 con 0x7f3b4c072f70 2023-07-18T03:07:03.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.005+0000 7f3b42ffd700 1 -- 172.21.15.101:0/2344505131 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b4800ee60 con 0x7f3b4c072f70 2023-07-18T03:07:03.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.005+0000 7f3b42ffd700 1 -- 172.21.15.101:0/2344505131 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b48003c60 con 0x7f3b4c072f70 2023-07-18T03:07:03.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.005+0000 7f3b53bff700 1 -- 172.21.15.101:0/2344505131 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b4c1a4220 con 0x7f3b4c072f70 2023-07-18T03:07:03.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.006+0000 7f3b53bff700 1 -- 172.21.15.101:0/2344505131 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b4c101c10 con 0x7f3b4c072f70 2023-07-18T03:07:03.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.008+0000 7f3b42ffd700 1 -- 172.21.15.101:0/2344505131 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f3b48005030 con 0x7f3b4c072f70 2023-07-18T03:07:03.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.009+0000 7f3b42ffd700 1 --2- 172.21.15.101:0/2344505131 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f3b380a1150 0x7f3b380a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:03.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.010+0000 7f3b42ffd700 1 -- 172.21.15.101:0/2344505131 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f3b48014070 con 0x7f3b4c072f70 2023-07-18T03:07:03.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.010+0000 7f3b5119a700 1 --2- 172.21.15.101:0/2344505131 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f3b380a1150 0x7f3b380a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:03.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.010+0000 7f3b5119a700 1 --2- 172.21.15.101:0/2344505131 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f3b380a1150 0x7f3b380a3600 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c00b680 tx=0x7f3b3c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:03.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.012+0000 7f3b42ffd700 1 -- 172.21.15.101:0/2344505131 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3b48096910 con 0x7f3b4c072f70 2023-07-18T03:07:04.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.209+0000 7f3b53bff700 1 -- 172.21.15.101:0/2344505131 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f3b4c02cac0 con 0x7f3b4c072f70 2023-07-18T03:07:04.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.210+0000 7f3b42ffd700 1 -- 172.21.15.101:0/2344505131 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3b48099f30 con 0x7f3b4c072f70 2023-07-18T03:07:04.197 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 32 2023-07-18T03:07:04.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.212+0000 7f3b53bff700 1 -- 172.21.15.101:0/2344505131 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f3b380a1150 msgr2=0x7f3b380a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:04.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.212+0000 7f3b53bff700 1 --2- 172.21.15.101:0/2344505131 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f3b380a1150 0x7f3b380a3600 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c00b680 tx=0x7f3b3c005fd0 comp rx=0 tx=0).stop 2023-07-18T03:07:04.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.212+0000 7f3b53bff700 1 -- 172.21.15.101:0/2344505131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b4c072f70 msgr2=0x7f3b4c19be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:04.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.212+0000 7f3b53bff700 1 --2- 172.21.15.101:0/2344505131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b4c072f70 0x7f3b4c19be30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3b4800e9c0 tx=0x7f3b4800ecd0 comp rx=0 tx=0).stop 2023-07-18T03:07:04.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.212+0000 7f3b53bff700 1 -- 172.21.15.101:0/2344505131 shutdown_connections 2023-07-18T03:07:04.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.212+0000 7f3b53bff700 1 --2- 172.21.15.101:0/2344505131 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f3b380a1150 0x7f3b380a3600 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:04.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.212+0000 7f3b53bff700 1 --2- 172.21.15.101:0/2344505131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b4c072f70 0x7f3b4c19be30 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:04.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.212+0000 7f3b53bff700 1 --2- 172.21.15.101:0/2344505131 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b4c10c0c0 0x7f3b4c1a33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:04.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.212+0000 7f3b53bff700 1 --2- 172.21.15.101:0/2344505131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b4c074550 0x7f3b4c19c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:04.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.212+0000 7f3b53bff700 1 -- 172.21.15.101:0/2344505131 >> 172.21.15.101:0/2344505131 conn(0x7f3b4c0fb130 msgr2=0x7f3b4c0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:04.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.212+0000 7f3b53bff700 1 -- 172.21.15.101:0/2344505131 shutdown_connections 2023-07-18T03:07:04.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.212+0000 7f3b53bff700 1 -- 172.21.15.101:0/2344505131 wait complete. 2023-07-18T03:07:04.216 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:03 smithi101 ceph-mon[126379]: pgmap v341: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 7 op/s 2023-07-18T03:07:04.216 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:03 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1130890462' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:07:04.216 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:03 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/939792987' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:07:04.218 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:07:04.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:03 smithi155 ceph-mon[128885]: pgmap v341: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 7 op/s 2023-07-18T03:07:04.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:03 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1130890462' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:07:04.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:03 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/939792987' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:07:04.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.429+0000 7fc3d9bff700 1 -- 172.21.15.101:0/499892522 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3d40ffb00 msgr2=0x7fc3d40fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:04.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.429+0000 7fc3d9bff700 1 --2- 172.21.15.101:0/499892522 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3d40ffb00 0x7fc3d40fff10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4009990 tx=0x7fc3c4009ca0 comp rx=0 tx=0).stop 2023-07-18T03:07:04.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.429+0000 7fc3d9bff700 1 -- 172.21.15.101:0/499892522 shutdown_connections 2023-07-18T03:07:04.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.429+0000 7fc3d9bff700 1 --2- 172.21.15.101:0/499892522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3d4101680 0x7fc3d4107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:04.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.429+0000 7fc3d9bff700 1 --2- 172.21.15.101:0/499892522 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3d40ffb00 0x7fc3d40fff10 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:04.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.429+0000 7fc3d9bff700 1 --2- 172.21.15.101:0/499892522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3d4100cf0 0x7fc3d4101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:04.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.429+0000 7fc3d9bff700 1 -- 172.21.15.101:0/499892522 >> 172.21.15.101:0/499892522 conn(0x7fc3d40fb0e0 msgr2=0x7fc3d40fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:04.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.429+0000 7fc3d9bff700 1 -- 172.21.15.101:0/499892522 shutdown_connections 2023-07-18T03:07:04.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.429+0000 7fc3d9bff700 1 -- 172.21.15.101:0/499892522 wait complete. 2023-07-18T03:07:04.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.430+0000 7fc3d9bff700 1 Processor -- start 2023-07-18T03:07:04.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.430+0000 7fc3d9bff700 1 -- start start 2023-07-18T03:07:04.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.430+0000 7fc3d9bff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3d40ffb00 0x7fc3d40742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:04.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.430+0000 7fc3d9bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3d4100cf0 0x7fc3d4072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:04.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.430+0000 7fc3d9bff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3d4101680 0x7fc3d4072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:04.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.430+0000 7fc3d9bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d40735e0 con 0x7fc3d4100cf0 2023-07-18T03:07:04.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.430+0000 7fc3d9bff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d4073750 con 0x7fc3d4101680 2023-07-18T03:07:04.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.430+0000 7fc3d9bff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d419f840 con 0x7fc3d40ffb00 2023-07-18T03:07:04.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.431+0000 7fc3d2ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3d4100cf0 0x7fc3d4072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:04.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.431+0000 7fc3d2ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3d4100cf0 0x7fc3d4072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:39900/0 (socket says 172.21.15.101:39900) 2023-07-18T03:07:04.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.431+0000 7fc3d2ffd700 1 -- 172.21.15.101:0/1334166431 learned_addr learned my addr 172.21.15.101:0/1334166431 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:04.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.431+0000 7fc3d3fff700 1 --2- 172.21.15.101:0/1334166431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3d4101680 0x7fc3d4072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:04.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.431+0000 7fc3d37fe700 1 --2- 172.21.15.101:0/1334166431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3d40ffb00 0x7fc3d40742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:04.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.431+0000 7fc3d2ffd700 1 -- 172.21.15.101:0/1334166431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3d40ffb00 msgr2=0x7fc3d40742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:04.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.431+0000 7fc3d2ffd700 1 --2- 172.21.15.101:0/1334166431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3d40ffb00 0x7fc3d40742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:04.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.431+0000 7fc3d2ffd700 1 -- 172.21.15.101:0/1334166431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3d4101680 msgr2=0x7fc3d4072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:04.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.431+0000 7fc3d2ffd700 1 --2- 172.21.15.101:0/1334166431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3d4101680 0x7fc3d4072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:04.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.431+0000 7fc3d2ffd700 1 -- 172.21.15.101:0/1334166431 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c40095f0 con 0x7fc3d4100cf0 2023-07-18T03:07:04.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.431+0000 7fc3d3fff700 1 --2- 172.21.15.101:0/1334166431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3d4101680 0x7fc3d4072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:07:04.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.432+0000 7fc3d2ffd700 1 --2- 172.21.15.101:0/1334166431 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3d4100cf0 0x7fc3d4072900 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc3c800e9a0 tx=0x7fc3c800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:04.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.432+0000 7fc3d0ff9700 1 -- 172.21.15.101:0/1334166431 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8009790 con 0x7fc3d4100cf0 2023-07-18T03:07:04.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.432+0000 7fc3d0ff9700 1 -- 172.21.15.101:0/1334166431 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3c8010430 con 0x7fc3d4100cf0 2023-07-18T03:07:04.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.432+0000 7fc3d0ff9700 1 -- 172.21.15.101:0/1334166431 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8018680 con 0x7fc3d4100cf0 2023-07-18T03:07:04.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.432+0000 7fc3d9bff700 1 -- 172.21.15.101:0/1334166431 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3d419fb20 con 0x7fc3d4100cf0 2023-07-18T03:07:04.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.432+0000 7fc3d9bff700 1 -- 172.21.15.101:0/1334166431 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3d419ffe0 con 0x7fc3d4100cf0 2023-07-18T03:07:04.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.433+0000 7fc3d9bff700 1 -- 172.21.15.101:0/1334166431 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3d410b2d0 con 0x7fc3d4100cf0 2023-07-18T03:07:04.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.435+0000 7fc3d0ff9700 1 -- 172.21.15.101:0/1334166431 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fc3c80105a0 con 0x7fc3d4100cf0 2023-07-18T03:07:04.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.436+0000 7fc3d0ff9700 1 --2- 172.21.15.101:0/1334166431 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc3bc0a11a0 0x7fc3bc0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:04.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.436+0000 7fc3d0ff9700 1 -- 172.21.15.101:0/1334166431 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fc3c8014070 con 0x7fc3d4100cf0 2023-07-18T03:07:04.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.437+0000 7fc3d37fe700 1 --2- 172.21.15.101:0/1334166431 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc3bc0a11a0 0x7fc3bc0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:04.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.437+0000 7fc3d37fe700 1 --2- 172.21.15.101:0/1334166431 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc3bc0a11a0 0x7fc3bc0a3650 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc3c400b2e0 tx=0x7fc3c4005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:04.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.440+0000 7fc3d0ff9700 1 -- 172.21.15.101:0/1334166431 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc3c8051790 con 0x7fc3d4100cf0 2023-07-18T03:07:04.444 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:03 smithi179 ceph-mon[129229]: pgmap v341: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 7 op/s 2023-07-18T03:07:04.444 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:03 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1130890462' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:07:04.444 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:03 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/939792987' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:07:04.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.682+0000 7fc3d9bff700 1 -- 172.21.15.101:0/1334166431 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3d4106990 con 0x7fc3d4100cf0 2023-07-18T03:07:04.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.683+0000 7fc3d0ff9700 1 -- 172.21.15.101:0/1334166431 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10858 (secure 0 0 0) 0x7fc3c8096220 con 0x7fc3d4100cf0 2023-07-18T03:07:04.671 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:07:04.671 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/4192846769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":4192846769},{"type":"v1","addr":"172.21.15.101:6839","nonce":4192846769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6841/3824340175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6840","nonce":3824340175},{"type":"v1","addr":"172.21.15.101:6841","nonce":3824340175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:04:59.361185+0000","modified":"2023-07-18T03:05:31.985959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.155:6839/2055462483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2055462483},{"type":"v1","addr":"172.21.15.155:6839","nonce":2055462483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:07:04.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.685+0000 7fc3d9bff700 1 -- 172.21.15.101:0/1334166431 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc3bc0a11a0 msgr2=0x7fc3bc0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:04.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.685+0000 7fc3d9bff700 1 --2- 172.21.15.101:0/1334166431 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc3bc0a11a0 0x7fc3bc0a3650 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc3c400b2e0 tx=0x7fc3c4005960 comp rx=0 tx=0).stop 2023-07-18T03:07:04.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.685+0000 7fc3d9bff700 1 -- 172.21.15.101:0/1334166431 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3d4100cf0 msgr2=0x7fc3d4072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:04.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.685+0000 7fc3d9bff700 1 --2- 172.21.15.101:0/1334166431 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3d4100cf0 0x7fc3d4072900 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc3c800e9a0 tx=0x7fc3c800ecb0 comp rx=0 tx=0).stop 2023-07-18T03:07:04.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.686+0000 7fc3d9bff700 1 -- 172.21.15.101:0/1334166431 shutdown_connections 2023-07-18T03:07:04.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.686+0000 7fc3d9bff700 1 --2- 172.21.15.101:0/1334166431 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc3bc0a11a0 0x7fc3bc0a3650 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:04.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.686+0000 7fc3d9bff700 1 --2- 172.21.15.101:0/1334166431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3d4101680 0x7fc3d4072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:04.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.686+0000 7fc3d9bff700 1 --2- 172.21.15.101:0/1334166431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3d40ffb00 0x7fc3d40742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:04.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.686+0000 7fc3d9bff700 1 --2- 172.21.15.101:0/1334166431 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3d4100cf0 0x7fc3d4072900 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:04.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.686+0000 7fc3d9bff700 1 -- 172.21.15.101:0/1334166431 >> 172.21.15.101:0/1334166431 conn(0x7fc3d40fb0e0 msgr2=0x7fc3d40fd300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:04.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.686+0000 7fc3d9bff700 1 -- 172.21.15.101:0/1334166431 shutdown_connections 2023-07-18T03:07:04.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.686+0000 7fc3d9bff700 1 -- 172.21.15.101:0/1334166431 wait complete. 2023-07-18T03:07:04.675 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 24 2023-07-18T03:07:04.692 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:07:04.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.901+0000 7fd2023ff700 1 -- 172.21.15.101:0/926574581 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1fc100550 msgr2=0x7fd1fc102930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:04.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.901+0000 7fd2023ff700 1 --2- 172.21.15.101:0/926574581 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1fc100550 0x7fd1fc102930 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec0098f0 tx=0x7fd1ec009c00 comp rx=0 tx=0).stop 2023-07-18T03:07:04.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.901+0000 7fd2023ff700 1 -- 172.21.15.101:0/926574581 shutdown_connections 2023-07-18T03:07:04.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.901+0000 7fd2023ff700 1 --2- 172.21.15.101:0/926574581 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1fc100550 0x7fd1fc102930 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:04.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.901+0000 7fd2023ff700 1 --2- 172.21.15.101:0/926574581 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd1fc105790 0x7fd1fc107bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:04.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.901+0000 7fd2023ff700 1 --2- 172.21.15.101:0/926574581 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1fc102e70 0x7fd1fc105250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:04.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.901+0000 7fd2023ff700 1 -- 172.21.15.101:0/926574581 >> 172.21.15.101:0/926574581 conn(0x7fd1fc0f9fd0 msgr2=0x7fd1fc0fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:04.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.901+0000 7fd2023ff700 1 -- 172.21.15.101:0/926574581 shutdown_connections 2023-07-18T03:07:04.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.901+0000 7fd2023ff700 1 -- 172.21.15.101:0/926574581 wait complete. 2023-07-18T03:07:04.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.902+0000 7fd2023ff700 1 Processor -- start 2023-07-18T03:07:04.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.903+0000 7fd2023ff700 1 -- start start 2023-07-18T03:07:04.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.903+0000 7fd2023ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1fc100550 0x7fd1fc197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:04.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.903+0000 7fd2023ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd1fc102e70 0x7fd1fc197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:04.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.903+0000 7fd2023ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1fc105790 0x7fd1fc19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:04.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.903+0000 7fd2023ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1fc19f580 con 0x7fd1fc102e70 2023-07-18T03:07:04.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.903+0000 7fd2023ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1fc19f6c0 con 0x7fd1fc105790 2023-07-18T03:07:04.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.903+0000 7fd2023ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1fc19f850 con 0x7fd1fc100550 2023-07-18T03:07:04.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.903+0000 7fd1f3fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd1fc102e70 0x7fd1fc197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:04.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.903+0000 7fd1f3fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd1fc102e70 0x7fd1fc197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:39926/0 (socket says 172.21.15.101:39926) 2023-07-18T03:07:04.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.903+0000 7fd1f3fff700 1 -- 172.21.15.101:0/4253742241 learned_addr learned my addr 172.21.15.101:0/4253742241 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:04.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.904+0000 7fd1fbfff700 1 --2- 172.21.15.101:0/4253742241 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1fc100550 0x7fd1fc197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:04.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.904+0000 7fd20099c700 1 --2- 172.21.15.101:0/4253742241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1fc105790 0x7fd1fc19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:04.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.904+0000 7fd1f3fff700 1 -- 172.21.15.101:0/4253742241 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1fc100550 msgr2=0x7fd1fc197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:04.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.904+0000 7fd1f3fff700 1 --2- 172.21.15.101:0/4253742241 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1fc100550 0x7fd1fc197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:04.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.904+0000 7fd1f3fff700 1 -- 172.21.15.101:0/4253742241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1fc105790 msgr2=0x7fd1fc19efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:04.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.904+0000 7fd1f3fff700 1 --2- 172.21.15.101:0/4253742241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1fc105790 0x7fd1fc19efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:04.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.904+0000 7fd1f3fff700 1 -- 172.21.15.101:0/4253742241 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ec009520 con 0x7fd1fc102e70 2023-07-18T03:07:04.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.904+0000 7fd1f3fff700 1 --2- 172.21.15.101:0/4253742241 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd1fc102e70 0x7fd1fc197f40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd1e400d950 tx=0x7fd1e400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:04.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.904+0000 7fd1f9ffb700 1 -- 172.21.15.101:0/4253742241 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4009940 con 0x7fd1fc102e70 2023-07-18T03:07:04.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.904+0000 7fd1f9ffb700 1 -- 172.21.15.101:0/4253742241 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1e4010460 con 0x7fd1fc102e70 2023-07-18T03:07:04.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.904+0000 7fd2023ff700 1 -- 172.21.15.101:0/4253742241 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1fc19fb30 con 0x7fd1fc102e70 2023-07-18T03:07:04.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.904+0000 7fd1f9ffb700 1 -- 172.21.15.101:0/4253742241 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e400f620 con 0x7fd1fc102e70 2023-07-18T03:07:04.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.904+0000 7fd2023ff700 1 -- 172.21.15.101:0/4253742241 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1fc1a0090 con 0x7fd1fc102e70 2023-07-18T03:07:04.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.905+0000 7fd2023ff700 1 -- 172.21.15.101:0/4253742241 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1fc107350 con 0x7fd1fc102e70 2023-07-18T03:07:04.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.907+0000 7fd1f9ffb700 1 -- 172.21.15.101:0/4253742241 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fd1e40108d0 con 0x7fd1fc102e70 2023-07-18T03:07:04.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.909+0000 7fd1f9ffb700 1 --2- 172.21.15.101:0/4253742241 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd1e80a1220 0x7fd1e80a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:04.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.909+0000 7fd1f9ffb700 1 -- 172.21.15.101:0/4253742241 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fd1e40ccab0 con 0x7fd1fc102e70 2023-07-18T03:07:04.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.909+0000 7fd1fbfff700 1 --2- 172.21.15.101:0/4253742241 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd1e80a1220 0x7fd1e80a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:04.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.910+0000 7fd1fbfff700 1 --2- 172.21.15.101:0/4253742241 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd1e80a1220 0x7fd1e80a36d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec00fa40 tx=0x7fd1ec019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:04.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:04.912+0000 7fd1f9ffb700 1 -- 172.21.15.101:0/4253742241 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd1e4095e30 con 0x7fd1fc102e70 2023-07-18T03:07:05.100 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.113+0000 7fd2023ff700 1 -- 172.21.15.101:0/4253742241 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1fc02c850 con 0x7fd1fc102e70 2023-07-18T03:07:05.101 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.114+0000 7fd1f9ffb700 1 -- 172.21.15.101:0/4253742241 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24725 (secure 0 0 0) 0x7fd1e4016090 con 0x7fd1fc102e70 2023-07-18T03:07:05.102 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:07:05.102 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":91,"fsid":"b1782daa-2516-11ee-9b34-001a4aab830c","created":"2023-07-18T02:57:22.972483+0000","modified":"2023-07-18T03:06:24.250777+0000","last_up_change":"2023-07-18T03:04:11.429530+0000","last_in_change":"2023-07-18T03:03:54.158340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:00:43.353489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:04:57.771616+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-18T03:04:58.769590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-18T03:05:02.358830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc76b5d1-38a0-4f1a-9335-756bfc40a62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6803","nonce":3610719929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6805","nonce":3610719929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6809","nonce":3610719929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6807","nonce":3610719929}]},"public_addr":"172.21.15.101:6803/3610719929","cluster_addr":"172.21.15.101:6805/3610719929","heartbeat_back_addr":"172.21.15.101:6809/3610719929","heartbeat_front_addr":"172.21.15.101:6807/3610719929","state":["exists","up"]},{"osd":1,"uuid":"2a299573-0b34-4af9-aaae-b523e5058ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6801","nonce":1030229579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6803","nonce":1030229579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6807","nonce":1030229579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6805","nonce":1030229579}]},"public_addr":"172.21.15.155:6801/1030229579","cluster_addr":"172.21.15.155:6803/1030229579","heartbeat_back_addr":"172.21.15.155:6807/1030229579","heartbeat_front_addr":"172.21.15.155:6805/1030229579","state":["exists","up"]},{"osd":2,"uuid":"123b3749-c636-48fa-9e33-6229b5575dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6801","nonce":1349487507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6803","nonce":1349487507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6807","nonce":1349487507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6805","nonce":1349487507}]},"public_addr":"172.21.15.179:6801/1349487507","cluster_addr":"172.21.15.179:6803/1349487507","heartbeat_back_addr":"172.21.15.179:6807/1349487507","heartbeat_front_addr":"172.21.15.179:6805/1349487507","state":["exists","up"]},{"osd":3,"uuid":"4b0c409d-2954-4127-833a-b196bf226d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6811","nonce":4226577463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6813","nonce":4226577463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6817","nonce":4226577463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6815","nonce":4226577463}]},"public_addr":"172.21.15.101:6811/4226577463","cluster_addr":"172.21.15.101:6813/4226577463","heartbeat_back_addr":"172.21.15.101:6817/4226577463","heartbeat_front_addr":"172.21.15.101:6815/4226577463","state":["exists","up"]},{"osd":4,"uuid":"2137f2df-984e-45ce-baf6-b4ac12c7e3c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6809","nonce":1951135805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6811","nonce":1951135805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6815","nonce":1951135805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6813","nonce":1951135805}]},"public_addr":"172.21.15.155:6809/1951135805","cluster_addr":"172.21.15.155:6811/1951135805","heartbeat_back_addr":"172.21.15.155:6815/1951135805","heartbeat_front_addr":"172.21.15.155:6813/1951135805","state":["exists","up"]},{"osd":5,"uuid":"1020bfc5-8c2f-482e-9588-66c7f98e46e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6809","nonce":2746037021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6811","nonce":2746037021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6815","nonce":2746037021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6813","nonce":2746037021}]},"public_addr":"172.21.15.179:6809/2746037021","cluster_addr":"172.21.15.179:6811/2746037021","heartbeat_back_addr":"172.21.15.179:6815/2746037021","heartbeat_front_addr":"172.21.15.179:6813/2746037021","state":["exists","up"]},{"osd":6,"uuid":"2cd99260-5c36-4603-9338-cf697970ce58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6819","nonce":958044931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6821","nonce":958044931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6825","nonce":958044931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6823","nonce":958044931}]},"public_addr":"172.21.15.101:6819/958044931","cluster_addr":"172.21.15.101:6821/958044931","heartbeat_back_addr":"172.21.15.101:6825/958044931","heartbeat_front_addr":"172.21.15.101:6823/958044931","state":["exists","up"]},{"osd":7,"uuid":"ed23fd0a-678f-4e15-b42c-cfe873315714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6817","nonce":3256322291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6819","nonce":3256322291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6823","nonce":3256322291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6821","nonce":3256322291}]},"public_addr":"172.21.15.155:6817/3256322291","cluster_addr":"172.21.15.155:6819/3256322291","heartbeat_back_addr":"172.21.15.155:6823/3256322291","heartbeat_front_addr":"172.21.15.155:6821/3256322291","state":["exists","up"]},{"osd":8,"uuid":"eca3d8e6-7baf-47a5-b991-328001c7ebef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6817","nonce":3520612559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6819","nonce":3520612559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6823","nonce":3520612559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6821","nonce":3520612559}]},"public_addr":"172.21.15.179:6817/3520612559","cluster_addr":"172.21.15.179:6819/3520612559","heartbeat_back_addr":"172.21.15.179:6823/3520612559","heartbeat_front_addr":"172.21.15.179:6821/3520612559","state":["exists","up"]},{"osd":9,"uuid":"465a7ffb-67e4-49ef-812c-1b96feacfebf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6827","nonce":2166787667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6829","nonce":2166787667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6833","nonce":2166787667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6831","nonce":2166787667}]},"public_addr":"172.21.15.101:6827/2166787667","cluster_addr":"172.21.15.101:6829/2166787667","heartbeat_back_addr":"172.21.15.101:6833/2166787667","heartbeat_front_addr":"172.21.15.101:6831/2166787667","state":["exists","up"]},{"osd":10,"uuid":"1897a027-f25d-434f-8df5-5fc2bd866578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6825","nonce":3682446349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6827","nonce":3682446349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6831","nonce":3682446349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6829","nonce":3682446349}]},"public_addr":"172.21.15.155:6825/3682446349","cluster_addr":"172.21.15.155:6827/3682446349","heartbeat_back_addr":"172.21.15.155:6831/3682446349","heartbeat_front_addr":"172.21.15.155:6829/3682446349","state":["exists","up"]},{"osd":11,"uuid":"b62effaf-45e2-4585-b8d0-db448f3474ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6825","nonce":2602640858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6827","nonce":2602640858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6831","nonce":2602640858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6829","nonce":2602640858}]},"public_addr":"172.21.15.179:6825/2602640858","cluster_addr":"172.21.15.179:6827/2602640858","heartbeat_back_addr":"172.21.15.179:6831/2602640858","heartbeat_front_addr":"172.21.15.179:6829/2602640858","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T02:59:56.074519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:19.358142+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:41.128966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:08.227054+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:31.442592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:52.751685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:19.251054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:40.698241+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:01.339150+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:27.955910+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:49.136980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:04:10.084982+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.101:6800/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:6801/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/604228291":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/4055187849":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/2844574324":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/1603248215":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6801/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6800/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/3307578182":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/373189792":"2023-07-19T02:57:56.243200+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:07:05.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.117+0000 7fd2023ff700 1 -- 172.21.15.101:0/4253742241 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd1e80a1220 msgr2=0x7fd1e80a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:05.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.117+0000 7fd2023ff700 1 --2- 172.21.15.101:0/4253742241 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd1e80a1220 0x7fd1e80a36d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec00fa40 tx=0x7fd1ec019040 comp rx=0 tx=0).stop 2023-07-18T03:07:05.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.117+0000 7fd2023ff700 1 -- 172.21.15.101:0/4253742241 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd1fc102e70 msgr2=0x7fd1fc197f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:05.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.117+0000 7fd2023ff700 1 --2- 172.21.15.101:0/4253742241 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd1fc102e70 0x7fd1fc197f40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd1e400d950 tx=0x7fd1e400dc60 comp rx=0 tx=0).stop 2023-07-18T03:07:05.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.117+0000 7fd2023ff700 1 -- 172.21.15.101:0/4253742241 shutdown_connections 2023-07-18T03:07:05.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.117+0000 7fd2023ff700 1 --2- 172.21.15.101:0/4253742241 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fd1e80a1220 0x7fd1e80a36d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:05.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.117+0000 7fd2023ff700 1 --2- 172.21.15.101:0/4253742241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1fc105790 0x7fd1fc19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:05.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.117+0000 7fd2023ff700 1 --2- 172.21.15.101:0/4253742241 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1fc100550 0x7fd1fc197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:05.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.117+0000 7fd2023ff700 1 --2- 172.21.15.101:0/4253742241 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd1fc102e70 0x7fd1fc197f40 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:05.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.117+0000 7fd2023ff700 1 -- 172.21.15.101:0/4253742241 >> 172.21.15.101:0/4253742241 conn(0x7fd1fc0f9fd0 msgr2=0x7fd1fc0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:05.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.117+0000 7fd2023ff700 1 -- 172.21.15.101:0/4253742241 shutdown_connections 2023-07-18T03:07:05.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.117+0000 7fd2023ff700 1 -- 172.21.15.101:0/4253742241 wait complete. 2023-07-18T03:07:05.124 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:00:43.353489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-18T03:04:57.771616+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-18T03:04:58.769590+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-18T03:05:02.358830+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-18T03:07:05.125 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-18T03:07:05.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:04 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2344505131' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-18T03:07:05.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:04 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1334166431' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:07:05.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:04 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/2344505131' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-18T03:07:05.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:04 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1334166431' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:07:05.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.338+0000 7fee55fff700 1 -- 172.21.15.101:0/3779053333 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fee50100cd0 msgr2=0x7fee50101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:05.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.338+0000 7fee55fff700 1 --2- 172.21.15.101:0/3779053333 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fee50100cd0 0x7fee50101120 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fee44009990 tx=0x7fee44009ca0 comp rx=0 tx=0).stop 2023-07-18T03:07:05.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.338+0000 7fee55fff700 1 -- 172.21.15.101:0/3779053333 shutdown_connections 2023-07-18T03:07:05.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.338+0000 7fee55fff700 1 --2- 172.21.15.101:0/3779053333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee50101660 0x7fee50107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:05.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.338+0000 7fee55fff700 1 --2- 172.21.15.101:0/3779053333 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee500ffae0 0x7fee500ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:05.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.338+0000 7fee55fff700 1 --2- 172.21.15.101:0/3779053333 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fee50100cd0 0x7fee50101120 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:05.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.338+0000 7fee55fff700 1 -- 172.21.15.101:0/3779053333 >> 172.21.15.101:0/3779053333 conn(0x7fee500fb0e0 msgr2=0x7fee500fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:05.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.338+0000 7fee55fff700 1 -- 172.21.15.101:0/3779053333 shutdown_connections 2023-07-18T03:07:05.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.338+0000 7fee55fff700 1 -- 172.21.15.101:0/3779053333 wait complete. 2023-07-18T03:07:05.328 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.339+0000 7fee55fff700 1 Processor -- start 2023-07-18T03:07:05.328 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.339+0000 7fee55fff700 1 -- start start 2023-07-18T03:07:05.328 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.339+0000 7fee55fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee500ffae0 0x7fee50197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:05.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.339+0000 7fee55fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fee50100cd0 0x7fee50197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:05.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.339+0000 7fee55fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee50101660 0x7fee5019f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:05.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.339+0000 7fee55fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5019f610 con 0x7fee50100cd0 2023-07-18T03:07:05.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.339+0000 7fee55fff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5019f750 con 0x7fee50101660 2023-07-18T03:07:05.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.339+0000 7fee55fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5019f8e0 con 0x7fee500ffae0 2023-07-18T03:07:05.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.340+0000 7fee4effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fee50100cd0 0x7fee50197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:05.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.340+0000 7fee4f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee500ffae0 0x7fee50197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:05.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.340+0000 7fee4effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fee50100cd0 0x7fee50197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:39952/0 (socket says 172.21.15.101:39952) 2023-07-18T03:07:05.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.340+0000 7fee4effd700 1 -- 172.21.15.101:0/126823864 learned_addr learned my addr 172.21.15.101:0/126823864 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:05.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.340+0000 7fee4ffff700 1 --2- 172.21.15.101:0/126823864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee50101660 0x7fee5019f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:05.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.340+0000 7fee4effd700 1 -- 172.21.15.101:0/126823864 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee500ffae0 msgr2=0x7fee50197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:05.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.340+0000 7fee4effd700 1 --2- 172.21.15.101:0/126823864 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee500ffae0 0x7fee50197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:05.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.340+0000 7fee4effd700 1 -- 172.21.15.101:0/126823864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee50101660 msgr2=0x7fee5019f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:05.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.340+0000 7fee4effd700 1 --2- 172.21.15.101:0/126823864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee50101660 0x7fee5019f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:05.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.340+0000 7fee4effd700 1 -- 172.21.15.101:0/126823864 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee440095f0 con 0x7fee50100cd0 2023-07-18T03:07:05.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.340+0000 7fee4effd700 1 --2- 172.21.15.101:0/126823864 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fee50100cd0 0x7fee50197fd0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fee44004760 tx=0x7fee44004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:05.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.340+0000 7fee4cff9700 1 -- 172.21.15.101:0/126823864 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4401d070 con 0x7fee50100cd0 2023-07-18T03:07:05.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.340+0000 7fee55fff700 1 -- 172.21.15.101:0/126823864 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee5019fb60 con 0x7fee50100cd0 2023-07-18T03:07:05.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.340+0000 7fee4cff9700 1 -- 172.21.15.101:0/126823864 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee44004830 con 0x7fee50100cd0 2023-07-18T03:07:05.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.341+0000 7fee4cff9700 1 -- 172.21.15.101:0/126823864 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee44005350 con 0x7fee50100cd0 2023-07-18T03:07:05.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.341+0000 7fee55fff700 1 -- 172.21.15.101:0/126823864 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee5019ff80 con 0x7fee50100cd0 2023-07-18T03:07:05.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.342+0000 7fee55fff700 1 -- 172.21.15.101:0/126823864 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee5010b300 con 0x7fee50100cd0 2023-07-18T03:07:05.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.344+0000 7fee4cff9700 1 -- 172.21.15.101:0/126823864 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fee44005530 con 0x7fee50100cd0 2023-07-18T03:07:05.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.345+0000 7fee4cff9700 1 --2- 172.21.15.101:0/126823864 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fee380a1150 0x7fee380a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:05.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.345+0000 7fee4cff9700 1 -- 172.21.15.101:0/126823864 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fee440cde70 con 0x7fee50100cd0 2023-07-18T03:07:05.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.345+0000 7fee4f7fe700 1 --2- 172.21.15.101:0/126823864 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fee380a1150 0x7fee380a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:05.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.346+0000 7fee4f7fe700 1 --2- 172.21.15.101:0/126823864 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fee380a1150 0x7fee380a3600 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fee50100b30 tx=0x7fee40009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:05.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.349+0000 7fee4cff9700 1 -- 172.21.15.101:0/126823864 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fee4405a790 con 0x7fee50100cd0 2023-07-18T03:07:05.443 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:04 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/2344505131' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-18T03:07:05.444 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:04 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1334166431' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:07:05.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.549+0000 7fee55fff700 1 -- 172.21.15.101:0/126823864 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fee501a05f0 con 0x7fee50100cd0 2023-07-18T03:07:05.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.550+0000 7fee4cff9700 1 -- 172.21.15.101:0/126823864 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7fee44097280 con 0x7fee50100cd0 2023-07-18T03:07:05.537 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 1 2023-07-18T03:07:05.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.552+0000 7fee55fff700 1 -- 172.21.15.101:0/126823864 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fee380a1150 msgr2=0x7fee380a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:05.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.552+0000 7fee55fff700 1 --2- 172.21.15.101:0/126823864 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fee380a1150 0x7fee380a3600 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fee50100b30 tx=0x7fee40009260 comp rx=0 tx=0).stop 2023-07-18T03:07:05.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.552+0000 7fee55fff700 1 -- 172.21.15.101:0/126823864 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fee50100cd0 msgr2=0x7fee50197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:05.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.552+0000 7fee55fff700 1 --2- 172.21.15.101:0/126823864 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fee50100cd0 0x7fee50197fd0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fee44004760 tx=0x7fee44004790 comp rx=0 tx=0).stop 2023-07-18T03:07:05.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.552+0000 7fee55fff700 1 -- 172.21.15.101:0/126823864 shutdown_connections 2023-07-18T03:07:05.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.552+0000 7fee55fff700 1 --2- 172.21.15.101:0/126823864 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fee380a1150 0x7fee380a3600 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:05.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.552+0000 7fee55fff700 1 --2- 172.21.15.101:0/126823864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee50101660 0x7fee5019f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:05.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.552+0000 7fee55fff700 1 --2- 172.21.15.101:0/126823864 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee500ffae0 0x7fee50197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:05.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.552+0000 7fee55fff700 1 --2- 172.21.15.101:0/126823864 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fee50100cd0 0x7fee50197fd0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:05.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.552+0000 7fee55fff700 1 -- 172.21.15.101:0/126823864 >> 172.21.15.101:0/126823864 conn(0x7fee500fb0e0 msgr2=0x7fee500fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:05.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.552+0000 7fee55fff700 1 -- 172.21.15.101:0/126823864 shutdown_connections 2023-07-18T03:07:05.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.552+0000 7fee55fff700 1 -- 172.21.15.101:0/126823864 wait complete. 2023-07-18T03:07:05.558 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-18T03:07:05.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.769+0000 7fb5317ff700 1 -- 172.21.15.101:0/3087146422 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb52c101670 msgr2=0x7fb52c107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:05.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.769+0000 7fb5317ff700 1 --2- 172.21.15.101:0/3087146422 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb52c101670 0x7fb52c107f30 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb52400b0a0 tx=0x7fb52400b3b0 comp rx=0 tx=0).stop 2023-07-18T03:07:05.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.769+0000 7fb5317ff700 1 -- 172.21.15.101:0/3087146422 shutdown_connections 2023-07-18T03:07:05.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.769+0000 7fb5317ff700 1 --2- 172.21.15.101:0/3087146422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb52c0ffaf0 0x7fb52c0fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:05.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.769+0000 7fb5317ff700 1 --2- 172.21.15.101:0/3087146422 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb52c101670 0x7fb52c107f30 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:05.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.769+0000 7fb5317ff700 1 --2- 172.21.15.101:0/3087146422 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb52c100ce0 0x7fb52c101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:05.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.769+0000 7fb5317ff700 1 -- 172.21.15.101:0/3087146422 >> 172.21.15.101:0/3087146422 conn(0x7fb52c0fb130 msgr2=0x7fb52c0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:05.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.769+0000 7fb5317ff700 1 -- 172.21.15.101:0/3087146422 shutdown_connections 2023-07-18T03:07:05.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.769+0000 7fb5317ff700 1 -- 172.21.15.101:0/3087146422 wait complete. 2023-07-18T03:07:05.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.770+0000 7fb5317ff700 1 Processor -- start 2023-07-18T03:07:05.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.770+0000 7fb5317ff700 1 -- start start 2023-07-18T03:07:05.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.770+0000 7fb5317ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb52c0ffaf0 0x7fb52c195920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:05.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.771+0000 7fb5317ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb52c100ce0 0x7fb52c195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:05.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.771+0000 7fb5317ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb52c101670 0x7fb52c19ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:05.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.771+0000 7fb5317ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c19d4a0 con 0x7fb52c101670 2023-07-18T03:07:05.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.771+0000 7fb5317ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c19d5e0 con 0x7fb52c0ffaf0 2023-07-18T03:07:05.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.771+0000 7fb5317ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c19d770 con 0x7fb52c100ce0 2023-07-18T03:07:05.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.771+0000 7fb52b7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb52c101670 0x7fb52c19ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:05.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.771+0000 7fb52b7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb52c101670 0x7fb52c19ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:39978/0 (socket says 172.21.15.101:39978) 2023-07-18T03:07:05.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.771+0000 7fb52b7fe700 1 -- 172.21.15.101:0/3273783383 learned_addr learned my addr 172.21.15.101:0/3273783383 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:05.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.771+0000 7fb523fff700 1 --2- 172.21.15.101:0/3273783383 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb52c100ce0 0x7fb52c195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:05.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.771+0000 7fb52affd700 1 --2- 172.21.15.101:0/3273783383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb52c0ffaf0 0x7fb52c195920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:05.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.771+0000 7fb52b7fe700 1 -- 172.21.15.101:0/3273783383 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb52c100ce0 msgr2=0x7fb52c195e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:05.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.771+0000 7fb52b7fe700 1 --2- 172.21.15.101:0/3273783383 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb52c100ce0 0x7fb52c195e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:05.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.771+0000 7fb52b7fe700 1 -- 172.21.15.101:0/3273783383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb52c0ffaf0 msgr2=0x7fb52c195920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:05.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.771+0000 7fb52b7fe700 1 --2- 172.21.15.101:0/3273783383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb52c0ffaf0 0x7fb52c195920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:05.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.771+0000 7fb52b7fe700 1 -- 172.21.15.101:0/3273783383 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb524009d00 con 0x7fb52c101670 2023-07-18T03:07:05.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.771+0000 7fb52b7fe700 1 --2- 172.21.15.101:0/3273783383 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb52c101670 0x7fb52c19ced0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb52400d010 tx=0x7fb52400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:05.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.772+0000 7fb528ff9700 1 -- 172.21.15.101:0/3273783383 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52400e040 con 0x7fb52c101670 2023-07-18T03:07:05.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.772+0000 7fb5317ff700 1 -- 172.21.15.101:0/3273783383 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb52c19d9f0 con 0x7fb52c101670 2023-07-18T03:07:05.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.772+0000 7fb528ff9700 1 -- 172.21.15.101:0/3273783383 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb524003e10 con 0x7fb52c101670 2023-07-18T03:07:05.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.772+0000 7fb528ff9700 1 -- 172.21.15.101:0/3273783383 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5240052a0 con 0x7fb52c101670 2023-07-18T03:07:05.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.772+0000 7fb5317ff700 1 -- 172.21.15.101:0/3273783383 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb52c19de10 con 0x7fb52c101670 2023-07-18T03:07:05.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.773+0000 7fb5317ff700 1 -- 172.21.15.101:0/3273783383 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb52c1069d0 con 0x7fb52c101670 2023-07-18T03:07:05.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.775+0000 7fb528ff9700 1 -- 172.21.15.101:0/3273783383 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fb52401d480 con 0x7fb52c101670 2023-07-18T03:07:05.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.776+0000 7fb528ff9700 1 --2- 172.21.15.101:0/3273783383 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb5180a11d0 0x7fb5180a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:05.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.776+0000 7fb528ff9700 1 -- 172.21.15.101:0/3273783383 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fb5240cdf80 con 0x7fb52c101670 2023-07-18T03:07:05.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.777+0000 7fb52affd700 1 --2- 172.21.15.101:0/3273783383 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb5180a11d0 0x7fb5180a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:05.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.777+0000 7fb52affd700 1 --2- 172.21.15.101:0/3273783383 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb5180a11d0 0x7fb5180a3680 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb52c100b40 tx=0x7fb51c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:05.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.780+0000 7fb528ff9700 1 -- 172.21.15.101:0/3273783383 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb524100790 con 0x7fb52c101670 2023-07-18T03:07:05.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.979+0000 7fb5317ff700 1 -- 172.21.15.101:0/3273783383 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb52c0611c0 con 0x7fb52c101670 2023-07-18T03:07:05.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.980+0000 7fb528ff9700 1 -- 172.21.15.101:0/3273783383 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb5240973f0 con 0x7fb52c101670 2023-07-18T03:07:05.967 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 32 2023-07-18T03:07:05.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.982+0000 7fb5317ff700 1 -- 172.21.15.101:0/3273783383 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb5180a11d0 msgr2=0x7fb5180a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:05.970 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.982+0000 7fb5317ff700 1 --2- 172.21.15.101:0/3273783383 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb5180a11d0 0x7fb5180a3680 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb52c100b40 tx=0x7fb51c009260 comp rx=0 tx=0).stop 2023-07-18T03:07:05.970 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.982+0000 7fb5317ff700 1 -- 172.21.15.101:0/3273783383 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb52c101670 msgr2=0x7fb52c19ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:05.970 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.982+0000 7fb5317ff700 1 --2- 172.21.15.101:0/3273783383 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb52c101670 0x7fb52c19ced0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb52400d010 tx=0x7fb52400bc80 comp rx=0 tx=0).stop 2023-07-18T03:07:05.970 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.982+0000 7fb5317ff700 1 -- 172.21.15.101:0/3273783383 shutdown_connections 2023-07-18T03:07:05.970 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.982+0000 7fb5317ff700 1 --2- 172.21.15.101:0/3273783383 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb5180a11d0 0x7fb5180a3680 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:05.970 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.982+0000 7fb5317ff700 1 --2- 172.21.15.101:0/3273783383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb52c0ffaf0 0x7fb52c195920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:05.970 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.982+0000 7fb5317ff700 1 --2- 172.21.15.101:0/3273783383 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb52c101670 0x7fb52c19ced0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:05.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.982+0000 7fb5317ff700 1 --2- 172.21.15.101:0/3273783383 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb52c100ce0 0x7fb52c195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:05.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.982+0000 7fb5317ff700 1 -- 172.21.15.101:0/3273783383 >> 172.21.15.101:0/3273783383 conn(0x7fb52c0fb130 msgr2=0x7fb52c0fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:05.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.982+0000 7fb5317ff700 1 -- 172.21.15.101:0/3273783383 shutdown_connections 2023-07-18T03:07:05.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:05.982+0000 7fb5317ff700 1 -- 172.21.15.101:0/3273783383 wait complete. 2023-07-18T03:07:05.989 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-18T03:07:06.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.201+0000 7efe28bff700 1 -- 172.21.15.101:0/4116358708 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efe240744f0 msgr2=0x7efe24072940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:06.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.201+0000 7efe28bff700 1 --2- 172.21.15.101:0/4116358708 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efe240744f0 0x7efe24072940 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7efe14009990 tx=0x7efe14009ca0 comp rx=0 tx=0).stop 2023-07-18T03:07:06.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.201+0000 7efe28bff700 1 -- 172.21.15.101:0/4116358708 shutdown_connections 2023-07-18T03:07:06.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.201+0000 7efe28bff700 1 --2- 172.21.15.101:0/4116358708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe24107990 0x7efe24109e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:06.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.201+0000 7efe28bff700 1 --2- 172.21.15.101:0/4116358708 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efe240744f0 0x7efe24072940 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:06.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.201+0000 7efe28bff700 1 --2- 172.21.15.101:0/4116358708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe24072f10 0x7efe24073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:06.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.201+0000 7efe28bff700 1 -- 172.21.15.101:0/4116358708 >> 172.21.15.101:0/4116358708 conn(0x7efe240fb120 msgr2=0x7efe240fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:06.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.201+0000 7efe28bff700 1 -- 172.21.15.101:0/4116358708 shutdown_connections 2023-07-18T03:07:06.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.201+0000 7efe28bff700 1 -- 172.21.15.101:0/4116358708 wait complete. 2023-07-18T03:07:06.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.202+0000 7efe28bff700 1 Processor -- start 2023-07-18T03:07:06.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.202+0000 7efe28bff700 1 -- start start 2023-07-18T03:07:06.192 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.202+0000 7efe28bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efe24072f10 0x7efe24197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:06.192 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.202+0000 7efe28bff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe240744f0 0x7efe24197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:06.192 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.202+0000 7efe28bff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe24107990 0x7efe2419f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:06.192 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.202+0000 7efe28bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2419f5d0 con 0x7efe24072f10 2023-07-18T03:07:06.192 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.202+0000 7efe28bff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2419f710 con 0x7efe240744f0 2023-07-18T03:07:06.192 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.202+0000 7efe28bff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2419f8a0 con 0x7efe24107990 2023-07-18T03:07:06.192 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.203+0000 7efe2259c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efe24072f10 0x7efe24197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:06.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.203+0000 7efe21d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe240744f0 0x7efe24197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:06.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.203+0000 7efe2259c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efe24072f10 0x7efe24197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:36602/0 (socket says 172.21.15.101:36602) 2023-07-18T03:07:06.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.203+0000 7efe2259c700 1 -- 172.21.15.101:0/1683258084 learned_addr learned my addr 172.21.15.101:0/1683258084 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:06.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.203+0000 7efe22d9d700 1 --2- 172.21.15.101:0/1683258084 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe24107990 0x7efe2419f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:06.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.203+0000 7efe2259c700 1 -- 172.21.15.101:0/1683258084 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe24107990 msgr2=0x7efe2419f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:06.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.203+0000 7efe2259c700 1 --2- 172.21.15.101:0/1683258084 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe24107990 0x7efe2419f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:06.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.203+0000 7efe2259c700 1 -- 172.21.15.101:0/1683258084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe240744f0 msgr2=0x7efe24197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:06.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.203+0000 7efe2259c700 1 --2- 172.21.15.101:0/1683258084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe240744f0 0x7efe24197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:06.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.203+0000 7efe2259c700 1 -- 172.21.15.101:0/1683258084 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe18009520 con 0x7efe24072f10 2023-07-18T03:07:06.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.203+0000 7efe2259c700 1 --2- 172.21.15.101:0/1683258084 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efe24072f10 0x7efe24197a50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7efe14004740 tx=0x7efe14009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:06.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.204+0000 7efe137fe700 1 -- 172.21.15.101:0/1683258084 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1401d070 con 0x7efe24072f10 2023-07-18T03:07:06.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.204+0000 7efe137fe700 1 -- 172.21.15.101:0/1683258084 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe14004830 con 0x7efe24072f10 2023-07-18T03:07:06.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.204+0000 7efe28bff700 1 -- 172.21.15.101:0/1683258084 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe140095f0 con 0x7efe24072f10 2023-07-18T03:07:06.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.204+0000 7efe137fe700 1 -- 172.21.15.101:0/1683258084 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe140053c0 con 0x7efe24072f10 2023-07-18T03:07:06.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.204+0000 7efe28bff700 1 -- 172.21.15.101:0/1683258084 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe2419fdb0 con 0x7efe24072f10 2023-07-18T03:07:06.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.205+0000 7efe28bff700 1 -- 172.21.15.101:0/1683258084 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe241060d0 con 0x7efe24072f10 2023-07-18T03:07:06.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.207+0000 7efe137fe700 1 -- 172.21.15.101:0/1683258084 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7efe14005580 con 0x7efe24072f10 2023-07-18T03:07:06.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.208+0000 7efe137fe700 1 --2- 172.21.15.101:0/1683258084 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7efe0c0a11d0 0x7efe0c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:06.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.208+0000 7efe137fe700 1 -- 172.21.15.101:0/1683258084 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7efe140cddf0 con 0x7efe24072f10 2023-07-18T03:07:06.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.208+0000 7efe21d9b700 1 --2- 172.21.15.101:0/1683258084 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7efe0c0a11d0 0x7efe0c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:06.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.209+0000 7efe21d9b700 1 --2- 172.21.15.101:0/1683258084 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7efe0c0a11d0 0x7efe0c0a3680 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7efe18009df0 tx=0x7efe18009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:06.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.212+0000 7efe137fe700 1 -- 172.21.15.101:0/1683258084 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7efe1405a790 con 0x7efe24072f10 2023-07-18T03:07:06.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:05 smithi101 ceph-mon[126379]: pgmap v342: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 6 op/s 2023-07-18T03:07:06.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:05 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/4253742241' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:07:06.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:05 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/126823864' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:07:06.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:05 smithi155 ceph-mon[128885]: pgmap v342: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 6 op/s 2023-07-18T03:07:06.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:05 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/4253742241' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:07:06.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:05 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/126823864' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:07:06.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.412+0000 7efe28bff700 1 -- 172.21.15.101:0/1683258084 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efe240611c0 con 0x7efe24072f10 2023-07-18T03:07:06.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.412+0000 7efe137fe700 1 -- 172.21.15.101:0/1683258084 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7efe14097200 con 0x7efe24072f10 2023-07-18T03:07:06.400 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 64 2023-07-18T03:07:06.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.414+0000 7efe28bff700 1 -- 172.21.15.101:0/1683258084 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7efe0c0a11d0 msgr2=0x7efe0c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:06.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.414+0000 7efe28bff700 1 --2- 172.21.15.101:0/1683258084 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7efe0c0a11d0 0x7efe0c0a3680 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7efe18009df0 tx=0x7efe18009260 comp rx=0 tx=0).stop 2023-07-18T03:07:06.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.415+0000 7efe28bff700 1 -- 172.21.15.101:0/1683258084 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efe24072f10 msgr2=0x7efe24197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:06.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.415+0000 7efe28bff700 1 --2- 172.21.15.101:0/1683258084 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efe24072f10 0x7efe24197a50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7efe14004740 tx=0x7efe14009580 comp rx=0 tx=0).stop 2023-07-18T03:07:06.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.415+0000 7efe28bff700 1 -- 172.21.15.101:0/1683258084 shutdown_connections 2023-07-18T03:07:06.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.415+0000 7efe28bff700 1 --2- 172.21.15.101:0/1683258084 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7efe0c0a11d0 0x7efe0c0a3680 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:06.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.415+0000 7efe28bff700 1 --2- 172.21.15.101:0/1683258084 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe24107990 0x7efe2419f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:06.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.415+0000 7efe28bff700 1 --2- 172.21.15.101:0/1683258084 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efe24072f10 0x7efe24197a50 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:06.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.415+0000 7efe28bff700 1 --2- 172.21.15.101:0/1683258084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe240744f0 0x7efe24197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:06.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.415+0000 7efe28bff700 1 -- 172.21.15.101:0/1683258084 >> 172.21.15.101:0/1683258084 conn(0x7efe240fb120 msgr2=0x7efe240fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:06.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.415+0000 7efe28bff700 1 -- 172.21.15.101:0/1683258084 shutdown_connections 2023-07-18T03:07:06.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.415+0000 7efe28bff700 1 -- 172.21.15.101:0/1683258084 wait complete. 2023-07-18T03:07:06.421 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-18T03:07:06.443 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:05 smithi179 ceph-mon[129229]: pgmap v342: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 6 op/s 2023-07-18T03:07:06.444 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:05 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/4253742241' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:07:06.444 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:05 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/126823864' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:07:06.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.635+0000 7f75a13ff700 1 -- 172.21.15.101:0/1324613141 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f759c100cd0 msgr2=0x7f759c101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:06.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.635+0000 7f75a13ff700 1 --2- 172.21.15.101:0/1324613141 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f759c100cd0 0x7f759c101120 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7584009990 tx=0x7f7584009ca0 comp rx=0 tx=0).stop 2023-07-18T03:07:06.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.635+0000 7f75a13ff700 1 -- 172.21.15.101:0/1324613141 shutdown_connections 2023-07-18T03:07:06.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.635+0000 7f75a13ff700 1 --2- 172.21.15.101:0/1324613141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f759c0ffae0 0x7f759c0ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:06.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.635+0000 7f75a13ff700 1 --2- 172.21.15.101:0/1324613141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f759c101660 0x7f759c107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:06.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.635+0000 7f75a13ff700 1 --2- 172.21.15.101:0/1324613141 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f759c100cd0 0x7f759c101120 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:06.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.635+0000 7f75a13ff700 1 -- 172.21.15.101:0/1324613141 >> 172.21.15.101:0/1324613141 conn(0x7f759c0fb0e0 msgr2=0x7f759c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:06.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.635+0000 7f75a13ff700 1 -- 172.21.15.101:0/1324613141 shutdown_connections 2023-07-18T03:07:06.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.635+0000 7f75a13ff700 1 -- 172.21.15.101:0/1324613141 wait complete. 2023-07-18T03:07:06.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.636+0000 7f75a13ff700 1 Processor -- start 2023-07-18T03:07:06.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.636+0000 7f75a13ff700 1 -- start start 2023-07-18T03:07:06.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.636+0000 7f75a13ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f759c0ffae0 0x7f759c0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:06.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.636+0000 7f75a13ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f759c100cd0 0x7f759c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:06.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.637+0000 7f75a13ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f759c101660 0x7f759c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:06.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.637+0000 7f75a13ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759c0735e0 con 0x7f759c100cd0 2023-07-18T03:07:06.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.637+0000 7f75a13ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759c073750 con 0x7f759c101660 2023-07-18T03:07:06.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.637+0000 7f75a13ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759c19f8e0 con 0x7f759c0ffae0 2023-07-18T03:07:06.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.637+0000 7f7593fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f759c100cd0 0x7f759c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:06.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.637+0000 7f7593fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f759c100cd0 0x7f759c072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:36630/0 (socket says 172.21.15.101:36630) 2023-07-18T03:07:06.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.637+0000 7f7593fff700 1 -- 172.21.15.101:0/786431695 learned_addr learned my addr 172.21.15.101:0/786431695 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:06.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.637+0000 7f759b7fe700 1 --2- 172.21.15.101:0/786431695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f759c101660 0x7f759c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:06.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.637+0000 7f759affd700 1 --2- 172.21.15.101:0/786431695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f759c0ffae0 0x7f759c0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:06.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.637+0000 7f7593fff700 1 -- 172.21.15.101:0/786431695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f759c0ffae0 msgr2=0x7f759c0742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:06.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.637+0000 7f7593fff700 1 --2- 172.21.15.101:0/786431695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f759c0ffae0 0x7f759c0742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:06.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.637+0000 7f7593fff700 1 -- 172.21.15.101:0/786431695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f759c101660 msgr2=0x7f759c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:06.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.637+0000 7f7593fff700 1 --2- 172.21.15.101:0/786431695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f759c101660 0x7f759c072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:06.628 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.637+0000 7f7593fff700 1 -- 172.21.15.101:0/786431695 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75840095f0 con 0x7f759c100cd0 2023-07-18T03:07:06.628 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.637+0000 7f7593fff700 1 --2- 172.21.15.101:0/786431695 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f759c100cd0 0x7f759c072900 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7584004760 tx=0x7f7584004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:06.628 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.638+0000 7f7598ff9700 1 -- 172.21.15.101:0/786431695 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758401d070 con 0x7f759c100cd0 2023-07-18T03:07:06.628 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.638+0000 7f7598ff9700 1 -- 172.21.15.101:0/786431695 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7584004830 con 0x7f759c100cd0 2023-07-18T03:07:06.628 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.638+0000 7f75a13ff700 1 -- 172.21.15.101:0/786431695 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f759c19fb60 con 0x7f759c100cd0 2023-07-18T03:07:06.628 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.638+0000 7f7598ff9700 1 -- 172.21.15.101:0/786431695 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7584005350 con 0x7f759c100cd0 2023-07-18T03:07:06.628 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.638+0000 7f75a13ff700 1 -- 172.21.15.101:0/786431695 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f759c19ff80 con 0x7f759c100cd0 2023-07-18T03:07:06.629 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.639+0000 7f75a13ff700 1 -- 172.21.15.101:0/786431695 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f759c10b300 con 0x7f759c100cd0 2023-07-18T03:07:06.629 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.641+0000 7f7598ff9700 1 -- 172.21.15.101:0/786431695 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f7584005510 con 0x7f759c100cd0 2023-07-18T03:07:06.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.642+0000 7f7598ff9700 1 --2- 172.21.15.101:0/786431695 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f75880a11d0 0x7f75880a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:06.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.642+0000 7f7598ff9700 1 -- 172.21.15.101:0/786431695 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f75840cde00 con 0x7f759c100cd0 2023-07-18T03:07:06.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.642+0000 7f759affd700 1 --2- 172.21.15.101:0/786431695 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f75880a11d0 0x7f75880a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:06.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.643+0000 7f759affd700 1 --2- 172.21.15.101:0/786431695 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f75880a11d0 0x7f75880a3680 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f759c100b30 tx=0x7f758c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:06.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.646+0000 7f7598ff9700 1 -- 172.21.15.101:0/786431695 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f758405a790 con 0x7f759c100cd0 2023-07-18T03:07:06.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.843+0000 7f75a13ff700 1 -- 172.21.15.101:0/786431695 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f759c1069e0 con 0x7f759c100cd0 2023-07-18T03:07:06.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.844+0000 7f7598ff9700 1 -- 172.21.15.101:0/786431695 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7584097210 con 0x7f759c100cd0 2023-07-18T03:07:06.831 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 32 2023-07-18T03:07:06.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.846+0000 7f75a13ff700 1 -- 172.21.15.101:0/786431695 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f75880a11d0 msgr2=0x7f75880a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:06.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.846+0000 7f75a13ff700 1 --2- 172.21.15.101:0/786431695 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f75880a11d0 0x7f75880a3680 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f759c100b30 tx=0x7f758c009260 comp rx=0 tx=0).stop 2023-07-18T03:07:06.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.846+0000 7f75a13ff700 1 -- 172.21.15.101:0/786431695 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f759c100cd0 msgr2=0x7f759c072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:06.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.846+0000 7f75a13ff700 1 --2- 172.21.15.101:0/786431695 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f759c100cd0 0x7f759c072900 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7584004760 tx=0x7f7584004790 comp rx=0 tx=0).stop 2023-07-18T03:07:06.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.846+0000 7f75a13ff700 1 -- 172.21.15.101:0/786431695 shutdown_connections 2023-07-18T03:07:06.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.846+0000 7f75a13ff700 1 --2- 172.21.15.101:0/786431695 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f75880a11d0 0x7f75880a3680 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:06.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.846+0000 7f75a13ff700 1 --2- 172.21.15.101:0/786431695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f759c101660 0x7f759c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:06.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.846+0000 7f75a13ff700 1 --2- 172.21.15.101:0/786431695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f759c0ffae0 0x7f759c0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:06.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.846+0000 7f75a13ff700 1 --2- 172.21.15.101:0/786431695 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f759c100cd0 0x7f759c072900 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:06.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.846+0000 7f75a13ff700 1 -- 172.21.15.101:0/786431695 >> 172.21.15.101:0/786431695 conn(0x7f759c0fb0e0 msgr2=0x7f759c0fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:06.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.846+0000 7f75a13ff700 1 -- 172.21.15.101:0/786431695 shutdown_connections 2023-07-18T03:07:06.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:06.846+0000 7f75a13ff700 1 -- 172.21.15.101:0/786431695 wait complete. 2023-07-18T03:07:06.853 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:07:07.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.064+0000 7f82b2d9d700 1 -- 172.21.15.101:0/1819241462 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f82ac0ff9e0 msgr2=0x7f82ac0ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:07.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.064+0000 7f82b2d9d700 1 --2- 172.21.15.101:0/1819241462 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f82ac0ff9e0 0x7f82ac0ffdf0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f82a8009990 tx=0x7f82a8009ca0 comp rx=0 tx=0).stop 2023-07-18T03:07:07.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.065+0000 7f82b2d9d700 1 -- 172.21.15.101:0/1819241462 shutdown_connections 2023-07-18T03:07:07.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.065+0000 7f82b2d9d700 1 --2- 172.21.15.101:0/1819241462 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82ac101560 0x7f82ac107e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:07.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.065+0000 7f82b2d9d700 1 --2- 172.21.15.101:0/1819241462 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f82ac0ff9e0 0x7f82ac0ffdf0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:07.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.065+0000 7f82b2d9d700 1 --2- 172.21.15.101:0/1819241462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82ac100bd0 0x7f82ac101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:07.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.065+0000 7f82b2d9d700 1 -- 172.21.15.101:0/1819241462 >> 172.21.15.101:0/1819241462 conn(0x7f82ac0fafe0 msgr2=0x7f82ac0fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:07.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.065+0000 7f82b2d9d700 1 -- 172.21.15.101:0/1819241462 shutdown_connections 2023-07-18T03:07:07.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.065+0000 7f82b2d9d700 1 -- 172.21.15.101:0/1819241462 wait complete. 2023-07-18T03:07:07.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.066+0000 7f82b2d9d700 1 Processor -- start 2023-07-18T03:07:07.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.066+0000 7f82b2d9d700 1 -- start start 2023-07-18T03:07:07.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.066+0000 7f82b2d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82ac0ff9e0 0x7f82ac195560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:07.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.066+0000 7f82b2d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82ac100bd0 0x7f82ac195aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:07.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.066+0000 7f82b2d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f82ac101560 0x7f82ac19cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:07.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.066+0000 7f82b2d9d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82ac19d0e0 con 0x7f82ac101560 2023-07-18T03:07:07.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.066+0000 7f82b2d9d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82ac19d220 con 0x7f82ac0ff9e0 2023-07-18T03:07:07.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.066+0000 7f82b2d9d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82ac19d3b0 con 0x7f82ac100bd0 2023-07-18T03:07:07.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.066+0000 7f82b159a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82ac100bd0 0x7f82ac195aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:07.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.067+0000 7f82b259c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f82ac101560 0x7f82ac19cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:07.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.067+0000 7f82b159a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82ac100bd0 0x7f82ac195aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:45716/0 (socket says 172.21.15.101:45716) 2023-07-18T03:07:07.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.067+0000 7f82b159a700 1 -- 172.21.15.101:0/3293944976 learned_addr learned my addr 172.21.15.101:0/3293944976 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:07.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.067+0000 7f82b1d9b700 1 --2- 172.21.15.101:0/3293944976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82ac0ff9e0 0x7f82ac195560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:07.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.067+0000 7f82b259c700 1 -- 172.21.15.101:0/3293944976 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82ac100bd0 msgr2=0x7f82ac195aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:07.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.067+0000 7f82b259c700 1 --2- 172.21.15.101:0/3293944976 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82ac100bd0 0x7f82ac195aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:07.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.067+0000 7f82b259c700 1 -- 172.21.15.101:0/3293944976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82ac0ff9e0 msgr2=0x7f82ac195560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:07.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.067+0000 7f82b259c700 1 --2- 172.21.15.101:0/3293944976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82ac0ff9e0 0x7f82ac195560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:07.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.067+0000 7f82b259c700 1 -- 172.21.15.101:0/3293944976 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a80095f0 con 0x7f82ac101560 2023-07-18T03:07:07.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.067+0000 7f82b159a700 1 --2- 172.21.15.101:0/3293944976 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82ac100bd0 0x7f82ac195aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:07:07.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.067+0000 7f82b259c700 1 --2- 172.21.15.101:0/3293944976 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f82ac101560 0x7f82ac19cb10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f82a400efc0 tx=0x7f82a400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:07.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.067+0000 7f82a2ffd700 1 -- 172.21.15.101:0/3293944976 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a4018070 con 0x7f82ac101560 2023-07-18T03:07:07.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.067+0000 7f82a2ffd700 1 -- 172.21.15.101:0/3293944976 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82a400f040 con 0x7f82ac101560 2023-07-18T03:07:07.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.067+0000 7f82b2d9d700 1 -- 172.21.15.101:0/3293944976 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82ac19d630 con 0x7f82ac101560 2023-07-18T03:07:07.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.067+0000 7f82a2ffd700 1 -- 172.21.15.101:0/3293944976 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a400cd40 con 0x7f82ac101560 2023-07-18T03:07:07.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.068+0000 7f82b2d9d700 1 -- 172.21.15.101:0/3293944976 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82ac19db90 con 0x7f82ac101560 2023-07-18T03:07:07.059 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.069+0000 7f82b2d9d700 1 -- 172.21.15.101:0/3293944976 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82ac10b200 con 0x7f82ac101560 2023-07-18T03:07:07.059 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.070+0000 7f82a2ffd700 1 -- 172.21.15.101:0/3293944976 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f82a4003a00 con 0x7f82ac101560 2023-07-18T03:07:07.059 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.071+0000 7f82a2ffd700 1 --2- 172.21.15.101:0/3293944976 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f82980a12c0 0x7f82980a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:07.059 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.072+0000 7f82a2ffd700 1 -- 172.21.15.101:0/3293944976 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f82a40cc7b0 con 0x7f82ac101560 2023-07-18T03:07:07.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.072+0000 7f82b1d9b700 1 --2- 172.21.15.101:0/3293944976 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f82980a12c0 0x7f82980a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:07.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.072+0000 7f82b1d9b700 1 --2- 172.21.15.101:0/3293944976 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f82980a12c0 0x7f82980a3770 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f82a800b2e0 tx=0x7f82a8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:07.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.075+0000 7f82a2ffd700 1 -- 172.21.15.101:0/3293944976 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f82a4095bc0 con 0x7f82ac101560 2023-07-18T03:07:07.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:06 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3273783383' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:07:07.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:06 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1683258084' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:07:07.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:06 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/786431695' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-18T03:07:07.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:06 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/3273783383' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:07:07.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:06 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1683258084' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:07:07.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:06 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/786431695' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-18T03:07:07.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.318+0000 7f82b2d9d700 1 -- 172.21.15.101:0/3293944976 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82ac10b3c0 con 0x7f82ac101560 2023-07-18T03:07:07.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.319+0000 7f82a2ffd700 1 -- 172.21.15.101:0/3293944976 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10858 (secure 0 0 0) 0x7f82a40991e0 con 0x7f82ac101560 2023-07-18T03:07:07.306 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:07:07.306 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/4192846769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":4192846769},{"type":"v1","addr":"172.21.15.101:6839","nonce":4192846769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6841/3824340175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6840","nonce":3824340175},{"type":"v1","addr":"172.21.15.101:6841","nonce":3824340175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:04:59.361185+0000","modified":"2023-07-18T03:05:31.985959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.155:6839/2055462483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2055462483},{"type":"v1","addr":"172.21.15.155:6839","nonce":2055462483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:07:07.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.321+0000 7f82b2d9d700 1 -- 172.21.15.101:0/3293944976 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f82980a12c0 msgr2=0x7f82980a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:07.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.321+0000 7f82b2d9d700 1 --2- 172.21.15.101:0/3293944976 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f82980a12c0 0x7f82980a3770 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f82a800b2e0 tx=0x7f82a8005fb0 comp rx=0 tx=0).stop 2023-07-18T03:07:07.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.321+0000 7f82b2d9d700 1 -- 172.21.15.101:0/3293944976 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f82ac101560 msgr2=0x7f82ac19cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:07.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.321+0000 7f82b2d9d700 1 --2- 172.21.15.101:0/3293944976 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f82ac101560 0x7f82ac19cb10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f82a400efc0 tx=0x7f82a400c3a0 comp rx=0 tx=0).stop 2023-07-18T03:07:07.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.321+0000 7f82b2d9d700 1 -- 172.21.15.101:0/3293944976 shutdown_connections 2023-07-18T03:07:07.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.321+0000 7f82b2d9d700 1 --2- 172.21.15.101:0/3293944976 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f82980a12c0 0x7f82980a3770 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:07.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.321+0000 7f82b2d9d700 1 --2- 172.21.15.101:0/3293944976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82ac0ff9e0 0x7f82ac195560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:07.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.321+0000 7f82b2d9d700 1 --2- 172.21.15.101:0/3293944976 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f82ac101560 0x7f82ac19cb10 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:07.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.321+0000 7f82b2d9d700 1 --2- 172.21.15.101:0/3293944976 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82ac100bd0 0x7f82ac195aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:07.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.321+0000 7f82b2d9d700 1 -- 172.21.15.101:0/3293944976 >> 172.21.15.101:0/3293944976 conn(0x7f82ac0fafe0 msgr2=0x7f82ac0fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:07.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.321+0000 7f82b2d9d700 1 -- 172.21.15.101:0/3293944976 shutdown_connections 2023-07-18T03:07:07.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.321+0000 7f82b2d9d700 1 -- 172.21.15.101:0/3293944976 wait complete. 2023-07-18T03:07:07.314 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 24 2023-07-18T03:07:07.328 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:07:07.444 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:06 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/3273783383' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:07:07.444 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:06 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1683258084' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:07:07.445 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:06 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/786431695' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-18T03:07:07.524 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.537+0000 7fe592fff700 1 -- 172.21.15.101:0/2737015126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe58c0ffb00 msgr2=0x7fe58c0fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:07.525 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.537+0000 7fe592fff700 1 --2- 172.21.15.101:0/2737015126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe58c0ffb00 0x7fe58c0fff10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe580009990 tx=0x7fe580009ca0 comp rx=0 tx=0).stop 2023-07-18T03:07:07.525 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.537+0000 7fe592fff700 1 -- 172.21.15.101:0/2737015126 shutdown_connections 2023-07-18T03:07:07.525 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.537+0000 7fe592fff700 1 --2- 172.21.15.101:0/2737015126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe58c0ffb00 0x7fe58c0fff10 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:07.525 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.537+0000 7fe592fff700 1 --2- 172.21.15.101:0/2737015126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe58c101680 0x7fe58c107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:07.525 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.537+0000 7fe592fff700 1 --2- 172.21.15.101:0/2737015126 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe58c100cf0 0x7fe58c101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:07.525 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.537+0000 7fe592fff700 1 -- 172.21.15.101:0/2737015126 >> 172.21.15.101:0/2737015126 conn(0x7fe58c0fb0e0 msgr2=0x7fe58c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:07.525 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.537+0000 7fe592fff700 1 -- 172.21.15.101:0/2737015126 shutdown_connections 2023-07-18T03:07:07.526 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.537+0000 7fe592fff700 1 -- 172.21.15.101:0/2737015126 wait complete. 2023-07-18T03:07:07.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.538+0000 7fe592fff700 1 Processor -- start 2023-07-18T03:07:07.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.538+0000 7fe592fff700 1 -- start start 2023-07-18T03:07:07.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.538+0000 7fe592fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe58c0ffb00 0x7fe58c195820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:07.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.538+0000 7fe592fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe58c100cf0 0x7fe58c195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:07.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.538+0000 7fe592fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe58c101680 0x7fe58c19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:07.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.538+0000 7fe592fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58c19d3a0 con 0x7fe58c101680 2023-07-18T03:07:07.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.538+0000 7fe592fff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58c19d4e0 con 0x7fe58c0ffb00 2023-07-18T03:07:07.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.538+0000 7fe592fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58c19d670 con 0x7fe58c100cf0 2023-07-18T03:07:07.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.539+0000 7fe58bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe58c100cf0 0x7fe58c195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:07.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.539+0000 7fe590d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe58c0ffb00 0x7fe58c195820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:07.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.539+0000 7fe58bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe58c100cf0 0x7fe58c195d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:45740/0 (socket says 172.21.15.101:45740) 2023-07-18T03:07:07.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.539+0000 7fe58bfff700 1 -- 172.21.15.101:0/1192787490 learned_addr learned my addr 172.21.15.101:0/1192787490 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:07.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.539+0000 7fe59159c700 1 --2- 172.21.15.101:0/1192787490 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe58c101680 0x7fe58c19cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:07.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.539+0000 7fe590d9b700 1 -- 172.21.15.101:0/1192787490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe58c100cf0 msgr2=0x7fe58c195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:07.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.539+0000 7fe590d9b700 1 --2- 172.21.15.101:0/1192787490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe58c100cf0 0x7fe58c195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:07.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.539+0000 7fe590d9b700 1 -- 172.21.15.101:0/1192787490 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe58c101680 msgr2=0x7fe58c19cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:07.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.539+0000 7fe590d9b700 1 --2- 172.21.15.101:0/1192787490 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe58c101680 0x7fe58c19cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:07.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.539+0000 7fe590d9b700 1 -- 172.21.15.101:0/1192787490 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5800095f0 con 0x7fe58c0ffb00 2023-07-18T03:07:07.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.539+0000 7fe58bfff700 1 --2- 172.21.15.101:0/1192787490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe58c100cf0 0x7fe58c195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:07:07.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.539+0000 7fe59159c700 1 --2- 172.21.15.101:0/1192787490 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe58c101680 0x7fe58c19cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:07:07.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.539+0000 7fe590d9b700 1 --2- 172.21.15.101:0/1192787490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe58c0ffb00 0x7fe58c195820 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe5800098e0 tx=0x7fe580005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:07.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.540+0000 7fe589ffb700 1 -- 172.21.15.101:0/1192787490 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58001d070 con 0x7fe58c0ffb00 2023-07-18T03:07:07.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.540+0000 7fe589ffb700 1 -- 172.21.15.101:0/1192787490 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe580021410 con 0x7fe58c0ffb00 2023-07-18T03:07:07.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.540+0000 7fe592fff700 1 -- 172.21.15.101:0/1192787490 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe58c19d8f0 con 0x7fe58c0ffb00 2023-07-18T03:07:07.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.540+0000 7fe589ffb700 1 -- 172.21.15.101:0/1192787490 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe580017650 con 0x7fe58c0ffb00 2023-07-18T03:07:07.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.540+0000 7fe592fff700 1 -- 172.21.15.101:0/1192787490 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe58c19dd10 con 0x7fe58c0ffb00 2023-07-18T03:07:07.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.541+0000 7fe592fff700 1 -- 172.21.15.101:0/1192787490 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe58c106ff0 con 0x7fe58c0ffb00 2023-07-18T03:07:07.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.543+0000 7fe589ffb700 1 -- 172.21.15.101:0/1192787490 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fe580021580 con 0x7fe58c0ffb00 2023-07-18T03:07:07.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.544+0000 7fe589ffb700 1 --2- 172.21.15.101:0/1192787490 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe57c0a11a0 0x7fe57c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:07.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.544+0000 7fe589ffb700 1 -- 172.21.15.101:0/1192787490 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fe5800ce670 con 0x7fe58c0ffb00 2023-07-18T03:07:07.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.545+0000 7fe58bfff700 1 --2- 172.21.15.101:0/1192787490 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe57c0a11a0 0x7fe57c0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:07.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.545+0000 7fe58bfff700 1 --2- 172.21.15.101:0/1192787490 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe57c0a11a0 0x7fe57c0a3650 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe578011550 tx=0x7fe578009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:07.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.546+0000 7fe589ffb700 1 -- 172.21.15.101:0/1192787490 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe580097a80 con 0x7fe58c0ffb00 2023-07-18T03:07:07.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.746+0000 7fe592fff700 1 -- 172.21.15.101:0/1192787490 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe58c02ca20 con 0x7fe58c0ffb00 2023-07-18T03:07:07.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.748+0000 7fe589ffb700 1 -- 172.21.15.101:0/1192787490 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24725 (secure 0 0 0) 0x7fe58009b0a0 con 0x7fe58c0ffb00 2023-07-18T03:07:07.735 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:07:07.736 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":91,"fsid":"b1782daa-2516-11ee-9b34-001a4aab830c","created":"2023-07-18T02:57:22.972483+0000","modified":"2023-07-18T03:06:24.250777+0000","last_up_change":"2023-07-18T03:04:11.429530+0000","last_in_change":"2023-07-18T03:03:54.158340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:00:43.353489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:04:57.771616+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-18T03:04:58.769590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-18T03:05:02.358830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc76b5d1-38a0-4f1a-9335-756bfc40a62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6803","nonce":3610719929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6805","nonce":3610719929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6809","nonce":3610719929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6807","nonce":3610719929}]},"public_addr":"172.21.15.101:6803/3610719929","cluster_addr":"172.21.15.101:6805/3610719929","heartbeat_back_addr":"172.21.15.101:6809/3610719929","heartbeat_front_addr":"172.21.15.101:6807/3610719929","state":["exists","up"]},{"osd":1,"uuid":"2a299573-0b34-4af9-aaae-b523e5058ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6801","nonce":1030229579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6803","nonce":1030229579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6807","nonce":1030229579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6805","nonce":1030229579}]},"public_addr":"172.21.15.155:6801/1030229579","cluster_addr":"172.21.15.155:6803/1030229579","heartbeat_back_addr":"172.21.15.155:6807/1030229579","heartbeat_front_addr":"172.21.15.155:6805/1030229579","state":["exists","up"]},{"osd":2,"uuid":"123b3749-c636-48fa-9e33-6229b5575dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6801","nonce":1349487507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6803","nonce":1349487507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6807","nonce":1349487507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6805","nonce":1349487507}]},"public_addr":"172.21.15.179:6801/1349487507","cluster_addr":"172.21.15.179:6803/1349487507","heartbeat_back_addr":"172.21.15.179:6807/1349487507","heartbeat_front_addr":"172.21.15.179:6805/1349487507","state":["exists","up"]},{"osd":3,"uuid":"4b0c409d-2954-4127-833a-b196bf226d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6811","nonce":4226577463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6813","nonce":4226577463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6817","nonce":4226577463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6815","nonce":4226577463}]},"public_addr":"172.21.15.101:6811/4226577463","cluster_addr":"172.21.15.101:6813/4226577463","heartbeat_back_addr":"172.21.15.101:6817/4226577463","heartbeat_front_addr":"172.21.15.101:6815/4226577463","state":["exists","up"]},{"osd":4,"uuid":"2137f2df-984e-45ce-baf6-b4ac12c7e3c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6809","nonce":1951135805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6811","nonce":1951135805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6815","nonce":1951135805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6813","nonce":1951135805}]},"public_addr":"172.21.15.155:6809/1951135805","cluster_addr":"172.21.15.155:6811/1951135805","heartbeat_back_addr":"172.21.15.155:6815/1951135805","heartbeat_front_addr":"172.21.15.155:6813/1951135805","state":["exists","up"]},{"osd":5,"uuid":"1020bfc5-8c2f-482e-9588-66c7f98e46e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6809","nonce":2746037021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6811","nonce":2746037021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6815","nonce":2746037021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6813","nonce":2746037021}]},"public_addr":"172.21.15.179:6809/2746037021","cluster_addr":"172.21.15.179:6811/2746037021","heartbeat_back_addr":"172.21.15.179:6815/2746037021","heartbeat_front_addr":"172.21.15.179:6813/2746037021","state":["exists","up"]},{"osd":6,"uuid":"2cd99260-5c36-4603-9338-cf697970ce58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6819","nonce":958044931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6821","nonce":958044931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6825","nonce":958044931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6823","nonce":958044931}]},"public_addr":"172.21.15.101:6819/958044931","cluster_addr":"172.21.15.101:6821/958044931","heartbeat_back_addr":"172.21.15.101:6825/958044931","heartbeat_front_addr":"172.21.15.101:6823/958044931","state":["exists","up"]},{"osd":7,"uuid":"ed23fd0a-678f-4e15-b42c-cfe873315714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6817","nonce":3256322291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6819","nonce":3256322291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6823","nonce":3256322291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6821","nonce":3256322291}]},"public_addr":"172.21.15.155:6817/3256322291","cluster_addr":"172.21.15.155:6819/3256322291","heartbeat_back_addr":"172.21.15.155:6823/3256322291","heartbeat_front_addr":"172.21.15.155:6821/3256322291","state":["exists","up"]},{"osd":8,"uuid":"eca3d8e6-7baf-47a5-b991-328001c7ebef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6817","nonce":3520612559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6819","nonce":3520612559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6823","nonce":3520612559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6821","nonce":3520612559}]},"public_addr":"172.21.15.179:6817/3520612559","cluster_addr":"172.21.15.179:6819/3520612559","heartbeat_back_addr":"172.21.15.179:6823/3520612559","heartbeat_front_addr":"172.21.15.179:6821/3520612559","state":["exists","up"]},{"osd":9,"uuid":"465a7ffb-67e4-49ef-812c-1b96feacfebf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6827","nonce":2166787667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6829","nonce":2166787667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6833","nonce":2166787667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6831","nonce":2166787667}]},"public_addr":"172.21.15.101:6827/2166787667","cluster_addr":"172.21.15.101:6829/2166787667","heartbeat_back_addr":"172.21.15.101:6833/2166787667","heartbeat_front_addr":"172.21.15.101:6831/2166787667","state":["exists","up"]},{"osd":10,"uuid":"1897a027-f25d-434f-8df5-5fc2bd866578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6825","nonce":3682446349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6827","nonce":3682446349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6831","nonce":3682446349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6829","nonce":3682446349}]},"public_addr":"172.21.15.155:6825/3682446349","cluster_addr":"172.21.15.155:6827/3682446349","heartbeat_back_addr":"172.21.15.155:6831/3682446349","heartbeat_front_addr":"172.21.15.155:6829/3682446349","state":["exists","up"]},{"osd":11,"uuid":"b62effaf-45e2-4585-b8d0-db448f3474ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6825","nonce":2602640858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6827","nonce":2602640858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6831","nonce":2602640858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6829","nonce":2602640858}]},"public_addr":"172.21.15.179:6825/2602640858","cluster_addr":"172.21.15.179:6827/2602640858","heartbeat_back_addr":"172.21.15.179:6831/2602640858","heartbeat_front_addr":"172.21.15.179:6829/2602640858","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T02:59:56.074519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:19.358142+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:41.128966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:08.227054+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:31.442592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:52.751685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:19.251054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:40.698241+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:01.339150+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:27.955910+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:49.136980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:04:10.084982+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.101:6800/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:6801/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/604228291":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/4055187849":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/2844574324":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/1603248215":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6801/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6800/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/3307578182":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/373189792":"2023-07-19T02:57:56.243200+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:07:07.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.750+0000 7fe592fff700 1 -- 172.21.15.101:0/1192787490 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe57c0a11a0 msgr2=0x7fe57c0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:07.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.750+0000 7fe592fff700 1 --2- 172.21.15.101:0/1192787490 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe57c0a11a0 0x7fe57c0a3650 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe578011550 tx=0x7fe578009260 comp rx=0 tx=0).stop 2023-07-18T03:07:07.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.750+0000 7fe592fff700 1 -- 172.21.15.101:0/1192787490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe58c0ffb00 msgr2=0x7fe58c195820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:07.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.750+0000 7fe592fff700 1 --2- 172.21.15.101:0/1192787490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe58c0ffb00 0x7fe58c195820 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe5800098e0 tx=0x7fe580005740 comp rx=0 tx=0).stop 2023-07-18T03:07:07.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.750+0000 7fe592fff700 1 -- 172.21.15.101:0/1192787490 shutdown_connections 2023-07-18T03:07:07.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.750+0000 7fe592fff700 1 --2- 172.21.15.101:0/1192787490 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe57c0a11a0 0x7fe57c0a3650 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:07.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.750+0000 7fe592fff700 1 --2- 172.21.15.101:0/1192787490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe58c0ffb00 0x7fe58c195820 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:07.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.750+0000 7fe592fff700 1 --2- 172.21.15.101:0/1192787490 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe58c101680 0x7fe58c19cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:07.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.750+0000 7fe592fff700 1 --2- 172.21.15.101:0/1192787490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe58c100cf0 0x7fe58c195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:07.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.750+0000 7fe592fff700 1 -- 172.21.15.101:0/1192787490 >> 172.21.15.101:0/1192787490 conn(0x7fe58c0fb0e0 msgr2=0x7fe58c104130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:07.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.750+0000 7fe592fff700 1 -- 172.21.15.101:0/1192787490 shutdown_connections 2023-07-18T03:07:07.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.750+0000 7fe592fff700 1 -- 172.21.15.101:0/1192787490 wait complete. 2023-07-18T03:07:07.757 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-07-18T03:04:59.361185+0000', 'modified': '2023-07-18T03:05:31.985959+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24562, 'mds_1': 24557, 'mds_2': 24541}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24488': {'gid': 24488, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.155:6835/943645270', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 943645270}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 943645270}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24515': {'gid': 24515, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.155:6837/48029036', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6836', 'nonce': 48029036}, {'type': 'v1', 'addr': '172.21.15.155:6837', 'nonce': 48029036}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24529': {'gid': 24529, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.179:6835/1268903554', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6834', 'nonce': 1268903554}, {'type': 'v1', 'addr': '172.21.15.179:6835', 'nonce': 1268903554}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24541': {'gid': 24541, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.179:6837/3458593442', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6836', 'nonce': 3458593442}, {'type': 'v1', 'addr': '172.21.15.179:6837', 'nonce': 3458593442}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24557': {'gid': 24557, 'name': 'b', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.155:6839/2055462483', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6838', 'nonce': 2055462483}, {'type': 'v1', 'addr': '172.21.15.155:6839', 'nonce': 2055462483}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24562': {'gid': 24562, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.179:6839/2061828474', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6838', 'nonce': 2061828474}, {'type': 'v1', 'addr': '172.21.15.179:6839', 'nonce': 2061828474}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-18T03:07:07.758 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-07-18T03:07:07.759 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-07-18T03:07:07.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.972+0000 7fb7983ff700 1 -- 172.21.15.101:0/2045685915 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7901005a0 msgr2=0x7fb790102980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:07.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.972+0000 7fb7983ff700 1 --2- 172.21.15.101:0/2045685915 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7901005a0 0x7fb790102980 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb78c0098f0 tx=0x7fb78c009c00 comp rx=0 tx=0).stop 2023-07-18T03:07:07.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.972+0000 7fb7983ff700 1 -- 172.21.15.101:0/2045685915 shutdown_connections 2023-07-18T03:07:07.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.972+0000 7fb7983ff700 1 --2- 172.21.15.101:0/2045685915 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7901005a0 0x7fb790102980 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:07.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.972+0000 7fb7983ff700 1 --2- 172.21.15.101:0/2045685915 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb7901057e0 0x7fb790107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:07.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.972+0000 7fb7983ff700 1 --2- 172.21.15.101:0/2045685915 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb790102ec0 0x7fb7901052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:07.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.972+0000 7fb7983ff700 1 -- 172.21.15.101:0/2045685915 >> 172.21.15.101:0/2045685915 conn(0x7fb7900fa000 msgr2=0x7fb7900fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:07.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.972+0000 7fb7983ff700 1 -- 172.21.15.101:0/2045685915 shutdown_connections 2023-07-18T03:07:07.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.972+0000 7fb7983ff700 1 -- 172.21.15.101:0/2045685915 wait complete. 2023-07-18T03:07:07.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.973+0000 7fb7983ff700 1 Processor -- start 2023-07-18T03:07:07.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.973+0000 7fb7983ff700 1 -- start start 2023-07-18T03:07:07.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.974+0000 7fb7983ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7901005a0 0x7fb7901958a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:07.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.974+0000 7fb7983ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb790102ec0 0x7fb790195de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:07.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.974+0000 7fb7983ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb7901057e0 0x7fb79019ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:07.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.974+0000 7fb7983ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79019d420 con 0x7fb7901057e0 2023-07-18T03:07:07.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.974+0000 7fb7983ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79019d560 con 0x7fb790102ec0 2023-07-18T03:07:07.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.974+0000 7fb7983ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79019d6f0 con 0x7fb7901005a0 2023-07-18T03:07:07.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.974+0000 7fb79699c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb7901057e0 0x7fb79019ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:07.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.974+0000 7fb79699c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb7901057e0 0x7fb79019ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:36674/0 (socket says 172.21.15.101:36674) 2023-07-18T03:07:07.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.974+0000 7fb79699c700 1 -- 172.21.15.101:0/3011213845 learned_addr learned my addr 172.21.15.101:0/3011213845 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:07.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.974+0000 7fb79599a700 1 --2- 172.21.15.101:0/3011213845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb790102ec0 0x7fb790195de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:07.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.974+0000 7fb79619b700 1 --2- 172.21.15.101:0/3011213845 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7901005a0 0x7fb7901958a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:07.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.974+0000 7fb79699c700 1 -- 172.21.15.101:0/3011213845 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7901005a0 msgr2=0x7fb7901958a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:07.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.974+0000 7fb79699c700 1 --2- 172.21.15.101:0/3011213845 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7901005a0 0x7fb7901958a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:07.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.975+0000 7fb79699c700 1 -- 172.21.15.101:0/3011213845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb790102ec0 msgr2=0x7fb790195de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:07.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.975+0000 7fb79699c700 1 --2- 172.21.15.101:0/3011213845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb790102ec0 0x7fb790195de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:07.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.975+0000 7fb79699c700 1 -- 172.21.15.101:0/3011213845 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb780009520 con 0x7fb7901057e0 2023-07-18T03:07:07.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.975+0000 7fb79699c700 1 --2- 172.21.15.101:0/3011213845 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb7901057e0 0x7fb79019ce50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb78800e3c0 tx=0x7fb78800e6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:07.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.975+0000 7fb7877fe700 1 -- 172.21.15.101:0/3011213845 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7880090d0 con 0x7fb7901057e0 2023-07-18T03:07:07.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.975+0000 7fb7983ff700 1 -- 172.21.15.101:0/3011213845 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb788009d00 con 0x7fb7901057e0 2023-07-18T03:07:07.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.975+0000 7fb7877fe700 1 -- 172.21.15.101:0/3011213845 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb78800f040 con 0x7fb7901057e0 2023-07-18T03:07:07.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.975+0000 7fb7877fe700 1 -- 172.21.15.101:0/3011213845 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb788007c20 con 0x7fb7901057e0 2023-07-18T03:07:07.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.975+0000 7fb7983ff700 1 -- 172.21.15.101:0/3011213845 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb78c009520 con 0x7fb7901057e0 2023-07-18T03:07:07.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.978+0000 7fb7877fe700 1 -- 172.21.15.101:0/3011213845 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fb788004bd0 con 0x7fb7901057e0 2023-07-18T03:07:07.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.979+0000 7fb7abe0fb80 1 Processor -- start 2023-07-18T03:07:07.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.979+0000 7fb7abe0fb80 1 -- start start 2023-07-18T03:07:07.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.980+0000 7fb7abe0fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d96982480 0x562d96982890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:07.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.980+0000 7fb7abe0fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d96982dd0 0x562d96983690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:07.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.980+0000 7fb7abe0fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d96983c20 0x562d96984090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:07.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.980+0000 7fb7abe0fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d96887c60 con 0x562d96982dd0 2023-07-18T03:07:07.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.980+0000 7fb7abe0fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d968da9f0 con 0x562d96982480 2023-07-18T03:07:07.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.980+0000 7fb7abe0fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d968d3e80 con 0x562d96983c20 2023-07-18T03:07:07.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.980+0000 7fb7877fe700 1 --2- 172.21.15.101:0/3011213845 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb77c0a11d0 0x7fb77c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:07.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.980+0000 7fb79599a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d96982480 0x562d96982890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:07.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.980+0000 7fb79599a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d96982480 0x562d96982890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:36776/0 (socket says 172.21.15.101:36776) 2023-07-18T03:07:07.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.980+0000 7fb79599a700 1 -- 172.21.15.101:0/2941338708 learned_addr learned my addr 172.21.15.101:0/2941338708 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:07.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.980+0000 7fb7877fe700 1 -- 172.21.15.101:0/3011213845 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fb7880ccc20 con 0x7fb7901057e0 2023-07-18T03:07:07.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.980+0000 7fb79599a700 1 --2- 172.21.15.101:0/2941338708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d96983c20 0x562d96984090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:07.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.980+0000 7fb79699c700 1 --2- 172.21.15.101:0/3011213845 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb77c0a11d0 0x7fb77c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:07.970 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.981+0000 7fb79619b700 1 --2- 172.21.15.101:0/2941338708 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d96982dd0 0x562d96983690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:07.970 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.981+0000 7fb79599a700 1 -- 172.21.15.101:0/2941338708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d96983c20 msgr2=0x562d96984090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:07.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.981+0000 7fb79599a700 1 --2- 172.21.15.101:0/2941338708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d96983c20 0x562d96984090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:07.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.981+0000 7fb79599a700 1 -- 172.21.15.101:0/2941338708 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d96982dd0 msgr2=0x562d96983690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:07.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.981+0000 7fb79599a700 1 --2- 172.21.15.101:0/2941338708 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d96982dd0 0x562d96983690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:07.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.981+0000 7fb79599a700 1 -- 172.21.15.101:0/2941338708 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d96701d90 con 0x562d96982480 2023-07-18T03:07:07.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.981+0000 7fb79619b700 1 --2- 172.21.15.101:0/2941338708 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d96982dd0 0x562d96983690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:07:07.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.981+0000 7fb79699c700 1 --2- 172.21.15.101:0/3011213845 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb77c0a11d0 0x7fb77c0a3680 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb7880961f0 tx=0x7fb788009230 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:07.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.981+0000 7fb79599a700 1 --2- 172.21.15.101:0/2941338708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d96982480 0x562d96982890 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb780011a70 tx=0x7fb780009a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab54ba0661b1d650 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:07.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.983+0000 7fb784bfe700 1 -- 172.21.15.101:0/2941338708 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb780011110 con 0x562d96982480 2023-07-18T03:07:07.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.984+0000 7fb784bfe700 1 -- 172.21.15.101:0/2941338708 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7800044f0 con 0x562d96982480 2023-07-18T03:07:07.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.984+0000 7fb784bfe700 1 -- 172.21.15.101:0/2941338708 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78000fdc0 con 0x562d96982480 2023-07-18T03:07:07.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.984+0000 7fb7abe0fb80 1 -- 172.21.15.101:0/2941338708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d96982480 msgr2=0x562d96982890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:07.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.984+0000 7fb7abe0fb80 1 --2- 172.21.15.101:0/2941338708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d96982480 0x562d96982890 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb780011a70 tx=0x7fb780009a10 comp rx=0 tx=0).stop 2023-07-18T03:07:07.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.984+0000 7fb7abe0fb80 1 -- 172.21.15.101:0/2941338708 shutdown_connections 2023-07-18T03:07:07.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.984+0000 7fb7abe0fb80 1 --2- 172.21.15.101:0/2941338708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d96982480 0x562d96982890 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:07.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.984+0000 7fb7abe0fb80 1 --2- 172.21.15.101:0/2941338708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d96983c20 0x562d96984090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:07.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.984+0000 7fb7abe0fb80 1 --2- 172.21.15.101:0/2941338708 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d96982dd0 0x562d96983690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:07.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.984+0000 7fb7abe0fb80 1 -- 172.21.15.101:0/2941338708 >> 172.21.15.101:0/2941338708 conn(0x562d96977d50 msgr2=0x562d9697d180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:07.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.985+0000 7fb7abe0fb80 1 -- 172.21.15.101:0/2941338708 shutdown_connections 2023-07-18T03:07:07.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.985+0000 7fb7abe0fb80 1 -- 172.21.15.101:0/2941338708 wait complete. 2023-07-18T03:07:07.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.985+0000 7fb7abe0fb80 1 -- start start 2023-07-18T03:07:07.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.985+0000 7fb7abe0fb80 1 Processor -- start 2023-07-18T03:07:07.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.986+0000 7fb7abe0fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d96982480 0x562d9698c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:07.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.986+0000 7fb7abe0fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d96982dd0 0x562d9698c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:07.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.986+0000 7fb7abe0fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d96983c20 0x562d969936f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:07.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.986+0000 7fb7abe0fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d96873a70 con 0x562d96982dd0 2023-07-18T03:07:07.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.986+0000 7fb7abe0fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d96873770 con 0x562d96983c20 2023-07-18T03:07:07.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.986+0000 7fb7abe0fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d967578d0 con 0x562d96982480 2023-07-18T03:07:07.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.986+0000 7fb7abe0fb80 10 client.0 _ll_register_callbacks cb 0x562d96871710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:07:07.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.986+0000 7fb79619b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d96982dd0 0x562d9698c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:07.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.986+0000 7fb79619b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d96982dd0 0x562d9698c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:36684/0 (socket says 172.21.15.101:36684) 2023-07-18T03:07:07.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.986+0000 7fb79619b700 1 -- 172.21.15.101:0/1833576304 learned_addr learned my addr 172.21.15.101:0/1833576304 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:07.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.986+0000 7fb79599a700 1 --2- 172.21.15.101:0/1833576304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d96982480 0x562d9698c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:07.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.986+0000 7fb79599a700 1 --2- 172.21.15.101:0/1833576304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d96983c20 0x562d969936f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:07.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.986+0000 7fb79619b700 1 -- 172.21.15.101:0/1833576304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d96982480 msgr2=0x562d9698c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:07.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.986+0000 7fb79619b700 1 --2- 172.21.15.101:0/1833576304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d96982480 0x562d9698c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:07.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.986+0000 7fb79619b700 1 -- 172.21.15.101:0/1833576304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d96983c20 msgr2=0x562d969936f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:07.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.986+0000 7fb79619b700 1 --2- 172.21.15.101:0/1833576304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d96983c20 0x562d969936f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:07.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.986+0000 7fb79619b700 1 -- 172.21.15.101:0/1833576304 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb780019040 con 0x562d96982dd0 2023-07-18T03:07:07.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.986+0000 7fb79599a700 1 --2- 172.21.15.101:0/1833576304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d96983c20 0x562d969936f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:07:07.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.987+0000 7fb79619b700 1 --2- 172.21.15.101:0/1833576304 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d96982dd0 0x562d9698c6b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb78c0098f0 tx=0x7fb78c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e7099556b037417 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:07.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.987+0000 7fb779ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:07:07.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.987+0000 7fb779ffb700 1 -- 172.21.15.101:0/1833576304 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78c01c070 con 0x562d96982dd0 2023-07-18T03:07:07.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.987+0000 7fb7abe0fb80 1 -- 172.21.15.101:0/1833576304 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562d9682d4a0 con 0x562d96982dd0 2023-07-18T03:07:07.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.987+0000 7fb779ffb700 1 -- 172.21.15.101:0/1833576304 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fb78c005240 con 0x562d96982dd0 2023-07-18T03:07:07.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.987+0000 7fb779ffb700 1 -- 172.21.15.101:0/1833576304 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb78c00bbf0 con 0x562d96982dd0 2023-07-18T03:07:07.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.987+0000 7fb7abe0fb80 10 client.15255 fetch_fsmap learned FSMap version 24 2023-07-18T03:07:07.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.987+0000 7fb7abe0fb80 1 -- 172.21.15.101:0/1833576304 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x562d966eac10 con 0x562d96982dd0 2023-07-18T03:07:07.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.987+0000 7fb779ffb700 1 -- 172.21.15.101:0/1833576304 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562d9682d4a0 con 0x562d96982dd0 2023-07-18T03:07:07.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.988+0000 7fb779ffb700 1 -- 172.21.15.101:0/1833576304 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb78c015020 con 0x562d96982dd0 2023-07-18T03:07:07.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.988+0000 7fb7abe0fb80 10 client.15255 fetch_fsmap finished waiting for FSMap version 24 2023-07-18T03:07:07.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.988+0000 7fb7abe0fb80 10 client.15255 resolve_mds: resolved name 'c' to daemon mds.c 2023-07-18T03:07:07.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.988+0000 7fb7abe0fb80 20 client.15255 populate_metadata read hostname 'smithi101' 2023-07-18T03:07:07.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.988+0000 7fb7abe0fb80 1 --2- 172.21.15.101:0/1833576304 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x562d96997440 0x562d969967f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:07.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.988+0000 7fb7abe0fb80 4 client.15255 mds_command: new command op to 24541 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:07:07.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.988+0000 7fb7abe0fb80 1 -- 172.21.15.101:0/1833576304 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562d968cce60 con 0x562d96997440 2023-07-18T03:07:07.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.988+0000 7fb79599a700 1 --2- 172.21.15.101:0/1833576304 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x562d96997440 0x562d969967f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:07.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.989+0000 7fb79599a700 1 --2- 172.21.15.101:0/1833576304 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x562d96997440 0x562d969967f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=16c7a6ecc021e087 server_cookie=1ffdc4729ae204b6 in_seq=0 out_seq=0 2023-07-18T03:07:07.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.989+0000 7fb779ffb700 10 client.15255 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:07:07.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.990+0000 7fb779ffb700 1 -- 172.21.15.101:0/1833576304 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562d968cce60 con 0x562d96997440 2023-07-18T03:07:07.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.990+0000 7fb779ffb700 10 client.15255 handle_command_reply: tid=0 2023-07-18T03:07:07.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.990+0000 7fb7abe0fb80 1 client.15255 shutdown 2023-07-18T03:07:07.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.990+0000 7fb7abe0fb80 1 -- 172.21.15.101:0/1833576304 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d96982dd0 msgr2=0x562d9698c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:07.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.990+0000 7fb7abe0fb80 1 --2- 172.21.15.101:0/1833576304 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d96982dd0 0x562d9698c6b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb78c0098f0 tx=0x7fb78c00b840 comp rx=0 tx=0).stop 2023-07-18T03:07:07.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.990+0000 7fb7abe0fb80 1 -- 172.21.15.101:0/1833576304 shutdown_connections 2023-07-18T03:07:07.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.990+0000 7fb7abe0fb80 1 --2- 172.21.15.101:0/1833576304 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x562d96997440 0x562d969967f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:07.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.990+0000 7fb7abe0fb80 1 --2- 172.21.15.101:0/1833576304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d96983c20 0x562d969936f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:07.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.990+0000 7fb7abe0fb80 1 --2- 172.21.15.101:0/1833576304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d96982480 0x562d9698c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:07.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.990+0000 7fb7abe0fb80 1 --2- 172.21.15.101:0/1833576304 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d96982dd0 0x562d9698c6b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:07.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.990+0000 7fb7abe0fb80 1 -- 172.21.15.101:0/1833576304 >> 172.21.15.101:0/1833576304 conn(0x562d96977d50 msgr2=0x562d9697c860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:07.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.990+0000 7fb779ffb700 0 client.15255 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:07:07.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.990+0000 7fb7abe0fb80 1 -- 172.21.15.101:0/1833576304 shutdown_connections 2023-07-18T03:07:07.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.990+0000 7fb7abe0fb80 1 -- 172.21.15.101:0/1833576304 wait complete. 2023-07-18T03:07:07.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:07.991+0000 7fb7abe0fb80 20 client.15255 trim_cache size 0 max 16384 2023-07-18T03:07:07.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.008+0000 7fb7abe0fb80 1 Processor -- start 2023-07-18T03:07:07.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.009+0000 7fb7abe0fb80 1 -- start start 2023-07-18T03:07:07.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.009+0000 7fb7abe0fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d96982480 0x562d9697c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:07.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.009+0000 7fb7abe0fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d96982dd0 0x562d969936b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:07.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.009+0000 7fb7abe0fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d96983c20 0x562d9698a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:07.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.009+0000 7fb7abe0fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d968cce60 con 0x562d96982480 2023-07-18T03:07:07.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.009+0000 7fb7abe0fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d9682d4a0 con 0x562d96982dd0 2023-07-18T03:07:07.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.009+0000 7fb7abe0fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d9697ec00 con 0x562d96983c20 2023-07-18T03:07:08.001 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.009+0000 7fb79599a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d96982480 0x562d9697c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:08.001 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.009+0000 7fb79599a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d96982480 0x562d9697c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:36700/0 (socket says 172.21.15.101:36700) 2023-07-18T03:07:08.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.009+0000 7fb79599a700 1 -- 172.21.15.101:0/3621263518 learned_addr learned my addr 172.21.15.101:0/3621263518 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:08.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.009+0000 7fb79619b700 1 --2- 172.21.15.101:0/3621263518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d96982dd0 0x562d969936b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:08.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.009+0000 7fb79599a700 1 --2- 172.21.15.101:0/3621263518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d96983c20 0x562d9698a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:08.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.010+0000 7fb79599a700 1 -- 172.21.15.101:0/3621263518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d96983c20 msgr2=0x562d9698a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:08.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.010+0000 7fb79599a700 1 --2- 172.21.15.101:0/3621263518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d96983c20 0x562d9698a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.010+0000 7fb79599a700 1 -- 172.21.15.101:0/3621263518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d96982dd0 msgr2=0x562d969936b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:08.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.010+0000 7fb79599a700 1 --2- 172.21.15.101:0/3621263518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d96982dd0 0x562d969936b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.010+0000 7fb79599a700 1 -- 172.21.15.101:0/3621263518 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d96728200 con 0x562d96982480 2023-07-18T03:07:08.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.010+0000 7fb79619b700 1 --2- 172.21.15.101:0/3621263518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d96982dd0 0x562d969936b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:07:08.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.010+0000 7fb79599a700 1 --2- 172.21.15.101:0/3621263518 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d96982480 0x562d9697c860 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb780011a40 tx=0x7fb780004660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8ae8d55b4caa77d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:08.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.010+0000 7fb7853ff700 1 -- 172.21.15.101:0/3621263518 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb780011210 con 0x562d96982480 2023-07-18T03:07:08.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.010+0000 7fb7853ff700 1 -- 172.21.15.101:0/3621263518 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb780011370 con 0x562d96982480 2023-07-18T03:07:08.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.010+0000 7fb7853ff700 1 -- 172.21.15.101:0/3621263518 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7800054b0 con 0x562d96982480 2023-07-18T03:07:08.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.010+0000 7fb7abe0fb80 1 -- 172.21.15.101:0/3621263518 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d96982480 msgr2=0x562d9697c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:08.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.011+0000 7fb7abe0fb80 1 --2- 172.21.15.101:0/3621263518 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d96982480 0x562d9697c860 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb780011a40 tx=0x7fb780004660 comp rx=0 tx=0).stop 2023-07-18T03:07:08.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.011+0000 7fb7abe0fb80 1 -- 172.21.15.101:0/3621263518 shutdown_connections 2023-07-18T03:07:08.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.011+0000 7fb7abe0fb80 1 --2- 172.21.15.101:0/3621263518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d96983c20 0x562d9698a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.011+0000 7fb7abe0fb80 1 --2- 172.21.15.101:0/3621263518 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d96982480 0x562d9697c860 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.011+0000 7fb7abe0fb80 1 --2- 172.21.15.101:0/3621263518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d96982dd0 0x562d969936b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.011+0000 7fb7abe0fb80 1 -- 172.21.15.101:0/3621263518 >> 172.21.15.101:0/3621263518 conn(0x562d96977d50 msgr2=0x562d9698ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:08.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.011+0000 7fb7abe0fb80 1 -- 172.21.15.101:0/3621263518 shutdown_connections 2023-07-18T03:07:08.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.011+0000 7fb7abe0fb80 1 -- 172.21.15.101:0/3621263518 wait complete. 2023-07-18T03:07:08.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.011+0000 7fb7abe0fb80 1 -- start start 2023-07-18T03:07:08.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.011+0000 7fb7abe0fb80 1 Processor -- start 2023-07-18T03:07:08.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.012+0000 7fb7abe0fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d96982480 0x562d96990930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:08.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.012+0000 7fb7abe0fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d96982dd0 0x562d96990e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:08.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.012+0000 7fb7abe0fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d96983c20 0x562d96991d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:08.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.012+0000 7fb7abe0fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d969922e0 con 0x562d96982dd0 2023-07-18T03:07:08.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.012+0000 7fb7abe0fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d96992450 con 0x562d96983c20 2023-07-18T03:07:08.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.012+0000 7fb7abe0fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d96992610 con 0x562d96982480 2023-07-18T03:07:08.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.012+0000 7fb7abe0fb80 10 client.0 _ll_register_callbacks cb 0x7fb78c009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:07:08.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.012+0000 7fb79619b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d96982dd0 0x562d96990e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:08.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.012+0000 7fb79619b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d96982dd0 0x562d96990e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:36716/0 (socket says 172.21.15.101:36716) 2023-07-18T03:07:08.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.012+0000 7fb79619b700 1 -- 172.21.15.101:0/2013114899 learned_addr learned my addr 172.21.15.101:0/2013114899 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:08.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.012+0000 7fb79599a700 1 --2- 172.21.15.101:0/2013114899 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d96982480 0x562d96990930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:08.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.012+0000 7fb79599a700 1 --2- 172.21.15.101:0/2013114899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d96983c20 0x562d96991d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:08.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.012+0000 7fb79619b700 1 -- 172.21.15.101:0/2013114899 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d96982480 msgr2=0x562d96990930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:08.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.012+0000 7fb79619b700 1 --2- 172.21.15.101:0/2013114899 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d96982480 0x562d96990930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.012+0000 7fb79619b700 1 -- 172.21.15.101:0/2013114899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d96983c20 msgr2=0x562d96991d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:08.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.012+0000 7fb79619b700 1 --2- 172.21.15.101:0/2013114899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d96983c20 0x562d96991d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.012+0000 7fb79619b700 1 -- 172.21.15.101:0/2013114899 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d966eac10 con 0x562d96982dd0 2023-07-18T03:07:08.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.012+0000 7fb79599a700 1 --2- 172.21.15.101:0/2013114899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d96983c20 0x562d96991d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:07:08.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.012+0000 7fb79619b700 1 --2- 172.21.15.101:0/2013114899 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d96982dd0 0x562d96990e70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb78c00c010 tx=0x7fb78c003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39e5483a903d8f18 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:08.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.012+0000 7fb779ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:07:08.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.013+0000 7fb779ffb700 1 -- 172.21.15.101:0/2013114899 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78c01c040 con 0x562d96982dd0 2023-07-18T03:07:08.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.013+0000 7fb7abe0fb80 1 -- 172.21.15.101:0/2013114899 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb7800044f0 con 0x562d96982dd0 2023-07-18T03:07:08.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.013+0000 7fb779ffb700 1 -- 172.21.15.101:0/2013114899 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fb78c000e40 con 0x562d96982dd0 2023-07-18T03:07:08.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.013+0000 7fb779ffb700 1 -- 172.21.15.101:0/2013114899 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb78c003c00 con 0x562d96982dd0 2023-07-18T03:07:08.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.013+0000 7fb7abe0fb80 10 client.15261 fetch_fsmap learned FSMap version 24 2023-07-18T03:07:08.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.013+0000 7fb7abe0fb80 1 -- 172.21.15.101:0/2013114899 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x562d96701d90 con 0x562d96982dd0 2023-07-18T03:07:08.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.013+0000 7fb779ffb700 1 -- 172.21.15.101:0/2013114899 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7800044f0 con 0x562d96982dd0 2023-07-18T03:07:08.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.013+0000 7fb779ffb700 1 -- 172.21.15.101:0/2013114899 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb78c03d070 con 0x562d96982dd0 2023-07-18T03:07:08.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.013+0000 7fb7abe0fb80 10 client.15261 fetch_fsmap finished waiting for FSMap version 24 2023-07-18T03:07:08.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.013+0000 7fb7abe0fb80 10 client.15261 resolve_mds: resolved name 'c' to daemon mds.c 2023-07-18T03:07:08.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.013+0000 7fb7abe0fb80 20 client.15261 populate_metadata read hostname 'smithi101' 2023-07-18T03:07:08.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.013+0000 7fb7abe0fb80 1 --2- 172.21.15.101:0/2013114899 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x562d969f04c0 0x562d969ed2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:08.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.013+0000 7fb7abe0fb80 4 client.15261 mds_command: new command op to 24541 tid=0[{"prefix": "status"}] 2023-07-18T03:07:08.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.013+0000 7fb7abe0fb80 1 -- 172.21.15.101:0/2013114899 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562d9688f000 con 0x562d969f04c0 2023-07-18T03:07:08.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.014+0000 7fb79599a700 1 --2- 172.21.15.101:0/2013114899 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x562d969f04c0 0x562d969ed2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:08.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.014+0000 7fb79599a700 1 --2- 172.21.15.101:0/2013114899 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x562d969f04c0 0x562d969ed2b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=29d0363f21f29966 server_cookie=74d5db3cb50356b9 in_seq=0 out_seq=0 2023-07-18T03:07:08.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.014+0000 7fb779ffb700 10 client.15261 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:07:08.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.015+0000 7fb779ffb700 1 -- 172.21.15.101:0/2013114899 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x562d9688f000 con 0x562d969f04c0 2023-07-18T03:07:08.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.015+0000 7fb779ffb700 10 client.15261 handle_command_reply: tid=0 2023-07-18T03:07:08.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.015+0000 7fb7abe0fb80 1 client.15261 shutdown 2023-07-18T03:07:08.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.015+0000 7fb7abe0fb80 1 -- 172.21.15.101:0/2013114899 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d96982dd0 msgr2=0x562d96990e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:08.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.015+0000 7fb7abe0fb80 1 --2- 172.21.15.101:0/2013114899 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d96982dd0 0x562d96990e70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb78c00c010 tx=0x7fb78c003960 comp rx=0 tx=0).stop 2023-07-18T03:07:08.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.015+0000 7fb7abe0fb80 1 -- 172.21.15.101:0/2013114899 shutdown_connections 2023-07-18T03:07:08.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.015+0000 7fb7abe0fb80 1 --2- 172.21.15.101:0/2013114899 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x562d969f04c0 0x562d969ed2b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.015+0000 7fb7abe0fb80 1 --2- 172.21.15.101:0/2013114899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d96983c20 0x562d96991d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.015+0000 7fb779ffb700 0 client.15261 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:07:08.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.015+0000 7fb7abe0fb80 1 --2- 172.21.15.101:0/2013114899 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d96982480 0x562d96990930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.015+0000 7fb7abe0fb80 1 --2- 172.21.15.101:0/2013114899 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d96982dd0 0x562d96990e70 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.015+0000 7fb7abe0fb80 1 -- 172.21.15.101:0/2013114899 >> 172.21.15.101:0/2013114899 conn(0x562d96977d50 msgr2=0x562d9697c860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:08.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.015+0000 7fb7abe0fb80 1 -- 172.21.15.101:0/2013114899 shutdown_connections 2023-07-18T03:07:08.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.015+0000 7fb7abe0fb80 1 -- 172.21.15.101:0/2013114899 wait complete. 2023-07-18T03:07:08.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.015+0000 7fb7abe0fb80 20 client.15261 trim_cache size 0 max 16384 2023-07-18T03:07:08.014 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:07:08.014 INFO:teuthology.orchestra.run.smithi101.stdout: "cluster_fsid": "b1782daa-2516-11ee-9b34-001a4aab830c", 2023-07-18T03:07:08.015 INFO:teuthology.orchestra.run.smithi101.stdout: "whoami": 2, 2023-07-18T03:07:08.015 INFO:teuthology.orchestra.run.smithi101.stdout: "id": 24541, 2023-07-18T03:07:08.015 INFO:teuthology.orchestra.run.smithi101.stdout: "want_state": "up:active", 2023-07-18T03:07:08.015 INFO:teuthology.orchestra.run.smithi101.stdout: "state": "up:active", 2023-07-18T03:07:08.015 INFO:teuthology.orchestra.run.smithi101.stdout: "fs_name": "cephfs", 2023-07-18T03:07:08.015 INFO:teuthology.orchestra.run.smithi101.stdout: "rank_uptime": 97.034400485000006, 2023-07-18T03:07:08.015 INFO:teuthology.orchestra.run.smithi101.stdout: "mdsmap_epoch": 24, 2023-07-18T03:07:08.016 INFO:teuthology.orchestra.run.smithi101.stdout: "osdmap_epoch": 91, 2023-07-18T03:07:08.016 INFO:teuthology.orchestra.run.smithi101.stdout: "osdmap_epoch_barrier": 89, 2023-07-18T03:07:08.016 INFO:teuthology.orchestra.run.smithi101.stdout: "uptime": 147.27828572999999 2023-07-18T03:07:08.016 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:07:08.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.016+0000 7fb7983ff700 1 -- 172.21.15.101:0/3011213845 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb77c0a11d0 msgr2=0x7fb77c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:08.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.016+0000 7fb7983ff700 1 --2- 172.21.15.101:0/3011213845 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb77c0a11d0 0x7fb77c0a3680 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb7880961f0 tx=0x7fb788009230 comp rx=0 tx=0).stop 2023-07-18T03:07:08.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.016+0000 7fb7983ff700 1 -- 172.21.15.101:0/3011213845 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb7901057e0 msgr2=0x7fb79019ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:08.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.016+0000 7fb7983ff700 1 --2- 172.21.15.101:0/3011213845 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb7901057e0 0x7fb79019ce50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb78800e3c0 tx=0x7fb78800e6d0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.016+0000 7fb7983ff700 1 -- 172.21.15.101:0/3011213845 shutdown_connections 2023-07-18T03:07:08.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.016+0000 7fb7983ff700 1 --2- 172.21.15.101:0/3011213845 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb77c0a11d0 0x7fb77c0a3680 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.016+0000 7fb7983ff700 1 --2- 172.21.15.101:0/3011213845 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb7901057e0 0x7fb79019ce50 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.016+0000 7fb7983ff700 1 --2- 172.21.15.101:0/3011213845 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7901005a0 0x7fb7901958a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.016+0000 7fb7983ff700 1 --2- 172.21.15.101:0/3011213845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb790102ec0 0x7fb790195de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.016+0000 7fb7983ff700 1 -- 172.21.15.101:0/3011213845 >> 172.21.15.101:0/3011213845 conn(0x7fb7900fa000 msgr2=0x7fb790101220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:08.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.016+0000 7fb7983ff700 1 -- 172.21.15.101:0/3011213845 shutdown_connections 2023-07-18T03:07:08.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.016+0000 7fb7983ff700 1 -- 172.21.15.101:0/3011213845 wait complete. 2023-07-18T03:07:08.019 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:07 smithi101 ceph-mon[126379]: pgmap v343: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 426 B/s wr, 6 op/s 2023-07-18T03:07:08.019 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:07 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3293944976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:07:08.019 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:07 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1192787490' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:07:08.021 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-07-18T03:07:08.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.231+0000 7fb264bff700 1 -- 172.21.15.101:0/2068914170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb260074550 msgr2=0x7fb2600729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:08.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.231+0000 7fb264bff700 1 --2- 172.21.15.101:0/2068914170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb260074550 0x7fb2600729a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb250009990 tx=0x7fb250009ca0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.231+0000 7fb264bff700 1 -- 172.21.15.101:0/2068914170 shutdown_connections 2023-07-18T03:07:08.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.231+0000 7fb264bff700 1 --2- 172.21.15.101:0/2068914170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb260074550 0x7fb2600729a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.231+0000 7fb264bff700 1 --2- 172.21.15.101:0/2068914170 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb2601079a0 0x7fb260109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.231+0000 7fb264bff700 1 --2- 172.21.15.101:0/2068914170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb260072f70 0x7fb2600733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.231+0000 7fb264bff700 1 -- 172.21.15.101:0/2068914170 >> 172.21.15.101:0/2068914170 conn(0x7fb2600fb0e0 msgr2=0x7fb2600fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:08.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.231+0000 7fb264bff700 1 -- 172.21.15.101:0/2068914170 shutdown_connections 2023-07-18T03:07:08.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.231+0000 7fb264bff700 1 -- 172.21.15.101:0/2068914170 wait complete. 2023-07-18T03:07:08.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.232+0000 7fb264bff700 1 Processor -- start 2023-07-18T03:07:08.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.232+0000 7fb264bff700 1 -- start start 2023-07-18T03:07:08.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.233+0000 7fb264bff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb260072f70 0x7fb260197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:08.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.233+0000 7fb264bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb260074550 0x7fb260197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:08.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.233+0000 7fb264bff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2601079a0 0x7fb26019f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:08.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.233+0000 7fb264bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26019f620 con 0x7fb260074550 2023-07-18T03:07:08.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.233+0000 7fb264bff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26019f760 con 0x7fb2601079a0 2023-07-18T03:07:08.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.233+0000 7fb264bff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26019f8f0 con 0x7fb260072f70 2023-07-18T03:07:08.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.233+0000 7fb25e59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb260072f70 0x7fb260197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:08.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.233+0000 7fb25e59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb260072f70 0x7fb260197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:45806/0 (socket says 172.21.15.101:45806) 2023-07-18T03:07:08.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.233+0000 7fb25e59c700 1 -- 172.21.15.101:0/3136757488 learned_addr learned my addr 172.21.15.101:0/3136757488 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:08.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.233+0000 7fb25dd9b700 1 --2- 172.21.15.101:0/3136757488 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb260074550 0x7fb260197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:08.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.233+0000 7fb25ed9d700 1 --2- 172.21.15.101:0/3136757488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2601079a0 0x7fb26019f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:08.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.233+0000 7fb25e59c700 1 -- 172.21.15.101:0/3136757488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2601079a0 msgr2=0x7fb26019f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:08.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.233+0000 7fb25e59c700 1 --2- 172.21.15.101:0/3136757488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2601079a0 0x7fb26019f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.233+0000 7fb25e59c700 1 -- 172.21.15.101:0/3136757488 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb260074550 msgr2=0x7fb260197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:08.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.233+0000 7fb25e59c700 1 --2- 172.21.15.101:0/3136757488 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb260074550 0x7fb260197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.233+0000 7fb25e59c700 1 -- 172.21.15.101:0/3136757488 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2500095f0 con 0x7fb260072f70 2023-07-18T03:07:08.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.234+0000 7fb25e59c700 1 --2- 172.21.15.101:0/3136757488 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb260072f70 0x7fb260197aa0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb250004710 tx=0x7fb250009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:08.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.234+0000 7fb24f7fe700 1 -- 172.21.15.101:0/3136757488 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25001d070 con 0x7fb260072f70 2023-07-18T03:07:08.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.234+0000 7fb264bff700 1 -- 172.21.15.101:0/3136757488 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb26019fb70 con 0x7fb260072f70 2023-07-18T03:07:08.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.234+0000 7fb24f7fe700 1 -- 172.21.15.101:0/3136757488 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb250021410 con 0x7fb260072f70 2023-07-18T03:07:08.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.234+0000 7fb24f7fe700 1 -- 172.21.15.101:0/3136757488 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb250017650 con 0x7fb260072f70 2023-07-18T03:07:08.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.234+0000 7fb264bff700 1 -- 172.21.15.101:0/3136757488 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb26019ff90 con 0x7fb260072f70 2023-07-18T03:07:08.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.237+0000 7fb24f7fe700 1 -- 172.21.15.101:0/3136757488 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fb250021580 con 0x7fb260072f70 2023-07-18T03:07:08.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.238+0000 7fb27853fb80 1 Processor -- start 2023-07-18T03:07:08.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.238+0000 7fb27853fb80 1 -- start start 2023-07-18T03:07:08.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.238+0000 7fb24f7fe700 1 --2- 172.21.15.101:0/3136757488 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb2480a1150 0x7fb2480a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:08.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.239+0000 7fb27853fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f5c64480 0x5619f5c64890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:08.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.239+0000 7fb27853fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5619f5c64dd0 0x5619f5c65690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:08.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.239+0000 7fb27853fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5619f5c65c20 0x5619f5c66090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:08.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.239+0000 7fb27853fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f5b69c60 con 0x5619f5c65c20 2023-07-18T03:07:08.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.239+0000 7fb27853fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f5bbc9f0 con 0x5619f5c64480 2023-07-18T03:07:08.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.239+0000 7fb27853fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f5bb5e80 con 0x5619f5c64dd0 2023-07-18T03:07:08.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.239+0000 7fb24f7fe700 1 -- 172.21.15.101:0/3136757488 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fb2500ce0e0 con 0x7fb260072f70 2023-07-18T03:07:08.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.239+0000 7fb25e59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f5c64480 0x5619f5c64890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:08.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.239+0000 7fb25ed9d700 1 --2- 172.21.15.101:0/3136757488 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb2480a1150 0x7fb2480a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:08.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.239+0000 7fb25dd9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5619f5c64dd0 0x5619f5c65690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:08.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.239+0000 7fb25e59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f5c64480 0x5619f5c64890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:36838/0 (socket says 172.21.15.101:36838) 2023-07-18T03:07:08.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.239+0000 7fb25e59c700 1 -- 172.21.15.101:0/54722057 learned_addr learned my addr 172.21.15.101:0/54722057 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:08.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.239+0000 7fb25ed9d700 1 --2- 172.21.15.101:0/54722057 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5619f5c65c20 0x5619f5c66090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:08.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.239+0000 7fb25e59c700 1 -- 172.21.15.101:0/54722057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5619f5c64dd0 msgr2=0x5619f5c65690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:08.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.239+0000 7fb25e59c700 1 --2- 172.21.15.101:0/54722057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5619f5c64dd0 0x5619f5c65690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.239+0000 7fb25e59c700 1 -- 172.21.15.101:0/54722057 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5619f5c65c20 msgr2=0x5619f5c66090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:08.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.239+0000 7fb25e59c700 1 --2- 172.21.15.101:0/54722057 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5619f5c65c20 0x5619f5c66090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.239+0000 7fb25e59c700 1 -- 172.21.15.101:0/54722057 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619f59e3d90 con 0x5619f5c64480 2023-07-18T03:07:08.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.240+0000 7fb25e59c700 1 --2- 172.21.15.101:0/54722057 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f5c64480 0x5619f5c64890 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb2500a3040 tx=0x7fb25009b020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e8a819c545e465f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:08.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.240+0000 7fb24cff9700 1 -- 172.21.15.101:0/54722057 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25009ab10 con 0x5619f5c64480 2023-07-18T03:07:08.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.240+0000 7fb25ed9d700 1 --2- 172.21.15.101:0/3136757488 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb2480a1150 0x7fb2480a3600 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb258009ec0 tx=0x7fb258009920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:08.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.240+0000 7fb24cff9700 1 -- 172.21.15.101:0/54722057 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb25009b1b0 con 0x5619f5c64480 2023-07-18T03:07:08.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.240+0000 7fb24cff9700 1 -- 172.21.15.101:0/54722057 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb250021880 con 0x5619f5c64480 2023-07-18T03:07:08.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.240+0000 7fb27853fb80 1 -- 172.21.15.101:0/54722057 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f5c64480 msgr2=0x5619f5c64890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:08.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.240+0000 7fb27853fb80 1 --2- 172.21.15.101:0/54722057 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f5c64480 0x5619f5c64890 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb2500a3040 tx=0x7fb25009b020 comp rx=0 tx=0).stop 2023-07-18T03:07:08.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.241+0000 7fb27853fb80 1 -- 172.21.15.101:0/54722057 shutdown_connections 2023-07-18T03:07:08.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.241+0000 7fb27853fb80 1 --2- 172.21.15.101:0/54722057 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f5c64480 0x5619f5c64890 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.241+0000 7fb27853fb80 1 --2- 172.21.15.101:0/54722057 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5619f5c65c20 0x5619f5c66090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.241+0000 7fb27853fb80 1 --2- 172.21.15.101:0/54722057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5619f5c64dd0 0x5619f5c65690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.241+0000 7fb27853fb80 1 -- 172.21.15.101:0/54722057 >> 172.21.15.101:0/54722057 conn(0x5619f5c59d50 msgr2=0x5619f5c5f180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:08.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.241+0000 7fb27853fb80 1 -- 172.21.15.101:0/54722057 shutdown_connections 2023-07-18T03:07:08.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.241+0000 7fb27853fb80 1 -- 172.21.15.101:0/54722057 wait complete. 2023-07-18T03:07:08.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.241+0000 7fb27853fb80 1 -- start start 2023-07-18T03:07:08.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.241+0000 7fb27853fb80 1 Processor -- start 2023-07-18T03:07:08.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.242+0000 7fb27853fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f5c64480 0x5619f5c6dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:08.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.242+0000 7fb27853fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5619f5c64dd0 0x5619f5c6e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:08.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.242+0000 7fb27853fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5619f5c65c20 0x5619f5c75280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:08.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.242+0000 7fb27853fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f5b55a70 con 0x5619f5c65c20 2023-07-18T03:07:08.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.242+0000 7fb27853fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f5b55770 con 0x5619f5c64480 2023-07-18T03:07:08.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.242+0000 7fb27853fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f5a398d0 con 0x5619f5c64dd0 2023-07-18T03:07:08.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.242+0000 7fb27853fb80 10 client.0 _ll_register_callbacks cb 0x5619f5b53710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:07:08.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.242+0000 7fb25ed9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5619f5c65c20 0x5619f5c75280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:08.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.242+0000 7fb25e59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f5c64480 0x5619f5c6dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:08.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.242+0000 7fb25e59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f5c64480 0x5619f5c6dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:36844/0 (socket says 172.21.15.101:36844) 2023-07-18T03:07:08.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.242+0000 7fb25e59c700 1 -- 172.21.15.101:0/3720801827 learned_addr learned my addr 172.21.15.101:0/3720801827 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:08.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.242+0000 7fb25dd9b700 1 --2- 172.21.15.101:0/3720801827 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5619f5c64dd0 0x5619f5c6e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:08.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.242+0000 7fb25e59c700 1 -- 172.21.15.101:0/3720801827 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5619f5c64dd0 msgr2=0x5619f5c6e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:08.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.242+0000 7fb25e59c700 1 --2- 172.21.15.101:0/3720801827 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5619f5c64dd0 0x5619f5c6e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.242+0000 7fb25e59c700 1 -- 172.21.15.101:0/3720801827 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5619f5c65c20 msgr2=0x5619f5c75280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:08.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.242+0000 7fb25e59c700 1 --2- 172.21.15.101:0/3720801827 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5619f5c65c20 0x5619f5c75280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.243+0000 7fb25e59c700 1 -- 172.21.15.101:0/3720801827 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb26019ff90 con 0x5619f5c64480 2023-07-18T03:07:08.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.243+0000 7fb25ed9d700 1 --2- 172.21.15.101:0/3720801827 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5619f5c65c20 0x5619f5c75280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:07:08.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.243+0000 7fb25e59c700 1 --2- 172.21.15.101:0/3720801827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f5c64480 0x5619f5c6dd00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb25009b140 tx=0x7fb250099760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d484e1faee102e09 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:08.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.243+0000 7fb245ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:07:08.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.243+0000 7fb245ffb700 1 -- 172.21.15.101:0/3720801827 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25009a6a0 con 0x5619f5c64480 2023-07-18T03:07:08.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.243+0000 7fb27853fb80 1 -- 172.21.15.101:0/3720801827 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5619f5b0f4a0 con 0x5619f5c64480 2023-07-18T03:07:08.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.243+0000 7fb245ffb700 1 -- 172.21.15.101:0/3720801827 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fb25009a800 con 0x5619f5c64480 2023-07-18T03:07:08.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.243+0000 7fb245ffb700 1 -- 172.21.15.101:0/3720801827 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2500a30e0 con 0x5619f5c64480 2023-07-18T03:07:08.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.243+0000 7fb27853fb80 10 client.25295 fetch_fsmap learned FSMap version 24 2023-07-18T03:07:08.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.243+0000 7fb27853fb80 1 -- 172.21.15.101:0/3720801827 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5619f59ccc10 con 0x5619f5c64480 2023-07-18T03:07:08.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.243+0000 7fb245ffb700 1 -- 172.21.15.101:0/3720801827 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5619f5b0f4a0 con 0x5619f5c64480 2023-07-18T03:07:08.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.244+0000 7fb245ffb700 1 -- 172.21.15.101:0/3720801827 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb25008b1a0 con 0x5619f5c64480 2023-07-18T03:07:08.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.244+0000 7fb27853fb80 10 client.25295 fetch_fsmap finished waiting for FSMap version 24 2023-07-18T03:07:08.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.244+0000 7fb27853fb80 10 client.25295 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-18T03:07:08.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.244+0000 7fb27853fb80 20 client.25295 populate_metadata read hostname 'smithi101' 2023-07-18T03:07:08.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.244+0000 7fb27853fb80 1 --2- 172.21.15.101:0/3720801827 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5619f5c78c00 0x5619f5c781b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:08.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.244+0000 7fb27853fb80 4 client.25295 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:07:08.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.244+0000 7fb27853fb80 1 -- 172.21.15.101:0/3720801827 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5619f5baee60 con 0x5619f5c78c00 2023-07-18T03:07:08.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.244+0000 7fb25dd9b700 1 --2- 172.21.15.101:0/3720801827 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5619f5c78c00 0x5619f5c781b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:08.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.245+0000 7fb245ffb700 10 client.25295 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:07:08.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.245+0000 7fb25dd9b700 1 --2- 172.21.15.101:0/3720801827 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5619f5c78c00 0x5619f5c781b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=18fd267843dd239e server_cookie=fc043c5b041fd7f2 in_seq=0 out_seq=0 2023-07-18T03:07:08.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.246+0000 7fb245ffb700 1 -- 172.21.15.101:0/3720801827 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5619f5baee60 con 0x5619f5c78c00 2023-07-18T03:07:08.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.246+0000 7fb245ffb700 10 client.25295 handle_command_reply: tid=0 2023-07-18T03:07:08.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.246+0000 7fb27853fb80 1 client.25295 shutdown 2023-07-18T03:07:08.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.246+0000 7fb27853fb80 1 -- 172.21.15.101:0/3720801827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f5c64480 msgr2=0x5619f5c6dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:08.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.246+0000 7fb27853fb80 1 --2- 172.21.15.101:0/3720801827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f5c64480 0x5619f5c6dd00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb25009b140 tx=0x7fb250099760 comp rx=0 tx=0).stop 2023-07-18T03:07:08.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.246+0000 7fb27853fb80 1 -- 172.21.15.101:0/3720801827 shutdown_connections 2023-07-18T03:07:08.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.246+0000 7fb27853fb80 1 --2- 172.21.15.101:0/3720801827 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5619f5c78c00 0x5619f5c781b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.246+0000 7fb27853fb80 1 --2- 172.21.15.101:0/3720801827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f5c64480 0x5619f5c6dd00 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.246+0000 7fb27853fb80 1 --2- 172.21.15.101:0/3720801827 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5619f5c65c20 0x5619f5c75280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.246+0000 7fb245ffb700 0 client.25295 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:07:08.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.246+0000 7fb27853fb80 1 --2- 172.21.15.101:0/3720801827 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5619f5c64dd0 0x5619f5c6e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.246+0000 7fb27853fb80 1 -- 172.21.15.101:0/3720801827 >> 172.21.15.101:0/3720801827 conn(0x5619f5c59d50 msgr2=0x5619f5c5e860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:08.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.246+0000 7fb27853fb80 1 -- 172.21.15.101:0/3720801827 shutdown_connections 2023-07-18T03:07:08.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.246+0000 7fb27853fb80 1 -- 172.21.15.101:0/3720801827 wait complete. 2023-07-18T03:07:08.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.246+0000 7fb27853fb80 20 client.25295 trim_cache size 0 max 16384 2023-07-18T03:07:08.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.265+0000 7fb27853fb80 1 Processor -- start 2023-07-18T03:07:08.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.265+0000 7fb27853fb80 1 -- start start 2023-07-18T03:07:08.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.265+0000 7fb27853fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f5c64480 0x5619f5c6bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:08.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.265+0000 7fb27853fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5619f5c64dd0 0x5619f5c6c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:08.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.265+0000 7fb27853fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5619f5c65c20 0x5619f5c75210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:08.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.265+0000 7fb27853fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f5c75750 con 0x5619f5c64dd0 2023-07-18T03:07:08.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.265+0000 7fb27853fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f5c79ee0 con 0x5619f5c64480 2023-07-18T03:07:08.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.265+0000 7fb27853fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f5baee60 con 0x5619f5c65c20 2023-07-18T03:07:08.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.265+0000 7fb25dd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5619f5c64dd0 0x5619f5c6c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:08.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.266+0000 7fb25dd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5619f5c64dd0 0x5619f5c6c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:36750/0 (socket says 172.21.15.101:36750) 2023-07-18T03:07:08.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.266+0000 7fb25dd9b700 1 -- 172.21.15.101:0/2104799093 learned_addr learned my addr 172.21.15.101:0/2104799093 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:08.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.266+0000 7fb25ed9d700 1 --2- 172.21.15.101:0/2104799093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5619f5c65c20 0x5619f5c75210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:08.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.266+0000 7fb25e59c700 1 --2- 172.21.15.101:0/2104799093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f5c64480 0x5619f5c6bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:08.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.266+0000 7fb25dd9b700 1 -- 172.21.15.101:0/2104799093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5619f5c65c20 msgr2=0x5619f5c75210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:08.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.266+0000 7fb25dd9b700 1 --2- 172.21.15.101:0/2104799093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5619f5c65c20 0x5619f5c75210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.266+0000 7fb25dd9b700 1 -- 172.21.15.101:0/2104799093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f5c64480 msgr2=0x5619f5c6bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:08.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.266+0000 7fb25dd9b700 1 --2- 172.21.15.101:0/2104799093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f5c64480 0x5619f5c6bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.266+0000 7fb25dd9b700 1 -- 172.21.15.101:0/2104799093 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619f5c66510 con 0x5619f5c64dd0 2023-07-18T03:07:08.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.266+0000 7fb25dd9b700 1 --2- 172.21.15.101:0/2104799093 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5619f5c64dd0 0x5619f5c6c510 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb254004790 tx=0x7fb254009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ba39beb562c9ab server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:08.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.266+0000 7fb24d7fa700 1 -- 172.21.15.101:0/2104799093 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb254017070 con 0x5619f5c64dd0 2023-07-18T03:07:08.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.266+0000 7fb24d7fa700 1 -- 172.21.15.101:0/2104799093 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb254018040 con 0x5619f5c64dd0 2023-07-18T03:07:08.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.266+0000 7fb24d7fa700 1 -- 172.21.15.101:0/2104799093 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb254010690 con 0x5619f5c64dd0 2023-07-18T03:07:08.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.267+0000 7fb27853fb80 1 -- 172.21.15.101:0/2104799093 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5619f5c64dd0 msgr2=0x5619f5c6c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:08.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.267+0000 7fb27853fb80 1 --2- 172.21.15.101:0/2104799093 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5619f5c64dd0 0x5619f5c6c510 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb254004790 tx=0x7fb254009260 comp rx=0 tx=0).stop 2023-07-18T03:07:08.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.267+0000 7fb27853fb80 1 -- 172.21.15.101:0/2104799093 shutdown_connections 2023-07-18T03:07:08.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.267+0000 7fb27853fb80 1 --2- 172.21.15.101:0/2104799093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f5c64480 0x5619f5c6bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.267+0000 7fb27853fb80 1 --2- 172.21.15.101:0/2104799093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5619f5c65c20 0x5619f5c75210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.267+0000 7fb27853fb80 1 --2- 172.21.15.101:0/2104799093 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5619f5c64dd0 0x5619f5c6c510 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.267+0000 7fb27853fb80 1 -- 172.21.15.101:0/2104799093 >> 172.21.15.101:0/2104799093 conn(0x5619f5c59d50 msgr2=0x5619f5c75ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:08.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.267+0000 7fb27853fb80 1 -- 172.21.15.101:0/2104799093 shutdown_connections 2023-07-18T03:07:08.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.267+0000 7fb27853fb80 1 -- 172.21.15.101:0/2104799093 wait complete. 2023-07-18T03:07:08.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.268+0000 7fb27853fb80 1 -- start start 2023-07-18T03:07:08.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.268+0000 7fb27853fb80 1 Processor -- start 2023-07-18T03:07:08.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.268+0000 7fb27853fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f5c64480 0x5619f5c633f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:08.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.268+0000 7fb27853fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5619f5c64dd0 0x5619f5c63930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:08.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.268+0000 7fb27853fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5619f5c65c20 0x5619f5ccca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:08.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.268+0000 7fb27853fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f5cccfc0 con 0x5619f5c64dd0 2023-07-18T03:07:08.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.268+0000 7fb27853fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f5ccd100 con 0x5619f5c64480 2023-07-18T03:07:08.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.268+0000 7fb27853fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f5ccd270 con 0x5619f5c65c20 2023-07-18T03:07:08.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.268+0000 7fb27853fb80 10 client.0 _ll_register_callbacks cb 0x7fb254009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:07:08.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.268+0000 7fb25dd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5619f5c64dd0 0x5619f5c63930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:08.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.269+0000 7fb25dd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5619f5c64dd0 0x5619f5c63930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:36752/0 (socket says 172.21.15.101:36752) 2023-07-18T03:07:08.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.269+0000 7fb25dd9b700 1 -- 172.21.15.101:0/523136027 learned_addr learned my addr 172.21.15.101:0/523136027 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:08.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.269+0000 7fb25ed9d700 1 --2- 172.21.15.101:0/523136027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5619f5c65c20 0x5619f5ccca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:08.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.269+0000 7fb25e59c700 1 --2- 172.21.15.101:0/523136027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f5c64480 0x5619f5c633f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:08.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.269+0000 7fb25dd9b700 1 -- 172.21.15.101:0/523136027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5619f5c65c20 msgr2=0x5619f5ccca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:08.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.269+0000 7fb25dd9b700 1 --2- 172.21.15.101:0/523136027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5619f5c65c20 0x5619f5ccca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.269+0000 7fb25dd9b700 1 -- 172.21.15.101:0/523136027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f5c64480 msgr2=0x5619f5c633f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:08.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.269+0000 7fb25dd9b700 1 --2- 172.21.15.101:0/523136027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f5c64480 0x5619f5c633f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.269+0000 7fb25dd9b700 1 -- 172.21.15.101:0/523136027 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb254009b30 con 0x5619f5c64dd0 2023-07-18T03:07:08.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.269+0000 7fb25e59c700 1 --2- 172.21.15.101:0/523136027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f5c64480 0x5619f5c633f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:07:08.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.269+0000 7fb25ed9d700 1 --2- 172.21.15.101:0/523136027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5619f5c65c20 0x5619f5ccca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:07:08.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.269+0000 7fb25dd9b700 1 --2- 172.21.15.101:0/523136027 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5619f5c64dd0 0x5619f5c63930 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb25401ffb0 tx=0x7fb25401e650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=189b3cec0d0cbb1c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:08.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.269+0000 7fb245ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:07:08.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.269+0000 7fb245ffb700 1 -- 172.21.15.101:0/523136027 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb254017070 con 0x5619f5c64dd0 2023-07-18T03:07:08.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.269+0000 7fb27853fb80 1 -- 172.21.15.101:0/523136027 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb25009b1b0 con 0x5619f5c64dd0 2023-07-18T03:07:08.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.269+0000 7fb245ffb700 1 -- 172.21.15.101:0/523136027 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fb254003900 con 0x5619f5c64dd0 2023-07-18T03:07:08.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.270+0000 7fb245ffb700 1 -- 172.21.15.101:0/523136027 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb254003ac0 con 0x5619f5c64dd0 2023-07-18T03:07:08.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.270+0000 7fb27853fb80 10 client.15279 fetch_fsmap learned FSMap version 24 2023-07-18T03:07:08.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.270+0000 7fb27853fb80 1 -- 172.21.15.101:0/523136027 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5619f5a0a200 con 0x5619f5c64dd0 2023-07-18T03:07:08.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.270+0000 7fb245ffb700 1 -- 172.21.15.101:0/523136027 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb25009b1b0 con 0x5619f5c64dd0 2023-07-18T03:07:08.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.270+0000 7fb245ffb700 1 -- 172.21.15.101:0/523136027 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb254025070 con 0x5619f5c64dd0 2023-07-18T03:07:08.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.270+0000 7fb27853fb80 10 client.15279 fetch_fsmap finished waiting for FSMap version 24 2023-07-18T03:07:08.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.270+0000 7fb27853fb80 10 client.15279 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-18T03:07:08.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.270+0000 7fb27853fb80 20 client.15279 populate_metadata read hostname 'smithi101' 2023-07-18T03:07:08.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.270+0000 7fb27853fb80 1 --2- 172.21.15.101:0/523136027 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5619f5cd2020 0x5619f5ccea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:08.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.270+0000 7fb27853fb80 4 client.15279 mds_command: new command op to 24557 tid=0[{"prefix": "status"}] 2023-07-18T03:07:08.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.270+0000 7fb27853fb80 1 -- 172.21.15.101:0/523136027 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5619f5b71000 con 0x5619f5cd2020 2023-07-18T03:07:08.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.270+0000 7fb25e59c700 1 --2- 172.21.15.101:0/523136027 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5619f5cd2020 0x5619f5ccea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:08.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.271+0000 7fb245ffb700 10 client.15279 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:07:08.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.271+0000 7fb25e59c700 1 --2- 172.21.15.101:0/523136027 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5619f5cd2020 0x5619f5ccea40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8e23d57e6c9dcf69 server_cookie=4f905372da6d5074 in_seq=0 out_seq=0 2023-07-18T03:07:08.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.271+0000 7fb245ffb700 1 -- 172.21.15.101:0/523136027 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5619f5b71000 con 0x5619f5cd2020 2023-07-18T03:07:08.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.271+0000 7fb245ffb700 10 client.15279 handle_command_reply: tid=0 2023-07-18T03:07:08.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.271+0000 7fb27853fb80 1 client.15279 shutdown 2023-07-18T03:07:08.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.271+0000 7fb27853fb80 1 -- 172.21.15.101:0/523136027 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5619f5c64dd0 msgr2=0x5619f5c63930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:08.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.271+0000 7fb27853fb80 1 --2- 172.21.15.101:0/523136027 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5619f5c64dd0 0x5619f5c63930 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb25401ffb0 tx=0x7fb25401e650 comp rx=0 tx=0).stop 2023-07-18T03:07:08.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.271+0000 7fb27853fb80 1 -- 172.21.15.101:0/523136027 shutdown_connections 2023-07-18T03:07:08.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.271+0000 7fb27853fb80 1 --2- 172.21.15.101:0/523136027 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5619f5cd2020 0x5619f5ccea40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.271+0000 7fb27853fb80 1 --2- 172.21.15.101:0/523136027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f5c64480 0x5619f5c633f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.271+0000 7fb27853fb80 1 --2- 172.21.15.101:0/523136027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5619f5c65c20 0x5619f5ccca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.271+0000 7fb27853fb80 1 --2- 172.21.15.101:0/523136027 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5619f5c64dd0 0x5619f5c63930 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.271+0000 7fb245ffb700 0 client.15279 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:07:08.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.271+0000 7fb27853fb80 1 -- 172.21.15.101:0/523136027 >> 172.21.15.101:0/523136027 conn(0x5619f5c59d50 msgr2=0x5619f5c6bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:08.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.271+0000 7fb27853fb80 1 -- 172.21.15.101:0/523136027 shutdown_connections 2023-07-18T03:07:08.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.272+0000 7fb27853fb80 1 -- 172.21.15.101:0/523136027 wait complete. 2023-07-18T03:07:08.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.272+0000 7fb27853fb80 20 client.15279 trim_cache size 0 max 16384 2023-07-18T03:07:08.270 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:07:08.270 INFO:teuthology.orchestra.run.smithi101.stdout: "cluster_fsid": "b1782daa-2516-11ee-9b34-001a4aab830c", 2023-07-18T03:07:08.270 INFO:teuthology.orchestra.run.smithi101.stdout: "whoami": 1, 2023-07-18T03:07:08.270 INFO:teuthology.orchestra.run.smithi101.stdout: "id": 24557, 2023-07-18T03:07:08.270 INFO:teuthology.orchestra.run.smithi101.stdout: "want_state": "up:active", 2023-07-18T03:07:08.271 INFO:teuthology.orchestra.run.smithi101.stdout: "state": "up:active", 2023-07-18T03:07:08.271 INFO:teuthology.orchestra.run.smithi101.stdout: "fs_name": "cephfs", 2023-07-18T03:07:08.271 INFO:teuthology.orchestra.run.smithi101.stdout: "rank_uptime": 98.297244691000003, 2023-07-18T03:07:08.271 INFO:teuthology.orchestra.run.smithi101.stdout: "mdsmap_epoch": 24, 2023-07-18T03:07:08.271 INFO:teuthology.orchestra.run.smithi101.stdout: "osdmap_epoch": 91, 2023-07-18T03:07:08.271 INFO:teuthology.orchestra.run.smithi101.stdout: "osdmap_epoch_barrier": 89, 2023-07-18T03:07:08.271 INFO:teuthology.orchestra.run.smithi101.stdout: "uptime": 133.86985865899999 2023-07-18T03:07:08.272 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:07:08.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.272+0000 7fb264bff700 1 -- 172.21.15.101:0/3136757488 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb2480a1150 msgr2=0x7fb2480a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:08.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.272+0000 7fb264bff700 1 --2- 172.21.15.101:0/3136757488 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb2480a1150 0x7fb2480a3600 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb258009ec0 tx=0x7fb258009920 comp rx=0 tx=0).stop 2023-07-18T03:07:08.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.272+0000 7fb264bff700 1 -- 172.21.15.101:0/3136757488 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb260072f70 msgr2=0x7fb260197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:08.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.272+0000 7fb264bff700 1 --2- 172.21.15.101:0/3136757488 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb260072f70 0x7fb260197aa0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb250004710 tx=0x7fb250009580 comp rx=0 tx=0).stop 2023-07-18T03:07:08.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.272+0000 7fb264bff700 1 -- 172.21.15.101:0/3136757488 shutdown_connections 2023-07-18T03:07:08.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.272+0000 7fb264bff700 1 --2- 172.21.15.101:0/3136757488 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb2480a1150 0x7fb2480a3600 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.272+0000 7fb264bff700 1 --2- 172.21.15.101:0/3136757488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2601079a0 0x7fb26019f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.272+0000 7fb264bff700 1 --2- 172.21.15.101:0/3136757488 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb260072f70 0x7fb260197aa0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.272+0000 7fb264bff700 1 --2- 172.21.15.101:0/3136757488 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb260074550 0x7fb260197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.272+0000 7fb264bff700 1 -- 172.21.15.101:0/3136757488 >> 172.21.15.101:0/3136757488 conn(0x7fb2600fb0e0 msgr2=0x7fb2600fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:08.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.272+0000 7fb264bff700 1 -- 172.21.15.101:0/3136757488 shutdown_connections 2023-07-18T03:07:08.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.272+0000 7fb264bff700 1 -- 172.21.15.101:0/3136757488 wait complete. 2023-07-18T03:07:08.278 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-07-18T03:07:08.288 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:07 smithi155 ceph-mon[128885]: pgmap v343: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 426 B/s wr, 6 op/s 2023-07-18T03:07:08.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:07 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/3293944976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:07:08.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:07 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1192787490' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:07:08.444 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:07 smithi179 ceph-mon[129229]: pgmap v343: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 426 B/s wr, 6 op/s 2023-07-18T03:07:08.444 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:07 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/3293944976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:07:08.444 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:07 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1192787490' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:07:08.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.490+0000 7fc5f69ff700 1 -- 172.21.15.101:0/793399679 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc5f00ffb00 msgr2=0x7fc5f00fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:08.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.490+0000 7fc5f69ff700 1 --2- 172.21.15.101:0/793399679 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc5f00ffb00 0x7fc5f00fff10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0009990 tx=0x7fc5e0009ca0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.490+0000 7fc5f69ff700 1 -- 172.21.15.101:0/793399679 shutdown_connections 2023-07-18T03:07:08.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.490+0000 7fc5f69ff700 1 --2- 172.21.15.101:0/793399679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5f0101680 0x7fc5f0107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.490+0000 7fc5f69ff700 1 --2- 172.21.15.101:0/793399679 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc5f00ffb00 0x7fc5f00fff10 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.490+0000 7fc5f69ff700 1 --2- 172.21.15.101:0/793399679 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc5f0100cf0 0x7fc5f0101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.490+0000 7fc5f69ff700 1 -- 172.21.15.101:0/793399679 >> 172.21.15.101:0/793399679 conn(0x7fc5f00fb120 msgr2=0x7fc5f00fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:08.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.490+0000 7fc5f69ff700 1 -- 172.21.15.101:0/793399679 shutdown_connections 2023-07-18T03:07:08.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.490+0000 7fc5f69ff700 1 -- 172.21.15.101:0/793399679 wait complete. 2023-07-18T03:07:08.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.491+0000 7fc5f69ff700 1 Processor -- start 2023-07-18T03:07:08.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.491+0000 7fc5f69ff700 1 -- start start 2023-07-18T03:07:08.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.491+0000 7fc5f69ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5f00ffb00 0x7fc5f0197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:08.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.492+0000 7fc5f69ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc5f0100cf0 0x7fc5f0198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:08.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.492+0000 7fc5f69ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc5f0101680 0x7fc5f019f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:08.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.492+0000 7fc5f69ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f019f690 con 0x7fc5f0101680 2023-07-18T03:07:08.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.492+0000 7fc5f69ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f019f7d0 con 0x7fc5f00ffb00 2023-07-18T03:07:08.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.492+0000 7fc5f69ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f019f960 con 0x7fc5f0100cf0 2023-07-18T03:07:08.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.492+0000 7fc5f4f9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc5f0101680 0x7fc5f019f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:08.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.492+0000 7fc5ef7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc5f0100cf0 0x7fc5f0198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:08.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.492+0000 7fc5f4f9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc5f0101680 0x7fc5f019f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:36778/0 (socket says 172.21.15.101:36778) 2023-07-18T03:07:08.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.492+0000 7fc5f4f9c700 1 -- 172.21.15.101:0/552680155 learned_addr learned my addr 172.21.15.101:0/552680155 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:08.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.492+0000 7fc5effff700 1 --2- 172.21.15.101:0/552680155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5f00ffb00 0x7fc5f0197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:08.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.492+0000 7fc5f4f9c700 1 -- 172.21.15.101:0/552680155 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc5f0100cf0 msgr2=0x7fc5f0198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:08.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.492+0000 7fc5f4f9c700 1 --2- 172.21.15.101:0/552680155 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc5f0100cf0 0x7fc5f0198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.492+0000 7fc5f4f9c700 1 -- 172.21.15.101:0/552680155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5f00ffb00 msgr2=0x7fc5f0197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:08.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.492+0000 7fc5f4f9c700 1 --2- 172.21.15.101:0/552680155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5f00ffb00 0x7fc5f0197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.492+0000 7fc5f4f9c700 1 -- 172.21.15.101:0/552680155 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e00095f0 con 0x7fc5f0101680 2023-07-18T03:07:08.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.492+0000 7fc5effff700 1 --2- 172.21.15.101:0/552680155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5f00ffb00 0x7fc5f0197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:07:08.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.492+0000 7fc5f4f9c700 1 --2- 172.21.15.101:0/552680155 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc5f0101680 0x7fc5f019f0c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc5e800ef30 tx=0x7fc5e800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:08.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.493+0000 7fc5ed7fa700 1 -- 172.21.15.101:0/552680155 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e8018070 con 0x7fc5f0101680 2023-07-18T03:07:08.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.493+0000 7fc5ed7fa700 1 -- 172.21.15.101:0/552680155 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5e800f040 con 0x7fc5f0101680 2023-07-18T03:07:08.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.493+0000 7fc5f69ff700 1 -- 172.21.15.101:0/552680155 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f019fbe0 con 0x7fc5f0101680 2023-07-18T03:07:08.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.493+0000 7fc5ed7fa700 1 -- 172.21.15.101:0/552680155 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e800cdb0 con 0x7fc5f0101680 2023-07-18T03:07:08.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.493+0000 7fc5f69ff700 1 -- 172.21.15.101:0/552680155 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f01a00f0 con 0x7fc5f0101680 2023-07-18T03:07:08.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.496+0000 7fc5ed7fa700 1 -- 172.21.15.101:0/552680155 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fc5e8003a00 con 0x7fc5f0101680 2023-07-18T03:07:08.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.497+0000 7fc60a322b80 1 Processor -- start 2023-07-18T03:07:08.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.497+0000 7fc5ed7fa700 1 --2- 172.21.15.101:0/552680155 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc5d80a1150 0x7fc5d80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:08.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.497+0000 7fc60a322b80 1 -- start start 2023-07-18T03:07:08.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.497+0000 7fc5ed7fa700 1 -- 172.21.15.101:0/552680155 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fc5e80cc820 con 0x7fc5f0101680 2023-07-18T03:07:08.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.497+0000 7fc60a322b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a8de073480 0x55a8de073890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:08.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.498+0000 7fc60a322b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8de073dd0 0x55a8de074690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:08.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.498+0000 7fc60a322b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a8de074c20 0x55a8de075090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:08.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.498+0000 7fc60a322b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8ddf78c60 con 0x55a8de073480 2023-07-18T03:07:08.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.498+0000 7fc60a322b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8ddfcb9f0 con 0x55a8de074c20 2023-07-18T03:07:08.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.498+0000 7fc60a322b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8ddfc4e80 con 0x55a8de073dd0 2023-07-18T03:07:08.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.498+0000 7fc5ef7fe700 1 --2- 172.21.15.101:0/552680155 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc5d80a1150 0x7fc5d80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:08.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.498+0000 7fc5effff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a8de073480 0x55a8de073890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:08.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.498+0000 7fc5effff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a8de073480 0x55a8de073890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:36782/0 (socket says 172.21.15.101:36782) 2023-07-18T03:07:08.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.498+0000 7fc5effff700 1 -- 172.21.15.101:0/2329668878 learned_addr learned my addr 172.21.15.101:0/2329668878 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:08.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.498+0000 7fc5ef7fe700 1 --2- 172.21.15.101:0/2329668878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8de073dd0 0x55a8de074690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:08.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.498+0000 7fc5f4f9c700 1 --2- 172.21.15.101:0/2329668878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a8de074c20 0x55a8de075090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:08.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.498+0000 7fc5effff700 1 -- 172.21.15.101:0/2329668878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8de073dd0 msgr2=0x55a8de074690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:08.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.498+0000 7fc5effff700 1 --2- 172.21.15.101:0/2329668878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8de073dd0 0x55a8de074690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.498+0000 7fc5effff700 1 -- 172.21.15.101:0/2329668878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a8de074c20 msgr2=0x55a8de075090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:08.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.498+0000 7fc5effff700 1 --2- 172.21.15.101:0/2329668878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a8de074c20 0x55a8de075090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.498+0000 7fc5effff700 1 -- 172.21.15.101:0/2329668878 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8dddf2d90 con 0x55a8de073480 2023-07-18T03:07:08.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.498+0000 7fc5ef7fe700 1 --2- 172.21.15.101:0/552680155 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc5d80a1150 0x7fc5d80a3600 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4013570 tx=0x7fc5e4009f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:08.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.499+0000 7fc5effff700 1 --2- 172.21.15.101:0/2329668878 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a8de073480 0x55a8de073890 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc5e000b2e0 tx=0x7fc5e0005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57bacf13f3f8785e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:08.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.499+0000 7fc5d67fc700 1 -- 172.21.15.101:0/2329668878 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e0005620 con 0x55a8de073480 2023-07-18T03:07:08.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.499+0000 7fc5d67fc700 1 -- 172.21.15.101:0/2329668878 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5e001e040 con 0x55a8de073480 2023-07-18T03:07:08.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.499+0000 7fc5d67fc700 1 -- 172.21.15.101:0/2329668878 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e0004aa0 con 0x55a8de073480 2023-07-18T03:07:08.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.499+0000 7fc60a322b80 1 -- 172.21.15.101:0/2329668878 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a8de073480 msgr2=0x55a8de073890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:08.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.499+0000 7fc60a322b80 1 --2- 172.21.15.101:0/2329668878 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a8de073480 0x55a8de073890 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc5e000b2e0 tx=0x7fc5e0005c00 comp rx=0 tx=0).stop 2023-07-18T03:07:08.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.500+0000 7fc60a322b80 1 -- 172.21.15.101:0/2329668878 shutdown_connections 2023-07-18T03:07:08.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.500+0000 7fc60a322b80 1 --2- 172.21.15.101:0/2329668878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a8de074c20 0x55a8de075090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.500+0000 7fc60a322b80 1 --2- 172.21.15.101:0/2329668878 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a8de073480 0x55a8de073890 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.500+0000 7fc60a322b80 1 --2- 172.21.15.101:0/2329668878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8de073dd0 0x55a8de074690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.500+0000 7fc60a322b80 1 -- 172.21.15.101:0/2329668878 >> 172.21.15.101:0/2329668878 conn(0x55a8de068d50 msgr2=0x55a8de06e180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:08.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.500+0000 7fc60a322b80 1 -- 172.21.15.101:0/2329668878 shutdown_connections 2023-07-18T03:07:08.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.500+0000 7fc60a322b80 1 -- 172.21.15.101:0/2329668878 wait complete. 2023-07-18T03:07:08.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.500+0000 7fc60a322b80 1 -- start start 2023-07-18T03:07:08.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.500+0000 7fc60a322b80 1 Processor -- start 2023-07-18T03:07:08.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.501+0000 7fc60a322b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8de073480 0x55a8de07d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:08.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.501+0000 7fc60a322b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a8de073dd0 0x55a8de07d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:08.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.501+0000 7fc60a322b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a8de074c20 0x55a8de084600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:08.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.501+0000 7fc60a322b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8ddf64a70 con 0x55a8de073dd0 2023-07-18T03:07:08.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.501+0000 7fc60a322b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8ddf64770 con 0x55a8de074c20 2023-07-18T03:07:08.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.501+0000 7fc60a322b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8dde488d0 con 0x55a8de073480 2023-07-18T03:07:08.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.501+0000 7fc60a322b80 10 client.0 _ll_register_callbacks cb 0x55a8ddf62710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:07:08.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.501+0000 7fc5ef7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a8de073dd0 0x55a8de07d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:08.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.501+0000 7fc5f4f9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a8de074c20 0x55a8de084600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:08.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.501+0000 7fc5ef7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a8de073dd0 0x55a8de07d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:36798/0 (socket says 172.21.15.101:36798) 2023-07-18T03:07:08.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.501+0000 7fc5effff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8de073480 0x55a8de07d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:08.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.501+0000 7fc5ef7fe700 1 -- 172.21.15.101:0/3988463423 learned_addr learned my addr 172.21.15.101:0/3988463423 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:08.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.502+0000 7fc5ef7fe700 1 -- 172.21.15.101:0/3988463423 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8de073480 msgr2=0x55a8de07d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:08.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.502+0000 7fc5ef7fe700 1 --2- 172.21.15.101:0/3988463423 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8de073480 0x55a8de07d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.502+0000 7fc5ef7fe700 1 -- 172.21.15.101:0/3988463423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a8de074c20 msgr2=0x55a8de084600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:08.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.502+0000 7fc5ef7fe700 1 --2- 172.21.15.101:0/3988463423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a8de074c20 0x55a8de084600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.502+0000 7fc5ef7fe700 1 -- 172.21.15.101:0/3988463423 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f0108870 con 0x55a8de073dd0 2023-07-18T03:07:08.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.502+0000 7fc5ef7fe700 1 --2- 172.21.15.101:0/3988463423 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a8de073dd0 0x55a8de07d5c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc5e40036d0 tx=0x7fc5e40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4123fc1bbe6a3bdd server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:08.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.502+0000 7fc5cf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:07:08.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.502+0000 7fc5cf7fe700 1 -- 172.21.15.101:0/3988463423 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e4018040 con 0x55a8de073dd0 2023-07-18T03:07:08.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.502+0000 7fc5cf7fe700 1 -- 172.21.15.101:0/3988463423 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fc5e4004a30 con 0x55a8de073dd0 2023-07-18T03:07:08.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.502+0000 7fc60a322b80 1 -- 172.21.15.101:0/3988463423 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a8ddf1e4a0 con 0x55a8de073dd0 2023-07-18T03:07:08.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.502+0000 7fc5cf7fe700 1 -- 172.21.15.101:0/3988463423 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5e40122e0 con 0x55a8de073dd0 2023-07-18T03:07:08.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.502+0000 7fc60a322b80 10 client.15291 fetch_fsmap learned FSMap version 24 2023-07-18T03:07:08.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.502+0000 7fc60a322b80 1 -- 172.21.15.101:0/3988463423 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55a8ddddbc10 con 0x55a8de073dd0 2023-07-18T03:07:08.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.503+0000 7fc5cf7fe700 1 -- 172.21.15.101:0/3988463423 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a8ddf1e4a0 con 0x55a8de073dd0 2023-07-18T03:07:08.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.503+0000 7fc5cf7fe700 1 -- 172.21.15.101:0/3988463423 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc5e4026070 con 0x55a8de073dd0 2023-07-18T03:07:08.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.503+0000 7fc60a322b80 10 client.15291 fetch_fsmap finished waiting for FSMap version 24 2023-07-18T03:07:08.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.503+0000 7fc60a322b80 10 client.15291 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-18T03:07:08.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.503+0000 7fc60a322b80 20 client.15291 populate_metadata read hostname 'smithi101' 2023-07-18T03:07:08.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.503+0000 7fc60a322b80 1 --2- 172.21.15.101:0/3988463423 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a8de088320 0x55a8de087750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:08.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.503+0000 7fc60a322b80 4 client.15291 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:07:08.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.503+0000 7fc60a322b80 1 -- 172.21.15.101:0/3988463423 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a8ddfbde60 con 0x55a8de088320 2023-07-18T03:07:08.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.503+0000 7fc5effff700 1 --2- 172.21.15.101:0/3988463423 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a8de088320 0x55a8de087750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:08.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.504+0000 7fc5cf7fe700 10 client.15291 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:07:08.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.504+0000 7fc5effff700 1 --2- 172.21.15.101:0/3988463423 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a8de088320 0x55a8de087750 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34041530a006aa2c server_cookie=8efed8fe3eb711a4 in_seq=0 out_seq=0 2023-07-18T03:07:08.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.504+0000 7fc5cf7fe700 1 -- 172.21.15.101:0/3988463423 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a8ddfbde60 con 0x55a8de088320 2023-07-18T03:07:08.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.504+0000 7fc5cf7fe700 10 client.15291 handle_command_reply: tid=0 2023-07-18T03:07:08.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.505+0000 7fc60a322b80 1 client.15291 shutdown 2023-07-18T03:07:08.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.505+0000 7fc60a322b80 1 -- 172.21.15.101:0/3988463423 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a8de073dd0 msgr2=0x55a8de07d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:08.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.505+0000 7fc60a322b80 1 --2- 172.21.15.101:0/3988463423 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a8de073dd0 0x55a8de07d5c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc5e40036d0 tx=0x7fc5e40047c0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.505+0000 7fc60a322b80 1 -- 172.21.15.101:0/3988463423 shutdown_connections 2023-07-18T03:07:08.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.505+0000 7fc60a322b80 1 --2- 172.21.15.101:0/3988463423 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a8de088320 0x55a8de087750 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.505+0000 7fc60a322b80 1 --2- 172.21.15.101:0/3988463423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a8de074c20 0x55a8de084600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.505+0000 7fc60a322b80 1 --2- 172.21.15.101:0/3988463423 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8de073480 0x55a8de07d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.505+0000 7fc60a322b80 1 --2- 172.21.15.101:0/3988463423 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a8de073dd0 0x55a8de07d5c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.505+0000 7fc5cf7fe700 0 client.15291 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:07:08.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.505+0000 7fc60a322b80 1 -- 172.21.15.101:0/3988463423 >> 172.21.15.101:0/3988463423 conn(0x55a8de068d50 msgr2=0x55a8de06d860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:08.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.505+0000 7fc60a322b80 1 -- 172.21.15.101:0/3988463423 shutdown_connections 2023-07-18T03:07:08.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.505+0000 7fc60a322b80 1 -- 172.21.15.101:0/3988463423 wait complete. 2023-07-18T03:07:08.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.505+0000 7fc60a322b80 20 client.15291 trim_cache size 0 max 16384 2023-07-18T03:07:08.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.523+0000 7fc60a322b80 1 Processor -- start 2023-07-18T03:07:08.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.523+0000 7fc60a322b80 1 -- start start 2023-07-18T03:07:08.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.524+0000 7fc60a322b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a8de073480 0x55a8de07b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:08.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.524+0000 7fc60a322b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a8de073dd0 0x55a8de0845c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:08.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.524+0000 7fc60a322b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8de074c20 0x55a8de087750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:08.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.524+0000 7fc60a322b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8ddf1e4a0 con 0x55a8de073480 2023-07-18T03:07:08.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.524+0000 7fc60a322b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8de07bad0 con 0x55a8de073dd0 2023-07-18T03:07:08.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.524+0000 7fc60a322b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8de0862f0 con 0x55a8de074c20 2023-07-18T03:07:08.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.524+0000 7fc5effff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a8de073480 0x55a8de07b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:08.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.524+0000 7fc5effff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a8de073480 0x55a8de07b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:36812/0 (socket says 172.21.15.101:36812) 2023-07-18T03:07:08.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.524+0000 7fc5effff700 1 -- 172.21.15.101:0/3290383333 learned_addr learned my addr 172.21.15.101:0/3290383333 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:08.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.524+0000 7fc5f4f9c700 1 --2- 172.21.15.101:0/3290383333 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8de074c20 0x55a8de087750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:08.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.524+0000 7fc5ef7fe700 1 --2- 172.21.15.101:0/3290383333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a8de073dd0 0x55a8de0845c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:08.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.524+0000 7fc5effff700 1 -- 172.21.15.101:0/3290383333 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8de074c20 msgr2=0x55a8de087750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:08.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.524+0000 7fc5effff700 1 --2- 172.21.15.101:0/3290383333 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8de074c20 0x55a8de087750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.524+0000 7fc5effff700 1 -- 172.21.15.101:0/3290383333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a8de073dd0 msgr2=0x55a8de0845c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:08.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.524+0000 7fc5effff700 1 --2- 172.21.15.101:0/3290383333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a8de073dd0 0x55a8de0845c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.524+0000 7fc5effff700 1 -- 172.21.15.101:0/3290383333 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8de075510 con 0x55a8de073480 2023-07-18T03:07:08.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.524+0000 7fc5effff700 1 --2- 172.21.15.101:0/3290383333 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a8de073480 0x55a8de07b030 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0000c00 tx=0x7fc5e0005cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc513acf4096a0f6 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:08.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.525+0000 7fc5d6ffd700 1 -- 172.21.15.101:0/3290383333 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e0035070 con 0x55a8de073480 2023-07-18T03:07:08.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.525+0000 7fc5d6ffd700 1 -- 172.21.15.101:0/3290383333 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5e0005540 con 0x55a8de073480 2023-07-18T03:07:08.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.525+0000 7fc5d6ffd700 1 -- 172.21.15.101:0/3290383333 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e0004ca0 con 0x55a8de073480 2023-07-18T03:07:08.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.525+0000 7fc60a322b80 1 -- 172.21.15.101:0/3290383333 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a8de073480 msgr2=0x55a8de07b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:08.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.525+0000 7fc60a322b80 1 --2- 172.21.15.101:0/3290383333 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a8de073480 0x55a8de07b030 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0000c00 tx=0x7fc5e0005cb0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.525+0000 7fc60a322b80 1 -- 172.21.15.101:0/3290383333 shutdown_connections 2023-07-18T03:07:08.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.525+0000 7fc60a322b80 1 --2- 172.21.15.101:0/3290383333 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8de074c20 0x55a8de087750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.525+0000 7fc60a322b80 1 --2- 172.21.15.101:0/3290383333 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a8de073480 0x55a8de07b030 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.525+0000 7fc60a322b80 1 --2- 172.21.15.101:0/3290383333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a8de073dd0 0x55a8de0845c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.525+0000 7fc60a322b80 1 -- 172.21.15.101:0/3290383333 >> 172.21.15.101:0/3290383333 conn(0x55a8de068d50 msgr2=0x55a8de07bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:08.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.525+0000 7fc60a322b80 1 -- 172.21.15.101:0/3290383333 shutdown_connections 2023-07-18T03:07:08.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.525+0000 7fc60a322b80 1 -- 172.21.15.101:0/3290383333 wait complete. 2023-07-18T03:07:08.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.526+0000 7fc60a322b80 1 -- start start 2023-07-18T03:07:08.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.526+0000 7fc60a322b80 1 Processor -- start 2023-07-18T03:07:08.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.526+0000 7fc60a322b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8de073480 0x55a8de07ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:08.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.526+0000 7fc60a322b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a8de073dd0 0x55a8de07f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:08.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.526+0000 7fc60a322b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a8de074c20 0x55a8de0841f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:08.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.526+0000 7fc60a322b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8de084730 con 0x55a8de073dd0 2023-07-18T03:07:08.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.526+0000 7fc60a322b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8de0848a0 con 0x55a8de074c20 2023-07-18T03:07:08.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.526+0000 7fc60a322b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8de0dc1e0 con 0x55a8de073480 2023-07-18T03:07:08.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.526+0000 7fc60a322b80 10 client.0 _ll_register_callbacks cb 0x7fc5e4004350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:07:08.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.527+0000 7fc5ef7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a8de073dd0 0x55a8de07f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:08.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.527+0000 7fc5effff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8de073480 0x55a8de07ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:08.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.527+0000 7fc5f4f9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a8de074c20 0x55a8de0841f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:08.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.527+0000 7fc5ef7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a8de073dd0 0x55a8de07f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:36816/0 (socket says 172.21.15.101:36816) 2023-07-18T03:07:08.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.527+0000 7fc5f4f9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a8de074c20 0x55a8de0841f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:36908/0 (socket says 172.21.15.101:36908) 2023-07-18T03:07:08.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.527+0000 7fc5f4f9c700 1 -- 172.21.15.101:0/306129115 learned_addr learned my addr 172.21.15.101:0/306129115 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:08.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.527+0000 7fc5effff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8de073480 0x55a8de07ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:45862/0 (socket says 172.21.15.101:45862) 2023-07-18T03:07:08.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.527+0000 7fc5effff700 1 -- 172.21.15.101:0/306129115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a8de074c20 msgr2=0x55a8de0841f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:08.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.527+0000 7fc5effff700 1 --2- 172.21.15.101:0/306129115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a8de074c20 0x55a8de0841f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.527+0000 7fc5effff700 1 -- 172.21.15.101:0/306129115 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a8de073dd0 msgr2=0x55a8de07f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:08.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.527+0000 7fc5effff700 1 --2- 172.21.15.101:0/306129115 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a8de073dd0 0x55a8de07f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.527+0000 7fc5effff700 1 -- 172.21.15.101:0/306129115 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a8dddf2d90 con 0x55a8de073480 2023-07-18T03:07:08.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.527+0000 7fc5ef7fe700 1 --2- 172.21.15.101:0/306129115 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a8de073dd0 0x55a8de07f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:07:08.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.527+0000 7fc5effff700 1 --2- 172.21.15.101:0/306129115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8de073480 0x55a8de07ee20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x55a8de07e180 tx=0x7fc5e0004500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ac77c3f5240d32a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:08.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.527+0000 7fc5cf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:07:08.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.528+0000 7fc5cf7fe700 1 -- 172.21.15.101:0/306129115 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e0035040 con 0x55a8de073480 2023-07-18T03:07:08.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.528+0000 7fc60a322b80 1 -- 172.21.15.101:0/306129115 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc5e001e040 con 0x55a8de073480 2023-07-18T03:07:08.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.528+0000 7fc5cf7fe700 1 -- 172.21.15.101:0/306129115 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fc5e00313d0 con 0x55a8de073480 2023-07-18T03:07:08.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.528+0000 7fc5cf7fe700 1 -- 172.21.15.101:0/306129115 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5e0031590 con 0x55a8de073480 2023-07-18T03:07:08.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.528+0000 7fc60a322b80 10 client.25354 fetch_fsmap learned FSMap version 24 2023-07-18T03:07:08.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.528+0000 7fc60a322b80 1 -- 172.21.15.101:0/306129115 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55a8dde19200 con 0x55a8de073480 2023-07-18T03:07:08.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.528+0000 7fc5cf7fe700 1 -- 172.21.15.101:0/306129115 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5e001e040 con 0x55a8de073480 2023-07-18T03:07:08.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.528+0000 7fc5cf7fe700 1 -- 172.21.15.101:0/306129115 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc5e0038070 con 0x55a8de073480 2023-07-18T03:07:08.524 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.528+0000 7fc60a322b80 10 client.25354 fetch_fsmap finished waiting for FSMap version 24 2023-07-18T03:07:08.524 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.528+0000 7fc60a322b80 10 client.25354 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-18T03:07:08.524 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.528+0000 7fc60a322b80 20 client.25354 populate_metadata read hostname 'smithi101' 2023-07-18T03:07:08.524 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.528+0000 7fc60a322b80 1 --2- 172.21.15.101:0/306129115 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a8de0e0a50 0x55a8de0dd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:08.524 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.528+0000 7fc60a322b80 4 client.25354 mds_command: new command op to 24562 tid=0[{"prefix": "status"}] 2023-07-18T03:07:08.524 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.528+0000 7fc60a322b80 1 -- 172.21.15.101:0/306129115 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a8ddf80000 con 0x55a8de0e0a50 2023-07-18T03:07:08.524 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.529+0000 7fc5ef7fe700 1 --2- 172.21.15.101:0/306129115 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a8de0e0a50 0x55a8de0dd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:08.525 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.529+0000 7fc5cf7fe700 10 client.25354 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:07:08.525 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.529+0000 7fc5ef7fe700 1 --2- 172.21.15.101:0/306129115 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a8de0e0a50 0x55a8de0dd3a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf4d5fa08451735 server_cookie=cbf1c8742d7aaf37 in_seq=0 out_seq=0 2023-07-18T03:07:08.525 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.530+0000 7fc5cf7fe700 1 -- 172.21.15.101:0/306129115 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x55a8ddf80000 con 0x55a8de0e0a50 2023-07-18T03:07:08.525 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.530+0000 7fc5cf7fe700 10 client.25354 handle_command_reply: tid=0 2023-07-18T03:07:08.525 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.530+0000 7fc60a322b80 1 client.25354 shutdown 2023-07-18T03:07:08.525 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.530+0000 7fc60a322b80 1 -- 172.21.15.101:0/306129115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8de073480 msgr2=0x55a8de07ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:08.525 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.530+0000 7fc60a322b80 1 --2- 172.21.15.101:0/306129115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8de073480 0x55a8de07ee20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x55a8de07e180 tx=0x7fc5e0004500 comp rx=0 tx=0).stop 2023-07-18T03:07:08.525 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.530+0000 7fc60a322b80 1 -- 172.21.15.101:0/306129115 shutdown_connections 2023-07-18T03:07:08.526 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.530+0000 7fc60a322b80 1 --2- 172.21.15.101:0/306129115 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a8de0e0a50 0x55a8de0dd3a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.526 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.530+0000 7fc60a322b80 1 --2- 172.21.15.101:0/306129115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a8de074c20 0x55a8de0841f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.526 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.530+0000 7fc60a322b80 1 --2- 172.21.15.101:0/306129115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8de073480 0x55a8de07ee20 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.526 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.530+0000 7fc60a322b80 1 --2- 172.21.15.101:0/306129115 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a8de073dd0 0x55a8de07f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.526 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.530+0000 7fc5cf7fe700 0 client.25354 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:07:08.526 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.530+0000 7fc60a322b80 1 -- 172.21.15.101:0/306129115 >> 172.21.15.101:0/306129115 conn(0x55a8de068d50 msgr2=0x55a8de087750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:08.526 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.530+0000 7fc60a322b80 1 -- 172.21.15.101:0/306129115 shutdown_connections 2023-07-18T03:07:08.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.530+0000 7fc60a322b80 1 -- 172.21.15.101:0/306129115 wait complete. 2023-07-18T03:07:08.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.530+0000 7fc60a322b80 20 client.25354 trim_cache size 0 max 16384 2023-07-18T03:07:08.527 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:07:08.527 INFO:teuthology.orchestra.run.smithi101.stdout: "cluster_fsid": "b1782daa-2516-11ee-9b34-001a4aab830c", 2023-07-18T03:07:08.527 INFO:teuthology.orchestra.run.smithi101.stdout: "whoami": 0, 2023-07-18T03:07:08.527 INFO:teuthology.orchestra.run.smithi101.stdout: "id": 24562, 2023-07-18T03:07:08.528 INFO:teuthology.orchestra.run.smithi101.stdout: "want_state": "up:active", 2023-07-18T03:07:08.528 INFO:teuthology.orchestra.run.smithi101.stdout: "state": "up:active", 2023-07-18T03:07:08.528 INFO:teuthology.orchestra.run.smithi101.stdout: "fs_name": "cephfs", 2023-07-18T03:07:08.528 INFO:teuthology.orchestra.run.smithi101.stdout: "rank_uptime": 128.698829526, 2023-07-18T03:07:08.528 INFO:teuthology.orchestra.run.smithi101.stdout: "mdsmap_epoch": 24, 2023-07-18T03:07:08.528 INFO:teuthology.orchestra.run.smithi101.stdout: "osdmap_epoch": 91, 2023-07-18T03:07:08.528 INFO:teuthology.orchestra.run.smithi101.stdout: "osdmap_epoch_barrier": 89, 2023-07-18T03:07:08.529 INFO:teuthology.orchestra.run.smithi101.stdout: "uptime": 143.285823389 2023-07-18T03:07:08.529 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:07:08.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.531+0000 7fc5f69ff700 1 -- 172.21.15.101:0/552680155 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc5d80a1150 msgr2=0x7fc5d80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:08.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.531+0000 7fc5f69ff700 1 --2- 172.21.15.101:0/552680155 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc5d80a1150 0x7fc5d80a3600 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4013570 tx=0x7fc5e4009f90 comp rx=0 tx=0).stop 2023-07-18T03:07:08.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.531+0000 7fc5f69ff700 1 -- 172.21.15.101:0/552680155 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc5f0101680 msgr2=0x7fc5f019f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:08.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.531+0000 7fc5f69ff700 1 --2- 172.21.15.101:0/552680155 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc5f0101680 0x7fc5f019f0c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc5e800ef30 tx=0x7fc5e800c3a0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.531+0000 7fc5f69ff700 1 -- 172.21.15.101:0/552680155 shutdown_connections 2023-07-18T03:07:08.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.531+0000 7fc5f69ff700 1 --2- 172.21.15.101:0/552680155 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc5d80a1150 0x7fc5d80a3600 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.531+0000 7fc5f69ff700 1 --2- 172.21.15.101:0/552680155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5f00ffb00 0x7fc5f0197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.531+0000 7fc5f69ff700 1 --2- 172.21.15.101:0/552680155 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc5f0101680 0x7fc5f019f0c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.531+0000 7fc5f69ff700 1 --2- 172.21.15.101:0/552680155 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc5f0100cf0 0x7fc5f0198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:08.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.531+0000 7fc5f69ff700 1 -- 172.21.15.101:0/552680155 >> 172.21.15.101:0/552680155 conn(0x7fc5f00fb120 msgr2=0x7fc5f0106650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:08.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.531+0000 7fc5f69ff700 1 -- 172.21.15.101:0/552680155 shutdown_connections 2023-07-18T03:07:08.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:08.531+0000 7fc5f69ff700 1 -- 172.21.15.101:0/552680155 wait complete. 2023-07-18T03:07:08.536 INFO:teuthology.run_tasks:Running task kclient... 2023-07-18T03:07:08.550 INFO:tasks.kclient:Mounting kernel clients... 2023-07-18T03:07:08.550 INFO:tasks.kclient:config is {'client.0': {}} 2023-07-18T03:07:08.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:07:08.551 DEBUG:teuthology.orchestra.run.smithi101:> ip netns list 2023-07-18T03:07:08.595 INFO:teuthology.orchestra.run.smithi101.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-07-18T03:07:08.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:07:08.596 DEBUG:teuthology.orchestra.run.smithi101:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-07-18T03:07:08.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:07:08.704 DEBUG:teuthology.orchestra.run.smithi101:> sudo ip link delete ceph-brx 2023-07-18T03:07:08.811 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2023-07-18T03:07:08.812 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-18T03:07:08.812 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-18T03:07:08.813 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph fs ls 2023-07-18T03:07:09.535 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:07:10.146 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:09 smithi101 ceph-mon[126379]: pgmap v344: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 341 B/s wr, 6 op/s 2023-07-18T03:07:10.194 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:09 smithi179 ceph-mon[129229]: pgmap v344: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 341 B/s wr, 6 op/s 2023-07-18T03:07:10.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:09 smithi155 ceph-mon[128885]: pgmap v344: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 341 B/s wr, 6 op/s 2023-07-18T03:07:11.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.117+0000 7fb63ee49700 1 -- 172.21.15.101:0/4235741207 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb6380731c0 msgr2=0x7fb63810bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:11.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.117+0000 7fb63ee49700 1 --2- 172.21.15.101:0/4235741207 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb6380731c0 0x7fb63810bea0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb624009990 tx=0x7fb624009ca0 comp rx=0 tx=0).stop 2023-07-18T03:07:11.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.118+0000 7fb63ee49700 1 -- 172.21.15.101:0/4235741207 shutdown_connections 2023-07-18T03:07:11.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.118+0000 7fb63ee49700 1 --2- 172.21.15.101:0/4235741207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6380728b0 0x7fb638072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:11.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.118+0000 7fb63ee49700 1 --2- 172.21.15.101:0/4235741207 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb63810c3e0 0x7fb63810e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:11.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.118+0000 7fb63ee49700 1 --2- 172.21.15.101:0/4235741207 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb6380731c0 0x7fb63810bea0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:11.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.118+0000 7fb63ee49700 1 -- 172.21.15.101:0/4235741207 >> 172.21.15.101:0/4235741207 conn(0x7fb6380fb660 msgr2=0x7fb6380fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:11.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.118+0000 7fb63ee49700 1 -- 172.21.15.101:0/4235741207 shutdown_connections 2023-07-18T03:07:11.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.118+0000 7fb63ee49700 1 -- 172.21.15.101:0/4235741207 wait complete. 2023-07-18T03:07:11.108 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.119+0000 7fb63ee49700 1 Processor -- start 2023-07-18T03:07:11.108 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.119+0000 7fb63ee49700 1 -- start start 2023-07-18T03:07:11.108 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.119+0000 7fb63ee49700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6380728b0 0x7fb638197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:11.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.119+0000 7fb63ee49700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6380731c0 0x7fb638198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:11.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.119+0000 7fb63ee49700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb63810c3e0 0x7fb63819c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:11.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.119+0000 7fb63ee49700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63819ce40 con 0x7fb63810c3e0 2023-07-18T03:07:11.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.119+0000 7fb63ee49700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63819cfb0 con 0x7fb6380731c0 2023-07-18T03:07:11.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.119+0000 7fb63ee49700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63819d170 con 0x7fb6380728b0 2023-07-18T03:07:11.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.119+0000 7fb63d3e6700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb63810c3e0 0x7fb63819c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:11.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.120+0000 7fb63cbe5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6380728b0 0x7fb638197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:11.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.120+0000 7fb63cbe5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6380728b0 0x7fb638197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:45872/0 (socket says 172.21.15.101:45872) 2023-07-18T03:07:11.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.120+0000 7fb63cbe5700 1 -- 172.21.15.101:0/345265287 learned_addr learned my addr 172.21.15.101:0/345265287 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:11.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.120+0000 7fb63d3e6700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb63810c3e0 0x7fb63819c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:36838/0 (socket says 172.21.15.101:36838) 2023-07-18T03:07:11.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.120+0000 7fb637fff700 1 --2- 172.21.15.101:0/345265287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6380731c0 0x7fb638198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:11.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.120+0000 7fb63cbe5700 1 -- 172.21.15.101:0/345265287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6380731c0 msgr2=0x7fb638198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:11.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.120+0000 7fb63cbe5700 1 --2- 172.21.15.101:0/345265287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6380731c0 0x7fb638198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:11.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.120+0000 7fb63cbe5700 1 -- 172.21.15.101:0/345265287 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb63810c3e0 msgr2=0x7fb63819c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:11.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.120+0000 7fb63cbe5700 1 --2- 172.21.15.101:0/345265287 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb63810c3e0 0x7fb63819c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:11.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.120+0000 7fb63cbe5700 1 -- 172.21.15.101:0/345265287 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6240095f0 con 0x7fb6380728b0 2023-07-18T03:07:11.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.120+0000 7fb63cbe5700 1 --2- 172.21.15.101:0/345265287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6380728b0 0x7fb638197df0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb62c00e9a0 tx=0x7fb62c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:11.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.120+0000 7fb635ffb700 1 -- 172.21.15.101:0/345265287 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62c009790 con 0x7fb6380728b0 2023-07-18T03:07:11.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.120+0000 7fb63ee49700 1 -- 172.21.15.101:0/345265287 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb63819d450 con 0x7fb6380728b0 2023-07-18T03:07:11.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.121+0000 7fb63ee49700 1 -- 172.21.15.101:0/345265287 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6381a4c70 con 0x7fb6380728b0 2023-07-18T03:07:11.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.121+0000 7fb635ffb700 1 -- 172.21.15.101:0/345265287 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb62c010430 con 0x7fb6380728b0 2023-07-18T03:07:11.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.121+0000 7fb635ffb700 1 -- 172.21.15.101:0/345265287 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62c018680 con 0x7fb6380728b0 2023-07-18T03:07:11.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.121+0000 7fb63ee49700 1 -- 172.21.15.101:0/345265287 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb63804e210 con 0x7fb6380728b0 2023-07-18T03:07:11.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.124+0000 7fb635ffb700 1 -- 172.21.15.101:0/345265287 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fb62c010810 con 0x7fb6380728b0 2023-07-18T03:07:11.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.125+0000 7fb635ffb700 1 --2- 172.21.15.101:0/345265287 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb6280a1150 0x7fb6280a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:11.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.125+0000 7fb635ffb700 1 -- 172.21.15.101:0/345265287 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fb62c014070 con 0x7fb6380728b0 2023-07-18T03:07:11.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.125+0000 7fb637fff700 1 --2- 172.21.15.101:0/345265287 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb6280a1150 0x7fb6280a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:11.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.126+0000 7fb637fff700 1 --2- 172.21.15.101:0/345265287 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb6280a1150 0x7fb6280a3600 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb638199410 tx=0x7fb624005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:11.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.127+0000 7fb635ffb700 1 -- 172.21.15.101:0/345265287 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb62c051790 con 0x7fb6380728b0 2023-07-18T03:07:11.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.366+0000 7fb63ee49700 1 -- 172.21.15.101:0/345265287 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb6381a4ee0 con 0x7fb6380728b0 2023-07-18T03:07:11.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.367+0000 7fb635ffb700 1 -- 172.21.15.101:0/345265287 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+88 (secure 0 0 0) 0x7fb62c0962e0 con 0x7fb6380728b0 2023-07-18T03:07:11.354 INFO:teuthology.orchestra.run.smithi101.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-07-18T03:07:11.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.369+0000 7fb63ee49700 1 -- 172.21.15.101:0/345265287 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb6280a1150 msgr2=0x7fb6280a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:11.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.369+0000 7fb63ee49700 1 --2- 172.21.15.101:0/345265287 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb6280a1150 0x7fb6280a3600 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb638199410 tx=0x7fb624005fb0 comp rx=0 tx=0).stop 2023-07-18T03:07:11.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.369+0000 7fb63ee49700 1 -- 172.21.15.101:0/345265287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6380728b0 msgr2=0x7fb638197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:11.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.369+0000 7fb63ee49700 1 --2- 172.21.15.101:0/345265287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6380728b0 0x7fb638197df0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb62c00e9a0 tx=0x7fb62c00ecb0 comp rx=0 tx=0).stop 2023-07-18T03:07:11.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.369+0000 7fb63ee49700 1 -- 172.21.15.101:0/345265287 shutdown_connections 2023-07-18T03:07:11.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.369+0000 7fb63ee49700 1 --2- 172.21.15.101:0/345265287 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb6280a1150 0x7fb6280a3600 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:11.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.369+0000 7fb63ee49700 1 --2- 172.21.15.101:0/345265287 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb63810c3e0 0x7fb63819c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:11.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.369+0000 7fb63ee49700 1 --2- 172.21.15.101:0/345265287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6380728b0 0x7fb638197df0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:11.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.369+0000 7fb63ee49700 1 --2- 172.21.15.101:0/345265287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6380731c0 0x7fb638198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:11.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.369+0000 7fb63ee49700 1 -- 172.21.15.101:0/345265287 >> 172.21.15.101:0/345265287 conn(0x7fb6380fb660 msgr2=0x7fb63810e480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:11.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.369+0000 7fb63ee49700 1 -- 172.21.15.101:0/345265287 shutdown_connections 2023-07-18T03:07:11.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:11.369+0000 7fb63ee49700 1 -- 172.21.15.101:0/345265287 wait complete. 2023-07-18T03:07:11.810 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-18T03:07:11.810 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-18T03:07:11.810 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi101.front.sepia.ceph.com 2023-07-18T03:07:11.811 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-18T03:07:11.811 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-18T03:07:11.811 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-18T03:07:11.811 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-18T03:07:11.811 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-18T03:07:11.811 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-18T03:07:11.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:07:11.812 DEBUG:teuthology.orchestra.run.smithi101:> ip addr 2023-07-18T03:07:11.859 INFO:teuthology.orchestra.run.smithi101.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-18T03:07:11.859 INFO:teuthology.orchestra.run.smithi101.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-18T03:07:11.860 INFO:teuthology.orchestra.run.smithi101.stdout: inet 127.0.0.1/8 scope host lo 2023-07-18T03:07:11.860 INFO:teuthology.orchestra.run.smithi101.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:07:11.860 INFO:teuthology.orchestra.run.smithi101.stdout: inet6 ::1/128 scope host 2023-07-18T03:07:11.861 INFO:teuthology.orchestra.run.smithi101.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:07:11.861 INFO:teuthology.orchestra.run.smithi101.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-18T03:07:11.861 INFO:teuthology.orchestra.run.smithi101.stdout: link/ether 0c:c4:7a:d8:a1:f2 brd ff:ff:ff:ff:ff:ff 2023-07-18T03:07:11.861 INFO:teuthology.orchestra.run.smithi101.stdout: altname enp7s0f0 2023-07-18T03:07:11.862 INFO:teuthology.orchestra.run.smithi101.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-18T03:07:11.862 INFO:teuthology.orchestra.run.smithi101.stdout: link/ether 0c:c4:7a:d8:a1:f3 brd ff:ff:ff:ff:ff:ff 2023-07-18T03:07:11.862 INFO:teuthology.orchestra.run.smithi101.stdout: altname enp7s0f1 2023-07-18T03:07:11.862 INFO:teuthology.orchestra.run.smithi101.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-18T03:07:11.863 INFO:teuthology.orchestra.run.smithi101.stdout: link/ether 0c:c4:7a:8f:d7:22 brd ff:ff:ff:ff:ff:ff 2023-07-18T03:07:11.863 INFO:teuthology.orchestra.run.smithi101.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-18T03:07:11.863 INFO:teuthology.orchestra.run.smithi101.stdout: link/ether 0c:c4:7a:8f:d7:23 brd ff:ff:ff:ff:ff:ff 2023-07-18T03:07:11.864 INFO:teuthology.orchestra.run.smithi101.stdout: inet 172.21.15.101/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-18T03:07:11.864 INFO:teuthology.orchestra.run.smithi101.stdout: valid_lft 41301sec preferred_lft 41301sec 2023-07-18T03:07:11.864 INFO:teuthology.orchestra.run.smithi101.stdout: inet6 fe80::ac5b:f81b:4a3:d684/64 scope link noprefixroute 2023-07-18T03:07:11.864 INFO:teuthology.orchestra.run.smithi101.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:07:11.865 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-07-18T03:07:11.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:07:11.866 DEBUG:teuthology.orchestra.run.smithi101:> (cd / && exec bash -c ' 2023-07-18T03:07:11.866 DEBUG:teuthology.orchestra.run.smithi101:> set -e 2023-07-18T03:07:11.866 DEBUG:teuthology.orchestra.run.smithi101:> sudo ip link add name ceph-brx type bridge 2023-07-18T03:07:11.867 DEBUG:teuthology.orchestra.run.smithi101:> sudo ip addr flush dev ceph-brx 2023-07-18T03:07:11.867 DEBUG:teuthology.orchestra.run.smithi101:> sudo ip link set ceph-brx up 2023-07-18T03:07:11.868 DEBUG:teuthology.orchestra.run.smithi101:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-07-18T03:07:11.868 DEBUG:teuthology.orchestra.run.smithi101:> ') 2023-07-18T03:07:12.065 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:11 smithi101 ceph-mon[126379]: pgmap v345: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2023-07-18T03:07:12.065 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:11 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/345265287' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-18T03:07:12.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:07:12.105 DEBUG:teuthology.orchestra.run.smithi101:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-18T03:07:12.172 INFO:teuthology.orchestra.run.smithi101.stdout:1 2023-07-18T03:07:12.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:07:12.175 DEBUG:teuthology.orchestra.run.smithi101:> route 2023-07-18T03:07:12.194 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:11 smithi179 ceph-mon[129229]: pgmap v345: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2023-07-18T03:07:12.194 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:11 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/345265287' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-18T03:07:12.224 INFO:teuthology.orchestra.run.smithi101.stdout:Kernel IP routing table 2023-07-18T03:07:12.224 INFO:teuthology.orchestra.run.smithi101.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-18T03:07:12.225 INFO:teuthology.orchestra.run.smithi101.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-18T03:07:12.225 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-18T03:07:12.225 INFO:teuthology.orchestra.run.smithi101.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-18T03:07:12.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:07:12.226 DEBUG:teuthology.orchestra.run.smithi101:> (cd / && exec bash -c ' 2023-07-18T03:07:12.227 DEBUG:teuthology.orchestra.run.smithi101:> set -e 2023-07-18T03:07:12.227 DEBUG:teuthology.orchestra.run.smithi101:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-18T03:07:12.227 DEBUG:teuthology.orchestra.run.smithi101:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-18T03:07:12.227 DEBUG:teuthology.orchestra.run.smithi101:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-18T03:07:12.228 DEBUG:teuthology.orchestra.run.smithi101:> ') 2023-07-18T03:07:12.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:11 smithi155 ceph-mon[128885]: pgmap v345: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2023-07-18T03:07:12.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:11 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/345265287' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-18T03:07:12.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:07:12.401 DEBUG:teuthology.orchestra.run.smithi101:> ip netns list 2023-07-18T03:07:12.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:07:12.445 DEBUG:teuthology.orchestra.run.smithi101:> ip netns list-id 2023-07-18T03:07:12.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:07:12.528 DEBUG:teuthology.orchestra.run.smithi101:> (cd / && exec bash -c ' 2023-07-18T03:07:12.528 DEBUG:teuthology.orchestra.run.smithi101:> set -e 2023-07-18T03:07:12.529 DEBUG:teuthology.orchestra.run.smithi101:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-18T03:07:12.529 DEBUG:teuthology.orchestra.run.smithi101:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-07-18T03:07:12.529 DEBUG:teuthology.orchestra.run.smithi101:> ') 2023-07-18T03:07:12.710 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-07-18T03:07:12.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:07:12.711 DEBUG:teuthology.orchestra.run.smithi101:> (cd / && exec bash -c ' 2023-07-18T03:07:12.712 DEBUG:teuthology.orchestra.run.smithi101:> set -e 2023-07-18T03:07:12.712 DEBUG:teuthology.orchestra.run.smithi101:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-07-18T03:07:12.713 DEBUG:teuthology.orchestra.run.smithi101:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-07-18T03:07:12.713 DEBUG:teuthology.orchestra.run.smithi101:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-07-18T03:07:12.713 DEBUG:teuthology.orchestra.run.smithi101:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-07-18T03:07:12.713 DEBUG:teuthology.orchestra.run.smithi101:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-07-18T03:07:12.714 DEBUG:teuthology.orchestra.run.smithi101:> ') 2023-07-18T03:07:12.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:07:12.963 DEBUG:teuthology.orchestra.run.smithi101:> (cd / && exec bash -c ' 2023-07-18T03:07:12.964 DEBUG:teuthology.orchestra.run.smithi101:> set -e 2023-07-18T03:07:12.964 DEBUG:teuthology.orchestra.run.smithi101:> sudo ip link set brx.0 up 2023-07-18T03:07:12.964 DEBUG:teuthology.orchestra.run.smithi101:> sudo ip link set dev brx.0 master ceph-brx 2023-07-18T03:07:12.964 DEBUG:teuthology.orchestra.run.smithi101:> ') 2023-07-18T03:07:13.136 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-18T03:07:13.137 DEBUG:teuthology.orchestra.run.smithi101:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-18T03:07:13.180 INFO:teuthology.orchestra.run.smithi101.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-18T03:07:13.180 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-18T03:07:13.181 DEBUG:teuthology.orchestra.run.smithi101:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-18T03:07:13.262 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2023-07-18T03:07:13.263 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2023-07-18T03:07:13.263 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/_nogroup/sv_1/e0ae3bb7-7d8a-4636-a330-38d2ea2c61af 2023-07-18T03:07:13.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:07:13.264 DEBUG:teuthology.orchestra.run.smithi101:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/_nogroup/sv_1/e0ae3bb7-7d8a-4636-a330-38d2ea2c61af /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2023-07-18T03:07:13.406 INFO:teuthology.orchestra.run.smithi101.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2023-07-18T03:07:13.406 INFO:teuthology.orchestra.run.smithi101.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2023-07-18T03:07:13.407 INFO:teuthology.orchestra.run.smithi101.stdout:invalid new device string format 2023-07-18T03:07:13.887 INFO:teuthology.orchestra.run.smithi101.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2023-07-18T03:07:13.887 INFO:teuthology.orchestra.run.smithi101.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2023-07-18T03:07:13.888 INFO:teuthology.orchestra.run.smithi101.stdout:invalid new device string format 2023-07-18T03:07:13.888 INFO:teuthology.orchestra.run.smithi101.stdout:mount.ceph: resolved to: "172.21.15.101:3300,172.21.15.155:3300,172.21.15.179:3300" 2023-07-18T03:07:13.888 INFO:teuthology.orchestra.run.smithi101.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.101:3300,172.21.15.155:3300,172.21.15.179:3300:/volumes/_nogroup/sv_1/e0ae3bb7-7d8a-4636-a330-38d2ea2c61af 2023-07-18T03:07:13.888 INFO:teuthology.orchestra.run.smithi101.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,key=0,fsid=b1782daa-2516-11ee-9b34-001a4aab830c" will pass to kernel 2023-07-18T03:07:13.895 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-07-18T03:07:13.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:07:13.895 DEBUG:teuthology.orchestra.run.smithi101:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-18T03:07:13.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:07:13.967 DEBUG:teuthology.orchestra.run.smithi101:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-18T03:07:14.054 INFO:teuthology.orchestra.run.smithi101.stdout:client25328 2023-07-18T03:07:14.054 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-07-18T03:07:14.054 DEBUG:teuthology.orchestra.run.smithi101:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:07:14.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:07:14.098 DEBUG:teuthology.orchestra.run.smithi101:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-18T03:07:14.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:13 smithi179 ceph-mon[129229]: pgmap v346: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-18T03:07:14.222 INFO:teuthology.orchestra.run.smithi101.stdout:client25328 2023-07-18T03:07:14.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:07:14.223 DEBUG:teuthology.orchestra.run.smithi101:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-18T03:07:14.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:13 smithi101 ceph-mon[126379]: pgmap v346: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-18T03:07:14.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:13 smithi155 ceph-mon[128885]: pgmap v346: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-18T03:07:14.308 INFO:teuthology.orchestra.run.smithi101.stdout:client25328 2023-07-18T03:07:14.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:07:14.309 DEBUG:teuthology.orchestra.run.smithi101:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/b1782daa-2516-11ee-9b34-001a4aab830c.client25328/status') 2023-07-18T03:07:14.455 INFO:teuthology.orchestra.run.smithi101.stdout:instance: client.25328 (3)172.21.15.101:0/114712741 2023-07-18T03:07:14.456 INFO:teuthology.orchestra.run.smithi101.stdout:blocklisted: false 2023-07-18T03:07:14.456 INFO:teuthology.orchestra.run.smithi101.stderr:0+1 records in 2023-07-18T03:07:14.456 INFO:teuthology.orchestra.run.smithi101.stderr:0+1 records out 2023-07-18T03:07:14.456 INFO:teuthology.orchestra.run.smithi101.stderr:71 bytes copied, 5.0932e-05 s, 1.4 MB/s 2023-07-18T03:07:14.459 INFO:teuthology.run_tasks:Running task check-counter... 2023-07-18T03:07:14.470 INFO:teuthology.run_tasks:Running task fwd_scrub... 2023-07-18T03:07:14.485 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:07:14.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.695+0000 7f77b87ff700 1 -- 172.21.15.101:0/1465433482 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f77b00ffb00 msgr2=0x7f77b00fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:14.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.695+0000 7f77b87ff700 1 --2- 172.21.15.101:0/1465433482 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f77b00ffb00 0x7f77b00fff10 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f77ac009990 tx=0x7f77ac009ca0 comp rx=0 tx=0).stop 2023-07-18T03:07:14.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.695+0000 7f77b87ff700 1 -- 172.21.15.101:0/1465433482 shutdown_connections 2023-07-18T03:07:14.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.695+0000 7f77b87ff700 1 --2- 172.21.15.101:0/1465433482 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f77b0101680 0x7f77b0107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:14.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.695+0000 7f77b87ff700 1 --2- 172.21.15.101:0/1465433482 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f77b00ffb00 0x7f77b00fff10 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:14.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.695+0000 7f77b87ff700 1 --2- 172.21.15.101:0/1465433482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77b0100cf0 0x7f77b0101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:14.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.695+0000 7f77b87ff700 1 -- 172.21.15.101:0/1465433482 >> 172.21.15.101:0/1465433482 conn(0x7f77b00fb0e0 msgr2=0x7f77b00fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:14.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.695+0000 7f77b87ff700 1 -- 172.21.15.101:0/1465433482 shutdown_connections 2023-07-18T03:07:14.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.695+0000 7f77b87ff700 1 -- 172.21.15.101:0/1465433482 wait complete. 2023-07-18T03:07:14.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.696+0000 7f77b87ff700 1 Processor -- start 2023-07-18T03:07:14.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.697+0000 7f77b87ff700 1 -- start start 2023-07-18T03:07:14.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.697+0000 7f77b87ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f77b00ffb00 0x7f77b0195840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:14.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.697+0000 7f77b87ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77b0100cf0 0x7f77b0195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:14.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.697+0000 7f77b87ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f77b0101680 0x7f77b019cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:14.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.697+0000 7f77b87ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b019d3c0 con 0x7f77b0101680 2023-07-18T03:07:14.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.697+0000 7f77b87ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b019d500 con 0x7f77b0100cf0 2023-07-18T03:07:14.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.697+0000 7f77b87ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b019d690 con 0x7f77b00ffb00 2023-07-18T03:07:14.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.697+0000 7f77b659b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f77b00ffb00 0x7f77b0195840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:14.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.697+0000 7f77b5d9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77b0100cf0 0x7f77b0195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:14.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.697+0000 7f77b6d9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f77b0101680 0x7f77b019cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:14.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.697+0000 7f77b659b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f77b00ffb00 0x7f77b0195840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:45898/0 (socket says 172.21.15.101:45898) 2023-07-18T03:07:14.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.697+0000 7f77b659b700 1 -- 172.21.15.101:0/1644856949 learned_addr learned my addr 172.21.15.101:0/1644856949 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:14.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.698+0000 7f77b659b700 1 -- 172.21.15.101:0/1644856949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77b0100cf0 msgr2=0x7f77b0195d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:14.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.698+0000 7f77b659b700 1 --2- 172.21.15.101:0/1644856949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77b0100cf0 0x7f77b0195d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:14.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.698+0000 7f77b659b700 1 -- 172.21.15.101:0/1644856949 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f77b0101680 msgr2=0x7f77b019cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:14.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.698+0000 7f77b659b700 1 --2- 172.21.15.101:0/1644856949 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f77b0101680 0x7f77b019cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:14.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.698+0000 7f77b659b700 1 -- 172.21.15.101:0/1644856949 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77ac0095f0 con 0x7f77b00ffb00 2023-07-18T03:07:14.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.698+0000 7f77b659b700 1 --2- 172.21.15.101:0/1644856949 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f77b00ffb00 0x7f77b0195840 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f77ac004730 tx=0x7f77ac004760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:14.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.698+0000 7f77a77fe700 1 -- 172.21.15.101:0/1644856949 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac01d070 con 0x7f77b00ffb00 2023-07-18T03:07:14.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.698+0000 7f77b87ff700 1 -- 172.21.15.101:0/1644856949 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77b019d910 con 0x7f77b00ffb00 2023-07-18T03:07:14.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.698+0000 7f77a77fe700 1 -- 172.21.15.101:0/1644856949 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77ac021410 con 0x7f77b00ffb00 2023-07-18T03:07:14.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.698+0000 7f77a77fe700 1 -- 172.21.15.101:0/1644856949 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac017650 con 0x7f77b00ffb00 2023-07-18T03:07:14.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.698+0000 7f77b87ff700 1 -- 172.21.15.101:0/1644856949 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77b019dd90 con 0x7f77b00ffb00 2023-07-18T03:07:14.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.699+0000 7f77b87ff700 1 -- 172.21.15.101:0/1644856949 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77b010b320 con 0x7f77b00ffb00 2023-07-18T03:07:14.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.702+0000 7f77a77fe700 1 -- 172.21.15.101:0/1644856949 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f77ac021580 con 0x7f77b00ffb00 2023-07-18T03:07:14.690 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.703+0000 7f77a77fe700 1 --2- 172.21.15.101:0/1644856949 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f779c0a11a0 0x7f779c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:14.690 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.703+0000 7f77a77fe700 1 -- 172.21.15.101:0/1644856949 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f77ac0ce750 con 0x7f77b00ffb00 2023-07-18T03:07:14.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.703+0000 7f77b5d9a700 1 --2- 172.21.15.101:0/1644856949 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f779c0a11a0 0x7f779c0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:14.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.704+0000 7f77b5d9a700 1 --2- 172.21.15.101:0/1644856949 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f779c0a11a0 0x7f779c0a3650 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f77a0009c60 tx=0x7f77a0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:14.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.705+0000 7f77a77fe700 1 -- 172.21.15.101:0/1644856949 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f77ac097b60 con 0x7f77b00ffb00 2023-07-18T03:07:14.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.914+0000 7f77b87ff700 1 -- 172.21.15.101:0/1644856949 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77b01069a0 con 0x7f77b00ffb00 2023-07-18T03:07:14.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.916+0000 7f77a77fe700 1 -- 172.21.15.101:0/1644856949 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24725 (secure 0 0 0) 0x7f77ac09b180 con 0x7f77b00ffb00 2023-07-18T03:07:14.904 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:07:14.905 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":91,"fsid":"b1782daa-2516-11ee-9b34-001a4aab830c","created":"2023-07-18T02:57:22.972483+0000","modified":"2023-07-18T03:06:24.250777+0000","last_up_change":"2023-07-18T03:04:11.429530+0000","last_in_change":"2023-07-18T03:03:54.158340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:00:43.353489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:04:57.771616+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-18T03:04:58.769590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-18T03:05:02.358830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc76b5d1-38a0-4f1a-9335-756bfc40a62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6803","nonce":3610719929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6805","nonce":3610719929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6809","nonce":3610719929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6807","nonce":3610719929}]},"public_addr":"172.21.15.101:6803/3610719929","cluster_addr":"172.21.15.101:6805/3610719929","heartbeat_back_addr":"172.21.15.101:6809/3610719929","heartbeat_front_addr":"172.21.15.101:6807/3610719929","state":["exists","up"]},{"osd":1,"uuid":"2a299573-0b34-4af9-aaae-b523e5058ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6801","nonce":1030229579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6803","nonce":1030229579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6807","nonce":1030229579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6805","nonce":1030229579}]},"public_addr":"172.21.15.155:6801/1030229579","cluster_addr":"172.21.15.155:6803/1030229579","heartbeat_back_addr":"172.21.15.155:6807/1030229579","heartbeat_front_addr":"172.21.15.155:6805/1030229579","state":["exists","up"]},{"osd":2,"uuid":"123b3749-c636-48fa-9e33-6229b5575dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6801","nonce":1349487507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6803","nonce":1349487507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6807","nonce":1349487507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6805","nonce":1349487507}]},"public_addr":"172.21.15.179:6801/1349487507","cluster_addr":"172.21.15.179:6803/1349487507","heartbeat_back_addr":"172.21.15.179:6807/1349487507","heartbeat_front_addr":"172.21.15.179:6805/1349487507","state":["exists","up"]},{"osd":3,"uuid":"4b0c409d-2954-4127-833a-b196bf226d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6811","nonce":4226577463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6813","nonce":4226577463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6817","nonce":4226577463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6815","nonce":4226577463}]},"public_addr":"172.21.15.101:6811/4226577463","cluster_addr":"172.21.15.101:6813/4226577463","heartbeat_back_addr":"172.21.15.101:6817/4226577463","heartbeat_front_addr":"172.21.15.101:6815/4226577463","state":["exists","up"]},{"osd":4,"uuid":"2137f2df-984e-45ce-baf6-b4ac12c7e3c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6809","nonce":1951135805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6811","nonce":1951135805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6815","nonce":1951135805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6813","nonce":1951135805}]},"public_addr":"172.21.15.155:6809/1951135805","cluster_addr":"172.21.15.155:6811/1951135805","heartbeat_back_addr":"172.21.15.155:6815/1951135805","heartbeat_front_addr":"172.21.15.155:6813/1951135805","state":["exists","up"]},{"osd":5,"uuid":"1020bfc5-8c2f-482e-9588-66c7f98e46e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6809","nonce":2746037021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6811","nonce":2746037021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6815","nonce":2746037021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6813","nonce":2746037021}]},"public_addr":"172.21.15.179:6809/2746037021","cluster_addr":"172.21.15.179:6811/2746037021","heartbeat_back_addr":"172.21.15.179:6815/2746037021","heartbeat_front_addr":"172.21.15.179:6813/2746037021","state":["exists","up"]},{"osd":6,"uuid":"2cd99260-5c36-4603-9338-cf697970ce58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6819","nonce":958044931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6821","nonce":958044931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6825","nonce":958044931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6823","nonce":958044931}]},"public_addr":"172.21.15.101:6819/958044931","cluster_addr":"172.21.15.101:6821/958044931","heartbeat_back_addr":"172.21.15.101:6825/958044931","heartbeat_front_addr":"172.21.15.101:6823/958044931","state":["exists","up"]},{"osd":7,"uuid":"ed23fd0a-678f-4e15-b42c-cfe873315714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6817","nonce":3256322291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6819","nonce":3256322291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6823","nonce":3256322291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6821","nonce":3256322291}]},"public_addr":"172.21.15.155:6817/3256322291","cluster_addr":"172.21.15.155:6819/3256322291","heartbeat_back_addr":"172.21.15.155:6823/3256322291","heartbeat_front_addr":"172.21.15.155:6821/3256322291","state":["exists","up"]},{"osd":8,"uuid":"eca3d8e6-7baf-47a5-b991-328001c7ebef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6817","nonce":3520612559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6819","nonce":3520612559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6823","nonce":3520612559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6821","nonce":3520612559}]},"public_addr":"172.21.15.179:6817/3520612559","cluster_addr":"172.21.15.179:6819/3520612559","heartbeat_back_addr":"172.21.15.179:6823/3520612559","heartbeat_front_addr":"172.21.15.179:6821/3520612559","state":["exists","up"]},{"osd":9,"uuid":"465a7ffb-67e4-49ef-812c-1b96feacfebf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6827","nonce":2166787667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6829","nonce":2166787667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6833","nonce":2166787667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6831","nonce":2166787667}]},"public_addr":"172.21.15.101:6827/2166787667","cluster_addr":"172.21.15.101:6829/2166787667","heartbeat_back_addr":"172.21.15.101:6833/2166787667","heartbeat_front_addr":"172.21.15.101:6831/2166787667","state":["exists","up"]},{"osd":10,"uuid":"1897a027-f25d-434f-8df5-5fc2bd866578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6825","nonce":3682446349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6827","nonce":3682446349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6831","nonce":3682446349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6829","nonce":3682446349}]},"public_addr":"172.21.15.155:6825/3682446349","cluster_addr":"172.21.15.155:6827/3682446349","heartbeat_back_addr":"172.21.15.155:6831/3682446349","heartbeat_front_addr":"172.21.15.155:6829/3682446349","state":["exists","up"]},{"osd":11,"uuid":"b62effaf-45e2-4585-b8d0-db448f3474ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6825","nonce":2602640858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6827","nonce":2602640858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6831","nonce":2602640858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6829","nonce":2602640858}]},"public_addr":"172.21.15.179:6825/2602640858","cluster_addr":"172.21.15.179:6827/2602640858","heartbeat_back_addr":"172.21.15.179:6831/2602640858","heartbeat_front_addr":"172.21.15.179:6829/2602640858","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T02:59:56.074519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:19.358142+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:41.128966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:08.227054+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:31.442592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:52.751685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:19.251054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:40.698241+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:01.339150+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:27.955910+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:49.136980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:04:10.084982+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.101:6800/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:6801/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/604228291":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/4055187849":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/2844574324":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/1603248215":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6801/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6800/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/3307578182":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/373189792":"2023-07-19T02:57:56.243200+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:07:14.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.919+0000 7f77b87ff700 1 -- 172.21.15.101:0/1644856949 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f779c0a11a0 msgr2=0x7f779c0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:14.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.919+0000 7f77b87ff700 1 --2- 172.21.15.101:0/1644856949 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f779c0a11a0 0x7f779c0a3650 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f77a0009c60 tx=0x7f77a0009260 comp rx=0 tx=0).stop 2023-07-18T03:07:14.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.919+0000 7f77b87ff700 1 -- 172.21.15.101:0/1644856949 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f77b00ffb00 msgr2=0x7f77b0195840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:14.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.919+0000 7f77b87ff700 1 --2- 172.21.15.101:0/1644856949 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f77b00ffb00 0x7f77b0195840 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f77ac004730 tx=0x7f77ac004760 comp rx=0 tx=0).stop 2023-07-18T03:07:14.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.919+0000 7f77b87ff700 1 -- 172.21.15.101:0/1644856949 shutdown_connections 2023-07-18T03:07:14.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.919+0000 7f77b87ff700 1 --2- 172.21.15.101:0/1644856949 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f779c0a11a0 0x7f779c0a3650 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:14.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.919+0000 7f77b87ff700 1 --2- 172.21.15.101:0/1644856949 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f77b0101680 0x7f77b019cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:14.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.919+0000 7f77b87ff700 1 --2- 172.21.15.101:0/1644856949 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f77b00ffb00 0x7f77b0195840 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:14.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.919+0000 7f77b87ff700 1 --2- 172.21.15.101:0/1644856949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77b0100cf0 0x7f77b0195d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:14.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.919+0000 7f77b87ff700 1 -- 172.21.15.101:0/1644856949 >> 172.21.15.101:0/1644856949 conn(0x7f77b00fb0e0 msgr2=0x7f77b00fd320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:14.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.919+0000 7f77b87ff700 1 -- 172.21.15.101:0/1644856949 shutdown_connections 2023-07-18T03:07:14.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:14.919+0000 7f77b87ff700 1 -- 172.21.15.101:0/1644856949 wait complete. 2023-07-18T03:07:14.925 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:00:43.353489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-18T03:04:57.771616+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-18T03:04:58.769590+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-18T03:05:02.358830+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-18T03:07:14.926 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-18T03:07:15.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.139+0000 7f1da49ff700 1 -- 172.21.15.101:0/1484216095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1da00ffae0 msgr2=0x7f1da00ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:15.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.139+0000 7f1da49ff700 1 --2- 172.21.15.101:0/1484216095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1da00ffae0 0x7f1da00ffef0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1d90009990 tx=0x7f1d90009ca0 comp rx=0 tx=0).stop 2023-07-18T03:07:15.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.139+0000 7f1da49ff700 1 -- 172.21.15.101:0/1484216095 shutdown_connections 2023-07-18T03:07:15.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.139+0000 7f1da49ff700 1 --2- 172.21.15.101:0/1484216095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1da00ffae0 0x7f1da00ffef0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:15.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.139+0000 7f1da49ff700 1 --2- 172.21.15.101:0/1484216095 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1da0101660 0x7f1da0107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:15.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.139+0000 7f1da49ff700 1 --2- 172.21.15.101:0/1484216095 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1da0100cd0 0x7f1da0101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:15.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.139+0000 7f1da49ff700 1 -- 172.21.15.101:0/1484216095 >> 172.21.15.101:0/1484216095 conn(0x7f1da00fb0e0 msgr2=0x7f1da00fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:15.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.139+0000 7f1da49ff700 1 -- 172.21.15.101:0/1484216095 shutdown_connections 2023-07-18T03:07:15.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.139+0000 7f1da49ff700 1 -- 172.21.15.101:0/1484216095 wait complete. 2023-07-18T03:07:15.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.141+0000 7f1da49ff700 1 Processor -- start 2023-07-18T03:07:15.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.141+0000 7f1da49ff700 1 -- start start 2023-07-18T03:07:15.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.141+0000 7f1da49ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1da00ffae0 0x7f1da01958c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:15.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.141+0000 7f1da49ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1da0100cd0 0x7f1da0195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:15.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.142+0000 7f1da49ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1da0101660 0x7f1da019ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:15.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.142+0000 7f1da49ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da019d440 con 0x7f1da00ffae0 2023-07-18T03:07:15.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.142+0000 7f1da49ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da019d580 con 0x7f1da0100cd0 2023-07-18T03:07:15.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.142+0000 7f1da49ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da019d710 con 0x7f1da0101660 2023-07-18T03:07:15.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.142+0000 7f1d9e59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1da00ffae0 0x7f1da01958c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:15.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.142+0000 7f1d9e59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1da00ffae0 0x7f1da01958c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:36870/0 (socket says 172.21.15.101:36870) 2023-07-18T03:07:15.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.142+0000 7f1d9e59c700 1 -- 172.21.15.101:0/816670413 learned_addr learned my addr 172.21.15.101:0/816670413 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:15.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.142+0000 7f1d9ed9d700 1 --2- 172.21.15.101:0/816670413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1da0101660 0x7f1da019ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:15.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.142+0000 7f1d9dd9b700 1 --2- 172.21.15.101:0/816670413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1da0100cd0 0x7f1da0195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:15.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.142+0000 7f1d9e59c700 1 -- 172.21.15.101:0/816670413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1da0101660 msgr2=0x7f1da019ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:15.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.142+0000 7f1d9e59c700 1 --2- 172.21.15.101:0/816670413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1da0101660 0x7f1da019ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:15.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.142+0000 7f1d9e59c700 1 -- 172.21.15.101:0/816670413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1da0100cd0 msgr2=0x7f1da0195e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:15.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.142+0000 7f1d9e59c700 1 --2- 172.21.15.101:0/816670413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1da0100cd0 0x7f1da0195e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:15.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.142+0000 7f1d9e59c700 1 -- 172.21.15.101:0/816670413 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d94009520 con 0x7f1da00ffae0 2023-07-18T03:07:15.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.142+0000 7f1d9e59c700 1 --2- 172.21.15.101:0/816670413 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1da00ffae0 0x7f1da01958c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1d90004730 tx=0x7f1d90004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:15.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.143+0000 7f1d8f7fe700 1 -- 172.21.15.101:0/816670413 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9001d070 con 0x7f1da00ffae0 2023-07-18T03:07:15.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.143+0000 7f1da49ff700 1 -- 172.21.15.101:0/816670413 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d98009c40 con 0x7f1da00ffae0 2023-07-18T03:07:15.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.143+0000 7f1d8f7fe700 1 -- 172.21.15.101:0/816670413 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d90021410 con 0x7f1da00ffae0 2023-07-18T03:07:15.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.143+0000 7f1d8f7fe700 1 -- 172.21.15.101:0/816670413 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d90017650 con 0x7f1da00ffae0 2023-07-18T03:07:15.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.143+0000 7f1da49ff700 1 -- 172.21.15.101:0/816670413 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d900095f0 con 0x7f1da00ffae0 2023-07-18T03:07:15.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.144+0000 7f1da49ff700 1 -- 172.21.15.101:0/816670413 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1da010b300 con 0x7f1da00ffae0 2023-07-18T03:07:15.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.146+0000 7f1d8f7fe700 1 -- 172.21.15.101:0/816670413 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f1d90021810 con 0x7f1da00ffae0 2023-07-18T03:07:15.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.147+0000 7f1d8f7fe700 1 --2- 172.21.15.101:0/816670413 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f1d880a1150 0x7f1d880a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:15.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.147+0000 7f1d8f7fe700 1 -- 172.21.15.101:0/816670413 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f1d900ce0f0 con 0x7f1da00ffae0 2023-07-18T03:07:15.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.147+0000 7f1d9dd9b700 1 --2- 172.21.15.101:0/816670413 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f1d880a1150 0x7f1d880a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:15.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.148+0000 7f1d9dd9b700 1 --2- 172.21.15.101:0/816670413 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f1d880a1150 0x7f1d880a3600 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1d94009df0 tx=0x7f1d94009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:15.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.151+0000 7f1d8f7fe700 1 -- 172.21.15.101:0/816670413 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1d90097500 con 0x7f1da00ffae0 2023-07-18T03:07:15.194 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:14 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1644856949' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:07:15.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:14 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1644856949' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:07:15.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:14 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1644856949' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:07:15.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.358+0000 7f1da49ff700 1 -- 172.21.15.101:0/816670413 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1da019e130 con 0x7f1da00ffae0 2023-07-18T03:07:15.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.359+0000 7f1d8f7fe700 1 -- 172.21.15.101:0/816670413 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1d9009ab20 con 0x7f1da00ffae0 2023-07-18T03:07:15.347 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 1 2023-07-18T03:07:15.349 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.361+0000 7f1da49ff700 1 -- 172.21.15.101:0/816670413 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f1d880a1150 msgr2=0x7f1d880a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:15.349 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.361+0000 7f1da49ff700 1 --2- 172.21.15.101:0/816670413 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f1d880a1150 0x7f1d880a3600 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1d94009df0 tx=0x7f1d94009260 comp rx=0 tx=0).stop 2023-07-18T03:07:15.349 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.361+0000 7f1da49ff700 1 -- 172.21.15.101:0/816670413 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1da00ffae0 msgr2=0x7f1da01958c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:15.349 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.361+0000 7f1da49ff700 1 --2- 172.21.15.101:0/816670413 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1da00ffae0 0x7f1da01958c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1d90004730 tx=0x7f1d90004760 comp rx=0 tx=0).stop 2023-07-18T03:07:15.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.361+0000 7f1da49ff700 1 -- 172.21.15.101:0/816670413 shutdown_connections 2023-07-18T03:07:15.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.361+0000 7f1da49ff700 1 --2- 172.21.15.101:0/816670413 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f1d880a1150 0x7f1d880a3600 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:15.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.361+0000 7f1da49ff700 1 --2- 172.21.15.101:0/816670413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1da0101660 0x7f1da019ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:15.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.361+0000 7f1da49ff700 1 --2- 172.21.15.101:0/816670413 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1da00ffae0 0x7f1da01958c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:15.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.361+0000 7f1da49ff700 1 --2- 172.21.15.101:0/816670413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1da0100cd0 0x7f1da0195e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:15.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.361+0000 7f1da49ff700 1 -- 172.21.15.101:0/816670413 >> 172.21.15.101:0/816670413 conn(0x7f1da00fb0e0 msgr2=0x7f1da00fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:15.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.362+0000 7f1da49ff700 1 -- 172.21.15.101:0/816670413 shutdown_connections 2023-07-18T03:07:15.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.362+0000 7f1da49ff700 1 -- 172.21.15.101:0/816670413 wait complete. 2023-07-18T03:07:15.368 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-18T03:07:15.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.581+0000 7f6c339ff700 1 -- 172.21.15.101:0/2962307879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c2c0ffae0 msgr2=0x7f6c2c0ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:15.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.581+0000 7f6c339ff700 1 --2- 172.21.15.101:0/2962307879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c2c0ffae0 0x7f6c2c0ffef0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6c28009990 tx=0x7f6c28009ca0 comp rx=0 tx=0).stop 2023-07-18T03:07:15.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.581+0000 7f6c339ff700 1 -- 172.21.15.101:0/2962307879 shutdown_connections 2023-07-18T03:07:15.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.581+0000 7f6c339ff700 1 --2- 172.21.15.101:0/2962307879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c2c101660 0x7f6c2c107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:15.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.581+0000 7f6c339ff700 1 --2- 172.21.15.101:0/2962307879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c2c0ffae0 0x7f6c2c0ffef0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:15.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.581+0000 7f6c339ff700 1 --2- 172.21.15.101:0/2962307879 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c2c100cd0 0x7f6c2c101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:15.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.581+0000 7f6c339ff700 1 -- 172.21.15.101:0/2962307879 >> 172.21.15.101:0/2962307879 conn(0x7f6c2c0fb0e0 msgr2=0x7f6c2c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:15.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.581+0000 7f6c339ff700 1 -- 172.21.15.101:0/2962307879 shutdown_connections 2023-07-18T03:07:15.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.581+0000 7f6c339ff700 1 -- 172.21.15.101:0/2962307879 wait complete. 2023-07-18T03:07:15.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.582+0000 7f6c339ff700 1 Processor -- start 2023-07-18T03:07:15.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.582+0000 7f6c339ff700 1 -- start start 2023-07-18T03:07:15.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.582+0000 7f6c339ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c2c0ffae0 0x7f6c2c197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:15.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.582+0000 7f6c339ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c2c100cd0 0x7f6c2c198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:15.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.582+0000 7f6c339ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c2c101660 0x7f6c2c19f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:15.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.582+0000 7f6c339ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c19f650 con 0x7f6c2c100cd0 2023-07-18T03:07:15.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.582+0000 7f6c339ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c19f790 con 0x7f6c2c101660 2023-07-18T03:07:15.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.582+0000 7f6c339ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c19f920 con 0x7f6c2c0ffae0 2023-07-18T03:07:15.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.583+0000 7f6c3179b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c2c0ffae0 0x7f6c2c197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:15.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.583+0000 7f6c3179b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c2c0ffae0 0x7f6c2c197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:45928/0 (socket says 172.21.15.101:45928) 2023-07-18T03:07:15.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.583+0000 7f6c3179b700 1 -- 172.21.15.101:0/3906377069 learned_addr learned my addr 172.21.15.101:0/3906377069 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:15.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.583+0000 7f6c30f9a700 1 --2- 172.21.15.101:0/3906377069 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c2c100cd0 0x7f6c2c198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:15.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.583+0000 7f6c31f9c700 1 --2- 172.21.15.101:0/3906377069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c2c101660 0x7f6c2c19f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:15.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.583+0000 7f6c3179b700 1 -- 172.21.15.101:0/3906377069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c2c101660 msgr2=0x7f6c2c19f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:15.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.583+0000 7f6c3179b700 1 --2- 172.21.15.101:0/3906377069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c2c101660 0x7f6c2c19f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:15.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.583+0000 7f6c3179b700 1 -- 172.21.15.101:0/3906377069 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c2c100cd0 msgr2=0x7f6c2c198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:15.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.583+0000 7f6c3179b700 1 --2- 172.21.15.101:0/3906377069 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c2c100cd0 0x7f6c2c198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:15.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.583+0000 7f6c3179b700 1 -- 172.21.15.101:0/3906377069 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c280095f0 con 0x7f6c2c0ffae0 2023-07-18T03:07:15.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.583+0000 7f6c3179b700 1 --2- 172.21.15.101:0/3906377069 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c2c0ffae0 0x7f6c2c197ad0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6c28005720 tx=0x7f6c28009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:15.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.584+0000 7f6c227fc700 1 -- 172.21.15.101:0/3906377069 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2801d070 con 0x7f6c2c0ffae0 2023-07-18T03:07:15.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.584+0000 7f6c339ff700 1 -- 172.21.15.101:0/3906377069 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c2c19fba0 con 0x7f6c2c0ffae0 2023-07-18T03:07:15.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.584+0000 7f6c227fc700 1 -- 172.21.15.101:0/3906377069 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c28021410 con 0x7f6c2c0ffae0 2023-07-18T03:07:15.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.584+0000 7f6c227fc700 1 -- 172.21.15.101:0/3906377069 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c28017650 con 0x7f6c2c0ffae0 2023-07-18T03:07:15.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.584+0000 7f6c339ff700 1 -- 172.21.15.101:0/3906377069 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c2c1a0020 con 0x7f6c2c0ffae0 2023-07-18T03:07:15.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.585+0000 7f6c339ff700 1 -- 172.21.15.101:0/3906377069 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c2c192160 con 0x7f6c2c0ffae0 2023-07-18T03:07:15.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.587+0000 7f6c227fc700 1 -- 172.21.15.101:0/3906377069 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f6c28021580 con 0x7f6c2c0ffae0 2023-07-18T03:07:15.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.588+0000 7f6c227fc700 1 --2- 172.21.15.101:0/3906377069 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f6c180a1150 0x7f6c180a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:15.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.588+0000 7f6c227fc700 1 -- 172.21.15.101:0/3906377069 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f6c280ce680 con 0x7f6c2c0ffae0 2023-07-18T03:07:15.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.588+0000 7f6c30f9a700 1 --2- 172.21.15.101:0/3906377069 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f6c180a1150 0x7f6c180a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:15.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.589+0000 7f6c30f9a700 1 --2- 172.21.15.101:0/3906377069 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f6c180a1150 0x7f6c180a3600 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c009c60 tx=0x7f6c1c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:15.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.590+0000 7f6c227fc700 1 -- 172.21.15.101:0/3906377069 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6c28097a90 con 0x7f6c2c0ffae0 2023-07-18T03:07:15.778 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.791+0000 7f6c339ff700 1 -- 172.21.15.101:0/3906377069 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6c2c02ccd0 con 0x7f6c2c0ffae0 2023-07-18T03:07:15.779 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.792+0000 7f6c227fc700 1 -- 172.21.15.101:0/3906377069 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6c2c02ccd0 con 0x7f6c2c0ffae0 2023-07-18T03:07:15.779 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 32 2023-07-18T03:07:15.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.794+0000 7f6c339ff700 1 -- 172.21.15.101:0/3906377069 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f6c180a1150 msgr2=0x7f6c180a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:15.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.794+0000 7f6c339ff700 1 --2- 172.21.15.101:0/3906377069 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f6c180a1150 0x7f6c180a3600 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c009c60 tx=0x7f6c1c009260 comp rx=0 tx=0).stop 2023-07-18T03:07:15.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.794+0000 7f6c339ff700 1 -- 172.21.15.101:0/3906377069 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c2c0ffae0 msgr2=0x7f6c2c197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:15.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.794+0000 7f6c339ff700 1 --2- 172.21.15.101:0/3906377069 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c2c0ffae0 0x7f6c2c197ad0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6c28005720 tx=0x7f6c28009580 comp rx=0 tx=0).stop 2023-07-18T03:07:15.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.794+0000 7f6c339ff700 1 -- 172.21.15.101:0/3906377069 shutdown_connections 2023-07-18T03:07:15.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.794+0000 7f6c339ff700 1 --2- 172.21.15.101:0/3906377069 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f6c180a1150 0x7f6c180a3600 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:15.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.794+0000 7f6c339ff700 1 --2- 172.21.15.101:0/3906377069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c2c101660 0x7f6c2c19f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:15.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.794+0000 7f6c339ff700 1 --2- 172.21.15.101:0/3906377069 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c2c0ffae0 0x7f6c2c197ad0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:15.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.794+0000 7f6c339ff700 1 --2- 172.21.15.101:0/3906377069 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c2c100cd0 0x7f6c2c198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:15.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.794+0000 7f6c339ff700 1 -- 172.21.15.101:0/3906377069 >> 172.21.15.101:0/3906377069 conn(0x7f6c2c0fb0e0 msgr2=0x7f6c2c106610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:15.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.794+0000 7f6c339ff700 1 -- 172.21.15.101:0/3906377069 shutdown_connections 2023-07-18T03:07:15.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:15.794+0000 7f6c339ff700 1 -- 172.21.15.101:0/3906377069 wait complete. 2023-07-18T03:07:15.802 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-18T03:07:16.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.016+0000 7fc9929ff700 1 -- 172.21.15.101:0/1375417566 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc98c100d30 msgr2=0x7fc98c1011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:16.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.016+0000 7fc9929ff700 1 --2- 172.21.15.101:0/1375417566 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc98c100d30 0x7fc98c1011a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc974009990 tx=0x7fc974009ca0 comp rx=0 tx=0).stop 2023-07-18T03:07:16.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.016+0000 7fc9929ff700 1 -- 172.21.15.101:0/1375417566 shutdown_connections 2023-07-18T03:07:16.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.016+0000 7fc9929ff700 1 --2- 172.21.15.101:0/1375417566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc98c0ffaa0 0x7fc98c0ffeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:16.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.016+0000 7fc9929ff700 1 --2- 172.21.15.101:0/1375417566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc98c1016e0 0x7fc98c107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:16.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.016+0000 7fc9929ff700 1 --2- 172.21.15.101:0/1375417566 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc98c100d30 0x7fc98c1011a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:16.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.016+0000 7fc9929ff700 1 -- 172.21.15.101:0/1375417566 >> 172.21.15.101:0/1375417566 conn(0x7fc98c0fb0c0 msgr2=0x7fc98c0fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:16.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.016+0000 7fc9929ff700 1 -- 172.21.15.101:0/1375417566 shutdown_connections 2023-07-18T03:07:16.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.016+0000 7fc9929ff700 1 -- 172.21.15.101:0/1375417566 wait complete. 2023-07-18T03:07:16.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.017+0000 7fc9929ff700 1 Processor -- start 2023-07-18T03:07:16.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.017+0000 7fc9929ff700 1 -- start start 2023-07-18T03:07:16.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.018+0000 7fc9929ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc98c0ffaa0 0x7fc98c1979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:16.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.018+0000 7fc9929ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc98c100d30 0x7fc98c197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:16.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.018+0000 7fc9929ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc98c1016e0 0x7fc98c19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:16.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.018+0000 7fc9929ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98c19f570 con 0x7fc98c100d30 2023-07-18T03:07:16.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.018+0000 7fc9929ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98c19f6b0 con 0x7fc98c1016e0 2023-07-18T03:07:16.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.018+0000 7fc9929ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98c19f840 con 0x7fc98c0ffaa0 2023-07-18T03:07:16.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.018+0000 7fc9837fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc98c100d30 0x7fc98c197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:16.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.018+0000 7fc9837fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc98c100d30 0x7fc98c197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:36272/0 (socket says 172.21.15.101:36272) 2023-07-18T03:07:16.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.018+0000 7fc9837fe700 1 -- 172.21.15.101:0/1189888827 learned_addr learned my addr 172.21.15.101:0/1189888827 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:16.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.018+0000 7fc990f9c700 1 --2- 172.21.15.101:0/1189888827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc98c1016e0 0x7fc98c19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:16.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.018+0000 7fc98bfff700 1 --2- 172.21.15.101:0/1189888827 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc98c0ffaa0 0x7fc98c1979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:16.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.018+0000 7fc9837fe700 1 -- 172.21.15.101:0/1189888827 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc98c0ffaa0 msgr2=0x7fc98c1979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:16.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.018+0000 7fc9837fe700 1 --2- 172.21.15.101:0/1189888827 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc98c0ffaa0 0x7fc98c1979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:16.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.018+0000 7fc9837fe700 1 -- 172.21.15.101:0/1189888827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc98c1016e0 msgr2=0x7fc98c19efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:16.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.018+0000 7fc9837fe700 1 --2- 172.21.15.101:0/1189888827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc98c1016e0 0x7fc98c19efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:16.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.018+0000 7fc9837fe700 1 -- 172.21.15.101:0/1189888827 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9740095f0 con 0x7fc98c100d30 2023-07-18T03:07:16.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.018+0000 7fc9837fe700 1 --2- 172.21.15.101:0/1189888827 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc98c100d30 0x7fc98c197f30 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc974004730 tx=0x7fc974004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:16.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.019+0000 7fc989ffb700 1 -- 172.21.15.101:0/1189888827 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97401d070 con 0x7fc98c100d30 2023-07-18T03:07:16.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.019+0000 7fc989ffb700 1 -- 172.21.15.101:0/1189888827 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc974004c90 con 0x7fc98c100d30 2023-07-18T03:07:16.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.019+0000 7fc9929ff700 1 -- 172.21.15.101:0/1189888827 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc98c19fac0 con 0x7fc98c100d30 2023-07-18T03:07:16.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.019+0000 7fc989ffb700 1 -- 172.21.15.101:0/1189888827 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc974005570 con 0x7fc98c100d30 2023-07-18T03:07:16.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.019+0000 7fc9929ff700 1 -- 172.21.15.101:0/1189888827 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc98c19fee0 con 0x7fc98c100d30 2023-07-18T03:07:16.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.020+0000 7fc9929ff700 1 -- 172.21.15.101:0/1189888827 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc98c10b2c0 con 0x7fc98c100d30 2023-07-18T03:07:16.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.022+0000 7fc989ffb700 1 -- 172.21.15.101:0/1189888827 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fc97400fc70 con 0x7fc98c100d30 2023-07-18T03:07:16.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.023+0000 7fc989ffb700 1 --2- 172.21.15.101:0/1189888827 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc9780a11d0 0x7fc9780a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:16.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.023+0000 7fc989ffb700 1 -- 172.21.15.101:0/1189888827 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fc9740ce400 con 0x7fc98c100d30 2023-07-18T03:07:16.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.023+0000 7fc98bfff700 1 --2- 172.21.15.101:0/1189888827 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc9780a11d0 0x7fc9780a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:16.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.024+0000 7fc98bfff700 1 --2- 172.21.15.101:0/1189888827 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc9780a11d0 0x7fc9780a3680 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc97c009380 tx=0x7fc97c00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:16.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.025+0000 7fc989ffb700 1 -- 172.21.15.101:0/1189888827 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc974097870 con 0x7fc98c100d30 2023-07-18T03:07:16.194 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:15 smithi179 ceph-mon[129229]: pgmap v347: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:07:16.194 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:15 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/816670413' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:07:16.194 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:15 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/3906377069' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:07:16.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.224+0000 7fc9929ff700 1 -- 172.21.15.101:0/1189888827 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc98c1a0550 con 0x7fc98c100d30 2023-07-18T03:07:16.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.224+0000 7fc989ffb700 1 -- 172.21.15.101:0/1189888827 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc97409ae90 con 0x7fc98c100d30 2023-07-18T03:07:16.213 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 64 2023-07-18T03:07:16.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.226+0000 7fc9929ff700 1 -- 172.21.15.101:0/1189888827 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc9780a11d0 msgr2=0x7fc9780a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:16.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.226+0000 7fc9929ff700 1 --2- 172.21.15.101:0/1189888827 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc9780a11d0 0x7fc9780a3680 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc97c009380 tx=0x7fc97c00b040 comp rx=0 tx=0).stop 2023-07-18T03:07:16.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.226+0000 7fc9929ff700 1 -- 172.21.15.101:0/1189888827 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc98c100d30 msgr2=0x7fc98c197f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:16.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.226+0000 7fc9929ff700 1 --2- 172.21.15.101:0/1189888827 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc98c100d30 0x7fc98c197f30 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc974004730 tx=0x7fc974004760 comp rx=0 tx=0).stop 2023-07-18T03:07:16.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.226+0000 7fc9929ff700 1 -- 172.21.15.101:0/1189888827 shutdown_connections 2023-07-18T03:07:16.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.226+0000 7fc9929ff700 1 --2- 172.21.15.101:0/1189888827 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fc9780a11d0 0x7fc9780a3680 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:16.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.226+0000 7fc9929ff700 1 --2- 172.21.15.101:0/1189888827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc98c1016e0 0x7fc98c19efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:16.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.226+0000 7fc9929ff700 1 --2- 172.21.15.101:0/1189888827 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc98c0ffaa0 0x7fc98c1979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:16.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.226+0000 7fc9929ff700 1 --2- 172.21.15.101:0/1189888827 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc98c100d30 0x7fc98c197f30 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:16.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.226+0000 7fc9929ff700 1 -- 172.21.15.101:0/1189888827 >> 172.21.15.101:0/1189888827 conn(0x7fc98c0fb0c0 msgr2=0x7fc98c0fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:16.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.226+0000 7fc9929ff700 1 -- 172.21.15.101:0/1189888827 shutdown_connections 2023-07-18T03:07:16.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.226+0000 7fc9929ff700 1 -- 172.21.15.101:0/1189888827 wait complete. 2023-07-18T03:07:16.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:15 smithi101 ceph-mon[126379]: pgmap v347: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:07:16.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:15 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/816670413' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:07:16.230 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:15 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3906377069' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:07:16.232 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-18T03:07:16.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:15 smithi155 ceph-mon[128885]: pgmap v347: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:07:16.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:15 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/816670413' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:07:16.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:15 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/3906377069' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:07:16.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.444+0000 7f96199ff700 1 -- 172.21.15.101:0/3622118382 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f96140ffaf0 msgr2=0x7f96140fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:16.432 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.444+0000 7f96199ff700 1 --2- 172.21.15.101:0/3622118382 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f96140ffaf0 0x7f96140fff00 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9604009990 tx=0x7f9604009ca0 comp rx=0 tx=0).stop 2023-07-18T03:07:16.432 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.444+0000 7f96199ff700 1 -- 172.21.15.101:0/3622118382 shutdown_connections 2023-07-18T03:07:16.432 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.444+0000 7f96199ff700 1 --2- 172.21.15.101:0/3622118382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9614101670 0x7f9614107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:16.432 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.444+0000 7f96199ff700 1 --2- 172.21.15.101:0/3622118382 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f96140ffaf0 0x7f96140fff00 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:16.432 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.444+0000 7f96199ff700 1 --2- 172.21.15.101:0/3622118382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9614100ce0 0x7f9614101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:16.433 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.444+0000 7f96199ff700 1 -- 172.21.15.101:0/3622118382 >> 172.21.15.101:0/3622118382 conn(0x7f96140fb130 msgr2=0x7f96140fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:16.433 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.444+0000 7f96199ff700 1 -- 172.21.15.101:0/3622118382 shutdown_connections 2023-07-18T03:07:16.433 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.444+0000 7f96199ff700 1 -- 172.21.15.101:0/3622118382 wait complete. 2023-07-18T03:07:16.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.445+0000 7f96199ff700 1 Processor -- start 2023-07-18T03:07:16.435 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.445+0000 7f96199ff700 1 -- start start 2023-07-18T03:07:16.435 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.445+0000 7f96199ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96140ffaf0 0x7f96140742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:16.435 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.445+0000 7f96199ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9614100ce0 0x7f9614072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:16.435 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.445+0000 7f96199ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9614101670 0x7f9614072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:16.435 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.445+0000 7f96199ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9614199720 con 0x7f9614100ce0 2023-07-18T03:07:16.435 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.445+0000 7f96199ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9614073380 con 0x7f96140ffaf0 2023-07-18T03:07:16.435 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.445+0000 7f96199ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9614073540 con 0x7f9614101670 2023-07-18T03:07:16.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.446+0000 7f96127fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9614100ce0 0x7f9614072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:16.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.446+0000 7f96127fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9614100ce0 0x7f9614072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:36296/0 (socket says 172.21.15.101:36296) 2023-07-18T03:07:16.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.446+0000 7f96127fc700 1 -- 172.21.15.101:0/3753809799 learned_addr learned my addr 172.21.15.101:0/3753809799 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:16.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.446+0000 7f9612ffd700 1 --2- 172.21.15.101:0/3753809799 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96140ffaf0 0x7f96140742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:16.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.446+0000 7f96137fe700 1 --2- 172.21.15.101:0/3753809799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9614101670 0x7f9614072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:16.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.446+0000 7f96127fc700 1 -- 172.21.15.101:0/3753809799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9614101670 msgr2=0x7f9614072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:16.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.446+0000 7f96127fc700 1 --2- 172.21.15.101:0/3753809799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9614101670 0x7f9614072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:16.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.446+0000 7f96127fc700 1 -- 172.21.15.101:0/3753809799 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96140ffaf0 msgr2=0x7f96140742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:16.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.446+0000 7f96127fc700 1 --2- 172.21.15.101:0/3753809799 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96140ffaf0 0x7f96140742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:16.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.446+0000 7f96127fc700 1 -- 172.21.15.101:0/3753809799 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96040095f0 con 0x7f9614100ce0 2023-07-18T03:07:16.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.446+0000 7f96127fc700 1 --2- 172.21.15.101:0/3753809799 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9614100ce0 0x7f9614072900 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f960800e9a0 tx=0x7f960800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:16.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.446+0000 7f96189fd700 1 -- 172.21.15.101:0/3753809799 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9608009790 con 0x7f9614100ce0 2023-07-18T03:07:16.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.447+0000 7f96189fd700 1 -- 172.21.15.101:0/3753809799 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9608010430 con 0x7f9614100ce0 2023-07-18T03:07:16.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.447+0000 7f96199ff700 1 -- 172.21.15.101:0/3753809799 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96141a40d0 con 0x7f9614100ce0 2023-07-18T03:07:16.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.447+0000 7f96189fd700 1 -- 172.21.15.101:0/3753809799 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9608018680 con 0x7f9614100ce0 2023-07-18T03:07:16.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.447+0000 7f96199ff700 1 -- 172.21.15.101:0/3753809799 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96141a4550 con 0x7f9614100ce0 2023-07-18T03:07:16.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.448+0000 7f96199ff700 1 -- 172.21.15.101:0/3753809799 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96141070b0 con 0x7f9614100ce0 2023-07-18T03:07:16.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.450+0000 7f96189fd700 1 -- 172.21.15.101:0/3753809799 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f96080105a0 con 0x7f9614100ce0 2023-07-18T03:07:16.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.451+0000 7f96189fd700 1 --2- 172.21.15.101:0/3753809799 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f95fc0a11d0 0x7f95fc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:16.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.451+0000 7f96189fd700 1 -- 172.21.15.101:0/3753809799 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f9608014070 con 0x7f9614100ce0 2023-07-18T03:07:16.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.451+0000 7f9612ffd700 1 --2- 172.21.15.101:0/3753809799 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f95fc0a11d0 0x7f95fc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:16.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.452+0000 7f9612ffd700 1 --2- 172.21.15.101:0/3753809799 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f95fc0a11d0 0x7f95fc0a3680 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f960400b2e0 tx=0x7f9604005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:16.442 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.455+0000 7f96189fd700 1 -- 172.21.15.101:0/3753809799 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9608051790 con 0x7f9614100ce0 2023-07-18T03:07:16.644 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.656+0000 7f96199ff700 1 -- 172.21.15.101:0/3753809799 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9614061120 con 0x7f9614100ce0 2023-07-18T03:07:16.645 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.657+0000 7f96189fd700 1 -- 172.21.15.101:0/3753809799 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f96080961b0 con 0x7f9614100ce0 2023-07-18T03:07:16.645 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 32 2023-07-18T03:07:16.647 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.659+0000 7f96199ff700 1 -- 172.21.15.101:0/3753809799 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f95fc0a11d0 msgr2=0x7f95fc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:16.647 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.659+0000 7f96199ff700 1 --2- 172.21.15.101:0/3753809799 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f95fc0a11d0 0x7f95fc0a3680 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f960400b2e0 tx=0x7f9604005fb0 comp rx=0 tx=0).stop 2023-07-18T03:07:16.648 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.659+0000 7f96199ff700 1 -- 172.21.15.101:0/3753809799 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9614100ce0 msgr2=0x7f9614072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:16.648 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.659+0000 7f96199ff700 1 --2- 172.21.15.101:0/3753809799 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9614100ce0 0x7f9614072900 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f960800e9a0 tx=0x7f960800ecb0 comp rx=0 tx=0).stop 2023-07-18T03:07:16.648 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.659+0000 7f96199ff700 1 -- 172.21.15.101:0/3753809799 shutdown_connections 2023-07-18T03:07:16.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.659+0000 7f96199ff700 1 --2- 172.21.15.101:0/3753809799 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f95fc0a11d0 0x7f95fc0a3680 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:16.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.659+0000 7f96199ff700 1 --2- 172.21.15.101:0/3753809799 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96140ffaf0 0x7f96140742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:16.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.659+0000 7f96199ff700 1 --2- 172.21.15.101:0/3753809799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9614101670 0x7f9614072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:16.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.659+0000 7f96199ff700 1 --2- 172.21.15.101:0/3753809799 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9614100ce0 0x7f9614072900 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:16.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.659+0000 7f96199ff700 1 -- 172.21.15.101:0/3753809799 >> 172.21.15.101:0/3753809799 conn(0x7f96140fb130 msgr2=0x7f96141041f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:16.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.659+0000 7f96199ff700 1 -- 172.21.15.101:0/3753809799 shutdown_connections 2023-07-18T03:07:16.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.659+0000 7f96199ff700 1 -- 172.21.15.101:0/3753809799 wait complete. 2023-07-18T03:07:16.665 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:07:16.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.874+0000 7fcbf95ff700 1 -- 172.21.15.101:0/2355120842 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcbf4100cf0 msgr2=0x7fcbf4101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:16.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.874+0000 7fcbf95ff700 1 --2- 172.21.15.101:0/2355120842 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcbf4100cf0 0x7fcbf4101140 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8009990 tx=0x7fcbe8009ca0 comp rx=0 tx=0).stop 2023-07-18T03:07:16.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.874+0000 7fcbf95ff700 1 -- 172.21.15.101:0/2355120842 shutdown_connections 2023-07-18T03:07:16.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.874+0000 7fcbf95ff700 1 --2- 172.21.15.101:0/2355120842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbf4101680 0x7fcbf4107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:16.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.874+0000 7fcbf95ff700 1 --2- 172.21.15.101:0/2355120842 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbf40ffb00 0x7fcbf40fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:16.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.874+0000 7fcbf95ff700 1 --2- 172.21.15.101:0/2355120842 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcbf4100cf0 0x7fcbf4101140 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:16.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.874+0000 7fcbf95ff700 1 -- 172.21.15.101:0/2355120842 >> 172.21.15.101:0/2355120842 conn(0x7fcbf40fb0e0 msgr2=0x7fcbf40fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:16.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.874+0000 7fcbf95ff700 1 -- 172.21.15.101:0/2355120842 shutdown_connections 2023-07-18T03:07:16.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.874+0000 7fcbf95ff700 1 -- 172.21.15.101:0/2355120842 wait complete. 2023-07-18T03:07:16.864 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.875+0000 7fcbf95ff700 1 Processor -- start 2023-07-18T03:07:16.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.875+0000 7fcbf95ff700 1 -- start start 2023-07-18T03:07:16.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.875+0000 7fcbf95ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcbf40ffb00 0x7fcbf40742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:16.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.875+0000 7fcbf95ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbf4100cf0 0x7fcbf4072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:16.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.875+0000 7fcbf95ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbf4101680 0x7fcbf4072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:16.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.876+0000 7fcbf95ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf40735e0 con 0x7fcbf40ffb00 2023-07-18T03:07:16.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.876+0000 7fcbf95ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf4073750 con 0x7fcbf4100cf0 2023-07-18T03:07:16.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.876+0000 7fcbf95ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf419f890 con 0x7fcbf4101680 2023-07-18T03:07:16.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.876+0000 7fcbf2ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcbf40ffb00 0x7fcbf40742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:16.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.876+0000 7fcbf2ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcbf40ffb00 0x7fcbf40742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:36312/0 (socket says 172.21.15.101:36312) 2023-07-18T03:07:16.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.876+0000 7fcbf2ffd700 1 -- 172.21.15.101:0/2922364421 learned_addr learned my addr 172.21.15.101:0/2922364421 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:16.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.876+0000 7fcbf37fe700 1 --2- 172.21.15.101:0/2922364421 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbf4101680 0x7fcbf4072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:16.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.876+0000 7fcbf27fc700 1 --2- 172.21.15.101:0/2922364421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbf4100cf0 0x7fcbf4072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:16.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.876+0000 7fcbf2ffd700 1 -- 172.21.15.101:0/2922364421 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbf4101680 msgr2=0x7fcbf4072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:16.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.876+0000 7fcbf2ffd700 1 --2- 172.21.15.101:0/2922364421 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbf4101680 0x7fcbf4072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:16.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.876+0000 7fcbf2ffd700 1 -- 172.21.15.101:0/2922364421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbf4100cf0 msgr2=0x7fcbf4072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:16.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.876+0000 7fcbf2ffd700 1 --2- 172.21.15.101:0/2922364421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbf4100cf0 0x7fcbf4072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:16.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.876+0000 7fcbf2ffd700 1 -- 172.21.15.101:0/2922364421 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbec009c40 con 0x7fcbf40ffb00 2023-07-18T03:07:16.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.876+0000 7fcbf2ffd700 1 --2- 172.21.15.101:0/2922364421 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcbf40ffb00 0x7fcbf40742c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fcbe400e970 tx=0x7fcbe400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:16.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.877+0000 7fcbdbfff700 1 -- 172.21.15.101:0/2922364421 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe4009790 con 0x7fcbf40ffb00 2023-07-18T03:07:16.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.877+0000 7fcbf95ff700 1 -- 172.21.15.101:0/2922364421 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbe80095f0 con 0x7fcbf40ffb00 2023-07-18T03:07:16.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.877+0000 7fcbdbfff700 1 -- 172.21.15.101:0/2922364421 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcbe400c7c0 con 0x7fcbf40ffb00 2023-07-18T03:07:16.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.877+0000 7fcbdbfff700 1 -- 172.21.15.101:0/2922364421 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe4003c20 con 0x7fcbf40ffb00 2023-07-18T03:07:16.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.877+0000 7fcbf95ff700 1 -- 172.21.15.101:0/2922364421 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbf419fef0 con 0x7fcbf40ffb00 2023-07-18T03:07:16.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.878+0000 7fcbf95ff700 1 -- 172.21.15.101:0/2922364421 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbf4107070 con 0x7fcbf40ffb00 2023-07-18T03:07:16.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.880+0000 7fcbdbfff700 1 -- 172.21.15.101:0/2922364421 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fcbe400c930 con 0x7fcbf40ffb00 2023-07-18T03:07:16.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.881+0000 7fcbdbfff700 1 --2- 172.21.15.101:0/2922364421 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fcbdc0a11a0 0x7fcbdc0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:16.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.881+0000 7fcbdbfff700 1 -- 172.21.15.101:0/2922364421 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fcbe4014070 con 0x7fcbf40ffb00 2023-07-18T03:07:16.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.881+0000 7fcbf27fc700 1 --2- 172.21.15.101:0/2922364421 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fcbdc0a11a0 0x7fcbdc0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:16.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.882+0000 7fcbf27fc700 1 --2- 172.21.15.101:0/2922364421 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fcbdc0a11a0 0x7fcbdc0a3650 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fcbe800b2e0 tx=0x7fcbe8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:16.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:16.883+0000 7fcbdbfff700 1 -- 172.21.15.101:0/2922364421 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fcbe4096370 con 0x7fcbf40ffb00 2023-07-18T03:07:17.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.087+0000 7fcbf95ff700 1 -- 172.21.15.101:0/2922364421 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcbf402cac0 con 0x7fcbf40ffb00 2023-07-18T03:07:17.076 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.088+0000 7fcbdbfff700 1 -- 172.21.15.101:0/2922364421 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24725 (secure 0 0 0) 0x7fcbe401f340 con 0x7fcbf40ffb00 2023-07-18T03:07:17.076 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:07:17.077 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":91,"fsid":"b1782daa-2516-11ee-9b34-001a4aab830c","created":"2023-07-18T02:57:22.972483+0000","modified":"2023-07-18T03:06:24.250777+0000","last_up_change":"2023-07-18T03:04:11.429530+0000","last_in_change":"2023-07-18T03:03:54.158340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:00:43.353489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:04:57.771616+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-18T03:04:58.769590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-18T03:05:02.358830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc76b5d1-38a0-4f1a-9335-756bfc40a62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6803","nonce":3610719929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6805","nonce":3610719929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6809","nonce":3610719929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6807","nonce":3610719929}]},"public_addr":"172.21.15.101:6803/3610719929","cluster_addr":"172.21.15.101:6805/3610719929","heartbeat_back_addr":"172.21.15.101:6809/3610719929","heartbeat_front_addr":"172.21.15.101:6807/3610719929","state":["exists","up"]},{"osd":1,"uuid":"2a299573-0b34-4af9-aaae-b523e5058ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6801","nonce":1030229579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6803","nonce":1030229579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6807","nonce":1030229579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6805","nonce":1030229579}]},"public_addr":"172.21.15.155:6801/1030229579","cluster_addr":"172.21.15.155:6803/1030229579","heartbeat_back_addr":"172.21.15.155:6807/1030229579","heartbeat_front_addr":"172.21.15.155:6805/1030229579","state":["exists","up"]},{"osd":2,"uuid":"123b3749-c636-48fa-9e33-6229b5575dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6801","nonce":1349487507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6803","nonce":1349487507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6807","nonce":1349487507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6805","nonce":1349487507}]},"public_addr":"172.21.15.179:6801/1349487507","cluster_addr":"172.21.15.179:6803/1349487507","heartbeat_back_addr":"172.21.15.179:6807/1349487507","heartbeat_front_addr":"172.21.15.179:6805/1349487507","state":["exists","up"]},{"osd":3,"uuid":"4b0c409d-2954-4127-833a-b196bf226d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6811","nonce":4226577463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6813","nonce":4226577463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6817","nonce":4226577463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6815","nonce":4226577463}]},"public_addr":"172.21.15.101:6811/4226577463","cluster_addr":"172.21.15.101:6813/4226577463","heartbeat_back_addr":"172.21.15.101:6817/4226577463","heartbeat_front_addr":"172.21.15.101:6815/4226577463","state":["exists","up"]},{"osd":4,"uuid":"2137f2df-984e-45ce-baf6-b4ac12c7e3c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6809","nonce":1951135805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6811","nonce":1951135805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6815","nonce":1951135805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6813","nonce":1951135805}]},"public_addr":"172.21.15.155:6809/1951135805","cluster_addr":"172.21.15.155:6811/1951135805","heartbeat_back_addr":"172.21.15.155:6815/1951135805","heartbeat_front_addr":"172.21.15.155:6813/1951135805","state":["exists","up"]},{"osd":5,"uuid":"1020bfc5-8c2f-482e-9588-66c7f98e46e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6809","nonce":2746037021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6811","nonce":2746037021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6815","nonce":2746037021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6813","nonce":2746037021}]},"public_addr":"172.21.15.179:6809/2746037021","cluster_addr":"172.21.15.179:6811/2746037021","heartbeat_back_addr":"172.21.15.179:6815/2746037021","heartbeat_front_addr":"172.21.15.179:6813/2746037021","state":["exists","up"]},{"osd":6,"uuid":"2cd99260-5c36-4603-9338-cf697970ce58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6819","nonce":958044931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6821","nonce":958044931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6825","nonce":958044931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6823","nonce":958044931}]},"public_addr":"172.21.15.101:6819/958044931","cluster_addr":"172.21.15.101:6821/958044931","heartbeat_back_addr":"172.21.15.101:6825/958044931","heartbeat_front_addr":"172.21.15.101:6823/958044931","state":["exists","up"]},{"osd":7,"uuid":"ed23fd0a-678f-4e15-b42c-cfe873315714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6817","nonce":3256322291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6819","nonce":3256322291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6823","nonce":3256322291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6821","nonce":3256322291}]},"public_addr":"172.21.15.155:6817/3256322291","cluster_addr":"172.21.15.155:6819/3256322291","heartbeat_back_addr":"172.21.15.155:6823/3256322291","heartbeat_front_addr":"172.21.15.155:6821/3256322291","state":["exists","up"]},{"osd":8,"uuid":"eca3d8e6-7baf-47a5-b991-328001c7ebef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6817","nonce":3520612559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6819","nonce":3520612559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6823","nonce":3520612559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6821","nonce":3520612559}]},"public_addr":"172.21.15.179:6817/3520612559","cluster_addr":"172.21.15.179:6819/3520612559","heartbeat_back_addr":"172.21.15.179:6823/3520612559","heartbeat_front_addr":"172.21.15.179:6821/3520612559","state":["exists","up"]},{"osd":9,"uuid":"465a7ffb-67e4-49ef-812c-1b96feacfebf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6827","nonce":2166787667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6829","nonce":2166787667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6833","nonce":2166787667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6831","nonce":2166787667}]},"public_addr":"172.21.15.101:6827/2166787667","cluster_addr":"172.21.15.101:6829/2166787667","heartbeat_back_addr":"172.21.15.101:6833/2166787667","heartbeat_front_addr":"172.21.15.101:6831/2166787667","state":["exists","up"]},{"osd":10,"uuid":"1897a027-f25d-434f-8df5-5fc2bd866578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6825","nonce":3682446349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6827","nonce":3682446349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6831","nonce":3682446349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6829","nonce":3682446349}]},"public_addr":"172.21.15.155:6825/3682446349","cluster_addr":"172.21.15.155:6827/3682446349","heartbeat_back_addr":"172.21.15.155:6831/3682446349","heartbeat_front_addr":"172.21.15.155:6829/3682446349","state":["exists","up"]},{"osd":11,"uuid":"b62effaf-45e2-4585-b8d0-db448f3474ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6825","nonce":2602640858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6827","nonce":2602640858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6831","nonce":2602640858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6829","nonce":2602640858}]},"public_addr":"172.21.15.179:6825/2602640858","cluster_addr":"172.21.15.179:6827/2602640858","heartbeat_back_addr":"172.21.15.179:6831/2602640858","heartbeat_front_addr":"172.21.15.179:6829/2602640858","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T02:59:56.074519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:19.358142+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:41.128966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:08.227054+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:31.442592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:52.751685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:19.251054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:40.698241+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:01.339150+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:27.955910+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:49.136980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:04:10.084982+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.101:6800/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:6801/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/604228291":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/4055187849":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/2844574324":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/1603248215":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6801/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6800/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/3307578182":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/373189792":"2023-07-19T02:57:56.243200+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:07:17.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.091+0000 7fcbf95ff700 1 -- 172.21.15.101:0/2922364421 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fcbdc0a11a0 msgr2=0x7fcbdc0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:17.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.091+0000 7fcbf95ff700 1 --2- 172.21.15.101:0/2922364421 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fcbdc0a11a0 0x7fcbdc0a3650 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fcbe800b2e0 tx=0x7fcbe8005fb0 comp rx=0 tx=0).stop 2023-07-18T03:07:17.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.091+0000 7fcbf95ff700 1 -- 172.21.15.101:0/2922364421 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcbf40ffb00 msgr2=0x7fcbf40742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:17.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.091+0000 7fcbf95ff700 1 --2- 172.21.15.101:0/2922364421 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcbf40ffb00 0x7fcbf40742c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fcbe400e970 tx=0x7fcbe400ec80 comp rx=0 tx=0).stop 2023-07-18T03:07:17.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.091+0000 7fcbf95ff700 1 -- 172.21.15.101:0/2922364421 shutdown_connections 2023-07-18T03:07:17.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.091+0000 7fcbf95ff700 1 --2- 172.21.15.101:0/2922364421 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fcbdc0a11a0 0x7fcbdc0a3650 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:17.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.091+0000 7fcbf95ff700 1 --2- 172.21.15.101:0/2922364421 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbf4101680 0x7fcbf4072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:17.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.091+0000 7fcbf95ff700 1 --2- 172.21.15.101:0/2922364421 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcbf40ffb00 0x7fcbf40742c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:17.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.091+0000 7fcbf95ff700 1 --2- 172.21.15.101:0/2922364421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbf4100cf0 0x7fcbf4072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:17.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.091+0000 7fcbf95ff700 1 -- 172.21.15.101:0/2922364421 >> 172.21.15.101:0/2922364421 conn(0x7fcbf40fb0e0 msgr2=0x7fcbf41041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:17.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.091+0000 7fcbf95ff700 1 -- 172.21.15.101:0/2922364421 shutdown_connections 2023-07-18T03:07:17.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.091+0000 7fcbf95ff700 1 -- 172.21.15.101:0/2922364421 wait complete. 2023-07-18T03:07:17.097 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:00:43.353489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-18T03:04:57.771616+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-18T03:04:58.769590+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-18T03:05:02.358830+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-18T03:07:17.098 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-18T03:07:17.194 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:16 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1189888827' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:07:17.194 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:16 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/3753809799' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-18T03:07:17.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:16 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1189888827' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:07:17.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:16 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3753809799' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-18T03:07:17.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:16 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1189888827' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:07:17.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:16 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/3753809799' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-18T03:07:17.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.310+0000 7fe201bff700 1 -- 172.21.15.101:0/1818400310 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe1fc101670 msgr2=0x7fe1fc107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:17.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.310+0000 7fe201bff700 1 --2- 172.21.15.101:0/1818400310 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe1fc101670 0x7fe1fc107f30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe1f400b0a0 tx=0x7fe1f400b3b0 comp rx=0 tx=0).stop 2023-07-18T03:07:17.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.310+0000 7fe201bff700 1 -- 172.21.15.101:0/1818400310 shutdown_connections 2023-07-18T03:07:17.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.310+0000 7fe201bff700 1 --2- 172.21.15.101:0/1818400310 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1fc0ffaf0 0x7fe1fc0fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:17.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.310+0000 7fe201bff700 1 --2- 172.21.15.101:0/1818400310 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe1fc101670 0x7fe1fc107f30 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:17.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.310+0000 7fe201bff700 1 --2- 172.21.15.101:0/1818400310 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1fc100ce0 0x7fe1fc101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:17.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.310+0000 7fe201bff700 1 -- 172.21.15.101:0/1818400310 >> 172.21.15.101:0/1818400310 conn(0x7fe1fc0fb130 msgr2=0x7fe1fc0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:17.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.310+0000 7fe201bff700 1 -- 172.21.15.101:0/1818400310 shutdown_connections 2023-07-18T03:07:17.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.310+0000 7fe201bff700 1 -- 172.21.15.101:0/1818400310 wait complete. 2023-07-18T03:07:17.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.312+0000 7fe201bff700 1 Processor -- start 2023-07-18T03:07:17.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.312+0000 7fe201bff700 1 -- start start 2023-07-18T03:07:17.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.312+0000 7fe201bff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1fc0ffaf0 0x7fe1fc0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:17.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.313+0000 7fe201bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe1fc100ce0 0x7fe1fc072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:17.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.313+0000 7fe201bff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1fc101670 0x7fe1fc072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:17.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.313+0000 7fe201bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1fc0735e0 con 0x7fe1fc100ce0 2023-07-18T03:07:17.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.313+0000 7fe201bff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1fc073750 con 0x7fe1fc101670 2023-07-18T03:07:17.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.313+0000 7fe201bff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1fc19f9c0 con 0x7fe1fc0ffaf0 2023-07-18T03:07:17.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.313+0000 7fe1faffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe1fc100ce0 0x7fe1fc072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:17.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.313+0000 7fe1faffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe1fc100ce0 0x7fe1fc072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:36340/0 (socket says 172.21.15.101:36340) 2023-07-18T03:07:17.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.313+0000 7fe1faffd700 1 -- 172.21.15.101:0/3973451071 learned_addr learned my addr 172.21.15.101:0/3973451071 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:17.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.313+0000 7fe1fb7fe700 1 --2- 172.21.15.101:0/3973451071 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1fc0ffaf0 0x7fe1fc0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:17.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.313+0000 7fe1fbfff700 1 --2- 172.21.15.101:0/3973451071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1fc101670 0x7fe1fc072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:17.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.313+0000 7fe1faffd700 1 -- 172.21.15.101:0/3973451071 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1fc0ffaf0 msgr2=0x7fe1fc0742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:17.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.313+0000 7fe1faffd700 1 --2- 172.21.15.101:0/3973451071 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1fc0ffaf0 0x7fe1fc0742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:17.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.313+0000 7fe1faffd700 1 -- 172.21.15.101:0/3973451071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1fc101670 msgr2=0x7fe1fc072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:17.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.313+0000 7fe1faffd700 1 --2- 172.21.15.101:0/3973451071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1fc101670 0x7fe1fc072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:17.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.313+0000 7fe1faffd700 1 -- 172.21.15.101:0/3973451071 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f4009d00 con 0x7fe1fc100ce0 2023-07-18T03:07:17.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.313+0000 7fe1faffd700 1 --2- 172.21.15.101:0/3973451071 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe1fc100ce0 0x7fe1fc072900 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe1f000e9a0 tx=0x7fe1f000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:17.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.314+0000 7fe1f8ff9700 1 -- 172.21.15.101:0/3973451071 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0009790 con 0x7fe1fc100ce0 2023-07-18T03:07:17.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.314+0000 7fe1f8ff9700 1 -- 172.21.15.101:0/3973451071 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe1f0010430 con 0x7fe1fc100ce0 2023-07-18T03:07:17.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.314+0000 7fe201bff700 1 -- 172.21.15.101:0/3973451071 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1fc19fca0 con 0x7fe1fc100ce0 2023-07-18T03:07:17.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.314+0000 7fe1f8ff9700 1 -- 172.21.15.101:0/3973451071 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0018680 con 0x7fe1fc100ce0 2023-07-18T03:07:17.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.314+0000 7fe201bff700 1 -- 172.21.15.101:0/3973451071 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1fc1a0180 con 0x7fe1fc100ce0 2023-07-18T03:07:17.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.315+0000 7fe201bff700 1 -- 172.21.15.101:0/3973451071 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1fc106980 con 0x7fe1fc100ce0 2023-07-18T03:07:17.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.317+0000 7fe1f8ff9700 1 -- 172.21.15.101:0/3973451071 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fe1f00105a0 con 0x7fe1fc100ce0 2023-07-18T03:07:17.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.319+0000 7fe1f8ff9700 1 --2- 172.21.15.101:0/3973451071 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe1e40a1150 0x7fe1e40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:17.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.319+0000 7fe1f8ff9700 1 -- 172.21.15.101:0/3973451071 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fe1f0014070 con 0x7fe1fc100ce0 2023-07-18T03:07:17.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.319+0000 7fe1fb7fe700 1 --2- 172.21.15.101:0/3973451071 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe1e40a1150 0x7fe1e40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:17.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.320+0000 7fe1fb7fe700 1 --2- 172.21.15.101:0/3973451071 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe1e40a1150 0x7fe1e40a3600 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc100b40 tx=0x7fe1ec009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:17.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.323+0000 7fe1f8ff9700 1 -- 172.21.15.101:0/3973451071 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe1f0051790 con 0x7fe1fc100ce0 2023-07-18T03:07:17.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.519+0000 7fe201bff700 1 -- 172.21.15.101:0/3973451071 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe1fc0611f0 con 0x7fe1fc100ce0 2023-07-18T03:07:17.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.519+0000 7fe1f8ff9700 1 -- 172.21.15.101:0/3973451071 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe1f0096210 con 0x7fe1fc100ce0 2023-07-18T03:07:17.507 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 1 2023-07-18T03:07:17.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.522+0000 7fe201bff700 1 -- 172.21.15.101:0/3973451071 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe1e40a1150 msgr2=0x7fe1e40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:17.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.522+0000 7fe201bff700 1 --2- 172.21.15.101:0/3973451071 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe1e40a1150 0x7fe1e40a3600 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc100b40 tx=0x7fe1ec009260 comp rx=0 tx=0).stop 2023-07-18T03:07:17.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.522+0000 7fe201bff700 1 -- 172.21.15.101:0/3973451071 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe1fc100ce0 msgr2=0x7fe1fc072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:17.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.522+0000 7fe201bff700 1 --2- 172.21.15.101:0/3973451071 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe1fc100ce0 0x7fe1fc072900 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe1f000e9a0 tx=0x7fe1f000ecb0 comp rx=0 tx=0).stop 2023-07-18T03:07:17.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.522+0000 7fe201bff700 1 -- 172.21.15.101:0/3973451071 shutdown_connections 2023-07-18T03:07:17.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.522+0000 7fe201bff700 1 --2- 172.21.15.101:0/3973451071 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fe1e40a1150 0x7fe1e40a3600 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:17.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.522+0000 7fe201bff700 1 --2- 172.21.15.101:0/3973451071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1fc101670 0x7fe1fc072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:17.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.522+0000 7fe201bff700 1 --2- 172.21.15.101:0/3973451071 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1fc0ffaf0 0x7fe1fc0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:17.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.522+0000 7fe201bff700 1 --2- 172.21.15.101:0/3973451071 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe1fc100ce0 0x7fe1fc072900 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:17.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.522+0000 7fe201bff700 1 -- 172.21.15.101:0/3973451071 >> 172.21.15.101:0/3973451071 conn(0x7fe1fc0fb130 msgr2=0x7fe1fc0fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:17.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.522+0000 7fe201bff700 1 -- 172.21.15.101:0/3973451071 shutdown_connections 2023-07-18T03:07:17.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.522+0000 7fe201bff700 1 -- 172.21.15.101:0/3973451071 wait complete. 2023-07-18T03:07:17.528 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-18T03:07:17.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.739+0000 7f20433ff700 1 -- 172.21.15.101:0/3797790016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f203c100cd0 msgr2=0x7f203c101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:17.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.739+0000 7f20433ff700 1 --2- 172.21.15.101:0/3797790016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f203c100cd0 0x7f203c101120 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f202c009990 tx=0x7f202c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:07:17.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.739+0000 7f20433ff700 1 -- 172.21.15.101:0/3797790016 shutdown_connections 2023-07-18T03:07:17.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.739+0000 7f20433ff700 1 --2- 172.21.15.101:0/3797790016 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f203c101660 0x7f203c107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:17.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.739+0000 7f20433ff700 1 --2- 172.21.15.101:0/3797790016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f203c0ffae0 0x7f203c0ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:17.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.739+0000 7f20433ff700 1 --2- 172.21.15.101:0/3797790016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f203c100cd0 0x7f203c101120 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:17.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.739+0000 7f20433ff700 1 -- 172.21.15.101:0/3797790016 >> 172.21.15.101:0/3797790016 conn(0x7f203c0fb0e0 msgr2=0x7f203c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:17.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.739+0000 7f20433ff700 1 -- 172.21.15.101:0/3797790016 shutdown_connections 2023-07-18T03:07:17.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.739+0000 7f20433ff700 1 -- 172.21.15.101:0/3797790016 wait complete. 2023-07-18T03:07:17.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.740+0000 7f20433ff700 1 Processor -- start 2023-07-18T03:07:17.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.741+0000 7f20433ff700 1 -- start start 2023-07-18T03:07:17.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.741+0000 7f20433ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f203c0ffae0 0x7f203c0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:17.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.741+0000 7f20433ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f203c100cd0 0x7f203c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:17.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.741+0000 7f20433ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f203c101660 0x7f203c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:17.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.741+0000 7f20433ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203c0735e0 con 0x7f203c100cd0 2023-07-18T03:07:17.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.741+0000 7f20433ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203c073750 con 0x7f203c101660 2023-07-18T03:07:17.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.741+0000 7f20433ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203c19f8e0 con 0x7f203c0ffae0 2023-07-18T03:07:17.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.741+0000 7f204099a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f203c100cd0 0x7f203c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:17.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.741+0000 7f204099a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f203c100cd0 0x7f203c072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:36372/0 (socket says 172.21.15.101:36372) 2023-07-18T03:07:17.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.741+0000 7f204099a700 1 -- 172.21.15.101:0/129402822 learned_addr learned my addr 172.21.15.101:0/129402822 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:17.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.741+0000 7f204199c700 1 --2- 172.21.15.101:0/129402822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f203c101660 0x7f203c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:17.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.741+0000 7f204119b700 1 --2- 172.21.15.101:0/129402822 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f203c0ffae0 0x7f203c0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:17.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.741+0000 7f204099a700 1 -- 172.21.15.101:0/129402822 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f203c0ffae0 msgr2=0x7f203c0742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:17.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.741+0000 7f204099a700 1 --2- 172.21.15.101:0/129402822 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f203c0ffae0 0x7f203c0742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:17.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.741+0000 7f204099a700 1 -- 172.21.15.101:0/129402822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f203c101660 msgr2=0x7f203c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:17.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.741+0000 7f204099a700 1 --2- 172.21.15.101:0/129402822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f203c101660 0x7f203c072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:17.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.742+0000 7f204099a700 1 -- 172.21.15.101:0/129402822 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2038009520 con 0x7f203c100cd0 2023-07-18T03:07:17.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.742+0000 7f204199c700 1 --2- 172.21.15.101:0/129402822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f203c101660 0x7f203c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:07:17.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.742+0000 7f204099a700 1 --2- 172.21.15.101:0/129402822 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f203c100cd0 0x7f203c072900 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f202c004760 tx=0x7f202c004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:17.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.742+0000 7f20327fc700 1 -- 172.21.15.101:0/129402822 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c01d070 con 0x7f203c100cd0 2023-07-18T03:07:17.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.742+0000 7f20327fc700 1 -- 172.21.15.101:0/129402822 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f202c004830 con 0x7f203c100cd0 2023-07-18T03:07:17.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.742+0000 7f20433ff700 1 -- 172.21.15.101:0/129402822 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f202c0095f0 con 0x7f203c100cd0 2023-07-18T03:07:17.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.742+0000 7f20327fc700 1 -- 172.21.15.101:0/129402822 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c005350 con 0x7f203c100cd0 2023-07-18T03:07:17.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.742+0000 7f20433ff700 1 -- 172.21.15.101:0/129402822 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f203c19fd90 con 0x7f203c100cd0 2023-07-18T03:07:17.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.743+0000 7f20433ff700 1 -- 172.21.15.101:0/129402822 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f203c106a30 con 0x7f203c100cd0 2023-07-18T03:07:17.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.745+0000 7f20327fc700 1 -- 172.21.15.101:0/129402822 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f202c005510 con 0x7f203c100cd0 2023-07-18T03:07:17.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.746+0000 7f20327fc700 1 --2- 172.21.15.101:0/129402822 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f20280a1150 0x7f20280a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:17.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.746+0000 7f20327fc700 1 -- 172.21.15.101:0/129402822 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f202c0cde80 con 0x7f203c100cd0 2023-07-18T03:07:17.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.747+0000 7f204119b700 1 --2- 172.21.15.101:0/129402822 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f20280a1150 0x7f20280a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:17.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.747+0000 7f204119b700 1 --2- 172.21.15.101:0/129402822 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f20280a1150 0x7f20280a3600 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f203c100b30 tx=0x7f2038009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:17.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.750+0000 7f20327fc700 1 -- 172.21.15.101:0/129402822 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f202c05a790 con 0x7f203c100cd0 2023-07-18T03:07:17.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.951+0000 7f20433ff700 1 -- 172.21.15.101:0/129402822 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f203c10b300 con 0x7f203c100cd0 2023-07-18T03:07:17.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.952+0000 7f20327fc700 1 -- 172.21.15.101:0/129402822 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f202c097290 con 0x7f203c100cd0 2023-07-18T03:07:17.940 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 32 2023-07-18T03:07:17.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.954+0000 7f20433ff700 1 -- 172.21.15.101:0/129402822 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f20280a1150 msgr2=0x7f20280a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:17.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.954+0000 7f20433ff700 1 --2- 172.21.15.101:0/129402822 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f20280a1150 0x7f20280a3600 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f203c100b30 tx=0x7f2038009260 comp rx=0 tx=0).stop 2023-07-18T03:07:17.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.954+0000 7f20433ff700 1 -- 172.21.15.101:0/129402822 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f203c100cd0 msgr2=0x7f203c072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:17.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.954+0000 7f20433ff700 1 --2- 172.21.15.101:0/129402822 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f203c100cd0 0x7f203c072900 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f202c004760 tx=0x7f202c004790 comp rx=0 tx=0).stop 2023-07-18T03:07:17.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.954+0000 7f20433ff700 1 -- 172.21.15.101:0/129402822 shutdown_connections 2023-07-18T03:07:17.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.954+0000 7f20433ff700 1 --2- 172.21.15.101:0/129402822 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f20280a1150 0x7f20280a3600 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:17.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.954+0000 7f20433ff700 1 --2- 172.21.15.101:0/129402822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f203c101660 0x7f203c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:17.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.954+0000 7f20433ff700 1 --2- 172.21.15.101:0/129402822 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f203c0ffae0 0x7f203c0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:17.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.954+0000 7f20433ff700 1 --2- 172.21.15.101:0/129402822 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f203c100cd0 0x7f203c072900 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:17.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.954+0000 7f20433ff700 1 -- 172.21.15.101:0/129402822 >> 172.21.15.101:0/129402822 conn(0x7f203c0fb0e0 msgr2=0x7f203c0fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:17.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.954+0000 7f20433ff700 1 -- 172.21.15.101:0/129402822 shutdown_connections 2023-07-18T03:07:17.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:17.955+0000 7f20433ff700 1 -- 172.21.15.101:0/129402822 wait complete. 2023-07-18T03:07:17.961 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-18T03:07:18.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.173+0000 7fda8d9ff700 1 -- 172.21.15.101:0/2344330871 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda880ffaf0 msgr2=0x7fda880fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:18.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.173+0000 7fda8d9ff700 1 --2- 172.21.15.101:0/2344330871 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda880ffaf0 0x7fda880fff00 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fda78009990 tx=0x7fda78009ca0 comp rx=0 tx=0).stop 2023-07-18T03:07:18.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.173+0000 7fda8d9ff700 1 -- 172.21.15.101:0/2344330871 shutdown_connections 2023-07-18T03:07:18.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.173+0000 7fda8d9ff700 1 --2- 172.21.15.101:0/2344330871 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda88101670 0x7fda88107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:18.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.173+0000 7fda8d9ff700 1 --2- 172.21.15.101:0/2344330871 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda880ffaf0 0x7fda880fff00 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:18.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.173+0000 7fda8d9ff700 1 --2- 172.21.15.101:0/2344330871 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda88100ce0 0x7fda88101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:18.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.173+0000 7fda8d9ff700 1 -- 172.21.15.101:0/2344330871 >> 172.21.15.101:0/2344330871 conn(0x7fda880fb130 msgr2=0x7fda880fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:18.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.173+0000 7fda8d9ff700 1 -- 172.21.15.101:0/2344330871 shutdown_connections 2023-07-18T03:07:18.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.173+0000 7fda8d9ff700 1 -- 172.21.15.101:0/2344330871 wait complete. 2023-07-18T03:07:18.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.174+0000 7fda8d9ff700 1 Processor -- start 2023-07-18T03:07:18.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.174+0000 7fda8d9ff700 1 -- start start 2023-07-18T03:07:18.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.174+0000 7fda8d9ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda880ffaf0 0x7fda880742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:18.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.174+0000 7fda8d9ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda88100ce0 0x7fda88072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:18.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.174+0000 7fda8d9ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda88101670 0x7fda88072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:18.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.174+0000 7fda8d9ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda880735e0 con 0x7fda88101670 2023-07-18T03:07:18.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.174+0000 7fda8d9ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda88073750 con 0x7fda880ffaf0 2023-07-18T03:07:18.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.174+0000 7fda8d9ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda8819f930 con 0x7fda88100ce0 2023-07-18T03:07:18.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.175+0000 7fda877fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda88101670 0x7fda88072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:18.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.175+0000 7fda877fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda88101670 0x7fda88072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:36392/0 (socket says 172.21.15.101:36392) 2023-07-18T03:07:18.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.175+0000 7fda877fe700 1 -- 172.21.15.101:0/3296910028 learned_addr learned my addr 172.21.15.101:0/3296910028 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:18.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.175+0000 7fda86ffd700 1 --2- 172.21.15.101:0/3296910028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda880ffaf0 0x7fda880742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:18.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.175+0000 7fda867fc700 1 --2- 172.21.15.101:0/3296910028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda88100ce0 0x7fda88072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:18.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.175+0000 7fda877fe700 1 -- 172.21.15.101:0/3296910028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda88100ce0 msgr2=0x7fda88072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:18.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.175+0000 7fda877fe700 1 --2- 172.21.15.101:0/3296910028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda88100ce0 0x7fda88072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:18.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.175+0000 7fda877fe700 1 -- 172.21.15.101:0/3296910028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda880ffaf0 msgr2=0x7fda880742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:18.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.175+0000 7fda877fe700 1 --2- 172.21.15.101:0/3296910028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda880ffaf0 0x7fda880742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:18.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.175+0000 7fda877fe700 1 -- 172.21.15.101:0/3296910028 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda780095f0 con 0x7fda88101670 2023-07-18T03:07:18.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.175+0000 7fda877fe700 1 --2- 172.21.15.101:0/3296910028 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda88101670 0x7fda88072e40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fda80007ee0 tx=0x7fda8000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:18.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.175+0000 7fda8c9fd700 1 -- 172.21.15.101:0/3296910028 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda80017070 con 0x7fda88101670 2023-07-18T03:07:18.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.176+0000 7fda8c9fd700 1 -- 172.21.15.101:0/3296910028 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda8000f040 con 0x7fda88101670 2023-07-18T03:07:18.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.176+0000 7fda8d9ff700 1 -- 172.21.15.101:0/3296910028 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda8819fc10 con 0x7fda88101670 2023-07-18T03:07:18.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.176+0000 7fda8c9fd700 1 -- 172.21.15.101:0/3296910028 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda80005580 con 0x7fda88101670 2023-07-18T03:07:18.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.176+0000 7fda8d9ff700 1 -- 172.21.15.101:0/3296910028 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda881a0120 con 0x7fda88101670 2023-07-18T03:07:18.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.177+0000 7fda8d9ff700 1 -- 172.21.15.101:0/3296910028 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda88107080 con 0x7fda88101670 2023-07-18T03:07:18.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.179+0000 7fda8c9fd700 1 -- 172.21.15.101:0/3296910028 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fda8000db20 con 0x7fda88101670 2023-07-18T03:07:18.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.180+0000 7fda8c9fd700 1 --2- 172.21.15.101:0/3296910028 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fda700a1150 0x7fda700a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:18.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.180+0000 7fda8c9fd700 1 -- 172.21.15.101:0/3296910028 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fda800cc970 con 0x7fda88101670 2023-07-18T03:07:18.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.180+0000 7fda86ffd700 1 --2- 172.21.15.101:0/3296910028 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fda700a1150 0x7fda700a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:18.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.181+0000 7fda86ffd700 1 --2- 172.21.15.101:0/3296910028 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fda700a1150 0x7fda700a3600 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fda7800b2e0 tx=0x7fda78005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:18.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.182+0000 7fda8c9fd700 1 -- 172.21.15.101:0/3296910028 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fda80095cf0 con 0x7fda88101670 2023-07-18T03:07:18.194 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:17 smithi179 ceph-mon[129229]: pgmap v348: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2023-07-18T03:07:18.194 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:17 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/2922364421' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:07:18.194 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:17 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/3973451071' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:07:18.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:17 smithi101 ceph-mon[126379]: pgmap v348: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2023-07-18T03:07:18.232 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:17 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2922364421' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:07:18.232 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:17 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3973451071' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:07:18.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:17 smithi155 ceph-mon[128885]: pgmap v348: 129 pgs: 129 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2023-07-18T03:07:18.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:17 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/2922364421' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:07:18.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:17 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/3973451071' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:07:18.370 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.383+0000 7fda8d9ff700 1 -- 172.21.15.101:0/3296910028 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fda8802caa0 con 0x7fda88101670 2023-07-18T03:07:18.371 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.383+0000 7fda8c9fd700 1 -- 172.21.15.101:0/3296910028 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7fda80099310 con 0x7fda88101670 2023-07-18T03:07:18.371 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 64 2023-07-18T03:07:18.373 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.386+0000 7fda8d9ff700 1 -- 172.21.15.101:0/3296910028 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fda700a1150 msgr2=0x7fda700a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:18.374 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.386+0000 7fda8d9ff700 1 --2- 172.21.15.101:0/3296910028 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fda700a1150 0x7fda700a3600 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fda7800b2e0 tx=0x7fda78005fb0 comp rx=0 tx=0).stop 2023-07-18T03:07:18.374 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.386+0000 7fda8d9ff700 1 -- 172.21.15.101:0/3296910028 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda88101670 msgr2=0x7fda88072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:18.374 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.386+0000 7fda8d9ff700 1 --2- 172.21.15.101:0/3296910028 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda88101670 0x7fda88072e40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fda80007ee0 tx=0x7fda8000d3b0 comp rx=0 tx=0).stop 2023-07-18T03:07:18.374 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.386+0000 7fda8d9ff700 1 -- 172.21.15.101:0/3296910028 shutdown_connections 2023-07-18T03:07:18.374 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.386+0000 7fda8d9ff700 1 --2- 172.21.15.101:0/3296910028 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fda700a1150 0x7fda700a3600 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:18.374 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.386+0000 7fda8d9ff700 1 --2- 172.21.15.101:0/3296910028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda880ffaf0 0x7fda880742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:18.375 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.386+0000 7fda8d9ff700 1 --2- 172.21.15.101:0/3296910028 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda88101670 0x7fda88072e40 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:18.375 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.386+0000 7fda8d9ff700 1 --2- 172.21.15.101:0/3296910028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda88100ce0 0x7fda88072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:18.375 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.386+0000 7fda8d9ff700 1 -- 172.21.15.101:0/3296910028 >> 172.21.15.101:0/3296910028 conn(0x7fda880fb130 msgr2=0x7fda881041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:18.375 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.386+0000 7fda8d9ff700 1 -- 172.21.15.101:0/3296910028 shutdown_connections 2023-07-18T03:07:18.375 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.386+0000 7fda8d9ff700 1 -- 172.21.15.101:0/3296910028 wait complete. 2023-07-18T03:07:18.392 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-18T03:07:18.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.605+0000 7f2303fff700 1 -- 172.21.15.101:0/2068882161 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f22fc072f70 msgr2=0x7f22fc0733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:18.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.605+0000 7f2303fff700 1 --2- 172.21.15.101:0/2068882161 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f22fc072f70 0x7f22fc0733e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f22ec009990 tx=0x7f22ec009ca0 comp rx=0 tx=0).stop 2023-07-18T03:07:18.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.605+0000 7f2303fff700 1 -- 172.21.15.101:0/2068882161 shutdown_connections 2023-07-18T03:07:18.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.605+0000 7f2303fff700 1 --2- 172.21.15.101:0/2068882161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22fc1079b0 0x7f22fc109e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:18.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.605+0000 7f2303fff700 1 --2- 172.21.15.101:0/2068882161 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22fc074550 0x7f22fc0729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:18.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.605+0000 7f2303fff700 1 --2- 172.21.15.101:0/2068882161 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f22fc072f70 0x7f22fc0733e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:18.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.605+0000 7f2303fff700 1 -- 172.21.15.101:0/2068882161 >> 172.21.15.101:0/2068882161 conn(0x7f22fc0fb130 msgr2=0x7f22fc0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:18.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.605+0000 7f2303fff700 1 -- 172.21.15.101:0/2068882161 shutdown_connections 2023-07-18T03:07:18.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.605+0000 7f2303fff700 1 -- 172.21.15.101:0/2068882161 wait complete. 2023-07-18T03:07:18.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.606+0000 7f2303fff700 1 Processor -- start 2023-07-18T03:07:18.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.606+0000 7f2303fff700 1 -- start start 2023-07-18T03:07:18.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.607+0000 7f2303fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22fc072f70 0x7f22fc197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:18.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.607+0000 7f2303fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22fc074550 0x7f22fc198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:18.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.607+0000 7f2303fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f22fc1079b0 0x7f22fc19f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:18.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.607+0000 7f2303fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22fc19f650 con 0x7f22fc1079b0 2023-07-18T03:07:18.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.607+0000 7f2303fff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22fc19f790 con 0x7f22fc074550 2023-07-18T03:07:18.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.607+0000 7f2303fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22fc19f920 con 0x7f22fc072f70 2023-07-18T03:07:18.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.607+0000 7f2301d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22fc072f70 0x7f22fc197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:18.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.607+0000 7f230159a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22fc074550 0x7f22fc198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:18.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.607+0000 7f230159a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22fc074550 0x7f22fc198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60288/0 (socket says 172.21.15.101:60288) 2023-07-18T03:07:18.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.607+0000 7f230159a700 1 -- 172.21.15.101:0/1099257748 learned_addr learned my addr 172.21.15.101:0/1099257748 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:18.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.607+0000 7f2301d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22fc072f70 0x7f22fc197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:48154/0 (socket says 172.21.15.101:48154) 2023-07-18T03:07:18.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.607+0000 7f230259c700 1 --2- 172.21.15.101:0/1099257748 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f22fc1079b0 0x7f22fc19f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:18.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.607+0000 7f230159a700 1 -- 172.21.15.101:0/1099257748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22fc072f70 msgr2=0x7f22fc197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:18.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.607+0000 7f230159a700 1 --2- 172.21.15.101:0/1099257748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22fc072f70 0x7f22fc197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:18.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.607+0000 7f230159a700 1 -- 172.21.15.101:0/1099257748 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f22fc1079b0 msgr2=0x7f22fc19f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:18.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.607+0000 7f230159a700 1 --2- 172.21.15.101:0/1099257748 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f22fc1079b0 0x7f22fc19f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:18.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.607+0000 7f230159a700 1 -- 172.21.15.101:0/1099257748 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f4009c40 con 0x7f22fc074550 2023-07-18T03:07:18.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.607+0000 7f2301d9b700 1 --2- 172.21.15.101:0/1099257748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22fc072f70 0x7f22fc197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:07:18.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.607+0000 7f230259c700 1 --2- 172.21.15.101:0/1099257748 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f22fc1079b0 0x7f22fc19f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:07:18.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.608+0000 7f230159a700 1 --2- 172.21.15.101:0/1099257748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22fc074550 0x7f22fc198010 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f22ec004710 tx=0x7f22ec009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:18.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.608+0000 7f22f2ffd700 1 -- 172.21.15.101:0/1099257748 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ec01d070 con 0x7f22fc074550 2023-07-18T03:07:18.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.608+0000 7f2303fff700 1 -- 172.21.15.101:0/1099257748 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22ec0095f0 con 0x7f22fc074550 2023-07-18T03:07:18.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.608+0000 7f22f2ffd700 1 -- 172.21.15.101:0/1099257748 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22ec021410 con 0x7f22fc074550 2023-07-18T03:07:18.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.608+0000 7f22f2ffd700 1 -- 172.21.15.101:0/1099257748 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ec017650 con 0x7f22fc074550 2023-07-18T03:07:18.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.608+0000 7f2303fff700 1 -- 172.21.15.101:0/1099257748 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22fc19fe90 con 0x7f22fc074550 2023-07-18T03:07:18.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.609+0000 7f2303fff700 1 -- 172.21.15.101:0/1099257748 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22fc106060 con 0x7f22fc074550 2023-07-18T03:07:18.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.611+0000 7f22f2ffd700 1 -- 172.21.15.101:0/1099257748 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f22ec021580 con 0x7f22fc074550 2023-07-18T03:07:18.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.613+0000 7f22f2ffd700 1 --2- 172.21.15.101:0/1099257748 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f22e80a1150 0x7f22e80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:18.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.613+0000 7f22f2ffd700 1 -- 172.21.15.101:0/1099257748 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f22ec0ce6f0 con 0x7f22fc074550 2023-07-18T03:07:18.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.613+0000 7f2301d9b700 1 --2- 172.21.15.101:0/1099257748 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f22e80a1150 0x7f22e80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:18.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.613+0000 7f2301d9b700 1 --2- 172.21.15.101:0/1099257748 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f22e80a1150 0x7f22e80a3600 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f22fc074920 tx=0x7f22f8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:18.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.615+0000 7f22f2ffd700 1 -- 172.21.15.101:0/1099257748 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f22ec097b00 con 0x7f22fc074550 2023-07-18T03:07:18.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.814+0000 7f2303fff700 1 -- 172.21.15.101:0/1099257748 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f22fc02c850 con 0x7f22fc074550 2023-07-18T03:07:18.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.815+0000 7f22f2ffd700 1 -- 172.21.15.101:0/1099257748 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f22ec09b120 con 0x7f22fc074550 2023-07-18T03:07:18.803 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 32 2023-07-18T03:07:18.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.818+0000 7f2303fff700 1 -- 172.21.15.101:0/1099257748 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f22e80a1150 msgr2=0x7f22e80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:18.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.818+0000 7f2303fff700 1 --2- 172.21.15.101:0/1099257748 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f22e80a1150 0x7f22e80a3600 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f22fc074920 tx=0x7f22f8009260 comp rx=0 tx=0).stop 2023-07-18T03:07:18.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.818+0000 7f2303fff700 1 -- 172.21.15.101:0/1099257748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22fc074550 msgr2=0x7f22fc198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:18.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.818+0000 7f2303fff700 1 --2- 172.21.15.101:0/1099257748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22fc074550 0x7f22fc198010 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f22ec004710 tx=0x7f22ec009580 comp rx=0 tx=0).stop 2023-07-18T03:07:18.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.818+0000 7f2303fff700 1 -- 172.21.15.101:0/1099257748 shutdown_connections 2023-07-18T03:07:18.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.818+0000 7f2303fff700 1 --2- 172.21.15.101:0/1099257748 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f22e80a1150 0x7f22e80a3600 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:18.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.818+0000 7f2303fff700 1 --2- 172.21.15.101:0/1099257748 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f22fc1079b0 0x7f22fc19f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:18.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.818+0000 7f2303fff700 1 --2- 172.21.15.101:0/1099257748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22fc072f70 0x7f22fc197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:18.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.818+0000 7f2303fff700 1 --2- 172.21.15.101:0/1099257748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22fc074550 0x7f22fc198010 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:18.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.818+0000 7f2303fff700 1 -- 172.21.15.101:0/1099257748 >> 172.21.15.101:0/1099257748 conn(0x7f22fc0fb130 msgr2=0x7f22fc0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:18.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.818+0000 7f2303fff700 1 -- 172.21.15.101:0/1099257748 shutdown_connections 2023-07-18T03:07:18.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:18.818+0000 7f2303fff700 1 -- 172.21.15.101:0/1099257748 wait complete. 2023-07-18T03:07:18.824 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2023-07-18T03:07:18.824 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:07:19.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.036+0000 7fb2367ff700 1 -- 172.21.15.101:0/784935071 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb230100cf0 msgr2=0x7fb230101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:19.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.036+0000 7fb2367ff700 1 --2- 172.21.15.101:0/784935071 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb230100cf0 0x7fb230101140 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb224009990 tx=0x7fb224009ca0 comp rx=0 tx=0).stop 2023-07-18T03:07:19.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.036+0000 7fb2367ff700 1 -- 172.21.15.101:0/784935071 shutdown_connections 2023-07-18T03:07:19.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.036+0000 7fb2367ff700 1 --2- 172.21.15.101:0/784935071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb230101680 0x7fb230107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:19.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.036+0000 7fb2367ff700 1 --2- 172.21.15.101:0/784935071 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb2300ffb00 0x7fb2300fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:19.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.036+0000 7fb2367ff700 1 --2- 172.21.15.101:0/784935071 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb230100cf0 0x7fb230101140 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:19.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.036+0000 7fb2367ff700 1 -- 172.21.15.101:0/784935071 >> 172.21.15.101:0/784935071 conn(0x7fb2300fb0e0 msgr2=0x7fb2300fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:19.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.036+0000 7fb2367ff700 1 -- 172.21.15.101:0/784935071 shutdown_connections 2023-07-18T03:07:19.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.036+0000 7fb2367ff700 1 -- 172.21.15.101:0/784935071 wait complete. 2023-07-18T03:07:19.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.037+0000 7fb2367ff700 1 Processor -- start 2023-07-18T03:07:19.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.037+0000 7fb2367ff700 1 -- start start 2023-07-18T03:07:19.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.038+0000 7fb2367ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2300ffb00 0x7fb230195800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:19.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.038+0000 7fb2367ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb230100cf0 0x7fb230195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:19.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.038+0000 7fb2367ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb230101680 0x7fb23019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:19.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.038+0000 7fb2367ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb23019d380 con 0x7fb230101680 2023-07-18T03:07:19.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.038+0000 7fb2367ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb23019d4c0 con 0x7fb2300ffb00 2023-07-18T03:07:19.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.038+0000 7fb2367ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb23019d650 con 0x7fb230100cf0 2023-07-18T03:07:19.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.038+0000 7fb22ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2300ffb00 0x7fb230195800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:19.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.038+0000 7fb234d9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb230101680 0x7fb23019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:19.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.038+0000 7fb22ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2300ffb00 0x7fb230195800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60304/0 (socket says 172.21.15.101:60304) 2023-07-18T03:07:19.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.038+0000 7fb22ffff700 1 -- 172.21.15.101:0/3584328588 learned_addr learned my addr 172.21.15.101:0/3584328588 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:19.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.038+0000 7fb22f7fe700 1 --2- 172.21.15.101:0/3584328588 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb230100cf0 0x7fb230195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:19.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.038+0000 7fb234d9c700 1 -- 172.21.15.101:0/3584328588 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb230100cf0 msgr2=0x7fb230195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:19.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.038+0000 7fb234d9c700 1 --2- 172.21.15.101:0/3584328588 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb230100cf0 0x7fb230195d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:19.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.038+0000 7fb234d9c700 1 -- 172.21.15.101:0/3584328588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2300ffb00 msgr2=0x7fb230195800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:19.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.038+0000 7fb234d9c700 1 --2- 172.21.15.101:0/3584328588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2300ffb00 0x7fb230195800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:19.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.038+0000 7fb234d9c700 1 -- 172.21.15.101:0/3584328588 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb228009c40 con 0x7fb230101680 2023-07-18T03:07:19.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.038+0000 7fb22ffff700 1 --2- 172.21.15.101:0/3584328588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2300ffb00 0x7fb230195800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:07:19.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.039+0000 7fb234d9c700 1 --2- 172.21.15.101:0/3584328588 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb230101680 0x7fb23019cdb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb22800efb0 tx=0x7fb22800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:19.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.039+0000 7fb22d7fa700 1 -- 172.21.15.101:0/3584328588 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb228003a00 con 0x7fb230101680 2023-07-18T03:07:19.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.039+0000 7fb22d7fa700 1 -- 172.21.15.101:0/3584328588 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb22800f040 con 0x7fb230101680 2023-07-18T03:07:19.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.039+0000 7fb2367ff700 1 -- 172.21.15.101:0/3584328588 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2240095f0 con 0x7fb230101680 2023-07-18T03:07:19.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.039+0000 7fb22d7fa700 1 -- 172.21.15.101:0/3584328588 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb228004540 con 0x7fb230101680 2023-07-18T03:07:19.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.039+0000 7fb2367ff700 1 -- 172.21.15.101:0/3584328588 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb23019dd00 con 0x7fb230101680 2023-07-18T03:07:19.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.040+0000 7fb2367ff700 1 -- 172.21.15.101:0/3584328588 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb230107030 con 0x7fb230101680 2023-07-18T03:07:19.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.042+0000 7fb22d7fa700 1 -- 172.21.15.101:0/3584328588 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fb228003bf0 con 0x7fb230101680 2023-07-18T03:07:19.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.043+0000 7fb22d7fa700 1 --2- 172.21.15.101:0/3584328588 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb2180a1220 0x7fb2180a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:19.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.044+0000 7fb22d7fa700 1 -- 172.21.15.101:0/3584328588 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fb2280cc850 con 0x7fb230101680 2023-07-18T03:07:19.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.044+0000 7fb22ffff700 1 --2- 172.21.15.101:0/3584328588 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb2180a1220 0x7fb2180a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:19.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.044+0000 7fb22ffff700 1 --2- 172.21.15.101:0/3584328588 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb2180a1220 0x7fb2180a36d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb220005950 tx=0x7fb220009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:19.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.047+0000 7fb22d7fa700 1 -- 172.21.15.101:0/3584328588 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb228095c60 con 0x7fb230101680 2023-07-18T03:07:19.194 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:18 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/129402822' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:07:19.194 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:18 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/3296910028' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:07:19.194 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:18 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1099257748' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-18T03:07:19.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:18 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/129402822' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:07:19.232 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:18 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3296910028' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:07:19.232 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:18 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1099257748' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-18T03:07:19.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.291+0000 7fb2367ff700 1 -- 172.21.15.101:0/3584328588 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb23002c850 con 0x7fb230101680 2023-07-18T03:07:19.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.292+0000 7fb22d7fa700 1 -- 172.21.15.101:0/3584328588 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10858 (secure 0 0 0) 0x7fb228099280 con 0x7fb230101680 2023-07-18T03:07:19.280 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:07:19.280 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/4192846769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":4192846769},{"type":"v1","addr":"172.21.15.101:6839","nonce":4192846769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6841/3824340175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6840","nonce":3824340175},{"type":"v1","addr":"172.21.15.101:6841","nonce":3824340175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:04:59.361185+0000","modified":"2023-07-18T03:05:31.985959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.155:6839/2055462483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2055462483},{"type":"v1","addr":"172.21.15.155:6839","nonce":2055462483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:07:19.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.294+0000 7fb2367ff700 1 -- 172.21.15.101:0/3584328588 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb2180a1220 msgr2=0x7fb2180a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:19.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.294+0000 7fb2367ff700 1 --2- 172.21.15.101:0/3584328588 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb2180a1220 0x7fb2180a36d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb220005950 tx=0x7fb220009260 comp rx=0 tx=0).stop 2023-07-18T03:07:19.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.294+0000 7fb2367ff700 1 -- 172.21.15.101:0/3584328588 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb230101680 msgr2=0x7fb23019cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:19.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.294+0000 7fb2367ff700 1 --2- 172.21.15.101:0/3584328588 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb230101680 0x7fb23019cdb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb22800efb0 tx=0x7fb22800c3a0 comp rx=0 tx=0).stop 2023-07-18T03:07:19.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.294+0000 7fb2367ff700 1 -- 172.21.15.101:0/3584328588 shutdown_connections 2023-07-18T03:07:19.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.294+0000 7fb2367ff700 1 --2- 172.21.15.101:0/3584328588 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fb2180a1220 0x7fb2180a36d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:19.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.294+0000 7fb2367ff700 1 --2- 172.21.15.101:0/3584328588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2300ffb00 0x7fb230195800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:19.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.294+0000 7fb2367ff700 1 --2- 172.21.15.101:0/3584328588 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb230101680 0x7fb23019cdb0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:19.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.294+0000 7fb2367ff700 1 --2- 172.21.15.101:0/3584328588 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb230100cf0 0x7fb230195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:19.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.294+0000 7fb2367ff700 1 -- 172.21.15.101:0/3584328588 >> 172.21.15.101:0/3584328588 conn(0x7fb2300fb0e0 msgr2=0x7fb230104170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:19.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.294+0000 7fb2367ff700 1 -- 172.21.15.101:0/3584328588 shutdown_connections 2023-07-18T03:07:19.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.295+0000 7fb2367ff700 1 -- 172.21.15.101:0/3584328588 wait complete. 2023-07-18T03:07:19.284 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 24 2023-07-18T03:07:19.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:18 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/129402822' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:07:19.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:18 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/3296910028' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:07:19.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:18 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1099257748' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-18T03:07:19.301 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2023-07-18T03:07:19.301 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2023-07-18T03:07:19.301 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-18T03:07:19.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.512+0000 7f15417ff700 1 -- 172.21.15.101:0/4244065489 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f153c072f10 msgr2=0x7f153c073380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:19.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.512+0000 7f15417ff700 1 --2- 172.21.15.101:0/4244065489 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f153c072f10 0x7f153c073380 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1530009990 tx=0x7f1530009ca0 comp rx=0 tx=0).stop 2023-07-18T03:07:19.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.512+0000 7f15417ff700 1 -- 172.21.15.101:0/4244065489 shutdown_connections 2023-07-18T03:07:19.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.512+0000 7f15417ff700 1 --2- 172.21.15.101:0/4244065489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f153c107990 0x7f153c109e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:19.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.512+0000 7f15417ff700 1 --2- 172.21.15.101:0/4244065489 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f153c0744f0 0x7f153c072940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:19.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.512+0000 7f15417ff700 1 --2- 172.21.15.101:0/4244065489 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f153c072f10 0x7f153c073380 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:19.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.512+0000 7f15417ff700 1 -- 172.21.15.101:0/4244065489 >> 172.21.15.101:0/4244065489 conn(0x7f153c0fb0e0 msgr2=0x7f153c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:19.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.512+0000 7f15417ff700 1 -- 172.21.15.101:0/4244065489 shutdown_connections 2023-07-18T03:07:19.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.512+0000 7f15417ff700 1 -- 172.21.15.101:0/4244065489 wait complete. 2023-07-18T03:07:19.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.513+0000 7f15417ff700 1 Processor -- start 2023-07-18T03:07:19.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.513+0000 7f15417ff700 1 -- start start 2023-07-18T03:07:19.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.513+0000 7f15417ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f153c072f10 0x7f153c197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:19.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.513+0000 7f15417ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f153c0744f0 0x7f153c197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:19.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.513+0000 7f15417ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f153c107990 0x7f153c19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:19.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.513+0000 7f15417ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153c19f590 con 0x7f153c107990 2023-07-18T03:07:19.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.513+0000 7f15417ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153c19f6d0 con 0x7f153c072f10 2023-07-18T03:07:19.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.513+0000 7f15417ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153c19f860 con 0x7f153c0744f0 2023-07-18T03:07:19.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.513+0000 7f153b7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f153c107990 0x7f153c19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:19.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.514+0000 7f153b7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f153c107990 0x7f153c19efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:36454/0 (socket says 172.21.15.101:36454) 2023-07-18T03:07:19.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.514+0000 7f153b7fe700 1 -- 172.21.15.101:0/1929699643 learned_addr learned my addr 172.21.15.101:0/1929699643 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:19.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.514+0000 7f153a7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f153c0744f0 0x7f153c197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:19.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.514+0000 7f153affd700 1 --2- 172.21.15.101:0/1929699643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f153c072f10 0x7f153c197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:19.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.514+0000 7f153b7fe700 1 -- 172.21.15.101:0/1929699643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f153c0744f0 msgr2=0x7f153c197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:19.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.514+0000 7f153b7fe700 1 --2- 172.21.15.101:0/1929699643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f153c0744f0 0x7f153c197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:19.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.514+0000 7f153b7fe700 1 -- 172.21.15.101:0/1929699643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f153c072f10 msgr2=0x7f153c197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:19.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.514+0000 7f153b7fe700 1 --2- 172.21.15.101:0/1929699643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f153c072f10 0x7f153c197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:19.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.514+0000 7f153b7fe700 1 -- 172.21.15.101:0/1929699643 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15300095f0 con 0x7f153c107990 2023-07-18T03:07:19.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.514+0000 7f153b7fe700 1 --2- 172.21.15.101:0/1929699643 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f153c107990 0x7f153c19efc0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1534007ee0 tx=0x7f153400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:19.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.514+0000 7f1523fff700 1 -- 172.21.15.101:0/1929699643 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1534017070 con 0x7f153c107990 2023-07-18T03:07:19.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.514+0000 7f15417ff700 1 -- 172.21.15.101:0/1929699643 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f153c19fb40 con 0x7f153c107990 2023-07-18T03:07:19.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.514+0000 7f1523fff700 1 -- 172.21.15.101:0/1929699643 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f153400f040 con 0x7f153c107990 2023-07-18T03:07:19.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.514+0000 7f1523fff700 1 -- 172.21.15.101:0/1929699643 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1534005550 con 0x7f153c107990 2023-07-18T03:07:19.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.514+0000 7f15417ff700 1 -- 172.21.15.101:0/1929699643 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f153c1a0050 con 0x7f153c107990 2023-07-18T03:07:19.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.516+0000 7f15417ff700 1 -- 172.21.15.101:0/1929699643 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f153c1060d0 con 0x7f153c107990 2023-07-18T03:07:19.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.517+0000 7f1523fff700 1 -- 172.21.15.101:0/1929699643 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f153400db20 con 0x7f153c107990 2023-07-18T03:07:19.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.519+0000 7f1523fff700 1 --2- 172.21.15.101:0/1929699643 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f15240a11a0 0x7f15240a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:19.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.519+0000 7f1523fff700 1 -- 172.21.15.101:0/1929699643 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f15340cc3b0 con 0x7f153c107990 2023-07-18T03:07:19.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.519+0000 7f153affd700 1 --2- 172.21.15.101:0/1929699643 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f15240a11a0 0x7f15240a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:19.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.520+0000 7f153affd700 1 --2- 172.21.15.101:0/1929699643 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f15240a11a0 0x7f15240a3650 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f152c009c60 tx=0x7f152c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:19.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.523+0000 7f1523fff700 1 -- 172.21.15.101:0/1929699643 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1534095790 con 0x7f153c107990 2023-07-18T03:07:19.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.714+0000 7f15417ff700 1 -- 172.21.15.101:0/1929699643 --> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f153c060950 con 0x7f15240a11a0 2023-07-18T03:07:19.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.726+0000 7f1523fff700 1 -- 172.21.15.101:0/1929699643 <== mgr.14150 v2:172.21.15.101:6800/3963742842 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427075 (secure 0 0 0) 0x7f153c060950 con 0x7f15240a11a0 2023-07-18T03:07:19.718 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:07:19.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.731+0000 7f15417ff700 1 -- 172.21.15.101:0/1929699643 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f15240a11a0 msgr2=0x7f15240a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:19.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.731+0000 7f15417ff700 1 --2- 172.21.15.101:0/1929699643 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f15240a11a0 0x7f15240a3650 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f152c009c60 tx=0x7f152c009260 comp rx=0 tx=0).stop 2023-07-18T03:07:19.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.731+0000 7f15417ff700 1 -- 172.21.15.101:0/1929699643 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f153c107990 msgr2=0x7f153c19efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:19.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.731+0000 7f15417ff700 1 --2- 172.21.15.101:0/1929699643 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f153c107990 0x7f153c19efc0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1534007ee0 tx=0x7f153400d3b0 comp rx=0 tx=0).stop 2023-07-18T03:07:19.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.731+0000 7f15417ff700 1 -- 172.21.15.101:0/1929699643 shutdown_connections 2023-07-18T03:07:19.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.731+0000 7f15417ff700 1 --2- 172.21.15.101:0/1929699643 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f15240a11a0 0x7f15240a3650 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:19.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.731+0000 7f15417ff700 1 --2- 172.21.15.101:0/1929699643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f153c072f10 0x7f153c197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:19.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.731+0000 7f15417ff700 1 --2- 172.21.15.101:0/1929699643 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f153c107990 0x7f153c19efc0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:19.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.731+0000 7f15417ff700 1 --2- 172.21.15.101:0/1929699643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f153c0744f0 0x7f153c197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:19.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.731+0000 7f15417ff700 1 -- 172.21.15.101:0/1929699643 >> 172.21.15.101:0/1929699643 conn(0x7f153c0fb0e0 msgr2=0x7f153c0fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:19.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.732+0000 7f15417ff700 1 -- 172.21.15.101:0/1929699643 shutdown_connections 2023-07-18T03:07:19.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.732+0000 7f15417ff700 1 -- 172.21.15.101:0/1929699643 wait complete. 2023-07-18T03:07:19.722 INFO:teuthology.orchestra.run.smithi101.stderr:dumped all 2023-07-18T03:07:19.737 INFO:teuthology.orchestra.run.smithi101.stdout:{"pg_ready":true,"pg_map":{"version":349,"stamp":"2023-07-18T03:07:18.885623+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661440,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":791,"num_read_kb":523,"num_write":349,"num_write_kb":3064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":785,"ondisk_log_size":785,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335600,"kb_used_data":12176,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606224,"statfs":{"total":1151940427776,"available":1151596773376,"internally_reserved":0,"allocated":12468224,"data_stored":8045618,"data_compressed":389565,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5695,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":77,"num_read_kb":47,"num_write":5,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002975"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.887431+0000","last_change":"2023-07-18T03:04:59.805689+0000","last_active":"2023-07-18T03:06:52.887431+0000","last_peered":"2023-07-18T03:06:52.887431+0000","last_clean":"2023-07-18T03:06:52.887431+0000","last_became_active":"2023-07-18T03:04:59.805224+0000","last_became_peered":"2023-07-18T03:04:59.805224+0000","last_unstale":"2023-07-18T03:06:52.887431+0000","last_undegraded":"2023-07-18T03:06:52.887431+0000","last_fullsized":"2023-07-18T03:06:52.887431+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:06:51.978483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.886501+0000","last_change":"2023-07-18T03:04:59.805013+0000","last_active":"2023-07-18T03:06:52.886501+0000","last_peered":"2023-07-18T03:06:52.886501+0000","last_clean":"2023-07-18T03:06:52.886501+0000","last_became_active":"2023-07-18T03:04:59.804702+0000","last_became_peered":"2023-07-18T03:04:59.804702+0000","last_unstale":"2023-07-18T03:06:52.886501+0000","last_undegraded":"2023-07-18T03:06:52.886501+0000","last_fullsized":"2023-07-18T03:06:52.886501+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:50:03.882265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.884770+0000","last_change":"2023-07-18T03:04:59.809035+0000","last_active":"2023-07-18T03:06:52.884770+0000","last_peered":"2023-07-18T03:06:52.884770+0000","last_clean":"2023-07-18T03:06:52.884770+0000","last_became_active":"2023-07-18T03:04:59.808630+0000","last_became_peered":"2023-07-18T03:04:59.808630+0000","last_unstale":"2023-07-18T03:06:52.884770+0000","last_undegraded":"2023-07-18T03:06:52.884770+0000","last_fullsized":"2023-07-18T03:06:52.884770+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:09:21.776254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.881696+0000","last_change":"2023-07-18T03:04:59.803510+0000","last_active":"2023-07-18T03:06:52.881696+0000","last_peered":"2023-07-18T03:06:52.881696+0000","last_clean":"2023-07-18T03:06:52.881696+0000","last_became_active":"2023-07-18T03:04:59.803060+0000","last_became_peered":"2023-07-18T03:04:59.803060+0000","last_unstale":"2023-07-18T03:06:52.881696+0000","last_undegraded":"2023-07-18T03:06:52.881696+0000","last_fullsized":"2023-07-18T03:06:52.881696+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:05:42.674904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.878377+0000","last_change":"2023-07-18T03:04:59.806911+0000","last_active":"2023-07-18T03:06:52.878377+0000","last_peered":"2023-07-18T03:06:52.878377+0000","last_clean":"2023-07-18T03:06:52.878377+0000","last_became_active":"2023-07-18T03:04:59.806597+0000","last_became_peered":"2023-07-18T03:04:59.806597+0000","last_unstale":"2023-07-18T03:06:52.878377+0000","last_undegraded":"2023-07-18T03:06:52.878377+0000","last_fullsized":"2023-07-18T03:06:52.878377+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:23:04.609208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.876882+0000","last_change":"2023-07-18T03:04:59.809263+0000","last_active":"2023-07-18T03:06:52.876882+0000","last_peered":"2023-07-18T03:06:52.876882+0000","last_clean":"2023-07-18T03:06:52.876882+0000","last_became_active":"2023-07-18T03:04:59.808956+0000","last_became_peered":"2023-07-18T03:04:59.808956+0000","last_unstale":"2023-07-18T03:06:52.876882+0000","last_undegraded":"2023-07-18T03:06:52.876882+0000","last_fullsized":"2023-07-18T03:06:52.876882+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:22:13.425291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.881590+0000","last_change":"2023-07-18T03:04:59.804946+0000","last_active":"2023-07-18T03:06:52.881590+0000","last_peered":"2023-07-18T03:06:52.881590+0000","last_clean":"2023-07-18T03:06:52.881590+0000","last_became_active":"2023-07-18T03:04:59.804610+0000","last_became_peered":"2023-07-18T03:04:59.804610+0000","last_unstale":"2023-07-18T03:06:52.881590+0000","last_undegraded":"2023-07-18T03:06:52.881590+0000","last_fullsized":"2023-07-18T03:06:52.881590+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:35:53.147695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.886091+0000","last_change":"2023-07-18T03:04:59.801796+0000","last_active":"2023-07-18T03:06:52.886091+0000","last_peered":"2023-07-18T03:06:52.886091+0000","last_clean":"2023-07-18T03:06:52.886091+0000","last_became_active":"2023-07-18T03:04:59.801432+0000","last_became_peered":"2023-07-18T03:04:59.801432+0000","last_unstale":"2023-07-18T03:06:52.886091+0000","last_undegraded":"2023-07-18T03:06:52.886091+0000","last_fullsized":"2023-07-18T03:06:52.886091+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:32:07.392642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.887741+0000","last_change":"2023-07-18T03:04:59.798191+0000","last_active":"2023-07-18T03:06:52.887741+0000","last_peered":"2023-07-18T03:06:52.887741+0000","last_clean":"2023-07-18T03:06:52.887741+0000","last_became_active":"2023-07-18T03:04:59.797863+0000","last_became_peered":"2023-07-18T03:04:59.797863+0000","last_unstale":"2023-07-18T03:06:52.887741+0000","last_undegraded":"2023-07-18T03:06:52.887741+0000","last_fullsized":"2023-07-18T03:06:52.887741+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:43:29.304091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.881104+0000","last_change":"2023-07-18T03:04:59.809867+0000","last_active":"2023-07-18T03:06:52.881104+0000","last_peered":"2023-07-18T03:06:52.881104+0000","last_clean":"2023-07-18T03:06:52.881104+0000","last_became_active":"2023-07-18T03:04:59.809568+0000","last_became_peered":"2023-07-18T03:04:59.809568+0000","last_unstale":"2023-07-18T03:06:52.881104+0000","last_undegraded":"2023-07-18T03:06:52.881104+0000","last_fullsized":"2023-07-18T03:06:52.881104+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:49:15.008273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.886428+0000","last_change":"2023-07-18T03:04:59.805920+0000","last_active":"2023-07-18T03:06:52.886428+0000","last_peered":"2023-07-18T03:06:52.886428+0000","last_clean":"2023-07-18T03:06:52.886428+0000","last_became_active":"2023-07-18T03:04:59.805559+0000","last_became_peered":"2023-07-18T03:04:59.805559+0000","last_unstale":"2023-07-18T03:06:52.886428+0000","last_undegraded":"2023-07-18T03:06:52.886428+0000","last_fullsized":"2023-07-18T03:06:52.886428+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:27:52.167301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.880048+0000","last_change":"2023-07-18T03:04:59.797940+0000","last_active":"2023-07-18T03:06:52.880048+0000","last_peered":"2023-07-18T03:06:52.880048+0000","last_clean":"2023-07-18T03:06:52.880048+0000","last_became_active":"2023-07-18T03:04:59.797575+0000","last_became_peered":"2023-07-18T03:04:59.797575+0000","last_unstale":"2023-07-18T03:06:52.880048+0000","last_undegraded":"2023-07-18T03:06:52.880048+0000","last_fullsized":"2023-07-18T03:06:52.880048+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:58:43.120283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.894941+0000","last_change":"2023-07-18T03:04:59.798026+0000","last_active":"2023-07-18T03:06:52.894941+0000","last_peered":"2023-07-18T03:06:52.894941+0000","last_clean":"2023-07-18T03:06:52.894941+0000","last_became_active":"2023-07-18T03:04:59.797734+0000","last_became_peered":"2023-07-18T03:04:59.797734+0000","last_unstale":"2023-07-18T03:06:52.894941+0000","last_undegraded":"2023-07-18T03:06:52.894941+0000","last_fullsized":"2023-07-18T03:06:52.894941+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:33:40.010231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.886275+0000","last_change":"2023-07-18T03:04:59.804964+0000","last_active":"2023-07-18T03:06:52.886275+0000","last_peered":"2023-07-18T03:06:52.886275+0000","last_clean":"2023-07-18T03:06:52.886275+0000","last_became_active":"2023-07-18T03:04:59.804660+0000","last_became_peered":"2023-07-18T03:04:59.804660+0000","last_unstale":"2023-07-18T03:06:52.886275+0000","last_undegraded":"2023-07-18T03:06:52.886275+0000","last_fullsized":"2023-07-18T03:06:52.886275+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:11:48.365954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.885687+0000","last_change":"2023-07-18T03:04:59.809384+0000","last_active":"2023-07-18T03:06:52.885687+0000","last_peered":"2023-07-18T03:06:52.885687+0000","last_clean":"2023-07-18T03:06:52.885687+0000","last_became_active":"2023-07-18T03:04:59.808969+0000","last_became_peered":"2023-07-18T03:04:59.808969+0000","last_unstale":"2023-07-18T03:06:52.885687+0000","last_undegraded":"2023-07-18T03:06:52.885687+0000","last_fullsized":"2023-07-18T03:06:52.885687+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:16:18.639337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.884787+0000","last_change":"2023-07-18T03:04:59.808344+0000","last_active":"2023-07-18T03:06:52.884787+0000","last_peered":"2023-07-18T03:06:52.884787+0000","last_clean":"2023-07-18T03:06:52.884787+0000","last_became_active":"2023-07-18T03:04:59.807995+0000","last_became_peered":"2023-07-18T03:04:59.807995+0000","last_unstale":"2023-07-18T03:06:52.884787+0000","last_undegraded":"2023-07-18T03:06:52.884787+0000","last_fullsized":"2023-07-18T03:06:52.884787+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:15:44.462832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.887652+0000","last_change":"2023-07-18T03:04:59.806584+0000","last_active":"2023-07-18T03:06:52.887652+0000","last_peered":"2023-07-18T03:06:52.887652+0000","last_clean":"2023-07-18T03:06:52.887652+0000","last_became_active":"2023-07-18T03:04:59.804118+0000","last_became_peered":"2023-07-18T03:04:59.804118+0000","last_unstale":"2023-07-18T03:06:52.887652+0000","last_undegraded":"2023-07-18T03:06:52.887652+0000","last_fullsized":"2023-07-18T03:06:52.887652+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:42:18.047326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.890768+0000","last_change":"2023-07-18T03:04:59.812610+0000","last_active":"2023-07-18T03:06:52.890768+0000","last_peered":"2023-07-18T03:06:52.890768+0000","last_clean":"2023-07-18T03:06:52.890768+0000","last_became_active":"2023-07-18T03:04:59.812230+0000","last_became_peered":"2023-07-18T03:04:59.812230+0000","last_unstale":"2023-07-18T03:06:52.890768+0000","last_undegraded":"2023-07-18T03:06:52.890768+0000","last_fullsized":"2023-07-18T03:06:52.890768+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:07:54.810466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.893873+0000","last_change":"2023-07-18T03:04:59.811880+0000","last_active":"2023-07-18T03:06:52.893873+0000","last_peered":"2023-07-18T03:06:52.893873+0000","last_clean":"2023-07-18T03:06:52.893873+0000","last_became_active":"2023-07-18T03:04:59.811561+0000","last_became_peered":"2023-07-18T03:04:59.811561+0000","last_unstale":"2023-07-18T03:06:52.893873+0000","last_undegraded":"2023-07-18T03:06:52.893873+0000","last_fullsized":"2023-07-18T03:06:52.893873+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:53:25.407425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.256306+0000","last_change":"2023-07-18T03:05:03.880843+0000","last_active":"2023-07-18T03:06:24.256306+0000","last_peered":"2023-07-18T03:06:24.256306+0000","last_clean":"2023-07-18T03:06:24.256306+0000","last_became_active":"2023-07-18T03:05:03.880391+0000","last_became_peered":"2023-07-18T03:05:03.880391+0000","last_unstale":"2023-07-18T03:06:24.256306+0000","last_undegraded":"2023-07-18T03:06:24.256306+0000","last_fullsized":"2023-07-18T03:06:24.256306+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:25:52.190341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255219+0000","last_change":"2023-07-18T03:04:58.848771+0000","last_active":"2023-07-18T03:06:24.255219+0000","last_peered":"2023-07-18T03:06:24.255219+0000","last_clean":"2023-07-18T03:06:24.255219+0000","last_became_active":"2023-07-18T03:04:58.848405+0000","last_became_peered":"2023-07-18T03:04:58.848405+0000","last_unstale":"2023-07-18T03:06:24.255219+0000","last_undegraded":"2023-07-18T03:06:24.255219+0000","last_fullsized":"2023-07-18T03:06:24.255219+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:09:13.965066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.881139+0000","last_change":"2023-07-18T03:04:59.805749+0000","last_active":"2023-07-18T03:06:52.881139+0000","last_peered":"2023-07-18T03:06:52.881139+0000","last_clean":"2023-07-18T03:06:52.881139+0000","last_became_active":"2023-07-18T03:04:59.805388+0000","last_became_peered":"2023-07-18T03:04:59.805388+0000","last_unstale":"2023-07-18T03:06:52.881139+0000","last_undegraded":"2023-07-18T03:06:52.881139+0000","last_fullsized":"2023-07-18T03:06:52.881139+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:36:19.960742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255676+0000","last_change":"2023-07-18T03:05:03.868819+0000","last_active":"2023-07-18T03:06:24.255676+0000","last_peered":"2023-07-18T03:06:24.255676+0000","last_clean":"2023-07-18T03:06:24.255676+0000","last_became_active":"2023-07-18T03:05:03.868387+0000","last_became_peered":"2023-07-18T03:05:03.868387+0000","last_unstale":"2023-07-18T03:06:24.255676+0000","last_undegraded":"2023-07-18T03:06:24.255676+0000","last_fullsized":"2023-07-18T03:06:24.255676+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:34:30.099371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.375503+0000","last_change":"2023-07-18T03:04:58.856084+0000","last_active":"2023-07-18T03:06:24.375503+0000","last_peered":"2023-07-18T03:06:24.375503+0000","last_clean":"2023-07-18T03:06:24.375503+0000","last_became_active":"2023-07-18T03:04:58.855768+0000","last_became_peered":"2023-07-18T03:04:58.855768+0000","last_unstale":"2023-07-18T03:06:24.375503+0000","last_undegraded":"2023-07-18T03:06:24.375503+0000","last_fullsized":"2023-07-18T03:06:24.375503+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:58:35.159671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.885217+0000","last_change":"2023-07-18T03:04:59.809512+0000","last_active":"2023-07-18T03:06:52.885217+0000","last_peered":"2023-07-18T03:06:52.885217+0000","last_clean":"2023-07-18T03:06:52.885217+0000","last_became_active":"2023-07-18T03:04:59.808112+0000","last_became_peered":"2023-07-18T03:04:59.808112+0000","last_unstale":"2023-07-18T03:06:52.885217+0000","last_undegraded":"2023-07-18T03:06:52.885217+0000","last_fullsized":"2023-07-18T03:06:52.885217+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:42:34.279489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255283+0000","last_change":"2023-07-18T03:05:03.874728+0000","last_active":"2023-07-18T03:06:24.255283+0000","last_peered":"2023-07-18T03:06:24.255283+0000","last_clean":"2023-07-18T03:06:24.255283+0000","last_became_active":"2023-07-18T03:05:03.874262+0000","last_became_peered":"2023-07-18T03:05:03.874262+0000","last_unstale":"2023-07-18T03:06:24.255283+0000","last_undegraded":"2023-07-18T03:06:24.255283+0000","last_fullsized":"2023-07-18T03:06:24.255283+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:31:54.083323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.261317+0000","last_change":"2023-07-18T03:04:58.849140+0000","last_active":"2023-07-18T03:06:24.261317+0000","last_peered":"2023-07-18T03:06:24.261317+0000","last_clean":"2023-07-18T03:06:24.261317+0000","last_became_active":"2023-07-18T03:04:58.848693+0000","last_became_peered":"2023-07-18T03:04:58.848693+0000","last_unstale":"2023-07-18T03:06:24.261317+0000","last_undegraded":"2023-07-18T03:06:24.261317+0000","last_fullsized":"2023-07-18T03:06:24.261317+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:07:50.859417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.892017+0000","last_change":"2023-07-18T03:04:59.807248+0000","last_active":"2023-07-18T03:06:52.892017+0000","last_peered":"2023-07-18T03:06:52.892017+0000","last_clean":"2023-07-18T03:06:52.892017+0000","last_became_active":"2023-07-18T03:04:59.806807+0000","last_became_peered":"2023-07-18T03:04:59.806807+0000","last_unstale":"2023-07-18T03:06:52.892017+0000","last_undegraded":"2023-07-18T03:06:52.892017+0000","last_fullsized":"2023-07-18T03:06:52.892017+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:07:33.904712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255269+0000","last_change":"2023-07-18T03:05:03.869075+0000","last_active":"2023-07-18T03:06:24.255269+0000","last_peered":"2023-07-18T03:06:24.255269+0000","last_clean":"2023-07-18T03:06:24.255269+0000","last_became_active":"2023-07-18T03:05:03.868653+0000","last_became_peered":"2023-07-18T03:05:03.868653+0000","last_unstale":"2023-07-18T03:06:24.255269+0000","last_undegraded":"2023-07-18T03:06:24.255269+0000","last_fullsized":"2023-07-18T03:06:24.255269+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:05:18.894653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.259374+0000","last_change":"2023-07-18T03:04:58.843879+0000","last_active":"2023-07-18T03:06:24.259374+0000","last_peered":"2023-07-18T03:06:24.259374+0000","last_clean":"2023-07-18T03:06:24.259374+0000","last_became_active":"2023-07-18T03:04:58.843525+0000","last_became_peered":"2023-07-18T03:04:58.843525+0000","last_unstale":"2023-07-18T03:06:24.259374+0000","last_undegraded":"2023-07-18T03:06:24.259374+0000","last_fullsized":"2023-07-18T03:06:24.259374+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:32:03.232823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.886046+0000","last_change":"2023-07-18T03:04:59.803597+0000","last_active":"2023-07-18T03:06:52.886046+0000","last_peered":"2023-07-18T03:06:52.886046+0000","last_clean":"2023-07-18T03:06:52.886046+0000","last_became_active":"2023-07-18T03:04:59.803293+0000","last_became_peered":"2023-07-18T03:04:59.803293+0000","last_unstale":"2023-07-18T03:06:52.886046+0000","last_undegraded":"2023-07-18T03:06:52.886046+0000","last_fullsized":"2023-07-18T03:06:52.886046+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:31:55.196443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255711+0000","last_change":"2023-07-18T03:05:23.944721+0000","last_active":"2023-07-18T03:06:24.255711+0000","last_peered":"2023-07-18T03:06:24.255711+0000","last_clean":"2023-07-18T03:06:24.255711+0000","last_became_active":"2023-07-18T03:05:23.944101+0000","last_became_peered":"2023-07-18T03:05:23.944101+0000","last_unstale":"2023-07-18T03:06:24.255711+0000","last_undegraded":"2023-07-18T03:06:24.255711+0000","last_fullsized":"2023-07-18T03:06:24.255711+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:20:50.819953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":145,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:07:14.145822+0000","last_change":"2023-07-18T03:04:58.847990+0000","last_active":"2023-07-18T03:07:14.145822+0000","last_peered":"2023-07-18T03:07:14.145822+0000","last_clean":"2023-07-18T03:07:14.145822+0000","last_became_active":"2023-07-18T03:04:58.847634+0000","last_became_peered":"2023-07-18T03:04:58.847634+0000","last_unstale":"2023-07-18T03:07:14.145822+0000","last_undegraded":"2023-07-18T03:07:14.145822+0000","last_fullsized":"2023-07-18T03:07:14.145822+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:14:18.454279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1105,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.881401+0000","last_change":"2023-07-18T03:04:59.797076+0000","last_active":"2023-07-18T03:06:52.881401+0000","last_peered":"2023-07-18T03:06:52.881401+0000","last_clean":"2023-07-18T03:06:52.881401+0000","last_became_active":"2023-07-18T03:04:59.796753+0000","last_became_peered":"2023-07-18T03:04:59.796753+0000","last_unstale":"2023-07-18T03:06:52.881401+0000","last_undegraded":"2023-07-18T03:06:52.881401+0000","last_fullsized":"2023-07-18T03:06:52.881401+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:46:03.085633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255734+0000","last_change":"2023-07-18T03:05:03.881793+0000","last_active":"2023-07-18T03:06:24.255734+0000","last_peered":"2023-07-18T03:06:24.255734+0000","last_clean":"2023-07-18T03:06:24.255734+0000","last_became_active":"2023-07-18T03:05:03.881341+0000","last_became_peered":"2023-07-18T03:05:03.881341+0000","last_unstale":"2023-07-18T03:06:24.255734+0000","last_undegraded":"2023-07-18T03:06:24.255734+0000","last_fullsized":"2023-07-18T03:06:24.255734+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:40:23.370420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.375558+0000","last_change":"2023-07-18T03:04:58.846732+0000","last_active":"2023-07-18T03:06:24.375558+0000","last_peered":"2023-07-18T03:06:24.375558+0000","last_clean":"2023-07-18T03:06:24.375558+0000","last_became_active":"2023-07-18T03:04:58.846321+0000","last_became_peered":"2023-07-18T03:04:58.846321+0000","last_unstale":"2023-07-18T03:06:24.375558+0000","last_undegraded":"2023-07-18T03:06:24.375558+0000","last_fullsized":"2023-07-18T03:06:24.375558+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:50:00.381631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.879809+0000","last_change":"2023-07-18T03:05:24.093277+0000","last_active":"2023-07-18T03:06:52.879809+0000","last_peered":"2023-07-18T03:06:52.879809+0000","last_clean":"2023-07-18T03:06:52.879809+0000","last_became_active":"2023-07-18T03:05:24.092842+0000","last_became_peered":"2023-07-18T03:05:24.092842+0000","last_unstale":"2023-07-18T03:06:52.879809+0000","last_undegraded":"2023-07-18T03:06:52.879809+0000","last_fullsized":"2023-07-18T03:06:52.879809+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:22:01.737422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.257633+0000","last_change":"2023-07-18T03:05:03.880782+0000","last_active":"2023-07-18T03:06:24.257633+0000","last_peered":"2023-07-18T03:06:24.257633+0000","last_clean":"2023-07-18T03:06:24.257633+0000","last_became_active":"2023-07-18T03:05:03.880362+0000","last_became_peered":"2023-07-18T03:05:03.880362+0000","last_unstale":"2023-07-18T03:06:24.257633+0000","last_undegraded":"2023-07-18T03:06:24.257633+0000","last_fullsized":"2023-07-18T03:06:24.257633+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:10:14.772789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.750818+0000","last_change":"2023-07-18T03:04:58.846095+0000","last_active":"2023-07-18T03:06:24.750818+0000","last_peered":"2023-07-18T03:06:24.750818+0000","last_clean":"2023-07-18T03:06:24.750818+0000","last_became_active":"2023-07-18T03:04:58.845242+0000","last_became_peered":"2023-07-18T03:04:58.845242+0000","last_unstale":"2023-07-18T03:06:24.750818+0000","last_undegraded":"2023-07-18T03:06:24.750818+0000","last_fullsized":"2023-07-18T03:06:24.750818+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:36:00.462709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.889059+0000","last_change":"2023-07-18T03:04:59.803024+0000","last_active":"2023-07-18T03:06:52.889059+0000","last_peered":"2023-07-18T03:06:52.889059+0000","last_clean":"2023-07-18T03:06:52.889059+0000","last_became_active":"2023-07-18T03:04:59.802687+0000","last_became_peered":"2023-07-18T03:04:59.802687+0000","last_unstale":"2023-07-18T03:06:52.889059+0000","last_undegraded":"2023-07-18T03:06:52.889059+0000","last_fullsized":"2023-07-18T03:06:52.889059+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:59:27.108157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.259440+0000","last_change":"2023-07-18T03:05:03.872690+0000","last_active":"2023-07-18T03:06:24.259440+0000","last_peered":"2023-07-18T03:06:24.259440+0000","last_clean":"2023-07-18T03:06:24.259440+0000","last_became_active":"2023-07-18T03:05:03.872187+0000","last_became_peered":"2023-07-18T03:05:03.872187+0000","last_unstale":"2023-07-18T03:06:24.259440+0000","last_undegraded":"2023-07-18T03:06:24.259440+0000","last_fullsized":"2023-07-18T03:06:24.259440+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:06:57.712294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.261033+0000","last_change":"2023-07-18T03:04:58.842877+0000","last_active":"2023-07-18T03:06:24.261033+0000","last_peered":"2023-07-18T03:06:24.261033+0000","last_clean":"2023-07-18T03:06:24.261033+0000","last_became_active":"2023-07-18T03:04:58.842516+0000","last_became_peered":"2023-07-18T03:04:58.842516+0000","last_unstale":"2023-07-18T03:06:24.261033+0000","last_undegraded":"2023-07-18T03:06:24.261033+0000","last_fullsized":"2023-07-18T03:06:24.261033+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:24:49.503369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.884379+0000","last_change":"2023-07-18T03:04:59.807592+0000","last_active":"2023-07-18T03:06:52.884379+0000","last_peered":"2023-07-18T03:06:52.884379+0000","last_clean":"2023-07-18T03:06:52.884379+0000","last_became_active":"2023-07-18T03:04:59.807276+0000","last_became_peered":"2023-07-18T03:04:59.807276+0000","last_unstale":"2023-07-18T03:06:52.884379+0000","last_undegraded":"2023-07-18T03:06:52.884379+0000","last_fullsized":"2023-07-18T03:06:52.884379+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:16:05.749019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.374960+0000","last_change":"2023-07-18T03:05:03.876355+0000","last_active":"2023-07-18T03:06:24.374960+0000","last_peered":"2023-07-18T03:06:24.374960+0000","last_clean":"2023-07-18T03:06:24.374960+0000","last_became_active":"2023-07-18T03:05:03.875845+0000","last_became_peered":"2023-07-18T03:05:03.875845+0000","last_unstale":"2023-07-18T03:06:24.374960+0000","last_undegraded":"2023-07-18T03:06:24.374960+0000","last_fullsized":"2023-07-18T03:06:24.374960+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:13:06.076832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'4","reported_seq":139,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:07:15.146806+0000","last_change":"2023-07-18T03:04:58.840389+0000","last_active":"2023-07-18T03:07:15.146806+0000","last_peered":"2023-07-18T03:07:15.146806+0000","last_clean":"2023-07-18T03:07:15.146806+0000","last_became_active":"2023-07-18T03:04:58.839974+0000","last_became_peered":"2023-07-18T03:04:58.839974+0000","last_unstale":"2023-07-18T03:07:15.146806+0000","last_undegraded":"2023-07-18T03:07:15.146806+0000","last_fullsized":"2023-07-18T03:07:15.146806+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:40:06.944095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.891931+0000","last_change":"2023-07-18T03:04:59.807681+0000","last_active":"2023-07-18T03:06:52.891931+0000","last_peered":"2023-07-18T03:06:52.891931+0000","last_clean":"2023-07-18T03:06:52.891931+0000","last_became_active":"2023-07-18T03:04:59.807365+0000","last_became_peered":"2023-07-18T03:04:59.807365+0000","last_unstale":"2023-07-18T03:06:52.891931+0000","last_undegraded":"2023-07-18T03:06:52.891931+0000","last_fullsized":"2023-07-18T03:06:52.891931+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:02:22.715114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.258920+0000","last_change":"2023-07-18T03:05:24.091413+0000","last_active":"2023-07-18T03:06:24.258920+0000","last_peered":"2023-07-18T03:06:24.258920+0000","last_clean":"2023-07-18T03:06:24.258920+0000","last_became_active":"2023-07-18T03:05:24.090815+0000","last_became_peered":"2023-07-18T03:05:24.090815+0000","last_unstale":"2023-07-18T03:06:24.258920+0000","last_undegraded":"2023-07-18T03:06:24.258920+0000","last_fullsized":"2023-07-18T03:06:24.258920+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:25:43.324509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.268967+0000","last_change":"2023-07-18T03:06:24.268967+0000","last_active":"2023-07-18T03:06:24.268967+0000","last_peered":"2023-07-18T03:06:24.268967+0000","last_clean":"2023-07-18T03:06:24.268967+0000","last_became_active":"2023-07-18T03:06:24.263424+0000","last_became_peered":"2023-07-18T03:06:24.263424+0000","last_unstale":"2023-07-18T03:06:24.268967+0000","last_undegraded":"2023-07-18T03:06:24.268967+0000","last_fullsized":"2023-07-18T03:06:24.268967+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:13:59.331501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.895054+0000","last_change":"2023-07-18T03:05:24.092870+0000","last_active":"2023-07-18T03:06:52.895054+0000","last_peered":"2023-07-18T03:06:52.895054+0000","last_clean":"2023-07-18T03:06:52.895054+0000","last_became_active":"2023-07-18T03:05:24.092474+0000","last_became_peered":"2023-07-18T03:05:24.092474+0000","last_unstale":"2023-07-18T03:06:52.895054+0000","last_undegraded":"2023-07-18T03:06:52.895054+0000","last_fullsized":"2023-07-18T03:06:52.895054+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:25:17.349689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255481+0000","last_change":"2023-07-18T03:05:03.874444+0000","last_active":"2023-07-18T03:06:24.255481+0000","last_peered":"2023-07-18T03:06:24.255481+0000","last_clean":"2023-07-18T03:06:24.255481+0000","last_became_active":"2023-07-18T03:05:03.873737+0000","last_became_peered":"2023-07-18T03:05:03.873737+0000","last_unstale":"2023-07-18T03:06:24.255481+0000","last_undegraded":"2023-07-18T03:06:24.255481+0000","last_fullsized":"2023-07-18T03:06:24.255481+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:07:21.954472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.261721+0000","last_change":"2023-07-18T03:04:58.845207+0000","last_active":"2023-07-18T03:06:24.261721+0000","last_peered":"2023-07-18T03:06:24.261721+0000","last_clean":"2023-07-18T03:06:24.261721+0000","last_became_active":"2023-07-18T03:04:58.844895+0000","last_became_peered":"2023-07-18T03:04:58.844895+0000","last_unstale":"2023-07-18T03:06:24.261721+0000","last_undegraded":"2023-07-18T03:06:24.261721+0000","last_fullsized":"2023-07-18T03:06:24.261721+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:59:56.250544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.886957+0000","last_change":"2023-07-18T03:04:59.807075+0000","last_active":"2023-07-18T03:06:52.886957+0000","last_peered":"2023-07-18T03:06:52.886957+0000","last_clean":"2023-07-18T03:06:52.886957+0000","last_became_active":"2023-07-18T03:04:59.806345+0000","last_became_peered":"2023-07-18T03:04:59.806345+0000","last_unstale":"2023-07-18T03:06:52.886957+0000","last_undegraded":"2023-07-18T03:06:52.886957+0000","last_fullsized":"2023-07-18T03:06:52.886957+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:50:55.760441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.375376+0000","last_change":"2023-07-18T03:05:24.091132+0000","last_active":"2023-07-18T03:06:24.375376+0000","last_peered":"2023-07-18T03:06:24.375376+0000","last_clean":"2023-07-18T03:06:24.375376+0000","last_became_active":"2023-07-18T03:05:24.090653+0000","last_became_peered":"2023-07-18T03:05:24.090653+0000","last_unstale":"2023-07-18T03:06:24.375376+0000","last_undegraded":"2023-07-18T03:06:24.375376+0000","last_fullsized":"2023-07-18T03:06:24.375376+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:14:06.294348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"91'38","reported_seq":289,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:07:16.164902+0000","last_change":"2023-07-18T03:04:58.844865+0000","last_active":"2023-07-18T03:07:16.164902+0000","last_peered":"2023-07-18T03:07:16.164902+0000","last_clean":"2023-07-18T03:07:16.164902+0000","last_became_active":"2023-07-18T03:04:58.844547+0000","last_became_peered":"2023-07-18T03:04:58.844547+0000","last_unstale":"2023-07-18T03:07:16.164902+0000","last_undegraded":"2023-07-18T03:07:16.164902+0000","last_fullsized":"2023-07-18T03:07:16.164902+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:09:40.602341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66781,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":221,"num_read_kb":176,"num_write":39,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.882102+0000","last_change":"2023-07-18T03:04:59.803024+0000","last_active":"2023-07-18T03:06:52.882102+0000","last_peered":"2023-07-18T03:06:52.882102+0000","last_clean":"2023-07-18T03:06:52.882102+0000","last_became_active":"2023-07-18T03:04:59.802705+0000","last_became_peered":"2023-07-18T03:04:59.802705+0000","last_unstale":"2023-07-18T03:06:52.882102+0000","last_undegraded":"2023-07-18T03:06:52.882102+0000","last_fullsized":"2023-07-18T03:06:52.882102+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:46:31.950692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.262478+0000","last_change":"2023-07-18T03:05:03.876009+0000","last_active":"2023-07-18T03:06:24.262478+0000","last_peered":"2023-07-18T03:06:24.262478+0000","last_clean":"2023-07-18T03:06:24.262478+0000","last_became_active":"2023-07-18T03:05:03.875328+0000","last_became_peered":"2023-07-18T03:05:03.875328+0000","last_unstale":"2023-07-18T03:06:24.262478+0000","last_undegraded":"2023-07-18T03:06:24.262478+0000","last_fullsized":"2023-07-18T03:06:24.262478+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:55:54.930973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.256224+0000","last_change":"2023-07-18T03:04:58.846573+0000","last_active":"2023-07-18T03:06:24.256224+0000","last_peered":"2023-07-18T03:06:24.256224+0000","last_clean":"2023-07-18T03:06:24.256224+0000","last_became_active":"2023-07-18T03:04:58.846271+0000","last_became_peered":"2023-07-18T03:04:58.846271+0000","last_unstale":"2023-07-18T03:06:24.256224+0000","last_undegraded":"2023-07-18T03:06:24.256224+0000","last_fullsized":"2023-07-18T03:06:24.256224+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:22:31.518590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.894308+0000","last_change":"2023-07-18T03:04:59.801552+0000","last_active":"2023-07-18T03:06:52.894308+0000","last_peered":"2023-07-18T03:06:52.894308+0000","last_clean":"2023-07-18T03:06:52.894308+0000","last_became_active":"2023-07-18T03:04:59.801224+0000","last_became_peered":"2023-07-18T03:04:59.801224+0000","last_unstale":"2023-07-18T03:06:52.894308+0000","last_undegraded":"2023-07-18T03:06:52.894308+0000","last_fullsized":"2023-07-18T03:06:52.894308+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:44:10.394026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.257271+0000","last_change":"2023-07-18T03:05:03.881125+0000","last_active":"2023-07-18T03:06:24.257271+0000","last_peered":"2023-07-18T03:06:24.257271+0000","last_clean":"2023-07-18T03:06:24.257271+0000","last_became_active":"2023-07-18T03:05:03.880723+0000","last_became_peered":"2023-07-18T03:05:03.880723+0000","last_unstale":"2023-07-18T03:06:24.257271+0000","last_undegraded":"2023-07-18T03:06:24.257271+0000","last_fullsized":"2023-07-18T03:06:24.257271+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:28:40.857218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.261012+0000","last_change":"2023-07-18T03:04:58.847118+0000","last_active":"2023-07-18T03:06:24.261012+0000","last_peered":"2023-07-18T03:06:24.261012+0000","last_clean":"2023-07-18T03:06:24.261012+0000","last_became_active":"2023-07-18T03:04:58.846805+0000","last_became_peered":"2023-07-18T03:04:58.846805+0000","last_unstale":"2023-07-18T03:06:24.261012+0000","last_undegraded":"2023-07-18T03:06:24.261012+0000","last_fullsized":"2023-07-18T03:06:24.261012+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:48:50.417535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.895015+0000","last_change":"2023-07-18T03:04:59.803914+0000","last_active":"2023-07-18T03:06:52.895015+0000","last_peered":"2023-07-18T03:06:52.895015+0000","last_clean":"2023-07-18T03:06:52.895015+0000","last_became_active":"2023-07-18T03:04:59.803589+0000","last_became_peered":"2023-07-18T03:04:59.803589+0000","last_unstale":"2023-07-18T03:06:52.895015+0000","last_undegraded":"2023-07-18T03:06:52.895015+0000","last_fullsized":"2023-07-18T03:06:52.895015+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:24:46.537710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255012+0000","last_change":"2023-07-18T03:05:03.877787+0000","last_active":"2023-07-18T03:06:24.255012+0000","last_peered":"2023-07-18T03:06:24.255012+0000","last_clean":"2023-07-18T03:06:24.255012+0000","last_became_active":"2023-07-18T03:05:03.877449+0000","last_became_peered":"2023-07-18T03:05:03.877449+0000","last_unstale":"2023-07-18T03:06:24.255012+0000","last_undegraded":"2023-07-18T03:06:24.255012+0000","last_fullsized":"2023-07-18T03:06:24.255012+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:54:47.075087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":137,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:07:12.144954+0000","last_change":"2023-07-18T03:04:58.844252+0000","last_active":"2023-07-18T03:07:12.144954+0000","last_peered":"2023-07-18T03:07:12.144954+0000","last_clean":"2023-07-18T03:07:12.144954+0000","last_became_active":"2023-07-18T03:04:58.843798+0000","last_became_peered":"2023-07-18T03:04:58.843798+0000","last_unstale":"2023-07-18T03:07:12.144954+0000","last_undegraded":"2023-07-18T03:07:12.144954+0000","last_fullsized":"2023-07-18T03:07:12.144954+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:24:14.970709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.883399+0000","last_change":"2023-07-18T03:04:59.806274+0000","last_active":"2023-07-18T03:06:52.883399+0000","last_peered":"2023-07-18T03:06:52.883399+0000","last_clean":"2023-07-18T03:06:52.883399+0000","last_became_active":"2023-07-18T03:04:59.805876+0000","last_became_peered":"2023-07-18T03:04:59.805876+0000","last_unstale":"2023-07-18T03:06:52.883399+0000","last_undegraded":"2023-07-18T03:06:52.883399+0000","last_fullsized":"2023-07-18T03:06:52.883399+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:00:53.965603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.751210+0000","last_change":"2023-07-18T03:05:03.876783+0000","last_active":"2023-07-18T03:06:24.751210+0000","last_peered":"2023-07-18T03:06:24.751210+0000","last_clean":"2023-07-18T03:06:24.751210+0000","last_became_active":"2023-07-18T03:05:03.876385+0000","last_became_peered":"2023-07-18T03:05:03.876385+0000","last_unstale":"2023-07-18T03:06:24.751210+0000","last_undegraded":"2023-07-18T03:06:24.751210+0000","last_fullsized":"2023-07-18T03:06:24.751210+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:12:22.516248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.259648+0000","last_change":"2023-07-18T03:04:58.851107+0000","last_active":"2023-07-18T03:06:24.259648+0000","last_peered":"2023-07-18T03:06:24.259648+0000","last_clean":"2023-07-18T03:06:24.259648+0000","last_became_active":"2023-07-18T03:04:58.850848+0000","last_became_peered":"2023-07-18T03:04:58.850848+0000","last_unstale":"2023-07-18T03:06:24.259648+0000","last_undegraded":"2023-07-18T03:06:24.259648+0000","last_fullsized":"2023-07-18T03:06:24.259648+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:39:44.591519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.885106+0000","last_change":"2023-07-18T03:04:59.803683+0000","last_active":"2023-07-18T03:06:52.885106+0000","last_peered":"2023-07-18T03:06:52.885106+0000","last_clean":"2023-07-18T03:06:52.885106+0000","last_became_active":"2023-07-18T03:04:59.803365+0000","last_became_peered":"2023-07-18T03:04:59.803365+0000","last_unstale":"2023-07-18T03:06:52.885106+0000","last_undegraded":"2023-07-18T03:06:52.885106+0000","last_fullsized":"2023-07-18T03:06:52.885106+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:30:10.236288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.260879+0000","last_change":"2023-07-18T03:05:03.875856+0000","last_active":"2023-07-18T03:06:24.260879+0000","last_peered":"2023-07-18T03:06:24.260879+0000","last_clean":"2023-07-18T03:06:24.260879+0000","last_became_active":"2023-07-18T03:05:03.875426+0000","last_became_peered":"2023-07-18T03:05:03.875426+0000","last_unstale":"2023-07-18T03:06:24.260879+0000","last_undegraded":"2023-07-18T03:06:24.260879+0000","last_fullsized":"2023-07-18T03:06:24.260879+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:48:39.948769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.261226+0000","last_change":"2023-07-18T03:04:58.844102+0000","last_active":"2023-07-18T03:06:24.261226+0000","last_peered":"2023-07-18T03:06:24.261226+0000","last_clean":"2023-07-18T03:06:24.261226+0000","last_became_active":"2023-07-18T03:04:58.843740+0000","last_became_peered":"2023-07-18T03:04:58.843740+0000","last_unstale":"2023-07-18T03:06:24.261226+0000","last_undegraded":"2023-07-18T03:06:24.261226+0000","last_fullsized":"2023-07-18T03:06:24.261226+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:19:46.540756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.883036+0000","last_change":"2023-07-18T03:05:24.091161+0000","last_active":"2023-07-18T03:06:52.883036+0000","last_peered":"2023-07-18T03:06:52.883036+0000","last_clean":"2023-07-18T03:06:52.883036+0000","last_became_active":"2023-07-18T03:05:24.090631+0000","last_became_peered":"2023-07-18T03:05:24.090631+0000","last_unstale":"2023-07-18T03:06:52.883036+0000","last_undegraded":"2023-07-18T03:06:52.883036+0000","last_fullsized":"2023-07-18T03:06:52.883036+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:21:10.676261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.259807+0000","last_change":"2023-07-18T03:05:24.092328+0000","last_active":"2023-07-18T03:06:24.259807+0000","last_peered":"2023-07-18T03:06:24.259807+0000","last_clean":"2023-07-18T03:06:24.259807+0000","last_became_active":"2023-07-18T03:05:24.091839+0000","last_became_peered":"2023-07-18T03:05:24.091839+0000","last_unstale":"2023-07-18T03:06:24.259807+0000","last_undegraded":"2023-07-18T03:06:24.259807+0000","last_fullsized":"2023-07-18T03:06:24.259807+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:42:18.354994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.262294+0000","last_change":"2023-07-18T03:06:24.261616+0000","last_active":"2023-07-18T03:06:24.262294+0000","last_peered":"2023-07-18T03:06:24.262294+0000","last_clean":"2023-07-18T03:06:24.262294+0000","last_became_active":"2023-07-18T03:06:24.261250+0000","last_became_peered":"2023-07-18T03:06:24.261250+0000","last_unstale":"2023-07-18T03:06:24.262294+0000","last_undegraded":"2023-07-18T03:06:24.262294+0000","last_fullsized":"2023-07-18T03:06:24.262294+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:19:24.589920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.893730+0000","last_change":"2023-07-18T03:05:24.092486+0000","last_active":"2023-07-18T03:06:52.893730+0000","last_peered":"2023-07-18T03:06:52.893730+0000","last_clean":"2023-07-18T03:06:52.893730+0000","last_became_active":"2023-07-18T03:05:24.092189+0000","last_became_peered":"2023-07-18T03:05:24.092189+0000","last_unstale":"2023-07-18T03:06:52.893730+0000","last_undegraded":"2023-07-18T03:06:52.893730+0000","last_fullsized":"2023-07-18T03:06:52.893730+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:09:16.496072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.751014+0000","last_change":"2023-07-18T03:05:03.868851+0000","last_active":"2023-07-18T03:06:24.751014+0000","last_peered":"2023-07-18T03:06:24.751014+0000","last_clean":"2023-07-18T03:06:24.751014+0000","last_became_active":"2023-07-18T03:05:03.868433+0000","last_became_peered":"2023-07-18T03:05:03.868433+0000","last_unstale":"2023-07-18T03:06:24.751014+0000","last_undegraded":"2023-07-18T03:06:24.751014+0000","last_fullsized":"2023-07-18T03:06:24.751014+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:36:46.180540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.258040+0000","last_change":"2023-07-18T03:04:58.848059+0000","last_active":"2023-07-18T03:06:24.258040+0000","last_peered":"2023-07-18T03:06:24.258040+0000","last_clean":"2023-07-18T03:06:24.258040+0000","last_became_active":"2023-07-18T03:04:58.847700+0000","last_became_peered":"2023-07-18T03:04:58.847700+0000","last_unstale":"2023-07-18T03:06:24.258040+0000","last_undegraded":"2023-07-18T03:06:24.258040+0000","last_fullsized":"2023-07-18T03:06:24.258040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:22:42.461972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.886520+0000","last_change":"2023-07-18T03:05:23.939939+0000","last_active":"2023-07-18T03:06:52.886520+0000","last_peered":"2023-07-18T03:06:52.886520+0000","last_clean":"2023-07-18T03:06:52.886520+0000","last_became_active":"2023-07-18T03:05:23.939607+0000","last_became_peered":"2023-07-18T03:05:23.939607+0000","last_unstale":"2023-07-18T03:06:52.886520+0000","last_undegraded":"2023-07-18T03:06:52.886520+0000","last_fullsized":"2023-07-18T03:06:52.886520+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:47:48.796749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.751185+0000","last_change":"2023-07-18T03:05:03.886724+0000","last_active":"2023-07-18T03:06:24.751185+0000","last_peered":"2023-07-18T03:06:24.751185+0000","last_clean":"2023-07-18T03:06:24.751185+0000","last_became_active":"2023-07-18T03:05:03.886172+0000","last_became_peered":"2023-07-18T03:05:03.886172+0000","last_unstale":"2023-07-18T03:06:24.751185+0000","last_undegraded":"2023-07-18T03:06:24.751185+0000","last_fullsized":"2023-07-18T03:06:24.751185+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:35:26.182278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.886032+0000","last_change":"2023-07-18T03:04:59.804273+0000","last_active":"2023-07-18T03:06:52.886032+0000","last_peered":"2023-07-18T03:06:52.886032+0000","last_clean":"2023-07-18T03:06:52.886032+0000","last_became_active":"2023-07-18T03:04:59.803987+0000","last_became_peered":"2023-07-18T03:04:59.803987+0000","last_unstale":"2023-07-18T03:06:52.886032+0000","last_undegraded":"2023-07-18T03:06:52.886032+0000","last_fullsized":"2023-07-18T03:06:52.886032+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:53:36.335539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":139,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:07:14.146943+0000","last_change":"2023-07-18T03:04:58.846381+0000","last_active":"2023-07-18T03:07:14.146943+0000","last_peered":"2023-07-18T03:07:14.146943+0000","last_clean":"2023-07-18T03:07:14.146943+0000","last_became_active":"2023-07-18T03:04:58.845757+0000","last_became_peered":"2023-07-18T03:04:58.845757+0000","last_unstale":"2023-07-18T03:07:14.146943+0000","last_undegraded":"2023-07-18T03:07:14.146943+0000","last_fullsized":"2023-07-18T03:07:14.146943+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:00:14.653137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.258055+0000","last_change":"2023-07-18T03:05:03.873626+0000","last_active":"2023-07-18T03:06:24.258055+0000","last_peered":"2023-07-18T03:06:24.258055+0000","last_clean":"2023-07-18T03:06:24.258055+0000","last_became_active":"2023-07-18T03:05:03.872965+0000","last_became_peered":"2023-07-18T03:05:03.872965+0000","last_unstale":"2023-07-18T03:06:24.258055+0000","last_undegraded":"2023-07-18T03:06:24.258055+0000","last_fullsized":"2023-07-18T03:06:24.258055+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:37:38.450618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.881350+0000","last_change":"2023-07-18T03:04:59.801153+0000","last_active":"2023-07-18T03:06:52.881350+0000","last_peered":"2023-07-18T03:06:52.881350+0000","last_clean":"2023-07-18T03:06:52.881350+0000","last_became_active":"2023-07-18T03:04:59.800798+0000","last_became_peered":"2023-07-18T03:04:59.800798+0000","last_unstale":"2023-07-18T03:06:52.881350+0000","last_undegraded":"2023-07-18T03:06:52.881350+0000","last_fullsized":"2023-07-18T03:06:52.881350+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:49:57.147042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255477+0000","last_change":"2023-07-18T03:04:58.846047+0000","last_active":"2023-07-18T03:06:24.255477+0000","last_peered":"2023-07-18T03:06:24.255477+0000","last_clean":"2023-07-18T03:06:24.255477+0000","last_became_active":"2023-07-18T03:04:58.845678+0000","last_became_peered":"2023-07-18T03:04:58.845678+0000","last_unstale":"2023-07-18T03:06:24.255477+0000","last_undegraded":"2023-07-18T03:06:24.255477+0000","last_fullsized":"2023-07-18T03:06:24.255477+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:50:39.012582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":385,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.257405+0000","last_change":"2023-07-18T03:03:54.168837+0000","last_active":"2023-07-18T03:06:24.257405+0000","last_peered":"2023-07-18T03:06:24.257405+0000","last_clean":"2023-07-18T03:06:24.257405+0000","last_became_active":"2023-07-18T03:03:54.168298+0000","last_became_peered":"2023-07-18T03:03:54.168298+0000","last_unstale":"2023-07-18T03:06:24.257405+0000","last_undegraded":"2023-07-18T03:06:24.257405+0000","last_fullsized":"2023-07-18T03:06:24.257405+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:00:43.469168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:00:43.469168+0000","last_clean_scrub_stamp":"2023-07-18T03:00:43.469168+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:03:45.014912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.261432+0000","last_change":"2023-07-18T03:05:03.886611+0000","last_active":"2023-07-18T03:06:24.261432+0000","last_peered":"2023-07-18T03:06:24.261432+0000","last_clean":"2023-07-18T03:06:24.261432+0000","last_became_active":"2023-07-18T03:05:03.886099+0000","last_became_peered":"2023-07-18T03:05:03.886099+0000","last_unstale":"2023-07-18T03:06:24.261432+0000","last_undegraded":"2023-07-18T03:06:24.261432+0000","last_fullsized":"2023-07-18T03:06:24.261432+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:26:40.808017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.893733+0000","last_change":"2023-07-18T03:04:59.804478+0000","last_active":"2023-07-18T03:06:52.893733+0000","last_peered":"2023-07-18T03:06:52.893733+0000","last_clean":"2023-07-18T03:06:52.893733+0000","last_became_active":"2023-07-18T03:04:59.804162+0000","last_became_peered":"2023-07-18T03:04:59.804162+0000","last_unstale":"2023-07-18T03:06:52.893733+0000","last_undegraded":"2023-07-18T03:06:52.893733+0000","last_fullsized":"2023-07-18T03:06:52.893733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:41:57.294071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.267166+0000","last_change":"2023-07-18T03:06:24.266708+0000","last_active":"2023-07-18T03:06:24.267166+0000","last_peered":"2023-07-18T03:06:24.267166+0000","last_clean":"2023-07-18T03:06:24.267166+0000","last_became_active":"2023-07-18T03:06:24.262606+0000","last_became_peered":"2023-07-18T03:06:24.262606+0000","last_unstale":"2023-07-18T03:06:24.267166+0000","last_undegraded":"2023-07-18T03:06:24.267166+0000","last_fullsized":"2023-07-18T03:06:24.267166+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:20:59.875839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.257365+0000","last_change":"2023-07-18T03:05:03.872788+0000","last_active":"2023-07-18T03:06:24.257365+0000","last_peered":"2023-07-18T03:06:24.257365+0000","last_clean":"2023-07-18T03:06:24.257365+0000","last_became_active":"2023-07-18T03:05:03.872343+0000","last_became_peered":"2023-07-18T03:05:03.872343+0000","last_unstale":"2023-07-18T03:06:24.257365+0000","last_undegraded":"2023-07-18T03:06:24.257365+0000","last_fullsized":"2023-07-18T03:06:24.257365+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:20:43.912026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.266064+0000","last_change":"2023-07-18T03:06:24.265727+0000","last_active":"2023-07-18T03:06:24.266064+0000","last_peered":"2023-07-18T03:06:24.266064+0000","last_clean":"2023-07-18T03:06:24.266064+0000","last_became_active":"2023-07-18T03:06:24.262373+0000","last_became_peered":"2023-07-18T03:06:24.262373+0000","last_unstale":"2023-07-18T03:06:24.266064+0000","last_undegraded":"2023-07-18T03:06:24.266064+0000","last_fullsized":"2023-07-18T03:06:24.266064+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:38:45.610472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.893085+0000","last_change":"2023-07-18T03:04:59.804706+0000","last_active":"2023-07-18T03:06:52.893085+0000","last_peered":"2023-07-18T03:06:52.893085+0000","last_clean":"2023-07-18T03:06:52.893085+0000","last_became_active":"2023-07-18T03:04:59.804445+0000","last_became_peered":"2023-07-18T03:04:59.804445+0000","last_unstale":"2023-07-18T03:06:52.893085+0000","last_undegraded":"2023-07-18T03:06:52.893085+0000","last_fullsized":"2023-07-18T03:06:52.893085+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:54:52.824231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.374912+0000","last_change":"2023-07-18T03:05:03.886065+0000","last_active":"2023-07-18T03:06:24.374912+0000","last_peered":"2023-07-18T03:06:24.374912+0000","last_clean":"2023-07-18T03:06:24.374912+0000","last_became_active":"2023-07-18T03:05:03.885625+0000","last_became_peered":"2023-07-18T03:05:03.885625+0000","last_unstale":"2023-07-18T03:06:24.374912+0000","last_undegraded":"2023-07-18T03:06:24.374912+0000","last_fullsized":"2023-07-18T03:06:24.374912+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:32:31.156943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:07:10.696055+0000","last_change":"2023-07-18T03:04:58.849246+0000","last_active":"2023-07-18T03:07:10.696055+0000","last_peered":"2023-07-18T03:07:10.696055+0000","last_clean":"2023-07-18T03:07:10.696055+0000","last_became_active":"2023-07-18T03:04:58.848888+0000","last_became_peered":"2023-07-18T03:04:58.848888+0000","last_unstale":"2023-07-18T03:07:10.696055+0000","last_undegraded":"2023-07-18T03:07:10.696055+0000","last_fullsized":"2023-07-18T03:07:10.696055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:47:14.002970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.883435+0000","last_change":"2023-07-18T03:04:59.804561+0000","last_active":"2023-07-18T03:06:52.883435+0000","last_peered":"2023-07-18T03:06:52.883435+0000","last_clean":"2023-07-18T03:06:52.883435+0000","last_became_active":"2023-07-18T03:04:59.803659+0000","last_became_peered":"2023-07-18T03:04:59.803659+0000","last_unstale":"2023-07-18T03:06:52.883435+0000","last_undegraded":"2023-07-18T03:06:52.883435+0000","last_fullsized":"2023-07-18T03:06:52.883435+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:55:58.968273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255507+0000","last_change":"2023-07-18T03:05:03.871418+0000","last_active":"2023-07-18T03:06:24.255507+0000","last_peered":"2023-07-18T03:06:24.255507+0000","last_clean":"2023-07-18T03:06:24.255507+0000","last_became_active":"2023-07-18T03:05:03.871054+0000","last_became_peered":"2023-07-18T03:05:03.871054+0000","last_unstale":"2023-07-18T03:06:24.255507+0000","last_undegraded":"2023-07-18T03:06:24.255507+0000","last_fullsized":"2023-07-18T03:06:24.255507+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:59:35.196165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255933+0000","last_change":"2023-07-18T03:04:58.850505+0000","last_active":"2023-07-18T03:06:24.255933+0000","last_peered":"2023-07-18T03:06:24.255933+0000","last_clean":"2023-07-18T03:06:24.255933+0000","last_became_active":"2023-07-18T03:04:58.850234+0000","last_became_peered":"2023-07-18T03:04:58.850234+0000","last_unstale":"2023-07-18T03:06:24.255933+0000","last_undegraded":"2023-07-18T03:06:24.255933+0000","last_fullsized":"2023-07-18T03:06:24.255933+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:36:14.805265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.886643+0000","last_change":"2023-07-18T03:04:59.807348+0000","last_active":"2023-07-18T03:06:52.886643+0000","last_peered":"2023-07-18T03:06:52.886643+0000","last_clean":"2023-07-18T03:06:52.886643+0000","last_became_active":"2023-07-18T03:04:59.807013+0000","last_became_peered":"2023-07-18T03:04:59.807013+0000","last_unstale":"2023-07-18T03:06:52.886643+0000","last_undegraded":"2023-07-18T03:06:52.886643+0000","last_fullsized":"2023-07-18T03:06:52.886643+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:11:05.544642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255623+0000","last_change":"2023-07-18T03:05:03.886773+0000","last_active":"2023-07-18T03:06:24.255623+0000","last_peered":"2023-07-18T03:06:24.255623+0000","last_clean":"2023-07-18T03:06:24.255623+0000","last_became_active":"2023-07-18T03:05:03.886259+0000","last_became_peered":"2023-07-18T03:05:03.886259+0000","last_unstale":"2023-07-18T03:06:24.255623+0000","last_undegraded":"2023-07-18T03:06:24.255623+0000","last_fullsized":"2023-07-18T03:06:24.255623+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:12:05.068449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.261480+0000","last_change":"2023-07-18T03:04:58.855746+0000","last_active":"2023-07-18T03:06:24.261480+0000","last_peered":"2023-07-18T03:06:24.261480+0000","last_clean":"2023-07-18T03:06:24.261480+0000","last_became_active":"2023-07-18T03:04:58.855366+0000","last_became_peered":"2023-07-18T03:04:58.855366+0000","last_unstale":"2023-07-18T03:06:24.261480+0000","last_undegraded":"2023-07-18T03:06:24.261480+0000","last_fullsized":"2023-07-18T03:06:24.261480+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:04:33.740458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.879818+0000","last_change":"2023-07-18T03:04:59.800321+0000","last_active":"2023-07-18T03:06:52.879818+0000","last_peered":"2023-07-18T03:06:52.879818+0000","last_clean":"2023-07-18T03:06:52.879818+0000","last_became_active":"2023-07-18T03:04:59.800018+0000","last_became_peered":"2023-07-18T03:04:59.800018+0000","last_unstale":"2023-07-18T03:06:52.879818+0000","last_undegraded":"2023-07-18T03:06:52.879818+0000","last_fullsized":"2023-07-18T03:06:52.879818+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:13:00.845492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255150+0000","last_change":"2023-07-18T03:05:03.877541+0000","last_active":"2023-07-18T03:06:24.255150+0000","last_peered":"2023-07-18T03:06:24.255150+0000","last_clean":"2023-07-18T03:06:24.255150+0000","last_became_active":"2023-07-18T03:05:03.877164+0000","last_became_peered":"2023-07-18T03:05:03.877164+0000","last_unstale":"2023-07-18T03:06:24.255150+0000","last_undegraded":"2023-07-18T03:06:24.255150+0000","last_fullsized":"2023-07-18T03:06:24.255150+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:58:38.384423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.375090+0000","last_change":"2023-07-18T03:04:58.845643+0000","last_active":"2023-07-18T03:06:24.375090+0000","last_peered":"2023-07-18T03:06:24.375090+0000","last_clean":"2023-07-18T03:06:24.375090+0000","last_became_active":"2023-07-18T03:04:58.845254+0000","last_became_peered":"2023-07-18T03:04:58.845254+0000","last_unstale":"2023-07-18T03:06:24.375090+0000","last_undegraded":"2023-07-18T03:06:24.375090+0000","last_fullsized":"2023-07-18T03:06:24.375090+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:03:15.745367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.887203+0000","last_change":"2023-07-18T03:04:59.803679+0000","last_active":"2023-07-18T03:06:52.887203+0000","last_peered":"2023-07-18T03:06:52.887203+0000","last_clean":"2023-07-18T03:06:52.887203+0000","last_became_active":"2023-07-18T03:04:59.802841+0000","last_became_peered":"2023-07-18T03:04:59.802841+0000","last_unstale":"2023-07-18T03:06:52.887203+0000","last_undegraded":"2023-07-18T03:06:52.887203+0000","last_fullsized":"2023-07-18T03:06:52.887203+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:51:12.763232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255483+0000","last_change":"2023-07-18T03:05:03.874093+0000","last_active":"2023-07-18T03:06:24.255483+0000","last_peered":"2023-07-18T03:06:24.255483+0000","last_clean":"2023-07-18T03:06:24.255483+0000","last_became_active":"2023-07-18T03:05:03.873710+0000","last_became_peered":"2023-07-18T03:05:03.873710+0000","last_unstale":"2023-07-18T03:06:24.255483+0000","last_undegraded":"2023-07-18T03:06:24.255483+0000","last_fullsized":"2023-07-18T03:06:24.255483+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:42:17.007476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255671+0000","last_change":"2023-07-18T03:04:58.846686+0000","last_active":"2023-07-18T03:06:24.255671+0000","last_peered":"2023-07-18T03:06:24.255671+0000","last_clean":"2023-07-18T03:06:24.255671+0000","last_became_active":"2023-07-18T03:04:58.846371+0000","last_became_peered":"2023-07-18T03:04:58.846371+0000","last_unstale":"2023-07-18T03:06:24.255671+0000","last_undegraded":"2023-07-18T03:06:24.255671+0000","last_fullsized":"2023-07-18T03:06:24.255671+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:25:47.154236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.886475+0000","last_change":"2023-07-18T03:04:59.811917+0000","last_active":"2023-07-18T03:06:52.886475+0000","last_peered":"2023-07-18T03:06:52.886475+0000","last_clean":"2023-07-18T03:06:52.886475+0000","last_became_active":"2023-07-18T03:04:59.811551+0000","last_became_peered":"2023-07-18T03:04:59.811551+0000","last_unstale":"2023-07-18T03:06:52.886475+0000","last_undegraded":"2023-07-18T03:06:52.886475+0000","last_fullsized":"2023-07-18T03:06:52.886475+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:28:00.926747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255450+0000","last_change":"2023-07-18T03:05:03.886072+0000","last_active":"2023-07-18T03:06:24.255450+0000","last_peered":"2023-07-18T03:06:24.255450+0000","last_clean":"2023-07-18T03:06:24.255450+0000","last_became_active":"2023-07-18T03:05:03.885652+0000","last_became_peered":"2023-07-18T03:05:03.885652+0000","last_unstale":"2023-07-18T03:06:24.255450+0000","last_undegraded":"2023-07-18T03:06:24.255450+0000","last_fullsized":"2023-07-18T03:06:24.255450+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:37:43.631888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.259677+0000","last_change":"2023-07-18T03:04:58.849069+0000","last_active":"2023-07-18T03:06:24.259677+0000","last_peered":"2023-07-18T03:06:24.259677+0000","last_clean":"2023-07-18T03:06:24.259677+0000","last_became_active":"2023-07-18T03:04:58.848744+0000","last_became_peered":"2023-07-18T03:04:58.848744+0000","last_unstale":"2023-07-18T03:06:24.259677+0000","last_undegraded":"2023-07-18T03:06:24.259677+0000","last_fullsized":"2023-07-18T03:06:24.259677+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:19:13.922970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.895462+0000","last_change":"2023-07-18T03:04:59.806976+0000","last_active":"2023-07-18T03:06:52.895462+0000","last_peered":"2023-07-18T03:06:52.895462+0000","last_clean":"2023-07-18T03:06:52.895462+0000","last_became_active":"2023-07-18T03:04:59.805750+0000","last_became_peered":"2023-07-18T03:04:59.805750+0000","last_unstale":"2023-07-18T03:06:52.895462+0000","last_undegraded":"2023-07-18T03:06:52.895462+0000","last_fullsized":"2023-07-18T03:06:52.895462+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:59:17.717720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.261590+0000","last_change":"2023-07-18T03:05:03.871210+0000","last_active":"2023-07-18T03:06:24.261590+0000","last_peered":"2023-07-18T03:06:24.261590+0000","last_clean":"2023-07-18T03:06:24.261590+0000","last_became_active":"2023-07-18T03:05:03.870826+0000","last_became_peered":"2023-07-18T03:05:03.870826+0000","last_unstale":"2023-07-18T03:06:24.261590+0000","last_undegraded":"2023-07-18T03:06:24.261590+0000","last_fullsized":"2023-07-18T03:06:24.261590+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:49:03.743543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.256155+0000","last_change":"2023-07-18T03:04:58.855673+0000","last_active":"2023-07-18T03:06:24.256155+0000","last_peered":"2023-07-18T03:06:24.256155+0000","last_clean":"2023-07-18T03:06:24.256155+0000","last_became_active":"2023-07-18T03:04:58.855338+0000","last_became_peered":"2023-07-18T03:04:58.855338+0000","last_unstale":"2023-07-18T03:06:24.256155+0000","last_undegraded":"2023-07-18T03:06:24.256155+0000","last_fullsized":"2023-07-18T03:06:24.256155+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:07:40.935595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.884252+0000","last_change":"2023-07-18T03:04:59.802488+0000","last_active":"2023-07-18T03:06:52.884252+0000","last_peered":"2023-07-18T03:06:52.884252+0000","last_clean":"2023-07-18T03:06:52.884252+0000","last_became_active":"2023-07-18T03:04:59.802184+0000","last_became_peered":"2023-07-18T03:04:59.802184+0000","last_unstale":"2023-07-18T03:06:52.884252+0000","last_undegraded":"2023-07-18T03:06:52.884252+0000","last_fullsized":"2023-07-18T03:06:52.884252+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:00:59.367787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.259218+0000","last_change":"2023-07-18T03:05:03.881585+0000","last_active":"2023-07-18T03:06:24.259218+0000","last_peered":"2023-07-18T03:06:24.259218+0000","last_clean":"2023-07-18T03:06:24.259218+0000","last_became_active":"2023-07-18T03:05:03.880917+0000","last_became_peered":"2023-07-18T03:05:03.880917+0000","last_unstale":"2023-07-18T03:06:24.259218+0000","last_undegraded":"2023-07-18T03:06:24.259218+0000","last_fullsized":"2023-07-18T03:06:24.259218+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:44:11.175469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.262020+0000","last_change":"2023-07-18T03:04:58.852387+0000","last_active":"2023-07-18T03:06:24.262020+0000","last_peered":"2023-07-18T03:06:24.262020+0000","last_clean":"2023-07-18T03:06:24.262020+0000","last_became_active":"2023-07-18T03:04:58.852113+0000","last_became_peered":"2023-07-18T03:04:58.852113+0000","last_unstale":"2023-07-18T03:06:24.262020+0000","last_undegraded":"2023-07-18T03:06:24.262020+0000","last_fullsized":"2023-07-18T03:06:24.262020+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:12:15.450770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.882718+0000","last_change":"2023-07-18T03:04:59.811569+0000","last_active":"2023-07-18T03:06:52.882718+0000","last_peered":"2023-07-18T03:06:52.882718+0000","last_clean":"2023-07-18T03:06:52.882718+0000","last_became_active":"2023-07-18T03:04:59.811229+0000","last_became_peered":"2023-07-18T03:04:59.811229+0000","last_unstale":"2023-07-18T03:06:52.882718+0000","last_undegraded":"2023-07-18T03:06:52.882718+0000","last_fullsized":"2023-07-18T03:06:52.882718+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:20:39.366577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.260963+0000","last_change":"2023-07-18T03:05:03.872530+0000","last_active":"2023-07-18T03:06:24.260963+0000","last_peered":"2023-07-18T03:06:24.260963+0000","last_clean":"2023-07-18T03:06:24.260963+0000","last_became_active":"2023-07-18T03:05:03.871607+0000","last_became_peered":"2023-07-18T03:05:03.871607+0000","last_unstale":"2023-07-18T03:06:24.260963+0000","last_undegraded":"2023-07-18T03:06:24.260963+0000","last_fullsized":"2023-07-18T03:06:24.260963+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:13:21.796882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.751209+0000","last_change":"2023-07-18T03:04:58.847933+0000","last_active":"2023-07-18T03:06:24.751209+0000","last_peered":"2023-07-18T03:06:24.751209+0000","last_clean":"2023-07-18T03:06:24.751209+0000","last_became_active":"2023-07-18T03:04:58.847630+0000","last_became_peered":"2023-07-18T03:04:58.847630+0000","last_unstale":"2023-07-18T03:06:24.751209+0000","last_undegraded":"2023-07-18T03:06:24.751209+0000","last_fullsized":"2023-07-18T03:06:24.751209+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:34:44.655423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.886617+0000","last_change":"2023-07-18T03:04:59.803981+0000","last_active":"2023-07-18T03:06:52.886617+0000","last_peered":"2023-07-18T03:06:52.886617+0000","last_clean":"2023-07-18T03:06:52.886617+0000","last_became_active":"2023-07-18T03:04:59.803692+0000","last_became_peered":"2023-07-18T03:04:59.803692+0000","last_unstale":"2023-07-18T03:06:52.886617+0000","last_undegraded":"2023-07-18T03:06:52.886617+0000","last_fullsized":"2023-07-18T03:06:52.886617+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:58:45.567557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.884681+0000","last_change":"2023-07-18T03:04:59.803770+0000","last_active":"2023-07-18T03:06:52.884681+0000","last_peered":"2023-07-18T03:06:52.884681+0000","last_clean":"2023-07-18T03:06:52.884681+0000","last_became_active":"2023-07-18T03:04:59.803355+0000","last_became_peered":"2023-07-18T03:04:59.803355+0000","last_unstale":"2023-07-18T03:06:52.884681+0000","last_undegraded":"2023-07-18T03:06:52.884681+0000","last_fullsized":"2023-07-18T03:06:52.884681+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:07:11.955948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.886539+0000","last_change":"2023-07-18T03:04:59.808946+0000","last_active":"2023-07-18T03:06:52.886539+0000","last_peered":"2023-07-18T03:06:52.886539+0000","last_clean":"2023-07-18T03:06:52.886539+0000","last_became_active":"2023-07-18T03:04:59.808646+0000","last_became_peered":"2023-07-18T03:04:59.808646+0000","last_unstale":"2023-07-18T03:06:52.886539+0000","last_undegraded":"2023-07-18T03:06:52.886539+0000","last_fullsized":"2023-07-18T03:06:52.886539+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:10:55.288876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.883434+0000","last_change":"2023-07-18T03:04:59.811933+0000","last_active":"2023-07-18T03:06:52.883434+0000","last_peered":"2023-07-18T03:06:52.883434+0000","last_clean":"2023-07-18T03:06:52.883434+0000","last_became_active":"2023-07-18T03:04:59.811620+0000","last_became_peered":"2023-07-18T03:04:59.811620+0000","last_unstale":"2023-07-18T03:06:52.883434+0000","last_undegraded":"2023-07-18T03:06:52.883434+0000","last_fullsized":"2023-07-18T03:06:52.883434+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:21:21.164245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.896076+0000","last_change":"2023-07-18T03:04:59.803942+0000","last_active":"2023-07-18T03:06:52.896076+0000","last_peered":"2023-07-18T03:06:52.896076+0000","last_clean":"2023-07-18T03:06:52.896076+0000","last_became_active":"2023-07-18T03:04:59.803576+0000","last_became_peered":"2023-07-18T03:04:59.803576+0000","last_unstale":"2023-07-18T03:06:52.896076+0000","last_undegraded":"2023-07-18T03:06:52.896076+0000","last_fullsized":"2023-07-18T03:06:52.896076+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:10:05.182871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.893685+0000","last_change":"2023-07-18T03:04:59.802594+0000","last_active":"2023-07-18T03:06:52.893685+0000","last_peered":"2023-07-18T03:06:52.893685+0000","last_clean":"2023-07-18T03:06:52.893685+0000","last_became_active":"2023-07-18T03:04:59.802215+0000","last_became_peered":"2023-07-18T03:04:59.802215+0000","last_unstale":"2023-07-18T03:06:52.893685+0000","last_undegraded":"2023-07-18T03:06:52.893685+0000","last_fullsized":"2023-07-18T03:06:52.893685+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:57:57.598480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.883798+0000","last_change":"2023-07-18T03:04:59.803519+0000","last_active":"2023-07-18T03:06:52.883798+0000","last_peered":"2023-07-18T03:06:52.883798+0000","last_clean":"2023-07-18T03:06:52.883798+0000","last_became_active":"2023-07-18T03:04:59.803170+0000","last_became_peered":"2023-07-18T03:04:59.803170+0000","last_unstale":"2023-07-18T03:06:52.883798+0000","last_undegraded":"2023-07-18T03:06:52.883798+0000","last_fullsized":"2023-07-18T03:06:52.883798+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:40:11.785249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.881716+0000","last_change":"2023-07-18T03:04:59.813494+0000","last_active":"2023-07-18T03:06:52.881716+0000","last_peered":"2023-07-18T03:06:52.881716+0000","last_clean":"2023-07-18T03:06:52.881716+0000","last_became_active":"2023-07-18T03:04:59.813175+0000","last_became_peered":"2023-07-18T03:04:59.813175+0000","last_unstale":"2023-07-18T03:06:52.881716+0000","last_undegraded":"2023-07-18T03:06:52.881716+0000","last_fullsized":"2023-07-18T03:06:52.881716+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:58:52.954498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.884923+0000","last_change":"2023-07-18T03:04:59.806597+0000","last_active":"2023-07-18T03:06:52.884923+0000","last_peered":"2023-07-18T03:06:52.884923+0000","last_clean":"2023-07-18T03:06:52.884923+0000","last_became_active":"2023-07-18T03:04:59.806220+0000","last_became_peered":"2023-07-18T03:04:59.806220+0000","last_unstale":"2023-07-18T03:06:52.884923+0000","last_undegraded":"2023-07-18T03:06:52.884923+0000","last_fullsized":"2023-07-18T03:06:52.884923+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:07:15.022728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.888455+0000","last_change":"2023-07-18T03:04:59.809051+0000","last_active":"2023-07-18T03:06:52.888455+0000","last_peered":"2023-07-18T03:06:52.888455+0000","last_clean":"2023-07-18T03:06:52.888455+0000","last_became_active":"2023-07-18T03:04:59.808783+0000","last_became_peered":"2023-07-18T03:04:59.808783+0000","last_unstale":"2023-07-18T03:06:52.888455+0000","last_undegraded":"2023-07-18T03:06:52.888455+0000","last_fullsized":"2023-07-18T03:06:52.888455+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:55:28.682728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.883966+0000","last_change":"2023-07-18T03:04:59.805741+0000","last_active":"2023-07-18T03:06:52.883966+0000","last_peered":"2023-07-18T03:06:52.883966+0000","last_clean":"2023-07-18T03:06:52.883966+0000","last_became_active":"2023-07-18T03:04:59.805412+0000","last_became_peered":"2023-07-18T03:04:59.805412+0000","last_unstale":"2023-07-18T03:06:52.883966+0000","last_undegraded":"2023-07-18T03:06:52.883966+0000","last_fullsized":"2023-07-18T03:06:52.883966+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:56:58.924751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.891407+0000","last_change":"2023-07-18T03:04:59.812052+0000","last_active":"2023-07-18T03:06:52.891407+0000","last_peered":"2023-07-18T03:06:52.891407+0000","last_clean":"2023-07-18T03:06:52.891407+0000","last_became_active":"2023-07-18T03:04:59.811756+0000","last_became_peered":"2023-07-18T03:04:59.811756+0000","last_unstale":"2023-07-18T03:06:52.891407+0000","last_undegraded":"2023-07-18T03:06:52.891407+0000","last_fullsized":"2023-07-18T03:06:52.891407+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:41:23.926360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.896400+0000","last_change":"2023-07-18T03:04:59.803009+0000","last_active":"2023-07-18T03:06:52.896400+0000","last_peered":"2023-07-18T03:06:52.896400+0000","last_clean":"2023-07-18T03:06:52.896400+0000","last_became_active":"2023-07-18T03:04:59.802687+0000","last_became_peered":"2023-07-18T03:04:59.802687+0000","last_unstale":"2023-07-18T03:06:52.896400+0000","last_undegraded":"2023-07-18T03:06:52.896400+0000","last_fullsized":"2023-07-18T03:06:52.896400+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:31:26.686903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.880496+0000","last_change":"2023-07-18T03:04:59.795562+0000","last_active":"2023-07-18T03:06:52.880496+0000","last_peered":"2023-07-18T03:06:52.880496+0000","last_clean":"2023-07-18T03:06:52.880496+0000","last_became_active":"2023-07-18T03:04:59.795241+0000","last_became_peered":"2023-07-18T03:04:59.795241+0000","last_unstale":"2023-07-18T03:06:52.880496+0000","last_undegraded":"2023-07-18T03:06:52.880496+0000","last_fullsized":"2023-07-18T03:06:52.880496+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:44:43.907380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281146,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70826,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":655,"num_read_kb":404,"num_write":166,"num_write_kb":124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":212478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449128,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":999424,"data_stored":507139,"data_compressed":23524,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.56000000000000005,"15min":0.55300000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.52200000000000002,"15min":0.51400000000000001},"min":{"1min":0.41099999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.69199999999999995}]},{"osd":1,"last update":"Tue Jul 18 03:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.64300000000000002,"15min":0.63500000000000001},"min":{"1min":0.47799999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.85299999999999998,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.624,"15min":0.621},"min":{"1min":0.47099999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.82299999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.76400000000000001}]},{"osd":2,"last update":"Tue Jul 18 03:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.48299999999999998,"15min":0.47399999999999998},"min":{"1min":0.35199999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.71399999999999997,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.48999999999999999,"15min":0.47599999999999998},"min":{"1min":0.36699999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.32500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":4,"last update":"Tue Jul 18 03:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.66500000000000004,"15min":0.65100000000000002},"min":{"1min":0.53200000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.64200000000000002,"15min":0.63500000000000001},"min":{"1min":0.48999999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.84799999999999998,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.84399999999999997}]},{"osd":5,"last update":"Tue Jul 18 03:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.622,"15min":0.621},"min":{"1min":0.45400000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.82099999999999995,"5min":1.375,"15min":1.375},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.61399999999999999,"15min":0.59299999999999997},"min":{"1min":0.378,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.40400000000000003}]},{"osd":6,"last update":"Tue Jul 18 03:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.68400000000000005,"15min":0.67000000000000004},"min":{"1min":0.52200000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.629},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.70499999999999996,"15min":0.69299999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.64900000000000002}]},{"osd":7,"last update":"Tue Jul 18 03:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.79200000000000004,"15min":0.78000000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1319999999999999,"5min":1.248,"15min":1.248},"last":0.745},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.76900000000000002,"15min":0.76300000000000001},"min":{"1min":0.56499999999999995,"5min":0.502,"15min":0.502},"max":{"1min":1.024,"5min":1.288,"15min":1.288},"last":0.55500000000000005}]},{"osd":8,"last update":"Tue Jul 18 03:07:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.76500000000000001,"15min":0.77000000000000002},"min":{"1min":0.50800000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.83899999999999997,"15min":0.84199999999999997},"min":{"1min":0.58999999999999997,"5min":0.498,"15min":0.498},"max":{"1min":0.98599999999999999,"5min":1.091,"15min":1.091},"last":0.496}]},{"osd":9,"last update":"Tue Jul 18 03:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.78300000000000003,"15min":0.76800000000000002},"min":{"1min":0.66700000000000004,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.145,"5min":1.304,"15min":1.304},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.81899999999999995,"15min":0.80400000000000005},"min":{"1min":0.63400000000000001,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1659999999999999,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.71199999999999997}]},{"osd":10,"last update":"Tue Jul 18 03:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.78200000000000003,"15min":0.77200000000000002},"min":{"1min":0.57999999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.054,"5min":1.163,"15min":1.163},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.79900000000000004,"15min":0.78900000000000003},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.026,"5min":1.496,"15min":1.496},"last":0.78400000000000003}]}]},{"osd":10,"up_from":71,"seq":304942678061,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1120,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1146880,"data_stored":1161118,"data_compressed":59283,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:07:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.44600000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.71099999999999997,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.66300000000000003,"15min":0.66900000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.70799999999999996}]},{"osd":1,"last update":"Tue Jul 18 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.47699999999999998,"15min":0.48099999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.61499999999999999,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.505,"15min":0.51400000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70999999999999996,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.58999999999999997}]},{"osd":2,"last update":"Tue Jul 18 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.64700000000000002,"15min":0.63400000000000001},"min":{"1min":0.437,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.64700000000000002,"15min":0.64300000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.78900000000000003,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.72699999999999998}]},{"osd":3,"last update":"Tue Jul 18 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.67200000000000004,"15min":0.68500000000000005},"min":{"1min":0.48199999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80000000000000004,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65500000000000003,"15min":0.66300000000000003},"min":{"1min":0.498,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.66100000000000003}]},{"osd":4,"last update":"Tue Jul 18 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.59799999999999998,"15min":0.61399999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.58399999999999996,"15min":0.57999999999999996},"min":{"1min":0.375,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.82999999999999996,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.80900000000000005}]},{"osd":5,"last update":"Tue Jul 18 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71299999999999997,"15min":0.72099999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.753,"5min":0.751,"15min":0.751},"min":{"1min":0.49299999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.90000000000000002}]},{"osd":6,"last update":"Tue Jul 18 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.745,"15min":0.75800000000000001},"min":{"1min":0.50700000000000001,"5min":0.501,"15min":0.501},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.79300000000000004,"15min":0.80100000000000005},"min":{"1min":0.61599999999999999,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.96899999999999997,"5min":1.077,"15min":1.077},"last":0.94099999999999995}]},{"osd":7,"last update":"Tue Jul 18 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.55800000000000005}]},{"osd":8,"last update":"Tue Jul 18 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.79200000000000004,"15min":0.80500000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.91000000000000003,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.81799999999999995,"15min":0.82499999999999996},"min":{"1min":0.60299999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.0169999999999999,"5min":1.032,"15min":1.032},"last":0.95899999999999996}]},{"osd":9,"last update":"Tue Jul 18 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.83799999999999997,"15min":0.84399999999999997},"min":{"1min":0.66500000000000004,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.84699999999999998,"15min":0.85499999999999998},"min":{"1min":0.63300000000000001,"5min":0.626,"15min":0.626},"max":{"1min":0.996,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.98799999999999999}]},{"osd":11,"last update":"Tue Jul 18 03:06:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.82699999999999996,"15min":0.83199999999999996},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.96599999999999997,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.80000000000000004,"15min":0.80900000000000005},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.92800000000000005,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.83099999999999996}]}]},{"osd":9,"up_from":66,"seq":283467841584,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":1015808,"data_stored":511449,"data_compressed":23524,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.52000000000000002,"15min":0.54800000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.629,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.55000000000000004,"15min":0.57199999999999995},"min":{"1min":0.32600000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.79700000000000004,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.39800000000000002}]},{"osd":1,"last update":"Tue Jul 18 03:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.68999999999999995,"15min":0.72099999999999997},"min":{"1min":0.41499999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.83599999999999997,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.69699999999999995,"15min":0.73099999999999998},"min":{"1min":0.46200000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.81699999999999995,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.58099999999999996}]},{"osd":2,"last update":"Tue Jul 18 03:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.68899999999999995,"15min":0.71099999999999997},"min":{"1min":0.45800000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.78100000000000003,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.71899999999999997,"15min":0.74199999999999999},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.81299999999999994,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.77600000000000002}]},{"osd":3,"last update":"Tue Jul 18 03:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.63500000000000001,"15min":0.67600000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.76500000000000001,"5min":1.038,"15min":1.038},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.61199999999999999,"15min":0.65000000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.67600000000000005,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.46200000000000002}]},{"osd":4,"last update":"Tue Jul 18 03:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.77000000000000002,"15min":0.81699999999999995},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.85999999999999999,"5min":1.47,"15min":1.47},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.77900000000000003,"15min":0.81799999999999995},"min":{"1min":0.47599999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.90700000000000003,"5min":1.619,"15min":1.619},"last":0.68200000000000005}]},{"osd":5,"last update":"Tue Jul 18 03:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.79800000000000004,"15min":0.82199999999999995},"min":{"1min":0.53500000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.89300000000000002,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.78000000000000003,"15min":0.80800000000000005},"min":{"1min":0.53300000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.90100000000000002,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.82199999999999995}]},{"osd":6,"last update":"Tue Jul 18 03:04:30 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.64300000000000002}]},{"osd":7,"last update":"Tue Jul 18 03:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.84799999999999998,"15min":0.89600000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.86699999999999999,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.89500000000000002,"15min":0.92300000000000004},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.81399999999999995,"5min":4.4009999999999998,"15min":4.4009999999999998},"last":0.64600000000000002}]},{"osd":8,"last update":"Tue Jul 18 03:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.876,"15min":0.91600000000000004},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.90900000000000003,"15min":0.94599999999999995},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.012,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.76500000000000001}]},{"osd":10,"last update":"Tue Jul 18 03:06:53 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.86099999999999999,"15min":0.89500000000000002},"min":{"1min":0.60199999999999998,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1020000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.873,"15min":0.92500000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.86099999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.754}]},{"osd":11,"last update":"Tue Jul 18 03:07:09 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.82799999999999996,"15min":0.83099999999999996},"min":{"1min":0.69399999999999995,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":0.96399999999999997,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.79400000000000004,"15min":0.79600000000000004},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":0.93700000000000006,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.79300000000000004}]}]},{"osd":8,"up_from":58,"seq":249108103221,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":1007616,"data_stored":507251,"data_compressed":23524,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.69099999999999995,"15min":0.64000000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.80700000000000005,"5min":7.0359999999999996,"15min":7.0359999999999996},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.58199999999999996,"15min":0.56999999999999995},"min":{"1min":0.42499999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.75700000000000001,"5min":2.883,"15min":2.883},"last":0.55700000000000005}]},{"osd":1,"last update":"Tue Jul 18 03:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.67500000000000004,"15min":0.63900000000000001},"min":{"1min":0.45200000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.95099999999999996,"5min":3.3340000000000001,"15min":3.3340000000000001},"last":0.752},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.67100000000000004,"15min":0.61599999999999999},"min":{"1min":0.55500000000000005,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.99199999999999999,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":0.82899999999999996}]},{"osd":2,"last update":"Tue Jul 18 03:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51400000000000001,"15min":0.47099999999999997},"min":{"1min":0.40100000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70199999999999996,"5min":3.2850000000000001,"15min":3.2850000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.53300000000000003,"15min":0.502},"min":{"1min":0.38400000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.753,"5min":2.012,"15min":2.012},"last":0.63900000000000001}]},{"osd":3,"last update":"Tue Jul 18 03:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64600000000000002,"15min":0.623},"min":{"1min":0.46899999999999997,"5min":0.432,"15min":0.432},"max":{"1min":0.88100000000000001,"5min":2.9100000000000001,"15min":2.9100000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.67000000000000004,"15min":0.65000000000000002},"min":{"1min":0.49399999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.996,"5min":3.6059999999999999,"15min":3.6059999999999999},"last":0.60499999999999998}]},{"osd":4,"last update":"Tue Jul 18 03:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.67600000000000005,"15min":0.63800000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.04,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.73299999999999998,"15min":0.69999999999999996},"min":{"1min":0.51700000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0940000000000001,"5min":3.3650000000000002,"15min":3.3650000000000002},"last":0.84599999999999997}]},{"osd":5,"last update":"Tue Jul 18 03:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.64800000000000002,"15min":0.60399999999999998},"min":{"1min":0.47499999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.0269999999999999,"5min":2.4279999999999999,"15min":2.4279999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.65800000000000003,"15min":0.61399999999999999},"min":{"1min":0.41899999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.135,"5min":2.7410000000000001,"15min":2.7410000000000001},"last":0.71799999999999997}]},{"osd":6,"last update":"Tue Jul 18 03:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76400000000000001,"15min":0.72599999999999998},"min":{"1min":0.46200000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.94399999999999995,"5min":3.5779999999999998,"15min":3.5779999999999998},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.77700000000000002,"15min":0.745},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.97899999999999998,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":0.92300000000000004}]},{"osd":7,"last update":"Tue Jul 18 03:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.78700000000000003,"15min":0.73899999999999999},"min":{"1min":0.67000000000000004,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.081,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.77900000000000003,"15min":0.73399999999999999},"min":{"1min":0.628,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.157,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":0.89200000000000002}]},{"osd":9,"last update":"Tue Jul 18 03:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.82899999999999996,"15min":0.78800000000000003},"min":{"1min":0.627,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1379999999999999,"5min":3.544,"15min":3.544},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.83099999999999996,"15min":0.79700000000000004},"min":{"1min":0.66500000000000004,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.1259999999999999,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":0.97699999999999998}]},{"osd":10,"last update":"Tue Jul 18 03:06:55 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.82799999999999996,"15min":0.81699999999999995},"min":{"1min":0.59299999999999997,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1519999999999999,"5min":2.8570000000000002,"15min":2.8570000000000002},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.879,"15min":0.88100000000000001},"min":{"1min":0.58699999999999997,"5min":0.502,"15min":0.502},"max":{"1min":1.1950000000000001,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":0.90400000000000003}]},{"osd":11,"last update":"Tue Jul 18 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.86899999999999999,"15min":0.86599999999999999},"min":{"1min":0.69399999999999995,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.0720000000000001,"5min":1.107,"15min":1.107},"last":0.878},{"interface":"front","average":{"1min":0.871,"5min":0.85299999999999998,"15min":0.84999999999999998},"min":{"1min":0.65800000000000003,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.81299999999999994}]}]},{"osd":7,"up_from":51,"seq":219043332153,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":1036,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":1060864,"data_stored":1103226,"data_compressed":59283,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.55500000000000005,"15min":0.54300000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.495},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59999999999999998,"15min":0.60299999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82599999999999996,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.44500000000000001}]},{"osd":1,"last update":"Tue Jul 18 03:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.434,"15min":0.42399999999999999},"min":{"1min":0.28699999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.5},{"interface":"front","average":{"1min":0.503,"5min":0.45400000000000001,"15min":0.438},"min":{"1min":0.317,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.74299999999999999,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.55100000000000005}]},{"osd":2,"last update":"Tue Jul 18 03:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.60599999999999998,"15min":0.57699999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.90600000000000003,"5min":1.111,"15min":1.111},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.60699999999999998,"15min":0.58699999999999997},"min":{"1min":0.375,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.89500000000000002,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.63500000000000001}]},{"osd":3,"last update":"Tue Jul 18 03:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.63500000000000001,"15min":0.60599999999999998},"min":{"1min":0.45000000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.83999999999999997,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.63200000000000001,"15min":0.64400000000000002},"min":{"1min":0.41699999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.83699999999999997,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.58499999999999996}]},{"osd":4,"last update":"Tue Jul 18 03:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.5,"15min":0.48799999999999999},"min":{"1min":0.34799999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.55700000000000005,"15min":0.55100000000000005},"min":{"1min":0.36699999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.52500000000000002}]},{"osd":5,"last update":"Tue Jul 18 03:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.68899999999999995,"15min":0.65700000000000003},"min":{"1min":0.51600000000000001,"5min":0.501,"15min":0.501},"max":{"1min":1.0249999999999999,"5min":1.077,"15min":1.077},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.66700000000000004,"15min":0.65600000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.93400000000000005,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.69199999999999995}]},{"osd":6,"last update":"Tue Jul 18 03:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67200000000000004},"min":{"1min":0.51800000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.89400000000000002,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.73299999999999998,"15min":0.70899999999999996},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.86099999999999999}]},{"osd":8,"last update":"Tue Jul 18 03:07:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.72299999999999998,"15min":0.69699999999999995},"min":{"1min":0.53600000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.997,"5min":1.022,"15min":1.022},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.745,"5min":0.68799999999999994,"15min":0.65600000000000003},"min":{"1min":0.55200000000000005,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.94999999999999996,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.82299999999999995}]},{"osd":9,"last update":"Tue Jul 18 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.76000000000000001,"15min":0.73799999999999999},"min":{"1min":0.435,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":3.1880000000000002,"5min":3.1880000000000002,"15min":3.1880000000000002},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.77900000000000003,"15min":0.76800000000000002},"min":{"1min":0.58399999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.88800000000000001}]},{"osd":10,"last update":"Tue Jul 18 03:06:55 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71799999999999997,"15min":0.72299999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.76100000000000001,"15min":0.755},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.91400000000000003}]},{"osd":11,"last update":"Tue Jul 18 03:07:06 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.83599999999999997,"15min":0.82599999999999996},"min":{"1min":0.76100000000000001,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.84899999999999998,"15min":0.83699999999999997},"min":{"1min":0.754,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.93700000000000006}]}]},{"osd":6,"up_from":46,"seq":197568495678,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":991232,"data_stored":502529,"data_compressed":23524,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.52000000000000002,"15min":0.55800000000000005},"min":{"1min":0.307,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.48,"5min":3.1579999999999999,"15min":3.1579999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.53400000000000003,"15min":0.55000000000000004},"min":{"1min":0.36699999999999999,"5min":0.318,"15min":0.318},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":0.38900000000000001}]},{"osd":1,"last update":"Tue Jul 18 03:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.64300000000000002,"15min":0.60899999999999999},"min":{"1min":0.499,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.71699999999999997,"15min":0.73699999999999999},"min":{"1min":0.48299999999999998,"5min":0.44,"15min":0.44},"max":{"1min":1.226,"5min":3.0369999999999999,"15min":3.0369999999999999},"last":0.66700000000000004}]},{"osd":2,"last update":"Tue Jul 18 03:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.67300000000000004,"15min":0.66100000000000003},"min":{"1min":0.47799999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.46300000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.63600000000000001}]},{"osd":3,"last update":"Tue Jul 18 03:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56499999999999995,"15min":0.57299999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.69499999999999995,"15min":0.74199999999999999},"min":{"1min":0.40100000000000002,"5min":0.377,"15min":0.377},"max":{"1min":1.546,"5min":3.226,"15min":3.226},"last":0.45400000000000001}]},{"osd":4,"last update":"Tue Jul 18 03:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69299999999999995,"15min":0.69399999999999995},"min":{"1min":0.47499999999999998,"5min":0.441,"15min":0.441},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.70699999999999996,"15min":0.70099999999999996},"min":{"1min":0.55200000000000005,"5min":0.438,"15min":0.438},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.75900000000000001}]},{"osd":5,"last update":"Tue Jul 18 03:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.78500000000000003,"15min":0.83599999999999997},"min":{"1min":0.54500000000000004,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1060000000000001,"5min":3.125,"15min":3.125},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.76000000000000001,"15min":0.78000000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.4039999999999999,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.86799999999999999}]},{"osd":7,"last update":"Tue Jul 18 03:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.82499999999999996,"15min":0.83699999999999997},"min":{"1min":0.59499999999999997,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1619999999999999,"5min":3.0870000000000002,"15min":3.0870000000000002},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75900000000000001,"15min":0.73599999999999999},"min":{"1min":0.52300000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.95399999999999996}]},{"osd":8,"last update":"Tue Jul 18 03:06:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.85099999999999998,"15min":0.876},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0640000000000001,"5min":3.1909999999999998,"15min":3.1909999999999998},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.99099999999999999,"5min":1.046,"15min":1.046},"last":1.0289999999999999}]},{"osd":9,"last update":"Tue Jul 18 03:06:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.75600000000000001,"15min":0.76500000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.83799999999999997,"15min":0.85199999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.73299999999999998}]},{"osd":10,"last update":"Tue Jul 18 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.88100000000000001,"15min":0.879},"min":{"1min":0.752,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.873,"5min":0.83199999999999996,"15min":0.82199999999999995},"min":{"1min":0.68700000000000006,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":0.92000000000000004}]},{"osd":11,"last update":"Tue Jul 18 03:07:05 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.86499999999999999,"15min":0.86099999999999999},"min":{"1min":0.61799999999999999,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.85299999999999998,"15min":0.85499999999999998},"min":{"1min":0.67000000000000004,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.516,"5min":1.583,"15min":1.583},"last":0.88700000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":1040,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":1064960,"data_stored":569220,"data_compressed":23524,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:06:32 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.55000000000000004,"15min":0.52400000000000002},"min":{"1min":0.39200000000000002,"5min":0.29799999999999999,"15min":0.27800000000000002},"max":{"1min":0.65500000000000003,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55200000000000005,"15min":0.52700000000000002},"min":{"1min":0.376,"5min":0.33100000000000002,"15min":0.29899999999999999},"max":{"1min":0.73399999999999999,"5min":1.421,"15min":1.421},"last":0.54600000000000004}]},{"osd":2,"last update":"Tue Jul 18 03:06:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.59799999999999998,"15min":0.53000000000000003},"min":{"1min":0.39500000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.0009999999999999,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.59799999999999998,"15min":0.55400000000000005},"min":{"1min":0.50900000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.91000000000000003,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.88800000000000001}]},{"osd":3,"last update":"Tue Jul 18 03:06:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.622,"15min":0.59999999999999998},"min":{"1min":0.378,"5min":0.312,"15min":0.312},"max":{"1min":1.014,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.627,"15min":0.627},"min":{"1min":0.47999999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.026,"5min":1.504,"15min":1.504},"last":0.63400000000000001}]},{"osd":4,"last update":"Tue Jul 18 03:06:46 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.48199999999999998,"15min":0.41399999999999998},"min":{"1min":0.379,"5min":0.215,"15min":0.215},"max":{"1min":0.68500000000000005,"5min":1.784,"15min":1.784},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.50600000000000001,"15min":0.435},"min":{"1min":0.32800000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.83599999999999997,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.65500000000000003}]},{"osd":5,"last update":"Tue Jul 18 03:07:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.64400000000000002,"15min":0.54200000000000004},"min":{"1min":0.436,"5min":0.375,"15min":0.375},"max":{"1min":0.84499999999999997,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.65300000000000002,"15min":0.57699999999999996},"min":{"1min":0.441,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.94299999999999995,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.91600000000000004}]},{"osd":6,"last update":"Tue Jul 18 03:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.66400000000000003,"15min":0.60999999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.87,"5min":2.6600000000000001,"15min":2.6600000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.67300000000000004,"15min":0.64500000000000002},"min":{"1min":0.54600000000000004,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.89900000000000002,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.73299999999999998}]},{"osd":7,"last update":"Tue Jul 18 03:06:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.61699999999999999,"15min":0.57099999999999995},"min":{"1min":0.45400000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.67600000000000005,"15min":0.65600000000000003},"min":{"1min":0.42299999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.90100000000000002,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":0.85799999999999998}]},{"osd":8,"last update":"Tue Jul 18 03:07:12 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.77600000000000002,"15min":0.78200000000000003},"min":{"1min":0.47699999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.997,"5min":2.6419999999999999,"15min":2.6419999999999999},"last":0.997},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.748,"15min":0.68999999999999995},"min":{"1min":0.52400000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.03,"5min":1.698,"15min":1.698},"last":0.93200000000000005}]},{"osd":9,"last update":"Tue Jul 18 03:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.83899999999999997,"15min":0.85299999999999998},"min":{"1min":0.58499999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.96299999999999997,"5min":2.6789999999999998,"15min":2.6789999999999998},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.78000000000000003,"15min":0.752},"min":{"1min":0.68100000000000005,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0129999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.875}]},{"osd":10,"last update":"Tue Jul 18 03:06:57 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79000000000000004,"15min":0.77200000000000002},"min":{"1min":0.57399999999999995,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0589999999999999,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.77700000000000002,"15min":0.76500000000000001},"min":{"1min":0.56699999999999995,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.002,"5min":1.359,"15min":1.359},"last":0.96799999999999997}]},{"osd":11,"last update":"Tue Jul 18 03:07:02 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.90600000000000003,"15min":0.90800000000000003},"min":{"1min":0.72299999999999998,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.1060000000000001,"5min":1.512,"15min":1.512},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":1.0249999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":1015808,"data_stored":503917,"data_compressed":23524,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Jul 18 03:06:31 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61499999999999999,"15min":0.55800000000000005},"min":{"1min":0.33700000000000002,"5min":0.33600000000000002,"15min":0.28299999999999997},"max":{"1min":0.88900000000000001,"5min":1.075,"15min":1.075},"last":0.627},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.64000000000000001,"15min":0.56999999999999995},"min":{"1min":0.371,"5min":0.36099999999999999,"15min":0.30199999999999999},"max":{"1min":1.1060000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.80500000000000005}]},{"osd":2,"last update":"Tue Jul 18 03:06:54 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.66800000000000004,"15min":0.57399999999999995},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":5.3300000000000001,"5min":5.3300000000000001,"15min":5.3300000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.67400000000000004,"15min":0.57899999999999996},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.01,"5min":1.204,"15min":1.204},"last":0.49099999999999999}]},{"osd":3,"last update":"Tue Jul 18 03:07:04 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54200000000000004,"15min":0.54400000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.53300000000000003,"15min":0.53100000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.45400000000000001}]},{"osd":4,"last update":"Tue Jul 18 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.68600000000000005,"15min":0.624},"min":{"1min":0.51700000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.748},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.66500000000000004,"15min":0.61199999999999999},"min":{"1min":0.49099999999999999,"5min":0.433,"15min":0.433},"max":{"1min":0.97499999999999998,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.59299999999999997}]},{"osd":5,"last update":"Tue Jul 18 03:05:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.64700000000000002,"15min":0.63500000000000001},"min":{"1min":0.46899999999999997,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.028,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.68200000000000005,"15min":0.65900000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.998,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.79600000000000004}]},{"osd":6,"last update":"Tue Jul 18 03:06:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.60099999999999998,"15min":0.59099999999999997},"min":{"1min":0.40400000000000003,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.97299999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.63400000000000001,"15min":0.60399999999999998},"min":{"1min":0.40300000000000002,"5min":0.372,"15min":0.372},"max":{"1min":1.1639999999999999,"5min":1.216,"15min":1.216},"last":0.55900000000000005}]},{"osd":7,"last update":"Tue Jul 18 03:07:04 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.80600000000000005,"15min":0.77500000000000002},"min":{"1min":0.57299999999999995,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":5.2869999999999999,"5min":5.2869999999999999,"15min":5.2869999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.024,"5min":0.86899999999999999,"15min":0.84299999999999997},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":5.3579999999999997,"5min":5.3579999999999997,"15min":5.3579999999999997},"last":0.88100000000000001}]},{"osd":8,"last update":"Tue Jul 18 03:07:08 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":0.80200000000000005,"15min":0.76400000000000001},"min":{"1min":0.52900000000000003,"5min":0.495,"15min":0.495},"max":{"1min":6.96,"5min":6.96,"15min":6.96},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.0980000000000001,"5min":0.873,"15min":0.84799999999999998},"min":{"1min":0.54300000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":6.9909999999999997,"5min":6.9909999999999997,"15min":6.9909999999999997},"last":0.90000000000000002}]},{"osd":9,"last update":"Tue Jul 18 03:06:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.77900000000000003,"15min":0.78700000000000003},"min":{"1min":0.46300000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0409999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.85699999999999998,"15min":0.873},"min":{"1min":0.497,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1879999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.85699999999999998}]},{"osd":10,"last update":"Tue Jul 18 03:06:54 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":0.96699999999999997,"15min":0.94099999999999995},"min":{"1min":0.70899999999999996,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":6.9359999999999999,"5min":6.9359999999999999,"15min":6.9359999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.748,"5min":0.78000000000000003,"15min":0.78900000000000003},"min":{"1min":0.57899999999999996,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.99299999999999999,"5min":1.304,"15min":1.304},"last":0.69399999999999995}]},{"osd":11,"last update":"Tue Jul 18 03:06:16 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.81299999999999994,"15min":0.80900000000000005},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.0760000000000001,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.88600000000000001,"15min":0.88200000000000001},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.3160000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.93700000000000006}]}]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1064,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1089536,"data_stored":571053,"data_compressed":23524,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.56799999999999995,"15min":0.54000000000000004},"min":{"1min":0.38800000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58399999999999996,"15min":0.55500000000000005},"min":{"1min":0.41399999999999998,"5min":0.373,"15min":0.34599999999999997},"max":{"1min":0.89600000000000002,"5min":1.726,"15min":1.726},"last":0.46899999999999997}]},{"osd":1,"last update":"Tue Jul 18 03:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.61499999999999999,"15min":0.52700000000000002},"min":{"1min":0.49199999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.89200000000000002,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.60399999999999998,"15min":0.52000000000000002},"min":{"1min":0.47299999999999998,"5min":0.34300000000000003,"15min":0.32100000000000001},"max":{"1min":0.82499999999999996,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.57899999999999996}]},{"osd":3,"last update":"Tue Jul 18 03:06:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59299999999999997,"15min":0.56299999999999994},"min":{"1min":0.439,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93899999999999995,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.64400000000000002,"15min":0.59299999999999997},"min":{"1min":0.40899999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.127,"5min":3.234,"15min":3.234},"last":0.60599999999999998}]},{"osd":4,"last update":"Tue Jul 18 03:07:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66400000000000003,"15min":0.66300000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.73599999999999999,"15min":0.74199999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.95699999999999996,"5min":1.353,"15min":1.353},"last":0.55400000000000005}]},{"osd":5,"last update":"Tue Jul 18 03:07:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.51000000000000001,"15min":0.46200000000000002},"min":{"1min":0.34899999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.91500000000000004,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.53800000000000003,"15min":0.47199999999999998},"min":{"1min":0.41399999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.0309999999999999,"5min":1.335,"15min":1.335},"last":0.39900000000000002}]},{"osd":6,"last update":"Tue Jul 18 03:06:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.67700000000000005,"15min":0.64600000000000002},"min":{"1min":0.55700000000000005,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.95399999999999996,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.622},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.72799999999999998,"15min":0.69399999999999995},"min":{"1min":0.56999999999999995,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.6619999999999999,"5min":1.665,"15min":1.665},"last":0.67700000000000005}]},{"osd":7,"last update":"Tue Jul 18 03:07:04 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.84899999999999998,"15min":0.85299999999999998},"min":{"1min":0.61399999999999999,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0129999999999999,"5min":1.532,"15min":1.532},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.78000000000000003,"15min":0.78100000000000003},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.66300000000000003}]},{"osd":8,"last update":"Tue Jul 18 03:07:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.70099999999999996,"15min":0.64100000000000001},"min":{"1min":0.48799999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.159,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.70999999999999996,"15min":0.66700000000000004},"min":{"1min":0.61799999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.079,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.69799999999999995}]},{"osd":9,"last update":"Tue Jul 18 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.753,"15min":0.74199999999999999},"min":{"1min":0.49299999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.2549999999999999,"5min":1.825,"15min":1.825},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.81599999999999995,"15min":0.79300000000000004},"min":{"1min":0.70899999999999996,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.1020000000000001,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.80600000000000005}]},{"osd":10,"last update":"Tue Jul 18 03:06:54 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.82999999999999996,"15min":0.79300000000000004},"min":{"1min":0.79200000000000004,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1000000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.78400000000000003,"15min":0.753},"min":{"1min":0.71299999999999997,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0089999999999999,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.749}]},{"osd":11,"last update":"Tue Jul 18 03:07:04 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.92400000000000004,"15min":0.92600000000000005},"min":{"1min":0.73399999999999999,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1839999999999999,"5min":1.76,"15min":1.76},"last":0.875},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.873,"15min":0.876},"min":{"1min":0.67400000000000004,"5min":0.624,"15min":0.624},"max":{"1min":1.1160000000000001,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.78800000000000003}]}]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":1003520,"data_stored":502723,"data_compressed":23524,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:06:25 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.499,"15min":0.42699999999999999},"min":{"1min":0.34999999999999998,"5min":0.312,"15min":0.312},"max":{"1min":1.8440000000000001,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":0.748},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.504,"15min":0.433},"min":{"1min":0.35899999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.93300000000000005}]},{"osd":1,"last update":"Tue Jul 18 03:06:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.60499999999999998,"15min":0.57199999999999995},"min":{"1min":0.45600000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.79300000000000004,"5min":1.115,"15min":1.115},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.66700000000000004,"15min":0.59499999999999997},"min":{"1min":0.44400000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":0.79200000000000004}]},{"osd":2,"last update":"Tue Jul 18 03:06:25 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.68100000000000005,"15min":0.61399999999999999},"min":{"1min":0.437,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0960000000000001,"5min":1.26,"15min":1.26},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.63700000000000001,"15min":0.60299999999999998},"min":{"1min":0.46600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.048,"5min":1.212,"15min":1.212},"last":0.78200000000000003}]},{"osd":4,"last update":"Tue Jul 18 03:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.70299999999999996,"15min":0.63300000000000001},"min":{"1min":0.56999999999999995,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.706,"5min":1.706,"15min":1.706},"last":0.871},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.69999999999999996,"15min":0.629},"min":{"1min":0.55100000000000005,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0629999999999999,"5min":1.143,"15min":1.143},"last":0.89400000000000002}]},{"osd":5,"last update":"Tue Jul 18 03:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.72099999999999997,"15min":0.64900000000000002},"min":{"1min":0.53200000000000003,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.73799999999999999,"15min":0.68999999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0329999999999999,"5min":1.218,"15min":1.218},"last":0.72799999999999998}]},{"osd":6,"last update":"Tue Jul 18 03:06:29 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.71399999999999997,"15min":0.65800000000000003},"min":{"1min":0.434,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.65300000000000002,"15min":0.61499999999999999},"min":{"1min":0.371,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":2.669,"5min":2.669,"15min":2.669},"last":0.69899999999999995}]},{"osd":7,"last update":"Tue Jul 18 03:06:54 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.76300000000000001,"15min":0.73099999999999998},"min":{"1min":0.56999999999999995,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.78600000000000003,"15min":0.75900000000000001},"min":{"1min":0.58799999999999997,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.1659999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.80300000000000005}]},{"osd":8,"last update":"Tue Jul 18 03:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.76200000000000001}]},{"osd":9,"last update":"Tue Jul 18 03:06:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.76200000000000001,"15min":0.78300000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":0.88},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.86199999999999999,"15min":0.876},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.91600000000000004}]},{"osd":10,"last update":"Tue Jul 18 03:06:54 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.89500000000000002,"15min":0.90700000000000003},"min":{"1min":0.63600000000000001,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.77,"5min":1.77,"15min":1.77},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.875,"15min":0.88500000000000001},"min":{"1min":0.66500000000000004,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.159,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.97499999999999998}]},{"osd":11,"last update":"Tue Jul 18 03:07:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84199999999999997,"15min":0.84099999999999997},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.72499999999999998,"15min":0.71899999999999997},"min":{"1min":0.58999999999999997,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.81699999999999995}]}]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":1003520,"data_stored":507131,"data_compressed":23524,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.56200000000000006,"15min":0.54600000000000004},"min":{"1min":0.39600000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.439},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.56899999999999995,"15min":0.54500000000000004},"min":{"1min":0.42799999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.48999999999999999}]},{"osd":1,"last update":"Tue Jul 18 03:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40699999999999997,"15min":0.36799999999999999},"min":{"1min":0.28599999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.72099999999999997,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.41799999999999998,"15min":0.38400000000000001},"min":{"1min":0.314,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.58599999999999997,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.41299999999999998}]},{"osd":2,"last update":"Tue Jul 18 03:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.59899999999999998,"15min":0.57499999999999996},"min":{"1min":0.43099999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.85699999999999998,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.622,"15min":0.57299999999999995},"min":{"1min":0.41899999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.88100000000000001,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.60099999999999998}]},{"osd":3,"last update":"Tue Jul 18 03:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.71399999999999997,"15min":0.61699999999999999},"min":{"1min":0.51100000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":5.726,"5min":5.726,"15min":5.726},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.65300000000000002,"15min":0.59999999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":4.141,"5min":4.141,"15min":4.141},"last":0.439}]},{"osd":5,"last update":"Tue Jul 18 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.67600000000000005,"15min":0.61799999999999999},"min":{"1min":0.42699999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.86399999999999999,"5min":1.214,"15min":1.214},"last":0.62},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.65700000000000003,"15min":0.61499999999999999},"min":{"1min":0.48599999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.85099999999999998,"5min":1.306,"15min":1.306},"last":0.68300000000000005}]},{"osd":6,"last update":"Tue Jul 18 03:06:27 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.754,"15min":0.73799999999999999},"min":{"1min":0.46500000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":4.2850000000000001,"5min":4.2850000000000001,"15min":4.2850000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.69599999999999995,"15min":0.66900000000000004},"min":{"1min":0.41299999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":4.7130000000000001,"5min":4.7130000000000001,"15min":4.7130000000000001},"last":0.52800000000000002}]},{"osd":7,"last update":"Tue Jul 18 03:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.39700000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.89800000000000002,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.65300000000000002,"15min":0.64700000000000002},"min":{"1min":0.38700000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.98799999999999999,"5min":1.034,"15min":1.034},"last":0.58199999999999996}]},{"osd":8,"last update":"Tue Jul 18 03:06:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73999999999999999,"15min":0.73499999999999999},"min":{"1min":0.53300000000000003,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.99399999999999999,"5min":1.155,"15min":1.155},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.75800000000000001,"15min":0.74399999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.008,"5min":1.135,"15min":1.135},"last":0.72199999999999998}]},{"osd":9,"last update":"Tue Jul 18 03:06:35 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.77300000000000002,"15min":0.73799999999999999},"min":{"1min":0.56299999999999994,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":4.6719999999999997,"5min":4.6719999999999997,"15min":4.6719999999999997},"last":0.63200000000000001},{"interface":"front","average":{"1min":1.0169999999999999,"5min":0.86499999999999999,"15min":0.82499999999999996},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":5.0060000000000002,"5min":5.0060000000000002,"15min":5.0060000000000002},"last":0.73399999999999999}]},{"osd":10,"last update":"Tue Jul 18 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.78100000000000003,"15min":0.79400000000000004},"min":{"1min":0.54500000000000004,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97799999999999998,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68400000000000005,"15min":0.69699999999999995},"min":{"1min":0.47699999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91200000000000003,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.56799999999999995}]},{"osd":11,"last update":"Tue Jul 18 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.80300000000000005,"15min":0.80000000000000004},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.76300000000000001,"15min":0.76000000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.746}]}]},{"osd":5,"up_from":38,"seq":163208757315,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1069056,"data_stored":1098862,"data_compressed":59283,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56100000000000005,"15min":0.53500000000000003},"min":{"1min":0.40600000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.80500000000000005,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.53700000000000003,"15min":0.53400000000000003},"min":{"1min":0.379,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.78100000000000003,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.69399999999999995}]},{"osd":1,"last update":"Tue Jul 18 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.60199999999999998,"15min":0.55200000000000005},"min":{"1min":0.435,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.57799999999999996,"15min":0.55200000000000005},"min":{"1min":0.42399999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.74399999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.65400000000000003}]},{"osd":2,"last update":"Tue Jul 18 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.42099999999999999,"15min":0.38500000000000001},"min":{"1min":0.33800000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.436,"15min":0.40300000000000002},"min":{"1min":0.311,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.57499999999999996}]},{"osd":3,"last update":"Tue Jul 18 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.60599999999999998,"15min":0.57399999999999995},"min":{"1min":0.44500000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.81999999999999995,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.60999999999999999,"15min":0.57699999999999996},"min":{"1min":0.53500000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77600000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.72599999999999998}]},{"osd":4,"last update":"Tue Jul 18 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60599999999999998,"15min":0.57599999999999996},"min":{"1min":0.41899999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.77600000000000002,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.63500000000000001,"15min":0.57999999999999996},"min":{"1min":0.39900000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.86899999999999999,"5min":0.871,"15min":0.871},"last":0.84899999999999998}]},{"osd":6,"last update":"Tue Jul 18 03:06:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.627,"15min":0.60699999999999998},"min":{"1min":0.48299999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.75800000000000001,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.75,"5min":0.66900000000000004,"15min":0.63200000000000001},"min":{"1min":0.58799999999999997,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.83299999999999996}]},{"osd":7,"last update":"Tue Jul 18 03:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.48199999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.83099999999999996,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.69899999999999995,"15min":0.69299999999999995},"min":{"1min":0.49099999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.85399999999999998,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.80800000000000005}]},{"osd":8,"last update":"Tue Jul 18 03:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.56899999999999995,"15min":0.57399999999999995},"min":{"1min":0.45100000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.85899999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.63500000000000001,"15min":0.628},"min":{"1min":0.40799999999999997,"5min":0.371,"15min":0.371},"max":{"1min":0.84499999999999997,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.89500000000000002}]},{"osd":9,"last update":"Tue Jul 18 03:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76200000000000001,"15min":0.77000000000000002},"min":{"1min":0.52600000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.88300000000000001,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.76000000000000001,"15min":0.76600000000000001},"min":{"1min":0.64900000000000002,"5min":0.501,"15min":0.501},"max":{"1min":0.90300000000000002,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.90400000000000003}]},{"osd":10,"last update":"Tue Jul 18 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.75,"15min":0.73599999999999999},"min":{"1min":0.54800000000000004,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.92000000000000004,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.754,"15min":0.74299999999999999},"min":{"1min":0.56299999999999994,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.92400000000000004,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.877}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34117,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34117,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34117,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35759,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35759,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35759,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35759,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35759,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":66781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":68614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":68311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-18T03:07:19.747 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-18T03:07:19.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.991+0000 7fbe04bff700 1 -- 172.21.15.101:0/3647542648 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbe000ffb00 msgr2=0x7fbe000fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:19.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.991+0000 7fbe04bff700 1 --2- 172.21.15.101:0/3647542648 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbe000ffb00 0x7fbe000fff10 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0009990 tx=0x7fbdf0009ca0 comp rx=0 tx=0).stop 2023-07-18T03:07:19.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.991+0000 7fbe04bff700 1 -- 172.21.15.101:0/3647542648 shutdown_connections 2023-07-18T03:07:19.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.991+0000 7fbe04bff700 1 --2- 172.21.15.101:0/3647542648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe00101680 0x7fbe00107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:19.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.991+0000 7fbe04bff700 1 --2- 172.21.15.101:0/3647542648 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbe000ffb00 0x7fbe000fff10 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:19.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.991+0000 7fbe04bff700 1 --2- 172.21.15.101:0/3647542648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe00100cf0 0x7fbe00101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:19.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.991+0000 7fbe04bff700 1 -- 172.21.15.101:0/3647542648 >> 172.21.15.101:0/3647542648 conn(0x7fbe000fb0e0 msgr2=0x7fbe000fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:19.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.991+0000 7fbe04bff700 1 -- 172.21.15.101:0/3647542648 shutdown_connections 2023-07-18T03:07:19.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.991+0000 7fbe04bff700 1 -- 172.21.15.101:0/3647542648 wait complete. 2023-07-18T03:07:19.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.992+0000 7fbe04bff700 1 Processor -- start 2023-07-18T03:07:19.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.992+0000 7fbe04bff700 1 -- start start 2023-07-18T03:07:19.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.993+0000 7fbe04bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbe000ffb00 0x7fbe000742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:19.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.993+0000 7fbe04bff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe00100cf0 0x7fbe00072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:19.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.993+0000 7fbe04bff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe00101680 0x7fbe00072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:19.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.993+0000 7fbe04bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe000735e0 con 0x7fbe000ffb00 2023-07-18T03:07:19.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.993+0000 7fbe04bff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe00073750 con 0x7fbe00100cf0 2023-07-18T03:07:19.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.993+0000 7fbe04bff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0019f8a0 con 0x7fbe00101680 2023-07-18T03:07:19.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.993+0000 7fbdfe59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbe000ffb00 0x7fbe000742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:19.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.993+0000 7fbdfdd9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe00100cf0 0x7fbe00072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:19.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.993+0000 7fbdfdd9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe00100cf0 0x7fbe00072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60330/0 (socket says 172.21.15.101:60330) 2023-07-18T03:07:19.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.993+0000 7fbdfdd9b700 1 -- 172.21.15.101:0/919305194 learned_addr learned my addr 172.21.15.101:0/919305194 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:19.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.993+0000 7fbdfed9d700 1 --2- 172.21.15.101:0/919305194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe00101680 0x7fbe00072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:19.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.993+0000 7fbdfe59c700 1 -- 172.21.15.101:0/919305194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe00101680 msgr2=0x7fbe00072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:19.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.993+0000 7fbdfe59c700 1 --2- 172.21.15.101:0/919305194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe00101680 0x7fbe00072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:19.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.993+0000 7fbdfe59c700 1 -- 172.21.15.101:0/919305194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe00100cf0 msgr2=0x7fbe00072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:19.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.993+0000 7fbdfe59c700 1 --2- 172.21.15.101:0/919305194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe00100cf0 0x7fbe00072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:19.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.993+0000 7fbdfe59c700 1 -- 172.21.15.101:0/919305194 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf00095f0 con 0x7fbe000ffb00 2023-07-18T03:07:19.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.994+0000 7fbdfe59c700 1 --2- 172.21.15.101:0/919305194 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbe000ffb00 0x7fbe000742c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0000c00 tx=0x7fbdf0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:19.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.994+0000 7fbdef7fe700 1 -- 172.21.15.101:0/919305194 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf001d070 con 0x7fbe000ffb00 2023-07-18T03:07:19.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.994+0000 7fbdef7fe700 1 -- 172.21.15.101:0/919305194 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbdf0003f60 con 0x7fbe000ffb00 2023-07-18T03:07:19.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.994+0000 7fbe04bff700 1 -- 172.21.15.101:0/919305194 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe0019fb20 con 0x7fbe000ffb00 2023-07-18T03:07:19.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.994+0000 7fbdef7fe700 1 -- 172.21.15.101:0/919305194 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf0022b40 con 0x7fbe000ffb00 2023-07-18T03:07:19.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.994+0000 7fbe04bff700 1 -- 172.21.15.101:0/919305194 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe0019ff40 con 0x7fbe000ffb00 2023-07-18T03:07:19.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.995+0000 7fbe04bff700 1 -- 172.21.15.101:0/919305194 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe0010b320 con 0x7fbe000ffb00 2023-07-18T03:07:19.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.997+0000 7fbdef7fe700 1 -- 172.21.15.101:0/919305194 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fbdf0003aa0 con 0x7fbe000ffb00 2023-07-18T03:07:19.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.998+0000 7fbdef7fe700 1 --2- 172.21.15.101:0/919305194 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fbde80a11a0 0x7fbde80a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:19.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.999+0000 7fbdef7fe700 1 -- 172.21.15.101:0/919305194 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fbdf00ce400 con 0x7fbe000ffb00 2023-07-18T03:07:19.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.999+0000 7fbdfdd9b700 1 --2- 172.21.15.101:0/919305194 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fbde80a11a0 0x7fbde80a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:19.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:19.999+0000 7fbdfdd9b700 1 --2- 172.21.15.101:0/919305194 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fbde80a11a0 0x7fbde80a3650 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4005950 tx=0x7fbdf400b500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:19.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.002+0000 7fbdef7fe700 1 -- 172.21.15.101:0/919305194 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fbdf0097930 con 0x7fbe000ffb00 2023-07-18T03:07:20.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.197+0000 7fbe04bff700 1 -- 172.21.15.101:0/919305194 --> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbe00060950 con 0x7fbde80a11a0 2023-07-18T03:07:20.194 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:19 smithi179 ceph-mon[129229]: pgmap v349: 129 pgs: 129 active+clean; 646 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 6 op/s 2023-07-18T03:07:20.194 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:19 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/3584328588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:07:20.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.209+0000 7fbdef7fe700 1 -- 172.21.15.101:0/919305194 <== mgr.14150 v2:172.21.15.101:6800/3963742842 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427075 (secure 0 0 0) 0x7fbe00060950 con 0x7fbde80a11a0 2023-07-18T03:07:20.200 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:07:20.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.214+0000 7fbe04bff700 1 -- 172.21.15.101:0/919305194 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fbde80a11a0 msgr2=0x7fbde80a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:20.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.214+0000 7fbe04bff700 1 --2- 172.21.15.101:0/919305194 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fbde80a11a0 0x7fbde80a3650 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4005950 tx=0x7fbdf400b500 comp rx=0 tx=0).stop 2023-07-18T03:07:20.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.214+0000 7fbe04bff700 1 -- 172.21.15.101:0/919305194 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbe000ffb00 msgr2=0x7fbe000742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:20.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.214+0000 7fbe04bff700 1 --2- 172.21.15.101:0/919305194 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbe000ffb00 0x7fbe000742c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0000c00 tx=0x7fbdf0005740 comp rx=0 tx=0).stop 2023-07-18T03:07:20.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.215+0000 7fbe04bff700 1 -- 172.21.15.101:0/919305194 shutdown_connections 2023-07-18T03:07:20.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.215+0000 7fbe04bff700 1 --2- 172.21.15.101:0/919305194 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fbde80a11a0 0x7fbde80a3650 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:20.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.215+0000 7fbe04bff700 1 --2- 172.21.15.101:0/919305194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe00101680 0x7fbe00072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:20.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.215+0000 7fbe04bff700 1 --2- 172.21.15.101:0/919305194 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbe000ffb00 0x7fbe000742c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:20.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.215+0000 7fbe04bff700 1 --2- 172.21.15.101:0/919305194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe00100cf0 0x7fbe00072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:20.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.215+0000 7fbe04bff700 1 -- 172.21.15.101:0/919305194 >> 172.21.15.101:0/919305194 conn(0x7fbe000fb0e0 msgr2=0x7fbe000fd320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:20.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.215+0000 7fbe04bff700 1 -- 172.21.15.101:0/919305194 shutdown_connections 2023-07-18T03:07:20.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.215+0000 7fbe04bff700 1 -- 172.21.15.101:0/919305194 wait complete. 2023-07-18T03:07:20.204 INFO:teuthology.orchestra.run.smithi101.stderr:dumped all 2023-07-18T03:07:20.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:19 smithi101 ceph-mon[126379]: pgmap v349: 129 pgs: 129 active+clean; 646 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 6 op/s 2023-07-18T03:07:20.218 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:19 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3584328588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:07:20.220 INFO:teuthology.orchestra.run.smithi101.stdout:{"pg_ready":true,"pg_map":{"version":349,"stamp":"2023-07-18T03:07:18.885623+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661440,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":791,"num_read_kb":523,"num_write":349,"num_write_kb":3064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":785,"ondisk_log_size":785,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335600,"kb_used_data":12176,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606224,"statfs":{"total":1151940427776,"available":1151596773376,"internally_reserved":0,"allocated":12468224,"data_stored":8045618,"data_compressed":389565,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5695,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":77,"num_read_kb":47,"num_write":5,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002975"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.887431+0000","last_change":"2023-07-18T03:04:59.805689+0000","last_active":"2023-07-18T03:06:52.887431+0000","last_peered":"2023-07-18T03:06:52.887431+0000","last_clean":"2023-07-18T03:06:52.887431+0000","last_became_active":"2023-07-18T03:04:59.805224+0000","last_became_peered":"2023-07-18T03:04:59.805224+0000","last_unstale":"2023-07-18T03:06:52.887431+0000","last_undegraded":"2023-07-18T03:06:52.887431+0000","last_fullsized":"2023-07-18T03:06:52.887431+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:06:51.978483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.886501+0000","last_change":"2023-07-18T03:04:59.805013+0000","last_active":"2023-07-18T03:06:52.886501+0000","last_peered":"2023-07-18T03:06:52.886501+0000","last_clean":"2023-07-18T03:06:52.886501+0000","last_became_active":"2023-07-18T03:04:59.804702+0000","last_became_peered":"2023-07-18T03:04:59.804702+0000","last_unstale":"2023-07-18T03:06:52.886501+0000","last_undegraded":"2023-07-18T03:06:52.886501+0000","last_fullsized":"2023-07-18T03:06:52.886501+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:50:03.882265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.884770+0000","last_change":"2023-07-18T03:04:59.809035+0000","last_active":"2023-07-18T03:06:52.884770+0000","last_peered":"2023-07-18T03:06:52.884770+0000","last_clean":"2023-07-18T03:06:52.884770+0000","last_became_active":"2023-07-18T03:04:59.808630+0000","last_became_peered":"2023-07-18T03:04:59.808630+0000","last_unstale":"2023-07-18T03:06:52.884770+0000","last_undegraded":"2023-07-18T03:06:52.884770+0000","last_fullsized":"2023-07-18T03:06:52.884770+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:09:21.776254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.881696+0000","last_change":"2023-07-18T03:04:59.803510+0000","last_active":"2023-07-18T03:06:52.881696+0000","last_peered":"2023-07-18T03:06:52.881696+0000","last_clean":"2023-07-18T03:06:52.881696+0000","last_became_active":"2023-07-18T03:04:59.803060+0000","last_became_peered":"2023-07-18T03:04:59.803060+0000","last_unstale":"2023-07-18T03:06:52.881696+0000","last_undegraded":"2023-07-18T03:06:52.881696+0000","last_fullsized":"2023-07-18T03:06:52.881696+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:05:42.674904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.878377+0000","last_change":"2023-07-18T03:04:59.806911+0000","last_active":"2023-07-18T03:06:52.878377+0000","last_peered":"2023-07-18T03:06:52.878377+0000","last_clean":"2023-07-18T03:06:52.878377+0000","last_became_active":"2023-07-18T03:04:59.806597+0000","last_became_peered":"2023-07-18T03:04:59.806597+0000","last_unstale":"2023-07-18T03:06:52.878377+0000","last_undegraded":"2023-07-18T03:06:52.878377+0000","last_fullsized":"2023-07-18T03:06:52.878377+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:23:04.609208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.876882+0000","last_change":"2023-07-18T03:04:59.809263+0000","last_active":"2023-07-18T03:06:52.876882+0000","last_peered":"2023-07-18T03:06:52.876882+0000","last_clean":"2023-07-18T03:06:52.876882+0000","last_became_active":"2023-07-18T03:04:59.808956+0000","last_became_peered":"2023-07-18T03:04:59.808956+0000","last_unstale":"2023-07-18T03:06:52.876882+0000","last_undegraded":"2023-07-18T03:06:52.876882+0000","last_fullsized":"2023-07-18T03:06:52.876882+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:22:13.425291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.881590+0000","last_change":"2023-07-18T03:04:59.804946+0000","last_active":"2023-07-18T03:06:52.881590+0000","last_peered":"2023-07-18T03:06:52.881590+0000","last_clean":"2023-07-18T03:06:52.881590+0000","last_became_active":"2023-07-18T03:04:59.804610+0000","last_became_peered":"2023-07-18T03:04:59.804610+0000","last_unstale":"2023-07-18T03:06:52.881590+0000","last_undegraded":"2023-07-18T03:06:52.881590+0000","last_fullsized":"2023-07-18T03:06:52.881590+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:35:53.147695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.886091+0000","last_change":"2023-07-18T03:04:59.801796+0000","last_active":"2023-07-18T03:06:52.886091+0000","last_peered":"2023-07-18T03:06:52.886091+0000","last_clean":"2023-07-18T03:06:52.886091+0000","last_became_active":"2023-07-18T03:04:59.801432+0000","last_became_peered":"2023-07-18T03:04:59.801432+0000","last_unstale":"2023-07-18T03:06:52.886091+0000","last_undegraded":"2023-07-18T03:06:52.886091+0000","last_fullsized":"2023-07-18T03:06:52.886091+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:32:07.392642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.887741+0000","last_change":"2023-07-18T03:04:59.798191+0000","last_active":"2023-07-18T03:06:52.887741+0000","last_peered":"2023-07-18T03:06:52.887741+0000","last_clean":"2023-07-18T03:06:52.887741+0000","last_became_active":"2023-07-18T03:04:59.797863+0000","last_became_peered":"2023-07-18T03:04:59.797863+0000","last_unstale":"2023-07-18T03:06:52.887741+0000","last_undegraded":"2023-07-18T03:06:52.887741+0000","last_fullsized":"2023-07-18T03:06:52.887741+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:43:29.304091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.881104+0000","last_change":"2023-07-18T03:04:59.809867+0000","last_active":"2023-07-18T03:06:52.881104+0000","last_peered":"2023-07-18T03:06:52.881104+0000","last_clean":"2023-07-18T03:06:52.881104+0000","last_became_active":"2023-07-18T03:04:59.809568+0000","last_became_peered":"2023-07-18T03:04:59.809568+0000","last_unstale":"2023-07-18T03:06:52.881104+0000","last_undegraded":"2023-07-18T03:06:52.881104+0000","last_fullsized":"2023-07-18T03:06:52.881104+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:49:15.008273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.886428+0000","last_change":"2023-07-18T03:04:59.805920+0000","last_active":"2023-07-18T03:06:52.886428+0000","last_peered":"2023-07-18T03:06:52.886428+0000","last_clean":"2023-07-18T03:06:52.886428+0000","last_became_active":"2023-07-18T03:04:59.805559+0000","last_became_peered":"2023-07-18T03:04:59.805559+0000","last_unstale":"2023-07-18T03:06:52.886428+0000","last_undegraded":"2023-07-18T03:06:52.886428+0000","last_fullsized":"2023-07-18T03:06:52.886428+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:27:52.167301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.880048+0000","last_change":"2023-07-18T03:04:59.797940+0000","last_active":"2023-07-18T03:06:52.880048+0000","last_peered":"2023-07-18T03:06:52.880048+0000","last_clean":"2023-07-18T03:06:52.880048+0000","last_became_active":"2023-07-18T03:04:59.797575+0000","last_became_peered":"2023-07-18T03:04:59.797575+0000","last_unstale":"2023-07-18T03:06:52.880048+0000","last_undegraded":"2023-07-18T03:06:52.880048+0000","last_fullsized":"2023-07-18T03:06:52.880048+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:58:43.120283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.894941+0000","last_change":"2023-07-18T03:04:59.798026+0000","last_active":"2023-07-18T03:06:52.894941+0000","last_peered":"2023-07-18T03:06:52.894941+0000","last_clean":"2023-07-18T03:06:52.894941+0000","last_became_active":"2023-07-18T03:04:59.797734+0000","last_became_peered":"2023-07-18T03:04:59.797734+0000","last_unstale":"2023-07-18T03:06:52.894941+0000","last_undegraded":"2023-07-18T03:06:52.894941+0000","last_fullsized":"2023-07-18T03:06:52.894941+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:33:40.010231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.886275+0000","last_change":"2023-07-18T03:04:59.804964+0000","last_active":"2023-07-18T03:06:52.886275+0000","last_peered":"2023-07-18T03:06:52.886275+0000","last_clean":"2023-07-18T03:06:52.886275+0000","last_became_active":"2023-07-18T03:04:59.804660+0000","last_became_peered":"2023-07-18T03:04:59.804660+0000","last_unstale":"2023-07-18T03:06:52.886275+0000","last_undegraded":"2023-07-18T03:06:52.886275+0000","last_fullsized":"2023-07-18T03:06:52.886275+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:11:48.365954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.885687+0000","last_change":"2023-07-18T03:04:59.809384+0000","last_active":"2023-07-18T03:06:52.885687+0000","last_peered":"2023-07-18T03:06:52.885687+0000","last_clean":"2023-07-18T03:06:52.885687+0000","last_became_active":"2023-07-18T03:04:59.808969+0000","last_became_peered":"2023-07-18T03:04:59.808969+0000","last_unstale":"2023-07-18T03:06:52.885687+0000","last_undegraded":"2023-07-18T03:06:52.885687+0000","last_fullsized":"2023-07-18T03:06:52.885687+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:16:18.639337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.884787+0000","last_change":"2023-07-18T03:04:59.808344+0000","last_active":"2023-07-18T03:06:52.884787+0000","last_peered":"2023-07-18T03:06:52.884787+0000","last_clean":"2023-07-18T03:06:52.884787+0000","last_became_active":"2023-07-18T03:04:59.807995+0000","last_became_peered":"2023-07-18T03:04:59.807995+0000","last_unstale":"2023-07-18T03:06:52.884787+0000","last_undegraded":"2023-07-18T03:06:52.884787+0000","last_fullsized":"2023-07-18T03:06:52.884787+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:15:44.462832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.887652+0000","last_change":"2023-07-18T03:04:59.806584+0000","last_active":"2023-07-18T03:06:52.887652+0000","last_peered":"2023-07-18T03:06:52.887652+0000","last_clean":"2023-07-18T03:06:52.887652+0000","last_became_active":"2023-07-18T03:04:59.804118+0000","last_became_peered":"2023-07-18T03:04:59.804118+0000","last_unstale":"2023-07-18T03:06:52.887652+0000","last_undegraded":"2023-07-18T03:06:52.887652+0000","last_fullsized":"2023-07-18T03:06:52.887652+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:42:18.047326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.890768+0000","last_change":"2023-07-18T03:04:59.812610+0000","last_active":"2023-07-18T03:06:52.890768+0000","last_peered":"2023-07-18T03:06:52.890768+0000","last_clean":"2023-07-18T03:06:52.890768+0000","last_became_active":"2023-07-18T03:04:59.812230+0000","last_became_peered":"2023-07-18T03:04:59.812230+0000","last_unstale":"2023-07-18T03:06:52.890768+0000","last_undegraded":"2023-07-18T03:06:52.890768+0000","last_fullsized":"2023-07-18T03:06:52.890768+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:07:54.810466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.893873+0000","last_change":"2023-07-18T03:04:59.811880+0000","last_active":"2023-07-18T03:06:52.893873+0000","last_peered":"2023-07-18T03:06:52.893873+0000","last_clean":"2023-07-18T03:06:52.893873+0000","last_became_active":"2023-07-18T03:04:59.811561+0000","last_became_peered":"2023-07-18T03:04:59.811561+0000","last_unstale":"2023-07-18T03:06:52.893873+0000","last_undegraded":"2023-07-18T03:06:52.893873+0000","last_fullsized":"2023-07-18T03:06:52.893873+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:53:25.407425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.256306+0000","last_change":"2023-07-18T03:05:03.880843+0000","last_active":"2023-07-18T03:06:24.256306+0000","last_peered":"2023-07-18T03:06:24.256306+0000","last_clean":"2023-07-18T03:06:24.256306+0000","last_became_active":"2023-07-18T03:05:03.880391+0000","last_became_peered":"2023-07-18T03:05:03.880391+0000","last_unstale":"2023-07-18T03:06:24.256306+0000","last_undegraded":"2023-07-18T03:06:24.256306+0000","last_fullsized":"2023-07-18T03:06:24.256306+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:25:52.190341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255219+0000","last_change":"2023-07-18T03:04:58.848771+0000","last_active":"2023-07-18T03:06:24.255219+0000","last_peered":"2023-07-18T03:06:24.255219+0000","last_clean":"2023-07-18T03:06:24.255219+0000","last_became_active":"2023-07-18T03:04:58.848405+0000","last_became_peered":"2023-07-18T03:04:58.848405+0000","last_unstale":"2023-07-18T03:06:24.255219+0000","last_undegraded":"2023-07-18T03:06:24.255219+0000","last_fullsized":"2023-07-18T03:06:24.255219+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:09:13.965066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.881139+0000","last_change":"2023-07-18T03:04:59.805749+0000","last_active":"2023-07-18T03:06:52.881139+0000","last_peered":"2023-07-18T03:06:52.881139+0000","last_clean":"2023-07-18T03:06:52.881139+0000","last_became_active":"2023-07-18T03:04:59.805388+0000","last_became_peered":"2023-07-18T03:04:59.805388+0000","last_unstale":"2023-07-18T03:06:52.881139+0000","last_undegraded":"2023-07-18T03:06:52.881139+0000","last_fullsized":"2023-07-18T03:06:52.881139+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:36:19.960742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255676+0000","last_change":"2023-07-18T03:05:03.868819+0000","last_active":"2023-07-18T03:06:24.255676+0000","last_peered":"2023-07-18T03:06:24.255676+0000","last_clean":"2023-07-18T03:06:24.255676+0000","last_became_active":"2023-07-18T03:05:03.868387+0000","last_became_peered":"2023-07-18T03:05:03.868387+0000","last_unstale":"2023-07-18T03:06:24.255676+0000","last_undegraded":"2023-07-18T03:06:24.255676+0000","last_fullsized":"2023-07-18T03:06:24.255676+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:34:30.099371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.375503+0000","last_change":"2023-07-18T03:04:58.856084+0000","last_active":"2023-07-18T03:06:24.375503+0000","last_peered":"2023-07-18T03:06:24.375503+0000","last_clean":"2023-07-18T03:06:24.375503+0000","last_became_active":"2023-07-18T03:04:58.855768+0000","last_became_peered":"2023-07-18T03:04:58.855768+0000","last_unstale":"2023-07-18T03:06:24.375503+0000","last_undegraded":"2023-07-18T03:06:24.375503+0000","last_fullsized":"2023-07-18T03:06:24.375503+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:58:35.159671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.885217+0000","last_change":"2023-07-18T03:04:59.809512+0000","last_active":"2023-07-18T03:06:52.885217+0000","last_peered":"2023-07-18T03:06:52.885217+0000","last_clean":"2023-07-18T03:06:52.885217+0000","last_became_active":"2023-07-18T03:04:59.808112+0000","last_became_peered":"2023-07-18T03:04:59.808112+0000","last_unstale":"2023-07-18T03:06:52.885217+0000","last_undegraded":"2023-07-18T03:06:52.885217+0000","last_fullsized":"2023-07-18T03:06:52.885217+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:42:34.279489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255283+0000","last_change":"2023-07-18T03:05:03.874728+0000","last_active":"2023-07-18T03:06:24.255283+0000","last_peered":"2023-07-18T03:06:24.255283+0000","last_clean":"2023-07-18T03:06:24.255283+0000","last_became_active":"2023-07-18T03:05:03.874262+0000","last_became_peered":"2023-07-18T03:05:03.874262+0000","last_unstale":"2023-07-18T03:06:24.255283+0000","last_undegraded":"2023-07-18T03:06:24.255283+0000","last_fullsized":"2023-07-18T03:06:24.255283+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:31:54.083323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.261317+0000","last_change":"2023-07-18T03:04:58.849140+0000","last_active":"2023-07-18T03:06:24.261317+0000","last_peered":"2023-07-18T03:06:24.261317+0000","last_clean":"2023-07-18T03:06:24.261317+0000","last_became_active":"2023-07-18T03:04:58.848693+0000","last_became_peered":"2023-07-18T03:04:58.848693+0000","last_unstale":"2023-07-18T03:06:24.261317+0000","last_undegraded":"2023-07-18T03:06:24.261317+0000","last_fullsized":"2023-07-18T03:06:24.261317+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:07:50.859417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.892017+0000","last_change":"2023-07-18T03:04:59.807248+0000","last_active":"2023-07-18T03:06:52.892017+0000","last_peered":"2023-07-18T03:06:52.892017+0000","last_clean":"2023-07-18T03:06:52.892017+0000","last_became_active":"2023-07-18T03:04:59.806807+0000","last_became_peered":"2023-07-18T03:04:59.806807+0000","last_unstale":"2023-07-18T03:06:52.892017+0000","last_undegraded":"2023-07-18T03:06:52.892017+0000","last_fullsized":"2023-07-18T03:06:52.892017+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:07:33.904712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255269+0000","last_change":"2023-07-18T03:05:03.869075+0000","last_active":"2023-07-18T03:06:24.255269+0000","last_peered":"2023-07-18T03:06:24.255269+0000","last_clean":"2023-07-18T03:06:24.255269+0000","last_became_active":"2023-07-18T03:05:03.868653+0000","last_became_peered":"2023-07-18T03:05:03.868653+0000","last_unstale":"2023-07-18T03:06:24.255269+0000","last_undegraded":"2023-07-18T03:06:24.255269+0000","last_fullsized":"2023-07-18T03:06:24.255269+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:05:18.894653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.259374+0000","last_change":"2023-07-18T03:04:58.843879+0000","last_active":"2023-07-18T03:06:24.259374+0000","last_peered":"2023-07-18T03:06:24.259374+0000","last_clean":"2023-07-18T03:06:24.259374+0000","last_became_active":"2023-07-18T03:04:58.843525+0000","last_became_peered":"2023-07-18T03:04:58.843525+0000","last_unstale":"2023-07-18T03:06:24.259374+0000","last_undegraded":"2023-07-18T03:06:24.259374+0000","last_fullsized":"2023-07-18T03:06:24.259374+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:32:03.232823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.886046+0000","last_change":"2023-07-18T03:04:59.803597+0000","last_active":"2023-07-18T03:06:52.886046+0000","last_peered":"2023-07-18T03:06:52.886046+0000","last_clean":"2023-07-18T03:06:52.886046+0000","last_became_active":"2023-07-18T03:04:59.803293+0000","last_became_peered":"2023-07-18T03:04:59.803293+0000","last_unstale":"2023-07-18T03:06:52.886046+0000","last_undegraded":"2023-07-18T03:06:52.886046+0000","last_fullsized":"2023-07-18T03:06:52.886046+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:31:55.196443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255711+0000","last_change":"2023-07-18T03:05:23.944721+0000","last_active":"2023-07-18T03:06:24.255711+0000","last_peered":"2023-07-18T03:06:24.255711+0000","last_clean":"2023-07-18T03:06:24.255711+0000","last_became_active":"2023-07-18T03:05:23.944101+0000","last_became_peered":"2023-07-18T03:05:23.944101+0000","last_unstale":"2023-07-18T03:06:24.255711+0000","last_undegraded":"2023-07-18T03:06:24.255711+0000","last_fullsized":"2023-07-18T03:06:24.255711+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:20:50.819953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":145,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:07:14.145822+0000","last_change":"2023-07-18T03:04:58.847990+0000","last_active":"2023-07-18T03:07:14.145822+0000","last_peered":"2023-07-18T03:07:14.145822+0000","last_clean":"2023-07-18T03:07:14.145822+0000","last_became_active":"2023-07-18T03:04:58.847634+0000","last_became_peered":"2023-07-18T03:04:58.847634+0000","last_unstale":"2023-07-18T03:07:14.145822+0000","last_undegraded":"2023-07-18T03:07:14.145822+0000","last_fullsized":"2023-07-18T03:07:14.145822+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:14:18.454279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1105,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.881401+0000","last_change":"2023-07-18T03:04:59.797076+0000","last_active":"2023-07-18T03:06:52.881401+0000","last_peered":"2023-07-18T03:06:52.881401+0000","last_clean":"2023-07-18T03:06:52.881401+0000","last_became_active":"2023-07-18T03:04:59.796753+0000","last_became_peered":"2023-07-18T03:04:59.796753+0000","last_unstale":"2023-07-18T03:06:52.881401+0000","last_undegraded":"2023-07-18T03:06:52.881401+0000","last_fullsized":"2023-07-18T03:06:52.881401+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:46:03.085633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255734+0000","last_change":"2023-07-18T03:05:03.881793+0000","last_active":"2023-07-18T03:06:24.255734+0000","last_peered":"2023-07-18T03:06:24.255734+0000","last_clean":"2023-07-18T03:06:24.255734+0000","last_became_active":"2023-07-18T03:05:03.881341+0000","last_became_peered":"2023-07-18T03:05:03.881341+0000","last_unstale":"2023-07-18T03:06:24.255734+0000","last_undegraded":"2023-07-18T03:06:24.255734+0000","last_fullsized":"2023-07-18T03:06:24.255734+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:40:23.370420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.375558+0000","last_change":"2023-07-18T03:04:58.846732+0000","last_active":"2023-07-18T03:06:24.375558+0000","last_peered":"2023-07-18T03:06:24.375558+0000","last_clean":"2023-07-18T03:06:24.375558+0000","last_became_active":"2023-07-18T03:04:58.846321+0000","last_became_peered":"2023-07-18T03:04:58.846321+0000","last_unstale":"2023-07-18T03:06:24.375558+0000","last_undegraded":"2023-07-18T03:06:24.375558+0000","last_fullsized":"2023-07-18T03:06:24.375558+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:50:00.381631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.879809+0000","last_change":"2023-07-18T03:05:24.093277+0000","last_active":"2023-07-18T03:06:52.879809+0000","last_peered":"2023-07-18T03:06:52.879809+0000","last_clean":"2023-07-18T03:06:52.879809+0000","last_became_active":"2023-07-18T03:05:24.092842+0000","last_became_peered":"2023-07-18T03:05:24.092842+0000","last_unstale":"2023-07-18T03:06:52.879809+0000","last_undegraded":"2023-07-18T03:06:52.879809+0000","last_fullsized":"2023-07-18T03:06:52.879809+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:22:01.737422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.257633+0000","last_change":"2023-07-18T03:05:03.880782+0000","last_active":"2023-07-18T03:06:24.257633+0000","last_peered":"2023-07-18T03:06:24.257633+0000","last_clean":"2023-07-18T03:06:24.257633+0000","last_became_active":"2023-07-18T03:05:03.880362+0000","last_became_peered":"2023-07-18T03:05:03.880362+0000","last_unstale":"2023-07-18T03:06:24.257633+0000","last_undegraded":"2023-07-18T03:06:24.257633+0000","last_fullsized":"2023-07-18T03:06:24.257633+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:10:14.772789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.750818+0000","last_change":"2023-07-18T03:04:58.846095+0000","last_active":"2023-07-18T03:06:24.750818+0000","last_peered":"2023-07-18T03:06:24.750818+0000","last_clean":"2023-07-18T03:06:24.750818+0000","last_became_active":"2023-07-18T03:04:58.845242+0000","last_became_peered":"2023-07-18T03:04:58.845242+0000","last_unstale":"2023-07-18T03:06:24.750818+0000","last_undegraded":"2023-07-18T03:06:24.750818+0000","last_fullsized":"2023-07-18T03:06:24.750818+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:36:00.462709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.889059+0000","last_change":"2023-07-18T03:04:59.803024+0000","last_active":"2023-07-18T03:06:52.889059+0000","last_peered":"2023-07-18T03:06:52.889059+0000","last_clean":"2023-07-18T03:06:52.889059+0000","last_became_active":"2023-07-18T03:04:59.802687+0000","last_became_peered":"2023-07-18T03:04:59.802687+0000","last_unstale":"2023-07-18T03:06:52.889059+0000","last_undegraded":"2023-07-18T03:06:52.889059+0000","last_fullsized":"2023-07-18T03:06:52.889059+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:59:27.108157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.259440+0000","last_change":"2023-07-18T03:05:03.872690+0000","last_active":"2023-07-18T03:06:24.259440+0000","last_peered":"2023-07-18T03:06:24.259440+0000","last_clean":"2023-07-18T03:06:24.259440+0000","last_became_active":"2023-07-18T03:05:03.872187+0000","last_became_peered":"2023-07-18T03:05:03.872187+0000","last_unstale":"2023-07-18T03:06:24.259440+0000","last_undegraded":"2023-07-18T03:06:24.259440+0000","last_fullsized":"2023-07-18T03:06:24.259440+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:06:57.712294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.261033+0000","last_change":"2023-07-18T03:04:58.842877+0000","last_active":"2023-07-18T03:06:24.261033+0000","last_peered":"2023-07-18T03:06:24.261033+0000","last_clean":"2023-07-18T03:06:24.261033+0000","last_became_active":"2023-07-18T03:04:58.842516+0000","last_became_peered":"2023-07-18T03:04:58.842516+0000","last_unstale":"2023-07-18T03:06:24.261033+0000","last_undegraded":"2023-07-18T03:06:24.261033+0000","last_fullsized":"2023-07-18T03:06:24.261033+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:24:49.503369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.884379+0000","last_change":"2023-07-18T03:04:59.807592+0000","last_active":"2023-07-18T03:06:52.884379+0000","last_peered":"2023-07-18T03:06:52.884379+0000","last_clean":"2023-07-18T03:06:52.884379+0000","last_became_active":"2023-07-18T03:04:59.807276+0000","last_became_peered":"2023-07-18T03:04:59.807276+0000","last_unstale":"2023-07-18T03:06:52.884379+0000","last_undegraded":"2023-07-18T03:06:52.884379+0000","last_fullsized":"2023-07-18T03:06:52.884379+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:16:05.749019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.374960+0000","last_change":"2023-07-18T03:05:03.876355+0000","last_active":"2023-07-18T03:06:24.374960+0000","last_peered":"2023-07-18T03:06:24.374960+0000","last_clean":"2023-07-18T03:06:24.374960+0000","last_became_active":"2023-07-18T03:05:03.875845+0000","last_became_peered":"2023-07-18T03:05:03.875845+0000","last_unstale":"2023-07-18T03:06:24.374960+0000","last_undegraded":"2023-07-18T03:06:24.374960+0000","last_fullsized":"2023-07-18T03:06:24.374960+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:13:06.076832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'4","reported_seq":139,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:07:15.146806+0000","last_change":"2023-07-18T03:04:58.840389+0000","last_active":"2023-07-18T03:07:15.146806+0000","last_peered":"2023-07-18T03:07:15.146806+0000","last_clean":"2023-07-18T03:07:15.146806+0000","last_became_active":"2023-07-18T03:04:58.839974+0000","last_became_peered":"2023-07-18T03:04:58.839974+0000","last_unstale":"2023-07-18T03:07:15.146806+0000","last_undegraded":"2023-07-18T03:07:15.146806+0000","last_fullsized":"2023-07-18T03:07:15.146806+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:40:06.944095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.891931+0000","last_change":"2023-07-18T03:04:59.807681+0000","last_active":"2023-07-18T03:06:52.891931+0000","last_peered":"2023-07-18T03:06:52.891931+0000","last_clean":"2023-07-18T03:06:52.891931+0000","last_became_active":"2023-07-18T03:04:59.807365+0000","last_became_peered":"2023-07-18T03:04:59.807365+0000","last_unstale":"2023-07-18T03:06:52.891931+0000","last_undegraded":"2023-07-18T03:06:52.891931+0000","last_fullsized":"2023-07-18T03:06:52.891931+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:02:22.715114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.258920+0000","last_change":"2023-07-18T03:05:24.091413+0000","last_active":"2023-07-18T03:06:24.258920+0000","last_peered":"2023-07-18T03:06:24.258920+0000","last_clean":"2023-07-18T03:06:24.258920+0000","last_became_active":"2023-07-18T03:05:24.090815+0000","last_became_peered":"2023-07-18T03:05:24.090815+0000","last_unstale":"2023-07-18T03:06:24.258920+0000","last_undegraded":"2023-07-18T03:06:24.258920+0000","last_fullsized":"2023-07-18T03:06:24.258920+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:25:43.324509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.268967+0000","last_change":"2023-07-18T03:06:24.268967+0000","last_active":"2023-07-18T03:06:24.268967+0000","last_peered":"2023-07-18T03:06:24.268967+0000","last_clean":"2023-07-18T03:06:24.268967+0000","last_became_active":"2023-07-18T03:06:24.263424+0000","last_became_peered":"2023-07-18T03:06:24.263424+0000","last_unstale":"2023-07-18T03:06:24.268967+0000","last_undegraded":"2023-07-18T03:06:24.268967+0000","last_fullsized":"2023-07-18T03:06:24.268967+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:13:59.331501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.895054+0000","last_change":"2023-07-18T03:05:24.092870+0000","last_active":"2023-07-18T03:06:52.895054+0000","last_peered":"2023-07-18T03:06:52.895054+0000","last_clean":"2023-07-18T03:06:52.895054+0000","last_became_active":"2023-07-18T03:05:24.092474+0000","last_became_peered":"2023-07-18T03:05:24.092474+0000","last_unstale":"2023-07-18T03:06:52.895054+0000","last_undegraded":"2023-07-18T03:06:52.895054+0000","last_fullsized":"2023-07-18T03:06:52.895054+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:25:17.349689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255481+0000","last_change":"2023-07-18T03:05:03.874444+0000","last_active":"2023-07-18T03:06:24.255481+0000","last_peered":"2023-07-18T03:06:24.255481+0000","last_clean":"2023-07-18T03:06:24.255481+0000","last_became_active":"2023-07-18T03:05:03.873737+0000","last_became_peered":"2023-07-18T03:05:03.873737+0000","last_unstale":"2023-07-18T03:06:24.255481+0000","last_undegraded":"2023-07-18T03:06:24.255481+0000","last_fullsized":"2023-07-18T03:06:24.255481+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:07:21.954472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.261721+0000","last_change":"2023-07-18T03:04:58.845207+0000","last_active":"2023-07-18T03:06:24.261721+0000","last_peered":"2023-07-18T03:06:24.261721+0000","last_clean":"2023-07-18T03:06:24.261721+0000","last_became_active":"2023-07-18T03:04:58.844895+0000","last_became_peered":"2023-07-18T03:04:58.844895+0000","last_unstale":"2023-07-18T03:06:24.261721+0000","last_undegraded":"2023-07-18T03:06:24.261721+0000","last_fullsized":"2023-07-18T03:06:24.261721+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:59:56.250544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.886957+0000","last_change":"2023-07-18T03:04:59.807075+0000","last_active":"2023-07-18T03:06:52.886957+0000","last_peered":"2023-07-18T03:06:52.886957+0000","last_clean":"2023-07-18T03:06:52.886957+0000","last_became_active":"2023-07-18T03:04:59.806345+0000","last_became_peered":"2023-07-18T03:04:59.806345+0000","last_unstale":"2023-07-18T03:06:52.886957+0000","last_undegraded":"2023-07-18T03:06:52.886957+0000","last_fullsized":"2023-07-18T03:06:52.886957+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:50:55.760441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.375376+0000","last_change":"2023-07-18T03:05:24.091132+0000","last_active":"2023-07-18T03:06:24.375376+0000","last_peered":"2023-07-18T03:06:24.375376+0000","last_clean":"2023-07-18T03:06:24.375376+0000","last_became_active":"2023-07-18T03:05:24.090653+0000","last_became_peered":"2023-07-18T03:05:24.090653+0000","last_unstale":"2023-07-18T03:06:24.375376+0000","last_undegraded":"2023-07-18T03:06:24.375376+0000","last_fullsized":"2023-07-18T03:06:24.375376+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:14:06.294348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"91'38","reported_seq":289,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:07:16.164902+0000","last_change":"2023-07-18T03:04:58.844865+0000","last_active":"2023-07-18T03:07:16.164902+0000","last_peered":"2023-07-18T03:07:16.164902+0000","last_clean":"2023-07-18T03:07:16.164902+0000","last_became_active":"2023-07-18T03:04:58.844547+0000","last_became_peered":"2023-07-18T03:04:58.844547+0000","last_unstale":"2023-07-18T03:07:16.164902+0000","last_undegraded":"2023-07-18T03:07:16.164902+0000","last_fullsized":"2023-07-18T03:07:16.164902+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:09:40.602341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66781,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":221,"num_read_kb":176,"num_write":39,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.882102+0000","last_change":"2023-07-18T03:04:59.803024+0000","last_active":"2023-07-18T03:06:52.882102+0000","last_peered":"2023-07-18T03:06:52.882102+0000","last_clean":"2023-07-18T03:06:52.882102+0000","last_became_active":"2023-07-18T03:04:59.802705+0000","last_became_peered":"2023-07-18T03:04:59.802705+0000","last_unstale":"2023-07-18T03:06:52.882102+0000","last_undegraded":"2023-07-18T03:06:52.882102+0000","last_fullsized":"2023-07-18T03:06:52.882102+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:46:31.950692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.262478+0000","last_change":"2023-07-18T03:05:03.876009+0000","last_active":"2023-07-18T03:06:24.262478+0000","last_peered":"2023-07-18T03:06:24.262478+0000","last_clean":"2023-07-18T03:06:24.262478+0000","last_became_active":"2023-07-18T03:05:03.875328+0000","last_became_peered":"2023-07-18T03:05:03.875328+0000","last_unstale":"2023-07-18T03:06:24.262478+0000","last_undegraded":"2023-07-18T03:06:24.262478+0000","last_fullsized":"2023-07-18T03:06:24.262478+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:55:54.930973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.256224+0000","last_change":"2023-07-18T03:04:58.846573+0000","last_active":"2023-07-18T03:06:24.256224+0000","last_peered":"2023-07-18T03:06:24.256224+0000","last_clean":"2023-07-18T03:06:24.256224+0000","last_became_active":"2023-07-18T03:04:58.846271+0000","last_became_peered":"2023-07-18T03:04:58.846271+0000","last_unstale":"2023-07-18T03:06:24.256224+0000","last_undegraded":"2023-07-18T03:06:24.256224+0000","last_fullsized":"2023-07-18T03:06:24.256224+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:22:31.518590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.894308+0000","last_change":"2023-07-18T03:04:59.801552+0000","last_active":"2023-07-18T03:06:52.894308+0000","last_peered":"2023-07-18T03:06:52.894308+0000","last_clean":"2023-07-18T03:06:52.894308+0000","last_became_active":"2023-07-18T03:04:59.801224+0000","last_became_peered":"2023-07-18T03:04:59.801224+0000","last_unstale":"2023-07-18T03:06:52.894308+0000","last_undegraded":"2023-07-18T03:06:52.894308+0000","last_fullsized":"2023-07-18T03:06:52.894308+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:44:10.394026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.257271+0000","last_change":"2023-07-18T03:05:03.881125+0000","last_active":"2023-07-18T03:06:24.257271+0000","last_peered":"2023-07-18T03:06:24.257271+0000","last_clean":"2023-07-18T03:06:24.257271+0000","last_became_active":"2023-07-18T03:05:03.880723+0000","last_became_peered":"2023-07-18T03:05:03.880723+0000","last_unstale":"2023-07-18T03:06:24.257271+0000","last_undegraded":"2023-07-18T03:06:24.257271+0000","last_fullsized":"2023-07-18T03:06:24.257271+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:28:40.857218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.261012+0000","last_change":"2023-07-18T03:04:58.847118+0000","last_active":"2023-07-18T03:06:24.261012+0000","last_peered":"2023-07-18T03:06:24.261012+0000","last_clean":"2023-07-18T03:06:24.261012+0000","last_became_active":"2023-07-18T03:04:58.846805+0000","last_became_peered":"2023-07-18T03:04:58.846805+0000","last_unstale":"2023-07-18T03:06:24.261012+0000","last_undegraded":"2023-07-18T03:06:24.261012+0000","last_fullsized":"2023-07-18T03:06:24.261012+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:48:50.417535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.895015+0000","last_change":"2023-07-18T03:04:59.803914+0000","last_active":"2023-07-18T03:06:52.895015+0000","last_peered":"2023-07-18T03:06:52.895015+0000","last_clean":"2023-07-18T03:06:52.895015+0000","last_became_active":"2023-07-18T03:04:59.803589+0000","last_became_peered":"2023-07-18T03:04:59.803589+0000","last_unstale":"2023-07-18T03:06:52.895015+0000","last_undegraded":"2023-07-18T03:06:52.895015+0000","last_fullsized":"2023-07-18T03:06:52.895015+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:24:46.537710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255012+0000","last_change":"2023-07-18T03:05:03.877787+0000","last_active":"2023-07-18T03:06:24.255012+0000","last_peered":"2023-07-18T03:06:24.255012+0000","last_clean":"2023-07-18T03:06:24.255012+0000","last_became_active":"2023-07-18T03:05:03.877449+0000","last_became_peered":"2023-07-18T03:05:03.877449+0000","last_unstale":"2023-07-18T03:06:24.255012+0000","last_undegraded":"2023-07-18T03:06:24.255012+0000","last_fullsized":"2023-07-18T03:06:24.255012+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:54:47.075087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":137,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:07:12.144954+0000","last_change":"2023-07-18T03:04:58.844252+0000","last_active":"2023-07-18T03:07:12.144954+0000","last_peered":"2023-07-18T03:07:12.144954+0000","last_clean":"2023-07-18T03:07:12.144954+0000","last_became_active":"2023-07-18T03:04:58.843798+0000","last_became_peered":"2023-07-18T03:04:58.843798+0000","last_unstale":"2023-07-18T03:07:12.144954+0000","last_undegraded":"2023-07-18T03:07:12.144954+0000","last_fullsized":"2023-07-18T03:07:12.144954+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:24:14.970709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.883399+0000","last_change":"2023-07-18T03:04:59.806274+0000","last_active":"2023-07-18T03:06:52.883399+0000","last_peered":"2023-07-18T03:06:52.883399+0000","last_clean":"2023-07-18T03:06:52.883399+0000","last_became_active":"2023-07-18T03:04:59.805876+0000","last_became_peered":"2023-07-18T03:04:59.805876+0000","last_unstale":"2023-07-18T03:06:52.883399+0000","last_undegraded":"2023-07-18T03:06:52.883399+0000","last_fullsized":"2023-07-18T03:06:52.883399+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:00:53.965603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.751210+0000","last_change":"2023-07-18T03:05:03.876783+0000","last_active":"2023-07-18T03:06:24.751210+0000","last_peered":"2023-07-18T03:06:24.751210+0000","last_clean":"2023-07-18T03:06:24.751210+0000","last_became_active":"2023-07-18T03:05:03.876385+0000","last_became_peered":"2023-07-18T03:05:03.876385+0000","last_unstale":"2023-07-18T03:06:24.751210+0000","last_undegraded":"2023-07-18T03:06:24.751210+0000","last_fullsized":"2023-07-18T03:06:24.751210+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:12:22.516248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.259648+0000","last_change":"2023-07-18T03:04:58.851107+0000","last_active":"2023-07-18T03:06:24.259648+0000","last_peered":"2023-07-18T03:06:24.259648+0000","last_clean":"2023-07-18T03:06:24.259648+0000","last_became_active":"2023-07-18T03:04:58.850848+0000","last_became_peered":"2023-07-18T03:04:58.850848+0000","last_unstale":"2023-07-18T03:06:24.259648+0000","last_undegraded":"2023-07-18T03:06:24.259648+0000","last_fullsized":"2023-07-18T03:06:24.259648+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:39:44.591519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.885106+0000","last_change":"2023-07-18T03:04:59.803683+0000","last_active":"2023-07-18T03:06:52.885106+0000","last_peered":"2023-07-18T03:06:52.885106+0000","last_clean":"2023-07-18T03:06:52.885106+0000","last_became_active":"2023-07-18T03:04:59.803365+0000","last_became_peered":"2023-07-18T03:04:59.803365+0000","last_unstale":"2023-07-18T03:06:52.885106+0000","last_undegraded":"2023-07-18T03:06:52.885106+0000","last_fullsized":"2023-07-18T03:06:52.885106+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:30:10.236288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.260879+0000","last_change":"2023-07-18T03:05:03.875856+0000","last_active":"2023-07-18T03:06:24.260879+0000","last_peered":"2023-07-18T03:06:24.260879+0000","last_clean":"2023-07-18T03:06:24.260879+0000","last_became_active":"2023-07-18T03:05:03.875426+0000","last_became_peered":"2023-07-18T03:05:03.875426+0000","last_unstale":"2023-07-18T03:06:24.260879+0000","last_undegraded":"2023-07-18T03:06:24.260879+0000","last_fullsized":"2023-07-18T03:06:24.260879+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:48:39.948769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.261226+0000","last_change":"2023-07-18T03:04:58.844102+0000","last_active":"2023-07-18T03:06:24.261226+0000","last_peered":"2023-07-18T03:06:24.261226+0000","last_clean":"2023-07-18T03:06:24.261226+0000","last_became_active":"2023-07-18T03:04:58.843740+0000","last_became_peered":"2023-07-18T03:04:58.843740+0000","last_unstale":"2023-07-18T03:06:24.261226+0000","last_undegraded":"2023-07-18T03:06:24.261226+0000","last_fullsized":"2023-07-18T03:06:24.261226+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:19:46.540756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.883036+0000","last_change":"2023-07-18T03:05:24.091161+0000","last_active":"2023-07-18T03:06:52.883036+0000","last_peered":"2023-07-18T03:06:52.883036+0000","last_clean":"2023-07-18T03:06:52.883036+0000","last_became_active":"2023-07-18T03:05:24.090631+0000","last_became_peered":"2023-07-18T03:05:24.090631+0000","last_unstale":"2023-07-18T03:06:52.883036+0000","last_undegraded":"2023-07-18T03:06:52.883036+0000","last_fullsized":"2023-07-18T03:06:52.883036+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:21:10.676261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.259807+0000","last_change":"2023-07-18T03:05:24.092328+0000","last_active":"2023-07-18T03:06:24.259807+0000","last_peered":"2023-07-18T03:06:24.259807+0000","last_clean":"2023-07-18T03:06:24.259807+0000","last_became_active":"2023-07-18T03:05:24.091839+0000","last_became_peered":"2023-07-18T03:05:24.091839+0000","last_unstale":"2023-07-18T03:06:24.259807+0000","last_undegraded":"2023-07-18T03:06:24.259807+0000","last_fullsized":"2023-07-18T03:06:24.259807+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:42:18.354994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.262294+0000","last_change":"2023-07-18T03:06:24.261616+0000","last_active":"2023-07-18T03:06:24.262294+0000","last_peered":"2023-07-18T03:06:24.262294+0000","last_clean":"2023-07-18T03:06:24.262294+0000","last_became_active":"2023-07-18T03:06:24.261250+0000","last_became_peered":"2023-07-18T03:06:24.261250+0000","last_unstale":"2023-07-18T03:06:24.262294+0000","last_undegraded":"2023-07-18T03:06:24.262294+0000","last_fullsized":"2023-07-18T03:06:24.262294+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:19:24.589920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.893730+0000","last_change":"2023-07-18T03:05:24.092486+0000","last_active":"2023-07-18T03:06:52.893730+0000","last_peered":"2023-07-18T03:06:52.893730+0000","last_clean":"2023-07-18T03:06:52.893730+0000","last_became_active":"2023-07-18T03:05:24.092189+0000","last_became_peered":"2023-07-18T03:05:24.092189+0000","last_unstale":"2023-07-18T03:06:52.893730+0000","last_undegraded":"2023-07-18T03:06:52.893730+0000","last_fullsized":"2023-07-18T03:06:52.893730+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:09:16.496072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.751014+0000","last_change":"2023-07-18T03:05:03.868851+0000","last_active":"2023-07-18T03:06:24.751014+0000","last_peered":"2023-07-18T03:06:24.751014+0000","last_clean":"2023-07-18T03:06:24.751014+0000","last_became_active":"2023-07-18T03:05:03.868433+0000","last_became_peered":"2023-07-18T03:05:03.868433+0000","last_unstale":"2023-07-18T03:06:24.751014+0000","last_undegraded":"2023-07-18T03:06:24.751014+0000","last_fullsized":"2023-07-18T03:06:24.751014+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:36:46.180540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.258040+0000","last_change":"2023-07-18T03:04:58.848059+0000","last_active":"2023-07-18T03:06:24.258040+0000","last_peered":"2023-07-18T03:06:24.258040+0000","last_clean":"2023-07-18T03:06:24.258040+0000","last_became_active":"2023-07-18T03:04:58.847700+0000","last_became_peered":"2023-07-18T03:04:58.847700+0000","last_unstale":"2023-07-18T03:06:24.258040+0000","last_undegraded":"2023-07-18T03:06:24.258040+0000","last_fullsized":"2023-07-18T03:06:24.258040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:22:42.461972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.886520+0000","last_change":"2023-07-18T03:05:23.939939+0000","last_active":"2023-07-18T03:06:52.886520+0000","last_peered":"2023-07-18T03:06:52.886520+0000","last_clean":"2023-07-18T03:06:52.886520+0000","last_became_active":"2023-07-18T03:05:23.939607+0000","last_became_peered":"2023-07-18T03:05:23.939607+0000","last_unstale":"2023-07-18T03:06:52.886520+0000","last_undegraded":"2023-07-18T03:06:52.886520+0000","last_fullsized":"2023-07-18T03:06:52.886520+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:47:48.796749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.751185+0000","last_change":"2023-07-18T03:05:03.886724+0000","last_active":"2023-07-18T03:06:24.751185+0000","last_peered":"2023-07-18T03:06:24.751185+0000","last_clean":"2023-07-18T03:06:24.751185+0000","last_became_active":"2023-07-18T03:05:03.886172+0000","last_became_peered":"2023-07-18T03:05:03.886172+0000","last_unstale":"2023-07-18T03:06:24.751185+0000","last_undegraded":"2023-07-18T03:06:24.751185+0000","last_fullsized":"2023-07-18T03:06:24.751185+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:35:26.182278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.886032+0000","last_change":"2023-07-18T03:04:59.804273+0000","last_active":"2023-07-18T03:06:52.886032+0000","last_peered":"2023-07-18T03:06:52.886032+0000","last_clean":"2023-07-18T03:06:52.886032+0000","last_became_active":"2023-07-18T03:04:59.803987+0000","last_became_peered":"2023-07-18T03:04:59.803987+0000","last_unstale":"2023-07-18T03:06:52.886032+0000","last_undegraded":"2023-07-18T03:06:52.886032+0000","last_fullsized":"2023-07-18T03:06:52.886032+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:53:36.335539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":139,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:07:14.146943+0000","last_change":"2023-07-18T03:04:58.846381+0000","last_active":"2023-07-18T03:07:14.146943+0000","last_peered":"2023-07-18T03:07:14.146943+0000","last_clean":"2023-07-18T03:07:14.146943+0000","last_became_active":"2023-07-18T03:04:58.845757+0000","last_became_peered":"2023-07-18T03:04:58.845757+0000","last_unstale":"2023-07-18T03:07:14.146943+0000","last_undegraded":"2023-07-18T03:07:14.146943+0000","last_fullsized":"2023-07-18T03:07:14.146943+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:00:14.653137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.258055+0000","last_change":"2023-07-18T03:05:03.873626+0000","last_active":"2023-07-18T03:06:24.258055+0000","last_peered":"2023-07-18T03:06:24.258055+0000","last_clean":"2023-07-18T03:06:24.258055+0000","last_became_active":"2023-07-18T03:05:03.872965+0000","last_became_peered":"2023-07-18T03:05:03.872965+0000","last_unstale":"2023-07-18T03:06:24.258055+0000","last_undegraded":"2023-07-18T03:06:24.258055+0000","last_fullsized":"2023-07-18T03:06:24.258055+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:37:38.450618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.881350+0000","last_change":"2023-07-18T03:04:59.801153+0000","last_active":"2023-07-18T03:06:52.881350+0000","last_peered":"2023-07-18T03:06:52.881350+0000","last_clean":"2023-07-18T03:06:52.881350+0000","last_became_active":"2023-07-18T03:04:59.800798+0000","last_became_peered":"2023-07-18T03:04:59.800798+0000","last_unstale":"2023-07-18T03:06:52.881350+0000","last_undegraded":"2023-07-18T03:06:52.881350+0000","last_fullsized":"2023-07-18T03:06:52.881350+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:49:57.147042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255477+0000","last_change":"2023-07-18T03:04:58.846047+0000","last_active":"2023-07-18T03:06:24.255477+0000","last_peered":"2023-07-18T03:06:24.255477+0000","last_clean":"2023-07-18T03:06:24.255477+0000","last_became_active":"2023-07-18T03:04:58.845678+0000","last_became_peered":"2023-07-18T03:04:58.845678+0000","last_unstale":"2023-07-18T03:06:24.255477+0000","last_undegraded":"2023-07-18T03:06:24.255477+0000","last_fullsized":"2023-07-18T03:06:24.255477+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:50:39.012582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":385,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.257405+0000","last_change":"2023-07-18T03:03:54.168837+0000","last_active":"2023-07-18T03:06:24.257405+0000","last_peered":"2023-07-18T03:06:24.257405+0000","last_clean":"2023-07-18T03:06:24.257405+0000","last_became_active":"2023-07-18T03:03:54.168298+0000","last_became_peered":"2023-07-18T03:03:54.168298+0000","last_unstale":"2023-07-18T03:06:24.257405+0000","last_undegraded":"2023-07-18T03:06:24.257405+0000","last_fullsized":"2023-07-18T03:06:24.257405+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:00:43.469168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:00:43.469168+0000","last_clean_scrub_stamp":"2023-07-18T03:00:43.469168+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:03:45.014912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.261432+0000","last_change":"2023-07-18T03:05:03.886611+0000","last_active":"2023-07-18T03:06:24.261432+0000","last_peered":"2023-07-18T03:06:24.261432+0000","last_clean":"2023-07-18T03:06:24.261432+0000","last_became_active":"2023-07-18T03:05:03.886099+0000","last_became_peered":"2023-07-18T03:05:03.886099+0000","last_unstale":"2023-07-18T03:06:24.261432+0000","last_undegraded":"2023-07-18T03:06:24.261432+0000","last_fullsized":"2023-07-18T03:06:24.261432+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:26:40.808017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.893733+0000","last_change":"2023-07-18T03:04:59.804478+0000","last_active":"2023-07-18T03:06:52.893733+0000","last_peered":"2023-07-18T03:06:52.893733+0000","last_clean":"2023-07-18T03:06:52.893733+0000","last_became_active":"2023-07-18T03:04:59.804162+0000","last_became_peered":"2023-07-18T03:04:59.804162+0000","last_unstale":"2023-07-18T03:06:52.893733+0000","last_undegraded":"2023-07-18T03:06:52.893733+0000","last_fullsized":"2023-07-18T03:06:52.893733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:41:57.294071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.267166+0000","last_change":"2023-07-18T03:06:24.266708+0000","last_active":"2023-07-18T03:06:24.267166+0000","last_peered":"2023-07-18T03:06:24.267166+0000","last_clean":"2023-07-18T03:06:24.267166+0000","last_became_active":"2023-07-18T03:06:24.262606+0000","last_became_peered":"2023-07-18T03:06:24.262606+0000","last_unstale":"2023-07-18T03:06:24.267166+0000","last_undegraded":"2023-07-18T03:06:24.267166+0000","last_fullsized":"2023-07-18T03:06:24.267166+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:20:59.875839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.257365+0000","last_change":"2023-07-18T03:05:03.872788+0000","last_active":"2023-07-18T03:06:24.257365+0000","last_peered":"2023-07-18T03:06:24.257365+0000","last_clean":"2023-07-18T03:06:24.257365+0000","last_became_active":"2023-07-18T03:05:03.872343+0000","last_became_peered":"2023-07-18T03:05:03.872343+0000","last_unstale":"2023-07-18T03:06:24.257365+0000","last_undegraded":"2023-07-18T03:06:24.257365+0000","last_fullsized":"2023-07-18T03:06:24.257365+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:20:43.912026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.266064+0000","last_change":"2023-07-18T03:06:24.265727+0000","last_active":"2023-07-18T03:06:24.266064+0000","last_peered":"2023-07-18T03:06:24.266064+0000","last_clean":"2023-07-18T03:06:24.266064+0000","last_became_active":"2023-07-18T03:06:24.262373+0000","last_became_peered":"2023-07-18T03:06:24.262373+0000","last_unstale":"2023-07-18T03:06:24.266064+0000","last_undegraded":"2023-07-18T03:06:24.266064+0000","last_fullsized":"2023-07-18T03:06:24.266064+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:38:45.610472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.893085+0000","last_change":"2023-07-18T03:04:59.804706+0000","last_active":"2023-07-18T03:06:52.893085+0000","last_peered":"2023-07-18T03:06:52.893085+0000","last_clean":"2023-07-18T03:06:52.893085+0000","last_became_active":"2023-07-18T03:04:59.804445+0000","last_became_peered":"2023-07-18T03:04:59.804445+0000","last_unstale":"2023-07-18T03:06:52.893085+0000","last_undegraded":"2023-07-18T03:06:52.893085+0000","last_fullsized":"2023-07-18T03:06:52.893085+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:54:52.824231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.374912+0000","last_change":"2023-07-18T03:05:03.886065+0000","last_active":"2023-07-18T03:06:24.374912+0000","last_peered":"2023-07-18T03:06:24.374912+0000","last_clean":"2023-07-18T03:06:24.374912+0000","last_became_active":"2023-07-18T03:05:03.885625+0000","last_became_peered":"2023-07-18T03:05:03.885625+0000","last_unstale":"2023-07-18T03:06:24.374912+0000","last_undegraded":"2023-07-18T03:06:24.374912+0000","last_fullsized":"2023-07-18T03:06:24.374912+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:32:31.156943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:07:10.696055+0000","last_change":"2023-07-18T03:04:58.849246+0000","last_active":"2023-07-18T03:07:10.696055+0000","last_peered":"2023-07-18T03:07:10.696055+0000","last_clean":"2023-07-18T03:07:10.696055+0000","last_became_active":"2023-07-18T03:04:58.848888+0000","last_became_peered":"2023-07-18T03:04:58.848888+0000","last_unstale":"2023-07-18T03:07:10.696055+0000","last_undegraded":"2023-07-18T03:07:10.696055+0000","last_fullsized":"2023-07-18T03:07:10.696055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:47:14.002970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.883435+0000","last_change":"2023-07-18T03:04:59.804561+0000","last_active":"2023-07-18T03:06:52.883435+0000","last_peered":"2023-07-18T03:06:52.883435+0000","last_clean":"2023-07-18T03:06:52.883435+0000","last_became_active":"2023-07-18T03:04:59.803659+0000","last_became_peered":"2023-07-18T03:04:59.803659+0000","last_unstale":"2023-07-18T03:06:52.883435+0000","last_undegraded":"2023-07-18T03:06:52.883435+0000","last_fullsized":"2023-07-18T03:06:52.883435+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:55:58.968273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255507+0000","last_change":"2023-07-18T03:05:03.871418+0000","last_active":"2023-07-18T03:06:24.255507+0000","last_peered":"2023-07-18T03:06:24.255507+0000","last_clean":"2023-07-18T03:06:24.255507+0000","last_became_active":"2023-07-18T03:05:03.871054+0000","last_became_peered":"2023-07-18T03:05:03.871054+0000","last_unstale":"2023-07-18T03:06:24.255507+0000","last_undegraded":"2023-07-18T03:06:24.255507+0000","last_fullsized":"2023-07-18T03:06:24.255507+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:59:35.196165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255933+0000","last_change":"2023-07-18T03:04:58.850505+0000","last_active":"2023-07-18T03:06:24.255933+0000","last_peered":"2023-07-18T03:06:24.255933+0000","last_clean":"2023-07-18T03:06:24.255933+0000","last_became_active":"2023-07-18T03:04:58.850234+0000","last_became_peered":"2023-07-18T03:04:58.850234+0000","last_unstale":"2023-07-18T03:06:24.255933+0000","last_undegraded":"2023-07-18T03:06:24.255933+0000","last_fullsized":"2023-07-18T03:06:24.255933+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:36:14.805265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.886643+0000","last_change":"2023-07-18T03:04:59.807348+0000","last_active":"2023-07-18T03:06:52.886643+0000","last_peered":"2023-07-18T03:06:52.886643+0000","last_clean":"2023-07-18T03:06:52.886643+0000","last_became_active":"2023-07-18T03:04:59.807013+0000","last_became_peered":"2023-07-18T03:04:59.807013+0000","last_unstale":"2023-07-18T03:06:52.886643+0000","last_undegraded":"2023-07-18T03:06:52.886643+0000","last_fullsized":"2023-07-18T03:06:52.886643+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:11:05.544642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255623+0000","last_change":"2023-07-18T03:05:03.886773+0000","last_active":"2023-07-18T03:06:24.255623+0000","last_peered":"2023-07-18T03:06:24.255623+0000","last_clean":"2023-07-18T03:06:24.255623+0000","last_became_active":"2023-07-18T03:05:03.886259+0000","last_became_peered":"2023-07-18T03:05:03.886259+0000","last_unstale":"2023-07-18T03:06:24.255623+0000","last_undegraded":"2023-07-18T03:06:24.255623+0000","last_fullsized":"2023-07-18T03:06:24.255623+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:12:05.068449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.261480+0000","last_change":"2023-07-18T03:04:58.855746+0000","last_active":"2023-07-18T03:06:24.261480+0000","last_peered":"2023-07-18T03:06:24.261480+0000","last_clean":"2023-07-18T03:06:24.261480+0000","last_became_active":"2023-07-18T03:04:58.855366+0000","last_became_peered":"2023-07-18T03:04:58.855366+0000","last_unstale":"2023-07-18T03:06:24.261480+0000","last_undegraded":"2023-07-18T03:06:24.261480+0000","last_fullsized":"2023-07-18T03:06:24.261480+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:04:33.740458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.879818+0000","last_change":"2023-07-18T03:04:59.800321+0000","last_active":"2023-07-18T03:06:52.879818+0000","last_peered":"2023-07-18T03:06:52.879818+0000","last_clean":"2023-07-18T03:06:52.879818+0000","last_became_active":"2023-07-18T03:04:59.800018+0000","last_became_peered":"2023-07-18T03:04:59.800018+0000","last_unstale":"2023-07-18T03:06:52.879818+0000","last_undegraded":"2023-07-18T03:06:52.879818+0000","last_fullsized":"2023-07-18T03:06:52.879818+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:13:00.845492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255150+0000","last_change":"2023-07-18T03:05:03.877541+0000","last_active":"2023-07-18T03:06:24.255150+0000","last_peered":"2023-07-18T03:06:24.255150+0000","last_clean":"2023-07-18T03:06:24.255150+0000","last_became_active":"2023-07-18T03:05:03.877164+0000","last_became_peered":"2023-07-18T03:05:03.877164+0000","last_unstale":"2023-07-18T03:06:24.255150+0000","last_undegraded":"2023-07-18T03:06:24.255150+0000","last_fullsized":"2023-07-18T03:06:24.255150+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:58:38.384423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.375090+0000","last_change":"2023-07-18T03:04:58.845643+0000","last_active":"2023-07-18T03:06:24.375090+0000","last_peered":"2023-07-18T03:06:24.375090+0000","last_clean":"2023-07-18T03:06:24.375090+0000","last_became_active":"2023-07-18T03:04:58.845254+0000","last_became_peered":"2023-07-18T03:04:58.845254+0000","last_unstale":"2023-07-18T03:06:24.375090+0000","last_undegraded":"2023-07-18T03:06:24.375090+0000","last_fullsized":"2023-07-18T03:06:24.375090+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:03:15.745367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.887203+0000","last_change":"2023-07-18T03:04:59.803679+0000","last_active":"2023-07-18T03:06:52.887203+0000","last_peered":"2023-07-18T03:06:52.887203+0000","last_clean":"2023-07-18T03:06:52.887203+0000","last_became_active":"2023-07-18T03:04:59.802841+0000","last_became_peered":"2023-07-18T03:04:59.802841+0000","last_unstale":"2023-07-18T03:06:52.887203+0000","last_undegraded":"2023-07-18T03:06:52.887203+0000","last_fullsized":"2023-07-18T03:06:52.887203+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:51:12.763232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255483+0000","last_change":"2023-07-18T03:05:03.874093+0000","last_active":"2023-07-18T03:06:24.255483+0000","last_peered":"2023-07-18T03:06:24.255483+0000","last_clean":"2023-07-18T03:06:24.255483+0000","last_became_active":"2023-07-18T03:05:03.873710+0000","last_became_peered":"2023-07-18T03:05:03.873710+0000","last_unstale":"2023-07-18T03:06:24.255483+0000","last_undegraded":"2023-07-18T03:06:24.255483+0000","last_fullsized":"2023-07-18T03:06:24.255483+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:42:17.007476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255671+0000","last_change":"2023-07-18T03:04:58.846686+0000","last_active":"2023-07-18T03:06:24.255671+0000","last_peered":"2023-07-18T03:06:24.255671+0000","last_clean":"2023-07-18T03:06:24.255671+0000","last_became_active":"2023-07-18T03:04:58.846371+0000","last_became_peered":"2023-07-18T03:04:58.846371+0000","last_unstale":"2023-07-18T03:06:24.255671+0000","last_undegraded":"2023-07-18T03:06:24.255671+0000","last_fullsized":"2023-07-18T03:06:24.255671+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:25:47.154236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.886475+0000","last_change":"2023-07-18T03:04:59.811917+0000","last_active":"2023-07-18T03:06:52.886475+0000","last_peered":"2023-07-18T03:06:52.886475+0000","last_clean":"2023-07-18T03:06:52.886475+0000","last_became_active":"2023-07-18T03:04:59.811551+0000","last_became_peered":"2023-07-18T03:04:59.811551+0000","last_unstale":"2023-07-18T03:06:52.886475+0000","last_undegraded":"2023-07-18T03:06:52.886475+0000","last_fullsized":"2023-07-18T03:06:52.886475+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:28:00.926747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255450+0000","last_change":"2023-07-18T03:05:03.886072+0000","last_active":"2023-07-18T03:06:24.255450+0000","last_peered":"2023-07-18T03:06:24.255450+0000","last_clean":"2023-07-18T03:06:24.255450+0000","last_became_active":"2023-07-18T03:05:03.885652+0000","last_became_peered":"2023-07-18T03:05:03.885652+0000","last_unstale":"2023-07-18T03:06:24.255450+0000","last_undegraded":"2023-07-18T03:06:24.255450+0000","last_fullsized":"2023-07-18T03:06:24.255450+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:37:43.631888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.259677+0000","last_change":"2023-07-18T03:04:58.849069+0000","last_active":"2023-07-18T03:06:24.259677+0000","last_peered":"2023-07-18T03:06:24.259677+0000","last_clean":"2023-07-18T03:06:24.259677+0000","last_became_active":"2023-07-18T03:04:58.848744+0000","last_became_peered":"2023-07-18T03:04:58.848744+0000","last_unstale":"2023-07-18T03:06:24.259677+0000","last_undegraded":"2023-07-18T03:06:24.259677+0000","last_fullsized":"2023-07-18T03:06:24.259677+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:19:13.922970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.895462+0000","last_change":"2023-07-18T03:04:59.806976+0000","last_active":"2023-07-18T03:06:52.895462+0000","last_peered":"2023-07-18T03:06:52.895462+0000","last_clean":"2023-07-18T03:06:52.895462+0000","last_became_active":"2023-07-18T03:04:59.805750+0000","last_became_peered":"2023-07-18T03:04:59.805750+0000","last_unstale":"2023-07-18T03:06:52.895462+0000","last_undegraded":"2023-07-18T03:06:52.895462+0000","last_fullsized":"2023-07-18T03:06:52.895462+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:59:17.717720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.261590+0000","last_change":"2023-07-18T03:05:03.871210+0000","last_active":"2023-07-18T03:06:24.261590+0000","last_peered":"2023-07-18T03:06:24.261590+0000","last_clean":"2023-07-18T03:06:24.261590+0000","last_became_active":"2023-07-18T03:05:03.870826+0000","last_became_peered":"2023-07-18T03:05:03.870826+0000","last_unstale":"2023-07-18T03:06:24.261590+0000","last_undegraded":"2023-07-18T03:06:24.261590+0000","last_fullsized":"2023-07-18T03:06:24.261590+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:49:03.743543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.256155+0000","last_change":"2023-07-18T03:04:58.855673+0000","last_active":"2023-07-18T03:06:24.256155+0000","last_peered":"2023-07-18T03:06:24.256155+0000","last_clean":"2023-07-18T03:06:24.256155+0000","last_became_active":"2023-07-18T03:04:58.855338+0000","last_became_peered":"2023-07-18T03:04:58.855338+0000","last_unstale":"2023-07-18T03:06:24.256155+0000","last_undegraded":"2023-07-18T03:06:24.256155+0000","last_fullsized":"2023-07-18T03:06:24.256155+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:07:40.935595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.884252+0000","last_change":"2023-07-18T03:04:59.802488+0000","last_active":"2023-07-18T03:06:52.884252+0000","last_peered":"2023-07-18T03:06:52.884252+0000","last_clean":"2023-07-18T03:06:52.884252+0000","last_became_active":"2023-07-18T03:04:59.802184+0000","last_became_peered":"2023-07-18T03:04:59.802184+0000","last_unstale":"2023-07-18T03:06:52.884252+0000","last_undegraded":"2023-07-18T03:06:52.884252+0000","last_fullsized":"2023-07-18T03:06:52.884252+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:00:59.367787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.259218+0000","last_change":"2023-07-18T03:05:03.881585+0000","last_active":"2023-07-18T03:06:24.259218+0000","last_peered":"2023-07-18T03:06:24.259218+0000","last_clean":"2023-07-18T03:06:24.259218+0000","last_became_active":"2023-07-18T03:05:03.880917+0000","last_became_peered":"2023-07-18T03:05:03.880917+0000","last_unstale":"2023-07-18T03:06:24.259218+0000","last_undegraded":"2023-07-18T03:06:24.259218+0000","last_fullsized":"2023-07-18T03:06:24.259218+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:44:11.175469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.262020+0000","last_change":"2023-07-18T03:04:58.852387+0000","last_active":"2023-07-18T03:06:24.262020+0000","last_peered":"2023-07-18T03:06:24.262020+0000","last_clean":"2023-07-18T03:06:24.262020+0000","last_became_active":"2023-07-18T03:04:58.852113+0000","last_became_peered":"2023-07-18T03:04:58.852113+0000","last_unstale":"2023-07-18T03:06:24.262020+0000","last_undegraded":"2023-07-18T03:06:24.262020+0000","last_fullsized":"2023-07-18T03:06:24.262020+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:12:15.450770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.882718+0000","last_change":"2023-07-18T03:04:59.811569+0000","last_active":"2023-07-18T03:06:52.882718+0000","last_peered":"2023-07-18T03:06:52.882718+0000","last_clean":"2023-07-18T03:06:52.882718+0000","last_became_active":"2023-07-18T03:04:59.811229+0000","last_became_peered":"2023-07-18T03:04:59.811229+0000","last_unstale":"2023-07-18T03:06:52.882718+0000","last_undegraded":"2023-07-18T03:06:52.882718+0000","last_fullsized":"2023-07-18T03:06:52.882718+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:20:39.366577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.260963+0000","last_change":"2023-07-18T03:05:03.872530+0000","last_active":"2023-07-18T03:06:24.260963+0000","last_peered":"2023-07-18T03:06:24.260963+0000","last_clean":"2023-07-18T03:06:24.260963+0000","last_became_active":"2023-07-18T03:05:03.871607+0000","last_became_peered":"2023-07-18T03:05:03.871607+0000","last_unstale":"2023-07-18T03:06:24.260963+0000","last_undegraded":"2023-07-18T03:06:24.260963+0000","last_fullsized":"2023-07-18T03:06:24.260963+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:13:21.796882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.751209+0000","last_change":"2023-07-18T03:04:58.847933+0000","last_active":"2023-07-18T03:06:24.751209+0000","last_peered":"2023-07-18T03:06:24.751209+0000","last_clean":"2023-07-18T03:06:24.751209+0000","last_became_active":"2023-07-18T03:04:58.847630+0000","last_became_peered":"2023-07-18T03:04:58.847630+0000","last_unstale":"2023-07-18T03:06:24.751209+0000","last_undegraded":"2023-07-18T03:06:24.751209+0000","last_fullsized":"2023-07-18T03:06:24.751209+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:34:44.655423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.886617+0000","last_change":"2023-07-18T03:04:59.803981+0000","last_active":"2023-07-18T03:06:52.886617+0000","last_peered":"2023-07-18T03:06:52.886617+0000","last_clean":"2023-07-18T03:06:52.886617+0000","last_became_active":"2023-07-18T03:04:59.803692+0000","last_became_peered":"2023-07-18T03:04:59.803692+0000","last_unstale":"2023-07-18T03:06:52.886617+0000","last_undegraded":"2023-07-18T03:06:52.886617+0000","last_fullsized":"2023-07-18T03:06:52.886617+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:58:45.567557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.884681+0000","last_change":"2023-07-18T03:04:59.803770+0000","last_active":"2023-07-18T03:06:52.884681+0000","last_peered":"2023-07-18T03:06:52.884681+0000","last_clean":"2023-07-18T03:06:52.884681+0000","last_became_active":"2023-07-18T03:04:59.803355+0000","last_became_peered":"2023-07-18T03:04:59.803355+0000","last_unstale":"2023-07-18T03:06:52.884681+0000","last_undegraded":"2023-07-18T03:06:52.884681+0000","last_fullsized":"2023-07-18T03:06:52.884681+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:07:11.955948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.886539+0000","last_change":"2023-07-18T03:04:59.808946+0000","last_active":"2023-07-18T03:06:52.886539+0000","last_peered":"2023-07-18T03:06:52.886539+0000","last_clean":"2023-07-18T03:06:52.886539+0000","last_became_active":"2023-07-18T03:04:59.808646+0000","last_became_peered":"2023-07-18T03:04:59.808646+0000","last_unstale":"2023-07-18T03:06:52.886539+0000","last_undegraded":"2023-07-18T03:06:52.886539+0000","last_fullsized":"2023-07-18T03:06:52.886539+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:10:55.288876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.883434+0000","last_change":"2023-07-18T03:04:59.811933+0000","last_active":"2023-07-18T03:06:52.883434+0000","last_peered":"2023-07-18T03:06:52.883434+0000","last_clean":"2023-07-18T03:06:52.883434+0000","last_became_active":"2023-07-18T03:04:59.811620+0000","last_became_peered":"2023-07-18T03:04:59.811620+0000","last_unstale":"2023-07-18T03:06:52.883434+0000","last_undegraded":"2023-07-18T03:06:52.883434+0000","last_fullsized":"2023-07-18T03:06:52.883434+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:21:21.164245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.896076+0000","last_change":"2023-07-18T03:04:59.803942+0000","last_active":"2023-07-18T03:06:52.896076+0000","last_peered":"2023-07-18T03:06:52.896076+0000","last_clean":"2023-07-18T03:06:52.896076+0000","last_became_active":"2023-07-18T03:04:59.803576+0000","last_became_peered":"2023-07-18T03:04:59.803576+0000","last_unstale":"2023-07-18T03:06:52.896076+0000","last_undegraded":"2023-07-18T03:06:52.896076+0000","last_fullsized":"2023-07-18T03:06:52.896076+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:10:05.182871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.893685+0000","last_change":"2023-07-18T03:04:59.802594+0000","last_active":"2023-07-18T03:06:52.893685+0000","last_peered":"2023-07-18T03:06:52.893685+0000","last_clean":"2023-07-18T03:06:52.893685+0000","last_became_active":"2023-07-18T03:04:59.802215+0000","last_became_peered":"2023-07-18T03:04:59.802215+0000","last_unstale":"2023-07-18T03:06:52.893685+0000","last_undegraded":"2023-07-18T03:06:52.893685+0000","last_fullsized":"2023-07-18T03:06:52.893685+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:57:57.598480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.883798+0000","last_change":"2023-07-18T03:04:59.803519+0000","last_active":"2023-07-18T03:06:52.883798+0000","last_peered":"2023-07-18T03:06:52.883798+0000","last_clean":"2023-07-18T03:06:52.883798+0000","last_became_active":"2023-07-18T03:04:59.803170+0000","last_became_peered":"2023-07-18T03:04:59.803170+0000","last_unstale":"2023-07-18T03:06:52.883798+0000","last_undegraded":"2023-07-18T03:06:52.883798+0000","last_fullsized":"2023-07-18T03:06:52.883798+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:40:11.785249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.881716+0000","last_change":"2023-07-18T03:04:59.813494+0000","last_active":"2023-07-18T03:06:52.881716+0000","last_peered":"2023-07-18T03:06:52.881716+0000","last_clean":"2023-07-18T03:06:52.881716+0000","last_became_active":"2023-07-18T03:04:59.813175+0000","last_became_peered":"2023-07-18T03:04:59.813175+0000","last_unstale":"2023-07-18T03:06:52.881716+0000","last_undegraded":"2023-07-18T03:06:52.881716+0000","last_fullsized":"2023-07-18T03:06:52.881716+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:58:52.954498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.884923+0000","last_change":"2023-07-18T03:04:59.806597+0000","last_active":"2023-07-18T03:06:52.884923+0000","last_peered":"2023-07-18T03:06:52.884923+0000","last_clean":"2023-07-18T03:06:52.884923+0000","last_became_active":"2023-07-18T03:04:59.806220+0000","last_became_peered":"2023-07-18T03:04:59.806220+0000","last_unstale":"2023-07-18T03:06:52.884923+0000","last_undegraded":"2023-07-18T03:06:52.884923+0000","last_fullsized":"2023-07-18T03:06:52.884923+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:07:15.022728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.888455+0000","last_change":"2023-07-18T03:04:59.809051+0000","last_active":"2023-07-18T03:06:52.888455+0000","last_peered":"2023-07-18T03:06:52.888455+0000","last_clean":"2023-07-18T03:06:52.888455+0000","last_became_active":"2023-07-18T03:04:59.808783+0000","last_became_peered":"2023-07-18T03:04:59.808783+0000","last_unstale":"2023-07-18T03:06:52.888455+0000","last_undegraded":"2023-07-18T03:06:52.888455+0000","last_fullsized":"2023-07-18T03:06:52.888455+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:55:28.682728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.883966+0000","last_change":"2023-07-18T03:04:59.805741+0000","last_active":"2023-07-18T03:06:52.883966+0000","last_peered":"2023-07-18T03:06:52.883966+0000","last_clean":"2023-07-18T03:06:52.883966+0000","last_became_active":"2023-07-18T03:04:59.805412+0000","last_became_peered":"2023-07-18T03:04:59.805412+0000","last_unstale":"2023-07-18T03:06:52.883966+0000","last_undegraded":"2023-07-18T03:06:52.883966+0000","last_fullsized":"2023-07-18T03:06:52.883966+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:56:58.924751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.891407+0000","last_change":"2023-07-18T03:04:59.812052+0000","last_active":"2023-07-18T03:06:52.891407+0000","last_peered":"2023-07-18T03:06:52.891407+0000","last_clean":"2023-07-18T03:06:52.891407+0000","last_became_active":"2023-07-18T03:04:59.811756+0000","last_became_peered":"2023-07-18T03:04:59.811756+0000","last_unstale":"2023-07-18T03:06:52.891407+0000","last_undegraded":"2023-07-18T03:06:52.891407+0000","last_fullsized":"2023-07-18T03:06:52.891407+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:41:23.926360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.896400+0000","last_change":"2023-07-18T03:04:59.803009+0000","last_active":"2023-07-18T03:06:52.896400+0000","last_peered":"2023-07-18T03:06:52.896400+0000","last_clean":"2023-07-18T03:06:52.896400+0000","last_became_active":"2023-07-18T03:04:59.802687+0000","last_became_peered":"2023-07-18T03:04:59.802687+0000","last_unstale":"2023-07-18T03:06:52.896400+0000","last_undegraded":"2023-07-18T03:06:52.896400+0000","last_fullsized":"2023-07-18T03:06:52.896400+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:31:26.686903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.880496+0000","last_change":"2023-07-18T03:04:59.795562+0000","last_active":"2023-07-18T03:06:52.880496+0000","last_peered":"2023-07-18T03:06:52.880496+0000","last_clean":"2023-07-18T03:06:52.880496+0000","last_became_active":"2023-07-18T03:04:59.795241+0000","last_became_peered":"2023-07-18T03:04:59.795241+0000","last_unstale":"2023-07-18T03:06:52.880496+0000","last_undegraded":"2023-07-18T03:06:52.880496+0000","last_fullsized":"2023-07-18T03:06:52.880496+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:44:43.907380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281146,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70826,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":655,"num_read_kb":404,"num_write":166,"num_write_kb":124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":212478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449128,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":999424,"data_stored":507139,"data_compressed":23524,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.56000000000000005,"15min":0.55300000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.52200000000000002,"15min":0.51400000000000001},"min":{"1min":0.41099999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.69199999999999995}]},{"osd":1,"last update":"Tue Jul 18 03:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.64300000000000002,"15min":0.63500000000000001},"min":{"1min":0.47799999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.85299999999999998,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.624,"15min":0.621},"min":{"1min":0.47099999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.82299999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.76400000000000001}]},{"osd":2,"last update":"Tue Jul 18 03:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.48299999999999998,"15min":0.47399999999999998},"min":{"1min":0.35199999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.71399999999999997,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.48999999999999999,"15min":0.47599999999999998},"min":{"1min":0.36699999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.32500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":4,"last update":"Tue Jul 18 03:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.66500000000000004,"15min":0.65100000000000002},"min":{"1min":0.53200000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.64200000000000002,"15min":0.63500000000000001},"min":{"1min":0.48999999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.84799999999999998,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.84399999999999997}]},{"osd":5,"last update":"Tue Jul 18 03:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.622,"15min":0.621},"min":{"1min":0.45400000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.82099999999999995,"5min":1.375,"15min":1.375},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.61399999999999999,"15min":0.59299999999999997},"min":{"1min":0.378,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.40400000000000003}]},{"osd":6,"last update":"Tue Jul 18 03:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.68400000000000005,"15min":0.67000000000000004},"min":{"1min":0.52200000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.629},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.70499999999999996,"15min":0.69299999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.64900000000000002}]},{"osd":7,"last update":"Tue Jul 18 03:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.79200000000000004,"15min":0.78000000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1319999999999999,"5min":1.248,"15min":1.248},"last":0.745},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.76900000000000002,"15min":0.76300000000000001},"min":{"1min":0.56499999999999995,"5min":0.502,"15min":0.502},"max":{"1min":1.024,"5min":1.288,"15min":1.288},"last":0.55500000000000005}]},{"osd":8,"last update":"Tue Jul 18 03:07:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.76500000000000001,"15min":0.77000000000000002},"min":{"1min":0.50800000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.83899999999999997,"15min":0.84199999999999997},"min":{"1min":0.58999999999999997,"5min":0.498,"15min":0.498},"max":{"1min":0.98599999999999999,"5min":1.091,"15min":1.091},"last":0.496}]},{"osd":9,"last update":"Tue Jul 18 03:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.78300000000000003,"15min":0.76800000000000002},"min":{"1min":0.66700000000000004,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.145,"5min":1.304,"15min":1.304},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.81899999999999995,"15min":0.80400000000000005},"min":{"1min":0.63400000000000001,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1659999999999999,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.71199999999999997}]},{"osd":10,"last update":"Tue Jul 18 03:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.78200000000000003,"15min":0.77200000000000002},"min":{"1min":0.57999999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.054,"5min":1.163,"15min":1.163},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.79900000000000004,"15min":0.78900000000000003},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.026,"5min":1.496,"15min":1.496},"last":0.78400000000000003}]}]},{"osd":10,"up_from":71,"seq":304942678061,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1120,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1146880,"data_stored":1161118,"data_compressed":59283,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:07:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.44600000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.71099999999999997,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.66300000000000003,"15min":0.66900000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.70799999999999996}]},{"osd":1,"last update":"Tue Jul 18 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.47699999999999998,"15min":0.48099999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.61499999999999999,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.505,"15min":0.51400000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70999999999999996,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.58999999999999997}]},{"osd":2,"last update":"Tue Jul 18 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.64700000000000002,"15min":0.63400000000000001},"min":{"1min":0.437,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.64700000000000002,"15min":0.64300000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.78900000000000003,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.72699999999999998}]},{"osd":3,"last update":"Tue Jul 18 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.67200000000000004,"15min":0.68500000000000005},"min":{"1min":0.48199999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80000000000000004,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65500000000000003,"15min":0.66300000000000003},"min":{"1min":0.498,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.66100000000000003}]},{"osd":4,"last update":"Tue Jul 18 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.59799999999999998,"15min":0.61399999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.58399999999999996,"15min":0.57999999999999996},"min":{"1min":0.375,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.82999999999999996,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.80900000000000005}]},{"osd":5,"last update":"Tue Jul 18 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71299999999999997,"15min":0.72099999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.753,"5min":0.751,"15min":0.751},"min":{"1min":0.49299999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.90000000000000002}]},{"osd":6,"last update":"Tue Jul 18 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.745,"15min":0.75800000000000001},"min":{"1min":0.50700000000000001,"5min":0.501,"15min":0.501},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.79300000000000004,"15min":0.80100000000000005},"min":{"1min":0.61599999999999999,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.96899999999999997,"5min":1.077,"15min":1.077},"last":0.94099999999999995}]},{"osd":7,"last update":"Tue Jul 18 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.55800000000000005}]},{"osd":8,"last update":"Tue Jul 18 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.79200000000000004,"15min":0.80500000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.91000000000000003,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.81799999999999995,"15min":0.82499999999999996},"min":{"1min":0.60299999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.0169999999999999,"5min":1.032,"15min":1.032},"last":0.95899999999999996}]},{"osd":9,"last update":"Tue Jul 18 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.83799999999999997,"15min":0.84399999999999997},"min":{"1min":0.66500000000000004,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.84699999999999998,"15min":0.85499999999999998},"min":{"1min":0.63300000000000001,"5min":0.626,"15min":0.626},"max":{"1min":0.996,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.98799999999999999}]},{"osd":11,"last update":"Tue Jul 18 03:06:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.82699999999999996,"15min":0.83199999999999996},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.96599999999999997,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.80000000000000004,"15min":0.80900000000000005},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.92800000000000005,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.83099999999999996}]}]},{"osd":9,"up_from":66,"seq":283467841584,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":1015808,"data_stored":511449,"data_compressed":23524,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.52000000000000002,"15min":0.54800000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.629,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.55000000000000004,"15min":0.57199999999999995},"min":{"1min":0.32600000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.79700000000000004,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.39800000000000002}]},{"osd":1,"last update":"Tue Jul 18 03:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.68999999999999995,"15min":0.72099999999999997},"min":{"1min":0.41499999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.83599999999999997,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.69699999999999995,"15min":0.73099999999999998},"min":{"1min":0.46200000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.81699999999999995,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.58099999999999996}]},{"osd":2,"last update":"Tue Jul 18 03:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.68899999999999995,"15min":0.71099999999999997},"min":{"1min":0.45800000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.78100000000000003,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.71899999999999997,"15min":0.74199999999999999},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.81299999999999994,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.77600000000000002}]},{"osd":3,"last update":"Tue Jul 18 03:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.63500000000000001,"15min":0.67600000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.76500000000000001,"5min":1.038,"15min":1.038},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.61199999999999999,"15min":0.65000000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.67600000000000005,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.46200000000000002}]},{"osd":4,"last update":"Tue Jul 18 03:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.77000000000000002,"15min":0.81699999999999995},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.85999999999999999,"5min":1.47,"15min":1.47},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.77900000000000003,"15min":0.81799999999999995},"min":{"1min":0.47599999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.90700000000000003,"5min":1.619,"15min":1.619},"last":0.68200000000000005}]},{"osd":5,"last update":"Tue Jul 18 03:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.79800000000000004,"15min":0.82199999999999995},"min":{"1min":0.53500000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.89300000000000002,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.78000000000000003,"15min":0.80800000000000005},"min":{"1min":0.53300000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.90100000000000002,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.82199999999999995}]},{"osd":6,"last update":"Tue Jul 18 03:04:30 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.64300000000000002}]},{"osd":7,"last update":"Tue Jul 18 03:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.84799999999999998,"15min":0.89600000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.86699999999999999,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.89500000000000002,"15min":0.92300000000000004},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.81399999999999995,"5min":4.4009999999999998,"15min":4.4009999999999998},"last":0.64600000000000002}]},{"osd":8,"last update":"Tue Jul 18 03:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.876,"15min":0.91600000000000004},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.90900000000000003,"15min":0.94599999999999995},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.012,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.76500000000000001}]},{"osd":10,"last update":"Tue Jul 18 03:06:53 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.86099999999999999,"15min":0.89500000000000002},"min":{"1min":0.60199999999999998,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1020000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.873,"15min":0.92500000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.86099999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.754}]},{"osd":11,"last update":"Tue Jul 18 03:07:09 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.82799999999999996,"15min":0.83099999999999996},"min":{"1min":0.69399999999999995,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":0.96399999999999997,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.79400000000000004,"15min":0.79600000000000004},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":0.93700000000000006,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.79300000000000004}]}]},{"osd":8,"up_from":58,"seq":249108103221,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":1007616,"data_stored":507251,"data_compressed":23524,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.69099999999999995,"15min":0.64000000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.80700000000000005,"5min":7.0359999999999996,"15min":7.0359999999999996},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.58199999999999996,"15min":0.56999999999999995},"min":{"1min":0.42499999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.75700000000000001,"5min":2.883,"15min":2.883},"last":0.55700000000000005}]},{"osd":1,"last update":"Tue Jul 18 03:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.67500000000000004,"15min":0.63900000000000001},"min":{"1min":0.45200000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.95099999999999996,"5min":3.3340000000000001,"15min":3.3340000000000001},"last":0.752},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.67100000000000004,"15min":0.61599999999999999},"min":{"1min":0.55500000000000005,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.99199999999999999,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":0.82899999999999996}]},{"osd":2,"last update":"Tue Jul 18 03:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51400000000000001,"15min":0.47099999999999997},"min":{"1min":0.40100000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70199999999999996,"5min":3.2850000000000001,"15min":3.2850000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.53300000000000003,"15min":0.502},"min":{"1min":0.38400000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.753,"5min":2.012,"15min":2.012},"last":0.63900000000000001}]},{"osd":3,"last update":"Tue Jul 18 03:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64600000000000002,"15min":0.623},"min":{"1min":0.46899999999999997,"5min":0.432,"15min":0.432},"max":{"1min":0.88100000000000001,"5min":2.9100000000000001,"15min":2.9100000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.67000000000000004,"15min":0.65000000000000002},"min":{"1min":0.49399999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.996,"5min":3.6059999999999999,"15min":3.6059999999999999},"last":0.60499999999999998}]},{"osd":4,"last update":"Tue Jul 18 03:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.67600000000000005,"15min":0.63800000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.04,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.73299999999999998,"15min":0.69999999999999996},"min":{"1min":0.51700000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0940000000000001,"5min":3.3650000000000002,"15min":3.3650000000000002},"last":0.84599999999999997}]},{"osd":5,"last update":"Tue Jul 18 03:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.64800000000000002,"15min":0.60399999999999998},"min":{"1min":0.47499999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.0269999999999999,"5min":2.4279999999999999,"15min":2.4279999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.65800000000000003,"15min":0.61399999999999999},"min":{"1min":0.41899999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.135,"5min":2.7410000000000001,"15min":2.7410000000000001},"last":0.71799999999999997}]},{"osd":6,"last update":"Tue Jul 18 03:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76400000000000001,"15min":0.72599999999999998},"min":{"1min":0.46200000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.94399999999999995,"5min":3.5779999999999998,"15min":3.5779999999999998},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.77700000000000002,"15min":0.745},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.97899999999999998,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":0.92300000000000004}]},{"osd":7,"last update":"Tue Jul 18 03:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.78700000000000003,"15min":0.73899999999999999},"min":{"1min":0.67000000000000004,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.081,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.77900000000000003,"15min":0.73399999999999999},"min":{"1min":0.628,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.157,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":0.89200000000000002}]},{"osd":9,"last update":"Tue Jul 18 03:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.82899999999999996,"15min":0.78800000000000003},"min":{"1min":0.627,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1379999999999999,"5min":3.544,"15min":3.544},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.83099999999999996,"15min":0.79700000000000004},"min":{"1min":0.66500000000000004,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.1259999999999999,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":0.97699999999999998}]},{"osd":10,"last update":"Tue Jul 18 03:06:55 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.82799999999999996,"15min":0.81699999999999995},"min":{"1min":0.59299999999999997,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1519999999999999,"5min":2.8570000000000002,"15min":2.8570000000000002},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.879,"15min":0.88100000000000001},"min":{"1min":0.58699999999999997,"5min":0.502,"15min":0.502},"max":{"1min":1.1950000000000001,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":0.90400000000000003}]},{"osd":11,"last update":"Tue Jul 18 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.86899999999999999,"15min":0.86599999999999999},"min":{"1min":0.69399999999999995,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.0720000000000001,"5min":1.107,"15min":1.107},"last":0.878},{"interface":"front","average":{"1min":0.871,"5min":0.85299999999999998,"15min":0.84999999999999998},"min":{"1min":0.65800000000000003,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.81299999999999994}]}]},{"osd":7,"up_from":51,"seq":219043332153,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":1036,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":1060864,"data_stored":1103226,"data_compressed":59283,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.55500000000000005,"15min":0.54300000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.495},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59999999999999998,"15min":0.60299999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82599999999999996,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.44500000000000001}]},{"osd":1,"last update":"Tue Jul 18 03:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.434,"15min":0.42399999999999999},"min":{"1min":0.28699999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.5},{"interface":"front","average":{"1min":0.503,"5min":0.45400000000000001,"15min":0.438},"min":{"1min":0.317,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.74299999999999999,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.55100000000000005}]},{"osd":2,"last update":"Tue Jul 18 03:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.60599999999999998,"15min":0.57699999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.90600000000000003,"5min":1.111,"15min":1.111},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.60699999999999998,"15min":0.58699999999999997},"min":{"1min":0.375,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.89500000000000002,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.63500000000000001}]},{"osd":3,"last update":"Tue Jul 18 03:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.63500000000000001,"15min":0.60599999999999998},"min":{"1min":0.45000000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.83999999999999997,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.63200000000000001,"15min":0.64400000000000002},"min":{"1min":0.41699999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.83699999999999997,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.58499999999999996}]},{"osd":4,"last update":"Tue Jul 18 03:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.5,"15min":0.48799999999999999},"min":{"1min":0.34799999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.55700000000000005,"15min":0.55100000000000005},"min":{"1min":0.36699999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.52500000000000002}]},{"osd":5,"last update":"Tue Jul 18 03:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.68899999999999995,"15min":0.65700000000000003},"min":{"1min":0.51600000000000001,"5min":0.501,"15min":0.501},"max":{"1min":1.0249999999999999,"5min":1.077,"15min":1.077},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.66700000000000004,"15min":0.65600000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.93400000000000005,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.69199999999999995}]},{"osd":6,"last update":"Tue Jul 18 03:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67200000000000004},"min":{"1min":0.51800000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.89400000000000002,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.73299999999999998,"15min":0.70899999999999996},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.86099999999999999}]},{"osd":8,"last update":"Tue Jul 18 03:07:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.72299999999999998,"15min":0.69699999999999995},"min":{"1min":0.53600000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.997,"5min":1.022,"15min":1.022},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.745,"5min":0.68799999999999994,"15min":0.65600000000000003},"min":{"1min":0.55200000000000005,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.94999999999999996,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.82299999999999995}]},{"osd":9,"last update":"Tue Jul 18 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.76000000000000001,"15min":0.73799999999999999},"min":{"1min":0.435,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":3.1880000000000002,"5min":3.1880000000000002,"15min":3.1880000000000002},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.77900000000000003,"15min":0.76800000000000002},"min":{"1min":0.58399999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.88800000000000001}]},{"osd":10,"last update":"Tue Jul 18 03:06:55 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71799999999999997,"15min":0.72299999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.76100000000000001,"15min":0.755},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.91400000000000003}]},{"osd":11,"last update":"Tue Jul 18 03:07:06 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.83599999999999997,"15min":0.82599999999999996},"min":{"1min":0.76100000000000001,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.84899999999999998,"15min":0.83699999999999997},"min":{"1min":0.754,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.93700000000000006}]}]},{"osd":6,"up_from":46,"seq":197568495678,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":991232,"data_stored":502529,"data_compressed":23524,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.52000000000000002,"15min":0.55800000000000005},"min":{"1min":0.307,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.48,"5min":3.1579999999999999,"15min":3.1579999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.53400000000000003,"15min":0.55000000000000004},"min":{"1min":0.36699999999999999,"5min":0.318,"15min":0.318},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":0.38900000000000001}]},{"osd":1,"last update":"Tue Jul 18 03:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.64300000000000002,"15min":0.60899999999999999},"min":{"1min":0.499,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.71699999999999997,"15min":0.73699999999999999},"min":{"1min":0.48299999999999998,"5min":0.44,"15min":0.44},"max":{"1min":1.226,"5min":3.0369999999999999,"15min":3.0369999999999999},"last":0.66700000000000004}]},{"osd":2,"last update":"Tue Jul 18 03:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.67300000000000004,"15min":0.66100000000000003},"min":{"1min":0.47799999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.46300000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.63600000000000001}]},{"osd":3,"last update":"Tue Jul 18 03:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56499999999999995,"15min":0.57299999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.69499999999999995,"15min":0.74199999999999999},"min":{"1min":0.40100000000000002,"5min":0.377,"15min":0.377},"max":{"1min":1.546,"5min":3.226,"15min":3.226},"last":0.45400000000000001}]},{"osd":4,"last update":"Tue Jul 18 03:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69299999999999995,"15min":0.69399999999999995},"min":{"1min":0.47499999999999998,"5min":0.441,"15min":0.441},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.70699999999999996,"15min":0.70099999999999996},"min":{"1min":0.55200000000000005,"5min":0.438,"15min":0.438},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.75900000000000001}]},{"osd":5,"last update":"Tue Jul 18 03:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.78500000000000003,"15min":0.83599999999999997},"min":{"1min":0.54500000000000004,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1060000000000001,"5min":3.125,"15min":3.125},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.76000000000000001,"15min":0.78000000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.4039999999999999,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.86799999999999999}]},{"osd":7,"last update":"Tue Jul 18 03:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.82499999999999996,"15min":0.83699999999999997},"min":{"1min":0.59499999999999997,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1619999999999999,"5min":3.0870000000000002,"15min":3.0870000000000002},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75900000000000001,"15min":0.73599999999999999},"min":{"1min":0.52300000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.95399999999999996}]},{"osd":8,"last update":"Tue Jul 18 03:06:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.85099999999999998,"15min":0.876},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0640000000000001,"5min":3.1909999999999998,"15min":3.1909999999999998},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.99099999999999999,"5min":1.046,"15min":1.046},"last":1.0289999999999999}]},{"osd":9,"last update":"Tue Jul 18 03:06:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.75600000000000001,"15min":0.76500000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.83799999999999997,"15min":0.85199999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.73299999999999998}]},{"osd":10,"last update":"Tue Jul 18 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.88100000000000001,"15min":0.879},"min":{"1min":0.752,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.873,"5min":0.83199999999999996,"15min":0.82199999999999995},"min":{"1min":0.68700000000000006,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":0.92000000000000004}]},{"osd":11,"last update":"Tue Jul 18 03:07:05 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.86499999999999999,"15min":0.86099999999999999},"min":{"1min":0.61799999999999999,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.85299999999999998,"15min":0.85499999999999998},"min":{"1min":0.67000000000000004,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.516,"5min":1.583,"15min":1.583},"last":0.88700000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":1040,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":1064960,"data_stored":569220,"data_compressed":23524,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:06:32 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.55000000000000004,"15min":0.52400000000000002},"min":{"1min":0.39200000000000002,"5min":0.29799999999999999,"15min":0.27800000000000002},"max":{"1min":0.65500000000000003,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55200000000000005,"15min":0.52700000000000002},"min":{"1min":0.376,"5min":0.33100000000000002,"15min":0.29899999999999999},"max":{"1min":0.73399999999999999,"5min":1.421,"15min":1.421},"last":0.54600000000000004}]},{"osd":2,"last update":"Tue Jul 18 03:06:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.59799999999999998,"15min":0.53000000000000003},"min":{"1min":0.39500000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.0009999999999999,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.59799999999999998,"15min":0.55400000000000005},"min":{"1min":0.50900000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.91000000000000003,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.88800000000000001}]},{"osd":3,"last update":"Tue Jul 18 03:06:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.622,"15min":0.59999999999999998},"min":{"1min":0.378,"5min":0.312,"15min":0.312},"max":{"1min":1.014,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.627,"15min":0.627},"min":{"1min":0.47999999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.026,"5min":1.504,"15min":1.504},"last":0.63400000000000001}]},{"osd":4,"last update":"Tue Jul 18 03:06:46 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.48199999999999998,"15min":0.41399999999999998},"min":{"1min":0.379,"5min":0.215,"15min":0.215},"max":{"1min":0.68500000000000005,"5min":1.784,"15min":1.784},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.50600000000000001,"15min":0.435},"min":{"1min":0.32800000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.83599999999999997,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.65500000000000003}]},{"osd":5,"last update":"Tue Jul 18 03:07:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.64400000000000002,"15min":0.54200000000000004},"min":{"1min":0.436,"5min":0.375,"15min":0.375},"max":{"1min":0.84499999999999997,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.65300000000000002,"15min":0.57699999999999996},"min":{"1min":0.441,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.94299999999999995,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.91600000000000004}]},{"osd":6,"last update":"Tue Jul 18 03:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.66400000000000003,"15min":0.60999999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.87,"5min":2.6600000000000001,"15min":2.6600000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.67300000000000004,"15min":0.64500000000000002},"min":{"1min":0.54600000000000004,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.89900000000000002,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.73299999999999998}]},{"osd":7,"last update":"Tue Jul 18 03:06:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.61699999999999999,"15min":0.57099999999999995},"min":{"1min":0.45400000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.67600000000000005,"15min":0.65600000000000003},"min":{"1min":0.42299999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.90100000000000002,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":0.85799999999999998}]},{"osd":8,"last update":"Tue Jul 18 03:07:12 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.77600000000000002,"15min":0.78200000000000003},"min":{"1min":0.47699999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.997,"5min":2.6419999999999999,"15min":2.6419999999999999},"last":0.997},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.748,"15min":0.68999999999999995},"min":{"1min":0.52400000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.03,"5min":1.698,"15min":1.698},"last":0.93200000000000005}]},{"osd":9,"last update":"Tue Jul 18 03:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.83899999999999997,"15min":0.85299999999999998},"min":{"1min":0.58499999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.96299999999999997,"5min":2.6789999999999998,"15min":2.6789999999999998},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.78000000000000003,"15min":0.752},"min":{"1min":0.68100000000000005,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0129999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.875}]},{"osd":10,"last update":"Tue Jul 18 03:06:57 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79000000000000004,"15min":0.77200000000000002},"min":{"1min":0.57399999999999995,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0589999999999999,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.77700000000000002,"15min":0.76500000000000001},"min":{"1min":0.56699999999999995,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.002,"5min":1.359,"15min":1.359},"last":0.96799999999999997}]},{"osd":11,"last update":"Tue Jul 18 03:07:02 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.90600000000000003,"15min":0.90800000000000003},"min":{"1min":0.72299999999999998,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.1060000000000001,"5min":1.512,"15min":1.512},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":1.0249999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":1015808,"data_stored":503917,"data_compressed":23524,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Jul 18 03:06:31 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61499999999999999,"15min":0.55800000000000005},"min":{"1min":0.33700000000000002,"5min":0.33600000000000002,"15min":0.28299999999999997},"max":{"1min":0.88900000000000001,"5min":1.075,"15min":1.075},"last":0.627},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.64000000000000001,"15min":0.56999999999999995},"min":{"1min":0.371,"5min":0.36099999999999999,"15min":0.30199999999999999},"max":{"1min":1.1060000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.80500000000000005}]},{"osd":2,"last update":"Tue Jul 18 03:06:54 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.66800000000000004,"15min":0.57399999999999995},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":5.3300000000000001,"5min":5.3300000000000001,"15min":5.3300000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.67400000000000004,"15min":0.57899999999999996},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.01,"5min":1.204,"15min":1.204},"last":0.49099999999999999}]},{"osd":3,"last update":"Tue Jul 18 03:07:04 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54200000000000004,"15min":0.54400000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.53300000000000003,"15min":0.53100000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.45400000000000001}]},{"osd":4,"last update":"Tue Jul 18 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.68600000000000005,"15min":0.624},"min":{"1min":0.51700000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.748},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.66500000000000004,"15min":0.61199999999999999},"min":{"1min":0.49099999999999999,"5min":0.433,"15min":0.433},"max":{"1min":0.97499999999999998,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.59299999999999997}]},{"osd":5,"last update":"Tue Jul 18 03:05:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.64700000000000002,"15min":0.63500000000000001},"min":{"1min":0.46899999999999997,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.028,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.68200000000000005,"15min":0.65900000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.998,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.79600000000000004}]},{"osd":6,"last update":"Tue Jul 18 03:06:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.60099999999999998,"15min":0.59099999999999997},"min":{"1min":0.40400000000000003,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.97299999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.63400000000000001,"15min":0.60399999999999998},"min":{"1min":0.40300000000000002,"5min":0.372,"15min":0.372},"max":{"1min":1.1639999999999999,"5min":1.216,"15min":1.216},"last":0.55900000000000005}]},{"osd":7,"last update":"Tue Jul 18 03:07:04 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.80600000000000005,"15min":0.77500000000000002},"min":{"1min":0.57299999999999995,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":5.2869999999999999,"5min":5.2869999999999999,"15min":5.2869999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.024,"5min":0.86899999999999999,"15min":0.84299999999999997},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":5.3579999999999997,"5min":5.3579999999999997,"15min":5.3579999999999997},"last":0.88100000000000001}]},{"osd":8,"last update":"Tue Jul 18 03:07:08 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":0.80200000000000005,"15min":0.76400000000000001},"min":{"1min":0.52900000000000003,"5min":0.495,"15min":0.495},"max":{"1min":6.96,"5min":6.96,"15min":6.96},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.0980000000000001,"5min":0.873,"15min":0.84799999999999998},"min":{"1min":0.54300000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":6.9909999999999997,"5min":6.9909999999999997,"15min":6.9909999999999997},"last":0.90000000000000002}]},{"osd":9,"last update":"Tue Jul 18 03:06:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.77900000000000003,"15min":0.78700000000000003},"min":{"1min":0.46300000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0409999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.85699999999999998,"15min":0.873},"min":{"1min":0.497,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1879999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.85699999999999998}]},{"osd":10,"last update":"Tue Jul 18 03:06:54 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":0.96699999999999997,"15min":0.94099999999999995},"min":{"1min":0.70899999999999996,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":6.9359999999999999,"5min":6.9359999999999999,"15min":6.9359999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.748,"5min":0.78000000000000003,"15min":0.78900000000000003},"min":{"1min":0.57899999999999996,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.99299999999999999,"5min":1.304,"15min":1.304},"last":0.69399999999999995}]},{"osd":11,"last update":"Tue Jul 18 03:06:16 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.81299999999999994,"15min":0.80900000000000005},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.0760000000000001,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.88600000000000001,"15min":0.88200000000000001},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.3160000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.93700000000000006}]}]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1064,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1089536,"data_stored":571053,"data_compressed":23524,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.56799999999999995,"15min":0.54000000000000004},"min":{"1min":0.38800000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58399999999999996,"15min":0.55500000000000005},"min":{"1min":0.41399999999999998,"5min":0.373,"15min":0.34599999999999997},"max":{"1min":0.89600000000000002,"5min":1.726,"15min":1.726},"last":0.46899999999999997}]},{"osd":1,"last update":"Tue Jul 18 03:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.61499999999999999,"15min":0.52700000000000002},"min":{"1min":0.49199999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.89200000000000002,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.60399999999999998,"15min":0.52000000000000002},"min":{"1min":0.47299999999999998,"5min":0.34300000000000003,"15min":0.32100000000000001},"max":{"1min":0.82499999999999996,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.57899999999999996}]},{"osd":3,"last update":"Tue Jul 18 03:06:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59299999999999997,"15min":0.56299999999999994},"min":{"1min":0.439,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93899999999999995,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.64400000000000002,"15min":0.59299999999999997},"min":{"1min":0.40899999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.127,"5min":3.234,"15min":3.234},"last":0.60599999999999998}]},{"osd":4,"last update":"Tue Jul 18 03:07:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66400000000000003,"15min":0.66300000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.73599999999999999,"15min":0.74199999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.95699999999999996,"5min":1.353,"15min":1.353},"last":0.55400000000000005}]},{"osd":5,"last update":"Tue Jul 18 03:07:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.51000000000000001,"15min":0.46200000000000002},"min":{"1min":0.34899999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.91500000000000004,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.53800000000000003,"15min":0.47199999999999998},"min":{"1min":0.41399999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.0309999999999999,"5min":1.335,"15min":1.335},"last":0.39900000000000002}]},{"osd":6,"last update":"Tue Jul 18 03:06:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.67700000000000005,"15min":0.64600000000000002},"min":{"1min":0.55700000000000005,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.95399999999999996,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.622},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.72799999999999998,"15min":0.69399999999999995},"min":{"1min":0.56999999999999995,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.6619999999999999,"5min":1.665,"15min":1.665},"last":0.67700000000000005}]},{"osd":7,"last update":"Tue Jul 18 03:07:04 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.84899999999999998,"15min":0.85299999999999998},"min":{"1min":0.61399999999999999,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0129999999999999,"5min":1.532,"15min":1.532},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.78000000000000003,"15min":0.78100000000000003},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.66300000000000003}]},{"osd":8,"last update":"Tue Jul 18 03:07:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.70099999999999996,"15min":0.64100000000000001},"min":{"1min":0.48799999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.159,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.70999999999999996,"15min":0.66700000000000004},"min":{"1min":0.61799999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.079,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.69799999999999995}]},{"osd":9,"last update":"Tue Jul 18 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.753,"15min":0.74199999999999999},"min":{"1min":0.49299999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.2549999999999999,"5min":1.825,"15min":1.825},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.81599999999999995,"15min":0.79300000000000004},"min":{"1min":0.70899999999999996,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.1020000000000001,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.80600000000000005}]},{"osd":10,"last update":"Tue Jul 18 03:06:54 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.82999999999999996,"15min":0.79300000000000004},"min":{"1min":0.79200000000000004,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1000000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.78400000000000003,"15min":0.753},"min":{"1min":0.71299999999999997,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0089999999999999,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.749}]},{"osd":11,"last update":"Tue Jul 18 03:07:04 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.92400000000000004,"15min":0.92600000000000005},"min":{"1min":0.73399999999999999,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1839999999999999,"5min":1.76,"15min":1.76},"last":0.875},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.873,"15min":0.876},"min":{"1min":0.67400000000000004,"5min":0.624,"15min":0.624},"max":{"1min":1.1160000000000001,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.78800000000000003}]}]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":1003520,"data_stored":502723,"data_compressed":23524,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:06:25 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.499,"15min":0.42699999999999999},"min":{"1min":0.34999999999999998,"5min":0.312,"15min":0.312},"max":{"1min":1.8440000000000001,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":0.748},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.504,"15min":0.433},"min":{"1min":0.35899999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.93300000000000005}]},{"osd":1,"last update":"Tue Jul 18 03:06:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.60499999999999998,"15min":0.57199999999999995},"min":{"1min":0.45600000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.79300000000000004,"5min":1.115,"15min":1.115},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.66700000000000004,"15min":0.59499999999999997},"min":{"1min":0.44400000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":0.79200000000000004}]},{"osd":2,"last update":"Tue Jul 18 03:06:25 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.68100000000000005,"15min":0.61399999999999999},"min":{"1min":0.437,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0960000000000001,"5min":1.26,"15min":1.26},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.63700000000000001,"15min":0.60299999999999998},"min":{"1min":0.46600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.048,"5min":1.212,"15min":1.212},"last":0.78200000000000003}]},{"osd":4,"last update":"Tue Jul 18 03:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.70299999999999996,"15min":0.63300000000000001},"min":{"1min":0.56999999999999995,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.706,"5min":1.706,"15min":1.706},"last":0.871},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.69999999999999996,"15min":0.629},"min":{"1min":0.55100000000000005,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0629999999999999,"5min":1.143,"15min":1.143},"last":0.89400000000000002}]},{"osd":5,"last update":"Tue Jul 18 03:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.72099999999999997,"15min":0.64900000000000002},"min":{"1min":0.53200000000000003,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.73799999999999999,"15min":0.68999999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0329999999999999,"5min":1.218,"15min":1.218},"last":0.72799999999999998}]},{"osd":6,"last update":"Tue Jul 18 03:06:29 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.71399999999999997,"15min":0.65800000000000003},"min":{"1min":0.434,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.65300000000000002,"15min":0.61499999999999999},"min":{"1min":0.371,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":2.669,"5min":2.669,"15min":2.669},"last":0.69899999999999995}]},{"osd":7,"last update":"Tue Jul 18 03:06:54 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.76300000000000001,"15min":0.73099999999999998},"min":{"1min":0.56999999999999995,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.78600000000000003,"15min":0.75900000000000001},"min":{"1min":0.58799999999999997,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.1659999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.80300000000000005}]},{"osd":8,"last update":"Tue Jul 18 03:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.76200000000000001}]},{"osd":9,"last update":"Tue Jul 18 03:06:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.76200000000000001,"15min":0.78300000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":0.88},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.86199999999999999,"15min":0.876},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.91600000000000004}]},{"osd":10,"last update":"Tue Jul 18 03:06:54 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.89500000000000002,"15min":0.90700000000000003},"min":{"1min":0.63600000000000001,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.77,"5min":1.77,"15min":1.77},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.875,"15min":0.88500000000000001},"min":{"1min":0.66500000000000004,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.159,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.97499999999999998}]},{"osd":11,"last update":"Tue Jul 18 03:07:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84199999999999997,"15min":0.84099999999999997},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.72499999999999998,"15min":0.71899999999999997},"min":{"1min":0.58999999999999997,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.81699999999999995}]}]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":1003520,"data_stored":507131,"data_compressed":23524,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.56200000000000006,"15min":0.54600000000000004},"min":{"1min":0.39600000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.439},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.56899999999999995,"15min":0.54500000000000004},"min":{"1min":0.42799999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.48999999999999999}]},{"osd":1,"last update":"Tue Jul 18 03:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40699999999999997,"15min":0.36799999999999999},"min":{"1min":0.28599999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.72099999999999997,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.41799999999999998,"15min":0.38400000000000001},"min":{"1min":0.314,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.58599999999999997,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.41299999999999998}]},{"osd":2,"last update":"Tue Jul 18 03:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.59899999999999998,"15min":0.57499999999999996},"min":{"1min":0.43099999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.85699999999999998,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.622,"15min":0.57299999999999995},"min":{"1min":0.41899999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.88100000000000001,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.60099999999999998}]},{"osd":3,"last update":"Tue Jul 18 03:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.71399999999999997,"15min":0.61699999999999999},"min":{"1min":0.51100000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":5.726,"5min":5.726,"15min":5.726},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.65300000000000002,"15min":0.59999999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":4.141,"5min":4.141,"15min":4.141},"last":0.439}]},{"osd":5,"last update":"Tue Jul 18 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.67600000000000005,"15min":0.61799999999999999},"min":{"1min":0.42699999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.86399999999999999,"5min":1.214,"15min":1.214},"last":0.62},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.65700000000000003,"15min":0.61499999999999999},"min":{"1min":0.48599999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.85099999999999998,"5min":1.306,"15min":1.306},"last":0.68300000000000005}]},{"osd":6,"last update":"Tue Jul 18 03:06:27 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.754,"15min":0.73799999999999999},"min":{"1min":0.46500000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":4.2850000000000001,"5min":4.2850000000000001,"15min":4.2850000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.69599999999999995,"15min":0.66900000000000004},"min":{"1min":0.41299999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":4.7130000000000001,"5min":4.7130000000000001,"15min":4.7130000000000001},"last":0.52800000000000002}]},{"osd":7,"last update":"Tue Jul 18 03:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.39700000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.89800000000000002,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.65300000000000002,"15min":0.64700000000000002},"min":{"1min":0.38700000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.98799999999999999,"5min":1.034,"15min":1.034},"last":0.58199999999999996}]},{"osd":8,"last update":"Tue Jul 18 03:06:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73999999999999999,"15min":0.73499999999999999},"min":{"1min":0.53300000000000003,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.99399999999999999,"5min":1.155,"15min":1.155},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.75800000000000001,"15min":0.74399999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.008,"5min":1.135,"15min":1.135},"last":0.72199999999999998}]},{"osd":9,"last update":"Tue Jul 18 03:06:35 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.77300000000000002,"15min":0.73799999999999999},"min":{"1min":0.56299999999999994,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":4.6719999999999997,"5min":4.6719999999999997,"15min":4.6719999999999997},"last":0.63200000000000001},{"interface":"front","average":{"1min":1.0169999999999999,"5min":0.86499999999999999,"15min":0.82499999999999996},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":5.0060000000000002,"5min":5.0060000000000002,"15min":5.0060000000000002},"last":0.73399999999999999}]},{"osd":10,"last update":"Tue Jul 18 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.78100000000000003,"15min":0.79400000000000004},"min":{"1min":0.54500000000000004,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97799999999999998,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68400000000000005,"15min":0.69699999999999995},"min":{"1min":0.47699999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91200000000000003,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.56799999999999995}]},{"osd":11,"last update":"Tue Jul 18 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.80300000000000005,"15min":0.80000000000000004},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.76300000000000001,"15min":0.76000000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.746}]}]},{"osd":5,"up_from":38,"seq":163208757315,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1069056,"data_stored":1098862,"data_compressed":59283,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56100000000000005,"15min":0.53500000000000003},"min":{"1min":0.40600000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.80500000000000005,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.53700000000000003,"15min":0.53400000000000003},"min":{"1min":0.379,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.78100000000000003,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.69399999999999995}]},{"osd":1,"last update":"Tue Jul 18 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.60199999999999998,"15min":0.55200000000000005},"min":{"1min":0.435,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.57799999999999996,"15min":0.55200000000000005},"min":{"1min":0.42399999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.74399999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.65400000000000003}]},{"osd":2,"last update":"Tue Jul 18 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.42099999999999999,"15min":0.38500000000000001},"min":{"1min":0.33800000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.436,"15min":0.40300000000000002},"min":{"1min":0.311,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.57499999999999996}]},{"osd":3,"last update":"Tue Jul 18 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.60599999999999998,"15min":0.57399999999999995},"min":{"1min":0.44500000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.81999999999999995,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.60999999999999999,"15min":0.57699999999999996},"min":{"1min":0.53500000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77600000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.72599999999999998}]},{"osd":4,"last update":"Tue Jul 18 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60599999999999998,"15min":0.57599999999999996},"min":{"1min":0.41899999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.77600000000000002,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.63500000000000001,"15min":0.57999999999999996},"min":{"1min":0.39900000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.86899999999999999,"5min":0.871,"15min":0.871},"last":0.84899999999999998}]},{"osd":6,"last update":"Tue Jul 18 03:06:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.627,"15min":0.60699999999999998},"min":{"1min":0.48299999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.75800000000000001,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.75,"5min":0.66900000000000004,"15min":0.63200000000000001},"min":{"1min":0.58799999999999997,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.83299999999999996}]},{"osd":7,"last update":"Tue Jul 18 03:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.48199999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.83099999999999996,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.69899999999999995,"15min":0.69299999999999995},"min":{"1min":0.49099999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.85399999999999998,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.80800000000000005}]},{"osd":8,"last update":"Tue Jul 18 03:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.56899999999999995,"15min":0.57399999999999995},"min":{"1min":0.45100000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.85899999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.63500000000000001,"15min":0.628},"min":{"1min":0.40799999999999997,"5min":0.371,"15min":0.371},"max":{"1min":0.84499999999999997,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.89500000000000002}]},{"osd":9,"last update":"Tue Jul 18 03:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76200000000000001,"15min":0.77000000000000002},"min":{"1min":0.52600000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.88300000000000001,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.76000000000000001,"15min":0.76600000000000001},"min":{"1min":0.64900000000000002,"5min":0.501,"15min":0.501},"max":{"1min":0.90300000000000002,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.90400000000000003}]},{"osd":10,"last update":"Tue Jul 18 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.75,"15min":0.73599999999999999},"min":{"1min":0.54800000000000004,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.92000000000000004,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.754,"15min":0.74299999999999999},"min":{"1min":0.56299999999999994,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.92400000000000004,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.877}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34117,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34117,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34117,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35759,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35759,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35759,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35759,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35759,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":66781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":68614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":68311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-18T03:07:20.229 INFO:tasks.fwd_scrub.ceph_manager:clean! 2023-07-18T03:07:20.230 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-18T03:07:20.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:19 smithi155 ceph-mon[128885]: pgmap v349: 129 pgs: 129 active+clean; 646 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 6 op/s 2023-07-18T03:07:20.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:19 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/3584328588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:07:20.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.470+0000 7f82fc1ff700 1 -- 172.21.15.101:0/2350220943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82f4101680 msgr2=0x7f82f4107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:20.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.470+0000 7f82fc1ff700 1 --2- 172.21.15.101:0/2350220943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82f4101680 0x7f82f4107f40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00b0a0 tx=0x7f82ec00b3b0 comp rx=0 tx=0).stop 2023-07-18T03:07:20.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.470+0000 7f82fc1ff700 1 -- 172.21.15.101:0/2350220943 shutdown_connections 2023-07-18T03:07:20.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.470+0000 7f82fc1ff700 1 --2- 172.21.15.101:0/2350220943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82f40ffb00 0x7f82f40fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:20.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.470+0000 7f82fc1ff700 1 --2- 172.21.15.101:0/2350220943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82f4101680 0x7f82f4107f40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:20.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.470+0000 7f82fc1ff700 1 --2- 172.21.15.101:0/2350220943 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f82f4100cf0 0x7f82f4101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:20.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.470+0000 7f82fc1ff700 1 -- 172.21.15.101:0/2350220943 >> 172.21.15.101:0/2350220943 conn(0x7f82f40fb0e0 msgr2=0x7f82f40fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:20.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.470+0000 7f82fc1ff700 1 -- 172.21.15.101:0/2350220943 shutdown_connections 2023-07-18T03:07:20.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.471+0000 7f82fc1ff700 1 -- 172.21.15.101:0/2350220943 wait complete. 2023-07-18T03:07:20.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.471+0000 7f82fc1ff700 1 Processor -- start 2023-07-18T03:07:20.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.472+0000 7f82fc1ff700 1 -- start start 2023-07-18T03:07:20.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.472+0000 7f82fc1ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82f40ffb00 0x7f82f41978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:20.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.472+0000 7f82fc1ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f82f4100cf0 0x7f82f4197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:20.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.472+0000 7f82fc1ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82f4101680 0x7f82f419eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:20.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.472+0000 7f82fc1ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f419f470 con 0x7f82f4100cf0 2023-07-18T03:07:20.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.472+0000 7f82fc1ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f419f5b0 con 0x7f82f4101680 2023-07-18T03:07:20.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.472+0000 7f82fc1ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f419f740 con 0x7f82f40ffb00 2023-07-18T03:07:20.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.472+0000 7f82f979a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f82f4100cf0 0x7f82f4197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:20.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.472+0000 7f82f979a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f82f4100cf0 0x7f82f4197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:36482/0 (socket says 172.21.15.101:36482) 2023-07-18T03:07:20.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.472+0000 7f82f979a700 1 -- 172.21.15.101:0/615372853 learned_addr learned my addr 172.21.15.101:0/615372853 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:20.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.472+0000 7f82f9f9b700 1 --2- 172.21.15.101:0/615372853 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82f40ffb00 0x7f82f41978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:20.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.472+0000 7f82fa79c700 1 --2- 172.21.15.101:0/615372853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82f4101680 0x7f82f419eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:20.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.473+0000 7f82f979a700 1 -- 172.21.15.101:0/615372853 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82f40ffb00 msgr2=0x7f82f41978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:20.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.473+0000 7f82f979a700 1 --2- 172.21.15.101:0/615372853 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82f40ffb00 0x7f82f41978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:20.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.473+0000 7f82f979a700 1 -- 172.21.15.101:0/615372853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82f4101680 msgr2=0x7f82f419eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:20.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.473+0000 7f82f979a700 1 --2- 172.21.15.101:0/615372853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82f4101680 0x7f82f419eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:20.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.473+0000 7f82f979a700 1 -- 172.21.15.101:0/615372853 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82ec009d00 con 0x7f82f4100cf0 2023-07-18T03:07:20.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.473+0000 7f82f979a700 1 --2- 172.21.15.101:0/615372853 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f82f4100cf0 0x7f82f4197e30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f82e400e9a0 tx=0x7f82e400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:20.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.473+0000 7f82eaffd700 1 -- 172.21.15.101:0/615372853 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e4009790 con 0x7f82f4100cf0 2023-07-18T03:07:20.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.473+0000 7f82fc1ff700 1 -- 172.21.15.101:0/615372853 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82f419fa20 con 0x7f82f4100cf0 2023-07-18T03:07:20.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.473+0000 7f82eaffd700 1 -- 172.21.15.101:0/615372853 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82e4010430 con 0x7f82f4100cf0 2023-07-18T03:07:20.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.473+0000 7f82eaffd700 1 -- 172.21.15.101:0/615372853 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e4018680 con 0x7f82f4100cf0 2023-07-18T03:07:20.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.473+0000 7f82fc1ff700 1 -- 172.21.15.101:0/615372853 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82f419ff30 con 0x7f82f4100cf0 2023-07-18T03:07:20.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.474+0000 7f82fc1ff700 1 -- 172.21.15.101:0/615372853 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82f410b270 con 0x7f82f4100cf0 2023-07-18T03:07:20.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.476+0000 7f82eaffd700 1 -- 172.21.15.101:0/615372853 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f82e40105a0 con 0x7f82f4100cf0 2023-07-18T03:07:20.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.477+0000 7f82eaffd700 1 --2- 172.21.15.101:0/615372853 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f82e00a11a0 0x7f82e00a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:20.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.478+0000 7f82eaffd700 1 -- 172.21.15.101:0/615372853 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f82e4014070 con 0x7f82f4100cf0 2023-07-18T03:07:20.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.478+0000 7f82f9f9b700 1 --2- 172.21.15.101:0/615372853 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f82e00a11a0 0x7f82e00a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:20.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.478+0000 7f82f9f9b700 1 --2- 172.21.15.101:0/615372853 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f82e00a11a0 0x7f82e00a3650 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f82f4100b50 tx=0x7f82f0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:20.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.481+0000 7f82eaffd700 1 -- 172.21.15.101:0/615372853 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f82e4051790 con 0x7f82f4100cf0 2023-07-18T03:07:20.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.680+0000 7f82fc1ff700 1 -- 172.21.15.101:0/615372853 --> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f82f4107540 con 0x7f82e00a11a0 2023-07-18T03:07:20.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.692+0000 7f82eaffd700 1 -- 172.21.15.101:0/615372853 <== mgr.14150 v2:172.21.15.101:6800/3963742842 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427075 (secure 0 0 0) 0x7f82f4107540 con 0x7f82e00a11a0 2023-07-18T03:07:20.683 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:07:20.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.698+0000 7f82fc1ff700 1 -- 172.21.15.101:0/615372853 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f82e00a11a0 msgr2=0x7f82e00a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:20.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.698+0000 7f82fc1ff700 1 --2- 172.21.15.101:0/615372853 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f82e00a11a0 0x7f82e00a3650 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f82f4100b50 tx=0x7f82f0009260 comp rx=0 tx=0).stop 2023-07-18T03:07:20.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.698+0000 7f82fc1ff700 1 -- 172.21.15.101:0/615372853 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f82f4100cf0 msgr2=0x7f82f4197e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:20.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.698+0000 7f82fc1ff700 1 --2- 172.21.15.101:0/615372853 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f82f4100cf0 0x7f82f4197e30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f82e400e9a0 tx=0x7f82e400ecb0 comp rx=0 tx=0).stop 2023-07-18T03:07:20.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.698+0000 7f82fc1ff700 1 -- 172.21.15.101:0/615372853 shutdown_connections 2023-07-18T03:07:20.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.698+0000 7f82fc1ff700 1 --2- 172.21.15.101:0/615372853 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f82e00a11a0 0x7f82e00a3650 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:20.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.698+0000 7f82fc1ff700 1 --2- 172.21.15.101:0/615372853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82f4101680 0x7f82f419eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:20.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.698+0000 7f82fc1ff700 1 --2- 172.21.15.101:0/615372853 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82f40ffb00 0x7f82f41978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:20.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.698+0000 7f82fc1ff700 1 --2- 172.21.15.101:0/615372853 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f82f4100cf0 0x7f82f4197e30 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:20.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.698+0000 7f82fc1ff700 1 -- 172.21.15.101:0/615372853 >> 172.21.15.101:0/615372853 conn(0x7f82f40fb0e0 msgr2=0x7f82f40fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:20.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.698+0000 7f82fc1ff700 1 -- 172.21.15.101:0/615372853 shutdown_connections 2023-07-18T03:07:20.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.698+0000 7f82fc1ff700 1 -- 172.21.15.101:0/615372853 wait complete. 2023-07-18T03:07:20.688 INFO:teuthology.orchestra.run.smithi101.stderr:dumped all 2023-07-18T03:07:20.704 INFO:teuthology.orchestra.run.smithi101.stdout:{"pg_ready":true,"pg_map":{"version":349,"stamp":"2023-07-18T03:07:18.885623+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661440,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":791,"num_read_kb":523,"num_write":349,"num_write_kb":3064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":785,"ondisk_log_size":785,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335600,"kb_used_data":12176,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606224,"statfs":{"total":1151940427776,"available":1151596773376,"internally_reserved":0,"allocated":12468224,"data_stored":8045618,"data_compressed":389565,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5695,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":77,"num_read_kb":47,"num_write":5,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002975"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.887431+0000","last_change":"2023-07-18T03:04:59.805689+0000","last_active":"2023-07-18T03:06:52.887431+0000","last_peered":"2023-07-18T03:06:52.887431+0000","last_clean":"2023-07-18T03:06:52.887431+0000","last_became_active":"2023-07-18T03:04:59.805224+0000","last_became_peered":"2023-07-18T03:04:59.805224+0000","last_unstale":"2023-07-18T03:06:52.887431+0000","last_undegraded":"2023-07-18T03:06:52.887431+0000","last_fullsized":"2023-07-18T03:06:52.887431+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:06:51.978483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.886501+0000","last_change":"2023-07-18T03:04:59.805013+0000","last_active":"2023-07-18T03:06:52.886501+0000","last_peered":"2023-07-18T03:06:52.886501+0000","last_clean":"2023-07-18T03:06:52.886501+0000","last_became_active":"2023-07-18T03:04:59.804702+0000","last_became_peered":"2023-07-18T03:04:59.804702+0000","last_unstale":"2023-07-18T03:06:52.886501+0000","last_undegraded":"2023-07-18T03:06:52.886501+0000","last_fullsized":"2023-07-18T03:06:52.886501+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:50:03.882265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.884770+0000","last_change":"2023-07-18T03:04:59.809035+0000","last_active":"2023-07-18T03:06:52.884770+0000","last_peered":"2023-07-18T03:06:52.884770+0000","last_clean":"2023-07-18T03:06:52.884770+0000","last_became_active":"2023-07-18T03:04:59.808630+0000","last_became_peered":"2023-07-18T03:04:59.808630+0000","last_unstale":"2023-07-18T03:06:52.884770+0000","last_undegraded":"2023-07-18T03:06:52.884770+0000","last_fullsized":"2023-07-18T03:06:52.884770+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:09:21.776254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.881696+0000","last_change":"2023-07-18T03:04:59.803510+0000","last_active":"2023-07-18T03:06:52.881696+0000","last_peered":"2023-07-18T03:06:52.881696+0000","last_clean":"2023-07-18T03:06:52.881696+0000","last_became_active":"2023-07-18T03:04:59.803060+0000","last_became_peered":"2023-07-18T03:04:59.803060+0000","last_unstale":"2023-07-18T03:06:52.881696+0000","last_undegraded":"2023-07-18T03:06:52.881696+0000","last_fullsized":"2023-07-18T03:06:52.881696+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:05:42.674904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.878377+0000","last_change":"2023-07-18T03:04:59.806911+0000","last_active":"2023-07-18T03:06:52.878377+0000","last_peered":"2023-07-18T03:06:52.878377+0000","last_clean":"2023-07-18T03:06:52.878377+0000","last_became_active":"2023-07-18T03:04:59.806597+0000","last_became_peered":"2023-07-18T03:04:59.806597+0000","last_unstale":"2023-07-18T03:06:52.878377+0000","last_undegraded":"2023-07-18T03:06:52.878377+0000","last_fullsized":"2023-07-18T03:06:52.878377+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:23:04.609208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.876882+0000","last_change":"2023-07-18T03:04:59.809263+0000","last_active":"2023-07-18T03:06:52.876882+0000","last_peered":"2023-07-18T03:06:52.876882+0000","last_clean":"2023-07-18T03:06:52.876882+0000","last_became_active":"2023-07-18T03:04:59.808956+0000","last_became_peered":"2023-07-18T03:04:59.808956+0000","last_unstale":"2023-07-18T03:06:52.876882+0000","last_undegraded":"2023-07-18T03:06:52.876882+0000","last_fullsized":"2023-07-18T03:06:52.876882+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:22:13.425291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.881590+0000","last_change":"2023-07-18T03:04:59.804946+0000","last_active":"2023-07-18T03:06:52.881590+0000","last_peered":"2023-07-18T03:06:52.881590+0000","last_clean":"2023-07-18T03:06:52.881590+0000","last_became_active":"2023-07-18T03:04:59.804610+0000","last_became_peered":"2023-07-18T03:04:59.804610+0000","last_unstale":"2023-07-18T03:06:52.881590+0000","last_undegraded":"2023-07-18T03:06:52.881590+0000","last_fullsized":"2023-07-18T03:06:52.881590+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:35:53.147695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.886091+0000","last_change":"2023-07-18T03:04:59.801796+0000","last_active":"2023-07-18T03:06:52.886091+0000","last_peered":"2023-07-18T03:06:52.886091+0000","last_clean":"2023-07-18T03:06:52.886091+0000","last_became_active":"2023-07-18T03:04:59.801432+0000","last_became_peered":"2023-07-18T03:04:59.801432+0000","last_unstale":"2023-07-18T03:06:52.886091+0000","last_undegraded":"2023-07-18T03:06:52.886091+0000","last_fullsized":"2023-07-18T03:06:52.886091+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:32:07.392642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.887741+0000","last_change":"2023-07-18T03:04:59.798191+0000","last_active":"2023-07-18T03:06:52.887741+0000","last_peered":"2023-07-18T03:06:52.887741+0000","last_clean":"2023-07-18T03:06:52.887741+0000","last_became_active":"2023-07-18T03:04:59.797863+0000","last_became_peered":"2023-07-18T03:04:59.797863+0000","last_unstale":"2023-07-18T03:06:52.887741+0000","last_undegraded":"2023-07-18T03:06:52.887741+0000","last_fullsized":"2023-07-18T03:06:52.887741+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:43:29.304091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.881104+0000","last_change":"2023-07-18T03:04:59.809867+0000","last_active":"2023-07-18T03:06:52.881104+0000","last_peered":"2023-07-18T03:06:52.881104+0000","last_clean":"2023-07-18T03:06:52.881104+0000","last_became_active":"2023-07-18T03:04:59.809568+0000","last_became_peered":"2023-07-18T03:04:59.809568+0000","last_unstale":"2023-07-18T03:06:52.881104+0000","last_undegraded":"2023-07-18T03:06:52.881104+0000","last_fullsized":"2023-07-18T03:06:52.881104+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:49:15.008273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.886428+0000","last_change":"2023-07-18T03:04:59.805920+0000","last_active":"2023-07-18T03:06:52.886428+0000","last_peered":"2023-07-18T03:06:52.886428+0000","last_clean":"2023-07-18T03:06:52.886428+0000","last_became_active":"2023-07-18T03:04:59.805559+0000","last_became_peered":"2023-07-18T03:04:59.805559+0000","last_unstale":"2023-07-18T03:06:52.886428+0000","last_undegraded":"2023-07-18T03:06:52.886428+0000","last_fullsized":"2023-07-18T03:06:52.886428+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:27:52.167301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.880048+0000","last_change":"2023-07-18T03:04:59.797940+0000","last_active":"2023-07-18T03:06:52.880048+0000","last_peered":"2023-07-18T03:06:52.880048+0000","last_clean":"2023-07-18T03:06:52.880048+0000","last_became_active":"2023-07-18T03:04:59.797575+0000","last_became_peered":"2023-07-18T03:04:59.797575+0000","last_unstale":"2023-07-18T03:06:52.880048+0000","last_undegraded":"2023-07-18T03:06:52.880048+0000","last_fullsized":"2023-07-18T03:06:52.880048+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:58:43.120283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.894941+0000","last_change":"2023-07-18T03:04:59.798026+0000","last_active":"2023-07-18T03:06:52.894941+0000","last_peered":"2023-07-18T03:06:52.894941+0000","last_clean":"2023-07-18T03:06:52.894941+0000","last_became_active":"2023-07-18T03:04:59.797734+0000","last_became_peered":"2023-07-18T03:04:59.797734+0000","last_unstale":"2023-07-18T03:06:52.894941+0000","last_undegraded":"2023-07-18T03:06:52.894941+0000","last_fullsized":"2023-07-18T03:06:52.894941+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:33:40.010231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.886275+0000","last_change":"2023-07-18T03:04:59.804964+0000","last_active":"2023-07-18T03:06:52.886275+0000","last_peered":"2023-07-18T03:06:52.886275+0000","last_clean":"2023-07-18T03:06:52.886275+0000","last_became_active":"2023-07-18T03:04:59.804660+0000","last_became_peered":"2023-07-18T03:04:59.804660+0000","last_unstale":"2023-07-18T03:06:52.886275+0000","last_undegraded":"2023-07-18T03:06:52.886275+0000","last_fullsized":"2023-07-18T03:06:52.886275+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:11:48.365954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.885687+0000","last_change":"2023-07-18T03:04:59.809384+0000","last_active":"2023-07-18T03:06:52.885687+0000","last_peered":"2023-07-18T03:06:52.885687+0000","last_clean":"2023-07-18T03:06:52.885687+0000","last_became_active":"2023-07-18T03:04:59.808969+0000","last_became_peered":"2023-07-18T03:04:59.808969+0000","last_unstale":"2023-07-18T03:06:52.885687+0000","last_undegraded":"2023-07-18T03:06:52.885687+0000","last_fullsized":"2023-07-18T03:06:52.885687+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:16:18.639337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.884787+0000","last_change":"2023-07-18T03:04:59.808344+0000","last_active":"2023-07-18T03:06:52.884787+0000","last_peered":"2023-07-18T03:06:52.884787+0000","last_clean":"2023-07-18T03:06:52.884787+0000","last_became_active":"2023-07-18T03:04:59.807995+0000","last_became_peered":"2023-07-18T03:04:59.807995+0000","last_unstale":"2023-07-18T03:06:52.884787+0000","last_undegraded":"2023-07-18T03:06:52.884787+0000","last_fullsized":"2023-07-18T03:06:52.884787+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:15:44.462832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.887652+0000","last_change":"2023-07-18T03:04:59.806584+0000","last_active":"2023-07-18T03:06:52.887652+0000","last_peered":"2023-07-18T03:06:52.887652+0000","last_clean":"2023-07-18T03:06:52.887652+0000","last_became_active":"2023-07-18T03:04:59.804118+0000","last_became_peered":"2023-07-18T03:04:59.804118+0000","last_unstale":"2023-07-18T03:06:52.887652+0000","last_undegraded":"2023-07-18T03:06:52.887652+0000","last_fullsized":"2023-07-18T03:06:52.887652+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:42:18.047326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.890768+0000","last_change":"2023-07-18T03:04:59.812610+0000","last_active":"2023-07-18T03:06:52.890768+0000","last_peered":"2023-07-18T03:06:52.890768+0000","last_clean":"2023-07-18T03:06:52.890768+0000","last_became_active":"2023-07-18T03:04:59.812230+0000","last_became_peered":"2023-07-18T03:04:59.812230+0000","last_unstale":"2023-07-18T03:06:52.890768+0000","last_undegraded":"2023-07-18T03:06:52.890768+0000","last_fullsized":"2023-07-18T03:06:52.890768+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:07:54.810466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.893873+0000","last_change":"2023-07-18T03:04:59.811880+0000","last_active":"2023-07-18T03:06:52.893873+0000","last_peered":"2023-07-18T03:06:52.893873+0000","last_clean":"2023-07-18T03:06:52.893873+0000","last_became_active":"2023-07-18T03:04:59.811561+0000","last_became_peered":"2023-07-18T03:04:59.811561+0000","last_unstale":"2023-07-18T03:06:52.893873+0000","last_undegraded":"2023-07-18T03:06:52.893873+0000","last_fullsized":"2023-07-18T03:06:52.893873+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:53:25.407425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.256306+0000","last_change":"2023-07-18T03:05:03.880843+0000","last_active":"2023-07-18T03:06:24.256306+0000","last_peered":"2023-07-18T03:06:24.256306+0000","last_clean":"2023-07-18T03:06:24.256306+0000","last_became_active":"2023-07-18T03:05:03.880391+0000","last_became_peered":"2023-07-18T03:05:03.880391+0000","last_unstale":"2023-07-18T03:06:24.256306+0000","last_undegraded":"2023-07-18T03:06:24.256306+0000","last_fullsized":"2023-07-18T03:06:24.256306+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:25:52.190341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255219+0000","last_change":"2023-07-18T03:04:58.848771+0000","last_active":"2023-07-18T03:06:24.255219+0000","last_peered":"2023-07-18T03:06:24.255219+0000","last_clean":"2023-07-18T03:06:24.255219+0000","last_became_active":"2023-07-18T03:04:58.848405+0000","last_became_peered":"2023-07-18T03:04:58.848405+0000","last_unstale":"2023-07-18T03:06:24.255219+0000","last_undegraded":"2023-07-18T03:06:24.255219+0000","last_fullsized":"2023-07-18T03:06:24.255219+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:09:13.965066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.881139+0000","last_change":"2023-07-18T03:04:59.805749+0000","last_active":"2023-07-18T03:06:52.881139+0000","last_peered":"2023-07-18T03:06:52.881139+0000","last_clean":"2023-07-18T03:06:52.881139+0000","last_became_active":"2023-07-18T03:04:59.805388+0000","last_became_peered":"2023-07-18T03:04:59.805388+0000","last_unstale":"2023-07-18T03:06:52.881139+0000","last_undegraded":"2023-07-18T03:06:52.881139+0000","last_fullsized":"2023-07-18T03:06:52.881139+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:36:19.960742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255676+0000","last_change":"2023-07-18T03:05:03.868819+0000","last_active":"2023-07-18T03:06:24.255676+0000","last_peered":"2023-07-18T03:06:24.255676+0000","last_clean":"2023-07-18T03:06:24.255676+0000","last_became_active":"2023-07-18T03:05:03.868387+0000","last_became_peered":"2023-07-18T03:05:03.868387+0000","last_unstale":"2023-07-18T03:06:24.255676+0000","last_undegraded":"2023-07-18T03:06:24.255676+0000","last_fullsized":"2023-07-18T03:06:24.255676+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:34:30.099371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.375503+0000","last_change":"2023-07-18T03:04:58.856084+0000","last_active":"2023-07-18T03:06:24.375503+0000","last_peered":"2023-07-18T03:06:24.375503+0000","last_clean":"2023-07-18T03:06:24.375503+0000","last_became_active":"2023-07-18T03:04:58.855768+0000","last_became_peered":"2023-07-18T03:04:58.855768+0000","last_unstale":"2023-07-18T03:06:24.375503+0000","last_undegraded":"2023-07-18T03:06:24.375503+0000","last_fullsized":"2023-07-18T03:06:24.375503+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:58:35.159671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.885217+0000","last_change":"2023-07-18T03:04:59.809512+0000","last_active":"2023-07-18T03:06:52.885217+0000","last_peered":"2023-07-18T03:06:52.885217+0000","last_clean":"2023-07-18T03:06:52.885217+0000","last_became_active":"2023-07-18T03:04:59.808112+0000","last_became_peered":"2023-07-18T03:04:59.808112+0000","last_unstale":"2023-07-18T03:06:52.885217+0000","last_undegraded":"2023-07-18T03:06:52.885217+0000","last_fullsized":"2023-07-18T03:06:52.885217+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:42:34.279489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255283+0000","last_change":"2023-07-18T03:05:03.874728+0000","last_active":"2023-07-18T03:06:24.255283+0000","last_peered":"2023-07-18T03:06:24.255283+0000","last_clean":"2023-07-18T03:06:24.255283+0000","last_became_active":"2023-07-18T03:05:03.874262+0000","last_became_peered":"2023-07-18T03:05:03.874262+0000","last_unstale":"2023-07-18T03:06:24.255283+0000","last_undegraded":"2023-07-18T03:06:24.255283+0000","last_fullsized":"2023-07-18T03:06:24.255283+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:31:54.083323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.261317+0000","last_change":"2023-07-18T03:04:58.849140+0000","last_active":"2023-07-18T03:06:24.261317+0000","last_peered":"2023-07-18T03:06:24.261317+0000","last_clean":"2023-07-18T03:06:24.261317+0000","last_became_active":"2023-07-18T03:04:58.848693+0000","last_became_peered":"2023-07-18T03:04:58.848693+0000","last_unstale":"2023-07-18T03:06:24.261317+0000","last_undegraded":"2023-07-18T03:06:24.261317+0000","last_fullsized":"2023-07-18T03:06:24.261317+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:07:50.859417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.892017+0000","last_change":"2023-07-18T03:04:59.807248+0000","last_active":"2023-07-18T03:06:52.892017+0000","last_peered":"2023-07-18T03:06:52.892017+0000","last_clean":"2023-07-18T03:06:52.892017+0000","last_became_active":"2023-07-18T03:04:59.806807+0000","last_became_peered":"2023-07-18T03:04:59.806807+0000","last_unstale":"2023-07-18T03:06:52.892017+0000","last_undegraded":"2023-07-18T03:06:52.892017+0000","last_fullsized":"2023-07-18T03:06:52.892017+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:07:33.904712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255269+0000","last_change":"2023-07-18T03:05:03.869075+0000","last_active":"2023-07-18T03:06:24.255269+0000","last_peered":"2023-07-18T03:06:24.255269+0000","last_clean":"2023-07-18T03:06:24.255269+0000","last_became_active":"2023-07-18T03:05:03.868653+0000","last_became_peered":"2023-07-18T03:05:03.868653+0000","last_unstale":"2023-07-18T03:06:24.255269+0000","last_undegraded":"2023-07-18T03:06:24.255269+0000","last_fullsized":"2023-07-18T03:06:24.255269+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:05:18.894653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.259374+0000","last_change":"2023-07-18T03:04:58.843879+0000","last_active":"2023-07-18T03:06:24.259374+0000","last_peered":"2023-07-18T03:06:24.259374+0000","last_clean":"2023-07-18T03:06:24.259374+0000","last_became_active":"2023-07-18T03:04:58.843525+0000","last_became_peered":"2023-07-18T03:04:58.843525+0000","last_unstale":"2023-07-18T03:06:24.259374+0000","last_undegraded":"2023-07-18T03:06:24.259374+0000","last_fullsized":"2023-07-18T03:06:24.259374+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:32:03.232823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.886046+0000","last_change":"2023-07-18T03:04:59.803597+0000","last_active":"2023-07-18T03:06:52.886046+0000","last_peered":"2023-07-18T03:06:52.886046+0000","last_clean":"2023-07-18T03:06:52.886046+0000","last_became_active":"2023-07-18T03:04:59.803293+0000","last_became_peered":"2023-07-18T03:04:59.803293+0000","last_unstale":"2023-07-18T03:06:52.886046+0000","last_undegraded":"2023-07-18T03:06:52.886046+0000","last_fullsized":"2023-07-18T03:06:52.886046+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:31:55.196443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255711+0000","last_change":"2023-07-18T03:05:23.944721+0000","last_active":"2023-07-18T03:06:24.255711+0000","last_peered":"2023-07-18T03:06:24.255711+0000","last_clean":"2023-07-18T03:06:24.255711+0000","last_became_active":"2023-07-18T03:05:23.944101+0000","last_became_peered":"2023-07-18T03:05:23.944101+0000","last_unstale":"2023-07-18T03:06:24.255711+0000","last_undegraded":"2023-07-18T03:06:24.255711+0000","last_fullsized":"2023-07-18T03:06:24.255711+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:20:50.819953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":145,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:07:14.145822+0000","last_change":"2023-07-18T03:04:58.847990+0000","last_active":"2023-07-18T03:07:14.145822+0000","last_peered":"2023-07-18T03:07:14.145822+0000","last_clean":"2023-07-18T03:07:14.145822+0000","last_became_active":"2023-07-18T03:04:58.847634+0000","last_became_peered":"2023-07-18T03:04:58.847634+0000","last_unstale":"2023-07-18T03:07:14.145822+0000","last_undegraded":"2023-07-18T03:07:14.145822+0000","last_fullsized":"2023-07-18T03:07:14.145822+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:14:18.454279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1105,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.881401+0000","last_change":"2023-07-18T03:04:59.797076+0000","last_active":"2023-07-18T03:06:52.881401+0000","last_peered":"2023-07-18T03:06:52.881401+0000","last_clean":"2023-07-18T03:06:52.881401+0000","last_became_active":"2023-07-18T03:04:59.796753+0000","last_became_peered":"2023-07-18T03:04:59.796753+0000","last_unstale":"2023-07-18T03:06:52.881401+0000","last_undegraded":"2023-07-18T03:06:52.881401+0000","last_fullsized":"2023-07-18T03:06:52.881401+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:46:03.085633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255734+0000","last_change":"2023-07-18T03:05:03.881793+0000","last_active":"2023-07-18T03:06:24.255734+0000","last_peered":"2023-07-18T03:06:24.255734+0000","last_clean":"2023-07-18T03:06:24.255734+0000","last_became_active":"2023-07-18T03:05:03.881341+0000","last_became_peered":"2023-07-18T03:05:03.881341+0000","last_unstale":"2023-07-18T03:06:24.255734+0000","last_undegraded":"2023-07-18T03:06:24.255734+0000","last_fullsized":"2023-07-18T03:06:24.255734+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:40:23.370420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.375558+0000","last_change":"2023-07-18T03:04:58.846732+0000","last_active":"2023-07-18T03:06:24.375558+0000","last_peered":"2023-07-18T03:06:24.375558+0000","last_clean":"2023-07-18T03:06:24.375558+0000","last_became_active":"2023-07-18T03:04:58.846321+0000","last_became_peered":"2023-07-18T03:04:58.846321+0000","last_unstale":"2023-07-18T03:06:24.375558+0000","last_undegraded":"2023-07-18T03:06:24.375558+0000","last_fullsized":"2023-07-18T03:06:24.375558+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:50:00.381631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.879809+0000","last_change":"2023-07-18T03:05:24.093277+0000","last_active":"2023-07-18T03:06:52.879809+0000","last_peered":"2023-07-18T03:06:52.879809+0000","last_clean":"2023-07-18T03:06:52.879809+0000","last_became_active":"2023-07-18T03:05:24.092842+0000","last_became_peered":"2023-07-18T03:05:24.092842+0000","last_unstale":"2023-07-18T03:06:52.879809+0000","last_undegraded":"2023-07-18T03:06:52.879809+0000","last_fullsized":"2023-07-18T03:06:52.879809+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:22:01.737422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.257633+0000","last_change":"2023-07-18T03:05:03.880782+0000","last_active":"2023-07-18T03:06:24.257633+0000","last_peered":"2023-07-18T03:06:24.257633+0000","last_clean":"2023-07-18T03:06:24.257633+0000","last_became_active":"2023-07-18T03:05:03.880362+0000","last_became_peered":"2023-07-18T03:05:03.880362+0000","last_unstale":"2023-07-18T03:06:24.257633+0000","last_undegraded":"2023-07-18T03:06:24.257633+0000","last_fullsized":"2023-07-18T03:06:24.257633+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:10:14.772789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.750818+0000","last_change":"2023-07-18T03:04:58.846095+0000","last_active":"2023-07-18T03:06:24.750818+0000","last_peered":"2023-07-18T03:06:24.750818+0000","last_clean":"2023-07-18T03:06:24.750818+0000","last_became_active":"2023-07-18T03:04:58.845242+0000","last_became_peered":"2023-07-18T03:04:58.845242+0000","last_unstale":"2023-07-18T03:06:24.750818+0000","last_undegraded":"2023-07-18T03:06:24.750818+0000","last_fullsized":"2023-07-18T03:06:24.750818+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:36:00.462709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.889059+0000","last_change":"2023-07-18T03:04:59.803024+0000","last_active":"2023-07-18T03:06:52.889059+0000","last_peered":"2023-07-18T03:06:52.889059+0000","last_clean":"2023-07-18T03:06:52.889059+0000","last_became_active":"2023-07-18T03:04:59.802687+0000","last_became_peered":"2023-07-18T03:04:59.802687+0000","last_unstale":"2023-07-18T03:06:52.889059+0000","last_undegraded":"2023-07-18T03:06:52.889059+0000","last_fullsized":"2023-07-18T03:06:52.889059+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:59:27.108157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.259440+0000","last_change":"2023-07-18T03:05:03.872690+0000","last_active":"2023-07-18T03:06:24.259440+0000","last_peered":"2023-07-18T03:06:24.259440+0000","last_clean":"2023-07-18T03:06:24.259440+0000","last_became_active":"2023-07-18T03:05:03.872187+0000","last_became_peered":"2023-07-18T03:05:03.872187+0000","last_unstale":"2023-07-18T03:06:24.259440+0000","last_undegraded":"2023-07-18T03:06:24.259440+0000","last_fullsized":"2023-07-18T03:06:24.259440+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:06:57.712294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.261033+0000","last_change":"2023-07-18T03:04:58.842877+0000","last_active":"2023-07-18T03:06:24.261033+0000","last_peered":"2023-07-18T03:06:24.261033+0000","last_clean":"2023-07-18T03:06:24.261033+0000","last_became_active":"2023-07-18T03:04:58.842516+0000","last_became_peered":"2023-07-18T03:04:58.842516+0000","last_unstale":"2023-07-18T03:06:24.261033+0000","last_undegraded":"2023-07-18T03:06:24.261033+0000","last_fullsized":"2023-07-18T03:06:24.261033+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:24:49.503369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.884379+0000","last_change":"2023-07-18T03:04:59.807592+0000","last_active":"2023-07-18T03:06:52.884379+0000","last_peered":"2023-07-18T03:06:52.884379+0000","last_clean":"2023-07-18T03:06:52.884379+0000","last_became_active":"2023-07-18T03:04:59.807276+0000","last_became_peered":"2023-07-18T03:04:59.807276+0000","last_unstale":"2023-07-18T03:06:52.884379+0000","last_undegraded":"2023-07-18T03:06:52.884379+0000","last_fullsized":"2023-07-18T03:06:52.884379+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:16:05.749019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.374960+0000","last_change":"2023-07-18T03:05:03.876355+0000","last_active":"2023-07-18T03:06:24.374960+0000","last_peered":"2023-07-18T03:06:24.374960+0000","last_clean":"2023-07-18T03:06:24.374960+0000","last_became_active":"2023-07-18T03:05:03.875845+0000","last_became_peered":"2023-07-18T03:05:03.875845+0000","last_unstale":"2023-07-18T03:06:24.374960+0000","last_undegraded":"2023-07-18T03:06:24.374960+0000","last_fullsized":"2023-07-18T03:06:24.374960+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:13:06.076832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'4","reported_seq":139,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:07:15.146806+0000","last_change":"2023-07-18T03:04:58.840389+0000","last_active":"2023-07-18T03:07:15.146806+0000","last_peered":"2023-07-18T03:07:15.146806+0000","last_clean":"2023-07-18T03:07:15.146806+0000","last_became_active":"2023-07-18T03:04:58.839974+0000","last_became_peered":"2023-07-18T03:04:58.839974+0000","last_unstale":"2023-07-18T03:07:15.146806+0000","last_undegraded":"2023-07-18T03:07:15.146806+0000","last_fullsized":"2023-07-18T03:07:15.146806+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:40:06.944095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.891931+0000","last_change":"2023-07-18T03:04:59.807681+0000","last_active":"2023-07-18T03:06:52.891931+0000","last_peered":"2023-07-18T03:06:52.891931+0000","last_clean":"2023-07-18T03:06:52.891931+0000","last_became_active":"2023-07-18T03:04:59.807365+0000","last_became_peered":"2023-07-18T03:04:59.807365+0000","last_unstale":"2023-07-18T03:06:52.891931+0000","last_undegraded":"2023-07-18T03:06:52.891931+0000","last_fullsized":"2023-07-18T03:06:52.891931+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:02:22.715114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.258920+0000","last_change":"2023-07-18T03:05:24.091413+0000","last_active":"2023-07-18T03:06:24.258920+0000","last_peered":"2023-07-18T03:06:24.258920+0000","last_clean":"2023-07-18T03:06:24.258920+0000","last_became_active":"2023-07-18T03:05:24.090815+0000","last_became_peered":"2023-07-18T03:05:24.090815+0000","last_unstale":"2023-07-18T03:06:24.258920+0000","last_undegraded":"2023-07-18T03:06:24.258920+0000","last_fullsized":"2023-07-18T03:06:24.258920+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:25:43.324509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.268967+0000","last_change":"2023-07-18T03:06:24.268967+0000","last_active":"2023-07-18T03:06:24.268967+0000","last_peered":"2023-07-18T03:06:24.268967+0000","last_clean":"2023-07-18T03:06:24.268967+0000","last_became_active":"2023-07-18T03:06:24.263424+0000","last_became_peered":"2023-07-18T03:06:24.263424+0000","last_unstale":"2023-07-18T03:06:24.268967+0000","last_undegraded":"2023-07-18T03:06:24.268967+0000","last_fullsized":"2023-07-18T03:06:24.268967+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:13:59.331501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.895054+0000","last_change":"2023-07-18T03:05:24.092870+0000","last_active":"2023-07-18T03:06:52.895054+0000","last_peered":"2023-07-18T03:06:52.895054+0000","last_clean":"2023-07-18T03:06:52.895054+0000","last_became_active":"2023-07-18T03:05:24.092474+0000","last_became_peered":"2023-07-18T03:05:24.092474+0000","last_unstale":"2023-07-18T03:06:52.895054+0000","last_undegraded":"2023-07-18T03:06:52.895054+0000","last_fullsized":"2023-07-18T03:06:52.895054+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:25:17.349689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255481+0000","last_change":"2023-07-18T03:05:03.874444+0000","last_active":"2023-07-18T03:06:24.255481+0000","last_peered":"2023-07-18T03:06:24.255481+0000","last_clean":"2023-07-18T03:06:24.255481+0000","last_became_active":"2023-07-18T03:05:03.873737+0000","last_became_peered":"2023-07-18T03:05:03.873737+0000","last_unstale":"2023-07-18T03:06:24.255481+0000","last_undegraded":"2023-07-18T03:06:24.255481+0000","last_fullsized":"2023-07-18T03:06:24.255481+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:07:21.954472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.261721+0000","last_change":"2023-07-18T03:04:58.845207+0000","last_active":"2023-07-18T03:06:24.261721+0000","last_peered":"2023-07-18T03:06:24.261721+0000","last_clean":"2023-07-18T03:06:24.261721+0000","last_became_active":"2023-07-18T03:04:58.844895+0000","last_became_peered":"2023-07-18T03:04:58.844895+0000","last_unstale":"2023-07-18T03:06:24.261721+0000","last_undegraded":"2023-07-18T03:06:24.261721+0000","last_fullsized":"2023-07-18T03:06:24.261721+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:59:56.250544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.886957+0000","last_change":"2023-07-18T03:04:59.807075+0000","last_active":"2023-07-18T03:06:52.886957+0000","last_peered":"2023-07-18T03:06:52.886957+0000","last_clean":"2023-07-18T03:06:52.886957+0000","last_became_active":"2023-07-18T03:04:59.806345+0000","last_became_peered":"2023-07-18T03:04:59.806345+0000","last_unstale":"2023-07-18T03:06:52.886957+0000","last_undegraded":"2023-07-18T03:06:52.886957+0000","last_fullsized":"2023-07-18T03:06:52.886957+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:50:55.760441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.375376+0000","last_change":"2023-07-18T03:05:24.091132+0000","last_active":"2023-07-18T03:06:24.375376+0000","last_peered":"2023-07-18T03:06:24.375376+0000","last_clean":"2023-07-18T03:06:24.375376+0000","last_became_active":"2023-07-18T03:05:24.090653+0000","last_became_peered":"2023-07-18T03:05:24.090653+0000","last_unstale":"2023-07-18T03:06:24.375376+0000","last_undegraded":"2023-07-18T03:06:24.375376+0000","last_fullsized":"2023-07-18T03:06:24.375376+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:14:06.294348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"91'38","reported_seq":289,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:07:16.164902+0000","last_change":"2023-07-18T03:04:58.844865+0000","last_active":"2023-07-18T03:07:16.164902+0000","last_peered":"2023-07-18T03:07:16.164902+0000","last_clean":"2023-07-18T03:07:16.164902+0000","last_became_active":"2023-07-18T03:04:58.844547+0000","last_became_peered":"2023-07-18T03:04:58.844547+0000","last_unstale":"2023-07-18T03:07:16.164902+0000","last_undegraded":"2023-07-18T03:07:16.164902+0000","last_fullsized":"2023-07-18T03:07:16.164902+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:09:40.602341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66781,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":221,"num_read_kb":176,"num_write":39,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.882102+0000","last_change":"2023-07-18T03:04:59.803024+0000","last_active":"2023-07-18T03:06:52.882102+0000","last_peered":"2023-07-18T03:06:52.882102+0000","last_clean":"2023-07-18T03:06:52.882102+0000","last_became_active":"2023-07-18T03:04:59.802705+0000","last_became_peered":"2023-07-18T03:04:59.802705+0000","last_unstale":"2023-07-18T03:06:52.882102+0000","last_undegraded":"2023-07-18T03:06:52.882102+0000","last_fullsized":"2023-07-18T03:06:52.882102+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:46:31.950692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.262478+0000","last_change":"2023-07-18T03:05:03.876009+0000","last_active":"2023-07-18T03:06:24.262478+0000","last_peered":"2023-07-18T03:06:24.262478+0000","last_clean":"2023-07-18T03:06:24.262478+0000","last_became_active":"2023-07-18T03:05:03.875328+0000","last_became_peered":"2023-07-18T03:05:03.875328+0000","last_unstale":"2023-07-18T03:06:24.262478+0000","last_undegraded":"2023-07-18T03:06:24.262478+0000","last_fullsized":"2023-07-18T03:06:24.262478+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:55:54.930973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.256224+0000","last_change":"2023-07-18T03:04:58.846573+0000","last_active":"2023-07-18T03:06:24.256224+0000","last_peered":"2023-07-18T03:06:24.256224+0000","last_clean":"2023-07-18T03:06:24.256224+0000","last_became_active":"2023-07-18T03:04:58.846271+0000","last_became_peered":"2023-07-18T03:04:58.846271+0000","last_unstale":"2023-07-18T03:06:24.256224+0000","last_undegraded":"2023-07-18T03:06:24.256224+0000","last_fullsized":"2023-07-18T03:06:24.256224+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:22:31.518590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.894308+0000","last_change":"2023-07-18T03:04:59.801552+0000","last_active":"2023-07-18T03:06:52.894308+0000","last_peered":"2023-07-18T03:06:52.894308+0000","last_clean":"2023-07-18T03:06:52.894308+0000","last_became_active":"2023-07-18T03:04:59.801224+0000","last_became_peered":"2023-07-18T03:04:59.801224+0000","last_unstale":"2023-07-18T03:06:52.894308+0000","last_undegraded":"2023-07-18T03:06:52.894308+0000","last_fullsized":"2023-07-18T03:06:52.894308+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:44:10.394026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.257271+0000","last_change":"2023-07-18T03:05:03.881125+0000","last_active":"2023-07-18T03:06:24.257271+0000","last_peered":"2023-07-18T03:06:24.257271+0000","last_clean":"2023-07-18T03:06:24.257271+0000","last_became_active":"2023-07-18T03:05:03.880723+0000","last_became_peered":"2023-07-18T03:05:03.880723+0000","last_unstale":"2023-07-18T03:06:24.257271+0000","last_undegraded":"2023-07-18T03:06:24.257271+0000","last_fullsized":"2023-07-18T03:06:24.257271+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:28:40.857218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.261012+0000","last_change":"2023-07-18T03:04:58.847118+0000","last_active":"2023-07-18T03:06:24.261012+0000","last_peered":"2023-07-18T03:06:24.261012+0000","last_clean":"2023-07-18T03:06:24.261012+0000","last_became_active":"2023-07-18T03:04:58.846805+0000","last_became_peered":"2023-07-18T03:04:58.846805+0000","last_unstale":"2023-07-18T03:06:24.261012+0000","last_undegraded":"2023-07-18T03:06:24.261012+0000","last_fullsized":"2023-07-18T03:06:24.261012+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:48:50.417535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.895015+0000","last_change":"2023-07-18T03:04:59.803914+0000","last_active":"2023-07-18T03:06:52.895015+0000","last_peered":"2023-07-18T03:06:52.895015+0000","last_clean":"2023-07-18T03:06:52.895015+0000","last_became_active":"2023-07-18T03:04:59.803589+0000","last_became_peered":"2023-07-18T03:04:59.803589+0000","last_unstale":"2023-07-18T03:06:52.895015+0000","last_undegraded":"2023-07-18T03:06:52.895015+0000","last_fullsized":"2023-07-18T03:06:52.895015+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:24:46.537710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255012+0000","last_change":"2023-07-18T03:05:03.877787+0000","last_active":"2023-07-18T03:06:24.255012+0000","last_peered":"2023-07-18T03:06:24.255012+0000","last_clean":"2023-07-18T03:06:24.255012+0000","last_became_active":"2023-07-18T03:05:03.877449+0000","last_became_peered":"2023-07-18T03:05:03.877449+0000","last_unstale":"2023-07-18T03:06:24.255012+0000","last_undegraded":"2023-07-18T03:06:24.255012+0000","last_fullsized":"2023-07-18T03:06:24.255012+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:54:47.075087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":137,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:07:12.144954+0000","last_change":"2023-07-18T03:04:58.844252+0000","last_active":"2023-07-18T03:07:12.144954+0000","last_peered":"2023-07-18T03:07:12.144954+0000","last_clean":"2023-07-18T03:07:12.144954+0000","last_became_active":"2023-07-18T03:04:58.843798+0000","last_became_peered":"2023-07-18T03:04:58.843798+0000","last_unstale":"2023-07-18T03:07:12.144954+0000","last_undegraded":"2023-07-18T03:07:12.144954+0000","last_fullsized":"2023-07-18T03:07:12.144954+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:24:14.970709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.883399+0000","last_change":"2023-07-18T03:04:59.806274+0000","last_active":"2023-07-18T03:06:52.883399+0000","last_peered":"2023-07-18T03:06:52.883399+0000","last_clean":"2023-07-18T03:06:52.883399+0000","last_became_active":"2023-07-18T03:04:59.805876+0000","last_became_peered":"2023-07-18T03:04:59.805876+0000","last_unstale":"2023-07-18T03:06:52.883399+0000","last_undegraded":"2023-07-18T03:06:52.883399+0000","last_fullsized":"2023-07-18T03:06:52.883399+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:00:53.965603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.751210+0000","last_change":"2023-07-18T03:05:03.876783+0000","last_active":"2023-07-18T03:06:24.751210+0000","last_peered":"2023-07-18T03:06:24.751210+0000","last_clean":"2023-07-18T03:06:24.751210+0000","last_became_active":"2023-07-18T03:05:03.876385+0000","last_became_peered":"2023-07-18T03:05:03.876385+0000","last_unstale":"2023-07-18T03:06:24.751210+0000","last_undegraded":"2023-07-18T03:06:24.751210+0000","last_fullsized":"2023-07-18T03:06:24.751210+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:12:22.516248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.259648+0000","last_change":"2023-07-18T03:04:58.851107+0000","last_active":"2023-07-18T03:06:24.259648+0000","last_peered":"2023-07-18T03:06:24.259648+0000","last_clean":"2023-07-18T03:06:24.259648+0000","last_became_active":"2023-07-18T03:04:58.850848+0000","last_became_peered":"2023-07-18T03:04:58.850848+0000","last_unstale":"2023-07-18T03:06:24.259648+0000","last_undegraded":"2023-07-18T03:06:24.259648+0000","last_fullsized":"2023-07-18T03:06:24.259648+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:39:44.591519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.885106+0000","last_change":"2023-07-18T03:04:59.803683+0000","last_active":"2023-07-18T03:06:52.885106+0000","last_peered":"2023-07-18T03:06:52.885106+0000","last_clean":"2023-07-18T03:06:52.885106+0000","last_became_active":"2023-07-18T03:04:59.803365+0000","last_became_peered":"2023-07-18T03:04:59.803365+0000","last_unstale":"2023-07-18T03:06:52.885106+0000","last_undegraded":"2023-07-18T03:06:52.885106+0000","last_fullsized":"2023-07-18T03:06:52.885106+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:30:10.236288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.260879+0000","last_change":"2023-07-18T03:05:03.875856+0000","last_active":"2023-07-18T03:06:24.260879+0000","last_peered":"2023-07-18T03:06:24.260879+0000","last_clean":"2023-07-18T03:06:24.260879+0000","last_became_active":"2023-07-18T03:05:03.875426+0000","last_became_peered":"2023-07-18T03:05:03.875426+0000","last_unstale":"2023-07-18T03:06:24.260879+0000","last_undegraded":"2023-07-18T03:06:24.260879+0000","last_fullsized":"2023-07-18T03:06:24.260879+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:48:39.948769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.261226+0000","last_change":"2023-07-18T03:04:58.844102+0000","last_active":"2023-07-18T03:06:24.261226+0000","last_peered":"2023-07-18T03:06:24.261226+0000","last_clean":"2023-07-18T03:06:24.261226+0000","last_became_active":"2023-07-18T03:04:58.843740+0000","last_became_peered":"2023-07-18T03:04:58.843740+0000","last_unstale":"2023-07-18T03:06:24.261226+0000","last_undegraded":"2023-07-18T03:06:24.261226+0000","last_fullsized":"2023-07-18T03:06:24.261226+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:19:46.540756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.883036+0000","last_change":"2023-07-18T03:05:24.091161+0000","last_active":"2023-07-18T03:06:52.883036+0000","last_peered":"2023-07-18T03:06:52.883036+0000","last_clean":"2023-07-18T03:06:52.883036+0000","last_became_active":"2023-07-18T03:05:24.090631+0000","last_became_peered":"2023-07-18T03:05:24.090631+0000","last_unstale":"2023-07-18T03:06:52.883036+0000","last_undegraded":"2023-07-18T03:06:52.883036+0000","last_fullsized":"2023-07-18T03:06:52.883036+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:21:10.676261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.259807+0000","last_change":"2023-07-18T03:05:24.092328+0000","last_active":"2023-07-18T03:06:24.259807+0000","last_peered":"2023-07-18T03:06:24.259807+0000","last_clean":"2023-07-18T03:06:24.259807+0000","last_became_active":"2023-07-18T03:05:24.091839+0000","last_became_peered":"2023-07-18T03:05:24.091839+0000","last_unstale":"2023-07-18T03:06:24.259807+0000","last_undegraded":"2023-07-18T03:06:24.259807+0000","last_fullsized":"2023-07-18T03:06:24.259807+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:42:18.354994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.262294+0000","last_change":"2023-07-18T03:06:24.261616+0000","last_active":"2023-07-18T03:06:24.262294+0000","last_peered":"2023-07-18T03:06:24.262294+0000","last_clean":"2023-07-18T03:06:24.262294+0000","last_became_active":"2023-07-18T03:06:24.261250+0000","last_became_peered":"2023-07-18T03:06:24.261250+0000","last_unstale":"2023-07-18T03:06:24.262294+0000","last_undegraded":"2023-07-18T03:06:24.262294+0000","last_fullsized":"2023-07-18T03:06:24.262294+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:19:24.589920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.893730+0000","last_change":"2023-07-18T03:05:24.092486+0000","last_active":"2023-07-18T03:06:52.893730+0000","last_peered":"2023-07-18T03:06:52.893730+0000","last_clean":"2023-07-18T03:06:52.893730+0000","last_became_active":"2023-07-18T03:05:24.092189+0000","last_became_peered":"2023-07-18T03:05:24.092189+0000","last_unstale":"2023-07-18T03:06:52.893730+0000","last_undegraded":"2023-07-18T03:06:52.893730+0000","last_fullsized":"2023-07-18T03:06:52.893730+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:09:16.496072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.751014+0000","last_change":"2023-07-18T03:05:03.868851+0000","last_active":"2023-07-18T03:06:24.751014+0000","last_peered":"2023-07-18T03:06:24.751014+0000","last_clean":"2023-07-18T03:06:24.751014+0000","last_became_active":"2023-07-18T03:05:03.868433+0000","last_became_peered":"2023-07-18T03:05:03.868433+0000","last_unstale":"2023-07-18T03:06:24.751014+0000","last_undegraded":"2023-07-18T03:06:24.751014+0000","last_fullsized":"2023-07-18T03:06:24.751014+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:36:46.180540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.258040+0000","last_change":"2023-07-18T03:04:58.848059+0000","last_active":"2023-07-18T03:06:24.258040+0000","last_peered":"2023-07-18T03:06:24.258040+0000","last_clean":"2023-07-18T03:06:24.258040+0000","last_became_active":"2023-07-18T03:04:58.847700+0000","last_became_peered":"2023-07-18T03:04:58.847700+0000","last_unstale":"2023-07-18T03:06:24.258040+0000","last_undegraded":"2023-07-18T03:06:24.258040+0000","last_fullsized":"2023-07-18T03:06:24.258040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:22:42.461972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.886520+0000","last_change":"2023-07-18T03:05:23.939939+0000","last_active":"2023-07-18T03:06:52.886520+0000","last_peered":"2023-07-18T03:06:52.886520+0000","last_clean":"2023-07-18T03:06:52.886520+0000","last_became_active":"2023-07-18T03:05:23.939607+0000","last_became_peered":"2023-07-18T03:05:23.939607+0000","last_unstale":"2023-07-18T03:06:52.886520+0000","last_undegraded":"2023-07-18T03:06:52.886520+0000","last_fullsized":"2023-07-18T03:06:52.886520+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:47:48.796749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.751185+0000","last_change":"2023-07-18T03:05:03.886724+0000","last_active":"2023-07-18T03:06:24.751185+0000","last_peered":"2023-07-18T03:06:24.751185+0000","last_clean":"2023-07-18T03:06:24.751185+0000","last_became_active":"2023-07-18T03:05:03.886172+0000","last_became_peered":"2023-07-18T03:05:03.886172+0000","last_unstale":"2023-07-18T03:06:24.751185+0000","last_undegraded":"2023-07-18T03:06:24.751185+0000","last_fullsized":"2023-07-18T03:06:24.751185+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:35:26.182278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.886032+0000","last_change":"2023-07-18T03:04:59.804273+0000","last_active":"2023-07-18T03:06:52.886032+0000","last_peered":"2023-07-18T03:06:52.886032+0000","last_clean":"2023-07-18T03:06:52.886032+0000","last_became_active":"2023-07-18T03:04:59.803987+0000","last_became_peered":"2023-07-18T03:04:59.803987+0000","last_unstale":"2023-07-18T03:06:52.886032+0000","last_undegraded":"2023-07-18T03:06:52.886032+0000","last_fullsized":"2023-07-18T03:06:52.886032+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:53:36.335539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":139,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:07:14.146943+0000","last_change":"2023-07-18T03:04:58.846381+0000","last_active":"2023-07-18T03:07:14.146943+0000","last_peered":"2023-07-18T03:07:14.146943+0000","last_clean":"2023-07-18T03:07:14.146943+0000","last_became_active":"2023-07-18T03:04:58.845757+0000","last_became_peered":"2023-07-18T03:04:58.845757+0000","last_unstale":"2023-07-18T03:07:14.146943+0000","last_undegraded":"2023-07-18T03:07:14.146943+0000","last_fullsized":"2023-07-18T03:07:14.146943+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:00:14.653137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.258055+0000","last_change":"2023-07-18T03:05:03.873626+0000","last_active":"2023-07-18T03:06:24.258055+0000","last_peered":"2023-07-18T03:06:24.258055+0000","last_clean":"2023-07-18T03:06:24.258055+0000","last_became_active":"2023-07-18T03:05:03.872965+0000","last_became_peered":"2023-07-18T03:05:03.872965+0000","last_unstale":"2023-07-18T03:06:24.258055+0000","last_undegraded":"2023-07-18T03:06:24.258055+0000","last_fullsized":"2023-07-18T03:06:24.258055+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:37:38.450618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.881350+0000","last_change":"2023-07-18T03:04:59.801153+0000","last_active":"2023-07-18T03:06:52.881350+0000","last_peered":"2023-07-18T03:06:52.881350+0000","last_clean":"2023-07-18T03:06:52.881350+0000","last_became_active":"2023-07-18T03:04:59.800798+0000","last_became_peered":"2023-07-18T03:04:59.800798+0000","last_unstale":"2023-07-18T03:06:52.881350+0000","last_undegraded":"2023-07-18T03:06:52.881350+0000","last_fullsized":"2023-07-18T03:06:52.881350+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:49:57.147042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255477+0000","last_change":"2023-07-18T03:04:58.846047+0000","last_active":"2023-07-18T03:06:24.255477+0000","last_peered":"2023-07-18T03:06:24.255477+0000","last_clean":"2023-07-18T03:06:24.255477+0000","last_became_active":"2023-07-18T03:04:58.845678+0000","last_became_peered":"2023-07-18T03:04:58.845678+0000","last_unstale":"2023-07-18T03:06:24.255477+0000","last_undegraded":"2023-07-18T03:06:24.255477+0000","last_fullsized":"2023-07-18T03:06:24.255477+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:50:39.012582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":385,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.257405+0000","last_change":"2023-07-18T03:03:54.168837+0000","last_active":"2023-07-18T03:06:24.257405+0000","last_peered":"2023-07-18T03:06:24.257405+0000","last_clean":"2023-07-18T03:06:24.257405+0000","last_became_active":"2023-07-18T03:03:54.168298+0000","last_became_peered":"2023-07-18T03:03:54.168298+0000","last_unstale":"2023-07-18T03:06:24.257405+0000","last_undegraded":"2023-07-18T03:06:24.257405+0000","last_fullsized":"2023-07-18T03:06:24.257405+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:00:43.469168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:00:43.469168+0000","last_clean_scrub_stamp":"2023-07-18T03:00:43.469168+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:03:45.014912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.261432+0000","last_change":"2023-07-18T03:05:03.886611+0000","last_active":"2023-07-18T03:06:24.261432+0000","last_peered":"2023-07-18T03:06:24.261432+0000","last_clean":"2023-07-18T03:06:24.261432+0000","last_became_active":"2023-07-18T03:05:03.886099+0000","last_became_peered":"2023-07-18T03:05:03.886099+0000","last_unstale":"2023-07-18T03:06:24.261432+0000","last_undegraded":"2023-07-18T03:06:24.261432+0000","last_fullsized":"2023-07-18T03:06:24.261432+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:26:40.808017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.893733+0000","last_change":"2023-07-18T03:04:59.804478+0000","last_active":"2023-07-18T03:06:52.893733+0000","last_peered":"2023-07-18T03:06:52.893733+0000","last_clean":"2023-07-18T03:06:52.893733+0000","last_became_active":"2023-07-18T03:04:59.804162+0000","last_became_peered":"2023-07-18T03:04:59.804162+0000","last_unstale":"2023-07-18T03:06:52.893733+0000","last_undegraded":"2023-07-18T03:06:52.893733+0000","last_fullsized":"2023-07-18T03:06:52.893733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:41:57.294071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.267166+0000","last_change":"2023-07-18T03:06:24.266708+0000","last_active":"2023-07-18T03:06:24.267166+0000","last_peered":"2023-07-18T03:06:24.267166+0000","last_clean":"2023-07-18T03:06:24.267166+0000","last_became_active":"2023-07-18T03:06:24.262606+0000","last_became_peered":"2023-07-18T03:06:24.262606+0000","last_unstale":"2023-07-18T03:06:24.267166+0000","last_undegraded":"2023-07-18T03:06:24.267166+0000","last_fullsized":"2023-07-18T03:06:24.267166+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:20:59.875839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.257365+0000","last_change":"2023-07-18T03:05:03.872788+0000","last_active":"2023-07-18T03:06:24.257365+0000","last_peered":"2023-07-18T03:06:24.257365+0000","last_clean":"2023-07-18T03:06:24.257365+0000","last_became_active":"2023-07-18T03:05:03.872343+0000","last_became_peered":"2023-07-18T03:05:03.872343+0000","last_unstale":"2023-07-18T03:06:24.257365+0000","last_undegraded":"2023-07-18T03:06:24.257365+0000","last_fullsized":"2023-07-18T03:06:24.257365+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:20:43.912026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.266064+0000","last_change":"2023-07-18T03:06:24.265727+0000","last_active":"2023-07-18T03:06:24.266064+0000","last_peered":"2023-07-18T03:06:24.266064+0000","last_clean":"2023-07-18T03:06:24.266064+0000","last_became_active":"2023-07-18T03:06:24.262373+0000","last_became_peered":"2023-07-18T03:06:24.262373+0000","last_unstale":"2023-07-18T03:06:24.266064+0000","last_undegraded":"2023-07-18T03:06:24.266064+0000","last_fullsized":"2023-07-18T03:06:24.266064+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:38:45.610472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.893085+0000","last_change":"2023-07-18T03:04:59.804706+0000","last_active":"2023-07-18T03:06:52.893085+0000","last_peered":"2023-07-18T03:06:52.893085+0000","last_clean":"2023-07-18T03:06:52.893085+0000","last_became_active":"2023-07-18T03:04:59.804445+0000","last_became_peered":"2023-07-18T03:04:59.804445+0000","last_unstale":"2023-07-18T03:06:52.893085+0000","last_undegraded":"2023-07-18T03:06:52.893085+0000","last_fullsized":"2023-07-18T03:06:52.893085+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:54:52.824231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.374912+0000","last_change":"2023-07-18T03:05:03.886065+0000","last_active":"2023-07-18T03:06:24.374912+0000","last_peered":"2023-07-18T03:06:24.374912+0000","last_clean":"2023-07-18T03:06:24.374912+0000","last_became_active":"2023-07-18T03:05:03.885625+0000","last_became_peered":"2023-07-18T03:05:03.885625+0000","last_unstale":"2023-07-18T03:06:24.374912+0000","last_undegraded":"2023-07-18T03:06:24.374912+0000","last_fullsized":"2023-07-18T03:06:24.374912+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:32:31.156943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:07:10.696055+0000","last_change":"2023-07-18T03:04:58.849246+0000","last_active":"2023-07-18T03:07:10.696055+0000","last_peered":"2023-07-18T03:07:10.696055+0000","last_clean":"2023-07-18T03:07:10.696055+0000","last_became_active":"2023-07-18T03:04:58.848888+0000","last_became_peered":"2023-07-18T03:04:58.848888+0000","last_unstale":"2023-07-18T03:07:10.696055+0000","last_undegraded":"2023-07-18T03:07:10.696055+0000","last_fullsized":"2023-07-18T03:07:10.696055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:47:14.002970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.883435+0000","last_change":"2023-07-18T03:04:59.804561+0000","last_active":"2023-07-18T03:06:52.883435+0000","last_peered":"2023-07-18T03:06:52.883435+0000","last_clean":"2023-07-18T03:06:52.883435+0000","last_became_active":"2023-07-18T03:04:59.803659+0000","last_became_peered":"2023-07-18T03:04:59.803659+0000","last_unstale":"2023-07-18T03:06:52.883435+0000","last_undegraded":"2023-07-18T03:06:52.883435+0000","last_fullsized":"2023-07-18T03:06:52.883435+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:55:58.968273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255507+0000","last_change":"2023-07-18T03:05:03.871418+0000","last_active":"2023-07-18T03:06:24.255507+0000","last_peered":"2023-07-18T03:06:24.255507+0000","last_clean":"2023-07-18T03:06:24.255507+0000","last_became_active":"2023-07-18T03:05:03.871054+0000","last_became_peered":"2023-07-18T03:05:03.871054+0000","last_unstale":"2023-07-18T03:06:24.255507+0000","last_undegraded":"2023-07-18T03:06:24.255507+0000","last_fullsized":"2023-07-18T03:06:24.255507+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:59:35.196165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255933+0000","last_change":"2023-07-18T03:04:58.850505+0000","last_active":"2023-07-18T03:06:24.255933+0000","last_peered":"2023-07-18T03:06:24.255933+0000","last_clean":"2023-07-18T03:06:24.255933+0000","last_became_active":"2023-07-18T03:04:58.850234+0000","last_became_peered":"2023-07-18T03:04:58.850234+0000","last_unstale":"2023-07-18T03:06:24.255933+0000","last_undegraded":"2023-07-18T03:06:24.255933+0000","last_fullsized":"2023-07-18T03:06:24.255933+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:36:14.805265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.886643+0000","last_change":"2023-07-18T03:04:59.807348+0000","last_active":"2023-07-18T03:06:52.886643+0000","last_peered":"2023-07-18T03:06:52.886643+0000","last_clean":"2023-07-18T03:06:52.886643+0000","last_became_active":"2023-07-18T03:04:59.807013+0000","last_became_peered":"2023-07-18T03:04:59.807013+0000","last_unstale":"2023-07-18T03:06:52.886643+0000","last_undegraded":"2023-07-18T03:06:52.886643+0000","last_fullsized":"2023-07-18T03:06:52.886643+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:11:05.544642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255623+0000","last_change":"2023-07-18T03:05:03.886773+0000","last_active":"2023-07-18T03:06:24.255623+0000","last_peered":"2023-07-18T03:06:24.255623+0000","last_clean":"2023-07-18T03:06:24.255623+0000","last_became_active":"2023-07-18T03:05:03.886259+0000","last_became_peered":"2023-07-18T03:05:03.886259+0000","last_unstale":"2023-07-18T03:06:24.255623+0000","last_undegraded":"2023-07-18T03:06:24.255623+0000","last_fullsized":"2023-07-18T03:06:24.255623+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:12:05.068449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.261480+0000","last_change":"2023-07-18T03:04:58.855746+0000","last_active":"2023-07-18T03:06:24.261480+0000","last_peered":"2023-07-18T03:06:24.261480+0000","last_clean":"2023-07-18T03:06:24.261480+0000","last_became_active":"2023-07-18T03:04:58.855366+0000","last_became_peered":"2023-07-18T03:04:58.855366+0000","last_unstale":"2023-07-18T03:06:24.261480+0000","last_undegraded":"2023-07-18T03:06:24.261480+0000","last_fullsized":"2023-07-18T03:06:24.261480+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:04:33.740458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.879818+0000","last_change":"2023-07-18T03:04:59.800321+0000","last_active":"2023-07-18T03:06:52.879818+0000","last_peered":"2023-07-18T03:06:52.879818+0000","last_clean":"2023-07-18T03:06:52.879818+0000","last_became_active":"2023-07-18T03:04:59.800018+0000","last_became_peered":"2023-07-18T03:04:59.800018+0000","last_unstale":"2023-07-18T03:06:52.879818+0000","last_undegraded":"2023-07-18T03:06:52.879818+0000","last_fullsized":"2023-07-18T03:06:52.879818+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:13:00.845492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255150+0000","last_change":"2023-07-18T03:05:03.877541+0000","last_active":"2023-07-18T03:06:24.255150+0000","last_peered":"2023-07-18T03:06:24.255150+0000","last_clean":"2023-07-18T03:06:24.255150+0000","last_became_active":"2023-07-18T03:05:03.877164+0000","last_became_peered":"2023-07-18T03:05:03.877164+0000","last_unstale":"2023-07-18T03:06:24.255150+0000","last_undegraded":"2023-07-18T03:06:24.255150+0000","last_fullsized":"2023-07-18T03:06:24.255150+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:58:38.384423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.375090+0000","last_change":"2023-07-18T03:04:58.845643+0000","last_active":"2023-07-18T03:06:24.375090+0000","last_peered":"2023-07-18T03:06:24.375090+0000","last_clean":"2023-07-18T03:06:24.375090+0000","last_became_active":"2023-07-18T03:04:58.845254+0000","last_became_peered":"2023-07-18T03:04:58.845254+0000","last_unstale":"2023-07-18T03:06:24.375090+0000","last_undegraded":"2023-07-18T03:06:24.375090+0000","last_fullsized":"2023-07-18T03:06:24.375090+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:03:15.745367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.887203+0000","last_change":"2023-07-18T03:04:59.803679+0000","last_active":"2023-07-18T03:06:52.887203+0000","last_peered":"2023-07-18T03:06:52.887203+0000","last_clean":"2023-07-18T03:06:52.887203+0000","last_became_active":"2023-07-18T03:04:59.802841+0000","last_became_peered":"2023-07-18T03:04:59.802841+0000","last_unstale":"2023-07-18T03:06:52.887203+0000","last_undegraded":"2023-07-18T03:06:52.887203+0000","last_fullsized":"2023-07-18T03:06:52.887203+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:51:12.763232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255483+0000","last_change":"2023-07-18T03:05:03.874093+0000","last_active":"2023-07-18T03:06:24.255483+0000","last_peered":"2023-07-18T03:06:24.255483+0000","last_clean":"2023-07-18T03:06:24.255483+0000","last_became_active":"2023-07-18T03:05:03.873710+0000","last_became_peered":"2023-07-18T03:05:03.873710+0000","last_unstale":"2023-07-18T03:06:24.255483+0000","last_undegraded":"2023-07-18T03:06:24.255483+0000","last_fullsized":"2023-07-18T03:06:24.255483+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:42:17.007476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255671+0000","last_change":"2023-07-18T03:04:58.846686+0000","last_active":"2023-07-18T03:06:24.255671+0000","last_peered":"2023-07-18T03:06:24.255671+0000","last_clean":"2023-07-18T03:06:24.255671+0000","last_became_active":"2023-07-18T03:04:58.846371+0000","last_became_peered":"2023-07-18T03:04:58.846371+0000","last_unstale":"2023-07-18T03:06:24.255671+0000","last_undegraded":"2023-07-18T03:06:24.255671+0000","last_fullsized":"2023-07-18T03:06:24.255671+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:25:47.154236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.886475+0000","last_change":"2023-07-18T03:04:59.811917+0000","last_active":"2023-07-18T03:06:52.886475+0000","last_peered":"2023-07-18T03:06:52.886475+0000","last_clean":"2023-07-18T03:06:52.886475+0000","last_became_active":"2023-07-18T03:04:59.811551+0000","last_became_peered":"2023-07-18T03:04:59.811551+0000","last_unstale":"2023-07-18T03:06:52.886475+0000","last_undegraded":"2023-07-18T03:06:52.886475+0000","last_fullsized":"2023-07-18T03:06:52.886475+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:28:00.926747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.255450+0000","last_change":"2023-07-18T03:05:03.886072+0000","last_active":"2023-07-18T03:06:24.255450+0000","last_peered":"2023-07-18T03:06:24.255450+0000","last_clean":"2023-07-18T03:06:24.255450+0000","last_became_active":"2023-07-18T03:05:03.885652+0000","last_became_peered":"2023-07-18T03:05:03.885652+0000","last_unstale":"2023-07-18T03:06:24.255450+0000","last_undegraded":"2023-07-18T03:06:24.255450+0000","last_fullsized":"2023-07-18T03:06:24.255450+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:37:43.631888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.259677+0000","last_change":"2023-07-18T03:04:58.849069+0000","last_active":"2023-07-18T03:06:24.259677+0000","last_peered":"2023-07-18T03:06:24.259677+0000","last_clean":"2023-07-18T03:06:24.259677+0000","last_became_active":"2023-07-18T03:04:58.848744+0000","last_became_peered":"2023-07-18T03:04:58.848744+0000","last_unstale":"2023-07-18T03:06:24.259677+0000","last_undegraded":"2023-07-18T03:06:24.259677+0000","last_fullsized":"2023-07-18T03:06:24.259677+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:19:13.922970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.895462+0000","last_change":"2023-07-18T03:04:59.806976+0000","last_active":"2023-07-18T03:06:52.895462+0000","last_peered":"2023-07-18T03:06:52.895462+0000","last_clean":"2023-07-18T03:06:52.895462+0000","last_became_active":"2023-07-18T03:04:59.805750+0000","last_became_peered":"2023-07-18T03:04:59.805750+0000","last_unstale":"2023-07-18T03:06:52.895462+0000","last_undegraded":"2023-07-18T03:06:52.895462+0000","last_fullsized":"2023-07-18T03:06:52.895462+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:59:17.717720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.261590+0000","last_change":"2023-07-18T03:05:03.871210+0000","last_active":"2023-07-18T03:06:24.261590+0000","last_peered":"2023-07-18T03:06:24.261590+0000","last_clean":"2023-07-18T03:06:24.261590+0000","last_became_active":"2023-07-18T03:05:03.870826+0000","last_became_peered":"2023-07-18T03:05:03.870826+0000","last_unstale":"2023-07-18T03:06:24.261590+0000","last_undegraded":"2023-07-18T03:06:24.261590+0000","last_fullsized":"2023-07-18T03:06:24.261590+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:49:03.743543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.256155+0000","last_change":"2023-07-18T03:04:58.855673+0000","last_active":"2023-07-18T03:06:24.256155+0000","last_peered":"2023-07-18T03:06:24.256155+0000","last_clean":"2023-07-18T03:06:24.256155+0000","last_became_active":"2023-07-18T03:04:58.855338+0000","last_became_peered":"2023-07-18T03:04:58.855338+0000","last_unstale":"2023-07-18T03:06:24.256155+0000","last_undegraded":"2023-07-18T03:06:24.256155+0000","last_fullsized":"2023-07-18T03:06:24.256155+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:07:40.935595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.884252+0000","last_change":"2023-07-18T03:04:59.802488+0000","last_active":"2023-07-18T03:06:52.884252+0000","last_peered":"2023-07-18T03:06:52.884252+0000","last_clean":"2023-07-18T03:06:52.884252+0000","last_became_active":"2023-07-18T03:04:59.802184+0000","last_became_peered":"2023-07-18T03:04:59.802184+0000","last_unstale":"2023-07-18T03:06:52.884252+0000","last_undegraded":"2023-07-18T03:06:52.884252+0000","last_fullsized":"2023-07-18T03:06:52.884252+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:00:59.367787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.259218+0000","last_change":"2023-07-18T03:05:03.881585+0000","last_active":"2023-07-18T03:06:24.259218+0000","last_peered":"2023-07-18T03:06:24.259218+0000","last_clean":"2023-07-18T03:06:24.259218+0000","last_became_active":"2023-07-18T03:05:03.880917+0000","last_became_peered":"2023-07-18T03:05:03.880917+0000","last_unstale":"2023-07-18T03:06:24.259218+0000","last_undegraded":"2023-07-18T03:06:24.259218+0000","last_fullsized":"2023-07-18T03:06:24.259218+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:44:11.175469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.262020+0000","last_change":"2023-07-18T03:04:58.852387+0000","last_active":"2023-07-18T03:06:24.262020+0000","last_peered":"2023-07-18T03:06:24.262020+0000","last_clean":"2023-07-18T03:06:24.262020+0000","last_became_active":"2023-07-18T03:04:58.852113+0000","last_became_peered":"2023-07-18T03:04:58.852113+0000","last_unstale":"2023-07-18T03:06:24.262020+0000","last_undegraded":"2023-07-18T03:06:24.262020+0000","last_fullsized":"2023-07-18T03:06:24.262020+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:12:15.450770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.882718+0000","last_change":"2023-07-18T03:04:59.811569+0000","last_active":"2023-07-18T03:06:52.882718+0000","last_peered":"2023-07-18T03:06:52.882718+0000","last_clean":"2023-07-18T03:06:52.882718+0000","last_became_active":"2023-07-18T03:04:59.811229+0000","last_became_peered":"2023-07-18T03:04:59.811229+0000","last_unstale":"2023-07-18T03:06:52.882718+0000","last_undegraded":"2023-07-18T03:06:52.882718+0000","last_fullsized":"2023-07-18T03:06:52.882718+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:20:39.366577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.260963+0000","last_change":"2023-07-18T03:05:03.872530+0000","last_active":"2023-07-18T03:06:24.260963+0000","last_peered":"2023-07-18T03:06:24.260963+0000","last_clean":"2023-07-18T03:06:24.260963+0000","last_became_active":"2023-07-18T03:05:03.871607+0000","last_became_peered":"2023-07-18T03:05:03.871607+0000","last_unstale":"2023-07-18T03:06:24.260963+0000","last_undegraded":"2023-07-18T03:06:24.260963+0000","last_fullsized":"2023-07-18T03:06:24.260963+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:05:02.845904+0000","last_clean_scrub_stamp":"2023-07-18T03:05:02.845904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:13:21.796882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:24.751209+0000","last_change":"2023-07-18T03:04:58.847933+0000","last_active":"2023-07-18T03:06:24.751209+0000","last_peered":"2023-07-18T03:06:24.751209+0000","last_clean":"2023-07-18T03:06:24.751209+0000","last_became_active":"2023-07-18T03:04:58.847630+0000","last_became_peered":"2023-07-18T03:04:58.847630+0000","last_unstale":"2023-07-18T03:06:24.751209+0000","last_undegraded":"2023-07-18T03:06:24.751209+0000","last_fullsized":"2023-07-18T03:06:24.751209+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:57.821748+0000","last_clean_scrub_stamp":"2023-07-18T03:04:57.821748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:34:44.655423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.886617+0000","last_change":"2023-07-18T03:04:59.803981+0000","last_active":"2023-07-18T03:06:52.886617+0000","last_peered":"2023-07-18T03:06:52.886617+0000","last_clean":"2023-07-18T03:06:52.886617+0000","last_became_active":"2023-07-18T03:04:59.803692+0000","last_became_peered":"2023-07-18T03:04:59.803692+0000","last_unstale":"2023-07-18T03:06:52.886617+0000","last_undegraded":"2023-07-18T03:06:52.886617+0000","last_fullsized":"2023-07-18T03:06:52.886617+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:58:45.567557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.884681+0000","last_change":"2023-07-18T03:04:59.803770+0000","last_active":"2023-07-18T03:06:52.884681+0000","last_peered":"2023-07-18T03:06:52.884681+0000","last_clean":"2023-07-18T03:06:52.884681+0000","last_became_active":"2023-07-18T03:04:59.803355+0000","last_became_peered":"2023-07-18T03:04:59.803355+0000","last_unstale":"2023-07-18T03:06:52.884681+0000","last_undegraded":"2023-07-18T03:06:52.884681+0000","last_fullsized":"2023-07-18T03:06:52.884681+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:07:11.955948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.886539+0000","last_change":"2023-07-18T03:04:59.808946+0000","last_active":"2023-07-18T03:06:52.886539+0000","last_peered":"2023-07-18T03:06:52.886539+0000","last_clean":"2023-07-18T03:06:52.886539+0000","last_became_active":"2023-07-18T03:04:59.808646+0000","last_became_peered":"2023-07-18T03:04:59.808646+0000","last_unstale":"2023-07-18T03:06:52.886539+0000","last_undegraded":"2023-07-18T03:06:52.886539+0000","last_fullsized":"2023-07-18T03:06:52.886539+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:10:55.288876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.883434+0000","last_change":"2023-07-18T03:04:59.811933+0000","last_active":"2023-07-18T03:06:52.883434+0000","last_peered":"2023-07-18T03:06:52.883434+0000","last_clean":"2023-07-18T03:06:52.883434+0000","last_became_active":"2023-07-18T03:04:59.811620+0000","last_became_peered":"2023-07-18T03:04:59.811620+0000","last_unstale":"2023-07-18T03:06:52.883434+0000","last_undegraded":"2023-07-18T03:06:52.883434+0000","last_fullsized":"2023-07-18T03:06:52.883434+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:21:21.164245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.896076+0000","last_change":"2023-07-18T03:04:59.803942+0000","last_active":"2023-07-18T03:06:52.896076+0000","last_peered":"2023-07-18T03:06:52.896076+0000","last_clean":"2023-07-18T03:06:52.896076+0000","last_became_active":"2023-07-18T03:04:59.803576+0000","last_became_peered":"2023-07-18T03:04:59.803576+0000","last_unstale":"2023-07-18T03:06:52.896076+0000","last_undegraded":"2023-07-18T03:06:52.896076+0000","last_fullsized":"2023-07-18T03:06:52.896076+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:10:05.182871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.893685+0000","last_change":"2023-07-18T03:04:59.802594+0000","last_active":"2023-07-18T03:06:52.893685+0000","last_peered":"2023-07-18T03:06:52.893685+0000","last_clean":"2023-07-18T03:06:52.893685+0000","last_became_active":"2023-07-18T03:04:59.802215+0000","last_became_peered":"2023-07-18T03:04:59.802215+0000","last_unstale":"2023-07-18T03:06:52.893685+0000","last_undegraded":"2023-07-18T03:06:52.893685+0000","last_fullsized":"2023-07-18T03:06:52.893685+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:57:57.598480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.883798+0000","last_change":"2023-07-18T03:04:59.803519+0000","last_active":"2023-07-18T03:06:52.883798+0000","last_peered":"2023-07-18T03:06:52.883798+0000","last_clean":"2023-07-18T03:06:52.883798+0000","last_became_active":"2023-07-18T03:04:59.803170+0000","last_became_peered":"2023-07-18T03:04:59.803170+0000","last_unstale":"2023-07-18T03:06:52.883798+0000","last_undegraded":"2023-07-18T03:06:52.883798+0000","last_fullsized":"2023-07-18T03:06:52.883798+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:40:11.785249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.881716+0000","last_change":"2023-07-18T03:04:59.813494+0000","last_active":"2023-07-18T03:06:52.881716+0000","last_peered":"2023-07-18T03:06:52.881716+0000","last_clean":"2023-07-18T03:06:52.881716+0000","last_became_active":"2023-07-18T03:04:59.813175+0000","last_became_peered":"2023-07-18T03:04:59.813175+0000","last_unstale":"2023-07-18T03:06:52.881716+0000","last_undegraded":"2023-07-18T03:06:52.881716+0000","last_fullsized":"2023-07-18T03:06:52.881716+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:58:52.954498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.884923+0000","last_change":"2023-07-18T03:04:59.806597+0000","last_active":"2023-07-18T03:06:52.884923+0000","last_peered":"2023-07-18T03:06:52.884923+0000","last_clean":"2023-07-18T03:06:52.884923+0000","last_became_active":"2023-07-18T03:04:59.806220+0000","last_became_peered":"2023-07-18T03:04:59.806220+0000","last_unstale":"2023-07-18T03:06:52.884923+0000","last_undegraded":"2023-07-18T03:06:52.884923+0000","last_fullsized":"2023-07-18T03:06:52.884923+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:07:15.022728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.888455+0000","last_change":"2023-07-18T03:04:59.809051+0000","last_active":"2023-07-18T03:06:52.888455+0000","last_peered":"2023-07-18T03:06:52.888455+0000","last_clean":"2023-07-18T03:06:52.888455+0000","last_became_active":"2023-07-18T03:04:59.808783+0000","last_became_peered":"2023-07-18T03:04:59.808783+0000","last_unstale":"2023-07-18T03:06:52.888455+0000","last_undegraded":"2023-07-18T03:06:52.888455+0000","last_fullsized":"2023-07-18T03:06:52.888455+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:55:28.682728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.883966+0000","last_change":"2023-07-18T03:04:59.805741+0000","last_active":"2023-07-18T03:06:52.883966+0000","last_peered":"2023-07-18T03:06:52.883966+0000","last_clean":"2023-07-18T03:06:52.883966+0000","last_became_active":"2023-07-18T03:04:59.805412+0000","last_became_peered":"2023-07-18T03:04:59.805412+0000","last_unstale":"2023-07-18T03:06:52.883966+0000","last_undegraded":"2023-07-18T03:06:52.883966+0000","last_fullsized":"2023-07-18T03:06:52.883966+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:56:58.924751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.891407+0000","last_change":"2023-07-18T03:04:59.812052+0000","last_active":"2023-07-18T03:06:52.891407+0000","last_peered":"2023-07-18T03:06:52.891407+0000","last_clean":"2023-07-18T03:06:52.891407+0000","last_became_active":"2023-07-18T03:04:59.811756+0000","last_became_peered":"2023-07-18T03:04:59.811756+0000","last_unstale":"2023-07-18T03:06:52.891407+0000","last_undegraded":"2023-07-18T03:06:52.891407+0000","last_fullsized":"2023-07-18T03:06:52.891407+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:41:23.926360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.896400+0000","last_change":"2023-07-18T03:04:59.803009+0000","last_active":"2023-07-18T03:06:52.896400+0000","last_peered":"2023-07-18T03:06:52.896400+0000","last_clean":"2023-07-18T03:06:52.896400+0000","last_became_active":"2023-07-18T03:04:59.802687+0000","last_became_peered":"2023-07-18T03:04:59.802687+0000","last_unstale":"2023-07-18T03:06:52.896400+0000","last_undegraded":"2023-07-18T03:06:52.896400+0000","last_fullsized":"2023-07-18T03:06:52.896400+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:31:26.686903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-18T03:06:52.880496+0000","last_change":"2023-07-18T03:04:59.795562+0000","last_active":"2023-07-18T03:06:52.880496+0000","last_peered":"2023-07-18T03:06:52.880496+0000","last_clean":"2023-07-18T03:06:52.880496+0000","last_became_active":"2023-07-18T03:04:59.795241+0000","last_became_peered":"2023-07-18T03:04:59.795241+0000","last_unstale":"2023-07-18T03:06:52.880496+0000","last_undegraded":"2023-07-18T03:06:52.880496+0000","last_fullsized":"2023-07-18T03:06:52.880496+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:04:58.823479+0000","last_clean_scrub_stamp":"2023-07-18T03:04:58.823479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:44:43.907380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281146,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70826,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":655,"num_read_kb":404,"num_write":166,"num_write_kb":124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":212478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449128,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":999424,"data_stored":507139,"data_compressed":23524,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.56000000000000005,"15min":0.55300000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.52200000000000002,"15min":0.51400000000000001},"min":{"1min":0.41099999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.69199999999999995}]},{"osd":1,"last update":"Tue Jul 18 03:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.64300000000000002,"15min":0.63500000000000001},"min":{"1min":0.47799999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.85299999999999998,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.624,"15min":0.621},"min":{"1min":0.47099999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.82299999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.76400000000000001}]},{"osd":2,"last update":"Tue Jul 18 03:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.48299999999999998,"15min":0.47399999999999998},"min":{"1min":0.35199999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.71399999999999997,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.48999999999999999,"15min":0.47599999999999998},"min":{"1min":0.36699999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.32500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":4,"last update":"Tue Jul 18 03:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.66500000000000004,"15min":0.65100000000000002},"min":{"1min":0.53200000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.64200000000000002,"15min":0.63500000000000001},"min":{"1min":0.48999999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.84799999999999998,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.84399999999999997}]},{"osd":5,"last update":"Tue Jul 18 03:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.622,"15min":0.621},"min":{"1min":0.45400000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.82099999999999995,"5min":1.375,"15min":1.375},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.61399999999999999,"15min":0.59299999999999997},"min":{"1min":0.378,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.40400000000000003}]},{"osd":6,"last update":"Tue Jul 18 03:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.68400000000000005,"15min":0.67000000000000004},"min":{"1min":0.52200000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.629},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.70499999999999996,"15min":0.69299999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.64900000000000002}]},{"osd":7,"last update":"Tue Jul 18 03:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.79200000000000004,"15min":0.78000000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1319999999999999,"5min":1.248,"15min":1.248},"last":0.745},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.76900000000000002,"15min":0.76300000000000001},"min":{"1min":0.56499999999999995,"5min":0.502,"15min":0.502},"max":{"1min":1.024,"5min":1.288,"15min":1.288},"last":0.55500000000000005}]},{"osd":8,"last update":"Tue Jul 18 03:07:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.76500000000000001,"15min":0.77000000000000002},"min":{"1min":0.50800000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.83899999999999997,"15min":0.84199999999999997},"min":{"1min":0.58999999999999997,"5min":0.498,"15min":0.498},"max":{"1min":0.98599999999999999,"5min":1.091,"15min":1.091},"last":0.496}]},{"osd":9,"last update":"Tue Jul 18 03:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.78300000000000003,"15min":0.76800000000000002},"min":{"1min":0.66700000000000004,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.145,"5min":1.304,"15min":1.304},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.81899999999999995,"15min":0.80400000000000005},"min":{"1min":0.63400000000000001,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1659999999999999,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.71199999999999997}]},{"osd":10,"last update":"Tue Jul 18 03:06:14 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.78200000000000003,"15min":0.77200000000000002},"min":{"1min":0.57999999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.054,"5min":1.163,"15min":1.163},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.79900000000000004,"15min":0.78900000000000003},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.026,"5min":1.496,"15min":1.496},"last":0.78400000000000003}]}]},{"osd":10,"up_from":71,"seq":304942678061,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1120,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1146880,"data_stored":1161118,"data_compressed":59283,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:07:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.44600000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.71099999999999997,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.66300000000000003,"15min":0.66900000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.70799999999999996}]},{"osd":1,"last update":"Tue Jul 18 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.47699999999999998,"15min":0.48099999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.61499999999999999,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.505,"15min":0.51400000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70999999999999996,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.58999999999999997}]},{"osd":2,"last update":"Tue Jul 18 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.64700000000000002,"15min":0.63400000000000001},"min":{"1min":0.437,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.64700000000000002,"15min":0.64300000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.78900000000000003,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.72699999999999998}]},{"osd":3,"last update":"Tue Jul 18 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.67200000000000004,"15min":0.68500000000000005},"min":{"1min":0.48199999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80000000000000004,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65500000000000003,"15min":0.66300000000000003},"min":{"1min":0.498,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.66100000000000003}]},{"osd":4,"last update":"Tue Jul 18 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.59799999999999998,"15min":0.61399999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.58399999999999996,"15min":0.57999999999999996},"min":{"1min":0.375,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.82999999999999996,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.80900000000000005}]},{"osd":5,"last update":"Tue Jul 18 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71299999999999997,"15min":0.72099999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.753,"5min":0.751,"15min":0.751},"min":{"1min":0.49299999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.90000000000000002}]},{"osd":6,"last update":"Tue Jul 18 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.745,"15min":0.75800000000000001},"min":{"1min":0.50700000000000001,"5min":0.501,"15min":0.501},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.79300000000000004,"15min":0.80100000000000005},"min":{"1min":0.61599999999999999,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.96899999999999997,"5min":1.077,"15min":1.077},"last":0.94099999999999995}]},{"osd":7,"last update":"Tue Jul 18 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.55800000000000005}]},{"osd":8,"last update":"Tue Jul 18 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.79200000000000004,"15min":0.80500000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.91000000000000003,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.81799999999999995,"15min":0.82499999999999996},"min":{"1min":0.60299999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.0169999999999999,"5min":1.032,"15min":1.032},"last":0.95899999999999996}]},{"osd":9,"last update":"Tue Jul 18 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.83799999999999997,"15min":0.84399999999999997},"min":{"1min":0.66500000000000004,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.84699999999999998,"15min":0.85499999999999998},"min":{"1min":0.63300000000000001,"5min":0.626,"15min":0.626},"max":{"1min":0.996,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.98799999999999999}]},{"osd":11,"last update":"Tue Jul 18 03:06:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.82699999999999996,"15min":0.83199999999999996},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.96599999999999997,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.80000000000000004,"15min":0.80900000000000005},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.92800000000000005,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.83099999999999996}]}]},{"osd":9,"up_from":66,"seq":283467841584,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":1015808,"data_stored":511449,"data_compressed":23524,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.52000000000000002,"15min":0.54800000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.629,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.55000000000000004,"15min":0.57199999999999995},"min":{"1min":0.32600000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.79700000000000004,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.39800000000000002}]},{"osd":1,"last update":"Tue Jul 18 03:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.68999999999999995,"15min":0.72099999999999997},"min":{"1min":0.41499999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.83599999999999997,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.69699999999999995,"15min":0.73099999999999998},"min":{"1min":0.46200000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.81699999999999995,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.58099999999999996}]},{"osd":2,"last update":"Tue Jul 18 03:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.68899999999999995,"15min":0.71099999999999997},"min":{"1min":0.45800000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.78100000000000003,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.71899999999999997,"15min":0.74199999999999999},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.81299999999999994,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.77600000000000002}]},{"osd":3,"last update":"Tue Jul 18 03:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.63500000000000001,"15min":0.67600000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.76500000000000001,"5min":1.038,"15min":1.038},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.61199999999999999,"15min":0.65000000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.67600000000000005,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.46200000000000002}]},{"osd":4,"last update":"Tue Jul 18 03:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.77000000000000002,"15min":0.81699999999999995},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.85999999999999999,"5min":1.47,"15min":1.47},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.77900000000000003,"15min":0.81799999999999995},"min":{"1min":0.47599999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.90700000000000003,"5min":1.619,"15min":1.619},"last":0.68200000000000005}]},{"osd":5,"last update":"Tue Jul 18 03:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.79800000000000004,"15min":0.82199999999999995},"min":{"1min":0.53500000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.89300000000000002,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.78000000000000003,"15min":0.80800000000000005},"min":{"1min":0.53300000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.90100000000000002,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.82199999999999995}]},{"osd":6,"last update":"Tue Jul 18 03:04:30 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.64300000000000002}]},{"osd":7,"last update":"Tue Jul 18 03:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.84799999999999998,"15min":0.89600000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.86699999999999999,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.89500000000000002,"15min":0.92300000000000004},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.81399999999999995,"5min":4.4009999999999998,"15min":4.4009999999999998},"last":0.64600000000000002}]},{"osd":8,"last update":"Tue Jul 18 03:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.876,"15min":0.91600000000000004},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.90900000000000003,"15min":0.94599999999999995},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.012,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.76500000000000001}]},{"osd":10,"last update":"Tue Jul 18 03:06:53 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.86099999999999999,"15min":0.89500000000000002},"min":{"1min":0.60199999999999998,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1020000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.873,"15min":0.92500000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.86099999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.754}]},{"osd":11,"last update":"Tue Jul 18 03:07:09 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.82799999999999996,"15min":0.83099999999999996},"min":{"1min":0.69399999999999995,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":0.96399999999999997,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.79400000000000004,"15min":0.79600000000000004},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":0.93700000000000006,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.79300000000000004}]}]},{"osd":8,"up_from":58,"seq":249108103221,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":1007616,"data_stored":507251,"data_compressed":23524,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.69099999999999995,"15min":0.64000000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.80700000000000005,"5min":7.0359999999999996,"15min":7.0359999999999996},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.58199999999999996,"15min":0.56999999999999995},"min":{"1min":0.42499999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.75700000000000001,"5min":2.883,"15min":2.883},"last":0.55700000000000005}]},{"osd":1,"last update":"Tue Jul 18 03:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.67500000000000004,"15min":0.63900000000000001},"min":{"1min":0.45200000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.95099999999999996,"5min":3.3340000000000001,"15min":3.3340000000000001},"last":0.752},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.67100000000000004,"15min":0.61599999999999999},"min":{"1min":0.55500000000000005,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.99199999999999999,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":0.82899999999999996}]},{"osd":2,"last update":"Tue Jul 18 03:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51400000000000001,"15min":0.47099999999999997},"min":{"1min":0.40100000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70199999999999996,"5min":3.2850000000000001,"15min":3.2850000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.53300000000000003,"15min":0.502},"min":{"1min":0.38400000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.753,"5min":2.012,"15min":2.012},"last":0.63900000000000001}]},{"osd":3,"last update":"Tue Jul 18 03:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64600000000000002,"15min":0.623},"min":{"1min":0.46899999999999997,"5min":0.432,"15min":0.432},"max":{"1min":0.88100000000000001,"5min":2.9100000000000001,"15min":2.9100000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.67000000000000004,"15min":0.65000000000000002},"min":{"1min":0.49399999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.996,"5min":3.6059999999999999,"15min":3.6059999999999999},"last":0.60499999999999998}]},{"osd":4,"last update":"Tue Jul 18 03:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.67600000000000005,"15min":0.63800000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.04,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.73299999999999998,"15min":0.69999999999999996},"min":{"1min":0.51700000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0940000000000001,"5min":3.3650000000000002,"15min":3.3650000000000002},"last":0.84599999999999997}]},{"osd":5,"last update":"Tue Jul 18 03:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.64800000000000002,"15min":0.60399999999999998},"min":{"1min":0.47499999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.0269999999999999,"5min":2.4279999999999999,"15min":2.4279999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.65800000000000003,"15min":0.61399999999999999},"min":{"1min":0.41899999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.135,"5min":2.7410000000000001,"15min":2.7410000000000001},"last":0.71799999999999997}]},{"osd":6,"last update":"Tue Jul 18 03:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76400000000000001,"15min":0.72599999999999998},"min":{"1min":0.46200000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.94399999999999995,"5min":3.5779999999999998,"15min":3.5779999999999998},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.77700000000000002,"15min":0.745},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.97899999999999998,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":0.92300000000000004}]},{"osd":7,"last update":"Tue Jul 18 03:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.78700000000000003,"15min":0.73899999999999999},"min":{"1min":0.67000000000000004,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.081,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.77900000000000003,"15min":0.73399999999999999},"min":{"1min":0.628,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.157,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":0.89200000000000002}]},{"osd":9,"last update":"Tue Jul 18 03:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.82899999999999996,"15min":0.78800000000000003},"min":{"1min":0.627,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1379999999999999,"5min":3.544,"15min":3.544},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.83099999999999996,"15min":0.79700000000000004},"min":{"1min":0.66500000000000004,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.1259999999999999,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":0.97699999999999998}]},{"osd":10,"last update":"Tue Jul 18 03:06:55 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.82799999999999996,"15min":0.81699999999999995},"min":{"1min":0.59299999999999997,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1519999999999999,"5min":2.8570000000000002,"15min":2.8570000000000002},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.879,"15min":0.88100000000000001},"min":{"1min":0.58699999999999997,"5min":0.502,"15min":0.502},"max":{"1min":1.1950000000000001,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":0.90400000000000003}]},{"osd":11,"last update":"Tue Jul 18 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.86899999999999999,"15min":0.86599999999999999},"min":{"1min":0.69399999999999995,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.0720000000000001,"5min":1.107,"15min":1.107},"last":0.878},{"interface":"front","average":{"1min":0.871,"5min":0.85299999999999998,"15min":0.84999999999999998},"min":{"1min":0.65800000000000003,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.81299999999999994}]}]},{"osd":7,"up_from":51,"seq":219043332153,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":1036,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":1060864,"data_stored":1103226,"data_compressed":59283,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.55500000000000005,"15min":0.54300000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.495},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59999999999999998,"15min":0.60299999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82599999999999996,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.44500000000000001}]},{"osd":1,"last update":"Tue Jul 18 03:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.434,"15min":0.42399999999999999},"min":{"1min":0.28699999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.5},{"interface":"front","average":{"1min":0.503,"5min":0.45400000000000001,"15min":0.438},"min":{"1min":0.317,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.74299999999999999,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.55100000000000005}]},{"osd":2,"last update":"Tue Jul 18 03:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.60599999999999998,"15min":0.57699999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.90600000000000003,"5min":1.111,"15min":1.111},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.60699999999999998,"15min":0.58699999999999997},"min":{"1min":0.375,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.89500000000000002,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.63500000000000001}]},{"osd":3,"last update":"Tue Jul 18 03:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.63500000000000001,"15min":0.60599999999999998},"min":{"1min":0.45000000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.83999999999999997,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.63200000000000001,"15min":0.64400000000000002},"min":{"1min":0.41699999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.83699999999999997,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.58499999999999996}]},{"osd":4,"last update":"Tue Jul 18 03:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.5,"15min":0.48799999999999999},"min":{"1min":0.34799999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.55700000000000005,"15min":0.55100000000000005},"min":{"1min":0.36699999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.52500000000000002}]},{"osd":5,"last update":"Tue Jul 18 03:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.68899999999999995,"15min":0.65700000000000003},"min":{"1min":0.51600000000000001,"5min":0.501,"15min":0.501},"max":{"1min":1.0249999999999999,"5min":1.077,"15min":1.077},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.66700000000000004,"15min":0.65600000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.93400000000000005,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.69199999999999995}]},{"osd":6,"last update":"Tue Jul 18 03:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67200000000000004},"min":{"1min":0.51800000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.89400000000000002,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.73299999999999998,"15min":0.70899999999999996},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.86099999999999999}]},{"osd":8,"last update":"Tue Jul 18 03:07:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.72299999999999998,"15min":0.69699999999999995},"min":{"1min":0.53600000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.997,"5min":1.022,"15min":1.022},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.745,"5min":0.68799999999999994,"15min":0.65600000000000003},"min":{"1min":0.55200000000000005,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.94999999999999996,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.82299999999999995}]},{"osd":9,"last update":"Tue Jul 18 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.76000000000000001,"15min":0.73799999999999999},"min":{"1min":0.435,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":3.1880000000000002,"5min":3.1880000000000002,"15min":3.1880000000000002},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.77900000000000003,"15min":0.76800000000000002},"min":{"1min":0.58399999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.88800000000000001}]},{"osd":10,"last update":"Tue Jul 18 03:06:55 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71799999999999997,"15min":0.72299999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.76100000000000001,"15min":0.755},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.91400000000000003}]},{"osd":11,"last update":"Tue Jul 18 03:07:06 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.83599999999999997,"15min":0.82599999999999996},"min":{"1min":0.76100000000000001,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.84899999999999998,"15min":0.83699999999999997},"min":{"1min":0.754,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.93700000000000006}]}]},{"osd":6,"up_from":46,"seq":197568495678,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":991232,"data_stored":502529,"data_compressed":23524,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.52000000000000002,"15min":0.55800000000000005},"min":{"1min":0.307,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.48,"5min":3.1579999999999999,"15min":3.1579999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.53400000000000003,"15min":0.55000000000000004},"min":{"1min":0.36699999999999999,"5min":0.318,"15min":0.318},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":0.38900000000000001}]},{"osd":1,"last update":"Tue Jul 18 03:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.64300000000000002,"15min":0.60899999999999999},"min":{"1min":0.499,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.71699999999999997,"15min":0.73699999999999999},"min":{"1min":0.48299999999999998,"5min":0.44,"15min":0.44},"max":{"1min":1.226,"5min":3.0369999999999999,"15min":3.0369999999999999},"last":0.66700000000000004}]},{"osd":2,"last update":"Tue Jul 18 03:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.67300000000000004,"15min":0.66100000000000003},"min":{"1min":0.47799999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.46300000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.63600000000000001}]},{"osd":3,"last update":"Tue Jul 18 03:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56499999999999995,"15min":0.57299999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.69499999999999995,"15min":0.74199999999999999},"min":{"1min":0.40100000000000002,"5min":0.377,"15min":0.377},"max":{"1min":1.546,"5min":3.226,"15min":3.226},"last":0.45400000000000001}]},{"osd":4,"last update":"Tue Jul 18 03:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69299999999999995,"15min":0.69399999999999995},"min":{"1min":0.47499999999999998,"5min":0.441,"15min":0.441},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.70699999999999996,"15min":0.70099999999999996},"min":{"1min":0.55200000000000005,"5min":0.438,"15min":0.438},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.75900000000000001}]},{"osd":5,"last update":"Tue Jul 18 03:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.78500000000000003,"15min":0.83599999999999997},"min":{"1min":0.54500000000000004,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1060000000000001,"5min":3.125,"15min":3.125},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.76000000000000001,"15min":0.78000000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.4039999999999999,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.86799999999999999}]},{"osd":7,"last update":"Tue Jul 18 03:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.82499999999999996,"15min":0.83699999999999997},"min":{"1min":0.59499999999999997,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1619999999999999,"5min":3.0870000000000002,"15min":3.0870000000000002},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75900000000000001,"15min":0.73599999999999999},"min":{"1min":0.52300000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.95399999999999996}]},{"osd":8,"last update":"Tue Jul 18 03:06:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.85099999999999998,"15min":0.876},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0640000000000001,"5min":3.1909999999999998,"15min":3.1909999999999998},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.99099999999999999,"5min":1.046,"15min":1.046},"last":1.0289999999999999}]},{"osd":9,"last update":"Tue Jul 18 03:06:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.75600000000000001,"15min":0.76500000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.83799999999999997,"15min":0.85199999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.73299999999999998}]},{"osd":10,"last update":"Tue Jul 18 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.88100000000000001,"15min":0.879},"min":{"1min":0.752,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.873,"5min":0.83199999999999996,"15min":0.82199999999999995},"min":{"1min":0.68700000000000006,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":0.92000000000000004}]},{"osd":11,"last update":"Tue Jul 18 03:07:05 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.86499999999999999,"15min":0.86099999999999999},"min":{"1min":0.61799999999999999,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.85299999999999998,"15min":0.85499999999999998},"min":{"1min":0.67000000000000004,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.516,"5min":1.583,"15min":1.583},"last":0.88700000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":1040,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":1064960,"data_stored":569220,"data_compressed":23524,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:06:32 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.55000000000000004,"15min":0.52400000000000002},"min":{"1min":0.39200000000000002,"5min":0.29799999999999999,"15min":0.27800000000000002},"max":{"1min":0.65500000000000003,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55200000000000005,"15min":0.52700000000000002},"min":{"1min":0.376,"5min":0.33100000000000002,"15min":0.29899999999999999},"max":{"1min":0.73399999999999999,"5min":1.421,"15min":1.421},"last":0.54600000000000004}]},{"osd":2,"last update":"Tue Jul 18 03:06:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.59799999999999998,"15min":0.53000000000000003},"min":{"1min":0.39500000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.0009999999999999,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.59799999999999998,"15min":0.55400000000000005},"min":{"1min":0.50900000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.91000000000000003,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.88800000000000001}]},{"osd":3,"last update":"Tue Jul 18 03:06:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.622,"15min":0.59999999999999998},"min":{"1min":0.378,"5min":0.312,"15min":0.312},"max":{"1min":1.014,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.627,"15min":0.627},"min":{"1min":0.47999999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.026,"5min":1.504,"15min":1.504},"last":0.63400000000000001}]},{"osd":4,"last update":"Tue Jul 18 03:06:46 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.48199999999999998,"15min":0.41399999999999998},"min":{"1min":0.379,"5min":0.215,"15min":0.215},"max":{"1min":0.68500000000000005,"5min":1.784,"15min":1.784},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.50600000000000001,"15min":0.435},"min":{"1min":0.32800000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.83599999999999997,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.65500000000000003}]},{"osd":5,"last update":"Tue Jul 18 03:07:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.64400000000000002,"15min":0.54200000000000004},"min":{"1min":0.436,"5min":0.375,"15min":0.375},"max":{"1min":0.84499999999999997,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.65300000000000002,"15min":0.57699999999999996},"min":{"1min":0.441,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.94299999999999995,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.91600000000000004}]},{"osd":6,"last update":"Tue Jul 18 03:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.66400000000000003,"15min":0.60999999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.87,"5min":2.6600000000000001,"15min":2.6600000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.67300000000000004,"15min":0.64500000000000002},"min":{"1min":0.54600000000000004,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.89900000000000002,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.73299999999999998}]},{"osd":7,"last update":"Tue Jul 18 03:06:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.61699999999999999,"15min":0.57099999999999995},"min":{"1min":0.45400000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.67600000000000005,"15min":0.65600000000000003},"min":{"1min":0.42299999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.90100000000000002,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":0.85799999999999998}]},{"osd":8,"last update":"Tue Jul 18 03:07:12 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.77600000000000002,"15min":0.78200000000000003},"min":{"1min":0.47699999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.997,"5min":2.6419999999999999,"15min":2.6419999999999999},"last":0.997},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.748,"15min":0.68999999999999995},"min":{"1min":0.52400000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.03,"5min":1.698,"15min":1.698},"last":0.93200000000000005}]},{"osd":9,"last update":"Tue Jul 18 03:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.83899999999999997,"15min":0.85299999999999998},"min":{"1min":0.58499999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.96299999999999997,"5min":2.6789999999999998,"15min":2.6789999999999998},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.78000000000000003,"15min":0.752},"min":{"1min":0.68100000000000005,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0129999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.875}]},{"osd":10,"last update":"Tue Jul 18 03:06:57 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79000000000000004,"15min":0.77200000000000002},"min":{"1min":0.57399999999999995,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0589999999999999,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.77700000000000002,"15min":0.76500000000000001},"min":{"1min":0.56699999999999995,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.002,"5min":1.359,"15min":1.359},"last":0.96799999999999997}]},{"osd":11,"last update":"Tue Jul 18 03:07:02 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.90600000000000003,"15min":0.90800000000000003},"min":{"1min":0.72299999999999998,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.1060000000000001,"5min":1.512,"15min":1.512},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":1.0249999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":1015808,"data_stored":503917,"data_compressed":23524,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Jul 18 03:06:31 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61499999999999999,"15min":0.55800000000000005},"min":{"1min":0.33700000000000002,"5min":0.33600000000000002,"15min":0.28299999999999997},"max":{"1min":0.88900000000000001,"5min":1.075,"15min":1.075},"last":0.627},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.64000000000000001,"15min":0.56999999999999995},"min":{"1min":0.371,"5min":0.36099999999999999,"15min":0.30199999999999999},"max":{"1min":1.1060000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.80500000000000005}]},{"osd":2,"last update":"Tue Jul 18 03:06:54 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.66800000000000004,"15min":0.57399999999999995},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":5.3300000000000001,"5min":5.3300000000000001,"15min":5.3300000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.67400000000000004,"15min":0.57899999999999996},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.01,"5min":1.204,"15min":1.204},"last":0.49099999999999999}]},{"osd":3,"last update":"Tue Jul 18 03:07:04 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54200000000000004,"15min":0.54400000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.53300000000000003,"15min":0.53100000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.45400000000000001}]},{"osd":4,"last update":"Tue Jul 18 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.68600000000000005,"15min":0.624},"min":{"1min":0.51700000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.748},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.66500000000000004,"15min":0.61199999999999999},"min":{"1min":0.49099999999999999,"5min":0.433,"15min":0.433},"max":{"1min":0.97499999999999998,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.59299999999999997}]},{"osd":5,"last update":"Tue Jul 18 03:05:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.64700000000000002,"15min":0.63500000000000001},"min":{"1min":0.46899999999999997,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.028,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.68200000000000005,"15min":0.65900000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.998,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.79600000000000004}]},{"osd":6,"last update":"Tue Jul 18 03:06:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.60099999999999998,"15min":0.59099999999999997},"min":{"1min":0.40400000000000003,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.97299999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.63400000000000001,"15min":0.60399999999999998},"min":{"1min":0.40300000000000002,"5min":0.372,"15min":0.372},"max":{"1min":1.1639999999999999,"5min":1.216,"15min":1.216},"last":0.55900000000000005}]},{"osd":7,"last update":"Tue Jul 18 03:07:04 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.80600000000000005,"15min":0.77500000000000002},"min":{"1min":0.57299999999999995,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":5.2869999999999999,"5min":5.2869999999999999,"15min":5.2869999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.024,"5min":0.86899999999999999,"15min":0.84299999999999997},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":5.3579999999999997,"5min":5.3579999999999997,"15min":5.3579999999999997},"last":0.88100000000000001}]},{"osd":8,"last update":"Tue Jul 18 03:07:08 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":0.80200000000000005,"15min":0.76400000000000001},"min":{"1min":0.52900000000000003,"5min":0.495,"15min":0.495},"max":{"1min":6.96,"5min":6.96,"15min":6.96},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.0980000000000001,"5min":0.873,"15min":0.84799999999999998},"min":{"1min":0.54300000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":6.9909999999999997,"5min":6.9909999999999997,"15min":6.9909999999999997},"last":0.90000000000000002}]},{"osd":9,"last update":"Tue Jul 18 03:06:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.77900000000000003,"15min":0.78700000000000003},"min":{"1min":0.46300000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0409999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.85699999999999998,"15min":0.873},"min":{"1min":0.497,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1879999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.85699999999999998}]},{"osd":10,"last update":"Tue Jul 18 03:06:54 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":0.96699999999999997,"15min":0.94099999999999995},"min":{"1min":0.70899999999999996,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":6.9359999999999999,"5min":6.9359999999999999,"15min":6.9359999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.748,"5min":0.78000000000000003,"15min":0.78900000000000003},"min":{"1min":0.57899999999999996,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.99299999999999999,"5min":1.304,"15min":1.304},"last":0.69399999999999995}]},{"osd":11,"last update":"Tue Jul 18 03:06:16 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.81299999999999994,"15min":0.80900000000000005},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.0760000000000001,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.88600000000000001,"15min":0.88200000000000001},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.3160000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.93700000000000006}]}]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1064,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1089536,"data_stored":571053,"data_compressed":23524,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.56799999999999995,"15min":0.54000000000000004},"min":{"1min":0.38800000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58399999999999996,"15min":0.55500000000000005},"min":{"1min":0.41399999999999998,"5min":0.373,"15min":0.34599999999999997},"max":{"1min":0.89600000000000002,"5min":1.726,"15min":1.726},"last":0.46899999999999997}]},{"osd":1,"last update":"Tue Jul 18 03:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.61499999999999999,"15min":0.52700000000000002},"min":{"1min":0.49199999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.89200000000000002,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.60399999999999998,"15min":0.52000000000000002},"min":{"1min":0.47299999999999998,"5min":0.34300000000000003,"15min":0.32100000000000001},"max":{"1min":0.82499999999999996,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.57899999999999996}]},{"osd":3,"last update":"Tue Jul 18 03:06:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59299999999999997,"15min":0.56299999999999994},"min":{"1min":0.439,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93899999999999995,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.64400000000000002,"15min":0.59299999999999997},"min":{"1min":0.40899999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.127,"5min":3.234,"15min":3.234},"last":0.60599999999999998}]},{"osd":4,"last update":"Tue Jul 18 03:07:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66400000000000003,"15min":0.66300000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.73599999999999999,"15min":0.74199999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.95699999999999996,"5min":1.353,"15min":1.353},"last":0.55400000000000005}]},{"osd":5,"last update":"Tue Jul 18 03:07:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.51000000000000001,"15min":0.46200000000000002},"min":{"1min":0.34899999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.91500000000000004,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.53800000000000003,"15min":0.47199999999999998},"min":{"1min":0.41399999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.0309999999999999,"5min":1.335,"15min":1.335},"last":0.39900000000000002}]},{"osd":6,"last update":"Tue Jul 18 03:06:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.67700000000000005,"15min":0.64600000000000002},"min":{"1min":0.55700000000000005,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.95399999999999996,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.622},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.72799999999999998,"15min":0.69399999999999995},"min":{"1min":0.56999999999999995,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.6619999999999999,"5min":1.665,"15min":1.665},"last":0.67700000000000005}]},{"osd":7,"last update":"Tue Jul 18 03:07:04 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.84899999999999998,"15min":0.85299999999999998},"min":{"1min":0.61399999999999999,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0129999999999999,"5min":1.532,"15min":1.532},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.78000000000000003,"15min":0.78100000000000003},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.66300000000000003}]},{"osd":8,"last update":"Tue Jul 18 03:07:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.70099999999999996,"15min":0.64100000000000001},"min":{"1min":0.48799999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.159,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.70999999999999996,"15min":0.66700000000000004},"min":{"1min":0.61799999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.079,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.69799999999999995}]},{"osd":9,"last update":"Tue Jul 18 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.753,"15min":0.74199999999999999},"min":{"1min":0.49299999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.2549999999999999,"5min":1.825,"15min":1.825},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.81599999999999995,"15min":0.79300000000000004},"min":{"1min":0.70899999999999996,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.1020000000000001,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.80600000000000005}]},{"osd":10,"last update":"Tue Jul 18 03:06:54 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.82999999999999996,"15min":0.79300000000000004},"min":{"1min":0.79200000000000004,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1000000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.78400000000000003,"15min":0.753},"min":{"1min":0.71299999999999997,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0089999999999999,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.749}]},{"osd":11,"last update":"Tue Jul 18 03:07:04 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.92400000000000004,"15min":0.92600000000000005},"min":{"1min":0.73399999999999999,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1839999999999999,"5min":1.76,"15min":1.76},"last":0.875},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.873,"15min":0.876},"min":{"1min":0.67400000000000004,"5min":0.624,"15min":0.624},"max":{"1min":1.1160000000000001,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.78800000000000003}]}]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":1003520,"data_stored":502723,"data_compressed":23524,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:06:25 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.499,"15min":0.42699999999999999},"min":{"1min":0.34999999999999998,"5min":0.312,"15min":0.312},"max":{"1min":1.8440000000000001,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":0.748},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.504,"15min":0.433},"min":{"1min":0.35899999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.93300000000000005}]},{"osd":1,"last update":"Tue Jul 18 03:06:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.60499999999999998,"15min":0.57199999999999995},"min":{"1min":0.45600000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.79300000000000004,"5min":1.115,"15min":1.115},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.66700000000000004,"15min":0.59499999999999997},"min":{"1min":0.44400000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":0.79200000000000004}]},{"osd":2,"last update":"Tue Jul 18 03:06:25 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.68100000000000005,"15min":0.61399999999999999},"min":{"1min":0.437,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0960000000000001,"5min":1.26,"15min":1.26},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.63700000000000001,"15min":0.60299999999999998},"min":{"1min":0.46600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.048,"5min":1.212,"15min":1.212},"last":0.78200000000000003}]},{"osd":4,"last update":"Tue Jul 18 03:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.70299999999999996,"15min":0.63300000000000001},"min":{"1min":0.56999999999999995,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.706,"5min":1.706,"15min":1.706},"last":0.871},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.69999999999999996,"15min":0.629},"min":{"1min":0.55100000000000005,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0629999999999999,"5min":1.143,"15min":1.143},"last":0.89400000000000002}]},{"osd":5,"last update":"Tue Jul 18 03:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.72099999999999997,"15min":0.64900000000000002},"min":{"1min":0.53200000000000003,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.73799999999999999,"15min":0.68999999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0329999999999999,"5min":1.218,"15min":1.218},"last":0.72799999999999998}]},{"osd":6,"last update":"Tue Jul 18 03:06:29 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.71399999999999997,"15min":0.65800000000000003},"min":{"1min":0.434,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.65300000000000002,"15min":0.61499999999999999},"min":{"1min":0.371,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":2.669,"5min":2.669,"15min":2.669},"last":0.69899999999999995}]},{"osd":7,"last update":"Tue Jul 18 03:06:54 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.76300000000000001,"15min":0.73099999999999998},"min":{"1min":0.56999999999999995,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.78600000000000003,"15min":0.75900000000000001},"min":{"1min":0.58799999999999997,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.1659999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.80300000000000005}]},{"osd":8,"last update":"Tue Jul 18 03:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.76200000000000001}]},{"osd":9,"last update":"Tue Jul 18 03:06:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.76200000000000001,"15min":0.78300000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":0.88},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.86199999999999999,"15min":0.876},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.91600000000000004}]},{"osd":10,"last update":"Tue Jul 18 03:06:54 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.89500000000000002,"15min":0.90700000000000003},"min":{"1min":0.63600000000000001,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.77,"5min":1.77,"15min":1.77},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.875,"15min":0.88500000000000001},"min":{"1min":0.66500000000000004,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.159,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.97499999999999998}]},{"osd":11,"last update":"Tue Jul 18 03:07:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84199999999999997,"15min":0.84099999999999997},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.72499999999999998,"15min":0.71899999999999997},"min":{"1min":0.58999999999999997,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.81699999999999995}]}]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":1003520,"data_stored":507131,"data_compressed":23524,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.56200000000000006,"15min":0.54600000000000004},"min":{"1min":0.39600000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.439},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.56899999999999995,"15min":0.54500000000000004},"min":{"1min":0.42799999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.48999999999999999}]},{"osd":1,"last update":"Tue Jul 18 03:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40699999999999997,"15min":0.36799999999999999},"min":{"1min":0.28599999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.72099999999999997,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.41799999999999998,"15min":0.38400000000000001},"min":{"1min":0.314,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.58599999999999997,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.41299999999999998}]},{"osd":2,"last update":"Tue Jul 18 03:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.59899999999999998,"15min":0.57499999999999996},"min":{"1min":0.43099999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.85699999999999998,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.622,"15min":0.57299999999999995},"min":{"1min":0.41899999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.88100000000000001,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.60099999999999998}]},{"osd":3,"last update":"Tue Jul 18 03:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.71399999999999997,"15min":0.61699999999999999},"min":{"1min":0.51100000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":5.726,"5min":5.726,"15min":5.726},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.65300000000000002,"15min":0.59999999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":4.141,"5min":4.141,"15min":4.141},"last":0.439}]},{"osd":5,"last update":"Tue Jul 18 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.67600000000000005,"15min":0.61799999999999999},"min":{"1min":0.42699999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.86399999999999999,"5min":1.214,"15min":1.214},"last":0.62},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.65700000000000003,"15min":0.61499999999999999},"min":{"1min":0.48599999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.85099999999999998,"5min":1.306,"15min":1.306},"last":0.68300000000000005}]},{"osd":6,"last update":"Tue Jul 18 03:06:27 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.754,"15min":0.73799999999999999},"min":{"1min":0.46500000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":4.2850000000000001,"5min":4.2850000000000001,"15min":4.2850000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.69599999999999995,"15min":0.66900000000000004},"min":{"1min":0.41299999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":4.7130000000000001,"5min":4.7130000000000001,"15min":4.7130000000000001},"last":0.52800000000000002}]},{"osd":7,"last update":"Tue Jul 18 03:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.39700000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.89800000000000002,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.65300000000000002,"15min":0.64700000000000002},"min":{"1min":0.38700000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.98799999999999999,"5min":1.034,"15min":1.034},"last":0.58199999999999996}]},{"osd":8,"last update":"Tue Jul 18 03:06:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73999999999999999,"15min":0.73499999999999999},"min":{"1min":0.53300000000000003,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.99399999999999999,"5min":1.155,"15min":1.155},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.75800000000000001,"15min":0.74399999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.008,"5min":1.135,"15min":1.135},"last":0.72199999999999998}]},{"osd":9,"last update":"Tue Jul 18 03:06:35 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.77300000000000002,"15min":0.73799999999999999},"min":{"1min":0.56299999999999994,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":4.6719999999999997,"5min":4.6719999999999997,"15min":4.6719999999999997},"last":0.63200000000000001},{"interface":"front","average":{"1min":1.0169999999999999,"5min":0.86499999999999999,"15min":0.82499999999999996},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":5.0060000000000002,"5min":5.0060000000000002,"15min":5.0060000000000002},"last":0.73399999999999999}]},{"osd":10,"last update":"Tue Jul 18 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.78100000000000003,"15min":0.79400000000000004},"min":{"1min":0.54500000000000004,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97799999999999998,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68400000000000005,"15min":0.69699999999999995},"min":{"1min":0.47699999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91200000000000003,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.56799999999999995}]},{"osd":11,"last update":"Tue Jul 18 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.80300000000000005,"15min":0.80000000000000004},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.76300000000000001,"15min":0.76000000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.746}]}]},{"osd":5,"up_from":38,"seq":163208757315,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1069056,"data_stored":1098862,"data_compressed":59283,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56100000000000005,"15min":0.53500000000000003},"min":{"1min":0.40600000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.80500000000000005,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.53700000000000003,"15min":0.53400000000000003},"min":{"1min":0.379,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.78100000000000003,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.69399999999999995}]},{"osd":1,"last update":"Tue Jul 18 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.60199999999999998,"15min":0.55200000000000005},"min":{"1min":0.435,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.57799999999999996,"15min":0.55200000000000005},"min":{"1min":0.42399999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.74399999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.65400000000000003}]},{"osd":2,"last update":"Tue Jul 18 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.42099999999999999,"15min":0.38500000000000001},"min":{"1min":0.33800000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.436,"15min":0.40300000000000002},"min":{"1min":0.311,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.57499999999999996}]},{"osd":3,"last update":"Tue Jul 18 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.60599999999999998,"15min":0.57399999999999995},"min":{"1min":0.44500000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.81999999999999995,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.60999999999999999,"15min":0.57699999999999996},"min":{"1min":0.53500000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77600000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.72599999999999998}]},{"osd":4,"last update":"Tue Jul 18 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60599999999999998,"15min":0.57599999999999996},"min":{"1min":0.41899999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.77600000000000002,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.63500000000000001,"15min":0.57999999999999996},"min":{"1min":0.39900000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.86899999999999999,"5min":0.871,"15min":0.871},"last":0.84899999999999998}]},{"osd":6,"last update":"Tue Jul 18 03:06:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.627,"15min":0.60699999999999998},"min":{"1min":0.48299999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.75800000000000001,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.75,"5min":0.66900000000000004,"15min":0.63200000000000001},"min":{"1min":0.58799999999999997,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.83299999999999996}]},{"osd":7,"last update":"Tue Jul 18 03:06:48 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.48199999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.83099999999999996,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.69899999999999995,"15min":0.69299999999999995},"min":{"1min":0.49099999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.85399999999999998,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.80800000000000005}]},{"osd":8,"last update":"Tue Jul 18 03:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.56899999999999995,"15min":0.57399999999999995},"min":{"1min":0.45100000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.85899999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.63500000000000001,"15min":0.628},"min":{"1min":0.40799999999999997,"5min":0.371,"15min":0.371},"max":{"1min":0.84499999999999997,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.89500000000000002}]},{"osd":9,"last update":"Tue Jul 18 03:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76200000000000001,"15min":0.77000000000000002},"min":{"1min":0.52600000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.88300000000000001,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.76000000000000001,"15min":0.76600000000000001},"min":{"1min":0.64900000000000002,"5min":0.501,"15min":0.501},"max":{"1min":0.90300000000000002,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.90400000000000003}]},{"osd":10,"last update":"Tue Jul 18 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.75,"15min":0.73599999999999999},"min":{"1min":0.54800000000000004,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.92000000000000004,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.754,"15min":0.74299999999999999},"min":{"1min":0.56299999999999994,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.92400000000000004,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.877}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34117,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34117,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34117,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35759,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35759,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35759,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35759,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35759,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":66781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":68614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":68311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-18T03:07:20.713 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:07:20.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.954+0000 7ffa645ff700 1 -- 172.21.15.101:0/2541933758 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffa5c1005f0 msgr2=0x7ffa5c1029d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:20.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.954+0000 7ffa645ff700 1 --2- 172.21.15.101:0/2541933758 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffa5c1005f0 0x7ffa5c1029d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ffa58009990 tx=0x7ffa58009ca0 comp rx=0 tx=0).stop 2023-07-18T03:07:20.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.954+0000 7ffa645ff700 1 -- 172.21.15.101:0/2541933758 shutdown_connections 2023-07-18T03:07:20.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.954+0000 7ffa645ff700 1 --2- 172.21.15.101:0/2541933758 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa5c105830 0x7ffa5c107c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:20.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.954+0000 7ffa645ff700 1 --2- 172.21.15.101:0/2541933758 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffa5c1005f0 0x7ffa5c1029d0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:20.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.954+0000 7ffa645ff700 1 --2- 172.21.15.101:0/2541933758 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa5c102f10 0x7ffa5c1052f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:20.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.954+0000 7ffa645ff700 1 -- 172.21.15.101:0/2541933758 >> 172.21.15.101:0/2541933758 conn(0x7ffa5c0f9fd0 msgr2=0x7ffa5c0fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:20.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.954+0000 7ffa645ff700 1 -- 172.21.15.101:0/2541933758 shutdown_connections 2023-07-18T03:07:20.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.954+0000 7ffa645ff700 1 -- 172.21.15.101:0/2541933758 wait complete. 2023-07-18T03:07:20.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.955+0000 7ffa645ff700 1 Processor -- start 2023-07-18T03:07:20.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.956+0000 7ffa645ff700 1 -- start start 2023-07-18T03:07:20.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.956+0000 7ffa645ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffa5c1005f0 0x7ffa5c0fff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:20.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.956+0000 7ffa645ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa5c102f10 0x7ffa5c0fe5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:20.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.956+0000 7ffa645ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa5c105830 0x7ffa5c0feae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:20.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.956+0000 7ffa645ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5c0ff280 con 0x7ffa5c1005f0 2023-07-18T03:07:20.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.956+0000 7ffa645ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5c0ff3f0 con 0x7ffa5c105830 2023-07-18T03:07:20.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.956+0000 7ffa645ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5c19f840 con 0x7ffa5c102f10 2023-07-18T03:07:20.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.956+0000 7ffa6239b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffa5c1005f0 0x7ffa5c0fff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:20.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.956+0000 7ffa6239b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffa5c1005f0 0x7ffa5c0fff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:36490/0 (socket says 172.21.15.101:36490) 2023-07-18T03:07:20.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.956+0000 7ffa6239b700 1 -- 172.21.15.101:0/3604314416 learned_addr learned my addr 172.21.15.101:0/3604314416 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:20.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.956+0000 7ffa62b9c700 1 --2- 172.21.15.101:0/3604314416 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa5c105830 0x7ffa5c0feae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:20.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.956+0000 7ffa61b9a700 1 --2- 172.21.15.101:0/3604314416 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa5c102f10 0x7ffa5c0fe5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:20.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.956+0000 7ffa6239b700 1 -- 172.21.15.101:0/3604314416 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa5c102f10 msgr2=0x7ffa5c0fe5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:20.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.956+0000 7ffa6239b700 1 --2- 172.21.15.101:0/3604314416 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa5c102f10 0x7ffa5c0fe5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:20.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.956+0000 7ffa6239b700 1 -- 172.21.15.101:0/3604314416 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa5c105830 msgr2=0x7ffa5c0feae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:20.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.956+0000 7ffa6239b700 1 --2- 172.21.15.101:0/3604314416 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa5c105830 0x7ffa5c0feae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:20.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.956+0000 7ffa6239b700 1 -- 172.21.15.101:0/3604314416 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa580095f0 con 0x7ffa5c1005f0 2023-07-18T03:07:20.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.957+0000 7ffa6239b700 1 --2- 172.21.15.101:0/3604314416 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffa5c1005f0 0x7ffa5c0fff60 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ffa58004740 tx=0x7ffa58009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:20.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.957+0000 7ffa537fe700 1 -- 172.21.15.101:0/3604314416 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5801d070 con 0x7ffa5c1005f0 2023-07-18T03:07:20.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.957+0000 7ffa537fe700 1 -- 172.21.15.101:0/3604314416 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa58004830 con 0x7ffa5c1005f0 2023-07-18T03:07:20.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.957+0000 7ffa645ff700 1 -- 172.21.15.101:0/3604314416 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa5c19fac0 con 0x7ffa5c1005f0 2023-07-18T03:07:20.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.957+0000 7ffa537fe700 1 -- 172.21.15.101:0/3604314416 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa58005450 con 0x7ffa5c1005f0 2023-07-18T03:07:20.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.957+0000 7ffa645ff700 1 -- 172.21.15.101:0/3604314416 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa5c19fe80 con 0x7ffa5c1005f0 2023-07-18T03:07:20.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.958+0000 7ffa645ff700 1 -- 172.21.15.101:0/3604314416 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa5c0fbb40 con 0x7ffa5c1005f0 2023-07-18T03:07:20.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.960+0000 7ffa537fe700 1 -- 172.21.15.101:0/3604314416 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7ffa5800fb00 con 0x7ffa5c1005f0 2023-07-18T03:07:20.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.962+0000 7ffa537fe700 1 --2- 172.21.15.101:0/3604314416 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7ffa480a11a0 0x7ffa480a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:20.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.962+0000 7ffa537fe700 1 -- 172.21.15.101:0/3604314416 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7ffa580cdea0 con 0x7ffa5c1005f0 2023-07-18T03:07:20.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.962+0000 7ffa61b9a700 1 --2- 172.21.15.101:0/3604314416 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7ffa480a11a0 0x7ffa480a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:20.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.963+0000 7ffa61b9a700 1 --2- 172.21.15.101:0/3604314416 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7ffa480a11a0 0x7ffa480a3650 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c009bf0 tx=0x7ffa4c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:20.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:20.966+0000 7ffa537fe700 1 -- 172.21.15.101:0/3604314416 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ffa5805a790 con 0x7ffa5c1005f0 2023-07-18T03:07:21.156 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.169+0000 7ffa645ff700 1 -- 172.21.15.101:0/3604314416 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa5c02c850 con 0x7ffa5c1005f0 2023-07-18T03:07:21.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.171+0000 7ffa537fe700 1 -- 172.21.15.101:0/3604314416 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24725 (secure 0 0 0) 0x7ffa580972b0 con 0x7ffa5c1005f0 2023-07-18T03:07:21.158 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:07:21.158 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":91,"fsid":"b1782daa-2516-11ee-9b34-001a4aab830c","created":"2023-07-18T02:57:22.972483+0000","modified":"2023-07-18T03:06:24.250777+0000","last_up_change":"2023-07-18T03:04:11.429530+0000","last_in_change":"2023-07-18T03:03:54.158340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:00:43.353489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:04:57.771616+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-18T03:04:58.769590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-18T03:05:02.358830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc76b5d1-38a0-4f1a-9335-756bfc40a62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6803","nonce":3610719929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6805","nonce":3610719929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6809","nonce":3610719929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6807","nonce":3610719929}]},"public_addr":"172.21.15.101:6803/3610719929","cluster_addr":"172.21.15.101:6805/3610719929","heartbeat_back_addr":"172.21.15.101:6809/3610719929","heartbeat_front_addr":"172.21.15.101:6807/3610719929","state":["exists","up"]},{"osd":1,"uuid":"2a299573-0b34-4af9-aaae-b523e5058ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6801","nonce":1030229579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6803","nonce":1030229579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6807","nonce":1030229579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6805","nonce":1030229579}]},"public_addr":"172.21.15.155:6801/1030229579","cluster_addr":"172.21.15.155:6803/1030229579","heartbeat_back_addr":"172.21.15.155:6807/1030229579","heartbeat_front_addr":"172.21.15.155:6805/1030229579","state":["exists","up"]},{"osd":2,"uuid":"123b3749-c636-48fa-9e33-6229b5575dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6801","nonce":1349487507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6803","nonce":1349487507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6807","nonce":1349487507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6805","nonce":1349487507}]},"public_addr":"172.21.15.179:6801/1349487507","cluster_addr":"172.21.15.179:6803/1349487507","heartbeat_back_addr":"172.21.15.179:6807/1349487507","heartbeat_front_addr":"172.21.15.179:6805/1349487507","state":["exists","up"]},{"osd":3,"uuid":"4b0c409d-2954-4127-833a-b196bf226d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6811","nonce":4226577463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6813","nonce":4226577463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6817","nonce":4226577463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6815","nonce":4226577463}]},"public_addr":"172.21.15.101:6811/4226577463","cluster_addr":"172.21.15.101:6813/4226577463","heartbeat_back_addr":"172.21.15.101:6817/4226577463","heartbeat_front_addr":"172.21.15.101:6815/4226577463","state":["exists","up"]},{"osd":4,"uuid":"2137f2df-984e-45ce-baf6-b4ac12c7e3c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6809","nonce":1951135805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6811","nonce":1951135805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6815","nonce":1951135805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6813","nonce":1951135805}]},"public_addr":"172.21.15.155:6809/1951135805","cluster_addr":"172.21.15.155:6811/1951135805","heartbeat_back_addr":"172.21.15.155:6815/1951135805","heartbeat_front_addr":"172.21.15.155:6813/1951135805","state":["exists","up"]},{"osd":5,"uuid":"1020bfc5-8c2f-482e-9588-66c7f98e46e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6809","nonce":2746037021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6811","nonce":2746037021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6815","nonce":2746037021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6813","nonce":2746037021}]},"public_addr":"172.21.15.179:6809/2746037021","cluster_addr":"172.21.15.179:6811/2746037021","heartbeat_back_addr":"172.21.15.179:6815/2746037021","heartbeat_front_addr":"172.21.15.179:6813/2746037021","state":["exists","up"]},{"osd":6,"uuid":"2cd99260-5c36-4603-9338-cf697970ce58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6819","nonce":958044931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6821","nonce":958044931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6825","nonce":958044931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6823","nonce":958044931}]},"public_addr":"172.21.15.101:6819/958044931","cluster_addr":"172.21.15.101:6821/958044931","heartbeat_back_addr":"172.21.15.101:6825/958044931","heartbeat_front_addr":"172.21.15.101:6823/958044931","state":["exists","up"]},{"osd":7,"uuid":"ed23fd0a-678f-4e15-b42c-cfe873315714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6817","nonce":3256322291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6819","nonce":3256322291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6823","nonce":3256322291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6821","nonce":3256322291}]},"public_addr":"172.21.15.155:6817/3256322291","cluster_addr":"172.21.15.155:6819/3256322291","heartbeat_back_addr":"172.21.15.155:6823/3256322291","heartbeat_front_addr":"172.21.15.155:6821/3256322291","state":["exists","up"]},{"osd":8,"uuid":"eca3d8e6-7baf-47a5-b991-328001c7ebef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6817","nonce":3520612559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6819","nonce":3520612559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6823","nonce":3520612559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6821","nonce":3520612559}]},"public_addr":"172.21.15.179:6817/3520612559","cluster_addr":"172.21.15.179:6819/3520612559","heartbeat_back_addr":"172.21.15.179:6823/3520612559","heartbeat_front_addr":"172.21.15.179:6821/3520612559","state":["exists","up"]},{"osd":9,"uuid":"465a7ffb-67e4-49ef-812c-1b96feacfebf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6827","nonce":2166787667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6829","nonce":2166787667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6833","nonce":2166787667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6831","nonce":2166787667}]},"public_addr":"172.21.15.101:6827/2166787667","cluster_addr":"172.21.15.101:6829/2166787667","heartbeat_back_addr":"172.21.15.101:6833/2166787667","heartbeat_front_addr":"172.21.15.101:6831/2166787667","state":["exists","up"]},{"osd":10,"uuid":"1897a027-f25d-434f-8df5-5fc2bd866578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6825","nonce":3682446349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6827","nonce":3682446349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6831","nonce":3682446349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6829","nonce":3682446349}]},"public_addr":"172.21.15.155:6825/3682446349","cluster_addr":"172.21.15.155:6827/3682446349","heartbeat_back_addr":"172.21.15.155:6831/3682446349","heartbeat_front_addr":"172.21.15.155:6829/3682446349","state":["exists","up"]},{"osd":11,"uuid":"b62effaf-45e2-4585-b8d0-db448f3474ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6825","nonce":2602640858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6827","nonce":2602640858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6831","nonce":2602640858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6829","nonce":2602640858}]},"public_addr":"172.21.15.179:6825/2602640858","cluster_addr":"172.21.15.179:6827/2602640858","heartbeat_back_addr":"172.21.15.179:6831/2602640858","heartbeat_front_addr":"172.21.15.179:6829/2602640858","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T02:59:56.074519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:19.358142+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:41.128966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:08.227054+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:31.442592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:52.751685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:19.251054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:40.698241+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:01.339150+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:27.955910+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:49.136980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:04:10.084982+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.101:6800/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:6801/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/604228291":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/4055187849":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/2844574324":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/1603248215":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6801/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6800/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/3307578182":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/373189792":"2023-07-19T02:57:56.243200+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:07:21.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.173+0000 7ffa645ff700 1 -- 172.21.15.101:0/3604314416 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7ffa480a11a0 msgr2=0x7ffa480a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:21.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.173+0000 7ffa645ff700 1 --2- 172.21.15.101:0/3604314416 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7ffa480a11a0 0x7ffa480a3650 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c009bf0 tx=0x7ffa4c009260 comp rx=0 tx=0).stop 2023-07-18T03:07:21.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.173+0000 7ffa645ff700 1 -- 172.21.15.101:0/3604314416 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffa5c1005f0 msgr2=0x7ffa5c0fff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:21.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.173+0000 7ffa645ff700 1 --2- 172.21.15.101:0/3604314416 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffa5c1005f0 0x7ffa5c0fff60 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ffa58004740 tx=0x7ffa58009580 comp rx=0 tx=0).stop 2023-07-18T03:07:21.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.174+0000 7ffa645ff700 1 -- 172.21.15.101:0/3604314416 shutdown_connections 2023-07-18T03:07:21.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.174+0000 7ffa645ff700 1 --2- 172.21.15.101:0/3604314416 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7ffa480a11a0 0x7ffa480a3650 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:21.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.174+0000 7ffa645ff700 1 --2- 172.21.15.101:0/3604314416 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa5c105830 0x7ffa5c0feae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:21.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.174+0000 7ffa645ff700 1 --2- 172.21.15.101:0/3604314416 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffa5c1005f0 0x7ffa5c0fff60 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:21.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.174+0000 7ffa645ff700 1 --2- 172.21.15.101:0/3604314416 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa5c102f10 0x7ffa5c0fe5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:21.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.174+0000 7ffa645ff700 1 -- 172.21.15.101:0/3604314416 >> 172.21.15.101:0/3604314416 conn(0x7ffa5c0f9fd0 msgr2=0x7ffa5c101250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:21.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.174+0000 7ffa645ff700 1 -- 172.21.15.101:0/3604314416 shutdown_connections 2023-07-18T03:07:21.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.174+0000 7ffa645ff700 1 -- 172.21.15.101:0/3604314416 wait complete. 2023-07-18T03:07:21.179 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:00:43.353489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-18T03:04:57.771616+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-18T03:04:58.769590+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-18T03:05:02.358830+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-18T03:07:21.180 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-18T03:07:21.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:20 smithi101 ceph-mon[126379]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:07:21.232 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:20 smithi101 ceph-mon[126379]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:07:21.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:20 smithi155 ceph-mon[128885]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:07:21.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:20 smithi155 ceph-mon[128885]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:07:21.379 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.392+0000 7f38447ff700 1 -- 172.21.15.101:0/262913935 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f383c0ffaf0 msgr2=0x7f383c0fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:21.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.392+0000 7f38447ff700 1 --2- 172.21.15.101:0/262913935 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f383c0ffaf0 0x7f383c0fff00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3838009990 tx=0x7f3838009ca0 comp rx=0 tx=0).stop 2023-07-18T03:07:21.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.392+0000 7f38447ff700 1 -- 172.21.15.101:0/262913935 shutdown_connections 2023-07-18T03:07:21.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.392+0000 7f38447ff700 1 --2- 172.21.15.101:0/262913935 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f383c101670 0x7f383c107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:21.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.392+0000 7f38447ff700 1 --2- 172.21.15.101:0/262913935 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f383c0ffaf0 0x7f383c0fff00 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:21.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.392+0000 7f38447ff700 1 --2- 172.21.15.101:0/262913935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f383c100ce0 0x7f383c101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:21.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.393+0000 7f38447ff700 1 -- 172.21.15.101:0/262913935 >> 172.21.15.101:0/262913935 conn(0x7f383c0fb130 msgr2=0x7f383c0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:21.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.393+0000 7f38447ff700 1 -- 172.21.15.101:0/262913935 shutdown_connections 2023-07-18T03:07:21.381 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.393+0000 7f38447ff700 1 -- 172.21.15.101:0/262913935 wait complete. 2023-07-18T03:07:21.382 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.394+0000 7f38447ff700 1 Processor -- start 2023-07-18T03:07:21.382 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.394+0000 7f38447ff700 1 -- start start 2023-07-18T03:07:21.382 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.394+0000 7f38447ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f383c0ffaf0 0x7f383c1958d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:21.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.394+0000 7f38447ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f383c100ce0 0x7f383c195e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:21.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.394+0000 7f38447ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f383c101670 0x7f383c19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:21.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.394+0000 7f38447ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f383c19d450 con 0x7f383c101670 2023-07-18T03:07:21.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.394+0000 7f38447ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f383c19d590 con 0x7f383c0ffaf0 2023-07-18T03:07:21.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.394+0000 7f38447ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f383c19d720 con 0x7f383c100ce0 2023-07-18T03:07:21.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.394+0000 7f3842d9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f383c101670 0x7f383c19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:21.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.394+0000 7f3841d9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f383c100ce0 0x7f383c195e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:21.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.394+0000 7f384259b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f383c0ffaf0 0x7f383c1958d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:21.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.395+0000 7f3842d9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f383c101670 0x7f383c19ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:36510/0 (socket says 172.21.15.101:36510) 2023-07-18T03:07:21.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.395+0000 7f3841d9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f383c100ce0 0x7f383c195e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:48276/0 (socket says 172.21.15.101:48276) 2023-07-18T03:07:21.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.395+0000 7f3842d9c700 1 -- 172.21.15.101:0/1594227083 learned_addr learned my addr 172.21.15.101:0/1594227083 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:21.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.395+0000 7f3842d9c700 1 -- 172.21.15.101:0/1594227083 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f383c100ce0 msgr2=0x7f383c195e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:21.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.395+0000 7f3842d9c700 1 --2- 172.21.15.101:0/1594227083 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f383c100ce0 0x7f383c195e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:21.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.395+0000 7f3842d9c700 1 -- 172.21.15.101:0/1594227083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f383c0ffaf0 msgr2=0x7f383c1958d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:21.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.395+0000 7f3842d9c700 1 --2- 172.21.15.101:0/1594227083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f383c0ffaf0 0x7f383c1958d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:21.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.395+0000 7f3842d9c700 1 -- 172.21.15.101:0/1594227083 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38380095f0 con 0x7f383c101670 2023-07-18T03:07:21.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.395+0000 7f3841d9a700 1 --2- 172.21.15.101:0/1594227083 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f383c100ce0 0x7f383c195e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:07:21.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.395+0000 7f384259b700 1 --2- 172.21.15.101:0/1594227083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f383c0ffaf0 0x7f383c1958d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:07:21.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.395+0000 7f3842d9c700 1 --2- 172.21.15.101:0/1594227083 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f383c101670 0x7f383c19ce80 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f383400ef30 tx=0x7f383400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:21.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.395+0000 7f38337fe700 1 -- 172.21.15.101:0/1594227083 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3834018070 con 0x7f383c101670 2023-07-18T03:07:21.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.395+0000 7f38337fe700 1 -- 172.21.15.101:0/1594227083 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f383400f040 con 0x7f383c101670 2023-07-18T03:07:21.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.395+0000 7f38447ff700 1 -- 172.21.15.101:0/1594227083 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f383c19d9a0 con 0x7f383c101670 2023-07-18T03:07:21.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.395+0000 7f38337fe700 1 -- 172.21.15.101:0/1594227083 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383400cd00 con 0x7f383c101670 2023-07-18T03:07:21.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.396+0000 7f38447ff700 1 -- 172.21.15.101:0/1594227083 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f383c19df00 con 0x7f383c101670 2023-07-18T03:07:21.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.397+0000 7f38447ff700 1 -- 172.21.15.101:0/1594227083 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f383c107080 con 0x7f383c101670 2023-07-18T03:07:21.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.398+0000 7f38337fe700 1 -- 172.21.15.101:0/1594227083 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f3834003a00 con 0x7f383c101670 2023-07-18T03:07:21.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.401+0000 7f38337fe700 1 --2- 172.21.15.101:0/1594227083 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f38280a11d0 0x7f38280a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:21.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.401+0000 7f38337fe700 1 -- 172.21.15.101:0/1594227083 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f38340cc6f0 con 0x7f383c101670 2023-07-18T03:07:21.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.401+0000 7f384259b700 1 --2- 172.21.15.101:0/1594227083 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f38280a11d0 0x7f38280a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:21.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.401+0000 7f384259b700 1 --2- 172.21.15.101:0/1594227083 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f38280a11d0 0x7f38280a3680 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f383800b2e0 tx=0x7f3838005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:21.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.404+0000 7f38337fe700 1 -- 172.21.15.101:0/1594227083 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3834095b00 con 0x7f383c101670 2023-07-18T03:07:21.444 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:20 smithi179 ceph-mon[129229]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:07:21.444 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:20 smithi179 ceph-mon[129229]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:07:21.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.600+0000 7f38447ff700 1 -- 172.21.15.101:0/1594227083 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f383c061120 con 0x7f383c101670 2023-07-18T03:07:21.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.600+0000 7f38337fe700 1 -- 172.21.15.101:0/1594227083 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3834099120 con 0x7f383c101670 2023-07-18T03:07:21.587 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 1 2023-07-18T03:07:21.589 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.603+0000 7f38447ff700 1 -- 172.21.15.101:0/1594227083 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f38280a11d0 msgr2=0x7f38280a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:21.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.603+0000 7f38447ff700 1 --2- 172.21.15.101:0/1594227083 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f38280a11d0 0x7f38280a3680 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f383800b2e0 tx=0x7f3838005fb0 comp rx=0 tx=0).stop 2023-07-18T03:07:21.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.603+0000 7f38447ff700 1 -- 172.21.15.101:0/1594227083 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f383c101670 msgr2=0x7f383c19ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:21.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.603+0000 7f38447ff700 1 --2- 172.21.15.101:0/1594227083 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f383c101670 0x7f383c19ce80 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f383400ef30 tx=0x7f383400c3a0 comp rx=0 tx=0).stop 2023-07-18T03:07:21.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.603+0000 7f38447ff700 1 -- 172.21.15.101:0/1594227083 shutdown_connections 2023-07-18T03:07:21.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.603+0000 7f38447ff700 1 --2- 172.21.15.101:0/1594227083 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f38280a11d0 0x7f38280a3680 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:21.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.603+0000 7f38447ff700 1 --2- 172.21.15.101:0/1594227083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f383c0ffaf0 0x7f383c1958d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:21.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.603+0000 7f38447ff700 1 --2- 172.21.15.101:0/1594227083 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f383c101670 0x7f383c19ce80 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:21.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.603+0000 7f38447ff700 1 --2- 172.21.15.101:0/1594227083 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f383c100ce0 0x7f383c195e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:21.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.603+0000 7f38447ff700 1 -- 172.21.15.101:0/1594227083 >> 172.21.15.101:0/1594227083 conn(0x7f383c0fb130 msgr2=0x7f383c1041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:21.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.603+0000 7f38447ff700 1 -- 172.21.15.101:0/1594227083 shutdown_connections 2023-07-18T03:07:21.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.603+0000 7f38447ff700 1 -- 172.21.15.101:0/1594227083 wait complete. 2023-07-18T03:07:21.608 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-18T03:07:21.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.822+0000 7f5ce0dff700 1 -- 172.21.15.101:0/2772423666 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5cdc102e80 msgr2=0x7f5cdc105260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:21.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.822+0000 7f5ce0dff700 1 --2- 172.21.15.101:0/2772423666 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5cdc102e80 0x7f5cdc105260 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4009990 tx=0x7f5cc4009ca0 comp rx=0 tx=0).stop 2023-07-18T03:07:21.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.822+0000 7f5ce0dff700 1 -- 172.21.15.101:0/2772423666 shutdown_connections 2023-07-18T03:07:21.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.822+0000 7f5ce0dff700 1 --2- 172.21.15.101:0/2772423666 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5cdc100560 0x7f5cdc102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:21.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.822+0000 7f5ce0dff700 1 --2- 172.21.15.101:0/2772423666 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5cdc1057a0 0x7f5cdc107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:21.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.822+0000 7f5ce0dff700 1 --2- 172.21.15.101:0/2772423666 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5cdc102e80 0x7f5cdc105260 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:21.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.822+0000 7f5ce0dff700 1 -- 172.21.15.101:0/2772423666 >> 172.21.15.101:0/2772423666 conn(0x7f5cdc0fa000 msgr2=0x7f5cdc0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:21.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.822+0000 7f5ce0dff700 1 -- 172.21.15.101:0/2772423666 shutdown_connections 2023-07-18T03:07:21.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.822+0000 7f5ce0dff700 1 -- 172.21.15.101:0/2772423666 wait complete. 2023-07-18T03:07:21.814 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.823+0000 7f5ce0dff700 1 Processor -- start 2023-07-18T03:07:21.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.823+0000 7f5ce0dff700 1 -- start start 2023-07-18T03:07:21.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.824+0000 7f5ce0dff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5cdc100560 0x7f5cdc197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:21.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.824+0000 7f5ce0dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5cdc102e80 0x7f5cdc197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:21.816 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.824+0000 7f5ce0dff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5cdc1057a0 0x7f5cdc19f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:21.816 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.824+0000 7f5ce0dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cdc19f5f0 con 0x7f5cdc102e80 2023-07-18T03:07:21.816 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.824+0000 7f5ce0dff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cdc19f730 con 0x7f5cdc100560 2023-07-18T03:07:21.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.824+0000 7f5ce0dff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cdc19f8c0 con 0x7f5cdc1057a0 2023-07-18T03:07:21.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.824+0000 7f5cd3fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5cdc102e80 0x7f5cdc197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:21.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.824+0000 7f5cd3fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5cdc102e80 0x7f5cdc197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:36526/0 (socket says 172.21.15.101:36526) 2023-07-18T03:07:21.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.824+0000 7f5cdad9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5cdc1057a0 0x7f5cdc19f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:21.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.824+0000 7f5cda59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5cdc100560 0x7f5cdc197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:21.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.824+0000 7f5cd3fff700 1 -- 172.21.15.101:0/1783679905 learned_addr learned my addr 172.21.15.101:0/1783679905 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:21.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.824+0000 7f5cd3fff700 1 -- 172.21.15.101:0/1783679905 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5cdc1057a0 msgr2=0x7f5cdc19f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:21.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.824+0000 7f5cd3fff700 1 --2- 172.21.15.101:0/1783679905 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5cdc1057a0 0x7f5cdc19f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:21.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.824+0000 7f5cd3fff700 1 -- 172.21.15.101:0/1783679905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5cdc100560 msgr2=0x7f5cdc197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:21.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.824+0000 7f5cd3fff700 1 --2- 172.21.15.101:0/1783679905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5cdc100560 0x7f5cdc197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:21.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.824+0000 7f5cd3fff700 1 -- 172.21.15.101:0/1783679905 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc40095f0 con 0x7f5cdc102e80 2023-07-18T03:07:21.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.824+0000 7f5cd3fff700 1 --2- 172.21.15.101:0/1783679905 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5cdc102e80 0x7f5cdc197fb0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4004740 tx=0x7f5cc4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:21.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.825+0000 7f5cd37fe700 1 -- 172.21.15.101:0/1783679905 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc401d070 con 0x7f5cdc102e80 2023-07-18T03:07:21.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.825+0000 7f5cd37fe700 1 -- 172.21.15.101:0/1783679905 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5cc4004830 con 0x7f5cdc102e80 2023-07-18T03:07:21.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.825+0000 7f5ce0dff700 1 -- 172.21.15.101:0/1783679905 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cdc19fb40 con 0x7f5cdc102e80 2023-07-18T03:07:21.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.825+0000 7f5cd37fe700 1 -- 172.21.15.101:0/1783679905 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc40053c0 con 0x7f5cdc102e80 2023-07-18T03:07:21.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.825+0000 7f5ce0dff700 1 -- 172.21.15.101:0/1783679905 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cdc19ff00 con 0x7f5cdc102e80 2023-07-18T03:07:21.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.826+0000 7f5ce0dff700 1 -- 172.21.15.101:0/1783679905 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cdc107360 con 0x7f5cdc102e80 2023-07-18T03:07:21.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.828+0000 7f5cd37fe700 1 -- 172.21.15.101:0/1783679905 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f5cc4005580 con 0x7f5cdc102e80 2023-07-18T03:07:21.822 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.829+0000 7f5cd37fe700 1 --2- 172.21.15.101:0/1783679905 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f5cc80a1150 0x7f5cc80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:21.822 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.829+0000 7f5cd37fe700 1 -- 172.21.15.101:0/1783679905 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f5cc40cde50 con 0x7f5cdc102e80 2023-07-18T03:07:21.822 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.829+0000 7f5cda59c700 1 --2- 172.21.15.101:0/1783679905 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f5cc80a1150 0x7f5cc80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:21.823 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.830+0000 7f5cda59c700 1 --2- 172.21.15.101:0/1783679905 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f5cc80a1150 0x7f5cc80a3600 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc009c60 tx=0x7f5ccc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:21.823 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:21.832+0000 7f5cd37fe700 1 -- 172.21.15.101:0/1783679905 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5cc405a790 con 0x7f5cdc102e80 2023-07-18T03:07:22.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.031+0000 7f5ce0dff700 1 -- 172.21.15.101:0/1783679905 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5cdc10b2f0 con 0x7f5cdc102e80 2023-07-18T03:07:22.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.032+0000 7f5cd37fe700 1 -- 172.21.15.101:0/1783679905 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5cc40972c0 con 0x7f5cdc102e80 2023-07-18T03:07:22.019 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 32 2023-07-18T03:07:22.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.034+0000 7f5ce0dff700 1 -- 172.21.15.101:0/1783679905 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f5cc80a1150 msgr2=0x7f5cc80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:22.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.034+0000 7f5ce0dff700 1 --2- 172.21.15.101:0/1783679905 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f5cc80a1150 0x7f5cc80a3600 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc009c60 tx=0x7f5ccc009260 comp rx=0 tx=0).stop 2023-07-18T03:07:22.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.034+0000 7f5ce0dff700 1 -- 172.21.15.101:0/1783679905 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5cdc102e80 msgr2=0x7f5cdc197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:22.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.034+0000 7f5ce0dff700 1 --2- 172.21.15.101:0/1783679905 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5cdc102e80 0x7f5cdc197fb0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4004740 tx=0x7f5cc4009580 comp rx=0 tx=0).stop 2023-07-18T03:07:22.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.034+0000 7f5ce0dff700 1 -- 172.21.15.101:0/1783679905 shutdown_connections 2023-07-18T03:07:22.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.034+0000 7f5ce0dff700 1 --2- 172.21.15.101:0/1783679905 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f5cc80a1150 0x7f5cc80a3600 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:22.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.034+0000 7f5ce0dff700 1 --2- 172.21.15.101:0/1783679905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5cdc100560 0x7f5cdc197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:22.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.034+0000 7f5ce0dff700 1 --2- 172.21.15.101:0/1783679905 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5cdc1057a0 0x7f5cdc19f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:22.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.034+0000 7f5ce0dff700 1 --2- 172.21.15.101:0/1783679905 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5cdc102e80 0x7f5cdc197fb0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:22.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.034+0000 7f5ce0dff700 1 -- 172.21.15.101:0/1783679905 >> 172.21.15.101:0/1783679905 conn(0x7f5cdc0fa000 msgr2=0x7f5cdc0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:22.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.034+0000 7f5ce0dff700 1 -- 172.21.15.101:0/1783679905 shutdown_connections 2023-07-18T03:07:22.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.034+0000 7f5ce0dff700 1 -- 172.21.15.101:0/1783679905 wait complete. 2023-07-18T03:07:22.039 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-18T03:07:22.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:21 smithi101 ceph-mon[126379]: from='client.15387 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:07:22.232 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:21 smithi101 ceph-mon[126379]: pgmap v350: 129 pgs: 129 active+clean; 646 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 6 op/s 2023-07-18T03:07:22.232 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:21 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3604314416' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:07:22.232 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:21 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1594227083' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:07:22.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.255+0000 7f6de17ff700 1 -- 172.21.15.101:0/3906854301 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6ddc0ffaf0 msgr2=0x7f6ddc0fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:22.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.255+0000 7f6de17ff700 1 --2- 172.21.15.101:0/3906854301 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6ddc0ffaf0 0x7f6ddc0fff00 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc009990 tx=0x7f6dcc009ca0 comp rx=0 tx=0).stop 2023-07-18T03:07:22.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.255+0000 7f6de17ff700 1 -- 172.21.15.101:0/3906854301 shutdown_connections 2023-07-18T03:07:22.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.255+0000 7f6de17ff700 1 --2- 172.21.15.101:0/3906854301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ddc101670 0x7f6ddc107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:22.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.255+0000 7f6de17ff700 1 --2- 172.21.15.101:0/3906854301 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6ddc0ffaf0 0x7f6ddc0fff00 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:22.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.255+0000 7f6de17ff700 1 --2- 172.21.15.101:0/3906854301 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ddc100ce0 0x7f6ddc101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:22.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.255+0000 7f6de17ff700 1 -- 172.21.15.101:0/3906854301 >> 172.21.15.101:0/3906854301 conn(0x7f6ddc0fb130 msgr2=0x7f6ddc0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:22.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.255+0000 7f6de17ff700 1 -- 172.21.15.101:0/3906854301 shutdown_connections 2023-07-18T03:07:22.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.255+0000 7f6de17ff700 1 -- 172.21.15.101:0/3906854301 wait complete. 2023-07-18T03:07:22.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.256+0000 7f6de17ff700 1 Processor -- start 2023-07-18T03:07:22.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.256+0000 7f6de17ff700 1 -- start start 2023-07-18T03:07:22.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.257+0000 7f6de17ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ddc0ffaf0 0x7f6ddc197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:22.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.257+0000 7f6de17ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6ddc100ce0 0x7f6ddc197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:22.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.257+0000 7f6de17ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ddc101670 0x7f6ddc19f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:22.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.257+0000 7f6de17ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc19f620 con 0x7f6ddc100ce0 2023-07-18T03:07:22.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.257+0000 7f6de17ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc19f760 con 0x7f6ddc0ffaf0 2023-07-18T03:07:22.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.257+0000 7f6de17ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc19f8f0 con 0x7f6ddc101670 2023-07-18T03:07:22.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.257+0000 7f6dda7fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6ddc100ce0 0x7f6ddc197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:22.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.257+0000 7f6dda7fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6ddc100ce0 0x7f6ddc197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:36556/0 (socket says 172.21.15.101:36556) 2023-07-18T03:07:22.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.257+0000 7f6dda7fc700 1 -- 172.21.15.101:0/1212249758 learned_addr learned my addr 172.21.15.101:0/1212249758 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:22.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.257+0000 7f6ddaffd700 1 --2- 172.21.15.101:0/1212249758 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ddc0ffaf0 0x7f6ddc197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:22.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.257+0000 7f6ddb7fe700 1 --2- 172.21.15.101:0/1212249758 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ddc101670 0x7f6ddc19f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:22.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.257+0000 7f6dda7fc700 1 -- 172.21.15.101:0/1212249758 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ddc101670 msgr2=0x7f6ddc19f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:22.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.257+0000 7f6dda7fc700 1 --2- 172.21.15.101:0/1212249758 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ddc101670 0x7f6ddc19f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:22.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.257+0000 7f6dda7fc700 1 -- 172.21.15.101:0/1212249758 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ddc0ffaf0 msgr2=0x7f6ddc197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:22.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.257+0000 7f6dda7fc700 1 --2- 172.21.15.101:0/1212249758 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ddc0ffaf0 0x7f6ddc197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:22.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.257+0000 7f6dda7fc700 1 -- 172.21.15.101:0/1212249758 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dcc0095f0 con 0x7f6ddc100ce0 2023-07-18T03:07:22.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.258+0000 7f6dda7fc700 1 --2- 172.21.15.101:0/1212249758 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6ddc100ce0 0x7f6ddc197fe0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6dd000e9a0 tx=0x7f6dd000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:22.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.258+0000 7f6dc3fff700 1 -- 172.21.15.101:0/1212249758 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd0009790 con 0x7f6ddc100ce0 2023-07-18T03:07:22.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.258+0000 7f6de17ff700 1 -- 172.21.15.101:0/1212249758 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ddc19fbd0 con 0x7f6ddc100ce0 2023-07-18T03:07:22.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.258+0000 7f6dc3fff700 1 -- 172.21.15.101:0/1212249758 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6dd000cbd0 con 0x7f6ddc100ce0 2023-07-18T03:07:22.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.258+0000 7f6dc3fff700 1 -- 172.21.15.101:0/1212249758 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd0018680 con 0x7f6ddc100ce0 2023-07-18T03:07:22.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.258+0000 7f6de17ff700 1 -- 172.21.15.101:0/1212249758 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ddc1a0130 con 0x7f6ddc100ce0 2023-07-18T03:07:22.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.259+0000 7f6de17ff700 1 -- 172.21.15.101:0/1212249758 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ddc1921d0 con 0x7f6ddc100ce0 2023-07-18T03:07:22.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.261+0000 7f6dc3fff700 1 -- 172.21.15.101:0/1212249758 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f6dd000cd40 con 0x7f6ddc100ce0 2023-07-18T03:07:22.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.262+0000 7f6dc3fff700 1 --2- 172.21.15.101:0/1212249758 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f6dc40a1150 0x7f6dc40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:22.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.262+0000 7f6dc3fff700 1 -- 172.21.15.101:0/1212249758 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f6dd0014070 con 0x7f6ddc100ce0 2023-07-18T03:07:22.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.262+0000 7f6ddaffd700 1 --2- 172.21.15.101:0/1212249758 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f6dc40a1150 0x7f6dc40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:22.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.263+0000 7f6ddaffd700 1 --2- 172.21.15.101:0/1212249758 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f6dc40a1150 0x7f6dc40a3600 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc00b2e0 tx=0x7f6dcc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:22.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.266+0000 7f6dc3fff700 1 -- 172.21.15.101:0/1212249758 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6dd0051790 con 0x7f6ddc100ce0 2023-07-18T03:07:22.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:21 smithi155 ceph-mon[128885]: from='client.15387 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:07:22.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:21 smithi155 ceph-mon[128885]: pgmap v350: 129 pgs: 129 active+clean; 646 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 6 op/s 2023-07-18T03:07:22.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:21 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/3604314416' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:07:22.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:21 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1594227083' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:07:22.444 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:21 smithi179 ceph-mon[129229]: from='client.15387 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:07:22.444 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:21 smithi179 ceph-mon[129229]: pgmap v350: 129 pgs: 129 active+clean; 646 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 6 op/s 2023-07-18T03:07:22.445 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:21 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/3604314416' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:07:22.445 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:21 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1594227083' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:07:22.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.466+0000 7f6de17ff700 1 -- 172.21.15.101:0/1212249758 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6ddc0611c0 con 0x7f6ddc100ce0 2023-07-18T03:07:22.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.467+0000 7f6dc3fff700 1 -- 172.21.15.101:0/1212249758 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6dd0096300 con 0x7f6ddc100ce0 2023-07-18T03:07:22.454 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 64 2023-07-18T03:07:22.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.469+0000 7f6de17ff700 1 -- 172.21.15.101:0/1212249758 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f6dc40a1150 msgr2=0x7f6dc40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:22.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.469+0000 7f6de17ff700 1 --2- 172.21.15.101:0/1212249758 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f6dc40a1150 0x7f6dc40a3600 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc00b2e0 tx=0x7f6dcc005fb0 comp rx=0 tx=0).stop 2023-07-18T03:07:22.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.469+0000 7f6de17ff700 1 -- 172.21.15.101:0/1212249758 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6ddc100ce0 msgr2=0x7f6ddc197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:22.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.469+0000 7f6de17ff700 1 --2- 172.21.15.101:0/1212249758 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6ddc100ce0 0x7f6ddc197fe0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6dd000e9a0 tx=0x7f6dd000ecb0 comp rx=0 tx=0).stop 2023-07-18T03:07:22.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.469+0000 7f6de17ff700 1 -- 172.21.15.101:0/1212249758 shutdown_connections 2023-07-18T03:07:22.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.469+0000 7f6de17ff700 1 --2- 172.21.15.101:0/1212249758 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f6dc40a1150 0x7f6dc40a3600 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:22.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.469+0000 7f6de17ff700 1 --2- 172.21.15.101:0/1212249758 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ddc0ffaf0 0x7f6ddc197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:22.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.469+0000 7f6de17ff700 1 --2- 172.21.15.101:0/1212249758 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ddc101670 0x7f6ddc19f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:22.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.469+0000 7f6de17ff700 1 --2- 172.21.15.101:0/1212249758 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6ddc100ce0 0x7f6ddc197fe0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:22.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.469+0000 7f6de17ff700 1 -- 172.21.15.101:0/1212249758 >> 172.21.15.101:0/1212249758 conn(0x7f6ddc0fb130 msgr2=0x7f6ddc1065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:22.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.469+0000 7f6de17ff700 1 -- 172.21.15.101:0/1212249758 shutdown_connections 2023-07-18T03:07:22.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.469+0000 7f6de17ff700 1 -- 172.21.15.101:0/1212249758 wait complete. 2023-07-18T03:07:22.474 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-18T03:07:22.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.704+0000 7fceb85ff700 1 -- 172.21.15.101:0/719984118 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fceb0074550 msgr2=0x7fceb00729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:22.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.704+0000 7fceb85ff700 1 --2- 172.21.15.101:0/719984118 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fceb0074550 0x7fceb00729a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fceac0098f0 tx=0x7fceac009c00 comp rx=0 tx=0).stop 2023-07-18T03:07:22.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.704+0000 7fceb85ff700 1 -- 172.21.15.101:0/719984118 shutdown_connections 2023-07-18T03:07:22.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.704+0000 7fceb85ff700 1 --2- 172.21.15.101:0/719984118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fceb01079b0 0x7fceb0109e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:22.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.704+0000 7fceb85ff700 1 --2- 172.21.15.101:0/719984118 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fceb0074550 0x7fceb00729a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:22.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.704+0000 7fceb85ff700 1 --2- 172.21.15.101:0/719984118 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fceb0072f70 0x7fceb00733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:22.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.704+0000 7fceb85ff700 1 -- 172.21.15.101:0/719984118 >> 172.21.15.101:0/719984118 conn(0x7fceb00fb130 msgr2=0x7fceb00fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:22.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.704+0000 7fceb85ff700 1 -- 172.21.15.101:0/719984118 shutdown_connections 2023-07-18T03:07:22.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.704+0000 7fceb85ff700 1 -- 172.21.15.101:0/719984118 wait complete. 2023-07-18T03:07:22.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.705+0000 7fceb85ff700 1 Processor -- start 2023-07-18T03:07:22.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.705+0000 7fceb85ff700 1 -- start start 2023-07-18T03:07:22.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.705+0000 7fceb85ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fceb0072f70 0x7fceb0197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:22.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.705+0000 7fceb85ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fceb0074550 0x7fceb0198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:22.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.705+0000 7fceb85ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fceb01079b0 0x7fceb019f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:22.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.705+0000 7fceb85ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb019f690 con 0x7fceb0074550 2023-07-18T03:07:22.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.705+0000 7fceb85ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb019f7d0 con 0x7fceb0072f70 2023-07-18T03:07:22.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.705+0000 7fceb85ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb019f960 con 0x7fceb01079b0 2023-07-18T03:07:22.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.706+0000 7fceb5b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fceb0074550 0x7fceb0198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:22.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.706+0000 7fceb639b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fceb0072f70 0x7fceb0197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:22.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.706+0000 7fceb5b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fceb0074550 0x7fceb0198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:36574/0 (socket says 172.21.15.101:36574) 2023-07-18T03:07:22.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.706+0000 7fceb5b9a700 1 -- 172.21.15.101:0/359616690 learned_addr learned my addr 172.21.15.101:0/359616690 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:22.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.706+0000 7fceb6b9c700 1 --2- 172.21.15.101:0/359616690 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fceb01079b0 0x7fceb019f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:22.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.706+0000 7fceb5b9a700 1 -- 172.21.15.101:0/359616690 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fceb01079b0 msgr2=0x7fceb019f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:22.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.706+0000 7fceb5b9a700 1 --2- 172.21.15.101:0/359616690 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fceb01079b0 0x7fceb019f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:22.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.706+0000 7fceb5b9a700 1 -- 172.21.15.101:0/359616690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fceb0072f70 msgr2=0x7fceb0197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:22.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.706+0000 7fceb5b9a700 1 --2- 172.21.15.101:0/359616690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fceb0072f70 0x7fceb0197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:22.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.706+0000 7fceb5b9a700 1 -- 172.21.15.101:0/359616690 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea0009520 con 0x7fceb0074550 2023-07-18T03:07:22.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.706+0000 7fceb639b700 1 --2- 172.21.15.101:0/359616690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fceb0072f70 0x7fceb0197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:07:22.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.706+0000 7fceb5b9a700 1 --2- 172.21.15.101:0/359616690 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fceb0074550 0x7fceb0198050 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fcea000ea50 tx=0x7fcea000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:22.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.707+0000 7fcea77fe700 1 -- 172.21.15.101:0/359616690 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea000cbd0 con 0x7fceb0074550 2023-07-18T03:07:22.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.707+0000 7fcea77fe700 1 -- 172.21.15.101:0/359616690 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcea000cd30 con 0x7fceb0074550 2023-07-18T03:07:22.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.707+0000 7fcea77fe700 1 -- 172.21.15.101:0/359616690 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea0018680 con 0x7fceb0074550 2023-07-18T03:07:22.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.707+0000 7fceb85ff700 1 -- 172.21.15.101:0/359616690 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcea8009d00 con 0x7fceb0074550 2023-07-18T03:07:22.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.707+0000 7fceb85ff700 1 -- 172.21.15.101:0/359616690 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fceac009520 con 0x7fceb0074550 2023-07-18T03:07:22.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.708+0000 7fceb85ff700 1 -- 172.21.15.101:0/359616690 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fceb0106060 con 0x7fceb0074550 2023-07-18T03:07:22.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.710+0000 7fcea77fe700 1 -- 172.21.15.101:0/359616690 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fcea0010430 con 0x7fceb0074550 2023-07-18T03:07:22.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.712+0000 7fcea77fe700 1 --2- 172.21.15.101:0/359616690 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fce9c0a11d0 0x7fce9c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:22.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.712+0000 7fcea77fe700 1 -- 172.21.15.101:0/359616690 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fcea0014070 con 0x7fceb0074550 2023-07-18T03:07:22.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.712+0000 7fceb639b700 1 --2- 172.21.15.101:0/359616690 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fce9c0a11d0 0x7fce9c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:22.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.713+0000 7fceb639b700 1 --2- 172.21.15.101:0/359616690 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fce9c0a11d0 0x7fce9c0a3680 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fceac009fd0 tx=0x7fceac019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:22.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.715+0000 7fcea77fe700 1 -- 172.21.15.101:0/359616690 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fcea0096470 con 0x7fceb0074550 2023-07-18T03:07:22.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.928+0000 7fceb85ff700 1 -- 172.21.15.101:0/359616690 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fceb002c850 con 0x7fceb0074550 2023-07-18T03:07:22.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.929+0000 7fcea77fe700 1 -- 172.21.15.101:0/359616690 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7fcea0099a90 con 0x7fceb0074550 2023-07-18T03:07:22.916 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 32 2023-07-18T03:07:22.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.931+0000 7fceb85ff700 1 -- 172.21.15.101:0/359616690 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fce9c0a11d0 msgr2=0x7fce9c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:22.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.931+0000 7fceb85ff700 1 --2- 172.21.15.101:0/359616690 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fce9c0a11d0 0x7fce9c0a3680 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fceac009fd0 tx=0x7fceac019040 comp rx=0 tx=0).stop 2023-07-18T03:07:22.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.931+0000 7fceb85ff700 1 -- 172.21.15.101:0/359616690 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fceb0074550 msgr2=0x7fceb0198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:22.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.931+0000 7fceb85ff700 1 --2- 172.21.15.101:0/359616690 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fceb0074550 0x7fceb0198050 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fcea000ea50 tx=0x7fcea000ed60 comp rx=0 tx=0).stop 2023-07-18T03:07:22.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.931+0000 7fceb85ff700 1 -- 172.21.15.101:0/359616690 shutdown_connections 2023-07-18T03:07:22.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.931+0000 7fceb85ff700 1 --2- 172.21.15.101:0/359616690 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fce9c0a11d0 0x7fce9c0a3680 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:22.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.931+0000 7fceb85ff700 1 --2- 172.21.15.101:0/359616690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fceb0072f70 0x7fceb0197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:22.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.931+0000 7fceb85ff700 1 --2- 172.21.15.101:0/359616690 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fceb01079b0 0x7fceb019f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:22.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.931+0000 7fceb85ff700 1 --2- 172.21.15.101:0/359616690 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fceb0074550 0x7fceb0198050 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:22.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.931+0000 7fceb85ff700 1 -- 172.21.15.101:0/359616690 >> 172.21.15.101:0/359616690 conn(0x7fceb00fb130 msgr2=0x7fceb00fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:22.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.931+0000 7fceb85ff700 1 -- 172.21.15.101:0/359616690 shutdown_connections 2023-07-18T03:07:22.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:22.931+0000 7fceb85ff700 1 -- 172.21.15.101:0/359616690 wait complete. 2023-07-18T03:07:22.936 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:07:23.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.159+0000 7f7875dff700 1 -- 172.21.15.101:0/2303022059 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f78700744f0 msgr2=0x7f7870072940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:23.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.159+0000 7f7875dff700 1 --2- 172.21.15.101:0/2303022059 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f78700744f0 0x7f7870072940 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7860009990 tx=0x7f7860009ca0 comp rx=0 tx=0).stop 2023-07-18T03:07:23.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.159+0000 7f7875dff700 1 -- 172.21.15.101:0/2303022059 shutdown_connections 2023-07-18T03:07:23.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.159+0000 7f7875dff700 1 --2- 172.21.15.101:0/2303022059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78701079a0 0x7f7870109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:23.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.159+0000 7f7875dff700 1 --2- 172.21.15.101:0/2303022059 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f78700744f0 0x7f7870072940 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:23.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.159+0000 7f7875dff700 1 --2- 172.21.15.101:0/2303022059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7870072f10 0x7f7870073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:23.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.159+0000 7f7875dff700 1 -- 172.21.15.101:0/2303022059 >> 172.21.15.101:0/2303022059 conn(0x7f78700fb110 msgr2=0x7f78700fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:23.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.159+0000 7f7875dff700 1 -- 172.21.15.101:0/2303022059 shutdown_connections 2023-07-18T03:07:23.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.159+0000 7f7875dff700 1 -- 172.21.15.101:0/2303022059 wait complete. 2023-07-18T03:07:23.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.160+0000 7f7875dff700 1 Processor -- start 2023-07-18T03:07:23.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.160+0000 7f7875dff700 1 -- start start 2023-07-18T03:07:23.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.160+0000 7f7875dff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7870072f10 0x7f7870197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:23.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.160+0000 7f7875dff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78700744f0 0x7f7870197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:23.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.160+0000 7f7875dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f78701079a0 0x7f787019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:23.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.161+0000 7f7875dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787019f5b0 con 0x7f78701079a0 2023-07-18T03:07:23.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.161+0000 7f7875dff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787019f6f0 con 0x7f7870072f10 2023-07-18T03:07:23.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.161+0000 7f7875dff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787019f880 con 0x7f78700744f0 2023-07-18T03:07:23.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.161+0000 7f786f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7870072f10 0x7f7870197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:23.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.161+0000 7f786f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7870072f10 0x7f7870197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60458/0 (socket says 172.21.15.101:60458) 2023-07-18T03:07:23.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.161+0000 7f786f7fe700 1 -- 172.21.15.101:0/814609035 learned_addr learned my addr 172.21.15.101:0/814609035 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:23.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.161+0000 7f7867fff700 1 --2- 172.21.15.101:0/814609035 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78700744f0 0x7f7870197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:23.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.161+0000 7f786ffff700 1 --2- 172.21.15.101:0/814609035 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f78701079a0 0x7f787019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:23.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.161+0000 7f786f7fe700 1 -- 172.21.15.101:0/814609035 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78700744f0 msgr2=0x7f7870197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:23.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.161+0000 7f786f7fe700 1 --2- 172.21.15.101:0/814609035 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78700744f0 0x7f7870197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:23.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.161+0000 7f786f7fe700 1 -- 172.21.15.101:0/814609035 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f78701079a0 msgr2=0x7f787019efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:23.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.161+0000 7f786f7fe700 1 --2- 172.21.15.101:0/814609035 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f78701079a0 0x7f787019efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:23.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.161+0000 7f786f7fe700 1 -- 172.21.15.101:0/814609035 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7868009c40 con 0x7f7870072f10 2023-07-18T03:07:23.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.161+0000 7f7867fff700 1 --2- 172.21.15.101:0/814609035 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78700744f0 0x7f7870197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:07:23.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.161+0000 7f786f7fe700 1 --2- 172.21.15.101:0/814609035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7870072f10 0x7f7870197a30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f786000b680 tx=0x7f7860005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:23.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.162+0000 7f786d7fa700 1 -- 172.21.15.101:0/814609035 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786001d070 con 0x7f7870072f10 2023-07-18T03:07:23.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.162+0000 7f786d7fa700 1 -- 172.21.15.101:0/814609035 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7860022470 con 0x7f7870072f10 2023-07-18T03:07:23.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.162+0000 7f7875dff700 1 -- 172.21.15.101:0/814609035 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78600095f0 con 0x7f7870072f10 2023-07-18T03:07:23.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.162+0000 7f786d7fa700 1 -- 172.21.15.101:0/814609035 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7860017650 con 0x7f7870072f10 2023-07-18T03:07:23.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.162+0000 7f7875dff700 1 -- 172.21.15.101:0/814609035 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f787019fdf0 con 0x7f7870072f10 2023-07-18T03:07:23.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.163+0000 7f7875dff700 1 -- 172.21.15.101:0/814609035 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78701060e0 con 0x7f7870072f10 2023-07-18T03:07:23.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.166+0000 7f786d7fa700 1 -- 172.21.15.101:0/814609035 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f78600225e0 con 0x7f7870072f10 2023-07-18T03:07:23.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.167+0000 7f786d7fa700 1 --2- 172.21.15.101:0/814609035 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f785c0a1220 0x7f785c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:23.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.168+0000 7f786d7fa700 1 -- 172.21.15.101:0/814609035 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f78600ce270 con 0x7f7870072f10 2023-07-18T03:07:23.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.168+0000 7f7867fff700 1 --2- 172.21.15.101:0/814609035 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f785c0a1220 0x7f785c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:23.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.168+0000 7f786d7fa700 1 -- 172.21.15.101:0/814609035 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f78600976e0 con 0x7f7870072f10 2023-07-18T03:07:23.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.168+0000 7f7867fff700 1 --2- 172.21.15.101:0/814609035 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f785c0a1220 0x7f785c0a36d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7858011490 tx=0x7f7858009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:23.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:22 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1783679905' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:07:23.232 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:22 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1212249758' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:07:23.232 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:22 smithi101 ceph-mon[126379]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:07:23.232 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:22 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/359616690' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-18T03:07:23.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:22 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1783679905' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:07:23.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:22 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1212249758' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:07:23.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:22 smithi155 ceph-mon[128885]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:07:23.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:22 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/359616690' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-18T03:07:23.432 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.445+0000 7f7875dff700 1 -- 172.21.15.101:0/814609035 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f787002c850 con 0x7f7870072f10 2023-07-18T03:07:23.433 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.446+0000 7f786d7fa700 1 -- 172.21.15.101:0/814609035 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10858 (secure 0 0 0) 0x7f786009ad00 con 0x7f7870072f10 2023-07-18T03:07:23.433 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:07:23.434 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/4192846769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":4192846769},{"type":"v1","addr":"172.21.15.101:6839","nonce":4192846769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6841/3824340175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6840","nonce":3824340175},{"type":"v1","addr":"172.21.15.101:6841","nonce":3824340175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:04:59.361185+0000","modified":"2023-07-18T03:05:31.985959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.155:6839/2055462483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2055462483},{"type":"v1","addr":"172.21.15.155:6839","nonce":2055462483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:07:23.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.449+0000 7f7875dff700 1 -- 172.21.15.101:0/814609035 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f785c0a1220 msgr2=0x7f785c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:23.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.449+0000 7f7875dff700 1 --2- 172.21.15.101:0/814609035 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f785c0a1220 0x7f785c0a36d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7858011490 tx=0x7f7858009260 comp rx=0 tx=0).stop 2023-07-18T03:07:23.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.449+0000 7f7875dff700 1 -- 172.21.15.101:0/814609035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7870072f10 msgr2=0x7f7870197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:23.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.449+0000 7f7875dff700 1 --2- 172.21.15.101:0/814609035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7870072f10 0x7f7870197a30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f786000b680 tx=0x7f7860005740 comp rx=0 tx=0).stop 2023-07-18T03:07:23.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.449+0000 7f7875dff700 1 -- 172.21.15.101:0/814609035 shutdown_connections 2023-07-18T03:07:23.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.449+0000 7f7875dff700 1 --2- 172.21.15.101:0/814609035 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f785c0a1220 0x7f785c0a36d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:23.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.449+0000 7f7875dff700 1 --2- 172.21.15.101:0/814609035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7870072f10 0x7f7870197a30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:23.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.449+0000 7f7875dff700 1 --2- 172.21.15.101:0/814609035 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f78701079a0 0x7f787019efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:23.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.449+0000 7f7875dff700 1 --2- 172.21.15.101:0/814609035 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78700744f0 0x7f7870197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:23.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.449+0000 7f7875dff700 1 -- 172.21.15.101:0/814609035 >> 172.21.15.101:0/814609035 conn(0x7f78700fb110 msgr2=0x7f78700fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:23.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.450+0000 7f7875dff700 1 -- 172.21.15.101:0/814609035 shutdown_connections 2023-07-18T03:07:23.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.450+0000 7f7875dff700 1 -- 172.21.15.101:0/814609035 wait complete. 2023-07-18T03:07:23.438 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 24 2023-07-18T03:07:23.444 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:22 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1783679905' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:07:23.444 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:22 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1212249758' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:07:23.444 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:22 smithi179 ceph-mon[129229]: from='mgr.14150 172.21.15.101:0/381704919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:07:23.444 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:22 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/359616690' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-18T03:07:23.456 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:07:23.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.674+0000 7f8f6f7ff700 1 -- 172.21.15.101:0/3424134265 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8f680ffb00 msgr2=0x7f8f680fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:23.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.674+0000 7f8f6f7ff700 1 --2- 172.21.15.101:0/3424134265 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8f680ffb00 0x7f8f680fff10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8f6400b0a0 tx=0x7f8f6400b3b0 comp rx=0 tx=0).stop 2023-07-18T03:07:23.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.674+0000 7f8f6f7ff700 1 -- 172.21.15.101:0/3424134265 shutdown_connections 2023-07-18T03:07:23.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.674+0000 7f8f6f7ff700 1 --2- 172.21.15.101:0/3424134265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8f68101680 0x7f8f68107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:23.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.674+0000 7f8f6f7ff700 1 --2- 172.21.15.101:0/3424134265 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8f680ffb00 0x7f8f680fff10 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:23.662 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.674+0000 7f8f6f7ff700 1 --2- 172.21.15.101:0/3424134265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f68100cf0 0x7f8f68101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:23.662 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.674+0000 7f8f6f7ff700 1 -- 172.21.15.101:0/3424134265 >> 172.21.15.101:0/3424134265 conn(0x7f8f680fb0e0 msgr2=0x7f8f680fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:23.662 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.674+0000 7f8f6f7ff700 1 -- 172.21.15.101:0/3424134265 shutdown_connections 2023-07-18T03:07:23.662 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.674+0000 7f8f6f7ff700 1 -- 172.21.15.101:0/3424134265 wait complete. 2023-07-18T03:07:23.663 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.675+0000 7f8f6f7ff700 1 Processor -- start 2023-07-18T03:07:23.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.675+0000 7f8f6f7ff700 1 -- start start 2023-07-18T03:07:23.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.676+0000 7f8f6f7ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f680ffb00 0x7f8f68197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:23.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.676+0000 7f8f6f7ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8f68100cf0 0x7f8f68198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:23.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.676+0000 7f8f6f7ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8f68101680 0x7f8f6819f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:23.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.676+0000 7f8f6f7ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6819f780 con 0x7f8f68101680 2023-07-18T03:07:23.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.676+0000 7f8f6f7ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6819f8f0 con 0x7f8f680ffb00 2023-07-18T03:07:23.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.676+0000 7f8f6f7ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6819fab0 con 0x7f8f68100cf0 2023-07-18T03:07:23.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.676+0000 7f8f6d59b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f680ffb00 0x7f8f68197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:23.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.676+0000 7f8f6cd9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8f68100cf0 0x7f8f68198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:23.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.676+0000 7f8f6d59b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f680ffb00 0x7f8f68197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60488/0 (socket says 172.21.15.101:60488) 2023-07-18T03:07:23.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.676+0000 7f8f6d59b700 1 -- 172.21.15.101:0/2599096139 learned_addr learned my addr 172.21.15.101:0/2599096139 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:23.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.676+0000 7f8f6dd9c700 1 --2- 172.21.15.101:0/2599096139 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8f68101680 0x7f8f6819f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:23.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.676+0000 7f8f6d59b700 1 -- 172.21.15.101:0/2599096139 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8f68100cf0 msgr2=0x7f8f68198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:23.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.676+0000 7f8f6d59b700 1 --2- 172.21.15.101:0/2599096139 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8f68100cf0 0x7f8f68198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:23.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.676+0000 7f8f6d59b700 1 -- 172.21.15.101:0/2599096139 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8f68101680 msgr2=0x7f8f6819f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:23.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.676+0000 7f8f6d59b700 1 --2- 172.21.15.101:0/2599096139 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8f68101680 0x7f8f6819f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:23.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.676+0000 7f8f6d59b700 1 -- 172.21.15.101:0/2599096139 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f58009520 con 0x7f8f680ffb00 2023-07-18T03:07:23.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.676+0000 7f8f6d59b700 1 --2- 172.21.15.101:0/2599096139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f680ffb00 0x7f8f68197c40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8f6400b070 tx=0x7f8f6400ba50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:23.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.677+0000 7f8f5e7fc700 1 -- 172.21.15.101:0/2599096139 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6400e040 con 0x7f8f680ffb00 2023-07-18T03:07:23.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.677+0000 7f8f5e7fc700 1 -- 172.21.15.101:0/2599096139 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f64003c00 con 0x7f8f680ffb00 2023-07-18T03:07:23.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.677+0000 7f8f6f7ff700 1 -- 172.21.15.101:0/2599096139 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f64009d00 con 0x7f8f680ffb00 2023-07-18T03:07:23.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.677+0000 7f8f5e7fc700 1 -- 172.21.15.101:0/2599096139 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f640052a0 con 0x7f8f680ffb00 2023-07-18T03:07:23.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.677+0000 7f8f6f7ff700 1 -- 172.21.15.101:0/2599096139 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f6819ffc0 con 0x7f8f680ffb00 2023-07-18T03:07:23.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.678+0000 7f8f6f7ff700 1 -- 172.21.15.101:0/2599096139 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f6810b320 con 0x7f8f680ffb00 2023-07-18T03:07:23.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.680+0000 7f8f5e7fc700 1 -- 172.21.15.101:0/2599096139 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f8f6401d480 con 0x7f8f680ffb00 2023-07-18T03:07:23.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.681+0000 7f8f5e7fc700 1 --2- 172.21.15.101:0/2599096139 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8f540a11a0 0x7f8f540a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:23.668 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.681+0000 7f8f5e7fc700 1 -- 172.21.15.101:0/2599096139 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f8f640cdd30 con 0x7f8f680ffb00 2023-07-18T03:07:23.668 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.681+0000 7f8f6cd9a700 1 --2- 172.21.15.101:0/2599096139 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8f540a11a0 0x7f8f540a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:23.668 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.682+0000 7f8f6cd9a700 1 --2- 172.21.15.101:0/2599096139 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8f540a11a0 0x7f8f540a3650 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8f58009df0 tx=0x7f8f58009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:23.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.683+0000 7f8f5e7fc700 1 -- 172.21.15.101:0/2599096139 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8f64097140 con 0x7f8f680ffb00 2023-07-18T03:07:23.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.904+0000 7f8f6f7ff700 1 -- 172.21.15.101:0/2599096139 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8f681a0570 con 0x7f8f680ffb00 2023-07-18T03:07:23.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.905+0000 7f8f5e7fc700 1 -- 172.21.15.101:0/2599096139 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24725 (secure 0 0 0) 0x7f8f6409a760 con 0x7f8f680ffb00 2023-07-18T03:07:23.893 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:07:23.893 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":91,"fsid":"b1782daa-2516-11ee-9b34-001a4aab830c","created":"2023-07-18T02:57:22.972483+0000","modified":"2023-07-18T03:06:24.250777+0000","last_up_change":"2023-07-18T03:04:11.429530+0000","last_in_change":"2023-07-18T03:03:54.158340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:00:43.353489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:04:57.771616+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-18T03:04:58.769590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-18T03:05:02.358830+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc76b5d1-38a0-4f1a-9335-756bfc40a62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6803","nonce":3610719929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6805","nonce":3610719929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6809","nonce":3610719929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6807","nonce":3610719929}]},"public_addr":"172.21.15.101:6803/3610719929","cluster_addr":"172.21.15.101:6805/3610719929","heartbeat_back_addr":"172.21.15.101:6809/3610719929","heartbeat_front_addr":"172.21.15.101:6807/3610719929","state":["exists","up"]},{"osd":1,"uuid":"2a299573-0b34-4af9-aaae-b523e5058ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6801","nonce":1030229579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6803","nonce":1030229579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6807","nonce":1030229579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6805","nonce":1030229579}]},"public_addr":"172.21.15.155:6801/1030229579","cluster_addr":"172.21.15.155:6803/1030229579","heartbeat_back_addr":"172.21.15.155:6807/1030229579","heartbeat_front_addr":"172.21.15.155:6805/1030229579","state":["exists","up"]},{"osd":2,"uuid":"123b3749-c636-48fa-9e33-6229b5575dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6801","nonce":1349487507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6803","nonce":1349487507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6807","nonce":1349487507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6805","nonce":1349487507}]},"public_addr":"172.21.15.179:6801/1349487507","cluster_addr":"172.21.15.179:6803/1349487507","heartbeat_back_addr":"172.21.15.179:6807/1349487507","heartbeat_front_addr":"172.21.15.179:6805/1349487507","state":["exists","up"]},{"osd":3,"uuid":"4b0c409d-2954-4127-833a-b196bf226d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6811","nonce":4226577463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6813","nonce":4226577463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6817","nonce":4226577463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6815","nonce":4226577463}]},"public_addr":"172.21.15.101:6811/4226577463","cluster_addr":"172.21.15.101:6813/4226577463","heartbeat_back_addr":"172.21.15.101:6817/4226577463","heartbeat_front_addr":"172.21.15.101:6815/4226577463","state":["exists","up"]},{"osd":4,"uuid":"2137f2df-984e-45ce-baf6-b4ac12c7e3c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6809","nonce":1951135805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6811","nonce":1951135805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6815","nonce":1951135805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6813","nonce":1951135805}]},"public_addr":"172.21.15.155:6809/1951135805","cluster_addr":"172.21.15.155:6811/1951135805","heartbeat_back_addr":"172.21.15.155:6815/1951135805","heartbeat_front_addr":"172.21.15.155:6813/1951135805","state":["exists","up"]},{"osd":5,"uuid":"1020bfc5-8c2f-482e-9588-66c7f98e46e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6809","nonce":2746037021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6811","nonce":2746037021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6815","nonce":2746037021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6813","nonce":2746037021}]},"public_addr":"172.21.15.179:6809/2746037021","cluster_addr":"172.21.15.179:6811/2746037021","heartbeat_back_addr":"172.21.15.179:6815/2746037021","heartbeat_front_addr":"172.21.15.179:6813/2746037021","state":["exists","up"]},{"osd":6,"uuid":"2cd99260-5c36-4603-9338-cf697970ce58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6819","nonce":958044931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6821","nonce":958044931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6825","nonce":958044931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6823","nonce":958044931}]},"public_addr":"172.21.15.101:6819/958044931","cluster_addr":"172.21.15.101:6821/958044931","heartbeat_back_addr":"172.21.15.101:6825/958044931","heartbeat_front_addr":"172.21.15.101:6823/958044931","state":["exists","up"]},{"osd":7,"uuid":"ed23fd0a-678f-4e15-b42c-cfe873315714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6817","nonce":3256322291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6819","nonce":3256322291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6823","nonce":3256322291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6821","nonce":3256322291}]},"public_addr":"172.21.15.155:6817/3256322291","cluster_addr":"172.21.15.155:6819/3256322291","heartbeat_back_addr":"172.21.15.155:6823/3256322291","heartbeat_front_addr":"172.21.15.155:6821/3256322291","state":["exists","up"]},{"osd":8,"uuid":"eca3d8e6-7baf-47a5-b991-328001c7ebef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6817","nonce":3520612559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6819","nonce":3520612559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6823","nonce":3520612559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6821","nonce":3520612559}]},"public_addr":"172.21.15.179:6817/3520612559","cluster_addr":"172.21.15.179:6819/3520612559","heartbeat_back_addr":"172.21.15.179:6823/3520612559","heartbeat_front_addr":"172.21.15.179:6821/3520612559","state":["exists","up"]},{"osd":9,"uuid":"465a7ffb-67e4-49ef-812c-1b96feacfebf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6827","nonce":2166787667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6829","nonce":2166787667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6833","nonce":2166787667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6831","nonce":2166787667}]},"public_addr":"172.21.15.101:6827/2166787667","cluster_addr":"172.21.15.101:6829/2166787667","heartbeat_back_addr":"172.21.15.101:6833/2166787667","heartbeat_front_addr":"172.21.15.101:6831/2166787667","state":["exists","up"]},{"osd":10,"uuid":"1897a027-f25d-434f-8df5-5fc2bd866578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6825","nonce":3682446349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6827","nonce":3682446349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6831","nonce":3682446349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6829","nonce":3682446349}]},"public_addr":"172.21.15.155:6825/3682446349","cluster_addr":"172.21.15.155:6827/3682446349","heartbeat_back_addr":"172.21.15.155:6831/3682446349","heartbeat_front_addr":"172.21.15.155:6829/3682446349","state":["exists","up"]},{"osd":11,"uuid":"b62effaf-45e2-4585-b8d0-db448f3474ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6825","nonce":2602640858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6827","nonce":2602640858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6831","nonce":2602640858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6829","nonce":2602640858}]},"public_addr":"172.21.15.179:6825/2602640858","cluster_addr":"172.21.15.179:6827/2602640858","heartbeat_back_addr":"172.21.15.179:6831/2602640858","heartbeat_front_addr":"172.21.15.179:6829/2602640858","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T02:59:56.074519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:19.358142+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:41.128966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:08.227054+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:31.442592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:52.751685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:19.251054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:40.698241+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:01.339150+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:27.955910+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:49.136980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:04:10.084982+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.101:6800/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:6801/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/604228291":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/4055187849":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/2844574324":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/1603248215":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6801/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6800/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/3307578182":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/373189792":"2023-07-19T02:57:56.243200+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:07:23.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.908+0000 7f8f6f7ff700 1 -- 172.21.15.101:0/2599096139 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8f540a11a0 msgr2=0x7f8f540a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:23.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.908+0000 7f8f6f7ff700 1 --2- 172.21.15.101:0/2599096139 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8f540a11a0 0x7f8f540a3650 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8f58009df0 tx=0x7f8f58009260 comp rx=0 tx=0).stop 2023-07-18T03:07:23.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.908+0000 7f8f6f7ff700 1 -- 172.21.15.101:0/2599096139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f680ffb00 msgr2=0x7f8f68197c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:23.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.908+0000 7f8f6f7ff700 1 --2- 172.21.15.101:0/2599096139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f680ffb00 0x7f8f68197c40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8f6400b070 tx=0x7f8f6400ba50 comp rx=0 tx=0).stop 2023-07-18T03:07:23.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.908+0000 7f8f6f7ff700 1 -- 172.21.15.101:0/2599096139 shutdown_connections 2023-07-18T03:07:23.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.908+0000 7f8f6f7ff700 1 --2- 172.21.15.101:0/2599096139 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8f540a11a0 0x7f8f540a3650 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:23.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.908+0000 7f8f6f7ff700 1 --2- 172.21.15.101:0/2599096139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f680ffb00 0x7f8f68197c40 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:23.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.908+0000 7f8f6f7ff700 1 --2- 172.21.15.101:0/2599096139 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8f68101680 0x7f8f6819f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:23.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.908+0000 7f8f6f7ff700 1 --2- 172.21.15.101:0/2599096139 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8f68100cf0 0x7f8f68198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:23.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.908+0000 7f8f6f7ff700 1 -- 172.21.15.101:0/2599096139 >> 172.21.15.101:0/2599096139 conn(0x7f8f680fb0e0 msgr2=0x7f8f68106570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:23.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.908+0000 7f8f6f7ff700 1 -- 172.21.15.101:0/2599096139 shutdown_connections 2023-07-18T03:07:23.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:23.908+0000 7f8f6f7ff700 1 -- 172.21.15.101:0/2599096139 wait complete. 2023-07-18T03:07:23.915 DEBUG:tasks.fwd_scrub:Yielding 2023-07-18T03:07:23.915 INFO:teuthology.run_tasks:Running task exec... 2023-07-18T03:07:23.926 INFO:teuthology.task.exec:Executing custom commands... 2023-07-18T03:07:23.926 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi101.front.sepia.ceph.com 2023-07-18T03:07:23.927 DEBUG:teuthology.orchestra.run.smithi101:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2023-07-18T03:07:23.927 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2023-07-18T03:07:23.927 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-18T03:07:23.928 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-18T03:07:24.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.161+0000 7f29325ff700 1 -- 172.21.15.101:0/2648746409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f292c0ffae0 msgr2=0x7f292c0ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:24.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.161+0000 7f29325ff700 1 --2- 172.21.15.101:0/2648746409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f292c0ffae0 0x7f292c0ffef0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f291c009990 tx=0x7f291c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.161+0000 7f29325ff700 1 -- 172.21.15.101:0/2648746409 shutdown_connections 2023-07-18T03:07:24.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.161+0000 7f29325ff700 1 --2- 172.21.15.101:0/2648746409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f292c101660 0x7f292c107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.161+0000 7f29325ff700 1 --2- 172.21.15.101:0/2648746409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f292c0ffae0 0x7f292c0ffef0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.161+0000 7f29325ff700 1 --2- 172.21.15.101:0/2648746409 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f292c100cd0 0x7f292c101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.161+0000 7f29325ff700 1 -- 172.21.15.101:0/2648746409 >> 172.21.15.101:0/2648746409 conn(0x7f292c0fb0e0 msgr2=0x7f292c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:24.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.162+0000 7f29325ff700 1 -- 172.21.15.101:0/2648746409 shutdown_connections 2023-07-18T03:07:24.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.162+0000 7f29325ff700 1 -- 172.21.15.101:0/2648746409 wait complete. 2023-07-18T03:07:24.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.163+0000 7f29325ff700 1 Processor -- start 2023-07-18T03:07:24.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.163+0000 7f29325ff700 1 -- start start 2023-07-18T03:07:24.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.163+0000 7f29325ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f292c0ffae0 0x7f292c197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:24.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.163+0000 7f95b13ff700 1 -- 172.21.15.101:0/1697025838 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95ac100d00 msgr2=0x7f95ac101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:24.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.163+0000 7f95b13ff700 1 --2- 172.21.15.101:0/1697025838 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95ac100d00 0x7f95ac101150 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f95a0009990 tx=0x7f95a0009ca0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.163+0000 7f29325ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f292c100cd0 0x7f292c197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:24.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.163+0000 7f29325ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f292c101660 0x7f292c19f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:24.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.163+0000 7f29325ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292c19f600 con 0x7f292c101660 2023-07-18T03:07:24.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.163+0000 7f29325ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292c19f740 con 0x7f292c100cd0 2023-07-18T03:07:24.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.163+0000 7f29325ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292c19f8d0 con 0x7f292c0ffae0 2023-07-18T03:07:24.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.163+0000 7f95b13ff700 1 -- 172.21.15.101:0/1697025838 shutdown_connections 2023-07-18T03:07:24.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.163+0000 7f95b13ff700 1 --2- 172.21.15.101:0/1697025838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95ac0ffb10 0x7f95ac0fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.163+0000 7f95b13ff700 1 --2- 172.21.15.101:0/1697025838 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f95ac101690 0x7f95ac107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.156 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.163+0000 7f95b13ff700 1 --2- 172.21.15.101:0/1697025838 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95ac100d00 0x7f95ac101150 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.156 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.163+0000 7f95b13ff700 1 -- 172.21.15.101:0/1697025838 >> 172.21.15.101:0/1697025838 conn(0x7f95ac0fb130 msgr2=0x7f95ac0fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:24.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.164+0000 7f292bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f292c0ffae0 0x7f292c197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:24.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.164+0000 7f95b13ff700 1 -- 172.21.15.101:0/1697025838 shutdown_connections 2023-07-18T03:07:24.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.164+0000 7f292bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f292c0ffae0 0x7f292c197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:48416/0 (socket says 172.21.15.101:48416) 2023-07-18T03:07:24.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.164+0000 7f292bfff700 1 -- 172.21.15.101:0/1590296860 learned_addr learned my addr 172.21.15.101:0/1590296860 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:24.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.164+0000 7f2930b9c700 1 --2- 172.21.15.101:0/1590296860 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f292c101660 0x7f292c19f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:24.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.164+0000 7f95b13ff700 1 -- 172.21.15.101:0/1697025838 wait complete. 2023-07-18T03:07:24.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.164+0000 7f292b7fe700 1 --2- 172.21.15.101:0/1590296860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f292c100cd0 0x7f292c197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:24.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.164+0000 7f292bfff700 1 -- 172.21.15.101:0/1590296860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f292c100cd0 msgr2=0x7f292c197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:24.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.164+0000 7f292bfff700 1 --2- 172.21.15.101:0/1590296860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f292c100cd0 0x7f292c197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.164+0000 7f292bfff700 1 -- 172.21.15.101:0/1590296860 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f292c101660 msgr2=0x7f292c19f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:24.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.164+0000 7f292bfff700 1 --2- 172.21.15.101:0/1590296860 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f292c101660 0x7f292c19f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.164+0000 7f292bfff700 1 -- 172.21.15.101:0/1590296860 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2924009c40 con 0x7f292c0ffae0 2023-07-18T03:07:24.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.164+0000 7f292bfff700 1 --2- 172.21.15.101:0/1590296860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f292c0ffae0 0x7f292c197a80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f291c004710 tx=0x7f291c009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:24.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.165+0000 7f29297fa700 1 -- 172.21.15.101:0/1590296860 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291c01d070 con 0x7f292c0ffae0 2023-07-18T03:07:24.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.165+0000 7f29297fa700 1 -- 172.21.15.101:0/1590296860 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f291c021410 con 0x7f292c0ffae0 2023-07-18T03:07:24.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.165+0000 7f29297fa700 1 -- 172.21.15.101:0/1590296860 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291c017650 con 0x7f292c0ffae0 2023-07-18T03:07:24.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.165+0000 7f29325ff700 1 -- 172.21.15.101:0/1590296860 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f291c0095f0 con 0x7f292c0ffae0 2023-07-18T03:07:24.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.165+0000 7f29325ff700 1 -- 172.21.15.101:0/1590296860 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f292c102190 con 0x7f292c0ffae0 2023-07-18T03:07:24.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.165+0000 7f95b13ff700 1 Processor -- start 2023-07-18T03:07:24.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.165+0000 7f95b13ff700 1 -- start start 2023-07-18T03:07:24.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.166+0000 7f95b13ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f95ac0ffb10 0x7f95ac072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:24.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.166+0000 7f95b13ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95ac100d00 0x7f95ac072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:24.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.166+0000 7f95b13ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95ac101690 0x7f95ac073380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:24.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.166+0000 7f95b13ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac19f8c0 con 0x7f95ac0ffb10 2023-07-18T03:07:24.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.166+0000 7f95b13ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac19fa00 con 0x7f95ac100d00 2023-07-18T03:07:24.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.166+0000 7f95b13ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac19fb40 con 0x7f95ac101690 2023-07-18T03:07:24.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.166+0000 7f95ab7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95ac101690 0x7f95ac073380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:24.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.166+0000 7f95aaffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f95ac0ffb10 0x7f95ac072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:24.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.166+0000 7f95ab7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95ac101690 0x7f95ac073380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:48422/0 (socket says 172.21.15.101:48422) 2023-07-18T03:07:24.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.166+0000 7f95ab7fe700 1 -- 172.21.15.101:0/1157994141 learned_addr learned my addr 172.21.15.101:0/1157994141 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:24.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.166+0000 7f95aa7fc700 1 --2- 172.21.15.101:0/1157994141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95ac100d00 0x7f95ac072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:24.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.167+0000 7f95ab7fe700 1 -- 172.21.15.101:0/1157994141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95ac100d00 msgr2=0x7f95ac072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:24.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.167+0000 7f95ab7fe700 1 --2- 172.21.15.101:0/1157994141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95ac100d00 0x7f95ac072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.167+0000 7f95ab7fe700 1 -- 172.21.15.101:0/1157994141 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f95ac0ffb10 msgr2=0x7f95ac072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:24.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.167+0000 7f95ab7fe700 1 --2- 172.21.15.101:0/1157994141 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f95ac0ffb10 0x7f95ac072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.167+0000 7f95ab7fe700 1 -- 172.21.15.101:0/1157994141 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a00095f0 con 0x7f95ac101690 2023-07-18T03:07:24.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.167+0000 7f95ab7fe700 1 --2- 172.21.15.101:0/1157994141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95ac101690 0x7f95ac073380 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f95a4007ee0 tx=0x7f95a400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:24.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.167+0000 7f9593fff700 1 -- 172.21.15.101:0/1157994141 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a4017070 con 0x7f95ac101690 2023-07-18T03:07:24.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.167+0000 7f95b13ff700 1 -- 172.21.15.101:0/1157994141 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95ac19fdd0 con 0x7f95ac101690 2023-07-18T03:07:24.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.167+0000 7f9593fff700 1 -- 172.21.15.101:0/1157994141 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95a400f040 con 0x7f95ac101690 2023-07-18T03:07:24.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.167+0000 7f9593fff700 1 -- 172.21.15.101:0/1157994141 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a4005580 con 0x7f95ac101690 2023-07-18T03:07:24.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.168+0000 7f95b13ff700 1 -- 172.21.15.101:0/1157994141 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95ac1a0280 con 0x7f95ac101690 2023-07-18T03:07:24.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.169+0000 7f95b13ff700 1 -- 172.21.15.101:0/1157994141 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95ac04e2f0 con 0x7f95ac101690 2023-07-18T03:07:24.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.169+0000 7f29297fa700 1 -- 172.21.15.101:0/1590296860 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f291c021580 con 0x7f292c0ffae0 2023-07-18T03:07:24.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.170+0000 7f29297fa700 1 --2- 172.21.15.101:0/1590296860 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f29140a11d0 0x7f29140a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:24.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.171+0000 7f29297fa700 1 -- 172.21.15.101:0/1590296860 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f291c0cdf20 con 0x7f292c0ffae0 2023-07-18T03:07:24.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.171+0000 7f292b7fe700 1 --2- 172.21.15.101:0/1590296860 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f29140a11d0 0x7f29140a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:24.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.171+0000 7f2945f1fb80 1 Processor -- start 2023-07-18T03:07:24.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.171+0000 7f2945f1fb80 1 -- start start 2023-07-18T03:07:24.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.172+0000 7f292b7fe700 1 --2- 172.21.15.101:0/1590296860 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f29140a11d0 0x7f29140a3680 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2920004170 tx=0x7f2920018040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:24.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.172+0000 7f2945f1fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae4340550 0x555ae4340960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:24.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.172+0000 7f2945f1fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae4340ea0 0x555ae4341760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:24.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.172+0000 7f2945f1fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ae4341cf0 0x555ae4342160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:24.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.172+0000 7f2945f1fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ae4245d30 con 0x555ae4341cf0 2023-07-18T03:07:24.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.172+0000 7f2945f1fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ae4298ac0 con 0x555ae4340ea0 2023-07-18T03:07:24.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.172+0000 7f2945f1fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ae4291f50 con 0x555ae4340550 2023-07-18T03:07:24.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.172+0000 7f292b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae4340ea0 0x555ae4341760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:24.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.172+0000 7f292bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae4340550 0x555ae4340960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:24.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.172+0000 7f292b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae4340ea0 0x555ae4341760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60540/0 (socket says 172.21.15.101:60540) 2023-07-18T03:07:24.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.172+0000 7f292bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae4340550 0x555ae4340960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:48436/0 (socket says 172.21.15.101:48436) 2023-07-18T03:07:24.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.172+0000 7f292b7fe700 1 -- 172.21.15.101:0/1156990194 learned_addr learned my addr 172.21.15.101:0/1156990194 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:24.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.172+0000 7f2930b9c700 1 --2- 172.21.15.101:0/1156990194 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ae4341cf0 0x555ae4342160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:24.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.172+0000 7f9593fff700 1 -- 172.21.15.101:0/1157994141 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f95a40057a0 con 0x7f95ac101690 2023-07-18T03:07:24.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.172+0000 7f292b7fe700 1 -- 172.21.15.101:0/1156990194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae4340550 msgr2=0x555ae4340960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:24.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.172+0000 7f292b7fe700 1 --2- 172.21.15.101:0/1156990194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae4340550 0x555ae4340960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.172+0000 7f292b7fe700 1 -- 172.21.15.101:0/1156990194 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ae4341cf0 msgr2=0x555ae4342160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:24.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.172+0000 7f292b7fe700 1 --2- 172.21.15.101:0/1156990194 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ae4341cf0 0x555ae4342160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.172+0000 7f292b7fe700 1 -- 172.21.15.101:0/1156990194 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ae40bfe60 con 0x555ae4340ea0 2023-07-18T03:07:24.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.173+0000 7f292b7fe700 1 --2- 172.21.15.101:0/1156990194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae4340ea0 0x555ae4341760 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2920015c50 tx=0x7f292001c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a3b93fc91f5663c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:24.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.173+0000 7f29127fc700 1 -- 172.21.15.101:0/1156990194 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2920028070 con 0x555ae4340ea0 2023-07-18T03:07:24.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.173+0000 7f29127fc700 1 -- 172.21.15.101:0/1156990194 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2920004830 con 0x555ae4340ea0 2023-07-18T03:07:24.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.173+0000 7f29127fc700 1 -- 172.21.15.101:0/1156990194 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292001b040 con 0x555ae4340ea0 2023-07-18T03:07:24.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.174+0000 7f9593fff700 1 --2- 172.21.15.101:0/1157994141 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f95940a11d0 0x7f95940a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:24.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.174+0000 7f2945f1fb80 1 -- 172.21.15.101:0/1156990194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae4340ea0 msgr2=0x555ae4341760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:24.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.174+0000 7f2945f1fb80 1 --2- 172.21.15.101:0/1156990194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae4340ea0 0x555ae4341760 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2920015c50 tx=0x7f292001c000 comp rx=0 tx=0).stop 2023-07-18T03:07:24.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.174+0000 7f2945f1fb80 1 -- 172.21.15.101:0/1156990194 shutdown_connections 2023-07-18T03:07:24.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.174+0000 7f2945f1fb80 1 --2- 172.21.15.101:0/1156990194 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ae4341cf0 0x555ae4342160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.174+0000 7f2945f1fb80 1 --2- 172.21.15.101:0/1156990194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae4340550 0x555ae4340960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.174+0000 7f2945f1fb80 1 --2- 172.21.15.101:0/1156990194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae4340ea0 0x555ae4341760 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.174+0000 7f2945f1fb80 1 -- 172.21.15.101:0/1156990194 >> 172.21.15.101:0/1156990194 conn(0x555ae4335e20 msgr2=0x555ae433b250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:24.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.174+0000 7f9593fff700 1 -- 172.21.15.101:0/1157994141 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f95a40cc330 con 0x7f95ac101690 2023-07-18T03:07:24.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.174+0000 7f95aaffd700 1 --2- 172.21.15.101:0/1157994141 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f95940a11d0 0x7f95940a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:24.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.174+0000 7f2945f1fb80 1 -- 172.21.15.101:0/1156990194 shutdown_connections 2023-07-18T03:07:24.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.174+0000 7f2945f1fb80 1 -- 172.21.15.101:0/1156990194 wait complete. 2023-07-18T03:07:24.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.174+0000 7f2945f1fb80 1 -- start start 2023-07-18T03:07:24.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.174+0000 7f95aaffd700 1 --2- 172.21.15.101:0/1157994141 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f95940a11d0 0x7f95940a3680 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f95ac074210 tx=0x7f959c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:24.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.175+0000 7f2945f1fb80 1 Processor -- start 2023-07-18T03:07:24.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.175+0000 7f2945f1fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ae4340550 0x555ae434a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:24.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.175+0000 7f2945f1fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae4340ea0 0x555ae434a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:24.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.175+0000 7f2945f1fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae4341cf0 0x555ae43516c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:24.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.175+0000 7f2945f1fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ae4231b40 con 0x555ae4340550 2023-07-18T03:07:24.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.175+0000 7f2945f1fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ae4231840 con 0x555ae4340ea0 2023-07-18T03:07:24.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.175+0000 7f2945f1fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ae41159a0 con 0x555ae4341cf0 2023-07-18T03:07:24.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.175+0000 7f2945f1fb80 10 client.0 _ll_register_callbacks cb 0x555ae422f7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:07:24.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.175+0000 7f292b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae4340ea0 0x555ae434a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:24.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.175+0000 7f292bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ae4340550 0x555ae434a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:24.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.175+0000 7f292b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae4340ea0 0x555ae434a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60556/0 (socket says 172.21.15.101:60556) 2023-07-18T03:07:24.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.175+0000 7f292b7fe700 1 -- 172.21.15.101:0/373117353 learned_addr learned my addr 172.21.15.101:0/373117353 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:24.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.175+0000 7f2930b9c700 1 --2- 172.21.15.101:0/373117353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae4341cf0 0x555ae43516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:24.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.176+0000 7f292b7fe700 1 -- 172.21.15.101:0/373117353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae4341cf0 msgr2=0x555ae43516c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:24.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.176+0000 7f292b7fe700 1 --2- 172.21.15.101:0/373117353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae4341cf0 0x555ae43516c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.176+0000 7f292b7fe700 1 -- 172.21.15.101:0/373117353 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ae4340550 msgr2=0x555ae434a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:24.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.176+0000 7f292b7fe700 1 --2- 172.21.15.101:0/373117353 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ae4340550 0x555ae434a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.176+0000 7f292b7fe700 1 -- 172.21.15.101:0/373117353 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2920009d40 con 0x555ae4340ea0 2023-07-18T03:07:24.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.176+0000 7f292b7fe700 1 --2- 172.21.15.101:0/373117353 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae4340ea0 0x555ae434a680 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2920004c20 tx=0x7f2920004c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac04da0cb4fe2ac0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:24.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.176+0000 7f290b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:07:24.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.176+0000 7f290b7fe700 1 -- 172.21.15.101:0/373117353 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292001b070 con 0x555ae4340ea0 2023-07-18T03:07:24.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.176+0000 7f2945f1fb80 1 -- 172.21.15.101:0/373117353 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555ae41eb570 con 0x555ae4340ea0 2023-07-18T03:07:24.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.176+0000 7f290b7fe700 1 -- 172.21.15.101:0/373117353 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f29200054a0 con 0x555ae4340ea0 2023-07-18T03:07:24.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.178+0000 7f290b7fe700 1 -- 172.21.15.101:0/373117353 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2920028050 con 0x555ae4340ea0 2023-07-18T03:07:24.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.178+0000 7f2945f1fb80 10 client.25472 fetch_fsmap learned FSMap version 24 2023-07-18T03:07:24.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.178+0000 7f2945f1fb80 1 -- 172.21.15.101:0/373117353 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x555ae40a8ce0 con 0x555ae4340ea0 2023-07-18T03:07:24.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.178+0000 7f9593fff700 1 -- 172.21.15.101:0/1157994141 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f95a40957b0 con 0x7f95ac101690 2023-07-18T03:07:24.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.179+0000 7f290b7fe700 1 -- 172.21.15.101:0/373117353 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555ae41eb570 con 0x555ae4340ea0 2023-07-18T03:07:24.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.179+0000 7f290b7fe700 1 -- 172.21.15.101:0/373117353 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2920021410 con 0x555ae4340ea0 2023-07-18T03:07:24.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.179+0000 7f2945f1fb80 10 client.25472 fetch_fsmap finished waiting for FSMap version 24 2023-07-18T03:07:24.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.179+0000 7f2945f1fb80 10 client.25472 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:07:24.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.179+0000 7f2945f1fb80 20 client.25472 populate_metadata read hostname 'smithi101' 2023-07-18T03:07:24.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.179+0000 7f2945f1fb80 1 --2- 172.21.15.101:0/373117353 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x555ae4355de0 0x555ae43581f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:24.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.179+0000 7f2945f1fb80 4 client.25472 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:07:24.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.179+0000 7f2945f1fb80 1 -- 172.21.15.101:0/373117353 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555ae428af30 con 0x555ae4355de0 2023-07-18T03:07:24.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.179+0000 7f292bfff700 1 --2- 172.21.15.101:0/373117353 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x555ae4355de0 0x555ae43581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:24.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.180+0000 7f292bfff700 1 --2- 172.21.15.101:0/373117353 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x555ae4355de0 0x555ae43581f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72fd3394ce97e644 server_cookie=e0123c571a29ae20 in_seq=0 out_seq=0 2023-07-18T03:07:24.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.180+0000 7f290b7fe700 10 client.25472 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:07:24.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.180+0000 7f290b7fe700 1 -- 172.21.15.101:0/373117353 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555ae428af30 con 0x555ae4355de0 2023-07-18T03:07:24.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.180+0000 7f290b7fe700 10 client.25472 handle_command_reply: tid=0 2023-07-18T03:07:24.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.180+0000 7f2945f1fb80 1 client.25472 shutdown 2023-07-18T03:07:24.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.180+0000 7f2945f1fb80 1 -- 172.21.15.101:0/373117353 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae4340ea0 msgr2=0x555ae434a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:24.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.180+0000 7f2945f1fb80 1 --2- 172.21.15.101:0/373117353 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae4340ea0 0x555ae434a680 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2920004c20 tx=0x7f2920004c50 comp rx=0 tx=0).stop 2023-07-18T03:07:24.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.181+0000 7f2945f1fb80 1 -- 172.21.15.101:0/373117353 shutdown_connections 2023-07-18T03:07:24.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.181+0000 7f2945f1fb80 1 --2- 172.21.15.101:0/373117353 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x555ae4355de0 0x555ae43581f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.181+0000 7f2945f1fb80 1 --2- 172.21.15.101:0/373117353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae4341cf0 0x555ae43516c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.181+0000 7f290b7fe700 0 client.25472 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:07:24.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.181+0000 7f2945f1fb80 1 --2- 172.21.15.101:0/373117353 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ae4340550 0x555ae434a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.181+0000 7f2945f1fb80 1 --2- 172.21.15.101:0/373117353 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae4340ea0 0x555ae434a680 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.181+0000 7f2945f1fb80 1 -- 172.21.15.101:0/373117353 >> 172.21.15.101:0/373117353 conn(0x555ae4335e20 msgr2=0x555ae433a930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:24.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.181+0000 7f2945f1fb80 1 -- 172.21.15.101:0/373117353 shutdown_connections 2023-07-18T03:07:24.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.181+0000 7f2945f1fb80 1 -- 172.21.15.101:0/373117353 wait complete. 2023-07-18T03:07:24.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.181+0000 7f2945f1fb80 20 client.25472 trim_cache size 0 max 16384 2023-07-18T03:07:24.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.215+0000 7f2945f1fb80 1 Processor -- start 2023-07-18T03:07:24.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.215+0000 7f2945f1fb80 1 -- start start 2023-07-18T03:07:24.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.216+0000 7f2945f1fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae4340550 0x555ae433a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:24.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.216+0000 7f2945f1fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ae4340ea0 0x555ae4351650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:24.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.216+0000 7f2945f1fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae4341cf0 0x555ae4344bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:24.205 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.216+0000 7f2945f1fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ae433cd70 con 0x555ae4340ea0 2023-07-18T03:07:24.205 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.216+0000 7f2945f1fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ae4347e50 con 0x555ae4340550 2023-07-18T03:07:24.205 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.216+0000 7f2945f1fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ae4355ab0 con 0x555ae4341cf0 2023-07-18T03:07:24.205 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.216+0000 7f292b7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ae4340ea0 0x555ae4351650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:24.205 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.216+0000 7f2930b9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae4341cf0 0x555ae4344bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:24.205 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.216+0000 7f2930b9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae4341cf0 0x555ae4344bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:48446/0 (socket says 172.21.15.101:48446) 2023-07-18T03:07:24.206 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.216+0000 7f2930b9c700 1 -- 172.21.15.101:0/850496119 learned_addr learned my addr 172.21.15.101:0/850496119 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:24.206 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.216+0000 7f292bfff700 1 --2- 172.21.15.101:0/850496119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae4340550 0x555ae433a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:24.206 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.217+0000 7f2930b9c700 1 -- 172.21.15.101:0/850496119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae4340550 msgr2=0x555ae433a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:24.206 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.217+0000 7f2930b9c700 1 --2- 172.21.15.101:0/850496119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae4340550 0x555ae433a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.206 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.217+0000 7f2930b9c700 1 -- 172.21.15.101:0/850496119 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ae4340ea0 msgr2=0x555ae4351650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:24.207 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.217+0000 7f2930b9c700 1 --2- 172.21.15.101:0/850496119 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ae4340ea0 0x555ae4351650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.207 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.217+0000 7f2930b9c700 1 -- 172.21.15.101:0/850496119 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ae4342390 con 0x555ae4341cf0 2023-07-18T03:07:24.207 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.217+0000 7f2930b9c700 1 --2- 172.21.15.101:0/850496119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae4341cf0 0x555ae4344bb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f292400e920 tx=0x7f2924009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c0a859f9d6bef14 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:24.207 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.217+0000 7f2912ffd700 1 -- 172.21.15.101:0/850496119 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292400e040 con 0x555ae4341cf0 2023-07-18T03:07:24.207 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.217+0000 7f2912ffd700 1 -- 172.21.15.101:0/850496119 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f292400f040 con 0x555ae4341cf0 2023-07-18T03:07:24.208 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.217+0000 7f2912ffd700 1 -- 172.21.15.101:0/850496119 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29240047b0 con 0x555ae4341cf0 2023-07-18T03:07:24.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.218+0000 7f2945f1fb80 1 -- 172.21.15.101:0/850496119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae4341cf0 msgr2=0x555ae4344bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:24.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.218+0000 7f2945f1fb80 1 --2- 172.21.15.101:0/850496119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae4341cf0 0x555ae4344bb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f292400e920 tx=0x7f2924009920 comp rx=0 tx=0).stop 2023-07-18T03:07:24.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.218+0000 7f2945f1fb80 1 -- 172.21.15.101:0/850496119 shutdown_connections 2023-07-18T03:07:24.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.218+0000 7f2945f1fb80 1 --2- 172.21.15.101:0/850496119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae4340550 0x555ae433a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.218+0000 7f2945f1fb80 1 --2- 172.21.15.101:0/850496119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae4341cf0 0x555ae4344bb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.218+0000 7f2945f1fb80 1 --2- 172.21.15.101:0/850496119 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ae4340ea0 0x555ae4351650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.218+0000 7f2945f1fb80 1 -- 172.21.15.101:0/850496119 >> 172.21.15.101:0/850496119 conn(0x555ae4335e20 msgr2=0x555ae435cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:24.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.218+0000 7f2945f1fb80 1 -- 172.21.15.101:0/850496119 shutdown_connections 2023-07-18T03:07:24.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.218+0000 7f2945f1fb80 1 -- 172.21.15.101:0/850496119 wait complete. 2023-07-18T03:07:24.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.219+0000 7f2945f1fb80 1 -- start start 2023-07-18T03:07:24.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.219+0000 7f2945f1fb80 1 Processor -- start 2023-07-18T03:07:24.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.219+0000 7f2945f1fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ae4340550 0x555ae434e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:24.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.219+0000 7f2945f1fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae4340ea0 0x555ae434ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:24.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.219+0000 7f2945f1fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae4341cf0 0x555ae434fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:24.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.219+0000 7f2945f1fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ae4350160 con 0x555ae4340550 2023-07-18T03:07:24.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.219+0000 7f2945f1fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ae43502d0 con 0x555ae4341cf0 2023-07-18T03:07:24.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.219+0000 7f2945f1fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ae4350490 con 0x555ae4340ea0 2023-07-18T03:07:24.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.219+0000 7f2945f1fb80 10 client.0 _ll_register_callbacks cb 0x7f292400e280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:07:24.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.220+0000 7f292bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ae4340550 0x555ae434e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:24.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.220+0000 7f292b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae4340ea0 0x555ae434ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:24.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.220+0000 7f292bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ae4340550 0x555ae434e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:36666/0 (socket says 172.21.15.101:36666) 2023-07-18T03:07:24.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.220+0000 7f292bfff700 1 -- 172.21.15.101:0/2190499605 learned_addr learned my addr 172.21.15.101:0/2190499605 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:24.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.220+0000 7f2930b9c700 1 --2- 172.21.15.101:0/2190499605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae4341cf0 0x555ae434fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:24.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.220+0000 7f292bfff700 1 -- 172.21.15.101:0/2190499605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae4340ea0 msgr2=0x555ae434ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:24.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.220+0000 7f292bfff700 1 --2- 172.21.15.101:0/2190499605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae4340ea0 0x555ae434ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.220+0000 7f292bfff700 1 -- 172.21.15.101:0/2190499605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae4341cf0 msgr2=0x555ae434fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:24.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.220+0000 7f292bfff700 1 --2- 172.21.15.101:0/2190499605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae4341cf0 0x555ae434fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.220+0000 7f292bfff700 1 -- 172.21.15.101:0/2190499605 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f292400e5d0 con 0x555ae4340550 2023-07-18T03:07:24.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.220+0000 7f292bfff700 1 --2- 172.21.15.101:0/2190499605 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ae4340550 0x555ae434e6d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f291c009ce0 tx=0x7f291c09f080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f796d046be3059a0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:24.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.220+0000 7f290b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:07:24.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.220+0000 7f290b7fe700 1 -- 172.21.15.101:0/2190499605 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291c0b9270 con 0x555ae4340550 2023-07-18T03:07:24.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.221+0000 7f290b7fe700 1 -- 172.21.15.101:0/2190499605 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f291c09b2b0 con 0x555ae4340550 2023-07-18T03:07:24.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.221+0000 7f2945f1fb80 1 -- 172.21.15.101:0/2190499605 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2920004830 con 0x555ae4340550 2023-07-18T03:07:24.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.221+0000 7f290b7fe700 1 -- 172.21.15.101:0/2190499605 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f291c0b93d0 con 0x555ae4340550 2023-07-18T03:07:24.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.221+0000 7f2945f1fb80 10 client.15453 fetch_fsmap learned FSMap version 24 2023-07-18T03:07:24.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.221+0000 7f2945f1fb80 1 -- 172.21.15.101:0/2190499605 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x555ae40e62d0 con 0x555ae4340550 2023-07-18T03:07:24.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.221+0000 7f290b7fe700 1 -- 172.21.15.101:0/2190499605 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f291c07d020 con 0x555ae4340550 2023-07-18T03:07:24.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.221+0000 7f290b7fe700 1 -- 172.21.15.101:0/2190499605 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f291c0a3010 con 0x555ae4340550 2023-07-18T03:07:24.218 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.221+0000 7f2945f1fb80 10 client.15453 fetch_fsmap finished waiting for FSMap version 24 2023-07-18T03:07:24.218 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.221+0000 7f2945f1fb80 10 client.15453 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:07:24.218 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.221+0000 7f2945f1fb80 20 client.15453 populate_metadata read hostname 'smithi101' 2023-07-18T03:07:24.218 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.221+0000 7f2945f1fb80 1 --2- 172.21.15.101:0/2190499605 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x555ae43abc60 0x555ae43ae040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:24.218 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.221+0000 7f2945f1fb80 4 client.15453 mds_command: new command op to 24562 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-18T03:07:24.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.221+0000 7f2945f1fb80 1 -- 172.21.15.101:0/2190499605 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x555ae424d0d0 con 0x555ae43abc60 2023-07-18T03:07:24.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.222+0000 7f292b7fe700 1 --2- 172.21.15.101:0/2190499605 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x555ae43abc60 0x555ae43ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:24.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.222+0000 7f292b7fe700 1 --2- 172.21.15.101:0/2190499605 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x555ae43abc60 0x555ae43ae040 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6d1ad08042adb36 server_cookie=ef72064e9033c5f0 in_seq=0 out_seq=0 2023-07-18T03:07:24.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.222+0000 7f290b7fe700 10 client.15453 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:07:24.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.223+0000 7f290b7fe700 1 -- 172.21.15.101:0/2190499605 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555ae424d0d0 con 0x555ae43abc60 2023-07-18T03:07:24.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.223+0000 7f290b7fe700 10 client.15453 handle_command_reply: tid=0 2023-07-18T03:07:24.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.223+0000 7f2945f1fb80 1 client.15453 shutdown 2023-07-18T03:07:24.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.223+0000 7f2945f1fb80 1 -- 172.21.15.101:0/2190499605 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ae4340550 msgr2=0x555ae434e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:24.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.223+0000 7f2945f1fb80 1 --2- 172.21.15.101:0/2190499605 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ae4340550 0x555ae434e6d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f291c009ce0 tx=0x7f291c09f080 comp rx=0 tx=0).stop 2023-07-18T03:07:24.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.223+0000 7f2945f1fb80 1 -- 172.21.15.101:0/2190499605 shutdown_connections 2023-07-18T03:07:24.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.223+0000 7f2945f1fb80 1 --2- 172.21.15.101:0/2190499605 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x555ae43abc60 0x555ae43ae040 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.223+0000 7f2945f1fb80 1 --2- 172.21.15.101:0/2190499605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae4341cf0 0x555ae434fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.223+0000 7f2945f1fb80 1 --2- 172.21.15.101:0/2190499605 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ae4340550 0x555ae434e6d0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.223+0000 7f2945f1fb80 1 --2- 172.21.15.101:0/2190499605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae4340ea0 0x555ae434ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.223+0000 7f290b7fe700 0 client.15453 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:07:24.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.223+0000 7f2945f1fb80 1 -- 172.21.15.101:0/2190499605 >> 172.21.15.101:0/2190499605 conn(0x555ae4335e20 msgr2=0x555ae433a930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:24.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.224+0000 7f2945f1fb80 1 -- 172.21.15.101:0/2190499605 shutdown_connections 2023-07-18T03:07:24.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.224+0000 7f2945f1fb80 1 -- 172.21.15.101:0/2190499605 wait complete. 2023-07-18T03:07:24.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.224+0000 7f2945f1fb80 20 client.15453 trim_cache size 0 max 16384 2023-07-18T03:07:24.222 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:07:24.222 INFO:teuthology.orchestra.run.smithi101.stdout: "return_code": 0, 2023-07-18T03:07:24.223 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_tag": "e9f04a4a-1085-4d30-9d0c-3ba038aacad0", 2023-07-18T03:07:24.223 INFO:teuthology.orchestra.run.smithi101.stdout: "mode": "asynchronous" 2023-07-18T03:07:24.223 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:07:24.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.224+0000 7f29325ff700 1 -- 172.21.15.101:0/1590296860 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f29140a11d0 msgr2=0x7f29140a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:24.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.224+0000 7f29325ff700 1 --2- 172.21.15.101:0/1590296860 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f29140a11d0 0x7f29140a3680 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2920004170 tx=0x7f2920018040 comp rx=0 tx=0).stop 2023-07-18T03:07:24.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.224+0000 7f29325ff700 1 -- 172.21.15.101:0/1590296860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f292c0ffae0 msgr2=0x7f292c197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:24.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.224+0000 7f29325ff700 1 --2- 172.21.15.101:0/1590296860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f292c0ffae0 0x7f292c197a80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f291c004710 tx=0x7f291c009580 comp rx=0 tx=0).stop 2023-07-18T03:07:24.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.224+0000 7f29325ff700 1 -- 172.21.15.101:0/1590296860 shutdown_connections 2023-07-18T03:07:24.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.224+0000 7f29325ff700 1 --2- 172.21.15.101:0/1590296860 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f29140a11d0 0x7f29140a3680 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.224+0000 7f29325ff700 1 --2- 172.21.15.101:0/1590296860 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f292c101660 0x7f292c19f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.224+0000 7f29325ff700 1 --2- 172.21.15.101:0/1590296860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f292c0ffae0 0x7f292c197a80 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.224+0000 7f29325ff700 1 --2- 172.21.15.101:0/1590296860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f292c100cd0 0x7f292c197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.224+0000 7f29325ff700 1 -- 172.21.15.101:0/1590296860 >> 172.21.15.101:0/1590296860 conn(0x7f292c0fb0e0 msgr2=0x7f292c1065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:24.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.225+0000 7f29325ff700 1 -- 172.21.15.101:0/1590296860 shutdown_connections 2023-07-18T03:07:24.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.225+0000 7f29325ff700 1 -- 172.21.15.101:0/1590296860 wait complete. 2023-07-18T03:07:24.226 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:23 smithi101 ceph-mon[126379]: pgmap v351: 129 pgs: 129 active+clean; 646 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 767 B/s wr, 7 op/s 2023-07-18T03:07:24.226 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:23 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/814609035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:07:24.226 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:23 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2599096139' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:07:24.228 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:07:24.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:23 smithi155 ceph-mon[128885]: pgmap v351: 129 pgs: 129 active+clean; 646 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 767 B/s wr, 7 op/s 2023-07-18T03:07:24.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:23 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/814609035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:07:24.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:23 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/2599096139' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:07:24.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.456+0000 7f1ff2bff700 1 -- 172.21.15.101:0/2788577904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fec100ce0 msgr2=0x7f1fec101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:24.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.456+0000 7f1ff2bff700 1 --2- 172.21.15.101:0/2788577904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fec100ce0 0x7f1fec101130 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8009990 tx=0x7f1fd8009ca0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.456+0000 7f1ff2bff700 1 -- 172.21.15.101:0/2788577904 shutdown_connections 2023-07-18T03:07:24.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.456+0000 7f1ff2bff700 1 --2- 172.21.15.101:0/2788577904 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1fec101670 0x7f1fec107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.456+0000 7f1ff2bff700 1 --2- 172.21.15.101:0/2788577904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1fec0ffaf0 0x7f1fec0fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.456+0000 7f1ff2bff700 1 --2- 172.21.15.101:0/2788577904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fec100ce0 0x7f1fec101130 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.456+0000 7f1ff2bff700 1 -- 172.21.15.101:0/2788577904 >> 172.21.15.101:0/2788577904 conn(0x7f1fec0fb130 msgr2=0x7f1fec0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:24.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.456+0000 7f1ff2bff700 1 -- 172.21.15.101:0/2788577904 shutdown_connections 2023-07-18T03:07:24.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.456+0000 7f1ff2bff700 1 -- 172.21.15.101:0/2788577904 wait complete. 2023-07-18T03:07:24.444 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:23 smithi179 ceph-mon[129229]: pgmap v351: 129 pgs: 129 active+clean; 646 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 767 B/s wr, 7 op/s 2023-07-18T03:07:24.445 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:23 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/814609035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:07:24.445 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:23 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/2599096139' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:07:24.446 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.457+0000 7f1ff2bff700 1 Processor -- start 2023-07-18T03:07:24.446 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.457+0000 7f1ff2bff700 1 -- start start 2023-07-18T03:07:24.446 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.457+0000 7f1ff2bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1fec0ffaf0 0x7f1fec0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:24.446 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.457+0000 7f1ff2bff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1fec100ce0 0x7f1fec072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:24.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.457+0000 7f1ff2bff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fec101670 0x7f1fec072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:24.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.457+0000 7f1ff2bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fec0735e0 con 0x7f1fec0ffaf0 2023-07-18T03:07:24.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.457+0000 7f1ff2bff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fec073750 con 0x7f1fec101670 2023-07-18T03:07:24.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.457+0000 7f1ff2bff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fec19f940 con 0x7f1fec100ce0 2023-07-18T03:07:24.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.457+0000 7f1ff099b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1fec0ffaf0 0x7f1fec0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:24.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.458+0000 7f1febfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1fec100ce0 0x7f1fec072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:24.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.458+0000 7f1ff119c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fec101670 0x7f1fec072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:24.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.458+0000 7f1ff099b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1fec0ffaf0 0x7f1fec0742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:36696/0 (socket says 172.21.15.101:36696) 2023-07-18T03:07:24.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.458+0000 7f1ff119c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fec101670 0x7f1fec072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60586/0 (socket says 172.21.15.101:60586) 2023-07-18T03:07:24.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.458+0000 7f1ff099b700 1 -- 172.21.15.101:0/2756919337 learned_addr learned my addr 172.21.15.101:0/2756919337 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:24.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.458+0000 7f1febfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1fec100ce0 0x7f1fec072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:48464/0 (socket says 172.21.15.101:48464) 2023-07-18T03:07:24.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.458+0000 7f1ff099b700 1 -- 172.21.15.101:0/2756919337 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1fec100ce0 msgr2=0x7f1fec072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:24.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.458+0000 7f1ff099b700 1 --2- 172.21.15.101:0/2756919337 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1fec100ce0 0x7f1fec072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.458+0000 7f1ff099b700 1 -- 172.21.15.101:0/2756919337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fec101670 msgr2=0x7f1fec072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:24.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.458+0000 7f1ff099b700 1 --2- 172.21.15.101:0/2756919337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fec101670 0x7f1fec072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.458+0000 7f1ff099b700 1 -- 172.21.15.101:0/2756919337 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd80095f0 con 0x7f1fec0ffaf0 2023-07-18T03:07:24.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.458+0000 7f1ff099b700 1 --2- 172.21.15.101:0/2756919337 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1fec0ffaf0 0x7f1fec0742c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1fe000efc0 tx=0x7f1fe000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:24.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.458+0000 7f1fe9ffb700 1 -- 172.21.15.101:0/2756919337 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe0018070 con 0x7f1fec0ffaf0 2023-07-18T03:07:24.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.458+0000 7f1fe9ffb700 1 -- 172.21.15.101:0/2756919337 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fe000f040 con 0x7f1fec0ffaf0 2023-07-18T03:07:24.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.459+0000 7f1fe9ffb700 1 -- 172.21.15.101:0/2756919337 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe0004280 con 0x7f1fec0ffaf0 2023-07-18T03:07:24.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.459+0000 7f1ff2bff700 1 -- 172.21.15.101:0/2756919337 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fec19fbc0 con 0x7f1fec0ffaf0 2023-07-18T03:07:24.450 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.459+0000 7f1ff2bff700 1 -- 172.21.15.101:0/2756919337 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fec1a0080 con 0x7f1fec0ffaf0 2023-07-18T03:07:24.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.462+0000 7f1fe9ffb700 1 -- 172.21.15.101:0/2756919337 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f1fe0003a00 con 0x7f1fec0ffaf0 2023-07-18T03:07:24.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.463+0000 7f20065ecb80 1 Processor -- start 2023-07-18T03:07:24.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.463+0000 7f1fe9ffb700 1 --2- 172.21.15.101:0/2756919337 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f1fdc0a1150 0x7f1fdc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:24.452 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.463+0000 7f1fe9ffb700 1 -- 172.21.15.101:0/2756919337 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f1fe00ccbb0 con 0x7f1fec0ffaf0 2023-07-18T03:07:24.452 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.463+0000 7f20065ecb80 1 -- start start 2023-07-18T03:07:24.452 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.463+0000 7f1ff119c700 1 --2- 172.21.15.101:0/2756919337 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f1fdc0a1150 0x7f1fdc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:24.452 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.464+0000 7f20065ecb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a65dd74ce0 0x55a65dd750f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:24.452 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.464+0000 7f20065ecb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a65dd75630 0x55a65dd75ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:24.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.464+0000 7f20065ecb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a65dd76480 0x55a65dd768f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:24.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.464+0000 7f20065ecb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a65dc71cb0 con 0x55a65dd74ce0 2023-07-18T03:07:24.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.464+0000 7f20065ecb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a65dcc4a40 con 0x55a65dd76480 2023-07-18T03:07:24.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.464+0000 7f20065ecb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a65dcbded0 con 0x55a65dd75630 2023-07-18T03:07:24.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.464+0000 7f1ff099b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a65dd74ce0 0x55a65dd750f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:24.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.464+0000 7f1ff099b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a65dd74ce0 0x55a65dd750f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:36698/0 (socket says 172.21.15.101:36698) 2023-07-18T03:07:24.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.464+0000 7f1ff099b700 1 -- 172.21.15.101:0/3002749045 learned_addr learned my addr 172.21.15.101:0/3002749045 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:24.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.464+0000 7f1ff119c700 1 --2- 172.21.15.101:0/2756919337 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f1fdc0a1150 0x7f1fdc0a3600 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1fec074920 tx=0x7f1fe4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:24.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.464+0000 7f1ff119c700 1 --2- 172.21.15.101:0/3002749045 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a65dd76480 0x55a65dd768f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:24.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.464+0000 7f1febfff700 1 --2- 172.21.15.101:0/3002749045 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a65dd75630 0x55a65dd75ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:24.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.465+0000 7f1ff099b700 1 -- 172.21.15.101:0/3002749045 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a65dd75630 msgr2=0x55a65dd75ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:24.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.465+0000 7f1ff099b700 1 --2- 172.21.15.101:0/3002749045 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a65dd75630 0x55a65dd75ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.465+0000 7f1ff099b700 1 -- 172.21.15.101:0/3002749045 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a65dd76480 msgr2=0x55a65dd768f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:24.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.465+0000 7f1ff099b700 1 --2- 172.21.15.101:0/3002749045 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a65dd76480 0x55a65dd768f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.465+0000 7f1ff099b700 1 -- 172.21.15.101:0/3002749045 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a65daebde0 con 0x55a65dd74ce0 2023-07-18T03:07:24.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.465+0000 7f1ff099b700 1 --2- 172.21.15.101:0/3002749045 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a65dd74ce0 0x55a65dd750f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1fe00090a0 tx=0x7f1fe00959c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d3904a0ffdb0f59 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:24.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.465+0000 7f1fd6ffd700 1 -- 172.21.15.101:0/3002749045 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe00995e0 con 0x55a65dd74ce0 2023-07-18T03:07:24.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.465+0000 7f1fd6ffd700 1 -- 172.21.15.101:0/3002749045 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fe0099c80 con 0x55a65dd74ce0 2023-07-18T03:07:24.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.465+0000 7f1fd6ffd700 1 -- 172.21.15.101:0/3002749045 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe0096570 con 0x55a65dd74ce0 2023-07-18T03:07:24.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.466+0000 7f20065ecb80 1 -- 172.21.15.101:0/3002749045 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a65dd74ce0 msgr2=0x55a65dd750f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:24.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.466+0000 7f20065ecb80 1 --2- 172.21.15.101:0/3002749045 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a65dd74ce0 0x55a65dd750f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1fe00090a0 tx=0x7f1fe00959c0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.466+0000 7f20065ecb80 1 -- 172.21.15.101:0/3002749045 shutdown_connections 2023-07-18T03:07:24.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.466+0000 7f20065ecb80 1 --2- 172.21.15.101:0/3002749045 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a65dd76480 0x55a65dd768f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.466+0000 7f20065ecb80 1 --2- 172.21.15.101:0/3002749045 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a65dd74ce0 0x55a65dd750f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.466+0000 7f20065ecb80 1 --2- 172.21.15.101:0/3002749045 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a65dd75630 0x55a65dd75ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.466+0000 7f20065ecb80 1 -- 172.21.15.101:0/3002749045 >> 172.21.15.101:0/3002749045 conn(0x55a65dd61da0 msgr2=0x55a65dd671d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:24.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.466+0000 7f20065ecb80 1 -- 172.21.15.101:0/3002749045 shutdown_connections 2023-07-18T03:07:24.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.466+0000 7f20065ecb80 1 -- 172.21.15.101:0/3002749045 wait complete. 2023-07-18T03:07:24.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.467+0000 7f20065ecb80 1 -- start start 2023-07-18T03:07:24.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.467+0000 7f20065ecb80 1 Processor -- start 2023-07-18T03:07:24.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.468+0000 7f20065ecb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a65dd74ce0 0x55a65dd7e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:24.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.468+0000 7f20065ecb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a65dd75630 0x55a65dd7eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:24.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.468+0000 7f20065ecb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a65dd76480 0x55a65dd85f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:24.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.468+0000 7f20065ecb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a65dc5dac0 con 0x55a65dd74ce0 2023-07-18T03:07:24.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.468+0000 7f20065ecb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a65dc5d7c0 con 0x55a65dd75630 2023-07-18T03:07:24.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.468+0000 7f20065ecb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a65db41920 con 0x55a65dd76480 2023-07-18T03:07:24.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.468+0000 7f20065ecb80 10 client.0 _ll_register_callbacks cb 0x55a65dc5b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:07:24.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.468+0000 7f1febfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a65dd75630 0x55a65dd7eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:24.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.468+0000 7f1ff099b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a65dd74ce0 0x55a65dd7e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:24.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.468+0000 7f1febfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a65dd75630 0x55a65dd7eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60618/0 (socket says 172.21.15.101:60618) 2023-07-18T03:07:24.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.468+0000 7f1febfff700 1 -- 172.21.15.101:0/40817275 learned_addr learned my addr 172.21.15.101:0/40817275 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:24.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.468+0000 7f1ff119c700 1 --2- 172.21.15.101:0/40817275 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a65dd76480 0x55a65dd85f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:24.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.468+0000 7f1febfff700 1 -- 172.21.15.101:0/40817275 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a65dd76480 msgr2=0x55a65dd85f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:24.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.468+0000 7f1febfff700 1 --2- 172.21.15.101:0/40817275 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a65dd76480 0x55a65dd85f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.468+0000 7f1febfff700 1 -- 172.21.15.101:0/40817275 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a65dd74ce0 msgr2=0x55a65dd7e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:24.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.468+0000 7f1febfff700 1 --2- 172.21.15.101:0/40817275 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a65dd74ce0 0x55a65dd7e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.468+0000 7f1febfff700 1 -- 172.21.15.101:0/40817275 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fec1a0080 con 0x55a65dd75630 2023-07-18T03:07:24.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.468+0000 7f1ff099b700 1 --2- 172.21.15.101:0/40817275 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a65dd74ce0 0x55a65dd7e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:07:24.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.469+0000 7f1febfff700 1 --2- 172.21.15.101:0/40817275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a65dd75630 0x55a65dd7eec0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8015000 tx=0x7f1fd8009570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15ec74b15610e7f3 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:24.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.469+0000 7f1fcbfff700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:07:24.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.469+0000 7f1fcbfff700 1 -- 172.21.15.101:0/40817275 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd801d030 con 0x55a65dd75630 2023-07-18T03:07:24.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.469+0000 7f20065ecb80 1 -- 172.21.15.101:0/40817275 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a65dc174f0 con 0x55a65dd75630 2023-07-18T03:07:24.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.469+0000 7f1fcbfff700 1 -- 172.21.15.101:0/40817275 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f1fd8004500 con 0x55a65dd75630 2023-07-18T03:07:24.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.469+0000 7f1fcbfff700 1 -- 172.21.15.101:0/40817275 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fd80046c0 con 0x55a65dd75630 2023-07-18T03:07:24.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.469+0000 7f20065ecb80 10 client.25490 fetch_fsmap learned FSMap version 24 2023-07-18T03:07:24.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.469+0000 7f20065ecb80 1 -- 172.21.15.101:0/40817275 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55a65dad4c60 con 0x55a65dd75630 2023-07-18T03:07:24.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.469+0000 7f1fcbfff700 1 -- 172.21.15.101:0/40817275 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a65dc174f0 con 0x55a65dd75630 2023-07-18T03:07:24.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.470+0000 7f1fcbfff700 1 -- 172.21.15.101:0/40817275 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1fd800f980 con 0x55a65dd75630 2023-07-18T03:07:24.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.470+0000 7f20065ecb80 10 client.25490 fetch_fsmap finished waiting for FSMap version 24 2023-07-18T03:07:24.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.470+0000 7f20065ecb80 10 client.25490 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:07:24.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.470+0000 7f20065ecb80 20 client.25490 populate_metadata read hostname 'smithi101' 2023-07-18T03:07:24.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.470+0000 7f20065ecb80 1 --2- 172.21.15.101:0/40817275 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a65dd8a7b0 0x55a65dd8cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:24.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.470+0000 7f20065ecb80 4 client.25490 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:07:24.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.470+0000 7f20065ecb80 1 -- 172.21.15.101:0/40817275 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a65dcb6eb0 con 0x55a65dd8a7b0 2023-07-18T03:07:24.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.470+0000 7f1ff099b700 1 --2- 172.21.15.101:0/40817275 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a65dd8a7b0 0x55a65dd8cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:24.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.471+0000 7f1fcbfff700 10 client.25490 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:07:24.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.471+0000 7f1ff099b700 1 --2- 172.21.15.101:0/40817275 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a65dd8a7b0 0x55a65dd8cbc0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23d2e6fc472a60e3 server_cookie=cee0ea49e4eafc16 in_seq=0 out_seq=0 2023-07-18T03:07:24.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.471+0000 7f1fcbfff700 1 -- 172.21.15.101:0/40817275 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a65dcb6eb0 con 0x55a65dd8a7b0 2023-07-18T03:07:24.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.471+0000 7f1fcbfff700 10 client.25490 handle_command_reply: tid=0 2023-07-18T03:07:24.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.471+0000 7f20065ecb80 1 client.25490 shutdown 2023-07-18T03:07:24.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.472+0000 7f20065ecb80 1 -- 172.21.15.101:0/40817275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a65dd75630 msgr2=0x55a65dd7eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:24.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.472+0000 7f20065ecb80 1 --2- 172.21.15.101:0/40817275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a65dd75630 0x55a65dd7eec0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8015000 tx=0x7f1fd8009570 comp rx=0 tx=0).stop 2023-07-18T03:07:24.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.472+0000 7f20065ecb80 1 -- 172.21.15.101:0/40817275 shutdown_connections 2023-07-18T03:07:24.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.472+0000 7f20065ecb80 1 --2- 172.21.15.101:0/40817275 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a65dd8a7b0 0x55a65dd8cbc0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.472+0000 7f20065ecb80 1 --2- 172.21.15.101:0/40817275 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a65dd76480 0x55a65dd85f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.472+0000 7f20065ecb80 1 --2- 172.21.15.101:0/40817275 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a65dd74ce0 0x55a65dd7e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.472+0000 7f20065ecb80 1 --2- 172.21.15.101:0/40817275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a65dd75630 0x55a65dd7eec0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.472+0000 7f1fcbfff700 0 client.25490 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:07:24.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.472+0000 7f20065ecb80 1 -- 172.21.15.101:0/40817275 >> 172.21.15.101:0/40817275 conn(0x55a65dd61da0 msgr2=0x55a65dd668b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:24.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.472+0000 7f20065ecb80 1 -- 172.21.15.101:0/40817275 shutdown_connections 2023-07-18T03:07:24.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.472+0000 7f20065ecb80 1 -- 172.21.15.101:0/40817275 wait complete. 2023-07-18T03:07:24.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.472+0000 7f20065ecb80 20 client.25490 trim_cache size 0 max 16384 2023-07-18T03:07:24.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.489+0000 7f20065ecb80 1 Processor -- start 2023-07-18T03:07:24.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.489+0000 7f20065ecb80 1 -- start start 2023-07-18T03:07:24.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.490+0000 7f20065ecb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a65dd74ce0 0x55a65dd7c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:24.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.490+0000 7f20065ecb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a65dd75630 0x55a65dd85ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:24.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.490+0000 7f20065ecb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a65dd76480 0x55a65dd793c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:24.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.490+0000 7f20065ecb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a65dd61970 con 0x55a65dd76480 2023-07-18T03:07:24.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.490+0000 7f20065ecb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a65dd75bf0 con 0x55a65dd74ce0 2023-07-18T03:07:24.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.490+0000 7f20065ecb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a65dd7f600 con 0x55a65dd75630 2023-07-18T03:07:24.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.490+0000 7f1ff119c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a65dd76480 0x55a65dd793c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:24.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.490+0000 7f1ff099b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a65dd74ce0 0x55a65dd7c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:24.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.490+0000 7f1ff099b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a65dd74ce0 0x55a65dd7c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60634/0 (socket says 172.21.15.101:60634) 2023-07-18T03:07:24.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.490+0000 7f1febfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a65dd75630 0x55a65dd85ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:24.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.490+0000 7f1ff099b700 1 -- 172.21.15.101:0/3160866058 learned_addr learned my addr 172.21.15.101:0/3160866058 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:24.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.490+0000 7f1ff099b700 1 -- 172.21.15.101:0/3160866058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a65dd75630 msgr2=0x55a65dd85ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:24.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.490+0000 7f1ff099b700 1 --2- 172.21.15.101:0/3160866058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a65dd75630 0x55a65dd85ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.490+0000 7f1ff099b700 1 -- 172.21.15.101:0/3160866058 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a65dd76480 msgr2=0x55a65dd793c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:24.480 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.490+0000 7f1ff099b700 1 --2- 172.21.15.101:0/3160866058 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a65dd76480 0x55a65dd793c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.480 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.490+0000 7f1ff099b700 1 -- 172.21.15.101:0/3160866058 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a65db12250 con 0x55a65dd74ce0 2023-07-18T03:07:24.480 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.491+0000 7f1ff099b700 1 --2- 172.21.15.101:0/3160866058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a65dd74ce0 0x55a65dd7c890 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0099c10 tx=0x7f1fe009f040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f33df546046f83e0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:24.480 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.491+0000 7f1fd77fe700 1 -- 172.21.15.101:0/3160866058 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe009dcb0 con 0x55a65dd74ce0 2023-07-18T03:07:24.480 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.491+0000 7f1fd77fe700 1 -- 172.21.15.101:0/3160866058 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fe00a2040 con 0x55a65dd74ce0 2023-07-18T03:07:24.480 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.491+0000 7f1fd77fe700 1 -- 172.21.15.101:0/3160866058 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe00b2280 con 0x55a65dd74ce0 2023-07-18T03:07:24.480 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.492+0000 7f20065ecb80 1 -- 172.21.15.101:0/3160866058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a65dd74ce0 msgr2=0x55a65dd7c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:24.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.492+0000 7f20065ecb80 1 --2- 172.21.15.101:0/3160866058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a65dd74ce0 0x55a65dd7c890 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0099c10 tx=0x7f1fe009f040 comp rx=0 tx=0).stop 2023-07-18T03:07:24.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.492+0000 7f20065ecb80 1 -- 172.21.15.101:0/3160866058 shutdown_connections 2023-07-18T03:07:24.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.492+0000 7f20065ecb80 1 --2- 172.21.15.101:0/3160866058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a65dd74ce0 0x55a65dd7c890 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.492+0000 7f20065ecb80 1 --2- 172.21.15.101:0/3160866058 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a65dd76480 0x55a65dd793c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.492+0000 7f20065ecb80 1 --2- 172.21.15.101:0/3160866058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a65dd75630 0x55a65dd85ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.492+0000 7f20065ecb80 1 -- 172.21.15.101:0/3160866058 >> 172.21.15.101:0/3160866058 conn(0x55a65dd61da0 msgr2=0x55a65dd7d600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:24.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.492+0000 7f20065ecb80 1 -- 172.21.15.101:0/3160866058 shutdown_connections 2023-07-18T03:07:24.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.492+0000 7f20065ecb80 1 -- 172.21.15.101:0/3160866058 wait complete. 2023-07-18T03:07:24.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.492+0000 7f20065ecb80 1 -- start start 2023-07-18T03:07:24.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.493+0000 7f20065ecb80 1 Processor -- start 2023-07-18T03:07:24.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.493+0000 7f20065ecb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a65dd74ce0 0x55a65dd81b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:24.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.493+0000 7f20065ecb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a65dd75630 0x55a65dd82080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:24.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.493+0000 7f20065ecb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a65dd76480 0x55a65dd850a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:24.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.494+0000 7f20065ecb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a65dd855e0 con 0x55a65dd74ce0 2023-07-18T03:07:24.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.494+0000 7f20065ecb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a65dd85750 con 0x55a65dd76480 2023-07-18T03:07:24.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.494+0000 7f20065ecb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a65dd85910 con 0x55a65dd75630 2023-07-18T03:07:24.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.494+0000 7f20065ecb80 10 client.0 _ll_register_callbacks cb 0x7f1fd8005350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:07:24.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.494+0000 7f1ff099b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a65dd74ce0 0x55a65dd81b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:24.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.494+0000 7f1ff099b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a65dd74ce0 0x55a65dd81b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:36720/0 (socket says 172.21.15.101:36720) 2023-07-18T03:07:24.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.494+0000 7f1ff099b700 1 -- 172.21.15.101:0/221557438 learned_addr learned my addr 172.21.15.101:0/221557438 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:24.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.494+0000 7f1ff119c700 1 --2- 172.21.15.101:0/221557438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a65dd76480 0x55a65dd850a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:24.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.494+0000 7f1febfff700 1 --2- 172.21.15.101:0/221557438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a65dd75630 0x55a65dd82080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:24.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.494+0000 7f1ff119c700 1 -- 172.21.15.101:0/221557438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a65dd75630 msgr2=0x55a65dd82080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:24.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.494+0000 7f1ff119c700 1 --2- 172.21.15.101:0/221557438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a65dd75630 0x55a65dd82080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.494+0000 7f1ff119c700 1 -- 172.21.15.101:0/221557438 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a65dd74ce0 msgr2=0x55a65dd81b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:24.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.494+0000 7f1ff119c700 1 --2- 172.21.15.101:0/221557438 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a65dd74ce0 0x55a65dd81b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.494+0000 7f1ff119c700 1 -- 172.21.15.101:0/221557438 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a65daebde0 con 0x55a65dd76480 2023-07-18T03:07:24.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.494+0000 7f1ff099b700 1 --2- 172.21.15.101:0/221557438 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a65dd74ce0 0x55a65dd81b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:07:24.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.494+0000 7f1ff119c700 1 --2- 172.21.15.101:0/221557438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a65dd76480 0x55a65dd850a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1fe400a010 tx=0x7f1fe40047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fab16f4d435bc9c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:24.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.494+0000 7f1fcbfff700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:07:24.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.495+0000 7f1fcbfff700 1 -- 172.21.15.101:0/221557438 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe401ee30 con 0x55a65dd76480 2023-07-18T03:07:24.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.495+0000 7f20065ecb80 1 -- 172.21.15.101:0/221557438 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1fe0099c80 con 0x55a65dd76480 2023-07-18T03:07:24.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.495+0000 7f1fcbfff700 1 -- 172.21.15.101:0/221557438 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f1fe4004260 con 0x55a65dd76480 2023-07-18T03:07:24.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.495+0000 7f1fcbfff700 1 -- 172.21.15.101:0/221557438 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fe4004420 con 0x55a65dd76480 2023-07-18T03:07:24.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.495+0000 7f20065ecb80 10 client.25496 fetch_fsmap learned FSMap version 24 2023-07-18T03:07:24.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.495+0000 7f20065ecb80 1 -- 172.21.15.101:0/221557438 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55a65daea2b0 con 0x55a65dd76480 2023-07-18T03:07:24.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.495+0000 7f1fcbfff700 1 -- 172.21.15.101:0/221557438 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fe0099c80 con 0x55a65dd76480 2023-07-18T03:07:24.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.495+0000 7f1fcbfff700 1 -- 172.21.15.101:0/221557438 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1fe40038e0 con 0x55a65dd76480 2023-07-18T03:07:24.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.495+0000 7f20065ecb80 10 client.25496 fetch_fsmap finished waiting for FSMap version 24 2023-07-18T03:07:24.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.495+0000 7f20065ecb80 10 client.25496 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:07:24.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.495+0000 7f20065ecb80 20 client.25496 populate_metadata read hostname 'smithi101' 2023-07-18T03:07:24.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.495+0000 7f20065ecb80 1 --2- 172.21.15.101:0/221557438 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a65dddf600 0x55a65dde1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:24.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.495+0000 7f20065ecb80 4 client.25496 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:07:24.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.495+0000 7f20065ecb80 1 -- 172.21.15.101:0/221557438 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a65dc79050 con 0x55a65dddf600 2023-07-18T03:07:24.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.496+0000 7f1ff099b700 1 --2- 172.21.15.101:0/221557438 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a65dddf600 0x55a65dde1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:24.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.496+0000 7f1fcbfff700 10 client.25496 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:07:24.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.496+0000 7f1ff099b700 1 --2- 172.21.15.101:0/221557438 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a65dddf600 0x55a65dde1a10 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4251824be13d96f2 server_cookie=b4de3d9a01c6633b in_seq=0 out_seq=0 2023-07-18T03:07:24.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.496+0000 7f1fcbfff700 1 -- 172.21.15.101:0/221557438 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55a65dc79050 con 0x55a65dddf600 2023-07-18T03:07:24.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.496+0000 7f1fcbfff700 10 client.25496 handle_command_reply: tid=0 2023-07-18T03:07:24.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.496+0000 7f20065ecb80 1 client.25496 shutdown 2023-07-18T03:07:24.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.497+0000 7f20065ecb80 1 -- 172.21.15.101:0/221557438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a65dd76480 msgr2=0x55a65dd850a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:24.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.497+0000 7f20065ecb80 1 --2- 172.21.15.101:0/221557438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a65dd76480 0x55a65dd850a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1fe400a010 tx=0x7f1fe40047c0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.497+0000 7f20065ecb80 1 -- 172.21.15.101:0/221557438 shutdown_connections 2023-07-18T03:07:24.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.497+0000 7f20065ecb80 1 --2- 172.21.15.101:0/221557438 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a65dddf600 0x55a65dde1a10 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.497+0000 7f20065ecb80 1 --2- 172.21.15.101:0/221557438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a65dd76480 0x55a65dd850a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.497+0000 7f1fcbfff700 0 client.25496 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:07:24.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.497+0000 7f20065ecb80 1 --2- 172.21.15.101:0/221557438 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a65dd74ce0 0x55a65dd81b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.497+0000 7f20065ecb80 1 --2- 172.21.15.101:0/221557438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a65dd75630 0x55a65dd82080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.497+0000 7f20065ecb80 1 -- 172.21.15.101:0/221557438 >> 172.21.15.101:0/221557438 conn(0x55a65dd61da0 msgr2=0x55a65dd7d600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:24.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.497+0000 7f20065ecb80 1 -- 172.21.15.101:0/221557438 shutdown_connections 2023-07-18T03:07:24.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.497+0000 7f20065ecb80 1 -- 172.21.15.101:0/221557438 wait complete. 2023-07-18T03:07:24.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.497+0000 7f20065ecb80 20 client.25496 trim_cache size 0 max 16384 2023-07-18T03:07:24.491 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:07:24.492 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:07:24.492 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:07:24.492 INFO:teuthology.orchestra.run.smithi101.stdout: "e9f04a4a-1085-4d30-9d0c-3ba038aacad0": { 2023-07-18T03:07:24.492 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:07:24.492 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "e9f04a4a-1085-4d30-9d0c-3ba038aacad0", 2023-07-18T03:07:24.492 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:07:24.492 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:07:24.493 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:07:24.493 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:07:24.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.497+0000 7f1ff2bff700 1 -- 172.21.15.101:0/2756919337 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f1fdc0a1150 msgr2=0x7f1fdc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:24.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.497+0000 7f1ff2bff700 1 --2- 172.21.15.101:0/2756919337 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f1fdc0a1150 0x7f1fdc0a3600 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1fec074920 tx=0x7f1fe4009260 comp rx=0 tx=0).stop 2023-07-18T03:07:24.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.497+0000 7f1ff2bff700 1 -- 172.21.15.101:0/2756919337 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1fec0ffaf0 msgr2=0x7f1fec0742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:24.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.497+0000 7f1ff2bff700 1 --2- 172.21.15.101:0/2756919337 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1fec0ffaf0 0x7f1fec0742c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1fe000efc0 tx=0x7f1fe000c3a0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.498+0000 7f1ff2bff700 1 -- 172.21.15.101:0/2756919337 shutdown_connections 2023-07-18T03:07:24.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.498+0000 7f1ff2bff700 1 --2- 172.21.15.101:0/2756919337 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f1fdc0a1150 0x7f1fdc0a3600 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.498+0000 7f1ff2bff700 1 --2- 172.21.15.101:0/2756919337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fec101670 0x7f1fec072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.498+0000 7f1ff2bff700 1 --2- 172.21.15.101:0/2756919337 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1fec0ffaf0 0x7f1fec0742c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.498+0000 7f1ff2bff700 1 --2- 172.21.15.101:0/2756919337 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1fec100ce0 0x7f1fec072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.498+0000 7f1ff2bff700 1 -- 172.21.15.101:0/2756919337 >> 172.21.15.101:0/2756919337 conn(0x7f1fec0fb130 msgr2=0x7f1fec104220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:24.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.498+0000 7f1ff2bff700 1 -- 172.21.15.101:0/2756919337 shutdown_connections 2023-07-18T03:07:24.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.498+0000 7f1ff2bff700 1 -- 172.21.15.101:0/2756919337 wait complete. 2023-07-18T03:07:24.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.500+0000 7f95b13ff700 1 -- 172.21.15.101:0/1157994141 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f95ac106a30 con 0x7f95ac101690 2023-07-18T03:07:24.502 INFO:tasks.cephfs.filesystem:scrub status for tag:e9f04a4a-1085-4d30-9d0c-3ba038aacad0 - {'path': '/', 'tag': 'e9f04a4a-1085-4d30-9d0c-3ba038aacad0', 'options': 'recursive,force'} 2023-07-18T03:07:24.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.989+0000 7f9593fff700 1 -- 172.21.15.101:0/1157994141 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mgrmap(e 19) v1 ==== 135733+0+0 (secure 0 0 0) 0x7f95a4095b80 con 0x7f95ac101690 2023-07-18T03:07:24.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.994+0000 7f9593fff700 1 -- 172.21.15.101:0/1157994141 <== mon.1 v2:172.21.15.179:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) v1 ==== 92+0+0 (secure 0 0 0) 0x7f95a40ac5b0 con 0x7f95ac101690 2023-07-18T03:07:24.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.996+0000 7f95b13ff700 1 -- 172.21.15.101:0/1157994141 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f95940a11d0 msgr2=0x7f95940a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:24.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.996+0000 7f95b13ff700 1 --2- 172.21.15.101:0/1157994141 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f95940a11d0 0x7f95940a3680 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f95ac074210 tx=0x7f959c009260 comp rx=0 tx=0).stop 2023-07-18T03:07:24.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.996+0000 7f95b13ff700 1 -- 172.21.15.101:0/1157994141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95ac101690 msgr2=0x7f95ac073380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:24.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.996+0000 7f95b13ff700 1 --2- 172.21.15.101:0/1157994141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95ac101690 0x7f95ac073380 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f95a4007ee0 tx=0x7f95a400d3b0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.996+0000 7f95b13ff700 1 -- 172.21.15.101:0/1157994141 shutdown_connections 2023-07-18T03:07:24.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.996+0000 7f95b13ff700 1 --2- 172.21.15.101:0/1157994141 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f95940a11d0 0x7f95940a3680 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.996+0000 7f95b13ff700 1 --2- 172.21.15.101:0/1157994141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95ac101690 0x7f95ac073380 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.996+0000 7f95b13ff700 1 --2- 172.21.15.101:0/1157994141 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f95ac0ffb10 0x7f95ac072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.996+0000 7f95b13ff700 1 --2- 172.21.15.101:0/1157994141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95ac100d00 0x7f95ac072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:24.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.996+0000 7f95b13ff700 1 -- 172.21.15.101:0/1157994141 >> 172.21.15.101:0/1157994141 conn(0x7f95ac0fb130 msgr2=0x7f95ac0fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:24.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.996+0000 7f95b13ff700 1 -- 172.21.15.101:0/1157994141 shutdown_connections 2023-07-18T03:07:24.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:24.996+0000 7f95b13ff700 1 -- 172.21.15.101:0/1157994141 wait complete. 2023-07-18T03:07:25.004 DEBUG:teuthology.orchestra.run.smithi101:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2023-07-18T03:07:25.218 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.231+0000 7f7c4ebff700 1 -- 172.21.15.101:0/3599746158 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c48100d00 msgr2=0x7f7c48101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:25.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.231+0000 7f7c4ebff700 1 --2- 172.21.15.101:0/3599746158 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c48100d00 0x7f7c48101150 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7c340098f0 tx=0x7f7c34009c00 comp rx=0 tx=0).stop 2023-07-18T03:07:25.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.231+0000 7f7c4ebff700 1 -- 172.21.15.101:0/3599746158 shutdown_connections 2023-07-18T03:07:25.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.231+0000 7f7c4ebff700 1 --2- 172.21.15.101:0/3599746158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c480ffb10 0x7f7c480fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:25.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.231+0000 7f7c4ebff700 1 --2- 172.21.15.101:0/3599746158 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7c48101690 0x7f7c48107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:25.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.231+0000 7f7c4ebff700 1 --2- 172.21.15.101:0/3599746158 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c48100d00 0x7f7c48101150 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:25.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.231+0000 7f7c4ebff700 1 -- 172.21.15.101:0/3599746158 >> 172.21.15.101:0/3599746158 conn(0x7f7c480fb130 msgr2=0x7f7c480fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:25.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.231+0000 7f7c4ebff700 1 -- 172.21.15.101:0/3599746158 shutdown_connections 2023-07-18T03:07:25.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.232+0000 7f7c4ebff700 1 -- 172.21.15.101:0/3599746158 wait complete. 2023-07-18T03:07:25.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.233+0000 7f7c4ebff700 1 Processor -- start 2023-07-18T03:07:25.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.233+0000 7f7c4ebff700 1 -- start start 2023-07-18T03:07:25.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.233+0000 7f7c4ebff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c480ffb10 0x7f7c48072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:25.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.233+0000 7f7c4ebff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c48100d00 0x7f7c48072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:25.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.233+0000 7f7c4ebff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7c48101690 0x7f7c48073380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:25.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.233+0000 7f7c4ebff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4819f8e0 con 0x7f7c48101690 2023-07-18T03:07:25.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.233+0000 7f7c4ebff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4819fa20 con 0x7f7c480ffb10 2023-07-18T03:07:25.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.233+0000 7f7c4ebff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4819fb60 con 0x7f7c48100d00 2023-07-18T03:07:25.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.234+0000 7f7c47fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c48100d00 0x7f7c48072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:25.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.234+0000 7f7c47fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c48100d00 0x7f7c48072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:48534/0 (socket says 172.21.15.101:48534) 2023-07-18T03:07:25.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.234+0000 7f7c47fff700 1 -- 172.21.15.101:0/1006437802 learned_addr learned my addr 172.21.15.101:0/1006437802 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:25.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.234+0000 7f7c4c99b700 1 --2- 172.21.15.101:0/1006437802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c480ffb10 0x7f7c48072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:25.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.234+0000 7f7c4d19c700 1 --2- 172.21.15.101:0/1006437802 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7c48101690 0x7f7c48073380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:25.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.234+0000 7f7c47fff700 1 -- 172.21.15.101:0/1006437802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c480ffb10 msgr2=0x7f7c48072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:25.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.234+0000 7f7c47fff700 1 --2- 172.21.15.101:0/1006437802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c480ffb10 0x7f7c48072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:25.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.234+0000 7f7c47fff700 1 -- 172.21.15.101:0/1006437802 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7c48101690 msgr2=0x7f7c48073380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:25.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.234+0000 7f7c47fff700 1 --2- 172.21.15.101:0/1006437802 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7c48101690 0x7f7c48073380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:25.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.234+0000 7f7c47fff700 1 -- 172.21.15.101:0/1006437802 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c3c0095f0 con 0x7f7c48100d00 2023-07-18T03:07:25.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.234+0000 7f7c4d19c700 1 --2- 172.21.15.101:0/1006437802 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7c48101690 0x7f7c48073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:07:25.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.235+0000 7f7c47fff700 1 --2- 172.21.15.101:0/1006437802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c48100d00 0x7f7c48072e40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7c34009fd0 tx=0x7f7c3400b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:25.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.235+0000 7f7c45ffb700 1 -- 172.21.15.101:0/1006437802 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3401c070 con 0x7f7c48100d00 2023-07-18T03:07:25.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.235+0000 7f7c45ffb700 1 -- 172.21.15.101:0/1006437802 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c3400f7d0 con 0x7f7c48100d00 2023-07-18T03:07:25.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.235+0000 7f7c45ffb700 1 -- 172.21.15.101:0/1006437802 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c34004b90 con 0x7f7c48100d00 2023-07-18T03:07:25.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.235+0000 7f7c4ebff700 1 -- 172.21.15.101:0/1006437802 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c34009520 con 0x7f7c48100d00 2023-07-18T03:07:25.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.235+0000 7f7c4ebff700 1 -- 172.21.15.101:0/1006437802 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c481a0080 con 0x7f7c48100d00 2023-07-18T03:07:25.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.237+0000 7f7c337fe700 1 -- 172.21.15.101:0/1006437802 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c4804e2f0 con 0x7f7c48100d00 2023-07-18T03:07:25.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.240+0000 7f7c45ffb700 1 -- 172.21.15.101:0/1006437802 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 19) v1 ==== 135733+0+0 (secure 0 0 0) 0x7f7c34020070 con 0x7f7c48100d00 2023-07-18T03:07:25.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.242+0000 7f7c45ffb700 1 --2- 172.21.15.101:0/1006437802 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f7c380a1220 0x7f7c380a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:25.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.242+0000 7f7c45ffb700 1 -- 172.21.15.101:0/1006437802 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f7c340cd9b0 con 0x7f7c48100d00 2023-07-18T03:07:25.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.242+0000 7f7c4c99b700 1 -- 172.21.15.101:0/1006437802 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f7c380a1220 msgr2=0x7f7c380a36d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.101:6800/3963742842 2023-07-18T03:07:25.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.242+0000 7f7c4c99b700 1 --2- 172.21.15.101:0/1006437802 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f7c380a1220 0x7f7c380a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-18T03:07:25.232 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:07:25 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: ignoring --setuser ceph since I am not root 2023-07-18T03:07:25.232 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:07:25 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: ignoring --setgroup ceph since I am not root 2023-07-18T03:07:25.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.246+0000 7f7c45ffb700 1 -- 172.21.15.101:0/1006437802 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7c3405c790 con 0x7f7c48100d00 2023-07-18T03:07:25.233 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:24 smithi101 ceph-mon[126379]: scrub queued for path: / 2023-07-18T03:07:25.233 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:24 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:07:25.233 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:24 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:07:25.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:24 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:07:25.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:24 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1157994141' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-07-18T03:07:25.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:24 smithi101 ceph-mon[126379]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-07-18T03:07:25.239 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 03:07:25 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: ignoring --setuser ceph since I am not root 2023-07-18T03:07:25.239 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 03:07:25 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: ignoring --setgroup ceph since I am not root 2023-07-18T03:07:25.239 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 03:07:25 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T03:07:25.160+0000 7f8fca2e3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-18T03:07:25.240 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 03:07:25 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T03:07:25.252+0000 7f8fca2e3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-18T03:07:25.240 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:24 smithi155 ceph-mon[128885]: scrub queued for path: / 2023-07-18T03:07:25.240 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:24 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:07:25.240 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:24 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:07:25.240 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:24 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:07:25.241 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:24 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1157994141' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-07-18T03:07:25.241 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:24 smithi155 ceph-mon[128885]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-07-18T03:07:25.241 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 03:07:25 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: ignoring --setuser ceph since I am not root 2023-07-18T03:07:25.241 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 03:07:25 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: ignoring --setgroup ceph since I am not root 2023-07-18T03:07:25.242 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 03:07:25 smithi179 ceph-mgr[131117]: -- 172.21.15.179:0/1453357894 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55999b76b1e0 con 0x55999b740c00 2023-07-18T03:07:25.242 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 03:07:25 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T03:07:25.163+0000 7f7eabcca200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-18T03:07:25.242 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:24 smithi179 ceph-mon[129229]: scrub queued for path: / 2023-07-18T03:07:25.242 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:24 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:07:25.242 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:24 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:07:25.242 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:24 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:07:25.242 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:24 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1157994141' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-07-18T03:07:25.243 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:24 smithi179 ceph-mon[129229]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-07-18T03:07:25.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.443+0000 7f7c4c99b700 1 -- 172.21.15.101:0/1006437802 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f7c380a1220 msgr2=0x7f7c380a36d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.101:6800/3963742842 2023-07-18T03:07:25.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.443+0000 7f7c4c99b700 1 --2- 172.21.15.101:0/1006437802 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f7c380a1220 0x7f7c380a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-18T03:07:25.471 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.485+0000 7f7c337fe700 1 -- 172.21.15.101:0/1006437802 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f7c48061120 con 0x7f7c48100d00 2023-07-18T03:07:25.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.492+0000 7f7c45ffb700 1 -- 172.21.15.101:0/1006437802 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v23)=0 v23) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7c34027020 con 0x7f7c48100d00 2023-07-18T03:07:25.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.495+0000 7f7c337fe700 1 -- 172.21.15.101:0/1006437802 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f7c380a1220 msgr2=0x7f7c380a36d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-18T03:07:25.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.495+0000 7f7c337fe700 1 --2- 172.21.15.101:0/1006437802 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f7c380a1220 0x7f7c380a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:25.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.495+0000 7f7c337fe700 1 -- 172.21.15.101:0/1006437802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c48100d00 msgr2=0x7f7c48072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:25.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.495+0000 7f7c337fe700 1 --2- 172.21.15.101:0/1006437802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c48100d00 0x7f7c48072e40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7c34009fd0 tx=0x7f7c3400b790 comp rx=0 tx=0).stop 2023-07-18T03:07:25.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.495+0000 7f7c337fe700 1 -- 172.21.15.101:0/1006437802 shutdown_connections 2023-07-18T03:07:25.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.495+0000 7f7c337fe700 1 --2- 172.21.15.101:0/1006437802 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f7c380a1220 0x7f7c380a36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:25.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.495+0000 7f7c337fe700 1 --2- 172.21.15.101:0/1006437802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c480ffb10 0x7f7c48072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:25.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.495+0000 7f7c337fe700 1 --2- 172.21.15.101:0/1006437802 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7c48101690 0x7f7c48073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:25.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.495+0000 7f7c337fe700 1 --2- 172.21.15.101:0/1006437802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c48100d00 0x7f7c48072e40 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:25.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.495+0000 7f7c337fe700 1 -- 172.21.15.101:0/1006437802 >> 172.21.15.101:0/1006437802 conn(0x7f7c480fb130 msgr2=0x7f7c48104260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:25.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.495+0000 7f7c337fe700 1 -- 172.21.15.101:0/1006437802 shutdown_connections 2023-07-18T03:07:25.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.495+0000 7f7c337fe700 1 -- 172.21.15.101:0/1006437802 wait complete. 2023-07-18T03:07:25.501 DEBUG:teuthology.orchestra.run.smithi101:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2023-07-18T03:07:25.521 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 03:07:25 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T03:07:25.253+0000 7f7eabcca200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-18T03:07:25.521 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 03:07:25 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T03:07:25.344+0000 7f7eabcca200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-18T03:07:25.521 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 03:07:25 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T03:07:25.446+0000 7f7eabcca200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-18T03:07:25.539 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 03:07:25 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T03:07:25.342+0000 7f8fca2e3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-18T03:07:25.539 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 03:07:25 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T03:07:25.435+0000 7f8fca2e3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-18T03:07:25.564 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:07:25 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:07:25.287+0000 7f024068f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-18T03:07:25.565 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:07:25 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:07:25.400+0000 7f024068f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-18T03:07:25.565 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:07:25 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:07:25.524+0000 7f024068f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-18T03:07:25.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.722+0000 7fa0891ff700 1 -- 172.21.15.101:0/1107628723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa084072f70 msgr2=0x7fa0840733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:25.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.722+0000 7fa0891ff700 1 --2- 172.21.15.101:0/1107628723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa084072f70 0x7fa0840733e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa078009990 tx=0x7fa078009ca0 comp rx=0 tx=0).stop 2023-07-18T03:07:25.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.722+0000 7fa0891ff700 1 -- 172.21.15.101:0/1107628723 shutdown_connections 2023-07-18T03:07:25.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.722+0000 7fa0891ff700 1 --2- 172.21.15.101:0/1107628723 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa08410c0e0 0x7fa08410e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:25.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.722+0000 7fa0891ff700 1 --2- 172.21.15.101:0/1107628723 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa084074550 0x7fa0840729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:25.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.722+0000 7fa0891ff700 1 --2- 172.21.15.101:0/1107628723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa084072f70 0x7fa0840733e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:25.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.722+0000 7fa0891ff700 1 -- 172.21.15.101:0/1107628723 >> 172.21.15.101:0/1107628723 conn(0x7fa0840fb130 msgr2=0x7fa0840fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:25.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.722+0000 7fa0891ff700 1 -- 172.21.15.101:0/1107628723 shutdown_connections 2023-07-18T03:07:25.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.722+0000 7fa0891ff700 1 -- 172.21.15.101:0/1107628723 wait complete. 2023-07-18T03:07:25.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.723+0000 7fa0891ff700 1 Processor -- start 2023-07-18T03:07:25.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.723+0000 7fa0891ff700 1 -- start start 2023-07-18T03:07:25.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.723+0000 7fa0891ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa084072f70 0x7fa08419bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:25.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.723+0000 7fa0891ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa084074550 0x7fa08419c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:25.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.723+0000 7fa0891ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa08410c0e0 0x7fa08419d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:25.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.723+0000 7fa0891ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08419d9d0 con 0x7fa084072f70 2023-07-18T03:07:25.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.723+0000 7fa0891ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08419db40 con 0x7fa08410c0e0 2023-07-18T03:07:25.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.723+0000 7fa0891ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08419dd00 con 0x7fa084074550 2023-07-18T03:07:25.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.724+0000 7fa08259c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa084074550 0x7fa08419c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:25.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.724+0000 7fa082d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa084072f70 0x7fa08419bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:25.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.724+0000 7fa08259c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa084074550 0x7fa08419c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:48554/0 (socket says 172.21.15.101:48554) 2023-07-18T03:07:25.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.724+0000 7fa08359e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa08410c0e0 0x7fa08419d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:25.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.724+0000 7fa08259c700 1 -- 172.21.15.101:0/3791477609 learned_addr learned my addr 172.21.15.101:0/3791477609 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:25.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.724+0000 7fa08259c700 1 -- 172.21.15.101:0/3791477609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa08410c0e0 msgr2=0x7fa08419d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:25.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.724+0000 7fa08259c700 1 --2- 172.21.15.101:0/3791477609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa08410c0e0 0x7fa08419d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:25.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.724+0000 7fa08259c700 1 -- 172.21.15.101:0/3791477609 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa084072f70 msgr2=0x7fa08419bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:25.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.724+0000 7fa08259c700 1 --2- 172.21.15.101:0/3791477609 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa084072f70 0x7fa08419bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:25.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.724+0000 7fa08259c700 1 -- 172.21.15.101:0/3791477609 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0780095f0 con 0x7fa084074550 2023-07-18T03:07:25.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.724+0000 7fa08259c700 1 --2- 172.21.15.101:0/3791477609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa084074550 0x7fa08419c4f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa078005720 tx=0x7fa078009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:25.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.724+0000 7fa06bfff700 1 -- 172.21.15.101:0/3791477609 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07801d070 con 0x7fa084074550 2023-07-18T03:07:25.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.725+0000 7fa0891ff700 1 -- 172.21.15.101:0/3791477609 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa08419df80 con 0x7fa084074550 2023-07-18T03:07:25.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.725+0000 7fa06bfff700 1 -- 172.21.15.101:0/3791477609 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa078021410 con 0x7fa084074550 2023-07-18T03:07:25.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.725+0000 7fa06bfff700 1 -- 172.21.15.101:0/3791477609 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa078017650 con 0x7fa084074550 2023-07-18T03:07:25.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.725+0000 7fa0891ff700 1 -- 172.21.15.101:0/3791477609 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0841a47a0 con 0x7fa084074550 2023-07-18T03:07:25.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.726+0000 7fa0891ff700 1 -- 172.21.15.101:0/3791477609 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa084101c30 con 0x7fa084074550 2023-07-18T03:07:25.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.728+0000 7fa06bfff700 1 -- 172.21.15.101:0/3791477609 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 19) v1 ==== 135733+0+0 (secure 0 0 0) 0x7fa078021580 con 0x7fa084074550 2023-07-18T03:07:25.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.729+0000 7fa06bfff700 1 --2- 172.21.15.101:0/3791477609 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa06c0a1220 0x7fa06c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:25.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.729+0000 7fa06bfff700 1 -- 172.21.15.101:0/3791477609 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fa0780cddf0 con 0x7fa084074550 2023-07-18T03:07:25.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.729+0000 7fa082d9d700 1 -- 172.21.15.101:0/3791477609 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa06c0a1220 msgr2=0x7fa06c0a36d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.101:6800/3963742842 2023-07-18T03:07:25.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.729+0000 7fa082d9d700 1 --2- 172.21.15.101:0/3791477609 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa06c0a1220 0x7fa06c0a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-18T03:07:25.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.731+0000 7fa06bfff700 1 -- 172.21.15.101:0/3791477609 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa078097200 con 0x7fa084074550 2023-07-18T03:07:25.901 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 03:07:25 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T03:07:25.606+0000 7f8fca2e3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-18T03:07:25.905 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:07:25 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:07:25.673+0000 7f024068f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-18T03:07:25.916 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 03:07:25 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T03:07:25.621+0000 7f7eabcca200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-18T03:07:25.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.930+0000 7fa082d9d700 1 -- 172.21.15.101:0/3791477609 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa06c0a1220 msgr2=0x7fa06c0a36d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.101:6800/3963742842 2023-07-18T03:07:25.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.930+0000 7fa082d9d700 1 --2- 172.21.15.101:0/3791477609 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa06c0a1220 0x7fa06c0a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-18T03:07:25.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.949+0000 7fa0891ff700 1 -- 172.21.15.101:0/3791477609 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fa08402c850 con 0x7fa084074550 2023-07-18T03:07:25.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.957+0000 7fa06bfff700 1 -- 172.21.15.101:0/3791477609 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v24)=0 v24) v1 ==== 133+0+0 (secure 0 0 0) 0x7fa0780260c0 con 0x7fa084074550 2023-07-18T03:07:25.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.960+0000 7fa0891ff700 1 -- 172.21.15.101:0/3791477609 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa06c0a1220 msgr2=0x7fa06c0a36d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-18T03:07:25.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.960+0000 7fa0891ff700 1 --2- 172.21.15.101:0/3791477609 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa06c0a1220 0x7fa06c0a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:25.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.960+0000 7fa0891ff700 1 -- 172.21.15.101:0/3791477609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa084074550 msgr2=0x7fa08419c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:25.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.960+0000 7fa0891ff700 1 --2- 172.21.15.101:0/3791477609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa084074550 0x7fa08419c4f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa078005720 tx=0x7fa078009580 comp rx=0 tx=0).stop 2023-07-18T03:07:25.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.960+0000 7fa0891ff700 1 -- 172.21.15.101:0/3791477609 shutdown_connections 2023-07-18T03:07:25.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.960+0000 7fa0891ff700 1 --2- 172.21.15.101:0/3791477609 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7fa06c0a1220 0x7fa06c0a36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:25.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.960+0000 7fa0891ff700 1 --2- 172.21.15.101:0/3791477609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa08410c0e0 0x7fa08419d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:25.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.960+0000 7fa0891ff700 1 --2- 172.21.15.101:0/3791477609 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa084072f70 0x7fa08419bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:25.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.960+0000 7fa0891ff700 1 --2- 172.21.15.101:0/3791477609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa084074550 0x7fa08419c4f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:25.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.960+0000 7fa0891ff700 1 -- 172.21.15.101:0/3791477609 >> 172.21.15.101:0/3791477609 conn(0x7fa0840fb130 msgr2=0x7fa0840fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:25.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.960+0000 7fa0891ff700 1 -- 172.21.15.101:0/3791477609 shutdown_connections 2023-07-18T03:07:25.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:25.960+0000 7fa0891ff700 1 -- 172.21.15.101:0/3791477609 wait complete. 2023-07-18T03:07:25.974 DEBUG:teuthology.orchestra.run.smithi101:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1M' 2023-07-18T03:07:26.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:26.193+0000 7f8d019ff700 1 -- 172.21.15.101:0/4054738498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cfc100d00 msgr2=0x7f8cfc101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:26.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:26.193+0000 7f8d019ff700 1 --2- 172.21.15.101:0/4054738498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cfc100d00 0x7f8cfc101150 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0009990 tx=0x7f8cf0009ca0 comp rx=0 tx=0).stop 2023-07-18T03:07:26.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:26.193+0000 7f8d019ff700 1 -- 172.21.15.101:0/4054738498 shutdown_connections 2023-07-18T03:07:26.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:26.193+0000 7f8d019ff700 1 --2- 172.21.15.101:0/4054738498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cfc101690 0x7f8cfc107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:26.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:26.193+0000 7f8d019ff700 1 --2- 172.21.15.101:0/4054738498 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8cfc0ffb10 0x7f8cfc0fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:26.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:26.193+0000 7f8d019ff700 1 --2- 172.21.15.101:0/4054738498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cfc100d00 0x7f8cfc101150 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:26.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:26.193+0000 7f8d019ff700 1 -- 172.21.15.101:0/4054738498 >> 172.21.15.101:0/4054738498 conn(0x7f8cfc0fb130 msgr2=0x7f8cfc0fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:26.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:26.193+0000 7f8d019ff700 1 -- 172.21.15.101:0/4054738498 shutdown_connections 2023-07-18T03:07:26.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:26.193+0000 7f8d019ff700 1 -- 172.21.15.101:0/4054738498 wait complete. 2023-07-18T03:07:26.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:26.194+0000 7f8d019ff700 1 Processor -- start 2023-07-18T03:07:26.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:26.194+0000 7f8d019ff700 1 -- start start 2023-07-18T03:07:26.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:26.194+0000 7f8d019ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cfc0ffb10 0x7f8cfc197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:26.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:26.194+0000 7f8d019ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cfc100d00 0x7f8cfc198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:26.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:26.194+0000 7f8d019ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8cfc101690 0x7f8cfc198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:26.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:26.194+0000 7f8d019ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc199550 con 0x7f8cfc101690 2023-07-18T03:07:26.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:26.194+0000 7f8d019ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc1996c0 con 0x7f8cfc100d00 2023-07-18T03:07:26.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:26.194+0000 7f8d019ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc199880 con 0x7f8cfc0ffb10 2023-07-18T03:07:26.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:26.195+0000 7f8cfaffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cfc0ffb10 0x7f8cfc197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:26.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:26.195+0000 7f8cfaffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cfc0ffb10 0x7f8cfc197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:58510/0 (socket says 172.21.15.101:58510) 2023-07-18T03:07:26.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:26.195+0000 7f8cfaffd700 1 -- 172.21.15.101:0/35905659 learned_addr learned my addr 172.21.15.101:0/35905659 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:26.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:26.195+0000 7f8cfa7fc700 1 --2- 172.21.15.101:0/35905659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cfc100d00 0x7f8cfc198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:26.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:26.195+0000 7f8cfb7fe700 1 --2- 172.21.15.101:0/35905659 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8cfc101690 0x7f8cfc198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:26.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:26.195+0000 7f8cfaffd700 1 -- 172.21.15.101:0/35905659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cfc100d00 msgr2=0x7f8cfc198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:26.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:26.195+0000 7f8cfaffd700 1 --2- 172.21.15.101:0/35905659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cfc100d00 0x7f8cfc198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:26.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:26.195+0000 7f8cfaffd700 1 -- 172.21.15.101:0/35905659 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8cfc101690 msgr2=0x7f8cfc198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:26.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:26.195+0000 7f8cfaffd700 1 --2- 172.21.15.101:0/35905659 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8cfc101690 0x7f8cfc198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:26.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:26.195+0000 7f8cfaffd700 1 -- 172.21.15.101:0/35905659 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cf00095f0 con 0x7f8cfc0ffb10 2023-07-18T03:07:26.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:26.195+0000 7f8cfaffd700 1 --2- 172.21.15.101:0/35905659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cfc0ffb10 0x7f8cfc197b30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8cec00e970 tx=0x7f8cec00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:26.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:26.196+0000 7f8d009fd700 1 -- 172.21.15.101:0/35905659 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cec009790 con 0x7f8cfc0ffb10 2023-07-18T03:07:26.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:26.196+0000 7f8d009fd700 1 -- 172.21.15.101:0/35905659 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8cec00c7e0 con 0x7f8cfc0ffb10 2023-07-18T03:07:26.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:26.196+0000 7f8d019ff700 1 -- 172.21.15.101:0/35905659 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cfc199b60 con 0x7f8cfc0ffb10 2023-07-18T03:07:26.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:26.196+0000 7f8d009fd700 1 -- 172.21.15.101:0/35905659 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cec003c70 con 0x7f8cfc0ffb10 2023-07-18T03:07:26.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:26.196+0000 7f8d019ff700 1 -- 172.21.15.101:0/35905659 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cfc1a0320 con 0x7f8cfc0ffb10 2023-07-18T03:07:26.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:26.197+0000 7f8d019ff700 1 -- 172.21.15.101:0/35905659 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cfc04e2f0 con 0x7f8cfc0ffb10 2023-07-18T03:07:26.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:26.199+0000 7f8d009fd700 1 -- 172.21.15.101:0/35905659 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 19) v1 ==== 135733+0+0 (secure 0 0 0) 0x7f8cec00cc50 con 0x7f8cfc0ffb10 2023-07-18T03:07:26.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:26.200+0000 7f8d009fd700 1 --2- 172.21.15.101:0/35905659 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8ce40a1220 0x7f8ce40a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:26.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:26.200+0000 7f8d009fd700 1 -- 172.21.15.101:0/35905659 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f8cec014070 con 0x7f8cfc0ffb10 2023-07-18T03:07:26.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:26.200+0000 7f8cfa7fc700 1 -- 172.21.15.101:0/35905659 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8ce40a1220 msgr2=0x7f8ce40a36d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.101:6800/3963742842 2023-07-18T03:07:26.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:26.200+0000 7f8cfa7fc700 1 --2- 172.21.15.101:0/35905659 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8ce40a1220 0x7f8ce40a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-18T03:07:26.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:26.202+0000 7f8d009fd700 1 -- 172.21.15.101:0/35905659 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8cec096800 con 0x7f8cfc0ffb10 2023-07-18T03:07:26.194 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 03:07:25 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T03:07:25.928+0000 7f7eabcca200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-18T03:07:26.194 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 03:07:26 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T03:07:26.019+0000 7f7eabcca200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-18T03:07:26.194 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 03:07:26 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T03:07:26.148+0000 7f7eabcca200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-18T03:07:26.194 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:25 smithi179 ceph-mon[129229]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-07-18T03:07:26.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:25 smithi179 ceph-mon[129229]: mgrmap e19: x(active, since 9m), standbys: z, y 2023-07-18T03:07:26.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:25 smithi179 ceph-mon[129229]: from='client.? ' entity='client.admin' 2023-07-18T03:07:26.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:25 smithi179 ceph-mon[129229]: from='client.? ' entity='client.admin' 2023-07-18T03:07:26.216 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 03:07:25 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T03:07:25.914+0000 7f8fca2e3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-18T03:07:26.216 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 03:07:26 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T03:07:26.004+0000 7f8fca2e3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-18T03:07:26.216 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 03:07:26 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T03:07:26.134+0000 7f8fca2e3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-18T03:07:26.217 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:25 smithi155 ceph-mon[128885]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-07-18T03:07:26.217 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:25 smithi155 ceph-mon[128885]: mgrmap e19: x(active, since 9m), standbys: z, y 2023-07-18T03:07:26.217 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:25 smithi155 ceph-mon[128885]: from='client.? ' entity='client.admin' 2023-07-18T03:07:26.217 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:25 smithi155 ceph-mon[128885]: from='client.? ' entity='client.admin' 2023-07-18T03:07:26.223 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:07:25 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:07:25.918+0000 7f024068f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-18T03:07:26.223 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:25 smithi101 ceph-mon[126379]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-07-18T03:07:26.223 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:25 smithi101 ceph-mon[126379]: mgrmap e19: x(active, since 9m), standbys: z, y 2023-07-18T03:07:26.223 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:25 smithi101 ceph-mon[126379]: from='client.? ' entity='client.admin' 2023-07-18T03:07:26.223 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:25 smithi101 ceph-mon[126379]: from='client.? ' entity='client.admin' 2023-07-18T03:07:26.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:26.401+0000 7f8cfa7fc700 1 -- 172.21.15.101:0/35905659 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8ce40a1220 msgr2=0x7f8ce40a36d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.101:6800/3963742842 2023-07-18T03:07:26.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:26.401+0000 7f8cfa7fc700 1 --2- 172.21.15.101:0/35905659 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8ce40a1220 0x7f8ce40a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-18T03:07:26.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:26.437+0000 7f8d019ff700 1 -- 172.21.15.101:0/35905659 --> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f8cfc1a05a0 con 0x7f8ce40a1220 2023-07-18T03:07:26.481 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:07:26 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:07:26.341+0000 7f024068f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-18T03:07:26.482 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:07:26 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:07:26.458+0000 7f024068f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-18T03:07:26.539 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 03:07:26 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T03:07:26.229+0000 7f8fca2e3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-18T03:07:26.652 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 03:07:26 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T03:07:26.244+0000 7f7eabcca200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-18T03:07:26.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:26.802+0000 7f8cfa7fc700 1 -- 172.21.15.101:0/35905659 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8ce40a1220 msgr2=0x7f8ce40a36d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.101:6800/3963742842 2023-07-18T03:07:26.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:26.802+0000 7f8cfa7fc700 1 --2- 172.21.15.101:0/35905659 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8ce40a1220 0x7f8ce40a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-07-18T03:07:26.944 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 03:07:26 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T03:07:26.664+0000 7f7eabcca200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-18T03:07:26.981 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:07:26 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:07:26.619+0000 7f024068f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-18T03:07:26.982 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:07:26 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:07:26.741+0000 7f024068f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-18T03:07:27.039 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 03:07:26 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T03:07:26.648+0000 7f8fca2e3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-18T03:07:27.539 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 03:07:27 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T03:07:27.208+0000 7f8fca2e3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-18T03:07:27.539 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 03:07:27 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T03:07:27.378+0000 7f8fca2e3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-18T03:07:27.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:27.603+0000 7f8cfa7fc700 1 -- 172.21.15.101:0/35905659 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8ce40a1220 msgr2=0x7f8ce40a36d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.101:6800/3963742842 2023-07-18T03:07:27.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:27.603+0000 7f8cfa7fc700 1 --2- 172.21.15.101:0/35905659 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8ce40a1220 0x7f8ce40a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-07-18T03:07:27.694 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 03:07:27 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T03:07:27.226+0000 7f7eabcca200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-18T03:07:27.694 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 03:07:27 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T03:07:27.397+0000 7f7eabcca200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-18T03:07:27.731 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:07:27 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:07:27.272+0000 7f024068f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-18T03:07:28.395 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 03:07:28 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T03:07:28.145+0000 7f8fca2e3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-18T03:07:28.395 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 03:07:28 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T03:07:28.237+0000 7f8fca2e3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-18T03:07:28.414 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 03:07:28 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T03:07:28.164+0000 7f7eabcca200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-18T03:07:28.415 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 03:07:28 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T03:07:28.256+0000 7f7eabcca200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-18T03:07:28.481 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:07:28 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:07:28.010+0000 7f024068f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-18T03:07:28.482 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:07:28 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:07:28.216+0000 7f024068f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-18T03:07:28.671 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 03:07:28 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T03:07:28.407+0000 7f8fca2e3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-18T03:07:28.693 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 03:07:28 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T03:07:28.427+0000 7f7eabcca200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-18T03:07:28.944 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 03:07:28 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T03:07:28.705+0000 7f7eabcca200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-18T03:07:28.944 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 03:07:28 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T03:07:28.932+0000 7f7eabcca200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-18T03:07:29.039 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 03:07:28 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T03:07:28.683+0000 7f8fca2e3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-18T03:07:29.039 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 03:07:28 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T03:07:28.911+0000 7f8fca2e3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-18T03:07:29.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:29.205+0000 7f8cfa7fc700 1 -- 172.21.15.101:0/35905659 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8ce40a1220 msgr2=0x7f8ce40a36d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.101:6800/3963742842 2023-07-18T03:07:29.192 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:29.205+0000 7f8cfa7fc700 1 --2- 172.21.15.101:0/35905659 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8ce40a1220 0x7f8ce40a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-07-18T03:07:29.306 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 03:07:29 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T03:07:29.057+0000 7f8fca2e3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-18T03:07:29.328 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 03:07:29 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T03:07:29.080+0000 7f7eabcca200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-18T03:07:29.480 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:07:29 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:07:29.152+0000 7f024068f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-18T03:07:29.481 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:07:29 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:07:29.271+0000 7f024068f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-18T03:07:29.694 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 03:07:29 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T03:07:29.340+0000 7f7eabcca200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-18T03:07:29.732 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:07:29 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:07:29.493+0000 7f024068f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-18T03:07:29.789 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 03:07:29 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T03:07:29.318+0000 7f8fca2e3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-18T03:07:30.113 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:07:29 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:07:29.851+0000 7f024068f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-18T03:07:30.289 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 03:07:30 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T03:07:30.016+0000 7f8fca2e3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-18T03:07:30.289 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 03:07:30 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T03:07:30.145+0000 7f8fca2e3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-18T03:07:30.348 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 03:07:30 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T03:07:30.040+0000 7f7eabcca200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-18T03:07:30.349 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 03:07:30 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T03:07:30.170+0000 7f7eabcca200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-18T03:07:30.481 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:07:30 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:07:30.126+0000 7f024068f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-18T03:07:30.482 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:07:30 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:07:30.319+0000 7f024068f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-18T03:07:30.652 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 03:07:30 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T03:07:30.336+0000 7f8fca2e3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-18T03:07:30.652 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 03:07:30 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T03:07:30.449+0000 7f8fca2e3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-18T03:07:30.676 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 03:07:30 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T03:07:30.361+0000 7f7eabcca200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-18T03:07:30.676 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 03:07:30 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T03:07:30.475+0000 7f7eabcca200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-18T03:07:30.944 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 03:07:30 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T03:07:30.688+0000 7f7eabcca200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-18T03:07:30.981 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:07:30 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:07:30.659+0000 7f024068f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-18T03:07:31.039 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 03:07:30 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T03:07:30.664+0000 7f8fca2e3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-18T03:07:31.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:31.596+0000 7f8d009fd700 1 -- 172.21.15.101:0/35905659 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mgrmap(e 20) v1 ==== 135733+0+0 (secure 0 0 0) 0x7f8cec00c950 con 0x7f8cfc0ffb10 2023-07-18T03:07:31.789 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 03:07:31 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y[130338]: 2023-07-18T03:07:31.518+0000 7f8fca2e3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-18T03:07:31.789 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:31 smithi155 ceph-mon[128885]: Standby manager daemon y restarted 2023-07-18T03:07:31.790 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:31 smithi155 ceph-mon[128885]: Standby manager daemon y started 2023-07-18T03:07:31.790 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:31 smithi155 ceph-mon[128885]: from='mgr.? 172.21.15.155:0/480679832' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-18T03:07:31.790 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:31 smithi155 ceph-mon[128885]: from='mgr.? 172.21.15.155:0/480679832' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-18T03:07:31.790 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:31 smithi155 ceph-mon[128885]: from='mgr.? 172.21.15.155:0/480679832' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-18T03:07:31.790 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:31 smithi155 ceph-mon[128885]: from='mgr.? 172.21.15.155:0/480679832' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-18T03:07:31.790 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:31 smithi155 ceph-mon[128885]: Standby manager daemon z restarted 2023-07-18T03:07:31.791 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:31 smithi155 ceph-mon[128885]: Standby manager daemon z started 2023-07-18T03:07:31.791 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:31 smithi155 ceph-mon[128885]: from='mgr.? 172.21.15.179:0/2109985019' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-18T03:07:31.791 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:31 smithi155 ceph-mon[128885]: from='mgr.? 172.21.15.179:0/2109985019' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-18T03:07:31.791 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:31 smithi155 ceph-mon[128885]: from='mgr.? 172.21.15.179:0/2109985019' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-18T03:07:31.791 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:31 smithi155 ceph-mon[128885]: from='mgr.? 172.21.15.179:0/2109985019' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-18T03:07:31.944 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 03:07:31 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z[131093]: 2023-07-18T03:07:31.549+0000 7f7eabcca200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-18T03:07:31.945 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:31 smithi179 ceph-mon[129229]: Standby manager daemon y restarted 2023-07-18T03:07:31.945 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:31 smithi179 ceph-mon[129229]: Standby manager daemon y started 2023-07-18T03:07:31.945 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:31 smithi179 ceph-mon[129229]: from='mgr.? 172.21.15.155:0/480679832' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-18T03:07:31.946 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:31 smithi179 ceph-mon[129229]: from='mgr.? 172.21.15.155:0/480679832' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-18T03:07:31.946 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:31 smithi179 ceph-mon[129229]: from='mgr.? 172.21.15.155:0/480679832' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-18T03:07:31.946 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:31 smithi179 ceph-mon[129229]: from='mgr.? 172.21.15.155:0/480679832' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-18T03:07:31.946 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:31 smithi179 ceph-mon[129229]: Standby manager daemon z restarted 2023-07-18T03:07:31.946 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:31 smithi179 ceph-mon[129229]: Standby manager daemon z started 2023-07-18T03:07:31.946 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:31 smithi179 ceph-mon[129229]: from='mgr.? 172.21.15.179:0/2109985019' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-18T03:07:31.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:31 smithi179 ceph-mon[129229]: from='mgr.? 172.21.15.179:0/2109985019' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-18T03:07:31.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:31 smithi179 ceph-mon[129229]: from='mgr.? 172.21.15.179:0/2109985019' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-18T03:07:31.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:31 smithi179 ceph-mon[129229]: from='mgr.? 172.21.15.179:0/2109985019' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-18T03:07:31.981 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:07:31 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:07:31.585+0000 7f024068f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-18T03:07:31.982 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:07:31 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:07:31.762+0000 7f024068f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-18T03:07:31.982 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:31 smithi101 ceph-mon[126379]: Standby manager daemon y restarted 2023-07-18T03:07:31.982 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:31 smithi101 ceph-mon[126379]: Standby manager daemon y started 2023-07-18T03:07:31.982 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:31 smithi101 ceph-mon[126379]: from='mgr.? 172.21.15.155:0/480679832' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-18T03:07:31.983 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:31 smithi101 ceph-mon[126379]: from='mgr.? 172.21.15.155:0/480679832' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-18T03:07:31.983 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:31 smithi101 ceph-mon[126379]: from='mgr.? 172.21.15.155:0/480679832' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-18T03:07:31.983 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:31 smithi101 ceph-mon[126379]: from='mgr.? 172.21.15.155:0/480679832' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-18T03:07:31.983 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:31 smithi101 ceph-mon[126379]: Standby manager daemon z restarted 2023-07-18T03:07:31.983 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:31 smithi101 ceph-mon[126379]: Standby manager daemon z started 2023-07-18T03:07:31.983 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:31 smithi101 ceph-mon[126379]: from='mgr.? 172.21.15.179:0/2109985019' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-18T03:07:31.983 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:31 smithi101 ceph-mon[126379]: from='mgr.? 172.21.15.179:0/2109985019' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-18T03:07:31.984 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:31 smithi101 ceph-mon[126379]: from='mgr.? 172.21.15.179:0/2109985019' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-18T03:07:31.984 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:31 smithi101 ceph-mon[126379]: from='mgr.? 172.21.15.179:0/2109985019' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-18T03:07:32.395 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:32.407+0000 7f8cfa7fc700 1 -- 172.21.15.101:0/35905659 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8ce40a1220 msgr2=0x7f8ce40a36d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.101:6800/3963742842 2023-07-18T03:07:32.395 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:32.407+0000 7f8cfa7fc700 1 --2- 172.21.15.101:0/35905659 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8ce40a1220 0x7f8ce40a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-07-18T03:07:32.426 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:07:32 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:07:32.013+0000 7f024068f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-18T03:07:32.427 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:07:32 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:07:32.163+0000 7f024068f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-18T03:07:32.732 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:07:32 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:07:32.438+0000 7f024068f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-18T03:07:32.732 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:32 smithi101 ceph-mon[126379]: mgrmap e20: x(active, since 9m), standbys: y, z 2023-07-18T03:07:32.944 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:32 smithi179 ceph-mon[129229]: mgrmap e20: x(active, since 9m), standbys: y, z 2023-07-18T03:07:33.039 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:32 smithi155 ceph-mon[128885]: mgrmap e20: x(active, since 9m), standbys: y, z 2023-07-18T03:07:33.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:33.512+0000 7f8d009fd700 1 -- 172.21.15.101:0/35905659 <== mon.1 v2:172.21.15.179:3300/0 8 ==== mgrmap(e 21) v1 ==== 135400+0+0 (secure 0 0 0) 0x7f8cec0cac30 con 0x7f8cfc0ffb10 2023-07-18T03:07:33.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:33.512+0000 7f8d009fd700 1 -- 172.21.15.101:0/35905659 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8ce40a1220 msgr2=0x7f8ce40a36d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-18T03:07:33.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:33.512+0000 7f8d009fd700 1 --2- 172.21.15.101:0/35905659 >> [v2:172.21.15.101:6800/3963742842,v1:172.21.15.101:6801/3963742842] conn(0x7f8ce40a1220 0x7f8ce40a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:33.762 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:07:33.495+0000 7f024068f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-18T03:07:33.763 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:07:33.715+0000 7f019c3a7700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-18T03:07:33.763 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:07:33.715+0000 7f019c3a7700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-18T03:07:33.763 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:07:33.715+0000 7f019c3a7700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-18T03:07:33.763 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:07:33.715+0000 7f019c3a7700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-18T03:07:33.763 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:07:33.715+0000 7f019c3a7700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-18T03:07:33.763 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:07:33.716+0000 7f019fbae700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-18T03:07:33.764 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:07:33.716+0000 7f019fbae700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-18T03:07:33.764 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:07:33.716+0000 7f019fbae700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-18T03:07:33.764 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:07:33.716+0000 7f019fbae700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-18T03:07:33.764 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:07:33.716+0000 7f019fbae700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-18T03:07:33.764 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-mon[126379]: Active manager daemon x restarted 2023-07-18T03:07:33.765 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-mon[126379]: Activating manager daemon x 2023-07-18T03:07:33.765 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-mon[126379]: osdmap e92: 12 total, 12 up, 12 in 2023-07-18T03:07:33.765 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-mon[126379]: mgrmap e21: x(active, starting, since 0.0130431s), standbys: y, z 2023-07-18T03:07:33.765 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-18T03:07:33.765 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T03:07:33.765 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T03:07:33.765 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-18T03:07:33.766 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-18T03:07:33.766 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-18T03:07:33.766 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-18T03:07:33.766 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-18T03:07:33.766 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-18T03:07:33.766 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-18T03:07:33.766 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-18T03:07:33.767 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-18T03:07:33.767 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-18T03:07:33.767 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-18T03:07:33.767 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-18T03:07:33.767 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-18T03:07:33.767 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-18T03:07:33.767 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-18T03:07:33.767 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-18T03:07:33.768 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-18T03:07:33.768 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-18T03:07:33.768 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-18T03:07:33.768 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-18T03:07:33.768 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-18T03:07:33.768 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-18T03:07:33.768 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-18T03:07:33.769 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-18T03:07:33.769 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-18T03:07:33.769 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-18T03:07:33.769 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-18T03:07:33.769 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-18T03:07:33.769 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-18T03:07:33.769 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-18T03:07:33.770 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-mon[126379]: Manager daemon x is now available 2023-07-18T03:07:33.770 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:07:33.770 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:33 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:07:33.944 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:33 smithi179 ceph-mon[129229]: Active manager daemon x restarted 2023-07-18T03:07:33.945 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:33 smithi179 ceph-mon[129229]: Activating manager daemon x 2023-07-18T03:07:33.945 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:33 smithi179 ceph-mon[129229]: osdmap e92: 12 total, 12 up, 12 in 2023-07-18T03:07:33.945 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:33 smithi179 ceph-mon[129229]: mgrmap e21: x(active, starting, since 0.0130431s), standbys: y, z 2023-07-18T03:07:33.945 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:33 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-18T03:07:33.945 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:33 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T03:07:33.945 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:33 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T03:07:33.946 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:33 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-18T03:07:33.946 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:33 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-18T03:07:33.946 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:33 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-18T03:07:33.946 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:33 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-18T03:07:33.946 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:33 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-18T03:07:33.946 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:33 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-18T03:07:33.946 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:33 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-18T03:07:33.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:33 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-18T03:07:33.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:33 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-18T03:07:33.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:33 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-18T03:07:33.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:33 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-18T03:07:33.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:33 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-18T03:07:33.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:33 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-18T03:07:33.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:33 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-18T03:07:33.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:33 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-18T03:07:33.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:33 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-18T03:07:33.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:33 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-18T03:07:33.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:33 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-18T03:07:33.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:33 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-18T03:07:33.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:33 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-18T03:07:33.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:33 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-18T03:07:33.948 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:33 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-18T03:07:33.949 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:33 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-18T03:07:33.949 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:33 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-18T03:07:33.949 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:33 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-18T03:07:33.949 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:33 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-18T03:07:33.949 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:33 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-18T03:07:33.949 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:33 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-18T03:07:33.949 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:33 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-18T03:07:33.950 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:33 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-18T03:07:33.950 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:33 smithi179 ceph-mon[129229]: Manager daemon x is now available 2023-07-18T03:07:33.950 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:33 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:07:33.950 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:33 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:07:34.039 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:33 smithi155 ceph-mon[128885]: Active manager daemon x restarted 2023-07-18T03:07:34.040 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:33 smithi155 ceph-mon[128885]: Activating manager daemon x 2023-07-18T03:07:34.040 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:33 smithi155 ceph-mon[128885]: osdmap e92: 12 total, 12 up, 12 in 2023-07-18T03:07:34.040 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:33 smithi155 ceph-mon[128885]: mgrmap e21: x(active, starting, since 0.0130431s), standbys: y, z 2023-07-18T03:07:34.040 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:33 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-18T03:07:34.040 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:33 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T03:07:34.041 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:33 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T03:07:34.041 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:33 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-18T03:07:34.041 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:33 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-18T03:07:34.041 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:33 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-18T03:07:34.041 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:33 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-18T03:07:34.041 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:33 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-18T03:07:34.041 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:33 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-18T03:07:34.041 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:33 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-18T03:07:34.042 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:33 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-18T03:07:34.042 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:33 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-18T03:07:34.042 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:33 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-18T03:07:34.042 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:33 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-18T03:07:34.042 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:33 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-18T03:07:34.042 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:33 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-18T03:07:34.042 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:33 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-18T03:07:34.043 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:33 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-18T03:07:34.043 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:33 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-18T03:07:34.043 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:33 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-18T03:07:34.043 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:33 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-18T03:07:34.043 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:33 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-18T03:07:34.043 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:33 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-18T03:07:34.043 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:33 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-18T03:07:34.043 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:33 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-18T03:07:34.044 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:33 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-18T03:07:34.044 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:33 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-18T03:07:34.044 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:33 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-18T03:07:34.044 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:33 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-18T03:07:34.044 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:33 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-18T03:07:34.044 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:33 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-18T03:07:34.044 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:33 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-18T03:07:34.045 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:33 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-18T03:07:34.045 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:33 smithi155 ceph-mon[128885]: Manager daemon x is now available 2023-07-18T03:07:34.045 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:33 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:07:34.045 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:33 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:07:34.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.517+0000 7f8d009fd700 1 -- 172.21.15.101:0/35905659 <== mon.1 v2:172.21.15.179:3300/0 9 ==== mgrmap(e 22) v1 ==== 135527+0+0 (secure 0 0 0) 0x7f8cec0cda70 con 0x7f8cfc0ffb10 2023-07-18T03:07:34.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.518+0000 7f8d009fd700 1 --2- 172.21.15.101:0/35905659 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f8ce401b4d0 0x7f8ce401b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:34.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.518+0000 7f8d009fd700 1 -- 172.21.15.101:0/35905659 --> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f8cfc1a05a0 con 0x7f8ce401b4d0 2023-07-18T03:07:34.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.521+0000 7f8cfa7fc700 1 --2- 172.21.15.101:0/35905659 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f8ce401b4d0 0x7f8ce401b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:34.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.525+0000 7f8cfa7fc700 1 --2- 172.21.15.101:0/35905659 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f8ce401b4d0 0x7f8ce401b8a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8cf000b2e0 tx=0x7f8cf0005fb0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:34.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.586+0000 7f8d009fd700 1 -- 172.21.15.101:0/35905659 <== mgr.15486 v2:172.21.15.101:6800/486424816 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f8cfc1a05a0 con 0x7f8ce401b4d0 2023-07-18T03:07:34.575 INFO:teuthology.orchestra.run.smithi101.stdout:Schedule set for path / 2023-07-18T03:07:34.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.589+0000 7f8d019ff700 1 -- 172.21.15.101:0/35905659 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f8ce401b4d0 msgr2=0x7f8ce401b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:34.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.589+0000 7f8d019ff700 1 --2- 172.21.15.101:0/35905659 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f8ce401b4d0 0x7f8ce401b8a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8cf000b2e0 tx=0x7f8cf0005fb0 comp rx=0 tx=0).stop 2023-07-18T03:07:34.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.589+0000 7f8d019ff700 1 -- 172.21.15.101:0/35905659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cfc0ffb10 msgr2=0x7f8cfc197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:34.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.589+0000 7f8d019ff700 1 --2- 172.21.15.101:0/35905659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cfc0ffb10 0x7f8cfc197b30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8cec00e970 tx=0x7f8cec00ec80 comp rx=0 tx=0).stop 2023-07-18T03:07:34.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.589+0000 7f8cfb7fe700 1 -- 172.21.15.101:0/35905659 reap_dead start 2023-07-18T03:07:34.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.589+0000 7f8d019ff700 1 -- 172.21.15.101:0/35905659 shutdown_connections 2023-07-18T03:07:34.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.589+0000 7f8d019ff700 1 -- 172.21.15.101:0/35905659 >> 172.21.15.101:0/35905659 conn(0x7f8cfc0fb130 msgr2=0x7f8cfc0fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:34.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.589+0000 7f8d019ff700 1 -- 172.21.15.101:0/35905659 shutdown_connections 2023-07-18T03:07:34.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.589+0000 7f8d019ff700 1 -- 172.21.15.101:0/35905659 wait complete. 2023-07-18T03:07:34.597 DEBUG:teuthology.orchestra.run.smithi101:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6M3h' 2023-07-18T03:07:34.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.818+0000 7f101a79d700 1 -- 172.21.15.101:0/884583498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10140ff940 msgr2=0x7f10140ffd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:34.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.818+0000 7f101a79d700 1 --2- 172.21.15.101:0/884583498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10140ff940 0x7f10140ffd50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f10100098f0 tx=0x7f1010009c00 comp rx=0 tx=0).stop 2023-07-18T03:07:34.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.818+0000 7f101a79d700 1 -- 172.21.15.101:0/884583498 shutdown_connections 2023-07-18T03:07:34.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.818+0000 7f101a79d700 1 --2- 172.21.15.101:0/884583498 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f10141014c0 0x7f1014107d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:34.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.818+0000 7f101a79d700 1 --2- 172.21.15.101:0/884583498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10140ff940 0x7f10140ffd50 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:34.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.818+0000 7f101a79d700 1 --2- 172.21.15.101:0/884583498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1014100b30 0x7f1014100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:34.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.818+0000 7f101a79d700 1 -- 172.21.15.101:0/884583498 >> 172.21.15.101:0/884583498 conn(0x7f10140faf60 msgr2=0x7f10140fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:34.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.819+0000 7f101a79d700 1 -- 172.21.15.101:0/884583498 shutdown_connections 2023-07-18T03:07:34.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.819+0000 7f101a79d700 1 -- 172.21.15.101:0/884583498 wait complete. 2023-07-18T03:07:34.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.820+0000 7f101a79d700 1 Processor -- start 2023-07-18T03:07:34.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.820+0000 7f101a79d700 1 -- start start 2023-07-18T03:07:34.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.820+0000 7f101a79d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f10140ff940 0x7f10141978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:34.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.820+0000 7f101a79d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1014100b30 0x7f1014197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:34.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.820+0000 7f101a79d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10141014c0 0x7f101419ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:34.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.820+0000 7f101a79d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101419f460 con 0x7f10140ff940 2023-07-18T03:07:34.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.820+0000 7f101a79d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101419f5a0 con 0x7f1014100b30 2023-07-18T03:07:34.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.820+0000 7f101a79d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101419f730 con 0x7f10141014c0 2023-07-18T03:07:34.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.820+0000 7f101979b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f10140ff940 0x7f10141978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:34.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.820+0000 7f1018f9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1014100b30 0x7f1014197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:34.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.820+0000 7f101979b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f10140ff940 0x7f10141978e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:56250/0 (socket says 172.21.15.101:56250) 2023-07-18T03:07:34.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.820+0000 7f101979b700 1 -- 172.21.15.101:0/1492389429 learned_addr learned my addr 172.21.15.101:0/1492389429 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:34.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.820+0000 7f1019f9c700 1 --2- 172.21.15.101:0/1492389429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10141014c0 0x7f101419ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:34.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.821+0000 7f101979b700 1 -- 172.21.15.101:0/1492389429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10141014c0 msgr2=0x7f101419ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:34.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.821+0000 7f101979b700 1 --2- 172.21.15.101:0/1492389429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10141014c0 0x7f101419ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:34.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.821+0000 7f101979b700 1 -- 172.21.15.101:0/1492389429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1014100b30 msgr2=0x7f1014197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:34.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.821+0000 7f101979b700 1 --2- 172.21.15.101:0/1492389429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1014100b30 0x7f1014197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:34.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.821+0000 7f101979b700 1 -- 172.21.15.101:0/1492389429 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10040095f0 con 0x7f10140ff940 2023-07-18T03:07:34.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.821+0000 7f1018f9a700 1 --2- 172.21.15.101:0/1492389429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1014100b30 0x7f1014197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:07:34.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.821+0000 7f101979b700 1 --2- 172.21.15.101:0/1492389429 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f10140ff940 0x7f10141978e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1010009fd0 tx=0x7f101000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:34.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.821+0000 7f100a7fc700 1 -- 172.21.15.101:0/1492389429 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101001c070 con 0x7f10140ff940 2023-07-18T03:07:34.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.821+0000 7f101a79d700 1 -- 172.21.15.101:0/1492389429 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1010009520 con 0x7f10140ff940 2023-07-18T03:07:34.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.821+0000 7f100a7fc700 1 -- 172.21.15.101:0/1492389429 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f101000f7d0 con 0x7f10140ff940 2023-07-18T03:07:34.813 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.821+0000 7f100a7fc700 1 -- 172.21.15.101:0/1492389429 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1010003b00 con 0x7f10140ff940 2023-07-18T03:07:34.813 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.821+0000 7f101a79d700 1 -- 172.21.15.101:0/1492389429 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f101419fca0 con 0x7f10140ff940 2023-07-18T03:07:34.813 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.823+0000 7f101a79d700 1 -- 172.21.15.101:0/1492389429 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1014191fc0 con 0x7f10140ff940 2023-07-18T03:07:34.813 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.824+0000 7f100a7fc700 1 -- 172.21.15.101:0/1492389429 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 22) v1 ==== 135527+0+0 (secure 0 0 0) 0x7f1010020030 con 0x7f10140ff940 2023-07-18T03:07:34.814 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.826+0000 7f100a7fc700 1 --2- 172.21.15.101:0/1492389429 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f10000a0f50 0x7f10000a3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:34.814 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.826+0000 7f100a7fc700 1 -- 172.21.15.101:0/1492389429 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f10100cd9e0 con 0x7f10140ff940 2023-07-18T03:07:34.814 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.826+0000 7f1018f9a700 1 --2- 172.21.15.101:0/1492389429 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f10000a0f50 0x7f10000a3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:34.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.827+0000 7f1018f9a700 1 --2- 172.21.15.101:0/1492389429 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f10000a0f50 0x7f10000a3400 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1004005fd0 tx=0x7f1004009310 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:34.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:34.829+0000 7f100a7fc700 1 -- 172.21.15.101:0/1492389429 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1010097000 con 0x7f10140ff940 2023-07-18T03:07:34.944 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:34 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-18T03:07:34.944 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:34 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-18T03:07:34.945 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:34 smithi179 ceph-mon[129229]: [18/Jul/2023:03:07:33] ENGINE Bus STARTING 2023-07-18T03:07:34.945 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:34 smithi179 ceph-mon[129229]: [18/Jul/2023:03:07:34] ENGINE Serving on http://172.21.15.101:8765 2023-07-18T03:07:34.945 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:34 smithi179 ceph-mon[129229]: mgrmap e22: x(active, since 1.02062s), standbys: y, z 2023-07-18T03:07:34.982 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:34 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-18T03:07:34.982 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:34 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-18T03:07:34.982 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:34 smithi101 ceph-mon[126379]: [18/Jul/2023:03:07:33] ENGINE Bus STARTING 2023-07-18T03:07:34.982 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:34 smithi101 ceph-mon[126379]: [18/Jul/2023:03:07:34] ENGINE Serving on http://172.21.15.101:8765 2023-07-18T03:07:34.982 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:34 smithi101 ceph-mon[126379]: mgrmap e22: x(active, since 1.02062s), standbys: y, z 2023-07-18T03:07:35.039 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:34 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-18T03:07:35.040 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:34 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-18T03:07:35.040 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:34 smithi155 ceph-mon[128885]: [18/Jul/2023:03:07:33] ENGINE Bus STARTING 2023-07-18T03:07:35.040 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:34 smithi155 ceph-mon[128885]: [18/Jul/2023:03:07:34] ENGINE Serving on http://172.21.15.101:8765 2023-07-18T03:07:35.040 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:34 smithi155 ceph-mon[128885]: mgrmap e22: x(active, since 1.02062s), standbys: y, z 2023-07-18T03:07:35.059 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.071+0000 7f101a79d700 1 -- 172.21.15.101:0/1492389429 --> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}) v1 -- 0x7f1014060950 con 0x7f10000a0f50 2023-07-18T03:07:35.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.124+0000 7f100a7fc700 1 -- 172.21.15.101:0/1492389429 <== mgr.15486 v2:172.21.15.101:6800/486424816 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f1014060950 con 0x7f10000a0f50 2023-07-18T03:07:35.113 INFO:teuthology.orchestra.run.smithi101.stdout:Retention added to path / 2023-07-18T03:07:35.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.127+0000 7f101a79d700 1 -- 172.21.15.101:0/1492389429 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f10000a0f50 msgr2=0x7f10000a3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:35.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.127+0000 7f101a79d700 1 --2- 172.21.15.101:0/1492389429 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f10000a0f50 0x7f10000a3400 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1004005fd0 tx=0x7f1004009310 comp rx=0 tx=0).stop 2023-07-18T03:07:35.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.127+0000 7f101a79d700 1 -- 172.21.15.101:0/1492389429 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f10140ff940 msgr2=0x7f10141978e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:35.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.127+0000 7f101a79d700 1 --2- 172.21.15.101:0/1492389429 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f10140ff940 0x7f10141978e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1010009fd0 tx=0x7f101000b790 comp rx=0 tx=0).stop 2023-07-18T03:07:35.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.127+0000 7f101a79d700 1 -- 172.21.15.101:0/1492389429 shutdown_connections 2023-07-18T03:07:35.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.127+0000 7f101a79d700 1 --2- 172.21.15.101:0/1492389429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10141014c0 0x7f101419ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:35.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.127+0000 7f101a79d700 1 --2- 172.21.15.101:0/1492389429 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f10140ff940 0x7f10141978e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:35.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.127+0000 7f101a79d700 1 --2- 172.21.15.101:0/1492389429 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f10000a0f50 0x7f10000a3400 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:35.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.127+0000 7f101a79d700 1 --2- 172.21.15.101:0/1492389429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1014100b30 0x7f1014197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:35.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.127+0000 7f101a79d700 1 -- 172.21.15.101:0/1492389429 >> 172.21.15.101:0/1492389429 conn(0x7f10140faf60 msgr2=0x7f10141064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:35.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.127+0000 7f101a79d700 1 -- 172.21.15.101:0/1492389429 shutdown_connections 2023-07-18T03:07:35.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.127+0000 7f101a79d700 1 -- 172.21.15.101:0/1492389429 wait complete. 2023-07-18T03:07:35.134 DEBUG:teuthology.orchestra.run.smithi101:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-07-18T03:07:35.338 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.350+0000 7f82df1ff700 1 -- 172.21.15.101:0/3158236580 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f82d8074550 msgr2=0x7f82d80729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:35.339 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.350+0000 7f82df1ff700 1 --2- 172.21.15.101:0/3158236580 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f82d8074550 0x7f82d80729a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f82cc009990 tx=0x7f82cc009ca0 comp rx=0 tx=0).stop 2023-07-18T03:07:35.339 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.350+0000 7f82df1ff700 1 -- 172.21.15.101:0/3158236580 shutdown_connections 2023-07-18T03:07:35.339 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.350+0000 7f82df1ff700 1 --2- 172.21.15.101:0/3158236580 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82d81079d0 0x7f82d8109e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:35.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.350+0000 7f82df1ff700 1 --2- 172.21.15.101:0/3158236580 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f82d8074550 0x7f82d80729a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:35.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.350+0000 7f82df1ff700 1 --2- 172.21.15.101:0/3158236580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82d8072f70 0x7f82d80733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:35.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.350+0000 7f82df1ff700 1 -- 172.21.15.101:0/3158236580 >> 172.21.15.101:0/3158236580 conn(0x7f82d80fb130 msgr2=0x7f82d80fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:35.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.350+0000 7f82df1ff700 1 -- 172.21.15.101:0/3158236580 shutdown_connections 2023-07-18T03:07:35.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.350+0000 7f82df1ff700 1 -- 172.21.15.101:0/3158236580 wait complete. 2023-07-18T03:07:35.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.351+0000 7f82df1ff700 1 Processor -- start 2023-07-18T03:07:35.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.351+0000 7f82df1ff700 1 -- start start 2023-07-18T03:07:35.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.351+0000 7f82df1ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82d8072f70 0x7f82d8197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:35.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.352+0000 7f82df1ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82d8074550 0x7f82d81980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:35.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.352+0000 7f82df1ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f82d81079d0 0x7f82d8198f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:35.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.352+0000 7f82df1ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d81995c0 con 0x7f82d81079d0 2023-07-18T03:07:35.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.352+0000 7f82df1ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d8199730 con 0x7f82d8072f70 2023-07-18T03:07:35.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.352+0000 7f82df1ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d81998f0 con 0x7f82d8074550 2023-07-18T03:07:35.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.352+0000 7f82d7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82d8074550 0x7f82d81980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:35.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.352+0000 7f82dd79c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f82d81079d0 0x7f82d8198f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:35.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.352+0000 7f82d7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82d8074550 0x7f82d81980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:58660/0 (socket says 172.21.15.101:58660) 2023-07-18T03:07:35.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.352+0000 7f82d7fff700 1 -- 172.21.15.101:0/3628363513 learned_addr learned my addr 172.21.15.101:0/3628363513 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:35.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.352+0000 7f82dcf9b700 1 --2- 172.21.15.101:0/3628363513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82d8072f70 0x7f82d8197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:35.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.352+0000 7f82d7fff700 1 -- 172.21.15.101:0/3628363513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82d8072f70 msgr2=0x7f82d8197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:35.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.352+0000 7f82d7fff700 1 --2- 172.21.15.101:0/3628363513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82d8072f70 0x7f82d8197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:35.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.352+0000 7f82d7fff700 1 -- 172.21.15.101:0/3628363513 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f82d81079d0 msgr2=0x7f82d8198f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:35.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.352+0000 7f82d7fff700 1 --2- 172.21.15.101:0/3628363513 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f82d81079d0 0x7f82d8198f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:35.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.352+0000 7f82d7fff700 1 -- 172.21.15.101:0/3628363513 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c4009520 con 0x7f82d8074550 2023-07-18T03:07:35.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.353+0000 7f82d7fff700 1 --2- 172.21.15.101:0/3628363513 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82d8074550 0x7f82d81980e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f82c400ea50 tx=0x7f82c400ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:35.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.353+0000 7f82d5ffb700 1 -- 172.21.15.101:0/3628363513 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c400cbd0 con 0x7f82d8074550 2023-07-18T03:07:35.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.353+0000 7f82d5ffb700 1 -- 172.21.15.101:0/3628363513 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82c400cd30 con 0x7f82d8074550 2023-07-18T03:07:35.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.353+0000 7f82df1ff700 1 -- 172.21.15.101:0/3628363513 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82cc0095f0 con 0x7f82d8074550 2023-07-18T03:07:35.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.353+0000 7f82d5ffb700 1 -- 172.21.15.101:0/3628363513 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c4018680 con 0x7f82d8074550 2023-07-18T03:07:35.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.353+0000 7f82df1ff700 1 -- 172.21.15.101:0/3628363513 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82d8199dc0 con 0x7f82d8074550 2023-07-18T03:07:35.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.354+0000 7f82df1ff700 1 -- 172.21.15.101:0/3628363513 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82d8106080 con 0x7f82d8074550 2023-07-18T03:07:35.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.356+0000 7f82d5ffb700 1 -- 172.21.15.101:0/3628363513 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 22) v1 ==== 135527+0+0 (secure 0 0 0) 0x7f82c4010430 con 0x7f82d8074550 2023-07-18T03:07:35.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.357+0000 7f82d5ffb700 1 --2- 172.21.15.101:0/3628363513 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f82c80a0fd0 0x7f82c80a3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:35.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.358+0000 7f82d5ffb700 1 -- 172.21.15.101:0/3628363513 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f82c4014070 con 0x7f82d8074550 2023-07-18T03:07:35.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.358+0000 7f82dcf9b700 1 --2- 172.21.15.101:0/3628363513 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f82c80a0fd0 0x7f82c80a3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:35.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.358+0000 7f82dcf9b700 1 --2- 172.21.15.101:0/3628363513 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f82c80a0fd0 0x7f82c80a3480 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f82cc00b2e0 tx=0x7f82cc005fb0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:35.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.359+0000 7f82d5ffb700 1 -- 172.21.15.101:0/3628363513 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f82c4096de0 con 0x7f82d8074550 2023-07-18T03:07:35.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.614+0000 7f82df1ff700 1 -- 172.21.15.101:0/3628363513 --> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f82d8060950 con 0x7f82c80a0fd0 2023-07-18T03:07:35.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.621+0000 7f82d5ffb700 1 -- 172.21.15.101:0/3628363513 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mgrmap(e 23) v1 ==== 135892+0+0 (secure 0 0 0) 0x7f82c40971b0 con 0x7f82d8074550 2023-07-18T03:07:35.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.629+0000 7f82d5ffb700 1 -- 172.21.15.101:0/3628363513 <== mgr.15486 v2:172.21.15.101:6800/486424816 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+283 (secure 0 0 0) 0x7f82d8060950 con 0x7f82c80a0fd0 2023-07-18T03:07:35.617 INFO:teuthology.orchestra.run.smithi101.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1M", "retention": {"h": 3, "M": 6}, "start": "2023-07-18T00:00:00", "created": "2023-07-18T03:07:34", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2023-07-18T03:07:35.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.631+0000 7f82df1ff700 1 -- 172.21.15.101:0/3628363513 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f82c80a0fd0 msgr2=0x7f82c80a3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:35.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.631+0000 7f82df1ff700 1 --2- 172.21.15.101:0/3628363513 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f82c80a0fd0 0x7f82c80a3480 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f82cc00b2e0 tx=0x7f82cc005fb0 comp rx=0 tx=0).stop 2023-07-18T03:07:35.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.631+0000 7f82df1ff700 1 -- 172.21.15.101:0/3628363513 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82d8074550 msgr2=0x7f82d81980e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:35.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.631+0000 7f82df1ff700 1 --2- 172.21.15.101:0/3628363513 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82d8074550 0x7f82d81980e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f82c400ea50 tx=0x7f82c400ed60 comp rx=0 tx=0).stop 2023-07-18T03:07:35.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.632+0000 7f82df1ff700 1 -- 172.21.15.101:0/3628363513 shutdown_connections 2023-07-18T03:07:35.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.632+0000 7f82df1ff700 1 --2- 172.21.15.101:0/3628363513 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f82c80a0fd0 0x7f82c80a3480 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:35.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.632+0000 7f82df1ff700 1 --2- 172.21.15.101:0/3628363513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82d8072f70 0x7f82d8197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:35.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.632+0000 7f82df1ff700 1 --2- 172.21.15.101:0/3628363513 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f82d81079d0 0x7f82d8198f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:35.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.632+0000 7f82df1ff700 1 --2- 172.21.15.101:0/3628363513 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82d8074550 0x7f82d81980e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:35.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.632+0000 7f82df1ff700 1 -- 172.21.15.101:0/3628363513 >> 172.21.15.101:0/3628363513 conn(0x7f82d80fb130 msgr2=0x7f82d80fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:35.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.632+0000 7f82df1ff700 1 -- 172.21.15.101:0/3628363513 shutdown_connections 2023-07-18T03:07:35.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.632+0000 7f82df1ff700 1 -- 172.21.15.101:0/3628363513 wait complete. 2023-07-18T03:07:35.640 DEBUG:teuthology.orchestra.run.smithi101:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-07-18T03:07:35.848 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.859+0000 7f90fc7ff700 1 -- 172.21.15.101:0/981123272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90f4100d00 msgr2=0x7f90f4101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:35.848 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.859+0000 7f90fc7ff700 1 --2- 172.21.15.101:0/981123272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90f4100d00 0x7f90f4101150 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f90e4009990 tx=0x7f90e4009ca0 comp rx=0 tx=0).stop 2023-07-18T03:07:35.848 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.860+0000 7f90fc7ff700 1 -- 172.21.15.101:0/981123272 shutdown_connections 2023-07-18T03:07:35.848 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.860+0000 7f90fc7ff700 1 --2- 172.21.15.101:0/981123272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90f4101690 0x7f90f4107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:35.848 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.860+0000 7f90fc7ff700 1 --2- 172.21.15.101:0/981123272 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f90f40ffb10 0x7f90f40fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:35.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.860+0000 7f90fc7ff700 1 --2- 172.21.15.101:0/981123272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90f4100d00 0x7f90f4101150 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:35.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.860+0000 7f90fc7ff700 1 -- 172.21.15.101:0/981123272 >> 172.21.15.101:0/981123272 conn(0x7f90f40fb130 msgr2=0x7f90f40fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:35.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.860+0000 7f90fc7ff700 1 -- 172.21.15.101:0/981123272 shutdown_connections 2023-07-18T03:07:35.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.860+0000 7f90fc7ff700 1 -- 172.21.15.101:0/981123272 wait complete. 2023-07-18T03:07:35.851 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.861+0000 7f90fc7ff700 1 Processor -- start 2023-07-18T03:07:35.851 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.861+0000 7f90fc7ff700 1 -- start start 2023-07-18T03:07:35.851 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.861+0000 7f90fc7ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90f40ffb10 0x7f90f4072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:35.851 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.861+0000 7f90fc7ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f90f4100d00 0x7f90f4072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:35.851 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.861+0000 7f90fc7ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90f4101690 0x7f90f4073380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:35.851 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.861+0000 7f90fc7ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f419f890 con 0x7f90f4100d00 2023-07-18T03:07:35.852 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.861+0000 7f90fc7ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f419f9d0 con 0x7f90f40ffb10 2023-07-18T03:07:35.852 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.861+0000 7f90fc7ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f419fb10 con 0x7f90f4101690 2023-07-18T03:07:35.852 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.862+0000 7f90fad9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90f4101690 0x7f90f4073380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:35.852 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.862+0000 7f90f9d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f90f4100d00 0x7f90f4072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:35.852 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.862+0000 7f90fad9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90f4101690 0x7f90f4073380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:51026/0 (socket says 172.21.15.101:51026) 2023-07-18T03:07:35.852 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.862+0000 7f90fad9c700 1 -- 172.21.15.101:0/2842814521 learned_addr learned my addr 172.21.15.101:0/2842814521 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:35.852 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.862+0000 7f90fa59b700 1 --2- 172.21.15.101:0/2842814521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90f40ffb10 0x7f90f4072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:35.853 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.862+0000 7f90fad9c700 1 -- 172.21.15.101:0/2842814521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90f40ffb10 msgr2=0x7f90f4072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:35.853 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.862+0000 7f90fad9c700 1 --2- 172.21.15.101:0/2842814521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90f40ffb10 0x7f90f4072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:35.853 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.862+0000 7f90fad9c700 1 -- 172.21.15.101:0/2842814521 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f90f4100d00 msgr2=0x7f90f4072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:35.853 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.862+0000 7f90fad9c700 1 --2- 172.21.15.101:0/2842814521 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f90f4100d00 0x7f90f4072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:35.853 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.862+0000 7f90fad9c700 1 -- 172.21.15.101:0/2842814521 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f0009520 con 0x7f90f4101690 2023-07-18T03:07:35.853 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.862+0000 7f90f9d9a700 1 --2- 172.21.15.101:0/2842814521 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f90f4100d00 0x7f90f4072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:07:35.853 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.862+0000 7f90fad9c700 1 --2- 172.21.15.101:0/2842814521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90f4101690 0x7f90f4073380 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f90ec00ef30 tx=0x7f90ec00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:35.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.862+0000 7f90eb7fe700 1 -- 172.21.15.101:0/2842814521 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ec018070 con 0x7f90f4101690 2023-07-18T03:07:35.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.862+0000 7f90fc7ff700 1 -- 172.21.15.101:0/2842814521 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90e40095f0 con 0x7f90f4101690 2023-07-18T03:07:35.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.862+0000 7f90eb7fe700 1 -- 172.21.15.101:0/2842814521 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90ec00f040 con 0x7f90f4101690 2023-07-18T03:07:35.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.862+0000 7f90eb7fe700 1 -- 172.21.15.101:0/2842814521 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ec00cd00 con 0x7f90f4101690 2023-07-18T03:07:35.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.863+0000 7f90fc7ff700 1 -- 172.21.15.101:0/2842814521 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90f41a00c0 con 0x7f90f4101690 2023-07-18T03:07:35.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.863+0000 7f90fc7ff700 1 -- 172.21.15.101:0/2842814521 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90f41070c0 con 0x7f90f4101690 2023-07-18T03:07:35.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.865+0000 7f90eb7fe700 1 -- 172.21.15.101:0/2842814521 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 23) v1 ==== 135892+0+0 (secure 0 0 0) 0x7f90ec003a00 con 0x7f90f4101690 2023-07-18T03:07:35.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.867+0000 7f90eb7fe700 1 --2- 172.21.15.101:0/2842814521 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f90e00a1350 0x7f90e00a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:35.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.867+0000 7f90eb7fe700 1 -- 172.21.15.101:0/2842814521 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f90ec0ccf10 con 0x7f90f4101690 2023-07-18T03:07:35.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.867+0000 7f90fa59b700 1 --2- 172.21.15.101:0/2842814521 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f90e00a1350 0x7f90e00a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:35.856 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.867+0000 7f90fa59b700 1 --2- 172.21.15.101:0/2842814521 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f90e00a1350 0x7f90e00a3800 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f90f4074210 tx=0x7f90f0009260 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:35.857 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:35.869+0000 7f90eb7fe700 1 -- 172.21.15.101:0/2842814521 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f90ec095e90 con 0x7f90f4101690 2023-07-18T03:07:35.944 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:35 smithi179 ceph-mon[129229]: [18/Jul/2023:03:07:34] ENGINE Serving on https://172.21.15.101:7150 2023-07-18T03:07:35.945 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:35 smithi179 ceph-mon[129229]: [18/Jul/2023:03:07:34] ENGINE Bus STARTED 2023-07-18T03:07:35.945 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:35 smithi179 ceph-mon[129229]: from='client.25561 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:07:35.945 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:35 smithi179 ceph-mon[129229]: pgmap v3: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:07:35.945 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:35 smithi179 ceph-mon[129229]: from='client.15546 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:07:35.945 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:35 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:07:35.982 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:35 smithi101 ceph-mon[126379]: [18/Jul/2023:03:07:34] ENGINE Serving on https://172.21.15.101:7150 2023-07-18T03:07:35.982 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:35 smithi101 ceph-mon[126379]: [18/Jul/2023:03:07:34] ENGINE Bus STARTED 2023-07-18T03:07:35.982 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:35 smithi101 ceph-mon[126379]: from='client.25561 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:07:35.982 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:35 smithi101 ceph-mon[126379]: pgmap v3: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:07:35.982 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:35 smithi101 ceph-mon[126379]: from='client.15546 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:07:35.983 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:35 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:07:36.039 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:35 smithi155 ceph-mon[128885]: [18/Jul/2023:03:07:34] ENGINE Serving on https://172.21.15.101:7150 2023-07-18T03:07:36.040 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:35 smithi155 ceph-mon[128885]: [18/Jul/2023:03:07:34] ENGINE Bus STARTED 2023-07-18T03:07:36.040 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:35 smithi155 ceph-mon[128885]: from='client.25561 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:07:36.040 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:35 smithi155 ceph-mon[128885]: pgmap v3: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:07:36.040 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:35 smithi155 ceph-mon[128885]: from='client.15546 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:07:36.040 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:35 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:07:36.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:36.115+0000 7f90fc7ff700 1 -- 172.21.15.101:0/2842814521 --> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f90f4060950 con 0x7f90e00a1350 2023-07-18T03:07:36.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:36.123+0000 7f90eb7fe700 1 -- 172.21.15.101:0/2842814521 <== mgr.15486 v2:172.21.15.101:6800/486424816 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f90f4060950 con 0x7f90e00a1350 2023-07-18T03:07:36.111 INFO:teuthology.orchestra.run.smithi101.stdout:/ 1M 6M3h 2023-07-18T03:07:36.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:36.126+0000 7f90fc7ff700 1 -- 172.21.15.101:0/2842814521 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f90e00a1350 msgr2=0x7f90e00a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:36.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:36.126+0000 7f90fc7ff700 1 --2- 172.21.15.101:0/2842814521 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f90e00a1350 0x7f90e00a3800 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f90f4074210 tx=0x7f90f0009260 comp rx=0 tx=0).stop 2023-07-18T03:07:36.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:36.126+0000 7f90fc7ff700 1 -- 172.21.15.101:0/2842814521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90f4101690 msgr2=0x7f90f4073380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:36.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:36.126+0000 7f90fc7ff700 1 --2- 172.21.15.101:0/2842814521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90f4101690 0x7f90f4073380 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f90ec00ef30 tx=0x7f90ec00c3a0 comp rx=0 tx=0).stop 2023-07-18T03:07:36.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:36.126+0000 7f90fc7ff700 1 -- 172.21.15.101:0/2842814521 shutdown_connections 2023-07-18T03:07:36.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:36.126+0000 7f90fc7ff700 1 --2- 172.21.15.101:0/2842814521 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f90e00a1350 0x7f90e00a3800 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:36.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:36.126+0000 7f90fc7ff700 1 --2- 172.21.15.101:0/2842814521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90f40ffb10 0x7f90f4072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:36.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:36.126+0000 7f90fc7ff700 1 --2- 172.21.15.101:0/2842814521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90f4101690 0x7f90f4073380 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:36.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:36.126+0000 7f90fc7ff700 1 --2- 172.21.15.101:0/2842814521 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f90f4100d00 0x7f90f4072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:36.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:36.126+0000 7f90fc7ff700 1 -- 172.21.15.101:0/2842814521 >> 172.21.15.101:0/2842814521 conn(0x7f90f40fb130 msgr2=0x7f90f4104200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:36.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:36.126+0000 7f90fc7ff700 1 -- 172.21.15.101:0/2842814521 shutdown_connections 2023-07-18T03:07:36.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:36.126+0000 7f90fc7ff700 1 -- 172.21.15.101:0/2842814521 wait complete. 2023-07-18T03:07:36.133 DEBUG:teuthology.orchestra.run.smithi101:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2023-07-18T03:07:36.211 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2023-07-18T03:07:36.222 INFO:teuthology.run_tasks:Running task workunit... 2023-07-18T03:07:36.234 INFO:tasks.workunit:Pulling workunits from ref 81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-18T03:07:36.235 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-07-18T03:07:36.235 INFO:tasks.workunit:timeout=3h 2023-07-18T03:07:36.236 INFO:tasks.workunit:cleanup=True 2023-07-18T03:07:36.236 DEBUG:teuthology.orchestra.run.smithi101:> stat -- /home/ubuntu/cephtest/mnt.0 2023-07-18T03:07:36.281 INFO:teuthology.orchestra.run.smithi101.stdout: File: /home/ubuntu/cephtest/mnt.0 2023-07-18T03:07:36.282 INFO:teuthology.orchestra.run.smithi101.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2023-07-18T03:07:36.282 INFO:teuthology.orchestra.run.smithi101.stdout:Device: 79h/121d Inode: 1099511628283 Links: 2 2023-07-18T03:07:36.282 INFO:teuthology.orchestra.run.smithi101.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-18T03:07:36.283 INFO:teuthology.orchestra.run.smithi101.stdout:Context: system_u:object_r:cephfs_t:s0 2023-07-18T03:07:36.283 INFO:teuthology.orchestra.run.smithi101.stdout:Access: 2023-07-18 03:05:33.449052970 +0000 2023-07-18T03:07:36.283 INFO:teuthology.orchestra.run.smithi101.stdout:Modify: 2023-07-18 03:05:33.449052970 +0000 2023-07-18T03:07:36.283 INFO:teuthology.orchestra.run.smithi101.stdout:Change: 2023-07-18 03:07:13.974431183 +0000 2023-07-18T03:07:36.284 INFO:teuthology.orchestra.run.smithi101.stdout: Birth: 2023-07-18 03:05:33.449052970 +0000 2023-07-18T03:07:36.284 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2023-07-18T03:07:36.285 DEBUG:teuthology.orchestra.run.smithi101:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2023-07-18T03:07:36.407 DEBUG:teuthology.orchestra.run.smithi101:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-18T03:07:36.453 INFO:tasks.workunit.client.0.smithi101.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-07-18T03:07:36.732 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:36 smithi101 ceph-mon[126379]: pgmap v4: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:07:36.732 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:36 smithi101 ceph-mon[126379]: from='client.25615 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:07:36.732 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:36 smithi101 ceph-mon[126379]: mgrmap e23: x(active, since 2s), standbys: y, z 2023-07-18T03:07:36.733 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:36 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:36.733 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:36 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:36.944 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:36 smithi179 ceph-mon[129229]: pgmap v4: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:07:36.945 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:36 smithi179 ceph-mon[129229]: from='client.25615 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:07:36.945 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:36 smithi179 ceph-mon[129229]: mgrmap e23: x(active, since 2s), standbys: y, z 2023-07-18T03:07:36.945 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:36 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:36.946 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:36 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:37.039 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:36 smithi155 ceph-mon[128885]: pgmap v4: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:07:37.040 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:36 smithi155 ceph-mon[128885]: from='client.25615 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:07:37.040 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:36 smithi155 ceph-mon[128885]: mgrmap e23: x(active, since 2s), standbys: y, z 2023-07-18T03:07:37.040 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:36 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:37.040 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:36 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:37.944 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:37 smithi179 ceph-mon[129229]: from='client.25621 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:07:37.945 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:37 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:37.945 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:37 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:37.945 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:37 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:37.945 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:37 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:37.945 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:37 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:07:37.945 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:37 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:07:37.946 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:37 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:07:37.946 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:37 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:07:37.946 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:37 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:37.946 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:37 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:37.946 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:37 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:07:37.946 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:37 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:07:37.946 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:37 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:07:37.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:37 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:07:37.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:37 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:37.947 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:37 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:37.982 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:37 smithi101 ceph-mon[126379]: from='client.25621 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:07:37.982 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:37 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:37.982 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:37 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:37.982 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:37 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:37.983 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:37 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:37.983 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:37 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:07:37.983 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:37 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:07:37.983 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:37 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:07:37.983 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:37 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:07:37.983 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:37 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:37.983 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:37 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:37.983 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:37 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:07:37.984 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:37 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:07:37.984 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:37 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:07:37.984 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:37 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:07:37.984 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:37 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:37.984 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:37 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:38.039 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:37 smithi155 ceph-mon[128885]: from='client.25621 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:07:38.040 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:37 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:38.040 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:37 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:38.040 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:37 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:38.040 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:37 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:38.040 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:37 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:07:38.040 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:37 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:07:38.040 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:37 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:07:38.041 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:37 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:07:38.041 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:37 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:38.041 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:37 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:38.041 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:37 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:07:38.041 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:37 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:07:38.041 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:37 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:07:38.041 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:37 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:07:38.042 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:37 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:38.042 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:37 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:38.944 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:38 smithi179 ceph-mon[129229]: Adjusting osd_memory_target on smithi155 to 196.2M 2023-07-18T03:07:38.944 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:38 smithi179 ceph-mon[129229]: Unable to set osd_memory_target on smithi155 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-07-18T03:07:38.945 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:38 smithi179 ceph-mon[129229]: Adjusting osd_memory_target on smithi179 to 196.2M 2023-07-18T03:07:38.945 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:38 smithi179 ceph-mon[129229]: Unable to set osd_memory_target on smithi179 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-07-18T03:07:38.945 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:38 smithi179 ceph-mon[129229]: pgmap v5: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:07:38.982 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:38 smithi101 ceph-mon[126379]: Adjusting osd_memory_target on smithi155 to 196.2M 2023-07-18T03:07:38.982 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:38 smithi101 ceph-mon[126379]: Unable to set osd_memory_target on smithi155 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-07-18T03:07:38.982 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:38 smithi101 ceph-mon[126379]: Adjusting osd_memory_target on smithi179 to 196.2M 2023-07-18T03:07:38.982 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:38 smithi101 ceph-mon[126379]: Unable to set osd_memory_target on smithi179 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-07-18T03:07:38.982 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:38 smithi101 ceph-mon[126379]: pgmap v5: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:07:39.039 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:38 smithi155 ceph-mon[128885]: Adjusting osd_memory_target on smithi155 to 196.2M 2023-07-18T03:07:39.039 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:38 smithi155 ceph-mon[128885]: Unable to set osd_memory_target on smithi155 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-07-18T03:07:39.039 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:38 smithi155 ceph-mon[128885]: Adjusting osd_memory_target on smithi179 to 196.2M 2023-07-18T03:07:39.040 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:38 smithi155 ceph-mon[128885]: Unable to set osd_memory_target on smithi179 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-07-18T03:07:39.040 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:38 smithi155 ceph-mon[128885]: pgmap v5: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:07:40.194 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:39 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:40.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:39 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:40.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:39 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:07:40.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:39 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:07:40.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:39 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:07:40.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:39 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:07:40.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:39 smithi179 ceph-mon[129229]: Adjusting osd_memory_target on smithi101 to 220.9M 2023-07-18T03:07:40.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:39 smithi179 ceph-mon[129229]: Unable to set osd_memory_target on smithi101 to 231653990: error parsing value: Value '231653990' is below minimum 939524096 2023-07-18T03:07:40.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:39 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:07:40.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:39 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:07:40.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:39 smithi179 ceph-mon[129229]: Updating smithi101:/etc/ceph/ceph.conf 2023-07-18T03:07:40.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:39 smithi179 ceph-mon[129229]: Updating smithi155:/etc/ceph/ceph.conf 2023-07-18T03:07:40.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:39 smithi179 ceph-mon[129229]: Updating smithi179:/etc/ceph/ceph.conf 2023-07-18T03:07:40.232 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:39 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:40.232 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:39 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:40.232 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:39 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:07:40.233 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:39 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:07:40.233 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:39 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:07:40.233 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:39 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:07:40.233 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:39 smithi101 ceph-mon[126379]: Adjusting osd_memory_target on smithi101 to 220.9M 2023-07-18T03:07:40.233 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:39 smithi101 ceph-mon[126379]: Unable to set osd_memory_target on smithi101 to 231653990: error parsing value: Value '231653990' is below minimum 939524096 2023-07-18T03:07:40.233 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:39 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:07:40.233 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:39 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:07:40.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:39 smithi101 ceph-mon[126379]: Updating smithi101:/etc/ceph/ceph.conf 2023-07-18T03:07:40.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:39 smithi101 ceph-mon[126379]: Updating smithi155:/etc/ceph/ceph.conf 2023-07-18T03:07:40.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:39 smithi101 ceph-mon[126379]: Updating smithi179:/etc/ceph/ceph.conf 2023-07-18T03:07:40.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:39 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:40.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:39 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:40.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:39 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:07:40.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:39 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:07:40.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:39 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:07:40.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:39 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:07:40.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:39 smithi155 ceph-mon[128885]: Adjusting osd_memory_target on smithi101 to 220.9M 2023-07-18T03:07:40.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:39 smithi155 ceph-mon[128885]: Unable to set osd_memory_target on smithi101 to 231653990: error parsing value: Value '231653990' is below minimum 939524096 2023-07-18T03:07:40.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:39 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:07:40.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:39 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:07:40.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:39 smithi155 ceph-mon[128885]: Updating smithi101:/etc/ceph/ceph.conf 2023-07-18T03:07:40.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:39 smithi155 ceph-mon[128885]: Updating smithi155:/etc/ceph/ceph.conf 2023-07-18T03:07:40.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:39 smithi155 ceph-mon[128885]: Updating smithi179:/etc/ceph/ceph.conf 2023-07-18T03:07:41.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:41 smithi155 ceph-mon[128885]: Updating smithi155:/var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-18T03:07:41.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:41 smithi155 ceph-mon[128885]: Updating smithi179:/var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-18T03:07:41.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:41 smithi155 ceph-mon[128885]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2023-07-18T03:07:41.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:41 smithi155 ceph-mon[128885]: Updating smithi179:/etc/ceph/ceph.client.admin.keyring 2023-07-18T03:07:41.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:41 smithi155 ceph-mon[128885]: Updating smithi101:/var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-18T03:07:41.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:41 smithi155 ceph-mon[128885]: pgmap v6: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:07:41.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:41 smithi155 ceph-mon[128885]: Updating smithi155:/var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-18T03:07:41.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:41 smithi155 ceph-mon[128885]: Updating smithi179:/var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-18T03:07:41.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:41 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:41.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:41 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:41.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:41 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:41.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:41 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:41.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:41 smithi155 ceph-mon[128885]: Updating smithi101:/etc/ceph/ceph.client.admin.keyring 2023-07-18T03:07:41.444 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:41 smithi179 ceph-mon[129229]: Updating smithi155:/var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-18T03:07:41.445 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:41 smithi179 ceph-mon[129229]: Updating smithi179:/var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-18T03:07:41.445 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:41 smithi179 ceph-mon[129229]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2023-07-18T03:07:41.445 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:41 smithi179 ceph-mon[129229]: Updating smithi179:/etc/ceph/ceph.client.admin.keyring 2023-07-18T03:07:41.445 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:41 smithi179 ceph-mon[129229]: Updating smithi101:/var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-18T03:07:41.445 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:41 smithi179 ceph-mon[129229]: pgmap v6: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:07:41.445 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:41 smithi179 ceph-mon[129229]: Updating smithi155:/var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-18T03:07:41.446 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:41 smithi179 ceph-mon[129229]: Updating smithi179:/var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-18T03:07:41.446 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:41 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:41.446 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:41 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:41.446 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:41 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:41.446 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:41 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:41.446 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:41 smithi179 ceph-mon[129229]: Updating smithi101:/etc/ceph/ceph.client.admin.keyring 2023-07-18T03:07:41.482 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:41 smithi101 ceph-mon[126379]: Updating smithi155:/var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-18T03:07:41.482 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:41 smithi101 ceph-mon[126379]: Updating smithi179:/var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-18T03:07:41.482 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:41 smithi101 ceph-mon[126379]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2023-07-18T03:07:41.482 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:41 smithi101 ceph-mon[126379]: Updating smithi179:/etc/ceph/ceph.client.admin.keyring 2023-07-18T03:07:41.483 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:41 smithi101 ceph-mon[126379]: Updating smithi101:/var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-18T03:07:41.483 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:41 smithi101 ceph-mon[126379]: pgmap v6: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:07:41.483 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:41 smithi101 ceph-mon[126379]: Updating smithi155:/var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-18T03:07:41.483 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:41 smithi101 ceph-mon[126379]: Updating smithi179:/var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-18T03:07:41.483 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:41 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:41.483 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:41 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:41.483 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:41 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:41.484 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:41 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:41.484 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:41 smithi101 ceph-mon[126379]: Updating smithi101:/etc/ceph/ceph.client.admin.keyring 2023-07-18T03:07:42.444 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:42 smithi179 ceph-mon[129229]: Updating smithi101:/var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-18T03:07:42.445 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:42 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:42.445 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:42 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:42.445 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:42 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:42.445 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:42 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:07:42.445 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:42 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:07:42.446 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:42 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:07:42.446 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:42 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:42.482 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:42 smithi101 ceph-mon[126379]: Updating smithi101:/var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-18T03:07:42.482 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:42 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:42.482 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:42 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:42.482 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:42 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:42.483 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:42 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:07:42.483 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:42 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:07:42.483 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:42 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:07:42.483 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:42 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:42.539 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:42 smithi155 ceph-mon[128885]: Updating smithi101:/var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-18T03:07:42.540 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:42 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:42.540 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:42 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:42.540 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:42 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:42.540 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:42 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:07:42.540 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:42 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:07:42.540 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:42 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:07:42.540 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:42 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:07:43.444 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:43 smithi179 ceph-mon[129229]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 132 KiB/s wr, 51 op/s 2023-07-18T03:07:43.482 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:43 smithi101 ceph-mon[126379]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 132 KiB/s wr, 51 op/s 2023-07-18T03:07:43.539 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:43 smithi155 ceph-mon[128885]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 132 KiB/s wr, 51 op/s 2023-07-18T03:07:45.194 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:44 smithi179 ceph-mon[129229]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 103 KiB/s wr, 39 op/s 2023-07-18T03:07:45.232 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:44 smithi101 ceph-mon[126379]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 103 KiB/s wr, 39 op/s 2023-07-18T03:07:45.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:44 smithi155 ceph-mon[128885]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 103 KiB/s wr, 39 op/s 2023-07-18T03:07:47.194 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:46 smithi179 ceph-mon[129229]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 84 KiB/s wr, 35 op/s 2023-07-18T03:07:47.232 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:46 smithi101 ceph-mon[126379]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 84 KiB/s wr, 35 op/s 2023-07-18T03:07:47.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:46 smithi155 ceph-mon[128885]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 84 KiB/s wr, 35 op/s 2023-07-18T03:07:49.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:48 smithi179 ceph-mon[129229]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 77 KiB/s wr, 32 op/s 2023-07-18T03:07:49.232 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:48 smithi101 ceph-mon[126379]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 77 KiB/s wr, 32 op/s 2023-07-18T03:07:49.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:48 smithi155 ceph-mon[128885]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 77 KiB/s wr, 32 op/s 2023-07-18T03:07:51.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:50 smithi179 ceph-mon[129229]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 77 KiB/s wr, 32 op/s 2023-07-18T03:07:51.232 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:50 smithi101 ceph-mon[126379]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 77 KiB/s wr, 32 op/s 2023-07-18T03:07:51.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:50 smithi155 ceph-mon[128885]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 77 KiB/s wr, 32 op/s 2023-07-18T03:07:53.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:52 smithi179 ceph-mon[129229]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 77 KiB/s wr, 34 op/s 2023-07-18T03:07:53.232 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:52 smithi101 ceph-mon[126379]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 77 KiB/s wr, 34 op/s 2023-07-18T03:07:53.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:52 smithi155 ceph-mon[128885]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 77 KiB/s wr, 34 op/s 2023-07-18T03:07:54.503 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:07:54.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.726+0000 7fe60ddff700 1 -- 172.21.15.101:0/1430615357 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe6080ffad0 msgr2=0x7fe6080fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:54.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.726+0000 7fe60ddff700 1 --2- 172.21.15.101:0/1430615357 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe6080ffad0 0x7fe6080fff40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc009990 tx=0x7fe5fc009ca0 comp rx=0 tx=0).stop 2023-07-18T03:07:54.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.726+0000 7fe60ddff700 1 -- 172.21.15.101:0/1430615357 shutdown_connections 2023-07-18T03:07:54.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.726+0000 7fe60ddff700 1 --2- 172.21.15.101:0/1430615357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe608105ae0 0x7fe608105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:54.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.726+0000 7fe60ddff700 1 --2- 172.21.15.101:0/1430615357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe608100480 0x7fe60810c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:54.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.726+0000 7fe60ddff700 1 --2- 172.21.15.101:0/1430615357 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe6080ffad0 0x7fe6080fff40 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:54.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.726+0000 7fe60ddff700 1 -- 172.21.15.101:0/1430615357 >> 172.21.15.101:0/1430615357 conn(0x7fe6080fb5f0 msgr2=0x7fe6080fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:54.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.727+0000 7fe60ddff700 1 -- 172.21.15.101:0/1430615357 shutdown_connections 2023-07-18T03:07:54.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.727+0000 7fe60ddff700 1 -- 172.21.15.101:0/1430615357 wait complete. 2023-07-18T03:07:54.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.728+0000 7fe60ddff700 1 Processor -- start 2023-07-18T03:07:54.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.728+0000 7fe60ddff700 1 -- start start 2023-07-18T03:07:54.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.728+0000 7fe60ddff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6080ffad0 0x7fe608106990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:54.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.728+0000 7fe60ddff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe608100480 0x7fe608106ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:54.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.728+0000 7fe60ddff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe608105ae0 0x7fe608107410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:54.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.728+0000 7fe60ddff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6081a84d0 con 0x7fe608100480 2023-07-18T03:07:54.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.728+0000 7fe60ddff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6081a8640 con 0x7fe608105ae0 2023-07-18T03:07:54.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.728+0000 7fe60ddff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6081a8800 con 0x7fe6080ffad0 2023-07-18T03:07:54.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.729+0000 7fe6077fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6080ffad0 0x7fe608106990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:54.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.729+0000 7fe607fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe608105ae0 0x7fe608107410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:54.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.729+0000 7fe606ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe608100480 0x7fe608106ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:54.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.729+0000 7fe6077fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6080ffad0 0x7fe608106990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:46712/0 (socket says 172.21.15.101:46712) 2023-07-18T03:07:54.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.729+0000 7fe6077fe700 1 -- 172.21.15.101:0/2040136348 learned_addr learned my addr 172.21.15.101:0/2040136348 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:54.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.729+0000 7fe6077fe700 1 -- 172.21.15.101:0/2040136348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe608105ae0 msgr2=0x7fe608107410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:54.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.729+0000 7fe6077fe700 1 --2- 172.21.15.101:0/2040136348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe608105ae0 0x7fe608107410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:54.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.729+0000 7fe6077fe700 1 -- 172.21.15.101:0/2040136348 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe608100480 msgr2=0x7fe608106ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:54.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.729+0000 7fe6077fe700 1 --2- 172.21.15.101:0/2040136348 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe608100480 0x7fe608106ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:54.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.729+0000 7fe6077fe700 1 -- 172.21.15.101:0/2040136348 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5fc0095f0 con 0x7fe6080ffad0 2023-07-18T03:07:54.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.729+0000 7fe607fff700 1 --2- 172.21.15.101:0/2040136348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe608105ae0 0x7fe608107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:07:54.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.729+0000 7fe606ffd700 1 --2- 172.21.15.101:0/2040136348 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe608100480 0x7fe608106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:07:54.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.729+0000 7fe6077fe700 1 --2- 172.21.15.101:0/2040136348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6080ffad0 0x7fe608106990 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe5f800e9a0 tx=0x7fe5f800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:54.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.730+0000 7fe604ff9700 1 -- 172.21.15.101:0/2040136348 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8009790 con 0x7fe6080ffad0 2023-07-18T03:07:54.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.730+0000 7fe604ff9700 1 -- 172.21.15.101:0/2040136348 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5f8010430 con 0x7fe6080ffad0 2023-07-18T03:07:54.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.730+0000 7fe604ff9700 1 -- 172.21.15.101:0/2040136348 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8018680 con 0x7fe6080ffad0 2023-07-18T03:07:54.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.730+0000 7fe60ddff700 1 -- 172.21.15.101:0/2040136348 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6081a8ae0 con 0x7fe6080ffad0 2023-07-18T03:07:54.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.730+0000 7fe60ddff700 1 -- 172.21.15.101:0/2040136348 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6081a8ff0 con 0x7fe6080ffad0 2023-07-18T03:07:54.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.734+0000 7fe604ff9700 1 -- 172.21.15.101:0/2040136348 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 23) v1 ==== 135892+0+0 (secure 0 0 0) 0x7fe5f80105a0 con 0x7fe6080ffad0 2023-07-18T03:07:54.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.735+0000 7fe604ff9700 1 --2- 172.21.15.101:0/2040136348 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fe5f00a13d0 0x7fe5f00a3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:54.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.735+0000 7fe604ff9700 1 -- 172.21.15.101:0/2040136348 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fe5f8014070 con 0x7fe6080ffad0 2023-07-18T03:07:54.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.735+0000 7fe606ffd700 1 --2- 172.21.15.101:0/2040136348 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fe5f00a13d0 0x7fe5f00a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:54.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.736+0000 7fe606ffd700 1 --2- 172.21.15.101:0/2040136348 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fe5f00a13d0 0x7fe5f00a3880 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe608108230 tx=0x7fe5fc005fb0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:54.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.736+0000 7fe62163bb80 1 Processor -- start 2023-07-18T03:07:54.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.737+0000 7fe62163bb80 1 -- start start 2023-07-18T03:07:54.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.737+0000 7fe62163bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56506ee522d0 0x56506ee526a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:54.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.737+0000 7fe62163bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56506ee52be0 0x56506ee5d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:54.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.737+0000 7fe62163bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56506ee5dad0 0x56506ee5feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:54.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.737+0000 7fe62163bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56506ed58cb0 con 0x56506ee52be0 2023-07-18T03:07:54.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.737+0000 7fe62163bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56506edaba40 con 0x56506ee5dad0 2023-07-18T03:07:54.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.737+0000 7fe62163bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56506eda4ed0 con 0x56506ee522d0 2023-07-18T03:07:54.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.737+0000 7fe606ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56506ee52be0 0x56506ee5d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:54.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.737+0000 7fe606ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56506ee52be0 0x56506ee5d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:42020/0 (socket says 172.21.15.101:42020) 2023-07-18T03:07:54.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.737+0000 7fe606ffd700 1 -- 172.21.15.101:0/2159166182 learned_addr learned my addr 172.21.15.101:0/2159166182 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:54.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.738+0000 7fe6077fe700 1 --2- 172.21.15.101:0/2159166182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56506ee522d0 0x56506ee526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:54.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.738+0000 7fe607fff700 1 --2- 172.21.15.101:0/2159166182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56506ee5dad0 0x56506ee5feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:54.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.738+0000 7fe606ffd700 1 -- 172.21.15.101:0/2159166182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56506ee522d0 msgr2=0x56506ee526a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:54.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.738+0000 7fe606ffd700 1 --2- 172.21.15.101:0/2159166182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56506ee522d0 0x56506ee526a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:54.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.738+0000 7fe606ffd700 1 -- 172.21.15.101:0/2159166182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56506ee5dad0 msgr2=0x56506ee5feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:54.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.738+0000 7fe606ffd700 1 --2- 172.21.15.101:0/2159166182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56506ee5dad0 0x56506ee5feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:54.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.738+0000 7fe606ffd700 1 -- 172.21.15.101:0/2159166182 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56506ebd2de0 con 0x56506ee52be0 2023-07-18T03:07:54.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.738+0000 7fe607fff700 1 --2- 172.21.15.101:0/2159166182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56506ee5dad0 0x56506ee5feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:07:54.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.738+0000 7fe606ffd700 1 --2- 172.21.15.101:0/2159166182 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56506ee52be0 0x56506ee5d590 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc009990 tx=0x7fe5fc020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42608bb61767cf4b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:54.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.738+0000 7fe5edffb700 1 -- 172.21.15.101:0/2159166182 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc02b070 con 0x56506ee52be0 2023-07-18T03:07:54.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.738+0000 7fe5edffb700 1 -- 172.21.15.101:0/2159166182 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5fc01e040 con 0x56506ee52be0 2023-07-18T03:07:54.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.738+0000 7fe5edffb700 1 -- 172.21.15.101:0/2159166182 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc024e80 con 0x56506ee52be0 2023-07-18T03:07:54.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.739+0000 7fe62163bb80 1 -- 172.21.15.101:0/2159166182 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56506ee52be0 msgr2=0x56506ee5d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:54.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.739+0000 7fe62163bb80 1 --2- 172.21.15.101:0/2159166182 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56506ee52be0 0x56506ee5d590 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc009990 tx=0x7fe5fc020000 comp rx=0 tx=0).stop 2023-07-18T03:07:54.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.739+0000 7fe62163bb80 1 -- 172.21.15.101:0/2159166182 shutdown_connections 2023-07-18T03:07:54.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.739+0000 7fe62163bb80 1 --2- 172.21.15.101:0/2159166182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56506ee5dad0 0x56506ee5feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:54.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.739+0000 7fe62163bb80 1 --2- 172.21.15.101:0/2159166182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56506ee522d0 0x56506ee526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:54.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.739+0000 7fe62163bb80 1 --2- 172.21.15.101:0/2159166182 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56506ee52be0 0x56506ee5d590 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:54.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.739+0000 7fe62163bb80 1 -- 172.21.15.101:0/2159166182 >> 172.21.15.101:0/2159166182 conn(0x56506ee48da0 msgr2=0x56506ee4e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:54.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.739+0000 7fe62163bb80 1 -- 172.21.15.101:0/2159166182 shutdown_connections 2023-07-18T03:07:54.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.739+0000 7fe62163bb80 1 -- 172.21.15.101:0/2159166182 wait complete. 2023-07-18T03:07:54.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.740+0000 7fe62163bb80 1 -- start start 2023-07-18T03:07:54.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.740+0000 7fe62163bb80 1 Processor -- start 2023-07-18T03:07:54.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.740+0000 7fe62163bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56506ee522d0 0x56506ee5d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:54.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.741+0000 7fe62163bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56506ee52be0 0x56506ee5a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:54.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.741+0000 7fe62163bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56506ee5dad0 0x56506ee5ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:54.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.741+0000 7fe62163bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56506ed44ac0 con 0x56506ee52be0 2023-07-18T03:07:54.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.741+0000 7fe62163bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56506ed447c0 con 0x56506ee522d0 2023-07-18T03:07:54.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.741+0000 7fe62163bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56506ec28920 con 0x56506ee5dad0 2023-07-18T03:07:54.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.741+0000 7fe62163bb80 10 client.0 _ll_register_callbacks cb 0x56506ed42760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:07:54.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.741+0000 7fe606ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56506ee52be0 0x56506ee5a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:54.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.741+0000 7fe606ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56506ee52be0 0x56506ee5a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:42032/0 (socket says 172.21.15.101:42032) 2023-07-18T03:07:54.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.741+0000 7fe606ffd700 1 -- 172.21.15.101:0/2386969111 learned_addr learned my addr 172.21.15.101:0/2386969111 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:54.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.741+0000 7fe607fff700 1 --2- 172.21.15.101:0/2386969111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56506ee5dad0 0x56506ee5ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:54.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.741+0000 7fe6077fe700 1 --2- 172.21.15.101:0/2386969111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56506ee522d0 0x56506ee5d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:54.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.741+0000 7fe606ffd700 1 -- 172.21.15.101:0/2386969111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56506ee5dad0 msgr2=0x56506ee5ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:54.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.741+0000 7fe606ffd700 1 --2- 172.21.15.101:0/2386969111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56506ee5dad0 0x56506ee5ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:54.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.741+0000 7fe606ffd700 1 -- 172.21.15.101:0/2386969111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56506ee522d0 msgr2=0x56506ee5d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:54.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.741+0000 7fe606ffd700 1 --2- 172.21.15.101:0/2386969111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56506ee522d0 0x56506ee5d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:54.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.741+0000 7fe606ffd700 1 -- 172.21.15.101:0/2386969111 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe60810d210 con 0x56506ee52be0 2023-07-18T03:07:54.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.741+0000 7fe607fff700 1 --2- 172.21.15.101:0/2386969111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56506ee5dad0 0x56506ee5ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:07:54.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.742+0000 7fe606ffd700 1 --2- 172.21.15.101:0/2386969111 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56506ee52be0 0x56506ee5a710 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc024a20 tx=0x7fe5fc00fea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b95bbc5305e5b450 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:54.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.742+0000 7fe5e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:07:54.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.742+0000 7fe5e6ffd700 1 -- 172.21.15.101:0/2386969111 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc03b070 con 0x56506ee52be0 2023-07-18T03:07:54.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.742+0000 7fe5e6ffd700 1 -- 172.21.15.101:0/2386969111 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fe5fc024620 con 0x56506ee52be0 2023-07-18T03:07:54.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.742+0000 7fe62163bb80 1 -- 172.21.15.101:0/2386969111 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56506ecfe4f0 con 0x56506ee52be0 2023-07-18T03:07:54.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.742+0000 7fe5e6ffd700 1 -- 172.21.15.101:0/2386969111 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5fc02b040 con 0x56506ee52be0 2023-07-18T03:07:54.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.742+0000 7fe62163bb80 10 client.15570 fetch_fsmap learned FSMap version 24 2023-07-18T03:07:54.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.742+0000 7fe62163bb80 1 -- 172.21.15.101:0/2386969111 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56506ebbbc60 con 0x56506ee52be0 2023-07-18T03:07:54.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.742+0000 7fe5e6ffd700 1 -- 172.21.15.101:0/2386969111 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56506ecfe4f0 con 0x56506ee52be0 2023-07-18T03:07:54.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.743+0000 7fe5e6ffd700 1 -- 172.21.15.101:0/2386969111 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe5fc03e020 con 0x56506ee52be0 2023-07-18T03:07:54.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.743+0000 7fe62163bb80 10 client.15570 fetch_fsmap finished waiting for FSMap version 24 2023-07-18T03:07:54.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.743+0000 7fe62163bb80 10 client.15570 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:07:54.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.743+0000 7fe62163bb80 20 client.15570 populate_metadata read hostname 'smithi101' 2023-07-18T03:07:54.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.743+0000 7fe62163bb80 1 --2- 172.21.15.101:0/2386969111 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56506ee6cdc0 0x56506ee6f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:54.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.743+0000 7fe62163bb80 4 client.15570 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:07:54.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.743+0000 7fe62163bb80 1 -- 172.21.15.101:0/2386969111 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56506ed9deb0 con 0x56506ee6cdc0 2023-07-18T03:07:54.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.743+0000 7fe6077fe700 1 --2- 172.21.15.101:0/2386969111 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56506ee6cdc0 0x56506ee6f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:54.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.744+0000 7fe5e6ffd700 10 client.15570 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:07:54.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.744+0000 7fe6077fe700 1 --2- 172.21.15.101:0/2386969111 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56506ee6cdc0 0x56506ee6f1d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd129bf46d32a56e server_cookie=f854b92ca3435edf in_seq=0 out_seq=0 2023-07-18T03:07:54.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.744+0000 7fe5e6ffd700 1 -- 172.21.15.101:0/2386969111 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56506ed9deb0 con 0x56506ee6cdc0 2023-07-18T03:07:54.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.744+0000 7fe5e6ffd700 10 client.15570 handle_command_reply: tid=0 2023-07-18T03:07:54.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.744+0000 7fe62163bb80 1 client.15570 shutdown 2023-07-18T03:07:54.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.745+0000 7fe62163bb80 1 -- 172.21.15.101:0/2386969111 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56506ee52be0 msgr2=0x56506ee5a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:54.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.745+0000 7fe62163bb80 1 --2- 172.21.15.101:0/2386969111 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56506ee52be0 0x56506ee5a710 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc024a20 tx=0x7fe5fc00fea0 comp rx=0 tx=0).stop 2023-07-18T03:07:54.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.745+0000 7fe62163bb80 1 -- 172.21.15.101:0/2386969111 shutdown_connections 2023-07-18T03:07:54.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.745+0000 7fe62163bb80 1 --2- 172.21.15.101:0/2386969111 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56506ee6cdc0 0x56506ee6f1d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:54.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.745+0000 7fe62163bb80 1 --2- 172.21.15.101:0/2386969111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56506ee522d0 0x56506ee5d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:54.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.745+0000 7fe62163bb80 1 --2- 172.21.15.101:0/2386969111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56506ee5dad0 0x56506ee5ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:54.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.745+0000 7fe62163bb80 1 --2- 172.21.15.101:0/2386969111 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56506ee52be0 0x56506ee5a710 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:54.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.745+0000 7fe5e6ffd700 0 client.15570 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:07:54.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.745+0000 7fe62163bb80 1 -- 172.21.15.101:0/2386969111 >> 172.21.15.101:0/2386969111 conn(0x56506ee48da0 msgr2=0x56506ee4e290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:54.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.745+0000 7fe62163bb80 1 -- 172.21.15.101:0/2386969111 shutdown_connections 2023-07-18T03:07:54.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.745+0000 7fe62163bb80 1 -- 172.21.15.101:0/2386969111 wait complete. 2023-07-18T03:07:54.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.745+0000 7fe62163bb80 20 client.15570 trim_cache size 0 max 16384 2023-07-18T03:07:54.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.766+0000 7fe62163bb80 1 Processor -- start 2023-07-18T03:07:54.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.766+0000 7fe62163bb80 1 -- start start 2023-07-18T03:07:54.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.766+0000 7fe62163bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56506ee522d0 0x56506ee5cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:54.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.767+0000 7fe62163bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56506ee52be0 0x56506ee55af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:54.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.767+0000 7fe62163bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56506ee5dad0 0x56506ee59fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:54.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.767+0000 7fe62163bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56506ee4fc50 con 0x56506ee52be0 2023-07-18T03:07:54.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.767+0000 7fe62163bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56506ee53dd0 con 0x56506ee5dad0 2023-07-18T03:07:54.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.767+0000 7fe62163bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56506ee59ca0 con 0x56506ee522d0 2023-07-18T03:07:54.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.767+0000 7fe606ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56506ee52be0 0x56506ee55af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:54.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.767+0000 7fe6077fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56506ee522d0 0x56506ee5cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:54.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.767+0000 7fe606ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56506ee52be0 0x56506ee55af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:42034/0 (socket says 172.21.15.101:42034) 2023-07-18T03:07:54.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.767+0000 7fe606ffd700 1 -- 172.21.15.101:0/982151210 learned_addr learned my addr 172.21.15.101:0/982151210 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:54.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.767+0000 7fe607fff700 1 --2- 172.21.15.101:0/982151210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56506ee5dad0 0x56506ee59fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:54.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.767+0000 7fe606ffd700 1 -- 172.21.15.101:0/982151210 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56506ee522d0 msgr2=0x56506ee5cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:54.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.767+0000 7fe606ffd700 1 --2- 172.21.15.101:0/982151210 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56506ee522d0 0x56506ee5cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:54.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.767+0000 7fe606ffd700 1 -- 172.21.15.101:0/982151210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56506ee5dad0 msgr2=0x56506ee59fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:54.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.767+0000 7fe606ffd700 1 --2- 172.21.15.101:0/982151210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56506ee5dad0 0x56506ee59fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:54.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.767+0000 7fe606ffd700 1 -- 172.21.15.101:0/982151210 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56506ebf9250 con 0x56506ee52be0 2023-07-18T03:07:54.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.767+0000 7fe6077fe700 1 --2- 172.21.15.101:0/982151210 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56506ee522d0 0x56506ee5cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:07:54.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.767+0000 7fe607fff700 1 --2- 172.21.15.101:0/982151210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56506ee5dad0 0x56506ee59fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:07:54.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.768+0000 7fe606ffd700 1 --2- 172.21.15.101:0/982151210 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56506ee52be0 0x56506ee55af0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc024fd0 tx=0x7fe5fc0092b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=144c3c73e0f92c01 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:54.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.768+0000 7fe5ee7fc700 1 -- 172.21.15.101:0/982151210 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc03b030 con 0x56506ee52be0 2023-07-18T03:07:54.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.768+0000 7fe5ee7fc700 1 -- 172.21.15.101:0/982151210 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5fc037e90 con 0x56506ee52be0 2023-07-18T03:07:54.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.768+0000 7fe5ee7fc700 1 -- 172.21.15.101:0/982151210 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc02b040 con 0x56506ee52be0 2023-07-18T03:07:54.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.769+0000 7fe62163bb80 1 -- 172.21.15.101:0/982151210 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56506ee52be0 msgr2=0x56506ee55af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:54.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.769+0000 7fe62163bb80 1 --2- 172.21.15.101:0/982151210 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56506ee52be0 0x56506ee55af0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc024fd0 tx=0x7fe5fc0092b0 comp rx=0 tx=0).stop 2023-07-18T03:07:54.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.769+0000 7fe62163bb80 1 -- 172.21.15.101:0/982151210 shutdown_connections 2023-07-18T03:07:54.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.769+0000 7fe62163bb80 1 --2- 172.21.15.101:0/982151210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56506ee5dad0 0x56506ee59fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:54.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.769+0000 7fe62163bb80 1 --2- 172.21.15.101:0/982151210 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56506ee522d0 0x56506ee5cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:54.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.769+0000 7fe62163bb80 1 --2- 172.21.15.101:0/982151210 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56506ee52be0 0x56506ee55af0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:54.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.769+0000 7fe62163bb80 1 -- 172.21.15.101:0/982151210 >> 172.21.15.101:0/982151210 conn(0x56506ee48da0 msgr2=0x56506ee73af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:54.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.769+0000 7fe62163bb80 1 -- 172.21.15.101:0/982151210 shutdown_connections 2023-07-18T03:07:54.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.769+0000 7fe62163bb80 1 -- 172.21.15.101:0/982151210 wait complete. 2023-07-18T03:07:54.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.769+0000 7fe62163bb80 1 -- start start 2023-07-18T03:07:54.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.769+0000 7fe62163bb80 1 Processor -- start 2023-07-18T03:07:54.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.770+0000 7fe62163bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56506ee522d0 0x56506ee62680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:54.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.770+0000 7fe62163bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56506ee52be0 0x56506ee62bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:54.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.770+0000 7fe62163bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56506ee5dad0 0x56506eebfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:54.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.770+0000 7fe62163bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56506eec0180 con 0x56506ee5dad0 2023-07-18T03:07:54.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.770+0000 7fe62163bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56506eec02c0 con 0x56506ee52be0 2023-07-18T03:07:54.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.770+0000 7fe62163bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56506eec0450 con 0x56506ee522d0 2023-07-18T03:07:54.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.770+0000 7fe62163bb80 10 client.0 _ll_register_callbacks cb 0x7fe5fc00bd80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:07:54.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.770+0000 7fe607fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56506ee5dad0 0x56506eebfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:54.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.770+0000 7fe607fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56506ee5dad0 0x56506eebfc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:42042/0 (socket says 172.21.15.101:42042) 2023-07-18T03:07:54.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.770+0000 7fe606ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56506ee52be0 0x56506ee62bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:54.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.770+0000 7fe607fff700 1 -- 172.21.15.101:0/1451363598 learned_addr learned my addr 172.21.15.101:0/1451363598 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:54.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.770+0000 7fe6077fe700 1 --2- 172.21.15.101:0/1451363598 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56506ee522d0 0x56506ee62680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:54.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.770+0000 7fe607fff700 1 -- 172.21.15.101:0/1451363598 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56506ee522d0 msgr2=0x56506ee62680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:54.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.770+0000 7fe607fff700 1 --2- 172.21.15.101:0/1451363598 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56506ee522d0 0x56506ee62680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:54.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.770+0000 7fe607fff700 1 -- 172.21.15.101:0/1451363598 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56506ee52be0 msgr2=0x56506ee62bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:54.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.770+0000 7fe607fff700 1 --2- 172.21.15.101:0/1451363598 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56506ee52be0 0x56506ee62bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:54.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.771+0000 7fe607fff700 1 -- 172.21.15.101:0/1451363598 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6081a8ff0 con 0x56506ee5dad0 2023-07-18T03:07:54.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.771+0000 7fe6077fe700 1 --2- 172.21.15.101:0/1451363598 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56506ee522d0 0x56506ee62680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:07:54.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.771+0000 7fe607fff700 1 --2- 172.21.15.101:0/1451363598 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56506ee5dad0 0x56506eebfc40 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe600004730 tx=0x7fe600004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff405393b3661f6 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:54.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.771+0000 7fe5e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:07:54.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.771+0000 7fe5e6ffd700 1 -- 172.21.15.101:0/1451363598 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60000ed70 con 0x56506ee5dad0 2023-07-18T03:07:54.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.771+0000 7fe62163bb80 1 -- 172.21.15.101:0/1451363598 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe5fc01e040 con 0x56506ee5dad0 2023-07-18T03:07:54.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.771+0000 7fe5e6ffd700 1 -- 172.21.15.101:0/1451363598 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fe600021420 con 0x56506ee5dad0 2023-07-18T03:07:54.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.771+0000 7fe5e6ffd700 1 -- 172.21.15.101:0/1451363598 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe60000eed0 con 0x56506ee5dad0 2023-07-18T03:07:54.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.771+0000 7fe62163bb80 10 client.15576 fetch_fsmap learned FSMap version 24 2023-07-18T03:07:54.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.771+0000 7fe62163bb80 1 -- 172.21.15.101:0/1451363598 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56506ebbbc60 con 0x56506ee5dad0 2023-07-18T03:07:54.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.772+0000 7fe5e6ffd700 1 -- 172.21.15.101:0/1451363598 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5fc01e040 con 0x56506ee5dad0 2023-07-18T03:07:54.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.772+0000 7fe5e6ffd700 1 -- 172.21.15.101:0/1451363598 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe600018680 con 0x56506ee5dad0 2023-07-18T03:07:54.769 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.772+0000 7fe62163bb80 10 client.15576 fetch_fsmap finished waiting for FSMap version 24 2023-07-18T03:07:54.769 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.772+0000 7fe62163bb80 10 client.15576 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:07:54.769 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.772+0000 7fe62163bb80 20 client.15576 populate_metadata read hostname 'smithi101' 2023-07-18T03:07:54.769 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.772+0000 7fe62163bb80 1 --2- 172.21.15.101:0/1451363598 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56506eec1c70 0x56506eec4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:54.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.772+0000 7fe62163bb80 4 client.15576 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:07:54.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.772+0000 7fe62163bb80 1 -- 172.21.15.101:0/1451363598 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56506ed60050 con 0x56506eec1c70 2023-07-18T03:07:54.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.772+0000 7fe6077fe700 1 --2- 172.21.15.101:0/1451363598 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56506eec1c70 0x56506eec4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:54.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.773+0000 7fe5e6ffd700 10 client.15576 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:07:54.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.773+0000 7fe6077fe700 1 --2- 172.21.15.101:0/1451363598 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56506eec1c70 0x56506eec4080 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f19d3abfcc870579 server_cookie=7afc4d29a86c928a in_seq=0 out_seq=0 2023-07-18T03:07:54.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.773+0000 7fe5e6ffd700 1 -- 172.21.15.101:0/1451363598 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56506ed60050 con 0x56506eec1c70 2023-07-18T03:07:54.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.773+0000 7fe5e6ffd700 10 client.15576 handle_command_reply: tid=0 2023-07-18T03:07:54.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.773+0000 7fe62163bb80 1 client.15576 shutdown 2023-07-18T03:07:54.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.773+0000 7fe62163bb80 1 -- 172.21.15.101:0/1451363598 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56506ee5dad0 msgr2=0x56506eebfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:54.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.773+0000 7fe62163bb80 1 --2- 172.21.15.101:0/1451363598 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56506ee5dad0 0x56506eebfc40 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe600004730 tx=0x7fe600004760 comp rx=0 tx=0).stop 2023-07-18T03:07:54.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.773+0000 7fe62163bb80 1 -- 172.21.15.101:0/1451363598 shutdown_connections 2023-07-18T03:07:54.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.773+0000 7fe62163bb80 1 --2- 172.21.15.101:0/1451363598 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56506eec1c70 0x56506eec4080 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:54.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.773+0000 7fe62163bb80 1 --2- 172.21.15.101:0/1451363598 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56506ee5dad0 0x56506eebfc40 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:54.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.773+0000 7fe62163bb80 1 --2- 172.21.15.101:0/1451363598 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56506ee522d0 0x56506ee62680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:54.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.773+0000 7fe5e6ffd700 0 client.15576 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:07:54.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.773+0000 7fe62163bb80 1 --2- 172.21.15.101:0/1451363598 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56506ee52be0 0x56506ee62bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:54.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.773+0000 7fe62163bb80 1 -- 172.21.15.101:0/1451363598 >> 172.21.15.101:0/1451363598 conn(0x56506ee48da0 msgr2=0x56506ee55af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:54.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.773+0000 7fe62163bb80 1 -- 172.21.15.101:0/1451363598 shutdown_connections 2023-07-18T03:07:54.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.773+0000 7fe62163bb80 1 -- 172.21.15.101:0/1451363598 wait complete. 2023-07-18T03:07:54.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.774+0000 7fe62163bb80 20 client.15576 trim_cache size 0 max 16384 2023-07-18T03:07:54.773 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:07:54.773 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "no active scrubs running", 2023-07-18T03:07:54.773 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": {} 2023-07-18T03:07:54.773 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:07:54.774 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.774+0000 7fe60ddff700 1 -- 172.21.15.101:0/2040136348 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fe5f00a13d0 msgr2=0x7fe5f00a3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:54.774 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.774+0000 7fe60ddff700 1 --2- 172.21.15.101:0/2040136348 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fe5f00a13d0 0x7fe5f00a3880 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe608108230 tx=0x7fe5fc005fb0 comp rx=0 tx=0).stop 2023-07-18T03:07:54.774 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.774+0000 7fe60ddff700 1 -- 172.21.15.101:0/2040136348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6080ffad0 msgr2=0x7fe608106990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:54.774 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.774+0000 7fe60ddff700 1 --2- 172.21.15.101:0/2040136348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6080ffad0 0x7fe608106990 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe5f800e9a0 tx=0x7fe5f800ecb0 comp rx=0 tx=0).stop 2023-07-18T03:07:54.774 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.774+0000 7fe60ddff700 1 -- 172.21.15.101:0/2040136348 shutdown_connections 2023-07-18T03:07:54.775 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.774+0000 7fe60ddff700 1 --2- 172.21.15.101:0/2040136348 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fe5f00a13d0 0x7fe5f00a3880 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:54.775 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.774+0000 7fe60ddff700 1 --2- 172.21.15.101:0/2040136348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe608105ae0 0x7fe608107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:54.775 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.774+0000 7fe60ddff700 1 --2- 172.21.15.101:0/2040136348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6080ffad0 0x7fe608106990 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:54.775 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.774+0000 7fe60ddff700 1 --2- 172.21.15.101:0/2040136348 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe608100480 0x7fe608106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:54.775 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.774+0000 7fe60ddff700 1 -- 172.21.15.101:0/2040136348 >> 172.21.15.101:0/2040136348 conn(0x7fe6080fb5f0 msgr2=0x7fe60810ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:54.775 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.774+0000 7fe60ddff700 1 -- 172.21.15.101:0/2040136348 shutdown_connections 2023-07-18T03:07:54.775 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.774+0000 7fe60ddff700 1 -- 172.21.15.101:0/2040136348 wait complete. 2023-07-18T03:07:54.781 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-18T03:07:54.781 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:07:54.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.997+0000 7f7d70d9d700 1 -- 172.21.15.101:0/645660040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d6c105a00 msgr2=0x7f7d6c105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:54.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.997+0000 7f7d70d9d700 1 --2- 172.21.15.101:0/645660040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d6c105a00 0x7f7d6c105dd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0098f0 tx=0x7f7d5c009c00 comp rx=0 tx=0).stop 2023-07-18T03:07:54.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.997+0000 7f7d70d9d700 1 -- 172.21.15.101:0/645660040 shutdown_connections 2023-07-18T03:07:54.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.997+0000 7f7d70d9d700 1 --2- 172.21.15.101:0/645660040 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7d6c1003a0 0x7f7d6c10c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:54.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.997+0000 7f7d70d9d700 1 --2- 172.21.15.101:0/645660040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d6c105a00 0x7f7d6c105dd0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:54.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.997+0000 7f7d70d9d700 1 --2- 172.21.15.101:0/645660040 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d6c0ff9f0 0x7f7d6c0ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:54.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.997+0000 7f7d70d9d700 1 -- 172.21.15.101:0/645660040 >> 172.21.15.101:0/645660040 conn(0x7f7d6c0fb510 msgr2=0x7f7d6c0fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:54.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.997+0000 7f7d70d9d700 1 -- 172.21.15.101:0/645660040 shutdown_connections 2023-07-18T03:07:54.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.998+0000 7f7d70d9d700 1 -- 172.21.15.101:0/645660040 wait complete. 2023-07-18T03:07:54.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.998+0000 7f7d70d9d700 1 Processor -- start 2023-07-18T03:07:54.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.999+0000 7f7d70d9d700 1 -- start start 2023-07-18T03:07:54.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.999+0000 7f7d70d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7d6c0ff9f0 0x7f7d6c19d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:54.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.999+0000 7f7d70d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d6c1003a0 0x7f7d6c19de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:54.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.999+0000 7f7d70d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d6c105a00 0x7f7d6c197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:54.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.999+0000 7f7d70d9d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c197fe0 con 0x7f7d6c0ff9f0 2023-07-18T03:07:54.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.999+0000 7f7d70d9d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c198150 con 0x7f7d6c105a00 2023-07-18T03:07:54.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.999+0000 7f7d70d9d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c198310 con 0x7f7d6c1003a0 2023-07-18T03:07:54.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.999+0000 7f7d6b7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7d6c0ff9f0 0x7f7d6c19d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:54.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.999+0000 7f7d6b7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7d6c0ff9f0 0x7f7d6c19d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:42068/0 (socket says 172.21.15.101:42068) 2023-07-18T03:07:54.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.999+0000 7f7d6b7fe700 1 -- 172.21.15.101:0/1616402788 learned_addr learned my addr 172.21.15.101:0/1616402788 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:54.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.999+0000 7f7d6affd700 1 --2- 172.21.15.101:0/1616402788 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d6c1003a0 0x7f7d6c19de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:54.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:54.999+0000 7f7d6bfff700 1 --2- 172.21.15.101:0/1616402788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d6c105a00 0x7f7d6c197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:54.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.000+0000 7f7d6b7fe700 1 -- 172.21.15.101:0/1616402788 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d6c1003a0 msgr2=0x7f7d6c19de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:54.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.000+0000 7f7d6b7fe700 1 --2- 172.21.15.101:0/1616402788 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d6c1003a0 0x7f7d6c19de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:54.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.000+0000 7f7d6b7fe700 1 -- 172.21.15.101:0/1616402788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d6c105a00 msgr2=0x7f7d6c197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:54.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.000+0000 7f7d6b7fe700 1 --2- 172.21.15.101:0/1616402788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d6c105a00 0x7f7d6c197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:54.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.000+0000 7f7d6b7fe700 1 -- 172.21.15.101:0/1616402788 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d64009d00 con 0x7f7d6c0ff9f0 2023-07-18T03:07:54.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.000+0000 7f7d6bfff700 1 --2- 172.21.15.101:0/1616402788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d6c105a00 0x7f7d6c197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:07:54.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.000+0000 7f7d6affd700 1 --2- 172.21.15.101:0/1616402788 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d6c1003a0 0x7f7d6c19de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:07:54.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.000+0000 7f7d6b7fe700 1 --2- 172.21.15.101:0/1616402788 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7d6c0ff9f0 0x7f7d6c19d900 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c009fd0 tx=0x7f7d5c00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:54.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.000+0000 7f7d68ff9700 1 -- 172.21.15.101:0/1616402788 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c01c070 con 0x7f7d6c0ff9f0 2023-07-18T03:07:54.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.000+0000 7f7d68ff9700 1 -- 172.21.15.101:0/1616402788 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d5c00bb20 con 0x7f7d6c0ff9f0 2023-07-18T03:07:54.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.000+0000 7f7d70d9d700 1 -- 172.21.15.101:0/1616402788 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d5c009520 con 0x7f7d6c0ff9f0 2023-07-18T03:07:54.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.000+0000 7f7d68ff9700 1 -- 172.21.15.101:0/1616402788 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c004930 con 0x7f7d6c0ff9f0 2023-07-18T03:07:54.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.001+0000 7f7d70d9d700 1 -- 172.21.15.101:0/1616402788 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d6c1a4710 con 0x7f7d6c0ff9f0 2023-07-18T03:07:54.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.002+0000 7f7d527fc700 1 -- 172.21.15.101:0/1616402788 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d6c067220 con 0x7f7d6c0ff9f0 2023-07-18T03:07:54.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.004+0000 7f7d68ff9700 1 -- 172.21.15.101:0/1616402788 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 23) v1 ==== 135892+0+0 (secure 0 0 0) 0x7f7d5c020070 con 0x7f7d6c0ff9f0 2023-07-18T03:07:54.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.005+0000 7f7d68ff9700 1 --2- 172.21.15.101:0/1616402788 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f7d540a1440 0x7f7d540a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:54.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.006+0000 7f7d68ff9700 1 -- 172.21.15.101:0/1616402788 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f7d5c0cddc0 con 0x7f7d6c0ff9f0 2023-07-18T03:07:54.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.006+0000 7f7d6affd700 1 --2- 172.21.15.101:0/1616402788 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f7d540a1440 0x7f7d540a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:54.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.006+0000 7f7d6affd700 1 --2- 172.21.15.101:0/1616402788 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f7d540a1440 0x7f7d540a38f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c0fee40 tx=0x7f7d60009260 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:54.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.009+0000 7f7d68ff9700 1 -- 172.21.15.101:0/1616402788 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7d5c096d40 con 0x7f7d6c0ff9f0 2023-07-18T03:07:55.194 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:54 smithi179 ceph-mon[129229]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2023-07-18T03:07:55.232 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:54 smithi101 ceph-mon[126379]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2023-07-18T03:07:55.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.258+0000 7f7d527fc700 1 -- 172.21.15.101:0/1616402788 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d6c198e80 con 0x7f7d6c0ff9f0 2023-07-18T03:07:55.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.258+0000 7f7d68ff9700 1 -- 172.21.15.101:0/1616402788 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10858 (secure 0 0 0) 0x7f7d5c09a360 con 0x7f7d6c0ff9f0 2023-07-18T03:07:55.247 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:07:55.247 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/4192846769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":4192846769},{"type":"v1","addr":"172.21.15.101:6839","nonce":4192846769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6841/3824340175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6840","nonce":3824340175},{"type":"v1","addr":"172.21.15.101:6841","nonce":3824340175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:04:59.361185+0000","modified":"2023-07-18T03:05:31.985959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.155:6839/2055462483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2055462483},{"type":"v1","addr":"172.21.15.155:6839","nonce":2055462483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:07:55.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.261+0000 7f7d527fc700 1 -- 172.21.15.101:0/1616402788 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f7d540a1440 msgr2=0x7f7d540a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:55.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.261+0000 7f7d527fc700 1 --2- 172.21.15.101:0/1616402788 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f7d540a1440 0x7f7d540a38f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c0fee40 tx=0x7f7d60009260 comp rx=0 tx=0).stop 2023-07-18T03:07:55.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.261+0000 7f7d527fc700 1 -- 172.21.15.101:0/1616402788 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7d6c0ff9f0 msgr2=0x7f7d6c19d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:55.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.261+0000 7f7d527fc700 1 --2- 172.21.15.101:0/1616402788 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7d6c0ff9f0 0x7f7d6c19d900 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c009fd0 tx=0x7f7d5c00fa00 comp rx=0 tx=0).stop 2023-07-18T03:07:55.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.261+0000 7f7d527fc700 1 -- 172.21.15.101:0/1616402788 shutdown_connections 2023-07-18T03:07:55.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.261+0000 7f7d527fc700 1 --2- 172.21.15.101:0/1616402788 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f7d540a1440 0x7f7d540a38f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.261+0000 7f7d527fc700 1 --2- 172.21.15.101:0/1616402788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d6c105a00 0x7f7d6c197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.261+0000 7f7d527fc700 1 --2- 172.21.15.101:0/1616402788 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7d6c0ff9f0 0x7f7d6c19d900 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.261+0000 7f7d527fc700 1 --2- 172.21.15.101:0/1616402788 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d6c1003a0 0x7f7d6c19de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.261+0000 7f7d527fc700 1 -- 172.21.15.101:0/1616402788 >> 172.21.15.101:0/1616402788 conn(0x7f7d6c0fb510 msgr2=0x7f7d6c0fc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:55.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.261+0000 7f7d527fc700 1 -- 172.21.15.101:0/1616402788 shutdown_connections 2023-07-18T03:07:55.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.261+0000 7f7d527fc700 1 -- 172.21.15.101:0/1616402788 wait complete. 2023-07-18T03:07:55.251 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 24 2023-07-18T03:07:55.269 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-18T03:07:55.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:54 smithi155 ceph-mon[128885]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2023-07-18T03:07:55.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.485+0000 7ff0407ff700 1 -- 172.21.15.101:0/2390767508 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0380728b0 msgr2=0x7ff038072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:55.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.485+0000 7ff0407ff700 1 --2- 172.21.15.101:0/2390767508 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0380728b0 0x7ff038072c80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff034009990 tx=0x7ff034009ca0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.485+0000 7ff0407ff700 1 -- 172.21.15.101:0/2390767508 shutdown_connections 2023-07-18T03:07:55.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.485+0000 7ff0407ff700 1 --2- 172.21.15.101:0/2390767508 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff03810c320 0x7ff03810e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.485+0000 7ff0407ff700 1 --2- 172.21.15.101:0/2390767508 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0380728b0 0x7ff038072c80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.485+0000 7ff0407ff700 1 --2- 172.21.15.101:0/2390767508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0380731c0 0x7ff03810bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.485+0000 7ff0407ff700 1 -- 172.21.15.101:0/2390767508 >> 172.21.15.101:0/2390767508 conn(0x7ff0380fb5e0 msgr2=0x7ff0380fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:55.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.485+0000 7ff0407ff700 1 -- 172.21.15.101:0/2390767508 shutdown_connections 2023-07-18T03:07:55.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.485+0000 7ff0407ff700 1 -- 172.21.15.101:0/2390767508 wait complete. 2023-07-18T03:07:55.480 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.486+0000 7ff0407ff700 1 Processor -- start 2023-07-18T03:07:55.480 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.486+0000 7ff0407ff700 1 -- start start 2023-07-18T03:07:55.480 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.486+0000 7ff0407ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0380728b0 0x7ff038197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:55.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.486+0000 7ff0407ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff0380731c0 0x7ff038198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:55.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.487+0000 7ff0407ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff03810c320 0x7ff03819c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:55.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.487+0000 7ff0407ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03819cdf0 con 0x7ff0380731c0 2023-07-18T03:07:55.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.487+0000 7ff0407ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03819cf60 con 0x7ff0380728b0 2023-07-18T03:07:55.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.487+0000 7ff0407ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03819d120 con 0x7ff03810c320 2023-07-18T03:07:55.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.487+0000 7ff03dd9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff0380731c0 0x7ff038198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:55.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.487+0000 7ff03dd9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff0380731c0 0x7ff038198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:42088/0 (socket says 172.21.15.101:42088) 2023-07-18T03:07:55.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.487+0000 7ff03dd9a700 1 -- 172.21.15.101:0/2867605256 learned_addr learned my addr 172.21.15.101:0/2867605256 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:55.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.487+0000 7ff03e59b700 1 --2- 172.21.15.101:0/2867605256 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0380728b0 0x7ff038197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:55.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.487+0000 7ff03ed9c700 1 --2- 172.21.15.101:0/2867605256 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff03810c320 0x7ff03819c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:55.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.487+0000 7ff03dd9a700 1 -- 172.21.15.101:0/2867605256 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff03810c320 msgr2=0x7ff03819c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:55.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.487+0000 7ff03dd9a700 1 --2- 172.21.15.101:0/2867605256 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff03810c320 0x7ff03819c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.487+0000 7ff03dd9a700 1 -- 172.21.15.101:0/2867605256 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0380728b0 msgr2=0x7ff038197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:55.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.487+0000 7ff03dd9a700 1 --2- 172.21.15.101:0/2867605256 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0380728b0 0x7ff038197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.487+0000 7ff03dd9a700 1 -- 172.21.15.101:0/2867605256 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0340095f0 con 0x7ff0380731c0 2023-07-18T03:07:55.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.487+0000 7ff03ed9c700 1 --2- 172.21.15.101:0/2867605256 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff03810c320 0x7ff03819c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:07:55.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.487+0000 7ff03dd9a700 1 --2- 172.21.15.101:0/2867605256 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff0380731c0 0x7ff038198350 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff02800e9a0 tx=0x7ff02800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:55.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.488+0000 7ff02f7fe700 1 -- 172.21.15.101:0/2867605256 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff028009790 con 0x7ff0380731c0 2023-07-18T03:07:55.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.488+0000 7ff02f7fe700 1 -- 172.21.15.101:0/2867605256 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff02800cbd0 con 0x7ff0380731c0 2023-07-18T03:07:55.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.488+0000 7ff0407ff700 1 -- 172.21.15.101:0/2867605256 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff03819d400 con 0x7ff0380731c0 2023-07-18T03:07:55.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.488+0000 7ff02f7fe700 1 -- 172.21.15.101:0/2867605256 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff028018680 con 0x7ff0380731c0 2023-07-18T03:07:55.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.488+0000 7ff0407ff700 1 -- 172.21.15.101:0/2867605256 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0381a4c40 con 0x7ff0380731c0 2023-07-18T03:07:55.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.491+0000 7ff02f7fe700 1 -- 172.21.15.101:0/2867605256 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 23) v1 ==== 135892+0+0 (secure 0 0 0) 0x7ff02800cd40 con 0x7ff0380731c0 2023-07-18T03:07:55.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.492+0000 7ff054207b80 1 Processor -- start 2023-07-18T03:07:55.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.493+0000 7ff054207b80 1 -- start start 2023-07-18T03:07:55.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.493+0000 7ff054207b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636b00cc2d0 0x5636b00cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:55.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.493+0000 7ff054207b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636b00ccbe0 0x5636b00d7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:55.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.493+0000 7ff054207b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636b00d7ad0 0x5636b00d9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:55.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.493+0000 7ff054207b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636affd2cb0 con 0x5636b00cc2d0 2023-07-18T03:07:55.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.493+0000 7ff054207b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636b0025a40 con 0x5636b00ccbe0 2023-07-18T03:07:55.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.493+0000 7ff054207b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636b001eed0 con 0x5636b00d7ad0 2023-07-18T03:07:55.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.493+0000 7ff02f7fe700 1 --2- 172.21.15.101:0/2867605256 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff0240a13d0 0x7ff0240a3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:55.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.493+0000 7ff03ed9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636b00cc2d0 0x5636b00cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:55.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.493+0000 7ff03e59b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636b00ccbe0 0x5636b00d7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:55.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.493+0000 7ff03e59b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636b00ccbe0 0x5636b00d7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:57510/0 (socket says 172.21.15.101:57510) 2023-07-18T03:07:55.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.493+0000 7ff03e59b700 1 -- 172.21.15.101:0/3750640340 learned_addr learned my addr 172.21.15.101:0/3750640340 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:55.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.493+0000 7ff03dd9a700 1 --2- 172.21.15.101:0/3750640340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636b00d7ad0 0x5636b00d9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:55.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.494+0000 7ff03ed9c700 1 --2- 172.21.15.101:0/2867605256 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff0240a13d0 0x7ff0240a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:55.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.494+0000 7ff02f7fe700 1 -- 172.21.15.101:0/2867605256 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7ff028014070 con 0x7ff0380731c0 2023-07-18T03:07:55.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.494+0000 7ff03e59b700 1 -- 172.21.15.101:0/3750640340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636b00d7ad0 msgr2=0x5636b00d9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:55.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.494+0000 7ff03e59b700 1 --2- 172.21.15.101:0/3750640340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636b00d7ad0 0x5636b00d9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.494+0000 7ff03e59b700 1 -- 172.21.15.101:0/3750640340 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636b00cc2d0 msgr2=0x5636b00cc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:55.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.494+0000 7ff03e59b700 1 --2- 172.21.15.101:0/3750640340 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636b00cc2d0 0x5636b00cc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.494+0000 7ff03e59b700 1 -- 172.21.15.101:0/3750640340 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636afe4cde0 con 0x5636b00ccbe0 2023-07-18T03:07:55.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.494+0000 7ff03ed9c700 1 --2- 172.21.15.101:0/3750640340 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636b00cc2d0 0x5636b00cc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:07:55.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.494+0000 7ff03e59b700 1 --2- 172.21.15.101:0/3750640340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636b00ccbe0 0x5636b00d7590 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff034005ea0 tx=0x7ff034005ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa4fa4219c06aab9 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:55.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.494+0000 7ff03ed9c700 1 --2- 172.21.15.101:0/2867605256 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff0240a13d0 0x7ff0240a3880 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff0300094f0 tx=0x7ff030004b40 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:55.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.495+0000 7ff02cff9700 1 -- 172.21.15.101:0/3750640340 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03401d070 con 0x5636b00ccbe0 2023-07-18T03:07:55.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.495+0000 7ff02cff9700 1 -- 172.21.15.101:0/3750640340 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff03401e040 con 0x5636b00ccbe0 2023-07-18T03:07:55.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.495+0000 7ff02cff9700 1 -- 172.21.15.101:0/3750640340 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff034004960 con 0x5636b00ccbe0 2023-07-18T03:07:55.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.495+0000 7ff054207b80 1 -- 172.21.15.101:0/3750640340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636b00ccbe0 msgr2=0x5636b00d7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:55.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.495+0000 7ff054207b80 1 --2- 172.21.15.101:0/3750640340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636b00ccbe0 0x5636b00d7590 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff034005ea0 tx=0x7ff034005ed0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.496+0000 7ff054207b80 1 -- 172.21.15.101:0/3750640340 shutdown_connections 2023-07-18T03:07:55.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.496+0000 7ff054207b80 1 --2- 172.21.15.101:0/3750640340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636b00d7ad0 0x5636b00d9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.496+0000 7ff054207b80 1 --2- 172.21.15.101:0/3750640340 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636b00cc2d0 0x5636b00cc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.496+0000 7ff054207b80 1 --2- 172.21.15.101:0/3750640340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636b00ccbe0 0x5636b00d7590 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.496+0000 7ff054207b80 1 -- 172.21.15.101:0/3750640340 >> 172.21.15.101:0/3750640340 conn(0x5636b00c2da0 msgr2=0x5636b00c86a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:55.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.496+0000 7ff054207b80 1 -- 172.21.15.101:0/3750640340 shutdown_connections 2023-07-18T03:07:55.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.496+0000 7ff054207b80 1 -- 172.21.15.101:0/3750640340 wait complete. 2023-07-18T03:07:55.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.496+0000 7ff054207b80 1 -- start start 2023-07-18T03:07:55.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.496+0000 7ff054207b80 1 Processor -- start 2023-07-18T03:07:55.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.497+0000 7ff054207b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636b00cc2d0 0x5636b00d7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:55.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.497+0000 7ff054207b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636b00ccbe0 0x5636b00d46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:55.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.497+0000 7ff054207b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636b00d7ad0 0x5636b00d4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:55.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.497+0000 7ff054207b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636affbeac0 con 0x5636b00cc2d0 2023-07-18T03:07:55.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.497+0000 7ff054207b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636affbe7c0 con 0x5636b00d7ad0 2023-07-18T03:07:55.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.497+0000 7ff054207b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636afea2920 con 0x5636b00ccbe0 2023-07-18T03:07:55.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.497+0000 7ff054207b80 10 client.0 _ll_register_callbacks cb 0x5636affbc760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:07:55.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.497+0000 7ff03ed9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636b00d7ad0 0x5636b00d4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:55.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.497+0000 7ff03ed9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636b00d7ad0 0x5636b00d4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:57516/0 (socket says 172.21.15.101:57516) 2023-07-18T03:07:55.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.497+0000 7ff03ed9c700 1 -- 172.21.15.101:0/2216586089 learned_addr learned my addr 172.21.15.101:0/2216586089 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:55.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.497+0000 7ff03dd9a700 1 --2- 172.21.15.101:0/2216586089 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636b00ccbe0 0x5636b00d46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:55.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.497+0000 7ff03e59b700 1 --2- 172.21.15.101:0/2216586089 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636b00cc2d0 0x5636b00d7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:55.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.498+0000 7ff03ed9c700 1 -- 172.21.15.101:0/2216586089 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636b00ccbe0 msgr2=0x5636b00d46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:55.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.498+0000 7ff03ed9c700 1 --2- 172.21.15.101:0/2216586089 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636b00ccbe0 0x5636b00d46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.498+0000 7ff03ed9c700 1 -- 172.21.15.101:0/2216586089 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636b00cc2d0 msgr2=0x5636b00d7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:55.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.498+0000 7ff03ed9c700 1 --2- 172.21.15.101:0/2216586089 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636b00cc2d0 0x5636b00d7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.498+0000 7ff03ed9c700 1 -- 172.21.15.101:0/2216586089 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff03810f3a0 con 0x5636b00d7ad0 2023-07-18T03:07:55.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.498+0000 7ff03ed9c700 1 --2- 172.21.15.101:0/2216586089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636b00d7ad0 0x5636b00d4d20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff03001aa30 tx=0x7ff030004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=124c59dd5c0ea6f3 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:55.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.498+0000 7ff021ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:07:55.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.498+0000 7ff021ffb700 1 -- 172.21.15.101:0/2216586089 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03001f070 con 0x5636b00d7ad0 2023-07-18T03:07:55.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.498+0000 7ff054207b80 1 -- 172.21.15.101:0/2216586089 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5636aff784f0 con 0x5636b00d7ad0 2023-07-18T03:07:55.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.498+0000 7ff021ffb700 1 -- 172.21.15.101:0/2216586089 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7ff03001a0d0 con 0x5636b00d7ad0 2023-07-18T03:07:55.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.499+0000 7ff021ffb700 1 -- 172.21.15.101:0/2216586089 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff03000e030 con 0x5636b00d7ad0 2023-07-18T03:07:55.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.499+0000 7ff054207b80 10 client.25622 fetch_fsmap learned FSMap version 24 2023-07-18T03:07:55.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.499+0000 7ff054207b80 1 -- 172.21.15.101:0/2216586089 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5636afe35c60 con 0x5636b00d7ad0 2023-07-18T03:07:55.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.499+0000 7ff021ffb700 1 -- 172.21.15.101:0/2216586089 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5636aff784f0 con 0x5636b00d7ad0 2023-07-18T03:07:55.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.499+0000 7ff021ffb700 1 -- 172.21.15.101:0/2216586089 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff030024860 con 0x5636b00d7ad0 2023-07-18T03:07:55.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.499+0000 7ff054207b80 10 client.25622 fetch_fsmap finished waiting for FSMap version 24 2023-07-18T03:07:55.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.499+0000 7ff054207b80 10 client.25622 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:07:55.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.499+0000 7ff054207b80 20 client.25622 populate_metadata read hostname 'smithi101' 2023-07-18T03:07:55.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.499+0000 7ff054207b80 1 --2- 172.21.15.101:0/2216586089 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5636b00e7380 0x5636b00e9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:55.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.499+0000 7ff054207b80 4 client.25622 mds_command: new command op to 24541 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:07:55.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.499+0000 7ff054207b80 1 -- 172.21.15.101:0/2216586089 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5636b0017eb0 con 0x5636b00e7380 2023-07-18T03:07:55.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.500+0000 7ff03e59b700 1 --2- 172.21.15.101:0/2216586089 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5636b00e7380 0x5636b00e9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:55.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.500+0000 7ff021ffb700 10 client.25622 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:07:55.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.500+0000 7ff03e59b700 1 --2- 172.21.15.101:0/2216586089 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5636b00e7380 0x5636b00e9790 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=152c1a79b04c6ca3 server_cookie=e79cd303b3538939 in_seq=0 out_seq=0 2023-07-18T03:07:55.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.501+0000 7ff021ffb700 1 -- 172.21.15.101:0/2216586089 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5636b0017eb0 con 0x5636b00e7380 2023-07-18T03:07:55.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.501+0000 7ff021ffb700 10 client.25622 handle_command_reply: tid=0 2023-07-18T03:07:55.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.501+0000 7ff054207b80 1 client.25622 shutdown 2023-07-18T03:07:55.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.502+0000 7ff054207b80 1 -- 172.21.15.101:0/2216586089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636b00d7ad0 msgr2=0x5636b00d4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:55.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.502+0000 7ff054207b80 1 --2- 172.21.15.101:0/2216586089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636b00d7ad0 0x5636b00d4d20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff03001aa30 tx=0x7ff030004ad0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.502+0000 7ff054207b80 1 -- 172.21.15.101:0/2216586089 shutdown_connections 2023-07-18T03:07:55.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.502+0000 7ff054207b80 1 --2- 172.21.15.101:0/2216586089 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5636b00e7380 0x5636b00e9790 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.502+0000 7ff054207b80 1 --2- 172.21.15.101:0/2216586089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636b00d7ad0 0x5636b00d4d20 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.502+0000 7ff054207b80 1 --2- 172.21.15.101:0/2216586089 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636b00cc2d0 0x5636b00d7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.502+0000 7ff054207b80 1 --2- 172.21.15.101:0/2216586089 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636b00ccbe0 0x5636b00d46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.502+0000 7ff021ffb700 0 client.25622 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:07:55.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.502+0000 7ff054207b80 1 -- 172.21.15.101:0/2216586089 >> 172.21.15.101:0/2216586089 conn(0x5636b00c2da0 msgr2=0x5636b00c8290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:55.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.502+0000 7ff054207b80 1 -- 172.21.15.101:0/2216586089 shutdown_connections 2023-07-18T03:07:55.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.502+0000 7ff054207b80 1 -- 172.21.15.101:0/2216586089 wait complete. 2023-07-18T03:07:55.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.502+0000 7ff054207b80 20 client.25622 trim_cache size 0 max 16384 2023-07-18T03:07:55.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.516+0000 7ff054207b80 1 Processor -- start 2023-07-18T03:07:55.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.516+0000 7ff054207b80 1 -- start start 2023-07-18T03:07:55.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.517+0000 7ff054207b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636b00cc2d0 0x5636b00de4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:55.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.517+0000 7ff054207b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636b00ccbe0 0x5636b00e3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:55.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.517+0000 7ff054207b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636b00d7ad0 0x5636b00ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:55.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.517+0000 7ff054207b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636b00cd540 con 0x5636b00d7ad0 2023-07-18T03:07:55.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.517+0000 7ff054207b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636b00d5cd0 con 0x5636b00cc2d0 2023-07-18T03:07:55.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.517+0000 7ff054207b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636b00d2c50 con 0x5636b00ccbe0 2023-07-18T03:07:55.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.517+0000 7ff03ed9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636b00d7ad0 0x5636b00ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:55.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.517+0000 7ff03ed9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636b00d7ad0 0x5636b00ce1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:42114/0 (socket says 172.21.15.101:42114) 2023-07-18T03:07:55.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.517+0000 7ff03dd9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636b00ccbe0 0x5636b00e3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:55.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.517+0000 7ff03ed9c700 1 -- 172.21.15.101:0/2429375444 learned_addr learned my addr 172.21.15.101:0/2429375444 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:55.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.517+0000 7ff03e59b700 1 --2- 172.21.15.101:0/2429375444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636b00cc2d0 0x5636b00de4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:55.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.517+0000 7ff03ed9c700 1 -- 172.21.15.101:0/2429375444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636b00ccbe0 msgr2=0x5636b00e3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:55.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.517+0000 7ff03ed9c700 1 --2- 172.21.15.101:0/2429375444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636b00ccbe0 0x5636b00e3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.517+0000 7ff03ed9c700 1 -- 172.21.15.101:0/2429375444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636b00cc2d0 msgr2=0x5636b00de4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:55.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.517+0000 7ff03ed9c700 1 --2- 172.21.15.101:0/2429375444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636b00cc2d0 0x5636b00de4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.517+0000 7ff03ed9c700 1 -- 172.21.15.101:0/2429375444 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636afe73250 con 0x5636b00d7ad0 2023-07-18T03:07:55.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.518+0000 7ff03ed9c700 1 --2- 172.21.15.101:0/2429375444 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636b00d7ad0 0x5636b00ce1a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff0300099b0 tx=0x7ff030009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7762efefa362ee15 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:55.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.518+0000 7ff02d7fa700 1 -- 172.21.15.101:0/2429375444 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03001f070 con 0x5636b00d7ad0 2023-07-18T03:07:55.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.518+0000 7ff02d7fa700 1 -- 172.21.15.101:0/2429375444 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff03000f040 con 0x5636b00d7ad0 2023-07-18T03:07:55.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.518+0000 7ff02d7fa700 1 -- 172.21.15.101:0/2429375444 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff030017a40 con 0x5636b00d7ad0 2023-07-18T03:07:55.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.519+0000 7ff054207b80 1 -- 172.21.15.101:0/2429375444 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636b00d7ad0 msgr2=0x5636b00ce1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:55.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.519+0000 7ff054207b80 1 --2- 172.21.15.101:0/2429375444 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636b00d7ad0 0x5636b00ce1a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff0300099b0 tx=0x7ff030009f00 comp rx=0 tx=0).stop 2023-07-18T03:07:55.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.519+0000 7ff054207b80 1 -- 172.21.15.101:0/2429375444 shutdown_connections 2023-07-18T03:07:55.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.519+0000 7ff054207b80 1 --2- 172.21.15.101:0/2429375444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636b00cc2d0 0x5636b00de4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.519+0000 7ff054207b80 1 --2- 172.21.15.101:0/2429375444 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636b00d7ad0 0x5636b00ce1a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.519+0000 7ff054207b80 1 --2- 172.21.15.101:0/2429375444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636b00ccbe0 0x5636b00e3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.519+0000 7ff054207b80 1 -- 172.21.15.101:0/2429375444 >> 172.21.15.101:0/2429375444 conn(0x5636b00c2da0 msgr2=0x5636b00c6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:55.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.519+0000 7ff054207b80 1 -- 172.21.15.101:0/2429375444 shutdown_connections 2023-07-18T03:07:55.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.519+0000 7ff054207b80 1 -- 172.21.15.101:0/2429375444 wait complete. 2023-07-18T03:07:55.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.519+0000 7ff054207b80 1 -- start start 2023-07-18T03:07:55.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.519+0000 7ff054207b80 1 Processor -- start 2023-07-18T03:07:55.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.520+0000 7ff054207b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636b00cc2d0 0x5636b00ddee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:55.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.520+0000 7ff054207b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636b00ccbe0 0x5636b00de420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:55.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.520+0000 7ff054207b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636b00d7ad0 0x5636b00dc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:55.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.520+0000 7ff054207b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636b00dca90 con 0x5636b00ccbe0 2023-07-18T03:07:55.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.520+0000 7ff054207b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636b00dcc00 con 0x5636b00cc2d0 2023-07-18T03:07:55.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.520+0000 7ff054207b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636b00dcdc0 con 0x5636b00d7ad0 2023-07-18T03:07:55.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.520+0000 7ff054207b80 10 client.0 _ll_register_callbacks cb 0x7ff03001a610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:07:55.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.520+0000 7ff03dd9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636b00ccbe0 0x5636b00de420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:55.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.520+0000 7ff03dd9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636b00ccbe0 0x5636b00de420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:42120/0 (socket says 172.21.15.101:42120) 2023-07-18T03:07:55.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.520+0000 7ff03dd9a700 1 -- 172.21.15.101:0/1160218167 learned_addr learned my addr 172.21.15.101:0/1160218167 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:55.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.520+0000 7ff03e59b700 1 --2- 172.21.15.101:0/1160218167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636b00cc2d0 0x5636b00ddee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:55.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.520+0000 7ff03ed9c700 1 --2- 172.21.15.101:0/1160218167 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636b00d7ad0 0x5636b00dc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:55.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.521+0000 7ff03dd9a700 1 -- 172.21.15.101:0/1160218167 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636b00d7ad0 msgr2=0x5636b00dc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:55.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.521+0000 7ff03dd9a700 1 --2- 172.21.15.101:0/1160218167 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636b00d7ad0 0x5636b00dc490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.521+0000 7ff03dd9a700 1 -- 172.21.15.101:0/1160218167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636b00cc2d0 msgr2=0x5636b00ddee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:55.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.521+0000 7ff03dd9a700 1 --2- 172.21.15.101:0/1160218167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636b00cc2d0 0x5636b00ddee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.521+0000 7ff03dd9a700 1 -- 172.21.15.101:0/1160218167 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5636afe35c60 con 0x5636b00ccbe0 2023-07-18T03:07:55.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.521+0000 7ff03e59b700 1 --2- 172.21.15.101:0/1160218167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636b00cc2d0 0x5636b00ddee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:07:55.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.521+0000 7ff03dd9a700 1 --2- 172.21.15.101:0/1160218167 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636b00ccbe0 0x5636b00de420 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff0280cae30 tx=0x7ff02806c0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1664170030b44044 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:55.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.521+0000 7ff021ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:07:55.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.521+0000 7ff021ffb700 1 -- 172.21.15.101:0/1160218167 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff028099730 con 0x5636b00ccbe0 2023-07-18T03:07:55.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.521+0000 7ff054207b80 1 -- 172.21.15.101:0/1160218167 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff03401e040 con 0x5636b00ccbe0 2023-07-18T03:07:55.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.521+0000 7ff021ffb700 1 -- 172.21.15.101:0/1160218167 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7ff02809a040 con 0x5636b00ccbe0 2023-07-18T03:07:55.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.521+0000 7ff021ffb700 1 -- 172.21.15.101:0/1160218167 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff028016330 con 0x5636b00ccbe0 2023-07-18T03:07:55.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.521+0000 7ff054207b80 10 client.15600 fetch_fsmap learned FSMap version 24 2023-07-18T03:07:55.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.521+0000 7ff054207b80 1 -- 172.21.15.101:0/1160218167 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5636afe4b2b0 con 0x5636b00ccbe0 2023-07-18T03:07:55.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.522+0000 7ff021ffb700 1 -- 172.21.15.101:0/1160218167 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff03401e040 con 0x5636b00ccbe0 2023-07-18T03:07:55.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.522+0000 7ff021ffb700 1 -- 172.21.15.101:0/1160218167 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff0280a2000 con 0x5636b00ccbe0 2023-07-18T03:07:55.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.522+0000 7ff054207b80 10 client.15600 fetch_fsmap finished waiting for FSMap version 24 2023-07-18T03:07:55.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.522+0000 7ff054207b80 10 client.15600 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:07:55.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.522+0000 7ff054207b80 20 client.15600 populate_metadata read hostname 'smithi101' 2023-07-18T03:07:55.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.522+0000 7ff054207b80 1 --2- 172.21.15.101:0/1160218167 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5636b013c530 0x5636b013e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:55.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.522+0000 7ff054207b80 4 client.15600 mds_command: new command op to 24541 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:07:55.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.522+0000 7ff054207b80 1 -- 172.21.15.101:0/1160218167 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5636affda050 con 0x5636b013c530 2023-07-18T03:07:55.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.522+0000 7ff03e59b700 1 --2- 172.21.15.101:0/1160218167 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5636b013c530 0x5636b013e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:55.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.523+0000 7ff03e59b700 1 --2- 172.21.15.101:0/1160218167 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5636b013c530 0x5636b013e940 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=30c73a270c59799b server_cookie=dc29e7db15c5ca46 in_seq=0 out_seq=0 2023-07-18T03:07:55.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.523+0000 7ff021ffb700 10 client.15600 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:07:55.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.523+0000 7ff021ffb700 1 -- 172.21.15.101:0/1160218167 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5636affda050 con 0x5636b013c530 2023-07-18T03:07:55.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.523+0000 7ff021ffb700 10 client.15600 handle_command_reply: tid=0 2023-07-18T03:07:55.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.523+0000 7ff054207b80 1 client.15600 shutdown 2023-07-18T03:07:55.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.523+0000 7ff054207b80 1 -- 172.21.15.101:0/1160218167 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636b00ccbe0 msgr2=0x5636b00de420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:55.524 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.523+0000 7ff054207b80 1 --2- 172.21.15.101:0/1160218167 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636b00ccbe0 0x5636b00de420 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff0280cae30 tx=0x7ff02806c0b0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.524 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.523+0000 7ff054207b80 1 -- 172.21.15.101:0/1160218167 shutdown_connections 2023-07-18T03:07:55.524 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.523+0000 7ff054207b80 1 --2- 172.21.15.101:0/1160218167 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5636b013c530 0x5636b013e940 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.524 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.523+0000 7ff054207b80 1 --2- 172.21.15.101:0/1160218167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636b00cc2d0 0x5636b00ddee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.524 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.523+0000 7ff054207b80 1 --2- 172.21.15.101:0/1160218167 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636b00d7ad0 0x5636b00dc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.524 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.523+0000 7ff054207b80 1 --2- 172.21.15.101:0/1160218167 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636b00ccbe0 0x5636b00de420 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.524 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.523+0000 7ff021ffb700 0 client.15600 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:07:55.525 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.523+0000 7ff054207b80 1 -- 172.21.15.101:0/1160218167 >> 172.21.15.101:0/1160218167 conn(0x5636b00c2da0 msgr2=0x5636b00ce1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:55.525 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.524+0000 7ff054207b80 1 -- 172.21.15.101:0/1160218167 shutdown_connections 2023-07-18T03:07:55.525 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.524+0000 7ff054207b80 1 -- 172.21.15.101:0/1160218167 wait complete. 2023-07-18T03:07:55.525 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.524+0000 7ff054207b80 20 client.15600 trim_cache size 0 max 16384 2023-07-18T03:07:55.525 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:07:55.526 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.524+0000 7ff0407ff700 1 -- 172.21.15.101:0/2867605256 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff0240a13d0 msgr2=0x7ff0240a3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:55.526 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.524+0000 7ff0407ff700 1 --2- 172.21.15.101:0/2867605256 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff0240a13d0 0x7ff0240a3880 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff0300094f0 tx=0x7ff030004b40 comp rx=0 tx=0).stop 2023-07-18T03:07:55.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.524+0000 7ff0407ff700 1 -- 172.21.15.101:0/2867605256 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff0380731c0 msgr2=0x7ff038198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:55.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.524+0000 7ff0407ff700 1 --2- 172.21.15.101:0/2867605256 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff0380731c0 0x7ff038198350 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff02800e9a0 tx=0x7ff02800ecb0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.524+0000 7ff0407ff700 1 -- 172.21.15.101:0/2867605256 shutdown_connections 2023-07-18T03:07:55.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.524+0000 7ff0407ff700 1 --2- 172.21.15.101:0/2867605256 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff0240a13d0 0x7ff0240a3880 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.524+0000 7ff0407ff700 1 --2- 172.21.15.101:0/2867605256 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0380728b0 0x7ff038197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.524+0000 7ff0407ff700 1 --2- 172.21.15.101:0/2867605256 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff03810c320 0x7ff03819c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.524+0000 7ff0407ff700 1 --2- 172.21.15.101:0/2867605256 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff0380731c0 0x7ff038198350 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.524+0000 7ff0407ff700 1 -- 172.21.15.101:0/2867605256 >> 172.21.15.101:0/2867605256 conn(0x7ff0380fb5e0 msgr2=0x7ff03810e470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:55.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.525+0000 7ff0407ff700 1 -- 172.21.15.101:0/2867605256 shutdown_connections 2023-07-18T03:07:55.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.525+0000 7ff0407ff700 1 -- 172.21.15.101:0/2867605256 wait complete. 2023-07-18T03:07:55.531 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-18T03:07:55.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.763+0000 7f3fd35ff700 1 -- 172.21.15.101:0/3668723928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3fcc105b10 msgr2=0x7f3fcc105ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:55.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.763+0000 7f3fd35ff700 1 --2- 172.21.15.101:0/3668723928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3fcc105b10 0x7f3fcc105ee0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3fc80098f0 tx=0x7f3fc8009c00 comp rx=0 tx=0).stop 2023-07-18T03:07:55.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.763+0000 7f3fd35ff700 1 -- 172.21.15.101:0/3668723928 shutdown_connections 2023-07-18T03:07:55.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.763+0000 7f3fd35ff700 1 --2- 172.21.15.101:0/3668723928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3fcc105b10 0x7f3fcc105ee0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.763+0000 7f3fd35ff700 1 --2- 172.21.15.101:0/3668723928 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3fcc1004b0 0x7f3fcc10c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.763+0000 7f3fd35ff700 1 --2- 172.21.15.101:0/3668723928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3fcc0ffb00 0x7f3fcc0fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.763+0000 7f3fd35ff700 1 -- 172.21.15.101:0/3668723928 >> 172.21.15.101:0/3668723928 conn(0x7f3fcc0fb600 msgr2=0x7f3fcc0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:55.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.763+0000 7f3fd35ff700 1 -- 172.21.15.101:0/3668723928 shutdown_connections 2023-07-18T03:07:55.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.763+0000 7f3fd35ff700 1 -- 172.21.15.101:0/3668723928 wait complete. 2023-07-18T03:07:55.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.764+0000 7f3fd35ff700 1 Processor -- start 2023-07-18T03:07:55.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.764+0000 7f3fd35ff700 1 -- start start 2023-07-18T03:07:55.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.764+0000 7f3fd35ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3fcc0ffb00 0x7f3fcc072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:55.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.765+0000 7f3fd35ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3fcc1004b0 0x7f3fcc072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:55.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.765+0000 7f3fd35ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3fcc105b10 0x7f3fcc1a3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:55.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.765+0000 7f3fd35ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc073580 con 0x7f3fcc1004b0 2023-07-18T03:07:55.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.765+0000 7f3fd35ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc1a4390 con 0x7f3fcc0ffb00 2023-07-18T03:07:55.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.765+0000 7f3fd35ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc1a44d0 con 0x7f3fcc105b10 2023-07-18T03:07:55.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.765+0000 7f3fd0b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3fcc1004b0 0x7f3fcc072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:55.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.765+0000 7f3fd0b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3fcc1004b0 0x7f3fcc072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:42130/0 (socket says 172.21.15.101:42130) 2023-07-18T03:07:55.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.765+0000 7f3fd139b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3fcc0ffb00 0x7f3fcc072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:55.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.765+0000 7f3fd0b9a700 1 -- 172.21.15.101:0/321815979 learned_addr learned my addr 172.21.15.101:0/321815979 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:55.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.765+0000 7f3fd1b9c700 1 --2- 172.21.15.101:0/321815979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3fcc105b10 0x7f3fcc1a3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:55.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.765+0000 7f3fd139b700 1 -- 172.21.15.101:0/321815979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3fcc105b10 msgr2=0x7f3fcc1a3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:55.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.765+0000 7f3fd139b700 1 --2- 172.21.15.101:0/321815979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3fcc105b10 0x7f3fcc1a3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.765+0000 7f3fd139b700 1 -- 172.21.15.101:0/321815979 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3fcc1004b0 msgr2=0x7f3fcc072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:55.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.765+0000 7f3fd139b700 1 --2- 172.21.15.101:0/321815979 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3fcc1004b0 0x7f3fcc072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.765+0000 7f3fd139b700 1 -- 172.21.15.101:0/321815979 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc8009520 con 0x7f3fcc0ffb00 2023-07-18T03:07:55.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.765+0000 7f3fd0b9a700 1 --2- 172.21.15.101:0/321815979 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3fcc1004b0 0x7f3fcc072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:07:55.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.766+0000 7f3fd139b700 1 --2- 172.21.15.101:0/321815979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3fcc0ffb00 0x7f3fcc072810 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8009810 tx=0x7f3fc800fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:55.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.766+0000 7f3fc27fc700 1 -- 172.21.15.101:0/321815979 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc801c070 con 0x7f3fcc0ffb00 2023-07-18T03:07:55.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.766+0000 7f3fd35ff700 1 -- 172.21.15.101:0/321815979 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fcc1a46d0 con 0x7f3fcc0ffb00 2023-07-18T03:07:55.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.766+0000 7f3fc27fc700 1 -- 172.21.15.101:0/321815979 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fc800bba0 con 0x7f3fcc0ffb00 2023-07-18T03:07:55.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.766+0000 7f3fc27fc700 1 -- 172.21.15.101:0/321815979 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc8003970 con 0x7f3fcc0ffb00 2023-07-18T03:07:55.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.766+0000 7f3fd35ff700 1 -- 172.21.15.101:0/321815979 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fcc1a4af0 con 0x7f3fcc0ffb00 2023-07-18T03:07:55.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.769+0000 7f3fc27fc700 1 -- 172.21.15.101:0/321815979 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 23) v1 ==== 135892+0+0 (secure 0 0 0) 0x7f3fc8020030 con 0x7f3fcc0ffb00 2023-07-18T03:07:55.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.771+0000 7f3fe6e7cb80 1 Processor -- start 2023-07-18T03:07:55.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.771+0000 7f3fe6e7cb80 1 -- start start 2023-07-18T03:07:55.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.771+0000 7f3fe6e7cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55821175f2d0 0x55821175f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:55.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.771+0000 7f3fe6e7cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55821175fbe0 0x55821176a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:55.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.771+0000 7f3fe6e7cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55821176aad0 0x55821176ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:55.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.771+0000 7f3fe6e7cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558211665cb0 con 0x55821176aad0 2023-07-18T03:07:55.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.771+0000 7f3fe6e7cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582116b8a40 con 0x55821175fbe0 2023-07-18T03:07:55.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.771+0000 7f3fe6e7cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582116b1ed0 con 0x55821175f2d0 2023-07-18T03:07:55.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.772+0000 7f3fd1b9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55821175f2d0 0x55821175f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:55.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.772+0000 7f3fd1b9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55821175f2d0 0x55821175f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:46854/0 (socket says 172.21.15.101:46854) 2023-07-18T03:07:55.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.772+0000 7f3fd1b9c700 1 -- 172.21.15.101:0/2669970719 learned_addr learned my addr 172.21.15.101:0/2669970719 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:55.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.772+0000 7f3fc27fc700 1 --2- 172.21.15.101:0/321815979 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f3fb80a1350 0x7f3fb80a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:55.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.772+0000 7f3fd0b9a700 1 --2- 172.21.15.101:0/2669970719 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55821176aad0 0x55821176ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:55.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.772+0000 7f3fd139b700 1 --2- 172.21.15.101:0/2669970719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55821175fbe0 0x55821176a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:55.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.772+0000 7f3fc27fc700 1 -- 172.21.15.101:0/321815979 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f3fc80cdff0 con 0x7f3fcc0ffb00 2023-07-18T03:07:55.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.772+0000 7f3fd1b9c700 1 --2- 172.21.15.101:0/321815979 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f3fb80a1350 0x7f3fb80a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:55.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.772+0000 7f3fd0b9a700 1 -- 172.21.15.101:0/2669970719 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55821175f2d0 msgr2=0x55821175f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:55.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.772+0000 7f3fd0b9a700 1 --2- 172.21.15.101:0/2669970719 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55821175f2d0 0x55821175f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.772+0000 7f3fd0b9a700 1 -- 172.21.15.101:0/2669970719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55821175fbe0 msgr2=0x55821176a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:55.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.772+0000 7f3fd0b9a700 1 --2- 172.21.15.101:0/2669970719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55821175fbe0 0x55821176a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.772+0000 7f3fd0b9a700 1 -- 172.21.15.101:0/2669970719 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582114dfde0 con 0x55821176aad0 2023-07-18T03:07:55.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.772+0000 7f3fd139b700 1 --2- 172.21.15.101:0/2669970719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55821175fbe0 0x55821176a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:07:55.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.773+0000 7f3fd0b9a700 1 --2- 172.21.15.101:0/2669970719 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55821176aad0 0x55821176ceb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc011540 tx=0x7f3fbc011850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f2c74fd62b83d77 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:55.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.773+0000 7f3fd1b9c700 1 --2- 172.21.15.101:0/321815979 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f3fb80a1350 0x7f3fb80a3800 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3fc40060b0 tx=0x7f3fc4004b40 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:55.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.773+0000 7f3fb77fe700 1 -- 172.21.15.101:0/2669970719 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc004230 con 0x55821176aad0 2023-07-18T03:07:55.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.773+0000 7f3fb77fe700 1 -- 172.21.15.101:0/2669970719 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fbc004390 con 0x55821176aad0 2023-07-18T03:07:55.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.773+0000 7f3fb77fe700 1 -- 172.21.15.101:0/2669970719 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc0198e0 con 0x55821176aad0 2023-07-18T03:07:55.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.774+0000 7f3fe6e7cb80 1 -- 172.21.15.101:0/2669970719 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55821176aad0 msgr2=0x55821176ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:55.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.774+0000 7f3fe6e7cb80 1 --2- 172.21.15.101:0/2669970719 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55821176aad0 0x55821176ceb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc011540 tx=0x7f3fbc011850 comp rx=0 tx=0).stop 2023-07-18T03:07:55.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.774+0000 7f3fe6e7cb80 1 -- 172.21.15.101:0/2669970719 shutdown_connections 2023-07-18T03:07:55.769 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.774+0000 7f3fe6e7cb80 1 --2- 172.21.15.101:0/2669970719 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55821176aad0 0x55821176ceb0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.769 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.774+0000 7f3fe6e7cb80 1 --2- 172.21.15.101:0/2669970719 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55821175f2d0 0x55821175f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.769 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.774+0000 7f3fe6e7cb80 1 --2- 172.21.15.101:0/2669970719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55821175fbe0 0x55821176a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.769 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.774+0000 7f3fe6e7cb80 1 -- 172.21.15.101:0/2669970719 >> 172.21.15.101:0/2669970719 conn(0x558211755da0 msgr2=0x55821175b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:55.769 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.774+0000 7f3fe6e7cb80 1 -- 172.21.15.101:0/2669970719 shutdown_connections 2023-07-18T03:07:55.769 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.774+0000 7f3fe6e7cb80 1 -- 172.21.15.101:0/2669970719 wait complete. 2023-07-18T03:07:55.769 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.774+0000 7f3fe6e7cb80 1 -- start start 2023-07-18T03:07:55.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.775+0000 7f3fe6e7cb80 1 Processor -- start 2023-07-18T03:07:55.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.775+0000 7f3fe6e7cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55821175f2d0 0x55821176a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:55.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.775+0000 7f3fe6e7cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55821175fbe0 0x558211767760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:55.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.775+0000 7f3fe6e7cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55821176aad0 0x558211767e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:55.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.776+0000 7f3fe6e7cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558211651ac0 con 0x55821176aad0 2023-07-18T03:07:55.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.776+0000 7f3fe6e7cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582116517c0 con 0x55821175f2d0 2023-07-18T03:07:55.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.776+0000 7f3fe6e7cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x558211535920 con 0x55821175fbe0 2023-07-18T03:07:55.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.776+0000 7f3fe6e7cb80 10 client.0 _ll_register_callbacks cb 0x55821164f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:07:55.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.776+0000 7f3fd0b9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55821175fbe0 0x558211767760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:55.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.776+0000 7f3fd0b9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55821175fbe0 0x558211767760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:46856/0 (socket says 172.21.15.101:46856) 2023-07-18T03:07:55.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.776+0000 7f3fd1b9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55821176aad0 0x558211767e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:55.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.776+0000 7f3fd0b9a700 1 -- 172.21.15.101:0/1868501643 learned_addr learned my addr 172.21.15.101:0/1868501643 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:55.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.776+0000 7f3fd139b700 1 --2- 172.21.15.101:0/1868501643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55821175f2d0 0x55821176a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:55.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.776+0000 7f3fd0b9a700 1 -- 172.21.15.101:0/1868501643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55821175f2d0 msgr2=0x55821176a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:55.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.776+0000 7f3fd0b9a700 1 --2- 172.21.15.101:0/1868501643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55821175f2d0 0x55821176a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.776+0000 7f3fd0b9a700 1 -- 172.21.15.101:0/1868501643 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55821176aad0 msgr2=0x558211767e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:55.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.776+0000 7f3fd0b9a700 1 --2- 172.21.15.101:0/1868501643 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55821176aad0 0x558211767e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.776+0000 7f3fd0b9a700 1 -- 172.21.15.101:0/1868501643 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fcc1a4af0 con 0x55821175fbe0 2023-07-18T03:07:55.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.777+0000 7f3fd0b9a700 1 --2- 172.21.15.101:0/1868501643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55821175fbe0 0x558211767760 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc009850 tx=0x7f3fbc0056a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ced4f1d6c1521026 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:55.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.777+0000 7f3fb4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:07:55.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.777+0000 7f3fb4ff9700 1 -- 172.21.15.101:0/1868501643 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc004b60 con 0x55821175fbe0 2023-07-18T03:07:55.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.777+0000 7f3fe6e7cb80 1 -- 172.21.15.101:0/1868501643 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55821160b4f0 con 0x55821175fbe0 2023-07-18T03:07:55.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.777+0000 7f3fb4ff9700 1 -- 172.21.15.101:0/1868501643 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f3fbc004cc0 con 0x55821175fbe0 2023-07-18T03:07:55.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.777+0000 7f3fb4ff9700 1 -- 172.21.15.101:0/1868501643 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fbc004e80 con 0x55821175fbe0 2023-07-18T03:07:55.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.777+0000 7f3fe6e7cb80 10 client.25675 fetch_fsmap learned FSMap version 24 2023-07-18T03:07:55.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.777+0000 7f3fe6e7cb80 1 -- 172.21.15.101:0/1868501643 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f3fbc009d60 con 0x55821175fbe0 2023-07-18T03:07:55.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.778+0000 7f3fb4ff9700 1 -- 172.21.15.101:0/1868501643 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55821160b4f0 con 0x55821175fbe0 2023-07-18T03:07:55.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.778+0000 7f3fb4ff9700 1 -- 172.21.15.101:0/1868501643 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3fbc021040 con 0x55821175fbe0 2023-07-18T03:07:55.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.778+0000 7f3fe6e7cb80 10 client.25675 fetch_fsmap finished waiting for FSMap version 24 2023-07-18T03:07:55.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.778+0000 7f3fe6e7cb80 10 client.25675 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:07:55.774 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.778+0000 7f3fe6e7cb80 20 client.25675 populate_metadata read hostname 'smithi101' 2023-07-18T03:07:55.774 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.778+0000 7f3fe6e7cb80 1 --2- 172.21.15.101:0/1868501643 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55821177a380 0x55821177c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:55.774 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.778+0000 7f3fe6e7cb80 4 client.25675 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:07:55.774 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.778+0000 7f3fe6e7cb80 1 -- 172.21.15.101:0/1868501643 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5582116aaeb0 con 0x55821177a380 2023-07-18T03:07:55.774 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.778+0000 7f3fd139b700 1 --2- 172.21.15.101:0/1868501643 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55821177a380 0x55821177c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:55.774 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.779+0000 7f3fb4ff9700 10 client.25675 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:07:55.774 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.779+0000 7f3fd139b700 1 --2- 172.21.15.101:0/1868501643 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55821177a380 0x55821177c790 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=714a9c936db27997 server_cookie=54090f5ced7c64ae in_seq=0 out_seq=0 2023-07-18T03:07:55.775 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.780+0000 7f3fb4ff9700 1 -- 172.21.15.101:0/1868501643 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5582116aaeb0 con 0x55821177a380 2023-07-18T03:07:55.775 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.780+0000 7f3fb4ff9700 10 client.25675 handle_command_reply: tid=0 2023-07-18T03:07:55.775 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.780+0000 7f3fe6e7cb80 1 client.25675 shutdown 2023-07-18T03:07:55.775 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.780+0000 7f3fe6e7cb80 1 -- 172.21.15.101:0/1868501643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55821175fbe0 msgr2=0x558211767760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:55.775 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.780+0000 7f3fe6e7cb80 1 --2- 172.21.15.101:0/1868501643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55821175fbe0 0x558211767760 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc009850 tx=0x7f3fbc0056a0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.775 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.780+0000 7f3fe6e7cb80 1 -- 172.21.15.101:0/1868501643 shutdown_connections 2023-07-18T03:07:55.775 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.780+0000 7f3fe6e7cb80 1 --2- 172.21.15.101:0/1868501643 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55821177a380 0x55821177c790 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.775 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.780+0000 7f3fe6e7cb80 1 --2- 172.21.15.101:0/1868501643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55821175f2d0 0x55821176a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.776 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.780+0000 7f3fb4ff9700 0 client.25675 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:07:55.776 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.780+0000 7f3fe6e7cb80 1 --2- 172.21.15.101:0/1868501643 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55821176aad0 0x558211767e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.776 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.780+0000 7f3fe6e7cb80 1 --2- 172.21.15.101:0/1868501643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55821175fbe0 0x558211767760 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.776 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.780+0000 7f3fe6e7cb80 1 -- 172.21.15.101:0/1868501643 >> 172.21.15.101:0/1868501643 conn(0x558211755da0 msgr2=0x55821175b230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:55.776 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.780+0000 7f3fe6e7cb80 1 -- 172.21.15.101:0/1868501643 shutdown_connections 2023-07-18T03:07:55.776 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.780+0000 7f3fe6e7cb80 1 -- 172.21.15.101:0/1868501643 wait complete. 2023-07-18T03:07:55.776 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.780+0000 7f3fe6e7cb80 20 client.25675 trim_cache size 0 max 16384 2023-07-18T03:07:55.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.794+0000 7f3fe6e7cb80 1 Processor -- start 2023-07-18T03:07:55.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.794+0000 7f3fe6e7cb80 1 -- start start 2023-07-18T03:07:55.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.794+0000 7f3fe6e7cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55821175f2d0 0x558211767720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:55.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.794+0000 7f3fe6e7cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55821175fbe0 0x5582117810b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:55.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.794+0000 7f3fe6e7cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55821176aad0 0x55821176a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:55.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.794+0000 7f3fe6e7cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582117611f0 con 0x55821175f2d0 2023-07-18T03:07:55.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.794+0000 7f3fe6e7cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55821175cad0 con 0x55821175fbe0 2023-07-18T03:07:55.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.794+0000 7f3fe6e7cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582117815f0 con 0x55821176aad0 2023-07-18T03:07:55.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.794+0000 7f3fd139b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55821175f2d0 0x558211767720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:55.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.794+0000 7f3fd139b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55821175f2d0 0x558211767720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:42158/0 (socket says 172.21.15.101:42158) 2023-07-18T03:07:55.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.794+0000 7f3fd139b700 1 -- 172.21.15.101:0/3681104773 learned_addr learned my addr 172.21.15.101:0/3681104773 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:55.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.795+0000 7f3fd0b9a700 1 --2- 172.21.15.101:0/3681104773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55821175fbe0 0x5582117810b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:55.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.795+0000 7f3fd1b9c700 1 --2- 172.21.15.101:0/3681104773 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55821176aad0 0x55821176a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:55.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.795+0000 7f3fd139b700 1 -- 172.21.15.101:0/3681104773 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55821176aad0 msgr2=0x55821176a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:55.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.795+0000 7f3fd139b700 1 --2- 172.21.15.101:0/3681104773 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55821176aad0 0x55821176a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.795+0000 7f3fd139b700 1 -- 172.21.15.101:0/3681104773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55821175fbe0 msgr2=0x5582117810b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:55.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.795+0000 7f3fd139b700 1 --2- 172.21.15.101:0/3681104773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55821175fbe0 0x5582117810b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.795+0000 7f3fd139b700 1 -- 172.21.15.101:0/3681104773 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582114c8c60 con 0x55821175f2d0 2023-07-18T03:07:55.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.795+0000 7f3fd139b700 1 --2- 172.21.15.101:0/3681104773 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55821175f2d0 0x558211767720 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8009340 tx=0x7f3fc8066540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7f4748cc451a83c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:55.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.795+0000 7f3fb7fff700 1 -- 172.21.15.101:0/3681104773 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc80b3070 con 0x55821175f2d0 2023-07-18T03:07:55.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.795+0000 7f3fb7fff700 1 -- 172.21.15.101:0/3681104773 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fc809ac50 con 0x55821175f2d0 2023-07-18T03:07:55.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.795+0000 7f3fb7fff700 1 -- 172.21.15.101:0/3681104773 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc8097360 con 0x55821175f2d0 2023-07-18T03:07:55.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.796+0000 7f3fe6e7cb80 1 -- 172.21.15.101:0/3681104773 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55821175f2d0 msgr2=0x558211767720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:55.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.796+0000 7f3fe6e7cb80 1 --2- 172.21.15.101:0/3681104773 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55821175f2d0 0x558211767720 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8009340 tx=0x7f3fc8066540 comp rx=0 tx=0).stop 2023-07-18T03:07:55.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.796+0000 7f3fe6e7cb80 1 -- 172.21.15.101:0/3681104773 shutdown_connections 2023-07-18T03:07:55.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.796+0000 7f3fe6e7cb80 1 --2- 172.21.15.101:0/3681104773 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55821176aad0 0x55821176a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.796+0000 7f3fe6e7cb80 1 --2- 172.21.15.101:0/3681104773 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55821175f2d0 0x558211767720 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.796+0000 7f3fe6e7cb80 1 --2- 172.21.15.101:0/3681104773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55821175fbe0 0x5582117810b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.796+0000 7f3fe6e7cb80 1 -- 172.21.15.101:0/3681104773 >> 172.21.15.101:0/3681104773 conn(0x558211755da0 msgr2=0x558211759e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:55.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.796+0000 7f3fe6e7cb80 1 -- 172.21.15.101:0/3681104773 shutdown_connections 2023-07-18T03:07:55.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.796+0000 7f3fe6e7cb80 1 -- 172.21.15.101:0/3681104773 wait complete. 2023-07-18T03:07:55.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.797+0000 7f3fe6e7cb80 1 -- start start 2023-07-18T03:07:55.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.797+0000 7f3fe6e7cb80 1 Processor -- start 2023-07-18T03:07:55.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.797+0000 7f3fe6e7cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55821175f2d0 0x55821176f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:55.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.797+0000 7f3fe6e7cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55821175fbe0 0x55821176fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:55.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.797+0000 7f3fe6e7cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55821176aad0 0x5582117cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:55.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.797+0000 7f3fe6e7cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558211770370 con 0x55821175f2d0 2023-07-18T03:07:55.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.797+0000 7f3fe6e7cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582117cd740 con 0x55821175fbe0 2023-07-18T03:07:55.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.797+0000 7f3fe6e7cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582117cd880 con 0x55821176aad0 2023-07-18T03:07:55.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.797+0000 7f3fe6e7cb80 10 client.0 _ll_register_callbacks cb 0x7f3fc80cb180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:07:55.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.798+0000 7f3fd139b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55821175f2d0 0x55821176f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:55.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.798+0000 7f3fd139b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55821175f2d0 0x55821176f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:42160/0 (socket says 172.21.15.101:42160) 2023-07-18T03:07:55.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.798+0000 7f3fd139b700 1 -- 172.21.15.101:0/1604390185 learned_addr learned my addr 172.21.15.101:0/1604390185 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:55.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.798+0000 7f3fd0b9a700 1 --2- 172.21.15.101:0/1604390185 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55821175fbe0 0x55821176fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:55.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.798+0000 7f3fd1b9c700 1 --2- 172.21.15.101:0/1604390185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55821176aad0 0x5582117cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:55.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.798+0000 7f3fd139b700 1 -- 172.21.15.101:0/1604390185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55821176aad0 msgr2=0x5582117cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:55.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.798+0000 7f3fd139b700 1 --2- 172.21.15.101:0/1604390185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55821176aad0 0x5582117cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.798+0000 7f3fd139b700 1 -- 172.21.15.101:0/1604390185 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55821175fbe0 msgr2=0x55821176fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:55.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.798+0000 7f3fd139b700 1 --2- 172.21.15.101:0/1604390185 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55821175fbe0 0x55821176fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.798+0000 7f3fd139b700 1 -- 172.21.15.101:0/1604390185 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fcc1a46d0 con 0x55821175f2d0 2023-07-18T03:07:55.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.798+0000 7f3fd139b700 1 --2- 172.21.15.101:0/1604390185 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55821175f2d0 0x55821176f6f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3fc809abe0 tx=0x7f3fc8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9f99432b8fe5fd2 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:55.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.798+0000 7f3fb4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:07:55.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.799+0000 7f3fb4ff9700 1 -- 172.21.15.101:0/1604390185 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc80b3050 con 0x55821175f2d0 2023-07-18T03:07:55.793 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.799+0000 7f3fb4ff9700 1 -- 172.21.15.101:0/1604390185 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f3fc8098d90 con 0x55821175f2d0 2023-07-18T03:07:55.793 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.799+0000 7f3fe6e7cb80 1 -- 172.21.15.101:0/1604390185 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3fbc004390 con 0x55821175f2d0 2023-07-18T03:07:55.793 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.799+0000 7f3fb4ff9700 1 -- 172.21.15.101:0/1604390185 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fc809ec80 con 0x55821175f2d0 2023-07-18T03:07:55.793 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.799+0000 7f3fe6e7cb80 10 client.15618 fetch_fsmap learned FSMap version 24 2023-07-18T03:07:55.793 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.799+0000 7f3fe6e7cb80 1 -- 172.21.15.101:0/1604390185 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x558211506250 con 0x55821175f2d0 2023-07-18T03:07:55.793 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.799+0000 7f3fb4ff9700 1 -- 172.21.15.101:0/1604390185 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fbc004390 con 0x55821175f2d0 2023-07-18T03:07:55.793 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.799+0000 7f3fb4ff9700 1 -- 172.21.15.101:0/1604390185 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3fc809aeb0 con 0x55821175f2d0 2023-07-18T03:07:55.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.799+0000 7f3fe6e7cb80 10 client.15618 fetch_fsmap finished waiting for FSMap version 24 2023-07-18T03:07:55.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.799+0000 7f3fe6e7cb80 10 client.15618 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:07:55.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.799+0000 7f3fe6e7cb80 20 client.15618 populate_metadata read hostname 'smithi101' 2023-07-18T03:07:55.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.799+0000 7f3fe6e7cb80 1 --2- 172.21.15.101:0/1604390185 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5582117ceae0 0x5582117d0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:55.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.799+0000 7f3fe6e7cb80 4 client.15618 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:07:55.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.799+0000 7f3fe6e7cb80 1 -- 172.21.15.101:0/1604390185 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55821166d050 con 0x5582117ceae0 2023-07-18T03:07:55.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.800+0000 7f3fd0b9a700 1 --2- 172.21.15.101:0/1604390185 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5582117ceae0 0x5582117d0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:55.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.800+0000 7f3fb4ff9700 10 client.15618 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:07:55.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.800+0000 7f3fd0b9a700 1 --2- 172.21.15.101:0/1604390185 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5582117ceae0 0x5582117d0ef0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=19b8f6161564c065 server_cookie=8b2b5adf6a19de in_seq=0 out_seq=0 2023-07-18T03:07:55.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.801+0000 7f3fb4ff9700 1 -- 172.21.15.101:0/1604390185 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55821166d050 con 0x5582117ceae0 2023-07-18T03:07:55.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.801+0000 7f3fb4ff9700 10 client.15618 handle_command_reply: tid=0 2023-07-18T03:07:55.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.801+0000 7f3fe6e7cb80 1 client.15618 shutdown 2023-07-18T03:07:55.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.801+0000 7f3fe6e7cb80 1 -- 172.21.15.101:0/1604390185 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55821175f2d0 msgr2=0x55821176f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:55.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.801+0000 7f3fe6e7cb80 1 --2- 172.21.15.101:0/1604390185 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55821175f2d0 0x55821176f6f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3fc809abe0 tx=0x7f3fc8000bc0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.801+0000 7f3fe6e7cb80 1 -- 172.21.15.101:0/1604390185 shutdown_connections 2023-07-18T03:07:55.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.801+0000 7f3fe6e7cb80 1 --2- 172.21.15.101:0/1604390185 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5582117ceae0 0x5582117d0ef0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.801+0000 7f3fe6e7cb80 1 --2- 172.21.15.101:0/1604390185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55821176aad0 0x5582117cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.801+0000 7f3fe6e7cb80 1 --2- 172.21.15.101:0/1604390185 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55821175f2d0 0x55821176f6f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.801+0000 7f3fe6e7cb80 1 --2- 172.21.15.101:0/1604390185 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55821175fbe0 0x55821176fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.801+0000 7f3fe6e7cb80 1 -- 172.21.15.101:0/1604390185 >> 172.21.15.101:0/1604390185 conn(0x558211755da0 msgr2=0x5582117767b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:55.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.801+0000 7f3fb4ff9700 0 client.15618 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:07:55.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.801+0000 7f3fe6e7cb80 1 -- 172.21.15.101:0/1604390185 shutdown_connections 2023-07-18T03:07:55.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.801+0000 7f3fe6e7cb80 1 -- 172.21.15.101:0/1604390185 wait complete. 2023-07-18T03:07:55.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.802+0000 7f3fe6e7cb80 20 client.15618 trim_cache size 0 max 16384 2023-07-18T03:07:55.798 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:07:55.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.802+0000 7f3fd35ff700 1 -- 172.21.15.101:0/321815979 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f3fb80a1350 msgr2=0x7f3fb80a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:55.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.802+0000 7f3fd35ff700 1 --2- 172.21.15.101:0/321815979 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f3fb80a1350 0x7f3fb80a3800 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3fc40060b0 tx=0x7f3fc4004b40 comp rx=0 tx=0).stop 2023-07-18T03:07:55.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.802+0000 7f3fd35ff700 1 -- 172.21.15.101:0/321815979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3fcc0ffb00 msgr2=0x7f3fcc072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:55.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.802+0000 7f3fd35ff700 1 --2- 172.21.15.101:0/321815979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3fcc0ffb00 0x7f3fcc072810 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8009810 tx=0x7f3fc800fa00 comp rx=0 tx=0).stop 2023-07-18T03:07:55.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.802+0000 7f3fd35ff700 1 -- 172.21.15.101:0/321815979 shutdown_connections 2023-07-18T03:07:55.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.802+0000 7f3fd35ff700 1 --2- 172.21.15.101:0/321815979 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f3fb80a1350 0x7f3fb80a3800 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.802+0000 7f3fd35ff700 1 --2- 172.21.15.101:0/321815979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3fcc0ffb00 0x7f3fcc072810 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.802+0000 7f3fd35ff700 1 --2- 172.21.15.101:0/321815979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3fcc105b10 0x7f3fcc1a3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.802+0000 7f3fd35ff700 1 --2- 172.21.15.101:0/321815979 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3fcc1004b0 0x7f3fcc072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:55.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.802+0000 7f3fd35ff700 1 -- 172.21.15.101:0/321815979 >> 172.21.15.101:0/321815979 conn(0x7f3fcc0fb600 msgr2=0x7f3fcc0fbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:55.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.802+0000 7f3fd35ff700 1 -- 172.21.15.101:0/321815979 shutdown_connections 2023-07-18T03:07:55.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:55.802+0000 7f3fd35ff700 1 -- 172.21.15.101:0/321815979 wait complete. 2023-07-18T03:07:55.812 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-18T03:07:56.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.032+0000 7fc2bd9ff700 1 -- 172.21.15.101:0/696740905 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc2b8105ae0 msgr2=0x7fc2b8105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:56.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.032+0000 7fc2bd9ff700 1 --2- 172.21.15.101:0/696740905 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc2b8105ae0 0x7fc2b8105eb0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8009990 tx=0x7fc2a8009ca0 comp rx=0 tx=0).stop 2023-07-18T03:07:56.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.032+0000 7fc2bd9ff700 1 -- 172.21.15.101:0/696740905 shutdown_connections 2023-07-18T03:07:56.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.032+0000 7fc2bd9ff700 1 --2- 172.21.15.101:0/696740905 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2b8100480 0x7fc2b810c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:56.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.032+0000 7fc2bd9ff700 1 --2- 172.21.15.101:0/696740905 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc2b8105ae0 0x7fc2b8105eb0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:56.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.032+0000 7fc2bd9ff700 1 --2- 172.21.15.101:0/696740905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2b80ffad0 0x7fc2b80fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:56.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.032+0000 7fc2bd9ff700 1 -- 172.21.15.101:0/696740905 >> 172.21.15.101:0/696740905 conn(0x7fc2b80fb5f0 msgr2=0x7fc2b80fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:56.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.032+0000 7fc2bd9ff700 1 -- 172.21.15.101:0/696740905 shutdown_connections 2023-07-18T03:07:56.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.032+0000 7fc2bd9ff700 1 -- 172.21.15.101:0/696740905 wait complete. 2023-07-18T03:07:56.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.033+0000 7fc2bd9ff700 1 Processor -- start 2023-07-18T03:07:56.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.033+0000 7fc2bd9ff700 1 -- start start 2023-07-18T03:07:56.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.034+0000 7fc2bd9ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc2b80ffad0 0x7fc2b8072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:56.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.034+0000 7fc2bd9ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2b8100480 0x7fc2b8072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:56.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.034+0000 7fc2bd9ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2b8105ae0 0x7fc2b8073290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:56.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.034+0000 7fc2bd9ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b81a8580 con 0x7fc2b80ffad0 2023-07-18T03:07:56.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.034+0000 7fc2bd9ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b81a86c0 con 0x7fc2b8100480 2023-07-18T03:07:56.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.034+0000 7fc2bd9ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b81a8800 con 0x7fc2b8105ae0 2023-07-18T03:07:56.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.034+0000 7fc2b6ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc2b80ffad0 0x7fc2b8072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:56.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.034+0000 7fc2b6ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc2b80ffad0 0x7fc2b8072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:46100/0 (socket says 172.21.15.101:46100) 2023-07-18T03:07:56.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.034+0000 7fc2b6ffd700 1 -- 172.21.15.101:0/1659756605 learned_addr learned my addr 172.21.15.101:0/1659756605 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:56.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.034+0000 7fc2b67fc700 1 --2- 172.21.15.101:0/1659756605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2b8100480 0x7fc2b8072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:56.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.034+0000 7fc2b77fe700 1 --2- 172.21.15.101:0/1659756605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2b8105ae0 0x7fc2b8073290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:56.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.034+0000 7fc2b6ffd700 1 -- 172.21.15.101:0/1659756605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2b8105ae0 msgr2=0x7fc2b8073290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:56.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.035+0000 7fc2b6ffd700 1 --2- 172.21.15.101:0/1659756605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2b8105ae0 0x7fc2b8073290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:56.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.035+0000 7fc2b6ffd700 1 -- 172.21.15.101:0/1659756605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2b8100480 msgr2=0x7fc2b8072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:56.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.035+0000 7fc2b6ffd700 1 --2- 172.21.15.101:0/1659756605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2b8100480 0x7fc2b8072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:56.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.035+0000 7fc2b6ffd700 1 -- 172.21.15.101:0/1659756605 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a80095f0 con 0x7fc2b80ffad0 2023-07-18T03:07:56.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.035+0000 7fc2b6ffd700 1 --2- 172.21.15.101:0/1659756605 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc2b80ffad0 0x7fc2b8072810 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8005910 tx=0x7fc2a8004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:56.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.035+0000 7fc2bc9fd700 1 -- 172.21.15.101:0/1659756605 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a801d070 con 0x7fc2b80ffad0 2023-07-18T03:07:56.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.035+0000 7fc2bd9ff700 1 -- 172.21.15.101:0/1659756605 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2b81a8a80 con 0x7fc2b80ffad0 2023-07-18T03:07:56.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.035+0000 7fc2bd9ff700 1 -- 172.21.15.101:0/1659756605 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2b81a8f00 con 0x7fc2b80ffad0 2023-07-18T03:07:56.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.035+0000 7fc2bc9fd700 1 -- 172.21.15.101:0/1659756605 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2a8022470 con 0x7fc2b80ffad0 2023-07-18T03:07:56.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.035+0000 7fc2bc9fd700 1 -- 172.21.15.101:0/1659756605 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a8017650 con 0x7fc2b80ffad0 2023-07-18T03:07:56.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.038+0000 7fc2bc9fd700 1 -- 172.21.15.101:0/1659756605 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 23) v1 ==== 135892+0+0 (secure 0 0 0) 0x7fc2a80225e0 con 0x7fc2b80ffad0 2023-07-18T03:07:56.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.040+0000 7fc2d129fb80 1 Processor -- start 2023-07-18T03:07:56.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.040+0000 7fc2d129fb80 1 -- start start 2023-07-18T03:07:56.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.041+0000 7fc2d129fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9702902d0 0x55c9702906a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:56.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.041+0000 7fc2d129fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c970290be0 0x55c97029b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:56.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.041+0000 7fc2d129fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c97029bad0 0x55c97029deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:56.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.041+0000 7fc2d129fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c970196cb0 con 0x55c970290be0 2023-07-18T03:07:56.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.041+0000 7fc2d129fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9701e9a40 con 0x55c97029bad0 2023-07-18T03:07:56.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.041+0000 7fc2d129fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9701e2ed0 con 0x55c9702902d0 2023-07-18T03:07:56.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.041+0000 7fc2bc9fd700 1 --2- 172.21.15.101:0/1659756605 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc2a00a13d0 0x7fc2a00a3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:56.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.041+0000 7fc2bc9fd700 1 -- 172.21.15.101:0/1659756605 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fc2a80ce440 con 0x7fc2b80ffad0 2023-07-18T03:07:56.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.041+0000 7fc2b77fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9702902d0 0x55c9702906a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:56.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.041+0000 7fc2b6ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c970290be0 0x55c97029b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:56.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.041+0000 7fc2b77fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9702902d0 0x55c9702906a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:37516/0 (socket says 172.21.15.101:37516) 2023-07-18T03:07:56.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.041+0000 7fc2b77fe700 1 -- 172.21.15.101:0/2528007756 learned_addr learned my addr 172.21.15.101:0/2528007756 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:56.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.041+0000 7fc2b67fc700 1 --2- 172.21.15.101:0/2528007756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c97029bad0 0x55c97029deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:56.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.042+0000 7fc2b77fe700 1 --2- 172.21.15.101:0/1659756605 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc2a00a13d0 0x7fc2a00a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:56.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.042+0000 7fc2b77fe700 1 -- 172.21.15.101:0/2528007756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c97029bad0 msgr2=0x55c97029deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:56.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.042+0000 7fc2b77fe700 1 --2- 172.21.15.101:0/2528007756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c97029bad0 0x55c97029deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:56.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.042+0000 7fc2b77fe700 1 -- 172.21.15.101:0/2528007756 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c970290be0 msgr2=0x55c97029b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:56.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.042+0000 7fc2b77fe700 1 --2- 172.21.15.101:0/2528007756 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c970290be0 0x55c97029b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:56.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.042+0000 7fc2b77fe700 1 -- 172.21.15.101:0/2528007756 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c970010de0 con 0x55c9702902d0 2023-07-18T03:07:56.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.042+0000 7fc2b77fe700 1 --2- 172.21.15.101:0/2528007756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9702902d0 0x55c9702906a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0019500 tx=0x7fc2b0019810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8913068cdbdc63e8 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:56.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.042+0000 7fc2b77fe700 1 --2- 172.21.15.101:0/1659756605 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc2a00a13d0 0x7fc2a00a3880 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0019b60 tx=0x7fc2b001d000 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:56.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.043+0000 7fc29dffb700 1 -- 172.21.15.101:0/2528007756 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b0025070 con 0x55c9702902d0 2023-07-18T03:07:56.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.043+0000 7fc29dffb700 1 -- 172.21.15.101:0/2528007756 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2b00123b0 con 0x55c9702902d0 2023-07-18T03:07:56.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.043+0000 7fc29dffb700 1 -- 172.21.15.101:0/2528007756 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b000e040 con 0x55c9702902d0 2023-07-18T03:07:56.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.043+0000 7fc2d129fb80 1 -- 172.21.15.101:0/2528007756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9702902d0 msgr2=0x55c9702906a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:56.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.043+0000 7fc2d129fb80 1 --2- 172.21.15.101:0/2528007756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9702902d0 0x55c9702906a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0019500 tx=0x7fc2b0019810 comp rx=0 tx=0).stop 2023-07-18T03:07:56.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.044+0000 7fc2d129fb80 1 -- 172.21.15.101:0/2528007756 shutdown_connections 2023-07-18T03:07:56.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.044+0000 7fc2d129fb80 1 --2- 172.21.15.101:0/2528007756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c97029bad0 0x55c97029deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:56.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.044+0000 7fc2d129fb80 1 --2- 172.21.15.101:0/2528007756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9702902d0 0x55c9702906a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:56.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.044+0000 7fc2d129fb80 1 --2- 172.21.15.101:0/2528007756 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c970290be0 0x55c97029b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:56.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.044+0000 7fc2d129fb80 1 -- 172.21.15.101:0/2528007756 >> 172.21.15.101:0/2528007756 conn(0x55c970286da0 msgr2=0x55c97028c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:56.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.044+0000 7fc2d129fb80 1 -- 172.21.15.101:0/2528007756 shutdown_connections 2023-07-18T03:07:56.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.044+0000 7fc2d129fb80 1 -- 172.21.15.101:0/2528007756 wait complete. 2023-07-18T03:07:56.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.044+0000 7fc2d129fb80 1 -- start start 2023-07-18T03:07:56.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.044+0000 7fc2d129fb80 1 Processor -- start 2023-07-18T03:07:56.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.045+0000 7fc2d129fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9702902d0 0x55c97029b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:56.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.045+0000 7fc2d129fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c970290be0 0x55c970298800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:56.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.045+0000 7fc2d129fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c97029bad0 0x55c970298f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:56.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.045+0000 7fc2d129fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c970182ac0 con 0x55c97029bad0 2023-07-18T03:07:56.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.045+0000 7fc2d129fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9701827c0 con 0x55c9702902d0 2023-07-18T03:07:56.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.045+0000 7fc2d129fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c970066920 con 0x55c970290be0 2023-07-18T03:07:56.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.045+0000 7fc2d129fb80 10 client.0 _ll_register_callbacks cb 0x55c970180760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:07:56.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.045+0000 7fc2b67fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c970290be0 0x55c970298800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:56.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.045+0000 7fc2b77fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c97029bad0 0x55c970298f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:56.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.045+0000 7fc2b67fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c970290be0 0x55c970298800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:37524/0 (socket says 172.21.15.101:37524) 2023-07-18T03:07:56.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.045+0000 7fc2b67fc700 1 -- 172.21.15.101:0/3066326882 learned_addr learned my addr 172.21.15.101:0/3066326882 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:56.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.045+0000 7fc2b6ffd700 1 --2- 172.21.15.101:0/3066326882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9702902d0 0x55c97029b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:56.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.046+0000 7fc2b67fc700 1 -- 172.21.15.101:0/3066326882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9702902d0 msgr2=0x55c97029b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:56.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.046+0000 7fc2b67fc700 1 --2- 172.21.15.101:0/3066326882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9702902d0 0x55c97029b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:56.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.046+0000 7fc2b67fc700 1 -- 172.21.15.101:0/3066326882 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c97029bad0 msgr2=0x55c970298f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:56.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.046+0000 7fc2b67fc700 1 --2- 172.21.15.101:0/3066326882 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c97029bad0 0x55c970298f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:56.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.046+0000 7fc2b67fc700 1 -- 172.21.15.101:0/3066326882 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2b0012cc0 con 0x55c970290be0 2023-07-18T03:07:56.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.046+0000 7fc2b67fc700 1 --2- 172.21.15.101:0/3066326882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c970290be0 0x55c970298800 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac011780 tx=0x7fc2ac011a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74faa8e094e172a1 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:56.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.046+0000 7fc296ffd700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:07:56.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.046+0000 7fc296ffd700 1 -- 172.21.15.101:0/3066326882 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac016070 con 0x55c970290be0 2023-07-18T03:07:56.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.046+0000 7fc2d129fb80 1 -- 172.21.15.101:0/3066326882 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c97013c4f0 con 0x55c970290be0 2023-07-18T03:07:56.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.046+0000 7fc296ffd700 1 -- 172.21.15.101:0/3066326882 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fc2ac004830 con 0x55c970290be0 2023-07-18T03:07:56.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.046+0000 7fc296ffd700 1 -- 172.21.15.101:0/3066326882 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2ac004a30 con 0x55c970290be0 2023-07-18T03:07:56.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.046+0000 7fc2d129fb80 10 client.25693 fetch_fsmap learned FSMap version 24 2023-07-18T03:07:56.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.047+0000 7fc2d129fb80 1 -- 172.21.15.101:0/3066326882 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c96fff9c60 con 0x55c970290be0 2023-07-18T03:07:56.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.047+0000 7fc296ffd700 1 -- 172.21.15.101:0/3066326882 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c97013c4f0 con 0x55c970290be0 2023-07-18T03:07:56.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.047+0000 7fc296ffd700 1 -- 172.21.15.101:0/3066326882 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc2ac01a910 con 0x55c970290be0 2023-07-18T03:07:56.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.047+0000 7fc2d129fb80 10 client.25693 fetch_fsmap finished waiting for FSMap version 24 2023-07-18T03:07:56.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.047+0000 7fc2d129fb80 10 client.25693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:07:56.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.047+0000 7fc2d129fb80 20 client.25693 populate_metadata read hostname 'smithi101' 2023-07-18T03:07:56.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.047+0000 7fc2d129fb80 1 --2- 172.21.15.101:0/3066326882 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c9702ab5f0 0x55c9702ada00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:56.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.047+0000 7fc2d129fb80 4 client.25693 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:07:56.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.047+0000 7fc2d129fb80 1 -- 172.21.15.101:0/3066326882 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c9701dbeb0 con 0x55c9702ab5f0 2023-07-18T03:07:56.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.047+0000 7fc2b6ffd700 1 --2- 172.21.15.101:0/3066326882 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c9702ab5f0 0x55c9702ada00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:56.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.048+0000 7fc296ffd700 10 client.25693 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:07:56.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.048+0000 7fc2b6ffd700 1 --2- 172.21.15.101:0/3066326882 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c9702ab5f0 0x55c9702ada00 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48a33366827ba03c server_cookie=3b748562cff0f7e7 in_seq=0 out_seq=0 2023-07-18T03:07:56.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.049+0000 7fc296ffd700 1 -- 172.21.15.101:0/3066326882 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c9701dbeb0 con 0x55c9702ab5f0 2023-07-18T03:07:56.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.049+0000 7fc296ffd700 10 client.25693 handle_command_reply: tid=0 2023-07-18T03:07:56.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.049+0000 7fc2d129fb80 1 client.25693 shutdown 2023-07-18T03:07:56.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.049+0000 7fc2d129fb80 1 -- 172.21.15.101:0/3066326882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c970290be0 msgr2=0x55c970298800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:56.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.049+0000 7fc2d129fb80 1 --2- 172.21.15.101:0/3066326882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c970290be0 0x55c970298800 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac011780 tx=0x7fc2ac011a90 comp rx=0 tx=0).stop 2023-07-18T03:07:56.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.049+0000 7fc2d129fb80 1 -- 172.21.15.101:0/3066326882 shutdown_connections 2023-07-18T03:07:56.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.049+0000 7fc2d129fb80 1 --2- 172.21.15.101:0/3066326882 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c9702ab5f0 0x55c9702ada00 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:56.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.049+0000 7fc2d129fb80 1 --2- 172.21.15.101:0/3066326882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9702902d0 0x55c97029b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:56.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.049+0000 7fc2d129fb80 1 --2- 172.21.15.101:0/3066326882 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c97029bad0 0x55c970298f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:56.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.049+0000 7fc2d129fb80 1 --2- 172.21.15.101:0/3066326882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c970290be0 0x55c970298800 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:56.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.049+0000 7fc296ffd700 0 client.25693 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:07:56.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.049+0000 7fc2d129fb80 1 -- 172.21.15.101:0/3066326882 >> 172.21.15.101:0/3066326882 conn(0x55c970286da0 msgr2=0x55c97028c230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:56.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.049+0000 7fc2d129fb80 1 -- 172.21.15.101:0/3066326882 shutdown_connections 2023-07-18T03:07:56.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.049+0000 7fc2d129fb80 1 -- 172.21.15.101:0/3066326882 wait complete. 2023-07-18T03:07:56.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.049+0000 7fc2d129fb80 20 client.25693 trim_cache size 0 max 16384 2023-07-18T03:07:56.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.066+0000 7fc2d129fb80 1 Processor -- start 2023-07-18T03:07:56.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.066+0000 7fc2d129fb80 1 -- start start 2023-07-18T03:07:56.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.066+0000 7fc2d129fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9702902d0 0x55c970293b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:56.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.066+0000 7fc2d129fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c970290be0 0x55c970294070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:56.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.066+0000 7fc2d129fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c97029bad0 0x55c970297fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:56.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.066+0000 7fc2d129fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c97028d2e0 con 0x55c970290be0 2023-07-18T03:07:56.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.066+0000 7fc2d129fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c970291540 con 0x55c97029bad0 2023-07-18T03:07:56.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.066+0000 7fc2d129fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c970283fa0 con 0x55c9702902d0 2023-07-18T03:07:56.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.066+0000 7fc2b67fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c970290be0 0x55c970294070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:56.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.066+0000 7fc2b6ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9702902d0 0x55c970293b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:56.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.066+0000 7fc2b67fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c970290be0 0x55c970294070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:46120/0 (socket says 172.21.15.101:46120) 2023-07-18T03:07:56.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.066+0000 7fc2b67fc700 1 -- 172.21.15.101:0/204590120 learned_addr learned my addr 172.21.15.101:0/204590120 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:56.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.067+0000 7fc2b77fe700 1 --2- 172.21.15.101:0/204590120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c97029bad0 0x55c970297fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:56.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.067+0000 7fc2b67fc700 1 -- 172.21.15.101:0/204590120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9702902d0 msgr2=0x55c970293b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:56.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.067+0000 7fc2b67fc700 1 --2- 172.21.15.101:0/204590120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9702902d0 0x55c970293b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:56.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.067+0000 7fc2b67fc700 1 -- 172.21.15.101:0/204590120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c97029bad0 msgr2=0x55c970297fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:56.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.067+0000 7fc2b67fc700 1 --2- 172.21.15.101:0/204590120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c97029bad0 0x55c970297fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:56.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.067+0000 7fc2b67fc700 1 -- 172.21.15.101:0/204590120 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c970037250 con 0x55c970290be0 2023-07-18T03:07:56.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.067+0000 7fc2b6ffd700 1 --2- 172.21.15.101:0/204590120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9702902d0 0x55c970293b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:07:56.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.067+0000 7fc2b77fe700 1 --2- 172.21.15.101:0/204590120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c97029bad0 0x55c970297fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:07:56.061 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.067+0000 7fc2b67fc700 1 --2- 172.21.15.101:0/204590120 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c970290be0 0x55c970294070 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac01ad70 tx=0x7fc2ac009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=851c474272a5f4ce server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:56.061 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.067+0000 7fc29e7fc700 1 -- 172.21.15.101:0/204590120 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac02e070 con 0x55c970290be0 2023-07-18T03:07:56.061 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.067+0000 7fc29e7fc700 1 -- 172.21.15.101:0/204590120 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2ac019390 con 0x55c970290be0 2023-07-18T03:07:56.061 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.067+0000 7fc29e7fc700 1 -- 172.21.15.101:0/204590120 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac016070 con 0x55c970290be0 2023-07-18T03:07:56.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.068+0000 7fc2d129fb80 1 -- 172.21.15.101:0/204590120 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c970290be0 msgr2=0x55c970294070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:56.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.068+0000 7fc2d129fb80 1 --2- 172.21.15.101:0/204590120 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c970290be0 0x55c970294070 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac01ad70 tx=0x7fc2ac009750 comp rx=0 tx=0).stop 2023-07-18T03:07:56.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.068+0000 7fc2d129fb80 1 -- 172.21.15.101:0/204590120 shutdown_connections 2023-07-18T03:07:56.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.068+0000 7fc2d129fb80 1 --2- 172.21.15.101:0/204590120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c97029bad0 0x55c970297fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:56.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.068+0000 7fc2d129fb80 1 --2- 172.21.15.101:0/204590120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9702902d0 0x55c970293b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:56.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.068+0000 7fc2d129fb80 1 --2- 172.21.15.101:0/204590120 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c970290be0 0x55c970294070 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:56.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.068+0000 7fc2d129fb80 1 -- 172.21.15.101:0/204590120 >> 172.21.15.101:0/204590120 conn(0x55c970286da0 msgr2=0x55c9702a77b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:56.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.068+0000 7fc2d129fb80 1 -- 172.21.15.101:0/204590120 shutdown_connections 2023-07-18T03:07:56.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.068+0000 7fc2d129fb80 1 -- 172.21.15.101:0/204590120 wait complete. 2023-07-18T03:07:56.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.069+0000 7fc2d129fb80 1 -- start start 2023-07-18T03:07:56.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.069+0000 7fc2d129fb80 1 Processor -- start 2023-07-18T03:07:56.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.069+0000 7fc2d129fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9702902d0 0x55c9702a2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:56.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.069+0000 7fc2d129fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c970290be0 0x55c9702a0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:56.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.069+0000 7fc2d129fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c97029bad0 0x55c9702a0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:56.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.069+0000 7fc2d129fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9702a1570 con 0x55c97029bad0 2023-07-18T03:07:56.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.069+0000 7fc2d129fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9702a16e0 con 0x55c9702902d0 2023-07-18T03:07:56.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.069+0000 7fc2d129fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9702ff480 con 0x55c970290be0 2023-07-18T03:07:56.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.069+0000 7fc2d129fb80 10 client.0 _ll_register_callbacks cb 0x7fc2ac009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:07:56.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.070+0000 7fc2b6ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9702902d0 0x55c9702a2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:56.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.070+0000 7fc2b77fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c97029bad0 0x55c9702a0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:56.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.070+0000 7fc2b67fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c970290be0 0x55c9702a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:56.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.070+0000 7fc2b6ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9702902d0 0x55c9702a2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:49274/0 (socket says 172.21.15.101:49274) 2023-07-18T03:07:56.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.070+0000 7fc2b6ffd700 1 -- 172.21.15.101:0/1685592544 learned_addr learned my addr 172.21.15.101:0/1685592544 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:56.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.070+0000 7fc2b6ffd700 1 -- 172.21.15.101:0/1685592544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c970290be0 msgr2=0x55c9702a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:56.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.070+0000 7fc2b6ffd700 1 --2- 172.21.15.101:0/1685592544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c970290be0 0x55c9702a0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:56.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.070+0000 7fc2b6ffd700 1 -- 172.21.15.101:0/1685592544 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c97029bad0 msgr2=0x55c9702a0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:56.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.070+0000 7fc2b6ffd700 1 --2- 172.21.15.101:0/1685592544 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c97029bad0 0x55c9702a0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:56.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.070+0000 7fc2b6ffd700 1 -- 172.21.15.101:0/1685592544 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2b81a8f00 con 0x55c9702902d0 2023-07-18T03:07:56.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.070+0000 7fc2b6ffd700 1 --2- 172.21.15.101:0/1685592544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9702902d0 0x55c9702a2530 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc2a809af00 tx=0x7fc2a801a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51afcb0711b69f69 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:56.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.070+0000 7fc296ffd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:07:56.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.070+0000 7fc296ffd700 1 -- 172.21.15.101:0/1685592544 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a809b080 con 0x55c9702902d0 2023-07-18T03:07:56.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.071+0000 7fc2d129fb80 1 -- 172.21.15.101:0/1685592544 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc2b00123b0 con 0x55c9702902d0 2023-07-18T03:07:56.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.071+0000 7fc296ffd700 1 -- 172.21.15.101:0/1685592544 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fc2a809f0b0 con 0x55c9702902d0 2023-07-18T03:07:56.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.071+0000 7fc296ffd700 1 -- 172.21.15.101:0/1685592544 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2a80a3440 con 0x55c9702902d0 2023-07-18T03:07:56.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.071+0000 7fc2d129fb80 10 client.25664 fetch_fsmap learned FSMap version 24 2023-07-18T03:07:56.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.071+0000 7fc2d129fb80 1 -- 172.21.15.101:0/1685592544 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c96fff9c60 con 0x55c9702902d0 2023-07-18T03:07:56.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.071+0000 7fc296ffd700 1 -- 172.21.15.101:0/1685592544 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2a80cb080 con 0x55c9702902d0 2023-07-18T03:07:56.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.071+0000 7fc296ffd700 1 -- 172.21.15.101:0/1685592544 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc2a80cb1f0 con 0x55c9702902d0 2023-07-18T03:07:56.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.071+0000 7fc2d129fb80 10 client.25664 fetch_fsmap finished waiting for FSMap version 24 2023-07-18T03:07:56.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.071+0000 7fc2d129fb80 10 client.25664 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:07:56.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.071+0000 7fc2d129fb80 20 client.25664 populate_metadata read hostname 'smithi101' 2023-07-18T03:07:56.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.071+0000 7fc2d129fb80 1 --2- 172.21.15.101:0/1685592544 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c9703006c0 0x55c970302aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:56.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.071+0000 7fc2d129fb80 4 client.25664 mds_command: new command op to 24562 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:07:56.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.071+0000 7fc2d129fb80 1 -- 172.21.15.101:0/1685592544 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c97019e050 con 0x55c9703006c0 2023-07-18T03:07:56.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.072+0000 7fc2b67fc700 1 --2- 172.21.15.101:0/1685592544 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c9703006c0 0x55c970302aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:56.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.072+0000 7fc2b67fc700 1 --2- 172.21.15.101:0/1685592544 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c9703006c0 0x55c970302aa0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=708b5c4fd8875ad0 server_cookie=f6ccf6a7b97b13db in_seq=0 out_seq=0 2023-07-18T03:07:56.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.072+0000 7fc296ffd700 10 client.25664 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:07:56.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.072+0000 7fc296ffd700 1 -- 172.21.15.101:0/1685592544 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c97019e050 con 0x55c9703006c0 2023-07-18T03:07:56.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.072+0000 7fc296ffd700 10 client.25664 handle_command_reply: tid=0 2023-07-18T03:07:56.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.072+0000 7fc2d129fb80 1 client.25664 shutdown 2023-07-18T03:07:56.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.073+0000 7fc2d129fb80 1 -- 172.21.15.101:0/1685592544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9702902d0 msgr2=0x55c9702a2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:56.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.073+0000 7fc2d129fb80 1 --2- 172.21.15.101:0/1685592544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9702902d0 0x55c9702a2530 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc2a809af00 tx=0x7fc2a801a040 comp rx=0 tx=0).stop 2023-07-18T03:07:56.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.073+0000 7fc2d129fb80 1 -- 172.21.15.101:0/1685592544 shutdown_connections 2023-07-18T03:07:56.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.073+0000 7fc2d129fb80 1 --2- 172.21.15.101:0/1685592544 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c9703006c0 0x55c970302aa0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:56.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.073+0000 7fc2d129fb80 1 --2- 172.21.15.101:0/1685592544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c9702902d0 0x55c9702a2530 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:56.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.073+0000 7fc2d129fb80 1 --2- 172.21.15.101:0/1685592544 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c97029bad0 0x55c9702a0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:56.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.073+0000 7fc296ffd700 0 client.25664 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:07:56.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.073+0000 7fc2d129fb80 1 --2- 172.21.15.101:0/1685592544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c970290be0 0x55c9702a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:56.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.073+0000 7fc2d129fb80 1 -- 172.21.15.101:0/1685592544 >> 172.21.15.101:0/1685592544 conn(0x55c970286da0 msgr2=0x55c970293b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:56.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.073+0000 7fc2d129fb80 1 -- 172.21.15.101:0/1685592544 shutdown_connections 2023-07-18T03:07:56.070 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.073+0000 7fc2d129fb80 1 -- 172.21.15.101:0/1685592544 wait complete. 2023-07-18T03:07:56.070 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.073+0000 7fc2d129fb80 20 client.25664 trim_cache size 0 max 16384 2023-07-18T03:07:56.070 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:07:56.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.073+0000 7fc2bd9ff700 1 -- 172.21.15.101:0/1659756605 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc2a00a13d0 msgr2=0x7fc2a00a3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:56.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.073+0000 7fc2bd9ff700 1 --2- 172.21.15.101:0/1659756605 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc2a00a13d0 0x7fc2a00a3880 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0019b60 tx=0x7fc2b001d000 comp rx=0 tx=0).stop 2023-07-18T03:07:56.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.073+0000 7fc2bd9ff700 1 -- 172.21.15.101:0/1659756605 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc2b80ffad0 msgr2=0x7fc2b8072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:56.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.073+0000 7fc2bd9ff700 1 --2- 172.21.15.101:0/1659756605 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc2b80ffad0 0x7fc2b8072810 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8005910 tx=0x7fc2a8004710 comp rx=0 tx=0).stop 2023-07-18T03:07:56.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.074+0000 7fc2bd9ff700 1 -- 172.21.15.101:0/1659756605 shutdown_connections 2023-07-18T03:07:56.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.074+0000 7fc2bd9ff700 1 --2- 172.21.15.101:0/1659756605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2b8105ae0 0x7fc2b8073290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:56.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.074+0000 7fc2bd9ff700 1 --2- 172.21.15.101:0/1659756605 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc2b80ffad0 0x7fc2b8072810 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:56.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.074+0000 7fc2bd9ff700 1 --2- 172.21.15.101:0/1659756605 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc2a00a13d0 0x7fc2a00a3880 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:56.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.074+0000 7fc2bd9ff700 1 --2- 172.21.15.101:0/1659756605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2b8100480 0x7fc2b8072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:56.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.074+0000 7fc2bd9ff700 1 -- 172.21.15.101:0/1659756605 >> 172.21.15.101:0/1659756605 conn(0x7fc2b80fb5f0 msgr2=0x7fc2b80fbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:56.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.074+0000 7fc2bd9ff700 1 -- 172.21.15.101:0/1659756605 shutdown_connections 2023-07-18T03:07:56.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:56.074+0000 7fc2bd9ff700 1 -- 172.21.15.101:0/1659756605 wait complete. 2023-07-18T03:07:56.078 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:55 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1616402788' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:07:56.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:55 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1616402788' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:07:56.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:55 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1616402788' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:07:57.081 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-18T03:07:57.081 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-18T03:07:57.142 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:56 smithi101 ceph-mon[126379]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2023-07-18T03:07:57.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:56 smithi179 ceph-mon[129229]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2023-07-18T03:07:57.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:56 smithi155 ceph-mon[128885]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2023-07-18T03:07:57.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.301+0000 7facd13ff700 1 -- 172.21.15.101:0/2887468763 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faccc0731c0 msgr2=0x7faccc10bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:57.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.301+0000 7facd13ff700 1 --2- 172.21.15.101:0/2887468763 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faccc0731c0 0x7faccc10bdd0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7facc0009990 tx=0x7facc0009ca0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.301+0000 7facd13ff700 1 -- 172.21.15.101:0/2887468763 shutdown_connections 2023-07-18T03:07:57.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.301+0000 7facd13ff700 1 --2- 172.21.15.101:0/2887468763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faccc10c310 0x7faccc10e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.301+0000 7facd13ff700 1 --2- 172.21.15.101:0/2887468763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faccc0728b0 0x7faccc072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.301+0000 7facd13ff700 1 --2- 172.21.15.101:0/2887468763 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faccc0731c0 0x7faccc10bdd0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.302+0000 7facd13ff700 1 -- 172.21.15.101:0/2887468763 >> 172.21.15.101:0/2887468763 conn(0x7faccc0fb5f0 msgr2=0x7faccc0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:57.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.302+0000 7facd13ff700 1 -- 172.21.15.101:0/2887468763 shutdown_connections 2023-07-18T03:07:57.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.302+0000 7facd13ff700 1 -- 172.21.15.101:0/2887468763 wait complete. 2023-07-18T03:07:57.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.303+0000 7facd13ff700 1 Processor -- start 2023-07-18T03:07:57.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.303+0000 7facd13ff700 1 -- start start 2023-07-18T03:07:57.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.303+0000 7facd13ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faccc0728b0 0x7faccc197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:57.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.303+0000 7facd13ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faccc0731c0 0x7faccc1982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:57.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.303+0000 7facd13ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faccc10c310 0x7faccc19c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:57.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.303+0000 7facd13ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faccc19cd80 con 0x7faccc10c310 2023-07-18T03:07:57.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.303+0000 7facd13ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faccc19cef0 con 0x7faccc0731c0 2023-07-18T03:07:57.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.303+0000 7facd13ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faccc19d0b0 con 0x7faccc0728b0 2023-07-18T03:07:57.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.303+0000 7faccb7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faccc10c310 0x7faccc19c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:57.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.303+0000 7faccaffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faccc0728b0 0x7faccc197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:57.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.303+0000 7faccb7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faccc10c310 0x7faccc19c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:46140/0 (socket says 172.21.15.101:46140) 2023-07-18T03:07:57.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.304+0000 7faccb7fe700 1 -- 172.21.15.101:0/942006199 learned_addr learned my addr 172.21.15.101:0/942006199 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:57.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.304+0000 7facca7fc700 1 --2- 172.21.15.101:0/942006199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faccc0731c0 0x7faccc1982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:57.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.304+0000 7faccb7fe700 1 -- 172.21.15.101:0/942006199 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faccc0728b0 msgr2=0x7faccc197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:57.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.304+0000 7faccb7fe700 1 --2- 172.21.15.101:0/942006199 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faccc0728b0 0x7faccc197da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.304+0000 7faccb7fe700 1 -- 172.21.15.101:0/942006199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faccc0731c0 msgr2=0x7faccc1982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:57.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.304+0000 7faccb7fe700 1 --2- 172.21.15.101:0/942006199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faccc0731c0 0x7faccc1982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.304+0000 7faccb7fe700 1 -- 172.21.15.101:0/942006199 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc00095f0 con 0x7faccc10c310 2023-07-18T03:07:57.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.304+0000 7facca7fc700 1 --2- 172.21.15.101:0/942006199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faccc0731c0 0x7faccc1982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:07:57.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.304+0000 7faccb7fe700 1 --2- 172.21.15.101:0/942006199 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faccc10c310 0x7faccc19c740 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7facc400ef30 tx=0x7facc400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:57.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.304+0000 7facb3fff700 1 -- 172.21.15.101:0/942006199 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc4018070 con 0x7faccc10c310 2023-07-18T03:07:57.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.304+0000 7facb3fff700 1 -- 172.21.15.101:0/942006199 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facc400f040 con 0x7faccc10c310 2023-07-18T03:07:57.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.304+0000 7facd13ff700 1 -- 172.21.15.101:0/942006199 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faccc19d330 con 0x7faccc10c310 2023-07-18T03:07:57.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.304+0000 7facb3fff700 1 -- 172.21.15.101:0/942006199 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc400cd00 con 0x7faccc10c310 2023-07-18T03:07:57.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.305+0000 7facd13ff700 1 -- 172.21.15.101:0/942006199 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faccc1a4bd0 con 0x7faccc10c310 2023-07-18T03:07:57.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.308+0000 7facb3fff700 1 -- 172.21.15.101:0/942006199 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 23) v1 ==== 135892+0+0 (secure 0 0 0) 0x7facc4003a00 con 0x7faccc10c310 2023-07-18T03:07:57.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.309+0000 7facb3fff700 1 --2- 172.21.15.101:0/942006199 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7facb40a1350 0x7facb40a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:57.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.309+0000 7facb3fff700 1 -- 172.21.15.101:0/942006199 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7facc40ccce0 con 0x7faccc10c310 2023-07-18T03:07:57.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.310+0000 7faccaffd700 1 --2- 172.21.15.101:0/942006199 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7facb40a1350 0x7facb40a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:57.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.310+0000 7face4d4ab80 1 Processor -- start 2023-07-18T03:07:57.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.310+0000 7face4d4ab80 1 -- start start 2023-07-18T03:07:57.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.310+0000 7face4d4ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635e7523350 0x5635e7523720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:57.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.310+0000 7face4d4ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635e7523c60 0x5635e752e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:57.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.310+0000 7face4d4ab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635e752eb50 0x5635e7530f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:57.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.310+0000 7face4d4ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e7429d30 con 0x5635e7523c60 2023-07-18T03:07:57.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.310+0000 7face4d4ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e747cac0 con 0x5635e7523350 2023-07-18T03:07:57.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.310+0000 7face4d4ab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e7475f50 con 0x5635e752eb50 2023-07-18T03:07:57.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.311+0000 7faccaffd700 1 --2- 172.21.15.101:0/942006199 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7facb40a1350 0x7facb40a3800 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7facbc009c60 tx=0x7facbc009260 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:57.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.311+0000 7facca7fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635e7523c60 0x5635e752e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:57.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.311+0000 7facca7fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635e7523c60 0x5635e752e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:46154/0 (socket says 172.21.15.101:46154) 2023-07-18T03:07:57.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.311+0000 7facca7fc700 1 -- 172.21.15.101:0/3710200422 learned_addr learned my addr 172.21.15.101:0/3710200422 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:57.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.311+0000 7faccb7fe700 1 --2- 172.21.15.101:0/3710200422 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635e752eb50 0x5635e7530f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:57.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.311+0000 7faccaffd700 1 --2- 172.21.15.101:0/3710200422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635e7523350 0x5635e7523720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:57.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.311+0000 7facca7fc700 1 -- 172.21.15.101:0/3710200422 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635e752eb50 msgr2=0x5635e7530f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:57.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.311+0000 7facca7fc700 1 --2- 172.21.15.101:0/3710200422 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635e752eb50 0x5635e7530f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.311+0000 7facca7fc700 1 -- 172.21.15.101:0/3710200422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635e7523350 msgr2=0x5635e7523720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:57.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.311+0000 7facca7fc700 1 --2- 172.21.15.101:0/3710200422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635e7523350 0x5635e7523720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.311+0000 7facca7fc700 1 -- 172.21.15.101:0/3710200422 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635e72a3e60 con 0x5635e7523c60 2023-07-18T03:07:57.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.311+0000 7facca7fc700 1 --2- 172.21.15.101:0/3710200422 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635e7523c60 0x5635e752e610 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7facc0015000 tx=0x7facc0005ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b59359c4e3954ea server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:57.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.311+0000 7facb17fa700 1 -- 172.21.15.101:0/3710200422 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc001d050 con 0x5635e7523c60 2023-07-18T03:07:57.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.312+0000 7facb17fa700 1 -- 172.21.15.101:0/3710200422 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facc001e040 con 0x5635e7523c60 2023-07-18T03:07:57.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.312+0000 7facb17fa700 1 -- 172.21.15.101:0/3710200422 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc0005470 con 0x5635e7523c60 2023-07-18T03:07:57.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.312+0000 7face4d4ab80 1 -- 172.21.15.101:0/3710200422 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635e7523c60 msgr2=0x5635e752e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:57.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.312+0000 7face4d4ab80 1 --2- 172.21.15.101:0/3710200422 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635e7523c60 0x5635e752e610 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7facc0015000 tx=0x7facc0005ea0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.312+0000 7face4d4ab80 1 -- 172.21.15.101:0/3710200422 shutdown_connections 2023-07-18T03:07:57.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.312+0000 7face4d4ab80 1 --2- 172.21.15.101:0/3710200422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635e7523350 0x5635e7523720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.312+0000 7face4d4ab80 1 --2- 172.21.15.101:0/3710200422 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635e752eb50 0x5635e7530f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.312+0000 7face4d4ab80 1 --2- 172.21.15.101:0/3710200422 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635e7523c60 0x5635e752e610 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.312+0000 7face4d4ab80 1 -- 172.21.15.101:0/3710200422 >> 172.21.15.101:0/3710200422 conn(0x5635e7519e20 msgr2=0x5635e751f720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:57.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.313+0000 7face4d4ab80 1 -- 172.21.15.101:0/3710200422 shutdown_connections 2023-07-18T03:07:57.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.313+0000 7face4d4ab80 1 -- 172.21.15.101:0/3710200422 wait complete. 2023-07-18T03:07:57.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.313+0000 7face4d4ab80 1 -- start start 2023-07-18T03:07:57.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.313+0000 7face4d4ab80 1 Processor -- start 2023-07-18T03:07:57.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.314+0000 7face4d4ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635e7523350 0x5635e752e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:57.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.314+0000 7face4d4ab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635e7523c60 0x5635e752b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:57.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.314+0000 7face4d4ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635e752eb50 0x5635e752be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:57.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.314+0000 7face4d4ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e7415b40 con 0x5635e7523350 2023-07-18T03:07:57.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.314+0000 7face4d4ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e7415840 con 0x5635e752eb50 2023-07-18T03:07:57.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.314+0000 7face4d4ab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e72f99a0 con 0x5635e7523c60 2023-07-18T03:07:57.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.314+0000 7face4d4ab80 10 client.0 _ll_register_callbacks cb 0x5635e74137e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:07:57.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.314+0000 7faccaffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635e7523350 0x5635e752e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:57.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.314+0000 7faccaffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635e7523350 0x5635e752e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:46168/0 (socket says 172.21.15.101:46168) 2023-07-18T03:07:57.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.314+0000 7faccaffd700 1 -- 172.21.15.101:0/2113598492 learned_addr learned my addr 172.21.15.101:0/2113598492 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:57.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.314+0000 7facca7fc700 1 --2- 172.21.15.101:0/2113598492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635e7523c60 0x5635e752b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:57.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.314+0000 7faccb7fe700 1 --2- 172.21.15.101:0/2113598492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635e752eb50 0x5635e752be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:57.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.315+0000 7faccaffd700 1 -- 172.21.15.101:0/2113598492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635e7523c60 msgr2=0x5635e752b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:57.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.315+0000 7faccaffd700 1 --2- 172.21.15.101:0/2113598492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635e7523c60 0x5635e752b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.315+0000 7faccaffd700 1 -- 172.21.15.101:0/2113598492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635e752eb50 msgr2=0x5635e752be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:57.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.315+0000 7faccaffd700 1 --2- 172.21.15.101:0/2113598492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635e752eb50 0x5635e752be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.315+0000 7faccaffd700 1 -- 172.21.15.101:0/2113598492 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faccc10f390 con 0x5635e7523350 2023-07-18T03:07:57.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.315+0000 7faccaffd700 1 --2- 172.21.15.101:0/2113598492 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635e7523350 0x5635e752e500 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7facbc003900 tx=0x7facbc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fff56630103e3435 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:57.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.315+0000 7facaeff5700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:07:57.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.315+0000 7facaeff5700 1 -- 172.21.15.101:0/2113598492 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc022070 con 0x5635e7523350 2023-07-18T03:07:57.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.315+0000 7facaeff5700 1 -- 172.21.15.101:0/2113598492 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7facbc01d620 con 0x5635e7523350 2023-07-18T03:07:57.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.315+0000 7face4d4ab80 1 -- 172.21.15.101:0/2113598492 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5635e73cf570 con 0x5635e7523350 2023-07-18T03:07:57.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.316+0000 7facaeff5700 1 -- 172.21.15.101:0/2113598492 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7facbc004830 con 0x5635e7523350 2023-07-18T03:07:57.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.316+0000 7face4d4ab80 10 client.15648 fetch_fsmap learned FSMap version 24 2023-07-18T03:07:57.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.316+0000 7face4d4ab80 1 -- 172.21.15.101:0/2113598492 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5635e728cce0 con 0x5635e7523350 2023-07-18T03:07:57.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.316+0000 7facaeff5700 1 -- 172.21.15.101:0/2113598492 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facbc030020 con 0x5635e7523350 2023-07-18T03:07:57.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.316+0000 7facaeff5700 1 -- 172.21.15.101:0/2113598492 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7facbc005020 con 0x5635e7523350 2023-07-18T03:07:57.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.316+0000 7face4d4ab80 10 client.15648 fetch_fsmap finished waiting for FSMap version 24 2023-07-18T03:07:57.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.316+0000 7face4d4ab80 10 client.15648 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:07:57.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.316+0000 7face4d4ab80 20 client.15648 populate_metadata read hostname 'smithi101' 2023-07-18T03:07:57.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.316+0000 7face4d4ab80 1 --2- 172.21.15.101:0/2113598492 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5635e753e4e0 0x5635e75408f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:57.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.316+0000 7face4d4ab80 4 client.15648 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:07:57.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.316+0000 7face4d4ab80 1 -- 172.21.15.101:0/2113598492 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5635e746ef30 con 0x5635e753e4e0 2023-07-18T03:07:57.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.317+0000 7facca7fc700 1 --2- 172.21.15.101:0/2113598492 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5635e753e4e0 0x5635e75408f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:57.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.317+0000 7facaeff5700 10 client.15648 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:07:57.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.317+0000 7facca7fc700 1 --2- 172.21.15.101:0/2113598492 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5635e753e4e0 0x5635e75408f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e09b360a59aeab1 server_cookie=291d89f10020e023 in_seq=0 out_seq=0 2023-07-18T03:07:57.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.318+0000 7facaeff5700 1 -- 172.21.15.101:0/2113598492 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5635e746ef30 con 0x5635e753e4e0 2023-07-18T03:07:57.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.318+0000 7facaeff5700 10 client.15648 handle_command_reply: tid=0 2023-07-18T03:07:57.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.318+0000 7face4d4ab80 1 client.15648 shutdown 2023-07-18T03:07:57.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.318+0000 7face4d4ab80 1 -- 172.21.15.101:0/2113598492 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635e7523350 msgr2=0x5635e752e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:57.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.318+0000 7face4d4ab80 1 --2- 172.21.15.101:0/2113598492 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635e7523350 0x5635e752e500 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7facbc003900 tx=0x7facbc0047c0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.318+0000 7face4d4ab80 1 -- 172.21.15.101:0/2113598492 shutdown_connections 2023-07-18T03:07:57.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.318+0000 7face4d4ab80 1 --2- 172.21.15.101:0/2113598492 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5635e753e4e0 0x5635e75408f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.318+0000 7face4d4ab80 1 --2- 172.21.15.101:0/2113598492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635e752eb50 0x5635e752be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.318+0000 7face4d4ab80 1 --2- 172.21.15.101:0/2113598492 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635e7523350 0x5635e752e500 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.318+0000 7face4d4ab80 1 --2- 172.21.15.101:0/2113598492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635e7523c60 0x5635e752b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.318+0000 7facaeff5700 0 client.15648 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:07:57.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.318+0000 7face4d4ab80 1 -- 172.21.15.101:0/2113598492 >> 172.21.15.101:0/2113598492 conn(0x5635e7519e20 msgr2=0x5635e751f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:57.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.319+0000 7face4d4ab80 1 -- 172.21.15.101:0/2113598492 shutdown_connections 2023-07-18T03:07:57.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.319+0000 7face4d4ab80 1 -- 172.21.15.101:0/2113598492 wait complete. 2023-07-18T03:07:57.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.319+0000 7face4d4ab80 20 client.15648 trim_cache size 0 max 16384 2023-07-18T03:07:57.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.337+0000 7face4d4ab80 1 Processor -- start 2023-07-18T03:07:57.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.337+0000 7face4d4ab80 1 -- start start 2023-07-18T03:07:57.327 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.337+0000 7face4d4ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635e7523350 0x5635e752e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:57.327 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.337+0000 7face4d4ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635e7523c60 0x5635e752afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:57.327 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.337+0000 7face4d4ab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635e752eb50 0x5635e752b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:57.328 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.338+0000 7face4d4ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e753ade0 con 0x5635e7523350 2023-07-18T03:07:57.328 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.338+0000 7face4d4ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e75199f0 con 0x5635e7523c60 2023-07-18T03:07:57.328 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.338+0000 7face4d4ab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e75241e0 con 0x5635e752eb50 2023-07-18T03:07:57.328 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.338+0000 7faccaffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635e7523350 0x5635e752e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:57.328 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.338+0000 7faccaffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635e7523350 0x5635e752e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:46180/0 (socket says 172.21.15.101:46180) 2023-07-18T03:07:57.328 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.338+0000 7faccb7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635e752eb50 0x5635e752b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:57.328 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.338+0000 7faccaffd700 1 -- 172.21.15.101:0/2109995322 learned_addr learned my addr 172.21.15.101:0/2109995322 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:57.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.338+0000 7facca7fc700 1 --2- 172.21.15.101:0/2109995322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635e7523c60 0x5635e752afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:57.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.338+0000 7faccb7fe700 1 -- 172.21.15.101:0/2109995322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635e7523c60 msgr2=0x5635e752afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:57.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.338+0000 7faccb7fe700 1 --2- 172.21.15.101:0/2109995322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635e7523c60 0x5635e752afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.338+0000 7faccb7fe700 1 -- 172.21.15.101:0/2109995322 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635e7523350 msgr2=0x5635e752e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:57.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.338+0000 7faccb7fe700 1 --2- 172.21.15.101:0/2109995322 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635e7523350 0x5635e752e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.338+0000 7faccb7fe700 1 -- 172.21.15.101:0/2109995322 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635e72ca2d0 con 0x5635e752eb50 2023-07-18T03:07:57.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.338+0000 7facca7fc700 1 --2- 172.21.15.101:0/2109995322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635e7523c60 0x5635e752afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:07:57.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.339+0000 7faccb7fe700 1 --2- 172.21.15.101:0/2109995322 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635e752eb50 0x5635e752b510 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7facc40c9fc0 tx=0x7facc40b9040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc947ebeffcc356e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:57.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.339+0000 7facb1ffb700 1 -- 172.21.15.101:0/2109995322 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc4004ad0 con 0x5635e752eb50 2023-07-18T03:07:57.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.339+0000 7facb1ffb700 1 -- 172.21.15.101:0/2109995322 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facc400ce60 con 0x5635e752eb50 2023-07-18T03:07:57.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.339+0000 7facb1ffb700 1 -- 172.21.15.101:0/2109995322 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc4099b80 con 0x5635e752eb50 2023-07-18T03:07:57.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.340+0000 7face4d4ab80 1 -- 172.21.15.101:0/2109995322 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635e752eb50 msgr2=0x5635e752b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:57.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.340+0000 7face4d4ab80 1 --2- 172.21.15.101:0/2109995322 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635e752eb50 0x5635e752b510 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7facc40c9fc0 tx=0x7facc40b9040 comp rx=0 tx=0).stop 2023-07-18T03:07:57.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.340+0000 7face4d4ab80 1 -- 172.21.15.101:0/2109995322 shutdown_connections 2023-07-18T03:07:57.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.340+0000 7face4d4ab80 1 --2- 172.21.15.101:0/2109995322 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635e752eb50 0x5635e752b510 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.340+0000 7face4d4ab80 1 --2- 172.21.15.101:0/2109995322 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635e7523350 0x5635e752e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.340+0000 7face4d4ab80 1 --2- 172.21.15.101:0/2109995322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635e7523c60 0x5635e752afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.340+0000 7face4d4ab80 1 -- 172.21.15.101:0/2109995322 >> 172.21.15.101:0/2109995322 conn(0x5635e7519e20 msgr2=0x5635e751de90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:57.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.340+0000 7face4d4ab80 1 -- 172.21.15.101:0/2109995322 shutdown_connections 2023-07-18T03:07:57.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.340+0000 7face4d4ab80 1 -- 172.21.15.101:0/2109995322 wait complete. 2023-07-18T03:07:57.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.340+0000 7face4d4ab80 1 -- start start 2023-07-18T03:07:57.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.340+0000 7face4d4ab80 1 Processor -- start 2023-07-18T03:07:57.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.341+0000 7face4d4ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635e7523350 0x5635e7533d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:57.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.341+0000 7face4d4ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635e7523c60 0x5635e7534280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:57.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.341+0000 7face4d4ab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635e752eb50 0x5635e75347c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:57.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.341+0000 7face4d4ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e75924a0 con 0x5635e7523350 2023-07-18T03:07:57.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.341+0000 7face4d4ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e7592610 con 0x5635e7523c60 2023-07-18T03:07:57.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.341+0000 7face4d4ab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e75927d0 con 0x5635e752eb50 2023-07-18T03:07:57.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.341+0000 7face4d4ab80 10 client.0 _ll_register_callbacks cb 0x7facbc0043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:07:57.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.341+0000 7faccaffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635e7523350 0x5635e7533d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:57.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.341+0000 7faccaffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635e7523350 0x5635e7533d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:46182/0 (socket says 172.21.15.101:46182) 2023-07-18T03:07:57.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.341+0000 7faccaffd700 1 -- 172.21.15.101:0/1810343565 learned_addr learned my addr 172.21.15.101:0/1810343565 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:57.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.341+0000 7facca7fc700 1 --2- 172.21.15.101:0/1810343565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635e7523c60 0x5635e7534280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:57.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.341+0000 7faccb7fe700 1 --2- 172.21.15.101:0/1810343565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635e752eb50 0x5635e75347c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:57.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.342+0000 7faccaffd700 1 -- 172.21.15.101:0/1810343565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635e752eb50 msgr2=0x5635e75347c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:57.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.342+0000 7faccaffd700 1 --2- 172.21.15.101:0/1810343565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635e752eb50 0x5635e75347c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.342+0000 7faccaffd700 1 -- 172.21.15.101:0/1810343565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635e7523c60 msgr2=0x5635e7534280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:57.338 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.342+0000 7faccaffd700 1 --2- 172.21.15.101:0/1810343565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635e7523c60 0x5635e7534280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.338 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.342+0000 7faccaffd700 1 -- 172.21.15.101:0/1810343565 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5635e72a3e60 con 0x5635e7523350 2023-07-18T03:07:57.338 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.342+0000 7facca7fc700 1 --2- 172.21.15.101:0/1810343565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635e7523c60 0x5635e7534280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:07:57.338 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.342+0000 7faccaffd700 1 --2- 172.21.15.101:0/1810343565 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635e7523350 0x5635e7533d40 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7facbc003900 tx=0x7facbc01d8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=621ed52ea40217d5 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:57.338 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.342+0000 7facaeff5700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:07:57.338 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.342+0000 7facaeff5700 1 -- 172.21.15.101:0/1810343565 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc022040 con 0x5635e7523350 2023-07-18T03:07:57.338 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.342+0000 7facaeff5700 1 -- 172.21.15.101:0/1810343565 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7facbc031080 con 0x5635e7523350 2023-07-18T03:07:57.339 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.342+0000 7face4d4ab80 1 -- 172.21.15.101:0/1810343565 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7facc001e040 con 0x5635e7523350 2023-07-18T03:07:57.339 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.342+0000 7facaeff5700 1 -- 172.21.15.101:0/1810343565 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7facbc009da0 con 0x5635e7523350 2023-07-18T03:07:57.339 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.342+0000 7face4d4ab80 10 client.15654 fetch_fsmap learned FSMap version 24 2023-07-18T03:07:57.339 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.342+0000 7face4d4ab80 1 -- 172.21.15.101:0/1810343565 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7faccc1a4bd0 con 0x5635e7523350 2023-07-18T03:07:57.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.343+0000 7facaeff5700 1 -- 172.21.15.101:0/1810343565 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facc001e040 con 0x5635e7523350 2023-07-18T03:07:57.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.343+0000 7facaeff5700 1 -- 172.21.15.101:0/1810343565 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7facbc034420 con 0x5635e7523350 2023-07-18T03:07:57.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.343+0000 7face4d4ab80 10 client.15654 fetch_fsmap finished waiting for FSMap version 24 2023-07-18T03:07:57.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.343+0000 7face4d4ab80 10 client.15654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:07:57.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.343+0000 7face4d4ab80 20 client.15654 populate_metadata read hostname 'smithi101' 2023-07-18T03:07:57.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.343+0000 7face4d4ab80 1 --2- 172.21.15.101:0/1810343565 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5635e7593f20 0x5635e7596330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:57.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.343+0000 7face4d4ab80 4 client.15654 mds_command: new command op to 24562 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-18T03:07:57.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.343+0000 7face4d4ab80 1 -- 172.21.15.101:0/1810343565 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5635e74310d0 con 0x5635e7593f20 2023-07-18T03:07:57.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.343+0000 7facca7fc700 1 --2- 172.21.15.101:0/1810343565 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5635e7593f20 0x5635e7596330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:57.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.344+0000 7facca7fc700 1 --2- 172.21.15.101:0/1810343565 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5635e7593f20 0x5635e7596330 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b39ac19e1bd6b18 server_cookie=adc7ad52e96c5270 in_seq=0 out_seq=0 2023-07-18T03:07:57.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.344+0000 7facaeff5700 10 client.15654 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:07:57.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.345+0000 7facaeff5700 1 -- 172.21.15.101:0/1810343565 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5635e74310d0 con 0x5635e7593f20 2023-07-18T03:07:57.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.345+0000 7facaeff5700 10 client.15654 handle_command_reply: tid=0 2023-07-18T03:07:57.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.345+0000 7face4d4ab80 1 client.15654 shutdown 2023-07-18T03:07:57.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.345+0000 7face4d4ab80 1 -- 172.21.15.101:0/1810343565 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635e7523350 msgr2=0x5635e7533d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:57.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.345+0000 7face4d4ab80 1 --2- 172.21.15.101:0/1810343565 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635e7523350 0x5635e7533d40 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7facbc003900 tx=0x7facbc01d8d0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.346+0000 7face4d4ab80 1 -- 172.21.15.101:0/1810343565 shutdown_connections 2023-07-18T03:07:57.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.346+0000 7face4d4ab80 1 --2- 172.21.15.101:0/1810343565 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5635e7593f20 0x5635e7596330 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.346+0000 7face4d4ab80 1 --2- 172.21.15.101:0/1810343565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635e752eb50 0x5635e75347c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.346+0000 7face4d4ab80 1 --2- 172.21.15.101:0/1810343565 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635e7523350 0x5635e7533d40 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.346+0000 7face4d4ab80 1 --2- 172.21.15.101:0/1810343565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635e7523c60 0x5635e7534280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.346+0000 7face4d4ab80 1 -- 172.21.15.101:0/1810343565 >> 172.21.15.101:0/1810343565 conn(0x5635e7519e20 msgr2=0x5635e7525460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:57.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.346+0000 7facaeff5700 0 client.15654 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:07:57.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.346+0000 7face4d4ab80 1 -- 172.21.15.101:0/1810343565 shutdown_connections 2023-07-18T03:07:57.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.346+0000 7face4d4ab80 1 -- 172.21.15.101:0/1810343565 wait complete. 2023-07-18T03:07:57.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.346+0000 7face4d4ab80 20 client.15654 trim_cache size 0 max 16384 2023-07-18T03:07:57.344 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:07:57.344 INFO:teuthology.orchestra.run.smithi101.stdout: "return_code": 0, 2023-07-18T03:07:57.345 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_tag": "d4e3ad42-c7f2-4aca-8340-8ca42d448977", 2023-07-18T03:07:57.345 INFO:teuthology.orchestra.run.smithi101.stdout: "mode": "asynchronous" 2023-07-18T03:07:57.345 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:07:57.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.347+0000 7facd13ff700 1 -- 172.21.15.101:0/942006199 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7facb40a1350 msgr2=0x7facb40a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:57.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.347+0000 7facd13ff700 1 --2- 172.21.15.101:0/942006199 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7facb40a1350 0x7facb40a3800 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7facbc009c60 tx=0x7facbc009260 comp rx=0 tx=0).stop 2023-07-18T03:07:57.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.347+0000 7facd13ff700 1 -- 172.21.15.101:0/942006199 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faccc10c310 msgr2=0x7faccc19c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:57.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.347+0000 7facd13ff700 1 --2- 172.21.15.101:0/942006199 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faccc10c310 0x7faccc19c740 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7facc400ef30 tx=0x7facc400c3a0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.347+0000 7facd13ff700 1 -- 172.21.15.101:0/942006199 shutdown_connections 2023-07-18T03:07:57.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.347+0000 7facd13ff700 1 --2- 172.21.15.101:0/942006199 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faccc10c310 0x7faccc19c740 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.347+0000 7facd13ff700 1 --2- 172.21.15.101:0/942006199 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faccc0728b0 0x7faccc197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.347+0000 7facd13ff700 1 --2- 172.21.15.101:0/942006199 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7facb40a1350 0x7facb40a3800 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.347+0000 7facd13ff700 1 --2- 172.21.15.101:0/942006199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faccc0731c0 0x7faccc1982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.347+0000 7facd13ff700 1 -- 172.21.15.101:0/942006199 >> 172.21.15.101:0/942006199 conn(0x7faccc0fb5f0 msgr2=0x7faccc103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:57.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.347+0000 7facd13ff700 1 -- 172.21.15.101:0/942006199 shutdown_connections 2023-07-18T03:07:57.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.347+0000 7facd13ff700 1 -- 172.21.15.101:0/942006199 wait complete. 2023-07-18T03:07:57.353 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:07:57.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.572+0000 7fd9185ff700 1 -- 172.21.15.101:0/219755920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd91010c3a0 msgr2=0x7fd91010e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:57.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.572+0000 7fd9185ff700 1 --2- 172.21.15.101:0/219755920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd91010c3a0 0x7fd91010e780 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd908009f90 tx=0x7fd90800b330 comp rx=0 tx=0).stop 2023-07-18T03:07:57.562 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.572+0000 7fd9185ff700 1 -- 172.21.15.101:0/219755920 shutdown_connections 2023-07-18T03:07:57.562 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.572+0000 7fd9185ff700 1 --2- 172.21.15.101:0/219755920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd91010c3a0 0x7fd91010e780 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.562 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.572+0000 7fd9185ff700 1 --2- 172.21.15.101:0/219755920 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd910072850 0x7fd910072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.572+0000 7fd9185ff700 1 --2- 172.21.15.101:0/219755920 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd910073160 0x7fd91010be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.572+0000 7fd9185ff700 1 -- 172.21.15.101:0/219755920 >> 172.21.15.101:0/219755920 conn(0x7fd9100fb600 msgr2=0x7fd9100fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:57.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.572+0000 7fd9185ff700 1 -- 172.21.15.101:0/219755920 shutdown_connections 2023-07-18T03:07:57.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.572+0000 7fd9185ff700 1 -- 172.21.15.101:0/219755920 wait complete. 2023-07-18T03:07:57.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.573+0000 7fd9185ff700 1 Processor -- start 2023-07-18T03:07:57.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.573+0000 7fd9185ff700 1 -- start start 2023-07-18T03:07:57.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.573+0000 7fd9185ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd910072850 0x7fd910197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:57.567 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.573+0000 7fd9185ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd910073160 0x7fd9101984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:57.567 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.573+0000 7fd9185ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd91010c3a0 0x7fd91019c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:57.567 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.573+0000 7fd9185ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91019cf80 con 0x7fd910073160 2023-07-18T03:07:57.567 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.573+0000 7fd9185ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91019d0f0 con 0x7fd910072850 2023-07-18T03:07:57.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.573+0000 7fd9185ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91019d2b0 con 0x7fd91010c3a0 2023-07-18T03:07:57.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.574+0000 7fd915b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd910073160 0x7fd9101984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:57.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.574+0000 7fd915b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd910073160 0x7fd9101984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:46194/0 (socket says 172.21.15.101:46194) 2023-07-18T03:07:57.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.574+0000 7fd915b9a700 1 -- 172.21.15.101:0/1728870281 learned_addr learned my addr 172.21.15.101:0/1728870281 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:57.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.574+0000 7fd91639b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd910072850 0x7fd910197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:57.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.574+0000 7fd916b9c700 1 --2- 172.21.15.101:0/1728870281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd91010c3a0 0x7fd91019c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:57.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.574+0000 7fd915b9a700 1 -- 172.21.15.101:0/1728870281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd91010c3a0 msgr2=0x7fd91019c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:57.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.574+0000 7fd915b9a700 1 --2- 172.21.15.101:0/1728870281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd91010c3a0 0x7fd91019c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.574+0000 7fd915b9a700 1 -- 172.21.15.101:0/1728870281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd910072850 msgr2=0x7fd910197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:57.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.574+0000 7fd915b9a700 1 --2- 172.21.15.101:0/1728870281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd910072850 0x7fd910197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.574+0000 7fd915b9a700 1 -- 172.21.15.101:0/1728870281 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9000095f0 con 0x7fd910073160 2023-07-18T03:07:57.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.574+0000 7fd916b9c700 1 --2- 172.21.15.101:0/1728870281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd91010c3a0 0x7fd91019c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:07:57.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.574+0000 7fd91639b700 1 --2- 172.21.15.101:0/1728870281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd910072850 0x7fd910197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:07:57.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.575+0000 7fd915b9a700 1 --2- 172.21.15.101:0/1728870281 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd910073160 0x7fd9101984e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd90000ec10 tx=0x7fd90000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:57.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.575+0000 7fd9077fe700 1 -- 172.21.15.101:0/1728870281 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90000cc90 con 0x7fd910073160 2023-07-18T03:07:57.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.575+0000 7fd9185ff700 1 -- 172.21.15.101:0/1728870281 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd908009c40 con 0x7fd910073160 2023-07-18T03:07:57.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.575+0000 7fd9077fe700 1 -- 172.21.15.101:0/1728870281 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9000044e0 con 0x7fd910073160 2023-07-18T03:07:57.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.575+0000 7fd9077fe700 1 -- 172.21.15.101:0/1728870281 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd900010430 con 0x7fd910073160 2023-07-18T03:07:57.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.575+0000 7fd9185ff700 1 -- 172.21.15.101:0/1728870281 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd91019d780 con 0x7fd910073160 2023-07-18T03:07:57.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.578+0000 7fd9077fe700 1 -- 172.21.15.101:0/1728870281 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 23) v1 ==== 135892+0+0 (secure 0 0 0) 0x7fd900003fb0 con 0x7fd910073160 2023-07-18T03:07:57.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.579+0000 7fd92bec2b80 1 Processor -- start 2023-07-18T03:07:57.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.580+0000 7fd92bec2b80 1 -- start start 2023-07-18T03:07:57.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.580+0000 7fd92bec2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd558342d0 0x55cd558346a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:57.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.580+0000 7fd92bec2b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd55834be0 0x55cd5583f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:57.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.580+0000 7fd92bec2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd5583fad0 0x55cd55841eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:57.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.580+0000 7fd92bec2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5573acb0 con 0x55cd558342d0 2023-07-18T03:07:57.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.580+0000 7fd9077fe700 1 --2- 172.21.15.101:0/1728870281 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd8fc0a13d0 0x7fd8fc0a3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:57.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.580+0000 7fd92bec2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5578da40 con 0x55cd5583fad0 2023-07-18T03:07:57.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.580+0000 7fd92bec2b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd55786ed0 con 0x55cd55834be0 2023-07-18T03:07:57.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.580+0000 7fd916b9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd558342d0 0x55cd558346a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:57.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.581+0000 7fd91639b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd55834be0 0x55cd5583f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:57.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.581+0000 7fd916b9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd558342d0 0x55cd558346a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:46206/0 (socket says 172.21.15.101:46206) 2023-07-18T03:07:57.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.581+0000 7fd916b9c700 1 -- 172.21.15.101:0/581747582 learned_addr learned my addr 172.21.15.101:0/581747582 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:57.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.581+0000 7fd9077fe700 1 -- 172.21.15.101:0/1728870281 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fd900014070 con 0x7fd910073160 2023-07-18T03:07:57.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.581+0000 7fd915b9a700 1 --2- 172.21.15.101:0/581747582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd5583fad0 0x55cd55841eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:57.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.581+0000 7fd916b9c700 1 --2- 172.21.15.101:0/1728870281 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd8fc0a13d0 0x7fd8fc0a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:57.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.581+0000 7fd91639b700 1 -- 172.21.15.101:0/581747582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd5583fad0 msgr2=0x55cd55841eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:57.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.581+0000 7fd91639b700 1 --2- 172.21.15.101:0/581747582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd5583fad0 0x55cd55841eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.581+0000 7fd91639b700 1 -- 172.21.15.101:0/581747582 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd558342d0 msgr2=0x55cd558346a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:57.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.581+0000 7fd91639b700 1 --2- 172.21.15.101:0/581747582 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd558342d0 0x55cd558346a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.581+0000 7fd91639b700 1 -- 172.21.15.101:0/581747582 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd555b4de0 con 0x55cd55834be0 2023-07-18T03:07:57.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.581+0000 7fd91639b700 1 --2- 172.21.15.101:0/581747582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd55834be0 0x55cd5583f590 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd90c010450 tx=0x7fd90c010760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb07c9566c4b39b9 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:57.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.582+0000 7fd916b9c700 1 --2- 172.21.15.101:0/1728870281 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd8fc0a13d0 0x7fd8fc0a3880 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd908007970 tx=0x7fd908009700 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:57.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.582+0000 7fd904bfe700 1 -- 172.21.15.101:0/581747582 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c01c480 con 0x55cd55834be0 2023-07-18T03:07:57.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.582+0000 7fd904bfe700 1 -- 172.21.15.101:0/581747582 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd90c015070 con 0x55cd55834be0 2023-07-18T03:07:57.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.582+0000 7fd904bfe700 1 -- 172.21.15.101:0/581747582 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c011900 con 0x55cd55834be0 2023-07-18T03:07:57.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.583+0000 7fd92bec2b80 1 -- 172.21.15.101:0/581747582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd55834be0 msgr2=0x55cd5583f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:57.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.583+0000 7fd92bec2b80 1 --2- 172.21.15.101:0/581747582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd55834be0 0x55cd5583f590 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd90c010450 tx=0x7fd90c010760 comp rx=0 tx=0).stop 2023-07-18T03:07:57.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.583+0000 7fd92bec2b80 1 -- 172.21.15.101:0/581747582 shutdown_connections 2023-07-18T03:07:57.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.583+0000 7fd92bec2b80 1 --2- 172.21.15.101:0/581747582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd5583fad0 0x55cd55841eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.583+0000 7fd92bec2b80 1 --2- 172.21.15.101:0/581747582 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd558342d0 0x55cd558346a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.583+0000 7fd92bec2b80 1 --2- 172.21.15.101:0/581747582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd55834be0 0x55cd5583f590 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.583+0000 7fd92bec2b80 1 -- 172.21.15.101:0/581747582 >> 172.21.15.101:0/581747582 conn(0x55cd5582ada0 msgr2=0x55cd558306a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:57.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.583+0000 7fd92bec2b80 1 -- 172.21.15.101:0/581747582 shutdown_connections 2023-07-18T03:07:57.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.583+0000 7fd92bec2b80 1 -- 172.21.15.101:0/581747582 wait complete. 2023-07-18T03:07:57.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.583+0000 7fd92bec2b80 1 -- start start 2023-07-18T03:07:57.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.584+0000 7fd92bec2b80 1 Processor -- start 2023-07-18T03:07:57.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.584+0000 7fd92bec2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd558342d0 0x55cd5583f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:57.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.585+0000 7fd92bec2b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd55834be0 0x55cd5583c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:57.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.585+0000 7fd92bec2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd5583fad0 0x55cd5583cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:57.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.585+0000 7fd92bec2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd55726ac0 con 0x55cd5583fad0 2023-07-18T03:07:57.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.585+0000 7fd92bec2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd557267c0 con 0x55cd558342d0 2023-07-18T03:07:57.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.585+0000 7fd92bec2b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5560a920 con 0x55cd55834be0 2023-07-18T03:07:57.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.585+0000 7fd92bec2b80 10 client.0 _ll_register_callbacks cb 0x55cd55724760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:07:57.581 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.585+0000 7fd916b9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd5583fad0 0x55cd5583cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:57.581 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.585+0000 7fd91639b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd558342d0 0x55cd5583f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:57.581 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.585+0000 7fd916b9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd5583fad0 0x55cd5583cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:46220/0 (socket says 172.21.15.101:46220) 2023-07-18T03:07:57.582 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.585+0000 7fd916b9c700 1 -- 172.21.15.101:0/1335908504 learned_addr learned my addr 172.21.15.101:0/1335908504 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:57.582 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.585+0000 7fd915b9a700 1 --2- 172.21.15.101:0/1335908504 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd55834be0 0x55cd5583c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:57.582 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.585+0000 7fd91639b700 1 -- 172.21.15.101:0/1335908504 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd55834be0 msgr2=0x55cd5583c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:57.582 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.585+0000 7fd91639b700 1 --2- 172.21.15.101:0/1335908504 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd55834be0 0x55cd5583c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.582 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.585+0000 7fd91639b700 1 -- 172.21.15.101:0/1335908504 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd5583fad0 msgr2=0x55cd5583cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:57.582 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.585+0000 7fd91639b700 1 --2- 172.21.15.101:0/1335908504 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd5583fad0 0x55cd5583cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.582 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.585+0000 7fd91639b700 1 -- 172.21.15.101:0/1335908504 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd90c00bcd0 con 0x55cd558342d0 2023-07-18T03:07:57.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.586+0000 7fd91639b700 1 --2- 172.21.15.101:0/1335908504 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd558342d0 0x55cd5583f450 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd90c000c00 tx=0x7fd90c01cf00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ea7ced93b5b7878 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:57.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.586+0000 7fd8f9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:07:57.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.586+0000 7fd8f9ffb700 1 -- 172.21.15.101:0/1335908504 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c020070 con 0x55cd558342d0 2023-07-18T03:07:57.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.586+0000 7fd92bec2b80 1 -- 172.21.15.101:0/1335908504 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cd556e04f0 con 0x55cd558342d0 2023-07-18T03:07:57.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.586+0000 7fd8f9ffb700 1 -- 172.21.15.101:0/1335908504 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fd90c017070 con 0x55cd558342d0 2023-07-18T03:07:57.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.586+0000 7fd8f9ffb700 1 -- 172.21.15.101:0/1335908504 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd90c0053d0 con 0x55cd558342d0 2023-07-18T03:07:57.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.586+0000 7fd92bec2b80 10 client.25694 fetch_fsmap learned FSMap version 24 2023-07-18T03:07:57.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.586+0000 7fd92bec2b80 1 -- 172.21.15.101:0/1335908504 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55cd5559dc60 con 0x55cd558342d0 2023-07-18T03:07:57.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.587+0000 7fd8f9ffb700 1 -- 172.21.15.101:0/1335908504 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cd556e04f0 con 0x55cd558342d0 2023-07-18T03:07:57.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.587+0000 7fd8f9ffb700 1 -- 172.21.15.101:0/1335908504 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd90c01c6e0 con 0x55cd558342d0 2023-07-18T03:07:57.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.587+0000 7fd92bec2b80 10 client.25694 fetch_fsmap finished waiting for FSMap version 24 2023-07-18T03:07:57.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.587+0000 7fd92bec2b80 10 client.25694 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:07:57.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.587+0000 7fd92bec2b80 20 client.25694 populate_metadata read hostname 'smithi101' 2023-07-18T03:07:57.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.587+0000 7fd92bec2b80 1 --2- 172.21.15.101:0/1335908504 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55cd5584f1c0 0x55cd558515d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:57.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.587+0000 7fd92bec2b80 4 client.25694 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:07:57.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.587+0000 7fd92bec2b80 1 -- 172.21.15.101:0/1335908504 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cd5577feb0 con 0x55cd5584f1c0 2023-07-18T03:07:57.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.587+0000 7fd915b9a700 1 --2- 172.21.15.101:0/1335908504 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55cd5584f1c0 0x55cd558515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:57.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.588+0000 7fd915b9a700 1 --2- 172.21.15.101:0/1335908504 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55cd5584f1c0 0x55cd558515d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c3d741b3c2e117b server_cookie=33882e5d9518bcb in_seq=0 out_seq=0 2023-07-18T03:07:57.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.588+0000 7fd8f9ffb700 10 client.25694 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:07:57.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.588+0000 7fd8f9ffb700 1 -- 172.21.15.101:0/1335908504 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cd5577feb0 con 0x55cd5584f1c0 2023-07-18T03:07:57.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.588+0000 7fd8f9ffb700 10 client.25694 handle_command_reply: tid=0 2023-07-18T03:07:57.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.588+0000 7fd92bec2b80 1 client.25694 shutdown 2023-07-18T03:07:57.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.589+0000 7fd92bec2b80 1 -- 172.21.15.101:0/1335908504 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd558342d0 msgr2=0x55cd5583f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:57.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.589+0000 7fd92bec2b80 1 --2- 172.21.15.101:0/1335908504 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd558342d0 0x55cd5583f450 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd90c000c00 tx=0x7fd90c01cf00 comp rx=0 tx=0).stop 2023-07-18T03:07:57.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.589+0000 7fd92bec2b80 1 -- 172.21.15.101:0/1335908504 shutdown_connections 2023-07-18T03:07:57.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.589+0000 7fd92bec2b80 1 --2- 172.21.15.101:0/1335908504 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55cd5584f1c0 0x55cd558515d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.589+0000 7fd92bec2b80 1 --2- 172.21.15.101:0/1335908504 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd558342d0 0x55cd5583f450 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.589+0000 7fd92bec2b80 1 --2- 172.21.15.101:0/1335908504 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd5583fad0 0x55cd5583cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.589+0000 7fd92bec2b80 1 --2- 172.21.15.101:0/1335908504 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd55834be0 0x55cd5583c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.589+0000 7fd8f9ffb700 0 client.25694 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:07:57.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.589+0000 7fd92bec2b80 1 -- 172.21.15.101:0/1335908504 >> 172.21.15.101:0/1335908504 conn(0x55cd5582ada0 msgr2=0x55cd558301d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:57.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.589+0000 7fd92bec2b80 1 -- 172.21.15.101:0/1335908504 shutdown_connections 2023-07-18T03:07:57.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.589+0000 7fd92bec2b80 1 -- 172.21.15.101:0/1335908504 wait complete. 2023-07-18T03:07:57.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.589+0000 7fd92bec2b80 20 client.25694 trim_cache size 0 max 16384 2023-07-18T03:07:57.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.610+0000 7fd92bec2b80 1 Processor -- start 2023-07-18T03:07:57.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.610+0000 7fd92bec2b80 1 -- start start 2023-07-18T03:07:57.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.611+0000 7fd92bec2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd558342d0 0x55cd5583eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:57.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.611+0000 7fd92bec2b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd55834be0 0x55cd55839a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:57.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.611+0000 7fd92bec2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd5583fad0 0x55cd55839f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:57.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.611+0000 7fd92bec2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd55834930 con 0x55cd558342d0 2023-07-18T03:07:57.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.611+0000 7fd92bec2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd55831d50 con 0x55cd5583fad0 2023-07-18T03:07:57.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.611+0000 7fd92bec2b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5583dcb0 con 0x55cd55834be0 2023-07-18T03:07:57.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.611+0000 7fd91639b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd558342d0 0x55cd5583eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:57.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.611+0000 7fd91639b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd558342d0 0x55cd5583eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:46222/0 (socket says 172.21.15.101:46222) 2023-07-18T03:07:57.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.611+0000 7fd91639b700 1 -- 172.21.15.101:0/3790599938 learned_addr learned my addr 172.21.15.101:0/3790599938 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:57.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.611+0000 7fd916b9c700 1 --2- 172.21.15.101:0/3790599938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd5583fad0 0x55cd55839f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:57.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.611+0000 7fd915b9a700 1 --2- 172.21.15.101:0/3790599938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd55834be0 0x55cd55839a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:57.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.611+0000 7fd91639b700 1 -- 172.21.15.101:0/3790599938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd55834be0 msgr2=0x55cd55839a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:57.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.611+0000 7fd91639b700 1 --2- 172.21.15.101:0/3790599938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd55834be0 0x55cd55839a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.612+0000 7fd91639b700 1 -- 172.21.15.101:0/3790599938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd5583fad0 msgr2=0x55cd55839f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:57.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.612+0000 7fd91639b700 1 --2- 172.21.15.101:0/3790599938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd5583fad0 0x55cd55839f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.612+0000 7fd91639b700 1 -- 172.21.15.101:0/3790599938 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd555db250 con 0x55cd558342d0 2023-07-18T03:07:57.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.612+0000 7fd915b9a700 1 --2- 172.21.15.101:0/3790599938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd55834be0 0x55cd55839a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:07:57.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.612+0000 7fd91639b700 1 --2- 172.21.15.101:0/3790599938 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd558342d0 0x55cd5583eff0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd90c01cd60 tx=0x7fd90c01cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511197d06fc6ed86 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:57.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.612+0000 7fd9053ff700 1 -- 172.21.15.101:0/3790599938 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c020070 con 0x55cd558342d0 2023-07-18T03:07:57.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.612+0000 7fd9053ff700 1 -- 172.21.15.101:0/3790599938 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd90c005000 con 0x55cd558342d0 2023-07-18T03:07:57.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.612+0000 7fd9053ff700 1 -- 172.21.15.101:0/3790599938 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c01c790 con 0x55cd558342d0 2023-07-18T03:07:57.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.613+0000 7fd92bec2b80 1 -- 172.21.15.101:0/3790599938 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd558342d0 msgr2=0x55cd5583eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:57.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.613+0000 7fd92bec2b80 1 --2- 172.21.15.101:0/3790599938 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd558342d0 0x55cd5583eff0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd90c01cd60 tx=0x7fd90c01cbc0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.613+0000 7fd92bec2b80 1 -- 172.21.15.101:0/3790599938 shutdown_connections 2023-07-18T03:07:57.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.613+0000 7fd92bec2b80 1 --2- 172.21.15.101:0/3790599938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd5583fad0 0x55cd55839f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.613+0000 7fd92bec2b80 1 --2- 172.21.15.101:0/3790599938 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd558342d0 0x55cd5583eff0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.613+0000 7fd92bec2b80 1 --2- 172.21.15.101:0/3790599938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd55834be0 0x55cd55839a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.613+0000 7fd92bec2b80 1 -- 172.21.15.101:0/3790599938 >> 172.21.15.101:0/3790599938 conn(0x55cd5582ada0 msgr2=0x55cd5584b720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:57.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.613+0000 7fd92bec2b80 1 -- 172.21.15.101:0/3790599938 shutdown_connections 2023-07-18T03:07:57.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.613+0000 7fd92bec2b80 1 -- 172.21.15.101:0/3790599938 wait complete. 2023-07-18T03:07:57.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.613+0000 7fd92bec2b80 1 -- start start 2023-07-18T03:07:57.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.614+0000 7fd92bec2b80 1 Processor -- start 2023-07-18T03:07:57.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.614+0000 7fd92bec2b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd558342d0 0x55cd55846540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:57.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.614+0000 7fd92bec2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd55834be0 0x55cd558447c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:57.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.614+0000 7fd92bec2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd5583fad0 0x55cd55844d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:57.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.614+0000 7fd92bec2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd558453d0 con 0x55cd55834be0 2023-07-18T03:07:57.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.614+0000 7fd92bec2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd55845540 con 0x55cd5583fad0 2023-07-18T03:07:57.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.614+0000 7fd92bec2b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd558a30b0 con 0x55cd558342d0 2023-07-18T03:07:57.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.614+0000 7fd92bec2b80 10 client.0 _ll_register_callbacks cb 0x7fd9000091e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:07:57.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.614+0000 7fd915b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd55834be0 0x55cd558447c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:57.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.614+0000 7fd915b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd55834be0 0x55cd558447c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:46224/0 (socket says 172.21.15.101:46224) 2023-07-18T03:07:57.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.614+0000 7fd915b9a700 1 -- 172.21.15.101:0/1083831167 learned_addr learned my addr 172.21.15.101:0/1083831167 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:07:57.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.615+0000 7fd91639b700 1 --2- 172.21.15.101:0/1083831167 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd558342d0 0x55cd55846540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:57.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.615+0000 7fd916b9c700 1 --2- 172.21.15.101:0/1083831167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd5583fad0 0x55cd55844d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:57.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.615+0000 7fd915b9a700 1 -- 172.21.15.101:0/1083831167 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd558342d0 msgr2=0x55cd55846540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:57.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.615+0000 7fd915b9a700 1 --2- 172.21.15.101:0/1083831167 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd558342d0 0x55cd55846540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.615+0000 7fd915b9a700 1 -- 172.21.15.101:0/1083831167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd5583fad0 msgr2=0x55cd55844d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:07:57.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.615+0000 7fd915b9a700 1 --2- 172.21.15.101:0/1083831167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd5583fad0 0x55cd55844d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.615+0000 7fd915b9a700 1 -- 172.21.15.101:0/1083831167 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd5559dc60 con 0x55cd55834be0 2023-07-18T03:07:57.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.615+0000 7fd916b9c700 1 --2- 172.21.15.101:0/1083831167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd5583fad0 0x55cd55844d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:07:57.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.615+0000 7fd915b9a700 1 --2- 172.21.15.101:0/1083831167 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd55834be0 0x55cd558447c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd900096cd0 tx=0x7fd90009a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b73a5dbfd4db22a3 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:07:57.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.615+0000 7fd8f9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:07:57.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.615+0000 7fd8f9ffb700 1 -- 172.21.15.101:0/1083831167 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90009e290 con 0x55cd55834be0 2023-07-18T03:07:57.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.615+0000 7fd8f9ffb700 1 -- 172.21.15.101:0/1083831167 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fd9000a26b0 con 0x55cd55834be0 2023-07-18T03:07:57.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.615+0000 7fd92bec2b80 1 -- 172.21.15.101:0/1083831167 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd90c015070 con 0x55cd55834be0 2023-07-18T03:07:57.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.616+0000 7fd8f9ffb700 1 -- 172.21.15.101:0/1083831167 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd900004260 con 0x55cd55834be0 2023-07-18T03:07:57.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.616+0000 7fd92bec2b80 10 client.15672 fetch_fsmap learned FSMap version 24 2023-07-18T03:07:57.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.616+0000 7fd92bec2b80 1 -- 172.21.15.101:0/1083831167 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55cd555b32b0 con 0x55cd55834be0 2023-07-18T03:07:57.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.616+0000 7fd8f9ffb700 1 -- 172.21.15.101:0/1083831167 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd90c015070 con 0x55cd55834be0 2023-07-18T03:07:57.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.616+0000 7fd8f9ffb700 1 -- 172.21.15.101:0/1083831167 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd9000b0760 con 0x55cd55834be0 2023-07-18T03:07:57.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.616+0000 7fd92bec2b80 10 client.15672 fetch_fsmap finished waiting for FSMap version 24 2023-07-18T03:07:57.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.616+0000 7fd92bec2b80 10 client.15672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:07:57.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.616+0000 7fd92bec2b80 20 client.15672 populate_metadata read hostname 'smithi101' 2023-07-18T03:07:57.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.616+0000 7fd92bec2b80 1 --2- 172.21.15.101:0/1083831167 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55cd558a4700 0x55cd558a6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:07:57.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.616+0000 7fd92bec2b80 4 client.15672 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:07:57.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.616+0000 7fd92bec2b80 1 -- 172.21.15.101:0/1083831167 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cd55742050 con 0x55cd558a4700 2023-07-18T03:07:57.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.617+0000 7fd91639b700 1 --2- 172.21.15.101:0/1083831167 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55cd558a4700 0x55cd558a6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:07:57.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.617+0000 7fd8f9ffb700 10 client.15672 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:07:57.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.617+0000 7fd91639b700 1 --2- 172.21.15.101:0/1083831167 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55cd558a4700 0x55cd558a6b10 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7b70e08471b3198 server_cookie=87b27e429036dd75 in_seq=0 out_seq=0 2023-07-18T03:07:57.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.617+0000 7fd8f9ffb700 1 -- 172.21.15.101:0/1083831167 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55cd55742050 con 0x55cd558a4700 2023-07-18T03:07:57.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.617+0000 7fd8f9ffb700 10 client.15672 handle_command_reply: tid=0 2023-07-18T03:07:57.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.617+0000 7fd92bec2b80 1 client.15672 shutdown 2023-07-18T03:07:57.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.618+0000 7fd92bec2b80 1 -- 172.21.15.101:0/1083831167 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd55834be0 msgr2=0x55cd558447c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:57.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.618+0000 7fd92bec2b80 1 --2- 172.21.15.101:0/1083831167 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd55834be0 0x55cd558447c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd900096cd0 tx=0x7fd90009a0a0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.618+0000 7fd92bec2b80 1 -- 172.21.15.101:0/1083831167 shutdown_connections 2023-07-18T03:07:57.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.618+0000 7fd92bec2b80 1 --2- 172.21.15.101:0/1083831167 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55cd558a4700 0x55cd558a6b10 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.618+0000 7fd92bec2b80 1 --2- 172.21.15.101:0/1083831167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd5583fad0 0x55cd55844d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.618+0000 7fd92bec2b80 1 --2- 172.21.15.101:0/1083831167 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd558342d0 0x55cd55846540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.618+0000 7fd92bec2b80 1 --2- 172.21.15.101:0/1083831167 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd55834be0 0x55cd558447c0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.618+0000 7fd92bec2b80 1 -- 172.21.15.101:0/1083831167 >> 172.21.15.101:0/1083831167 conn(0x55cd5582ada0 msgr2=0x55cd5582f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:57.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.618+0000 7fd8f9ffb700 0 client.15672 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:07:57.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.618+0000 7fd92bec2b80 1 -- 172.21.15.101:0/1083831167 shutdown_connections 2023-07-18T03:07:57.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.618+0000 7fd92bec2b80 1 -- 172.21.15.101:0/1083831167 wait complete. 2023-07-18T03:07:57.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.618+0000 7fd92bec2b80 20 client.15672 trim_cache size 0 max 16384 2023-07-18T03:07:57.616 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:07:57.616 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:07:57.616 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:07:57.616 INFO:teuthology.orchestra.run.smithi101.stdout: "d4e3ad42-c7f2-4aca-8340-8ca42d448977": { 2023-07-18T03:07:57.616 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:07:57.616 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "d4e3ad42-c7f2-4aca-8340-8ca42d448977", 2023-07-18T03:07:57.617 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:07:57.617 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:07:57.617 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:07:57.617 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:07:57.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.619+0000 7fd9185ff700 1 -- 172.21.15.101:0/1728870281 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd8fc0a13d0 msgr2=0x7fd8fc0a3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:57.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.619+0000 7fd9185ff700 1 --2- 172.21.15.101:0/1728870281 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd8fc0a13d0 0x7fd8fc0a3880 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd908007970 tx=0x7fd908009700 comp rx=0 tx=0).stop 2023-07-18T03:07:57.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.619+0000 7fd9185ff700 1 -- 172.21.15.101:0/1728870281 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd910073160 msgr2=0x7fd9101984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:07:57.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.619+0000 7fd9185ff700 1 --2- 172.21.15.101:0/1728870281 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd910073160 0x7fd9101984e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd90000ec10 tx=0x7fd90000ef20 comp rx=0 tx=0).stop 2023-07-18T03:07:57.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.619+0000 7fd9185ff700 1 -- 172.21.15.101:0/1728870281 shutdown_connections 2023-07-18T03:07:57.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.619+0000 7fd9185ff700 1 --2- 172.21.15.101:0/1728870281 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd8fc0a13d0 0x7fd8fc0a3880 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.619+0000 7fd9185ff700 1 --2- 172.21.15.101:0/1728870281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd910072850 0x7fd910197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.619+0000 7fd9185ff700 1 --2- 172.21.15.101:0/1728870281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd91010c3a0 0x7fd91019c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.619+0000 7fd9185ff700 1 --2- 172.21.15.101:0/1728870281 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd910073160 0x7fd9101984e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:07:57.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.619+0000 7fd9185ff700 1 -- 172.21.15.101:0/1728870281 >> 172.21.15.101:0/1728870281 conn(0x7fd9100fb600 msgr2=0x7fd910103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:07:57.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.619+0000 7fd9185ff700 1 -- 172.21.15.101:0/1728870281 shutdown_connections 2023-07-18T03:07:57.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:07:57.619+0000 7fd9185ff700 1 -- 172.21.15.101:0/1728870281 wait complete. 2023-07-18T03:07:57.626 INFO:tasks.cephfs.filesystem:scrub status for tag:d4e3ad42-c7f2-4aca-8340-8ca42d448977 - {'path': '/', 'tag': 'd4e3ad42-c7f2-4aca-8340-8ca42d448977', 'options': 'recursive,force'} 2023-07-18T03:07:59.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:58 smithi179 ceph-mon[129229]: scrub queued for path: / 2023-07-18T03:07:59.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:58 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:07:59.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:58 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:07:59.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:58 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:07:59.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:07:58 smithi179 ceph-mon[129229]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:07:59.232 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:58 smithi101 ceph-mon[126379]: scrub queued for path: / 2023-07-18T03:07:59.232 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:58 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:07:59.233 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:58 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:07:59.233 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:58 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:07:59.233 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:07:58 smithi101 ceph-mon[126379]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:07:59.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:58 smithi155 ceph-mon[128885]: scrub queued for path: / 2023-07-18T03:07:59.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:58 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:07:59.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:58 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:07:59.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:58 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:07:59.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:07:58 smithi155 ceph-mon[128885]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:08:00.482 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:08:00 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:08:00.135+0000 7f017d76a700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-18T03:08:00.482 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:08:00 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:08:00.135+0000 7f017d76a700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-18T03:08:00.483 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:08:00 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:08:00.135+0000 7f017d76a700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-18T03:08:00.483 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:08:00 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:08:00.135+0000 7f017d76a700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-18T03:08:00.483 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:08:00 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:08:00.135+0000 7f017d76a700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-18T03:08:01.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:08:00 smithi179 ceph-mon[129229]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:08:01.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:08:00 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:08:01.232 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:08:00 smithi101 ceph-mon[126379]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:08:01.232 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:08:00 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:08:01.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:08:00 smithi155 ceph-mon[128885]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:08:01.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:08:00 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:08:03.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:08:02 smithi179 ceph-mon[129229]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-07-18T03:08:03.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:08:02 smithi179 ceph-mon[129229]: mgrmap e24: x(active, since 28s), standbys: y, z 2023-07-18T03:08:03.232 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:08:02 smithi101 ceph-mon[126379]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-07-18T03:08:03.232 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:08:02 smithi101 ceph-mon[126379]: mgrmap e24: x(active, since 28s), standbys: y, z 2023-07-18T03:08:03.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:08:02 smithi155 ceph-mon[128885]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-07-18T03:08:03.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:08:02 smithi155 ceph-mon[128885]: mgrmap e24: x(active, since 28s), standbys: y, z 2023-07-18T03:08:05.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:08:04 smithi179 ceph-mon[129229]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-07-18T03:08:05.232 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:08:04 smithi101 ceph-mon[126379]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-07-18T03:08:05.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:08:04 smithi155 ceph-mon[128885]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-07-18T03:08:07.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:08:06 smithi179 ceph-mon[129229]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2023-07-18T03:08:07.232 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:08:06 smithi101 ceph-mon[126379]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2023-07-18T03:08:07.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:08:06 smithi155 ceph-mon[128885]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2023-07-18T03:08:09.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:08:08 smithi179 ceph-mon[129229]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 17 op/s 2023-07-18T03:08:09.232 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:08:08 smithi101 ceph-mon[126379]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 17 op/s 2023-07-18T03:08:09.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:08:08 smithi155 ceph-mon[128885]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 17 op/s 2023-07-18T03:08:11.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:08:10 smithi179 ceph-mon[129229]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 17 op/s 2023-07-18T03:08:11.232 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:08:10 smithi101 ceph-mon[126379]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 17 op/s 2023-07-18T03:08:11.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:08:10 smithi155 ceph-mon[128885]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 17 op/s 2023-07-18T03:08:13.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:08:12 smithi179 ceph-mon[129229]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2023-07-18T03:08:13.232 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:08:12 smithi101 ceph-mon[126379]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2023-07-18T03:08:13.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:08:12 smithi155 ceph-mon[128885]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2023-07-18T03:08:15.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:08:14 smithi179 ceph-mon[129229]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 17 op/s 2023-07-18T03:08:15.232 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:08:14 smithi101 ceph-mon[126379]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 17 op/s 2023-07-18T03:08:15.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:08:14 smithi155 ceph-mon[128885]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 17 op/s 2023-07-18T03:08:17.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:08:16 smithi179 ceph-mon[129229]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2023-07-18T03:08:17.233 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:08:16 smithi101 ceph-mon[126379]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2023-07-18T03:08:17.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:08:16 smithi155 ceph-mon[128885]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2023-07-18T03:08:19.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:08:18 smithi179 ceph-mon[129229]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2023-07-18T03:08:19.232 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:08:18 smithi101 ceph-mon[126379]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2023-07-18T03:08:19.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:08:18 smithi155 ceph-mon[128885]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2023-07-18T03:08:21.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:08:20 smithi179 ceph-mon[129229]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 7 op/s 2023-07-18T03:08:21.232 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:08:20 smithi101 ceph-mon[126379]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 7 op/s 2023-07-18T03:08:21.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:08:20 smithi155 ceph-mon[128885]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 7 op/s 2023-07-18T03:08:23.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:08:22 smithi179 ceph-mon[129229]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 7 op/s 2023-07-18T03:08:23.233 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:08:22 smithi101 ceph-mon[126379]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 7 op/s 2023-07-18T03:08:23.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:08:22 smithi155 ceph-mon[128885]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 7 op/s 2023-07-18T03:08:25.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:08:24 smithi179 ceph-mon[129229]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2023-07-18T03:08:25.233 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:08:24 smithi101 ceph-mon[126379]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2023-07-18T03:08:25.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:08:24 smithi155 ceph-mon[128885]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2023-07-18T03:08:27.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:08:26 smithi179 ceph-mon[129229]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2023-07-18T03:08:27.233 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:08:26 smithi101 ceph-mon[126379]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2023-07-18T03:08:27.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:08:26 smithi155 ceph-mon[128885]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2023-07-18T03:08:27.627 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:08:27.970 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.981+0000 7f82fdb9d700 1 -- 172.21.15.101:0/3241879050 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82f81013b0 msgr2=0x7f82f8109a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:27.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.981+0000 7f82fdb9d700 1 --2- 172.21.15.101:0/3241879050 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82f81013b0 0x7f82f8109a80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f82e4009990 tx=0x7f82e4009ca0 comp rx=0 tx=0).stop 2023-07-18T03:08:27.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.981+0000 7f82fdb9d700 1 -- 172.21.15.101:0/3241879050 shutdown_connections 2023-07-18T03:08:27.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.981+0000 7f82fdb9d700 1 --2- 172.21.15.101:0/3241879050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82f8109fc0 0x7f82f810c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:27.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.981+0000 7f82fdb9d700 1 --2- 172.21.15.101:0/3241879050 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f82f8100aa0 0x7f82f8100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:27.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.981+0000 7f82fdb9d700 1 --2- 172.21.15.101:0/3241879050 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82f81013b0 0x7f82f8109a80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:27.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.981+0000 7f82fdb9d700 1 -- 172.21.15.101:0/3241879050 >> 172.21.15.101:0/3241879050 conn(0x7f82f80745d0 msgr2=0x7f82f80749d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:08:27.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.981+0000 7f82fdb9d700 1 -- 172.21.15.101:0/3241879050 shutdown_connections 2023-07-18T03:08:27.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.981+0000 7f82fdb9d700 1 -- 172.21.15.101:0/3241879050 wait complete. 2023-07-18T03:08:27.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.983+0000 7f82fdb9d700 1 Processor -- start 2023-07-18T03:08:27.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.983+0000 7f82fdb9d700 1 -- start start 2023-07-18T03:08:27.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.983+0000 7f82fdb9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82f8100aa0 0x7f82f80fe8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:27.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.984+0000 7f82fdb9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82f81013b0 0x7f82f80fee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:27.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.984+0000 7f82fdb9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f82f8109fc0 0x7f82f81a3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:27.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.984+0000 7f82fdb9d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f80ff640 con 0x7f82f8109fc0 2023-07-18T03:08:27.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.984+0000 7f82fdb9d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f81a4100 con 0x7f82f8100aa0 2023-07-18T03:08:27.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.984+0000 7f82fdb9d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f81a4240 con 0x7f82f81013b0 2023-07-18T03:08:27.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.984+0000 7f82fcb9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82f8100aa0 0x7f82f80fe8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:27.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.984+0000 7f82fcb9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82f8100aa0 0x7f82f80fe8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:48090/0 (socket says 172.21.15.101:48090) 2023-07-18T03:08:27.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.984+0000 7f82fcb9b700 1 -- 172.21.15.101:0/54362199 learned_addr learned my addr 172.21.15.101:0/54362199 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:08:27.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.984+0000 7f82fd39c700 1 --2- 172.21.15.101:0/54362199 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f82f8109fc0 0x7f82f81a3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:27.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.984+0000 7f82f7fff700 1 --2- 172.21.15.101:0/54362199 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82f81013b0 0x7f82f80fee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:27.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.984+0000 7f82fcb9b700 1 -- 172.21.15.101:0/54362199 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82f81013b0 msgr2=0x7f82f80fee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:27.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.984+0000 7f82fcb9b700 1 --2- 172.21.15.101:0/54362199 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82f81013b0 0x7f82f80fee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:27.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.984+0000 7f82fcb9b700 1 -- 172.21.15.101:0/54362199 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f82f8109fc0 msgr2=0x7f82f81a3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:27.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.984+0000 7f82fcb9b700 1 --2- 172.21.15.101:0/54362199 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f82f8109fc0 0x7f82f81a3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:27.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.984+0000 7f82fcb9b700 1 -- 172.21.15.101:0/54362199 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e40095f0 con 0x7f82f8100aa0 2023-07-18T03:08:27.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.985+0000 7f82fd39c700 1 --2- 172.21.15.101:0/54362199 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f82f8109fc0 0x7f82f81a3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:08:27.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.985+0000 7f82fcb9b700 1 --2- 172.21.15.101:0/54362199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82f8100aa0 0x7f82f80fe8d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00e9c0 tx=0x7f82ec00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:08:27.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.985+0000 7f82f5ffb700 1 -- 172.21.15.101:0/54362199 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ec0097b0 con 0x7f82f8100aa0 2023-07-18T03:08:27.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.985+0000 7f82f5ffb700 1 -- 172.21.15.101:0/54362199 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82ec004cf0 con 0x7f82f8100aa0 2023-07-18T03:08:27.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.985+0000 7f82fdb9d700 1 -- 172.21.15.101:0/54362199 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82f81a44a0 con 0x7f82f8100aa0 2023-07-18T03:08:27.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.985+0000 7f82f5ffb700 1 -- 172.21.15.101:0/54362199 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ec005600 con 0x7f82f8100aa0 2023-07-18T03:08:27.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.985+0000 7f82fdb9d700 1 -- 172.21.15.101:0/54362199 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82f81a4980 con 0x7f82f8100aa0 2023-07-18T03:08:27.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.990+0000 7f82f5ffb700 1 -- 172.21.15.101:0/54362199 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) v1 ==== 135949+0+0 (secure 0 0 0) 0x7f82ec00cc60 con 0x7f82f8100aa0 2023-07-18T03:08:27.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.992+0000 7f8312704b80 1 Processor -- start 2023-07-18T03:08:27.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.992+0000 7f8312704b80 1 -- start start 2023-07-18T03:08:27.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.993+0000 7f8312704b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5c9d114f0 0x55a5c9d118c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:27.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.993+0000 7f8312704b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a5c9d11e00 0x55a5c9d1b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:27.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.993+0000 7f8312704b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5c9d1bd30 0x55a5c9d1e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:27.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.993+0000 7f8312704b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5c9c16cb0 con 0x55a5c9d11e00 2023-07-18T03:08:27.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.993+0000 7f8312704b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5c9c69a40 con 0x55a5c9d114f0 2023-07-18T03:08:27.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.993+0000 7f8312704b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5c9c62ed0 con 0x55a5c9d1bd30 2023-07-18T03:08:27.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.993+0000 7f82f7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5c9d1bd30 0x55a5c9d1e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:27.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.993+0000 7f82f7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5c9d1bd30 0x55a5c9d1e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:59358/0 (socket says 172.21.15.101:59358) 2023-07-18T03:08:27.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.993+0000 7f82f7fff700 1 -- 172.21.15.101:0/2170112429 learned_addr learned my addr 172.21.15.101:0/2170112429 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:08:27.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.993+0000 7f82f5ffb700 1 --2- 172.21.15.101:0/54362199 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f82e80a1500 0x7f82e80a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:27.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.993+0000 7f82fd39c700 1 --2- 172.21.15.101:0/2170112429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5c9d114f0 0x55a5c9d118c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:27.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.994+0000 7f82f5ffb700 1 -- 172.21.15.101:0/54362199 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f82ec014070 con 0x7f82f8100aa0 2023-07-18T03:08:27.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.994+0000 7f82fd39c700 1 -- 172.21.15.101:0/2170112429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5c9d1bd30 msgr2=0x55a5c9d1e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:08:27.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.994+0000 7f82fcb9b700 1 --2- 172.21.15.101:0/2170112429 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a5c9d11e00 0x55a5c9d1b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:27.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.994+0000 7f82fd39c700 1 --2- 172.21.15.101:0/2170112429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5c9d1bd30 0x55a5c9d1e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:27.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.994+0000 7f82fd39c700 1 -- 172.21.15.101:0/2170112429 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a5c9d11e00 msgr2=0x55a5c9d1b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:08:27.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.994+0000 7f82fd39c700 1 --2- 172.21.15.101:0/2170112429 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a5c9d11e00 0x55a5c9d1b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:27.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.994+0000 7f82fd39c700 1 -- 172.21.15.101:0/2170112429 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5c9a90de0 con 0x55a5c9d114f0 2023-07-18T03:08:27.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.994+0000 7f82fcb9b700 1 --2- 172.21.15.101:0/2170112429 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a5c9d11e00 0x55a5c9d1b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:08:27.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.995+0000 7f82fd39c700 1 --2- 172.21.15.101:0/2170112429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5c9d114f0 0x55a5c9d118c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f82f8073240 tx=0x7f82f00077e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aac35bb8d46aa92e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:08:27.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.995+0000 7f82fd39c700 1 --2- 172.21.15.101:0/54362199 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f82e80a1500 0x7f82e80a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:27.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.995+0000 7f82e2ffd700 1 -- 172.21.15.101:0/2170112429 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f000e070 con 0x55a5c9d114f0 2023-07-18T03:08:27.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.995+0000 7f82e2ffd700 1 -- 172.21.15.101:0/2170112429 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82f000f040 con 0x55a5c9d114f0 2023-07-18T03:08:27.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.995+0000 7f82e2ffd700 1 -- 172.21.15.101:0/2170112429 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f001f650 con 0x55a5c9d114f0 2023-07-18T03:08:27.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.996+0000 7f8312704b80 1 -- 172.21.15.101:0/2170112429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5c9d114f0 msgr2=0x55a5c9d118c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:27.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.996+0000 7f82fd39c700 1 --2- 172.21.15.101:0/54362199 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f82e80a1500 0x7f82e80a39b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f82f00094f0 tx=0x7f82f001f890 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:08:27.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.996+0000 7f8312704b80 1 --2- 172.21.15.101:0/2170112429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5c9d114f0 0x55a5c9d118c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f82f8073240 tx=0x7f82f00077e0 comp rx=0 tx=0).stop 2023-07-18T03:08:27.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.996+0000 7f8312704b80 1 -- 172.21.15.101:0/2170112429 shutdown_connections 2023-07-18T03:08:27.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.996+0000 7f8312704b80 1 --2- 172.21.15.101:0/2170112429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5c9d114f0 0x55a5c9d118c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:27.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.996+0000 7f8312704b80 1 --2- 172.21.15.101:0/2170112429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5c9d1bd30 0x55a5c9d1e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:27.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.996+0000 7f8312704b80 1 --2- 172.21.15.101:0/2170112429 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a5c9d11e00 0x55a5c9d1b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:27.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.996+0000 7f8312704b80 1 -- 172.21.15.101:0/2170112429 >> 172.21.15.101:0/2170112429 conn(0x55a5c9d06ff0 msgr2=0x55a5c9d0c930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:08:27.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.996+0000 7f8312704b80 1 -- 172.21.15.101:0/2170112429 shutdown_connections 2023-07-18T03:08:27.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.996+0000 7f8312704b80 1 -- 172.21.15.101:0/2170112429 wait complete. 2023-07-18T03:08:27.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.997+0000 7f8312704b80 1 -- start start 2023-07-18T03:08:27.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.997+0000 7f8312704b80 1 Processor -- start 2023-07-18T03:08:27.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.998+0000 7f8312704b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5c9d114f0 0x55a5c9d1b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:27.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.998+0000 7f8312704b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5c9d11e00 0x55a5c9d18930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:27.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.998+0000 7f8312704b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a5c9d1bd30 0x55a5c9d18f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:27.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.998+0000 7f8312704b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5c9c02ac0 con 0x55a5c9d1bd30 2023-07-18T03:08:27.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.998+0000 7f8312704b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5c9c027c0 con 0x55a5c9d114f0 2023-07-18T03:08:27.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.998+0000 7f8312704b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5c9ae6920 con 0x55a5c9d11e00 2023-07-18T03:08:27.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.998+0000 7f8312704b80 10 client.0 _ll_register_callbacks cb 0x55a5c9c00760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:08:27.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.999+0000 7f82fcb9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5c9d114f0 0x55a5c9d1b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:27.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.999+0000 7f82fcb9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5c9d114f0 0x55a5c9d1b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:48112/0 (socket says 172.21.15.101:48112) 2023-07-18T03:08:27.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.999+0000 7f82fcb9b700 1 -- 172.21.15.101:0/1674846853 learned_addr learned my addr 172.21.15.101:0/1674846853 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:08:27.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.999+0000 7f82fd39c700 1 --2- 172.21.15.101:0/1674846853 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a5c9d1bd30 0x55a5c9d18f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:27.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.999+0000 7f82f7fff700 1 --2- 172.21.15.101:0/1674846853 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5c9d11e00 0x55a5c9d18930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:27.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.999+0000 7f82fcb9b700 1 -- 172.21.15.101:0/1674846853 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5c9d11e00 msgr2=0x55a5c9d18930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:08:27.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.999+0000 7f82fcb9b700 1 --2- 172.21.15.101:0/1674846853 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5c9d11e00 0x55a5c9d18930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:27.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.999+0000 7f82fcb9b700 1 -- 172.21.15.101:0/1674846853 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a5c9d1bd30 msgr2=0x55a5c9d18f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:08:27.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.999+0000 7f82fcb9b700 1 --2- 172.21.15.101:0/1674846853 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a5c9d1bd30 0x55a5c9d18f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:27.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.999+0000 7f82fcb9b700 1 -- 172.21.15.101:0/1674846853 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82f0009090 con 0x55a5c9d114f0 2023-07-18T03:08:27.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.999+0000 7f82f7fff700 1 --2- 172.21.15.101:0/1674846853 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5c9d11e00 0x55a5c9d18930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:08:27.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.999+0000 7f82fcb9b700 1 --2- 172.21.15.101:0/1674846853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5c9d114f0 0x55a5c9d1b6b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f82ec0965a0 tx=0x7f82ec004f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2165f7e05551ec4e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:08:27.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.999+0000 7f82d7fff700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:08:27.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:27.999+0000 7f82d7fff700 1 -- 172.21.15.101:0/1674846853 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ec0cd040 con 0x55a5c9d114f0 2023-07-18T03:08:27.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.000+0000 7f82d7fff700 1 -- 172.21.15.101:0/1674846853 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f82ec0cd3b0 con 0x55a5c9d114f0 2023-07-18T03:08:27.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.000+0000 7f8312704b80 1 -- 172.21.15.101:0/1674846853 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a5c9bbc4f0 con 0x55a5c9d114f0 2023-07-18T03:08:27.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.000+0000 7f82d7fff700 1 -- 172.21.15.101:0/1674846853 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82ec09e0d0 con 0x55a5c9d114f0 2023-07-18T03:08:27.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.000+0000 7f8312704b80 10 client.25718 fetch_fsmap learned FSMap version 24 2023-07-18T03:08:27.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.000+0000 7f8312704b80 1 -- 172.21.15.101:0/1674846853 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55a5c9a79c60 con 0x55a5c9d114f0 2023-07-18T03:08:27.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.000+0000 7f82d7fff700 1 -- 172.21.15.101:0/1674846853 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a5c9bbc4f0 con 0x55a5c9d114f0 2023-07-18T03:08:27.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.001+0000 7f82d7fff700 1 -- 172.21.15.101:0/1674846853 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f82ec099690 con 0x55a5c9d114f0 2023-07-18T03:08:28.000 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.001+0000 7f8312704b80 10 client.25718 fetch_fsmap finished waiting for FSMap version 24 2023-07-18T03:08:28.000 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.001+0000 7f8312704b80 10 client.25718 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:08:28.000 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.001+0000 7f8312704b80 20 client.25718 populate_metadata read hostname 'smithi101' 2023-07-18T03:08:28.000 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.001+0000 7f8312704b80 1 --2- 172.21.15.101:0/1674846853 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a5c9d2b640 0x55a5c9d2da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:28.000 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.001+0000 7f8312704b80 4 client.25718 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:08:28.001 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.001+0000 7f8312704b80 1 -- 172.21.15.101:0/1674846853 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a5c9c5beb0 con 0x55a5c9d2b640 2023-07-18T03:08:28.001 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.002+0000 7f82f7fff700 1 --2- 172.21.15.101:0/1674846853 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a5c9d2b640 0x55a5c9d2da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:28.001 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.002+0000 7f82d7fff700 10 client.25718 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:08:28.001 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.002+0000 7f82f7fff700 1 --2- 172.21.15.101:0/1674846853 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a5c9d2b640 0x55a5c9d2da50 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb23f121fccbed92 server_cookie=af66fc7566f7db93 in_seq=0 out_seq=0 2023-07-18T03:08:28.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.003+0000 7f82d7fff700 1 -- 172.21.15.101:0/1674846853 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a5c9c5beb0 con 0x55a5c9d2b640 2023-07-18T03:08:28.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.003+0000 7f82d7fff700 10 client.25718 handle_command_reply: tid=0 2023-07-18T03:08:28.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.003+0000 7f8312704b80 1 client.25718 shutdown 2023-07-18T03:08:28.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.003+0000 7f8312704b80 1 -- 172.21.15.101:0/1674846853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5c9d114f0 msgr2=0x55a5c9d1b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:28.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.003+0000 7f8312704b80 1 --2- 172.21.15.101:0/1674846853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5c9d114f0 0x55a5c9d1b6b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f82ec0965a0 tx=0x7f82ec004f10 comp rx=0 tx=0).stop 2023-07-18T03:08:28.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.004+0000 7f8312704b80 1 -- 172.21.15.101:0/1674846853 shutdown_connections 2023-07-18T03:08:28.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.004+0000 7f8312704b80 1 --2- 172.21.15.101:0/1674846853 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a5c9d2b640 0x55a5c9d2da50 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:28.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.004+0000 7f8312704b80 1 --2- 172.21.15.101:0/1674846853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5c9d114f0 0x55a5c9d1b6b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:28.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.004+0000 7f8312704b80 1 --2- 172.21.15.101:0/1674846853 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a5c9d1bd30 0x55a5c9d18f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:28.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.004+0000 7f82d7fff700 0 client.25718 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:08:28.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.004+0000 7f8312704b80 1 --2- 172.21.15.101:0/1674846853 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5c9d11e00 0x55a5c9d18930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:28.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.004+0000 7f8312704b80 1 -- 172.21.15.101:0/1674846853 >> 172.21.15.101:0/1674846853 conn(0x55a5c9d06ff0 msgr2=0x55a5c9d0c520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:08:28.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.004+0000 7f8312704b80 1 -- 172.21.15.101:0/1674846853 shutdown_connections 2023-07-18T03:08:28.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.004+0000 7f8312704b80 1 -- 172.21.15.101:0/1674846853 wait complete. 2023-07-18T03:08:28.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.004+0000 7f8312704b80 20 client.25718 trim_cache size 0 max 16384 2023-07-18T03:08:28.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.037+0000 7f8312704b80 1 Processor -- start 2023-07-18T03:08:28.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.037+0000 7f8312704b80 1 -- start start 2023-07-18T03:08:28.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.037+0000 7f8312704b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a5c9d114f0 0x55a5c9d1b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:28.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.037+0000 7f8312704b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5c9d11e00 0x55a5c9d180f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:28.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.037+0000 7f8312704b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5c9d1bd30 0x55a5c9d18630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:28.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.037+0000 7f8312704b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5c9d12770 con 0x55a5c9d114f0 2023-07-18T03:08:28.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.038+0000 7f8312704b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5c9d12980 con 0x55a5c9d11e00 2023-07-18T03:08:28.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.038+0000 7f8312704b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5c9d2b420 con 0x55a5c9d1bd30 2023-07-18T03:08:28.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.038+0000 7f82f7fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5c9d11e00 0x55a5c9d180f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:28.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.038+0000 7f82fd39c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5c9d1bd30 0x55a5c9d18630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:28.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.038+0000 7f82f7fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5c9d11e00 0x55a5c9d180f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:48126/0 (socket says 172.21.15.101:48126) 2023-07-18T03:08:28.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.038+0000 7f82f7fff700 1 -- 172.21.15.101:0/880313558 learned_addr learned my addr 172.21.15.101:0/880313558 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:08:28.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.038+0000 7f82fcb9b700 1 --2- 172.21.15.101:0/880313558 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a5c9d114f0 0x55a5c9d1b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:28.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.039+0000 7f82f7fff700 1 -- 172.21.15.101:0/880313558 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5c9d1bd30 msgr2=0x55a5c9d18630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:08:28.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.039+0000 7f82f7fff700 1 --2- 172.21.15.101:0/880313558 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5c9d1bd30 0x55a5c9d18630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:28.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.039+0000 7f82f7fff700 1 -- 172.21.15.101:0/880313558 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a5c9d114f0 msgr2=0x55a5c9d1b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:08:28.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.039+0000 7f82f7fff700 1 --2- 172.21.15.101:0/880313558 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a5c9d114f0 0x55a5c9d1b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:28.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.039+0000 7f82f7fff700 1 -- 172.21.15.101:0/880313558 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5c9ab7250 con 0x55a5c9d11e00 2023-07-18T03:08:28.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.039+0000 7f82fcb9b700 1 --2- 172.21.15.101:0/880313558 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a5c9d114f0 0x55a5c9d1b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:08:28.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.039+0000 7f82f7fff700 1 --2- 172.21.15.101:0/880313558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5c9d11e00 0x55a5c9d180f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x55a5c9d19ee0 tx=0x7f82e4009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf5a59880bf18c6e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:08:28.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.039+0000 7f82e37fe700 1 -- 172.21.15.101:0/880313558 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e401d030 con 0x55a5c9d11e00 2023-07-18T03:08:28.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.039+0000 7f82e37fe700 1 -- 172.21.15.101:0/880313558 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82e401e040 con 0x55a5c9d11e00 2023-07-18T03:08:28.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.039+0000 7f82e37fe700 1 -- 172.21.15.101:0/880313558 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e4004500 con 0x55a5c9d11e00 2023-07-18T03:08:28.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.040+0000 7f8312704b80 1 -- 172.21.15.101:0/880313558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5c9d11e00 msgr2=0x55a5c9d180f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:28.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.040+0000 7f8312704b80 1 --2- 172.21.15.101:0/880313558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5c9d11e00 0x55a5c9d180f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x55a5c9d19ee0 tx=0x7f82e4009310 comp rx=0 tx=0).stop 2023-07-18T03:08:28.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.040+0000 7f8312704b80 1 -- 172.21.15.101:0/880313558 shutdown_connections 2023-07-18T03:08:28.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.040+0000 7f8312704b80 1 --2- 172.21.15.101:0/880313558 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5c9d1bd30 0x55a5c9d18630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:28.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.040+0000 7f8312704b80 1 --2- 172.21.15.101:0/880313558 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a5c9d114f0 0x55a5c9d1b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:28.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.040+0000 7f8312704b80 1 --2- 172.21.15.101:0/880313558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5c9d11e00 0x55a5c9d180f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:28.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.040+0000 7f8312704b80 1 -- 172.21.15.101:0/880313558 >> 172.21.15.101:0/880313558 conn(0x55a5c9d06ff0 msgr2=0x55a5c9d27980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:08:28.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.041+0000 7f8312704b80 1 -- 172.21.15.101:0/880313558 shutdown_connections 2023-07-18T03:08:28.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.041+0000 7f8312704b80 1 -- 172.21.15.101:0/880313558 wait complete. 2023-07-18T03:08:28.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.041+0000 7f8312704b80 1 -- start start 2023-07-18T03:08:28.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.042+0000 7f8312704b80 1 Processor -- start 2023-07-18T03:08:28.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.042+0000 7f8312704b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5c9d114f0 0x55a5c9d20f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:28.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.042+0000 7f8312704b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a5c9d11e00 0x55a5c9d214b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:28.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.042+0000 7f8312704b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5c9d1bd30 0x55a5c9d219f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:28.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.042+0000 7f8312704b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5c9d87d20 con 0x55a5c9d11e00 2023-07-18T03:08:28.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.042+0000 7f8312704b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5c9d87e60 con 0x55a5c9d114f0 2023-07-18T03:08:28.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.042+0000 7f8312704b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5c9d88020 con 0x55a5c9d1bd30 2023-07-18T03:08:28.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.042+0000 7f8312704b80 10 client.0 _ll_register_callbacks cb 0x7f82ec099cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:08:28.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.043+0000 7f82fcb9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5c9d114f0 0x55a5c9d20f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:28.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.043+0000 7f82f7fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a5c9d11e00 0x55a5c9d214b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:28.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.043+0000 7f82fcb9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5c9d114f0 0x55a5c9d20f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:48140/0 (socket says 172.21.15.101:48140) 2023-07-18T03:08:28.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.043+0000 7f82fcb9b700 1 -- 172.21.15.101:0/2481490321 learned_addr learned my addr 172.21.15.101:0/2481490321 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:08:28.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.043+0000 7f82fd39c700 1 --2- 172.21.15.101:0/2481490321 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5c9d1bd30 0x55a5c9d219f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:28.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.043+0000 7f82fcb9b700 1 -- 172.21.15.101:0/2481490321 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5c9d1bd30 msgr2=0x55a5c9d219f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:08:28.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.043+0000 7f82fcb9b700 1 --2- 172.21.15.101:0/2481490321 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5c9d1bd30 0x55a5c9d219f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:28.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.043+0000 7f82fcb9b700 1 -- 172.21.15.101:0/2481490321 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a5c9d11e00 msgr2=0x55a5c9d214b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:08:28.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.043+0000 7f82fcb9b700 1 --2- 172.21.15.101:0/2481490321 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a5c9d11e00 0x55a5c9d214b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:28.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.043+0000 7f82fcb9b700 1 -- 172.21.15.101:0/2481490321 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a5c9a79c60 con 0x55a5c9d114f0 2023-07-18T03:08:28.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.044+0000 7f82fcb9b700 1 --2- 172.21.15.101:0/2481490321 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5c9d114f0 0x55a5c9d20f70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00c5b0 tx=0x7f82ec0b8570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad75255295f8c076 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:08:28.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.044+0000 7f82d7fff700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:08:28.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.044+0000 7f82d7fff700 1 -- 172.21.15.101:0/2481490321 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ec0c8070 con 0x55a5c9d114f0 2023-07-18T03:08:28.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.044+0000 7f8312704b80 1 -- 172.21.15.101:0/2481490321 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f82f000f040 con 0x55a5c9d114f0 2023-07-18T03:08:28.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.044+0000 7f82d7fff700 1 -- 172.21.15.101:0/2481490321 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f82ec0cd3b0 con 0x55a5c9d114f0 2023-07-18T03:08:28.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.044+0000 7f82d7fff700 1 -- 172.21.15.101:0/2481490321 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82ec0cd040 con 0x55a5c9d114f0 2023-07-18T03:08:28.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.044+0000 7f8312704b80 10 client.25724 fetch_fsmap learned FSMap version 24 2023-07-18T03:08:28.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.044+0000 7f8312704b80 1 -- 172.21.15.101:0/2481490321 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f82f810d0f0 con 0x55a5c9d114f0 2023-07-18T03:08:28.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.045+0000 7f82d7fff700 1 -- 172.21.15.101:0/2481490321 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82f000f040 con 0x55a5c9d114f0 2023-07-18T03:08:28.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.045+0000 7f82d7fff700 1 -- 172.21.15.101:0/2481490321 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f82ec0ca2b0 con 0x55a5c9d114f0 2023-07-18T03:08:28.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.045+0000 7f8312704b80 10 client.25724 fetch_fsmap finished waiting for FSMap version 24 2023-07-18T03:08:28.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.045+0000 7f8312704b80 10 client.25724 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:08:28.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.045+0000 7f8312704b80 20 client.25724 populate_metadata read hostname 'smithi101' 2023-07-18T03:08:28.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.045+0000 7f8312704b80 1 --2- 172.21.15.101:0/2481490321 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a5c9d89460 0x55a5c9d8b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:28.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.045+0000 7f8312704b80 4 client.25724 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:08:28.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.045+0000 7f8312704b80 1 -- 172.21.15.101:0/2481490321 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a5c9c1e050 con 0x55a5c9d89460 2023-07-18T03:08:28.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.046+0000 7f82f7fff700 1 --2- 172.21.15.101:0/2481490321 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a5c9d89460 0x55a5c9d8b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:28.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.046+0000 7f82f7fff700 1 --2- 172.21.15.101:0/2481490321 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a5c9d89460 0x55a5c9d8b870 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90089da38b2a9b0c server_cookie=ef51146b2ae66f8b in_seq=0 out_seq=0 2023-07-18T03:08:28.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.047+0000 7f82d7fff700 10 client.25724 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:08:28.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.047+0000 7f82d7fff700 1 -- 172.21.15.101:0/2481490321 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a5c9c1e050 con 0x55a5c9d89460 2023-07-18T03:08:28.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.047+0000 7f82d7fff700 10 client.25724 handle_command_reply: tid=0 2023-07-18T03:08:28.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.047+0000 7f8312704b80 1 client.25724 shutdown 2023-07-18T03:08:28.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.047+0000 7f8312704b80 1 -- 172.21.15.101:0/2481490321 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5c9d114f0 msgr2=0x55a5c9d20f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:28.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.047+0000 7f8312704b80 1 --2- 172.21.15.101:0/2481490321 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5c9d114f0 0x55a5c9d20f70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00c5b0 tx=0x7f82ec0b8570 comp rx=0 tx=0).stop 2023-07-18T03:08:28.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.047+0000 7f8312704b80 1 -- 172.21.15.101:0/2481490321 shutdown_connections 2023-07-18T03:08:28.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.047+0000 7f8312704b80 1 --2- 172.21.15.101:0/2481490321 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a5c9d89460 0x55a5c9d8b870 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:28.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.047+0000 7f8312704b80 1 --2- 172.21.15.101:0/2481490321 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5c9d114f0 0x55a5c9d20f70 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:28.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.047+0000 7f8312704b80 1 --2- 172.21.15.101:0/2481490321 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5c9d1bd30 0x55a5c9d219f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:28.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.047+0000 7f8312704b80 1 --2- 172.21.15.101:0/2481490321 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a5c9d11e00 0x55a5c9d214b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:28.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.047+0000 7f8312704b80 1 -- 172.21.15.101:0/2481490321 >> 172.21.15.101:0/2481490321 conn(0x55a5c9d06ff0 msgr2=0x55a5c9d10440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:08:28.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.047+0000 7f82d7fff700 0 client.25724 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:08:28.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.047+0000 7f8312704b80 1 -- 172.21.15.101:0/2481490321 shutdown_connections 2023-07-18T03:08:28.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.048+0000 7f8312704b80 1 -- 172.21.15.101:0/2481490321 wait complete. 2023-07-18T03:08:28.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.048+0000 7f8312704b80 20 client.25724 trim_cache size 0 max 16384 2023-07-18T03:08:28.051 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:08:28.051 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "no active scrubs running", 2023-07-18T03:08:28.051 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": {} 2023-07-18T03:08:28.052 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:08:28.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.048+0000 7f82fdb9d700 1 -- 172.21.15.101:0/54362199 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f82e80a1500 msgr2=0x7f82e80a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:28.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.048+0000 7f82fdb9d700 1 --2- 172.21.15.101:0/54362199 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f82e80a1500 0x7f82e80a39b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f82f00094f0 tx=0x7f82f001f890 comp rx=0 tx=0).stop 2023-07-18T03:08:28.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.048+0000 7f82fdb9d700 1 -- 172.21.15.101:0/54362199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82f8100aa0 msgr2=0x7f82f80fe8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:28.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.048+0000 7f82fdb9d700 1 --2- 172.21.15.101:0/54362199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82f8100aa0 0x7f82f80fe8d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00e9c0 tx=0x7f82ec00ecd0 comp rx=0 tx=0).stop 2023-07-18T03:08:28.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.049+0000 7f82fdb9d700 1 -- 172.21.15.101:0/54362199 shutdown_connections 2023-07-18T03:08:28.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.049+0000 7f82fdb9d700 1 --2- 172.21.15.101:0/54362199 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f82e80a1500 0x7f82e80a39b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:28.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.049+0000 7f82fdb9d700 1 --2- 172.21.15.101:0/54362199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82f8100aa0 0x7f82f80fe8d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:28.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.049+0000 7f82fdb9d700 1 --2- 172.21.15.101:0/54362199 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f82f8109fc0 0x7f82f81a3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:28.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.049+0000 7f82fdb9d700 1 --2- 172.21.15.101:0/54362199 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82f81013b0 0x7f82f80fee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:28.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.049+0000 7f82fdb9d700 1 -- 172.21.15.101:0/54362199 >> 172.21.15.101:0/54362199 conn(0x7f82f80745d0 msgr2=0x7f82f80fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:08:28.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.049+0000 7f82fdb9d700 1 -- 172.21.15.101:0/54362199 shutdown_connections 2023-07-18T03:08:28.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.049+0000 7f82fdb9d700 1 -- 172.21.15.101:0/54362199 wait complete. 2023-07-18T03:08:28.065 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-18T03:08:28.066 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:08:28.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.409+0000 7f14f27ff700 1 -- 172.21.15.101:0/3389104459 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f14ec0731c0 msgr2=0x7f14ec10bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:28.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.409+0000 7f14f27ff700 1 --2- 172.21.15.101:0/3389104459 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f14ec0731c0 0x7f14ec10bdd0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f14e0009990 tx=0x7f14e0009ca0 comp rx=0 tx=0).stop 2023-07-18T03:08:28.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.410+0000 7f14f27ff700 1 -- 172.21.15.101:0/3389104459 shutdown_connections 2023-07-18T03:08:28.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.410+0000 7f14f27ff700 1 --2- 172.21.15.101:0/3389104459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14ec10c310 0x7f14ec10e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:28.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.410+0000 7f14f27ff700 1 --2- 172.21.15.101:0/3389104459 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14ec0728b0 0x7f14ec072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:28.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.410+0000 7f14f27ff700 1 --2- 172.21.15.101:0/3389104459 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f14ec0731c0 0x7f14ec10bdd0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:28.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.410+0000 7f14f27ff700 1 -- 172.21.15.101:0/3389104459 >> 172.21.15.101:0/3389104459 conn(0x7f14ec0fb590 msgr2=0x7f14ec0fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:08:28.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.410+0000 7f14f27ff700 1 -- 172.21.15.101:0/3389104459 shutdown_connections 2023-07-18T03:08:28.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.410+0000 7f14f27ff700 1 -- 172.21.15.101:0/3389104459 wait complete. 2023-07-18T03:08:28.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.411+0000 7f14f27ff700 1 Processor -- start 2023-07-18T03:08:28.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.412+0000 7f14f27ff700 1 -- start start 2023-07-18T03:08:28.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.412+0000 7f14f27ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f14ec0728b0 0x7f14ec197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:28.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.412+0000 7f14f27ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14ec0731c0 0x7f14ec1982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:28.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.412+0000 7f14f27ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14ec10c310 0x7f14ec19c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:28.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.412+0000 7f14f27ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec19cd80 con 0x7f14ec0728b0 2023-07-18T03:08:28.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.412+0000 7f14f27ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec19cef0 con 0x7f14ec10c310 2023-07-18T03:08:28.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.412+0000 7f14f27ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec19d0b0 con 0x7f14ec0731c0 2023-07-18T03:08:28.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.413+0000 7f14eb7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14ec0731c0 0x7f14ec1982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:28.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.413+0000 7f14eb7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14ec0731c0 0x7f14ec1982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:59394/0 (socket says 172.21.15.101:59394) 2023-07-18T03:08:28.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.413+0000 7f14eb7fe700 1 -- 172.21.15.101:0/2971387739 learned_addr learned my addr 172.21.15.101:0/2971387739 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:08:28.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.413+0000 7f14f0d9c700 1 --2- 172.21.15.101:0/2971387739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14ec10c310 0x7f14ec19c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:28.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.413+0000 7f14ebfff700 1 --2- 172.21.15.101:0/2971387739 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f14ec0728b0 0x7f14ec197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:28.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.413+0000 7f14eb7fe700 1 -- 172.21.15.101:0/2971387739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14ec10c310 msgr2=0x7f14ec19c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:28.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.413+0000 7f14eb7fe700 1 --2- 172.21.15.101:0/2971387739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14ec10c310 0x7f14ec19c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:28.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.413+0000 7f14eb7fe700 1 -- 172.21.15.101:0/2971387739 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f14ec0728b0 msgr2=0x7f14ec197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:28.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.413+0000 7f14eb7fe700 1 --2- 172.21.15.101:0/2971387739 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f14ec0728b0 0x7f14ec197da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:28.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.413+0000 7f14eb7fe700 1 -- 172.21.15.101:0/2971387739 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e00095f0 con 0x7f14ec0731c0 2023-07-18T03:08:28.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.413+0000 7f14f0d9c700 1 --2- 172.21.15.101:0/2971387739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14ec10c310 0x7f14ec19c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:08:28.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.413+0000 7f14ebfff700 1 --2- 172.21.15.101:0/2971387739 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f14ec0728b0 0x7f14ec197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:08:28.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.414+0000 7f14eb7fe700 1 --2- 172.21.15.101:0/2971387739 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14ec0731c0 0x7f14ec1982e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f14e0005910 tx=0x7f14e00047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:08:28.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.414+0000 7f14e97fa700 1 -- 172.21.15.101:0/2971387739 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e001d070 con 0x7f14ec0731c0 2023-07-18T03:08:28.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.414+0000 7f14f27ff700 1 -- 172.21.15.101:0/2971387739 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14ec19d330 con 0x7f14ec0731c0 2023-07-18T03:08:28.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.414+0000 7f14e97fa700 1 -- 172.21.15.101:0/2971387739 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14e0022470 con 0x7f14ec0731c0 2023-07-18T03:08:28.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.414+0000 7f14e97fa700 1 -- 172.21.15.101:0/2971387739 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e0017650 con 0x7f14ec0731c0 2023-07-18T03:08:28.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.414+0000 7f14f27ff700 1 -- 172.21.15.101:0/2971387739 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14ec1a4bd0 con 0x7f14ec0731c0 2023-07-18T03:08:28.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.416+0000 7f14f27ff700 1 -- 172.21.15.101:0/2971387739 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14ec106c30 con 0x7f14ec0731c0 2023-07-18T03:08:28.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.419+0000 7f14e97fa700 1 -- 172.21.15.101:0/2971387739 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 24) v1 ==== 135949+0+0 (secure 0 0 0) 0x7f14e00225e0 con 0x7f14ec0731c0 2023-07-18T03:08:28.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.421+0000 7f14e97fa700 1 --2- 172.21.15.101:0/2971387739 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f14d40a14b0 0x7f14d40a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:28.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.422+0000 7f14e97fa700 1 -- 172.21.15.101:0/2971387739 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f14e00ce650 con 0x7f14ec0731c0 2023-07-18T03:08:28.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.422+0000 7f14ebfff700 1 --2- 172.21.15.101:0/2971387739 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f14d40a14b0 0x7f14d40a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:28.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.423+0000 7f14ebfff700 1 --2- 172.21.15.101:0/2971387739 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f14d40a14b0 0x7f14d40a3960 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f14dc009cb0 tx=0x7f14dc009260 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:08:28.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.425+0000 7f14e97fa700 1 -- 172.21.15.101:0/2971387739 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f14e005c790 con 0x7f14ec0731c0 2023-07-18T03:08:28.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.913+0000 7f14f27ff700 1 -- 172.21.15.101:0/2971387739 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14ec067300 con 0x7f14ec0731c0 2023-07-18T03:08:28.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.914+0000 7f14e97fa700 1 -- 172.21.15.101:0/2971387739 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10858 (secure 0 0 0) 0x7f14e00974f0 con 0x7f14ec0731c0 2023-07-18T03:08:28.903 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:08:28.903 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/4192846769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":4192846769},{"type":"v1","addr":"172.21.15.101:6839","nonce":4192846769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6841/3824340175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6840","nonce":3824340175},{"type":"v1","addr":"172.21.15.101:6841","nonce":3824340175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:04:59.361185+0000","modified":"2023-07-18T03:05:31.985959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.155:6839/2055462483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2055462483},{"type":"v1","addr":"172.21.15.155:6839","nonce":2055462483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:08:28.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.917+0000 7f14f27ff700 1 -- 172.21.15.101:0/2971387739 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f14d40a14b0 msgr2=0x7f14d40a3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:28.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.917+0000 7f14f27ff700 1 --2- 172.21.15.101:0/2971387739 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f14d40a14b0 0x7f14d40a3960 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f14dc009cb0 tx=0x7f14dc009260 comp rx=0 tx=0).stop 2023-07-18T03:08:28.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.917+0000 7f14f27ff700 1 -- 172.21.15.101:0/2971387739 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14ec0731c0 msgr2=0x7f14ec1982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:28.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.917+0000 7f14f27ff700 1 --2- 172.21.15.101:0/2971387739 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14ec0731c0 0x7f14ec1982e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f14e0005910 tx=0x7f14e00047c0 comp rx=0 tx=0).stop 2023-07-18T03:08:28.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.917+0000 7f14f27ff700 1 -- 172.21.15.101:0/2971387739 shutdown_connections 2023-07-18T03:08:28.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.917+0000 7f14f27ff700 1 --2- 172.21.15.101:0/2971387739 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f14d40a14b0 0x7f14d40a3960 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:28.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.917+0000 7f14f27ff700 1 --2- 172.21.15.101:0/2971387739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14ec10c310 0x7f14ec19c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:28.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.917+0000 7f14f27ff700 1 --2- 172.21.15.101:0/2971387739 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f14ec0728b0 0x7f14ec197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:28.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.917+0000 7f14f27ff700 1 --2- 172.21.15.101:0/2971387739 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14ec0731c0 0x7f14ec1982e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:28.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.917+0000 7f14f27ff700 1 -- 172.21.15.101:0/2971387739 >> 172.21.15.101:0/2971387739 conn(0x7f14ec0fb590 msgr2=0x7f14ec103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:08:28.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.917+0000 7f14f27ff700 1 -- 172.21.15.101:0/2971387739 shutdown_connections 2023-07-18T03:08:28.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:28.917+0000 7f14f27ff700 1 -- 172.21.15.101:0/2971387739 wait complete. 2023-07-18T03:08:28.908 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 24 2023-07-18T03:08:28.936 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-18T03:08:29.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:08:28 smithi179 ceph-mon[129229]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:08:29.233 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:08:28 smithi101 ceph-mon[126379]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:08:29.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.281+0000 7f353559d700 1 -- 172.21.15.101:0/2430212429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3530100aa0 msgr2=0x7f3530100e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:29.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.281+0000 7f353559d700 1 --2- 172.21.15.101:0/2430212429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3530100aa0 0x7f3530100e70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3518009990 tx=0x7f3518009ca0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.282+0000 7f353559d700 1 -- 172.21.15.101:0/2430212429 shutdown_connections 2023-07-18T03:08:29.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.282+0000 7f353559d700 1 --2- 172.21.15.101:0/2430212429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3530100aa0 0x7f3530100e70 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.282+0000 7f353559d700 1 --2- 172.21.15.101:0/2430212429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3530109fc0 0x7f353010c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.282+0000 7f353559d700 1 --2- 172.21.15.101:0/2430212429 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f35301013b0 0x7f3530109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.282+0000 7f353559d700 1 -- 172.21.15.101:0/2430212429 >> 172.21.15.101:0/2430212429 conn(0x7f35300745d0 msgr2=0x7f35300749d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:08:29.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.282+0000 7f353559d700 1 -- 172.21.15.101:0/2430212429 shutdown_connections 2023-07-18T03:08:29.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.282+0000 7f353559d700 1 -- 172.21.15.101:0/2430212429 wait complete. 2023-07-18T03:08:29.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.284+0000 7f353559d700 1 Processor -- start 2023-07-18T03:08:29.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.284+0000 7f353559d700 1 -- start start 2023-07-18T03:08:29.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.284+0000 7f353559d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3530100aa0 0x7f35300fe8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:29.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.284+0000 7f353559d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35301013b0 0x7f35300fee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:29.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.284+0000 7f353559d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3530109fc0 0x7f3530070760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:29.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.284+0000 7f353559d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35300ff680 con 0x7f3530100aa0 2023-07-18T03:08:29.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.284+0000 7f353559d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3530070ca0 con 0x7f3530109fc0 2023-07-18T03:08:29.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.284+0000 7f353559d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3530070e60 con 0x7f35301013b0 2023-07-18T03:08:29.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.285+0000 7f352ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3530100aa0 0x7f35300fe8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:29.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.285+0000 7f352ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3530100aa0 0x7f35300fe8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:56620/0 (socket says 172.21.15.101:56620) 2023-07-18T03:08:29.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.285+0000 7f352ffff700 1 -- 172.21.15.101:0/1800546498 learned_addr learned my addr 172.21.15.101:0/1800546498 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:08:29.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.285+0000 7f3534d9c700 1 --2- 172.21.15.101:0/1800546498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3530109fc0 0x7f3530070760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:29.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.285+0000 7f352ffff700 1 -- 172.21.15.101:0/1800546498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35301013b0 msgr2=0x7f35300fee30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-18T03:08:29.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.285+0000 7f352ffff700 1 --2- 172.21.15.101:0/1800546498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35301013b0 0x7f35300fee30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.285+0000 7f352ffff700 1 -- 172.21.15.101:0/1800546498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3530109fc0 msgr2=0x7f3530070760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:29.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.286+0000 7f352ffff700 1 --2- 172.21.15.101:0/1800546498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3530109fc0 0x7f3530070760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.286+0000 7f352ffff700 1 -- 172.21.15.101:0/1800546498 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35180095f0 con 0x7f3530100aa0 2023-07-18T03:08:29.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.286+0000 7f3534d9c700 1 --2- 172.21.15.101:0/1800546498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3530109fc0 0x7f3530070760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:08:29.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.286+0000 7f352ffff700 1 --2- 172.21.15.101:0/1800546498 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3530100aa0 0x7f35300fe8f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3518005740 tx=0x7f3518005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:08:29.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.286+0000 7f352d7fa700 1 -- 172.21.15.101:0/1800546498 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351801d070 con 0x7f3530100aa0 2023-07-18T03:08:29.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.286+0000 7f352d7fa700 1 -- 172.21.15.101:0/1800546498 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3518022470 con 0x7f3530100aa0 2023-07-18T03:08:29.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.286+0000 7f353559d700 1 -- 172.21.15.101:0/1800546498 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35300710e0 con 0x7f3530100aa0 2023-07-18T03:08:29.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.287+0000 7f352d7fa700 1 -- 172.21.15.101:0/1800546498 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3518017650 con 0x7f3530100aa0 2023-07-18T03:08:29.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.288+0000 7f353559d700 1 -- 172.21.15.101:0/1800546498 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35300715e0 con 0x7f3530100aa0 2023-07-18T03:08:29.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.292+0000 7f352d7fa700 1 -- 172.21.15.101:0/1800546498 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 24) v1 ==== 135949+0+0 (secure 0 0 0) 0x7f35180228c0 con 0x7f3530100aa0 2023-07-18T03:08:29.284 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.294+0000 7f354a069b80 1 Processor -- start 2023-07-18T03:08:29.284 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.294+0000 7f354a069b80 1 -- start start 2023-07-18T03:08:29.284 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.295+0000 7f354a069b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55930e5524f0 0x55930e5528c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:29.284 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.295+0000 7f354a069b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55930e552e00 0x55930e55c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:29.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.295+0000 7f354a069b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55930e55cd30 0x55930e55f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:29.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.295+0000 7f354a069b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55930e457cb0 con 0x55930e55cd30 2023-07-18T03:08:29.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.295+0000 7f354a069b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55930e4aaa40 con 0x55930e552e00 2023-07-18T03:08:29.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.295+0000 7f354a069b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55930e4a3ed0 con 0x55930e5524f0 2023-07-18T03:08:29.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.295+0000 7f352d7fa700 1 --2- 172.21.15.101:0/1800546498 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f351c0a1500 0x7f351c0a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:29.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.295+0000 7f3534d9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55930e5524f0 0x55930e5528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:29.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.295+0000 7f3534d9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55930e5524f0 0x55930e5528c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:59426/0 (socket says 172.21.15.101:59426) 2023-07-18T03:08:29.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.295+0000 7f3534d9c700 1 -- 172.21.15.101:0/1774471563 learned_addr learned my addr 172.21.15.101:0/1774471563 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:08:29.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.295+0000 7f352d7fa700 1 -- 172.21.15.101:0/1800546498 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f35180ce3b0 con 0x7f3530100aa0 2023-07-18T03:08:29.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.295+0000 7f352f7fe700 1 --2- 172.21.15.101:0/1774471563 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55930e55cd30 0x55930e55f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:29.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.296+0000 7f352ffff700 1 --2- 172.21.15.101:0/1774471563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55930e552e00 0x55930e55c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:29.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.296+0000 7f352f7fe700 1 -- 172.21.15.101:0/1774471563 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55930e5524f0 msgr2=0x55930e5528c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:08:29.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.296+0000 7f352f7fe700 1 --2- 172.21.15.101:0/1774471563 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55930e5524f0 0x55930e5528c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.296+0000 7f352f7fe700 1 -- 172.21.15.101:0/1774471563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55930e552e00 msgr2=0x55930e55c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:08:29.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.296+0000 7f352f7fe700 1 --2- 172.21.15.101:0/1774471563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55930e552e00 0x55930e55c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.296+0000 7f352f7fe700 1 -- 172.21.15.101:0/1774471563 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55930e2d1de0 con 0x55930e55cd30 2023-07-18T03:08:29.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.296+0000 7f3534d9c700 1 --2- 172.21.15.101:0/1774471563 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55930e5524f0 0x55930e5528c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:08:29.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.296+0000 7f352ffff700 1 --2- 172.21.15.101:0/1774471563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55930e552e00 0x55930e55c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:08:29.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.296+0000 7f3534d9c700 1 --2- 172.21.15.101:0/1800546498 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f351c0a1500 0x7f351c0a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:29.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.296+0000 7f352f7fe700 1 --2- 172.21.15.101:0/1774471563 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55930e55cd30 0x55930e55f110 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3520009b30 tx=0x7f3520009e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fc695644dd9fecc server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:08:29.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.297+0000 7f35267fc700 1 -- 172.21.15.101:0/1774471563 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3520004230 con 0x55930e55cd30 2023-07-18T03:08:29.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.297+0000 7f35267fc700 1 -- 172.21.15.101:0/1774471563 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3520004cf0 con 0x55930e55cd30 2023-07-18T03:08:29.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.297+0000 7f35267fc700 1 -- 172.21.15.101:0/1774471563 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f352000eeb0 con 0x55930e55cd30 2023-07-18T03:08:29.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.298+0000 7f354a069b80 1 -- 172.21.15.101:0/1774471563 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55930e55cd30 msgr2=0x55930e55f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:29.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.298+0000 7f354a069b80 1 --2- 172.21.15.101:0/1774471563 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55930e55cd30 0x55930e55f110 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3520009b30 tx=0x7f3520009e40 comp rx=0 tx=0).stop 2023-07-18T03:08:29.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.298+0000 7f3534d9c700 1 --2- 172.21.15.101:0/1800546498 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f351c0a1500 0x7f351c0a39b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3528003fd0 tx=0x7f3528004ad0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:08:29.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.298+0000 7f354a069b80 1 -- 172.21.15.101:0/1774471563 shutdown_connections 2023-07-18T03:08:29.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.298+0000 7f354a069b80 1 --2- 172.21.15.101:0/1774471563 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55930e55cd30 0x55930e55f110 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.298+0000 7f354a069b80 1 --2- 172.21.15.101:0/1774471563 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55930e5524f0 0x55930e5528c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.298+0000 7f354a069b80 1 --2- 172.21.15.101:0/1774471563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55930e552e00 0x55930e55c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.298+0000 7f354a069b80 1 -- 172.21.15.101:0/1774471563 >> 172.21.15.101:0/1774471563 conn(0x55930e547ff0 msgr2=0x55930e54d930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:08:29.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.298+0000 7f354a069b80 1 -- 172.21.15.101:0/1774471563 shutdown_connections 2023-07-18T03:08:29.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.298+0000 7f354a069b80 1 -- 172.21.15.101:0/1774471563 wait complete. 2023-07-18T03:08:29.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.299+0000 7f354a069b80 1 -- start start 2023-07-18T03:08:29.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.300+0000 7f354a069b80 1 Processor -- start 2023-07-18T03:08:29.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.300+0000 7f354a069b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55930e5524f0 0x55930e55c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:29.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.300+0000 7f354a069b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55930e552e00 0x55930e559930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:29.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.300+0000 7f354a069b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55930e55cd30 0x55930e559f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:29.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.300+0000 7f354a069b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55930e443ac0 con 0x55930e5524f0 2023-07-18T03:08:29.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.300+0000 7f354a069b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55930e4437c0 con 0x55930e552e00 2023-07-18T03:08:29.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.300+0000 7f354a069b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55930e327920 con 0x55930e55cd30 2023-07-18T03:08:29.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.300+0000 7f354a069b80 10 client.0 _ll_register_callbacks cb 0x55930e441760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:08:29.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.301+0000 7f352ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55930e5524f0 0x55930e55c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:29.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.301+0000 7f352ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55930e5524f0 0x55930e55c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:56646/0 (socket says 172.21.15.101:56646) 2023-07-18T03:08:29.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.301+0000 7f352ffff700 1 -- 172.21.15.101:0/621397097 learned_addr learned my addr 172.21.15.101:0/621397097 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:08:29.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.301+0000 7f3534d9c700 1 --2- 172.21.15.101:0/621397097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55930e55cd30 0x55930e559f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:29.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.301+0000 7f352f7fe700 1 --2- 172.21.15.101:0/621397097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55930e552e00 0x55930e559930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:29.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.301+0000 7f352ffff700 1 -- 172.21.15.101:0/621397097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55930e55cd30 msgr2=0x55930e559f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:08:29.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.301+0000 7f352ffff700 1 --2- 172.21.15.101:0/621397097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55930e55cd30 0x55930e559f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.301+0000 7f352ffff700 1 -- 172.21.15.101:0/621397097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55930e552e00 msgr2=0x55930e559930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:08:29.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.301+0000 7f352ffff700 1 --2- 172.21.15.101:0/621397097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55930e552e00 0x55930e559930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.301+0000 7f352ffff700 1 -- 172.21.15.101:0/621397097 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3528003ae0 con 0x55930e5524f0 2023-07-18T03:08:29.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:08:28 smithi155 ceph-mon[128885]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:08:29.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.302+0000 7f352ffff700 1 --2- 172.21.15.101:0/621397097 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55930e5524f0 0x55930e55c6b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3518029000 tx=0x7f351809ae10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed2520d28c91215 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:08:29.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.302+0000 7f350f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:08:29.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.302+0000 7f350f7fe700 1 -- 172.21.15.101:0/621397097 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35180cbea0 con 0x55930e5524f0 2023-07-18T03:08:29.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.302+0000 7f350f7fe700 1 -- 172.21.15.101:0/621397097 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f351809af10 con 0x55930e5524f0 2023-07-18T03:08:29.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.302+0000 7f354a069b80 1 -- 172.21.15.101:0/621397097 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55930e3fd4f0 con 0x55930e5524f0 2023-07-18T03:08:29.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.302+0000 7f350f7fe700 1 -- 172.21.15.101:0/621397097 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f351809ef40 con 0x55930e5524f0 2023-07-18T03:08:29.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.303+0000 7f354a069b80 10 client.15711 fetch_fsmap learned FSMap version 24 2023-07-18T03:08:29.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.303+0000 7f354a069b80 1 -- 172.21.15.101:0/621397097 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f35300715e0 con 0x55930e5524f0 2023-07-18T03:08:29.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.303+0000 7f350f7fe700 1 -- 172.21.15.101:0/621397097 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55930e3fd4f0 con 0x55930e5524f0 2023-07-18T03:08:29.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.303+0000 7f350f7fe700 1 -- 172.21.15.101:0/621397097 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3518089030 con 0x55930e5524f0 2023-07-18T03:08:29.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.303+0000 7f354a069b80 10 client.15711 fetch_fsmap finished waiting for FSMap version 24 2023-07-18T03:08:29.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.303+0000 7f354a069b80 10 client.15711 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:08:29.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.303+0000 7f354a069b80 20 client.15711 populate_metadata read hostname 'smithi101' 2023-07-18T03:08:29.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.303+0000 7f354a069b80 1 --2- 172.21.15.101:0/621397097 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55930e56c310 0x55930e56e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:29.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.303+0000 7f354a069b80 4 client.15711 mds_command: new command op to 24541 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:08:29.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.303+0000 7f354a069b80 1 -- 172.21.15.101:0/621397097 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55930e49ceb0 con 0x55930e56c310 2023-07-18T03:08:29.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.304+0000 7f352f7fe700 1 --2- 172.21.15.101:0/621397097 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55930e56c310 0x55930e56e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:29.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.305+0000 7f352f7fe700 1 --2- 172.21.15.101:0/621397097 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55930e56c310 0x55930e56e6f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9eea0776317a3a4e server_cookie=3b4d825dce1e894d in_seq=0 out_seq=0 2023-07-18T03:08:29.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.305+0000 7f350f7fe700 10 client.15711 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:08:29.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.306+0000 7f350f7fe700 1 -- 172.21.15.101:0/621397097 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55930e49ceb0 con 0x55930e56c310 2023-07-18T03:08:29.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.306+0000 7f350f7fe700 10 client.15711 handle_command_reply: tid=0 2023-07-18T03:08:29.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.306+0000 7f354a069b80 1 client.15711 shutdown 2023-07-18T03:08:29.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.306+0000 7f354a069b80 1 -- 172.21.15.101:0/621397097 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55930e5524f0 msgr2=0x55930e55c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:29.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.306+0000 7f354a069b80 1 --2- 172.21.15.101:0/621397097 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55930e5524f0 0x55930e55c6b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3518029000 tx=0x7f351809ae10 comp rx=0 tx=0).stop 2023-07-18T03:08:29.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.306+0000 7f354a069b80 1 -- 172.21.15.101:0/621397097 shutdown_connections 2023-07-18T03:08:29.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.306+0000 7f354a069b80 1 --2- 172.21.15.101:0/621397097 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55930e56c310 0x55930e56e6f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.306+0000 7f354a069b80 1 --2- 172.21.15.101:0/621397097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55930e55cd30 0x55930e559f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.306+0000 7f354a069b80 1 --2- 172.21.15.101:0/621397097 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55930e5524f0 0x55930e55c6b0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.306+0000 7f354a069b80 1 --2- 172.21.15.101:0/621397097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55930e552e00 0x55930e559930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.306+0000 7f354a069b80 1 -- 172.21.15.101:0/621397097 >> 172.21.15.101:0/621397097 conn(0x55930e547ff0 msgr2=0x55930e54d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:08:29.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.307+0000 7f350f7fe700 0 client.15711 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:08:29.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.307+0000 7f354a069b80 1 -- 172.21.15.101:0/621397097 shutdown_connections 2023-07-18T03:08:29.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.307+0000 7f354a069b80 1 -- 172.21.15.101:0/621397097 wait complete. 2023-07-18T03:08:29.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.307+0000 7f354a069b80 20 client.15711 trim_cache size 0 max 16384 2023-07-18T03:08:29.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.332+0000 7f354a069b80 1 Processor -- start 2023-07-18T03:08:29.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.333+0000 7f354a069b80 1 -- start start 2023-07-18T03:08:29.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.333+0000 7f354a069b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55930e5524f0 0x55930e563840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:29.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.333+0000 7f354a069b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55930e552e00 0x55930e554c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:29.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.333+0000 7f354a069b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55930e55cd30 0x55930e555190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:29.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.333+0000 7f354a069b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55930e5539b0 con 0x55930e55cd30 2023-07-18T03:08:29.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.333+0000 7f354a069b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55930e5541a0 con 0x55930e5524f0 2023-07-18T03:08:29.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.333+0000 7f354a069b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55930e5686f0 con 0x55930e552e00 2023-07-18T03:08:29.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.334+0000 7f352ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55930e5524f0 0x55930e563840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:29.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.334+0000 7f352ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55930e5524f0 0x55930e563840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:48214/0 (socket says 172.21.15.101:48214) 2023-07-18T03:08:29.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.334+0000 7f352ffff700 1 -- 172.21.15.101:0/2956997749 learned_addr learned my addr 172.21.15.101:0/2956997749 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:08:29.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.334+0000 7f352f7fe700 1 --2- 172.21.15.101:0/2956997749 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55930e552e00 0x55930e554c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:29.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.334+0000 7f3534d9c700 1 --2- 172.21.15.101:0/2956997749 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55930e55cd30 0x55930e555190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:29.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.334+0000 7f352ffff700 1 -- 172.21.15.101:0/2956997749 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55930e552e00 msgr2=0x55930e554c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:08:29.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.334+0000 7f352ffff700 1 --2- 172.21.15.101:0/2956997749 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55930e552e00 0x55930e554c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.334+0000 7f352ffff700 1 -- 172.21.15.101:0/2956997749 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55930e55cd30 msgr2=0x55930e555190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:08:29.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.334+0000 7f352ffff700 1 --2- 172.21.15.101:0/2956997749 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55930e55cd30 0x55930e555190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.334+0000 7f352ffff700 1 -- 172.21.15.101:0/2956997749 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3520009810 con 0x55930e5524f0 2023-07-18T03:08:29.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.335+0000 7f352ffff700 1 --2- 172.21.15.101:0/2956997749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55930e5524f0 0x55930e563840 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f351800b2e0 tx=0x7f35180994c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93defeef5aeca2f4 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:08:29.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.335+0000 7f3526ffd700 1 -- 172.21.15.101:0/2956997749 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35180cbea0 con 0x55930e5524f0 2023-07-18T03:08:29.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.335+0000 7f3526ffd700 1 -- 172.21.15.101:0/2956997749 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3518085e90 con 0x55930e5524f0 2023-07-18T03:08:29.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.335+0000 7f3526ffd700 1 -- 172.21.15.101:0/2956997749 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351809ef40 con 0x55930e5524f0 2023-07-18T03:08:29.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.336+0000 7f354a069b80 1 -- 172.21.15.101:0/2956997749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55930e5524f0 msgr2=0x55930e563840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:29.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.336+0000 7f354a069b80 1 --2- 172.21.15.101:0/2956997749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55930e5524f0 0x55930e563840 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f351800b2e0 tx=0x7f35180994c0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.336+0000 7f354a069b80 1 -- 172.21.15.101:0/2956997749 shutdown_connections 2023-07-18T03:08:29.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.336+0000 7f354a069b80 1 --2- 172.21.15.101:0/2956997749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55930e5524f0 0x55930e563840 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.336+0000 7f354a069b80 1 --2- 172.21.15.101:0/2956997749 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55930e55cd30 0x55930e555190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.336+0000 7f354a069b80 1 --2- 172.21.15.101:0/2956997749 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55930e552e00 0x55930e554c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.336+0000 7f354a069b80 1 -- 172.21.15.101:0/2956997749 >> 172.21.15.101:0/2956997749 conn(0x55930e547ff0 msgr2=0x55930e567fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:08:29.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.336+0000 7f354a069b80 1 -- 172.21.15.101:0/2956997749 shutdown_connections 2023-07-18T03:08:29.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.336+0000 7f354a069b80 1 -- 172.21.15.101:0/2956997749 wait complete. 2023-07-18T03:08:29.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.337+0000 7f354a069b80 1 -- start start 2023-07-18T03:08:29.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.337+0000 7f354a069b80 1 Processor -- start 2023-07-18T03:08:29.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.338+0000 7f354a069b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55930e5524f0 0x55930e561f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:29.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.338+0000 7f354a069b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55930e552e00 0x55930e5624c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:29.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.338+0000 7f354a069b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55930e55cd30 0x55930e562a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:29.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.338+0000 7f354a069b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55930e5c0200 con 0x55930e5524f0 2023-07-18T03:08:29.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.338+0000 7f354a069b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55930e5c0340 con 0x55930e552e00 2023-07-18T03:08:29.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.338+0000 7f354a069b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55930e5c0500 con 0x55930e55cd30 2023-07-18T03:08:29.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.338+0000 7f354a069b80 10 client.0 _ll_register_callbacks cb 0x7f3518097250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:08:29.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.338+0000 7f352ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55930e5524f0 0x55930e561f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:29.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.338+0000 7f352ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55930e5524f0 0x55930e561f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:56660/0 (socket says 172.21.15.101:56660) 2023-07-18T03:08:29.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.338+0000 7f352ffff700 1 -- 172.21.15.101:0/3283616670 learned_addr learned my addr 172.21.15.101:0/3283616670 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:08:29.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.338+0000 7f352f7fe700 1 --2- 172.21.15.101:0/3283616670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55930e552e00 0x55930e5624c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:29.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.338+0000 7f3534d9c700 1 --2- 172.21.15.101:0/3283616670 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55930e55cd30 0x55930e562a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:29.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.339+0000 7f352ffff700 1 -- 172.21.15.101:0/3283616670 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55930e55cd30 msgr2=0x55930e562a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:08:29.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.339+0000 7f352ffff700 1 --2- 172.21.15.101:0/3283616670 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55930e55cd30 0x55930e562a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.339+0000 7f352ffff700 1 -- 172.21.15.101:0/3283616670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55930e552e00 msgr2=0x55930e5624c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:08:29.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.339+0000 7f352ffff700 1 --2- 172.21.15.101:0/3283616670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55930e552e00 0x55930e5624c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.339+0000 7f352ffff700 1 -- 172.21.15.101:0/3283616670 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35300715e0 con 0x55930e5524f0 2023-07-18T03:08:29.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.339+0000 7f352ffff700 1 --2- 172.21.15.101:0/3283616670 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55930e5524f0 0x55930e561f80 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3518097980 tx=0x7f3518005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d964cec96cf8523a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:08:29.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.339+0000 7f350f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:08:29.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.339+0000 7f350f7fe700 1 -- 172.21.15.101:0/3283616670 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35180cbea0 con 0x55930e5524f0 2023-07-18T03:08:29.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.339+0000 7f350f7fe700 1 -- 172.21.15.101:0/3283616670 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f3518094030 con 0x55930e5524f0 2023-07-18T03:08:29.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.339+0000 7f354a069b80 1 -- 172.21.15.101:0/3283616670 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3520004cf0 con 0x55930e5524f0 2023-07-18T03:08:29.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.339+0000 7f350f7fe700 1 -- 172.21.15.101:0/3283616670 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f351809a6c0 con 0x55930e5524f0 2023-07-18T03:08:29.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.340+0000 7f354a069b80 10 client.15717 fetch_fsmap learned FSMap version 24 2023-07-18T03:08:29.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.340+0000 7f354a069b80 1 -- 172.21.15.101:0/3283616670 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55930e2bac60 con 0x55930e5524f0 2023-07-18T03:08:29.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.340+0000 7f350f7fe700 1 -- 172.21.15.101:0/3283616670 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3520004cf0 con 0x55930e5524f0 2023-07-18T03:08:29.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.340+0000 7f350f7fe700 1 -- 172.21.15.101:0/3283616670 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3518089070 con 0x55930e5524f0 2023-07-18T03:08:29.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.340+0000 7f354a069b80 10 client.15717 fetch_fsmap finished waiting for FSMap version 24 2023-07-18T03:08:29.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.340+0000 7f354a069b80 10 client.15717 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:08:29.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.340+0000 7f354a069b80 20 client.15717 populate_metadata read hostname 'smithi101' 2023-07-18T03:08:29.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.340+0000 7f354a069b80 1 --2- 172.21.15.101:0/3283616670 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55930e5c1750 0x55930e5c3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:29.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.340+0000 7f354a069b80 4 client.15717 mds_command: new command op to 24541 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:08:29.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.340+0000 7f354a069b80 1 -- 172.21.15.101:0/3283616670 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55930e45f050 con 0x55930e5c1750 2023-07-18T03:08:29.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.341+0000 7f352f7fe700 1 --2- 172.21.15.101:0/3283616670 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55930e5c1750 0x55930e5c3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:29.339 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.341+0000 7f350f7fe700 10 client.15717 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:08:29.339 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.341+0000 7f352f7fe700 1 --2- 172.21.15.101:0/3283616670 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55930e5c1750 0x55930e5c3b30 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d8db6f837981f571 server_cookie=57039130fa556b04 in_seq=0 out_seq=0 2023-07-18T03:08:29.339 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.342+0000 7f350f7fe700 1 -- 172.21.15.101:0/3283616670 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55930e45f050 con 0x55930e5c1750 2023-07-18T03:08:29.339 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.342+0000 7f350f7fe700 10 client.15717 handle_command_reply: tid=0 2023-07-18T03:08:29.339 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.342+0000 7f354a069b80 1 client.15717 shutdown 2023-07-18T03:08:29.339 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.342+0000 7f354a069b80 1 -- 172.21.15.101:0/3283616670 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55930e5524f0 msgr2=0x55930e561f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:29.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.342+0000 7f354a069b80 1 --2- 172.21.15.101:0/3283616670 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55930e5524f0 0x55930e561f80 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3518097980 tx=0x7f3518005e50 comp rx=0 tx=0).stop 2023-07-18T03:08:29.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.342+0000 7f354a069b80 1 -- 172.21.15.101:0/3283616670 shutdown_connections 2023-07-18T03:08:29.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.342+0000 7f354a069b80 1 --2- 172.21.15.101:0/3283616670 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55930e5c1750 0x55930e5c3b30 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.342+0000 7f354a069b80 1 --2- 172.21.15.101:0/3283616670 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55930e55cd30 0x55930e562a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.342+0000 7f354a069b80 1 --2- 172.21.15.101:0/3283616670 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55930e5524f0 0x55930e561f80 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.342+0000 7f354a069b80 1 --2- 172.21.15.101:0/3283616670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55930e552e00 0x55930e5624c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.342+0000 7f354a069b80 1 -- 172.21.15.101:0/3283616670 >> 172.21.15.101:0/3283616670 conn(0x55930e547ff0 msgr2=0x55930e554c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:08:29.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.342+0000 7f350f7fe700 0 client.15717 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:08:29.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.342+0000 7f354a069b80 1 -- 172.21.15.101:0/3283616670 shutdown_connections 2023-07-18T03:08:29.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.342+0000 7f354a069b80 1 -- 172.21.15.101:0/3283616670 wait complete. 2023-07-18T03:08:29.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.343+0000 7f354a069b80 20 client.15717 trim_cache size 0 max 16384 2023-07-18T03:08:29.341 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:08:29.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.343+0000 7f353559d700 1 -- 172.21.15.101:0/1800546498 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f351c0a1500 msgr2=0x7f351c0a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:29.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.343+0000 7f353559d700 1 --2- 172.21.15.101:0/1800546498 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f351c0a1500 0x7f351c0a39b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3528003fd0 tx=0x7f3528004ad0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.343+0000 7f353559d700 1 -- 172.21.15.101:0/1800546498 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3530100aa0 msgr2=0x7f35300fe8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:29.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.343+0000 7f353559d700 1 --2- 172.21.15.101:0/1800546498 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3530100aa0 0x7f35300fe8f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3518005740 tx=0x7f3518005770 comp rx=0 tx=0).stop 2023-07-18T03:08:29.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.344+0000 7f353559d700 1 -- 172.21.15.101:0/1800546498 shutdown_connections 2023-07-18T03:08:29.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.344+0000 7f353559d700 1 --2- 172.21.15.101:0/1800546498 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f351c0a1500 0x7f351c0a39b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.344+0000 7f353559d700 1 --2- 172.21.15.101:0/1800546498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3530109fc0 0x7f3530070760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.344+0000 7f353559d700 1 --2- 172.21.15.101:0/1800546498 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3530100aa0 0x7f35300fe8f0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.344+0000 7f353559d700 1 --2- 172.21.15.101:0/1800546498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35301013b0 0x7f35300fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.344+0000 7f353559d700 1 -- 172.21.15.101:0/1800546498 >> 172.21.15.101:0/1800546498 conn(0x7f35300745d0 msgr2=0x7f3530103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:08:29.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.344+0000 7f353559d700 1 -- 172.21.15.101:0/1800546498 shutdown_connections 2023-07-18T03:08:29.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.344+0000 7f353559d700 1 -- 172.21.15.101:0/1800546498 wait complete. 2023-07-18T03:08:29.361 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-18T03:08:29.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.708+0000 7f067dbff700 1 -- 172.21.15.101:0/1089819634 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06780731c0 msgr2=0x7f067810bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:29.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.708+0000 7f067dbff700 1 --2- 172.21.15.101:0/1089819634 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06780731c0 0x7f067810bdd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f066c009990 tx=0x7f066c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.708+0000 7f067dbff700 1 -- 172.21.15.101:0/1089819634 shutdown_connections 2023-07-18T03:08:29.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.708+0000 7f067dbff700 1 --2- 172.21.15.101:0/1089819634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06780728b0 0x7f0678072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.708+0000 7f067dbff700 1 --2- 172.21.15.101:0/1089819634 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f067810c310 0x7f067810e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.708+0000 7f067dbff700 1 --2- 172.21.15.101:0/1089819634 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06780731c0 0x7f067810bdd0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.708+0000 7f067dbff700 1 -- 172.21.15.101:0/1089819634 >> 172.21.15.101:0/1089819634 conn(0x7f06780fb5f0 msgr2=0x7f06780fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:08:29.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.708+0000 7f067dbff700 1 -- 172.21.15.101:0/1089819634 shutdown_connections 2023-07-18T03:08:29.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.708+0000 7f067dbff700 1 -- 172.21.15.101:0/1089819634 wait complete. 2023-07-18T03:08:29.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.710+0000 7f067dbff700 1 Processor -- start 2023-07-18T03:08:29.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.710+0000 7f067dbff700 1 -- start start 2023-07-18T03:08:29.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.710+0000 7f067dbff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06780728b0 0x7f0678197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:29.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.710+0000 7f067dbff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f06780731c0 0x7f0678198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:29.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.710+0000 7f067dbff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f067810c310 0x7f067819c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:29.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.710+0000 7f067dbff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067819cdd0 con 0x7f06780731c0 2023-07-18T03:08:29.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.710+0000 7f067dbff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067819cf40 con 0x7f06780728b0 2023-07-18T03:08:29.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.710+0000 7f067dbff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067819d100 con 0x7f067810c310 2023-07-18T03:08:29.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.711+0000 7f0676ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f06780731c0 0x7f0678198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:29.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.711+0000 7f0676ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f06780731c0 0x7f0678198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:56682/0 (socket says 172.21.15.101:56682) 2023-07-18T03:08:29.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.711+0000 7f0676ffd700 1 -- 172.21.15.101:0/2420192530 learned_addr learned my addr 172.21.15.101:0/2420192530 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:08:29.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.711+0000 7f06777fe700 1 --2- 172.21.15.101:0/2420192530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06780728b0 0x7f0678197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:29.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.711+0000 7f0677fff700 1 --2- 172.21.15.101:0/2420192530 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f067810c310 0x7f067819c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:29.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.712+0000 7f0676ffd700 1 -- 172.21.15.101:0/2420192530 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f067810c310 msgr2=0x7f067819c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:29.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.712+0000 7f0676ffd700 1 --2- 172.21.15.101:0/2420192530 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f067810c310 0x7f067819c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.712+0000 7f0676ffd700 1 -- 172.21.15.101:0/2420192530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06780728b0 msgr2=0x7f0678197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:29.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.712+0000 7f0676ffd700 1 --2- 172.21.15.101:0/2420192530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06780728b0 0x7f0678197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.712+0000 7f0676ffd700 1 -- 172.21.15.101:0/2420192530 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f066c0095f0 con 0x7f06780731c0 2023-07-18T03:08:29.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.712+0000 7f06777fe700 1 --2- 172.21.15.101:0/2420192530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06780728b0 0x7f0678197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:08:29.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.712+0000 7f0676ffd700 1 --2- 172.21.15.101:0/2420192530 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f06780731c0 0x7f0678198330 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f066c005910 tx=0x7f066c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:08:29.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.713+0000 7f0674ff9700 1 -- 172.21.15.101:0/2420192530 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c01d070 con 0x7f06780731c0 2023-07-18T03:08:29.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.713+0000 7f0674ff9700 1 -- 172.21.15.101:0/2420192530 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f066c022470 con 0x7f06780731c0 2023-07-18T03:08:29.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.713+0000 7f067dbff700 1 -- 172.21.15.101:0/2420192530 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f067819d380 con 0x7f06780731c0 2023-07-18T03:08:29.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.713+0000 7f0674ff9700 1 -- 172.21.15.101:0/2420192530 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c017650 con 0x7f06780731c0 2023-07-18T03:08:29.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.713+0000 7f067dbff700 1 -- 172.21.15.101:0/2420192530 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06781a4c20 con 0x7f06780731c0 2023-07-18T03:08:29.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.718+0000 7f0674ff9700 1 -- 172.21.15.101:0/2420192530 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 24) v1 ==== 135949+0+0 (secure 0 0 0) 0x7f066c00fad0 con 0x7f06780731c0 2023-07-18T03:08:29.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.719+0000 7f06914dcb80 1 Processor -- start 2023-07-18T03:08:29.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.719+0000 7f06914dcb80 1 -- start start 2023-07-18T03:08:29.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.720+0000 7f06914dcb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d98f787ae0 0x55d98f787eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:29.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.720+0000 7f06914dcb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d98f7883f0 0x55d98f792da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:29.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.720+0000 7f06914dcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d98f7932e0 0x55d98f7956c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:29.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.720+0000 7f06914dcb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98f685cb0 con 0x55d98f787ae0 2023-07-18T03:08:29.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.720+0000 7f06914dcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98f6d8a40 con 0x55d98f7932e0 2023-07-18T03:08:29.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.720+0000 7f06914dcb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98f6d1ed0 con 0x55d98f7883f0 2023-07-18T03:08:29.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.721+0000 7f0676ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d98f7932e0 0x55d98f7956c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:29.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.721+0000 7f0677fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d98f787ae0 0x55d98f787eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:29.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.721+0000 7f06777fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d98f7883f0 0x55d98f792da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:29.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.721+0000 7f0676ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d98f7932e0 0x55d98f7956c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:48262/0 (socket says 172.21.15.101:48262) 2023-07-18T03:08:29.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.721+0000 7f0676ffd700 1 -- 172.21.15.101:0/3638111476 learned_addr learned my addr 172.21.15.101:0/3638111476 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:08:29.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.721+0000 7f0677fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d98f787ae0 0x55d98f787eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:56690/0 (socket says 172.21.15.101:56690) 2023-07-18T03:08:29.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.721+0000 7f0677fff700 1 -- 172.21.15.101:0/3638111476 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d98f7883f0 msgr2=0x55d98f792da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:08:29.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.721+0000 7f0677fff700 1 --2- 172.21.15.101:0/3638111476 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d98f7883f0 0x55d98f792da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.721+0000 7f0677fff700 1 -- 172.21.15.101:0/3638111476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d98f7932e0 msgr2=0x55d98f7956c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:08:29.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.721+0000 7f0677fff700 1 --2- 172.21.15.101:0/3638111476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d98f7932e0 0x55d98f7956c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.721+0000 7f0677fff700 1 -- 172.21.15.101:0/3638111476 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d98f4ffde0 con 0x55d98f787ae0 2023-07-18T03:08:29.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.722+0000 7f0677fff700 1 --2- 172.21.15.101:0/3638111476 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d98f787ae0 0x55d98f787eb0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0670003e80 tx=0x7f0670009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55c6ebed163bd817 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:08:29.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.722+0000 7f0674ff9700 1 --2- 172.21.15.101:0/2420192530 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f06600a14e0 0x7f06600a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:29.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.722+0000 7f065dffb700 1 -- 172.21.15.101:0/3638111476 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0670019070 con 0x55d98f787ae0 2023-07-18T03:08:29.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.722+0000 7f0674ff9700 1 -- 172.21.15.101:0/2420192530 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f066c0ce750 con 0x7f06780731c0 2023-07-18T03:08:29.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.722+0000 7f065dffb700 1 -- 172.21.15.101:0/3638111476 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f067000f040 con 0x55d98f787ae0 2023-07-18T03:08:29.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.722+0000 7f065dffb700 1 -- 172.21.15.101:0/3638111476 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06700078c0 con 0x55d98f787ae0 2023-07-18T03:08:29.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.723+0000 7f06914dcb80 1 -- 172.21.15.101:0/3638111476 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d98f787ae0 msgr2=0x55d98f787eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:29.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.723+0000 7f06914dcb80 1 --2- 172.21.15.101:0/3638111476 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d98f787ae0 0x55d98f787eb0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0670003e80 tx=0x7f0670009ba0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.723+0000 7f06777fe700 1 --2- 172.21.15.101:0/2420192530 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f06600a14e0 0x7f06600a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:29.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.723+0000 7f06914dcb80 1 -- 172.21.15.101:0/3638111476 shutdown_connections 2023-07-18T03:08:29.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.723+0000 7f06914dcb80 1 --2- 172.21.15.101:0/3638111476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d98f7932e0 0x55d98f7956c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.723+0000 7f06914dcb80 1 --2- 172.21.15.101:0/3638111476 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d98f787ae0 0x55d98f787eb0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.723+0000 7f06914dcb80 1 --2- 172.21.15.101:0/3638111476 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d98f7883f0 0x55d98f792da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.723+0000 7f06914dcb80 1 -- 172.21.15.101:0/3638111476 >> 172.21.15.101:0/3638111476 conn(0x55d98f775da0 msgr2=0x55d98f77b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:08:29.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.724+0000 7f06914dcb80 1 -- 172.21.15.101:0/3638111476 shutdown_connections 2023-07-18T03:08:29.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.724+0000 7f06914dcb80 1 -- 172.21.15.101:0/3638111476 wait complete. 2023-07-18T03:08:29.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.724+0000 7f06777fe700 1 --2- 172.21.15.101:0/2420192530 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f06600a14e0 0x7f06600a3990 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0668013570 tx=0x7f0668009260 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:08:29.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.725+0000 7f06914dcb80 1 -- start start 2023-07-18T03:08:29.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.725+0000 7f06914dcb80 1 Processor -- start 2023-07-18T03:08:29.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.726+0000 7f06914dcb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d98f787ae0 0x55d98f792d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:29.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.726+0000 7f06914dcb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d98f7883f0 0x55d98f790000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:29.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.726+0000 7f06914dcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d98f7932e0 0x55d98f790740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:29.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.726+0000 7f06914dcb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98f671ac0 con 0x55d98f787ae0 2023-07-18T03:08:29.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.726+0000 7f06914dcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98f6717c0 con 0x55d98f7932e0 2023-07-18T03:08:29.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.726+0000 7f06914dcb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98f555920 con 0x55d98f7883f0 2023-07-18T03:08:29.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.726+0000 7f06914dcb80 10 client.0 _ll_register_callbacks cb 0x55d98f66f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:08:29.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.726+0000 7f0677fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d98f7932e0 0x55d98f790740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:29.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.726+0000 7f0676ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d98f7883f0 0x55d98f790000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:29.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.726+0000 7f0677fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d98f7932e0 0x55d98f790740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:48272/0 (socket says 172.21.15.101:48272) 2023-07-18T03:08:29.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.726+0000 7f0677fff700 1 -- 172.21.15.101:0/2012544435 learned_addr learned my addr 172.21.15.101:0/2012544435 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:08:29.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.727+0000 7f0677fff700 1 -- 172.21.15.101:0/2012544435 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d98f7883f0 msgr2=0x55d98f790000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:08:29.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.727+0000 7f0677fff700 1 --2- 172.21.15.101:0/2012544435 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d98f7883f0 0x55d98f790000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.727+0000 7f0677fff700 1 -- 172.21.15.101:0/2012544435 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d98f787ae0 msgr2=0x55d98f792d10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-18T03:08:29.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.727+0000 7f0677fff700 1 --2- 172.21.15.101:0/2012544435 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d98f787ae0 0x55d98f792d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.727+0000 7f0677fff700 1 -- 172.21.15.101:0/2012544435 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0670003ae0 con 0x55d98f7932e0 2023-07-18T03:08:29.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.727+0000 7f0676ffd700 1 --2- 172.21.15.101:0/2012544435 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d98f7883f0 0x55d98f790000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:08:29.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.727+0000 7f0677fff700 1 --2- 172.21.15.101:0/2012544435 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d98f7932e0 0x55d98f790740 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0670004aa0 tx=0x7f0670004b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66b3f44146038f61 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:08:29.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.727+0000 7f0656ffd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:08:29.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.727+0000 7f0656ffd700 1 -- 172.21.15.101:0/2012544435 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0670019040 con 0x55d98f7932e0 2023-07-18T03:08:29.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.728+0000 7f0656ffd700 1 -- 172.21.15.101:0/2012544435 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f0670007a90 con 0x55d98f7932e0 2023-07-18T03:08:29.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.728+0000 7f06914dcb80 1 -- 172.21.15.101:0/2012544435 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d98f62b4f0 con 0x55d98f7932e0 2023-07-18T03:08:29.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.728+0000 7f0656ffd700 1 -- 172.21.15.101:0/2012544435 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0670007c50 con 0x55d98f7932e0 2023-07-18T03:08:29.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.728+0000 7f06914dcb80 10 client.25760 fetch_fsmap learned FSMap version 24 2023-07-18T03:08:29.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.728+0000 7f06914dcb80 1 -- 172.21.15.101:0/2012544435 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55d98f4e8c60 con 0x55d98f7932e0 2023-07-18T03:08:29.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.729+0000 7f0656ffd700 1 -- 172.21.15.101:0/2012544435 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d98f62b4f0 con 0x55d98f7932e0 2023-07-18T03:08:29.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.729+0000 7f0656ffd700 1 -- 172.21.15.101:0/2012544435 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f067001f450 con 0x55d98f7932e0 2023-07-18T03:08:29.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.729+0000 7f06914dcb80 10 client.25760 fetch_fsmap finished waiting for FSMap version 24 2023-07-18T03:08:29.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.729+0000 7f06914dcb80 10 client.25760 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:08:29.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.729+0000 7f06914dcb80 20 client.25760 populate_metadata read hostname 'smithi101' 2023-07-18T03:08:29.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.729+0000 7f06914dcb80 1 --2- 172.21.15.101:0/2012544435 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55d98f7a2b20 0x55d98f7a4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:29.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.729+0000 7f06914dcb80 4 client.25760 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:08:29.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.729+0000 7f06914dcb80 1 -- 172.21.15.101:0/2012544435 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d98f6caeb0 con 0x55d98f7a2b20 2023-07-18T03:08:29.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.729+0000 7f06777fe700 1 --2- 172.21.15.101:0/2012544435 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55d98f7a2b20 0x55d98f7a4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:29.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.730+0000 7f06777fe700 1 --2- 172.21.15.101:0/2012544435 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55d98f7a2b20 0x55d98f7a4f30 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=df89f7f4b7c7819b server_cookie=bb5f3646e24ed2e9 in_seq=0 out_seq=0 2023-07-18T03:08:29.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.730+0000 7f0656ffd700 10 client.25760 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:08:29.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.731+0000 7f0656ffd700 1 -- 172.21.15.101:0/2012544435 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d98f6caeb0 con 0x55d98f7a2b20 2023-07-18T03:08:29.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.731+0000 7f0656ffd700 10 client.25760 handle_command_reply: tid=0 2023-07-18T03:08:29.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.731+0000 7f06914dcb80 1 client.25760 shutdown 2023-07-18T03:08:29.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.731+0000 7f06914dcb80 1 -- 172.21.15.101:0/2012544435 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d98f7932e0 msgr2=0x55d98f790740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:29.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.731+0000 7f06914dcb80 1 --2- 172.21.15.101:0/2012544435 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d98f7932e0 0x55d98f790740 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0670004aa0 tx=0x7f0670004b80 comp rx=0 tx=0).stop 2023-07-18T03:08:29.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.731+0000 7f06914dcb80 1 -- 172.21.15.101:0/2012544435 shutdown_connections 2023-07-18T03:08:29.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.731+0000 7f06914dcb80 1 --2- 172.21.15.101:0/2012544435 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55d98f7a2b20 0x55d98f7a4f30 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.731+0000 7f06914dcb80 1 --2- 172.21.15.101:0/2012544435 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d98f7932e0 0x55d98f790740 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.731+0000 7f06914dcb80 1 --2- 172.21.15.101:0/2012544435 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d98f787ae0 0x55d98f792d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.731+0000 7f0656ffd700 0 client.25760 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:08:29.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.731+0000 7f06914dcb80 1 --2- 172.21.15.101:0/2012544435 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d98f7883f0 0x55d98f790000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.732+0000 7f06914dcb80 1 -- 172.21.15.101:0/2012544435 >> 172.21.15.101:0/2012544435 conn(0x55d98f775da0 msgr2=0x55d98f77b290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:08:29.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.732+0000 7f06914dcb80 1 -- 172.21.15.101:0/2012544435 shutdown_connections 2023-07-18T03:08:29.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.732+0000 7f06914dcb80 1 -- 172.21.15.101:0/2012544435 wait complete. 2023-07-18T03:08:29.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.732+0000 7f06914dcb80 20 client.25760 trim_cache size 0 max 16384 2023-07-18T03:08:29.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.758+0000 7f06914dcb80 1 Processor -- start 2023-07-18T03:08:29.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.758+0000 7f06914dcb80 1 -- start start 2023-07-18T03:08:29.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.758+0000 7f06914dcb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d98f787ae0 0x55d98f779e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:29.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.758+0000 7f06914dcb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d98f7883f0 0x55d98f79ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:29.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.758+0000 7f06914dcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d98f7932e0 0x55d98f7928b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:29.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.758+0000 7f06914dcb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98f7759e0 con 0x55d98f7883f0 2023-07-18T03:08:29.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.758+0000 7f06914dcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98f7899e0 con 0x55d98f7932e0 2023-07-18T03:08:29.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.758+0000 7f06914dcb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98f77a380 con 0x55d98f787ae0 2023-07-18T03:08:29.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.759+0000 7f06777fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d98f787ae0 0x55d98f779e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:29.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.759+0000 7f06777fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d98f787ae0 0x55d98f779e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:59508/0 (socket says 172.21.15.101:59508) 2023-07-18T03:08:29.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.759+0000 7f06777fe700 1 -- 172.21.15.101:0/4141073388 learned_addr learned my addr 172.21.15.101:0/4141073388 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:08:29.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.759+0000 7f0677fff700 1 --2- 172.21.15.101:0/4141073388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d98f7932e0 0x55d98f7928b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:29.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.759+0000 7f0676ffd700 1 --2- 172.21.15.101:0/4141073388 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d98f7883f0 0x55d98f79ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:29.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.759+0000 7f06777fe700 1 -- 172.21.15.101:0/4141073388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d98f7932e0 msgr2=0x55d98f7928b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:08:29.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.759+0000 7f06777fe700 1 --2- 172.21.15.101:0/4141073388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d98f7932e0 0x55d98f7928b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.759+0000 7f06777fe700 1 -- 172.21.15.101:0/4141073388 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d98f7883f0 msgr2=0x55d98f79ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:08:29.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.759+0000 7f06777fe700 1 --2- 172.21.15.101:0/4141073388 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d98f7883f0 0x55d98f79ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.759+0000 7f06777fe700 1 -- 172.21.15.101:0/4141073388 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d98f526250 con 0x55d98f787ae0 2023-07-18T03:08:29.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.760+0000 7f06777fe700 1 --2- 172.21.15.101:0/4141073388 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d98f787ae0 0x55d98f779e10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x55d98f7a5810 tx=0x7f0668011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8afe84ed6b9a4f45 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:08:29.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.760+0000 7f065e7fc700 1 -- 172.21.15.101:0/4141073388 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0668024040 con 0x55d98f787ae0 2023-07-18T03:08:29.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.760+0000 7f065e7fc700 1 -- 172.21.15.101:0/4141073388 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0668004510 con 0x55d98f787ae0 2023-07-18T03:08:29.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.760+0000 7f065e7fc700 1 -- 172.21.15.101:0/4141073388 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0668025e50 con 0x55d98f787ae0 2023-07-18T03:08:29.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.761+0000 7f06914dcb80 1 -- 172.21.15.101:0/4141073388 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d98f787ae0 msgr2=0x55d98f779e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:29.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.761+0000 7f06914dcb80 1 --2- 172.21.15.101:0/4141073388 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d98f787ae0 0x55d98f779e10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x55d98f7a5810 tx=0x7f0668011000 comp rx=0 tx=0).stop 2023-07-18T03:08:29.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.761+0000 7f06914dcb80 1 -- 172.21.15.101:0/4141073388 shutdown_connections 2023-07-18T03:08:29.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.761+0000 7f06914dcb80 1 --2- 172.21.15.101:0/4141073388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d98f7932e0 0x55d98f7928b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.761+0000 7f06914dcb80 1 --2- 172.21.15.101:0/4141073388 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d98f787ae0 0x55d98f779e10 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.761+0000 7f06914dcb80 1 --2- 172.21.15.101:0/4141073388 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d98f7883f0 0x55d98f79ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.761+0000 7f06914dcb80 1 -- 172.21.15.101:0/4141073388 >> 172.21.15.101:0/4141073388 conn(0x55d98f775da0 msgr2=0x55d98f7a9890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:08:29.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.761+0000 7f06914dcb80 1 -- 172.21.15.101:0/4141073388 shutdown_connections 2023-07-18T03:08:29.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.761+0000 7f06914dcb80 1 -- 172.21.15.101:0/4141073388 wait complete. 2023-07-18T03:08:29.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.762+0000 7f06914dcb80 1 -- start start 2023-07-18T03:08:29.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.762+0000 7f06914dcb80 1 Processor -- start 2023-07-18T03:08:29.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.763+0000 7f06914dcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d98f787ae0 0x55d98f797b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:29.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.763+0000 7f06914dcb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d98f7883f0 0x55d98f798060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:29.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.763+0000 7f06914dcb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d98f7932e0 0x55d98f7f59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:29.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.763+0000 7f06914dcb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98f7f5f20 con 0x55d98f7883f0 2023-07-18T03:08:29.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.763+0000 7f06914dcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98f7f6060 con 0x55d98f787ae0 2023-07-18T03:08:29.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.763+0000 7f06914dcb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98f7f61d0 con 0x55d98f7932e0 2023-07-18T03:08:29.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.763+0000 7f06914dcb80 10 client.0 _ll_register_callbacks cb 0x7f066c0d4400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:08:29.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.764+0000 7f06777fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d98f787ae0 0x55d98f797b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:29.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.764+0000 7f06777fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d98f787ae0 0x55d98f797b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:48288/0 (socket says 172.21.15.101:48288) 2023-07-18T03:08:29.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.764+0000 7f06777fe700 1 -- 172.21.15.101:0/1885758793 learned_addr learned my addr 172.21.15.101:0/1885758793 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:08:29.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.764+0000 7f0676ffd700 1 --2- 172.21.15.101:0/1885758793 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d98f7883f0 0x55d98f798060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:29.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.764+0000 7f0677fff700 1 --2- 172.21.15.101:0/1885758793 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d98f7932e0 0x55d98f7f59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:29.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.764+0000 7f06777fe700 1 -- 172.21.15.101:0/1885758793 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d98f7932e0 msgr2=0x55d98f7f59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:08:29.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.764+0000 7f06777fe700 1 --2- 172.21.15.101:0/1885758793 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d98f7932e0 0x55d98f7f59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.764+0000 7f06777fe700 1 -- 172.21.15.101:0/1885758793 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d98f7883f0 msgr2=0x55d98f798060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:08:29.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.764+0000 7f06777fe700 1 --2- 172.21.15.101:0/1885758793 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d98f7883f0 0x55d98f798060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.764+0000 7f06777fe700 1 -- 172.21.15.101:0/1885758793 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06680117b0 con 0x55d98f787ae0 2023-07-18T03:08:29.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.764+0000 7f0677fff700 1 --2- 172.21.15.101:0/1885758793 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d98f7932e0 0x55d98f7f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:08:29.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.765+0000 7f06777fe700 1 --2- 172.21.15.101:0/1885758793 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d98f787ae0 0x55d98f797b20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f06680057c0 tx=0x7f0668025780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c2234dcc2f0cdce server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:08:29.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.765+0000 7f0656ffd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:08:29.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.765+0000 7f0656ffd700 1 -- 172.21.15.101:0/1885758793 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0668024040 con 0x55d98f787ae0 2023-07-18T03:08:29.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.765+0000 7f0656ffd700 1 -- 172.21.15.101:0/1885758793 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f0668025440 con 0x55d98f787ae0 2023-07-18T03:08:29.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.765+0000 7f06914dcb80 1 -- 172.21.15.101:0/1885758793 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f067000f040 con 0x55d98f787ae0 2023-07-18T03:08:29.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.765+0000 7f0656ffd700 1 -- 172.21.15.101:0/1885758793 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0668004360 con 0x55d98f787ae0 2023-07-18T03:08:29.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.765+0000 7f06914dcb80 10 client.25766 fetch_fsmap learned FSMap version 24 2023-07-18T03:08:29.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.765+0000 7f06914dcb80 1 -- 172.21.15.101:0/1885758793 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55d98f4fe2b0 con 0x55d98f787ae0 2023-07-18T03:08:29.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.766+0000 7f0656ffd700 1 -- 172.21.15.101:0/1885758793 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f067000f040 con 0x55d98f787ae0 2023-07-18T03:08:29.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.766+0000 7f0656ffd700 1 -- 172.21.15.101:0/1885758793 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0668003cf0 con 0x55d98f787ae0 2023-07-18T03:08:29.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.766+0000 7f06914dcb80 10 client.25766 fetch_fsmap finished waiting for FSMap version 24 2023-07-18T03:08:29.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.766+0000 7f06914dcb80 10 client.25766 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:08:29.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.766+0000 7f06914dcb80 20 client.25766 populate_metadata read hostname 'smithi101' 2023-07-18T03:08:29.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.766+0000 7f06914dcb80 1 --2- 172.21.15.101:0/1885758793 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55d98f7f77f0 0x55d98f7f9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:29.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.766+0000 7f06914dcb80 4 client.25766 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:08:29.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.766+0000 7f06914dcb80 1 -- 172.21.15.101:0/1885758793 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d98f68d050 con 0x55d98f7f77f0 2023-07-18T03:08:29.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.767+0000 7f0676ffd700 1 --2- 172.21.15.101:0/1885758793 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55d98f7f77f0 0x55d98f7f9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:29.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.767+0000 7f0676ffd700 1 --2- 172.21.15.101:0/1885758793 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55d98f7f77f0 0x55d98f7f9c00 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=208be3737dd47e92 server_cookie=8960b16d30c068f7 in_seq=0 out_seq=0 2023-07-18T03:08:29.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.767+0000 7f0656ffd700 10 client.25766 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:08:29.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.767+0000 7f0656ffd700 1 -- 172.21.15.101:0/1885758793 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d98f68d050 con 0x55d98f7f77f0 2023-07-18T03:08:29.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.767+0000 7f0656ffd700 10 client.25766 handle_command_reply: tid=0 2023-07-18T03:08:29.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.768+0000 7f06914dcb80 1 client.25766 shutdown 2023-07-18T03:08:29.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.768+0000 7f06914dcb80 1 -- 172.21.15.101:0/1885758793 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d98f787ae0 msgr2=0x55d98f797b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:29.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.768+0000 7f06914dcb80 1 --2- 172.21.15.101:0/1885758793 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d98f787ae0 0x55d98f797b20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f06680057c0 tx=0x7f0668025780 comp rx=0 tx=0).stop 2023-07-18T03:08:29.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.768+0000 7f06914dcb80 1 -- 172.21.15.101:0/1885758793 shutdown_connections 2023-07-18T03:08:29.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.768+0000 7f06914dcb80 1 --2- 172.21.15.101:0/1885758793 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55d98f7f77f0 0x55d98f7f9c00 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.768+0000 7f06914dcb80 1 --2- 172.21.15.101:0/1885758793 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d98f787ae0 0x55d98f797b20 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.768+0000 7f06914dcb80 1 --2- 172.21.15.101:0/1885758793 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d98f7932e0 0x55d98f7f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.768+0000 7f06914dcb80 1 --2- 172.21.15.101:0/1885758793 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d98f7883f0 0x55d98f798060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.768+0000 7f06914dcb80 1 -- 172.21.15.101:0/1885758793 >> 172.21.15.101:0/1885758793 conn(0x55d98f775da0 msgr2=0x55d98f79ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:08:29.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.768+0000 7f0656ffd700 0 client.25766 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:08:29.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.768+0000 7f06914dcb80 1 -- 172.21.15.101:0/1885758793 shutdown_connections 2023-07-18T03:08:29.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.768+0000 7f06914dcb80 1 -- 172.21.15.101:0/1885758793 wait complete. 2023-07-18T03:08:29.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.769+0000 7f06914dcb80 20 client.25766 trim_cache size 0 max 16384 2023-07-18T03:08:29.766 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:08:29.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.769+0000 7f067dbff700 1 -- 172.21.15.101:0/2420192530 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f06600a14e0 msgr2=0x7f06600a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:29.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.769+0000 7f067dbff700 1 --2- 172.21.15.101:0/2420192530 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f06600a14e0 0x7f06600a3990 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0668013570 tx=0x7f0668009260 comp rx=0 tx=0).stop 2023-07-18T03:08:29.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.769+0000 7f067dbff700 1 -- 172.21.15.101:0/2420192530 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f06780731c0 msgr2=0x7f0678198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:29.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.769+0000 7f067dbff700 1 --2- 172.21.15.101:0/2420192530 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f06780731c0 0x7f0678198330 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f066c005910 tx=0x7f066c005740 comp rx=0 tx=0).stop 2023-07-18T03:08:29.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.769+0000 7f067dbff700 1 -- 172.21.15.101:0/2420192530 shutdown_connections 2023-07-18T03:08:29.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.769+0000 7f067dbff700 1 --2- 172.21.15.101:0/2420192530 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f06600a14e0 0x7f06600a3990 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.769+0000 7f067dbff700 1 --2- 172.21.15.101:0/2420192530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06780728b0 0x7f0678197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.769+0000 7f067dbff700 1 --2- 172.21.15.101:0/2420192530 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f067810c310 0x7f067819c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.769+0000 7f067dbff700 1 --2- 172.21.15.101:0/2420192530 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f06780731c0 0x7f0678198330 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:29.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.769+0000 7f067dbff700 1 -- 172.21.15.101:0/2420192530 >> 172.21.15.101:0/2420192530 conn(0x7f06780fb5f0 msgr2=0x7f067810e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:08:29.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.770+0000 7f067dbff700 1 -- 172.21.15.101:0/2420192530 shutdown_connections 2023-07-18T03:08:29.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:29.770+0000 7f067dbff700 1 -- 172.21.15.101:0/2420192530 wait complete. 2023-07-18T03:08:29.786 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-18T03:08:30.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.134+0000 7f87073ff700 1 -- 172.21.15.101:0/2857113923 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8700105af0 msgr2=0x7f8700105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:30.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.134+0000 7f87073ff700 1 --2- 172.21.15.101:0/2857113923 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8700105af0 0x7f8700105ec0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f86fc009990 tx=0x7f86fc009ca0 comp rx=0 tx=0).stop 2023-07-18T03:08:30.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.135+0000 7f87073ff700 1 -- 172.21.15.101:0/2857113923 shutdown_connections 2023-07-18T03:08:30.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.135+0000 7f87073ff700 1 --2- 172.21.15.101:0/2857113923 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8700100490 0x7f870010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:30.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.135+0000 7f87073ff700 1 --2- 172.21.15.101:0/2857113923 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8700105af0 0x7f8700105ec0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:30.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.135+0000 7f87073ff700 1 --2- 172.21.15.101:0/2857113923 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87000ffae0 0x7f87000fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:30.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.135+0000 7f87073ff700 1 -- 172.21.15.101:0/2857113923 >> 172.21.15.101:0/2857113923 conn(0x7f87000fb5e0 msgr2=0x7f87000fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:08:30.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.135+0000 7f87073ff700 1 -- 172.21.15.101:0/2857113923 shutdown_connections 2023-07-18T03:08:30.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.135+0000 7f87073ff700 1 -- 172.21.15.101:0/2857113923 wait complete. 2023-07-18T03:08:30.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.136+0000 7f87073ff700 1 Processor -- start 2023-07-18T03:08:30.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.136+0000 7f87073ff700 1 -- start start 2023-07-18T03:08:30.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.137+0000 7f87073ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87000ffae0 0x7f8700106980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:30.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.137+0000 7f87073ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8700100490 0x7f8700106ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:30.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.137+0000 7f87073ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8700105af0 0x7f8700107400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:30.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.137+0000 7f87073ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87001a8440 con 0x7f8700105af0 2023-07-18T03:08:30.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.137+0000 7f87073ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87001a8580 con 0x7f8700100490 2023-07-18T03:08:30.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.137+0000 7f87073ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87001a8740 con 0x7f87000ffae0 2023-07-18T03:08:30.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.138+0000 7f870599c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8700105af0 0x7f8700107400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:30.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.138+0000 7f870599c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8700105af0 0x7f8700107400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:56736/0 (socket says 172.21.15.101:56736) 2023-07-18T03:08:30.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.138+0000 7f870599c700 1 -- 172.21.15.101:0/1771931090 learned_addr learned my addr 172.21.15.101:0/1771931090 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:08:30.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.138+0000 7f870519b700 1 --2- 172.21.15.101:0/1771931090 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87000ffae0 0x7f8700106980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:30.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.138+0000 7f870499a700 1 --2- 172.21.15.101:0/1771931090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8700100490 0x7f8700106ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:30.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.138+0000 7f870499a700 1 -- 172.21.15.101:0/1771931090 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87000ffae0 msgr2=0x7f8700106980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:30.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.138+0000 7f870499a700 1 --2- 172.21.15.101:0/1771931090 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87000ffae0 0x7f8700106980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:30.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.138+0000 7f870499a700 1 -- 172.21.15.101:0/1771931090 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8700105af0 msgr2=0x7f8700107400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:30.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.138+0000 7f870499a700 1 --2- 172.21.15.101:0/1771931090 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8700105af0 0x7f8700107400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:30.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.138+0000 7f870499a700 1 -- 172.21.15.101:0/1771931090 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86fc0095f0 con 0x7f8700100490 2023-07-18T03:08:30.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.138+0000 7f870499a700 1 --2- 172.21.15.101:0/1771931090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8700100490 0x7f8700106ec0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f86f000ea20 tx=0x7f86f000ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:08:30.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.139+0000 7f86f67fc700 1 -- 172.21.15.101:0/1771931090 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f000cbd0 con 0x7f8700100490 2023-07-18T03:08:30.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.139+0000 7f86f67fc700 1 -- 172.21.15.101:0/1771931090 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86f000ee60 con 0x7f8700100490 2023-07-18T03:08:30.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.139+0000 7f86f67fc700 1 -- 172.21.15.101:0/1771931090 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f0018680 con 0x7f8700100490 2023-07-18T03:08:30.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.139+0000 7f87073ff700 1 -- 172.21.15.101:0/1771931090 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87001a8a20 con 0x7f8700100490 2023-07-18T03:08:30.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.139+0000 7f87073ff700 1 -- 172.21.15.101:0/1771931090 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87001a8ee0 con 0x7f8700100490 2023-07-18T03:08:30.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.145+0000 7f86f67fc700 1 -- 172.21.15.101:0/1771931090 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) v1 ==== 135949+0+0 (secure 0 0 0) 0x7f86f0010430 con 0x7f8700100490 2023-07-18T03:08:30.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.145+0000 7f871accfb80 1 Processor -- start 2023-07-18T03:08:30.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.146+0000 7f871accfb80 1 -- start start 2023-07-18T03:08:30.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.146+0000 7f871accfb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56045736c2d0 0x56045736c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:30.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.146+0000 7f871accfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56045736cbe0 0x560457377590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:30.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.146+0000 7f871accfb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560457377ad0 0x560457379eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:30.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.146+0000 7f871accfb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560457272cb0 con 0x56045736c2d0 2023-07-18T03:08:30.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.146+0000 7f871accfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604572c5a40 con 0x56045736cbe0 2023-07-18T03:08:30.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.146+0000 7f871accfb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604572beed0 con 0x560457377ad0 2023-07-18T03:08:30.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.147+0000 7f870519b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56045736cbe0 0x560457377590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:30.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.147+0000 7f870519b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56045736cbe0 0x560457377590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:48322/0 (socket says 172.21.15.101:48322) 2023-07-18T03:08:30.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.147+0000 7f870519b700 1 -- 172.21.15.101:0/3364012339 learned_addr learned my addr 172.21.15.101:0/3364012339 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:08:30.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.147+0000 7f870599c700 1 --2- 172.21.15.101:0/3364012339 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56045736c2d0 0x56045736c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:30.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.147+0000 7f870519b700 1 -- 172.21.15.101:0/3364012339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560457377ad0 msgr2=0x560457379eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-18T03:08:30.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.147+0000 7f870519b700 1 --2- 172.21.15.101:0/3364012339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560457377ad0 0x560457379eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:30.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.147+0000 7f870519b700 1 -- 172.21.15.101:0/3364012339 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56045736c2d0 msgr2=0x56045736c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:08:30.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.147+0000 7f870519b700 1 --2- 172.21.15.101:0/3364012339 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56045736c2d0 0x56045736c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:30.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.147+0000 7f870519b700 1 -- 172.21.15.101:0/3364012339 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604570ecde0 con 0x56045736cbe0 2023-07-18T03:08:30.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.148+0000 7f870519b700 1 --2- 172.21.15.101:0/3364012339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56045736cbe0 0x560457377590 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f86fc005f50 tx=0x7f86fc00b610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0421a83c6b0febd server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:08:30.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.148+0000 7f86f67fc700 1 --2- 172.21.15.101:0/1771931090 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f86ec0a14e0 0x7f86ec0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:30.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.148+0000 7f86f67fc700 1 -- 172.21.15.101:0/1771931090 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f86f0014070 con 0x7f8700100490 2023-07-18T03:08:30.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.148+0000 7f86eb7fe700 1 -- 172.21.15.101:0/3364012339 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86fc01d070 con 0x56045736cbe0 2023-07-18T03:08:30.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.148+0000 7f86eb7fe700 1 -- 172.21.15.101:0/3364012339 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86fc01e040 con 0x56045736cbe0 2023-07-18T03:08:30.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.148+0000 7f86eb7fe700 1 -- 172.21.15.101:0/3364012339 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86fc004820 con 0x56045736cbe0 2023-07-18T03:08:30.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.149+0000 7f870599c700 1 --2- 172.21.15.101:0/1771931090 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f86ec0a14e0 0x7f86ec0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:30.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.149+0000 7f871accfb80 1 -- 172.21.15.101:0/3364012339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56045736cbe0 msgr2=0x560457377590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:30.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.149+0000 7f871accfb80 1 --2- 172.21.15.101:0/3364012339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56045736cbe0 0x560457377590 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f86fc005f50 tx=0x7f86fc00b610 comp rx=0 tx=0).stop 2023-07-18T03:08:30.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.150+0000 7f871accfb80 1 -- 172.21.15.101:0/3364012339 shutdown_connections 2023-07-18T03:08:30.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.150+0000 7f871accfb80 1 --2- 172.21.15.101:0/3364012339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560457377ad0 0x560457379eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:30.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.150+0000 7f871accfb80 1 --2- 172.21.15.101:0/3364012339 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56045736c2d0 0x56045736c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:30.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.150+0000 7f871accfb80 1 --2- 172.21.15.101:0/3364012339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56045736cbe0 0x560457377590 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:30.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.150+0000 7f871accfb80 1 -- 172.21.15.101:0/3364012339 >> 172.21.15.101:0/3364012339 conn(0x560457362da0 msgr2=0x5604573686a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:08:30.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.150+0000 7f870599c700 1 --2- 172.21.15.101:0/1771931090 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f86ec0a14e0 0x7f86ec0a3990 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f86f8009550 tx=0x7f86f8009040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:08:30.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.150+0000 7f871accfb80 1 -- 172.21.15.101:0/3364012339 shutdown_connections 2023-07-18T03:08:30.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.150+0000 7f871accfb80 1 -- 172.21.15.101:0/3364012339 wait complete. 2023-07-18T03:08:30.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.151+0000 7f871accfb80 1 -- start start 2023-07-18T03:08:30.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.151+0000 7f871accfb80 1 Processor -- start 2023-07-18T03:08:30.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.152+0000 7f871accfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56045736c2d0 0x5604573774f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:30.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.152+0000 7f871accfb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56045736cbe0 0x560457374700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:30.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.152+0000 7f871accfb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560457377ad0 0x560457374d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:30.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.152+0000 7f871accfb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56045725eac0 con 0x56045736cbe0 2023-07-18T03:08:30.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.152+0000 7f871accfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56045725e7c0 con 0x56045736c2d0 2023-07-18T03:08:30.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.152+0000 7f871accfb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x560457142920 con 0x560457377ad0 2023-07-18T03:08:30.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.152+0000 7f871accfb80 10 client.0 _ll_register_callbacks cb 0x56045725c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:08:30.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.152+0000 7f870599c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560457377ad0 0x560457374d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:30.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.152+0000 7f870599c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560457377ad0 0x560457374d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:59552/0 (socket says 172.21.15.101:59552) 2023-07-18T03:08:30.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.152+0000 7f870599c700 1 -- 172.21.15.101:0/2600384674 learned_addr learned my addr 172.21.15.101:0/2600384674 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:08:30.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.153+0000 7f870499a700 1 --2- 172.21.15.101:0/2600384674 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56045736cbe0 0x560457374700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:30.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.153+0000 7f870599c700 1 -- 172.21.15.101:0/2600384674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56045736c2d0 msgr2=0x5604573774f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:08:30.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.153+0000 7f870599c700 1 --2- 172.21.15.101:0/2600384674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56045736c2d0 0x5604573774f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:30.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.153+0000 7f870599c700 1 -- 172.21.15.101:0/2600384674 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56045736cbe0 msgr2=0x560457374700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:08:30.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.153+0000 7f870599c700 1 --2- 172.21.15.101:0/2600384674 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56045736cbe0 0x560457374700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:30.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.153+0000 7f870599c700 1 -- 172.21.15.101:0/2600384674 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f870010d220 con 0x560457377ad0 2023-07-18T03:08:30.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.153+0000 7f870499a700 1 --2- 172.21.15.101:0/2600384674 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56045736cbe0 0x560457374700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:08:30.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.153+0000 7f870599c700 1 --2- 172.21.15.101:0/2600384674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560457377ad0 0x560457374d30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f86f8016e00 tx=0x7f86f8004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bc624a34c075470 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:08:30.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.153+0000 7f86e8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:08:30.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.154+0000 7f86e8ff9700 1 -- 172.21.15.101:0/2600384674 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f8007bb0 con 0x560457377ad0 2023-07-18T03:08:30.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.154+0000 7f86e8ff9700 1 -- 172.21.15.101:0/2600384674 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f86f8015a20 con 0x560457377ad0 2023-07-18T03:08:30.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.154+0000 7f871accfb80 1 -- 172.21.15.101:0/2600384674 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5604572184f0 con 0x560457377ad0 2023-07-18T03:08:30.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.154+0000 7f86e8ff9700 1 -- 172.21.15.101:0/2600384674 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86f8015be0 con 0x560457377ad0 2023-07-18T03:08:30.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.154+0000 7f871accfb80 10 client.25801 fetch_fsmap learned FSMap version 24 2023-07-18T03:08:30.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.154+0000 7f871accfb80 1 -- 172.21.15.101:0/2600384674 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5604570d5c60 con 0x560457377ad0 2023-07-18T03:08:30.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.155+0000 7f86e8ff9700 1 -- 172.21.15.101:0/2600384674 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5604572184f0 con 0x560457377ad0 2023-07-18T03:08:30.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.155+0000 7f86e8ff9700 1 -- 172.21.15.101:0/2600384674 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f86f80223a0 con 0x560457377ad0 2023-07-18T03:08:30.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.155+0000 7f871accfb80 10 client.25801 fetch_fsmap finished waiting for FSMap version 24 2023-07-18T03:08:30.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.155+0000 7f871accfb80 10 client.25801 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:08:30.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.155+0000 7f871accfb80 20 client.25801 populate_metadata read hostname 'smithi101' 2023-07-18T03:08:30.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.155+0000 7f871accfb80 1 --2- 172.21.15.101:0/2600384674 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x560457387400 0x560457389810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:30.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.155+0000 7f871accfb80 4 client.25801 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:08:30.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.155+0000 7f871accfb80 1 -- 172.21.15.101:0/2600384674 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5604572b7eb0 con 0x560457387400 2023-07-18T03:08:30.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.156+0000 7f870519b700 1 --2- 172.21.15.101:0/2600384674 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x560457387400 0x560457389810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:30.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.156+0000 7f870519b700 1 --2- 172.21.15.101:0/2600384674 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x560457387400 0x560457389810 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fb1b7657a394c2f server_cookie=d6b5a1c1824a3329 in_seq=0 out_seq=0 2023-07-18T03:08:30.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.156+0000 7f86e8ff9700 10 client.25801 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:08:30.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.157+0000 7f86e8ff9700 1 -- 172.21.15.101:0/2600384674 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5604572b7eb0 con 0x560457387400 2023-07-18T03:08:30.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.157+0000 7f86e8ff9700 10 client.25801 handle_command_reply: tid=0 2023-07-18T03:08:30.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.157+0000 7f871accfb80 1 client.25801 shutdown 2023-07-18T03:08:30.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.157+0000 7f871accfb80 1 -- 172.21.15.101:0/2600384674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560457377ad0 msgr2=0x560457374d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:30.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.157+0000 7f871accfb80 1 --2- 172.21.15.101:0/2600384674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560457377ad0 0x560457374d30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f86f8016e00 tx=0x7f86f8004b40 comp rx=0 tx=0).stop 2023-07-18T03:08:30.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.158+0000 7f871accfb80 1 -- 172.21.15.101:0/2600384674 shutdown_connections 2023-07-18T03:08:30.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.158+0000 7f871accfb80 1 --2- 172.21.15.101:0/2600384674 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x560457387400 0x560457389810 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:30.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.158+0000 7f871accfb80 1 --2- 172.21.15.101:0/2600384674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56045736c2d0 0x5604573774f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:30.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.158+0000 7f871accfb80 1 --2- 172.21.15.101:0/2600384674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560457377ad0 0x560457374d30 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:30.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.158+0000 7f871accfb80 1 --2- 172.21.15.101:0/2600384674 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56045736cbe0 0x560457374700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:30.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.158+0000 7f871accfb80 1 -- 172.21.15.101:0/2600384674 >> 172.21.15.101:0/2600384674 conn(0x560457362da0 msgr2=0x560457368230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:08:30.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.158+0000 7f86e8ff9700 0 client.25801 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:08:30.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.158+0000 7f871accfb80 1 -- 172.21.15.101:0/2600384674 shutdown_connections 2023-07-18T03:08:30.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.158+0000 7f871accfb80 1 -- 172.21.15.101:0/2600384674 wait complete. 2023-07-18T03:08:30.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.158+0000 7f871accfb80 20 client.25801 trim_cache size 0 max 16384 2023-07-18T03:08:30.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.184+0000 7f871accfb80 1 Processor -- start 2023-07-18T03:08:30.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.184+0000 7f871accfb80 1 -- start start 2023-07-18T03:08:30.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.184+0000 7f871accfb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56045736c2d0 0x56045736f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:30.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.184+0000 7f871accfb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56045736cbe0 0x5604573681d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:30.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.184+0000 7f871accfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560457377ad0 0x560457377020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:30.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.184+0000 7f871accfb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56045736ddd0 con 0x56045736c2d0 2023-07-18T03:08:30.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.184+0000 7f871accfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560457372c60 con 0x560457377ad0 2023-07-18T03:08:30.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.184+0000 7f871accfb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x560457389b20 con 0x56045736cbe0 2023-07-18T03:08:30.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.185+0000 7f870499a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56045736cbe0 0x5604573681d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:30.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.185+0000 7f870499a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56045736cbe0 0x5604573681d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:59564/0 (socket says 172.21.15.101:59564) 2023-07-18T03:08:30.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.185+0000 7f870499a700 1 -- 172.21.15.101:0/1970500499 learned_addr learned my addr 172.21.15.101:0/1970500499 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:08:30.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.185+0000 7f870599c700 1 --2- 172.21.15.101:0/1970500499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560457377ad0 0x560457377020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:30.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.185+0000 7f870519b700 1 --2- 172.21.15.101:0/1970500499 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56045736c2d0 0x56045736f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:30.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.186+0000 7f870499a700 1 -- 172.21.15.101:0/1970500499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560457377ad0 msgr2=0x560457377020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:08:30.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.186+0000 7f870499a700 1 --2- 172.21.15.101:0/1970500499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560457377ad0 0x560457377020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:30.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.186+0000 7f870499a700 1 -- 172.21.15.101:0/1970500499 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56045736c2d0 msgr2=0x56045736f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:08:30.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.186+0000 7f870499a700 1 --2- 172.21.15.101:0/1970500499 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56045736c2d0 0x56045736f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:30.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.186+0000 7f870499a700 1 -- 172.21.15.101:0/1970500499 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604573645e0 con 0x56045736cbe0 2023-07-18T03:08:30.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.186+0000 7f870599c700 1 --2- 172.21.15.101:0/1970500499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560457377ad0 0x560457377020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:08:30.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.186+0000 7f870499a700 1 --2- 172.21.15.101:0/1970500499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56045736cbe0 0x5604573681d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f86f00963c0 tx=0x7f86f0099c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5bbb10ad3dd39e8 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:08:30.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.186+0000 7f86ebfff700 1 -- 172.21.15.101:0/1970500499 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f0099820 con 0x56045736cbe0 2023-07-18T03:08:30.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.186+0000 7f86ebfff700 1 -- 172.21.15.101:0/1970500499 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86f000cd30 con 0x56045736cbe0 2023-07-18T03:08:30.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.186+0000 7f86ebfff700 1 -- 172.21.15.101:0/1970500499 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f009a380 con 0x56045736cbe0 2023-07-18T03:08:30.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.187+0000 7f871accfb80 1 -- 172.21.15.101:0/1970500499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56045736cbe0 msgr2=0x5604573681d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:30.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.187+0000 7f871accfb80 1 --2- 172.21.15.101:0/1970500499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56045736cbe0 0x5604573681d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f86f00963c0 tx=0x7f86f0099c90 comp rx=0 tx=0).stop 2023-07-18T03:08:30.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.187+0000 7f871accfb80 1 -- 172.21.15.101:0/1970500499 shutdown_connections 2023-07-18T03:08:30.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.187+0000 7f871accfb80 1 --2- 172.21.15.101:0/1970500499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560457377ad0 0x560457377020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:30.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.187+0000 7f871accfb80 1 --2- 172.21.15.101:0/1970500499 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56045736c2d0 0x56045736f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:30.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.187+0000 7f871accfb80 1 --2- 172.21.15.101:0/1970500499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56045736cbe0 0x5604573681d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:30.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.187+0000 7f871accfb80 1 -- 172.21.15.101:0/1970500499 >> 172.21.15.101:0/1970500499 conn(0x560457362da0 msgr2=0x560457389410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:08:30.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.187+0000 7f871accfb80 1 -- 172.21.15.101:0/1970500499 shutdown_connections 2023-07-18T03:08:30.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.187+0000 7f871accfb80 1 -- 172.21.15.101:0/1970500499 wait complete. 2023-07-18T03:08:30.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.188+0000 7f871accfb80 1 -- start start 2023-07-18T03:08:30.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.188+0000 7f871accfb80 1 Processor -- start 2023-07-18T03:08:30.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.189+0000 7f871accfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56045736c2d0 0x56045737e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:30.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.189+0000 7f871accfb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56045736cbe0 0x56045737c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:30.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.189+0000 7f871accfb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560457377ad0 0x56045737cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:30.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.189+0000 7f871accfb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56045737d020 con 0x56045736cbe0 2023-07-18T03:08:30.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.189+0000 7f871accfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56045737d190 con 0x56045736c2d0 2023-07-18T03:08:30.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.189+0000 7f871accfb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56045737d350 con 0x560457377ad0 2023-07-18T03:08:30.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.189+0000 7f871accfb80 10 client.0 _ll_register_callbacks cb 0x56045725c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:08:30.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.189+0000 7f870519b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56045736c2d0 0x56045737e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:30.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.190+0000 7f870519b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56045736c2d0 0x56045737e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:48358/0 (socket says 172.21.15.101:48358) 2023-07-18T03:08:30.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.190+0000 7f870519b700 1 -- 172.21.15.101:0/4125655626 learned_addr learned my addr 172.21.15.101:0/4125655626 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:08:30.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.190+0000 7f870499a700 1 --2- 172.21.15.101:0/4125655626 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56045736cbe0 0x56045737c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:30.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.190+0000 7f870599c700 1 --2- 172.21.15.101:0/4125655626 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560457377ad0 0x56045737cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:30.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.190+0000 7f870519b700 1 -- 172.21.15.101:0/4125655626 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560457377ad0 msgr2=0x56045737cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:08:30.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.190+0000 7f870519b700 1 --2- 172.21.15.101:0/4125655626 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560457377ad0 0x56045737cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:30.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.190+0000 7f870519b700 1 -- 172.21.15.101:0/4125655626 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56045736cbe0 msgr2=0x56045737c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:08:30.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.190+0000 7f870519b700 1 --2- 172.21.15.101:0/4125655626 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56045736cbe0 0x56045737c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:30.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.190+0000 7f870519b700 1 -- 172.21.15.101:0/4125655626 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87001a8ee0 con 0x56045736c2d0 2023-07-18T03:08:30.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.191+0000 7f870519b700 1 --2- 172.21.15.101:0/4125655626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56045736c2d0 0x56045737e270 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f86fc00b6f0 tx=0x7f86fc005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e90509b0ef8fb95 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:08:30.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.191+0000 7f86e8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:08:30.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.191+0000 7f86e8ff9700 1 -- 172.21.15.101:0/4125655626 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86fc030040 con 0x56045736c2d0 2023-07-18T03:08:30.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.191+0000 7f871accfb80 1 -- 172.21.15.101:0/4125655626 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f86fc01e040 con 0x56045736c2d0 2023-07-18T03:08:30.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.191+0000 7f86e8ff9700 1 -- 172.21.15.101:0/4125655626 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f86fc005980 con 0x56045736c2d0 2023-07-18T03:08:30.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.191+0000 7f86e8ff9700 1 -- 172.21.15.101:0/4125655626 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86fc01d040 con 0x56045736c2d0 2023-07-18T03:08:30.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.191+0000 7f871accfb80 10 client.25781 fetch_fsmap learned FSMap version 24 2023-07-18T03:08:30.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.191+0000 7f871accfb80 1 -- 172.21.15.101:0/4125655626 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560457113250 con 0x56045736c2d0 2023-07-18T03:08:30.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.192+0000 7f86e8ff9700 1 -- 172.21.15.101:0/4125655626 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86fc01e040 con 0x56045736c2d0 2023-07-18T03:08:30.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.193+0000 7f86e8ff9700 1 -- 172.21.15.101:0/4125655626 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f86fc004500 con 0x56045736c2d0 2023-07-18T03:08:30.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.193+0000 7f871accfb80 10 client.25781 fetch_fsmap finished waiting for FSMap version 24 2023-07-18T03:08:30.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.193+0000 7f871accfb80 10 client.25781 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:08:30.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.193+0000 7f871accfb80 20 client.25781 populate_metadata read hostname 'smithi101' 2023-07-18T03:08:30.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.193+0000 7f871accfb80 1 --2- 172.21.15.101:0/4125655626 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5604573db370 0x5604573dd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:30.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.193+0000 7f871accfb80 4 client.25781 mds_command: new command op to 24562 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:08:30.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.193+0000 7f871accfb80 1 -- 172.21.15.101:0/4125655626 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56045727a050 con 0x5604573db370 2023-07-18T03:08:30.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.193+0000 7f870499a700 1 --2- 172.21.15.101:0/4125655626 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5604573db370 0x5604573dd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:30.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.194+0000 7f86e8ff9700 10 client.25781 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:08:30.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.194+0000 7f870499a700 1 --2- 172.21.15.101:0/4125655626 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5604573db370 0x5604573dd780 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24967558918adde9 server_cookie=3fc9523475436567 in_seq=0 out_seq=0 2023-07-18T03:08:30.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.194+0000 7f86e8ff9700 1 -- 172.21.15.101:0/4125655626 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56045727a050 con 0x5604573db370 2023-07-18T03:08:30.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.194+0000 7f86e8ff9700 10 client.25781 handle_command_reply: tid=0 2023-07-18T03:08:30.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.194+0000 7f871accfb80 1 client.25781 shutdown 2023-07-18T03:08:30.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.195+0000 7f871accfb80 1 -- 172.21.15.101:0/4125655626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56045736c2d0 msgr2=0x56045737e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:30.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.195+0000 7f871accfb80 1 --2- 172.21.15.101:0/4125655626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56045736c2d0 0x56045737e270 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f86fc00b6f0 tx=0x7f86fc005740 comp rx=0 tx=0).stop 2023-07-18T03:08:30.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.195+0000 7f871accfb80 1 -- 172.21.15.101:0/4125655626 shutdown_connections 2023-07-18T03:08:30.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.195+0000 7f871accfb80 1 --2- 172.21.15.101:0/4125655626 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5604573db370 0x5604573dd780 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:30.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.195+0000 7f871accfb80 1 --2- 172.21.15.101:0/4125655626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56045736c2d0 0x56045737e270 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:30.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.195+0000 7f871accfb80 1 --2- 172.21.15.101:0/4125655626 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560457377ad0 0x56045737cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:30.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.195+0000 7f871accfb80 1 --2- 172.21.15.101:0/4125655626 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56045736cbe0 0x56045737c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:30.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.195+0000 7f871accfb80 1 -- 172.21.15.101:0/4125655626 >> 172.21.15.101:0/4125655626 conn(0x560457362da0 msgr2=0x5604573678b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:08:30.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.195+0000 7f86e8ff9700 0 client.25781 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:08:30.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.195+0000 7f871accfb80 1 -- 172.21.15.101:0/4125655626 shutdown_connections 2023-07-18T03:08:30.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.195+0000 7f871accfb80 1 -- 172.21.15.101:0/4125655626 wait complete. 2023-07-18T03:08:30.198 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.195+0000 7f871accfb80 20 client.25781 trim_cache size 0 max 16384 2023-07-18T03:08:30.198 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:08:30.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.196+0000 7f87073ff700 1 -- 172.21.15.101:0/1771931090 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f86ec0a14e0 msgr2=0x7f86ec0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:30.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.196+0000 7f87073ff700 1 --2- 172.21.15.101:0/1771931090 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f86ec0a14e0 0x7f86ec0a3990 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f86f8009550 tx=0x7f86f8009040 comp rx=0 tx=0).stop 2023-07-18T03:08:30.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.196+0000 7f87073ff700 1 -- 172.21.15.101:0/1771931090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8700100490 msgr2=0x7f8700106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:30.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.196+0000 7f87073ff700 1 --2- 172.21.15.101:0/1771931090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8700100490 0x7f8700106ec0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f86f000ea20 tx=0x7f86f000ed30 comp rx=0 tx=0).stop 2023-07-18T03:08:30.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.196+0000 7f87073ff700 1 -- 172.21.15.101:0/1771931090 shutdown_connections 2023-07-18T03:08:30.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.196+0000 7f87073ff700 1 --2- 172.21.15.101:0/1771931090 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8700105af0 0x7f8700107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:30.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.196+0000 7f87073ff700 1 --2- 172.21.15.101:0/1771931090 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87000ffae0 0x7f8700106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:30.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.196+0000 7f87073ff700 1 --2- 172.21.15.101:0/1771931090 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f86ec0a14e0 0x7f86ec0a3990 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:30.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.196+0000 7f87073ff700 1 --2- 172.21.15.101:0/1771931090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8700100490 0x7f8700106ec0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:30.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.196+0000 7f87073ff700 1 -- 172.21.15.101:0/1771931090 >> 172.21.15.101:0/1771931090 conn(0x7f87000fb5e0 msgr2=0x7f870010adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:08:30.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.196+0000 7f87073ff700 1 -- 172.21.15.101:0/1771931090 shutdown_connections 2023-07-18T03:08:30.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:30.196+0000 7f87073ff700 1 -- 172.21.15.101:0/1771931090 wait complete. 2023-07-18T03:08:30.203 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:08:29 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/2971387739' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:08:30.204 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:08:29 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2971387739' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:08:30.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:08:29 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/2971387739' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:08:31.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:08:30 smithi179 ceph-mon[129229]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-18T03:08:31.206 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-18T03:08:31.206 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-18T03:08:31.232 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:08:30 smithi101 ceph-mon[126379]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-18T03:08:31.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:08:30 smithi155 ceph-mon[128885]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-18T03:08:31.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.419+0000 7f7498b9d700 1 -- 172.21.15.101:0/999206747 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f74940ff9d0 msgr2=0x7f74940ffe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:31.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.419+0000 7f7498b9d700 1 --2- 172.21.15.101:0/999206747 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f74940ff9d0 0x7f74940ffe40 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7488009990 tx=0x7f7488009ca0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.419+0000 7f7498b9d700 1 -- 172.21.15.101:0/999206747 shutdown_connections 2023-07-18T03:08:31.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.419+0000 7f7498b9d700 1 --2- 172.21.15.101:0/999206747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74941059e0 0x7f7494105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.419+0000 7f7498b9d700 1 --2- 172.21.15.101:0/999206747 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7494100380 0x7f749410c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.419+0000 7f7498b9d700 1 --2- 172.21.15.101:0/999206747 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f74940ff9d0 0x7f74940ffe40 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.419+0000 7f7498b9d700 1 -- 172.21.15.101:0/999206747 >> 172.21.15.101:0/999206747 conn(0x7f74940fb4b0 msgr2=0x7f74940fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:08:31.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.419+0000 7f7498b9d700 1 -- 172.21.15.101:0/999206747 shutdown_connections 2023-07-18T03:08:31.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.419+0000 7f7498b9d700 1 -- 172.21.15.101:0/999206747 wait complete. 2023-07-18T03:08:31.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.420+0000 7f7498b9d700 1 Processor -- start 2023-07-18T03:08:31.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.420+0000 7f7498b9d700 1 -- start start 2023-07-18T03:08:31.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.421+0000 7f7498b9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74940ff9d0 0x7f749419d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:31.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.421+0000 7f7498b9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7494100380 0x7f749419ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:31.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.421+0000 7f7498b9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f74941059e0 0x7f7494197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:31.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.421+0000 7f7498b9d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7494198070 con 0x7f74941059e0 2023-07-18T03:08:31.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.421+0000 7f7498b9d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74941981e0 con 0x7f74940ff9d0 2023-07-18T03:08:31.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.421+0000 7f7498b9d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74941983a0 con 0x7f7494100380 2023-07-18T03:08:31.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.421+0000 7f74937fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74940ff9d0 0x7f749419d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:31.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.421+0000 7f74937fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74940ff9d0 0x7f749419d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:48380/0 (socket says 172.21.15.101:48380) 2023-07-18T03:08:31.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.421+0000 7f7493fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f74941059e0 0x7f7494197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:31.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.421+0000 7f74937fe700 1 -- 172.21.15.101:0/3066416692 learned_addr learned my addr 172.21.15.101:0/3066416692 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:08:31.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.421+0000 7f7492ffd700 1 --2- 172.21.15.101:0/3066416692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7494100380 0x7f749419ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:31.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.422+0000 7f7493fff700 1 -- 172.21.15.101:0/3066416692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7494100380 msgr2=0x7f749419ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:31.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.422+0000 7f7493fff700 1 --2- 172.21.15.101:0/3066416692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7494100380 0x7f749419ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.422+0000 7f7493fff700 1 -- 172.21.15.101:0/3066416692 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74940ff9d0 msgr2=0x7f749419d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:31.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.422+0000 7f7493fff700 1 --2- 172.21.15.101:0/3066416692 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74940ff9d0 0x7f749419d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.414 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.422+0000 7f7493fff700 1 -- 172.21.15.101:0/3066416692 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74880095f0 con 0x7f74941059e0 2023-07-18T03:08:31.414 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.422+0000 7f74937fe700 1 --2- 172.21.15.101:0/3066416692 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74940ff9d0 0x7f749419d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:08:31.414 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.422+0000 7f7493fff700 1 --2- 172.21.15.101:0/3066416692 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f74941059e0 0x7f7494197a00 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f748c007f10 tx=0x7f748c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:08:31.414 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.422+0000 7f7490ff9700 1 -- 172.21.15.101:0/3066416692 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c009ea0 con 0x7f74941059e0 2023-07-18T03:08:31.414 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.422+0000 7f7498b9d700 1 -- 172.21.15.101:0/3066416692 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7494198680 con 0x7f74941059e0 2023-07-18T03:08:31.414 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.422+0000 7f7490ff9700 1 -- 172.21.15.101:0/3066416692 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f748c00f040 con 0x7f74941059e0 2023-07-18T03:08:31.414 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.422+0000 7f7490ff9700 1 -- 172.21.15.101:0/3066416692 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c004400 con 0x7f74941059e0 2023-07-18T03:08:31.415 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.422+0000 7f7498b9d700 1 -- 172.21.15.101:0/3066416692 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74941a4980 con 0x7f74941059e0 2023-07-18T03:08:31.415 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.425+0000 7f7490ff9700 1 -- 172.21.15.101:0/3066416692 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 24) v1 ==== 135949+0+0 (secure 0 0 0) 0x7f748c004ad0 con 0x7f74941059e0 2023-07-18T03:08:31.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.427+0000 7f74ad781b80 1 Processor -- start 2023-07-18T03:08:31.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.427+0000 7f7490ff9700 1 --2- 172.21.15.101:0/3066416692 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f747c0a1480 0x7f747c0a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:31.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.427+0000 7f74ad781b80 1 -- start start 2023-07-18T03:08:31.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.427+0000 7f7490ff9700 1 -- 172.21.15.101:0/3066416692 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f748c0ccc20 con 0x7f74941059e0 2023-07-18T03:08:31.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.427+0000 7f7492ffd700 1 --2- 172.21.15.101:0/3066416692 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f747c0a1480 0x7f747c0a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:31.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.427+0000 7f74ad781b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569ce6fc570 0x5569ce6fc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:31.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.427+0000 7f74ad781b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569ce6fce80 0x5569ce706870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:31.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.427+0000 7f74ad781b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569ce706db0 0x5569ce709190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:31.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.427+0000 7f74ad781b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569ce601d30 con 0x5569ce706db0 2023-07-18T03:08:31.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.427+0000 7f74ad781b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569ce654ac0 con 0x5569ce6fc570 2023-07-18T03:08:31.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.427+0000 7f74ad781b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569ce64df50 con 0x5569ce6fce80 2023-07-18T03:08:31.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.428+0000 7f7493fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569ce706db0 0x5569ce709190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:31.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.428+0000 7f7492ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569ce6fce80 0x5569ce706870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:31.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.428+0000 7f7493fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569ce706db0 0x5569ce709190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:56818/0 (socket says 172.21.15.101:56818) 2023-07-18T03:08:31.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.428+0000 7f7493fff700 1 -- 172.21.15.101:0/900209306 learned_addr learned my addr 172.21.15.101:0/900209306 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:08:31.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.428+0000 7f74937fe700 1 --2- 172.21.15.101:0/900209306 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569ce6fc570 0x5569ce6fc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:31.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.428+0000 7f7493fff700 1 -- 172.21.15.101:0/900209306 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569ce6fce80 msgr2=0x5569ce706870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:08:31.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.428+0000 7f7493fff700 1 --2- 172.21.15.101:0/900209306 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569ce6fce80 0x5569ce706870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.428+0000 7f7493fff700 1 -- 172.21.15.101:0/900209306 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569ce6fc570 msgr2=0x5569ce6fc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:08:31.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.428+0000 7f7493fff700 1 --2- 172.21.15.101:0/900209306 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569ce6fc570 0x5569ce6fc940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.428+0000 7f7493fff700 1 -- 172.21.15.101:0/900209306 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569ce47be60 con 0x5569ce706db0 2023-07-18T03:08:31.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.428+0000 7f7492ffd700 1 --2- 172.21.15.101:0/3066416692 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f747c0a1480 0x7f747c0a3930 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7488005cd0 tx=0x7f748800b6b0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:08:31.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.428+0000 7f7493fff700 1 --2- 172.21.15.101:0/900209306 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569ce706db0 0x5569ce709190 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f748c095bf0 tx=0x7f748c099230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a067f7d191fd831 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:08:31.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.428+0000 7f7479ffb700 1 -- 172.21.15.101:0/900209306 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c0996f0 con 0x5569ce706db0 2023-07-18T03:08:31.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.428+0000 7f7479ffb700 1 -- 172.21.15.101:0/900209306 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f748c09d720 con 0x5569ce706db0 2023-07-18T03:08:31.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.428+0000 7f7479ffb700 1 -- 172.21.15.101:0/900209306 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c098e40 con 0x5569ce706db0 2023-07-18T03:08:31.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.429+0000 7f74ad781b80 1 -- 172.21.15.101:0/900209306 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569ce706db0 msgr2=0x5569ce709190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:31.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.429+0000 7f74ad781b80 1 --2- 172.21.15.101:0/900209306 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569ce706db0 0x5569ce709190 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f748c095bf0 tx=0x7f748c099230 comp rx=0 tx=0).stop 2023-07-18T03:08:31.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.429+0000 7f74ad781b80 1 -- 172.21.15.101:0/900209306 shutdown_connections 2023-07-18T03:08:31.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.429+0000 7f74ad781b80 1 --2- 172.21.15.101:0/900209306 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569ce6fc570 0x5569ce6fc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.429+0000 7f74ad781b80 1 --2- 172.21.15.101:0/900209306 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569ce706db0 0x5569ce709190 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.429+0000 7f74ad781b80 1 --2- 172.21.15.101:0/900209306 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569ce6fce80 0x5569ce706870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.429+0000 7f74ad781b80 1 -- 172.21.15.101:0/900209306 >> 172.21.15.101:0/900209306 conn(0x5569ce6f2070 msgr2=0x5569ce6f79b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:08:31.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.429+0000 7f74ad781b80 1 -- 172.21.15.101:0/900209306 shutdown_connections 2023-07-18T03:08:31.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.429+0000 7f74ad781b80 1 -- 172.21.15.101:0/900209306 wait complete. 2023-07-18T03:08:31.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.430+0000 7f74ad781b80 1 -- start start 2023-07-18T03:08:31.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.430+0000 7f74ad781b80 1 Processor -- start 2023-07-18T03:08:31.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.430+0000 7f74ad781b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569ce6fc570 0x5569ce706810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:31.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.430+0000 7f74ad781b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569ce6fce80 0x5569ce703a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:31.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.431+0000 7f74ad781b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569ce706db0 0x5569ce704160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:31.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.431+0000 7f74ad781b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569ce5edb40 con 0x5569ce6fc570 2023-07-18T03:08:31.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.431+0000 7f74ad781b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569ce5ed840 con 0x5569ce6fce80 2023-07-18T03:08:31.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.431+0000 7f74ad781b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569ce4d19a0 con 0x5569ce706db0 2023-07-18T03:08:31.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.431+0000 7f74ad781b80 10 client.0 _ll_register_callbacks cb 0x5569ce5eb7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:08:31.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.431+0000 7f74937fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569ce6fc570 0x5569ce706810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:31.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.431+0000 7f74937fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569ce6fc570 0x5569ce706810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:56824/0 (socket says 172.21.15.101:56824) 2023-07-18T03:08:31.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.431+0000 7f7492ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569ce6fce80 0x5569ce703a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:31.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.431+0000 7f74937fe700 1 -- 172.21.15.101:0/2463258313 learned_addr learned my addr 172.21.15.101:0/2463258313 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:08:31.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.431+0000 7f7493fff700 1 --2- 172.21.15.101:0/2463258313 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569ce706db0 0x5569ce704160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:31.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.431+0000 7f74937fe700 1 -- 172.21.15.101:0/2463258313 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569ce706db0 msgr2=0x5569ce704160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:08:31.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.431+0000 7f74937fe700 1 --2- 172.21.15.101:0/2463258313 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569ce706db0 0x5569ce704160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.431+0000 7f74937fe700 1 -- 172.21.15.101:0/2463258313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569ce6fce80 msgr2=0x5569ce703a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:08:31.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.431+0000 7f74937fe700 1 --2- 172.21.15.101:0/2463258313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569ce6fce80 0x5569ce703a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.431+0000 7f74937fe700 1 -- 172.21.15.101:0/2463258313 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74941a4980 con 0x5569ce6fc570 2023-07-18T03:08:31.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.431+0000 7f74937fe700 1 --2- 172.21.15.101:0/2463258313 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569ce6fc570 0x5569ce706810 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7484011d10 tx=0x7f748400f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da1bc57242141ad0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:08:31.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.431+0000 7f7472ffd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:08:31.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.432+0000 7f7472ffd700 1 -- 172.21.15.101:0/2463258313 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7484016070 con 0x5569ce6fc570 2023-07-18T03:08:31.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.432+0000 7f74ad781b80 1 -- 172.21.15.101:0/2463258313 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5569ce5a7570 con 0x5569ce6fc570 2023-07-18T03:08:31.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.432+0000 7f7472ffd700 1 -- 172.21.15.101:0/2463258313 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f748400fd10 con 0x5569ce6fc570 2023-07-18T03:08:31.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.432+0000 7f7472ffd700 1 -- 172.21.15.101:0/2463258313 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7484004050 con 0x5569ce6fc570 2023-07-18T03:08:31.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.432+0000 7f74ad781b80 10 client.15762 fetch_fsmap learned FSMap version 24 2023-07-18T03:08:31.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.432+0000 7f74ad781b80 1 -- 172.21.15.101:0/2463258313 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5569ce464ce0 con 0x5569ce6fc570 2023-07-18T03:08:31.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.432+0000 7f7472ffd700 1 -- 172.21.15.101:0/2463258313 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5569ce5a7570 con 0x5569ce6fc570 2023-07-18T03:08:31.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.432+0000 7f7472ffd700 1 -- 172.21.15.101:0/2463258313 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7484019370 con 0x5569ce6fc570 2023-07-18T03:08:31.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.432+0000 7f74ad781b80 10 client.15762 fetch_fsmap finished waiting for FSMap version 24 2023-07-18T03:08:31.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.432+0000 7f74ad781b80 10 client.15762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:08:31.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.432+0000 7f74ad781b80 20 client.15762 populate_metadata read hostname 'smithi101' 2023-07-18T03:08:31.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.432+0000 7f74ad781b80 1 --2- 172.21.15.101:0/2463258313 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5569ce716820 0x5569ce718c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:31.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.432+0000 7f74ad781b80 4 client.15762 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:08:31.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.432+0000 7f74ad781b80 1 -- 172.21.15.101:0/2463258313 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5569ce646f30 con 0x5569ce716820 2023-07-18T03:08:31.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.433+0000 7f7492ffd700 1 --2- 172.21.15.101:0/2463258313 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5569ce716820 0x5569ce718c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:31.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.433+0000 7f7472ffd700 10 client.15762 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:08:31.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.433+0000 7f7492ffd700 1 --2- 172.21.15.101:0/2463258313 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5569ce716820 0x5569ce718c30 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a81dfbd7f0afee5 server_cookie=1953a6f7209f45ad in_seq=0 out_seq=0 2023-07-18T03:08:31.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.434+0000 7f7472ffd700 1 -- 172.21.15.101:0/2463258313 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5569ce646f30 con 0x5569ce716820 2023-07-18T03:08:31.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.434+0000 7f7472ffd700 10 client.15762 handle_command_reply: tid=0 2023-07-18T03:08:31.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.434+0000 7f74ad781b80 1 client.15762 shutdown 2023-07-18T03:08:31.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.434+0000 7f74ad781b80 1 -- 172.21.15.101:0/2463258313 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569ce6fc570 msgr2=0x5569ce706810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:31.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.434+0000 7f74ad781b80 1 --2- 172.21.15.101:0/2463258313 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569ce6fc570 0x5569ce706810 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7484011d10 tx=0x7f748400f6a0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.434+0000 7f74ad781b80 1 -- 172.21.15.101:0/2463258313 shutdown_connections 2023-07-18T03:08:31.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.434+0000 7f74ad781b80 1 --2- 172.21.15.101:0/2463258313 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5569ce716820 0x5569ce718c30 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.434+0000 7f74ad781b80 1 --2- 172.21.15.101:0/2463258313 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569ce706db0 0x5569ce704160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.434+0000 7f74ad781b80 1 --2- 172.21.15.101:0/2463258313 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569ce6fc570 0x5569ce706810 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.434+0000 7f7472ffd700 0 client.15762 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:08:31.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.434+0000 7f74ad781b80 1 --2- 172.21.15.101:0/2463258313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569ce6fce80 0x5569ce703a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.434+0000 7f74ad781b80 1 -- 172.21.15.101:0/2463258313 >> 172.21.15.101:0/2463258313 conn(0x5569ce6f2070 msgr2=0x5569ce6f75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:08:31.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.435+0000 7f74ad781b80 1 -- 172.21.15.101:0/2463258313 shutdown_connections 2023-07-18T03:08:31.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.435+0000 7f74ad781b80 1 -- 172.21.15.101:0/2463258313 wait complete. 2023-07-18T03:08:31.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.435+0000 7f74ad781b80 20 client.15762 trim_cache size 0 max 16384 2023-07-18T03:08:31.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.453+0000 7f74ad781b80 1 Processor -- start 2023-07-18T03:08:31.442 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.453+0000 7f74ad781b80 1 -- start start 2023-07-18T03:08:31.442 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.453+0000 7f74ad781b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569ce6fc570 0x5569ce7039d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:31.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.453+0000 7f74ad781b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569ce6fce80 0x5569ce706340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:31.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.453+0000 7f74ad781b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569ce706db0 0x5569ce700c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:31.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.453+0000 7f74ad781b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569ce6f8660 con 0x5569ce6fc570 2023-07-18T03:08:31.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.453+0000 7f74ad781b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569ce6f1bf0 con 0x5569ce706db0 2023-07-18T03:08:31.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.453+0000 7f74ad781b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569ce713ed0 con 0x5569ce6fce80 2023-07-18T03:08:31.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.453+0000 7f74937fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569ce6fc570 0x5569ce7039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:31.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.453+0000 7f7493fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569ce706db0 0x5569ce700c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:31.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.453+0000 7f7493fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569ce706db0 0x5569ce700c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:48420/0 (socket says 172.21.15.101:48420) 2023-07-18T03:08:31.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.453+0000 7f7493fff700 1 -- 172.21.15.101:0/2098398700 learned_addr learned my addr 172.21.15.101:0/2098398700 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:08:31.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.454+0000 7f7493fff700 1 -- 172.21.15.101:0/2098398700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569ce6fce80 msgr2=0x5569ce706340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:08:31.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.454+0000 7f7493fff700 1 --2- 172.21.15.101:0/2098398700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569ce6fce80 0x5569ce706340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.454+0000 7f7493fff700 1 -- 172.21.15.101:0/2098398700 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569ce6fc570 msgr2=0x5569ce7039d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:08:31.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.454+0000 7f7493fff700 1 --2- 172.21.15.101:0/2098398700 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569ce6fc570 0x5569ce7039d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.454+0000 7f7493fff700 1 -- 172.21.15.101:0/2098398700 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569ce6f63b0 con 0x5569ce706db0 2023-07-18T03:08:31.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.454+0000 7f74937fe700 1 --2- 172.21.15.101:0/2098398700 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569ce6fc570 0x5569ce7039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:08:31.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.454+0000 7f7493fff700 1 --2- 172.21.15.101:0/2098398700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569ce706db0 0x5569ce700c90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f748c0993a0 tx=0x7f748c00b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f35bbb5e49695094 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:08:31.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.454+0000 7f747a7fc700 1 -- 172.21.15.101:0/2098398700 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c0996f0 con 0x5569ce706db0 2023-07-18T03:08:31.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.454+0000 7f747a7fc700 1 -- 172.21.15.101:0/2098398700 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f748c0a1b40 con 0x5569ce706db0 2023-07-18T03:08:31.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.454+0000 7f747a7fc700 1 -- 172.21.15.101:0/2098398700 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c099bb0 con 0x5569ce706db0 2023-07-18T03:08:31.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.455+0000 7f74ad781b80 1 -- 172.21.15.101:0/2098398700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569ce706db0 msgr2=0x5569ce700c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:31.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.455+0000 7f74ad781b80 1 --2- 172.21.15.101:0/2098398700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569ce706db0 0x5569ce700c90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f748c0993a0 tx=0x7f748c00b040 comp rx=0 tx=0).stop 2023-07-18T03:08:31.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.455+0000 7f74ad781b80 1 -- 172.21.15.101:0/2098398700 shutdown_connections 2023-07-18T03:08:31.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.455+0000 7f74ad781b80 1 --2- 172.21.15.101:0/2098398700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569ce706db0 0x5569ce700c90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.455+0000 7f74ad781b80 1 --2- 172.21.15.101:0/2098398700 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569ce6fc570 0x5569ce7039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.455+0000 7f74ad781b80 1 --2- 172.21.15.101:0/2098398700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569ce6fce80 0x5569ce706340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.455+0000 7f74ad781b80 1 -- 172.21.15.101:0/2098398700 >> 172.21.15.101:0/2098398700 conn(0x5569ce6f2070 msgr2=0x5569ce7032c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:08:31.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.455+0000 7f74ad781b80 1 -- 172.21.15.101:0/2098398700 shutdown_connections 2023-07-18T03:08:31.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.455+0000 7f74ad781b80 1 -- 172.21.15.101:0/2098398700 wait complete. 2023-07-18T03:08:31.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.456+0000 7f74ad781b80 1 -- start start 2023-07-18T03:08:31.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.456+0000 7f74ad781b80 1 Processor -- start 2023-07-18T03:08:31.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.456+0000 7f74ad781b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569ce6fc570 0x5569ce70c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:31.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.456+0000 7f74ad781b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569ce6fce80 0x5569ce70c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:31.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.456+0000 7f74ad781b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569ce706db0 0x5569ce76a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:31.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.456+0000 7f74ad781b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569ce70cf90 con 0x5569ce706db0 2023-07-18T03:08:31.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.456+0000 7f74ad781b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569ce70d100 con 0x5569ce6fc570 2023-07-18T03:08:31.450 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.456+0000 7f74ad781b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569ce76ac60 con 0x5569ce6fce80 2023-07-18T03:08:31.450 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.457+0000 7f74ad781b80 10 client.0 _ll_register_callbacks cb 0x7f74840117e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:08:31.450 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.457+0000 7f74937fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569ce6fc570 0x5569ce70c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:31.450 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.457+0000 7f74937fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569ce6fc570 0x5569ce70c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:48436/0 (socket says 172.21.15.101:48436) 2023-07-18T03:08:31.450 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.457+0000 7f74937fe700 1 -- 172.21.15.101:0/2734835336 learned_addr learned my addr 172.21.15.101:0/2734835336 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:08:31.450 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.457+0000 7f7492ffd700 1 --2- 172.21.15.101:0/2734835336 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569ce6fce80 0x5569ce70c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:31.450 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.457+0000 7f7493fff700 1 --2- 172.21.15.101:0/2734835336 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569ce706db0 0x5569ce76a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:31.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.457+0000 7f74937fe700 1 -- 172.21.15.101:0/2734835336 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569ce6fce80 msgr2=0x5569ce70c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:08:31.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.457+0000 7f74937fe700 1 --2- 172.21.15.101:0/2734835336 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569ce6fce80 0x5569ce70c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.457+0000 7f74937fe700 1 -- 172.21.15.101:0/2734835336 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569ce706db0 msgr2=0x5569ce76a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:08:31.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.457+0000 7f74937fe700 1 --2- 172.21.15.101:0/2734835336 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569ce706db0 0x5569ce76a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.457+0000 7f74937fe700 1 -- 172.21.15.101:0/2734835336 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5569ce464ce0 con 0x5569ce6fc570 2023-07-18T03:08:31.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.457+0000 7f74937fe700 1 --2- 172.21.15.101:0/2734835336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569ce6fc570 0x5569ce70c280 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f74840096b0 tx=0x7f748400ffc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a926555102bfd71c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:08:31.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.457+0000 7f7472ffd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:08:31.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.457+0000 7f7472ffd700 1 -- 172.21.15.101:0/2734835336 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7484016070 con 0x5569ce6fc570 2023-07-18T03:08:31.452 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.457+0000 7f74ad781b80 1 -- 172.21.15.101:0/2734835336 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f748c09d720 con 0x5569ce6fc570 2023-07-18T03:08:31.452 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.457+0000 7f7472ffd700 1 -- 172.21.15.101:0/2734835336 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f7484000e40 con 0x5569ce6fc570 2023-07-18T03:08:31.452 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.458+0000 7f7472ffd700 1 -- 172.21.15.101:0/2734835336 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7484019370 con 0x5569ce6fc570 2023-07-18T03:08:31.452 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.458+0000 7f74ad781b80 10 client.25799 fetch_fsmap learned FSMap version 24 2023-07-18T03:08:31.452 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.458+0000 7f74ad781b80 1 -- 172.21.15.101:0/2734835336 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5569ce47be60 con 0x5569ce6fc570 2023-07-18T03:08:31.452 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.458+0000 7f7472ffd700 1 -- 172.21.15.101:0/2734835336 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f748c09d720 con 0x5569ce6fc570 2023-07-18T03:08:31.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.458+0000 7f7472ffd700 1 -- 172.21.15.101:0/2734835336 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f748400f3c0 con 0x5569ce6fc570 2023-07-18T03:08:31.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.458+0000 7f74ad781b80 10 client.25799 fetch_fsmap finished waiting for FSMap version 24 2023-07-18T03:08:31.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.458+0000 7f74ad781b80 10 client.25799 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:08:31.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.458+0000 7f74ad781b80 20 client.25799 populate_metadata read hostname 'smithi101' 2023-07-18T03:08:31.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.458+0000 7f74ad781b80 1 --2- 172.21.15.101:0/2734835336 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5569ce76c390 0x5569ce76e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:31.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.458+0000 7f74ad781b80 4 client.25799 mds_command: new command op to 24562 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-18T03:08:31.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.458+0000 7f74ad781b80 1 -- 172.21.15.101:0/2734835336 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5569ce6090d0 con 0x5569ce76c390 2023-07-18T03:08:31.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.459+0000 7f7492ffd700 1 --2- 172.21.15.101:0/2734835336 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5569ce76c390 0x5569ce76e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:31.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.459+0000 7f7492ffd700 1 --2- 172.21.15.101:0/2734835336 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5569ce76c390 0x5569ce76e7a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0a98a1b72b112ea server_cookie=9a823d394efc8927 in_seq=0 out_seq=0 2023-07-18T03:08:31.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.459+0000 7f7472ffd700 10 client.25799 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:08:31.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.460+0000 7f7472ffd700 1 -- 172.21.15.101:0/2734835336 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5569ce6090d0 con 0x5569ce76c390 2023-07-18T03:08:31.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.461+0000 7f7472ffd700 10 client.25799 handle_command_reply: tid=0 2023-07-18T03:08:31.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.461+0000 7f74ad781b80 1 client.25799 shutdown 2023-07-18T03:08:31.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.461+0000 7f74ad781b80 1 -- 172.21.15.101:0/2734835336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569ce6fc570 msgr2=0x5569ce70c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:31.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.461+0000 7f74ad781b80 1 --2- 172.21.15.101:0/2734835336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569ce6fc570 0x5569ce70c280 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f74840096b0 tx=0x7f748400ffc0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.461+0000 7f74ad781b80 1 -- 172.21.15.101:0/2734835336 shutdown_connections 2023-07-18T03:08:31.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.461+0000 7f74ad781b80 1 --2- 172.21.15.101:0/2734835336 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5569ce76c390 0x5569ce76e7a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.461+0000 7f74ad781b80 1 --2- 172.21.15.101:0/2734835336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569ce6fc570 0x5569ce70c280 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.461+0000 7f74ad781b80 1 --2- 172.21.15.101:0/2734835336 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569ce706db0 0x5569ce76a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.461+0000 7f74ad781b80 1 --2- 172.21.15.101:0/2734835336 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569ce6fce80 0x5569ce70c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.461+0000 7f74ad781b80 1 -- 172.21.15.101:0/2734835336 >> 172.21.15.101:0/2734835336 conn(0x5569ce6f2070 msgr2=0x5569ce6fbd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:08:31.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.461+0000 7f7472ffd700 0 client.25799 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:08:31.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.461+0000 7f74ad781b80 1 -- 172.21.15.101:0/2734835336 shutdown_connections 2023-07-18T03:08:31.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.461+0000 7f74ad781b80 1 -- 172.21.15.101:0/2734835336 wait complete. 2023-07-18T03:08:31.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.461+0000 7f74ad781b80 20 client.25799 trim_cache size 0 max 16384 2023-07-18T03:08:31.457 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:08:31.457 INFO:teuthology.orchestra.run.smithi101.stdout: "return_code": 0, 2023-07-18T03:08:31.458 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_tag": "2b06628a-eab7-4b15-8803-1c11ca43fa88", 2023-07-18T03:08:31.458 INFO:teuthology.orchestra.run.smithi101.stdout: "mode": "asynchronous" 2023-07-18T03:08:31.458 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:08:31.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.462+0000 7f7498b9d700 1 -- 172.21.15.101:0/3066416692 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f747c0a1480 msgr2=0x7f747c0a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:31.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.462+0000 7f7498b9d700 1 --2- 172.21.15.101:0/3066416692 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f747c0a1480 0x7f747c0a3930 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7488005cd0 tx=0x7f748800b6b0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.462+0000 7f7498b9d700 1 -- 172.21.15.101:0/3066416692 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f74941059e0 msgr2=0x7f7494197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:31.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.462+0000 7f7498b9d700 1 --2- 172.21.15.101:0/3066416692 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f74941059e0 0x7f7494197a00 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f748c007f10 tx=0x7f748c00d3b0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.462+0000 7f7498b9d700 1 -- 172.21.15.101:0/3066416692 shutdown_connections 2023-07-18T03:08:31.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.462+0000 7f7498b9d700 1 --2- 172.21.15.101:0/3066416692 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f747c0a1480 0x7f747c0a3930 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.462+0000 7f7498b9d700 1 --2- 172.21.15.101:0/3066416692 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74940ff9d0 0x7f749419d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.462+0000 7f7498b9d700 1 --2- 172.21.15.101:0/3066416692 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f74941059e0 0x7f7494197a00 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.462+0000 7f7498b9d700 1 --2- 172.21.15.101:0/3066416692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7494100380 0x7f749419ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.462+0000 7f7498b9d700 1 -- 172.21.15.101:0/3066416692 >> 172.21.15.101:0/3066416692 conn(0x7f74940fb4b0 msgr2=0x7f7494072740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:08:31.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.462+0000 7f7498b9d700 1 -- 172.21.15.101:0/3066416692 shutdown_connections 2023-07-18T03:08:31.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.462+0000 7f7498b9d700 1 -- 172.21.15.101:0/3066416692 wait complete. 2023-07-18T03:08:31.469 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:08:31.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.723+0000 7fe8cc1ff700 1 -- 172.21.15.101:0/118485090 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe8c4105af0 msgr2=0x7fe8c4105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:31.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.723+0000 7fe8cc1ff700 1 --2- 172.21.15.101:0/118485090 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe8c4105af0 0x7fe8c4105ec0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0009990 tx=0x7fe8c0009ca0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.723+0000 7fe8cc1ff700 1 -- 172.21.15.101:0/118485090 shutdown_connections 2023-07-18T03:08:31.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.723+0000 7fe8cc1ff700 1 --2- 172.21.15.101:0/118485090 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8c4100490 0x7fe8c410c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.723+0000 7fe8cc1ff700 1 --2- 172.21.15.101:0/118485090 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe8c4105af0 0x7fe8c4105ec0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.723+0000 7fe8cc1ff700 1 --2- 172.21.15.101:0/118485090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8c40ffae0 0x7fe8c40fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.723+0000 7fe8cc1ff700 1 -- 172.21.15.101:0/118485090 >> 172.21.15.101:0/118485090 conn(0x7fe8c40fb5e0 msgr2=0x7fe8c40fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:08:31.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.723+0000 7fe8cc1ff700 1 -- 172.21.15.101:0/118485090 shutdown_connections 2023-07-18T03:08:31.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.724+0000 7fe8cc1ff700 1 -- 172.21.15.101:0/118485090 wait complete. 2023-07-18T03:08:31.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.724+0000 7fe8cc1ff700 1 Processor -- start 2023-07-18T03:08:31.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.725+0000 7fe8cc1ff700 1 -- start start 2023-07-18T03:08:31.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.725+0000 7fe8cc1ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8c40ffae0 0x7fe8c4197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:31.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.725+0000 7fe8cc1ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8c4100490 0x7fe8c4198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:31.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.725+0000 7fe8cc1ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe8c4105af0 0x7fe8c419c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:31.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.725+0000 7fe8cc1ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c419ce10 con 0x7fe8c4105af0 2023-07-18T03:08:31.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.725+0000 7fe8cc1ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c419cf80 con 0x7fe8c40ffae0 2023-07-18T03:08:31.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.725+0000 7fe8cc1ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c419d140 con 0x7fe8c4100490 2023-07-18T03:08:31.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.725+0000 7fe8c979a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8c4100490 0x7fe8c4198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:31.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.725+0000 7fe8ca79c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe8c4105af0 0x7fe8c419c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:31.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.725+0000 7fe8c9f9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8c40ffae0 0x7fe8c4197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:31.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.725+0000 7fe8c979a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8c4100490 0x7fe8c4198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:59648/0 (socket says 172.21.15.101:59648) 2023-07-18T03:08:31.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.725+0000 7fe8c979a700 1 -- 172.21.15.101:0/1775391099 learned_addr learned my addr 172.21.15.101:0/1775391099 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:08:31.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.726+0000 7fe8c979a700 1 -- 172.21.15.101:0/1775391099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8c40ffae0 msgr2=0x7fe8c4197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:31.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.726+0000 7fe8c979a700 1 --2- 172.21.15.101:0/1775391099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8c40ffae0 0x7fe8c4197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.726+0000 7fe8c979a700 1 -- 172.21.15.101:0/1775391099 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe8c4105af0 msgr2=0x7fe8c419c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:31.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.726+0000 7fe8c979a700 1 --2- 172.21.15.101:0/1775391099 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe8c4105af0 0x7fe8c419c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.726+0000 7fe8c979a700 1 -- 172.21.15.101:0/1775391099 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c00095f0 con 0x7fe8c4100490 2023-07-18T03:08:31.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.726+0000 7fe8c979a700 1 --2- 172.21.15.101:0/1775391099 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8c4100490 0x7fe8c4198370 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe8b400e9a0 tx=0x7fe8b400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:08:31.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.726+0000 7fe8baffd700 1 -- 172.21.15.101:0/1775391099 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4009790 con 0x7fe8c4100490 2023-07-18T03:08:31.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.726+0000 7fe8cc1ff700 1 -- 172.21.15.101:0/1775391099 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8c419d420 con 0x7fe8c4100490 2023-07-18T03:08:31.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.726+0000 7fe8baffd700 1 -- 172.21.15.101:0/1775391099 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8b4010430 con 0x7fe8c4100490 2023-07-18T03:08:31.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.726+0000 7fe8baffd700 1 -- 172.21.15.101:0/1775391099 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4018680 con 0x7fe8c4100490 2023-07-18T03:08:31.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.726+0000 7fe8cc1ff700 1 -- 172.21.15.101:0/1775391099 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c41a4c60 con 0x7fe8c4100490 2023-07-18T03:08:31.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.730+0000 7fe8baffd700 1 -- 172.21.15.101:0/1775391099 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 24) v1 ==== 135949+0+0 (secure 0 0 0) 0x7fe8b40105a0 con 0x7fe8c4100490 2023-07-18T03:08:31.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.731+0000 7fe8dfc13b80 1 Processor -- start 2023-07-18T03:08:31.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.731+0000 7fe8dfc13b80 1 -- start start 2023-07-18T03:08:31.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.731+0000 7fe8dfc13b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569c3d262d0 0x5569c3d266a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:31.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.731+0000 7fe8dfc13b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569c3d26be0 0x5569c3d31590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:31.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.731+0000 7fe8dfc13b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569c3d31ad0 0x5569c3d33eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:31.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.731+0000 7fe8dfc13b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569c3c2ccb0 con 0x5569c3d26be0 2023-07-18T03:08:31.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.731+0000 7fe8dfc13b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569c3c7fa40 con 0x5569c3d262d0 2023-07-18T03:08:31.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.731+0000 7fe8dfc13b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569c3c78ed0 con 0x5569c3d31ad0 2023-07-18T03:08:31.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.731+0000 7fe8baffd700 1 --2- 172.21.15.101:0/1775391099 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fe8b00a13e0 0x7fe8b00a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:31.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.732+0000 7fe8c979a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569c3d31ad0 0x5569c3d33eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:31.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.732+0000 7fe8c9f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569c3d26be0 0x5569c3d31590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:31.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.732+0000 7fe8c979a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569c3d31ad0 0x5569c3d33eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:59662/0 (socket says 172.21.15.101:59662) 2023-07-18T03:08:31.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.732+0000 7fe8c979a700 1 -- 172.21.15.101:0/1871449088 learned_addr learned my addr 172.21.15.101:0/1871449088 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:08:31.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.732+0000 7fe8ca79c700 1 --2- 172.21.15.101:0/1871449088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569c3d262d0 0x5569c3d266a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:31.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.732+0000 7fe8ca79c700 1 --2- 172.21.15.101:0/1775391099 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fe8b00a13e0 0x7fe8b00a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:31.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.732+0000 7fe8baffd700 1 -- 172.21.15.101:0/1775391099 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fe8b4014070 con 0x7fe8c4100490 2023-07-18T03:08:31.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.732+0000 7fe8c9f9b700 1 -- 172.21.15.101:0/1871449088 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569c3d31ad0 msgr2=0x5569c3d33eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:08:31.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.732+0000 7fe8c9f9b700 1 --2- 172.21.15.101:0/1871449088 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569c3d31ad0 0x5569c3d33eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.732+0000 7fe8c9f9b700 1 -- 172.21.15.101:0/1871449088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569c3d262d0 msgr2=0x5569c3d266a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:08:31.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.732+0000 7fe8c9f9b700 1 --2- 172.21.15.101:0/1871449088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569c3d262d0 0x5569c3d266a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.732+0000 7fe8c9f9b700 1 -- 172.21.15.101:0/1871449088 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569c3aa6de0 con 0x5569c3d26be0 2023-07-18T03:08:31.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.732+0000 7fe8c979a700 1 --2- 172.21.15.101:0/1871449088 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569c3d31ad0 0x5569c3d33eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:08:31.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.732+0000 7fe8c9f9b700 1 --2- 172.21.15.101:0/1871449088 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569c3d26be0 0x5569c3d31590 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0000c00 tx=0x7fe8c0005ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98907fb88b22bcde server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:08:31.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.732+0000 7fe8ca79c700 1 --2- 172.21.15.101:0/1775391099 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fe8b00a13e0 0x7fe8b00a3890 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc009ec0 tx=0x7fe8bc00b040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:08:31.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.733+0000 7fe8affff700 1 -- 172.21.15.101:0/1871449088 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c001d070 con 0x5569c3d26be0 2023-07-18T03:08:31.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.733+0000 7fe8affff700 1 -- 172.21.15.101:0/1871449088 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8c001e040 con 0x5569c3d26be0 2023-07-18T03:08:31.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.733+0000 7fe8affff700 1 -- 172.21.15.101:0/1871449088 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c0004740 con 0x5569c3d26be0 2023-07-18T03:08:31.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.733+0000 7fe8dfc13b80 1 -- 172.21.15.101:0/1871449088 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569c3d26be0 msgr2=0x5569c3d31590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:31.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.733+0000 7fe8dfc13b80 1 --2- 172.21.15.101:0/1871449088 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569c3d26be0 0x5569c3d31590 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0000c00 tx=0x7fe8c0005ea0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.733+0000 7fe8dfc13b80 1 -- 172.21.15.101:0/1871449088 shutdown_connections 2023-07-18T03:08:31.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.733+0000 7fe8dfc13b80 1 --2- 172.21.15.101:0/1871449088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569c3d262d0 0x5569c3d266a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.733+0000 7fe8dfc13b80 1 --2- 172.21.15.101:0/1871449088 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569c3d31ad0 0x5569c3d33eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.733+0000 7fe8dfc13b80 1 --2- 172.21.15.101:0/1871449088 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569c3d26be0 0x5569c3d31590 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.734+0000 7fe8dfc13b80 1 -- 172.21.15.101:0/1871449088 >> 172.21.15.101:0/1871449088 conn(0x5569c3d1cda0 msgr2=0x5569c3d226a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:08:31.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.734+0000 7fe8dfc13b80 1 -- 172.21.15.101:0/1871449088 shutdown_connections 2023-07-18T03:08:31.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.734+0000 7fe8dfc13b80 1 -- 172.21.15.101:0/1871449088 wait complete. 2023-07-18T03:08:31.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.734+0000 7fe8dfc13b80 1 -- start start 2023-07-18T03:08:31.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.735+0000 7fe8dfc13b80 1 Processor -- start 2023-07-18T03:08:31.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.735+0000 7fe8dfc13b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569c3d262d0 0x5569c3d31520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:31.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.735+0000 7fe8dfc13b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569c3d26be0 0x5569c3d2e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:31.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.735+0000 7fe8dfc13b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569c3d31ad0 0x5569c3d2ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:31.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.735+0000 7fe8dfc13b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569c3c18ac0 con 0x5569c3d262d0 2023-07-18T03:08:31.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.735+0000 7fe8dfc13b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569c3c187c0 con 0x5569c3d31ad0 2023-07-18T03:08:31.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.735+0000 7fe8dfc13b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569c3afc920 con 0x5569c3d26be0 2023-07-18T03:08:31.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.735+0000 7fe8dfc13b80 10 client.0 _ll_register_callbacks cb 0x5569c3c16760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:08:31.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.736+0000 7fe8c9f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569c3d262d0 0x5569c3d31520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:31.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.736+0000 7fe8c9f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569c3d262d0 0x5569c3d31520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:56884/0 (socket says 172.21.15.101:56884) 2023-07-18T03:08:31.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.736+0000 7fe8c9f9b700 1 -- 172.21.15.101:0/2389166290 learned_addr learned my addr 172.21.15.101:0/2389166290 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:08:31.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.736+0000 7fe8c979a700 1 --2- 172.21.15.101:0/2389166290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569c3d26be0 0x5569c3d2e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:31.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.736+0000 7fe8ca79c700 1 --2- 172.21.15.101:0/2389166290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569c3d31ad0 0x5569c3d2ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:31.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.736+0000 7fe8c9f9b700 1 -- 172.21.15.101:0/2389166290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569c3d26be0 msgr2=0x5569c3d2e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:08:31.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.736+0000 7fe8c9f9b700 1 --2- 172.21.15.101:0/2389166290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569c3d26be0 0x5569c3d2e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.736+0000 7fe8c9f9b700 1 -- 172.21.15.101:0/2389166290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569c3d31ad0 msgr2=0x5569c3d2ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:08:31.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.736+0000 7fe8c9f9b700 1 --2- 172.21.15.101:0/2389166290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569c3d31ad0 0x5569c3d2ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.736+0000 7fe8c9f9b700 1 -- 172.21.15.101:0/2389166290 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c41a4c60 con 0x5569c3d262d0 2023-07-18T03:08:31.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.736+0000 7fe8c9f9b700 1 --2- 172.21.15.101:0/2389166290 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569c3d262d0 0x5569c3d31520 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe8c00098e0 tx=0x7fe8c000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c7f3090729eac36 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:08:31.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.736+0000 7fe8ad7fa700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:08:31.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.736+0000 7fe8ad7fa700 1 -- 172.21.15.101:0/2389166290 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c001d040 con 0x5569c3d262d0 2023-07-18T03:08:31.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.736+0000 7fe8ad7fa700 1 -- 172.21.15.101:0/2389166290 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fe8c0005980 con 0x5569c3d262d0 2023-07-18T03:08:31.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.736+0000 7fe8dfc13b80 1 -- 172.21.15.101:0/2389166290 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5569c3bd24f0 con 0x5569c3d262d0 2023-07-18T03:08:31.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.737+0000 7fe8ad7fa700 1 -- 172.21.15.101:0/2389166290 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8c00056a0 con 0x5569c3d262d0 2023-07-18T03:08:31.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.737+0000 7fe8dfc13b80 10 client.15780 fetch_fsmap learned FSMap version 24 2023-07-18T03:08:31.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.737+0000 7fe8dfc13b80 1 -- 172.21.15.101:0/2389166290 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7fe8c410d220 con 0x5569c3d262d0 2023-07-18T03:08:31.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.737+0000 7fe8ad7fa700 1 -- 172.21.15.101:0/2389166290 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5569c3bd24f0 con 0x5569c3d262d0 2023-07-18T03:08:31.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.737+0000 7fe8ad7fa700 1 -- 172.21.15.101:0/2389166290 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe8c0004a50 con 0x5569c3d262d0 2023-07-18T03:08:31.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.737+0000 7fe8dfc13b80 10 client.15780 fetch_fsmap finished waiting for FSMap version 24 2023-07-18T03:08:31.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.737+0000 7fe8dfc13b80 10 client.15780 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:08:31.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.737+0000 7fe8dfc13b80 20 client.15780 populate_metadata read hostname 'smithi101' 2023-07-18T03:08:31.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.737+0000 7fe8dfc13b80 1 --2- 172.21.15.101:0/2389166290 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5569c3d40f40 0x5569c3d43320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:31.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.737+0000 7fe8dfc13b80 4 client.15780 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:08:31.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.737+0000 7fe8dfc13b80 1 -- 172.21.15.101:0/2389166290 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5569c3c71eb0 con 0x5569c3d40f40 2023-07-18T03:08:31.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.738+0000 7fe8c979a700 1 --2- 172.21.15.101:0/2389166290 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5569c3d40f40 0x5569c3d43320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:31.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.738+0000 7fe8ad7fa700 10 client.15780 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:08:31.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.738+0000 7fe8c979a700 1 --2- 172.21.15.101:0/2389166290 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5569c3d40f40 0x5569c3d43320 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f0b62d50cf3e0 server_cookie=14e5497b5ee1c749 in_seq=0 out_seq=0 2023-07-18T03:08:31.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.739+0000 7fe8ad7fa700 1 -- 172.21.15.101:0/2389166290 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5569c3c71eb0 con 0x5569c3d40f40 2023-07-18T03:08:31.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.739+0000 7fe8ad7fa700 10 client.15780 handle_command_reply: tid=0 2023-07-18T03:08:31.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.739+0000 7fe8dfc13b80 1 client.15780 shutdown 2023-07-18T03:08:31.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.739+0000 7fe8dfc13b80 1 -- 172.21.15.101:0/2389166290 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569c3d262d0 msgr2=0x5569c3d31520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:31.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.739+0000 7fe8dfc13b80 1 --2- 172.21.15.101:0/2389166290 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569c3d262d0 0x5569c3d31520 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe8c00098e0 tx=0x7fe8c000f690 comp rx=0 tx=0).stop 2023-07-18T03:08:31.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.739+0000 7fe8dfc13b80 1 -- 172.21.15.101:0/2389166290 shutdown_connections 2023-07-18T03:08:31.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.739+0000 7fe8dfc13b80 1 --2- 172.21.15.101:0/2389166290 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5569c3d40f40 0x5569c3d43320 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.739+0000 7fe8dfc13b80 1 --2- 172.21.15.101:0/2389166290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569c3d31ad0 0x5569c3d2ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.739+0000 7fe8ad7fa700 0 client.15780 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:08:31.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.739+0000 7fe8dfc13b80 1 --2- 172.21.15.101:0/2389166290 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569c3d262d0 0x5569c3d31520 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.739+0000 7fe8dfc13b80 1 --2- 172.21.15.101:0/2389166290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569c3d26be0 0x5569c3d2e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.739+0000 7fe8dfc13b80 1 -- 172.21.15.101:0/2389166290 >> 172.21.15.101:0/2389166290 conn(0x5569c3d1cda0 msgr2=0x5569c3d22230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:08:31.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.739+0000 7fe8dfc13b80 1 -- 172.21.15.101:0/2389166290 shutdown_connections 2023-07-18T03:08:31.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.739+0000 7fe8dfc13b80 1 -- 172.21.15.101:0/2389166290 wait complete. 2023-07-18T03:08:31.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.739+0000 7fe8dfc13b80 20 client.15780 trim_cache size 0 max 16384 2023-07-18T03:08:31.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.756+0000 7fe8dfc13b80 1 Processor -- start 2023-07-18T03:08:31.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.756+0000 7fe8dfc13b80 1 -- start start 2023-07-18T03:08:31.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.757+0000 7fe8dfc13b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569c3d262d0 0x5569c3d31050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:31.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.757+0000 7fe8dfc13b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569c3d26be0 0x5569c3d2b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:31.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.757+0000 7fe8dfc13b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569c3d31ad0 0x5569c3d2bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:31.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.757+0000 7fe8dfc13b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569c3bd24f0 con 0x5569c3d31ad0 2023-07-18T03:08:31.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.757+0000 7fe8dfc13b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569c3d1eec0 con 0x5569c3d26be0 2023-07-18T03:08:31.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.757+0000 7fe8dfc13b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569c3d27540 con 0x5569c3d262d0 2023-07-18T03:08:31.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.757+0000 7fe8ca79c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569c3d31ad0 0x5569c3d2bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:31.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.757+0000 7fe8ca79c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569c3d31ad0 0x5569c3d2bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:56890/0 (socket says 172.21.15.101:56890) 2023-07-18T03:08:31.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.757+0000 7fe8c9f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569c3d262d0 0x5569c3d31050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:31.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.757+0000 7fe8ca79c700 1 -- 172.21.15.101:0/3767733620 learned_addr learned my addr 172.21.15.101:0/3767733620 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:08:31.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.757+0000 7fe8c979a700 1 --2- 172.21.15.101:0/3767733620 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569c3d26be0 0x5569c3d2b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:31.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.757+0000 7fe8c9f9b700 1 -- 172.21.15.101:0/3767733620 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569c3d26be0 msgr2=0x5569c3d2b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:08:31.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.758+0000 7fe8c9f9b700 1 --2- 172.21.15.101:0/3767733620 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569c3d26be0 0x5569c3d2b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.758+0000 7fe8c9f9b700 1 -- 172.21.15.101:0/3767733620 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569c3d31ad0 msgr2=0x5569c3d2bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:08:31.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.758+0000 7fe8c9f9b700 1 --2- 172.21.15.101:0/3767733620 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569c3d31ad0 0x5569c3d2bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.758+0000 7fe8c9f9b700 1 -- 172.21.15.101:0/3767733620 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569c3a8fc60 con 0x5569c3d262d0 2023-07-18T03:08:31.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.758+0000 7fe8ca79c700 1 --2- 172.21.15.101:0/3767733620 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569c3d31ad0 0x5569c3d2bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:08:31.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.758+0000 7fe8c9f9b700 1 --2- 172.21.15.101:0/3767733620 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569c3d262d0 0x5569c3d31050 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0009990 tx=0x7fe8c0009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69e10d4d44ddf41b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:08:31.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.758+0000 7fe8b8ff9700 1 -- 172.21.15.101:0/3767733620 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c00224a0 con 0x5569c3d262d0 2023-07-18T03:08:31.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.758+0000 7fe8b8ff9700 1 -- 172.21.15.101:0/3767733620 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8c00058e0 con 0x5569c3d262d0 2023-07-18T03:08:31.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.758+0000 7fe8b8ff9700 1 -- 172.21.15.101:0/3767733620 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c0004500 con 0x5569c3d262d0 2023-07-18T03:08:31.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.759+0000 7fe8dfc13b80 1 -- 172.21.15.101:0/3767733620 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569c3d262d0 msgr2=0x5569c3d31050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:31.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.759+0000 7fe8dfc13b80 1 --2- 172.21.15.101:0/3767733620 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569c3d262d0 0x5569c3d31050 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0009990 tx=0x7fe8c0009310 comp rx=0 tx=0).stop 2023-07-18T03:08:31.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.759+0000 7fe8dfc13b80 1 -- 172.21.15.101:0/3767733620 shutdown_connections 2023-07-18T03:08:31.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.759+0000 7fe8dfc13b80 1 --2- 172.21.15.101:0/3767733620 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569c3d31ad0 0x5569c3d2bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.759+0000 7fe8dfc13b80 1 --2- 172.21.15.101:0/3767733620 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569c3d262d0 0x5569c3d31050 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.759+0000 7fe8dfc13b80 1 --2- 172.21.15.101:0/3767733620 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569c3d26be0 0x5569c3d2b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.759+0000 7fe8dfc13b80 1 -- 172.21.15.101:0/3767733620 >> 172.21.15.101:0/3767733620 conn(0x5569c3d1cda0 msgr2=0x5569c3d3d760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:08:31.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.759+0000 7fe8dfc13b80 1 -- 172.21.15.101:0/3767733620 shutdown_connections 2023-07-18T03:08:31.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.759+0000 7fe8dfc13b80 1 -- 172.21.15.101:0/3767733620 wait complete. 2023-07-18T03:08:31.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.759+0000 7fe8dfc13b80 1 -- start start 2023-07-18T03:08:31.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.760+0000 7fe8dfc13b80 1 Processor -- start 2023-07-18T03:08:31.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.760+0000 7fe8dfc13b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569c3d262d0 0x5569c3d38350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:31.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.760+0000 7fe8dfc13b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569c3d26be0 0x5569c3d36680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:31.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.760+0000 7fe8dfc13b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569c3d31ad0 0x5569c3d36bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:31.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.760+0000 7fe8dfc13b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569c3d37100 con 0x5569c3d31ad0 2023-07-18T03:08:31.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.760+0000 7fe8dfc13b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569c3d37270 con 0x5569c3d262d0 2023-07-18T03:08:31.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.760+0000 7fe8dfc13b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569c3d37430 con 0x5569c3d26be0 2023-07-18T03:08:31.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.760+0000 7fe8dfc13b80 10 client.0 _ll_register_callbacks cb 0x7fe8bc009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:08:31.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.760+0000 7fe8ca79c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569c3d31ad0 0x5569c3d36bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:31.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.760+0000 7fe8ca79c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569c3d31ad0 0x5569c3d36bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:56892/0 (socket says 172.21.15.101:56892) 2023-07-18T03:08:31.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.760+0000 7fe8ca79c700 1 -- 172.21.15.101:0/855992671 learned_addr learned my addr 172.21.15.101:0/855992671 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:08:31.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.760+0000 7fe8c9f9b700 1 --2- 172.21.15.101:0/855992671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569c3d262d0 0x5569c3d38350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:31.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.760+0000 7fe8c979a700 1 --2- 172.21.15.101:0/855992671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569c3d26be0 0x5569c3d36680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:31.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.761+0000 7fe8ca79c700 1 -- 172.21.15.101:0/855992671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569c3d26be0 msgr2=0x5569c3d36680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:08:31.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.761+0000 7fe8ca79c700 1 --2- 172.21.15.101:0/855992671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569c3d26be0 0x5569c3d36680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.761+0000 7fe8ca79c700 1 -- 172.21.15.101:0/855992671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569c3d262d0 msgr2=0x5569c3d38350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:08:31.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.761+0000 7fe8ca79c700 1 --2- 172.21.15.101:0/855992671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569c3d262d0 0x5569c3d38350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.761+0000 7fe8ca79c700 1 -- 172.21.15.101:0/855992671 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c410d220 con 0x5569c3d31ad0 2023-07-18T03:08:31.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.761+0000 7fe8c979a700 1 --2- 172.21.15.101:0/855992671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569c3d26be0 0x5569c3d36680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:08:31.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.761+0000 7fe8c9f9b700 1 --2- 172.21.15.101:0/855992671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569c3d262d0 0x5569c3d38350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:08:31.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.761+0000 7fe8ca79c700 1 --2- 172.21.15.101:0/855992671 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569c3d31ad0 0x5569c3d36bc0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc01aa20 tx=0x7fe8bc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d901b66b9adb2ca8 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:08:31.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.761+0000 7fe8ad7fa700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:08:31.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.761+0000 7fe8ad7fa700 1 -- 172.21.15.101:0/855992671 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc00e070 con 0x5569c3d31ad0 2023-07-18T03:08:31.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.761+0000 7fe8dfc13b80 1 -- 172.21.15.101:0/855992671 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe8c001e040 con 0x5569c3d31ad0 2023-07-18T03:08:31.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.761+0000 7fe8ad7fa700 1 -- 172.21.15.101:0/855992671 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fe8bc004440 con 0x5569c3d31ad0 2023-07-18T03:08:31.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.762+0000 7fe8ad7fa700 1 -- 172.21.15.101:0/855992671 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8bc003e60 con 0x5569c3d31ad0 2023-07-18T03:08:31.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.762+0000 7fe8dfc13b80 10 client.15786 fetch_fsmap learned FSMap version 24 2023-07-18T03:08:31.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.762+0000 7fe8dfc13b80 1 -- 172.21.15.101:0/855992671 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5569c3acd250 con 0x5569c3d31ad0 2023-07-18T03:08:31.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.762+0000 7fe8ad7fa700 1 -- 172.21.15.101:0/855992671 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8c001e040 con 0x5569c3d31ad0 2023-07-18T03:08:31.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.762+0000 7fe8ad7fa700 1 -- 172.21.15.101:0/855992671 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe8bc029c70 con 0x5569c3d31ad0 2023-07-18T03:08:31.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.762+0000 7fe8dfc13b80 10 client.15786 fetch_fsmap finished waiting for FSMap version 24 2023-07-18T03:08:31.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.762+0000 7fe8dfc13b80 10 client.15786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:08:31.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.762+0000 7fe8dfc13b80 20 client.15786 populate_metadata read hostname 'smithi101' 2023-07-18T03:08:31.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.762+0000 7fe8dfc13b80 1 --2- 172.21.15.101:0/855992671 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5569c3d96200 0x5569c3d985e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:08:31.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.762+0000 7fe8dfc13b80 4 client.15786 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:08:31.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.762+0000 7fe8dfc13b80 1 -- 172.21.15.101:0/855992671 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5569c3c34050 con 0x5569c3d96200 2023-07-18T03:08:31.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.763+0000 7fe8c9f9b700 1 --2- 172.21.15.101:0/855992671 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5569c3d96200 0x5569c3d985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:08:31.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.763+0000 7fe8ad7fa700 10 client.15786 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:08:31.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.763+0000 7fe8c9f9b700 1 --2- 172.21.15.101:0/855992671 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5569c3d96200 0x5569c3d985e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3633ff8946cbf525 server_cookie=5b288d3c9a006ebd in_seq=0 out_seq=0 2023-07-18T03:08:31.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.763+0000 7fe8ad7fa700 1 -- 172.21.15.101:0/855992671 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5569c3c34050 con 0x5569c3d96200 2023-07-18T03:08:31.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.763+0000 7fe8ad7fa700 10 client.15786 handle_command_reply: tid=0 2023-07-18T03:08:31.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.763+0000 7fe8dfc13b80 1 client.15786 shutdown 2023-07-18T03:08:31.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.764+0000 7fe8dfc13b80 1 -- 172.21.15.101:0/855992671 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569c3d31ad0 msgr2=0x5569c3d36bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:31.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.764+0000 7fe8dfc13b80 1 --2- 172.21.15.101:0/855992671 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569c3d31ad0 0x5569c3d36bc0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc01aa20 tx=0x7fe8bc004ad0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.764+0000 7fe8dfc13b80 1 -- 172.21.15.101:0/855992671 shutdown_connections 2023-07-18T03:08:31.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.764+0000 7fe8dfc13b80 1 --2- 172.21.15.101:0/855992671 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5569c3d96200 0x5569c3d985e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.764+0000 7fe8dfc13b80 1 --2- 172.21.15.101:0/855992671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569c3d262d0 0x5569c3d38350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.764+0000 7fe8dfc13b80 1 --2- 172.21.15.101:0/855992671 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5569c3d31ad0 0x5569c3d36bc0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.764+0000 7fe8dfc13b80 1 --2- 172.21.15.101:0/855992671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569c3d26be0 0x5569c3d36680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.764+0000 7fe8dfc13b80 1 -- 172.21.15.101:0/855992671 >> 172.21.15.101:0/855992671 conn(0x5569c3d1cda0 msgr2=0x5569c3d283e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:08:31.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.764+0000 7fe8ad7fa700 0 client.15786 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:08:31.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.764+0000 7fe8dfc13b80 1 -- 172.21.15.101:0/855992671 shutdown_connections 2023-07-18T03:08:31.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.764+0000 7fe8dfc13b80 1 -- 172.21.15.101:0/855992671 wait complete. 2023-07-18T03:08:31.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.764+0000 7fe8dfc13b80 20 client.15786 trim_cache size 0 max 16384 2023-07-18T03:08:31.768 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:08:31.768 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:08:31.768 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:08:31.768 INFO:teuthology.orchestra.run.smithi101.stdout: "2b06628a-eab7-4b15-8803-1c11ca43fa88": { 2023-07-18T03:08:31.768 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:08:31.768 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "2b06628a-eab7-4b15-8803-1c11ca43fa88", 2023-07-18T03:08:31.768 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:08:31.769 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:08:31.769 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:08:31.769 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:08:31.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.764+0000 7fe8cc1ff700 1 -- 172.21.15.101:0/1775391099 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fe8b00a13e0 msgr2=0x7fe8b00a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:31.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.764+0000 7fe8cc1ff700 1 --2- 172.21.15.101:0/1775391099 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fe8b00a13e0 0x7fe8b00a3890 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc009ec0 tx=0x7fe8bc00b040 comp rx=0 tx=0).stop 2023-07-18T03:08:31.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.764+0000 7fe8cc1ff700 1 -- 172.21.15.101:0/1775391099 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8c4100490 msgr2=0x7fe8c4198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:08:31.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.764+0000 7fe8cc1ff700 1 --2- 172.21.15.101:0/1775391099 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8c4100490 0x7fe8c4198370 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe8b400e9a0 tx=0x7fe8b400ecb0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.765+0000 7fe8cc1ff700 1 -- 172.21.15.101:0/1775391099 shutdown_connections 2023-07-18T03:08:31.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.765+0000 7fe8cc1ff700 1 --2- 172.21.15.101:0/1775391099 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fe8b00a13e0 0x7fe8b00a3890 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.765+0000 7fe8cc1ff700 1 --2- 172.21.15.101:0/1775391099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8c40ffae0 0x7fe8c4197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.765+0000 7fe8cc1ff700 1 --2- 172.21.15.101:0/1775391099 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe8c4105af0 0x7fe8c419c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.765+0000 7fe8cc1ff700 1 --2- 172.21.15.101:0/1775391099 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8c4100490 0x7fe8c4198370 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:08:31.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.765+0000 7fe8cc1ff700 1 -- 172.21.15.101:0/1775391099 >> 172.21.15.101:0/1775391099 conn(0x7fe8c40fb5e0 msgr2=0x7fe8c410ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:08:31.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.765+0000 7fe8cc1ff700 1 -- 172.21.15.101:0/1775391099 shutdown_connections 2023-07-18T03:08:31.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:08:31.765+0000 7fe8cc1ff700 1 -- 172.21.15.101:0/1775391099 wait complete. 2023-07-18T03:08:31.772 INFO:tasks.cephfs.filesystem:scrub status for tag:2b06628a-eab7-4b15-8803-1c11ca43fa88 - {'path': '/', 'tag': '2b06628a-eab7-4b15-8803-1c11ca43fa88', 'options': 'recursive,force'} 2023-07-18T03:08:33.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:08:32 smithi179 ceph-mon[129229]: scrub queued for path: / 2023-07-18T03:08:33.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:08:32 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:08:33.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:08:32 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:08:33.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:08:32 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:08:33.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:08:32 smithi179 ceph-mon[129229]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:08:33.233 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:08:32 smithi101 ceph-mon[126379]: scrub queued for path: / 2023-07-18T03:08:33.233 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:08:32 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:08:33.233 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:08:32 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:08:33.233 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:08:32 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:08:33.233 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:08:32 smithi101 ceph-mon[126379]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:08:33.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:08:32 smithi155 ceph-mon[128885]: scrub queued for path: / 2023-07-18T03:08:33.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:08:32 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:08:33.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:08:32 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:08:33.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:08:32 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:08:33.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:08:32 smithi155 ceph-mon[128885]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:08:33.534 INFO:tasks.workunit.client.0.smithi101.stderr:Updating files: 64% (7543/11710) Updating files: 65% (7612/11710) Updating files: 66% (7729/11710) Updating files: 67% (7846/11710) Updating files: 68% (7963/11710) Updating files: 69% (8080/11710) Updating files: 70% (8197/11710) Updating files: 71% (8315/11710) Updating files: 72% (8432/11710) Updating files: 73% (8549/11710) Updating files: 74% (8666/11710) Updating files: 75% (8783/11710) Updating files: 76% (8900/11710) Updating files: 77% (9017/11710) Updating files: 78% (9134/11710) Updating files: 79% (9251/11710) Updating files: 80% (9368/11710) Updating files: 81% (9486/11710) Updating files: 82% (9603/11710) Updating files: 83% (9720/11710) Updating files: 84% (9837/11710) Updating files: 85% (9954/11710) Updating files: 86% (10071/11710) Updating files: 87% (10188/11710) Updating files: 88% (10305/11710) Updating files: 89% (10422/11710) Updating files: 90% (10539/11710) Updating files: 91% (10657/11710) Updating files: 92% (10774/11710) Updating files: 93% (10891/11710) Updating files: 94% (11008/11710) Updating files: 95% (11125/11710) Updating files: 96% (11242/11710) Updating files: 97% (11359/11710) Updating files: 98% (11476/11710) Updating files: 99% (11593/11710) Updating files: 100% (11710/11710) Updating files: 100% (11710/11710), done. 2023-07-18T03:08:34.329 INFO:tasks.workunit.client.0.smithi101.stderr:Note: switching to '81cf785aa7781c4e08381c540f890cb9bc987ef9'. 2023-07-18T03:08:34.329 INFO:tasks.workunit.client.0.smithi101.stderr: 2023-07-18T03:08:34.329 INFO:tasks.workunit.client.0.smithi101.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-07-18T03:08:34.330 INFO:tasks.workunit.client.0.smithi101.stderr:changes and commit them, and you can discard any commits you make in this 2023-07-18T03:08:34.330 INFO:tasks.workunit.client.0.smithi101.stderr:state without impacting any branches by switching back to a branch. 2023-07-18T03:08:34.330 INFO:tasks.workunit.client.0.smithi101.stderr: 2023-07-18T03:08:34.330 INFO:tasks.workunit.client.0.smithi101.stderr:If you want to create a new branch to retain commits you create, you may 2023-07-18T03:08:34.330 INFO:tasks.workunit.client.0.smithi101.stderr:do so (now or later) by using -c with the switch command. Example: 2023-07-18T03:08:34.330 INFO:tasks.workunit.client.0.smithi101.stderr: 2023-07-18T03:08:34.330 INFO:tasks.workunit.client.0.smithi101.stderr: git switch -c 2023-07-18T03:08:34.331 INFO:tasks.workunit.client.0.smithi101.stderr: 2023-07-18T03:08:34.331 INFO:tasks.workunit.client.0.smithi101.stderr:Or undo this operation with: 2023-07-18T03:08:34.331 INFO:tasks.workunit.client.0.smithi101.stderr: 2023-07-18T03:08:34.331 INFO:tasks.workunit.client.0.smithi101.stderr: git switch - 2023-07-18T03:08:34.331 INFO:tasks.workunit.client.0.smithi101.stderr: 2023-07-18T03:08:34.331 INFO:tasks.workunit.client.0.smithi101.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-07-18T03:08:34.331 INFO:tasks.workunit.client.0.smithi101.stderr: 2023-07-18T03:08:34.332 INFO:tasks.workunit.client.0.smithi101.stderr:HEAD is now at 81cf785aa77 Merge remote-tracking branch 'temp/wip-61599' into wip-rishabh-2023Jul13-m-quota 2023-07-18T03:08:34.344 DEBUG:teuthology.orchestra.run.smithi101:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-07-18T03:08:34.396 INFO:tasks.workunit.client.0.smithi101.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-07-18T03:08:34.401 INFO:tasks.workunit.client.0.smithi101.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-07-18T03:08:34.401 INFO:tasks.workunit.client.0.smithi101.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-07-18T03:08:35.026 INFO:tasks.workunit.client.0.smithi101.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-07-18T03:08:35.166 INFO:tasks.workunit.client.0.smithi101.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-07-18T03:08:35.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:08:34 smithi179 ceph-mon[129229]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:08:35.230 INFO:tasks.workunit.client.0.smithi101.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-07-18T03:08:35.232 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:08:34 smithi101 ceph-mon[126379]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:08:35.233 INFO:tasks.workunit.client.0.smithi101.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-07-18T03:08:35.234 INFO:tasks.workunit.client.0.smithi101.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-07-18T03:08:35.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:08:34 smithi155 ceph-mon[128885]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:08:35.296 INFO:tasks.workunit.client.0.smithi101.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-07-18T03:08:35.303 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-07-18T03:08:35.303 DEBUG:teuthology.orchestra.run.smithi101:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-07-18T03:08:35.346 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2023-07-18T03:08:35.347 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2023-07-18T03:08:35.347 DEBUG:teuthology.orchestra.run.smithi101:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=81cf785aa7781c4e08381c540f890cb9bc987ef9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2023-07-18T03:08:35.443 INFO:tasks.workunit.client.0.smithi101.stderr:+ dbench 1 2023-07-18T03:08:35.457 INFO:tasks.workunit.client.0.smithi101.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2023-07-18T03:08:35.457 INFO:tasks.workunit.client.0.smithi101.stdout: 2023-07-18T03:08:35.457 INFO:tasks.workunit.client.0.smithi101.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2023-07-18T03:08:35.458 INFO:tasks.workunit.client.0.smithi101.stdout:0 of 1 processes prepared for launch 0 sec 2023-07-18T03:08:35.558 INFO:tasks.workunit.client.0.smithi101.stdout:1 of 1 processes prepared for launch 0 sec 2023-07-18T03:08:35.559 INFO:tasks.workunit.client.0.smithi101.stdout:releasing clients 2023-07-18T03:08:36.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:08:35 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:08:36.233 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:08:35 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:08:36.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:08:35 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:08:36.559 INFO:tasks.workunit.client.0.smithi101.stdout: 1 3169 34.12 MB/sec warmup 1 sec latency 5.931 ms 2023-07-18T03:08:37.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:08:36 smithi179 ceph-mon[129229]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-07-18T03:08:37.233 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:08:36 smithi101 ceph-mon[126379]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-07-18T03:08:37.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:08:36 smithi155 ceph-mon[128885]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-07-18T03:08:37.559 INFO:tasks.workunit.client.0.smithi101.stdout: 1 5959 23.54 MB/sec warmup 2 sec latency 6.624 ms 2023-07-18T03:08:38.559 INFO:tasks.workunit.client.0.smithi101.stdout: 1 8387 20.46 MB/sec warmup 3 sec latency 6.061 ms 2023-07-18T03:08:39.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:08:38 smithi179 ceph-mon[129229]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-07-18T03:08:39.233 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:08:38 smithi101 ceph-mon[126379]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-07-18T03:08:39.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:08:38 smithi155 ceph-mon[128885]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-07-18T03:08:39.559 INFO:tasks.workunit.client.0.smithi101.stdout: 1 11429 19.68 MB/sec warmup 4 sec latency 8.126 ms 2023-07-18T03:08:40.559 INFO:tasks.workunit.client.0.smithi101.stdout: 1 14566 19.48 MB/sec warmup 5 sec latency 15.952 ms 2023-07-18T03:08:41.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:08:40 smithi179 ceph-mon[129229]: pgmap v36: 129 pgs: 129 active+clean; 6.0 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 447 KiB/s wr, 68 op/s 2023-07-18T03:08:41.233 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:08:40 smithi101 ceph-mon[126379]: pgmap v36: 129 pgs: 129 active+clean; 6.0 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 447 KiB/s wr, 68 op/s 2023-07-18T03:08:41.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:08:40 smithi155 ceph-mon[128885]: pgmap v36: 129 pgs: 129 active+clean; 6.0 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 447 KiB/s wr, 68 op/s 2023-07-18T03:08:41.559 INFO:tasks.workunit.client.0.smithi101.stdout: 1 17584 18.66 MB/sec warmup 6 sec latency 9.798 ms 2023-07-18T03:08:42.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:08:41 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:08:42.233 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:08:41 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:08:42.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:08:41 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:08:42.559 INFO:tasks.workunit.client.0.smithi101.stdout: 1 20112 17.64 MB/sec warmup 7 sec latency 10.340 ms 2023-07-18T03:08:43.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:08:42 smithi179 ceph-mon[129229]: pgmap v37: 129 pgs: 129 active+clean; 6.8 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 530 KiB/s wr, 71 op/s 2023-07-18T03:08:43.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:08:42 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:08:43.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:08:42 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:08:43.233 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:08:42 smithi101 ceph-mon[126379]: pgmap v37: 129 pgs: 129 active+clean; 6.8 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 530 KiB/s wr, 71 op/s 2023-07-18T03:08:43.233 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:08:42 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:08:43.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:08:42 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:08:43.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:08:42 smithi155 ceph-mon[128885]: pgmap v37: 129 pgs: 129 active+clean; 6.8 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 530 KiB/s wr, 71 op/s 2023-07-18T03:08:43.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:08:42 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:08:43.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:08:42 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:08:43.559 INFO:tasks.workunit.client.0.smithi101.stdout: 1 22488 17.22 MB/sec warmup 8 sec latency 9.196 ms 2023-07-18T03:08:44.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:08:43 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:08:44.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:08:43 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:08:44.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:08:43 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:08:44.233 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:08:43 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:08:44.233 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:08:43 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:08:44.233 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:08:43 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:08:44.290 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:08:43 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:08:44.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:08:43 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:08:44.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:08:43 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:08:44.559 INFO:tasks.workunit.client.0.smithi101.stdout: 1 25494 17.21 MB/sec warmup 9 sec latency 12.545 ms 2023-07-18T03:08:45.445 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:08:45 smithi179 ceph-mon[129229]: pgmap v38: 129 pgs: 129 active+clean; 6.8 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 530 KiB/s wr, 71 op/s 2023-07-18T03:08:45.483 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:08:45 smithi101 ceph-mon[126379]: pgmap v38: 129 pgs: 129 active+clean; 6.8 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 530 KiB/s wr, 71 op/s 2023-07-18T03:08:45.540 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:08:45 smithi155 ceph-mon[128885]: pgmap v38: 129 pgs: 129 active+clean; 6.8 MiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 530 KiB/s wr, 71 op/s 2023-07-18T03:08:45.559 INFO:tasks.workunit.client.0.smithi101.stdout: 1 28562 17.27 MB/sec warmup 10 sec latency 17.117 ms 2023-07-18T03:08:46.560 INFO:tasks.workunit.client.0.smithi101.stdout: 1 31025 16.55 MB/sec warmup 11 sec latency 6.402 ms 2023-07-18T03:08:47.445 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:08:47 smithi179 ceph-mon[129229]: pgmap v39: 129 pgs: 129 active+clean; 13 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 1.1 MiB/s wr, 155 op/s 2023-07-18T03:08:47.483 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:08:47 smithi101 ceph-mon[126379]: pgmap v39: 129 pgs: 129 active+clean; 13 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 1.1 MiB/s wr, 155 op/s 2023-07-18T03:08:47.541 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:08:47 smithi155 ceph-mon[128885]: pgmap v39: 129 pgs: 129 active+clean; 13 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 1.1 MiB/s wr, 155 op/s 2023-07-18T03:08:47.560 INFO:tasks.workunit.client.0.smithi101.stdout: 1 32573 16.09 MB/sec warmup 12 sec latency 195.208 ms 2023-07-18T03:08:48.560 INFO:tasks.workunit.client.0.smithi101.stdout: 1 35582 16.21 MB/sec warmup 13 sec latency 65.043 ms 2023-07-18T03:08:49.445 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:08:49 smithi179 ceph-mon[129229]: pgmap v40: 129 pgs: 129 active+clean; 13 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 1.1 MiB/s wr, 153 op/s 2023-07-18T03:08:49.483 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:08:49 smithi101 ceph-mon[126379]: pgmap v40: 129 pgs: 129 active+clean; 13 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 1.1 MiB/s wr, 153 op/s 2023-07-18T03:08:49.540 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:08:49 smithi155 ceph-mon[128885]: pgmap v40: 129 pgs: 129 active+clean; 13 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 1.1 MiB/s wr, 153 op/s 2023-07-18T03:08:49.560 INFO:tasks.workunit.client.0.smithi101.stdout: 1 38065 15.73 MB/sec warmup 14 sec latency 28.118 ms 2023-07-18T03:08:50.560 INFO:tasks.workunit.client.0.smithi101.stdout: 1 40390 15.65 MB/sec warmup 15 sec latency 6.557 ms 2023-07-18T03:08:51.446 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:08:51 smithi179 ceph-mon[129229]: pgmap v41: 129 pgs: 129 active+clean; 19 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 1.7 MiB/s wr, 228 op/s 2023-07-18T03:08:51.483 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:08:51 smithi101 ceph-mon[126379]: pgmap v41: 129 pgs: 129 active+clean; 19 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 1.7 MiB/s wr, 228 op/s 2023-07-18T03:08:51.540 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:08:51 smithi155 ceph-mon[128885]: pgmap v41: 129 pgs: 129 active+clean; 19 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 1.7 MiB/s wr, 228 op/s 2023-07-18T03:08:51.560 INFO:tasks.workunit.client.0.smithi101.stdout: 1 43274 15.67 MB/sec warmup 16 sec latency 12.753 ms 2023-07-18T03:08:52.560 INFO:tasks.workunit.client.0.smithi101.stdout: 1 46255 15.78 MB/sec warmup 17 sec latency 35.601 ms 2023-07-18T03:08:53.445 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:08:53 smithi179 ceph-mon[129229]: pgmap v42: 129 pgs: 129 active+clean; 19 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 1.4 MiB/s wr, 169 op/s 2023-07-18T03:08:53.483 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:08:53 smithi101 ceph-mon[126379]: pgmap v42: 129 pgs: 129 active+clean; 19 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 1.4 MiB/s wr, 169 op/s 2023-07-18T03:08:53.540 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:08:53 smithi155 ceph-mon[128885]: pgmap v42: 129 pgs: 129 active+clean; 19 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 1.4 MiB/s wr, 169 op/s 2023-07-18T03:08:53.561 INFO:tasks.workunit.client.0.smithi101.stdout: 1 47815 15.27 MB/sec warmup 18 sec latency 140.086 ms 2023-07-18T03:08:54.560 INFO:tasks.workunit.client.0.smithi101.stdout: 1 50494 15.22 MB/sec warmup 19 sec latency 6.253 ms 2023-07-18T03:08:55.446 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:08:55 smithi179 ceph-mon[129229]: pgmap v43: 129 pgs: 129 active+clean; 19 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2023-07-18T03:08:55.483 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:08:55 smithi101 ceph-mon[126379]: pgmap v43: 129 pgs: 129 active+clean; 19 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2023-07-18T03:08:55.541 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:08:55 smithi155 ceph-mon[128885]: pgmap v43: 129 pgs: 129 active+clean; 19 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2023-07-18T03:08:55.560 INFO:tasks.workunit.client.0.smithi101.stdout: 1 53530 15.37 MB/sec warmup 20 sec latency 7.361 ms 2023-07-18T03:08:56.560 INFO:tasks.workunit.client.0.smithi101.stdout: 1 55997 15.18 MB/sec warmup 21 sec latency 7.812 ms 2023-07-18T03:08:57.446 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:08:57 smithi179 ceph-mon[129229]: pgmap v44: 129 pgs: 129 active+clean; 25 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 2.0 MiB/s wr, 245 op/s 2023-07-18T03:08:57.483 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:08:57 smithi101 ceph-mon[126379]: pgmap v44: 129 pgs: 129 active+clean; 25 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 2.0 MiB/s wr, 245 op/s 2023-07-18T03:08:57.540 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:08:57 smithi155 ceph-mon[128885]: pgmap v44: 129 pgs: 129 active+clean; 25 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 2.0 MiB/s wr, 245 op/s 2023-07-18T03:08:57.560 INFO:tasks.workunit.client.0.smithi101.stdout: 1 57694 14.89 MB/sec warmup 22 sec latency 100.111 ms 2023-07-18T03:08:58.560 INFO:tasks.workunit.client.0.smithi101.stdout: 1 60567 15.03 MB/sec warmup 23 sec latency 79.287 ms 2023-07-18T03:08:59.446 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:08:59 smithi179 ceph-mon[129229]: pgmap v45: 129 pgs: 129 active+clean; 25 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2023-07-18T03:08:59.483 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:08:59 smithi101 ceph-mon[126379]: pgmap v45: 129 pgs: 129 active+clean; 25 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2023-07-18T03:08:59.541 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:08:59 smithi155 ceph-mon[128885]: pgmap v45: 129 pgs: 129 active+clean; 25 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2023-07-18T03:08:59.560 INFO:tasks.workunit.client.0.smithi101.stdout: 1 62829 14.74 MB/sec warmup 24 sec latency 9.903 ms 2023-07-18T03:09:00.561 INFO:tasks.workunit.client.0.smithi101.stdout: 1 64877 14.64 MB/sec warmup 25 sec latency 26.494 ms 2023-07-18T03:09:01.446 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:01 smithi179 ceph-mon[129229]: pgmap v46: 129 pgs: 129 active+clean; 31 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 236 op/s 2023-07-18T03:09:01.483 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:01 smithi101 ceph-mon[126379]: pgmap v46: 129 pgs: 129 active+clean; 31 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 236 op/s 2023-07-18T03:09:01.541 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:01 smithi155 ceph-mon[128885]: pgmap v46: 129 pgs: 129 active+clean; 31 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 236 op/s 2023-07-18T03:09:01.561 INFO:tasks.workunit.client.0.smithi101.stdout: 1 65270 14.24 MB/sec warmup 26 sec latency 371.268 ms 2023-07-18T03:09:01.773 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:09:01.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.988+0000 7f24161ff700 1 -- 172.21.15.101:0/660714429 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2410105af0 msgr2=0x7f2410105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:01.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.988+0000 7f24161ff700 1 --2- 172.21.15.101:0/660714429 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2410105af0 0x7f2410105ec0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2400009990 tx=0x7f2400009ca0 comp rx=0 tx=0).stop 2023-07-18T03:09:01.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.988+0000 7f24161ff700 1 -- 172.21.15.101:0/660714429 shutdown_connections 2023-07-18T03:09:01.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.988+0000 7f24161ff700 1 --2- 172.21.15.101:0/660714429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2410100490 0x7f241010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:01.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.988+0000 7f24161ff700 1 --2- 172.21.15.101:0/660714429 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2410105af0 0x7f2410105ec0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:01.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.988+0000 7f24161ff700 1 --2- 172.21.15.101:0/660714429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24100ffae0 0x7f24100fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:01.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.988+0000 7f24161ff700 1 -- 172.21.15.101:0/660714429 >> 172.21.15.101:0/660714429 conn(0x7f24100fb5e0 msgr2=0x7f24100fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:01.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.988+0000 7f24161ff700 1 -- 172.21.15.101:0/660714429 shutdown_connections 2023-07-18T03:09:01.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.988+0000 7f24161ff700 1 -- 172.21.15.101:0/660714429 wait complete. 2023-07-18T03:09:01.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.990+0000 7f24161ff700 1 Processor -- start 2023-07-18T03:09:01.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.990+0000 7f24161ff700 1 -- start start 2023-07-18T03:09:01.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.990+0000 7f24161ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24100ffae0 0x7f2410072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:01.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.990+0000 7f24161ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2410100490 0x7f2410072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:01.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.991+0000 7f24161ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2410105af0 0x7f24101a3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:01.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.991+0000 7f24161ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2410073580 con 0x7f2410100490 2023-07-18T03:09:01.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.991+0000 7f24161ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24101a4300 con 0x7f2410105af0 2023-07-18T03:09:01.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.991+0000 7f24161ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24101a4440 con 0x7f24100ffae0 2023-07-18T03:09:01.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.991+0000 7f240effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2410100490 0x7f2410072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:01.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.991+0000 7f240effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2410100490 0x7f2410072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:35132/0 (socket says 172.21.15.101:35132) 2023-07-18T03:09:01.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.991+0000 7f240effd700 1 -- 172.21.15.101:0/1485616014 learned_addr learned my addr 172.21.15.101:0/1485616014 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:01.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.991+0000 7f240f7fe700 1 --2- 172.21.15.101:0/1485616014 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24100ffae0 0x7f2410072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:01.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.991+0000 7f240ffff700 1 --2- 172.21.15.101:0/1485616014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2410105af0 0x7f24101a3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:01.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.991+0000 7f240effd700 1 -- 172.21.15.101:0/1485616014 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24100ffae0 msgr2=0x7f2410072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:01.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.991+0000 7f240effd700 1 --2- 172.21.15.101:0/1485616014 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24100ffae0 0x7f2410072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:01.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.991+0000 7f240effd700 1 -- 172.21.15.101:0/1485616014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2410105af0 msgr2=0x7f24101a3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:01.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.991+0000 7f240effd700 1 --2- 172.21.15.101:0/1485616014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2410105af0 0x7f24101a3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:01.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.991+0000 7f240effd700 1 -- 172.21.15.101:0/1485616014 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24000095f0 con 0x7f2410100490 2023-07-18T03:09:01.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.991+0000 7f240effd700 1 --2- 172.21.15.101:0/1485616014 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2410100490 0x7f2410072d50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f240400e9a0 tx=0x7f240400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:01.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.992+0000 7f240cff9700 1 -- 172.21.15.101:0/1485616014 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2404009790 con 0x7f2410100490 2023-07-18T03:09:01.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.992+0000 7f240cff9700 1 -- 172.21.15.101:0/1485616014 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2404010430 con 0x7f2410100490 2023-07-18T03:09:01.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.992+0000 7f24161ff700 1 -- 172.21.15.101:0/1485616014 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24101a46a0 con 0x7f2410100490 2023-07-18T03:09:01.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.992+0000 7f240cff9700 1 -- 172.21.15.101:0/1485616014 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2404018680 con 0x7f2410100490 2023-07-18T03:09:01.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.992+0000 7f24161ff700 1 -- 172.21.15.101:0/1485616014 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24101a4bd0 con 0x7f2410100490 2023-07-18T03:09:01.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.995+0000 7f240cff9700 1 -- 172.21.15.101:0/1485616014 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 24) v1 ==== 135949+0+0 (secure 0 0 0) 0x7f24040105a0 con 0x7f2410100490 2023-07-18T03:09:01.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.996+0000 7f2429beab80 1 Processor -- start 2023-07-18T03:09:01.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.996+0000 7f240cff9700 1 --2- 172.21.15.101:0/1485616014 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f23f80a1460 0x7f23f80a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:01.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.996+0000 7f2429beab80 1 -- start start 2023-07-18T03:09:01.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.996+0000 7f240cff9700 1 -- 172.21.15.101:0/1485616014 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f2404014070 con 0x7f2410100490 2023-07-18T03:09:01.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.997+0000 7f240ffff700 1 --2- 172.21.15.101:0/1485616014 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f23f80a1460 0x7f23f80a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:01.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.997+0000 7f2429beab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ded10362d0 0x55ded10366a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:01.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.997+0000 7f2429beab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ded1036be0 0x55ded1041590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:01.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.997+0000 7f2429beab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ded1041ad0 0x55ded1043eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:01.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.997+0000 7f2429beab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ded0f3ccb0 con 0x55ded1036be0 2023-07-18T03:09:01.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.997+0000 7f2429beab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ded0f8fa40 con 0x55ded10362d0 2023-07-18T03:09:01.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.997+0000 7f2429beab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ded0f88ed0 con 0x55ded1041ad0 2023-07-18T03:09:01.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.997+0000 7f240effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ded1036be0 0x55ded1041590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:01.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.997+0000 7f240effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ded1036be0 0x55ded1041590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:35136/0 (socket says 172.21.15.101:35136) 2023-07-18T03:09:01.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.997+0000 7f240effd700 1 -- 172.21.15.101:0/1552205190 learned_addr learned my addr 172.21.15.101:0/1552205190 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:01.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.997+0000 7f240f7fe700 1 --2- 172.21.15.101:0/1552205190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ded10362d0 0x55ded10366a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:01.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.997+0000 7f240ffff700 1 --2- 172.21.15.101:0/1485616014 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f23f80a1460 0x7f23f80a3910 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2408009550 tx=0x7f2408009130 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:01.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.997+0000 7f240ffff700 1 --2- 172.21.15.101:0/1552205190 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ded1041ad0 0x55ded1043eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:01.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.997+0000 7f240effd700 1 -- 172.21.15.101:0/1552205190 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ded1041ad0 msgr2=0x55ded1043eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:01.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.997+0000 7f240effd700 1 --2- 172.21.15.101:0/1552205190 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ded1041ad0 0x55ded1043eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:01.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.997+0000 7f240effd700 1 -- 172.21.15.101:0/1552205190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ded10362d0 msgr2=0x55ded10366a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:01.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.997+0000 7f240effd700 1 --2- 172.21.15.101:0/1552205190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ded10362d0 0x55ded10366a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:01.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.997+0000 7f240effd700 1 -- 172.21.15.101:0/1552205190 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ded0db6de0 con 0x55ded1036be0 2023-07-18T03:09:01.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.998+0000 7f240effd700 1 --2- 172.21.15.101:0/1552205190 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ded1036be0 0x55ded1041590 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2404099f50 tx=0x7f24040999f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27b1289ea0ff8554 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:01.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.998+0000 7f23f5ffb700 1 -- 172.21.15.101:0/1552205190 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24040a2410 con 0x55ded1036be0 2023-07-18T03:09:01.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.998+0000 7f23f5ffb700 1 -- 172.21.15.101:0/1552205190 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2404010880 con 0x55ded1036be0 2023-07-18T03:09:01.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.998+0000 7f23f5ffb700 1 -- 172.21.15.101:0/1552205190 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2404099760 con 0x55ded1036be0 2023-07-18T03:09:01.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.999+0000 7f2429beab80 1 -- 172.21.15.101:0/1552205190 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ded1036be0 msgr2=0x55ded1041590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:01.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.999+0000 7f2429beab80 1 --2- 172.21.15.101:0/1552205190 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ded1036be0 0x55ded1041590 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2404099f50 tx=0x7f24040999f0 comp rx=0 tx=0).stop 2023-07-18T03:09:01.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.999+0000 7f2429beab80 1 -- 172.21.15.101:0/1552205190 shutdown_connections 2023-07-18T03:09:01.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.999+0000 7f2429beab80 1 --2- 172.21.15.101:0/1552205190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ded10362d0 0x55ded10366a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:01.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.999+0000 7f2429beab80 1 --2- 172.21.15.101:0/1552205190 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ded1041ad0 0x55ded1043eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:01.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.999+0000 7f2429beab80 1 --2- 172.21.15.101:0/1552205190 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ded1036be0 0x55ded1041590 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:01.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.999+0000 7f2429beab80 1 -- 172.21.15.101:0/1552205190 >> 172.21.15.101:0/1552205190 conn(0x55ded102cda0 msgr2=0x55ded10326a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:01.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.999+0000 7f2429beab80 1 -- 172.21.15.101:0/1552205190 shutdown_connections 2023-07-18T03:09:01.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.999+0000 7f2429beab80 1 -- 172.21.15.101:0/1552205190 wait complete. 2023-07-18T03:09:01.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:01.999+0000 7f2429beab80 1 -- start start 2023-07-18T03:09:01.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.000+0000 7f2429beab80 1 Processor -- start 2023-07-18T03:09:01.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.000+0000 7f2429beab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ded10362d0 0x55ded1041530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:01.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.000+0000 7f2429beab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ded1036be0 0x55ded103e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:01.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.001+0000 7f2429beab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ded1041ad0 0x55ded103eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:01.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.001+0000 7f2429beab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ded0f28ac0 con 0x55ded1041ad0 2023-07-18T03:09:01.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.001+0000 7f2429beab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ded0f287c0 con 0x55ded10362d0 2023-07-18T03:09:01.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.001+0000 7f2429beab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ded0e0c920 con 0x55ded1036be0 2023-07-18T03:09:01.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.001+0000 7f2429beab80 10 client.0 _ll_register_callbacks cb 0x55ded0f26760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:09:01.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.001+0000 7f240ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ded1041ad0 0x55ded103eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:01.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.001+0000 7f240effd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ded1036be0 0x55ded103e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:01.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.001+0000 7f240effd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ded1036be0 0x55ded103e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:36774/0 (socket says 172.21.15.101:36774) 2023-07-18T03:09:01.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.001+0000 7f240ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ded1041ad0 0x55ded103eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:35142/0 (socket says 172.21.15.101:35142) 2023-07-18T03:09:01.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.001+0000 7f240effd700 1 -- 172.21.15.101:0/266885534 learned_addr learned my addr 172.21.15.101:0/266885534 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:01.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.001+0000 7f240f7fe700 1 --2- 172.21.15.101:0/266885534 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ded10362d0 0x55ded1041530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:01.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.001+0000 7f240effd700 1 -- 172.21.15.101:0/266885534 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ded10362d0 msgr2=0x55ded1041530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:01.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.001+0000 7f240effd700 1 --2- 172.21.15.101:0/266885534 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ded10362d0 0x55ded1041530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:01.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.001+0000 7f240effd700 1 -- 172.21.15.101:0/266885534 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ded1041ad0 msgr2=0x55ded103eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:01.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.001+0000 7f240effd700 1 --2- 172.21.15.101:0/266885534 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ded1041ad0 0x55ded103eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:01.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.001+0000 7f240effd700 1 -- 172.21.15.101:0/266885534 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24101a4bd0 con 0x55ded1036be0 2023-07-18T03:09:01.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.001+0000 7f240effd700 1 --2- 172.21.15.101:0/266885534 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ded1036be0 0x55ded103e7b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f24040a1ee0 tx=0x7f2404094e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4b3460918ac33db server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:01.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.001+0000 7f23eeffd700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:09:01.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.002+0000 7f23eeffd700 1 -- 172.21.15.101:0/266885534 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24040a2410 con 0x55ded1036be0 2023-07-18T03:09:01.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.002+0000 7f2429beab80 1 -- 172.21.15.101:0/266885534 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ded0ee24f0 con 0x55ded1036be0 2023-07-18T03:09:01.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.002+0000 7f23eeffd700 1 -- 172.21.15.101:0/266885534 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f24040996c0 con 0x55ded1036be0 2023-07-18T03:09:01.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.002+0000 7f23eeffd700 1 -- 172.21.15.101:0/266885534 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24040b8180 con 0x55ded1036be0 2023-07-18T03:09:01.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.002+0000 7f2429beab80 10 client.25852 fetch_fsmap learned FSMap version 25 2023-07-18T03:09:01.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.002+0000 7f2429beab80 1 -- 172.21.15.101:0/266885534 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ded0d9fc60 con 0x55ded1036be0 2023-07-18T03:09:01.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.002+0000 7f23eeffd700 1 -- 172.21.15.101:0/266885534 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f240407c020 con 0x55ded1036be0 2023-07-18T03:09:01.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.002+0000 7f23eeffd700 1 -- 172.21.15.101:0/266885534 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f24040a2080 con 0x55ded1036be0 2023-07-18T03:09:01.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.002+0000 7f2429beab80 10 client.25852 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:09:01.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.002+0000 7f2429beab80 10 client.25852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:09:01.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.002+0000 7f2429beab80 20 client.25852 populate_metadata read hostname 'smithi101' 2023-07-18T03:09:01.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.002+0000 7f2429beab80 1 --2- 172.21.15.101:0/266885534 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ded1051200 0x55ded1053610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:01.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.002+0000 7f2429beab80 4 client.25852 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:09:01.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.002+0000 7f2429beab80 1 -- 172.21.15.101:0/266885534 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ded0f81eb0 con 0x55ded1051200 2023-07-18T03:09:01.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.003+0000 7f240f7fe700 1 --2- 172.21.15.101:0/266885534 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ded1051200 0x55ded1053610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:02.000 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.003+0000 7f23eeffd700 10 client.25852 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:09:02.000 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.003+0000 7f240f7fe700 1 --2- 172.21.15.101:0/266885534 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ded1051200 0x55ded1053610 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ab365d824f9a789 server_cookie=61304766cc7fc310 in_seq=0 out_seq=0 2023-07-18T03:09:02.000 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.004+0000 7f23eeffd700 1 -- 172.21.15.101:0/266885534 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ded0f81eb0 con 0x55ded1051200 2023-07-18T03:09:02.000 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.004+0000 7f23eeffd700 10 client.25852 handle_command_reply: tid=0 2023-07-18T03:09:02.001 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.004+0000 7f2429beab80 1 client.25852 shutdown 2023-07-18T03:09:02.001 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.004+0000 7f2429beab80 1 -- 172.21.15.101:0/266885534 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ded1036be0 msgr2=0x55ded103e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:02.001 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.004+0000 7f2429beab80 1 --2- 172.21.15.101:0/266885534 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ded1036be0 0x55ded103e7b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f24040a1ee0 tx=0x7f2404094e80 comp rx=0 tx=0).stop 2023-07-18T03:09:02.001 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.004+0000 7f2429beab80 1 -- 172.21.15.101:0/266885534 shutdown_connections 2023-07-18T03:09:02.001 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.004+0000 7f2429beab80 1 --2- 172.21.15.101:0/266885534 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ded1051200 0x55ded1053610 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.001 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.004+0000 7f2429beab80 1 --2- 172.21.15.101:0/266885534 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ded10362d0 0x55ded1041530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.001 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.004+0000 7f2429beab80 1 --2- 172.21.15.101:0/266885534 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ded1041ad0 0x55ded103eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.001 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.004+0000 7f2429beab80 1 --2- 172.21.15.101:0/266885534 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ded1036be0 0x55ded103e7b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.004+0000 7f2429beab80 1 -- 172.21.15.101:0/266885534 >> 172.21.15.101:0/266885534 conn(0x55ded102cda0 msgr2=0x55ded1032230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:02.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.004+0000 7f23eeffd700 0 client.25852 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:09:02.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.004+0000 7f2429beab80 1 -- 172.21.15.101:0/266885534 shutdown_connections 2023-07-18T03:09:02.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.004+0000 7f2429beab80 1 -- 172.21.15.101:0/266885534 wait complete. 2023-07-18T03:09:02.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.004+0000 7f2429beab80 20 client.25852 trim_cache size 0 max 16384 2023-07-18T03:09:02.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.021+0000 7f2429beab80 1 Processor -- start 2023-07-18T03:09:02.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.021+0000 7f2429beab80 1 -- start start 2023-07-18T03:09:02.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.022+0000 7f2429beab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ded10362d0 0x55ded10366a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:02.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.022+0000 7f2429beab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ded1036be0 0x55ded1030e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:02.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.022+0000 7f2429beab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ded1041ad0 0x55ded1039ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:02.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.022+0000 7f2429beab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ded1037e70 con 0x55ded1036be0 2023-07-18T03:09:02.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.022+0000 7f2429beab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ded102eec0 con 0x55ded1041ad0 2023-07-18T03:09:02.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.022+0000 7f2429beab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ded103f940 con 0x55ded10362d0 2023-07-18T03:09:02.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.022+0000 7f240effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ded1036be0 0x55ded1030e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:02.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.022+0000 7f240f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ded10362d0 0x55ded10366a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:02.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.022+0000 7f240f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ded10362d0 0x55ded10366a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:36784/0 (socket says 172.21.15.101:36784) 2023-07-18T03:09:02.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.022+0000 7f240f7fe700 1 -- 172.21.15.101:0/2901701968 learned_addr learned my addr 172.21.15.101:0/2901701968 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:02.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.022+0000 7f240ffff700 1 --2- 172.21.15.101:0/2901701968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ded1041ad0 0x55ded1039ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:02.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.022+0000 7f240f7fe700 1 -- 172.21.15.101:0/2901701968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ded1041ad0 msgr2=0x55ded1039ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:02.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.022+0000 7f240f7fe700 1 --2- 172.21.15.101:0/2901701968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ded1041ad0 0x55ded1039ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.022+0000 7f240f7fe700 1 -- 172.21.15.101:0/2901701968 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ded1036be0 msgr2=0x55ded1030e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:02.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.022+0000 7f240f7fe700 1 --2- 172.21.15.101:0/2901701968 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ded1036be0 0x55ded1030e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.022+0000 7f240f7fe700 1 -- 172.21.15.101:0/2901701968 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ded0ddd250 con 0x55ded10362d0 2023-07-18T03:09:02.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.022+0000 7f240effd700 1 --2- 172.21.15.101:0/2901701968 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ded1036be0 0x55ded1030e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:09:02.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.022+0000 7f240f7fe700 1 --2- 172.21.15.101:0/2901701968 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ded10362d0 0x55ded10366a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f240000b9e0 tx=0x7f2400005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8951e33018ef9446 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:02.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.023+0000 7f23f67fc700 1 -- 172.21.15.101:0/2901701968 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240001d040 con 0x55ded10362d0 2023-07-18T03:09:02.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.023+0000 7f23f67fc700 1 -- 172.21.15.101:0/2901701968 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f240001e040 con 0x55ded10362d0 2023-07-18T03:09:02.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.023+0000 7f23f67fc700 1 -- 172.21.15.101:0/2901701968 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24000052a0 con 0x55ded10362d0 2023-07-18T03:09:02.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.023+0000 7f2429beab80 1 -- 172.21.15.101:0/2901701968 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ded10362d0 msgr2=0x55ded10366a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:02.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.023+0000 7f2429beab80 1 --2- 172.21.15.101:0/2901701968 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ded10362d0 0x55ded10366a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f240000b9e0 tx=0x7f2400005fd0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.023+0000 7f2429beab80 1 -- 172.21.15.101:0/2901701968 shutdown_connections 2023-07-18T03:09:02.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.023+0000 7f2429beab80 1 --2- 172.21.15.101:0/2901701968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ded1041ad0 0x55ded1039ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.023+0000 7f2429beab80 1 --2- 172.21.15.101:0/2901701968 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ded10362d0 0x55ded10366a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.023+0000 7f2429beab80 1 --2- 172.21.15.101:0/2901701968 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ded1036be0 0x55ded1030e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.023+0000 7f2429beab80 1 -- 172.21.15.101:0/2901701968 >> 172.21.15.101:0/2901701968 conn(0x55ded102cda0 msgr2=0x55ded104d700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:02.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.023+0000 7f2429beab80 1 -- 172.21.15.101:0/2901701968 shutdown_connections 2023-07-18T03:09:02.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.024+0000 7f2429beab80 1 -- 172.21.15.101:0/2901701968 wait complete. 2023-07-18T03:09:02.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.024+0000 7f2429beab80 1 -- start start 2023-07-18T03:09:02.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.024+0000 7f2429beab80 1 Processor -- start 2023-07-18T03:09:02.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.025+0000 7f2429beab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ded10362d0 0x55ded109e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:02.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.025+0000 7f2429beab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ded1036be0 0x55ded109e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:02.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.025+0000 7f2429beab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ded1041ad0 0x55ded1046490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:02.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.025+0000 7f2429beab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ded1046ae0 con 0x55ded10362d0 2023-07-18T03:09:02.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.025+0000 7f2429beab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ded1046c50 con 0x55ded1036be0 2023-07-18T03:09:02.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.025+0000 7f2429beab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ded1046e10 con 0x55ded1041ad0 2023-07-18T03:09:02.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.025+0000 7f2429beab80 10 client.0 _ll_register_callbacks cb 0x7f2404005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:09:02.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.025+0000 7f240f7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ded10362d0 0x55ded109e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:02.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.025+0000 7f240f7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ded10362d0 0x55ded109e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:35160/0 (socket says 172.21.15.101:35160) 2023-07-18T03:09:02.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.025+0000 7f240f7fe700 1 -- 172.21.15.101:0/2529276239 learned_addr learned my addr 172.21.15.101:0/2529276239 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:02.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.025+0000 7f240ffff700 1 --2- 172.21.15.101:0/2529276239 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ded1041ad0 0x55ded1046490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:02.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.025+0000 7f240effd700 1 --2- 172.21.15.101:0/2529276239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ded1036be0 0x55ded109e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:02.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.025+0000 7f240f7fe700 1 -- 172.21.15.101:0/2529276239 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ded1041ad0 msgr2=0x55ded1046490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:02.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.025+0000 7f240f7fe700 1 --2- 172.21.15.101:0/2529276239 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ded1041ad0 0x55ded1046490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.025+0000 7f240f7fe700 1 -- 172.21.15.101:0/2529276239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ded1036be0 msgr2=0x55ded109e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:02.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.025+0000 7f240f7fe700 1 --2- 172.21.15.101:0/2529276239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ded1036be0 0x55ded109e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.025+0000 7f240f7fe700 1 -- 172.21.15.101:0/2529276239 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ded0d9fc60 con 0x55ded10362d0 2023-07-18T03:09:02.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.025+0000 7f240f7fe700 1 --2- 172.21.15.101:0/2529276239 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ded10362d0 0x55ded109e380 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2400009350 tx=0x7f24000295d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53063b3e301f35ae server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:02.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.025+0000 7f23eeffd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:09:02.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.026+0000 7f23eeffd700 1 -- 172.21.15.101:0/2529276239 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240001d040 con 0x55ded10362d0 2023-07-18T03:09:02.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.026+0000 7f2429beab80 1 -- 172.21.15.101:0/2529276239 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2404010880 con 0x55ded10362d0 2023-07-18T03:09:02.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.026+0000 7f23eeffd700 1 -- 172.21.15.101:0/2529276239 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f240000bb30 con 0x55ded10362d0 2023-07-18T03:09:02.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.026+0000 7f23eeffd700 1 -- 172.21.15.101:0/2529276239 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24000292d0 con 0x55ded10362d0 2023-07-18T03:09:02.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.026+0000 7f2429beab80 10 client.15804 fetch_fsmap learned FSMap version 25 2023-07-18T03:09:02.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.026+0000 7f2429beab80 1 -- 172.21.15.101:0/2529276239 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f241010d220 con 0x55ded10362d0 2023-07-18T03:09:02.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.026+0000 7f23eeffd700 1 -- 172.21.15.101:0/2529276239 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2404010880 con 0x55ded10362d0 2023-07-18T03:09:02.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.026+0000 7f23eeffd700 1 -- 172.21.15.101:0/2529276239 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f2400004760 con 0x55ded10362d0 2023-07-18T03:09:02.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.026+0000 7f2429beab80 10 client.15804 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:09:02.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.026+0000 7f2429beab80 10 client.15804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:09:02.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.026+0000 7f2429beab80 20 client.15804 populate_metadata read hostname 'smithi101' 2023-07-18T03:09:02.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.026+0000 7f2429beab80 1 --2- 172.21.15.101:0/2529276239 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ded10a5da0 0x55ded10a81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:02.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.026+0000 7f2429beab80 4 client.15804 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:09:02.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.026+0000 7f2429beab80 1 -- 172.21.15.101:0/2529276239 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ded0f44050 con 0x55ded10a5da0 2023-07-18T03:09:02.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.027+0000 7f240effd700 1 --2- 172.21.15.101:0/2529276239 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ded10a5da0 0x55ded10a81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:02.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.027+0000 7f240effd700 1 --2- 172.21.15.101:0/2529276239 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ded10a5da0 0x55ded10a81b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d1c66dc12da07c2 server_cookie=e070b80c0ec70fd3 in_seq=0 out_seq=0 2023-07-18T03:09:02.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.027+0000 7f23eeffd700 10 client.15804 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:09:02.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.027+0000 7f23eeffd700 1 -- 172.21.15.101:0/2529276239 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ded0f44050 con 0x55ded10a5da0 2023-07-18T03:09:02.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.027+0000 7f23eeffd700 10 client.15804 handle_command_reply: tid=0 2023-07-18T03:09:02.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.027+0000 7f2429beab80 1 client.15804 shutdown 2023-07-18T03:09:02.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.027+0000 7f2429beab80 1 -- 172.21.15.101:0/2529276239 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ded10362d0 msgr2=0x55ded109e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:02.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.027+0000 7f2429beab80 1 --2- 172.21.15.101:0/2529276239 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ded10362d0 0x55ded109e380 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2400009350 tx=0x7f24000295d0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.027+0000 7f2429beab80 1 -- 172.21.15.101:0/2529276239 shutdown_connections 2023-07-18T03:09:02.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.027+0000 7f2429beab80 1 --2- 172.21.15.101:0/2529276239 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ded10a5da0 0x55ded10a81b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.027+0000 7f2429beab80 1 --2- 172.21.15.101:0/2529276239 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ded1041ad0 0x55ded1046490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.027+0000 7f2429beab80 1 --2- 172.21.15.101:0/2529276239 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ded10362d0 0x55ded109e380 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.027+0000 7f23eeffd700 0 client.15804 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:09:02.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.027+0000 7f2429beab80 1 --2- 172.21.15.101:0/2529276239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ded1036be0 0x55ded109e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.027+0000 7f2429beab80 1 -- 172.21.15.101:0/2529276239 >> 172.21.15.101:0/2529276239 conn(0x55ded102cda0 msgr2=0x55ded1039ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:02.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.028+0000 7f2429beab80 1 -- 172.21.15.101:0/2529276239 shutdown_connections 2023-07-18T03:09:02.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.028+0000 7f2429beab80 1 -- 172.21.15.101:0/2529276239 wait complete. 2023-07-18T03:09:02.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.028+0000 7f2429beab80 20 client.15804 trim_cache size 0 max 16384 2023-07-18T03:09:02.043 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:09:02.043 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "no active scrubs running", 2023-07-18T03:09:02.044 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": {} 2023-07-18T03:09:02.044 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:09:02.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.028+0000 7f24161ff700 1 -- 172.21.15.101:0/1485616014 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f23f80a1460 msgr2=0x7f23f80a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:02.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.028+0000 7f24161ff700 1 --2- 172.21.15.101:0/1485616014 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f23f80a1460 0x7f23f80a3910 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2408009550 tx=0x7f2408009130 comp rx=0 tx=0).stop 2023-07-18T03:09:02.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.028+0000 7f24161ff700 1 -- 172.21.15.101:0/1485616014 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2410100490 msgr2=0x7f2410072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:02.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.028+0000 7f24161ff700 1 --2- 172.21.15.101:0/1485616014 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2410100490 0x7f2410072d50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f240400e9a0 tx=0x7f240400ecb0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.028+0000 7f24161ff700 1 -- 172.21.15.101:0/1485616014 shutdown_connections 2023-07-18T03:09:02.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.028+0000 7f24161ff700 1 --2- 172.21.15.101:0/1485616014 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f23f80a1460 0x7f23f80a3910 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.028+0000 7f24161ff700 1 --2- 172.21.15.101:0/1485616014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2410105af0 0x7f24101a3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.028+0000 7f24161ff700 1 --2- 172.21.15.101:0/1485616014 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24100ffae0 0x7f2410072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.028+0000 7f24161ff700 1 --2- 172.21.15.101:0/1485616014 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2410100490 0x7f2410072d50 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.028+0000 7f24161ff700 1 -- 172.21.15.101:0/1485616014 >> 172.21.15.101:0/1485616014 conn(0x7f24100fb5e0 msgr2=0x7f24101035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:02.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.029+0000 7f24161ff700 1 -- 172.21.15.101:0/1485616014 shutdown_connections 2023-07-18T03:09:02.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.029+0000 7f24161ff700 1 -- 172.21.15.101:0/1485616014 wait complete. 2023-07-18T03:09:02.047 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-18T03:09:02.047 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:09:02.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.258+0000 7f88a85ff700 1 -- 172.21.15.101:0/1057676663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88a0105b00 msgr2=0x7f88a0105ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:02.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.258+0000 7f88a85ff700 1 --2- 172.21.15.101:0/1057676663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88a0105b00 0x7f88a0105ed0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f889c009990 tx=0x7f889c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.258+0000 7f88a85ff700 1 -- 172.21.15.101:0/1057676663 shutdown_connections 2023-07-18T03:09:02.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.258+0000 7f88a85ff700 1 --2- 172.21.15.101:0/1057676663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88a0105b00 0x7f88a0105ed0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.258+0000 7f88a85ff700 1 --2- 172.21.15.101:0/1057676663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88a01004a0 0x7f88a010c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.258+0000 7f88a85ff700 1 --2- 172.21.15.101:0/1057676663 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f88a00ffaf0 0x7f88a00fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.258+0000 7f88a85ff700 1 -- 172.21.15.101:0/1057676663 >> 172.21.15.101:0/1057676663 conn(0x7f88a00fb630 msgr2=0x7f88a00fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:02.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.258+0000 7f88a85ff700 1 -- 172.21.15.101:0/1057676663 shutdown_connections 2023-07-18T03:09:02.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.258+0000 7f88a85ff700 1 -- 172.21.15.101:0/1057676663 wait complete. 2023-07-18T03:09:02.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.259+0000 7f88a85ff700 1 Processor -- start 2023-07-18T03:09:02.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.259+0000 7f88a85ff700 1 -- start start 2023-07-18T03:09:02.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.259+0000 7f88a85ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88a00ffaf0 0x7f88a019ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:02.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.259+0000 7f88a85ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f88a01004a0 0x7f88a019bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:02.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.259+0000 7f88a85ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88a0105b00 0x7f88a0195ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:02.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.259+0000 7f88a85ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a0196120 con 0x7f88a01004a0 2023-07-18T03:09:02.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.259+0000 7f88a85ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a0196290 con 0x7f88a00ffaf0 2023-07-18T03:09:02.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.259+0000 7f88a85ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a0196450 con 0x7f88a0105b00 2023-07-18T03:09:02.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.260+0000 7f88a5b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f88a01004a0 0x7f88a019bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:02.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.260+0000 7f88a639b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88a00ffaf0 0x7f88a019ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:02.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.260+0000 7f88a5b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f88a01004a0 0x7f88a019bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:35188/0 (socket says 172.21.15.101:35188) 2023-07-18T03:09:02.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.260+0000 7f88a5b9a700 1 -- 172.21.15.101:0/313120455 learned_addr learned my addr 172.21.15.101:0/313120455 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:02.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.260+0000 7f88a6b9c700 1 --2- 172.21.15.101:0/313120455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88a0105b00 0x7f88a0195ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:02.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.260+0000 7f88a5b9a700 1 -- 172.21.15.101:0/313120455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88a0105b00 msgr2=0x7f88a0195ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:02.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.260+0000 7f88a5b9a700 1 --2- 172.21.15.101:0/313120455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88a0105b00 0x7f88a0195ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.260+0000 7f88a5b9a700 1 -- 172.21.15.101:0/313120455 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88a00ffaf0 msgr2=0x7f88a019ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:02.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.260+0000 7f88a5b9a700 1 --2- 172.21.15.101:0/313120455 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88a00ffaf0 0x7f88a019ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.260+0000 7f88a5b9a700 1 -- 172.21.15.101:0/313120455 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f889c0095f0 con 0x7f88a01004a0 2023-07-18T03:09:02.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.260+0000 7f88a6b9c700 1 --2- 172.21.15.101:0/313120455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88a0105b00 0x7f88a0195ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:09:02.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.260+0000 7f88a639b700 1 --2- 172.21.15.101:0/313120455 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88a00ffaf0 0x7f88a019ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:09:02.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.260+0000 7f88a5b9a700 1 --2- 172.21.15.101:0/313120455 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f88a01004a0 0x7f88a019bf80 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f889000e9a0 tx=0x7f889000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:02.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.261+0000 7f88977fe700 1 -- 172.21.15.101:0/313120455 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8890009790 con 0x7f88a01004a0 2023-07-18T03:09:02.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.261+0000 7f88a85ff700 1 -- 172.21.15.101:0/313120455 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88a0196730 con 0x7f88a01004a0 2023-07-18T03:09:02.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.261+0000 7f88977fe700 1 -- 172.21.15.101:0/313120455 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f889000cbd0 con 0x7f88a01004a0 2023-07-18T03:09:02.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.261+0000 7f88977fe700 1 -- 172.21.15.101:0/313120455 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8890018680 con 0x7f88a01004a0 2023-07-18T03:09:02.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.261+0000 7f88a85ff700 1 -- 172.21.15.101:0/313120455 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88a01a2a30 con 0x7f88a01004a0 2023-07-18T03:09:02.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.262+0000 7f88a85ff700 1 -- 172.21.15.101:0/313120455 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88a0067300 con 0x7f88a01004a0 2023-07-18T03:09:02.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.264+0000 7f88977fe700 1 -- 172.21.15.101:0/313120455 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 24) v1 ==== 135949+0+0 (secure 0 0 0) 0x7f889000cd40 con 0x7f88a01004a0 2023-07-18T03:09:02.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.265+0000 7f88977fe700 1 --2- 172.21.15.101:0/313120455 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f888c0a1430 0x7f888c0a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:02.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.265+0000 7f88977fe700 1 -- 172.21.15.101:0/313120455 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f8890014070 con 0x7f88a01004a0 2023-07-18T03:09:02.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.265+0000 7f88a639b700 1 --2- 172.21.15.101:0/313120455 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f888c0a1430 0x7f888c0a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:02.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.266+0000 7f88a639b700 1 --2- 172.21.15.101:0/313120455 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f888c0a1430 0x7f888c0a38e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f889c00b2e0 tx=0x7f889c005fd0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:02.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.269+0000 7f88977fe700 1 -- 172.21.15.101:0/313120455 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8890051790 con 0x7f88a01004a0 2023-07-18T03:09:02.446 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:02 smithi179 ceph-mon[129229]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:09:02.483 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:02 smithi101 ceph-mon[126379]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:09:02.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.509+0000 7f88a85ff700 1 -- 172.21.15.101:0/313120455 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f88a01a2cf0 con 0x7f88a01004a0 2023-07-18T03:09:02.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.510+0000 7f88977fe700 1 -- 172.21.15.101:0/313120455 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10861 (secure 0 0 0) 0x7f88900963e0 con 0x7f88a01004a0 2023-07-18T03:09:02.500 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:09:02.500 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/4192846769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":4192846769},{"type":"v1","addr":"172.21.15.101:6839","nonce":4192846769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6841/3824340175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6840","nonce":3824340175},{"type":"v1","addr":"172.21.15.101:6841","nonce":3824340175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:04:59.361185+0000","modified":"2023-07-18T03:09:01.105419+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.155:6839/2055462483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2055462483},{"type":"v1","addr":"172.21.15.155:6839","nonce":2055462483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:09:02.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.512+0000 7f88a85ff700 1 -- 172.21.15.101:0/313120455 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f888c0a1430 msgr2=0x7f888c0a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:02.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.512+0000 7f88a85ff700 1 --2- 172.21.15.101:0/313120455 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f888c0a1430 0x7f888c0a38e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f889c00b2e0 tx=0x7f889c005fd0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.512+0000 7f88a85ff700 1 -- 172.21.15.101:0/313120455 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f88a01004a0 msgr2=0x7f88a019bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:02.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.512+0000 7f88a85ff700 1 --2- 172.21.15.101:0/313120455 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f88a01004a0 0x7f88a019bf80 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f889000e9a0 tx=0x7f889000ecb0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.512+0000 7f88a85ff700 1 -- 172.21.15.101:0/313120455 shutdown_connections 2023-07-18T03:09:02.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.512+0000 7f88a85ff700 1 --2- 172.21.15.101:0/313120455 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f888c0a1430 0x7f888c0a38e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.512+0000 7f88a85ff700 1 --2- 172.21.15.101:0/313120455 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88a00ffaf0 0x7f88a019ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.512+0000 7f88a85ff700 1 --2- 172.21.15.101:0/313120455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88a0105b00 0x7f88a0195ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.512+0000 7f88a85ff700 1 --2- 172.21.15.101:0/313120455 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f88a01004a0 0x7f88a019bf80 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.512+0000 7f88a85ff700 1 -- 172.21.15.101:0/313120455 >> 172.21.15.101:0/313120455 conn(0x7f88a00fb630 msgr2=0x7f88a00fbcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:02.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.513+0000 7f88a85ff700 1 -- 172.21.15.101:0/313120455 shutdown_connections 2023-07-18T03:09:02.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.513+0000 7f88a85ff700 1 -- 172.21.15.101:0/313120455 wait complete. 2023-07-18T03:09:02.504 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 25 2023-07-18T03:09:02.521 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-18T03:09:02.541 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:02 smithi155 ceph-mon[128885]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:09:02.561 INFO:tasks.workunit.client.0.smithi101.stdout: 1 65270 13.71 MB/sec warmup 27 sec latency 1371.329 ms 2023-07-18T03:09:02.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.730+0000 7f86473ff700 1 -- 172.21.15.101:0/2416650096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86400728b0 msgr2=0x7f8640072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:02.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.730+0000 7f86473ff700 1 --2- 172.21.15.101:0/2416650096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86400728b0 0x7f8640072c80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f863c009990 tx=0x7f863c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.730+0000 7f86473ff700 1 -- 172.21.15.101:0/2416650096 shutdown_connections 2023-07-18T03:09:02.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.730+0000 7f86473ff700 1 --2- 172.21.15.101:0/2416650096 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8640110af0 0x7f8640112ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.730+0000 7f86473ff700 1 --2- 172.21.15.101:0/2416650096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86400728b0 0x7f8640072c80 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.730+0000 7f86473ff700 1 --2- 172.21.15.101:0/2416650096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86400731c0 0x7f86401105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.730+0000 7f86473ff700 1 -- 172.21.15.101:0/2416650096 >> 172.21.15.101:0/2416650096 conn(0x7f86400fb5e0 msgr2=0x7f86400fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:02.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.731+0000 7f86473ff700 1 -- 172.21.15.101:0/2416650096 shutdown_connections 2023-07-18T03:09:02.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.731+0000 7f86473ff700 1 -- 172.21.15.101:0/2416650096 wait complete. 2023-07-18T03:09:02.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.732+0000 7f86473ff700 1 Processor -- start 2023-07-18T03:09:02.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.732+0000 7f86473ff700 1 -- start start 2023-07-18T03:09:02.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.732+0000 7f86473ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86400728b0 0x7f864019c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:02.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.732+0000 7f86473ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f86400731c0 0x7f864019c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:02.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.732+0000 7f86473ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8640110af0 0x7f86401a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:02.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.732+0000 7f86473ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86401a1210 con 0x7f86400731c0 2023-07-18T03:09:02.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.732+0000 7f86473ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86401a1380 con 0x7f86400728b0 2023-07-18T03:09:02.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.732+0000 7f86473ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86401a1540 con 0x7f8640110af0 2023-07-18T03:09:02.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.732+0000 7f864499a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f86400731c0 0x7f864019c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:02.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.732+0000 7f864499a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f86400731c0 0x7f864019c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:35212/0 (socket says 172.21.15.101:35212) 2023-07-18T03:09:02.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.732+0000 7f864499a700 1 -- 172.21.15.101:0/2478028663 learned_addr learned my addr 172.21.15.101:0/2478028663 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:02.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.732+0000 7f864599c700 1 --2- 172.21.15.101:0/2478028663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8640110af0 0x7f86401a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:02.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.732+0000 7f864519b700 1 --2- 172.21.15.101:0/2478028663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86400728b0 0x7f864019c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:02.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.733+0000 7f864499a700 1 -- 172.21.15.101:0/2478028663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8640110af0 msgr2=0x7f86401a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:02.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.733+0000 7f864499a700 1 --2- 172.21.15.101:0/2478028663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8640110af0 0x7f86401a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.733+0000 7f864499a700 1 -- 172.21.15.101:0/2478028663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86400728b0 msgr2=0x7f864019c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:02.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.733+0000 7f864499a700 1 --2- 172.21.15.101:0/2478028663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86400728b0 0x7f864019c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.733+0000 7f864499a700 1 -- 172.21.15.101:0/2478028663 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f863c0095f0 con 0x7f86400731c0 2023-07-18T03:09:02.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.733+0000 7f864599c700 1 --2- 172.21.15.101:0/2478028663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8640110af0 0x7f86401a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:09:02.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.733+0000 7f864499a700 1 --2- 172.21.15.101:0/2478028663 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f86400731c0 0x7f864019c770 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f863000e9a0 tx=0x7f863000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:02.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.733+0000 7f86367fc700 1 -- 172.21.15.101:0/2478028663 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8630009790 con 0x7f86400731c0 2023-07-18T03:09:02.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.733+0000 7f86367fc700 1 -- 172.21.15.101:0/2478028663 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8630010430 con 0x7f86400731c0 2023-07-18T03:09:02.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.733+0000 7f86473ff700 1 -- 172.21.15.101:0/2478028663 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86401a1820 con 0x7f86400731c0 2023-07-18T03:09:02.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.733+0000 7f86367fc700 1 -- 172.21.15.101:0/2478028663 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8630018680 con 0x7f86400731c0 2023-07-18T03:09:02.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.733+0000 7f86473ff700 1 -- 172.21.15.101:0/2478028663 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86401a9060 con 0x7f86400731c0 2023-07-18T03:09:02.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.737+0000 7f86367fc700 1 -- 172.21.15.101:0/2478028663 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 24) v1 ==== 135949+0+0 (secure 0 0 0) 0x7f86300105a0 con 0x7f86400731c0 2023-07-18T03:09:02.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.738+0000 7f865ac9db80 1 Processor -- start 2023-07-18T03:09:02.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.738+0000 7f865ac9db80 1 -- start start 2023-07-18T03:09:02.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.739+0000 7f86367fc700 1 --2- 172.21.15.101:0/2478028663 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f862c0a13e0 0x7f862c0a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:02.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.739+0000 7f865ac9db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c38b9742d0 0x55c38b9746a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:02.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.739+0000 7f865ac9db80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c38b974be0 0x55c38b97f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:02.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.739+0000 7f865ac9db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38b97fad0 0x55c38b981eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:02.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.739+0000 7f86367fc700 1 -- 172.21.15.101:0/2478028663 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f8630014070 con 0x7f86400731c0 2023-07-18T03:09:02.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.739+0000 7f865ac9db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c38b87acb0 con 0x55c38b9742d0 2023-07-18T03:09:02.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.739+0000 7f865ac9db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c38b8cda40 con 0x55c38b97fad0 2023-07-18T03:09:02.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.739+0000 7f865ac9db80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c38b8c6ed0 con 0x55c38b974be0 2023-07-18T03:09:02.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.739+0000 7f864599c700 1 --2- 172.21.15.101:0/2478028663 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f862c0a13e0 0x7f862c0a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:02.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.739+0000 7f864599c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38b97fad0 0x55c38b981eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:02.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.739+0000 7f864519b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c38b9742d0 0x55c38b9746a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:02.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.739+0000 7f864599c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38b97fad0 0x55c38b981eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:51186/0 (socket says 172.21.15.101:51186) 2023-07-18T03:09:02.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.739+0000 7f864599c700 1 -- 172.21.15.101:0/605397364 learned_addr learned my addr 172.21.15.101:0/605397364 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:02.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.739+0000 7f864499a700 1 --2- 172.21.15.101:0/605397364 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c38b974be0 0x55c38b97f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:02.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.740+0000 7f864519b700 1 -- 172.21.15.101:0/605397364 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c38b974be0 msgr2=0x55c38b97f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:02.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.740+0000 7f864519b700 1 --2- 172.21.15.101:0/605397364 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c38b974be0 0x55c38b97f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.740+0000 7f864519b700 1 -- 172.21.15.101:0/605397364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38b97fad0 msgr2=0x55c38b981eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:02.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.740+0000 7f864519b700 1 --2- 172.21.15.101:0/605397364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38b97fad0 0x55c38b981eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.740+0000 7f864519b700 1 -- 172.21.15.101:0/605397364 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c38b6f4de0 con 0x55c38b9742d0 2023-07-18T03:09:02.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.740+0000 7f864599c700 1 --2- 172.21.15.101:0/2478028663 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f862c0a13e0 0x7f862c0a3890 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f86380042f0 tx=0x7f8638003ac0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:02.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.740+0000 7f864519b700 1 --2- 172.21.15.101:0/605397364 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c38b9742d0 0x55c38b9746a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f863c000c00 tx=0x7f863c005ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63a28871f8cf7566 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:02.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.740+0000 7f862b7fe700 1 -- 172.21.15.101:0/605397364 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863c01d030 con 0x55c38b9742d0 2023-07-18T03:09:02.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.740+0000 7f862b7fe700 1 -- 172.21.15.101:0/605397364 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f863c01e040 con 0x55c38b9742d0 2023-07-18T03:09:02.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.740+0000 7f862b7fe700 1 -- 172.21.15.101:0/605397364 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863c004760 con 0x55c38b9742d0 2023-07-18T03:09:02.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.741+0000 7f865ac9db80 1 -- 172.21.15.101:0/605397364 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c38b9742d0 msgr2=0x55c38b9746a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:02.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.741+0000 7f865ac9db80 1 --2- 172.21.15.101:0/605397364 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c38b9742d0 0x55c38b9746a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f863c000c00 tx=0x7f863c005ea0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.741+0000 7f865ac9db80 1 -- 172.21.15.101:0/605397364 shutdown_connections 2023-07-18T03:09:02.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.741+0000 7f865ac9db80 1 --2- 172.21.15.101:0/605397364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38b97fad0 0x55c38b981eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.741+0000 7f865ac9db80 1 --2- 172.21.15.101:0/605397364 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c38b9742d0 0x55c38b9746a0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.741+0000 7f865ac9db80 1 --2- 172.21.15.101:0/605397364 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c38b974be0 0x55c38b97f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.741+0000 7f865ac9db80 1 -- 172.21.15.101:0/605397364 >> 172.21.15.101:0/605397364 conn(0x55c38b96ada0 msgr2=0x55c38b9706a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:02.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.741+0000 7f865ac9db80 1 -- 172.21.15.101:0/605397364 shutdown_connections 2023-07-18T03:09:02.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.741+0000 7f865ac9db80 1 -- 172.21.15.101:0/605397364 wait complete. 2023-07-18T03:09:02.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.742+0000 7f865ac9db80 1 -- start start 2023-07-18T03:09:02.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.742+0000 7f865ac9db80 1 Processor -- start 2023-07-18T03:09:02.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.742+0000 7f865ac9db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c38b9742d0 0x55c38b97f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:02.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.742+0000 7f865ac9db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38b974be0 0x55c38b97c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:02.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.742+0000 7f865ac9db80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c38b97fad0 0x55c38b97cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:02.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.742+0000 7f865ac9db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c38b866ac0 con 0x55c38b9742d0 2023-07-18T03:09:02.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.742+0000 7f865ac9db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c38b8667c0 con 0x55c38b974be0 2023-07-18T03:09:02.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.742+0000 7f865ac9db80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c38b74a920 con 0x55c38b97fad0 2023-07-18T03:09:02.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.742+0000 7f865ac9db80 10 client.0 _ll_register_callbacks cb 0x55c38b864760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:09:02.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.743+0000 7f864519b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c38b9742d0 0x55c38b97f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:02.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.743+0000 7f864519b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c38b9742d0 0x55c38b97f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:35226/0 (socket says 172.21.15.101:35226) 2023-07-18T03:09:02.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.743+0000 7f864519b700 1 -- 172.21.15.101:0/2057903523 learned_addr learned my addr 172.21.15.101:0/2057903523 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:02.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.743+0000 7f864499a700 1 --2- 172.21.15.101:0/2057903523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38b974be0 0x55c38b97c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:02.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.743+0000 7f864599c700 1 --2- 172.21.15.101:0/2057903523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c38b97fad0 0x55c38b97cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:02.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.743+0000 7f864519b700 1 -- 172.21.15.101:0/2057903523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c38b97fad0 msgr2=0x55c38b97cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:02.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.743+0000 7f864519b700 1 --2- 172.21.15.101:0/2057903523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c38b97fad0 0x55c38b97cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.743+0000 7f864519b700 1 -- 172.21.15.101:0/2057903523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38b974be0 msgr2=0x55c38b97c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:02.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.743+0000 7f864519b700 1 --2- 172.21.15.101:0/2057903523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38b974be0 0x55c38b97c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.743+0000 7f864519b700 1 -- 172.21.15.101:0/2057903523 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8640113790 con 0x55c38b9742d0 2023-07-18T03:09:02.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.743+0000 7f864499a700 1 --2- 172.21.15.101:0/2057903523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38b974be0 0x55c38b97c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:09:02.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.743+0000 7f864519b700 1 --2- 172.21.15.101:0/2057903523 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c38b9742d0 0x55c38b97f500 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f863c005990 tx=0x7f863c00bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b9b3c62ead975a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:02.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.743+0000 7f8628ff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:09:02.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.744+0000 7f8628ff9700 1 -- 172.21.15.101:0/2057903523 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863c01d040 con 0x55c38b9742d0 2023-07-18T03:09:02.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.744+0000 7f865ac9db80 1 -- 172.21.15.101:0/2057903523 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c38b8204f0 con 0x55c38b9742d0 2023-07-18T03:09:02.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.744+0000 7f8628ff9700 1 -- 172.21.15.101:0/2057903523 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f863c00f870 con 0x55c38b9742d0 2023-07-18T03:09:02.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.744+0000 7f8628ff9700 1 -- 172.21.15.101:0/2057903523 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f863c00fa30 con 0x55c38b9742d0 2023-07-18T03:09:02.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.744+0000 7f865ac9db80 10 client.15822 fetch_fsmap learned FSMap version 25 2023-07-18T03:09:02.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.744+0000 7f865ac9db80 1 -- 172.21.15.101:0/2057903523 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c38b6ddc60 con 0x55c38b9742d0 2023-07-18T03:09:02.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.744+0000 7f8628ff9700 1 -- 172.21.15.101:0/2057903523 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c38b8204f0 con 0x55c38b9742d0 2023-07-18T03:09:02.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.744+0000 7f8628ff9700 1 -- 172.21.15.101:0/2057903523 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f863c02c3f0 con 0x55c38b9742d0 2023-07-18T03:09:02.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.744+0000 7f865ac9db80 10 client.15822 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:09:02.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.744+0000 7f865ac9db80 10 client.15822 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:09:02.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.744+0000 7f865ac9db80 20 client.15822 populate_metadata read hostname 'smithi101' 2023-07-18T03:09:02.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.744+0000 7f865ac9db80 1 --2- 172.21.15.101:0/2057903523 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55c38b98eff0 0x55c38b991400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:02.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.744+0000 7f865ac9db80 4 client.15822 mds_command: new command op to 24541 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:09:02.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.744+0000 7f865ac9db80 1 -- 172.21.15.101:0/2057903523 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c38b8bfeb0 con 0x55c38b98eff0 2023-07-18T03:09:02.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.745+0000 7f864499a700 1 --2- 172.21.15.101:0/2057903523 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55c38b98eff0 0x55c38b991400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:02.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.745+0000 7f864499a700 1 --2- 172.21.15.101:0/2057903523 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55c38b98eff0 0x55c38b991400 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=436791e37b98e200 server_cookie=54c8e1b8cd94480b in_seq=0 out_seq=0 2023-07-18T03:09:02.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.745+0000 7f8628ff9700 10 client.15822 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:09:02.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.746+0000 7f8628ff9700 1 -- 172.21.15.101:0/2057903523 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c38b8bfeb0 con 0x55c38b98eff0 2023-07-18T03:09:02.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.746+0000 7f8628ff9700 10 client.15822 handle_command_reply: tid=0 2023-07-18T03:09:02.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.746+0000 7f865ac9db80 1 client.15822 shutdown 2023-07-18T03:09:02.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.746+0000 7f865ac9db80 1 -- 172.21.15.101:0/2057903523 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c38b9742d0 msgr2=0x55c38b97f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:02.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.746+0000 7f865ac9db80 1 --2- 172.21.15.101:0/2057903523 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c38b9742d0 0x55c38b97f500 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f863c005990 tx=0x7f863c00bf60 comp rx=0 tx=0).stop 2023-07-18T03:09:02.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.746+0000 7f865ac9db80 1 -- 172.21.15.101:0/2057903523 shutdown_connections 2023-07-18T03:09:02.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.746+0000 7f865ac9db80 1 --2- 172.21.15.101:0/2057903523 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55c38b98eff0 0x55c38b991400 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.746+0000 7f865ac9db80 1 --2- 172.21.15.101:0/2057903523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c38b97fad0 0x55c38b97cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.746+0000 7f865ac9db80 1 --2- 172.21.15.101:0/2057903523 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c38b9742d0 0x55c38b97f500 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.746+0000 7f8628ff9700 0 client.15822 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:09:02.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.746+0000 7f865ac9db80 1 --2- 172.21.15.101:0/2057903523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38b974be0 0x55c38b97c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.746+0000 7f865ac9db80 1 -- 172.21.15.101:0/2057903523 >> 172.21.15.101:0/2057903523 conn(0x55c38b96ada0 msgr2=0x55c38b970290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:02.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.746+0000 7f865ac9db80 1 -- 172.21.15.101:0/2057903523 shutdown_connections 2023-07-18T03:09:02.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.746+0000 7f865ac9db80 1 -- 172.21.15.101:0/2057903523 wait complete. 2023-07-18T03:09:02.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.746+0000 7f865ac9db80 20 client.15822 trim_cache size 0 max 16384 2023-07-18T03:09:02.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.760+0000 7f865ac9db80 1 Processor -- start 2023-07-18T03:09:02.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.760+0000 7f865ac9db80 1 -- start start 2023-07-18T03:09:02.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.760+0000 7f865ac9db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c38b9742d0 0x55c38b9913d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:02.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.760+0000 7f865ac9db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38b974be0 0x55c38b97f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:02.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.760+0000 7f865ac9db80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c38b97fad0 0x55c38b977ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:02.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.760+0000 7f865ac9db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c38b96a970 con 0x55c38b9742d0 2023-07-18T03:09:02.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.760+0000 7f865ac9db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c38b9761a0 con 0x55c38b974be0 2023-07-18T03:09:02.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.760+0000 7f865ac9db80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c38b96c6d0 con 0x55c38b97fad0 2023-07-18T03:09:02.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.760+0000 7f864519b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c38b9742d0 0x55c38b9913d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:02.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.760+0000 7f864519b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c38b9742d0 0x55c38b9913d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:35240/0 (socket says 172.21.15.101:35240) 2023-07-18T03:09:02.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.760+0000 7f864519b700 1 -- 172.21.15.101:0/2215125791 learned_addr learned my addr 172.21.15.101:0/2215125791 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:02.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.760+0000 7f864599c700 1 --2- 172.21.15.101:0/2215125791 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c38b97fad0 0x55c38b977ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:02.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.761+0000 7f864499a700 1 --2- 172.21.15.101:0/2215125791 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38b974be0 0x55c38b97f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:02.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.761+0000 7f864519b700 1 -- 172.21.15.101:0/2215125791 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c38b97fad0 msgr2=0x55c38b977ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:02.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.761+0000 7f864519b700 1 --2- 172.21.15.101:0/2215125791 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c38b97fad0 0x55c38b977ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.761+0000 7f864519b700 1 -- 172.21.15.101:0/2215125791 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38b974be0 msgr2=0x55c38b97f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:02.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.761+0000 7f864519b700 1 --2- 172.21.15.101:0/2215125791 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38b974be0 0x55c38b97f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.761+0000 7f864519b700 1 -- 172.21.15.101:0/2215125791 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c38b71b250 con 0x55c38b9742d0 2023-07-18T03:09:02.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.761+0000 7f864499a700 1 --2- 172.21.15.101:0/2215125791 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38b974be0 0x55c38b97f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:09:02.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.761+0000 7f864599c700 1 --2- 172.21.15.101:0/2215125791 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c38b97fad0 0x55c38b977ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:09:02.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.761+0000 7f864519b700 1 --2- 172.21.15.101:0/2215125791 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c38b9742d0 0x55c38b9913d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f863c000c00 tx=0x7f863c02c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42211f7310ac3aac server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:02.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.761+0000 7f862bfff700 1 -- 172.21.15.101:0/2215125791 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863c0059c0 con 0x55c38b9742d0 2023-07-18T03:09:02.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.761+0000 7f862bfff700 1 -- 172.21.15.101:0/2215125791 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f863c02c5f0 con 0x55c38b9742d0 2023-07-18T03:09:02.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.761+0000 7f862bfff700 1 -- 172.21.15.101:0/2215125791 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863c00f670 con 0x55c38b9742d0 2023-07-18T03:09:02.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.762+0000 7f865ac9db80 1 -- 172.21.15.101:0/2215125791 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c38b9742d0 msgr2=0x55c38b9913d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:02.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.762+0000 7f865ac9db80 1 --2- 172.21.15.101:0/2215125791 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c38b9742d0 0x55c38b9913d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f863c000c00 tx=0x7f863c02c8d0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.762+0000 7f865ac9db80 1 -- 172.21.15.101:0/2215125791 shutdown_connections 2023-07-18T03:09:02.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.762+0000 7f865ac9db80 1 --2- 172.21.15.101:0/2215125791 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c38b97fad0 0x55c38b977ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.762+0000 7f865ac9db80 1 --2- 172.21.15.101:0/2215125791 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c38b9742d0 0x55c38b9913d0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.762+0000 7f865ac9db80 1 --2- 172.21.15.101:0/2215125791 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38b974be0 0x55c38b97f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.762+0000 7f865ac9db80 1 -- 172.21.15.101:0/2215125791 >> 172.21.15.101:0/2215125791 conn(0x55c38b96ada0 msgr2=0x55c38b995d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:02.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.762+0000 7f865ac9db80 1 -- 172.21.15.101:0/2215125791 shutdown_connections 2023-07-18T03:09:02.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.762+0000 7f865ac9db80 1 -- 172.21.15.101:0/2215125791 wait complete. 2023-07-18T03:09:02.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.762+0000 7f865ac9db80 1 -- start start 2023-07-18T03:09:02.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.762+0000 7f865ac9db80 1 Processor -- start 2023-07-18T03:09:02.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.763+0000 7f865ac9db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38b9742d0 0x55c38b986620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:02.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.763+0000 7f865ac9db80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c38b974be0 0x55c38b9839b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:02.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.763+0000 7f865ac9db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c38b97fad0 0x55c38b984030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:02.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.763+0000 7f865ac9db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c38b984770 con 0x55c38b97fad0 2023-07-18T03:09:02.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.763+0000 7f865ac9db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c38b9e1e70 con 0x55c38b9742d0 2023-07-18T03:09:02.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.763+0000 7f865ac9db80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c38b9e1fb0 con 0x55c38b974be0 2023-07-18T03:09:02.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.763+0000 7f865ac9db80 10 client.0 _ll_register_callbacks cb 0x7f863801e050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:09:02.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.763+0000 7f864519b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38b9742d0 0x55c38b986620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:02.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.763+0000 7f864519b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38b9742d0 0x55c38b986620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:51222/0 (socket says 172.21.15.101:51222) 2023-07-18T03:09:02.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.763+0000 7f864519b700 1 -- 172.21.15.101:0/1073467358 learned_addr learned my addr 172.21.15.101:0/1073467358 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:02.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.763+0000 7f864599c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c38b97fad0 0x55c38b984030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:02.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.763+0000 7f864499a700 1 --2- 172.21.15.101:0/1073467358 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c38b974be0 0x55c38b9839b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:02.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.764+0000 7f864519b700 1 -- 172.21.15.101:0/1073467358 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c38b974be0 msgr2=0x55c38b9839b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:02.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.764+0000 7f864519b700 1 --2- 172.21.15.101:0/1073467358 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c38b974be0 0x55c38b9839b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.764+0000 7f864519b700 1 -- 172.21.15.101:0/1073467358 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c38b97fad0 msgr2=0x55c38b984030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:02.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.764+0000 7f864519b700 1 --2- 172.21.15.101:0/1073467358 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c38b97fad0 0x55c38b984030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.764+0000 7f864519b700 1 -- 172.21.15.101:0/1073467358 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c38b6ddc60 con 0x55c38b9742d0 2023-07-18T03:09:02.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.764+0000 7f864519b700 1 --2- 172.21.15.101:0/1073467358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38b9742d0 0x55c38b986620 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f863c02c8a0 tx=0x7f863c00bf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2585e8a3dec1938c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:02.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.764+0000 7f8628ff9700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:09:02.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.764+0000 7f8628ff9700 1 -- 172.21.15.101:0/1073467358 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863c01d030 con 0x55c38b9742d0 2023-07-18T03:09:02.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.764+0000 7f865ac9db80 1 -- 172.21.15.101:0/1073467358 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f863c01e040 con 0x55c38b9742d0 2023-07-18T03:09:02.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.764+0000 7f8628ff9700 1 -- 172.21.15.101:0/1073467358 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f863c000e40 con 0x55c38b9742d0 2023-07-18T03:09:02.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.764+0000 7f8628ff9700 1 -- 172.21.15.101:0/1073467358 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f863c02c3f0 con 0x55c38b9742d0 2023-07-18T03:09:02.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.764+0000 7f865ac9db80 10 client.25859 fetch_fsmap learned FSMap version 25 2023-07-18T03:09:02.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.764+0000 7f865ac9db80 1 -- 172.21.15.101:0/1073467358 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c38b6f32b0 con 0x55c38b9742d0 2023-07-18T03:09:02.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.765+0000 7f8628ff9700 1 -- 172.21.15.101:0/1073467358 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f863c01e040 con 0x55c38b9742d0 2023-07-18T03:09:02.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.765+0000 7f8628ff9700 1 -- 172.21.15.101:0/1073467358 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f863c00b850 con 0x55c38b9742d0 2023-07-18T03:09:02.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.765+0000 7f865ac9db80 10 client.25859 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:09:02.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.765+0000 7f865ac9db80 10 client.25859 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:09:02.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.765+0000 7f865ac9db80 20 client.25859 populate_metadata read hostname 'smithi101' 2023-07-18T03:09:02.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.765+0000 7f865ac9db80 1 --2- 172.21.15.101:0/1073467358 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55c38b9e3190 0x55c38b9e5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:02.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.765+0000 7f865ac9db80 4 client.25859 mds_command: new command op to 24541 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:09:02.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.765+0000 7f865ac9db80 1 -- 172.21.15.101:0/1073467358 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c38b882050 con 0x55c38b9e3190 2023-07-18T03:09:02.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.765+0000 7f864499a700 1 --2- 172.21.15.101:0/1073467358 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55c38b9e3190 0x55c38b9e5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:02.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.766+0000 7f8628ff9700 10 client.25859 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:09:02.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.766+0000 7f864499a700 1 --2- 172.21.15.101:0/1073467358 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55c38b9e3190 0x55c38b9e5570 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2f4aae4562c4a466 server_cookie=12ae4e7f988e442b in_seq=0 out_seq=0 2023-07-18T03:09:02.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.766+0000 7f8628ff9700 1 -- 172.21.15.101:0/1073467358 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c38b882050 con 0x55c38b9e3190 2023-07-18T03:09:02.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.766+0000 7f8628ff9700 10 client.25859 handle_command_reply: tid=0 2023-07-18T03:09:02.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.766+0000 7f865ac9db80 1 client.25859 shutdown 2023-07-18T03:09:02.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.766+0000 7f865ac9db80 1 -- 172.21.15.101:0/1073467358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38b9742d0 msgr2=0x55c38b986620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:02.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.766+0000 7f865ac9db80 1 --2- 172.21.15.101:0/1073467358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38b9742d0 0x55c38b986620 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f863c02c8a0 tx=0x7f863c00bf40 comp rx=0 tx=0).stop 2023-07-18T03:09:02.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.766+0000 7f865ac9db80 1 -- 172.21.15.101:0/1073467358 shutdown_connections 2023-07-18T03:09:02.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.766+0000 7f865ac9db80 1 --2- 172.21.15.101:0/1073467358 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55c38b9e3190 0x55c38b9e5570 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.766+0000 7f865ac9db80 1 --2- 172.21.15.101:0/1073467358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c38b9742d0 0x55c38b986620 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.767+0000 7f8628ff9700 0 client.25859 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:09:02.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.767+0000 7f865ac9db80 1 --2- 172.21.15.101:0/1073467358 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c38b97fad0 0x55c38b984030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.767+0000 7f865ac9db80 1 --2- 172.21.15.101:0/1073467358 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c38b974be0 0x55c38b9839b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.767+0000 7f865ac9db80 1 -- 172.21.15.101:0/1073467358 >> 172.21.15.101:0/1073467358 conn(0x55c38b96ada0 msgr2=0x55c38b977ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:02.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.767+0000 7f865ac9db80 1 -- 172.21.15.101:0/1073467358 shutdown_connections 2023-07-18T03:09:02.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.767+0000 7f865ac9db80 1 -- 172.21.15.101:0/1073467358 wait complete. 2023-07-18T03:09:02.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.767+0000 7f865ac9db80 20 client.25859 trim_cache size 0 max 16384 2023-07-18T03:09:02.765 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:09:02.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.767+0000 7f86473ff700 1 -- 172.21.15.101:0/2478028663 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f862c0a13e0 msgr2=0x7f862c0a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:02.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.767+0000 7f86473ff700 1 --2- 172.21.15.101:0/2478028663 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f862c0a13e0 0x7f862c0a3890 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f86380042f0 tx=0x7f8638003ac0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.767+0000 7f86473ff700 1 -- 172.21.15.101:0/2478028663 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f86400731c0 msgr2=0x7f864019c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:02.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.767+0000 7f86473ff700 1 --2- 172.21.15.101:0/2478028663 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f86400731c0 0x7f864019c770 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f863000e9a0 tx=0x7f863000ecb0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.767+0000 7f86473ff700 1 -- 172.21.15.101:0/2478028663 shutdown_connections 2023-07-18T03:09:02.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.767+0000 7f86473ff700 1 --2- 172.21.15.101:0/2478028663 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f862c0a13e0 0x7f862c0a3890 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.767+0000 7f86473ff700 1 --2- 172.21.15.101:0/2478028663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86400728b0 0x7f864019c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.767+0000 7f86473ff700 1 --2- 172.21.15.101:0/2478028663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8640110af0 0x7f86401a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.767+0000 7f86473ff700 1 --2- 172.21.15.101:0/2478028663 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f86400731c0 0x7f864019c770 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:02.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.767+0000 7f86473ff700 1 -- 172.21.15.101:0/2478028663 >> 172.21.15.101:0/2478028663 conn(0x7f86400fb5e0 msgr2=0x7f86400fc240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:02.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.768+0000 7f86473ff700 1 -- 172.21.15.101:0/2478028663 shutdown_connections 2023-07-18T03:09:02.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:02.768+0000 7f86473ff700 1 -- 172.21.15.101:0/2478028663 wait complete. 2023-07-18T03:09:02.774 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-18T03:09:03.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.025+0000 7f99579ff700 1 -- 172.21.15.101:0/2053568889 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9950100490 msgr2=0x7f995010c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:03.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.025+0000 7f99579ff700 1 --2- 172.21.15.101:0/2053568889 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9950100490 0x7f995010c920 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f994800b0a0 tx=0x7f994800b3b0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.025+0000 7f99579ff700 1 -- 172.21.15.101:0/2053568889 shutdown_connections 2023-07-18T03:09:03.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.025+0000 7f99579ff700 1 --2- 172.21.15.101:0/2053568889 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9950105af0 0x7f9950105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.025+0000 7f99579ff700 1 --2- 172.21.15.101:0/2053568889 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9950100490 0x7f995010c920 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.025+0000 7f99579ff700 1 --2- 172.21.15.101:0/2053568889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99500ffae0 0x7f99500fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.025+0000 7f99579ff700 1 -- 172.21.15.101:0/2053568889 >> 172.21.15.101:0/2053568889 conn(0x7f99500fb5e0 msgr2=0x7f99500fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:03.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.025+0000 7f99579ff700 1 -- 172.21.15.101:0/2053568889 shutdown_connections 2023-07-18T03:09:03.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.025+0000 7f99579ff700 1 -- 172.21.15.101:0/2053568889 wait complete. 2023-07-18T03:09:03.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.026+0000 7f99579ff700 1 Processor -- start 2023-07-18T03:09:03.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.026+0000 7f99579ff700 1 -- start start 2023-07-18T03:09:03.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.026+0000 7f99579ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99500ffae0 0x7f9950072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:03.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.026+0000 7f99579ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9950100490 0x7f9950072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:03.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.026+0000 7f99579ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9950105af0 0x7f99501a3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:03.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.026+0000 7f99579ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9950073580 con 0x7f9950100490 2023-07-18T03:09:03.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.026+0000 7f99579ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99501a43f0 con 0x7f99500ffae0 2023-07-18T03:09:03.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.026+0000 7f99579ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99501a4530 con 0x7f9950105af0 2023-07-18T03:09:03.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.027+0000 7f9954f9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9950100490 0x7f9950072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:03.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.027+0000 7f9955f9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9950105af0 0x7f99501a3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:03.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.027+0000 7f9954f9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9950100490 0x7f9950072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:35280/0 (socket says 172.21.15.101:35280) 2023-07-18T03:09:03.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.027+0000 7f9954f9a700 1 -- 172.21.15.101:0/1001775803 learned_addr learned my addr 172.21.15.101:0/1001775803 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:03.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.027+0000 7f995579b700 1 --2- 172.21.15.101:0/1001775803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99500ffae0 0x7f9950072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:03.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.027+0000 7f9954f9a700 1 -- 172.21.15.101:0/1001775803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9950105af0 msgr2=0x7f99501a3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:03.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.027+0000 7f9954f9a700 1 --2- 172.21.15.101:0/1001775803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9950105af0 0x7f99501a3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.027+0000 7f9954f9a700 1 -- 172.21.15.101:0/1001775803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99500ffae0 msgr2=0x7f9950072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:03.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.027+0000 7f9954f9a700 1 --2- 172.21.15.101:0/1001775803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99500ffae0 0x7f9950072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.027+0000 7f9954f9a700 1 -- 172.21.15.101:0/1001775803 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9948009d00 con 0x7f9950100490 2023-07-18T03:09:03.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.027+0000 7f9955f9c700 1 --2- 172.21.15.101:0/1001775803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9950105af0 0x7f99501a3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:09:03.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.027+0000 7f9954f9a700 1 --2- 172.21.15.101:0/1001775803 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9950100490 0x7f9950072d50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f994000e9a0 tx=0x7f994000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:03.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.028+0000 7f99467fc700 1 -- 172.21.15.101:0/1001775803 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9940009790 con 0x7f9950100490 2023-07-18T03:09:03.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.028+0000 7f99467fc700 1 -- 172.21.15.101:0/1001775803 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9940010430 con 0x7f9950100490 2023-07-18T03:09:03.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.028+0000 7f99579ff700 1 -- 172.21.15.101:0/1001775803 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99501a4790 con 0x7f9950100490 2023-07-18T03:09:03.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.028+0000 7f99467fc700 1 -- 172.21.15.101:0/1001775803 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9940018680 con 0x7f9950100490 2023-07-18T03:09:03.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.028+0000 7f99579ff700 1 -- 172.21.15.101:0/1001775803 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99501a4cc0 con 0x7f9950100490 2023-07-18T03:09:03.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.031+0000 7f99467fc700 1 -- 172.21.15.101:0/1001775803 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 24) v1 ==== 135949+0+0 (secure 0 0 0) 0x7f99400105a0 con 0x7f9950100490 2023-07-18T03:09:03.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.033+0000 7f996b3f2b80 1 Processor -- start 2023-07-18T03:09:03.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.033+0000 7f996b3f2b80 1 -- start start 2023-07-18T03:09:03.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.033+0000 7f996b3f2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c6eca82d0 0x557c6eca86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:03.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.033+0000 7f996b3f2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c6eca8be0 0x557c6ecb3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:03.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.033+0000 7f996b3f2b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c6ecb3ad0 0x557c6ecb5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:03.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.033+0000 7f996b3f2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c6ebaecb0 con 0x557c6eca82d0 2023-07-18T03:09:03.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.033+0000 7f996b3f2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c6ec01a40 con 0x557c6eca8be0 2023-07-18T03:09:03.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.033+0000 7f996b3f2b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c6ebfaed0 con 0x557c6ecb3ad0 2023-07-18T03:09:03.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.033+0000 7f99467fc700 1 --2- 172.21.15.101:0/1001775803 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f993c0a1460 0x7f993c0a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:03.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.034+0000 7f995579b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c6eca8be0 0x557c6ecb3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:03.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.034+0000 7f995579b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c6eca8be0 0x557c6ecb3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:51258/0 (socket says 172.21.15.101:51258) 2023-07-18T03:09:03.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.034+0000 7f995579b700 1 -- 172.21.15.101:0/844010720 learned_addr learned my addr 172.21.15.101:0/844010720 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:03.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.034+0000 7f9955f9c700 1 --2- 172.21.15.101:0/844010720 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c6eca82d0 0x557c6eca86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:03.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.034+0000 7f99467fc700 1 -- 172.21.15.101:0/1001775803 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f9940014070 con 0x7f9950100490 2023-07-18T03:09:03.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.034+0000 7f9954f9a700 1 --2- 172.21.15.101:0/844010720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c6ecb3ad0 0x557c6ecb5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:03.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.034+0000 7f9955f9c700 1 --2- 172.21.15.101:0/1001775803 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f993c0a1460 0x7f993c0a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:03.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.034+0000 7f995579b700 1 -- 172.21.15.101:0/844010720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c6ecb3ad0 msgr2=0x557c6ecb5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:03.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.034+0000 7f995579b700 1 --2- 172.21.15.101:0/844010720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c6ecb3ad0 0x557c6ecb5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.034+0000 7f995579b700 1 -- 172.21.15.101:0/844010720 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c6eca82d0 msgr2=0x557c6eca86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:03.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.034+0000 7f995579b700 1 --2- 172.21.15.101:0/844010720 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c6eca82d0 0x557c6eca86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.034+0000 7f995579b700 1 -- 172.21.15.101:0/844010720 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c6ea28de0 con 0x557c6eca8be0 2023-07-18T03:09:03.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.034+0000 7f995579b700 1 --2- 172.21.15.101:0/844010720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c6eca8be0 0x557c6ecb3590 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f994c009d00 tx=0x7f994c011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37c6b58dce58e18f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:03.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.034+0000 7f9955f9c700 1 --2- 172.21.15.101:0/1001775803 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f993c0a1460 0x7f993c0a3910 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9950074920 tx=0x7f9948006230 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:03.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.035+0000 7f993b7fe700 1 -- 172.21.15.101:0/844010720 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994c004230 con 0x557c6eca8be0 2023-07-18T03:09:03.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.035+0000 7f993b7fe700 1 -- 172.21.15.101:0/844010720 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f994c004390 con 0x557c6eca8be0 2023-07-18T03:09:03.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.035+0000 7f993b7fe700 1 -- 172.21.15.101:0/844010720 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994c019770 con 0x557c6eca8be0 2023-07-18T03:09:03.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.035+0000 7f996b3f2b80 1 -- 172.21.15.101:0/844010720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c6eca8be0 msgr2=0x557c6ecb3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:03.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.035+0000 7f996b3f2b80 1 --2- 172.21.15.101:0/844010720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c6eca8be0 0x557c6ecb3590 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f994c009d00 tx=0x7f994c011720 comp rx=0 tx=0).stop 2023-07-18T03:09:03.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.035+0000 7f996b3f2b80 1 -- 172.21.15.101:0/844010720 shutdown_connections 2023-07-18T03:09:03.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.035+0000 7f996b3f2b80 1 --2- 172.21.15.101:0/844010720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c6ecb3ad0 0x557c6ecb5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.035+0000 7f996b3f2b80 1 --2- 172.21.15.101:0/844010720 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c6eca82d0 0x557c6eca86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.035+0000 7f996b3f2b80 1 --2- 172.21.15.101:0/844010720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c6eca8be0 0x557c6ecb3590 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.035+0000 7f996b3f2b80 1 -- 172.21.15.101:0/844010720 >> 172.21.15.101:0/844010720 conn(0x557c6ec9eda0 msgr2=0x557c6eca46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:03.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.036+0000 7f996b3f2b80 1 -- 172.21.15.101:0/844010720 shutdown_connections 2023-07-18T03:09:03.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.036+0000 7f996b3f2b80 1 -- 172.21.15.101:0/844010720 wait complete. 2023-07-18T03:09:03.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.036+0000 7f996b3f2b80 1 -- start start 2023-07-18T03:09:03.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.036+0000 7f996b3f2b80 1 Processor -- start 2023-07-18T03:09:03.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.037+0000 7f996b3f2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c6eca82d0 0x557c6ecb3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:03.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.037+0000 7f996b3f2b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c6eca8be0 0x557c6ecb0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:03.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.037+0000 7f996b3f2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c6ecb3ad0 0x557c6ecb0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:03.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.037+0000 7f996b3f2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c6eb9aac0 con 0x557c6ecb3ad0 2023-07-18T03:09:03.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.037+0000 7f996b3f2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c6eb9a7c0 con 0x557c6eca82d0 2023-07-18T03:09:03.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.037+0000 7f996b3f2b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c6ea7e920 con 0x557c6eca8be0 2023-07-18T03:09:03.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.037+0000 7f996b3f2b80 10 client.0 _ll_register_callbacks cb 0x557c6eb98760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:09:03.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.037+0000 7f9955f9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c6ecb3ad0 0x557c6ecb0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:03.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.037+0000 7f995579b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c6eca82d0 0x557c6ecb3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:03.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.037+0000 7f995579b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c6eca82d0 0x557c6ecb3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:51270/0 (socket says 172.21.15.101:51270) 2023-07-18T03:09:03.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.037+0000 7f9955f9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c6ecb3ad0 0x557c6ecb0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:35292/0 (socket says 172.21.15.101:35292) 2023-07-18T03:09:03.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.037+0000 7f995579b700 1 -- 172.21.15.101:0/3099221481 learned_addr learned my addr 172.21.15.101:0/3099221481 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:03.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.037+0000 7f9954f9a700 1 --2- 172.21.15.101:0/3099221481 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c6eca8be0 0x557c6ecb0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:03.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.037+0000 7f9955f9c700 1 -- 172.21.15.101:0/3099221481 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c6eca8be0 msgr2=0x557c6ecb0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:03.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.037+0000 7f9955f9c700 1 --2- 172.21.15.101:0/3099221481 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c6eca8be0 0x557c6ecb0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.037+0000 7f9955f9c700 1 -- 172.21.15.101:0/3099221481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c6eca82d0 msgr2=0x557c6ecb3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:03.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.037+0000 7f9955f9c700 1 --2- 172.21.15.101:0/3099221481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c6eca82d0 0x557c6ecb3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.038+0000 7f9955f9c700 1 -- 172.21.15.101:0/3099221481 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f994c009960 con 0x557c6ecb3ad0 2023-07-18T03:09:03.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.038+0000 7f995579b700 1 --2- 172.21.15.101:0/3099221481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c6eca82d0 0x557c6ecb3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:09:03.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.038+0000 7f9955f9c700 1 --2- 172.21.15.101:0/3099221481 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c6ecb3ad0 0x557c6ecb0f20 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f99480062a0 tx=0x7f99480099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f20f8183e7c501b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:03.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.038+0000 7f9938ff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:09:03.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.038+0000 7f9938ff9700 1 -- 172.21.15.101:0/3099221481 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994800e040 con 0x557c6ecb3ad0 2023-07-18T03:09:03.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.038+0000 7f996b3f2b80 1 -- 172.21.15.101:0/3099221481 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c6eb544f0 con 0x557c6ecb3ad0 2023-07-18T03:09:03.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.039+0000 7f9938ff9700 1 -- 172.21.15.101:0/3099221481 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f9948020860 con 0x557c6ecb3ad0 2023-07-18T03:09:03.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.039+0000 7f9938ff9700 1 -- 172.21.15.101:0/3099221481 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9948020a20 con 0x557c6ecb3ad0 2023-07-18T03:09:03.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.039+0000 7f996b3f2b80 10 client.15840 fetch_fsmap learned FSMap version 25 2023-07-18T03:09:03.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.039+0000 7f996b3f2b80 1 -- 172.21.15.101:0/3099221481 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557c6ea11c60 con 0x557c6ecb3ad0 2023-07-18T03:09:03.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.039+0000 7f9938ff9700 1 -- 172.21.15.101:0/3099221481 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557c6eb544f0 con 0x557c6ecb3ad0 2023-07-18T03:09:03.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.039+0000 7f9938ff9700 1 -- 172.21.15.101:0/3099221481 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f994802a070 con 0x557c6ecb3ad0 2023-07-18T03:09:03.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.039+0000 7f996b3f2b80 10 client.15840 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:09:03.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.039+0000 7f996b3f2b80 10 client.15840 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:09:03.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.039+0000 7f996b3f2b80 20 client.15840 populate_metadata read hostname 'smithi101' 2023-07-18T03:09:03.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.039+0000 7f996b3f2b80 1 --2- 172.21.15.101:0/3099221481 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x557c6ecc3300 0x557c6ecc56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:03.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.039+0000 7f996b3f2b80 4 client.15840 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:09:03.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.039+0000 7f996b3f2b80 1 -- 172.21.15.101:0/3099221481 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c6ebf3eb0 con 0x557c6ecc3300 2023-07-18T03:09:03.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.040+0000 7f995579b700 1 --2- 172.21.15.101:0/3099221481 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x557c6ecc3300 0x557c6ecc56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:03.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.040+0000 7f9938ff9700 10 client.15840 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:09:03.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.040+0000 7f995579b700 1 --2- 172.21.15.101:0/3099221481 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x557c6ecc3300 0x557c6ecc56e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ec527ab284a493a4 server_cookie=d967252b1fd97091 in_seq=0 out_seq=0 2023-07-18T03:09:03.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.041+0000 7f9938ff9700 1 -- 172.21.15.101:0/3099221481 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557c6ebf3eb0 con 0x557c6ecc3300 2023-07-18T03:09:03.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.041+0000 7f9938ff9700 10 client.15840 handle_command_reply: tid=0 2023-07-18T03:09:03.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.041+0000 7f996b3f2b80 1 client.15840 shutdown 2023-07-18T03:09:03.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.041+0000 7f996b3f2b80 1 -- 172.21.15.101:0/3099221481 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c6ecb3ad0 msgr2=0x557c6ecb0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:03.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.041+0000 7f996b3f2b80 1 --2- 172.21.15.101:0/3099221481 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c6ecb3ad0 0x557c6ecb0f20 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f99480062a0 tx=0x7f99480099e0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.041+0000 7f996b3f2b80 1 -- 172.21.15.101:0/3099221481 shutdown_connections 2023-07-18T03:09:03.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.041+0000 7f996b3f2b80 1 --2- 172.21.15.101:0/3099221481 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x557c6ecc3300 0x557c6ecc56e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.041+0000 7f996b3f2b80 1 --2- 172.21.15.101:0/3099221481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c6eca82d0 0x557c6ecb3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.041+0000 7f996b3f2b80 1 --2- 172.21.15.101:0/3099221481 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c6ecb3ad0 0x557c6ecb0f20 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.041+0000 7f9938ff9700 0 client.15840 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:09:03.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.041+0000 7f996b3f2b80 1 --2- 172.21.15.101:0/3099221481 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c6eca8be0 0x557c6ecb0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.041+0000 7f996b3f2b80 1 -- 172.21.15.101:0/3099221481 >> 172.21.15.101:0/3099221481 conn(0x557c6ec9eda0 msgr2=0x557c6eca4230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:03.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.041+0000 7f996b3f2b80 1 -- 172.21.15.101:0/3099221481 shutdown_connections 2023-07-18T03:09:03.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.041+0000 7f996b3f2b80 1 -- 172.21.15.101:0/3099221481 wait complete. 2023-07-18T03:09:03.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.042+0000 7f996b3f2b80 20 client.15840 trim_cache size 0 max 16384 2023-07-18T03:09:03.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.055+0000 7f996b3f2b80 1 Processor -- start 2023-07-18T03:09:03.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.055+0000 7f996b3f2b80 1 -- start start 2023-07-18T03:09:03.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.055+0000 7f996b3f2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c6eca82d0 0x557c6ecb0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:03.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.055+0000 7f996b3f2b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c6eca8be0 0x557c6ecab1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:03.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.055+0000 7f996b3f2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c6ecb3ad0 0x557c6ecba4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:03.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.055+0000 7f996b3f2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c6ec9e970 con 0x557c6ecb3ad0 2023-07-18T03:09:03.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.055+0000 7f996b3f2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c6ecb1d70 con 0x557c6eca82d0 2023-07-18T03:09:03.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.055+0000 7f996b3f2b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c6ecab6f0 con 0x557c6eca8be0 2023-07-18T03:09:03.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.056+0000 7f9955f9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c6ecb3ad0 0x557c6ecba4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:03.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.056+0000 7f9955f9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c6ecb3ad0 0x557c6ecba4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:35298/0 (socket says 172.21.15.101:35298) 2023-07-18T03:09:03.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.056+0000 7f995579b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c6eca82d0 0x557c6ecb0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:03.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.056+0000 7f9955f9c700 1 -- 172.21.15.101:0/940505682 learned_addr learned my addr 172.21.15.101:0/940505682 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:03.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.056+0000 7f9954f9a700 1 --2- 172.21.15.101:0/940505682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c6eca8be0 0x557c6ecab1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:03.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.056+0000 7f9955f9c700 1 -- 172.21.15.101:0/940505682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c6eca8be0 msgr2=0x557c6ecab1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:03.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.056+0000 7f9955f9c700 1 --2- 172.21.15.101:0/940505682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c6eca8be0 0x557c6ecab1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.056+0000 7f9955f9c700 1 -- 172.21.15.101:0/940505682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c6eca82d0 msgr2=0x557c6ecb0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:03.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.056+0000 7f9955f9c700 1 --2- 172.21.15.101:0/940505682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c6eca82d0 0x557c6ecb0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.056+0000 7f9955f9c700 1 -- 172.21.15.101:0/940505682 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c6ea4f250 con 0x557c6ecb3ad0 2023-07-18T03:09:03.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.056+0000 7f995579b700 1 --2- 172.21.15.101:0/940505682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c6eca82d0 0x557c6ecb0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:09:03.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.056+0000 7f9954f9a700 1 --2- 172.21.15.101:0/940505682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c6eca8be0 0x557c6ecab1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:09:03.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.056+0000 7f9955f9c700 1 --2- 172.21.15.101:0/940505682 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c6ecb3ad0 0x557c6ecba4b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9948009cd0 tx=0x7f994801e3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e9b5cfd79867b1 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:03.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.056+0000 7f993bfff700 1 -- 172.21.15.101:0/940505682 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9948007980 con 0x557c6ecb3ad0 2023-07-18T03:09:03.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.056+0000 7f993bfff700 1 -- 172.21.15.101:0/940505682 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9948027040 con 0x557c6ecb3ad0 2023-07-18T03:09:03.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.056+0000 7f993bfff700 1 -- 172.21.15.101:0/940505682 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9948020cf0 con 0x557c6ecb3ad0 2023-07-18T03:09:03.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.057+0000 7f996b3f2b80 1 -- 172.21.15.101:0/940505682 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c6ecb3ad0 msgr2=0x557c6ecba4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:03.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.057+0000 7f996b3f2b80 1 --2- 172.21.15.101:0/940505682 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c6ecb3ad0 0x557c6ecba4b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9948009cd0 tx=0x7f994801e3d0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.057+0000 7f996b3f2b80 1 -- 172.21.15.101:0/940505682 shutdown_connections 2023-07-18T03:09:03.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.057+0000 7f996b3f2b80 1 --2- 172.21.15.101:0/940505682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c6eca82d0 0x557c6ecb0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.057+0000 7f996b3f2b80 1 --2- 172.21.15.101:0/940505682 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c6ecb3ad0 0x557c6ecba4b0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.057+0000 7f996b3f2b80 1 --2- 172.21.15.101:0/940505682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c6eca8be0 0x557c6ecab1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.057+0000 7f996b3f2b80 1 -- 172.21.15.101:0/940505682 >> 172.21.15.101:0/940505682 conn(0x557c6ec9eda0 msgr2=0x557c6eca38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:03.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.057+0000 7f996b3f2b80 1 -- 172.21.15.101:0/940505682 shutdown_connections 2023-07-18T03:09:03.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.057+0000 7f996b3f2b80 1 -- 172.21.15.101:0/940505682 wait complete. 2023-07-18T03:09:03.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.057+0000 7f996b3f2b80 1 -- start start 2023-07-18T03:09:03.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.058+0000 7f996b3f2b80 1 Processor -- start 2023-07-18T03:09:03.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.058+0000 7f996b3f2b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c6eca82d0 0x557c6ecb9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:03.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.058+0000 7f996b3f2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c6eca8be0 0x557c6ecba410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:03.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.058+0000 7f996b3f2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c6ecb3ad0 0x557c6ecb8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:03.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.058+0000 7f996b3f2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c6ecb8ba0 con 0x557c6eca8be0 2023-07-18T03:09:03.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.058+0000 7f996b3f2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c6ecb8d10 con 0x557c6ecb3ad0 2023-07-18T03:09:03.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.058+0000 7f996b3f2b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c6ecb8ed0 con 0x557c6eca82d0 2023-07-18T03:09:03.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.058+0000 7f996b3f2b80 10 client.0 _ll_register_callbacks cb 0x7f9948009340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:09:03.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.058+0000 7f9954f9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c6eca8be0 0x557c6ecba410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:03.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.058+0000 7f9954f9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c6eca8be0 0x557c6ecba410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:35310/0 (socket says 172.21.15.101:35310) 2023-07-18T03:09:03.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.058+0000 7f9954f9a700 1 -- 172.21.15.101:0/2305155262 learned_addr learned my addr 172.21.15.101:0/2305155262 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:03.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.058+0000 7f9955f9c700 1 --2- 172.21.15.101:0/2305155262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c6ecb3ad0 0x557c6ecb8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:03.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.059+0000 7f995579b700 1 --2- 172.21.15.101:0/2305155262 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c6eca82d0 0x557c6ecb9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:03.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.059+0000 7f9954f9a700 1 -- 172.21.15.101:0/2305155262 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c6eca82d0 msgr2=0x557c6ecb9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:03.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.059+0000 7f9954f9a700 1 --2- 172.21.15.101:0/2305155262 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c6eca82d0 0x557c6ecb9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.059+0000 7f9954f9a700 1 -- 172.21.15.101:0/2305155262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c6ecb3ad0 msgr2=0x557c6ecb8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:03.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.059+0000 7f9954f9a700 1 --2- 172.21.15.101:0/2305155262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c6ecb3ad0 0x557c6ecb8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.059+0000 7f9954f9a700 1 -- 172.21.15.101:0/2305155262 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557c6ea28de0 con 0x557c6eca8be0 2023-07-18T03:09:03.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.059+0000 7f9955f9c700 1 --2- 172.21.15.101:0/2305155262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c6ecb3ad0 0x557c6ecb8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:09:03.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.059+0000 7f9954f9a700 1 --2- 172.21.15.101:0/2305155262 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c6eca8be0 0x557c6ecba410 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f99400964c0 tx=0x7f994009a500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dfc1c7b6d7537ac server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:03.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.059+0000 7f9938ff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:09:03.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.059+0000 7f9938ff9700 1 -- 172.21.15.101:0/2305155262 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9940094dd0 con 0x557c6eca8be0 2023-07-18T03:09:03.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.059+0000 7f996b3f2b80 1 -- 172.21.15.101:0/2305155262 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f994c004390 con 0x557c6eca8be0 2023-07-18T03:09:03.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.059+0000 7f9938ff9700 1 -- 172.21.15.101:0/2305155262 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f99400a2410 con 0x557c6eca8be0 2023-07-18T03:09:03.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.059+0000 7f9938ff9700 1 -- 172.21.15.101:0/2305155262 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99400b8250 con 0x557c6eca8be0 2023-07-18T03:09:03.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.059+0000 7f996b3f2b80 10 client.15846 fetch_fsmap learned FSMap version 25 2023-07-18T03:09:03.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.059+0000 7f996b3f2b80 1 -- 172.21.15.101:0/2305155262 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557c6ea11c60 con 0x557c6eca8be0 2023-07-18T03:09:03.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.060+0000 7f9938ff9700 1 -- 172.21.15.101:0/2305155262 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9940078e90 con 0x557c6eca8be0 2023-07-18T03:09:03.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.060+0000 7f9938ff9700 1 -- 172.21.15.101:0/2305155262 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f99400a1f80 con 0x557c6eca8be0 2023-07-18T03:09:03.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.060+0000 7f996b3f2b80 10 client.15846 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:09:03.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.060+0000 7f996b3f2b80 10 client.15846 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:09:03.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.060+0000 7f996b3f2b80 20 client.15846 populate_metadata read hostname 'smithi101' 2023-07-18T03:09:03.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.060+0000 7f996b3f2b80 1 --2- 172.21.15.101:0/2305155262 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x557c6ed18260 0x557c6ed1a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:03.061 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.060+0000 7f996b3f2b80 4 client.15846 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:09:03.061 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.060+0000 7f996b3f2b80 1 -- 172.21.15.101:0/2305155262 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557c6ebb6050 con 0x557c6ed18260 2023-07-18T03:09:03.061 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.060+0000 7f995579b700 1 --2- 172.21.15.101:0/2305155262 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x557c6ed18260 0x557c6ed1a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:03.061 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.061+0000 7f9938ff9700 10 client.15846 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:09:03.061 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.061+0000 7f995579b700 1 --2- 172.21.15.101:0/2305155262 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x557c6ed18260 0x557c6ed1a670 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=50ff426188c26e5d server_cookie=3f5bef0adbe67e78 in_seq=0 out_seq=0 2023-07-18T03:09:03.061 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.061+0000 7f9938ff9700 1 -- 172.21.15.101:0/2305155262 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557c6ebb6050 con 0x557c6ed18260 2023-07-18T03:09:03.061 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.061+0000 7f9938ff9700 10 client.15846 handle_command_reply: tid=0 2023-07-18T03:09:03.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.061+0000 7f996b3f2b80 1 client.15846 shutdown 2023-07-18T03:09:03.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.061+0000 7f996b3f2b80 1 -- 172.21.15.101:0/2305155262 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c6eca8be0 msgr2=0x557c6ecba410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:03.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.061+0000 7f996b3f2b80 1 --2- 172.21.15.101:0/2305155262 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c6eca8be0 0x557c6ecba410 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f99400964c0 tx=0x7f994009a500 comp rx=0 tx=0).stop 2023-07-18T03:09:03.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.061+0000 7f996b3f2b80 1 -- 172.21.15.101:0/2305155262 shutdown_connections 2023-07-18T03:09:03.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.061+0000 7f996b3f2b80 1 --2- 172.21.15.101:0/2305155262 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x557c6ed18260 0x557c6ed1a670 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.061+0000 7f996b3f2b80 1 --2- 172.21.15.101:0/2305155262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c6ecb3ad0 0x557c6ecb8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.061+0000 7f9938ff9700 0 client.15846 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:09:03.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.061+0000 7f996b3f2b80 1 --2- 172.21.15.101:0/2305155262 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c6eca82d0 0x557c6ecb9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.061+0000 7f996b3f2b80 1 --2- 172.21.15.101:0/2305155262 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c6eca8be0 0x557c6ecba410 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.061+0000 7f996b3f2b80 1 -- 172.21.15.101:0/2305155262 >> 172.21.15.101:0/2305155262 conn(0x557c6ec9eda0 msgr2=0x557c6eca4140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:03.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.061+0000 7f996b3f2b80 1 -- 172.21.15.101:0/2305155262 shutdown_connections 2023-07-18T03:09:03.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.061+0000 7f996b3f2b80 1 -- 172.21.15.101:0/2305155262 wait complete. 2023-07-18T03:09:03.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.061+0000 7f996b3f2b80 20 client.15846 trim_cache size 0 max 16384 2023-07-18T03:09:03.063 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:09:03.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.062+0000 7f99579ff700 1 -- 172.21.15.101:0/1001775803 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f993c0a1460 msgr2=0x7f993c0a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:03.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.062+0000 7f99579ff700 1 --2- 172.21.15.101:0/1001775803 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f993c0a1460 0x7f993c0a3910 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9950074920 tx=0x7f9948006230 comp rx=0 tx=0).stop 2023-07-18T03:09:03.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.062+0000 7f99579ff700 1 -- 172.21.15.101:0/1001775803 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9950100490 msgr2=0x7f9950072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:03.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.062+0000 7f99579ff700 1 --2- 172.21.15.101:0/1001775803 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9950100490 0x7f9950072d50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f994000e9a0 tx=0x7f994000ecb0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.062+0000 7f99579ff700 1 -- 172.21.15.101:0/1001775803 shutdown_connections 2023-07-18T03:09:03.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.062+0000 7f99579ff700 1 --2- 172.21.15.101:0/1001775803 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f993c0a1460 0x7f993c0a3910 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.062+0000 7f99579ff700 1 --2- 172.21.15.101:0/1001775803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99500ffae0 0x7f9950072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.062+0000 7f99579ff700 1 --2- 172.21.15.101:0/1001775803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9950105af0 0x7f99501a3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.062+0000 7f99579ff700 1 --2- 172.21.15.101:0/1001775803 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9950100490 0x7f9950072d50 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.062+0000 7f99579ff700 1 -- 172.21.15.101:0/1001775803 >> 172.21.15.101:0/1001775803 conn(0x7f99500fb5e0 msgr2=0x7f99501035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:03.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.062+0000 7f99579ff700 1 -- 172.21.15.101:0/1001775803 shutdown_connections 2023-07-18T03:09:03.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.062+0000 7f99579ff700 1 -- 172.21.15.101:0/1001775803 wait complete. 2023-07-18T03:09:03.069 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-18T03:09:03.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.280+0000 7f383ebff700 1 -- 172.21.15.101:0/3080328769 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3838105af0 msgr2=0x7f3838105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:03.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.280+0000 7f383ebff700 1 --2- 172.21.15.101:0/3080328769 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3838105af0 0x7f3838105ec0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f382c009990 tx=0x7f382c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.280+0000 7f383ebff700 1 -- 172.21.15.101:0/3080328769 shutdown_connections 2023-07-18T03:09:03.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.280+0000 7f383ebff700 1 --2- 172.21.15.101:0/3080328769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3838100490 0x7f383810c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.280+0000 7f383ebff700 1 --2- 172.21.15.101:0/3080328769 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3838105af0 0x7f3838105ec0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.280+0000 7f383ebff700 1 --2- 172.21.15.101:0/3080328769 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38380ffae0 0x7f38380fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.281+0000 7f383ebff700 1 -- 172.21.15.101:0/3080328769 >> 172.21.15.101:0/3080328769 conn(0x7f38380fb5e0 msgr2=0x7f38380fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:03.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.281+0000 7f383ebff700 1 -- 172.21.15.101:0/3080328769 shutdown_connections 2023-07-18T03:09:03.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.281+0000 7f383ebff700 1 -- 172.21.15.101:0/3080328769 wait complete. 2023-07-18T03:09:03.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.282+0000 7f383ebff700 1 Processor -- start 2023-07-18T03:09:03.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.282+0000 7f383ebff700 1 -- start start 2023-07-18T03:09:03.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.282+0000 7f383ebff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f38380ffae0 0x7f3838072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:03.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.282+0000 7f383ebff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3838100490 0x7f3838072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:03.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.282+0000 7f383ebff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3838105af0 0x7f38381a3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:03.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.282+0000 7f383ebff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3838073580 con 0x7f38380ffae0 2023-07-18T03:09:03.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.282+0000 7f383ebff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38381a4350 con 0x7f3838100490 2023-07-18T03:09:03.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.282+0000 7f383ebff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38381a4490 con 0x7f3838105af0 2023-07-18T03:09:03.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.282+0000 7f383c99b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f38380ffae0 0x7f3838072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:03.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.282+0000 7f383c99b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f38380ffae0 0x7f3838072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:35338/0 (socket says 172.21.15.101:35338) 2023-07-18T03:09:03.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.282+0000 7f383c99b700 1 -- 172.21.15.101:0/2410800876 learned_addr learned my addr 172.21.15.101:0/2410800876 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:03.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.282+0000 7f3837fff700 1 --2- 172.21.15.101:0/2410800876 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3838100490 0x7f3838072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:03.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.282+0000 7f383d19c700 1 --2- 172.21.15.101:0/2410800876 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3838105af0 0x7f38381a3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:03.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.283+0000 7f383c99b700 1 -- 172.21.15.101:0/2410800876 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3838105af0 msgr2=0x7f38381a3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:03.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.283+0000 7f383c99b700 1 --2- 172.21.15.101:0/2410800876 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3838105af0 0x7f38381a3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.283+0000 7f383c99b700 1 -- 172.21.15.101:0/2410800876 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3838100490 msgr2=0x7f3838072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:03.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.283+0000 7f383c99b700 1 --2- 172.21.15.101:0/2410800876 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3838100490 0x7f3838072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.283+0000 7f383c99b700 1 -- 172.21.15.101:0/2410800876 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f382c0095f0 con 0x7f38380ffae0 2023-07-18T03:09:03.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.283+0000 7f383c99b700 1 --2- 172.21.15.101:0/2410800876 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f38380ffae0 0x7f3838072810 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f382c005910 tx=0x7f382c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:03.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.283+0000 7f3835ffb700 1 -- 172.21.15.101:0/2410800876 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c01d070 con 0x7f38380ffae0 2023-07-18T03:09:03.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.283+0000 7f3835ffb700 1 -- 172.21.15.101:0/2410800876 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f382c022470 con 0x7f38380ffae0 2023-07-18T03:09:03.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.283+0000 7f3835ffb700 1 -- 172.21.15.101:0/2410800876 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c017650 con 0x7f38380ffae0 2023-07-18T03:09:03.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.283+0000 7f383ebff700 1 -- 172.21.15.101:0/2410800876 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38381a4690 con 0x7f38380ffae0 2023-07-18T03:09:03.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.283+0000 7f383ebff700 1 -- 172.21.15.101:0/2410800876 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38381a4a60 con 0x7f38380ffae0 2023-07-18T03:09:03.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.288+0000 7f3835ffb700 1 -- 172.21.15.101:0/2410800876 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 24) v1 ==== 135949+0+0 (secure 0 0 0) 0x7f382c00fad0 con 0x7f38380ffae0 2023-07-18T03:09:03.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.289+0000 7f3852460b80 1 Processor -- start 2023-07-18T03:09:03.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.289+0000 7f3852460b80 1 -- start start 2023-07-18T03:09:03.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.289+0000 7f3852460b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561291ed42d0 0x561291ed46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:03.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.289+0000 7f3852460b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561291ed4be0 0x561291edf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:03.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.289+0000 7f3852460b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561291edfad0 0x561291ee1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:03.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.289+0000 7f3852460b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561291ddacb0 con 0x561291ed4be0 2023-07-18T03:09:03.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.289+0000 7f3852460b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561291e2da40 con 0x561291ed42d0 2023-07-18T03:09:03.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.289+0000 7f3852460b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x561291e26ed0 con 0x561291edfad0 2023-07-18T03:09:03.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.289+0000 7f383c99b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561291ed4be0 0x561291edf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:03.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.290+0000 7f3837fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561291edfad0 0x561291ee1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:03.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.290+0000 7f383c99b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561291ed4be0 0x561291edf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:35354/0 (socket says 172.21.15.101:35354) 2023-07-18T03:09:03.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.290+0000 7f383c99b700 1 -- 172.21.15.101:0/2461412359 learned_addr learned my addr 172.21.15.101:0/2461412359 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:03.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.290+0000 7f383d19c700 1 --2- 172.21.15.101:0/2461412359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561291ed42d0 0x561291ed46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:03.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.290+0000 7f3835ffb700 1 --2- 172.21.15.101:0/2410800876 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f38280a13e0 0x7f38280a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:03.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.290+0000 7f383c99b700 1 -- 172.21.15.101:0/2461412359 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561291edfad0 msgr2=0x561291ee1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:03.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.290+0000 7f383c99b700 1 --2- 172.21.15.101:0/2461412359 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561291edfad0 0x561291ee1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.290+0000 7f383c99b700 1 -- 172.21.15.101:0/2461412359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561291ed42d0 msgr2=0x561291ed46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:03.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.290+0000 7f383c99b700 1 --2- 172.21.15.101:0/2461412359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561291ed42d0 0x561291ed46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.290+0000 7f383c99b700 1 -- 172.21.15.101:0/2461412359 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561291c54de0 con 0x561291ed4be0 2023-07-18T03:09:03.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.290+0000 7f3837fff700 1 --2- 172.21.15.101:0/2461412359 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561291edfad0 0x561291ee1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:09:03.284 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.290+0000 7f3835ffb700 1 -- 172.21.15.101:0/2410800876 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f382c0ce770 con 0x7f38380ffae0 2023-07-18T03:09:03.284 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.290+0000 7f383d19c700 1 --2- 172.21.15.101:0/2410800876 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f38280a13e0 0x7f38280a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:03.284 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.290+0000 7f383c99b700 1 --2- 172.21.15.101:0/2461412359 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561291ed4be0 0x561291edf590 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f382c0cbb00 tx=0x7f382c0668a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76772089b417c7a7 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:03.284 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.291+0000 7f3822ffd700 1 -- 172.21.15.101:0/2461412359 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c09b100 con 0x561291ed4be0 2023-07-18T03:09:03.284 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.291+0000 7f3822ffd700 1 -- 172.21.15.101:0/2461412359 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f382c09ae70 con 0x561291ed4be0 2023-07-18T03:09:03.284 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.291+0000 7f3822ffd700 1 -- 172.21.15.101:0/2461412359 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c09ac90 con 0x561291ed4be0 2023-07-18T03:09:03.284 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.291+0000 7f383d19c700 1 --2- 172.21.15.101:0/2410800876 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f38280a13e0 0x7f38280a3890 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3830009790 tx=0x7f3830004ad0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:03.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.291+0000 7f3852460b80 1 -- 172.21.15.101:0/2461412359 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561291ed4be0 msgr2=0x561291edf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:03.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.291+0000 7f3852460b80 1 --2- 172.21.15.101:0/2461412359 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561291ed4be0 0x561291edf590 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f382c0cbb00 tx=0x7f382c0668a0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.291+0000 7f3852460b80 1 -- 172.21.15.101:0/2461412359 shutdown_connections 2023-07-18T03:09:03.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.291+0000 7f3852460b80 1 --2- 172.21.15.101:0/2461412359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561291ed42d0 0x561291ed46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.291+0000 7f3852460b80 1 --2- 172.21.15.101:0/2461412359 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561291edfad0 0x561291ee1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.291+0000 7f3852460b80 1 --2- 172.21.15.101:0/2461412359 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561291ed4be0 0x561291edf590 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.291+0000 7f3852460b80 1 -- 172.21.15.101:0/2461412359 >> 172.21.15.101:0/2461412359 conn(0x561291ecada0 msgr2=0x561291ed06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:03.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.292+0000 7f3852460b80 1 -- 172.21.15.101:0/2461412359 shutdown_connections 2023-07-18T03:09:03.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.292+0000 7f3852460b80 1 -- 172.21.15.101:0/2461412359 wait complete. 2023-07-18T03:09:03.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.292+0000 7f3852460b80 1 -- start start 2023-07-18T03:09:03.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.292+0000 7f3852460b80 1 Processor -- start 2023-07-18T03:09:03.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.293+0000 7f3852460b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561291ed42d0 0x561291ee4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:03.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.293+0000 7f3852460b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561291ed4be0 0x561291ee5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:03.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.293+0000 7f3852460b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561291edfad0 0x561291ede9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:03.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.293+0000 7f3852460b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561291dc6ac0 con 0x561291ed42d0 2023-07-18T03:09:03.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.293+0000 7f3852460b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561291dc67c0 con 0x561291ed4be0 2023-07-18T03:09:03.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.293+0000 7f3852460b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x561291caa920 con 0x561291edfad0 2023-07-18T03:09:03.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.293+0000 7f3852460b80 10 client.0 _ll_register_callbacks cb 0x561291dc4760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:09:03.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.293+0000 7f383c99b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561291ed42d0 0x561291ee4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:03.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.293+0000 7f383c99b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561291ed42d0 0x561291ee4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:35368/0 (socket says 172.21.15.101:35368) 2023-07-18T03:09:03.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.293+0000 7f383c99b700 1 -- 172.21.15.101:0/618034067 learned_addr learned my addr 172.21.15.101:0/618034067 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:03.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.293+0000 7f3837fff700 1 --2- 172.21.15.101:0/618034067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561291ed4be0 0x561291ee5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:03.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.293+0000 7f383d19c700 1 --2- 172.21.15.101:0/618034067 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561291edfad0 0x561291ede9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:03.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.293+0000 7f383c99b700 1 -- 172.21.15.101:0/618034067 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561291edfad0 msgr2=0x561291ede9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:03.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.293+0000 7f383c99b700 1 --2- 172.21.15.101:0/618034067 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561291edfad0 0x561291ede9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.293+0000 7f383c99b700 1 -- 172.21.15.101:0/618034067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561291ed4be0 msgr2=0x561291ee5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:03.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.293+0000 7f383c99b700 1 --2- 172.21.15.101:0/618034067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561291ed4be0 0x561291ee5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.293+0000 7f383c99b700 1 -- 172.21.15.101:0/618034067 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38240098e0 con 0x561291ed42d0 2023-07-18T03:09:03.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.294+0000 7f383c99b700 1 --2- 172.21.15.101:0/618034067 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561291ed42d0 0x561291ee4b20 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f382c0cc170 tx=0x7f382c0cea30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba0479ef0fb35815 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:03.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.294+0000 7f3817fff700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:09:03.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.294+0000 7f3817fff700 1 -- 172.21.15.101:0/618034067 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c0d4260 con 0x561291ed42d0 2023-07-18T03:09:03.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.294+0000 7f3817fff700 1 -- 172.21.15.101:0/618034067 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f382c09b330 con 0x561291ed42d0 2023-07-18T03:09:03.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.294+0000 7f3852460b80 1 -- 172.21.15.101:0/618034067 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561291d804f0 con 0x561291ed42d0 2023-07-18T03:09:03.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.294+0000 7f3817fff700 1 -- 172.21.15.101:0/618034067 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f382c09ac90 con 0x561291ed42d0 2023-07-18T03:09:03.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.294+0000 7f3852460b80 10 client.15858 fetch_fsmap learned FSMap version 25 2023-07-18T03:09:03.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.294+0000 7f3852460b80 1 -- 172.21.15.101:0/618034067 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f38381a4a60 con 0x561291ed42d0 2023-07-18T03:09:03.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.294+0000 7f3817fff700 1 -- 172.21.15.101:0/618034067 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561291d804f0 con 0x561291ed42d0 2023-07-18T03:09:03.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.294+0000 7f3817fff700 1 -- 172.21.15.101:0/618034067 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f382c0971a0 con 0x561291ed42d0 2023-07-18T03:09:03.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.294+0000 7f3852460b80 10 client.15858 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:09:03.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.294+0000 7f3852460b80 10 client.15858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:09:03.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.294+0000 7f3852460b80 20 client.15858 populate_metadata read hostname 'smithi101' 2023-07-18T03:09:03.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.294+0000 7f3852460b80 1 --2- 172.21.15.101:0/618034067 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x561291eeebd0 0x561291ef0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:03.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.294+0000 7f3852460b80 4 client.15858 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:09:03.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.294+0000 7f3852460b80 1 -- 172.21.15.101:0/618034067 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561291e1feb0 con 0x561291eeebd0 2023-07-18T03:09:03.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.295+0000 7f3837fff700 1 --2- 172.21.15.101:0/618034067 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x561291eeebd0 0x561291ef0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:03.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.295+0000 7f3837fff700 1 --2- 172.21.15.101:0/618034067 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x561291eeebd0 0x561291ef0fb0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73b5497c3703b26c server_cookie=c950e318959a8d92 in_seq=0 out_seq=0 2023-07-18T03:09:03.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.295+0000 7f3817fff700 10 client.15858 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:09:03.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.296+0000 7f3817fff700 1 -- 172.21.15.101:0/618034067 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561291e1feb0 con 0x561291eeebd0 2023-07-18T03:09:03.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.296+0000 7f3817fff700 10 client.15858 handle_command_reply: tid=0 2023-07-18T03:09:03.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.296+0000 7f3852460b80 1 client.15858 shutdown 2023-07-18T03:09:03.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.296+0000 7f3852460b80 1 -- 172.21.15.101:0/618034067 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561291ed42d0 msgr2=0x561291ee4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:03.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.296+0000 7f3852460b80 1 --2- 172.21.15.101:0/618034067 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561291ed42d0 0x561291ee4b20 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f382c0cc170 tx=0x7f382c0cea30 comp rx=0 tx=0).stop 2023-07-18T03:09:03.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.296+0000 7f3852460b80 1 -- 172.21.15.101:0/618034067 shutdown_connections 2023-07-18T03:09:03.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.296+0000 7f3852460b80 1 --2- 172.21.15.101:0/618034067 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x561291eeebd0 0x561291ef0fb0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.296+0000 7f3852460b80 1 --2- 172.21.15.101:0/618034067 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561291edfad0 0x561291ede9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.296+0000 7f3852460b80 1 --2- 172.21.15.101:0/618034067 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561291ed42d0 0x561291ee4b20 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.296+0000 7f3817fff700 0 client.15858 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:09:03.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.296+0000 7f3852460b80 1 --2- 172.21.15.101:0/618034067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561291ed4be0 0x561291ee5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.296+0000 7f3852460b80 1 -- 172.21.15.101:0/618034067 >> 172.21.15.101:0/618034067 conn(0x561291ecada0 msgr2=0x561291ed0230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:03.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.296+0000 7f3852460b80 1 -- 172.21.15.101:0/618034067 shutdown_connections 2023-07-18T03:09:03.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.296+0000 7f3852460b80 1 -- 172.21.15.101:0/618034067 wait complete. 2023-07-18T03:09:03.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.297+0000 7f3852460b80 20 client.15858 trim_cache size 0 max 16384 2023-07-18T03:09:03.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.310+0000 7f3852460b80 1 Processor -- start 2023-07-18T03:09:03.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.310+0000 7f3852460b80 1 -- start start 2023-07-18T03:09:03.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.310+0000 7f3852460b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561291ed42d0 0x561291ede980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:03.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.310+0000 7f3852460b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561291ed4be0 0x561291ee46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:03.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.310+0000 7f3852460b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561291edfad0 0x561291ee4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:03.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.310+0000 7f3852460b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561291ed5de0 con 0x561291ed42d0 2023-07-18T03:09:03.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.310+0000 7f3852460b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561291ed14e0 con 0x561291edfad0 2023-07-18T03:09:03.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.310+0000 7f3852460b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x561291ed61a0 con 0x561291ed4be0 2023-07-18T03:09:03.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.311+0000 7f383c99b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561291ed42d0 0x561291ede980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:03.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.311+0000 7f3837fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561291ed4be0 0x561291ee46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:03.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.311+0000 7f383c99b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561291ed42d0 0x561291ede980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:35370/0 (socket says 172.21.15.101:35370) 2023-07-18T03:09:03.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.311+0000 7f383c99b700 1 -- 172.21.15.101:0/3890877686 learned_addr learned my addr 172.21.15.101:0/3890877686 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:03.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.311+0000 7f383d19c700 1 --2- 172.21.15.101:0/3890877686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561291edfad0 0x561291ee4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:03.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.311+0000 7f383c99b700 1 -- 172.21.15.101:0/3890877686 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561291ed4be0 msgr2=0x561291ee46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:03.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.311+0000 7f383c99b700 1 --2- 172.21.15.101:0/3890877686 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561291ed4be0 0x561291ee46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.311+0000 7f383c99b700 1 -- 172.21.15.101:0/3890877686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561291edfad0 msgr2=0x561291ee4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:03.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.311+0000 7f383c99b700 1 --2- 172.21.15.101:0/3890877686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561291edfad0 0x561291ee4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.311+0000 7f383c99b700 1 -- 172.21.15.101:0/3890877686 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561291c3dc60 con 0x561291ed42d0 2023-07-18T03:09:03.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.311+0000 7f3837fff700 1 --2- 172.21.15.101:0/3890877686 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561291ed4be0 0x561291ee46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:09:03.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.311+0000 7f383d19c700 1 --2- 172.21.15.101:0/3890877686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561291edfad0 0x561291ee4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:09:03.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.311+0000 7f383c99b700 1 --2- 172.21.15.101:0/3890877686 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561291ed42d0 0x561291ede980 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f382c09afe0 tx=0x7f382c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc524a5a8c15e60 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:03.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.311+0000 7f38237fe700 1 -- 172.21.15.101:0/3890877686 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c0ae070 con 0x561291ed42d0 2023-07-18T03:09:03.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.311+0000 7f38237fe700 1 -- 172.21.15.101:0/3890877686 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f382c00fd80 con 0x561291ed42d0 2023-07-18T03:09:03.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.311+0000 7f38237fe700 1 -- 172.21.15.101:0/3890877686 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c09ac90 con 0x561291ed42d0 2023-07-18T03:09:03.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.312+0000 7f3852460b80 1 -- 172.21.15.101:0/3890877686 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561291ed42d0 msgr2=0x561291ede980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:03.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.312+0000 7f3852460b80 1 --2- 172.21.15.101:0/3890877686 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561291ed42d0 0x561291ede980 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f382c09afe0 tx=0x7f382c009310 comp rx=0 tx=0).stop 2023-07-18T03:09:03.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.312+0000 7f3852460b80 1 -- 172.21.15.101:0/3890877686 shutdown_connections 2023-07-18T03:09:03.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.312+0000 7f3852460b80 1 --2- 172.21.15.101:0/3890877686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561291edfad0 0x561291ee4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.312+0000 7f3852460b80 1 --2- 172.21.15.101:0/3890877686 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561291ed42d0 0x561291ede980 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.312+0000 7f3852460b80 1 --2- 172.21.15.101:0/3890877686 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561291ed4be0 0x561291ee46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.312+0000 7f3852460b80 1 -- 172.21.15.101:0/3890877686 >> 172.21.15.101:0/3890877686 conn(0x561291ecada0 msgr2=0x561291ede270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:03.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.312+0000 7f3852460b80 1 -- 172.21.15.101:0/3890877686 shutdown_connections 2023-07-18T03:09:03.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.312+0000 7f3852460b80 1 -- 172.21.15.101:0/3890877686 wait complete. 2023-07-18T03:09:03.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.312+0000 7f3852460b80 1 -- start start 2023-07-18T03:09:03.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.313+0000 7f3852460b80 1 Processor -- start 2023-07-18T03:09:03.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.313+0000 7f3852460b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561291ed42d0 0x561291f36be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:03.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.313+0000 7f3852460b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561291ed4be0 0x561291f37120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:03.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.313+0000 7f3852460b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561291edfad0 0x561291f3bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:03.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.313+0000 7f3852460b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561291f3c0d0 con 0x561291ed42d0 2023-07-18T03:09:03.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.313+0000 7f3852460b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561291f3c210 con 0x561291ed4be0 2023-07-18T03:09:03.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.313+0000 7f3852460b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x561291f3c3d0 con 0x561291edfad0 2023-07-18T03:09:03.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.313+0000 7f3852460b80 10 client.0 _ll_register_callbacks cb 0x7f3830009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:09:03.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.313+0000 7f383c99b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561291ed42d0 0x561291f36be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:03.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.313+0000 7f383c99b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561291ed42d0 0x561291f36be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:35380/0 (socket says 172.21.15.101:35380) 2023-07-18T03:09:03.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.313+0000 7f383c99b700 1 -- 172.21.15.101:0/2137695638 learned_addr learned my addr 172.21.15.101:0/2137695638 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:03.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.314+0000 7f383d19c700 1 --2- 172.21.15.101:0/2137695638 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561291edfad0 0x561291f3bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:03.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.314+0000 7f3837fff700 1 --2- 172.21.15.101:0/2137695638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561291ed4be0 0x561291f37120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:03.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.314+0000 7f383c99b700 1 -- 172.21.15.101:0/2137695638 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561291edfad0 msgr2=0x561291f3bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:03.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.314+0000 7f383c99b700 1 --2- 172.21.15.101:0/2137695638 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561291edfad0 0x561291f3bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.314+0000 7f383c99b700 1 -- 172.21.15.101:0/2137695638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561291ed4be0 msgr2=0x561291f37120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:03.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.314+0000 7f383c99b700 1 --2- 172.21.15.101:0/2137695638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561291ed4be0 0x561291f37120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.314+0000 7f383c99b700 1 -- 172.21.15.101:0/2137695638 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3824009d80 con 0x561291ed42d0 2023-07-18T03:09:03.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.314+0000 7f3837fff700 1 --2- 172.21.15.101:0/2137695638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561291ed4be0 0x561291f37120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:09:03.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.314+0000 7f383c99b700 1 --2- 172.21.15.101:0/2137695638 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561291ed42d0 0x561291f36be0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f382c000c00 tx=0x7f382c009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8644c8f802e49e6 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:03.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.314+0000 7f3817fff700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:09:03.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.314+0000 7f3817fff700 1 -- 172.21.15.101:0/2137695638 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c0ae040 con 0x561291ed42d0 2023-07-18T03:09:03.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.314+0000 7f3852460b80 1 -- 172.21.15.101:0/2137695638 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f382c09ae70 con 0x561291ed42d0 2023-07-18T03:09:03.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.314+0000 7f3817fff700 1 -- 172.21.15.101:0/2137695638 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f382c09b330 con 0x561291ed42d0 2023-07-18T03:09:03.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.314+0000 7f3817fff700 1 -- 172.21.15.101:0/2137695638 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f382c09ac90 con 0x561291ed42d0 2023-07-18T03:09:03.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.315+0000 7f3852460b80 10 client.15864 fetch_fsmap learned FSMap version 25 2023-07-18T03:09:03.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.315+0000 7f3852460b80 1 -- 172.21.15.101:0/2137695638 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f38381a4a60 con 0x561291ed42d0 2023-07-18T03:09:03.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.315+0000 7f3817fff700 1 -- 172.21.15.101:0/2137695638 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f382c09ae70 con 0x561291ed42d0 2023-07-18T03:09:03.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.315+0000 7f3817fff700 1 -- 172.21.15.101:0/2137695638 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f382c097120 con 0x561291ed42d0 2023-07-18T03:09:03.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.315+0000 7f3852460b80 10 client.15864 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:09:03.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.315+0000 7f3852460b80 10 client.15864 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:09:03.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.315+0000 7f3852460b80 20 client.15864 populate_metadata read hostname 'smithi101' 2023-07-18T03:09:03.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.315+0000 7f3852460b80 1 --2- 172.21.15.101:0/2137695638 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x561291f43660 0x561291f45a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:03.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.315+0000 7f3852460b80 4 client.15864 mds_command: new command op to 24562 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:09:03.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.315+0000 7f3852460b80 1 -- 172.21.15.101:0/2137695638 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561291de2050 con 0x561291f43660 2023-07-18T03:09:03.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.315+0000 7f3837fff700 1 --2- 172.21.15.101:0/2137695638 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x561291f43660 0x561291f45a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:03.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.316+0000 7f3817fff700 10 client.15864 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:09:03.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.316+0000 7f3837fff700 1 --2- 172.21.15.101:0/2137695638 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x561291f43660 0x561291f45a40 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc876b8b4b304766 server_cookie=805c61d360c58f33 in_seq=0 out_seq=0 2023-07-18T03:09:03.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.316+0000 7f3817fff700 1 -- 172.21.15.101:0/2137695638 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561291de2050 con 0x561291f43660 2023-07-18T03:09:03.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.316+0000 7f3817fff700 10 client.15864 handle_command_reply: tid=0 2023-07-18T03:09:03.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.316+0000 7f3852460b80 1 client.15864 shutdown 2023-07-18T03:09:03.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.316+0000 7f3852460b80 1 -- 172.21.15.101:0/2137695638 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561291ed42d0 msgr2=0x561291f36be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:03.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.316+0000 7f3852460b80 1 --2- 172.21.15.101:0/2137695638 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561291ed42d0 0x561291f36be0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f382c000c00 tx=0x7f382c009380 comp rx=0 tx=0).stop 2023-07-18T03:09:03.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.316+0000 7f3852460b80 1 -- 172.21.15.101:0/2137695638 shutdown_connections 2023-07-18T03:09:03.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.316+0000 7f3852460b80 1 --2- 172.21.15.101:0/2137695638 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x561291f43660 0x561291f45a40 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.316+0000 7f3852460b80 1 --2- 172.21.15.101:0/2137695638 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561291edfad0 0x561291f3bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.316+0000 7f3852460b80 1 --2- 172.21.15.101:0/2137695638 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561291ed42d0 0x561291f36be0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.316+0000 7f3817fff700 0 client.15864 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:09:03.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.316+0000 7f3852460b80 1 --2- 172.21.15.101:0/2137695638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561291ed4be0 0x561291f37120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.316+0000 7f3852460b80 1 -- 172.21.15.101:0/2137695638 >> 172.21.15.101:0/2137695638 conn(0x561291ecada0 msgr2=0x561291ed6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:03.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.316+0000 7f3852460b80 1 -- 172.21.15.101:0/2137695638 shutdown_connections 2023-07-18T03:09:03.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.316+0000 7f3852460b80 1 -- 172.21.15.101:0/2137695638 wait complete. 2023-07-18T03:09:03.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.317+0000 7f3852460b80 20 client.15864 trim_cache size 0 max 16384 2023-07-18T03:09:03.315 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:09:03.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.317+0000 7f383ebff700 1 -- 172.21.15.101:0/2410800876 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f38280a13e0 msgr2=0x7f38280a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:03.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.317+0000 7f383ebff700 1 --2- 172.21.15.101:0/2410800876 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f38280a13e0 0x7f38280a3890 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3830009790 tx=0x7f3830004ad0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.317+0000 7f383ebff700 1 -- 172.21.15.101:0/2410800876 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f38380ffae0 msgr2=0x7f3838072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:03.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.317+0000 7f383ebff700 1 --2- 172.21.15.101:0/2410800876 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f38380ffae0 0x7f3838072810 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f382c005910 tx=0x7f382c005740 comp rx=0 tx=0).stop 2023-07-18T03:09:03.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.317+0000 7f383ebff700 1 -- 172.21.15.101:0/2410800876 shutdown_connections 2023-07-18T03:09:03.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.317+0000 7f383ebff700 1 --2- 172.21.15.101:0/2410800876 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3838105af0 0x7f38381a3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.317+0000 7f383ebff700 1 --2- 172.21.15.101:0/2410800876 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f38380ffae0 0x7f3838072810 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.317+0000 7f383ebff700 1 --2- 172.21.15.101:0/2410800876 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f38280a13e0 0x7f38280a3890 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.317+0000 7f383ebff700 1 --2- 172.21.15.101:0/2410800876 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3838100490 0x7f3838072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:03.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.317+0000 7f383ebff700 1 -- 172.21.15.101:0/2410800876 >> 172.21.15.101:0/2410800876 conn(0x7f38380fb5e0 msgr2=0x7f38381035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:03.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.317+0000 7f383ebff700 1 -- 172.21.15.101:0/2410800876 shutdown_connections 2023-07-18T03:09:03.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:03.317+0000 7f383ebff700 1 -- 172.21.15.101:0/2410800876 wait complete. 2023-07-18T03:09:03.322 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:03 smithi101 ceph-mon[126379]: pgmap v47: 129 pgs: 129 active+clean; 31 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 1.3 MiB/s wr, 163 op/s 2023-07-18T03:09:03.322 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:03 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/313120455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:09:03.446 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:03 smithi179 ceph-mon[129229]: pgmap v47: 129 pgs: 129 active+clean; 31 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 1.3 MiB/s wr, 163 op/s 2023-07-18T03:09:03.446 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:03 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/313120455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:09:03.541 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:03 smithi155 ceph-mon[128885]: pgmap v47: 129 pgs: 129 active+clean; 31 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 1.3 MiB/s wr, 163 op/s 2023-07-18T03:09:03.541 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:03 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/313120455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:09:03.561 INFO:tasks.workunit.client.0.smithi101.stdout: 1 65270 13.22 MB/sec warmup 28 sec latency 2371.401 ms 2023-07-18T03:09:04.324 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-18T03:09:04.325 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-18T03:09:04.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.539+0000 7fbf94dff700 1 -- 172.21.15.101:0/1235135454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf900728b0 msgr2=0x7fbf90072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:04.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.539+0000 7fbf94dff700 1 --2- 172.21.15.101:0/1235135454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf900728b0 0x7fbf90072c80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbf80009990 tx=0x7fbf80009ca0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.540+0000 7fbf94dff700 1 -- 172.21.15.101:0/1235135454 shutdown_connections 2023-07-18T03:09:04.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.540+0000 7fbf94dff700 1 --2- 172.21.15.101:0/1235135454 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbf9010c320 0x7fbf9010e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.540+0000 7fbf94dff700 1 --2- 172.21.15.101:0/1235135454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf900728b0 0x7fbf90072c80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.540+0000 7fbf94dff700 1 --2- 172.21.15.101:0/1235135454 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf900731c0 0x7fbf9010bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.540+0000 7fbf94dff700 1 -- 172.21.15.101:0/1235135454 >> 172.21.15.101:0/1235135454 conn(0x7fbf900fb5e0 msgr2=0x7fbf900fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:04.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.540+0000 7fbf94dff700 1 -- 172.21.15.101:0/1235135454 shutdown_connections 2023-07-18T03:09:04.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.540+0000 7fbf94dff700 1 -- 172.21.15.101:0/1235135454 wait complete. 2023-07-18T03:09:04.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.541+0000 7fbf94dff700 1 Processor -- start 2023-07-18T03:09:04.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.541+0000 7fbf94dff700 1 -- start start 2023-07-18T03:09:04.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.541+0000 7fbf94dff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf900728b0 0x7fbf90197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:04.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.541+0000 7fbf94dff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf900731c0 0x7fbf901983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:04.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.541+0000 7fbf94dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbf9010c320 0x7fbf9019c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:04.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.541+0000 7fbf94dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9019ce60 con 0x7fbf9010c320 2023-07-18T03:09:04.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.541+0000 7fbf94dff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9019cfd0 con 0x7fbf900728b0 2023-07-18T03:09:04.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.541+0000 7fbf94dff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9019d190 con 0x7fbf900731c0 2023-07-18T03:09:04.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.541+0000 7fbf8e59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf900728b0 0x7fbf90197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:04.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.541+0000 7fbf8ed9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbf9010c320 0x7fbf9019c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:04.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.541+0000 7fbf8dd9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf900731c0 0x7fbf901983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:04.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.541+0000 7fbf8e59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf900728b0 0x7fbf90197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:51356/0 (socket says 172.21.15.101:51356) 2023-07-18T03:09:04.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.541+0000 7fbf8ed9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbf9010c320 0x7fbf9019c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:35398/0 (socket says 172.21.15.101:35398) 2023-07-18T03:09:04.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.542+0000 7fbf8e59c700 1 -- 172.21.15.101:0/349971123 learned_addr learned my addr 172.21.15.101:0/349971123 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:04.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.542+0000 7fbf8dd9b700 1 -- 172.21.15.101:0/349971123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf900728b0 msgr2=0x7fbf90197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:04.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.542+0000 7fbf8dd9b700 1 --2- 172.21.15.101:0/349971123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf900728b0 0x7fbf90197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.542+0000 7fbf8dd9b700 1 -- 172.21.15.101:0/349971123 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbf9010c320 msgr2=0x7fbf9019c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:04.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.542+0000 7fbf8dd9b700 1 --2- 172.21.15.101:0/349971123 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbf9010c320 0x7fbf9019c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.542+0000 7fbf8dd9b700 1 -- 172.21.15.101:0/349971123 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf800095f0 con 0x7fbf900731c0 2023-07-18T03:09:04.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.542+0000 7fbf8e59c700 1 --2- 172.21.15.101:0/349971123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf900728b0 0x7fbf90197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:09:04.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.542+0000 7fbf8ed9d700 1 --2- 172.21.15.101:0/349971123 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbf9010c320 0x7fbf9019c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:09:04.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.542+0000 7fbf8dd9b700 1 --2- 172.21.15.101:0/349971123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf900731c0 0x7fbf901983c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbf8400e9f0 tx=0x7fbf8400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:04.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.542+0000 7fbf7f7fe700 1 -- 172.21.15.101:0/349971123 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8400c9c0 con 0x7fbf900731c0 2023-07-18T03:09:04.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.542+0000 7fbf7f7fe700 1 -- 172.21.15.101:0/349971123 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf84010430 con 0x7fbf900731c0 2023-07-18T03:09:04.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.542+0000 7fbf94dff700 1 -- 172.21.15.101:0/349971123 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf9019d470 con 0x7fbf900731c0 2023-07-18T03:09:04.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.542+0000 7fbf7f7fe700 1 -- 172.21.15.101:0/349971123 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf84018680 con 0x7fbf900731c0 2023-07-18T03:09:04.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.543+0000 7fbf94dff700 1 -- 172.21.15.101:0/349971123 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf901a4cb0 con 0x7fbf900731c0 2023-07-18T03:09:04.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.546+0000 7fbf7f7fe700 1 -- 172.21.15.101:0/349971123 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 24) v1 ==== 135949+0+0 (secure 0 0 0) 0x7fbf84010860 con 0x7fbf900731c0 2023-07-18T03:09:04.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.547+0000 7fbfa862bb80 1 Processor -- start 2023-07-18T03:09:04.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.547+0000 7fbfa862bb80 1 -- start start 2023-07-18T03:09:04.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.547+0000 7fbfa862bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a6f1e7350 0x562a6f1e7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:04.544 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.547+0000 7fbfa862bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a6f1e7c60 0x562a6f1f2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:04.544 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.547+0000 7fbfa862bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a6f1f2b50 0x562a6f1f4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:04.544 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.547+0000 7fbfa862bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6f0edd30 con 0x562a6f1f2b50 2023-07-18T03:09:04.544 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.547+0000 7fbfa862bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6f140ac0 con 0x562a6f1e7350 2023-07-18T03:09:04.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.547+0000 7fbfa862bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6f139f50 con 0x562a6f1e7c60 2023-07-18T03:09:04.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.547+0000 7fbf7f7fe700 1 --2- 172.21.15.101:0/349971123 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fbf780a13e0 0x7fbf780a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:04.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.547+0000 7fbf8ed9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a6f1e7350 0x562a6f1e7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:04.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.547+0000 7fbf8dd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a6f1f2b50 0x562a6f1f4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:04.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.548+0000 7fbf8ed9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a6f1e7350 0x562a6f1e7720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:51372/0 (socket says 172.21.15.101:51372) 2023-07-18T03:09:04.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.548+0000 7fbf8ed9d700 1 -- 172.21.15.101:0/4013817158 learned_addr learned my addr 172.21.15.101:0/4013817158 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:04.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.548+0000 7fbf8e59c700 1 --2- 172.21.15.101:0/4013817158 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a6f1e7c60 0x562a6f1f2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:04.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.548+0000 7fbf7f7fe700 1 -- 172.21.15.101:0/349971123 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fbf84014070 con 0x7fbf900731c0 2023-07-18T03:09:04.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.548+0000 7fbf8ed9d700 1 --2- 172.21.15.101:0/349971123 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fbf780a13e0 0x7fbf780a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:04.547 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.548+0000 7fbf8e59c700 1 -- 172.21.15.101:0/4013817158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a6f1e7350 msgr2=0x562a6f1e7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:04.547 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.548+0000 7fbf8e59c700 1 --2- 172.21.15.101:0/4013817158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a6f1e7350 0x562a6f1e7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.547 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.548+0000 7fbf8e59c700 1 -- 172.21.15.101:0/4013817158 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a6f1f2b50 msgr2=0x562a6f1f4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:04.547 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.548+0000 7fbf8e59c700 1 --2- 172.21.15.101:0/4013817158 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a6f1f2b50 0x562a6f1f4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.548+0000 7fbf8e59c700 1 -- 172.21.15.101:0/4013817158 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a6ef67e60 con 0x562a6f1e7c60 2023-07-18T03:09:04.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.548+0000 7fbf8e59c700 1 --2- 172.21.15.101:0/4013817158 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a6f1e7c60 0x562a6f1f2610 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbf80005910 tx=0x7fbf80005ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1505e276969670f8 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:04.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.548+0000 7fbf8ed9d700 1 --2- 172.21.15.101:0/349971123 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fbf780a13e0 0x7fbf780a3890 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbf88003fd0 tx=0x7fbf88004b40 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:04.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.549+0000 7fbf7cff9700 1 -- 172.21.15.101:0/4013817158 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8001d050 con 0x562a6f1e7c60 2023-07-18T03:09:04.549 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.549+0000 7fbf7cff9700 1 -- 172.21.15.101:0/4013817158 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf8001e040 con 0x562a6f1e7c60 2023-07-18T03:09:04.549 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.549+0000 7fbf7cff9700 1 -- 172.21.15.101:0/4013817158 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf800054f0 con 0x562a6f1e7c60 2023-07-18T03:09:04.549 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.549+0000 7fbfa862bb80 1 -- 172.21.15.101:0/4013817158 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a6f1e7c60 msgr2=0x562a6f1f2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:04.549 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.549+0000 7fbfa862bb80 1 --2- 172.21.15.101:0/4013817158 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a6f1e7c60 0x562a6f1f2610 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbf80005910 tx=0x7fbf80005ec0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.550 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.549+0000 7fbfa862bb80 1 -- 172.21.15.101:0/4013817158 shutdown_connections 2023-07-18T03:09:04.550 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.549+0000 7fbfa862bb80 1 --2- 172.21.15.101:0/4013817158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a6f1e7350 0x562a6f1e7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.550 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.549+0000 7fbfa862bb80 1 --2- 172.21.15.101:0/4013817158 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a6f1f2b50 0x562a6f1f4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.550 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.549+0000 7fbfa862bb80 1 --2- 172.21.15.101:0/4013817158 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a6f1e7c60 0x562a6f1f2610 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.550 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.549+0000 7fbfa862bb80 1 -- 172.21.15.101:0/4013817158 >> 172.21.15.101:0/4013817158 conn(0x562a6f1dde20 msgr2=0x562a6f1e3720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:04.550 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.549+0000 7fbfa862bb80 1 -- 172.21.15.101:0/4013817158 shutdown_connections 2023-07-18T03:09:04.550 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.549+0000 7fbfa862bb80 1 -- 172.21.15.101:0/4013817158 wait complete. 2023-07-18T03:09:04.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.550+0000 7fbfa862bb80 1 -- start start 2023-07-18T03:09:04.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.550+0000 7fbfa862bb80 1 Processor -- start 2023-07-18T03:09:04.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.550+0000 7fbfa862bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a6f1e7350 0x562a6f1f25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:04.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.551+0000 7fbfa862bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a6f1e7c60 0x562a6f1ef7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:04.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.551+0000 7fbfa862bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a6f1f2b50 0x562a6f1eff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:04.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.551+0000 7fbfa862bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6f0d9b40 con 0x562a6f1e7350 2023-07-18T03:09:04.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.551+0000 7fbfa862bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6f0d9840 con 0x562a6f1e7c60 2023-07-18T03:09:04.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.551+0000 7fbfa862bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6efbd9a0 con 0x562a6f1f2b50 2023-07-18T03:09:04.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.551+0000 7fbfa862bb80 10 client.0 _ll_register_callbacks cb 0x562a6f0d77e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:09:04.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.551+0000 7fbf8e59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a6f1e7350 0x562a6f1f25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:04.553 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.551+0000 7fbf8e59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a6f1e7350 0x562a6f1f25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:35408/0 (socket says 172.21.15.101:35408) 2023-07-18T03:09:04.553 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.551+0000 7fbf8e59c700 1 -- 172.21.15.101:0/1841896752 learned_addr learned my addr 172.21.15.101:0/1841896752 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:04.553 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.551+0000 7fbf8ed9d700 1 --2- 172.21.15.101:0/1841896752 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a6f1f2b50 0x562a6f1eff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:04.553 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.551+0000 7fbf8dd9b700 1 --2- 172.21.15.101:0/1841896752 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a6f1e7c60 0x562a6f1ef7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:04.553 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.551+0000 7fbf8e59c700 1 -- 172.21.15.101:0/1841896752 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a6f1f2b50 msgr2=0x562a6f1eff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:04.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.551+0000 7fbf8e59c700 1 --2- 172.21.15.101:0/1841896752 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a6f1f2b50 0x562a6f1eff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.551+0000 7fbf8e59c700 1 -- 172.21.15.101:0/1841896752 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a6f1e7c60 msgr2=0x562a6f1ef7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:04.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.551+0000 7fbf8e59c700 1 --2- 172.21.15.101:0/1841896752 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a6f1e7c60 0x562a6f1ef7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.551+0000 7fbf8e59c700 1 -- 172.21.15.101:0/1841896752 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf9010f3a0 con 0x562a6f1e7350 2023-07-18T03:09:04.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.551+0000 7fbf8e59c700 1 --2- 172.21.15.101:0/1841896752 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a6f1e7350 0x562a6f1f25e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fbf80009ce0 tx=0x7fbf800053f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d6acb3546fda98 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:04.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.551+0000 7fbf75ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:09:04.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.552+0000 7fbf75ffb700 1 -- 172.21.15.101:0/1841896752 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf800049e0 con 0x562a6f1e7350 2023-07-18T03:09:04.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.552+0000 7fbf75ffb700 1 -- 172.21.15.101:0/1841896752 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fbf80004b40 con 0x562a6f1e7350 2023-07-18T03:09:04.555 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.552+0000 7fbfa862bb80 1 -- 172.21.15.101:0/1841896752 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562a6f093570 con 0x562a6f1e7350 2023-07-18T03:09:04.555 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.552+0000 7fbf75ffb700 1 -- 172.21.15.101:0/1841896752 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf80004d00 con 0x562a6f1e7350 2023-07-18T03:09:04.555 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.552+0000 7fbfa862bb80 10 client.15876 fetch_fsmap learned FSMap version 25 2023-07-18T03:09:04.555 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.552+0000 7fbfa862bb80 1 -- 172.21.15.101:0/1841896752 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562a6ef50ce0 con 0x562a6f1e7350 2023-07-18T03:09:04.555 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.552+0000 7fbf75ffb700 1 -- 172.21.15.101:0/1841896752 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562a6f093570 con 0x562a6f1e7350 2023-07-18T03:09:04.555 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.552+0000 7fbf75ffb700 1 -- 172.21.15.101:0/1841896752 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fbf800234f0 con 0x562a6f1e7350 2023-07-18T03:09:04.555 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.553+0000 7fbfa862bb80 10 client.15876 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:09:04.556 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.553+0000 7fbfa862bb80 10 client.15876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:09:04.556 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.553+0000 7fbfa862bb80 20 client.15876 populate_metadata read hostname 'smithi101' 2023-07-18T03:09:04.556 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.553+0000 7fbfa862bb80 1 --2- 172.21.15.101:0/1841896752 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x562a6f202040 0x562a6f204420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:04.556 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.553+0000 7fbfa862bb80 4 client.15876 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:09:04.556 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.553+0000 7fbfa862bb80 1 -- 172.21.15.101:0/1841896752 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562a6f132f30 con 0x562a6f202040 2023-07-18T03:09:04.556 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.553+0000 7fbf8dd9b700 1 --2- 172.21.15.101:0/1841896752 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x562a6f202040 0x562a6f204420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:04.557 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.553+0000 7fbf75ffb700 10 client.15876 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:09:04.557 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.553+0000 7fbf8dd9b700 1 --2- 172.21.15.101:0/1841896752 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x562a6f202040 0x562a6f204420 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bcf1fece736d5de server_cookie=215e079cc313ee43 in_seq=0 out_seq=0 2023-07-18T03:09:04.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.554+0000 7fbf75ffb700 1 -- 172.21.15.101:0/1841896752 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562a6f132f30 con 0x562a6f202040 2023-07-18T03:09:04.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.554+0000 7fbf75ffb700 10 client.15876 handle_command_reply: tid=0 2023-07-18T03:09:04.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.554+0000 7fbfa862bb80 1 client.15876 shutdown 2023-07-18T03:09:04.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.554+0000 7fbfa862bb80 1 -- 172.21.15.101:0/1841896752 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a6f1e7350 msgr2=0x562a6f1f25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:04.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.554+0000 7fbfa862bb80 1 --2- 172.21.15.101:0/1841896752 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a6f1e7350 0x562a6f1f25e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fbf80009ce0 tx=0x7fbf800053f0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.554+0000 7fbfa862bb80 1 -- 172.21.15.101:0/1841896752 shutdown_connections 2023-07-18T03:09:04.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.554+0000 7fbfa862bb80 1 --2- 172.21.15.101:0/1841896752 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x562a6f202040 0x562a6f204420 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.554+0000 7fbfa862bb80 1 --2- 172.21.15.101:0/1841896752 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a6f1f2b50 0x562a6f1eff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.554+0000 7fbfa862bb80 1 --2- 172.21.15.101:0/1841896752 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a6f1e7350 0x562a6f1f25e0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.554+0000 7fbf75ffb700 0 client.15876 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:09:04.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.554+0000 7fbfa862bb80 1 --2- 172.21.15.101:0/1841896752 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a6f1e7c60 0x562a6f1ef7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.554+0000 7fbfa862bb80 1 -- 172.21.15.101:0/1841896752 >> 172.21.15.101:0/1841896752 conn(0x562a6f1dde20 msgr2=0x562a6f1e3310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:04.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.554+0000 7fbfa862bb80 1 -- 172.21.15.101:0/1841896752 shutdown_connections 2023-07-18T03:09:04.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.555+0000 7fbfa862bb80 1 -- 172.21.15.101:0/1841896752 wait complete. 2023-07-18T03:09:04.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.555+0000 7fbfa862bb80 20 client.15876 trim_cache size 0 max 16384 2023-07-18T03:09:04.561 INFO:tasks.workunit.client.0.smithi101.stdout: 1 65270 12.77 MB/sec warmup 29 sec latency 3371.469 ms 2023-07-18T03:09:04.564 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.572+0000 7fbfa862bb80 1 Processor -- start 2023-07-18T03:09:04.564 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.572+0000 7fbfa862bb80 1 -- start start 2023-07-18T03:09:04.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.572+0000 7fbfa862bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a6f1e7350 0x562a6f1ef7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:04.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.572+0000 7fbfa862bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a6f1e7c60 0x562a6f1f2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:04.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.572+0000 7fbfa862bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a6f1f2b50 0x562a6f1ecaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:04.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.572+0000 7fbfa862bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6f1e85c0 con 0x562a6f1f2b50 2023-07-18T03:09:04.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.572+0000 7fbfa862bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6f1dd9f0 con 0x562a6f1e7350 2023-07-18T03:09:04.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.572+0000 7fbfa862bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6f1e9220 con 0x562a6f1e7c60 2023-07-18T03:09:04.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.572+0000 7fbf8ed9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a6f1f2b50 0x562a6f1ecaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:04.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.572+0000 7fbf8ed9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a6f1f2b50 0x562a6f1ecaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:35416/0 (socket says 172.21.15.101:35416) 2023-07-18T03:09:04.567 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.572+0000 7fbf8ed9d700 1 -- 172.21.15.101:0/501941163 learned_addr learned my addr 172.21.15.101:0/501941163 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:04.567 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.573+0000 7fbf8e59c700 1 --2- 172.21.15.101:0/501941163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a6f1e7350 0x562a6f1ef7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:04.567 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.573+0000 7fbf8dd9b700 1 --2- 172.21.15.101:0/501941163 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a6f1e7c60 0x562a6f1f2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:04.567 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.573+0000 7fbf8ed9d700 1 -- 172.21.15.101:0/501941163 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a6f1e7c60 msgr2=0x562a6f1f2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:04.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.573+0000 7fbf8ed9d700 1 --2- 172.21.15.101:0/501941163 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a6f1e7c60 0x562a6f1f2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.573+0000 7fbf8ed9d700 1 -- 172.21.15.101:0/501941163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a6f1e7350 msgr2=0x562a6f1ef7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:04.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.573+0000 7fbf8ed9d700 1 --2- 172.21.15.101:0/501941163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a6f1e7350 0x562a6f1ef7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.573+0000 7fbf8ed9d700 1 -- 172.21.15.101:0/501941163 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a6f1df660 con 0x562a6f1f2b50 2023-07-18T03:09:04.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.573+0000 7fbf8dd9b700 1 --2- 172.21.15.101:0/501941163 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a6f1e7c60 0x562a6f1f2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:09:04.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.573+0000 7fbf8ed9d700 1 --2- 172.21.15.101:0/501941163 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a6f1f2b50 0x562a6f1ecaa0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbf88009240 tx=0x7fbf88009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84622391fc94518a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:04.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.573+0000 7fbf7d7fa700 1 -- 172.21.15.101:0/501941163 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf88004db0 con 0x562a6f1f2b50 2023-07-18T03:09:04.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.573+0000 7fbf7d7fa700 1 -- 172.21.15.101:0/501941163 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf88023040 con 0x562a6f1f2b50 2023-07-18T03:09:04.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.573+0000 7fbf7d7fa700 1 -- 172.21.15.101:0/501941163 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8801b690 con 0x562a6f1f2b50 2023-07-18T03:09:04.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.574+0000 7fbfa862bb80 1 -- 172.21.15.101:0/501941163 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a6f1f2b50 msgr2=0x562a6f1ecaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:04.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.574+0000 7fbfa862bb80 1 --2- 172.21.15.101:0/501941163 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a6f1f2b50 0x562a6f1ecaa0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbf88009240 tx=0x7fbf88009920 comp rx=0 tx=0).stop 2023-07-18T03:09:04.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.574+0000 7fbfa862bb80 1 -- 172.21.15.101:0/501941163 shutdown_connections 2023-07-18T03:09:04.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.574+0000 7fbfa862bb80 1 --2- 172.21.15.101:0/501941163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a6f1e7350 0x562a6f1ef7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.574+0000 7fbfa862bb80 1 --2- 172.21.15.101:0/501941163 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a6f1f2b50 0x562a6f1ecaa0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.574+0000 7fbfa862bb80 1 --2- 172.21.15.101:0/501941163 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a6f1e7c60 0x562a6f1f2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.574+0000 7fbfa862bb80 1 -- 172.21.15.101:0/501941163 >> 172.21.15.101:0/501941163 conn(0x562a6f1dde20 msgr2=0x562a6f1ef090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:04.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.574+0000 7fbfa862bb80 1 -- 172.21.15.101:0/501941163 shutdown_connections 2023-07-18T03:09:04.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.574+0000 7fbfa862bb80 1 -- 172.21.15.101:0/501941163 wait complete. 2023-07-18T03:09:04.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.574+0000 7fbfa862bb80 1 -- start start 2023-07-18T03:09:04.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.575+0000 7fbfa862bb80 1 Processor -- start 2023-07-18T03:09:04.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.575+0000 7fbfa862bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a6f1e7350 0x562a6f1f82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:04.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.575+0000 7fbfa862bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a6f1e7c60 0x562a6f1f8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:04.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.575+0000 7fbfa862bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a6f1f2b50 0x562a6f255f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:04.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.575+0000 7fbfa862bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6f256440 con 0x562a6f1f2b50 2023-07-18T03:09:04.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.575+0000 7fbfa862bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6f256580 con 0x562a6f1e7350 2023-07-18T03:09:04.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.575+0000 7fbfa862bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6f2566c0 con 0x562a6f1e7c60 2023-07-18T03:09:04.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.575+0000 7fbfa862bb80 10 client.0 _ll_register_callbacks cb 0x7fbf84096380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:09:04.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.575+0000 7fbf8ed9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a6f1f2b50 0x562a6f255f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:04.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.575+0000 7fbf8ed9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a6f1f2b50 0x562a6f255f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:35420/0 (socket says 172.21.15.101:35420) 2023-07-18T03:09:04.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.575+0000 7fbf8ed9d700 1 -- 172.21.15.101:0/3608532843 learned_addr learned my addr 172.21.15.101:0/3608532843 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:04.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.575+0000 7fbf8e59c700 1 --2- 172.21.15.101:0/3608532843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a6f1e7350 0x562a6f1f82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:04.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.576+0000 7fbf8dd9b700 1 --2- 172.21.15.101:0/3608532843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a6f1e7c60 0x562a6f1f8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:04.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.576+0000 7fbf8ed9d700 1 -- 172.21.15.101:0/3608532843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a6f1e7c60 msgr2=0x562a6f1f8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:04.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.576+0000 7fbf8ed9d700 1 --2- 172.21.15.101:0/3608532843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a6f1e7c60 0x562a6f1f8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.576+0000 7fbf8ed9d700 1 -- 172.21.15.101:0/3608532843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a6f1e7350 msgr2=0x562a6f1f82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:04.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.576+0000 7fbf8ed9d700 1 --2- 172.21.15.101:0/3608532843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a6f1e7350 0x562a6f1f82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.581 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.576+0000 7fbf8ed9d700 1 -- 172.21.15.101:0/3608532843 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf880048e0 con 0x562a6f1f2b50 2023-07-18T03:09:04.581 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.576+0000 7fbf8dd9b700 1 --2- 172.21.15.101:0/3608532843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a6f1e7c60 0x562a6f1f8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:09:04.581 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.576+0000 7fbf8ed9d700 1 --2- 172.21.15.101:0/3608532843 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a6f1f2b50 0x562a6f255f00 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbf880041b0 tx=0x7fbf88007b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb76aa6e0986eefd server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:04.581 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.576+0000 7fbf75ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:09:04.581 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.576+0000 7fbf75ffb700 1 -- 172.21.15.101:0/3608532843 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8801b860 con 0x562a6f1f2b50 2023-07-18T03:09:04.582 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.576+0000 7fbf75ffb700 1 -- 172.21.15.101:0/3608532843 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fbf88007e40 con 0x562a6f1f2b50 2023-07-18T03:09:04.582 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.576+0000 7fbfa862bb80 1 -- 172.21.15.101:0/3608532843 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf8001e040 con 0x562a6f1f2b50 2023-07-18T03:09:04.582 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.576+0000 7fbf75ffb700 1 -- 172.21.15.101:0/3608532843 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf8801b9c0 con 0x562a6f1f2b50 2023-07-18T03:09:04.582 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.576+0000 7fbfa862bb80 10 client.15882 fetch_fsmap learned FSMap version 25 2023-07-18T03:09:04.582 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.577+0000 7fbfa862bb80 1 -- 172.21.15.101:0/3608532843 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562a6ef8e2d0 con 0x562a6f1f2b50 2023-07-18T03:09:04.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.577+0000 7fbf75ffb700 1 -- 172.21.15.101:0/3608532843 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf8001e040 con 0x562a6f1f2b50 2023-07-18T03:09:04.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.577+0000 7fbf75ffb700 1 -- 172.21.15.101:0/3608532843 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fbf8801ad60 con 0x562a6f1f2b50 2023-07-18T03:09:04.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.577+0000 7fbfa862bb80 10 client.15882 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:09:04.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.577+0000 7fbfa862bb80 10 client.15882 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:09:04.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.577+0000 7fbfa862bb80 20 client.15882 populate_metadata read hostname 'smithi101' 2023-07-18T03:09:04.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.577+0000 7fbfa862bb80 1 --2- 172.21.15.101:0/3608532843 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x562a6f257dc0 0x562a6f25a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:04.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.577+0000 7fbfa862bb80 4 client.15882 mds_command: new command op to 24562 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-18T03:09:04.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.577+0000 7fbfa862bb80 1 -- 172.21.15.101:0/3608532843 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x562a6f0f50d0 con 0x562a6f257dc0 2023-07-18T03:09:04.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.577+0000 7fbf8e59c700 1 --2- 172.21.15.101:0/3608532843 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x562a6f257dc0 0x562a6f25a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:04.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.578+0000 7fbf75ffb700 10 client.15882 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:09:04.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.578+0000 7fbf8e59c700 1 --2- 172.21.15.101:0/3608532843 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x562a6f257dc0 0x562a6f25a1d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfe7e41922c3cde9 server_cookie=d09f2daec2672812 in_seq=0 out_seq=0 2023-07-18T03:09:04.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.579+0000 7fbf75ffb700 1 -- 172.21.15.101:0/3608532843 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x562a6f0f50d0 con 0x562a6f257dc0 2023-07-18T03:09:04.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.579+0000 7fbf75ffb700 10 client.15882 handle_command_reply: tid=0 2023-07-18T03:09:04.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.579+0000 7fbfa862bb80 1 client.15882 shutdown 2023-07-18T03:09:04.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.579+0000 7fbfa862bb80 1 -- 172.21.15.101:0/3608532843 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a6f1f2b50 msgr2=0x562a6f255f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:04.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.579+0000 7fbfa862bb80 1 --2- 172.21.15.101:0/3608532843 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a6f1f2b50 0x562a6f255f00 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbf880041b0 tx=0x7fbf88007b60 comp rx=0 tx=0).stop 2023-07-18T03:09:04.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.579+0000 7fbfa862bb80 1 -- 172.21.15.101:0/3608532843 shutdown_connections 2023-07-18T03:09:04.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.579+0000 7fbfa862bb80 1 --2- 172.21.15.101:0/3608532843 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x562a6f257dc0 0x562a6f25a1d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.579+0000 7fbfa862bb80 1 --2- 172.21.15.101:0/3608532843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a6f1e7350 0x562a6f1f82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.579+0000 7fbfa862bb80 1 --2- 172.21.15.101:0/3608532843 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a6f1f2b50 0x562a6f255f00 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.579+0000 7fbfa862bb80 1 --2- 172.21.15.101:0/3608532843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a6f1e7c60 0x562a6f1f8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.579+0000 7fbfa862bb80 1 -- 172.21.15.101:0/3608532843 >> 172.21.15.101:0/3608532843 conn(0x562a6f1dde20 msgr2=0x562a6f1e9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:04.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.580+0000 7fbf75ffb700 0 client.15882 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:09:04.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.580+0000 7fbfa862bb80 1 -- 172.21.15.101:0/3608532843 shutdown_connections 2023-07-18T03:09:04.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.580+0000 7fbfa862bb80 1 -- 172.21.15.101:0/3608532843 wait complete. 2023-07-18T03:09:04.589 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.580+0000 7fbfa862bb80 20 client.15882 trim_cache size 0 max 16384 2023-07-18T03:09:04.589 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:09:04.589 INFO:teuthology.orchestra.run.smithi101.stdout: "return_code": 0, 2023-07-18T03:09:04.589 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_tag": "e1976b04-545c-4f4a-b690-4d59d9363daf", 2023-07-18T03:09:04.589 INFO:teuthology.orchestra.run.smithi101.stdout: "mode": "asynchronous" 2023-07-18T03:09:04.590 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:09:04.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.580+0000 7fbf94dff700 1 -- 172.21.15.101:0/349971123 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fbf780a13e0 msgr2=0x7fbf780a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:04.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.580+0000 7fbf94dff700 1 --2- 172.21.15.101:0/349971123 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fbf780a13e0 0x7fbf780a3890 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbf88003fd0 tx=0x7fbf88004b40 comp rx=0 tx=0).stop 2023-07-18T03:09:04.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.580+0000 7fbf94dff700 1 -- 172.21.15.101:0/349971123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf900731c0 msgr2=0x7fbf901983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:04.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.580+0000 7fbf94dff700 1 --2- 172.21.15.101:0/349971123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf900731c0 0x7fbf901983c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbf8400e9f0 tx=0x7fbf8400ed00 comp rx=0 tx=0).stop 2023-07-18T03:09:04.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.580+0000 7fbf94dff700 1 -- 172.21.15.101:0/349971123 shutdown_connections 2023-07-18T03:09:04.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.580+0000 7fbf94dff700 1 --2- 172.21.15.101:0/349971123 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fbf780a13e0 0x7fbf780a3890 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.580+0000 7fbf94dff700 1 --2- 172.21.15.101:0/349971123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf900728b0 0x7fbf90197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.580+0000 7fbf94dff700 1 --2- 172.21.15.101:0/349971123 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbf9010c320 0x7fbf9019c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.580+0000 7fbf94dff700 1 --2- 172.21.15.101:0/349971123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf900731c0 0x7fbf901983c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.580+0000 7fbf94dff700 1 -- 172.21.15.101:0/349971123 >> 172.21.15.101:0/349971123 conn(0x7fbf900fb5e0 msgr2=0x7fbf9010e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:04.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.581+0000 7fbf94dff700 1 -- 172.21.15.101:0/349971123 shutdown_connections 2023-07-18T03:09:04.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.581+0000 7fbf94dff700 1 -- 172.21.15.101:0/349971123 wait complete. 2023-07-18T03:09:04.593 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:09:04.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.802+0000 7f7d151ff700 1 -- 172.21.15.101:0/4001220650 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7d10105b10 msgr2=0x7f7d10105ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:04.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.802+0000 7f7d151ff700 1 --2- 172.21.15.101:0/4001220650 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7d10105b10 0x7f7d10105ee0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7d00009990 tx=0x7f7d00009ca0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.802+0000 7f7d151ff700 1 -- 172.21.15.101:0/4001220650 shutdown_connections 2023-07-18T03:09:04.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.802+0000 7f7d151ff700 1 --2- 172.21.15.101:0/4001220650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d101004b0 0x7f7d1010c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.802+0000 7f7d151ff700 1 --2- 172.21.15.101:0/4001220650 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7d10105b10 0x7f7d10105ee0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.793 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.802+0000 7f7d151ff700 1 --2- 172.21.15.101:0/4001220650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d100ffb00 0x7f7d100fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.793 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.802+0000 7f7d151ff700 1 -- 172.21.15.101:0/4001220650 >> 172.21.15.101:0/4001220650 conn(0x7f7d100fb600 msgr2=0x7f7d100fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:04.793 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.802+0000 7f7d151ff700 1 -- 172.21.15.101:0/4001220650 shutdown_connections 2023-07-18T03:09:04.793 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.802+0000 7f7d151ff700 1 -- 172.21.15.101:0/4001220650 wait complete. 2023-07-18T03:09:04.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.803+0000 7f7d151ff700 1 Processor -- start 2023-07-18T03:09:04.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.803+0000 7f7d151ff700 1 -- start start 2023-07-18T03:09:04.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.803+0000 7f7d151ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7d100ffb00 0x7f7d1019b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:04.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.803+0000 7f7d151ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d101004b0 0x7f7d1019bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:04.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.803+0000 7f7d151ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d10105b10 0x7f7d10195a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:04.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.803+0000 7f7d151ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d101960d0 con 0x7f7d100ffb00 2023-07-18T03:09:04.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.803+0000 7f7d151ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d10196240 con 0x7f7d10105b10 2023-07-18T03:09:04.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.803+0000 7f7d151ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d10196400 con 0x7f7d101004b0 2023-07-18T03:09:04.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.804+0000 7f7d0ed9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7d100ffb00 0x7f7d1019b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:04.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.804+0000 7f7d0ed9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7d100ffb00 0x7f7d1019b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:35430/0 (socket says 172.21.15.101:35430) 2023-07-18T03:09:04.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.804+0000 7f7d0ed9d700 1 -- 172.21.15.101:0/2849678862 learned_addr learned my addr 172.21.15.101:0/2849678862 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:04.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.804+0000 7f7d07fff700 1 --2- 172.21.15.101:0/2849678862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d101004b0 0x7f7d1019bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:04.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.804+0000 7f7d0f59e700 1 --2- 172.21.15.101:0/2849678862 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d10105b10 0x7f7d10195a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:04.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.804+0000 7f7d0ed9d700 1 -- 172.21.15.101:0/2849678862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d101004b0 msgr2=0x7f7d1019bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:04.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.804+0000 7f7d0ed9d700 1 --2- 172.21.15.101:0/2849678862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d101004b0 0x7f7d1019bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.804+0000 7f7d0ed9d700 1 -- 172.21.15.101:0/2849678862 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d10105b10 msgr2=0x7f7d10195a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:04.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.804+0000 7f7d0ed9d700 1 --2- 172.21.15.101:0/2849678862 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d10105b10 0x7f7d10195a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.804+0000 7f7d0ed9d700 1 -- 172.21.15.101:0/2849678862 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d000095f0 con 0x7f7d100ffb00 2023-07-18T03:09:04.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.804+0000 7f7d0f59e700 1 --2- 172.21.15.101:0/2849678862 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d10105b10 0x7f7d10195a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:09:04.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.804+0000 7f7d0ed9d700 1 --2- 172.21.15.101:0/2849678862 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7d100ffb00 0x7f7d1019b9f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7d00005910 tx=0x7f7d00005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:04.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.805+0000 7f7d0cd99700 1 -- 172.21.15.101:0/2849678862 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0001d070 con 0x7f7d100ffb00 2023-07-18T03:09:04.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.805+0000 7f7d0cd99700 1 -- 172.21.15.101:0/2849678862 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d00022470 con 0x7f7d100ffb00 2023-07-18T03:09:04.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.805+0000 7f7d0cd99700 1 -- 172.21.15.101:0/2849678862 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d00017650 con 0x7f7d100ffb00 2023-07-18T03:09:04.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.805+0000 7f7d151ff700 1 -- 172.21.15.101:0/2849678862 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d10196680 con 0x7f7d100ffb00 2023-07-18T03:09:04.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.805+0000 7f7d151ff700 1 -- 172.21.15.101:0/2849678862 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d101a29e0 con 0x7f7d100ffb00 2023-07-18T03:09:04.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.808+0000 7f7d0cd99700 1 -- 172.21.15.101:0/2849678862 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 24) v1 ==== 135949+0+0 (secure 0 0 0) 0x7f7d000225e0 con 0x7f7d100ffb00 2023-07-18T03:09:04.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.809+0000 7f7d28a49b80 1 Processor -- start 2023-07-18T03:09:04.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.809+0000 7f7d28a49b80 1 -- start start 2023-07-18T03:09:04.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.809+0000 7f7d0cd99700 1 --2- 172.21.15.101:0/2849678862 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f7cfc0a1460 0x7f7cfc0a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:04.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.810+0000 7f7d0cd99700 1 -- 172.21.15.101:0/2849678862 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f7d000ce6a0 con 0x7f7d100ffb00 2023-07-18T03:09:04.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.810+0000 7f7d28a49b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b7ca52d0 0x55d3b7ca56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:04.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.810+0000 7f7d0f59e700 1 --2- 172.21.15.101:0/2849678862 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f7cfc0a1460 0x7f7cfc0a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:04.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.810+0000 7f7d28a49b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b7ca5be0 0x55d3b7cb49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:04.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.810+0000 7f7d28a49b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3b7cb4ee0 0x55d3b7cb72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:04.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.810+0000 7f7d28a49b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b7babcb0 con 0x55d3b7cb4ee0 2023-07-18T03:09:04.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.810+0000 7f7d28a49b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b7bfea40 con 0x55d3b7ca52d0 2023-07-18T03:09:04.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.810+0000 7f7d28a49b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b7bf7ed0 con 0x55d3b7ca5be0 2023-07-18T03:09:04.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.810+0000 7f7d07fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b7ca5be0 0x55d3b7cb49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:04.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.810+0000 7f7d0f59e700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3b7cb4ee0 0x55d3b7cb72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:04.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.810+0000 7f7d0ed9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b7ca52d0 0x55d3b7ca56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:04.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.810+0000 7f7d07fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b7ca5be0 0x55d3b7cb49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:37028/0 (socket says 172.21.15.101:37028) 2023-07-18T03:09:04.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.810+0000 7f7d07fff700 1 -- 172.21.15.101:0/3519206694 learned_addr learned my addr 172.21.15.101:0/3519206694 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:04.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.810+0000 7f7d0ed9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b7ca52d0 0x55d3b7ca56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:51448/0 (socket says 172.21.15.101:51448) 2023-07-18T03:09:04.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.811+0000 7f7d07fff700 1 -- 172.21.15.101:0/3519206694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b7ca52d0 msgr2=0x55d3b7ca56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:04.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.811+0000 7f7d07fff700 1 --2- 172.21.15.101:0/3519206694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b7ca52d0 0x55d3b7ca56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.811+0000 7f7d07fff700 1 -- 172.21.15.101:0/3519206694 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3b7cb4ee0 msgr2=0x55d3b7cb72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:04.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.811+0000 7f7d07fff700 1 --2- 172.21.15.101:0/3519206694 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3b7cb4ee0 0x55d3b7cb72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.811+0000 7f7d07fff700 1 -- 172.21.15.101:0/3519206694 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3b7a25de0 con 0x55d3b7ca5be0 2023-07-18T03:09:04.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.811+0000 7f7d0ed9d700 1 --2- 172.21.15.101:0/3519206694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b7ca52d0 0x55d3b7ca56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:09:04.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.811+0000 7f7d0f59e700 1 --2- 172.21.15.101:0/3519206694 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3b7cb4ee0 0x55d3b7cb72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:09:04.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.811+0000 7f7d0f59e700 1 --2- 172.21.15.101:0/2849678862 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f7cfc0a1460 0x7f7cfc0a3910 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7d08009fd0 tx=0x7f7d08007b00 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:04.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.811+0000 7f7d07fff700 1 --2- 172.21.15.101:0/3519206694 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b7ca5be0 0x55d3b7cb49a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8009d20 tx=0x7f7cf8011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a0a5a5773399237 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:04.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.811+0000 7f7d057fa700 1 -- 172.21.15.101:0/3519206694 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8019630 con 0x55d3b7ca5be0 2023-07-18T03:09:04.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.811+0000 7f7d057fa700 1 -- 172.21.15.101:0/3519206694 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7cf8009e70 con 0x55d3b7ca5be0 2023-07-18T03:09:04.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.811+0000 7f7d057fa700 1 -- 172.21.15.101:0/3519206694 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf800ea60 con 0x55d3b7ca5be0 2023-07-18T03:09:04.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.812+0000 7f7d28a49b80 1 -- 172.21.15.101:0/3519206694 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b7ca5be0 msgr2=0x55d3b7cb49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:04.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.812+0000 7f7d28a49b80 1 --2- 172.21.15.101:0/3519206694 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b7ca5be0 0x55d3b7cb49a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8009d20 tx=0x7f7cf8011720 comp rx=0 tx=0).stop 2023-07-18T03:09:04.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.812+0000 7f7d28a49b80 1 -- 172.21.15.101:0/3519206694 shutdown_connections 2023-07-18T03:09:04.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.812+0000 7f7d28a49b80 1 --2- 172.21.15.101:0/3519206694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b7ca52d0 0x55d3b7ca56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.812+0000 7f7d28a49b80 1 --2- 172.21.15.101:0/3519206694 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3b7cb4ee0 0x55d3b7cb72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.812+0000 7f7d28a49b80 1 --2- 172.21.15.101:0/3519206694 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b7ca5be0 0x55d3b7cb49a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.812+0000 7f7d28a49b80 1 -- 172.21.15.101:0/3519206694 >> 172.21.15.101:0/3519206694 conn(0x55d3b7c9bda0 msgr2=0x55d3b7ca16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:04.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.812+0000 7f7d28a49b80 1 -- 172.21.15.101:0/3519206694 shutdown_connections 2023-07-18T03:09:04.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.812+0000 7f7d28a49b80 1 -- 172.21.15.101:0/3519206694 wait complete. 2023-07-18T03:09:04.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.813+0000 7f7d28a49b80 1 -- start start 2023-07-18T03:09:04.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.813+0000 7f7d28a49b80 1 Processor -- start 2023-07-18T03:09:04.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.813+0000 7f7d28a49b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b7ca52d0 0x55d3b7cb4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:04.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.813+0000 7f7d28a49b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b7ca5be0 0x55d3b7cb1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:04.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.813+0000 7f7d28a49b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3b7cb4ee0 0x55d3b7cb22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:04.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.813+0000 7f7d28a49b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b7b97ac0 con 0x55d3b7cb4ee0 2023-07-18T03:09:04.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.813+0000 7f7d28a49b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b7b977c0 con 0x55d3b7ca52d0 2023-07-18T03:09:04.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.813+0000 7f7d28a49b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b7a7b920 con 0x55d3b7ca5be0 2023-07-18T03:09:04.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.813+0000 7f7d28a49b80 10 client.0 _ll_register_callbacks cb 0x55d3b7b95760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:09:04.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.814+0000 7f7d07fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b7ca5be0 0x55d3b7cb1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:04.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.814+0000 7f7d0f59e700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3b7cb4ee0 0x55d3b7cb22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:04.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.814+0000 7f7d07fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b7ca5be0 0x55d3b7cb1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:37042/0 (socket says 172.21.15.101:37042) 2023-07-18T03:09:04.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.814+0000 7f7d07fff700 1 -- 172.21.15.101:0/3391154055 learned_addr learned my addr 172.21.15.101:0/3391154055 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:04.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.814+0000 7f7d0ed9d700 1 --2- 172.21.15.101:0/3391154055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b7ca52d0 0x55d3b7cb4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:04.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.814+0000 7f7d07fff700 1 -- 172.21.15.101:0/3391154055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b7ca52d0 msgr2=0x55d3b7cb4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:04.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.814+0000 7f7d07fff700 1 --2- 172.21.15.101:0/3391154055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b7ca52d0 0x55d3b7cb4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.814+0000 7f7d07fff700 1 -- 172.21.15.101:0/3391154055 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3b7cb4ee0 msgr2=0x55d3b7cb22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:04.813 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.814+0000 7f7d07fff700 1 --2- 172.21.15.101:0/3391154055 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3b7cb4ee0 0x55d3b7cb22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.813 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.814+0000 7f7d07fff700 1 -- 172.21.15.101:0/3391154055 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d101a29e0 con 0x55d3b7ca5be0 2023-07-18T03:09:04.813 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.814+0000 7f7d07fff700 1 --2- 172.21.15.101:0/3391154055 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b7ca5be0 0x55d3b7cb1ba0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7cf801c040 tx=0x7f7cf8005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c276ecc911ab5db server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:04.813 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.814+0000 7f7cf67fc700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:09:04.813 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.815+0000 7f7cf67fc700 1 -- 172.21.15.101:0/3391154055 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8004b60 con 0x55d3b7ca5be0 2023-07-18T03:09:04.814 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.815+0000 7f7d28a49b80 1 -- 172.21.15.101:0/3391154055 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d3b7b514f0 con 0x55d3b7ca5be0 2023-07-18T03:09:04.814 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.815+0000 7f7cf67fc700 1 -- 172.21.15.101:0/3391154055 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f7cf8004cc0 con 0x55d3b7ca5be0 2023-07-18T03:09:04.814 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.815+0000 7f7cf67fc700 1 -- 172.21.15.101:0/3391154055 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7cf8004e80 con 0x55d3b7ca5be0 2023-07-18T03:09:04.814 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.815+0000 7f7d28a49b80 10 client.25948 fetch_fsmap learned FSMap version 25 2023-07-18T03:09:04.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.815+0000 7f7d28a49b80 1 -- 172.21.15.101:0/3391154055 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f7d0801a470 con 0x55d3b7ca5be0 2023-07-18T03:09:04.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.815+0000 7f7cf67fc700 1 -- 172.21.15.101:0/3391154055 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d3b7b514f0 con 0x55d3b7ca5be0 2023-07-18T03:09:04.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.816+0000 7f7cf67fc700 1 -- 172.21.15.101:0/3391154055 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f7cf8029020 con 0x55d3b7ca5be0 2023-07-18T03:09:04.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.816+0000 7f7d28a49b80 10 client.25948 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:09:04.816 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.816+0000 7f7d28a49b80 10 client.25948 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:09:04.816 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.816+0000 7f7d28a49b80 20 client.25948 populate_metadata read hostname 'smithi101' 2023-07-18T03:09:04.816 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.816+0000 7f7d28a49b80 1 --2- 172.21.15.101:0/3391154055 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55d3b7cc48a0 0x55d3b7cc6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:04.816 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.816+0000 7f7d28a49b80 4 client.25948 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:09:04.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.816+0000 7f7d28a49b80 1 -- 172.21.15.101:0/3391154055 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d3b7bf0eb0 con 0x55d3b7cc48a0 2023-07-18T03:09:04.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.816+0000 7f7d0ed9d700 1 --2- 172.21.15.101:0/3391154055 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55d3b7cc48a0 0x55d3b7cc6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:04.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.817+0000 7f7d0ed9d700 1 --2- 172.21.15.101:0/3391154055 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55d3b7cc48a0 0x55d3b7cc6cb0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb245316af089d65 server_cookie=12f16d2684a495b6 in_seq=0 out_seq=0 2023-07-18T03:09:04.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.817+0000 7f7cf67fc700 10 client.25948 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:09:04.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.817+0000 7f7cf67fc700 1 -- 172.21.15.101:0/3391154055 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d3b7bf0eb0 con 0x55d3b7cc48a0 2023-07-18T03:09:04.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.817+0000 7f7cf67fc700 10 client.25948 handle_command_reply: tid=0 2023-07-18T03:09:04.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.817+0000 7f7d28a49b80 1 client.25948 shutdown 2023-07-18T03:09:04.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.817+0000 7f7d28a49b80 1 -- 172.21.15.101:0/3391154055 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b7ca5be0 msgr2=0x55d3b7cb1ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:04.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.817+0000 7f7d28a49b80 1 --2- 172.21.15.101:0/3391154055 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b7ca5be0 0x55d3b7cb1ba0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7cf801c040 tx=0x7f7cf8005720 comp rx=0 tx=0).stop 2023-07-18T03:09:04.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.817+0000 7f7d28a49b80 1 -- 172.21.15.101:0/3391154055 shutdown_connections 2023-07-18T03:09:04.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.818+0000 7f7d28a49b80 1 --2- 172.21.15.101:0/3391154055 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55d3b7cc48a0 0x55d3b7cc6cb0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.818+0000 7f7d28a49b80 1 --2- 172.21.15.101:0/3391154055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b7ca52d0 0x55d3b7cb4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.818+0000 7f7cf67fc700 0 client.25948 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:09:04.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.818+0000 7f7d28a49b80 1 --2- 172.21.15.101:0/3391154055 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3b7cb4ee0 0x55d3b7cb22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.818+0000 7f7d28a49b80 1 --2- 172.21.15.101:0/3391154055 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b7ca5be0 0x55d3b7cb1ba0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.818+0000 7f7d28a49b80 1 -- 172.21.15.101:0/3391154055 >> 172.21.15.101:0/3391154055 conn(0x55d3b7c9bda0 msgr2=0x55d3b7ca1290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:04.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.818+0000 7f7d28a49b80 1 -- 172.21.15.101:0/3391154055 shutdown_connections 2023-07-18T03:09:04.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.818+0000 7f7d28a49b80 1 -- 172.21.15.101:0/3391154055 wait complete. 2023-07-18T03:09:04.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.818+0000 7f7d28a49b80 20 client.25948 trim_cache size 0 max 16384 2023-07-18T03:09:04.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.835+0000 7f7d28a49b80 1 Processor -- start 2023-07-18T03:09:04.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.835+0000 7f7d28a49b80 1 -- start start 2023-07-18T03:09:04.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.835+0000 7f7d28a49b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3b7ca52d0 0x55d3b7cb13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:04.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.835+0000 7f7d28a49b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b7ca5be0 0x55d3b7cc00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:04.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.835+0000 7f7d28a49b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b7cb4ee0 0x55d3b7cacee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:04.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.835+0000 7f7d28a49b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b7ca6540 con 0x55d3b7ca52d0 2023-07-18T03:09:04.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.835+0000 7f7d28a49b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b7ca5fb0 con 0x55d3b7ca5be0 2023-07-18T03:09:04.826 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.835+0000 7f7d28a49b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b7c98ef0 con 0x55d3b7cb4ee0 2023-07-18T03:09:04.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.836+0000 7f7d0ed9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3b7ca52d0 0x55d3b7cb13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:04.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.836+0000 7f7d0ed9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3b7ca52d0 0x55d3b7cb13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:35452/0 (socket says 172.21.15.101:35452) 2023-07-18T03:09:04.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.836+0000 7f7d0ed9d700 1 -- 172.21.15.101:0/3930019170 learned_addr learned my addr 172.21.15.101:0/3930019170 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:04.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.836+0000 7f7d07fff700 1 --2- 172.21.15.101:0/3930019170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b7ca5be0 0x55d3b7cc00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:04.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.836+0000 7f7d0f59e700 1 --2- 172.21.15.101:0/3930019170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b7cb4ee0 0x55d3b7cacee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:04.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.836+0000 7f7d0ed9d700 1 -- 172.21.15.101:0/3930019170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b7cb4ee0 msgr2=0x55d3b7cacee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:04.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.836+0000 7f7d0ed9d700 1 --2- 172.21.15.101:0/3930019170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b7cb4ee0 0x55d3b7cacee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.836+0000 7f7d0ed9d700 1 -- 172.21.15.101:0/3930019170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b7ca5be0 msgr2=0x55d3b7cc00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:04.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.836+0000 7f7d0ed9d700 1 --2- 172.21.15.101:0/3930019170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b7ca5be0 0x55d3b7cc00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.836+0000 7f7d0ed9d700 1 -- 172.21.15.101:0/3930019170 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf8009980 con 0x55d3b7ca52d0 2023-07-18T03:09:04.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.836+0000 7f7d0f59e700 1 --2- 172.21.15.101:0/3930019170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b7cb4ee0 0x55d3b7cacee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:09:04.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.836+0000 7f7d07fff700 1 --2- 172.21.15.101:0/3930019170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b7ca5be0 0x55d3b7cc00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:09:04.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.836+0000 7f7d0ed9d700 1 --2- 172.21.15.101:0/3930019170 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3b7ca52d0 0x55d3b7cb13a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7d0009b190 tx=0x7f7d00009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2acf98679affb3a1 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:04.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.836+0000 7f7d05ffb700 1 -- 172.21.15.101:0/3930019170 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d000a10b0 con 0x55d3b7ca52d0 2023-07-18T03:09:04.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.836+0000 7f7d05ffb700 1 -- 172.21.15.101:0/3930019170 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d0009b200 con 0x55d3b7ca52d0 2023-07-18T03:09:04.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.836+0000 7f7d05ffb700 1 -- 172.21.15.101:0/3930019170 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0009b460 con 0x55d3b7ca52d0 2023-07-18T03:09:04.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.837+0000 7f7d28a49b80 1 -- 172.21.15.101:0/3930019170 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3b7ca52d0 msgr2=0x55d3b7cb13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:04.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.837+0000 7f7d28a49b80 1 --2- 172.21.15.101:0/3930019170 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3b7ca52d0 0x55d3b7cb13a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7d0009b190 tx=0x7f7d00009ca0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.837+0000 7f7d28a49b80 1 -- 172.21.15.101:0/3930019170 shutdown_connections 2023-07-18T03:09:04.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.837+0000 7f7d28a49b80 1 --2- 172.21.15.101:0/3930019170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b7cb4ee0 0x55d3b7cacee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.837+0000 7f7d28a49b80 1 --2- 172.21.15.101:0/3930019170 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3b7ca52d0 0x55d3b7cb13a0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.837+0000 7f7d28a49b80 1 --2- 172.21.15.101:0/3930019170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b7ca5be0 0x55d3b7cc00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.837+0000 7f7d28a49b80 1 -- 172.21.15.101:0/3930019170 >> 172.21.15.101:0/3930019170 conn(0x55d3b7c9bda0 msgr2=0x55d3b7c9fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:04.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.837+0000 7f7d28a49b80 1 -- 172.21.15.101:0/3930019170 shutdown_connections 2023-07-18T03:09:04.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.837+0000 7f7d28a49b80 1 -- 172.21.15.101:0/3930019170 wait complete. 2023-07-18T03:09:04.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.838+0000 7f7d28a49b80 1 -- start start 2023-07-18T03:09:04.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.838+0000 7f7d28a49b80 1 Processor -- start 2023-07-18T03:09:04.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.838+0000 7f7d28a49b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b7ca52d0 0x55d3b7cbb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:04.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.838+0000 7f7d28a49b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b7ca5be0 0x55d3b7cb9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:04.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.838+0000 7f7d28a49b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3b7cb4ee0 0x55d3b7cba000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:04.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.838+0000 7f7d28a49b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b7cba540 con 0x55d3b7cb4ee0 2023-07-18T03:09:04.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.838+0000 7f7d28a49b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b7cba6b0 con 0x55d3b7ca52d0 2023-07-18T03:09:04.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.838+0000 7f7d28a49b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b7cba870 con 0x55d3b7ca5be0 2023-07-18T03:09:04.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.838+0000 7f7d28a49b80 10 client.0 _ll_register_callbacks cb 0x7f7d00005c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:09:04.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.838+0000 7f7d0f59e700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3b7cb4ee0 0x55d3b7cba000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:04.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.839+0000 7f7d0ed9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b7ca52d0 0x55d3b7cbb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:04.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.839+0000 7f7d0f59e700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3b7cb4ee0 0x55d3b7cba000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:35454/0 (socket says 172.21.15.101:35454) 2023-07-18T03:09:04.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.839+0000 7f7d0f59e700 1 -- 172.21.15.101:0/339486621 learned_addr learned my addr 172.21.15.101:0/339486621 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:04.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.839+0000 7f7d07fff700 1 --2- 172.21.15.101:0/339486621 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b7ca5be0 0x55d3b7cb9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:04.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.839+0000 7f7d0f59e700 1 -- 172.21.15.101:0/339486621 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b7ca5be0 msgr2=0x55d3b7cb9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:04.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.839+0000 7f7d0f59e700 1 --2- 172.21.15.101:0/339486621 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b7ca5be0 0x55d3b7cb9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.839+0000 7f7d0f59e700 1 -- 172.21.15.101:0/339486621 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b7ca52d0 msgr2=0x55d3b7cbb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:04.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.839+0000 7f7d0f59e700 1 --2- 172.21.15.101:0/339486621 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b7ca52d0 0x55d3b7cbb7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.839+0000 7f7d0f59e700 1 -- 172.21.15.101:0/339486621 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d0801a470 con 0x55d3b7cb4ee0 2023-07-18T03:09:04.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.839+0000 7f7d0f59e700 1 --2- 172.21.15.101:0/339486621 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3b7cb4ee0 0x55d3b7cba000 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7d0801aeb0 tx=0x7f7d08004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e311c4593aa007c7 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:04.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.839+0000 7f7cf67fc700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:09:04.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.839+0000 7f7cf67fc700 1 -- 172.21.15.101:0/339486621 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0800e050 con 0x55d3b7cb4ee0 2023-07-18T03:09:04.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.839+0000 7f7d28a49b80 1 -- 172.21.15.101:0/339486621 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7cf8009e70 con 0x55d3b7cb4ee0 2023-07-18T03:09:04.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.839+0000 7f7cf67fc700 1 -- 172.21.15.101:0/339486621 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f7d08017d30 con 0x55d3b7cb4ee0 2023-07-18T03:09:04.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.840+0000 7f7cf67fc700 1 -- 172.21.15.101:0/339486621 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d08003a00 con 0x55d3b7cb4ee0 2023-07-18T03:09:04.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.840+0000 7f7d28a49b80 10 client.15900 fetch_fsmap learned FSMap version 25 2023-07-18T03:09:04.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.840+0000 7f7d28a49b80 1 -- 172.21.15.101:0/339486621 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f7d10196680 con 0x55d3b7cb4ee0 2023-07-18T03:09:04.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.840+0000 7f7cf67fc700 1 -- 172.21.15.101:0/339486621 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7cf8009e70 con 0x55d3b7cb4ee0 2023-07-18T03:09:04.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.840+0000 7f7cf67fc700 1 -- 172.21.15.101:0/339486621 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f7d08025450 con 0x55d3b7cb4ee0 2023-07-18T03:09:04.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.840+0000 7f7d28a49b80 10 client.15900 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:09:04.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.840+0000 7f7d28a49b80 10 client.15900 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:09:04.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.840+0000 7f7d28a49b80 20 client.15900 populate_metadata read hostname 'smithi101' 2023-07-18T03:09:04.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.840+0000 7f7d28a49b80 1 --2- 172.21.15.101:0/339486621 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55d3b7d19cd0 0x55d3b7d1c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:04.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.840+0000 7f7d28a49b80 4 client.15900 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:09:04.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.840+0000 7f7d28a49b80 1 -- 172.21.15.101:0/339486621 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d3b7bb3050 con 0x55d3b7d19cd0 2023-07-18T03:09:04.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.841+0000 7f7d0ed9d700 1 --2- 172.21.15.101:0/339486621 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55d3b7d19cd0 0x55d3b7d1c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:04.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.841+0000 7f7cf67fc700 10 client.15900 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:09:04.839 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.841+0000 7f7d0ed9d700 1 --2- 172.21.15.101:0/339486621 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55d3b7d19cd0 0x55d3b7d1c0e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3cac2dc5a5640ea server_cookie=4b1cecdc8392a0e2 in_seq=0 out_seq=0 2023-07-18T03:09:04.839 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.841+0000 7f7cf67fc700 1 -- 172.21.15.101:0/339486621 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55d3b7bb3050 con 0x55d3b7d19cd0 2023-07-18T03:09:04.839 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.841+0000 7f7cf67fc700 10 client.15900 handle_command_reply: tid=0 2023-07-18T03:09:04.839 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.841+0000 7f7d28a49b80 1 client.15900 shutdown 2023-07-18T03:09:04.839 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.841+0000 7f7d28a49b80 1 -- 172.21.15.101:0/339486621 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3b7cb4ee0 msgr2=0x55d3b7cba000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:04.839 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.841+0000 7f7d28a49b80 1 --2- 172.21.15.101:0/339486621 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3b7cb4ee0 0x55d3b7cba000 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7d0801aeb0 tx=0x7f7d08004b40 comp rx=0 tx=0).stop 2023-07-18T03:09:04.840 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.842+0000 7f7d28a49b80 1 -- 172.21.15.101:0/339486621 shutdown_connections 2023-07-18T03:09:04.840 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.842+0000 7f7d28a49b80 1 --2- 172.21.15.101:0/339486621 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55d3b7d19cd0 0x55d3b7d1c0e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.840 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.842+0000 7f7d28a49b80 1 --2- 172.21.15.101:0/339486621 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b7ca52d0 0x55d3b7cbb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.840 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.842+0000 7f7d28a49b80 1 --2- 172.21.15.101:0/339486621 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3b7cb4ee0 0x55d3b7cba000 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.840 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.842+0000 7f7d28a49b80 1 --2- 172.21.15.101:0/339486621 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b7ca5be0 0x55d3b7cb9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.840 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.842+0000 7f7cf67fc700 0 client.15900 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:09:04.840 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.842+0000 7f7d28a49b80 1 -- 172.21.15.101:0/339486621 >> 172.21.15.101:0/339486621 conn(0x55d3b7c9bda0 msgr2=0x55d3b7cacee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:04.841 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.842+0000 7f7d28a49b80 1 -- 172.21.15.101:0/339486621 shutdown_connections 2023-07-18T03:09:04.841 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.842+0000 7f7d28a49b80 1 -- 172.21.15.101:0/339486621 wait complete. 2023-07-18T03:09:04.841 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.842+0000 7f7d28a49b80 20 client.15900 trim_cache size 0 max 16384 2023-07-18T03:09:04.841 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:09:04.842 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "scrub active (2 inodes in the stack)", 2023-07-18T03:09:04.842 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:09:04.842 INFO:teuthology.orchestra.run.smithi101.stdout: "e1976b04-545c-4f4a-b690-4d59d9363daf": { 2023-07-18T03:09:04.842 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:09:04.842 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "e1976b04-545c-4f4a-b690-4d59d9363daf", 2023-07-18T03:09:04.843 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:09:04.843 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:09:04.843 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:09:04.843 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:09:04.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.842+0000 7f7d151ff700 1 -- 172.21.15.101:0/2849678862 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f7cfc0a1460 msgr2=0x7f7cfc0a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:04.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.842+0000 7f7d151ff700 1 --2- 172.21.15.101:0/2849678862 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f7cfc0a1460 0x7f7cfc0a3910 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7d08009fd0 tx=0x7f7d08007b00 comp rx=0 tx=0).stop 2023-07-18T03:09:04.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.842+0000 7f7d151ff700 1 -- 172.21.15.101:0/2849678862 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7d100ffb00 msgr2=0x7f7d1019b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:04.845 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.842+0000 7f7d151ff700 1 --2- 172.21.15.101:0/2849678862 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7d100ffb00 0x7f7d1019b9f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7d00005910 tx=0x7f7d00005720 comp rx=0 tx=0).stop 2023-07-18T03:09:04.845 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.843+0000 7f7d151ff700 1 -- 172.21.15.101:0/2849678862 shutdown_connections 2023-07-18T03:09:04.845 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.843+0000 7f7d151ff700 1 --2- 172.21.15.101:0/2849678862 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f7cfc0a1460 0x7f7cfc0a3910 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.845 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.843+0000 7f7d151ff700 1 --2- 172.21.15.101:0/2849678862 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d10105b10 0x7f7d10195a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.845 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.843+0000 7f7d151ff700 1 --2- 172.21.15.101:0/2849678862 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7d100ffb00 0x7f7d1019b9f0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.846 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.843+0000 7f7d151ff700 1 --2- 172.21.15.101:0/2849678862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d101004b0 0x7f7d1019bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:04.846 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.843+0000 7f7d151ff700 1 -- 172.21.15.101:0/2849678862 >> 172.21.15.101:0/2849678862 conn(0x7f7d100fb600 msgr2=0x7f7d100fbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:04.846 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.843+0000 7f7d151ff700 1 -- 172.21.15.101:0/2849678862 shutdown_connections 2023-07-18T03:09:04.846 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:04.843+0000 7f7d151ff700 1 -- 172.21.15.101:0/2849678862 wait complete. 2023-07-18T03:09:04.849 INFO:tasks.cephfs.filesystem:scrub status for tag:e1976b04-545c-4f4a-b690-4d59d9363daf - {'path': '/', 'tag': 'e1976b04-545c-4f4a-b690-4d59d9363daf', 'options': 'recursive,force'} 2023-07-18T03:09:05.446 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:05 smithi179 ceph-mon[129229]: pgmap v48: 129 pgs: 129 active+clean; 31 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 1.2 MiB/s wr, 158 op/s 2023-07-18T03:09:05.483 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:05 smithi101 ceph-mon[126379]: pgmap v48: 129 pgs: 129 active+clean; 31 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 1.2 MiB/s wr, 158 op/s 2023-07-18T03:09:05.541 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:05 smithi155 ceph-mon[128885]: pgmap v48: 129 pgs: 129 active+clean; 31 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 1.2 MiB/s wr, 158 op/s 2023-07-18T03:09:05.561 INFO:tasks.workunit.client.0.smithi101.stdout: 1 65270 12.34 MB/sec warmup 30 sec latency 4371.506 ms 2023-07-18T03:09:06.446 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:06 smithi179 ceph-mon[129229]: scrub queued for path: / 2023-07-18T03:09:06.446 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:06 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:09:06.446 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:06 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:09:06.483 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:06 smithi101 ceph-mon[126379]: scrub queued for path: / 2023-07-18T03:09:06.484 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:06 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:09:06.484 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:06 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:09:06.541 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:06 smithi155 ceph-mon[128885]: scrub queued for path: / 2023-07-18T03:09:06.541 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:06 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:09:06.541 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:06 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:09:06.561 INFO:tasks.workunit.client.0.smithi101.stdout: 1 65270 11.94 MB/sec warmup 31 sec latency 5371.608 ms 2023-07-18T03:09:07.446 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:07 smithi179 ceph-mon[129229]: pgmap v49: 129 pgs: 129 active+clean; 33 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 1.5 MiB/s wr, 211 op/s 2023-07-18T03:09:07.446 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:07 smithi179 ceph-mon[129229]: mgrmap e25: x(active, since 92s), standbys: y, z 2023-07-18T03:09:07.483 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:07 smithi101 ceph-mon[126379]: pgmap v49: 129 pgs: 129 active+clean; 33 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 1.5 MiB/s wr, 211 op/s 2023-07-18T03:09:07.484 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:07 smithi101 ceph-mon[126379]: mgrmap e25: x(active, since 92s), standbys: y, z 2023-07-18T03:09:07.541 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:07 smithi155 ceph-mon[128885]: pgmap v49: 129 pgs: 129 active+clean; 33 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 1.5 MiB/s wr, 211 op/s 2023-07-18T03:09:07.541 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:07 smithi155 ceph-mon[128885]: mgrmap e25: x(active, since 92s), standbys: y, z 2023-07-18T03:09:07.561 INFO:tasks.workunit.client.0.smithi101.stdout: 1 65270 11.57 MB/sec warmup 32 sec latency 6371.743 ms 2023-07-18T03:09:08.446 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:08 smithi179 ceph-mon[129229]: pgmap v50: 129 pgs: 129 active+clean; 33 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 889 KiB/s wr, 130 op/s 2023-07-18T03:09:08.483 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:08 smithi101 ceph-mon[126379]: pgmap v50: 129 pgs: 129 active+clean; 33 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 889 KiB/s wr, 130 op/s 2023-07-18T03:09:08.541 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:08 smithi155 ceph-mon[128885]: pgmap v50: 129 pgs: 129 active+clean; 33 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 889 KiB/s wr, 130 op/s 2023-07-18T03:09:08.561 INFO:tasks.workunit.client.0.smithi101.stdout: 1 65270 11.22 MB/sec warmup 33 sec latency 7371.881 ms 2023-07-18T03:09:09.562 INFO:tasks.workunit.client.0.smithi101.stdout: 1 65270 10.89 MB/sec warmup 34 sec latency 8372.026 ms 2023-07-18T03:09:10.562 INFO:tasks.workunit.client.0.smithi101.stdout: 1 65270 10.58 MB/sec warmup 35 sec latency 9372.184 ms 2023-07-18T03:09:11.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:10 smithi179 ceph-mon[129229]: pgmap v51: 129 pgs: 129 active+clean; 63 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 3.6 MiB/s wr, 168 op/s 2023-07-18T03:09:11.233 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:10 smithi101 ceph-mon[126379]: pgmap v51: 129 pgs: 129 active+clean; 63 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 3.6 MiB/s wr, 168 op/s 2023-07-18T03:09:11.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:10 smithi155 ceph-mon[128885]: pgmap v51: 129 pgs: 129 active+clean; 63 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 3.6 MiB/s wr, 168 op/s 2023-07-18T03:09:11.562 INFO:tasks.workunit.client.0.smithi101.stdout: 1 65270 10.28 MB/sec warmup 36 sec latency 10372.301 ms 2023-07-18T03:09:12.562 INFO:tasks.workunit.client.0.smithi101.stdout: 1 65270 10.01 MB/sec warmup 37 sec latency 11372.458 ms 2023-07-18T03:09:13.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:12 smithi179 ceph-mon[129229]: pgmap v52: 129 pgs: 129 active+clean; 63 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 3.1 MiB/s wr, 93 op/s 2023-07-18T03:09:13.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:12 smithi101 ceph-mon[126379]: pgmap v52: 129 pgs: 129 active+clean; 63 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 3.1 MiB/s wr, 93 op/s 2023-07-18T03:09:13.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:12 smithi155 ceph-mon[128885]: pgmap v52: 129 pgs: 129 active+clean; 63 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 3.1 MiB/s wr, 93 op/s 2023-07-18T03:09:13.562 INFO:tasks.workunit.client.0.smithi101.stdout: 1 65270 9.74 MB/sec warmup 38 sec latency 12372.559 ms 2023-07-18T03:09:14.562 INFO:tasks.workunit.client.0.smithi101.stdout: 1 65270 9.49 MB/sec warmup 39 sec latency 13372.698 ms 2023-07-18T03:09:15.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:14 smithi179 ceph-mon[129229]: pgmap v53: 129 pgs: 129 active+clean; 63 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 3.0 MiB/s wr, 91 op/s 2023-07-18T03:09:15.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:14 smithi101 ceph-mon[126379]: pgmap v53: 129 pgs: 129 active+clean; 63 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 3.0 MiB/s wr, 91 op/s 2023-07-18T03:09:15.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:14 smithi155 ceph-mon[128885]: pgmap v53: 129 pgs: 129 active+clean; 63 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 3.0 MiB/s wr, 91 op/s 2023-07-18T03:09:15.562 INFO:tasks.workunit.client.0.smithi101.stdout: 1 65400 9.26 MB/sec warmup 40 sec latency 14219.137 ms 2023-07-18T03:09:16.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:15 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:09:16.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:15 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:09:16.233 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:15 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:09:16.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:15 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:09:16.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:15 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:09:16.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:15 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:09:16.563 INFO:tasks.workunit.client.0.smithi101.stdout: 1 67489 9.34 MB/sec warmup 41 sec latency 22.576 ms 2023-07-18T03:09:17.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:16 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:09:17.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:16 smithi179 ceph-mon[129229]: pgmap v54: 129 pgs: 129 active+clean; 86 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 6.1 MiB/s wr, 166 op/s 2023-07-18T03:09:17.233 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:16 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:09:17.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:16 smithi101 ceph-mon[126379]: pgmap v54: 129 pgs: 129 active+clean; 86 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 6.1 MiB/s wr, 166 op/s 2023-07-18T03:09:17.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:16 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:09:17.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:16 smithi155 ceph-mon[128885]: pgmap v54: 129 pgs: 129 active+clean; 86 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 6.1 MiB/s wr, 166 op/s 2023-07-18T03:09:17.563 INFO:tasks.workunit.client.0.smithi101.stdout: 1 67745 9.14 MB/sec warmup 42 sec latency 733.010 ms 2023-07-18T03:09:18.563 INFO:tasks.workunit.client.0.smithi101.stdout: 1 67745 8.93 MB/sec warmup 43 sec latency 1733.126 ms 2023-07-18T03:09:19.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:18 smithi179 ceph-mon[129229]: pgmap v55: 129 pgs: 129 active+clean; 86 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 5.9 MiB/s wr, 112 op/s 2023-07-18T03:09:19.233 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:18 smithi101 ceph-mon[126379]: pgmap v55: 129 pgs: 129 active+clean; 86 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 5.9 MiB/s wr, 112 op/s 2023-07-18T03:09:19.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:18 smithi155 ceph-mon[128885]: pgmap v55: 129 pgs: 129 active+clean; 86 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 5.9 MiB/s wr, 112 op/s 2023-07-18T03:09:19.563 INFO:tasks.workunit.client.0.smithi101.stdout: 1 67745 8.73 MB/sec warmup 44 sec latency 2733.240 ms 2023-07-18T03:09:20.563 INFO:tasks.workunit.client.0.smithi101.stdout: 1 67745 8.53 MB/sec warmup 45 sec latency 3733.357 ms 2023-07-18T03:09:21.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:20 smithi179 ceph-mon[129229]: pgmap v56: 129 pgs: 129 active+clean; 50 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 6.2 MiB/s wr, 194 op/s 2023-07-18T03:09:21.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:20 smithi101 ceph-mon[126379]: pgmap v56: 129 pgs: 129 active+clean; 50 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 6.2 MiB/s wr, 194 op/s 2023-07-18T03:09:21.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:20 smithi155 ceph-mon[128885]: pgmap v56: 129 pgs: 129 active+clean; 50 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 6.2 MiB/s wr, 194 op/s 2023-07-18T03:09:21.563 INFO:tasks.workunit.client.0.smithi101.stdout: 1 67793 8.35 MB/sec warmup 46 sec latency 3915.326 ms 2023-07-18T03:09:22.564 INFO:tasks.workunit.client.0.smithi101.stdout: 1 67793 8.18 MB/sec warmup 47 sec latency 1770.729 ms 2023-07-18T03:09:23.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:22 smithi179 ceph-mon[129229]: pgmap v57: 129 pgs: 129 active+clean; 50 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 3.5 MiB/s wr, 156 op/s 2023-07-18T03:09:23.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:22 smithi101 ceph-mon[126379]: pgmap v57: 129 pgs: 129 active+clean; 50 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 3.5 MiB/s wr, 156 op/s 2023-07-18T03:09:23.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:22 smithi155 ceph-mon[128885]: pgmap v57: 129 pgs: 129 active+clean; 50 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 3.5 MiB/s wr, 156 op/s 2023-07-18T03:09:23.563 INFO:tasks.workunit.client.0.smithi101.stdout: 1 67793 8.01 MB/sec warmup 48 sec latency 2770.835 ms 2023-07-18T03:09:24.564 INFO:tasks.workunit.client.0.smithi101.stdout: 1 67793 7.84 MB/sec warmup 49 sec latency 3770.942 ms 2023-07-18T03:09:25.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:24 smithi179 ceph-mon[129229]: pgmap v58: 129 pgs: 129 active+clean; 50 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 3.5 MiB/s wr, 156 op/s 2023-07-18T03:09:25.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:24 smithi101 ceph-mon[126379]: pgmap v58: 129 pgs: 129 active+clean; 50 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 3.5 MiB/s wr, 156 op/s 2023-07-18T03:09:25.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:24 smithi155 ceph-mon[128885]: pgmap v58: 129 pgs: 129 active+clean; 50 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 3.5 MiB/s wr, 156 op/s 2023-07-18T03:09:25.564 INFO:tasks.workunit.client.0.smithi101.stdout: 1 67793 7.69 MB/sec warmup 50 sec latency 4771.080 ms 2023-07-18T03:09:26.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:25 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:09:26.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:25 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:09:26.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:25 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:09:26.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:25 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:09:26.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:25 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:09:26.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:25 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:09:26.564 INFO:tasks.workunit.client.0.smithi101.stdout: 1 67805 7.53 MB/sec warmup 51 sec latency 4955.491 ms 2023-07-18T03:09:27.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:26 smithi179 ceph-mon[129229]: pgmap v59: 129 pgs: 129 active+clean; 50 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 3.5 MiB/s wr, 162 op/s 2023-07-18T03:09:27.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:26 smithi101 ceph-mon[126379]: pgmap v59: 129 pgs: 129 active+clean; 50 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 3.5 MiB/s wr, 162 op/s 2023-07-18T03:09:27.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:26 smithi155 ceph-mon[128885]: pgmap v59: 129 pgs: 129 active+clean; 50 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 3.5 MiB/s wr, 162 op/s 2023-07-18T03:09:27.564 INFO:tasks.workunit.client.0.smithi101.stdout: 1 67805 7.39 MB/sec warmup 52 sec latency 1799.689 ms 2023-07-18T03:09:28.564 INFO:tasks.workunit.client.0.smithi101.stdout: 1 67805 7.25 MB/sec warmup 53 sec latency 2799.849 ms 2023-07-18T03:09:29.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:28 smithi179 ceph-mon[129229]: pgmap v60: 129 pgs: 129 active+clean; 50 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 377 KiB/s wr, 87 op/s 2023-07-18T03:09:29.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:28 smithi101 ceph-mon[126379]: pgmap v60: 129 pgs: 129 active+clean; 50 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 377 KiB/s wr, 87 op/s 2023-07-18T03:09:29.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:28 smithi155 ceph-mon[128885]: pgmap v60: 129 pgs: 129 active+clean; 50 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 377 KiB/s wr, 87 op/s 2023-07-18T03:09:29.564 INFO:tasks.workunit.client.0.smithi101.stdout: 1 67805 7.12 MB/sec warmup 54 sec latency 3799.963 ms 2023-07-18T03:09:30.565 INFO:tasks.workunit.client.0.smithi101.stdout: 1 67805 6.99 MB/sec warmup 55 sec latency 4800.108 ms 2023-07-18T03:09:31.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:30 smithi179 ceph-mon[129229]: pgmap v61: 129 pgs: 129 active+clean; 50 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 383 KiB/s wr, 92 op/s 2023-07-18T03:09:31.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:30 smithi101 ceph-mon[126379]: pgmap v61: 129 pgs: 129 active+clean; 50 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 383 KiB/s wr, 92 op/s 2023-07-18T03:09:31.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:30 smithi155 ceph-mon[128885]: pgmap v61: 129 pgs: 129 active+clean; 50 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 383 KiB/s wr, 92 op/s 2023-07-18T03:09:31.564 INFO:tasks.workunit.client.0.smithi101.stdout: 1 69058 6.96 MB/sec warmup 56 sec latency 4984.200 ms 2023-07-18T03:09:32.565 INFO:tasks.workunit.client.0.smithi101.stdout: 1 71233 7.07 MB/sec warmup 57 sec latency 59.574 ms 2023-07-18T03:09:33.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:32 smithi179 ceph-mon[129229]: pgmap v62: 129 pgs: 129 active+clean; 50 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-07-18T03:09:33.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:32 smithi101 ceph-mon[126379]: pgmap v62: 129 pgs: 129 active+clean; 50 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-07-18T03:09:33.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:32 smithi155 ceph-mon[128885]: pgmap v62: 129 pgs: 129 active+clean; 50 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-07-18T03:09:33.565 INFO:tasks.workunit.client.0.smithi101.stdout: 1 73138 7.05 MB/sec warmup 58 sec latency 13.554 ms 2023-07-18T03:09:34.565 INFO:tasks.workunit.client.0.smithi101.stdout: 1 74911 7.16 MB/sec warmup 59 sec latency 10.521 ms 2023-07-18T03:09:34.850 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:09:35.059 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.070+0000 7f4aa85ff700 1 -- 172.21.15.101:0/692800128 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4aa0105af0 msgr2=0x7f4aa0105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:35.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.070+0000 7f4aa85ff700 1 --2- 172.21.15.101:0/692800128 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4aa0105af0 0x7f4aa0105ec0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c009990 tx=0x7f4a9c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.070+0000 7f4aa85ff700 1 -- 172.21.15.101:0/692800128 shutdown_connections 2023-07-18T03:09:35.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.070+0000 7f4aa85ff700 1 --2- 172.21.15.101:0/692800128 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4aa0105af0 0x7f4aa0105ec0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.070+0000 7f4aa85ff700 1 --2- 172.21.15.101:0/692800128 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4aa0100490 0x7f4aa010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.070+0000 7f4aa85ff700 1 --2- 172.21.15.101:0/692800128 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4aa00ffae0 0x7f4aa00fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.070+0000 7f4aa85ff700 1 -- 172.21.15.101:0/692800128 >> 172.21.15.101:0/692800128 conn(0x7f4aa00fb5e0 msgr2=0x7f4aa00fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:35.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.070+0000 7f4aa85ff700 1 -- 172.21.15.101:0/692800128 shutdown_connections 2023-07-18T03:09:35.061 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.070+0000 7f4aa85ff700 1 -- 172.21.15.101:0/692800128 wait complete. 2023-07-18T03:09:35.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.071+0000 7f4aa85ff700 1 Processor -- start 2023-07-18T03:09:35.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.071+0000 7f4aa85ff700 1 -- start start 2023-07-18T03:09:35.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.072+0000 7f4aa85ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4aa00ffae0 0x7f4aa019b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:35.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.072+0000 7f4aa85ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4aa0100490 0x7f4aa019bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:35.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.072+0000 7f4aa85ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4aa0105af0 0x7f4aa0195a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:35.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.072+0000 7f4aa85ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa01960a0 con 0x7f4aa00ffae0 2023-07-18T03:09:35.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.072+0000 7f4aa85ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa0196210 con 0x7f4aa0100490 2023-07-18T03:09:35.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.072+0000 7f4aa85ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa01963d0 con 0x7f4aa0105af0 2023-07-18T03:09:35.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.072+0000 7f4aa639b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4aa00ffae0 0x7f4aa019b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:35.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.072+0000 7f4aa5b9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4aa0100490 0x7f4aa019bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:35.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.072+0000 7f4aa5b9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4aa0100490 0x7f4aa019bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:34066/0 (socket says 172.21.15.101:34066) 2023-07-18T03:09:35.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.072+0000 7f4aa5b9a700 1 -- 172.21.15.101:0/287231014 learned_addr learned my addr 172.21.15.101:0/287231014 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:35.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.072+0000 7f4aa6b9c700 1 --2- 172.21.15.101:0/287231014 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4aa0105af0 0x7f4aa0195a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:35.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.072+0000 7f4aa639b700 1 -- 172.21.15.101:0/287231014 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4aa0105af0 msgr2=0x7f4aa0195a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:35.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.072+0000 7f4aa639b700 1 --2- 172.21.15.101:0/287231014 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4aa0105af0 0x7f4aa0195a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.072+0000 7f4aa639b700 1 -- 172.21.15.101:0/287231014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4aa0100490 msgr2=0x7f4aa019bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:35.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.072+0000 7f4aa639b700 1 --2- 172.21.15.101:0/287231014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4aa0100490 0x7f4aa019bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.072+0000 7f4aa639b700 1 -- 172.21.15.101:0/287231014 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a9c0095f0 con 0x7f4aa00ffae0 2023-07-18T03:09:35.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.072+0000 7f4aa5b9a700 1 --2- 172.21.15.101:0/287231014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4aa0100490 0x7f4aa019bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:09:35.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.073+0000 7f4aa639b700 1 --2- 172.21.15.101:0/287231014 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4aa00ffae0 0x7f4aa019b9c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c0098e0 tx=0x7f4a9c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:35.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.073+0000 7f4a977fe700 1 -- 172.21.15.101:0/287231014 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c01d070 con 0x7f4aa00ffae0 2023-07-18T03:09:35.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.073+0000 7f4aa85ff700 1 -- 172.21.15.101:0/287231014 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aa0196650 con 0x7f4aa00ffae0 2023-07-18T03:09:35.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.073+0000 7f4a977fe700 1 -- 172.21.15.101:0/287231014 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a9c005030 con 0x7f4aa00ffae0 2023-07-18T03:09:35.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.073+0000 7f4a977fe700 1 -- 172.21.15.101:0/287231014 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c004ab0 con 0x7f4aa00ffae0 2023-07-18T03:09:35.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.073+0000 7f4aa85ff700 1 -- 172.21.15.101:0/287231014 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aa01a29b0 con 0x7f4aa00ffae0 2023-07-18T03:09:35.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.076+0000 7f4a977fe700 1 -- 172.21.15.101:0/287231014 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f4a9c005410 con 0x7f4aa00ffae0 2023-07-18T03:09:35.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.078+0000 7f4abbfc4b80 1 Processor -- start 2023-07-18T03:09:35.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.078+0000 7f4abbfc4b80 1 -- start start 2023-07-18T03:09:35.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.078+0000 7f4a977fe700 1 --2- 172.21.15.101:0/287231014 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4a8c0a12c0 0x7f4a8c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:35.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.078+0000 7f4abbfc4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558e7580d6e0 0x558e7580dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:35.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.078+0000 7f4a977fe700 1 -- 172.21.15.101:0/287231014 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f4a9c0ce180 con 0x7f4aa00ffae0 2023-07-18T03:09:35.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.078+0000 7f4abbfc4b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e7580dff0 0x558e758189a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:35.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.078+0000 7f4abbfc4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e75818ee0 0x558e7581b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:35.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.078+0000 7f4abbfc4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e7570fcb0 con 0x558e7580d6e0 2023-07-18T03:09:35.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.078+0000 7f4abbfc4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e75762a40 con 0x558e75818ee0 2023-07-18T03:09:35.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.078+0000 7f4abbfc4b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e7575bed0 con 0x558e7580dff0 2023-07-18T03:09:35.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.078+0000 7f4aa6b9c700 1 --2- 172.21.15.101:0/287231014 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4a8c0a12c0 0x7f4a8c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:35.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.078+0000 7f4aa5b9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e7580dff0 0x558e758189a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:35.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.078+0000 7f4aa5b9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e7580dff0 0x558e758189a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:37218/0 (socket says 172.21.15.101:37218) 2023-07-18T03:09:35.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.078+0000 7f4aa5b9a700 1 -- 172.21.15.101:0/559854787 learned_addr learned my addr 172.21.15.101:0/559854787 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:35.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.079+0000 7f4aa639b700 1 --2- 172.21.15.101:0/559854787 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558e7580d6e0 0x558e7580dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:35.070 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.079+0000 7f4aa6b9c700 1 --2- 172.21.15.101:0/559854787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e75818ee0 0x558e7581b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:35.070 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.079+0000 7f4aa5b9a700 1 -- 172.21.15.101:0/559854787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e75818ee0 msgr2=0x558e7581b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:35.070 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.079+0000 7f4aa5b9a700 1 --2- 172.21.15.101:0/559854787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e75818ee0 0x558e7581b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.070 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.079+0000 7f4aa5b9a700 1 -- 172.21.15.101:0/559854787 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558e7580d6e0 msgr2=0x558e7580dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:35.070 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.079+0000 7f4aa5b9a700 1 --2- 172.21.15.101:0/559854787 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558e7580d6e0 0x558e7580dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.070 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.079+0000 7f4aa5b9a700 1 -- 172.21.15.101:0/559854787 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e75589de0 con 0x558e7580dff0 2023-07-18T03:09:35.070 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.079+0000 7f4aa5b9a700 1 --2- 172.21.15.101:0/559854787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e7580dff0 0x558e758189a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4a90011680 tx=0x7f4a90011990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4356cdb6625ce447 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:35.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.079+0000 7f4aa6b9c700 1 --2- 172.21.15.101:0/287231014 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4a8c0a12c0 0x7f4a8c0a3770 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4a98003fd0 tx=0x7f4a98004040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:35.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.079+0000 7f4a94bfe700 1 -- 172.21.15.101:0/559854787 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a90004020 con 0x558e7580dff0 2023-07-18T03:09:35.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.079+0000 7f4a94bfe700 1 -- 172.21.15.101:0/559854787 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a90003680 con 0x558e7580dff0 2023-07-18T03:09:35.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.079+0000 7f4a94bfe700 1 -- 172.21.15.101:0/559854787 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a90019650 con 0x558e7580dff0 2023-07-18T03:09:35.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.080+0000 7f4abbfc4b80 1 -- 172.21.15.101:0/559854787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e7580dff0 msgr2=0x558e758189a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:35.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.080+0000 7f4abbfc4b80 1 --2- 172.21.15.101:0/559854787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e7580dff0 0x558e758189a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4a90011680 tx=0x7f4a90011990 comp rx=0 tx=0).stop 2023-07-18T03:09:35.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.080+0000 7f4abbfc4b80 1 -- 172.21.15.101:0/559854787 shutdown_connections 2023-07-18T03:09:35.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.080+0000 7f4abbfc4b80 1 --2- 172.21.15.101:0/559854787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e75818ee0 0x558e7581b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.075 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.080+0000 7f4abbfc4b80 1 --2- 172.21.15.101:0/559854787 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558e7580d6e0 0x558e7580dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.075 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.080+0000 7f4abbfc4b80 1 --2- 172.21.15.101:0/559854787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e7580dff0 0x558e758189a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.075 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.080+0000 7f4abbfc4b80 1 -- 172.21.15.101:0/559854787 >> 172.21.15.101:0/559854787 conn(0x558e757ffda0 msgr2=0x558e758056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:35.076 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.080+0000 7f4abbfc4b80 1 -- 172.21.15.101:0/559854787 shutdown_connections 2023-07-18T03:09:35.076 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.080+0000 7f4abbfc4b80 1 -- 172.21.15.101:0/559854787 wait complete. 2023-07-18T03:09:35.076 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.081+0000 7f4abbfc4b80 1 -- start start 2023-07-18T03:09:35.076 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.081+0000 7f4abbfc4b80 1 Processor -- start 2023-07-18T03:09:35.076 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.081+0000 7f4abbfc4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558e7580d6e0 0x558e75818970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:35.076 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.081+0000 7f4abbfc4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e7580dff0 0x558e75815bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:35.076 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.081+0000 7f4abbfc4b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e75818ee0 0x558e75816330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:35.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.081+0000 7f4abbfc4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e756fbac0 con 0x558e7580d6e0 2023-07-18T03:09:35.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.081+0000 7f4abbfc4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e756fb7c0 con 0x558e7580dff0 2023-07-18T03:09:35.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.081+0000 7f4abbfc4b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e755df920 con 0x558e75818ee0 2023-07-18T03:09:35.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.082+0000 7f4abbfc4b80 10 client.0 _ll_register_callbacks cb 0x558e756f9760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:09:35.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.082+0000 7f4aa5b9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e7580dff0 0x558e75815bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:35.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.082+0000 7f4aa5b9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e7580dff0 0x558e75815bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:34092/0 (socket says 172.21.15.101:34092) 2023-07-18T03:09:35.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.082+0000 7f4aa5b9a700 1 -- 172.21.15.101:0/2009057048 learned_addr learned my addr 172.21.15.101:0/2009057048 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:35.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.082+0000 7f4aa6b9c700 1 --2- 172.21.15.101:0/2009057048 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e75818ee0 0x558e75816330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:35.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.082+0000 7f4aa639b700 1 --2- 172.21.15.101:0/2009057048 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558e7580d6e0 0x558e75818970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:35.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.082+0000 7f4aa6b9c700 1 -- 172.21.15.101:0/2009057048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e7580dff0 msgr2=0x558e75815bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:35.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.082+0000 7f4aa6b9c700 1 --2- 172.21.15.101:0/2009057048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e7580dff0 0x558e75815bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.082+0000 7f4aa6b9c700 1 -- 172.21.15.101:0/2009057048 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558e7580d6e0 msgr2=0x558e75818970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:35.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.082+0000 7f4aa6b9c700 1 --2- 172.21.15.101:0/2009057048 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558e7580d6e0 0x558e75818970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.082+0000 7f4aa6b9c700 1 -- 172.21.15.101:0/2009057048 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a90009e10 con 0x558e75818ee0 2023-07-18T03:09:35.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.083+0000 7f4aa6b9c700 1 --2- 172.21.15.101:0/2009057048 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e75818ee0 0x558e75816330 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4a98007780 tx=0x7f4a98004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2992660eee8bdb47 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:35.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.083+0000 7f4a89ffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:09:35.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.083+0000 7f4a89ffb700 1 -- 172.21.15.101:0/2009057048 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9801a880 con 0x558e75818ee0 2023-07-18T03:09:35.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.083+0000 7f4abbfc4b80 1 -- 172.21.15.101:0/2009057048 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558e756b54f0 con 0x558e75818ee0 2023-07-18T03:09:35.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.083+0000 7f4a89ffb700 1 -- 172.21.15.101:0/2009057048 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f4a9801a9e0 con 0x558e75818ee0 2023-07-18T03:09:35.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.083+0000 7f4a89ffb700 1 -- 172.21.15.101:0/2009057048 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a9801aba0 con 0x558e75818ee0 2023-07-18T03:09:35.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.083+0000 7f4abbfc4b80 10 client.25966 fetch_fsmap learned FSMap version 25 2023-07-18T03:09:35.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.083+0000 7f4abbfc4b80 1 -- 172.21.15.101:0/2009057048 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558e75572c60 con 0x558e75818ee0 2023-07-18T03:09:35.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.083+0000 7f4a89ffb700 1 -- 172.21.15.101:0/2009057048 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558e756b54f0 con 0x558e75818ee0 2023-07-18T03:09:35.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.083+0000 7f4a89ffb700 1 -- 172.21.15.101:0/2009057048 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f4a98022030 con 0x558e75818ee0 2023-07-18T03:09:35.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.084+0000 7f4abbfc4b80 10 client.25966 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:09:35.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.084+0000 7f4abbfc4b80 10 client.25966 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:09:35.081 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.084+0000 7f4abbfc4b80 20 client.25966 populate_metadata read hostname 'smithi101' 2023-07-18T03:09:35.081 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.084+0000 7f4abbfc4b80 1 --2- 172.21.15.101:0/2009057048 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x558e758289c0 0x558e7582add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:35.081 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.084+0000 7f4abbfc4b80 4 client.25966 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:09:35.081 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.084+0000 7f4abbfc4b80 1 -- 172.21.15.101:0/2009057048 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558e75754eb0 con 0x558e758289c0 2023-07-18T03:09:35.081 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.084+0000 7f4aa639b700 1 --2- 172.21.15.101:0/2009057048 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x558e758289c0 0x558e7582add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:35.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.084+0000 7f4aa639b700 1 --2- 172.21.15.101:0/2009057048 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x558e758289c0 0x558e7582add0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40ea5d81b199951e server_cookie=ea9e610e5d763c7d in_seq=0 out_seq=0 2023-07-18T03:09:35.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.084+0000 7f4a89ffb700 10 client.25966 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:09:35.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.085+0000 7f4a89ffb700 1 -- 172.21.15.101:0/2009057048 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558e75754eb0 con 0x558e758289c0 2023-07-18T03:09:35.083 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.085+0000 7f4a89ffb700 10 client.25966 handle_command_reply: tid=0 2023-07-18T03:09:35.083 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.085+0000 7f4abbfc4b80 1 client.25966 shutdown 2023-07-18T03:09:35.083 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.085+0000 7f4abbfc4b80 1 -- 172.21.15.101:0/2009057048 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e75818ee0 msgr2=0x558e75816330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:35.083 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.085+0000 7f4abbfc4b80 1 --2- 172.21.15.101:0/2009057048 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e75818ee0 0x558e75816330 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4a98007780 tx=0x7f4a98004b40 comp rx=0 tx=0).stop 2023-07-18T03:09:35.083 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.085+0000 7f4abbfc4b80 1 -- 172.21.15.101:0/2009057048 shutdown_connections 2023-07-18T03:09:35.083 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.085+0000 7f4abbfc4b80 1 --2- 172.21.15.101:0/2009057048 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x558e758289c0 0x558e7582add0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.083 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.085+0000 7f4abbfc4b80 1 --2- 172.21.15.101:0/2009057048 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e75818ee0 0x558e75816330 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.084 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.085+0000 7f4a89ffb700 0 client.25966 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:09:35.084 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.085+0000 7f4abbfc4b80 1 --2- 172.21.15.101:0/2009057048 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558e7580d6e0 0x558e75818970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.084 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.085+0000 7f4abbfc4b80 1 --2- 172.21.15.101:0/2009057048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e7580dff0 0x558e75815bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.084 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.085+0000 7f4abbfc4b80 1 -- 172.21.15.101:0/2009057048 >> 172.21.15.101:0/2009057048 conn(0x558e757ffda0 msgr2=0x558e75805230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:35.084 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.085+0000 7f4abbfc4b80 1 -- 172.21.15.101:0/2009057048 shutdown_connections 2023-07-18T03:09:35.084 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.085+0000 7f4abbfc4b80 1 -- 172.21.15.101:0/2009057048 wait complete. 2023-07-18T03:09:35.084 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.086+0000 7f4abbfc4b80 20 client.25966 trim_cache size 0 max 16384 2023-07-18T03:09:35.100 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.111+0000 7f4abbfc4b80 1 Processor -- start 2023-07-18T03:09:35.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.112+0000 7f4abbfc4b80 1 -- start start 2023-07-18T03:09:35.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.112+0000 7f4abbfc4b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e757ffda0 0x558e75818510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:35.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.112+0000 7f4abbfc4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e7580d6e0 0x558e7581f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:35.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.112+0000 7f4abbfc4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558e7580dff0 0x558e75812e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:35.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.112+0000 7f4abbfc4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e758171d0 con 0x558e7580dff0 2023-07-18T03:09:35.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.112+0000 7f4abbfc4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e7580de30 con 0x558e7580d6e0 2023-07-18T03:09:35.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.112+0000 7f4abbfc4b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e7581fe30 con 0x558e757ffda0 2023-07-18T03:09:35.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.113+0000 7f4aa6b9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558e7580dff0 0x558e75812e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:35.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.113+0000 7f4aa6b9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558e7580dff0 0x558e75812e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:41836/0 (socket says 172.21.15.101:41836) 2023-07-18T03:09:35.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.113+0000 7f4aa6b9c700 1 -- 172.21.15.101:0/2297387527 learned_addr learned my addr 172.21.15.101:0/2297387527 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:35.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.113+0000 7f4aa639b700 1 --2- 172.21.15.101:0/2297387527 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e757ffda0 0x558e75818510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:35.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.114+0000 7f4aa6b9c700 1 -- 172.21.15.101:0/2297387527 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e757ffda0 msgr2=0x558e75818510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:35.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.114+0000 7f4aa6b9c700 1 --2- 172.21.15.101:0/2297387527 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e757ffda0 0x558e75818510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.114+0000 7f4aa6b9c700 1 -- 172.21.15.101:0/2297387527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e7580d6e0 msgr2=0x558e7581f8f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-18T03:09:35.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.114+0000 7f4aa6b9c700 1 --2- 172.21.15.101:0/2297387527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e7580d6e0 0x558e7581f8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.114+0000 7f4aa6b9c700 1 -- 172.21.15.101:0/2297387527 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e755b0250 con 0x558e7580dff0 2023-07-18T03:09:35.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.114+0000 7f4aa6b9c700 1 --2- 172.21.15.101:0/2297387527 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558e7580dff0 0x558e75812e60 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4a980091b0 tx=0x7f4a980091e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92c39cba05c0ddda server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:35.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.115+0000 7f4a953ff700 1 -- 172.21.15.101:0/2297387527 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a98003b70 con 0x558e7580dff0 2023-07-18T03:09:35.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.115+0000 7f4a953ff700 1 -- 172.21.15.101:0/2297387527 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a9800f040 con 0x558e7580dff0 2023-07-18T03:09:35.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.115+0000 7f4a953ff700 1 -- 172.21.15.101:0/2297387527 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a98019a90 con 0x558e7580dff0 2023-07-18T03:09:35.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.116+0000 7f4abbfc4b80 1 -- 172.21.15.101:0/2297387527 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558e7580dff0 msgr2=0x558e75812e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:35.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.116+0000 7f4abbfc4b80 1 --2- 172.21.15.101:0/2297387527 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558e7580dff0 0x558e75812e60 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4a980091b0 tx=0x7f4a980091e0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.116+0000 7f4abbfc4b80 1 -- 172.21.15.101:0/2297387527 shutdown_connections 2023-07-18T03:09:35.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.116+0000 7f4abbfc4b80 1 --2- 172.21.15.101:0/2297387527 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558e7580dff0 0x558e75812e60 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.116+0000 7f4abbfc4b80 1 --2- 172.21.15.101:0/2297387527 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e757ffda0 0x558e75818510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.116+0000 7f4abbfc4b80 1 --2- 172.21.15.101:0/2297387527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e7580d6e0 0x558e7581f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.116+0000 7f4abbfc4b80 1 -- 172.21.15.101:0/2297387527 >> 172.21.15.101:0/2297387527 conn(0x558e758164a0 msgr2=0x558e75810ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:35.108 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.116+0000 7f4abbfc4b80 1 -- 172.21.15.101:0/2297387527 shutdown_connections 2023-07-18T03:09:35.108 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.117+0000 7f4abbfc4b80 1 -- 172.21.15.101:0/2297387527 wait complete. 2023-07-18T03:09:35.108 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.117+0000 7f4abbfc4b80 1 -- start start 2023-07-18T03:09:35.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.118+0000 7f4abbfc4b80 1 Processor -- start 2023-07-18T03:09:35.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.118+0000 7f4abbfc4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e757ffda0 0x558e758075f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:35.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.118+0000 7f4abbfc4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558e7580d6e0 0x558e75807b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:35.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.118+0000 7f4abbfc4b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e7580dff0 0x558e75808070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:35.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.118+0000 7f4abbfc4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e7580b4b0 con 0x558e7580d6e0 2023-07-18T03:09:35.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.118+0000 7f4abbfc4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e7581f930 con 0x558e757ffda0 2023-07-18T03:09:35.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.118+0000 7f4abbfc4b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e7581faf0 con 0x558e7580dff0 2023-07-18T03:09:35.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.118+0000 7f4abbfc4b80 10 client.0 _ll_register_callbacks cb 0x7f4a98004620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:09:35.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.119+0000 7f4aa639b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e757ffda0 0x558e758075f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:35.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.119+0000 7f4aa639b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e757ffda0 0x558e758075f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:34102/0 (socket says 172.21.15.101:34102) 2023-07-18T03:09:35.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.119+0000 7f4aa639b700 1 -- 172.21.15.101:0/4272703006 learned_addr learned my addr 172.21.15.101:0/4272703006 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:35.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.119+0000 7f4aa6b9c700 1 --2- 172.21.15.101:0/4272703006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e7580dff0 0x558e75808070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:35.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.119+0000 7f4aa5b9a700 1 --2- 172.21.15.101:0/4272703006 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558e7580d6e0 0x558e75807b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:35.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.120+0000 7f4aa639b700 1 -- 172.21.15.101:0/4272703006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e7580dff0 msgr2=0x558e75808070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:35.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.120+0000 7f4aa639b700 1 --2- 172.21.15.101:0/4272703006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e7580dff0 0x558e75808070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.120+0000 7f4aa639b700 1 -- 172.21.15.101:0/4272703006 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558e7580d6e0 msgr2=0x558e75807b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:35.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.120+0000 7f4aa639b700 1 --2- 172.21.15.101:0/4272703006 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558e7580d6e0 0x558e75807b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.120+0000 7f4aa639b700 1 -- 172.21.15.101:0/4272703006 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558e75572c60 con 0x558e757ffda0 2023-07-18T03:09:35.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.120+0000 7f4aa639b700 1 --2- 172.21.15.101:0/4272703006 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e757ffda0 0x558e758075f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c09aec0 tx=0x7f4a9c09ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e58587232cc3b6af server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:35.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.120+0000 7f4a89ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:09:35.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.121+0000 7f4a89ffb700 1 -- 172.21.15.101:0/4272703006 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c0b3070 con 0x558e757ffda0 2023-07-18T03:09:35.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.121+0000 7f4a89ffb700 1 -- 172.21.15.101:0/4272703006 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f4a9c0b7030 con 0x558e757ffda0 2023-07-18T03:09:35.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.121+0000 7f4abbfc4b80 1 -- 172.21.15.101:0/4272703006 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a90003680 con 0x558e757ffda0 2023-07-18T03:09:35.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.121+0000 7f4a89ffb700 1 -- 172.21.15.101:0/4272703006 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a9c0b92e0 con 0x558e757ffda0 2023-07-18T03:09:35.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.121+0000 7f4abbfc4b80 10 client.25946 fetch_fsmap learned FSMap version 25 2023-07-18T03:09:35.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.121+0000 7f4abbfc4b80 1 -- 172.21.15.101:0/4272703006 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558e755882b0 con 0x558e757ffda0 2023-07-18T03:09:35.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.122+0000 7f4a89ffb700 1 -- 172.21.15.101:0/4272703006 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a90003680 con 0x558e757ffda0 2023-07-18T03:09:35.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.122+0000 7f4a89ffb700 1 -- 172.21.15.101:0/4272703006 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f4a9c09a420 con 0x558e757ffda0 2023-07-18T03:09:35.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.122+0000 7f4abbfc4b80 10 client.25946 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:09:35.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.122+0000 7f4abbfc4b80 10 client.25946 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:09:35.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.122+0000 7f4abbfc4b80 20 client.25946 populate_metadata read hostname 'smithi101' 2023-07-18T03:09:35.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.122+0000 7f4abbfc4b80 1 --2- 172.21.15.101:0/4272703006 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x558e75818ee0 0x558e758950a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:35.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.122+0000 7f4abbfc4b80 4 client.25946 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:09:35.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.122+0000 7f4abbfc4b80 1 -- 172.21.15.101:0/4272703006 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558e75717050 con 0x558e75818ee0 2023-07-18T03:09:35.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.122+0000 7f4aa5b9a700 1 --2- 172.21.15.101:0/4272703006 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x558e75818ee0 0x558e758950a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:35.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.123+0000 7f4aa5b9a700 1 --2- 172.21.15.101:0/4272703006 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x558e75818ee0 0x558e758950a0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92e2e1030ebbf614 server_cookie=200d2e04825c3efa in_seq=0 out_seq=0 2023-07-18T03:09:35.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.123+0000 7f4a89ffb700 10 client.25946 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:09:35.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.124+0000 7f4a89ffb700 1 -- 172.21.15.101:0/4272703006 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558e75717050 con 0x558e75818ee0 2023-07-18T03:09:35.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.124+0000 7f4a89ffb700 10 client.25946 handle_command_reply: tid=0 2023-07-18T03:09:35.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.124+0000 7f4abbfc4b80 1 client.25946 shutdown 2023-07-18T03:09:35.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.125+0000 7f4abbfc4b80 1 -- 172.21.15.101:0/4272703006 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e757ffda0 msgr2=0x558e758075f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:35.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.125+0000 7f4abbfc4b80 1 --2- 172.21.15.101:0/4272703006 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e757ffda0 0x558e758075f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c09aec0 tx=0x7f4a9c09ef60 comp rx=0 tx=0).stop 2023-07-18T03:09:35.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.125+0000 7f4abbfc4b80 1 -- 172.21.15.101:0/4272703006 shutdown_connections 2023-07-18T03:09:35.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.125+0000 7f4abbfc4b80 1 --2- 172.21.15.101:0/4272703006 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x558e75818ee0 0x558e758950a0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.125+0000 7f4abbfc4b80 1 --2- 172.21.15.101:0/4272703006 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e757ffda0 0x558e758075f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.125+0000 7f4abbfc4b80 1 --2- 172.21.15.101:0/4272703006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e7580dff0 0x558e75808070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.125+0000 7f4abbfc4b80 1 --2- 172.21.15.101:0/4272703006 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558e7580d6e0 0x558e75807b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.125+0000 7f4abbfc4b80 1 -- 172.21.15.101:0/4272703006 >> 172.21.15.101:0/4272703006 conn(0x558e758164a0 msgr2=0x558e7580f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:35.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.125+0000 7f4a89ffb700 0 client.25946 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:09:35.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.125+0000 7f4abbfc4b80 1 -- 172.21.15.101:0/4272703006 shutdown_connections 2023-07-18T03:09:35.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.125+0000 7f4abbfc4b80 1 -- 172.21.15.101:0/4272703006 wait complete. 2023-07-18T03:09:35.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.127+0000 7f4abbfc4b80 20 client.25946 trim_cache size 0 max 16384 2023-07-18T03:09:35.118 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:09:35.119 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "no active scrubs running", 2023-07-18T03:09:35.119 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": {} 2023-07-18T03:09:35.119 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:09:35.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.129+0000 7f4aa85ff700 1 -- 172.21.15.101:0/287231014 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4a8c0a12c0 msgr2=0x7f4a8c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:35.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.129+0000 7f4aa85ff700 1 --2- 172.21.15.101:0/287231014 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4a8c0a12c0 0x7f4a8c0a3770 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4a98003fd0 tx=0x7f4a98004040 comp rx=0 tx=0).stop 2023-07-18T03:09:35.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.129+0000 7f4aa85ff700 1 -- 172.21.15.101:0/287231014 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4aa00ffae0 msgr2=0x7f4aa019b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:35.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.129+0000 7f4aa85ff700 1 --2- 172.21.15.101:0/287231014 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4aa00ffae0 0x7f4aa019b9c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c0098e0 tx=0x7f4a9c004730 comp rx=0 tx=0).stop 2023-07-18T03:09:35.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.130+0000 7f4aa85ff700 1 -- 172.21.15.101:0/287231014 shutdown_connections 2023-07-18T03:09:35.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.130+0000 7f4aa85ff700 1 --2- 172.21.15.101:0/287231014 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4aa0105af0 0x7f4aa0195a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.130+0000 7f4aa85ff700 1 --2- 172.21.15.101:0/287231014 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4aa00ffae0 0x7f4aa019b9c0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.130+0000 7f4aa85ff700 1 --2- 172.21.15.101:0/287231014 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4a8c0a12c0 0x7f4a8c0a3770 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.130+0000 7f4aa85ff700 1 --2- 172.21.15.101:0/287231014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4aa0100490 0x7f4aa019bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.130+0000 7f4aa85ff700 1 -- 172.21.15.101:0/287231014 >> 172.21.15.101:0/287231014 conn(0x7f4aa00fb5e0 msgr2=0x7f4aa00fbc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:35.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.131+0000 7f4aa85ff700 1 -- 172.21.15.101:0/287231014 shutdown_connections 2023-07-18T03:09:35.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.131+0000 7f4aa85ff700 1 -- 172.21.15.101:0/287231014 wait complete. 2023-07-18T03:09:35.139 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-18T03:09:35.139 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:09:35.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:34 smithi179 ceph-mon[129229]: pgmap v63: 129 pgs: 129 active+clean; 50 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-07-18T03:09:35.203 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:34 smithi101 ceph-mon[126379]: pgmap v63: 129 pgs: 129 active+clean; 50 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-07-18T03:09:35.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:34 smithi155 ceph-mon[128885]: pgmap v63: 129 pgs: 129 active+clean; 50 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-07-18T03:09:35.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.357+0000 7f7814dff700 1 -- 172.21.15.101:0/3030734987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7810072830 msgr2=0x7f7810072c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:35.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.357+0000 7f7814dff700 1 --2- 172.21.15.101:0/3030734987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7810072830 0x7f7810072c00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7800009990 tx=0x7f7800009ca0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.357+0000 7f7814dff700 1 -- 172.21.15.101:0/3030734987 shutdown_connections 2023-07-18T03:09:35.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.357+0000 7f7814dff700 1 --2- 172.21.15.101:0/3030734987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f781010c3d0 0x7f781010e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.357+0000 7f7814dff700 1 --2- 172.21.15.101:0/3030734987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7810072830 0x7f7810072c00 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.357+0000 7f7814dff700 1 --2- 172.21.15.101:0/3030734987 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7810073140 0x7f781010be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.357+0000 7f7814dff700 1 -- 172.21.15.101:0/3030734987 >> 172.21.15.101:0/3030734987 conn(0x7f78100fb610 msgr2=0x7f78100fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:35.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.357+0000 7f7814dff700 1 -- 172.21.15.101:0/3030734987 shutdown_connections 2023-07-18T03:09:35.348 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.357+0000 7f7814dff700 1 -- 172.21.15.101:0/3030734987 wait complete. 2023-07-18T03:09:35.349 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.358+0000 7f7814dff700 1 Processor -- start 2023-07-18T03:09:35.349 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.358+0000 7f7814dff700 1 -- start start 2023-07-18T03:09:35.349 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.359+0000 7f7814dff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7810072830 0x7f7810197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:35.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.359+0000 7f7814dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7810073140 0x7f78101983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:35.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.359+0000 7f7814dff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f781010c3d0 0x7f781019c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:35.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.359+0000 7f7814dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f781019ce70 con 0x7f7810073140 2023-07-18T03:09:35.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.359+0000 7f7814dff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f781019cfe0 con 0x7f7810072830 2023-07-18T03:09:35.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.359+0000 7f7814dff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f781019d1a0 con 0x7f781010c3d0 2023-07-18T03:09:35.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.359+0000 7f780dd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7810073140 0x7f78101983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:35.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.359+0000 7f780ed9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f781010c3d0 0x7f781019c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:35.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.359+0000 7f780ed9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f781010c3d0 0x7f781019c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:37274/0 (socket says 172.21.15.101:37274) 2023-07-18T03:09:35.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.359+0000 7f780ed9d700 1 -- 172.21.15.101:0/1731436417 learned_addr learned my addr 172.21.15.101:0/1731436417 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:35.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.359+0000 7f780e59c700 1 --2- 172.21.15.101:0/1731436417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7810072830 0x7f7810197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:35.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.359+0000 7f780ed9d700 1 -- 172.21.15.101:0/1731436417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7810072830 msgr2=0x7f7810197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:35.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.359+0000 7f780ed9d700 1 --2- 172.21.15.101:0/1731436417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7810072830 0x7f7810197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.359+0000 7f780ed9d700 1 -- 172.21.15.101:0/1731436417 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7810073140 msgr2=0x7f78101983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:35.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.359+0000 7f780ed9d700 1 --2- 172.21.15.101:0/1731436417 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7810073140 0x7f78101983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.359+0000 7f780ed9d700 1 -- 172.21.15.101:0/1731436417 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7804009520 con 0x7f781010c3d0 2023-07-18T03:09:35.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.359+0000 7f780dd9b700 1 --2- 172.21.15.101:0/1731436417 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7810073140 0x7f78101983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:09:35.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.360+0000 7f780ed9d700 1 --2- 172.21.15.101:0/1731436417 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f781010c3d0 0x7f781019c830 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7808007ee0 tx=0x7f780800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:35.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.360+0000 7f77ff7fe700 1 -- 172.21.15.101:0/1731436417 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780800dd20 con 0x7f781010c3d0 2023-07-18T03:09:35.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.360+0000 7f77ff7fe700 1 -- 172.21.15.101:0/1731436417 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f780800f040 con 0x7f781010c3d0 2023-07-18T03:09:35.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.360+0000 7f7814dff700 1 -- 172.21.15.101:0/1731436417 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78000095f0 con 0x7f781010c3d0 2023-07-18T03:09:35.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.360+0000 7f77ff7fe700 1 -- 172.21.15.101:0/1731436417 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78080048a0 con 0x7f781010c3d0 2023-07-18T03:09:35.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.360+0000 7f7814dff700 1 -- 172.21.15.101:0/1731436417 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78101a4ae0 con 0x7f781010c3d0 2023-07-18T03:09:35.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.361+0000 7f7814dff700 1 -- 172.21.15.101:0/1731436417 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78101075e0 con 0x7f781010c3d0 2023-07-18T03:09:35.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.363+0000 7f77ff7fe700 1 -- 172.21.15.101:0/1731436417 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f7808003a00 con 0x7f781010c3d0 2023-07-18T03:09:35.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.364+0000 7f77ff7fe700 1 --2- 172.21.15.101:0/1731436417 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f77f80a1380 0x7f77f80a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:35.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.364+0000 7f77ff7fe700 1 -- 172.21.15.101:0/1731436417 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f78080cc990 con 0x7f781010c3d0 2023-07-18T03:09:35.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.364+0000 7f780e59c700 1 --2- 172.21.15.101:0/1731436417 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f77f80a1380 0x7f77f80a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:35.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.365+0000 7f780e59c700 1 --2- 172.21.15.101:0/1731436417 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f77f80a1380 0x7f77f80a3830 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f780000b2e0 tx=0x7f7800005fd0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:35.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.368+0000 7f77ff7fe700 1 -- 172.21.15.101:0/1731436417 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f78080959f0 con 0x7f781010c3d0 2023-07-18T03:09:35.566 INFO:tasks.workunit.client.0.smithi101.stdout: 1 76898 7.16 MB/sec warmup 60 sec latency 29.670 ms 2023-07-18T03:09:35.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.635+0000 7f7814dff700 1 -- 172.21.15.101:0/1731436417 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78101991e0 con 0x7f781010c3d0 2023-07-18T03:09:35.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.635+0000 7f77ff7fe700 1 -- 172.21.15.101:0/1731436417 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10861 (secure 0 0 0) 0x7f7808099010 con 0x7f781010c3d0 2023-07-18T03:09:35.625 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:09:35.626 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/4192846769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":4192846769},{"type":"v1","addr":"172.21.15.101:6839","nonce":4192846769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6841/3824340175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6840","nonce":3824340175},{"type":"v1","addr":"172.21.15.101:6841","nonce":3824340175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:04:59.361185+0000","modified":"2023-07-18T03:09:01.105419+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.155:6839/2055462483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2055462483},{"type":"v1","addr":"172.21.15.155:6839","nonce":2055462483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:09:35.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.638+0000 7f7814dff700 1 -- 172.21.15.101:0/1731436417 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f77f80a1380 msgr2=0x7f77f80a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:35.628 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.638+0000 7f7814dff700 1 --2- 172.21.15.101:0/1731436417 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f77f80a1380 0x7f77f80a3830 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f780000b2e0 tx=0x7f7800005fd0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.628 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.638+0000 7f7814dff700 1 -- 172.21.15.101:0/1731436417 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f781010c3d0 msgr2=0x7f781019c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:35.629 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.638+0000 7f7814dff700 1 --2- 172.21.15.101:0/1731436417 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f781010c3d0 0x7f781019c830 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7808007ee0 tx=0x7f780800d3b0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.629 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.638+0000 7f7814dff700 1 -- 172.21.15.101:0/1731436417 shutdown_connections 2023-07-18T03:09:35.629 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.638+0000 7f7814dff700 1 --2- 172.21.15.101:0/1731436417 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f77f80a1380 0x7f77f80a3830 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.638+0000 7f7814dff700 1 --2- 172.21.15.101:0/1731436417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7810072830 0x7f7810197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.638+0000 7f7814dff700 1 --2- 172.21.15.101:0/1731436417 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f781010c3d0 0x7f781019c830 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.638+0000 7f7814dff700 1 --2- 172.21.15.101:0/1731436417 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7810073140 0x7f78101983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.631 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.638+0000 7f7814dff700 1 -- 172.21.15.101:0/1731436417 >> 172.21.15.101:0/1731436417 conn(0x7f78100fb610 msgr2=0x7f781010e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:35.631 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.638+0000 7f7814dff700 1 -- 172.21.15.101:0/1731436417 shutdown_connections 2023-07-18T03:09:35.631 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.638+0000 7f7814dff700 1 -- 172.21.15.101:0/1731436417 wait complete. 2023-07-18T03:09:35.632 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 25 2023-07-18T03:09:35.646 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-18T03:09:35.851 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.861+0000 7fed0bbff700 1 -- 172.21.15.101:0/2110464326 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed040ffad0 msgr2=0x7fed040fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:35.851 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.861+0000 7fed0bbff700 1 --2- 172.21.15.101:0/2110464326 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed040ffad0 0x7fed040fff40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fecf4009990 tx=0x7fecf4009ca0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.851 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.861+0000 7fed0bbff700 1 -- 172.21.15.101:0/2110464326 shutdown_connections 2023-07-18T03:09:35.851 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.861+0000 7fed0bbff700 1 --2- 172.21.15.101:0/2110464326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed04105ae0 0x7fed04105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.851 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.861+0000 7fed0bbff700 1 --2- 172.21.15.101:0/2110464326 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fed04100480 0x7fed0410c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.852 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.861+0000 7fed0bbff700 1 --2- 172.21.15.101:0/2110464326 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed040ffad0 0x7fed040fff40 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.852 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.861+0000 7fed0bbff700 1 -- 172.21.15.101:0/2110464326 >> 172.21.15.101:0/2110464326 conn(0x7fed040fb5f0 msgr2=0x7fed040fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:35.852 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.861+0000 7fed0bbff700 1 -- 172.21.15.101:0/2110464326 shutdown_connections 2023-07-18T03:09:35.852 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.861+0000 7fed0bbff700 1 -- 172.21.15.101:0/2110464326 wait complete. 2023-07-18T03:09:35.853 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.862+0000 7fed0bbff700 1 Processor -- start 2023-07-18T03:09:35.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.862+0000 7fed0bbff700 1 -- start start 2023-07-18T03:09:35.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.863+0000 7fed0bbff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed040ffad0 0x7fed0419b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:35.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.863+0000 7fed0bbff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed04100480 0x7fed0419bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:35.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.863+0000 7fed0bbff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fed04105ae0 0x7fed04195a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:35.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.863+0000 7fed0bbff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed041960b0 con 0x7fed04105ae0 2023-07-18T03:09:35.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.863+0000 7fed0bbff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed04196220 con 0x7fed04100480 2023-07-18T03:09:35.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.863+0000 7fed0bbff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed041963e0 con 0x7fed040ffad0 2023-07-18T03:09:35.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.863+0000 7fed0919a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed04100480 0x7fed0419bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:35.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.863+0000 7fed0999b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed040ffad0 0x7fed0419b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:35.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.863+0000 7fed0919a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed04100480 0x7fed0419bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:37644/0 (socket says 172.21.15.101:37644) 2023-07-18T03:09:35.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.863+0000 7fed0919a700 1 -- 172.21.15.101:0/1587240353 learned_addr learned my addr 172.21.15.101:0/1587240353 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:35.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.863+0000 7fed0a19c700 1 --2- 172.21.15.101:0/1587240353 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fed04105ae0 0x7fed04195a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:35.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.863+0000 7fed0919a700 1 -- 172.21.15.101:0/1587240353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed040ffad0 msgr2=0x7fed0419b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:35.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.863+0000 7fed0919a700 1 --2- 172.21.15.101:0/1587240353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed040ffad0 0x7fed0419b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.856 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.863+0000 7fed0919a700 1 -- 172.21.15.101:0/1587240353 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fed04105ae0 msgr2=0x7fed04195a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:35.856 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.863+0000 7fed0919a700 1 --2- 172.21.15.101:0/1587240353 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fed04105ae0 0x7fed04195a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.856 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.863+0000 7fed0919a700 1 -- 172.21.15.101:0/1587240353 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf40095f0 con 0x7fed04100480 2023-07-18T03:09:35.856 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.863+0000 7fed0999b700 1 --2- 172.21.15.101:0/1587240353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed040ffad0 0x7fed0419b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:09:35.856 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.864+0000 7fed0919a700 1 --2- 172.21.15.101:0/1587240353 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed04100480 0x7fed0419bf10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fecf4005910 tx=0x7fecf4005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:35.856 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.864+0000 7fecfaffd700 1 -- 172.21.15.101:0/1587240353 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf401d070 con 0x7fed04100480 2023-07-18T03:09:35.856 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.864+0000 7fecfaffd700 1 -- 172.21.15.101:0/1587240353 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fecf4022470 con 0x7fed04100480 2023-07-18T03:09:35.857 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.864+0000 7fed0bbff700 1 -- 172.21.15.101:0/1587240353 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed04196660 con 0x7fed04100480 2023-07-18T03:09:35.857 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.864+0000 7fecfaffd700 1 -- 172.21.15.101:0/1587240353 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf4017650 con 0x7fed04100480 2023-07-18T03:09:35.857 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.864+0000 7fed0bbff700 1 -- 172.21.15.101:0/1587240353 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed041a29c0 con 0x7fed04100480 2023-07-18T03:09:35.857 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.867+0000 7fecfaffd700 1 -- 172.21.15.101:0/1587240353 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fecf40225e0 con 0x7fed04100480 2023-07-18T03:09:35.858 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.869+0000 7fed1f437b80 1 Processor -- start 2023-07-18T03:09:35.858 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.869+0000 7fed1f437b80 1 -- start start 2023-07-18T03:09:35.859 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.870+0000 7fecfaffd700 1 --2- 172.21.15.101:0/1587240353 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fecf00a1340 0x7fecf00a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:35.860 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.870+0000 7fed1f437b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4d06e72d0 0x55d4d06e76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:35.860 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.870+0000 7fed1f437b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d06e7be0 0x55d4d06f2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:35.860 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.870+0000 7fed1f437b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4d06f2ad0 0x55d4d06f4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:35.861 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.870+0000 7fed1f437b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4d05edcb0 con 0x55d4d06f2ad0 2023-07-18T03:09:35.861 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.870+0000 7fed1f437b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4d0640a40 con 0x55d4d06e7be0 2023-07-18T03:09:35.861 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.870+0000 7fed1f437b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4d0639ed0 con 0x55d4d06e72d0 2023-07-18T03:09:35.861 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.870+0000 7fecfaffd700 1 -- 172.21.15.101:0/1587240353 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fecf40ce320 con 0x7fed04100480 2023-07-18T03:09:35.861 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.870+0000 7fed0999b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4d06e72d0 0x55d4d06e76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:35.861 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.870+0000 7fed0a19c700 1 --2- 172.21.15.101:0/1587240353 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fecf00a1340 0x7fecf00a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:35.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.870+0000 7fed0999b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4d06e72d0 0x55d4d06e76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:60540/0 (socket says 172.21.15.101:60540) 2023-07-18T03:09:35.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.870+0000 7fed0999b700 1 -- 172.21.15.101:0/1655596124 learned_addr learned my addr 172.21.15.101:0/1655596124 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:35.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.870+0000 7fed0919a700 1 --2- 172.21.15.101:0/1655596124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d06e7be0 0x55d4d06f2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:35.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.870+0000 7fed0a19c700 1 --2- 172.21.15.101:0/1655596124 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4d06f2ad0 0x55d4d06f4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:35.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.870+0000 7fed0999b700 1 -- 172.21.15.101:0/1655596124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d06e7be0 msgr2=0x55d4d06f2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:35.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.870+0000 7fed0999b700 1 --2- 172.21.15.101:0/1655596124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d06e7be0 0x55d4d06f2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.870+0000 7fed0999b700 1 -- 172.21.15.101:0/1655596124 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4d06f2ad0 msgr2=0x55d4d06f4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:35.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.870+0000 7fed0999b700 1 --2- 172.21.15.101:0/1655596124 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4d06f2ad0 0x55d4d06f4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.870+0000 7fed0999b700 1 -- 172.21.15.101:0/1655596124 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4d0467de0 con 0x55d4d06e72d0 2023-07-18T03:09:35.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.870+0000 7fed0919a700 1 --2- 172.21.15.101:0/1655596124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d06e7be0 0x55d4d06f2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:09:35.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.871+0000 7fed0999b700 1 --2- 172.21.15.101:0/1655596124 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4d06e72d0 0x55d4d06e76a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fed000115f0 tx=0x7fed00011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afb8a466c782fc36 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:35.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.871+0000 7fed0a19c700 1 --2- 172.21.15.101:0/1587240353 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fecf00a1340 0x7fecf00a37f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fecfc009ec0 tx=0x7fecfc0092f0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:35.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.871+0000 7feceffff700 1 -- 172.21.15.101:0/1655596124 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed00019630 con 0x55d4d06e72d0 2023-07-18T03:09:35.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.871+0000 7feceffff700 1 -- 172.21.15.101:0/1655596124 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed00019c50 con 0x55d4d06e72d0 2023-07-18T03:09:35.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.871+0000 7feceffff700 1 -- 172.21.15.101:0/1655596124 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0000ebe0 con 0x55d4d06e72d0 2023-07-18T03:09:35.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.872+0000 7fed1f437b80 1 -- 172.21.15.101:0/1655596124 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4d06e72d0 msgr2=0x55d4d06e76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:35.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.872+0000 7fed1f437b80 1 --2- 172.21.15.101:0/1655596124 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4d06e72d0 0x55d4d06e76a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fed000115f0 tx=0x7fed00011900 comp rx=0 tx=0).stop 2023-07-18T03:09:35.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.872+0000 7fed1f437b80 1 -- 172.21.15.101:0/1655596124 shutdown_connections 2023-07-18T03:09:35.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.872+0000 7fed1f437b80 1 --2- 172.21.15.101:0/1655596124 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4d06f2ad0 0x55d4d06f4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.872+0000 7fed1f437b80 1 --2- 172.21.15.101:0/1655596124 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4d06e72d0 0x55d4d06e76a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.872+0000 7fed1f437b80 1 --2- 172.21.15.101:0/1655596124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d06e7be0 0x55d4d06f2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.872+0000 7fed1f437b80 1 -- 172.21.15.101:0/1655596124 >> 172.21.15.101:0/1655596124 conn(0x55d4d06ddda0 msgr2=0x55d4d06e36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:35.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.872+0000 7fed1f437b80 1 -- 172.21.15.101:0/1655596124 shutdown_connections 2023-07-18T03:09:35.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.872+0000 7fed1f437b80 1 -- 172.21.15.101:0/1655596124 wait complete. 2023-07-18T03:09:35.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.872+0000 7fed1f437b80 1 -- start start 2023-07-18T03:09:35.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.872+0000 7fed1f437b80 1 Processor -- start 2023-07-18T03:09:35.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.873+0000 7fed1f437b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4d06e72d0 0x55d4d06f2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:35.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.873+0000 7fed1f437b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d06e7be0 0x55d4d06ef710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:35.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.873+0000 7fed1f437b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4d06f2ad0 0x55d4d06efd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:35.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.873+0000 7fed1f437b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4d05d9ac0 con 0x55d4d06f2ad0 2023-07-18T03:09:35.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.873+0000 7fed1f437b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4d05d97c0 con 0x55d4d06e7be0 2023-07-18T03:09:35.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.873+0000 7fed1f437b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4d04bd920 con 0x55d4d06e72d0 2023-07-18T03:09:35.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.873+0000 7fed1f437b80 10 client.0 _ll_register_callbacks cb 0x55d4d05d7760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:09:35.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.873+0000 7fed0919a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d06e7be0 0x55d4d06ef710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:35.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.873+0000 7fed0919a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d06e7be0 0x55d4d06ef710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:37650/0 (socket says 172.21.15.101:37650) 2023-07-18T03:09:35.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.873+0000 7fed0919a700 1 -- 172.21.15.101:0/4136274233 learned_addr learned my addr 172.21.15.101:0/4136274233 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:35.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.873+0000 7fed0a19c700 1 --2- 172.21.15.101:0/4136274233 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4d06f2ad0 0x55d4d06efd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:35.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.873+0000 7fed0999b700 1 --2- 172.21.15.101:0/4136274233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4d06e72d0 0x55d4d06f2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:35.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.873+0000 7fed0919a700 1 -- 172.21.15.101:0/4136274233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4d06e72d0 msgr2=0x55d4d06f2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:35.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.873+0000 7fed0919a700 1 --2- 172.21.15.101:0/4136274233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4d06e72d0 0x55d4d06f2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.873+0000 7fed0919a700 1 -- 172.21.15.101:0/4136274233 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4d06f2ad0 msgr2=0x55d4d06efd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:35.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.873+0000 7fed0919a700 1 --2- 172.21.15.101:0/4136274233 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4d06f2ad0 0x55d4d06efd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.873+0000 7fed0919a700 1 -- 172.21.15.101:0/4136274233 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed041a29c0 con 0x55d4d06e7be0 2023-07-18T03:09:35.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.873+0000 7fed0a19c700 1 --2- 172.21.15.101:0/4136274233 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4d06f2ad0 0x55d4d06efd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:09:35.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.874+0000 7fed0919a700 1 --2- 172.21.15.101:0/4136274233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d06e7be0 0x55d4d06ef710 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fecf4097b40 tx=0x7fecf409ac10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=695977ef3c4101e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:35.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.874+0000 7feced7fa700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:09:35.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.874+0000 7feced7fa700 1 -- 172.21.15.101:0/4136274233 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf409f100 con 0x55d4d06e7be0 2023-07-18T03:09:35.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.874+0000 7feced7fa700 1 -- 172.21.15.101:0/4136274233 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fecf40a3490 con 0x55d4d06e7be0 2023-07-18T03:09:35.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.874+0000 7fed1f437b80 1 -- 172.21.15.101:0/4136274233 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d4d05934f0 con 0x55d4d06e7be0 2023-07-18T03:09:35.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.874+0000 7feced7fa700 1 -- 172.21.15.101:0/4136274233 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fecf409b330 con 0x55d4d06e7be0 2023-07-18T03:09:35.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.874+0000 7fed1f437b80 10 client.25964 fetch_fsmap learned FSMap version 25 2023-07-18T03:09:35.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.874+0000 7fed1f437b80 1 -- 172.21.15.101:0/4136274233 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fed00009d50 con 0x55d4d06e7be0 2023-07-18T03:09:35.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.874+0000 7feced7fa700 1 -- 172.21.15.101:0/4136274233 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d4d05934f0 con 0x55d4d06e7be0 2023-07-18T03:09:35.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.874+0000 7feced7fa700 1 -- 172.21.15.101:0/4136274233 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fecf409a5c0 con 0x55d4d06e7be0 2023-07-18T03:09:35.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.875+0000 7fed1f437b80 10 client.25964 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:09:35.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.875+0000 7fed1f437b80 10 client.25964 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:09:35.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.875+0000 7fed1f437b80 20 client.25964 populate_metadata read hostname 'smithi101' 2023-07-18T03:09:35.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.875+0000 7fed1f437b80 1 --2- 172.21.15.101:0/4136274233 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55d4d0701fe0 0x55d4d07043f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:35.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.875+0000 7fed1f437b80 4 client.25964 mds_command: new command op to 24541 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:09:35.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.875+0000 7fed1f437b80 1 -- 172.21.15.101:0/4136274233 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d4d0632eb0 con 0x55d4d0701fe0 2023-07-18T03:09:35.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.875+0000 7fed0999b700 1 --2- 172.21.15.101:0/4136274233 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55d4d0701fe0 0x55d4d07043f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:35.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.875+0000 7fed0999b700 1 --2- 172.21.15.101:0/4136274233 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55d4d0701fe0 0x55d4d07043f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d3fc61a9ecf1a313 server_cookie=5d72fad7599a2f3f in_seq=0 out_seq=0 2023-07-18T03:09:35.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.875+0000 7feced7fa700 10 client.25964 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:09:35.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.876+0000 7feced7fa700 1 -- 172.21.15.101:0/4136274233 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d4d0632eb0 con 0x55d4d0701fe0 2023-07-18T03:09:35.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.876+0000 7feced7fa700 10 client.25964 handle_command_reply: tid=0 2023-07-18T03:09:35.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.876+0000 7fed1f437b80 1 client.25964 shutdown 2023-07-18T03:09:35.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.876+0000 7fed1f437b80 1 -- 172.21.15.101:0/4136274233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d06e7be0 msgr2=0x55d4d06ef710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:35.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.876+0000 7fed1f437b80 1 --2- 172.21.15.101:0/4136274233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d06e7be0 0x55d4d06ef710 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fecf4097b40 tx=0x7fecf409ac10 comp rx=0 tx=0).stop 2023-07-18T03:09:35.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.876+0000 7fed1f437b80 1 -- 172.21.15.101:0/4136274233 shutdown_connections 2023-07-18T03:09:35.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.876+0000 7fed1f437b80 1 --2- 172.21.15.101:0/4136274233 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55d4d0701fe0 0x55d4d07043f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.876+0000 7fed1f437b80 1 --2- 172.21.15.101:0/4136274233 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4d06f2ad0 0x55d4d06efd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.876+0000 7feced7fa700 0 client.25964 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:09:35.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.876+0000 7fed1f437b80 1 --2- 172.21.15.101:0/4136274233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4d06e72d0 0x55d4d06f2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.876+0000 7fed1f437b80 1 --2- 172.21.15.101:0/4136274233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d06e7be0 0x55d4d06ef710 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.876+0000 7fed1f437b80 1 -- 172.21.15.101:0/4136274233 >> 172.21.15.101:0/4136274233 conn(0x55d4d06ddda0 msgr2=0x55d4d06e3230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:35.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.877+0000 7fed1f437b80 1 -- 172.21.15.101:0/4136274233 shutdown_connections 2023-07-18T03:09:35.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.877+0000 7fed1f437b80 1 -- 172.21.15.101:0/4136274233 wait complete. 2023-07-18T03:09:35.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.877+0000 7fed1f437b80 20 client.25964 trim_cache size 0 max 16384 2023-07-18T03:09:35.882 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.893+0000 7fed1f437b80 1 Processor -- start 2023-07-18T03:09:35.882 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.893+0000 7fed1f437b80 1 -- start start 2023-07-18T03:09:35.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.893+0000 7fed1f437b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4d06e72d0 0x55d4d06ea9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:35.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.893+0000 7fed1f437b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4d06e7be0 0x55d4d06eaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:35.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.893+0000 7fed1f437b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d06f2ad0 0x55d4d06ec980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:35.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.893+0000 7fed1f437b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4d06f0120 con 0x55d4d06e7be0 2023-07-18T03:09:35.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.893+0000 7fed1f437b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4d06dd970 con 0x55d4d06f2ad0 2023-07-18T03:09:35.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.893+0000 7fed1f437b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4d06edc70 con 0x55d4d06e72d0 2023-07-18T03:09:35.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.894+0000 7fed0a19c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d06f2ad0 0x55d4d06ec980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:35.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.894+0000 7fed0999b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4d06e72d0 0x55d4d06ea9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:35.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.894+0000 7fed0a19c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d06f2ad0 0x55d4d06ec980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:37666/0 (socket says 172.21.15.101:37666) 2023-07-18T03:09:35.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.894+0000 7fed0a19c700 1 -- 172.21.15.101:0/2798258357 learned_addr learned my addr 172.21.15.101:0/2798258357 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:35.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.894+0000 7fed0999b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4d06e72d0 0x55d4d06ea9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:60544/0 (socket says 172.21.15.101:60544) 2023-07-18T03:09:35.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.894+0000 7fed0919a700 1 --2- 172.21.15.101:0/2798258357 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4d06e7be0 0x55d4d06eaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:35.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.894+0000 7fed0999b700 1 -- 172.21.15.101:0/2798258357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d06f2ad0 msgr2=0x55d4d06ec980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:35.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.894+0000 7fed0999b700 1 --2- 172.21.15.101:0/2798258357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d06f2ad0 0x55d4d06ec980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.894+0000 7fed0999b700 1 -- 172.21.15.101:0/2798258357 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4d06e7be0 msgr2=0x55d4d06eaf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:35.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.894+0000 7fed0999b700 1 --2- 172.21.15.101:0/2798258357 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4d06e7be0 0x55d4d06eaf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.894+0000 7fed0999b700 1 -- 172.21.15.101:0/2798258357 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4d0450c60 con 0x55d4d06e72d0 2023-07-18T03:09:35.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.894+0000 7fed0a19c700 1 --2- 172.21.15.101:0/2798258357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d06f2ad0 0x55d4d06ec980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:09:35.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.894+0000 7fed0999b700 1 --2- 172.21.15.101:0/2798258357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4d06e72d0 0x55d4d06ea9f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fed0001c040 tx=0x7fed000058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4dfe6393b8b7461 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:35.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.895+0000 7fecf8bff700 1 -- 172.21.15.101:0/2798258357 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed000052b0 con 0x55d4d06e72d0 2023-07-18T03:09:35.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.895+0000 7fecf8bff700 1 -- 172.21.15.101:0/2798258357 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed00005ce0 con 0x55d4d06e72d0 2023-07-18T03:09:35.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.895+0000 7fecf8bff700 1 -- 172.21.15.101:0/2798258357 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0000ea70 con 0x55d4d06e72d0 2023-07-18T03:09:35.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.895+0000 7fed1f437b80 1 -- 172.21.15.101:0/2798258357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4d06e72d0 msgr2=0x55d4d06ea9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:35.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.895+0000 7fed1f437b80 1 --2- 172.21.15.101:0/2798258357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4d06e72d0 0x55d4d06ea9f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fed0001c040 tx=0x7fed000058e0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.895+0000 7fed1f437b80 1 -- 172.21.15.101:0/2798258357 shutdown_connections 2023-07-18T03:09:35.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.895+0000 7fed1f437b80 1 --2- 172.21.15.101:0/2798258357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d06f2ad0 0x55d4d06ec980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.895+0000 7fed1f437b80 1 --2- 172.21.15.101:0/2798258357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4d06e72d0 0x55d4d06ea9f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.895+0000 7fed1f437b80 1 --2- 172.21.15.101:0/2798258357 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4d06e7be0 0x55d4d06eaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.895+0000 7fed1f437b80 1 -- 172.21.15.101:0/2798258357 >> 172.21.15.101:0/2798258357 conn(0x55d4d06ddda0 msgr2=0x55d4d06f20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:35.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.895+0000 7fed1f437b80 1 -- 172.21.15.101:0/2798258357 shutdown_connections 2023-07-18T03:09:35.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.895+0000 7fed1f437b80 1 -- 172.21.15.101:0/2798258357 wait complete. 2023-07-18T03:09:35.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.896+0000 7fed1f437b80 1 -- start start 2023-07-18T03:09:35.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.896+0000 7fed1f437b80 1 Processor -- start 2023-07-18T03:09:35.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.897+0000 7fed1f437b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d06e72d0 0x55d4d06f9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:35.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.897+0000 7fed1f437b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4d06e7be0 0x55d4d06f7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:35.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.897+0000 7fed1f437b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4d06f2ad0 0x55d4d06f7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:35.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.897+0000 7fed1f437b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4d06f80c0 con 0x55d4d06f2ad0 2023-07-18T03:09:35.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.897+0000 7fed1f437b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4d06f8230 con 0x55d4d06e72d0 2023-07-18T03:09:35.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.897+0000 7fed1f437b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4d06f83f0 con 0x55d4d06e7be0 2023-07-18T03:09:35.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.897+0000 7fed1f437b80 10 client.0 _ll_register_callbacks cb 0x7fecf409aea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:09:35.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.897+0000 7fed0999b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d06e72d0 0x55d4d06f9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:35.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.897+0000 7fed0999b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d06e72d0 0x55d4d06f9310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:37670/0 (socket says 172.21.15.101:37670) 2023-07-18T03:09:35.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.897+0000 7fed0999b700 1 -- 172.21.15.101:0/411835424 learned_addr learned my addr 172.21.15.101:0/411835424 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:35.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.897+0000 7fed0919a700 1 --2- 172.21.15.101:0/411835424 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4d06e7be0 0x55d4d06f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:35.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.897+0000 7fed0a19c700 1 --2- 172.21.15.101:0/411835424 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4d06f2ad0 0x55d4d06f7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:35.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.897+0000 7fed0999b700 1 -- 172.21.15.101:0/411835424 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4d06e7be0 msgr2=0x55d4d06f7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:35.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.897+0000 7fed0999b700 1 --2- 172.21.15.101:0/411835424 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4d06e7be0 0x55d4d06f7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.897+0000 7fed0999b700 1 -- 172.21.15.101:0/411835424 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4d06f2ad0 msgr2=0x55d4d06f7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:35.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.897+0000 7fed0999b700 1 --2- 172.21.15.101:0/411835424 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4d06f2ad0 0x55d4d06f7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.897+0000 7fed0999b700 1 -- 172.21.15.101:0/411835424 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecfc008570 con 0x55d4d06e72d0 2023-07-18T03:09:35.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.897+0000 7fed0a19c700 1 --2- 172.21.15.101:0/411835424 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4d06f2ad0 0x55d4d06f7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:09:35.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.898+0000 7fed0999b700 1 --2- 172.21.15.101:0/411835424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d06e72d0 0x55d4d06f9310 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fed00000c00 tx=0x7fed00005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e69155dc84f665e1 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:35.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.898+0000 7feced7fa700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:09:35.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.898+0000 7feced7fa700 1 -- 172.21.15.101:0/411835424 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed000052b0 con 0x55d4d06e72d0 2023-07-18T03:09:35.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.898+0000 7feced7fa700 1 -- 172.21.15.101:0/411835424 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fed00005410 con 0x55d4d06e72d0 2023-07-18T03:09:35.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.898+0000 7fed1f437b80 1 -- 172.21.15.101:0/411835424 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed00019c50 con 0x55d4d06e72d0 2023-07-18T03:09:35.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.898+0000 7feced7fa700 1 -- 172.21.15.101:0/411835424 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed00019430 con 0x55d4d06e72d0 2023-07-18T03:09:35.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.898+0000 7fed1f437b80 10 client.25970 fetch_fsmap learned FSMap version 25 2023-07-18T03:09:35.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.898+0000 7fed1f437b80 1 -- 172.21.15.101:0/411835424 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d4d0467de0 con 0x55d4d06e72d0 2023-07-18T03:09:35.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.898+0000 7feced7fa700 1 -- 172.21.15.101:0/411835424 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed00019c50 con 0x55d4d06e72d0 2023-07-18T03:09:35.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.899+0000 7feced7fa700 1 -- 172.21.15.101:0/411835424 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fed0003d070 con 0x55d4d06e72d0 2023-07-18T03:09:35.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.899+0000 7fed1f437b80 10 client.25970 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:09:35.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.899+0000 7fed1f437b80 10 client.25970 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:09:35.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.899+0000 7fed1f437b80 20 client.25970 populate_metadata read hostname 'smithi101' 2023-07-18T03:09:35.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.899+0000 7fed1f437b80 1 --2- 172.21.15.101:0/411835424 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55d4d0756d60 0x55d4d0759140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:35.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.899+0000 7fed1f437b80 4 client.25970 mds_command: new command op to 24541 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:09:35.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.899+0000 7fed1f437b80 1 -- 172.21.15.101:0/411835424 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d4d05f5050 con 0x55d4d0756d60 2023-07-18T03:09:35.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.899+0000 7fed0919a700 1 --2- 172.21.15.101:0/411835424 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55d4d0756d60 0x55d4d0759140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:35.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.899+0000 7fed0919a700 1 --2- 172.21.15.101:0/411835424 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55d4d0756d60 0x55d4d0759140 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=383ffbd2805c3f58 server_cookie=9f17990f9027dd5c in_seq=0 out_seq=0 2023-07-18T03:09:35.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.899+0000 7feced7fa700 10 client.25970 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:09:35.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.900+0000 7feced7fa700 1 -- 172.21.15.101:0/411835424 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d4d05f5050 con 0x55d4d0756d60 2023-07-18T03:09:35.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.900+0000 7feced7fa700 10 client.25970 handle_command_reply: tid=0 2023-07-18T03:09:35.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.900+0000 7fed1f437b80 1 client.25970 shutdown 2023-07-18T03:09:35.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.900+0000 7fed1f437b80 1 -- 172.21.15.101:0/411835424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d06e72d0 msgr2=0x55d4d06f9310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:35.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.900+0000 7fed1f437b80 1 --2- 172.21.15.101:0/411835424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d06e72d0 0x55d4d06f9310 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fed00000c00 tx=0x7fed00005e50 comp rx=0 tx=0).stop 2023-07-18T03:09:35.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.900+0000 7fed1f437b80 1 -- 172.21.15.101:0/411835424 shutdown_connections 2023-07-18T03:09:35.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.900+0000 7fed1f437b80 1 --2- 172.21.15.101:0/411835424 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55d4d0756d60 0x55d4d0759140 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.900+0000 7fed1f437b80 1 --2- 172.21.15.101:0/411835424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4d06e72d0 0x55d4d06f9310 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.900+0000 7fed1f437b80 1 --2- 172.21.15.101:0/411835424 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4d06f2ad0 0x55d4d06f7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.900+0000 7fed1f437b80 1 --2- 172.21.15.101:0/411835424 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4d06e7be0 0x55d4d06f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.900+0000 7feced7fa700 0 client.25970 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:09:35.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.900+0000 7fed1f437b80 1 -- 172.21.15.101:0/411835424 >> 172.21.15.101:0/411835424 conn(0x55d4d06ddda0 msgr2=0x55d4d06e93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:35.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.900+0000 7fed1f437b80 1 -- 172.21.15.101:0/411835424 shutdown_connections 2023-07-18T03:09:35.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.900+0000 7fed1f437b80 1 -- 172.21.15.101:0/411835424 wait complete. 2023-07-18T03:09:35.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.900+0000 7fed1f437b80 20 client.25970 trim_cache size 0 max 16384 2023-07-18T03:09:35.899 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:09:35.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.901+0000 7fed0bbff700 1 -- 172.21.15.101:0/1587240353 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fecf00a1340 msgr2=0x7fecf00a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:35.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.901+0000 7fed0bbff700 1 --2- 172.21.15.101:0/1587240353 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fecf00a1340 0x7fecf00a37f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fecfc009ec0 tx=0x7fecfc0092f0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.901+0000 7fed0bbff700 1 -- 172.21.15.101:0/1587240353 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed04100480 msgr2=0x7fed0419bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:35.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.901+0000 7fed0bbff700 1 --2- 172.21.15.101:0/1587240353 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed04100480 0x7fed0419bf10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fecf4005910 tx=0x7fecf4005720 comp rx=0 tx=0).stop 2023-07-18T03:09:35.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.901+0000 7fed0bbff700 1 -- 172.21.15.101:0/1587240353 shutdown_connections 2023-07-18T03:09:35.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.901+0000 7fed0bbff700 1 --2- 172.21.15.101:0/1587240353 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fed04105ae0 0x7fed04195a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.901+0000 7fed0bbff700 1 --2- 172.21.15.101:0/1587240353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed040ffad0 0x7fed0419b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.901+0000 7fed0bbff700 1 --2- 172.21.15.101:0/1587240353 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fecf00a1340 0x7fecf00a37f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.901+0000 7fed0bbff700 1 --2- 172.21.15.101:0/1587240353 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed04100480 0x7fed0419bf10 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:35.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.901+0000 7fed0bbff700 1 -- 172.21.15.101:0/1587240353 >> 172.21.15.101:0/1587240353 conn(0x7fed040fb5f0 msgr2=0x7fed040fbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:35.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.901+0000 7fed0bbff700 1 -- 172.21.15.101:0/1587240353 shutdown_connections 2023-07-18T03:09:35.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:35.901+0000 7fed0bbff700 1 -- 172.21.15.101:0/1587240353 wait complete. 2023-07-18T03:09:35.908 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-18T03:09:36.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.126+0000 7f4c887ff700 1 -- 172.21.15.101:0/1868636005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c800ffad0 msgr2=0x7f4c800fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:36.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.126+0000 7f4c887ff700 1 --2- 172.21.15.101:0/1868636005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c800ffad0 0x7f4c800fff40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4c70009990 tx=0x7f4c70009ca0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.126+0000 7f4c887ff700 1 -- 172.21.15.101:0/1868636005 shutdown_connections 2023-07-18T03:09:36.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.126+0000 7f4c887ff700 1 --2- 172.21.15.101:0/1868636005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c80100480 0x7f4c8010c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.126+0000 7f4c887ff700 1 --2- 172.21.15.101:0/1868636005 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4c80105ae0 0x7f4c80105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.126+0000 7f4c887ff700 1 --2- 172.21.15.101:0/1868636005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c800ffad0 0x7f4c800fff40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.126+0000 7f4c887ff700 1 -- 172.21.15.101:0/1868636005 >> 172.21.15.101:0/1868636005 conn(0x7f4c800fb5f0 msgr2=0x7f4c800fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:36.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.126+0000 7f4c887ff700 1 -- 172.21.15.101:0/1868636005 shutdown_connections 2023-07-18T03:09:36.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.126+0000 7f4c887ff700 1 -- 172.21.15.101:0/1868636005 wait complete. 2023-07-18T03:09:36.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.127+0000 7f4c887ff700 1 Processor -- start 2023-07-18T03:09:36.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.127+0000 7f4c887ff700 1 -- start start 2023-07-18T03:09:36.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.128+0000 7f4c887ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4c800ffad0 0x7f4c80072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:36.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.128+0000 7f4c887ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c80100480 0x7f4c80072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:36.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.128+0000 7f4c887ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c80105ae0 0x7f4c801a3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:36.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.128+0000 7f4c887ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c80073580 con 0x7f4c800ffad0 2023-07-18T03:09:36.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.128+0000 7f4c887ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c801a42f0 con 0x7f4c80105ae0 2023-07-18T03:09:36.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.128+0000 7f4c887ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c801a4430 con 0x7f4c80100480 2023-07-18T03:09:36.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.128+0000 7f4c8659b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4c800ffad0 0x7f4c80072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:36.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.128+0000 7f4c85d9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c80100480 0x7f4c80072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:36.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.128+0000 7f4c85d9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c80100480 0x7f4c80072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:60570/0 (socket says 172.21.15.101:60570) 2023-07-18T03:09:36.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.128+0000 7f4c8659b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4c800ffad0 0x7f4c80072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:43280/0 (socket says 172.21.15.101:43280) 2023-07-18T03:09:36.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.128+0000 7f4c85d9a700 1 -- 172.21.15.101:0/2864802178 learned_addr learned my addr 172.21.15.101:0/2864802178 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:36.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.128+0000 7f4c86d9c700 1 --2- 172.21.15.101:0/2864802178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c80105ae0 0x7f4c801a3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:36.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.128+0000 7f4c85d9a700 1 -- 172.21.15.101:0/2864802178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c80105ae0 msgr2=0x7f4c801a3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:36.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.128+0000 7f4c85d9a700 1 --2- 172.21.15.101:0/2864802178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c80105ae0 0x7f4c801a3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.128+0000 7f4c85d9a700 1 -- 172.21.15.101:0/2864802178 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4c800ffad0 msgr2=0x7f4c80072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:36.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.128+0000 7f4c85d9a700 1 --2- 172.21.15.101:0/2864802178 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4c800ffad0 0x7f4c80072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.129+0000 7f4c85d9a700 1 -- 172.21.15.101:0/2864802178 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c700095f0 con 0x7f4c80100480 2023-07-18T03:09:36.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.129+0000 7f4c8659b700 1 --2- 172.21.15.101:0/2864802178 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4c800ffad0 0x7f4c80072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:09:36.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.129+0000 7f4c85d9a700 1 --2- 172.21.15.101:0/2864802178 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c80100480 0x7f4c80072d50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4c70005910 tx=0x7f4c70004710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:36.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.129+0000 7f4c777fe700 1 -- 172.21.15.101:0/2864802178 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c7001d070 con 0x7f4c80100480 2023-07-18T03:09:36.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.129+0000 7f4c777fe700 1 -- 172.21.15.101:0/2864802178 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c70022470 con 0x7f4c80100480 2023-07-18T03:09:36.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.129+0000 7f4c887ff700 1 -- 172.21.15.101:0/2864802178 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c801a4630 con 0x7f4c80100480 2023-07-18T03:09:36.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.129+0000 7f4c777fe700 1 -- 172.21.15.101:0/2864802178 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c70017650 con 0x7f4c80100480 2023-07-18T03:09:36.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.129+0000 7f4c887ff700 1 -- 172.21.15.101:0/2864802178 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c801a4ab0 con 0x7f4c80100480 2023-07-18T03:09:36.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.132+0000 7f4c777fe700 1 -- 172.21.15.101:0/2864802178 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f4c700225e0 con 0x7f4c80100480 2023-07-18T03:09:36.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.133+0000 7f4c9c08eb80 1 Processor -- start 2023-07-18T03:09:36.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.134+0000 7f4c777fe700 1 --2- 172.21.15.101:0/2864802178 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4c6c0a1340 0x7f4c6c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:36.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.134+0000 7f4c9c08eb80 1 -- start start 2023-07-18T03:09:36.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.134+0000 7f4c777fe700 1 -- 172.21.15.101:0/2864802178 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f4c700ce250 con 0x7f4c80100480 2023-07-18T03:09:36.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.134+0000 7f4c86d9c700 1 --2- 172.21.15.101:0/2864802178 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4c6c0a1340 0x7f4c6c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:36.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.134+0000 7f4c9c08eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d01b6002d0 0x55d01b6006a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:36.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.134+0000 7f4c9c08eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d01b600be0 0x55d01b60b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:36.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.134+0000 7f4c9c08eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d01b60bad0 0x55d01b60deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:36.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.134+0000 7f4c9c08eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d01b506cb0 con 0x55d01b60bad0 2023-07-18T03:09:36.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.134+0000 7f4c9c08eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d01b559a40 con 0x55d01b6002d0 2023-07-18T03:09:36.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.134+0000 7f4c9c08eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d01b552ed0 con 0x55d01b600be0 2023-07-18T03:09:36.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.134+0000 7f4c8659b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d01b6002d0 0x55d01b6006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:36.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.134+0000 7f4c85d9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d01b600be0 0x55d01b60b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:36.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.134+0000 7f4c8659b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d01b6002d0 0x55d01b6006a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:37704/0 (socket says 172.21.15.101:37704) 2023-07-18T03:09:36.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.134+0000 7f4c8659b700 1 -- 172.21.15.101:0/154515506 learned_addr learned my addr 172.21.15.101:0/154515506 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:36.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.135+0000 7f4c8659b700 1 -- 172.21.15.101:0/154515506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d01b600be0 msgr2=0x55d01b60b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:36.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.135+0000 7f4c8659b700 1 --2- 172.21.15.101:0/154515506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d01b600be0 0x55d01b60b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.135+0000 7f4c8659b700 1 -- 172.21.15.101:0/154515506 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d01b60bad0 msgr2=0x55d01b60deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:36.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.135+0000 7f4c86d9c700 1 --2- 172.21.15.101:0/154515506 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d01b60bad0 0x55d01b60deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:36.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.135+0000 7f4c8659b700 1 --2- 172.21.15.101:0/154515506 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d01b60bad0 0x55d01b60deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.135+0000 7f4c8659b700 1 -- 172.21.15.101:0/154515506 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d01b380de0 con 0x55d01b6002d0 2023-07-18T03:09:36.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.135+0000 7f4c85d9a700 1 --2- 172.21.15.101:0/154515506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d01b600be0 0x55d01b60b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:09:36.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.135+0000 7f4c86d9c700 1 --2- 172.21.15.101:0/154515506 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d01b60bad0 0x55d01b60deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:09:36.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.135+0000 7f4c86d9c700 1 --2- 172.21.15.101:0/2864802178 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4c6c0a1340 0x7f4c6c0a37f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4c80074920 tx=0x7f4c78009920 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:36.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.135+0000 7f4c8659b700 1 --2- 172.21.15.101:0/154515506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d01b6002d0 0x55d01b6006a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c0115f0 tx=0x7f4c7c011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b1d2884a22bde0a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:36.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.135+0000 7f4c74ff9700 1 -- 172.21.15.101:0/154515506 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c019630 con 0x55d01b6002d0 2023-07-18T03:09:36.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.135+0000 7f4c74ff9700 1 -- 172.21.15.101:0/154515506 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c7c019c50 con 0x55d01b6002d0 2023-07-18T03:09:36.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.135+0000 7f4c74ff9700 1 -- 172.21.15.101:0/154515506 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c00ec80 con 0x55d01b6002d0 2023-07-18T03:09:36.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.136+0000 7f4c9c08eb80 1 -- 172.21.15.101:0/154515506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d01b6002d0 msgr2=0x55d01b6006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:36.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.136+0000 7f4c9c08eb80 1 --2- 172.21.15.101:0/154515506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d01b6002d0 0x55d01b6006a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c0115f0 tx=0x7f4c7c011900 comp rx=0 tx=0).stop 2023-07-18T03:09:36.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.136+0000 7f4c9c08eb80 1 -- 172.21.15.101:0/154515506 shutdown_connections 2023-07-18T03:09:36.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.136+0000 7f4c9c08eb80 1 --2- 172.21.15.101:0/154515506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d01b6002d0 0x55d01b6006a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.136+0000 7f4c9c08eb80 1 --2- 172.21.15.101:0/154515506 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d01b60bad0 0x55d01b60deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.136+0000 7f4c9c08eb80 1 --2- 172.21.15.101:0/154515506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d01b600be0 0x55d01b60b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.136+0000 7f4c9c08eb80 1 -- 172.21.15.101:0/154515506 >> 172.21.15.101:0/154515506 conn(0x55d01b5f6da0 msgr2=0x55d01b5fc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:36.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.136+0000 7f4c9c08eb80 1 -- 172.21.15.101:0/154515506 shutdown_connections 2023-07-18T03:09:36.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.136+0000 7f4c9c08eb80 1 -- 172.21.15.101:0/154515506 wait complete. 2023-07-18T03:09:36.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.137+0000 7f4c9c08eb80 1 -- start start 2023-07-18T03:09:36.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.137+0000 7f4c9c08eb80 1 Processor -- start 2023-07-18T03:09:36.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.137+0000 7f4c9c08eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d01b6002d0 0x55d01b60b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:36.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.137+0000 7f4c9c08eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d01b600be0 0x55d01b608800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:36.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.137+0000 7f4c9c08eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d01b60bad0 0x55d01b608f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:36.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.137+0000 7f4c9c08eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d01b4f2ac0 con 0x55d01b60bad0 2023-07-18T03:09:36.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.137+0000 7f4c9c08eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d01b4f27c0 con 0x55d01b600be0 2023-07-18T03:09:36.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.137+0000 7f4c9c08eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d01b3d6920 con 0x55d01b6002d0 2023-07-18T03:09:36.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.137+0000 7f4c9c08eb80 10 client.0 _ll_register_callbacks cb 0x55d01b4f0760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:09:36.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.138+0000 7f4c8659b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d01b6002d0 0x55d01b60b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:36.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.138+0000 7f4c8659b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d01b6002d0 0x55d01b60b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:60592/0 (socket says 172.21.15.101:60592) 2023-07-18T03:09:36.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.138+0000 7f4c8659b700 1 -- 172.21.15.101:0/2446424909 learned_addr learned my addr 172.21.15.101:0/2446424909 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:36.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.138+0000 7f4c86d9c700 1 --2- 172.21.15.101:0/2446424909 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d01b60bad0 0x55d01b608f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:36.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.138+0000 7f4c85d9a700 1 --2- 172.21.15.101:0/2446424909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d01b600be0 0x55d01b608800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:36.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.138+0000 7f4c8659b700 1 -- 172.21.15.101:0/2446424909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d01b600be0 msgr2=0x55d01b608800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:36.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.138+0000 7f4c8659b700 1 --2- 172.21.15.101:0/2446424909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d01b600be0 0x55d01b608800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.138+0000 7f4c8659b700 1 -- 172.21.15.101:0/2446424909 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d01b60bad0 msgr2=0x55d01b608f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:36.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.138+0000 7f4c8659b700 1 --2- 172.21.15.101:0/2446424909 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d01b60bad0 0x55d01b608f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.138+0000 7f4c8659b700 1 -- 172.21.15.101:0/2446424909 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c801a4ab0 con 0x55d01b6002d0 2023-07-18T03:09:36.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.138+0000 7f4c85d9a700 1 --2- 172.21.15.101:0/2446424909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d01b600be0 0x55d01b608800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:09:36.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.138+0000 7f4c8659b700 1 --2- 172.21.15.101:0/2446424909 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d01b6002d0 0x55d01b60b510 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c011c50 tx=0x7f4c7c00eb00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4f14e3b48ccdd16 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:36.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.138+0000 7f4c69ffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:09:36.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.139+0000 7f4c69ffb700 1 -- 172.21.15.101:0/2446424909 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c019630 con 0x55d01b6002d0 2023-07-18T03:09:36.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.139+0000 7f4c9c08eb80 1 -- 172.21.15.101:0/2446424909 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d01b4ac4f0 con 0x55d01b6002d0 2023-07-18T03:09:36.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.139+0000 7f4c69ffb700 1 -- 172.21.15.101:0/2446424909 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f4c7c005ce0 con 0x55d01b6002d0 2023-07-18T03:09:36.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.139+0000 7f4c69ffb700 1 -- 172.21.15.101:0/2446424909 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c7c019790 con 0x55d01b6002d0 2023-07-18T03:09:36.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.139+0000 7f4c9c08eb80 10 client.26008 fetch_fsmap learned FSMap version 25 2023-07-18T03:09:36.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.139+0000 7f4c9c08eb80 1 -- 172.21.15.101:0/2446424909 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f4c7c009d50 con 0x55d01b6002d0 2023-07-18T03:09:36.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.139+0000 7f4c69ffb700 1 -- 172.21.15.101:0/2446424909 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d01b4ac4f0 con 0x55d01b6002d0 2023-07-18T03:09:36.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.139+0000 7f4c69ffb700 1 -- 172.21.15.101:0/2446424909 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f4c7c029020 con 0x55d01b6002d0 2023-07-18T03:09:36.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.139+0000 7f4c9c08eb80 10 client.26008 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:09:36.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.139+0000 7f4c9c08eb80 10 client.26008 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:09:36.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.139+0000 7f4c9c08eb80 20 client.26008 populate_metadata read hostname 'smithi101' 2023-07-18T03:09:36.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.139+0000 7f4c9c08eb80 1 --2- 172.21.15.101:0/2446424909 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55d01b61b280 0x55d01b61d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:36.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.139+0000 7f4c9c08eb80 4 client.26008 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:09:36.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.139+0000 7f4c9c08eb80 1 -- 172.21.15.101:0/2446424909 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d01b54beb0 con 0x55d01b61b280 2023-07-18T03:09:36.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.140+0000 7f4c85d9a700 1 --2- 172.21.15.101:0/2446424909 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55d01b61b280 0x55d01b61d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:36.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.140+0000 7f4c69ffb700 10 client.26008 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:09:36.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.140+0000 7f4c85d9a700 1 --2- 172.21.15.101:0/2446424909 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55d01b61b280 0x55d01b61d690 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6ddb14e85d08bda3 server_cookie=321fe75c509decf4 in_seq=0 out_seq=0 2023-07-18T03:09:36.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.141+0000 7f4c69ffb700 1 -- 172.21.15.101:0/2446424909 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d01b54beb0 con 0x55d01b61b280 2023-07-18T03:09:36.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.141+0000 7f4c69ffb700 10 client.26008 handle_command_reply: tid=0 2023-07-18T03:09:36.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.141+0000 7f4c9c08eb80 1 client.26008 shutdown 2023-07-18T03:09:36.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.141+0000 7f4c9c08eb80 1 -- 172.21.15.101:0/2446424909 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d01b6002d0 msgr2=0x55d01b60b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:36.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.141+0000 7f4c9c08eb80 1 --2- 172.21.15.101:0/2446424909 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d01b6002d0 0x55d01b60b510 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c011c50 tx=0x7f4c7c00eb00 comp rx=0 tx=0).stop 2023-07-18T03:09:36.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.141+0000 7f4c9c08eb80 1 -- 172.21.15.101:0/2446424909 shutdown_connections 2023-07-18T03:09:36.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.141+0000 7f4c9c08eb80 1 --2- 172.21.15.101:0/2446424909 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55d01b61b280 0x55d01b61d690 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.141+0000 7f4c9c08eb80 1 --2- 172.21.15.101:0/2446424909 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d01b60bad0 0x55d01b608f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.141+0000 7f4c9c08eb80 1 --2- 172.21.15.101:0/2446424909 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d01b6002d0 0x55d01b60b510 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.141+0000 7f4c9c08eb80 1 --2- 172.21.15.101:0/2446424909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d01b600be0 0x55d01b608800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.141+0000 7f4c69ffb700 0 client.26008 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:09:36.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.141+0000 7f4c9c08eb80 1 -- 172.21.15.101:0/2446424909 >> 172.21.15.101:0/2446424909 conn(0x55d01b5f6da0 msgr2=0x55d01b5fc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:36.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.142+0000 7f4c9c08eb80 1 -- 172.21.15.101:0/2446424909 shutdown_connections 2023-07-18T03:09:36.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.142+0000 7f4c9c08eb80 1 -- 172.21.15.101:0/2446424909 wait complete. 2023-07-18T03:09:36.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.142+0000 7f4c9c08eb80 20 client.26008 trim_cache size 0 max 16384 2023-07-18T03:09:36.156 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.155+0000 7f4c9c08eb80 1 Processor -- start 2023-07-18T03:09:36.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.155+0000 7f4c9c08eb80 1 -- start start 2023-07-18T03:09:36.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.156+0000 7f4c9c08eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d01b6002d0 0x55d01b60b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:36.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.156+0000 7f4c9c08eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d01b600be0 0x55d01b607fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:36.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.156+0000 7f4c9c08eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d01b60bad0 0x55d01b608500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:36.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.156+0000 7f4c9c08eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d01b609de0 con 0x55d01b6002d0 2023-07-18T03:09:36.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.156+0000 7f4c9c08eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d01b5f3fb0 con 0x55d01b60bad0 2023-07-18T03:09:36.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.156+0000 7f4c9c08eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d01b5f69a0 con 0x55d01b600be0 2023-07-18T03:09:36.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.156+0000 7f4c85d9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d01b600be0 0x55d01b607fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:36.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.156+0000 7f4c8659b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d01b6002d0 0x55d01b60b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:36.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.156+0000 7f4c85d9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d01b600be0 0x55d01b607fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:60606/0 (socket says 172.21.15.101:60606) 2023-07-18T03:09:36.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.156+0000 7f4c85d9a700 1 -- 172.21.15.101:0/2747149772 learned_addr learned my addr 172.21.15.101:0/2747149772 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:36.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.156+0000 7f4c86d9c700 1 --2- 172.21.15.101:0/2747149772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d01b60bad0 0x55d01b608500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:36.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.156+0000 7f4c8659b700 1 -- 172.21.15.101:0/2747149772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d01b600be0 msgr2=0x55d01b607fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:36.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.156+0000 7f4c8659b700 1 --2- 172.21.15.101:0/2747149772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d01b600be0 0x55d01b607fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.156+0000 7f4c8659b700 1 -- 172.21.15.101:0/2747149772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d01b60bad0 msgr2=0x55d01b608500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:36.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.156+0000 7f4c8659b700 1 --2- 172.21.15.101:0/2747149772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d01b60bad0 0x55d01b608500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.156+0000 7f4c8659b700 1 -- 172.21.15.101:0/2747149772 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d01b369c60 con 0x55d01b6002d0 2023-07-18T03:09:36.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.156+0000 7f4c86d9c700 1 --2- 172.21.15.101:0/2747149772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d01b60bad0 0x55d01b608500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:09:36.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.157+0000 7f4c8659b700 1 --2- 172.21.15.101:0/2747149772 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d01b6002d0 0x55d01b60b0b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c00e750 tx=0x7f4c7c011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805063dbba587f1c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:36.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.157+0000 7f4c757fa700 1 -- 172.21.15.101:0/2747149772 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c00ec20 con 0x55d01b6002d0 2023-07-18T03:09:36.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.157+0000 7f4c757fa700 1 -- 172.21.15.101:0/2747149772 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c7c005ce0 con 0x55d01b6002d0 2023-07-18T03:09:36.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.157+0000 7f4c757fa700 1 -- 172.21.15.101:0/2747149772 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c004570 con 0x55d01b6002d0 2023-07-18T03:09:36.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.157+0000 7f4c9c08eb80 1 -- 172.21.15.101:0/2747149772 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d01b6002d0 msgr2=0x55d01b60b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:36.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.157+0000 7f4c9c08eb80 1 --2- 172.21.15.101:0/2747149772 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d01b6002d0 0x55d01b60b0b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c00e750 tx=0x7f4c7c011900 comp rx=0 tx=0).stop 2023-07-18T03:09:36.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.158+0000 7f4c9c08eb80 1 -- 172.21.15.101:0/2747149772 shutdown_connections 2023-07-18T03:09:36.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.158+0000 7f4c9c08eb80 1 --2- 172.21.15.101:0/2747149772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d01b60bad0 0x55d01b608500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.158+0000 7f4c9c08eb80 1 --2- 172.21.15.101:0/2747149772 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d01b6002d0 0x55d01b60b0b0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.158+0000 7f4c9c08eb80 1 --2- 172.21.15.101:0/2747149772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d01b600be0 0x55d01b607fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.158+0000 7f4c9c08eb80 1 -- 172.21.15.101:0/2747149772 >> 172.21.15.101:0/2747149772 conn(0x55d01b5f6da0 msgr2=0x55d01b603b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:36.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.158+0000 7f4c9c08eb80 1 -- 172.21.15.101:0/2747149772 shutdown_connections 2023-07-18T03:09:36.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.158+0000 7f4c9c08eb80 1 -- 172.21.15.101:0/2747149772 wait complete. 2023-07-18T03:09:36.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.158+0000 7f4c9c08eb80 1 -- start start 2023-07-18T03:09:36.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.158+0000 7f4c9c08eb80 1 Processor -- start 2023-07-18T03:09:36.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.159+0000 7f4c9c08eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d01b6002d0 0x55d01b6116a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:36.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.159+0000 7f4c9c08eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d01b600be0 0x55d01b611be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:36.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.159+0000 7f4c9c08eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d01b60bad0 0x55d01b66e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:36.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.159+0000 7f4c9c08eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d01b612350 con 0x55d01b600be0 2023-07-18T03:09:36.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.159+0000 7f4c9c08eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d01b6124c0 con 0x55d01b60bad0 2023-07-18T03:09:36.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.159+0000 7f4c9c08eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d01b66e6a0 con 0x55d01b6002d0 2023-07-18T03:09:36.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.159+0000 7f4c9c08eb80 10 client.0 _ll_register_callbacks cb 0x7f4c70009290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:09:36.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.159+0000 7f4c85d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d01b600be0 0x55d01b611be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:36.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.159+0000 7f4c8659b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d01b6002d0 0x55d01b6116a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:36.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.159+0000 7f4c85d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d01b600be0 0x55d01b611be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:43296/0 (socket says 172.21.15.101:43296) 2023-07-18T03:09:36.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.159+0000 7f4c85d9a700 1 -- 172.21.15.101:0/2316837270 learned_addr learned my addr 172.21.15.101:0/2316837270 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:36.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.159+0000 7f4c86d9c700 1 --2- 172.21.15.101:0/2316837270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d01b60bad0 0x55d01b66e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:36.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.159+0000 7f4c8659b700 1 -- 172.21.15.101:0/2316837270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d01b60bad0 msgr2=0x55d01b66e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:36.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.159+0000 7f4c8659b700 1 --2- 172.21.15.101:0/2316837270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d01b60bad0 0x55d01b66e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.159+0000 7f4c8659b700 1 -- 172.21.15.101:0/2316837270 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d01b600be0 msgr2=0x55d01b611be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:36.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.159+0000 7f4c8659b700 1 --2- 172.21.15.101:0/2316837270 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d01b600be0 0x55d01b611be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.159+0000 7f4c8659b700 1 -- 172.21.15.101:0/2316837270 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c7c009d50 con 0x55d01b6002d0 2023-07-18T03:09:36.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.160+0000 7f4c8659b700 1 --2- 172.21.15.101:0/2316837270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d01b6002d0 0x55d01b6116a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c00a010 tx=0x7f4c7c005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7adc8b2fdefb9225 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:36.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.160+0000 7f4c69ffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:09:36.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.160+0000 7f4c69ffb700 1 -- 172.21.15.101:0/2316837270 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c0046c0 con 0x55d01b6002d0 2023-07-18T03:09:36.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.160+0000 7f4c9c08eb80 1 -- 172.21.15.101:0/2316837270 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4c7c019c50 con 0x55d01b6002d0 2023-07-18T03:09:36.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.160+0000 7f4c69ffb700 1 -- 172.21.15.101:0/2316837270 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f4c7c000e40 con 0x55d01b6002d0 2023-07-18T03:09:36.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.160+0000 7f4c69ffb700 1 -- 172.21.15.101:0/2316837270 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c7c019430 con 0x55d01b6002d0 2023-07-18T03:09:36.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.160+0000 7f4c9c08eb80 10 client.26014 fetch_fsmap learned FSMap version 25 2023-07-18T03:09:36.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.160+0000 7f4c9c08eb80 1 -- 172.21.15.101:0/2316837270 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d01b3a7250 con 0x55d01b6002d0 2023-07-18T03:09:36.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.161+0000 7f4c69ffb700 1 -- 172.21.15.101:0/2316837270 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c7c019c50 con 0x55d01b6002d0 2023-07-18T03:09:36.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.161+0000 7f4c69ffb700 1 -- 172.21.15.101:0/2316837270 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f4c7c029070 con 0x55d01b6002d0 2023-07-18T03:09:36.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.161+0000 7f4c9c08eb80 10 client.26014 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:09:36.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.161+0000 7f4c9c08eb80 10 client.26014 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:09:36.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.161+0000 7f4c9c08eb80 20 client.26014 populate_metadata read hostname 'smithi101' 2023-07-18T03:09:36.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.161+0000 7f4c9c08eb80 1 --2- 172.21.15.101:0/2316837270 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55d01b66f9e0 0x55d01b671df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:36.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.161+0000 7f4c9c08eb80 4 client.26014 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:09:36.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.161+0000 7f4c9c08eb80 1 -- 172.21.15.101:0/2316837270 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d01b50e050 con 0x55d01b66f9e0 2023-07-18T03:09:36.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.161+0000 7f4c85d9a700 1 --2- 172.21.15.101:0/2316837270 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55d01b66f9e0 0x55d01b671df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:36.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.161+0000 7f4c85d9a700 1 --2- 172.21.15.101:0/2316837270 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55d01b66f9e0 0x55d01b671df0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=90de87043f6cb4f2 server_cookie=c6344b0e0d8937d8 in_seq=0 out_seq=0 2023-07-18T03:09:36.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.161+0000 7f4c69ffb700 10 client.26014 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:09:36.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.162+0000 7f4c69ffb700 1 -- 172.21.15.101:0/2316837270 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d01b50e050 con 0x55d01b66f9e0 2023-07-18T03:09:36.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.162+0000 7f4c69ffb700 10 client.26014 handle_command_reply: tid=0 2023-07-18T03:09:36.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.162+0000 7f4c9c08eb80 1 client.26014 shutdown 2023-07-18T03:09:36.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.162+0000 7f4c9c08eb80 1 -- 172.21.15.101:0/2316837270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d01b6002d0 msgr2=0x55d01b6116a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:36.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.162+0000 7f4c9c08eb80 1 --2- 172.21.15.101:0/2316837270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d01b6002d0 0x55d01b6116a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c00a010 tx=0x7f4c7c005e50 comp rx=0 tx=0).stop 2023-07-18T03:09:36.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.162+0000 7f4c9c08eb80 1 -- 172.21.15.101:0/2316837270 shutdown_connections 2023-07-18T03:09:36.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.162+0000 7f4c9c08eb80 1 --2- 172.21.15.101:0/2316837270 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55d01b66f9e0 0x55d01b671df0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.162+0000 7f4c9c08eb80 1 --2- 172.21.15.101:0/2316837270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d01b60bad0 0x55d01b66e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.162+0000 7f4c69ffb700 0 client.26014 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:09:36.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.162+0000 7f4c9c08eb80 1 --2- 172.21.15.101:0/2316837270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d01b6002d0 0x55d01b6116a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.162+0000 7f4c9c08eb80 1 --2- 172.21.15.101:0/2316837270 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d01b600be0 0x55d01b611be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.162+0000 7f4c9c08eb80 1 -- 172.21.15.101:0/2316837270 >> 172.21.15.101:0/2316837270 conn(0x55d01b5f6da0 msgr2=0x55d01b5fb8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:36.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.162+0000 7f4c9c08eb80 1 -- 172.21.15.101:0/2316837270 shutdown_connections 2023-07-18T03:09:36.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.162+0000 7f4c9c08eb80 1 -- 172.21.15.101:0/2316837270 wait complete. 2023-07-18T03:09:36.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.162+0000 7f4c9c08eb80 20 client.26014 trim_cache size 0 max 16384 2023-07-18T03:09:36.170 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:09:36.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.163+0000 7f4c887ff700 1 -- 172.21.15.101:0/2864802178 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4c6c0a1340 msgr2=0x7f4c6c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:36.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.163+0000 7f4c887ff700 1 --2- 172.21.15.101:0/2864802178 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4c6c0a1340 0x7f4c6c0a37f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4c80074920 tx=0x7f4c78009920 comp rx=0 tx=0).stop 2023-07-18T03:09:36.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.163+0000 7f4c887ff700 1 -- 172.21.15.101:0/2864802178 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c80100480 msgr2=0x7f4c80072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:36.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.163+0000 7f4c887ff700 1 --2- 172.21.15.101:0/2864802178 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c80100480 0x7f4c80072d50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4c70005910 tx=0x7f4c70004710 comp rx=0 tx=0).stop 2023-07-18T03:09:36.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.163+0000 7f4c887ff700 1 -- 172.21.15.101:0/2864802178 shutdown_connections 2023-07-18T03:09:36.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.163+0000 7f4c887ff700 1 --2- 172.21.15.101:0/2864802178 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4c6c0a1340 0x7f4c6c0a37f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.163+0000 7f4c887ff700 1 --2- 172.21.15.101:0/2864802178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c80105ae0 0x7f4c801a3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.163+0000 7f4c887ff700 1 --2- 172.21.15.101:0/2864802178 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4c800ffad0 0x7f4c80072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.163+0000 7f4c887ff700 1 --2- 172.21.15.101:0/2864802178 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c80100480 0x7f4c80072d50 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.163+0000 7f4c887ff700 1 -- 172.21.15.101:0/2864802178 >> 172.21.15.101:0/2864802178 conn(0x7f4c800fb5f0 msgr2=0x7f4c801035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:36.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.163+0000 7f4c887ff700 1 -- 172.21.15.101:0/2864802178 shutdown_connections 2023-07-18T03:09:36.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.163+0000 7f4c887ff700 1 -- 172.21.15.101:0/2864802178 wait complete. 2023-07-18T03:09:36.173 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:35 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1731436417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:09:36.174 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-18T03:09:36.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:35 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1731436417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:09:36.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:35 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1731436417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:09:36.378 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.389+0000 7f6d3ddff700 1 -- 172.21.15.101:0/2596526595 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d380ffae0 msgr2=0x7f6d380fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:36.379 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.389+0000 7f6d3ddff700 1 --2- 172.21.15.101:0/2596526595 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d380ffae0 0x7f6d380fff50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c009990 tx=0x7f6d2c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.379 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.389+0000 7f6d3ddff700 1 -- 172.21.15.101:0/2596526595 shutdown_connections 2023-07-18T03:09:36.379 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.389+0000 7f6d3ddff700 1 --2- 172.21.15.101:0/2596526595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d38100490 0x7f6d3810c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.379 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.389+0000 7f6d3ddff700 1 --2- 172.21.15.101:0/2596526595 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6d38105af0 0x7f6d38105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.379 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.389+0000 7f6d3ddff700 1 --2- 172.21.15.101:0/2596526595 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d380ffae0 0x7f6d380fff50 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.389+0000 7f6d3ddff700 1 -- 172.21.15.101:0/2596526595 >> 172.21.15.101:0/2596526595 conn(0x7f6d380fb5e0 msgr2=0x7f6d380fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:36.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.389+0000 7f6d3ddff700 1 -- 172.21.15.101:0/2596526595 shutdown_connections 2023-07-18T03:09:36.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.389+0000 7f6d3ddff700 1 -- 172.21.15.101:0/2596526595 wait complete. 2023-07-18T03:09:36.382 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.390+0000 7f6d3ddff700 1 Processor -- start 2023-07-18T03:09:36.382 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.391+0000 7f6d3ddff700 1 -- start start 2023-07-18T03:09:36.382 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.391+0000 7f6d3ddff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d380ffae0 0x7f6d38072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:36.382 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.391+0000 7f6d3ddff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d38100490 0x7f6d38072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:36.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.391+0000 7f6d3ddff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6d38105af0 0x7f6d381a3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:36.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.391+0000 7f6d3ddff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d38073580 con 0x7f6d38105af0 2023-07-18T03:09:36.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.391+0000 7f6d3ddff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d381a4320 con 0x7f6d38100490 2023-07-18T03:09:36.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.391+0000 7f6d3ddff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d381a4460 con 0x7f6d380ffae0 2023-07-18T03:09:36.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.391+0000 7f6d377fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d380ffae0 0x7f6d38072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:36.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.391+0000 7f6d377fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d380ffae0 0x7f6d38072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:60628/0 (socket says 172.21.15.101:60628) 2023-07-18T03:09:36.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.391+0000 7f6d377fe700 1 -- 172.21.15.101:0/2343525300 learned_addr learned my addr 172.21.15.101:0/2343525300 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:36.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.391+0000 7f6d37fff700 1 --2- 172.21.15.101:0/2343525300 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6d38105af0 0x7f6d381a3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:36.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.391+0000 7f6d36ffd700 1 --2- 172.21.15.101:0/2343525300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d38100490 0x7f6d38072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:36.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.391+0000 7f6d377fe700 1 -- 172.21.15.101:0/2343525300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d38100490 msgr2=0x7f6d38072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:36.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.391+0000 7f6d377fe700 1 --2- 172.21.15.101:0/2343525300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d38100490 0x7f6d38072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.391+0000 7f6d377fe700 1 -- 172.21.15.101:0/2343525300 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6d38105af0 msgr2=0x7f6d381a3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:36.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.391+0000 7f6d377fe700 1 --2- 172.21.15.101:0/2343525300 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6d38105af0 0x7f6d381a3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.392+0000 7f6d377fe700 1 -- 172.21.15.101:0/2343525300 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d2c0095f0 con 0x7f6d380ffae0 2023-07-18T03:09:36.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.392+0000 7f6d377fe700 1 --2- 172.21.15.101:0/2343525300 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d380ffae0 0x7f6d38072810 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6d2800e9a0 tx=0x7f6d2800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:36.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.392+0000 7f6d34ff9700 1 -- 172.21.15.101:0/2343525300 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d28009790 con 0x7f6d380ffae0 2023-07-18T03:09:36.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.392+0000 7f6d34ff9700 1 -- 172.21.15.101:0/2343525300 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d2800cbd0 con 0x7f6d380ffae0 2023-07-18T03:09:36.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.392+0000 7f6d34ff9700 1 -- 172.21.15.101:0/2343525300 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d28018680 con 0x7f6d380ffae0 2023-07-18T03:09:36.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.392+0000 7f6d3ddff700 1 -- 172.21.15.101:0/2343525300 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d381a46c0 con 0x7f6d380ffae0 2023-07-18T03:09:36.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.392+0000 7f6d3ddff700 1 -- 172.21.15.101:0/2343525300 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d381a4ba0 con 0x7f6d380ffae0 2023-07-18T03:09:36.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.395+0000 7f6d34ff9700 1 -- 172.21.15.101:0/2343525300 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f6d2800cd40 con 0x7f6d380ffae0 2023-07-18T03:09:36.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.396+0000 7f6d5166bb80 1 Processor -- start 2023-07-18T03:09:36.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.397+0000 7f6d5166bb80 1 -- start start 2023-07-18T03:09:36.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.397+0000 7f6d34ff9700 1 --2- 172.21.15.101:0/2343525300 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6d200a1340 0x7f6d200a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:36.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.397+0000 7f6d34ff9700 1 -- 172.21.15.101:0/2343525300 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f6d28014070 con 0x7f6d380ffae0 2023-07-18T03:09:36.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.397+0000 7f6d5166bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572087eb2d0 0x5572087eb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:36.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.397+0000 7f6d37fff700 1 --2- 172.21.15.101:0/2343525300 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6d200a1340 0x7f6d200a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:36.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.397+0000 7f6d5166bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572087ebbe0 0x5572087f6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:36.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.397+0000 7f6d5166bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5572087f6ad0 0x5572087f8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:36.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.397+0000 7f6d5166bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572086f1cb0 con 0x5572087f6ad0 2023-07-18T03:09:36.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.397+0000 7f6d5166bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557208744a40 con 0x5572087eb2d0 2023-07-18T03:09:36.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.397+0000 7f6d5166bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55720873ded0 con 0x5572087ebbe0 2023-07-18T03:09:36.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.397+0000 7f6d377fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572087eb2d0 0x5572087eb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:36.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.397+0000 7f6d377fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572087eb2d0 0x5572087eb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:37746/0 (socket says 172.21.15.101:37746) 2023-07-18T03:09:36.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.397+0000 7f6d377fe700 1 -- 172.21.15.101:0/3258652754 learned_addr learned my addr 172.21.15.101:0/3258652754 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:36.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.397+0000 7f6d36ffd700 1 --2- 172.21.15.101:0/3258652754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572087ebbe0 0x5572087f6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:36.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.398+0000 7f6d37fff700 1 --2- 172.21.15.101:0/3258652754 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5572087f6ad0 0x5572087f8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:36.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.398+0000 7f6d377fe700 1 -- 172.21.15.101:0/3258652754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572087ebbe0 msgr2=0x5572087f6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:36.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.398+0000 7f6d377fe700 1 --2- 172.21.15.101:0/3258652754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572087ebbe0 0x5572087f6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.398+0000 7f6d377fe700 1 -- 172.21.15.101:0/3258652754 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5572087f6ad0 msgr2=0x5572087f8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:36.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.398+0000 7f6d377fe700 1 --2- 172.21.15.101:0/3258652754 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5572087f6ad0 0x5572087f8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.398+0000 7f6d377fe700 1 -- 172.21.15.101:0/3258652754 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55720856bde0 con 0x5572087eb2d0 2023-07-18T03:09:36.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.398+0000 7f6d36ffd700 1 --2- 172.21.15.101:0/3258652754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572087ebbe0 0x5572087f6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:09:36.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.398+0000 7f6d37fff700 1 --2- 172.21.15.101:0/3258652754 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5572087f6ad0 0x5572087f8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:09:36.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.398+0000 7f6d377fe700 1 --2- 172.21.15.101:0/3258652754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572087eb2d0 0x5572087eb6a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6d280969e0 tx=0x7f6d28099f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d010937ea047c3fa server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:36.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.398+0000 7f6d37fff700 1 --2- 172.21.15.101:0/2343525300 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6d200a1340 0x7f6d200a37f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6d38074920 tx=0x7f6d30009920 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:36.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.398+0000 7f6d1dffb700 1 -- 172.21.15.101:0/3258652754 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2809a1d0 con 0x5572087eb2d0 2023-07-18T03:09:36.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.398+0000 7f6d1dffb700 1 -- 172.21.15.101:0/3258652754 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d2809a330 con 0x5572087eb2d0 2023-07-18T03:09:36.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.398+0000 7f6d1dffb700 1 -- 172.21.15.101:0/3258652754 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d28094b80 con 0x5572087eb2d0 2023-07-18T03:09:36.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.399+0000 7f6d5166bb80 1 -- 172.21.15.101:0/3258652754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572087eb2d0 msgr2=0x5572087eb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:36.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.399+0000 7f6d5166bb80 1 --2- 172.21.15.101:0/3258652754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572087eb2d0 0x5572087eb6a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6d280969e0 tx=0x7f6d28099f70 comp rx=0 tx=0).stop 2023-07-18T03:09:36.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.399+0000 7f6d5166bb80 1 -- 172.21.15.101:0/3258652754 shutdown_connections 2023-07-18T03:09:36.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.399+0000 7f6d5166bb80 1 --2- 172.21.15.101:0/3258652754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572087eb2d0 0x5572087eb6a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.394 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.399+0000 7f6d5166bb80 1 --2- 172.21.15.101:0/3258652754 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5572087f6ad0 0x5572087f8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.394 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.399+0000 7f6d5166bb80 1 --2- 172.21.15.101:0/3258652754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572087ebbe0 0x5572087f6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.394 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.399+0000 7f6d5166bb80 1 -- 172.21.15.101:0/3258652754 >> 172.21.15.101:0/3258652754 conn(0x5572087e1da0 msgr2=0x5572087e76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:36.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.399+0000 7f6d5166bb80 1 -- 172.21.15.101:0/3258652754 shutdown_connections 2023-07-18T03:09:36.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.399+0000 7f6d5166bb80 1 -- 172.21.15.101:0/3258652754 wait complete. 2023-07-18T03:09:36.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.400+0000 7f6d5166bb80 1 -- start start 2023-07-18T03:09:36.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.400+0000 7f6d5166bb80 1 Processor -- start 2023-07-18T03:09:36.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.400+0000 7f6d5166bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5572087eb2d0 0x5572087f3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:36.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.400+0000 7f6d5166bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572087ebbe0 0x5572087f3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:36.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.400+0000 7f6d5166bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572087f6ad0 0x5572087f4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:36.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.400+0000 7f6d5166bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572086ddac0 con 0x5572087eb2d0 2023-07-18T03:09:36.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.400+0000 7f6d5166bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572086dd7c0 con 0x5572087f6ad0 2023-07-18T03:09:36.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.400+0000 7f6d5166bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572085c1920 con 0x5572087ebbe0 2023-07-18T03:09:36.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.400+0000 7f6d5166bb80 10 client.0 _ll_register_callbacks cb 0x5572086db760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:09:36.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.400+0000 7f6d377fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5572087eb2d0 0x5572087f3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:36.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.401+0000 7f6d37fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572087f6ad0 0x5572087f4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:36.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.401+0000 7f6d36ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572087ebbe0 0x5572087f3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:36.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.401+0000 7f6d37fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572087f6ad0 0x5572087f4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:37762/0 (socket says 172.21.15.101:37762) 2023-07-18T03:09:36.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.401+0000 7f6d36ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572087ebbe0 0x5572087f3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:60644/0 (socket says 172.21.15.101:60644) 2023-07-18T03:09:36.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.401+0000 7f6d377fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5572087eb2d0 0x5572087f3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:43334/0 (socket says 172.21.15.101:43334) 2023-07-18T03:09:36.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.401+0000 7f6d37fff700 1 -- 172.21.15.101:0/2235380616 learned_addr learned my addr 172.21.15.101:0/2235380616 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:36.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.401+0000 7f6d37fff700 1 -- 172.21.15.101:0/2235380616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572087ebbe0 msgr2=0x5572087f3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:36.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.401+0000 7f6d37fff700 1 --2- 172.21.15.101:0/2235380616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572087ebbe0 0x5572087f3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.401+0000 7f6d37fff700 1 -- 172.21.15.101:0/2235380616 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5572087eb2d0 msgr2=0x5572087f3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:36.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.401+0000 7f6d37fff700 1 --2- 172.21.15.101:0/2235380616 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5572087eb2d0 0x5572087f3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.401+0000 7f6d37fff700 1 -- 172.21.15.101:0/2235380616 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d381a4ba0 con 0x5572087f6ad0 2023-07-18T03:09:36.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.401+0000 7f6d36ffd700 1 --2- 172.21.15.101:0/2235380616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572087ebbe0 0x5572087f3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:09:36.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.401+0000 7f6d377fe700 1 --2- 172.21.15.101:0/2235380616 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5572087eb2d0 0x5572087f3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:09:36.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.401+0000 7f6d37fff700 1 --2- 172.21.15.101:0/2235380616 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572087f6ad0 0x5572087f4300 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6d3001ae60 tx=0x7f6d30004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7633399e617d3f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:36.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.401+0000 7f6d16ffd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:09:36.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.402+0000 7f6d16ffd700 1 -- 172.21.15.101:0/2235380616 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3000e040 con 0x5572087f6ad0 2023-07-18T03:09:36.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.402+0000 7f6d16ffd700 1 -- 172.21.15.101:0/2235380616 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f6d3000f2d0 con 0x5572087f6ad0 2023-07-18T03:09:36.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.402+0000 7f6d5166bb80 1 -- 172.21.15.101:0/2235380616 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5572086974f0 con 0x5572087f6ad0 2023-07-18T03:09:36.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.402+0000 7f6d16ffd700 1 -- 172.21.15.101:0/2235380616 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d30018bb0 con 0x5572087f6ad0 2023-07-18T03:09:36.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.402+0000 7f6d5166bb80 10 client.26000 fetch_fsmap learned FSMap version 25 2023-07-18T03:09:36.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.402+0000 7f6d5166bb80 1 -- 172.21.15.101:0/2235380616 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557208554c60 con 0x5572087f6ad0 2023-07-18T03:09:36.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.402+0000 7f6d16ffd700 1 -- 172.21.15.101:0/2235380616 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5572086974f0 con 0x5572087f6ad0 2023-07-18T03:09:36.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.402+0000 7f6d16ffd700 1 -- 172.21.15.101:0/2235380616 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f6d30008a30 con 0x5572087f6ad0 2023-07-18T03:09:36.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.402+0000 7f6d5166bb80 10 client.26000 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:09:36.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.402+0000 7f6d5166bb80 10 client.26000 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:09:36.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.402+0000 7f6d5166bb80 20 client.26000 populate_metadata read hostname 'smithi101' 2023-07-18T03:09:36.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.402+0000 7f6d5166bb80 1 --2- 172.21.15.101:0/2235380616 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557208806410 0x557208808820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:36.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.402+0000 7f6d5166bb80 4 client.26000 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:09:36.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.402+0000 7f6d5166bb80 1 -- 172.21.15.101:0/2235380616 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557208736eb0 con 0x557208806410 2023-07-18T03:09:36.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.403+0000 7f6d377fe700 1 --2- 172.21.15.101:0/2235380616 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557208806410 0x557208808820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:36.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.403+0000 7f6d377fe700 1 --2- 172.21.15.101:0/2235380616 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557208806410 0x557208808820 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af34fe63c38175d1 server_cookie=ace44db1cdc349fd in_seq=0 out_seq=0 2023-07-18T03:09:36.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.403+0000 7f6d16ffd700 10 client.26000 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:09:36.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.404+0000 7f6d16ffd700 1 -- 172.21.15.101:0/2235380616 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557208736eb0 con 0x557208806410 2023-07-18T03:09:36.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.404+0000 7f6d16ffd700 10 client.26000 handle_command_reply: tid=0 2023-07-18T03:09:36.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.404+0000 7f6d5166bb80 1 client.26000 shutdown 2023-07-18T03:09:36.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.404+0000 7f6d5166bb80 1 -- 172.21.15.101:0/2235380616 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572087f6ad0 msgr2=0x5572087f4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:36.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.404+0000 7f6d5166bb80 1 --2- 172.21.15.101:0/2235380616 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572087f6ad0 0x5572087f4300 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6d3001ae60 tx=0x7f6d30004ad0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.404+0000 7f6d5166bb80 1 -- 172.21.15.101:0/2235380616 shutdown_connections 2023-07-18T03:09:36.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.404+0000 7f6d5166bb80 1 --2- 172.21.15.101:0/2235380616 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557208806410 0x557208808820 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.404+0000 7f6d5166bb80 1 --2- 172.21.15.101:0/2235380616 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572087f6ad0 0x5572087f4300 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.404+0000 7f6d16ffd700 0 client.26000 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:09:36.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.404+0000 7f6d5166bb80 1 --2- 172.21.15.101:0/2235380616 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5572087eb2d0 0x5572087f3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.404+0000 7f6d5166bb80 1 --2- 172.21.15.101:0/2235380616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572087ebbe0 0x5572087f3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.404+0000 7f6d5166bb80 1 -- 172.21.15.101:0/2235380616 >> 172.21.15.101:0/2235380616 conn(0x5572087e1da0 msgr2=0x5572087e7290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:36.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.405+0000 7f6d5166bb80 1 -- 172.21.15.101:0/2235380616 shutdown_connections 2023-07-18T03:09:36.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.405+0000 7f6d5166bb80 1 -- 172.21.15.101:0/2235380616 wait complete. 2023-07-18T03:09:36.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.405+0000 7f6d5166bb80 20 client.26000 trim_cache size 0 max 16384 2023-07-18T03:09:36.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.418+0000 7f6d5166bb80 1 Processor -- start 2023-07-18T03:09:36.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.418+0000 7f6d5166bb80 1 -- start start 2023-07-18T03:09:36.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.418+0000 7f6d5166bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5572087eb2d0 0x557208801d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:36.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.418+0000 7f6d5166bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572087ebbe0 0x5572087faec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:36.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.419+0000 7f6d5166bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572087f6ad0 0x5572087f4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:36.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.419+0000 7f6d5166bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557208802df0 con 0x5572087eb2d0 2023-07-18T03:09:36.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.419+0000 7f6d5166bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572088022d0 con 0x5572087f6ad0 2023-07-18T03:09:36.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.419+0000 7f6d5166bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572087ee6a0 con 0x5572087ebbe0 2023-07-18T03:09:36.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.419+0000 7f6d377fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5572087eb2d0 0x557208801d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:36.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.419+0000 7f6d36ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572087ebbe0 0x5572087faec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:36.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.419+0000 7f6d377fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5572087eb2d0 0x557208801d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:43336/0 (socket says 172.21.15.101:43336) 2023-07-18T03:09:36.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.419+0000 7f6d377fe700 1 -- 172.21.15.101:0/2297810092 learned_addr learned my addr 172.21.15.101:0/2297810092 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:36.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.419+0000 7f6d37fff700 1 --2- 172.21.15.101:0/2297810092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572087f6ad0 0x5572087f4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:36.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.419+0000 7f6d377fe700 1 -- 172.21.15.101:0/2297810092 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572087ebbe0 msgr2=0x5572087faec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:36.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.419+0000 7f6d377fe700 1 --2- 172.21.15.101:0/2297810092 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572087ebbe0 0x5572087faec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.419+0000 7f6d377fe700 1 -- 172.21.15.101:0/2297810092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572087f6ad0 msgr2=0x5572087f4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:36.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.419+0000 7f6d377fe700 1 --2- 172.21.15.101:0/2297810092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572087f6ad0 0x5572087f4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.419+0000 7f6d377fe700 1 -- 172.21.15.101:0/2297810092 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557208592250 con 0x5572087eb2d0 2023-07-18T03:09:36.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.419+0000 7f6d377fe700 1 --2- 172.21.15.101:0/2297810092 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5572087eb2d0 0x557208801d90 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6d280cab70 tx=0x7f6d28099ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d9a39a03a128c62 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:36.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.420+0000 7f6d1e7fc700 1 -- 172.21.15.101:0/2297810092 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d28099460 con 0x5572087eb2d0 2023-07-18T03:09:36.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.420+0000 7f6d1e7fc700 1 -- 172.21.15.101:0/2297810092 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d280995c0 con 0x5572087eb2d0 2023-07-18T03:09:36.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.420+0000 7f6d1e7fc700 1 -- 172.21.15.101:0/2297810092 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d28099130 con 0x5572087eb2d0 2023-07-18T03:09:36.414 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.420+0000 7f6d5166bb80 1 -- 172.21.15.101:0/2297810092 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5572087eb2d0 msgr2=0x557208801d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:36.414 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.420+0000 7f6d5166bb80 1 --2- 172.21.15.101:0/2297810092 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5572087eb2d0 0x557208801d90 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6d280cab70 tx=0x7f6d28099ab0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.415 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.420+0000 7f6d5166bb80 1 -- 172.21.15.101:0/2297810092 shutdown_connections 2023-07-18T03:09:36.415 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.420+0000 7f6d5166bb80 1 --2- 172.21.15.101:0/2297810092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572087f6ad0 0x5572087f4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.415 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.420+0000 7f6d5166bb80 1 --2- 172.21.15.101:0/2297810092 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5572087eb2d0 0x557208801d90 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.415 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.420+0000 7f6d5166bb80 1 --2- 172.21.15.101:0/2297810092 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572087ebbe0 0x5572087faec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.415 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.420+0000 7f6d5166bb80 1 -- 172.21.15.101:0/2297810092 >> 172.21.15.101:0/2297810092 conn(0x5572087e1da0 msgr2=0x557208801680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:36.415 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.420+0000 7f6d5166bb80 1 -- 172.21.15.101:0/2297810092 shutdown_connections 2023-07-18T03:09:36.415 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.420+0000 7f6d5166bb80 1 -- 172.21.15.101:0/2297810092 wait complete. 2023-07-18T03:09:36.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.421+0000 7f6d5166bb80 1 -- start start 2023-07-18T03:09:36.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.421+0000 7f6d5166bb80 1 Processor -- start 2023-07-18T03:09:36.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.422+0000 7f6d5166bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572087eb2d0 0x5572087f8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:36.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.422+0000 7f6d5166bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5572087ebbe0 0x5572087f90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:36.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.422+0000 7f6d5166bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572087f6ad0 0x5572088592d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:36.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.422+0000 7f6d5166bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572087f9850 con 0x5572087ebbe0 2023-07-18T03:09:36.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.422+0000 7f6d5166bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557208859840 con 0x5572087eb2d0 2023-07-18T03:09:36.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.422+0000 7f6d5166bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x557208859a00 con 0x5572087f6ad0 2023-07-18T03:09:36.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.422+0000 7f6d5166bb80 10 client.0 _ll_register_callbacks cb 0x7f6d3001a810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:09:36.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.422+0000 7f6d377fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572087eb2d0 0x5572087f8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:36.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.422+0000 7f6d36ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5572087ebbe0 0x5572087f90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:36.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.422+0000 7f6d377fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572087eb2d0 0x5572087f8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:37784/0 (socket says 172.21.15.101:37784) 2023-07-18T03:09:36.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.422+0000 7f6d377fe700 1 -- 172.21.15.101:0/805364370 learned_addr learned my addr 172.21.15.101:0/805364370 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:36.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.422+0000 7f6d37fff700 1 --2- 172.21.15.101:0/805364370 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572087f6ad0 0x5572088592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:36.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.422+0000 7f6d37fff700 1 -- 172.21.15.101:0/805364370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572087eb2d0 msgr2=0x5572087f8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:36.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.422+0000 7f6d37fff700 1 --2- 172.21.15.101:0/805364370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572087eb2d0 0x5572087f8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.422+0000 7f6d37fff700 1 -- 172.21.15.101:0/805364370 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5572087ebbe0 msgr2=0x5572087f90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:36.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.422+0000 7f6d37fff700 1 --2- 172.21.15.101:0/805364370 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5572087ebbe0 0x5572087f90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.422+0000 7f6d37fff700 1 -- 172.21.15.101:0/805364370 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55720856bde0 con 0x5572087f6ad0 2023-07-18T03:09:36.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.422+0000 7f6d36ffd700 1 --2- 172.21.15.101:0/805364370 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5572087ebbe0 0x5572087f90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:09:36.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.423+0000 7f6d37fff700 1 --2- 172.21.15.101:0/805364370 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572087f6ad0 0x5572088592d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6d30008e10 tx=0x7f6d30006110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eeb559224d0c3277 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:36.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.423+0000 7f6d16ffd700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:09:36.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.423+0000 7f6d16ffd700 1 -- 172.21.15.101:0/805364370 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3000e040 con 0x5572087f6ad0 2023-07-18T03:09:36.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.423+0000 7f6d5166bb80 1 -- 172.21.15.101:0/805364370 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d2809a330 con 0x5572087f6ad0 2023-07-18T03:09:36.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.423+0000 7f6d16ffd700 1 -- 172.21.15.101:0/805364370 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f6d30004d50 con 0x5572087f6ad0 2023-07-18T03:09:36.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.423+0000 7f6d16ffd700 1 -- 172.21.15.101:0/805364370 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d30018a00 con 0x5572087f6ad0 2023-07-18T03:09:36.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.423+0000 7f6d5166bb80 10 client.26032 fetch_fsmap learned FSMap version 25 2023-07-18T03:09:36.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.423+0000 7f6d5166bb80 1 -- 172.21.15.101:0/805364370 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55720856a2b0 con 0x5572087f6ad0 2023-07-18T03:09:36.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.423+0000 7f6d16ffd700 1 -- 172.21.15.101:0/805364370 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d2809a330 con 0x5572087f6ad0 2023-07-18T03:09:36.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.423+0000 7f6d16ffd700 1 -- 172.21.15.101:0/805364370 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f6d30008510 con 0x5572087f6ad0 2023-07-18T03:09:36.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.424+0000 7f6d5166bb80 10 client.26032 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:09:36.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.424+0000 7f6d5166bb80 10 client.26032 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:09:36.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.424+0000 7f6d5166bb80 20 client.26032 populate_metadata read hostname 'smithi101' 2023-07-18T03:09:36.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.424+0000 7f6d5166bb80 1 --2- 172.21.15.101:0/805364370 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55720885b0e0 0x55720885d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:36.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.424+0000 7f6d5166bb80 4 client.26032 mds_command: new command op to 24562 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:09:36.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.424+0000 7f6d5166bb80 1 -- 172.21.15.101:0/805364370 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5572086f9050 con 0x55720885b0e0 2023-07-18T03:09:36.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.424+0000 7f6d377fe700 1 --2- 172.21.15.101:0/805364370 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55720885b0e0 0x55720885d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:36.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.424+0000 7f6d16ffd700 10 client.26032 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:09:36.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.424+0000 7f6d377fe700 1 --2- 172.21.15.101:0/805364370 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55720885b0e0 0x55720885d4f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c355352990d8f359 server_cookie=c17979681640a702 in_seq=0 out_seq=0 2023-07-18T03:09:36.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.425+0000 7f6d16ffd700 1 -- 172.21.15.101:0/805364370 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5572086f9050 con 0x55720885b0e0 2023-07-18T03:09:36.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.425+0000 7f6d16ffd700 10 client.26032 handle_command_reply: tid=0 2023-07-18T03:09:36.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.425+0000 7f6d5166bb80 1 client.26032 shutdown 2023-07-18T03:09:36.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.425+0000 7f6d5166bb80 1 -- 172.21.15.101:0/805364370 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572087f6ad0 msgr2=0x5572088592d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:36.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.425+0000 7f6d5166bb80 1 --2- 172.21.15.101:0/805364370 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572087f6ad0 0x5572088592d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6d30008e10 tx=0x7f6d30006110 comp rx=0 tx=0).stop 2023-07-18T03:09:36.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.425+0000 7f6d5166bb80 1 -- 172.21.15.101:0/805364370 shutdown_connections 2023-07-18T03:09:36.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.425+0000 7f6d5166bb80 1 --2- 172.21.15.101:0/805364370 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55720885b0e0 0x55720885d4f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.425+0000 7f6d5166bb80 1 --2- 172.21.15.101:0/805364370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572087eb2d0 0x5572087f8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.425+0000 7f6d5166bb80 1 --2- 172.21.15.101:0/805364370 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572087f6ad0 0x5572088592d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.425+0000 7f6d16ffd700 0 client.26032 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:09:36.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.425+0000 7f6d5166bb80 1 --2- 172.21.15.101:0/805364370 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5572087ebbe0 0x5572087f90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.425+0000 7f6d5166bb80 1 -- 172.21.15.101:0/805364370 >> 172.21.15.101:0/805364370 conn(0x5572087e1da0 msgr2=0x5572087f4060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:36.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.425+0000 7f6d5166bb80 1 -- 172.21.15.101:0/805364370 shutdown_connections 2023-07-18T03:09:36.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.425+0000 7f6d5166bb80 1 -- 172.21.15.101:0/805364370 wait complete. 2023-07-18T03:09:36.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.425+0000 7f6d5166bb80 20 client.26032 trim_cache size 0 max 16384 2023-07-18T03:09:36.423 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:09:36.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.426+0000 7f6d3ddff700 1 -- 172.21.15.101:0/2343525300 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6d200a1340 msgr2=0x7f6d200a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:36.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.426+0000 7f6d3ddff700 1 --2- 172.21.15.101:0/2343525300 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6d200a1340 0x7f6d200a37f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6d38074920 tx=0x7f6d30009920 comp rx=0 tx=0).stop 2023-07-18T03:09:36.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.426+0000 7f6d3ddff700 1 -- 172.21.15.101:0/2343525300 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d380ffae0 msgr2=0x7f6d38072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:36.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.426+0000 7f6d3ddff700 1 --2- 172.21.15.101:0/2343525300 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d380ffae0 0x7f6d38072810 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6d2800e9a0 tx=0x7f6d2800ecb0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.426+0000 7f6d3ddff700 1 -- 172.21.15.101:0/2343525300 shutdown_connections 2023-07-18T03:09:36.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.426+0000 7f6d3ddff700 1 --2- 172.21.15.101:0/2343525300 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6d38105af0 0x7f6d381a3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.426+0000 7f6d3ddff700 1 --2- 172.21.15.101:0/2343525300 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d380ffae0 0x7f6d38072810 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.426+0000 7f6d3ddff700 1 --2- 172.21.15.101:0/2343525300 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6d200a1340 0x7f6d200a37f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.426+0000 7f6d3ddff700 1 --2- 172.21.15.101:0/2343525300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d38100490 0x7f6d38072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:36.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.426+0000 7f6d3ddff700 1 -- 172.21.15.101:0/2343525300 >> 172.21.15.101:0/2343525300 conn(0x7f6d380fb5e0 msgr2=0x7f6d381035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:36.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.426+0000 7f6d3ddff700 1 -- 172.21.15.101:0/2343525300 shutdown_connections 2023-07-18T03:09:36.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:36.426+0000 7f6d3ddff700 1 -- 172.21.15.101:0/2343525300 wait complete. 2023-07-18T03:09:36.565 INFO:tasks.workunit.client.0.smithi101.stdout: 1 78398 7.24 MB/sec warmup 61 sec latency 18.765 ms 2023-07-18T03:09:37.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:36 smithi179 ceph-mon[129229]: pgmap v64: 129 pgs: 129 active+clean; 54 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 411 KiB/s wr, 78 op/s 2023-07-18T03:09:37.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:36 smithi101 ceph-mon[126379]: pgmap v64: 129 pgs: 129 active+clean; 54 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 411 KiB/s wr, 78 op/s 2023-07-18T03:09:37.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:36 smithi155 ceph-mon[128885]: pgmap v64: 129 pgs: 129 active+clean; 54 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 411 KiB/s wr, 78 op/s 2023-07-18T03:09:37.434 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-18T03:09:37.435 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-18T03:09:37.565 INFO:tasks.workunit.client.0.smithi101.stdout: 1 79609 7.21 MB/sec warmup 62 sec latency 40.748 ms 2023-07-18T03:09:37.651 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.660+0000 7f873759e700 1 -- 172.21.15.101:0/743838455 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f87380ff9f0 msgr2=0x7f87380ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:37.651 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.660+0000 7f873759e700 1 --2- 172.21.15.101:0/743838455 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f87380ff9f0 0x7f87380ffe60 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f872c009990 tx=0x7f872c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.651 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.661+0000 7f873759e700 1 -- 172.21.15.101:0/743838455 shutdown_connections 2023-07-18T03:09:37.651 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.661+0000 7f873759e700 1 --2- 172.21.15.101:0/743838455 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8738105a00 0x7f8738105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.661+0000 7f873759e700 1 --2- 172.21.15.101:0/743838455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87381003a0 0x7f873810c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.661+0000 7f873759e700 1 --2- 172.21.15.101:0/743838455 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f87380ff9f0 0x7f87380ffe60 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.661+0000 7f873759e700 1 -- 172.21.15.101:0/743838455 >> 172.21.15.101:0/743838455 conn(0x7f87380fb510 msgr2=0x7f87380fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:37.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.661+0000 7f873759e700 1 -- 172.21.15.101:0/743838455 shutdown_connections 2023-07-18T03:09:37.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.661+0000 7f873759e700 1 -- 172.21.15.101:0/743838455 wait complete. 2023-07-18T03:09:37.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.662+0000 7f873759e700 1 Processor -- start 2023-07-18T03:09:37.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.662+0000 7f873759e700 1 -- start start 2023-07-18T03:09:37.654 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.662+0000 7f873759e700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f87380ff9f0 0x7f873819b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:37.654 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.662+0000 7f873759e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87381003a0 0x7f873819bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:37.654 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.662+0000 7f873759e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8738105a00 0x7f8738195780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:37.654 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.662+0000 7f873759e700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8738195df0 con 0x7f87380ff9f0 2023-07-18T03:09:37.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.662+0000 7f873759e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8738195f60 con 0x7f8738105a00 2023-07-18T03:09:37.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.662+0000 7f873759e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8738196120 con 0x7f87381003a0 2023-07-18T03:09:37.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.662+0000 7f873659c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f87380ff9f0 0x7f873819b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:37.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.662+0000 7f873659c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f87380ff9f0 0x7f873819b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:43364/0 (socket says 172.21.15.101:43364) 2023-07-18T03:09:37.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.662+0000 7f873659c700 1 -- 172.21.15.101:0/4021928103 learned_addr learned my addr 172.21.15.101:0/4021928103 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:37.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.662+0000 7f8736d9d700 1 --2- 172.21.15.101:0/4021928103 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8738105a00 0x7f8738195780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:37.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.662+0000 7f8735d9b700 1 --2- 172.21.15.101:0/4021928103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87381003a0 0x7f873819bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:37.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.663+0000 7f873659c700 1 -- 172.21.15.101:0/4021928103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87381003a0 msgr2=0x7f873819bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:37.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.663+0000 7f873659c700 1 --2- 172.21.15.101:0/4021928103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87381003a0 0x7f873819bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.663+0000 7f873659c700 1 -- 172.21.15.101:0/4021928103 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8738105a00 msgr2=0x7f8738195780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:37.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.663+0000 7f873659c700 1 --2- 172.21.15.101:0/4021928103 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8738105a00 0x7f8738195780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.663+0000 7f873659c700 1 -- 172.21.15.101:0/4021928103 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8730009c40 con 0x7f87380ff9f0 2023-07-18T03:09:37.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.663+0000 7f8735d9b700 1 --2- 172.21.15.101:0/4021928103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87381003a0 0x7f873819bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:09:37.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.663+0000 7f8736d9d700 1 --2- 172.21.15.101:0/4021928103 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8738105a00 0x7f8738195780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:09:37.657 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.663+0000 7f873659c700 1 --2- 172.21.15.101:0/4021928103 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f87380ff9f0 0x7f873819b710 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f872800e970 tx=0x7f872800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:37.657 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.663+0000 7f87277fe700 1 -- 172.21.15.101:0/4021928103 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8728009790 con 0x7f87380ff9f0 2023-07-18T03:09:37.657 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.663+0000 7f87277fe700 1 -- 172.21.15.101:0/4021928103 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f872800c7c0 con 0x7f87380ff9f0 2023-07-18T03:09:37.657 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.663+0000 7f87277fe700 1 -- 172.21.15.101:0/4021928103 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8728003c20 con 0x7f87380ff9f0 2023-07-18T03:09:37.657 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.663+0000 7f873759e700 1 -- 172.21.15.101:0/4021928103 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f872c0095f0 con 0x7f87380ff9f0 2023-07-18T03:09:37.657 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.663+0000 7f873759e700 1 -- 172.21.15.101:0/4021928103 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87381a2520 con 0x7f87380ff9f0 2023-07-18T03:09:37.659 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.666+0000 7f87277fe700 1 -- 172.21.15.101:0/4021928103 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f872800c930 con 0x7f87380ff9f0 2023-07-18T03:09:37.659 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.668+0000 7f8750509b80 1 Processor -- start 2023-07-18T03:09:37.659 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.668+0000 7f8750509b80 1 -- start start 2023-07-18T03:09:37.659 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.668+0000 7f8750509b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a51cfc570 0x555a51cfc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:37.659 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.668+0000 7f87277fe700 1 --2- 172.21.15.101:0/4021928103 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f87200a13e0 0x7f87200a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:37.659 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.668+0000 7f8750509b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555a51cfce80 0x555a51d0ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:37.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.668+0000 7f8750509b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a51d0b1c0 0x555a51d0d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:37.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.668+0000 7f8750509b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a51c01d30 con 0x555a51cfce80 2023-07-18T03:09:37.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.668+0000 7f8750509b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a51c54ac0 con 0x555a51d0b1c0 2023-07-18T03:09:37.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.668+0000 7f8750509b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a51c4df50 con 0x555a51cfc570 2023-07-18T03:09:37.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.668+0000 7f87277fe700 1 -- 172.21.15.101:0/4021928103 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f8728014070 con 0x7f87380ff9f0 2023-07-18T03:09:37.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.668+0000 7f8736d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a51cfc570 0x555a51cfc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:37.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.668+0000 7f8735d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555a51cfce80 0x555a51d0ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:37.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.668+0000 7f8736d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a51cfc570 0x555a51cfc940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:60682/0 (socket says 172.21.15.101:60682) 2023-07-18T03:09:37.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.668+0000 7f8736d9d700 1 -- 172.21.15.101:0/2577615545 learned_addr learned my addr 172.21.15.101:0/2577615545 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:37.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.669+0000 7f873659c700 1 --2- 172.21.15.101:0/4021928103 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f87200a13e0 0x7f87200a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:37.662 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.669+0000 7f8736d9d700 1 --2- 172.21.15.101:0/2577615545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a51d0b1c0 0x555a51d0d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:37.662 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.669+0000 7f8736d9d700 1 -- 172.21.15.101:0/2577615545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a51d0b1c0 msgr2=0x555a51d0d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:37.662 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.669+0000 7f8736d9d700 1 --2- 172.21.15.101:0/2577615545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a51d0b1c0 0x555a51d0d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.663 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.669+0000 7f8736d9d700 1 -- 172.21.15.101:0/2577615545 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555a51cfce80 msgr2=0x555a51d0ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:37.663 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.669+0000 7f8736d9d700 1 --2- 172.21.15.101:0/2577615545 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555a51cfce80 0x555a51d0ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.663 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.669+0000 7f8736d9d700 1 -- 172.21.15.101:0/2577615545 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a51a7be60 con 0x555a51cfc570 2023-07-18T03:09:37.663 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.669+0000 7f8736d9d700 1 --2- 172.21.15.101:0/2577615545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a51cfc570 0x555a51cfc940 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f873000ea70 tx=0x7f8730009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa1f5267543c9b1e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:37.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.669+0000 7f873659c700 1 --2- 172.21.15.101:0/4021928103 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f87200a13e0 0x7f87200a3890 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8728096430 tx=0x7f872800cf70 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:37.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.669+0000 7f8724ff9700 1 -- 172.21.15.101:0/2577615545 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8730004db0 con 0x555a51cfc570 2023-07-18T03:09:37.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.669+0000 7f8724ff9700 1 -- 172.21.15.101:0/2577615545 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f873000f040 con 0x555a51cfc570 2023-07-18T03:09:37.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.669+0000 7f8724ff9700 1 -- 172.21.15.101:0/2577615545 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8730018c40 con 0x555a51cfc570 2023-07-18T03:09:37.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.670+0000 7f8750509b80 1 -- 172.21.15.101:0/2577615545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a51cfc570 msgr2=0x555a51cfc940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:37.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.670+0000 7f8750509b80 1 --2- 172.21.15.101:0/2577615545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a51cfc570 0x555a51cfc940 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f873000ea70 tx=0x7f8730009920 comp rx=0 tx=0).stop 2023-07-18T03:09:37.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.670+0000 7f8750509b80 1 -- 172.21.15.101:0/2577615545 shutdown_connections 2023-07-18T03:09:37.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.670+0000 7f8750509b80 1 --2- 172.21.15.101:0/2577615545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a51d0b1c0 0x555a51d0d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.670+0000 7f8750509b80 1 --2- 172.21.15.101:0/2577615545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a51cfc570 0x555a51cfc940 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.670+0000 7f8750509b80 1 --2- 172.21.15.101:0/2577615545 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555a51cfce80 0x555a51d0ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.670+0000 7f8750509b80 1 -- 172.21.15.101:0/2577615545 >> 172.21.15.101:0/2577615545 conn(0x555a51cf2070 msgr2=0x555a51cf79b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:37.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.670+0000 7f8750509b80 1 -- 172.21.15.101:0/2577615545 shutdown_connections 2023-07-18T03:09:37.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.670+0000 7f8750509b80 1 -- 172.21.15.101:0/2577615545 wait complete. 2023-07-18T03:09:37.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.671+0000 7f8750509b80 1 -- start start 2023-07-18T03:09:37.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.671+0000 7f8750509b80 1 Processor -- start 2023-07-18T03:09:37.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.672+0000 7f8750509b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a51cfc570 0x555a51d0abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:37.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.672+0000 7f8750509b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555a51cfce80 0x555a51d07df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:37.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.672+0000 7f8750509b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a51d0b1c0 0x555a51d08420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:37.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.672+0000 7f8750509b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a51bedb40 con 0x555a51cfce80 2023-07-18T03:09:37.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.672+0000 7f8750509b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a51bed840 con 0x555a51cfc570 2023-07-18T03:09:37.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.672+0000 7f8750509b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a51ad19a0 con 0x555a51d0b1c0 2023-07-18T03:09:37.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.672+0000 7f8750509b80 10 client.0 _ll_register_callbacks cb 0x555a51beb7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:09:37.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.672+0000 7f8736d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a51cfc570 0x555a51d0abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:37.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.672+0000 7f8735d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555a51cfce80 0x555a51d07df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:37.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.672+0000 7f8736d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a51cfc570 0x555a51d0abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:37828/0 (socket says 172.21.15.101:37828) 2023-07-18T03:09:37.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.672+0000 7f8736d9d700 1 -- 172.21.15.101:0/1806075060 learned_addr learned my addr 172.21.15.101:0/1806075060 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:37.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.672+0000 7f8736d9d700 1 --2- 172.21.15.101:0/1806075060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a51d0b1c0 0x555a51d08420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:37.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.673+0000 7f8735d9b700 1 -- 172.21.15.101:0/1806075060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a51d0b1c0 msgr2=0x555a51d08420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:37.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.673+0000 7f8735d9b700 1 --2- 172.21.15.101:0/1806075060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a51d0b1c0 0x555a51d08420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.673+0000 7f8735d9b700 1 -- 172.21.15.101:0/1806075060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a51cfc570 msgr2=0x555a51d0abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:37.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.673+0000 7f8735d9b700 1 --2- 172.21.15.101:0/1806075060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a51cfc570 0x555a51d0abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.673+0000 7f8735d9b700 1 -- 172.21.15.101:0/1806075060 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f873000e6e0 con 0x555a51cfce80 2023-07-18T03:09:37.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.673+0000 7f8735d9b700 1 --2- 172.21.15.101:0/1806075060 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555a51cfce80 0x555a51d07df0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f872c00b2e0 tx=0x7f872c005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=915b3853b233e6a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:37.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.673+0000 7f871dffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:09:37.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.673+0000 7f871dffb700 1 -- 172.21.15.101:0/1806075060 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c01d070 con 0x555a51cfce80 2023-07-18T03:09:37.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.673+0000 7f871dffb700 1 -- 172.21.15.101:0/1806075060 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f872c0056c0 con 0x555a51cfce80 2023-07-18T03:09:37.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.673+0000 7f8750509b80 1 -- 172.21.15.101:0/1806075060 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555a51ba7570 con 0x555a51cfce80 2023-07-18T03:09:37.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.674+0000 7f871dffb700 1 -- 172.21.15.101:0/1806075060 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f872c004910 con 0x555a51cfce80 2023-07-18T03:09:37.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.674+0000 7f8750509b80 10 client.15984 fetch_fsmap learned FSMap version 25 2023-07-18T03:09:37.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.674+0000 7f8750509b80 1 -- 172.21.15.101:0/1806075060 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555a51a64ce0 con 0x555a51cfce80 2023-07-18T03:09:37.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.674+0000 7f871dffb700 1 -- 172.21.15.101:0/1806075060 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555a51ba7570 con 0x555a51cfce80 2023-07-18T03:09:37.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.674+0000 7f871dffb700 1 -- 172.21.15.101:0/1806075060 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f872c00fc20 con 0x555a51cfce80 2023-07-18T03:09:37.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.674+0000 7f8750509b80 10 client.15984 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:09:37.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.674+0000 7f8750509b80 10 client.15984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:09:37.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.674+0000 7f8750509b80 20 client.15984 populate_metadata read hostname 'smithi101' 2023-07-18T03:09:37.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.674+0000 7f8750509b80 1 --2- 172.21.15.101:0/1806075060 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x555a51d1abb0 0x555a51d1cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:37.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.674+0000 7f8750509b80 4 client.15984 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:09:37.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.674+0000 7f8750509b80 1 -- 172.21.15.101:0/1806075060 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555a51c46f30 con 0x555a51d1abb0 2023-07-18T03:09:37.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.674+0000 7f8736d9d700 1 --2- 172.21.15.101:0/1806075060 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x555a51d1abb0 0x555a51d1cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:37.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.675+0000 7f8736d9d700 1 --2- 172.21.15.101:0/1806075060 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x555a51d1abb0 0x555a51d1cfc0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bb3d4b8c7d6e904 server_cookie=5ca8e563b635e08d in_seq=0 out_seq=0 2023-07-18T03:09:37.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.675+0000 7f871dffb700 10 client.15984 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:09:37.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.676+0000 7f871dffb700 1 -- 172.21.15.101:0/1806075060 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555a51c46f30 con 0x555a51d1abb0 2023-07-18T03:09:37.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.676+0000 7f871dffb700 10 client.15984 handle_command_reply: tid=0 2023-07-18T03:09:37.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.677+0000 7f8750509b80 1 client.15984 shutdown 2023-07-18T03:09:37.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.677+0000 7f8750509b80 1 -- 172.21.15.101:0/1806075060 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555a51cfce80 msgr2=0x555a51d07df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:37.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.677+0000 7f8750509b80 1 --2- 172.21.15.101:0/1806075060 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555a51cfce80 0x555a51d07df0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f872c00b2e0 tx=0x7f872c005270 comp rx=0 tx=0).stop 2023-07-18T03:09:37.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.677+0000 7f8750509b80 1 -- 172.21.15.101:0/1806075060 shutdown_connections 2023-07-18T03:09:37.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.677+0000 7f8750509b80 1 --2- 172.21.15.101:0/1806075060 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x555a51d1abb0 0x555a51d1cfc0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.677+0000 7f8750509b80 1 --2- 172.21.15.101:0/1806075060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a51cfc570 0x555a51d0abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.677+0000 7f8750509b80 1 --2- 172.21.15.101:0/1806075060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a51d0b1c0 0x555a51d08420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.677+0000 7f871dffb700 0 client.15984 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:09:37.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.677+0000 7f8750509b80 1 --2- 172.21.15.101:0/1806075060 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555a51cfce80 0x555a51d07df0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.677+0000 7f8750509b80 1 -- 172.21.15.101:0/1806075060 >> 172.21.15.101:0/1806075060 conn(0x555a51cf2070 msgr2=0x555a51cf75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:37.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.677+0000 7f8750509b80 1 -- 172.21.15.101:0/1806075060 shutdown_connections 2023-07-18T03:09:37.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.677+0000 7f8750509b80 1 -- 172.21.15.101:0/1806075060 wait complete. 2023-07-18T03:09:37.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.677+0000 7f8750509b80 20 client.15984 trim_cache size 0 max 16384 2023-07-18T03:09:37.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.695+0000 7f8750509b80 1 Processor -- start 2023-07-18T03:09:37.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.695+0000 7f8750509b80 1 -- start start 2023-07-18T03:09:37.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.695+0000 7f8750509b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555a51cfc570 0x555a51d0a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:37.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.695+0000 7f8750509b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a51cfce80 0x555a51d05080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:37.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.695+0000 7f8750509b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a51d0b1c0 0x555a51d055c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:37.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.695+0000 7f8750509b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a51cfe290 con 0x555a51cfc570 2023-07-18T03:09:37.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.695+0000 7f8750509b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a51d18210 con 0x555a51d0b1c0 2023-07-18T03:09:37.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.695+0000 7f8750509b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a51cfda00 con 0x555a51cfce80 2023-07-18T03:09:37.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.696+0000 7f8736d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555a51cfc570 0x555a51d0a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:37.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.696+0000 7f8735d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a51cfce80 0x555a51d05080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:37.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.696+0000 7f8736d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555a51cfc570 0x555a51d0a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:43394/0 (socket says 172.21.15.101:43394) 2023-07-18T03:09:37.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.696+0000 7f8736d9d700 1 -- 172.21.15.101:0/184859771 learned_addr learned my addr 172.21.15.101:0/184859771 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:37.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.696+0000 7f8736d9d700 1 --2- 172.21.15.101:0/184859771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a51d0b1c0 0x555a51d055c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:37.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.696+0000 7f8735d9b700 1 -- 172.21.15.101:0/184859771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a51d0b1c0 msgr2=0x555a51d055c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:37.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.696+0000 7f8735d9b700 1 --2- 172.21.15.101:0/184859771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a51d0b1c0 0x555a51d055c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.696+0000 7f8735d9b700 1 -- 172.21.15.101:0/184859771 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555a51cfc570 msgr2=0x555a51d0a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:37.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.696+0000 7f8735d9b700 1 --2- 172.21.15.101:0/184859771 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555a51cfc570 0x555a51d0a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.696+0000 7f8735d9b700 1 -- 172.21.15.101:0/184859771 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a51aa22d0 con 0x555a51cfce80 2023-07-18T03:09:37.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.696+0000 7f8736d9d700 1 --2- 172.21.15.101:0/184859771 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555a51cfc570 0x555a51d0a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:09:37.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.696+0000 7f8735d9b700 1 --2- 172.21.15.101:0/184859771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a51cfce80 0x555a51d05080 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f872c023ab0 tx=0x7f872c009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c6e2617d55712d1 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:37.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.696+0000 7f87257fa700 1 -- 172.21.15.101:0/184859771 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c01d070 con 0x555a51cfce80 2023-07-18T03:09:37.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.696+0000 7f87257fa700 1 -- 172.21.15.101:0/184859771 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f872c01e040 con 0x555a51cfce80 2023-07-18T03:09:37.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.696+0000 7f87257fa700 1 -- 172.21.15.101:0/184859771 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c0238e0 con 0x555a51cfce80 2023-07-18T03:09:37.690 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.697+0000 7f8750509b80 1 -- 172.21.15.101:0/184859771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a51cfce80 msgr2=0x555a51d05080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:37.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.697+0000 7f8750509b80 1 --2- 172.21.15.101:0/184859771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a51cfce80 0x555a51d05080 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f872c023ab0 tx=0x7f872c009310 comp rx=0 tx=0).stop 2023-07-18T03:09:37.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.697+0000 7f8750509b80 1 -- 172.21.15.101:0/184859771 shutdown_connections 2023-07-18T03:09:37.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.697+0000 7f8750509b80 1 --2- 172.21.15.101:0/184859771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a51d0b1c0 0x555a51d055c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.697+0000 7f8750509b80 1 --2- 172.21.15.101:0/184859771 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555a51cfc570 0x555a51d0a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.697+0000 7f8750509b80 1 --2- 172.21.15.101:0/184859771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a51cfce80 0x555a51d05080 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.697+0000 7f8750509b80 1 -- 172.21.15.101:0/184859771 >> 172.21.15.101:0/184859771 conn(0x555a51cf2070 msgr2=0x555a51d03120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:37.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.697+0000 7f8750509b80 1 -- 172.21.15.101:0/184859771 shutdown_connections 2023-07-18T03:09:37.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.697+0000 7f8750509b80 1 -- 172.21.15.101:0/184859771 wait complete. 2023-07-18T03:09:37.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.698+0000 7f8750509b80 1 -- start start 2023-07-18T03:09:37.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.698+0000 7f8750509b80 1 Processor -- start 2023-07-18T03:09:37.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.698+0000 7f8750509b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555a51cfc570 0x555a51d01560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:37.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.698+0000 7f8750509b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a51cfce80 0x555a51d01aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:37.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.698+0000 7f8750509b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a51d0b1c0 0x555a51d02000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:37.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.698+0000 7f8750509b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a51d73400 con 0x555a51cfc570 2023-07-18T03:09:37.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.698+0000 7f8750509b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a51d73570 con 0x555a51cfce80 2023-07-18T03:09:37.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.698+0000 7f8750509b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a51d73730 con 0x555a51d0b1c0 2023-07-18T03:09:37.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.698+0000 7f8750509b80 10 client.0 _ll_register_callbacks cb 0x7f872c00bf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:09:37.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.699+0000 7f8736d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555a51cfc570 0x555a51d01560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:37.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.699+0000 7f8735d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a51cfce80 0x555a51d01aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:37.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.699+0000 7f8736d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555a51cfc570 0x555a51d01560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:43400/0 (socket says 172.21.15.101:43400) 2023-07-18T03:09:37.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.699+0000 7f8736d9d700 1 -- 172.21.15.101:0/2526823455 learned_addr learned my addr 172.21.15.101:0/2526823455 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:37.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.699+0000 7f8736d9d700 1 --2- 172.21.15.101:0/2526823455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a51d0b1c0 0x555a51d02000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:37.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.699+0000 7f8736d9d700 1 -- 172.21.15.101:0/2526823455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a51d0b1c0 msgr2=0x555a51d02000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:37.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.699+0000 7f8736d9d700 1 --2- 172.21.15.101:0/2526823455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a51d0b1c0 0x555a51d02000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.699+0000 7f8736d9d700 1 -- 172.21.15.101:0/2526823455 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a51cfce80 msgr2=0x555a51d01aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:37.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.699+0000 7f8736d9d700 1 --2- 172.21.15.101:0/2526823455 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a51cfce80 0x555a51d01aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.699+0000 7f8736d9d700 1 -- 172.21.15.101:0/2526823455 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a51a7be60 con 0x555a51cfc570 2023-07-18T03:09:37.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.699+0000 7f8736d9d700 1 --2- 172.21.15.101:0/2526823455 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555a51cfc570 0x555a51d01560 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x555a51d02540 tx=0x7f87300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d825ceb3d3b74a3c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:37.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.699+0000 7f871dffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:09:37.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.699+0000 7f871dffb700 1 -- 172.21.15.101:0/2526823455 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8730007d20 con 0x555a51cfc570 2023-07-18T03:09:37.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.700+0000 7f8750509b80 1 -- 172.21.15.101:0/2526823455 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f873000f040 con 0x555a51cfc570 2023-07-18T03:09:37.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.700+0000 7f871dffb700 1 -- 172.21.15.101:0/2526823455 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f8730003a00 con 0x555a51cfc570 2023-07-18T03:09:37.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.700+0000 7f871dffb700 1 -- 172.21.15.101:0/2526823455 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8730007e80 con 0x555a51cfc570 2023-07-18T03:09:37.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.700+0000 7f8750509b80 10 client.15990 fetch_fsmap learned FSMap version 25 2023-07-18T03:09:37.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.700+0000 7f8750509b80 1 -- 172.21.15.101:0/2526823455 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555a51a64ce0 con 0x555a51cfc570 2023-07-18T03:09:37.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.700+0000 7f871dffb700 1 -- 172.21.15.101:0/2526823455 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f873000f040 con 0x555a51cfc570 2023-07-18T03:09:37.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.700+0000 7f871dffb700 1 -- 172.21.15.101:0/2526823455 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f8730036020 con 0x555a51cfc570 2023-07-18T03:09:37.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.700+0000 7f8750509b80 10 client.15990 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:09:37.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.700+0000 7f8750509b80 10 client.15990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:09:37.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.700+0000 7f8750509b80 20 client.15990 populate_metadata read hostname 'smithi101' 2023-07-18T03:09:37.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.700+0000 7f8750509b80 1 --2- 172.21.15.101:0/2526823455 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x555a51d74fb0 0x555a51d773c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:37.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.700+0000 7f8750509b80 4 client.15990 mds_command: new command op to 24562 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-18T03:09:37.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.700+0000 7f8750509b80 1 -- 172.21.15.101:0/2526823455 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x555a51c090d0 con 0x555a51d74fb0 2023-07-18T03:09:37.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.701+0000 7f8735d9b700 1 --2- 172.21.15.101:0/2526823455 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x555a51d74fb0 0x555a51d773c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:37.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.701+0000 7f8735d9b700 1 --2- 172.21.15.101:0/2526823455 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x555a51d74fb0 0x555a51d773c0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=285e17e0d16b7519 server_cookie=c7389e148f458c44 in_seq=0 out_seq=0 2023-07-18T03:09:37.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.701+0000 7f871dffb700 10 client.15990 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:09:37.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.703+0000 7f871dffb700 1 -- 172.21.15.101:0/2526823455 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555a51c090d0 con 0x555a51d74fb0 2023-07-18T03:09:37.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.703+0000 7f871dffb700 10 client.15990 handle_command_reply: tid=0 2023-07-18T03:09:37.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.703+0000 7f8750509b80 1 client.15990 shutdown 2023-07-18T03:09:37.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.703+0000 7f8750509b80 1 -- 172.21.15.101:0/2526823455 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555a51cfc570 msgr2=0x555a51d01560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:37.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.703+0000 7f8750509b80 1 --2- 172.21.15.101:0/2526823455 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555a51cfc570 0x555a51d01560 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x555a51d02540 tx=0x7f87300091a0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.703+0000 7f8750509b80 1 -- 172.21.15.101:0/2526823455 shutdown_connections 2023-07-18T03:09:37.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.703+0000 7f8750509b80 1 --2- 172.21.15.101:0/2526823455 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x555a51d74fb0 0x555a51d773c0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.703+0000 7f8750509b80 1 --2- 172.21.15.101:0/2526823455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a51d0b1c0 0x555a51d02000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.703+0000 7f8750509b80 1 --2- 172.21.15.101:0/2526823455 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555a51cfc570 0x555a51d01560 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.703+0000 7f8750509b80 1 --2- 172.21.15.101:0/2526823455 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555a51cfce80 0x555a51d01aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.703+0000 7f8750509b80 1 -- 172.21.15.101:0/2526823455 >> 172.21.15.101:0/2526823455 conn(0x555a51cf2070 msgr2=0x555a51d065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:37.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.703+0000 7f871dffb700 0 client.15990 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:09:37.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.703+0000 7f8750509b80 1 -- 172.21.15.101:0/2526823455 shutdown_connections 2023-07-18T03:09:37.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.703+0000 7f8750509b80 1 -- 172.21.15.101:0/2526823455 wait complete. 2023-07-18T03:09:37.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.704+0000 7f8750509b80 20 client.15990 trim_cache size 0 max 16384 2023-07-18T03:09:37.704 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:09:37.704 INFO:teuthology.orchestra.run.smithi101.stdout: "return_code": 0, 2023-07-18T03:09:37.704 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_tag": "7e6b3aee-b6d8-4cc1-9064-d4034c0e8d33", 2023-07-18T03:09:37.704 INFO:teuthology.orchestra.run.smithi101.stdout: "mode": "asynchronous" 2023-07-18T03:09:37.705 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:09:37.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.704+0000 7f873759e700 1 -- 172.21.15.101:0/4021928103 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f87200a13e0 msgr2=0x7f87200a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:37.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.704+0000 7f873759e700 1 --2- 172.21.15.101:0/4021928103 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f87200a13e0 0x7f87200a3890 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8728096430 tx=0x7f872800cf70 comp rx=0 tx=0).stop 2023-07-18T03:09:37.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.704+0000 7f873759e700 1 -- 172.21.15.101:0/4021928103 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f87380ff9f0 msgr2=0x7f873819b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:37.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.704+0000 7f873759e700 1 --2- 172.21.15.101:0/4021928103 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f87380ff9f0 0x7f873819b710 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f872800e970 tx=0x7f872800ec80 comp rx=0 tx=0).stop 2023-07-18T03:09:37.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.704+0000 7f873759e700 1 -- 172.21.15.101:0/4021928103 shutdown_connections 2023-07-18T03:09:37.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.704+0000 7f873759e700 1 --2- 172.21.15.101:0/4021928103 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f87200a13e0 0x7f87200a3890 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.704+0000 7f873759e700 1 --2- 172.21.15.101:0/4021928103 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8738105a00 0x7f8738195780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.704+0000 7f873759e700 1 --2- 172.21.15.101:0/4021928103 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f87380ff9f0 0x7f873819b710 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.704+0000 7f873759e700 1 --2- 172.21.15.101:0/4021928103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87381003a0 0x7f873819bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.704+0000 7f873759e700 1 -- 172.21.15.101:0/4021928103 >> 172.21.15.101:0/4021928103 conn(0x7f87380fb510 msgr2=0x7f87380fca50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:37.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.704+0000 7f873759e700 1 -- 172.21.15.101:0/4021928103 shutdown_connections 2023-07-18T03:09:37.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.704+0000 7f873759e700 1 -- 172.21.15.101:0/4021928103 wait complete. 2023-07-18T03:09:37.712 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:09:37.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.936+0000 7f9ff7bff700 1 -- 172.21.15.101:0/1421417827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ff0105b10 msgr2=0x7f9ff0105ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:37.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.936+0000 7f9ff7bff700 1 --2- 172.21.15.101:0/1421417827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ff0105b10 0x7f9ff0105ee0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9fec009990 tx=0x7f9fec009ca0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.937+0000 7f9ff7bff700 1 -- 172.21.15.101:0/1421417827 shutdown_connections 2023-07-18T03:09:37.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.937+0000 7f9ff7bff700 1 --2- 172.21.15.101:0/1421417827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ff0105b10 0x7f9ff0105ee0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.937+0000 7f9ff7bff700 1 --2- 172.21.15.101:0/1421417827 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9ff01004b0 0x7f9ff010c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.937+0000 7f9ff7bff700 1 --2- 172.21.15.101:0/1421417827 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ff00ffb00 0x7f9ff00fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.937+0000 7f9ff7bff700 1 -- 172.21.15.101:0/1421417827 >> 172.21.15.101:0/1421417827 conn(0x7f9ff00fb600 msgr2=0x7f9ff00fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:37.928 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.937+0000 7f9ff7bff700 1 -- 172.21.15.101:0/1421417827 shutdown_connections 2023-07-18T03:09:37.928 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.937+0000 7f9ff7bff700 1 -- 172.21.15.101:0/1421417827 wait complete. 2023-07-18T03:09:37.929 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.938+0000 7f9ff7bff700 1 Processor -- start 2023-07-18T03:09:37.930 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.938+0000 7f9ff7bff700 1 -- start start 2023-07-18T03:09:37.930 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.938+0000 7f9ff7bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9ff00ffb00 0x7f9ff0072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:37.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.938+0000 7f9ff7bff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ff01004b0 0x7f9ff0072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:37.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.938+0000 7f9ff7bff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ff0105b10 0x7f9ff0073290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:37.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.938+0000 7f9ff7bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff01a4080 con 0x7f9ff00ffb00 2023-07-18T03:09:37.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.938+0000 7f9ff7bff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff01a41f0 con 0x7f9ff0105b10 2023-07-18T03:09:37.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.938+0000 7f9ff7bff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff01a43b0 con 0x7f9ff01004b0 2023-07-18T03:09:37.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.938+0000 7f9ff519a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ff01004b0 0x7f9ff0072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:37.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.939+0000 7f9ff519a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ff01004b0 0x7f9ff0072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:60726/0 (socket says 172.21.15.101:60726) 2023-07-18T03:09:37.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.939+0000 7f9ff519a700 1 -- 172.21.15.101:0/3886986565 learned_addr learned my addr 172.21.15.101:0/3886986565 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:37.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.939+0000 7f9ff619c700 1 --2- 172.21.15.101:0/3886986565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ff0105b10 0x7f9ff0073290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:37.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.939+0000 7f9ff599b700 1 --2- 172.21.15.101:0/3886986565 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9ff00ffb00 0x7f9ff0072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:37.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.939+0000 7f9ff519a700 1 -- 172.21.15.101:0/3886986565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ff0105b10 msgr2=0x7f9ff0073290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:37.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.939+0000 7f9ff519a700 1 --2- 172.21.15.101:0/3886986565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ff0105b10 0x7f9ff0073290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.939+0000 7f9ff519a700 1 -- 172.21.15.101:0/3886986565 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9ff00ffb00 msgr2=0x7f9ff0072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:37.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.939+0000 7f9ff519a700 1 --2- 172.21.15.101:0/3886986565 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9ff00ffb00 0x7f9ff0072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.939+0000 7f9ff519a700 1 -- 172.21.15.101:0/3886986565 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fec0095f0 con 0x7f9ff01004b0 2023-07-18T03:09:37.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.939+0000 7f9ff599b700 1 --2- 172.21.15.101:0/3886986565 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9ff00ffb00 0x7f9ff0072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:09:37.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.939+0000 7f9ff519a700 1 --2- 172.21.15.101:0/3886986565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ff01004b0 0x7f9ff0072d50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9fe000e9a0 tx=0x7f9fe000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:37.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.939+0000 7f9fe6ffd700 1 -- 172.21.15.101:0/3886986565 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0009790 con 0x7f9ff01004b0 2023-07-18T03:09:37.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.939+0000 7f9ff7bff700 1 -- 172.21.15.101:0/3886986565 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ff01a4690 con 0x7f9ff01004b0 2023-07-18T03:09:37.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.939+0000 7f9fe6ffd700 1 -- 172.21.15.101:0/3886986565 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fe0010430 con 0x7f9ff01004b0 2023-07-18T03:09:37.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.939+0000 7f9fe6ffd700 1 -- 172.21.15.101:0/3886986565 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0018680 con 0x7f9ff01004b0 2023-07-18T03:09:37.934 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.939+0000 7f9ff7bff700 1 -- 172.21.15.101:0/3886986565 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ff01a4bf0 con 0x7f9ff01004b0 2023-07-18T03:09:37.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.943+0000 7f9fe6ffd700 1 -- 172.21.15.101:0/3886986565 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f9fe00105a0 con 0x7f9ff01004b0 2023-07-18T03:09:37.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.944+0000 7fa00b504b80 1 Processor -- start 2023-07-18T03:09:37.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.944+0000 7fa00b504b80 1 -- start start 2023-07-18T03:09:37.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.944+0000 7f9fe6ffd700 1 --2- 172.21.15.101:0/3886986565 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9fdc0a12c0 0x7f9fdc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:37.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.944+0000 7fa00b504b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c22dba2d0 0x559c22dba6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:37.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.944+0000 7fa00b504b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c22dbabe0 0x559c22dc5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:37.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.944+0000 7fa00b504b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c22dc5ad0 0x559c22dc7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:37.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.944+0000 7fa00b504b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c22cc0cb0 con 0x559c22dbabe0 2023-07-18T03:09:37.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.944+0000 7fa00b504b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c22d13a40 con 0x559c22dc5ad0 2023-07-18T03:09:37.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.944+0000 7fa00b504b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c22d0ced0 con 0x559c22dba2d0 2023-07-18T03:09:37.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.944+0000 7f9fe6ffd700 1 -- 172.21.15.101:0/3886986565 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f9fe0014070 con 0x7f9ff01004b0 2023-07-18T03:09:37.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.944+0000 7f9ff599b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c22dba2d0 0x559c22dba6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:37.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.944+0000 7f9ff599b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c22dba2d0 0x559c22dba6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:60728/0 (socket says 172.21.15.101:60728) 2023-07-18T03:09:37.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.944+0000 7f9ff599b700 1 -- 172.21.15.101:0/1366724203 learned_addr learned my addr 172.21.15.101:0/1366724203 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:37.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.944+0000 7f9ff619c700 1 --2- 172.21.15.101:0/3886986565 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9fdc0a12c0 0x7f9fdc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:37.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.945+0000 7f9ff619c700 1 --2- 172.21.15.101:0/1366724203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c22dc5ad0 0x559c22dc7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:37.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.945+0000 7f9ff519a700 1 --2- 172.21.15.101:0/1366724203 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c22dbabe0 0x559c22dc5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:37.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.945+0000 7f9ff599b700 1 -- 172.21.15.101:0/1366724203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c22dc5ad0 msgr2=0x559c22dc7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:37.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.945+0000 7f9ff599b700 1 --2- 172.21.15.101:0/1366724203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c22dc5ad0 0x559c22dc7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.945+0000 7f9ff599b700 1 -- 172.21.15.101:0/1366724203 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c22dbabe0 msgr2=0x559c22dc5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:37.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.945+0000 7f9ff599b700 1 --2- 172.21.15.101:0/1366724203 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c22dbabe0 0x559c22dc5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.945+0000 7f9ff599b700 1 -- 172.21.15.101:0/1366724203 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c22b3ade0 con 0x559c22dba2d0 2023-07-18T03:09:37.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.945+0000 7f9ff599b700 1 --2- 172.21.15.101:0/1366724203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c22dba2d0 0x559c22dba6a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9fec005ea0 tx=0x7f9fec005ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13bc86322d2ab659 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:37.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.945+0000 7f9ff619c700 1 --2- 172.21.15.101:0/3886986565 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9fdc0a12c0 0x7f9fdc0a3770 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8009ec0 tx=0x7f9fe8009920 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:37.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.945+0000 7f9fdbfff700 1 -- 172.21.15.101:0/1366724203 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fec01d070 con 0x559c22dba2d0 2023-07-18T03:09:37.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.945+0000 7f9fdbfff700 1 -- 172.21.15.101:0/1366724203 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fec01e040 con 0x559c22dba2d0 2023-07-18T03:09:37.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.945+0000 7f9fdbfff700 1 -- 172.21.15.101:0/1366724203 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fec004c60 con 0x559c22dba2d0 2023-07-18T03:09:37.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.946+0000 7fa00b504b80 1 -- 172.21.15.101:0/1366724203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c22dba2d0 msgr2=0x559c22dba6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:37.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.946+0000 7fa00b504b80 1 --2- 172.21.15.101:0/1366724203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c22dba2d0 0x559c22dba6a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9fec005ea0 tx=0x7f9fec005ed0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.946+0000 7fa00b504b80 1 -- 172.21.15.101:0/1366724203 shutdown_connections 2023-07-18T03:09:37.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.946+0000 7fa00b504b80 1 --2- 172.21.15.101:0/1366724203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c22dc5ad0 0x559c22dc7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.946+0000 7fa00b504b80 1 --2- 172.21.15.101:0/1366724203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c22dba2d0 0x559c22dba6a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.946+0000 7fa00b504b80 1 --2- 172.21.15.101:0/1366724203 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c22dbabe0 0x559c22dc5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.946+0000 7fa00b504b80 1 -- 172.21.15.101:0/1366724203 >> 172.21.15.101:0/1366724203 conn(0x559c22db0da0 msgr2=0x559c22db66a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:37.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.946+0000 7fa00b504b80 1 -- 172.21.15.101:0/1366724203 shutdown_connections 2023-07-18T03:09:37.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.946+0000 7fa00b504b80 1 -- 172.21.15.101:0/1366724203 wait complete. 2023-07-18T03:09:37.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.947+0000 7fa00b504b80 1 -- start start 2023-07-18T03:09:37.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.947+0000 7fa00b504b80 1 Processor -- start 2023-07-18T03:09:37.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.947+0000 7fa00b504b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c22dba2d0 0x559c22dc53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:37.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.948+0000 7fa00b504b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c22dbabe0 0x559c22dc2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:37.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.948+0000 7fa00b504b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c22dc5ad0 0x559c22dc2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:37.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.948+0000 7fa00b504b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c22cacac0 con 0x559c22dba2d0 2023-07-18T03:09:37.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.948+0000 7fa00b504b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c22cac7c0 con 0x559c22dbabe0 2023-07-18T03:09:37.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.948+0000 7fa00b504b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c22b90920 con 0x559c22dc5ad0 2023-07-18T03:09:37.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.948+0000 7fa00b504b80 10 client.0 _ll_register_callbacks cb 0x559c22caa760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:09:37.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.948+0000 7f9ff519a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c22dbabe0 0x559c22dc2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:37.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.948+0000 7f9ff599b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c22dba2d0 0x559c22dc53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:37.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.948+0000 7f9ff619c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c22dc5ad0 0x559c22dc2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:37.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.948+0000 7f9ff519a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c22dbabe0 0x559c22dc2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:37898/0 (socket says 172.21.15.101:37898) 2023-07-18T03:09:37.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.948+0000 7f9ff519a700 1 -- 172.21.15.101:0/2059688290 learned_addr learned my addr 172.21.15.101:0/2059688290 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:37.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.948+0000 7f9ff519a700 1 -- 172.21.15.101:0/2059688290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c22dc5ad0 msgr2=0x559c22dc2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:37.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.948+0000 7f9ff519a700 1 --2- 172.21.15.101:0/2059688290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c22dc5ad0 0x559c22dc2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.948+0000 7f9ff519a700 1 -- 172.21.15.101:0/2059688290 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c22dba2d0 msgr2=0x559c22dc53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:37.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.948+0000 7f9ff519a700 1 --2- 172.21.15.101:0/2059688290 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c22dba2d0 0x559c22dc53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.948+0000 7f9ff519a700 1 -- 172.21.15.101:0/2059688290 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ff010d240 con 0x559c22dbabe0 2023-07-18T03:09:37.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.948+0000 7f9ff519a700 1 --2- 172.21.15.101:0/2059688290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c22dbabe0 0x559c22dc2670 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9fe00964b0 tx=0x7f9fe009a030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa8cc049731904e2 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:37.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.948+0000 7f9fd97fa700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:09:37.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.949+0000 7f9fd97fa700 1 -- 172.21.15.101:0/2059688290 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe00a2400 con 0x559c22dbabe0 2023-07-18T03:09:37.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.949+0000 7f9fd97fa700 1 -- 172.21.15.101:0/2059688290 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f9fe009a210 con 0x559c22dbabe0 2023-07-18T03:09:37.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.949+0000 7fa00b504b80 1 -- 172.21.15.101:0/2059688290 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559c22c664f0 con 0x559c22dbabe0 2023-07-18T03:09:37.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.949+0000 7f9fd97fa700 1 -- 172.21.15.101:0/2059688290 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fe009a3d0 con 0x559c22dbabe0 2023-07-18T03:09:37.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.949+0000 7fa00b504b80 10 client.26036 fetch_fsmap learned FSMap version 25 2023-07-18T03:09:37.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.949+0000 7fa00b504b80 1 -- 172.21.15.101:0/2059688290 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559c22b23c60 con 0x559c22dbabe0 2023-07-18T03:09:37.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.949+0000 7f9fd97fa700 1 -- 172.21.15.101:0/2059688290 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559c22c664f0 con 0x559c22dbabe0 2023-07-18T03:09:37.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.949+0000 7f9fd97fa700 1 -- 172.21.15.101:0/2059688290 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f9fe00a2090 con 0x559c22dbabe0 2023-07-18T03:09:37.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.949+0000 7fa00b504b80 10 client.26036 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:09:37.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.949+0000 7fa00b504b80 10 client.26036 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:09:37.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.949+0000 7fa00b504b80 20 client.26036 populate_metadata read hostname 'smithi101' 2023-07-18T03:09:37.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.949+0000 7fa00b504b80 1 --2- 172.21.15.101:0/2059688290 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x559c22dd5270 0x559c22dd7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:37.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.949+0000 7fa00b504b80 4 client.26036 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:09:37.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.949+0000 7fa00b504b80 1 -- 172.21.15.101:0/2059688290 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559c22d05eb0 con 0x559c22dd5270 2023-07-18T03:09:37.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.950+0000 7f9ff599b700 1 --2- 172.21.15.101:0/2059688290 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x559c22dd5270 0x559c22dd7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:37.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.950+0000 7f9fd97fa700 10 client.26036 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:09:37.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.950+0000 7f9ff599b700 1 --2- 172.21.15.101:0/2059688290 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x559c22dd5270 0x559c22dd7680 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5112459fc924d1e server_cookie=16c978ba2f04ae70 in_seq=0 out_seq=0 2023-07-18T03:09:37.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.951+0000 7f9fd97fa700 1 -- 172.21.15.101:0/2059688290 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559c22d05eb0 con 0x559c22dd5270 2023-07-18T03:09:37.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.951+0000 7f9fd97fa700 10 client.26036 handle_command_reply: tid=0 2023-07-18T03:09:37.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.951+0000 7fa00b504b80 1 client.26036 shutdown 2023-07-18T03:09:37.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.951+0000 7fa00b504b80 1 -- 172.21.15.101:0/2059688290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c22dbabe0 msgr2=0x559c22dc2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:37.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.951+0000 7fa00b504b80 1 --2- 172.21.15.101:0/2059688290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c22dbabe0 0x559c22dc2670 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9fe00964b0 tx=0x7f9fe009a030 comp rx=0 tx=0).stop 2023-07-18T03:09:37.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.951+0000 7fa00b504b80 1 -- 172.21.15.101:0/2059688290 shutdown_connections 2023-07-18T03:09:37.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.951+0000 7fa00b504b80 1 --2- 172.21.15.101:0/2059688290 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x559c22dd5270 0x559c22dd7680 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.951+0000 7fa00b504b80 1 --2- 172.21.15.101:0/2059688290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c22dc5ad0 0x559c22dc2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.951+0000 7fa00b504b80 1 --2- 172.21.15.101:0/2059688290 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c22dba2d0 0x559c22dc53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.951+0000 7f9fd97fa700 0 client.26036 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:09:37.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.951+0000 7fa00b504b80 1 --2- 172.21.15.101:0/2059688290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c22dbabe0 0x559c22dc2670 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.951+0000 7fa00b504b80 1 -- 172.21.15.101:0/2059688290 >> 172.21.15.101:0/2059688290 conn(0x559c22db0da0 msgr2=0x559c22db6230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:37.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.951+0000 7fa00b504b80 1 -- 172.21.15.101:0/2059688290 shutdown_connections 2023-07-18T03:09:37.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.951+0000 7fa00b504b80 1 -- 172.21.15.101:0/2059688290 wait complete. 2023-07-18T03:09:37.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.952+0000 7fa00b504b80 20 client.26036 trim_cache size 0 max 16384 2023-07-18T03:09:37.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.969+0000 7fa00b504b80 1 Processor -- start 2023-07-18T03:09:37.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.969+0000 7fa00b504b80 1 -- start start 2023-07-18T03:09:37.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.970+0000 7fa00b504b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c22dba2d0 0x559c22dc4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:37.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.970+0000 7fa00b504b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c22dbabe0 0x559c22dc1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:37.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.970+0000 7fa00b504b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c22dc5ad0 0x559c22dc2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:37.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.970+0000 7fa00b504b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c22dc3ca0 con 0x559c22dbabe0 2023-07-18T03:09:37.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.970+0000 7fa00b504b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c22dbbd90 con 0x559c22dc5ad0 2023-07-18T03:09:37.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.970+0000 7fa00b504b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c22dbc220 con 0x559c22dba2d0 2023-07-18T03:09:37.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.970+0000 7f9ff519a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c22dbabe0 0x559c22dc1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:37.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.970+0000 7f9ff599b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c22dba2d0 0x559c22dc4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:37.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.970+0000 7f9ff619c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c22dc5ad0 0x559c22dc2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:37.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.970+0000 7f9ff519a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c22dbabe0 0x559c22dc1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:43468/0 (socket says 172.21.15.101:43468) 2023-07-18T03:09:37.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.970+0000 7f9ff599b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c22dba2d0 0x559c22dc4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:60746/0 (socket says 172.21.15.101:60746) 2023-07-18T03:09:37.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.970+0000 7f9ff519a700 1 -- 172.21.15.101:0/3120626819 learned_addr learned my addr 172.21.15.101:0/3120626819 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:37.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.970+0000 7f9ff519a700 1 -- 172.21.15.101:0/3120626819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c22dba2d0 msgr2=0x559c22dc4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:37.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.970+0000 7f9ff519a700 1 --2- 172.21.15.101:0/3120626819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c22dba2d0 0x559c22dc4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.970+0000 7f9ff519a700 1 -- 172.21.15.101:0/3120626819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c22dc5ad0 msgr2=0x559c22dc2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:37.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.970+0000 7f9ff519a700 1 --2- 172.21.15.101:0/3120626819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c22dc5ad0 0x559c22dc2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.970+0000 7f9ff519a700 1 -- 172.21.15.101:0/3120626819 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c22b61250 con 0x559c22dbabe0 2023-07-18T03:09:37.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.971+0000 7f9ff519a700 1 --2- 172.21.15.101:0/3120626819 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c22dbabe0 0x559c22dc1e50 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0009230 tx=0x7f9fe009e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f66fb89a67c63b14 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:37.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.971+0000 7f9fe4bff700 1 -- 172.21.15.101:0/3120626819 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe00962f0 con 0x559c22dbabe0 2023-07-18T03:09:37.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.971+0000 7f9fe4bff700 1 -- 172.21.15.101:0/3120626819 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fe00a2400 con 0x559c22dbabe0 2023-07-18T03:09:37.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.971+0000 7f9fe4bff700 1 -- 172.21.15.101:0/3120626819 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe00a1f70 con 0x559c22dbabe0 2023-07-18T03:09:37.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.972+0000 7fa00b504b80 1 -- 172.21.15.101:0/3120626819 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c22dbabe0 msgr2=0x559c22dc1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:37.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.972+0000 7fa00b504b80 1 --2- 172.21.15.101:0/3120626819 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c22dbabe0 0x559c22dc1e50 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0009230 tx=0x7f9fe009e160 comp rx=0 tx=0).stop 2023-07-18T03:09:37.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.972+0000 7fa00b504b80 1 -- 172.21.15.101:0/3120626819 shutdown_connections 2023-07-18T03:09:37.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.972+0000 7fa00b504b80 1 --2- 172.21.15.101:0/3120626819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c22dc5ad0 0x559c22dc2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.972+0000 7fa00b504b80 1 --2- 172.21.15.101:0/3120626819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c22dba2d0 0x559c22dc4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.972+0000 7fa00b504b80 1 --2- 172.21.15.101:0/3120626819 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c22dbabe0 0x559c22dc1e50 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.972+0000 7fa00b504b80 1 -- 172.21.15.101:0/3120626819 >> 172.21.15.101:0/3120626819 conn(0x559c22db0da0 msgr2=0x559c22dd0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:37.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.972+0000 7fa00b504b80 1 -- 172.21.15.101:0/3120626819 shutdown_connections 2023-07-18T03:09:37.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.972+0000 7fa00b504b80 1 -- 172.21.15.101:0/3120626819 wait complete. 2023-07-18T03:09:37.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.972+0000 7fa00b504b80 1 -- start start 2023-07-18T03:09:37.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.972+0000 7fa00b504b80 1 Processor -- start 2023-07-18T03:09:37.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.973+0000 7fa00b504b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c22dba2d0 0x559c22dcc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:37.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.973+0000 7fa00b504b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c22dbabe0 0x559c22dca900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:37.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.973+0000 7fa00b504b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c22dc5ad0 0x559c22dcafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:37.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.973+0000 7fa00b504b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c22dcb670 con 0x559c22dbabe0 2023-07-18T03:09:37.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.973+0000 7fa00b504b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c22e29100 con 0x559c22dba2d0 2023-07-18T03:09:37.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.973+0000 7fa00b504b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c22e29240 con 0x559c22dc5ad0 2023-07-18T03:09:37.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.973+0000 7fa00b504b80 10 client.0 _ll_register_callbacks cb 0x7f9fe0099910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:09:37.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.973+0000 7f9ff519a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c22dbabe0 0x559c22dca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:37.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.973+0000 7f9ff599b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c22dba2d0 0x559c22dcc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:37.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.973+0000 7f9ff599b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c22dba2d0 0x559c22dcc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:37912/0 (socket says 172.21.15.101:37912) 2023-07-18T03:09:37.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.973+0000 7f9ff599b700 1 -- 172.21.15.101:0/3766988408 learned_addr learned my addr 172.21.15.101:0/3766988408 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:09:37.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.973+0000 7f9ff519a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c22dbabe0 0x559c22dca900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:43482/0 (socket says 172.21.15.101:43482) 2023-07-18T03:09:37.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.973+0000 7f9ff619c700 1 --2- 172.21.15.101:0/3766988408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c22dc5ad0 0x559c22dcafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:37.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.974+0000 7f9ff519a700 1 -- 172.21.15.101:0/3766988408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c22dc5ad0 msgr2=0x559c22dcafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:37.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.974+0000 7f9ff519a700 1 --2- 172.21.15.101:0/3766988408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c22dc5ad0 0x559c22dcafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.974+0000 7f9ff519a700 1 -- 172.21.15.101:0/3766988408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c22dba2d0 msgr2=0x559c22dcc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:09:37.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.974+0000 7f9ff519a700 1 --2- 172.21.15.101:0/3766988408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c22dba2d0 0x559c22dcc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.970 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.974+0000 7f9ff519a700 1 -- 172.21.15.101:0/3766988408 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559c22b23c60 con 0x559c22dbabe0 2023-07-18T03:09:37.970 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.974+0000 7f9ff599b700 1 --2- 172.21.15.101:0/3766988408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c22dba2d0 0x559c22dcc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:09:37.970 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.974+0000 7f9ff519a700 1 --2- 172.21.15.101:0/3766988408 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c22dbabe0 0x559c22dca900 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x559c22db5500 tx=0x7f9fe0098560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4361fd1c9f06ebf server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:09:37.970 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.974+0000 7f9fd97fa700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:09:37.970 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.974+0000 7f9fd97fa700 1 -- 172.21.15.101:0/3766988408 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe00962f0 con 0x559c22dbabe0 2023-07-18T03:09:37.970 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.974+0000 7f9fd97fa700 1 -- 172.21.15.101:0/3766988408 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f9fe00a1f70 con 0x559c22dbabe0 2023-07-18T03:09:37.970 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.974+0000 7fa00b504b80 1 -- 172.21.15.101:0/3766988408 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9fec01e040 con 0x559c22dbabe0 2023-07-18T03:09:37.970 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.974+0000 7f9fd97fa700 1 -- 172.21.15.101:0/3766988408 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fe0094c60 con 0x559c22dbabe0 2023-07-18T03:09:37.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.974+0000 7fa00b504b80 10 client.16008 fetch_fsmap learned FSMap version 25 2023-07-18T03:09:37.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.974+0000 7fa00b504b80 1 -- 172.21.15.101:0/3766988408 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559c22b392b0 con 0x559c22dbabe0 2023-07-18T03:09:37.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.975+0000 7f9fd97fa700 1 -- 172.21.15.101:0/3766988408 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fec01e040 con 0x559c22dbabe0 2023-07-18T03:09:37.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.975+0000 7f9fd97fa700 1 -- 172.21.15.101:0/3766988408 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f9fe0098fa0 con 0x559c22dbabe0 2023-07-18T03:09:37.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.975+0000 7fa00b504b80 10 client.16008 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:09:37.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.975+0000 7fa00b504b80 10 client.16008 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:09:37.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.975+0000 7fa00b504b80 20 client.16008 populate_metadata read hostname 'smithi101' 2023-07-18T03:09:37.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.975+0000 7fa00b504b80 1 --2- 172.21.15.101:0/3766988408 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x559c22e2a3f0 0x559c22e2c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:09:37.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.975+0000 7fa00b504b80 4 client.16008 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:09:37.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.975+0000 7fa00b504b80 1 -- 172.21.15.101:0/3766988408 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559c22cc8050 con 0x559c22e2a3f0 2023-07-18T03:09:37.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.975+0000 7f9ff599b700 1 --2- 172.21.15.101:0/3766988408 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x559c22e2a3f0 0x559c22e2c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:09:37.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.976+0000 7f9ff599b700 1 --2- 172.21.15.101:0/3766988408 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x559c22e2a3f0 0x559c22e2c7d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=906b450aa358b437 server_cookie=f9af06b5bd0ae4dc in_seq=0 out_seq=0 2023-07-18T03:09:37.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.976+0000 7f9fd97fa700 10 client.16008 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:09:37.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.976+0000 7f9fd97fa700 1 -- 172.21.15.101:0/3766988408 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x559c22cc8050 con 0x559c22e2a3f0 2023-07-18T03:09:37.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.976+0000 7f9fd97fa700 10 client.16008 handle_command_reply: tid=0 2023-07-18T03:09:37.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.976+0000 7fa00b504b80 1 client.16008 shutdown 2023-07-18T03:09:37.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.976+0000 7fa00b504b80 1 -- 172.21.15.101:0/3766988408 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c22dbabe0 msgr2=0x559c22dca900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:37.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.976+0000 7fa00b504b80 1 --2- 172.21.15.101:0/3766988408 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c22dbabe0 0x559c22dca900 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x559c22db5500 tx=0x7f9fe0098560 comp rx=0 tx=0).stop 2023-07-18T03:09:37.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.976+0000 7fa00b504b80 1 -- 172.21.15.101:0/3766988408 shutdown_connections 2023-07-18T03:09:37.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.976+0000 7fa00b504b80 1 --2- 172.21.15.101:0/3766988408 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x559c22e2a3f0 0x559c22e2c7d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.976+0000 7fa00b504b80 1 --2- 172.21.15.101:0/3766988408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c22dba2d0 0x559c22dcc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.976+0000 7f9fd97fa700 0 client.16008 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:09:37.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.976+0000 7fa00b504b80 1 --2- 172.21.15.101:0/3766988408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c22dc5ad0 0x559c22dcafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.976+0000 7fa00b504b80 1 --2- 172.21.15.101:0/3766988408 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c22dbabe0 0x559c22dca900 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.976+0000 7fa00b504b80 1 -- 172.21.15.101:0/3766988408 >> 172.21.15.101:0/3766988408 conn(0x559c22db0da0 msgr2=0x559c22dbc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:37.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.976+0000 7fa00b504b80 1 -- 172.21.15.101:0/3766988408 shutdown_connections 2023-07-18T03:09:37.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.976+0000 7fa00b504b80 1 -- 172.21.15.101:0/3766988408 wait complete. 2023-07-18T03:09:37.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.977+0000 7fa00b504b80 20 client.16008 trim_cache size 0 max 16384 2023-07-18T03:09:37.975 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:09:37.975 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:09:37.975 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:09:37.975 INFO:teuthology.orchestra.run.smithi101.stdout: "7e6b3aee-b6d8-4cc1-9064-d4034c0e8d33": { 2023-07-18T03:09:37.975 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:09:37.976 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "7e6b3aee-b6d8-4cc1-9064-d4034c0e8d33", 2023-07-18T03:09:37.976 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:09:37.976 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:09:37.976 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:09:37.976 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:09:37.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.977+0000 7f9ff7bff700 1 -- 172.21.15.101:0/3886986565 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9fdc0a12c0 msgr2=0x7f9fdc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:37.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.977+0000 7f9ff7bff700 1 --2- 172.21.15.101:0/3886986565 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9fdc0a12c0 0x7f9fdc0a3770 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8009ec0 tx=0x7f9fe8009920 comp rx=0 tx=0).stop 2023-07-18T03:09:37.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.977+0000 7f9ff7bff700 1 -- 172.21.15.101:0/3886986565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ff01004b0 msgr2=0x7f9ff0072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:09:37.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.977+0000 7f9ff7bff700 1 --2- 172.21.15.101:0/3886986565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ff01004b0 0x7f9ff0072d50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9fe000e9a0 tx=0x7f9fe000ecb0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.977+0000 7f9ff7bff700 1 -- 172.21.15.101:0/3886986565 shutdown_connections 2023-07-18T03:09:37.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.977+0000 7f9ff7bff700 1 --2- 172.21.15.101:0/3886986565 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9fdc0a12c0 0x7f9fdc0a3770 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.977+0000 7f9ff7bff700 1 --2- 172.21.15.101:0/3886986565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ff0105b10 0x7f9ff0073290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.977+0000 7f9ff7bff700 1 --2- 172.21.15.101:0/3886986565 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9ff00ffb00 0x7f9ff0072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.977+0000 7f9ff7bff700 1 --2- 172.21.15.101:0/3886986565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ff01004b0 0x7f9ff0072d50 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:09:37.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.977+0000 7f9ff7bff700 1 -- 172.21.15.101:0/3886986565 >> 172.21.15.101:0/3886986565 conn(0x7f9ff00fb600 msgr2=0x7f9ff00fbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:09:37.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.977+0000 7f9ff7bff700 1 -- 172.21.15.101:0/3886986565 shutdown_connections 2023-07-18T03:09:37.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:09:37.977+0000 7f9ff7bff700 1 -- 172.21.15.101:0/3886986565 wait complete. 2023-07-18T03:09:37.984 INFO:tasks.cephfs.filesystem:scrub status for tag:7e6b3aee-b6d8-4cc1-9064-d4034c0e8d33 - {'path': '/', 'tag': '7e6b3aee-b6d8-4cc1-9064-d4034c0e8d33', 'options': 'recursive,force'} 2023-07-18T03:09:38.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:37 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:09:38.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:37 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:09:38.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:37 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:09:38.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:37 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:09:38.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:37 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:09:38.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:37 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:09:38.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:37 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:09:38.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:37 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:09:38.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:37 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:09:38.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:37 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:09:38.235 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:37 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:09:38.235 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:37 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:09:38.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:37 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:09:38.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:37 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:09:38.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:37 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:09:38.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:37 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:09:38.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:37 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:09:38.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:37 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:09:38.565 INFO:tasks.workunit.client.0.smithi101.stdout: 1 81790 7.29 MB/sec warmup 63 sec latency 15.372 ms 2023-07-18T03:09:39.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:38 smithi179 ceph-mon[129229]: pgmap v65: 129 pgs: 129 active+clean; 54 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 395 KiB/s wr, 72 op/s 2023-07-18T03:09:39.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:38 smithi179 ceph-mon[129229]: scrub queued for path: / 2023-07-18T03:09:39.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:38 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:09:39.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:38 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:09:39.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:38 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:09:39.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:38 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:09:39.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:38 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:09:39.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:38 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:09:39.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:38 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:09:39.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:38 smithi101 ceph-mon[126379]: pgmap v65: 129 pgs: 129 active+clean; 54 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 395 KiB/s wr, 72 op/s 2023-07-18T03:09:39.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:38 smithi101 ceph-mon[126379]: scrub queued for path: / 2023-07-18T03:09:39.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:38 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:09:39.235 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:38 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:09:39.235 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:38 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:09:39.235 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:38 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:09:39.235 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:38 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:09:39.235 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:38 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:09:39.235 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:38 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:09:39.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:38 smithi155 ceph-mon[128885]: pgmap v65: 129 pgs: 129 active+clean; 54 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 395 KiB/s wr, 72 op/s 2023-07-18T03:09:39.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:38 smithi155 ceph-mon[128885]: scrub queued for path: / 2023-07-18T03:09:39.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:38 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:09:39.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:38 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:09:39.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:38 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:09:39.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:38 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:09:39.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:38 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:09:39.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:38 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:09:39.293 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:38 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:09:39.565 INFO:tasks.workunit.client.0.smithi101.stdout: 1 82874 7.23 MB/sec warmup 64 sec latency 13.306 ms 2023-07-18T03:09:40.565 INFO:tasks.workunit.client.0.smithi101.stdout: 1 85410 7.38 MB/sec warmup 65 sec latency 11.550 ms 2023-07-18T03:09:41.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:40 smithi179 ceph-mon[129229]: pgmap v66: 129 pgs: 129 active+clean; 59 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 947 KiB/s wr, 161 op/s 2023-07-18T03:09:41.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:40 smithi101 ceph-mon[126379]: pgmap v66: 129 pgs: 129 active+clean; 59 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 947 KiB/s wr, 161 op/s 2023-07-18T03:09:41.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:40 smithi155 ceph-mon[128885]: pgmap v66: 129 pgs: 129 active+clean; 59 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 947 KiB/s wr, 161 op/s 2023-07-18T03:09:41.566 INFO:tasks.workunit.client.0.smithi101.stdout: 1 86735 7.35 MB/sec warmup 66 sec latency 23.701 ms 2023-07-18T03:09:42.566 INFO:tasks.workunit.client.0.smithi101.stdout: 1 88967 7.44 MB/sec warmup 67 sec latency 37.327 ms 2023-07-18T03:09:43.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:42 smithi179 ceph-mon[129229]: pgmap v67: 129 pgs: 129 active+clean; 59 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 941 KiB/s wr, 156 op/s 2023-07-18T03:09:43.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:42 smithi101 ceph-mon[126379]: pgmap v67: 129 pgs: 129 active+clean; 59 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 941 KiB/s wr, 156 op/s 2023-07-18T03:09:43.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:42 smithi155 ceph-mon[128885]: pgmap v67: 129 pgs: 129 active+clean; 59 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 941 KiB/s wr, 156 op/s 2023-07-18T03:09:43.566 INFO:tasks.workunit.client.0.smithi101.stdout: 1 90394 7.42 MB/sec warmup 68 sec latency 14.575 ms 2023-07-18T03:09:44.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:43 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:09:44.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:43 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:09:44.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:43 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:09:44.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:43 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:09:44.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:43 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:09:44.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:43 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:09:44.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:43 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:09:44.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:43 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:09:44.235 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:43 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:09:44.235 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:43 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:09:44.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:43 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:09:44.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:43 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:09:44.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:43 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:09:44.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:43 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:09:44.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:43 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:09:44.566 INFO:tasks.workunit.client.0.smithi101.stdout: 1 92503 7.51 MB/sec warmup 69 sec latency 18.787 ms 2023-07-18T03:09:45.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:44 smithi179 ceph-mon[129229]: pgmap v68: 129 pgs: 129 active+clean; 59 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 941 KiB/s wr, 156 op/s 2023-07-18T03:09:45.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:44 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:09:45.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:44 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:09:45.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:44 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:09:45.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:44 smithi101 ceph-mon[126379]: pgmap v68: 129 pgs: 129 active+clean; 59 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 941 KiB/s wr, 156 op/s 2023-07-18T03:09:45.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:44 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:09:45.235 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:44 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:09:45.235 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:44 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:09:45.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:44 smithi155 ceph-mon[128885]: pgmap v68: 129 pgs: 129 active+clean; 59 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 941 KiB/s wr, 156 op/s 2023-07-18T03:09:45.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:44 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:09:45.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:44 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:09:45.293 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:44 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:09:45.566 INFO:tasks.workunit.client.0.smithi101.stdout: 1 93799 7.48 MB/sec warmup 70 sec latency 14.621 ms 2023-07-18T03:09:46.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:45 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:09:46.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:45 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:09:46.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:45 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:09:46.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:45 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:09:46.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:45 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:09:46.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:45 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:09:46.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:45 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:09:46.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:45 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:09:46.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:45 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:09:46.568 INFO:tasks.workunit.client.0.smithi101.stdout: 1 96074 7.56 MB/sec warmup 71 sec latency 27.572 ms 2023-07-18T03:09:47.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:46 smithi179 ceph-mon[129229]: pgmap v69: 129 pgs: 129 active+clean; 64 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 243 op/s 2023-07-18T03:09:47.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:46 smithi101 ceph-mon[126379]: pgmap v69: 129 pgs: 129 active+clean; 64 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 243 op/s 2023-07-18T03:09:47.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:46 smithi155 ceph-mon[128885]: pgmap v69: 129 pgs: 129 active+clean; 64 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 243 op/s 2023-07-18T03:09:47.566 INFO:tasks.workunit.client.0.smithi101.stdout: 1 97346 7.53 MB/sec warmup 72 sec latency 20.899 ms 2023-07-18T03:09:48.566 INFO:tasks.workunit.client.0.smithi101.stdout: 1 99602 7.62 MB/sec warmup 73 sec latency 28.493 ms 2023-07-18T03:09:49.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:48 smithi179 ceph-mon[129229]: pgmap v70: 129 pgs: 129 active+clean; 64 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 175 op/s 2023-07-18T03:09:49.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:48 smithi101 ceph-mon[126379]: pgmap v70: 129 pgs: 129 active+clean; 64 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 175 op/s 2023-07-18T03:09:49.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:48 smithi155 ceph-mon[128885]: pgmap v70: 129 pgs: 129 active+clean; 64 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 175 op/s 2023-07-18T03:09:49.566 INFO:tasks.workunit.client.0.smithi101.stdout: 1 100891 7.59 MB/sec warmup 74 sec latency 21.741 ms 2023-07-18T03:09:50.566 INFO:tasks.workunit.client.0.smithi101.stdout: 1 103172 7.67 MB/sec warmup 75 sec latency 11.008 ms 2023-07-18T03:09:51.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:50 smithi179 ceph-mon[129229]: pgmap v71: 129 pgs: 129 active+clean; 69 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 270 op/s 2023-07-18T03:09:51.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:50 smithi101 ceph-mon[126379]: pgmap v71: 129 pgs: 129 active+clean; 69 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 270 op/s 2023-07-18T03:09:51.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:50 smithi155 ceph-mon[128885]: pgmap v71: 129 pgs: 129 active+clean; 69 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 270 op/s 2023-07-18T03:09:51.566 INFO:tasks.workunit.client.0.smithi101.stdout: 1 104548 7.64 MB/sec warmup 76 sec latency 14.400 ms 2023-07-18T03:09:52.567 INFO:tasks.workunit.client.0.smithi101.stdout: 1 106707 7.72 MB/sec warmup 77 sec latency 23.410 ms 2023-07-18T03:09:53.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:52 smithi179 ceph-mon[129229]: pgmap v72: 129 pgs: 129 active+clean; 69 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 1.2 MiB/s wr, 181 op/s 2023-07-18T03:09:53.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:52 smithi101 ceph-mon[126379]: pgmap v72: 129 pgs: 129 active+clean; 69 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 1.2 MiB/s wr, 181 op/s 2023-07-18T03:09:53.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:52 smithi155 ceph-mon[128885]: pgmap v72: 129 pgs: 129 active+clean; 69 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 1.2 MiB/s wr, 181 op/s 2023-07-18T03:09:53.567 INFO:tasks.workunit.client.0.smithi101.stdout: 1 108118 7.69 MB/sec warmup 78 sec latency 14.802 ms 2023-07-18T03:09:54.567 INFO:tasks.workunit.client.0.smithi101.stdout: 1 110233 7.77 MB/sec warmup 79 sec latency 31.331 ms 2023-07-18T03:09:55.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:54 smithi179 ceph-mon[129229]: pgmap v73: 129 pgs: 129 active+clean; 69 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 1.2 MiB/s wr, 181 op/s 2023-07-18T03:09:55.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:54 smithi101 ceph-mon[126379]: pgmap v73: 129 pgs: 129 active+clean; 69 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 1.2 MiB/s wr, 181 op/s 2023-07-18T03:09:55.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:54 smithi155 ceph-mon[128885]: pgmap v73: 129 pgs: 129 active+clean; 69 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 1.2 MiB/s wr, 181 op/s 2023-07-18T03:09:55.567 INFO:tasks.workunit.client.0.smithi101.stdout: 1 111541 7.74 MB/sec warmup 80 sec latency 38.157 ms 2023-07-18T03:09:56.567 INFO:tasks.workunit.client.0.smithi101.stdout: 1 113788 7.81 MB/sec warmup 81 sec latency 58.013 ms 2023-07-18T03:09:57.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:56 smithi179 ceph-mon[129229]: pgmap v74: 129 pgs: 129 active+clean; 74 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 272 op/s 2023-07-18T03:09:57.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:56 smithi101 ceph-mon[126379]: pgmap v74: 129 pgs: 129 active+clean; 74 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 272 op/s 2023-07-18T03:09:57.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:56 smithi155 ceph-mon[128885]: pgmap v74: 129 pgs: 129 active+clean; 74 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 272 op/s 2023-07-18T03:09:57.567 INFO:tasks.workunit.client.0.smithi101.stdout: 1 114795 7.74 MB/sec warmup 82 sec latency 97.754 ms 2023-07-18T03:09:58.567 INFO:tasks.workunit.client.0.smithi101.stdout: 1 117328 7.85 MB/sec warmup 83 sec latency 10.326 ms 2023-07-18T03:09:59.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:09:58 smithi179 ceph-mon[129229]: pgmap v75: 129 pgs: 129 active+clean; 74 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 1.4 MiB/s wr, 185 op/s 2023-07-18T03:09:59.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:09:58 smithi101 ceph-mon[126379]: pgmap v75: 129 pgs: 129 active+clean; 74 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 1.4 MiB/s wr, 185 op/s 2023-07-18T03:09:59.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:09:58 smithi155 ceph-mon[128885]: pgmap v75: 129 pgs: 129 active+clean; 74 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 1.4 MiB/s wr, 185 op/s 2023-07-18T03:09:59.567 INFO:tasks.workunit.client.0.smithi101.stdout: 1 118277 7.79 MB/sec warmup 84 sec latency 32.515 ms 2023-07-18T03:10:00.567 INFO:tasks.workunit.client.0.smithi101.stdout: 1 120334 7.83 MB/sec warmup 85 sec latency 58.330 ms 2023-07-18T03:10:01.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:00 smithi179 ceph-mon[129229]: pgmap v76: 129 pgs: 129 active+clean; 79 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 272 op/s 2023-07-18T03:10:01.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:00 smithi179 ceph-mon[129229]: overall HEALTH_OK 2023-07-18T03:10:01.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:00 smithi101 ceph-mon[126379]: pgmap v76: 129 pgs: 129 active+clean; 79 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 272 op/s 2023-07-18T03:10:01.235 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:00 smithi101 ceph-mon[126379]: overall HEALTH_OK 2023-07-18T03:10:01.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:00 smithi155 ceph-mon[128885]: pgmap v76: 129 pgs: 129 active+clean; 79 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 272 op/s 2023-07-18T03:10:01.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:00 smithi155 ceph-mon[128885]: overall HEALTH_OK 2023-07-18T03:10:01.567 INFO:tasks.workunit.client.0.smithi101.stdout: 1 121009 7.80 MB/sec warmup 86 sec latency 57.457 ms 2023-07-18T03:10:02.568 INFO:tasks.workunit.client.0.smithi101.stdout: 1 122993 7.79 MB/sec warmup 87 sec latency 16.464 ms 2023-07-18T03:10:03.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:02 smithi101 ceph-mon[126379]: pgmap v77: 129 pgs: 129 active+clean; 79 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.2 MiB/s wr, 176 op/s 2023-07-18T03:10:03.291 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:02 smithi155 ceph-mon[128885]: pgmap v77: 129 pgs: 129 active+clean; 79 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.2 MiB/s wr, 176 op/s 2023-07-18T03:10:03.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:02 smithi179 ceph-mon[129229]: pgmap v77: 129 pgs: 129 active+clean; 79 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.2 MiB/s wr, 176 op/s 2023-07-18T03:10:03.568 INFO:tasks.workunit.client.0.smithi101.stdout: 1 124588 7.84 MB/sec warmup 88 sec latency 10.537 ms 2023-07-18T03:10:04.568 INFO:tasks.workunit.client.0.smithi101.stdout: 1 126546 7.83 MB/sec warmup 89 sec latency 40.836 ms 2023-07-18T03:10:05.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:04 smithi101 ceph-mon[126379]: pgmap v78: 129 pgs: 129 active+clean; 79 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.2 MiB/s wr, 176 op/s 2023-07-18T03:10:05.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:04 smithi155 ceph-mon[128885]: pgmap v78: 129 pgs: 129 active+clean; 79 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.2 MiB/s wr, 176 op/s 2023-07-18T03:10:05.446 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:04 smithi179 ceph-mon[129229]: pgmap v78: 129 pgs: 129 active+clean; 79 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.2 MiB/s wr, 176 op/s 2023-07-18T03:10:05.568 INFO:tasks.workunit.client.0.smithi101.stdout: 1 128119 7.88 MB/sec warmup 90 sec latency 11.351 ms 2023-07-18T03:10:06.568 INFO:tasks.workunit.client.0.smithi101.stdout: 1 130135 7.87 MB/sec warmup 91 sec latency 14.784 ms 2023-07-18T03:10:07.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:06 smithi101 ceph-mon[126379]: pgmap v79: 129 pgs: 129 active+clean; 83 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 287 op/s 2023-07-18T03:10:07.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:06 smithi155 ceph-mon[128885]: pgmap v79: 129 pgs: 129 active+clean; 83 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 287 op/s 2023-07-18T03:10:07.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:06 smithi179 ceph-mon[129229]: pgmap v79: 129 pgs: 129 active+clean; 83 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 287 op/s 2023-07-18T03:10:07.568 INFO:tasks.workunit.client.0.smithi101.stdout: 1 131619 7.92 MB/sec warmup 92 sec latency 42.548 ms 2023-07-18T03:10:07.985 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:10:08.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.220+0000 7fecd4b9d700 1 -- 172.21.15.101:0/1184332636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecd0073160 msgr2=0x7fecd010bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:08.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.220+0000 7fecd4b9d700 1 --2- 172.21.15.101:0/1184332636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecd0073160 0x7fecd010bd50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fecc4009990 tx=0x7fecc4009ca0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.220+0000 7fecd4b9d700 1 -- 172.21.15.101:0/1184332636 shutdown_connections 2023-07-18T03:10:08.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.220+0000 7fecd4b9d700 1 --2- 172.21.15.101:0/1184332636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fecd010c290 0x7fecd010e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.220+0000 7fecd4b9d700 1 --2- 172.21.15.101:0/1184332636 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fecd0072850 0x7fecd0072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.220+0000 7fecd4b9d700 1 --2- 172.21.15.101:0/1184332636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecd0073160 0x7fecd010bd50 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.220+0000 7fecd4b9d700 1 -- 172.21.15.101:0/1184332636 >> 172.21.15.101:0/1184332636 conn(0x7fecd00fb510 msgr2=0x7fecd00fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:08.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.220+0000 7fecd4b9d700 1 -- 172.21.15.101:0/1184332636 shutdown_connections 2023-07-18T03:10:08.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.220+0000 7fecd4b9d700 1 -- 172.21.15.101:0/1184332636 wait complete. 2023-07-18T03:10:08.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.222+0000 7fecd4b9d700 1 Processor -- start 2023-07-18T03:10:08.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.222+0000 7fecd4b9d700 1 -- start start 2023-07-18T03:10:08.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.223+0000 7fecd4b9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fecd0072850 0x7fecd019da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:08.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.223+0000 7fecd4b9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecd0073160 0x7fecd019df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:08.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.223+0000 7fecd4b9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fecd010c290 0x7fecd0197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:08.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.223+0000 7fecd4b9d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd0198120 con 0x7fecd010c290 2023-07-18T03:10:08.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.223+0000 7fecd4b9d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd0198290 con 0x7fecd0072850 2023-07-18T03:10:08.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.223+0000 7fecd4b9d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd0198450 con 0x7fecd0073160 2023-07-18T03:10:08.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.223+0000 7feccf7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fecd0072850 0x7fecd019da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:08.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.223+0000 7feccf7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fecd0072850 0x7fecd019da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:32820/0 (socket says 172.21.15.101:32820) 2023-07-18T03:10:08.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.223+0000 7feccf7fe700 1 -- 172.21.15.101:0/3220500409 learned_addr learned my addr 172.21.15.101:0/3220500409 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:08.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.223+0000 7fecceffd700 1 --2- 172.21.15.101:0/3220500409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecd0073160 0x7fecd019df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:08.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.223+0000 7feccffff700 1 --2- 172.21.15.101:0/3220500409 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fecd010c290 0x7fecd0197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:08.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.223+0000 7feccf7fe700 1 -- 172.21.15.101:0/3220500409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecd0073160 msgr2=0x7fecd019df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:08.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.224+0000 7feccf7fe700 1 --2- 172.21.15.101:0/3220500409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecd0073160 0x7fecd019df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.224+0000 7feccf7fe700 1 -- 172.21.15.101:0/3220500409 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fecd010c290 msgr2=0x7fecd0197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:08.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.224+0000 7feccf7fe700 1 --2- 172.21.15.101:0/3220500409 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fecd010c290 0x7fecd0197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.224+0000 7feccf7fe700 1 -- 172.21.15.101:0/3220500409 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc0009520 con 0x7fecd0072850 2023-07-18T03:10:08.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.224+0000 7feccffff700 1 --2- 172.21.15.101:0/3220500409 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fecd010c290 0x7fecd0197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:10:08.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.224+0000 7feccf7fe700 1 --2- 172.21.15.101:0/3220500409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fecd0072850 0x7fecd019da40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fecc000eb30 tx=0x7fecc000ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:08.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.224+0000 7fecccff9700 1 -- 172.21.15.101:0/3220500409 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc0009880 con 0x7fecd0072850 2023-07-18T03:10:08.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.224+0000 7fecccff9700 1 -- 172.21.15.101:0/3220500409 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fecc00044e0 con 0x7fecd0072850 2023-07-18T03:10:08.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.224+0000 7fecccff9700 1 -- 172.21.15.101:0/3220500409 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc0003e90 con 0x7fecd0072850 2023-07-18T03:10:08.218 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.224+0000 7fecd4b9d700 1 -- 172.21.15.101:0/3220500409 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecc40095f0 con 0x7fecd0072850 2023-07-18T03:10:08.218 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.224+0000 7fecd4b9d700 1 -- 172.21.15.101:0/3220500409 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecd01a4850 con 0x7fecd0072850 2023-07-18T03:10:08.218 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.228+0000 7fecccff9700 1 -- 172.21.15.101:0/3220500409 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fecc0004000 con 0x7fecd0072850 2023-07-18T03:10:08.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.229+0000 7fece9739b80 1 Processor -- start 2023-07-18T03:10:08.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.229+0000 7fece9739b80 1 -- start start 2023-07-18T03:10:08.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.229+0000 7fecccff9700 1 --2- 172.21.15.101:0/3220500409 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fecb80a1360 0x7fecb80a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:08.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.230+0000 7fece9739b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb4055d4f0 0x55fb4055d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:08.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.230+0000 7fece9739b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb4055de00 0x55fb405677f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:08.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.230+0000 7fece9739b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb40567d30 0x55fb4056a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:08.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.230+0000 7fecccff9700 1 -- 172.21.15.101:0/3220500409 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fecc0014070 con 0x7fecd0072850 2023-07-18T03:10:08.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.230+0000 7fece9739b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb40462cb0 con 0x55fb40567d30 2023-07-18T03:10:08.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.230+0000 7fece9739b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb404b5a40 con 0x55fb4055d4f0 2023-07-18T03:10:08.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.230+0000 7fece9739b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb404aeed0 con 0x55fb4055de00 2023-07-18T03:10:08.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.230+0000 7feccffff700 1 --2- 172.21.15.101:0/3220500409 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fecb80a1360 0x7fecb80a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:08.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.230+0000 7feccf7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb4055d4f0 0x55fb4055d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:08.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.230+0000 7feccf7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb4055d4f0 0x55fb4055d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:32826/0 (socket says 172.21.15.101:32826) 2023-07-18T03:10:08.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.230+0000 7feccf7fe700 1 -- 172.21.15.101:0/3281937870 learned_addr learned my addr 172.21.15.101:0/3281937870 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:08.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.230+0000 7fecceffd700 1 --2- 172.21.15.101:0/3281937870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb4055de00 0x55fb405677f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:08.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.230+0000 7feccffff700 1 --2- 172.21.15.101:0/3281937870 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb40567d30 0x55fb4056a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:08.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.230+0000 7feccf7fe700 1 -- 172.21.15.101:0/3281937870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb4055de00 msgr2=0x55fb405677f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:08.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.230+0000 7feccf7fe700 1 --2- 172.21.15.101:0/3281937870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb4055de00 0x55fb405677f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.230+0000 7feccf7fe700 1 -- 172.21.15.101:0/3281937870 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb40567d30 msgr2=0x55fb4056a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:08.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.230+0000 7feccf7fe700 1 --2- 172.21.15.101:0/3281937870 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb40567d30 0x55fb4056a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.230+0000 7feccf7fe700 1 -- 172.21.15.101:0/3281937870 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb402dcde0 con 0x55fb4055d4f0 2023-07-18T03:10:08.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.230+0000 7feccffff700 1 --2- 172.21.15.101:0/3281937870 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb40567d30 0x55fb4056a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:10:08.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.231+0000 7feccf7fe700 1 --2- 172.21.15.101:0/3281937870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb4055d4f0 0x55fb4055d8c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fecc00a21a0 tx=0x7fecc0099e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e1bd0002f7c2b3f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:08.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.231+0000 7feccffff700 1 --2- 172.21.15.101:0/3220500409 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fecb80a1360 0x7fecb80a3810 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fecd019e740 tx=0x7fecc8009920 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:08.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.231+0000 7fecb5ffb700 1 -- 172.21.15.101:0/3281937870 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc000cc90 con 0x55fb4055d4f0 2023-07-18T03:10:08.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.231+0000 7fecb5ffb700 1 -- 172.21.15.101:0/3281937870 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fecc009a280 con 0x55fb4055d4f0 2023-07-18T03:10:08.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.231+0000 7fecb5ffb700 1 -- 172.21.15.101:0/3281937870 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc00cd170 con 0x55fb4055d4f0 2023-07-18T03:10:08.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.232+0000 7fece9739b80 1 -- 172.21.15.101:0/3281937870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb4055d4f0 msgr2=0x55fb4055d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:08.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.232+0000 7fece9739b80 1 --2- 172.21.15.101:0/3281937870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb4055d4f0 0x55fb4055d8c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fecc00a21a0 tx=0x7fecc0099e40 comp rx=0 tx=0).stop 2023-07-18T03:10:08.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.232+0000 7fece9739b80 1 -- 172.21.15.101:0/3281937870 shutdown_connections 2023-07-18T03:10:08.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.232+0000 7fece9739b80 1 --2- 172.21.15.101:0/3281937870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb4055d4f0 0x55fb4055d8c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.232+0000 7fece9739b80 1 --2- 172.21.15.101:0/3281937870 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb40567d30 0x55fb4056a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.232+0000 7fece9739b80 1 --2- 172.21.15.101:0/3281937870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb4055de00 0x55fb405677f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.232+0000 7fece9739b80 1 -- 172.21.15.101:0/3281937870 >> 172.21.15.101:0/3281937870 conn(0x55fb40552ff0 msgr2=0x55fb40558930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:08.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.232+0000 7fece9739b80 1 -- 172.21.15.101:0/3281937870 shutdown_connections 2023-07-18T03:10:08.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.232+0000 7fece9739b80 1 -- 172.21.15.101:0/3281937870 wait complete. 2023-07-18T03:10:08.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.232+0000 7fece9739b80 1 -- start start 2023-07-18T03:10:08.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.233+0000 7fece9739b80 1 Processor -- start 2023-07-18T03:10:08.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.233+0000 7fece9739b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb4055d4f0 0x55fb40567740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:08.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.233+0000 7fece9739b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb4055de00 0x55fb405649c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:08.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.233+0000 7fece9739b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb40567d30 0x55fb40565060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:08.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.233+0000 7fece9739b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb4044eac0 con 0x55fb4055de00 2023-07-18T03:10:08.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.233+0000 7fece9739b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb4044e7c0 con 0x55fb4055d4f0 2023-07-18T03:10:08.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.233+0000 7fece9739b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb40332920 con 0x55fb40567d30 2023-07-18T03:10:08.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.233+0000 7fece9739b80 10 client.0 _ll_register_callbacks cb 0x55fb4044c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:10:08.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.234+0000 7fecceffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb4055de00 0x55fb405649c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:08.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.234+0000 7feccffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb40567d30 0x55fb40565060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:08.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.234+0000 7fecceffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb4055de00 0x55fb405649c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:44618/0 (socket says 172.21.15.101:44618) 2023-07-18T03:10:08.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.234+0000 7fecceffd700 1 -- 172.21.15.101:0/2998303409 learned_addr learned my addr 172.21.15.101:0/2998303409 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:08.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.234+0000 7feccffff700 1 --2- 172.21.15.101:0/2998303409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb40567d30 0x55fb40565060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:50476/0 (socket says 172.21.15.101:50476) 2023-07-18T03:10:08.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.234+0000 7feccf7fe700 1 --2- 172.21.15.101:0/2998303409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb4055d4f0 0x55fb40567740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:08.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.234+0000 7fecceffd700 1 -- 172.21.15.101:0/2998303409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb40567d30 msgr2=0x55fb40565060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:08.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.234+0000 7fecceffd700 1 --2- 172.21.15.101:0/2998303409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb40567d30 0x55fb40565060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.234+0000 7fecceffd700 1 -- 172.21.15.101:0/2998303409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb4055d4f0 msgr2=0x55fb40567740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:08.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.234+0000 7fecceffd700 1 --2- 172.21.15.101:0/2998303409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb4055d4f0 0x55fb40567740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.234+0000 7fecceffd700 1 -- 172.21.15.101:0/2998303409 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecd01a4850 con 0x55fb4055de00 2023-07-18T03:10:08.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.234+0000 7feccf7fe700 1 --2- 172.21.15.101:0/2998303409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb4055d4f0 0x55fb40567740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:10:08.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.234+0000 7fecceffd700 1 --2- 172.21.15.101:0/2998303409 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb4055de00 0x55fb405649c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fecc4015000 tx=0x7fecc4009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa9196264ded9522 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:08.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.234+0000 7fecaeffd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:10:08.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.234+0000 7fecaeffd700 1 -- 172.21.15.101:0/2998303409 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc401d030 con 0x55fb4055de00 2023-07-18T03:10:08.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.234+0000 7fece9739b80 1 -- 172.21.15.101:0/2998303409 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fb404084f0 con 0x55fb4055de00 2023-07-18T03:10:08.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.234+0000 7fecaeffd700 1 -- 172.21.15.101:0/2998303409 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fecc4004500 con 0x55fb4055de00 2023-07-18T03:10:08.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.235+0000 7fecaeffd700 1 -- 172.21.15.101:0/2998303409 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fecc402d7b0 con 0x55fb4055de00 2023-07-18T03:10:08.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.235+0000 7fece9739b80 10 client.16020 fetch_fsmap learned FSMap version 25 2023-07-18T03:10:08.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.235+0000 7fece9739b80 1 -- 172.21.15.101:0/2998303409 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55fb402c5c60 con 0x55fb4055de00 2023-07-18T03:10:08.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.235+0000 7fecaeffd700 1 -- 172.21.15.101:0/2998303409 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fb404084f0 con 0x55fb4055de00 2023-07-18T03:10:08.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.235+0000 7fecaeffd700 1 -- 172.21.15.101:0/2998303409 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fecc4004920 con 0x55fb4055de00 2023-07-18T03:10:08.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.235+0000 7fece9739b80 10 client.16020 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:10:08.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.235+0000 7fece9739b80 10 client.16020 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:10:08.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.235+0000 7fece9739b80 20 client.16020 populate_metadata read hostname 'smithi101' 2023-07-18T03:10:08.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.235+0000 7fece9739b80 1 --2- 172.21.15.101:0/2998303409 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55fb40577850 0x55fb40579c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:08.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.235+0000 7fece9739b80 4 client.16020 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:10:08.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.235+0000 7fece9739b80 1 -- 172.21.15.101:0/2998303409 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fb404a7eb0 con 0x55fb40577850 2023-07-18T03:10:08.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.236+0000 7feccf7fe700 1 --2- 172.21.15.101:0/2998303409 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55fb40577850 0x55fb40579c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:08.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.236+0000 7feccf7fe700 1 --2- 172.21.15.101:0/2998303409 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55fb40577850 0x55fb40579c60 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48be6b2e77e4d69 server_cookie=da94bd5ccbe8c495 in_seq=0 out_seq=0 2023-07-18T03:10:08.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.236+0000 7fecaeffd700 10 client.16020 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:10:08.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.236+0000 7fecaeffd700 1 -- 172.21.15.101:0/2998303409 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55fb404a7eb0 con 0x55fb40577850 2023-07-18T03:10:08.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.237+0000 7fecaeffd700 10 client.16020 handle_command_reply: tid=0 2023-07-18T03:10:08.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.237+0000 7fece9739b80 1 client.16020 shutdown 2023-07-18T03:10:08.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.237+0000 7fece9739b80 1 -- 172.21.15.101:0/2998303409 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb4055de00 msgr2=0x55fb405649c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:08.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.237+0000 7fece9739b80 1 --2- 172.21.15.101:0/2998303409 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb4055de00 0x55fb405649c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fecc4015000 tx=0x7fecc4009570 comp rx=0 tx=0).stop 2023-07-18T03:10:08.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.237+0000 7fece9739b80 1 -- 172.21.15.101:0/2998303409 shutdown_connections 2023-07-18T03:10:08.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.237+0000 7fece9739b80 1 --2- 172.21.15.101:0/2998303409 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55fb40577850 0x55fb40579c60 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.237+0000 7fece9739b80 1 --2- 172.21.15.101:0/2998303409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb4055d4f0 0x55fb40567740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.237+0000 7fece9739b80 1 --2- 172.21.15.101:0/2998303409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb40567d30 0x55fb40565060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.237+0000 7fecaeffd700 0 client.16020 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:10:08.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.237+0000 7fece9739b80 1 --2- 172.21.15.101:0/2998303409 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb4055de00 0x55fb405649c0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.237+0000 7fece9739b80 1 -- 172.21.15.101:0/2998303409 >> 172.21.15.101:0/2998303409 conn(0x55fb40552ff0 msgr2=0x55fb40558520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:08.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.237+0000 7fece9739b80 1 -- 172.21.15.101:0/2998303409 shutdown_connections 2023-07-18T03:10:08.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.237+0000 7fece9739b80 1 -- 172.21.15.101:0/2998303409 wait complete. 2023-07-18T03:10:08.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.237+0000 7fece9739b80 20 client.16020 trim_cache size 0 max 16384 2023-07-18T03:10:08.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.257+0000 7fece9739b80 1 Processor -- start 2023-07-18T03:10:08.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.257+0000 7fece9739b80 1 -- start start 2023-07-18T03:10:08.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.257+0000 7fece9739b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb4055d4f0 0x55fb40567370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:08.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.258+0000 7fece9739b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb4055de00 0x55fb4055fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:08.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.258+0000 7fece9739b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb40567d30 0x55fb40560230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:08.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.258+0000 7fece9739b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb40574e40 con 0x55fb4055d4f0 2023-07-18T03:10:08.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.258+0000 7fece9739b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb4055dc40 con 0x55fb4055de00 2023-07-18T03:10:08.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.258+0000 7fece9739b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb40562f20 con 0x55fb40567d30 2023-07-18T03:10:08.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.258+0000 7feccf7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb4055d4f0 0x55fb40567370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:08.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.258+0000 7feccffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb40567d30 0x55fb40560230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:08.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.258+0000 7feccffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb40567d30 0x55fb40560230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:50478/0 (socket says 172.21.15.101:50478) 2023-07-18T03:10:08.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.258+0000 7feccffff700 1 -- 172.21.15.101:0/2280266129 learned_addr learned my addr 172.21.15.101:0/2280266129 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:08.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.258+0000 7fecceffd700 1 --2- 172.21.15.101:0/2280266129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb4055de00 0x55fb4055fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:08.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.258+0000 7feccf7fe700 1 -- 172.21.15.101:0/2280266129 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb40567d30 msgr2=0x55fb40560230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:08.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.258+0000 7feccf7fe700 1 --2- 172.21.15.101:0/2280266129 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb40567d30 0x55fb40560230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.258+0000 7feccf7fe700 1 -- 172.21.15.101:0/2280266129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb4055de00 msgr2=0x55fb4055fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:08.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.258+0000 7feccf7fe700 1 --2- 172.21.15.101:0/2280266129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb4055de00 0x55fb4055fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.258+0000 7feccf7fe700 1 -- 172.21.15.101:0/2280266129 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb40303250 con 0x55fb4055d4f0 2023-07-18T03:10:08.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.258+0000 7feccf7fe700 1 --2- 172.21.15.101:0/2280266129 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb4055d4f0 0x55fb40567370 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fecc000eb00 tx=0x7fecc0099c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf182a213a95a3a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:08.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.259+0000 7fecb67fc700 1 -- 172.21.15.101:0/2280266129 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc000cc90 con 0x55fb4055d4f0 2023-07-18T03:10:08.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.259+0000 7fecb67fc700 1 -- 172.21.15.101:0/2280266129 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fecc00a26d0 con 0x55fb4055d4f0 2023-07-18T03:10:08.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.259+0000 7fecb67fc700 1 -- 172.21.15.101:0/2280266129 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc00cd3d0 con 0x55fb4055d4f0 2023-07-18T03:10:08.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.259+0000 7fece9739b80 1 -- 172.21.15.101:0/2280266129 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb4055d4f0 msgr2=0x55fb40567370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:08.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.259+0000 7fece9739b80 1 --2- 172.21.15.101:0/2280266129 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb4055d4f0 0x55fb40567370 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fecc000eb00 tx=0x7fecc0099c60 comp rx=0 tx=0).stop 2023-07-18T03:10:08.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.259+0000 7fece9739b80 1 -- 172.21.15.101:0/2280266129 shutdown_connections 2023-07-18T03:10:08.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.259+0000 7fece9739b80 1 --2- 172.21.15.101:0/2280266129 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb40567d30 0x55fb40560230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.259+0000 7fece9739b80 1 --2- 172.21.15.101:0/2280266129 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb4055d4f0 0x55fb40567370 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.259+0000 7fece9739b80 1 --2- 172.21.15.101:0/2280266129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb4055de00 0x55fb4055fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.259+0000 7fece9739b80 1 -- 172.21.15.101:0/2280266129 >> 172.21.15.101:0/2280266129 conn(0x55fb40552ff0 msgr2=0x55fb40573a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:08.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.260+0000 7fece9739b80 1 -- 172.21.15.101:0/2280266129 shutdown_connections 2023-07-18T03:10:08.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.260+0000 7fece9739b80 1 -- 172.21.15.101:0/2280266129 wait complete. 2023-07-18T03:10:08.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.260+0000 7fece9739b80 1 -- start start 2023-07-18T03:10:08.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.260+0000 7fece9739b80 1 Processor -- start 2023-07-18T03:10:08.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.261+0000 7fece9739b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb4055d4f0 0x55fb4056e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:08.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.261+0000 7fece9739b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb4055de00 0x55fb4056c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:08.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.261+0000 7fece9739b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb40567d30 0x55fb4056cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:08.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.261+0000 7fece9739b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb4056d450 con 0x55fb4055d4f0 2023-07-18T03:10:08.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.261+0000 7fece9739b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb4056d5c0 con 0x55fb40567d30 2023-07-18T03:10:08.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.261+0000 7fece9739b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb4056d780 con 0x55fb4055de00 2023-07-18T03:10:08.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.261+0000 7fece9739b80 10 client.0 _ll_register_callbacks cb 0x7fecc400bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:10:08.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.261+0000 7fecceffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb4055de00 0x55fb4056c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:08.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.261+0000 7fecceffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb4055de00 0x55fb4056c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:50494/0 (socket says 172.21.15.101:50494) 2023-07-18T03:10:08.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.261+0000 7fecceffd700 1 -- 172.21.15.101:0/1105936237 learned_addr learned my addr 172.21.15.101:0/1105936237 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:08.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.261+0000 7feccf7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb4055d4f0 0x55fb4056e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:08.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.261+0000 7feccffff700 1 --2- 172.21.15.101:0/1105936237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb40567d30 0x55fb4056cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:08.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.261+0000 7fecceffd700 1 -- 172.21.15.101:0/1105936237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb40567d30 msgr2=0x55fb4056cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:08.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.261+0000 7fecceffd700 1 --2- 172.21.15.101:0/1105936237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb40567d30 0x55fb4056cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.261+0000 7fecceffd700 1 -- 172.21.15.101:0/1105936237 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb4055d4f0 msgr2=0x55fb4056e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:08.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.261+0000 7fecceffd700 1 --2- 172.21.15.101:0/1105936237 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb4055d4f0 0x55fb4056e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.261+0000 7fecceffd700 1 -- 172.21.15.101:0/1105936237 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb402dcde0 con 0x55fb4055de00 2023-07-18T03:10:08.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.262+0000 7fecaeffd700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:10:08.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.262+0000 7fecceffd700 1 --2- 172.21.15.101:0/1105936237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb4055de00 0x55fb4056c9d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x55fb40552bb0 tx=0x7fecc4005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea9ff0c2aa72e286 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:08.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.262+0000 7fecaeffd700 1 -- 172.21.15.101:0/1105936237 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc401d040 con 0x55fb4055de00 2023-07-18T03:10:08.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.262+0000 7fece9739b80 1 -- 172.21.15.101:0/1105936237 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fecc009a280 con 0x55fb4055de00 2023-07-18T03:10:08.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.262+0000 7fecaeffd700 1 -- 172.21.15.101:0/1105936237 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fecc4000e40 con 0x55fb4055de00 2023-07-18T03:10:08.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.262+0000 7fecaeffd700 1 -- 172.21.15.101:0/1105936237 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fecc4004d30 con 0x55fb4055de00 2023-07-18T03:10:08.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.262+0000 7fece9739b80 10 client.26086 fetch_fsmap learned FSMap version 25 2023-07-18T03:10:08.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.262+0000 7fece9739b80 1 -- 172.21.15.101:0/1105936237 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55fb402db2b0 con 0x55fb4055de00 2023-07-18T03:10:08.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.263+0000 7fecaeffd700 1 -- 172.21.15.101:0/1105936237 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fecc009a280 con 0x55fb4055de00 2023-07-18T03:10:08.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.263+0000 7fecaeffd700 1 -- 172.21.15.101:0/1105936237 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fecc400f460 con 0x55fb4055de00 2023-07-18T03:10:08.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.263+0000 7fece9739b80 10 client.26086 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:10:08.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.263+0000 7fece9739b80 10 client.26086 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:10:08.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.263+0000 7fece9739b80 20 client.26086 populate_metadata read hostname 'smithi101' 2023-07-18T03:10:08.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.263+0000 7fece9739b80 1 --2- 172.21.15.101:0/1105936237 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55fb405cc830 0x55fb405cec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:08.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.263+0000 7fece9739b80 4 client.26086 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:10:08.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.263+0000 7fece9739b80 1 -- 172.21.15.101:0/1105936237 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fb4046a050 con 0x55fb405cc830 2023-07-18T03:10:08.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.263+0000 7feccf7fe700 1 --2- 172.21.15.101:0/1105936237 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55fb405cc830 0x55fb405cec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:08.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.263+0000 7fecaeffd700 10 client.26086 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:10:08.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.263+0000 7feccf7fe700 1 --2- 172.21.15.101:0/1105936237 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55fb405cc830 0x55fb405cec40 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3961a09b8b96a4b server_cookie=3d8995ec317e1e49 in_seq=0 out_seq=0 2023-07-18T03:10:08.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.264+0000 7fecaeffd700 1 -- 172.21.15.101:0/1105936237 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55fb4046a050 con 0x55fb405cc830 2023-07-18T03:10:08.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.264+0000 7fecaeffd700 10 client.26086 handle_command_reply: tid=0 2023-07-18T03:10:08.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.264+0000 7fece9739b80 1 client.26086 shutdown 2023-07-18T03:10:08.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.264+0000 7fece9739b80 1 -- 172.21.15.101:0/1105936237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb4055de00 msgr2=0x55fb4056c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:08.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.264+0000 7fece9739b80 1 --2- 172.21.15.101:0/1105936237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb4055de00 0x55fb4056c9d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x55fb40552bb0 tx=0x7fecc4005f00 comp rx=0 tx=0).stop 2023-07-18T03:10:08.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.264+0000 7fece9739b80 1 -- 172.21.15.101:0/1105936237 shutdown_connections 2023-07-18T03:10:08.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.264+0000 7fece9739b80 1 --2- 172.21.15.101:0/1105936237 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55fb405cc830 0x55fb405cec40 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.264+0000 7fece9739b80 1 --2- 172.21.15.101:0/1105936237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fb40567d30 0x55fb4056cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.264+0000 7fece9739b80 1 --2- 172.21.15.101:0/1105936237 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb4055d4f0 0x55fb4056e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.264+0000 7fece9739b80 1 --2- 172.21.15.101:0/1105936237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb4055de00 0x55fb4056c9d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.264+0000 7fecaeffd700 0 client.26086 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:10:08.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.264+0000 7fece9739b80 1 -- 172.21.15.101:0/1105936237 >> 172.21.15.101:0/1105936237 conn(0x55fb40552ff0 msgr2=0x55fb4055fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:08.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.264+0000 7fece9739b80 1 -- 172.21.15.101:0/1105936237 shutdown_connections 2023-07-18T03:10:08.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.264+0000 7fece9739b80 1 -- 172.21.15.101:0/1105936237 wait complete. 2023-07-18T03:10:08.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.264+0000 7fece9739b80 20 client.26086 trim_cache size 0 max 16384 2023-07-18T03:10:08.264 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:10:08.264 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "no active scrubs running", 2023-07-18T03:10:08.264 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": {} 2023-07-18T03:10:08.264 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:10:08.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.265+0000 7fecd4b9d700 1 -- 172.21.15.101:0/3220500409 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fecb80a1360 msgr2=0x7fecb80a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:08.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.265+0000 7fecd4b9d700 1 --2- 172.21.15.101:0/3220500409 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fecb80a1360 0x7fecb80a3810 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fecd019e740 tx=0x7fecc8009920 comp rx=0 tx=0).stop 2023-07-18T03:10:08.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.265+0000 7fecd4b9d700 1 -- 172.21.15.101:0/3220500409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fecd0072850 msgr2=0x7fecd019da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:08.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.265+0000 7fecd4b9d700 1 --2- 172.21.15.101:0/3220500409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fecd0072850 0x7fecd019da40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fecc000eb30 tx=0x7fecc000ee40 comp rx=0 tx=0).stop 2023-07-18T03:10:08.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.265+0000 7fecd4b9d700 1 -- 172.21.15.101:0/3220500409 shutdown_connections 2023-07-18T03:10:08.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.265+0000 7fecd4b9d700 1 --2- 172.21.15.101:0/3220500409 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fecb80a1360 0x7fecb80a3810 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.265+0000 7fecd4b9d700 1 --2- 172.21.15.101:0/3220500409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fecd0072850 0x7fecd019da40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.265+0000 7fecd4b9d700 1 --2- 172.21.15.101:0/3220500409 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fecd010c290 0x7fecd0197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.265+0000 7fecd4b9d700 1 --2- 172.21.15.101:0/3220500409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecd0073160 0x7fecd019df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.265+0000 7fecd4b9d700 1 -- 172.21.15.101:0/3220500409 >> 172.21.15.101:0/3220500409 conn(0x7fecd00fb510 msgr2=0x7fecd0103cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:08.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.265+0000 7fecd4b9d700 1 -- 172.21.15.101:0/3220500409 shutdown_connections 2023-07-18T03:10:08.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.265+0000 7fecd4b9d700 1 -- 172.21.15.101:0/3220500409 wait complete. 2023-07-18T03:10:08.274 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-18T03:10:08.275 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:10:08.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.500+0000 7f2c4c5ff700 1 -- 172.21.15.101:0/3227968261 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2c440728b0 msgr2=0x7f2c44072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:08.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.500+0000 7f2c4c5ff700 1 --2- 172.21.15.101:0/3227968261 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2c440728b0 0x7f2c44072c80 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2c40009990 tx=0x7f2c40009ca0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.500+0000 7f2c4c5ff700 1 -- 172.21.15.101:0/3227968261 shutdown_connections 2023-07-18T03:10:08.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.500+0000 7f2c4c5ff700 1 --2- 172.21.15.101:0/3227968261 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c4410c330 0x7f2c4410e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.500+0000 7f2c4c5ff700 1 --2- 172.21.15.101:0/3227968261 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2c440728b0 0x7f2c44072c80 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.500+0000 7f2c4c5ff700 1 --2- 172.21.15.101:0/3227968261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c440731c0 0x7f2c4410bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.500+0000 7f2c4c5ff700 1 -- 172.21.15.101:0/3227968261 >> 172.21.15.101:0/3227968261 conn(0x7f2c440fb630 msgr2=0x7f2c440fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:08.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.500+0000 7f2c4c5ff700 1 -- 172.21.15.101:0/3227968261 shutdown_connections 2023-07-18T03:10:08.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.501+0000 7f2c4c5ff700 1 -- 172.21.15.101:0/3227968261 wait complete. 2023-07-18T03:10:08.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.502+0000 7f2c4c5ff700 1 Processor -- start 2023-07-18T03:10:08.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.502+0000 7f2c4c5ff700 1 -- start start 2023-07-18T03:10:08.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.502+0000 7f2c4c5ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c440728b0 0x7f2c44197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:08.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.502+0000 7f2c4c5ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c440731c0 0x7f2c44198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:08.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.502+0000 7f2c4c5ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2c4410c330 0x7f2c4419c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:08.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.502+0000 7f2c4c5ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4419ce00 con 0x7f2c4410c330 2023-07-18T03:10:08.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.502+0000 7f2c4c5ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4419cf70 con 0x7f2c440731c0 2023-07-18T03:10:08.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.502+0000 7f2c4c5ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4419d130 con 0x7f2c440728b0 2023-07-18T03:10:08.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.502+0000 7f2c4a39b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c440728b0 0x7f2c44197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:08.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.502+0000 7f2c4a39b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c440728b0 0x7f2c44197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:50526/0 (socket says 172.21.15.101:50526) 2023-07-18T03:10:08.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.502+0000 7f2c4a39b700 1 -- 172.21.15.101:0/713378807 learned_addr learned my addr 172.21.15.101:0/713378807 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:08.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.502+0000 7f2c4ab9c700 1 --2- 172.21.15.101:0/713378807 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2c4410c330 0x7f2c4419c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:08.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.502+0000 7f2c49b9a700 1 --2- 172.21.15.101:0/713378807 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c440731c0 0x7f2c44198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:08.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.503+0000 7f2c4a39b700 1 -- 172.21.15.101:0/713378807 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c440731c0 msgr2=0x7f2c44198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:08.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.503+0000 7f2c4a39b700 1 --2- 172.21.15.101:0/713378807 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c440731c0 0x7f2c44198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.503+0000 7f2c4a39b700 1 -- 172.21.15.101:0/713378807 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2c4410c330 msgr2=0x7f2c4419c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:08.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.503+0000 7f2c4a39b700 1 --2- 172.21.15.101:0/713378807 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2c4410c330 0x7f2c4419c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.503+0000 7f2c4a39b700 1 -- 172.21.15.101:0/713378807 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c400095f0 con 0x7f2c440728b0 2023-07-18T03:10:08.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.503+0000 7f2c4a39b700 1 --2- 172.21.15.101:0/713378807 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c440728b0 0x7f2c44197e20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2c40005910 tx=0x7f2c40005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:08.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.503+0000 7f2c3b7fe700 1 -- 172.21.15.101:0/713378807 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4001d070 con 0x7f2c440728b0 2023-07-18T03:10:08.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.503+0000 7f2c4c5ff700 1 -- 172.21.15.101:0/713378807 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c4419d3b0 con 0x7f2c440728b0 2023-07-18T03:10:08.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.503+0000 7f2c3b7fe700 1 -- 172.21.15.101:0/713378807 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c40022470 con 0x7f2c440728b0 2023-07-18T03:10:08.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.503+0000 7f2c3b7fe700 1 -- 172.21.15.101:0/713378807 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c40017650 con 0x7f2c440728b0 2023-07-18T03:10:08.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.503+0000 7f2c4c5ff700 1 -- 172.21.15.101:0/713378807 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c441a4c50 con 0x7f2c440728b0 2023-07-18T03:10:08.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.504+0000 7f2c4c5ff700 1 -- 172.21.15.101:0/713378807 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c44067300 con 0x7f2c440728b0 2023-07-18T03:10:08.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.506+0000 7f2c3b7fe700 1 -- 172.21.15.101:0/713378807 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f2c4000fad0 con 0x7f2c440728b0 2023-07-18T03:10:08.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.508+0000 7f2c3b7fe700 1 --2- 172.21.15.101:0/713378807 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2c300a1380 0x7f2c300a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:08.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.508+0000 7f2c3b7fe700 1 -- 172.21.15.101:0/713378807 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f2c400ce6b0 con 0x7f2c440728b0 2023-07-18T03:10:08.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.508+0000 7f2c49b9a700 1 --2- 172.21.15.101:0/713378807 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2c300a1380 0x7f2c300a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:08.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.509+0000 7f2c49b9a700 1 --2- 172.21.15.101:0/713378807 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2c300a1380 0x7f2c300a3830 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2c44199430 tx=0x7f2c340073d0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:08.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.510+0000 7f2c3b7fe700 1 -- 172.21.15.101:0/713378807 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2c400977a0 con 0x7f2c440728b0 2023-07-18T03:10:08.568 INFO:tasks.workunit.client.0.smithi101.stdout: 1 133673 7.91 MB/sec warmup 93 sec latency 18.431 ms 2023-07-18T03:10:08.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.769+0000 7f2c4c5ff700 1 -- 172.21.15.101:0/713378807 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2c44199170 con 0x7f2c440728b0 2023-07-18T03:10:08.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.770+0000 7f2c3b7fe700 1 -- 172.21.15.101:0/713378807 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10861 (secure 0 0 0) 0x7f2c40027090 con 0x7f2c440728b0 2023-07-18T03:10:08.760 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:10:08.761 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/4192846769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":4192846769},{"type":"v1","addr":"172.21.15.101:6839","nonce":4192846769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6841/3824340175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6840","nonce":3824340175},{"type":"v1","addr":"172.21.15.101:6841","nonce":3824340175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:04:59.361185+0000","modified":"2023-07-18T03:09:01.105419+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.155:6839/2055462483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2055462483},{"type":"v1","addr":"172.21.15.155:6839","nonce":2055462483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:10:08.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.772+0000 7f2c4c5ff700 1 -- 172.21.15.101:0/713378807 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2c300a1380 msgr2=0x7f2c300a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:08.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.772+0000 7f2c4c5ff700 1 --2- 172.21.15.101:0/713378807 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2c300a1380 0x7f2c300a3830 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2c44199430 tx=0x7f2c340073d0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.772+0000 7f2c4c5ff700 1 -- 172.21.15.101:0/713378807 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c440728b0 msgr2=0x7f2c44197e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:08.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.772+0000 7f2c4c5ff700 1 --2- 172.21.15.101:0/713378807 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c440728b0 0x7f2c44197e20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2c40005910 tx=0x7f2c40005740 comp rx=0 tx=0).stop 2023-07-18T03:10:08.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.772+0000 7f2c4c5ff700 1 -- 172.21.15.101:0/713378807 shutdown_connections 2023-07-18T03:10:08.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.772+0000 7f2c4c5ff700 1 --2- 172.21.15.101:0/713378807 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2c4410c330 0x7f2c4419c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.772+0000 7f2c4c5ff700 1 --2- 172.21.15.101:0/713378807 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c440728b0 0x7f2c44197e20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.772+0000 7f2c4c5ff700 1 --2- 172.21.15.101:0/713378807 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2c300a1380 0x7f2c300a3830 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.772+0000 7f2c4c5ff700 1 --2- 172.21.15.101:0/713378807 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c440731c0 0x7f2c44198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.772+0000 7f2c4c5ff700 1 -- 172.21.15.101:0/713378807 >> 172.21.15.101:0/713378807 conn(0x7f2c440fb630 msgr2=0x7f2c4410e400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:08.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.772+0000 7f2c4c5ff700 1 -- 172.21.15.101:0/713378807 shutdown_connections 2023-07-18T03:10:08.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:08.772+0000 7f2c4c5ff700 1 -- 172.21.15.101:0/713378807 wait complete. 2023-07-18T03:10:08.765 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 25 2023-07-18T03:10:08.782 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-18T03:10:08.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.003+0000 7f7a47dff700 1 -- 172.21.15.101:0/3223585096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a400ffae0 msgr2=0x7f7a400fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:08.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.003+0000 7f7a47dff700 1 --2- 172.21.15.101:0/3223585096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a400ffae0 0x7f7a400fff50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7a300098f0 tx=0x7f7a30009c00 comp rx=0 tx=0).stop 2023-07-18T03:10:08.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.003+0000 7f7a47dff700 1 -- 172.21.15.101:0/3223585096 shutdown_connections 2023-07-18T03:10:08.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.003+0000 7f7a47dff700 1 --2- 172.21.15.101:0/3223585096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a40105af0 0x7f7a40105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.003+0000 7f7a47dff700 1 --2- 172.21.15.101:0/3223585096 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7a40100490 0x7f7a4010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.003+0000 7f7a47dff700 1 --2- 172.21.15.101:0/3223585096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a400ffae0 0x7f7a400fff50 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.003+0000 7f7a47dff700 1 -- 172.21.15.101:0/3223585096 >> 172.21.15.101:0/3223585096 conn(0x7f7a400fb5e0 msgr2=0x7f7a400fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:08.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.004+0000 7f7a47dff700 1 -- 172.21.15.101:0/3223585096 shutdown_connections 2023-07-18T03:10:08.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.004+0000 7f7a47dff700 1 -- 172.21.15.101:0/3223585096 wait complete. 2023-07-18T03:10:08.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.005+0000 7f7a47dff700 1 Processor -- start 2023-07-18T03:10:08.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.005+0000 7f7a47dff700 1 -- start start 2023-07-18T03:10:08.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.005+0000 7f7a47dff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a400ffae0 0x7f7a40072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:08.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.005+0000 7f7a47dff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a40100490 0x7f7a40072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:08.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.005+0000 7f7a47dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7a40105af0 0x7f7a401a3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:08.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.005+0000 7f7a47dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a40073580 con 0x7f7a40105af0 2023-07-18T03:10:08.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.005+0000 7f7a47dff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a401a4380 con 0x7f7a400ffae0 2023-07-18T03:10:08.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.005+0000 7f7a47dff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a401a44c0 con 0x7f7a40100490 2023-07-18T03:10:08.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.005+0000 7f7a4539a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a40100490 0x7f7a40072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:08.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.005+0000 7f7a45b9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a400ffae0 0x7f7a40072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:08.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.005+0000 7f7a4539a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a40100490 0x7f7a40072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:50534/0 (socket says 172.21.15.101:50534) 2023-07-18T03:10:08.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.005+0000 7f7a4539a700 1 -- 172.21.15.101:0/3762621743 learned_addr learned my addr 172.21.15.101:0/3762621743 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:08.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.006+0000 7f7a4639c700 1 --2- 172.21.15.101:0/3762621743 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7a40105af0 0x7f7a401a3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:08.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.006+0000 7f7a4539a700 1 -- 172.21.15.101:0/3762621743 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a400ffae0 msgr2=0x7f7a40072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:08.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.006+0000 7f7a4539a700 1 --2- 172.21.15.101:0/3762621743 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a400ffae0 0x7f7a40072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.006+0000 7f7a4539a700 1 -- 172.21.15.101:0/3762621743 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7a40105af0 msgr2=0x7f7a401a3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:08.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.006+0000 7f7a4539a700 1 --2- 172.21.15.101:0/3762621743 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7a40105af0 0x7f7a401a3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:08.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.006+0000 7f7a4539a700 1 -- 172.21.15.101:0/3762621743 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a3c0095f0 con 0x7f7a40100490 2023-07-18T03:10:09.000 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.006+0000 7f7a4539a700 1 --2- 172.21.15.101:0/3762621743 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a40100490 0x7f7a40072d50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7a30009810 tx=0x7f7a3000fa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:09.000 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.006+0000 7f7a36ffd700 1 -- 172.21.15.101:0/3762621743 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3001c070 con 0x7f7a40100490 2023-07-18T03:10:09.000 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.006+0000 7f7a36ffd700 1 -- 172.21.15.101:0/3762621743 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a3000bba0 con 0x7f7a40100490 2023-07-18T03:10:09.000 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.006+0000 7f7a47dff700 1 -- 172.21.15.101:0/3762621743 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a30009520 con 0x7f7a40100490 2023-07-18T03:10:09.001 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.006+0000 7f7a36ffd700 1 -- 172.21.15.101:0/3762621743 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a300049c0 con 0x7f7a40100490 2023-07-18T03:10:09.001 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.007+0000 7f7a47dff700 1 -- 172.21.15.101:0/3762621743 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a401a4950 con 0x7f7a40100490 2023-07-18T03:10:09.001 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.010+0000 7f7a36ffd700 1 -- 172.21.15.101:0/3762621743 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f7a30020070 con 0x7f7a40100490 2023-07-18T03:10:09.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.012+0000 7f7a5b7cfb80 1 Processor -- start 2023-07-18T03:10:09.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.012+0000 7f7a5b7cfb80 1 -- start start 2023-07-18T03:10:09.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.012+0000 7f7a36ffd700 1 --2- 172.21.15.101:0/3762621743 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f7a2c0a12c0 0x7f7a2c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.012+0000 7f7a5b7cfb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae8e202d0 0x555ae8e206a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.012+0000 7f7a5b7cfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae8e20be0 0x555ae8e2b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.012+0000 7f7a5b7cfb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ae8e2bad0 0x555ae8e2deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.012+0000 7f7a5b7cfb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ae8d26cb0 con 0x555ae8e2bad0 2023-07-18T03:10:09.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.012+0000 7f7a5b7cfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ae8d79a40 con 0x555ae8e20be0 2023-07-18T03:10:09.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.012+0000 7f7a5b7cfb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ae8d72ed0 con 0x555ae8e202d0 2023-07-18T03:10:09.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.012+0000 7f7a36ffd700 1 -- 172.21.15.101:0/3762621743 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f7a300cddc0 con 0x7f7a40100490 2023-07-18T03:10:09.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.013+0000 7f7a45b9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae8e202d0 0x555ae8e206a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.013+0000 7f7a45b9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae8e202d0 0x555ae8e206a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:50538/0 (socket says 172.21.15.101:50538) 2023-07-18T03:10:09.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.013+0000 7f7a4639c700 1 --2- 172.21.15.101:0/3762621743 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f7a2c0a12c0 0x7f7a2c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.013+0000 7f7a45b9b700 1 -- 172.21.15.101:0/3772910724 learned_addr learned my addr 172.21.15.101:0/3772910724 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:09.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.013+0000 7f7a4539a700 1 --2- 172.21.15.101:0/3772910724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae8e20be0 0x555ae8e2b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.013+0000 7f7a4639c700 1 --2- 172.21.15.101:0/3772910724 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ae8e2bad0 0x555ae8e2deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.013+0000 7f7a45b9b700 1 -- 172.21.15.101:0/3772910724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae8e20be0 msgr2=0x555ae8e2b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:09.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.013+0000 7f7a45b9b700 1 --2- 172.21.15.101:0/3772910724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae8e20be0 0x555ae8e2b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.013+0000 7f7a45b9b700 1 -- 172.21.15.101:0/3772910724 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ae8e2bad0 msgr2=0x555ae8e2deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:09.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.013+0000 7f7a45b9b700 1 --2- 172.21.15.101:0/3772910724 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ae8e2bad0 0x555ae8e2deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.013+0000 7f7a45b9b700 1 -- 172.21.15.101:0/3772910724 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ae8ba0de0 con 0x555ae8e202d0 2023-07-18T03:10:09.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.013+0000 7f7a4639c700 1 --2- 172.21.15.101:0/3772910724 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ae8e2bad0 0x555ae8e2deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:10:09.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.013+0000 7f7a45b9b700 1 --2- 172.21.15.101:0/3772910724 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae8e202d0 0x555ae8e206a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c009f40 tx=0x7f7a3c011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c72046759365db35 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:09.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.013+0000 7f7a4639c700 1 --2- 172.21.15.101:0/3762621743 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f7a2c0a12c0 0x7f7a2c0a3770 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7a40074920 tx=0x7f7a3800b040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:09.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.013+0000 7f7a2bfff700 1 -- 172.21.15.101:0/3772910724 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c004310 con 0x555ae8e202d0 2023-07-18T03:10:09.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.013+0000 7f7a2bfff700 1 -- 172.21.15.101:0/3772910724 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a3c004470 con 0x555ae8e202d0 2023-07-18T03:10:09.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.014+0000 7f7a2bfff700 1 -- 172.21.15.101:0/3772910724 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c019910 con 0x555ae8e202d0 2023-07-18T03:10:09.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.014+0000 7f7a5b7cfb80 1 -- 172.21.15.101:0/3772910724 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae8e202d0 msgr2=0x555ae8e206a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:09.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.014+0000 7f7a5b7cfb80 1 --2- 172.21.15.101:0/3772910724 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae8e202d0 0x555ae8e206a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c009f40 tx=0x7f7a3c011720 comp rx=0 tx=0).stop 2023-07-18T03:10:09.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.014+0000 7f7a5b7cfb80 1 -- 172.21.15.101:0/3772910724 shutdown_connections 2023-07-18T03:10:09.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.014+0000 7f7a5b7cfb80 1 --2- 172.21.15.101:0/3772910724 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ae8e2bad0 0x555ae8e2deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.014+0000 7f7a5b7cfb80 1 --2- 172.21.15.101:0/3772910724 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae8e202d0 0x555ae8e206a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.014+0000 7f7a5b7cfb80 1 --2- 172.21.15.101:0/3772910724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae8e20be0 0x555ae8e2b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.014+0000 7f7a5b7cfb80 1 -- 172.21.15.101:0/3772910724 >> 172.21.15.101:0/3772910724 conn(0x555ae8e16da0 msgr2=0x555ae8e1c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:09.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.014+0000 7f7a5b7cfb80 1 -- 172.21.15.101:0/3772910724 shutdown_connections 2023-07-18T03:10:09.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.014+0000 7f7a5b7cfb80 1 -- 172.21.15.101:0/3772910724 wait complete. 2023-07-18T03:10:09.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.015+0000 7f7a5b7cfb80 1 -- start start 2023-07-18T03:10:09.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.015+0000 7f7a5b7cfb80 1 Processor -- start 2023-07-18T03:10:09.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.015+0000 7f7a5b7cfb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ae8e202d0 0x555ae8e2b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.016+0000 7f7a5b7cfb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae8e20be0 0x555ae8e287f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.016+0000 7f7a5b7cfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae8e2bad0 0x555ae8e28f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.016+0000 7f7a5b7cfb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ae8d12ac0 con 0x555ae8e202d0 2023-07-18T03:10:09.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.016+0000 7f7a5b7cfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ae8d127c0 con 0x555ae8e2bad0 2023-07-18T03:10:09.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.016+0000 7f7a5b7cfb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ae8bf6920 con 0x555ae8e20be0 2023-07-18T03:10:09.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.016+0000 7f7a5b7cfb80 10 client.0 _ll_register_callbacks cb 0x555ae8d10760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:10:09.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.016+0000 7f7a4639c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae8e2bad0 0x555ae8e28f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.016+0000 7f7a4639c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae8e2bad0 0x555ae8e28f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:32902/0 (socket says 172.21.15.101:32902) 2023-07-18T03:10:09.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.016+0000 7f7a4639c700 1 -- 172.21.15.101:0/4196827551 learned_addr learned my addr 172.21.15.101:0/4196827551 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:09.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.016+0000 7f7a4539a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae8e20be0 0x555ae8e287f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.016+0000 7f7a45b9b700 1 --2- 172.21.15.101:0/4196827551 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ae8e202d0 0x555ae8e2b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.016+0000 7f7a4639c700 1 -- 172.21.15.101:0/4196827551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae8e20be0 msgr2=0x555ae8e287f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:09.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.016+0000 7f7a4639c700 1 --2- 172.21.15.101:0/4196827551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae8e20be0 0x555ae8e287f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.016+0000 7f7a4639c700 1 -- 172.21.15.101:0/4196827551 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ae8e202d0 msgr2=0x555ae8e2b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:09.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.016+0000 7f7a4639c700 1 --2- 172.21.15.101:0/4196827551 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ae8e202d0 0x555ae8e2b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.016+0000 7f7a4639c700 1 -- 172.21.15.101:0/4196827551 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a3c009ba0 con 0x555ae8e2bad0 2023-07-18T03:10:09.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.016+0000 7f7a45b9b700 1 --2- 172.21.15.101:0/4196827551 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ae8e202d0 0x555ae8e2b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:10:09.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.016+0000 7f7a4539a700 1 --2- 172.21.15.101:0/4196827551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae8e20be0 0x555ae8e287f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:10:09.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.016+0000 7f7a4639c700 1 --2- 172.21.15.101:0/4196827551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae8e2bad0 0x555ae8e28f30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7a38009420 tx=0x7f7a38004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba7969c3883a697a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:09.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.016+0000 7f7a297fa700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:10:09.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.017+0000 7f7a297fa700 1 -- 172.21.15.101:0/4196827551 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a38016a60 con 0x555ae8e2bad0 2023-07-18T03:10:09.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.017+0000 7f7a5b7cfb80 1 -- 172.21.15.101:0/4196827551 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555ae8ccc4f0 con 0x555ae8e2bad0 2023-07-18T03:10:09.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.017+0000 7f7a297fa700 1 -- 172.21.15.101:0/4196827551 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f7a38016bc0 con 0x555ae8e2bad0 2023-07-18T03:10:09.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.017+0000 7f7a297fa700 1 -- 172.21.15.101:0/4196827551 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a38016d80 con 0x555ae8e2bad0 2023-07-18T03:10:09.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.017+0000 7f7a5b7cfb80 10 client.26078 fetch_fsmap learned FSMap version 25 2023-07-18T03:10:09.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.017+0000 7f7a5b7cfb80 1 -- 172.21.15.101:0/4196827551 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555ae8b89c60 con 0x555ae8e2bad0 2023-07-18T03:10:09.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.017+0000 7f7a297fa700 1 -- 172.21.15.101:0/4196827551 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555ae8ccc4f0 con 0x555ae8e2bad0 2023-07-18T03:10:09.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.017+0000 7f7a297fa700 1 -- 172.21.15.101:0/4196827551 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f7a380213a0 con 0x555ae8e2bad0 2023-07-18T03:10:09.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.017+0000 7f7a5b7cfb80 10 client.26078 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:10:09.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.017+0000 7f7a5b7cfb80 10 client.26078 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:10:09.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.017+0000 7f7a5b7cfb80 20 client.26078 populate_metadata read hostname 'smithi101' 2023-07-18T03:10:09.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.017+0000 7f7a5b7cfb80 1 --2- 172.21.15.101:0/4196827551 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x555ae8e3b480 0x555ae8e3d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.017+0000 7f7a5b7cfb80 4 client.26078 mds_command: new command op to 24541 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:10:09.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.017+0000 7f7a5b7cfb80 1 -- 172.21.15.101:0/4196827551 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555ae8d6beb0 con 0x555ae8e3b480 2023-07-18T03:10:09.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.018+0000 7f7a45b9b700 1 --2- 172.21.15.101:0/4196827551 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x555ae8e3b480 0x555ae8e3d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.018+0000 7f7a297fa700 10 client.26078 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:10:09.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.018+0000 7f7a45b9b700 1 --2- 172.21.15.101:0/4196827551 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x555ae8e3b480 0x555ae8e3d890 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e1bd7db6381e324c server_cookie=ab0058e41b991a80 in_seq=0 out_seq=0 2023-07-18T03:10:09.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.019+0000 7f7a297fa700 1 -- 172.21.15.101:0/4196827551 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555ae8d6beb0 con 0x555ae8e3b480 2023-07-18T03:10:09.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.019+0000 7f7a297fa700 10 client.26078 handle_command_reply: tid=0 2023-07-18T03:10:09.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.019+0000 7f7a5b7cfb80 1 client.26078 shutdown 2023-07-18T03:10:09.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.019+0000 7f7a5b7cfb80 1 -- 172.21.15.101:0/4196827551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae8e2bad0 msgr2=0x555ae8e28f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:09.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.019+0000 7f7a5b7cfb80 1 --2- 172.21.15.101:0/4196827551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae8e2bad0 0x555ae8e28f30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7a38009420 tx=0x7f7a38004ad0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.019+0000 7f7a5b7cfb80 1 -- 172.21.15.101:0/4196827551 shutdown_connections 2023-07-18T03:10:09.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.019+0000 7f7a5b7cfb80 1 --2- 172.21.15.101:0/4196827551 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x555ae8e3b480 0x555ae8e3d890 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.019+0000 7f7a5b7cfb80 1 --2- 172.21.15.101:0/4196827551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae8e2bad0 0x555ae8e28f30 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.019+0000 7f7a297fa700 0 client.26078 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:10:09.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.019+0000 7f7a5b7cfb80 1 --2- 172.21.15.101:0/4196827551 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ae8e202d0 0x555ae8e2b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.019+0000 7f7a5b7cfb80 1 --2- 172.21.15.101:0/4196827551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae8e20be0 0x555ae8e287f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.019+0000 7f7a5b7cfb80 1 -- 172.21.15.101:0/4196827551 >> 172.21.15.101:0/4196827551 conn(0x555ae8e16da0 msgr2=0x555ae8e1c230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:09.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.019+0000 7f7a5b7cfb80 1 -- 172.21.15.101:0/4196827551 shutdown_connections 2023-07-18T03:10:09.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.019+0000 7f7a5b7cfb80 1 -- 172.21.15.101:0/4196827551 wait complete. 2023-07-18T03:10:09.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.019+0000 7f7a5b7cfb80 20 client.26078 trim_cache size 0 max 16384 2023-07-18T03:10:09.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.033+0000 7f7a5b7cfb80 1 Processor -- start 2023-07-18T03:10:09.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.033+0000 7f7a5b7cfb80 1 -- start start 2023-07-18T03:10:09.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.034+0000 7f7a5b7cfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae8e202d0 0x555ae8e2b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.034+0000 7f7a5b7cfb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ae8e20be0 0x555ae8e25a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.034+0000 7f7a5b7cfb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae8e2bad0 0x555ae8e25fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.034+0000 7f7a5b7cfb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ae8e29b40 con 0x555ae8e20be0 2023-07-18T03:10:09.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.034+0000 7f7a5b7cfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ae8e1b1d0 con 0x555ae8e202d0 2023-07-18T03:10:09.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.034+0000 7f7a5b7cfb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ae8e208f0 con 0x555ae8e2bad0 2023-07-18T03:10:09.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.034+0000 7f7a45b9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae8e202d0 0x555ae8e2b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.034+0000 7f7a4539a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ae8e20be0 0x555ae8e25a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.034+0000 7f7a45b9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae8e202d0 0x555ae8e2b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:32912/0 (socket says 172.21.15.101:32912) 2023-07-18T03:10:09.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.034+0000 7f7a45b9b700 1 -- 172.21.15.101:0/3347258504 learned_addr learned my addr 172.21.15.101:0/3347258504 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:09.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.034+0000 7f7a4639c700 1 --2- 172.21.15.101:0/3347258504 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae8e2bad0 0x555ae8e25fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.034+0000 7f7a45b9b700 1 -- 172.21.15.101:0/3347258504 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae8e2bad0 msgr2=0x555ae8e25fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:09.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.034+0000 7f7a45b9b700 1 --2- 172.21.15.101:0/3347258504 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae8e2bad0 0x555ae8e25fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.034+0000 7f7a45b9b700 1 -- 172.21.15.101:0/3347258504 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ae8e20be0 msgr2=0x555ae8e25a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:09.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.034+0000 7f7a45b9b700 1 --2- 172.21.15.101:0/3347258504 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ae8e20be0 0x555ae8e25a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.034+0000 7f7a45b9b700 1 -- 172.21.15.101:0/3347258504 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ae8bc7250 con 0x555ae8e202d0 2023-07-18T03:10:09.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.034+0000 7f7a4539a700 1 --2- 172.21.15.101:0/3347258504 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ae8e20be0 0x555ae8e25a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:10:09.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.035+0000 7f7a45b9b700 1 --2- 172.21.15.101:0/3347258504 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae8e202d0 0x555ae8e2b0a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c011a70 tx=0x7f7a3c005a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=deee6faff416ce09 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:09.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.035+0000 7f7a34bff700 1 -- 172.21.15.101:0/3347258504 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c004190 con 0x555ae8e202d0 2023-07-18T03:10:09.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.035+0000 7f7a34bff700 1 -- 172.21.15.101:0/3347258504 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a3c0042f0 con 0x555ae8e202d0 2023-07-18T03:10:09.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.035+0000 7f7a34bff700 1 -- 172.21.15.101:0/3347258504 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c003d90 con 0x555ae8e202d0 2023-07-18T03:10:09.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.036+0000 7f7a5b7cfb80 1 -- 172.21.15.101:0/3347258504 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae8e202d0 msgr2=0x555ae8e2b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:09.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.036+0000 7f7a5b7cfb80 1 --2- 172.21.15.101:0/3347258504 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae8e202d0 0x555ae8e2b0a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c011a70 tx=0x7f7a3c005a90 comp rx=0 tx=0).stop 2023-07-18T03:10:09.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.036+0000 7f7a5b7cfb80 1 -- 172.21.15.101:0/3347258504 shutdown_connections 2023-07-18T03:10:09.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.036+0000 7f7a5b7cfb80 1 --2- 172.21.15.101:0/3347258504 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae8e202d0 0x555ae8e2b0a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.036+0000 7f7a5b7cfb80 1 --2- 172.21.15.101:0/3347258504 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae8e2bad0 0x555ae8e25fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.036+0000 7f7a5b7cfb80 1 --2- 172.21.15.101:0/3347258504 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ae8e20be0 0x555ae8e25a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.036+0000 7f7a5b7cfb80 1 -- 172.21.15.101:0/3347258504 >> 172.21.15.101:0/3347258504 conn(0x555ae8e16da0 msgr2=0x555ae8e36cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:09.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.036+0000 7f7a5b7cfb80 1 -- 172.21.15.101:0/3347258504 shutdown_connections 2023-07-18T03:10:09.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.036+0000 7f7a5b7cfb80 1 -- 172.21.15.101:0/3347258504 wait complete. 2023-07-18T03:10:09.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.036+0000 7f7a5b7cfb80 1 -- start start 2023-07-18T03:10:09.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.037+0000 7f7a5b7cfb80 1 Processor -- start 2023-07-18T03:10:09.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.037+0000 7f7a5b7cfb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ae8e202d0 0x555ae8e32320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.037+0000 7f7a5b7cfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae8e20be0 0x555ae8e30650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.037+0000 7f7a5b7cfb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae8e2bad0 0x555ae8e30b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.037+0000 7f7a5b7cfb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ae8e310d0 con 0x555ae8e202d0 2023-07-18T03:10:09.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.037+0000 7f7a5b7cfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ae8e31240 con 0x555ae8e20be0 2023-07-18T03:10:09.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.037+0000 7f7a5b7cfb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ae8e31400 con 0x555ae8e2bad0 2023-07-18T03:10:09.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.037+0000 7f7a5b7cfb80 10 client.0 _ll_register_callbacks cb 0x7f7a38017800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:10:09.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.038+0000 7f7a4639c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae8e2bad0 0x555ae8e30b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.038+0000 7f7a4639c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae8e2bad0 0x555ae8e30b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:50568/0 (socket says 172.21.15.101:50568) 2023-07-18T03:10:09.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.038+0000 7f7a4639c700 1 -- 172.21.15.101:0/1475887201 learned_addr learned my addr 172.21.15.101:0/1475887201 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:09.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.038+0000 7f7a4539a700 1 --2- 172.21.15.101:0/1475887201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae8e20be0 0x555ae8e30650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.038+0000 7f7a45b9b700 1 --2- 172.21.15.101:0/1475887201 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ae8e202d0 0x555ae8e32320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.038+0000 7f7a4639c700 1 -- 172.21.15.101:0/1475887201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae8e20be0 msgr2=0x555ae8e30650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:09.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.038+0000 7f7a4639c700 1 --2- 172.21.15.101:0/1475887201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae8e20be0 0x555ae8e30650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.038+0000 7f7a4639c700 1 -- 172.21.15.101:0/1475887201 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ae8e202d0 msgr2=0x555ae8e32320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:09.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.038+0000 7f7a4639c700 1 --2- 172.21.15.101:0/1475887201 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ae8e202d0 0x555ae8e32320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.038+0000 7f7a4639c700 1 -- 172.21.15.101:0/1475887201 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a401a4950 con 0x555ae8e2bad0 2023-07-18T03:10:09.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.038+0000 7f7a4639c700 1 --2- 172.21.15.101:0/1475887201 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae8e2bad0 0x555ae8e30b90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7a38007bf0 tx=0x7f7a38009130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c43caa1c4498d9df server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:09.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.038+0000 7f7a297fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:10:09.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.039+0000 7f7a297fa700 1 -- 172.21.15.101:0/1475887201 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a380166a0 con 0x555ae8e2bad0 2023-07-18T03:10:09.059 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.039+0000 7f7a297fa700 1 -- 172.21.15.101:0/1475887201 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f7a38021c60 con 0x555ae8e2bad0 2023-07-18T03:10:09.061 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.039+0000 7f7a5b7cfb80 1 -- 172.21.15.101:0/1475887201 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555ae8d2e050 con 0x555ae8e2bad0 2023-07-18T03:10:09.061 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.039+0000 7f7a297fa700 1 -- 172.21.15.101:0/1475887201 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a38021ae0 con 0x555ae8e2bad0 2023-07-18T03:10:09.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.039+0000 7f7a5b7cfb80 10 client.26110 fetch_fsmap learned FSMap version 25 2023-07-18T03:10:09.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.039+0000 7f7a5b7cfb80 1 -- 172.21.15.101:0/1475887201 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f7a3c01e040 con 0x555ae8e2bad0 2023-07-18T03:10:09.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.039+0000 7f7a297fa700 1 -- 172.21.15.101:0/1475887201 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555ae8d2e050 con 0x555ae8e2bad0 2023-07-18T03:10:09.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.039+0000 7f7a297fa700 1 -- 172.21.15.101:0/1475887201 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f7a38038020 con 0x555ae8e2bad0 2023-07-18T03:10:09.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.040+0000 7f7a5b7cfb80 10 client.26110 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:10:09.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.040+0000 7f7a5b7cfb80 10 client.26110 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:10:09.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.040+0000 7f7a5b7cfb80 20 client.26110 populate_metadata read hostname 'smithi101' 2023-07-18T03:10:09.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.040+0000 7f7a5b7cfb80 1 --2- 172.21.15.101:0/1475887201 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x555ae8e90700 0x555ae8e92b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.040+0000 7f7a5b7cfb80 4 client.26110 mds_command: new command op to 24541 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:10:09.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.040+0000 7f7a5b7cfb80 1 -- 172.21.15.101:0/1475887201 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555ae8bc1f70 con 0x555ae8e90700 2023-07-18T03:10:09.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.040+0000 7f7a45b9b700 1 --2- 172.21.15.101:0/1475887201 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x555ae8e90700 0x555ae8e92b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.040+0000 7f7a45b9b700 1 --2- 172.21.15.101:0/1475887201 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x555ae8e90700 0x555ae8e92b10 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ce00061d8f2640d2 server_cookie=153d9855a8c220a7 in_seq=0 out_seq=0 2023-07-18T03:10:09.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.040+0000 7f7a297fa700 10 client.26110 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:10:09.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.042+0000 7f7a297fa700 1 -- 172.21.15.101:0/1475887201 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555ae8bc1f70 con 0x555ae8e90700 2023-07-18T03:10:09.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.042+0000 7f7a297fa700 10 client.26110 handle_command_reply: tid=0 2023-07-18T03:10:09.070 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.042+0000 7f7a5b7cfb80 1 client.26110 shutdown 2023-07-18T03:10:09.070 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.042+0000 7f7a5b7cfb80 1 -- 172.21.15.101:0/1475887201 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae8e2bad0 msgr2=0x555ae8e30b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:09.070 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.042+0000 7f7a5b7cfb80 1 --2- 172.21.15.101:0/1475887201 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae8e2bad0 0x555ae8e30b90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7a38007bf0 tx=0x7f7a38009130 comp rx=0 tx=0).stop 2023-07-18T03:10:09.070 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.043+0000 7f7a5b7cfb80 1 -- 172.21.15.101:0/1475887201 shutdown_connections 2023-07-18T03:10:09.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.043+0000 7f7a5b7cfb80 1 --2- 172.21.15.101:0/1475887201 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x555ae8e90700 0x555ae8e92b10 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.043+0000 7f7a5b7cfb80 1 --2- 172.21.15.101:0/1475887201 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ae8e2bad0 0x555ae8e30b90 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.043+0000 7f7a5b7cfb80 1 --2- 172.21.15.101:0/1475887201 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ae8e202d0 0x555ae8e32320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.043+0000 7f7a5b7cfb80 1 --2- 172.21.15.101:0/1475887201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ae8e20be0 0x555ae8e30650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.043+0000 7f7a297fa700 0 client.26110 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:10:09.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.043+0000 7f7a5b7cfb80 1 -- 172.21.15.101:0/1475887201 >> 172.21.15.101:0/1475887201 conn(0x555ae8e16da0 msgr2=0x555ae8e223e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:09.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.043+0000 7f7a5b7cfb80 1 -- 172.21.15.101:0/1475887201 shutdown_connections 2023-07-18T03:10:09.075 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.043+0000 7f7a5b7cfb80 1 -- 172.21.15.101:0/1475887201 wait complete. 2023-07-18T03:10:09.075 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.043+0000 7f7a5b7cfb80 20 client.26110 trim_cache size 0 max 16384 2023-07-18T03:10:09.077 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:10:09.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.044+0000 7f7a47dff700 1 -- 172.21.15.101:0/3762621743 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f7a2c0a12c0 msgr2=0x7f7a2c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:09.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.044+0000 7f7a47dff700 1 --2- 172.21.15.101:0/3762621743 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f7a2c0a12c0 0x7f7a2c0a3770 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7a40074920 tx=0x7f7a3800b040 comp rx=0 tx=0).stop 2023-07-18T03:10:09.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.044+0000 7f7a47dff700 1 -- 172.21.15.101:0/3762621743 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a40100490 msgr2=0x7f7a40072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:09.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.044+0000 7f7a47dff700 1 --2- 172.21.15.101:0/3762621743 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a40100490 0x7f7a40072d50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7a30009810 tx=0x7f7a3000fa70 comp rx=0 tx=0).stop 2023-07-18T03:10:09.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.044+0000 7f7a47dff700 1 -- 172.21.15.101:0/3762621743 shutdown_connections 2023-07-18T03:10:09.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.044+0000 7f7a47dff700 1 --2- 172.21.15.101:0/3762621743 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f7a2c0a12c0 0x7f7a2c0a3770 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.044+0000 7f7a47dff700 1 --2- 172.21.15.101:0/3762621743 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a400ffae0 0x7f7a40072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.044+0000 7f7a47dff700 1 --2- 172.21.15.101:0/3762621743 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7a40105af0 0x7f7a401a3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.044+0000 7f7a47dff700 1 --2- 172.21.15.101:0/3762621743 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a40100490 0x7f7a40072d50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.044+0000 7f7a47dff700 1 -- 172.21.15.101:0/3762621743 >> 172.21.15.101:0/3762621743 conn(0x7f7a400fb5e0 msgr2=0x7f7a401035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:09.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.045+0000 7f7a47dff700 1 -- 172.21.15.101:0/3762621743 shutdown_connections 2023-07-18T03:10:09.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.045+0000 7f7a47dff700 1 -- 172.21.15.101:0/3762621743 wait complete. 2023-07-18T03:10:09.079 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:08 smithi101 ceph-mon[126379]: pgmap v80: 129 pgs: 129 active+clean; 83 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 1.1 MiB/s wr, 197 op/s 2023-07-18T03:10:09.080 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:08 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/713378807' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:10:09.080 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-18T03:10:09.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:08 smithi155 ceph-mon[128885]: pgmap v80: 129 pgs: 129 active+clean; 83 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 1.1 MiB/s wr, 197 op/s 2023-07-18T03:10:09.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:08 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/713378807' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:10:09.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.306+0000 7f29e6bff700 1 -- 172.21.15.101:0/3294222623 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f29e00731c0 msgr2=0x7f29e010bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:09.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.306+0000 7f29e6bff700 1 --2- 172.21.15.101:0/3294222623 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f29e00731c0 0x7f29e010bde0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f29cc009990 tx=0x7f29cc009ca0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.306+0000 7f29e6bff700 1 -- 172.21.15.101:0/3294222623 shutdown_connections 2023-07-18T03:10:09.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.306+0000 7f29e6bff700 1 --2- 172.21.15.101:0/3294222623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29e010c320 0x7f29e010e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.306+0000 7f29e6bff700 1 --2- 172.21.15.101:0/3294222623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29e00728b0 0x7f29e0072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.306+0000 7f29e6bff700 1 --2- 172.21.15.101:0/3294222623 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f29e00731c0 0x7f29e010bde0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.306+0000 7f29e6bff700 1 -- 172.21.15.101:0/3294222623 >> 172.21.15.101:0/3294222623 conn(0x7f29e00fb5e0 msgr2=0x7f29e00fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:09.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.306+0000 7f29e6bff700 1 -- 172.21.15.101:0/3294222623 shutdown_connections 2023-07-18T03:10:09.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.306+0000 7f29e6bff700 1 -- 172.21.15.101:0/3294222623 wait complete. 2023-07-18T03:10:09.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.308+0000 7f29e6bff700 1 Processor -- start 2023-07-18T03:10:09.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.308+0000 7f29e6bff700 1 -- start start 2023-07-18T03:10:09.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.308+0000 7f29e6bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f29e00728b0 0x7f29e0197d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.308+0000 7f29e6bff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29e00731c0 0x7f29e01982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.308+0000 7f29e6bff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29e010c320 0x7f29e019c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.308+0000 7f29e6bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e019cd60 con 0x7f29e00728b0 2023-07-18T03:10:09.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.308+0000 7f29e6bff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e019ced0 con 0x7f29e010c320 2023-07-18T03:10:09.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.308+0000 7f29e6bff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e019d090 con 0x7f29e00731c0 2023-07-18T03:10:09.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.309+0000 7f29dffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29e00731c0 0x7f29e01982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.309+0000 7f29dffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29e00731c0 0x7f29e01982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:50588/0 (socket says 172.21.15.101:50588) 2023-07-18T03:10:09.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.309+0000 7f29dffff700 1 -- 172.21.15.101:0/2410387049 learned_addr learned my addr 172.21.15.101:0/2410387049 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:09.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.309+0000 7f29e519c700 1 --2- 172.21.15.101:0/2410387049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29e010c320 0x7f29e019c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.309+0000 7f29e499b700 1 --2- 172.21.15.101:0/2410387049 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f29e00728b0 0x7f29e0197d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.309+0000 7f29dffff700 1 -- 172.21.15.101:0/2410387049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29e010c320 msgr2=0x7f29e019c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:09.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.309+0000 7f29dffff700 1 --2- 172.21.15.101:0/2410387049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29e010c320 0x7f29e019c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.309+0000 7f29dffff700 1 -- 172.21.15.101:0/2410387049 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f29e00728b0 msgr2=0x7f29e0197d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:09.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.309+0000 7f29dffff700 1 --2- 172.21.15.101:0/2410387049 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f29e00728b0 0x7f29e0197d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.309+0000 7f29dffff700 1 -- 172.21.15.101:0/2410387049 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29cc0095f0 con 0x7f29e00731c0 2023-07-18T03:10:09.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.309+0000 7f29e519c700 1 --2- 172.21.15.101:0/2410387049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29e010c320 0x7f29e019c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:10:09.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.310+0000 7f29dffff700 1 --2- 172.21.15.101:0/2410387049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29e00731c0 0x7f29e01982c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f29cc000c00 tx=0x7f29cc004790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:09.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.310+0000 7f29ddffb700 1 -- 172.21.15.101:0/2410387049 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc01d070 con 0x7f29e00731c0 2023-07-18T03:10:09.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.310+0000 7f29ddffb700 1 -- 172.21.15.101:0/2410387049 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29cc021410 con 0x7f29e00731c0 2023-07-18T03:10:09.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.310+0000 7f29e6bff700 1 -- 172.21.15.101:0/2410387049 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29e019d310 con 0x7f29e00731c0 2023-07-18T03:10:09.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.310+0000 7f29ddffb700 1 -- 172.21.15.101:0/2410387049 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc017650 con 0x7f29e00731c0 2023-07-18T03:10:09.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.310+0000 7f29e6bff700 1 -- 172.21.15.101:0/2410387049 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e01a4bb0 con 0x7f29e00731c0 2023-07-18T03:10:09.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.314+0000 7f29ddffb700 1 -- 172.21.15.101:0/2410387049 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f29cc021580 con 0x7f29e00731c0 2023-07-18T03:10:09.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.315+0000 7f29fa5ddb80 1 Processor -- start 2023-07-18T03:10:09.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.315+0000 7f29fa5ddb80 1 -- start start 2023-07-18T03:10:09.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.315+0000 7f29ddffb700 1 --2- 172.21.15.101:0/2410387049 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f29d00a1340 0x7f29d00a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.315+0000 7f29ddffb700 1 -- 172.21.15.101:0/2410387049 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f29cc0ce2f0 con 0x7f29e00731c0 2023-07-18T03:10:09.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.315+0000 7f29e519c700 1 --2- 172.21.15.101:0/2410387049 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f29d00a1340 0x7f29d00a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.315+0000 7f29fa5ddb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56254118c2d0 0x56254118c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.315+0000 7f29fa5ddb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56254118cbe0 0x56254119fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.316+0000 7f29fa5ddb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5625411a02e0 0x5625411a26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.316+0000 7f29fa5ddb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562541092cb0 con 0x5625411a02e0 2023-07-18T03:10:09.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.316+0000 7f29fa5ddb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625410e5a40 con 0x56254118cbe0 2023-07-18T03:10:09.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.316+0000 7f29fa5ddb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625410deed0 con 0x56254118c2d0 2023-07-18T03:10:09.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.316+0000 7f29e499b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56254118c2d0 0x56254118c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.316+0000 7f29e499b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56254118c2d0 0x56254118c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:50594/0 (socket says 172.21.15.101:50594) 2023-07-18T03:10:09.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.316+0000 7f29e499b700 1 -- 172.21.15.101:0/58122960 learned_addr learned my addr 172.21.15.101:0/58122960 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:09.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.316+0000 7f29e519c700 1 --2- 172.21.15.101:0/58122960 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5625411a02e0 0x5625411a26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.316+0000 7f29dffff700 1 --2- 172.21.15.101:0/58122960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56254118cbe0 0x56254119fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.316+0000 7f29e499b700 1 -- 172.21.15.101:0/58122960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56254118cbe0 msgr2=0x56254119fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:09.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.316+0000 7f29e499b700 1 --2- 172.21.15.101:0/58122960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56254118cbe0 0x56254119fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.316+0000 7f29e499b700 1 -- 172.21.15.101:0/58122960 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5625411a02e0 msgr2=0x5625411a26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:09.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.316+0000 7f29e499b700 1 --2- 172.21.15.101:0/58122960 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5625411a02e0 0x5625411a26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.316+0000 7f29e499b700 1 -- 172.21.15.101:0/58122960 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562540f0cde0 con 0x56254118c2d0 2023-07-18T03:10:09.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.316+0000 7f29e519c700 1 --2- 172.21.15.101:0/2410387049 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f29d00a1340 0x7f29d00a37f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f29d8004230 tx=0x7f29d80045f0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:09.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.317+0000 7f29e499b700 1 --2- 172.21.15.101:0/58122960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56254118c2d0 0x56254118c6a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f29d4009d80 tx=0x7f29d4011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae7ca38eb7bbf2dc server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:09.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.317+0000 7f29caffd700 1 -- 172.21.15.101:0/58122960 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d4004230 con 0x56254118c2d0 2023-07-18T03:10:09.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.317+0000 7f29caffd700 1 -- 172.21.15.101:0/58122960 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29d4004390 con 0x56254118c2d0 2023-07-18T03:10:09.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.317+0000 7f29caffd700 1 -- 172.21.15.101:0/58122960 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d40198c0 con 0x56254118c2d0 2023-07-18T03:10:09.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.318+0000 7f29fa5ddb80 1 -- 172.21.15.101:0/58122960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56254118c2d0 msgr2=0x56254118c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:09.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.318+0000 7f29fa5ddb80 1 --2- 172.21.15.101:0/58122960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56254118c2d0 0x56254118c6a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f29d4009d80 tx=0x7f29d4011720 comp rx=0 tx=0).stop 2023-07-18T03:10:09.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.318+0000 7f29fa5ddb80 1 -- 172.21.15.101:0/58122960 shutdown_connections 2023-07-18T03:10:09.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.318+0000 7f29fa5ddb80 1 --2- 172.21.15.101:0/58122960 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5625411a02e0 0x5625411a26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.318+0000 7f29fa5ddb80 1 --2- 172.21.15.101:0/58122960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56254118c2d0 0x56254118c6a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.318+0000 7f29fa5ddb80 1 --2- 172.21.15.101:0/58122960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56254118cbe0 0x56254119fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.318+0000 7f29fa5ddb80 1 -- 172.21.15.101:0/58122960 >> 172.21.15.101:0/58122960 conn(0x562541182da0 msgr2=0x5625411886a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:09.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.318+0000 7f29fa5ddb80 1 -- 172.21.15.101:0/58122960 shutdown_connections 2023-07-18T03:10:09.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.318+0000 7f29fa5ddb80 1 -- 172.21.15.101:0/58122960 wait complete. 2023-07-18T03:10:09.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.318+0000 7f29fa5ddb80 1 -- start start 2023-07-18T03:10:09.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.319+0000 7f29fa5ddb80 1 Processor -- start 2023-07-18T03:10:09.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.319+0000 7f29fa5ddb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56254118c2d0 0x56254119fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.319+0000 7f29fa5ddb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56254118cbe0 0x56254119d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.319+0000 7f29fa5ddb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5625411a02e0 0x56254119d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.319+0000 7f29fa5ddb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254107eac0 con 0x56254118cbe0 2023-07-18T03:10:09.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.319+0000 7f29fa5ddb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254107e7c0 con 0x5625411a02e0 2023-07-18T03:10:09.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.319+0000 7f29fa5ddb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562540f62920 con 0x56254118c2d0 2023-07-18T03:10:09.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.319+0000 7f29fa5ddb80 10 client.0 _ll_register_callbacks cb 0x56254107c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:10:09.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.319+0000 7f29e499b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56254118c2d0 0x56254119fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.319+0000 7f29e499b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56254118c2d0 0x56254119fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:50598/0 (socket says 172.21.15.101:50598) 2023-07-18T03:10:09.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.319+0000 7f29e519c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5625411a02e0 0x56254119d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.319+0000 7f29e499b700 1 -- 172.21.15.101:0/2972325892 learned_addr learned my addr 172.21.15.101:0/2972325892 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:09.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.319+0000 7f29dffff700 1 --2- 172.21.15.101:0/2972325892 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56254118cbe0 0x56254119d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.320+0000 7f29e519c700 1 -- 172.21.15.101:0/2972325892 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56254118c2d0 msgr2=0x56254119fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:09.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.320+0000 7f29e519c700 1 --2- 172.21.15.101:0/2972325892 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56254118c2d0 0x56254119fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.320+0000 7f29e519c700 1 -- 172.21.15.101:0/2972325892 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56254118cbe0 msgr2=0x56254119d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:09.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.320+0000 7f29e519c700 1 --2- 172.21.15.101:0/2972325892 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56254118cbe0 0x56254119d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.320+0000 7f29e519c700 1 -- 172.21.15.101:0/2972325892 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29d40099e0 con 0x5625411a02e0 2023-07-18T03:10:09.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.320+0000 7f29e499b700 1 --2- 172.21.15.101:0/2972325892 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56254118c2d0 0x56254119fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:10:09.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.320+0000 7f29e519c700 1 --2- 172.21.15.101:0/2972325892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5625411a02e0 0x56254119d730 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f29d8004a60 tx=0x7f29d8004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b30db8f5c7a4386 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:09.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.320+0000 7f29bffff700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:10:09.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.320+0000 7f29bffff700 1 -- 172.21.15.101:0/2972325892 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d802c400 con 0x5625411a02e0 2023-07-18T03:10:09.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.320+0000 7f29bffff700 1 -- 172.21.15.101:0/2972325892 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f29d802c560 con 0x5625411a02e0 2023-07-18T03:10:09.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.320+0000 7f29fa5ddb80 1 -- 172.21.15.101:0/2972325892 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5625410384f0 con 0x5625411a02e0 2023-07-18T03:10:09.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.321+0000 7f29bffff700 1 -- 172.21.15.101:0/2972325892 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29d802c720 con 0x5625411a02e0 2023-07-18T03:10:09.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.321+0000 7f29fa5ddb80 10 client.26096 fetch_fsmap learned FSMap version 25 2023-07-18T03:10:09.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.321+0000 7f29fa5ddb80 1 -- 172.21.15.101:0/2972325892 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562540ef5c60 con 0x5625411a02e0 2023-07-18T03:10:09.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.321+0000 7f29bffff700 1 -- 172.21.15.101:0/2972325892 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5625410384f0 con 0x5625411a02e0 2023-07-18T03:10:09.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.321+0000 7f29bffff700 1 -- 172.21.15.101:0/2972325892 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f29d802a020 con 0x5625411a02e0 2023-07-18T03:10:09.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.321+0000 7f29fa5ddb80 10 client.26096 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:10:09.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.321+0000 7f29fa5ddb80 10 client.26096 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:10:09.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.321+0000 7f29fa5ddb80 20 client.26096 populate_metadata read hostname 'smithi101' 2023-07-18T03:10:09.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.321+0000 7f29fa5ddb80 1 --2- 172.21.15.101:0/2972325892 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5625411afc70 0x5625411b2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.321+0000 7f29fa5ddb80 4 client.26096 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:10:09.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.321+0000 7f29fa5ddb80 1 -- 172.21.15.101:0/2972325892 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5625410d7eb0 con 0x5625411afc70 2023-07-18T03:10:09.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.321+0000 7f29e499b700 1 --2- 172.21.15.101:0/2972325892 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5625411afc70 0x5625411b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.322+0000 7f29e499b700 1 --2- 172.21.15.101:0/2972325892 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5625411afc70 0x5625411b2080 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eae85df8cf461e server_cookie=b075f0c2dc4e2390 in_seq=0 out_seq=0 2023-07-18T03:10:09.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.322+0000 7f29bffff700 10 client.26096 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:10:09.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.322+0000 7f29bffff700 1 -- 172.21.15.101:0/2972325892 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5625410d7eb0 con 0x5625411afc70 2023-07-18T03:10:09.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.322+0000 7f29bffff700 10 client.26096 handle_command_reply: tid=0 2023-07-18T03:10:09.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.323+0000 7f29fa5ddb80 1 client.26096 shutdown 2023-07-18T03:10:09.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.323+0000 7f29fa5ddb80 1 -- 172.21.15.101:0/2972325892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5625411a02e0 msgr2=0x56254119d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:09.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.323+0000 7f29fa5ddb80 1 --2- 172.21.15.101:0/2972325892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5625411a02e0 0x56254119d730 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f29d8004a60 tx=0x7f29d8004b40 comp rx=0 tx=0).stop 2023-07-18T03:10:09.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.323+0000 7f29fa5ddb80 1 -- 172.21.15.101:0/2972325892 shutdown_connections 2023-07-18T03:10:09.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.323+0000 7f29fa5ddb80 1 --2- 172.21.15.101:0/2972325892 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5625411afc70 0x5625411b2080 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.323+0000 7f29fa5ddb80 1 --2- 172.21.15.101:0/2972325892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5625411a02e0 0x56254119d730 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.323+0000 7f29bffff700 0 client.26096 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:10:09.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.323+0000 7f29fa5ddb80 1 --2- 172.21.15.101:0/2972325892 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56254118c2d0 0x56254119fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.323+0000 7f29fa5ddb80 1 --2- 172.21.15.101:0/2972325892 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56254118cbe0 0x56254119d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.323+0000 7f29fa5ddb80 1 -- 172.21.15.101:0/2972325892 >> 172.21.15.101:0/2972325892 conn(0x562541182da0 msgr2=0x562541188290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:09.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.323+0000 7f29fa5ddb80 1 -- 172.21.15.101:0/2972325892 shutdown_connections 2023-07-18T03:10:09.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.323+0000 7f29fa5ddb80 1 -- 172.21.15.101:0/2972325892 wait complete. 2023-07-18T03:10:09.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.323+0000 7f29fa5ddb80 20 client.26096 trim_cache size 0 max 16384 2023-07-18T03:10:09.328 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.338+0000 7f29fa5ddb80 1 Processor -- start 2023-07-18T03:10:09.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.338+0000 7f29fa5ddb80 1 -- start start 2023-07-18T03:10:09.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.338+0000 7f29fa5ddb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56254118c2d0 0x56254119d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.338+0000 7f29fa5ddb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56254118cbe0 0x56254119c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.338+0000 7f29fa5ddb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625411a02e0 0x5625411ab4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.338+0000 7f29fa5ddb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254119e5b0 con 0x56254118c2d0 2023-07-18T03:10:09.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.338+0000 7f29fa5ddb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625411afa50 con 0x56254118cbe0 2023-07-18T03:10:09.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.338+0000 7f29fa5ddb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562541182970 con 0x5625411a02e0 2023-07-18T03:10:09.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.338+0000 7f29e499b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56254118c2d0 0x56254119d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.338+0000 7f29e519c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625411a02e0 0x5625411ab4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.338+0000 7f29e499b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56254118c2d0 0x56254119d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:44752/0 (socket says 172.21.15.101:44752) 2023-07-18T03:10:09.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.338+0000 7f29e499b700 1 -- 172.21.15.101:0/1535075060 learned_addr learned my addr 172.21.15.101:0/1535075060 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:09.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.338+0000 7f29e519c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625411a02e0 0x5625411ab4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:50608/0 (socket says 172.21.15.101:50608) 2023-07-18T03:10:09.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.338+0000 7f29dffff700 1 --2- 172.21.15.101:0/1535075060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56254118cbe0 0x56254119c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.339+0000 7f29e519c700 1 -- 172.21.15.101:0/1535075060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56254118cbe0 msgr2=0x56254119c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:09.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.339+0000 7f29e519c700 1 --2- 172.21.15.101:0/1535075060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56254118cbe0 0x56254119c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.339+0000 7f29e519c700 1 -- 172.21.15.101:0/1535075060 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56254118c2d0 msgr2=0x56254119d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:09.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.339+0000 7f29e519c700 1 --2- 172.21.15.101:0/1535075060 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56254118c2d0 0x56254119d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.339+0000 7f29e519c700 1 -- 172.21.15.101:0/1535075060 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562540f33250 con 0x5625411a02e0 2023-07-18T03:10:09.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.339+0000 7f29e519c700 1 --2- 172.21.15.101:0/1535075060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625411a02e0 0x5625411ab4a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f29d8009fd0 tx=0x7f29d80098a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2ca8dec353084fc server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:09.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.339+0000 7f29cb7fe700 1 -- 172.21.15.101:0/1535075060 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d8023ea0 con 0x5625411a02e0 2023-07-18T03:10:09.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.339+0000 7f29cb7fe700 1 -- 172.21.15.101:0/1535075060 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29d800f040 con 0x5625411a02e0 2023-07-18T03:10:09.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.339+0000 7f29cb7fe700 1 -- 172.21.15.101:0/1535075060 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d8039800 con 0x5625411a02e0 2023-07-18T03:10:09.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.340+0000 7f29fa5ddb80 1 -- 172.21.15.101:0/1535075060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625411a02e0 msgr2=0x5625411ab4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:09.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.340+0000 7f29fa5ddb80 1 --2- 172.21.15.101:0/1535075060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625411a02e0 0x5625411ab4a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f29d8009fd0 tx=0x7f29d80098a0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.340+0000 7f29fa5ddb80 1 -- 172.21.15.101:0/1535075060 shutdown_connections 2023-07-18T03:10:09.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.340+0000 7f29fa5ddb80 1 --2- 172.21.15.101:0/1535075060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625411a02e0 0x5625411ab4a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.340+0000 7f29fa5ddb80 1 --2- 172.21.15.101:0/1535075060 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56254118c2d0 0x56254119d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.340+0000 7f29fa5ddb80 1 --2- 172.21.15.101:0/1535075060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56254118cbe0 0x56254119c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.340+0000 7f29fa5ddb80 1 -- 172.21.15.101:0/1535075060 >> 172.21.15.101:0/1535075060 conn(0x562541182da0 msgr2=0x562541186e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:09.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.340+0000 7f29fa5ddb80 1 -- 172.21.15.101:0/1535075060 shutdown_connections 2023-07-18T03:10:09.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.340+0000 7f29fa5ddb80 1 -- 172.21.15.101:0/1535075060 wait complete. 2023-07-18T03:10:09.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.340+0000 7f29fa5ddb80 1 -- start start 2023-07-18T03:10:09.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.341+0000 7f29fa5ddb80 1 Processor -- start 2023-07-18T03:10:09.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.341+0000 7f29fa5ddb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56254118c2d0 0x5625411a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.341+0000 7f29fa5ddb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56254118cbe0 0x5625411a5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.341+0000 7f29fa5ddb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5625411a02e0 0x562541202b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.341+0000 7f29fa5ddb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562541203070 con 0x56254118cbe0 2023-07-18T03:10:09.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.341+0000 7f29fa5ddb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625412031b0 con 0x5625411a02e0 2023-07-18T03:10:09.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.341+0000 7f29fa5ddb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562541203320 con 0x56254118c2d0 2023-07-18T03:10:09.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.341+0000 7f29fa5ddb80 10 client.0 _ll_register_callbacks cb 0x7f29d800ec10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:10:09.338 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.341+0000 7f29e499b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56254118c2d0 0x5625411a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.338 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.342+0000 7f29dffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56254118cbe0 0x5625411a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.338 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.342+0000 7f29e499b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56254118c2d0 0x5625411a4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:50612/0 (socket says 172.21.15.101:50612) 2023-07-18T03:10:09.338 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.342+0000 7f29e499b700 1 -- 172.21.15.101:0/1608050242 learned_addr learned my addr 172.21.15.101:0/1608050242 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:09.338 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.342+0000 7f29e519c700 1 --2- 172.21.15.101:0/1608050242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5625411a02e0 0x562541202b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.338 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.342+0000 7f29e499b700 1 -- 172.21.15.101:0/1608050242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5625411a02e0 msgr2=0x562541202b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:09.338 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.342+0000 7f29e499b700 1 --2- 172.21.15.101:0/1608050242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5625411a02e0 0x562541202b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.339 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.342+0000 7f29e499b700 1 -- 172.21.15.101:0/1608050242 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56254118cbe0 msgr2=0x5625411a5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:09.339 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.342+0000 7f29e499b700 1 --2- 172.21.15.101:0/1608050242 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56254118cbe0 0x5625411a5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.339 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.342+0000 7f29e499b700 1 -- 172.21.15.101:0/1608050242 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562540ef5c60 con 0x56254118c2d0 2023-07-18T03:10:09.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.342+0000 7f29e499b700 1 --2- 172.21.15.101:0/1608050242 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56254118c2d0 0x5625411a4ed0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f29d4003f40 tx=0x7f29d4003ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bdfde3a5d6eed20 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:09.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.342+0000 7f29bffff700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:10:09.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.342+0000 7f29bffff700 1 -- 172.21.15.101:0/1608050242 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d4029040 con 0x56254118c2d0 2023-07-18T03:10:09.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.342+0000 7f29fa5ddb80 1 -- 172.21.15.101:0/1608050242 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f29d4004390 con 0x56254118c2d0 2023-07-18T03:10:09.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.342+0000 7f29bffff700 1 -- 172.21.15.101:0/1608050242 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f29d4003680 con 0x56254118c2d0 2023-07-18T03:10:09.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.343+0000 7f29bffff700 1 -- 172.21.15.101:0/1608050242 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29d402a620 con 0x56254118c2d0 2023-07-18T03:10:09.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.343+0000 7f29fa5ddb80 10 client.26128 fetch_fsmap learned FSMap version 25 2023-07-18T03:10:09.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.343+0000 7f29fa5ddb80 1 -- 172.21.15.101:0/1608050242 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562540f0b2b0 con 0x56254118c2d0 2023-07-18T03:10:09.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.343+0000 7f29bffff700 1 -- 172.21.15.101:0/1608050242 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29d4004390 con 0x56254118c2d0 2023-07-18T03:10:09.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.343+0000 7f29bffff700 1 -- 172.21.15.101:0/1608050242 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f29d4011440 con 0x56254118c2d0 2023-07-18T03:10:09.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.343+0000 7f29fa5ddb80 10 client.26128 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:10:09.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.343+0000 7f29fa5ddb80 10 client.26128 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:10:09.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.343+0000 7f29fa5ddb80 20 client.26128 populate_metadata read hostname 'smithi101' 2023-07-18T03:10:09.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.343+0000 7f29fa5ddb80 1 --2- 172.21.15.101:0/1608050242 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x562541204a30 0x562541206e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.343+0000 7f29fa5ddb80 4 client.26128 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:10:09.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.343+0000 7f29fa5ddb80 1 -- 172.21.15.101:0/1608050242 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56254109a050 con 0x562541204a30 2023-07-18T03:10:09.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.344+0000 7f29dffff700 1 --2- 172.21.15.101:0/1608050242 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x562541204a30 0x562541206e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.344+0000 7f29dffff700 1 --2- 172.21.15.101:0/1608050242 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x562541204a30 0x562541206e40 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7c746a738c22ae7 server_cookie=955578286733d9ee in_seq=0 out_seq=0 2023-07-18T03:10:09.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.344+0000 7f29bffff700 10 client.26128 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:10:09.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.344+0000 7f29bffff700 1 -- 172.21.15.101:0/1608050242 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56254109a050 con 0x562541204a30 2023-07-18T03:10:09.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.344+0000 7f29bffff700 10 client.26128 handle_command_reply: tid=0 2023-07-18T03:10:09.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.344+0000 7f29fa5ddb80 1 client.26128 shutdown 2023-07-18T03:10:09.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.344+0000 7f29fa5ddb80 1 -- 172.21.15.101:0/1608050242 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56254118c2d0 msgr2=0x5625411a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:09.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.344+0000 7f29fa5ddb80 1 --2- 172.21.15.101:0/1608050242 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56254118c2d0 0x5625411a4ed0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f29d4003f40 tx=0x7f29d4003ea0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.344+0000 7f29fa5ddb80 1 -- 172.21.15.101:0/1608050242 shutdown_connections 2023-07-18T03:10:09.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.344+0000 7f29fa5ddb80 1 --2- 172.21.15.101:0/1608050242 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x562541204a30 0x562541206e40 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.344+0000 7f29fa5ddb80 1 --2- 172.21.15.101:0/1608050242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5625411a02e0 0x562541202b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.344+0000 7f29bffff700 0 client.26128 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:10:09.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.344+0000 7f29fa5ddb80 1 --2- 172.21.15.101:0/1608050242 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56254118c2d0 0x5625411a4ed0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.344+0000 7f29fa5ddb80 1 --2- 172.21.15.101:0/1608050242 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56254118cbe0 0x5625411a5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.344+0000 7f29fa5ddb80 1 -- 172.21.15.101:0/1608050242 >> 172.21.15.101:0/1608050242 conn(0x562541182da0 msgr2=0x5625411ab4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:09.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.344+0000 7f29fa5ddb80 1 -- 172.21.15.101:0/1608050242 shutdown_connections 2023-07-18T03:10:09.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.344+0000 7f29fa5ddb80 1 -- 172.21.15.101:0/1608050242 wait complete. 2023-07-18T03:10:09.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.345+0000 7f29fa5ddb80 20 client.26128 trim_cache size 0 max 16384 2023-07-18T03:10:09.347 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:10:09.348 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.345+0000 7f29e6bff700 1 -- 172.21.15.101:0/2410387049 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f29d00a1340 msgr2=0x7f29d00a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:09.348 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.345+0000 7f29e6bff700 1 --2- 172.21.15.101:0/2410387049 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f29d00a1340 0x7f29d00a37f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f29d8004230 tx=0x7f29d80045f0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.348 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.345+0000 7f29e6bff700 1 -- 172.21.15.101:0/2410387049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29e00731c0 msgr2=0x7f29e01982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:09.348 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.345+0000 7f29e6bff700 1 --2- 172.21.15.101:0/2410387049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29e00731c0 0x7f29e01982c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f29cc000c00 tx=0x7f29cc004790 comp rx=0 tx=0).stop 2023-07-18T03:10:09.349 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.345+0000 7f29e6bff700 1 -- 172.21.15.101:0/2410387049 shutdown_connections 2023-07-18T03:10:09.349 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.345+0000 7f29e6bff700 1 --2- 172.21.15.101:0/2410387049 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f29d00a1340 0x7f29d00a37f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.349 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.345+0000 7f29e6bff700 1 --2- 172.21.15.101:0/2410387049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29e010c320 0x7f29e019c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.349 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.345+0000 7f29e6bff700 1 --2- 172.21.15.101:0/2410387049 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f29e00728b0 0x7f29e0197d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.349 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.345+0000 7f29e6bff700 1 --2- 172.21.15.101:0/2410387049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29e00731c0 0x7f29e01982c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.349 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.345+0000 7f29e6bff700 1 -- 172.21.15.101:0/2410387049 >> 172.21.15.101:0/2410387049 conn(0x7f29e00fb5e0 msgr2=0x7f29e010e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:09.349 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.345+0000 7f29e6bff700 1 -- 172.21.15.101:0/2410387049 shutdown_connections 2023-07-18T03:10:09.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.345+0000 7f29e6bff700 1 -- 172.21.15.101:0/2410387049 wait complete. 2023-07-18T03:10:09.355 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-18T03:10:09.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:08 smithi179 ceph-mon[129229]: pgmap v80: 129 pgs: 129 active+clean; 83 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 1.1 MiB/s wr, 197 op/s 2023-07-18T03:10:09.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:08 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/713378807' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:10:09.568 INFO:tasks.workunit.client.0.smithi101.stdout: 1 135185 7.95 MB/sec warmup 94 sec latency 24.731 ms 2023-07-18T03:10:09.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.582+0000 7f38681ff700 1 -- 172.21.15.101:0/465194496 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38600ffad0 msgr2=0x7f38600fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:09.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.582+0000 7f38681ff700 1 --2- 172.21.15.101:0/465194496 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38600ffad0 0x7f38600fff40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3850009990 tx=0x7f3850009ca0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.583+0000 7f38681ff700 1 -- 172.21.15.101:0/465194496 shutdown_connections 2023-07-18T03:10:09.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.583+0000 7f38681ff700 1 --2- 172.21.15.101:0/465194496 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3860100480 0x7f386010c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.583+0000 7f38681ff700 1 --2- 172.21.15.101:0/465194496 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3860105ae0 0x7f3860105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.583+0000 7f38681ff700 1 --2- 172.21.15.101:0/465194496 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38600ffad0 0x7f38600fff40 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.583+0000 7f38681ff700 1 -- 172.21.15.101:0/465194496 >> 172.21.15.101:0/465194496 conn(0x7f38600fb5f0 msgr2=0x7f38600fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:09.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.583+0000 7f38681ff700 1 -- 172.21.15.101:0/465194496 shutdown_connections 2023-07-18T03:10:09.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.583+0000 7f38681ff700 1 -- 172.21.15.101:0/465194496 wait complete. 2023-07-18T03:10:09.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.584+0000 7f38681ff700 1 Processor -- start 2023-07-18T03:10:09.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.584+0000 7f38681ff700 1 -- start start 2023-07-18T03:10:09.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.585+0000 7f38681ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f38600ffad0 0x7f386019ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.585+0000 7f38681ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3860100480 0x7f386019bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.585+0000 7f38681ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3860105ae0 0x7f3860195a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.585+0000 7f38681ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38601960e0 con 0x7f38600ffad0 2023-07-18T03:10:09.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.585+0000 7f38681ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3860196250 con 0x7f3860105ae0 2023-07-18T03:10:09.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.585+0000 7f38681ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3860196410 con 0x7f3860100480 2023-07-18T03:10:09.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.585+0000 7f3865f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f38600ffad0 0x7f386019ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.585+0000 7f386579a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3860100480 0x7f386019bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.585+0000 7f386579a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3860100480 0x7f386019bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:50628/0 (socket says 172.21.15.101:50628) 2023-07-18T03:10:09.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.585+0000 7f386579a700 1 -- 172.21.15.101:0/977022578 learned_addr learned my addr 172.21.15.101:0/977022578 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:09.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.585+0000 7f386679c700 1 --2- 172.21.15.101:0/977022578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3860105ae0 0x7f3860195a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.586+0000 7f386579a700 1 -- 172.21.15.101:0/977022578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3860105ae0 msgr2=0x7f3860195a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:09.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.586+0000 7f386579a700 1 --2- 172.21.15.101:0/977022578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3860105ae0 0x7f3860195a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.586+0000 7f386579a700 1 -- 172.21.15.101:0/977022578 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f38600ffad0 msgr2=0x7f386019ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:09.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.586+0000 7f386579a700 1 --2- 172.21.15.101:0/977022578 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f38600ffad0 0x7f386019ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.586+0000 7f386579a700 1 -- 172.21.15.101:0/977022578 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38500095f0 con 0x7f3860100480 2023-07-18T03:10:09.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.586+0000 7f386579a700 1 --2- 172.21.15.101:0/977022578 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3860100480 0x7f386019bf40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3850005910 tx=0x7f3850005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:09.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.586+0000 7f3856ffd700 1 -- 172.21.15.101:0/977022578 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385001d070 con 0x7f3860100480 2023-07-18T03:10:09.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.586+0000 7f3856ffd700 1 -- 172.21.15.101:0/977022578 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3850022470 con 0x7f3860100480 2023-07-18T03:10:09.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.586+0000 7f38681ff700 1 -- 172.21.15.101:0/977022578 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3860196690 con 0x7f3860100480 2023-07-18T03:10:09.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.586+0000 7f3856ffd700 1 -- 172.21.15.101:0/977022578 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3850017650 con 0x7f3860100480 2023-07-18T03:10:09.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.586+0000 7f38681ff700 1 -- 172.21.15.101:0/977022578 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3860072920 con 0x7f3860100480 2023-07-18T03:10:09.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.590+0000 7f3856ffd700 1 -- 172.21.15.101:0/977022578 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f38500225e0 con 0x7f3860100480 2023-07-18T03:10:09.581 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.591+0000 7f387bbd2b80 1 Processor -- start 2023-07-18T03:10:09.582 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.591+0000 7f387bbd2b80 1 -- start start 2023-07-18T03:10:09.582 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.591+0000 7f3856ffd700 1 --2- 172.21.15.101:0/977022578 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f384c0a5750 0x7f384c0a7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.582 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.591+0000 7f387bbd2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1707712d0 0x55f1707716a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.591+0000 7f387bbd2b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f170771be0 0x55f17077c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.591+0000 7f387bbd2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f17077cad0 0x55f17077eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.591+0000 7f387bbd2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f170677cb0 con 0x55f17077cad0 2023-07-18T03:10:09.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.591+0000 7f387bbd2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1706caa40 con 0x55f1707712d0 2023-07-18T03:10:09.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.591+0000 7f387bbd2b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1706c3ed0 con 0x55f170771be0 2023-07-18T03:10:09.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.591+0000 7f3856ffd700 1 -- 172.21.15.101:0/977022578 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f38500ce320 con 0x7f3860100480 2023-07-18T03:10:09.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.592+0000 7f386579a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f170771be0 0x55f17077c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.592+0000 7f386579a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f170771be0 0x55f17077c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:50644/0 (socket says 172.21.15.101:50644) 2023-07-18T03:10:09.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.592+0000 7f3865f9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1707712d0 0x55f1707716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.592+0000 7f386579a700 1 -- 172.21.15.101:0/2933057838 learned_addr learned my addr 172.21.15.101:0/2933057838 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:09.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.592+0000 7f386679c700 1 --2- 172.21.15.101:0/977022578 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f384c0a5750 0x7f384c0a7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.592+0000 7f386579a700 1 -- 172.21.15.101:0/2933057838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1707712d0 msgr2=0x55f1707716a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:09.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.592+0000 7f386579a700 1 --2- 172.21.15.101:0/2933057838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1707712d0 0x55f1707716a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.592+0000 7f386579a700 1 -- 172.21.15.101:0/2933057838 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f17077cad0 msgr2=0x55f17077eeb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-18T03:10:09.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.592+0000 7f386679c700 1 --2- 172.21.15.101:0/2933057838 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f17077cad0 0x55f17077eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.592+0000 7f386579a700 1 --2- 172.21.15.101:0/2933057838 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f17077cad0 0x55f17077eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.592+0000 7f386579a700 1 -- 172.21.15.101:0/2933057838 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1704f1de0 con 0x55f170771be0 2023-07-18T03:10:09.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.592+0000 7f386679c700 1 --2- 172.21.15.101:0/2933057838 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f17077cad0 0x55f17077eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:10:09.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.592+0000 7f386579a700 1 --2- 172.21.15.101:0/2933057838 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f170771be0 0x55f17077c590 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3850097b40 tx=0x7f3850005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5ee9a408feb5622 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:09.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.592+0000 7f384bfff700 1 -- 172.21.15.101:0/2933057838 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385009ac10 con 0x55f170771be0 2023-07-18T03:10:09.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.593+0000 7f384bfff700 1 -- 172.21.15.101:0/2933057838 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f385009aa30 con 0x55f170771be0 2023-07-18T03:10:09.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.593+0000 7f384bfff700 1 -- 172.21.15.101:0/2933057838 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38500228c0 con 0x55f170771be0 2023-07-18T03:10:09.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.593+0000 7f387bbd2b80 1 -- 172.21.15.101:0/2933057838 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f170771be0 msgr2=0x55f17077c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:09.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.593+0000 7f386679c700 1 --2- 172.21.15.101:0/977022578 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f384c0a5750 0x7f384c0a7c00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f386019c700 tx=0x7f3858009920 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:09.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.593+0000 7f387bbd2b80 1 --2- 172.21.15.101:0/2933057838 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f170771be0 0x55f17077c590 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3850097b40 tx=0x7f3850005f10 comp rx=0 tx=0).stop 2023-07-18T03:10:09.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.593+0000 7f387bbd2b80 1 -- 172.21.15.101:0/2933057838 shutdown_connections 2023-07-18T03:10:09.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.593+0000 7f387bbd2b80 1 --2- 172.21.15.101:0/2933057838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1707712d0 0x55f1707716a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.593+0000 7f387bbd2b80 1 --2- 172.21.15.101:0/2933057838 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f17077cad0 0x55f17077eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.589 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.593+0000 7f387bbd2b80 1 --2- 172.21.15.101:0/2933057838 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f170771be0 0x55f17077c590 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.589 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.593+0000 7f387bbd2b80 1 -- 172.21.15.101:0/2933057838 >> 172.21.15.101:0/2933057838 conn(0x55f170767da0 msgr2=0x55f17076d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:09.589 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.593+0000 7f387bbd2b80 1 -- 172.21.15.101:0/2933057838 shutdown_connections 2023-07-18T03:10:09.589 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.594+0000 7f387bbd2b80 1 -- 172.21.15.101:0/2933057838 wait complete. 2023-07-18T03:10:09.589 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.595+0000 7f387bbd2b80 1 -- start start 2023-07-18T03:10:09.589 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.595+0000 7f387bbd2b80 1 Processor -- start 2023-07-18T03:10:09.589 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.596+0000 7f387bbd2b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f1707712d0 0x55f170781c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.596+0000 7f387bbd2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f170771be0 0x55f170782160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.596+0000 7f387bbd2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f17077cad0 0x55f17077ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.596+0000 7f387bbd2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f170663ac0 con 0x55f170771be0 2023-07-18T03:10:09.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.596+0000 7f387bbd2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1706637c0 con 0x55f17077cad0 2023-07-18T03:10:09.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.596+0000 7f387bbd2b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f170547920 con 0x55f1707712d0 2023-07-18T03:10:09.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.596+0000 7f387bbd2b80 10 client.0 _ll_register_callbacks cb 0x55f170661760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:10:09.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.596+0000 7f3865f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f1707712d0 0x55f170781c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.596+0000 7f3865f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f1707712d0 0x55f170781c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:50660/0 (socket says 172.21.15.101:50660) 2023-07-18T03:10:09.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.596+0000 7f3865f9b700 1 -- 172.21.15.101:0/2978557742 learned_addr learned my addr 172.21.15.101:0/2978557742 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:09.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.596+0000 7f386579a700 1 --2- 172.21.15.101:0/2978557742 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f170771be0 0x55f170782160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.596+0000 7f386679c700 1 --2- 172.21.15.101:0/2978557742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f17077cad0 0x55f17077ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.596+0000 7f3865f9b700 1 -- 172.21.15.101:0/2978557742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f17077cad0 msgr2=0x55f17077ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:09.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.596+0000 7f3865f9b700 1 --2- 172.21.15.101:0/2978557742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f17077cad0 0x55f17077ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.596+0000 7f3865f9b700 1 -- 172.21.15.101:0/2978557742 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f170771be0 msgr2=0x55f170782160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:09.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.596+0000 7f3865f9b700 1 --2- 172.21.15.101:0/2978557742 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f170771be0 0x55f170782160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.596+0000 7f3865f9b700 1 -- 172.21.15.101:0/2978557742 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3860072920 con 0x55f1707712d0 2023-07-18T03:10:09.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.597+0000 7f3865f9b700 1 --2- 172.21.15.101:0/2978557742 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f1707712d0 0x55f170781c20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f385c005c50 tx=0x7f385c012a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a5b6f0b684365b3 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:09.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.597+0000 7f38497fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:10:09.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.597+0000 7f38497fa700 1 -- 172.21.15.101:0/2978557742 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c017070 con 0x55f1707712d0 2023-07-18T03:10:09.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.597+0000 7f387bbd2b80 1 -- 172.21.15.101:0/2978557742 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f17061d4f0 con 0x55f1707712d0 2023-07-18T03:10:09.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.597+0000 7f38497fa700 1 -- 172.21.15.101:0/2978557742 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f385c00f480 con 0x55f1707712d0 2023-07-18T03:10:09.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.597+0000 7f38497fa700 1 -- 172.21.15.101:0/2978557742 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f385c0122e0 con 0x55f1707712d0 2023-07-18T03:10:09.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.597+0000 7f387bbd2b80 10 client.26140 fetch_fsmap learned FSMap version 25 2023-07-18T03:10:09.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.597+0000 7f387bbd2b80 1 -- 172.21.15.101:0/2978557742 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f1704dac60 con 0x55f1707712d0 2023-07-18T03:10:09.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.597+0000 7f38497fa700 1 -- 172.21.15.101:0/2978557742 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f17061d4f0 con 0x55f1707712d0 2023-07-18T03:10:09.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.597+0000 7f38497fa700 1 -- 172.21.15.101:0/2978557742 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f385c010830 con 0x55f1707712d0 2023-07-18T03:10:09.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.598+0000 7f387bbd2b80 10 client.26140 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:10:09.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.598+0000 7f387bbd2b80 10 client.26140 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:10:09.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.598+0000 7f387bbd2b80 20 client.26140 populate_metadata read hostname 'smithi101' 2023-07-18T03:10:09.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.598+0000 7f387bbd2b80 1 --2- 172.21.15.101:0/2978557742 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f17078c120 0x55f17078e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.598+0000 7f387bbd2b80 4 client.26140 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:10:09.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.598+0000 7f387bbd2b80 1 -- 172.21.15.101:0/2978557742 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f1706bceb0 con 0x55f17078c120 2023-07-18T03:10:09.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.598+0000 7f386579a700 1 --2- 172.21.15.101:0/2978557742 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f17078c120 0x55f17078e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.598+0000 7f386579a700 1 --2- 172.21.15.101:0/2978557742 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f17078c120 0x55f17078e530 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66df45ebfd53abfc server_cookie=45a1a68f58f6a05b in_seq=0 out_seq=0 2023-07-18T03:10:09.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.598+0000 7f38497fa700 10 client.26140 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:10:09.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.599+0000 7f38497fa700 1 -- 172.21.15.101:0/2978557742 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f1706bceb0 con 0x55f17078c120 2023-07-18T03:10:09.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.599+0000 7f38497fa700 10 client.26140 handle_command_reply: tid=0 2023-07-18T03:10:09.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.600+0000 7f387bbd2b80 1 client.26140 shutdown 2023-07-18T03:10:09.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.600+0000 7f387bbd2b80 1 -- 172.21.15.101:0/2978557742 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f1707712d0 msgr2=0x55f170781c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:09.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.600+0000 7f387bbd2b80 1 --2- 172.21.15.101:0/2978557742 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f1707712d0 0x55f170781c20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f385c005c50 tx=0x7f385c012a20 comp rx=0 tx=0).stop 2023-07-18T03:10:09.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.600+0000 7f387bbd2b80 1 -- 172.21.15.101:0/2978557742 shutdown_connections 2023-07-18T03:10:09.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.600+0000 7f387bbd2b80 1 --2- 172.21.15.101:0/2978557742 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f17078c120 0x55f17078e530 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.600+0000 7f387bbd2b80 1 --2- 172.21.15.101:0/2978557742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f17077cad0 0x55f17077ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.600+0000 7f387bbd2b80 1 --2- 172.21.15.101:0/2978557742 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f1707712d0 0x55f170781c20 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.600+0000 7f38497fa700 0 client.26140 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:10:09.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.600+0000 7f387bbd2b80 1 --2- 172.21.15.101:0/2978557742 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f170771be0 0x55f170782160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.600+0000 7f387bbd2b80 1 -- 172.21.15.101:0/2978557742 >> 172.21.15.101:0/2978557742 conn(0x55f170767da0 msgr2=0x55f17076d290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:09.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.600+0000 7f387bbd2b80 1 -- 172.21.15.101:0/2978557742 shutdown_connections 2023-07-18T03:10:09.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.600+0000 7f387bbd2b80 1 -- 172.21.15.101:0/2978557742 wait complete. 2023-07-18T03:10:09.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.600+0000 7f387bbd2b80 20 client.26140 trim_cache size 0 max 16384 2023-07-18T03:10:09.604 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.614+0000 7f387bbd2b80 1 Processor -- start 2023-07-18T03:10:09.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.614+0000 7f387bbd2b80 1 -- start start 2023-07-18T03:10:09.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.614+0000 7f387bbd2b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f1707712d0 0x55f170773af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.614+0000 7f387bbd2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f170771be0 0x55f1707817b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.614+0000 7f387bbd2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f17077cad0 0x55f170781cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.614+0000 7f387bbd2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f17077cff0 con 0x55f17077cad0 2023-07-18T03:10:09.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.614+0000 7f387bbd2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f17076ed50 con 0x55f170771be0 2023-07-18T03:10:09.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.614+0000 7f387bbd2b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f17077d450 con 0x55f1707712d0 2023-07-18T03:10:09.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.614+0000 7f3865f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f1707712d0 0x55f170773af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.614+0000 7f3865f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f1707712d0 0x55f170773af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:50670/0 (socket says 172.21.15.101:50670) 2023-07-18T03:10:09.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.614+0000 7f3865f9b700 1 -- 172.21.15.101:0/543650072 learned_addr learned my addr 172.21.15.101:0/543650072 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:09.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.614+0000 7f386679c700 1 --2- 172.21.15.101:0/543650072 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f17077cad0 0x55f170781cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.615+0000 7f386579a700 1 --2- 172.21.15.101:0/543650072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f170771be0 0x55f1707817b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.615+0000 7f3865f9b700 1 -- 172.21.15.101:0/543650072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f170771be0 msgr2=0x55f1707817b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:09.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.615+0000 7f3865f9b700 1 --2- 172.21.15.101:0/543650072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f170771be0 0x55f1707817b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.615+0000 7f3865f9b700 1 -- 172.21.15.101:0/543650072 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f17077cad0 msgr2=0x55f170781cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:09.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.615+0000 7f3865f9b700 1 --2- 172.21.15.101:0/543650072 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f17077cad0 0x55f170781cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.615+0000 7f3865f9b700 1 -- 172.21.15.101:0/543650072 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f170518250 con 0x55f1707712d0 2023-07-18T03:10:09.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.615+0000 7f3865f9b700 1 --2- 172.21.15.101:0/543650072 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f1707712d0 0x55f170773af0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f385c010c60 tx=0x7f385c009480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5427ce1eb1fccee2 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:09.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.615+0000 7f3854ff9700 1 -- 172.21.15.101:0/543650072 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c017040 con 0x55f1707712d0 2023-07-18T03:10:09.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.615+0000 7f3854ff9700 1 -- 172.21.15.101:0/543650072 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f385c0050c0 con 0x55f1707712d0 2023-07-18T03:10:09.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.615+0000 7f3854ff9700 1 -- 172.21.15.101:0/543650072 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c0038e0 con 0x55f1707712d0 2023-07-18T03:10:09.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.616+0000 7f387bbd2b80 1 -- 172.21.15.101:0/543650072 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f1707712d0 msgr2=0x55f170773af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:09.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.616+0000 7f387bbd2b80 1 --2- 172.21.15.101:0/543650072 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f1707712d0 0x55f170773af0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f385c010c60 tx=0x7f385c009480 comp rx=0 tx=0).stop 2023-07-18T03:10:09.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.616+0000 7f387bbd2b80 1 -- 172.21.15.101:0/543650072 shutdown_connections 2023-07-18T03:10:09.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.616+0000 7f387bbd2b80 1 --2- 172.21.15.101:0/543650072 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f17077cad0 0x55f170781cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.616+0000 7f387bbd2b80 1 --2- 172.21.15.101:0/543650072 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f1707712d0 0x55f170773af0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.616+0000 7f387bbd2b80 1 --2- 172.21.15.101:0/543650072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f170771be0 0x55f1707817b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.616+0000 7f387bbd2b80 1 -- 172.21.15.101:0/543650072 >> 172.21.15.101:0/543650072 conn(0x55f170767da0 msgr2=0x55f1707733e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:09.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.616+0000 7f387bbd2b80 1 -- 172.21.15.101:0/543650072 shutdown_connections 2023-07-18T03:10:09.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.617+0000 7f387bbd2b80 1 -- 172.21.15.101:0/543650072 wait complete. 2023-07-18T03:10:09.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.617+0000 7f387bbd2b80 1 -- start start 2023-07-18T03:10:09.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.617+0000 7f387bbd2b80 1 Processor -- start 2023-07-18T03:10:09.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.618+0000 7f387bbd2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1707712d0 0x55f1707d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.618+0000 7f387bbd2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f170771be0 0x55f1707d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.618+0000 7f387bbd2b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f17077cad0 0x55f1707d8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.618+0000 7f387bbd2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1707d9500 con 0x55f170771be0 2023-07-18T03:10:09.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.618+0000 7f387bbd2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1707d9640 con 0x55f1707712d0 2023-07-18T03:10:09.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.618+0000 7f387bbd2b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1707d9800 con 0x55f17077cad0 2023-07-18T03:10:09.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.618+0000 7f387bbd2b80 10 client.0 _ll_register_callbacks cb 0x7f385c009dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:10:09.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.618+0000 7f386679c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f17077cad0 0x55f1707d8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.618+0000 7f386579a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f170771be0 0x55f1707d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.618+0000 7f3865f9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1707712d0 0x55f1707d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.618+0000 7f3865f9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1707712d0 0x55f1707d4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:33040/0 (socket says 172.21.15.101:33040) 2023-07-18T03:10:09.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.618+0000 7f386579a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f170771be0 0x55f1707d46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:44812/0 (socket says 172.21.15.101:44812) 2023-07-18T03:10:09.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.618+0000 7f3865f9b700 1 -- 172.21.15.101:0/2130397898 learned_addr learned my addr 172.21.15.101:0/2130397898 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:09.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.618+0000 7f386679c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f17077cad0 0x55f1707d8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:50686/0 (socket says 172.21.15.101:50686) 2023-07-18T03:10:09.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.618+0000 7f386679c700 1 -- 172.21.15.101:0/2130397898 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1707712d0 msgr2=0x55f1707d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:09.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.618+0000 7f386679c700 1 --2- 172.21.15.101:0/2130397898 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1707712d0 0x55f1707d4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.618+0000 7f386679c700 1 -- 172.21.15.101:0/2130397898 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f170771be0 msgr2=0x55f1707d46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:09.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.618+0000 7f386679c700 1 --2- 172.21.15.101:0/2130397898 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f170771be0 0x55f1707d46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.619+0000 7f386679c700 1 -- 172.21.15.101:0/2130397898 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f1704dac60 con 0x55f17077cad0 2023-07-18T03:10:09.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.619+0000 7f386679c700 1 --2- 172.21.15.101:0/2130397898 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f17077cad0 0x55f1707d8fc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3858013e80 tx=0x7f385800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c1eb5f7380059ab server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:09.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.619+0000 7f38497fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:10:09.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.619+0000 7f38497fa700 1 -- 172.21.15.101:0/2130397898 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385800e040 con 0x55f17077cad0 2023-07-18T03:10:09.628 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.619+0000 7f387bbd2b80 1 -- 172.21.15.101:0/2130397898 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f385009aa30 con 0x55f17077cad0 2023-07-18T03:10:09.628 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.619+0000 7f38497fa700 1 -- 172.21.15.101:0/2130397898 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f385801ad70 con 0x55f17077cad0 2023-07-18T03:10:09.628 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.619+0000 7f38497fa700 1 -- 172.21.15.101:0/2130397898 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3858009020 con 0x55f17077cad0 2023-07-18T03:10:09.629 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.619+0000 7f387bbd2b80 10 client.26146 fetch_fsmap learned FSMap version 25 2023-07-18T03:10:09.629 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.619+0000 7f387bbd2b80 1 -- 172.21.15.101:0/2130397898 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f1704f02b0 con 0x55f17077cad0 2023-07-18T03:10:09.629 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.620+0000 7f38497fa700 1 -- 172.21.15.101:0/2130397898 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f385009aa30 con 0x55f17077cad0 2023-07-18T03:10:09.629 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.620+0000 7f38497fa700 1 -- 172.21.15.101:0/2130397898 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f3858025070 con 0x55f17077cad0 2023-07-18T03:10:09.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.620+0000 7f387bbd2b80 10 client.26146 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:10:09.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.620+0000 7f387bbd2b80 10 client.26146 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:10:09.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.620+0000 7f387bbd2b80 20 client.26146 populate_metadata read hostname 'smithi101' 2023-07-18T03:10:09.631 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.620+0000 7f387bbd2b80 1 --2- 172.21.15.101:0/2130397898 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f1707e1100 0x55f1707e3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:09.631 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.620+0000 7f387bbd2b80 4 client.26146 mds_command: new command op to 24562 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:10:09.631 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.620+0000 7f387bbd2b80 1 -- 172.21.15.101:0/2130397898 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f17067f050 con 0x55f1707e1100 2023-07-18T03:10:09.632 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.620+0000 7f3865f9b700 1 --2- 172.21.15.101:0/2130397898 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f1707e1100 0x55f1707e3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:09.632 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.621+0000 7f38497fa700 10 client.26146 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:10:09.632 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.621+0000 7f3865f9b700 1 --2- 172.21.15.101:0/2130397898 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f1707e1100 0x55f1707e3510 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67d45a912dbc13a9 server_cookie=833c4f95cc2a50a in_seq=0 out_seq=0 2023-07-18T03:10:09.632 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.621+0000 7f38497fa700 1 -- 172.21.15.101:0/2130397898 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f17067f050 con 0x55f1707e1100 2023-07-18T03:10:09.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.621+0000 7f38497fa700 10 client.26146 handle_command_reply: tid=0 2023-07-18T03:10:09.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.621+0000 7f387bbd2b80 1 client.26146 shutdown 2023-07-18T03:10:09.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.621+0000 7f387bbd2b80 1 -- 172.21.15.101:0/2130397898 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f17077cad0 msgr2=0x55f1707d8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:09.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.621+0000 7f387bbd2b80 1 --2- 172.21.15.101:0/2130397898 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f17077cad0 0x55f1707d8fc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3858013e80 tx=0x7f385800b040 comp rx=0 tx=0).stop 2023-07-18T03:10:09.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.621+0000 7f387bbd2b80 1 -- 172.21.15.101:0/2130397898 shutdown_connections 2023-07-18T03:10:09.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.621+0000 7f387bbd2b80 1 --2- 172.21.15.101:0/2130397898 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f1707e1100 0x55f1707e3510 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.621+0000 7f387bbd2b80 1 --2- 172.21.15.101:0/2130397898 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1707712d0 0x55f1707d4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.634 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.621+0000 7f387bbd2b80 1 --2- 172.21.15.101:0/2130397898 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f17077cad0 0x55f1707d8fc0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.634 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.621+0000 7f38497fa700 0 client.26146 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:10:09.634 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.621+0000 7f387bbd2b80 1 --2- 172.21.15.101:0/2130397898 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f170771be0 0x55f1707d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.634 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.621+0000 7f387bbd2b80 1 -- 172.21.15.101:0/2130397898 >> 172.21.15.101:0/2130397898 conn(0x55f170767da0 msgr2=0x55f170773c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:09.634 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.621+0000 7f387bbd2b80 1 -- 172.21.15.101:0/2130397898 shutdown_connections 2023-07-18T03:10:09.634 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.621+0000 7f387bbd2b80 1 -- 172.21.15.101:0/2130397898 wait complete. 2023-07-18T03:10:09.634 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.622+0000 7f387bbd2b80 20 client.26146 trim_cache size 0 max 16384 2023-07-18T03:10:09.635 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:10:09.636 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.622+0000 7f38681ff700 1 -- 172.21.15.101:0/977022578 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f384c0a5750 msgr2=0x7f384c0a7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:09.636 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.622+0000 7f38681ff700 1 --2- 172.21.15.101:0/977022578 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f384c0a5750 0x7f384c0a7c00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f386019c700 tx=0x7f3858009920 comp rx=0 tx=0).stop 2023-07-18T03:10:09.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.622+0000 7f38681ff700 1 -- 172.21.15.101:0/977022578 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3860100480 msgr2=0x7f386019bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:09.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.622+0000 7f38681ff700 1 --2- 172.21.15.101:0/977022578 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3860100480 0x7f386019bf40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3850005910 tx=0x7f3850005720 comp rx=0 tx=0).stop 2023-07-18T03:10:09.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.622+0000 7f38681ff700 1 -- 172.21.15.101:0/977022578 shutdown_connections 2023-07-18T03:10:09.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.622+0000 7f38681ff700 1 --2- 172.21.15.101:0/977022578 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f384c0a5750 0x7f384c0a7c00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.622+0000 7f38681ff700 1 --2- 172.21.15.101:0/977022578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3860105ae0 0x7f3860195a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.622+0000 7f38681ff700 1 --2- 172.21.15.101:0/977022578 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f38600ffad0 0x7f386019ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.638 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.622+0000 7f38681ff700 1 --2- 172.21.15.101:0/977022578 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3860100480 0x7f386019bf40 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:09.638 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.622+0000 7f38681ff700 1 -- 172.21.15.101:0/977022578 >> 172.21.15.101:0/977022578 conn(0x7f38600fb5f0 msgr2=0x7f38600fbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:09.638 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.622+0000 7f38681ff700 1 -- 172.21.15.101:0/977022578 shutdown_connections 2023-07-18T03:10:09.638 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:09.622+0000 7f38681ff700 1 -- 172.21.15.101:0/977022578 wait complete. 2023-07-18T03:10:10.568 INFO:tasks.workunit.client.0.smithi101.stdout: 1 137021 7.93 MB/sec warmup 95 sec latency 33.959 ms 2023-07-18T03:10:10.639 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-18T03:10:10.639 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-18T03:10:10.861 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.869+0000 7f17bc1ff700 1 -- 172.21.15.101:0/341354638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17b4105af0 msgr2=0x7f17b4105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:10.861 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.869+0000 7f17bc1ff700 1 --2- 172.21.15.101:0/341354638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17b4105af0 0x7f17b4105ec0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f17b0009990 tx=0x7f17b0009ca0 comp rx=0 tx=0).stop 2023-07-18T03:10:10.861 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.869+0000 7f17bc1ff700 1 -- 172.21.15.101:0/341354638 shutdown_connections 2023-07-18T03:10:10.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.869+0000 7f17bc1ff700 1 --2- 172.21.15.101:0/341354638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17b4105af0 0x7f17b4105ec0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:10.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.869+0000 7f17bc1ff700 1 --2- 172.21.15.101:0/341354638 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17b4100490 0x7f17b410c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:10.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.869+0000 7f17bc1ff700 1 --2- 172.21.15.101:0/341354638 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f17b40ffae0 0x7f17b40fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:10.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.869+0000 7f17bc1ff700 1 -- 172.21.15.101:0/341354638 >> 172.21.15.101:0/341354638 conn(0x7f17b40fb5e0 msgr2=0x7f17b40fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:10.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.870+0000 7f17bc1ff700 1 -- 172.21.15.101:0/341354638 shutdown_connections 2023-07-18T03:10:10.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.870+0000 7f17bc1ff700 1 -- 172.21.15.101:0/341354638 wait complete. 2023-07-18T03:10:10.864 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.870+0000 7f17bc1ff700 1 Processor -- start 2023-07-18T03:10:10.864 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.871+0000 7f17bc1ff700 1 -- start start 2023-07-18T03:10:10.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.871+0000 7f17bc1ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f17b40ffae0 0x7f17b4072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:10.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.871+0000 7f17bc1ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17b4100490 0x7f17b4072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:10.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.871+0000 7f17bc1ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17b4105af0 0x7f17b41a3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:10.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.871+0000 7f17bc1ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b4073580 con 0x7f17b40ffae0 2023-07-18T03:10:10.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.871+0000 7f17bc1ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b41a4320 con 0x7f17b4100490 2023-07-18T03:10:10.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.871+0000 7f17bc1ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b41a4460 con 0x7f17b4105af0 2023-07-18T03:10:10.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.871+0000 7f17b9f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f17b40ffae0 0x7f17b4072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:10.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.871+0000 7f17b979a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17b4100490 0x7f17b4072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:10.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.871+0000 7f17b979a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17b4100490 0x7f17b4072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:33060/0 (socket says 172.21.15.101:33060) 2023-07-18T03:10:10.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.871+0000 7f17b9f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f17b40ffae0 0x7f17b4072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:44828/0 (socket says 172.21.15.101:44828) 2023-07-18T03:10:10.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.871+0000 7f17b979a700 1 -- 172.21.15.101:0/1396475171 learned_addr learned my addr 172.21.15.101:0/1396475171 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:10.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.871+0000 7f17ba79c700 1 --2- 172.21.15.101:0/1396475171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17b4105af0 0x7f17b41a3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:10.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.872+0000 7f17b979a700 1 -- 172.21.15.101:0/1396475171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17b4105af0 msgr2=0x7f17b41a3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:10.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.872+0000 7f17b979a700 1 --2- 172.21.15.101:0/1396475171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17b4105af0 0x7f17b41a3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:10.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.872+0000 7f17b979a700 1 -- 172.21.15.101:0/1396475171 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f17b40ffae0 msgr2=0x7f17b4072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:10.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.872+0000 7f17b979a700 1 --2- 172.21.15.101:0/1396475171 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f17b40ffae0 0x7f17b4072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:10.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.872+0000 7f17b979a700 1 -- 172.21.15.101:0/1396475171 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b00095f0 con 0x7f17b4100490 2023-07-18T03:10:10.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.872+0000 7f17b9f9b700 1 --2- 172.21.15.101:0/1396475171 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f17b40ffae0 0x7f17b4072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:10:10.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.872+0000 7f17ba79c700 1 --2- 172.21.15.101:0/1396475171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17b4105af0 0x7f17b41a3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:10:10.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.872+0000 7f17b979a700 1 --2- 172.21.15.101:0/1396475171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17b4100490 0x7f17b4072d50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f17a400e9a0 tx=0x7f17a400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:10.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.872+0000 7f17aaffd700 1 -- 172.21.15.101:0/1396475171 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a4009790 con 0x7f17b4100490 2023-07-18T03:10:10.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.872+0000 7f17aaffd700 1 -- 172.21.15.101:0/1396475171 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17a4010430 con 0x7f17b4100490 2023-07-18T03:10:10.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.872+0000 7f17bc1ff700 1 -- 172.21.15.101:0/1396475171 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17b41a46c0 con 0x7f17b4100490 2023-07-18T03:10:10.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.872+0000 7f17aaffd700 1 -- 172.21.15.101:0/1396475171 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a4018680 con 0x7f17b4100490 2023-07-18T03:10:10.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.872+0000 7f17bc1ff700 1 -- 172.21.15.101:0/1396475171 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17b41a4ba0 con 0x7f17b4100490 2023-07-18T03:10:10.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.876+0000 7f17aaffd700 1 -- 172.21.15.101:0/1396475171 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f17a40105a0 con 0x7f17b4100490 2023-07-18T03:10:10.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.877+0000 7f17cfa3db80 1 Processor -- start 2023-07-18T03:10:10.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.877+0000 7f17cfa3db80 1 -- start start 2023-07-18T03:10:10.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.878+0000 7f17cfa3db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559014408350 0x559014408720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:10.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.878+0000 7f17cfa3db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559014408c60 0x559014413610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:10.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.878+0000 7f17cfa3db80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559014413b50 0x559014415f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:10.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.878+0000 7f17cfa3db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55901430ed30 con 0x559014408350 2023-07-18T03:10:10.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.878+0000 7f17cfa3db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559014361ac0 con 0x559014408c60 2023-07-18T03:10:10.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.878+0000 7f17cfa3db80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55901435af50 con 0x559014413b50 2023-07-18T03:10:10.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.878+0000 7f17b9f9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559014408c60 0x559014413610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:10.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.878+0000 7f17b979a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559014413b50 0x559014415f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:10.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.878+0000 7f17b9f9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559014408c60 0x559014413610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:33074/0 (socket says 172.21.15.101:33074) 2023-07-18T03:10:10.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.878+0000 7f17b9f9b700 1 -- 172.21.15.101:0/973693309 learned_addr learned my addr 172.21.15.101:0/973693309 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:10.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.878+0000 7f17aaffd700 1 --2- 172.21.15.101:0/1396475171 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f17a00a12c0 0x7f17a00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:10.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.878+0000 7f17ba79c700 1 --2- 172.21.15.101:0/973693309 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559014408350 0x559014408720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:10.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.878+0000 7f17b9f9b700 1 -- 172.21.15.101:0/973693309 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559014413b50 msgr2=0x559014415f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:10.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.878+0000 7f17b9f9b700 1 --2- 172.21.15.101:0/973693309 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559014413b50 0x559014415f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:10.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.878+0000 7f17b9f9b700 1 -- 172.21.15.101:0/973693309 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559014408350 msgr2=0x559014408720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:10.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.878+0000 7f17b9f9b700 1 --2- 172.21.15.101:0/973693309 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559014408350 0x559014408720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:10.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.879+0000 7f17b9f9b700 1 -- 172.21.15.101:0/973693309 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559014188e60 con 0x559014408c60 2023-07-18T03:10:10.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.878+0000 7f17aaffd700 1 -- 172.21.15.101:0/1396475171 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f17a4014070 con 0x7f17b4100490 2023-07-18T03:10:10.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.879+0000 7f17ba79c700 1 --2- 172.21.15.101:0/1396475171 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f17a00a12c0 0x7f17a00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:10.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.879+0000 7f17b9f9b700 1 --2- 172.21.15.101:0/973693309 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559014408c60 0x559014413610 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f17b0005ee0 tx=0x7f17b0005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67618db5cdb88940 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:10.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.879+0000 7f179ffff700 1 -- 172.21.15.101:0/973693309 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b001d070 con 0x559014408c60 2023-07-18T03:10:10.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.879+0000 7f179ffff700 1 -- 172.21.15.101:0/973693309 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17b001e040 con 0x559014408c60 2023-07-18T03:10:10.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.879+0000 7f179ffff700 1 -- 172.21.15.101:0/973693309 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b0004dc0 con 0x559014408c60 2023-07-18T03:10:10.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.879+0000 7f17ba79c700 1 --2- 172.21.15.101:0/1396475171 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f17a00a12c0 0x7f17a00a3770 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f17ac009fd0 tx=0x7f17ac009040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:10.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.880+0000 7f17cfa3db80 1 -- 172.21.15.101:0/973693309 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559014408c60 msgr2=0x559014413610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:10.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.880+0000 7f17cfa3db80 1 --2- 172.21.15.101:0/973693309 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559014408c60 0x559014413610 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f17b0005ee0 tx=0x7f17b0005f10 comp rx=0 tx=0).stop 2023-07-18T03:10:10.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.880+0000 7f17cfa3db80 1 -- 172.21.15.101:0/973693309 shutdown_connections 2023-07-18T03:10:10.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.880+0000 7f17cfa3db80 1 --2- 172.21.15.101:0/973693309 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559014413b50 0x559014415f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:10.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.880+0000 7f17cfa3db80 1 --2- 172.21.15.101:0/973693309 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559014408350 0x559014408720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:10.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.880+0000 7f17cfa3db80 1 --2- 172.21.15.101:0/973693309 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559014408c60 0x559014413610 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:10.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.880+0000 7f17cfa3db80 1 -- 172.21.15.101:0/973693309 >> 172.21.15.101:0/973693309 conn(0x5590143fee20 msgr2=0x559014404720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:10.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.880+0000 7f17cfa3db80 1 -- 172.21.15.101:0/973693309 shutdown_connections 2023-07-18T03:10:10.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.880+0000 7f17cfa3db80 1 -- 172.21.15.101:0/973693309 wait complete. 2023-07-18T03:10:10.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.881+0000 7f17cfa3db80 1 -- start start 2023-07-18T03:10:10.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.881+0000 7f17cfa3db80 1 Processor -- start 2023-07-18T03:10:10.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.881+0000 7f17cfa3db80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559014408350 0x5590144135d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:10.882 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.881+0000 7f17cfa3db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559014408c60 0x559014410850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:10.882 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.881+0000 7f17cfa3db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559014413b50 0x559014410f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:10.882 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.881+0000 7f17cfa3db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590142fab40 con 0x559014413b50 2023-07-18T03:10:10.882 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.881+0000 7f17cfa3db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590142fa840 con 0x559014408c60 2023-07-18T03:10:10.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.881+0000 7f17cfa3db80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590141de9a0 con 0x559014408350 2023-07-18T03:10:10.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.881+0000 7f17cfa3db80 10 client.0 _ll_register_callbacks cb 0x5590142f87e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:10:10.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.882+0000 7f17b9f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559014408350 0x5590144135d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:10.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.882+0000 7f17ba79c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559014413b50 0x559014410f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:10.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.882+0000 7f17b9f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559014408350 0x5590144135d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:50718/0 (socket says 172.21.15.101:50718) 2023-07-18T03:10:10.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.882+0000 7f17b9f9b700 1 -- 172.21.15.101:0/2430979879 learned_addr learned my addr 172.21.15.101:0/2430979879 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:10.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.882+0000 7f17b979a700 1 --2- 172.21.15.101:0/2430979879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559014408c60 0x559014410850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:10.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.882+0000 7f17b9f9b700 1 -- 172.21.15.101:0/2430979879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559014408c60 msgr2=0x559014410850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:10.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.882+0000 7f17b9f9b700 1 --2- 172.21.15.101:0/2430979879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559014408c60 0x559014410850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:10.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.882+0000 7f17b9f9b700 1 -- 172.21.15.101:0/2430979879 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559014413b50 msgr2=0x559014410f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:10.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.882+0000 7f17b9f9b700 1 --2- 172.21.15.101:0/2430979879 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559014413b50 0x559014410f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:10.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.882+0000 7f17b9f9b700 1 -- 172.21.15.101:0/2430979879 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17b410d220 con 0x559014408350 2023-07-18T03:10:10.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.882+0000 7f17b979a700 1 --2- 172.21.15.101:0/2430979879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559014408c60 0x559014410850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:10:10.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.882+0000 7f17b9f9b700 1 --2- 172.21.15.101:0/2430979879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559014408350 0x5590144135d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f17b00049e0 tx=0x7f17b0022b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfd847ffe1604619 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:10.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.882+0000 7f179d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:10:10.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.882+0000 7f179d7fa700 1 -- 172.21.15.101:0/2430979879 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b001d030 con 0x559014408350 2023-07-18T03:10:10.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.883+0000 7f179d7fa700 1 -- 172.21.15.101:0/2430979879 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f17b000f660 con 0x559014408350 2023-07-18T03:10:10.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.883+0000 7f17cfa3db80 1 -- 172.21.15.101:0/2430979879 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5590142b4570 con 0x559014408350 2023-07-18T03:10:10.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.883+0000 7f179d7fa700 1 -- 172.21.15.101:0/2430979879 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17b0004760 con 0x559014408350 2023-07-18T03:10:10.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.883+0000 7f17cfa3db80 10 client.26158 fetch_fsmap learned FSMap version 25 2023-07-18T03:10:10.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.883+0000 7f17cfa3db80 1 -- 172.21.15.101:0/2430979879 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559014171ce0 con 0x559014408350 2023-07-18T03:10:10.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.883+0000 7f179d7fa700 1 -- 172.21.15.101:0/2430979879 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5590142b4570 con 0x559014408350 2023-07-18T03:10:10.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.883+0000 7f179d7fa700 1 -- 172.21.15.101:0/2430979879 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f17b002c520 con 0x559014408350 2023-07-18T03:10:10.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.883+0000 7f17cfa3db80 10 client.26158 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:10:10.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.884+0000 7f17cfa3db80 10 client.26158 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:10:10.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.884+0000 7f17cfa3db80 20 client.26158 populate_metadata read hostname 'smithi101' 2023-07-18T03:10:10.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.884+0000 7f17cfa3db80 1 --2- 172.21.15.101:0/2430979879 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x559014422f90 0x5590144253a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:10.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.884+0000 7f17cfa3db80 4 client.26158 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:10:10.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.884+0000 7f17cfa3db80 1 -- 172.21.15.101:0/2430979879 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559014353f30 con 0x559014422f90 2023-07-18T03:10:10.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.884+0000 7f17b979a700 1 --2- 172.21.15.101:0/2430979879 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x559014422f90 0x5590144253a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:10.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.884+0000 7f17b979a700 1 --2- 172.21.15.101:0/2430979879 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x559014422f90 0x5590144253a0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db1effafb3d06a21 server_cookie=6a38916826c3f543 in_seq=0 out_seq=0 2023-07-18T03:10:10.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.884+0000 7f179d7fa700 10 client.26158 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:10:10.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.885+0000 7f179d7fa700 1 -- 172.21.15.101:0/2430979879 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559014353f30 con 0x559014422f90 2023-07-18T03:10:10.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.885+0000 7f179d7fa700 10 client.26158 handle_command_reply: tid=0 2023-07-18T03:10:10.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.885+0000 7f17cfa3db80 1 client.26158 shutdown 2023-07-18T03:10:10.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.885+0000 7f17cfa3db80 1 -- 172.21.15.101:0/2430979879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559014408350 msgr2=0x5590144135d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:10.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.886+0000 7f17cfa3db80 1 --2- 172.21.15.101:0/2430979879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559014408350 0x5590144135d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f17b00049e0 tx=0x7f17b0022b50 comp rx=0 tx=0).stop 2023-07-18T03:10:10.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.886+0000 7f17cfa3db80 1 -- 172.21.15.101:0/2430979879 shutdown_connections 2023-07-18T03:10:10.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.886+0000 7f17cfa3db80 1 --2- 172.21.15.101:0/2430979879 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x559014422f90 0x5590144253a0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:10.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.886+0000 7f17cfa3db80 1 --2- 172.21.15.101:0/2430979879 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559014413b50 0x559014410f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:10.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.886+0000 7f17cfa3db80 1 --2- 172.21.15.101:0/2430979879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559014408350 0x5590144135d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:10.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.886+0000 7f17cfa3db80 1 --2- 172.21.15.101:0/2430979879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559014408c60 0x559014410850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:10.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.886+0000 7f179d7fa700 0 client.26158 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:10:10.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.886+0000 7f17cfa3db80 1 -- 172.21.15.101:0/2430979879 >> 172.21.15.101:0/2430979879 conn(0x5590143fee20 msgr2=0x5590144042b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:10.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.886+0000 7f17cfa3db80 1 -- 172.21.15.101:0/2430979879 shutdown_connections 2023-07-18T03:10:10.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.886+0000 7f17cfa3db80 1 -- 172.21.15.101:0/2430979879 wait complete. 2023-07-18T03:10:10.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.886+0000 7f17cfa3db80 20 client.26158 trim_cache size 0 max 16384 2023-07-18T03:10:10.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.904+0000 7f17cfa3db80 1 Processor -- start 2023-07-18T03:10:10.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.904+0000 7f17cfa3db80 1 -- start start 2023-07-18T03:10:10.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.904+0000 7f17cfa3db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559014408350 0x5590144048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:10.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.905+0000 7f17cfa3db80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559014408c60 0x55901440b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:10.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.905+0000 7f17cfa3db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559014413b50 0x55901441a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:10.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.905+0000 7f17cfa3db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559014409600 con 0x559014413b50 2023-07-18T03:10:10.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.905+0000 7f17cfa3db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559014411e30 con 0x559014408350 2023-07-18T03:10:10.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.905+0000 7f17cfa3db80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590143fea20 con 0x559014408c60 2023-07-18T03:10:10.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.905+0000 7f17b979a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559014408c60 0x55901440b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:10.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.905+0000 7f17b979a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559014408c60 0x55901440b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:50730/0 (socket says 172.21.15.101:50730) 2023-07-18T03:10:10.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.905+0000 7f17b979a700 1 -- 172.21.15.101:0/831845093 learned_addr learned my addr 172.21.15.101:0/831845093 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:10.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.905+0000 7f17ba79c700 1 --2- 172.21.15.101:0/831845093 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559014413b50 0x55901441a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:10.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.905+0000 7f17b9f9b700 1 --2- 172.21.15.101:0/831845093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559014408350 0x5590144048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:10.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.905+0000 7f17b979a700 1 -- 172.21.15.101:0/831845093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559014408350 msgr2=0x5590144048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:10.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.905+0000 7f17b979a700 1 --2- 172.21.15.101:0/831845093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559014408350 0x5590144048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:10.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.905+0000 7f17b979a700 1 -- 172.21.15.101:0/831845093 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559014413b50 msgr2=0x55901441a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:10.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.905+0000 7f17b979a700 1 --2- 172.21.15.101:0/831845093 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559014413b50 0x55901441a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:10.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.905+0000 7f17b979a700 1 -- 172.21.15.101:0/831845093 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590141af2d0 con 0x559014408c60 2023-07-18T03:10:10.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.905+0000 7f17b979a700 1 --2- 172.21.15.101:0/831845093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559014408c60 0x55901440b230 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f17a4096a20 tx=0x7f17a409d040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f68fcf04dca9e14 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:10.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.906+0000 7f17a8ff9700 1 -- 172.21.15.101:0/831845093 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a409dfe0 con 0x559014408c60 2023-07-18T03:10:10.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.906+0000 7f17a8ff9700 1 -- 172.21.15.101:0/831845093 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17a40a2400 con 0x559014408c60 2023-07-18T03:10:10.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.906+0000 7f17a8ff9700 1 -- 172.21.15.101:0/831845093 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a40cf310 con 0x559014408c60 2023-07-18T03:10:10.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.906+0000 7f17cfa3db80 1 -- 172.21.15.101:0/831845093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559014408c60 msgr2=0x55901440b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:10.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.906+0000 7f17cfa3db80 1 --2- 172.21.15.101:0/831845093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559014408c60 0x55901440b230 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f17a4096a20 tx=0x7f17a409d040 comp rx=0 tx=0).stop 2023-07-18T03:10:10.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.906+0000 7f17cfa3db80 1 -- 172.21.15.101:0/831845093 shutdown_connections 2023-07-18T03:10:10.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.906+0000 7f17cfa3db80 1 --2- 172.21.15.101:0/831845093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559014408350 0x5590144048d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:10.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.906+0000 7f17cfa3db80 1 --2- 172.21.15.101:0/831845093 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559014413b50 0x55901441a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:10.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.906+0000 7f17cfa3db80 1 --2- 172.21.15.101:0/831845093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559014408c60 0x55901440b230 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:10.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.906+0000 7f17cfa3db80 1 -- 172.21.15.101:0/831845093 >> 172.21.15.101:0/831845093 conn(0x5590143fee20 msgr2=0x5590144041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:10.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.907+0000 7f17cfa3db80 1 -- 172.21.15.101:0/831845093 shutdown_connections 2023-07-18T03:10:10.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.907+0000 7f17cfa3db80 1 -- 172.21.15.101:0/831845093 wait complete. 2023-07-18T03:10:10.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.907+0000 7f17cfa3db80 1 -- start start 2023-07-18T03:10:10.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.907+0000 7f17cfa3db80 1 Processor -- start 2023-07-18T03:10:10.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.908+0000 7f17cfa3db80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559014408350 0x559014471430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:10.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.908+0000 7f17cfa3db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559014408c60 0x559014471970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:10.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.908+0000 7f17cfa3db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559014413b50 0x559014419670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:10.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.908+0000 7f17cfa3db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559014419c50 con 0x559014408c60 2023-07-18T03:10:10.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.908+0000 7f17cfa3db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559014419dc0 con 0x559014413b50 2023-07-18T03:10:10.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.908+0000 7f17cfa3db80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x559014419f80 con 0x559014408350 2023-07-18T03:10:10.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.908+0000 7f17cfa3db80 10 client.0 _ll_register_callbacks cb 0x7f17b00227e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:10:10.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.908+0000 7f17b9f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559014408350 0x559014471430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:10.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.908+0000 7f17b9f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559014408350 0x559014471430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:50744/0 (socket says 172.21.15.101:50744) 2023-07-18T03:10:10.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.908+0000 7f17b9f9b700 1 -- 172.21.15.101:0/2150515339 learned_addr learned my addr 172.21.15.101:0/2150515339 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:10.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.908+0000 7f17b979a700 1 --2- 172.21.15.101:0/2150515339 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559014408c60 0x559014471970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:10.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.908+0000 7f17ba79c700 1 --2- 172.21.15.101:0/2150515339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559014413b50 0x559014419670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:10.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.908+0000 7f17b9f9b700 1 -- 172.21.15.101:0/2150515339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559014413b50 msgr2=0x559014419670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:10.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.908+0000 7f17b9f9b700 1 --2- 172.21.15.101:0/2150515339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559014413b50 0x559014419670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:10.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.908+0000 7f17b9f9b700 1 -- 172.21.15.101:0/2150515339 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559014408c60 msgr2=0x559014471970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:10.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.908+0000 7f17b9f9b700 1 --2- 172.21.15.101:0/2150515339 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559014408c60 0x559014471970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:10.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.908+0000 7f17b9f9b700 1 -- 172.21.15.101:0/2150515339 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17b41a4ba0 con 0x559014408350 2023-07-18T03:10:10.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.908+0000 7f17ba79c700 1 --2- 172.21.15.101:0/2150515339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559014413b50 0x559014419670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:10:10.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.910+0000 7f17b9f9b700 1 --2- 172.21.15.101:0/2150515339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559014408350 0x559014471430 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f17b0006010 tx=0x7f17b002e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ada15bedc2b1da71 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:10.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.910+0000 7f179d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:10:10.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.910+0000 7f179d7fa700 1 -- 172.21.15.101:0/2150515339 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b0045070 con 0x559014408350 2023-07-18T03:10:10.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.910+0000 7f17cfa3db80 1 -- 172.21.15.101:0/2150515339 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f17b001e040 con 0x559014408350 2023-07-18T03:10:10.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.910+0000 7f179d7fa700 1 -- 172.21.15.101:0/2150515339 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f17b0000e40 con 0x559014408350 2023-07-18T03:10:10.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.910+0000 7f179d7fa700 1 -- 172.21.15.101:0/2150515339 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17b001d040 con 0x559014408350 2023-07-18T03:10:10.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.910+0000 7f17cfa3db80 10 client.26164 fetch_fsmap learned FSMap version 25 2023-07-18T03:10:10.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.910+0000 7f17cfa3db80 1 -- 172.21.15.101:0/2150515339 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559014187330 con 0x559014408350 2023-07-18T03:10:10.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.910+0000 7f179d7fa700 1 -- 172.21.15.101:0/2150515339 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17b001e040 con 0x559014408350 2023-07-18T03:10:10.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.910+0000 7f179d7fa700 1 -- 172.21.15.101:0/2150515339 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f17b0048030 con 0x559014408350 2023-07-18T03:10:10.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.910+0000 7f17cfa3db80 10 client.26164 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:10:10.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.910+0000 7f17cfa3db80 10 client.26164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:10:10.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.910+0000 7f17cfa3db80 20 client.26164 populate_metadata read hostname 'smithi101' 2023-07-18T03:10:10.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.910+0000 7f17cfa3db80 1 --2- 172.21.15.101:0/2150515339 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x559014478e40 0x55901447b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:10.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.910+0000 7f17cfa3db80 4 client.26164 mds_command: new command op to 24562 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-18T03:10:10.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.910+0000 7f17cfa3db80 1 -- 172.21.15.101:0/2150515339 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5590143160d0 con 0x559014478e40 2023-07-18T03:10:10.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.910+0000 7f17b979a700 1 --2- 172.21.15.101:0/2150515339 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x559014478e40 0x55901447b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:10.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.910+0000 7f17b979a700 1 --2- 172.21.15.101:0/2150515339 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x559014478e40 0x55901447b220 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=720ce756121d9e17 server_cookie=7f0b2aa3700dc852 in_seq=0 out_seq=0 2023-07-18T03:10:10.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.910+0000 7f179d7fa700 10 client.26164 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:10:10.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.911+0000 7f179d7fa700 1 -- 172.21.15.101:0/2150515339 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5590143160d0 con 0x559014478e40 2023-07-18T03:10:10.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.911+0000 7f179d7fa700 10 client.26164 handle_command_reply: tid=0 2023-07-18T03:10:10.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.911+0000 7f17cfa3db80 1 client.26164 shutdown 2023-07-18T03:10:10.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.911+0000 7f17cfa3db80 1 -- 172.21.15.101:0/2150515339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559014408350 msgr2=0x559014471430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:10.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.911+0000 7f17cfa3db80 1 --2- 172.21.15.101:0/2150515339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559014408350 0x559014471430 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f17b0006010 tx=0x7f17b002e000 comp rx=0 tx=0).stop 2023-07-18T03:10:10.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.911+0000 7f17cfa3db80 1 -- 172.21.15.101:0/2150515339 shutdown_connections 2023-07-18T03:10:10.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.911+0000 7f17cfa3db80 1 --2- 172.21.15.101:0/2150515339 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x559014478e40 0x55901447b220 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:10.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.911+0000 7f17cfa3db80 1 --2- 172.21.15.101:0/2150515339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559014413b50 0x559014419670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:10.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.911+0000 7f17cfa3db80 1 --2- 172.21.15.101:0/2150515339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559014408350 0x559014471430 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:10.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.912+0000 7f179d7fa700 0 client.26164 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:10:10.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.912+0000 7f17cfa3db80 1 --2- 172.21.15.101:0/2150515339 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559014408c60 0x559014471970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:10.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.912+0000 7f17cfa3db80 1 -- 172.21.15.101:0/2150515339 >> 172.21.15.101:0/2150515339 conn(0x5590143fee20 msgr2=0x55901440a220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:10.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.912+0000 7f17cfa3db80 1 -- 172.21.15.101:0/2150515339 shutdown_connections 2023-07-18T03:10:10.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.912+0000 7f17cfa3db80 1 -- 172.21.15.101:0/2150515339 wait complete. 2023-07-18T03:10:10.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.912+0000 7f17cfa3db80 20 client.26164 trim_cache size 0 max 16384 2023-07-18T03:10:10.912 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:10:10.912 INFO:teuthology.orchestra.run.smithi101.stdout: "return_code": 0, 2023-07-18T03:10:10.912 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_tag": "124dbb6f-f1a4-44fe-a8d7-4ce3d960f1ea", 2023-07-18T03:10:10.912 INFO:teuthology.orchestra.run.smithi101.stdout: "mode": "asynchronous" 2023-07-18T03:10:10.912 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:10:10.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.912+0000 7f17bc1ff700 1 -- 172.21.15.101:0/1396475171 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f17a00a12c0 msgr2=0x7f17a00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:10.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.912+0000 7f17bc1ff700 1 --2- 172.21.15.101:0/1396475171 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f17a00a12c0 0x7f17a00a3770 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f17ac009fd0 tx=0x7f17ac009040 comp rx=0 tx=0).stop 2023-07-18T03:10:10.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.912+0000 7f17bc1ff700 1 -- 172.21.15.101:0/1396475171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17b4100490 msgr2=0x7f17b4072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:10.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.912+0000 7f17bc1ff700 1 --2- 172.21.15.101:0/1396475171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17b4100490 0x7f17b4072d50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f17a400e9a0 tx=0x7f17a400ecb0 comp rx=0 tx=0).stop 2023-07-18T03:10:10.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.912+0000 7f17bc1ff700 1 -- 172.21.15.101:0/1396475171 shutdown_connections 2023-07-18T03:10:10.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.912+0000 7f17bc1ff700 1 --2- 172.21.15.101:0/1396475171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17b4105af0 0x7f17b41a3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:10.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.912+0000 7f17bc1ff700 1 --2- 172.21.15.101:0/1396475171 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f17b40ffae0 0x7f17b4072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:10.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.912+0000 7f17bc1ff700 1 --2- 172.21.15.101:0/1396475171 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f17a00a12c0 0x7f17a00a3770 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:10.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.912+0000 7f17bc1ff700 1 --2- 172.21.15.101:0/1396475171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17b4100490 0x7f17b4072d50 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:10.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.912+0000 7f17bc1ff700 1 -- 172.21.15.101:0/1396475171 >> 172.21.15.101:0/1396475171 conn(0x7f17b40fb5e0 msgr2=0x7f17b40fbc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:10.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.913+0000 7f17bc1ff700 1 -- 172.21.15.101:0/1396475171 shutdown_connections 2023-07-18T03:10:10.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:10.913+0000 7f17bc1ff700 1 -- 172.21.15.101:0/1396475171 wait complete. 2023-07-18T03:10:10.922 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:10:11.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.169+0000 7f15f37ff700 1 -- 172.21.15.101:0/2327329091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15ec0fe690 msgr2=0x7f15ec102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:11.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.169+0000 7f15f37ff700 1 --2- 172.21.15.101:0/2327329091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15ec0fe690 0x7f15ec102900 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f15dc009990 tx=0x7f15dc009ca0 comp rx=0 tx=0).stop 2023-07-18T03:10:11.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.170+0000 7f15f37ff700 1 -- 172.21.15.101:0/2327329091 shutdown_connections 2023-07-18T03:10:11.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.170+0000 7f15f37ff700 1 --2- 172.21.15.101:0/2327329091 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f15ec102e40 0x7f15ec1032e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:11.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.170+0000 7f15f37ff700 1 --2- 172.21.15.101:0/2327329091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15ec0fdd80 0x7f15ec0fe150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:11.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.170+0000 7f15f37ff700 1 --2- 172.21.15.101:0/2327329091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15ec0fe690 0x7f15ec102900 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:11.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.170+0000 7f15f37ff700 1 -- 172.21.15.101:0/2327329091 >> 172.21.15.101:0/2327329091 conn(0x7f15ec075b70 msgr2=0x7f15ec075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:11.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.170+0000 7f15f37ff700 1 -- 172.21.15.101:0/2327329091 shutdown_connections 2023-07-18T03:10:11.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.171+0000 7f15f37ff700 1 -- 172.21.15.101:0/2327329091 wait complete. 2023-07-18T03:10:11.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.173+0000 7f15f37ff700 1 Processor -- start 2023-07-18T03:10:11.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.173+0000 7f15f37ff700 1 -- start start 2023-07-18T03:10:11.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.175+0000 7f15f37ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15ec0fdd80 0x7f15ec10dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:11.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.175+0000 7f15f37ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f15ec0fe690 0x7f15ec10e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:11.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.175+0000 7f15f37ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15ec102e40 0x7f15ec10ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:11.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.175+0000 7f15f37ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ec070840 con 0x7f15ec0fe690 2023-07-18T03:10:11.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.175+0000 7f15f37ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ec0709b0 con 0x7f15ec102e40 2023-07-18T03:10:11.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.175+0000 7f15f37ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ec070b70 con 0x7f15ec0fdd80 2023-07-18T03:10:11.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.175+0000 7f15f0d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f15ec0fe690 0x7f15ec10e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:11.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.175+0000 7f15f0d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f15ec0fe690 0x7f15ec10e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:44872/0 (socket says 172.21.15.101:44872) 2023-07-18T03:10:11.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.175+0000 7f15f0d9a700 1 -- 172.21.15.101:0/842055924 learned_addr learned my addr 172.21.15.101:0/842055924 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:11.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.176+0000 7f15f159b700 1 --2- 172.21.15.101:0/842055924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15ec0fdd80 0x7f15ec10dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:11.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.176+0000 7f15f1d9c700 1 --2- 172.21.15.101:0/842055924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15ec102e40 0x7f15ec10ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:11.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.176+0000 7f15f0d9a700 1 -- 172.21.15.101:0/842055924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15ec0fdd80 msgr2=0x7f15ec10dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:11.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.176+0000 7f15f0d9a700 1 --2- 172.21.15.101:0/842055924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15ec0fdd80 0x7f15ec10dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:11.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.176+0000 7f15f0d9a700 1 -- 172.21.15.101:0/842055924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15ec102e40 msgr2=0x7f15ec10ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:11.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.176+0000 7f15f0d9a700 1 --2- 172.21.15.101:0/842055924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15ec102e40 0x7f15ec10ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:11.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.176+0000 7f15f0d9a700 1 -- 172.21.15.101:0/842055924 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15dc0095f0 con 0x7f15ec0fe690 2023-07-18T03:10:11.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.176+0000 7f15f159b700 1 --2- 172.21.15.101:0/842055924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15ec0fdd80 0x7f15ec10dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:10:11.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.177+0000 7f15f0d9a700 1 --2- 172.21.15.101:0/842055924 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f15ec0fe690 0x7f15ec10e4e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f15dc005910 tx=0x7f15dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:11.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.177+0000 7f15e27fc700 1 -- 172.21.15.101:0/842055924 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15dc01d070 con 0x7f15ec0fe690 2023-07-18T03:10:11.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.178+0000 7f15e27fc700 1 -- 172.21.15.101:0/842055924 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15dc022470 con 0x7f15ec0fe690 2023-07-18T03:10:11.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.178+0000 7f15e27fc700 1 -- 172.21.15.101:0/842055924 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15dc017650 con 0x7f15ec0fe690 2023-07-18T03:10:11.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.178+0000 7f15f37ff700 1 -- 172.21.15.101:0/842055924 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15ec070df0 con 0x7f15ec0fe690 2023-07-18T03:10:11.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.178+0000 7f15f37ff700 1 -- 172.21.15.101:0/842055924 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15ec071250 con 0x7f15ec0fe690 2023-07-18T03:10:11.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.182+0000 7f15e27fc700 1 -- 172.21.15.101:0/842055924 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f15dc00fad0 con 0x7f15ec0fe690 2023-07-18T03:10:11.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.183+0000 7f15e27fc700 1 --2- 172.21.15.101:0/842055924 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f15d80a1340 0x7f15d80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:11.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.183+0000 7f160709ab80 1 Processor -- start 2023-07-18T03:10:11.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.184+0000 7f15e27fc700 1 -- 172.21.15.101:0/842055924 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f15dc0ce4b0 con 0x7f15ec0fe690 2023-07-18T03:10:11.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.184+0000 7f160709ab80 1 -- start start 2023-07-18T03:10:11.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.184+0000 7f15f159b700 1 --2- 172.21.15.101:0/842055924 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f15d80a1340 0x7f15d80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:11.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.184+0000 7f160709ab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e73b93b2d0 0x55e73b93b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:11.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.184+0000 7f160709ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e73b93bbe0 0x55e73b946590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:11.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.184+0000 7f160709ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e73b946ad0 0x55e73b948eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:11.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.184+0000 7f160709ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e73b841cb0 con 0x55e73b93bbe0 2023-07-18T03:10:11.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.184+0000 7f160709ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e73b894a40 con 0x55e73b946ad0 2023-07-18T03:10:11.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.184+0000 7f160709ab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e73b88ded0 con 0x55e73b93b2d0 2023-07-18T03:10:11.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.184+0000 7f15f159b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e73b93b2d0 0x55e73b93b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:11.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.184+0000 7f15f159b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e73b93b2d0 0x55e73b93b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:50770/0 (socket says 172.21.15.101:50770) 2023-07-18T03:10:11.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.184+0000 7f15f159b700 1 -- 172.21.15.101:0/3509298835 learned_addr learned my addr 172.21.15.101:0/3509298835 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:11.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.185+0000 7f15f0d9a700 1 --2- 172.21.15.101:0/3509298835 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e73b93bbe0 0x55e73b946590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:11.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.185+0000 7f15f1d9c700 1 --2- 172.21.15.101:0/3509298835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e73b946ad0 0x55e73b948eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:11.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.185+0000 7f15f159b700 1 --2- 172.21.15.101:0/842055924 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f15d80a1340 0x7f15d80a37f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f15e8005950 tx=0x7f15e800b410 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:11.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.185+0000 7f15f159b700 1 -- 172.21.15.101:0/3509298835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e73b946ad0 msgr2=0x55e73b948eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:11.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.185+0000 7f15f159b700 1 --2- 172.21.15.101:0/3509298835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e73b946ad0 0x55e73b948eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:11.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.185+0000 7f15f159b700 1 -- 172.21.15.101:0/3509298835 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e73b93bbe0 msgr2=0x55e73b946590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:11.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.185+0000 7f15f159b700 1 --2- 172.21.15.101:0/3509298835 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e73b93bbe0 0x55e73b946590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:11.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.185+0000 7f15f159b700 1 -- 172.21.15.101:0/3509298835 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e73b6bbde0 con 0x55e73b93b2d0 2023-07-18T03:10:11.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.185+0000 7f15f159b700 1 --2- 172.21.15.101:0/3509298835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e73b93b2d0 0x55e73b93b6a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f15e80116b0 tx=0x7f15e801b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=429846f52c552cba server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:11.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.185+0000 7f15d77fe700 1 -- 172.21.15.101:0/3509298835 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e8026670 con 0x55e73b93b2d0 2023-07-18T03:10:11.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.185+0000 7f15d77fe700 1 -- 172.21.15.101:0/3509298835 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15e8011e50 con 0x55e73b93b2d0 2023-07-18T03:10:11.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.185+0000 7f15d77fe700 1 -- 172.21.15.101:0/3509298835 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e801db40 con 0x55e73b93b2d0 2023-07-18T03:10:11.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.186+0000 7f160709ab80 1 -- 172.21.15.101:0/3509298835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e73b93b2d0 msgr2=0x55e73b93b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:11.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.186+0000 7f160709ab80 1 --2- 172.21.15.101:0/3509298835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e73b93b2d0 0x55e73b93b6a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f15e80116b0 tx=0x7f15e801b000 comp rx=0 tx=0).stop 2023-07-18T03:10:11.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.186+0000 7f160709ab80 1 -- 172.21.15.101:0/3509298835 shutdown_connections 2023-07-18T03:10:11.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.186+0000 7f160709ab80 1 --2- 172.21.15.101:0/3509298835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e73b946ad0 0x55e73b948eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:11.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.186+0000 7f160709ab80 1 --2- 172.21.15.101:0/3509298835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e73b93b2d0 0x55e73b93b6a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:11.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.186+0000 7f160709ab80 1 --2- 172.21.15.101:0/3509298835 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e73b93bbe0 0x55e73b946590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:11.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.186+0000 7f160709ab80 1 -- 172.21.15.101:0/3509298835 >> 172.21.15.101:0/3509298835 conn(0x55e73b931da0 msgr2=0x55e73b9376a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:11.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.186+0000 7f160709ab80 1 -- 172.21.15.101:0/3509298835 shutdown_connections 2023-07-18T03:10:11.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.186+0000 7f160709ab80 1 -- 172.21.15.101:0/3509298835 wait complete. 2023-07-18T03:10:11.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.187+0000 7f160709ab80 1 -- start start 2023-07-18T03:10:11.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.187+0000 7f160709ab80 1 Processor -- start 2023-07-18T03:10:11.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.188+0000 7f160709ab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e73b93b2d0 0x55e73b9438a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:11.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.188+0000 7f160709ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e73b93bbe0 0x55e73b943de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:11.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.188+0000 7f160709ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e73b946ad0 0x55e73b944320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:11.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.188+0000 7f160709ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e73b82dac0 con 0x55e73b946ad0 2023-07-18T03:10:11.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.188+0000 7f160709ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e73b82d7c0 con 0x55e73b93bbe0 2023-07-18T03:10:11.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.188+0000 7f160709ab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e73b711920 con 0x55e73b93b2d0 2023-07-18T03:10:11.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.188+0000 7f160709ab80 10 client.0 _ll_register_callbacks cb 0x55e73b82b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:10:11.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.188+0000 7f15f1d9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e73b946ad0 0x55e73b944320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:11.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.188+0000 7f15f0d9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e73b93bbe0 0x55e73b943de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:11.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.188+0000 7f15f0d9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e73b93bbe0 0x55e73b943de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:33144/0 (socket says 172.21.15.101:33144) 2023-07-18T03:10:11.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.188+0000 7f15f0d9a700 1 -- 172.21.15.101:0/1309089920 learned_addr learned my addr 172.21.15.101:0/1309089920 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:11.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.188+0000 7f15f159b700 1 --2- 172.21.15.101:0/1309089920 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e73b93b2d0 0x55e73b9438a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:11.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.188+0000 7f15f1d9c700 1 -- 172.21.15.101:0/1309089920 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e73b93b2d0 msgr2=0x55e73b9438a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:11.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.188+0000 7f15f1d9c700 1 --2- 172.21.15.101:0/1309089920 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e73b93b2d0 0x55e73b9438a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:11.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.188+0000 7f15f1d9c700 1 -- 172.21.15.101:0/1309089920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e73b93bbe0 msgr2=0x55e73b943de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:11.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.188+0000 7f15f1d9c700 1 --2- 172.21.15.101:0/1309089920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e73b93bbe0 0x55e73b943de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:11.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.188+0000 7f15f1d9c700 1 -- 172.21.15.101:0/1309089920 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15e8011440 con 0x55e73b946ad0 2023-07-18T03:10:11.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.188+0000 7f15f159b700 1 --2- 172.21.15.101:0/1309089920 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e73b93b2d0 0x55e73b9438a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:10:11.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.188+0000 7f15f0d9a700 1 --2- 172.21.15.101:0/1309089920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e73b93bbe0 0x55e73b943de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:10:11.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.189+0000 7f15f1d9c700 1 --2- 172.21.15.101:0/1309089920 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e73b946ad0 0x55e73b944320 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f15e4011e60 tx=0x7f15e4014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd84927a1f4add18 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:11.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.189+0000 7f15d4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:10:11.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.189+0000 7f15d4ff9700 1 -- 172.21.15.101:0/1309089920 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e400e040 con 0x55e73b946ad0 2023-07-18T03:10:11.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.189+0000 7f160709ab80 1 -- 172.21.15.101:0/1309089920 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e73b7e74f0 con 0x55e73b946ad0 2023-07-18T03:10:11.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.189+0000 7f15d4ff9700 1 -- 172.21.15.101:0/1309089920 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f15e4004bb0 con 0x55e73b946ad0 2023-07-18T03:10:11.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.189+0000 7f15d4ff9700 1 -- 172.21.15.101:0/1309089920 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15e4004d70 con 0x55e73b946ad0 2023-07-18T03:10:11.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.189+0000 7f160709ab80 10 client.16110 fetch_fsmap learned FSMap version 25 2023-07-18T03:10:11.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.189+0000 7f160709ab80 1 -- 172.21.15.101:0/1309089920 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e73b6a4c60 con 0x55e73b946ad0 2023-07-18T03:10:11.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.189+0000 7f15d4ff9700 1 -- 172.21.15.101:0/1309089920 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e73b7e74f0 con 0x55e73b946ad0 2023-07-18T03:10:11.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.190+0000 7f15d4ff9700 1 -- 172.21.15.101:0/1309089920 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f15e401c030 con 0x55e73b946ad0 2023-07-18T03:10:11.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.190+0000 7f160709ab80 10 client.16110 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:10:11.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.190+0000 7f160709ab80 10 client.16110 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:10:11.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.190+0000 7f160709ab80 20 client.16110 populate_metadata read hostname 'smithi101' 2023-07-18T03:10:11.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.190+0000 7f160709ab80 1 --2- 172.21.15.101:0/1309089920 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55e73b9565a0 0x55e73b9589b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:11.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.190+0000 7f160709ab80 4 client.16110 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:10:11.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.190+0000 7f160709ab80 1 -- 172.21.15.101:0/1309089920 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e73b886eb0 con 0x55e73b9565a0 2023-07-18T03:10:11.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.190+0000 7f15f159b700 1 --2- 172.21.15.101:0/1309089920 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55e73b9565a0 0x55e73b9589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:11.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.191+0000 7f15f159b700 1 --2- 172.21.15.101:0/1309089920 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55e73b9565a0 0x55e73b9589b0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fbd915f8e313c7f server_cookie=849d99597b0973f2 in_seq=0 out_seq=0 2023-07-18T03:10:11.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.191+0000 7f15d4ff9700 10 client.16110 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:10:11.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.191+0000 7f15d4ff9700 1 -- 172.21.15.101:0/1309089920 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e73b886eb0 con 0x55e73b9565a0 2023-07-18T03:10:11.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.191+0000 7f15d4ff9700 10 client.16110 handle_command_reply: tid=0 2023-07-18T03:10:11.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.193+0000 7f160709ab80 1 client.16110 shutdown 2023-07-18T03:10:11.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.193+0000 7f160709ab80 1 -- 172.21.15.101:0/1309089920 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e73b946ad0 msgr2=0x55e73b944320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:11.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.193+0000 7f160709ab80 1 --2- 172.21.15.101:0/1309089920 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e73b946ad0 0x55e73b944320 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f15e4011e60 tx=0x7f15e4014600 comp rx=0 tx=0).stop 2023-07-18T03:10:11.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.193+0000 7f160709ab80 1 -- 172.21.15.101:0/1309089920 shutdown_connections 2023-07-18T03:10:11.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.193+0000 7f160709ab80 1 --2- 172.21.15.101:0/1309089920 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55e73b9565a0 0x55e73b9589b0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:11.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.193+0000 7f160709ab80 1 --2- 172.21.15.101:0/1309089920 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e73b946ad0 0x55e73b944320 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:11.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.193+0000 7f15d4ff9700 0 client.16110 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:10:11.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.193+0000 7f160709ab80 1 --2- 172.21.15.101:0/1309089920 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e73b93b2d0 0x55e73b9438a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:11.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.193+0000 7f160709ab80 1 --2- 172.21.15.101:0/1309089920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e73b93bbe0 0x55e73b943de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:11.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.193+0000 7f160709ab80 1 -- 172.21.15.101:0/1309089920 >> 172.21.15.101:0/1309089920 conn(0x55e73b931da0 msgr2=0x55e73b937230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:11.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.193+0000 7f160709ab80 1 -- 172.21.15.101:0/1309089920 shutdown_connections 2023-07-18T03:10:11.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.193+0000 7f160709ab80 1 -- 172.21.15.101:0/1309089920 wait complete. 2023-07-18T03:10:11.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.193+0000 7f160709ab80 20 client.16110 trim_cache size 0 max 16384 2023-07-18T03:10:11.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.210+0000 7f160709ab80 1 Processor -- start 2023-07-18T03:10:11.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.210+0000 7f160709ab80 1 -- start start 2023-07-18T03:10:11.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.210+0000 7f160709ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e73b93b2d0 0x55e73b94d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:11.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.210+0000 7f160709ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e73b93bbe0 0x55e73b93ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:11.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.210+0000 7f160709ab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e73b946ad0 0x55e73b93f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:11.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.210+0000 7f160709ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e73b946430 con 0x55e73b93bbe0 2023-07-18T03:10:11.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.210+0000 7f160709ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e73b953bb0 con 0x55e73b93b2d0 2023-07-18T03:10:11.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.210+0000 7f160709ab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e73b938d00 con 0x55e73b946ad0 2023-07-18T03:10:11.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.211+0000 7f15f159b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e73b93b2d0 0x55e73b94d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:11.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.211+0000 7f15f159b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e73b93b2d0 0x55e73b94d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:33156/0 (socket says 172.21.15.101:33156) 2023-07-18T03:10:11.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.211+0000 7f15f159b700 1 -- 172.21.15.101:0/4211217773 learned_addr learned my addr 172.21.15.101:0/4211217773 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:11.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.211+0000 7f15f0d9a700 1 --2- 172.21.15.101:0/4211217773 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e73b93bbe0 0x55e73b93ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:11.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.211+0000 7f15f1d9c700 1 --2- 172.21.15.101:0/4211217773 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e73b946ad0 0x55e73b93f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:11.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.211+0000 7f15f159b700 1 -- 172.21.15.101:0/4211217773 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e73b946ad0 msgr2=0x55e73b93f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:11.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.211+0000 7f15f159b700 1 --2- 172.21.15.101:0/4211217773 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e73b946ad0 0x55e73b93f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:11.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.211+0000 7f15f159b700 1 -- 172.21.15.101:0/4211217773 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e73b93bbe0 msgr2=0x55e73b93ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:11.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.211+0000 7f15f159b700 1 --2- 172.21.15.101:0/4211217773 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e73b93bbe0 0x55e73b93ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:11.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.211+0000 7f15f159b700 1 -- 172.21.15.101:0/4211217773 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e73b6e2250 con 0x55e73b93b2d0 2023-07-18T03:10:11.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.211+0000 7f15f159b700 1 --2- 172.21.15.101:0/4211217773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e73b93b2d0 0x55e73b94d4f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f15e801d610 tx=0x7f15e8028040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9da55822ade7d05e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:11.207 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.211+0000 7f15d7fff700 1 -- 172.21.15.101:0/4211217773 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e802b040 con 0x55e73b93b2d0 2023-07-18T03:10:11.207 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.212+0000 7f15d7fff700 1 -- 172.21.15.101:0/4211217773 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15e8036070 con 0x55e73b93b2d0 2023-07-18T03:10:11.207 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.212+0000 7f15d7fff700 1 -- 172.21.15.101:0/4211217773 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e80322b0 con 0x55e73b93b2d0 2023-07-18T03:10:11.207 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.212+0000 7f160709ab80 1 -- 172.21.15.101:0/4211217773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e73b93b2d0 msgr2=0x55e73b94d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:11.207 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.212+0000 7f160709ab80 1 --2- 172.21.15.101:0/4211217773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e73b93b2d0 0x55e73b94d4f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f15e801d610 tx=0x7f15e8028040 comp rx=0 tx=0).stop 2023-07-18T03:10:11.207 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.212+0000 7f160709ab80 1 -- 172.21.15.101:0/4211217773 shutdown_connections 2023-07-18T03:10:11.208 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.212+0000 7f160709ab80 1 --2- 172.21.15.101:0/4211217773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e73b93b2d0 0x55e73b94d4f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:11.208 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.212+0000 7f160709ab80 1 --2- 172.21.15.101:0/4211217773 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e73b946ad0 0x55e73b93f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:11.208 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.212+0000 7f160709ab80 1 --2- 172.21.15.101:0/4211217773 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e73b93bbe0 0x55e73b93ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:11.208 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.212+0000 7f160709ab80 1 -- 172.21.15.101:0/4211217773 >> 172.21.15.101:0/4211217773 conn(0x55e73b931da0 msgr2=0x55e73b952850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:11.208 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.212+0000 7f160709ab80 1 -- 172.21.15.101:0/4211217773 shutdown_connections 2023-07-18T03:10:11.208 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.212+0000 7f160709ab80 1 -- 172.21.15.101:0/4211217773 wait complete. 2023-07-18T03:10:11.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.213+0000 7f160709ab80 1 -- start start 2023-07-18T03:10:11.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.213+0000 7f160709ab80 1 Processor -- start 2023-07-18T03:10:11.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.214+0000 7f160709ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e73b93b2d0 0x55e73b99e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:11.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.214+0000 7f160709ab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e73b93bbe0 0x55e73b99eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:11.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.214+0000 7f160709ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e73b946ad0 0x55e73b9a9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:11.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.214+0000 7f160709ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e73b9a99b0 con 0x55e73b93b2d0 2023-07-18T03:10:11.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.214+0000 7f160709ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e73b9a9af0 con 0x55e73b946ad0 2023-07-18T03:10:11.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.214+0000 7f160709ab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e73b9a9c80 con 0x55e73b93bbe0 2023-07-18T03:10:11.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.214+0000 7f160709ab80 10 client.0 _ll_register_callbacks cb 0x7f15e40117c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:10:11.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.214+0000 7f15f159b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e73b93b2d0 0x55e73b99e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:11.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.214+0000 7f15f1d9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e73b946ad0 0x55e73b9a9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:11.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.214+0000 7f15f1d9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e73b946ad0 0x55e73b9a9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:33168/0 (socket says 172.21.15.101:33168) 2023-07-18T03:10:11.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.214+0000 7f15f1d9c700 1 -- 172.21.15.101:0/4009428134 learned_addr learned my addr 172.21.15.101:0/4009428134 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:11.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.214+0000 7f15f0d9a700 1 --2- 172.21.15.101:0/4009428134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e73b93bbe0 0x55e73b99eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:11.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.215+0000 7f15f1d9c700 1 -- 172.21.15.101:0/4009428134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e73b93bbe0 msgr2=0x55e73b99eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:11.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.215+0000 7f15f1d9c700 1 --2- 172.21.15.101:0/4009428134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e73b93bbe0 0x55e73b99eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:11.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.215+0000 7f15f1d9c700 1 -- 172.21.15.101:0/4009428134 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e73b93b2d0 msgr2=0x55e73b99e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:11.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.215+0000 7f15f1d9c700 1 --2- 172.21.15.101:0/4009428134 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e73b93b2d0 0x55e73b99e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:11.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.215+0000 7f15f1d9c700 1 -- 172.21.15.101:0/4009428134 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e73b6bbde0 con 0x55e73b946ad0 2023-07-18T03:10:11.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.215+0000 7f15f0d9a700 1 --2- 172.21.15.101:0/4009428134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e73b93bbe0 0x55e73b99eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:10:11.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.215+0000 7f15f1d9c700 1 --2- 172.21.15.101:0/4009428134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e73b946ad0 0x55e73b9a9470 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f15e4009960 tx=0x7f15e4009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec4e913183061156 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:11.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.215+0000 7f15d4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:10:11.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.215+0000 7f15d4ff9700 1 -- 172.21.15.101:0/4009428134 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e400e040 con 0x55e73b946ad0 2023-07-18T03:10:11.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.215+0000 7f160709ab80 1 -- 172.21.15.101:0/4009428134 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f15e8011e50 con 0x55e73b946ad0 2023-07-18T03:10:11.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.215+0000 7f15d4ff9700 1 -- 172.21.15.101:0/4009428134 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f15e4004860 con 0x55e73b946ad0 2023-07-18T03:10:11.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.215+0000 7f15d4ff9700 1 -- 172.21.15.101:0/4009428134 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15e4004a40 con 0x55e73b946ad0 2023-07-18T03:10:11.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.215+0000 7f160709ab80 10 client.26156 fetch_fsmap learned FSMap version 25 2023-07-18T03:10:11.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.215+0000 7f160709ab80 1 -- 172.21.15.101:0/4009428134 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e73b6ba2b0 con 0x55e73b946ad0 2023-07-18T03:10:11.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.216+0000 7f15d4ff9700 1 -- 172.21.15.101:0/4009428134 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15e8011e50 con 0x55e73b946ad0 2023-07-18T03:10:11.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.216+0000 7f15d4ff9700 1 -- 172.21.15.101:0/4009428134 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f15e4014320 con 0x55e73b946ad0 2023-07-18T03:10:11.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.216+0000 7f160709ab80 10 client.26156 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:10:11.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.216+0000 7f160709ab80 10 client.26156 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:10:11.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.216+0000 7f160709ab80 20 client.26156 populate_metadata read hostname 'smithi101' 2023-07-18T03:10:11.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.216+0000 7f160709ab80 1 --2- 172.21.15.101:0/4009428134 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55e73b9ab260 0x55e73b9ad670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:11.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.216+0000 7f160709ab80 4 client.26156 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:10:11.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.216+0000 7f160709ab80 1 -- 172.21.15.101:0/4009428134 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e73b849050 con 0x55e73b9ab260 2023-07-18T03:10:11.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.216+0000 7f15f159b700 1 --2- 172.21.15.101:0/4009428134 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55e73b9ab260 0x55e73b9ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:11.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.217+0000 7f15d4ff9700 10 client.26156 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:10:11.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.217+0000 7f15f159b700 1 --2- 172.21.15.101:0/4009428134 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55e73b9ab260 0x55e73b9ad670 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfa94389c48099cd server_cookie=b250fb3bfbfb1771 in_seq=0 out_seq=0 2023-07-18T03:10:11.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.217+0000 7f15d4ff9700 1 -- 172.21.15.101:0/4009428134 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55e73b849050 con 0x55e73b9ab260 2023-07-18T03:10:11.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.217+0000 7f15d4ff9700 10 client.26156 handle_command_reply: tid=0 2023-07-18T03:10:11.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.217+0000 7f160709ab80 1 client.26156 shutdown 2023-07-18T03:10:11.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.217+0000 7f160709ab80 1 -- 172.21.15.101:0/4009428134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e73b946ad0 msgr2=0x55e73b9a9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:11.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.217+0000 7f160709ab80 1 --2- 172.21.15.101:0/4009428134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e73b946ad0 0x55e73b9a9470 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f15e4009960 tx=0x7f15e4009510 comp rx=0 tx=0).stop 2023-07-18T03:10:11.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.217+0000 7f160709ab80 1 -- 172.21.15.101:0/4009428134 shutdown_connections 2023-07-18T03:10:11.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.217+0000 7f160709ab80 1 --2- 172.21.15.101:0/4009428134 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55e73b9ab260 0x55e73b9ad670 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:11.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.217+0000 7f160709ab80 1 --2- 172.21.15.101:0/4009428134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e73b946ad0 0x55e73b9a9470 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:11.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.217+0000 7f15d4ff9700 0 client.26156 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:10:11.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.217+0000 7f160709ab80 1 --2- 172.21.15.101:0/4009428134 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e73b93b2d0 0x55e73b99e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:11.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.217+0000 7f160709ab80 1 --2- 172.21.15.101:0/4009428134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e73b93bbe0 0x55e73b99eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:11.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.217+0000 7f160709ab80 1 -- 172.21.15.101:0/4009428134 >> 172.21.15.101:0/4009428134 conn(0x55e73b931da0 msgr2=0x55e73b943e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:11.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.217+0000 7f160709ab80 1 -- 172.21.15.101:0/4009428134 shutdown_connections 2023-07-18T03:10:11.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.217+0000 7f160709ab80 1 -- 172.21.15.101:0/4009428134 wait complete. 2023-07-18T03:10:11.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.218+0000 7f160709ab80 20 client.26156 trim_cache size 0 max 16384 2023-07-18T03:10:11.218 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:10:11.218 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:10:11.218 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:10:11.218 INFO:teuthology.orchestra.run.smithi101.stdout: "124dbb6f-f1a4-44fe-a8d7-4ce3d960f1ea": { 2023-07-18T03:10:11.218 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:10:11.219 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "124dbb6f-f1a4-44fe-a8d7-4ce3d960f1ea", 2023-07-18T03:10:11.219 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:10:11.219 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:10:11.219 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:10:11.219 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:10:11.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.218+0000 7f15f37ff700 1 -- 172.21.15.101:0/842055924 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f15d80a1340 msgr2=0x7f15d80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:11.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.218+0000 7f15f37ff700 1 --2- 172.21.15.101:0/842055924 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f15d80a1340 0x7f15d80a37f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f15e8005950 tx=0x7f15e800b410 comp rx=0 tx=0).stop 2023-07-18T03:10:11.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.218+0000 7f15f37ff700 1 -- 172.21.15.101:0/842055924 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f15ec0fe690 msgr2=0x7f15ec10e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:11.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.218+0000 7f15f37ff700 1 --2- 172.21.15.101:0/842055924 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f15ec0fe690 0x7f15ec10e4e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f15dc005910 tx=0x7f15dc004730 comp rx=0 tx=0).stop 2023-07-18T03:10:11.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.218+0000 7f15f37ff700 1 -- 172.21.15.101:0/842055924 shutdown_connections 2023-07-18T03:10:11.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.218+0000 7f15f37ff700 1 --2- 172.21.15.101:0/842055924 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f15d80a1340 0x7f15d80a37f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:11.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.218+0000 7f15f37ff700 1 --2- 172.21.15.101:0/842055924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15ec102e40 0x7f15ec10ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:11.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.218+0000 7f15f37ff700 1 --2- 172.21.15.101:0/842055924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15ec0fdd80 0x7f15ec10dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:11.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.218+0000 7f15f37ff700 1 --2- 172.21.15.101:0/842055924 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f15ec0fe690 0x7f15ec10e4e0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:11.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.218+0000 7f15f37ff700 1 -- 172.21.15.101:0/842055924 >> 172.21.15.101:0/842055924 conn(0x7f15ec075b70 msgr2=0x7f15ec0fb620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:11.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.218+0000 7f15f37ff700 1 -- 172.21.15.101:0/842055924 shutdown_connections 2023-07-18T03:10:11.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:11.218+0000 7f15f37ff700 1 -- 172.21.15.101:0/842055924 wait complete. 2023-07-18T03:10:11.224 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:10 smithi101 ceph-mon[126379]: pgmap v81: 129 pgs: 129 active+clean; 87 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 307 op/s 2023-07-18T03:10:11.227 INFO:tasks.cephfs.filesystem:scrub status for tag:124dbb6f-f1a4-44fe-a8d7-4ce3d960f1ea - {'path': '/', 'tag': '124dbb6f-f1a4-44fe-a8d7-4ce3d960f1ea', 'options': 'recursive,force'} 2023-07-18T03:10:11.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:10 smithi155 ceph-mon[128885]: pgmap v81: 129 pgs: 129 active+clean; 87 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 307 op/s 2023-07-18T03:10:11.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:10 smithi179 ceph-mon[129229]: pgmap v81: 129 pgs: 129 active+clean; 87 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 307 op/s 2023-07-18T03:10:11.568 INFO:tasks.workunit.client.0.smithi101.stdout: 1 138694 7.98 MB/sec warmup 96 sec latency 52.588 ms 2023-07-18T03:10:12.235 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:11 smithi101 ceph-mon[126379]: scrub queued for path: / 2023-07-18T03:10:12.235 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:11 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:10:12.235 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:11 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:10:12.235 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:11 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:10:12.235 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:11 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:10:12.236 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:11 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:10:12.236 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:11 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:10:12.236 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:11 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:10:12.236 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:11 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:10:12.236 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:11 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:10:12.236 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:11 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:10:12.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:11 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:10:12.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:11 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:10:12.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:11 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:10:12.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:11 smithi155 ceph-mon[128885]: scrub queued for path: / 2023-07-18T03:10:12.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:11 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:10:12.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:11 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:10:12.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:11 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:10:12.293 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:11 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:10:12.293 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:11 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:10:12.293 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:11 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:10:12.293 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:11 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:10:12.293 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:11 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:10:12.293 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:11 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:10:12.293 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:11 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:10:12.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:11 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:10:12.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:11 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:10:12.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:11 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:10:12.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:11 smithi179 ceph-mon[129229]: scrub queued for path: / 2023-07-18T03:10:12.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:11 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:10:12.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:11 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:10:12.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:11 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:10:12.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:11 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:10:12.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:11 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:10:12.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:11 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:10:12.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:11 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:10:12.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:11 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:10:12.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:11 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:10:12.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:11 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:10:12.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:11 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:10:12.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:11 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:10:12.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:11 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:10:12.568 INFO:tasks.workunit.client.0.smithi101.stdout: 1 140053 7.96 MB/sec warmup 97 sec latency 50.149 ms 2023-07-18T03:10:13.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:12 smithi101 ceph-mon[126379]: pgmap v82: 129 pgs: 129 active+clean; 87 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 1.8 MiB/s wr, 220 op/s 2023-07-18T03:10:13.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:12 smithi155 ceph-mon[128885]: pgmap v82: 129 pgs: 129 active+clean; 87 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 1.8 MiB/s wr, 220 op/s 2023-07-18T03:10:13.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:12 smithi179 ceph-mon[129229]: pgmap v82: 129 pgs: 129 active+clean; 87 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 1.8 MiB/s wr, 220 op/s 2023-07-18T03:10:13.568 INFO:tasks.workunit.client.0.smithi101.stdout: 1 142221 8.02 MB/sec warmup 98 sec latency 16.895 ms 2023-07-18T03:10:14.568 INFO:tasks.workunit.client.0.smithi101.stdout: 1 144002 8.00 MB/sec warmup 99 sec latency 13.509 ms 2023-07-18T03:10:15.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:14 smithi101 ceph-mon[126379]: pgmap v83: 129 pgs: 129 active+clean; 87 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 1.8 MiB/s wr, 220 op/s 2023-07-18T03:10:15.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:14 smithi155 ceph-mon[128885]: pgmap v83: 129 pgs: 129 active+clean; 87 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 1.8 MiB/s wr, 220 op/s 2023-07-18T03:10:15.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:14 smithi179 ceph-mon[129229]: pgmap v83: 129 pgs: 129 active+clean; 87 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 1.8 MiB/s wr, 220 op/s 2023-07-18T03:10:15.568 INFO:tasks.workunit.client.0.smithi101.stdout: 1 145837 8.05 MB/sec warmup 100 sec latency 17.185 ms 2023-07-18T03:10:16.569 INFO:tasks.workunit.client.0.smithi101.stdout: 1 147191 8.02 MB/sec warmup 101 sec latency 78.456 ms 2023-07-18T03:10:17.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:16 smithi101 ceph-mon[126379]: pgmap v84: 129 pgs: 129 active+clean; 94 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 332 op/s 2023-07-18T03:10:17.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:16 smithi155 ceph-mon[128885]: pgmap v84: 129 pgs: 129 active+clean; 94 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 332 op/s 2023-07-18T03:10:17.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:16 smithi179 ceph-mon[129229]: pgmap v84: 129 pgs: 129 active+clean; 94 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 332 op/s 2023-07-18T03:10:17.569 INFO:tasks.workunit.client.0.smithi101.stdout: 1 149265 8.08 MB/sec warmup 102 sec latency 57.965 ms 2023-07-18T03:10:18.569 INFO:tasks.workunit.client.0.smithi101.stdout: 1 150695 8.05 MB/sec warmup 103 sec latency 17.600 ms 2023-07-18T03:10:19.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:18 smithi101 ceph-mon[126379]: pgmap v85: 129 pgs: 129 active+clean; 94 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 221 op/s 2023-07-18T03:10:19.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:18 smithi155 ceph-mon[128885]: pgmap v85: 129 pgs: 129 active+clean; 94 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 221 op/s 2023-07-18T03:10:19.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:18 smithi179 ceph-mon[129229]: pgmap v85: 129 pgs: 129 active+clean; 94 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 221 op/s 2023-07-18T03:10:19.573 INFO:tasks.workunit.client.0.smithi101.stdout: 1 152851 8.11 MB/sec warmup 104 sec latency 20.234 ms 2023-07-18T03:10:20.569 INFO:tasks.workunit.client.0.smithi101.stdout: 1 154610 8.08 MB/sec warmup 105 sec latency 19.759 ms 2023-07-18T03:10:21.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:20 smithi101 ceph-mon[126379]: pgmap v86: 129 pgs: 129 active+clean; 99 MiB data, 811 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 334 op/s 2023-07-18T03:10:21.235 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:20 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:10:21.235 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:20 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:10:21.235 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:20 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:10:21.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:20 smithi155 ceph-mon[128885]: pgmap v86: 129 pgs: 129 active+clean; 99 MiB data, 811 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 334 op/s 2023-07-18T03:10:21.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:20 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:10:21.293 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:20 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:10:21.293 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:20 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:10:21.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:20 smithi179 ceph-mon[129229]: pgmap v86: 129 pgs: 129 active+clean; 99 MiB data, 811 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 334 op/s 2023-07-18T03:10:21.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:20 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:10:21.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:20 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:10:21.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:20 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:10:21.569 INFO:tasks.workunit.client.0.smithi101.stdout: 1 156405 8.13 MB/sec warmup 106 sec latency 27.523 ms 2023-07-18T03:10:22.569 INFO:tasks.workunit.client.0.smithi101.stdout: 1 157637 8.11 MB/sec warmup 107 sec latency 69.074 ms 2023-07-18T03:10:23.234 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:22 smithi101 ceph-mon[126379]: pgmap v87: 129 pgs: 129 active+clean; 99 MiB data, 811 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 224 op/s 2023-07-18T03:10:23.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:22 smithi155 ceph-mon[128885]: pgmap v87: 129 pgs: 129 active+clean; 99 MiB data, 811 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 224 op/s 2023-07-18T03:10:23.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:22 smithi179 ceph-mon[129229]: pgmap v87: 129 pgs: 129 active+clean; 99 MiB data, 811 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 224 op/s 2023-07-18T03:10:23.569 INFO:tasks.workunit.client.0.smithi101.stdout: 1 159861 8.16 MB/sec warmup 108 sec latency 62.783 ms 2023-07-18T03:10:24.569 INFO:tasks.workunit.client.0.smithi101.stdout: 1 161251 8.13 MB/sec warmup 109 sec latency 76.733 ms 2023-07-18T03:10:25.235 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:24 smithi101 ceph-mon[126379]: pgmap v88: 129 pgs: 129 active+clean; 99 MiB data, 811 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 224 op/s 2023-07-18T03:10:25.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:24 smithi155 ceph-mon[128885]: pgmap v88: 129 pgs: 129 active+clean; 99 MiB data, 811 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 224 op/s 2023-07-18T03:10:25.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:24 smithi179 ceph-mon[129229]: pgmap v88: 129 pgs: 129 active+clean; 99 MiB data, 811 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 224 op/s 2023-07-18T03:10:25.569 INFO:tasks.workunit.client.0.smithi101.stdout: 1 163470 8.18 MB/sec warmup 110 sec latency 11.348 ms 2023-07-18T03:10:26.569 INFO:tasks.workunit.client.0.smithi101.stdout: 1 164676 8.16 MB/sec warmup 111 sec latency 57.143 ms 2023-07-18T03:10:27.235 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:26 smithi101 ceph-mon[126379]: pgmap v89: 129 pgs: 129 active+clean; 103 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 337 op/s 2023-07-18T03:10:27.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:26 smithi155 ceph-mon[128885]: pgmap v89: 129 pgs: 129 active+clean; 103 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 337 op/s 2023-07-18T03:10:27.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:26 smithi179 ceph-mon[129229]: pgmap v89: 129 pgs: 129 active+clean; 103 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 337 op/s 2023-07-18T03:10:27.570 INFO:tasks.workunit.client.0.smithi101.stdout: 1 166951 8.21 MB/sec warmup 112 sec latency 32.413 ms 2023-07-18T03:10:28.570 INFO:tasks.workunit.client.0.smithi101.stdout: 1 167991 8.16 MB/sec warmup 113 sec latency 21.597 ms 2023-07-18T03:10:29.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:28 smithi155 ceph-mon[128885]: pgmap v90: 129 pgs: 129 active+clean; 103 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 225 op/s 2023-07-18T03:10:29.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:28 smithi179 ceph-mon[129229]: pgmap v90: 129 pgs: 129 active+clean; 103 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 225 op/s 2023-07-18T03:10:29.484 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:28 smithi101 ceph-mon[126379]: pgmap v90: 129 pgs: 129 active+clean; 103 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 225 op/s 2023-07-18T03:10:29.570 INFO:tasks.workunit.client.0.smithi101.stdout: 1 170502 8.23 MB/sec warmup 114 sec latency 11.445 ms 2023-07-18T03:10:30.570 INFO:tasks.workunit.client.0.smithi101.stdout: 1 171525 8.18 MB/sec warmup 115 sec latency 26.106 ms 2023-07-18T03:10:31.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:30 smithi155 ceph-mon[128885]: pgmap v91: 129 pgs: 129 active+clean; 108 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 333 op/s 2023-07-18T03:10:31.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:30 smithi179 ceph-mon[129229]: pgmap v91: 129 pgs: 129 active+clean; 108 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 333 op/s 2023-07-18T03:10:31.484 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:30 smithi101 ceph-mon[126379]: pgmap v91: 129 pgs: 129 active+clean; 108 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 333 op/s 2023-07-18T03:10:31.570 INFO:tasks.workunit.client.0.smithi101.stdout: 1 174043 8.26 MB/sec warmup 116 sec latency 16.121 ms 2023-07-18T03:10:32.570 INFO:tasks.workunit.client.0.smithi101.stdout: 1 174977 8.20 MB/sec warmup 117 sec latency 61.086 ms 2023-07-18T03:10:33.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:33 smithi155 ceph-mon[128885]: pgmap v92: 129 pgs: 129 active+clean; 108 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 220 op/s 2023-07-18T03:10:33.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:33 smithi179 ceph-mon[129229]: pgmap v92: 129 pgs: 129 active+clean; 108 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 220 op/s 2023-07-18T03:10:33.485 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:33 smithi101 ceph-mon[126379]: pgmap v92: 129 pgs: 129 active+clean; 108 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 220 op/s 2023-07-18T03:10:33.570 INFO:tasks.workunit.client.0.smithi101.stdout: 1 177339 8.27 MB/sec warmup 118 sec latency 29.194 ms 2023-07-18T03:10:34.570 INFO:tasks.workunit.client.0.smithi101.stdout: 1 178433 8.23 MB/sec warmup 119 sec latency 14.722 ms 2023-07-18T03:10:35.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:35 smithi155 ceph-mon[128885]: pgmap v93: 129 pgs: 129 active+clean; 108 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 220 op/s 2023-07-18T03:10:35.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:35 smithi179 ceph-mon[129229]: pgmap v93: 129 pgs: 129 active+clean; 108 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 220 op/s 2023-07-18T03:10:35.485 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:35 smithi101 ceph-mon[126379]: pgmap v93: 129 pgs: 129 active+clean; 108 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 220 op/s 2023-07-18T03:10:36.571 INFO:tasks.workunit.client.0.smithi101.stdout: 1 181902 2.60 MB/sec execute 1 sec latency 46.998 ms 2023-07-18T03:10:37.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:37 smithi155 ceph-mon[128885]: pgmap v94: 129 pgs: 129 active+clean; 112 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 321 op/s 2023-07-18T03:10:37.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:37 smithi179 ceph-mon[129229]: pgmap v94: 129 pgs: 129 active+clean; 112 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 321 op/s 2023-07-18T03:10:37.484 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:37 smithi101 ceph-mon[126379]: pgmap v94: 129 pgs: 129 active+clean; 112 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 321 op/s 2023-07-18T03:10:37.570 INFO:tasks.workunit.client.0.smithi101.stdout: 1 183893 7.11 MB/sec execute 2 sec latency 157.043 ms 2023-07-18T03:10:38.571 INFO:tasks.workunit.client.0.smithi101.stdout: 1 184940 6.92 MB/sec execute 3 sec latency 62.583 ms 2023-07-18T03:10:39.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:39 smithi155 ceph-mon[128885]: pgmap v95: 129 pgs: 129 active+clean; 112 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 208 op/s 2023-07-18T03:10:39.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:39 smithi179 ceph-mon[129229]: pgmap v95: 129 pgs: 129 active+clean; 112 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 208 op/s 2023-07-18T03:10:39.485 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:39 smithi101 ceph-mon[126379]: pgmap v95: 129 pgs: 129 active+clean; 112 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 208 op/s 2023-07-18T03:10:39.571 INFO:tasks.workunit.client.0.smithi101.stdout: 1 187047 7.20 MB/sec execute 4 sec latency 15.419 ms 2023-07-18T03:10:40.571 INFO:tasks.workunit.client.0.smithi101.stdout: 1 188557 7.98 MB/sec execute 5 sec latency 17.751 ms 2023-07-18T03:10:41.228 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:10:41.291 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:41 smithi101 ceph-mon[126379]: pgmap v96: 129 pgs: 129 active+clean; 116 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 313 op/s 2023-07-18T03:10:41.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:41 smithi155 ceph-mon[128885]: pgmap v96: 129 pgs: 129 active+clean; 116 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 313 op/s 2023-07-18T03:10:41.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.452+0000 7f7ffc99d700 1 -- 172.21.15.101:0/3468137798 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7ff80ff9d0 msgr2=0x7f7ff80ffe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:41.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.452+0000 7f7ffc99d700 1 --2- 172.21.15.101:0/3468137798 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7ff80ff9d0 0x7f7ff80ffe40 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7fec009990 tx=0x7f7fec009ca0 comp rx=0 tx=0).stop 2023-07-18T03:10:41.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.453+0000 7f7ffc99d700 1 -- 172.21.15.101:0/3468137798 shutdown_connections 2023-07-18T03:10:41.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.453+0000 7f7ffc99d700 1 --2- 172.21.15.101:0/3468137798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ff81059e0 0x7f7ff8105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:41.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.453+0000 7f7ffc99d700 1 --2- 172.21.15.101:0/3468137798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ff8100380 0x7f7ff810c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:41.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.453+0000 7f7ffc99d700 1 --2- 172.21.15.101:0/3468137798 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7ff80ff9d0 0x7f7ff80ffe40 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:41.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.453+0000 7f7ffc99d700 1 -- 172.21.15.101:0/3468137798 >> 172.21.15.101:0/3468137798 conn(0x7f7ff80fb4b0 msgr2=0x7f7ff80fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:41.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.453+0000 7f7ffc99d700 1 -- 172.21.15.101:0/3468137798 shutdown_connections 2023-07-18T03:10:41.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.453+0000 7f7ffc99d700 1 -- 172.21.15.101:0/3468137798 wait complete. 2023-07-18T03:10:41.446 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.454+0000 7f7ffc99d700 1 Processor -- start 2023-07-18T03:10:41.446 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.454+0000 7f7ffc99d700 1 -- start start 2023-07-18T03:10:41.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.455+0000 7f7ffc99d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ff80ff9d0 0x7f7ff8072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:41.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.455+0000 7f7ffc99d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7ff8100380 0x7f7ff8072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:41.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.455+0000 7f7ffc99d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ff81059e0 0x7f7ff81a3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:41.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.455+0000 7f7ffc99d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff8073580 con 0x7f7ff8100380 2023-07-18T03:10:41.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.455+0000 7f7ffc99d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff81a4010 con 0x7f7ff80ff9d0 2023-07-18T03:10:41.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.455+0000 7f7ffc99d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff81a4150 con 0x7f7ff81059e0 2023-07-18T03:10:41.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.455+0000 7f7ff77fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ff80ff9d0 0x7f7ff8072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:41.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.455+0000 7f7ff6ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7ff8100380 0x7f7ff8072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:41.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.455+0000 7f7ff77fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ff80ff9d0 0x7f7ff8072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:33946/0 (socket says 172.21.15.101:33946) 2023-07-18T03:10:41.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.455+0000 7f7ff77fe700 1 -- 172.21.15.101:0/3746809574 learned_addr learned my addr 172.21.15.101:0/3746809574 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:41.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.455+0000 7f7ff7fff700 1 --2- 172.21.15.101:0/3746809574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ff81059e0 0x7f7ff81a3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:41.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.456+0000 7f7ff6ffd700 1 -- 172.21.15.101:0/3746809574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ff81059e0 msgr2=0x7f7ff81a3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:41.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.456+0000 7f7ff6ffd700 1 --2- 172.21.15.101:0/3746809574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ff81059e0 0x7f7ff81a3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:41.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.456+0000 7f7ff6ffd700 1 -- 172.21.15.101:0/3746809574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ff80ff9d0 msgr2=0x7f7ff8072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:41.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.456+0000 7f7ff6ffd700 1 --2- 172.21.15.101:0/3746809574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ff80ff9d0 0x7f7ff8072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:41.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.456+0000 7f7ff6ffd700 1 -- 172.21.15.101:0/3746809574 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fec0095f0 con 0x7f7ff8100380 2023-07-18T03:10:41.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.456+0000 7f7ff77fe700 1 --2- 172.21.15.101:0/3746809574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ff80ff9d0 0x7f7ff8072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:10:41.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:41 smithi179 ceph-mon[129229]: pgmap v96: 129 pgs: 129 active+clean; 116 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 313 op/s 2023-07-18T03:10:41.450 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.456+0000 7f7ff6ffd700 1 --2- 172.21.15.101:0/3746809574 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7ff8100380 0x7f7ff8072d50 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7fec000c00 tx=0x7f7fec005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:41.450 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.456+0000 7f7ff4ff9700 1 -- 172.21.15.101:0/3746809574 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fec01d070 con 0x7f7ff8100380 2023-07-18T03:10:41.450 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.456+0000 7f7ffc99d700 1 -- 172.21.15.101:0/3746809574 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ff81a4350 con 0x7f7ff8100380 2023-07-18T03:10:41.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.456+0000 7f7ff4ff9700 1 -- 172.21.15.101:0/3746809574 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7fec021410 con 0x7f7ff8100380 2023-07-18T03:10:41.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.456+0000 7f7ff4ff9700 1 -- 172.21.15.101:0/3746809574 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fec017650 con 0x7f7ff8100380 2023-07-18T03:10:41.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.456+0000 7f7ffc99d700 1 -- 172.21.15.101:0/3746809574 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ff81a4780 con 0x7f7ff8100380 2023-07-18T03:10:41.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.459+0000 7f7ff4ff9700 1 -- 172.21.15.101:0/3746809574 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f7fec0217f0 con 0x7f7ff8100380 2023-07-18T03:10:41.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.461+0000 7f7ff4ff9700 1 --2- 172.21.15.101:0/3746809574 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f7fe00a1360 0x7f7fe00a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:41.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.461+0000 7f801147cb80 1 Processor -- start 2023-07-18T03:10:41.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.461+0000 7f7ff4ff9700 1 -- 172.21.15.101:0/3746809574 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f7fec0ce390 con 0x7f7ff8100380 2023-07-18T03:10:41.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.461+0000 7f7ff77fe700 1 --2- 172.21.15.101:0/3746809574 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f7fe00a1360 0x7f7fe00a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:41.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.461+0000 7f801147cb80 1 -- start start 2023-07-18T03:10:41.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.461+0000 7f801147cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd8da0b4f0 0x55cd8da0b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:41.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.461+0000 7f801147cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd8da0be00 0x55cd8da1e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:41.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.461+0000 7f801147cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd8da1e540 0x55cd8da20920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:41.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.461+0000 7f801147cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd8d910cb0 con 0x55cd8da0be00 2023-07-18T03:10:41.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.461+0000 7f801147cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd8d963a40 con 0x55cd8da0b4f0 2023-07-18T03:10:41.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.461+0000 7f801147cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd8d95ced0 con 0x55cd8da1e540 2023-07-18T03:10:41.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.462+0000 7f7ff77fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd8da0b4f0 0x55cd8da0b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:41.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.462+0000 7f7ff7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd8da1e540 0x55cd8da20920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:41.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.462+0000 7f7ff77fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd8da0b4f0 0x55cd8da0b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:33960/0 (socket says 172.21.15.101:33960) 2023-07-18T03:10:41.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.462+0000 7f7ff77fe700 1 -- 172.21.15.101:0/4254320028 learned_addr learned my addr 172.21.15.101:0/4254320028 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:41.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.462+0000 7f7ff6ffd700 1 --2- 172.21.15.101:0/4254320028 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd8da0be00 0x55cd8da1e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:41.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.462+0000 7f7ff77fe700 1 --2- 172.21.15.101:0/3746809574 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f7fe00a1360 0x7f7fe00a3810 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8005950 tx=0x7f7fe8009260 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:41.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.462+0000 7f7ff77fe700 1 -- 172.21.15.101:0/4254320028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd8da1e540 msgr2=0x55cd8da20920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:41.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.462+0000 7f7ff77fe700 1 --2- 172.21.15.101:0/4254320028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd8da1e540 0x55cd8da20920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:41.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.462+0000 7f7ff77fe700 1 -- 172.21.15.101:0/4254320028 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd8da0be00 msgr2=0x55cd8da1e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:41.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.462+0000 7f7ff77fe700 1 --2- 172.21.15.101:0/4254320028 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd8da0be00 0x55cd8da1e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:41.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.462+0000 7f7ff77fe700 1 -- 172.21.15.101:0/4254320028 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd8d78ade0 con 0x55cd8da0b4f0 2023-07-18T03:10:41.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.462+0000 7f7ff77fe700 1 --2- 172.21.15.101:0/4254320028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd8da0b4f0 0x55cd8da0b8c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7fe801c610 tx=0x7f7fe801c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d26e0728a17b338b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:41.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.463+0000 7f7fddffb700 1 -- 172.21.15.101:0/4254320028 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe8004020 con 0x55cd8da0b4f0 2023-07-18T03:10:41.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.463+0000 7f7fddffb700 1 -- 172.21.15.101:0/4254320028 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7fe8003680 con 0x55cd8da0b4f0 2023-07-18T03:10:41.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.463+0000 7f7fddffb700 1 -- 172.21.15.101:0/4254320028 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe8024610 con 0x55cd8da0b4f0 2023-07-18T03:10:41.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.463+0000 7f801147cb80 1 -- 172.21.15.101:0/4254320028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd8da0b4f0 msgr2=0x55cd8da0b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:41.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.463+0000 7f801147cb80 1 --2- 172.21.15.101:0/4254320028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd8da0b4f0 0x55cd8da0b8c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7fe801c610 tx=0x7f7fe801c000 comp rx=0 tx=0).stop 2023-07-18T03:10:41.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.464+0000 7f801147cb80 1 -- 172.21.15.101:0/4254320028 shutdown_connections 2023-07-18T03:10:41.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.464+0000 7f801147cb80 1 --2- 172.21.15.101:0/4254320028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd8da0b4f0 0x55cd8da0b8c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:41.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.464+0000 7f801147cb80 1 --2- 172.21.15.101:0/4254320028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd8da1e540 0x55cd8da20920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:41.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.464+0000 7f801147cb80 1 --2- 172.21.15.101:0/4254320028 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd8da0be00 0x55cd8da1e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:41.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.464+0000 7f801147cb80 1 -- 172.21.15.101:0/4254320028 >> 172.21.15.101:0/4254320028 conn(0x55cd8da00ff0 msgr2=0x55cd8da06930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:41.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.464+0000 7f801147cb80 1 -- 172.21.15.101:0/4254320028 shutdown_connections 2023-07-18T03:10:41.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.464+0000 7f801147cb80 1 -- 172.21.15.101:0/4254320028 wait complete. 2023-07-18T03:10:41.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.464+0000 7f801147cb80 1 -- start start 2023-07-18T03:10:41.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.464+0000 7f801147cb80 1 Processor -- start 2023-07-18T03:10:41.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.465+0000 7f801147cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd8da0b4f0 0x55cd8da1ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:41.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.465+0000 7f801147cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd8da0be00 0x55cd8da1b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:41.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.465+0000 7f801147cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd8da1e540 0x55cd8da1b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:41.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.465+0000 7f801147cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd8d8fcac0 con 0x55cd8da0b4f0 2023-07-18T03:10:41.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.465+0000 7f801147cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd8d8fc7c0 con 0x55cd8da1e540 2023-07-18T03:10:41.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.465+0000 7f801147cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd8d7e0920 con 0x55cd8da0be00 2023-07-18T03:10:41.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.465+0000 7f801147cb80 10 client.0 _ll_register_callbacks cb 0x55cd8d8fa760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:10:41.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.465+0000 7f7ff6ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd8da0be00 0x55cd8da1b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:41.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.465+0000 7f7ff6ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd8da0be00 0x55cd8da1b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:43202/0 (socket says 172.21.15.101:43202) 2023-07-18T03:10:41.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.465+0000 7f7ff6ffd700 1 -- 172.21.15.101:0/2600402595 learned_addr learned my addr 172.21.15.101:0/2600402595 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:41.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.465+0000 7f7ff77fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd8da0b4f0 0x55cd8da1ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:41.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.465+0000 7f7ff7fff700 1 --2- 172.21.15.101:0/2600402595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd8da1e540 0x55cd8da1b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:41.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.466+0000 7f7ff6ffd700 1 -- 172.21.15.101:0/2600402595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd8da1e540 msgr2=0x55cd8da1b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:41.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.466+0000 7f7ff6ffd700 1 --2- 172.21.15.101:0/2600402595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd8da1e540 0x55cd8da1b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:41.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.466+0000 7f7ff6ffd700 1 -- 172.21.15.101:0/2600402595 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd8da0b4f0 msgr2=0x55cd8da1ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:41.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.466+0000 7f7ff6ffd700 1 --2- 172.21.15.101:0/2600402595 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd8da0b4f0 0x55cd8da1ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:41.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.466+0000 7f7ff6ffd700 1 -- 172.21.15.101:0/2600402595 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe8017050 con 0x55cd8da0be00 2023-07-18T03:10:41.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.466+0000 7f7ff6ffd700 1 --2- 172.21.15.101:0/2600402595 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd8da0be00 0x55cd8da1b150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7fec09b0d0 tx=0x7f7fec09aaa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c5324f5bbf4fd01 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:41.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.466+0000 7f7fd6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:10:41.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.466+0000 7f7fd6ffd700 1 -- 172.21.15.101:0/2600402595 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fec09f170 con 0x55cd8da0be00 2023-07-18T03:10:41.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.466+0000 7f7fd6ffd700 1 -- 172.21.15.101:0/2600402595 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f7fec0a3500 con 0x55cd8da0be00 2023-07-18T03:10:41.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.466+0000 7f801147cb80 1 -- 172.21.15.101:0/2600402595 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cd8d8b64f0 con 0x55cd8da0be00 2023-07-18T03:10:41.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.466+0000 7f7fd6ffd700 1 -- 172.21.15.101:0/2600402595 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7fec09a630 con 0x55cd8da0be00 2023-07-18T03:10:41.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.466+0000 7f801147cb80 10 client.26194 fetch_fsmap learned FSMap version 25 2023-07-18T03:10:41.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.466+0000 7f801147cb80 1 -- 172.21.15.101:0/2600402595 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55cd8d773c60 con 0x55cd8da0be00 2023-07-18T03:10:41.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.467+0000 7f7fd6ffd700 1 -- 172.21.15.101:0/2600402595 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7fec0c9080 con 0x55cd8da0be00 2023-07-18T03:10:41.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.467+0000 7f7fd6ffd700 1 -- 172.21.15.101:0/2600402595 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f7fec00fad0 con 0x55cd8da0be00 2023-07-18T03:10:41.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.467+0000 7f801147cb80 10 client.26194 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:10:41.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.467+0000 7f801147cb80 10 client.26194 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:10:41.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.467+0000 7f801147cb80 20 client.26194 populate_metadata read hostname 'smithi101' 2023-07-18T03:10:41.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.467+0000 7f801147cb80 1 --2- 172.21.15.101:0/2600402595 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55cd8da2db40 0x55cd8da2ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:41.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.467+0000 7f801147cb80 4 client.26194 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:10:41.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.467+0000 7f801147cb80 1 -- 172.21.15.101:0/2600402595 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cd8d955eb0 con 0x55cd8da2db40 2023-07-18T03:10:41.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.467+0000 7f7ff77fe700 1 --2- 172.21.15.101:0/2600402595 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55cd8da2db40 0x55cd8da2ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:41.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.468+0000 7f7fd6ffd700 10 client.26194 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:10:41.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.468+0000 7f7ff77fe700 1 --2- 172.21.15.101:0/2600402595 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55cd8da2db40 0x55cd8da2ff50 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6af9aebbb0982aac server_cookie=70f0963fc26fea98 in_seq=0 out_seq=0 2023-07-18T03:10:41.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.468+0000 7f7fd6ffd700 1 -- 172.21.15.101:0/2600402595 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cd8d955eb0 con 0x55cd8da2db40 2023-07-18T03:10:41.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.469+0000 7f7fd6ffd700 10 client.26194 handle_command_reply: tid=0 2023-07-18T03:10:41.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.469+0000 7f801147cb80 1 client.26194 shutdown 2023-07-18T03:10:41.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.469+0000 7f801147cb80 1 -- 172.21.15.101:0/2600402595 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd8da0be00 msgr2=0x55cd8da1b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:41.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.469+0000 7f801147cb80 1 --2- 172.21.15.101:0/2600402595 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd8da0be00 0x55cd8da1b150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7fec09b0d0 tx=0x7f7fec09aaa0 comp rx=0 tx=0).stop 2023-07-18T03:10:41.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.469+0000 7f801147cb80 1 -- 172.21.15.101:0/2600402595 shutdown_connections 2023-07-18T03:10:41.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.469+0000 7f801147cb80 1 --2- 172.21.15.101:0/2600402595 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55cd8da2db40 0x55cd8da2ff50 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:41.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.469+0000 7f801147cb80 1 --2- 172.21.15.101:0/2600402595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd8da1e540 0x55cd8da1b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:41.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.469+0000 7f801147cb80 1 --2- 172.21.15.101:0/2600402595 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd8da0b4f0 0x55cd8da1ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:41.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.469+0000 7f7fd6ffd700 0 client.26194 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:10:41.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.469+0000 7f801147cb80 1 --2- 172.21.15.101:0/2600402595 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd8da0be00 0x55cd8da1b150 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:41.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.469+0000 7f801147cb80 1 -- 172.21.15.101:0/2600402595 >> 172.21.15.101:0/2600402595 conn(0x55cd8da00ff0 msgr2=0x55cd8da06520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:41.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.469+0000 7f801147cb80 1 -- 172.21.15.101:0/2600402595 shutdown_connections 2023-07-18T03:10:41.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.469+0000 7f801147cb80 1 -- 172.21.15.101:0/2600402595 wait complete. 2023-07-18T03:10:41.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.469+0000 7f801147cb80 20 client.26194 trim_cache size 0 max 16384 2023-07-18T03:10:41.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.490+0000 7f801147cb80 1 Processor -- start 2023-07-18T03:10:41.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.490+0000 7f801147cb80 1 -- start start 2023-07-18T03:10:41.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.490+0000 7f801147cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd8da0b4f0 0x55cd8da1a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:41.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.490+0000 7f801147cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd8da0be00 0x55cd8da1da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:41.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.490+0000 7f801147cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd8da1e540 0x55cd8da164a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:41.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.490+0000 7f801147cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd8da2a8c0 con 0x55cd8da0b4f0 2023-07-18T03:10:41.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.490+0000 7f801147cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd8da15fa0 con 0x55cd8da0be00 2023-07-18T03:10:41.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.490+0000 7f801147cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd8d9fe140 con 0x55cd8da1e540 2023-07-18T03:10:41.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.491+0000 7f7ff77fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd8da0b4f0 0x55cd8da1a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:41.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.491+0000 7f7ff7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd8da1e540 0x55cd8da164a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:41.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.491+0000 7f7ff6ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd8da0be00 0x55cd8da1da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:41.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.491+0000 7f7ff77fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd8da0b4f0 0x55cd8da1a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:47314/0 (socket says 172.21.15.101:47314) 2023-07-18T03:10:41.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.491+0000 7f7ff77fe700 1 -- 172.21.15.101:0/2261946661 learned_addr learned my addr 172.21.15.101:0/2261946661 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:41.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.491+0000 7f7ff7fff700 1 -- 172.21.15.101:0/2261946661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd8da0be00 msgr2=0x55cd8da1da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:41.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.491+0000 7f7ff7fff700 1 --2- 172.21.15.101:0/2261946661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd8da0be00 0x55cd8da1da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:41.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.491+0000 7f7ff7fff700 1 -- 172.21.15.101:0/2261946661 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd8da0b4f0 msgr2=0x55cd8da1a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:41.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.491+0000 7f7ff7fff700 1 --2- 172.21.15.101:0/2261946661 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd8da0b4f0 0x55cd8da1a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:41.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.491+0000 7f7ff7fff700 1 -- 172.21.15.101:0/2261946661 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd8da02830 con 0x55cd8da1e540 2023-07-18T03:10:41.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.491+0000 7f7ff6ffd700 1 --2- 172.21.15.101:0/2261946661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd8da0be00 0x55cd8da1da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:10:41.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.491+0000 7f7ff7fff700 1 --2- 172.21.15.101:0/2261946661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd8da1e540 0x55cd8da164a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7ff00117f0 tx=0x7f7ff0009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50356ecf07274db5 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:41.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.492+0000 7f7fde7fc700 1 -- 172.21.15.101:0/2261946661 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff000e040 con 0x55cd8da1e540 2023-07-18T03:10:41.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.492+0000 7f7fde7fc700 1 -- 172.21.15.101:0/2261946661 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ff000f040 con 0x55cd8da1e540 2023-07-18T03:10:41.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.492+0000 7f7fde7fc700 1 -- 172.21.15.101:0/2261946661 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff0007760 con 0x55cd8da1e540 2023-07-18T03:10:41.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.492+0000 7f801147cb80 1 -- 172.21.15.101:0/2261946661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd8da1e540 msgr2=0x55cd8da164a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:41.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.492+0000 7f801147cb80 1 --2- 172.21.15.101:0/2261946661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd8da1e540 0x55cd8da164a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7ff00117f0 tx=0x7f7ff0009920 comp rx=0 tx=0).stop 2023-07-18T03:10:41.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.492+0000 7f801147cb80 1 -- 172.21.15.101:0/2261946661 shutdown_connections 2023-07-18T03:10:41.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.492+0000 7f801147cb80 1 --2- 172.21.15.101:0/2261946661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd8da1e540 0x55cd8da164a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:41.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.492+0000 7f801147cb80 1 --2- 172.21.15.101:0/2261946661 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd8da0b4f0 0x55cd8da1a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:41.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.492+0000 7f801147cb80 1 --2- 172.21.15.101:0/2261946661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd8da0be00 0x55cd8da1da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:41.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.492+0000 7f801147cb80 1 -- 172.21.15.101:0/2261946661 >> 172.21.15.101:0/2261946661 conn(0x55cd8da00ff0 msgr2=0x55cd8da05060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:41.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.492+0000 7f801147cb80 1 -- 172.21.15.101:0/2261946661 shutdown_connections 2023-07-18T03:10:41.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.492+0000 7f801147cb80 1 -- 172.21.15.101:0/2261946661 wait complete. 2023-07-18T03:10:41.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.493+0000 7f801147cb80 1 -- start start 2023-07-18T03:10:41.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.493+0000 7f801147cb80 1 Processor -- start 2023-07-18T03:10:41.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.493+0000 7f801147cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd8da0b4f0 0x55cd8da23790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:41.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.493+0000 7f801147cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd8da0be00 0x55cd8da23cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:41.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.493+0000 7f801147cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd8da1e540 0x55cd8da24210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:41.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.493+0000 7f801147cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd8da81a10 con 0x55cd8da1e540 2023-07-18T03:10:41.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.493+0000 7f801147cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd8da81b50 con 0x55cd8da0be00 2023-07-18T03:10:41.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.493+0000 7f801147cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd8da81ce0 con 0x55cd8da0b4f0 2023-07-18T03:10:41.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.494+0000 7f801147cb80 10 client.0 _ll_register_callbacks cb 0x7f7fec09af10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:10:41.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.494+0000 7f7ff77fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd8da0b4f0 0x55cd8da23790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:41.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.494+0000 7f7ff77fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd8da0b4f0 0x55cd8da23790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:43220/0 (socket says 172.21.15.101:43220) 2023-07-18T03:10:41.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.494+0000 7f7ff77fe700 1 -- 172.21.15.101:0/2708462338 learned_addr learned my addr 172.21.15.101:0/2708462338 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:41.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.494+0000 7f7ff6ffd700 1 --2- 172.21.15.101:0/2708462338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd8da0be00 0x55cd8da23cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:41.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.494+0000 7f7ff7fff700 1 --2- 172.21.15.101:0/2708462338 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd8da1e540 0x55cd8da24210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:41.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.494+0000 7f7ff77fe700 1 -- 172.21.15.101:0/2708462338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd8da0be00 msgr2=0x55cd8da23cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:41.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.494+0000 7f7ff77fe700 1 --2- 172.21.15.101:0/2708462338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd8da0be00 0x55cd8da23cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:41.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.494+0000 7f7ff77fe700 1 -- 172.21.15.101:0/2708462338 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd8da1e540 msgr2=0x55cd8da24210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:41.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.494+0000 7f7ff77fe700 1 --2- 172.21.15.101:0/2708462338 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd8da1e540 0x55cd8da24210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:41.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.494+0000 7f7ff77fe700 1 -- 172.21.15.101:0/2708462338 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd8d773c60 con 0x55cd8da0b4f0 2023-07-18T03:10:41.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.494+0000 7f7ff77fe700 1 --2- 172.21.15.101:0/2708462338 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd8da0b4f0 0x55cd8da23790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7fe801ccf0 tx=0x7f7fe801cd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51729ca1314903ae server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:41.492 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.494+0000 7f7fd6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:10:41.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.494+0000 7f7fd6ffd700 1 -- 172.21.15.101:0/2708462338 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe80352d0 con 0x55cd8da0b4f0 2023-07-18T03:10:41.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.495+0000 7f7fd6ffd700 1 -- 172.21.15.101:0/2708462338 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f7fe8034070 con 0x55cd8da0b4f0 2023-07-18T03:10:41.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.495+0000 7f801147cb80 1 -- 172.21.15.101:0/2708462338 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7fe8003680 con 0x55cd8da0b4f0 2023-07-18T03:10:41.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.495+0000 7f7fd6ffd700 1 -- 172.21.15.101:0/2708462338 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7fe8035430 con 0x55cd8da0b4f0 2023-07-18T03:10:41.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.495+0000 7f801147cb80 10 client.26200 fetch_fsmap learned FSMap version 25 2023-07-18T03:10:41.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.495+0000 7f801147cb80 1 -- 172.21.15.101:0/2708462338 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f7ff00114d0 con 0x55cd8da0b4f0 2023-07-18T03:10:41.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.495+0000 7f7fd6ffd700 1 -- 172.21.15.101:0/2708462338 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7fe8003680 con 0x55cd8da0b4f0 2023-07-18T03:10:41.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.495+0000 7f7fd6ffd700 1 -- 172.21.15.101:0/2708462338 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f7fe801c640 con 0x55cd8da0b4f0 2023-07-18T03:10:41.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.495+0000 7f801147cb80 10 client.26200 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:10:41.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.495+0000 7f801147cb80 10 client.26200 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:10:41.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.495+0000 7f801147cb80 20 client.26200 populate_metadata read hostname 'smithi101' 2023-07-18T03:10:41.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.495+0000 7f801147cb80 1 --2- 172.21.15.101:0/2708462338 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55cd8da83470 0x55cd8da85880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:41.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.495+0000 7f801147cb80 4 client.26200 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:10:41.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.495+0000 7f801147cb80 1 -- 172.21.15.101:0/2708462338 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cd8d918050 con 0x55cd8da83470 2023-07-18T03:10:41.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.496+0000 7f7ff6ffd700 1 --2- 172.21.15.101:0/2708462338 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55cd8da83470 0x55cd8da85880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:41.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.496+0000 7f7ff6ffd700 1 --2- 172.21.15.101:0/2708462338 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55cd8da83470 0x55cd8da85880 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a3684eff9caa082 server_cookie=cfc40d83e199d9c3 in_seq=0 out_seq=0 2023-07-18T03:10:41.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.496+0000 7f7fd6ffd700 10 client.26200 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:10:41.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.496+0000 7f7fd6ffd700 1 -- 172.21.15.101:0/2708462338 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55cd8d918050 con 0x55cd8da83470 2023-07-18T03:10:41.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.496+0000 7f7fd6ffd700 10 client.26200 handle_command_reply: tid=0 2023-07-18T03:10:41.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.496+0000 7f801147cb80 1 client.26200 shutdown 2023-07-18T03:10:41.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.496+0000 7f801147cb80 1 -- 172.21.15.101:0/2708462338 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd8da0b4f0 msgr2=0x55cd8da23790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:41.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.496+0000 7f801147cb80 1 --2- 172.21.15.101:0/2708462338 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd8da0b4f0 0x55cd8da23790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7fe801ccf0 tx=0x7f7fe801cd20 comp rx=0 tx=0).stop 2023-07-18T03:10:41.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.497+0000 7f801147cb80 1 -- 172.21.15.101:0/2708462338 shutdown_connections 2023-07-18T03:10:41.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.497+0000 7f801147cb80 1 --2- 172.21.15.101:0/2708462338 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55cd8da83470 0x55cd8da85880 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:41.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.497+0000 7f801147cb80 1 --2- 172.21.15.101:0/2708462338 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd8da1e540 0x55cd8da24210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:41.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.497+0000 7f801147cb80 1 --2- 172.21.15.101:0/2708462338 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd8da0b4f0 0x55cd8da23790 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:41.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.497+0000 7f801147cb80 1 --2- 172.21.15.101:0/2708462338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cd8da0be00 0x55cd8da23cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:41.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.497+0000 7f7fd6ffd700 0 client.26200 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:10:41.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.497+0000 7f801147cb80 1 -- 172.21.15.101:0/2708462338 >> 172.21.15.101:0/2708462338 conn(0x55cd8da00ff0 msgr2=0x55cd8da0a440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:41.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.497+0000 7f801147cb80 1 -- 172.21.15.101:0/2708462338 shutdown_connections 2023-07-18T03:10:41.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.497+0000 7f801147cb80 1 -- 172.21.15.101:0/2708462338 wait complete. 2023-07-18T03:10:41.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.497+0000 7f801147cb80 20 client.26200 trim_cache size 0 max 16384 2023-07-18T03:10:41.503 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:10:41.503 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "no active scrubs running", 2023-07-18T03:10:41.503 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": {} 2023-07-18T03:10:41.504 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:10:41.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.497+0000 7f7ffc99d700 1 -- 172.21.15.101:0/3746809574 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f7fe00a1360 msgr2=0x7f7fe00a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:41.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.497+0000 7f7ffc99d700 1 --2- 172.21.15.101:0/3746809574 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f7fe00a1360 0x7f7fe00a3810 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8005950 tx=0x7f7fe8009260 comp rx=0 tx=0).stop 2023-07-18T03:10:41.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.497+0000 7f7ffc99d700 1 -- 172.21.15.101:0/3746809574 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7ff8100380 msgr2=0x7f7ff8072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:41.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.497+0000 7f7ffc99d700 1 --2- 172.21.15.101:0/3746809574 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7ff8100380 0x7f7ff8072d50 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7fec000c00 tx=0x7f7fec005740 comp rx=0 tx=0).stop 2023-07-18T03:10:41.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.498+0000 7f7ffc99d700 1 -- 172.21.15.101:0/3746809574 shutdown_connections 2023-07-18T03:10:41.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.498+0000 7f7ffc99d700 1 --2- 172.21.15.101:0/3746809574 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f7fe00a1360 0x7f7fe00a3810 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:41.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.498+0000 7f7ffc99d700 1 --2- 172.21.15.101:0/3746809574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ff80ff9d0 0x7f7ff8072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:41.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.498+0000 7f7ffc99d700 1 --2- 172.21.15.101:0/3746809574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ff81059e0 0x7f7ff81a3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:41.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.498+0000 7f7ffc99d700 1 --2- 172.21.15.101:0/3746809574 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7ff8100380 0x7f7ff8072d50 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:41.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.498+0000 7f7ffc99d700 1 -- 172.21.15.101:0/3746809574 >> 172.21.15.101:0/3746809574 conn(0x7f7ff80fb4b0 msgr2=0x7f7ff81067e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:41.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.498+0000 7f7ffc99d700 1 -- 172.21.15.101:0/3746809574 shutdown_connections 2023-07-18T03:10:41.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.498+0000 7f7ffc99d700 1 -- 172.21.15.101:0/3746809574 wait complete. 2023-07-18T03:10:41.509 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-18T03:10:41.509 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:10:41.571 INFO:tasks.workunit.client.0.smithi101.stdout: 1 190772 8.48 MB/sec execute 6 sec latency 14.293 ms 2023-07-18T03:10:41.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.727+0000 7f6b27fff700 1 -- 172.21.15.101:0/2963458120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b2010c3b0 msgr2=0x7f6b2010e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:41.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.727+0000 7f6b27fff700 1 --2- 172.21.15.101:0/2963458120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b2010c3b0 0x7f6b2010e790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6b18009f90 tx=0x7f6b1800b330 comp rx=0 tx=0).stop 2023-07-18T03:10:41.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.728+0000 7f6b27fff700 1 -- 172.21.15.101:0/2963458120 shutdown_connections 2023-07-18T03:10:41.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.728+0000 7f6b27fff700 1 --2- 172.21.15.101:0/2963458120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b2010c3b0 0x7f6b2010e790 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:41.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.728+0000 7f6b27fff700 1 --2- 172.21.15.101:0/2963458120 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6b20072850 0x7f6b20072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:41.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.728+0000 7f6b27fff700 1 --2- 172.21.15.101:0/2963458120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b20073160 0x7f6b2010be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:41.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.728+0000 7f6b27fff700 1 -- 172.21.15.101:0/2963458120 >> 172.21.15.101:0/2963458120 conn(0x7f6b200fb610 msgr2=0x7f6b200fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:41.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.728+0000 7f6b27fff700 1 -- 172.21.15.101:0/2963458120 shutdown_connections 2023-07-18T03:10:41.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.728+0000 7f6b27fff700 1 -- 172.21.15.101:0/2963458120 wait complete. 2023-07-18T03:10:41.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.729+0000 7f6b27fff700 1 Processor -- start 2023-07-18T03:10:41.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.729+0000 7f6b27fff700 1 -- start start 2023-07-18T03:10:41.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.729+0000 7f6b27fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b20072850 0x7f6b20197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:41.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.729+0000 7f6b27fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6b20073160 0x7f6b20198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:41.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.729+0000 7f6b27fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b2010c3b0 0x7f6b2019c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:41.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.729+0000 7f6b27fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2019cef0 con 0x7f6b20073160 2023-07-18T03:10:41.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.729+0000 7f6b27fff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2019d060 con 0x7f6b20072850 2023-07-18T03:10:41.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.729+0000 7f6b27fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2019d220 con 0x7f6b2010c3b0 2023-07-18T03:10:41.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.729+0000 7f6b2559a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6b20073160 0x7f6b20198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:41.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.729+0000 7f6b25d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b20072850 0x7f6b20197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:41.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.729+0000 7f6b25d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b20072850 0x7f6b20197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:34006/0 (socket says 172.21.15.101:34006) 2023-07-18T03:10:41.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.729+0000 7f6b2659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b2010c3b0 0x7f6b2019c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:41.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.729+0000 7f6b2559a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6b20073160 0x7f6b20198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:47352/0 (socket says 172.21.15.101:47352) 2023-07-18T03:10:41.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.729+0000 7f6b25d9b700 1 -- 172.21.15.101:0/2807344120 learned_addr learned my addr 172.21.15.101:0/2807344120 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:41.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.730+0000 7f6b2659c700 1 -- 172.21.15.101:0/2807344120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b20072850 msgr2=0x7f6b20197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:41.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.730+0000 7f6b2659c700 1 --2- 172.21.15.101:0/2807344120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b20072850 0x7f6b20197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:41.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.730+0000 7f6b2659c700 1 -- 172.21.15.101:0/2807344120 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6b20073160 msgr2=0x7f6b20198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:41.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.730+0000 7f6b2659c700 1 --2- 172.21.15.101:0/2807344120 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6b20073160 0x7f6b20198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:41.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.730+0000 7f6b2659c700 1 -- 172.21.15.101:0/2807344120 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b1c009520 con 0x7f6b2010c3b0 2023-07-18T03:10:41.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.730+0000 7f6b2559a700 1 --2- 172.21.15.101:0/2807344120 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6b20073160 0x7f6b20198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:10:41.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.730+0000 7f6b2659c700 1 --2- 172.21.15.101:0/2807344120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b2010c3b0 0x7f6b2019c8b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800b6c0 tx=0x7f6b18009490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:41.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.730+0000 7f6b16ffd700 1 -- 172.21.15.101:0/2807344120 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1800e040 con 0x7f6b2010c3b0 2023-07-18T03:10:41.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.730+0000 7f6b27fff700 1 -- 172.21.15.101:0/2807344120 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b100095f0 con 0x7f6b2010c3b0 2023-07-18T03:10:41.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.730+0000 7f6b16ffd700 1 -- 172.21.15.101:0/2807344120 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b1801d470 con 0x7f6b2010c3b0 2023-07-18T03:10:41.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.730+0000 7f6b16ffd700 1 -- 172.21.15.101:0/2807344120 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b18005280 con 0x7f6b2010c3b0 2023-07-18T03:10:41.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.730+0000 7f6b27fff700 1 -- 172.21.15.101:0/2807344120 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b18009c40 con 0x7f6b2010c3b0 2023-07-18T03:10:41.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.731+0000 7f6b27fff700 1 -- 172.21.15.101:0/2807344120 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b20106cd0 con 0x7f6b2010c3b0 2023-07-18T03:10:41.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.733+0000 7f6b16ffd700 1 -- 172.21.15.101:0/2807344120 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f6b1801d5e0 con 0x7f6b2010c3b0 2023-07-18T03:10:41.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.735+0000 7f6b16ffd700 1 --2- 172.21.15.101:0/2807344120 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6b0c0a1300 0x7f6b0c0a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:41.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.735+0000 7f6b16ffd700 1 -- 172.21.15.101:0/2807344120 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f6b180ce580 con 0x7f6b2010c3b0 2023-07-18T03:10:41.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.735+0000 7f6b25d9b700 1 --2- 172.21.15.101:0/2807344120 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6b0c0a1300 0x7f6b0c0a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:41.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.735+0000 7f6b25d9b700 1 --2- 172.21.15.101:0/2807344120 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6b0c0a1300 0x7f6b0c0a37b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c009ef0 tx=0x7f6b1c009260 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:41.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:41.737+0000 7f6b16ffd700 1 -- 172.21.15.101:0/2807344120 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6b18097670 con 0x7f6b2010c3b0 2023-07-18T03:10:41.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.008+0000 7f6b27fff700 1 -- 172.21.15.101:0/2807344120 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b20067300 con 0x7f6b2010c3b0 2023-07-18T03:10:42.000 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.009+0000 7f6b16ffd700 1 -- 172.21.15.101:0/2807344120 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10861 (secure 0 0 0) 0x7f6b1809ac90 con 0x7f6b2010c3b0 2023-07-18T03:10:42.001 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:10:42.001 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/4192846769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":4192846769},{"type":"v1","addr":"172.21.15.101:6839","nonce":4192846769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6841/3824340175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6840","nonce":3824340175},{"type":"v1","addr":"172.21.15.101:6841","nonce":3824340175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:04:59.361185+0000","modified":"2023-07-18T03:09:01.105419+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.155:6839/2055462483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2055462483},{"type":"v1","addr":"172.21.15.155:6839","nonce":2055462483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:10:42.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.011+0000 7f6b27fff700 1 -- 172.21.15.101:0/2807344120 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6b0c0a1300 msgr2=0x7f6b0c0a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:42.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.011+0000 7f6b27fff700 1 --2- 172.21.15.101:0/2807344120 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6b0c0a1300 0x7f6b0c0a37b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c009ef0 tx=0x7f6b1c009260 comp rx=0 tx=0).stop 2023-07-18T03:10:42.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.011+0000 7f6b27fff700 1 -- 172.21.15.101:0/2807344120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b2010c3b0 msgr2=0x7f6b2019c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:42.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.011+0000 7f6b27fff700 1 --2- 172.21.15.101:0/2807344120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b2010c3b0 0x7f6b2019c8b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800b6c0 tx=0x7f6b18009490 comp rx=0 tx=0).stop 2023-07-18T03:10:42.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.012+0000 7f6b27fff700 1 -- 172.21.15.101:0/2807344120 shutdown_connections 2023-07-18T03:10:42.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.012+0000 7f6b27fff700 1 --2- 172.21.15.101:0/2807344120 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6b0c0a1300 0x7f6b0c0a37b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.012+0000 7f6b27fff700 1 --2- 172.21.15.101:0/2807344120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b20072850 0x7f6b20197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.012+0000 7f6b27fff700 1 --2- 172.21.15.101:0/2807344120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b2010c3b0 0x7f6b2019c8b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.012+0000 7f6b27fff700 1 --2- 172.21.15.101:0/2807344120 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6b20073160 0x7f6b20198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.012+0000 7f6b27fff700 1 -- 172.21.15.101:0/2807344120 >> 172.21.15.101:0/2807344120 conn(0x7f6b200fb610 msgr2=0x7f6b20103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:42.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.012+0000 7f6b27fff700 1 -- 172.21.15.101:0/2807344120 shutdown_connections 2023-07-18T03:10:42.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.012+0000 7f6b27fff700 1 -- 172.21.15.101:0/2807344120 wait complete. 2023-07-18T03:10:42.009 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 25 2023-07-18T03:10:42.023 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-18T03:10:42.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.242+0000 7f8b7c7ff700 1 -- 172.21.15.101:0/3498812111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b74105af0 msgr2=0x7f8b74105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:42.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.242+0000 7f8b7c7ff700 1 --2- 172.21.15.101:0/3498812111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b74105af0 0x7f8b74105ec0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8b70009990 tx=0x7f8b70009ca0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.243+0000 7f8b7c7ff700 1 -- 172.21.15.101:0/3498812111 shutdown_connections 2023-07-18T03:10:42.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.243+0000 7f8b7c7ff700 1 --2- 172.21.15.101:0/3498812111 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8b74100490 0x7f8b7410c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.243+0000 7f8b7c7ff700 1 --2- 172.21.15.101:0/3498812111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b74105af0 0x7f8b74105ec0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.243+0000 7f8b7c7ff700 1 --2- 172.21.15.101:0/3498812111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b740ffae0 0x7f8b740fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.243+0000 7f8b7c7ff700 1 -- 172.21.15.101:0/3498812111 >> 172.21.15.101:0/3498812111 conn(0x7f8b740fb5e0 msgr2=0x7f8b740fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:42.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.243+0000 7f8b7c7ff700 1 -- 172.21.15.101:0/3498812111 shutdown_connections 2023-07-18T03:10:42.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.243+0000 7f8b7c7ff700 1 -- 172.21.15.101:0/3498812111 wait complete. 2023-07-18T03:10:42.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.244+0000 7f8b7c7ff700 1 Processor -- start 2023-07-18T03:10:42.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.244+0000 7f8b7c7ff700 1 -- start start 2023-07-18T03:10:42.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.244+0000 7f8b7c7ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b740ffae0 0x7f8b74197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.244+0000 7f8b7c7ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8b74100490 0x7f8b74198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.244+0000 7f8b7c7ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b74105af0 0x7f8b7419c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.244+0000 7f8b7c7ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7419cdc0 con 0x7f8b74100490 2023-07-18T03:10:42.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.244+0000 7f8b7c7ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7419cf30 con 0x7f8b74105af0 2023-07-18T03:10:42.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.244+0000 7f8b7c7ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7419d0f0 con 0x7f8b740ffae0 2023-07-18T03:10:42.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.245+0000 7f8b79d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8b74100490 0x7f8b74198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.245+0000 7f8b7a59b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b740ffae0 0x7f8b74197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.245+0000 7f8b79d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8b74100490 0x7f8b74198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:47372/0 (socket says 172.21.15.101:47372) 2023-07-18T03:10:42.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.245+0000 7f8b79d9a700 1 -- 172.21.15.101:0/2502298184 learned_addr learned my addr 172.21.15.101:0/2502298184 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:42.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.245+0000 7f8b7ad9c700 1 --2- 172.21.15.101:0/2502298184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b74105af0 0x7f8b7419c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.245+0000 7f8b7a59b700 1 -- 172.21.15.101:0/2502298184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b74105af0 msgr2=0x7f8b7419c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:42.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.245+0000 7f8b7a59b700 1 --2- 172.21.15.101:0/2502298184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b74105af0 0x7f8b7419c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.245+0000 7f8b7a59b700 1 -- 172.21.15.101:0/2502298184 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8b74100490 msgr2=0x7f8b74198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:42.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.245+0000 7f8b7a59b700 1 --2- 172.21.15.101:0/2502298184 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8b74100490 0x7f8b74198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.245+0000 7f8b7a59b700 1 -- 172.21.15.101:0/2502298184 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b64009520 con 0x7f8b740ffae0 2023-07-18T03:10:42.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.245+0000 7f8b79d9a700 1 --2- 172.21.15.101:0/2502298184 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8b74100490 0x7f8b74198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:10:42.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.245+0000 7f8b7a59b700 1 --2- 172.21.15.101:0/2502298184 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b740ffae0 0x7f8b74197de0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8b70009990 tx=0x7f8b70009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:42.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.245+0000 7f8b6b7fe700 1 -- 172.21.15.101:0/2502298184 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7001d070 con 0x7f8b740ffae0 2023-07-18T03:10:42.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.245+0000 7f8b6b7fe700 1 -- 172.21.15.101:0/2502298184 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b70021410 con 0x7f8b740ffae0 2023-07-18T03:10:42.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.245+0000 7f8b7c7ff700 1 -- 172.21.15.101:0/2502298184 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b6c009c40 con 0x7f8b740ffae0 2023-07-18T03:10:42.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.246+0000 7f8b6b7fe700 1 -- 172.21.15.101:0/2502298184 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b70017650 con 0x7f8b740ffae0 2023-07-18T03:10:42.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.246+0000 7f8b7c7ff700 1 -- 172.21.15.101:0/2502298184 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b700095f0 con 0x7f8b740ffae0 2023-07-18T03:10:42.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.249+0000 7f8b6b7fe700 1 -- 172.21.15.101:0/2502298184 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f8b700217a0 con 0x7f8b740ffae0 2023-07-18T03:10:42.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.251+0000 7f8b901e7b80 1 Processor -- start 2023-07-18T03:10:42.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.251+0000 7f8b901e7b80 1 -- start start 2023-07-18T03:10:42.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.251+0000 7f8b6b7fe700 1 --2- 172.21.15.101:0/2502298184 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f8b600a1340 0x7f8b600a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.251+0000 7f8b6b7fe700 1 -- 172.21.15.101:0/2502298184 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f8b70021e40 con 0x7f8b740ffae0 2023-07-18T03:10:42.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.251+0000 7f8b901e7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c32fe62d0 0x556c32fe66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.251+0000 7f8b901e7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c32fe6be0 0x556c32ff1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.251+0000 7f8b901e7b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c32ff1ad0 0x556c32ff3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.251+0000 7f8b901e7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c32eeccb0 con 0x556c32fe6be0 2023-07-18T03:10:42.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.251+0000 7f8b901e7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c32f3fa40 con 0x556c32fe62d0 2023-07-18T03:10:42.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.251+0000 7f8b901e7b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c32f38ed0 con 0x556c32ff1ad0 2023-07-18T03:10:42.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.252+0000 7f8b7ad9c700 1 --2- 172.21.15.101:0/2502298184 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f8b600a1340 0x7f8b600a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.252+0000 7f8b79d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c32fe6be0 0x556c32ff1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.252+0000 7f8b7a59b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c32fe62d0 0x556c32fe66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.252+0000 7f8b7a59b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c32fe62d0 0x556c32fe66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:34038/0 (socket says 172.21.15.101:34038) 2023-07-18T03:10:42.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.252+0000 7f8b7a59b700 1 -- 172.21.15.101:0/322471935 learned_addr learned my addr 172.21.15.101:0/322471935 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:42.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.252+0000 7f8b7ad9c700 1 --2- 172.21.15.101:0/322471935 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c32ff1ad0 0x556c32ff3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.252+0000 7f8b79d9a700 1 -- 172.21.15.101:0/322471935 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c32ff1ad0 msgr2=0x556c32ff3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:42.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.252+0000 7f8b79d9a700 1 --2- 172.21.15.101:0/322471935 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c32ff1ad0 0x556c32ff3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.252+0000 7f8b79d9a700 1 -- 172.21.15.101:0/322471935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c32fe62d0 msgr2=0x556c32fe66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:42.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.252+0000 7f8b79d9a700 1 --2- 172.21.15.101:0/322471935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c32fe62d0 0x556c32fe66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.252+0000 7f8b79d9a700 1 -- 172.21.15.101:0/322471935 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c32d66de0 con 0x556c32fe6be0 2023-07-18T03:10:42.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.252+0000 7f8b7ad9c700 1 --2- 172.21.15.101:0/322471935 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c32ff1ad0 0x556c32ff3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:10:42.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.252+0000 7f8b7ad9c700 1 --2- 172.21.15.101:0/2502298184 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f8b600a1340 0x7f8b600a37f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c009fd0 tx=0x7f8b6c009490 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:42.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.253+0000 7f8b79d9a700 1 --2- 172.21.15.101:0/322471935 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c32fe6be0 0x556c32ff1590 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8b640117e0 tx=0x7f8b64011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ea83a76528ce109 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:42.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.253+0000 7f8b68ff9700 1 -- 172.21.15.101:0/322471935 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b64004020 con 0x556c32fe6be0 2023-07-18T03:10:42.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.253+0000 7f8b68ff9700 1 -- 172.21.15.101:0/322471935 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b64004830 con 0x556c32fe6be0 2023-07-18T03:10:42.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.253+0000 7f8b68ff9700 1 -- 172.21.15.101:0/322471935 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b64005240 con 0x556c32fe6be0 2023-07-18T03:10:42.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.254+0000 7f8b901e7b80 1 -- 172.21.15.101:0/322471935 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c32fe6be0 msgr2=0x556c32ff1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:42.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.254+0000 7f8b901e7b80 1 --2- 172.21.15.101:0/322471935 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c32fe6be0 0x556c32ff1590 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8b640117e0 tx=0x7f8b64011af0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.254+0000 7f8b901e7b80 1 -- 172.21.15.101:0/322471935 shutdown_connections 2023-07-18T03:10:42.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.254+0000 7f8b901e7b80 1 --2- 172.21.15.101:0/322471935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c32fe62d0 0x556c32fe66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.254+0000 7f8b901e7b80 1 --2- 172.21.15.101:0/322471935 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c32ff1ad0 0x556c32ff3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.254+0000 7f8b901e7b80 1 --2- 172.21.15.101:0/322471935 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c32fe6be0 0x556c32ff1590 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.254+0000 7f8b901e7b80 1 -- 172.21.15.101:0/322471935 >> 172.21.15.101:0/322471935 conn(0x556c32fdcda0 msgr2=0x556c32fe26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:42.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.254+0000 7f8b901e7b80 1 -- 172.21.15.101:0/322471935 shutdown_connections 2023-07-18T03:10:42.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.254+0000 7f8b901e7b80 1 -- 172.21.15.101:0/322471935 wait complete. 2023-07-18T03:10:42.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.254+0000 7f8b901e7b80 1 -- start start 2023-07-18T03:10:42.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.255+0000 7f8b901e7b80 1 Processor -- start 2023-07-18T03:10:42.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.255+0000 7f8b901e7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c32fe62d0 0x556c32fee9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.255+0000 7f8b901e7b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c32ff1ad0 0x556c32feef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.255+0000 7f8b901e7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c32fef470 0x556c32ffd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.256+0000 7f8b901e7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c32ed8ac0 con 0x556c32fe62d0 2023-07-18T03:10:42.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.256+0000 7f8b901e7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c32ed87c0 con 0x556c32fef470 2023-07-18T03:10:42.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.256+0000 7f8b901e7b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c32dbc920 con 0x556c32ff1ad0 2023-07-18T03:10:42.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.256+0000 7f8b901e7b80 10 client.0 _ll_register_callbacks cb 0x556c32ed6760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:10:42.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.256+0000 7f8b79d9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c32ff1ad0 0x556c32feef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.256+0000 7f8b7a59b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c32fe62d0 0x556c32fee9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.256+0000 7f8b79d9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c32ff1ad0 0x556c32feef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:43294/0 (socket says 172.21.15.101:43294) 2023-07-18T03:10:42.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.256+0000 7f8b79d9a700 1 -- 172.21.15.101:0/3840726385 learned_addr learned my addr 172.21.15.101:0/3840726385 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:42.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.256+0000 7f8b7ad9c700 1 --2- 172.21.15.101:0/3840726385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c32fef470 0x556c32ffd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.256+0000 7f8b79d9a700 1 -- 172.21.15.101:0/3840726385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c32fef470 msgr2=0x556c32ffd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:42.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.256+0000 7f8b79d9a700 1 --2- 172.21.15.101:0/3840726385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c32fef470 0x556c32ffd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.256+0000 7f8b79d9a700 1 -- 172.21.15.101:0/3840726385 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c32fe62d0 msgr2=0x556c32fee9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:42.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.256+0000 7f8b79d9a700 1 --2- 172.21.15.101:0/3840726385 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c32fe62d0 0x556c32fee9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.256+0000 7f8b79d9a700 1 -- 172.21.15.101:0/3840726385 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b6c01abe0 con 0x556c32ff1ad0 2023-07-18T03:10:42.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.256+0000 7f8b79d9a700 1 --2- 172.21.15.101:0/3840726385 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c32ff1ad0 0x556c32feef30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8b64009940 tx=0x7f8b640042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53444e4005caba82 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:42.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.256+0000 7f8b5dffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:10:42.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.257+0000 7f8b5dffb700 1 -- 172.21.15.101:0/3840726385 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b64004590 con 0x556c32ff1ad0 2023-07-18T03:10:42.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.257+0000 7f8b901e7b80 1 -- 172.21.15.101:0/3840726385 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556c32e924f0 con 0x556c32ff1ad0 2023-07-18T03:10:42.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.257+0000 7f8b5dffb700 1 -- 172.21.15.101:0/3840726385 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f8b64004fe0 con 0x556c32ff1ad0 2023-07-18T03:10:42.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.257+0000 7f8b5dffb700 1 -- 172.21.15.101:0/3840726385 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b640051a0 con 0x556c32ff1ad0 2023-07-18T03:10:42.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.257+0000 7f8b901e7b80 10 client.26218 fetch_fsmap learned FSMap version 25 2023-07-18T03:10:42.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.257+0000 7f8b901e7b80 1 -- 172.21.15.101:0/3840726385 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f8b64011440 con 0x556c32ff1ad0 2023-07-18T03:10:42.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.257+0000 7f8b5dffb700 1 -- 172.21.15.101:0/3840726385 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556c32e924f0 con 0x556c32ff1ad0 2023-07-18T03:10:42.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.257+0000 7f8b5dffb700 1 -- 172.21.15.101:0/3840726385 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f8b64003af0 con 0x556c32ff1ad0 2023-07-18T03:10:42.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.257+0000 7f8b901e7b80 10 client.26218 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:10:42.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.257+0000 7f8b901e7b80 10 client.26218 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:10:42.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.257+0000 7f8b901e7b80 20 client.26218 populate_metadata read hostname 'smithi101' 2023-07-18T03:10:42.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.257+0000 7f8b901e7b80 1 --2- 172.21.15.101:0/3840726385 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x556c33001a10 0x556c33003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.258+0000 7f8b901e7b80 4 client.26218 mds_command: new command op to 24541 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:10:42.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.258+0000 7f8b901e7b80 1 -- 172.21.15.101:0/3840726385 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556c32f31eb0 con 0x556c33001a10 2023-07-18T03:10:42.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.258+0000 7f8b7a59b700 1 --2- 172.21.15.101:0/3840726385 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x556c33001a10 0x556c33003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.258+0000 7f8b5dffb700 10 client.26218 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:10:42.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.258+0000 7f8b7a59b700 1 --2- 172.21.15.101:0/3840726385 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x556c33001a10 0x556c33003e20 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=78a919a76903e4ee server_cookie=d91695de4f3b9cfc in_seq=0 out_seq=0 2023-07-18T03:10:42.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.260+0000 7f8b5dffb700 1 -- 172.21.15.101:0/3840726385 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556c32f31eb0 con 0x556c33001a10 2023-07-18T03:10:42.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.260+0000 7f8b5dffb700 10 client.26218 handle_command_reply: tid=0 2023-07-18T03:10:42.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.260+0000 7f8b901e7b80 1 client.26218 shutdown 2023-07-18T03:10:42.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.260+0000 7f8b901e7b80 1 -- 172.21.15.101:0/3840726385 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c32ff1ad0 msgr2=0x556c32feef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:42.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.260+0000 7f8b901e7b80 1 --2- 172.21.15.101:0/3840726385 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c32ff1ad0 0x556c32feef30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8b64009940 tx=0x7f8b640042f0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.260+0000 7f8b901e7b80 1 -- 172.21.15.101:0/3840726385 shutdown_connections 2023-07-18T03:10:42.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.260+0000 7f8b901e7b80 1 --2- 172.21.15.101:0/3840726385 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x556c33001a10 0x556c33003e20 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.260+0000 7f8b901e7b80 1 --2- 172.21.15.101:0/3840726385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c32fef470 0x556c32ffd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.260+0000 7f8b901e7b80 1 --2- 172.21.15.101:0/3840726385 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c32fe62d0 0x556c32fee9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.260+0000 7f8b5dffb700 0 client.26218 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:10:42.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.260+0000 7f8b901e7b80 1 --2- 172.21.15.101:0/3840726385 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c32ff1ad0 0x556c32feef30 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.260+0000 7f8b901e7b80 1 -- 172.21.15.101:0/3840726385 >> 172.21.15.101:0/3840726385 conn(0x556c32fdcda0 msgr2=0x556c32fe2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:42.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.260+0000 7f8b901e7b80 1 -- 172.21.15.101:0/3840726385 shutdown_connections 2023-07-18T03:10:42.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.260+0000 7f8b901e7b80 1 -- 172.21.15.101:0/3840726385 wait complete. 2023-07-18T03:10:42.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.260+0000 7f8b901e7b80 20 client.26218 trim_cache size 0 max 16384 2023-07-18T03:10:42.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.275+0000 7f8b901e7b80 1 Processor -- start 2023-07-18T03:10:42.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.275+0000 7f8b901e7b80 1 -- start start 2023-07-18T03:10:42.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.275+0000 7f8b901e7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c32fe62d0 0x556c32ffd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.275+0000 7f8b901e7b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c32fef470 0x556c32ff84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.275+0000 7f8b901e7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c32ff1ad0 0x556c32febc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.275+0000 7f8b901e7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c32fe7540 con 0x556c32fe62d0 2023-07-18T03:10:42.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.275+0000 7f8b901e7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c32fd9fa0 con 0x556c32ff1ad0 2023-07-18T03:10:42.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.275+0000 7f8b901e7b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c32ffd8e0 con 0x556c32fef470 2023-07-18T03:10:42.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.276+0000 7f8b79d9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c32fef470 0x556c32ff84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.276+0000 7f8b7a59b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c32fe62d0 0x556c32ffd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.276+0000 7f8b79d9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c32fef470 0x556c32ff84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:43304/0 (socket says 172.21.15.101:43304) 2023-07-18T03:10:42.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.276+0000 7f8b79d9a700 1 -- 172.21.15.101:0/2283004435 learned_addr learned my addr 172.21.15.101:0/2283004435 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:42.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.276+0000 7f8b7ad9c700 1 --2- 172.21.15.101:0/2283004435 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c32ff1ad0 0x556c32febc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.276+0000 7f8b79d9a700 1 -- 172.21.15.101:0/2283004435 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c32ff1ad0 msgr2=0x556c32febc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:42.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.276+0000 7f8b79d9a700 1 --2- 172.21.15.101:0/2283004435 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c32ff1ad0 0x556c32febc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.276+0000 7f8b79d9a700 1 -- 172.21.15.101:0/2283004435 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c32fe62d0 msgr2=0x556c32ffd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:42.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.276+0000 7f8b79d9a700 1 --2- 172.21.15.101:0/2283004435 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c32fe62d0 0x556c32ffd3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.276+0000 7f8b79d9a700 1 -- 172.21.15.101:0/2283004435 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c32d4fc60 con 0x556c32fef470 2023-07-18T03:10:42.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.276+0000 7f8b7a59b700 1 --2- 172.21.15.101:0/2283004435 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c32fe62d0 0x556c32ffd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:10:42.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.276+0000 7f8b7ad9c700 1 --2- 172.21.15.101:0/2283004435 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c32ff1ad0 0x556c32febc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:10:42.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.276+0000 7f8b79d9a700 1 --2- 172.21.15.101:0/2283004435 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c32fef470 0x556c32ff84e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8b64011730 tx=0x7f8b64011af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17c8be34195bf114 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:42.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.276+0000 7f8b697fa700 1 -- 172.21.15.101:0/2283004435 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6400f690 con 0x556c32fef470 2023-07-18T03:10:42.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.277+0000 7f8b697fa700 1 -- 172.21.15.101:0/2283004435 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b6400fd90 con 0x556c32fef470 2023-07-18T03:10:42.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.277+0000 7f8b697fa700 1 -- 172.21.15.101:0/2283004435 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b64003680 con 0x556c32fef470 2023-07-18T03:10:42.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.277+0000 7f8b901e7b80 1 -- 172.21.15.101:0/2283004435 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c32fef470 msgr2=0x556c32ff84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:42.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.277+0000 7f8b901e7b80 1 --2- 172.21.15.101:0/2283004435 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c32fef470 0x556c32ff84e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8b64011730 tx=0x7f8b64011af0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.277+0000 7f8b901e7b80 1 -- 172.21.15.101:0/2283004435 shutdown_connections 2023-07-18T03:10:42.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.277+0000 7f8b901e7b80 1 --2- 172.21.15.101:0/2283004435 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c32ff1ad0 0x556c32febc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.277+0000 7f8b901e7b80 1 --2- 172.21.15.101:0/2283004435 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c32fe62d0 0x556c32ffd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.277+0000 7f8b901e7b80 1 --2- 172.21.15.101:0/2283004435 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c32fef470 0x556c32ff84e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.277+0000 7f8b901e7b80 1 -- 172.21.15.101:0/2283004435 >> 172.21.15.101:0/2283004435 conn(0x556c32fdcda0 msgr2=0x556c32ffcc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:42.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.277+0000 7f8b901e7b80 1 -- 172.21.15.101:0/2283004435 shutdown_connections 2023-07-18T03:10:42.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.277+0000 7f8b901e7b80 1 -- 172.21.15.101:0/2283004435 wait complete. 2023-07-18T03:10:42.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.278+0000 7f8b901e7b80 1 -- start start 2023-07-18T03:10:42.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.278+0000 7f8b901e7b80 1 Processor -- start 2023-07-18T03:10:42.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.278+0000 7f8b901e7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c32fe62d0 0x556c32ff7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.278+0000 7f8b901e7b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c32fef470 0x556c32ff83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.278+0000 7f8b901e7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c32ff1ad0 0x556c32ff6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.278+0000 7f8b901e7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c32ff6910 con 0x556c32ff1ad0 2023-07-18T03:10:42.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.278+0000 7f8b901e7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c32ff6a80 con 0x556c32fe62d0 2023-07-18T03:10:42.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.278+0000 7f8b901e7b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c32ff6c40 con 0x556c32fef470 2023-07-18T03:10:42.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.279+0000 7f8b901e7b80 10 client.0 _ll_register_callbacks cb 0x7f8b70005e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:10:42.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.279+0000 7f8b79d9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c32fef470 0x556c32ff83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.279+0000 7f8b79d9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c32fef470 0x556c32ff83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:43314/0 (socket says 172.21.15.101:43314) 2023-07-18T03:10:42.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.279+0000 7f8b79d9a700 1 -- 172.21.15.101:0/1651727997 learned_addr learned my addr 172.21.15.101:0/1651727997 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:42.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.279+0000 7f8b7a59b700 1 --2- 172.21.15.101:0/1651727997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c32fe62d0 0x556c32ff7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.279+0000 7f8b7ad9c700 1 --2- 172.21.15.101:0/1651727997 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c32ff1ad0 0x556c32ff6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.279+0000 7f8b7a59b700 1 -- 172.21.15.101:0/1651727997 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c32fef470 msgr2=0x556c32ff83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:42.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.279+0000 7f8b7a59b700 1 --2- 172.21.15.101:0/1651727997 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c32fef470 0x556c32ff83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.279+0000 7f8b7a59b700 1 -- 172.21.15.101:0/1651727997 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c32ff1ad0 msgr2=0x556c32ff6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:42.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.279+0000 7f8b7a59b700 1 --2- 172.21.15.101:0/1651727997 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c32ff1ad0 0x556c32ff6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.279+0000 7f8b7a59b700 1 -- 172.21.15.101:0/1651727997 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b700095f0 con 0x556c32fe62d0 2023-07-18T03:10:42.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.279+0000 7f8b79d9a700 1 --2- 172.21.15.101:0/1651727997 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c32fef470 0x556c32ff83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:10:42.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.279+0000 7f8b7ad9c700 1 --2- 172.21.15.101:0/1651727997 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c32ff1ad0 0x556c32ff6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:10:42.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.279+0000 7f8b7a59b700 1 --2- 172.21.15.101:0/1651727997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c32fe62d0 0x556c32ff7e80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8b700975c0 tx=0x7f8b70009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=788635a884428014 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:42.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.279+0000 7f8b5dffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:10:42.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.279+0000 7f8b5dffb700 1 -- 172.21.15.101:0/1651727997 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b700a3480 con 0x556c32fe62d0 2023-07-18T03:10:42.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.280+0000 7f8b901e7b80 1 -- 172.21.15.101:0/1651727997 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8b64004830 con 0x556c32fe62d0 2023-07-18T03:10:42.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.280+0000 7f8b5dffb700 1 -- 172.21.15.101:0/1651727997 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f8b7009b320 con 0x556c32fe62d0 2023-07-18T03:10:42.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.280+0000 7f8b5dffb700 1 -- 172.21.15.101:0/1651727997 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b7009b4e0 con 0x556c32fe62d0 2023-07-18T03:10:42.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.280+0000 7f8b901e7b80 10 client.26198 fetch_fsmap learned FSMap version 25 2023-07-18T03:10:42.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.280+0000 7f8b901e7b80 1 -- 172.21.15.101:0/1651727997 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f8b64011440 con 0x556c32fe62d0 2023-07-18T03:10:42.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.280+0000 7f8b5dffb700 1 -- 172.21.15.101:0/1651727997 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b64004830 con 0x556c32fe62d0 2023-07-18T03:10:42.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.280+0000 7f8b5dffb700 1 -- 172.21.15.101:0/1651727997 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f8b700a3060 con 0x556c32fe62d0 2023-07-18T03:10:42.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.280+0000 7f8b901e7b80 10 client.26198 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:10:42.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.280+0000 7f8b901e7b80 10 client.26198 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:10:42.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.280+0000 7f8b901e7b80 20 client.26198 populate_metadata read hostname 'smithi101' 2023-07-18T03:10:42.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.280+0000 7f8b901e7b80 1 --2- 172.21.15.101:0/1651727997 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x556c33056640 0x556c33058a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.280+0000 7f8b901e7b80 4 client.26198 mds_command: new command op to 24541 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:10:42.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.280+0000 7f8b901e7b80 1 -- 172.21.15.101:0/1651727997 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556c32ef4050 con 0x556c33056640 2023-07-18T03:10:42.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.280+0000 7f8b79d9a700 1 --2- 172.21.15.101:0/1651727997 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x556c33056640 0x556c33058a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.281+0000 7f8b5dffb700 10 client.26198 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:10:42.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.281+0000 7f8b79d9a700 1 --2- 172.21.15.101:0/1651727997 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x556c33056640 0x556c33058a20 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bff87f5c2703f858 server_cookie=8a82dbaf8040a8da in_seq=0 out_seq=0 2023-07-18T03:10:42.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.282+0000 7f8b5dffb700 1 -- 172.21.15.101:0/1651727997 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556c32ef4050 con 0x556c33056640 2023-07-18T03:10:42.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.282+0000 7f8b5dffb700 10 client.26198 handle_command_reply: tid=0 2023-07-18T03:10:42.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.282+0000 7f8b901e7b80 1 client.26198 shutdown 2023-07-18T03:10:42.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.282+0000 7f8b901e7b80 1 -- 172.21.15.101:0/1651727997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c32fe62d0 msgr2=0x556c32ff7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:42.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.282+0000 7f8b901e7b80 1 --2- 172.21.15.101:0/1651727997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c32fe62d0 0x556c32ff7e80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8b700975c0 tx=0x7f8b70009310 comp rx=0 tx=0).stop 2023-07-18T03:10:42.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.282+0000 7f8b901e7b80 1 -- 172.21.15.101:0/1651727997 shutdown_connections 2023-07-18T03:10:42.284 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.282+0000 7f8b901e7b80 1 --2- 172.21.15.101:0/1651727997 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x556c33056640 0x556c33058a20 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.284 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.282+0000 7f8b901e7b80 1 --2- 172.21.15.101:0/1651727997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c32fe62d0 0x556c32ff7e80 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.284 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.282+0000 7f8b901e7b80 1 --2- 172.21.15.101:0/1651727997 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c32ff1ad0 0x556c32ff6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.284 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.282+0000 7f8b901e7b80 1 --2- 172.21.15.101:0/1651727997 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c32fef470 0x556c32ff83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.282+0000 7f8b5dffb700 0 client.26198 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:10:42.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.282+0000 7f8b901e7b80 1 -- 172.21.15.101:0/1651727997 >> 172.21.15.101:0/1651727997 conn(0x556c32fdcda0 msgr2=0x556c32fe83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:42.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.282+0000 7f8b901e7b80 1 -- 172.21.15.101:0/1651727997 shutdown_connections 2023-07-18T03:10:42.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.282+0000 7f8b901e7b80 1 -- 172.21.15.101:0/1651727997 wait complete. 2023-07-18T03:10:42.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.282+0000 7f8b901e7b80 20 client.26198 trim_cache size 0 max 16384 2023-07-18T03:10:42.286 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:10:42.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.283+0000 7f8b7c7ff700 1 -- 172.21.15.101:0/2502298184 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f8b600a1340 msgr2=0x7f8b600a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:42.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.283+0000 7f8b7c7ff700 1 --2- 172.21.15.101:0/2502298184 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f8b600a1340 0x7f8b600a37f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c009fd0 tx=0x7f8b6c009490 comp rx=0 tx=0).stop 2023-07-18T03:10:42.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.283+0000 7f8b7c7ff700 1 -- 172.21.15.101:0/2502298184 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b740ffae0 msgr2=0x7f8b74197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:42.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.283+0000 7f8b7c7ff700 1 --2- 172.21.15.101:0/2502298184 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b740ffae0 0x7f8b74197de0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8b70009990 tx=0x7f8b70009580 comp rx=0 tx=0).stop 2023-07-18T03:10:42.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.283+0000 7f8b7c7ff700 1 -- 172.21.15.101:0/2502298184 shutdown_connections 2023-07-18T03:10:42.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.283+0000 7f8b7c7ff700 1 --2- 172.21.15.101:0/2502298184 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f8b600a1340 0x7f8b600a37f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.283+0000 7f8b7c7ff700 1 --2- 172.21.15.101:0/2502298184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b74105af0 0x7f8b7419c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.283+0000 7f8b7c7ff700 1 --2- 172.21.15.101:0/2502298184 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b740ffae0 0x7f8b74197de0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.283+0000 7f8b7c7ff700 1 --2- 172.21.15.101:0/2502298184 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8b74100490 0x7f8b74198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.283+0000 7f8b7c7ff700 1 -- 172.21.15.101:0/2502298184 >> 172.21.15.101:0/2502298184 conn(0x7f8b740fb5e0 msgr2=0x7f8b7410ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:42.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.283+0000 7f8b7c7ff700 1 -- 172.21.15.101:0/2502298184 shutdown_connections 2023-07-18T03:10:42.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.283+0000 7f8b7c7ff700 1 -- 172.21.15.101:0/2502298184 wait complete. 2023-07-18T03:10:42.290 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:42 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2807344120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:10:42.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:42 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/2807344120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:10:42.293 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-18T03:10:42.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:42 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/2807344120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:10:42.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.512+0000 7f87661ff700 1 -- 172.21.15.101:0/4119961645 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8760105af0 msgr2=0x7f8760105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:42.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.513+0000 7f87661ff700 1 --2- 172.21.15.101:0/4119961645 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8760105af0 0x7f8760105ec0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f8750009990 tx=0x7f8750009ca0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.513+0000 7f87661ff700 1 -- 172.21.15.101:0/4119961645 shutdown_connections 2023-07-18T03:10:42.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.513+0000 7f87661ff700 1 --2- 172.21.15.101:0/4119961645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8760100490 0x7f876010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.513+0000 7f87661ff700 1 --2- 172.21.15.101:0/4119961645 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8760105af0 0x7f8760105ec0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.513+0000 7f87661ff700 1 --2- 172.21.15.101:0/4119961645 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87600ffae0 0x7f87600fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.513+0000 7f87661ff700 1 -- 172.21.15.101:0/4119961645 >> 172.21.15.101:0/4119961645 conn(0x7f87600fb5e0 msgr2=0x7f87600fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:42.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.513+0000 7f87661ff700 1 -- 172.21.15.101:0/4119961645 shutdown_connections 2023-07-18T03:10:42.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.513+0000 7f87661ff700 1 -- 172.21.15.101:0/4119961645 wait complete. 2023-07-18T03:10:42.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.514+0000 7f87661ff700 1 Processor -- start 2023-07-18T03:10:42.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.514+0000 7f87661ff700 1 -- start start 2023-07-18T03:10:42.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.514+0000 7f87661ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87600ffae0 0x7f876019b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.514+0000 7f87661ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8760100490 0x7f876019beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.514+0000 7f87661ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8760105af0 0x7f87601959e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.514+0000 7f87661ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8760196050 con 0x7f8760105af0 2023-07-18T03:10:42.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.514+0000 7f87661ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87601961c0 con 0x7f87600ffae0 2023-07-18T03:10:42.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.514+0000 7f87661ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8760196380 con 0x7f8760100490 2023-07-18T03:10:42.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.515+0000 7f875effd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8760100490 0x7f876019beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.515+0000 7f875effd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8760100490 0x7f876019beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:43328/0 (socket says 172.21.15.101:43328) 2023-07-18T03:10:42.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.515+0000 7f875effd700 1 -- 172.21.15.101:0/1910714057 learned_addr learned my addr 172.21.15.101:0/1910714057 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:42.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.515+0000 7f875f7fe700 1 --2- 172.21.15.101:0/1910714057 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87600ffae0 0x7f876019b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.515+0000 7f875ffff700 1 --2- 172.21.15.101:0/1910714057 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8760105af0 0x7f87601959e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.515+0000 7f875effd700 1 -- 172.21.15.101:0/1910714057 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87600ffae0 msgr2=0x7f876019b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:42.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.515+0000 7f875effd700 1 --2- 172.21.15.101:0/1910714057 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87600ffae0 0x7f876019b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.515+0000 7f875effd700 1 -- 172.21.15.101:0/1910714057 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8760105af0 msgr2=0x7f87601959e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:42.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.515+0000 7f875effd700 1 --2- 172.21.15.101:0/1910714057 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8760105af0 0x7f87601959e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.515+0000 7f875effd700 1 -- 172.21.15.101:0/1910714057 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87500095f0 con 0x7f8760100490 2023-07-18T03:10:42.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.515+0000 7f875ffff700 1 --2- 172.21.15.101:0/1910714057 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8760105af0 0x7f87601959e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:10:42.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.515+0000 7f875effd700 1 --2- 172.21.15.101:0/1910714057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8760100490 0x7f876019beb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f875400e9a0 tx=0x7f875400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:42.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.516+0000 7f875cff9700 1 -- 172.21.15.101:0/1910714057 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8754009790 con 0x7f8760100490 2023-07-18T03:10:42.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.516+0000 7f87661ff700 1 -- 172.21.15.101:0/1910714057 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8760196660 con 0x7f8760100490 2023-07-18T03:10:42.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.516+0000 7f875cff9700 1 -- 172.21.15.101:0/1910714057 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f875400cbd0 con 0x7f8760100490 2023-07-18T03:10:42.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.516+0000 7f875cff9700 1 -- 172.21.15.101:0/1910714057 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8754018680 con 0x7f8760100490 2023-07-18T03:10:42.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.516+0000 7f87661ff700 1 -- 172.21.15.101:0/1910714057 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8760072920 con 0x7f8760100490 2023-07-18T03:10:42.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.522+0000 7f8779b07b80 1 Processor -- start 2023-07-18T03:10:42.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.522+0000 7f875cff9700 1 -- 172.21.15.101:0/1910714057 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f875400cd40 con 0x7f8760100490 2023-07-18T03:10:42.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.524+0000 7f8779b07b80 1 -- start start 2023-07-18T03:10:42.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.524+0000 7f8779b07b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf6479eae0 0x55cf6479eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.524+0000 7f8779b07b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cf6479f3f0 0x55cf647a9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.524+0000 7f8779b07b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf647aa2e0 0x55cf647ac6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.524+0000 7f8779b07b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf6469ccb0 con 0x55cf6479f3f0 2023-07-18T03:10:42.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.524+0000 7f8779b07b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf646efa40 con 0x55cf647aa2e0 2023-07-18T03:10:42.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.524+0000 7f8779b07b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf646e8ed0 con 0x55cf6479eae0 2023-07-18T03:10:42.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.525+0000 7f875cff9700 1 --2- 172.21.15.101:0/1910714057 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f87480a1340 0x7f87480a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.525+0000 7f875effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf647aa2e0 0x55cf647ac6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.525+0000 7f875effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf647aa2e0 0x55cf647ac6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:34088/0 (socket says 172.21.15.101:34088) 2023-07-18T03:10:42.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.525+0000 7f875effd700 1 -- 172.21.15.101:0/1762305282 learned_addr learned my addr 172.21.15.101:0/1762305282 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:42.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.525+0000 7f875cff9700 1 -- 172.21.15.101:0/1910714057 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f8754014070 con 0x7f8760100490 2023-07-18T03:10:42.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.526+0000 7f875effd700 1 -- 172.21.15.101:0/1762305282 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf6479eae0 msgr2=0x55cf6479eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:42.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.526+0000 7f875effd700 1 --2- 172.21.15.101:0/1762305282 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf6479eae0 0x55cf6479eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.526+0000 7f875effd700 1 -- 172.21.15.101:0/1762305282 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cf6479f3f0 msgr2=0x55cf647a9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:42.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.526+0000 7f875effd700 1 --2- 172.21.15.101:0/1762305282 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cf6479f3f0 0x55cf647a9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.526+0000 7f875effd700 1 -- 172.21.15.101:0/1762305282 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf64516de0 con 0x55cf647aa2e0 2023-07-18T03:10:42.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.529+0000 7f875effd700 1 --2- 172.21.15.101:0/1762305282 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf647aa2e0 0x55cf647ac6c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f875407a860 tx=0x7f875406c0b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccd7c927a252be86 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:42.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.531+0000 7f875ffff700 1 --2- 172.21.15.101:0/1910714057 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f87480a1340 0x7f87480a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.531+0000 7f8745ffb700 1 -- 172.21.15.101:0/1762305282 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875409a040 con 0x55cf647aa2e0 2023-07-18T03:10:42.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.531+0000 7f8745ffb700 1 -- 172.21.15.101:0/1762305282 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f875409e070 con 0x55cf647aa2e0 2023-07-18T03:10:42.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.531+0000 7f8745ffb700 1 -- 172.21.15.101:0/1762305282 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8754065760 con 0x55cf647aa2e0 2023-07-18T03:10:42.524 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.532+0000 7f8779b07b80 1 -- 172.21.15.101:0/1762305282 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf647aa2e0 msgr2=0x55cf647ac6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:42.524 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.532+0000 7f8779b07b80 1 --2- 172.21.15.101:0/1762305282 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf647aa2e0 0x55cf647ac6c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f875407a860 tx=0x7f875406c0b0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.540+0000 7f8779b07b80 1 -- 172.21.15.101:0/1762305282 shutdown_connections 2023-07-18T03:10:42.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.540+0000 7f8779b07b80 1 --2- 172.21.15.101:0/1762305282 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf647aa2e0 0x55cf647ac6c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.540+0000 7f8779b07b80 1 --2- 172.21.15.101:0/1762305282 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf6479eae0 0x55cf6479eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.540+0000 7f8779b07b80 1 --2- 172.21.15.101:0/1762305282 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cf6479f3f0 0x55cf647a9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.540+0000 7f8779b07b80 1 -- 172.21.15.101:0/1762305282 >> 172.21.15.101:0/1762305282 conn(0x55cf6478cda0 msgr2=0x55cf647926a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:42.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.549+0000 7f875ffff700 1 --2- 172.21.15.101:0/1910714057 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f87480a1340 0x7f87480a37f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f876019c670 tx=0x7f8758009920 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:42.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.553+0000 7f8779b07b80 1 -- 172.21.15.101:0/1762305282 shutdown_connections 2023-07-18T03:10:42.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.555+0000 7f8779b07b80 1 -- 172.21.15.101:0/1762305282 wait complete. 2023-07-18T03:10:42.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.556+0000 7f8779b07b80 1 -- start start 2023-07-18T03:10:42.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.556+0000 7f8779b07b80 1 Processor -- start 2023-07-18T03:10:42.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.558+0000 7f8779b07b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf6479eae0 0x55cf647a70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.558+0000 7f8779b07b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf6479f3f0 0x55cf647a7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.558+0000 7f8779b07b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cf647aa2e0 0x55cf647a7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.558+0000 7f8779b07b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf64688ac0 con 0x55cf647aa2e0 2023-07-18T03:10:42.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.558+0000 7f8779b07b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf646887c0 con 0x55cf6479eae0 2023-07-18T03:10:42.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.558+0000 7f8779b07b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf6456c920 con 0x55cf6479f3f0 2023-07-18T03:10:42.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.558+0000 7f8779b07b80 10 client.0 _ll_register_callbacks cb 0x55cf64686760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:10:42.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.559+0000 7f875effd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf6479f3f0 0x55cf647a7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.559+0000 7f875f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf6479eae0 0x55cf647a70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.559+0000 7f875f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf6479eae0 0x55cf647a70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:34092/0 (socket says 172.21.15.101:34092) 2023-07-18T03:10:42.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.559+0000 7f875f7fe700 1 -- 172.21.15.101:0/1238035641 learned_addr learned my addr 172.21.15.101:0/1238035641 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:42.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.560+0000 7f875f7fe700 1 -- 172.21.15.101:0/1238035641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf6479f3f0 msgr2=0x55cf647a7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:42.555 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.560+0000 7f875f7fe700 1 --2- 172.21.15.101:0/1238035641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf6479f3f0 0x55cf647a7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.555 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.560+0000 7f875f7fe700 1 -- 172.21.15.101:0/1238035641 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cf647aa2e0 msgr2=0x55cf647a7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:42.555 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.560+0000 7f875f7fe700 1 --2- 172.21.15.101:0/1238035641 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cf647aa2e0 0x55cf647a7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.556 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.560+0000 7f875f7fe700 1 -- 172.21.15.101:0/1238035641 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8760072920 con 0x55cf6479eae0 2023-07-18T03:10:42.556 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.561+0000 7f875f7fe700 1 --2- 172.21.15.101:0/1238035641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf6479eae0 0x55cf647a70c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8750009990 tx=0x7f875000bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c16477acd04b3439 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:42.556 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.561+0000 7f873effd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:10:42.556 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.561+0000 7f873effd700 1 -- 172.21.15.101:0/1238035641 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875001d070 con 0x55cf6479eae0 2023-07-18T03:10:42.557 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.561+0000 7f873effd700 1 -- 172.21.15.101:0/1238035641 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f8750004500 con 0x55cf6479eae0 2023-07-18T03:10:42.557 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.561+0000 7f8779b07b80 1 -- 172.21.15.101:0/1238035641 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cf646424f0 con 0x55cf6479eae0 2023-07-18T03:10:42.557 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.563+0000 7f873effd700 1 -- 172.21.15.101:0/1238035641 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87500046c0 con 0x55cf6479eae0 2023-07-18T03:10:42.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.563+0000 7f8779b07b80 10 client.26210 fetch_fsmap learned FSMap version 25 2023-07-18T03:10:42.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.563+0000 7f8779b07b80 1 -- 172.21.15.101:0/1238035641 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55cf644ffc60 con 0x55cf6479eae0 2023-07-18T03:10:42.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.564+0000 7f873effd700 1 -- 172.21.15.101:0/1238035641 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cf646424f0 con 0x55cf6479eae0 2023-07-18T03:10:42.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.564+0000 7f873effd700 1 -- 172.21.15.101:0/1238035641 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f875000f820 con 0x55cf6479eae0 2023-07-18T03:10:42.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.564+0000 7f8779b07b80 10 client.26210 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:10:42.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.564+0000 7f8779b07b80 10 client.26210 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:10:42.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.564+0000 7f8779b07b80 20 client.26210 populate_metadata read hostname 'smithi101' 2023-07-18T03:10:42.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.564+0000 7f8779b07b80 1 --2- 172.21.15.101:0/1238035641 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55cf64798140 0x55cf6479a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.565+0000 7f8779b07b80 4 client.26210 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:10:42.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.565+0000 7f8779b07b80 1 -- 172.21.15.101:0/1238035641 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cf646e1eb0 con 0x55cf64798140 2023-07-18T03:10:42.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.565+0000 7f875effd700 1 --2- 172.21.15.101:0/1238035641 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55cf64798140 0x55cf6479a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.567+0000 7f875effd700 1 --2- 172.21.15.101:0/1238035641 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55cf64798140 0x55cf6479a550 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=37aab4b871d3633b server_cookie=45a3aba5e2269c5e in_seq=0 out_seq=0 2023-07-18T03:10:42.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.568+0000 7f873effd700 10 client.26210 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:10:42.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.569+0000 7f873effd700 1 -- 172.21.15.101:0/1238035641 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cf646e1eb0 con 0x55cf64798140 2023-07-18T03:10:42.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.569+0000 7f873effd700 10 client.26210 handle_command_reply: tid=0 2023-07-18T03:10:42.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.569+0000 7f8779b07b80 1 client.26210 shutdown 2023-07-18T03:10:42.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.569+0000 7f8779b07b80 1 -- 172.21.15.101:0/1238035641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf6479eae0 msgr2=0x55cf647a70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:42.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.569+0000 7f8779b07b80 1 --2- 172.21.15.101:0/1238035641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf6479eae0 0x55cf647a70c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8750009990 tx=0x7f875000bfd0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.562 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.570+0000 7f8779b07b80 1 -- 172.21.15.101:0/1238035641 shutdown_connections 2023-07-18T03:10:42.562 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.570+0000 7f8779b07b80 1 --2- 172.21.15.101:0/1238035641 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55cf64798140 0x55cf6479a550 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.570+0000 7f8779b07b80 1 --2- 172.21.15.101:0/1238035641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf6479eae0 0x55cf647a70c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.570+0000 7f8779b07b80 1 --2- 172.21.15.101:0/1238035641 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cf647aa2e0 0x55cf647a7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.570+0000 7f8779b07b80 1 --2- 172.21.15.101:0/1238035641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf6479f3f0 0x55cf647a7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.564 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.570+0000 7f8779b07b80 1 -- 172.21.15.101:0/1238035641 >> 172.21.15.101:0/1238035641 conn(0x55cf6478cda0 msgr2=0x55cf64792290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:42.564 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.570+0000 7f873effd700 0 client.26210 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:10:42.564 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.570+0000 7f8779b07b80 1 -- 172.21.15.101:0/1238035641 shutdown_connections 2023-07-18T03:10:42.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.570+0000 7f8779b07b80 1 -- 172.21.15.101:0/1238035641 wait complete. 2023-07-18T03:10:42.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.571+0000 7f8779b07b80 20 client.26210 trim_cache size 0 max 16384 2023-07-18T03:10:42.571 INFO:tasks.workunit.client.0.smithi101.stdout: 1 192019 8.43 MB/sec execute 7 sec latency 64.897 ms 2023-07-18T03:10:42.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.586+0000 7f8779b07b80 1 Processor -- start 2023-07-18T03:10:42.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.586+0000 7f8779b07b80 1 -- start start 2023-07-18T03:10:42.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.586+0000 7f8779b07b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf6479eae0 0x55cf647c1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.586+0000 7f8779b07b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cf6479f3f0 0x55cf647a79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.586+0000 7f8779b07b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf647aa2e0 0x55cf647a2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.586+0000 7f8779b07b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf6479a8a0 con 0x55cf6479f3f0 2023-07-18T03:10:42.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.586+0000 7f8779b07b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf647c2170 con 0x55cf647aa2e0 2023-07-18T03:10:42.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.586+0000 7f8779b07b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf6479f100 con 0x55cf6479eae0 2023-07-18T03:10:42.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.587+0000 7f875effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cf6479f3f0 0x55cf647a79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.587+0000 7f875ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf647aa2e0 0x55cf647a2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.587+0000 7f875f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf6479eae0 0x55cf647c1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.587+0000 7f875ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf647aa2e0 0x55cf647a2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:34094/0 (socket says 172.21.15.101:34094) 2023-07-18T03:10:42.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.587+0000 7f875f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf6479eae0 0x55cf647c1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:43360/0 (socket says 172.21.15.101:43360) 2023-07-18T03:10:42.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.587+0000 7f875ffff700 1 -- 172.21.15.101:0/252334125 learned_addr learned my addr 172.21.15.101:0/252334125 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:42.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.587+0000 7f875f7fe700 1 -- 172.21.15.101:0/252334125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf647aa2e0 msgr2=0x55cf647a2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:42.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.587+0000 7f875f7fe700 1 --2- 172.21.15.101:0/252334125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf647aa2e0 0x55cf647a2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.587+0000 7f875f7fe700 1 -- 172.21.15.101:0/252334125 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cf6479f3f0 msgr2=0x55cf647a79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:42.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.587+0000 7f875f7fe700 1 --2- 172.21.15.101:0/252334125 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cf6479f3f0 0x55cf647a79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.587+0000 7f875f7fe700 1 -- 172.21.15.101:0/252334125 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf6453d250 con 0x55cf6479eae0 2023-07-18T03:10:42.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.587+0000 7f875effd700 1 --2- 172.21.15.101:0/252334125 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cf6479f3f0 0x55cf647a79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:10:42.581 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.587+0000 7f875f7fe700 1 --2- 172.21.15.101:0/252334125 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf6479eae0 0x55cf647c1c30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8750009ce0 tx=0x7f8750005c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dff50cd9226c7996 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:42.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.587+0000 7f87467fc700 1 -- 172.21.15.101:0/252334125 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875001d070 con 0x55cf6479eae0 2023-07-18T03:10:42.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.587+0000 7f87467fc700 1 -- 172.21.15.101:0/252334125 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f875001e040 con 0x55cf6479eae0 2023-07-18T03:10:42.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.587+0000 7f87467fc700 1 -- 172.21.15.101:0/252334125 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875000fea0 con 0x55cf6479eae0 2023-07-18T03:10:42.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.588+0000 7f8779b07b80 1 -- 172.21.15.101:0/252334125 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf6479eae0 msgr2=0x55cf647c1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:42.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.588+0000 7f8779b07b80 1 --2- 172.21.15.101:0/252334125 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf6479eae0 0x55cf647c1c30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8750009ce0 tx=0x7f8750005c70 comp rx=0 tx=0).stop 2023-07-18T03:10:42.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.588+0000 7f8779b07b80 1 -- 172.21.15.101:0/252334125 shutdown_connections 2023-07-18T03:10:42.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.588+0000 7f8779b07b80 1 --2- 172.21.15.101:0/252334125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf647aa2e0 0x55cf647a2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.588+0000 7f8779b07b80 1 --2- 172.21.15.101:0/252334125 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf6479eae0 0x55cf647c1c30 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.588+0000 7f8779b07b80 1 --2- 172.21.15.101:0/252334125 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cf6479f3f0 0x55cf647a79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.589 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.588+0000 7f8779b07b80 1 -- 172.21.15.101:0/252334125 >> 172.21.15.101:0/252334125 conn(0x55cf6478cda0 msgr2=0x55cf6479a150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:42.589 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.588+0000 7f8779b07b80 1 -- 172.21.15.101:0/252334125 shutdown_connections 2023-07-18T03:10:42.589 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.588+0000 7f8779b07b80 1 -- 172.21.15.101:0/252334125 wait complete. 2023-07-18T03:10:42.589 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.589+0000 7f8779b07b80 1 -- start start 2023-07-18T03:10:42.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.589+0000 7f8779b07b80 1 Processor -- start 2023-07-18T03:10:42.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.589+0000 7f8779b07b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf6479eae0 0x55cf647a4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.589+0000 7f8779b07b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf6479f3f0 0x55cf647a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.590+0000 7f8779b07b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cf647aa2e0 0x55cf647a3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.590+0000 7f8779b07b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf647a38d0 con 0x55cf647aa2e0 2023-07-18T03:10:42.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.590+0000 7f8779b07b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf647a3a40 con 0x55cf6479f3f0 2023-07-18T03:10:42.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.590+0000 7f8779b07b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf647a3c00 con 0x55cf6479eae0 2023-07-18T03:10:42.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.590+0000 7f8779b07b80 10 client.0 _ll_register_callbacks cb 0x7f8750009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:10:42.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.590+0000 7f875f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf6479eae0 0x55cf647a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.590+0000 7f875f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf6479eae0 0x55cf647a4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:43376/0 (socket says 172.21.15.101:43376) 2023-07-18T03:10:42.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.590+0000 7f875f7fe700 1 -- 172.21.15.101:0/141350525 learned_addr learned my addr 172.21.15.101:0/141350525 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:42.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.590+0000 7f875ffff700 1 --2- 172.21.15.101:0/141350525 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cf647aa2e0 0x55cf647a3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.590+0000 7f875effd700 1 --2- 172.21.15.101:0/141350525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf6479f3f0 0x55cf647a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.590+0000 7f875f7fe700 1 -- 172.21.15.101:0/141350525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf6479f3f0 msgr2=0x55cf647a4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:42.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.590+0000 7f875f7fe700 1 --2- 172.21.15.101:0/141350525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf6479f3f0 0x55cf647a4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.590+0000 7f875f7fe700 1 -- 172.21.15.101:0/141350525 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cf647aa2e0 msgr2=0x55cf647a3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:42.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.590+0000 7f875f7fe700 1 --2- 172.21.15.101:0/141350525 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cf647aa2e0 0x55cf647a3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.590+0000 7f875f7fe700 1 -- 172.21.15.101:0/141350525 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cf64516de0 con 0x55cf6479eae0 2023-07-18T03:10:42.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.591+0000 7f875f7fe700 1 --2- 172.21.15.101:0/141350525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf6479eae0 0x55cf647a4720 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f875000bba0 tx=0x7f875000ff30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8a81fe568ef7a09 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:42.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.591+0000 7f873effd700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:10:42.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.591+0000 7f873effd700 1 -- 172.21.15.101:0/141350525 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875001d070 con 0x55cf6479eae0 2023-07-18T03:10:42.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.591+0000 7f8779b07b80 1 -- 172.21.15.101:0/141350525 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f875409e070 con 0x55cf6479eae0 2023-07-18T03:10:42.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.591+0000 7f873effd700 1 -- 172.21.15.101:0/141350525 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f8750000e40 con 0x55cf6479eae0 2023-07-18T03:10:42.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.591+0000 7f873effd700 1 -- 172.21.15.101:0/141350525 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f875000f870 con 0x55cf6479eae0 2023-07-18T03:10:42.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.591+0000 7f8779b07b80 10 client.26239 fetch_fsmap learned FSMap version 25 2023-07-18T03:10:42.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.591+0000 7f8779b07b80 1 -- 172.21.15.101:0/141350525 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55cf644ffc60 con 0x55cf6479eae0 2023-07-18T03:10:42.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.591+0000 7f873effd700 1 -- 172.21.15.101:0/141350525 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f875409e070 con 0x55cf6479eae0 2023-07-18T03:10:42.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.592+0000 7f873effd700 1 -- 172.21.15.101:0/141350525 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f8750022510 con 0x55cf6479eae0 2023-07-18T03:10:42.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.592+0000 7f8779b07b80 10 client.26239 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:10:42.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.592+0000 7f8779b07b80 10 client.26239 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:10:42.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.592+0000 7f8779b07b80 20 client.26239 populate_metadata read hostname 'smithi101' 2023-07-18T03:10:42.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.592+0000 7f8779b07b80 1 --2- 172.21.15.101:0/141350525 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55cf648278c0 0x55cf64829ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.592+0000 7f8779b07b80 4 client.26239 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:10:42.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.592+0000 7f8779b07b80 1 -- 172.21.15.101:0/141350525 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cf646a4050 con 0x55cf648278c0 2023-07-18T03:10:42.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.592+0000 7f875effd700 1 --2- 172.21.15.101:0/141350525 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55cf648278c0 0x55cf64829ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.592+0000 7f875effd700 1 --2- 172.21.15.101:0/141350525 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55cf648278c0 0x55cf64829ca0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=abc04f150e11586f server_cookie=b86c39bd30d773a4 in_seq=0 out_seq=0 2023-07-18T03:10:42.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.592+0000 7f873effd700 10 client.26239 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:10:42.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.593+0000 7f873effd700 1 -- 172.21.15.101:0/141350525 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cf646a4050 con 0x55cf648278c0 2023-07-18T03:10:42.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.593+0000 7f873effd700 10 client.26239 handle_command_reply: tid=0 2023-07-18T03:10:42.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.593+0000 7f8779b07b80 1 client.26239 shutdown 2023-07-18T03:10:42.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.593+0000 7f8779b07b80 1 -- 172.21.15.101:0/141350525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf6479eae0 msgr2=0x55cf647a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:42.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.593+0000 7f8779b07b80 1 --2- 172.21.15.101:0/141350525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf6479eae0 0x55cf647a4720 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f875000bba0 tx=0x7f875000ff30 comp rx=0 tx=0).stop 2023-07-18T03:10:42.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.593+0000 7f8779b07b80 1 -- 172.21.15.101:0/141350525 shutdown_connections 2023-07-18T03:10:42.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.593+0000 7f8779b07b80 1 --2- 172.21.15.101:0/141350525 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55cf648278c0 0x55cf64829ca0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.593+0000 7f8779b07b80 1 --2- 172.21.15.101:0/141350525 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cf647aa2e0 0x55cf647a3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.593+0000 7f8779b07b80 1 --2- 172.21.15.101:0/141350525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf6479eae0 0x55cf647a4720 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.593+0000 7f873effd700 0 client.26239 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:10:42.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.593+0000 7f8779b07b80 1 --2- 172.21.15.101:0/141350525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf6479f3f0 0x55cf647a4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.593+0000 7f8779b07b80 1 -- 172.21.15.101:0/141350525 >> 172.21.15.101:0/141350525 conn(0x55cf6478cda0 msgr2=0x55cf647a2440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:42.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.593+0000 7f8779b07b80 1 -- 172.21.15.101:0/141350525 shutdown_connections 2023-07-18T03:10:42.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.593+0000 7f8779b07b80 1 -- 172.21.15.101:0/141350525 wait complete. 2023-07-18T03:10:42.604 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.593+0000 7f8779b07b80 20 client.26239 trim_cache size 0 max 16384 2023-07-18T03:10:42.604 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:10:42.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.594+0000 7f87661ff700 1 -- 172.21.15.101:0/1910714057 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f87480a1340 msgr2=0x7f87480a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:42.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.594+0000 7f87661ff700 1 --2- 172.21.15.101:0/1910714057 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f87480a1340 0x7f87480a37f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f876019c670 tx=0x7f8758009920 comp rx=0 tx=0).stop 2023-07-18T03:10:42.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.594+0000 7f87661ff700 1 -- 172.21.15.101:0/1910714057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8760100490 msgr2=0x7f876019beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:42.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.594+0000 7f87661ff700 1 --2- 172.21.15.101:0/1910714057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8760100490 0x7f876019beb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f875400e9a0 tx=0x7f875400ecb0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.594+0000 7f87661ff700 1 -- 172.21.15.101:0/1910714057 shutdown_connections 2023-07-18T03:10:42.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.594+0000 7f87661ff700 1 --2- 172.21.15.101:0/1910714057 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f87480a1340 0x7f87480a37f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.594+0000 7f87661ff700 1 --2- 172.21.15.101:0/1910714057 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87600ffae0 0x7f876019b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.594+0000 7f87661ff700 1 --2- 172.21.15.101:0/1910714057 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8760105af0 0x7f87601959e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.594+0000 7f87661ff700 1 --2- 172.21.15.101:0/1910714057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8760100490 0x7f876019beb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.594+0000 7f87661ff700 1 -- 172.21.15.101:0/1910714057 >> 172.21.15.101:0/1910714057 conn(0x7f87600fb5e0 msgr2=0x7f87600fbc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:42.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.594+0000 7f87661ff700 1 -- 172.21.15.101:0/1910714057 shutdown_connections 2023-07-18T03:10:42.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.594+0000 7f87661ff700 1 -- 172.21.15.101:0/1910714057 wait complete. 2023-07-18T03:10:42.608 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-18T03:10:42.824 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.832+0000 7f289b39d700 1 -- 172.21.15.101:0/855767213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28940ff9d0 msgr2=0x7f28940ffe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:42.824 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.832+0000 7f289b39d700 1 --2- 172.21.15.101:0/855767213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28940ff9d0 0x7f28940ffe40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f28840098f0 tx=0x7f2884009c00 comp rx=0 tx=0).stop 2023-07-18T03:10:42.824 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.832+0000 7f289b39d700 1 -- 172.21.15.101:0/855767213 shutdown_connections 2023-07-18T03:10:42.824 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.832+0000 7f289b39d700 1 --2- 172.21.15.101:0/855767213 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2894100380 0x7f289410c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.824 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.833+0000 7f289b39d700 1 --2- 172.21.15.101:0/855767213 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f28941059e0 0x7f2894105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.833+0000 7f289b39d700 1 --2- 172.21.15.101:0/855767213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28940ff9d0 0x7f28940ffe40 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.833+0000 7f289b39d700 1 -- 172.21.15.101:0/855767213 >> 172.21.15.101:0/855767213 conn(0x7f28940fb4b0 msgr2=0x7f28940fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:42.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.833+0000 7f289b39d700 1 -- 172.21.15.101:0/855767213 shutdown_connections 2023-07-18T03:10:42.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.833+0000 7f289b39d700 1 -- 172.21.15.101:0/855767213 wait complete. 2023-07-18T03:10:42.826 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.834+0000 7f289b39d700 1 Processor -- start 2023-07-18T03:10:42.826 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.834+0000 7f289b39d700 1 -- start start 2023-07-18T03:10:42.826 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.835+0000 7f289b39d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28940ff9d0 0x7f289419b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.826 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.835+0000 7f289b39d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2894100380 0x7f289419bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.835+0000 7f289b39d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f28941059e0 0x7f28941958d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.835+0000 7f289b39d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2894195f30 con 0x7f28941059e0 2023-07-18T03:10:42.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.835+0000 7f289b39d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28941960a0 con 0x7f2894100380 2023-07-18T03:10:42.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.835+0000 7f289b39d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2894196260 con 0x7f28940ff9d0 2023-07-18T03:10:42.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.835+0000 7f289a39b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28940ff9d0 0x7f289419b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.835+0000 7f289a39b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28940ff9d0 0x7f289419b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:43384/0 (socket says 172.21.15.101:43384) 2023-07-18T03:10:42.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.835+0000 7f289a39b700 1 -- 172.21.15.101:0/363308484 learned_addr learned my addr 172.21.15.101:0/363308484 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:42.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.835+0000 7f2899b9a700 1 --2- 172.21.15.101:0/363308484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2894100380 0x7f289419bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.835+0000 7f289ab9c700 1 --2- 172.21.15.101:0/363308484 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f28941059e0 0x7f28941958d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.835+0000 7f289a39b700 1 -- 172.21.15.101:0/363308484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2894100380 msgr2=0x7f289419bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:42.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.835+0000 7f289a39b700 1 --2- 172.21.15.101:0/363308484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2894100380 0x7f289419bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.835+0000 7f289a39b700 1 -- 172.21.15.101:0/363308484 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f28941059e0 msgr2=0x7f28941958d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:42.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.835+0000 7f289a39b700 1 --2- 172.21.15.101:0/363308484 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f28941059e0 0x7f28941958d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.835+0000 7f289a39b700 1 -- 172.21.15.101:0/363308484 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f288c009d00 con 0x7f28940ff9d0 2023-07-18T03:10:42.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.836+0000 7f2899b9a700 1 --2- 172.21.15.101:0/363308484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2894100380 0x7f289419bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:10:42.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.836+0000 7f289ab9c700 1 --2- 172.21.15.101:0/363308484 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f28941059e0 0x7f28941958d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:10:42.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.836+0000 7f289a39b700 1 --2- 172.21.15.101:0/363308484 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28940ff9d0 0x7f289419b860 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f289000e970 tx=0x7f289000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:42.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.836+0000 7f288b7fe700 1 -- 172.21.15.101:0/363308484 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2890009790 con 0x7f28940ff9d0 2023-07-18T03:10:42.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.836+0000 7f288b7fe700 1 -- 172.21.15.101:0/363308484 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f289000eec0 con 0x7f28940ff9d0 2023-07-18T03:10:42.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.836+0000 7f289b39d700 1 -- 172.21.15.101:0/363308484 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2884009520 con 0x7f28940ff9d0 2023-07-18T03:10:42.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.836+0000 7f288b7fe700 1 -- 172.21.15.101:0/363308484 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2890003cf0 con 0x7f28940ff9d0 2023-07-18T03:10:42.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.836+0000 7f289b39d700 1 -- 172.21.15.101:0/363308484 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28941a2670 con 0x7f28940ff9d0 2023-07-18T03:10:42.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.840+0000 7f288b7fe700 1 -- 172.21.15.101:0/363308484 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f289000ccd0 con 0x7f28940ff9d0 2023-07-18T03:10:42.832 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.841+0000 7f28afe92b80 1 Processor -- start 2023-07-18T03:10:42.832 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.841+0000 7f28afe92b80 1 -- start start 2023-07-18T03:10:42.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.841+0000 7f28afe92b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fdcb144f0 0x555fdcb148c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.841+0000 7f28afe92b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fdcb14e00 0x555fdcb22c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.841+0000 7f28afe92b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fdcb23140 0x555fdcb25520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.842+0000 7f28afe92b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fdca19cb0 con 0x555fdcb23140 2023-07-18T03:10:42.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.842+0000 7f28afe92b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fdca6ca40 con 0x555fdcb14e00 2023-07-18T03:10:42.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.842+0000 7f28afe92b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fdca65ed0 con 0x555fdcb144f0 2023-07-18T03:10:42.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.842+0000 7f289a39b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fdcb14e00 0x555fdcb22c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.842+0000 7f289ab9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fdcb144f0 0x555fdcb148c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.842+0000 7f289ab9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fdcb144f0 0x555fdcb148c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:43386/0 (socket says 172.21.15.101:43386) 2023-07-18T03:10:42.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.842+0000 7f289ab9c700 1 -- 172.21.15.101:0/1884678030 learned_addr learned my addr 172.21.15.101:0/1884678030 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:42.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.842+0000 7f2899b9a700 1 --2- 172.21.15.101:0/1884678030 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fdcb23140 0x555fdcb25520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.842+0000 7f288b7fe700 1 --2- 172.21.15.101:0/363308484 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f28800a13e0 0x7f28800a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.842+0000 7f288b7fe700 1 -- 172.21.15.101:0/363308484 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f2890014070 con 0x7f28940ff9d0 2023-07-18T03:10:42.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.842+0000 7f289ab9c700 1 --2- 172.21.15.101:0/363308484 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f28800a13e0 0x7f28800a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.842+0000 7f289a39b700 1 -- 172.21.15.101:0/1884678030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fdcb144f0 msgr2=0x555fdcb148c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:42.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.842+0000 7f289a39b700 1 --2- 172.21.15.101:0/1884678030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fdcb144f0 0x555fdcb148c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.842+0000 7f289a39b700 1 -- 172.21.15.101:0/1884678030 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fdcb23140 msgr2=0x555fdcb25520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:42.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.842+0000 7f289a39b700 1 --2- 172.21.15.101:0/1884678030 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fdcb23140 0x555fdcb25520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.842+0000 7f289a39b700 1 -- 172.21.15.101:0/1884678030 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555fdc893de0 con 0x555fdcb14e00 2023-07-18T03:10:42.839 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.842+0000 7f289a39b700 1 --2- 172.21.15.101:0/1884678030 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fdcb14e00 0x555fdcb22c00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f28900ca5e0 tx=0x7f2890003780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a21a920ef46e5728 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:42.839 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.843+0000 7f2888bfe700 1 -- 172.21.15.101:0/1884678030 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28900998c0 con 0x555fdcb14e00 2023-07-18T03:10:42.839 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.843+0000 7f2888bfe700 1 -- 172.21.15.101:0/1884678030 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2890016330 con 0x555fdcb14e00 2023-07-18T03:10:42.839 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.843+0000 7f2888bfe700 1 -- 172.21.15.101:0/1884678030 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2890095d50 con 0x555fdcb14e00 2023-07-18T03:10:42.839 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.843+0000 7f28afe92b80 1 -- 172.21.15.101:0/1884678030 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fdcb14e00 msgr2=0x555fdcb22c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:42.840 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.843+0000 7f289ab9c700 1 --2- 172.21.15.101:0/363308484 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f28800a13e0 0x7f28800a3890 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f288c0095b0 tx=0x7f288c004b40 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:42.840 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.843+0000 7f28afe92b80 1 --2- 172.21.15.101:0/1884678030 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fdcb14e00 0x555fdcb22c00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f28900ca5e0 tx=0x7f2890003780 comp rx=0 tx=0).stop 2023-07-18T03:10:42.840 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.843+0000 7f28afe92b80 1 -- 172.21.15.101:0/1884678030 shutdown_connections 2023-07-18T03:10:42.840 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.843+0000 7f28afe92b80 1 --2- 172.21.15.101:0/1884678030 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fdcb23140 0x555fdcb25520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.841 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.843+0000 7f28afe92b80 1 --2- 172.21.15.101:0/1884678030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fdcb144f0 0x555fdcb148c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.841 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.843+0000 7f28afe92b80 1 --2- 172.21.15.101:0/1884678030 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fdcb14e00 0x555fdcb22c00 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.841 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.843+0000 7f28afe92b80 1 -- 172.21.15.101:0/1884678030 >> 172.21.15.101:0/1884678030 conn(0x555fdcb09ff0 msgr2=0x555fdcb0f930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:42.842 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.844+0000 7f28afe92b80 1 -- 172.21.15.101:0/1884678030 shutdown_connections 2023-07-18T03:10:42.842 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.844+0000 7f28afe92b80 1 -- 172.21.15.101:0/1884678030 wait complete. 2023-07-18T03:10:42.842 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.844+0000 7f28afe92b80 1 -- start start 2023-07-18T03:10:42.842 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.844+0000 7f28afe92b80 1 Processor -- start 2023-07-18T03:10:42.843 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.845+0000 7f28afe92b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fdcb144f0 0x555fdcb22b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.843 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.845+0000 7f28afe92b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fdcb14e00 0x555fdcb1fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.843 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.845+0000 7f28afe92b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fdcb23140 0x555fdcb204e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.845+0000 7f28afe92b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fdca05ac0 con 0x555fdcb144f0 2023-07-18T03:10:42.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.845+0000 7f28afe92b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fdca057c0 con 0x555fdcb23140 2023-07-18T03:10:42.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.845+0000 7f28afe92b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fdc8e9920 con 0x555fdcb14e00 2023-07-18T03:10:42.845 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.845+0000 7f28afe92b80 10 client.0 _ll_register_callbacks cb 0x555fdca03760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:10:42.845 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.845+0000 7f289a39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fdcb144f0 0x555fdcb22b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.845 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.845+0000 7f289a39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fdcb144f0 0x555fdcb22b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:47512/0 (socket says 172.21.15.101:47512) 2023-07-18T03:10:42.845 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.845+0000 7f289a39b700 1 -- 172.21.15.101:0/3250379688 learned_addr learned my addr 172.21.15.101:0/3250379688 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:42.846 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.846+0000 7f2899b9a700 1 --2- 172.21.15.101:0/3250379688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fdcb14e00 0x555fdcb1fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.846 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.846+0000 7f289a39b700 1 -- 172.21.15.101:0/3250379688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fdcb14e00 msgr2=0x555fdcb1fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:42.846 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.846+0000 7f289ab9c700 1 --2- 172.21.15.101:0/3250379688 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fdcb23140 0x555fdcb204e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.847 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.846+0000 7f289a39b700 1 --2- 172.21.15.101:0/3250379688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fdcb14e00 0x555fdcb1fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.847 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.846+0000 7f289a39b700 1 -- 172.21.15.101:0/3250379688 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fdcb23140 msgr2=0x555fdcb204e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:42.847 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.846+0000 7f289a39b700 1 --2- 172.21.15.101:0/3250379688 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fdcb23140 0x555fdcb204e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.847 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.846+0000 7f289a39b700 1 -- 172.21.15.101:0/3250379688 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28941a2670 con 0x555fdcb144f0 2023-07-18T03:10:42.847 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.846+0000 7f2899b9a700 1 --2- 172.21.15.101:0/3250379688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fdcb14e00 0x555fdcb1fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:10:42.848 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.846+0000 7f289ab9c700 1 --2- 172.21.15.101:0/3250379688 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fdcb23140 0x555fdcb204e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:10:42.848 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.846+0000 7f289a39b700 1 --2- 172.21.15.101:0/3250379688 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fdcb144f0 0x555fdcb22b90 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f28900a1e80 tx=0x7f28900a23b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5517ff2e6085ecd3 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:42.848 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.846+0000 7f287dffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:10:42.848 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.846+0000 7f287dffb700 1 -- 172.21.15.101:0/3250379688 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289009a3c0 con 0x555fdcb144f0 2023-07-18T03:10:42.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.847+0000 7f287dffb700 1 -- 172.21.15.101:0/3250379688 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f289009df90 con 0x555fdcb144f0 2023-07-18T03:10:42.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.847+0000 7f28afe92b80 1 -- 172.21.15.101:0/3250379688 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555fdc9bf4f0 con 0x555fdcb144f0 2023-07-18T03:10:42.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.847+0000 7f287dffb700 1 -- 172.21.15.101:0/3250379688 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2890095f60 con 0x555fdcb144f0 2023-07-18T03:10:42.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.847+0000 7f28afe92b80 10 client.16182 fetch_fsmap learned FSMap version 25 2023-07-18T03:10:42.850 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.847+0000 7f28afe92b80 1 -- 172.21.15.101:0/3250379688 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555fdc87cc60 con 0x555fdcb144f0 2023-07-18T03:10:42.850 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.847+0000 7f287dffb700 1 -- 172.21.15.101:0/3250379688 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555fdc9bf4f0 con 0x555fdcb144f0 2023-07-18T03:10:42.850 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.847+0000 7f287dffb700 1 -- 172.21.15.101:0/3250379688 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f28900a1f20 con 0x555fdcb144f0 2023-07-18T03:10:42.850 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.847+0000 7f28afe92b80 10 client.16182 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:10:42.850 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.847+0000 7f28afe92b80 10 client.16182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:10:42.850 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.847+0000 7f28afe92b80 20 client.16182 populate_metadata read hostname 'smithi101' 2023-07-18T03:10:42.850 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.847+0000 7f28afe92b80 1 --2- 172.21.15.101:0/3250379688 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x555fdcb32660 0x555fdcb34a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.851 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.847+0000 7f28afe92b80 4 client.16182 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:10:42.851 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.847+0000 7f28afe92b80 1 -- 172.21.15.101:0/3250379688 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555fdca5eeb0 con 0x555fdcb32660 2023-07-18T03:10:42.851 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.848+0000 7f2899b9a700 1 --2- 172.21.15.101:0/3250379688 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x555fdcb32660 0x555fdcb34a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.851 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.848+0000 7f2899b9a700 1 --2- 172.21.15.101:0/3250379688 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x555fdcb32660 0x555fdcb34a60 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1af67807ead5faf server_cookie=1125a8adba33dde4 in_seq=0 out_seq=0 2023-07-18T03:10:42.851 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.848+0000 7f287dffb700 10 client.16182 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:10:42.852 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.849+0000 7f287dffb700 1 -- 172.21.15.101:0/3250379688 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555fdca5eeb0 con 0x555fdcb32660 2023-07-18T03:10:42.852 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.849+0000 7f287dffb700 10 client.16182 handle_command_reply: tid=0 2023-07-18T03:10:42.853 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.849+0000 7f28afe92b80 1 client.16182 shutdown 2023-07-18T03:10:42.853 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.849+0000 7f28afe92b80 1 -- 172.21.15.101:0/3250379688 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fdcb144f0 msgr2=0x555fdcb22b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:42.853 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.849+0000 7f28afe92b80 1 --2- 172.21.15.101:0/3250379688 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fdcb144f0 0x555fdcb22b90 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f28900a1e80 tx=0x7f28900a23b0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.853 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.849+0000 7f28afe92b80 1 -- 172.21.15.101:0/3250379688 shutdown_connections 2023-07-18T03:10:42.853 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.849+0000 7f28afe92b80 1 --2- 172.21.15.101:0/3250379688 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x555fdcb32660 0x555fdcb34a60 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.853 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.849+0000 7f28afe92b80 1 --2- 172.21.15.101:0/3250379688 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fdcb23140 0x555fdcb204e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.853 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.849+0000 7f28afe92b80 1 --2- 172.21.15.101:0/3250379688 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fdcb144f0 0x555fdcb22b90 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.849+0000 7f28afe92b80 1 --2- 172.21.15.101:0/3250379688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fdcb14e00 0x555fdcb1fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.849+0000 7f287dffb700 0 client.16182 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:10:42.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.849+0000 7f28afe92b80 1 -- 172.21.15.101:0/3250379688 >> 172.21.15.101:0/3250379688 conn(0x555fdcb09ff0 msgr2=0x555fdcb0f520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:42.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.849+0000 7f28afe92b80 1 -- 172.21.15.101:0/3250379688 shutdown_connections 2023-07-18T03:10:42.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.849+0000 7f28afe92b80 1 -- 172.21.15.101:0/3250379688 wait complete. 2023-07-18T03:10:42.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.849+0000 7f28afe92b80 20 client.16182 trim_cache size 0 max 16384 2023-07-18T03:10:42.858 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.867+0000 7f28afe92b80 1 Processor -- start 2023-07-18T03:10:42.858 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.867+0000 7f28afe92b80 1 -- start start 2023-07-18T03:10:42.859 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.867+0000 7f28afe92b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fdcb144f0 0x555fdcb29bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.859 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.867+0000 7f28afe92b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fdcb14e00 0x555fdcb0eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.859 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.867+0000 7f28afe92b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fdcb23140 0x555fdcb226c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.859 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.867+0000 7f28afe92b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fdcb20ae0 con 0x555fdcb144f0 2023-07-18T03:10:42.860 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.867+0000 7f28afe92b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fdcb09b70 con 0x555fdcb14e00 2023-07-18T03:10:42.860 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.867+0000 7f28afe92b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fdcb2a100 con 0x555fdcb23140 2023-07-18T03:10:42.860 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.867+0000 7f289ab9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fdcb23140 0x555fdcb226c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.860 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.867+0000 7f2899b9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fdcb14e00 0x555fdcb0eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.860 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.867+0000 7f2899b9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fdcb14e00 0x555fdcb0eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:34144/0 (socket says 172.21.15.101:34144) 2023-07-18T03:10:42.860 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.867+0000 7f289ab9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fdcb23140 0x555fdcb226c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:43390/0 (socket says 172.21.15.101:43390) 2023-07-18T03:10:42.860 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.867+0000 7f2899b9a700 1 -- 172.21.15.101:0/3693210381 learned_addr learned my addr 172.21.15.101:0/3693210381 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:42.861 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.867+0000 7f289a39b700 1 --2- 172.21.15.101:0/3693210381 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fdcb144f0 0x555fdcb29bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.861 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.868+0000 7f2899b9a700 1 -- 172.21.15.101:0/3693210381 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fdcb23140 msgr2=0x555fdcb226c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:42.861 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.868+0000 7f2899b9a700 1 --2- 172.21.15.101:0/3693210381 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fdcb23140 0x555fdcb226c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.861 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.868+0000 7f2899b9a700 1 -- 172.21.15.101:0/3693210381 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fdcb144f0 msgr2=0x555fdcb29bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:42.861 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.868+0000 7f2899b9a700 1 --2- 172.21.15.101:0/3693210381 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fdcb144f0 0x555fdcb29bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.861 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.868+0000 7f2899b9a700 1 -- 172.21.15.101:0/3693210381 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555fdc8ba250 con 0x555fdcb14e00 2023-07-18T03:10:42.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.868+0000 7f2899b9a700 1 --2- 172.21.15.101:0/3693210381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fdcb14e00 0x555fdcb0eb00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f28840098c0 tx=0x7f2884005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7db1bbb9c8723af1 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:42.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.868+0000 7f28893ff700 1 -- 172.21.15.101:0/3693210381 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288401c040 con 0x555fdcb14e00 2023-07-18T03:10:42.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.868+0000 7f28893ff700 1 -- 172.21.15.101:0/3693210381 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f288401d040 con 0x555fdcb14e00 2023-07-18T03:10:42.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.868+0000 7f28893ff700 1 -- 172.21.15.101:0/3693210381 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288400b7c0 con 0x555fdcb14e00 2023-07-18T03:10:42.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.869+0000 7f28afe92b80 1 -- 172.21.15.101:0/3693210381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fdcb14e00 msgr2=0x555fdcb0eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:42.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.869+0000 7f28afe92b80 1 --2- 172.21.15.101:0/3693210381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fdcb14e00 0x555fdcb0eb00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f28840098c0 tx=0x7f2884005dc0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.869+0000 7f28afe92b80 1 -- 172.21.15.101:0/3693210381 shutdown_connections 2023-07-18T03:10:42.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.869+0000 7f28afe92b80 1 --2- 172.21.15.101:0/3693210381 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fdcb23140 0x555fdcb226c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.869+0000 7f28afe92b80 1 --2- 172.21.15.101:0/3693210381 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fdcb144f0 0x555fdcb29bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.869+0000 7f28afe92b80 1 --2- 172.21.15.101:0/3693210381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fdcb14e00 0x555fdcb0eb00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.869+0000 7f28afe92b80 1 -- 172.21.15.101:0/3693210381 >> 172.21.15.101:0/3693210381 conn(0x555fdcb09ff0 msgr2=0x555fdcb2ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:42.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.869+0000 7f28afe92b80 1 -- 172.21.15.101:0/3693210381 shutdown_connections 2023-07-18T03:10:42.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.869+0000 7f28afe92b80 1 -- 172.21.15.101:0/3693210381 wait complete. 2023-07-18T03:10:42.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.870+0000 7f28afe92b80 1 -- start start 2023-07-18T03:10:42.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.870+0000 7f28afe92b80 1 Processor -- start 2023-07-18T03:10:42.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.870+0000 7f28afe92b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fdcb144f0 0x555fdcb297a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.870+0000 7f28afe92b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fdcb14e00 0x555fdcb27ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.870+0000 7f28afe92b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fdcb23140 0x555fdcb28010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.870+0000 7f28afe92b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fdcb28550 con 0x555fdcb144f0 2023-07-18T03:10:42.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.870+0000 7f28afe92b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fdcb286c0 con 0x555fdcb23140 2023-07-18T03:10:42.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.870+0000 7f28afe92b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fdcb28830 con 0x555fdcb14e00 2023-07-18T03:10:42.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.870+0000 7f28afe92b80 10 client.0 _ll_register_callbacks cb 0x7f2890005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:10:42.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.871+0000 7f2899b9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fdcb14e00 0x555fdcb27ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.871+0000 7f289ab9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fdcb23140 0x555fdcb28010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.871+0000 7f2899b9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fdcb14e00 0x555fdcb27ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:43400/0 (socket says 172.21.15.101:43400) 2023-07-18T03:10:42.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.871+0000 7f2899b9a700 1 -- 172.21.15.101:0/3297389396 learned_addr learned my addr 172.21.15.101:0/3297389396 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:42.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.871+0000 7f289a39b700 1 --2- 172.21.15.101:0/3297389396 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fdcb144f0 0x555fdcb297a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.871+0000 7f2899b9a700 1 -- 172.21.15.101:0/3297389396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fdcb23140 msgr2=0x555fdcb28010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:42.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.871+0000 7f2899b9a700 1 --2- 172.21.15.101:0/3297389396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fdcb23140 0x555fdcb28010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.871+0000 7f2899b9a700 1 -- 172.21.15.101:0/3297389396 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fdcb144f0 msgr2=0x555fdcb297a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:42.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.871+0000 7f2899b9a700 1 --2- 172.21.15.101:0/3297389396 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fdcb144f0 0x555fdcb297a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.871+0000 7f2899b9a700 1 -- 172.21.15.101:0/3297389396 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f289410d110 con 0x555fdcb14e00 2023-07-18T03:10:42.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.871+0000 7f289a39b700 1 --2- 172.21.15.101:0/3297389396 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fdcb144f0 0x555fdcb297a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:10:42.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.871+0000 7f2899b9a700 1 --2- 172.21.15.101:0/3297389396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fdcb14e00 0x555fdcb27ad0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f288400a010 tx=0x7f2884005bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aca9cc7c4cbb4e24 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:42.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.871+0000 7f287dffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:10:42.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.871+0000 7f287dffb700 1 -- 172.21.15.101:0/3297389396 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288401c040 con 0x555fdcb14e00 2023-07-18T03:10:42.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.871+0000 7f28afe92b80 1 -- 172.21.15.101:0/3297389396 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2890016330 con 0x555fdcb14e00 2023-07-18T03:10:42.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.872+0000 7f287dffb700 1 -- 172.21.15.101:0/3297389396 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f288400f700 con 0x555fdcb14e00 2023-07-18T03:10:42.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.872+0000 7f287dffb700 1 -- 172.21.15.101:0/3297389396 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f288400f8c0 con 0x555fdcb14e00 2023-07-18T03:10:42.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.872+0000 7f28afe92b80 10 client.26254 fetch_fsmap learned FSMap version 25 2023-07-18T03:10:42.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.872+0000 7f28afe92b80 1 -- 172.21.15.101:0/3297389396 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555fdc8922b0 con 0x555fdcb14e00 2023-07-18T03:10:42.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.872+0000 7f287dffb700 1 -- 172.21.15.101:0/3297389396 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2890016330 con 0x555fdcb14e00 2023-07-18T03:10:42.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.872+0000 7f287dffb700 1 -- 172.21.15.101:0/3297389396 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f2884015020 con 0x555fdcb14e00 2023-07-18T03:10:42.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.872+0000 7f28afe92b80 10 client.26254 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:10:42.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.872+0000 7f28afe92b80 10 client.26254 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:10:42.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.872+0000 7f28afe92b80 20 client.26254 populate_metadata read hostname 'smithi101' 2023-07-18T03:10:42.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.872+0000 7f28afe92b80 1 --2- 172.21.15.101:0/3297389396 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x555fdcb87630 0x555fdcb89a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:42.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.872+0000 7f28afe92b80 4 client.26254 mds_command: new command op to 24562 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:10:42.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.872+0000 7f28afe92b80 1 -- 172.21.15.101:0/3297389396 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555fdca21050 con 0x555fdcb87630 2023-07-18T03:10:42.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.873+0000 7f289a39b700 1 --2- 172.21.15.101:0/3297389396 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x555fdcb87630 0x555fdcb89a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:42.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.873+0000 7f289a39b700 1 --2- 172.21.15.101:0/3297389396 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x555fdcb87630 0x555fdcb89a40 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d3794866312e70 server_cookie=7797bcfbb79357c3 in_seq=0 out_seq=0 2023-07-18T03:10:42.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.873+0000 7f287dffb700 10 client.26254 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:10:42.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.874+0000 7f287dffb700 1 -- 172.21.15.101:0/3297389396 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555fdca21050 con 0x555fdcb87630 2023-07-18T03:10:42.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.874+0000 7f287dffb700 10 client.26254 handle_command_reply: tid=0 2023-07-18T03:10:42.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.874+0000 7f28afe92b80 1 client.26254 shutdown 2023-07-18T03:10:42.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.874+0000 7f28afe92b80 1 -- 172.21.15.101:0/3297389396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fdcb14e00 msgr2=0x555fdcb27ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:42.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.874+0000 7f28afe92b80 1 --2- 172.21.15.101:0/3297389396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fdcb14e00 0x555fdcb27ad0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f288400a010 tx=0x7f2884005bc0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.874+0000 7f28afe92b80 1 -- 172.21.15.101:0/3297389396 shutdown_connections 2023-07-18T03:10:42.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.874+0000 7f28afe92b80 1 --2- 172.21.15.101:0/3297389396 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x555fdcb87630 0x555fdcb89a40 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.874+0000 7f28afe92b80 1 --2- 172.21.15.101:0/3297389396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fdcb23140 0x555fdcb28010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.874+0000 7f28afe92b80 1 --2- 172.21.15.101:0/3297389396 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fdcb144f0 0x555fdcb297a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.874+0000 7f28afe92b80 1 --2- 172.21.15.101:0/3297389396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fdcb14e00 0x555fdcb27ad0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.874+0000 7f287dffb700 0 client.26254 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:10:42.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.874+0000 7f28afe92b80 1 -- 172.21.15.101:0/3297389396 >> 172.21.15.101:0/3297389396 conn(0x555fdcb09ff0 msgr2=0x555fdcb1b140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:42.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.874+0000 7f28afe92b80 1 -- 172.21.15.101:0/3297389396 shutdown_connections 2023-07-18T03:10:42.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.874+0000 7f28afe92b80 1 -- 172.21.15.101:0/3297389396 wait complete. 2023-07-18T03:10:42.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.874+0000 7f28afe92b80 20 client.26254 trim_cache size 0 max 16384 2023-07-18T03:10:42.875 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:10:42.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.875+0000 7f289b39d700 1 -- 172.21.15.101:0/363308484 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f28800a13e0 msgr2=0x7f28800a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:42.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.875+0000 7f289b39d700 1 --2- 172.21.15.101:0/363308484 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f28800a13e0 0x7f28800a3890 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f288c0095b0 tx=0x7f288c004b40 comp rx=0 tx=0).stop 2023-07-18T03:10:42.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.875+0000 7f289b39d700 1 -- 172.21.15.101:0/363308484 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28940ff9d0 msgr2=0x7f289419b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:42.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.875+0000 7f289b39d700 1 --2- 172.21.15.101:0/363308484 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28940ff9d0 0x7f289419b860 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f289000e970 tx=0x7f289000ec80 comp rx=0 tx=0).stop 2023-07-18T03:10:42.877 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.875+0000 7f289b39d700 1 -- 172.21.15.101:0/363308484 shutdown_connections 2023-07-18T03:10:42.877 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.875+0000 7f289b39d700 1 --2- 172.21.15.101:0/363308484 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f28941059e0 0x7f28941958d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.877 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.875+0000 7f289b39d700 1 --2- 172.21.15.101:0/363308484 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28940ff9d0 0x7f289419b860 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.877 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.875+0000 7f289b39d700 1 --2- 172.21.15.101:0/363308484 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f28800a13e0 0x7f28800a3890 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.877 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.875+0000 7f289b39d700 1 --2- 172.21.15.101:0/363308484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2894100380 0x7f289419bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:42.877 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.875+0000 7f289b39d700 1 -- 172.21.15.101:0/363308484 >> 172.21.15.101:0/363308484 conn(0x7f28940fb4b0 msgr2=0x7f28940fca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:42.877 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.875+0000 7f289b39d700 1 -- 172.21.15.101:0/363308484 shutdown_connections 2023-07-18T03:10:42.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:42.875+0000 7f289b39d700 1 -- 172.21.15.101:0/363308484 wait complete. 2023-07-18T03:10:43.235 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:43 smithi101 ceph-mon[126379]: pgmap v97: 129 pgs: 129 active+clean; 116 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 205 op/s 2023-07-18T03:10:43.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:43 smithi155 ceph-mon[128885]: pgmap v97: 129 pgs: 129 active+clean; 116 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 205 op/s 2023-07-18T03:10:43.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:43 smithi179 ceph-mon[129229]: pgmap v97: 129 pgs: 129 active+clean; 116 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 205 op/s 2023-07-18T03:10:43.571 INFO:tasks.workunit.client.0.smithi101.stdout: 1 193989 8.27 MB/sec execute 8 sec latency 71.966 ms 2023-07-18T03:10:43.884 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-18T03:10:43.885 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-18T03:10:44.102 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.111+0000 7fa4a7fff700 1 -- 172.21.15.101:0/1797364369 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa4a8105a00 msgr2=0x7fa4a8105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:44.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.111+0000 7fa4a7fff700 1 --2- 172.21.15.101:0/1797364369 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa4a8105a00 0x7fa4a8105dd0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa498009990 tx=0x7fa498009ca0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.111+0000 7fa4a7fff700 1 -- 172.21.15.101:0/1797364369 shutdown_connections 2023-07-18T03:10:44.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.111+0000 7fa4a7fff700 1 --2- 172.21.15.101:0/1797364369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4a81003a0 0x7fa4a810c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.111+0000 7fa4a7fff700 1 --2- 172.21.15.101:0/1797364369 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa4a8105a00 0x7fa4a8105dd0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.111+0000 7fa4a7fff700 1 --2- 172.21.15.101:0/1797364369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4a80ff9f0 0x7fa4a80ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.111+0000 7fa4a7fff700 1 -- 172.21.15.101:0/1797364369 >> 172.21.15.101:0/1797364369 conn(0x7fa4a80fb510 msgr2=0x7fa4a80fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:44.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.111+0000 7fa4a7fff700 1 -- 172.21.15.101:0/1797364369 shutdown_connections 2023-07-18T03:10:44.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.111+0000 7fa4a7fff700 1 -- 172.21.15.101:0/1797364369 wait complete. 2023-07-18T03:10:44.108 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.112+0000 7fa4a7fff700 1 Processor -- start 2023-07-18T03:10:44.108 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.112+0000 7fa4a7fff700 1 -- start start 2023-07-18T03:10:44.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.112+0000 7fa4a7fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4a80ff9f0 0x7fa4a819d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:44.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.112+0000 7fa4a7fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa4a81003a0 0x7fa4a819dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:44.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.112+0000 7fa4a7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4a8105a00 0x7fa4a8197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:44.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.112+0000 7fa4a7fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a8198070 con 0x7fa4a81003a0 2023-07-18T03:10:44.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.112+0000 7fa4a7fff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a81981e0 con 0x7fa4a80ff9f0 2023-07-18T03:10:44.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.112+0000 7fa4a7fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a81983a0 con 0x7fa4a8105a00 2023-07-18T03:10:44.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.113+0000 7fa4a6ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4a80ff9f0 0x7fa4a819d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:44.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.113+0000 7fa4a6ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4a80ff9f0 0x7fa4a819d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:34178/0 (socket says 172.21.15.101:34178) 2023-07-18T03:10:44.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.113+0000 7fa4a6ffd700 1 -- 172.21.15.101:0/3510614178 learned_addr learned my addr 172.21.15.101:0/3510614178 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:44.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.113+0000 7fa4a67fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa4a81003a0 0x7fa4a819dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:44.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.113+0000 7fa4a77fe700 1 --2- 172.21.15.101:0/3510614178 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4a8105a00 0x7fa4a8197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:44.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.113+0000 7fa4a67fc700 1 -- 172.21.15.101:0/3510614178 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4a8105a00 msgr2=0x7fa4a8197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:44.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.113+0000 7fa4a67fc700 1 --2- 172.21.15.101:0/3510614178 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4a8105a00 0x7fa4a8197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.113+0000 7fa4a67fc700 1 -- 172.21.15.101:0/3510614178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4a80ff9f0 msgr2=0x7fa4a819d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:44.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.113+0000 7fa4a67fc700 1 --2- 172.21.15.101:0/3510614178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4a80ff9f0 0x7fa4a819d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.113+0000 7fa4a67fc700 1 -- 172.21.15.101:0/3510614178 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4980095f0 con 0x7fa4a81003a0 2023-07-18T03:10:44.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.113+0000 7fa4a6ffd700 1 --2- 172.21.15.101:0/3510614178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4a80ff9f0 0x7fa4a819d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:10:44.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.113+0000 7fa4a77fe700 1 --2- 172.21.15.101:0/3510614178 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4a8105a00 0x7fa4a8197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:10:44.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.113+0000 7fa4a67fc700 1 --2- 172.21.15.101:0/3510614178 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa4a81003a0 0x7fa4a819dee0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa49c00e9f0 tx=0x7fa49c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:44.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.113+0000 7fa48ffff700 1 -- 172.21.15.101:0/3510614178 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49c00c9c0 con 0x7fa4a81003a0 2023-07-18T03:10:44.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.114+0000 7fa48ffff700 1 -- 172.21.15.101:0/3510614178 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa49c010430 con 0x7fa4a81003a0 2023-07-18T03:10:44.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.114+0000 7fa4a7fff700 1 -- 172.21.15.101:0/3510614178 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4a8198680 con 0x7fa4a81003a0 2023-07-18T03:10:44.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.114+0000 7fa48ffff700 1 -- 172.21.15.101:0/3510614178 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49c018680 con 0x7fa4a81003a0 2023-07-18T03:10:44.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.114+0000 7fa4a7fff700 1 -- 172.21.15.101:0/3510614178 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4a81a4990 con 0x7fa4a81003a0 2023-07-18T03:10:44.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.117+0000 7fa48ffff700 1 -- 172.21.15.101:0/3510614178 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fa49c010860 con 0x7fa4a81003a0 2023-07-18T03:10:44.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.118+0000 7fa4c11fcb80 1 Processor -- start 2023-07-18T03:10:44.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.118+0000 7fa4c11fcb80 1 -- start start 2023-07-18T03:10:44.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.119+0000 7fa48ffff700 1 --2- 172.21.15.101:0/3510614178 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa4900a13e0 0x7fa4900a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:44.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.119+0000 7fa48ffff700 1 -- 172.21.15.101:0/3510614178 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fa49c014070 con 0x7fa4a81003a0 2023-07-18T03:10:44.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.119+0000 7fa4c11fcb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b33e504570 0x55b33e504940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:44.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.119+0000 7fa4c11fcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b33e504e80 0x55b33e50e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:44.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.119+0000 7fa4c11fcb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b33e50edb0 0x55b33e511190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:44.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.119+0000 7fa4c11fcb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b33e409d30 con 0x55b33e504570 2023-07-18T03:10:44.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.119+0000 7fa4c11fcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b33e45cac0 con 0x55b33e504e80 2023-07-18T03:10:44.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.119+0000 7fa4c11fcb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b33e455f50 con 0x55b33e50edb0 2023-07-18T03:10:44.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.119+0000 7fa4a77fe700 1 --2- 172.21.15.101:0/3510614178 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa4900a13e0 0x7fa4900a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:44.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.119+0000 7fa4a67fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b33e504e80 0x55b33e50e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:44.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.119+0000 7fa4a6ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b33e504570 0x55b33e504940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:44.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.119+0000 7fa4a67fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b33e504e80 0x55b33e50e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:34194/0 (socket says 172.21.15.101:34194) 2023-07-18T03:10:44.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.119+0000 7fa4a67fc700 1 -- 172.21.15.101:0/205598727 learned_addr learned my addr 172.21.15.101:0/205598727 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:44.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.119+0000 7fa4a77fe700 1 --2- 172.21.15.101:0/205598727 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b33e50edb0 0x55b33e511190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:44.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.119+0000 7fa4a67fc700 1 -- 172.21.15.101:0/205598727 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b33e50edb0 msgr2=0x55b33e511190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:44.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.119+0000 7fa4a67fc700 1 --2- 172.21.15.101:0/205598727 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b33e50edb0 0x55b33e511190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.119+0000 7fa4a67fc700 1 -- 172.21.15.101:0/205598727 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b33e504570 msgr2=0x55b33e504940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:44.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.119+0000 7fa4a67fc700 1 --2- 172.21.15.101:0/205598727 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b33e504570 0x55b33e504940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.119+0000 7fa4a67fc700 1 -- 172.21.15.101:0/205598727 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b33e283e60 con 0x55b33e504e80 2023-07-18T03:10:44.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.119+0000 7fa4a77fe700 1 --2- 172.21.15.101:0/205598727 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b33e50edb0 0x55b33e511190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:10:44.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.120+0000 7fa4a67fc700 1 --2- 172.21.15.101:0/205598727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b33e504e80 0x55b33e50e870 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa49c096920 tx=0x7fa49c099a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a811a93e9eb8a127 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:44.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.120+0000 7fa4a77fe700 1 --2- 172.21.15.101:0/3510614178 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa4900a13e0 0x7fa4900a3890 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0009790 tx=0x7fa4a0009720 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:44.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.120+0000 7fa48d7fa700 1 -- 172.21.15.101:0/205598727 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49c09dee0 con 0x55b33e504e80 2023-07-18T03:10:44.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.120+0000 7fa48d7fa700 1 -- 172.21.15.101:0/205598727 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa49c0a2300 con 0x55b33e504e80 2023-07-18T03:10:44.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.120+0000 7fa48d7fa700 1 -- 172.21.15.101:0/205598727 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49c0995a0 con 0x55b33e504e80 2023-07-18T03:10:44.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.121+0000 7fa4c11fcb80 1 -- 172.21.15.101:0/205598727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b33e504e80 msgr2=0x55b33e50e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:44.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.121+0000 7fa4c11fcb80 1 --2- 172.21.15.101:0/205598727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b33e504e80 0x55b33e50e870 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa49c096920 tx=0x7fa49c099a70 comp rx=0 tx=0).stop 2023-07-18T03:10:44.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.121+0000 7fa4c11fcb80 1 -- 172.21.15.101:0/205598727 shutdown_connections 2023-07-18T03:10:44.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.121+0000 7fa4c11fcb80 1 --2- 172.21.15.101:0/205598727 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b33e50edb0 0x55b33e511190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.121+0000 7fa4c11fcb80 1 --2- 172.21.15.101:0/205598727 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b33e504570 0x55b33e504940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.121+0000 7fa4c11fcb80 1 --2- 172.21.15.101:0/205598727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b33e504e80 0x55b33e50e870 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.121+0000 7fa4c11fcb80 1 -- 172.21.15.101:0/205598727 >> 172.21.15.101:0/205598727 conn(0x55b33e4fa070 msgr2=0x55b33e4ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:44.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.121+0000 7fa4c11fcb80 1 -- 172.21.15.101:0/205598727 shutdown_connections 2023-07-18T03:10:44.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.121+0000 7fa4c11fcb80 1 -- 172.21.15.101:0/205598727 wait complete. 2023-07-18T03:10:44.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.121+0000 7fa4c11fcb80 1 -- start start 2023-07-18T03:10:44.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.122+0000 7fa4c11fcb80 1 Processor -- start 2023-07-18T03:10:44.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.122+0000 7fa4c11fcb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b33e504570 0x55b33e50e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:44.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.122+0000 7fa4c11fcb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b33e504e80 0x55b33e50ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:44.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.122+0000 7fa4c11fcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b33e50edb0 0x55b33e50c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:44.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.122+0000 7fa4c11fcb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b33e3f5b40 con 0x55b33e504570 2023-07-18T03:10:44.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.122+0000 7fa4c11fcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b33e3f5840 con 0x55b33e50edb0 2023-07-18T03:10:44.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.122+0000 7fa4c11fcb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b33e2d99a0 con 0x55b33e504e80 2023-07-18T03:10:44.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.122+0000 7fa4c11fcb80 10 client.0 _ll_register_callbacks cb 0x55b33e3f37e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:10:44.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.122+0000 7fa4a6ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b33e504570 0x55b33e50e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:44.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.122+0000 7fa4a67fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b33e504e80 0x55b33e50ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:44.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.123+0000 7fa4a77fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b33e50edb0 0x55b33e50c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:44.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.123+0000 7fa4a67fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b33e504e80 0x55b33e50ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:43430/0 (socket says 172.21.15.101:43430) 2023-07-18T03:10:44.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.123+0000 7fa4a6ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b33e504570 0x55b33e50e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:47556/0 (socket says 172.21.15.101:47556) 2023-07-18T03:10:44.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.123+0000 7fa4a67fc700 1 -- 172.21.15.101:0/2012385512 learned_addr learned my addr 172.21.15.101:0/2012385512 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:44.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.123+0000 7fa4a67fc700 1 -- 172.21.15.101:0/2012385512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b33e50edb0 msgr2=0x55b33e50c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:44.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.123+0000 7fa4a67fc700 1 --2- 172.21.15.101:0/2012385512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b33e50edb0 0x55b33e50c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.123+0000 7fa4a67fc700 1 -- 172.21.15.101:0/2012385512 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b33e504570 msgr2=0x55b33e50e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:44.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.123+0000 7fa4a67fc700 1 --2- 172.21.15.101:0/2012385512 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b33e504570 0x55b33e50e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.123+0000 7fa4a67fc700 1 -- 172.21.15.101:0/2012385512 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4a81a4990 con 0x55b33e504e80 2023-07-18T03:10:44.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.123+0000 7fa4a77fe700 1 --2- 172.21.15.101:0/2012385512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b33e50edb0 0x55b33e50c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:10:44.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.123+0000 7fa4a6ffd700 1 --2- 172.21.15.101:0/2012385512 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b33e504570 0x55b33e50e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:10:44.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.123+0000 7fa4a67fc700 1 --2- 172.21.15.101:0/2012385512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b33e504e80 0x55b33e50ba70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa49c0caab0 tx=0x7fa49c099910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c926ba9d76b3e8d9 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:44.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.123+0000 7fa48aff5700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:10:44.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.123+0000 7fa48aff5700 1 -- 172.21.15.101:0/2012385512 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49c0b6070 con 0x55b33e504e80 2023-07-18T03:10:44.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.123+0000 7fa4c11fcb80 1 -- 172.21.15.101:0/2012385512 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b33e3af570 con 0x55b33e504e80 2023-07-18T03:10:44.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.123+0000 7fa48aff5700 1 -- 172.21.15.101:0/2012385512 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fa49c0105a0 con 0x55b33e504e80 2023-07-18T03:10:44.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.124+0000 7fa48aff5700 1 -- 172.21.15.101:0/2012385512 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa49c0b82b0 con 0x55b33e504e80 2023-07-18T03:10:44.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.124+0000 7fa4c11fcb80 10 client.26266 fetch_fsmap learned FSMap version 25 2023-07-18T03:10:44.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.124+0000 7fa4c11fcb80 1 -- 172.21.15.101:0/2012385512 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b33e26cce0 con 0x55b33e504e80 2023-07-18T03:10:44.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.124+0000 7fa48aff5700 1 -- 172.21.15.101:0/2012385512 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa49c07c080 con 0x55b33e504e80 2023-07-18T03:10:44.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.124+0000 7fa48aff5700 1 -- 172.21.15.101:0/2012385512 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fa49c0ca170 con 0x55b33e504e80 2023-07-18T03:10:44.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.124+0000 7fa4c11fcb80 10 client.26266 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:10:44.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.124+0000 7fa4c11fcb80 10 client.26266 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:10:44.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.124+0000 7fa4c11fcb80 20 client.26266 populate_metadata read hostname 'smithi101' 2023-07-18T03:10:44.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.124+0000 7fa4c11fcb80 1 --2- 172.21.15.101:0/2012385512 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b33e51e370 0x55b33e520780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:44.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.124+0000 7fa4c11fcb80 4 client.26266 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:10:44.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.124+0000 7fa4c11fcb80 1 -- 172.21.15.101:0/2012385512 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b33e44ef30 con 0x55b33e51e370 2023-07-18T03:10:44.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.124+0000 7fa4a6ffd700 1 --2- 172.21.15.101:0/2012385512 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b33e51e370 0x55b33e520780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:44.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.125+0000 7fa48aff5700 10 client.26266 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:10:44.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.125+0000 7fa4a6ffd700 1 --2- 172.21.15.101:0/2012385512 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b33e51e370 0x55b33e520780 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8144a43cfde500c6 server_cookie=f6e51be19b824312 in_seq=0 out_seq=0 2023-07-18T03:10:44.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.126+0000 7fa48aff5700 1 -- 172.21.15.101:0/2012385512 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b33e44ef30 con 0x55b33e51e370 2023-07-18T03:10:44.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.126+0000 7fa48aff5700 10 client.26266 handle_command_reply: tid=0 2023-07-18T03:10:44.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.126+0000 7fa4c11fcb80 1 client.26266 shutdown 2023-07-18T03:10:44.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.126+0000 7fa4c11fcb80 1 -- 172.21.15.101:0/2012385512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b33e504e80 msgr2=0x55b33e50ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:44.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.126+0000 7fa4c11fcb80 1 --2- 172.21.15.101:0/2012385512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b33e504e80 0x55b33e50ba70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa49c0caab0 tx=0x7fa49c099910 comp rx=0 tx=0).stop 2023-07-18T03:10:44.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.127+0000 7fa4c11fcb80 1 -- 172.21.15.101:0/2012385512 shutdown_connections 2023-07-18T03:10:44.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.127+0000 7fa4c11fcb80 1 --2- 172.21.15.101:0/2012385512 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b33e51e370 0x55b33e520780 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.127+0000 7fa4c11fcb80 1 --2- 172.21.15.101:0/2012385512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b33e50edb0 0x55b33e50c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.127+0000 7fa4c11fcb80 1 --2- 172.21.15.101:0/2012385512 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b33e504570 0x55b33e50e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.127+0000 7fa48aff5700 0 client.26266 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:10:44.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.127+0000 7fa4c11fcb80 1 --2- 172.21.15.101:0/2012385512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b33e504e80 0x55b33e50ba70 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.127+0000 7fa4c11fcb80 1 -- 172.21.15.101:0/2012385512 >> 172.21.15.101:0/2012385512 conn(0x55b33e4fa070 msgr2=0x55b33e4ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:44.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.127+0000 7fa4c11fcb80 1 -- 172.21.15.101:0/2012385512 shutdown_connections 2023-07-18T03:10:44.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.127+0000 7fa4c11fcb80 1 -- 172.21.15.101:0/2012385512 wait complete. 2023-07-18T03:10:44.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.127+0000 7fa4c11fcb80 20 client.26266 trim_cache size 0 max 16384 2023-07-18T03:10:44.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.146+0000 7fa4c11fcb80 1 Processor -- start 2023-07-18T03:10:44.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.146+0000 7fa4c11fcb80 1 -- start start 2023-07-18T03:10:44.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.147+0000 7fa4c11fcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b33e504570 0x55b33e515830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:44.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.147+0000 7fa4c11fcb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b33e504e80 0x55b33e503bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:44.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.147+0000 7fa4c11fcb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b33e50edb0 0x55b33e508ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:44.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.147+0000 7fa4c11fcb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b33e5076a0 con 0x55b33e50edb0 2023-07-18T03:10:44.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.147+0000 7fa4c11fcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b33e515d70 con 0x55b33e504570 2023-07-18T03:10:44.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.147+0000 7fa4c11fcb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b33e50cc90 con 0x55b33e504e80 2023-07-18T03:10:44.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.147+0000 7fa4a6ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b33e504570 0x55b33e515830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:44.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.147+0000 7fa4a6ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b33e504570 0x55b33e515830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:34214/0 (socket says 172.21.15.101:34214) 2023-07-18T03:10:44.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.147+0000 7fa4a67fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b33e504e80 0x55b33e503bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:44.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.147+0000 7fa4a6ffd700 1 -- 172.21.15.101:0/1806671719 learned_addr learned my addr 172.21.15.101:0/1806671719 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:44.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.147+0000 7fa4a77fe700 1 --2- 172.21.15.101:0/1806671719 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b33e50edb0 0x55b33e508ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:44.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.147+0000 7fa4a6ffd700 1 -- 172.21.15.101:0/1806671719 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b33e504e80 msgr2=0x55b33e503bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:44.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.147+0000 7fa4a6ffd700 1 --2- 172.21.15.101:0/1806671719 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b33e504e80 0x55b33e503bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.147+0000 7fa4a6ffd700 1 -- 172.21.15.101:0/1806671719 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b33e50edb0 msgr2=0x55b33e508ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:44.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.147+0000 7fa4a6ffd700 1 --2- 172.21.15.101:0/1806671719 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b33e50edb0 0x55b33e508ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.147+0000 7fa4a6ffd700 1 -- 172.21.15.101:0/1806671719 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b33e2aa2d0 con 0x55b33e504570 2023-07-18T03:10:44.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.147+0000 7fa4a77fe700 1 --2- 172.21.15.101:0/1806671719 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b33e50edb0 0x55b33e508ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:10:44.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.148+0000 7fa4a6ffd700 1 --2- 172.21.15.101:0/1806671719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b33e504570 0x55b33e515830 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa49800b2e0 tx=0x7fa498005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75502f79717aa633 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:44.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.148+0000 7fa48dffb700 1 -- 172.21.15.101:0/1806671719 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49802b070 con 0x55b33e504570 2023-07-18T03:10:44.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.148+0000 7fa48dffb700 1 -- 172.21.15.101:0/1806671719 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa49801e040 con 0x55b33e504570 2023-07-18T03:10:44.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.148+0000 7fa48dffb700 1 -- 172.21.15.101:0/1806671719 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49801d040 con 0x55b33e504570 2023-07-18T03:10:44.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.149+0000 7fa4c11fcb80 1 -- 172.21.15.101:0/1806671719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b33e504570 msgr2=0x55b33e515830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:44.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.149+0000 7fa4c11fcb80 1 --2- 172.21.15.101:0/1806671719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b33e504570 0x55b33e515830 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa49800b2e0 tx=0x7fa498005fd0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.149+0000 7fa4c11fcb80 1 -- 172.21.15.101:0/1806671719 shutdown_connections 2023-07-18T03:10:44.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.149+0000 7fa4c11fcb80 1 --2- 172.21.15.101:0/1806671719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b33e504570 0x55b33e515830 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.149+0000 7fa4c11fcb80 1 --2- 172.21.15.101:0/1806671719 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b33e50edb0 0x55b33e508ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.149+0000 7fa4c11fcb80 1 --2- 172.21.15.101:0/1806671719 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b33e504e80 0x55b33e503bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.149+0000 7fa4c11fcb80 1 -- 172.21.15.101:0/1806671719 >> 172.21.15.101:0/1806671719 conn(0x55b33e4fa070 msgr2=0x55b33e5034c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:44.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.149+0000 7fa4c11fcb80 1 -- 172.21.15.101:0/1806671719 shutdown_connections 2023-07-18T03:10:44.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.149+0000 7fa4c11fcb80 1 -- 172.21.15.101:0/1806671719 wait complete. 2023-07-18T03:10:44.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.149+0000 7fa4c11fcb80 1 -- start start 2023-07-18T03:10:44.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.149+0000 7fa4c11fcb80 1 Processor -- start 2023-07-18T03:10:44.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.150+0000 7fa4c11fcb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b33e504570 0x55b33e56b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:44.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.150+0000 7fa4c11fcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b33e504e80 0x55b33e56bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:44.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.150+0000 7fa4c11fcb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b33e50edb0 0x55b33e514790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:44.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.150+0000 7fa4c11fcb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b33e514ea0 con 0x55b33e50edb0 2023-07-18T03:10:44.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.150+0000 7fa4c11fcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b33e515010 con 0x55b33e504e80 2023-07-18T03:10:44.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.150+0000 7fa4c11fcb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b33e5151d0 con 0x55b33e504570 2023-07-18T03:10:44.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.150+0000 7fa4c11fcb80 10 client.0 _ll_register_callbacks cb 0x7fa49c005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:10:44.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.150+0000 7fa4a6ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b33e504570 0x55b33e56b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:44.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.150+0000 7fa4a6ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b33e504570 0x55b33e56b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:43452/0 (socket says 172.21.15.101:43452) 2023-07-18T03:10:44.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.150+0000 7fa4a6ffd700 1 -- 172.21.15.101:0/4166658755 learned_addr learned my addr 172.21.15.101:0/4166658755 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:44.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.150+0000 7fa4a67fc700 1 --2- 172.21.15.101:0/4166658755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b33e504e80 0x55b33e56bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:44.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.150+0000 7fa4a77fe700 1 --2- 172.21.15.101:0/4166658755 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b33e50edb0 0x55b33e514790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:44.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.151+0000 7fa4a6ffd700 1 -- 172.21.15.101:0/4166658755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b33e504e80 msgr2=0x55b33e56bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:44.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.151+0000 7fa4a6ffd700 1 --2- 172.21.15.101:0/4166658755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b33e504e80 0x55b33e56bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.151+0000 7fa4a6ffd700 1 -- 172.21.15.101:0/4166658755 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b33e50edb0 msgr2=0x55b33e514790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:44.156 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.151+0000 7fa4a6ffd700 1 --2- 172.21.15.101:0/4166658755 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b33e50edb0 0x55b33e514790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.156 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.151+0000 7fa4a6ffd700 1 -- 172.21.15.101:0/4166658755 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4a810d130 con 0x55b33e504570 2023-07-18T03:10:44.156 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.151+0000 7fa4a77fe700 1 --2- 172.21.15.101:0/4166658755 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b33e50edb0 0x55b33e514790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:10:44.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.151+0000 7fa4a6ffd700 1 --2- 172.21.15.101:0/4166658755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b33e504570 0x55b33e56b760 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa49800bfd0 tx=0x7fa49800f460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5ac8381cb2d57b7 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:44.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.151+0000 7fa48aff5700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:10:44.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.151+0000 7fa48aff5700 1 -- 172.21.15.101:0/4166658755 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49801d070 con 0x55b33e504570 2023-07-18T03:10:44.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.151+0000 7fa48aff5700 1 -- 172.21.15.101:0/4166658755 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fa498005260 con 0x55b33e504570 2023-07-18T03:10:44.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.151+0000 7fa4c11fcb80 1 -- 172.21.15.101:0/4166658755 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa49c0a2300 con 0x55b33e504570 2023-07-18T03:10:44.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.151+0000 7fa48aff5700 1 -- 172.21.15.101:0/4166658755 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa49802b040 con 0x55b33e504570 2023-07-18T03:10:44.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.151+0000 7fa4c11fcb80 10 client.26272 fetch_fsmap learned FSMap version 25 2023-07-18T03:10:44.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.151+0000 7fa4c11fcb80 1 -- 172.21.15.101:0/4166658755 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b33e282330 con 0x55b33e504570 2023-07-18T03:10:44.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.152+0000 7fa48aff5700 1 -- 172.21.15.101:0/4166658755 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa49c0a2300 con 0x55b33e504570 2023-07-18T03:10:44.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.152+0000 7fa48aff5700 1 -- 172.21.15.101:0/4166658755 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fa498030ce0 con 0x55b33e504570 2023-07-18T03:10:44.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.152+0000 7fa4c11fcb80 10 client.26272 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:10:44.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.152+0000 7fa4c11fcb80 10 client.26272 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:10:44.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.152+0000 7fa4c11fcb80 20 client.26272 populate_metadata read hostname 'smithi101' 2023-07-18T03:10:44.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.152+0000 7fa4c11fcb80 1 --2- 172.21.15.101:0/4166658755 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b33e5742c0 0x55b33e5766d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:44.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.152+0000 7fa4c11fcb80 4 client.26272 mds_command: new command op to 24562 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-18T03:10:44.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.152+0000 7fa4c11fcb80 1 -- 172.21.15.101:0/4166658755 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55b33e4110d0 con 0x55b33e5742c0 2023-07-18T03:10:44.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.152+0000 7fa4a67fc700 1 --2- 172.21.15.101:0/4166658755 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b33e5742c0 0x55b33e5766d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:44.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.153+0000 7fa48aff5700 10 client.26272 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:10:44.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.153+0000 7fa4a67fc700 1 --2- 172.21.15.101:0/4166658755 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b33e5742c0 0x55b33e5766d0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94a5d393d3368a8 server_cookie=1e0ae8cbc4480f70 in_seq=0 out_seq=0 2023-07-18T03:10:44.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.154+0000 7fa48aff5700 1 -- 172.21.15.101:0/4166658755 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b33e4110d0 con 0x55b33e5742c0 2023-07-18T03:10:44.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.154+0000 7fa48aff5700 10 client.26272 handle_command_reply: tid=0 2023-07-18T03:10:44.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.154+0000 7fa4c11fcb80 1 client.26272 shutdown 2023-07-18T03:10:44.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.154+0000 7fa4c11fcb80 1 -- 172.21.15.101:0/4166658755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b33e504570 msgr2=0x55b33e56b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:44.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.154+0000 7fa4c11fcb80 1 --2- 172.21.15.101:0/4166658755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b33e504570 0x55b33e56b760 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa49800bfd0 tx=0x7fa49800f460 comp rx=0 tx=0).stop 2023-07-18T03:10:44.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.154+0000 7fa4c11fcb80 1 -- 172.21.15.101:0/4166658755 shutdown_connections 2023-07-18T03:10:44.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.154+0000 7fa4c11fcb80 1 --2- 172.21.15.101:0/4166658755 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b33e5742c0 0x55b33e5766d0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.154+0000 7fa4c11fcb80 1 --2- 172.21.15.101:0/4166658755 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b33e50edb0 0x55b33e514790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.154+0000 7fa4c11fcb80 1 --2- 172.21.15.101:0/4166658755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b33e504570 0x55b33e56b760 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.154+0000 7fa48aff5700 0 client.26272 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:10:44.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.154+0000 7fa4c11fcb80 1 --2- 172.21.15.101:0/4166658755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b33e504e80 0x55b33e56bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.154+0000 7fa4c11fcb80 1 -- 172.21.15.101:0/4166658755 >> 172.21.15.101:0/4166658755 conn(0x55b33e4fa070 msgr2=0x55b33e503d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:44.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.154+0000 7fa4c11fcb80 1 -- 172.21.15.101:0/4166658755 shutdown_connections 2023-07-18T03:10:44.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.154+0000 7fa4c11fcb80 1 -- 172.21.15.101:0/4166658755 wait complete. 2023-07-18T03:10:44.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.154+0000 7fa4c11fcb80 20 client.26272 trim_cache size 0 max 16384 2023-07-18T03:10:44.163 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:10:44.163 INFO:teuthology.orchestra.run.smithi101.stdout: "return_code": 0, 2023-07-18T03:10:44.163 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_tag": "8696cdd4-3cd5-41de-be52-278beef00fbc", 2023-07-18T03:10:44.163 INFO:teuthology.orchestra.run.smithi101.stdout: "mode": "asynchronous" 2023-07-18T03:10:44.163 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:10:44.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.155+0000 7fa4a7fff700 1 -- 172.21.15.101:0/3510614178 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa4900a13e0 msgr2=0x7fa4900a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:44.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.155+0000 7fa4a7fff700 1 --2- 172.21.15.101:0/3510614178 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa4900a13e0 0x7fa4900a3890 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0009790 tx=0x7fa4a0009720 comp rx=0 tx=0).stop 2023-07-18T03:10:44.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.155+0000 7fa4a7fff700 1 -- 172.21.15.101:0/3510614178 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa4a81003a0 msgr2=0x7fa4a819dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:44.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.155+0000 7fa4a7fff700 1 --2- 172.21.15.101:0/3510614178 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa4a81003a0 0x7fa4a819dee0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa49c00e9f0 tx=0x7fa49c00ed00 comp rx=0 tx=0).stop 2023-07-18T03:10:44.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.155+0000 7fa4a7fff700 1 -- 172.21.15.101:0/3510614178 shutdown_connections 2023-07-18T03:10:44.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.155+0000 7fa4a7fff700 1 --2- 172.21.15.101:0/3510614178 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa4900a13e0 0x7fa4900a3890 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.155+0000 7fa4a7fff700 1 --2- 172.21.15.101:0/3510614178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4a80ff9f0 0x7fa4a819d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.155+0000 7fa4a7fff700 1 --2- 172.21.15.101:0/3510614178 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4a8105a00 0x7fa4a8197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.155+0000 7fa4a7fff700 1 --2- 172.21.15.101:0/3510614178 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa4a81003a0 0x7fa4a819dee0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.155+0000 7fa4a7fff700 1 -- 172.21.15.101:0/3510614178 >> 172.21.15.101:0/3510614178 conn(0x7fa4a80fb510 msgr2=0x7fa4a8072740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:44.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.155+0000 7fa4a7fff700 1 -- 172.21.15.101:0/3510614178 shutdown_connections 2023-07-18T03:10:44.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.155+0000 7fa4a7fff700 1 -- 172.21.15.101:0/3510614178 wait complete. 2023-07-18T03:10:44.167 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:10:44.379 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.387+0000 7f84f87ff700 1 -- 172.21.15.101:0/365886026 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f84f0105af0 msgr2=0x7f84f0105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:44.379 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.387+0000 7f84f87ff700 1 --2- 172.21.15.101:0/365886026 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f84f0105af0 0x7f84f0105ec0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f84ec009990 tx=0x7f84ec009ca0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.379 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.388+0000 7f84f87ff700 1 -- 172.21.15.101:0/365886026 shutdown_connections 2023-07-18T03:10:44.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.388+0000 7f84f87ff700 1 --2- 172.21.15.101:0/365886026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84f0100490 0x7f84f010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.388+0000 7f84f87ff700 1 --2- 172.21.15.101:0/365886026 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f84f0105af0 0x7f84f0105ec0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.388+0000 7f84f87ff700 1 --2- 172.21.15.101:0/365886026 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84f00ffae0 0x7f84f00fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.388+0000 7f84f87ff700 1 -- 172.21.15.101:0/365886026 >> 172.21.15.101:0/365886026 conn(0x7f84f00fb5e0 msgr2=0x7f84f00fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:44.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.388+0000 7f84f87ff700 1 -- 172.21.15.101:0/365886026 shutdown_connections 2023-07-18T03:10:44.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.388+0000 7f84f87ff700 1 -- 172.21.15.101:0/365886026 wait complete. 2023-07-18T03:10:44.382 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.389+0000 7f84f87ff700 1 Processor -- start 2023-07-18T03:10:44.382 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.389+0000 7f84f87ff700 1 -- start start 2023-07-18T03:10:44.382 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.389+0000 7f84f87ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84f00ffae0 0x7f84f019b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:44.382 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.389+0000 7f84f87ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f84f0100490 0x7f84f019bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:44.382 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.389+0000 7f84f87ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84f0105af0 0x7f84f0195a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:44.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.389+0000 7f84f87ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f0196080 con 0x7f84f0100490 2023-07-18T03:10:44.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.389+0000 7f84f87ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f01961f0 con 0x7f84f0105af0 2023-07-18T03:10:44.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.389+0000 7f84f87ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f01963b0 con 0x7f84f00ffae0 2023-07-18T03:10:44.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.389+0000 7f84f659b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84f00ffae0 0x7f84f019b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:44.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.389+0000 7f84f659b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84f00ffae0 0x7f84f019b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:43472/0 (socket says 172.21.15.101:43472) 2023-07-18T03:10:44.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.389+0000 7f84f659b700 1 -- 172.21.15.101:0/2161092438 learned_addr learned my addr 172.21.15.101:0/2161092438 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:44.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.389+0000 7f84f5d9a700 1 --2- 172.21.15.101:0/2161092438 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f84f0100490 0x7f84f019bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:44.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.389+0000 7f84f6d9c700 1 --2- 172.21.15.101:0/2161092438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84f0105af0 0x7f84f0195a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:44.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.390+0000 7f84f6d9c700 1 -- 172.21.15.101:0/2161092438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84f00ffae0 msgr2=0x7f84f019b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:44.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.390+0000 7f84f6d9c700 1 --2- 172.21.15.101:0/2161092438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84f00ffae0 0x7f84f019b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.390+0000 7f84f6d9c700 1 -- 172.21.15.101:0/2161092438 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f84f0100490 msgr2=0x7f84f019bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:44.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.390+0000 7f84f6d9c700 1 --2- 172.21.15.101:0/2161092438 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f84f0100490 0x7f84f019bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.390+0000 7f84f6d9c700 1 -- 172.21.15.101:0/2161092438 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84ec0095f0 con 0x7f84f0105af0 2023-07-18T03:10:44.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.390+0000 7f84f659b700 1 --2- 172.21.15.101:0/2161092438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84f00ffae0 0x7f84f019b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:10:44.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.390+0000 7f84f6d9c700 1 --2- 172.21.15.101:0/2161092438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84f0105af0 0x7f84f0195a10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f84e800efc0 tx=0x7f84e800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:44.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.390+0000 7f84e77fe700 1 -- 172.21.15.101:0/2161092438 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e8018070 con 0x7f84f0105af0 2023-07-18T03:10:44.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.390+0000 7f84e77fe700 1 -- 172.21.15.101:0/2161092438 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84e800f040 con 0x7f84f0105af0 2023-07-18T03:10:44.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.390+0000 7f84f87ff700 1 -- 172.21.15.101:0/2161092438 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84f0196630 con 0x7f84f0105af0 2023-07-18T03:10:44.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.390+0000 7f84e77fe700 1 -- 172.21.15.101:0/2161092438 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e800cd40 con 0x7f84f0105af0 2023-07-18T03:10:44.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.391+0000 7f84f87ff700 1 -- 172.21.15.101:0/2161092438 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84f01a2990 con 0x7f84f0105af0 2023-07-18T03:10:44.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.394+0000 7f84e77fe700 1 -- 172.21.15.101:0/2161092438 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f84e8003a00 con 0x7f84f0105af0 2023-07-18T03:10:44.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.395+0000 7f850c21ab80 1 Processor -- start 2023-07-18T03:10:44.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.396+0000 7f850c21ab80 1 -- start start 2023-07-18T03:10:44.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.396+0000 7f850c21ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e443c8b2d0 0x55e443c8b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:44.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.396+0000 7f850c21ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e443c8bbe0 0x55e443c96590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:44.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.396+0000 7f850c21ab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e443c96ad0 0x55e443c98eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:44.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.396+0000 7f850c21ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e443b91cb0 con 0x55e443c8bbe0 2023-07-18T03:10:44.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.396+0000 7f850c21ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e443be4a40 con 0x55e443c8b2d0 2023-07-18T03:10:44.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.396+0000 7f850c21ab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e443bdded0 con 0x55e443c96ad0 2023-07-18T03:10:44.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.396+0000 7f84f5d9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e443c8b2d0 0x55e443c8b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:44.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.396+0000 7f84f659b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e443c8bbe0 0x55e443c96590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:44.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.396+0000 7f84f5d9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e443c8b2d0 0x55e443c8b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:34250/0 (socket says 172.21.15.101:34250) 2023-07-18T03:10:44.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.396+0000 7f84f5d9a700 1 -- 172.21.15.101:0/688857173 learned_addr learned my addr 172.21.15.101:0/688857173 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:44.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.396+0000 7f84e77fe700 1 --2- 172.21.15.101:0/2161092438 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f84dc0a12c0 0x7f84dc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:44.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.396+0000 7f84f5d9a700 1 --2- 172.21.15.101:0/688857173 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e443c96ad0 0x55e443c98eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:44.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.396+0000 7f84e77fe700 1 -- 172.21.15.101:0/2161092438 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f84e80ccc40 con 0x7f84f0105af0 2023-07-18T03:10:44.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.397+0000 7f84f6d9c700 1 --2- 172.21.15.101:0/2161092438 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f84dc0a12c0 0x7f84dc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:44.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.397+0000 7f84f659b700 1 -- 172.21.15.101:0/688857173 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e443c96ad0 msgr2=0x55e443c98eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:44.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.397+0000 7f84f659b700 1 --2- 172.21.15.101:0/688857173 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e443c96ad0 0x55e443c98eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.397+0000 7f84f659b700 1 -- 172.21.15.101:0/688857173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e443c8b2d0 msgr2=0x55e443c8b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:44.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.397+0000 7f84f659b700 1 --2- 172.21.15.101:0/688857173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e443c8b2d0 0x55e443c8b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.397+0000 7f84f659b700 1 -- 172.21.15.101:0/688857173 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e443a0bde0 con 0x55e443c8bbe0 2023-07-18T03:10:44.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.397+0000 7f84f5d9a700 1 --2- 172.21.15.101:0/688857173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e443c8b2d0 0x55e443c8b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:10:44.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.397+0000 7f84f659b700 1 --2- 172.21.15.101:0/688857173 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e443c8bbe0 0x55e443c96590 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f84ec000c00 tx=0x7f84ec00b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32b32047514ca279 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:44.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.397+0000 7f84f6d9c700 1 --2- 172.21.15.101:0/2161092438 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f84dc0a12c0 0x7f84dc0a3770 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f84e80ca040 tx=0x7f84e80b7f90 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:44.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.397+0000 7f84e4ff9700 1 -- 172.21.15.101:0/688857173 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ec01d070 con 0x55e443c8bbe0 2023-07-18T03:10:44.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.397+0000 7f84e4ff9700 1 -- 172.21.15.101:0/688857173 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84ec01e040 con 0x55e443c8bbe0 2023-07-18T03:10:44.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.397+0000 7f84e4ff9700 1 -- 172.21.15.101:0/688857173 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ec004c40 con 0x55e443c8bbe0 2023-07-18T03:10:44.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.398+0000 7f850c21ab80 1 -- 172.21.15.101:0/688857173 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e443c8bbe0 msgr2=0x55e443c96590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:44.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.398+0000 7f850c21ab80 1 --2- 172.21.15.101:0/688857173 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e443c8bbe0 0x55e443c96590 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f84ec000c00 tx=0x7f84ec00b610 comp rx=0 tx=0).stop 2023-07-18T03:10:44.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.398+0000 7f850c21ab80 1 -- 172.21.15.101:0/688857173 shutdown_connections 2023-07-18T03:10:44.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.398+0000 7f850c21ab80 1 --2- 172.21.15.101:0/688857173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e443c8b2d0 0x55e443c8b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.398+0000 7f850c21ab80 1 --2- 172.21.15.101:0/688857173 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e443c96ad0 0x55e443c98eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.398+0000 7f850c21ab80 1 --2- 172.21.15.101:0/688857173 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e443c8bbe0 0x55e443c96590 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.398+0000 7f850c21ab80 1 -- 172.21.15.101:0/688857173 >> 172.21.15.101:0/688857173 conn(0x55e443c81da0 msgr2=0x55e443c876a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:44.394 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.398+0000 7f850c21ab80 1 -- 172.21.15.101:0/688857173 shutdown_connections 2023-07-18T03:10:44.394 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.398+0000 7f850c21ab80 1 -- 172.21.15.101:0/688857173 wait complete. 2023-07-18T03:10:44.394 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.399+0000 7f850c21ab80 1 -- start start 2023-07-18T03:10:44.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.399+0000 7f850c21ab80 1 Processor -- start 2023-07-18T03:10:44.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.399+0000 7f850c21ab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e443c8b2d0 0x55e443c964e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:44.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.399+0000 7f850c21ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e443c8bbe0 0x55e443c936f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:44.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.400+0000 7f850c21ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e443c96ad0 0x55e443c93d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:44.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.400+0000 7f850c21ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e443b7dac0 con 0x55e443c8bbe0 2023-07-18T03:10:44.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.400+0000 7f850c21ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e443b7d7c0 con 0x55e443c96ad0 2023-07-18T03:10:44.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.400+0000 7f850c21ab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e443a61920 con 0x55e443c8b2d0 2023-07-18T03:10:44.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.400+0000 7f850c21ab80 10 client.0 _ll_register_callbacks cb 0x55e443b7b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:10:44.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.400+0000 7f84f5d9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e443c8b2d0 0x55e443c964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:44.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.400+0000 7f84f659b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e443c8bbe0 0x55e443c936f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:44.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.400+0000 7f84f5d9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e443c8b2d0 0x55e443c964e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:43484/0 (socket says 172.21.15.101:43484) 2023-07-18T03:10:44.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.400+0000 7f84f5d9a700 1 -- 172.21.15.101:0/1509883632 learned_addr learned my addr 172.21.15.101:0/1509883632 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:44.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.400+0000 7f84f659b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e443c8bbe0 0x55e443c936f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:47608/0 (socket says 172.21.15.101:47608) 2023-07-18T03:10:44.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.400+0000 7f84f5d9a700 1 --2- 172.21.15.101:0/1509883632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e443c96ad0 0x55e443c93d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:44.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.400+0000 7f84f659b700 1 -- 172.21.15.101:0/1509883632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e443c8b2d0 msgr2=0x55e443c964e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:44.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.400+0000 7f84f659b700 1 --2- 172.21.15.101:0/1509883632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e443c8b2d0 0x55e443c964e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.400+0000 7f84f659b700 1 -- 172.21.15.101:0/1509883632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e443c96ad0 msgr2=0x55e443c93d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:44.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.400+0000 7f84f659b700 1 --2- 172.21.15.101:0/1509883632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e443c96ad0 0x55e443c93d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.400+0000 7f84f659b700 1 -- 172.21.15.101:0/1509883632 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84e0019040 con 0x55e443c8bbe0 2023-07-18T03:10:44.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.401+0000 7f84f659b700 1 --2- 172.21.15.101:0/1509883632 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e443c8bbe0 0x55e443c936f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f84ec000c00 tx=0x7f84ec00fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=603bdbbef2c1ab85 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:44.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.401+0000 7f84d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:10:44.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.401+0000 7f84d9ffb700 1 -- 172.21.15.101:0/1509883632 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ec01d040 con 0x55e443c8bbe0 2023-07-18T03:10:44.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.401+0000 7f850c21ab80 1 -- 172.21.15.101:0/1509883632 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e443b374f0 con 0x55e443c8bbe0 2023-07-18T03:10:44.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.401+0000 7f84d9ffb700 1 -- 172.21.15.101:0/1509883632 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f84ec00fe30 con 0x55e443c8bbe0 2023-07-18T03:10:44.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.401+0000 7f84d9ffb700 1 -- 172.21.15.101:0/1509883632 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84ec005bc0 con 0x55e443c8bbe0 2023-07-18T03:10:44.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.401+0000 7f850c21ab80 10 client.16215 fetch_fsmap learned FSMap version 25 2023-07-18T03:10:44.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.401+0000 7f850c21ab80 1 -- 172.21.15.101:0/1509883632 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f84f010d220 con 0x55e443c8bbe0 2023-07-18T03:10:44.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.402+0000 7f84d9ffb700 1 -- 172.21.15.101:0/1509883632 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e443b374f0 con 0x55e443c8bbe0 2023-07-18T03:10:44.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.402+0000 7f84d9ffb700 1 -- 172.21.15.101:0/1509883632 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f84ec022520 con 0x55e443c8bbe0 2023-07-18T03:10:44.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.402+0000 7f850c21ab80 10 client.16215 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:10:44.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.402+0000 7f850c21ab80 10 client.16215 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:10:44.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.402+0000 7f850c21ab80 20 client.16215 populate_metadata read hostname 'smithi101' 2023-07-18T03:10:44.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.402+0000 7f850c21ab80 1 --2- 172.21.15.101:0/1509883632 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55e443caa300 0x55e443cac710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:44.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.402+0000 7f850c21ab80 4 client.16215 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:10:44.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.402+0000 7f850c21ab80 1 -- 172.21.15.101:0/1509883632 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e443bd6eb0 con 0x55e443caa300 2023-07-18T03:10:44.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.402+0000 7f84f5d9a700 1 --2- 172.21.15.101:0/1509883632 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55e443caa300 0x55e443cac710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:44.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.403+0000 7f84d9ffb700 10 client.16215 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:10:44.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.403+0000 7f84f5d9a700 1 --2- 172.21.15.101:0/1509883632 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55e443caa300 0x55e443cac710 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13308621e0aa812a server_cookie=fc8b8614e5f55641 in_seq=0 out_seq=0 2023-07-18T03:10:44.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.404+0000 7f84d9ffb700 1 -- 172.21.15.101:0/1509883632 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e443bd6eb0 con 0x55e443caa300 2023-07-18T03:10:44.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.404+0000 7f84d9ffb700 10 client.16215 handle_command_reply: tid=0 2023-07-18T03:10:44.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.404+0000 7f850c21ab80 1 client.16215 shutdown 2023-07-18T03:10:44.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.404+0000 7f850c21ab80 1 -- 172.21.15.101:0/1509883632 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e443c8bbe0 msgr2=0x55e443c936f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:44.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.404+0000 7f850c21ab80 1 --2- 172.21.15.101:0/1509883632 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e443c8bbe0 0x55e443c936f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f84ec000c00 tx=0x7f84ec00fac0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.404+0000 7f850c21ab80 1 -- 172.21.15.101:0/1509883632 shutdown_connections 2023-07-18T03:10:44.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.404+0000 7f850c21ab80 1 --2- 172.21.15.101:0/1509883632 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55e443caa300 0x55e443cac710 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.404+0000 7f850c21ab80 1 --2- 172.21.15.101:0/1509883632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e443c96ad0 0x55e443c93d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.404+0000 7f850c21ab80 1 --2- 172.21.15.101:0/1509883632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e443c8b2d0 0x55e443c964e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.404+0000 7f84d9ffb700 0 client.16215 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:10:44.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.404+0000 7f850c21ab80 1 --2- 172.21.15.101:0/1509883632 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e443c8bbe0 0x55e443c936f0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.404+0000 7f850c21ab80 1 -- 172.21.15.101:0/1509883632 >> 172.21.15.101:0/1509883632 conn(0x55e443c81da0 msgr2=0x55e443c87230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:44.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.404+0000 7f850c21ab80 1 -- 172.21.15.101:0/1509883632 shutdown_connections 2023-07-18T03:10:44.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.404+0000 7f850c21ab80 1 -- 172.21.15.101:0/1509883632 wait complete. 2023-07-18T03:10:44.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.404+0000 7f850c21ab80 20 client.16215 trim_cache size 0 max 16384 2023-07-18T03:10:44.414 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.423+0000 7f850c21ab80 1 Processor -- start 2023-07-18T03:10:44.414 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.423+0000 7f850c21ab80 1 -- start start 2023-07-18T03:10:44.415 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.423+0000 7f850c21ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e443c8b2d0 0x55e443c85e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:44.415 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.423+0000 7f850c21ab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e443c8bbe0 0x55e443ca6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:44.415 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.423+0000 7f850c21ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e443c96ad0 0x55e443c96010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:44.415 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.423+0000 7f850c21ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e443c8e530 con 0x55e443c8b2d0 2023-07-18T03:10:44.415 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.423+0000 7f850c21ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e443c81970 con 0x55e443c96ad0 2023-07-18T03:10:44.415 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.423+0000 7f850c21ab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e443c93ce0 con 0x55e443c8bbe0 2023-07-18T03:10:44.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.424+0000 7f84f659b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e443c8bbe0 0x55e443ca6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:44.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.424+0000 7f84f5d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e443c8b2d0 0x55e443c85e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:44.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.424+0000 7f84f659b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e443c8bbe0 0x55e443ca6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:43494/0 (socket says 172.21.15.101:43494) 2023-07-18T03:10:44.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.424+0000 7f84f659b700 1 -- 172.21.15.101:0/1981501291 learned_addr learned my addr 172.21.15.101:0/1981501291 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:44.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.424+0000 7f84f5d9a700 1 --2- 172.21.15.101:0/1981501291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e443c96ad0 0x55e443c96010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:44.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.424+0000 7f84f659b700 1 -- 172.21.15.101:0/1981501291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e443c96ad0 msgr2=0x55e443c96010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:44.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.424+0000 7f84f659b700 1 --2- 172.21.15.101:0/1981501291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e443c96ad0 0x55e443c96010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.424+0000 7f84f659b700 1 -- 172.21.15.101:0/1981501291 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e443c8b2d0 msgr2=0x55e443c85e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:44.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.424+0000 7f84f659b700 1 --2- 172.21.15.101:0/1981501291 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e443c8b2d0 0x55e443c85e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.424+0000 7f84f659b700 1 -- 172.21.15.101:0/1981501291 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4439f4c60 con 0x55e443c8bbe0 2023-07-18T03:10:44.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.424+0000 7f84f659b700 1 --2- 172.21.15.101:0/1981501291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e443c8bbe0 0x55e443ca6b30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f84ec00ee20 tx=0x7f84ec0057a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba071342e4e9a660 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:44.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.425+0000 7f84e57fa700 1 -- 172.21.15.101:0/1981501291 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ec022600 con 0x55e443c8bbe0 2023-07-18T03:10:44.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.425+0000 7f84e57fa700 1 -- 172.21.15.101:0/1981501291 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84ec005910 con 0x55e443c8bbe0 2023-07-18T03:10:44.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.425+0000 7f84e57fa700 1 -- 172.21.15.101:0/1981501291 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ec00e800 con 0x55e443c8bbe0 2023-07-18T03:10:44.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.425+0000 7f850c21ab80 1 -- 172.21.15.101:0/1981501291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e443c8bbe0 msgr2=0x55e443ca6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:44.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.425+0000 7f850c21ab80 1 --2- 172.21.15.101:0/1981501291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e443c8bbe0 0x55e443ca6b30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f84ec00ee20 tx=0x7f84ec0057a0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.425+0000 7f850c21ab80 1 -- 172.21.15.101:0/1981501291 shutdown_connections 2023-07-18T03:10:44.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.425+0000 7f850c21ab80 1 --2- 172.21.15.101:0/1981501291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e443c96ad0 0x55e443c96010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.425+0000 7f850c21ab80 1 --2- 172.21.15.101:0/1981501291 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e443c8b2d0 0x55e443c85e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.425+0000 7f850c21ab80 1 --2- 172.21.15.101:0/1981501291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e443c8bbe0 0x55e443ca6b30 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.425+0000 7f850c21ab80 1 -- 172.21.15.101:0/1981501291 >> 172.21.15.101:0/1981501291 conn(0x55e443c81da0 msgr2=0x55e443cb1070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:44.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.426+0000 7f850c21ab80 1 -- 172.21.15.101:0/1981501291 shutdown_connections 2023-07-18T03:10:44.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.426+0000 7f850c21ab80 1 -- 172.21.15.101:0/1981501291 wait complete. 2023-07-18T03:10:44.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.426+0000 7f850c21ab80 1 -- start start 2023-07-18T03:10:44.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.426+0000 7f850c21ab80 1 Processor -- start 2023-07-18T03:10:44.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.427+0000 7f850c21ab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e443c8b2d0 0x55e443ca16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:44.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.427+0000 7f850c21ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e443c8bbe0 0x55e443c9fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:44.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.427+0000 7f850c21ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e443c96ad0 0x55e443c9ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:44.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.427+0000 7f850c21ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e443ca1fb0 con 0x55e443c96ad0 2023-07-18T03:10:44.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.427+0000 7f850c21ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e443ca04b0 con 0x55e443c8bbe0 2023-07-18T03:10:44.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.427+0000 7f850c21ab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e443ca0670 con 0x55e443c8b2d0 2023-07-18T03:10:44.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.427+0000 7f850c21ab80 10 client.0 _ll_register_callbacks cb 0x7f84e0009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:10:44.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.427+0000 7f84f659b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e443c8bbe0 0x55e443c9fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:44.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.427+0000 7f84f659b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e443c8bbe0 0x55e443c9fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:34268/0 (socket says 172.21.15.101:34268) 2023-07-18T03:10:44.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.427+0000 7f84f659b700 1 -- 172.21.15.101:0/223919739 learned_addr learned my addr 172.21.15.101:0/223919739 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:10:44.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.427+0000 7f84f5d9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e443c8b2d0 0x55e443ca16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:44.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.428+0000 7f84f5d9a700 1 --2- 172.21.15.101:0/223919739 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e443c96ad0 0x55e443c9ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:44.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.428+0000 7f84f659b700 1 -- 172.21.15.101:0/223919739 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e443c8b2d0 msgr2=0x55e443ca16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:44.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.428+0000 7f84f659b700 1 --2- 172.21.15.101:0/223919739 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e443c8b2d0 0x55e443ca16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.428+0000 7f84f659b700 1 -- 172.21.15.101:0/223919739 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e443c96ad0 msgr2=0x55e443c9ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:10:44.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.428+0000 7f84f659b700 1 --2- 172.21.15.101:0/223919739 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e443c96ad0 0x55e443c9ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.428+0000 7f84f659b700 1 -- 172.21.15.101:0/223919739 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84f010d220 con 0x55e443c8bbe0 2023-07-18T03:10:44.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.428+0000 7f84f5d9a700 1 --2- 172.21.15.101:0/223919739 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e443c96ad0 0x55e443c9ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:10:44.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.428+0000 7f84f659b700 1 --2- 172.21.15.101:0/223919739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e443c8bbe0 0x55e443c9fa00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f84ec00b6f0 tx=0x7f84ec022630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98cadb947902deb2 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:10:44.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.428+0000 7f84d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:10:44.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.428+0000 7f84d9ffb700 1 -- 172.21.15.101:0/223919739 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ec01d030 con 0x55e443c8bbe0 2023-07-18T03:10:44.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.428+0000 7f850c21ab80 1 -- 172.21.15.101:0/223919739 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f84ec01e040 con 0x55e443c8bbe0 2023-07-18T03:10:44.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.428+0000 7f84d9ffb700 1 -- 172.21.15.101:0/223919739 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f84ec000e40 con 0x55e443c8bbe0 2023-07-18T03:10:44.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.429+0000 7f84d9ffb700 1 -- 172.21.15.101:0/223919739 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84ec00b310 con 0x55e443c8bbe0 2023-07-18T03:10:44.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.429+0000 7f850c21ab80 10 client.26267 fetch_fsmap learned FSMap version 25 2023-07-18T03:10:44.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.429+0000 7f850c21ab80 1 -- 172.21.15.101:0/223919739 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e443a32250 con 0x55e443c8bbe0 2023-07-18T03:10:44.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.429+0000 7f84d9ffb700 1 -- 172.21.15.101:0/223919739 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84ec01e040 con 0x55e443c8bbe0 2023-07-18T03:10:44.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.429+0000 7f84d9ffb700 1 -- 172.21.15.101:0/223919739 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f84ec034460 con 0x55e443c8bbe0 2023-07-18T03:10:44.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.429+0000 7f850c21ab80 10 client.26267 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:10:44.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.429+0000 7f850c21ab80 10 client.26267 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:10:44.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.429+0000 7f850c21ab80 20 client.26267 populate_metadata read hostname 'smithi101' 2023-07-18T03:10:44.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.429+0000 7f850c21ab80 1 --2- 172.21.15.101:0/223919739 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55e443cff270 0x55e443d01680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:10:44.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.429+0000 7f850c21ab80 4 client.26267 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:10:44.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.429+0000 7f850c21ab80 1 -- 172.21.15.101:0/223919739 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e443b99050 con 0x55e443cff270 2023-07-18T03:10:44.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.430+0000 7f84f5d9a700 1 --2- 172.21.15.101:0/223919739 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55e443cff270 0x55e443d01680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:10:44.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.430+0000 7f84f5d9a700 1 --2- 172.21.15.101:0/223919739 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55e443cff270 0x55e443d01680 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a63a44a6ddfbc353 server_cookie=8381dae8f690d4ca in_seq=0 out_seq=0 2023-07-18T03:10:44.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.430+0000 7f84d9ffb700 10 client.26267 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:10:44.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.430+0000 7f84d9ffb700 1 -- 172.21.15.101:0/223919739 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55e443b99050 con 0x55e443cff270 2023-07-18T03:10:44.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.430+0000 7f84d9ffb700 10 client.26267 handle_command_reply: tid=0 2023-07-18T03:10:44.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.430+0000 7f850c21ab80 1 client.26267 shutdown 2023-07-18T03:10:44.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.430+0000 7f850c21ab80 1 -- 172.21.15.101:0/223919739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e443c8bbe0 msgr2=0x55e443c9fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:44.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.430+0000 7f850c21ab80 1 --2- 172.21.15.101:0/223919739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e443c8bbe0 0x55e443c9fa00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f84ec00b6f0 tx=0x7f84ec022630 comp rx=0 tx=0).stop 2023-07-18T03:10:44.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.431+0000 7f850c21ab80 1 -- 172.21.15.101:0/223919739 shutdown_connections 2023-07-18T03:10:44.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.431+0000 7f850c21ab80 1 --2- 172.21.15.101:0/223919739 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55e443cff270 0x55e443d01680 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.431+0000 7f850c21ab80 1 --2- 172.21.15.101:0/223919739 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e443c96ad0 0x55e443c9ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.431+0000 7f850c21ab80 1 --2- 172.21.15.101:0/223919739 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e443c8b2d0 0x55e443ca16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.431+0000 7f84d9ffb700 0 client.26267 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:10:44.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.431+0000 7f850c21ab80 1 --2- 172.21.15.101:0/223919739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e443c8bbe0 0x55e443c9fa00 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.431+0000 7f850c21ab80 1 -- 172.21.15.101:0/223919739 >> 172.21.15.101:0/223919739 conn(0x55e443c81da0 msgr2=0x55e443ca6b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:44.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.431+0000 7f850c21ab80 1 -- 172.21.15.101:0/223919739 shutdown_connections 2023-07-18T03:10:44.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.431+0000 7f850c21ab80 1 -- 172.21.15.101:0/223919739 wait complete. 2023-07-18T03:10:44.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.431+0000 7f850c21ab80 20 client.26267 trim_cache size 0 max 16384 2023-07-18T03:10:44.431 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:10:44.431 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:10:44.431 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:10:44.431 INFO:teuthology.orchestra.run.smithi101.stdout: "8696cdd4-3cd5-41de-be52-278beef00fbc": { 2023-07-18T03:10:44.431 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:10:44.431 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "8696cdd4-3cd5-41de-be52-278beef00fbc", 2023-07-18T03:10:44.431 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:10:44.432 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:10:44.432 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:10:44.432 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:10:44.433 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.432+0000 7f84f87ff700 1 -- 172.21.15.101:0/2161092438 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f84dc0a12c0 msgr2=0x7f84dc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:44.433 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.432+0000 7f84f87ff700 1 --2- 172.21.15.101:0/2161092438 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f84dc0a12c0 0x7f84dc0a3770 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f84e80ca040 tx=0x7f84e80b7f90 comp rx=0 tx=0).stop 2023-07-18T03:10:44.433 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.432+0000 7f84f87ff700 1 -- 172.21.15.101:0/2161092438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84f0105af0 msgr2=0x7f84f0195a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:10:44.433 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.432+0000 7f84f87ff700 1 --2- 172.21.15.101:0/2161092438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84f0105af0 0x7f84f0195a10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f84e800efc0 tx=0x7f84e800c3a0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.433 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.432+0000 7f84f87ff700 1 -- 172.21.15.101:0/2161092438 shutdown_connections 2023-07-18T03:10:44.433 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.432+0000 7f84f87ff700 1 --2- 172.21.15.101:0/2161092438 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f84dc0a12c0 0x7f84dc0a3770 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.432+0000 7f84f87ff700 1 --2- 172.21.15.101:0/2161092438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84f0105af0 0x7f84f0195a10 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.432+0000 7f84f87ff700 1 --2- 172.21.15.101:0/2161092438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84f00ffae0 0x7f84f019b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.432+0000 7f84f87ff700 1 --2- 172.21.15.101:0/2161092438 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f84f0100490 0x7f84f019bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:10:44.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.432+0000 7f84f87ff700 1 -- 172.21.15.101:0/2161092438 >> 172.21.15.101:0/2161092438 conn(0x7f84f00fb5e0 msgr2=0x7f84f01035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:10:44.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.432+0000 7f84f87ff700 1 -- 172.21.15.101:0/2161092438 shutdown_connections 2023-07-18T03:10:44.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:10:44.432+0000 7f84f87ff700 1 -- 172.21.15.101:0/2161092438 wait complete. 2023-07-18T03:10:44.443 INFO:tasks.cephfs.filesystem:scrub status for tag:8696cdd4-3cd5-41de-be52-278beef00fbc - {'path': '/', 'tag': '8696cdd4-3cd5-41de-be52-278beef00fbc', 'options': 'recursive,force'} 2023-07-18T03:10:44.571 INFO:tasks.workunit.client.0.smithi101.stdout: 1 195495 8.68 MB/sec execute 9 sec latency 21.867 ms 2023-07-18T03:10:45.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:45 smithi155 ceph-mon[128885]: pgmap v98: 129 pgs: 129 active+clean; 116 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 205 op/s 2023-07-18T03:10:45.293 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:45 smithi155 ceph-mon[128885]: scrub queued for path: / 2023-07-18T03:10:45.293 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:45 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:10:45.293 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:45 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:10:45.293 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:45 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:10:45.293 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:45 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:10:45.293 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:45 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:10:45.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:45 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:10:45.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:45 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:10:45.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:45 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:10:45.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:45 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:10:45.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:45 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:10:45.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:45 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:10:45.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:45 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:10:45.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:45 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:10:45.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:45 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:10:45.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:45 smithi179 ceph-mon[129229]: pgmap v98: 129 pgs: 129 active+clean; 116 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 205 op/s 2023-07-18T03:10:45.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:45 smithi179 ceph-mon[129229]: scrub queued for path: / 2023-07-18T03:10:45.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:45 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:10:45.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:45 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:10:45.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:45 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:10:45.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:45 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:10:45.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:45 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:10:45.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:45 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:10:45.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:45 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:10:45.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:45 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:10:45.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:45 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:10:45.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:45 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:10:45.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:45 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:10:45.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:45 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:10:45.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:45 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:10:45.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:45 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:10:45.485 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:45 smithi101 ceph-mon[126379]: pgmap v98: 129 pgs: 129 active+clean; 116 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 205 op/s 2023-07-18T03:10:45.485 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:45 smithi101 ceph-mon[126379]: scrub queued for path: / 2023-07-18T03:10:45.485 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:45 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:10:45.485 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:45 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:10:45.486 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:45 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:10:45.486 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:45 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:10:45.486 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:45 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:10:45.486 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:45 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:10:45.486 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:45 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:10:45.486 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:45 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:10:45.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:45 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:10:45.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:45 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:10:45.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:45 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:10:45.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:45 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:10:45.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:45 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:10:45.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:45 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:10:45.571 INFO:tasks.workunit.client.0.smithi101.stdout: 1 197503 8.52 MB/sec execute 10 sec latency 20.582 ms 2023-07-18T03:10:46.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:46 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:10:46.293 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:46 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:10:46.293 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:46 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:10:46.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:46 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:10:46.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:46 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:10:46.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:46 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:10:46.485 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:46 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:10:46.485 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:46 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:10:46.485 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:46 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:10:46.571 INFO:tasks.workunit.client.0.smithi101.stdout: 1 199080 8.84 MB/sec execute 11 sec latency 10.724 ms 2023-07-18T03:10:47.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:47 smithi155 ceph-mon[128885]: pgmap v99: 129 pgs: 129 active+clean; 120 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 312 op/s 2023-07-18T03:10:47.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:47 smithi179 ceph-mon[129229]: pgmap v99: 129 pgs: 129 active+clean; 120 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 312 op/s 2023-07-18T03:10:47.485 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:47 smithi101 ceph-mon[126379]: pgmap v99: 129 pgs: 129 active+clean; 120 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 312 op/s 2023-07-18T03:10:47.571 INFO:tasks.workunit.client.0.smithi101.stdout: 1 201131 8.75 MB/sec execute 12 sec latency 13.947 ms 2023-07-18T03:10:48.571 INFO:tasks.workunit.client.0.smithi101.stdout: 1 202550 8.95 MB/sec execute 13 sec latency 47.605 ms 2023-07-18T03:10:49.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:49 smithi155 ceph-mon[128885]: pgmap v100: 129 pgs: 129 active+clean; 120 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 211 op/s 2023-07-18T03:10:49.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:49 smithi179 ceph-mon[129229]: pgmap v100: 129 pgs: 129 active+clean; 120 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 211 op/s 2023-07-18T03:10:49.485 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:49 smithi101 ceph-mon[126379]: pgmap v100: 129 pgs: 129 active+clean; 120 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 211 op/s 2023-07-18T03:10:49.571 INFO:tasks.workunit.client.0.smithi101.stdout: 1 204223 8.71 MB/sec execute 14 sec latency 28.883 ms 2023-07-18T03:10:50.571 INFO:tasks.workunit.client.0.smithi101.stdout: 1 206057 9.01 MB/sec execute 15 sec latency 10.679 ms 2023-07-18T03:10:51.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:51 smithi155 ceph-mon[128885]: pgmap v101: 129 pgs: 129 active+clean; 125 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 325 op/s 2023-07-18T03:10:51.293 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:51 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:10:51.293 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:51 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:10:51.293 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:51 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:10:51.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:51 smithi179 ceph-mon[129229]: pgmap v101: 129 pgs: 129 active+clean; 125 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 325 op/s 2023-07-18T03:10:51.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:51 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:10:51.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:51 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:10:51.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:51 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:10:51.485 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:51 smithi101 ceph-mon[126379]: pgmap v101: 129 pgs: 129 active+clean; 125 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 325 op/s 2023-07-18T03:10:51.485 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:51 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:10:51.485 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:51 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:10:51.485 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:51 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:10:51.572 INFO:tasks.workunit.client.0.smithi101.stdout: 1 207707 8.81 MB/sec execute 16 sec latency 15.303 ms 2023-07-18T03:10:52.572 INFO:tasks.workunit.client.0.smithi101.stdout: 1 209596 9.08 MB/sec execute 17 sec latency 48.231 ms 2023-07-18T03:10:53.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:53 smithi155 ceph-mon[128885]: pgmap v102: 129 pgs: 129 active+clean; 125 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 219 op/s 2023-07-18T03:10:53.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:53 smithi179 ceph-mon[129229]: pgmap v102: 129 pgs: 129 active+clean; 125 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 219 op/s 2023-07-18T03:10:53.485 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:53 smithi101 ceph-mon[126379]: pgmap v102: 129 pgs: 129 active+clean; 125 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 219 op/s 2023-07-18T03:10:53.572 INFO:tasks.workunit.client.0.smithi101.stdout: 1 210836 8.88 MB/sec execute 18 sec latency 41.318 ms 2023-07-18T03:10:54.572 INFO:tasks.workunit.client.0.smithi101.stdout: 1 213107 9.13 MB/sec execute 19 sec latency 10.944 ms 2023-07-18T03:10:55.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:55 smithi155 ceph-mon[128885]: pgmap v103: 129 pgs: 129 active+clean; 125 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 219 op/s 2023-07-18T03:10:55.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:55 smithi179 ceph-mon[129229]: pgmap v103: 129 pgs: 129 active+clean; 125 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 219 op/s 2023-07-18T03:10:55.485 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:55 smithi101 ceph-mon[126379]: pgmap v103: 129 pgs: 129 active+clean; 125 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 219 op/s 2023-07-18T03:10:55.572 INFO:tasks.workunit.client.0.smithi101.stdout: 1 214715 8.96 MB/sec execute 20 sec latency 17.122 ms 2023-07-18T03:10:56.572 INFO:tasks.workunit.client.0.smithi101.stdout: 1 216760 9.19 MB/sec execute 21 sec latency 10.308 ms 2023-07-18T03:10:57.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:57 smithi155 ceph-mon[128885]: pgmap v104: 129 pgs: 129 active+clean; 130 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 329 op/s 2023-07-18T03:10:57.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:57 smithi179 ceph-mon[129229]: pgmap v104: 129 pgs: 129 active+clean; 130 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 329 op/s 2023-07-18T03:10:57.485 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:57 smithi101 ceph-mon[126379]: pgmap v104: 129 pgs: 129 active+clean; 130 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 329 op/s 2023-07-18T03:10:57.572 INFO:tasks.workunit.client.0.smithi101.stdout: 1 218562 9.03 MB/sec execute 22 sec latency 15.845 ms 2023-07-18T03:10:58.572 INFO:tasks.workunit.client.0.smithi101.stdout: 1 220209 9.21 MB/sec execute 23 sec latency 40.327 ms 2023-07-18T03:10:59.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:10:59 smithi155 ceph-mon[128885]: pgmap v105: 129 pgs: 129 active+clean; 130 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 223 op/s 2023-07-18T03:10:59.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:10:59 smithi179 ceph-mon[129229]: pgmap v105: 129 pgs: 129 active+clean; 130 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 223 op/s 2023-07-18T03:10:59.485 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:10:59 smithi101 ceph-mon[126379]: pgmap v105: 129 pgs: 129 active+clean; 130 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 223 op/s 2023-07-18T03:10:59.572 INFO:tasks.workunit.client.0.smithi101.stdout: 1 221524 9.05 MB/sec execute 24 sec latency 17.375 ms 2023-07-18T03:11:00.572 INFO:tasks.workunit.client.0.smithi101.stdout: 1 223681 9.19 MB/sec execute 25 sec latency 69.706 ms 2023-07-18T03:11:01.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:01 smithi155 ceph-mon[128885]: pgmap v106: 129 pgs: 129 active+clean; 134 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 331 op/s 2023-07-18T03:11:01.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:01 smithi179 ceph-mon[129229]: pgmap v106: 129 pgs: 129 active+clean; 134 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 331 op/s 2023-07-18T03:11:01.485 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:01 smithi101 ceph-mon[126379]: pgmap v106: 129 pgs: 129 active+clean; 134 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 331 op/s 2023-07-18T03:11:01.572 INFO:tasks.workunit.client.0.smithi101.stdout: 1 223960 8.89 MB/sec execute 26 sec latency 52.608 ms 2023-07-18T03:11:02.572 INFO:tasks.workunit.client.0.smithi101.stdout: 1 226067 8.86 MB/sec execute 27 sec latency 13.747 ms 2023-07-18T03:11:03.292 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:03 smithi155 ceph-mon[128885]: pgmap v107: 129 pgs: 129 active+clean; 134 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 218 op/s 2023-07-18T03:11:03.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:03 smithi179 ceph-mon[129229]: pgmap v107: 129 pgs: 129 active+clean; 134 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 218 op/s 2023-07-18T03:11:03.486 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:03 smithi101 ceph-mon[126379]: pgmap v107: 129 pgs: 129 active+clean; 134 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 218 op/s 2023-07-18T03:11:03.572 INFO:tasks.workunit.client.0.smithi101.stdout: 1 227469 8.94 MB/sec execute 28 sec latency 35.700 ms 2023-07-18T03:11:04.572 INFO:tasks.workunit.client.0.smithi101.stdout: 1 229485 8.88 MB/sec execute 29 sec latency 21.743 ms 2023-07-18T03:11:05.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:05 smithi179 ceph-mon[129229]: pgmap v108: 129 pgs: 129 active+clean; 134 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 218 op/s 2023-07-18T03:11:05.485 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:05 smithi101 ceph-mon[126379]: pgmap v108: 129 pgs: 129 active+clean; 134 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 218 op/s 2023-07-18T03:11:05.542 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:05 smithi155 ceph-mon[128885]: pgmap v108: 129 pgs: 129 active+clean; 134 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 218 op/s 2023-07-18T03:11:05.572 INFO:tasks.workunit.client.0.smithi101.stdout: 1 231021 8.98 MB/sec execute 30 sec latency 23.018 ms 2023-07-18T03:11:06.572 INFO:tasks.workunit.client.0.smithi101.stdout: 1 233095 8.94 MB/sec execute 31 sec latency 14.743 ms 2023-07-18T03:11:07.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:07 smithi179 ceph-mon[129229]: pgmap v109: 129 pgs: 129 active+clean; 138 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 350 op/s 2023-07-18T03:11:07.485 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:07 smithi101 ceph-mon[126379]: pgmap v109: 129 pgs: 129 active+clean; 138 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 350 op/s 2023-07-18T03:11:07.542 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:07 smithi155 ceph-mon[128885]: pgmap v109: 129 pgs: 129 active+clean; 138 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 350 op/s 2023-07-18T03:11:07.572 INFO:tasks.workunit.client.0.smithi101.stdout: 1 234613 9.02 MB/sec execute 32 sec latency 11.321 ms 2023-07-18T03:11:08.573 INFO:tasks.workunit.client.0.smithi101.stdout: 1 236549 8.96 MB/sec execute 33 sec latency 58.997 ms 2023-07-18T03:11:09.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:09 smithi179 ceph-mon[129229]: pgmap v110: 129 pgs: 129 active+clean; 138 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 240 op/s 2023-07-18T03:11:09.485 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:09 smithi101 ceph-mon[126379]: pgmap v110: 129 pgs: 129 active+clean; 138 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 240 op/s 2023-07-18T03:11:09.542 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:09 smithi155 ceph-mon[128885]: pgmap v110: 129 pgs: 129 active+clean; 138 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 240 op/s 2023-07-18T03:11:09.573 INFO:tasks.workunit.client.0.smithi101.stdout: 1 238057 9.05 MB/sec execute 34 sec latency 67.269 ms 2023-07-18T03:11:10.573 INFO:tasks.workunit.client.0.smithi101.stdout: 1 239345 8.94 MB/sec execute 35 sec latency 33.925 ms 2023-07-18T03:11:11.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:11 smithi179 ceph-mon[129229]: pgmap v111: 129 pgs: 129 active+clean; 144 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 347 op/s 2023-07-18T03:11:11.485 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:11 smithi101 ceph-mon[126379]: pgmap v111: 129 pgs: 129 active+clean; 144 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 347 op/s 2023-07-18T03:11:11.542 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:11 smithi155 ceph-mon[128885]: pgmap v111: 129 pgs: 129 active+clean; 144 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 347 op/s 2023-07-18T03:11:11.573 INFO:tasks.workunit.client.0.smithi101.stdout: 1 241508 9.07 MB/sec execute 36 sec latency 11.653 ms 2023-07-18T03:11:12.573 INFO:tasks.workunit.client.0.smithi101.stdout: 1 242769 8.97 MB/sec execute 37 sec latency 16.837 ms 2023-07-18T03:11:13.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:13 smithi179 ceph-mon[129229]: pgmap v112: 129 pgs: 129 active+clean; 144 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 239 op/s 2023-07-18T03:11:13.485 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:13 smithi101 ceph-mon[126379]: pgmap v112: 129 pgs: 129 active+clean; 144 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 239 op/s 2023-07-18T03:11:13.543 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:13 smithi155 ceph-mon[128885]: pgmap v112: 129 pgs: 129 active+clean; 144 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 239 op/s 2023-07-18T03:11:13.573 INFO:tasks.workunit.client.0.smithi101.stdout: 1 244995 9.09 MB/sec execute 38 sec latency 33.480 ms 2023-07-18T03:11:14.444 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:11:14.573 INFO:tasks.workunit.client.0.smithi101.stdout: 1 245970 8.92 MB/sec execute 39 sec latency 32.414 ms 2023-07-18T03:11:14.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.669+0000 7f6a5d7ff700 1 -- 172.21.15.101:0/4184429465 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a58073160 msgr2=0x7f6a5810be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:14.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.669+0000 7f6a5d7ff700 1 --2- 172.21.15.101:0/4184429465 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a58073160 0x7f6a5810be60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c009990 tx=0x7f6a4c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:11:14.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.669+0000 7f6a5d7ff700 1 -- 172.21.15.101:0/4184429465 shutdown_connections 2023-07-18T03:11:14.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.669+0000 7f6a5d7ff700 1 --2- 172.21.15.101:0/4184429465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a58072850 0x7f6a58072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:14.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.669+0000 7f6a5d7ff700 1 --2- 172.21.15.101:0/4184429465 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6a5810c3a0 0x7f6a5810e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:14.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.669+0000 7f6a5d7ff700 1 --2- 172.21.15.101:0/4184429465 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a58073160 0x7f6a5810be60 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:14.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.669+0000 7f6a5d7ff700 1 -- 172.21.15.101:0/4184429465 >> 172.21.15.101:0/4184429465 conn(0x7f6a580fb600 msgr2=0x7f6a580fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:14.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.669+0000 7f6a5d7ff700 1 -- 172.21.15.101:0/4184429465 shutdown_connections 2023-07-18T03:11:14.662 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.669+0000 7f6a5d7ff700 1 -- 172.21.15.101:0/4184429465 wait complete. 2023-07-18T03:11:14.663 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.670+0000 7f6a5d7ff700 1 Processor -- start 2023-07-18T03:11:14.663 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.670+0000 7f6a5d7ff700 1 -- start start 2023-07-18T03:11:14.663 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.670+0000 7f6a5d7ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6a58072850 0x7f6a58197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:14.663 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.670+0000 7f6a5d7ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a58073160 0x7f6a581983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:14.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.670+0000 7f6a5d7ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a5810c3a0 0x7f6a5819c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:14.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.670+0000 7f6a5d7ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5819ce90 con 0x7f6a58072850 2023-07-18T03:11:14.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.670+0000 7f6a5d7ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5819d000 con 0x7f6a5810c3a0 2023-07-18T03:11:14.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.670+0000 7f6a5d7ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5819d1c0 con 0x7f6a58073160 2023-07-18T03:11:14.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.671+0000 7f6a56ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6a58072850 0x7f6a58197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:14.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.671+0000 7f6a567fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a58073160 0x7f6a581983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:14.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.671+0000 7f6a577fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a5810c3a0 0x7f6a5819c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:14.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.671+0000 7f6a567fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a58073160 0x7f6a581983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:36134/0 (socket says 172.21.15.101:36134) 2023-07-18T03:11:14.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.671+0000 7f6a577fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a5810c3a0 0x7f6a5819c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:58180/0 (socket says 172.21.15.101:58180) 2023-07-18T03:11:14.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.671+0000 7f6a567fc700 1 -- 172.21.15.101:0/3896562400 learned_addr learned my addr 172.21.15.101:0/3896562400 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:14.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.671+0000 7f6a56ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6a58072850 0x7f6a58197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:42376/0 (socket says 172.21.15.101:42376) 2023-07-18T03:11:14.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.671+0000 7f6a577fe700 1 -- 172.21.15.101:0/3896562400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a58073160 msgr2=0x7f6a581983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:14.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.671+0000 7f6a577fe700 1 --2- 172.21.15.101:0/3896562400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a58073160 0x7f6a581983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:14.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.671+0000 7f6a577fe700 1 -- 172.21.15.101:0/3896562400 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6a58072850 msgr2=0x7f6a58197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:14.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.671+0000 7f6a577fe700 1 --2- 172.21.15.101:0/3896562400 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6a58072850 0x7f6a58197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:14.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.671+0000 7f6a577fe700 1 -- 172.21.15.101:0/3896562400 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a4c0095f0 con 0x7f6a5810c3a0 2023-07-18T03:11:14.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.671+0000 7f6a56ffd700 1 --2- 172.21.15.101:0/3896562400 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6a58072850 0x7f6a58197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:11:14.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.671+0000 7f6a567fc700 1 --2- 172.21.15.101:0/3896562400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a58073160 0x7f6a581983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:11:14.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.671+0000 7f6a577fe700 1 --2- 172.21.15.101:0/3896562400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a5810c3a0 0x7f6a5819c850 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6a58074920 tx=0x7f6a5000ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:14.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.672+0000 7f6a3ffff700 1 -- 172.21.15.101:0/3896562400 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a50018070 con 0x7f6a5810c3a0 2023-07-18T03:11:14.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.672+0000 7f6a3ffff700 1 -- 172.21.15.101:0/3896562400 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a5000f040 con 0x7f6a5810c3a0 2023-07-18T03:11:14.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.672+0000 7f6a3ffff700 1 -- 172.21.15.101:0/3896562400 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5000cd80 con 0x7f6a5810c3a0 2023-07-18T03:11:14.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.672+0000 7f6a5d7ff700 1 -- 172.21.15.101:0/3896562400 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a5819d440 con 0x7f6a5810c3a0 2023-07-18T03:11:14.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.672+0000 7f6a5d7ff700 1 -- 172.21.15.101:0/3896562400 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a581a4ce0 con 0x7f6a5810c3a0 2023-07-18T03:11:14.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.675+0000 7f6a3ffff700 1 -- 172.21.15.101:0/3896562400 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f6a50003a00 con 0x7f6a5810c3a0 2023-07-18T03:11:14.668 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.676+0000 7f6a71194b80 1 Processor -- start 2023-07-18T03:11:14.668 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.677+0000 7f6a71194b80 1 -- start start 2023-07-18T03:11:14.668 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.677+0000 7f6a3ffff700 1 --2- 172.21.15.101:0/3896562400 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6a400a12c0 0x7f6a400a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:14.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.677+0000 7f6a3ffff700 1 -- 172.21.15.101:0/3896562400 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f6a500ccc80 con 0x7f6a5810c3a0 2023-07-18T03:11:14.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.677+0000 7f6a71194b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5601a04602d0 0x5601a04606a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:14.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.677+0000 7f6a71194b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601a0460be0 0x5601a046b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:14.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.677+0000 7f6a71194b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601a046bad0 0x5601a046deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:14.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.677+0000 7f6a71194b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601a0366cb0 con 0x5601a04602d0 2023-07-18T03:11:14.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.677+0000 7f6a71194b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601a03b9a40 con 0x5601a046bad0 2023-07-18T03:11:14.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.677+0000 7f6a71194b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601a03b2ed0 con 0x5601a0460be0 2023-07-18T03:11:14.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.677+0000 7f6a567fc700 1 --2- 172.21.15.101:0/3896562400 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6a400a12c0 0x7f6a400a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:14.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.678+0000 7f6a577fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601a046bad0 0x5601a046deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:14.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.678+0000 7f6a567fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601a0460be0 0x5601a046b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:14.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.678+0000 7f6a56ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5601a04602d0 0x5601a04606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:14.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.678+0000 7f6a577fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601a046bad0 0x5601a046deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:58192/0 (socket says 172.21.15.101:58192) 2023-07-18T03:11:14.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.678+0000 7f6a577fe700 1 -- 172.21.15.101:0/409137615 learned_addr learned my addr 172.21.15.101:0/409137615 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:14.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.678+0000 7f6a577fe700 1 -- 172.21.15.101:0/409137615 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601a0460be0 msgr2=0x5601a046b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:14.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.678+0000 7f6a577fe700 1 --2- 172.21.15.101:0/409137615 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601a0460be0 0x5601a046b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:14.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.678+0000 7f6a577fe700 1 -- 172.21.15.101:0/409137615 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5601a04602d0 msgr2=0x5601a04606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:14.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.678+0000 7f6a577fe700 1 --2- 172.21.15.101:0/409137615 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5601a04602d0 0x5601a04606a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:14.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.678+0000 7f6a577fe700 1 -- 172.21.15.101:0/409137615 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601a01e0de0 con 0x5601a046bad0 2023-07-18T03:11:14.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.678+0000 7f6a56ffd700 1 --2- 172.21.15.101:0/409137615 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5601a04602d0 0x5601a04606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:11:14.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.678+0000 7f6a567fc700 1 --2- 172.21.15.101:0/3896562400 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6a400a12c0 0x7f6a400a3770 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c00b2e0 tx=0x7f6a4c00b930 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:14.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.678+0000 7f6a577fe700 1 --2- 172.21.15.101:0/409137615 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601a046bad0 0x5601a046deb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6a500090a0 tx=0x7f6a50095e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f20b514c41e37007 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:14.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.678+0000 7f6a3d7fa700 1 -- 172.21.15.101:0/409137615 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a50099390 con 0x5601a046bad0 2023-07-18T03:11:14.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.678+0000 7f6a3d7fa700 1 -- 172.21.15.101:0/409137615 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a5009ae90 con 0x5601a046bad0 2023-07-18T03:11:14.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.678+0000 7f6a3d7fa700 1 -- 172.21.15.101:0/409137615 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a50099c90 con 0x5601a046bad0 2023-07-18T03:11:14.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.679+0000 7f6a71194b80 1 -- 172.21.15.101:0/409137615 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601a046bad0 msgr2=0x5601a046deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:14.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.679+0000 7f6a71194b80 1 --2- 172.21.15.101:0/409137615 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601a046bad0 0x5601a046deb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6a500090a0 tx=0x7f6a50095e40 comp rx=0 tx=0).stop 2023-07-18T03:11:14.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.679+0000 7f6a71194b80 1 -- 172.21.15.101:0/409137615 shutdown_connections 2023-07-18T03:11:14.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.679+0000 7f6a71194b80 1 --2- 172.21.15.101:0/409137615 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601a046bad0 0x5601a046deb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:14.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.679+0000 7f6a71194b80 1 --2- 172.21.15.101:0/409137615 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5601a04602d0 0x5601a04606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:14.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.679+0000 7f6a71194b80 1 --2- 172.21.15.101:0/409137615 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601a0460be0 0x5601a046b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:14.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.679+0000 7f6a71194b80 1 -- 172.21.15.101:0/409137615 >> 172.21.15.101:0/409137615 conn(0x5601a0456da0 msgr2=0x5601a045c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:14.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.679+0000 7f6a71194b80 1 -- 172.21.15.101:0/409137615 shutdown_connections 2023-07-18T03:11:14.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.679+0000 7f6a71194b80 1 -- 172.21.15.101:0/409137615 wait complete. 2023-07-18T03:11:14.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.680+0000 7f6a71194b80 1 -- start start 2023-07-18T03:11:14.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.680+0000 7f6a71194b80 1 Processor -- start 2023-07-18T03:11:14.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.681+0000 7f6a71194b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601a04602d0 0x5601a046b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:14.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.681+0000 7f6a71194b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5601a0460be0 0x5601a04687e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:14.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.681+0000 7f6a71194b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601a046bad0 0x5601a0468f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:14.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.681+0000 7f6a71194b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601a0352ac0 con 0x5601a0460be0 2023-07-18T03:11:14.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.681+0000 7f6a71194b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601a03527c0 con 0x5601a046bad0 2023-07-18T03:11:14.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.681+0000 7f6a71194b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601a0236920 con 0x5601a04602d0 2023-07-18T03:11:14.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.681+0000 7f6a71194b80 10 client.0 _ll_register_callbacks cb 0x5601a0350760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:11:14.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.681+0000 7f6a56ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601a04602d0 0x5601a046b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:14.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.681+0000 7f6a567fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5601a0460be0 0x5601a04687e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:14.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.681+0000 7f6a56ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601a04602d0 0x5601a046b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:36162/0 (socket says 172.21.15.101:36162) 2023-07-18T03:11:14.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.681+0000 7f6a56ffd700 1 -- 172.21.15.101:0/3920677972 learned_addr learned my addr 172.21.15.101:0/3920677972 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:14.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.681+0000 7f6a577fe700 1 --2- 172.21.15.101:0/3920677972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601a046bad0 0x5601a0468f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:14.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.681+0000 7f6a567fc700 1 -- 172.21.15.101:0/3920677972 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601a04602d0 msgr2=0x5601a046b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:14.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.681+0000 7f6a567fc700 1 --2- 172.21.15.101:0/3920677972 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601a04602d0 0x5601a046b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:14.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.681+0000 7f6a567fc700 1 -- 172.21.15.101:0/3920677972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601a046bad0 msgr2=0x5601a0468f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:14.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.681+0000 7f6a567fc700 1 --2- 172.21.15.101:0/3920677972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601a046bad0 0x5601a0468f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:14.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.681+0000 7f6a567fc700 1 -- 172.21.15.101:0/3920677972 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a48009c90 con 0x5601a0460be0 2023-07-18T03:11:14.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.681+0000 7f6a577fe700 1 --2- 172.21.15.101:0/3920677972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601a046bad0 0x5601a0468f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:11:14.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.682+0000 7f6a567fc700 1 --2- 172.21.15.101:0/3920677972 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5601a0460be0 0x5601a04687e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c005910 tx=0x7f6a4c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=189350f8fd052dd6 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:14.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.682+0000 7f6a3aff5700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:11:14.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.682+0000 7f6a3aff5700 1 -- 172.21.15.101:0/3920677972 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c005050 con 0x5601a0460be0 2023-07-18T03:11:14.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.682+0000 7f6a3aff5700 1 -- 172.21.15.101:0/3920677972 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f6a4c0051b0 con 0x5601a0460be0 2023-07-18T03:11:14.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.682+0000 7f6a71194b80 1 -- 172.21.15.101:0/3920677972 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5601a030c4f0 con 0x5601a0460be0 2023-07-18T03:11:14.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.682+0000 7f6a3aff5700 1 -- 172.21.15.101:0/3920677972 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a4c005370 con 0x5601a0460be0 2023-07-18T03:11:14.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.682+0000 7f6a71194b80 10 client.16233 fetch_fsmap learned FSMap version 25 2023-07-18T03:11:14.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.682+0000 7f6a71194b80 1 -- 172.21.15.101:0/3920677972 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f6a581a4ce0 con 0x5601a0460be0 2023-07-18T03:11:14.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.682+0000 7f6a3aff5700 1 -- 172.21.15.101:0/3920677972 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5601a030c4f0 con 0x5601a0460be0 2023-07-18T03:11:14.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.683+0000 7f6a3aff5700 1 -- 172.21.15.101:0/3920677972 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f6a4c030020 con 0x5601a0460be0 2023-07-18T03:11:14.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.683+0000 7f6a71194b80 10 client.16233 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:11:14.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.683+0000 7f6a71194b80 10 client.16233 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:11:14.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.683+0000 7f6a71194b80 20 client.16233 populate_metadata read hostname 'smithi101' 2023-07-18T03:11:14.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.683+0000 7f6a71194b80 1 --2- 172.21.15.101:0/3920677972 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5601a047b2b0 0x5601a047d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:14.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.683+0000 7f6a71194b80 4 client.16233 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:11:14.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.683+0000 7f6a71194b80 1 -- 172.21.15.101:0/3920677972 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5601a03abeb0 con 0x5601a047b2b0 2023-07-18T03:11:14.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.683+0000 7f6a56ffd700 1 --2- 172.21.15.101:0/3920677972 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5601a047b2b0 0x5601a047d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:14.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.684+0000 7f6a56ffd700 1 --2- 172.21.15.101:0/3920677972 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5601a047b2b0 0x5601a047d6c0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c70cbdc20ecc7cbe server_cookie=4a9711e45dd46ca9 in_seq=0 out_seq=0 2023-07-18T03:11:14.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.684+0000 7f6a3aff5700 10 client.16233 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:11:14.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.684+0000 7f6a3aff5700 1 -- 172.21.15.101:0/3920677972 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5601a03abeb0 con 0x5601a047b2b0 2023-07-18T03:11:14.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.684+0000 7f6a3aff5700 10 client.16233 handle_command_reply: tid=0 2023-07-18T03:11:14.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.684+0000 7f6a71194b80 1 client.16233 shutdown 2023-07-18T03:11:14.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.684+0000 7f6a71194b80 1 -- 172.21.15.101:0/3920677972 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5601a0460be0 msgr2=0x5601a04687e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:14.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.684+0000 7f6a71194b80 1 --2- 172.21.15.101:0/3920677972 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5601a0460be0 0x5601a04687e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c005910 tx=0x7f6a4c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:11:14.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.685+0000 7f6a71194b80 1 -- 172.21.15.101:0/3920677972 shutdown_connections 2023-07-18T03:11:14.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.685+0000 7f6a71194b80 1 --2- 172.21.15.101:0/3920677972 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5601a047b2b0 0x5601a047d6c0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:14.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.685+0000 7f6a71194b80 1 --2- 172.21.15.101:0/3920677972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601a046bad0 0x5601a0468f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:14.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.685+0000 7f6a71194b80 1 --2- 172.21.15.101:0/3920677972 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601a04602d0 0x5601a046b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:14.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.685+0000 7f6a71194b80 1 --2- 172.21.15.101:0/3920677972 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5601a0460be0 0x5601a04687e0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:14.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.685+0000 7f6a71194b80 1 -- 172.21.15.101:0/3920677972 >> 172.21.15.101:0/3920677972 conn(0x5601a0456da0 msgr2=0x5601a045c230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:14.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.685+0000 7f6a3aff5700 0 client.16233 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:11:14.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.685+0000 7f6a71194b80 1 -- 172.21.15.101:0/3920677972 shutdown_connections 2023-07-18T03:11:14.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.685+0000 7f6a71194b80 1 -- 172.21.15.101:0/3920677972 wait complete. 2023-07-18T03:11:14.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.685+0000 7f6a71194b80 20 client.16233 trim_cache size 0 max 16384 2023-07-18T03:11:14.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.703+0000 7f6a71194b80 1 Processor -- start 2023-07-18T03:11:14.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.703+0000 7f6a71194b80 1 -- start start 2023-07-18T03:11:14.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.704+0000 7f6a71194b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601a04602d0 0x5601a0463b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:14.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.704+0000 7f6a71194b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601a0460be0 0x5601a0464070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:14.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.704+0000 7f6a71194b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5601a046bad0 0x5601a0467fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:14.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.704+0000 7f6a71194b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601a04615b0 con 0x5601a046bad0 2023-07-18T03:11:14.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.704+0000 7f6a71194b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601a04636b0 con 0x5601a04602d0 2023-07-18T03:11:14.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.704+0000 7f6a71194b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601a045b510 con 0x5601a0460be0 2023-07-18T03:11:14.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.704+0000 7f6a567fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601a0460be0 0x5601a0464070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:14.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.704+0000 7f6a56ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601a04602d0 0x5601a0463b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:14.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.704+0000 7f6a56ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601a04602d0 0x5601a0463b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:58210/0 (socket says 172.21.15.101:58210) 2023-07-18T03:11:14.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.704+0000 7f6a567fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601a0460be0 0x5601a0464070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:36176/0 (socket says 172.21.15.101:36176) 2023-07-18T03:11:14.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.704+0000 7f6a56ffd700 1 -- 172.21.15.101:0/453520387 learned_addr learned my addr 172.21.15.101:0/453520387 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:14.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.704+0000 7f6a577fe700 1 --2- 172.21.15.101:0/453520387 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5601a046bad0 0x5601a0467fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:14.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.704+0000 7f6a56ffd700 1 -- 172.21.15.101:0/453520387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601a0460be0 msgr2=0x5601a0464070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:14.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.704+0000 7f6a56ffd700 1 --2- 172.21.15.101:0/453520387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601a0460be0 0x5601a0464070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:14.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.704+0000 7f6a56ffd700 1 -- 172.21.15.101:0/453520387 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5601a046bad0 msgr2=0x5601a0467fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:14.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.704+0000 7f6a56ffd700 1 --2- 172.21.15.101:0/453520387 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5601a046bad0 0x5601a0467fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:14.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.704+0000 7f6a56ffd700 1 -- 172.21.15.101:0/453520387 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601a04585e0 con 0x5601a04602d0 2023-07-18T03:11:14.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.705+0000 7f6a56ffd700 1 --2- 172.21.15.101:0/453520387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601a04602d0 0x5601a0463b30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6a48005e90 tx=0x7f6a480139c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d574064392c1c573 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:14.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.705+0000 7f6a3dffb700 1 -- 172.21.15.101:0/453520387 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4801e9d0 con 0x5601a04602d0 2023-07-18T03:11:14.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.705+0000 7f6a3dffb700 1 -- 172.21.15.101:0/453520387 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a48018040 con 0x5601a04602d0 2023-07-18T03:11:14.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.705+0000 7f6a3dffb700 1 -- 172.21.15.101:0/453520387 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a48003970 con 0x5601a04602d0 2023-07-18T03:11:14.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.706+0000 7f6a71194b80 1 -- 172.21.15.101:0/453520387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601a04602d0 msgr2=0x5601a0463b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:14.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.706+0000 7f6a71194b80 1 --2- 172.21.15.101:0/453520387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601a04602d0 0x5601a0463b30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6a48005e90 tx=0x7f6a480139c0 comp rx=0 tx=0).stop 2023-07-18T03:11:14.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.706+0000 7f6a71194b80 1 -- 172.21.15.101:0/453520387 shutdown_connections 2023-07-18T03:11:14.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.706+0000 7f6a71194b80 1 --2- 172.21.15.101:0/453520387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601a04602d0 0x5601a0463b30 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:14.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.706+0000 7f6a71194b80 1 --2- 172.21.15.101:0/453520387 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5601a046bad0 0x5601a0467fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:14.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.706+0000 7f6a71194b80 1 --2- 172.21.15.101:0/453520387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601a0460be0 0x5601a0464070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:14.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.706+0000 7f6a71194b80 1 -- 172.21.15.101:0/453520387 >> 172.21.15.101:0/453520387 conn(0x5601a0456da0 msgr2=0x5601a047d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:14.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.706+0000 7f6a71194b80 1 -- 172.21.15.101:0/453520387 shutdown_connections 2023-07-18T03:11:14.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.706+0000 7f6a71194b80 1 -- 172.21.15.101:0/453520387 wait complete. 2023-07-18T03:11:14.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.706+0000 7f6a71194b80 1 -- start start 2023-07-18T03:11:14.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.706+0000 7f6a71194b80 1 Processor -- start 2023-07-18T03:11:14.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.707+0000 7f6a71194b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5601a04602d0 0x5601a04724c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:14.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.707+0000 7f6a71194b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601a0460be0 0x5601a04708e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:14.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.707+0000 7f6a71194b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601a046bad0 0x5601a0470e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:14.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.707+0000 7f6a71194b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601a0471500 con 0x5601a04602d0 2023-07-18T03:11:14.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.707+0000 7f6a71194b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601a0471670 con 0x5601a046bad0 2023-07-18T03:11:14.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.707+0000 7f6a71194b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601a04cf140 con 0x5601a0460be0 2023-07-18T03:11:14.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.707+0000 7f6a71194b80 10 client.0 _ll_register_callbacks cb 0x5601a0350760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:11:14.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.707+0000 7f6a577fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601a046bad0 0x5601a0470e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:14.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.707+0000 7f6a567fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601a0460be0 0x5601a04708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:14.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.707+0000 7f6a577fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601a046bad0 0x5601a0470e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:58222/0 (socket says 172.21.15.101:58222) 2023-07-18T03:11:14.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.707+0000 7f6a567fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601a0460be0 0x5601a04708e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:36190/0 (socket says 172.21.15.101:36190) 2023-07-18T03:11:14.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.707+0000 7f6a56ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5601a04602d0 0x5601a04724c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:14.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.707+0000 7f6a577fe700 1 -- 172.21.15.101:0/3976954835 learned_addr learned my addr 172.21.15.101:0/3976954835 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:14.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.708+0000 7f6a577fe700 1 -- 172.21.15.101:0/3976954835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601a0460be0 msgr2=0x5601a04708e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:14.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.708+0000 7f6a577fe700 1 --2- 172.21.15.101:0/3976954835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601a0460be0 0x5601a04708e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:14.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.708+0000 7f6a577fe700 1 -- 172.21.15.101:0/3976954835 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5601a04602d0 msgr2=0x5601a04724c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:14.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.708+0000 7f6a577fe700 1 --2- 172.21.15.101:0/3976954835 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5601a04602d0 0x5601a04724c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:14.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.708+0000 7f6a577fe700 1 -- 172.21.15.101:0/3976954835 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a480132c0 con 0x5601a046bad0 2023-07-18T03:11:14.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.708+0000 7f6a567fc700 1 --2- 172.21.15.101:0/3976954835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601a0460be0 0x5601a04708e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:11:14.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.708+0000 7f6a577fe700 1 --2- 172.21.15.101:0/3976954835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601a046bad0 0x5601a0470e20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6a50095f30 tx=0x7f6a50009ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb85243e560c49b3 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:14.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.708+0000 7f6a3aff5700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:11:14.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.708+0000 7f6a3aff5700 1 -- 172.21.15.101:0/3976954835 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a500b7070 con 0x5601a046bad0 2023-07-18T03:11:14.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.708+0000 7f6a71194b80 1 -- 172.21.15.101:0/3976954835 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a5009ae90 con 0x5601a046bad0 2023-07-18T03:11:14.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.708+0000 7f6a3aff5700 1 -- 172.21.15.101:0/3976954835 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f6a50004ad0 con 0x5601a046bad0 2023-07-18T03:11:14.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.708+0000 7f6a3aff5700 1 -- 172.21.15.101:0/3976954835 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a5009da60 con 0x5601a046bad0 2023-07-18T03:11:14.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.708+0000 7f6a71194b80 10 client.26285 fetch_fsmap learned FSMap version 25 2023-07-18T03:11:14.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.708+0000 7f6a71194b80 1 -- 172.21.15.101:0/3976954835 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5601a01c9c60 con 0x5601a046bad0 2023-07-18T03:11:14.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.708+0000 7f6a3aff5700 1 -- 172.21.15.101:0/3976954835 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a50063020 con 0x5601a046bad0 2023-07-18T03:11:14.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.709+0000 7f6a3aff5700 1 -- 172.21.15.101:0/3976954835 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f6a500c7030 con 0x5601a046bad0 2023-07-18T03:11:14.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.709+0000 7f6a71194b80 10 client.26285 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:11:14.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.709+0000 7f6a71194b80 10 client.26285 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:11:14.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.709+0000 7f6a71194b80 20 client.26285 populate_metadata read hostname 'smithi101' 2023-07-18T03:11:14.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.709+0000 7f6a71194b80 1 --2- 172.21.15.101:0/3976954835 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5601a04d0360 0x5601a04d2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:14.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.709+0000 7f6a71194b80 4 client.26285 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:11:14.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.709+0000 7f6a71194b80 1 -- 172.21.15.101:0/3976954835 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5601a036e050 con 0x5601a04d0360 2023-07-18T03:11:14.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.709+0000 7f6a56ffd700 1 --2- 172.21.15.101:0/3976954835 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5601a04d0360 0x5601a04d2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:14.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.709+0000 7f6a3aff5700 10 client.26285 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:11:14.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.709+0000 7f6a56ffd700 1 --2- 172.21.15.101:0/3976954835 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5601a04d0360 0x5601a04d2770 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68192ee27a4727ff server_cookie=ae36e07f70c0e33f in_seq=0 out_seq=0 2023-07-18T03:11:14.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.710+0000 7f6a3aff5700 1 -- 172.21.15.101:0/3976954835 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5601a036e050 con 0x5601a04d0360 2023-07-18T03:11:14.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.710+0000 7f6a3aff5700 10 client.26285 handle_command_reply: tid=0 2023-07-18T03:11:14.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.710+0000 7f6a71194b80 1 client.26285 shutdown 2023-07-18T03:11:14.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.710+0000 7f6a71194b80 1 -- 172.21.15.101:0/3976954835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601a046bad0 msgr2=0x5601a0470e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:14.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.710+0000 7f6a71194b80 1 --2- 172.21.15.101:0/3976954835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601a046bad0 0x5601a0470e20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6a50095f30 tx=0x7f6a50009ac0 comp rx=0 tx=0).stop 2023-07-18T03:11:14.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.710+0000 7f6a71194b80 1 -- 172.21.15.101:0/3976954835 shutdown_connections 2023-07-18T03:11:14.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.710+0000 7f6a71194b80 1 --2- 172.21.15.101:0/3976954835 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5601a04d0360 0x5601a04d2770 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:14.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.710+0000 7f6a71194b80 1 --2- 172.21.15.101:0/3976954835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601a046bad0 0x5601a0470e20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:14.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.710+0000 7f6a71194b80 1 --2- 172.21.15.101:0/3976954835 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5601a04602d0 0x5601a04724c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:14.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.710+0000 7f6a71194b80 1 --2- 172.21.15.101:0/3976954835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601a0460be0 0x5601a04708e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:14.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.710+0000 7f6a71194b80 1 -- 172.21.15.101:0/3976954835 >> 172.21.15.101:0/3976954835 conn(0x5601a0456da0 msgr2=0x5601a04622e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:14.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.710+0000 7f6a3aff5700 0 client.26285 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:11:14.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.710+0000 7f6a71194b80 1 -- 172.21.15.101:0/3976954835 shutdown_connections 2023-07-18T03:11:14.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.710+0000 7f6a71194b80 1 -- 172.21.15.101:0/3976954835 wait complete. 2023-07-18T03:11:14.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.710+0000 7f6a71194b80 20 client.26285 trim_cache size 0 max 16384 2023-07-18T03:11:14.712 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:11:14.712 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "no active scrubs running", 2023-07-18T03:11:14.712 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": {} 2023-07-18T03:11:14.712 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:11:14.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.711+0000 7f6a5d7ff700 1 -- 172.21.15.101:0/3896562400 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6a400a12c0 msgr2=0x7f6a400a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:14.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.711+0000 7f6a5d7ff700 1 --2- 172.21.15.101:0/3896562400 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6a400a12c0 0x7f6a400a3770 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c00b2e0 tx=0x7f6a4c00b930 comp rx=0 tx=0).stop 2023-07-18T03:11:14.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.711+0000 7f6a5d7ff700 1 -- 172.21.15.101:0/3896562400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a5810c3a0 msgr2=0x7f6a5819c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:14.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.711+0000 7f6a5d7ff700 1 --2- 172.21.15.101:0/3896562400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a5810c3a0 0x7f6a5819c850 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6a58074920 tx=0x7f6a5000ef90 comp rx=0 tx=0).stop 2023-07-18T03:11:14.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.711+0000 7f6a5d7ff700 1 -- 172.21.15.101:0/3896562400 shutdown_connections 2023-07-18T03:11:14.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.711+0000 7f6a5d7ff700 1 --2- 172.21.15.101:0/3896562400 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6a400a12c0 0x7f6a400a3770 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:14.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.711+0000 7f6a5d7ff700 1 --2- 172.21.15.101:0/3896562400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a5810c3a0 0x7f6a5819c850 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:14.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.711+0000 7f6a5d7ff700 1 --2- 172.21.15.101:0/3896562400 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6a58072850 0x7f6a58197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:14.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.711+0000 7f6a5d7ff700 1 --2- 172.21.15.101:0/3896562400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a58073160 0x7f6a581983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:14.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.711+0000 7f6a5d7ff700 1 -- 172.21.15.101:0/3896562400 >> 172.21.15.101:0/3896562400 conn(0x7f6a580fb600 msgr2=0x7f6a58103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:14.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.711+0000 7f6a5d7ff700 1 -- 172.21.15.101:0/3896562400 shutdown_connections 2023-07-18T03:11:14.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.711+0000 7f6a5d7ff700 1 -- 172.21.15.101:0/3896562400 wait complete. 2023-07-18T03:11:14.721 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-18T03:11:14.722 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:11:14.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.945+0000 7f9d69dff700 1 -- 172.21.15.101:0/3756827336 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d64072830 msgr2=0x7f9d64072c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:14.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.945+0000 7f9d69dff700 1 --2- 172.21.15.101:0/3756827336 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d64072830 0x7f9d64072c00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9d540098f0 tx=0x7f9d54009c00 comp rx=0 tx=0).stop 2023-07-18T03:11:14.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.945+0000 7f9d69dff700 1 -- 172.21.15.101:0/3756827336 shutdown_connections 2023-07-18T03:11:14.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.946+0000 7f9d69dff700 1 --2- 172.21.15.101:0/3756827336 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9d6410c3b0 0x7f9d6410e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:14.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.946+0000 7f9d69dff700 1 --2- 172.21.15.101:0/3756827336 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d64072830 0x7f9d64072c00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:14.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.946+0000 7f9d69dff700 1 --2- 172.21.15.101:0/3756827336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d64073140 0x7f9d6410be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:14.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.946+0000 7f9d69dff700 1 -- 172.21.15.101:0/3756827336 >> 172.21.15.101:0/3756827336 conn(0x7f9d640fb5d0 msgr2=0x7f9d640fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:14.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.946+0000 7f9d69dff700 1 -- 172.21.15.101:0/3756827336 shutdown_connections 2023-07-18T03:11:14.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.946+0000 7f9d69dff700 1 -- 172.21.15.101:0/3756827336 wait complete. 2023-07-18T03:11:14.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.947+0000 7f9d69dff700 1 Processor -- start 2023-07-18T03:11:14.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.947+0000 7f9d69dff700 1 -- start start 2023-07-18T03:11:14.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.947+0000 7f9d69dff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d64072830 0x7f9d64197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:14.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.947+0000 7f9d69dff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d64073140 0x7f9d64198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:14.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.947+0000 7f9d69dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9d6410c3b0 0x7f9d6419c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:14.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.947+0000 7f9d69dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6419ce00 con 0x7f9d6410c3b0 2023-07-18T03:11:14.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.947+0000 7f9d69dff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6419cf70 con 0x7f9d64072830 2023-07-18T03:11:14.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.947+0000 7f9d69dff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6419d130 con 0x7f9d64073140 2023-07-18T03:11:14.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.947+0000 7f9d637fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d64072830 0x7f9d64197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:14.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.947+0000 7f9d637fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d64072830 0x7f9d64197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:58238/0 (socket says 172.21.15.101:58238) 2023-07-18T03:11:14.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.947+0000 7f9d637fe700 1 -- 172.21.15.101:0/2371187621 learned_addr learned my addr 172.21.15.101:0/2371187621 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:14.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.947+0000 7f9d62ffd700 1 --2- 172.21.15.101:0/2371187621 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d64073140 0x7f9d64198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:14.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.947+0000 7f9d63fff700 1 --2- 172.21.15.101:0/2371187621 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9d6410c3b0 0x7f9d6419c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:14.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.948+0000 7f9d637fe700 1 -- 172.21.15.101:0/2371187621 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d64073140 msgr2=0x7f9d64198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:14.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.948+0000 7f9d637fe700 1 --2- 172.21.15.101:0/2371187621 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d64073140 0x7f9d64198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:14.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.948+0000 7f9d637fe700 1 -- 172.21.15.101:0/2371187621 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9d6410c3b0 msgr2=0x7f9d6419c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:14.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.948+0000 7f9d637fe700 1 --2- 172.21.15.101:0/2371187621 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9d6410c3b0 0x7f9d6419c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:14.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.948+0000 7f9d637fe700 1 -- 172.21.15.101:0/2371187621 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d580095f0 con 0x7f9d64072830 2023-07-18T03:11:14.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.948+0000 7f9d637fe700 1 --2- 172.21.15.101:0/2371187621 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d64072830 0x7f9d64197e20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9d54009810 tx=0x7f9d5400fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:14.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.948+0000 7f9d60ff9700 1 -- 172.21.15.101:0/2371187621 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5401c070 con 0x7f9d64072830 2023-07-18T03:11:14.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.948+0000 7f9d69dff700 1 -- 172.21.15.101:0/2371187621 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d54009520 con 0x7f9d64072830 2023-07-18T03:11:14.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.948+0000 7f9d60ff9700 1 -- 172.21.15.101:0/2371187621 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d5400bb20 con 0x7f9d64072830 2023-07-18T03:11:14.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.948+0000 7f9d60ff9700 1 -- 172.21.15.101:0/2371187621 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d54004930 con 0x7f9d64072830 2023-07-18T03:11:14.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.948+0000 7f9d69dff700 1 -- 172.21.15.101:0/2371187621 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d641a4a70 con 0x7f9d64072830 2023-07-18T03:11:14.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.950+0000 7f9d69dff700 1 -- 172.21.15.101:0/2371187621 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d641075c0 con 0x7f9d64072830 2023-07-18T03:11:14.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.952+0000 7f9d60ff9700 1 -- 172.21.15.101:0/2371187621 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f9d54020070 con 0x7f9d64072830 2023-07-18T03:11:14.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.953+0000 7f9d60ff9700 1 --2- 172.21.15.101:0/2371187621 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9d4c0a1300 0x7f9d4c0a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:14.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.953+0000 7f9d60ff9700 1 -- 172.21.15.101:0/2371187621 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f9d540cddc0 con 0x7f9d64072830 2023-07-18T03:11:14.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.953+0000 7f9d62ffd700 1 --2- 172.21.15.101:0/2371187621 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9d4c0a1300 0x7f9d4c0a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:14.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.954+0000 7f9d62ffd700 1 --2- 172.21.15.101:0/2371187621 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9d4c0a1300 0x7f9d4c0a37b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9d64199430 tx=0x7f9d58009310 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:14.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:14.955+0000 7f9d60ff9700 1 -- 172.21.15.101:0/2371187621 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9d54096eb0 con 0x7f9d64072830 2023-07-18T03:11:15.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.222+0000 7f9d69dff700 1 -- 172.21.15.101:0/2371187621 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d6419d2f0 con 0x7f9d64072830 2023-07-18T03:11:15.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.223+0000 7f9d60ff9700 1 -- 172.21.15.101:0/2371187621 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10861 (secure 0 0 0) 0x7f9d5409a4d0 con 0x7f9d64072830 2023-07-18T03:11:15.215 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:11:15.215 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/4192846769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":4192846769},{"type":"v1","addr":"172.21.15.101:6839","nonce":4192846769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6841/3824340175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6840","nonce":3824340175},{"type":"v1","addr":"172.21.15.101:6841","nonce":3824340175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:04:59.361185+0000","modified":"2023-07-18T03:09:01.105419+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.155:6839/2055462483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2055462483},{"type":"v1","addr":"172.21.15.155:6839","nonce":2055462483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:11:15.218 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.226+0000 7f9d69dff700 1 -- 172.21.15.101:0/2371187621 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9d4c0a1300 msgr2=0x7f9d4c0a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:15.218 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.226+0000 7f9d69dff700 1 --2- 172.21.15.101:0/2371187621 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9d4c0a1300 0x7f9d4c0a37b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9d64199430 tx=0x7f9d58009310 comp rx=0 tx=0).stop 2023-07-18T03:11:15.218 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.226+0000 7f9d69dff700 1 -- 172.21.15.101:0/2371187621 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d64072830 msgr2=0x7f9d64197e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:15.218 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.226+0000 7f9d69dff700 1 --2- 172.21.15.101:0/2371187621 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d64072830 0x7f9d64197e20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9d54009810 tx=0x7f9d5400fa00 comp rx=0 tx=0).stop 2023-07-18T03:11:15.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.226+0000 7f9d69dff700 1 -- 172.21.15.101:0/2371187621 shutdown_connections 2023-07-18T03:11:15.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.226+0000 7f9d69dff700 1 --2- 172.21.15.101:0/2371187621 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9d4c0a1300 0x7f9d4c0a37b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.226+0000 7f9d69dff700 1 --2- 172.21.15.101:0/2371187621 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d64072830 0x7f9d64197e20 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.226+0000 7f9d69dff700 1 --2- 172.21.15.101:0/2371187621 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9d6410c3b0 0x7f9d6419c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.226+0000 7f9d69dff700 1 --2- 172.21.15.101:0/2371187621 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d64073140 0x7f9d64198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.226+0000 7f9d69dff700 1 -- 172.21.15.101:0/2371187621 >> 172.21.15.101:0/2371187621 conn(0x7f9d640fb5d0 msgr2=0x7f9d6410e610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:15.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.226+0000 7f9d69dff700 1 -- 172.21.15.101:0/2371187621 shutdown_connections 2023-07-18T03:11:15.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.226+0000 7f9d69dff700 1 -- 172.21.15.101:0/2371187621 wait complete. 2023-07-18T03:11:15.221 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 25 2023-07-18T03:11:15.237 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-18T03:11:15.307 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:15 smithi101 ceph-mon[126379]: pgmap v113: 129 pgs: 129 active+clean; 144 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 239 op/s 2023-07-18T03:11:15.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:15 smithi179 ceph-mon[129229]: pgmap v113: 129 pgs: 129 active+clean; 144 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 239 op/s 2023-07-18T03:11:15.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.463+0000 7f524bbff700 1 -- 172.21.15.101:0/865455920 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52441015b0 msgr2=0x7f5244109aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:15.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.463+0000 7f524bbff700 1 --2- 172.21.15.101:0/865455920 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52441015b0 0x7f5244109aa0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5234009990 tx=0x7f5234009ca0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.463+0000 7f524bbff700 1 -- 172.21.15.101:0/865455920 shutdown_connections 2023-07-18T03:11:15.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.463+0000 7f524bbff700 1 --2- 172.21.15.101:0/865455920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5244100ca0 0x7f5244101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.463+0000 7f524bbff700 1 --2- 172.21.15.101:0/865455920 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f524410a070 0x7f524410c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.463+0000 7f524bbff700 1 --2- 172.21.15.101:0/865455920 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52441015b0 0x7f5244109aa0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.463+0000 7f524bbff700 1 -- 172.21.15.101:0/865455920 >> 172.21.15.101:0/865455920 conn(0x7f52440fa510 msgr2=0x7f52440fc920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:15.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.463+0000 7f524bbff700 1 -- 172.21.15.101:0/865455920 shutdown_connections 2023-07-18T03:11:15.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.463+0000 7f524bbff700 1 -- 172.21.15.101:0/865455920 wait complete. 2023-07-18T03:11:15.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.464+0000 7f524bbff700 1 Processor -- start 2023-07-18T03:11:15.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.464+0000 7f524bbff700 1 -- start start 2023-07-18T03:11:15.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.465+0000 7f524bbff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5244100ca0 0x7f5244197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:15.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.465+0000 7f524bbff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52441015b0 0x7f52441983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:15.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.465+0000 7f524bbff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f524410a070 0x7f524419c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:15.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.465+0000 7f524bbff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524419ce80 con 0x7f5244100ca0 2023-07-18T03:11:15.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.465+0000 7f524bbff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524419cff0 con 0x7f524410a070 2023-07-18T03:11:15.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.465+0000 7f524bbff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524419d1b0 con 0x7f52441015b0 2023-07-18T03:11:15.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.465+0000 7f524999b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5244100ca0 0x7f5244197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:15.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.465+0000 7f524999b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5244100ca0 0x7f5244197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:42448/0 (socket says 172.21.15.101:42448) 2023-07-18T03:11:15.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.465+0000 7f524999b700 1 -- 172.21.15.101:0/2169780831 learned_addr learned my addr 172.21.15.101:0/2169780831 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:15.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.465+0000 7f524a19c700 1 --2- 172.21.15.101:0/2169780831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f524410a070 0x7f524419c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:15.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.465+0000 7f524919a700 1 --2- 172.21.15.101:0/2169780831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52441015b0 0x7f52441983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:15.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.465+0000 7f524999b700 1 -- 172.21.15.101:0/2169780831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52441015b0 msgr2=0x7f52441983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:15.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.465+0000 7f524999b700 1 --2- 172.21.15.101:0/2169780831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52441015b0 0x7f52441983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.465+0000 7f524999b700 1 -- 172.21.15.101:0/2169780831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f524410a070 msgr2=0x7f524419c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:15.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.465+0000 7f524999b700 1 --2- 172.21.15.101:0/2169780831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f524410a070 0x7f524419c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.465+0000 7f524999b700 1 -- 172.21.15.101:0/2169780831 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52340095f0 con 0x7f5244100ca0 2023-07-18T03:11:15.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.466+0000 7f524999b700 1 --2- 172.21.15.101:0/2169780831 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5244100ca0 0x7f5244197ea0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f524000e9a0 tx=0x7f524000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:15.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.466+0000 7f523affd700 1 -- 172.21.15.101:0/2169780831 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5240009790 con 0x7f5244100ca0 2023-07-18T03:11:15.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.466+0000 7f524bbff700 1 -- 172.21.15.101:0/2169780831 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f524419d490 con 0x7f5244100ca0 2023-07-18T03:11:15.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.466+0000 7f523affd700 1 -- 172.21.15.101:0/2169780831 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5240010430 con 0x7f5244100ca0 2023-07-18T03:11:15.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.466+0000 7f523affd700 1 -- 172.21.15.101:0/2169780831 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5240018680 con 0x7f5244100ca0 2023-07-18T03:11:15.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.466+0000 7f524bbff700 1 -- 172.21.15.101:0/2169780831 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52441a4cd0 con 0x7f5244100ca0 2023-07-18T03:11:15.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.469+0000 7f523affd700 1 -- 172.21.15.101:0/2169780831 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f52400105a0 con 0x7f5244100ca0 2023-07-18T03:11:15.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.471+0000 7f525f43ab80 1 Processor -- start 2023-07-18T03:11:15.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.471+0000 7f525f43ab80 1 -- start start 2023-07-18T03:11:15.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.471+0000 7f525f43ab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56275b57b2d0 0x56275b57b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:15.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.471+0000 7f525f43ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56275b57bbe0 0x56275b586590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:15.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.471+0000 7f523affd700 1 --2- 172.21.15.101:0/2169780831 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f52300a12c0 0x7f52300a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:15.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.471+0000 7f525f43ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56275b586ad0 0x56275b588eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:15.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.471+0000 7f525f43ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56275b481cb0 con 0x56275b57bbe0 2023-07-18T03:11:15.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.471+0000 7f525f43ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56275b4d4a40 con 0x56275b586ad0 2023-07-18T03:11:15.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.471+0000 7f525f43ab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56275b4cded0 con 0x56275b57b2d0 2023-07-18T03:11:15.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.471+0000 7f523affd700 1 -- 172.21.15.101:0/2169780831 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f5240014070 con 0x7f5244100ca0 2023-07-18T03:11:15.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.472+0000 7f524919a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56275b57bbe0 0x56275b586590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:15.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.472+0000 7f524999b700 1 --2- 172.21.15.101:0/2169780831 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f52300a12c0 0x7f52300a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:15.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.472+0000 7f524a19c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56275b57b2d0 0x56275b57b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:15.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.472+0000 7f524919a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56275b57bbe0 0x56275b586590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:42462/0 (socket says 172.21.15.101:42462) 2023-07-18T03:11:15.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.472+0000 7f524919a700 1 -- 172.21.15.101:0/1441743755 learned_addr learned my addr 172.21.15.101:0/1441743755 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:15.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.472+0000 7f524a19c700 1 --2- 172.21.15.101:0/1441743755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56275b586ad0 0x56275b588eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:15.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.472+0000 7f524919a700 1 -- 172.21.15.101:0/1441743755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56275b57b2d0 msgr2=0x56275b57b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:15.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.472+0000 7f524919a700 1 --2- 172.21.15.101:0/1441743755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56275b57b2d0 0x56275b57b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.472+0000 7f524919a700 1 -- 172.21.15.101:0/1441743755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56275b586ad0 msgr2=0x56275b588eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:15.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.472+0000 7f524919a700 1 --2- 172.21.15.101:0/1441743755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56275b586ad0 0x56275b588eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.472+0000 7f524919a700 1 -- 172.21.15.101:0/1441743755 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56275b2fbde0 con 0x56275b57bbe0 2023-07-18T03:11:15.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.472+0000 7f524a19c700 1 --2- 172.21.15.101:0/1441743755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56275b57b2d0 0x56275b57b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:11:15.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.472+0000 7f524999b700 1 --2- 172.21.15.101:0/2169780831 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f52300a12c0 0x7f52300a3770 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5240096a10 tx=0x7f5240010b80 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:15.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.472+0000 7f524919a700 1 --2- 172.21.15.101:0/1441743755 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56275b57bbe0 0x56275b586590 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f5234015000 tx=0x7f5234005ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc608eea5d0f244c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:15.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.473+0000 7f522ffff700 1 -- 172.21.15.101:0/1441743755 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523401d070 con 0x56275b57bbe0 2023-07-18T03:11:15.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.473+0000 7f522ffff700 1 -- 172.21.15.101:0/1441743755 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f523401e040 con 0x56275b57bbe0 2023-07-18T03:11:15.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.473+0000 7f522ffff700 1 -- 172.21.15.101:0/1441743755 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5234004930 con 0x56275b57bbe0 2023-07-18T03:11:15.471 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.473+0000 7f525f43ab80 1 -- 172.21.15.101:0/1441743755 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56275b57bbe0 msgr2=0x56275b586590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:15.471 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.473+0000 7f525f43ab80 1 --2- 172.21.15.101:0/1441743755 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56275b57bbe0 0x56275b586590 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f5234015000 tx=0x7f5234005ea0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.471 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.474+0000 7f525f43ab80 1 -- 172.21.15.101:0/1441743755 shutdown_connections 2023-07-18T03:11:15.472 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.474+0000 7f525f43ab80 1 --2- 172.21.15.101:0/1441743755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56275b586ad0 0x56275b588eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.472 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.474+0000 7f525f43ab80 1 --2- 172.21.15.101:0/1441743755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56275b57b2d0 0x56275b57b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.472 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.474+0000 7f525f43ab80 1 --2- 172.21.15.101:0/1441743755 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56275b57bbe0 0x56275b586590 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.472 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.474+0000 7f525f43ab80 1 -- 172.21.15.101:0/1441743755 >> 172.21.15.101:0/1441743755 conn(0x56275b571da0 msgr2=0x56275b5776a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:15.472 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.474+0000 7f525f43ab80 1 -- 172.21.15.101:0/1441743755 shutdown_connections 2023-07-18T03:11:15.472 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.474+0000 7f525f43ab80 1 -- 172.21.15.101:0/1441743755 wait complete. 2023-07-18T03:11:15.472 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.474+0000 7f525f43ab80 1 -- start start 2023-07-18T03:11:15.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.475+0000 7f525f43ab80 1 Processor -- start 2023-07-18T03:11:15.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.475+0000 7f525f43ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56275b57b2d0 0x56275b586520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:15.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.475+0000 7f525f43ab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56275b57bbe0 0x56275b583730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:15.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.475+0000 7f525f43ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56275b586ad0 0x56275b583e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:15.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.475+0000 7f525f43ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56275b46dac0 con 0x56275b57b2d0 2023-07-18T03:11:15.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.475+0000 7f525f43ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56275b46d7c0 con 0x56275b586ad0 2023-07-18T03:11:15.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.475+0000 7f525f43ab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56275b351920 con 0x56275b57bbe0 2023-07-18T03:11:15.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.475+0000 7f525f43ab80 10 client.0 _ll_register_callbacks cb 0x56275b46b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:11:15.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.476+0000 7f524919a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56275b57bbe0 0x56275b583730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:15.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.476+0000 7f524919a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56275b57bbe0 0x56275b583730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:36244/0 (socket says 172.21.15.101:36244) 2023-07-18T03:11:15.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.476+0000 7f524919a700 1 -- 172.21.15.101:0/2068883238 learned_addr learned my addr 172.21.15.101:0/2068883238 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:15.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.476+0000 7f524a19c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56275b57b2d0 0x56275b586520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:15.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.476+0000 7f524a19c700 1 --2- 172.21.15.101:0/2068883238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56275b586ad0 0x56275b583e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:15.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.476+0000 7f524919a700 1 -- 172.21.15.101:0/2068883238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56275b586ad0 msgr2=0x56275b583e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:15.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.476+0000 7f524919a700 1 --2- 172.21.15.101:0/2068883238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56275b586ad0 0x56275b583e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.476+0000 7f524919a700 1 -- 172.21.15.101:0/2068883238 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56275b57b2d0 msgr2=0x56275b586520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:15.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.476+0000 7f524919a700 1 --2- 172.21.15.101:0/2068883238 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56275b57b2d0 0x56275b586520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.476+0000 7f524919a700 1 -- 172.21.15.101:0/2068883238 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f523c008850 con 0x56275b57bbe0 2023-07-18T03:11:15.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.476+0000 7f524a19c700 1 --2- 172.21.15.101:0/2068883238 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56275b57b2d0 0x56275b586520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:11:15.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.476+0000 7f524919a700 1 --2- 172.21.15.101:0/2068883238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56275b57bbe0 0x56275b583730 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5234005f50 tx=0x7f523400fb00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=760aaea71677b299 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:15.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.476+0000 7f522d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:11:15.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.477+0000 7f522d7fa700 1 -- 172.21.15.101:0/2068883238 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523401d040 con 0x56275b57bbe0 2023-07-18T03:11:15.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.477+0000 7f522d7fa700 1 -- 172.21.15.101:0/2068883238 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f523400fe20 con 0x56275b57bbe0 2023-07-18T03:11:15.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.477+0000 7f525f43ab80 1 -- 172.21.15.101:0/2068883238 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56275b4274f0 con 0x56275b57bbe0 2023-07-18T03:11:15.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.477+0000 7f522d7fa700 1 -- 172.21.15.101:0/2068883238 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5234005610 con 0x56275b57bbe0 2023-07-18T03:11:15.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.477+0000 7f525f43ab80 10 client.26326 fetch_fsmap learned FSMap version 25 2023-07-18T03:11:15.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.477+0000 7f525f43ab80 1 -- 172.21.15.101:0/2068883238 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f524410d170 con 0x56275b57bbe0 2023-07-18T03:11:15.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.477+0000 7f522d7fa700 1 -- 172.21.15.101:0/2068883238 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56275b4274f0 con 0x56275b57bbe0 2023-07-18T03:11:15.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.477+0000 7f522d7fa700 1 -- 172.21.15.101:0/2068883238 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f5234022400 con 0x56275b57bbe0 2023-07-18T03:11:15.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.477+0000 7f525f43ab80 10 client.26326 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:11:15.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.477+0000 7f525f43ab80 10 client.26326 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:11:15.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.477+0000 7f525f43ab80 20 client.26326 populate_metadata read hostname 'smithi101' 2023-07-18T03:11:15.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.477+0000 7f525f43ab80 1 --2- 172.21.15.101:0/2068883238 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x56275b595fe0 0x56275b5983f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:15.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.477+0000 7f525f43ab80 4 client.26326 mds_command: new command op to 24541 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:11:15.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.477+0000 7f525f43ab80 1 -- 172.21.15.101:0/2068883238 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56275b4c6eb0 con 0x56275b595fe0 2023-07-18T03:11:15.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.478+0000 7f524a19c700 1 --2- 172.21.15.101:0/2068883238 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x56275b595fe0 0x56275b5983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:15.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.478+0000 7f522d7fa700 10 client.26326 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:11:15.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.478+0000 7f524a19c700 1 --2- 172.21.15.101:0/2068883238 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x56275b595fe0 0x56275b5983f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3652e0ce1cff2091 server_cookie=519570050d864aba in_seq=0 out_seq=0 2023-07-18T03:11:15.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.479+0000 7f522d7fa700 1 -- 172.21.15.101:0/2068883238 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56275b4c6eb0 con 0x56275b595fe0 2023-07-18T03:11:15.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.479+0000 7f522d7fa700 10 client.26326 handle_command_reply: tid=0 2023-07-18T03:11:15.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.479+0000 7f525f43ab80 1 client.26326 shutdown 2023-07-18T03:11:15.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.479+0000 7f525f43ab80 1 -- 172.21.15.101:0/2068883238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56275b57bbe0 msgr2=0x56275b583730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:15.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.479+0000 7f525f43ab80 1 --2- 172.21.15.101:0/2068883238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56275b57bbe0 0x56275b583730 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5234005f50 tx=0x7f523400fb00 comp rx=0 tx=0).stop 2023-07-18T03:11:15.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.479+0000 7f525f43ab80 1 -- 172.21.15.101:0/2068883238 shutdown_connections 2023-07-18T03:11:15.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.479+0000 7f525f43ab80 1 --2- 172.21.15.101:0/2068883238 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x56275b595fe0 0x56275b5983f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.479+0000 7f525f43ab80 1 --2- 172.21.15.101:0/2068883238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56275b586ad0 0x56275b583e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.479+0000 7f525f43ab80 1 --2- 172.21.15.101:0/2068883238 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56275b57b2d0 0x56275b586520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.479+0000 7f525f43ab80 1 --2- 172.21.15.101:0/2068883238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56275b57bbe0 0x56275b583730 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.479+0000 7f522d7fa700 0 client.26326 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:11:15.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.479+0000 7f525f43ab80 1 -- 172.21.15.101:0/2068883238 >> 172.21.15.101:0/2068883238 conn(0x56275b571da0 msgr2=0x56275b577230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:15.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.479+0000 7f525f43ab80 1 -- 172.21.15.101:0/2068883238 shutdown_connections 2023-07-18T03:11:15.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.479+0000 7f525f43ab80 1 -- 172.21.15.101:0/2068883238 wait complete. 2023-07-18T03:11:15.480 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.479+0000 7f525f43ab80 20 client.26326 trim_cache size 0 max 16384 2023-07-18T03:11:15.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.493+0000 7f525f43ab80 1 Processor -- start 2023-07-18T03:11:15.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.493+0000 7f525f43ab80 1 -- start start 2023-07-18T03:11:15.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.494+0000 7f525f43ab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56275b57b2d0 0x56275b5836f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:15.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.494+0000 7f525f43ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56275b57bbe0 0x56275b575e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:15.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.494+0000 7f525f43ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56275b586ad0 0x56275b586050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:15.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.494+0000 7f525f43ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56275b427520 con 0x56275b586ad0 2023-07-18T03:11:15.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.494+0000 7f525f43ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56275b573ef0 con 0x56275b57bbe0 2023-07-18T03:11:15.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.494+0000 7f525f43ab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56275b57c540 con 0x56275b57b2d0 2023-07-18T03:11:15.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.494+0000 7f524919a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56275b57bbe0 0x56275b575e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:15.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.494+0000 7f524a19c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56275b57b2d0 0x56275b5836f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:15.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.494+0000 7f524919a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56275b57bbe0 0x56275b575e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:58290/0 (socket says 172.21.15.101:58290) 2023-07-18T03:11:15.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.494+0000 7f524919a700 1 -- 172.21.15.101:0/641697703 learned_addr learned my addr 172.21.15.101:0/641697703 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:15.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.494+0000 7f524a19c700 1 --2- 172.21.15.101:0/641697703 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56275b586ad0 0x56275b586050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:15.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.494+0000 7f524919a700 1 -- 172.21.15.101:0/641697703 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56275b57b2d0 msgr2=0x56275b5836f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:15.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.494+0000 7f524919a700 1 --2- 172.21.15.101:0/641697703 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56275b57b2d0 0x56275b5836f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.494+0000 7f524919a700 1 -- 172.21.15.101:0/641697703 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56275b586ad0 msgr2=0x56275b586050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:15.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.494+0000 7f524919a700 1 --2- 172.21.15.101:0/641697703 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56275b586ad0 0x56275b586050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.494+0000 7f524919a700 1 -- 172.21.15.101:0/641697703 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56275b2e4c60 con 0x56275b57bbe0 2023-07-18T03:11:15.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.494+0000 7f524a19c700 1 --2- 172.21.15.101:0/641697703 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56275b586ad0 0x56275b586050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:11:15.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.495+0000 7f524919a700 1 --2- 172.21.15.101:0/641697703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56275b57bbe0 0x56275b575e10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f523400bbb0 tx=0x7f5234009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c95c5efe7953101a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:15.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.495+0000 7f5238bff700 1 -- 172.21.15.101:0/641697703 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5234005bc0 con 0x56275b57bbe0 2023-07-18T03:11:15.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.495+0000 7f5238bff700 1 -- 172.21.15.101:0/641697703 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52340058e0 con 0x56275b57bbe0 2023-07-18T03:11:15.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.495+0000 7f5238bff700 1 -- 172.21.15.101:0/641697703 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523400eb00 con 0x56275b57bbe0 2023-07-18T03:11:15.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.495+0000 7f525f43ab80 1 -- 172.21.15.101:0/641697703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56275b57bbe0 msgr2=0x56275b575e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:15.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.495+0000 7f525f43ab80 1 --2- 172.21.15.101:0/641697703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56275b57bbe0 0x56275b575e10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f523400bbb0 tx=0x7f5234009310 comp rx=0 tx=0).stop 2023-07-18T03:11:15.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.496+0000 7f525f43ab80 1 -- 172.21.15.101:0/641697703 shutdown_connections 2023-07-18T03:11:15.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.496+0000 7f525f43ab80 1 --2- 172.21.15.101:0/641697703 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56275b586ad0 0x56275b586050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.496+0000 7f525f43ab80 1 --2- 172.21.15.101:0/641697703 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56275b57b2d0 0x56275b5836f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.492 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.496+0000 7f525f43ab80 1 --2- 172.21.15.101:0/641697703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56275b57bbe0 0x56275b575e10 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.492 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.496+0000 7f525f43ab80 1 -- 172.21.15.101:0/641697703 >> 172.21.15.101:0/641697703 conn(0x56275b571da0 msgr2=0x56275b59cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:15.492 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.496+0000 7f525f43ab80 1 -- 172.21.15.101:0/641697703 shutdown_connections 2023-07-18T03:11:15.492 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.496+0000 7f525f43ab80 1 -- 172.21.15.101:0/641697703 wait complete. 2023-07-18T03:11:15.492 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.496+0000 7f525f43ab80 1 -- start start 2023-07-18T03:11:15.492 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.496+0000 7f525f43ab80 1 Processor -- start 2023-07-18T03:11:15.492 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.497+0000 7f525f43ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56275b57b2d0 0x56275b58c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:15.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.497+0000 7f525f43ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56275b57bbe0 0x56275b58cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:15.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.497+0000 7f525f43ab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56275b586ad0 0x56275b5e8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:15.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.497+0000 7f525f43ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56275b58d350 con 0x56275b57bbe0 2023-07-18T03:11:15.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.497+0000 7f525f43ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56275b58d4c0 con 0x56275b57b2d0 2023-07-18T03:11:15.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.497+0000 7f525f43ab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56275b5e93a0 con 0x56275b586ad0 2023-07-18T03:11:15.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.497+0000 7f525f43ab80 10 client.0 _ll_register_callbacks cb 0x7f523c008510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:11:15.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.497+0000 7f524a19c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56275b57b2d0 0x56275b58c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:15.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.497+0000 7f524919a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56275b57bbe0 0x56275b58cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:15.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.497+0000 7f524a19c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56275b57b2d0 0x56275b58c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:58298/0 (socket says 172.21.15.101:58298) 2023-07-18T03:11:15.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.497+0000 7f524a19c700 1 -- 172.21.15.101:0/2396413721 learned_addr learned my addr 172.21.15.101:0/2396413721 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:15.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.497+0000 7f524a19c700 1 --2- 172.21.15.101:0/2396413721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56275b586ad0 0x56275b5e8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:15.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.498+0000 7f524919a700 1 -- 172.21.15.101:0/2396413721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56275b586ad0 msgr2=0x56275b5e8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:15.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.498+0000 7f524919a700 1 --2- 172.21.15.101:0/2396413721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56275b586ad0 0x56275b5e8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.498+0000 7f524919a700 1 -- 172.21.15.101:0/2396413721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56275b57b2d0 msgr2=0x56275b58c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:15.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.498+0000 7f524919a700 1 --2- 172.21.15.101:0/2396413721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56275b57b2d0 0x56275b58c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.498+0000 7f524919a700 1 -- 172.21.15.101:0/2396413721 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f524410d170 con 0x56275b57bbe0 2023-07-18T03:11:15.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.498+0000 7f524a19c700 1 --2- 172.21.15.101:0/2396413721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56275b57b2d0 0x56275b58c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:11:15.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.498+0000 7f524919a700 1 --2- 172.21.15.101:0/2396413721 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56275b57bbe0 0x56275b58cbe0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f5234022900 tx=0x7f5234005ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcd143c4aa2e34d1 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:15.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.498+0000 7f522d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:11:15.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.498+0000 7f522d7fa700 1 -- 172.21.15.101:0/2396413721 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523401d070 con 0x56275b57bbe0 2023-07-18T03:11:15.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.498+0000 7f522d7fa700 1 -- 172.21.15.101:0/2396413721 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f5234047030 con 0x56275b57bbe0 2023-07-18T03:11:15.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.498+0000 7f525f43ab80 1 -- 172.21.15.101:0/2396413721 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f523401e040 con 0x56275b57bbe0 2023-07-18T03:11:15.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.499+0000 7f522d7fa700 1 -- 172.21.15.101:0/2396413721 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5234005bc0 con 0x56275b57bbe0 2023-07-18T03:11:15.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.499+0000 7f525f43ab80 10 client.16263 fetch_fsmap learned FSMap version 25 2023-07-18T03:11:15.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.499+0000 7f525f43ab80 1 -- 172.21.15.101:0/2396413721 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56275b322250 con 0x56275b57bbe0 2023-07-18T03:11:15.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.499+0000 7f522d7fa700 1 -- 172.21.15.101:0/2396413721 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f523401e040 con 0x56275b57bbe0 2023-07-18T03:11:15.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.499+0000 7f522d7fa700 1 -- 172.21.15.101:0/2396413721 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f5234041070 con 0x56275b57bbe0 2023-07-18T03:11:15.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.499+0000 7f525f43ab80 10 client.16263 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:11:15.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.499+0000 7f525f43ab80 10 client.16263 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:11:15.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.499+0000 7f525f43ab80 20 client.16263 populate_metadata read hostname 'smithi101' 2023-07-18T03:11:15.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.499+0000 7f525f43ab80 1 --2- 172.21.15.101:0/2396413721 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x56275b5ea660 0x56275b5eca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:15.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.499+0000 7f525f43ab80 4 client.16263 mds_command: new command op to 24541 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:11:15.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.499+0000 7f525f43ab80 1 -- 172.21.15.101:0/2396413721 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56275b489050 con 0x56275b5ea660 2023-07-18T03:11:15.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.499+0000 7f524a19c700 1 --2- 172.21.15.101:0/2396413721 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x56275b5ea660 0x56275b5eca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:15.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.500+0000 7f524a19c700 1 --2- 172.21.15.101:0/2396413721 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x56275b5ea660 0x56275b5eca70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=70b056004efcd97b server_cookie=196cfc5a1ea62df in_seq=0 out_seq=0 2023-07-18T03:11:15.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.500+0000 7f522d7fa700 10 client.16263 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:11:15.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.500+0000 7f522d7fa700 1 -- 172.21.15.101:0/2396413721 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56275b489050 con 0x56275b5ea660 2023-07-18T03:11:15.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.500+0000 7f522d7fa700 10 client.16263 handle_command_reply: tid=0 2023-07-18T03:11:15.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.500+0000 7f525f43ab80 1 client.16263 shutdown 2023-07-18T03:11:15.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.500+0000 7f525f43ab80 1 -- 172.21.15.101:0/2396413721 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56275b57bbe0 msgr2=0x56275b58cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:15.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.500+0000 7f525f43ab80 1 --2- 172.21.15.101:0/2396413721 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56275b57bbe0 0x56275b58cbe0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f5234022900 tx=0x7f5234005ea0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.500+0000 7f525f43ab80 1 -- 172.21.15.101:0/2396413721 shutdown_connections 2023-07-18T03:11:15.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.500+0000 7f525f43ab80 1 --2- 172.21.15.101:0/2396413721 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x56275b5ea660 0x56275b5eca70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.500+0000 7f525f43ab80 1 --2- 172.21.15.101:0/2396413721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56275b57b2d0 0x56275b58c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.500+0000 7f525f43ab80 1 --2- 172.21.15.101:0/2396413721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56275b586ad0 0x56275b5e8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.500+0000 7f525f43ab80 1 --2- 172.21.15.101:0/2396413721 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56275b57bbe0 0x56275b58cbe0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.500+0000 7f525f43ab80 1 -- 172.21.15.101:0/2396413721 >> 172.21.15.101:0/2396413721 conn(0x56275b571da0 msgr2=0x56275b592700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:15.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.500+0000 7f522d7fa700 0 client.16263 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:11:15.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.500+0000 7f525f43ab80 1 -- 172.21.15.101:0/2396413721 shutdown_connections 2023-07-18T03:11:15.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.500+0000 7f525f43ab80 1 -- 172.21.15.101:0/2396413721 wait complete. 2023-07-18T03:11:15.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.501+0000 7f525f43ab80 20 client.16263 trim_cache size 0 max 16384 2023-07-18T03:11:15.501 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:11:15.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.501+0000 7f524bbff700 1 -- 172.21.15.101:0/2169780831 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f52300a12c0 msgr2=0x7f52300a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:15.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.501+0000 7f524bbff700 1 --2- 172.21.15.101:0/2169780831 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f52300a12c0 0x7f52300a3770 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5240096a10 tx=0x7f5240010b80 comp rx=0 tx=0).stop 2023-07-18T03:11:15.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.501+0000 7f524bbff700 1 -- 172.21.15.101:0/2169780831 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5244100ca0 msgr2=0x7f5244197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:15.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.501+0000 7f524bbff700 1 --2- 172.21.15.101:0/2169780831 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5244100ca0 0x7f5244197ea0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f524000e9a0 tx=0x7f524000ecb0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.501+0000 7f524bbff700 1 -- 172.21.15.101:0/2169780831 shutdown_connections 2023-07-18T03:11:15.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.501+0000 7f524bbff700 1 --2- 172.21.15.101:0/2169780831 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f52300a12c0 0x7f52300a3770 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.501+0000 7f524bbff700 1 --2- 172.21.15.101:0/2169780831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f524410a070 0x7f524419c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.501+0000 7f524bbff700 1 --2- 172.21.15.101:0/2169780831 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5244100ca0 0x7f5244197ea0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.501+0000 7f524bbff700 1 --2- 172.21.15.101:0/2169780831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52441015b0 0x7f52441983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.501+0000 7f524bbff700 1 -- 172.21.15.101:0/2169780831 >> 172.21.15.101:0/2169780831 conn(0x7f52440fa510 msgr2=0x7f524410c190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:15.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.501+0000 7f524bbff700 1 -- 172.21.15.101:0/2169780831 shutdown_connections 2023-07-18T03:11:15.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.501+0000 7f524bbff700 1 -- 172.21.15.101:0/2169780831 wait complete. 2023-07-18T03:11:15.512 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-18T03:11:15.542 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:15 smithi155 ceph-mon[128885]: pgmap v113: 129 pgs: 129 active+clean; 144 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 239 op/s 2023-07-18T03:11:15.573 INFO:tasks.workunit.client.0.smithi101.stdout: 1 248343 9.08 MB/sec execute 40 sec latency 16.880 ms 2023-07-18T03:11:15.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.743+0000 7f4ba35ff700 1 -- 172.21.15.101:0/452262969 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b9c0ffae0 msgr2=0x7f4b9c0fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:15.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.743+0000 7f4ba35ff700 1 --2- 172.21.15.101:0/452262969 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b9c0ffae0 0x7f4b9c0fff50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c009990 tx=0x7f4b8c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.743+0000 7f4ba35ff700 1 -- 172.21.15.101:0/452262969 shutdown_connections 2023-07-18T03:11:15.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.743+0000 7f4ba35ff700 1 --2- 172.21.15.101:0/452262969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b9c100490 0x7f4b9c10c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.743+0000 7f4ba35ff700 1 --2- 172.21.15.101:0/452262969 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4b9c105af0 0x7f4b9c105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.743+0000 7f4ba35ff700 1 --2- 172.21.15.101:0/452262969 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b9c0ffae0 0x7f4b9c0fff50 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.743+0000 7f4ba35ff700 1 -- 172.21.15.101:0/452262969 >> 172.21.15.101:0/452262969 conn(0x7f4b9c0fb5e0 msgr2=0x7f4b9c0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:15.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.743+0000 7f4ba35ff700 1 -- 172.21.15.101:0/452262969 shutdown_connections 2023-07-18T03:11:15.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.743+0000 7f4ba35ff700 1 -- 172.21.15.101:0/452262969 wait complete. 2023-07-18T03:11:15.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.744+0000 7f4ba35ff700 1 Processor -- start 2023-07-18T03:11:15.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.744+0000 7f4ba35ff700 1 -- start start 2023-07-18T03:11:15.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.745+0000 7f4ba35ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b9c0ffae0 0x7f4b9c197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:15.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.745+0000 7f4ba35ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4b9c100490 0x7f4b9c198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:15.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.745+0000 7f4ba35ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b9c105af0 0x7f4b9c19c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:15.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.745+0000 7f4ba35ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9c19cde0 con 0x7f4b9c100490 2023-07-18T03:11:15.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.745+0000 7f4ba35ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9c19cf50 con 0x7f4b9c105af0 2023-07-18T03:11:15.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.745+0000 7f4ba35ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9c19d110 con 0x7f4b9c0ffae0 2023-07-18T03:11:15.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.745+0000 7f4ba139b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b9c0ffae0 0x7f4b9c197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:15.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.745+0000 7f4ba139b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b9c0ffae0 0x7f4b9c197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:36274/0 (socket says 172.21.15.101:36274) 2023-07-18T03:11:15.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.745+0000 7f4ba139b700 1 -- 172.21.15.101:0/3247899486 learned_addr learned my addr 172.21.15.101:0/3247899486 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:15.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.745+0000 7f4ba0b9a700 1 --2- 172.21.15.101:0/3247899486 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4b9c100490 0x7f4b9c198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:15.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.745+0000 7f4ba1b9c700 1 --2- 172.21.15.101:0/3247899486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b9c105af0 0x7f4b9c19c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:15.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.745+0000 7f4ba139b700 1 -- 172.21.15.101:0/3247899486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b9c105af0 msgr2=0x7f4b9c19c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:15.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.745+0000 7f4ba139b700 1 --2- 172.21.15.101:0/3247899486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b9c105af0 0x7f4b9c19c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.745+0000 7f4ba139b700 1 -- 172.21.15.101:0/3247899486 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4b9c100490 msgr2=0x7f4b9c198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:15.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.745+0000 7f4ba139b700 1 --2- 172.21.15.101:0/3247899486 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4b9c100490 0x7f4b9c198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.745+0000 7f4ba139b700 1 -- 172.21.15.101:0/3247899486 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b8c0095f0 con 0x7f4b9c0ffae0 2023-07-18T03:11:15.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.746+0000 7f4ba139b700 1 --2- 172.21.15.101:0/3247899486 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b9c0ffae0 0x7f4b9c197e00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4b9800e9a0 tx=0x7f4b9800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:15.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.746+0000 7f4b927fc700 1 -- 172.21.15.101:0/3247899486 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b98009790 con 0x7f4b9c0ffae0 2023-07-18T03:11:15.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.746+0000 7f4b927fc700 1 -- 172.21.15.101:0/3247899486 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b98010430 con 0x7f4b9c0ffae0 2023-07-18T03:11:15.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.746+0000 7f4ba35ff700 1 -- 172.21.15.101:0/3247899486 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b9c19d3f0 con 0x7f4b9c0ffae0 2023-07-18T03:11:15.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.746+0000 7f4b927fc700 1 -- 172.21.15.101:0/3247899486 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b98018680 con 0x7f4b9c0ffae0 2023-07-18T03:11:15.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.746+0000 7f4ba35ff700 1 -- 172.21.15.101:0/3247899486 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b9c1a4c30 con 0x7f4b9c0ffae0 2023-07-18T03:11:15.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.750+0000 7f4b927fc700 1 -- 172.21.15.101:0/3247899486 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f4b980105a0 con 0x7f4b9c0ffae0 2023-07-18T03:11:15.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.751+0000 7f4bb6f6cb80 1 Processor -- start 2023-07-18T03:11:15.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.752+0000 7f4bb6f6cb80 1 -- start start 2023-07-18T03:11:15.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.752+0000 7f4bb6f6cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56162b436ae0 0x56162b436eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:15.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.752+0000 7f4bb6f6cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56162b4373f0 0x56162b441da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:15.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.752+0000 7f4bb6f6cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56162b4422e0 0x56162b4446c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:15.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.752+0000 7f4bb6f6cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56162b334cb0 con 0x56162b4373f0 2023-07-18T03:11:15.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.752+0000 7f4bb6f6cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56162b387a40 con 0x56162b4422e0 2023-07-18T03:11:15.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.752+0000 7f4bb6f6cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56162b380ed0 con 0x56162b436ae0 2023-07-18T03:11:15.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.753+0000 7f4ba1b9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56162b436ae0 0x56162b436eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:15.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.753+0000 7f4ba1b9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56162b436ae0 0x56162b436eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:36278/0 (socket says 172.21.15.101:36278) 2023-07-18T03:11:15.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.753+0000 7f4ba1b9c700 1 -- 172.21.15.101:0/2592625550 learned_addr learned my addr 172.21.15.101:0/2592625550 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:15.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.753+0000 7f4ba139b700 1 --2- 172.21.15.101:0/2592625550 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56162b4373f0 0x56162b441da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:15.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.753+0000 7f4ba0b9a700 1 --2- 172.21.15.101:0/2592625550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56162b4422e0 0x56162b4446c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:15.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.753+0000 7f4b927fc700 1 --2- 172.21.15.101:0/3247899486 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4b880a12c0 0x7f4b880a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:15.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.753+0000 7f4ba0b9a700 1 -- 172.21.15.101:0/2592625550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56162b436ae0 msgr2=0x56162b436eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:15.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.753+0000 7f4ba0b9a700 1 --2- 172.21.15.101:0/2592625550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56162b436ae0 0x56162b436eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.753+0000 7f4ba0b9a700 1 -- 172.21.15.101:0/2592625550 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56162b4373f0 msgr2=0x56162b441da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:15.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.753+0000 7f4ba0b9a700 1 --2- 172.21.15.101:0/2592625550 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56162b4373f0 0x56162b441da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.753+0000 7f4ba0b9a700 1 -- 172.21.15.101:0/2592625550 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56162b1aede0 con 0x56162b4422e0 2023-07-18T03:11:15.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.753+0000 7f4ba139b700 1 --2- 172.21.15.101:0/2592625550 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56162b4373f0 0x56162b441da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:11:15.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.753+0000 7f4b927fc700 1 -- 172.21.15.101:0/3247899486 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f4b98014070 con 0x7f4b9c0ffae0 2023-07-18T03:11:15.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.753+0000 7f4ba1b9c700 1 --2- 172.21.15.101:0/3247899486 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4b880a12c0 0x7f4b880a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:15.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.754+0000 7f4ba0b9a700 1 --2- 172.21.15.101:0/2592625550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56162b4422e0 0x56162b4446c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c00b2e0 tx=0x7f4b8c005ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a63e8b550f637d49 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:15.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.754+0000 7f4b877fe700 1 -- 172.21.15.101:0/2592625550 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c01d070 con 0x56162b4422e0 2023-07-18T03:11:15.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.754+0000 7f4ba1b9c700 1 --2- 172.21.15.101:0/3247899486 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4b880a12c0 0x7f4b880a3770 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4b940075c0 tx=0x7f4b94009040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:15.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.754+0000 7f4b877fe700 1 -- 172.21.15.101:0/2592625550 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b8c01e040 con 0x56162b4422e0 2023-07-18T03:11:15.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.754+0000 7f4b877fe700 1 -- 172.21.15.101:0/2592625550 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c0047f0 con 0x56162b4422e0 2023-07-18T03:11:15.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.755+0000 7f4bb6f6cb80 1 -- 172.21.15.101:0/2592625550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56162b4422e0 msgr2=0x56162b4446c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:15.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.755+0000 7f4bb6f6cb80 1 --2- 172.21.15.101:0/2592625550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56162b4422e0 0x56162b4446c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c00b2e0 tx=0x7f4b8c005ea0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.755+0000 7f4bb6f6cb80 1 -- 172.21.15.101:0/2592625550 shutdown_connections 2023-07-18T03:11:15.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.755+0000 7f4bb6f6cb80 1 --2- 172.21.15.101:0/2592625550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56162b4422e0 0x56162b4446c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.755+0000 7f4bb6f6cb80 1 --2- 172.21.15.101:0/2592625550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56162b436ae0 0x56162b436eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.755+0000 7f4bb6f6cb80 1 --2- 172.21.15.101:0/2592625550 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56162b4373f0 0x56162b441da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.755+0000 7f4bb6f6cb80 1 -- 172.21.15.101:0/2592625550 >> 172.21.15.101:0/2592625550 conn(0x56162b424da0 msgr2=0x56162b42a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:15.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.755+0000 7f4bb6f6cb80 1 -- 172.21.15.101:0/2592625550 shutdown_connections 2023-07-18T03:11:15.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.755+0000 7f4bb6f6cb80 1 -- 172.21.15.101:0/2592625550 wait complete. 2023-07-18T03:11:15.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.756+0000 7f4bb6f6cb80 1 -- start start 2023-07-18T03:11:15.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.756+0000 7f4bb6f6cb80 1 Processor -- start 2023-07-18T03:11:15.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.756+0000 7f4bb6f6cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56162b436ae0 0x56162b441ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:15.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.756+0000 7f4bb6f6cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56162b4373f0 0x56162b43eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:15.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.756+0000 7f4bb6f6cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56162b4422e0 0x56162b43f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:15.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.756+0000 7f4bb6f6cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56162b320ac0 con 0x56162b4373f0 2023-07-18T03:11:15.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.756+0000 7f4bb6f6cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56162b3207c0 con 0x56162b4422e0 2023-07-18T03:11:15.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.756+0000 7f4bb6f6cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56162b204920 con 0x56162b436ae0 2023-07-18T03:11:15.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.756+0000 7f4bb6f6cb80 10 client.0 _ll_register_callbacks cb 0x56162b31e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:11:15.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.756+0000 7f4ba139b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56162b436ae0 0x56162b441ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:15.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.757+0000 7f4ba139b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56162b436ae0 0x56162b441ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:36294/0 (socket says 172.21.15.101:36294) 2023-07-18T03:11:15.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.757+0000 7f4ba139b700 1 -- 172.21.15.101:0/3184228645 learned_addr learned my addr 172.21.15.101:0/3184228645 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:15.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.757+0000 7f4ba0b9a700 1 --2- 172.21.15.101:0/3184228645 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56162b4373f0 0x56162b43eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:15.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.757+0000 7f4ba1b9c700 1 --2- 172.21.15.101:0/3184228645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56162b4422e0 0x56162b43f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:15.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.757+0000 7f4ba139b700 1 -- 172.21.15.101:0/3184228645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56162b4422e0 msgr2=0x56162b43f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:15.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.757+0000 7f4ba139b700 1 --2- 172.21.15.101:0/3184228645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56162b4422e0 0x56162b43f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.757+0000 7f4ba139b700 1 -- 172.21.15.101:0/3184228645 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56162b4373f0 msgr2=0x56162b43eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:15.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.757+0000 7f4ba139b700 1 --2- 172.21.15.101:0/3184228645 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56162b4373f0 0x56162b43eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.757+0000 7f4ba139b700 1 -- 172.21.15.101:0/3184228645 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b9c10d220 con 0x56162b436ae0 2023-07-18T03:11:15.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.757+0000 7f4ba139b700 1 --2- 172.21.15.101:0/3184228645 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56162b436ae0 0x56162b441ce0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4b98099c20 tx=0x7f4b980cab00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=711516e9fad419e1 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:15.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.757+0000 7f4b84ff9700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:11:15.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.757+0000 7f4b84ff9700 1 -- 172.21.15.101:0/3184228645 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b98099f20 con 0x56162b436ae0 2023-07-18T03:11:15.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.758+0000 7f4b84ff9700 1 -- 172.21.15.101:0/3184228645 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f4b9809df50 con 0x56162b436ae0 2023-07-18T03:11:15.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.758+0000 7f4bb6f6cb80 1 -- 172.21.15.101:0/3184228645 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56162b2da4f0 con 0x56162b436ae0 2023-07-18T03:11:15.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.758+0000 7f4b84ff9700 1 -- 172.21.15.101:0/3184228645 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b980a2370 con 0x56162b436ae0 2023-07-18T03:11:15.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.758+0000 7f4bb6f6cb80 10 client.26344 fetch_fsmap learned FSMap version 25 2023-07-18T03:11:15.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.758+0000 7f4bb6f6cb80 1 -- 172.21.15.101:0/3184228645 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56162b197c60 con 0x56162b436ae0 2023-07-18T03:11:15.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.758+0000 7f4b84ff9700 1 -- 172.21.15.101:0/3184228645 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b980b1020 con 0x56162b436ae0 2023-07-18T03:11:15.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.758+0000 7f4b84ff9700 1 -- 172.21.15.101:0/3184228645 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f4b980a1f70 con 0x56162b436ae0 2023-07-18T03:11:15.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.758+0000 7f4bb6f6cb80 10 client.26344 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:11:15.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.758+0000 7f4bb6f6cb80 10 client.26344 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:11:15.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.758+0000 7f4bb6f6cb80 20 client.26344 populate_metadata read hostname 'smithi101' 2023-07-18T03:11:15.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.758+0000 7f4bb6f6cb80 1 --2- 172.21.15.101:0/3184228645 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x56162b451b60 0x56162b453f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:15.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.758+0000 7f4bb6f6cb80 4 client.26344 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:11:15.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.758+0000 7f4bb6f6cb80 1 -- 172.21.15.101:0/3184228645 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56162b379eb0 con 0x56162b451b60 2023-07-18T03:11:15.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.759+0000 7f4ba0b9a700 1 --2- 172.21.15.101:0/3184228645 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x56162b451b60 0x56162b453f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:15.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.759+0000 7f4ba0b9a700 1 --2- 172.21.15.101:0/3184228645 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x56162b451b60 0x56162b453f40 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8033d17f9333acf server_cookie=af374616eedac891 in_seq=0 out_seq=0 2023-07-18T03:11:15.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.759+0000 7f4b84ff9700 10 client.26344 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:11:15.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.760+0000 7f4b84ff9700 1 -- 172.21.15.101:0/3184228645 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56162b379eb0 con 0x56162b451b60 2023-07-18T03:11:15.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.760+0000 7f4b84ff9700 10 client.26344 handle_command_reply: tid=0 2023-07-18T03:11:15.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.760+0000 7f4bb6f6cb80 1 client.26344 shutdown 2023-07-18T03:11:15.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.760+0000 7f4bb6f6cb80 1 -- 172.21.15.101:0/3184228645 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56162b436ae0 msgr2=0x56162b441ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:15.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.760+0000 7f4bb6f6cb80 1 --2- 172.21.15.101:0/3184228645 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56162b436ae0 0x56162b441ce0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4b98099c20 tx=0x7f4b980cab00 comp rx=0 tx=0).stop 2023-07-18T03:11:15.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.760+0000 7f4bb6f6cb80 1 -- 172.21.15.101:0/3184228645 shutdown_connections 2023-07-18T03:11:15.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.760+0000 7f4bb6f6cb80 1 --2- 172.21.15.101:0/3184228645 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x56162b451b60 0x56162b453f40 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.760+0000 7f4bb6f6cb80 1 --2- 172.21.15.101:0/3184228645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56162b4422e0 0x56162b43f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.760+0000 7f4bb6f6cb80 1 --2- 172.21.15.101:0/3184228645 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56162b436ae0 0x56162b441ce0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.760+0000 7f4bb6f6cb80 1 --2- 172.21.15.101:0/3184228645 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56162b4373f0 0x56162b43eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.760+0000 7f4b84ff9700 0 client.26344 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:11:15.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.760+0000 7f4bb6f6cb80 1 -- 172.21.15.101:0/3184228645 >> 172.21.15.101:0/3184228645 conn(0x56162b424da0 msgr2=0x56162b42a290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:15.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.760+0000 7f4bb6f6cb80 1 -- 172.21.15.101:0/3184228645 shutdown_connections 2023-07-18T03:11:15.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.760+0000 7f4bb6f6cb80 1 -- 172.21.15.101:0/3184228645 wait complete. 2023-07-18T03:11:15.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.760+0000 7f4bb6f6cb80 20 client.26344 trim_cache size 0 max 16384 2023-07-18T03:11:15.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.774+0000 7f4bb6f6cb80 1 Processor -- start 2023-07-18T03:11:15.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.774+0000 7f4bb6f6cb80 1 -- start start 2023-07-18T03:11:15.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.774+0000 7f4bb6f6cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56162b436ae0 0x56162b43f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:15.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.774+0000 7f4bb6f6cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56162b4373f0 0x56162b441810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:15.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.774+0000 7f4bb6f6cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56162b4422e0 0x56162b43a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:15.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.774+0000 7f4bb6f6cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56162b43d4c0 con 0x56162b436ae0 2023-07-18T03:11:15.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.774+0000 7f4bb6f6cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56162b4266d0 con 0x56162b4373f0 2023-07-18T03:11:15.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.774+0000 7f4bb6f6cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56162b426890 con 0x56162b4422e0 2023-07-18T03:11:15.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.774+0000 7f4ba0b9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56162b4373f0 0x56162b441810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:15.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.774+0000 7f4ba139b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56162b436ae0 0x56162b43f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:15.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.775+0000 7f4ba0b9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56162b4373f0 0x56162b441810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:58344/0 (socket says 172.21.15.101:58344) 2023-07-18T03:11:15.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.775+0000 7f4ba0b9a700 1 -- 172.21.15.101:0/3633168375 learned_addr learned my addr 172.21.15.101:0/3633168375 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:15.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.775+0000 7f4ba1b9c700 1 --2- 172.21.15.101:0/3633168375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56162b4422e0 0x56162b43a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:15.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.775+0000 7f4ba0b9a700 1 -- 172.21.15.101:0/3633168375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56162b4422e0 msgr2=0x56162b43a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:15.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.775+0000 7f4ba0b9a700 1 --2- 172.21.15.101:0/3633168375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56162b4422e0 0x56162b43a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.775+0000 7f4ba0b9a700 1 -- 172.21.15.101:0/3633168375 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56162b436ae0 msgr2=0x56162b43f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:15.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.775+0000 7f4ba0b9a700 1 --2- 172.21.15.101:0/3633168375 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56162b436ae0 0x56162b43f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.775+0000 7f4ba0b9a700 1 -- 172.21.15.101:0/3633168375 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56162b1d5250 con 0x56162b4373f0 2023-07-18T03:11:15.769 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.775+0000 7f4ba139b700 1 --2- 172.21.15.101:0/3633168375 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56162b436ae0 0x56162b43f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:11:15.769 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.775+0000 7f4ba0b9a700 1 --2- 172.21.15.101:0/3633168375 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56162b4373f0 0x56162b441810 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c000c00 tx=0x7f4b8c009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2e2bb551b6959d7 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:15.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.775+0000 7f4b87fff700 1 -- 172.21.15.101:0/3633168375 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c01d040 con 0x56162b4373f0 2023-07-18T03:11:15.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.775+0000 7f4b87fff700 1 -- 172.21.15.101:0/3633168375 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b8c033d20 con 0x56162b4373f0 2023-07-18T03:11:15.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.775+0000 7f4b87fff700 1 -- 172.21.15.101:0/3633168375 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c0053d0 con 0x56162b4373f0 2023-07-18T03:11:15.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.776+0000 7f4bb6f6cb80 1 -- 172.21.15.101:0/3633168375 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56162b4373f0 msgr2=0x56162b441810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:15.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.776+0000 7f4bb6f6cb80 1 --2- 172.21.15.101:0/3633168375 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56162b4373f0 0x56162b441810 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c000c00 tx=0x7f4b8c009310 comp rx=0 tx=0).stop 2023-07-18T03:11:15.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.776+0000 7f4bb6f6cb80 1 -- 172.21.15.101:0/3633168375 shutdown_connections 2023-07-18T03:11:15.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.776+0000 7f4bb6f6cb80 1 --2- 172.21.15.101:0/3633168375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56162b4422e0 0x56162b43a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.776+0000 7f4bb6f6cb80 1 --2- 172.21.15.101:0/3633168375 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56162b436ae0 0x56162b43f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.776+0000 7f4bb6f6cb80 1 --2- 172.21.15.101:0/3633168375 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56162b4373f0 0x56162b441810 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.776+0000 7f4bb6f6cb80 1 -- 172.21.15.101:0/3633168375 >> 172.21.15.101:0/3633168375 conn(0x56162b424da0 msgr2=0x56162b428e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:15.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.776+0000 7f4bb6f6cb80 1 -- 172.21.15.101:0/3633168375 shutdown_connections 2023-07-18T03:11:15.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.776+0000 7f4bb6f6cb80 1 -- 172.21.15.101:0/3633168375 wait complete. 2023-07-18T03:11:15.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.777+0000 7f4bb6f6cb80 1 -- start start 2023-07-18T03:11:15.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.777+0000 7f4bb6f6cb80 1 Processor -- start 2023-07-18T03:11:15.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.777+0000 7f4bb6f6cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56162b436ae0 0x56162b446a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:15.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.777+0000 7f4bb6f6cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56162b4373f0 0x56162b446f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:15.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.777+0000 7f4bb6f6cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56162b4422e0 0x56162b4a49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:15.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.777+0000 7f4bb6f6cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56162b4a4f30 con 0x56162b4373f0 2023-07-18T03:11:15.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.777+0000 7f4bb6f6cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56162b4a5070 con 0x56162b436ae0 2023-07-18T03:11:15.774 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.777+0000 7f4bb6f6cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56162b4a51b0 con 0x56162b4422e0 2023-07-18T03:11:15.774 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.777+0000 7f4bb6f6cb80 10 client.0 _ll_register_callbacks cb 0x7f4b980962e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:11:15.774 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.778+0000 7f4ba139b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56162b436ae0 0x56162b446a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:15.774 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.778+0000 7f4ba1b9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56162b4422e0 0x56162b4a49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:15.774 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.778+0000 7f4ba0b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56162b4373f0 0x56162b446f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:15.774 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.778+0000 7f4ba139b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56162b436ae0 0x56162b446a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:58350/0 (socket says 172.21.15.101:58350) 2023-07-18T03:11:15.774 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.778+0000 7f4ba139b700 1 -- 172.21.15.101:0/2152220560 learned_addr learned my addr 172.21.15.101:0/2152220560 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:15.775 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.778+0000 7f4ba1b9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56162b4422e0 0x56162b4a49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:36316/0 (socket says 172.21.15.101:36316) 2023-07-18T03:11:15.775 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.778+0000 7f4ba139b700 1 -- 172.21.15.101:0/2152220560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56162b4422e0 msgr2=0x56162b4a49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:15.775 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.778+0000 7f4ba139b700 1 --2- 172.21.15.101:0/2152220560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56162b4422e0 0x56162b4a49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.775 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.778+0000 7f4ba139b700 1 -- 172.21.15.101:0/2152220560 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56162b4373f0 msgr2=0x56162b446f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:15.775 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.778+0000 7f4ba139b700 1 --2- 172.21.15.101:0/2152220560 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56162b4373f0 0x56162b446f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.775 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.778+0000 7f4ba139b700 1 -- 172.21.15.101:0/2152220560 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56162b197c60 con 0x56162b436ae0 2023-07-18T03:11:15.775 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.778+0000 7f4ba1b9c700 1 --2- 172.21.15.101:0/2152220560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56162b4422e0 0x56162b4a49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:11:15.775 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.778+0000 7f4ba0b9a700 1 --2- 172.21.15.101:0/2152220560 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56162b4373f0 0x56162b446f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:11:15.776 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.778+0000 7f4ba139b700 1 --2- 172.21.15.101:0/2152220560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56162b436ae0 0x56162b446a30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4b98009230 tx=0x7f4b9809a4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9253119d4e0911aa server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:15.776 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.778+0000 7f4b84ff9700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:11:15.776 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.779+0000 7f4b84ff9700 1 -- 172.21.15.101:0/2152220560 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b98099a60 con 0x56162b436ae0 2023-07-18T03:11:15.776 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.779+0000 7f4bb6f6cb80 1 -- 172.21.15.101:0/2152220560 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b8c01e040 con 0x56162b436ae0 2023-07-18T03:11:15.776 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.779+0000 7f4b84ff9700 1 -- 172.21.15.101:0/2152220560 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f4b980b2030 con 0x56162b436ae0 2023-07-18T03:11:15.776 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.779+0000 7f4b84ff9700 1 -- 172.21.15.101:0/2152220560 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b9809df50 con 0x56162b436ae0 2023-07-18T03:11:15.776 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.779+0000 7f4bb6f6cb80 10 client.26327 fetch_fsmap learned FSMap version 25 2023-07-18T03:11:15.777 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.779+0000 7f4bb6f6cb80 1 -- 172.21.15.101:0/2152220560 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56162b1aede0 con 0x56162b436ae0 2023-07-18T03:11:15.777 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.779+0000 7f4b84ff9700 1 -- 172.21.15.101:0/2152220560 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b8c01e040 con 0x56162b436ae0 2023-07-18T03:11:15.777 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.779+0000 7f4b84ff9700 1 -- 172.21.15.101:0/2152220560 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f4b980a1ee0 con 0x56162b436ae0 2023-07-18T03:11:15.777 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.779+0000 7f4bb6f6cb80 10 client.26327 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:11:15.777 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.779+0000 7f4bb6f6cb80 10 client.26327 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:11:15.777 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.779+0000 7f4bb6f6cb80 20 client.26327 populate_metadata read hostname 'smithi101' 2023-07-18T03:11:15.777 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.779+0000 7f4bb6f6cb80 1 --2- 172.21.15.101:0/2152220560 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x56162b4a6440 0x56162b4a8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:15.777 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.779+0000 7f4bb6f6cb80 4 client.26327 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:11:15.778 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.779+0000 7f4bb6f6cb80 1 -- 172.21.15.101:0/2152220560 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56162b33c050 con 0x56162b4a6440 2023-07-18T03:11:15.778 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.780+0000 7f4ba0b9a700 1 --2- 172.21.15.101:0/2152220560 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x56162b4a6440 0x56162b4a8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:15.779 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.780+0000 7f4ba0b9a700 1 --2- 172.21.15.101:0/2152220560 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x56162b4a6440 0x56162b4a8850 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c5633fe5d86643cc server_cookie=aaa6731481037da9 in_seq=0 out_seq=0 2023-07-18T03:11:15.779 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.780+0000 7f4b84ff9700 10 client.26327 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:11:15.779 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.780+0000 7f4b84ff9700 1 -- 172.21.15.101:0/2152220560 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56162b33c050 con 0x56162b4a6440 2023-07-18T03:11:15.779 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.780+0000 7f4b84ff9700 10 client.26327 handle_command_reply: tid=0 2023-07-18T03:11:15.779 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.780+0000 7f4bb6f6cb80 1 client.26327 shutdown 2023-07-18T03:11:15.780 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.780+0000 7f4bb6f6cb80 1 -- 172.21.15.101:0/2152220560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56162b436ae0 msgr2=0x56162b446a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:15.780 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.780+0000 7f4bb6f6cb80 1 --2- 172.21.15.101:0/2152220560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56162b436ae0 0x56162b446a30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4b98009230 tx=0x7f4b9809a4f0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.780 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.781+0000 7f4bb6f6cb80 1 -- 172.21.15.101:0/2152220560 shutdown_connections 2023-07-18T03:11:15.780 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.781+0000 7f4bb6f6cb80 1 --2- 172.21.15.101:0/2152220560 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x56162b4a6440 0x56162b4a8850 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.780 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.781+0000 7f4bb6f6cb80 1 --2- 172.21.15.101:0/2152220560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56162b436ae0 0x56162b446a30 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.780 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.781+0000 7f4b84ff9700 0 client.26327 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:11:15.780 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.781+0000 7f4bb6f6cb80 1 --2- 172.21.15.101:0/2152220560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56162b4422e0 0x56162b4a49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.781+0000 7f4bb6f6cb80 1 --2- 172.21.15.101:0/2152220560 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56162b4373f0 0x56162b446f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.781+0000 7f4bb6f6cb80 1 -- 172.21.15.101:0/2152220560 >> 172.21.15.101:0/2152220560 conn(0x56162b424da0 msgr2=0x56162b43a260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:15.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.781+0000 7f4bb6f6cb80 1 -- 172.21.15.101:0/2152220560 shutdown_connections 2023-07-18T03:11:15.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.781+0000 7f4bb6f6cb80 1 -- 172.21.15.101:0/2152220560 wait complete. 2023-07-18T03:11:15.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.781+0000 7f4bb6f6cb80 20 client.26327 trim_cache size 0 max 16384 2023-07-18T03:11:15.781 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:11:15.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.781+0000 7f4ba35ff700 1 -- 172.21.15.101:0/3247899486 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4b880a12c0 msgr2=0x7f4b880a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:15.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.781+0000 7f4ba35ff700 1 --2- 172.21.15.101:0/3247899486 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4b880a12c0 0x7f4b880a3770 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4b940075c0 tx=0x7f4b94009040 comp rx=0 tx=0).stop 2023-07-18T03:11:15.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.781+0000 7f4ba35ff700 1 -- 172.21.15.101:0/3247899486 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b9c0ffae0 msgr2=0x7f4b9c197e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:15.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.781+0000 7f4ba35ff700 1 --2- 172.21.15.101:0/3247899486 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b9c0ffae0 0x7f4b9c197e00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4b9800e9a0 tx=0x7f4b9800ecb0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.782+0000 7f4ba35ff700 1 -- 172.21.15.101:0/3247899486 shutdown_connections 2023-07-18T03:11:15.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.782+0000 7f4ba35ff700 1 --2- 172.21.15.101:0/3247899486 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4b880a12c0 0x7f4b880a3770 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.782+0000 7f4ba35ff700 1 --2- 172.21.15.101:0/3247899486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b9c105af0 0x7f4b9c19c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.782+0000 7f4ba35ff700 1 --2- 172.21.15.101:0/3247899486 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b9c0ffae0 0x7f4b9c197e00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.782+0000 7f4ba35ff700 1 --2- 172.21.15.101:0/3247899486 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4b9c100490 0x7f4b9c198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:15.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.782+0000 7f4ba35ff700 1 -- 172.21.15.101:0/3247899486 >> 172.21.15.101:0/3247899486 conn(0x7f4b9c0fb5e0 msgr2=0x7f4b9c10ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:15.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.782+0000 7f4ba35ff700 1 -- 172.21.15.101:0/3247899486 shutdown_connections 2023-07-18T03:11:15.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:15.782+0000 7f4ba35ff700 1 -- 172.21.15.101:0/3247899486 wait complete. 2023-07-18T03:11:15.791 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-18T03:11:16.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.014+0000 7fc1087ff700 1 -- 172.21.15.101:0/2038563117 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc100105af0 msgr2=0x7fc100105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:16.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.014+0000 7fc1087ff700 1 --2- 172.21.15.101:0/2038563117 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc100105af0 0x7fc100105ec0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc009990 tx=0x7fc0fc009ca0 comp rx=0 tx=0).stop 2023-07-18T03:11:16.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.014+0000 7fc1087ff700 1 -- 172.21.15.101:0/2038563117 shutdown_connections 2023-07-18T03:11:16.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.014+0000 7fc1087ff700 1 --2- 172.21.15.101:0/2038563117 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc100100490 0x7fc10010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:16.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.014+0000 7fc1087ff700 1 --2- 172.21.15.101:0/2038563117 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc100105af0 0x7fc100105ec0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:16.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.014+0000 7fc1087ff700 1 --2- 172.21.15.101:0/2038563117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1000ffae0 0x7fc1000fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:16.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.014+0000 7fc1087ff700 1 -- 172.21.15.101:0/2038563117 >> 172.21.15.101:0/2038563117 conn(0x7fc1000fb5e0 msgr2=0x7fc1000fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:16.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.014+0000 7fc1087ff700 1 -- 172.21.15.101:0/2038563117 shutdown_connections 2023-07-18T03:11:16.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.014+0000 7fc1087ff700 1 -- 172.21.15.101:0/2038563117 wait complete. 2023-07-18T03:11:16.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.015+0000 7fc1087ff700 1 Processor -- start 2023-07-18T03:11:16.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.015+0000 7fc1087ff700 1 -- start start 2023-07-18T03:11:16.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.015+0000 7fc1087ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1000ffae0 0x7fc100197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:16.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.015+0000 7fc1087ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc100100490 0x7fc100198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:16.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.015+0000 7fc1087ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc100105af0 0x7fc10019c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:16.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.015+0000 7fc1087ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10019ce20 con 0x7fc100105af0 2023-07-18T03:11:16.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.015+0000 7fc1087ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10019cf90 con 0x7fc100100490 2023-07-18T03:11:16.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.015+0000 7fc1087ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10019d150 con 0x7fc1000ffae0 2023-07-18T03:11:16.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.016+0000 7fc106d9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc100105af0 0x7fc10019c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:16.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.016+0000 7fc105d9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc100100490 0x7fc100198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:16.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.016+0000 7fc106d9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc100105af0 0x7fc10019c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:54720/0 (socket says 172.21.15.101:54720) 2023-07-18T03:11:16.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.016+0000 7fc106d9c700 1 -- 172.21.15.101:0/3272845993 learned_addr learned my addr 172.21.15.101:0/3272845993 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:16.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.016+0000 7fc10659b700 1 --2- 172.21.15.101:0/3272845993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1000ffae0 0x7fc100197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:16.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.016+0000 7fc106d9c700 1 -- 172.21.15.101:0/3272845993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1000ffae0 msgr2=0x7fc100197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:16.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.016+0000 7fc106d9c700 1 --2- 172.21.15.101:0/3272845993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1000ffae0 0x7fc100197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:16.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.016+0000 7fc106d9c700 1 -- 172.21.15.101:0/3272845993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc100100490 msgr2=0x7fc100198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:16.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.016+0000 7fc106d9c700 1 --2- 172.21.15.101:0/3272845993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc100100490 0x7fc100198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:16.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.016+0000 7fc106d9c700 1 -- 172.21.15.101:0/3272845993 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0fc0095f0 con 0x7fc100105af0 2023-07-18T03:11:16.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.016+0000 7fc105d9a700 1 --2- 172.21.15.101:0/3272845993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc100100490 0x7fc100198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:11:16.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.016+0000 7fc106d9c700 1 --2- 172.21.15.101:0/3272845993 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc100105af0 0x7fc10019c7e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc0f800ef30 tx=0x7fc0f800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:16.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.016+0000 7fc0f77fe700 1 -- 172.21.15.101:0/3272845993 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8018070 con 0x7fc100105af0 2023-07-18T03:11:16.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.017+0000 7fc0f77fe700 1 -- 172.21.15.101:0/3272845993 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0f800f040 con 0x7fc100105af0 2023-07-18T03:11:16.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.017+0000 7fc1087ff700 1 -- 172.21.15.101:0/3272845993 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc10019d3d0 con 0x7fc100105af0 2023-07-18T03:11:16.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.017+0000 7fc0f77fe700 1 -- 172.21.15.101:0/3272845993 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f800cd00 con 0x7fc100105af0 2023-07-18T03:11:16.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.017+0000 7fc1087ff700 1 -- 172.21.15.101:0/3272845993 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1001a4c70 con 0x7fc100105af0 2023-07-18T03:11:16.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.020+0000 7fc0f77fe700 1 -- 172.21.15.101:0/3272845993 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fc0f8003a00 con 0x7fc100105af0 2023-07-18T03:11:16.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.021+0000 7fc11c073b80 1 Processor -- start 2023-07-18T03:11:16.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.021+0000 7fc11c073b80 1 -- start start 2023-07-18T03:11:16.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.021+0000 7fc0f77fe700 1 --2- 172.21.15.101:0/3272845993 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc0ec0a12c0 0x7fc0ec0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:16.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.021+0000 7fc0f77fe700 1 -- 172.21.15.101:0/3272845993 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fc0f80ccb40 con 0x7fc100105af0 2023-07-18T03:11:16.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.022+0000 7fc11c073b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f36c66e2d0 0x55f36c66e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:16.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.022+0000 7fc11c073b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f36c66ebe0 0x55f36c679590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:16.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.022+0000 7fc105d9a700 1 --2- 172.21.15.101:0/3272845993 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc0ec0a12c0 0x7fc0ec0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:16.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.022+0000 7fc11c073b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f36c679ad0 0x55f36c67beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:16.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.022+0000 7fc11c073b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f36c574cb0 con 0x55f36c679ad0 2023-07-18T03:11:16.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.022+0000 7fc11c073b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f36c5c7a40 con 0x55f36c66e2d0 2023-07-18T03:11:16.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.022+0000 7fc11c073b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f36c5c0ed0 con 0x55f36c66ebe0 2023-07-18T03:11:16.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.022+0000 7fc10659b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f36c66e2d0 0x55f36c66e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:16.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.022+0000 7fc10659b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f36c66e2d0 0x55f36c66e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:55958/0 (socket says 172.21.15.101:55958) 2023-07-18T03:11:16.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.022+0000 7fc10659b700 1 -- 172.21.15.101:0/4241025274 learned_addr learned my addr 172.21.15.101:0/4241025274 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:16.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.022+0000 7fc105d9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f36c66ebe0 0x55f36c679590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:16.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.022+0000 7fc106d9c700 1 --2- 172.21.15.101:0/4241025274 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f36c679ad0 0x55f36c67beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:16.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.022+0000 7fc10659b700 1 -- 172.21.15.101:0/4241025274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f36c66ebe0 msgr2=0x55f36c679590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:16.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.022+0000 7fc10659b700 1 --2- 172.21.15.101:0/4241025274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f36c66ebe0 0x55f36c679590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:16.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.022+0000 7fc10659b700 1 -- 172.21.15.101:0/4241025274 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f36c679ad0 msgr2=0x55f36c67beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:16.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.022+0000 7fc10659b700 1 --2- 172.21.15.101:0/4241025274 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f36c679ad0 0x55f36c67beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:16.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.022+0000 7fc10659b700 1 -- 172.21.15.101:0/4241025274 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f36c3eede0 con 0x55f36c66e2d0 2023-07-18T03:11:16.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.023+0000 7fc10659b700 1 --2- 172.21.15.101:0/4241025274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f36c66e2d0 0x55f36c66e6a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc005ea0 tx=0x7fc0fc005ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef8c02d263ee5033 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:16.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.023+0000 7fc105d9a700 1 --2- 172.21.15.101:0/3272845993 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc0ec0a12c0 0x7fc0ec0a3770 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc100199450 tx=0x7fc0f0009cc0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:16.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.023+0000 7fc0f4ff9700 1 -- 172.21.15.101:0/4241025274 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc01d070 con 0x55f36c66e2d0 2023-07-18T03:11:16.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.023+0000 7fc0f4ff9700 1 -- 172.21.15.101:0/4241025274 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0fc01e040 con 0x55f36c66e2d0 2023-07-18T03:11:16.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.023+0000 7fc0f4ff9700 1 -- 172.21.15.101:0/4241025274 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc004880 con 0x55f36c66e2d0 2023-07-18T03:11:16.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.024+0000 7fc11c073b80 1 -- 172.21.15.101:0/4241025274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f36c66e2d0 msgr2=0x55f36c66e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:16.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.024+0000 7fc11c073b80 1 --2- 172.21.15.101:0/4241025274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f36c66e2d0 0x55f36c66e6a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc005ea0 tx=0x7fc0fc005ed0 comp rx=0 tx=0).stop 2023-07-18T03:11:16.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.024+0000 7fc11c073b80 1 -- 172.21.15.101:0/4241025274 shutdown_connections 2023-07-18T03:11:16.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.024+0000 7fc11c073b80 1 --2- 172.21.15.101:0/4241025274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f36c66e2d0 0x55f36c66e6a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:16.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.024+0000 7fc11c073b80 1 --2- 172.21.15.101:0/4241025274 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f36c679ad0 0x55f36c67beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:16.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.024+0000 7fc11c073b80 1 --2- 172.21.15.101:0/4241025274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f36c66ebe0 0x55f36c679590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:16.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.024+0000 7fc11c073b80 1 -- 172.21.15.101:0/4241025274 >> 172.21.15.101:0/4241025274 conn(0x55f36c664da0 msgr2=0x55f36c66a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:16.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.024+0000 7fc11c073b80 1 -- 172.21.15.101:0/4241025274 shutdown_connections 2023-07-18T03:11:16.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.024+0000 7fc11c073b80 1 -- 172.21.15.101:0/4241025274 wait complete. 2023-07-18T03:11:16.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.024+0000 7fc11c073b80 1 -- start start 2023-07-18T03:11:16.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.025+0000 7fc11c073b80 1 Processor -- start 2023-07-18T03:11:16.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.025+0000 7fc11c073b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f36c66e2d0 0x55f36c679510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:16.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.025+0000 7fc11c073b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f36c66ebe0 0x55f36c676790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:16.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.025+0000 7fc11c073b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f36c679ad0 0x55f36c676ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:16.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.025+0000 7fc11c073b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f36c560ac0 con 0x55f36c66e2d0 2023-07-18T03:11:16.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.025+0000 7fc11c073b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f36c5607c0 con 0x55f36c679ad0 2023-07-18T03:11:16.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.025+0000 7fc11c073b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f36c444920 con 0x55f36c66ebe0 2023-07-18T03:11:16.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.025+0000 7fc11c073b80 10 client.0 _ll_register_callbacks cb 0x55f36c55e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:11:16.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.025+0000 7fc105d9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f36c66ebe0 0x55f36c676790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:16.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.025+0000 7fc10659b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f36c66e2d0 0x55f36c679510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:16.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.025+0000 7fc105d9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f36c66ebe0 0x55f36c676790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:52854/0 (socket says 172.21.15.101:52854) 2023-07-18T03:11:16.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.025+0000 7fc105d9a700 1 -- 172.21.15.101:0/711317683 learned_addr learned my addr 172.21.15.101:0/711317683 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:16.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.025+0000 7fc106d9c700 1 --2- 172.21.15.101:0/711317683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f36c679ad0 0x55f36c676ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:16.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.026+0000 7fc106d9c700 1 -- 172.21.15.101:0/711317683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f36c66ebe0 msgr2=0x55f36c676790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:16.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.026+0000 7fc106d9c700 1 --2- 172.21.15.101:0/711317683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f36c66ebe0 0x55f36c676790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:16.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.026+0000 7fc106d9c700 1 -- 172.21.15.101:0/711317683 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f36c66e2d0 msgr2=0x55f36c679510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:16.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.026+0000 7fc106d9c700 1 --2- 172.21.15.101:0/711317683 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f36c66e2d0 0x55f36c679510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:16.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.026+0000 7fc106d9c700 1 -- 172.21.15.101:0/711317683 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc10010d220 con 0x55f36c679ad0 2023-07-18T03:11:16.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.026+0000 7fc105d9a700 1 --2- 172.21.15.101:0/711317683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f36c66ebe0 0x55f36c676790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:11:16.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.026+0000 7fc106d9c700 1 --2- 172.21.15.101:0/711317683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f36c679ad0 0x55f36c676ed0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc0f80c9ee0 tx=0x7fc0f8004b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=baadc606e2939fb3 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:16.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.026+0000 7fc0e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:11:16.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.026+0000 7fc0e9ffb700 1 -- 172.21.15.101:0/711317683 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f809d920 con 0x55f36c679ad0 2023-07-18T03:11:16.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.026+0000 7fc11c073b80 1 -- 172.21.15.101:0/711317683 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f36c51a4f0 con 0x55f36c679ad0 2023-07-18T03:11:16.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.026+0000 7fc0e9ffb700 1 -- 172.21.15.101:0/711317683 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fc0f80a1cb0 con 0x55f36c679ad0 2023-07-18T03:11:16.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.026+0000 7fc0e9ffb700 1 -- 172.21.15.101:0/711317683 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0f8098de0 con 0x55f36c679ad0 2023-07-18T03:11:16.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.026+0000 7fc11c073b80 10 client.26339 fetch_fsmap learned FSMap version 25 2023-07-18T03:11:16.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.026+0000 7fc11c073b80 1 -- 172.21.15.101:0/711317683 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f36c3d7c60 con 0x55f36c679ad0 2023-07-18T03:11:16.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.027+0000 7fc0e9ffb700 1 -- 172.21.15.101:0/711317683 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f36c51a4f0 con 0x55f36c679ad0 2023-07-18T03:11:16.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.027+0000 7fc0e9ffb700 1 -- 172.21.15.101:0/711317683 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fc0f809dbf0 con 0x55f36c679ad0 2023-07-18T03:11:16.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.027+0000 7fc11c073b80 10 client.26339 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:11:16.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.027+0000 7fc11c073b80 10 client.26339 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:11:16.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.027+0000 7fc11c073b80 20 client.26339 populate_metadata read hostname 'smithi101' 2023-07-18T03:11:16.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.027+0000 7fc11c073b80 1 --2- 172.21.15.101:0/711317683 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f36c689380 0x55f36c68b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:16.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.027+0000 7fc11c073b80 4 client.26339 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:11:16.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.027+0000 7fc11c073b80 1 -- 172.21.15.101:0/711317683 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f36c5b9eb0 con 0x55f36c689380 2023-07-18T03:11:16.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.027+0000 7fc10659b700 1 --2- 172.21.15.101:0/711317683 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f36c689380 0x55f36c68b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:16.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.028+0000 7fc10659b700 1 --2- 172.21.15.101:0/711317683 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f36c689380 0x55f36c68b790 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=176f590dc0c241bd server_cookie=dd004f8a8ace50b7 in_seq=0 out_seq=0 2023-07-18T03:11:16.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.028+0000 7fc0e9ffb700 10 client.26339 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:11:16.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.029+0000 7fc0e9ffb700 1 -- 172.21.15.101:0/711317683 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f36c5b9eb0 con 0x55f36c689380 2023-07-18T03:11:16.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.029+0000 7fc0e9ffb700 10 client.26339 handle_command_reply: tid=0 2023-07-18T03:11:16.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.029+0000 7fc11c073b80 1 client.26339 shutdown 2023-07-18T03:11:16.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.029+0000 7fc11c073b80 1 -- 172.21.15.101:0/711317683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f36c679ad0 msgr2=0x55f36c676ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:16.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.029+0000 7fc11c073b80 1 --2- 172.21.15.101:0/711317683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f36c679ad0 0x55f36c676ed0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc0f80c9ee0 tx=0x7fc0f8004b50 comp rx=0 tx=0).stop 2023-07-18T03:11:16.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.029+0000 7fc11c073b80 1 -- 172.21.15.101:0/711317683 shutdown_connections 2023-07-18T03:11:16.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.029+0000 7fc11c073b80 1 --2- 172.21.15.101:0/711317683 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f36c689380 0x55f36c68b790 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:16.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.029+0000 7fc11c073b80 1 --2- 172.21.15.101:0/711317683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f36c679ad0 0x55f36c676ed0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:16.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.029+0000 7fc11c073b80 1 --2- 172.21.15.101:0/711317683 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f36c66e2d0 0x55f36c679510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:16.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.029+0000 7fc0e9ffb700 0 client.26339 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:11:16.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.029+0000 7fc11c073b80 1 --2- 172.21.15.101:0/711317683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f36c66ebe0 0x55f36c676790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:16.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.029+0000 7fc11c073b80 1 -- 172.21.15.101:0/711317683 >> 172.21.15.101:0/711317683 conn(0x55f36c664da0 msgr2=0x55f36c66a290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:16.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.029+0000 7fc11c073b80 1 -- 172.21.15.101:0/711317683 shutdown_connections 2023-07-18T03:11:16.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.029+0000 7fc11c073b80 1 -- 172.21.15.101:0/711317683 wait complete. 2023-07-18T03:11:16.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.029+0000 7fc11c073b80 20 client.26339 trim_cache size 0 max 16384 2023-07-18T03:11:16.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.044+0000 7fc11c073b80 1 Processor -- start 2023-07-18T03:11:16.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.044+0000 7fc11c073b80 1 -- start start 2023-07-18T03:11:16.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.045+0000 7fc11c073b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f36c66e2d0 0x55f36c6711b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:16.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.045+0000 7fc11c073b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f36c66ebe0 0x55f36c671a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:16.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.045+0000 7fc11c073b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f36c679ad0 0x55f36c671fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:16.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.045+0000 7fc11c073b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f36c66f160 con 0x55f36c66ebe0 2023-07-18T03:11:16.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.045+0000 7fc11c073b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f36c666ec0 con 0x55f36c66e2d0 2023-07-18T03:11:16.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.045+0000 7fc11c073b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f36c66f540 con 0x55f36c679ad0 2023-07-18T03:11:16.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.045+0000 7fc10659b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f36c66e2d0 0x55f36c6711b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:16.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.045+0000 7fc10659b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f36c66e2d0 0x55f36c6711b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:55974/0 (socket says 172.21.15.101:55974) 2023-07-18T03:11:16.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.045+0000 7fc10659b700 1 -- 172.21.15.101:0/3296449401 learned_addr learned my addr 172.21.15.101:0/3296449401 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:16.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.045+0000 7fc105d9a700 1 --2- 172.21.15.101:0/3296449401 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f36c66ebe0 0x55f36c671a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:16.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.045+0000 7fc106d9c700 1 --2- 172.21.15.101:0/3296449401 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f36c679ad0 0x55f36c671fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:16.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.045+0000 7fc10659b700 1 -- 172.21.15.101:0/3296449401 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f36c679ad0 msgr2=0x55f36c671fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:16.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.045+0000 7fc10659b700 1 --2- 172.21.15.101:0/3296449401 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f36c679ad0 0x55f36c671fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:16.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.045+0000 7fc10659b700 1 -- 172.21.15.101:0/3296449401 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f36c66ebe0 msgr2=0x55f36c671a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:16.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.045+0000 7fc10659b700 1 --2- 172.21.15.101:0/3296449401 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f36c66ebe0 0x55f36c671a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:16.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.045+0000 7fc10659b700 1 -- 172.21.15.101:0/3296449401 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f36c415250 con 0x55f36c66e2d0 2023-07-18T03:11:16.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.045+0000 7fc106d9c700 1 --2- 172.21.15.101:0/3296449401 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f36c679ad0 0x55f36c671fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:11:16.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.045+0000 7fc10659b700 1 --2- 172.21.15.101:0/3296449401 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f36c66e2d0 0x55f36c6711b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc005ea0 tx=0x7fc0fc00b280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d24c890ef445b38a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:16.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.046+0000 7fc0f57fa700 1 -- 172.21.15.101:0/3296449401 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc01d040 con 0x55f36c66e2d0 2023-07-18T03:11:16.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.046+0000 7fc0f57fa700 1 -- 172.21.15.101:0/3296449401 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0fc00be60 con 0x55f36c66e2d0 2023-07-18T03:11:16.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.046+0000 7fc0f57fa700 1 -- 172.21.15.101:0/3296449401 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc004840 con 0x55f36c66e2d0 2023-07-18T03:11:16.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.046+0000 7fc11c073b80 1 -- 172.21.15.101:0/3296449401 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f36c66e2d0 msgr2=0x55f36c6711b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:16.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.046+0000 7fc11c073b80 1 --2- 172.21.15.101:0/3296449401 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f36c66e2d0 0x55f36c6711b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc005ea0 tx=0x7fc0fc00b280 comp rx=0 tx=0).stop 2023-07-18T03:11:16.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.046+0000 7fc11c073b80 1 -- 172.21.15.101:0/3296449401 shutdown_connections 2023-07-18T03:11:16.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.046+0000 7fc11c073b80 1 --2- 172.21.15.101:0/3296449401 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f36c66e2d0 0x55f36c6711b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:16.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.046+0000 7fc11c073b80 1 --2- 172.21.15.101:0/3296449401 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f36c679ad0 0x55f36c671fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:16.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.046+0000 7fc11c073b80 1 --2- 172.21.15.101:0/3296449401 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f36c66ebe0 0x55f36c671a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:16.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.046+0000 7fc11c073b80 1 -- 172.21.15.101:0/3296449401 >> 172.21.15.101:0/3296449401 conn(0x55f36c664da0 msgr2=0x55f36c6855e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:16.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.046+0000 7fc11c073b80 1 -- 172.21.15.101:0/3296449401 shutdown_connections 2023-07-18T03:11:16.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.046+0000 7fc11c073b80 1 -- 172.21.15.101:0/3296449401 wait complete. 2023-07-18T03:11:16.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.047+0000 7fc11c073b80 1 -- start start 2023-07-18T03:11:16.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.047+0000 7fc11c073b80 1 Processor -- start 2023-07-18T03:11:16.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.047+0000 7fc11c073b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f36c66e2d0 0x55f36c67e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:16.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.047+0000 7fc11c073b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f36c66ebe0 0x55f36c67ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:16.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.047+0000 7fc11c073b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f36c679ad0 0x55f36c6dc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:16.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.047+0000 7fc11c073b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f36c67f340 con 0x55f36c66e2d0 2023-07-18T03:11:16.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.048+0000 7fc11c073b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f36c6dc780 con 0x55f36c66ebe0 2023-07-18T03:11:16.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.048+0000 7fc11c073b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f36c6dc8f0 con 0x55f36c679ad0 2023-07-18T03:11:16.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.048+0000 7fc11c073b80 10 client.0 _ll_register_callbacks cb 0x7fc0f8099430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:11:16.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.048+0000 7fc10659b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f36c66e2d0 0x55f36c67e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:16.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.048+0000 7fc10659b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f36c66e2d0 0x55f36c67e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:54736/0 (socket says 172.21.15.101:54736) 2023-07-18T03:11:16.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.048+0000 7fc10659b700 1 -- 172.21.15.101:0/248021960 learned_addr learned my addr 172.21.15.101:0/248021960 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:16.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.048+0000 7fc106d9c700 1 --2- 172.21.15.101:0/248021960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f36c679ad0 0x55f36c6dc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:16.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.048+0000 7fc105d9a700 1 --2- 172.21.15.101:0/248021960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f36c66ebe0 0x55f36c67ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:16.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.048+0000 7fc10659b700 1 -- 172.21.15.101:0/248021960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f36c679ad0 msgr2=0x55f36c6dc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:16.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.048+0000 7fc10659b700 1 --2- 172.21.15.101:0/248021960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f36c679ad0 0x55f36c6dc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:16.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.048+0000 7fc10659b700 1 -- 172.21.15.101:0/248021960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f36c66ebe0 msgr2=0x55f36c67ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:16.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.048+0000 7fc10659b700 1 --2- 172.21.15.101:0/248021960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f36c66ebe0 0x55f36c67ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:16.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.048+0000 7fc10659b700 1 -- 172.21.15.101:0/248021960 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f36c3eede0 con 0x55f36c66e2d0 2023-07-18T03:11:16.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.048+0000 7fc105d9a700 1 --2- 172.21.15.101:0/248021960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f36c66ebe0 0x55f36c67ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:11:16.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.048+0000 7fc10659b700 1 --2- 172.21.15.101:0/248021960 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f36c66e2d0 0x55f36c67e680 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc005d10 tx=0x7fc0fc00bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e696f46fe6d496f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:16.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.048+0000 7fc0e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:11:16.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.049+0000 7fc0e9ffb700 1 -- 172.21.15.101:0/248021960 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc01d030 con 0x55f36c66e2d0 2023-07-18T03:11:16.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.049+0000 7fc0e9ffb700 1 -- 172.21.15.101:0/248021960 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fc0fc004700 con 0x55f36c66e2d0 2023-07-18T03:11:16.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.049+0000 7fc11c073b80 1 -- 172.21.15.101:0/248021960 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc0fc01e040 con 0x55f36c66e2d0 2023-07-18T03:11:16.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.049+0000 7fc0e9ffb700 1 -- 172.21.15.101:0/248021960 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0fc0048c0 con 0x55f36c66e2d0 2023-07-18T03:11:16.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.049+0000 7fc11c073b80 10 client.16299 fetch_fsmap learned FSMap version 25 2023-07-18T03:11:16.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.049+0000 7fc11c073b80 1 -- 172.21.15.101:0/248021960 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f36c3ed2b0 con 0x55f36c66e2d0 2023-07-18T03:11:16.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.049+0000 7fc0e9ffb700 1 -- 172.21.15.101:0/248021960 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0fc01e040 con 0x55f36c66e2d0 2023-07-18T03:11:16.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.049+0000 7fc0e9ffb700 1 -- 172.21.15.101:0/248021960 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fc0fc042080 con 0x55f36c66e2d0 2023-07-18T03:11:16.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.049+0000 7fc11c073b80 10 client.16299 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:11:16.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.049+0000 7fc11c073b80 10 client.16299 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:11:16.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.049+0000 7fc11c073b80 20 client.16299 populate_metadata read hostname 'smithi101' 2023-07-18T03:11:16.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.049+0000 7fc11c073b80 1 --2- 172.21.15.101:0/248021960 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f36c6ddb20 0x55f36c6dff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:16.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.049+0000 7fc11c073b80 4 client.16299 mds_command: new command op to 24562 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:11:16.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.049+0000 7fc11c073b80 1 -- 172.21.15.101:0/248021960 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f36c57c050 con 0x55f36c6ddb20 2023-07-18T03:11:16.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.050+0000 7fc105d9a700 1 --2- 172.21.15.101:0/248021960 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f36c6ddb20 0x55f36c6dff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:16.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.050+0000 7fc105d9a700 1 --2- 172.21.15.101:0/248021960 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f36c6ddb20 0x55f36c6dff00 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=deb3ca3e607c356 server_cookie=c13d7c9352889232 in_seq=0 out_seq=0 2023-07-18T03:11:16.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.050+0000 7fc0e9ffb700 10 client.16299 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:11:16.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.050+0000 7fc0e9ffb700 1 -- 172.21.15.101:0/248021960 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f36c57c050 con 0x55f36c6ddb20 2023-07-18T03:11:16.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.050+0000 7fc0e9ffb700 10 client.16299 handle_command_reply: tid=0 2023-07-18T03:11:16.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.050+0000 7fc11c073b80 1 client.16299 shutdown 2023-07-18T03:11:16.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.050+0000 7fc11c073b80 1 -- 172.21.15.101:0/248021960 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f36c66e2d0 msgr2=0x55f36c67e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:16.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.050+0000 7fc11c073b80 1 --2- 172.21.15.101:0/248021960 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f36c66e2d0 0x55f36c67e680 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc005d10 tx=0x7fc0fc00bdc0 comp rx=0 tx=0).stop 2023-07-18T03:11:16.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.050+0000 7fc11c073b80 1 -- 172.21.15.101:0/248021960 shutdown_connections 2023-07-18T03:11:16.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.050+0000 7fc11c073b80 1 --2- 172.21.15.101:0/248021960 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f36c6ddb20 0x55f36c6dff00 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:16.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.050+0000 7fc11c073b80 1 --2- 172.21.15.101:0/248021960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f36c679ad0 0x55f36c6dc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:16.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.050+0000 7fc11c073b80 1 --2- 172.21.15.101:0/248021960 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f36c66e2d0 0x55f36c67e680 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:16.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.050+0000 7fc0e9ffb700 0 client.16299 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:11:16.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.050+0000 7fc11c073b80 1 --2- 172.21.15.101:0/248021960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f36c66ebe0 0x55f36c67ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:16.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.050+0000 7fc11c073b80 1 -- 172.21.15.101:0/248021960 >> 172.21.15.101:0/248021960 conn(0x55f36c664da0 msgr2=0x55f36c671a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:16.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.051+0000 7fc11c073b80 1 -- 172.21.15.101:0/248021960 shutdown_connections 2023-07-18T03:11:16.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.051+0000 7fc11c073b80 1 -- 172.21.15.101:0/248021960 wait complete. 2023-07-18T03:11:16.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.051+0000 7fc11c073b80 20 client.16299 trim_cache size 0 max 16384 2023-07-18T03:11:16.052 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:11:16.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.051+0000 7fc1087ff700 1 -- 172.21.15.101:0/3272845993 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc0ec0a12c0 msgr2=0x7fc0ec0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:16.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.051+0000 7fc1087ff700 1 --2- 172.21.15.101:0/3272845993 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc0ec0a12c0 0x7fc0ec0a3770 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc100199450 tx=0x7fc0f0009cc0 comp rx=0 tx=0).stop 2023-07-18T03:11:16.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.051+0000 7fc1087ff700 1 -- 172.21.15.101:0/3272845993 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc100105af0 msgr2=0x7fc10019c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:16.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.051+0000 7fc1087ff700 1 --2- 172.21.15.101:0/3272845993 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc100105af0 0x7fc10019c7e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc0f800ef30 tx=0x7fc0f800c3a0 comp rx=0 tx=0).stop 2023-07-18T03:11:16.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.051+0000 7fc1087ff700 1 -- 172.21.15.101:0/3272845993 shutdown_connections 2023-07-18T03:11:16.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.051+0000 7fc1087ff700 1 --2- 172.21.15.101:0/3272845993 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc100105af0 0x7fc10019c7e0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:16.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.051+0000 7fc1087ff700 1 --2- 172.21.15.101:0/3272845993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1000ffae0 0x7fc100197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:16.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.051+0000 7fc1087ff700 1 --2- 172.21.15.101:0/3272845993 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc0ec0a12c0 0x7fc0ec0a3770 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:16.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.051+0000 7fc1087ff700 1 --2- 172.21.15.101:0/3272845993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc100100490 0x7fc100198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:16.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.052+0000 7fc1087ff700 1 -- 172.21.15.101:0/3272845993 >> 172.21.15.101:0/3272845993 conn(0x7fc1000fb5e0 msgr2=0x7fc10010ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:16.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.052+0000 7fc1087ff700 1 -- 172.21.15.101:0/3272845993 shutdown_connections 2023-07-18T03:11:16.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:16.052+0000 7fc1087ff700 1 -- 172.21.15.101:0/3272845993 wait complete. 2023-07-18T03:11:16.236 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:16 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2371187621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:11:16.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:16 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/2371187621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:11:16.543 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:16 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/2371187621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:11:16.573 INFO:tasks.workunit.client.0.smithi101.stdout: 1 249242 8.92 MB/sec execute 41 sec latency 27.819 ms 2023-07-18T03:11:17.062 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-18T03:11:17.062 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-18T03:11:17.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.307+0000 7ff161dff700 1 -- 172.21.15.101:0/1324323539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff15c105ae0 msgr2=0x7ff15c105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:17.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.307+0000 7ff161dff700 1 --2- 172.21.15.101:0/1324323539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff15c105ae0 0x7ff15c105eb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff14c009990 tx=0x7ff14c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.307+0000 7ff161dff700 1 -- 172.21.15.101:0/1324323539 shutdown_connections 2023-07-18T03:11:17.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.307+0000 7ff161dff700 1 --2- 172.21.15.101:0/1324323539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff15c105ae0 0x7ff15c105eb0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.307+0000 7ff161dff700 1 --2- 172.21.15.101:0/1324323539 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff15c100480 0x7ff15c10c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.307+0000 7ff161dff700 1 --2- 172.21.15.101:0/1324323539 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff15c0ffad0 0x7ff15c0fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.307+0000 7ff161dff700 1 -- 172.21.15.101:0/1324323539 >> 172.21.15.101:0/1324323539 conn(0x7ff15c0fb5f0 msgr2=0x7ff15c0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:17.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.308+0000 7ff161dff700 1 -- 172.21.15.101:0/1324323539 shutdown_connections 2023-07-18T03:11:17.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.308+0000 7ff161dff700 1 -- 172.21.15.101:0/1324323539 wait complete. 2023-07-18T03:11:17.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.309+0000 7ff161dff700 1 Processor -- start 2023-07-18T03:11:17.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.309+0000 7ff161dff700 1 -- start start 2023-07-18T03:11:17.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.309+0000 7ff161dff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff15c0ffad0 0x7ff15c197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:17.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.309+0000 7ff161dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff15c100480 0x7ff15c198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:17.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.309+0000 7ff161dff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff15c105ae0 0x7ff15c19c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:17.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.309+0000 7ff161dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15c19cdf0 con 0x7ff15c100480 2023-07-18T03:11:17.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.309+0000 7ff161dff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15c19cf60 con 0x7ff15c0ffad0 2023-07-18T03:11:17.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.309+0000 7ff161dff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15c19d120 con 0x7ff15c105ae0 2023-07-18T03:11:17.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.309+0000 7ff15b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff15c0ffad0 0x7ff15c197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:17.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.309+0000 7ff15b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff15c0ffad0 0x7ff15c197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:56016/0 (socket says 172.21.15.101:56016) 2023-07-18T03:11:17.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.309+0000 7ff15b7fe700 1 -- 172.21.15.101:0/1237194867 learned_addr learned my addr 172.21.15.101:0/1237194867 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:17.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.309+0000 7ff15affd700 1 --2- 172.21.15.101:0/1237194867 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff15c100480 0x7ff15c198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:17.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.309+0000 7ff15bfff700 1 --2- 172.21.15.101:0/1237194867 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff15c105ae0 0x7ff15c19c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:17.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.310+0000 7ff15b7fe700 1 -- 172.21.15.101:0/1237194867 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff15c105ae0 msgr2=0x7ff15c19c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:17.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.310+0000 7ff15b7fe700 1 --2- 172.21.15.101:0/1237194867 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff15c105ae0 0x7ff15c19c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.310+0000 7ff15b7fe700 1 -- 172.21.15.101:0/1237194867 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff15c100480 msgr2=0x7ff15c198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:17.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.310+0000 7ff15b7fe700 1 --2- 172.21.15.101:0/1237194867 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff15c100480 0x7ff15c198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.310+0000 7ff15b7fe700 1 -- 172.21.15.101:0/1237194867 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff14c0095f0 con 0x7ff15c0ffad0 2023-07-18T03:11:17.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.310+0000 7ff15b7fe700 1 --2- 172.21.15.101:0/1237194867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff15c0ffad0 0x7ff15c197e10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff14c005910 tx=0x7ff14c005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:17.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.310+0000 7ff158ff9700 1 -- 172.21.15.101:0/1237194867 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14c01d070 con 0x7ff15c0ffad0 2023-07-18T03:11:17.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.310+0000 7ff158ff9700 1 -- 172.21.15.101:0/1237194867 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff14c022470 con 0x7ff15c0ffad0 2023-07-18T03:11:17.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.310+0000 7ff161dff700 1 -- 172.21.15.101:0/1237194867 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff15c19d3a0 con 0x7ff15c0ffad0 2023-07-18T03:11:17.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.310+0000 7ff158ff9700 1 -- 172.21.15.101:0/1237194867 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14c017650 con 0x7ff15c0ffad0 2023-07-18T03:11:17.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.311+0000 7ff161dff700 1 -- 172.21.15.101:0/1237194867 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff15c1a4c40 con 0x7ff15c0ffad0 2023-07-18T03:11:17.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.314+0000 7ff158ff9700 1 -- 172.21.15.101:0/1237194867 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7ff14c0225e0 con 0x7ff15c0ffad0 2023-07-18T03:11:17.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.315+0000 7ff175820b80 1 Processor -- start 2023-07-18T03:11:17.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.315+0000 7ff175820b80 1 -- start start 2023-07-18T03:11:17.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.315+0000 7ff175820b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6fc67c350 0x55b6fc67c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:17.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.315+0000 7ff175820b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6fc67cc60 0x55b6fc687610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:17.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.316+0000 7ff175820b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b6fc687b50 0x55b6fc689f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:17.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.316+0000 7ff175820b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6fc582d30 con 0x55b6fc687b50 2023-07-18T03:11:17.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.316+0000 7ff175820b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6fc5d5ac0 con 0x55b6fc67c350 2023-07-18T03:11:17.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.316+0000 7ff175820b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6fc5cef50 con 0x55b6fc67cc60 2023-07-18T03:11:17.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.316+0000 7ff158ff9700 1 --2- 172.21.15.101:0/1237194867 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff1440a1340 0x7ff1440a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:17.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.316+0000 7ff15b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6fc67cc60 0x55b6fc687610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:17.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.316+0000 7ff15affd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b6fc687b50 0x55b6fc689f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:17.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.316+0000 7ff15b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6fc67cc60 0x55b6fc687610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:52892/0 (socket says 172.21.15.101:52892) 2023-07-18T03:11:17.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.316+0000 7ff15affd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b6fc687b50 0x55b6fc689f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:54754/0 (socket says 172.21.15.101:54754) 2023-07-18T03:11:17.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.316+0000 7ff15b7fe700 1 -- 172.21.15.101:0/3420560164 learned_addr learned my addr 172.21.15.101:0/3420560164 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:17.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.316+0000 7ff15bfff700 1 --2- 172.21.15.101:0/3420560164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6fc67c350 0x55b6fc67c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:17.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.316+0000 7ff15bfff700 1 --2- 172.21.15.101:0/1237194867 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff1440a1340 0x7ff1440a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:17.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.316+0000 7ff158ff9700 1 -- 172.21.15.101:0/1237194867 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7ff14c0ce320 con 0x7ff15c0ffad0 2023-07-18T03:11:17.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.316+0000 7ff15b7fe700 1 -- 172.21.15.101:0/3420560164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6fc67c350 msgr2=0x55b6fc67c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:17.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.316+0000 7ff15b7fe700 1 --2- 172.21.15.101:0/3420560164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6fc67c350 0x55b6fc67c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.316+0000 7ff15b7fe700 1 -- 172.21.15.101:0/3420560164 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b6fc687b50 msgr2=0x55b6fc689f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:17.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.316+0000 7ff15b7fe700 1 --2- 172.21.15.101:0/3420560164 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b6fc687b50 0x55b6fc689f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.316+0000 7ff15b7fe700 1 -- 172.21.15.101:0/3420560164 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6fc3fce60 con 0x55b6fc67cc60 2023-07-18T03:11:17.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.317+0000 7ff15b7fe700 1 --2- 172.21.15.101:0/3420560164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6fc67cc60 0x55b6fc687610 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff14c0cb600 tx=0x7ff14c009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f3d2216f01f59ff server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:17.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.317+0000 7ff15bfff700 1 --2- 172.21.15.101:0/1237194867 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff1440a1340 0x7ff1440a37f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff154009ec0 tx=0x7ff15400b040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:17.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.317+0000 7ff141ffb700 1 -- 172.21.15.101:0/3420560164 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14c09aa30 con 0x55b6fc67cc60 2023-07-18T03:11:17.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.317+0000 7ff141ffb700 1 -- 172.21.15.101:0/3420560164 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff14c09b0d0 con 0x55b6fc67cc60 2023-07-18T03:11:17.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.317+0000 7ff141ffb700 1 -- 172.21.15.101:0/3420560164 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14c09b590 con 0x55b6fc67cc60 2023-07-18T03:11:17.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.317+0000 7ff175820b80 1 -- 172.21.15.101:0/3420560164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6fc67cc60 msgr2=0x55b6fc687610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:17.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.317+0000 7ff175820b80 1 --2- 172.21.15.101:0/3420560164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6fc67cc60 0x55b6fc687610 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff14c0cb600 tx=0x7ff14c009310 comp rx=0 tx=0).stop 2023-07-18T03:11:17.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.318+0000 7ff175820b80 1 -- 172.21.15.101:0/3420560164 shutdown_connections 2023-07-18T03:11:17.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.318+0000 7ff175820b80 1 --2- 172.21.15.101:0/3420560164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6fc67c350 0x55b6fc67c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.318+0000 7ff175820b80 1 --2- 172.21.15.101:0/3420560164 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b6fc687b50 0x55b6fc689f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.318+0000 7ff175820b80 1 --2- 172.21.15.101:0/3420560164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6fc67cc60 0x55b6fc687610 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.318+0000 7ff175820b80 1 -- 172.21.15.101:0/3420560164 >> 172.21.15.101:0/3420560164 conn(0x55b6fc672e20 msgr2=0x55b6fc678720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:17.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.318+0000 7ff175820b80 1 -- 172.21.15.101:0/3420560164 shutdown_connections 2023-07-18T03:11:17.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.318+0000 7ff175820b80 1 -- 172.21.15.101:0/3420560164 wait complete. 2023-07-18T03:11:17.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.318+0000 7ff175820b80 1 -- start start 2023-07-18T03:11:17.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.318+0000 7ff175820b80 1 Processor -- start 2023-07-18T03:11:17.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.319+0000 7ff175820b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6fc67c350 0x55b6fc687f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:17.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.319+0000 7ff175820b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6fc67cc60 0x55b6fc688460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:17.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.319+0000 7ff175820b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b6fc687b50 0x55b6fc6889a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:17.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.319+0000 7ff175820b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6fc56eb40 con 0x55b6fc687b50 2023-07-18T03:11:17.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.319+0000 7ff175820b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6fc56e840 con 0x55b6fc67cc60 2023-07-18T03:11:17.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.319+0000 7ff175820b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6fc4529a0 con 0x55b6fc67c350 2023-07-18T03:11:17.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.319+0000 7ff175820b80 10 client.0 _ll_register_callbacks cb 0x55b6fc56c7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:11:17.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.319+0000 7ff15affd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6fc67cc60 0x55b6fc688460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:17.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.319+0000 7ff15b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6fc67c350 0x55b6fc687f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:17.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.319+0000 7ff15affd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6fc67cc60 0x55b6fc688460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:56030/0 (socket says 172.21.15.101:56030) 2023-07-18T03:11:17.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.319+0000 7ff15b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6fc67c350 0x55b6fc687f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:52904/0 (socket says 172.21.15.101:52904) 2023-07-18T03:11:17.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.319+0000 7ff15affd700 1 -- 172.21.15.101:0/3162461436 learned_addr learned my addr 172.21.15.101:0/3162461436 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:17.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.319+0000 7ff15bfff700 1 --2- 172.21.15.101:0/3162461436 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b6fc687b50 0x55b6fc6889a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:17.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.319+0000 7ff15affd700 1 -- 172.21.15.101:0/3162461436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6fc67c350 msgr2=0x55b6fc687f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:17.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.319+0000 7ff15affd700 1 --2- 172.21.15.101:0/3162461436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6fc67c350 0x55b6fc687f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.319+0000 7ff15affd700 1 -- 172.21.15.101:0/3162461436 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b6fc687b50 msgr2=0x55b6fc6889a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:17.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.319+0000 7ff15affd700 1 --2- 172.21.15.101:0/3162461436 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b6fc687b50 0x55b6fc6889a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.320+0000 7ff15affd700 1 -- 172.21.15.101:0/3162461436 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff15c1a4c40 con 0x55b6fc67cc60 2023-07-18T03:11:17.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.320+0000 7ff15affd700 1 --2- 172.21.15.101:0/3162461436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6fc67cc60 0x55b6fc688460 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff150011730 tx=0x7ff150011a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd2152d6b909ccef server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:17.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.320+0000 7ff13affd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:11:17.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.320+0000 7ff13affd700 1 -- 172.21.15.101:0/3162461436 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff150016070 con 0x55b6fc67cc60 2023-07-18T03:11:17.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.320+0000 7ff175820b80 1 -- 172.21.15.101:0/3162461436 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b6fc528570 con 0x55b6fc67cc60 2023-07-18T03:11:17.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.320+0000 7ff13affd700 1 -- 172.21.15.101:0/3162461436 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7ff15000f690 con 0x55b6fc67cc60 2023-07-18T03:11:17.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.320+0000 7ff13affd700 1 -- 172.21.15.101:0/3162461436 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff15000f850 con 0x55b6fc67cc60 2023-07-18T03:11:17.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.320+0000 7ff175820b80 10 client.26357 fetch_fsmap learned FSMap version 25 2023-07-18T03:11:17.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.320+0000 7ff175820b80 1 -- 172.21.15.101:0/3162461436 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b6fc3e5ce0 con 0x55b6fc67cc60 2023-07-18T03:11:17.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.321+0000 7ff13affd700 1 -- 172.21.15.101:0/3162461436 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b6fc528570 con 0x55b6fc67cc60 2023-07-18T03:11:17.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.321+0000 7ff13affd700 1 -- 172.21.15.101:0/3162461436 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff15001aaa0 con 0x55b6fc67cc60 2023-07-18T03:11:17.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.321+0000 7ff175820b80 10 client.26357 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:11:17.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.321+0000 7ff175820b80 10 client.26357 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:11:17.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.321+0000 7ff175820b80 20 client.26357 populate_metadata read hostname 'smithi101' 2023-07-18T03:11:17.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.321+0000 7ff175820b80 1 --2- 172.21.15.101:0/3162461436 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b6fc6971e0 0x55b6fc6995f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:17.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.321+0000 7ff175820b80 4 client.26357 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:11:17.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.321+0000 7ff175820b80 1 -- 172.21.15.101:0/3162461436 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b6fc5c7f30 con 0x55b6fc6971e0 2023-07-18T03:11:17.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.321+0000 7ff15b7fe700 1 --2- 172.21.15.101:0/3162461436 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b6fc6971e0 0x55b6fc6995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:17.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.322+0000 7ff13affd700 10 client.26357 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:11:17.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.322+0000 7ff15b7fe700 1 --2- 172.21.15.101:0/3162461436 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b6fc6971e0 0x55b6fc6995f0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44bb076e12debce5 server_cookie=7f802521ee0cecdb in_seq=0 out_seq=0 2023-07-18T03:11:17.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.323+0000 7ff13affd700 1 -- 172.21.15.101:0/3162461436 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b6fc5c7f30 con 0x55b6fc6971e0 2023-07-18T03:11:17.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.323+0000 7ff13affd700 10 client.26357 handle_command_reply: tid=0 2023-07-18T03:11:17.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.323+0000 7ff175820b80 1 client.26357 shutdown 2023-07-18T03:11:17.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.323+0000 7ff175820b80 1 -- 172.21.15.101:0/3162461436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6fc67cc60 msgr2=0x55b6fc688460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:17.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.323+0000 7ff175820b80 1 --2- 172.21.15.101:0/3162461436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6fc67cc60 0x55b6fc688460 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff150011730 tx=0x7ff150011a40 comp rx=0 tx=0).stop 2023-07-18T03:11:17.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.323+0000 7ff175820b80 1 -- 172.21.15.101:0/3162461436 shutdown_connections 2023-07-18T03:11:17.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.323+0000 7ff175820b80 1 --2- 172.21.15.101:0/3162461436 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b6fc6971e0 0x55b6fc6995f0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.323+0000 7ff175820b80 1 --2- 172.21.15.101:0/3162461436 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b6fc687b50 0x55b6fc6889a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.323+0000 7ff175820b80 1 --2- 172.21.15.101:0/3162461436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6fc67c350 0x55b6fc687f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.323+0000 7ff175820b80 1 --2- 172.21.15.101:0/3162461436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6fc67cc60 0x55b6fc688460 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.323+0000 7ff13affd700 0 client.26357 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:11:17.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.323+0000 7ff175820b80 1 -- 172.21.15.101:0/3162461436 >> 172.21.15.101:0/3162461436 conn(0x55b6fc672e20 msgr2=0x55b6fc678310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:17.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.323+0000 7ff175820b80 1 -- 172.21.15.101:0/3162461436 shutdown_connections 2023-07-18T03:11:17.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.323+0000 7ff175820b80 1 -- 172.21.15.101:0/3162461436 wait complete. 2023-07-18T03:11:17.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.323+0000 7ff175820b80 20 client.26357 trim_cache size 0 max 16384 2023-07-18T03:11:17.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.341+0000 7ff175820b80 1 Processor -- start 2023-07-18T03:11:17.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.341+0000 7ff175820b80 1 -- start start 2023-07-18T03:11:17.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.341+0000 7ff175820b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6fc67c350 0x55b6fc68bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:17.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.341+0000 7ff175820b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b6fc67cc60 0x55b6fc67eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:17.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.342+0000 7ff175820b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6fc687b50 0x55b6fc683da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:17.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.342+0000 7ff175820b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6fc67e220 con 0x55b6fc67cc60 2023-07-18T03:11:17.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.342+0000 7ff175820b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6fc6871b0 con 0x55b6fc67c350 2023-07-18T03:11:17.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.342+0000 7ff175820b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6fc693ae0 con 0x55b6fc687b50 2023-07-18T03:11:17.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.342+0000 7ff15affd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b6fc67cc60 0x55b6fc67eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:17.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.342+0000 7ff15bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6fc687b50 0x55b6fc683da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:17.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.342+0000 7ff15b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6fc67c350 0x55b6fc68bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:17.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.342+0000 7ff15affd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b6fc67cc60 0x55b6fc67eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:54778/0 (socket says 172.21.15.101:54778) 2023-07-18T03:11:17.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.342+0000 7ff15affd700 1 -- 172.21.15.101:0/2171817840 learned_addr learned my addr 172.21.15.101:0/2171817840 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:17.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.342+0000 7ff15affd700 1 -- 172.21.15.101:0/2171817840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6fc687b50 msgr2=0x55b6fc683da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:17.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.342+0000 7ff15affd700 1 --2- 172.21.15.101:0/2171817840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6fc687b50 0x55b6fc683da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.342+0000 7ff15affd700 1 -- 172.21.15.101:0/2171817840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6fc67c350 msgr2=0x55b6fc68bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:17.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.342+0000 7ff15affd700 1 --2- 172.21.15.101:0/2171817840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6fc67c350 0x55b6fc68bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.342+0000 7ff15affd700 1 -- 172.21.15.101:0/2171817840 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6fc4232d0 con 0x55b6fc67cc60 2023-07-18T03:11:17.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.342+0000 7ff15affd700 1 --2- 172.21.15.101:0/2171817840 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b6fc67cc60 0x55b6fc67eb70 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff150001eb0 tx=0x7ff150009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69305999a5d52703 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:17.339 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.343+0000 7ff1427fc700 1 -- 172.21.15.101:0/2171817840 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15002f070 con 0x55b6fc67cc60 2023-07-18T03:11:17.339 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.343+0000 7ff1427fc700 1 -- 172.21.15.101:0/2171817840 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff150019460 con 0x55b6fc67cc60 2023-07-18T03:11:17.339 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.343+0000 7ff1427fc700 1 -- 172.21.15.101:0/2171817840 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff150016050 con 0x55b6fc67cc60 2023-07-18T03:11:17.339 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.343+0000 7ff175820b80 1 -- 172.21.15.101:0/2171817840 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b6fc67cc60 msgr2=0x55b6fc67eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:17.339 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.343+0000 7ff175820b80 1 --2- 172.21.15.101:0/2171817840 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b6fc67cc60 0x55b6fc67eb70 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff150001eb0 tx=0x7ff150009480 comp rx=0 tx=0).stop 2023-07-18T03:11:17.339 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.344+0000 7ff175820b80 1 -- 172.21.15.101:0/2171817840 shutdown_connections 2023-07-18T03:11:17.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.344+0000 7ff175820b80 1 --2- 172.21.15.101:0/2171817840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6fc67c350 0x55b6fc68bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.344+0000 7ff175820b80 1 --2- 172.21.15.101:0/2171817840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6fc687b50 0x55b6fc683da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.344+0000 7ff175820b80 1 --2- 172.21.15.101:0/2171817840 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b6fc67cc60 0x55b6fc67eb70 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.344+0000 7ff175820b80 1 -- 172.21.15.101:0/2171817840 >> 172.21.15.101:0/2171817840 conn(0x55b6fc672e20 msgr2=0x55b6fc67e460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:17.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.344+0000 7ff175820b80 1 -- 172.21.15.101:0/2171817840 shutdown_connections 2023-07-18T03:11:17.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.344+0000 7ff175820b80 1 -- 172.21.15.101:0/2171817840 wait complete. 2023-07-18T03:11:17.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.344+0000 7ff175820b80 1 -- start start 2023-07-18T03:11:17.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.344+0000 7ff175820b80 1 Processor -- start 2023-07-18T03:11:17.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.345+0000 7ff175820b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6fc67c350 0x55b6fc6e5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:17.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.345+0000 7ff175820b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b6fc67cc60 0x55b6fc6e5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:17.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.345+0000 7ff175820b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6fc687b50 0x55b6fc6dfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:17.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.345+0000 7ff175820b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6fc6e0010 con 0x55b6fc67cc60 2023-07-18T03:11:17.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.345+0000 7ff175820b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6fc6e0180 con 0x55b6fc67c350 2023-07-18T03:11:17.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.345+0000 7ff175820b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6fc6e0340 con 0x55b6fc687b50 2023-07-18T03:11:17.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.345+0000 7ff175820b80 10 client.0 _ll_register_callbacks cb 0x55b6fc56c7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:11:17.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.345+0000 7ff15b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6fc67c350 0x55b6fc6e5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:17.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.345+0000 7ff15b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6fc67c350 0x55b6fc6e5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:56062/0 (socket says 172.21.15.101:56062) 2023-07-18T03:11:17.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.345+0000 7ff15b7fe700 1 -- 172.21.15.101:0/716582580 learned_addr learned my addr 172.21.15.101:0/716582580 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:17.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.345+0000 7ff15bfff700 1 --2- 172.21.15.101:0/716582580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6fc687b50 0x55b6fc6dfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:17.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.345+0000 7ff15affd700 1 --2- 172.21.15.101:0/716582580 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b6fc67cc60 0x55b6fc6e5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:17.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.345+0000 7ff15b7fe700 1 -- 172.21.15.101:0/716582580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6fc687b50 msgr2=0x55b6fc6dfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:17.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.345+0000 7ff15b7fe700 1 --2- 172.21.15.101:0/716582580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6fc687b50 0x55b6fc6dfa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.345+0000 7ff15b7fe700 1 -- 172.21.15.101:0/716582580 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b6fc67cc60 msgr2=0x55b6fc6e5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:17.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.345+0000 7ff15b7fe700 1 --2- 172.21.15.101:0/716582580 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b6fc67cc60 0x55b6fc6e5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.345+0000 7ff15b7fe700 1 -- 172.21.15.101:0/716582580 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b6fc3e5ce0 con 0x55b6fc67c350 2023-07-18T03:11:17.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.345+0000 7ff15bfff700 1 --2- 172.21.15.101:0/716582580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6fc687b50 0x55b6fc6dfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:11:17.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.346+0000 7ff15b7fe700 1 --2- 172.21.15.101:0/716582580 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6fc67c350 0x55b6fc6e5690 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff14c000c00 tx=0x7ff14c01a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c972a56765cb258b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:17.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.346+0000 7ff13affd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:11:17.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.346+0000 7ff13affd700 1 -- 172.21.15.101:0/716582580 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14c0a3490 con 0x55b6fc67c350 2023-07-18T03:11:17.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.346+0000 7ff175820b80 1 -- 172.21.15.101:0/716582580 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff14c09b0d0 con 0x55b6fc67c350 2023-07-18T03:11:17.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.346+0000 7ff13affd700 1 -- 172.21.15.101:0/716582580 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7ff14c09f100 con 0x55b6fc67c350 2023-07-18T03:11:17.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.346+0000 7ff13affd700 1 -- 172.21.15.101:0/716582580 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff14c09a5c0 con 0x55b6fc67c350 2023-07-18T03:11:17.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.346+0000 7ff175820b80 10 client.26363 fetch_fsmap learned FSMap version 25 2023-07-18T03:11:17.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.346+0000 7ff175820b80 1 -- 172.21.15.101:0/716582580 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b6fc3fb330 con 0x55b6fc67c350 2023-07-18T03:11:17.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.346+0000 7ff13affd700 1 -- 172.21.15.101:0/716582580 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff14c09b0d0 con 0x55b6fc67c350 2023-07-18T03:11:17.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.346+0000 7ff13affd700 1 -- 172.21.15.101:0/716582580 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff14c09b410 con 0x55b6fc67c350 2023-07-18T03:11:17.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.346+0000 7ff175820b80 10 client.26363 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:11:17.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.346+0000 7ff175820b80 10 client.26363 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:11:17.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.346+0000 7ff175820b80 20 client.26363 populate_metadata read hostname 'smithi101' 2023-07-18T03:11:17.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.346+0000 7ff175820b80 1 --2- 172.21.15.101:0/716582580 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b6fc6ed1f0 0x55b6fc6ef600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:17.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.346+0000 7ff175820b80 4 client.26363 mds_command: new command op to 24562 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-18T03:11:17.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.346+0000 7ff175820b80 1 -- 172.21.15.101:0/716582580 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55b6fc58a0d0 con 0x55b6fc6ed1f0 2023-07-18T03:11:17.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.347+0000 7ff15affd700 1 --2- 172.21.15.101:0/716582580 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b6fc6ed1f0 0x55b6fc6ef600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:17.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.347+0000 7ff15affd700 1 --2- 172.21.15.101:0/716582580 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b6fc6ed1f0 0x55b6fc6ef600 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c42ff996c29982bb server_cookie=eb8a0d96badfb4be in_seq=0 out_seq=0 2023-07-18T03:11:17.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.347+0000 7ff13affd700 10 client.26363 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:11:17.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.348+0000 7ff13affd700 1 -- 172.21.15.101:0/716582580 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b6fc58a0d0 con 0x55b6fc6ed1f0 2023-07-18T03:11:17.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.348+0000 7ff13affd700 10 client.26363 handle_command_reply: tid=0 2023-07-18T03:11:17.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.348+0000 7ff175820b80 1 client.26363 shutdown 2023-07-18T03:11:17.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.348+0000 7ff175820b80 1 -- 172.21.15.101:0/716582580 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6fc67c350 msgr2=0x55b6fc6e5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:17.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.348+0000 7ff175820b80 1 --2- 172.21.15.101:0/716582580 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6fc67c350 0x55b6fc6e5690 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff14c000c00 tx=0x7ff14c01a040 comp rx=0 tx=0).stop 2023-07-18T03:11:17.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.348+0000 7ff175820b80 1 -- 172.21.15.101:0/716582580 shutdown_connections 2023-07-18T03:11:17.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.348+0000 7ff175820b80 1 --2- 172.21.15.101:0/716582580 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b6fc6ed1f0 0x55b6fc6ef600 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.348+0000 7ff175820b80 1 --2- 172.21.15.101:0/716582580 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6fc67c350 0x55b6fc6e5690 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.348 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.348+0000 7ff175820b80 1 --2- 172.21.15.101:0/716582580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6fc687b50 0x55b6fc6dfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.348 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.348+0000 7ff175820b80 1 --2- 172.21.15.101:0/716582580 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b6fc67cc60 0x55b6fc6e5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.348 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.348+0000 7ff175820b80 1 -- 172.21.15.101:0/716582580 >> 172.21.15.101:0/716582580 conn(0x55b6fc672e20 msgr2=0x55b6fc67ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:17.348 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.348+0000 7ff13affd700 0 client.26363 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:11:17.348 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.348+0000 7ff175820b80 1 -- 172.21.15.101:0/716582580 shutdown_connections 2023-07-18T03:11:17.348 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.348+0000 7ff175820b80 1 -- 172.21.15.101:0/716582580 wait complete. 2023-07-18T03:11:17.348 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.349+0000 7ff175820b80 20 client.26363 trim_cache size 0 max 16384 2023-07-18T03:11:17.349 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:11:17.349 INFO:teuthology.orchestra.run.smithi101.stdout: "return_code": 0, 2023-07-18T03:11:17.349 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_tag": "596ffb58-c4df-4721-b069-b45be65bb2cc", 2023-07-18T03:11:17.349 INFO:teuthology.orchestra.run.smithi101.stdout: "mode": "asynchronous" 2023-07-18T03:11:17.349 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:11:17.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.349+0000 7ff161dff700 1 -- 172.21.15.101:0/1237194867 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff1440a1340 msgr2=0x7ff1440a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:17.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.349+0000 7ff161dff700 1 --2- 172.21.15.101:0/1237194867 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff1440a1340 0x7ff1440a37f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff154009ec0 tx=0x7ff15400b040 comp rx=0 tx=0).stop 2023-07-18T03:11:17.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.349+0000 7ff161dff700 1 -- 172.21.15.101:0/1237194867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff15c0ffad0 msgr2=0x7ff15c197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:17.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.349+0000 7ff161dff700 1 --2- 172.21.15.101:0/1237194867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff15c0ffad0 0x7ff15c197e10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff14c005910 tx=0x7ff14c005720 comp rx=0 tx=0).stop 2023-07-18T03:11:17.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.349+0000 7ff161dff700 1 -- 172.21.15.101:0/1237194867 shutdown_connections 2023-07-18T03:11:17.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.349+0000 7ff161dff700 1 --2- 172.21.15.101:0/1237194867 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff1440a1340 0x7ff1440a37f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.349+0000 7ff161dff700 1 --2- 172.21.15.101:0/1237194867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff15c0ffad0 0x7ff15c197e10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.349+0000 7ff161dff700 1 --2- 172.21.15.101:0/1237194867 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff15c105ae0 0x7ff15c19c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.349+0000 7ff161dff700 1 --2- 172.21.15.101:0/1237194867 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff15c100480 0x7ff15c198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.349+0000 7ff161dff700 1 -- 172.21.15.101:0/1237194867 >> 172.21.15.101:0/1237194867 conn(0x7ff15c0fb5f0 msgr2=0x7ff15c0fbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:17.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.349+0000 7ff161dff700 1 -- 172.21.15.101:0/1237194867 shutdown_connections 2023-07-18T03:11:17.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.349+0000 7ff161dff700 1 -- 172.21.15.101:0/1237194867 wait complete. 2023-07-18T03:11:17.357 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:17 smithi101 ceph-mon[126379]: pgmap v114: 129 pgs: 129 active+clean; 150 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 343 op/s 2023-07-18T03:11:17.359 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:11:17.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:17 smithi179 ceph-mon[129229]: pgmap v114: 129 pgs: 129 active+clean; 150 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 343 op/s 2023-07-18T03:11:17.543 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:17 smithi155 ceph-mon[128885]: pgmap v114: 129 pgs: 129 active+clean; 150 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 343 op/s 2023-07-18T03:11:17.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.581+0000 7f90e45ff700 1 -- 172.21.15.101:0/185146031 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f90dc101fa0 msgr2=0x7f90dc102370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:17.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.581+0000 7f90e45ff700 1 --2- 172.21.15.101:0/185146031 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f90dc101fa0 0x7f90dc102370 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f90d800b0d0 tx=0x7f90d800b3e0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.581+0000 7f90e45ff700 1 -- 172.21.15.101:0/185146031 shutdown_connections 2023-07-18T03:11:17.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.581+0000 7f90e45ff700 1 --2- 172.21.15.101:0/185146031 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90dc106fe0 0x7f90dc10ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.581+0000 7f90e45ff700 1 --2- 172.21.15.101:0/185146031 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f90dc101fa0 0x7f90dc102370 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.581+0000 7f90e45ff700 1 --2- 172.21.15.101:0/185146031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90dc102940 0x7f90dc106900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.581+0000 7f90e45ff700 1 -- 172.21.15.101:0/185146031 >> 172.21.15.101:0/185146031 conn(0x7f90dc0fd7d0 msgr2=0x7f90dc0ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:17.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.581+0000 7f90e45ff700 1 -- 172.21.15.101:0/185146031 shutdown_connections 2023-07-18T03:11:17.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.581+0000 7f90e45ff700 1 -- 172.21.15.101:0/185146031 wait complete. 2023-07-18T03:11:17.576 INFO:tasks.workunit.client.0.smithi101.stdout: 1 251535 9.01 MB/sec execute 42 sec latency 29.008 ms 2023-07-18T03:11:17.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.582+0000 7f90e45ff700 1 Processor -- start 2023-07-18T03:11:17.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.582+0000 7f90e45ff700 1 -- start start 2023-07-18T03:11:17.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.582+0000 7f90e45ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f90dc101fa0 0x7f90dc197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:17.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.582+0000 7f90e45ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90dc102940 0x7f90dc198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:17.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.583+0000 7f90e45ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90dc106fe0 0x7f90dc19c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:17.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.583+0000 7f90e45ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90dc19cfd0 con 0x7f90dc101fa0 2023-07-18T03:11:17.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.583+0000 7f90e45ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90dc19d140 con 0x7f90dc102940 2023-07-18T03:11:17.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.583+0000 7f90e45ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90dc19d300 con 0x7f90dc106fe0 2023-07-18T03:11:17.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.583+0000 7f90e239b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f90dc101fa0 0x7f90dc197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:17.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.583+0000 7f90e1b9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90dc102940 0x7f90dc198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:17.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.583+0000 7f90e239b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f90dc101fa0 0x7f90dc197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:54808/0 (socket says 172.21.15.101:54808) 2023-07-18T03:11:17.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.583+0000 7f90e239b700 1 -- 172.21.15.101:0/252650067 learned_addr learned my addr 172.21.15.101:0/252650067 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:17.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.583+0000 7f90e2b9c700 1 --2- 172.21.15.101:0/252650067 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90dc106fe0 0x7f90dc19c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:17.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.583+0000 7f90e239b700 1 -- 172.21.15.101:0/252650067 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90dc106fe0 msgr2=0x7f90dc19c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:17.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.583+0000 7f90e239b700 1 --2- 172.21.15.101:0/252650067 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90dc106fe0 0x7f90dc19c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.583+0000 7f90e239b700 1 -- 172.21.15.101:0/252650067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90dc102940 msgr2=0x7f90dc198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:17.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.583+0000 7f90e239b700 1 --2- 172.21.15.101:0/252650067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90dc102940 0x7f90dc198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.583+0000 7f90e239b700 1 -- 172.21.15.101:0/252650067 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90cc009520 con 0x7f90dc101fa0 2023-07-18T03:11:17.581 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.583+0000 7f90e239b700 1 --2- 172.21.15.101:0/252650067 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f90dc101fa0 0x7f90dc197ff0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f90d8000f80 tx=0x7f90d8007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:17.581 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.584+0000 7f90d37fe700 1 -- 172.21.15.101:0/252650067 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d800e040 con 0x7f90dc101fa0 2023-07-18T03:11:17.581 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.584+0000 7f90e45ff700 1 -- 172.21.15.101:0/252650067 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90d8009d00 con 0x7f90dc101fa0 2023-07-18T03:11:17.581 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.584+0000 7f90d37fe700 1 -- 172.21.15.101:0/252650067 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90d8005030 con 0x7f90dc101fa0 2023-07-18T03:11:17.581 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.584+0000 7f90d37fe700 1 -- 172.21.15.101:0/252650067 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d801d6d0 con 0x7f90dc101fa0 2023-07-18T03:11:17.581 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.584+0000 7f90e45ff700 1 -- 172.21.15.101:0/252650067 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90dc1a4c40 con 0x7f90dc101fa0 2023-07-18T03:11:17.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.587+0000 7f90d37fe700 1 -- 172.21.15.101:0/252650067 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f90d80054a0 con 0x7f90dc101fa0 2023-07-18T03:11:17.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.588+0000 7f90f7f4eb80 1 Processor -- start 2023-07-18T03:11:17.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.588+0000 7f90f7f4eb80 1 -- start start 2023-07-18T03:11:17.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.589+0000 7f90f7f4eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dddec9b2d0 0x55dddec9b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:17.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.589+0000 7f90f7f4eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dddec9bbe0 0x55dddeca6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:17.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.589+0000 7f90d37fe700 1 --2- 172.21.15.101:0/252650067 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f90c80a12c0 0x7f90c80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:17.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.589+0000 7f90f7f4eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dddeca6ad0 0x55dddeca8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:17.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.589+0000 7f90f7f4eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dddeba1cb0 con 0x55dddec9b2d0 2023-07-18T03:11:17.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.589+0000 7f90f7f4eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dddebf4a40 con 0x55dddec9bbe0 2023-07-18T03:11:17.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.589+0000 7f90f7f4eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dddebeded0 con 0x55dddeca6ad0 2023-07-18T03:11:17.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.589+0000 7f90d37fe700 1 -- 172.21.15.101:0/252650067 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f90d80cdfb0 con 0x7f90dc101fa0 2023-07-18T03:11:17.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.589+0000 7f90e2b9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dddec9b2d0 0x55dddec9b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:17.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.589+0000 7f90e2b9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dddec9b2d0 0x55dddec9b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:54820/0 (socket says 172.21.15.101:54820) 2023-07-18T03:11:17.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.589+0000 7f90e2b9c700 1 -- 172.21.15.101:0/459452073 learned_addr learned my addr 172.21.15.101:0/459452073 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:17.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.589+0000 7f90e239b700 1 --2- 172.21.15.101:0/459452073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dddec9bbe0 0x55dddeca6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:17.589 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.589+0000 7f90e1b9a700 1 --2- 172.21.15.101:0/252650067 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f90c80a12c0 0x7f90c80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:17.589 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.589+0000 7f90e2b9c700 1 --2- 172.21.15.101:0/459452073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dddeca6ad0 0x55dddeca8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:17.589 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.589+0000 7f90e239b700 1 -- 172.21.15.101:0/459452073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dddeca6ad0 msgr2=0x55dddeca8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:17.589 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.589+0000 7f90e239b700 1 --2- 172.21.15.101:0/459452073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dddeca6ad0 0x55dddeca8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.589 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.589+0000 7f90e239b700 1 -- 172.21.15.101:0/459452073 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dddec9b2d0 msgr2=0x55dddec9b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:17.589 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.589+0000 7f90e239b700 1 --2- 172.21.15.101:0/459452073 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dddec9b2d0 0x55dddec9b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.589+0000 7f90e239b700 1 -- 172.21.15.101:0/459452073 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dddea1bde0 con 0x55dddec9bbe0 2023-07-18T03:11:17.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.589+0000 7f90e2b9c700 1 --2- 172.21.15.101:0/459452073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dddeca6ad0 0x55dddeca8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:11:17.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.590+0000 7f90e239b700 1 --2- 172.21.15.101:0/459452073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dddec9bbe0 0x55dddeca6590 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f90d80cb350 tx=0x7f90d809ab30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c76ad26dce923931 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:17.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.590+0000 7f90e1b9a700 1 --2- 172.21.15.101:0/252650067 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f90c80a12c0 0x7f90c80a3770 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f90dc199600 tx=0x7f90cc009260 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:17.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.590+0000 7f90d0bfe700 1 -- 172.21.15.101:0/459452073 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d8064810 con 0x55dddec9bbe0 2023-07-18T03:11:17.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.590+0000 7f90d0bfe700 1 -- 172.21.15.101:0/459452073 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90d8063ab0 con 0x55dddec9bbe0 2023-07-18T03:11:17.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.590+0000 7f90d0bfe700 1 -- 172.21.15.101:0/459452073 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d80a3120 con 0x55dddec9bbe0 2023-07-18T03:11:17.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.591+0000 7f90f7f4eb80 1 -- 172.21.15.101:0/459452073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dddec9bbe0 msgr2=0x55dddeca6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:17.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.591+0000 7f90f7f4eb80 1 --2- 172.21.15.101:0/459452073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dddec9bbe0 0x55dddeca6590 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f90d80cb350 tx=0x7f90d809ab30 comp rx=0 tx=0).stop 2023-07-18T03:11:17.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.591+0000 7f90f7f4eb80 1 -- 172.21.15.101:0/459452073 shutdown_connections 2023-07-18T03:11:17.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.591+0000 7f90f7f4eb80 1 --2- 172.21.15.101:0/459452073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dddeca6ad0 0x55dddeca8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.591+0000 7f90f7f4eb80 1 --2- 172.21.15.101:0/459452073 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dddec9b2d0 0x55dddec9b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.591+0000 7f90f7f4eb80 1 --2- 172.21.15.101:0/459452073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dddec9bbe0 0x55dddeca6590 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.591+0000 7f90f7f4eb80 1 -- 172.21.15.101:0/459452073 >> 172.21.15.101:0/459452073 conn(0x55dddec91da0 msgr2=0x55dddec976a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:17.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.591+0000 7f90f7f4eb80 1 -- 172.21.15.101:0/459452073 shutdown_connections 2023-07-18T03:11:17.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.591+0000 7f90f7f4eb80 1 -- 172.21.15.101:0/459452073 wait complete. 2023-07-18T03:11:17.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.591+0000 7f90f7f4eb80 1 -- start start 2023-07-18T03:11:17.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.592+0000 7f90f7f4eb80 1 Processor -- start 2023-07-18T03:11:17.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.592+0000 7f90f7f4eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dddec9b2d0 0x55dddeca6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:17.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.592+0000 7f90f7f4eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dddec9bbe0 0x55dddeca73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:17.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.592+0000 7f90f7f4eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dddeca6ad0 0x55dddecb1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:17.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.592+0000 7f90f7f4eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dddeb8dac0 con 0x55dddeca6ad0 2023-07-18T03:11:17.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.592+0000 7f90f7f4eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dddeb8d7c0 con 0x55dddec9b2d0 2023-07-18T03:11:17.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.592+0000 7f90f7f4eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dddea71920 con 0x55dddec9bbe0 2023-07-18T03:11:17.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.593+0000 7f90f7f4eb80 10 client.0 _ll_register_callbacks cb 0x55dddeb8b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:11:17.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.593+0000 7f90e239b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dddec9b2d0 0x55dddeca6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:17.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.593+0000 7f90e239b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dddec9b2d0 0x55dddeca6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:56086/0 (socket says 172.21.15.101:56086) 2023-07-18T03:11:17.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.593+0000 7f90e239b700 1 -- 172.21.15.101:0/2402409275 learned_addr learned my addr 172.21.15.101:0/2402409275 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:17.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.593+0000 7f90e1b9a700 1 --2- 172.21.15.101:0/2402409275 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dddec9bbe0 0x55dddeca73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:17.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.593+0000 7f90e2b9c700 1 --2- 172.21.15.101:0/2402409275 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dddeca6ad0 0x55dddecb1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:17.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.593+0000 7f90e239b700 1 -- 172.21.15.101:0/2402409275 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dddec9bbe0 msgr2=0x55dddeca73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:17.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.593+0000 7f90e239b700 1 --2- 172.21.15.101:0/2402409275 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dddec9bbe0 0x55dddeca73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.593+0000 7f90e239b700 1 -- 172.21.15.101:0/2402409275 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dddeca6ad0 msgr2=0x55dddecb1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:17.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.593+0000 7f90e239b700 1 --2- 172.21.15.101:0/2402409275 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dddeca6ad0 0x55dddecb1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.593+0000 7f90e239b700 1 -- 172.21.15.101:0/2402409275 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90dc1a4c40 con 0x55dddec9b2d0 2023-07-18T03:11:17.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.593+0000 7f90e1b9a700 1 --2- 172.21.15.101:0/2402409275 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dddec9bbe0 0x55dddeca73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:11:17.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.593+0000 7f90e2b9c700 1 --2- 172.21.15.101:0/2402409275 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dddeca6ad0 0x55dddecb1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:11:17.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.593+0000 7f90e239b700 1 --2- 172.21.15.101:0/2402409275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dddec9b2d0 0x55dddeca6ea0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f90d8097260 tx=0x7f90d80074e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c2fee2bacbcc1e1 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:17.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.593+0000 7f90c5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:11:17.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.594+0000 7f90c5ffb700 1 -- 172.21.15.101:0/2402409275 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d806dee0 con 0x55dddec9b2d0 2023-07-18T03:11:17.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.594+0000 7f90c5ffb700 1 -- 172.21.15.101:0/2402409275 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f90d809ad60 con 0x55dddec9b2d0 2023-07-18T03:11:17.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.594+0000 7f90f7f4eb80 1 -- 172.21.15.101:0/2402409275 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dddeb474f0 con 0x55dddec9b2d0 2023-07-18T03:11:17.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.594+0000 7f90c5ffb700 1 -- 172.21.15.101:0/2402409275 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90d80051a0 con 0x55dddec9b2d0 2023-07-18T03:11:17.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.594+0000 7f90f7f4eb80 10 client.26375 fetch_fsmap learned FSMap version 25 2023-07-18T03:11:17.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.594+0000 7f90f7f4eb80 1 -- 172.21.15.101:0/2402409275 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55dddea04c60 con 0x55dddec9b2d0 2023-07-18T03:11:17.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.594+0000 7f90c5ffb700 1 -- 172.21.15.101:0/2402409275 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dddeb474f0 con 0x55dddec9b2d0 2023-07-18T03:11:17.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.594+0000 7f90c5ffb700 1 -- 172.21.15.101:0/2402409275 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f90d809a250 con 0x55dddec9b2d0 2023-07-18T03:11:17.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.594+0000 7f90f7f4eb80 10 client.26375 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:11:17.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.594+0000 7f90f7f4eb80 10 client.26375 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:11:17.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.594+0000 7f90f7f4eb80 20 client.26375 populate_metadata read hostname 'smithi101' 2023-07-18T03:11:17.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.594+0000 7f90f7f4eb80 1 --2- 172.21.15.101:0/2402409275 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55dddecb5ce0 0x55dddecb80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:17.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.594+0000 7f90f7f4eb80 4 client.26375 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:11:17.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.594+0000 7f90f7f4eb80 1 -- 172.21.15.101:0/2402409275 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dddebe6eb0 con 0x55dddecb5ce0 2023-07-18T03:11:17.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.595+0000 7f90e1b9a700 1 --2- 172.21.15.101:0/2402409275 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55dddecb5ce0 0x55dddecb80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:17.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.595+0000 7f90e1b9a700 1 --2- 172.21.15.101:0/2402409275 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55dddecb5ce0 0x55dddecb80c0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5413ca826ea448 server_cookie=15a7e0e1aa9aff32 in_seq=0 out_seq=0 2023-07-18T03:11:17.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.595+0000 7f90c5ffb700 10 client.26375 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:11:17.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.596+0000 7f90c5ffb700 1 -- 172.21.15.101:0/2402409275 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55dddebe6eb0 con 0x55dddecb5ce0 2023-07-18T03:11:17.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.596+0000 7f90c5ffb700 10 client.26375 handle_command_reply: tid=0 2023-07-18T03:11:17.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.596+0000 7f90f7f4eb80 1 client.26375 shutdown 2023-07-18T03:11:17.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.596+0000 7f90f7f4eb80 1 -- 172.21.15.101:0/2402409275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dddec9b2d0 msgr2=0x55dddeca6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:17.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.596+0000 7f90f7f4eb80 1 --2- 172.21.15.101:0/2402409275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dddec9b2d0 0x55dddeca6ea0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f90d8097260 tx=0x7f90d80074e0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.596+0000 7f90f7f4eb80 1 -- 172.21.15.101:0/2402409275 shutdown_connections 2023-07-18T03:11:17.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.596+0000 7f90f7f4eb80 1 --2- 172.21.15.101:0/2402409275 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55dddecb5ce0 0x55dddecb80c0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.596+0000 7f90f7f4eb80 1 --2- 172.21.15.101:0/2402409275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dddec9b2d0 0x55dddeca6ea0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.596+0000 7f90f7f4eb80 1 --2- 172.21.15.101:0/2402409275 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dddeca6ad0 0x55dddecb1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.596+0000 7f90f7f4eb80 1 --2- 172.21.15.101:0/2402409275 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dddec9bbe0 0x55dddeca73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.596+0000 7f90c5ffb700 0 client.26375 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:11:17.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.596+0000 7f90f7f4eb80 1 -- 172.21.15.101:0/2402409275 >> 172.21.15.101:0/2402409275 conn(0x55dddec91da0 msgr2=0x55dddec97230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:17.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.596+0000 7f90f7f4eb80 1 -- 172.21.15.101:0/2402409275 shutdown_connections 2023-07-18T03:11:17.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.596+0000 7f90f7f4eb80 1 -- 172.21.15.101:0/2402409275 wait complete. 2023-07-18T03:11:17.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.596+0000 7f90f7f4eb80 20 client.26375 trim_cache size 0 max 16384 2023-07-18T03:11:17.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.614+0000 7f90f7f4eb80 1 Processor -- start 2023-07-18T03:11:17.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.614+0000 7f90f7f4eb80 1 -- start start 2023-07-18T03:11:17.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.615+0000 7f90f7f4eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dddec9b2d0 0x55dddeca5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:17.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.615+0000 7f90f7f4eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dddec9bbe0 0x55dddeca5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:17.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.615+0000 7f90f7f4eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dddeca6ad0 0x55dddecb1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:17.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.615+0000 7f90f7f4eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dddeca7750 con 0x55dddec9b2d0 2023-07-18T03:11:17.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.615+0000 7f90f7f4eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dddeca61d0 con 0x55dddeca6ad0 2023-07-18T03:11:17.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.615+0000 7f90f7f4eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dddeca0a00 con 0x55dddec9bbe0 2023-07-18T03:11:17.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.615+0000 7f90e1b9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dddec9bbe0 0x55dddeca5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:17.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.615+0000 7f90e1b9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dddec9bbe0 0x55dddeca5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:52996/0 (socket says 172.21.15.101:52996) 2023-07-18T03:11:17.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.615+0000 7f90e1b9a700 1 -- 172.21.15.101:0/1604340489 learned_addr learned my addr 172.21.15.101:0/1604340489 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:17.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.615+0000 7f90e239b700 1 --2- 172.21.15.101:0/1604340489 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dddec9b2d0 0x55dddeca5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:17.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.615+0000 7f90e2b9c700 1 --2- 172.21.15.101:0/1604340489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dddeca6ad0 0x55dddecb1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:17.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.616+0000 7f90e1b9a700 1 -- 172.21.15.101:0/1604340489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dddeca6ad0 msgr2=0x55dddecb1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:17.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.616+0000 7f90e1b9a700 1 --2- 172.21.15.101:0/1604340489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dddeca6ad0 0x55dddecb1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.616+0000 7f90e1b9a700 1 -- 172.21.15.101:0/1604340489 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dddec9b2d0 msgr2=0x55dddeca5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:17.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.616+0000 7f90e1b9a700 1 --2- 172.21.15.101:0/1604340489 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dddec9b2d0 0x55dddeca5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.616+0000 7f90e1b9a700 1 -- 172.21.15.101:0/1604340489 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dddec935e0 con 0x55dddec9bbe0 2023-07-18T03:11:17.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.616+0000 7f90e2b9c700 1 --2- 172.21.15.101:0/1604340489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dddeca6ad0 0x55dddecb1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:11:17.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.616+0000 7f90e1b9a700 1 --2- 172.21.15.101:0/1604340489 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dddec9bbe0 0x55dddeca5670 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f90cc009bd0 tx=0x7f90cc014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=578f4682208047fd server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:17.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.616+0000 7f90d13ff700 1 -- 172.21.15.101:0/1604340489 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90cc024070 con 0x55dddec9bbe0 2023-07-18T03:11:17.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.616+0000 7f90d13ff700 1 -- 172.21.15.101:0/1604340489 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90cc026cb0 con 0x55dddec9bbe0 2023-07-18T03:11:17.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.616+0000 7f90d13ff700 1 -- 172.21.15.101:0/1604340489 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90cc01e420 con 0x55dddec9bbe0 2023-07-18T03:11:17.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.617+0000 7f90f7f4eb80 1 -- 172.21.15.101:0/1604340489 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dddec9bbe0 msgr2=0x55dddeca5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:17.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.617+0000 7f90f7f4eb80 1 --2- 172.21.15.101:0/1604340489 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dddec9bbe0 0x55dddeca5670 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f90cc009bd0 tx=0x7f90cc014000 comp rx=0 tx=0).stop 2023-07-18T03:11:17.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.617+0000 7f90f7f4eb80 1 -- 172.21.15.101:0/1604340489 shutdown_connections 2023-07-18T03:11:17.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.617+0000 7f90f7f4eb80 1 --2- 172.21.15.101:0/1604340489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dddeca6ad0 0x55dddecb1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.617+0000 7f90f7f4eb80 1 --2- 172.21.15.101:0/1604340489 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dddec9b2d0 0x55dddeca5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.617+0000 7f90f7f4eb80 1 --2- 172.21.15.101:0/1604340489 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dddec9bbe0 0x55dddeca5670 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.617+0000 7f90f7f4eb80 1 -- 172.21.15.101:0/1604340489 >> 172.21.15.101:0/1604340489 conn(0x55dddec91da0 msgr2=0x55dddeca0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:17.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.617+0000 7f90f7f4eb80 1 -- 172.21.15.101:0/1604340489 shutdown_connections 2023-07-18T03:11:17.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.617+0000 7f90f7f4eb80 1 -- 172.21.15.101:0/1604340489 wait complete. 2023-07-18T03:11:17.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.617+0000 7f90f7f4eb80 1 -- start start 2023-07-18T03:11:17.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.618+0000 7f90f7f4eb80 1 Processor -- start 2023-07-18T03:11:17.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.618+0000 7f90f7f4eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dddec9b2d0 0x55dddecabdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:17.616 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.618+0000 7f90f7f4eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dddec9bbe0 0x55dddecaa120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:17.616 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.618+0000 7f90f7f4eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dddeca6ad0 0x55dddecaa660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:17.616 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.618+0000 7f90f7f4eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dddecaabf0 con 0x55dddec9bbe0 2023-07-18T03:11:17.616 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.618+0000 7f90f7f4eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dddecaad30 con 0x55dddec9b2d0 2023-07-18T03:11:17.616 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.618+0000 7f90f7f4eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dddecaaef0 con 0x55dddeca6ad0 2023-07-18T03:11:17.616 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.618+0000 7f90f7f4eb80 10 client.0 _ll_register_callbacks cb 0x7f90d4009b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:11:17.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.618+0000 7f90e239b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dddec9b2d0 0x55dddecabdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:17.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.619+0000 7f90e239b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dddec9b2d0 0x55dddecabdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:56110/0 (socket says 172.21.15.101:56110) 2023-07-18T03:11:17.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.619+0000 7f90e239b700 1 -- 172.21.15.101:0/3795252865 learned_addr learned my addr 172.21.15.101:0/3795252865 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:17.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.619+0000 7f90e1b9a700 1 --2- 172.21.15.101:0/3795252865 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dddec9bbe0 0x55dddecaa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:17.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.619+0000 7f90e2b9c700 1 --2- 172.21.15.101:0/3795252865 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dddeca6ad0 0x55dddecaa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:17.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.619+0000 7f90e239b700 1 -- 172.21.15.101:0/3795252865 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dddeca6ad0 msgr2=0x55dddecaa660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:17.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.619+0000 7f90e239b700 1 --2- 172.21.15.101:0/3795252865 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dddeca6ad0 0x55dddecaa660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.619+0000 7f90e239b700 1 -- 172.21.15.101:0/3795252865 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dddec9bbe0 msgr2=0x55dddecaa120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:17.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.619+0000 7f90e239b700 1 --2- 172.21.15.101:0/3795252865 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dddec9bbe0 0x55dddecaa120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.619+0000 7f90e239b700 1 -- 172.21.15.101:0/3795252865 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90d4009e70 con 0x55dddec9b2d0 2023-07-18T03:11:17.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.619+0000 7f90e1b9a700 1 --2- 172.21.15.101:0/3795252865 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dddec9bbe0 0x55dddecaa120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:11:17.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.619+0000 7f90e239b700 1 --2- 172.21.15.101:0/3795252865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dddec9b2d0 0x55dddecabdf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f90d8009fd0 tx=0x7f90d8060eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f58f2111b94f9339 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:17.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.619+0000 7f90c5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:11:17.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.619+0000 7f90c5ffb700 1 -- 172.21.15.101:0/3795252865 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d80a2ef0 con 0x55dddec9b2d0 2023-07-18T03:11:17.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.619+0000 7f90c5ffb700 1 -- 172.21.15.101:0/3795252865 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f90d80051a0 con 0x55dddec9b2d0 2023-07-18T03:11:17.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.619+0000 7f90f7f4eb80 1 -- 172.21.15.101:0/3795252865 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f90d8063ab0 con 0x55dddec9b2d0 2023-07-18T03:11:17.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.620+0000 7f90c5ffb700 1 -- 172.21.15.101:0/3795252865 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90d80b90d0 con 0x55dddec9b2d0 2023-07-18T03:11:17.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.620+0000 7f90f7f4eb80 10 client.26381 fetch_fsmap learned FSMap version 25 2023-07-18T03:11:17.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.620+0000 7f90f7f4eb80 1 -- 172.21.15.101:0/3795252865 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f90cc017070 con 0x55dddec9b2d0 2023-07-18T03:11:17.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.620+0000 7f90c5ffb700 1 -- 172.21.15.101:0/3795252865 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90d8063ab0 con 0x55dddec9b2d0 2023-07-18T03:11:17.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.620+0000 7f90c5ffb700 1 -- 172.21.15.101:0/3795252865 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f90d8096ce0 con 0x55dddec9b2d0 2023-07-18T03:11:17.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.620+0000 7f90f7f4eb80 10 client.26381 fetch_fsmap finished waiting for FSMap version 25 2023-07-18T03:11:17.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.620+0000 7f90f7f4eb80 10 client.26381 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:11:17.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.620+0000 7f90f7f4eb80 20 client.26381 populate_metadata read hostname 'smithi101' 2023-07-18T03:11:17.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.620+0000 7f90f7f4eb80 1 --2- 172.21.15.101:0/3795252865 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ddded0aca0 0x55ddded0d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:17.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.620+0000 7f90f7f4eb80 4 client.26381 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:11:17.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.620+0000 7f90f7f4eb80 1 -- 172.21.15.101:0/3795252865 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55dddeba9050 con 0x55ddded0aca0 2023-07-18T03:11:17.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.620+0000 7f90e1b9a700 1 --2- 172.21.15.101:0/3795252865 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ddded0aca0 0x55ddded0d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:17.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.621+0000 7f90e1b9a700 1 --2- 172.21.15.101:0/3795252865 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ddded0aca0 0x55ddded0d0b0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6588fd5e736cdae server_cookie=68ac49e6eba04563 in_seq=0 out_seq=0 2023-07-18T03:11:17.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.621+0000 7f90c5ffb700 10 client.26381 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:11:17.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.621+0000 7f90c5ffb700 1 -- 172.21.15.101:0/3795252865 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55dddeba9050 con 0x55ddded0aca0 2023-07-18T03:11:17.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.621+0000 7f90c5ffb700 10 client.26381 handle_command_reply: tid=0 2023-07-18T03:11:17.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.621+0000 7f90f7f4eb80 1 client.26381 shutdown 2023-07-18T03:11:17.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.621+0000 7f90f7f4eb80 1 -- 172.21.15.101:0/3795252865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dddec9b2d0 msgr2=0x55dddecabdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:17.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.621+0000 7f90f7f4eb80 1 --2- 172.21.15.101:0/3795252865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dddec9b2d0 0x55dddecabdf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f90d8009fd0 tx=0x7f90d8060eb0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.621+0000 7f90f7f4eb80 1 -- 172.21.15.101:0/3795252865 shutdown_connections 2023-07-18T03:11:17.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.621+0000 7f90f7f4eb80 1 --2- 172.21.15.101:0/3795252865 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ddded0aca0 0x55ddded0d0b0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.621+0000 7f90f7f4eb80 1 --2- 172.21.15.101:0/3795252865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dddec9b2d0 0x55dddecabdf0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.621+0000 7f90f7f4eb80 1 --2- 172.21.15.101:0/3795252865 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dddeca6ad0 0x55dddecaa660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.621+0000 7f90c5ffb700 0 client.26381 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:11:17.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.621+0000 7f90f7f4eb80 1 --2- 172.21.15.101:0/3795252865 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dddec9bbe0 0x55dddecaa120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.622+0000 7f90f7f4eb80 1 -- 172.21.15.101:0/3795252865 >> 172.21.15.101:0/3795252865 conn(0x55dddec91da0 msgr2=0x55dddec9f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:17.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.622+0000 7f90f7f4eb80 1 -- 172.21.15.101:0/3795252865 shutdown_connections 2023-07-18T03:11:17.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.622+0000 7f90f7f4eb80 1 -- 172.21.15.101:0/3795252865 wait complete. 2023-07-18T03:11:17.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.622+0000 7f90f7f4eb80 20 client.26381 trim_cache size 0 max 16384 2023-07-18T03:11:17.624 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:11:17.625 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:11:17.625 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:11:17.625 INFO:teuthology.orchestra.run.smithi101.stdout: "596ffb58-c4df-4721-b069-b45be65bb2cc": { 2023-07-18T03:11:17.625 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:11:17.625 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "596ffb58-c4df-4721-b069-b45be65bb2cc", 2023-07-18T03:11:17.626 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:11:17.626 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:11:17.626 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:11:17.626 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:11:17.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.623+0000 7f90e45ff700 1 -- 172.21.15.101:0/252650067 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f90c80a12c0 msgr2=0x7f90c80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:17.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.623+0000 7f90e45ff700 1 --2- 172.21.15.101:0/252650067 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f90c80a12c0 0x7f90c80a3770 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f90dc199600 tx=0x7f90cc009260 comp rx=0 tx=0).stop 2023-07-18T03:11:17.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.623+0000 7f90e45ff700 1 -- 172.21.15.101:0/252650067 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f90dc101fa0 msgr2=0x7f90dc197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:17.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.623+0000 7f90e45ff700 1 --2- 172.21.15.101:0/252650067 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f90dc101fa0 0x7f90dc197ff0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f90d8000f80 tx=0x7f90d8007870 comp rx=0 tx=0).stop 2023-07-18T03:11:17.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.623+0000 7f90e45ff700 1 -- 172.21.15.101:0/252650067 shutdown_connections 2023-07-18T03:11:17.628 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.623+0000 7f90e45ff700 1 --2- 172.21.15.101:0/252650067 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90dc106fe0 0x7f90dc19c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.628 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.623+0000 7f90e45ff700 1 --2- 172.21.15.101:0/252650067 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f90dc101fa0 0x7f90dc197ff0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.628 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.623+0000 7f90e45ff700 1 --2- 172.21.15.101:0/252650067 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f90c80a12c0 0x7f90c80a3770 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.628 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.623+0000 7f90e45ff700 1 --2- 172.21.15.101:0/252650067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90dc102940 0x7f90dc198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:17.628 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.623+0000 7f90e45ff700 1 -- 172.21.15.101:0/252650067 >> 172.21.15.101:0/252650067 conn(0x7f90dc0fd7d0 msgr2=0x7f90dc0fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:17.628 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.623+0000 7f90e45ff700 1 -- 172.21.15.101:0/252650067 shutdown_connections 2023-07-18T03:11:17.628 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:17.623+0000 7f90e45ff700 1 -- 172.21.15.101:0/252650067 wait complete. 2023-07-18T03:11:17.634 INFO:tasks.cephfs.filesystem:scrub status for tag:596ffb58-c4df-4721-b069-b45be65bb2cc - {'path': '/', 'tag': '596ffb58-c4df-4721-b069-b45be65bb2cc', 'options': 'recursive,force'} 2023-07-18T03:11:18.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:18 smithi179 ceph-mon[129229]: scrub queued for path: / 2023-07-18T03:11:18.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:18 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:11:18.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:18 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:11:18.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:18 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:11:18.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:18 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:11:18.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:18 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:11:18.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:18 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:11:18.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:18 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:11:18.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:18 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:11:18.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:18 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:11:18.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:18 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:11:18.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:18 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:11:18.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:18 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:11:18.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:18 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:11:18.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:18 smithi101 ceph-mon[126379]: scrub queued for path: / 2023-07-18T03:11:18.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:18 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:11:18.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:18 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:11:18.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:18 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:11:18.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:18 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:11:18.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:18 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:11:18.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:18 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:11:18.492 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:18 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:11:18.492 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:18 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:11:18.492 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:18 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:11:18.492 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:18 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:11:18.492 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:18 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:11:18.492 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:18 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:11:18.493 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:18 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:11:18.543 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:18 smithi155 ceph-mon[128885]: scrub queued for path: / 2023-07-18T03:11:18.543 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:18 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:11:18.543 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:18 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:11:18.544 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:18 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:11:18.544 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:18 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:11:18.544 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:18 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:11:18.544 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:18 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:11:18.544 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:18 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:11:18.544 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:18 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:11:18.545 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:18 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:11:18.545 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:18 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:11:18.545 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:18 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:11:18.545 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:18 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:11:18.545 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:18 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:11:18.573 INFO:tasks.workunit.client.0.smithi101.stdout: 1 252619 8.94 MB/sec execute 43 sec latency 40.051 ms 2023-07-18T03:11:19.447 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:19 smithi179 ceph-mon[129229]: pgmap v115: 129 pgs: 129 active+clean; 150 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 210 op/s 2023-07-18T03:11:19.485 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:19 smithi101 ceph-mon[126379]: pgmap v115: 129 pgs: 129 active+clean; 150 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 210 op/s 2023-07-18T03:11:19.543 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:19 smithi155 ceph-mon[128885]: pgmap v115: 129 pgs: 129 active+clean; 150 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 210 op/s 2023-07-18T03:11:19.573 INFO:tasks.workunit.client.0.smithi101.stdout: 1 254858 9.02 MB/sec execute 44 sec latency 29.306 ms 2023-07-18T03:11:20.573 INFO:tasks.workunit.client.0.smithi101.stdout: 1 256160 8.97 MB/sec execute 45 sec latency 14.354 ms 2023-07-18T03:11:21.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:21 smithi179 ceph-mon[129229]: pgmap v116: 129 pgs: 129 active+clean; 156 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 320 op/s 2023-07-18T03:11:21.485 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:21 smithi101 ceph-mon[126379]: pgmap v116: 129 pgs: 129 active+clean; 156 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 320 op/s 2023-07-18T03:11:21.543 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:21 smithi155 ceph-mon[128885]: pgmap v116: 129 pgs: 129 active+clean; 156 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 320 op/s 2023-07-18T03:11:21.574 INFO:tasks.workunit.client.0.smithi101.stdout: 1 258412 9.04 MB/sec execute 46 sec latency 11.937 ms 2023-07-18T03:11:22.573 INFO:tasks.workunit.client.0.smithi101.stdout: 1 259634 8.99 MB/sec execute 47 sec latency 33.387 ms 2023-07-18T03:11:23.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:23 smithi179 ceph-mon[129229]: pgmap v117: 129 pgs: 129 active+clean; 156 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 213 op/s 2023-07-18T03:11:23.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:23 smithi101 ceph-mon[126379]: pgmap v117: 129 pgs: 129 active+clean; 156 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 213 op/s 2023-07-18T03:11:23.545 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:23 smithi155 ceph-mon[128885]: pgmap v117: 129 pgs: 129 active+clean; 156 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 213 op/s 2023-07-18T03:11:23.574 INFO:tasks.workunit.client.0.smithi101.stdout: 1 261737 9.03 MB/sec execute 48 sec latency 42.147 ms 2023-07-18T03:11:24.574 INFO:tasks.workunit.client.0.smithi101.stdout: 1 263059 9.01 MB/sec execute 49 sec latency 24.133 ms 2023-07-18T03:11:25.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:25 smithi179 ceph-mon[129229]: pgmap v118: 129 pgs: 129 active+clean; 156 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 213 op/s 2023-07-18T03:11:25.485 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:25 smithi101 ceph-mon[126379]: pgmap v118: 129 pgs: 129 active+clean; 156 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 213 op/s 2023-07-18T03:11:25.543 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:25 smithi155 ceph-mon[128885]: pgmap v118: 129 pgs: 129 active+clean; 156 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 213 op/s 2023-07-18T03:11:25.574 INFO:tasks.workunit.client.0.smithi101.stdout: 1 265278 9.05 MB/sec execute 50 sec latency 18.494 ms 2023-07-18T03:11:26.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:26 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:11:26.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:26 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:11:26.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:26 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:11:26.485 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:26 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:11:26.486 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:26 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:11:26.486 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:26 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:11:26.543 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:26 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:11:26.543 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:26 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:11:26.543 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:26 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:11:26.574 INFO:tasks.workunit.client.0.smithi101.stdout: 1 266593 9.03 MB/sec execute 51 sec latency 18.099 ms 2023-07-18T03:11:27.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:27 smithi179 ceph-mon[129229]: pgmap v119: 129 pgs: 129 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 318 op/s 2023-07-18T03:11:27.485 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:27 smithi101 ceph-mon[126379]: pgmap v119: 129 pgs: 129 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 318 op/s 2023-07-18T03:11:27.543 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:27 smithi155 ceph-mon[128885]: pgmap v119: 129 pgs: 129 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 318 op/s 2023-07-18T03:11:27.574 INFO:tasks.workunit.client.0.smithi101.stdout: 1 268619 9.01 MB/sec execute 52 sec latency 24.439 ms 2023-07-18T03:11:28.574 INFO:tasks.workunit.client.0.smithi101.stdout: 1 270110 9.05 MB/sec execute 53 sec latency 12.672 ms 2023-07-18T03:11:29.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:29 smithi179 ceph-mon[129229]: pgmap v120: 129 pgs: 129 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 214 op/s 2023-07-18T03:11:29.485 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:29 smithi101 ceph-mon[126379]: pgmap v120: 129 pgs: 129 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 214 op/s 2023-07-18T03:11:29.543 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:29 smithi155 ceph-mon[128885]: pgmap v120: 129 pgs: 129 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 214 op/s 2023-07-18T03:11:29.574 INFO:tasks.workunit.client.0.smithi101.stdout: 1 272032 9.02 MB/sec execute 54 sec latency 34.636 ms 2023-07-18T03:11:30.574 INFO:tasks.workunit.client.0.smithi101.stdout: 1 273596 9.07 MB/sec execute 55 sec latency 10.713 ms 2023-07-18T03:11:31.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:31 smithi179 ceph-mon[129229]: pgmap v121: 129 pgs: 129 active+clean; 166 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 326 op/s 2023-07-18T03:11:31.485 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:31 smithi101 ceph-mon[126379]: pgmap v121: 129 pgs: 129 active+clean; 166 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 326 op/s 2023-07-18T03:11:31.543 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:31 smithi155 ceph-mon[128885]: pgmap v121: 129 pgs: 129 active+clean; 166 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 326 op/s 2023-07-18T03:11:31.574 INFO:tasks.workunit.client.0.smithi101.stdout: 1 275642 9.05 MB/sec execute 56 sec latency 28.436 ms 2023-07-18T03:11:32.574 INFO:tasks.workunit.client.0.smithi101.stdout: 1 277136 9.09 MB/sec execute 57 sec latency 29.973 ms 2023-07-18T03:11:33.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:33 smithi179 ceph-mon[129229]: pgmap v122: 129 pgs: 129 active+clean; 166 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 216 op/s 2023-07-18T03:11:33.485 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:33 smithi101 ceph-mon[126379]: pgmap v122: 129 pgs: 129 active+clean; 166 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 216 op/s 2023-07-18T03:11:33.543 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:33 smithi155 ceph-mon[128885]: pgmap v122: 129 pgs: 129 active+clean; 166 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 216 op/s 2023-07-18T03:11:33.574 INFO:tasks.workunit.client.0.smithi101.stdout: 1 279091 9.05 MB/sec execute 58 sec latency 101.220 ms 2023-07-18T03:11:34.574 INFO:tasks.workunit.client.0.smithi101.stdout: 1 280514 9.10 MB/sec execute 59 sec latency 63.315 ms 2023-07-18T03:11:35.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:35 smithi179 ceph-mon[129229]: pgmap v123: 129 pgs: 129 active+clean; 166 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 216 op/s 2023-07-18T03:11:35.486 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:35 smithi101 ceph-mon[126379]: pgmap v123: 129 pgs: 129 active+clean; 166 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 216 op/s 2023-07-18T03:11:35.543 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:35 smithi155 ceph-mon[128885]: pgmap v123: 129 pgs: 129 active+clean; 166 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 216 op/s 2023-07-18T03:11:35.575 INFO:tasks.workunit.client.0.smithi101.stdout: 1 282198 9.04 MB/sec execute 60 sec latency 12.919 ms 2023-07-18T03:11:36.574 INFO:tasks.workunit.client.0.smithi101.stdout: 1 284110 9.11 MB/sec execute 61 sec latency 10.846 ms 2023-07-18T03:11:37.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:37 smithi179 ceph-mon[129229]: pgmap v124: 129 pgs: 129 active+clean; 171 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 315 op/s 2023-07-18T03:11:37.485 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:37 smithi101 ceph-mon[126379]: pgmap v124: 129 pgs: 129 active+clean; 171 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 315 op/s 2023-07-18T03:11:37.543 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:37 smithi155 ceph-mon[128885]: pgmap v124: 129 pgs: 129 active+clean; 171 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 315 op/s 2023-07-18T03:11:37.575 INFO:tasks.workunit.client.0.smithi101.stdout: 1 285680 9.06 MB/sec execute 62 sec latency 19.467 ms 2023-07-18T03:11:38.575 INFO:tasks.workunit.client.0.smithi101.stdout: 1 287673 9.13 MB/sec execute 63 sec latency 10.820 ms 2023-07-18T03:11:39.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:39 smithi179 ceph-mon[129229]: pgmap v125: 129 pgs: 129 active+clean; 171 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 211 op/s 2023-07-18T03:11:39.485 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:39 smithi101 ceph-mon[126379]: pgmap v125: 129 pgs: 129 active+clean; 171 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 211 op/s 2023-07-18T03:11:39.543 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:39 smithi155 ceph-mon[128885]: pgmap v125: 129 pgs: 129 active+clean; 171 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 211 op/s 2023-07-18T03:11:39.575 INFO:tasks.workunit.client.0.smithi101.stdout: 1 288742 9.07 MB/sec execute 64 sec latency 23.782 ms 2023-07-18T03:11:40.575 INFO:tasks.workunit.client.0.smithi101.stdout: 1 291104 9.14 MB/sec execute 65 sec latency 20.311 ms 2023-07-18T03:11:41.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:41 smithi179 ceph-mon[129229]: pgmap v126: 129 pgs: 129 active+clean; 177 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 319 op/s 2023-07-18T03:11:41.486 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:41 smithi101 ceph-mon[126379]: pgmap v126: 129 pgs: 129 active+clean; 177 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 319 op/s 2023-07-18T03:11:41.543 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:41 smithi155 ceph-mon[128885]: pgmap v126: 129 pgs: 129 active+clean; 177 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 319 op/s 2023-07-18T03:11:41.575 INFO:tasks.workunit.client.0.smithi101.stdout: 1 292097 9.04 MB/sec execute 66 sec latency 19.379 ms 2023-07-18T03:11:42.575 INFO:tasks.workunit.client.0.smithi101.stdout: 1 294559 9.14 MB/sec execute 67 sec latency 10.889 ms 2023-07-18T03:11:43.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:43 smithi179 ceph-mon[129229]: pgmap v127: 129 pgs: 129 active+clean; 177 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 208 op/s 2023-07-18T03:11:43.485 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:43 smithi101 ceph-mon[126379]: pgmap v127: 129 pgs: 129 active+clean; 177 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 208 op/s 2023-07-18T03:11:43.544 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:43 smithi155 ceph-mon[128885]: pgmap v127: 129 pgs: 129 active+clean; 177 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 208 op/s 2023-07-18T03:11:43.575 INFO:tasks.workunit.client.0.smithi101.stdout: 1 295542 9.05 MB/sec execute 68 sec latency 18.774 ms 2023-07-18T03:11:44.575 INFO:tasks.workunit.client.0.smithi101.stdout: 1 297418 9.07 MB/sec execute 69 sec latency 100.248 ms 2023-07-18T03:11:45.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:45 smithi179 ceph-mon[129229]: pgmap v128: 129 pgs: 129 active+clean; 177 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 208 op/s 2023-07-18T03:11:45.485 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:45 smithi101 ceph-mon[126379]: pgmap v128: 129 pgs: 129 active+clean; 177 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 208 op/s 2023-07-18T03:11:45.543 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:45 smithi155 ceph-mon[128885]: pgmap v128: 129 pgs: 129 active+clean; 177 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 208 op/s 2023-07-18T03:11:45.575 INFO:tasks.workunit.client.0.smithi101.stdout: 1 298566 9.04 MB/sec execute 70 sec latency 28.637 ms 2023-07-18T03:11:46.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:46 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:11:46.486 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:46 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:11:46.543 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:46 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:11:46.575 INFO:tasks.workunit.client.0.smithi101.stdout: 1 300734 9.07 MB/sec execute 71 sec latency 14.954 ms 2023-07-18T03:11:47.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:47 smithi179 ceph-mon[129229]: pgmap v129: 129 pgs: 129 active+clean; 182 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 305 op/s 2023-07-18T03:11:47.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:47 smithi179 ceph-mon[129229]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:11:47.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:47 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:11:47.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:47 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:11:47.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:47 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:11:47.486 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:47 smithi101 ceph-mon[126379]: pgmap v129: 129 pgs: 129 active+clean; 182 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 305 op/s 2023-07-18T03:11:47.486 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:47 smithi101 ceph-mon[126379]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:11:47.486 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:47 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:11:47.486 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:47 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:11:47.486 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:47 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:11:47.543 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:47 smithi155 ceph-mon[128885]: pgmap v129: 129 pgs: 129 active+clean; 182 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 305 op/s 2023-07-18T03:11:47.543 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:47 smithi155 ceph-mon[128885]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:11:47.543 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:47 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:11:47.544 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:47 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:11:47.544 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:47 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:11:47.575 INFO:tasks.workunit.client.0.smithi101.stdout: 1 302080 9.05 MB/sec execute 72 sec latency 17.194 ms 2023-07-18T03:11:47.635 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:11:47.858 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.867+0000 7fab1d1ff700 1 -- 172.21.15.101:0/198755140 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab18105ae0 msgr2=0x7fab18105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:47.858 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.867+0000 7fab1d1ff700 1 --2- 172.21.15.101:0/198755140 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab18105ae0 0x7fab18105eb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fab08009990 tx=0x7fab08009ca0 comp rx=0 tx=0).stop 2023-07-18T03:11:47.858 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.867+0000 7fab1d1ff700 1 -- 172.21.15.101:0/198755140 shutdown_connections 2023-07-18T03:11:47.859 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.867+0000 7fab1d1ff700 1 --2- 172.21.15.101:0/198755140 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fab18100480 0x7fab1810c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:47.859 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.867+0000 7fab1d1ff700 1 --2- 172.21.15.101:0/198755140 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab18105ae0 0x7fab18105eb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:47.859 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.867+0000 7fab1d1ff700 1 --2- 172.21.15.101:0/198755140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab180ffad0 0x7fab180fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:47.859 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.867+0000 7fab1d1ff700 1 -- 172.21.15.101:0/198755140 >> 172.21.15.101:0/198755140 conn(0x7fab180fb610 msgr2=0x7fab180fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:47.859 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.867+0000 7fab1d1ff700 1 -- 172.21.15.101:0/198755140 shutdown_connections 2023-07-18T03:11:47.859 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.867+0000 7fab1d1ff700 1 -- 172.21.15.101:0/198755140 wait complete. 2023-07-18T03:11:47.861 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.868+0000 7fab1d1ff700 1 Processor -- start 2023-07-18T03:11:47.861 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.868+0000 7fab1d1ff700 1 -- start start 2023-07-18T03:11:47.861 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.869+0000 7fab1d1ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab180ffad0 0x7fab18197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:47.861 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.869+0000 7fab1d1ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab18100480 0x7fab18198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:47.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.869+0000 7fab1d1ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fab18105ae0 0x7fab1819c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:47.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.869+0000 7fab1d1ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1819ce10 con 0x7fab18105ae0 2023-07-18T03:11:47.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.869+0000 7fab1d1ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1819cf80 con 0x7fab180ffad0 2023-07-18T03:11:47.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.869+0000 7fab1d1ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1819d140 con 0x7fab18100480 2023-07-18T03:11:47.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.869+0000 7fab16d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab180ffad0 0x7fab18197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:47.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.869+0000 7fab16d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab180ffad0 0x7fab18197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:51844/0 (socket says 172.21.15.101:51844) 2023-07-18T03:11:47.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.869+0000 7fab16d9d700 1 -- 172.21.15.101:0/2284894704 learned_addr learned my addr 172.21.15.101:0/2284894704 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:47.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.869+0000 7fab1659c700 1 --2- 172.21.15.101:0/2284894704 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab18100480 0x7fab18198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:47.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.869+0000 7fab1759e700 1 --2- 172.21.15.101:0/2284894704 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fab18105ae0 0x7fab1819c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:47.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.869+0000 7fab16d9d700 1 -- 172.21.15.101:0/2284894704 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab18100480 msgr2=0x7fab18198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:47.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.869+0000 7fab16d9d700 1 --2- 172.21.15.101:0/2284894704 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab18100480 0x7fab18198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:47.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.869+0000 7fab16d9d700 1 -- 172.21.15.101:0/2284894704 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fab18105ae0 msgr2=0x7fab1819c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:47.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.869+0000 7fab16d9d700 1 --2- 172.21.15.101:0/2284894704 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fab18105ae0 0x7fab1819c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:47.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.869+0000 7fab16d9d700 1 -- 172.21.15.101:0/2284894704 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab10009c40 con 0x7fab180ffad0 2023-07-18T03:11:47.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.870+0000 7fab16d9d700 1 --2- 172.21.15.101:0/2284894704 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab180ffad0 0x7fab18197e30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fab08005910 tx=0x7fab08005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:47.864 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.870+0000 7faafffff700 1 -- 172.21.15.101:0/2284894704 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0801d070 con 0x7fab180ffad0 2023-07-18T03:11:47.864 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.870+0000 7fab1d1ff700 1 -- 172.21.15.101:0/2284894704 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab080095f0 con 0x7fab180ffad0 2023-07-18T03:11:47.864 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.870+0000 7faafffff700 1 -- 172.21.15.101:0/2284894704 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab08022470 con 0x7fab180ffad0 2023-07-18T03:11:47.864 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.870+0000 7faafffff700 1 -- 172.21.15.101:0/2284894704 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab08017650 con 0x7fab180ffad0 2023-07-18T03:11:47.864 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.870+0000 7fab1d1ff700 1 -- 172.21.15.101:0/2284894704 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab181a4a80 con 0x7fab180ffad0 2023-07-18T03:11:47.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.875+0000 7faafffff700 1 -- 172.21.15.101:0/2284894704 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fab080225e0 con 0x7fab180ffad0 2023-07-18T03:11:47.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.876+0000 7fab30b2fb80 1 Processor -- start 2023-07-18T03:11:47.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.876+0000 7fab30b2fb80 1 -- start start 2023-07-18T03:11:47.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.877+0000 7fab30b2fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615656d62d0 0x5615656d66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:47.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.877+0000 7fab30b2fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5615656d6be0 0x5615656e1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:47.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.877+0000 7fab30b2fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615656e1ad0 0x5615656e3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:47.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.877+0000 7fab30b2fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615655dccb0 con 0x5615656d6be0 2023-07-18T03:11:47.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.877+0000 7fab30b2fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56156562fa40 con 0x5615656d62d0 2023-07-18T03:11:47.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.877+0000 7fab30b2fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x561565628ed0 con 0x5615656e1ad0 2023-07-18T03:11:47.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.877+0000 7fab1759e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615656d62d0 0x5615656d66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:47.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.877+0000 7fab1759e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615656d62d0 0x5615656d66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:51860/0 (socket says 172.21.15.101:51860) 2023-07-18T03:11:47.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.877+0000 7fab1759e700 1 -- 172.21.15.101:0/3429212743 learned_addr learned my addr 172.21.15.101:0/3429212743 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:47.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.877+0000 7fab1659c700 1 --2- 172.21.15.101:0/3429212743 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615656e1ad0 0x5615656e3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:47.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.877+0000 7fab16d9d700 1 --2- 172.21.15.101:0/3429212743 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5615656d6be0 0x5615656e1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:47.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.877+0000 7faafffff700 1 --2- 172.21.15.101:0/2284894704 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fab000a1340 0x7fab000a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:47.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.877+0000 7fab1759e700 1 -- 172.21.15.101:0/3429212743 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615656e1ad0 msgr2=0x5615656e3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:47.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.877+0000 7fab1759e700 1 --2- 172.21.15.101:0/3429212743 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615656e1ad0 0x5615656e3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:47.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.877+0000 7fab1759e700 1 -- 172.21.15.101:0/3429212743 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5615656d6be0 msgr2=0x5615656e1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:47.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.877+0000 7fab1759e700 1 --2- 172.21.15.101:0/3429212743 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5615656d6be0 0x5615656e1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:47.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.877+0000 7fab1759e700 1 -- 172.21.15.101:0/3429212743 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561565456de0 con 0x5615656d62d0 2023-07-18T03:11:47.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.878+0000 7faafffff700 1 -- 172.21.15.101:0/2284894704 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fab080ce2f0 con 0x7fab180ffad0 2023-07-18T03:11:47.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.878+0000 7fab1759e700 1 --2- 172.21.15.101:0/2284894704 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fab000a1340 0x7fab000a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:47.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.878+0000 7fab1759e700 1 --2- 172.21.15.101:0/3429212743 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615656d62d0 0x5615656d66a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fab1000e730 tx=0x7fab10009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9638fd96163f74fe server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:47.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.878+0000 7faafd7fa700 1 -- 172.21.15.101:0/3429212743 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1000e040 con 0x5615656d62d0 2023-07-18T03:11:47.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.878+0000 7faafd7fa700 1 -- 172.21.15.101:0/3429212743 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab1000f040 con 0x5615656d62d0 2023-07-18T03:11:47.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.878+0000 7faafd7fa700 1 -- 172.21.15.101:0/3429212743 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab10015870 con 0x5615656d62d0 2023-07-18T03:11:47.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.879+0000 7fab30b2fb80 1 -- 172.21.15.101:0/3429212743 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615656d62d0 msgr2=0x5615656d66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:47.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.879+0000 7fab1759e700 1 --2- 172.21.15.101:0/2284894704 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fab000a1340 0x7fab000a37f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fab10003fd0 tx=0x7fab10023000 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:47.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.879+0000 7fab30b2fb80 1 --2- 172.21.15.101:0/3429212743 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615656d62d0 0x5615656d66a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fab1000e730 tx=0x7fab10009920 comp rx=0 tx=0).stop 2023-07-18T03:11:47.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.879+0000 7fab30b2fb80 1 -- 172.21.15.101:0/3429212743 shutdown_connections 2023-07-18T03:11:47.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.879+0000 7fab30b2fb80 1 --2- 172.21.15.101:0/3429212743 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615656d62d0 0x5615656d66a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:47.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.879+0000 7fab30b2fb80 1 --2- 172.21.15.101:0/3429212743 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615656e1ad0 0x5615656e3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:47.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.879+0000 7fab30b2fb80 1 --2- 172.21.15.101:0/3429212743 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5615656d6be0 0x5615656e1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:47.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.879+0000 7fab30b2fb80 1 -- 172.21.15.101:0/3429212743 >> 172.21.15.101:0/3429212743 conn(0x5615656ccda0 msgr2=0x5615656d26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:47.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.879+0000 7fab30b2fb80 1 -- 172.21.15.101:0/3429212743 shutdown_connections 2023-07-18T03:11:47.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.879+0000 7fab30b2fb80 1 -- 172.21.15.101:0/3429212743 wait complete. 2023-07-18T03:11:47.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.880+0000 7fab30b2fb80 1 -- start start 2023-07-18T03:11:47.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.880+0000 7fab30b2fb80 1 Processor -- start 2023-07-18T03:11:47.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.881+0000 7fab30b2fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615656d62d0 0x5615656e1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:47.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.881+0000 7fab30b2fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5615656d6be0 0x5615656de760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:47.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.881+0000 7fab30b2fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615656e1ad0 0x5615656dee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:47.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.881+0000 7fab30b2fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615655c8ac0 con 0x5615656d6be0 2023-07-18T03:11:47.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.881+0000 7fab30b2fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615655c87c0 con 0x5615656e1ad0 2023-07-18T03:11:47.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.881+0000 7fab30b2fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615654ac920 con 0x5615656d62d0 2023-07-18T03:11:47.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.881+0000 7fab30b2fb80 10 client.0 _ll_register_callbacks cb 0x5615655c6760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:11:47.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.881+0000 7fab16d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615656d62d0 0x5615656e1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:47.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.881+0000 7fab1659c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5615656d6be0 0x5615656de760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:47.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.881+0000 7fab16d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615656d62d0 0x5615656e1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:48882/0 (socket says 172.21.15.101:48882) 2023-07-18T03:11:47.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.881+0000 7fab16d9d700 1 -- 172.21.15.101:0/978496818 learned_addr learned my addr 172.21.15.101:0/978496818 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:47.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.881+0000 7fab1759e700 1 --2- 172.21.15.101:0/978496818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615656e1ad0 0x5615656dee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:47.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.882+0000 7fab16d9d700 1 -- 172.21.15.101:0/978496818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615656e1ad0 msgr2=0x5615656dee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:47.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.882+0000 7fab16d9d700 1 --2- 172.21.15.101:0/978496818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615656e1ad0 0x5615656dee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:47.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.882+0000 7fab16d9d700 1 -- 172.21.15.101:0/978496818 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5615656d6be0 msgr2=0x5615656de760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:47.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.882+0000 7fab16d9d700 1 --2- 172.21.15.101:0/978496818 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5615656d6be0 0x5615656de760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:47.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.882+0000 7fab16d9d700 1 -- 172.21.15.101:0/978496818 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab1000e410 con 0x5615656d62d0 2023-07-18T03:11:47.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.882+0000 7fab16d9d700 1 --2- 172.21.15.101:0/978496818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615656d62d0 0x5615656e1550 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fab0809ae00 tx=0x7fab080996b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db2c6077103645f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:47.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.882+0000 7faafaff5700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:11:47.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.882+0000 7faafaff5700 1 -- 172.21.15.101:0/978496818 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0809a7f0 con 0x5615656d62d0 2023-07-18T03:11:47.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.882+0000 7faafaff5700 1 -- 172.21.15.101:0/978496818 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fab080cbe40 con 0x5615656d62d0 2023-07-18T03:11:47.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.882+0000 7fab30b2fb80 1 -- 172.21.15.101:0/978496818 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5615655824f0 con 0x5615656d62d0 2023-07-18T03:11:47.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.882+0000 7faafaff5700 1 -- 172.21.15.101:0/978496818 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab0809ac40 con 0x5615656d62d0 2023-07-18T03:11:47.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.882+0000 7fab30b2fb80 10 client.26416 fetch_fsmap learned FSMap version 26 2023-07-18T03:11:47.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.882+0000 7fab30b2fb80 1 -- 172.21.15.101:0/978496818 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56156543fc60 con 0x5615656d62d0 2023-07-18T03:11:47.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.882+0000 7faafaff5700 1 -- 172.21.15.101:0/978496818 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5615655824f0 con 0x5615656d62d0 2023-07-18T03:11:47.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.883+0000 7faafaff5700 1 -- 172.21.15.101:0/978496818 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fab080a3030 con 0x5615656d62d0 2023-07-18T03:11:47.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.883+0000 7fab30b2fb80 10 client.26416 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:11:47.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.883+0000 7fab30b2fb80 10 client.26416 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:11:47.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.883+0000 7fab30b2fb80 20 client.26416 populate_metadata read hostname 'smithi101' 2023-07-18T03:11:47.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.883+0000 7fab30b2fb80 1 --2- 172.21.15.101:0/978496818 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5615656f0f60 0x5615656f3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:47.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.883+0000 7fab30b2fb80 4 client.26416 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:11:47.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.883+0000 7fab30b2fb80 1 -- 172.21.15.101:0/978496818 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561565621eb0 con 0x5615656f0f60 2023-07-18T03:11:47.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.883+0000 7fab1659c700 1 --2- 172.21.15.101:0/978496818 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5615656f0f60 0x5615656f3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:47.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.883+0000 7faafaff5700 10 client.26416 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:11:47.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.883+0000 7fab1659c700 1 --2- 172.21.15.101:0/978496818 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5615656f0f60 0x5615656f3340 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c15ac98dd9650239 server_cookie=30b574066ed392a4 in_seq=0 out_seq=0 2023-07-18T03:11:47.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.884+0000 7faafaff5700 1 -- 172.21.15.101:0/978496818 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561565621eb0 con 0x5615656f0f60 2023-07-18T03:11:47.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.884+0000 7faafaff5700 10 client.26416 handle_command_reply: tid=0 2023-07-18T03:11:47.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.884+0000 7fab30b2fb80 1 client.26416 shutdown 2023-07-18T03:11:47.882 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.884+0000 7fab30b2fb80 1 -- 172.21.15.101:0/978496818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615656d62d0 msgr2=0x5615656e1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:47.882 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.884+0000 7fab30b2fb80 1 --2- 172.21.15.101:0/978496818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615656d62d0 0x5615656e1550 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fab0809ae00 tx=0x7fab080996b0 comp rx=0 tx=0).stop 2023-07-18T03:11:47.882 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.884+0000 7fab30b2fb80 1 -- 172.21.15.101:0/978496818 shutdown_connections 2023-07-18T03:11:47.882 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.884+0000 7fab30b2fb80 1 --2- 172.21.15.101:0/978496818 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5615656f0f60 0x5615656f3340 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:47.882 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.884+0000 7fab30b2fb80 1 --2- 172.21.15.101:0/978496818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615656e1ad0 0x5615656dee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:47.882 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.884+0000 7fab30b2fb80 1 --2- 172.21.15.101:0/978496818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615656d62d0 0x5615656e1550 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:47.882 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.884+0000 7faafaff5700 0 client.26416 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:11:47.882 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.884+0000 7fab30b2fb80 1 --2- 172.21.15.101:0/978496818 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5615656d6be0 0x5615656de760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:47.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.884+0000 7fab30b2fb80 1 -- 172.21.15.101:0/978496818 >> 172.21.15.101:0/978496818 conn(0x5615656ccda0 msgr2=0x5615656d2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:47.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.885+0000 7fab30b2fb80 1 -- 172.21.15.101:0/978496818 shutdown_connections 2023-07-18T03:11:47.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.885+0000 7fab30b2fb80 1 -- 172.21.15.101:0/978496818 wait complete. 2023-07-18T03:11:47.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.885+0000 7fab30b2fb80 20 client.26416 trim_cache size 0 max 16384 2023-07-18T03:11:47.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.903+0000 7fab30b2fb80 1 Processor -- start 2023-07-18T03:11:47.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.903+0000 7fab30b2fb80 1 -- start start 2023-07-18T03:11:47.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.903+0000 7fab30b2fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615656d62d0 0x5615656de710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:47.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.903+0000 7fab30b2fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615656d6be0 0x5615656d91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:47.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.903+0000 7fab30b2fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5615656e1ad0 0x5615656e1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:47.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.904+0000 7fab30b2fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615656ceec0 con 0x5615656e1ad0 2023-07-18T03:11:47.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.904+0000 7fab30b2fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615656d7540 con 0x5615656d6be0 2023-07-18T03:11:47.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.904+0000 7fab30b2fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615656da320 con 0x5615656d62d0 2023-07-18T03:11:47.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.904+0000 7fab1659c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615656d6be0 0x5615656d91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:47.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.904+0000 7fab1659c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615656d6be0 0x5615656d91b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:51876/0 (socket says 172.21.15.101:51876) 2023-07-18T03:11:47.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.904+0000 7fab1659c700 1 -- 172.21.15.101:0/305946711 learned_addr learned my addr 172.21.15.101:0/305946711 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:47.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.904+0000 7fab16d9d700 1 --2- 172.21.15.101:0/305946711 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615656d62d0 0x5615656de710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:47.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.904+0000 7fab1759e700 1 --2- 172.21.15.101:0/305946711 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5615656e1ad0 0x5615656e1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:47.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.904+0000 7fab16d9d700 1 -- 172.21.15.101:0/305946711 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615656d6be0 msgr2=0x5615656d91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:47.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.904+0000 7fab16d9d700 1 --2- 172.21.15.101:0/305946711 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615656d6be0 0x5615656d91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:47.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.904+0000 7fab16d9d700 1 -- 172.21.15.101:0/305946711 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5615656e1ad0 msgr2=0x5615656e1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:47.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.904+0000 7fab16d9d700 1 --2- 172.21.15.101:0/305946711 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5615656e1ad0 0x5615656e1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:47.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.904+0000 7fab16d9d700 1 -- 172.21.15.101:0/305946711 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56156547d250 con 0x5615656d62d0 2023-07-18T03:11:47.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.904+0000 7fab1659c700 1 --2- 172.21.15.101:0/305946711 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615656d6be0 0x5615656d91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:11:47.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.904+0000 7fab16d9d700 1 --2- 172.21.15.101:0/305946711 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615656d62d0 0x5615656de710 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fab0800f740 tx=0x7fab08009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=777d5d5931038b2f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:47.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.905+0000 7faafdffb700 1 -- 172.21.15.101:0/305946711 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab080a3230 con 0x5615656d62d0 2023-07-18T03:11:47.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.905+0000 7faafdffb700 1 -- 172.21.15.101:0/305946711 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab0809ac40 con 0x5615656d62d0 2023-07-18T03:11:47.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.905+0000 7faafdffb700 1 -- 172.21.15.101:0/305946711 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0809b190 con 0x5615656d62d0 2023-07-18T03:11:47.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.905+0000 7fab30b2fb80 1 -- 172.21.15.101:0/305946711 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615656d62d0 msgr2=0x5615656de710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:47.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.905+0000 7fab30b2fb80 1 --2- 172.21.15.101:0/305946711 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615656d62d0 0x5615656de710 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fab0800f740 tx=0x7fab08009310 comp rx=0 tx=0).stop 2023-07-18T03:11:47.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.905+0000 7fab30b2fb80 1 -- 172.21.15.101:0/305946711 shutdown_connections 2023-07-18T03:11:47.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.905+0000 7fab30b2fb80 1 --2- 172.21.15.101:0/305946711 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5615656e1ad0 0x5615656e1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:47.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.905+0000 7fab30b2fb80 1 --2- 172.21.15.101:0/305946711 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615656d62d0 0x5615656de710 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:47.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.905+0000 7fab30b2fb80 1 --2- 172.21.15.101:0/305946711 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615656d6be0 0x5615656d91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:47.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.905+0000 7fab30b2fb80 1 -- 172.21.15.101:0/305946711 >> 172.21.15.101:0/305946711 conn(0x5615656ccda0 msgr2=0x5615656de000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:47.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.906+0000 7fab30b2fb80 1 -- 172.21.15.101:0/305946711 shutdown_connections 2023-07-18T03:11:47.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.906+0000 7fab30b2fb80 1 -- 172.21.15.101:0/305946711 wait complete. 2023-07-18T03:11:47.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.906+0000 7fab30b2fb80 1 -- start start 2023-07-18T03:11:47.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.906+0000 7fab30b2fb80 1 Processor -- start 2023-07-18T03:11:47.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.907+0000 7fab30b2fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615656d62d0 0x5615656e8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:47.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.907+0000 7fab30b2fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615656d6be0 0x5615656e6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:47.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.907+0000 7fab30b2fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5615656e1ad0 0x5615656e7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:47.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.907+0000 7fab30b2fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615656e77e0 con 0x5615656e1ad0 2023-07-18T03:11:47.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.907+0000 7fab30b2fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615656e7950 con 0x5615656d62d0 2023-07-18T03:11:47.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.907+0000 7fab30b2fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x561565744e00 con 0x5615656d6be0 2023-07-18T03:11:47.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.907+0000 7fab30b2fb80 10 client.0 _ll_register_callbacks cb 0x5615655c6760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:11:47.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.907+0000 7fab16d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615656d62d0 0x5615656e8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:47.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.907+0000 7fab16d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615656d62d0 0x5615656e8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:51892/0 (socket says 172.21.15.101:51892) 2023-07-18T03:11:47.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.907+0000 7fab16d9d700 1 -- 172.21.15.101:0/3352632588 learned_addr learned my addr 172.21.15.101:0/3352632588 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:47.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.907+0000 7fab1659c700 1 --2- 172.21.15.101:0/3352632588 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615656d6be0 0x5615656e6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:47.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.907+0000 7fab1759e700 1 --2- 172.21.15.101:0/3352632588 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5615656e1ad0 0x5615656e7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:47.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.907+0000 7fab16d9d700 1 -- 172.21.15.101:0/3352632588 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615656d6be0 msgr2=0x5615656e6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:47.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.907+0000 7fab16d9d700 1 --2- 172.21.15.101:0/3352632588 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615656d6be0 0x5615656e6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:47.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.907+0000 7fab16d9d700 1 -- 172.21.15.101:0/3352632588 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5615656e1ad0 msgr2=0x5615656e7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:47.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.907+0000 7fab16d9d700 1 --2- 172.21.15.101:0/3352632588 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5615656e1ad0 0x5615656e7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:47.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.907+0000 7fab16d9d700 1 -- 172.21.15.101:0/3352632588 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab0c009c70 con 0x5615656d62d0 2023-07-18T03:11:47.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.908+0000 7fab16d9d700 1 --2- 172.21.15.101:0/3352632588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615656d62d0 0x5615656e8630 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fab080cb7a0 tx=0x7fab0809a820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b314e799f6918566 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:47.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.908+0000 7faafaff5700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:11:47.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.908+0000 7faafaff5700 1 -- 172.21.15.101:0/3352632588 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab080ce130 con 0x5615656d62d0 2023-07-18T03:11:47.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.908+0000 7faafaff5700 1 -- 172.21.15.101:0/3352632588 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fab080cbe40 con 0x5615656d62d0 2023-07-18T03:11:47.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.908+0000 7fab30b2fb80 1 -- 172.21.15.101:0/3352632588 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fab1000f040 con 0x5615656d62d0 2023-07-18T03:11:47.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.908+0000 7faafaff5700 1 -- 172.21.15.101:0/3352632588 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab0809b100 con 0x5615656d62d0 2023-07-18T03:11:47.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.908+0000 7fab30b2fb80 10 client.26399 fetch_fsmap learned FSMap version 26 2023-07-18T03:11:47.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.908+0000 7fab30b2fb80 1 -- 172.21.15.101:0/3352632588 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56156543fc60 con 0x5615656d62d0 2023-07-18T03:11:47.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.908+0000 7faafaff5700 1 -- 172.21.15.101:0/3352632588 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab1000f040 con 0x5615656d62d0 2023-07-18T03:11:47.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.908+0000 7faafaff5700 1 -- 172.21.15.101:0/3352632588 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fab080970b0 con 0x5615656d62d0 2023-07-18T03:11:47.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.908+0000 7fab30b2fb80 10 client.26399 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:11:47.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.908+0000 7fab30b2fb80 10 client.26399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:11:47.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.908+0000 7fab30b2fb80 20 client.26399 populate_metadata read hostname 'smithi101' 2023-07-18T03:11:47.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.908+0000 7fab30b2fb80 1 --2- 172.21.15.101:0/3352632588 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x561565746060 0x561565748440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:47.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.909+0000 7fab30b2fb80 4 client.26399 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:11:47.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.909+0000 7fab30b2fb80 1 -- 172.21.15.101:0/3352632588 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5615655e4050 con 0x561565746060 2023-07-18T03:11:47.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.909+0000 7fab1659c700 1 --2- 172.21.15.101:0/3352632588 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x561565746060 0x561565748440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:47.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.909+0000 7fab1659c700 1 --2- 172.21.15.101:0/3352632588 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x561565746060 0x561565748440 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cd48e419a3d444e server_cookie=7f79193ad0ec470e in_seq=0 out_seq=0 2023-07-18T03:11:47.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.910+0000 7faafaff5700 10 client.26399 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:11:47.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.910+0000 7faafaff5700 1 -- 172.21.15.101:0/3352632588 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5615655e4050 con 0x561565746060 2023-07-18T03:11:47.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.910+0000 7faafaff5700 10 client.26399 handle_command_reply: tid=0 2023-07-18T03:11:47.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.910+0000 7fab30b2fb80 1 client.26399 shutdown 2023-07-18T03:11:47.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.910+0000 7fab30b2fb80 1 -- 172.21.15.101:0/3352632588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615656d62d0 msgr2=0x5615656e8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:47.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.910+0000 7fab30b2fb80 1 --2- 172.21.15.101:0/3352632588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615656d62d0 0x5615656e8630 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fab080cb7a0 tx=0x7fab0809a820 comp rx=0 tx=0).stop 2023-07-18T03:11:47.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.910+0000 7fab30b2fb80 1 -- 172.21.15.101:0/3352632588 shutdown_connections 2023-07-18T03:11:47.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.910+0000 7fab30b2fb80 1 --2- 172.21.15.101:0/3352632588 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x561565746060 0x561565748440 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:47.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.910+0000 7fab30b2fb80 1 --2- 172.21.15.101:0/3352632588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615656d62d0 0x5615656e8630 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:47.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.910+0000 7fab30b2fb80 1 --2- 172.21.15.101:0/3352632588 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5615656e1ad0 0x5615656e7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:47.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.910+0000 7faafaff5700 0 client.26399 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:11:47.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.910+0000 7fab30b2fb80 1 --2- 172.21.15.101:0/3352632588 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615656d6be0 0x5615656e6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:47.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.910+0000 7fab30b2fb80 1 -- 172.21.15.101:0/3352632588 >> 172.21.15.101:0/3352632588 conn(0x5615656ccda0 msgr2=0x5615656d2140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:47.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.911+0000 7fab30b2fb80 1 -- 172.21.15.101:0/3352632588 shutdown_connections 2023-07-18T03:11:47.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.911+0000 7fab30b2fb80 1 -- 172.21.15.101:0/3352632588 wait complete. 2023-07-18T03:11:47.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.911+0000 7fab30b2fb80 20 client.26399 trim_cache size 0 max 16384 2023-07-18T03:11:47.911 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:11:47.911 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "no active scrubs running", 2023-07-18T03:11:47.911 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": {} 2023-07-18T03:11:47.911 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:11:47.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.911+0000 7fab1d1ff700 1 -- 172.21.15.101:0/2284894704 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fab000a1340 msgr2=0x7fab000a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:47.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.911+0000 7fab1d1ff700 1 --2- 172.21.15.101:0/2284894704 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fab000a1340 0x7fab000a37f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fab10003fd0 tx=0x7fab10023000 comp rx=0 tx=0).stop 2023-07-18T03:11:47.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.911+0000 7fab1d1ff700 1 -- 172.21.15.101:0/2284894704 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab180ffad0 msgr2=0x7fab18197e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:47.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.911+0000 7fab1d1ff700 1 --2- 172.21.15.101:0/2284894704 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab180ffad0 0x7fab18197e30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fab08005910 tx=0x7fab08005720 comp rx=0 tx=0).stop 2023-07-18T03:11:47.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.911+0000 7fab1d1ff700 1 -- 172.21.15.101:0/2284894704 shutdown_connections 2023-07-18T03:11:47.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.911+0000 7fab1d1ff700 1 --2- 172.21.15.101:0/2284894704 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fab000a1340 0x7fab000a37f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:47.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.911+0000 7fab1d1ff700 1 --2- 172.21.15.101:0/2284894704 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab180ffad0 0x7fab18197e30 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:47.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.911+0000 7fab1d1ff700 1 --2- 172.21.15.101:0/2284894704 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fab18105ae0 0x7fab1819c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:47.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.911+0000 7fab1d1ff700 1 --2- 172.21.15.101:0/2284894704 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab18100480 0x7fab18198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:47.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.911+0000 7fab1d1ff700 1 -- 172.21.15.101:0/2284894704 >> 172.21.15.101:0/2284894704 conn(0x7fab180fb610 msgr2=0x7fab1810ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:47.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.912+0000 7fab1d1ff700 1 -- 172.21.15.101:0/2284894704 shutdown_connections 2023-07-18T03:11:47.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:47.912+0000 7fab1d1ff700 1 -- 172.21.15.101:0/2284894704 wait complete. 2023-07-18T03:11:47.920 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-18T03:11:47.921 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:11:48.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.140+0000 7fc4909ff700 1 -- 172.21.15.101:0/3322815756 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc48c0ffaf0 msgr2=0x7fc48c0fff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:48.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.140+0000 7fc4909ff700 1 --2- 172.21.15.101:0/3322815756 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc48c0ffaf0 0x7fc48c0fff60 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc480009990 tx=0x7fc480009ca0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.140+0000 7fc4909ff700 1 -- 172.21.15.101:0/3322815756 shutdown_connections 2023-07-18T03:11:48.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.140+0000 7fc4909ff700 1 --2- 172.21.15.101:0/3322815756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc48c105b00 0x7fc48c105ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.140+0000 7fc4909ff700 1 --2- 172.21.15.101:0/3322815756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc48c1004a0 0x7fc48c10c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.140+0000 7fc4909ff700 1 --2- 172.21.15.101:0/3322815756 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc48c0ffaf0 0x7fc48c0fff60 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.140+0000 7fc4909ff700 1 -- 172.21.15.101:0/3322815756 >> 172.21.15.101:0/3322815756 conn(0x7fc48c0fb630 msgr2=0x7fc48c0fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:48.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.140+0000 7fc4909ff700 1 -- 172.21.15.101:0/3322815756 shutdown_connections 2023-07-18T03:11:48.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.140+0000 7fc4909ff700 1 -- 172.21.15.101:0/3322815756 wait complete. 2023-07-18T03:11:48.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.141+0000 7fc4909ff700 1 Processor -- start 2023-07-18T03:11:48.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.142+0000 7fc4909ff700 1 -- start start 2023-07-18T03:11:48.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.142+0000 7fc4909ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc48c0ffaf0 0x7fc48c197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:48.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.142+0000 7fc4909ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc48c1004a0 0x7fc48c198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:48.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.142+0000 7fc4909ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc48c105b00 0x7fc48c19c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:48.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.142+0000 7fc4909ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48c19cda0 con 0x7fc48c0ffaf0 2023-07-18T03:11:48.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.142+0000 7fc4909ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48c19cf10 con 0x7fc48c105b00 2023-07-18T03:11:48.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.142+0000 7fc4909ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48c19d0d0 con 0x7fc48c1004a0 2023-07-18T03:11:48.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.142+0000 7fc489d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc48c1004a0 0x7fc48c198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:48.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.142+0000 7fc48a59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc48c0ffaf0 0x7fc48c197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:48.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.142+0000 7fc489d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc48c1004a0 0x7fc48c198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:48938/0 (socket says 172.21.15.101:48938) 2023-07-18T03:11:48.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.142+0000 7fc489d9b700 1 -- 172.21.15.101:0/3426212169 learned_addr learned my addr 172.21.15.101:0/3426212169 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:48.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.142+0000 7fc48ad9d700 1 --2- 172.21.15.101:0/3426212169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc48c105b00 0x7fc48c19c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:48.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.143+0000 7fc489d9b700 1 -- 172.21.15.101:0/3426212169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc48c105b00 msgr2=0x7fc48c19c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:48.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.143+0000 7fc489d9b700 1 --2- 172.21.15.101:0/3426212169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc48c105b00 0x7fc48c19c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.143+0000 7fc489d9b700 1 -- 172.21.15.101:0/3426212169 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc48c0ffaf0 msgr2=0x7fc48c197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:48.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.143+0000 7fc489d9b700 1 --2- 172.21.15.101:0/3426212169 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc48c0ffaf0 0x7fc48c197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.143+0000 7fc489d9b700 1 -- 172.21.15.101:0/3426212169 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4800095f0 con 0x7fc48c1004a0 2023-07-18T03:11:48.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.143+0000 7fc489d9b700 1 --2- 172.21.15.101:0/3426212169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc48c1004a0 0x7fc48c198300 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc480005910 tx=0x7fc480004710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:48.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.143+0000 7fc47b7fe700 1 -- 172.21.15.101:0/3426212169 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48001d070 con 0x7fc48c1004a0 2023-07-18T03:11:48.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.143+0000 7fc47b7fe700 1 -- 172.21.15.101:0/3426212169 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc480022470 con 0x7fc48c1004a0 2023-07-18T03:11:48.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.143+0000 7fc4909ff700 1 -- 172.21.15.101:0/3426212169 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc48c19d350 con 0x7fc48c1004a0 2023-07-18T03:11:48.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.143+0000 7fc47b7fe700 1 -- 172.21.15.101:0/3426212169 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc480017650 con 0x7fc48c1004a0 2023-07-18T03:11:48.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.143+0000 7fc4909ff700 1 -- 172.21.15.101:0/3426212169 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc48c1a4bf0 con 0x7fc48c1004a0 2023-07-18T03:11:48.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.144+0000 7fc4909ff700 1 -- 172.21.15.101:0/3426212169 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc48c067300 con 0x7fc48c1004a0 2023-07-18T03:11:48.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.146+0000 7fc47b7fe700 1 -- 172.21.15.101:0/3426212169 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fc4800225e0 con 0x7fc48c1004a0 2023-07-18T03:11:48.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.148+0000 7fc47b7fe700 1 --2- 172.21.15.101:0/3426212169 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc4740a1300 0x7fc4740a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:48.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.148+0000 7fc47b7fe700 1 -- 172.21.15.101:0/3426212169 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fc4800ce260 con 0x7fc48c1004a0 2023-07-18T03:11:48.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.148+0000 7fc48a59c700 1 --2- 172.21.15.101:0/3426212169 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc4740a1300 0x7fc4740a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:48.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.148+0000 7fc48a59c700 1 --2- 172.21.15.101:0/3426212169 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc4740a1300 0x7fc4740a37b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc47c009be0 tx=0x7fc47c009260 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:48.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.153+0000 7fc47b7fe700 1 -- 172.21.15.101:0/3426212169 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc480097350 con 0x7fc48c1004a0 2023-07-18T03:11:48.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.414+0000 7fc4909ff700 1 -- 172.21.15.101:0/3426212169 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc48c1a4d80 con 0x7fc48c1004a0 2023-07-18T03:11:48.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.415+0000 7fc47b7fe700 1 -- 172.21.15.101:0/3426212169 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10858 (secure 0 0 0) 0x7fc48009a970 con 0x7fc48c1004a0 2023-07-18T03:11:48.406 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:11:48.406 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/4192846769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":4192846769},{"type":"v1","addr":"172.21.15.101:6839","nonce":4192846769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6841/3824340175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6840","nonce":3824340175},{"type":"v1","addr":"172.21.15.101:6841","nonce":3824340175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:04:59.361185+0000","modified":"2023-07-18T03:11:46.116493+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.155:6839/2055462483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2055462483},{"type":"v1","addr":"172.21.15.155:6839","nonce":2055462483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:11:48.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.417+0000 7fc4909ff700 1 -- 172.21.15.101:0/3426212169 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc4740a1300 msgr2=0x7fc4740a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:48.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.417+0000 7fc4909ff700 1 --2- 172.21.15.101:0/3426212169 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc4740a1300 0x7fc4740a37b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc47c009be0 tx=0x7fc47c009260 comp rx=0 tx=0).stop 2023-07-18T03:11:48.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.418+0000 7fc4909ff700 1 -- 172.21.15.101:0/3426212169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc48c1004a0 msgr2=0x7fc48c198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:48.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.418+0000 7fc4909ff700 1 --2- 172.21.15.101:0/3426212169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc48c1004a0 0x7fc48c198300 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc480005910 tx=0x7fc480004710 comp rx=0 tx=0).stop 2023-07-18T03:11:48.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.418+0000 7fc4909ff700 1 -- 172.21.15.101:0/3426212169 shutdown_connections 2023-07-18T03:11:48.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.418+0000 7fc4909ff700 1 --2- 172.21.15.101:0/3426212169 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc4740a1300 0x7fc4740a37b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.418+0000 7fc4909ff700 1 --2- 172.21.15.101:0/3426212169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc48c105b00 0x7fc48c19c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.418+0000 7fc4909ff700 1 --2- 172.21.15.101:0/3426212169 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc48c0ffaf0 0x7fc48c197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.418+0000 7fc4909ff700 1 --2- 172.21.15.101:0/3426212169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc48c1004a0 0x7fc48c198300 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.418+0000 7fc4909ff700 1 -- 172.21.15.101:0/3426212169 >> 172.21.15.101:0/3426212169 conn(0x7fc48c0fb630 msgr2=0x7fc48c0fbc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:48.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.418+0000 7fc4909ff700 1 -- 172.21.15.101:0/3426212169 shutdown_connections 2023-07-18T03:11:48.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.418+0000 7fc4909ff700 1 -- 172.21.15.101:0/3426212169 wait complete. 2023-07-18T03:11:48.410 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 26 2023-07-18T03:11:48.429 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-18T03:11:48.575 INFO:tasks.workunit.client.0.smithi101.stdout: 1 304128 9.04 MB/sec execute 73 sec latency 23.646 ms 2023-07-18T03:11:48.642 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.651+0000 7f09171ff700 1 -- 172.21.15.101:0/730870810 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0910105ae0 msgr2=0x7f0910105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:48.643 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.651+0000 7f09171ff700 1 --2- 172.21.15.101:0/730870810 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0910105ae0 0x7f0910105eb0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f0904009990 tx=0x7f0904009ca0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.643 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.651+0000 7f09171ff700 1 -- 172.21.15.101:0/730870810 shutdown_connections 2023-07-18T03:11:48.643 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.651+0000 7f09171ff700 1 --2- 172.21.15.101:0/730870810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0910100480 0x7f091010c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.643 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.651+0000 7f09171ff700 1 --2- 172.21.15.101:0/730870810 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0910105ae0 0x7f0910105eb0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.643 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.651+0000 7f09171ff700 1 --2- 172.21.15.101:0/730870810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09100ffad0 0x7f09100fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.644 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.651+0000 7f09171ff700 1 -- 172.21.15.101:0/730870810 >> 172.21.15.101:0/730870810 conn(0x7f09100fb5f0 msgr2=0x7f09100fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:48.644 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.652+0000 7f09171ff700 1 -- 172.21.15.101:0/730870810 shutdown_connections 2023-07-18T03:11:48.644 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.652+0000 7f09171ff700 1 -- 172.21.15.101:0/730870810 wait complete. 2023-07-18T03:11:48.645 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.653+0000 7f09171ff700 1 Processor -- start 2023-07-18T03:11:48.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.653+0000 7f09171ff700 1 -- start start 2023-07-18T03:11:48.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.653+0000 7f09171ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09100ffad0 0x7f0910072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:48.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.653+0000 7f09171ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0910100480 0x7f0910072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:48.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.653+0000 7f09171ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0910105ae0 0x7f09101a3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:48.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.653+0000 7f09171ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0910073580 con 0x7f0910100480 2023-07-18T03:11:48.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.653+0000 7f09171ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09101a4300 con 0x7f0910105ae0 2023-07-18T03:11:48.647 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.653+0000 7f09171ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09101a4440 con 0x7f09100ffad0 2023-07-18T03:11:48.647 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.654+0000 7f090ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0910100480 0x7f0910072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:48.647 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.654+0000 7f0914f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09100ffad0 0x7f0910072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:48.647 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.654+0000 7f0914f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09100ffad0 0x7f0910072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:48946/0 (socket says 172.21.15.101:48946) 2023-07-18T03:11:48.647 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.654+0000 7f0914f9b700 1 -- 172.21.15.101:0/728776450 learned_addr learned my addr 172.21.15.101:0/728776450 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:48.647 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.654+0000 7f091579c700 1 --2- 172.21.15.101:0/728776450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0910105ae0 0x7f09101a3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:48.647 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.654+0000 7f091579c700 1 -- 172.21.15.101:0/728776450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09100ffad0 msgr2=0x7f0910072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:48.647 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.654+0000 7f091579c700 1 --2- 172.21.15.101:0/728776450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09100ffad0 0x7f0910072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.648 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.654+0000 7f091579c700 1 -- 172.21.15.101:0/728776450 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0910100480 msgr2=0x7f0910072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:48.648 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.654+0000 7f091579c700 1 --2- 172.21.15.101:0/728776450 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0910100480 0x7f0910072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.648 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.654+0000 7f091579c700 1 -- 172.21.15.101:0/728776450 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09040095f0 con 0x7f0910105ae0 2023-07-18T03:11:48.648 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.654+0000 7f090ffff700 1 --2- 172.21.15.101:0/728776450 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0910100480 0x7f0910072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:11:48.648 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.654+0000 7f091579c700 1 --2- 172.21.15.101:0/728776450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0910105ae0 0x7f09101a3dc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f090800efc0 tx=0x7f090800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:48.648 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.654+0000 7f090dffb700 1 -- 172.21.15.101:0/728776450 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0908018070 con 0x7f0910105ae0 2023-07-18T03:11:48.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.655+0000 7f09171ff700 1 -- 172.21.15.101:0/728776450 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09101a4640 con 0x7f0910105ae0 2023-07-18T03:11:48.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.655+0000 7f090dffb700 1 -- 172.21.15.101:0/728776450 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f090800f040 con 0x7f0910105ae0 2023-07-18T03:11:48.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.655+0000 7f090dffb700 1 -- 172.21.15.101:0/728776450 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090800cd40 con 0x7f0910105ae0 2023-07-18T03:11:48.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.655+0000 7f09171ff700 1 -- 172.21.15.101:0/728776450 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09101a4b20 con 0x7f0910105ae0 2023-07-18T03:11:48.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.658+0000 7f090dffb700 1 -- 172.21.15.101:0/728776450 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f0908003a00 con 0x7f0910105ae0 2023-07-18T03:11:48.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.659+0000 7f092ab99b80 1 Processor -- start 2023-07-18T03:11:48.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.659+0000 7f090dffb700 1 --2- 172.21.15.101:0/728776450 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f09000a12c0 0x7f09000a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:48.651 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.659+0000 7f092ab99b80 1 -- start start 2023-07-18T03:11:48.651 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.659+0000 7f090dffb700 1 -- 172.21.15.101:0/728776450 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f09080ccbc0 con 0x7f0910105ae0 2023-07-18T03:11:48.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.660+0000 7f090ffff700 1 --2- 172.21.15.101:0/728776450 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f09000a12c0 0x7f09000a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:48.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.660+0000 7f092ab99b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598de2b12d0 0x5598de2b16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:48.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.660+0000 7f092ab99b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5598de2b1be0 0x5598de2bc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:48.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.660+0000 7f092ab99b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598de2bcad0 0x5598de2beeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:48.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.660+0000 7f092ab99b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598de1b7cb0 con 0x5598de2b12d0 2023-07-18T03:11:48.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.660+0000 7f092ab99b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598de20aa40 con 0x5598de2bcad0 2023-07-18T03:11:48.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.660+0000 7f092ab99b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598de203ed0 con 0x5598de2b1be0 2023-07-18T03:11:48.654 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.660+0000 7f0914f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598de2b12d0 0x5598de2b16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:48.654 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.660+0000 7f0914f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598de2b12d0 0x5598de2b16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:42200/0 (socket says 172.21.15.101:42200) 2023-07-18T03:11:48.654 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.660+0000 7f0914f9b700 1 -- 172.21.15.101:0/1323908738 learned_addr learned my addr 172.21.15.101:0/1323908738 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:48.654 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.660+0000 7f090ffff700 1 --2- 172.21.15.101:0/1323908738 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5598de2b1be0 0x5598de2bc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:48.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.660+0000 7f091579c700 1 --2- 172.21.15.101:0/1323908738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598de2bcad0 0x5598de2beeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:48.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.660+0000 7f090ffff700 1 --2- 172.21.15.101:0/728776450 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f09000a12c0 0x7f09000a3770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0910074000 tx=0x7f08fc009260 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:48.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.661+0000 7f0914f9b700 1 -- 172.21.15.101:0/1323908738 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5598de2b1be0 msgr2=0x5598de2bc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:48.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.661+0000 7f0914f9b700 1 --2- 172.21.15.101:0/1323908738 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5598de2b1be0 0x5598de2bc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.661+0000 7f0914f9b700 1 -- 172.21.15.101:0/1323908738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598de2bcad0 msgr2=0x5598de2beeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:48.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.661+0000 7f0914f9b700 1 --2- 172.21.15.101:0/1323908738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598de2bcad0 0x5598de2beeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.661+0000 7f0914f9b700 1 -- 172.21.15.101:0/1323908738 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598de031de0 con 0x5598de2b12d0 2023-07-18T03:11:48.657 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.661+0000 7f0914f9b700 1 --2- 172.21.15.101:0/1323908738 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598de2b12d0 0x5598de2b16a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f090400b610 tx=0x7f090400b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edd36f06f0f2731c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:48.657 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.661+0000 7f08faffd700 1 -- 172.21.15.101:0/1323908738 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090401d070 con 0x5598de2b12d0 2023-07-18T03:11:48.657 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.661+0000 7f08faffd700 1 -- 172.21.15.101:0/1323908738 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f090401e040 con 0x5598de2b12d0 2023-07-18T03:11:48.658 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.661+0000 7f08faffd700 1 -- 172.21.15.101:0/1323908738 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0904004980 con 0x5598de2b12d0 2023-07-18T03:11:48.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.662+0000 7f092ab99b80 1 -- 172.21.15.101:0/1323908738 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598de2b12d0 msgr2=0x5598de2b16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:48.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.662+0000 7f092ab99b80 1 --2- 172.21.15.101:0/1323908738 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598de2b12d0 0x5598de2b16a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f090400b610 tx=0x7f090400b640 comp rx=0 tx=0).stop 2023-07-18T03:11:48.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.662+0000 7f092ab99b80 1 -- 172.21.15.101:0/1323908738 shutdown_connections 2023-07-18T03:11:48.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.662+0000 7f092ab99b80 1 --2- 172.21.15.101:0/1323908738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598de2bcad0 0x5598de2beeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.662+0000 7f092ab99b80 1 --2- 172.21.15.101:0/1323908738 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598de2b12d0 0x5598de2b16a0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.662+0000 7f092ab99b80 1 --2- 172.21.15.101:0/1323908738 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5598de2b1be0 0x5598de2bc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.662+0000 7f092ab99b80 1 -- 172.21.15.101:0/1323908738 >> 172.21.15.101:0/1323908738 conn(0x5598de2a7da0 msgr2=0x5598de2ad6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:48.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.663+0000 7f092ab99b80 1 -- 172.21.15.101:0/1323908738 shutdown_connections 2023-07-18T03:11:48.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.663+0000 7f092ab99b80 1 -- 172.21.15.101:0/1323908738 wait complete. 2023-07-18T03:11:48.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.663+0000 7f092ab99b80 1 -- start start 2023-07-18T03:11:48.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.663+0000 7f092ab99b80 1 Processor -- start 2023-07-18T03:11:48.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.664+0000 7f092ab99b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598de2b12d0 0x5598de2bc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:48.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.664+0000 7f092ab99b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5598de2b1be0 0x5598de2b96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:48.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.664+0000 7f092ab99b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598de2bcad0 0x5598de2b9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:48.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.664+0000 7f092ab99b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598de1a3ac0 con 0x5598de2b12d0 2023-07-18T03:11:48.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.664+0000 7f092ab99b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598de1a37c0 con 0x5598de2bcad0 2023-07-18T03:11:48.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.664+0000 7f092ab99b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598de087920 con 0x5598de2b1be0 2023-07-18T03:11:48.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.664+0000 7f092ab99b80 10 client.0 _ll_register_callbacks cb 0x5598de1a1760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:11:48.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.664+0000 7f090ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5598de2b1be0 0x5598de2b96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:48.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.664+0000 7f090ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5598de2b1be0 0x5598de2b96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:48960/0 (socket says 172.21.15.101:48960) 2023-07-18T03:11:48.668 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.664+0000 7f0914f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598de2b12d0 0x5598de2bc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:48.668 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.664+0000 7f090ffff700 1 -- 172.21.15.101:0/2475838953 learned_addr learned my addr 172.21.15.101:0/2475838953 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:48.668 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.664+0000 7f091579c700 1 --2- 172.21.15.101:0/2475838953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598de2bcad0 0x5598de2b9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:48.668 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.664+0000 7f090ffff700 1 -- 172.21.15.101:0/2475838953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598de2bcad0 msgr2=0x5598de2b9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:48.668 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.664+0000 7f090ffff700 1 --2- 172.21.15.101:0/2475838953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598de2bcad0 0x5598de2b9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.668 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.664+0000 7f090ffff700 1 -- 172.21.15.101:0/2475838953 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598de2b12d0 msgr2=0x5598de2bc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:48.668 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.664+0000 7f090ffff700 1 --2- 172.21.15.101:0/2475838953 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598de2b12d0 0x5598de2bc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.665+0000 7f090ffff700 1 -- 172.21.15.101:0/2475838953 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f091010d210 con 0x5598de2b1be0 2023-07-18T03:11:48.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.665+0000 7f090ffff700 1 --2- 172.21.15.101:0/2475838953 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5598de2b1be0 0x5598de2b96d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f08fc004690 tx=0x7f08fc013040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3eba438372665dc server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:48.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.665+0000 7f08effff700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:11:48.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.665+0000 7f08effff700 1 -- 172.21.15.101:0/2475838953 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc021070 con 0x5598de2b1be0 2023-07-18T03:11:48.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.665+0000 7f092ab99b80 1 -- 172.21.15.101:0/2475838953 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5598de15d4f0 con 0x5598de2b1be0 2023-07-18T03:11:48.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.665+0000 7f08effff700 1 -- 172.21.15.101:0/2475838953 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f08fc01c440 con 0x5598de2b1be0 2023-07-18T03:11:48.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.665+0000 7f08effff700 1 -- 172.21.15.101:0/2475838953 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08fc01c600 con 0x5598de2b1be0 2023-07-18T03:11:48.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.665+0000 7f092ab99b80 10 client.26440 fetch_fsmap learned FSMap version 26 2023-07-18T03:11:48.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.665+0000 7f092ab99b80 1 -- 172.21.15.101:0/2475838953 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5598de01ac60 con 0x5598de2b1be0 2023-07-18T03:11:48.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.666+0000 7f08effff700 1 -- 172.21.15.101:0/2475838953 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5598de15d4f0 con 0x5598de2b1be0 2023-07-18T03:11:48.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.666+0000 7f08effff700 1 -- 172.21.15.101:0/2475838953 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f08fc01b940 con 0x5598de2b1be0 2023-07-18T03:11:48.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.666+0000 7f092ab99b80 10 client.26440 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:11:48.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.666+0000 7f092ab99b80 10 client.26440 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:11:48.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.666+0000 7f092ab99b80 20 client.26440 populate_metadata read hostname 'smithi101' 2023-07-18T03:11:48.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.666+0000 7f092ab99b80 1 --2- 172.21.15.101:0/2475838953 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5598de2cc3f0 0x5598de2ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:48.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.666+0000 7f092ab99b80 4 client.26440 mds_command: new command op to 24541 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:11:48.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.666+0000 7f092ab99b80 1 -- 172.21.15.101:0/2475838953 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5598de1fceb0 con 0x5598de2cc3f0 2023-07-18T03:11:48.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.666+0000 7f0914f9b700 1 --2- 172.21.15.101:0/2475838953 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5598de2cc3f0 0x5598de2ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:48.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.667+0000 7f08effff700 10 client.26440 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:11:48.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.667+0000 7f0914f9b700 1 --2- 172.21.15.101:0/2475838953 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5598de2cc3f0 0x5598de2ce800 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6821e8e273faec08 server_cookie=7adac8ec7810e020 in_seq=0 out_seq=0 2023-07-18T03:11:48.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.667+0000 7f08effff700 1 -- 172.21.15.101:0/2475838953 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5598de1fceb0 con 0x5598de2cc3f0 2023-07-18T03:11:48.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.667+0000 7f08effff700 10 client.26440 handle_command_reply: tid=0 2023-07-18T03:11:48.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.667+0000 7f092ab99b80 1 client.26440 shutdown 2023-07-18T03:11:48.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.668+0000 7f092ab99b80 1 -- 172.21.15.101:0/2475838953 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5598de2b1be0 msgr2=0x5598de2b96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:48.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.668+0000 7f092ab99b80 1 --2- 172.21.15.101:0/2475838953 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5598de2b1be0 0x5598de2b96d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f08fc004690 tx=0x7f08fc013040 comp rx=0 tx=0).stop 2023-07-18T03:11:48.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.668+0000 7f092ab99b80 1 -- 172.21.15.101:0/2475838953 shutdown_connections 2023-07-18T03:11:48.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.668+0000 7f092ab99b80 1 --2- 172.21.15.101:0/2475838953 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5598de2cc3f0 0x5598de2ce800 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.668+0000 7f092ab99b80 1 --2- 172.21.15.101:0/2475838953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598de2bcad0 0x5598de2b9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.668+0000 7f08effff700 0 client.26440 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:11:48.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.668+0000 7f092ab99b80 1 --2- 172.21.15.101:0/2475838953 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598de2b12d0 0x5598de2bc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.668+0000 7f092ab99b80 1 --2- 172.21.15.101:0/2475838953 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5598de2b1be0 0x5598de2b96d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.668+0000 7f092ab99b80 1 -- 172.21.15.101:0/2475838953 >> 172.21.15.101:0/2475838953 conn(0x5598de2a7da0 msgr2=0x5598de2ad290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:48.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.668+0000 7f092ab99b80 1 -- 172.21.15.101:0/2475838953 shutdown_connections 2023-07-18T03:11:48.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.668+0000 7f092ab99b80 1 -- 172.21.15.101:0/2475838953 wait complete. 2023-07-18T03:11:48.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.668+0000 7f092ab99b80 20 client.26440 trim_cache size 0 max 16384 2023-07-18T03:11:48.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.682+0000 7f092ab99b80 1 Processor -- start 2023-07-18T03:11:48.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.683+0000 7f092ab99b80 1 -- start start 2023-07-18T03:11:48.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.683+0000 7f092ab99b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598de2b12d0 0x5598de2beeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:48.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.683+0000 7f092ab99b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5598de2b1be0 0x5598de2c7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:48.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.683+0000 7f092ab99b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598de2bcad0 0x5598de2bbff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:48.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.683+0000 7f092ab99b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598de2b7c30 con 0x5598de2bcad0 2023-07-18T03:11:48.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.683+0000 7f092ab99b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598de2cea20 con 0x5598de2b12d0 2023-07-18T03:11:48.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.683+0000 7f092ab99b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598de2cebe0 con 0x5598de2b1be0 2023-07-18T03:11:48.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.683+0000 7f0914f9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598de2b12d0 0x5598de2beeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:48.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.683+0000 7f0914f9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598de2b12d0 0x5598de2beeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:51924/0 (socket says 172.21.15.101:51924) 2023-07-18T03:11:48.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.683+0000 7f091579c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598de2bcad0 0x5598de2bbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:48.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.683+0000 7f0914f9b700 1 -- 172.21.15.101:0/1588571287 learned_addr learned my addr 172.21.15.101:0/1588571287 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:48.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.684+0000 7f090ffff700 1 --2- 172.21.15.101:0/1588571287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5598de2b1be0 0x5598de2c7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:48.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.684+0000 7f0914f9b700 1 -- 172.21.15.101:0/1588571287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5598de2b1be0 msgr2=0x5598de2c7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:48.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.684+0000 7f0914f9b700 1 --2- 172.21.15.101:0/1588571287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5598de2b1be0 0x5598de2c7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.684+0000 7f0914f9b700 1 -- 172.21.15.101:0/1588571287 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598de2bcad0 msgr2=0x5598de2bbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:48.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.684+0000 7f0914f9b700 1 --2- 172.21.15.101:0/1588571287 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598de2bcad0 0x5598de2bbff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.684+0000 7f0914f9b700 1 -- 172.21.15.101:0/1588571287 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598de058250 con 0x5598de2b12d0 2023-07-18T03:11:48.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.684+0000 7f0914f9b700 1 --2- 172.21.15.101:0/1588571287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598de2b12d0 0x5598de2beeb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0904004570 tx=0x7f0904009550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c5f3d774ffe4693 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:48.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.684+0000 7f08fb7fe700 1 -- 172.21.15.101:0/1588571287 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090401d040 con 0x5598de2b12d0 2023-07-18T03:11:48.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.684+0000 7f08fb7fe700 1 -- 172.21.15.101:0/1588571287 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0904033040 con 0x5598de2b12d0 2023-07-18T03:11:48.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.684+0000 7f08fb7fe700 1 -- 172.21.15.101:0/1588571287 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090400f460 con 0x5598de2b12d0 2023-07-18T03:11:48.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.685+0000 7f092ab99b80 1 -- 172.21.15.101:0/1588571287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598de2b12d0 msgr2=0x5598de2beeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:48.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.685+0000 7f092ab99b80 1 --2- 172.21.15.101:0/1588571287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598de2b12d0 0x5598de2beeb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0904004570 tx=0x7f0904009550 comp rx=0 tx=0).stop 2023-07-18T03:11:48.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.685+0000 7f092ab99b80 1 -- 172.21.15.101:0/1588571287 shutdown_connections 2023-07-18T03:11:48.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.685+0000 7f092ab99b80 1 --2- 172.21.15.101:0/1588571287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598de2b12d0 0x5598de2beeb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.685+0000 7f092ab99b80 1 --2- 172.21.15.101:0/1588571287 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598de2bcad0 0x5598de2bbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.685+0000 7f092ab99b80 1 --2- 172.21.15.101:0/1588571287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5598de2b1be0 0x5598de2c7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.685+0000 7f092ab99b80 1 -- 172.21.15.101:0/1588571287 >> 172.21.15.101:0/1588571287 conn(0x5598de2a7da0 msgr2=0x5598de2d3120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:48.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.685+0000 7f092ab99b80 1 -- 172.21.15.101:0/1588571287 shutdown_connections 2023-07-18T03:11:48.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.685+0000 7f092ab99b80 1 -- 172.21.15.101:0/1588571287 wait complete. 2023-07-18T03:11:48.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.686+0000 7f092ab99b80 1 -- start start 2023-07-18T03:11:48.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.686+0000 7f092ab99b80 1 Processor -- start 2023-07-18T03:11:48.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.686+0000 7f092ab99b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598de2b12d0 0x5598de2bf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:48.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.687+0000 7f092ab99b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598de2b1be0 0x5598de2bfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:48.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.687+0000 7f092ab99b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5598de2bcad0 0x5598de31f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:48.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.687+0000 7f092ab99b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598de31f7a0 con 0x5598de2b12d0 2023-07-18T03:11:48.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.687+0000 7f092ab99b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598de31f8e0 con 0x5598de2b1be0 2023-07-18T03:11:48.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.687+0000 7f092ab99b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598de31fa20 con 0x5598de2bcad0 2023-07-18T03:11:48.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.687+0000 7f092ab99b80 10 client.0 _ll_register_callbacks cb 0x7f08fc004480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:11:48.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.687+0000 7f0914f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598de2b12d0 0x5598de2bf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:48.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.687+0000 7f090ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598de2b1be0 0x5598de2bfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:48.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.687+0000 7f091579c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5598de2bcad0 0x5598de31f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:48.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.687+0000 7f0914f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598de2b12d0 0x5598de2bf670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:42230/0 (socket says 172.21.15.101:42230) 2023-07-18T03:11:48.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.687+0000 7f0914f9b700 1 -- 172.21.15.101:0/1882479094 learned_addr learned my addr 172.21.15.101:0/1882479094 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:48.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.687+0000 7f091579c700 1 -- 172.21.15.101:0/1882479094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598de2b1be0 msgr2=0x5598de2bfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:48.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.687+0000 7f091579c700 1 --2- 172.21.15.101:0/1882479094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598de2b1be0 0x5598de2bfbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.687+0000 7f091579c700 1 -- 172.21.15.101:0/1882479094 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598de2b12d0 msgr2=0x5598de2bf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:48.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.687+0000 7f091579c700 1 --2- 172.21.15.101:0/1882479094 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598de2b12d0 0x5598de2bf670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.687+0000 7f091579c700 1 -- 172.21.15.101:0/1882479094 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5598de031de0 con 0x5598de2bcad0 2023-07-18T03:11:48.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.687+0000 7f090ffff700 1 --2- 172.21.15.101:0/1882479094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598de2b1be0 0x5598de2bfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:11:48.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.687+0000 7f0914f9b700 1 --2- 172.21.15.101:0/1882479094 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598de2b12d0 0x5598de2bf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:11:48.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.687+0000 7f091579c700 1 --2- 172.21.15.101:0/1882479094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5598de2bcad0 0x5598de31f260 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f09080c9f60 tx=0x7f0908099580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fc65d40d110f2b0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:48.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.687+0000 7f08effff700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:11:48.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.688+0000 7f08effff700 1 -- 172.21.15.101:0/1882479094 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0908099970 con 0x5598de2bcad0 2023-07-18T03:11:48.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.688+0000 7f092ab99b80 1 -- 172.21.15.101:0/1882479094 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f090401e040 con 0x5598de2bcad0 2023-07-18T03:11:48.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.688+0000 7f08effff700 1 -- 172.21.15.101:0/1882479094 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f090809d9a0 con 0x5598de2bcad0 2023-07-18T03:11:48.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.688+0000 7f08effff700 1 -- 172.21.15.101:0/1882479094 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09080a1d30 con 0x5598de2bcad0 2023-07-18T03:11:48.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.688+0000 7f092ab99b80 10 client.26446 fetch_fsmap learned FSMap version 26 2023-07-18T03:11:48.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.688+0000 7f092ab99b80 1 -- 172.21.15.101:0/1882479094 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5598de0302b0 con 0x5598de2bcad0 2023-07-18T03:11:48.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.688+0000 7f08effff700 1 -- 172.21.15.101:0/1882479094 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f090401e040 con 0x5598de2bcad0 2023-07-18T03:11:48.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.688+0000 7f08effff700 1 -- 172.21.15.101:0/1882479094 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0908095760 con 0x5598de2bcad0 2023-07-18T03:11:48.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.688+0000 7f092ab99b80 10 client.26446 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:11:48.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.688+0000 7f092ab99b80 10 client.26446 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:11:48.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.688+0000 7f092ab99b80 20 client.26446 populate_metadata read hostname 'smithi101' 2023-07-18T03:11:48.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.688+0000 7f092ab99b80 1 --2- 172.21.15.101:0/1882479094 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5598de321140 0x5598de323550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:48.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.688+0000 7f092ab99b80 4 client.26446 mds_command: new command op to 24541 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:11:48.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.688+0000 7f092ab99b80 1 -- 172.21.15.101:0/1882479094 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5598de1bf050 con 0x5598de321140 2023-07-18T03:11:48.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.689+0000 7f0914f9b700 1 --2- 172.21.15.101:0/1882479094 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5598de321140 0x5598de323550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:48.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.689+0000 7f0914f9b700 1 --2- 172.21.15.101:0/1882479094 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5598de321140 0x5598de323550 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=48611fed1f6ad259 server_cookie=c22247f628bf06bb in_seq=0 out_seq=0 2023-07-18T03:11:48.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.689+0000 7f08effff700 10 client.26446 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:11:48.690 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.689+0000 7f08effff700 1 -- 172.21.15.101:0/1882479094 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5598de1bf050 con 0x5598de321140 2023-07-18T03:11:48.690 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.689+0000 7f08effff700 10 client.26446 handle_command_reply: tid=0 2023-07-18T03:11:48.690 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.689+0000 7f092ab99b80 1 client.26446 shutdown 2023-07-18T03:11:48.690 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.689+0000 7f092ab99b80 1 -- 172.21.15.101:0/1882479094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5598de2bcad0 msgr2=0x5598de31f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:48.690 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.689+0000 7f092ab99b80 1 --2- 172.21.15.101:0/1882479094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5598de2bcad0 0x5598de31f260 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f09080c9f60 tx=0x7f0908099580 comp rx=0 tx=0).stop 2023-07-18T03:11:48.690 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.690+0000 7f092ab99b80 1 -- 172.21.15.101:0/1882479094 shutdown_connections 2023-07-18T03:11:48.690 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.690+0000 7f092ab99b80 1 --2- 172.21.15.101:0/1882479094 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5598de321140 0x5598de323550 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.690+0000 7f092ab99b80 1 --2- 172.21.15.101:0/1882479094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5598de2bcad0 0x5598de31f260 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.690+0000 7f092ab99b80 1 --2- 172.21.15.101:0/1882479094 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598de2b12d0 0x5598de2bf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.690+0000 7f092ab99b80 1 --2- 172.21.15.101:0/1882479094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598de2b1be0 0x5598de2bfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.690+0000 7f08effff700 0 client.26446 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:11:48.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.690+0000 7f092ab99b80 1 -- 172.21.15.101:0/1882479094 >> 172.21.15.101:0/1882479094 conn(0x5598de2a7da0 msgr2=0x5598de2c7c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:48.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.690+0000 7f092ab99b80 1 -- 172.21.15.101:0/1882479094 shutdown_connections 2023-07-18T03:11:48.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.690+0000 7f092ab99b80 1 -- 172.21.15.101:0/1882479094 wait complete. 2023-07-18T03:11:48.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.690+0000 7f092ab99b80 20 client.26446 trim_cache size 0 max 16384 2023-07-18T03:11:48.692 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:11:48.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.690+0000 7f09171ff700 1 -- 172.21.15.101:0/728776450 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f09000a12c0 msgr2=0x7f09000a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:48.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.690+0000 7f09171ff700 1 --2- 172.21.15.101:0/728776450 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f09000a12c0 0x7f09000a3770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0910074000 tx=0x7f08fc009260 comp rx=0 tx=0).stop 2023-07-18T03:11:48.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.690+0000 7f09171ff700 1 -- 172.21.15.101:0/728776450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0910105ae0 msgr2=0x7f09101a3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:48.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.690+0000 7f09171ff700 1 --2- 172.21.15.101:0/728776450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0910105ae0 0x7f09101a3dc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f090800efc0 tx=0x7f090800c3a0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.691+0000 7f09171ff700 1 -- 172.21.15.101:0/728776450 shutdown_connections 2023-07-18T03:11:48.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.691+0000 7f09171ff700 1 --2- 172.21.15.101:0/728776450 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f09000a12c0 0x7f09000a3770 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.691+0000 7f09171ff700 1 --2- 172.21.15.101:0/728776450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0910105ae0 0x7f09101a3dc0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.691+0000 7f09171ff700 1 --2- 172.21.15.101:0/728776450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09100ffad0 0x7f0910072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.691+0000 7f09171ff700 1 --2- 172.21.15.101:0/728776450 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0910100480 0x7f0910072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.691+0000 7f09171ff700 1 -- 172.21.15.101:0/728776450 >> 172.21.15.101:0/728776450 conn(0x7f09100fb5f0 msgr2=0x7f09101035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:48.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.691+0000 7f09171ff700 1 -- 172.21.15.101:0/728776450 shutdown_connections 2023-07-18T03:11:48.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.691+0000 7f09171ff700 1 -- 172.21.15.101:0/728776450 wait complete. 2023-07-18T03:11:48.701 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-18T03:11:48.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.930+0000 7f2c4b7ff700 1 -- 172.21.15.101:0/4041765761 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c44105af0 msgr2=0x7f2c44105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:48.922 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.930+0000 7f2c4b7ff700 1 --2- 172.21.15.101:0/4041765761 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c44105af0 0x7f2c44105ec0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2c40009990 tx=0x7f2c40009ca0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.922 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.930+0000 7f2c4b7ff700 1 -- 172.21.15.101:0/4041765761 shutdown_connections 2023-07-18T03:11:48.922 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.930+0000 7f2c4b7ff700 1 --2- 172.21.15.101:0/4041765761 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2c44100490 0x7f2c4410c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.922 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.930+0000 7f2c4b7ff700 1 --2- 172.21.15.101:0/4041765761 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c44105af0 0x7f2c44105ec0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.922 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.930+0000 7f2c4b7ff700 1 --2- 172.21.15.101:0/4041765761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c440ffae0 0x7f2c440fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.922 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.930+0000 7f2c4b7ff700 1 -- 172.21.15.101:0/4041765761 >> 172.21.15.101:0/4041765761 conn(0x7f2c440fb5e0 msgr2=0x7f2c440fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:48.923 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.930+0000 7f2c4b7ff700 1 -- 172.21.15.101:0/4041765761 shutdown_connections 2023-07-18T03:11:48.923 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.930+0000 7f2c4b7ff700 1 -- 172.21.15.101:0/4041765761 wait complete. 2023-07-18T03:11:48.925 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.931+0000 7f2c4b7ff700 1 Processor -- start 2023-07-18T03:11:48.925 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.931+0000 7f2c4b7ff700 1 -- start start 2023-07-18T03:11:48.925 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.932+0000 7f2c4b7ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c440ffae0 0x7f2c44197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:48.925 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.932+0000 7f2c4b7ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2c44100490 0x7f2c44198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:48.925 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.932+0000 7f2c4b7ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c44105af0 0x7f2c4419c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:48.925 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.932+0000 7f2c4b7ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4419cd50 con 0x7f2c44100490 2023-07-18T03:11:48.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.932+0000 7f2c4b7ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4419cec0 con 0x7f2c440ffae0 2023-07-18T03:11:48.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.932+0000 7f2c4b7ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4419d080 con 0x7f2c44105af0 2023-07-18T03:11:48.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.932+0000 7f2c4959b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c440ffae0 0x7f2c44197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:48.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.932+0000 7f2c4959b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c440ffae0 0x7f2c44197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:51948/0 (socket says 172.21.15.101:51948) 2023-07-18T03:11:48.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.932+0000 7f2c4959b700 1 -- 172.21.15.101:0/21463666 learned_addr learned my addr 172.21.15.101:0/21463666 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:48.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.932+0000 7f2c48d9a700 1 --2- 172.21.15.101:0/21463666 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2c44100490 0x7f2c44198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:48.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.932+0000 7f2c49d9c700 1 --2- 172.21.15.101:0/21463666 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c44105af0 0x7f2c4419c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:48.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.932+0000 7f2c4959b700 1 -- 172.21.15.101:0/21463666 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c44105af0 msgr2=0x7f2c4419c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:48.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.932+0000 7f2c4959b700 1 --2- 172.21.15.101:0/21463666 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c44105af0 0x7f2c4419c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.932+0000 7f2c4959b700 1 -- 172.21.15.101:0/21463666 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2c44100490 msgr2=0x7f2c44198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:48.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.932+0000 7f2c4959b700 1 --2- 172.21.15.101:0/21463666 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2c44100490 0x7f2c44198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.932+0000 7f2c4959b700 1 -- 172.21.15.101:0/21463666 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c400095f0 con 0x7f2c440ffae0 2023-07-18T03:11:48.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.933+0000 7f2c4959b700 1 --2- 172.21.15.101:0/21463666 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c440ffae0 0x7f2c44197de0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2c40005910 tx=0x7f2c40005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:48.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.933+0000 7f2c3a7fc700 1 -- 172.21.15.101:0/21463666 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4001d070 con 0x7f2c440ffae0 2023-07-18T03:11:48.928 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.933+0000 7f2c3a7fc700 1 -- 172.21.15.101:0/21463666 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c40022470 con 0x7f2c440ffae0 2023-07-18T03:11:48.928 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.933+0000 7f2c4b7ff700 1 -- 172.21.15.101:0/21463666 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c4419d300 con 0x7f2c440ffae0 2023-07-18T03:11:48.928 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.933+0000 7f2c3a7fc700 1 -- 172.21.15.101:0/21463666 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c40017650 con 0x7f2c440ffae0 2023-07-18T03:11:48.928 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.933+0000 7f2c4b7ff700 1 -- 172.21.15.101:0/21463666 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c441a4ba0 con 0x7f2c440ffae0 2023-07-18T03:11:48.929 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.936+0000 7f2c3a7fc700 1 -- 172.21.15.101:0/21463666 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f2c400225e0 con 0x7f2c440ffae0 2023-07-18T03:11:48.929 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.938+0000 7f2c5f1adb80 1 Processor -- start 2023-07-18T03:11:48.929 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.938+0000 7f2c5f1adb80 1 -- start start 2023-07-18T03:11:48.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.938+0000 7f2c5f1adb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ff3db92d0 0x559ff3db96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:48.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.938+0000 7f2c5f1adb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ff3db9be0 0x559ff3dc4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:48.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.938+0000 7f2c5f1adb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559ff3dc4ad0 0x559ff3dc6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:48.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.938+0000 7f2c5f1adb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ff3cbfcb0 con 0x559ff3dc4ad0 2023-07-18T03:11:48.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.938+0000 7f2c5f1adb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ff3d12a40 con 0x559ff3db92d0 2023-07-18T03:11:48.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.938+0000 7f2c5f1adb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ff3d0bed0 con 0x559ff3db9be0 2023-07-18T03:11:48.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.939+0000 7f2c49d9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ff3db92d0 0x559ff3db96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:48.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.939+0000 7f2c49d9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ff3db92d0 0x559ff3db96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:51960/0 (socket says 172.21.15.101:51960) 2023-07-18T03:11:48.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.939+0000 7f2c4959b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ff3db9be0 0x559ff3dc4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:48.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.939+0000 7f2c49d9c700 1 -- 172.21.15.101:0/2391583975 learned_addr learned my addr 172.21.15.101:0/2391583975 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:48.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.939+0000 7f2c48d9a700 1 --2- 172.21.15.101:0/2391583975 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559ff3dc4ad0 0x559ff3dc6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:48.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.939+0000 7f2c3a7fc700 1 --2- 172.21.15.101:0/21463666 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2c300a1340 0x7f2c300a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:48.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.939+0000 7f2c49d9c700 1 -- 172.21.15.101:0/2391583975 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ff3db9be0 msgr2=0x559ff3dc4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:48.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.939+0000 7f2c49d9c700 1 --2- 172.21.15.101:0/2391583975 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ff3db9be0 0x559ff3dc4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.939+0000 7f2c49d9c700 1 -- 172.21.15.101:0/2391583975 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559ff3dc4ad0 msgr2=0x559ff3dc6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:48.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.939+0000 7f2c49d9c700 1 --2- 172.21.15.101:0/2391583975 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559ff3dc4ad0 0x559ff3dc6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.939+0000 7f2c49d9c700 1 -- 172.21.15.101:0/2391583975 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ff3b39de0 con 0x559ff3db92d0 2023-07-18T03:11:48.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.939+0000 7f2c3a7fc700 1 -- 172.21.15.101:0/21463666 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f2c400ce320 con 0x7f2c440ffae0 2023-07-18T03:11:48.934 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.939+0000 7f2c49d9c700 1 --2- 172.21.15.101:0/21463666 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2c300a1340 0x7f2c300a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:48.934 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.940+0000 7f2c49d9c700 1 --2- 172.21.15.101:0/2391583975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ff3db92d0 0x559ff3db96a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c011790 tx=0x7f2c3c011aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4be3b229742860f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:48.934 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.940+0000 7f2c2f7fe700 1 -- 172.21.15.101:0/2391583975 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c00e040 con 0x559ff3db92d0 2023-07-18T03:11:48.934 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.940+0000 7f2c2f7fe700 1 -- 172.21.15.101:0/2391583975 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c3c00f040 con 0x559ff3db92d0 2023-07-18T03:11:48.934 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.940+0000 7f2c2f7fe700 1 -- 172.21.15.101:0/2391583975 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c0156c0 con 0x559ff3db92d0 2023-07-18T03:11:48.934 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.940+0000 7f2c49d9c700 1 --2- 172.21.15.101:0/21463666 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2c300a1340 0x7f2c300a37f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c022000 tx=0x7f2c3c01f000 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:48.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.941+0000 7f2c5f1adb80 1 -- 172.21.15.101:0/2391583975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ff3db92d0 msgr2=0x559ff3db96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:48.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.941+0000 7f2c5f1adb80 1 --2- 172.21.15.101:0/2391583975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ff3db92d0 0x559ff3db96a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c011790 tx=0x7f2c3c011aa0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.941+0000 7f2c5f1adb80 1 -- 172.21.15.101:0/2391583975 shutdown_connections 2023-07-18T03:11:48.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.941+0000 7f2c5f1adb80 1 --2- 172.21.15.101:0/2391583975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ff3db92d0 0x559ff3db96a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.941+0000 7f2c5f1adb80 1 --2- 172.21.15.101:0/2391583975 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559ff3dc4ad0 0x559ff3dc6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.941+0000 7f2c5f1adb80 1 --2- 172.21.15.101:0/2391583975 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ff3db9be0 0x559ff3dc4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.941+0000 7f2c5f1adb80 1 -- 172.21.15.101:0/2391583975 >> 172.21.15.101:0/2391583975 conn(0x559ff3dafda0 msgr2=0x559ff3db56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:48.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.941+0000 7f2c5f1adb80 1 -- 172.21.15.101:0/2391583975 shutdown_connections 2023-07-18T03:11:48.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.941+0000 7f2c5f1adb80 1 -- 172.21.15.101:0/2391583975 wait complete. 2023-07-18T03:11:48.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.941+0000 7f2c5f1adb80 1 -- start start 2023-07-18T03:11:48.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.942+0000 7f2c5f1adb80 1 Processor -- start 2023-07-18T03:11:48.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.942+0000 7f2c5f1adb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559ff3db92d0 0x559ff3dc44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:48.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.942+0000 7f2c5f1adb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ff3db9be0 0x559ff3dc1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:48.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.942+0000 7f2c5f1adb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ff3dc4ad0 0x559ff3dc1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:48.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.942+0000 7f2c5f1adb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ff3cabac0 con 0x559ff3db92d0 2023-07-18T03:11:48.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.942+0000 7f2c5f1adb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ff3cab7c0 con 0x559ff3dc4ad0 2023-07-18T03:11:48.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.942+0000 7f2c5f1adb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ff3b8f920 con 0x559ff3db9be0 2023-07-18T03:11:48.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.942+0000 7f2c5f1adb80 10 client.0 _ll_register_callbacks cb 0x559ff3ca9760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:11:48.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.942+0000 7f2c48d9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ff3db9be0 0x559ff3dc1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:48.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.942+0000 7f2c4959b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559ff3db92d0 0x559ff3dc44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:48.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.942+0000 7f2c48d9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ff3db9be0 0x559ff3dc1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:49018/0 (socket says 172.21.15.101:49018) 2023-07-18T03:11:48.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.942+0000 7f2c49d9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ff3dc4ad0 0x559ff3dc1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:48.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.942+0000 7f2c48d9a700 1 -- 172.21.15.101:0/875913024 learned_addr learned my addr 172.21.15.101:0/875913024 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:48.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.943+0000 7f2c48d9a700 1 -- 172.21.15.101:0/875913024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ff3dc4ad0 msgr2=0x559ff3dc1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:48.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.943+0000 7f2c48d9a700 1 --2- 172.21.15.101:0/875913024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ff3dc4ad0 0x559ff3dc1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.943+0000 7f2c48d9a700 1 -- 172.21.15.101:0/875913024 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559ff3db92d0 msgr2=0x559ff3dc44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:48.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.943+0000 7f2c48d9a700 1 --2- 172.21.15.101:0/875913024 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559ff3db92d0 0x559ff3dc44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.943+0000 7f2c48d9a700 1 -- 172.21.15.101:0/875913024 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c3c011380 con 0x559ff3db9be0 2023-07-18T03:11:48.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.943+0000 7f2c48d9a700 1 --2- 172.21.15.101:0/875913024 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ff3db9be0 0x559ff3dc1770 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2c340117c0 tx=0x7f2c34011ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=550686316a8f7c9f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:48.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.943+0000 7f2c2cff9700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:11:48.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.943+0000 7f2c2cff9700 1 -- 172.21.15.101:0/875913024 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c34016070 con 0x559ff3db9be0 2023-07-18T03:11:48.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.943+0000 7f2c5f1adb80 1 -- 172.21.15.101:0/875913024 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559ff3c654f0 con 0x559ff3db9be0 2023-07-18T03:11:48.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.943+0000 7f2c2cff9700 1 -- 172.21.15.101:0/875913024 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f2c3400f750 con 0x559ff3db9be0 2023-07-18T03:11:48.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.943+0000 7f2c2cff9700 1 -- 172.21.15.101:0/875913024 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c3400f910 con 0x559ff3db9be0 2023-07-18T03:11:48.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.943+0000 7f2c5f1adb80 10 client.26458 fetch_fsmap learned FSMap version 26 2023-07-18T03:11:48.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.943+0000 7f2c5f1adb80 1 -- 172.21.15.101:0/875913024 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x559ff3b22c60 con 0x559ff3db9be0 2023-07-18T03:11:48.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.944+0000 7f2c2cff9700 1 -- 172.21.15.101:0/875913024 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559ff3c654f0 con 0x559ff3db9be0 2023-07-18T03:11:48.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.944+0000 7f2c2cff9700 1 -- 172.21.15.101:0/875913024 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2c3401ab80 con 0x559ff3db9be0 2023-07-18T03:11:48.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.944+0000 7f2c5f1adb80 10 client.26458 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:11:48.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.944+0000 7f2c5f1adb80 10 client.26458 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:11:48.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.944+0000 7f2c5f1adb80 20 client.26458 populate_metadata read hostname 'smithi101' 2023-07-18T03:11:48.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.944+0000 7f2c5f1adb80 1 --2- 172.21.15.101:0/875913024 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x559ff3dd4490 0x559ff3dd68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:48.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.944+0000 7f2c5f1adb80 4 client.26458 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:11:48.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.944+0000 7f2c5f1adb80 1 -- 172.21.15.101:0/875913024 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559ff3d04eb0 con 0x559ff3dd4490 2023-07-18T03:11:48.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.944+0000 7f2c4959b700 1 --2- 172.21.15.101:0/875913024 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x559ff3dd4490 0x559ff3dd68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:48.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.945+0000 7f2c2cff9700 10 client.26458 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:11:48.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.945+0000 7f2c4959b700 1 --2- 172.21.15.101:0/875913024 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x559ff3dd4490 0x559ff3dd68a0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=263811543d13a8c8 server_cookie=f3e13c79c111b590 in_seq=0 out_seq=0 2023-07-18T03:11:48.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.946+0000 7f2c2cff9700 1 -- 172.21.15.101:0/875913024 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559ff3d04eb0 con 0x559ff3dd4490 2023-07-18T03:11:48.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.946+0000 7f2c2cff9700 10 client.26458 handle_command_reply: tid=0 2023-07-18T03:11:48.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.946+0000 7f2c5f1adb80 1 client.26458 shutdown 2023-07-18T03:11:48.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.946+0000 7f2c5f1adb80 1 -- 172.21.15.101:0/875913024 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ff3db9be0 msgr2=0x559ff3dc1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:48.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.946+0000 7f2c5f1adb80 1 --2- 172.21.15.101:0/875913024 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ff3db9be0 0x559ff3dc1770 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2c340117c0 tx=0x7f2c34011ad0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.946+0000 7f2c5f1adb80 1 -- 172.21.15.101:0/875913024 shutdown_connections 2023-07-18T03:11:48.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.946+0000 7f2c5f1adb80 1 --2- 172.21.15.101:0/875913024 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x559ff3dd4490 0x559ff3dd68a0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.946+0000 7f2c2cff9700 0 client.26458 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:11:48.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.946+0000 7f2c5f1adb80 1 --2- 172.21.15.101:0/875913024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ff3dc4ad0 0x559ff3dc1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.946+0000 7f2c5f1adb80 1 --2- 172.21.15.101:0/875913024 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559ff3db92d0 0x559ff3dc44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.946+0000 7f2c5f1adb80 1 --2- 172.21.15.101:0/875913024 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ff3db9be0 0x559ff3dc1770 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.946+0000 7f2c5f1adb80 1 -- 172.21.15.101:0/875913024 >> 172.21.15.101:0/875913024 conn(0x559ff3dafda0 msgr2=0x559ff3db5230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:48.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.946+0000 7f2c5f1adb80 1 -- 172.21.15.101:0/875913024 shutdown_connections 2023-07-18T03:11:48.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.946+0000 7f2c5f1adb80 1 -- 172.21.15.101:0/875913024 wait complete. 2023-07-18T03:11:48.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.947+0000 7f2c5f1adb80 20 client.26458 trim_cache size 0 max 16384 2023-07-18T03:11:48.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.960+0000 7f2c5f1adb80 1 Processor -- start 2023-07-18T03:11:48.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.960+0000 7f2c5f1adb80 1 -- start start 2023-07-18T03:11:48.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.961+0000 7f2c5f1adb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ff3db92d0 0x559ff3dd68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:48.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.961+0000 7f2c5f1adb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ff3db9be0 0x559ff3dcb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:48.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.961+0000 7f2c5f1adb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559ff3dc4ad0 0x559ff3dcfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:48.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.961+0000 7f2c5f1adb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ff3dbb230 con 0x559ff3dc4ad0 2023-07-18T03:11:48.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.961+0000 7f2c5f1adb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ff3dbfd50 con 0x559ff3db92d0 2023-07-18T03:11:48.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.961+0000 7f2c5f1adb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ff3dc23a0 con 0x559ff3db9be0 2023-07-18T03:11:48.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.961+0000 7f2c48d9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ff3db9be0 0x559ff3dcb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:48.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.961+0000 7f2c49d9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559ff3dc4ad0 0x559ff3dcfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:48.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.961+0000 7f2c48d9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ff3db9be0 0x559ff3dcb520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:49032/0 (socket says 172.21.15.101:49032) 2023-07-18T03:11:48.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.961+0000 7f2c48d9a700 1 -- 172.21.15.101:0/1183430468 learned_addr learned my addr 172.21.15.101:0/1183430468 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:48.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.961+0000 7f2c4959b700 1 --2- 172.21.15.101:0/1183430468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ff3db92d0 0x559ff3dd68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:48.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.961+0000 7f2c48d9a700 1 -- 172.21.15.101:0/1183430468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ff3db92d0 msgr2=0x559ff3dd68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:48.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.961+0000 7f2c48d9a700 1 --2- 172.21.15.101:0/1183430468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ff3db92d0 0x559ff3dd68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.961+0000 7f2c48d9a700 1 -- 172.21.15.101:0/1183430468 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559ff3dc4ad0 msgr2=0x559ff3dcfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:48.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.961+0000 7f2c48d9a700 1 --2- 172.21.15.101:0/1183430468 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559ff3dc4ad0 0x559ff3dcfcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.961+0000 7f2c48d9a700 1 -- 172.21.15.101:0/1183430468 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ff3b60250 con 0x559ff3db9be0 2023-07-18T03:11:48.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.961+0000 7f2c4959b700 1 --2- 172.21.15.101:0/1183430468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ff3db92d0 0x559ff3dd68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:11:48.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.961+0000 7f2c48d9a700 1 --2- 172.21.15.101:0/1183430468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ff3db9be0 0x559ff3dcb520 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2c34005a80 tx=0x7f2c34012ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6062f742814cb05 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:48.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.962+0000 7f2c2ffff700 1 -- 172.21.15.101:0/1183430468 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c34016070 con 0x559ff3db9be0 2023-07-18T03:11:48.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.962+0000 7f2c2ffff700 1 -- 172.21.15.101:0/1183430468 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c34019c80 con 0x559ff3db9be0 2023-07-18T03:11:48.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.962+0000 7f2c2ffff700 1 -- 172.21.15.101:0/1183430468 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3400fee0 con 0x559ff3db9be0 2023-07-18T03:11:48.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.962+0000 7f2c5f1adb80 1 -- 172.21.15.101:0/1183430468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ff3db9be0 msgr2=0x559ff3dcb520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:48.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.962+0000 7f2c5f1adb80 1 --2- 172.21.15.101:0/1183430468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ff3db9be0 0x559ff3dcb520 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2c34005a80 tx=0x7f2c34012ee0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.962+0000 7f2c5f1adb80 1 -- 172.21.15.101:0/1183430468 shutdown_connections 2023-07-18T03:11:48.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.962+0000 7f2c5f1adb80 1 --2- 172.21.15.101:0/1183430468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ff3db92d0 0x559ff3dd68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.962+0000 7f2c5f1adb80 1 --2- 172.21.15.101:0/1183430468 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559ff3dc4ad0 0x559ff3dcfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.962+0000 7f2c5f1adb80 1 --2- 172.21.15.101:0/1183430468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ff3db9be0 0x559ff3dcb520 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.962+0000 7f2c5f1adb80 1 -- 172.21.15.101:0/1183430468 >> 172.21.15.101:0/1183430468 conn(0x559ff3dafda0 msgr2=0x559ff3ddb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:48.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.962+0000 7f2c5f1adb80 1 -- 172.21.15.101:0/1183430468 shutdown_connections 2023-07-18T03:11:48.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.962+0000 7f2c5f1adb80 1 -- 172.21.15.101:0/1183430468 wait complete. 2023-07-18T03:11:48.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.963+0000 7f2c5f1adb80 1 -- start start 2023-07-18T03:11:48.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.963+0000 7f2c5f1adb80 1 Processor -- start 2023-07-18T03:11:48.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.963+0000 7f2c5f1adb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559ff3db92d0 0x559ff3dc9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:48.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.963+0000 7f2c5f1adb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ff3db9be0 0x559ff3dc9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:48.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.964+0000 7f2c5f1adb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ff3dc4ad0 0x559ff3e27350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:48.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.964+0000 7f2c5f1adb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ff3dc9c90 con 0x559ff3db92d0 2023-07-18T03:11:48.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.964+0000 7f2c5f1adb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ff3e27890 con 0x559ff3dc4ad0 2023-07-18T03:11:48.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.964+0000 7f2c5f1adb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ff3e27a50 con 0x559ff3db9be0 2023-07-18T03:11:48.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.964+0000 7f2c5f1adb80 10 client.0 _ll_register_callbacks cb 0x7f2c34009e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:11:48.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.964+0000 7f2c48d9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ff3db9be0 0x559ff3dc9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:48.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.964+0000 7f2c4959b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559ff3db92d0 0x559ff3dc9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:48.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.964+0000 7f2c48d9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ff3db9be0 0x559ff3dc9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:49048/0 (socket says 172.21.15.101:49048) 2023-07-18T03:11:48.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.964+0000 7f2c48d9a700 1 -- 172.21.15.101:0/1148747044 learned_addr learned my addr 172.21.15.101:0/1148747044 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:48.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.964+0000 7f2c49d9c700 1 --2- 172.21.15.101:0/1148747044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ff3dc4ad0 0x559ff3e27350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:48.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.964+0000 7f2c48d9a700 1 -- 172.21.15.101:0/1148747044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ff3dc4ad0 msgr2=0x559ff3e27350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:48.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.964+0000 7f2c48d9a700 1 --2- 172.21.15.101:0/1148747044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ff3dc4ad0 0x559ff3e27350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.964+0000 7f2c48d9a700 1 -- 172.21.15.101:0/1148747044 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559ff3db92d0 msgr2=0x559ff3dc9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:48.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.964+0000 7f2c48d9a700 1 --2- 172.21.15.101:0/1148747044 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559ff3db92d0 0x559ff3dc9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.964+0000 7f2c48d9a700 1 -- 172.21.15.101:0/1148747044 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559ff3b22c60 con 0x559ff3db9be0 2023-07-18T03:11:48.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.964+0000 7f2c48d9a700 1 --2- 172.21.15.101:0/1148747044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ff3db9be0 0x559ff3dc9540 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2c34009990 tx=0x7f2c34009f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28db8cfb289bdec3 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:48.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.964+0000 7f2c2cff9700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:11:48.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.965+0000 7f2c2cff9700 1 -- 172.21.15.101:0/1148747044 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c34016040 con 0x559ff3db9be0 2023-07-18T03:11:48.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.965+0000 7f2c2cff9700 1 -- 172.21.15.101:0/1148747044 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f2c34000e40 con 0x559ff3db9be0 2023-07-18T03:11:48.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.965+0000 7f2c5f1adb80 1 -- 172.21.15.101:0/1148747044 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c3c00f040 con 0x559ff3db9be0 2023-07-18T03:11:48.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.965+0000 7f2c2cff9700 1 -- 172.21.15.101:0/1148747044 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c3400fb00 con 0x559ff3db9be0 2023-07-18T03:11:48.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.965+0000 7f2c5f1adb80 10 client.26464 fetch_fsmap learned FSMap version 26 2023-07-18T03:11:48.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.965+0000 7f2c5f1adb80 1 -- 172.21.15.101:0/1148747044 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x559ff3b382b0 con 0x559ff3db9be0 2023-07-18T03:11:48.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.965+0000 7f2c2cff9700 1 -- 172.21.15.101:0/1148747044 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c3c00f040 con 0x559ff3db9be0 2023-07-18T03:11:48.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.965+0000 7f2c2cff9700 1 -- 172.21.15.101:0/1148747044 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2c34003680 con 0x559ff3db9be0 2023-07-18T03:11:48.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.965+0000 7f2c5f1adb80 10 client.26464 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:11:48.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.965+0000 7f2c5f1adb80 10 client.26464 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:11:48.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.965+0000 7f2c5f1adb80 20 client.26464 populate_metadata read hostname 'smithi101' 2023-07-18T03:11:48.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.965+0000 7f2c5f1adb80 1 --2- 172.21.15.101:0/1148747044 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x559ff3dd4490 0x559ff3e2af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:48.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.965+0000 7f2c5f1adb80 4 client.26464 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:11:48.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.965+0000 7f2c5f1adb80 1 -- 172.21.15.101:0/1148747044 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559ff3cc7050 con 0x559ff3dd4490 2023-07-18T03:11:48.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.966+0000 7f2c4959b700 1 --2- 172.21.15.101:0/1148747044 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x559ff3dd4490 0x559ff3e2af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:48.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.966+0000 7f2c2cff9700 10 client.26464 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:11:48.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.966+0000 7f2c4959b700 1 --2- 172.21.15.101:0/1148747044 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x559ff3dd4490 0x559ff3e2af10 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cd383e3f9a1fdfd9 server_cookie=15fd54cc7d9f80d2 in_seq=0 out_seq=0 2023-07-18T03:11:48.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.966+0000 7f2c2cff9700 1 -- 172.21.15.101:0/1148747044 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559ff3cc7050 con 0x559ff3dd4490 2023-07-18T03:11:48.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.966+0000 7f2c2cff9700 10 client.26464 handle_command_reply: tid=0 2023-07-18T03:11:48.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.966+0000 7f2c5f1adb80 1 client.26464 shutdown 2023-07-18T03:11:48.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.966+0000 7f2c5f1adb80 1 -- 172.21.15.101:0/1148747044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ff3db9be0 msgr2=0x559ff3dc9540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:48.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.966+0000 7f2c5f1adb80 1 --2- 172.21.15.101:0/1148747044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ff3db9be0 0x559ff3dc9540 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2c34009990 tx=0x7f2c34009f00 comp rx=0 tx=0).stop 2023-07-18T03:11:48.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.966+0000 7f2c5f1adb80 1 -- 172.21.15.101:0/1148747044 shutdown_connections 2023-07-18T03:11:48.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.966+0000 7f2c5f1adb80 1 --2- 172.21.15.101:0/1148747044 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x559ff3dd4490 0x559ff3e2af10 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.966+0000 7f2c5f1adb80 1 --2- 172.21.15.101:0/1148747044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ff3dc4ad0 0x559ff3e27350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.966+0000 7f2c5f1adb80 1 --2- 172.21.15.101:0/1148747044 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559ff3db92d0 0x559ff3dc9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.966+0000 7f2c5f1adb80 1 --2- 172.21.15.101:0/1148747044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559ff3db9be0 0x559ff3dc9540 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.966+0000 7f2c2cff9700 0 client.26464 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:11:48.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.966+0000 7f2c5f1adb80 1 -- 172.21.15.101:0/1148747044 >> 172.21.15.101:0/1148747044 conn(0x559ff3dafda0 msgr2=0x559ff3dd06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:48.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.967+0000 7f2c5f1adb80 1 -- 172.21.15.101:0/1148747044 shutdown_connections 2023-07-18T03:11:48.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.967+0000 7f2c5f1adb80 1 -- 172.21.15.101:0/1148747044 wait complete. 2023-07-18T03:11:48.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.967+0000 7f2c5f1adb80 20 client.26464 trim_cache size 0 max 16384 2023-07-18T03:11:48.977 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:11:48.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.967+0000 7f2c4b7ff700 1 -- 172.21.15.101:0/21463666 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2c300a1340 msgr2=0x7f2c300a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:48.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.967+0000 7f2c4b7ff700 1 --2- 172.21.15.101:0/21463666 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2c300a1340 0x7f2c300a37f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c022000 tx=0x7f2c3c01f000 comp rx=0 tx=0).stop 2023-07-18T03:11:48.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.967+0000 7f2c4b7ff700 1 -- 172.21.15.101:0/21463666 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c440ffae0 msgr2=0x7f2c44197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:48.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.967+0000 7f2c4b7ff700 1 --2- 172.21.15.101:0/21463666 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c440ffae0 0x7f2c44197de0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2c40005910 tx=0x7f2c40005720 comp rx=0 tx=0).stop 2023-07-18T03:11:48.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.967+0000 7f2c4b7ff700 1 -- 172.21.15.101:0/21463666 shutdown_connections 2023-07-18T03:11:48.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.967+0000 7f2c4b7ff700 1 --2- 172.21.15.101:0/21463666 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2c300a1340 0x7f2c300a37f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.967+0000 7f2c4b7ff700 1 --2- 172.21.15.101:0/21463666 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c440ffae0 0x7f2c44197de0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.968+0000 7f2c4b7ff700 1 --2- 172.21.15.101:0/21463666 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c44105af0 0x7f2c4419c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.968+0000 7f2c4b7ff700 1 --2- 172.21.15.101:0/21463666 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2c44100490 0x7f2c44198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:48.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.968+0000 7f2c4b7ff700 1 -- 172.21.15.101:0/21463666 >> 172.21.15.101:0/21463666 conn(0x7f2c440fb5e0 msgr2=0x7f2c4410adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:48.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.968+0000 7f2c4b7ff700 1 -- 172.21.15.101:0/21463666 shutdown_connections 2023-07-18T03:11:48.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:48.968+0000 7f2c4b7ff700 1 -- 172.21.15.101:0/21463666 wait complete. 2023-07-18T03:11:48.980 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-18T03:11:49.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.232+0000 7efe957ff700 1 -- 172.21.15.101:0/1102618709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe900ffad0 msgr2=0x7efe900fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:49.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.232+0000 7efe957ff700 1 --2- 172.21.15.101:0/1102618709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe900ffad0 0x7efe900fff40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7efe78009990 tx=0x7efe78009ca0 comp rx=0 tx=0).stop 2023-07-18T03:11:49.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.232+0000 7efe957ff700 1 -- 172.21.15.101:0/1102618709 shutdown_connections 2023-07-18T03:11:49.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.232+0000 7efe957ff700 1 --2- 172.21.15.101:0/1102618709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe90100480 0x7efe9010c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:49.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.232+0000 7efe957ff700 1 --2- 172.21.15.101:0/1102618709 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efe90105ae0 0x7efe90105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:49.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.232+0000 7efe957ff700 1 --2- 172.21.15.101:0/1102618709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe900ffad0 0x7efe900fff40 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:49.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.232+0000 7efe957ff700 1 -- 172.21.15.101:0/1102618709 >> 172.21.15.101:0/1102618709 conn(0x7efe900fb610 msgr2=0x7efe900fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:49.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.232+0000 7efe957ff700 1 -- 172.21.15.101:0/1102618709 shutdown_connections 2023-07-18T03:11:49.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.232+0000 7efe957ff700 1 -- 172.21.15.101:0/1102618709 wait complete. 2023-07-18T03:11:49.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.233+0000 7efe957ff700 1 Processor -- start 2023-07-18T03:11:49.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.234+0000 7efe957ff700 1 -- start start 2023-07-18T03:11:49.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.234+0000 7efe957ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efe900ffad0 0x7efe90072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:49.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.234+0000 7efe957ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe90100480 0x7efe90072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:49.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.234+0000 7efe957ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe90105ae0 0x7efe901a3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:49.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.234+0000 7efe957ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe90073580 con 0x7efe900ffad0 2023-07-18T03:11:49.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.234+0000 7efe957ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe901a4350 con 0x7efe90100480 2023-07-18T03:11:49.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.234+0000 7efe957ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe901a4490 con 0x7efe90105ae0 2023-07-18T03:11:49.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.234+0000 7efe8effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efe900ffad0 0x7efe90072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:49.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.234+0000 7efe87fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe90100480 0x7efe90072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:49.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.234+0000 7efe87fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe90100480 0x7efe90072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:52000/0 (socket says 172.21.15.101:52000) 2023-07-18T03:11:49.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.234+0000 7efe87fff700 1 -- 172.21.15.101:0/678955661 learned_addr learned my addr 172.21.15.101:0/678955661 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:49.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.234+0000 7efe8effd700 1 --2- 172.21.15.101:0/678955661 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efe900ffad0 0x7efe90072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:42288/0 (socket says 172.21.15.101:42288) 2023-07-18T03:11:49.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.234+0000 7efe8f7fe700 1 --2- 172.21.15.101:0/678955661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe90105ae0 0x7efe901a3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:49.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.235+0000 7efe87fff700 1 -- 172.21.15.101:0/678955661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe90105ae0 msgr2=0x7efe901a3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:49.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.235+0000 7efe87fff700 1 --2- 172.21.15.101:0/678955661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe90105ae0 0x7efe901a3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:49.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.235+0000 7efe87fff700 1 -- 172.21.15.101:0/678955661 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efe900ffad0 msgr2=0x7efe90072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:49.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.235+0000 7efe87fff700 1 --2- 172.21.15.101:0/678955661 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efe900ffad0 0x7efe90072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:49.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.235+0000 7efe87fff700 1 -- 172.21.15.101:0/678955661 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe80009520 con 0x7efe90100480 2023-07-18T03:11:49.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.235+0000 7efe8effd700 1 --2- 172.21.15.101:0/678955661 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efe900ffad0 0x7efe90072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:11:49.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.235+0000 7efe8f7fe700 1 --2- 172.21.15.101:0/678955661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe90105ae0 0x7efe901a3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:11:49.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.235+0000 7efe87fff700 1 --2- 172.21.15.101:0/678955661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe90100480 0x7efe90072d50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7efe78005910 tx=0x7efe78004710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:49.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.235+0000 7efe8cff9700 1 -- 172.21.15.101:0/678955661 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7801d070 con 0x7efe90100480 2023-07-18T03:11:49.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.235+0000 7efe957ff700 1 -- 172.21.15.101:0/678955661 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe780095f0 con 0x7efe90100480 2023-07-18T03:11:49.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.235+0000 7efe8cff9700 1 -- 172.21.15.101:0/678955661 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe78022470 con 0x7efe90100480 2023-07-18T03:11:49.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.235+0000 7efe8cff9700 1 -- 172.21.15.101:0/678955661 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe78017650 con 0x7efe90100480 2023-07-18T03:11:49.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.235+0000 7efe957ff700 1 -- 172.21.15.101:0/678955661 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe901a4980 con 0x7efe90100480 2023-07-18T03:11:49.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.239+0000 7efe8cff9700 1 -- 172.21.15.101:0/678955661 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7efe780225e0 con 0x7efe90100480 2023-07-18T03:11:49.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.240+0000 7efea9168b80 1 Processor -- start 2023-07-18T03:11:49.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.240+0000 7efea9168b80 1 -- start start 2023-07-18T03:11:49.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.240+0000 7efea9168b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7bffa92d0 0x55f7bffa96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:49.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.240+0000 7efea9168b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7bffa9be0 0x55f7bffb4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:49.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.240+0000 7efea9168b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7bffb4ad0 0x55f7bffb6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:49.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.240+0000 7efea9168b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7bfeafcb0 con 0x55f7bffb4ad0 2023-07-18T03:11:49.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.240+0000 7efea9168b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7bff02a40 con 0x55f7bffa92d0 2023-07-18T03:11:49.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.240+0000 7efea9168b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7bfefbed0 con 0x55f7bffa9be0 2023-07-18T03:11:49.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.241+0000 7efe8effd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7bffa9be0 0x55f7bffb4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:49.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.241+0000 7efe8f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7bffa92d0 0x55f7bffa96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:49.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.241+0000 7efe8effd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7bffa9be0 0x55f7bffb4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:49070/0 (socket says 172.21.15.101:49070) 2023-07-18T03:11:49.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.241+0000 7efe8cff9700 1 --2- 172.21.15.101:0/678955661 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7efe7c0a1340 0x7efe7c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:49.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.241+0000 7efe8effd700 1 -- 172.21.15.101:0/2767951161 learned_addr learned my addr 172.21.15.101:0/2767951161 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:49.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.241+0000 7efe8f7fe700 1 --2- 172.21.15.101:0/2767951161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7bffa92d0 0x55f7bffa96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:52004/0 (socket says 172.21.15.101:52004) 2023-07-18T03:11:49.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.241+0000 7efe87fff700 1 --2- 172.21.15.101:0/2767951161 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7bffb4ad0 0x55f7bffb6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:49.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.241+0000 7efe8cff9700 1 -- 172.21.15.101:0/678955661 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7efe780ce250 con 0x7efe90100480 2023-07-18T03:11:49.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.241+0000 7efe8f7fe700 1 --2- 172.21.15.101:0/678955661 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7efe7c0a1340 0x7efe7c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:49.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.241+0000 7efe8effd700 1 -- 172.21.15.101:0/2767951161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7bffa92d0 msgr2=0x55f7bffa96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:49.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.241+0000 7efe8effd700 1 --2- 172.21.15.101:0/2767951161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7bffa92d0 0x55f7bffa96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:49.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.241+0000 7efe8effd700 1 -- 172.21.15.101:0/2767951161 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7bffb4ad0 msgr2=0x55f7bffb6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:49.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.241+0000 7efe8effd700 1 --2- 172.21.15.101:0/2767951161 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7bffb4ad0 0x55f7bffb6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:49.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.241+0000 7efe8effd700 1 -- 172.21.15.101:0/2767951161 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7bfd29de0 con 0x55f7bffa9be0 2023-07-18T03:11:49.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.241+0000 7efe8effd700 1 --2- 172.21.15.101:0/2767951161 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7bffa9be0 0x55f7bffb4590 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7efe800117e0 tx=0x7efe80011af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=106b005b001cfdcc server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:49.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.242+0000 7efe857fa700 1 -- 172.21.15.101:0/2767951161 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe80019410 con 0x55f7bffa9be0 2023-07-18T03:11:49.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.242+0000 7efe857fa700 1 -- 172.21.15.101:0/2767951161 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe80019a30 con 0x55f7bffa9be0 2023-07-18T03:11:49.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.242+0000 7efe8f7fe700 1 --2- 172.21.15.101:0/678955661 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7efe7c0a1340 0x7efe7c0a37f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efe880060b0 tx=0x7efe88004b40 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:49.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.242+0000 7efe857fa700 1 -- 172.21.15.101:0/2767951161 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe80018b20 con 0x55f7bffa9be0 2023-07-18T03:11:49.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.242+0000 7efea9168b80 1 -- 172.21.15.101:0/2767951161 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7bffa9be0 msgr2=0x55f7bffb4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:49.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.242+0000 7efea9168b80 1 --2- 172.21.15.101:0/2767951161 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7bffa9be0 0x55f7bffb4590 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7efe800117e0 tx=0x7efe80011af0 comp rx=0 tx=0).stop 2023-07-18T03:11:49.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.242+0000 7efea9168b80 1 -- 172.21.15.101:0/2767951161 shutdown_connections 2023-07-18T03:11:49.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.242+0000 7efea9168b80 1 --2- 172.21.15.101:0/2767951161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7bffa92d0 0x55f7bffa96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:49.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.242+0000 7efea9168b80 1 --2- 172.21.15.101:0/2767951161 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7bffb4ad0 0x55f7bffb6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:49.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.242+0000 7efea9168b80 1 --2- 172.21.15.101:0/2767951161 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7bffa9be0 0x55f7bffb4590 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:49.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.242+0000 7efea9168b80 1 -- 172.21.15.101:0/2767951161 >> 172.21.15.101:0/2767951161 conn(0x55f7bff9fda0 msgr2=0x55f7bffa56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:49.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.243+0000 7efea9168b80 1 -- 172.21.15.101:0/2767951161 shutdown_connections 2023-07-18T03:11:49.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.243+0000 7efea9168b80 1 -- 172.21.15.101:0/2767951161 wait complete. 2023-07-18T03:11:49.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.243+0000 7efea9168b80 1 -- start start 2023-07-18T03:11:49.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.243+0000 7efea9168b80 1 Processor -- start 2023-07-18T03:11:49.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.244+0000 7efea9168b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7bffa92d0 0x55f7bffb4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:49.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.244+0000 7efea9168b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7bffa9be0 0x55f7bffb1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:49.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.244+0000 7efea9168b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7bffb4ad0 0x55f7bffb1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:49.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.244+0000 7efea9168b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7bfe9bac0 con 0x55f7bffb4ad0 2023-07-18T03:11:49.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.244+0000 7efea9168b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7bfe9b7c0 con 0x55f7bffa9be0 2023-07-18T03:11:49.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.244+0000 7efea9168b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7bfd7f920 con 0x55f7bffa92d0 2023-07-18T03:11:49.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.244+0000 7efea9168b80 10 client.0 _ll_register_callbacks cb 0x55f7bfe99760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:11:49.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.244+0000 7efe8effd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7bffa92d0 0x55f7bffb4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:49.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.244+0000 7efe87fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7bffa9be0 0x55f7bffb1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:49.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.244+0000 7efe8effd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7bffa92d0 0x55f7bffb4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:49086/0 (socket says 172.21.15.101:49086) 2023-07-18T03:11:49.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.244+0000 7efe8effd700 1 -- 172.21.15.101:0/3255290459 learned_addr learned my addr 172.21.15.101:0/3255290459 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:49.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.244+0000 7efe8f7fe700 1 --2- 172.21.15.101:0/3255290459 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7bffb4ad0 0x55f7bffb1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:49.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.244+0000 7efe87fff700 1 -- 172.21.15.101:0/3255290459 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7bffa92d0 msgr2=0x55f7bffb4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:49.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.244+0000 7efe87fff700 1 --2- 172.21.15.101:0/3255290459 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7bffa92d0 0x55f7bffb4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:49.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.244+0000 7efe87fff700 1 -- 172.21.15.101:0/3255290459 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7bffb4ad0 msgr2=0x55f7bffb1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:49.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.244+0000 7efe87fff700 1 --2- 172.21.15.101:0/3255290459 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7bffb4ad0 0x55f7bffb1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:49.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.244+0000 7efe87fff700 1 -- 172.21.15.101:0/3255290459 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe80011440 con 0x55f7bffa9be0 2023-07-18T03:11:49.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.244+0000 7efe8f7fe700 1 --2- 172.21.15.101:0/3255290459 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7bffb4ad0 0x55f7bffb1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:11:49.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.245+0000 7efe87fff700 1 --2- 172.21.15.101:0/3255290459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7bffa9be0 0x55f7bffb1730 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efe780cb5f0 tx=0x7efe780669a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c236e8283caeeb44 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:49.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.245+0000 7efe767fc700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:11:49.243 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:49 smithi101 ceph-mon[126379]: pgmap v130: 129 pgs: 129 active+clean; 182 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 205 op/s 2023-07-18T03:11:49.243 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:49 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3426212169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:11:49.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.245+0000 7efe767fc700 1 -- 172.21.15.101:0/3255290459 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7809b460 con 0x55f7bffa9be0 2023-07-18T03:11:49.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.245+0000 7efe767fc700 1 -- 172.21.15.101:0/3255290459 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7efe7809b000 con 0x55f7bffa9be0 2023-07-18T03:11:49.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.245+0000 7efea9168b80 1 -- 172.21.15.101:0/3255290459 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f7bfe554f0 con 0x55f7bffa9be0 2023-07-18T03:11:49.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.245+0000 7efe767fc700 1 -- 172.21.15.101:0/3255290459 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe7809a960 con 0x55f7bffa9be0 2023-07-18T03:11:49.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.245+0000 7efea9168b80 10 client.26453 fetch_fsmap learned FSMap version 26 2023-07-18T03:11:49.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.245+0000 7efea9168b80 1 -- 172.21.15.101:0/3255290459 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55f7bfd12c60 con 0x55f7bffa9be0 2023-07-18T03:11:49.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.245+0000 7efe767fc700 1 -- 172.21.15.101:0/3255290459 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f7bfe554f0 con 0x55f7bffa9be0 2023-07-18T03:11:49.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.245+0000 7efe767fc700 1 -- 172.21.15.101:0/3255290459 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efe780a2f30 con 0x55f7bffa9be0 2023-07-18T03:11:49.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.245+0000 7efea9168b80 10 client.26453 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:11:49.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.245+0000 7efea9168b80 10 client.26453 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:11:49.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.246+0000 7efea9168b80 20 client.26453 populate_metadata read hostname 'smithi101' 2023-07-18T03:11:49.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.246+0000 7efea9168b80 1 --2- 172.21.15.101:0/3255290459 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f7bffc4040 0x55f7bffc6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:49.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.246+0000 7efea9168b80 4 client.26453 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:11:49.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.246+0000 7efea9168b80 1 -- 172.21.15.101:0/3255290459 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f7bfef4eb0 con 0x55f7bffc4040 2023-07-18T03:11:49.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.246+0000 7efe8effd700 1 --2- 172.21.15.101:0/3255290459 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f7bffc4040 0x55f7bffc6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:49.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.246+0000 7efe8effd700 1 --2- 172.21.15.101:0/3255290459 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f7bffc4040 0x55f7bffc6450 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5375abf35702ab18 server_cookie=e5e3f54e1a6f907e in_seq=0 out_seq=0 2023-07-18T03:11:49.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.246+0000 7efe767fc700 10 client.26453 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:11:49.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.247+0000 7efe767fc700 1 -- 172.21.15.101:0/3255290459 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f7bfef4eb0 con 0x55f7bffc4040 2023-07-18T03:11:49.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.247+0000 7efe767fc700 10 client.26453 handle_command_reply: tid=0 2023-07-18T03:11:49.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.247+0000 7efea9168b80 1 client.26453 shutdown 2023-07-18T03:11:49.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.247+0000 7efea9168b80 1 -- 172.21.15.101:0/3255290459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7bffa9be0 msgr2=0x55f7bffb1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:49.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.247+0000 7efea9168b80 1 --2- 172.21.15.101:0/3255290459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7bffa9be0 0x55f7bffb1730 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efe780cb5f0 tx=0x7efe780669a0 comp rx=0 tx=0).stop 2023-07-18T03:11:49.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.247+0000 7efea9168b80 1 -- 172.21.15.101:0/3255290459 shutdown_connections 2023-07-18T03:11:49.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.247+0000 7efea9168b80 1 --2- 172.21.15.101:0/3255290459 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f7bffc4040 0x55f7bffc6450 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:49.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.247+0000 7efea9168b80 1 --2- 172.21.15.101:0/3255290459 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7bffb4ad0 0x55f7bffb1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:49.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.247+0000 7efe767fc700 0 client.26453 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:11:49.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.247+0000 7efea9168b80 1 --2- 172.21.15.101:0/3255290459 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7bffa92d0 0x55f7bffb4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:49.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.247+0000 7efea9168b80 1 --2- 172.21.15.101:0/3255290459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7bffa9be0 0x55f7bffb1730 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:49.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.247+0000 7efea9168b80 1 -- 172.21.15.101:0/3255290459 >> 172.21.15.101:0/3255290459 conn(0x55f7bff9fda0 msgr2=0x55f7bffa5290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:49.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.248+0000 7efea9168b80 1 -- 172.21.15.101:0/3255290459 shutdown_connections 2023-07-18T03:11:49.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.248+0000 7efea9168b80 1 -- 172.21.15.101:0/3255290459 wait complete. 2023-07-18T03:11:49.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.248+0000 7efea9168b80 20 client.26453 trim_cache size 0 max 16384 2023-07-18T03:11:49.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.261+0000 7efea9168b80 1 Processor -- start 2023-07-18T03:11:49.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.261+0000 7efea9168b80 1 -- start start 2023-07-18T03:11:49.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.261+0000 7efea9168b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7bffa92d0 0x55f7bffb16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:49.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.261+0000 7efea9168b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7bffa9be0 0x55f7bffcadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:49.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.261+0000 7efea9168b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7bffb4ad0 0x55f7bffc0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:49.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.261+0000 7efea9168b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7bffa1ec0 con 0x55f7bffa9be0 2023-07-18T03:11:49.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.261+0000 7efea9168b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7bffb2470 con 0x55f7bffb4ad0 2023-07-18T03:11:49.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.261+0000 7efea9168b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7bffab1a0 con 0x55f7bffa92d0 2023-07-18T03:11:49.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.261+0000 7efe8effd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7bffa92d0 0x55f7bffb16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:49.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.262+0000 7efe8effd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7bffa92d0 0x55f7bffb16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:49102/0 (socket says 172.21.15.101:49102) 2023-07-18T03:11:49.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.262+0000 7efe8effd700 1 -- 172.21.15.101:0/3435763296 learned_addr learned my addr 172.21.15.101:0/3435763296 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:49.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.262+0000 7efe8f7fe700 1 --2- 172.21.15.101:0/3435763296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7bffb4ad0 0x55f7bffc0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:49.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.262+0000 7efe87fff700 1 --2- 172.21.15.101:0/3435763296 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7bffa9be0 0x55f7bffcadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:49.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.262+0000 7efe8effd700 1 -- 172.21.15.101:0/3435763296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7bffb4ad0 msgr2=0x55f7bffc0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:49.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.262+0000 7efe8effd700 1 --2- 172.21.15.101:0/3435763296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7bffb4ad0 0x55f7bffc0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:49.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.262+0000 7efe8effd700 1 -- 172.21.15.101:0/3435763296 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7bffa9be0 msgr2=0x55f7bffcadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:49.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.262+0000 7efe8effd700 1 --2- 172.21.15.101:0/3435763296 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7bffa9be0 0x55f7bffcadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:49.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.262+0000 7efe8effd700 1 -- 172.21.15.101:0/3435763296 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7bfd50250 con 0x55f7bffa92d0 2023-07-18T03:11:49.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.262+0000 7efe87fff700 1 --2- 172.21.15.101:0/3435763296 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7bffa9be0 0x55f7bffcadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:11:49.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.262+0000 7efe8effd700 1 --2- 172.21.15.101:0/3435763296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7bffa92d0 0x55f7bffb16f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efe80000c00 tx=0x7efe80005040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c649c93b1355d8f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:49.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.262+0000 7efe85ffb700 1 -- 172.21.15.101:0/3435763296 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe800183b0 con 0x55f7bffa92d0 2023-07-18T03:11:49.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.262+0000 7efe85ffb700 1 -- 172.21.15.101:0/3435763296 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe800050b0 con 0x55f7bffa92d0 2023-07-18T03:11:49.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.262+0000 7efe85ffb700 1 -- 172.21.15.101:0/3435763296 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe800198d0 con 0x55f7bffa92d0 2023-07-18T03:11:49.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.263+0000 7efea9168b80 1 -- 172.21.15.101:0/3435763296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7bffa92d0 msgr2=0x55f7bffb16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:49.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.263+0000 7efea9168b80 1 --2- 172.21.15.101:0/3435763296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7bffa92d0 0x55f7bffb16f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efe80000c00 tx=0x7efe80005040 comp rx=0 tx=0).stop 2023-07-18T03:11:49.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.263+0000 7efea9168b80 1 -- 172.21.15.101:0/3435763296 shutdown_connections 2023-07-18T03:11:49.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.263+0000 7efea9168b80 1 --2- 172.21.15.101:0/3435763296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7bffb4ad0 0x55f7bffc0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:49.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.263+0000 7efea9168b80 1 --2- 172.21.15.101:0/3435763296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7bffa92d0 0x55f7bffb16f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:49.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.263+0000 7efea9168b80 1 --2- 172.21.15.101:0/3435763296 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7bffa9be0 0x55f7bffcadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:49.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.263+0000 7efea9168b80 1 -- 172.21.15.101:0/3435763296 >> 172.21.15.101:0/3435763296 conn(0x55f7bff9fda0 msgr2=0x55f7bffa1310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:49.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.263+0000 7efea9168b80 1 -- 172.21.15.101:0/3435763296 shutdown_connections 2023-07-18T03:11:49.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.263+0000 7efea9168b80 1 -- 172.21.15.101:0/3435763296 wait complete. 2023-07-18T03:11:49.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.263+0000 7efea9168b80 1 -- start start 2023-07-18T03:11:49.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.264+0000 7efea9168b80 1 Processor -- start 2023-07-18T03:11:49.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.264+0000 7efea9168b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7bffa92d0 0x55f7bffbb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:49.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.264+0000 7efea9168b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7bffa9be0 0x55f7bffb7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:49.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.264+0000 7efea9168b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7bffb4ad0 0x55f7bffb7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:49.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.264+0000 7efea9168b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7bffb8240 con 0x55f7bffa9be0 2023-07-18T03:11:49.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.264+0000 7efea9168b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7bffb8380 con 0x55f7bffb4ad0 2023-07-18T03:11:49.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.264+0000 7efea9168b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7bffb8540 con 0x55f7bffa92d0 2023-07-18T03:11:49.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.264+0000 7efea9168b80 10 client.0 _ll_register_callbacks cb 0x7efe7809add0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:11:49.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.264+0000 7efe8effd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7bffa92d0 0x55f7bffbb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:49.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.264+0000 7efe87fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7bffa9be0 0x55f7bffb7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:49.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.264+0000 7efe8effd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7bffa92d0 0x55f7bffbb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:49106/0 (socket says 172.21.15.101:49106) 2023-07-18T03:11:49.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.264+0000 7efe8effd700 1 -- 172.21.15.101:0/1559754707 learned_addr learned my addr 172.21.15.101:0/1559754707 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:49.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.264+0000 7efe8f7fe700 1 --2- 172.21.15.101:0/1559754707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7bffb4ad0 0x55f7bffb7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:49.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.265+0000 7efe8effd700 1 -- 172.21.15.101:0/1559754707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7bffb4ad0 msgr2=0x55f7bffb7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:49.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.265+0000 7efe8effd700 1 --2- 172.21.15.101:0/1559754707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7bffb4ad0 0x55f7bffb7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:49.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.265+0000 7efe8effd700 1 -- 172.21.15.101:0/1559754707 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7bffa9be0 msgr2=0x55f7bffb7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:49.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.265+0000 7efe8effd700 1 --2- 172.21.15.101:0/1559754707 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7bffa9be0 0x55f7bffb7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:49.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.265+0000 7efe8effd700 1 -- 172.21.15.101:0/1559754707 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f7bfd29de0 con 0x55f7bffa92d0 2023-07-18T03:11:49.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.265+0000 7efe8effd700 1 --2- 172.21.15.101:0/1559754707 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7bffa92d0 0x55f7bffbb440 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7efe80009d40 tx=0x7efe80019490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c717b3f0ae0a51a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:49.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.265+0000 7efe767fc700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:11:49.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.265+0000 7efe767fc700 1 -- 172.21.15.101:0/1559754707 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe80005220 con 0x55f7bffa92d0 2023-07-18T03:11:49.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.265+0000 7efe767fc700 1 -- 172.21.15.101:0/1559754707 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7efe80005380 con 0x55f7bffa92d0 2023-07-18T03:11:49.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.265+0000 7efea9168b80 1 -- 172.21.15.101:0/1559754707 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efe80019a30 con 0x55f7bffa92d0 2023-07-18T03:11:49.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.265+0000 7efe767fc700 1 -- 172.21.15.101:0/1559754707 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe80005540 con 0x55f7bffa92d0 2023-07-18T03:11:49.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.265+0000 7efea9168b80 10 client.26482 fetch_fsmap learned FSMap version 26 2023-07-18T03:11:49.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.266+0000 7efea9168b80 1 -- 172.21.15.101:0/1559754707 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55f7bfd282b0 con 0x55f7bffa92d0 2023-07-18T03:11:49.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.266+0000 7efe767fc700 1 -- 172.21.15.101:0/1559754707 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe80019a30 con 0x55f7bffa92d0 2023-07-18T03:11:49.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.266+0000 7efe767fc700 1 -- 172.21.15.101:0/1559754707 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efe80035070 con 0x55f7bffa92d0 2023-07-18T03:11:49.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.266+0000 7efea9168b80 10 client.26482 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:11:49.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.266+0000 7efea9168b80 10 client.26482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:11:49.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.266+0000 7efea9168b80 20 client.26482 populate_metadata read hostname 'smithi101' 2023-07-18T03:11:49.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.266+0000 7efea9168b80 1 --2- 172.21.15.101:0/1559754707 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f7c00190d0 0x55f7c001b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:49.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.266+0000 7efea9168b80 4 client.26482 mds_command: new command op to 24562 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:11:49.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.266+0000 7efea9168b80 1 -- 172.21.15.101:0/1559754707 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f7bfeb7050 con 0x55f7c00190d0 2023-07-18T03:11:49.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.266+0000 7efe87fff700 1 --2- 172.21.15.101:0/1559754707 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f7c00190d0 0x55f7c001b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:49.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.267+0000 7efe767fc700 10 client.26482 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:11:49.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.267+0000 7efe87fff700 1 --2- 172.21.15.101:0/1559754707 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f7c00190d0 0x55f7c001b4b0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4301e73d581419d server_cookie=cf972a0edc3b3ac3 in_seq=0 out_seq=0 2023-07-18T03:11:49.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.267+0000 7efe767fc700 1 -- 172.21.15.101:0/1559754707 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f7bfeb7050 con 0x55f7c00190d0 2023-07-18T03:11:49.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.267+0000 7efe767fc700 10 client.26482 handle_command_reply: tid=0 2023-07-18T03:11:49.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.267+0000 7efea9168b80 1 client.26482 shutdown 2023-07-18T03:11:49.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.267+0000 7efea9168b80 1 -- 172.21.15.101:0/1559754707 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7bffa92d0 msgr2=0x55f7bffbb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:49.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.267+0000 7efea9168b80 1 --2- 172.21.15.101:0/1559754707 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7bffa92d0 0x55f7bffbb440 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7efe80009d40 tx=0x7efe80019490 comp rx=0 tx=0).stop 2023-07-18T03:11:49.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.267+0000 7efea9168b80 1 -- 172.21.15.101:0/1559754707 shutdown_connections 2023-07-18T03:11:49.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.267+0000 7efea9168b80 1 --2- 172.21.15.101:0/1559754707 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f7c00190d0 0x55f7c001b4b0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:49.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.267+0000 7efea9168b80 1 --2- 172.21.15.101:0/1559754707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7bffb4ad0 0x55f7bffb7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:49.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.267+0000 7efea9168b80 1 --2- 172.21.15.101:0/1559754707 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7bffa92d0 0x55f7bffbb440 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:49.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.267+0000 7efe767fc700 0 client.26482 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:11:49.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.267+0000 7efea9168b80 1 --2- 172.21.15.101:0/1559754707 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7bffa9be0 0x55f7bffb7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:49.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.267+0000 7efea9168b80 1 -- 172.21.15.101:0/1559754707 >> 172.21.15.101:0/1559754707 conn(0x55f7bff9fda0 msgr2=0x55f7bffaca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:49.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.267+0000 7efea9168b80 1 -- 172.21.15.101:0/1559754707 shutdown_connections 2023-07-18T03:11:49.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.267+0000 7efea9168b80 1 -- 172.21.15.101:0/1559754707 wait complete. 2023-07-18T03:11:49.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.267+0000 7efea9168b80 20 client.26482 trim_cache size 0 max 16384 2023-07-18T03:11:49.274 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:11:49.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.268+0000 7efe957ff700 1 -- 172.21.15.101:0/678955661 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7efe7c0a1340 msgr2=0x7efe7c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:49.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.268+0000 7efe957ff700 1 --2- 172.21.15.101:0/678955661 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7efe7c0a1340 0x7efe7c0a37f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efe880060b0 tx=0x7efe88004b40 comp rx=0 tx=0).stop 2023-07-18T03:11:49.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.268+0000 7efe957ff700 1 -- 172.21.15.101:0/678955661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe90100480 msgr2=0x7efe90072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:49.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.268+0000 7efe957ff700 1 --2- 172.21.15.101:0/678955661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe90100480 0x7efe90072d50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7efe78005910 tx=0x7efe78004710 comp rx=0 tx=0).stop 2023-07-18T03:11:49.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.268+0000 7efe957ff700 1 -- 172.21.15.101:0/678955661 shutdown_connections 2023-07-18T03:11:49.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.268+0000 7efe957ff700 1 --2- 172.21.15.101:0/678955661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe90105ae0 0x7efe901a3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:49.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.268+0000 7efe957ff700 1 --2- 172.21.15.101:0/678955661 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efe900ffad0 0x7efe90072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:49.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.268+0000 7efe957ff700 1 --2- 172.21.15.101:0/678955661 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7efe7c0a1340 0x7efe7c0a37f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:49.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.268+0000 7efe957ff700 1 --2- 172.21.15.101:0/678955661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe90100480 0x7efe90072d50 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:49.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.268+0000 7efe957ff700 1 -- 172.21.15.101:0/678955661 >> 172.21.15.101:0/678955661 conn(0x7efe900fb610 msgr2=0x7efe901035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:49.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.268+0000 7efe957ff700 1 -- 172.21.15.101:0/678955661 shutdown_connections 2023-07-18T03:11:49.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:49.268+0000 7efe957ff700 1 -- 172.21.15.101:0/678955661 wait complete. 2023-07-18T03:11:49.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:49 smithi179 ceph-mon[129229]: pgmap v130: 129 pgs: 129 active+clean; 182 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 205 op/s 2023-07-18T03:11:49.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:49 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/3426212169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:11:49.543 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:49 smithi155 ceph-mon[128885]: pgmap v130: 129 pgs: 129 active+clean; 182 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 205 op/s 2023-07-18T03:11:49.543 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:49 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/3426212169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:11:49.575 INFO:tasks.workunit.client.0.smithi101.stdout: 1 305443 9.07 MB/sec execute 74 sec latency 87.239 ms 2023-07-18T03:11:50.284 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-18T03:11:50.284 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-18T03:11:50.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.506+0000 7f2cef7ff700 1 -- 172.21.15.101:0/3800244048 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ce80731c0 msgr2=0x7f2ce810bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:50.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.506+0000 7f2cef7ff700 1 --2- 172.21.15.101:0/3800244048 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ce80731c0 0x7f2ce810bdd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8009990 tx=0x7f2cd8009ca0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.506+0000 7f2cef7ff700 1 -- 172.21.15.101:0/3800244048 shutdown_connections 2023-07-18T03:11:50.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.506+0000 7f2cef7ff700 1 --2- 172.21.15.101:0/3800244048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ce810c310 0x7f2ce810e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.506+0000 7f2cef7ff700 1 --2- 172.21.15.101:0/3800244048 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2ce80728b0 0x7f2ce8072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.506+0000 7f2cef7ff700 1 --2- 172.21.15.101:0/3800244048 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ce80731c0 0x7f2ce810bdd0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.506+0000 7f2cef7ff700 1 -- 172.21.15.101:0/3800244048 >> 172.21.15.101:0/3800244048 conn(0x7f2ce80fb5f0 msgr2=0x7f2ce80fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:50.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.506+0000 7f2cef7ff700 1 -- 172.21.15.101:0/3800244048 shutdown_connections 2023-07-18T03:11:50.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.506+0000 7f2cef7ff700 1 -- 172.21.15.101:0/3800244048 wait complete. 2023-07-18T03:11:50.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.507+0000 7f2cef7ff700 1 Processor -- start 2023-07-18T03:11:50.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.507+0000 7f2cef7ff700 1 -- start start 2023-07-18T03:11:50.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.508+0000 7f2cef7ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ce80728b0 0x7f2ce8197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:50.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.508+0000 7f2cef7ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ce80731c0 0x7f2ce8198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:50.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.508+0000 7f2cef7ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2ce810c310 0x7f2ce819c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:50.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.508+0000 7f2cef7ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce819cdf0 con 0x7f2ce810c310 2023-07-18T03:11:50.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.508+0000 7f2cef7ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce819cf60 con 0x7f2ce80728b0 2023-07-18T03:11:50.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.508+0000 7f2cef7ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce819d120 con 0x7f2ce80731c0 2023-07-18T03:11:50.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.508+0000 7f2ced59b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ce80728b0 0x7f2ce8197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:50.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.508+0000 7f2cecd9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ce80731c0 0x7f2ce8198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:50.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.508+0000 7f2ced59b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ce80728b0 0x7f2ce8197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:52050/0 (socket says 172.21.15.101:52050) 2023-07-18T03:11:50.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.508+0000 7f2cedd9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2ce810c310 0x7f2ce819c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:50.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.508+0000 7f2ced59b700 1 -- 172.21.15.101:0/470539790 learned_addr learned my addr 172.21.15.101:0/470539790 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:50.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.508+0000 7f2cecd9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ce80731c0 0x7f2ce8198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:49136/0 (socket says 172.21.15.101:49136) 2023-07-18T03:11:50.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.508+0000 7f2ced59b700 1 -- 172.21.15.101:0/470539790 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ce80731c0 msgr2=0x7f2ce8198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:50.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.508+0000 7f2ced59b700 1 --2- 172.21.15.101:0/470539790 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ce80731c0 0x7f2ce8198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.508+0000 7f2ced59b700 1 -- 172.21.15.101:0/470539790 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2ce810c310 msgr2=0x7f2ce819c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:50.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.508+0000 7f2ced59b700 1 --2- 172.21.15.101:0/470539790 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2ce810c310 0x7f2ce819c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.508+0000 7f2ced59b700 1 -- 172.21.15.101:0/470539790 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd80095f0 con 0x7f2ce80728b0 2023-07-18T03:11:50.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.508+0000 7f2cedd9c700 1 --2- 172.21.15.101:0/470539790 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2ce810c310 0x7f2ce819c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:11:50.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.509+0000 7f2ced59b700 1 --2- 172.21.15.101:0/470539790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ce80728b0 0x7f2ce8197e10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2ce400e9a0 tx=0x7f2ce400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:50.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.509+0000 7f2cde7fc700 1 -- 172.21.15.101:0/470539790 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce4009790 con 0x7f2ce80728b0 2023-07-18T03:11:50.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.509+0000 7f2cde7fc700 1 -- 172.21.15.101:0/470539790 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ce4010430 con 0x7f2ce80728b0 2023-07-18T03:11:50.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.509+0000 7f2cef7ff700 1 -- 172.21.15.101:0/470539790 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ce819d400 con 0x7f2ce80728b0 2023-07-18T03:11:50.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.509+0000 7f2cde7fc700 1 -- 172.21.15.101:0/470539790 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce4018680 con 0x7f2ce80728b0 2023-07-18T03:11:50.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.510+0000 7f2cef7ff700 1 -- 172.21.15.101:0/470539790 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce81a4c40 con 0x7f2ce80728b0 2023-07-18T03:11:50.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.513+0000 7f2cde7fc700 1 -- 172.21.15.101:0/470539790 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f2ce40105a0 con 0x7f2ce80728b0 2023-07-18T03:11:50.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.514+0000 7f2d030d7b80 1 Processor -- start 2023-07-18T03:11:50.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.515+0000 7f2d030d7b80 1 -- start start 2023-07-18T03:11:50.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.515+0000 7f2cde7fc700 1 --2- 172.21.15.101:0/470539790 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2cd40a1340 0x7f2cd40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:50.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.515+0000 7f2d030d7b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593ae035350 0x5593ae035720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:50.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.515+0000 7f2d030d7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5593ae035c60 0x5593ae040610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:50.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.515+0000 7f2d030d7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593ae040b50 0x5593ae042f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:50.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.515+0000 7f2d030d7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593adf3bd30 con 0x5593ae040b50 2023-07-18T03:11:50.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.515+0000 7f2d030d7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593adf8eac0 con 0x5593ae035c60 2023-07-18T03:11:50.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.515+0000 7f2d030d7b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593adf87f50 con 0x5593ae035350 2023-07-18T03:11:50.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.515+0000 7f2cde7fc700 1 -- 172.21.15.101:0/470539790 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f2ce4014070 con 0x7f2ce80728b0 2023-07-18T03:11:50.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.516+0000 7f2ced59b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593ae035350 0x5593ae035720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:50.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.516+0000 7f2ced59b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593ae035350 0x5593ae035720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:49138/0 (socket says 172.21.15.101:49138) 2023-07-18T03:11:50.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.516+0000 7f2cecd9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5593ae035c60 0x5593ae040610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:50.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.516+0000 7f2ced59b700 1 -- 172.21.15.101:0/316562059 learned_addr learned my addr 172.21.15.101:0/316562059 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:50.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.516+0000 7f2cedd9c700 1 --2- 172.21.15.101:0/470539790 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2cd40a1340 0x7f2cd40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:50.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.516+0000 7f2cedd9c700 1 --2- 172.21.15.101:0/316562059 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593ae040b50 0x5593ae042f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:50.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.516+0000 7f2cecd9a700 1 -- 172.21.15.101:0/316562059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593ae035350 msgr2=0x5593ae035720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:50.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.516+0000 7f2cecd9a700 1 --2- 172.21.15.101:0/316562059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593ae035350 0x5593ae035720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.516+0000 7f2cecd9a700 1 -- 172.21.15.101:0/316562059 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593ae040b50 msgr2=0x5593ae042f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:50.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.516+0000 7f2cecd9a700 1 --2- 172.21.15.101:0/316562059 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593ae040b50 0x5593ae042f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.516+0000 7f2cecd9a700 1 -- 172.21.15.101:0/316562059 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593addb5e60 con 0x5593ae035c60 2023-07-18T03:11:50.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.516+0000 7f2cedd9c700 1 --2- 172.21.15.101:0/316562059 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593ae040b50 0x5593ae042f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:11:50.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.516+0000 7f2cecd9a700 1 --2- 172.21.15.101:0/316562059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5593ae035c60 0x5593ae040610 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2cd800b2e0 tx=0x7f2cd800b610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2da900b72e388c08 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:50.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.517+0000 7f2cedd9c700 1 --2- 172.21.15.101:0/470539790 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2cd40a1340 0x7f2cd40a37f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0009550 tx=0x7f2ce000b040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:50.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.516+0000 7f2cd37fe700 1 -- 172.21.15.101:0/316562059 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd801d070 con 0x5593ae035c60 2023-07-18T03:11:50.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.517+0000 7f2cd37fe700 1 -- 172.21.15.101:0/316562059 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2cd801e040 con 0x5593ae035c60 2023-07-18T03:11:50.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.517+0000 7f2cd37fe700 1 -- 172.21.15.101:0/316562059 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd80056b0 con 0x5593ae035c60 2023-07-18T03:11:50.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.517+0000 7f2d030d7b80 1 -- 172.21.15.101:0/316562059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5593ae035c60 msgr2=0x5593ae040610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:50.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.517+0000 7f2d030d7b80 1 --2- 172.21.15.101:0/316562059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5593ae035c60 0x5593ae040610 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2cd800b2e0 tx=0x7f2cd800b610 comp rx=0 tx=0).stop 2023-07-18T03:11:50.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.517+0000 7f2d030d7b80 1 -- 172.21.15.101:0/316562059 shutdown_connections 2023-07-18T03:11:50.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.517+0000 7f2d030d7b80 1 --2- 172.21.15.101:0/316562059 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593ae040b50 0x5593ae042f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.517+0000 7f2d030d7b80 1 --2- 172.21.15.101:0/316562059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593ae035350 0x5593ae035720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.517+0000 7f2d030d7b80 1 --2- 172.21.15.101:0/316562059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5593ae035c60 0x5593ae040610 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.517+0000 7f2d030d7b80 1 -- 172.21.15.101:0/316562059 >> 172.21.15.101:0/316562059 conn(0x5593ae02be20 msgr2=0x5593ae031720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:50.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.517+0000 7f2d030d7b80 1 -- 172.21.15.101:0/316562059 shutdown_connections 2023-07-18T03:11:50.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.517+0000 7f2d030d7b80 1 -- 172.21.15.101:0/316562059 wait complete. 2023-07-18T03:11:50.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.518+0000 7f2d030d7b80 1 -- start start 2023-07-18T03:11:50.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.518+0000 7f2d030d7b80 1 Processor -- start 2023-07-18T03:11:50.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.519+0000 7f2d030d7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5593ae035350 0x5593ae040570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:50.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.519+0000 7f2d030d7b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593ae035c60 0x5593ae03d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:50.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.519+0000 7f2d030d7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593ae040b50 0x5593ae03de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:50.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.519+0000 7f2d030d7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593adf27b40 con 0x5593ae040b50 2023-07-18T03:11:50.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.519+0000 7f2d030d7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593adf27840 con 0x5593ae035350 2023-07-18T03:11:50.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.519+0000 7f2d030d7b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593ade0b9a0 con 0x5593ae035c60 2023-07-18T03:11:50.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.519+0000 7f2d030d7b80 10 client.0 _ll_register_callbacks cb 0x5593adf257e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:11:50.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.519+0000 7f2cedd9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593ae040b50 0x5593ae03de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:50.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.519+0000 7f2cecd9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593ae035c60 0x5593ae03d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:50.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.519+0000 7f2cecd9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593ae035c60 0x5593ae03d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:49148/0 (socket says 172.21.15.101:49148) 2023-07-18T03:11:50.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.519+0000 7f2ced59b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5593ae035350 0x5593ae040570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:50.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.519+0000 7f2cecd9a700 1 -- 172.21.15.101:0/63060639 learned_addr learned my addr 172.21.15.101:0/63060639 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:50.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.519+0000 7f2cecd9a700 1 -- 172.21.15.101:0/63060639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5593ae035350 msgr2=0x5593ae040570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:50.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.519+0000 7f2cecd9a700 1 --2- 172.21.15.101:0/63060639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5593ae035350 0x5593ae040570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.519+0000 7f2cecd9a700 1 -- 172.21.15.101:0/63060639 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593ae040b50 msgr2=0x5593ae03de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:50.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.519+0000 7f2cecd9a700 1 --2- 172.21.15.101:0/63060639 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593ae040b50 0x5593ae03de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.519+0000 7f2cecd9a700 1 -- 172.21.15.101:0/63060639 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce810f390 con 0x5593ae035c60 2023-07-18T03:11:50.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.519+0000 7f2ced59b700 1 --2- 172.21.15.101:0/63060639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5593ae035350 0x5593ae040570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:11:50.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.520+0000 7f2cecd9a700 1 --2- 172.21.15.101:0/63060639 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593ae035c60 0x5593ae03d7f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2cd800b6c0 tx=0x7f2cd800bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84f4a9db21a67e3a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:50.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.520+0000 7f2cd0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:11:50.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.520+0000 7f2cd0ff9700 1 -- 172.21.15.101:0/63060639 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd801d050 con 0x5593ae035c60 2023-07-18T03:11:50.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.520+0000 7f2cd0ff9700 1 -- 172.21.15.101:0/63060639 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f2cd802f670 con 0x5593ae035c60 2023-07-18T03:11:50.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.520+0000 7f2d030d7b80 1 -- 172.21.15.101:0/63060639 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5593adee1570 con 0x5593ae035c60 2023-07-18T03:11:50.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.520+0000 7f2cd0ff9700 1 -- 172.21.15.101:0/63060639 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2cd802f830 con 0x5593ae035c60 2023-07-18T03:11:50.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.520+0000 7f2d030d7b80 10 client.26494 fetch_fsmap learned FSMap version 26 2023-07-18T03:11:50.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.520+0000 7f2d030d7b80 1 -- 172.21.15.101:0/63060639 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5593add9ece0 con 0x5593ae035c60 2023-07-18T03:11:50.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.520+0000 7f2cd0ff9700 1 -- 172.21.15.101:0/63060639 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5593adee1570 con 0x5593ae035c60 2023-07-18T03:11:50.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.521+0000 7f2cd0ff9700 1 -- 172.21.15.101:0/63060639 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2cd800fad0 con 0x5593ae035c60 2023-07-18T03:11:50.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.521+0000 7f2d030d7b80 10 client.26494 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:11:50.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.521+0000 7f2d030d7b80 10 client.26494 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:11:50.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.521+0000 7f2d030d7b80 20 client.26494 populate_metadata read hostname 'smithi101' 2023-07-18T03:11:50.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.521+0000 7f2d030d7b80 1 --2- 172.21.15.101:0/63060639 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5593ae04ff80 0x5593ae052390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:50.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.521+0000 7f2d030d7b80 4 client.26494 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:11:50.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.521+0000 7f2d030d7b80 1 -- 172.21.15.101:0/63060639 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5593adf80f30 con 0x5593ae04ff80 2023-07-18T03:11:50.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.521+0000 7f2ced59b700 1 --2- 172.21.15.101:0/63060639 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5593ae04ff80 0x5593ae052390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:50.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.522+0000 7f2cd0ff9700 10 client.26494 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:11:50.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.522+0000 7f2ced59b700 1 --2- 172.21.15.101:0/63060639 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5593ae04ff80 0x5593ae052390 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d9aebb41566422b server_cookie=64499fdeaa1c581 in_seq=0 out_seq=0 2023-07-18T03:11:50.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.523+0000 7f2cd0ff9700 1 -- 172.21.15.101:0/63060639 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5593adf80f30 con 0x5593ae04ff80 2023-07-18T03:11:50.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.523+0000 7f2cd0ff9700 10 client.26494 handle_command_reply: tid=0 2023-07-18T03:11:50.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.523+0000 7f2d030d7b80 1 client.26494 shutdown 2023-07-18T03:11:50.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.523+0000 7f2d030d7b80 1 -- 172.21.15.101:0/63060639 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593ae035c60 msgr2=0x5593ae03d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:50.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.523+0000 7f2d030d7b80 1 --2- 172.21.15.101:0/63060639 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593ae035c60 0x5593ae03d7f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2cd800b6c0 tx=0x7f2cd800bfd0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.523+0000 7f2d030d7b80 1 -- 172.21.15.101:0/63060639 shutdown_connections 2023-07-18T03:11:50.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.523+0000 7f2d030d7b80 1 --2- 172.21.15.101:0/63060639 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5593ae04ff80 0x5593ae052390 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.523+0000 7f2d030d7b80 1 --2- 172.21.15.101:0/63060639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5593ae035350 0x5593ae040570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.523+0000 7f2d030d7b80 1 --2- 172.21.15.101:0/63060639 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593ae040b50 0x5593ae03de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.523+0000 7f2d030d7b80 1 --2- 172.21.15.101:0/63060639 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593ae035c60 0x5593ae03d7f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.523+0000 7f2cd0ff9700 0 client.26494 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:11:50.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.523+0000 7f2d030d7b80 1 -- 172.21.15.101:0/63060639 >> 172.21.15.101:0/63060639 conn(0x5593ae02be20 msgr2=0x5593ae0312b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:50.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.524+0000 7f2d030d7b80 1 -- 172.21.15.101:0/63060639 shutdown_connections 2023-07-18T03:11:50.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.524+0000 7f2d030d7b80 1 -- 172.21.15.101:0/63060639 wait complete. 2023-07-18T03:11:50.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.524+0000 7f2d030d7b80 20 client.26494 trim_cache size 0 max 16384 2023-07-18T03:11:50.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.543+0000 7f2d030d7b80 1 Processor -- start 2023-07-18T03:11:50.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.543+0000 7f2d030d7b80 1 -- start start 2023-07-18T03:11:50.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.543+0000 7f2d030d7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593ae035350 0x5593ae038230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:50.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.543+0000 7f2d030d7b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593ae035c60 0x5593ae047530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:50.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.543+0000 7f2d030d7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5593ae040b50 0x5593ae031040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:50.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.543+0000 7f2d030d7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593ae032da0 con 0x5593ae035350 2023-07-18T03:11:50.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.543+0000 7f2d030d7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593ae0365c0 con 0x5593ae040b50 2023-07-18T03:11:50.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.543+0000 7f2d030d7b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593ae04c350 con 0x5593ae035c60 2023-07-18T03:11:50.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.543+0000 7f2cecd9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593ae035c60 0x5593ae047530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:50.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.544+0000 7f2cedd9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5593ae040b50 0x5593ae031040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:50.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.544+0000 7f2ced59b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593ae035350 0x5593ae038230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:50.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.544+0000 7f2cecd9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593ae035c60 0x5593ae047530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:49152/0 (socket says 172.21.15.101:49152) 2023-07-18T03:11:50.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.544+0000 7f2cecd9a700 1 -- 172.21.15.101:0/3589289287 learned_addr learned my addr 172.21.15.101:0/3589289287 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:50.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.544+0000 7f2cecd9a700 1 -- 172.21.15.101:0/3589289287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5593ae040b50 msgr2=0x5593ae031040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:50.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.544+0000 7f2cecd9a700 1 --2- 172.21.15.101:0/3589289287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5593ae040b50 0x5593ae031040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.544+0000 7f2cecd9a700 1 -- 172.21.15.101:0/3589289287 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593ae035350 msgr2=0x5593ae038230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:50.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.544+0000 7f2cecd9a700 1 --2- 172.21.15.101:0/3589289287 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593ae035350 0x5593ae038230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.544+0000 7f2cecd9a700 1 -- 172.21.15.101:0/3589289287 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593ae02d660 con 0x5593ae035c60 2023-07-18T03:11:50.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.544+0000 7f2cecd9a700 1 --2- 172.21.15.101:0/3589289287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593ae035c60 0x5593ae047530 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2cd800fe80 tx=0x7f2cd8009290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=661fd2b24e05ae73 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:50.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.544+0000 7f2cd3fff700 1 -- 172.21.15.101:0/3589289287 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd801d070 con 0x5593ae035c60 2023-07-18T03:11:50.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.545+0000 7f2cd3fff700 1 -- 172.21.15.101:0/3589289287 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2cd800bb30 con 0x5593ae035c60 2023-07-18T03:11:50.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.545+0000 7f2cd3fff700 1 -- 172.21.15.101:0/3589289287 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd800f9a0 con 0x5593ae035c60 2023-07-18T03:11:50.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.545+0000 7f2d030d7b80 1 -- 172.21.15.101:0/3589289287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593ae035c60 msgr2=0x5593ae047530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:50.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.545+0000 7f2d030d7b80 1 --2- 172.21.15.101:0/3589289287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593ae035c60 0x5593ae047530 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2cd800fe80 tx=0x7f2cd8009290 comp rx=0 tx=0).stop 2023-07-18T03:11:50.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.545+0000 7f2d030d7b80 1 -- 172.21.15.101:0/3589289287 shutdown_connections 2023-07-18T03:11:50.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.545+0000 7f2d030d7b80 1 --2- 172.21.15.101:0/3589289287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5593ae040b50 0x5593ae031040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.545+0000 7f2d030d7b80 1 --2- 172.21.15.101:0/3589289287 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593ae035350 0x5593ae038230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.545+0000 7f2d030d7b80 1 --2- 172.21.15.101:0/3589289287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593ae035c60 0x5593ae047530 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.545+0000 7f2d030d7b80 1 -- 172.21.15.101:0/3589289287 >> 172.21.15.101:0/3589289287 conn(0x5593ae02be20 msgr2=0x5593ae030930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:50.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.545+0000 7f2d030d7b80 1 -- 172.21.15.101:0/3589289287 shutdown_connections 2023-07-18T03:11:50.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.545+0000 7f2d030d7b80 1 -- 172.21.15.101:0/3589289287 wait complete. 2023-07-18T03:11:50.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.546+0000 7f2d030d7b80 1 -- start start 2023-07-18T03:11:50.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.546+0000 7f2d030d7b80 1 Processor -- start 2023-07-18T03:11:50.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.547+0000 7f2d030d7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593ae035350 0x5593ae047380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:50.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.547+0000 7f2d030d7b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593ae035c60 0x5593ae0456b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:50.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.547+0000 7f2d030d7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5593ae040b50 0x5593ae045bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:50.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.547+0000 7f2d030d7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593ae046130 con 0x5593ae035350 2023-07-18T03:11:50.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.547+0000 7f2d030d7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593ae0462a0 con 0x5593ae040b50 2023-07-18T03:11:50.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.547+0000 7f2d030d7b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593ae046460 con 0x5593ae035c60 2023-07-18T03:11:50.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.547+0000 7f2d030d7b80 10 client.0 _ll_register_callbacks cb 0x7f2ce4095b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:11:50.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.547+0000 7f2cecd9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593ae035c60 0x5593ae0456b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:50.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.547+0000 7f2ced59b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593ae035350 0x5593ae047380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:50.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.547+0000 7f2cedd9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5593ae040b50 0x5593ae045bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:50.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.547+0000 7f2cecd9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593ae035c60 0x5593ae0456b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:49162/0 (socket says 172.21.15.101:49162) 2023-07-18T03:11:50.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.547+0000 7f2cecd9a700 1 -- 172.21.15.101:0/236166326 learned_addr learned my addr 172.21.15.101:0/236166326 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:50.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.548+0000 7f2ced59b700 1 -- 172.21.15.101:0/236166326 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593ae035c60 msgr2=0x5593ae0456b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:50.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.548+0000 7f2ced59b700 1 --2- 172.21.15.101:0/236166326 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593ae035c60 0x5593ae0456b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.548+0000 7f2ced59b700 1 -- 172.21.15.101:0/236166326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5593ae040b50 msgr2=0x5593ae045bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:50.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.548+0000 7f2ced59b700 1 --2- 172.21.15.101:0/236166326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5593ae040b50 0x5593ae045bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.548+0000 7f2ced59b700 1 -- 172.21.15.101:0/236166326 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5593add9ece0 con 0x5593ae035350 2023-07-18T03:11:50.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.548+0000 7f2cecd9a700 1 --2- 172.21.15.101:0/236166326 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593ae035c60 0x5593ae0456b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:11:50.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.548+0000 7f2ced59b700 1 --2- 172.21.15.101:0/236166326 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593ae035350 0x5593ae047380 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4096440 tx=0x7f2ce40a1de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26a8e6101a8d5b94 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:50.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.548+0000 7f2cd0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:11:50.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.548+0000 7f2cd0ff9700 1 -- 172.21.15.101:0/236166326 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce40a1280 con 0x5593ae035350 2023-07-18T03:11:50.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.548+0000 7f2cd0ff9700 1 -- 172.21.15.101:0/236166326 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f2ce40a14c0 con 0x5593ae035350 2023-07-18T03:11:50.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.548+0000 7f2d030d7b80 1 -- 172.21.15.101:0/236166326 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5593adf430d0 con 0x5593ae035350 2023-07-18T03:11:50.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.548+0000 7f2cd0ff9700 1 -- 172.21.15.101:0/236166326 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ce40a2140 con 0x5593ae035350 2023-07-18T03:11:50.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.548+0000 7f2d030d7b80 10 client.16431 fetch_fsmap learned FSMap version 26 2023-07-18T03:11:50.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.548+0000 7f2d030d7b80 1 -- 172.21.15.101:0/236166326 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5593adddc2d0 con 0x5593ae035350 2023-07-18T03:11:50.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.549+0000 7f2cd0ff9700 1 -- 172.21.15.101:0/236166326 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5593adf430d0 con 0x5593ae035350 2023-07-18T03:11:50.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.549+0000 7f2cd0ff9700 1 -- 172.21.15.101:0/236166326 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2ce40a1700 con 0x5593ae035350 2023-07-18T03:11:50.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.549+0000 7f2d030d7b80 10 client.16431 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:11:50.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.549+0000 7f2d030d7b80 10 client.16431 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:11:50.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.549+0000 7f2d030d7b80 20 client.16431 populate_metadata read hostname 'smithi101' 2023-07-18T03:11:50.547 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.549+0000 7f2d030d7b80 1 --2- 172.21.15.101:0/236166326 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5593ae0a5290 0x5593ae0a76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:50.547 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.549+0000 7f2d030d7b80 4 client.16431 mds_command: new command op to 24562 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-18T03:11:50.547 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.549+0000 7f2d030d7b80 1 -- 172.21.15.101:0/236166326 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5593addd6ff0 con 0x5593ae0a5290 2023-07-18T03:11:50.547 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.549+0000 7f2cecd9a700 1 --2- 172.21.15.101:0/236166326 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5593ae0a5290 0x5593ae0a76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:50.547 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.549+0000 7f2cd0ff9700 10 client.16431 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:11:50.547 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.549+0000 7f2cecd9a700 1 --2- 172.21.15.101:0/236166326 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5593ae0a5290 0x5593ae0a76a0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66e1c0244295a82e server_cookie=714778c0d175673b in_seq=0 out_seq=0 2023-07-18T03:11:50.547 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.550+0000 7f2cd0ff9700 1 -- 172.21.15.101:0/236166326 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5593addd6ff0 con 0x5593ae0a5290 2023-07-18T03:11:50.547 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.550+0000 7f2cd0ff9700 10 client.16431 handle_command_reply: tid=0 2023-07-18T03:11:50.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.550+0000 7f2d030d7b80 1 client.16431 shutdown 2023-07-18T03:11:50.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.551+0000 7f2d030d7b80 1 -- 172.21.15.101:0/236166326 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593ae035350 msgr2=0x5593ae047380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:50.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.551+0000 7f2d030d7b80 1 --2- 172.21.15.101:0/236166326 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593ae035350 0x5593ae047380 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4096440 tx=0x7f2ce40a1de0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.551+0000 7f2d030d7b80 1 -- 172.21.15.101:0/236166326 shutdown_connections 2023-07-18T03:11:50.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.551+0000 7f2d030d7b80 1 --2- 172.21.15.101:0/236166326 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5593ae0a5290 0x5593ae0a76a0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.551+0000 7f2d030d7b80 1 --2- 172.21.15.101:0/236166326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5593ae040b50 0x5593ae045bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.551+0000 7f2d030d7b80 1 --2- 172.21.15.101:0/236166326 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593ae035350 0x5593ae047380 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.549 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.551+0000 7f2d030d7b80 1 --2- 172.21.15.101:0/236166326 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593ae035c60 0x5593ae0456b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.549 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.551+0000 7f2cd0ff9700 0 client.16431 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:11:50.549 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.551+0000 7f2d030d7b80 1 -- 172.21.15.101:0/236166326 >> 172.21.15.101:0/236166326 conn(0x5593ae02be20 msgr2=0x5593ae0311c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:50.549 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.551+0000 7f2d030d7b80 1 -- 172.21.15.101:0/236166326 shutdown_connections 2023-07-18T03:11:50.549 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.551+0000 7f2d030d7b80 1 -- 172.21.15.101:0/236166326 wait complete. 2023-07-18T03:11:50.549 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.551+0000 7f2d030d7b80 20 client.16431 trim_cache size 0 max 16384 2023-07-18T03:11:50.549 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:11:50.550 INFO:teuthology.orchestra.run.smithi101.stdout: "return_code": 0, 2023-07-18T03:11:50.550 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_tag": "d3886fa6-7130-456c-aed3-bcba5cd62880", 2023-07-18T03:11:50.550 INFO:teuthology.orchestra.run.smithi101.stdout: "mode": "asynchronous" 2023-07-18T03:11:50.550 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:11:50.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.552+0000 7f2cef7ff700 1 -- 172.21.15.101:0/470539790 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2cd40a1340 msgr2=0x7f2cd40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:50.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.552+0000 7f2cef7ff700 1 --2- 172.21.15.101:0/470539790 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2cd40a1340 0x7f2cd40a37f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0009550 tx=0x7f2ce000b040 comp rx=0 tx=0).stop 2023-07-18T03:11:50.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.552+0000 7f2cef7ff700 1 -- 172.21.15.101:0/470539790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ce80728b0 msgr2=0x7f2ce8197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:50.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.552+0000 7f2cef7ff700 1 --2- 172.21.15.101:0/470539790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ce80728b0 0x7f2ce8197e10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2ce400e9a0 tx=0x7f2ce400ecb0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.552+0000 7f2cef7ff700 1 -- 172.21.15.101:0/470539790 shutdown_connections 2023-07-18T03:11:50.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.552+0000 7f2cef7ff700 1 --2- 172.21.15.101:0/470539790 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2cd40a1340 0x7f2cd40a37f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.552+0000 7f2cef7ff700 1 --2- 172.21.15.101:0/470539790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ce80728b0 0x7f2ce8197e10 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.552+0000 7f2cef7ff700 1 --2- 172.21.15.101:0/470539790 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2ce810c310 0x7f2ce819c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.552+0000 7f2cef7ff700 1 --2- 172.21.15.101:0/470539790 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ce80731c0 0x7f2ce8198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.552+0000 7f2cef7ff700 1 -- 172.21.15.101:0/470539790 >> 172.21.15.101:0/470539790 conn(0x7f2ce80fb5f0 msgr2=0x7f2ce810e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:50.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.552+0000 7f2cef7ff700 1 -- 172.21.15.101:0/470539790 shutdown_connections 2023-07-18T03:11:50.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.552+0000 7f2cef7ff700 1 -- 172.21.15.101:0/470539790 wait complete. 2023-07-18T03:11:50.561 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:11:50.575 INFO:tasks.workunit.client.0.smithi101.stdout: 1 307267 9.02 MB/sec execute 75 sec latency 11.607 ms 2023-07-18T03:11:50.777 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.786+0000 7f1194bff700 1 -- 172.21.15.101:0/962111838 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1190105af0 msgr2=0x7f1190105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:50.778 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.786+0000 7f1194bff700 1 --2- 172.21.15.101:0/962111838 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1190105af0 0x7f1190105ec0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1180009990 tx=0x7f1180009ca0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.778 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.786+0000 7f1194bff700 1 -- 172.21.15.101:0/962111838 shutdown_connections 2023-07-18T03:11:50.778 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.786+0000 7f1194bff700 1 --2- 172.21.15.101:0/962111838 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1190100490 0x7f119010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.778 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.786+0000 7f1194bff700 1 --2- 172.21.15.101:0/962111838 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1190105af0 0x7f1190105ec0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.778 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.786+0000 7f1194bff700 1 --2- 172.21.15.101:0/962111838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11900ffae0 0x7f11900fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.778 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.786+0000 7f1194bff700 1 -- 172.21.15.101:0/962111838 >> 172.21.15.101:0/962111838 conn(0x7f11900fb5e0 msgr2=0x7f11900fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:50.778 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.786+0000 7f1194bff700 1 -- 172.21.15.101:0/962111838 shutdown_connections 2023-07-18T03:11:50.779 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.786+0000 7f1194bff700 1 -- 172.21.15.101:0/962111838 wait complete. 2023-07-18T03:11:50.780 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.787+0000 7f1194bff700 1 Processor -- start 2023-07-18T03:11:50.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.788+0000 7f1194bff700 1 -- start start 2023-07-18T03:11:50.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.788+0000 7f1194bff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11900ffae0 0x7f1190197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:50.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.788+0000 7f1194bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1190100490 0x7f1190198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:50.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.788+0000 7f1194bff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1190105af0 0x7f119019c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:50.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.788+0000 7f1194bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119019cdc0 con 0x7f1190100490 2023-07-18T03:11:50.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.788+0000 7f1194bff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119019cf30 con 0x7f11900ffae0 2023-07-18T03:11:50.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.788+0000 7f1194bff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119019d0f0 con 0x7f1190105af0 2023-07-18T03:11:50.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.788+0000 7f118e59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11900ffae0 0x7f1190197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:50.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.788+0000 7f118dd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1190100490 0x7f1190198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:50.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.788+0000 7f118e59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11900ffae0 0x7f1190197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:52114/0 (socket says 172.21.15.101:52114) 2023-07-18T03:11:50.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.788+0000 7f118e59c700 1 -- 172.21.15.101:0/2474684752 learned_addr learned my addr 172.21.15.101:0/2474684752 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:50.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.788+0000 7f118ed9d700 1 --2- 172.21.15.101:0/2474684752 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1190105af0 0x7f119019c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:50.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.789+0000 7f118e59c700 1 -- 172.21.15.101:0/2474684752 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1190105af0 msgr2=0x7f119019c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:50.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.789+0000 7f118e59c700 1 --2- 172.21.15.101:0/2474684752 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1190105af0 0x7f119019c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.789+0000 7f118e59c700 1 -- 172.21.15.101:0/2474684752 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1190100490 msgr2=0x7f1190198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:50.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.789+0000 7f118e59c700 1 --2- 172.21.15.101:0/2474684752 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1190100490 0x7f1190198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.789+0000 7f118e59c700 1 -- 172.21.15.101:0/2474684752 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11800095f0 con 0x7f11900ffae0 2023-07-18T03:11:50.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.789+0000 7f118e59c700 1 --2- 172.21.15.101:0/2474684752 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11900ffae0 0x7f1190197de0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1180005910 tx=0x7f1180005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:50.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.789+0000 7f117f7fe700 1 -- 172.21.15.101:0/2474684752 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118001d070 con 0x7f11900ffae0 2023-07-18T03:11:50.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.789+0000 7f1194bff700 1 -- 172.21.15.101:0/2474684752 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f119019d370 con 0x7f11900ffae0 2023-07-18T03:11:50.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.789+0000 7f117f7fe700 1 -- 172.21.15.101:0/2474684752 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1180022470 con 0x7f11900ffae0 2023-07-18T03:11:50.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.789+0000 7f117f7fe700 1 -- 172.21.15.101:0/2474684752 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1180017650 con 0x7f11900ffae0 2023-07-18T03:11:50.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.789+0000 7f1194bff700 1 -- 172.21.15.101:0/2474684752 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11901a4c10 con 0x7f11900ffae0 2023-07-18T03:11:50.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.793+0000 7f117f7fe700 1 -- 172.21.15.101:0/2474684752 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f11800225e0 con 0x7f11900ffae0 2023-07-18T03:11:50.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.794+0000 7f117f7fe700 1 --2- 172.21.15.101:0/2474684752 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f11780a1340 0x7f11780a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:50.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.794+0000 7f117f7fe700 1 -- 172.21.15.101:0/2474684752 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f11800ce320 con 0x7f11900ffae0 2023-07-18T03:11:50.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.794+0000 7f11a8486b80 1 Processor -- start 2023-07-18T03:11:50.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.794+0000 7f118dd9b700 1 --2- 172.21.15.101:0/2474684752 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f11780a1340 0x7f11780a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:50.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.795+0000 7f11a8486b80 1 -- start start 2023-07-18T03:11:50.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.795+0000 7f11a8486b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6e89592d0 0x55a6e89596a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:50.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.795+0000 7f11a8486b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a6e8959be0 0x55a6e8964590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:50.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.795+0000 7f11a8486b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a6e8964ad0 0x55a6e8966eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:50.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.795+0000 7f11a8486b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6e885fcb0 con 0x55a6e8959be0 2023-07-18T03:11:50.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.795+0000 7f11a8486b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6e88b2a40 con 0x55a6e89592d0 2023-07-18T03:11:50.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.795+0000 7f11a8486b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6e88abed0 con 0x55a6e8964ad0 2023-07-18T03:11:50.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.795+0000 7f118e59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6e89592d0 0x55a6e89596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:50.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.795+0000 7f118e59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6e89592d0 0x55a6e89596a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:52122/0 (socket says 172.21.15.101:52122) 2023-07-18T03:11:50.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.795+0000 7f118e59c700 1 -- 172.21.15.101:0/1449530447 learned_addr learned my addr 172.21.15.101:0/1449530447 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:50.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.795+0000 7f118dd9b700 1 --2- 172.21.15.101:0/1449530447 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a6e8959be0 0x55a6e8964590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:50.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.795+0000 7f118ed9d700 1 --2- 172.21.15.101:0/1449530447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a6e8964ad0 0x55a6e8966eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:50.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.795+0000 7f118dd9b700 1 --2- 172.21.15.101:0/2474684752 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f11780a1340 0x7f11780a37f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f11901993f0 tx=0x7f1184009260 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:50.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.795+0000 7f118e59c700 1 -- 172.21.15.101:0/1449530447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a6e8964ad0 msgr2=0x55a6e8966eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:50.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.796+0000 7f118e59c700 1 --2- 172.21.15.101:0/1449530447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a6e8964ad0 0x55a6e8966eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.796+0000 7f118e59c700 1 -- 172.21.15.101:0/1449530447 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a6e8959be0 msgr2=0x55a6e8964590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:50.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.796+0000 7f118e59c700 1 --2- 172.21.15.101:0/1449530447 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a6e8959be0 0x55a6e8964590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.796+0000 7f118e59c700 1 -- 172.21.15.101:0/1449530447 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6e86d9de0 con 0x55a6e89592d0 2023-07-18T03:11:50.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.796+0000 7f118e59c700 1 --2- 172.21.15.101:0/1449530447 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6e89592d0 0x55a6e89596a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f11800975d0 tx=0x7f1180005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c7b527fb6de6c67 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:50.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.796+0000 7f117cff9700 1 -- 172.21.15.101:0/1449530447 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118009aa30 con 0x55a6e89592d0 2023-07-18T03:11:50.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.796+0000 7f117cff9700 1 -- 172.21.15.101:0/1449530447 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f118009b0d0 con 0x55a6e89592d0 2023-07-18T03:11:50.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.796+0000 7f117cff9700 1 -- 172.21.15.101:0/1449530447 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118009b590 con 0x55a6e89592d0 2023-07-18T03:11:50.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.797+0000 7f11a8486b80 1 -- 172.21.15.101:0/1449530447 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6e89592d0 msgr2=0x55a6e89596a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:50.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.797+0000 7f11a8486b80 1 --2- 172.21.15.101:0/1449530447 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6e89592d0 0x55a6e89596a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f11800975d0 tx=0x7f1180005f10 comp rx=0 tx=0).stop 2023-07-18T03:11:50.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.797+0000 7f11a8486b80 1 -- 172.21.15.101:0/1449530447 shutdown_connections 2023-07-18T03:11:50.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.797+0000 7f11a8486b80 1 --2- 172.21.15.101:0/1449530447 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6e89592d0 0x55a6e89596a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.797+0000 7f11a8486b80 1 --2- 172.21.15.101:0/1449530447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a6e8964ad0 0x55a6e8966eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.797+0000 7f11a8486b80 1 --2- 172.21.15.101:0/1449530447 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a6e8959be0 0x55a6e8964590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.797+0000 7f11a8486b80 1 -- 172.21.15.101:0/1449530447 >> 172.21.15.101:0/1449530447 conn(0x55a6e894fda0 msgr2=0x55a6e89556a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:50.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.797+0000 7f11a8486b80 1 -- 172.21.15.101:0/1449530447 shutdown_connections 2023-07-18T03:11:50.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.797+0000 7f11a8486b80 1 -- 172.21.15.101:0/1449530447 wait complete. 2023-07-18T03:11:50.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.798+0000 7f11a8486b80 1 -- start start 2023-07-18T03:11:50.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.798+0000 7f11a8486b80 1 Processor -- start 2023-07-18T03:11:50.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.798+0000 7f11a8486b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a6e89592d0 0x55a6e8969b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:50.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.798+0000 7f11a8486b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a6e8959be0 0x55a6e896a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:50.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.798+0000 7f11a8486b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6e8964ad0 0x55a6e8963aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:50.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.798+0000 7f11a8486b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6e884bac0 con 0x55a6e89592d0 2023-07-18T03:11:50.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.798+0000 7f11a8486b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6e884b7c0 con 0x55a6e8964ad0 2023-07-18T03:11:50.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.798+0000 7f11a8486b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6e872f920 con 0x55a6e8959be0 2023-07-18T03:11:50.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.798+0000 7f11a8486b80 10 client.0 _ll_register_callbacks cb 0x55a6e8849760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:11:50.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.799+0000 7f118dd9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a6e8959be0 0x55a6e896a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:50.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.799+0000 7f118e59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a6e89592d0 0x55a6e8969b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:50.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.799+0000 7f118dd9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a6e8959be0 0x55a6e896a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:49194/0 (socket says 172.21.15.101:49194) 2023-07-18T03:11:50.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.799+0000 7f118ed9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6e8964ad0 0x55a6e8963aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:50.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.799+0000 7f118dd9b700 1 -- 172.21.15.101:0/1415017623 learned_addr learned my addr 172.21.15.101:0/1415017623 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:50.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.799+0000 7f118dd9b700 1 -- 172.21.15.101:0/1415017623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6e8964ad0 msgr2=0x55a6e8963aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:50.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.799+0000 7f118dd9b700 1 --2- 172.21.15.101:0/1415017623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6e8964ad0 0x55a6e8963aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.799+0000 7f118dd9b700 1 -- 172.21.15.101:0/1415017623 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a6e89592d0 msgr2=0x55a6e8969b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:50.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.799+0000 7f118dd9b700 1 --2- 172.21.15.101:0/1415017623 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a6e89592d0 0x55a6e8969b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.799+0000 7f118dd9b700 1 -- 172.21.15.101:0/1415017623 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11901a4c10 con 0x55a6e8959be0 2023-07-18T03:11:50.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.799+0000 7f118ed9d700 1 --2- 172.21.15.101:0/1415017623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6e8964ad0 0x55a6e8963aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:11:50.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.799+0000 7f118e59c700 1 --2- 172.21.15.101:0/1415017623 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a6e89592d0 0x55a6e8969b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:11:50.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.799+0000 7f118dd9b700 1 --2- 172.21.15.101:0/1415017623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a6e8959be0 0x55a6e896a0b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f118401d850 tx=0x7f11840047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18c3ae714ad4dc5d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:50.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.799+0000 7f1175ffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:11:50.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.800+0000 7f1175ffb700 1 -- 172.21.15.101:0/1415017623 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1184022070 con 0x55a6e8959be0 2023-07-18T03:11:50.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.800+0000 7f11a8486b80 1 -- 172.21.15.101:0/1415017623 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a6e88054f0 con 0x55a6e8959be0 2023-07-18T03:11:50.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.800+0000 7f1175ffb700 1 -- 172.21.15.101:0/1415017623 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f118401c6e0 con 0x55a6e8959be0 2023-07-18T03:11:50.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.800+0000 7f1175ffb700 1 -- 172.21.15.101:0/1415017623 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f118401c8a0 con 0x55a6e8959be0 2023-07-18T03:11:50.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.800+0000 7f11a8486b80 10 client.26512 fetch_fsmap learned FSMap version 26 2023-07-18T03:11:50.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.800+0000 7f11a8486b80 1 -- 172.21.15.101:0/1415017623 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55a6e86c2c60 con 0x55a6e8959be0 2023-07-18T03:11:50.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.800+0000 7f1175ffb700 1 -- 172.21.15.101:0/1415017623 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a6e88054f0 con 0x55a6e8959be0 2023-07-18T03:11:50.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.800+0000 7f1175ffb700 1 -- 172.21.15.101:0/1415017623 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1184027a50 con 0x55a6e8959be0 2023-07-18T03:11:50.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.800+0000 7f11a8486b80 10 client.26512 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:11:50.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.800+0000 7f11a8486b80 10 client.26512 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:11:50.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.800+0000 7f11a8486b80 20 client.26512 populate_metadata read hostname 'smithi101' 2023-07-18T03:11:50.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.800+0000 7f11a8486b80 1 --2- 172.21.15.101:0/1415017623 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a6e89740f0 0x55a6e8976500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:50.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.800+0000 7f11a8486b80 4 client.26512 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:11:50.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.800+0000 7f11a8486b80 1 -- 172.21.15.101:0/1415017623 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a6e88a4eb0 con 0x55a6e89740f0 2023-07-18T03:11:50.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.801+0000 7f118e59c700 1 --2- 172.21.15.101:0/1415017623 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a6e89740f0 0x55a6e8976500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:50.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.801+0000 7f118e59c700 1 --2- 172.21.15.101:0/1415017623 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a6e89740f0 0x55a6e8976500 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e62ceb67858ddba server_cookie=44115f6f10dcaf81 in_seq=0 out_seq=0 2023-07-18T03:11:50.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.801+0000 7f1175ffb700 10 client.26512 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:11:50.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.802+0000 7f1175ffb700 1 -- 172.21.15.101:0/1415017623 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a6e88a4eb0 con 0x55a6e89740f0 2023-07-18T03:11:50.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.802+0000 7f1175ffb700 10 client.26512 handle_command_reply: tid=0 2023-07-18T03:11:50.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.802+0000 7f11a8486b80 1 client.26512 shutdown 2023-07-18T03:11:50.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.802+0000 7f11a8486b80 1 -- 172.21.15.101:0/1415017623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a6e8959be0 msgr2=0x55a6e896a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:50.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.802+0000 7f11a8486b80 1 --2- 172.21.15.101:0/1415017623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a6e8959be0 0x55a6e896a0b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f118401d850 tx=0x7f11840047c0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.802+0000 7f11a8486b80 1 -- 172.21.15.101:0/1415017623 shutdown_connections 2023-07-18T03:11:50.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.802+0000 7f11a8486b80 1 --2- 172.21.15.101:0/1415017623 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a6e89740f0 0x55a6e8976500 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.802+0000 7f11a8486b80 1 --2- 172.21.15.101:0/1415017623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6e8964ad0 0x55a6e8963aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.802+0000 7f11a8486b80 1 --2- 172.21.15.101:0/1415017623 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a6e89592d0 0x55a6e8969b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.802+0000 7f1175ffb700 0 client.26512 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:11:50.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.802+0000 7f11a8486b80 1 --2- 172.21.15.101:0/1415017623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a6e8959be0 0x55a6e896a0b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.802+0000 7f11a8486b80 1 -- 172.21.15.101:0/1415017623 >> 172.21.15.101:0/1415017623 conn(0x55a6e894fda0 msgr2=0x55a6e8955290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:50.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.802+0000 7f11a8486b80 1 -- 172.21.15.101:0/1415017623 shutdown_connections 2023-07-18T03:11:50.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.802+0000 7f11a8486b80 1 -- 172.21.15.101:0/1415017623 wait complete. 2023-07-18T03:11:50.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.802+0000 7f11a8486b80 20 client.26512 trim_cache size 0 max 16384 2023-07-18T03:11:50.814 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.821+0000 7f11a8486b80 1 Processor -- start 2023-07-18T03:11:50.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.821+0000 7f11a8486b80 1 -- start start 2023-07-18T03:11:50.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.821+0000 7f11a8486b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a6e89592d0 0x55a6e8963bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:50.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.821+0000 7f11a8486b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6e8959be0 0x55a6e8962960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:50.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.821+0000 7f11a8486b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a6e8964ad0 0x55a6e8969740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:50.816 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.821+0000 7f11a8486b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6e88a4eb0 con 0x55a6e89592d0 2023-07-18T03:11:50.816 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.821+0000 7f11a8486b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6e88054f0 con 0x55a6e8959be0 2023-07-18T03:11:50.816 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.821+0000 7f11a8486b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6e894f9e0 con 0x55a6e8964ad0 2023-07-18T03:11:50.816 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.821+0000 7f118dd9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6e8959be0 0x55a6e8962960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:50.816 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.821+0000 7f118e59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a6e89592d0 0x55a6e8963bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:50.816 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.821+0000 7f118dd9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6e8959be0 0x55a6e8962960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:52140/0 (socket says 172.21.15.101:52140) 2023-07-18T03:11:50.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.821+0000 7f118dd9b700 1 -- 172.21.15.101:0/2444087996 learned_addr learned my addr 172.21.15.101:0/2444087996 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:50.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.821+0000 7f118ed9d700 1 --2- 172.21.15.101:0/2444087996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a6e8964ad0 0x55a6e8969740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:50.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.822+0000 7f118dd9b700 1 -- 172.21.15.101:0/2444087996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a6e8964ad0 msgr2=0x55a6e8969740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:50.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.822+0000 7f118dd9b700 1 --2- 172.21.15.101:0/2444087996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a6e8964ad0 0x55a6e8969740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.822+0000 7f118dd9b700 1 -- 172.21.15.101:0/2444087996 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a6e89592d0 msgr2=0x55a6e8963bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:50.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.822+0000 7f118dd9b700 1 --2- 172.21.15.101:0/2444087996 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a6e89592d0 0x55a6e8963bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.822+0000 7f118dd9b700 1 -- 172.21.15.101:0/2444087996 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6e8700250 con 0x55a6e8959be0 2023-07-18T03:11:50.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.822+0000 7f118e59c700 1 --2- 172.21.15.101:0/2444087996 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a6e89592d0 0x55a6e8963bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:11:50.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.822+0000 7f118dd9b700 1 --2- 172.21.15.101:0/2444087996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6e8959be0 0x55a6e8962960 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1184027e00 tx=0x7f1184011850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf857eb24c6f5c74 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:50.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.822+0000 7f117d7fa700 1 -- 172.21.15.101:0/2444087996 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1184022070 con 0x55a6e8959be0 2023-07-18T03:11:50.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.822+0000 7f117d7fa700 1 -- 172.21.15.101:0/2444087996 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11840263c0 con 0x55a6e8959be0 2023-07-18T03:11:50.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.822+0000 7f117d7fa700 1 -- 172.21.15.101:0/2444087996 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1184027510 con 0x55a6e8959be0 2023-07-18T03:11:50.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.823+0000 7f11a8486b80 1 -- 172.21.15.101:0/2444087996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6e8959be0 msgr2=0x55a6e8962960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:50.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.823+0000 7f11a8486b80 1 --2- 172.21.15.101:0/2444087996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6e8959be0 0x55a6e8962960 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1184027e00 tx=0x7f1184011850 comp rx=0 tx=0).stop 2023-07-18T03:11:50.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.823+0000 7f11a8486b80 1 -- 172.21.15.101:0/2444087996 shutdown_connections 2023-07-18T03:11:50.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.823+0000 7f11a8486b80 1 --2- 172.21.15.101:0/2444087996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a6e8964ad0 0x55a6e8969740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.823+0000 7f11a8486b80 1 --2- 172.21.15.101:0/2444087996 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a6e89592d0 0x55a6e8963bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.823+0000 7f11a8486b80 1 --2- 172.21.15.101:0/2444087996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6e8959be0 0x55a6e8962960 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.823+0000 7f11a8486b80 1 -- 172.21.15.101:0/2444087996 >> 172.21.15.101:0/2444087996 conn(0x55a6e894fda0 msgr2=0x55a6e8962250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:50.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.823+0000 7f11a8486b80 1 -- 172.21.15.101:0/2444087996 shutdown_connections 2023-07-18T03:11:50.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.823+0000 7f11a8486b80 1 -- 172.21.15.101:0/2444087996 wait complete. 2023-07-18T03:11:50.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.823+0000 7f11a8486b80 1 -- start start 2023-07-18T03:11:50.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.824+0000 7f11a8486b80 1 Processor -- start 2023-07-18T03:11:50.822 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.824+0000 7f11a8486b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a6e89592d0 0x55a6e89c18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:50.823 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.824+0000 7f11a8486b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6e8959be0 0x55a6e89c1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:50.823 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.824+0000 7f11a8486b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a6e8964ad0 0x55a6e89bbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:50.823 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.824+0000 7f11a8486b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6e89bc620 con 0x55a6e89592d0 2023-07-18T03:11:50.823 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.824+0000 7f11a8486b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6e89bc790 con 0x55a6e8959be0 2023-07-18T03:11:50.823 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.824+0000 7f11a8486b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6e89bc950 con 0x55a6e8964ad0 2023-07-18T03:11:50.823 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.824+0000 7f11a8486b80 10 client.0 _ll_register_callbacks cb 0x7f118401d410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:11:50.824 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.825+0000 7f118dd9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6e8959be0 0x55a6e89c1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:50.824 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.825+0000 7f118e59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a6e89592d0 0x55a6e89c18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:50.824 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.825+0000 7f118dd9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6e8959be0 0x55a6e89c1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:52152/0 (socket says 172.21.15.101:52152) 2023-07-18T03:11:50.824 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.825+0000 7f118dd9b700 1 -- 172.21.15.101:0/861905087 learned_addr learned my addr 172.21.15.101:0/861905087 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:11:50.824 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.825+0000 7f118ed9d700 1 --2- 172.21.15.101:0/861905087 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a6e8964ad0 0x55a6e89bbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:50.824 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.825+0000 7f118dd9b700 1 -- 172.21.15.101:0/861905087 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a6e8964ad0 msgr2=0x55a6e89bbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:50.824 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.825+0000 7f118dd9b700 1 --2- 172.21.15.101:0/861905087 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a6e8964ad0 0x55a6e89bbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.825+0000 7f118dd9b700 1 -- 172.21.15.101:0/861905087 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a6e89592d0 msgr2=0x55a6e89c18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:11:50.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.825+0000 7f118dd9b700 1 --2- 172.21.15.101:0/861905087 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a6e89592d0 0x55a6e89c18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.825+0000 7f118dd9b700 1 -- 172.21.15.101:0/861905087 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a6e86d9de0 con 0x55a6e8959be0 2023-07-18T03:11:50.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.825+0000 7f118dd9b700 1 --2- 172.21.15.101:0/861905087 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6e8959be0 0x55a6e89c1e30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f118401d850 tx=0x7f1184005750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d2801032615d78b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:11:50.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.825+0000 7f1175ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:11:50.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.826+0000 7f1175ffb700 1 -- 172.21.15.101:0/861905087 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1184022040 con 0x55a6e8959be0 2023-07-18T03:11:50.826 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.826+0000 7f1175ffb700 1 -- 172.21.15.101:0/861905087 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f11840049d0 con 0x55a6e8959be0 2023-07-18T03:11:50.826 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.826+0000 7f11a8486b80 1 -- 172.21.15.101:0/861905087 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f118009b0d0 con 0x55a6e8959be0 2023-07-18T03:11:50.826 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.826+0000 7f1175ffb700 1 -- 172.21.15.101:0/861905087 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1184004b90 con 0x55a6e8959be0 2023-07-18T03:11:50.826 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.826+0000 7f11a8486b80 10 client.26495 fetch_fsmap learned FSMap version 26 2023-07-18T03:11:50.826 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.826+0000 7f11a8486b80 1 -- 172.21.15.101:0/861905087 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55a6e86c2c60 con 0x55a6e8959be0 2023-07-18T03:11:50.826 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.826+0000 7f1175ffb700 1 -- 172.21.15.101:0/861905087 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f118009b0d0 con 0x55a6e8959be0 2023-07-18T03:11:50.826 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.826+0000 7f1175ffb700 1 -- 172.21.15.101:0/861905087 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1184047070 con 0x55a6e8959be0 2023-07-18T03:11:50.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.827+0000 7f11a8486b80 10 client.26495 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:11:50.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.827+0000 7f11a8486b80 10 client.26495 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:11:50.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.827+0000 7f11a8486b80 20 client.26495 populate_metadata read hostname 'smithi101' 2023-07-18T03:11:50.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.827+0000 7f11a8486b80 1 --2- 172.21.15.101:0/861905087 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a6e89c9450 0x55a6e89cb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:11:50.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.827+0000 7f11a8486b80 4 client.26495 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:11:50.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.827+0000 7f11a8486b80 1 -- 172.21.15.101:0/861905087 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a6e8867050 con 0x55a6e89c9450 2023-07-18T03:11:50.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.827+0000 7f118e59c700 1 --2- 172.21.15.101:0/861905087 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a6e89c9450 0x55a6e89cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:11:50.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.827+0000 7f118e59c700 1 --2- 172.21.15.101:0/861905087 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a6e89c9450 0x55a6e89cb860 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ca3e58c85d89553 server_cookie=ac0f871a2ea21932 in_seq=0 out_seq=0 2023-07-18T03:11:50.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.827+0000 7f1175ffb700 10 client.26495 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:11:50.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.828+0000 7f1175ffb700 1 -- 172.21.15.101:0/861905087 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55a6e8867050 con 0x55a6e89c9450 2023-07-18T03:11:50.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.828+0000 7f1175ffb700 10 client.26495 handle_command_reply: tid=0 2023-07-18T03:11:50.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.828+0000 7f11a8486b80 1 client.26495 shutdown 2023-07-18T03:11:50.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.828+0000 7f11a8486b80 1 -- 172.21.15.101:0/861905087 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6e8959be0 msgr2=0x55a6e89c1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:50.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.828+0000 7f11a8486b80 1 --2- 172.21.15.101:0/861905087 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6e8959be0 0x55a6e89c1e30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f118401d850 tx=0x7f1184005750 comp rx=0 tx=0).stop 2023-07-18T03:11:50.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.828+0000 7f11a8486b80 1 -- 172.21.15.101:0/861905087 shutdown_connections 2023-07-18T03:11:50.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.828+0000 7f11a8486b80 1 --2- 172.21.15.101:0/861905087 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a6e89c9450 0x55a6e89cb860 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.828+0000 7f11a8486b80 1 --2- 172.21.15.101:0/861905087 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a6e8964ad0 0x55a6e89bbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.828+0000 7f11a8486b80 1 --2- 172.21.15.101:0/861905087 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a6e89592d0 0x55a6e89c18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.828+0000 7f1175ffb700 0 client.26495 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:11:50.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.828+0000 7f11a8486b80 1 --2- 172.21.15.101:0/861905087 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a6e8959be0 0x55a6e89c1e30 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.828+0000 7f11a8486b80 1 -- 172.21.15.101:0/861905087 >> 172.21.15.101:0/861905087 conn(0x55a6e894fda0 msgr2=0x55a6e8963a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:50.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.828+0000 7f11a8486b80 1 -- 172.21.15.101:0/861905087 shutdown_connections 2023-07-18T03:11:50.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.828+0000 7f11a8486b80 1 -- 172.21.15.101:0/861905087 wait complete. 2023-07-18T03:11:50.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.828+0000 7f11a8486b80 20 client.26495 trim_cache size 0 max 16384 2023-07-18T03:11:50.830 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:11:50.830 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:11:50.830 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:11:50.831 INFO:teuthology.orchestra.run.smithi101.stdout: "d3886fa6-7130-456c-aed3-bcba5cd62880": { 2023-07-18T03:11:50.831 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:11:50.831 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "d3886fa6-7130-456c-aed3-bcba5cd62880", 2023-07-18T03:11:50.831 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:11:50.831 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:11:50.831 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:11:50.831 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:11:50.832 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.829+0000 7f1194bff700 1 -- 172.21.15.101:0/2474684752 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f11780a1340 msgr2=0x7f11780a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:50.832 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.829+0000 7f1194bff700 1 --2- 172.21.15.101:0/2474684752 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f11780a1340 0x7f11780a37f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f11901993f0 tx=0x7f1184009260 comp rx=0 tx=0).stop 2023-07-18T03:11:50.832 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.829+0000 7f1194bff700 1 -- 172.21.15.101:0/2474684752 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11900ffae0 msgr2=0x7f1190197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:11:50.832 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.829+0000 7f1194bff700 1 --2- 172.21.15.101:0/2474684752 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11900ffae0 0x7f1190197de0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1180005910 tx=0x7f1180005720 comp rx=0 tx=0).stop 2023-07-18T03:11:50.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.829+0000 7f1194bff700 1 -- 172.21.15.101:0/2474684752 shutdown_connections 2023-07-18T03:11:50.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.829+0000 7f1194bff700 1 --2- 172.21.15.101:0/2474684752 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f11780a1340 0x7f11780a37f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.829+0000 7f1194bff700 1 --2- 172.21.15.101:0/2474684752 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11900ffae0 0x7f1190197de0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.829+0000 7f1194bff700 1 --2- 172.21.15.101:0/2474684752 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1190105af0 0x7f119019c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.829+0000 7f1194bff700 1 --2- 172.21.15.101:0/2474684752 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1190100490 0x7f1190198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:11:50.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.829+0000 7f1194bff700 1 -- 172.21.15.101:0/2474684752 >> 172.21.15.101:0/2474684752 conn(0x7f11900fb5e0 msgr2=0x7f119010ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:11:50.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.829+0000 7f1194bff700 1 -- 172.21.15.101:0/2474684752 shutdown_connections 2023-07-18T03:11:50.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:11:50.829+0000 7f1194bff700 1 -- 172.21.15.101:0/2474684752 wait complete. 2023-07-18T03:11:50.837 INFO:tasks.cephfs.filesystem:scrub status for tag:d3886fa6-7130-456c-aed3-bcba5cd62880 - {'path': '/', 'tag': 'd3886fa6-7130-456c-aed3-bcba5cd62880', 'options': 'recursive,force'} 2023-07-18T03:11:51.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:51 smithi179 ceph-mon[129229]: pgmap v131: 129 pgs: 129 active+clean; 186 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.6 MiB/s wr, 313 op/s 2023-07-18T03:11:51.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:51 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:11:51.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:51 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:11:51.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:51 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:11:51.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:51 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:11:51.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:51 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:11:51.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:51 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:11:51.486 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:51 smithi101 ceph-mon[126379]: pgmap v131: 129 pgs: 129 active+clean; 186 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.6 MiB/s wr, 313 op/s 2023-07-18T03:11:51.486 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:51 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:11:51.486 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:51 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:11:51.486 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:51 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:11:51.486 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:51 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:11:51.486 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:51 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:11:51.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:51 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:11:51.543 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:51 smithi155 ceph-mon[128885]: pgmap v131: 129 pgs: 129 active+clean; 186 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.6 MiB/s wr, 313 op/s 2023-07-18T03:11:51.543 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:51 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:11:51.543 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:51 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:11:51.544 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:51 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:11:51.544 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:51 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:11:51.544 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:51 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:11:51.544 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:51 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:11:51.575 INFO:tasks.workunit.client.0.smithi101.stdout: 1 308937 9.08 MB/sec execute 76 sec latency 55.413 ms 2023-07-18T03:11:52.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:52 smithi179 ceph-mon[129229]: scrub queued for path: / 2023-07-18T03:11:52.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:52 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:11:52.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:52 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:11:52.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:52 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:11:52.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:52 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:11:52.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:52 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:11:52.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:52 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:11:52.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:52 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:11:52.485 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:52 smithi101 ceph-mon[126379]: scrub queued for path: / 2023-07-18T03:11:52.486 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:52 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:11:52.486 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:52 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:11:52.486 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:52 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:11:52.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:52 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:11:52.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:52 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:11:52.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:52 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:11:52.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:52 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:11:52.543 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:52 smithi155 ceph-mon[128885]: scrub queued for path: / 2023-07-18T03:11:52.544 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:52 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:11:52.544 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:52 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:11:52.544 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:52 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:11:52.544 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:52 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:11:52.544 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:52 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:11:52.544 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:52 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:11:52.544 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:52 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:11:52.576 INFO:tasks.workunit.client.0.smithi101.stdout: 1 310377 9.03 MB/sec execute 77 sec latency 14.344 ms 2023-07-18T03:11:53.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:53 smithi179 ceph-mon[129229]: pgmap v132: 129 pgs: 129 active+clean; 186 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 205 op/s 2023-07-18T03:11:53.486 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:53 smithi101 ceph-mon[126379]: pgmap v132: 129 pgs: 129 active+clean; 186 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 205 op/s 2023-07-18T03:11:53.543 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:53 smithi155 ceph-mon[128885]: pgmap v132: 129 pgs: 129 active+clean; 186 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 205 op/s 2023-07-18T03:11:53.576 INFO:tasks.workunit.client.0.smithi101.stdout: 1 312459 9.09 MB/sec execute 78 sec latency 11.702 ms 2023-07-18T03:11:54.576 INFO:tasks.workunit.client.0.smithi101.stdout: 1 313435 9.00 MB/sec execute 79 sec latency 82.806 ms 2023-07-18T03:11:55.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:55 smithi179 ceph-mon[129229]: pgmap v133: 129 pgs: 129 active+clean; 186 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 205 op/s 2023-07-18T03:11:55.486 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:55 smithi101 ceph-mon[126379]: pgmap v133: 129 pgs: 129 active+clean; 186 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 205 op/s 2023-07-18T03:11:55.543 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:55 smithi155 ceph-mon[128885]: pgmap v133: 129 pgs: 129 active+clean; 186 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 205 op/s 2023-07-18T03:11:55.576 INFO:tasks.workunit.client.0.smithi101.stdout: 1 315912 9.10 MB/sec execute 80 sec latency 20.711 ms 2023-07-18T03:11:56.576 INFO:tasks.workunit.client.0.smithi101.stdout: 1 317114 9.05 MB/sec execute 81 sec latency 13.297 ms 2023-07-18T03:11:57.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:57 smithi179 ceph-mon[129229]: pgmap v134: 129 pgs: 129 active+clean; 200 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.7 MiB/s wr, 307 op/s 2023-07-18T03:11:57.486 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:57 smithi101 ceph-mon[126379]: pgmap v134: 129 pgs: 129 active+clean; 200 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.7 MiB/s wr, 307 op/s 2023-07-18T03:11:57.543 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:57 smithi155 ceph-mon[128885]: pgmap v134: 129 pgs: 129 active+clean; 200 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.7 MiB/s wr, 307 op/s 2023-07-18T03:11:57.576 INFO:tasks.workunit.client.0.smithi101.stdout: 1 319491 9.11 MB/sec execute 82 sec latency 11.055 ms 2023-07-18T03:11:58.576 INFO:tasks.workunit.client.0.smithi101.stdout: 1 320631 9.07 MB/sec execute 83 sec latency 13.310 ms 2023-07-18T03:11:59.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:11:59 smithi179 ceph-mon[129229]: pgmap v135: 129 pgs: 129 active+clean; 200 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 210 op/s 2023-07-18T03:11:59.486 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:11:59 smithi101 ceph-mon[126379]: pgmap v135: 129 pgs: 129 active+clean; 200 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 210 op/s 2023-07-18T03:11:59.543 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:11:59 smithi155 ceph-mon[128885]: pgmap v135: 129 pgs: 129 active+clean; 200 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 210 op/s 2023-07-18T03:11:59.576 INFO:tasks.workunit.client.0.smithi101.stdout: 1 322937 9.11 MB/sec execute 84 sec latency 46.354 ms 2023-07-18T03:12:00.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:00 smithi179 ceph-mon[129229]: pgmap v136: 129 pgs: 129 active+clean; 203 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 317 op/s 2023-07-18T03:12:00.485 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:00 smithi101 ceph-mon[126379]: pgmap v136: 129 pgs: 129 active+clean; 203 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 317 op/s 2023-07-18T03:12:00.543 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:00 smithi155 ceph-mon[128885]: pgmap v136: 129 pgs: 129 active+clean; 203 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 317 op/s 2023-07-18T03:12:00.576 INFO:tasks.workunit.client.0.smithi101.stdout: 1 323803 9.03 MB/sec execute 85 sec latency 54.671 ms 2023-07-18T03:12:01.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:01 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:12:01.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:01 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:12:01.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:01 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:12:01.486 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:01 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:12:01.486 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:01 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:12:01.486 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:01 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:12:01.543 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:01 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:12:01.543 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:01 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:12:01.544 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:01 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:12:01.576 INFO:tasks.workunit.client.0.smithi101.stdout: 1 324621 8.98 MB/sec execute 86 sec latency 60.608 ms 2023-07-18T03:12:02.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:02 smithi179 ceph-mon[129229]: pgmap v137: 129 pgs: 129 active+clean; 203 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 209 op/s 2023-07-18T03:12:02.486 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:02 smithi101 ceph-mon[126379]: pgmap v137: 129 pgs: 129 active+clean; 203 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 209 op/s 2023-07-18T03:12:02.543 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:02 smithi155 ceph-mon[128885]: pgmap v137: 129 pgs: 129 active+clean; 203 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 209 op/s 2023-07-18T03:12:02.576 INFO:tasks.workunit.client.0.smithi101.stdout: 1 326608 9.03 MB/sec execute 87 sec latency 11.333 ms 2023-07-18T03:12:03.577 INFO:tasks.workunit.client.0.smithi101.stdout: 1 327870 8.99 MB/sec execute 88 sec latency 15.011 ms 2023-07-18T03:12:04.577 INFO:tasks.workunit.client.0.smithi101.stdout: 1 330022 9.03 MB/sec execute 89 sec latency 54.767 ms 2023-07-18T03:12:05.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:04 smithi179 ceph-mon[129229]: pgmap v138: 129 pgs: 129 active+clean; 203 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 209 op/s 2023-07-18T03:12:05.236 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:04 smithi101 ceph-mon[126379]: pgmap v138: 129 pgs: 129 active+clean; 203 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 209 op/s 2023-07-18T03:12:05.293 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:04 smithi155 ceph-mon[128885]: pgmap v138: 129 pgs: 129 active+clean; 203 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 209 op/s 2023-07-18T03:12:05.577 INFO:tasks.workunit.client.0.smithi101.stdout: 1 330923 8.96 MB/sec execute 90 sec latency 28.306 ms 2023-07-18T03:12:06.577 INFO:tasks.workunit.client.0.smithi101.stdout: 1 333248 9.00 MB/sec execute 91 sec latency 15.933 ms 2023-07-18T03:12:07.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:06 smithi179 ceph-mon[129229]: pgmap v139: 129 pgs: 129 active+clean; 209 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 334 op/s 2023-07-18T03:12:07.236 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:06 smithi101 ceph-mon[126379]: pgmap v139: 129 pgs: 129 active+clean; 209 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 334 op/s 2023-07-18T03:12:07.293 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:06 smithi155 ceph-mon[128885]: pgmap v139: 129 pgs: 129 active+clean; 209 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 334 op/s 2023-07-18T03:12:07.577 INFO:tasks.workunit.client.0.smithi101.stdout: 1 334229 8.96 MB/sec execute 92 sec latency 66.944 ms 2023-07-18T03:12:08.577 INFO:tasks.workunit.client.0.smithi101.stdout: 1 336437 8.99 MB/sec execute 93 sec latency 15.882 ms 2023-07-18T03:12:09.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:08 smithi179 ceph-mon[129229]: pgmap v140: 129 pgs: 129 active+clean; 209 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 232 op/s 2023-07-18T03:12:09.236 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:08 smithi101 ceph-mon[126379]: pgmap v140: 129 pgs: 129 active+clean; 209 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 232 op/s 2023-07-18T03:12:09.293 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:08 smithi155 ceph-mon[128885]: pgmap v140: 129 pgs: 129 active+clean; 209 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 232 op/s 2023-07-18T03:12:09.579 INFO:tasks.workunit.client.0.smithi101.stdout: 1 337658 8.97 MB/sec execute 94 sec latency 37.946 ms 2023-07-18T03:12:10.577 INFO:tasks.workunit.client.0.smithi101.stdout: 1 339475 8.96 MB/sec execute 95 sec latency 80.690 ms 2023-07-18T03:12:11.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:10 smithi179 ceph-mon[129229]: pgmap v141: 129 pgs: 129 active+clean; 207 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 325 op/s 2023-07-18T03:12:11.236 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:10 smithi101 ceph-mon[126379]: pgmap v141: 129 pgs: 129 active+clean; 207 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 325 op/s 2023-07-18T03:12:11.293 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:10 smithi155 ceph-mon[128885]: pgmap v141: 129 pgs: 129 active+clean; 207 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 325 op/s 2023-07-18T03:12:11.581 INFO:tasks.workunit.client.0.smithi101.stdout: 1 340894 8.98 MB/sec execute 96 sec latency 74.334 ms 2023-07-18T03:12:12.577 INFO:tasks.workunit.client.0.smithi101.stdout: 1 342210 8.94 MB/sec execute 97 sec latency 14.625 ms 2023-07-18T03:12:13.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:12 smithi179 ceph-mon[129229]: pgmap v142: 129 pgs: 129 active+clean; 207 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 218 op/s 2023-07-18T03:12:13.236 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:12 smithi101 ceph-mon[126379]: pgmap v142: 129 pgs: 129 active+clean; 207 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 218 op/s 2023-07-18T03:12:13.293 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:12 smithi155 ceph-mon[128885]: pgmap v142: 129 pgs: 129 active+clean; 207 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 218 op/s 2023-07-18T03:12:13.578 INFO:tasks.workunit.client.0.smithi101.stdout: 1 344223 8.98 MB/sec execute 98 sec latency 49.558 ms 2023-07-18T03:12:14.578 INFO:tasks.workunit.client.0.smithi101.stdout: 1 345057 8.91 MB/sec execute 99 sec latency 53.259 ms 2023-07-18T03:12:15.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:14 smithi179 ceph-mon[129229]: pgmap v143: 129 pgs: 129 active+clean; 207 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 218 op/s 2023-07-18T03:12:15.236 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:14 smithi101 ceph-mon[126379]: pgmap v143: 129 pgs: 129 active+clean; 207 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 218 op/s 2023-07-18T03:12:15.293 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:14 smithi155 ceph-mon[128885]: pgmap v143: 129 pgs: 129 active+clean; 207 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 218 op/s 2023-07-18T03:12:15.578 INFO:tasks.workunit.client.0.smithi101.stdout: 1 346657 8.89 MB/sec execute 100 sec latency 33.679 ms 2023-07-18T03:12:16.578 INFO:tasks.workunit.client.0.smithi101.stdout: 1 348164 8.91 MB/sec execute 101 sec latency 12.670 ms 2023-07-18T03:12:17.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:16 smithi179 ceph-mon[129229]: pgmap v144: 129 pgs: 129 active+clean; 212 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 317 op/s 2023-07-18T03:12:17.236 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:16 smithi101 ceph-mon[126379]: pgmap v144: 129 pgs: 129 active+clean; 212 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 317 op/s 2023-07-18T03:12:17.293 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:16 smithi155 ceph-mon[128885]: pgmap v144: 129 pgs: 129 active+clean; 212 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 317 op/s 2023-07-18T03:12:17.578 INFO:tasks.workunit.client.0.smithi101.stdout: 1 350006 8.90 MB/sec execute 102 sec latency 23.732 ms 2023-07-18T03:12:18.578 INFO:tasks.workunit.client.0.smithi101.stdout: 1 351546 8.93 MB/sec execute 103 sec latency 12.968 ms 2023-07-18T03:12:19.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:18 smithi179 ceph-mon[129229]: pgmap v145: 129 pgs: 129 active+clean; 212 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 192 op/s 2023-07-18T03:12:19.236 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:18 smithi101 ceph-mon[126379]: pgmap v145: 129 pgs: 129 active+clean; 212 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 192 op/s 2023-07-18T03:12:19.293 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:18 smithi155 ceph-mon[128885]: pgmap v145: 129 pgs: 129 active+clean; 212 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 192 op/s 2023-07-18T03:12:19.578 INFO:tasks.workunit.client.0.smithi101.stdout: 1 352853 8.89 MB/sec execute 104 sec latency 17.008 ms 2023-07-18T03:12:20.578 INFO:tasks.workunit.client.0.smithi101.stdout: 1 354817 8.93 MB/sec execute 105 sec latency 69.012 ms 2023-07-18T03:12:20.838 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:12:21.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.086+0000 7f4aa39ff700 1 -- 172.21.15.101:0/3813669291 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4a9c105af0 msgr2=0x7f4a9c105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:21.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.086+0000 7f4aa39ff700 1 --2- 172.21.15.101:0/3813669291 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4a9c105af0 0x7f4a9c105ec0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4a98009990 tx=0x7f4a98009ca0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.086+0000 7f4aa39ff700 1 -- 172.21.15.101:0/3813669291 shutdown_connections 2023-07-18T03:12:21.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.086+0000 7f4aa39ff700 1 --2- 172.21.15.101:0/3813669291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a9c100490 0x7f4a9c10c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.086+0000 7f4aa39ff700 1 --2- 172.21.15.101:0/3813669291 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4a9c105af0 0x7f4a9c105ec0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.086+0000 7f4aa39ff700 1 --2- 172.21.15.101:0/3813669291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a9c0ffae0 0x7f4a9c0fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.081 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.086+0000 7f4aa39ff700 1 -- 172.21.15.101:0/3813669291 >> 172.21.15.101:0/3813669291 conn(0x7f4a9c0fb5e0 msgr2=0x7f4a9c0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:21.081 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.086+0000 7f4aa39ff700 1 -- 172.21.15.101:0/3813669291 shutdown_connections 2023-07-18T03:12:21.081 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.086+0000 7f4aa39ff700 1 -- 172.21.15.101:0/3813669291 wait complete. 2023-07-18T03:12:21.085 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.087+0000 7f4aa39ff700 1 Processor -- start 2023-07-18T03:12:21.085 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.087+0000 7f4aa39ff700 1 -- start start 2023-07-18T03:12:21.085 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.088+0000 7f4aa39ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a9c0ffae0 0x7f4a9c197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:21.085 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.088+0000 7f4aa39ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4a9c100490 0x7f4a9c198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:21.086 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.088+0000 7f4aa39ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a9c105af0 0x7f4a9c19c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:21.086 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.088+0000 7f4aa39ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9c19cdd0 con 0x7f4a9c100490 2023-07-18T03:12:21.086 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.088+0000 7f4aa39ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9c19cf40 con 0x7f4a9c105af0 2023-07-18T03:12:21.086 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.088+0000 7f4aa39ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9c19d100 con 0x7f4a9c0ffae0 2023-07-18T03:12:21.087 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.088+0000 7f4aa179b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a9c0ffae0 0x7f4a9c197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:21.087 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.088+0000 7f4aa0f9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4a9c100490 0x7f4a9c198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:21.087 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.088+0000 7f4aa1f9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a9c105af0 0x7f4a9c19c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:21.087 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.088+0000 7f4aa0f9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4a9c100490 0x7f4a9c198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:44806/0 (socket says 172.21.15.101:44806) 2023-07-18T03:12:21.088 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.088+0000 7f4aa179b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a9c0ffae0 0x7f4a9c197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:51636/0 (socket says 172.21.15.101:51636) 2023-07-18T03:12:21.088 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.088+0000 7f4aa0f9a700 1 -- 172.21.15.101:0/2227539911 learned_addr learned my addr 172.21.15.101:0/2227539911 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:21.088 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.088+0000 7f4aa0f9a700 1 -- 172.21.15.101:0/2227539911 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a9c0ffae0 msgr2=0x7f4a9c197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:21.088 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.088+0000 7f4aa0f9a700 1 --2- 172.21.15.101:0/2227539911 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a9c0ffae0 0x7f4a9c197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.088 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.088+0000 7f4aa0f9a700 1 -- 172.21.15.101:0/2227539911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a9c105af0 msgr2=0x7f4a9c19c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:21.089 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.088+0000 7f4aa0f9a700 1 --2- 172.21.15.101:0/2227539911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a9c105af0 0x7f4a9c19c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.089 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.088+0000 7f4aa0f9a700 1 -- 172.21.15.101:0/2227539911 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a980095f0 con 0x7f4a9c100490 2023-07-18T03:12:21.089 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.088+0000 7f4aa179b700 1 --2- 172.21.15.101:0/2227539911 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a9c0ffae0 0x7f4a9c197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:12:21.089 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.088+0000 7f4aa1f9c700 1 --2- 172.21.15.101:0/2227539911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a9c105af0 0x7f4a9c19c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:12:21.089 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.089+0000 7f4aa0f9a700 1 --2- 172.21.15.101:0/2227539911 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4a9c100490 0x7f4a9c198330 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c00e9a0 tx=0x7f4a8c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:21.089 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.089+0000 7f4a927fc700 1 -- 172.21.15.101:0/2227539911 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c009790 con 0x7f4a9c100490 2023-07-18T03:12:21.089 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.089+0000 7f4aa39ff700 1 -- 172.21.15.101:0/2227539911 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a9c19d3e0 con 0x7f4a9c100490 2023-07-18T03:12:21.090 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.089+0000 7f4a927fc700 1 -- 172.21.15.101:0/2227539911 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a8c010430 con 0x7f4a9c100490 2023-07-18T03:12:21.090 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.089+0000 7f4a927fc700 1 -- 172.21.15.101:0/2227539911 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c018680 con 0x7f4a9c100490 2023-07-18T03:12:21.090 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.089+0000 7f4aa39ff700 1 -- 172.21.15.101:0/2227539911 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a9c1a4c20 con 0x7f4a9c100490 2023-07-18T03:12:21.090 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.092+0000 7f4a927fc700 1 -- 172.21.15.101:0/2227539911 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f4a8c0105a0 con 0x7f4a9c100490 2023-07-18T03:12:21.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.094+0000 7f4ab7301b80 1 Processor -- start 2023-07-18T03:12:21.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.094+0000 7f4a927fc700 1 --2- 172.21.15.101:0/2227539911 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4a880a12c0 0x7f4a880a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:21.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.094+0000 7f4a927fc700 1 -- 172.21.15.101:0/2227539911 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f4a8c014070 con 0x7f4a9c100490 2023-07-18T03:12:21.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.094+0000 7f4ab7301b80 1 -- start start 2023-07-18T03:12:21.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.094+0000 7f4aa1f9c700 1 --2- 172.21.15.101:0/2227539911 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4a880a12c0 0x7f4a880a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:21.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.094+0000 7f4ab7301b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56306607e2d0 0x56306607e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:21.096 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.094+0000 7f4ab7301b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56306607ebe0 0x563066089590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:21.096 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.094+0000 7f4ab7301b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563066089ad0 0x56306608beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:21.096 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.094+0000 7f4ab7301b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563065f84cb0 con 0x563066089ad0 2023-07-18T03:12:21.096 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.094+0000 7f4ab7301b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563065fd7a40 con 0x56306607ebe0 2023-07-18T03:12:21.096 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.094+0000 7f4ab7301b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563065fd0ed0 con 0x56306607e2d0 2023-07-18T03:12:21.096 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.094+0000 7f4aa179b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56306607e2d0 0x56306607e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:21.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.094+0000 7f4aa179b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56306607e2d0 0x56306607e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:51638/0 (socket says 172.21.15.101:51638) 2023-07-18T03:12:21.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.094+0000 7f4aa179b700 1 -- 172.21.15.101:0/957165143 learned_addr learned my addr 172.21.15.101:0/957165143 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:21.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.094+0000 7f4aa0f9a700 1 --2- 172.21.15.101:0/957165143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56306607ebe0 0x563066089590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:21.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.095+0000 7f4aa179b700 1 -- 172.21.15.101:0/957165143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56306607ebe0 msgr2=0x563066089590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:21.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.095+0000 7f4aa179b700 1 --2- 172.21.15.101:0/957165143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56306607ebe0 0x563066089590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.095+0000 7f4aa179b700 1 -- 172.21.15.101:0/957165143 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563066089ad0 msgr2=0x56306608beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:21.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.095+0000 7f4aa179b700 1 --2- 172.21.15.101:0/957165143 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563066089ad0 0x56306608beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.095+0000 7f4aa179b700 1 -- 172.21.15.101:0/957165143 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563065dfede0 con 0x56306607e2d0 2023-07-18T03:12:21.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.095+0000 7f4aa0f9a700 1 --2- 172.21.15.101:0/957165143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56306607ebe0 0x563066089590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:12:21.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.095+0000 7f4aa1f9c700 1 --2- 172.21.15.101:0/2227539911 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4a880a12c0 0x7f4a880a3770 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4a94003ea0 tx=0x7f4a94009920 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:21.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.095+0000 7f4aa179b700 1 --2- 172.21.15.101:0/957165143 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56306607e2d0 0x56306607e6a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4a98005ee0 tx=0x7f4a98005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40cac81cff05f871 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:21.100 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.095+0000 7f4a877fe700 1 -- 172.21.15.101:0/957165143 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9801d070 con 0x56306607e2d0 2023-07-18T03:12:21.100 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.095+0000 7f4a877fe700 1 -- 172.21.15.101:0/957165143 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a9801e040 con 0x56306607e2d0 2023-07-18T03:12:21.100 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.095+0000 7f4a877fe700 1 -- 172.21.15.101:0/957165143 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a98004cf0 con 0x56306607e2d0 2023-07-18T03:12:21.101 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.096+0000 7f4ab7301b80 1 -- 172.21.15.101:0/957165143 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56306607e2d0 msgr2=0x56306607e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:21.101 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.096+0000 7f4ab7301b80 1 --2- 172.21.15.101:0/957165143 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56306607e2d0 0x56306607e6a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4a98005ee0 tx=0x7f4a98005f10 comp rx=0 tx=0).stop 2023-07-18T03:12:21.101 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.096+0000 7f4ab7301b80 1 -- 172.21.15.101:0/957165143 shutdown_connections 2023-07-18T03:12:21.101 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.096+0000 7f4ab7301b80 1 --2- 172.21.15.101:0/957165143 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563066089ad0 0x56306608beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.102 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.096+0000 7f4ab7301b80 1 --2- 172.21.15.101:0/957165143 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56306607e2d0 0x56306607e6a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.102 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.096+0000 7f4ab7301b80 1 --2- 172.21.15.101:0/957165143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56306607ebe0 0x563066089590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.102 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.096+0000 7f4ab7301b80 1 -- 172.21.15.101:0/957165143 >> 172.21.15.101:0/957165143 conn(0x563066074da0 msgr2=0x56306607a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:21.102 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.096+0000 7f4ab7301b80 1 -- 172.21.15.101:0/957165143 shutdown_connections 2023-07-18T03:12:21.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.096+0000 7f4ab7301b80 1 -- 172.21.15.101:0/957165143 wait complete. 2023-07-18T03:12:21.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.097+0000 7f4ab7301b80 1 -- start start 2023-07-18T03:12:21.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.097+0000 7f4ab7301b80 1 Processor -- start 2023-07-18T03:12:21.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.097+0000 7f4ab7301b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56306607e2d0 0x5630660894a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:21.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.097+0000 7f4ab7301b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56306607ebe0 0x563066086720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:21.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.097+0000 7f4ab7301b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563066089ad0 0x563066086da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:21.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.098+0000 7f4ab7301b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563065f70ac0 con 0x563066089ad0 2023-07-18T03:12:21.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.098+0000 7f4ab7301b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563065f707c0 con 0x56306607ebe0 2023-07-18T03:12:21.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.098+0000 7f4ab7301b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563065e54920 con 0x56306607e2d0 2023-07-18T03:12:21.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.098+0000 7f4ab7301b80 10 client.0 _ll_register_callbacks cb 0x563065f6e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:12:21.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.098+0000 7f4aa179b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56306607e2d0 0x5630660894a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:21.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.098+0000 7f4aa179b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56306607e2d0 0x5630660894a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:51644/0 (socket says 172.21.15.101:51644) 2023-07-18T03:12:21.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.098+0000 7f4aa179b700 1 -- 172.21.15.101:0/1582133066 learned_addr learned my addr 172.21.15.101:0/1582133066 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:21.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.098+0000 7f4aa0f9a700 1 --2- 172.21.15.101:0/1582133066 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56306607ebe0 0x563066086720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:21.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.098+0000 7f4aa1f9c700 1 --2- 172.21.15.101:0/1582133066 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563066089ad0 0x563066086da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:21.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.098+0000 7f4aa179b700 1 -- 172.21.15.101:0/1582133066 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56306607ebe0 msgr2=0x563066086720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:21.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.098+0000 7f4aa179b700 1 --2- 172.21.15.101:0/1582133066 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56306607ebe0 0x563066086720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.098+0000 7f4aa179b700 1 -- 172.21.15.101:0/1582133066 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563066089ad0 msgr2=0x563066086da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:21.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.098+0000 7f4aa179b700 1 --2- 172.21.15.101:0/1582133066 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563066089ad0 0x563066086da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.098+0000 7f4aa179b700 1 -- 172.21.15.101:0/1582133066 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a9c1a4c20 con 0x56306607e2d0 2023-07-18T03:12:21.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.099+0000 7f4aa179b700 1 --2- 172.21.15.101:0/1582133066 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56306607e2d0 0x5630660894a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4a98009ce0 tx=0x7f4a98022b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fc0923d5880728f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:21.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.099+0000 7f4a84ff9700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:12:21.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.099+0000 7f4a84ff9700 1 -- 172.21.15.101:0/1582133066 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9801d030 con 0x56306607e2d0 2023-07-18T03:12:21.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.099+0000 7f4a84ff9700 1 -- 172.21.15.101:0/1582133066 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f4a98004500 con 0x56306607e2d0 2023-07-18T03:12:21.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.099+0000 7f4ab7301b80 1 -- 172.21.15.101:0/1582133066 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563065f2a4f0 con 0x56306607e2d0 2023-07-18T03:12:21.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.099+0000 7f4a84ff9700 1 -- 172.21.15.101:0/1582133066 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a980046f0 con 0x56306607e2d0 2023-07-18T03:12:21.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.099+0000 7f4ab7301b80 10 client.26530 fetch_fsmap learned FSMap version 26 2023-07-18T03:12:21.108 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.099+0000 7f4ab7301b80 1 -- 172.21.15.101:0/1582133066 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f4a9c10d220 con 0x56306607e2d0 2023-07-18T03:12:21.108 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.100+0000 7f4a84ff9700 1 -- 172.21.15.101:0/1582133066 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563065f2a4f0 con 0x56306607e2d0 2023-07-18T03:12:21.108 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.100+0000 7f4a84ff9700 1 -- 172.21.15.101:0/1582133066 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4a9802c490 con 0x56306607e2d0 2023-07-18T03:12:21.108 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.100+0000 7f4ab7301b80 10 client.26530 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:12:21.108 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.100+0000 7f4ab7301b80 10 client.26530 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:12:21.108 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.100+0000 7f4ab7301b80 20 client.26530 populate_metadata read hostname 'smithi101' 2023-07-18T03:12:21.108 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.100+0000 7f4ab7301b80 1 --2- 172.21.15.101:0/1582133066 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563066098f30 0x56306609b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:21.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.100+0000 7f4ab7301b80 4 client.26530 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:12:21.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.100+0000 7f4ab7301b80 1 -- 172.21.15.101:0/1582133066 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563065fc9eb0 con 0x563066098f30 2023-07-18T03:12:21.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.100+0000 7f4aa0f9a700 1 --2- 172.21.15.101:0/1582133066 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563066098f30 0x56306609b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:21.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.101+0000 7f4aa0f9a700 1 --2- 172.21.15.101:0/1582133066 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563066098f30 0x56306609b310 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1cfb2bd0e841b65 server_cookie=e0663c4a2e6b03b4 in_seq=0 out_seq=0 2023-07-18T03:12:21.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.101+0000 7f4a84ff9700 10 client.26530 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:12:21.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.101+0000 7f4a84ff9700 1 -- 172.21.15.101:0/1582133066 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563065fc9eb0 con 0x563066098f30 2023-07-18T03:12:21.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.101+0000 7f4a84ff9700 10 client.26530 handle_command_reply: tid=0 2023-07-18T03:12:21.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.101+0000 7f4ab7301b80 1 client.26530 shutdown 2023-07-18T03:12:21.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.102+0000 7f4ab7301b80 1 -- 172.21.15.101:0/1582133066 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56306607e2d0 msgr2=0x5630660894a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:21.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.102+0000 7f4ab7301b80 1 --2- 172.21.15.101:0/1582133066 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56306607e2d0 0x5630660894a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4a98009ce0 tx=0x7f4a98022b90 comp rx=0 tx=0).stop 2023-07-18T03:12:21.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.102+0000 7f4ab7301b80 1 -- 172.21.15.101:0/1582133066 shutdown_connections 2023-07-18T03:12:21.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.102+0000 7f4ab7301b80 1 --2- 172.21.15.101:0/1582133066 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563066098f30 0x56306609b310 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.102+0000 7f4ab7301b80 1 --2- 172.21.15.101:0/1582133066 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563066089ad0 0x563066086da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.102+0000 7f4ab7301b80 1 --2- 172.21.15.101:0/1582133066 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56306607e2d0 0x5630660894a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.102+0000 7f4ab7301b80 1 --2- 172.21.15.101:0/1582133066 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56306607ebe0 0x563066086720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.102+0000 7f4ab7301b80 1 -- 172.21.15.101:0/1582133066 >> 172.21.15.101:0/1582133066 conn(0x563066074da0 msgr2=0x56306607a230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:21.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.102+0000 7f4a84ff9700 0 client.26530 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:12:21.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.102+0000 7f4ab7301b80 1 -- 172.21.15.101:0/1582133066 shutdown_connections 2023-07-18T03:12:21.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.102+0000 7f4ab7301b80 1 -- 172.21.15.101:0/1582133066 wait complete. 2023-07-18T03:12:21.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.102+0000 7f4ab7301b80 20 client.26530 trim_cache size 0 max 16384 2023-07-18T03:12:21.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.119+0000 7f4ab7301b80 1 Processor -- start 2023-07-18T03:12:21.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.119+0000 7f4ab7301b80 1 -- start start 2023-07-18T03:12:21.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.120+0000 7f4ab7301b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56306607e2d0 0x563066089040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:21.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.120+0000 7f4ab7301b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56306607ebe0 0x563066081a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:21.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.120+0000 7f4ab7301b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563066089ad0 0x563066081fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:21.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.120+0000 7f4ab7301b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563066087680 con 0x563066089ad0 2023-07-18T03:12:21.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.120+0000 7f4ab7301b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563066084c80 con 0x56306607e2d0 2023-07-18T03:12:21.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.120+0000 7f4ab7301b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630660873a0 con 0x56306607ebe0 2023-07-18T03:12:21.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.120+0000 7f4aa179b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56306607e2d0 0x563066089040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:21.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.120+0000 7f4aa179b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56306607e2d0 0x563066089040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:52486/0 (socket says 172.21.15.101:52486) 2023-07-18T03:12:21.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.120+0000 7f4aa1f9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563066089ad0 0x563066081fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:21.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.120+0000 7f4aa179b700 1 -- 172.21.15.101:0/3391166873 learned_addr learned my addr 172.21.15.101:0/3391166873 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:21.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.120+0000 7f4aa0f9a700 1 --2- 172.21.15.101:0/3391166873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56306607ebe0 0x563066081a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:21.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.120+0000 7f4aa179b700 1 -- 172.21.15.101:0/3391166873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56306607ebe0 msgr2=0x563066081a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:21.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.120+0000 7f4aa179b700 1 --2- 172.21.15.101:0/3391166873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56306607ebe0 0x563066081a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.120+0000 7f4aa179b700 1 -- 172.21.15.101:0/3391166873 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563066089ad0 msgr2=0x563066081fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:21.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.120+0000 7f4aa179b700 1 --2- 172.21.15.101:0/3391166873 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563066089ad0 0x563066081fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.120+0000 7f4aa179b700 1 -- 172.21.15.101:0/3391166873 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563065de7c60 con 0x56306607e2d0 2023-07-18T03:12:21.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.121+0000 7f4aa179b700 1 --2- 172.21.15.101:0/3391166873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56306607e2d0 0x563066089040 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4a9802c7c0 tx=0x7f4a9800b2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98e728727c86b0fb server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:21.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.121+0000 7f4a87fff700 1 -- 172.21.15.101:0/3391166873 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9801d030 con 0x56306607e2d0 2023-07-18T03:12:21.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.121+0000 7f4a87fff700 1 -- 172.21.15.101:0/3391166873 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a9802c600 con 0x56306607e2d0 2023-07-18T03:12:21.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.121+0000 7f4a87fff700 1 -- 172.21.15.101:0/3391166873 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a980049c0 con 0x56306607e2d0 2023-07-18T03:12:21.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.121+0000 7f4ab7301b80 1 -- 172.21.15.101:0/3391166873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56306607e2d0 msgr2=0x563066089040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:21.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.121+0000 7f4ab7301b80 1 --2- 172.21.15.101:0/3391166873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56306607e2d0 0x563066089040 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4a9802c7c0 tx=0x7f4a9800b2c0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.121+0000 7f4ab7301b80 1 -- 172.21.15.101:0/3391166873 shutdown_connections 2023-07-18T03:12:21.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.121+0000 7f4ab7301b80 1 --2- 172.21.15.101:0/3391166873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56306607e2d0 0x563066089040 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.121+0000 7f4ab7301b80 1 --2- 172.21.15.101:0/3391166873 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563066089ad0 0x563066081fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.121+0000 7f4ab7301b80 1 --2- 172.21.15.101:0/3391166873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56306607ebe0 0x563066081a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.121+0000 7f4ab7301b80 1 -- 172.21.15.101:0/3391166873 >> 172.21.15.101:0/3391166873 conn(0x563066074da0 msgr2=0x5630660956e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:21.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.122+0000 7f4ab7301b80 1 -- 172.21.15.101:0/3391166873 shutdown_connections 2023-07-18T03:12:21.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.122+0000 7f4ab7301b80 1 -- 172.21.15.101:0/3391166873 wait complete. 2023-07-18T03:12:21.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.122+0000 7f4ab7301b80 1 -- start start 2023-07-18T03:12:21.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.122+0000 7f4ab7301b80 1 Processor -- start 2023-07-18T03:12:21.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.123+0000 7f4ab7301b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56306607e2d0 0x56306608e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:21.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.123+0000 7f4ab7301b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56306607ebe0 0x56306608e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:21.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.123+0000 7f4ab7301b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563066089ad0 0x5630660ebdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:21.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.123+0000 7f4ab7301b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56306608ed20 con 0x563066089ad0 2023-07-18T03:12:21.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.123+0000 7f4ab7301b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630660ec2f0 con 0x56306607e2d0 2023-07-18T03:12:21.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.123+0000 7f4ab7301b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630660ec430 con 0x56306607ebe0 2023-07-18T03:12:21.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.123+0000 7f4ab7301b80 10 client.0 _ll_register_callbacks cb 0x7f4a98009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:12:21.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.123+0000 7f4aa0f9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56306607ebe0 0x56306608e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:21.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.123+0000 7f4aa1f9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563066089ad0 0x5630660ebdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:21.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.123+0000 7f4aa179b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56306607e2d0 0x56306608e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:21.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.123+0000 7f4aa0f9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56306607ebe0 0x56306608e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:51662/0 (socket says 172.21.15.101:51662) 2023-07-18T03:12:21.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.123+0000 7f4aa0f9a700 1 -- 172.21.15.101:0/3307910688 learned_addr learned my addr 172.21.15.101:0/3307910688 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:21.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.123+0000 7f4aa0f9a700 1 -- 172.21.15.101:0/3307910688 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56306607e2d0 msgr2=0x56306608e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:21.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.123+0000 7f4aa0f9a700 1 --2- 172.21.15.101:0/3307910688 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56306607e2d0 0x56306608e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.123+0000 7f4aa0f9a700 1 -- 172.21.15.101:0/3307910688 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563066089ad0 msgr2=0x5630660ebdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:21.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.123+0000 7f4aa0f9a700 1 --2- 172.21.15.101:0/3307910688 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563066089ad0 0x5630660ebdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.123+0000 7f4aa0f9a700 1 -- 172.21.15.101:0/3307910688 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a9c10d220 con 0x56306607ebe0 2023-07-18T03:12:21.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.123+0000 7f4aa1f9c700 1 --2- 172.21.15.101:0/3307910688 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563066089ad0 0x5630660ebdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:12:21.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.124+0000 7f4aa0f9a700 1 --2- 172.21.15.101:0/3307910688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56306607ebe0 0x56306608e640 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c096a10 tx=0x7f4a8c0cc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d345fec9e3a06d98 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:21.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.124+0000 7f4a84ff9700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:12:21.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.124+0000 7f4a84ff9700 1 -- 172.21.15.101:0/3307910688 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c09a200 con 0x56306607ebe0 2023-07-18T03:12:21.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.124+0000 7f4a84ff9700 1 -- 172.21.15.101:0/3307910688 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f4a8c09a360 con 0x56306607ebe0 2023-07-18T03:12:21.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.124+0000 7f4ab7301b80 1 -- 172.21.15.101:0/3307910688 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a9801e040 con 0x56306607ebe0 2023-07-18T03:12:21.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.124+0000 7f4a84ff9700 1 -- 172.21.15.101:0/3307910688 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a8c0b82b0 con 0x56306607ebe0 2023-07-18T03:12:21.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.124+0000 7f4ab7301b80 10 client.26536 fetch_fsmap learned FSMap version 26 2023-07-18T03:12:21.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.124+0000 7f4ab7301b80 1 -- 172.21.15.101:0/3307910688 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x563065e25250 con 0x56306607ebe0 2023-07-18T03:12:21.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.124+0000 7f4a84ff9700 1 -- 172.21.15.101:0/3307910688 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a8c0ca080 con 0x56306607ebe0 2023-07-18T03:12:21.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.124+0000 7f4a84ff9700 1 -- 172.21.15.101:0/3307910688 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4a8c0995b0 con 0x56306607ebe0 2023-07-18T03:12:21.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.124+0000 7f4ab7301b80 10 client.26536 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:12:21.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.124+0000 7f4ab7301b80 10 client.26536 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:12:21.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.124+0000 7f4ab7301b80 20 client.26536 populate_metadata read hostname 'smithi101' 2023-07-18T03:12:21.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.124+0000 7f4ab7301b80 1 --2- 172.21.15.101:0/3307910688 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5630660edbd0 0x5630660effe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:21.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.124+0000 7f4ab7301b80 4 client.26536 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:12:21.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.124+0000 7f4ab7301b80 1 -- 172.21.15.101:0/3307910688 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563065f8c050 con 0x5630660edbd0 2023-07-18T03:12:21.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.125+0000 7f4aa179b700 1 --2- 172.21.15.101:0/3307910688 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5630660edbd0 0x5630660effe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:21.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.125+0000 7f4aa179b700 1 --2- 172.21.15.101:0/3307910688 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5630660edbd0 0x5630660effe0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94ad937fccabd5f server_cookie=fd91b7c8dd2ed156 in_seq=0 out_seq=0 2023-07-18T03:12:21.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.125+0000 7f4a84ff9700 10 client.26536 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:12:21.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.125+0000 7f4a84ff9700 1 -- 172.21.15.101:0/3307910688 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x563065f8c050 con 0x5630660edbd0 2023-07-18T03:12:21.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.125+0000 7f4a84ff9700 10 client.26536 handle_command_reply: tid=0 2023-07-18T03:12:21.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.125+0000 7f4ab7301b80 1 client.26536 shutdown 2023-07-18T03:12:21.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.125+0000 7f4ab7301b80 1 -- 172.21.15.101:0/3307910688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56306607ebe0 msgr2=0x56306608e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:21.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.125+0000 7f4ab7301b80 1 --2- 172.21.15.101:0/3307910688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56306607ebe0 0x56306608e640 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c096a10 tx=0x7f4a8c0cc040 comp rx=0 tx=0).stop 2023-07-18T03:12:21.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.125+0000 7f4ab7301b80 1 -- 172.21.15.101:0/3307910688 shutdown_connections 2023-07-18T03:12:21.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.125+0000 7f4ab7301b80 1 --2- 172.21.15.101:0/3307910688 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5630660edbd0 0x5630660effe0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.125+0000 7f4ab7301b80 1 --2- 172.21.15.101:0/3307910688 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56306607e2d0 0x56306608e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.125+0000 7f4ab7301b80 1 --2- 172.21.15.101:0/3307910688 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563066089ad0 0x5630660ebdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.125+0000 7f4ab7301b80 1 --2- 172.21.15.101:0/3307910688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56306607ebe0 0x56306608e640 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.125+0000 7f4a84ff9700 0 client.26536 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:12:21.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.125+0000 7f4ab7301b80 1 -- 172.21.15.101:0/3307910688 >> 172.21.15.101:0/3307910688 conn(0x563066074da0 msgr2=0x563066081a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:21.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.126+0000 7f4ab7301b80 1 -- 172.21.15.101:0/3307910688 shutdown_connections 2023-07-18T03:12:21.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.126+0000 7f4ab7301b80 1 -- 172.21.15.101:0/3307910688 wait complete. 2023-07-18T03:12:21.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.126+0000 7f4ab7301b80 20 client.26536 trim_cache size 0 max 16384 2023-07-18T03:12:21.128 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:12:21.128 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "no active scrubs running", 2023-07-18T03:12:21.128 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": {} 2023-07-18T03:12:21.128 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:12:21.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.126+0000 7f4aa39ff700 1 -- 172.21.15.101:0/2227539911 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4a880a12c0 msgr2=0x7f4a880a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:21.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.126+0000 7f4aa39ff700 1 --2- 172.21.15.101:0/2227539911 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4a880a12c0 0x7f4a880a3770 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4a94003ea0 tx=0x7f4a94009920 comp rx=0 tx=0).stop 2023-07-18T03:12:21.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.126+0000 7f4aa39ff700 1 -- 172.21.15.101:0/2227539911 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4a9c100490 msgr2=0x7f4a9c198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:21.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.126+0000 7f4aa39ff700 1 --2- 172.21.15.101:0/2227539911 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4a9c100490 0x7f4a9c198330 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c00e9a0 tx=0x7f4a8c00ecb0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.126+0000 7f4aa39ff700 1 -- 172.21.15.101:0/2227539911 shutdown_connections 2023-07-18T03:12:21.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.126+0000 7f4aa39ff700 1 --2- 172.21.15.101:0/2227539911 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4a880a12c0 0x7f4a880a3770 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.126+0000 7f4aa39ff700 1 --2- 172.21.15.101:0/2227539911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a9c105af0 0x7f4a9c19c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.126+0000 7f4aa39ff700 1 --2- 172.21.15.101:0/2227539911 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a9c0ffae0 0x7f4a9c197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.126+0000 7f4aa39ff700 1 --2- 172.21.15.101:0/2227539911 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4a9c100490 0x7f4a9c198330 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.126+0000 7f4aa39ff700 1 -- 172.21.15.101:0/2227539911 >> 172.21.15.101:0/2227539911 conn(0x7f4a9c0fb5e0 msgr2=0x7f4a9c10ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:21.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.126+0000 7f4aa39ff700 1 -- 172.21.15.101:0/2227539911 shutdown_connections 2023-07-18T03:12:21.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.127+0000 7f4aa39ff700 1 -- 172.21.15.101:0/2227539911 wait complete. 2023-07-18T03:12:21.136 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-18T03:12:21.137 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:12:21.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:20 smithi179 ceph-mon[129229]: pgmap v146: 129 pgs: 129 active+clean; 217 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 294 op/s 2023-07-18T03:12:21.199 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:20 smithi101 ceph-mon[126379]: pgmap v146: 129 pgs: 129 active+clean; 217 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 294 op/s 2023-07-18T03:12:21.293 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:20 smithi155 ceph-mon[128885]: pgmap v146: 129 pgs: 129 active+clean; 217 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 294 op/s 2023-07-18T03:12:21.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.365+0000 7fb2e29ff700 1 -- 172.21.15.101:0/3798142537 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb2dc102950 msgr2=0x7fb2dc106910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:21.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.365+0000 7fb2e29ff700 1 --2- 172.21.15.101:0/3798142537 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb2dc102950 0x7fb2dc106910 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb2d00099c0 tx=0x7fb2d0009cd0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.365+0000 7fb2e29ff700 1 -- 172.21.15.101:0/3798142537 shutdown_connections 2023-07-18T03:12:21.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.365+0000 7fb2e29ff700 1 --2- 172.21.15.101:0/3798142537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2dc106ff0 0x7fb2dc10ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.365+0000 7fb2e29ff700 1 --2- 172.21.15.101:0/3798142537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2dc101fb0 0x7fb2dc102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.365+0000 7fb2e29ff700 1 --2- 172.21.15.101:0/3798142537 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb2dc102950 0x7fb2dc106910 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.365+0000 7fb2e29ff700 1 -- 172.21.15.101:0/3798142537 >> 172.21.15.101:0/3798142537 conn(0x7fb2dc0fd800 msgr2=0x7fb2dc0ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:21.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.365+0000 7fb2e29ff700 1 -- 172.21.15.101:0/3798142537 shutdown_connections 2023-07-18T03:12:21.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.365+0000 7fb2e29ff700 1 -- 172.21.15.101:0/3798142537 wait complete. 2023-07-18T03:12:21.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.366+0000 7fb2e29ff700 1 Processor -- start 2023-07-18T03:12:21.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.367+0000 7fb2e29ff700 1 -- start start 2023-07-18T03:12:21.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.367+0000 7fb2e29ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2dc101fb0 0x7fb2dc197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:21.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.367+0000 7fb2e29ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2dc102950 0x7fb2dc198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:21.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.367+0000 7fb2e29ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb2dc106ff0 0x7fb2dc19c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:21.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.367+0000 7fb2e29ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc19cdc0 con 0x7fb2dc106ff0 2023-07-18T03:12:21.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.367+0000 7fb2e29ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc19cf30 con 0x7fb2dc102950 2023-07-18T03:12:21.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.367+0000 7fb2e29ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc19d0f0 con 0x7fb2dc101fb0 2023-07-18T03:12:21.365 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.367+0000 7fb2db7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2dc102950 0x7fb2dc198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:21.365 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.367+0000 7fb2dbfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2dc101fb0 0x7fb2dc197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:21.365 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.367+0000 7fb2db7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2dc102950 0x7fb2dc198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:52494/0 (socket says 172.21.15.101:52494) 2023-07-18T03:12:21.365 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.367+0000 7fb2db7fe700 1 -- 172.21.15.101:0/693338206 learned_addr learned my addr 172.21.15.101:0/693338206 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:21.365 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.367+0000 7fb2e0f9c700 1 --2- 172.21.15.101:0/693338206 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb2dc106ff0 0x7fb2dc19c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:21.365 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.368+0000 7fb2dbfff700 1 -- 172.21.15.101:0/693338206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2dc102950 msgr2=0x7fb2dc198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:21.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.368+0000 7fb2dbfff700 1 --2- 172.21.15.101:0/693338206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2dc102950 0x7fb2dc198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.368+0000 7fb2dbfff700 1 -- 172.21.15.101:0/693338206 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb2dc106ff0 msgr2=0x7fb2dc19c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:21.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.368+0000 7fb2dbfff700 1 --2- 172.21.15.101:0/693338206 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb2dc106ff0 0x7fb2dc19c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.368+0000 7fb2dbfff700 1 -- 172.21.15.101:0/693338206 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d4009c40 con 0x7fb2dc101fb0 2023-07-18T03:12:21.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.368+0000 7fb2e0f9c700 1 --2- 172.21.15.101:0/693338206 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb2dc106ff0 0x7fb2dc19c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:12:21.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.368+0000 7fb2db7fe700 1 --2- 172.21.15.101:0/693338206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2dc102950 0x7fb2dc198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:12:21.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.368+0000 7fb2dbfff700 1 --2- 172.21.15.101:0/693338206 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2dc101fb0 0x7fb2dc197de0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc00e9f0 tx=0x7fb2cc00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:21.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.368+0000 7fb2d97fa700 1 -- 172.21.15.101:0/693338206 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc00c9c0 con 0x7fb2dc101fb0 2023-07-18T03:12:21.367 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.368+0000 7fb2e29ff700 1 -- 172.21.15.101:0/693338206 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2d00095f0 con 0x7fb2dc101fb0 2023-07-18T03:12:21.367 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.368+0000 7fb2d97fa700 1 -- 172.21.15.101:0/693338206 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2cc010430 con 0x7fb2dc101fb0 2023-07-18T03:12:21.367 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.368+0000 7fb2d97fa700 1 -- 172.21.15.101:0/693338206 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc018680 con 0x7fb2dc101fb0 2023-07-18T03:12:21.367 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.368+0000 7fb2e29ff700 1 -- 172.21.15.101:0/693338206 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2dc1a4a30 con 0x7fb2dc101fb0 2023-07-18T03:12:21.367 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.369+0000 7fb2e29ff700 1 -- 172.21.15.101:0/693338206 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2dc067300 con 0x7fb2dc101fb0 2023-07-18T03:12:21.368 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.371+0000 7fb2d97fa700 1 -- 172.21.15.101:0/693338206 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fb2cc010860 con 0x7fb2dc101fb0 2023-07-18T03:12:21.368 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.373+0000 7fb2d97fa700 1 --2- 172.21.15.101:0/693338206 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fb2c40a1300 0x7fb2c40a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:21.369 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.373+0000 7fb2d97fa700 1 -- 172.21.15.101:0/693338206 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fb2cc014070 con 0x7fb2dc101fb0 2023-07-18T03:12:21.369 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.373+0000 7fb2db7fe700 1 --2- 172.21.15.101:0/693338206 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fb2c40a1300 0x7fb2c40a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:21.369 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.373+0000 7fb2db7fe700 1 --2- 172.21.15.101:0/693338206 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fb2c40a1300 0x7fb2c40a37b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb2d00036c0 tx=0x7fb2d001a040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:21.369 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.375+0000 7fb2d97fa700 1 -- 172.21.15.101:0/693338206 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb2cc0963f0 con 0x7fb2dc101fb0 2023-07-18T03:12:21.578 INFO:tasks.workunit.client.0.smithi101.stdout: 1 355680 8.86 MB/sec execute 106 sec latency 34.976 ms 2023-07-18T03:12:21.641 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.650+0000 7fb2e29ff700 1 -- 172.21.15.101:0/693338206 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb2dc1a4c50 con 0x7fb2dc101fb0 2023-07-18T03:12:21.642 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.651+0000 7fb2d97fa700 1 -- 172.21.15.101:0/693338206 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10858 (secure 0 0 0) 0x7fb2cc099a10 con 0x7fb2dc101fb0 2023-07-18T03:12:21.643 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:12:21.643 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/4192846769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":4192846769},{"type":"v1","addr":"172.21.15.101:6839","nonce":4192846769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6841/3824340175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6840","nonce":3824340175},{"type":"v1","addr":"172.21.15.101:6841","nonce":3824340175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:04:59.361185+0000","modified":"2023-07-18T03:11:46.116493+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.155:6839/2055462483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2055462483},{"type":"v1","addr":"172.21.15.155:6839","nonce":2055462483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:12:21.645 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.653+0000 7fb2e29ff700 1 -- 172.21.15.101:0/693338206 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fb2c40a1300 msgr2=0x7fb2c40a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:21.645 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.653+0000 7fb2e29ff700 1 --2- 172.21.15.101:0/693338206 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fb2c40a1300 0x7fb2c40a37b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb2d00036c0 tx=0x7fb2d001a040 comp rx=0 tx=0).stop 2023-07-18T03:12:21.645 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.654+0000 7fb2e29ff700 1 -- 172.21.15.101:0/693338206 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2dc101fb0 msgr2=0x7fb2dc197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:21.645 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.654+0000 7fb2e29ff700 1 --2- 172.21.15.101:0/693338206 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2dc101fb0 0x7fb2dc197de0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc00e9f0 tx=0x7fb2cc00ed00 comp rx=0 tx=0).stop 2023-07-18T03:12:21.645 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.654+0000 7fb2e29ff700 1 -- 172.21.15.101:0/693338206 shutdown_connections 2023-07-18T03:12:21.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.654+0000 7fb2e29ff700 1 --2- 172.21.15.101:0/693338206 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb2dc106ff0 0x7fb2dc19c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.654+0000 7fb2e29ff700 1 --2- 172.21.15.101:0/693338206 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2dc101fb0 0x7fb2dc197de0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.654+0000 7fb2e29ff700 1 --2- 172.21.15.101:0/693338206 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fb2c40a1300 0x7fb2c40a37b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.654+0000 7fb2e29ff700 1 --2- 172.21.15.101:0/693338206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2dc102950 0x7fb2dc198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.654+0000 7fb2e29ff700 1 -- 172.21.15.101:0/693338206 >> 172.21.15.101:0/693338206 conn(0x7fb2dc0fd800 msgr2=0x7fb2dc0fe1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:21.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.654+0000 7fb2e29ff700 1 -- 172.21.15.101:0/693338206 shutdown_connections 2023-07-18T03:12:21.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.654+0000 7fb2e29ff700 1 -- 172.21.15.101:0/693338206 wait complete. 2023-07-18T03:12:21.647 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 26 2023-07-18T03:12:21.664 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-18T03:12:21.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.884+0000 7fb3731ff700 1 -- 172.21.15.101:0/2584990207 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb36c101610 msgr2=0x7fb36c109b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:21.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.884+0000 7fb3731ff700 1 --2- 172.21.15.101:0/2584990207 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb36c101610 0x7fb36c109b00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb358009990 tx=0x7fb358009ca0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.884+0000 7fb3731ff700 1 -- 172.21.15.101:0/2584990207 shutdown_connections 2023-07-18T03:12:21.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.884+0000 7fb3731ff700 1 --2- 172.21.15.101:0/2584990207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb36c10a040 0x7fb36c10c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.884+0000 7fb3731ff700 1 --2- 172.21.15.101:0/2584990207 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb36c100d00 0x7fb36c1010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.884+0000 7fb3731ff700 1 --2- 172.21.15.101:0/2584990207 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb36c101610 0x7fb36c109b00 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.884+0000 7fb3731ff700 1 -- 172.21.15.101:0/2584990207 >> 172.21.15.101:0/2584990207 conn(0x7fb36c0fa4b0 msgr2=0x7fb36c0fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:21.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.884+0000 7fb3731ff700 1 -- 172.21.15.101:0/2584990207 shutdown_connections 2023-07-18T03:12:21.877 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.884+0000 7fb3731ff700 1 -- 172.21.15.101:0/2584990207 wait complete. 2023-07-18T03:12:21.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.885+0000 7fb3731ff700 1 Processor -- start 2023-07-18T03:12:21.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.885+0000 7fb3731ff700 1 -- start start 2023-07-18T03:12:21.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.886+0000 7fb3731ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb36c100d00 0x7fb36c19b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:21.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.886+0000 7fb3731ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb36c101610 0x7fb36c19bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:21.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.886+0000 7fb3731ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb36c10a040 0x7fb36c195a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:21.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.886+0000 7fb3731ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36c1960c0 con 0x7fb36c101610 2023-07-18T03:12:21.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.886+0000 7fb3731ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36c196230 con 0x7fb36c100d00 2023-07-18T03:12:21.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.886+0000 7fb3731ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36c1963f0 con 0x7fb36c10a040 2023-07-18T03:12:21.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.886+0000 7fb370f9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb36c100d00 0x7fb36c19b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:21.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.886+0000 7fb370f9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb36c100d00 0x7fb36c19b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:52508/0 (socket says 172.21.15.101:52508) 2023-07-18T03:12:21.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.886+0000 7fb36bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb36c101610 0x7fb36c19bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:21.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.886+0000 7fb370f9b700 1 -- 172.21.15.101:0/1546252647 learned_addr learned my addr 172.21.15.101:0/1546252647 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:21.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.886+0000 7fb37179c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb36c10a040 0x7fb36c195a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:21.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.886+0000 7fb37179c700 1 -- 172.21.15.101:0/1546252647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb36c100d00 msgr2=0x7fb36c19b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:21.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.886+0000 7fb37179c700 1 --2- 172.21.15.101:0/1546252647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb36c100d00 0x7fb36c19b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.886+0000 7fb37179c700 1 -- 172.21.15.101:0/1546252647 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb36c101610 msgr2=0x7fb36c19bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:21.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.886+0000 7fb37179c700 1 --2- 172.21.15.101:0/1546252647 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb36c101610 0x7fb36c19bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.886+0000 7fb37179c700 1 -- 172.21.15.101:0/1546252647 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3580095f0 con 0x7fb36c10a040 2023-07-18T03:12:21.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.886+0000 7fb370f9b700 1 --2- 172.21.15.101:0/1546252647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb36c100d00 0x7fb36c19b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:12:21.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.886+0000 7fb36bfff700 1 --2- 172.21.15.101:0/1546252647 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb36c101610 0x7fb36c19bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:12:21.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.886+0000 7fb37179c700 1 --2- 172.21.15.101:0/1546252647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb36c10a040 0x7fb36c195a50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb36400efc0 tx=0x7fb36400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:21.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.887+0000 7fb369ffb700 1 -- 172.21.15.101:0/1546252647 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb364018070 con 0x7fb36c10a040 2023-07-18T03:12:21.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.887+0000 7fb369ffb700 1 -- 172.21.15.101:0/1546252647 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb36400f040 con 0x7fb36c10a040 2023-07-18T03:12:21.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.887+0000 7fb369ffb700 1 -- 172.21.15.101:0/1546252647 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36400cd40 con 0x7fb36c10a040 2023-07-18T03:12:21.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.887+0000 7fb3731ff700 1 -- 172.21.15.101:0/1546252647 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb36c196670 con 0x7fb36c10a040 2023-07-18T03:12:21.882 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.887+0000 7fb3731ff700 1 -- 172.21.15.101:0/1546252647 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb36c1a29d0 con 0x7fb36c10a040 2023-07-18T03:12:21.882 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.890+0000 7fb369ffb700 1 -- 172.21.15.101:0/1546252647 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fb364003a00 con 0x7fb36c10a040 2023-07-18T03:12:21.882 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.891+0000 7fb386a81b80 1 Processor -- start 2023-07-18T03:12:21.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.891+0000 7fb386a81b80 1 -- start start 2023-07-18T03:12:21.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.891+0000 7fb369ffb700 1 --2- 172.21.15.101:0/1546252647 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fb35c0a1380 0x7fb35c0a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:21.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.892+0000 7fb369ffb700 1 -- 172.21.15.101:0/1546252647 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fb3640ccb40 con 0x7fb36c10a040 2023-07-18T03:12:21.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.892+0000 7fb386a81b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638c46b5eb0 0x5638c46b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:21.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.892+0000 7fb386a81b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638c46b4810 0x5638c46b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:21.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.892+0000 7fb36bfff700 1 --2- 172.21.15.101:0/1546252647 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fb35c0a1380 0x7fb35c0a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:21.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.892+0000 7fb386a81b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638c46c8170 0x5638c46ca550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:21.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.892+0000 7fb386a81b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638c45bacb0 con 0x5638c46b4810 2023-07-18T03:12:21.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.892+0000 7fb386a81b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638c460da40 con 0x5638c46c8170 2023-07-18T03:12:21.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.892+0000 7fb386a81b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638c4606ed0 con 0x5638c46b5eb0 2023-07-18T03:12:21.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.892+0000 7fb370f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638c46b5eb0 0x5638c46b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:21.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.892+0000 7fb370f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638c46b5eb0 0x5638c46b42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:51702/0 (socket says 172.21.15.101:51702) 2023-07-18T03:12:21.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.892+0000 7fb370f9b700 1 -- 172.21.15.101:0/604924040 learned_addr learned my addr 172.21.15.101:0/604924040 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:21.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.892+0000 7fb36bfff700 1 --2- 172.21.15.101:0/604924040 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638c46b4810 0x5638c46b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:21.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.892+0000 7fb37179c700 1 --2- 172.21.15.101:0/604924040 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638c46c8170 0x5638c46ca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:21.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.892+0000 7fb370f9b700 1 -- 172.21.15.101:0/604924040 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638c46c8170 msgr2=0x5638c46ca550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:21.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.893+0000 7fb370f9b700 1 --2- 172.21.15.101:0/604924040 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638c46c8170 0x5638c46ca550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.893+0000 7fb370f9b700 1 -- 172.21.15.101:0/604924040 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638c46b4810 msgr2=0x5638c46b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:21.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.893+0000 7fb370f9b700 1 --2- 172.21.15.101:0/604924040 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638c46b4810 0x5638c46b4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.893+0000 7fb370f9b700 1 -- 172.21.15.101:0/604924040 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638c4434de0 con 0x5638c46b5eb0 2023-07-18T03:12:21.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.893+0000 7fb37179c700 1 --2- 172.21.15.101:0/604924040 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638c46c8170 0x5638c46ca550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:12:21.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.893+0000 7fb370f9b700 1 --2- 172.21.15.101:0/604924040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638c46b5eb0 0x5638c46b42d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb3600117e0 tx=0x7fb360011af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4104cd96c098ba4b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:21.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.893+0000 7fb36bfff700 1 --2- 172.21.15.101:0/1546252647 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fb35c0a1380 0x7fb35c0a3830 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb35800c010 tx=0x7fb35800bc60 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:21.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.893+0000 7fb356ffd700 1 -- 172.21.15.101:0/604924040 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36000f480 con 0x5638c46b5eb0 2023-07-18T03:12:21.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.893+0000 7fb356ffd700 1 -- 172.21.15.101:0/604924040 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb36000faa0 con 0x5638c46b5eb0 2023-07-18T03:12:21.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.893+0000 7fb356ffd700 1 -- 172.21.15.101:0/604924040 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb360019990 con 0x5638c46b5eb0 2023-07-18T03:12:21.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.894+0000 7fb386a81b80 1 -- 172.21.15.101:0/604924040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638c46b5eb0 msgr2=0x5638c46b42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:21.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.894+0000 7fb386a81b80 1 --2- 172.21.15.101:0/604924040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638c46b5eb0 0x5638c46b42d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb3600117e0 tx=0x7fb360011af0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.894+0000 7fb386a81b80 1 -- 172.21.15.101:0/604924040 shutdown_connections 2023-07-18T03:12:21.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.894+0000 7fb386a81b80 1 --2- 172.21.15.101:0/604924040 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638c46c8170 0x5638c46ca550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.894+0000 7fb386a81b80 1 --2- 172.21.15.101:0/604924040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638c46b5eb0 0x5638c46b42d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.894+0000 7fb386a81b80 1 --2- 172.21.15.101:0/604924040 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638c46b4810 0x5638c46b4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.894+0000 7fb386a81b80 1 -- 172.21.15.101:0/604924040 >> 172.21.15.101:0/604924040 conn(0x5638c46aada0 msgr2=0x5638c46b06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:21.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.894+0000 7fb386a81b80 1 -- 172.21.15.101:0/604924040 shutdown_connections 2023-07-18T03:12:21.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.894+0000 7fb386a81b80 1 -- 172.21.15.101:0/604924040 wait complete. 2023-07-18T03:12:21.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.894+0000 7fb386a81b80 1 -- start start 2023-07-18T03:12:21.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.895+0000 7fb386a81b80 1 Processor -- start 2023-07-18T03:12:21.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.895+0000 7fb386a81b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638c46b4810 0x5638c46baff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:21.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.895+0000 7fb386a81b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638c46b5eb0 0x5638c46b8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:21.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.895+0000 7fb386a81b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638c46c8170 0x5638c46b8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:21.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.895+0000 7fb386a81b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638c45a6ac0 con 0x5638c46b5eb0 2023-07-18T03:12:21.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.895+0000 7fb386a81b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638c45a67c0 con 0x5638c46c8170 2023-07-18T03:12:21.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.895+0000 7fb386a81b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638c448a920 con 0x5638c46b4810 2023-07-18T03:12:21.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.895+0000 7fb386a81b80 10 client.0 _ll_register_callbacks cb 0x5638c45a4760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:12:21.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.895+0000 7fb370f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638c46b4810 0x5638c46baff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:21.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.895+0000 7fb370f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638c46b4810 0x5638c46baff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:51714/0 (socket says 172.21.15.101:51714) 2023-07-18T03:12:21.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.895+0000 7fb370f9b700 1 -- 172.21.15.101:0/697614323 learned_addr learned my addr 172.21.15.101:0/697614323 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:21.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.896+0000 7fb36bfff700 1 --2- 172.21.15.101:0/697614323 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638c46b5eb0 0x5638c46b8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:21.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.896+0000 7fb37179c700 1 --2- 172.21.15.101:0/697614323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638c46c8170 0x5638c46b8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:21.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.896+0000 7fb370f9b700 1 -- 172.21.15.101:0/697614323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638c46c8170 msgr2=0x5638c46b8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:21.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.896+0000 7fb370f9b700 1 --2- 172.21.15.101:0/697614323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638c46c8170 0x5638c46b8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.896+0000 7fb370f9b700 1 -- 172.21.15.101:0/697614323 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638c46b5eb0 msgr2=0x5638c46b8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:21.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.896+0000 7fb370f9b700 1 --2- 172.21.15.101:0/697614323 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638c46b5eb0 0x5638c46b8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.896+0000 7fb370f9b700 1 -- 172.21.15.101:0/697614323 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb360011440 con 0x5638c46b4810 2023-07-18T03:12:21.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.896+0000 7fb37179c700 1 --2- 172.21.15.101:0/697614323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638c46c8170 0x5638c46b8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:12:21.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.896+0000 7fb370f9b700 1 --2- 172.21.15.101:0/697614323 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638c46b4810 0x5638c46baff0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb360009810 tx=0x7fb36000f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be8c6b976fa870f0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:21.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.896+0000 7fb34bfff700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:12:21.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.896+0000 7fb34bfff700 1 -- 172.21.15.101:0/697614323 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3600052d0 con 0x5638c46b4810 2023-07-18T03:12:21.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.896+0000 7fb386a81b80 1 -- 172.21.15.101:0/697614323 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5638c45604f0 con 0x5638c46b4810 2023-07-18T03:12:21.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.896+0000 7fb34bfff700 1 -- 172.21.15.101:0/697614323 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fb360005430 con 0x5638c46b4810 2023-07-18T03:12:21.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.897+0000 7fb34bfff700 1 -- 172.21.15.101:0/697614323 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3600055f0 con 0x5638c46b4810 2023-07-18T03:12:21.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.897+0000 7fb386a81b80 10 client.26554 fetch_fsmap learned FSMap version 26 2023-07-18T03:12:21.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.897+0000 7fb386a81b80 1 -- 172.21.15.101:0/697614323 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5638c441dc60 con 0x5638c46b4810 2023-07-18T03:12:21.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.897+0000 7fb34bfff700 1 -- 172.21.15.101:0/697614323 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5638c45604f0 con 0x5638c46b4810 2023-07-18T03:12:21.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.897+0000 7fb34bfff700 1 -- 172.21.15.101:0/697614323 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb360019620 con 0x5638c46b4810 2023-07-18T03:12:21.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.897+0000 7fb386a81b80 10 client.26554 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:12:21.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.897+0000 7fb386a81b80 10 client.26554 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:12:21.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.897+0000 7fb386a81b80 20 client.26554 populate_metadata read hostname 'smithi101' 2023-07-18T03:12:21.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.897+0000 7fb386a81b80 1 --2- 172.21.15.101:0/697614323 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5638c46d7a70 0x5638c46d9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:21.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.897+0000 7fb386a81b80 4 client.26554 mds_command: new command op to 24541 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:12:21.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.897+0000 7fb386a81b80 1 -- 172.21.15.101:0/697614323 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5638c45ffeb0 con 0x5638c46d7a70 2023-07-18T03:12:21.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.897+0000 7fb36bfff700 1 --2- 172.21.15.101:0/697614323 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5638c46d7a70 0x5638c46d9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:21.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.898+0000 7fb34bfff700 10 client.26554 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:12:21.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.898+0000 7fb36bfff700 1 --2- 172.21.15.101:0/697614323 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5638c46d7a70 0x5638c46d9e80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7630c7cd205969bc server_cookie=715598228f995df8 in_seq=0 out_seq=0 2023-07-18T03:12:21.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.899+0000 7fb34bfff700 1 -- 172.21.15.101:0/697614323 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5638c45ffeb0 con 0x5638c46d7a70 2023-07-18T03:12:21.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.899+0000 7fb34bfff700 10 client.26554 handle_command_reply: tid=0 2023-07-18T03:12:21.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.899+0000 7fb386a81b80 1 client.26554 shutdown 2023-07-18T03:12:21.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.899+0000 7fb386a81b80 1 -- 172.21.15.101:0/697614323 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638c46b4810 msgr2=0x5638c46baff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:21.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.899+0000 7fb386a81b80 1 --2- 172.21.15.101:0/697614323 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638c46b4810 0x5638c46baff0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb360009810 tx=0x7fb36000f730 comp rx=0 tx=0).stop 2023-07-18T03:12:21.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.899+0000 7fb386a81b80 1 -- 172.21.15.101:0/697614323 shutdown_connections 2023-07-18T03:12:21.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.899+0000 7fb386a81b80 1 --2- 172.21.15.101:0/697614323 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5638c46d7a70 0x5638c46d9e80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.899+0000 7fb386a81b80 1 --2- 172.21.15.101:0/697614323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638c46c8170 0x5638c46b8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.899+0000 7fb386a81b80 1 --2- 172.21.15.101:0/697614323 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638c46b4810 0x5638c46baff0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.899+0000 7fb34bfff700 0 client.26554 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:12:21.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.899+0000 7fb386a81b80 1 --2- 172.21.15.101:0/697614323 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638c46b5eb0 0x5638c46b8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.899+0000 7fb386a81b80 1 -- 172.21.15.101:0/697614323 >> 172.21.15.101:0/697614323 conn(0x5638c46aada0 msgr2=0x5638c46b0290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:21.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.900+0000 7fb386a81b80 1 -- 172.21.15.101:0/697614323 shutdown_connections 2023-07-18T03:12:21.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.900+0000 7fb386a81b80 1 -- 172.21.15.101:0/697614323 wait complete. 2023-07-18T03:12:21.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.900+0000 7fb386a81b80 20 client.26554 trim_cache size 0 max 16384 2023-07-18T03:12:21.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.914+0000 7fb386a81b80 1 Processor -- start 2023-07-18T03:12:21.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.914+0000 7fb386a81b80 1 -- start start 2023-07-18T03:12:21.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.914+0000 7fb386a81b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638c46b4810 0x5638c46ced00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:21.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.914+0000 7fb386a81b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638c46b5eb0 0x5638c46bab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:21.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.914+0000 7fb386a81b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638c46c8170 0x5638c46c75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:21.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.914+0000 7fb386a81b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638c46b5b10 con 0x5638c46b5eb0 2023-07-18T03:12:21.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.914+0000 7fb386a81b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638c46b53d0 con 0x5638c46c8170 2023-07-18T03:12:21.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.914+0000 7fb386a81b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638c46c73a0 con 0x5638c46b4810 2023-07-18T03:12:21.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.915+0000 7fb370f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638c46b4810 0x5638c46ced00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:21.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.915+0000 7fb37179c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638c46c8170 0x5638c46c75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:21.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.915+0000 7fb370f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638c46b4810 0x5638c46ced00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:51716/0 (socket says 172.21.15.101:51716) 2023-07-18T03:12:21.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.915+0000 7fb370f9b700 1 -- 172.21.15.101:0/3741414211 learned_addr learned my addr 172.21.15.101:0/3741414211 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:21.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.915+0000 7fb36bfff700 1 --2- 172.21.15.101:0/3741414211 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638c46b5eb0 0x5638c46bab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:21.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.915+0000 7fb370f9b700 1 -- 172.21.15.101:0/3741414211 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638c46c8170 msgr2=0x5638c46c75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:21.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.915+0000 7fb370f9b700 1 --2- 172.21.15.101:0/3741414211 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638c46c8170 0x5638c46c75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.915+0000 7fb370f9b700 1 -- 172.21.15.101:0/3741414211 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638c46b5eb0 msgr2=0x5638c46bab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:21.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.915+0000 7fb370f9b700 1 --2- 172.21.15.101:0/3741414211 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638c46b5eb0 0x5638c46bab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.915+0000 7fb370f9b700 1 -- 172.21.15.101:0/3741414211 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638c46ac5e0 con 0x5638c46b4810 2023-07-18T03:12:21.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.915+0000 7fb36bfff700 1 --2- 172.21.15.101:0/3741414211 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638c46b5eb0 0x5638c46bab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:12:21.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.915+0000 7fb37179c700 1 --2- 172.21.15.101:0/3741414211 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638c46c8170 0x5638c46c75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:12:21.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.915+0000 7fb370f9b700 1 --2- 172.21.15.101:0/3741414211 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638c46b4810 0x5638c46ced00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb360011b30 tx=0x7fb36000ff20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c238220eddb311d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:21.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.919+0000 7fb3577fe700 1 -- 172.21.15.101:0/3741414211 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb360019610 con 0x5638c46b4810 2023-07-18T03:12:21.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.919+0000 7fb3577fe700 1 -- 172.21.15.101:0/3741414211 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb360009d50 con 0x5638c46b4810 2023-07-18T03:12:21.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.919+0000 7fb3577fe700 1 -- 172.21.15.101:0/3741414211 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb360003be0 con 0x5638c46b4810 2023-07-18T03:12:21.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.920+0000 7fb386a81b80 1 -- 172.21.15.101:0/3741414211 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638c46b4810 msgr2=0x5638c46ced00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:21.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.920+0000 7fb386a81b80 1 --2- 172.21.15.101:0/3741414211 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638c46b4810 0x5638c46ced00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb360011b30 tx=0x7fb36000ff20 comp rx=0 tx=0).stop 2023-07-18T03:12:21.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.920+0000 7fb386a81b80 1 -- 172.21.15.101:0/3741414211 shutdown_connections 2023-07-18T03:12:21.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.920+0000 7fb386a81b80 1 --2- 172.21.15.101:0/3741414211 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638c46c8170 0x5638c46c75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.920+0000 7fb386a81b80 1 --2- 172.21.15.101:0/3741414211 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638c46b4810 0x5638c46ced00 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.920+0000 7fb386a81b80 1 --2- 172.21.15.101:0/3741414211 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638c46b5eb0 0x5638c46bab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.920+0000 7fb386a81b80 1 -- 172.21.15.101:0/3741414211 >> 172.21.15.101:0/3741414211 conn(0x5638c46aada0 msgr2=0x5638c46c4150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:21.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.920+0000 7fb386a81b80 1 -- 172.21.15.101:0/3741414211 shutdown_connections 2023-07-18T03:12:21.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.920+0000 7fb386a81b80 1 -- 172.21.15.101:0/3741414211 wait complete. 2023-07-18T03:12:21.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.920+0000 7fb386a81b80 1 -- start start 2023-07-18T03:12:21.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.921+0000 7fb386a81b80 1 Processor -- start 2023-07-18T03:12:21.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.921+0000 7fb386a81b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638c46b4810 0x5638c46cca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:21.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.921+0000 7fb386a81b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638c46b5eb0 0x5638c46ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:21.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.921+0000 7fb386a81b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638c46c8170 0x5638c472a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:21.916 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.922+0000 7fb386a81b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638c472ae70 con 0x5638c46c8170 2023-07-18T03:12:21.916 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.922+0000 7fb386a81b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638c472afb0 con 0x5638c46b5eb0 2023-07-18T03:12:21.916 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.922+0000 7fb386a81b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638c472b0f0 con 0x5638c46b4810 2023-07-18T03:12:21.916 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.922+0000 7fb386a81b80 10 client.0 _ll_register_callbacks cb 0x7fb360005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:12:21.916 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.922+0000 7fb370f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638c46b4810 0x5638c46cca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:21.916 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.922+0000 7fb36bfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638c46b5eb0 0x5638c46ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:21.916 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.922+0000 7fb370f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638c46b4810 0x5638c46cca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:51722/0 (socket says 172.21.15.101:51722) 2023-07-18T03:12:21.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.922+0000 7fb370f9b700 1 -- 172.21.15.101:0/741547028 learned_addr learned my addr 172.21.15.101:0/741547028 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:21.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.922+0000 7fb37179c700 1 --2- 172.21.15.101:0/741547028 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638c46c8170 0x5638c472a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:21.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.922+0000 7fb370f9b700 1 -- 172.21.15.101:0/741547028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638c46b5eb0 msgr2=0x5638c46ccfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:21.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.922+0000 7fb370f9b700 1 --2- 172.21.15.101:0/741547028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638c46b5eb0 0x5638c46ccfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.922+0000 7fb370f9b700 1 -- 172.21.15.101:0/741547028 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638c46c8170 msgr2=0x5638c472a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:21.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.922+0000 7fb370f9b700 1 --2- 172.21.15.101:0/741547028 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638c46c8170 0x5638c472a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.922+0000 7fb370f9b700 1 -- 172.21.15.101:0/741547028 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb358034780 con 0x5638c46b4810 2023-07-18T03:12:21.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.922+0000 7fb36bfff700 1 --2- 172.21.15.101:0/741547028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638c46b5eb0 0x5638c46ccfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:12:21.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.922+0000 7fb37179c700 1 --2- 172.21.15.101:0/741547028 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638c46c8170 0x5638c472a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:12:21.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.922+0000 7fb370f9b700 1 --2- 172.21.15.101:0/741547028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638c46b4810 0x5638c46cca90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb360005e90 tx=0x7fb360019be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4704c02075b17a57 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:21.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.922+0000 7fb34bfff700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:12:21.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.923+0000 7fb34bfff700 1 -- 172.21.15.101:0/741547028 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3600056e0 con 0x5638c46b4810 2023-07-18T03:12:21.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.923+0000 7fb34bfff700 1 -- 172.21.15.101:0/741547028 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fb360000e40 con 0x5638c46b4810 2023-07-18T03:12:21.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.923+0000 7fb386a81b80 1 -- 172.21.15.101:0/741547028 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb36000faa0 con 0x5638c46b4810 2023-07-18T03:12:21.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.923+0000 7fb34bfff700 1 -- 172.21.15.101:0/741547028 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb360019ca0 con 0x5638c46b4810 2023-07-18T03:12:21.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.923+0000 7fb386a81b80 10 client.26560 fetch_fsmap learned FSMap version 26 2023-07-18T03:12:21.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.923+0000 7fb386a81b80 1 -- 172.21.15.101:0/741547028 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7fb36c1a29d0 con 0x5638c46b4810 2023-07-18T03:12:21.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.923+0000 7fb34bfff700 1 -- 172.21.15.101:0/741547028 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb36000faa0 con 0x5638c46b4810 2023-07-18T03:12:21.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.923+0000 7fb34bfff700 1 -- 172.21.15.101:0/741547028 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb360022760 con 0x5638c46b4810 2023-07-18T03:12:21.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.923+0000 7fb386a81b80 10 client.26560 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:12:21.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.923+0000 7fb386a81b80 10 client.26560 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:12:21.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.923+0000 7fb386a81b80 20 client.26560 populate_metadata read hostname 'smithi101' 2023-07-18T03:12:21.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.923+0000 7fb386a81b80 1 --2- 172.21.15.101:0/741547028 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5638c46d7a70 0x5638c472e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:21.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.923+0000 7fb386a81b80 4 client.26560 mds_command: new command op to 24541 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:12:21.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.923+0000 7fb386a81b80 1 -- 172.21.15.101:0/741547028 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5638c45c2050 con 0x5638c46d7a70 2023-07-18T03:12:21.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.924+0000 7fb36bfff700 1 --2- 172.21.15.101:0/741547028 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5638c46d7a70 0x5638c472e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:21.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.924+0000 7fb36bfff700 1 --2- 172.21.15.101:0/741547028 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5638c46d7a70 0x5638c472e380 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f451b27c6df71a3c server_cookie=8bdca6df572ef61d in_seq=0 out_seq=0 2023-07-18T03:12:21.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.924+0000 7fb34bfff700 10 client.26560 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:12:21.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.924+0000 7fb34bfff700 1 -- 172.21.15.101:0/741547028 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5638c45c2050 con 0x5638c46d7a70 2023-07-18T03:12:21.922 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.924+0000 7fb34bfff700 10 client.26560 handle_command_reply: tid=0 2023-07-18T03:12:21.922 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.924+0000 7fb386a81b80 1 client.26560 shutdown 2023-07-18T03:12:21.922 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.925+0000 7fb386a81b80 1 -- 172.21.15.101:0/741547028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638c46b4810 msgr2=0x5638c46cca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:21.922 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.925+0000 7fb386a81b80 1 --2- 172.21.15.101:0/741547028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638c46b4810 0x5638c46cca90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb360005e90 tx=0x7fb360019be0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.922 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.925+0000 7fb386a81b80 1 -- 172.21.15.101:0/741547028 shutdown_connections 2023-07-18T03:12:21.923 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.925+0000 7fb386a81b80 1 --2- 172.21.15.101:0/741547028 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5638c46d7a70 0x5638c472e380 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.923 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.925+0000 7fb386a81b80 1 --2- 172.21.15.101:0/741547028 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638c46c8170 0x5638c472a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.923 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.925+0000 7fb34bfff700 0 client.26560 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:12:21.923 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.925+0000 7fb386a81b80 1 --2- 172.21.15.101:0/741547028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638c46b4810 0x5638c46cca90 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.923 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.925+0000 7fb386a81b80 1 --2- 172.21.15.101:0/741547028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638c46b5eb0 0x5638c46ccfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.924 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.925+0000 7fb386a81b80 1 -- 172.21.15.101:0/741547028 >> 172.21.15.101:0/741547028 conn(0x5638c46aada0 msgr2=0x5638c46c75f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:21.924 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.925+0000 7fb386a81b80 1 -- 172.21.15.101:0/741547028 shutdown_connections 2023-07-18T03:12:21.924 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.925+0000 7fb386a81b80 1 -- 172.21.15.101:0/741547028 wait complete. 2023-07-18T03:12:21.924 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.925+0000 7fb386a81b80 20 client.26560 trim_cache size 0 max 16384 2023-07-18T03:12:21.925 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:12:21.925 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.925+0000 7fb3731ff700 1 -- 172.21.15.101:0/1546252647 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fb35c0a1380 msgr2=0x7fb35c0a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:21.925 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.926+0000 7fb3731ff700 1 --2- 172.21.15.101:0/1546252647 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fb35c0a1380 0x7fb35c0a3830 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb35800c010 tx=0x7fb35800bc60 comp rx=0 tx=0).stop 2023-07-18T03:12:21.925 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.926+0000 7fb3731ff700 1 -- 172.21.15.101:0/1546252647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb36c10a040 msgr2=0x7fb36c195a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:21.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.926+0000 7fb3731ff700 1 --2- 172.21.15.101:0/1546252647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb36c10a040 0x7fb36c195a50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb36400efc0 tx=0x7fb36400c3a0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.926+0000 7fb3731ff700 1 -- 172.21.15.101:0/1546252647 shutdown_connections 2023-07-18T03:12:21.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.926+0000 7fb3731ff700 1 --2- 172.21.15.101:0/1546252647 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fb35c0a1380 0x7fb35c0a3830 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.926+0000 7fb3731ff700 1 --2- 172.21.15.101:0/1546252647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb36c100d00 0x7fb36c19b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.926+0000 7fb3731ff700 1 --2- 172.21.15.101:0/1546252647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb36c10a040 0x7fb36c195a50 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.926+0000 7fb3731ff700 1 --2- 172.21.15.101:0/1546252647 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb36c101610 0x7fb36c19bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:21.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.926+0000 7fb3731ff700 1 -- 172.21.15.101:0/1546252647 >> 172.21.15.101:0/1546252647 conn(0x7fb36c0fa4b0 msgr2=0x7fb36c10c180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:21.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.926+0000 7fb3731ff700 1 -- 172.21.15.101:0/1546252647 shutdown_connections 2023-07-18T03:12:21.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:21.926+0000 7fb3731ff700 1 -- 172.21.15.101:0/1546252647 wait complete. 2023-07-18T03:12:21.932 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:21 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/693338206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:12:21.935 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-18T03:12:22.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.151+0000 7f473c5ff700 1 -- 172.21.15.101:0/2438946647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f473410a070 msgr2=0x7f473410c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:22.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.151+0000 7f473c5ff700 1 --2- 172.21.15.101:0/2438946647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f473410a070 0x7f473410c4b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f472c00b0a0 tx=0x7f472c00b3b0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.151+0000 7f473c5ff700 1 -- 172.21.15.101:0/2438946647 shutdown_connections 2023-07-18T03:12:22.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.151+0000 7f473c5ff700 1 --2- 172.21.15.101:0/2438946647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f473410a070 0x7f473410c4b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.151+0000 7f473c5ff700 1 --2- 172.21.15.101:0/2438946647 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4734100ca0 0x7f4734101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.151+0000 7f473c5ff700 1 --2- 172.21.15.101:0/2438946647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47341015b0 0x7f4734109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.151+0000 7f473c5ff700 1 -- 172.21.15.101:0/2438946647 >> 172.21.15.101:0/2438946647 conn(0x7f47340fa510 msgr2=0x7f47340fc920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:22.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.151+0000 7f473c5ff700 1 -- 172.21.15.101:0/2438946647 shutdown_connections 2023-07-18T03:12:22.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.151+0000 7f473c5ff700 1 -- 172.21.15.101:0/2438946647 wait complete. 2023-07-18T03:12:22.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.152+0000 7f473c5ff700 1 Processor -- start 2023-07-18T03:12:22.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.152+0000 7f473c5ff700 1 -- start start 2023-07-18T03:12:22.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.153+0000 7f473c5ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4734100ca0 0x7f4734198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:22.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.153+0000 7f473c5ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f47341015b0 0x7f4734198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:22.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.153+0000 7f473c5ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f473410a070 0x7f473419c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:22.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.153+0000 7f473c5ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473419d000 con 0x7f47341015b0 2023-07-18T03:12:22.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.153+0000 7f473c5ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473419d170 con 0x7f4734100ca0 2023-07-18T03:12:22.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.153+0000 7f473c5ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473419d330 con 0x7f473410a070 2023-07-18T03:12:22.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.153+0000 7f4739b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f47341015b0 0x7f4734198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:22.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.153+0000 7f4739b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f47341015b0 0x7f4734198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:44924/0 (socket says 172.21.15.101:44924) 2023-07-18T03:12:22.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.153+0000 7f473ab9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f473410a070 0x7f473419c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:22.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.153+0000 7f4739b9a700 1 -- 172.21.15.101:0/4273901994 learned_addr learned my addr 172.21.15.101:0/4273901994 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:22.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.153+0000 7f473a39b700 1 --2- 172.21.15.101:0/4273901994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4734100ca0 0x7f4734198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:22.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.153+0000 7f473ab9c700 1 -- 172.21.15.101:0/4273901994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4734100ca0 msgr2=0x7f4734198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:22.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.153+0000 7f473ab9c700 1 --2- 172.21.15.101:0/4273901994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4734100ca0 0x7f4734198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.153+0000 7f473ab9c700 1 -- 172.21.15.101:0/4273901994 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f47341015b0 msgr2=0x7f4734198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:22.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.153+0000 7f473ab9c700 1 --2- 172.21.15.101:0/4273901994 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f47341015b0 0x7f4734198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.153+0000 7f473ab9c700 1 -- 172.21.15.101:0/4273901994 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f472c009d00 con 0x7f473410a070 2023-07-18T03:12:22.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.154+0000 7f473ab9c700 1 --2- 172.21.15.101:0/4273901994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f473410a070 0x7f473419c9c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f472c009fd0 tx=0x7f472c007930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:22.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.154+0000 7f472b7fe700 1 -- 172.21.15.101:0/4273901994 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472c00e040 con 0x7f473410a070 2023-07-18T03:12:22.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.154+0000 7f473c5ff700 1 -- 172.21.15.101:0/4273901994 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f473419d5b0 con 0x7f473410a070 2023-07-18T03:12:22.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.154+0000 7f472b7fe700 1 -- 172.21.15.101:0/4273901994 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f472c01d470 con 0x7f473410a070 2023-07-18T03:12:22.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.154+0000 7f472b7fe700 1 -- 172.21.15.101:0/4273901994 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472c0052a0 con 0x7f473410a070 2023-07-18T03:12:22.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.154+0000 7f473c5ff700 1 -- 172.21.15.101:0/4273901994 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47341a4e50 con 0x7f473410a070 2023-07-18T03:12:22.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.157+0000 7f472b7fe700 1 -- 172.21.15.101:0/4273901994 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f472c01d5e0 con 0x7f473410a070 2023-07-18T03:12:22.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.158+0000 7f474fe1fb80 1 Processor -- start 2023-07-18T03:12:22.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.158+0000 7f474fe1fb80 1 -- start start 2023-07-18T03:12:22.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.159+0000 7f474fe1fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f6d51912d0 0x55f6d51916a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:22.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.159+0000 7f472b7fe700 1 --2- 172.21.15.101:0/4273901994 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f47200a12c0 0x7f47200a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:22.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.159+0000 7f474fe1fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6d5191be0 0x55f6d519c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:22.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.159+0000 7f474fe1fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6d519cad0 0x55f6d519eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:22.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.159+0000 7f474fe1fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d5097cb0 con 0x55f6d51912d0 2023-07-18T03:12:22.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.159+0000 7f474fe1fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d50eaa40 con 0x55f6d519cad0 2023-07-18T03:12:22.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.159+0000 7f474fe1fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d50e3ed0 con 0x55f6d5191be0 2023-07-18T03:12:22.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.159+0000 7f472b7fe700 1 -- 172.21.15.101:0/4273901994 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f472c0ce380 con 0x7f473410a070 2023-07-18T03:12:22.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.159+0000 7f473ab9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6d519cad0 0x55f6d519eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:22.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.159+0000 7f473ab9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6d519cad0 0x55f6d519eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:52556/0 (socket says 172.21.15.101:52556) 2023-07-18T03:12:22.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.159+0000 7f473ab9c700 1 -- 172.21.15.101:0/3568813232 learned_addr learned my addr 172.21.15.101:0/3568813232 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:22.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.159+0000 7f4739b9a700 1 --2- 172.21.15.101:0/3568813232 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f6d51912d0 0x55f6d51916a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:22.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.159+0000 7f473a39b700 1 --2- 172.21.15.101:0/4273901994 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f47200a12c0 0x7f47200a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:22.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.159+0000 7f4739b9a700 1 --2- 172.21.15.101:0/3568813232 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6d5191be0 0x55f6d519c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:22.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.159+0000 7f473ab9c700 1 -- 172.21.15.101:0/3568813232 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6d5191be0 msgr2=0x55f6d519c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:22.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.159+0000 7f473ab9c700 1 --2- 172.21.15.101:0/3568813232 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6d5191be0 0x55f6d519c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.159+0000 7f473ab9c700 1 -- 172.21.15.101:0/3568813232 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f6d51912d0 msgr2=0x55f6d51916a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:22.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.159+0000 7f473ab9c700 1 --2- 172.21.15.101:0/3568813232 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f6d51912d0 0x55f6d51916a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.159+0000 7f473ab9c700 1 -- 172.21.15.101:0/3568813232 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6d4f11de0 con 0x55f6d519cad0 2023-07-18T03:12:22.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.159+0000 7f4739b9a700 1 --2- 172.21.15.101:0/3568813232 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6d5191be0 0x55f6d519c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:12:22.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.160+0000 7f473ab9c700 1 --2- 172.21.15.101:0/3568813232 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6d519cad0 0x55f6d519eeb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f472c09ae30 tx=0x7f472c01dc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9745edfa833dec4 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:22.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.160+0000 7f473a39b700 1 --2- 172.21.15.101:0/4273901994 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f47200a12c0 0x7f47200a3770 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4730009c60 tx=0x7f4730009260 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:22.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.160+0000 7f4728bfe700 1 -- 172.21.15.101:0/3568813232 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472c09b130 con 0x55f6d519cad0 2023-07-18T03:12:22.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.160+0000 7f4728bfe700 1 -- 172.21.15.101:0/3568813232 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f472c09f160 con 0x55f6d519cad0 2023-07-18T03:12:22.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.160+0000 7f4728bfe700 1 -- 172.21.15.101:0/3568813232 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472c01db00 con 0x55f6d519cad0 2023-07-18T03:12:22.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.161+0000 7f474fe1fb80 1 -- 172.21.15.101:0/3568813232 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6d519cad0 msgr2=0x55f6d519eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:22.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.161+0000 7f474fe1fb80 1 --2- 172.21.15.101:0/3568813232 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6d519cad0 0x55f6d519eeb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f472c09ae30 tx=0x7f472c01dc80 comp rx=0 tx=0).stop 2023-07-18T03:12:22.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.161+0000 7f474fe1fb80 1 -- 172.21.15.101:0/3568813232 shutdown_connections 2023-07-18T03:12:22.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.161+0000 7f474fe1fb80 1 --2- 172.21.15.101:0/3568813232 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6d519cad0 0x55f6d519eeb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.161+0000 7f474fe1fb80 1 --2- 172.21.15.101:0/3568813232 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f6d51912d0 0x55f6d51916a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.161+0000 7f474fe1fb80 1 --2- 172.21.15.101:0/3568813232 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6d5191be0 0x55f6d519c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.161+0000 7f474fe1fb80 1 -- 172.21.15.101:0/3568813232 >> 172.21.15.101:0/3568813232 conn(0x55f6d5187da0 msgr2=0x55f6d518d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:22.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.161+0000 7f474fe1fb80 1 -- 172.21.15.101:0/3568813232 shutdown_connections 2023-07-18T03:12:22.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.161+0000 7f474fe1fb80 1 -- 172.21.15.101:0/3568813232 wait complete. 2023-07-18T03:12:22.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.161+0000 7f474fe1fb80 1 -- start start 2023-07-18T03:12:22.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.161+0000 7f474fe1fb80 1 Processor -- start 2023-07-18T03:12:22.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.162+0000 7f474fe1fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6d51912d0 0x55f6d519cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:22.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.162+0000 7f474fe1fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6d5191be0 0x55f6d519d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:22.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.162+0000 7f474fe1fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f6d519cad0 0x55f6d51a7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:22.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.162+0000 7f474fe1fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d5083ac0 con 0x55f6d519cad0 2023-07-18T03:12:22.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.162+0000 7f474fe1fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d50837c0 con 0x55f6d5191be0 2023-07-18T03:12:22.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.162+0000 7f474fe1fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d4f67920 con 0x55f6d51912d0 2023-07-18T03:12:22.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.162+0000 7f474fe1fb80 10 client.0 _ll_register_callbacks cb 0x55f6d5081760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:12:22.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.162+0000 7f473a39b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6d51912d0 0x55f6d519cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:22.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.162+0000 7f473a39b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6d51912d0 0x55f6d519cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:51762/0 (socket says 172.21.15.101:51762) 2023-07-18T03:12:22.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.162+0000 7f473a39b700 1 -- 172.21.15.101:0/1519557974 learned_addr learned my addr 172.21.15.101:0/1519557974 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:22.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.162+0000 7f473ab9c700 1 --2- 172.21.15.101:0/1519557974 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f6d519cad0 0x55f6d51a7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:22.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.162+0000 7f4739b9a700 1 --2- 172.21.15.101:0/1519557974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6d5191be0 0x55f6d519d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:22.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.163+0000 7f473a39b700 1 -- 172.21.15.101:0/1519557974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6d5191be0 msgr2=0x55f6d519d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:22.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.163+0000 7f473a39b700 1 --2- 172.21.15.101:0/1519557974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6d5191be0 0x55f6d519d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.163+0000 7f473a39b700 1 -- 172.21.15.101:0/1519557974 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f6d519cad0 msgr2=0x55f6d51a7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:22.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.163+0000 7f473a39b700 1 --2- 172.21.15.101:0/1519557974 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f6d519cad0 0x55f6d51a7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.163+0000 7f473a39b700 1 -- 172.21.15.101:0/1519557974 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f472c06b040 con 0x55f6d51912d0 2023-07-18T03:12:22.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.163+0000 7f4739b9a700 1 --2- 172.21.15.101:0/1519557974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6d5191be0 0x55f6d519d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:12:22.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.163+0000 7f473a39b700 1 --2- 172.21.15.101:0/1519557974 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6d51912d0 0x55f6d519cea0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f473000eb10 tx=0x7f47300047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85b142d85a742f1d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:22.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.163+0000 7f471dffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:12:22.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.163+0000 7f471dffb700 1 -- 172.21.15.101:0/1519557974 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473001cbc0 con 0x55f6d51912d0 2023-07-18T03:12:22.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.163+0000 7f474fe1fb80 1 -- 172.21.15.101:0/1519557974 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f6d503d4f0 con 0x55f6d51912d0 2023-07-18T03:12:22.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.163+0000 7f471dffb700 1 -- 172.21.15.101:0/1519557974 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f4730023070 con 0x55f6d51912d0 2023-07-18T03:12:22.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.164+0000 7f471dffb700 1 -- 172.21.15.101:0/1519557974 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f473001cd20 con 0x55f6d51912d0 2023-07-18T03:12:22.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.164+0000 7f474fe1fb80 10 client.26569 fetch_fsmap learned FSMap version 26 2023-07-18T03:12:22.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.164+0000 7f474fe1fb80 1 -- 172.21.15.101:0/1519557974 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55f6d4efac60 con 0x55f6d51912d0 2023-07-18T03:12:22.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.164+0000 7f471dffb700 1 -- 172.21.15.101:0/1519557974 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f6d503d4f0 con 0x55f6d51912d0 2023-07-18T03:12:22.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.164+0000 7f471dffb700 1 -- 172.21.15.101:0/1519557974 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f473001d410 con 0x55f6d51912d0 2023-07-18T03:12:22.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.164+0000 7f474fe1fb80 10 client.26569 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:12:22.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.164+0000 7f474fe1fb80 10 client.26569 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:12:22.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.164+0000 7f474fe1fb80 20 client.26569 populate_metadata read hostname 'smithi101' 2023-07-18T03:12:22.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.164+0000 7f474fe1fb80 1 --2- 172.21.15.101:0/1519557974 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55f6d51ac1b0 0x55f6d51ae5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:22.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.164+0000 7f474fe1fb80 4 client.26569 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:12:22.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.164+0000 7f474fe1fb80 1 -- 172.21.15.101:0/1519557974 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f6d50dceb0 con 0x55f6d51ac1b0 2023-07-18T03:12:22.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.164+0000 7f4739b9a700 1 --2- 172.21.15.101:0/1519557974 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55f6d51ac1b0 0x55f6d51ae5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:22.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.165+0000 7f471dffb700 10 client.26569 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:12:22.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.165+0000 7f4739b9a700 1 --2- 172.21.15.101:0/1519557974 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55f6d51ac1b0 0x55f6d51ae5c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b8b77632c6870e68 server_cookie=67df2b3cb9a427c7 in_seq=0 out_seq=0 2023-07-18T03:12:22.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.165+0000 7f471dffb700 1 -- 172.21.15.101:0/1519557974 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f6d50dceb0 con 0x55f6d51ac1b0 2023-07-18T03:12:22.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.165+0000 7f471dffb700 10 client.26569 handle_command_reply: tid=0 2023-07-18T03:12:22.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.166+0000 7f474fe1fb80 1 client.26569 shutdown 2023-07-18T03:12:22.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.166+0000 7f474fe1fb80 1 -- 172.21.15.101:0/1519557974 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6d51912d0 msgr2=0x55f6d519cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:22.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.166+0000 7f474fe1fb80 1 --2- 172.21.15.101:0/1519557974 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6d51912d0 0x55f6d519cea0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f473000eb10 tx=0x7f47300047c0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.166+0000 7f474fe1fb80 1 -- 172.21.15.101:0/1519557974 shutdown_connections 2023-07-18T03:12:22.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.166+0000 7f474fe1fb80 1 --2- 172.21.15.101:0/1519557974 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55f6d51ac1b0 0x55f6d51ae5c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.166+0000 7f474fe1fb80 1 --2- 172.21.15.101:0/1519557974 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f6d519cad0 0x55f6d51a7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.166+0000 7f474fe1fb80 1 --2- 172.21.15.101:0/1519557974 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6d51912d0 0x55f6d519cea0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.166+0000 7f474fe1fb80 1 --2- 172.21.15.101:0/1519557974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6d5191be0 0x55f6d519d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.166+0000 7f471dffb700 0 client.26569 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:12:22.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.166+0000 7f474fe1fb80 1 -- 172.21.15.101:0/1519557974 >> 172.21.15.101:0/1519557974 conn(0x55f6d5187da0 msgr2=0x55f6d518d290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:22.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.166+0000 7f474fe1fb80 1 -- 172.21.15.101:0/1519557974 shutdown_connections 2023-07-18T03:12:22.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.166+0000 7f474fe1fb80 1 -- 172.21.15.101:0/1519557974 wait complete. 2023-07-18T03:12:22.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.166+0000 7f474fe1fb80 20 client.26569 trim_cache size 0 max 16384 2023-07-18T03:12:22.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.180+0000 7f474fe1fb80 1 Processor -- start 2023-07-18T03:12:22.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.180+0000 7f474fe1fb80 1 -- start start 2023-07-18T03:12:22.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.180+0000 7f474fe1fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6d51912d0 0x55f6d51a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:22.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.180+0000 7f474fe1fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f6d5191be0 0x55f6d519b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:22.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.180+0000 7f474fe1fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6d519cad0 0x55f6d5196f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:22.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.180+0000 7f474fe1fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d5196a60 con 0x55f6d5191be0 2023-07-18T03:12:22.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.180+0000 7f474fe1fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d5187970 con 0x55f6d51912d0 2023-07-18T03:12:22.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.180+0000 7f474fe1fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d519c270 con 0x55f6d519cad0 2023-07-18T03:12:22.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.181+0000 7f473a39b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6d51912d0 0x55f6d51a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:22.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.181+0000 7f473a39b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6d51912d0 0x55f6d51a1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:52572/0 (socket says 172.21.15.101:52572) 2023-07-18T03:12:22.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.181+0000 7f473ab9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6d519cad0 0x55f6d5196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:22.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.181+0000 7f473a39b700 1 -- 172.21.15.101:0/2205661741 learned_addr learned my addr 172.21.15.101:0/2205661741 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:22.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.181+0000 7f4739b9a700 1 --2- 172.21.15.101:0/2205661741 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f6d5191be0 0x55f6d519b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:22.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.181+0000 7f473a39b700 1 -- 172.21.15.101:0/2205661741 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6d519cad0 msgr2=0x55f6d5196f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:22.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.181+0000 7f473a39b700 1 --2- 172.21.15.101:0/2205661741 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6d519cad0 0x55f6d5196f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.181+0000 7f473a39b700 1 -- 172.21.15.101:0/2205661741 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f6d5191be0 msgr2=0x55f6d519b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:22.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.181+0000 7f473a39b700 1 --2- 172.21.15.101:0/2205661741 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f6d5191be0 0x55f6d519b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.181+0000 7f473a39b700 1 -- 172.21.15.101:0/2205661741 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6d4f38250 con 0x55f6d51912d0 2023-07-18T03:12:22.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.181+0000 7f473a39b700 1 --2- 172.21.15.101:0/2205661741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6d51912d0 0x55f6d51a1e90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4730005a80 tx=0x7f4730009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8755dfc210554a71 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:22.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.181+0000 7f47293ff700 1 -- 172.21.15.101:0/2205661741 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4730005c20 con 0x55f6d51912d0 2023-07-18T03:12:22.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.182+0000 7f47293ff700 1 -- 172.21.15.101:0/2205661741 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4730021070 con 0x55f6d51912d0 2023-07-18T03:12:22.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.182+0000 7f47293ff700 1 -- 172.21.15.101:0/2205661741 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4730004040 con 0x55f6d51912d0 2023-07-18T03:12:22.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.182+0000 7f474fe1fb80 1 -- 172.21.15.101:0/2205661741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6d51912d0 msgr2=0x55f6d51a1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:22.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.182+0000 7f474fe1fb80 1 --2- 172.21.15.101:0/2205661741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6d51912d0 0x55f6d51a1e90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4730005a80 tx=0x7f4730009ed0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.182+0000 7f474fe1fb80 1 -- 172.21.15.101:0/2205661741 shutdown_connections 2023-07-18T03:12:22.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.182+0000 7f474fe1fb80 1 --2- 172.21.15.101:0/2205661741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6d51912d0 0x55f6d51a1e90 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.182+0000 7f474fe1fb80 1 --2- 172.21.15.101:0/2205661741 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6d519cad0 0x55f6d5196f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.182+0000 7f474fe1fb80 1 --2- 172.21.15.101:0/2205661741 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f6d5191be0 0x55f6d519b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.182+0000 7f474fe1fb80 1 -- 172.21.15.101:0/2205661741 >> 172.21.15.101:0/2205661741 conn(0x55f6d5187da0 msgr2=0x55f6d519cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:22.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.182+0000 7f474fe1fb80 1 -- 172.21.15.101:0/2205661741 shutdown_connections 2023-07-18T03:12:22.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.182+0000 7f474fe1fb80 1 -- 172.21.15.101:0/2205661741 wait complete. 2023-07-18T03:12:22.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.183+0000 7f474fe1fb80 1 -- start start 2023-07-18T03:12:22.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.183+0000 7f474fe1fb80 1 Processor -- start 2023-07-18T03:12:22.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.183+0000 7f474fe1fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6d51912d0 0x55f6d51a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:22.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.183+0000 7f474fe1fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6d5191be0 0x55f6d519e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:22.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.183+0000 7f474fe1fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f6d519cad0 0x55f6d519e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:22.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.183+0000 7f474fe1fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d519ea90 con 0x55f6d519cad0 2023-07-18T03:12:22.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.183+0000 7f474fe1fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d519ec00 con 0x55f6d51912d0 2023-07-18T03:12:22.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.183+0000 7f474fe1fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6d519edc0 con 0x55f6d5191be0 2023-07-18T03:12:22.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.183+0000 7f474fe1fb80 10 client.0 _ll_register_callbacks cb 0x55f6d5081760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:12:22.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.184+0000 7f4739b9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6d5191be0 0x55f6d519e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:22.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.184+0000 7f4739b9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6d5191be0 0x55f6d519e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:51784/0 (socket says 172.21.15.101:51784) 2023-07-18T03:12:22.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.184+0000 7f4739b9a700 1 -- 172.21.15.101:0/2421251526 learned_addr learned my addr 172.21.15.101:0/2421251526 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:22.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.184+0000 7f473ab9c700 1 --2- 172.21.15.101:0/2421251526 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f6d519cad0 0x55f6d519e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:22.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.184+0000 7f473a39b700 1 --2- 172.21.15.101:0/2421251526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6d51912d0 0x55f6d51a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:22.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.184+0000 7f4739b9a700 1 -- 172.21.15.101:0/2421251526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6d51912d0 msgr2=0x55f6d51a1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:22.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.184+0000 7f4739b9a700 1 --2- 172.21.15.101:0/2421251526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6d51912d0 0x55f6d51a1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.184+0000 7f4739b9a700 1 -- 172.21.15.101:0/2421251526 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f6d519cad0 msgr2=0x55f6d519e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:22.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.184+0000 7f4739b9a700 1 --2- 172.21.15.101:0/2421251526 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f6d519cad0 0x55f6d519e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.184+0000 7f4739b9a700 1 -- 172.21.15.101:0/2421251526 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f6d4efac60 con 0x55f6d5191be0 2023-07-18T03:12:22.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.184+0000 7f4739b9a700 1 --2- 172.21.15.101:0/2421251526 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6d5191be0 0x55f6d519e010 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4724009880 tx=0x7f4724020a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4673d49fa386239 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:22.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.184+0000 7f471dffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:12:22.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.184+0000 7f471dffb700 1 -- 172.21.15.101:0/2421251526 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4724004520 con 0x55f6d5191be0 2023-07-18T03:12:22.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.184+0000 7f474fe1fb80 1 -- 172.21.15.101:0/2421251526 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f472c09f160 con 0x55f6d5191be0 2023-07-18T03:12:22.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.184+0000 7f471dffb700 1 -- 172.21.15.101:0/2421251526 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f4724004830 con 0x55f6d5191be0 2023-07-18T03:12:22.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.185+0000 7f471dffb700 1 -- 172.21.15.101:0/2421251526 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47240049f0 con 0x55f6d5191be0 2023-07-18T03:12:22.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.185+0000 7f474fe1fb80 10 client.26575 fetch_fsmap learned FSMap version 26 2023-07-18T03:12:22.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.185+0000 7f474fe1fb80 1 -- 172.21.15.101:0/2421251526 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55f6d4f102b0 con 0x55f6d5191be0 2023-07-18T03:12:22.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.185+0000 7f471dffb700 1 -- 172.21.15.101:0/2421251526 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f472c09f160 con 0x55f6d5191be0 2023-07-18T03:12:22.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.185+0000 7f471dffb700 1 -- 172.21.15.101:0/2421251526 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f47240108f0 con 0x55f6d5191be0 2023-07-18T03:12:22.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.185+0000 7f474fe1fb80 10 client.26575 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:12:22.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.185+0000 7f474fe1fb80 10 client.26575 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:12:22.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.185+0000 7f474fe1fb80 20 client.26575 populate_metadata read hostname 'smithi101' 2023-07-18T03:12:22.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.185+0000 7f474fe1fb80 1 --2- 172.21.15.101:0/2421251526 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55f6d5201570 0x55f6d5203980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:22.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.185+0000 7f474fe1fb80 4 client.26575 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:12:22.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.185+0000 7f474fe1fb80 1 -- 172.21.15.101:0/2421251526 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f6d509f050 con 0x55f6d5201570 2023-07-18T03:12:22.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.185+0000 7f473a39b700 1 --2- 172.21.15.101:0/2421251526 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55f6d5201570 0x55f6d5203980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:22.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.186+0000 7f471dffb700 10 client.26575 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:12:22.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.186+0000 7f473a39b700 1 --2- 172.21.15.101:0/2421251526 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55f6d5201570 0x55f6d5203980 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d1e80990e9fb5eb7 server_cookie=1a3e650d83c081a in_seq=0 out_seq=0 2023-07-18T03:12:22.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.186+0000 7f471dffb700 1 -- 172.21.15.101:0/2421251526 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f6d509f050 con 0x55f6d5201570 2023-07-18T03:12:22.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.186+0000 7f471dffb700 10 client.26575 handle_command_reply: tid=0 2023-07-18T03:12:22.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.186+0000 7f474fe1fb80 1 client.26575 shutdown 2023-07-18T03:12:22.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.186+0000 7f474fe1fb80 1 -- 172.21.15.101:0/2421251526 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6d5191be0 msgr2=0x55f6d519e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:22.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.186+0000 7f474fe1fb80 1 --2- 172.21.15.101:0/2421251526 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6d5191be0 0x55f6d519e010 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4724009880 tx=0x7f4724020a40 comp rx=0 tx=0).stop 2023-07-18T03:12:22.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.186+0000 7f474fe1fb80 1 -- 172.21.15.101:0/2421251526 shutdown_connections 2023-07-18T03:12:22.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.186+0000 7f474fe1fb80 1 --2- 172.21.15.101:0/2421251526 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55f6d5201570 0x55f6d5203980 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.186+0000 7f474fe1fb80 1 --2- 172.21.15.101:0/2421251526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f6d51912d0 0x55f6d51a1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.186+0000 7f474fe1fb80 1 --2- 172.21.15.101:0/2421251526 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f6d519cad0 0x55f6d519e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.186+0000 7f471dffb700 0 client.26575 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:12:22.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.186+0000 7f474fe1fb80 1 --2- 172.21.15.101:0/2421251526 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6d5191be0 0x55f6d519e010 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.186+0000 7f474fe1fb80 1 -- 172.21.15.101:0/2421251526 >> 172.21.15.101:0/2421251526 conn(0x55f6d5187da0 msgr2=0x55f6d5196f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:22.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.186+0000 7f474fe1fb80 1 -- 172.21.15.101:0/2421251526 shutdown_connections 2023-07-18T03:12:22.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.186+0000 7f474fe1fb80 1 -- 172.21.15.101:0/2421251526 wait complete. 2023-07-18T03:12:22.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.187+0000 7f474fe1fb80 20 client.26575 trim_cache size 0 max 16384 2023-07-18T03:12:22.187 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:12:22.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.187+0000 7f473c5ff700 1 -- 172.21.15.101:0/4273901994 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f47200a12c0 msgr2=0x7f47200a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:22.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.187+0000 7f473c5ff700 1 --2- 172.21.15.101:0/4273901994 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f47200a12c0 0x7f47200a3770 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4730009c60 tx=0x7f4730009260 comp rx=0 tx=0).stop 2023-07-18T03:12:22.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.187+0000 7f473c5ff700 1 -- 172.21.15.101:0/4273901994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f473410a070 msgr2=0x7f473419c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:22.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.187+0000 7f473c5ff700 1 --2- 172.21.15.101:0/4273901994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f473410a070 0x7f473419c9c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f472c009fd0 tx=0x7f472c007930 comp rx=0 tx=0).stop 2023-07-18T03:12:22.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.187+0000 7f473c5ff700 1 -- 172.21.15.101:0/4273901994 shutdown_connections 2023-07-18T03:12:22.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.187+0000 7f473c5ff700 1 --2- 172.21.15.101:0/4273901994 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f47200a12c0 0x7f47200a3770 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.187+0000 7f473c5ff700 1 --2- 172.21.15.101:0/4273901994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4734100ca0 0x7f4734198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.187+0000 7f473c5ff700 1 --2- 172.21.15.101:0/4273901994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f473410a070 0x7f473419c9c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.187+0000 7f473c5ff700 1 --2- 172.21.15.101:0/4273901994 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f47341015b0 0x7f4734198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.187+0000 7f473c5ff700 1 -- 172.21.15.101:0/4273901994 >> 172.21.15.101:0/4273901994 conn(0x7f47340fa510 msgr2=0x7f47340fa8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:22.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.187+0000 7f473c5ff700 1 -- 172.21.15.101:0/4273901994 shutdown_connections 2023-07-18T03:12:22.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.187+0000 7f473c5ff700 1 -- 172.21.15.101:0/4273901994 wait complete. 2023-07-18T03:12:22.197 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-18T03:12:22.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:21 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/693338206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:12:22.293 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:21 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/693338206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:12:22.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.428+0000 7f0ae87ff700 1 -- 172.21.15.101:0/4274769612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ae00ffad0 msgr2=0x7f0ae00fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:22.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.428+0000 7f0ae87ff700 1 --2- 172.21.15.101:0/4274769612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ae00ffad0 0x7f0ae00fff40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0009990 tx=0x7f0ad0009ca0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.428+0000 7f0ae87ff700 1 -- 172.21.15.101:0/4274769612 shutdown_connections 2023-07-18T03:12:22.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.428+0000 7f0ae87ff700 1 --2- 172.21.15.101:0/4274769612 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0ae0100480 0x7f0ae010c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.428+0000 7f0ae87ff700 1 --2- 172.21.15.101:0/4274769612 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ae0105ae0 0x7f0ae0105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.428+0000 7f0ae87ff700 1 --2- 172.21.15.101:0/4274769612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ae00ffad0 0x7f0ae00fff40 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.428+0000 7f0ae87ff700 1 -- 172.21.15.101:0/4274769612 >> 172.21.15.101:0/4274769612 conn(0x7f0ae00fb5f0 msgr2=0x7f0ae00fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:22.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.428+0000 7f0ae87ff700 1 -- 172.21.15.101:0/4274769612 shutdown_connections 2023-07-18T03:12:22.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.428+0000 7f0ae87ff700 1 -- 172.21.15.101:0/4274769612 wait complete. 2023-07-18T03:12:22.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.429+0000 7f0ae87ff700 1 Processor -- start 2023-07-18T03:12:22.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.430+0000 7f0ae87ff700 1 -- start start 2023-07-18T03:12:22.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.430+0000 7f0ae87ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0ae00ffad0 0x7f0ae0072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:22.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.430+0000 7f0ae87ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ae0100480 0x7f0ae0072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:22.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.430+0000 7f0ae87ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ae0105ae0 0x7f0ae01a3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:22.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.430+0000 7f0ae87ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae0073580 con 0x7f0ae00ffad0 2023-07-18T03:12:22.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.430+0000 7f0ae87ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae01a4310 con 0x7f0ae0100480 2023-07-18T03:12:22.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.430+0000 7f0ae87ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae01a4450 con 0x7f0ae0105ae0 2023-07-18T03:12:22.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.430+0000 7f0ae5d9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ae0100480 0x7f0ae0072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:22.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.430+0000 7f0ae5d9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ae0100480 0x7f0ae0072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:52594/0 (socket says 172.21.15.101:52594) 2023-07-18T03:12:22.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.430+0000 7f0ae5d9a700 1 -- 172.21.15.101:0/1970387782 learned_addr learned my addr 172.21.15.101:0/1970387782 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:22.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.430+0000 7f0ae659b700 1 --2- 172.21.15.101:0/1970387782 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0ae00ffad0 0x7f0ae0072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:22.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.430+0000 7f0ae6d9c700 1 --2- 172.21.15.101:0/1970387782 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ae0105ae0 0x7f0ae01a3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:22.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.431+0000 7f0ae5d9a700 1 -- 172.21.15.101:0/1970387782 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ae0105ae0 msgr2=0x7f0ae01a3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:22.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.431+0000 7f0ae5d9a700 1 --2- 172.21.15.101:0/1970387782 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ae0105ae0 0x7f0ae01a3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.431+0000 7f0ae5d9a700 1 -- 172.21.15.101:0/1970387782 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0ae00ffad0 msgr2=0x7f0ae0072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:22.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.431+0000 7f0ae5d9a700 1 --2- 172.21.15.101:0/1970387782 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0ae00ffad0 0x7f0ae0072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.431+0000 7f0ae5d9a700 1 -- 172.21.15.101:0/1970387782 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad00095f0 con 0x7f0ae0100480 2023-07-18T03:12:22.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.431+0000 7f0ae5d9a700 1 --2- 172.21.15.101:0/1970387782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ae0100480 0x7f0ae0072d50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0005910 tx=0x7f0ad0004710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:22.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.431+0000 7f0ad77fe700 1 -- 172.21.15.101:0/1970387782 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad001d070 con 0x7f0ae0100480 2023-07-18T03:12:22.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.431+0000 7f0ad77fe700 1 -- 172.21.15.101:0/1970387782 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ad0022470 con 0x7f0ae0100480 2023-07-18T03:12:22.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.431+0000 7f0ad77fe700 1 -- 172.21.15.101:0/1970387782 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0017650 con 0x7f0ae0100480 2023-07-18T03:12:22.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.431+0000 7f0ae87ff700 1 -- 172.21.15.101:0/1970387782 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ae01a4650 con 0x7f0ae0100480 2023-07-18T03:12:22.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.431+0000 7f0ae87ff700 1 -- 172.21.15.101:0/1970387782 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ae01a4a20 con 0x7f0ae0100480 2023-07-18T03:12:22.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.434+0000 7f0ad77fe700 1 -- 172.21.15.101:0/1970387782 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f0ad00225e0 con 0x7f0ae0100480 2023-07-18T03:12:22.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.436+0000 7f0ad77fe700 1 --2- 172.21.15.101:0/1970387782 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0acc0a12c0 0x7f0acc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:22.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.436+0000 7f0afc141b80 1 Processor -- start 2023-07-18T03:12:22.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.436+0000 7f0ad77fe700 1 -- 172.21.15.101:0/1970387782 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f0ad00ce370 con 0x7f0ae0100480 2023-07-18T03:12:22.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.436+0000 7f0ae659b700 1 --2- 172.21.15.101:0/1970387782 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0acc0a12c0 0x7f0acc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:22.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.436+0000 7f0afc141b80 1 -- start start 2023-07-18T03:12:22.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.437+0000 7f0afc141b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562856a312d0 0x562856a316a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:22.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.437+0000 7f0afc141b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562856a31be0 0x562856a3c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:22.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.437+0000 7f0afc141b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562856a3cad0 0x562856a3eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:22.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.437+0000 7f0afc141b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562856937cb0 con 0x562856a3cad0 2023-07-18T03:12:22.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.437+0000 7f0afc141b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56285698aa40 con 0x562856a31be0 2023-07-18T03:12:22.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.437+0000 7f0afc141b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562856983ed0 con 0x562856a312d0 2023-07-18T03:12:22.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.437+0000 7f0ae659b700 1 --2- 172.21.15.101:0/1970387782 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0acc0a12c0 0x7f0acc0a3770 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0adc009be0 tx=0x7f0adc009260 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:22.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.437+0000 7f0ae5d9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562856a31be0 0x562856a3c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:22.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.437+0000 7f0ae5d9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562856a31be0 0x562856a3c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:52602/0 (socket says 172.21.15.101:52602) 2023-07-18T03:12:22.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.437+0000 7f0ae5d9a700 1 -- 172.21.15.101:0/2852970346 learned_addr learned my addr 172.21.15.101:0/2852970346 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:22.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.437+0000 7f0ae659b700 1 --2- 172.21.15.101:0/2852970346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562856a312d0 0x562856a316a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:22.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.437+0000 7f0ae6d9c700 1 --2- 172.21.15.101:0/2852970346 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562856a3cad0 0x562856a3eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:22.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.438+0000 7f0ae659b700 1 -- 172.21.15.101:0/2852970346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562856a31be0 msgr2=0x562856a3c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:22.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.438+0000 7f0ae659b700 1 --2- 172.21.15.101:0/2852970346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562856a31be0 0x562856a3c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.438+0000 7f0ae659b700 1 -- 172.21.15.101:0/2852970346 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562856a3cad0 msgr2=0x562856a3eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:22.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.438+0000 7f0ae659b700 1 --2- 172.21.15.101:0/2852970346 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562856a3cad0 0x562856a3eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.432 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.438+0000 7f0ae659b700 1 -- 172.21.15.101:0/2852970346 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628567b1de0 con 0x562856a312d0 2023-07-18T03:12:22.432 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.438+0000 7f0ae5d9a700 1 --2- 172.21.15.101:0/2852970346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562856a31be0 0x562856a3c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:12:22.432 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.438+0000 7f0ae659b700 1 --2- 172.21.15.101:0/2852970346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562856a312d0 0x562856a316a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0adc0117d0 tx=0x7f0adc014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd2cdfe7c58f925d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:22.432 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.438+0000 7f0ad4ff9700 1 -- 172.21.15.101:0/2852970346 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0adc004020 con 0x562856a312d0 2023-07-18T03:12:22.432 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.438+0000 7f0ad4ff9700 1 -- 172.21.15.101:0/2852970346 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0adc00e840 con 0x562856a312d0 2023-07-18T03:12:22.433 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.438+0000 7f0ad4ff9700 1 -- 172.21.15.101:0/2852970346 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0adc003a10 con 0x562856a312d0 2023-07-18T03:12:22.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.439+0000 7f0afc141b80 1 -- 172.21.15.101:0/2852970346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562856a312d0 msgr2=0x562856a316a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:22.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.439+0000 7f0afc141b80 1 --2- 172.21.15.101:0/2852970346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562856a312d0 0x562856a316a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0adc0117d0 tx=0x7f0adc014000 comp rx=0 tx=0).stop 2023-07-18T03:12:22.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.439+0000 7f0afc141b80 1 -- 172.21.15.101:0/2852970346 shutdown_connections 2023-07-18T03:12:22.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.439+0000 7f0afc141b80 1 --2- 172.21.15.101:0/2852970346 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562856a3cad0 0x562856a3eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.439+0000 7f0afc141b80 1 --2- 172.21.15.101:0/2852970346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562856a312d0 0x562856a316a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.439+0000 7f0afc141b80 1 --2- 172.21.15.101:0/2852970346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562856a31be0 0x562856a3c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.439+0000 7f0afc141b80 1 -- 172.21.15.101:0/2852970346 >> 172.21.15.101:0/2852970346 conn(0x562856a27da0 msgr2=0x562856a2d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:22.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.439+0000 7f0afc141b80 1 -- 172.21.15.101:0/2852970346 shutdown_connections 2023-07-18T03:12:22.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.439+0000 7f0afc141b80 1 -- 172.21.15.101:0/2852970346 wait complete. 2023-07-18T03:12:22.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.440+0000 7f0afc141b80 1 -- start start 2023-07-18T03:12:22.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.440+0000 7f0afc141b80 1 Processor -- start 2023-07-18T03:12:22.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.441+0000 7f0afc141b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562856a312d0 0x562856a3c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:22.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.441+0000 7f0afc141b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562856a31be0 0x562856a397b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:22.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.441+0000 7f0afc141b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562856a3cad0 0x562856a39e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:22.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.441+0000 7f0afc141b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562856923ac0 con 0x562856a3cad0 2023-07-18T03:12:22.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.441+0000 7f0afc141b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628569237c0 con 0x562856a31be0 2023-07-18T03:12:22.440 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.441+0000 7f0afc141b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562856807920 con 0x562856a312d0 2023-07-18T03:12:22.440 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.441+0000 7f0afc141b80 10 client.0 _ll_register_callbacks cb 0x562856921760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:12:22.440 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.441+0000 7f0ae659b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562856a312d0 0x562856a3c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:22.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.441+0000 7f0ae5d9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562856a31be0 0x562856a397b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:22.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.441+0000 7f0ae659b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562856a312d0 0x562856a3c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:51812/0 (socket says 172.21.15.101:51812) 2023-07-18T03:12:22.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.441+0000 7f0ae659b700 1 -- 172.21.15.101:0/4179330047 learned_addr learned my addr 172.21.15.101:0/4179330047 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:22.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.441+0000 7f0ae6d9c700 1 --2- 172.21.15.101:0/4179330047 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562856a3cad0 0x562856a39e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:22.442 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.441+0000 7f0ae5d9a700 1 -- 172.21.15.101:0/4179330047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562856a312d0 msgr2=0x562856a3c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:22.442 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.441+0000 7f0ae5d9a700 1 --2- 172.21.15.101:0/4179330047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562856a312d0 0x562856a3c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.442 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.441+0000 7f0ae5d9a700 1 -- 172.21.15.101:0/4179330047 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562856a3cad0 msgr2=0x562856a39e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:22.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.441+0000 7f0ae5d9a700 1 --2- 172.21.15.101:0/4179330047 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562856a3cad0 0x562856a39e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.441+0000 7f0ae5d9a700 1 -- 172.21.15.101:0/4179330047 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ae01a4a20 con 0x562856a31be0 2023-07-18T03:12:22.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.441+0000 7f0ae659b700 1 --2- 172.21.15.101:0/4179330047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562856a312d0 0x562856a3c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:12:22.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.442+0000 7f0ae5d9a700 1 --2- 172.21.15.101:0/4179330047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562856a31be0 0x562856a397b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0006010 tx=0x7f0ad009ad50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d07200ac174f9c05 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:22.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.442+0000 7f0ac9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:12:22.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.442+0000 7f0ac9ffb700 1 -- 172.21.15.101:0/4179330047 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad009a810 con 0x562856a31be0 2023-07-18T03:12:22.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.442+0000 7f0ac9ffb700 1 -- 172.21.15.101:0/4179330047 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f0ad009b120 con 0x562856a31be0 2023-07-18T03:12:22.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.442+0000 7f0afc141b80 1 -- 172.21.15.101:0/4179330047 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5628568dd4f0 con 0x562856a31be0 2023-07-18T03:12:22.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.442+0000 7f0ac9ffb700 1 -- 172.21.15.101:0/4179330047 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ad009f150 con 0x562856a31be0 2023-07-18T03:12:22.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.442+0000 7f0afc141b80 10 client.26567 fetch_fsmap learned FSMap version 26 2023-07-18T03:12:22.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.442+0000 7f0afc141b80 1 -- 172.21.15.101:0/4179330047 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f0adc017070 con 0x562856a31be0 2023-07-18T03:12:22.446 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.442+0000 7f0ac9ffb700 1 -- 172.21.15.101:0/4179330047 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ad00c9080 con 0x562856a31be0 2023-07-18T03:12:22.446 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.442+0000 7f0ac9ffb700 1 -- 172.21.15.101:0/4179330047 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0ad000fb10 con 0x562856a31be0 2023-07-18T03:12:22.446 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.442+0000 7f0afc141b80 10 client.26567 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:12:22.446 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.442+0000 7f0afc141b80 10 client.26567 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:12:22.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.443+0000 7f0afc141b80 20 client.26567 populate_metadata read hostname 'smithi101' 2023-07-18T03:12:22.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.443+0000 7f0afc141b80 1 --2- 172.21.15.101:0/4179330047 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x562856a4bf80 0x562856a4e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:22.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.443+0000 7f0afc141b80 4 client.26567 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:12:22.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.443+0000 7f0afc141b80 1 -- 172.21.15.101:0/4179330047 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56285697ceb0 con 0x562856a4bf80 2023-07-18T03:12:22.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.443+0000 7f0ae659b700 1 --2- 172.21.15.101:0/4179330047 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x562856a4bf80 0x562856a4e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:22.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.443+0000 7f0ac9ffb700 10 client.26567 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:12:22.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.443+0000 7f0ae659b700 1 --2- 172.21.15.101:0/4179330047 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x562856a4bf80 0x562856a4e390 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c3623873bd6b563 server_cookie=ec3a329e10bd401e in_seq=0 out_seq=0 2023-07-18T03:12:22.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.444+0000 7f0ac9ffb700 1 -- 172.21.15.101:0/4179330047 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56285697ceb0 con 0x562856a4bf80 2023-07-18T03:12:22.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.444+0000 7f0ac9ffb700 10 client.26567 handle_command_reply: tid=0 2023-07-18T03:12:22.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.444+0000 7f0afc141b80 1 client.26567 shutdown 2023-07-18T03:12:22.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.444+0000 7f0afc141b80 1 -- 172.21.15.101:0/4179330047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562856a31be0 msgr2=0x562856a397b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:22.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.444+0000 7f0afc141b80 1 --2- 172.21.15.101:0/4179330047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562856a31be0 0x562856a397b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0006010 tx=0x7f0ad009ad50 comp rx=0 tx=0).stop 2023-07-18T03:12:22.450 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.444+0000 7f0afc141b80 1 -- 172.21.15.101:0/4179330047 shutdown_connections 2023-07-18T03:12:22.450 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.444+0000 7f0afc141b80 1 --2- 172.21.15.101:0/4179330047 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x562856a4bf80 0x562856a4e390 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.450 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.444+0000 7f0afc141b80 1 --2- 172.21.15.101:0/4179330047 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562856a3cad0 0x562856a39e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.450 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.444+0000 7f0ac9ffb700 0 client.26567 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:12:22.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.444+0000 7f0afc141b80 1 --2- 172.21.15.101:0/4179330047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562856a312d0 0x562856a3c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.444+0000 7f0afc141b80 1 --2- 172.21.15.101:0/4179330047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562856a31be0 0x562856a397b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.444+0000 7f0afc141b80 1 -- 172.21.15.101:0/4179330047 >> 172.21.15.101:0/4179330047 conn(0x562856a27da0 msgr2=0x562856a2d230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:22.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.444+0000 7f0afc141b80 1 -- 172.21.15.101:0/4179330047 shutdown_connections 2023-07-18T03:12:22.452 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.445+0000 7f0afc141b80 1 -- 172.21.15.101:0/4179330047 wait complete. 2023-07-18T03:12:22.452 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.445+0000 7f0afc141b80 20 client.26567 trim_cache size 0 max 16384 2023-07-18T03:12:22.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.460+0000 7f0afc141b80 1 Processor -- start 2023-07-18T03:12:22.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.460+0000 7f0afc141b80 1 -- start start 2023-07-18T03:12:22.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.460+0000 7f0afc141b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562856a312d0 0x562856a487b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:22.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.460+0000 7f0afc141b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562856a31be0 0x562856a3c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:22.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.460+0000 7f0afc141b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562856a3cad0 0x562856a38f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:22.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.460+0000 7f0afc141b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562856a3ad20 con 0x562856a31be0 2023-07-18T03:12:22.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.460+0000 7f0afc141b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562856a24fa0 con 0x562856a3cad0 2023-07-18T03:12:22.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.460+0000 7f0afc141b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562856a27970 con 0x562856a312d0 2023-07-18T03:12:22.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.460+0000 7f0ae5d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562856a31be0 0x562856a3c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:22.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.460+0000 7f0ae5d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562856a31be0 0x562856a3c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:44974/0 (socket says 172.21.15.101:44974) 2023-07-18T03:12:22.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.460+0000 7f0ae5d9a700 1 -- 172.21.15.101:0/2982293286 learned_addr learned my addr 172.21.15.101:0/2982293286 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:22.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.460+0000 7f0ae659b700 1 --2- 172.21.15.101:0/2982293286 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562856a312d0 0x562856a487b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:22.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.461+0000 7f0ae6d9c700 1 --2- 172.21.15.101:0/2982293286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562856a3cad0 0x562856a38f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:22.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.461+0000 7f0ae5d9a700 1 -- 172.21.15.101:0/2982293286 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562856a312d0 msgr2=0x562856a487b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:22.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.461+0000 7f0ae5d9a700 1 --2- 172.21.15.101:0/2982293286 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562856a312d0 0x562856a487b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.461+0000 7f0ae5d9a700 1 -- 172.21.15.101:0/2982293286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562856a3cad0 msgr2=0x562856a38f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:22.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.461+0000 7f0ae5d9a700 1 --2- 172.21.15.101:0/2982293286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562856a3cad0 0x562856a38f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.461+0000 7f0ae5d9a700 1 -- 172.21.15.101:0/2982293286 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56285679ac60 con 0x562856a31be0 2023-07-18T03:12:22.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.461+0000 7f0ae6d9c700 1 --2- 172.21.15.101:0/2982293286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562856a3cad0 0x562856a38f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:12:22.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.461+0000 7f0ae659b700 1 --2- 172.21.15.101:0/2982293286 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562856a312d0 0x562856a487b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:12:22.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.461+0000 7f0ae5d9a700 1 --2- 172.21.15.101:0/2982293286 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562856a31be0 0x562856a3c060 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f0ad000ffb0 tx=0x7f0ad0009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb715e0731e6d8b9 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:22.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.461+0000 7f0ad57fa700 1 -- 172.21.15.101:0/2982293286 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0096f10 con 0x562856a31be0 2023-07-18T03:12:22.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.461+0000 7f0ad57fa700 1 -- 172.21.15.101:0/2982293286 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5628568dd4f0 con 0x562856a31be0 2023-07-18T03:12:22.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.461+0000 7f0ad57fa700 1 -- 172.21.15.101:0/2982293286 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad00970d0 con 0x562856a31be0 2023-07-18T03:12:22.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.462+0000 7f0afc141b80 1 -- 172.21.15.101:0/2982293286 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562856a31be0 msgr2=0x562856a3c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:22.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.462+0000 7f0afc141b80 1 --2- 172.21.15.101:0/2982293286 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562856a31be0 0x562856a3c060 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f0ad000ffb0 tx=0x7f0ad0009ca0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.462+0000 7f0afc141b80 1 -- 172.21.15.101:0/2982293286 shutdown_connections 2023-07-18T03:12:22.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.462+0000 7f0afc141b80 1 --2- 172.21.15.101:0/2982293286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562856a3cad0 0x562856a38f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.462+0000 7f0afc141b80 1 --2- 172.21.15.101:0/2982293286 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562856a312d0 0x562856a487b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.462+0000 7f0afc141b80 1 --2- 172.21.15.101:0/2982293286 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562856a31be0 0x562856a3c060 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.462+0000 7f0afc141b80 1 -- 172.21.15.101:0/2982293286 >> 172.21.15.101:0/2982293286 conn(0x562856a27da0 msgr2=0x562856a52cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:22.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.462+0000 7f0afc141b80 1 -- 172.21.15.101:0/2982293286 shutdown_connections 2023-07-18T03:12:22.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.462+0000 7f0afc141b80 1 -- 172.21.15.101:0/2982293286 wait complete. 2023-07-18T03:12:22.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.463+0000 7f0afc141b80 1 -- start start 2023-07-18T03:12:22.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.463+0000 7f0afc141b80 1 Processor -- start 2023-07-18T03:12:22.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.463+0000 7f0afc141b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562856a312d0 0x562856a416f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:22.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.463+0000 7f0afc141b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562856a31be0 0x562856a41c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:22.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.463+0000 7f0afc141b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562856a3cad0 0x562856a9ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:22.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.463+0000 7f0afc141b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562856a9f380 con 0x562856a31be0 2023-07-18T03:12:22.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.463+0000 7f0afc141b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562856a9f4c0 con 0x562856a312d0 2023-07-18T03:12:22.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.463+0000 7f0afc141b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562856a9f630 con 0x562856a3cad0 2023-07-18T03:12:22.471 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.463+0000 7f0afc141b80 10 client.0 _ll_register_callbacks cb 0x562856921760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:12:22.471 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.464+0000 7f0ae6d9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562856a3cad0 0x562856a9ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:22.471 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.464+0000 7f0ae6d9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562856a3cad0 0x562856a9ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:51820/0 (socket says 172.21.15.101:51820) 2023-07-18T03:12:22.471 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.464+0000 7f0ae659b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562856a312d0 0x562856a416f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:22.471 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.464+0000 7f0ae6d9c700 1 -- 172.21.15.101:0/436522877 learned_addr learned my addr 172.21.15.101:0/436522877 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:22.471 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.464+0000 7f0ae5d9a700 1 --2- 172.21.15.101:0/436522877 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562856a31be0 0x562856a41c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:22.471 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.464+0000 7f0ae6d9c700 1 -- 172.21.15.101:0/436522877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562856a312d0 msgr2=0x562856a416f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:22.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.464+0000 7f0ae6d9c700 1 --2- 172.21.15.101:0/436522877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562856a312d0 0x562856a416f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.464+0000 7f0ae6d9c700 1 -- 172.21.15.101:0/436522877 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562856a31be0 msgr2=0x562856a41c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:22.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.464+0000 7f0ae6d9c700 1 --2- 172.21.15.101:0/436522877 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562856a31be0 0x562856a41c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.464+0000 7f0ae6d9c700 1 -- 172.21.15.101:0/436522877 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5628567b1de0 con 0x562856a3cad0 2023-07-18T03:12:22.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.464+0000 7f0ae659b700 1 --2- 172.21.15.101:0/436522877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562856a312d0 0x562856a416f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:12:22.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.464+0000 7f0ae6d9c700 1 --2- 172.21.15.101:0/436522877 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562856a3cad0 0x562856a9ee40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8007f00 tx=0x7f0ad8007f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=741e9b1f3a3bb4f6 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:22.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.464+0000 7f0ac9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:12:22.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.464+0000 7f0ac9ffb700 1 -- 172.21.15.101:0/436522877 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad800e070 con 0x562856a3cad0 2023-07-18T03:12:22.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.464+0000 7f0ac9ffb700 1 -- 172.21.15.101:0/436522877 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f0ad8004db0 con 0x562856a3cad0 2023-07-18T03:12:22.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.464+0000 7f0afc141b80 1 -- 172.21.15.101:0/436522877 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0adc00e840 con 0x562856a3cad0 2023-07-18T03:12:22.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.465+0000 7f0ac9ffb700 1 -- 172.21.15.101:0/436522877 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ad80167b0 con 0x562856a3cad0 2023-07-18T03:12:22.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.465+0000 7f0afc141b80 10 client.26593 fetch_fsmap learned FSMap version 26 2023-07-18T03:12:22.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.465+0000 7f0afc141b80 1 -- 172.21.15.101:0/436522877 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f0adc017070 con 0x562856a3cad0 2023-07-18T03:12:22.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.465+0000 7f0ac9ffb700 1 -- 172.21.15.101:0/436522877 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0adc00e840 con 0x562856a3cad0 2023-07-18T03:12:22.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.465+0000 7f0ac9ffb700 1 -- 172.21.15.101:0/436522877 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0ad8020ac0 con 0x562856a3cad0 2023-07-18T03:12:22.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.465+0000 7f0afc141b80 10 client.26593 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:12:22.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.465+0000 7f0afc141b80 10 client.26593 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:12:22.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.465+0000 7f0afc141b80 20 client.26593 populate_metadata read hostname 'smithi101' 2023-07-18T03:12:22.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.465+0000 7f0afc141b80 1 --2- 172.21.15.101:0/436522877 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x562856aa0db0 0x562856aa31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:22.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.465+0000 7f0afc141b80 4 client.26593 mds_command: new command op to 24562 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:12:22.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.465+0000 7f0afc141b80 1 -- 172.21.15.101:0/436522877 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56285693f050 con 0x562856aa0db0 2023-07-18T03:12:22.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.466+0000 7f0ae659b700 1 --2- 172.21.15.101:0/436522877 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x562856aa0db0 0x562856aa31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:22.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.466+0000 7f0ae659b700 1 --2- 172.21.15.101:0/436522877 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x562856aa0db0 0x562856aa31c0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb6bd04538c14690 server_cookie=2ce6c36010af4b1b in_seq=0 out_seq=0 2023-07-18T03:12:22.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.466+0000 7f0ac9ffb700 10 client.26593 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:12:22.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.466+0000 7f0ac9ffb700 1 -- 172.21.15.101:0/436522877 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56285693f050 con 0x562856aa0db0 2023-07-18T03:12:22.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.466+0000 7f0ac9ffb700 10 client.26593 handle_command_reply: tid=0 2023-07-18T03:12:22.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.466+0000 7f0afc141b80 1 client.26593 shutdown 2023-07-18T03:12:22.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.467+0000 7f0afc141b80 1 -- 172.21.15.101:0/436522877 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562856a3cad0 msgr2=0x562856a9ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:22.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.467+0000 7f0afc141b80 1 --2- 172.21.15.101:0/436522877 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562856a3cad0 0x562856a9ee40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8007f00 tx=0x7f0ad8007f30 comp rx=0 tx=0).stop 2023-07-18T03:12:22.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.467+0000 7f0afc141b80 1 -- 172.21.15.101:0/436522877 shutdown_connections 2023-07-18T03:12:22.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.467+0000 7f0afc141b80 1 --2- 172.21.15.101:0/436522877 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x562856aa0db0 0x562856aa31c0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.467+0000 7f0afc141b80 1 --2- 172.21.15.101:0/436522877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562856a312d0 0x562856a416f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.467+0000 7f0ac9ffb700 0 client.26593 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:12:22.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.467+0000 7f0afc141b80 1 --2- 172.21.15.101:0/436522877 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562856a3cad0 0x562856a9ee40 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.467+0000 7f0afc141b80 1 --2- 172.21.15.101:0/436522877 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562856a31be0 0x562856a41c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.467+0000 7f0afc141b80 1 -- 172.21.15.101:0/436522877 >> 172.21.15.101:0/436522877 conn(0x562856a27da0 msgr2=0x562856a487b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:22.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.467+0000 7f0afc141b80 1 -- 172.21.15.101:0/436522877 shutdown_connections 2023-07-18T03:12:22.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.467+0000 7f0afc141b80 1 -- 172.21.15.101:0/436522877 wait complete. 2023-07-18T03:12:22.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.467+0000 7f0afc141b80 20 client.26593 trim_cache size 0 max 16384 2023-07-18T03:12:22.480 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:12:22.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.467+0000 7f0ae87ff700 1 -- 172.21.15.101:0/1970387782 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0acc0a12c0 msgr2=0x7f0acc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:22.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.468+0000 7f0ae87ff700 1 --2- 172.21.15.101:0/1970387782 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0acc0a12c0 0x7f0acc0a3770 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0adc009be0 tx=0x7f0adc009260 comp rx=0 tx=0).stop 2023-07-18T03:12:22.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.468+0000 7f0ae87ff700 1 -- 172.21.15.101:0/1970387782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ae0100480 msgr2=0x7f0ae0072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:22.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.468+0000 7f0ae87ff700 1 --2- 172.21.15.101:0/1970387782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ae0100480 0x7f0ae0072d50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0005910 tx=0x7f0ad0004710 comp rx=0 tx=0).stop 2023-07-18T03:12:22.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.468+0000 7f0ae87ff700 1 -- 172.21.15.101:0/1970387782 shutdown_connections 2023-07-18T03:12:22.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.468+0000 7f0ae87ff700 1 --2- 172.21.15.101:0/1970387782 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ae0105ae0 0x7f0ae01a3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.468+0000 7f0ae87ff700 1 --2- 172.21.15.101:0/1970387782 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0ae00ffad0 0x7f0ae0072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.468+0000 7f0ae87ff700 1 --2- 172.21.15.101:0/1970387782 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0acc0a12c0 0x7f0acc0a3770 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.468+0000 7f0ae87ff700 1 --2- 172.21.15.101:0/1970387782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ae0100480 0x7f0ae0072d50 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:22.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.468+0000 7f0ae87ff700 1 -- 172.21.15.101:0/1970387782 >> 172.21.15.101:0/1970387782 conn(0x7f0ae00fb5f0 msgr2=0x7f0ae01035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:22.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.468+0000 7f0ae87ff700 1 -- 172.21.15.101:0/1970387782 shutdown_connections 2023-07-18T03:12:22.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:22.468+0000 7f0ae87ff700 1 -- 172.21.15.101:0/1970387782 wait complete. 2023-07-18T03:12:22.578 INFO:tasks.workunit.client.0.smithi101.stdout: 1 357600 8.88 MB/sec execute 107 sec latency 31.164 ms 2023-07-18T03:12:23.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:22 smithi179 ceph-mon[129229]: pgmap v147: 129 pgs: 129 active+clean; 217 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 200 op/s 2023-07-18T03:12:23.236 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:22 smithi101 ceph-mon[126379]: pgmap v147: 129 pgs: 129 active+clean; 217 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 200 op/s 2023-07-18T03:12:23.293 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:22 smithi155 ceph-mon[128885]: pgmap v147: 129 pgs: 129 active+clean; 217 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 200 op/s 2023-07-18T03:12:23.485 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-18T03:12:23.486 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-18T03:12:23.578 INFO:tasks.workunit.client.0.smithi101.stdout: 1 358712 8.87 MB/sec execute 108 sec latency 58.255 ms 2023-07-18T03:12:23.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.712+0000 7f4ee6dff700 1 -- 172.21.15.101:0/1638923846 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ee0102900 msgr2=0x7f4ee01068c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:23.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.712+0000 7f4ee6dff700 1 --2- 172.21.15.101:0/1638923846 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ee0102900 0x7f4ee01068c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc0099c0 tx=0x7f4ecc009cd0 comp rx=0 tx=0).stop 2023-07-18T03:12:23.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.712+0000 7f4ee6dff700 1 -- 172.21.15.101:0/1638923846 shutdown_connections 2023-07-18T03:12:23.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.712+0000 7f4ee6dff700 1 --2- 172.21.15.101:0/1638923846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ee0106fa0 0x7f4ee010eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:23.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.712+0000 7f4ee6dff700 1 --2- 172.21.15.101:0/1638923846 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4ee0101f60 0x7f4ee0102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:23.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.712+0000 7f4ee6dff700 1 --2- 172.21.15.101:0/1638923846 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ee0102900 0x7f4ee01068c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:23.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.712+0000 7f4ee6dff700 1 -- 172.21.15.101:0/1638923846 >> 172.21.15.101:0/1638923846 conn(0x7f4ee00fd7d0 msgr2=0x7f4ee00ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:23.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.712+0000 7f4ee6dff700 1 -- 172.21.15.101:0/1638923846 shutdown_connections 2023-07-18T03:12:23.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.712+0000 7f4ee6dff700 1 -- 172.21.15.101:0/1638923846 wait complete. 2023-07-18T03:12:23.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.713+0000 7f4ee6dff700 1 Processor -- start 2023-07-18T03:12:23.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.713+0000 7f4ee6dff700 1 -- start start 2023-07-18T03:12:23.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.714+0000 7f4ee6dff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ee0101f60 0x7f4ee0197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:23.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.714+0000 7f4ee6dff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ee0102900 0x7f4ee0198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:23.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.714+0000 7f4ee6dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4ee0106fa0 0x7f4ee019c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:23.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.714+0000 7f4ee6dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee019cde0 con 0x7f4ee0106fa0 2023-07-18T03:12:23.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.714+0000 7f4ee6dff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee019cf50 con 0x7f4ee0101f60 2023-07-18T03:12:23.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.714+0000 7f4ee6dff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee019d110 con 0x7f4ee0102900 2023-07-18T03:12:23.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.714+0000 7f4ee4b9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ee0101f60 0x7f4ee0197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:23.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.714+0000 7f4edffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ee0102900 0x7f4ee0198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:23.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.714+0000 7f4ee4b9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ee0101f60 0x7f4ee0197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:52640/0 (socket says 172.21.15.101:52640) 2023-07-18T03:12:23.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.714+0000 7f4edffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ee0102900 0x7f4ee0198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:51830/0 (socket says 172.21.15.101:51830) 2023-07-18T03:12:23.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.714+0000 7f4ee4b9b700 1 -- 172.21.15.101:0/807181525 learned_addr learned my addr 172.21.15.101:0/807181525 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:23.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.714+0000 7f4ee539c700 1 --2- 172.21.15.101:0/807181525 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4ee0106fa0 0x7f4ee019c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:23.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.715+0000 7f4ee4b9b700 1 -- 172.21.15.101:0/807181525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ee0102900 msgr2=0x7f4ee0198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:23.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.715+0000 7f4ee4b9b700 1 --2- 172.21.15.101:0/807181525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ee0102900 0x7f4ee0198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:23.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.715+0000 7f4ee4b9b700 1 -- 172.21.15.101:0/807181525 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4ee0106fa0 msgr2=0x7f4ee019c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:23.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.715+0000 7f4ee4b9b700 1 --2- 172.21.15.101:0/807181525 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4ee0106fa0 0x7f4ee019c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:23.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.715+0000 7f4ee4b9b700 1 -- 172.21.15.101:0/807181525 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed4009520 con 0x7f4ee0101f60 2023-07-18T03:12:23.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.715+0000 7f4edffff700 1 --2- 172.21.15.101:0/807181525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ee0102900 0x7f4ee0198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:12:23.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.715+0000 7f4ee4b9b700 1 --2- 172.21.15.101:0/807181525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ee0101f60 0x7f4ee0197e00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4ed400eb30 tx=0x7f4ed400ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:23.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.715+0000 7f4eddffb700 1 -- 172.21.15.101:0/807181525 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4009880 con 0x7f4ee0101f60 2023-07-18T03:12:23.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.715+0000 7f4eddffb700 1 -- 172.21.15.101:0/807181525 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ed40044e0 con 0x7f4ee0101f60 2023-07-18T03:12:23.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.715+0000 7f4ee6dff700 1 -- 172.21.15.101:0/807181525 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ecc0095f0 con 0x7f4ee0101f60 2023-07-18T03:12:23.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.715+0000 7f4eddffb700 1 -- 172.21.15.101:0/807181525 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4003d80 con 0x7f4ee0101f60 2023-07-18T03:12:23.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.715+0000 7f4ee6dff700 1 -- 172.21.15.101:0/807181525 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ee01a4a50 con 0x7f4ee0101f60 2023-07-18T03:12:23.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.719+0000 7f4eddffb700 1 -- 172.21.15.101:0/807181525 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f4ed4003f20 con 0x7f4ee0101f60 2023-07-18T03:12:23.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.719+0000 7f4efa6ebb80 1 Processor -- start 2023-07-18T03:12:23.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.720+0000 7f4efa6ebb80 1 -- start start 2023-07-18T03:12:23.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.720+0000 7f4efa6ebb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f774c8350 0x557f774c8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:23.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.720+0000 7f4efa6ebb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f774c8c60 0x557f774d3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:23.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.720+0000 7f4efa6ebb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f774d3b50 0x557f774d5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:23.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.720+0000 7f4efa6ebb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f773ced30 con 0x557f774c8350 2023-07-18T03:12:23.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.720+0000 7f4efa6ebb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f77421ac0 con 0x557f774d3b50 2023-07-18T03:12:23.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.720+0000 7f4efa6ebb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f7741af50 con 0x557f774c8c60 2023-07-18T03:12:23.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.720+0000 7f4eddffb700 1 --2- 172.21.15.101:0/807181525 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4ed00a12c0 0x7f4ed00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:23.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.720+0000 7f4ee4b9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f774c8c60 0x557f774d3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:23.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.720+0000 7f4edffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f774d3b50 0x557f774d5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:23.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.720+0000 7f4ee4b9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f774c8c60 0x557f774d3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:51832/0 (socket says 172.21.15.101:51832) 2023-07-18T03:12:23.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.720+0000 7f4ee4b9b700 1 -- 172.21.15.101:0/3911673616 learned_addr learned my addr 172.21.15.101:0/3911673616 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:23.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.720+0000 7f4ee539c700 1 --2- 172.21.15.101:0/3911673616 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f774c8350 0x557f774c8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:23.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.721+0000 7f4ee539c700 1 --2- 172.21.15.101:0/807181525 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4ed00a12c0 0x7f4ed00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:23.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.721+0000 7f4eddffb700 1 -- 172.21.15.101:0/807181525 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f4ed4014070 con 0x7f4ee0101f60 2023-07-18T03:12:23.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.721+0000 7f4ee4b9b700 1 -- 172.21.15.101:0/3911673616 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f774d3b50 msgr2=0x557f774d5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:23.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.721+0000 7f4ee4b9b700 1 --2- 172.21.15.101:0/3911673616 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f774d3b50 0x557f774d5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:23.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.721+0000 7f4ee4b9b700 1 -- 172.21.15.101:0/3911673616 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f774c8350 msgr2=0x557f774c8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:23.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.721+0000 7f4ee4b9b700 1 --2- 172.21.15.101:0/3911673616 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f774c8350 0x557f774c8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:23.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.721+0000 7f4ee4b9b700 1 -- 172.21.15.101:0/3911673616 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f77248e60 con 0x557f774c8c60 2023-07-18T03:12:23.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.721+0000 7f4edffff700 1 --2- 172.21.15.101:0/3911673616 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f774d3b50 0x557f774d5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:12:23.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.721+0000 7f4ee4b9b700 1 --2- 172.21.15.101:0/3911673616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f774c8c60 0x557f774d3610 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4ed400cf90 tx=0x7f4ed4020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13802776373d6169 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:23.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.721+0000 7f4ee539c700 1 --2- 172.21.15.101:0/807181525 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4ed00a12c0 0x7f4ed00a3770 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8009ec0 tx=0x7f4ed800b040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:23.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.721+0000 7f4ecaffd700 1 -- 172.21.15.101:0/3911673616 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed400cbd0 con 0x557f774c8c60 2023-07-18T03:12:23.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.721+0000 7f4ecaffd700 1 -- 172.21.15.101:0/3911673616 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ed401e2e0 con 0x557f774c8c60 2023-07-18T03:12:23.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.721+0000 7f4ecaffd700 1 -- 172.21.15.101:0/3911673616 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4096b80 con 0x557f774c8c60 2023-07-18T03:12:23.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.722+0000 7f4efa6ebb80 1 -- 172.21.15.101:0/3911673616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f774c8c60 msgr2=0x557f774d3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:23.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.722+0000 7f4efa6ebb80 1 --2- 172.21.15.101:0/3911673616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f774c8c60 0x557f774d3610 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4ed400cf90 tx=0x7f4ed4020000 comp rx=0 tx=0).stop 2023-07-18T03:12:23.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.722+0000 7f4efa6ebb80 1 -- 172.21.15.101:0/3911673616 shutdown_connections 2023-07-18T03:12:23.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.722+0000 7f4efa6ebb80 1 --2- 172.21.15.101:0/3911673616 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f774d3b50 0x557f774d5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:23.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.722+0000 7f4efa6ebb80 1 --2- 172.21.15.101:0/3911673616 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f774c8350 0x557f774c8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:23.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.722+0000 7f4efa6ebb80 1 --2- 172.21.15.101:0/3911673616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f774c8c60 0x557f774d3610 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:23.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.722+0000 7f4efa6ebb80 1 -- 172.21.15.101:0/3911673616 >> 172.21.15.101:0/3911673616 conn(0x557f774bee20 msgr2=0x557f774c4720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:23.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.722+0000 7f4efa6ebb80 1 -- 172.21.15.101:0/3911673616 shutdown_connections 2023-07-18T03:12:23.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.722+0000 7f4efa6ebb80 1 -- 172.21.15.101:0/3911673616 wait complete. 2023-07-18T03:12:23.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.723+0000 7f4efa6ebb80 1 -- start start 2023-07-18T03:12:23.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.723+0000 7f4efa6ebb80 1 Processor -- start 2023-07-18T03:12:23.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.723+0000 7f4efa6ebb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f774c8350 0x557f774d3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:23.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.723+0000 7f4efa6ebb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f774c8c60 0x557f774d0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:23.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.723+0000 7f4efa6ebb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f774d3b50 0x557f774d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:23.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.723+0000 7f4efa6ebb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f773bab40 con 0x557f774c8350 2023-07-18T03:12:23.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.723+0000 7f4efa6ebb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f773ba840 con 0x557f774d3b50 2023-07-18T03:12:23.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.723+0000 7f4efa6ebb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f7729e9a0 con 0x557f774c8c60 2023-07-18T03:12:23.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.723+0000 7f4efa6ebb80 10 client.0 _ll_register_callbacks cb 0x557f773b87e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:12:23.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.724+0000 7f4ee4b9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f774c8350 0x557f774d3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:23.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.724+0000 7f4edffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f774c8c60 0x557f774d0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:23.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.724+0000 7f4edffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f774c8c60 0x557f774d0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:51846/0 (socket says 172.21.15.101:51846) 2023-07-18T03:12:23.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.724+0000 7f4edffff700 1 -- 172.21.15.101:0/1873092834 learned_addr learned my addr 172.21.15.101:0/1873092834 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:23.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.724+0000 7f4ee539c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f774d3b50 0x557f774d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:23.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.724+0000 7f4edffff700 1 -- 172.21.15.101:0/1873092834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f774d3b50 msgr2=0x557f774d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:23.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.724+0000 7f4edffff700 1 --2- 172.21.15.101:0/1873092834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f774d3b50 0x557f774d0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:23.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.724+0000 7f4edffff700 1 -- 172.21.15.101:0/1873092834 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f774c8350 msgr2=0x557f774d3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:23.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.724+0000 7f4edffff700 1 --2- 172.21.15.101:0/1873092834 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f774c8350 0x557f774d3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:23.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.724+0000 7f4edffff700 1 -- 172.21.15.101:0/1873092834 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ee010f3b0 con 0x557f774c8c60 2023-07-18T03:12:23.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.724+0000 7f4ee4b9b700 1 --2- 172.21.15.101:0/1873092834 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f774c8350 0x557f774d3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:12:23.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.724+0000 7f4ee539c700 1 --2- 172.21.15.101:0/1873092834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f774d3b50 0x557f774d0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:12:23.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.724+0000 7f4edffff700 1 --2- 172.21.15.101:0/1873092834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f774c8c60 0x557f774d0800 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc000c00 tx=0x7f4ecc003a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb1ffefff6c089ba server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:23.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.724+0000 7f4ebffff700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:12:23.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.725+0000 7f4ebffff700 1 -- 172.21.15.101:0/1873092834 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc005240 con 0x557f774c8c60 2023-07-18T03:12:23.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.725+0000 7f4efa6ebb80 1 -- 172.21.15.101:0/1873092834 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557f77374570 con 0x557f774c8c60 2023-07-18T03:12:23.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.725+0000 7f4ebffff700 1 -- 172.21.15.101:0/1873092834 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f4ecc003e30 con 0x557f774c8c60 2023-07-18T03:12:23.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.725+0000 7f4ebffff700 1 -- 172.21.15.101:0/1873092834 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ecc0053a0 con 0x557f774c8c60 2023-07-18T03:12:23.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.725+0000 7f4efa6ebb80 10 client.26605 fetch_fsmap learned FSMap version 26 2023-07-18T03:12:23.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.725+0000 7f4efa6ebb80 1 -- 172.21.15.101:0/1873092834 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f4ee01a4a50 con 0x557f774c8c60 2023-07-18T03:12:23.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.725+0000 7f4ebffff700 1 -- 172.21.15.101:0/1873092834 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557f77374570 con 0x557f774c8c60 2023-07-18T03:12:23.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.725+0000 7f4ebffff700 1 -- 172.21.15.101:0/1873092834 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4ecc0173f0 con 0x557f774c8c60 2023-07-18T03:12:23.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.726+0000 7f4efa6ebb80 10 client.26605 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:12:23.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.726+0000 7f4efa6ebb80 10 client.26605 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:12:23.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.726+0000 7f4efa6ebb80 20 client.26605 populate_metadata read hostname 'smithi101' 2023-07-18T03:12:23.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.726+0000 7f4efa6ebb80 1 --2- 172.21.15.101:0/1873092834 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557f774e3440 0x557f774e5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:23.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.726+0000 7f4efa6ebb80 4 client.26605 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:12:23.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.726+0000 7f4efa6ebb80 1 -- 172.21.15.101:0/1873092834 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f77413f30 con 0x557f774e3440 2023-07-18T03:12:23.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.726+0000 7f4ee4b9b700 1 --2- 172.21.15.101:0/1873092834 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557f774e3440 0x557f774e5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:23.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.726+0000 7f4ee4b9b700 1 --2- 172.21.15.101:0/1873092834 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557f774e3440 0x557f774e5850 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af3a565d01114ed0 server_cookie=845696bf902de525 in_seq=0 out_seq=0 2023-07-18T03:12:23.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.726+0000 7f4ebffff700 10 client.26605 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:12:23.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.727+0000 7f4ebffff700 1 -- 172.21.15.101:0/1873092834 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557f77413f30 con 0x557f774e3440 2023-07-18T03:12:23.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.728+0000 7f4ebffff700 10 client.26605 handle_command_reply: tid=0 2023-07-18T03:12:23.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.728+0000 7f4efa6ebb80 1 client.26605 shutdown 2023-07-18T03:12:23.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.728+0000 7f4efa6ebb80 1 -- 172.21.15.101:0/1873092834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f774c8c60 msgr2=0x557f774d0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:23.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.728+0000 7f4efa6ebb80 1 --2- 172.21.15.101:0/1873092834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f774c8c60 0x557f774d0800 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc000c00 tx=0x7f4ecc003a00 comp rx=0 tx=0).stop 2023-07-18T03:12:23.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.728+0000 7f4efa6ebb80 1 -- 172.21.15.101:0/1873092834 shutdown_connections 2023-07-18T03:12:23.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.728+0000 7f4efa6ebb80 1 --2- 172.21.15.101:0/1873092834 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557f774e3440 0x557f774e5850 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:23.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.728+0000 7f4efa6ebb80 1 --2- 172.21.15.101:0/1873092834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f774d3b50 0x557f774d0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:23.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.728+0000 7f4efa6ebb80 1 --2- 172.21.15.101:0/1873092834 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f774c8350 0x557f774d3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:23.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.728+0000 7f4efa6ebb80 1 --2- 172.21.15.101:0/1873092834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f774c8c60 0x557f774d0800 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:23.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.728+0000 7f4ebffff700 0 client.26605 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:12:23.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.728+0000 7f4efa6ebb80 1 -- 172.21.15.101:0/1873092834 >> 172.21.15.101:0/1873092834 conn(0x557f774bee20 msgr2=0x557f774c42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:23.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.728+0000 7f4efa6ebb80 1 -- 172.21.15.101:0/1873092834 shutdown_connections 2023-07-18T03:12:23.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.728+0000 7f4efa6ebb80 1 -- 172.21.15.101:0/1873092834 wait complete. 2023-07-18T03:12:23.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.728+0000 7f4efa6ebb80 20 client.26605 trim_cache size 0 max 16384 2023-07-18T03:12:23.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.747+0000 7f4efa6ebb80 1 Processor -- start 2023-07-18T03:12:23.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.747+0000 7f4efa6ebb80 1 -- start start 2023-07-18T03:12:23.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.747+0000 7f4efa6ebb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f774c8350 0x557f774da5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:23.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.747+0000 7f4efa6ebb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f774c8c60 0x557f774cab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:23.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.747+0000 7f4efa6ebb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f774d3b50 0x557f774cffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:23.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.747+0000 7f4efa6ebb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f774dfef0 con 0x557f774c8c60 2023-07-18T03:12:23.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.747+0000 7f4efa6ebb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f774c53f0 con 0x557f774c8350 2023-07-18T03:12:23.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.747+0000 7f4efa6ebb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f774c95f0 con 0x557f774d3b50 2023-07-18T03:12:23.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.747+0000 7f4edffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f774c8c60 0x557f774cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:23.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.747+0000 7f4ee4b9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f774c8350 0x557f774da5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:23.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.747+0000 7f4ee4b9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f774c8350 0x557f774da5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:52668/0 (socket says 172.21.15.101:52668) 2023-07-18T03:12:23.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.747+0000 7f4ee4b9b700 1 -- 172.21.15.101:0/1101463051 learned_addr learned my addr 172.21.15.101:0/1101463051 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:23.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.748+0000 7f4ee539c700 1 --2- 172.21.15.101:0/1101463051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f774d3b50 0x557f774cffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:23.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.748+0000 7f4ee4b9b700 1 -- 172.21.15.101:0/1101463051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f774d3b50 msgr2=0x557f774cffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:23.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.748+0000 7f4ee4b9b700 1 --2- 172.21.15.101:0/1101463051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f774d3b50 0x557f774cffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:23.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.748+0000 7f4ee4b9b700 1 -- 172.21.15.101:0/1101463051 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f774c8c60 msgr2=0x557f774cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:23.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.748+0000 7f4ee4b9b700 1 --2- 172.21.15.101:0/1101463051 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f774c8c60 0x557f774cab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:23.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.748+0000 7f4ee4b9b700 1 -- 172.21.15.101:0/1101463051 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f774c3160 con 0x557f774c8350 2023-07-18T03:12:23.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.748+0000 7f4ee539c700 1 --2- 172.21.15.101:0/1101463051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f774d3b50 0x557f774cffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:12:23.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.748+0000 7f4edffff700 1 --2- 172.21.15.101:0/1101463051 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f774c8c60 0x557f774cab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:12:23.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.748+0000 7f4ee4b9b700 1 --2- 172.21.15.101:0/1101463051 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f774c8350 0x557f774da5a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4ed400c5b0 tx=0x7f4ed4096660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bc63ae7f77323cc server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:23.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.748+0000 7f4ecb7fe700 1 -- 172.21.15.101:0/1101463051 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed40b1070 con 0x557f774c8350 2023-07-18T03:12:23.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.748+0000 7f4ecb7fe700 1 -- 172.21.15.101:0/1101463051 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ed40a2770 con 0x557f774c8350 2023-07-18T03:12:23.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.748+0000 7f4ecb7fe700 1 -- 172.21.15.101:0/1101463051 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4096a30 con 0x557f774c8350 2023-07-18T03:12:23.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.749+0000 7f4efa6ebb80 1 -- 172.21.15.101:0/1101463051 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f774c8350 msgr2=0x557f774da5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:23.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.749+0000 7f4efa6ebb80 1 --2- 172.21.15.101:0/1101463051 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f774c8350 0x557f774da5a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4ed400c5b0 tx=0x7f4ed4096660 comp rx=0 tx=0).stop 2023-07-18T03:12:23.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.749+0000 7f4efa6ebb80 1 -- 172.21.15.101:0/1101463051 shutdown_connections 2023-07-18T03:12:23.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.749+0000 7f4efa6ebb80 1 --2- 172.21.15.101:0/1101463051 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f774c8350 0x557f774da5a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:23.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.749+0000 7f4efa6ebb80 1 --2- 172.21.15.101:0/1101463051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f774d3b50 0x557f774cffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:23.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.749+0000 7f4efa6ebb80 1 --2- 172.21.15.101:0/1101463051 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f774c8c60 0x557f774cab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:23.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.749+0000 7f4efa6ebb80 1 -- 172.21.15.101:0/1101463051 >> 172.21.15.101:0/1101463051 conn(0x557f774bee20 msgr2=0x557f774ca460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:23.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.749+0000 7f4efa6ebb80 1 -- 172.21.15.101:0/1101463051 shutdown_connections 2023-07-18T03:12:23.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.749+0000 7f4efa6ebb80 1 -- 172.21.15.101:0/1101463051 wait complete. 2023-07-18T03:12:23.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.750+0000 7f4efa6ebb80 1 -- start start 2023-07-18T03:12:23.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.750+0000 7f4efa6ebb80 1 Processor -- start 2023-07-18T03:12:23.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.750+0000 7f4efa6ebb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f774c8350 0x557f774d8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:23.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.751+0000 7f4efa6ebb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f774c8c60 0x557f774d90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:23.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.751+0000 7f4efa6ebb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f774d3b50 0x557f774d9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:23.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.751+0000 7f4efa6ebb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f77537310 con 0x557f774c8c60 2023-07-18T03:12:23.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.751+0000 7f4efa6ebb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f77537450 con 0x557f774c8350 2023-07-18T03:12:23.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.751+0000 7f4efa6ebb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f77537590 con 0x557f774d3b50 2023-07-18T03:12:23.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.751+0000 7f4efa6ebb80 10 client.0 _ll_register_callbacks cb 0x7f4ecc00be50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:12:23.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.751+0000 7f4ee4b9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f774c8350 0x557f774d8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:23.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.751+0000 7f4ee4b9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f774c8350 0x557f774d8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:52682/0 (socket says 172.21.15.101:52682) 2023-07-18T03:12:23.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.751+0000 7f4ee4b9b700 1 -- 172.21.15.101:0/6758962 learned_addr learned my addr 172.21.15.101:0/6758962 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:23.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.751+0000 7f4ee539c700 1 --2- 172.21.15.101:0/6758962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f774d3b50 0x557f774d9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:23.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.751+0000 7f4edffff700 1 --2- 172.21.15.101:0/6758962 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f774c8c60 0x557f774d90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:23.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.751+0000 7f4ee4b9b700 1 -- 172.21.15.101:0/6758962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f774d3b50 msgr2=0x557f774d9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:23.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.751+0000 7f4ee4b9b700 1 --2- 172.21.15.101:0/6758962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f774d3b50 0x557f774d9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:23.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.751+0000 7f4ee4b9b700 1 -- 172.21.15.101:0/6758962 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f774c8c60 msgr2=0x557f774d90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:23.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.751+0000 7f4ee4b9b700 1 --2- 172.21.15.101:0/6758962 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f774c8c60 0x557f774d90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:23.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.751+0000 7f4ee4b9b700 1 -- 172.21.15.101:0/6758962 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ee01a4a50 con 0x557f774c8350 2023-07-18T03:12:23.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.751+0000 7f4ee4b9b700 1 --2- 172.21.15.101:0/6758962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f774c8350 0x557f774d8bb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4000c00 tx=0x7f4ed4095ff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=146015908ecf83d2 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:23.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.751+0000 7f4ebffff700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:12:23.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.752+0000 7f4ebffff700 1 -- 172.21.15.101:0/6758962 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed40b1050 con 0x557f774c8350 2023-07-18T03:12:23.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.752+0000 7f4ebffff700 1 -- 172.21.15.101:0/6758962 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f4ed409e350 con 0x557f774c8350 2023-07-18T03:12:23.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.752+0000 7f4efa6ebb80 1 -- 172.21.15.101:0/6758962 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ed401e2e0 con 0x557f774c8350 2023-07-18T03:12:23.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.752+0000 7f4ebffff700 1 -- 172.21.15.101:0/6758962 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ed409a320 con 0x557f774c8350 2023-07-18T03:12:23.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.752+0000 7f4efa6ebb80 10 client.26588 fetch_fsmap learned FSMap version 26 2023-07-18T03:12:23.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.752+0000 7f4efa6ebb80 1 -- 172.21.15.101:0/6758962 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x557f77248e60 con 0x557f774c8350 2023-07-18T03:12:23.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.752+0000 7f4ebffff700 1 -- 172.21.15.101:0/6758962 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ed401e2e0 con 0x557f774c8350 2023-07-18T03:12:23.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.752+0000 7f4ebffff700 1 -- 172.21.15.101:0/6758962 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4ed409a6a0 con 0x557f774c8350 2023-07-18T03:12:23.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.752+0000 7f4efa6ebb80 10 client.26588 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:12:23.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.752+0000 7f4efa6ebb80 10 client.26588 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:12:23.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.752+0000 7f4efa6ebb80 20 client.26588 populate_metadata read hostname 'smithi101' 2023-07-18T03:12:23.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.752+0000 7f4efa6ebb80 1 --2- 172.21.15.101:0/6758962 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557f775387f0 0x557f7753abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:23.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.752+0000 7f4efa6ebb80 4 client.26588 mds_command: new command op to 24562 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-18T03:12:23.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.752+0000 7f4efa6ebb80 1 -- 172.21.15.101:0/6758962 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x557f773d60d0 con 0x557f775387f0 2023-07-18T03:12:23.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.753+0000 7f4edffff700 1 --2- 172.21.15.101:0/6758962 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557f775387f0 0x557f7753abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:23.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.753+0000 7f4ebffff700 10 client.26588 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:12:23.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.753+0000 7f4edffff700 1 --2- 172.21.15.101:0/6758962 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557f775387f0 0x557f7753abd0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef00dbb986b3376a server_cookie=7925afd94de4f32 in_seq=0 out_seq=0 2023-07-18T03:12:23.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.754+0000 7f4ebffff700 1 -- 172.21.15.101:0/6758962 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557f773d60d0 con 0x557f775387f0 2023-07-18T03:12:23.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.754+0000 7f4ebffff700 10 client.26588 handle_command_reply: tid=0 2023-07-18T03:12:23.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.754+0000 7f4efa6ebb80 1 client.26588 shutdown 2023-07-18T03:12:23.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.754+0000 7f4efa6ebb80 1 -- 172.21.15.101:0/6758962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f774c8350 msgr2=0x557f774d8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:23.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.754+0000 7f4efa6ebb80 1 --2- 172.21.15.101:0/6758962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f774c8350 0x557f774d8bb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4000c00 tx=0x7f4ed4095ff0 comp rx=0 tx=0).stop 2023-07-18T03:12:23.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.754+0000 7f4efa6ebb80 1 -- 172.21.15.101:0/6758962 shutdown_connections 2023-07-18T03:12:23.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.754+0000 7f4efa6ebb80 1 --2- 172.21.15.101:0/6758962 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557f775387f0 0x557f7753abd0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:23.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.754+0000 7f4efa6ebb80 1 --2- 172.21.15.101:0/6758962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f774c8350 0x557f774d8bb0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:23.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.754+0000 7f4efa6ebb80 1 --2- 172.21.15.101:0/6758962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f774d3b50 0x557f774d9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:23.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.754+0000 7f4ebffff700 0 client.26588 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:12:23.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.754+0000 7f4efa6ebb80 1 --2- 172.21.15.101:0/6758962 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f774c8c60 0x557f774d90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:23.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.754+0000 7f4efa6ebb80 1 -- 172.21.15.101:0/6758962 >> 172.21.15.101:0/6758962 conn(0x557f774bee20 msgr2=0x557f774cacf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:23.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.754+0000 7f4efa6ebb80 1 -- 172.21.15.101:0/6758962 shutdown_connections 2023-07-18T03:12:23.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.754+0000 7f4efa6ebb80 1 -- 172.21.15.101:0/6758962 wait complete. 2023-07-18T03:12:23.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.755+0000 7f4efa6ebb80 20 client.26588 trim_cache size 0 max 16384 2023-07-18T03:12:23.755 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:12:23.755 INFO:teuthology.orchestra.run.smithi101.stdout: "return_code": 0, 2023-07-18T03:12:23.755 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_tag": "de1d7a1e-2310-4b40-9718-45790f2e1b07", 2023-07-18T03:12:23.755 INFO:teuthology.orchestra.run.smithi101.stdout: "mode": "asynchronous" 2023-07-18T03:12:23.755 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:12:23.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.755+0000 7f4ee6dff700 1 -- 172.21.15.101:0/807181525 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4ed00a12c0 msgr2=0x7f4ed00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:23.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.755+0000 7f4ee6dff700 1 --2- 172.21.15.101:0/807181525 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4ed00a12c0 0x7f4ed00a3770 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8009ec0 tx=0x7f4ed800b040 comp rx=0 tx=0).stop 2023-07-18T03:12:23.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.755+0000 7f4ee6dff700 1 -- 172.21.15.101:0/807181525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ee0101f60 msgr2=0x7f4ee0197e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:23.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.755+0000 7f4ee6dff700 1 --2- 172.21.15.101:0/807181525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ee0101f60 0x7f4ee0197e00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4ed400eb30 tx=0x7f4ed400ee40 comp rx=0 tx=0).stop 2023-07-18T03:12:23.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.755+0000 7f4ee6dff700 1 -- 172.21.15.101:0/807181525 shutdown_connections 2023-07-18T03:12:23.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.755+0000 7f4ee6dff700 1 --2- 172.21.15.101:0/807181525 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4ed00a12c0 0x7f4ed00a3770 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:23.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.755+0000 7f4ee6dff700 1 --2- 172.21.15.101:0/807181525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ee0101f60 0x7f4ee0197e00 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:23.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.755+0000 7f4ee6dff700 1 --2- 172.21.15.101:0/807181525 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4ee0106fa0 0x7f4ee019c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:23.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.755+0000 7f4ee6dff700 1 --2- 172.21.15.101:0/807181525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ee0102900 0x7f4ee0198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:23.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.755+0000 7f4ee6dff700 1 -- 172.21.15.101:0/807181525 >> 172.21.15.101:0/807181525 conn(0x7f4ee00fd7d0 msgr2=0x7f4ee00fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:23.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.755+0000 7f4ee6dff700 1 -- 172.21.15.101:0/807181525 shutdown_connections 2023-07-18T03:12:23.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:23.755+0000 7f4ee6dff700 1 -- 172.21.15.101:0/807181525 wait complete. 2023-07-18T03:12:23.770 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:12:23.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.004+0000 7fce735ff700 1 -- 172.21.15.101:0/3116053400 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fce6c0728b0 msgr2=0x7fce6c072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:23.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.004+0000 7fce735ff700 1 --2- 172.21.15.101:0/3116053400 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fce6c0728b0 0x7fce6c072c80 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fce68009990 tx=0x7fce68009ca0 comp rx=0 tx=0).stop 2023-07-18T03:12:23.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.004+0000 7fce735ff700 1 -- 172.21.15.101:0/3116053400 shutdown_connections 2023-07-18T03:12:23.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.004+0000 7fce735ff700 1 --2- 172.21.15.101:0/3116053400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce6c110af0 0x7fce6c112ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:23.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.004+0000 7fce735ff700 1 --2- 172.21.15.101:0/3116053400 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fce6c0728b0 0x7fce6c072c80 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:23.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.004+0000 7fce735ff700 1 --2- 172.21.15.101:0/3116053400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce6c0731c0 0x7fce6c1105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:23.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.004+0000 7fce735ff700 1 -- 172.21.15.101:0/3116053400 >> 172.21.15.101:0/3116053400 conn(0x7fce6c0fb5e0 msgr2=0x7fce6c0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:23.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.004+0000 7fce735ff700 1 -- 172.21.15.101:0/3116053400 shutdown_connections 2023-07-18T03:12:23.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.004+0000 7fce735ff700 1 -- 172.21.15.101:0/3116053400 wait complete. 2023-07-18T03:12:24.001 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.005+0000 7fce735ff700 1 Processor -- start 2023-07-18T03:12:24.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.005+0000 7fce735ff700 1 -- start start 2023-07-18T03:12:24.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.005+0000 7fce735ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce6c0728b0 0x7fce6c19c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:24.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.005+0000 7fce735ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce6c0731c0 0x7fce6c19c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:24.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.006+0000 7fce735ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fce6c110af0 0x7fce6c1a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:24.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.006+0000 7fce735ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6c1a1260 con 0x7fce6c110af0 2023-07-18T03:12:24.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.006+0000 7fce735ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6c1a13d0 con 0x7fce6c0728b0 2023-07-18T03:12:24.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.006+0000 7fce735ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6c1a1590 con 0x7fce6c0731c0 2023-07-18T03:12:24.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.006+0000 7fce7139b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce6c0728b0 0x7fce6c19c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:24.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.006+0000 7fce7139b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce6c0728b0 0x7fce6c19c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:52700/0 (socket says 172.21.15.101:52700) 2023-07-18T03:12:24.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.006+0000 7fce7139b700 1 -- 172.21.15.101:0/2039785032 learned_addr learned my addr 172.21.15.101:0/2039785032 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:24.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.006+0000 7fce70b9a700 1 --2- 172.21.15.101:0/2039785032 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce6c0731c0 0x7fce6c19c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:24.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.006+0000 7fce71b9c700 1 --2- 172.21.15.101:0/2039785032 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fce6c110af0 0x7fce6c1a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:24.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.006+0000 7fce7139b700 1 -- 172.21.15.101:0/2039785032 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce6c0731c0 msgr2=0x7fce6c19c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:24.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.006+0000 7fce7139b700 1 --2- 172.21.15.101:0/2039785032 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce6c0731c0 0x7fce6c19c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:24.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.006+0000 7fce7139b700 1 -- 172.21.15.101:0/2039785032 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fce6c110af0 msgr2=0x7fce6c1a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:24.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.006+0000 7fce7139b700 1 --2- 172.21.15.101:0/2039785032 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fce6c110af0 0x7fce6c1a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:24.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.006+0000 7fce7139b700 1 -- 172.21.15.101:0/2039785032 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce64009c40 con 0x7fce6c0728b0 2023-07-18T03:12:24.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.006+0000 7fce71b9c700 1 --2- 172.21.15.101:0/2039785032 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fce6c110af0 0x7fce6c1a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:12:24.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.006+0000 7fce7139b700 1 --2- 172.21.15.101:0/2039785032 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce6c0728b0 0x7fce6c19c280 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fce68005910 tx=0x7fce68004710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:24.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.007+0000 7fce627fc700 1 -- 172.21.15.101:0/2039785032 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6801d070 con 0x7fce6c0728b0 2023-07-18T03:12:24.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.007+0000 7fce735ff700 1 -- 172.21.15.101:0/2039785032 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce680095f0 con 0x7fce6c0728b0 2023-07-18T03:12:24.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.007+0000 7fce627fc700 1 -- 172.21.15.101:0/2039785032 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce68022470 con 0x7fce6c0728b0 2023-07-18T03:12:24.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.007+0000 7fce627fc700 1 -- 172.21.15.101:0/2039785032 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce68017650 con 0x7fce6c0728b0 2023-07-18T03:12:24.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.007+0000 7fce735ff700 1 -- 172.21.15.101:0/2039785032 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce6c1a8ed0 con 0x7fce6c0728b0 2023-07-18T03:12:24.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.011+0000 7fce627fc700 1 -- 172.21.15.101:0/2039785032 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fce680225e0 con 0x7fce6c0728b0 2023-07-18T03:12:24.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.011+0000 7fce86fcbb80 1 Processor -- start 2023-07-18T03:12:24.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.011+0000 7fce86fcbb80 1 -- start start 2023-07-18T03:12:24.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.012+0000 7fce86fcbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f24b63f2d0 0x55f24b63f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:24.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.012+0000 7fce86fcbb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f24b63fbe0 0x55f24b64a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:24.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.012+0000 7fce86fcbb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f24b64aad0 0x55f24b64ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:24.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.012+0000 7fce86fcbb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f24b545cb0 con 0x55f24b63fbe0 2023-07-18T03:12:24.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.012+0000 7fce86fcbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f24b598a40 con 0x55f24b63f2d0 2023-07-18T03:12:24.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.012+0000 7fce86fcbb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f24b591ed0 con 0x55f24b64aad0 2023-07-18T03:12:24.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.012+0000 7fce71b9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f24b63f2d0 0x55f24b63f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:24.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.012+0000 7fce71b9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f24b63f2d0 0x55f24b63f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:52708/0 (socket says 172.21.15.101:52708) 2023-07-18T03:12:24.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.012+0000 7fce71b9c700 1 -- 172.21.15.101:0/903357394 learned_addr learned my addr 172.21.15.101:0/903357394 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:24.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.012+0000 7fce7139b700 1 --2- 172.21.15.101:0/903357394 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f24b63fbe0 0x55f24b64a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:24.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.012+0000 7fce70b9a700 1 --2- 172.21.15.101:0/903357394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f24b64aad0 0x55f24b64ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:24.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.012+0000 7fce71b9c700 1 -- 172.21.15.101:0/903357394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f24b64aad0 msgr2=0x55f24b64ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:24.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.012+0000 7fce71b9c700 1 --2- 172.21.15.101:0/903357394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f24b64aad0 0x55f24b64ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:24.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.012+0000 7fce71b9c700 1 -- 172.21.15.101:0/903357394 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f24b63fbe0 msgr2=0x55f24b64a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:24.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.012+0000 7fce71b9c700 1 --2- 172.21.15.101:0/903357394 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f24b63fbe0 0x55f24b64a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:24.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.012+0000 7fce71b9c700 1 -- 172.21.15.101:0/903357394 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f24b3bfde0 con 0x55f24b63f2d0 2023-07-18T03:12:24.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.012+0000 7fce7139b700 1 --2- 172.21.15.101:0/903357394 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f24b63fbe0 0x55f24b64a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:12:24.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.013+0000 7fce627fc700 1 --2- 172.21.15.101:0/2039785032 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fce580a12c0 0x7fce580a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:24.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.013+0000 7fce71b9c700 1 --2- 172.21.15.101:0/903357394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f24b63f2d0 0x55f24b63f6a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fce64007550 tx=0x7fce64009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d7668eb5beb98eb server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:24.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.013+0000 7fce627fc700 1 -- 172.21.15.101:0/2039785032 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fce680ce2c0 con 0x7fce6c0728b0 2023-07-18T03:12:24.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.013+0000 7fce577fe700 1 -- 172.21.15.101:0/903357394 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6400e070 con 0x55f24b63f2d0 2023-07-18T03:12:24.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.013+0000 7fce577fe700 1 -- 172.21.15.101:0/903357394 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce6400f040 con 0x55f24b63f2d0 2023-07-18T03:12:24.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.013+0000 7fce7139b700 1 --2- 172.21.15.101:0/2039785032 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fce580a12c0 0x7fce580a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:24.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.013+0000 7fce577fe700 1 -- 172.21.15.101:0/903357394 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce64014ac0 con 0x55f24b63f2d0 2023-07-18T03:12:24.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.014+0000 7fce86fcbb80 1 -- 172.21.15.101:0/903357394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f24b63f2d0 msgr2=0x55f24b63f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:24.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.014+0000 7fce86fcbb80 1 --2- 172.21.15.101:0/903357394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f24b63f2d0 0x55f24b63f6a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fce64007550 tx=0x7fce64009920 comp rx=0 tx=0).stop 2023-07-18T03:12:24.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.014+0000 7fce7139b700 1 --2- 172.21.15.101:0/2039785032 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fce580a12c0 0x7fce580a3770 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fce680cb770 tx=0x7fce68022cd0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:24.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.014+0000 7fce86fcbb80 1 -- 172.21.15.101:0/903357394 shutdown_connections 2023-07-18T03:12:24.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.014+0000 7fce86fcbb80 1 --2- 172.21.15.101:0/903357394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f24b63f2d0 0x55f24b63f6a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:24.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.014+0000 7fce86fcbb80 1 --2- 172.21.15.101:0/903357394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f24b64aad0 0x55f24b64ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:24.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.014+0000 7fce86fcbb80 1 --2- 172.21.15.101:0/903357394 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f24b63fbe0 0x55f24b64a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:24.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.014+0000 7fce86fcbb80 1 -- 172.21.15.101:0/903357394 >> 172.21.15.101:0/903357394 conn(0x55f24b635da0 msgr2=0x55f24b63b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:24.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.014+0000 7fce86fcbb80 1 -- 172.21.15.101:0/903357394 shutdown_connections 2023-07-18T03:12:24.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.014+0000 7fce86fcbb80 1 -- 172.21.15.101:0/903357394 wait complete. 2023-07-18T03:12:24.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.015+0000 7fce86fcbb80 1 -- start start 2023-07-18T03:12:24.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.015+0000 7fce86fcbb80 1 Processor -- start 2023-07-18T03:12:24.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.016+0000 7fce86fcbb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f24b63f2d0 0x55f24b64a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:24.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.016+0000 7fce86fcbb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f24b63fbe0 0x55f24b647660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:24.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.016+0000 7fce86fcbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f24b64aad0 0x55f24b647c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:24.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.016+0000 7fce86fcbb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f24b531ac0 con 0x55f24b63f2d0 2023-07-18T03:12:24.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.016+0000 7fce86fcbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f24b5317c0 con 0x55f24b64aad0 2023-07-18T03:12:24.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.016+0000 7fce86fcbb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f24b415920 con 0x55f24b63fbe0 2023-07-18T03:12:24.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.016+0000 7fce86fcbb80 10 client.0 _ll_register_callbacks cb 0x55f24b52f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:12:24.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.016+0000 7fce71b9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f24b63f2d0 0x55f24b64a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:24.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.016+0000 7fce71b9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f24b64aad0 0x55f24b647c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:24.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.016+0000 7fce70b9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f24b63fbe0 0x55f24b647660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:24.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.016+0000 7fce71b9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f24b64aad0 0x55f24b647c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:52712/0 (socket says 172.21.15.101:52712) 2023-07-18T03:12:24.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.016+0000 7fce71b9c700 1 -- 172.21.15.101:0/2613495435 learned_addr learned my addr 172.21.15.101:0/2613495435 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:24.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.016+0000 7fce70b9a700 1 -- 172.21.15.101:0/2613495435 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f24b64aad0 msgr2=0x55f24b647c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:24.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.016+0000 7fce70b9a700 1 --2- 172.21.15.101:0/2613495435 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f24b64aad0 0x55f24b647c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:24.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.016+0000 7fce70b9a700 1 -- 172.21.15.101:0/2613495435 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f24b63f2d0 msgr2=0x55f24b64a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:24.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.016+0000 7fce70b9a700 1 --2- 172.21.15.101:0/2613495435 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f24b63f2d0 0x55f24b64a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:24.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.017+0000 7fce70b9a700 1 -- 172.21.15.101:0/2613495435 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce64009280 con 0x55f24b63fbe0 2023-07-18T03:12:24.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.017+0000 7fce70b9a700 1 --2- 172.21.15.101:0/2613495435 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f24b63fbe0 0x55f24b647660 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fce5c011730 tx=0x7fce5c011a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbd5c6df3095398 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:24.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.017+0000 7fce54ff9700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:12:24.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.017+0000 7fce54ff9700 1 -- 172.21.15.101:0/2613495435 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c016070 con 0x55f24b63fbe0 2023-07-18T03:12:24.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.017+0000 7fce54ff9700 1 -- 172.21.15.101:0/2613495435 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fce5c004830 con 0x55f24b63fbe0 2023-07-18T03:12:24.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.017+0000 7fce86fcbb80 1 -- 172.21.15.101:0/2613495435 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f24b4eb4f0 con 0x55f24b63fbe0 2023-07-18T03:12:24.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.017+0000 7fce54ff9700 1 -- 172.21.15.101:0/2613495435 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce5c004a30 con 0x55f24b63fbe0 2023-07-18T03:12:24.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.018+0000 7fce86fcbb80 10 client.26623 fetch_fsmap learned FSMap version 26 2023-07-18T03:12:24.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.018+0000 7fce86fcbb80 1 -- 172.21.15.101:0/2613495435 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55f24b3a8c60 con 0x55f24b63fbe0 2023-07-18T03:12:24.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.018+0000 7fce54ff9700 1 -- 172.21.15.101:0/2613495435 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f24b4eb4f0 con 0x55f24b63fbe0 2023-07-18T03:12:24.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.018+0000 7fce54ff9700 1 -- 172.21.15.101:0/2613495435 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fce5c01a860 con 0x55f24b63fbe0 2023-07-18T03:12:24.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.018+0000 7fce86fcbb80 10 client.26623 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:12:24.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.018+0000 7fce86fcbb80 10 client.26623 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:12:24.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.018+0000 7fce86fcbb80 20 client.26623 populate_metadata read hostname 'smithi101' 2023-07-18T03:12:24.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.018+0000 7fce86fcbb80 1 --2- 172.21.15.101:0/2613495435 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f24b65a430 0x55f24b65c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:24.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.018+0000 7fce86fcbb80 4 client.26623 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:12:24.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.018+0000 7fce86fcbb80 1 -- 172.21.15.101:0/2613495435 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f24b58aeb0 con 0x55f24b65a430 2023-07-18T03:12:24.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.018+0000 7fce71b9c700 1 --2- 172.21.15.101:0/2613495435 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f24b65a430 0x55f24b65c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:24.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.019+0000 7fce71b9c700 1 --2- 172.21.15.101:0/2613495435 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f24b65a430 0x55f24b65c840 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e1f4226d3e4cd7a server_cookie=d2412a3a9f758c91 in_seq=0 out_seq=0 2023-07-18T03:12:24.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.019+0000 7fce54ff9700 10 client.26623 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:12:24.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.019+0000 7fce54ff9700 1 -- 172.21.15.101:0/2613495435 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f24b58aeb0 con 0x55f24b65a430 2023-07-18T03:12:24.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.019+0000 7fce54ff9700 10 client.26623 handle_command_reply: tid=0 2023-07-18T03:12:24.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.019+0000 7fce86fcbb80 1 client.26623 shutdown 2023-07-18T03:12:24.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.020+0000 7fce86fcbb80 1 -- 172.21.15.101:0/2613495435 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f24b63fbe0 msgr2=0x55f24b647660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:24.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.020+0000 7fce86fcbb80 1 --2- 172.21.15.101:0/2613495435 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f24b63fbe0 0x55f24b647660 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fce5c011730 tx=0x7fce5c011a40 comp rx=0 tx=0).stop 2023-07-18T03:12:24.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.020+0000 7fce86fcbb80 1 -- 172.21.15.101:0/2613495435 shutdown_connections 2023-07-18T03:12:24.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.020+0000 7fce86fcbb80 1 --2- 172.21.15.101:0/2613495435 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f24b65a430 0x55f24b65c840 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:24.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.020+0000 7fce86fcbb80 1 --2- 172.21.15.101:0/2613495435 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f24b64aad0 0x55f24b647c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:24.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.020+0000 7fce86fcbb80 1 --2- 172.21.15.101:0/2613495435 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f24b63f2d0 0x55f24b64a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:24.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.020+0000 7fce54ff9700 0 client.26623 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:12:24.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.020+0000 7fce86fcbb80 1 --2- 172.21.15.101:0/2613495435 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f24b63fbe0 0x55f24b647660 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:24.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.020+0000 7fce86fcbb80 1 -- 172.21.15.101:0/2613495435 >> 172.21.15.101:0/2613495435 conn(0x55f24b635da0 msgr2=0x55f24b63b230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:24.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.020+0000 7fce86fcbb80 1 -- 172.21.15.101:0/2613495435 shutdown_connections 2023-07-18T03:12:24.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.020+0000 7fce86fcbb80 1 -- 172.21.15.101:0/2613495435 wait complete. 2023-07-18T03:12:24.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.020+0000 7fce86fcbb80 20 client.26623 trim_cache size 0 max 16384 2023-07-18T03:12:24.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.039+0000 7fce86fcbb80 1 Processor -- start 2023-07-18T03:12:24.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.039+0000 7fce86fcbb80 1 -- start start 2023-07-18T03:12:24.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.039+0000 7fce86fcbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f24b63f2d0 0x55f24b655c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:24.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.039+0000 7fce86fcbb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f24b63fbe0 0x55f24b642a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:24.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.039+0000 7fce86fcbb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f24b64aad0 0x55f24b63b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:24.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.039+0000 7fce86fcbb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f24b63c330 con 0x55f24b63fbe0 2023-07-18T03:12:24.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.039+0000 7fce86fcbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f24b640540 con 0x55f24b63f2d0 2023-07-18T03:12:24.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.039+0000 7fce86fcbb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f24b648c40 con 0x55f24b64aad0 2023-07-18T03:12:24.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.040+0000 7fce70b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f24b63fbe0 0x55f24b642a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:24.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.040+0000 7fce71b9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f24b63f2d0 0x55f24b655c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:24.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.040+0000 7fce70b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f24b63fbe0 0x55f24b642a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:45070/0 (socket says 172.21.15.101:45070) 2023-07-18T03:12:24.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.040+0000 7fce71b9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f24b63f2d0 0x55f24b655c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:52726/0 (socket says 172.21.15.101:52726) 2023-07-18T03:12:24.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.040+0000 7fce70b9a700 1 -- 172.21.15.101:0/1374280039 learned_addr learned my addr 172.21.15.101:0/1374280039 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:24.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.040+0000 7fce71b9c700 1 --2- 172.21.15.101:0/1374280039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f24b64aad0 0x55f24b63b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:24.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.040+0000 7fce70b9a700 1 -- 172.21.15.101:0/1374280039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f24b64aad0 msgr2=0x55f24b63b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:24.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.040+0000 7fce70b9a700 1 --2- 172.21.15.101:0/1374280039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f24b64aad0 0x55f24b63b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:24.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.040+0000 7fce70b9a700 1 -- 172.21.15.101:0/1374280039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f24b63f2d0 msgr2=0x55f24b655c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:24.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.040+0000 7fce70b9a700 1 --2- 172.21.15.101:0/1374280039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f24b63f2d0 0x55f24b655c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:24.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.040+0000 7fce70b9a700 1 -- 172.21.15.101:0/1374280039 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f24b3e6250 con 0x55f24b63fbe0 2023-07-18T03:12:24.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.040+0000 7fce71b9c700 1 --2- 172.21.15.101:0/1374280039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f24b63f2d0 0x55f24b655c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:12:24.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.041+0000 7fce70b9a700 1 --2- 172.21.15.101:0/1374280039 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f24b63fbe0 0x55f24b642a30 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fce5c009340 tx=0x7fce5c011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0af934d2b2461ce server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:24.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.041+0000 7fce57fff700 1 -- 172.21.15.101:0/1374280039 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c016070 con 0x55f24b63fbe0 2023-07-18T03:12:24.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.041+0000 7fce57fff700 1 -- 172.21.15.101:0/1374280039 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce5c01be70 con 0x55f24b63fbe0 2023-07-18T03:12:24.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.041+0000 7fce57fff700 1 -- 172.21.15.101:0/1374280039 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c003680 con 0x55f24b63fbe0 2023-07-18T03:12:24.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.041+0000 7fce86fcbb80 1 -- 172.21.15.101:0/1374280039 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f24b63fbe0 msgr2=0x55f24b642a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:24.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.041+0000 7fce86fcbb80 1 --2- 172.21.15.101:0/1374280039 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f24b63fbe0 0x55f24b642a30 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fce5c009340 tx=0x7fce5c011d90 comp rx=0 tx=0).stop 2023-07-18T03:12:24.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.042+0000 7fce86fcbb80 1 -- 172.21.15.101:0/1374280039 shutdown_connections 2023-07-18T03:12:24.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.042+0000 7fce86fcbb80 1 --2- 172.21.15.101:0/1374280039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f24b63f2d0 0x55f24b655c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:24.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.042+0000 7fce86fcbb80 1 --2- 172.21.15.101:0/1374280039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f24b64aad0 0x55f24b63b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:24.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.042+0000 7fce86fcbb80 1 --2- 172.21.15.101:0/1374280039 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f24b63fbe0 0x55f24b642a30 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:24.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.042+0000 7fce86fcbb80 1 -- 172.21.15.101:0/1374280039 >> 172.21.15.101:0/1374280039 conn(0x55f24b635da0 msgr2=0x55f24b639e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:24.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.042+0000 7fce86fcbb80 1 -- 172.21.15.101:0/1374280039 shutdown_connections 2023-07-18T03:12:24.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.042+0000 7fce86fcbb80 1 -- 172.21.15.101:0/1374280039 wait complete. 2023-07-18T03:12:24.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.042+0000 7fce86fcbb80 1 -- start start 2023-07-18T03:12:24.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.043+0000 7fce86fcbb80 1 Processor -- start 2023-07-18T03:12:24.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.043+0000 7fce86fcbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f24b63f2d0 0x55f24b651500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:24.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.043+0000 7fce86fcbb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f24b63fbe0 0x55f24b64d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:24.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.043+0000 7fce86fcbb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f24b64aad0 0x55f24b64de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:24.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.043+0000 7fce86fcbb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f24b64e530 con 0x55f24b63fbe0 2023-07-18T03:12:24.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.043+0000 7fce86fcbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f24b64e6a0 con 0x55f24b63f2d0 2023-07-18T03:12:24.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.043+0000 7fce86fcbb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f24b6ae2b0 con 0x55f24b64aad0 2023-07-18T03:12:24.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.043+0000 7fce86fcbb80 10 client.0 _ll_register_callbacks cb 0x7fce5c009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:12:24.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.043+0000 7fce70b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f24b63fbe0 0x55f24b64d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:24.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.043+0000 7fce70b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f24b63fbe0 0x55f24b64d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:45084/0 (socket says 172.21.15.101:45084) 2023-07-18T03:12:24.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.043+0000 7fce70b9a700 1 -- 172.21.15.101:0/1834702508 learned_addr learned my addr 172.21.15.101:0/1834702508 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:24.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.043+0000 7fce71b9c700 1 --2- 172.21.15.101:0/1834702508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f24b63f2d0 0x55f24b651500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:24.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.043+0000 7fce71b9c700 1 --2- 172.21.15.101:0/1834702508 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f24b64aad0 0x55f24b64de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:24.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.044+0000 7fce70b9a700 1 -- 172.21.15.101:0/1834702508 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f24b64aad0 msgr2=0x55f24b64de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:24.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.044+0000 7fce70b9a700 1 --2- 172.21.15.101:0/1834702508 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f24b64aad0 0x55f24b64de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:24.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.044+0000 7fce70b9a700 1 -- 172.21.15.101:0/1834702508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f24b63f2d0 msgr2=0x55f24b651500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:24.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.044+0000 7fce70b9a700 1 --2- 172.21.15.101:0/1834702508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f24b63f2d0 0x55f24b651500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:24.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.044+0000 7fce70b9a700 1 -- 172.21.15.101:0/1834702508 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce64028040 con 0x55f24b63fbe0 2023-07-18T03:12:24.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.044+0000 7fce70b9a700 1 --2- 172.21.15.101:0/1834702508 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f24b63fbe0 0x55f24b64d910 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fce5c01ad10 tx=0x7fce5c009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a63c8131039c22df server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:24.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.044+0000 7fce54ff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:12:24.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.044+0000 7fce54ff9700 1 -- 172.21.15.101:0/1834702508 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c016040 con 0x55f24b63fbe0 2023-07-18T03:12:24.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.044+0000 7fce86fcbb80 1 -- 172.21.15.101:0/1834702508 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fce6400f040 con 0x55f24b63fbe0 2023-07-18T03:12:24.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.044+0000 7fce54ff9700 1 -- 172.21.15.101:0/1834702508 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fce5c000e40 con 0x55f24b63fbe0 2023-07-18T03:12:24.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.044+0000 7fce54ff9700 1 -- 172.21.15.101:0/1834702508 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce5c0057a0 con 0x55f24b63fbe0 2023-07-18T03:12:24.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.044+0000 7fce86fcbb80 10 client.16560 fetch_fsmap learned FSMap version 26 2023-07-18T03:12:24.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.045+0000 7fce86fcbb80 1 -- 172.21.15.101:0/1834702508 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55f24b3a8c60 con 0x55f24b63fbe0 2023-07-18T03:12:24.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.045+0000 7fce54ff9700 1 -- 172.21.15.101:0/1834702508 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce6400f040 con 0x55f24b63fbe0 2023-07-18T03:12:24.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.045+0000 7fce54ff9700 1 -- 172.21.15.101:0/1834702508 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fce5c003a30 con 0x55f24b63fbe0 2023-07-18T03:12:24.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.045+0000 7fce86fcbb80 10 client.16560 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:12:24.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.045+0000 7fce86fcbb80 10 client.16560 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:12:24.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.045+0000 7fce86fcbb80 20 client.16560 populate_metadata read hostname 'smithi101' 2023-07-18T03:12:24.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.045+0000 7fce86fcbb80 1 --2- 172.21.15.101:0/1834702508 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f24b6af550 0x55f24b6b1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:24.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.045+0000 7fce86fcbb80 4 client.16560 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:12:24.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.045+0000 7fce86fcbb80 1 -- 172.21.15.101:0/1834702508 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f24b54d050 con 0x55f24b6af550 2023-07-18T03:12:24.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.045+0000 7fce71b9c700 1 --2- 172.21.15.101:0/1834702508 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f24b6af550 0x55f24b6b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:24.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.046+0000 7fce71b9c700 1 --2- 172.21.15.101:0/1834702508 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f24b6af550 0x55f24b6b1960 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6c25d5cc9f85c8a server_cookie=63e28e97ce766cfe in_seq=0 out_seq=0 2023-07-18T03:12:24.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.046+0000 7fce54ff9700 10 client.16560 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:12:24.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.046+0000 7fce54ff9700 1 -- 172.21.15.101:0/1834702508 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55f24b54d050 con 0x55f24b6af550 2023-07-18T03:12:24.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.046+0000 7fce54ff9700 10 client.16560 handle_command_reply: tid=0 2023-07-18T03:12:24.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.046+0000 7fce86fcbb80 1 client.16560 shutdown 2023-07-18T03:12:24.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.046+0000 7fce86fcbb80 1 -- 172.21.15.101:0/1834702508 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f24b63fbe0 msgr2=0x55f24b64d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:24.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.046+0000 7fce86fcbb80 1 --2- 172.21.15.101:0/1834702508 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f24b63fbe0 0x55f24b64d910 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fce5c01ad10 tx=0x7fce5c009740 comp rx=0 tx=0).stop 2023-07-18T03:12:24.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.046+0000 7fce86fcbb80 1 -- 172.21.15.101:0/1834702508 shutdown_connections 2023-07-18T03:12:24.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.046+0000 7fce86fcbb80 1 --2- 172.21.15.101:0/1834702508 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f24b6af550 0x55f24b6b1960 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:24.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.046+0000 7fce86fcbb80 1 --2- 172.21.15.101:0/1834702508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f24b63f2d0 0x55f24b651500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:24.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.046+0000 7fce86fcbb80 1 --2- 172.21.15.101:0/1834702508 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f24b64aad0 0x55f24b64de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:24.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.046+0000 7fce54ff9700 0 client.16560 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:12:24.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.046+0000 7fce86fcbb80 1 --2- 172.21.15.101:0/1834702508 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f24b63fbe0 0x55f24b64d910 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:24.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.046+0000 7fce86fcbb80 1 -- 172.21.15.101:0/1834702508 >> 172.21.15.101:0/1834702508 conn(0x55f24b635da0 msgr2=0x55f24b642a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:24.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.046+0000 7fce86fcbb80 1 -- 172.21.15.101:0/1834702508 shutdown_connections 2023-07-18T03:12:24.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.046+0000 7fce86fcbb80 1 -- 172.21.15.101:0/1834702508 wait complete. 2023-07-18T03:12:24.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.047+0000 7fce86fcbb80 20 client.16560 trim_cache size 0 max 16384 2023-07-18T03:12:24.054 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:12:24.054 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:12:24.054 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:12:24.054 INFO:teuthology.orchestra.run.smithi101.stdout: "de1d7a1e-2310-4b40-9718-45790f2e1b07": { 2023-07-18T03:12:24.055 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:12:24.055 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "de1d7a1e-2310-4b40-9718-45790f2e1b07", 2023-07-18T03:12:24.055 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:12:24.055 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:12:24.055 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:12:24.055 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:12:24.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.047+0000 7fce735ff700 1 -- 172.21.15.101:0/2039785032 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fce580a12c0 msgr2=0x7fce580a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:24.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.047+0000 7fce735ff700 1 --2- 172.21.15.101:0/2039785032 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fce580a12c0 0x7fce580a3770 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fce680cb770 tx=0x7fce68022cd0 comp rx=0 tx=0).stop 2023-07-18T03:12:24.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.047+0000 7fce735ff700 1 -- 172.21.15.101:0/2039785032 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce6c0728b0 msgr2=0x7fce6c19c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:24.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.047+0000 7fce735ff700 1 --2- 172.21.15.101:0/2039785032 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce6c0728b0 0x7fce6c19c280 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fce68005910 tx=0x7fce68004710 comp rx=0 tx=0).stop 2023-07-18T03:12:24.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.047+0000 7fce735ff700 1 -- 172.21.15.101:0/2039785032 shutdown_connections 2023-07-18T03:12:24.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.047+0000 7fce735ff700 1 --2- 172.21.15.101:0/2039785032 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fce580a12c0 0x7fce580a3770 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:24.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.047+0000 7fce735ff700 1 --2- 172.21.15.101:0/2039785032 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce6c0728b0 0x7fce6c19c280 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:24.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.047+0000 7fce735ff700 1 --2- 172.21.15.101:0/2039785032 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fce6c110af0 0x7fce6c1a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:24.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.047+0000 7fce735ff700 1 --2- 172.21.15.101:0/2039785032 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce6c0731c0 0x7fce6c19c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:24.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.047+0000 7fce735ff700 1 -- 172.21.15.101:0/2039785032 >> 172.21.15.101:0/2039785032 conn(0x7fce6c0fb5e0 msgr2=0x7fce6c0fc240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:24.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.047+0000 7fce735ff700 1 -- 172.21.15.101:0/2039785032 shutdown_connections 2023-07-18T03:12:24.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:24.047+0000 7fce735ff700 1 -- 172.21.15.101:0/2039785032 wait complete. 2023-07-18T03:12:24.058 INFO:tasks.cephfs.filesystem:scrub status for tag:de1d7a1e-2310-4b40-9718-45790f2e1b07 - {'path': '/', 'tag': 'de1d7a1e-2310-4b40-9718-45790f2e1b07', 'options': 'recursive,force'} 2023-07-18T03:12:24.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:23 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:12:24.199 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:23 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:24.199 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:23 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:12:24.199 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:23 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:24.199 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:23 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:12:24.199 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:23 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:24.236 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:23 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:12:24.236 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:23 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:24.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:23 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:12:24.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:23 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:24.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:23 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:12:24.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:23 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:24.293 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:23 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:12:24.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:23 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:24.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:23 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:12:24.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:23 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:24.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:23 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:12:24.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:23 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:24.579 INFO:tasks.workunit.client.0.smithi101.stdout: 1 360341 8.84 MB/sec execute 109 sec latency 20.372 ms 2023-07-18T03:12:25.199 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:24 smithi179 ceph-mon[129229]: pgmap v148: 129 pgs: 129 active+clean; 217 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 200 op/s 2023-07-18T03:12:25.199 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:24 smithi179 ceph-mon[129229]: scrub queued for path: / 2023-07-18T03:12:25.199 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:24 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:25.199 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:24 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:12:25.199 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:24 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:12:25.200 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:24 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:25.200 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:24 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:12:25.200 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:24 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:25.200 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:24 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:25.236 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:24 smithi101 ceph-mon[126379]: pgmap v148: 129 pgs: 129 active+clean; 217 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 200 op/s 2023-07-18T03:12:25.236 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:24 smithi101 ceph-mon[126379]: scrub queued for path: / 2023-07-18T03:12:25.236 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:24 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:25.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:24 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:12:25.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:24 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:12:25.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:24 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:25.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:24 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:12:25.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:24 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:25.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:24 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:25.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:24 smithi155 ceph-mon[128885]: pgmap v148: 129 pgs: 129 active+clean; 217 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 200 op/s 2023-07-18T03:12:25.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:24 smithi155 ceph-mon[128885]: scrub queued for path: / 2023-07-18T03:12:25.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:24 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:25.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:24 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:12:25.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:24 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:12:25.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:24 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:25.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:24 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:12:25.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:24 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:25.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:24 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:25.579 INFO:tasks.workunit.client.0.smithi101.stdout: 1 362009 8.88 MB/sec execute 110 sec latency 59.150 ms 2023-07-18T03:12:26.579 INFO:tasks.workunit.client.0.smithi101.stdout: 1 363201 8.85 MB/sec execute 111 sec latency 14.220 ms 2023-07-18T03:12:27.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:26 smithi179 ceph-mon[129229]: pgmap v149: 129 pgs: 129 active+clean; 223 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 295 op/s 2023-07-18T03:12:27.236 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:26 smithi101 ceph-mon[126379]: pgmap v149: 129 pgs: 129 active+clean; 223 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 295 op/s 2023-07-18T03:12:27.293 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:26 smithi155 ceph-mon[128885]: pgmap v149: 129 pgs: 129 active+clean; 223 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 295 op/s 2023-07-18T03:12:27.579 INFO:tasks.workunit.client.0.smithi101.stdout: 1 365161 8.85 MB/sec execute 112 sec latency 79.127 ms 2023-07-18T03:12:28.579 INFO:tasks.workunit.client.0.smithi101.stdout: 1 366242 8.82 MB/sec execute 113 sec latency 25.970 ms 2023-07-18T03:12:29.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:28 smithi179 ceph-mon[129229]: pgmap v150: 129 pgs: 129 active+clean; 223 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 196 op/s 2023-07-18T03:12:29.236 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:28 smithi101 ceph-mon[126379]: pgmap v150: 129 pgs: 129 active+clean; 223 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 196 op/s 2023-07-18T03:12:29.293 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:28 smithi155 ceph-mon[128885]: pgmap v150: 129 pgs: 129 active+clean; 223 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 196 op/s 2023-07-18T03:12:29.579 INFO:tasks.workunit.client.0.smithi101.stdout: 1 368306 8.85 MB/sec execute 114 sec latency 16.003 ms 2023-07-18T03:12:30.579 INFO:tasks.workunit.client.0.smithi101.stdout: 1 369464 8.83 MB/sec execute 115 sec latency 48.002 ms 2023-07-18T03:12:31.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:30 smithi179 ceph-mon[129229]: pgmap v151: 129 pgs: 129 active+clean; 228 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 296 op/s 2023-07-18T03:12:31.199 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:30 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:12:31.199 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:30 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:12:31.199 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:30 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:12:31.236 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:30 smithi101 ceph-mon[126379]: pgmap v151: 129 pgs: 129 active+clean; 228 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 296 op/s 2023-07-18T03:12:31.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:30 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:12:31.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:30 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:12:31.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:30 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:12:31.293 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:30 smithi155 ceph-mon[128885]: pgmap v151: 129 pgs: 129 active+clean; 228 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 296 op/s 2023-07-18T03:12:31.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:30 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:12:31.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:30 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:12:31.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:30 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:12:31.579 INFO:tasks.workunit.client.0.smithi101.stdout: 1 371286 8.81 MB/sec execute 116 sec latency 22.300 ms 2023-07-18T03:12:32.579 INFO:tasks.workunit.client.0.smithi101.stdout: 1 372827 8.84 MB/sec execute 117 sec latency 11.925 ms 2023-07-18T03:12:33.199 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:32 smithi179 ceph-mon[129229]: pgmap v152: 129 pgs: 129 active+clean; 228 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 194 op/s 2023-07-18T03:12:33.236 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:32 smithi101 ceph-mon[126379]: pgmap v152: 129 pgs: 129 active+clean; 228 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 194 op/s 2023-07-18T03:12:33.293 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:32 smithi155 ceph-mon[128885]: pgmap v152: 129 pgs: 129 active+clean; 228 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 194 op/s 2023-07-18T03:12:33.579 INFO:tasks.workunit.client.0.smithi101.stdout: 1 374080 8.81 MB/sec execute 118 sec latency 34.707 ms 2023-07-18T03:12:34.579 INFO:tasks.workunit.client.0.smithi101.stdout: 1 376198 8.85 MB/sec execute 119 sec latency 14.212 ms 2023-07-18T03:12:35.198 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:34 smithi179 ceph-mon[129229]: pgmap v153: 129 pgs: 129 active+clean; 228 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 194 op/s 2023-07-18T03:12:35.236 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:34 smithi101 ceph-mon[126379]: pgmap v153: 129 pgs: 129 active+clean; 228 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 194 op/s 2023-07-18T03:12:35.293 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:34 smithi155 ceph-mon[128885]: pgmap v153: 129 pgs: 129 active+clean; 228 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 194 op/s 2023-07-18T03:12:35.579 INFO:tasks.workunit.client.0.smithi101.stdout: 1 377144 8.79 MB/sec execute 120 sec latency 44.836 ms 2023-07-18T03:12:36.579 INFO:tasks.workunit.client.0.smithi101.stdout: 1 379287 8.82 MB/sec execute 121 sec latency 22.117 ms 2023-07-18T03:12:37.199 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:36 smithi179 ceph-mon[129229]: pgmap v154: 129 pgs: 129 active+clean; 233 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 294 op/s 2023-07-18T03:12:37.236 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:36 smithi101 ceph-mon[126379]: pgmap v154: 129 pgs: 129 active+clean; 233 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 294 op/s 2023-07-18T03:12:37.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:36 smithi155 ceph-mon[128885]: pgmap v154: 129 pgs: 129 active+clean; 233 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 294 op/s 2023-07-18T03:12:37.579 INFO:tasks.workunit.client.0.smithi101.stdout: 1 380449 8.80 MB/sec execute 122 sec latency 17.907 ms 2023-07-18T03:12:38.579 INFO:tasks.workunit.client.0.smithi101.stdout: 1 382542 8.82 MB/sec execute 123 sec latency 14.385 ms 2023-07-18T03:12:39.236 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:38 smithi101 ceph-mon[126379]: pgmap v155: 129 pgs: 129 active+clean; 233 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 199 op/s 2023-07-18T03:12:39.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:38 smithi155 ceph-mon[128885]: pgmap v155: 129 pgs: 129 active+clean; 233 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 199 op/s 2023-07-18T03:12:39.448 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:38 smithi179 ceph-mon[129229]: pgmap v155: 129 pgs: 129 active+clean; 233 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 199 op/s 2023-07-18T03:12:39.580 INFO:tasks.workunit.client.0.smithi101.stdout: 1 383771 8.80 MB/sec execute 124 sec latency 46.196 ms 2023-07-18T03:12:40.580 INFO:tasks.workunit.client.0.smithi101.stdout: 1 384930 8.78 MB/sec execute 125 sec latency 79.275 ms 2023-07-18T03:12:41.236 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:40 smithi101 ceph-mon[126379]: pgmap v156: 129 pgs: 129 active+clean; 238 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 296 op/s 2023-07-18T03:12:41.293 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:40 smithi155 ceph-mon[128885]: pgmap v156: 129 pgs: 129 active+clean; 238 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 296 op/s 2023-07-18T03:12:41.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:40 smithi179 ceph-mon[129229]: pgmap v156: 129 pgs: 129 active+clean; 238 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 296 op/s 2023-07-18T03:12:41.580 INFO:tasks.workunit.client.0.smithi101.stdout: 1 386914 8.81 MB/sec execute 126 sec latency 28.069 ms 2023-07-18T03:12:42.580 INFO:tasks.workunit.client.0.smithi101.stdout: 1 388169 8.79 MB/sec execute 127 sec latency 14.164 ms 2023-07-18T03:12:43.236 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:42 smithi101 ceph-mon[126379]: pgmap v157: 129 pgs: 129 active+clean; 238 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 197 op/s 2023-07-18T03:12:43.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:42 smithi155 ceph-mon[128885]: pgmap v157: 129 pgs: 129 active+clean; 238 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 197 op/s 2023-07-18T03:12:43.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:42 smithi179 ceph-mon[129229]: pgmap v157: 129 pgs: 129 active+clean; 238 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 197 op/s 2023-07-18T03:12:43.580 INFO:tasks.workunit.client.0.smithi101.stdout: 1 390232 8.81 MB/sec execute 128 sec latency 18.700 ms 2023-07-18T03:12:44.580 INFO:tasks.workunit.client.0.smithi101.stdout: 1 391164 8.76 MB/sec execute 129 sec latency 16.602 ms 2023-07-18T03:12:45.236 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:44 smithi101 ceph-mon[126379]: pgmap v158: 129 pgs: 129 active+clean; 238 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 197 op/s 2023-07-18T03:12:45.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:44 smithi155 ceph-mon[128885]: pgmap v158: 129 pgs: 129 active+clean; 238 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 197 op/s 2023-07-18T03:12:45.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:44 smithi179 ceph-mon[129229]: pgmap v158: 129 pgs: 129 active+clean; 238 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 197 op/s 2023-07-18T03:12:45.580 INFO:tasks.workunit.client.0.smithi101.stdout: 1 392815 8.75 MB/sec execute 130 sec latency 64.028 ms 2023-07-18T03:12:46.580 INFO:tasks.workunit.client.0.smithi101.stdout: 1 394235 8.77 MB/sec execute 131 sec latency 26.888 ms 2023-07-18T03:12:47.236 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:46 smithi101 ceph-mon[126379]: pgmap v159: 129 pgs: 129 active+clean; 243 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 298 op/s 2023-07-18T03:12:47.236 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:46 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:12:47.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:46 smithi155 ceph-mon[128885]: pgmap v159: 129 pgs: 129 active+clean; 243 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 298 op/s 2023-07-18T03:12:47.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:46 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:12:47.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:46 smithi179 ceph-mon[129229]: pgmap v159: 129 pgs: 129 active+clean; 243 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 298 op/s 2023-07-18T03:12:47.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:46 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:12:47.580 INFO:tasks.workunit.client.0.smithi101.stdout: 1 395543 8.74 MB/sec execute 132 sec latency 25.866 ms 2023-07-18T03:12:48.236 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:47 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:12:48.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:47 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:12:48.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:47 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:12:48.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:47 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:12:48.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:47 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:12:48.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:47 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:12:48.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:47 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:12:48.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:47 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:12:48.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:47 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:12:48.580 INFO:tasks.workunit.client.0.smithi101.stdout: 1 397554 8.78 MB/sec execute 133 sec latency 15.131 ms 2023-07-18T03:12:49.236 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:48 smithi101 ceph-mon[126379]: pgmap v160: 129 pgs: 129 active+clean; 243 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 197 op/s 2023-07-18T03:12:49.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:48 smithi155 ceph-mon[128885]: pgmap v160: 129 pgs: 129 active+clean; 243 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 197 op/s 2023-07-18T03:12:49.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:48 smithi179 ceph-mon[129229]: pgmap v160: 129 pgs: 129 active+clean; 243 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 197 op/s 2023-07-18T03:12:49.580 INFO:tasks.workunit.client.0.smithi101.stdout: 1 398530 8.73 MB/sec execute 134 sec latency 18.287 ms 2023-07-18T03:12:50.580 INFO:tasks.workunit.client.0.smithi101.stdout: 1 400326 8.75 MB/sec execute 135 sec latency 35.027 ms 2023-07-18T03:12:51.236 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:50 smithi101 ceph-mon[126379]: pgmap v161: 129 pgs: 129 active+clean; 248 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 291 op/s 2023-07-18T03:12:51.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:50 smithi155 ceph-mon[128885]: pgmap v161: 129 pgs: 129 active+clean; 248 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 291 op/s 2023-07-18T03:12:51.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:50 smithi179 ceph-mon[129229]: pgmap v161: 129 pgs: 129 active+clean; 248 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 291 op/s 2023-07-18T03:12:51.580 INFO:tasks.workunit.client.0.smithi101.stdout: 1 401464 8.73 MB/sec execute 136 sec latency 45.582 ms 2023-07-18T03:12:52.580 INFO:tasks.workunit.client.0.smithi101.stdout: 1 403289 8.72 MB/sec execute 137 sec latency 29.180 ms 2023-07-18T03:12:53.236 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:52 smithi101 ceph-mon[126379]: pgmap v162: 129 pgs: 129 active+clean; 248 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 194 op/s 2023-07-18T03:12:53.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:52 smithi155 ceph-mon[128885]: pgmap v162: 129 pgs: 129 active+clean; 248 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 194 op/s 2023-07-18T03:12:53.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:52 smithi179 ceph-mon[129229]: pgmap v162: 129 pgs: 129 active+clean; 248 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 194 op/s 2023-07-18T03:12:53.581 INFO:tasks.workunit.client.0.smithi101.stdout: 1 404818 8.74 MB/sec execute 138 sec latency 11.509 ms 2023-07-18T03:12:54.060 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:12:54.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.279+0000 7fdb409ff700 1 -- 172.21.15.101:0/2583753903 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdb3c100480 msgr2=0x7fdb3c10c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:54.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.279+0000 7fdb409ff700 1 --2- 172.21.15.101:0/2583753903 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdb3c100480 0x7fdb3c10c910 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fdb3400b0a0 tx=0x7fdb3400b3b0 comp rx=0 tx=0).stop 2023-07-18T03:12:54.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.279+0000 7fdb409ff700 1 -- 172.21.15.101:0/2583753903 shutdown_connections 2023-07-18T03:12:54.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.279+0000 7fdb409ff700 1 --2- 172.21.15.101:0/2583753903 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb3c105ae0 0x7fdb3c105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:54.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.279+0000 7fdb409ff700 1 --2- 172.21.15.101:0/2583753903 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdb3c100480 0x7fdb3c10c910 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:54.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.279+0000 7fdb409ff700 1 --2- 172.21.15.101:0/2583753903 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb3c0ffad0 0x7fdb3c0fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:54.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.279+0000 7fdb409ff700 1 -- 172.21.15.101:0/2583753903 >> 172.21.15.101:0/2583753903 conn(0x7fdb3c0fb5f0 msgr2=0x7fdb3c0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:54.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.279+0000 7fdb409ff700 1 -- 172.21.15.101:0/2583753903 shutdown_connections 2023-07-18T03:12:54.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.279+0000 7fdb409ff700 1 -- 172.21.15.101:0/2583753903 wait complete. 2023-07-18T03:12:54.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.280+0000 7fdb409ff700 1 Processor -- start 2023-07-18T03:12:54.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.280+0000 7fdb409ff700 1 -- start start 2023-07-18T03:12:54.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.281+0000 7fdb409ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb3c0ffad0 0x7fdb3c197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:54.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.281+0000 7fdb409ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb3c100480 0x7fdb3c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:54.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.281+0000 7fdb409ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdb3c105ae0 0x7fdb3c19c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:54.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.281+0000 7fdb409ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3c19ced0 con 0x7fdb3c105ae0 2023-07-18T03:12:54.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.281+0000 7fdb409ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3c19d040 con 0x7fdb3c0ffad0 2023-07-18T03:12:54.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.281+0000 7fdb409ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3c19d200 con 0x7fdb3c100480 2023-07-18T03:12:54.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.281+0000 7fdb3a59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb3c0ffad0 0x7fdb3c197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:54.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.281+0000 7fdb3a59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb3c0ffad0 0x7fdb3c197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:50522/0 (socket says 172.21.15.101:50522) 2023-07-18T03:12:54.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.281+0000 7fdb3a59c700 1 -- 172.21.15.101:0/3989200894 learned_addr learned my addr 172.21.15.101:0/3989200894 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:54.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.281+0000 7fdb3ad9d700 1 --2- 172.21.15.101:0/3989200894 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdb3c105ae0 0x7fdb3c19c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:54.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.281+0000 7fdb33fff700 1 --2- 172.21.15.101:0/3989200894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb3c100480 0x7fdb3c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:54.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.282+0000 7fdb3a59c700 1 -- 172.21.15.101:0/3989200894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb3c100480 msgr2=0x7fdb3c198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:54.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.282+0000 7fdb3a59c700 1 --2- 172.21.15.101:0/3989200894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb3c100480 0x7fdb3c198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:54.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.282+0000 7fdb3a59c700 1 -- 172.21.15.101:0/3989200894 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdb3c105ae0 msgr2=0x7fdb3c19c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:54.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.282+0000 7fdb3a59c700 1 --2- 172.21.15.101:0/3989200894 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdb3c105ae0 0x7fdb3c19c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:54.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.282+0000 7fdb3a59c700 1 -- 172.21.15.101:0/3989200894 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb34009d00 con 0x7fdb3c0ffad0 2023-07-18T03:12:54.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.282+0000 7fdb33fff700 1 --2- 172.21.15.101:0/3989200894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb3c100480 0x7fdb3c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:12:54.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.282+0000 7fdb3a59c700 1 --2- 172.21.15.101:0/3989200894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb3c0ffad0 0x7fdb3c197ef0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c00e9a0 tx=0x7fdb2c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:54.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.282+0000 7fdb337fe700 1 -- 172.21.15.101:0/3989200894 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c009790 con 0x7fdb3c0ffad0 2023-07-18T03:12:54.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.282+0000 7fdb337fe700 1 -- 172.21.15.101:0/3989200894 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb2c00cbd0 con 0x7fdb3c0ffad0 2023-07-18T03:12:54.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.282+0000 7fdb409ff700 1 -- 172.21.15.101:0/3989200894 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb3c19d4e0 con 0x7fdb3c0ffad0 2023-07-18T03:12:54.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.282+0000 7fdb337fe700 1 -- 172.21.15.101:0/3989200894 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c018680 con 0x7fdb3c0ffad0 2023-07-18T03:12:54.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.282+0000 7fdb409ff700 1 -- 172.21.15.101:0/3989200894 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb3c1a4d20 con 0x7fdb3c0ffad0 2023-07-18T03:12:54.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.285+0000 7fdb337fe700 1 -- 172.21.15.101:0/3989200894 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fdb2c00cd40 con 0x7fdb3c0ffad0 2023-07-18T03:12:54.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.286+0000 7fdb542eab80 1 Processor -- start 2023-07-18T03:12:54.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.287+0000 7fdb542eab80 1 -- start start 2023-07-18T03:12:54.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.287+0000 7fdb542eab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2367ab2d0 0x55c2367ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:54.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.287+0000 7fdb542eab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c2367abbe0 0x55c2367b6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:54.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.287+0000 7fdb542eab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2367b6ad0 0x55c2367b8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:54.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.287+0000 7fdb542eab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2366b1cb0 con 0x55c2367abbe0 2023-07-18T03:12:54.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.287+0000 7fdb542eab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c236704a40 con 0x55c2367b6ad0 2023-07-18T03:12:54.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.287+0000 7fdb542eab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2366fded0 con 0x55c2367ab2d0 2023-07-18T03:12:54.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.287+0000 7fdb33fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2367b6ad0 0x55c2367b8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:54.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.287+0000 7fdb3ad9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2367ab2d0 0x55c2367ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:54.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.287+0000 7fdb33fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2367b6ad0 0x55c2367b8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:50526/0 (socket says 172.21.15.101:50526) 2023-07-18T03:12:54.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.287+0000 7fdb33fff700 1 -- 172.21.15.101:0/3777556389 learned_addr learned my addr 172.21.15.101:0/3777556389 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:54.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.287+0000 7fdb3a59c700 1 --2- 172.21.15.101:0/3777556389 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c2367abbe0 0x55c2367b6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:54.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.287+0000 7fdb337fe700 1 --2- 172.21.15.101:0/3989200894 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fdb280a12c0 0x7fdb280a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:54.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.288+0000 7fdb33fff700 1 -- 172.21.15.101:0/3777556389 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2367ab2d0 msgr2=0x55c2367ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:54.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.288+0000 7fdb33fff700 1 --2- 172.21.15.101:0/3777556389 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2367ab2d0 0x55c2367ab6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:54.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.288+0000 7fdb33fff700 1 -- 172.21.15.101:0/3777556389 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c2367abbe0 msgr2=0x55c2367b6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:54.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.288+0000 7fdb33fff700 1 --2- 172.21.15.101:0/3777556389 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c2367abbe0 0x55c2367b6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:54.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.288+0000 7fdb33fff700 1 -- 172.21.15.101:0/3777556389 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c23652bde0 con 0x55c2367b6ad0 2023-07-18T03:12:54.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.288+0000 7fdb3a59c700 1 --2- 172.21.15.101:0/3777556389 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c2367abbe0 0x55c2367b6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:12:54.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.288+0000 7fdb337fe700 1 -- 172.21.15.101:0/3989200894 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fdb2c014070 con 0x7fdb3c0ffad0 2023-07-18T03:12:54.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.288+0000 7fdb3ad9d700 1 --2- 172.21.15.101:0/3989200894 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fdb280a12c0 0x7fdb280a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:54.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.288+0000 7fdb33fff700 1 --2- 172.21.15.101:0/3777556389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2367b6ad0 0x55c2367b8eb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdb24009d80 tx=0x7fdb240116c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de5c5ada0933873d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:54.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.288+0000 7fdb30ff9700 1 -- 172.21.15.101:0/3777556389 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb24011e20 con 0x55c2367b6ad0 2023-07-18T03:12:54.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.288+0000 7fdb30ff9700 1 -- 172.21.15.101:0/3777556389 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb24012910 con 0x55c2367b6ad0 2023-07-18T03:12:54.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.288+0000 7fdb30ff9700 1 -- 172.21.15.101:0/3777556389 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2400f840 con 0x55c2367b6ad0 2023-07-18T03:12:54.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.289+0000 7fdb542eab80 1 -- 172.21.15.101:0/3777556389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2367b6ad0 msgr2=0x55c2367b8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:54.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.289+0000 7fdb3ad9d700 1 --2- 172.21.15.101:0/3989200894 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fdb280a12c0 0x7fdb280a3770 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdb3400b780 tx=0x7fdb340099e0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:54.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.289+0000 7fdb542eab80 1 --2- 172.21.15.101:0/3777556389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2367b6ad0 0x55c2367b8eb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdb24009d80 tx=0x7fdb240116c0 comp rx=0 tx=0).stop 2023-07-18T03:12:54.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.289+0000 7fdb542eab80 1 -- 172.21.15.101:0/3777556389 shutdown_connections 2023-07-18T03:12:54.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.289+0000 7fdb542eab80 1 --2- 172.21.15.101:0/3777556389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2367b6ad0 0x55c2367b8eb0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:54.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.289+0000 7fdb542eab80 1 --2- 172.21.15.101:0/3777556389 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2367ab2d0 0x55c2367ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:54.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.289+0000 7fdb542eab80 1 --2- 172.21.15.101:0/3777556389 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c2367abbe0 0x55c2367b6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:54.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.289+0000 7fdb542eab80 1 -- 172.21.15.101:0/3777556389 >> 172.21.15.101:0/3777556389 conn(0x55c2367a1da0 msgr2=0x55c2367a76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:54.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.289+0000 7fdb542eab80 1 -- 172.21.15.101:0/3777556389 shutdown_connections 2023-07-18T03:12:54.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.289+0000 7fdb542eab80 1 -- 172.21.15.101:0/3777556389 wait complete. 2023-07-18T03:12:54.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.290+0000 7fdb542eab80 1 -- start start 2023-07-18T03:12:54.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.290+0000 7fdb542eab80 1 Processor -- start 2023-07-18T03:12:54.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.290+0000 7fdb542eab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2367ab2d0 0x55c2367b6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:54.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.290+0000 7fdb542eab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c2367abbe0 0x55c2367b3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:54.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.290+0000 7fdb542eab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2367b6ad0 0x55c2367b3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:54.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.290+0000 7fdb542eab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c23669dac0 con 0x55c2367abbe0 2023-07-18T03:12:54.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.290+0000 7fdb542eab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c23669d7c0 con 0x55c2367b6ad0 2023-07-18T03:12:54.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.290+0000 7fdb542eab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c236581920 con 0x55c2367ab2d0 2023-07-18T03:12:54.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.291+0000 7fdb542eab80 10 client.0 _ll_register_callbacks cb 0x55c23669b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:12:54.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.291+0000 7fdb3a59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2367ab2d0 0x55c2367b6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:54.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.291+0000 7fdb33fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c2367abbe0 0x55c2367b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:54.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.291+0000 7fdb3ad9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2367b6ad0 0x55c2367b3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:54.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.291+0000 7fdb3a59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2367ab2d0 0x55c2367b6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:50902/0 (socket says 172.21.15.101:50902) 2023-07-18T03:12:54.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.291+0000 7fdb3a59c700 1 -- 172.21.15.101:0/185051761 learned_addr learned my addr 172.21.15.101:0/185051761 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:54.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.291+0000 7fdb3a59c700 1 -- 172.21.15.101:0/185051761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2367b6ad0 msgr2=0x55c2367b3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:54.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.291+0000 7fdb3a59c700 1 --2- 172.21.15.101:0/185051761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2367b6ad0 0x55c2367b3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:54.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.291+0000 7fdb3a59c700 1 -- 172.21.15.101:0/185051761 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c2367abbe0 msgr2=0x55c2367b3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:54.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.291+0000 7fdb3a59c700 1 --2- 172.21.15.101:0/185051761 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c2367abbe0 0x55c2367b3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:54.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.291+0000 7fdb3a59c700 1 -- 172.21.15.101:0/185051761 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb240099b0 con 0x55c2367ab2d0 2023-07-18T03:12:54.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.291+0000 7fdb33fff700 1 --2- 172.21.15.101:0/185051761 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c2367abbe0 0x55c2367b3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:12:54.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.291+0000 7fdb3a59c700 1 --2- 172.21.15.101:0/185051761 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2367ab2d0 0x55c2367b6510 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c00e970 tx=0x7fdb2c0cabc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=662552221fd2d4d9 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:54.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.291+0000 7fdb21ffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:12:54.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.291+0000 7fdb21ffb700 1 -- 172.21.15.101:0/185051761 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c09a440 con 0x55c2367ab2d0 2023-07-18T03:12:54.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.292+0000 7fdb21ffb700 1 -- 172.21.15.101:0/185051761 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fdb2c099fe0 con 0x55c2367ab2d0 2023-07-18T03:12:54.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.292+0000 7fdb542eab80 1 -- 172.21.15.101:0/185051761 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c2366574f0 con 0x55c2367ab2d0 2023-07-18T03:12:54.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.292+0000 7fdb21ffb700 1 -- 172.21.15.101:0/185051761 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb2c0cf320 con 0x55c2367ab2d0 2023-07-18T03:12:54.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.292+0000 7fdb542eab80 10 client.26641 fetch_fsmap learned FSMap version 26 2023-07-18T03:12:54.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.292+0000 7fdb542eab80 1 -- 172.21.15.101:0/185051761 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55c236514c60 con 0x55c2367ab2d0 2023-07-18T03:12:54.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.292+0000 7fdb21ffb700 1 -- 172.21.15.101:0/185051761 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c2366574f0 con 0x55c2367ab2d0 2023-07-18T03:12:54.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.292+0000 7fdb21ffb700 1 -- 172.21.15.101:0/185051761 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdb2c0a1fa0 con 0x55c2367ab2d0 2023-07-18T03:12:54.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.292+0000 7fdb542eab80 10 client.26641 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:12:54.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.292+0000 7fdb542eab80 10 client.26641 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:12:54.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.292+0000 7fdb542eab80 20 client.26641 populate_metadata read hostname 'smithi101' 2023-07-18T03:12:54.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.292+0000 7fdb542eab80 1 --2- 172.21.15.101:0/185051761 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c2367c6260 0x55c2367c8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:54.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.292+0000 7fdb542eab80 4 client.26641 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:12:54.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.292+0000 7fdb542eab80 1 -- 172.21.15.101:0/185051761 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c2366f6eb0 con 0x55c2367c6260 2023-07-18T03:12:54.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.293+0000 7fdb33fff700 1 --2- 172.21.15.101:0/185051761 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c2367c6260 0x55c2367c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:54.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.293+0000 7fdb33fff700 1 --2- 172.21.15.101:0/185051761 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c2367c6260 0x55c2367c8640 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1766ddb46768bb5 server_cookie=d5ada7dda11c4a32 in_seq=0 out_seq=0 2023-07-18T03:12:54.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.293+0000 7fdb21ffb700 10 client.26641 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:12:54.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.294+0000 7fdb21ffb700 1 -- 172.21.15.101:0/185051761 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c2366f6eb0 con 0x55c2367c6260 2023-07-18T03:12:54.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.294+0000 7fdb21ffb700 10 client.26641 handle_command_reply: tid=0 2023-07-18T03:12:54.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.294+0000 7fdb542eab80 1 client.26641 shutdown 2023-07-18T03:12:54.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.294+0000 7fdb542eab80 1 -- 172.21.15.101:0/185051761 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2367ab2d0 msgr2=0x55c2367b6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:54.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.294+0000 7fdb542eab80 1 --2- 172.21.15.101:0/185051761 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2367ab2d0 0x55c2367b6510 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c00e970 tx=0x7fdb2c0cabc0 comp rx=0 tx=0).stop 2023-07-18T03:12:54.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.294+0000 7fdb542eab80 1 -- 172.21.15.101:0/185051761 shutdown_connections 2023-07-18T03:12:54.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.294+0000 7fdb542eab80 1 --2- 172.21.15.101:0/185051761 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c2367c6260 0x55c2367c8640 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:54.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.294+0000 7fdb542eab80 1 --2- 172.21.15.101:0/185051761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2367b6ad0 0x55c2367b3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:54.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.294+0000 7fdb542eab80 1 --2- 172.21.15.101:0/185051761 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2367ab2d0 0x55c2367b6510 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:54.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.294+0000 7fdb542eab80 1 --2- 172.21.15.101:0/185051761 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c2367abbe0 0x55c2367b3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:54.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.294+0000 7fdb21ffb700 0 client.26641 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:12:54.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.294+0000 7fdb542eab80 1 -- 172.21.15.101:0/185051761 >> 172.21.15.101:0/185051761 conn(0x55c2367a1da0 msgr2=0x55c2367a7230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:54.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.294+0000 7fdb542eab80 1 -- 172.21.15.101:0/185051761 shutdown_connections 2023-07-18T03:12:54.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.294+0000 7fdb542eab80 1 -- 172.21.15.101:0/185051761 wait complete. 2023-07-18T03:12:54.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.294+0000 7fdb542eab80 20 client.26641 trim_cache size 0 max 16384 2023-07-18T03:12:54.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.312+0000 7fdb542eab80 1 Processor -- start 2023-07-18T03:12:54.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.313+0000 7fdb542eab80 1 -- start start 2023-07-18T03:12:54.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.313+0000 7fdb542eab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c2367ab2d0 0x55c2367b6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:54.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.313+0000 7fdb542eab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2367abbe0 0x55c2367aea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:54.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.313+0000 7fdb542eab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2367b6ad0 0x55c2367aef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:54.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.313+0000 7fdb542eab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2367ac540 con 0x55c2367ab2d0 2023-07-18T03:12:54.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.313+0000 7fdb542eab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2367ac160 con 0x55c2367abbe0 2023-07-18T03:12:54.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.313+0000 7fdb542eab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2367c6040 con 0x55c2367b6ad0 2023-07-18T03:12:54.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.313+0000 7fdb3a59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c2367ab2d0 0x55c2367b6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:54.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.313+0000 7fdb33fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2367abbe0 0x55c2367aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:54.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.313+0000 7fdb3ad9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2367b6ad0 0x55c2367aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:54.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.313+0000 7fdb33fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2367abbe0 0x55c2367aea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:50554/0 (socket says 172.21.15.101:50554) 2023-07-18T03:12:54.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.313+0000 7fdb33fff700 1 -- 172.21.15.101:0/2060577891 learned_addr learned my addr 172.21.15.101:0/2060577891 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:54.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.314+0000 7fdb33fff700 1 -- 172.21.15.101:0/2060577891 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2367b6ad0 msgr2=0x55c2367aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:54.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.314+0000 7fdb33fff700 1 --2- 172.21.15.101:0/2060577891 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2367b6ad0 0x55c2367aef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:54.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.314+0000 7fdb33fff700 1 -- 172.21.15.101:0/2060577891 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c2367ab2d0 msgr2=0x55c2367b6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:54.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.314+0000 7fdb33fff700 1 --2- 172.21.15.101:0/2060577891 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c2367ab2d0 0x55c2367b6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:54.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.314+0000 7fdb33fff700 1 -- 172.21.15.101:0/2060577891 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c236552250 con 0x55c2367abbe0 2023-07-18T03:12:54.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.314+0000 7fdb3a59c700 1 --2- 172.21.15.101:0/2060577891 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c2367ab2d0 0x55c2367b6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:12:54.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.314+0000 7fdb33fff700 1 --2- 172.21.15.101:0/2060577891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2367abbe0 0x55c2367aea50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdb2401c040 tx=0x7fdb2400f440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10b7ff57fcda2a3d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:54.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.314+0000 7fdb317fa700 1 -- 172.21.15.101:0/2060577891 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb24009ed0 con 0x55c2367abbe0 2023-07-18T03:12:54.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.314+0000 7fdb317fa700 1 -- 172.21.15.101:0/2060577891 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb24029b50 con 0x55c2367abbe0 2023-07-18T03:12:54.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.314+0000 7fdb317fa700 1 -- 172.21.15.101:0/2060577891 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb24005310 con 0x55c2367abbe0 2023-07-18T03:12:54.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.315+0000 7fdb542eab80 1 -- 172.21.15.101:0/2060577891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2367abbe0 msgr2=0x55c2367aea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:54.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.315+0000 7fdb542eab80 1 --2- 172.21.15.101:0/2060577891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2367abbe0 0x55c2367aea50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdb2401c040 tx=0x7fdb2400f440 comp rx=0 tx=0).stop 2023-07-18T03:12:54.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.315+0000 7fdb542eab80 1 -- 172.21.15.101:0/2060577891 shutdown_connections 2023-07-18T03:12:54.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.315+0000 7fdb542eab80 1 --2- 172.21.15.101:0/2060577891 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2367b6ad0 0x55c2367aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:54.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.315+0000 7fdb542eab80 1 --2- 172.21.15.101:0/2060577891 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c2367ab2d0 0x55c2367b6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:54.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.315+0000 7fdb542eab80 1 --2- 172.21.15.101:0/2060577891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2367abbe0 0x55c2367aea50 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:54.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.315+0000 7fdb542eab80 1 -- 172.21.15.101:0/2060577891 >> 172.21.15.101:0/2060577891 conn(0x55c2367a1da0 msgr2=0x55c2367c1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:54.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.315+0000 7fdb542eab80 1 -- 172.21.15.101:0/2060577891 shutdown_connections 2023-07-18T03:12:54.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.315+0000 7fdb542eab80 1 -- 172.21.15.101:0/2060577891 wait complete. 2023-07-18T03:12:54.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.315+0000 7fdb542eab80 1 -- start start 2023-07-18T03:12:54.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.316+0000 7fdb542eab80 1 Processor -- start 2023-07-18T03:12:54.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.316+0000 7fdb542eab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c2367ab2d0 0x55c2367bd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:54.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.316+0000 7fdb542eab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2367abbe0 0x55c2367bb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:54.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.316+0000 7fdb542eab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2367b6ad0 0x55c2367bba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:54.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.316+0000 7fdb542eab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2367bdaa0 con 0x55c2367ab2d0 2023-07-18T03:12:54.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.316+0000 7fdb542eab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2367bbf70 con 0x55c2367b6ad0 2023-07-18T03:12:54.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.316+0000 7fdb542eab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2367bc130 con 0x55c2367abbe0 2023-07-18T03:12:54.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.316+0000 7fdb542eab80 10 client.0 _ll_register_callbacks cb 0x7fdb2c099b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:12:54.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.316+0000 7fdb3a59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c2367ab2d0 0x55c2367bd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:54.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.317+0000 7fdb33fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2367abbe0 0x55c2367bb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:54.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.317+0000 7fdb3a59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c2367ab2d0 0x55c2367bd1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:35634/0 (socket says 172.21.15.101:35634) 2023-07-18T03:12:54.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.317+0000 7fdb3ad9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2367b6ad0 0x55c2367bba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:54.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.317+0000 7fdb3a59c700 1 -- 172.21.15.101:0/3657991715 learned_addr learned my addr 172.21.15.101:0/3657991715 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:54.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.317+0000 7fdb33fff700 1 -- 172.21.15.101:0/3657991715 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2367b6ad0 msgr2=0x55c2367bba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:54.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.317+0000 7fdb33fff700 1 --2- 172.21.15.101:0/3657991715 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2367b6ad0 0x55c2367bba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:54.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.317+0000 7fdb33fff700 1 -- 172.21.15.101:0/3657991715 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c2367ab2d0 msgr2=0x55c2367bd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:54.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.317+0000 7fdb33fff700 1 --2- 172.21.15.101:0/3657991715 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c2367ab2d0 0x55c2367bd1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:54.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.317+0000 7fdb33fff700 1 -- 172.21.15.101:0/3657991715 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c236514c60 con 0x55c2367abbe0 2023-07-18T03:12:54.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.317+0000 7fdb33fff700 1 --2- 172.21.15.101:0/3657991715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2367abbe0 0x55c2367bb4f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdb24011a10 tx=0x7fdb24009f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ececff2bc318ca3c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:54.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.317+0000 7fdb21ffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:12:54.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.317+0000 7fdb21ffb700 1 -- 172.21.15.101:0/3657991715 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb24029160 con 0x55c2367abbe0 2023-07-18T03:12:54.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.317+0000 7fdb542eab80 1 -- 172.21.15.101:0/3657991715 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb24012910 con 0x55c2367abbe0 2023-07-18T03:12:54.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.317+0000 7fdb21ffb700 1 -- 172.21.15.101:0/3657991715 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fdb240292c0 con 0x55c2367abbe0 2023-07-18T03:12:54.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.318+0000 7fdb21ffb700 1 -- 172.21.15.101:0/3657991715 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb24012450 con 0x55c2367abbe0 2023-07-18T03:12:54.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.318+0000 7fdb542eab80 10 client.26647 fetch_fsmap learned FSMap version 26 2023-07-18T03:12:54.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.318+0000 7fdb542eab80 1 -- 172.21.15.101:0/3657991715 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55c23652bde0 con 0x55c2367abbe0 2023-07-18T03:12:54.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.318+0000 7fdb21ffb700 1 -- 172.21.15.101:0/3657991715 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb24012910 con 0x55c2367abbe0 2023-07-18T03:12:54.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.318+0000 7fdb21ffb700 1 -- 172.21.15.101:0/3657991715 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdb24022030 con 0x55c2367abbe0 2023-07-18T03:12:54.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.318+0000 7fdb542eab80 10 client.26647 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:12:54.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.318+0000 7fdb542eab80 10 client.26647 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:12:54.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.318+0000 7fdb542eab80 20 client.26647 populate_metadata read hostname 'smithi101' 2023-07-18T03:12:54.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.318+0000 7fdb542eab80 1 --2- 172.21.15.101:0/3657991715 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c23681b170 0x55c23681d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:54.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.318+0000 7fdb542eab80 4 client.26647 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:12:54.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.318+0000 7fdb542eab80 1 -- 172.21.15.101:0/3657991715 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c2366b9050 con 0x55c23681b170 2023-07-18T03:12:54.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.318+0000 7fdb3a59c700 1 --2- 172.21.15.101:0/3657991715 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c23681b170 0x55c23681d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:54.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.319+0000 7fdb21ffb700 10 client.26647 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:12:54.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.319+0000 7fdb3a59c700 1 --2- 172.21.15.101:0/3657991715 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c23681b170 0x55c23681d580 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6e1d6c8ae345e02 server_cookie=a15dd9b62b8bc352 in_seq=0 out_seq=0 2023-07-18T03:12:54.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.319+0000 7fdb21ffb700 1 -- 172.21.15.101:0/3657991715 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c2366b9050 con 0x55c23681b170 2023-07-18T03:12:54.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.319+0000 7fdb21ffb700 10 client.26647 handle_command_reply: tid=0 2023-07-18T03:12:54.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.319+0000 7fdb542eab80 1 client.26647 shutdown 2023-07-18T03:12:54.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.319+0000 7fdb542eab80 1 -- 172.21.15.101:0/3657991715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2367abbe0 msgr2=0x55c2367bb4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:54.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.319+0000 7fdb542eab80 1 --2- 172.21.15.101:0/3657991715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2367abbe0 0x55c2367bb4f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdb24011a10 tx=0x7fdb24009f60 comp rx=0 tx=0).stop 2023-07-18T03:12:54.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.319+0000 7fdb542eab80 1 -- 172.21.15.101:0/3657991715 shutdown_connections 2023-07-18T03:12:54.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.319+0000 7fdb542eab80 1 --2- 172.21.15.101:0/3657991715 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c23681b170 0x55c23681d580 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:54.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.319+0000 7fdb542eab80 1 --2- 172.21.15.101:0/3657991715 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2367b6ad0 0x55c2367bba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:54.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.319+0000 7fdb21ffb700 0 client.26647 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:12:54.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.319+0000 7fdb542eab80 1 --2- 172.21.15.101:0/3657991715 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c2367ab2d0 0x55c2367bd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:54.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.319+0000 7fdb542eab80 1 --2- 172.21.15.101:0/3657991715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2367abbe0 0x55c2367bb4f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:54.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.319+0000 7fdb542eab80 1 -- 172.21.15.101:0/3657991715 >> 172.21.15.101:0/3657991715 conn(0x55c2367a1da0 msgr2=0x55c2367aea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:54.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.319+0000 7fdb542eab80 1 -- 172.21.15.101:0/3657991715 shutdown_connections 2023-07-18T03:12:54.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.319+0000 7fdb542eab80 1 -- 172.21.15.101:0/3657991715 wait complete. 2023-07-18T03:12:54.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.320+0000 7fdb542eab80 20 client.26647 trim_cache size 0 max 16384 2023-07-18T03:12:54.320 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:12:54.321 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "no active scrubs running", 2023-07-18T03:12:54.321 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": {} 2023-07-18T03:12:54.321 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:12:54.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.320+0000 7fdb409ff700 1 -- 172.21.15.101:0/3989200894 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fdb280a12c0 msgr2=0x7fdb280a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:54.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.320+0000 7fdb409ff700 1 --2- 172.21.15.101:0/3989200894 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fdb280a12c0 0x7fdb280a3770 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdb3400b780 tx=0x7fdb340099e0 comp rx=0 tx=0).stop 2023-07-18T03:12:54.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.320+0000 7fdb409ff700 1 -- 172.21.15.101:0/3989200894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb3c0ffad0 msgr2=0x7fdb3c197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:54.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.320+0000 7fdb409ff700 1 --2- 172.21.15.101:0/3989200894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb3c0ffad0 0x7fdb3c197ef0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c00e9a0 tx=0x7fdb2c00ecb0 comp rx=0 tx=0).stop 2023-07-18T03:12:54.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.320+0000 7fdb409ff700 1 -- 172.21.15.101:0/3989200894 shutdown_connections 2023-07-18T03:12:54.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.320+0000 7fdb409ff700 1 --2- 172.21.15.101:0/3989200894 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fdb280a12c0 0x7fdb280a3770 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:54.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.320+0000 7fdb409ff700 1 --2- 172.21.15.101:0/3989200894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb3c0ffad0 0x7fdb3c197ef0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:54.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.320+0000 7fdb409ff700 1 --2- 172.21.15.101:0/3989200894 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdb3c105ae0 0x7fdb3c19c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:54.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.320+0000 7fdb409ff700 1 --2- 172.21.15.101:0/3989200894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb3c100480 0x7fdb3c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:54.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.320+0000 7fdb409ff700 1 -- 172.21.15.101:0/3989200894 >> 172.21.15.101:0/3989200894 conn(0x7fdb3c0fb5f0 msgr2=0x7fdb3c10ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:54.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.320+0000 7fdb409ff700 1 -- 172.21.15.101:0/3989200894 shutdown_connections 2023-07-18T03:12:54.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.320+0000 7fdb409ff700 1 -- 172.21.15.101:0/3989200894 wait complete. 2023-07-18T03:12:54.331 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-18T03:12:54.331 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:12:54.555 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.562+0000 7f255b3ff700 1 -- 172.21.15.101:0/45283562 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2554105b00 msgr2=0x7f2554105ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:54.556 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.562+0000 7f255b3ff700 1 --2- 172.21.15.101:0/45283562 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2554105b00 0x7f2554105ed0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f25500098f0 tx=0x7f2550009c00 comp rx=0 tx=0).stop 2023-07-18T03:12:54.556 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.562+0000 7f255b3ff700 1 -- 172.21.15.101:0/45283562 shutdown_connections 2023-07-18T03:12:54.556 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.562+0000 7f255b3ff700 1 --2- 172.21.15.101:0/45283562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25541004a0 0x7f255410c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:54.556 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.562+0000 7f255b3ff700 1 --2- 172.21.15.101:0/45283562 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2554105b00 0x7f2554105ed0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:54.556 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.562+0000 7f255b3ff700 1 --2- 172.21.15.101:0/45283562 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25540ffaf0 0x7f25540fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:54.556 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.562+0000 7f255b3ff700 1 -- 172.21.15.101:0/45283562 >> 172.21.15.101:0/45283562 conn(0x7f25540fb630 msgr2=0x7f25540fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:54.557 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.562+0000 7f255b3ff700 1 -- 172.21.15.101:0/45283562 shutdown_connections 2023-07-18T03:12:54.557 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.562+0000 7f255b3ff700 1 -- 172.21.15.101:0/45283562 wait complete. 2023-07-18T03:12:54.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.564+0000 7f255b3ff700 1 Processor -- start 2023-07-18T03:12:54.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.564+0000 7f255b3ff700 1 -- start start 2023-07-18T03:12:54.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.564+0000 7f255b3ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f25540ffaf0 0x7f255419ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:54.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.564+0000 7f255b3ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25541004a0 0x7f255419bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:54.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.564+0000 7f255b3ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2554105b00 0x7f2554195aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:54.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.565+0000 7f255b3ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2554196110 con 0x7f25540ffaf0 2023-07-18T03:12:54.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.565+0000 7f255b3ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2554196280 con 0x7f2554105b00 2023-07-18T03:12:54.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.565+0000 7f255b3ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2554196440 con 0x7f25541004a0 2023-07-18T03:12:54.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.565+0000 7f255919b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f25540ffaf0 0x7f255419ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:54.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.565+0000 7f255919b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f25540ffaf0 0x7f255419ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:35646/0 (socket says 172.21.15.101:35646) 2023-07-18T03:12:54.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.565+0000 7f255919b700 1 -- 172.21.15.101:0/1502258105 learned_addr learned my addr 172.21.15.101:0/1502258105 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:54.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.565+0000 7f255899a700 1 --2- 172.21.15.101:0/1502258105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25541004a0 0x7f255419bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:54.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.565+0000 7f255999c700 1 --2- 172.21.15.101:0/1502258105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2554105b00 0x7f2554195aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:54.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.565+0000 7f255919b700 1 -- 172.21.15.101:0/1502258105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25541004a0 msgr2=0x7f255419bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:54.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.565+0000 7f255919b700 1 --2- 172.21.15.101:0/1502258105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25541004a0 0x7f255419bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:54.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.565+0000 7f255919b700 1 -- 172.21.15.101:0/1502258105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2554105b00 msgr2=0x7f2554195aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:54.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.565+0000 7f255919b700 1 --2- 172.21.15.101:0/1502258105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2554105b00 0x7f2554195aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:54.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.565+0000 7f255919b700 1 -- 172.21.15.101:0/1502258105 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25440095f0 con 0x7f25540ffaf0 2023-07-18T03:12:54.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.565+0000 7f255919b700 1 --2- 172.21.15.101:0/1502258105 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f25540ffaf0 0x7f255419ba30 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f2550009810 tx=0x7f255000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:54.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.566+0000 7f254a7fc700 1 -- 172.21.15.101:0/1502258105 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255001c070 con 0x7f25540ffaf0 2023-07-18T03:12:54.562 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.566+0000 7f254a7fc700 1 -- 172.21.15.101:0/1502258105 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f255000bb20 con 0x7f25540ffaf0 2023-07-18T03:12:54.562 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.566+0000 7f255b3ff700 1 -- 172.21.15.101:0/1502258105 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2550009520 con 0x7f25540ffaf0 2023-07-18T03:12:54.562 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.566+0000 7f254a7fc700 1 -- 172.21.15.101:0/1502258105 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2550004930 con 0x7f25540ffaf0 2023-07-18T03:12:54.562 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.566+0000 7f255b3ff700 1 -- 172.21.15.101:0/1502258105 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25541a2840 con 0x7f25540ffaf0 2023-07-18T03:12:54.562 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.567+0000 7f255b3ff700 1 -- 172.21.15.101:0/1502258105 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2554067300 con 0x7f25540ffaf0 2023-07-18T03:12:54.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.569+0000 7f254a7fc700 1 -- 172.21.15.101:0/1502258105 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f2550020070 con 0x7f25540ffaf0 2023-07-18T03:12:54.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.570+0000 7f254a7fc700 1 --2- 172.21.15.101:0/1502258105 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f25400a1300 0x7f25400a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:54.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.570+0000 7f254a7fc700 1 -- 172.21.15.101:0/1502258105 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f25500cdd80 con 0x7f25540ffaf0 2023-07-18T03:12:54.564 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.571+0000 7f255899a700 1 --2- 172.21.15.101:0/1502258105 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f25400a1300 0x7f25400a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:54.564 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.571+0000 7f255899a700 1 --2- 172.21.15.101:0/1502258105 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f25400a1300 0x7f25400a37b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f25540fdf30 tx=0x7f2544009310 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:54.567 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.575+0000 7f254a7fc700 1 -- 172.21.15.101:0/1502258105 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2550096e70 con 0x7f25540ffaf0 2023-07-18T03:12:54.581 INFO:tasks.workunit.client.0.smithi101.stdout: 1 406092 8.72 MB/sec execute 139 sec latency 60.863 ms 2023-07-18T03:12:54.826 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.834+0000 7f255b3ff700 1 -- 172.21.15.101:0/1502258105 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2554196fb0 con 0x7f25540ffaf0 2023-07-18T03:12:54.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.835+0000 7f254a7fc700 1 -- 172.21.15.101:0/1502258105 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10858 (secure 0 0 0) 0x7f2550005140 con 0x7f25540ffaf0 2023-07-18T03:12:54.827 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:12:54.828 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/4192846769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":4192846769},{"type":"v1","addr":"172.21.15.101:6839","nonce":4192846769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6841/3824340175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6840","nonce":3824340175},{"type":"v1","addr":"172.21.15.101:6841","nonce":3824340175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:04:59.361185+0000","modified":"2023-07-18T03:11:46.116493+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.155:6839/2055462483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2055462483},{"type":"v1","addr":"172.21.15.155:6839","nonce":2055462483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:12:54.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.837+0000 7f255b3ff700 1 -- 172.21.15.101:0/1502258105 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f25400a1300 msgr2=0x7f25400a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:54.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.837+0000 7f255b3ff700 1 --2- 172.21.15.101:0/1502258105 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f25400a1300 0x7f25400a37b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f25540fdf30 tx=0x7f2544009310 comp rx=0 tx=0).stop 2023-07-18T03:12:54.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.837+0000 7f255b3ff700 1 -- 172.21.15.101:0/1502258105 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f25540ffaf0 msgr2=0x7f255419ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:54.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.837+0000 7f255b3ff700 1 --2- 172.21.15.101:0/1502258105 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f25540ffaf0 0x7f255419ba30 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f2550009810 tx=0x7f255000fa00 comp rx=0 tx=0).stop 2023-07-18T03:12:54.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.837+0000 7f255b3ff700 1 -- 172.21.15.101:0/1502258105 shutdown_connections 2023-07-18T03:12:54.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.837+0000 7f255b3ff700 1 --2- 172.21.15.101:0/1502258105 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f25400a1300 0x7f25400a37b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:54.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.837+0000 7f255b3ff700 1 --2- 172.21.15.101:0/1502258105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2554105b00 0x7f2554195aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:54.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.837+0000 7f255b3ff700 1 --2- 172.21.15.101:0/1502258105 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f25540ffaf0 0x7f255419ba30 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:54.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.837+0000 7f255b3ff700 1 --2- 172.21.15.101:0/1502258105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25541004a0 0x7f255419bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:54.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.837+0000 7f255b3ff700 1 -- 172.21.15.101:0/1502258105 >> 172.21.15.101:0/1502258105 conn(0x7f25540fb630 msgr2=0x7f2554103600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:54.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.837+0000 7f255b3ff700 1 -- 172.21.15.101:0/1502258105 shutdown_connections 2023-07-18T03:12:54.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:54.837+0000 7f255b3ff700 1 -- 172.21.15.101:0/1502258105 wait complete. 2023-07-18T03:12:54.832 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 26 2023-07-18T03:12:54.850 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-18T03:12:55.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.069+0000 7f39f2dff700 1 -- 172.21.15.101:0/2800145305 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f39ec0ffae0 msgr2=0x7f39ec0fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:55.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.069+0000 7f39f2dff700 1 --2- 172.21.15.101:0/2800145305 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f39ec0ffae0 0x7f39ec0fff50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f39d80098f0 tx=0x7f39d8009c00 comp rx=0 tx=0).stop 2023-07-18T03:12:55.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.069+0000 7f39f2dff700 1 -- 172.21.15.101:0/2800145305 shutdown_connections 2023-07-18T03:12:55.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.069+0000 7f39f2dff700 1 --2- 172.21.15.101:0/2800145305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39ec100490 0x7f39ec10c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.069+0000 7f39f2dff700 1 --2- 172.21.15.101:0/2800145305 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39ec105af0 0x7f39ec105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.069+0000 7f39f2dff700 1 --2- 172.21.15.101:0/2800145305 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f39ec0ffae0 0x7f39ec0fff50 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.069+0000 7f39f2dff700 1 -- 172.21.15.101:0/2800145305 >> 172.21.15.101:0/2800145305 conn(0x7f39ec0fb5e0 msgr2=0x7f39ec0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:55.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.069+0000 7f39f2dff700 1 -- 172.21.15.101:0/2800145305 shutdown_connections 2023-07-18T03:12:55.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.070+0000 7f39f2dff700 1 -- 172.21.15.101:0/2800145305 wait complete. 2023-07-18T03:12:55.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.070+0000 7f39f2dff700 1 Processor -- start 2023-07-18T03:12:55.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.071+0000 7f39f2dff700 1 -- start start 2023-07-18T03:12:55.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.071+0000 7f39f2dff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39ec0ffae0 0x7f39ec197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.071+0000 7f39f2dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f39ec100490 0x7f39ec1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.071+0000 7f39f2dff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39ec105af0 0x7f39ec19c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.071+0000 7f39f2dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ec19ce50 con 0x7f39ec100490 2023-07-18T03:12:55.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.071+0000 7f39f2dff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ec19cfc0 con 0x7f39ec105af0 2023-07-18T03:12:55.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.071+0000 7f39f2dff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ec19d180 con 0x7f39ec0ffae0 2023-07-18T03:12:55.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.071+0000 7f39ebfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f39ec100490 0x7f39ec1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.071+0000 7f39f139c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39ec105af0 0x7f39ec19c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.071+0000 7f39f139c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39ec105af0 0x7f39ec19c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:50598/0 (socket says 172.21.15.101:50598) 2023-07-18T03:12:55.070 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.071+0000 7f39ebfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f39ec100490 0x7f39ec1983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:35664/0 (socket says 172.21.15.101:35664) 2023-07-18T03:12:55.070 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.071+0000 7f39f139c700 1 -- 172.21.15.101:0/141252975 learned_addr learned my addr 172.21.15.101:0/141252975 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:55.070 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.071+0000 7f39f0b9b700 1 --2- 172.21.15.101:0/141252975 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39ec0ffae0 0x7f39ec197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.070 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.072+0000 7f39f139c700 1 -- 172.21.15.101:0/141252975 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39ec0ffae0 msgr2=0x7f39ec197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:55.070 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.072+0000 7f39f139c700 1 --2- 172.21.15.101:0/141252975 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39ec0ffae0 0x7f39ec197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.070 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.072+0000 7f39f139c700 1 -- 172.21.15.101:0/141252975 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f39ec100490 msgr2=0x7f39ec1983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:55.070 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.072+0000 7f39f139c700 1 --2- 172.21.15.101:0/141252975 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f39ec100490 0x7f39ec1983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.072+0000 7f39f139c700 1 -- 172.21.15.101:0/141252975 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e4009d00 con 0x7f39ec105af0 2023-07-18T03:12:55.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.072+0000 7f39ebfff700 1 --2- 172.21.15.101:0/141252975 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f39ec100490 0x7f39ec1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:12:55.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.072+0000 7f39f139c700 1 --2- 172.21.15.101:0/141252975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39ec105af0 0x7f39ec19c810 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f39e400e3c0 tx=0x7f39e400e6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:55.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.072+0000 7f39e9ffb700 1 -- 172.21.15.101:0/141252975 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e40090d0 con 0x7f39ec105af0 2023-07-18T03:12:55.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.072+0000 7f39e9ffb700 1 -- 172.21.15.101:0/141252975 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39e400f040 con 0x7f39ec105af0 2023-07-18T03:12:55.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.072+0000 7f39f2dff700 1 -- 172.21.15.101:0/141252975 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39e0009520 con 0x7f39ec105af0 2023-07-18T03:12:55.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.072+0000 7f39e9ffb700 1 -- 172.21.15.101:0/141252975 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e4007c20 con 0x7f39ec105af0 2023-07-18T03:12:55.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.072+0000 7f39f2dff700 1 -- 172.21.15.101:0/141252975 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39d8009520 con 0x7f39ec105af0 2023-07-18T03:12:55.075 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.076+0000 7f39e9ffb700 1 -- 172.21.15.101:0/141252975 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f39e4003a00 con 0x7f39ec105af0 2023-07-18T03:12:55.075 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.076+0000 7f3a06709b80 1 Processor -- start 2023-07-18T03:12:55.076 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.077+0000 7f3a06709b80 1 -- start start 2023-07-18T03:12:55.076 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.077+0000 7f3a06709b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b05578c2d0 0x55b05578c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.076 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.077+0000 7f3a06709b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b05578cbe0 0x55b055797590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.076 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.077+0000 7f3a06709b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b055797ad0 0x55b055799eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.076 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.077+0000 7f3a06709b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b055692cb0 con 0x55b055797ad0 2023-07-18T03:12:55.076 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.077+0000 7f3a06709b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0556e5a40 con 0x55b05578c2d0 2023-07-18T03:12:55.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.077+0000 7f3a06709b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0556deed0 con 0x55b05578cbe0 2023-07-18T03:12:55.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.077+0000 7f39ebfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b05578c2d0 0x55b05578c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.077+0000 7f39ebfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b05578c2d0 0x55b05578c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:50614/0 (socket says 172.21.15.101:50614) 2023-07-18T03:12:55.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.077+0000 7f39ebfff700 1 -- 172.21.15.101:0/147546549 learned_addr learned my addr 172.21.15.101:0/147546549 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:55.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.078+0000 7f39ebfff700 1 --2- 172.21.15.101:0/147546549 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b055797ad0 0x55b055799eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.078+0000 7f39f0b9b700 1 --2- 172.21.15.101:0/147546549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b05578cbe0 0x55b055797590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.078+0000 7f39f0b9b700 1 -- 172.21.15.101:0/147546549 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b05578c2d0 msgr2=0x55b05578c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:55.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.078+0000 7f39f0b9b700 1 --2- 172.21.15.101:0/147546549 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b05578c2d0 0x55b05578c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.078+0000 7f39f0b9b700 1 -- 172.21.15.101:0/147546549 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b055797ad0 msgr2=0x55b055799eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:55.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.078+0000 7f39f0b9b700 1 --2- 172.21.15.101:0/147546549 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b055797ad0 0x55b055799eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.078+0000 7f39f0b9b700 1 -- 172.21.15.101:0/147546549 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b05550cde0 con 0x55b05578cbe0 2023-07-18T03:12:55.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.078+0000 7f39e9ffb700 1 --2- 172.21.15.101:0/141252975 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f39dc0a1340 0x7f39dc0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.078+0000 7f39f0b9b700 1 --2- 172.21.15.101:0/147546549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b05578cbe0 0x55b055797590 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f39e0009e40 tx=0x7f39e0011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a8b72b24802450b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:55.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.078+0000 7f39e9ffb700 1 -- 172.21.15.101:0/141252975 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f39e40cce30 con 0x7f39ec105af0 2023-07-18T03:12:55.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.078+0000 7f39ebfff700 1 --2- 172.21.15.101:0/141252975 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f39dc0a1340 0x7f39dc0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.078+0000 7f39d6ffd700 1 -- 172.21.15.101:0/147546549 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e0019630 con 0x55b05578cbe0 2023-07-18T03:12:55.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.078+0000 7f39d6ffd700 1 -- 172.21.15.101:0/147546549 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39e0019c50 con 0x55b05578cbe0 2023-07-18T03:12:55.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.078+0000 7f39d6ffd700 1 -- 172.21.15.101:0/147546549 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e000ec20 con 0x55b05578cbe0 2023-07-18T03:12:55.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.079+0000 7f3a06709b80 1 -- 172.21.15.101:0/147546549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b05578cbe0 msgr2=0x55b055797590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:55.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.079+0000 7f39ebfff700 1 --2- 172.21.15.101:0/141252975 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f39dc0a1340 0x7f39dc0a37f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f39d80098c0 tx=0x7f39d8005dc0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:55.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.079+0000 7f3a06709b80 1 --2- 172.21.15.101:0/147546549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b05578cbe0 0x55b055797590 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f39e0009e40 tx=0x7f39e0011720 comp rx=0 tx=0).stop 2023-07-18T03:12:55.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.079+0000 7f3a06709b80 1 -- 172.21.15.101:0/147546549 shutdown_connections 2023-07-18T03:12:55.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.079+0000 7f3a06709b80 1 --2- 172.21.15.101:0/147546549 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b05578c2d0 0x55b05578c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.079+0000 7f3a06709b80 1 --2- 172.21.15.101:0/147546549 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b055797ad0 0x55b055799eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.079+0000 7f3a06709b80 1 --2- 172.21.15.101:0/147546549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b05578cbe0 0x55b055797590 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.079+0000 7f3a06709b80 1 -- 172.21.15.101:0/147546549 >> 172.21.15.101:0/147546549 conn(0x55b055782da0 msgr2=0x55b0557886a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:55.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.080+0000 7f3a06709b80 1 -- 172.21.15.101:0/147546549 shutdown_connections 2023-07-18T03:12:55.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.080+0000 7f3a06709b80 1 -- 172.21.15.101:0/147546549 wait complete. 2023-07-18T03:12:55.081 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.080+0000 7f3a06709b80 1 -- start start 2023-07-18T03:12:55.081 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.080+0000 7f3a06709b80 1 Processor -- start 2023-07-18T03:12:55.081 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.081+0000 7f3a06709b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b05578c2d0 0x55b055797550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.081 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.081+0000 7f3a06709b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b05578cbe0 0x55b055794760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.081 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.081+0000 7f3a06709b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b055797ad0 0x55b055794e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.081 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.081+0000 7f3a06709b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b05567eac0 con 0x55b05578c2d0 2023-07-18T03:12:55.081 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.081+0000 7f3a06709b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b05567e7c0 con 0x55b055797ad0 2023-07-18T03:12:55.081 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.081+0000 7f3a06709b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b055562920 con 0x55b05578cbe0 2023-07-18T03:12:55.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.081+0000 7f3a06709b80 10 client.0 _ll_register_callbacks cb 0x55b05567c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:12:55.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.081+0000 7f39f139c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b055797ad0 0x55b055794e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.081+0000 7f39f139c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b055797ad0 0x55b055794e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:50618/0 (socket says 172.21.15.101:50618) 2023-07-18T03:12:55.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.081+0000 7f39f139c700 1 -- 172.21.15.101:0/1971140127 learned_addr learned my addr 172.21.15.101:0/1971140127 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:55.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.081+0000 7f39f0b9b700 1 --2- 172.21.15.101:0/1971140127 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b05578c2d0 0x55b055797550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.081+0000 7f39ebfff700 1 --2- 172.21.15.101:0/1971140127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b05578cbe0 0x55b055794760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.082+0000 7f39f139c700 1 -- 172.21.15.101:0/1971140127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b05578cbe0 msgr2=0x55b055794760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:55.083 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.082+0000 7f39f139c700 1 --2- 172.21.15.101:0/1971140127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b05578cbe0 0x55b055794760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.083 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.082+0000 7f39f139c700 1 -- 172.21.15.101:0/1971140127 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b05578c2d0 msgr2=0x55b055797550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:55.083 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.082+0000 7f39f139c700 1 --2- 172.21.15.101:0/1971140127 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b05578c2d0 0x55b055797550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.083 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.082+0000 7f39f139c700 1 -- 172.21.15.101:0/1971140127 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39e0009a70 con 0x55b055797ad0 2023-07-18T03:12:55.083 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.082+0000 7f39f0b9b700 1 --2- 172.21.15.101:0/1971140127 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b05578c2d0 0x55b055797550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:12:55.083 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.082+0000 7f39f139c700 1 --2- 172.21.15.101:0/1971140127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b055797ad0 0x55b055794e50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f39e40095b0 tx=0x7f39e40995c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca76cd27d155e415 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:55.083 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.082+0000 7f39cbfff700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:12:55.084 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.082+0000 7f39cbfff700 1 -- 172.21.15.101:0/1971140127 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e40a1fa0 con 0x55b055797ad0 2023-07-18T03:12:55.084 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.082+0000 7f39cbfff700 1 -- 172.21.15.101:0/1971140127 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f39e40962f0 con 0x55b055797ad0 2023-07-18T03:12:55.084 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.082+0000 7f3a06709b80 1 -- 172.21.15.101:0/1971140127 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b0556384f0 con 0x55b055797ad0 2023-07-18T03:12:55.084 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.082+0000 7f39cbfff700 1 -- 172.21.15.101:0/1971140127 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39e40964b0 con 0x55b055797ad0 2023-07-18T03:12:55.084 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.082+0000 7f3a06709b80 10 client.26642 fetch_fsmap learned FSMap version 26 2023-07-18T03:12:55.084 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.082+0000 7f3a06709b80 1 -- 172.21.15.101:0/1971140127 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55b0554f5c60 con 0x55b055797ad0 2023-07-18T03:12:55.084 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.083+0000 7f39cbfff700 1 -- 172.21.15.101:0/1971140127 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b0556384f0 con 0x55b055797ad0 2023-07-18T03:12:55.084 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.083+0000 7f39cbfff700 1 -- 172.21.15.101:0/1971140127 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f39e40990d0 con 0x55b055797ad0 2023-07-18T03:12:55.085 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.083+0000 7f3a06709b80 10 client.26642 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:12:55.085 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.083+0000 7f3a06709b80 10 client.26642 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:12:55.085 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.083+0000 7f3a06709b80 20 client.26642 populate_metadata read hostname 'smithi101' 2023-07-18T03:12:55.085 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.083+0000 7f3a06709b80 1 --2- 172.21.15.101:0/1971140127 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55b0557a7370 0x55b0557a9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.085 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.083+0000 7f3a06709b80 4 client.26642 mds_command: new command op to 24541 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:12:55.085 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.083+0000 7f3a06709b80 1 -- 172.21.15.101:0/1971140127 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b0556d7eb0 con 0x55b0557a7370 2023-07-18T03:12:55.086 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.083+0000 7f39f0b9b700 1 --2- 172.21.15.101:0/1971140127 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55b0557a7370 0x55b0557a9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.086 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.084+0000 7f39f0b9b700 1 --2- 172.21.15.101:0/1971140127 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55b0557a7370 0x55b0557a9780 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=db3affd88ffd2f72 server_cookie=e41d29e35ce100bd in_seq=0 out_seq=0 2023-07-18T03:12:55.087 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.084+0000 7f39cbfff700 10 client.26642 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:12:55.087 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.084+0000 7f39cbfff700 1 -- 172.21.15.101:0/1971140127 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b0556d7eb0 con 0x55b0557a7370 2023-07-18T03:12:55.087 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.084+0000 7f39cbfff700 10 client.26642 handle_command_reply: tid=0 2023-07-18T03:12:55.087 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.085+0000 7f3a06709b80 1 client.26642 shutdown 2023-07-18T03:12:55.087 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.085+0000 7f3a06709b80 1 -- 172.21.15.101:0/1971140127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b055797ad0 msgr2=0x55b055794e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:55.087 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.085+0000 7f3a06709b80 1 --2- 172.21.15.101:0/1971140127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b055797ad0 0x55b055794e50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f39e40095b0 tx=0x7f39e40995c0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.087 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.085+0000 7f3a06709b80 1 -- 172.21.15.101:0/1971140127 shutdown_connections 2023-07-18T03:12:55.088 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.085+0000 7f3a06709b80 1 --2- 172.21.15.101:0/1971140127 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55b0557a7370 0x55b0557a9780 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.088 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.085+0000 7f3a06709b80 1 --2- 172.21.15.101:0/1971140127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b055797ad0 0x55b055794e50 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.088 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.085+0000 7f3a06709b80 1 --2- 172.21.15.101:0/1971140127 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b05578c2d0 0x55b055797550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.088 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.085+0000 7f39cbfff700 0 client.26642 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:12:55.088 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.085+0000 7f3a06709b80 1 --2- 172.21.15.101:0/1971140127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b05578cbe0 0x55b055794760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.088 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.085+0000 7f3a06709b80 1 -- 172.21.15.101:0/1971140127 >> 172.21.15.101:0/1971140127 conn(0x55b055782da0 msgr2=0x55b055788290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:55.088 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.085+0000 7f3a06709b80 1 -- 172.21.15.101:0/1971140127 shutdown_connections 2023-07-18T03:12:55.089 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.085+0000 7f3a06709b80 1 -- 172.21.15.101:0/1971140127 wait complete. 2023-07-18T03:12:55.089 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.085+0000 7f3a06709b80 20 client.26642 trim_cache size 0 max 16384 2023-07-18T03:12:55.091 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.099+0000 7f3a06709b80 1 Processor -- start 2023-07-18T03:12:55.091 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.099+0000 7f3a06709b80 1 -- start start 2023-07-18T03:12:55.092 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.099+0000 7f3a06709b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b05578c2d0 0x55b055797080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.092 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.099+0000 7f3a06709b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b05578cbe0 0x55b05578fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.092 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.099+0000 7f3a06709b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b055797ad0 0x55b05578ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.092 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.099+0000 7f3a06709b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b055789d20 con 0x55b05578cbe0 2023-07-18T03:12:55.092 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.099+0000 7f3a06709b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b055795450 con 0x55b05578c2d0 2023-07-18T03:12:55.093 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.099+0000 7f3a06709b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b05578d540 con 0x55b055797ad0 2023-07-18T03:12:55.093 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.100+0000 7f39f139c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b055797ad0 0x55b05578ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.094 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.100+0000 7f39f139c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b055797ad0 0x55b05578ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:50986/0 (socket says 172.21.15.101:50986) 2023-07-18T03:12:55.094 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.100+0000 7f39f139c700 1 -- 172.21.15.101:0/471934846 learned_addr learned my addr 172.21.15.101:0/471934846 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:55.094 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.100+0000 7f39ebfff700 1 --2- 172.21.15.101:0/471934846 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b05578cbe0 0x55b05578fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.094 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.100+0000 7f39f0b9b700 1 --2- 172.21.15.101:0/471934846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b05578c2d0 0x55b055797080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.094 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.100+0000 7f39f139c700 1 -- 172.21.15.101:0/471934846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b05578c2d0 msgr2=0x55b055797080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:55.094 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.100+0000 7f39f139c700 1 --2- 172.21.15.101:0/471934846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b05578c2d0 0x55b055797080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.100+0000 7f39f139c700 1 -- 172.21.15.101:0/471934846 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b05578cbe0 msgr2=0x55b05578fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:55.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.100+0000 7f39f139c700 1 --2- 172.21.15.101:0/471934846 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b05578cbe0 0x55b05578fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.100+0000 7f39f139c700 1 -- 172.21.15.101:0/471934846 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b055533250 con 0x55b055797ad0 2023-07-18T03:12:55.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.100+0000 7f39f139c700 1 --2- 172.21.15.101:0/471934846 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b055797ad0 0x55b05578ffd0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f39e4099da0 tx=0x7f39e4007f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30e80444dd414069 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:55.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.101+0000 7f39d77fe700 1 -- 172.21.15.101:0/471934846 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e40074e0 con 0x55b055797ad0 2023-07-18T03:12:55.096 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.101+0000 7f39d77fe700 1 -- 172.21.15.101:0/471934846 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39e40a1fa0 con 0x55b055797ad0 2023-07-18T03:12:55.096 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.101+0000 7f39d77fe700 1 -- 172.21.15.101:0/471934846 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e40962f0 con 0x55b055797ad0 2023-07-18T03:12:55.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.101+0000 7f3a06709b80 1 -- 172.21.15.101:0/471934846 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b055797ad0 msgr2=0x55b05578ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:55.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.101+0000 7f3a06709b80 1 --2- 172.21.15.101:0/471934846 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b055797ad0 0x55b05578ffd0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f39e4099da0 tx=0x7f39e4007f90 comp rx=0 tx=0).stop 2023-07-18T03:12:55.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.102+0000 7f3a06709b80 1 -- 172.21.15.101:0/471934846 shutdown_connections 2023-07-18T03:12:55.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.102+0000 7f3a06709b80 1 --2- 172.21.15.101:0/471934846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b05578c2d0 0x55b055797080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.102+0000 7f3a06709b80 1 --2- 172.21.15.101:0/471934846 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b055797ad0 0x55b05578ffd0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.102+0000 7f3a06709b80 1 --2- 172.21.15.101:0/471934846 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b05578cbe0 0x55b05578fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.102+0000 7f3a06709b80 1 -- 172.21.15.101:0/471934846 >> 172.21.15.101:0/471934846 conn(0x55b055782da0 msgr2=0x55b0557ae0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:55.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.102+0000 7f3a06709b80 1 -- 172.21.15.101:0/471934846 shutdown_connections 2023-07-18T03:12:55.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.102+0000 7f3a06709b80 1 -- 172.21.15.101:0/471934846 wait complete. 2023-07-18T03:12:55.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.102+0000 7f3a06709b80 1 -- start start 2023-07-18T03:12:55.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.102+0000 7f3a06709b80 1 Processor -- start 2023-07-18T03:12:55.100 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.103+0000 7f3a06709b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b05578c2d0 0x55b0557f6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.100 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.103+0000 7f3a06709b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b05578cbe0 0x55b0557f6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.100 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.103+0000 7f3a06709b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b055797ad0 0x55b0557f6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.100 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.103+0000 7f3a06709b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b05579c670 con 0x55b05578c2d0 2023-07-18T03:12:55.100 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.103+0000 7f3a06709b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b05579c7e0 con 0x55b05578cbe0 2023-07-18T03:12:55.100 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.103+0000 7f3a06709b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b05579c9a0 con 0x55b055797ad0 2023-07-18T03:12:55.100 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.103+0000 7f3a06709b80 10 client.0 _ll_register_callbacks cb 0x7f39e40999d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:12:55.101 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.103+0000 7f39f0b9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b05578c2d0 0x55b0557f6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.101 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.103+0000 7f39f139c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b055797ad0 0x55b0557f6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.101 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.103+0000 7f39ebfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b05578cbe0 0x55b0557f6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.101 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.103+0000 7f39f139c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b055797ad0 0x55b0557f6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:50990/0 (socket says 172.21.15.101:50990) 2023-07-18T03:12:55.101 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.103+0000 7f39f139c700 1 -- 172.21.15.101:0/323086680 learned_addr learned my addr 172.21.15.101:0/323086680 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:55.101 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.103+0000 7f39f0b9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b05578c2d0 0x55b0557f6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:35708/0 (socket says 172.21.15.101:35708) 2023-07-18T03:12:55.101 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.103+0000 7f39ebfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b05578cbe0 0x55b0557f6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:50634/0 (socket says 172.21.15.101:50634) 2023-07-18T03:12:55.101 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.104+0000 7f39f139c700 1 -- 172.21.15.101:0/323086680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b05578cbe0 msgr2=0x55b0557f6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:55.102 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.104+0000 7f39f139c700 1 --2- 172.21.15.101:0/323086680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b05578cbe0 0x55b0557f6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.102 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.104+0000 7f39f139c700 1 -- 172.21.15.101:0/323086680 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b05578c2d0 msgr2=0x55b0557f6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:55.102 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.104+0000 7f39f139c700 1 --2- 172.21.15.101:0/323086680 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b05578c2d0 0x55b0557f6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.102 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.104+0000 7f39f139c700 1 -- 172.21.15.101:0/323086680 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b0554f5c60 con 0x55b055797ad0 2023-07-18T03:12:55.102 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.104+0000 7f39f139c700 1 --2- 172.21.15.101:0/323086680 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b055797ad0 0x55b0557f6c90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f39e4005c30 tx=0x7f39e409a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e62c193519a9268e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:55.102 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.104+0000 7f39cbfff700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:12:55.102 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.104+0000 7f39cbfff700 1 -- 172.21.15.101:0/323086680 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e40074e0 con 0x55b055797ad0 2023-07-18T03:12:55.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.104+0000 7f39cbfff700 1 -- 172.21.15.101:0/323086680 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f39e4099e40 con 0x55b055797ad0 2023-07-18T03:12:55.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.104+0000 7f3a06709b80 1 -- 172.21.15.101:0/323086680 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f39e0019c50 con 0x55b055797ad0 2023-07-18T03:12:55.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.104+0000 7f39cbfff700 1 -- 172.21.15.101:0/323086680 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39e409a000 con 0x55b055797ad0 2023-07-18T03:12:55.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.104+0000 7f3a06709b80 10 client.26671 fetch_fsmap learned FSMap version 26 2023-07-18T03:12:55.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.104+0000 7f3a06709b80 1 -- 172.21.15.101:0/323086680 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55b05550b2b0 con 0x55b055797ad0 2023-07-18T03:12:55.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.105+0000 7f39cbfff700 1 -- 172.21.15.101:0/323086680 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39e0019c50 con 0x55b055797ad0 2023-07-18T03:12:55.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.105+0000 7f39cbfff700 1 -- 172.21.15.101:0/323086680 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f39e4098b40 con 0x55b055797ad0 2023-07-18T03:12:55.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.105+0000 7f3a06709b80 10 client.26671 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:12:55.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.105+0000 7f3a06709b80 10 client.26671 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:12:55.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.105+0000 7f3a06709b80 20 client.26671 populate_metadata read hostname 'smithi101' 2023-07-18T03:12:55.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.105+0000 7f3a06709b80 1 --2- 172.21.15.101:0/323086680 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55b055804130 0x55b055806510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.105+0000 7f3a06709b80 4 client.26671 mds_command: new command op to 24541 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:12:55.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.105+0000 7f3a06709b80 1 -- 172.21.15.101:0/323086680 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b05569a050 con 0x55b055804130 2023-07-18T03:12:55.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.105+0000 7f39f0b9b700 1 --2- 172.21.15.101:0/323086680 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55b055804130 0x55b055806510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.106+0000 7f39cbfff700 10 client.26671 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:12:55.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.106+0000 7f39f0b9b700 1 --2- 172.21.15.101:0/323086680 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55b055804130 0x55b055806510 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2d2c297ea8006bb3 server_cookie=671713f69be831e in_seq=0 out_seq=0 2023-07-18T03:12:55.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.106+0000 7f39cbfff700 1 -- 172.21.15.101:0/323086680 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b05569a050 con 0x55b055804130 2023-07-18T03:12:55.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.106+0000 7f39cbfff700 10 client.26671 handle_command_reply: tid=0 2023-07-18T03:12:55.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.106+0000 7f3a06709b80 1 client.26671 shutdown 2023-07-18T03:12:55.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.106+0000 7f3a06709b80 1 -- 172.21.15.101:0/323086680 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b055797ad0 msgr2=0x55b0557f6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:55.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.106+0000 7f3a06709b80 1 --2- 172.21.15.101:0/323086680 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b055797ad0 0x55b0557f6c90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f39e4005c30 tx=0x7f39e409a1e0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.107+0000 7f3a06709b80 1 -- 172.21.15.101:0/323086680 shutdown_connections 2023-07-18T03:12:55.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.107+0000 7f3a06709b80 1 --2- 172.21.15.101:0/323086680 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55b055804130 0x55b055806510 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.107+0000 7f3a06709b80 1 --2- 172.21.15.101:0/323086680 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b055797ad0 0x55b0557f6c90 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.107+0000 7f3a06709b80 1 --2- 172.21.15.101:0/323086680 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b05578c2d0 0x55b0557f6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.107+0000 7f3a06709b80 1 --2- 172.21.15.101:0/323086680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b05578cbe0 0x55b0557f6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.107+0000 7f39cbfff700 0 client.26671 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:12:55.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.107+0000 7f3a06709b80 1 -- 172.21.15.101:0/323086680 >> 172.21.15.101:0/323086680 conn(0x55b055782da0 msgr2=0x55b05578fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:55.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.107+0000 7f3a06709b80 1 -- 172.21.15.101:0/323086680 shutdown_connections 2023-07-18T03:12:55.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.107+0000 7f3a06709b80 1 -- 172.21.15.101:0/323086680 wait complete. 2023-07-18T03:12:55.108 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.107+0000 7f3a06709b80 20 client.26671 trim_cache size 0 max 16384 2023-07-18T03:12:55.108 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:12:55.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.107+0000 7f39f2dff700 1 -- 172.21.15.101:0/141252975 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f39dc0a1340 msgr2=0x7f39dc0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:55.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.107+0000 7f39f2dff700 1 --2- 172.21.15.101:0/141252975 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f39dc0a1340 0x7f39dc0a37f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f39d80098c0 tx=0x7f39d8005dc0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.107+0000 7f39f2dff700 1 -- 172.21.15.101:0/141252975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39ec105af0 msgr2=0x7f39ec19c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:55.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.107+0000 7f39f2dff700 1 --2- 172.21.15.101:0/141252975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39ec105af0 0x7f39ec19c810 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f39e400e3c0 tx=0x7f39e400e6d0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.108+0000 7f39f2dff700 1 -- 172.21.15.101:0/141252975 shutdown_connections 2023-07-18T03:12:55.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.108+0000 7f39f2dff700 1 --2- 172.21.15.101:0/141252975 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f39dc0a1340 0x7f39dc0a37f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.108+0000 7f39f2dff700 1 --2- 172.21.15.101:0/141252975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39ec105af0 0x7f39ec19c810 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.108+0000 7f39f2dff700 1 --2- 172.21.15.101:0/141252975 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39ec0ffae0 0x7f39ec197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.108+0000 7f39f2dff700 1 --2- 172.21.15.101:0/141252975 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f39ec100490 0x7f39ec1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.108+0000 7f39f2dff700 1 -- 172.21.15.101:0/141252975 >> 172.21.15.101:0/141252975 conn(0x7f39ec0fb5e0 msgr2=0x7f39ec10ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:55.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.108+0000 7f39f2dff700 1 -- 172.21.15.101:0/141252975 shutdown_connections 2023-07-18T03:12:55.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.108+0000 7f39f2dff700 1 -- 172.21.15.101:0/141252975 wait complete. 2023-07-18T03:12:55.115 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:54 smithi101 ceph-mon[126379]: pgmap v163: 129 pgs: 129 active+clean; 248 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 194 op/s 2023-07-18T03:12:55.116 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:54 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1502258105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:12:55.118 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-18T03:12:55.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:54 smithi155 ceph-mon[128885]: pgmap v163: 129 pgs: 129 active+clean; 248 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 194 op/s 2023-07-18T03:12:55.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:54 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1502258105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:12:55.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.337+0000 7f47cfbff700 1 -- 172.21.15.101:0/2074103796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47c80ff190 msgr2=0x7f47c810bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:55.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.337+0000 7f47cfbff700 1 --2- 172.21.15.101:0/2074103796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47c80ff190 0x7f47c810bef0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f47b8009990 tx=0x7f47b8009ca0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.337+0000 7f47cfbff700 1 -- 172.21.15.101:0/2074103796 shutdown_connections 2023-07-18T03:12:55.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.337+0000 7f47cfbff700 1 --2- 172.21.15.101:0/2074103796 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47c810c430 0x7f47c810e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.337+0000 7f47cfbff700 1 --2- 172.21.15.101:0/2074103796 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f47c80fe880 0x7f47c80fec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.337+0000 7f47cfbff700 1 --2- 172.21.15.101:0/2074103796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47c80ff190 0x7f47c810bef0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.337+0000 7f47cfbff700 1 -- 172.21.15.101:0/2074103796 >> 172.21.15.101:0/2074103796 conn(0x7f47c80fa4b0 msgr2=0x7f47c80fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:55.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.337+0000 7f47cfbff700 1 -- 172.21.15.101:0/2074103796 shutdown_connections 2023-07-18T03:12:55.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.337+0000 7f47cfbff700 1 -- 172.21.15.101:0/2074103796 wait complete. 2023-07-18T03:12:55.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.338+0000 7f47cfbff700 1 Processor -- start 2023-07-18T03:12:55.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.338+0000 7f47cfbff700 1 -- start start 2023-07-18T03:12:55.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.339+0000 7f47cfbff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f47c80fe880 0x7f47c8197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.339+0000 7f47cfbff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47c80ff190 0x7f47c81983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.339+0000 7f47cfbff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47c810c430 0x7f47c819c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.339+0000 7f47cfbff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c819ce50 con 0x7f47c80fe880 2023-07-18T03:12:55.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.339+0000 7f47cfbff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c819cfc0 con 0x7f47c810c430 2023-07-18T03:12:55.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.339+0000 7f47cfbff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c819d180 con 0x7f47c80ff190 2023-07-18T03:12:55.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.339+0000 7f47cd99b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f47c80fe880 0x7f47c8197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.339+0000 7f47cd19a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47c80ff190 0x7f47c81983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.339+0000 7f47cd99b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f47c80fe880 0x7f47c8197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:35728/0 (socket says 172.21.15.101:35728) 2023-07-18T03:12:55.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.339+0000 7f47cd99b700 1 -- 172.21.15.101:0/3813298536 learned_addr learned my addr 172.21.15.101:0/3813298536 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:55.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.339+0000 7f47ce19c700 1 --2- 172.21.15.101:0/3813298536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47c810c430 0x7f47c819c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.339+0000 7f47cd99b700 1 -- 172.21.15.101:0/3813298536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47c80ff190 msgr2=0x7f47c81983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:55.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.339+0000 7f47cd99b700 1 --2- 172.21.15.101:0/3813298536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47c80ff190 0x7f47c81983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.339+0000 7f47cd99b700 1 -- 172.21.15.101:0/3813298536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47c810c430 msgr2=0x7f47c819c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:55.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.339+0000 7f47cd99b700 1 --2- 172.21.15.101:0/3813298536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47c810c430 0x7f47c819c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.339+0000 7f47cd99b700 1 -- 172.21.15.101:0/3813298536 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b80095f0 con 0x7f47c80fe880 2023-07-18T03:12:55.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.339+0000 7f47ce19c700 1 --2- 172.21.15.101:0/3813298536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47c810c430 0x7f47c819c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:12:55.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.340+0000 7f47cd99b700 1 --2- 172.21.15.101:0/3813298536 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f47c80fe880 0x7f47c8197e70 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f47c400e9a0 tx=0x7f47c400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:55.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.340+0000 7f47beffd700 1 -- 172.21.15.101:0/3813298536 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c4009790 con 0x7f47c80fe880 2023-07-18T03:12:55.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.340+0000 7f47cfbff700 1 -- 172.21.15.101:0/3813298536 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c819d460 con 0x7f47c80fe880 2023-07-18T03:12:55.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.340+0000 7f47cfbff700 1 -- 172.21.15.101:0/3813298536 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c81a4ca0 con 0x7f47c80fe880 2023-07-18T03:12:55.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.340+0000 7f47beffd700 1 -- 172.21.15.101:0/3813298536 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47c4010430 con 0x7f47c80fe880 2023-07-18T03:12:55.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.340+0000 7f47beffd700 1 -- 172.21.15.101:0/3813298536 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c4018680 con 0x7f47c80fe880 2023-07-18T03:12:55.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.343+0000 7f47beffd700 1 -- 172.21.15.101:0/3813298536 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f47c4010830 con 0x7f47c80fe880 2023-07-18T03:12:55.338 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.345+0000 7f47beffd700 1 --2- 172.21.15.101:0/3813298536 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f47b40a12c0 0x7f47b40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.345+0000 7f47beffd700 1 -- 172.21.15.101:0/3813298536 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f47c4014070 con 0x7f47c80fe880 2023-07-18T03:12:55.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.345+0000 7f47e356bb80 1 Processor -- start 2023-07-18T03:12:55.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.345+0000 7f47cd19a700 1 --2- 172.21.15.101:0/3813298536 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f47b40a12c0 0x7f47b40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.345+0000 7f47e356bb80 1 -- start start 2023-07-18T03:12:55.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.346+0000 7f47e356bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56547f57d2d0 0x56547f57d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.346+0000 7f47e356bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56547f57dbe0 0x56547f588590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.346+0000 7f47e356bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56547f588ad0 0x56547f58aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.346+0000 7f47e356bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56547f483cb0 con 0x56547f57dbe0 2023-07-18T03:12:55.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.346+0000 7f47e356bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56547f4d6a40 con 0x56547f57d2d0 2023-07-18T03:12:55.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.346+0000 7f47e356bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56547f4cfed0 con 0x56547f588ad0 2023-07-18T03:12:55.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.346+0000 7f47cd99b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56547f57d2d0 0x56547f57d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.346+0000 7f47cd99b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56547f57d2d0 0x56547f57d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:50642/0 (socket says 172.21.15.101:50642) 2023-07-18T03:12:55.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.346+0000 7f47cd99b700 1 -- 172.21.15.101:0/3563768420 learned_addr learned my addr 172.21.15.101:0/3563768420 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:55.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.346+0000 7f47cd19a700 1 --2- 172.21.15.101:0/3563768420 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56547f57dbe0 0x56547f588590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.346+0000 7f47ce19c700 1 --2- 172.21.15.101:0/3563768420 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56547f588ad0 0x56547f58aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.346+0000 7f47cd99b700 1 -- 172.21.15.101:0/3563768420 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56547f588ad0 msgr2=0x56547f58aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:55.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.346+0000 7f47cd19a700 1 --2- 172.21.15.101:0/3813298536 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f47b40a12c0 0x7f47b40a3770 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f47b800b6f0 tx=0x7f47b800b280 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:55.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.346+0000 7f47cd99b700 1 --2- 172.21.15.101:0/3563768420 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56547f588ad0 0x56547f58aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.346+0000 7f47cd99b700 1 -- 172.21.15.101:0/3563768420 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56547f57dbe0 msgr2=0x56547f588590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:55.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.346+0000 7f47cd99b700 1 --2- 172.21.15.101:0/3563768420 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56547f57dbe0 0x56547f588590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.346+0000 7f47cd99b700 1 -- 172.21.15.101:0/3563768420 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56547f2fdde0 con 0x56547f57d2d0 2023-07-18T03:12:55.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.347+0000 7f47cd99b700 1 --2- 172.21.15.101:0/3563768420 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56547f57d2d0 0x56547f57d6a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f47c40ca620 tx=0x7f47c4099970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b7709c93a6c031b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:55.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.347+0000 7f47b3fff700 1 -- 172.21.15.101:0/3563768420 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c40a2390 con 0x56547f57d2d0 2023-07-18T03:12:55.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.347+0000 7f47b3fff700 1 -- 172.21.15.101:0/3563768420 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47c40982e0 con 0x56547f57d2d0 2023-07-18T03:12:55.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.347+0000 7f47b3fff700 1 -- 172.21.15.101:0/3563768420 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c40a1f00 con 0x56547f57d2d0 2023-07-18T03:12:55.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.348+0000 7f47e356bb80 1 -- 172.21.15.101:0/3563768420 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56547f57d2d0 msgr2=0x56547f57d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:55.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.348+0000 7f47e356bb80 1 --2- 172.21.15.101:0/3563768420 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56547f57d2d0 0x56547f57d6a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f47c40ca620 tx=0x7f47c4099970 comp rx=0 tx=0).stop 2023-07-18T03:12:55.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.348+0000 7f47e356bb80 1 -- 172.21.15.101:0/3563768420 shutdown_connections 2023-07-18T03:12:55.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.348+0000 7f47e356bb80 1 --2- 172.21.15.101:0/3563768420 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56547f57d2d0 0x56547f57d6a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.348+0000 7f47e356bb80 1 --2- 172.21.15.101:0/3563768420 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56547f588ad0 0x56547f58aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.348+0000 7f47e356bb80 1 --2- 172.21.15.101:0/3563768420 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56547f57dbe0 0x56547f588590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.348+0000 7f47e356bb80 1 -- 172.21.15.101:0/3563768420 >> 172.21.15.101:0/3563768420 conn(0x56547f573da0 msgr2=0x56547f5796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:55.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.348+0000 7f47e356bb80 1 -- 172.21.15.101:0/3563768420 shutdown_connections 2023-07-18T03:12:55.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.348+0000 7f47e356bb80 1 -- 172.21.15.101:0/3563768420 wait complete. 2023-07-18T03:12:55.348 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.348+0000 7f47e356bb80 1 -- start start 2023-07-18T03:12:55.348 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.348+0000 7f47e356bb80 1 Processor -- start 2023-07-18T03:12:55.348 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.349+0000 7f47e356bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56547f57d2d0 0x56547f588560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.348 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.349+0000 7f47e356bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56547f57dbe0 0x56547f5857e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.348 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.349+0000 7f47e356bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56547f588ad0 0x56547f585f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.348 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.349+0000 7f47e356bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56547f46fac0 con 0x56547f57dbe0 2023-07-18T03:12:55.348 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.349+0000 7f47e356bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56547f46f7c0 con 0x56547f57d2d0 2023-07-18T03:12:55.349 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.349+0000 7f47e356bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56547f353920 con 0x56547f588ad0 2023-07-18T03:12:55.349 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.349+0000 7f47e356bb80 10 client.0 _ll_register_callbacks cb 0x56547f46d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:12:55.349 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.349+0000 7f47cd99b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56547f57d2d0 0x56547f588560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.349 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.349+0000 7f47cd99b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56547f57d2d0 0x56547f588560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:50654/0 (socket says 172.21.15.101:50654) 2023-07-18T03:12:55.349 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.349+0000 7f47cd99b700 1 -- 172.21.15.101:0/2232720674 learned_addr learned my addr 172.21.15.101:0/2232720674 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:55.349 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.349+0000 7f47cd19a700 1 --2- 172.21.15.101:0/2232720674 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56547f57dbe0 0x56547f5857e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.349 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.349+0000 7f47ce19c700 1 --2- 172.21.15.101:0/2232720674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56547f588ad0 0x56547f585f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.349+0000 7f47cd99b700 1 -- 172.21.15.101:0/2232720674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56547f588ad0 msgr2=0x56547f585f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:55.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.349+0000 7f47cd99b700 1 --2- 172.21.15.101:0/2232720674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56547f588ad0 0x56547f585f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.349+0000 7f47cd99b700 1 -- 172.21.15.101:0/2232720674 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56547f57dbe0 msgr2=0x56547f5857e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:55.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.349+0000 7f47cd99b700 1 --2- 172.21.15.101:0/2232720674 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56547f57dbe0 0x56547f5857e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.349+0000 7f47cd99b700 1 -- 172.21.15.101:0/2232720674 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c81a4ca0 con 0x56547f57d2d0 2023-07-18T03:12:55.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.350+0000 7f47cd99b700 1 --2- 172.21.15.101:0/2232720674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56547f57d2d0 0x56547f588560 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f47c4099ed0 tx=0x7f47c40ca560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=929e99490a4b0c0a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:55.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.350+0000 7f47b17fa700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:12:55.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.350+0000 7f47b17fa700 1 -- 172.21.15.101:0/2232720674 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c40a2100 con 0x56547f57d2d0 2023-07-18T03:12:55.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.350+0000 7f47e356bb80 1 -- 172.21.15.101:0/2232720674 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56547f4294f0 con 0x56547f57d2d0 2023-07-18T03:12:55.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.350+0000 7f47b17fa700 1 -- 172.21.15.101:0/2232720674 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f47c4094b50 con 0x56547f57d2d0 2023-07-18T03:12:55.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.350+0000 7f47b17fa700 1 -- 172.21.15.101:0/2232720674 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47c409a3b0 con 0x56547f57d2d0 2023-07-18T03:12:55.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.350+0000 7f47e356bb80 10 client.26660 fetch_fsmap learned FSMap version 26 2023-07-18T03:12:55.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.350+0000 7f47e356bb80 1 -- 172.21.15.101:0/2232720674 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56547f2e6c60 con 0x56547f57d2d0 2023-07-18T03:12:55.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.350+0000 7f47b17fa700 1 -- 172.21.15.101:0/2232720674 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56547f4294f0 con 0x56547f57d2d0 2023-07-18T03:12:55.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.350+0000 7f47b17fa700 1 -- 172.21.15.101:0/2232720674 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f47c400ca10 con 0x56547f57d2d0 2023-07-18T03:12:55.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.351+0000 7f47e356bb80 10 client.26660 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:12:55.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.351+0000 7f47e356bb80 10 client.26660 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:12:55.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.351+0000 7f47e356bb80 20 client.26660 populate_metadata read hostname 'smithi101' 2023-07-18T03:12:55.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.351+0000 7f47e356bb80 1 --2- 172.21.15.101:0/2232720674 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x56547f598120 0x56547f59a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.351+0000 7f47e356bb80 4 client.26660 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:12:55.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.351+0000 7f47e356bb80 1 -- 172.21.15.101:0/2232720674 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56547f4c8eb0 con 0x56547f598120 2023-07-18T03:12:55.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.351+0000 7f47cd19a700 1 --2- 172.21.15.101:0/2232720674 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x56547f598120 0x56547f59a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.351+0000 7f47b17fa700 10 client.26660 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:12:55.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.351+0000 7f47cd19a700 1 --2- 172.21.15.101:0/2232720674 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x56547f598120 0x56547f59a530 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=79f8086cae03fbdd server_cookie=7d0edefac269f66 in_seq=0 out_seq=0 2023-07-18T03:12:55.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.352+0000 7f47b17fa700 1 -- 172.21.15.101:0/2232720674 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56547f4c8eb0 con 0x56547f598120 2023-07-18T03:12:55.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.352+0000 7f47b17fa700 10 client.26660 handle_command_reply: tid=0 2023-07-18T03:12:55.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.352+0000 7f47e356bb80 1 client.26660 shutdown 2023-07-18T03:12:55.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.353+0000 7f47e356bb80 1 -- 172.21.15.101:0/2232720674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56547f57d2d0 msgr2=0x56547f588560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:55.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.353+0000 7f47e356bb80 1 --2- 172.21.15.101:0/2232720674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56547f57d2d0 0x56547f588560 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f47c4099ed0 tx=0x7f47c40ca560 comp rx=0 tx=0).stop 2023-07-18T03:12:55.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.353+0000 7f47e356bb80 1 -- 172.21.15.101:0/2232720674 shutdown_connections 2023-07-18T03:12:55.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.353+0000 7f47e356bb80 1 --2- 172.21.15.101:0/2232720674 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x56547f598120 0x56547f59a530 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.353+0000 7f47e356bb80 1 --2- 172.21.15.101:0/2232720674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56547f57d2d0 0x56547f588560 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.353+0000 7f47e356bb80 1 --2- 172.21.15.101:0/2232720674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56547f588ad0 0x56547f585f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.353+0000 7f47b17fa700 0 client.26660 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:12:55.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.353+0000 7f47e356bb80 1 --2- 172.21.15.101:0/2232720674 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56547f57dbe0 0x56547f5857e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.353+0000 7f47e356bb80 1 -- 172.21.15.101:0/2232720674 >> 172.21.15.101:0/2232720674 conn(0x56547f573da0 msgr2=0x56547f579290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:55.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.353+0000 7f47e356bb80 1 -- 172.21.15.101:0/2232720674 shutdown_connections 2023-07-18T03:12:55.355 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.353+0000 7f47e356bb80 1 -- 172.21.15.101:0/2232720674 wait complete. 2023-07-18T03:12:55.355 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.353+0000 7f47e356bb80 20 client.26660 trim_cache size 0 max 16384 2023-07-18T03:12:55.369 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.377+0000 7f47e356bb80 1 Processor -- start 2023-07-18T03:12:55.370 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.377+0000 7f47e356bb80 1 -- start start 2023-07-18T03:12:55.370 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.377+0000 7f47e356bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56547f57d2d0 0x56547f588100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.370 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.377+0000 7f47e356bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56547f57dbe0 0x56547f584fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.370 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.377+0000 7f47e356bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56547f588ad0 0x56547f585500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.370 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.377+0000 7f47e356bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56547f586dc0 con 0x56547f588ad0 2023-07-18T03:12:55.371 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.377+0000 7f47e356bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56547f57ed60 con 0x56547f57dbe0 2023-07-18T03:12:55.371 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.377+0000 7f47e356bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56547f583d60 con 0x56547f57d2d0 2023-07-18T03:12:55.372 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.377+0000 7f47cd99b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56547f57d2d0 0x56547f588100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.372 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.377+0000 7f47ce19c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56547f588ad0 0x56547f585500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.372 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.377+0000 7f47cd19a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56547f57dbe0 0x56547f584fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.373 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.377+0000 7f47cd99b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56547f57d2d0 0x56547f588100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:51034/0 (socket says 172.21.15.101:51034) 2023-07-18T03:12:55.373 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.377+0000 7f47cd99b700 1 -- 172.21.15.101:0/2061679855 learned_addr learned my addr 172.21.15.101:0/2061679855 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:55.373 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.378+0000 7f47cd99b700 1 -- 172.21.15.101:0/2061679855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56547f57dbe0 msgr2=0x56547f584fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:55.373 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.378+0000 7f47cd99b700 1 --2- 172.21.15.101:0/2061679855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56547f57dbe0 0x56547f584fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.373 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.378+0000 7f47cd99b700 1 -- 172.21.15.101:0/2061679855 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56547f588ad0 msgr2=0x56547f585500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:55.373 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.378+0000 7f47cd99b700 1 --2- 172.21.15.101:0/2061679855 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56547f588ad0 0x56547f585500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.374 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.378+0000 7f47cd99b700 1 -- 172.21.15.101:0/2061679855 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56547f324250 con 0x56547f57d2d0 2023-07-18T03:12:55.374 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.378+0000 7f47cd99b700 1 --2- 172.21.15.101:0/2061679855 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56547f57d2d0 0x56547f588100 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f47c400e970 tx=0x7f47c4096330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61a6a0a6fe6191af server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:55.374 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.378+0000 7f47bcbff700 1 -- 172.21.15.101:0/2061679855 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c409a3b0 con 0x56547f57d2d0 2023-07-18T03:12:55.374 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.378+0000 7f47bcbff700 1 -- 172.21.15.101:0/2061679855 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47c400ca10 con 0x56547f57d2d0 2023-07-18T03:12:55.374 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.378+0000 7f47bcbff700 1 -- 172.21.15.101:0/2061679855 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c4099120 con 0x56547f57d2d0 2023-07-18T03:12:55.374 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.379+0000 7f47e356bb80 1 -- 172.21.15.101:0/2061679855 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56547f57d2d0 msgr2=0x56547f588100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:55.374 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.379+0000 7f47e356bb80 1 --2- 172.21.15.101:0/2061679855 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56547f57d2d0 0x56547f588100 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f47c400e970 tx=0x7f47c4096330 comp rx=0 tx=0).stop 2023-07-18T03:12:55.374 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.379+0000 7f47e356bb80 1 -- 172.21.15.101:0/2061679855 shutdown_connections 2023-07-18T03:12:55.375 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.379+0000 7f47e356bb80 1 --2- 172.21.15.101:0/2061679855 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56547f588ad0 0x56547f585500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.375 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.379+0000 7f47e356bb80 1 --2- 172.21.15.101:0/2061679855 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56547f57d2d0 0x56547f588100 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.375 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.379+0000 7f47e356bb80 1 --2- 172.21.15.101:0/2061679855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56547f57dbe0 0x56547f584fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.375 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.379+0000 7f47e356bb80 1 -- 172.21.15.101:0/2061679855 >> 172.21.15.101:0/2061679855 conn(0x56547f573da0 msgr2=0x56547f580b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:55.375 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.379+0000 7f47e356bb80 1 -- 172.21.15.101:0/2061679855 shutdown_connections 2023-07-18T03:12:55.375 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.379+0000 7f47e356bb80 1 -- 172.21.15.101:0/2061679855 wait complete. 2023-07-18T03:12:55.375 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.380+0000 7f47e356bb80 1 -- start start 2023-07-18T03:12:55.376 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.380+0000 7f47e356bb80 1 Processor -- start 2023-07-18T03:12:55.378 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.380+0000 7f47e356bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56547f57d2d0 0x56547f58d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.378 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.380+0000 7f47e356bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56547f57dbe0 0x56547f58dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.378 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.380+0000 7f47e356bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56547f588ad0 0x56547f5e4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.378 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.380+0000 7f47e356bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56547f5e54f0 con 0x56547f57d2d0 2023-07-18T03:12:55.379 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.380+0000 7f47e356bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56547f5e5660 con 0x56547f588ad0 2023-07-18T03:12:55.379 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.380+0000 7f47e356bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56547f5e5820 con 0x56547f57dbe0 2023-07-18T03:12:55.379 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.380+0000 7f47e356bb80 10 client.0 _ll_register_callbacks cb 0x7f47c0009f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:12:55.379 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.381+0000 7f47cd19a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56547f57dbe0 0x56547f58dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.379 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.381+0000 7f47cd99b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56547f57d2d0 0x56547f58d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.379 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.381+0000 7f47cd19a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56547f57dbe0 0x56547f58dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:51050/0 (socket says 172.21.15.101:51050) 2023-07-18T03:12:55.379 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.381+0000 7f47cd19a700 1 -- 172.21.15.101:0/3111702179 learned_addr learned my addr 172.21.15.101:0/3111702179 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:55.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.381+0000 7f47ce19c700 1 --2- 172.21.15.101:0/3111702179 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56547f588ad0 0x56547f5e4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.381+0000 7f47cd19a700 1 -- 172.21.15.101:0/3111702179 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56547f588ad0 msgr2=0x56547f5e4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:55.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.381+0000 7f47cd19a700 1 --2- 172.21.15.101:0/3111702179 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56547f588ad0 0x56547f5e4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.381+0000 7f47cd19a700 1 -- 172.21.15.101:0/3111702179 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56547f57d2d0 msgr2=0x56547f58d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:55.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.381+0000 7f47cd19a700 1 --2- 172.21.15.101:0/3111702179 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56547f57d2d0 0x56547f58d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.381+0000 7f47cd19a700 1 -- 172.21.15.101:0/3111702179 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56547f2e6c60 con 0x56547f57dbe0 2023-07-18T03:12:55.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.381+0000 7f47cd19a700 1 --2- 172.21.15.101:0/3111702179 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56547f57dbe0 0x56547f58dc00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f47b8009990 tx=0x7f47b8009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e894059ff9ae40f4 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:55.381 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.381+0000 7f47b17fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:12:55.381 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.381+0000 7f47b17fa700 1 -- 172.21.15.101:0/3111702179 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b80054c0 con 0x56547f57dbe0 2023-07-18T03:12:55.381 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.382+0000 7f47e356bb80 1 -- 172.21.15.101:0/3111702179 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f47c40982e0 con 0x56547f57dbe0 2023-07-18T03:12:55.381 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.382+0000 7f47b17fa700 1 -- 172.21.15.101:0/3111702179 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f47b8005620 con 0x56547f57dbe0 2023-07-18T03:12:55.381 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.382+0000 7f47b17fa700 1 -- 172.21.15.101:0/3111702179 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47b8033a60 con 0x56547f57dbe0 2023-07-18T03:12:55.381 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.382+0000 7f47e356bb80 10 client.26689 fetch_fsmap learned FSMap version 26 2023-07-18T03:12:55.381 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.382+0000 7f47e356bb80 1 -- 172.21.15.101:0/3111702179 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56547f2fc2b0 con 0x56547f57dbe0 2023-07-18T03:12:55.382 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.382+0000 7f47b17fa700 1 -- 172.21.15.101:0/3111702179 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47c40982e0 con 0x56547f57dbe0 2023-07-18T03:12:55.382 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.382+0000 7f47b17fa700 1 -- 172.21.15.101:0/3111702179 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f47b800fba0 con 0x56547f57dbe0 2023-07-18T03:12:55.382 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.382+0000 7f47e356bb80 10 client.26689 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:12:55.382 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.382+0000 7f47e356bb80 10 client.26689 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:12:55.382 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.382+0000 7f47e356bb80 20 client.26689 populate_metadata read hostname 'smithi101' 2023-07-18T03:12:55.382 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.382+0000 7f47e356bb80 1 --2- 172.21.15.101:0/3111702179 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x56547f5ecfe0 0x56547f5ef3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.382 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.382+0000 7f47e356bb80 4 client.26689 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:12:55.382 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.382+0000 7f47e356bb80 1 -- 172.21.15.101:0/3111702179 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56547f48b050 con 0x56547f5ecfe0 2023-07-18T03:12:55.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.382+0000 7f47cd99b700 1 --2- 172.21.15.101:0/3111702179 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x56547f5ecfe0 0x56547f5ef3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.383+0000 7f47b17fa700 10 client.26689 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:12:55.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.383+0000 7f47cd99b700 1 --2- 172.21.15.101:0/3111702179 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x56547f5ecfe0 0x56547f5ef3f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d25889429bd81cf9 server_cookie=5c6eda2371ac9e2c in_seq=0 out_seq=0 2023-07-18T03:12:55.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.383+0000 7f47b17fa700 1 -- 172.21.15.101:0/3111702179 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56547f48b050 con 0x56547f5ecfe0 2023-07-18T03:12:55.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.383+0000 7f47b17fa700 10 client.26689 handle_command_reply: tid=0 2023-07-18T03:12:55.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.383+0000 7f47e356bb80 1 client.26689 shutdown 2023-07-18T03:12:55.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.383+0000 7f47e356bb80 1 -- 172.21.15.101:0/3111702179 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56547f57dbe0 msgr2=0x56547f58dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:55.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.383+0000 7f47e356bb80 1 --2- 172.21.15.101:0/3111702179 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56547f57dbe0 0x56547f58dc00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f47b8009990 tx=0x7f47b8009ca0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.383+0000 7f47e356bb80 1 -- 172.21.15.101:0/3111702179 shutdown_connections 2023-07-18T03:12:55.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.383+0000 7f47e356bb80 1 --2- 172.21.15.101:0/3111702179 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x56547f5ecfe0 0x56547f5ef3f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.383+0000 7f47e356bb80 1 --2- 172.21.15.101:0/3111702179 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56547f588ad0 0x56547f5e4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.383+0000 7f47e356bb80 1 --2- 172.21.15.101:0/3111702179 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56547f57d2d0 0x56547f58d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.383+0000 7f47b17fa700 0 client.26689 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:12:55.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.383+0000 7f47e356bb80 1 --2- 172.21.15.101:0/3111702179 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56547f57dbe0 0x56547f58dc00 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.383+0000 7f47e356bb80 1 -- 172.21.15.101:0/3111702179 >> 172.21.15.101:0/3111702179 conn(0x56547f573da0 msgr2=0x56547f57f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:55.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.383+0000 7f47e356bb80 1 -- 172.21.15.101:0/3111702179 shutdown_connections 2023-07-18T03:12:55.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.383+0000 7f47e356bb80 1 -- 172.21.15.101:0/3111702179 wait complete. 2023-07-18T03:12:55.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.384+0000 7f47e356bb80 20 client.26689 trim_cache size 0 max 16384 2023-07-18T03:12:55.385 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:12:55.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.384+0000 7f47cfbff700 1 -- 172.21.15.101:0/3813298536 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f47b40a12c0 msgr2=0x7f47b40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:55.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.384+0000 7f47cfbff700 1 --2- 172.21.15.101:0/3813298536 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f47b40a12c0 0x7f47b40a3770 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f47b800b6f0 tx=0x7f47b800b280 comp rx=0 tx=0).stop 2023-07-18T03:12:55.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.384+0000 7f47cfbff700 1 -- 172.21.15.101:0/3813298536 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f47c80fe880 msgr2=0x7f47c8197e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:55.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.384+0000 7f47cfbff700 1 --2- 172.21.15.101:0/3813298536 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f47c80fe880 0x7f47c8197e70 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f47c400e9a0 tx=0x7f47c400ecb0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.385+0000 7f47cfbff700 1 -- 172.21.15.101:0/3813298536 shutdown_connections 2023-07-18T03:12:55.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.385+0000 7f47cfbff700 1 --2- 172.21.15.101:0/3813298536 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f47b40a12c0 0x7f47b40a3770 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.385+0000 7f47cfbff700 1 --2- 172.21.15.101:0/3813298536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47c810c430 0x7f47c819c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.385+0000 7f47cfbff700 1 --2- 172.21.15.101:0/3813298536 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f47c80fe880 0x7f47c8197e70 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.385+0000 7f47cfbff700 1 --2- 172.21.15.101:0/3813298536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47c80ff190 0x7f47c81983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.385+0000 7f47cfbff700 1 -- 172.21.15.101:0/3813298536 >> 172.21.15.101:0/3813298536 conn(0x7f47c80fa4b0 msgr2=0x7f47c810d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:55.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.385+0000 7f47cfbff700 1 -- 172.21.15.101:0/3813298536 shutdown_connections 2023-07-18T03:12:55.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.385+0000 7f47cfbff700 1 -- 172.21.15.101:0/3813298536 wait complete. 2023-07-18T03:12:55.402 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-18T03:12:55.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:54 smithi179 ceph-mon[129229]: pgmap v163: 129 pgs: 129 active+clean; 248 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 194 op/s 2023-07-18T03:12:55.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:54 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1502258105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:12:55.581 INFO:tasks.workunit.client.0.smithi101.stdout: 1 408017 8.74 MB/sec execute 140 sec latency 30.623 ms 2023-07-18T03:12:55.629 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.636+0000 7f9dc2fff700 1 -- 172.21.15.101:0/3697622678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9dbc105ae0 msgr2=0x7f9dbc105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:55.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.636+0000 7f9dc2fff700 1 --2- 172.21.15.101:0/3697622678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9dbc105ae0 0x7f9dbc105eb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9db0009990 tx=0x7f9db0009ca0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.637+0000 7f9dc2fff700 1 -- 172.21.15.101:0/3697622678 shutdown_connections 2023-07-18T03:12:55.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.637+0000 7f9dc2fff700 1 --2- 172.21.15.101:0/3697622678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9dbc100480 0x7f9dbc10c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.637+0000 7f9dc2fff700 1 --2- 172.21.15.101:0/3697622678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9dbc105ae0 0x7f9dbc105eb0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.637+0000 7f9dc2fff700 1 --2- 172.21.15.101:0/3697622678 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9dbc0ffad0 0x7f9dbc0fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.637+0000 7f9dc2fff700 1 -- 172.21.15.101:0/3697622678 >> 172.21.15.101:0/3697622678 conn(0x7f9dbc0fb5f0 msgr2=0x7f9dbc0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:55.631 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.637+0000 7f9dc2fff700 1 -- 172.21.15.101:0/3697622678 shutdown_connections 2023-07-18T03:12:55.631 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.637+0000 7f9dc2fff700 1 -- 172.21.15.101:0/3697622678 wait complete. 2023-07-18T03:12:55.634 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.638+0000 7f9dc2fff700 1 Processor -- start 2023-07-18T03:12:55.634 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.639+0000 7f9dc2fff700 1 -- start start 2023-07-18T03:12:55.634 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.639+0000 7f9dc2fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9dbc0ffad0 0x7f9dbc072830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.635 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.639+0000 7f9dc2fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9dbc100480 0x7f9dbc072d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.635 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.639+0000 7f9dc2fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9dbc105ae0 0x7f9dbc0732b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.635 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.639+0000 7f9dc2fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dbc199e20 con 0x7f9dbc0ffad0 2023-07-18T03:12:55.635 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.639+0000 7f9dc2fff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dbc199f90 con 0x7f9dbc100480 2023-07-18T03:12:55.635 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.639+0000 7f9dc2fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dbc19a150 con 0x7f9dbc105ae0 2023-07-18T03:12:55.635 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.640+0000 7f9dc159c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9dbc105ae0 0x7f9dbc0732b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.635 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.640+0000 7f9dc159c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9dbc105ae0 0x7f9dbc0732b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:51056/0 (socket says 172.21.15.101:51056) 2023-07-18T03:12:55.636 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.640+0000 7f9dc159c700 1 -- 172.21.15.101:0/3052167574 learned_addr learned my addr 172.21.15.101:0/3052167574 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:55.636 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.640+0000 7f9dc159c700 1 -- 172.21.15.101:0/3052167574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9dbc100480 msgr2=0x7f9dbc072d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:55.636 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.640+0000 7f9dc159c700 1 --2- 172.21.15.101:0/3052167574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9dbc100480 0x7f9dbc072d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.636 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.640+0000 7f9dc159c700 1 -- 172.21.15.101:0/3052167574 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9dbc0ffad0 msgr2=0x7f9dbc072830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:55.636 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.640+0000 7f9dc159c700 1 --2- 172.21.15.101:0/3052167574 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9dbc0ffad0 0x7f9dbc072830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.636 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.640+0000 7f9dc159c700 1 -- 172.21.15.101:0/3052167574 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db4009c40 con 0x7f9dbc105ae0 2023-07-18T03:12:55.636 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.641+0000 7f9dc159c700 1 --2- 172.21.15.101:0/3052167574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9dbc105ae0 0x7f9dbc0732b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9db400c370 tx=0x7f9db400c730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:55.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.642+0000 7f9db9ffb700 1 -- 172.21.15.101:0/3052167574 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db4017070 con 0x7f9dbc105ae0 2023-07-18T03:12:55.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.642+0000 7f9db9ffb700 1 -- 172.21.15.101:0/3052167574 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9db400f040 con 0x7f9dbc105ae0 2023-07-18T03:12:55.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.642+0000 7f9db9ffb700 1 -- 172.21.15.101:0/3052167574 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db400e050 con 0x7f9dbc105ae0 2023-07-18T03:12:55.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.642+0000 7f9dc2fff700 1 -- 172.21.15.101:0/3052167574 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9db00095f0 con 0x7f9dbc105ae0 2023-07-18T03:12:55.638 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.642+0000 7f9dc2fff700 1 -- 172.21.15.101:0/3052167574 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dbc19a700 con 0x7f9dbc105ae0 2023-07-18T03:12:55.645 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.652+0000 7f9dd68e2b80 1 Processor -- start 2023-07-18T03:12:55.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.654+0000 7f9dd68e2b80 1 -- start start 2023-07-18T03:12:55.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.654+0000 7f9db9ffb700 1 -- 172.21.15.101:0/3052167574 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f9db40090d0 con 0x7f9dbc105ae0 2023-07-18T03:12:55.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.655+0000 7f9dd68e2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fc0990e2d0 0x55fc0990e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.651 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.655+0000 7f9dd68e2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc0990ebe0 0x55fc09919590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.651 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.655+0000 7f9dd68e2b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fc09919ad0 0x55fc0991beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.651 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.655+0000 7f9dd68e2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc09814cb0 con 0x55fc0990e2d0 2023-07-18T03:12:55.651 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.655+0000 7f9dd68e2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc09867a40 con 0x55fc0990ebe0 2023-07-18T03:12:55.651 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.655+0000 7f9dd68e2b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc09860ed0 con 0x55fc09919ad0 2023-07-18T03:12:55.651 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.657+0000 7f9dbbfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fc09919ad0 0x55fc0991beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.651 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.658+0000 7f9dbbfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fc09919ad0 0x55fc0991beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:51064/0 (socket says 172.21.15.101:51064) 2023-07-18T03:12:55.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.658+0000 7f9dbbfff700 1 -- 172.21.15.101:0/3209220912 learned_addr learned my addr 172.21.15.101:0/3209220912 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:55.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.658+0000 7f9dbbfff700 1 --2- 172.21.15.101:0/3209220912 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fc0990e2d0 0x55fc0990e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.658+0000 7f9dbbfff700 1 -- 172.21.15.101:0/3209220912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc0990ebe0 msgr2=0x55fc09919590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:55.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.658+0000 7f9dbbfff700 1 --2- 172.21.15.101:0/3209220912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc0990ebe0 0x55fc09919590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.658+0000 7f9dbbfff700 1 -- 172.21.15.101:0/3209220912 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fc0990e2d0 msgr2=0x55fc0990e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:55.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.658+0000 7f9dbbfff700 1 --2- 172.21.15.101:0/3209220912 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fc0990e2d0 0x55fc0990e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.658+0000 7f9dbbfff700 1 -- 172.21.15.101:0/3209220912 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc0968ede0 con 0x55fc09919ad0 2023-07-18T03:12:55.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.659+0000 7f9dbbfff700 1 --2- 172.21.15.101:0/3209220912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fc09919ad0 0x55fc0991beb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9da80127f0 tx=0x7f9da800af90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ef7db8b08ffa34f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:55.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.659+0000 7f9db9ffb700 1 --2- 172.21.15.101:0/3052167574 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9dac0a1490 0x7f9dac0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.659+0000 7f9db9ffb700 1 -- 172.21.15.101:0/3052167574 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f9db40cc890 con 0x7f9dbc105ae0 2023-07-18T03:12:55.654 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.661+0000 7f9dc0d9b700 1 --2- 172.21.15.101:0/3052167574 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9dac0a1490 0x7f9dac0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.662+0000 7f9da6ffd700 1 -- 172.21.15.101:0/3209220912 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da8012230 con 0x55fc09919ad0 2023-07-18T03:12:55.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.662+0000 7f9da6ffd700 1 -- 172.21.15.101:0/3209220912 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9da8018070 con 0x55fc09919ad0 2023-07-18T03:12:55.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.662+0000 7f9da6ffd700 1 -- 172.21.15.101:0/3209220912 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da800d3e0 con 0x55fc09919ad0 2023-07-18T03:12:55.659 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.664+0000 7f9dd68e2b80 1 -- 172.21.15.101:0/3209220912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fc09919ad0 msgr2=0x55fc0991beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:55.659 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.664+0000 7f9dd68e2b80 1 --2- 172.21.15.101:0/3209220912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fc09919ad0 0x55fc0991beb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9da80127f0 tx=0x7f9da800af90 comp rx=0 tx=0).stop 2023-07-18T03:12:55.659 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.665+0000 7f9dd68e2b80 1 -- 172.21.15.101:0/3209220912 shutdown_connections 2023-07-18T03:12:55.659 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.665+0000 7f9dd68e2b80 1 --2- 172.21.15.101:0/3209220912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fc09919ad0 0x55fc0991beb0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.665+0000 7f9dd68e2b80 1 --2- 172.21.15.101:0/3209220912 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fc0990e2d0 0x55fc0990e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.665+0000 7f9dd68e2b80 1 --2- 172.21.15.101:0/3209220912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc0990ebe0 0x55fc09919590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.665+0000 7f9dd68e2b80 1 -- 172.21.15.101:0/3209220912 >> 172.21.15.101:0/3209220912 conn(0x55fc09904da0 msgr2=0x55fc0990a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:55.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.665+0000 7f9dc0d9b700 1 --2- 172.21.15.101:0/3052167574 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9dac0a1490 0x7f9dac0a3940 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9db00098e0 tx=0x7f9db0005fb0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:55.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.668+0000 7f9dd68e2b80 1 -- 172.21.15.101:0/3209220912 shutdown_connections 2023-07-18T03:12:55.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.671+0000 7f9dd68e2b80 1 -- 172.21.15.101:0/3209220912 wait complete. 2023-07-18T03:12:55.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.672+0000 7f9dd68e2b80 1 -- start start 2023-07-18T03:12:55.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.673+0000 7f9dd68e2b80 1 Processor -- start 2023-07-18T03:12:55.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.673+0000 7f9dd68e2b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fc0990e2d0 0x55fc09916870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.673+0000 7f9dd68e2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc0990ebe0 0x55fc09916db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.673+0000 7f9dd68e2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fc09919ad0 0x55fc099172f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.673+0000 7f9dd68e2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc09800ac0 con 0x55fc09919ad0 2023-07-18T03:12:55.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.674+0000 7f9dd68e2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc098007c0 con 0x55fc0990ebe0 2023-07-18T03:12:55.668 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.674+0000 7f9dd68e2b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc096e4920 con 0x55fc0990e2d0 2023-07-18T03:12:55.668 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.674+0000 7f9dd68e2b80 10 client.0 _ll_register_callbacks cb 0x55fc097fe760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:12:55.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.674+0000 7f9dc0d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fc0990e2d0 0x55fc09916870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.674+0000 7f9dc0d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fc0990e2d0 0x55fc09916870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:51076/0 (socket says 172.21.15.101:51076) 2023-07-18T03:12:55.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.674+0000 7f9dc0d9b700 1 -- 172.21.15.101:0/1874148804 learned_addr learned my addr 172.21.15.101:0/1874148804 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:55.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.674+0000 7f9dc159c700 1 --2- 172.21.15.101:0/1874148804 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fc09919ad0 0x55fc099172f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.674+0000 7f9dbbfff700 1 --2- 172.21.15.101:0/1874148804 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc0990ebe0 0x55fc09916db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.675+0000 7f9dc0d9b700 1 -- 172.21.15.101:0/1874148804 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc0990ebe0 msgr2=0x55fc09916db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:55.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.675+0000 7f9dc0d9b700 1 --2- 172.21.15.101:0/1874148804 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc0990ebe0 0x55fc09916db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.675+0000 7f9dc0d9b700 1 -- 172.21.15.101:0/1874148804 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fc09919ad0 msgr2=0x55fc099172f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:55.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.675+0000 7f9dc0d9b700 1 --2- 172.21.15.101:0/1874148804 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fc09919ad0 0x55fc099172f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.675+0000 7f9dc0d9b700 1 -- 172.21.15.101:0/1874148804 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9da80124d0 con 0x55fc0990e2d0 2023-07-18T03:12:55.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.675+0000 7f9dc159c700 1 --2- 172.21.15.101:0/1874148804 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fc09919ad0 0x55fc099172f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:12:55.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.675+0000 7f9dc0d9b700 1 --2- 172.21.15.101:0/1874148804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fc0990e2d0 0x55fc09916870 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9db00056a0 tx=0x7f9db000f740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=255d2e737bb53183 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:55.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.676+0000 7f9d9bfff700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:12:55.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.676+0000 7f9d9bfff700 1 -- 172.21.15.101:0/1874148804 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db001d070 con 0x55fc0990e2d0 2023-07-18T03:12:55.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.676+0000 7f9d9bfff700 1 -- 172.21.15.101:0/1874148804 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f9db000fdd0 con 0x55fc0990e2d0 2023-07-18T03:12:55.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.676+0000 7f9dd68e2b80 1 -- 172.21.15.101:0/1874148804 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fc097ba4f0 con 0x55fc0990e2d0 2023-07-18T03:12:55.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.677+0000 7f9d9bfff700 1 -- 172.21.15.101:0/1874148804 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9db00056d0 con 0x55fc0990e2d0 2023-07-18T03:12:55.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.677+0000 7f9dd68e2b80 10 client.26701 fetch_fsmap learned FSMap version 26 2023-07-18T03:12:55.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.677+0000 7f9dd68e2b80 1 -- 172.21.15.101:0/1874148804 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55fc09677c60 con 0x55fc0990e2d0 2023-07-18T03:12:55.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.677+0000 7f9d9bfff700 1 -- 172.21.15.101:0/1874148804 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fc097ba4f0 con 0x55fc0990e2d0 2023-07-18T03:12:55.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.678+0000 7f9d9bfff700 1 -- 172.21.15.101:0/1874148804 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9db0025520 con 0x55fc0990e2d0 2023-07-18T03:12:55.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.678+0000 7f9dd68e2b80 10 client.26701 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:12:55.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.678+0000 7f9dd68e2b80 10 client.26701 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:12:55.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.678+0000 7f9dd68e2b80 20 client.26701 populate_metadata read hostname 'smithi101' 2023-07-18T03:12:55.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.678+0000 7f9dd68e2b80 1 --2- 172.21.15.101:0/1874148804 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55fc0993a4a0 0x55fc0993c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.678+0000 7f9dd68e2b80 4 client.26701 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:12:55.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.678+0000 7f9dd68e2b80 1 -- 172.21.15.101:0/1874148804 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fc09859eb0 con 0x55fc0993a4a0 2023-07-18T03:12:55.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.679+0000 7f9dbbfff700 1 --2- 172.21.15.101:0/1874148804 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55fc0993a4a0 0x55fc0993c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.679+0000 7f9dbbfff700 1 --2- 172.21.15.101:0/1874148804 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55fc0993a4a0 0x55fc0993c8b0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbbb779352646c9b server_cookie=b4ba780629548a4e in_seq=0 out_seq=0 2023-07-18T03:12:55.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.680+0000 7f9d9bfff700 10 client.26701 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:12:55.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.680+0000 7f9d9bfff700 1 -- 172.21.15.101:0/1874148804 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55fc09859eb0 con 0x55fc0993a4a0 2023-07-18T03:12:55.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.680+0000 7f9d9bfff700 10 client.26701 handle_command_reply: tid=0 2023-07-18T03:12:55.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.680+0000 7f9dd68e2b80 1 client.26701 shutdown 2023-07-18T03:12:55.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.681+0000 7f9dd68e2b80 1 -- 172.21.15.101:0/1874148804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fc0990e2d0 msgr2=0x55fc09916870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:55.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.681+0000 7f9dd68e2b80 1 --2- 172.21.15.101:0/1874148804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fc0990e2d0 0x55fc09916870 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9db00056a0 tx=0x7f9db000f740 comp rx=0 tx=0).stop 2023-07-18T03:12:55.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.685+0000 7f9dd68e2b80 1 -- 172.21.15.101:0/1874148804 shutdown_connections 2023-07-18T03:12:55.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.685+0000 7f9dd68e2b80 1 --2- 172.21.15.101:0/1874148804 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55fc0993a4a0 0x55fc0993c8b0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.685+0000 7f9dd68e2b80 1 --2- 172.21.15.101:0/1874148804 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fc09919ad0 0x55fc099172f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.685+0000 7f9dd68e2b80 1 --2- 172.21.15.101:0/1874148804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fc0990e2d0 0x55fc09916870 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.685+0000 7f9dd68e2b80 1 --2- 172.21.15.101:0/1874148804 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc0990ebe0 0x55fc09916db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.685+0000 7f9dd68e2b80 1 -- 172.21.15.101:0/1874148804 >> 172.21.15.101:0/1874148804 conn(0x55fc09904da0 msgr2=0x55fc0990a230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:55.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.690+0000 7f9d9bfff700 0 client.26701 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:12:55.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.692+0000 7f9dd68e2b80 1 -- 172.21.15.101:0/1874148804 shutdown_connections 2023-07-18T03:12:55.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.693+0000 7f9dd68e2b80 1 -- 172.21.15.101:0/1874148804 wait complete. 2023-07-18T03:12:55.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.695+0000 7f9dd68e2b80 20 client.26701 trim_cache size 0 max 16384 2023-07-18T03:12:55.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.709+0000 7f9dd68e2b80 1 Processor -- start 2023-07-18T03:12:55.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.710+0000 7f9dd68e2b80 1 -- start start 2023-07-18T03:12:55.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.710+0000 7f9dd68e2b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fc0990e2d0 0x55fc09931580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.710+0000 7f9dd68e2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fc0990ebe0 0x55fc09916080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.710+0000 7f9dd68e2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc09919ad0 0x55fc09913b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.710+0000 7f9dd68e2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc0990f540 con 0x55fc0990ebe0 2023-07-18T03:12:55.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.710+0000 7f9dd68e2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc09917e50 con 0x55fc09919ad0 2023-07-18T03:12:55.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.710+0000 7f9dd68e2b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc09936f20 con 0x55fc0990e2d0 2023-07-18T03:12:55.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.710+0000 7f9dc0d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fc0990e2d0 0x55fc09931580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.710+0000 7f9dc0d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fc0990e2d0 0x55fc09931580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:51088/0 (socket says 172.21.15.101:51088) 2023-07-18T03:12:55.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.710+0000 7f9dc159c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc09919ad0 0x55fc09913b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.710+0000 7f9dc0d9b700 1 -- 172.21.15.101:0/4146255101 learned_addr learned my addr 172.21.15.101:0/4146255101 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:55.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.710+0000 7f9dbbfff700 1 --2- 172.21.15.101:0/4146255101 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fc0990ebe0 0x55fc09916080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.710+0000 7f9dc0d9b700 1 -- 172.21.15.101:0/4146255101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc09919ad0 msgr2=0x55fc09913b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:55.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.710+0000 7f9dc0d9b700 1 --2- 172.21.15.101:0/4146255101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc09919ad0 0x55fc09913b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.710+0000 7f9dc0d9b700 1 -- 172.21.15.101:0/4146255101 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fc0990ebe0 msgr2=0x55fc09916080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:55.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.710+0000 7f9dc0d9b700 1 --2- 172.21.15.101:0/4146255101 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fc0990ebe0 0x55fc09916080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.710+0000 7f9dc0d9b700 1 -- 172.21.15.101:0/4146255101 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc099090e0 con 0x55fc0990e2d0 2023-07-18T03:12:55.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.711+0000 7f9dc0d9b700 1 --2- 172.21.15.101:0/4146255101 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fc0990e2d0 0x55fc09931580 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9db000c010 tx=0x7f9db0005d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3aa26e8611aac21 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:55.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.711+0000 7f9da77fe700 1 -- 172.21.15.101:0/4146255101 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db003d070 con 0x55fc0990e2d0 2023-07-18T03:12:55.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.711+0000 7f9da77fe700 1 -- 172.21.15.101:0/4146255101 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9db00056d0 con 0x55fc0990e2d0 2023-07-18T03:12:55.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.711+0000 7f9da77fe700 1 -- 172.21.15.101:0/4146255101 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db002fb60 con 0x55fc0990e2d0 2023-07-18T03:12:55.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.712+0000 7f9dd68e2b80 1 -- 172.21.15.101:0/4146255101 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fc0990e2d0 msgr2=0x55fc09931580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:55.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.712+0000 7f9dd68e2b80 1 --2- 172.21.15.101:0/4146255101 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fc0990e2d0 0x55fc09931580 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9db000c010 tx=0x7f9db0005d00 comp rx=0 tx=0).stop 2023-07-18T03:12:55.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.712+0000 7f9dd68e2b80 1 -- 172.21.15.101:0/4146255101 shutdown_connections 2023-07-18T03:12:55.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.712+0000 7f9dd68e2b80 1 --2- 172.21.15.101:0/4146255101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc09919ad0 0x55fc09913b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.712+0000 7f9dd68e2b80 1 --2- 172.21.15.101:0/4146255101 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fc0990e2d0 0x55fc09931580 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.712+0000 7f9dd68e2b80 1 --2- 172.21.15.101:0/4146255101 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fc0990ebe0 0x55fc09916080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.712+0000 7f9dd68e2b80 1 -- 172.21.15.101:0/4146255101 >> 172.21.15.101:0/4146255101 conn(0x55fc09904da0 msgr2=0x55fc09911bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:55.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.712+0000 7f9dd68e2b80 1 -- 172.21.15.101:0/4146255101 shutdown_connections 2023-07-18T03:12:55.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.712+0000 7f9dd68e2b80 1 -- 172.21.15.101:0/4146255101 wait complete. 2023-07-18T03:12:55.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.712+0000 7f9dd68e2b80 1 -- start start 2023-07-18T03:12:55.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.712+0000 7f9dd68e2b80 1 Processor -- start 2023-07-18T03:12:55.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.713+0000 7f9dd68e2b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fc0990e2d0 0x55fc09931400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.713+0000 7f9dd68e2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc0990ebe0 0x55fc0992e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.713+0000 7f9dd68e2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fc09919ad0 0x55fc0992ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.713+0000 7f9dd68e2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc0992f1a0 con 0x55fc09919ad0 2023-07-18T03:12:55.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.713+0000 7f9dd68e2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc0992f310 con 0x55fc0990ebe0 2023-07-18T03:12:55.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.713+0000 7f9dd68e2b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc0992f4d0 con 0x55fc0990e2d0 2023-07-18T03:12:55.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.713+0000 7f9dd68e2b80 10 client.0 _ll_register_callbacks cb 0x7f9db000be40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:12:55.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.713+0000 7f9dc159c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fc09919ad0 0x55fc0992ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.713+0000 7f9dc0d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fc0990e2d0 0x55fc09931400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.713+0000 7f9dc0d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fc0990e2d0 0x55fc09931400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:51090/0 (socket says 172.21.15.101:51090) 2023-07-18T03:12:55.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.713+0000 7f9dc159c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fc09919ad0 0x55fc0992ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:35832/0 (socket says 172.21.15.101:35832) 2023-07-18T03:12:55.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.713+0000 7f9dc0d9b700 1 -- 172.21.15.101:0/3063688588 learned_addr learned my addr 172.21.15.101:0/3063688588 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:55.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.713+0000 7f9dbbfff700 1 --2- 172.21.15.101:0/3063688588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc0990ebe0 0x55fc0992e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.714+0000 7f9dc159c700 1 -- 172.21.15.101:0/3063688588 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fc0990e2d0 msgr2=0x55fc09931400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:55.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.714+0000 7f9dc159c700 1 --2- 172.21.15.101:0/3063688588 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fc0990e2d0 0x55fc09931400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.714+0000 7f9dc159c700 1 -- 172.21.15.101:0/3063688588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc0990ebe0 msgr2=0x55fc0992e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:55.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.714+0000 7f9dc159c700 1 --2- 172.21.15.101:0/3063688588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc0990ebe0 0x55fc0992e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.714+0000 7f9dc159c700 1 -- 172.21.15.101:0/3063688588 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fc09677c60 con 0x55fc09919ad0 2023-07-18T03:12:55.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.714+0000 7f9dc0d9b700 1 --2- 172.21.15.101:0/3063688588 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fc0990e2d0 0x55fc09931400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:12:55.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.714+0000 7f9dbbfff700 1 --2- 172.21.15.101:0/3063688588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc0990ebe0 0x55fc0992e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:12:55.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.714+0000 7f9dc159c700 1 --2- 172.21.15.101:0/3063688588 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fc09919ad0 0x55fc0992ec60 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9db40c9c30 tx=0x7f9db40b7ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d181a66ae7a64352 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:55.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.714+0000 7f9d9bfff700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:12:55.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.714+0000 7f9d9bfff700 1 -- 172.21.15.101:0/3063688588 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db4098f10 con 0x55fc09919ad0 2023-07-18T03:12:55.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.714+0000 7f9dd68e2b80 1 -- 172.21.15.101:0/3063688588 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9da8018070 con 0x55fc09919ad0 2023-07-18T03:12:55.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.714+0000 7f9d9bfff700 1 -- 172.21.15.101:0/3063688588 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f9db40995b0 con 0x55fc09919ad0 2023-07-18T03:12:55.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.714+0000 7f9d9bfff700 1 -- 172.21.15.101:0/3063688588 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9db409d5e0 con 0x55fc09919ad0 2023-07-18T03:12:55.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.714+0000 7f9dd68e2b80 10 client.16632 fetch_fsmap learned FSMap version 26 2023-07-18T03:12:55.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.714+0000 7f9dd68e2b80 1 -- 172.21.15.101:0/3063688588 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55fc096b5250 con 0x55fc09919ad0 2023-07-18T03:12:55.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.714+0000 7f9d9bfff700 1 -- 172.21.15.101:0/3063688588 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9da8018070 con 0x55fc09919ad0 2023-07-18T03:12:55.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.715+0000 7f9d9bfff700 1 -- 172.21.15.101:0/3063688588 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9db40ae040 con 0x55fc09919ad0 2023-07-18T03:12:55.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.715+0000 7f9dd68e2b80 10 client.16632 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:12:55.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.715+0000 7f9dd68e2b80 10 client.16632 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:12:55.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.715+0000 7f9dd68e2b80 20 client.16632 populate_metadata read hostname 'smithi101' 2023-07-18T03:12:55.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.715+0000 7f9dd68e2b80 1 --2- 172.21.15.101:0/3063688588 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55fc0998e7b0 0x55fc09990bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:55.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.715+0000 7f9dd68e2b80 4 client.16632 mds_command: new command op to 24562 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:12:55.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.715+0000 7f9dd68e2b80 1 -- 172.21.15.101:0/3063688588 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fc0981c050 con 0x55fc0998e7b0 2023-07-18T03:12:55.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.715+0000 7f9dc0d9b700 1 --2- 172.21.15.101:0/3063688588 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55fc0998e7b0 0x55fc09990bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:55.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.715+0000 7f9d9bfff700 10 client.16632 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:12:55.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.715+0000 7f9dc0d9b700 1 --2- 172.21.15.101:0/3063688588 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55fc0998e7b0 0x55fc09990bc0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2668177ede1d30c2 server_cookie=d26ba9206aaded39 in_seq=0 out_seq=0 2023-07-18T03:12:55.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.716+0000 7f9d9bfff700 1 -- 172.21.15.101:0/3063688588 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fc0981c050 con 0x55fc0998e7b0 2023-07-18T03:12:55.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.716+0000 7f9d9bfff700 10 client.16632 handle_command_reply: tid=0 2023-07-18T03:12:55.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.716+0000 7f9dd68e2b80 1 client.16632 shutdown 2023-07-18T03:12:55.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.716+0000 7f9dd68e2b80 1 -- 172.21.15.101:0/3063688588 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fc09919ad0 msgr2=0x55fc0992ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:55.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.716+0000 7f9dd68e2b80 1 --2- 172.21.15.101:0/3063688588 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fc09919ad0 0x55fc0992ec60 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9db40c9c30 tx=0x7f9db40b7ec0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.716+0000 7f9dd68e2b80 1 -- 172.21.15.101:0/3063688588 shutdown_connections 2023-07-18T03:12:55.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.716+0000 7f9dd68e2b80 1 --2- 172.21.15.101:0/3063688588 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55fc0998e7b0 0x55fc09990bc0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.716+0000 7f9dd68e2b80 1 --2- 172.21.15.101:0/3063688588 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fc09919ad0 0x55fc0992ec60 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.716+0000 7f9dd68e2b80 1 --2- 172.21.15.101:0/3063688588 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fc0990e2d0 0x55fc09931400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.716+0000 7f9d9bfff700 0 client.16632 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:12:55.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.716+0000 7f9dd68e2b80 1 --2- 172.21.15.101:0/3063688588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc0990ebe0 0x55fc0992e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.716+0000 7f9dd68e2b80 1 -- 172.21.15.101:0/3063688588 >> 172.21.15.101:0/3063688588 conn(0x55fc09904da0 msgr2=0x55fc09918180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:55.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.716+0000 7f9dd68e2b80 1 -- 172.21.15.101:0/3063688588 shutdown_connections 2023-07-18T03:12:55.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.716+0000 7f9dd68e2b80 1 -- 172.21.15.101:0/3063688588 wait complete. 2023-07-18T03:12:55.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.717+0000 7f9dd68e2b80 20 client.16632 trim_cache size 0 max 16384 2023-07-18T03:12:55.723 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:12:55.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.717+0000 7f9dc2fff700 1 -- 172.21.15.101:0/3052167574 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9dac0a1490 msgr2=0x7f9dac0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:55.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.717+0000 7f9dc2fff700 1 --2- 172.21.15.101:0/3052167574 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9dac0a1490 0x7f9dac0a3940 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9db00098e0 tx=0x7f9db0005fb0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.717+0000 7f9dc2fff700 1 -- 172.21.15.101:0/3052167574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9dbc105ae0 msgr2=0x7f9dbc0732b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:55.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.717+0000 7f9dc2fff700 1 --2- 172.21.15.101:0/3052167574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9dbc105ae0 0x7f9dbc0732b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9db400c370 tx=0x7f9db400c730 comp rx=0 tx=0).stop 2023-07-18T03:12:55.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.717+0000 7f9dc2fff700 1 -- 172.21.15.101:0/3052167574 shutdown_connections 2023-07-18T03:12:55.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.717+0000 7f9dc2fff700 1 --2- 172.21.15.101:0/3052167574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9dbc105ae0 0x7f9dbc0732b0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.717+0000 7f9dc2fff700 1 --2- 172.21.15.101:0/3052167574 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9dbc0ffad0 0x7f9dbc072830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.717+0000 7f9dc2fff700 1 --2- 172.21.15.101:0/3052167574 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9dac0a1490 0x7f9dac0a3940 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.717+0000 7f9dc2fff700 1 --2- 172.21.15.101:0/3052167574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9dbc100480 0x7f9dbc072d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:55.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.717+0000 7f9dc2fff700 1 -- 172.21.15.101:0/3052167574 >> 172.21.15.101:0/3052167574 conn(0x7f9dbc0fb5f0 msgr2=0x7f9dbc0fbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:55.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.717+0000 7f9dc2fff700 1 -- 172.21.15.101:0/3052167574 shutdown_connections 2023-07-18T03:12:55.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:55.717+0000 7f9dc2fff700 1 -- 172.21.15.101:0/3052167574 wait complete. 2023-07-18T03:12:56.581 INFO:tasks.workunit.client.0.smithi101.stdout: 1 408822 8.70 MB/sec execute 141 sec latency 43.228 ms 2023-07-18T03:12:56.728 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-18T03:12:56.729 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-18T03:12:56.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.944+0000 7f2dec7ff700 1 -- 172.21.15.101:0/1657764298 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2de4100480 msgr2=0x7f2de410c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:56.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.944+0000 7f2dec7ff700 1 --2- 172.21.15.101:0/1657764298 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2de4100480 0x7f2de410c910 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc00b0a0 tx=0x7f2ddc00b3b0 comp rx=0 tx=0).stop 2023-07-18T03:12:56.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.945+0000 7f2dec7ff700 1 -- 172.21.15.101:0/1657764298 shutdown_connections 2023-07-18T03:12:56.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.945+0000 7f2dec7ff700 1 --2- 172.21.15.101:0/1657764298 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2de4100480 0x7f2de410c910 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:56.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.945+0000 7f2dec7ff700 1 --2- 172.21.15.101:0/1657764298 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2de4105ae0 0x7f2de4105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:56.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.945+0000 7f2dec7ff700 1 --2- 172.21.15.101:0/1657764298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2de40ffad0 0x7f2de40fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:56.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.945+0000 7f2dec7ff700 1 -- 172.21.15.101:0/1657764298 >> 172.21.15.101:0/1657764298 conn(0x7f2de40fb5f0 msgr2=0x7f2de40fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:56.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.945+0000 7f2dec7ff700 1 -- 172.21.15.101:0/1657764298 shutdown_connections 2023-07-18T03:12:56.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.945+0000 7f2dec7ff700 1 -- 172.21.15.101:0/1657764298 wait complete. 2023-07-18T03:12:56.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.946+0000 7f2dec7ff700 1 Processor -- start 2023-07-18T03:12:56.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.946+0000 7f2dec7ff700 1 -- start start 2023-07-18T03:12:56.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.946+0000 7f2dec7ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2de40ffad0 0x7f2de4072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:56.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.946+0000 7f2dec7ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2de4100480 0x7f2de4072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:56.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.946+0000 7f2dec7ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2de4105ae0 0x7f2de41a3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:56.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.946+0000 7f2dec7ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de4073580 con 0x7f2de40ffad0 2023-07-18T03:12:56.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.946+0000 7f2dec7ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de41a4400 con 0x7f2de4100480 2023-07-18T03:12:56.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.946+0000 7f2dec7ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de41a4540 con 0x7f2de4105ae0 2023-07-18T03:12:56.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.947+0000 7f2dea59b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2de40ffad0 0x7f2de4072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:56.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.947+0000 7f2dea59b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2de40ffad0 0x7f2de4072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:43036/0 (socket says 172.21.15.101:43036) 2023-07-18T03:12:56.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.947+0000 7f2dea59b700 1 -- 172.21.15.101:0/5373935 learned_addr learned my addr 172.21.15.101:0/5373935 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:56.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.947+0000 7f2dead9c700 1 --2- 172.21.15.101:0/5373935 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2de4105ae0 0x7f2de41a3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:56.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.947+0000 7f2de9d9a700 1 --2- 172.21.15.101:0/5373935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2de4100480 0x7f2de4072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:56.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.947+0000 7f2dea59b700 1 -- 172.21.15.101:0/5373935 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2de4105ae0 msgr2=0x7f2de41a3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:56.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.947+0000 7f2dea59b700 1 --2- 172.21.15.101:0/5373935 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2de4105ae0 0x7f2de41a3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:56.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.947+0000 7f2dea59b700 1 -- 172.21.15.101:0/5373935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2de4100480 msgr2=0x7f2de4072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:56.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.947+0000 7f2dea59b700 1 --2- 172.21.15.101:0/5373935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2de4100480 0x7f2de4072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:56.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.947+0000 7f2dea59b700 1 -- 172.21.15.101:0/5373935 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ddc009d00 con 0x7f2de40ffad0 2023-07-18T03:12:56.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.947+0000 7f2dead9c700 1 --2- 172.21.15.101:0/5373935 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2de4105ae0 0x7f2de41a3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:12:56.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.947+0000 7f2dea59b700 1 --2- 172.21.15.101:0/5373935 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2de40ffad0 0x7f2de4072810 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2de000e9a0 tx=0x7f2de000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:56.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.948+0000 7f2ddb7fe700 1 -- 172.21.15.101:0/5373935 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de0009790 con 0x7f2de40ffad0 2023-07-18T03:12:56.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.948+0000 7f2ddb7fe700 1 -- 172.21.15.101:0/5373935 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2de0010430 con 0x7f2de40ffad0 2023-07-18T03:12:56.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.948+0000 7f2ddb7fe700 1 -- 172.21.15.101:0/5373935 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de0018680 con 0x7f2de40ffad0 2023-07-18T03:12:56.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.948+0000 7f2dec7ff700 1 -- 172.21.15.101:0/5373935 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2de41a47a0 con 0x7f2de40ffad0 2023-07-18T03:12:56.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.948+0000 7f2dec7ff700 1 -- 172.21.15.101:0/5373935 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2de41a4c30 con 0x7f2de40ffad0 2023-07-18T03:12:56.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.951+0000 7f2ddb7fe700 1 -- 172.21.15.101:0/5373935 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f2de00105a0 con 0x7f2de40ffad0 2023-07-18T03:12:56.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.952+0000 7f2ddb7fe700 1 --2- 172.21.15.101:0/5373935 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2dd00a1340 0x7f2dd00a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:56.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.952+0000 7f2e00064b80 1 Processor -- start 2023-07-18T03:12:56.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.952+0000 7f2ddb7fe700 1 -- 172.21.15.101:0/5373935 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f2de0014070 con 0x7f2de40ffad0 2023-07-18T03:12:56.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.952+0000 7f2e00064b80 1 -- start start 2023-07-18T03:12:56.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.953+0000 7f2de9d9a700 1 --2- 172.21.15.101:0/5373935 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2dd00a1340 0x7f2dd00a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:56.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.953+0000 7f2e00064b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560886f40350 0x560886f40720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:56.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.953+0000 7f2e00064b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560886f40c60 0x560886f4b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:56.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.953+0000 7f2e00064b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560886f4bb50 0x560886f4df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:56.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.953+0000 7f2e00064b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560886e46d30 con 0x560886f40350 2023-07-18T03:12:56.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.953+0000 7f2e00064b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560886e99ac0 con 0x560886f4bb50 2023-07-18T03:12:56.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.953+0000 7f2e00064b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x560886e92f50 con 0x560886f40c60 2023-07-18T03:12:56.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.953+0000 7f2dea59b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560886f40350 0x560886f40720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:56.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.953+0000 7f2de9d9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560886f40c60 0x560886f4b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:56.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.953+0000 7f2dea59b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560886f40350 0x560886f40720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:43040/0 (socket says 172.21.15.101:43040) 2023-07-18T03:12:56.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.953+0000 7f2dea59b700 1 -- 172.21.15.101:0/540897873 learned_addr learned my addr 172.21.15.101:0/540897873 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:56.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.953+0000 7f2dead9c700 1 --2- 172.21.15.101:0/540897873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560886f4bb50 0x560886f4df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:56.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.953+0000 7f2de9d9a700 1 --2- 172.21.15.101:0/5373935 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2dd00a1340 0x7f2dd00a37f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2de4074000 tx=0x7f2dd400b410 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:56.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.954+0000 7f2de9d9a700 1 -- 172.21.15.101:0/540897873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560886f4bb50 msgr2=0x560886f4df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:56.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.954+0000 7f2de9d9a700 1 --2- 172.21.15.101:0/540897873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560886f4bb50 0x560886f4df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:56.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.954+0000 7f2de9d9a700 1 -- 172.21.15.101:0/540897873 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560886f40350 msgr2=0x560886f40720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:56.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.954+0000 7f2de9d9a700 1 --2- 172.21.15.101:0/540897873 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560886f40350 0x560886f40720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:56.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.954+0000 7f2de9d9a700 1 -- 172.21.15.101:0/540897873 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560886cc0e60 con 0x560886f40c60 2023-07-18T03:12:56.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.954+0000 7f2dea59b700 1 --2- 172.21.15.101:0/540897873 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560886f40350 0x560886f40720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:12:56.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.954+0000 7f2de9d9a700 1 --2- 172.21.15.101:0/540897873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560886f40c60 0x560886f4b610 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4010f40 tx=0x7f2dd401b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6415335bc79c9d0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:56.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.954+0000 7f2dd8ff9700 1 -- 172.21.15.101:0/540897873 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd4011db0 con 0x560886f40c60 2023-07-18T03:12:56.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.954+0000 7f2dd8ff9700 1 -- 172.21.15.101:0/540897873 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2dd400b930 con 0x560886f40c60 2023-07-18T03:12:56.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.954+0000 7f2dd8ff9700 1 -- 172.21.15.101:0/540897873 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd401d7a0 con 0x560886f40c60 2023-07-18T03:12:56.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.955+0000 7f2e00064b80 1 -- 172.21.15.101:0/540897873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560886f40c60 msgr2=0x560886f4b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:56.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.955+0000 7f2e00064b80 1 --2- 172.21.15.101:0/540897873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560886f40c60 0x560886f4b610 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4010f40 tx=0x7f2dd401b000 comp rx=0 tx=0).stop 2023-07-18T03:12:56.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.955+0000 7f2e00064b80 1 -- 172.21.15.101:0/540897873 shutdown_connections 2023-07-18T03:12:56.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.955+0000 7f2e00064b80 1 --2- 172.21.15.101:0/540897873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560886f4bb50 0x560886f4df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:56.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.955+0000 7f2e00064b80 1 --2- 172.21.15.101:0/540897873 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560886f40350 0x560886f40720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:56.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.955+0000 7f2e00064b80 1 --2- 172.21.15.101:0/540897873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560886f40c60 0x560886f4b610 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:56.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.955+0000 7f2e00064b80 1 -- 172.21.15.101:0/540897873 >> 172.21.15.101:0/540897873 conn(0x560886f36e20 msgr2=0x560886f3c720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:56.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.955+0000 7f2e00064b80 1 -- 172.21.15.101:0/540897873 shutdown_connections 2023-07-18T03:12:56.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.955+0000 7f2e00064b80 1 -- 172.21.15.101:0/540897873 wait complete. 2023-07-18T03:12:56.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.956+0000 7f2e00064b80 1 -- start start 2023-07-18T03:12:56.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.956+0000 7f2e00064b80 1 Processor -- start 2023-07-18T03:12:56.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.956+0000 7f2e00064b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560886f40350 0x560886f4b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:56.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.956+0000 7f2e00064b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560886f40c60 0x560886f487d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:56.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.956+0000 7f2e00064b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560886f4bb50 0x560886f48ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:56.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.956+0000 7f2e00064b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560886e32b40 con 0x560886f40c60 2023-07-18T03:12:56.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.956+0000 7f2e00064b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560886e32840 con 0x560886f4bb50 2023-07-18T03:12:56.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.956+0000 7f2e00064b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x560886d169a0 con 0x560886f40350 2023-07-18T03:12:56.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.956+0000 7f2e00064b80 10 client.0 _ll_register_callbacks cb 0x560886e307e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:12:56.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.957+0000 7f2de9d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560886f40c60 0x560886f487d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:56.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.957+0000 7f2dea59b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560886f40350 0x560886f4b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:56.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.957+0000 7f2dea59b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560886f40350 0x560886f4b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:57672/0 (socket says 172.21.15.101:57672) 2023-07-18T03:12:56.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.957+0000 7f2dea59b700 1 -- 172.21.15.101:0/148479672 learned_addr learned my addr 172.21.15.101:0/148479672 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:56.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.957+0000 7f2dead9c700 1 --2- 172.21.15.101:0/148479672 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560886f4bb50 0x560886f48ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:56.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.957+0000 7f2dea59b700 1 -- 172.21.15.101:0/148479672 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560886f4bb50 msgr2=0x560886f48ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:56.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.957+0000 7f2dea59b700 1 --2- 172.21.15.101:0/148479672 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560886f4bb50 0x560886f48ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:56.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.957+0000 7f2dea59b700 1 -- 172.21.15.101:0/148479672 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560886f40c60 msgr2=0x560886f487d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:56.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.957+0000 7f2dea59b700 1 --2- 172.21.15.101:0/148479672 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560886f40c60 0x560886f487d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:56.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.957+0000 7f2dea59b700 1 -- 172.21.15.101:0/148479672 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2de41a4c30 con 0x560886f40350 2023-07-18T03:12:56.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.957+0000 7f2dea59b700 1 --2- 172.21.15.101:0/148479672 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560886f40350 0x560886f4b5c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2de0096410 tx=0x7f2de0099f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e358a1f212972d7 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:56.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.957+0000 7f2dcdffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:12:56.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.957+0000 7f2dcdffb700 1 -- 172.21.15.101:0/148479672 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de0099600 con 0x560886f40350 2023-07-18T03:12:56.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.957+0000 7f2dcdffb700 1 -- 172.21.15.101:0/148479672 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f2de00a2360 con 0x560886f40350 2023-07-18T03:12:56.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.957+0000 7f2e00064b80 1 -- 172.21.15.101:0/148479672 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560886e8bf30 con 0x560886f40350 2023-07-18T03:12:56.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.958+0000 7f2dcdffb700 1 -- 172.21.15.101:0/148479672 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2de00b82b0 con 0x560886f40350 2023-07-18T03:12:56.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.958+0000 7f2e00064b80 10 client.26719 fetch_fsmap learned FSMap version 26 2023-07-18T03:12:56.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.958+0000 7f2e00064b80 1 -- 172.21.15.101:0/148479672 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f2dd4010c20 con 0x560886f40350 2023-07-18T03:12:56.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.958+0000 7f2dcdffb700 1 -- 172.21.15.101:0/148479672 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2de00ca080 con 0x560886f40350 2023-07-18T03:12:56.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.958+0000 7f2dcdffb700 1 -- 172.21.15.101:0/148479672 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2de0098e70 con 0x560886f40350 2023-07-18T03:12:56.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.958+0000 7f2e00064b80 10 client.26719 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:12:56.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.958+0000 7f2e00064b80 10 client.26719 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:12:56.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.958+0000 7f2e00064b80 20 client.26719 populate_metadata read hostname 'smithi101' 2023-07-18T03:12:56.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.958+0000 7f2e00064b80 1 --2- 172.21.15.101:0/148479672 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x560886f5b3f0 0x560886f5d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:56.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.958+0000 7f2e00064b80 4 client.26719 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:12:56.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.958+0000 7f2e00064b80 1 -- 172.21.15.101:0/148479672 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560886e4e0d0 con 0x560886f5b3f0 2023-07-18T03:12:56.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.958+0000 7f2de9d9a700 1 --2- 172.21.15.101:0/148479672 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x560886f5b3f0 0x560886f5d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:56.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.959+0000 7f2de9d9a700 1 --2- 172.21.15.101:0/148479672 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x560886f5b3f0 0x560886f5d800 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b0bed574beb4c8 server_cookie=4bb4c3b56267c738 in_seq=0 out_seq=0 2023-07-18T03:12:56.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.959+0000 7f2dcdffb700 10 client.26719 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:12:56.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.961+0000 7f2dcdffb700 1 -- 172.21.15.101:0/148479672 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560886e4e0d0 con 0x560886f5b3f0 2023-07-18T03:12:56.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.961+0000 7f2dcdffb700 10 client.26719 handle_command_reply: tid=0 2023-07-18T03:12:56.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.961+0000 7f2e00064b80 1 client.26719 shutdown 2023-07-18T03:12:56.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.961+0000 7f2e00064b80 1 -- 172.21.15.101:0/148479672 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560886f40350 msgr2=0x560886f4b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:56.970 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.961+0000 7f2e00064b80 1 --2- 172.21.15.101:0/148479672 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560886f40350 0x560886f4b5c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2de0096410 tx=0x7f2de0099f90 comp rx=0 tx=0).stop 2023-07-18T03:12:56.970 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.961+0000 7f2e00064b80 1 -- 172.21.15.101:0/148479672 shutdown_connections 2023-07-18T03:12:56.970 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.961+0000 7f2e00064b80 1 --2- 172.21.15.101:0/148479672 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x560886f5b3f0 0x560886f5d800 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:56.970 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.961+0000 7f2e00064b80 1 --2- 172.21.15.101:0/148479672 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560886f4bb50 0x560886f48ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:56.970 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.961+0000 7f2e00064b80 1 --2- 172.21.15.101:0/148479672 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560886f40350 0x560886f4b5c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:56.970 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.961+0000 7f2e00064b80 1 --2- 172.21.15.101:0/148479672 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560886f40c60 0x560886f487d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:56.970 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.961+0000 7f2dcdffb700 0 client.26719 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:12:56.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.961+0000 7f2e00064b80 1 -- 172.21.15.101:0/148479672 >> 172.21.15.101:0/148479672 conn(0x560886f36e20 msgr2=0x560886f3c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:56.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.961+0000 7f2e00064b80 1 -- 172.21.15.101:0/148479672 shutdown_connections 2023-07-18T03:12:56.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.961+0000 7f2e00064b80 1 -- 172.21.15.101:0/148479672 wait complete. 2023-07-18T03:12:56.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.961+0000 7f2e00064b80 20 client.26719 trim_cache size 0 max 16384 2023-07-18T03:12:56.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.978+0000 7f2e00064b80 1 Processor -- start 2023-07-18T03:12:56.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.979+0000 7f2e00064b80 1 -- start start 2023-07-18T03:12:56.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.979+0000 7f2e00064b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560886f40350 0x560886f487d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:56.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.979+0000 7f2e00064b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560886f40c60 0x560886f4b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:56.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.979+0000 7f2e00064b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560886f4bb50 0x560886f45ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:56.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.979+0000 7f2e00064b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560886f443e0 con 0x560886f40350 2023-07-18T03:12:56.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.979+0000 7f2e00064b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560886f57390 con 0x560886f4bb50 2023-07-18T03:12:56.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.979+0000 7f2e00064b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x560886f62180 con 0x560886f40c60 2023-07-18T03:12:56.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.979+0000 7f2de9d9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560886f40c60 0x560886f4b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:56.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.979+0000 7f2de9d9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560886f40c60 0x560886f4b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:57682/0 (socket says 172.21.15.101:57682) 2023-07-18T03:12:56.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.979+0000 7f2de9d9a700 1 -- 172.21.15.101:0/720765612 learned_addr learned my addr 172.21.15.101:0/720765612 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:56.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.979+0000 7f2dea59b700 1 --2- 172.21.15.101:0/720765612 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560886f40350 0x560886f487d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:56.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.979+0000 7f2dead9c700 1 --2- 172.21.15.101:0/720765612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560886f4bb50 0x560886f45ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:56.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.979+0000 7f2de9d9a700 1 -- 172.21.15.101:0/720765612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560886f4bb50 msgr2=0x560886f45ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:56.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.979+0000 7f2de9d9a700 1 --2- 172.21.15.101:0/720765612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560886f4bb50 0x560886f45ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:56.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.979+0000 7f2de9d9a700 1 -- 172.21.15.101:0/720765612 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560886f40350 msgr2=0x560886f487d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:56.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.979+0000 7f2de9d9a700 1 --2- 172.21.15.101:0/720765612 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560886f40350 0x560886f487d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:56.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.979+0000 7f2de9d9a700 1 -- 172.21.15.101:0/720765612 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560886ca9ce0 con 0x560886f40c60 2023-07-18T03:12:56.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.980+0000 7f2de9d9a700 1 --2- 172.21.15.101:0/720765612 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560886f40c60 0x560886f4b0f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4026ee0 tx=0x7f2dd4005950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af82740cfd1af3fa server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:56.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.980+0000 7f2dd97fa700 1 -- 172.21.15.101:0/720765612 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd402a040 con 0x560886f40c60 2023-07-18T03:12:56.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.980+0000 7f2dd97fa700 1 -- 172.21.15.101:0/720765612 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2dd4011db0 con 0x560886f40c60 2023-07-18T03:12:56.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.980+0000 7f2dd97fa700 1 -- 172.21.15.101:0/720765612 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd4026670 con 0x560886f40c60 2023-07-18T03:12:56.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.981+0000 7f2e00064b80 1 -- 172.21.15.101:0/720765612 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560886f40c60 msgr2=0x560886f4b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:56.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.981+0000 7f2e00064b80 1 --2- 172.21.15.101:0/720765612 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560886f40c60 0x560886f4b0f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4026ee0 tx=0x7f2dd4005950 comp rx=0 tx=0).stop 2023-07-18T03:12:56.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.981+0000 7f2e00064b80 1 -- 172.21.15.101:0/720765612 shutdown_connections 2023-07-18T03:12:56.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.981+0000 7f2e00064b80 1 --2- 172.21.15.101:0/720765612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560886f4bb50 0x560886f45ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:56.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.981+0000 7f2e00064b80 1 --2- 172.21.15.101:0/720765612 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560886f40350 0x560886f487d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:56.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.981+0000 7f2e00064b80 1 --2- 172.21.15.101:0/720765612 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560886f40c60 0x560886f4b0f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:56.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.981+0000 7f2e00064b80 1 -- 172.21.15.101:0/720765612 >> 172.21.15.101:0/720765612 conn(0x560886f36e20 msgr2=0x560886f480c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:56.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.981+0000 7f2e00064b80 1 -- 172.21.15.101:0/720765612 shutdown_connections 2023-07-18T03:12:56.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.981+0000 7f2e00064b80 1 -- 172.21.15.101:0/720765612 wait complete. 2023-07-18T03:12:56.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.981+0000 7f2e00064b80 1 -- start start 2023-07-18T03:12:56.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.982+0000 7f2e00064b80 1 Processor -- start 2023-07-18T03:12:56.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.982+0000 7f2e00064b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560886f40350 0x560886fa9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:56.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.982+0000 7f2e00064b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560886f40c60 0x560886faa040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:56.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.982+0000 7f2e00064b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560886f4bb50 0x560886f518b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:56.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.982+0000 7f2e00064b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560886f51e90 con 0x560886f40c60 2023-07-18T03:12:56.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.982+0000 7f2e00064b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560886f52000 con 0x560886f4bb50 2023-07-18T03:12:56.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.982+0000 7f2e00064b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x560886f521c0 con 0x560886f40350 2023-07-18T03:12:56.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.982+0000 7f2e00064b80 10 client.0 _ll_register_callbacks cb 0x560886e307e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:12:56.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.982+0000 7f2dea59b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560886f40350 0x560886fa9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:56.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.982+0000 7f2dea59b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560886f40350 0x560886fa9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:57690/0 (socket says 172.21.15.101:57690) 2023-07-18T03:12:56.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.982+0000 7f2dea59b700 1 -- 172.21.15.101:0/3660028746 learned_addr learned my addr 172.21.15.101:0/3660028746 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:56.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.982+0000 7f2de9d9a700 1 --2- 172.21.15.101:0/3660028746 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560886f40c60 0x560886faa040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:56.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.982+0000 7f2dead9c700 1 --2- 172.21.15.101:0/3660028746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560886f4bb50 0x560886f518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:56.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.983+0000 7f2dea59b700 1 -- 172.21.15.101:0/3660028746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560886f4bb50 msgr2=0x560886f518b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:56.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.983+0000 7f2dea59b700 1 --2- 172.21.15.101:0/3660028746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560886f4bb50 0x560886f518b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:56.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.983+0000 7f2dea59b700 1 -- 172.21.15.101:0/3660028746 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560886f40c60 msgr2=0x560886faa040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:56.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.983+0000 7f2dea59b700 1 --2- 172.21.15.101:0/3660028746 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560886f40c60 0x560886faa040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:56.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.983+0000 7f2dea59b700 1 -- 172.21.15.101:0/3660028746 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560886cc0e60 con 0x560886f40350 2023-07-18T03:12:56.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.983+0000 7f2dea59b700 1 --2- 172.21.15.101:0/3660028746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560886f40350 0x560886fa9b00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2de0099200 tx=0x7f2de00ca690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db2ac279bbe41691 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:56.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.983+0000 7f2dcdffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:12:56.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.985+0000 7f2dcdffb700 1 -- 172.21.15.101:0/3660028746 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de00a20e0 con 0x560886f40350 2023-07-18T03:12:56.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.986+0000 7f2dcdffb700 1 -- 172.21.15.101:0/3660028746 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f2de00a2360 con 0x560886f40350 2023-07-18T03:12:56.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.986+0000 7f2e00064b80 1 -- 172.21.15.101:0/3660028746 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560886dec570 con 0x560886f40350 2023-07-18T03:12:56.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.986+0000 7f2dcdffb700 1 -- 172.21.15.101:0/3660028746 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2de0099a50 con 0x560886f40350 2023-07-18T03:12:56.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.986+0000 7f2e00064b80 10 client.26725 fetch_fsmap learned FSMap version 26 2023-07-18T03:12:56.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.986+0000 7f2e00064b80 1 -- 172.21.15.101:0/3660028746 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x560886ce72d0 con 0x560886f40350 2023-07-18T03:12:56.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.986+0000 7f2dcdffb700 1 -- 172.21.15.101:0/3660028746 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560886dec570 con 0x560886f40350 2023-07-18T03:12:56.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.986+0000 7f2dcdffb700 1 -- 172.21.15.101:0/3660028746 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2de000ca10 con 0x560886f40350 2023-07-18T03:12:56.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.986+0000 7f2e00064b80 10 client.26725 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:12:56.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.986+0000 7f2e00064b80 10 client.26725 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:12:56.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.986+0000 7f2e00064b80 20 client.26725 populate_metadata read hostname 'smithi101' 2023-07-18T03:12:56.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.986+0000 7f2e00064b80 1 --2- 172.21.15.101:0/3660028746 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x560886fb14c0 0x560886fb38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:56.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.986+0000 7f2e00064b80 4 client.26725 mds_command: new command op to 24562 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-18T03:12:56.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.986+0000 7f2e00064b80 1 -- 172.21.15.101:0/3660028746 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x560886ce1ff0 con 0x560886fb14c0 2023-07-18T03:12:56.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.986+0000 7f2de9d9a700 1 --2- 172.21.15.101:0/3660028746 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x560886fb14c0 0x560886fb38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:56.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.987+0000 7f2dcdffb700 10 client.26725 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:12:56.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.987+0000 7f2de9d9a700 1 --2- 172.21.15.101:0/3660028746 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x560886fb14c0 0x560886fb38a0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9de74ec10255c10 server_cookie=5119e5fe322ecf35 in_seq=0 out_seq=0 2023-07-18T03:12:56.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.989+0000 7f2dcdffb700 1 -- 172.21.15.101:0/3660028746 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x560886ce1ff0 con 0x560886fb14c0 2023-07-18T03:12:56.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.989+0000 7f2dcdffb700 10 client.26725 handle_command_reply: tid=0 2023-07-18T03:12:56.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.989+0000 7f2e00064b80 1 client.26725 shutdown 2023-07-18T03:12:56.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.989+0000 7f2e00064b80 1 -- 172.21.15.101:0/3660028746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560886f40350 msgr2=0x560886fa9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:56.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.989+0000 7f2e00064b80 1 --2- 172.21.15.101:0/3660028746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560886f40350 0x560886fa9b00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2de0099200 tx=0x7f2de00ca690 comp rx=0 tx=0).stop 2023-07-18T03:12:56.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.989+0000 7f2e00064b80 1 -- 172.21.15.101:0/3660028746 shutdown_connections 2023-07-18T03:12:56.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.989+0000 7f2e00064b80 1 --2- 172.21.15.101:0/3660028746 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x560886fb14c0 0x560886fb38a0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:56.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.989+0000 7f2e00064b80 1 --2- 172.21.15.101:0/3660028746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560886f4bb50 0x560886f518b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:56.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.989+0000 7f2e00064b80 1 --2- 172.21.15.101:0/3660028746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560886f40350 0x560886fa9b00 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:56.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.989+0000 7f2dcdffb700 0 client.26725 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:12:56.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.989+0000 7f2e00064b80 1 --2- 172.21.15.101:0/3660028746 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560886f40c60 0x560886faa040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:56.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.989+0000 7f2e00064b80 1 -- 172.21.15.101:0/3660028746 >> 172.21.15.101:0/3660028746 conn(0x560886f36e20 msgr2=0x560886f42cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:56.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.989+0000 7f2e00064b80 1 -- 172.21.15.101:0/3660028746 shutdown_connections 2023-07-18T03:12:56.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.989+0000 7f2e00064b80 1 -- 172.21.15.101:0/3660028746 wait complete. 2023-07-18T03:12:56.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.989+0000 7f2e00064b80 20 client.26725 trim_cache size 0 max 16384 2023-07-18T03:12:56.989 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:12:56.990 INFO:teuthology.orchestra.run.smithi101.stdout: "return_code": 0, 2023-07-18T03:12:56.990 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_tag": "9aa45bb6-e7f1-479a-b1f7-f457fe746e04", 2023-07-18T03:12:56.990 INFO:teuthology.orchestra.run.smithi101.stdout: "mode": "asynchronous" 2023-07-18T03:12:56.991 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:12:56.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.990+0000 7f2dec7ff700 1 -- 172.21.15.101:0/5373935 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2dd00a1340 msgr2=0x7f2dd00a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:56.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.990+0000 7f2dec7ff700 1 --2- 172.21.15.101:0/5373935 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2dd00a1340 0x7f2dd00a37f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2de4074000 tx=0x7f2dd400b410 comp rx=0 tx=0).stop 2023-07-18T03:12:56.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.990+0000 7f2dec7ff700 1 -- 172.21.15.101:0/5373935 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2de40ffad0 msgr2=0x7f2de4072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:56.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.990+0000 7f2dec7ff700 1 --2- 172.21.15.101:0/5373935 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2de40ffad0 0x7f2de4072810 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2de000e9a0 tx=0x7f2de000ecb0 comp rx=0 tx=0).stop 2023-07-18T03:12:56.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.990+0000 7f2dec7ff700 1 -- 172.21.15.101:0/5373935 shutdown_connections 2023-07-18T03:12:56.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.990+0000 7f2dec7ff700 1 --2- 172.21.15.101:0/5373935 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2de4105ae0 0x7f2de41a3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:56.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.990+0000 7f2dec7ff700 1 --2- 172.21.15.101:0/5373935 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2de40ffad0 0x7f2de4072810 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:56.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.990+0000 7f2dec7ff700 1 --2- 172.21.15.101:0/5373935 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2dd00a1340 0x7f2dd00a37f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:56.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.990+0000 7f2dec7ff700 1 --2- 172.21.15.101:0/5373935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2de4100480 0x7f2de4072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:56.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.990+0000 7f2dec7ff700 1 -- 172.21.15.101:0/5373935 >> 172.21.15.101:0/5373935 conn(0x7f2de40fb5f0 msgr2=0x7f2de40fbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:56.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.990+0000 7f2dec7ff700 1 -- 172.21.15.101:0/5373935 shutdown_connections 2023-07-18T03:12:56.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:56.990+0000 7f2dec7ff700 1 -- 172.21.15.101:0/5373935 wait complete. 2023-07-18T03:12:57.002 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:12:57.067 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:56 smithi101 ceph-mon[126379]: pgmap v164: 129 pgs: 129 active+clean; 253 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.5 MiB/s wr, 290 op/s 2023-07-18T03:12:57.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.229+0000 7fbcdf5ff700 1 -- 172.21.15.101:0/2936773962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcd80ffad0 msgr2=0x7fbcd80fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:57.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.229+0000 7fbcdf5ff700 1 --2- 172.21.15.101:0/2936773962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcd80ffad0 0x7fbcd80fff40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8009990 tx=0x7fbcc8009ca0 comp rx=0 tx=0).stop 2023-07-18T03:12:57.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.229+0000 7fbcdf5ff700 1 -- 172.21.15.101:0/2936773962 shutdown_connections 2023-07-18T03:12:57.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.229+0000 7fbcdf5ff700 1 --2- 172.21.15.101:0/2936773962 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbcd8100480 0x7fbcd810c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:57.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.229+0000 7fbcdf5ff700 1 --2- 172.21.15.101:0/2936773962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbcd8105ae0 0x7fbcd8105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:57.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.229+0000 7fbcdf5ff700 1 --2- 172.21.15.101:0/2936773962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcd80ffad0 0x7fbcd80fff40 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:57.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.229+0000 7fbcdf5ff700 1 -- 172.21.15.101:0/2936773962 >> 172.21.15.101:0/2936773962 conn(0x7fbcd80fb5f0 msgr2=0x7fbcd80fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:57.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.229+0000 7fbcdf5ff700 1 -- 172.21.15.101:0/2936773962 shutdown_connections 2023-07-18T03:12:57.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.229+0000 7fbcdf5ff700 1 -- 172.21.15.101:0/2936773962 wait complete. 2023-07-18T03:12:57.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.230+0000 7fbcdf5ff700 1 Processor -- start 2023-07-18T03:12:57.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.230+0000 7fbcdf5ff700 1 -- start start 2023-07-18T03:12:57.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.230+0000 7fbcdf5ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbcd80ffad0 0x7fbcd8197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:57.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.231+0000 7fbcdf5ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbcd8100480 0x7fbcd8198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:57.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.231+0000 7fbcdf5ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcd8105ae0 0x7fbcd819c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:57.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.231+0000 7fbcdf5ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd819ce00 con 0x7fbcd80ffad0 2023-07-18T03:12:57.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.231+0000 7fbcdf5ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd819cf70 con 0x7fbcd8105ae0 2023-07-18T03:12:57.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.231+0000 7fbcdf5ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd819d130 con 0x7fbcd8100480 2023-07-18T03:12:57.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.231+0000 7fbcdcb9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbcd8100480 0x7fbcd8198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:57.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.231+0000 7fbcdd39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbcd80ffad0 0x7fbcd8197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:57.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.231+0000 7fbcdcb9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbcd8100480 0x7fbcd8198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:57712/0 (socket says 172.21.15.101:57712) 2023-07-18T03:12:57.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.231+0000 7fbcdcb9a700 1 -- 172.21.15.101:0/1034565319 learned_addr learned my addr 172.21.15.101:0/1034565319 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:57.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.231+0000 7fbcddb9c700 1 --2- 172.21.15.101:0/1034565319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcd8105ae0 0x7fbcd819c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:57.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.231+0000 7fbcdcb9a700 1 -- 172.21.15.101:0/1034565319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcd8105ae0 msgr2=0x7fbcd819c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:57.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.231+0000 7fbcdcb9a700 1 --2- 172.21.15.101:0/1034565319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcd8105ae0 0x7fbcd819c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:57.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.231+0000 7fbcdcb9a700 1 -- 172.21.15.101:0/1034565319 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbcd80ffad0 msgr2=0x7fbcd8197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:57.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.231+0000 7fbcdcb9a700 1 --2- 172.21.15.101:0/1034565319 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbcd80ffad0 0x7fbcd8197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:57.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.231+0000 7fbcdcb9a700 1 -- 172.21.15.101:0/1034565319 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc80095f0 con 0x7fbcd8100480 2023-07-18T03:12:57.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.231+0000 7fbcdcb9a700 1 --2- 172.21.15.101:0/1034565319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbcd8100480 0x7fbcd8198360 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8005910 tx=0x7fbcc8005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:57.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.232+0000 7fbcce7fc700 1 -- 172.21.15.101:0/1034565319 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc801d070 con 0x7fbcd8100480 2023-07-18T03:12:57.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.232+0000 7fbcdf5ff700 1 -- 172.21.15.101:0/1034565319 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcd819d3b0 con 0x7fbcd8100480 2023-07-18T03:12:57.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.232+0000 7fbcce7fc700 1 -- 172.21.15.101:0/1034565319 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbcc8022470 con 0x7fbcd8100480 2023-07-18T03:12:57.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.232+0000 7fbcce7fc700 1 -- 172.21.15.101:0/1034565319 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8017650 con 0x7fbcd8100480 2023-07-18T03:12:57.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.232+0000 7fbcdf5ff700 1 -- 172.21.15.101:0/1034565319 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcd81a4c50 con 0x7fbcd8100480 2023-07-18T03:12:57.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.235+0000 7fbcce7fc700 1 -- 172.21.15.101:0/1034565319 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fbcc800fad0 con 0x7fbcd8100480 2023-07-18T03:12:57.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.236+0000 7fbcf2f82b80 1 Processor -- start 2023-07-18T03:12:57.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.236+0000 7fbcce7fc700 1 --2- 172.21.15.101:0/1034565319 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fbcc40a12c0 0x7fbcc40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:57.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.237+0000 7fbcf2f82b80 1 -- start start 2023-07-18T03:12:57.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.237+0000 7fbcce7fc700 1 -- 172.21.15.101:0/1034565319 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fbcc80ce4c0 con 0x7fbcd8100480 2023-07-18T03:12:57.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.237+0000 7fbcf2f82b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55945ca9c2d0 0x55945ca9c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:57.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.237+0000 7fbcf2f82b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55945ca9cbe0 0x55945caa7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:57.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.237+0000 7fbcf2f82b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55945caa7ad0 0x55945caa9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:57.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.237+0000 7fbcf2f82b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55945c9a2cb0 con 0x55945ca9cbe0 2023-07-18T03:12:57.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.237+0000 7fbcf2f82b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55945c9f5a40 con 0x55945ca9c2d0 2023-07-18T03:12:57.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.237+0000 7fbcf2f82b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55945c9eeed0 con 0x55945caa7ad0 2023-07-18T03:12:57.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.237+0000 7fbcddb9c700 1 --2- 172.21.15.101:0/1034565319 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fbcc40a12c0 0x7fbcc40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:57.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.237+0000 7fbcdd39b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55945ca9c2d0 0x55945ca9c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:57.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.237+0000 7fbcdd39b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55945ca9c2d0 0x55945ca9c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:42850/0 (socket says 172.21.15.101:42850) 2023-07-18T03:12:57.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.237+0000 7fbcdd39b700 1 -- 172.21.15.101:0/1366439531 learned_addr learned my addr 172.21.15.101:0/1366439531 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:57.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.237+0000 7fbcdcb9a700 1 --2- 172.21.15.101:0/1366439531 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55945ca9cbe0 0x55945caa7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:57.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.237+0000 7fbcddb9c700 1 --2- 172.21.15.101:0/1366439531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55945caa7ad0 0x55945caa9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:57.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.237+0000 7fbcdd39b700 1 -- 172.21.15.101:0/1366439531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55945caa7ad0 msgr2=0x55945caa9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:57.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.237+0000 7fbcdd39b700 1 --2- 172.21.15.101:0/1366439531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55945caa7ad0 0x55945caa9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:57.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.237+0000 7fbcdd39b700 1 -- 172.21.15.101:0/1366439531 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55945ca9cbe0 msgr2=0x55945caa7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:57.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.237+0000 7fbcdd39b700 1 --2- 172.21.15.101:0/1366439531 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55945ca9cbe0 0x55945caa7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:57.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.238+0000 7fbcdd39b700 1 -- 172.21.15.101:0/1366439531 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55945c81cde0 con 0x55945ca9c2d0 2023-07-18T03:12:57.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.238+0000 7fbcddb9c700 1 --2- 172.21.15.101:0/1034565319 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fbcc40a12c0 0x7fbcc40a3770 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0009c90 tx=0x7fbcd0004320 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:57.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.238+0000 7fbcdd39b700 1 --2- 172.21.15.101:0/1366439531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55945ca9c2d0 0x55945ca9c6a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4009d00 tx=0x7fbcd4011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f16c1f0a1d2c435 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:57.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.238+0000 7fbcc37fe700 1 -- 172.21.15.101:0/1366439531 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4019630 con 0x55945ca9c2d0 2023-07-18T03:12:57.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.238+0000 7fbcc37fe700 1 -- 172.21.15.101:0/1366439531 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbcd4019c50 con 0x55945ca9c2d0 2023-07-18T03:12:57.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.238+0000 7fbcc37fe700 1 -- 172.21.15.101:0/1366439531 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd400eb90 con 0x55945ca9c2d0 2023-07-18T03:12:57.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.239+0000 7fbcf2f82b80 1 -- 172.21.15.101:0/1366439531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55945ca9c2d0 msgr2=0x55945ca9c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:57.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.239+0000 7fbcf2f82b80 1 --2- 172.21.15.101:0/1366439531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55945ca9c2d0 0x55945ca9c6a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4009d00 tx=0x7fbcd4011720 comp rx=0 tx=0).stop 2023-07-18T03:12:57.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.239+0000 7fbcf2f82b80 1 -- 172.21.15.101:0/1366439531 shutdown_connections 2023-07-18T03:12:57.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.239+0000 7fbcf2f82b80 1 --2- 172.21.15.101:0/1366439531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55945ca9c2d0 0x55945ca9c6a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:57.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.239+0000 7fbcf2f82b80 1 --2- 172.21.15.101:0/1366439531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55945caa7ad0 0x55945caa9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:57.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.239+0000 7fbcf2f82b80 1 --2- 172.21.15.101:0/1366439531 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55945ca9cbe0 0x55945caa7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:57.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.239+0000 7fbcf2f82b80 1 -- 172.21.15.101:0/1366439531 >> 172.21.15.101:0/1366439531 conn(0x55945ca92da0 msgr2=0x55945ca986a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:57.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.239+0000 7fbcf2f82b80 1 -- 172.21.15.101:0/1366439531 shutdown_connections 2023-07-18T03:12:57.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.239+0000 7fbcf2f82b80 1 -- 172.21.15.101:0/1366439531 wait complete. 2023-07-18T03:12:57.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.240+0000 7fbcf2f82b80 1 -- start start 2023-07-18T03:12:57.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.240+0000 7fbcf2f82b80 1 Processor -- start 2023-07-18T03:12:57.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.240+0000 7fbcf2f82b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55945ca9c2d0 0x55945caa7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:57.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.240+0000 7fbcf2f82b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55945ca9cbe0 0x55945caa4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:57.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.241+0000 7fbcf2f82b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55945caa7ad0 0x55945caa4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:57.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.241+0000 7fbcf2f82b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55945c98eac0 con 0x55945caa7ad0 2023-07-18T03:12:57.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.241+0000 7fbcf2f82b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55945c98e7c0 con 0x55945ca9c2d0 2023-07-18T03:12:57.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.241+0000 7fbcf2f82b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55945c872920 con 0x55945ca9cbe0 2023-07-18T03:12:57.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.241+0000 7fbcf2f82b80 10 client.0 _ll_register_callbacks cb 0x55945c98c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:12:57.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.241+0000 7fbcdcb9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55945ca9cbe0 0x55945caa4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:57.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.241+0000 7fbcdd39b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55945ca9c2d0 0x55945caa7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:57.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.241+0000 7fbcdcb9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55945ca9cbe0 0x55945caa4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:57734/0 (socket says 172.21.15.101:57734) 2023-07-18T03:12:57.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.241+0000 7fbcddb9c700 1 --2- 172.21.15.101:0/4200726461 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55945caa7ad0 0x55945caa4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:57.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.241+0000 7fbcdcb9a700 1 -- 172.21.15.101:0/4200726461 learned_addr learned my addr 172.21.15.101:0/4200726461 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:57.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.241+0000 7fbcdd39b700 1 -- 172.21.15.101:0/4200726461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55945ca9cbe0 msgr2=0x55945caa4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:57.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.241+0000 7fbcdd39b700 1 --2- 172.21.15.101:0/4200726461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55945ca9cbe0 0x55945caa4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:57.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.241+0000 7fbcdd39b700 1 -- 172.21.15.101:0/4200726461 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55945caa7ad0 msgr2=0x55945caa4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:57.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.241+0000 7fbcdd39b700 1 --2- 172.21.15.101:0/4200726461 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55945caa7ad0 0x55945caa4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:57.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.241+0000 7fbcdd39b700 1 -- 172.21.15.101:0/4200726461 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcd4009960 con 0x55945ca9c2d0 2023-07-18T03:12:57.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.242+0000 7fbcdd39b700 1 --2- 172.21.15.101:0/4200726461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55945ca9c2d0 0x55945caa7550 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4000c00 tx=0x7fbcd40057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a766d048004fc5b7 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:57.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.242+0000 7fbcc0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:12:57.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.242+0000 7fbcc0ff9700 1 -- 172.21.15.101:0/4200726461 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4005440 con 0x55945ca9c2d0 2023-07-18T03:12:57.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.242+0000 7fbcf2f82b80 1 -- 172.21.15.101:0/4200726461 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55945c9484f0 con 0x55945ca9c2d0 2023-07-18T03:12:57.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.242+0000 7fbcc0ff9700 1 -- 172.21.15.101:0/4200726461 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fbcd4005ce0 con 0x55945ca9c2d0 2023-07-18T03:12:57.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.242+0000 7fbcc0ff9700 1 -- 172.21.15.101:0/4200726461 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbcd40055a0 con 0x55945ca9c2d0 2023-07-18T03:12:57.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.242+0000 7fbcf2f82b80 10 client.26708 fetch_fsmap learned FSMap version 26 2023-07-18T03:12:57.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.242+0000 7fbcf2f82b80 1 -- 172.21.15.101:0/4200726461 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55945c805c60 con 0x55945ca9c2d0 2023-07-18T03:12:57.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.242+0000 7fbcc0ff9700 1 -- 172.21.15.101:0/4200726461 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55945c9484f0 con 0x55945ca9c2d0 2023-07-18T03:12:57.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.242+0000 7fbcc0ff9700 1 -- 172.21.15.101:0/4200726461 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbcd4004b20 con 0x55945ca9c2d0 2023-07-18T03:12:57.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.242+0000 7fbcf2f82b80 10 client.26708 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:12:57.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.243+0000 7fbcf2f82b80 10 client.26708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:12:57.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.243+0000 7fbcf2f82b80 20 client.26708 populate_metadata read hostname 'smithi101' 2023-07-18T03:12:57.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.243+0000 7fbcf2f82b80 1 --2- 172.21.15.101:0/4200726461 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55945cab72c0 0x55945cab96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:57.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.243+0000 7fbcf2f82b80 4 client.26708 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:12:57.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.243+0000 7fbcf2f82b80 1 -- 172.21.15.101:0/4200726461 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55945c9e7eb0 con 0x55945cab72c0 2023-07-18T03:12:57.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.243+0000 7fbcdcb9a700 1 --2- 172.21.15.101:0/4200726461 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55945cab72c0 0x55945cab96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:57.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.243+0000 7fbcc0ff9700 10 client.26708 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:12:57.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.243+0000 7fbcdcb9a700 1 --2- 172.21.15.101:0/4200726461 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55945cab72c0 0x55945cab96d0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1595bb57dc26873b server_cookie=cc3c30add6550ed in_seq=0 out_seq=0 2023-07-18T03:12:57.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.244+0000 7fbcc0ff9700 1 -- 172.21.15.101:0/4200726461 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55945c9e7eb0 con 0x55945cab72c0 2023-07-18T03:12:57.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.244+0000 7fbcc0ff9700 10 client.26708 handle_command_reply: tid=0 2023-07-18T03:12:57.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.245+0000 7fbcf2f82b80 1 client.26708 shutdown 2023-07-18T03:12:57.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.245+0000 7fbcf2f82b80 1 -- 172.21.15.101:0/4200726461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55945ca9c2d0 msgr2=0x55945caa7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:57.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.245+0000 7fbcf2f82b80 1 --2- 172.21.15.101:0/4200726461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55945ca9c2d0 0x55945caa7550 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4000c00 tx=0x7fbcd40057c0 comp rx=0 tx=0).stop 2023-07-18T03:12:57.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.245+0000 7fbcf2f82b80 1 -- 172.21.15.101:0/4200726461 shutdown_connections 2023-07-18T03:12:57.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.245+0000 7fbcf2f82b80 1 --2- 172.21.15.101:0/4200726461 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55945cab72c0 0x55945cab96d0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:57.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.245+0000 7fbcf2f82b80 1 --2- 172.21.15.101:0/4200726461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55945ca9c2d0 0x55945caa7550 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:57.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.245+0000 7fbcf2f82b80 1 --2- 172.21.15.101:0/4200726461 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55945caa7ad0 0x55945caa4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:57.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.245+0000 7fbcc0ff9700 0 client.26708 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:12:57.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.245+0000 7fbcf2f82b80 1 --2- 172.21.15.101:0/4200726461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55945ca9cbe0 0x55945caa4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:57.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.245+0000 7fbcf2f82b80 1 -- 172.21.15.101:0/4200726461 >> 172.21.15.101:0/4200726461 conn(0x55945ca92da0 msgr2=0x55945ca98230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:57.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.245+0000 7fbcf2f82b80 1 -- 172.21.15.101:0/4200726461 shutdown_connections 2023-07-18T03:12:57.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.245+0000 7fbcf2f82b80 1 -- 172.21.15.101:0/4200726461 wait complete. 2023-07-18T03:12:57.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.245+0000 7fbcf2f82b80 20 client.26708 trim_cache size 0 max 16384 2023-07-18T03:12:57.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.262+0000 7fbcf2f82b80 1 Processor -- start 2023-07-18T03:12:57.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.262+0000 7fbcf2f82b80 1 -- start start 2023-07-18T03:12:57.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.263+0000 7fbcf2f82b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55945ca9c2d0 0x55945ca9f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:57.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.263+0000 7fbcf2f82b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55945ca9cbe0 0x55945caa7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:57.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.263+0000 7fbcf2f82b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55945caa7ad0 0x55945caa4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:57.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.263+0000 7fbcf2f82b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55945ca9d580 con 0x55945caa7ad0 2023-07-18T03:12:57.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.263+0000 7fbcf2f82b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55945caa5450 con 0x55945ca9c2d0 2023-07-18T03:12:57.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.263+0000 7fbcf2f82b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55945caa3cd0 con 0x55945ca9cbe0 2023-07-18T03:12:57.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.263+0000 7fbcdcb9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55945ca9cbe0 0x55945caa7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:57.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.263+0000 7fbcddb9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55945caa7ad0 0x55945caa4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:57.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.263+0000 7fbcdcb9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55945ca9cbe0 0x55945caa7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:57748/0 (socket says 172.21.15.101:57748) 2023-07-18T03:12:57.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.263+0000 7fbcdcb9a700 1 -- 172.21.15.101:0/2087921268 learned_addr learned my addr 172.21.15.101:0/2087921268 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:57.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.263+0000 7fbcdd39b700 1 --2- 172.21.15.101:0/2087921268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55945ca9c2d0 0x55945ca9f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:57.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.263+0000 7fbcdcb9a700 1 -- 172.21.15.101:0/2087921268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55945ca9c2d0 msgr2=0x55945ca9f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:57.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.263+0000 7fbcdcb9a700 1 --2- 172.21.15.101:0/2087921268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55945ca9c2d0 0x55945ca9f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:57.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.263+0000 7fbcdcb9a700 1 -- 172.21.15.101:0/2087921268 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55945caa7ad0 msgr2=0x55945caa4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:57.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.264+0000 7fbcdcb9a700 1 --2- 172.21.15.101:0/2087921268 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55945caa7ad0 0x55945caa4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:57.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.264+0000 7fbcdcb9a700 1 -- 172.21.15.101:0/2087921268 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55945c843250 con 0x55945ca9cbe0 2023-07-18T03:12:57.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.264+0000 7fbcdcb9a700 1 --2- 172.21.15.101:0/2087921268 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55945ca9cbe0 0x55945caa7080 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbcc80cbec0 tx=0x7fbcc801a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f550a3012cbcb0b4 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:57.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.264+0000 7fbcc3fff700 1 -- 172.21.15.101:0/2087921268 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc809f120 con 0x55945ca9cbe0 2023-07-18T03:12:57.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.264+0000 7fbcc3fff700 1 -- 172.21.15.101:0/2087921268 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbcc809b2d0 con 0x55945ca9cbe0 2023-07-18T03:12:57.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.264+0000 7fbcc3fff700 1 -- 172.21.15.101:0/2087921268 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8099700 con 0x55945ca9cbe0 2023-07-18T03:12:57.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.265+0000 7fbcf2f82b80 1 -- 172.21.15.101:0/2087921268 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55945ca9cbe0 msgr2=0x55945caa7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:57.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.265+0000 7fbcf2f82b80 1 --2- 172.21.15.101:0/2087921268 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55945ca9cbe0 0x55945caa7080 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbcc80cbec0 tx=0x7fbcc801a040 comp rx=0 tx=0).stop 2023-07-18T03:12:57.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.265+0000 7fbcf2f82b80 1 -- 172.21.15.101:0/2087921268 shutdown_connections 2023-07-18T03:12:57.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.265+0000 7fbcf2f82b80 1 --2- 172.21.15.101:0/2087921268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55945ca9c2d0 0x55945ca9f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:57.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.265+0000 7fbcf2f82b80 1 --2- 172.21.15.101:0/2087921268 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55945caa7ad0 0x55945caa4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:57.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.265+0000 7fbcf2f82b80 1 --2- 172.21.15.101:0/2087921268 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55945ca9cbe0 0x55945caa7080 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:57.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.265+0000 7fbcf2f82b80 1 -- 172.21.15.101:0/2087921268 >> 172.21.15.101:0/2087921268 conn(0x55945ca92da0 msgr2=0x55945ca9fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:57.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.265+0000 7fbcf2f82b80 1 -- 172.21.15.101:0/2087921268 shutdown_connections 2023-07-18T03:12:57.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.265+0000 7fbcf2f82b80 1 -- 172.21.15.101:0/2087921268 wait complete. 2023-07-18T03:12:57.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.265+0000 7fbcf2f82b80 1 -- start start 2023-07-18T03:12:57.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.265+0000 7fbcf2f82b80 1 Processor -- start 2023-07-18T03:12:57.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.266+0000 7fbcf2f82b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55945ca9c2d0 0x55945caac6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:57.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.266+0000 7fbcf2f82b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55945ca9cbe0 0x55945caacbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:57.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.266+0000 7fbcf2f82b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55945caa7ad0 0x55945cb0c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:57.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.266+0000 7fbcf2f82b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55945cb0ce60 con 0x55945ca9cbe0 2023-07-18T03:12:57.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.266+0000 7fbcf2f82b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55945cb0cfd0 con 0x55945caa7ad0 2023-07-18T03:12:57.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.266+0000 7fbcf2f82b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55945cb0d190 con 0x55945ca9c2d0 2023-07-18T03:12:57.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.266+0000 7fbcf2f82b80 10 client.0 _ll_register_callbacks cb 0x7fbcc8005c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:12:57.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.266+0000 7fbcdd39b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55945ca9c2d0 0x55945caac6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:57.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.266+0000 7fbcdd39b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55945ca9c2d0 0x55945caac6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:57756/0 (socket says 172.21.15.101:57756) 2023-07-18T03:12:57.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.266+0000 7fbcdd39b700 1 -- 172.21.15.101:0/2484859215 learned_addr learned my addr 172.21.15.101:0/2484859215 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:12:57.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.266+0000 7fbcdcb9a700 1 --2- 172.21.15.101:0/2484859215 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55945ca9cbe0 0x55945caacbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:57.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.266+0000 7fbcddb9c700 1 --2- 172.21.15.101:0/2484859215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55945caa7ad0 0x55945cb0c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:57.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.266+0000 7fbcdd39b700 1 -- 172.21.15.101:0/2484859215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55945caa7ad0 msgr2=0x55945cb0c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:57.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.266+0000 7fbcdd39b700 1 --2- 172.21.15.101:0/2484859215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55945caa7ad0 0x55945cb0c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:57.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.266+0000 7fbcdd39b700 1 -- 172.21.15.101:0/2484859215 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55945ca9cbe0 msgr2=0x55945caacbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:12:57.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.266+0000 7fbcdd39b700 1 --2- 172.21.15.101:0/2484859215 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55945ca9cbe0 0x55945caacbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:57.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.266+0000 7fbcdd39b700 1 -- 172.21.15.101:0/2484859215 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcd81a4c50 con 0x55945ca9c2d0 2023-07-18T03:12:57.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.267+0000 7fbcdd39b700 1 --2- 172.21.15.101:0/2484859215 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55945ca9c2d0 0x55945caac6a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4004440 tx=0x7fbcd4009300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=220196d5a4a58065 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:12:57.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.267+0000 7fbcc0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:12:57.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.267+0000 7fbcc0ff9700 1 -- 172.21.15.101:0/2484859215 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd400e3b0 con 0x55945ca9c2d0 2023-07-18T03:12:57.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.267+0000 7fbcf2f82b80 1 -- 172.21.15.101:0/2484859215 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbcd4019c50 con 0x55945ca9c2d0 2023-07-18T03:12:57.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.267+0000 7fbcc0ff9700 1 -- 172.21.15.101:0/2484859215 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fbcd4000e40 con 0x55945ca9c2d0 2023-07-18T03:12:57.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.267+0000 7fbcc0ff9700 1 -- 172.21.15.101:0/2484859215 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbcd400e890 con 0x55945ca9c2d0 2023-07-18T03:12:57.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.267+0000 7fbcf2f82b80 10 client.26743 fetch_fsmap learned FSMap version 26 2023-07-18T03:12:57.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.267+0000 7fbcf2f82b80 1 -- 172.21.15.101:0/2484859215 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55945c81b2b0 con 0x55945ca9c2d0 2023-07-18T03:12:57.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.267+0000 7fbcc0ff9700 1 -- 172.21.15.101:0/2484859215 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbcd4019c50 con 0x55945ca9c2d0 2023-07-18T03:12:57.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.267+0000 7fbcc0ff9700 1 -- 172.21.15.101:0/2484859215 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbcd4004860 con 0x55945ca9c2d0 2023-07-18T03:12:57.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.268+0000 7fbcf2f82b80 10 client.26743 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:12:57.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.268+0000 7fbcf2f82b80 10 client.26743 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:12:57.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.268+0000 7fbcf2f82b80 20 client.26743 populate_metadata read hostname 'smithi101' 2023-07-18T03:12:57.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.268+0000 7fbcf2f82b80 1 --2- 172.21.15.101:0/2484859215 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55945cb146f0 0x55945cb16b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:12:57.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.268+0000 7fbcf2f82b80 4 client.26743 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:12:57.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.268+0000 7fbcf2f82b80 1 -- 172.21.15.101:0/2484859215 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55945c9aa050 con 0x55945cb146f0 2023-07-18T03:12:57.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.268+0000 7fbcdcb9a700 1 --2- 172.21.15.101:0/2484859215 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55945cb146f0 0x55945cb16b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:12:57.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.268+0000 7fbcdcb9a700 1 --2- 172.21.15.101:0/2484859215 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55945cb146f0 0x55945cb16b00 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1db41a353f795416 server_cookie=9fd9b2209b16d00d in_seq=0 out_seq=0 2023-07-18T03:12:57.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.268+0000 7fbcc0ff9700 10 client.26743 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:12:57.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.269+0000 7fbcc0ff9700 1 -- 172.21.15.101:0/2484859215 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55945c9aa050 con 0x55945cb146f0 2023-07-18T03:12:57.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.269+0000 7fbcc0ff9700 10 client.26743 handle_command_reply: tid=0 2023-07-18T03:12:57.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.269+0000 7fbcf2f82b80 1 client.26743 shutdown 2023-07-18T03:12:57.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.269+0000 7fbcf2f82b80 1 -- 172.21.15.101:0/2484859215 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55945ca9c2d0 msgr2=0x55945caac6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:57.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.269+0000 7fbcf2f82b80 1 --2- 172.21.15.101:0/2484859215 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55945ca9c2d0 0x55945caac6a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4004440 tx=0x7fbcd4009300 comp rx=0 tx=0).stop 2023-07-18T03:12:57.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.269+0000 7fbcf2f82b80 1 -- 172.21.15.101:0/2484859215 shutdown_connections 2023-07-18T03:12:57.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.269+0000 7fbcf2f82b80 1 --2- 172.21.15.101:0/2484859215 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55945cb146f0 0x55945cb16b00 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:57.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.270+0000 7fbcf2f82b80 1 --2- 172.21.15.101:0/2484859215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55945caa7ad0 0x55945cb0c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:57.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.270+0000 7fbcf2f82b80 1 --2- 172.21.15.101:0/2484859215 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55945ca9c2d0 0x55945caac6a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:57.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.270+0000 7fbcf2f82b80 1 --2- 172.21.15.101:0/2484859215 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55945ca9cbe0 0x55945caacbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:57.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.270+0000 7fbcc0ff9700 0 client.26743 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:12:57.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.270+0000 7fbcf2f82b80 1 -- 172.21.15.101:0/2484859215 >> 172.21.15.101:0/2484859215 conn(0x55945ca92da0 msgr2=0x55945caa3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:57.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.270+0000 7fbcf2f82b80 1 -- 172.21.15.101:0/2484859215 shutdown_connections 2023-07-18T03:12:57.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.270+0000 7fbcf2f82b80 1 -- 172.21.15.101:0/2484859215 wait complete. 2023-07-18T03:12:57.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.270+0000 7fbcf2f82b80 20 client.26743 trim_cache size 0 max 16384 2023-07-18T03:12:57.272 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:12:57.272 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:12:57.272 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:12:57.272 INFO:teuthology.orchestra.run.smithi101.stdout: "9aa45bb6-e7f1-479a-b1f7-f457fe746e04": { 2023-07-18T03:12:57.272 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:12:57.273 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "9aa45bb6-e7f1-479a-b1f7-f457fe746e04", 2023-07-18T03:12:57.273 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:12:57.273 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:12:57.273 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:12:57.273 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:12:57.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.270+0000 7fbcdf5ff700 1 -- 172.21.15.101:0/1034565319 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fbcc40a12c0 msgr2=0x7fbcc40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:57.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.270+0000 7fbcdf5ff700 1 --2- 172.21.15.101:0/1034565319 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fbcc40a12c0 0x7fbcc40a3770 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0009c90 tx=0x7fbcd0004320 comp rx=0 tx=0).stop 2023-07-18T03:12:57.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.270+0000 7fbcdf5ff700 1 -- 172.21.15.101:0/1034565319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbcd8100480 msgr2=0x7fbcd8198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:12:57.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.270+0000 7fbcdf5ff700 1 --2- 172.21.15.101:0/1034565319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbcd8100480 0x7fbcd8198360 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8005910 tx=0x7fbcc8005740 comp rx=0 tx=0).stop 2023-07-18T03:12:57.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.270+0000 7fbcdf5ff700 1 -- 172.21.15.101:0/1034565319 shutdown_connections 2023-07-18T03:12:57.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.270+0000 7fbcdf5ff700 1 --2- 172.21.15.101:0/1034565319 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fbcc40a12c0 0x7fbcc40a3770 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:57.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.270+0000 7fbcdf5ff700 1 --2- 172.21.15.101:0/1034565319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcd8105ae0 0x7fbcd819c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:57.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.270+0000 7fbcdf5ff700 1 --2- 172.21.15.101:0/1034565319 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbcd80ffad0 0x7fbcd8197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:57.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.270+0000 7fbcdf5ff700 1 --2- 172.21.15.101:0/1034565319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbcd8100480 0x7fbcd8198360 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:12:57.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.270+0000 7fbcdf5ff700 1 -- 172.21.15.101:0/1034565319 >> 172.21.15.101:0/1034565319 conn(0x7fbcd80fb5f0 msgr2=0x7fbcd80fbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:12:57.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.271+0000 7fbcdf5ff700 1 -- 172.21.15.101:0/1034565319 shutdown_connections 2023-07-18T03:12:57.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:12:57.271+0000 7fbcdf5ff700 1 -- 172.21.15.101:0/1034565319 wait complete. 2023-07-18T03:12:57.282 INFO:tasks.cephfs.filesystem:scrub status for tag:9aa45bb6-e7f1-479a-b1f7-f457fe746e04 - {'path': '/', 'tag': '9aa45bb6-e7f1-479a-b1f7-f457fe746e04', 'options': 'recursive,force'} 2023-07-18T03:12:57.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:56 smithi155 ceph-mon[128885]: pgmap v164: 129 pgs: 129 active+clean; 253 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.5 MiB/s wr, 290 op/s 2023-07-18T03:12:57.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:56 smithi179 ceph-mon[129229]: pgmap v164: 129 pgs: 129 active+clean; 253 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.5 MiB/s wr, 290 op/s 2023-07-18T03:12:57.581 INFO:tasks.workunit.client.0.smithi101.stdout: 1 410726 8.71 MB/sec execute 142 sec latency 26.561 ms 2023-07-18T03:12:58.236 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:57 smithi101 ceph-mon[126379]: scrub queued for path: / 2023-07-18T03:12:58.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:57 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:58.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:57 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:12:58.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:57 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:12:58.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:57 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:58.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:57 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:12:58.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:57 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:12:58.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:57 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:58.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:57 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:12:58.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:57 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:58.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:57 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:58.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:57 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:12:58.239 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:57 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:58.239 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:57 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:12:58.239 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:57 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:58.239 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:57 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:12:58.239 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:57 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:58.239 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:57 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:58.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:57 smithi155 ceph-mon[128885]: scrub queued for path: / 2023-07-18T03:12:58.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:57 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:58.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:57 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:12:58.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:57 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:12:58.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:57 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:58.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:57 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:12:58.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:57 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:12:58.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:57 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:58.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:57 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:12:58.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:57 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:58.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:57 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:58.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:57 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:12:58.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:57 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:58.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:57 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:12:58.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:57 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:58.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:57 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:12:58.298 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:57 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:58.298 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:57 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:58.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:57 smithi179 ceph-mon[129229]: scrub queued for path: / 2023-07-18T03:12:58.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:57 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:58.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:57 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:12:58.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:57 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:12:58.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:57 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:58.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:57 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:12:58.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:57 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:12:58.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:57 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:58.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:57 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:12:58.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:57 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:58.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:57 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:58.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:57 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:12:58.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:57 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:58.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:57 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:12:58.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:57 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:58.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:57 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:12:58.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:57 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:58.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:57 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:12:58.583 INFO:tasks.workunit.client.0.smithi101.stdout: 1 412068 8.70 MB/sec execute 143 sec latency 19.449 ms 2023-07-18T03:12:59.236 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:12:58 smithi101 ceph-mon[126379]: pgmap v165: 129 pgs: 129 active+clean; 253 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 189 op/s 2023-07-18T03:12:59.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:12:58 smithi155 ceph-mon[128885]: pgmap v165: 129 pgs: 129 active+clean; 253 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 189 op/s 2023-07-18T03:12:59.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:12:58 smithi179 ceph-mon[129229]: pgmap v165: 129 pgs: 129 active+clean; 253 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 189 op/s 2023-07-18T03:12:59.581 INFO:tasks.workunit.client.0.smithi101.stdout: 1 413935 8.69 MB/sec execute 144 sec latency 16.955 ms 2023-07-18T03:13:00.581 INFO:tasks.workunit.client.0.smithi101.stdout: 1 415440 8.72 MB/sec execute 145 sec latency 54.985 ms 2023-07-18T03:13:01.236 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:00 smithi101 ceph-mon[126379]: pgmap v166: 129 pgs: 129 active+clean; 258 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 284 op/s 2023-07-18T03:13:01.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:00 smithi155 ceph-mon[128885]: pgmap v166: 129 pgs: 129 active+clean; 258 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 284 op/s 2023-07-18T03:13:01.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:00 smithi179 ceph-mon[129229]: pgmap v166: 129 pgs: 129 active+clean; 258 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 284 op/s 2023-07-18T03:13:01.581 INFO:tasks.workunit.client.0.smithi101.stdout: 1 416058 8.66 MB/sec execute 146 sec latency 89.213 ms 2023-07-18T03:13:02.581 INFO:tasks.workunit.client.0.smithi101.stdout: 1 417338 8.64 MB/sec execute 147 sec latency 53.155 ms 2023-07-18T03:13:03.236 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:02 smithi101 ceph-mon[126379]: pgmap v167: 129 pgs: 129 active+clean; 258 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 190 op/s 2023-07-18T03:13:03.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:02 smithi155 ceph-mon[128885]: pgmap v167: 129 pgs: 129 active+clean; 258 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 190 op/s 2023-07-18T03:13:03.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:02 smithi179 ceph-mon[129229]: pgmap v167: 129 pgs: 129 active+clean; 258 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 190 op/s 2023-07-18T03:13:03.581 INFO:tasks.workunit.client.0.smithi101.stdout: 1 418966 8.67 MB/sec execute 148 sec latency 18.500 ms 2023-07-18T03:13:04.581 INFO:tasks.workunit.client.0.smithi101.stdout: 1 420275 8.64 MB/sec execute 149 sec latency 51.743 ms 2023-07-18T03:13:05.236 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:04 smithi101 ceph-mon[126379]: pgmap v168: 129 pgs: 129 active+clean; 258 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 190 op/s 2023-07-18T03:13:05.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:04 smithi155 ceph-mon[128885]: pgmap v168: 129 pgs: 129 active+clean; 258 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 190 op/s 2023-07-18T03:13:05.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:04 smithi179 ceph-mon[129229]: pgmap v168: 129 pgs: 129 active+clean; 258 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 190 op/s 2023-07-18T03:13:05.582 INFO:tasks.workunit.client.0.smithi101.stdout: 1 422375 8.68 MB/sec execute 150 sec latency 16.813 ms 2023-07-18T03:13:06.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:06 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:13:06.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:06 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:13:06.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:06 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:13:06.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:06 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:13:06.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:06 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:13:06.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:06 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:13:06.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:05 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:13:06.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:05 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:13:06.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:05 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:13:06.582 INFO:tasks.workunit.client.0.smithi101.stdout: 1 423242 8.64 MB/sec execute 151 sec latency 61.331 ms 2023-07-18T03:13:07.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:07 smithi155 ceph-mon[128885]: pgmap v169: 129 pgs: 129 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 307 op/s 2023-07-18T03:13:07.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:07 smithi179 ceph-mon[129229]: pgmap v169: 129 pgs: 129 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 307 op/s 2023-07-18T03:13:07.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:07 smithi101 ceph-mon[126379]: pgmap v169: 129 pgs: 129 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 307 op/s 2023-07-18T03:13:07.582 INFO:tasks.workunit.client.0.smithi101.stdout: 1 425122 8.65 MB/sec execute 152 sec latency 48.784 ms 2023-07-18T03:13:08.582 INFO:tasks.workunit.client.0.smithi101.stdout: 1 426411 8.64 MB/sec execute 153 sec latency 13.949 ms 2023-07-18T03:13:09.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:09 smithi155 ceph-mon[128885]: pgmap v170: 129 pgs: 129 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 211 op/s 2023-07-18T03:13:09.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:09 smithi179 ceph-mon[129229]: pgmap v170: 129 pgs: 129 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 211 op/s 2023-07-18T03:13:09.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:09 smithi101 ceph-mon[126379]: pgmap v170: 129 pgs: 129 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 211 op/s 2023-07-18T03:13:09.582 INFO:tasks.workunit.client.0.smithi101.stdout: 1 428227 8.63 MB/sec execute 154 sec latency 17.471 ms 2023-07-18T03:13:10.582 INFO:tasks.workunit.client.0.smithi101.stdout: 1 429662 8.65 MB/sec execute 155 sec latency 20.931 ms 2023-07-18T03:13:11.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:11 smithi155 ceph-mon[128885]: pgmap v171: 129 pgs: 129 active+clean; 268 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 308 op/s 2023-07-18T03:13:11.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:11 smithi179 ceph-mon[129229]: pgmap v171: 129 pgs: 129 active+clean; 268 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 308 op/s 2023-07-18T03:13:11.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:11 smithi101 ceph-mon[126379]: pgmap v171: 129 pgs: 129 active+clean; 268 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 308 op/s 2023-07-18T03:13:11.582 INFO:tasks.workunit.client.0.smithi101.stdout: 1 430881 8.62 MB/sec execute 156 sec latency 64.853 ms 2023-07-18T03:13:12.582 INFO:tasks.workunit.client.0.smithi101.stdout: 1 432968 8.66 MB/sec execute 157 sec latency 16.255 ms 2023-07-18T03:13:13.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:13 smithi155 ceph-mon[128885]: pgmap v172: 129 pgs: 129 active+clean; 268 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 214 op/s 2023-07-18T03:13:13.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:13 smithi179 ceph-mon[129229]: pgmap v172: 129 pgs: 129 active+clean; 268 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 214 op/s 2023-07-18T03:13:13.486 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:13 smithi101 ceph-mon[126379]: pgmap v172: 129 pgs: 129 active+clean; 268 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 214 op/s 2023-07-18T03:13:13.582 INFO:tasks.workunit.client.0.smithi101.stdout: 1 434045 8.62 MB/sec execute 158 sec latency 14.791 ms 2023-07-18T03:13:14.582 INFO:tasks.workunit.client.0.smithi101.stdout: 1 436134 8.64 MB/sec execute 159 sec latency 15.138 ms 2023-07-18T03:13:15.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:15 smithi155 ceph-mon[128885]: pgmap v173: 129 pgs: 129 active+clean; 268 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 214 op/s 2023-07-18T03:13:15.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:15 smithi179 ceph-mon[129229]: pgmap v173: 129 pgs: 129 active+clean; 268 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 214 op/s 2023-07-18T03:13:15.486 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:15 smithi101 ceph-mon[126379]: pgmap v173: 129 pgs: 129 active+clean; 268 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 214 op/s 2023-07-18T03:13:15.582 INFO:tasks.workunit.client.0.smithi101.stdout: 1 437187 8.62 MB/sec execute 160 sec latency 22.366 ms 2023-07-18T03:13:16.582 INFO:tasks.workunit.client.0.smithi101.stdout: 1 438768 8.61 MB/sec execute 161 sec latency 64.707 ms 2023-07-18T03:13:17.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:17 smithi155 ceph-mon[128885]: pgmap v174: 129 pgs: 129 active+clean; 276 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 314 op/s 2023-07-18T03:13:17.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:17 smithi179 ceph-mon[129229]: pgmap v174: 129 pgs: 129 active+clean; 276 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 314 op/s 2023-07-18T03:13:17.486 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:17 smithi101 ceph-mon[126379]: pgmap v174: 129 pgs: 129 active+clean; 276 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 314 op/s 2023-07-18T03:13:17.582 INFO:tasks.workunit.client.0.smithi101.stdout: 1 440257 8.63 MB/sec execute 162 sec latency 38.208 ms 2023-07-18T03:13:18.582 INFO:tasks.workunit.client.0.smithi101.stdout: 1 441716 8.61 MB/sec execute 163 sec latency 19.012 ms 2023-07-18T03:13:19.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:19 smithi155 ceph-mon[128885]: pgmap v175: 129 pgs: 129 active+clean; 276 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 197 op/s 2023-07-18T03:13:19.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:19 smithi179 ceph-mon[129229]: pgmap v175: 129 pgs: 129 active+clean; 276 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 197 op/s 2023-07-18T03:13:19.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:19 smithi101 ceph-mon[126379]: pgmap v175: 129 pgs: 129 active+clean; 276 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 197 op/s 2023-07-18T03:13:19.583 INFO:tasks.workunit.client.0.smithi101.stdout: 1 443675 8.64 MB/sec execute 164 sec latency 12.461 ms 2023-07-18T03:13:20.583 INFO:tasks.workunit.client.0.smithi101.stdout: 1 444592 8.60 MB/sec execute 165 sec latency 77.043 ms 2023-07-18T03:13:21.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:21 smithi155 ceph-mon[128885]: pgmap v176: 129 pgs: 129 active+clean; 283 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 297 op/s 2023-07-18T03:13:21.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:21 smithi179 ceph-mon[129229]: pgmap v176: 129 pgs: 129 active+clean; 283 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 297 op/s 2023-07-18T03:13:21.486 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:21 smithi101 ceph-mon[126379]: pgmap v176: 129 pgs: 129 active+clean; 283 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 297 op/s 2023-07-18T03:13:21.583 INFO:tasks.workunit.client.0.smithi101.stdout: 1 445986 8.58 MB/sec execute 166 sec latency 248.280 ms 2023-07-18T03:13:22.583 INFO:tasks.workunit.client.0.smithi101.stdout: 1 447416 8.60 MB/sec execute 167 sec latency 25.227 ms 2023-07-18T03:13:23.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:23 smithi155 ceph-mon[128885]: pgmap v177: 129 pgs: 129 active+clean; 283 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 200 op/s 2023-07-18T03:13:23.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:23 smithi179 ceph-mon[129229]: pgmap v177: 129 pgs: 129 active+clean; 283 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 200 op/s 2023-07-18T03:13:23.486 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:23 smithi101 ceph-mon[126379]: pgmap v177: 129 pgs: 129 active+clean; 283 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 200 op/s 2023-07-18T03:13:23.583 INFO:tasks.workunit.client.0.smithi101.stdout: 1 449146 8.58 MB/sec execute 168 sec latency 30.683 ms 2023-07-18T03:13:24.583 INFO:tasks.workunit.client.0.smithi101.stdout: 1 450815 8.61 MB/sec execute 169 sec latency 32.195 ms 2023-07-18T03:13:25.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:25 smithi155 ceph-mon[128885]: pgmap v178: 129 pgs: 129 active+clean; 283 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 200 op/s 2023-07-18T03:13:25.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:25 smithi179 ceph-mon[129229]: pgmap v178: 129 pgs: 129 active+clean; 283 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 200 op/s 2023-07-18T03:13:25.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:25 smithi101 ceph-mon[126379]: pgmap v178: 129 pgs: 129 active+clean; 283 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 200 op/s 2023-07-18T03:13:25.585 INFO:tasks.workunit.client.0.smithi101.stdout: 1 452086 8.59 MB/sec execute 170 sec latency 15.579 ms 2023-07-18T03:13:26.583 INFO:tasks.workunit.client.0.smithi101.stdout: 1 454002 8.61 MB/sec execute 171 sec latency 49.689 ms 2023-07-18T03:13:27.283 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:13:27.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:27 smithi155 ceph-mon[128885]: pgmap v179: 129 pgs: 129 active+clean; 289 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 291 op/s 2023-07-18T03:13:27.345 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:27 smithi101 ceph-mon[126379]: pgmap v179: 129 pgs: 129 active+clean; 289 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 291 op/s 2023-07-18T03:13:27.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:27 smithi179 ceph-mon[129229]: pgmap v179: 129 pgs: 129 active+clean; 289 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 291 op/s 2023-07-18T03:13:27.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.526+0000 7fee5d3ff700 1 -- 172.21.15.101:0/1478269089 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee580ffae0 msgr2=0x7fee580fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:27.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.526+0000 7fee5d3ff700 1 --2- 172.21.15.101:0/1478269089 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee580ffae0 0x7fee580fff50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fee4c009990 tx=0x7fee4c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:13:27.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.526+0000 7fee5d3ff700 1 -- 172.21.15.101:0/1478269089 shutdown_connections 2023-07-18T03:13:27.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.526+0000 7fee5d3ff700 1 --2- 172.21.15.101:0/1478269089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee58100490 0x7fee5810c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:27.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.526+0000 7fee5d3ff700 1 --2- 172.21.15.101:0/1478269089 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fee58105af0 0x7fee58105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:27.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.526+0000 7fee5d3ff700 1 --2- 172.21.15.101:0/1478269089 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee580ffae0 0x7fee580fff50 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:27.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.526+0000 7fee5d3ff700 1 -- 172.21.15.101:0/1478269089 >> 172.21.15.101:0/1478269089 conn(0x7fee580fb5e0 msgr2=0x7fee580fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:13:27.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.526+0000 7fee5d3ff700 1 -- 172.21.15.101:0/1478269089 shutdown_connections 2023-07-18T03:13:27.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.526+0000 7fee5d3ff700 1 -- 172.21.15.101:0/1478269089 wait complete. 2023-07-18T03:13:27.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.527+0000 7fee5d3ff700 1 Processor -- start 2023-07-18T03:13:27.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.527+0000 7fee5d3ff700 1 -- start start 2023-07-18T03:13:27.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.528+0000 7fee5d3ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fee580ffae0 0x7fee5819b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:27.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.528+0000 7fee5d3ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee58100490 0x7fee5819bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:27.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.528+0000 7fee5d3ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee58105af0 0x7fee58195a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:27.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.528+0000 7fee5d3ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee581960b0 con 0x7fee580ffae0 2023-07-18T03:13:27.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.528+0000 7fee5d3ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee58196220 con 0x7fee58100490 2023-07-18T03:13:27.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.528+0000 7fee5d3ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee581963e0 con 0x7fee58105af0 2023-07-18T03:13:27.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.528+0000 7fee567fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee58100490 0x7fee5819bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:27.524 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.528+0000 7fee56ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fee580ffae0 0x7fee5819b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:27.524 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.528+0000 7fee567fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee58100490 0x7fee5819bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:50314/0 (socket says 172.21.15.101:50314) 2023-07-18T03:13:27.524 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.528+0000 7fee567fc700 1 -- 172.21.15.101:0/3808761122 learned_addr learned my addr 172.21.15.101:0/3808761122 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:13:27.524 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.528+0000 7fee577fe700 1 --2- 172.21.15.101:0/3808761122 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee58105af0 0x7fee58195a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:27.524 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.528+0000 7fee567fc700 1 -- 172.21.15.101:0/3808761122 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee58105af0 msgr2=0x7fee58195a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:27.524 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.528+0000 7fee567fc700 1 --2- 172.21.15.101:0/3808761122 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee58105af0 0x7fee58195a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:27.524 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.528+0000 7fee567fc700 1 -- 172.21.15.101:0/3808761122 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fee580ffae0 msgr2=0x7fee5819b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:27.525 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.529+0000 7fee567fc700 1 --2- 172.21.15.101:0/3808761122 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fee580ffae0 0x7fee5819b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:27.525 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.529+0000 7fee567fc700 1 -- 172.21.15.101:0/3808761122 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee50009c40 con 0x7fee58100490 2023-07-18T03:13:27.525 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.529+0000 7fee567fc700 1 --2- 172.21.15.101:0/3808761122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee58100490 0x7fee5819bf10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fee4c005910 tx=0x7fee4c005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:13:27.525 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.529+0000 7fee3ffff700 1 -- 172.21.15.101:0/3808761122 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4c01d070 con 0x7fee58100490 2023-07-18T03:13:27.525 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.529+0000 7fee3ffff700 1 -- 172.21.15.101:0/3808761122 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee4c022470 con 0x7fee58100490 2023-07-18T03:13:27.525 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.529+0000 7fee5d3ff700 1 -- 172.21.15.101:0/3808761122 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee4c0095f0 con 0x7fee58100490 2023-07-18T03:13:27.525 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.529+0000 7fee3ffff700 1 -- 172.21.15.101:0/3808761122 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4c017650 con 0x7fee58100490 2023-07-18T03:13:27.525 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.529+0000 7fee5d3ff700 1 -- 172.21.15.101:0/3808761122 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee581a27e0 con 0x7fee58100490 2023-07-18T03:13:27.526 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.532+0000 7fee3ffff700 1 -- 172.21.15.101:0/3808761122 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fee4c0225e0 con 0x7fee58100490 2023-07-18T03:13:27.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.534+0000 7fee3ffff700 1 --2- 172.21.15.101:0/3808761122 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fee400a1340 0x7fee400a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:27.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.534+0000 7fee70d0eb80 1 Processor -- start 2023-07-18T03:13:27.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.534+0000 7fee3ffff700 1 -- 172.21.15.101:0/3808761122 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fee4c0ce350 con 0x7fee58100490 2023-07-18T03:13:27.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.534+0000 7fee70d0eb80 1 -- start start 2023-07-18T03:13:27.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.534+0000 7fee56ffd700 1 --2- 172.21.15.101:0/3808761122 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fee400a1340 0x7fee400a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:27.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.534+0000 7fee70d0eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d053442d0 0x562d053446a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:27.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.534+0000 7fee70d0eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d05344be0 0x562d0534f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:27.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.535+0000 7fee70d0eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d0534fad0 0x562d05351eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:27.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.535+0000 7fee70d0eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d0524acb0 con 0x562d0534fad0 2023-07-18T03:13:27.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.535+0000 7fee70d0eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d0529da40 con 0x562d053442d0 2023-07-18T03:13:27.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.535+0000 7fee70d0eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d05296ed0 con 0x562d05344be0 2023-07-18T03:13:27.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.535+0000 7fee56ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d053442d0 0x562d053446a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:27.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.535+0000 7fee567fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d05344be0 0x562d0534f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:27.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.535+0000 7fee56ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d053442d0 0x562d053446a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:50330/0 (socket says 172.21.15.101:50330) 2023-07-18T03:13:27.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.535+0000 7fee56ffd700 1 -- 172.21.15.101:0/595834850 learned_addr learned my addr 172.21.15.101:0/595834850 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:13:27.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.535+0000 7fee567fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d05344be0 0x562d0534f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:57106/0 (socket says 172.21.15.101:57106) 2023-07-18T03:13:27.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.535+0000 7fee577fe700 1 --2- 172.21.15.101:0/595834850 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d0534fad0 0x562d05351eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:27.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.535+0000 7fee567fc700 1 -- 172.21.15.101:0/595834850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d053442d0 msgr2=0x562d053446a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:27.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.535+0000 7fee56ffd700 1 --2- 172.21.15.101:0/3808761122 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fee400a1340 0x7fee400a37f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fee48009c30 tx=0x7fee48009260 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:13:27.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.535+0000 7fee567fc700 1 --2- 172.21.15.101:0/595834850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d053442d0 0x562d053446a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:27.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.535+0000 7fee567fc700 1 -- 172.21.15.101:0/595834850 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d0534fad0 msgr2=0x562d05351eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:27.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.535+0000 7fee567fc700 1 --2- 172.21.15.101:0/595834850 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d0534fad0 0x562d05351eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:27.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.535+0000 7fee567fc700 1 -- 172.21.15.101:0/595834850 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d050c4de0 con 0x562d05344be0 2023-07-18T03:13:27.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.535+0000 7fee577fe700 1 --2- 172.21.15.101:0/595834850 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d0534fad0 0x562d05351eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:13:27.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.535+0000 7fee567fc700 1 --2- 172.21.15.101:0/595834850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d05344be0 0x562d0534f590 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fee4c0cb6f0 tx=0x7fee4c009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5333193d32b0de92 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:13:27.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.536+0000 7fee3d7fa700 1 -- 172.21.15.101:0/595834850 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4c09aa60 con 0x562d05344be0 2023-07-18T03:13:27.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.536+0000 7fee3d7fa700 1 -- 172.21.15.101:0/595834850 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee4c09b100 con 0x562d05344be0 2023-07-18T03:13:27.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.536+0000 7fee3d7fa700 1 -- 172.21.15.101:0/595834850 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4c09a5f0 con 0x562d05344be0 2023-07-18T03:13:27.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.536+0000 7fee70d0eb80 1 -- 172.21.15.101:0/595834850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d05344be0 msgr2=0x562d0534f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:27.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.536+0000 7fee70d0eb80 1 --2- 172.21.15.101:0/595834850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d05344be0 0x562d0534f590 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fee4c0cb6f0 tx=0x7fee4c009310 comp rx=0 tx=0).stop 2023-07-18T03:13:27.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.537+0000 7fee70d0eb80 1 -- 172.21.15.101:0/595834850 shutdown_connections 2023-07-18T03:13:27.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.537+0000 7fee70d0eb80 1 --2- 172.21.15.101:0/595834850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d053442d0 0x562d053446a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:27.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.537+0000 7fee70d0eb80 1 --2- 172.21.15.101:0/595834850 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d0534fad0 0x562d05351eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:27.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.537+0000 7fee70d0eb80 1 --2- 172.21.15.101:0/595834850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d05344be0 0x562d0534f590 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:27.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.537+0000 7fee70d0eb80 1 -- 172.21.15.101:0/595834850 >> 172.21.15.101:0/595834850 conn(0x562d0533ada0 msgr2=0x562d053406a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:13:27.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.537+0000 7fee70d0eb80 1 -- 172.21.15.101:0/595834850 shutdown_connections 2023-07-18T03:13:27.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.537+0000 7fee70d0eb80 1 -- 172.21.15.101:0/595834850 wait complete. 2023-07-18T03:13:27.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.537+0000 7fee70d0eb80 1 -- start start 2023-07-18T03:13:27.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.537+0000 7fee70d0eb80 1 Processor -- start 2023-07-18T03:13:27.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.538+0000 7fee70d0eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d053442d0 0x562d0534fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:27.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.538+0000 7fee70d0eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d05344be0 0x562d053503e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:27.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.538+0000 7fee70d0eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d0534fad0 0x562d05350920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:27.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.538+0000 7fee70d0eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d05236ac0 con 0x562d05344be0 2023-07-18T03:13:27.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.538+0000 7fee70d0eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d052367c0 con 0x562d0534fad0 2023-07-18T03:13:27.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.538+0000 7fee70d0eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d0511a920 con 0x562d053442d0 2023-07-18T03:13:27.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.538+0000 7fee70d0eb80 10 client.0 _ll_register_callbacks cb 0x562d05234760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:13:27.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.538+0000 7fee577fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d0534fad0 0x562d05350920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:27.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.538+0000 7fee577fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d0534fad0 0x562d05350920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:50338/0 (socket says 172.21.15.101:50338) 2023-07-18T03:13:27.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.538+0000 7fee577fe700 1 -- 172.21.15.101:0/3457970147 learned_addr learned my addr 172.21.15.101:0/3457970147 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:13:27.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.538+0000 7fee56ffd700 1 --2- 172.21.15.101:0/3457970147 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d053442d0 0x562d0534fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:27.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.538+0000 7fee567fc700 1 --2- 172.21.15.101:0/3457970147 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d05344be0 0x562d053503e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:27.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.539+0000 7fee577fe700 1 -- 172.21.15.101:0/3457970147 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d053442d0 msgr2=0x562d0534fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:27.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.539+0000 7fee577fe700 1 --2- 172.21.15.101:0/3457970147 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d053442d0 0x562d0534fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:27.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.539+0000 7fee577fe700 1 -- 172.21.15.101:0/3457970147 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d05344be0 msgr2=0x562d053503e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:27.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.539+0000 7fee577fe700 1 --2- 172.21.15.101:0/3457970147 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d05344be0 0x562d053503e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:27.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.539+0000 7fee577fe700 1 -- 172.21.15.101:0/3457970147 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee581a27e0 con 0x562d0534fad0 2023-07-18T03:13:27.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.539+0000 7fee577fe700 1 --2- 172.21.15.101:0/3457970147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d0534fad0 0x562d05350920 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fee50007c90 tx=0x7fee50007cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8de313f736dd877b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:13:27.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.539+0000 7fee3aff5700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:13:27.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.539+0000 7fee3aff5700 1 -- 172.21.15.101:0/3457970147 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5000e040 con 0x562d0534fad0 2023-07-18T03:13:27.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.539+0000 7fee70d0eb80 1 -- 172.21.15.101:0/3457970147 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562d051f04f0 con 0x562d0534fad0 2023-07-18T03:13:27.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.539+0000 7fee3aff5700 1 -- 172.21.15.101:0/3457970147 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fee50003c10 con 0x562d0534fad0 2023-07-18T03:13:27.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.539+0000 7fee3aff5700 1 -- 172.21.15.101:0/3457970147 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee50007e50 con 0x562d0534fad0 2023-07-18T03:13:27.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.539+0000 7fee70d0eb80 10 client.26726 fetch_fsmap learned FSMap version 26 2023-07-18T03:13:27.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.539+0000 7fee70d0eb80 1 -- 172.21.15.101:0/3457970147 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x562d050adc60 con 0x562d0534fad0 2023-07-18T03:13:27.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.540+0000 7fee3aff5700 1 -- 172.21.15.101:0/3457970147 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562d051f04f0 con 0x562d0534fad0 2023-07-18T03:13:27.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.540+0000 7fee3aff5700 1 -- 172.21.15.101:0/3457970147 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fee50004350 con 0x562d0534fad0 2023-07-18T03:13:27.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.540+0000 7fee70d0eb80 10 client.26726 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:13:27.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.540+0000 7fee70d0eb80 10 client.26726 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:13:27.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.540+0000 7fee70d0eb80 20 client.26726 populate_metadata read hostname 'smithi101' 2023-07-18T03:13:27.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.540+0000 7fee70d0eb80 1 --2- 172.21.15.101:0/3457970147 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x562d0535f1f0 0x562d05361600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:27.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.540+0000 7fee70d0eb80 4 client.26726 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:13:27.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.540+0000 7fee70d0eb80 1 -- 172.21.15.101:0/3457970147 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562d0528feb0 con 0x562d0535f1f0 2023-07-18T03:13:27.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.540+0000 7fee56ffd700 1 --2- 172.21.15.101:0/3457970147 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x562d0535f1f0 0x562d05361600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:27.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.541+0000 7fee3aff5700 10 client.26726 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:13:27.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.541+0000 7fee56ffd700 1 --2- 172.21.15.101:0/3457970147 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x562d0535f1f0 0x562d05361600 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f7859920ea1f8e9 server_cookie=4418ada2cffda4eb in_seq=0 out_seq=0 2023-07-18T03:13:27.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.541+0000 7fee3aff5700 1 -- 172.21.15.101:0/3457970147 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562d0528feb0 con 0x562d0535f1f0 2023-07-18T03:13:27.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.541+0000 7fee3aff5700 10 client.26726 handle_command_reply: tid=0 2023-07-18T03:13:27.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.541+0000 7fee70d0eb80 1 client.26726 shutdown 2023-07-18T03:13:27.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.541+0000 7fee70d0eb80 1 -- 172.21.15.101:0/3457970147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d0534fad0 msgr2=0x562d05350920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:27.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.541+0000 7fee70d0eb80 1 --2- 172.21.15.101:0/3457970147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d0534fad0 0x562d05350920 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fee50007c90 tx=0x7fee50007cc0 comp rx=0 tx=0).stop 2023-07-18T03:13:27.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.542+0000 7fee70d0eb80 1 -- 172.21.15.101:0/3457970147 shutdown_connections 2023-07-18T03:13:27.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.542+0000 7fee70d0eb80 1 --2- 172.21.15.101:0/3457970147 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x562d0535f1f0 0x562d05361600 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:27.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.542+0000 7fee70d0eb80 1 --2- 172.21.15.101:0/3457970147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d0534fad0 0x562d05350920 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:27.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.542+0000 7fee70d0eb80 1 --2- 172.21.15.101:0/3457970147 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d053442d0 0x562d0534fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:27.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.542+0000 7fee70d0eb80 1 --2- 172.21.15.101:0/3457970147 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d05344be0 0x562d053503e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:27.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.542+0000 7fee3aff5700 0 client.26726 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:13:27.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.542+0000 7fee70d0eb80 1 -- 172.21.15.101:0/3457970147 >> 172.21.15.101:0/3457970147 conn(0x562d0533ada0 msgr2=0x562d05340290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:13:27.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.542+0000 7fee70d0eb80 1 -- 172.21.15.101:0/3457970147 shutdown_connections 2023-07-18T03:13:27.544 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.542+0000 7fee70d0eb80 1 -- 172.21.15.101:0/3457970147 wait complete. 2023-07-18T03:13:27.544 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.542+0000 7fee70d0eb80 20 client.26726 trim_cache size 0 max 16384 2023-07-18T03:13:27.555 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.562+0000 7fee70d0eb80 1 Processor -- start 2023-07-18T03:13:27.555 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.562+0000 7fee70d0eb80 1 -- start start 2023-07-18T03:13:27.555 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.562+0000 7fee70d0eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d053442d0 0x562d0534e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:27.556 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.563+0000 7fee70d0eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d05344be0 0x562d05353ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:27.557 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.563+0000 7fee70d0eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d0534fad0 0x562d0535b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:27.557 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.563+0000 7fee70d0eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d0535c830 con 0x562d053442d0 2023-07-18T03:13:27.557 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.563+0000 7fee70d0eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d053461a0 con 0x562d0534fad0 2023-07-18T03:13:27.557 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.563+0000 7fee70d0eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d0534f180 con 0x562d05344be0 2023-07-18T03:13:27.557 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.563+0000 7fee577fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d0534fad0 0x562d0535b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:27.557 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.563+0000 7fee567fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d05344be0 0x562d05353ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:27.557 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.563+0000 7fee567fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d05344be0 0x562d05353ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:57112/0 (socket says 172.21.15.101:57112) 2023-07-18T03:13:27.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.563+0000 7fee567fc700 1 -- 172.21.15.101:0/3812866152 learned_addr learned my addr 172.21.15.101:0/3812866152 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:13:27.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.563+0000 7fee577fe700 1 --2- 172.21.15.101:0/3812866152 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d0534fad0 0x562d0535b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:50352/0 (socket says 172.21.15.101:50352) 2023-07-18T03:13:27.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.563+0000 7fee56ffd700 1 --2- 172.21.15.101:0/3812866152 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d053442d0 0x562d0534e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:27.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.563+0000 7fee577fe700 1 -- 172.21.15.101:0/3812866152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d05344be0 msgr2=0x562d05353ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:27.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.564+0000 7fee577fe700 1 --2- 172.21.15.101:0/3812866152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d05344be0 0x562d05353ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:27.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.564+0000 7fee577fe700 1 -- 172.21.15.101:0/3812866152 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d053442d0 msgr2=0x562d0534e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:27.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.564+0000 7fee577fe700 1 --2- 172.21.15.101:0/3812866152 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d053442d0 0x562d0534e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:27.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.564+0000 7fee577fe700 1 -- 172.21.15.101:0/3812866152 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d050eb250 con 0x562d0534fad0 2023-07-18T03:13:27.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.564+0000 7fee567fc700 1 --2- 172.21.15.101:0/3812866152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d05344be0 0x562d05353ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:13:27.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.564+0000 7fee577fe700 1 --2- 172.21.15.101:0/3812866152 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d0534fad0 0x562d0535b410 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fee500046b0 tx=0x7fee50013610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c35d89ef56bc29d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:13:27.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.564+0000 7fee3dffb700 1 -- 172.21.15.101:0/3812866152 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee50006080 con 0x562d0534fad0 2023-07-18T03:13:27.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.564+0000 7fee3dffb700 1 -- 172.21.15.101:0/3812866152 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee5000e030 con 0x562d0534fad0 2023-07-18T03:13:27.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.564+0000 7fee3dffb700 1 -- 172.21.15.101:0/3812866152 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee50013eb0 con 0x562d0534fad0 2023-07-18T03:13:27.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.565+0000 7fee70d0eb80 1 -- 172.21.15.101:0/3812866152 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d0534fad0 msgr2=0x562d0535b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:27.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.565+0000 7fee70d0eb80 1 --2- 172.21.15.101:0/3812866152 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d0534fad0 0x562d0535b410 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fee500046b0 tx=0x7fee50013610 comp rx=0 tx=0).stop 2023-07-18T03:13:27.562 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.565+0000 7fee70d0eb80 1 -- 172.21.15.101:0/3812866152 shutdown_connections 2023-07-18T03:13:27.562 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.565+0000 7fee70d0eb80 1 --2- 172.21.15.101:0/3812866152 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d0534fad0 0x562d0535b410 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:27.562 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.565+0000 7fee70d0eb80 1 --2- 172.21.15.101:0/3812866152 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d053442d0 0x562d0534e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:27.562 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.565+0000 7fee70d0eb80 1 --2- 172.21.15.101:0/3812866152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d05344be0 0x562d05353ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:27.562 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.565+0000 7fee70d0eb80 1 -- 172.21.15.101:0/3812866152 >> 172.21.15.101:0/3812866152 conn(0x562d0533ada0 msgr2=0x562d05365f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:13:27.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.565+0000 7fee70d0eb80 1 -- 172.21.15.101:0/3812866152 shutdown_connections 2023-07-18T03:13:27.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.566+0000 7fee70d0eb80 1 -- 172.21.15.101:0/3812866152 wait complete. 2023-07-18T03:13:27.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.566+0000 7fee70d0eb80 1 -- start start 2023-07-18T03:13:27.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.566+0000 7fee70d0eb80 1 Processor -- start 2023-07-18T03:13:27.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.567+0000 7fee70d0eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d053442d0 0x562d053ac540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:27.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.567+0000 7fee70d0eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d05344be0 0x562d053aca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:27.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.567+0000 7fee70d0eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d0534fad0 0x562d053ab2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:27.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.567+0000 7fee70d0eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d053ab860 con 0x562d053442d0 2023-07-18T03:13:27.564 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.567+0000 7fee70d0eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d053ab9d0 con 0x562d0534fad0 2023-07-18T03:13:27.564 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.567+0000 7fee70d0eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d053abb90 con 0x562d05344be0 2023-07-18T03:13:27.564 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.567+0000 7fee70d0eb80 10 client.0 _ll_register_callbacks cb 0x7fee500075e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:13:27.564 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.568+0000 7fee577fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d0534fad0 0x562d053ab2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:27.564 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.568+0000 7fee577fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d0534fad0 0x562d053ab2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:50362/0 (socket says 172.21.15.101:50362) 2023-07-18T03:13:27.564 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.568+0000 7fee577fe700 1 -- 172.21.15.101:0/2579627525 learned_addr learned my addr 172.21.15.101:0/2579627525 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:13:27.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.568+0000 7fee567fc700 1 --2- 172.21.15.101:0/2579627525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d05344be0 0x562d053aca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:27.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.568+0000 7fee56ffd700 1 --2- 172.21.15.101:0/2579627525 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d053442d0 0x562d053ac540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:27.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.568+0000 7fee577fe700 1 -- 172.21.15.101:0/2579627525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d05344be0 msgr2=0x562d053aca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:27.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.568+0000 7fee577fe700 1 --2- 172.21.15.101:0/2579627525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d05344be0 0x562d053aca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:27.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.568+0000 7fee577fe700 1 -- 172.21.15.101:0/2579627525 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d053442d0 msgr2=0x562d053ac540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:27.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.568+0000 7fee577fe700 1 --2- 172.21.15.101:0/2579627525 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d053442d0 0x562d053ac540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:27.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.568+0000 7fee577fe700 1 -- 172.21.15.101:0/2579627525 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d050c4de0 con 0x562d0534fad0 2023-07-18T03:13:27.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.568+0000 7fee577fe700 1 --2- 172.21.15.101:0/2579627525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d0534fad0 0x562d053ab2b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fee5002e040 tx=0x7fee50013950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd86335bf516c2c8 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:13:27.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.568+0000 7fee3aff5700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:13:27.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.569+0000 7fee3aff5700 1 -- 172.21.15.101:0/2579627525 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee50013d40 con 0x562d0534fad0 2023-07-18T03:13:27.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.569+0000 7fee3aff5700 1 -- 172.21.15.101:0/2579627525 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fee500048d0 con 0x562d0534fad0 2023-07-18T03:13:27.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.569+0000 7fee70d0eb80 1 -- 172.21.15.101:0/2579627525 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fee4c09b100 con 0x562d0534fad0 2023-07-18T03:13:27.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.569+0000 7fee3aff5700 1 -- 172.21.15.101:0/2579627525 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee50013ea0 con 0x562d0534fad0 2023-07-18T03:13:27.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.569+0000 7fee70d0eb80 10 client.26732 fetch_fsmap learned FSMap version 26 2023-07-18T03:13:27.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.569+0000 7fee70d0eb80 1 -- 172.21.15.101:0/2579627525 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x562d050adc60 con 0x562d0534fad0 2023-07-18T03:13:27.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.570+0000 7fee3aff5700 1 -- 172.21.15.101:0/2579627525 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee5003f020 con 0x562d0534fad0 2023-07-18T03:13:27.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.570+0000 7fee3aff5700 1 -- 172.21.15.101:0/2579627525 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fee50003c10 con 0x562d0534fad0 2023-07-18T03:13:27.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.570+0000 7fee70d0eb80 10 client.26732 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:13:27.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.570+0000 7fee70d0eb80 10 client.26732 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:13:27.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.570+0000 7fee70d0eb80 20 client.26732 populate_metadata read hostname 'smithi101' 2023-07-18T03:13:27.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.570+0000 7fee70d0eb80 1 --2- 172.21.15.101:0/2579627525 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x562d053b4070 0x562d053b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:27.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.570+0000 7fee70d0eb80 4 client.26732 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:13:27.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.570+0000 7fee70d0eb80 1 -- 172.21.15.101:0/2579627525 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562d05252050 con 0x562d053b4070 2023-07-18T03:13:27.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.570+0000 7fee56ffd700 1 --2- 172.21.15.101:0/2579627525 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x562d053b4070 0x562d053b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:27.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.571+0000 7fee56ffd700 1 --2- 172.21.15.101:0/2579627525 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x562d053b4070 0x562d053b6450 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=253a4e4118e68b46 server_cookie=d1a16fe2b4eea67d in_seq=0 out_seq=0 2023-07-18T03:13:27.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.571+0000 7fee3aff5700 10 client.26732 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:13:27.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.571+0000 7fee3aff5700 1 -- 172.21.15.101:0/2579627525 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562d05252050 con 0x562d053b4070 2023-07-18T03:13:27.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.571+0000 7fee3aff5700 10 client.26732 handle_command_reply: tid=0 2023-07-18T03:13:27.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.571+0000 7fee70d0eb80 1 client.26732 shutdown 2023-07-18T03:13:27.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.572+0000 7fee70d0eb80 1 -- 172.21.15.101:0/2579627525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d0534fad0 msgr2=0x562d053ab2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:27.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.572+0000 7fee70d0eb80 1 --2- 172.21.15.101:0/2579627525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d0534fad0 0x562d053ab2b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fee5002e040 tx=0x7fee50013950 comp rx=0 tx=0).stop 2023-07-18T03:13:27.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.572+0000 7fee70d0eb80 1 -- 172.21.15.101:0/2579627525 shutdown_connections 2023-07-18T03:13:27.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.572+0000 7fee70d0eb80 1 --2- 172.21.15.101:0/2579627525 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x562d053b4070 0x562d053b6450 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:27.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.572+0000 7fee70d0eb80 1 --2- 172.21.15.101:0/2579627525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562d0534fad0 0x562d053ab2b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:27.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.572+0000 7fee70d0eb80 1 --2- 172.21.15.101:0/2579627525 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562d053442d0 0x562d053ac540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:27.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.572+0000 7fee70d0eb80 1 --2- 172.21.15.101:0/2579627525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d05344be0 0x562d053aca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:27.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.572+0000 7fee70d0eb80 1 -- 172.21.15.101:0/2579627525 >> 172.21.15.101:0/2579627525 conn(0x562d0533ada0 msgr2=0x562d0535b410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:13:27.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.572+0000 7fee3aff5700 0 client.26732 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:13:27.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.572+0000 7fee70d0eb80 1 -- 172.21.15.101:0/2579627525 shutdown_connections 2023-07-18T03:13:27.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.572+0000 7fee70d0eb80 1 -- 172.21.15.101:0/2579627525 wait complete. 2023-07-18T03:13:27.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.572+0000 7fee70d0eb80 20 client.26732 trim_cache size 0 max 16384 2023-07-18T03:13:27.576 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:13:27.576 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "no active scrubs running", 2023-07-18T03:13:27.576 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": {} 2023-07-18T03:13:27.576 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:13:27.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.573+0000 7fee5d3ff700 1 -- 172.21.15.101:0/3808761122 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fee400a1340 msgr2=0x7fee400a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:27.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.573+0000 7fee5d3ff700 1 --2- 172.21.15.101:0/3808761122 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fee400a1340 0x7fee400a37f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fee48009c30 tx=0x7fee48009260 comp rx=0 tx=0).stop 2023-07-18T03:13:27.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.573+0000 7fee5d3ff700 1 -- 172.21.15.101:0/3808761122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee58100490 msgr2=0x7fee5819bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:27.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.573+0000 7fee5d3ff700 1 --2- 172.21.15.101:0/3808761122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee58100490 0x7fee5819bf10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fee4c005910 tx=0x7fee4c005720 comp rx=0 tx=0).stop 2023-07-18T03:13:27.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.573+0000 7fee5d3ff700 1 -- 172.21.15.101:0/3808761122 shutdown_connections 2023-07-18T03:13:27.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.573+0000 7fee5d3ff700 1 --2- 172.21.15.101:0/3808761122 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee58105af0 0x7fee58195a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:27.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.573+0000 7fee5d3ff700 1 --2- 172.21.15.101:0/3808761122 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fee580ffae0 0x7fee5819b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:27.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.573+0000 7fee5d3ff700 1 --2- 172.21.15.101:0/3808761122 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fee400a1340 0x7fee400a37f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:27.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.573+0000 7fee5d3ff700 1 --2- 172.21.15.101:0/3808761122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee58100490 0x7fee5819bf10 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:27.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.573+0000 7fee5d3ff700 1 -- 172.21.15.101:0/3808761122 >> 172.21.15.101:0/3808761122 conn(0x7fee580fb5e0 msgr2=0x7fee580fbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:13:27.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.574+0000 7fee5d3ff700 1 -- 172.21.15.101:0/3808761122 shutdown_connections 2023-07-18T03:13:27.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.574+0000 7fee5d3ff700 1 -- 172.21.15.101:0/3808761122 wait complete. 2023-07-18T03:13:27.583 INFO:tasks.workunit.client.0.smithi101.stdout: 1 454864 8.57 MB/sec execute 172 sec latency 16.648 ms 2023-07-18T03:13:27.586 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-18T03:13:27.586 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:13:27.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.808+0000 7f4a0119d700 1 -- 172.21.15.101:0/3684786350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49fc100380 msgr2=0x7f49fc10c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:27.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.808+0000 7f4a0119d700 1 --2- 172.21.15.101:0/3684786350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49fc100380 0x7f49fc10c810 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f49f400b0a0 tx=0x7f49f400b3b0 comp rx=0 tx=0).stop 2023-07-18T03:13:27.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.808+0000 7f4a0119d700 1 -- 172.21.15.101:0/3684786350 shutdown_connections 2023-07-18T03:13:27.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.808+0000 7f4a0119d700 1 --2- 172.21.15.101:0/3684786350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49fc1059e0 0x7f49fc105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:27.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.808+0000 7f4a0119d700 1 --2- 172.21.15.101:0/3684786350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49fc100380 0x7f49fc10c810 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:27.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.808+0000 7f4a0119d700 1 --2- 172.21.15.101:0/3684786350 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f49fc0ff9d0 0x7f49fc0ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:27.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.808+0000 7f4a0119d700 1 -- 172.21.15.101:0/3684786350 >> 172.21.15.101:0/3684786350 conn(0x7f49fc0fb4d0 msgr2=0x7f49fc0fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:13:27.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.808+0000 7f4a0119d700 1 -- 172.21.15.101:0/3684786350 shutdown_connections 2023-07-18T03:13:27.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.809+0000 7f4a0119d700 1 -- 172.21.15.101:0/3684786350 wait complete. 2023-07-18T03:13:27.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.809+0000 7f4a0119d700 1 Processor -- start 2023-07-18T03:13:27.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.810+0000 7f4a0119d700 1 -- start start 2023-07-18T03:13:27.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.810+0000 7f4a0119d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49fc0ff9d0 0x7f49fc072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:27.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.810+0000 7f4a0119d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f49fc100380 0x7f49fc072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:27.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.810+0000 7f4a0119d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49fc1059e0 0x7f49fc073290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:27.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.810+0000 7f4a0119d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49fc1a8370 con 0x7f49fc100380 2023-07-18T03:13:27.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.810+0000 7f4a0119d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49fc1a84b0 con 0x7f49fc0ff9d0 2023-07-18T03:13:27.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.810+0000 7f4a0119d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49fc1a85f0 con 0x7f49fc1059e0 2023-07-18T03:13:27.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.810+0000 7f49fb7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f49fc100380 0x7f49fc072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:27.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.811+0000 7f49fb7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f49fc100380 0x7f49fc072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:41462/0 (socket says 172.21.15.101:41462) 2023-07-18T03:13:27.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.811+0000 7f49fb7fe700 1 -- 172.21.15.101:0/608018886 learned_addr learned my addr 172.21.15.101:0/608018886 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:13:27.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.811+0000 7f49fbfff700 1 --2- 172.21.15.101:0/608018886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49fc0ff9d0 0x7f49fc072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:27.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.811+0000 7f4a0099c700 1 --2- 172.21.15.101:0/608018886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49fc1059e0 0x7f49fc073290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:27.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.811+0000 7f49fb7fe700 1 -- 172.21.15.101:0/608018886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49fc1059e0 msgr2=0x7f49fc073290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:27.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.811+0000 7f49fb7fe700 1 --2- 172.21.15.101:0/608018886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49fc1059e0 0x7f49fc073290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:27.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.811+0000 7f49fb7fe700 1 -- 172.21.15.101:0/608018886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49fc0ff9d0 msgr2=0x7f49fc072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:27.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.811+0000 7f49fb7fe700 1 --2- 172.21.15.101:0/608018886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49fc0ff9d0 0x7f49fc072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:27.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.811+0000 7f49fb7fe700 1 -- 172.21.15.101:0/608018886 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f4009d00 con 0x7f49fc100380 2023-07-18T03:13:27.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.811+0000 7f49fb7fe700 1 --2- 172.21.15.101:0/608018886 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f49fc100380 0x7f49fc072d50 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f49f000e9a0 tx=0x7f49f000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:13:27.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.812+0000 7f49f97fa700 1 -- 172.21.15.101:0/608018886 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f0009790 con 0x7f49fc100380 2023-07-18T03:13:27.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.812+0000 7f49f97fa700 1 -- 172.21.15.101:0/608018886 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49f0010430 con 0x7f49fc100380 2023-07-18T03:13:27.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.812+0000 7f49f97fa700 1 -- 172.21.15.101:0/608018886 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f0018680 con 0x7f49fc100380 2023-07-18T03:13:27.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.812+0000 7f4a0119d700 1 -- 172.21.15.101:0/608018886 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49fc1a88d0 con 0x7f49fc100380 2023-07-18T03:13:27.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.812+0000 7f4a0119d700 1 -- 172.21.15.101:0/608018886 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49fc1a8d90 con 0x7f49fc100380 2023-07-18T03:13:27.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.813+0000 7f4a0119d700 1 -- 172.21.15.101:0/608018886 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49fc067220 con 0x7f49fc100380 2023-07-18T03:13:27.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.815+0000 7f49f97fa700 1 -- 172.21.15.101:0/608018886 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f49f00105a0 con 0x7f49fc100380 2023-07-18T03:13:27.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.817+0000 7f49f97fa700 1 --2- 172.21.15.101:0/608018886 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f49e40a1430 0x7f49e40a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:27.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.817+0000 7f49f97fa700 1 -- 172.21.15.101:0/608018886 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f49f0014070 con 0x7f49fc100380 2023-07-18T03:13:27.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.817+0000 7f49fbfff700 1 --2- 172.21.15.101:0/608018886 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f49e40a1430 0x7f49e40a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:27.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.817+0000 7f49fbfff700 1 --2- 172.21.15.101:0/608018886 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f49e40a1430 0x7f49e40a38e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f49fc100b10 tx=0x7f49ec009260 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:13:27.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:27.819+0000 7f49f97fa700 1 -- 172.21.15.101:0/608018886 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f49f00964a0 con 0x7f49fc100380 2023-07-18T03:13:28.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.084+0000 7f4a0119d700 1 -- 172.21.15.101:0/608018886 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49fc073d40 con 0x7f49fc100380 2023-07-18T03:13:28.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.084+0000 7f49f97fa700 1 -- 172.21.15.101:0/608018886 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10858 (secure 0 0 0) 0x7f49f00ade60 con 0x7f49fc100380 2023-07-18T03:13:28.078 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:13:28.079 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/4192846769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":4192846769},{"type":"v1","addr":"172.21.15.101:6839","nonce":4192846769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6841/3824340175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6840","nonce":3824340175},{"type":"v1","addr":"172.21.15.101:6841","nonce":3824340175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:04:59.361185+0000","modified":"2023-07-18T03:11:46.116493+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.155:6839/2055462483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2055462483},{"type":"v1","addr":"172.21.15.155:6839","nonce":2055462483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:13:28.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.087+0000 7f4a0119d700 1 -- 172.21.15.101:0/608018886 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f49e40a1430 msgr2=0x7f49e40a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:28.081 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.087+0000 7f4a0119d700 1 --2- 172.21.15.101:0/608018886 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f49e40a1430 0x7f49e40a38e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f49fc100b10 tx=0x7f49ec009260 comp rx=0 tx=0).stop 2023-07-18T03:13:28.081 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.087+0000 7f4a0119d700 1 -- 172.21.15.101:0/608018886 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f49fc100380 msgr2=0x7f49fc072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:28.081 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.087+0000 7f4a0119d700 1 --2- 172.21.15.101:0/608018886 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f49fc100380 0x7f49fc072d50 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f49f000e9a0 tx=0x7f49f000ecb0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.081 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.087+0000 7f4a0119d700 1 -- 172.21.15.101:0/608018886 shutdown_connections 2023-07-18T03:13:28.081 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.087+0000 7f4a0119d700 1 --2- 172.21.15.101:0/608018886 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f49e40a1430 0x7f49e40a38e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.081 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.087+0000 7f4a0119d700 1 --2- 172.21.15.101:0/608018886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49fc0ff9d0 0x7f49fc072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.087+0000 7f4a0119d700 1 --2- 172.21.15.101:0/608018886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49fc1059e0 0x7f49fc073290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.087+0000 7f4a0119d700 1 --2- 172.21.15.101:0/608018886 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f49fc100380 0x7f49fc072d50 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.087+0000 7f4a0119d700 1 -- 172.21.15.101:0/608018886 >> 172.21.15.101:0/608018886 conn(0x7f49fc0fb4d0 msgr2=0x7f49fc106750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:13:28.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.088+0000 7f4a0119d700 1 -- 172.21.15.101:0/608018886 shutdown_connections 2023-07-18T03:13:28.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.088+0000 7f4a0119d700 1 -- 172.21.15.101:0/608018886 wait complete. 2023-07-18T03:13:28.083 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 26 2023-07-18T03:13:28.101 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-18T03:13:28.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.319+0000 7f16f0bff700 1 -- 172.21.15.101:0/3158386722 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f16ec105af0 msgr2=0x7f16ec105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:28.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.319+0000 7f16f0bff700 1 --2- 172.21.15.101:0/3158386722 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f16ec105af0 0x7f16ec105ec0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f16dc009990 tx=0x7f16dc009ca0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.319+0000 7f16f0bff700 1 -- 172.21.15.101:0/3158386722 shutdown_connections 2023-07-18T03:13:28.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.319+0000 7f16f0bff700 1 --2- 172.21.15.101:0/3158386722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16ec100490 0x7f16ec10c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.319+0000 7f16f0bff700 1 --2- 172.21.15.101:0/3158386722 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f16ec105af0 0x7f16ec105ec0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.319+0000 7f16f0bff700 1 --2- 172.21.15.101:0/3158386722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16ec0ffae0 0x7f16ec0fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.319+0000 7f16f0bff700 1 -- 172.21.15.101:0/3158386722 >> 172.21.15.101:0/3158386722 conn(0x7f16ec0fb5e0 msgr2=0x7f16ec0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:13:28.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.319+0000 7f16f0bff700 1 -- 172.21.15.101:0/3158386722 shutdown_connections 2023-07-18T03:13:28.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.319+0000 7f16f0bff700 1 -- 172.21.15.101:0/3158386722 wait complete. 2023-07-18T03:13:28.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.320+0000 7f16f0bff700 1 Processor -- start 2023-07-18T03:13:28.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.320+0000 7f16f0bff700 1 -- start start 2023-07-18T03:13:28.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.321+0000 7f16f0bff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16ec0ffae0 0x7f16ec197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.321+0000 7f16f0bff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16ec100490 0x7f16ec198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.321+0000 7f16f0bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f16ec105af0 0x7f16ec19c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.321+0000 7f16f0bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec19cdd0 con 0x7f16ec105af0 2023-07-18T03:13:28.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.321+0000 7f16f0bff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec19cf40 con 0x7f16ec100490 2023-07-18T03:13:28.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.321+0000 7f16f0bff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec19d100 con 0x7f16ec0ffae0 2023-07-18T03:13:28.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.321+0000 7f16e9d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16ec100490 0x7f16ec198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.321+0000 7f16ead9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f16ec105af0 0x7f16ec19c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.321+0000 7f16ea59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16ec0ffae0 0x7f16ec197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.321+0000 7f16e9d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16ec100490 0x7f16ec198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:50398/0 (socket says 172.21.15.101:50398) 2023-07-18T03:13:28.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.321+0000 7f16e9d9b700 1 -- 172.21.15.101:0/391087265 learned_addr learned my addr 172.21.15.101:0/391087265 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:13:28.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.321+0000 7f16e9d9b700 1 -- 172.21.15.101:0/391087265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16ec0ffae0 msgr2=0x7f16ec197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:28.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.321+0000 7f16e9d9b700 1 --2- 172.21.15.101:0/391087265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16ec0ffae0 0x7f16ec197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.321+0000 7f16e9d9b700 1 -- 172.21.15.101:0/391087265 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f16ec105af0 msgr2=0x7f16ec19c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:28.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.321+0000 7f16e9d9b700 1 --2- 172.21.15.101:0/391087265 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f16ec105af0 0x7f16ec19c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.321+0000 7f16e9d9b700 1 -- 172.21.15.101:0/391087265 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16dc0095f0 con 0x7f16ec100490 2023-07-18T03:13:28.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.321+0000 7f16ead9d700 1 --2- 172.21.15.101:0/391087265 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f16ec105af0 0x7f16ec19c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:13:28.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.321+0000 7f16ea59c700 1 --2- 172.21.15.101:0/391087265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16ec0ffae0 0x7f16ec197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:13:28.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.322+0000 7f16e9d9b700 1 --2- 172.21.15.101:0/391087265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16ec100490 0x7f16ec198330 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f16e000e9a0 tx=0x7f16e000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:13:28.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.322+0000 7f16db7fe700 1 -- 172.21.15.101:0/391087265 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e0009790 con 0x7f16ec100490 2023-07-18T03:13:28.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.322+0000 7f16f0bff700 1 -- 172.21.15.101:0/391087265 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16ec19d3e0 con 0x7f16ec100490 2023-07-18T03:13:28.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.322+0000 7f16db7fe700 1 -- 172.21.15.101:0/391087265 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16e0010430 con 0x7f16ec100490 2023-07-18T03:13:28.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.322+0000 7f16db7fe700 1 -- 172.21.15.101:0/391087265 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e0018680 con 0x7f16ec100490 2023-07-18T03:13:28.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.322+0000 7f16f0bff700 1 -- 172.21.15.101:0/391087265 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16ec1a4c20 con 0x7f16ec100490 2023-07-18T03:13:28.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.325+0000 7f16db7fe700 1 -- 172.21.15.101:0/391087265 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f16e00105a0 con 0x7f16ec100490 2023-07-18T03:13:28.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.327+0000 7f170452fb80 1 Processor -- start 2023-07-18T03:13:28.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.327+0000 7f170452fb80 1 -- start start 2023-07-18T03:13:28.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.327+0000 7f16db7fe700 1 --2- 172.21.15.101:0/391087265 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f16d40a12c0 0x7f16d40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.327+0000 7f16db7fe700 1 -- 172.21.15.101:0/391087265 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f16e0014070 con 0x7f16ec100490 2023-07-18T03:13:28.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.327+0000 7f170452fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e2644f92d0 0x55e2644f96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.327+0000 7f170452fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e2644f9be0 0x55e264504590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.327+0000 7f170452fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e264504ad0 0x55e264506eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.327+0000 7f170452fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2643ffcb0 con 0x55e264504ad0 2023-07-18T03:13:28.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.327+0000 7f170452fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e264452a40 con 0x55e2644f9be0 2023-07-18T03:13:28.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.327+0000 7f170452fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e26444bed0 con 0x55e2644f92d0 2023-07-18T03:13:28.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.328+0000 7f16ead9d700 1 --2- 172.21.15.101:0/391087265 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f16d40a12c0 0x7f16d40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.328+0000 7f16e9d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e2644f9be0 0x55e264504590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.328+0000 7f16ea59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e2644f92d0 0x55e2644f96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.328+0000 7f16e9d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e2644f9be0 0x55e264504590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:50412/0 (socket says 172.21.15.101:50412) 2023-07-18T03:13:28.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.328+0000 7f16e9d9b700 1 -- 172.21.15.101:0/659075382 learned_addr learned my addr 172.21.15.101:0/659075382 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:13:28.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.328+0000 7f16ead9d700 1 --2- 172.21.15.101:0/659075382 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e264504ad0 0x55e264506eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.328+0000 7f16ea59c700 1 -- 172.21.15.101:0/659075382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e2644f9be0 msgr2=0x55e264504590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:28.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.328+0000 7f16ea59c700 1 --2- 172.21.15.101:0/659075382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e2644f9be0 0x55e264504590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.328+0000 7f16ea59c700 1 -- 172.21.15.101:0/659075382 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e264504ad0 msgr2=0x55e264506eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:28.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.328+0000 7f16ea59c700 1 --2- 172.21.15.101:0/659075382 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e264504ad0 0x55e264506eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.328+0000 7f16ea59c700 1 -- 172.21.15.101:0/659075382 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e264279de0 con 0x55e2644f92d0 2023-07-18T03:13:28.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.328+0000 7f16e9d9b700 1 --2- 172.21.15.101:0/659075382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e2644f9be0 0x55e264504590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:13:28.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.328+0000 7f16ea59c700 1 --2- 172.21.15.101:0/659075382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e2644f92d0 0x55e2644f96a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f16dc000c00 tx=0x7f16dc005ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9071bd80f4dc29ff server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:13:28.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.329+0000 7f16ead9d700 1 --2- 172.21.15.101:0/391087265 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f16d40a12c0 0x7f16d40a3770 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f16e40043a0 tx=0x7f16e4009920 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:13:28.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.329+0000 7f16d8ff9700 1 -- 172.21.15.101:0/659075382 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16dc01d070 con 0x55e2644f92d0 2023-07-18T03:13:28.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.329+0000 7f16d8ff9700 1 -- 172.21.15.101:0/659075382 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16dc01e040 con 0x55e2644f92d0 2023-07-18T03:13:28.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.329+0000 7f16d8ff9700 1 -- 172.21.15.101:0/659075382 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16dc004d60 con 0x55e2644f92d0 2023-07-18T03:13:28.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.329+0000 7f170452fb80 1 -- 172.21.15.101:0/659075382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e2644f92d0 msgr2=0x55e2644f96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:28.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.329+0000 7f170452fb80 1 --2- 172.21.15.101:0/659075382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e2644f92d0 0x55e2644f96a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f16dc000c00 tx=0x7f16dc005ee0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.330+0000 7f170452fb80 1 -- 172.21.15.101:0/659075382 shutdown_connections 2023-07-18T03:13:28.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.330+0000 7f170452fb80 1 --2- 172.21.15.101:0/659075382 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e264504ad0 0x55e264506eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.330+0000 7f170452fb80 1 --2- 172.21.15.101:0/659075382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e2644f92d0 0x55e2644f96a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.330+0000 7f170452fb80 1 --2- 172.21.15.101:0/659075382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e2644f9be0 0x55e264504590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.327 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.330+0000 7f170452fb80 1 -- 172.21.15.101:0/659075382 >> 172.21.15.101:0/659075382 conn(0x55e2644efda0 msgr2=0x55e2644f56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:13:28.327 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.330+0000 7f170452fb80 1 -- 172.21.15.101:0/659075382 shutdown_connections 2023-07-18T03:13:28.327 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.330+0000 7f170452fb80 1 -- 172.21.15.101:0/659075382 wait complete. 2023-07-18T03:13:28.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.330+0000 7f170452fb80 1 -- start start 2023-07-18T03:13:28.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.330+0000 7f170452fb80 1 Processor -- start 2023-07-18T03:13:28.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.331+0000 7f170452fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e2644f92d0 0x55e2645044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.331+0000 7f170452fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e2644f9be0 0x55e2645017d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.331+0000 7f170452fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e264504ad0 0x55e264501e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.331+0000 7f170452fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2643ebac0 con 0x55e2644f92d0 2023-07-18T03:13:28.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.331+0000 7f170452fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2643eb7c0 con 0x55e2644f9be0 2023-07-18T03:13:28.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.331+0000 7f170452fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2642cf920 con 0x55e264504ad0 2023-07-18T03:13:28.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.331+0000 7f170452fb80 10 client.0 _ll_register_callbacks cb 0x55e2643e9760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:13:28.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.331+0000 7f16e9d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e2644f9be0 0x55e2645017d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.331+0000 7f16ead9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e264504ad0 0x55e264501e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.331+0000 7f16ea59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e2644f92d0 0x55e2645044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.331+0000 7f16e9d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e2644f9be0 0x55e2645017d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:50420/0 (socket says 172.21.15.101:50420) 2023-07-18T03:13:28.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.331+0000 7f16e9d9b700 1 -- 172.21.15.101:0/3985115391 learned_addr learned my addr 172.21.15.101:0/3985115391 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:13:28.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.331+0000 7f16e9d9b700 1 -- 172.21.15.101:0/3985115391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e264504ad0 msgr2=0x55e264501e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:28.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.331+0000 7f16e9d9b700 1 --2- 172.21.15.101:0/3985115391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e264504ad0 0x55e264501e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.331+0000 7f16e9d9b700 1 -- 172.21.15.101:0/3985115391 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e2644f92d0 msgr2=0x55e2645044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:28.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.331+0000 7f16e9d9b700 1 --2- 172.21.15.101:0/3985115391 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e2644f92d0 0x55e2645044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.331+0000 7f16e9d9b700 1 -- 172.21.15.101:0/3985115391 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16ec1a4c20 con 0x55e2644f9be0 2023-07-18T03:13:28.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.332+0000 7f16e9d9b700 1 --2- 172.21.15.101:0/3985115391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e2644f9be0 0x55e2645017d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f16e000c510 tx=0x7f16e0099fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4db8e11b92a37ac5 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:13:28.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.332+0000 7f16d1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:13:28.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.332+0000 7f16d1ffb700 1 -- 172.21.15.101:0/3985115391 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e0099690 con 0x55e2644f9be0 2023-07-18T03:13:28.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.332+0000 7f170452fb80 1 -- 172.21.15.101:0/3985115391 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e2643a54f0 con 0x55e2644f9be0 2023-07-18T03:13:28.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.332+0000 7f16d1ffb700 1 -- 172.21.15.101:0/3985115391 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f16e009dfd0 con 0x55e2644f9be0 2023-07-18T03:13:28.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.332+0000 7f16d1ffb700 1 -- 172.21.15.101:0/3985115391 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16e00a23f0 con 0x55e2644f9be0 2023-07-18T03:13:28.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.332+0000 7f170452fb80 10 client.26750 fetch_fsmap learned FSMap version 26 2023-07-18T03:13:28.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.332+0000 7f170452fb80 1 -- 172.21.15.101:0/3985115391 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f16ec10d220 con 0x55e2644f9be0 2023-07-18T03:13:28.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.332+0000 7f16d1ffb700 1 -- 172.21.15.101:0/3985115391 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e2643a54f0 con 0x55e2644f9be0 2023-07-18T03:13:28.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.333+0000 7f16d1ffb700 1 -- 172.21.15.101:0/3985115391 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f16e00b80d0 con 0x55e2644f9be0 2023-07-18T03:13:28.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.333+0000 7f170452fb80 10 client.26750 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:13:28.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.333+0000 7f170452fb80 10 client.26750 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:13:28.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.333+0000 7f170452fb80 20 client.26750 populate_metadata read hostname 'smithi101' 2023-07-18T03:13:28.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.333+0000 7f170452fb80 1 --2- 172.21.15.101:0/3985115391 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55e2645143e0 0x55e2645167f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.333+0000 7f170452fb80 4 client.26750 mds_command: new command op to 24541 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:13:28.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.333+0000 7f170452fb80 1 -- 172.21.15.101:0/3985115391 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e264444eb0 con 0x55e2645143e0 2023-07-18T03:13:28.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.333+0000 7f16ea59c700 1 --2- 172.21.15.101:0/3985115391 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55e2645143e0 0x55e2645167f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.334+0000 7f16ea59c700 1 --2- 172.21.15.101:0/3985115391 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55e2645143e0 0x55e2645167f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c2fa822b0d08f9eb server_cookie=e7de530c4ac6e571 in_seq=0 out_seq=0 2023-07-18T03:13:28.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.334+0000 7f16d1ffb700 10 client.26750 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:13:28.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.334+0000 7f16d1ffb700 1 -- 172.21.15.101:0/3985115391 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e264444eb0 con 0x55e2645143e0 2023-07-18T03:13:28.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.334+0000 7f16d1ffb700 10 client.26750 handle_command_reply: tid=0 2023-07-18T03:13:28.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.334+0000 7f170452fb80 1 client.26750 shutdown 2023-07-18T03:13:28.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.334+0000 7f170452fb80 1 -- 172.21.15.101:0/3985115391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e2644f9be0 msgr2=0x55e2645017d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:28.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.334+0000 7f170452fb80 1 --2- 172.21.15.101:0/3985115391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e2644f9be0 0x55e2645017d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f16e000c510 tx=0x7f16e0099fa0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.334+0000 7f170452fb80 1 -- 172.21.15.101:0/3985115391 shutdown_connections 2023-07-18T03:13:28.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.334+0000 7f170452fb80 1 --2- 172.21.15.101:0/3985115391 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55e2645143e0 0x55e2645167f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.334+0000 7f170452fb80 1 --2- 172.21.15.101:0/3985115391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e264504ad0 0x55e264501e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.334+0000 7f170452fb80 1 --2- 172.21.15.101:0/3985115391 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e2644f92d0 0x55e2645044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.334+0000 7f170452fb80 1 --2- 172.21.15.101:0/3985115391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e2644f9be0 0x55e2645017d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.334+0000 7f16d1ffb700 0 client.26750 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:13:28.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.334+0000 7f170452fb80 1 -- 172.21.15.101:0/3985115391 >> 172.21.15.101:0/3985115391 conn(0x55e2644efda0 msgr2=0x55e2644f5230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:13:28.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.335+0000 7f170452fb80 1 -- 172.21.15.101:0/3985115391 shutdown_connections 2023-07-18T03:13:28.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.335+0000 7f170452fb80 1 -- 172.21.15.101:0/3985115391 wait complete. 2023-07-18T03:13:28.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.335+0000 7f170452fb80 20 client.26750 trim_cache size 0 max 16384 2023-07-18T03:13:28.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.349+0000 7f170452fb80 1 Processor -- start 2023-07-18T03:13:28.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.349+0000 7f170452fb80 1 -- start start 2023-07-18T03:13:28.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.349+0000 7f170452fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e2644f92d0 0x55e264504080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.349+0000 7f170452fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e2644f9be0 0x55e264500f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.349+0000 7f170452fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e264504ad0 0x55e2645014d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.349+0000 7f170452fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e264502d40 con 0x55e2644f92d0 2023-07-18T03:13:28.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.349+0000 7f170452fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2644ef970 con 0x55e2644f9be0 2023-07-18T03:13:28.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.349+0000 7f170452fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2644fb1a0 con 0x55e264504ad0 2023-07-18T03:13:28.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.350+0000 7f16ead9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e264504ad0 0x55e2645014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.350+0000 7f16ead9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e264504ad0 0x55e2645014d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:57192/0 (socket says 172.21.15.101:57192) 2023-07-18T03:13:28.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.350+0000 7f16ead9d700 1 -- 172.21.15.101:0/4152539643 learned_addr learned my addr 172.21.15.101:0/4152539643 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:13:28.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.350+0000 7f16ea59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e2644f92d0 0x55e264504080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.350+0000 7f16e9d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e2644f9be0 0x55e264500f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.350+0000 7f16ead9d700 1 -- 172.21.15.101:0/4152539643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e2644f9be0 msgr2=0x55e264500f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:28.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.350+0000 7f16ead9d700 1 --2- 172.21.15.101:0/4152539643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e2644f9be0 0x55e264500f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.350+0000 7f16ead9d700 1 -- 172.21.15.101:0/4152539643 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e2644f92d0 msgr2=0x55e264504080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:28.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.350+0000 7f16ead9d700 1 --2- 172.21.15.101:0/4152539643 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e2644f92d0 0x55e264504080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.350+0000 7f16ead9d700 1 -- 172.21.15.101:0/4152539643 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e264262c60 con 0x55e264504ad0 2023-07-18T03:13:28.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.350+0000 7f16ea59c700 1 --2- 172.21.15.101:0/4152539643 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e2644f92d0 0x55e264504080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:13:28.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.350+0000 7f16ead9d700 1 --2- 172.21.15.101:0/4152539643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e264504ad0 0x55e2645014d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f16e4004970 tx=0x7f16e4004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3da447f562502ffe server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:13:28.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.351+0000 7f16d97fa700 1 -- 172.21.15.101:0/4152539643 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e4029070 con 0x55e264504ad0 2023-07-18T03:13:28.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.351+0000 7f16d97fa700 1 -- 172.21.15.101:0/4152539643 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16e4021040 con 0x55e264504ad0 2023-07-18T03:13:28.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.351+0000 7f16d97fa700 1 -- 172.21.15.101:0/4152539643 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e4020070 con 0x55e264504ad0 2023-07-18T03:13:28.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.351+0000 7f170452fb80 1 -- 172.21.15.101:0/4152539643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e264504ad0 msgr2=0x55e2645014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:28.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.351+0000 7f170452fb80 1 --2- 172.21.15.101:0/4152539643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e264504ad0 0x55e2645014d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f16e4004970 tx=0x7f16e4004b40 comp rx=0 tx=0).stop 2023-07-18T03:13:28.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.351+0000 7f170452fb80 1 -- 172.21.15.101:0/4152539643 shutdown_connections 2023-07-18T03:13:28.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.351+0000 7f170452fb80 1 --2- 172.21.15.101:0/4152539643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e264504ad0 0x55e2645014d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.351+0000 7f170452fb80 1 --2- 172.21.15.101:0/4152539643 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e2644f92d0 0x55e264504080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.348 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.351+0000 7f170452fb80 1 --2- 172.21.15.101:0/4152539643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e2644f9be0 0x55e264500f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.348 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.351+0000 7f170452fb80 1 -- 172.21.15.101:0/4152539643 >> 172.21.15.101:0/4152539643 conn(0x55e2644efda0 msgr2=0x55e2644fca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:13:28.348 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.351+0000 7f170452fb80 1 -- 172.21.15.101:0/4152539643 shutdown_connections 2023-07-18T03:13:28.348 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.352+0000 7f170452fb80 1 -- 172.21.15.101:0/4152539643 wait complete. 2023-07-18T03:13:28.348 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.352+0000 7f170452fb80 1 -- start start 2023-07-18T03:13:28.348 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.352+0000 7f170452fb80 1 Processor -- start 2023-07-18T03:13:28.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.352+0000 7f170452fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e2644f92d0 0x55e26450b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.353+0000 7f170452fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e2644f9be0 0x55e2645095c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.353+0000 7f170452fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e264504ad0 0x55e264509b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.353+0000 7f170452fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e26450a040 con 0x55e264504ad0 2023-07-18T03:13:28.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.353+0000 7f170452fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e26450a1b0 con 0x55e2644f92d0 2023-07-18T03:13:28.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.353+0000 7f170452fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e26450a370 con 0x55e2644f9be0 2023-07-18T03:13:28.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.353+0000 7f170452fb80 10 client.0 _ll_register_callbacks cb 0x7f16e0099990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:13:28.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.353+0000 7f16ea59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e2644f92d0 0x55e26450b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.353+0000 7f16ead9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e264504ad0 0x55e264509b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.353+0000 7f16e9d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e2644f9be0 0x55e2645095c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.353+0000 7f16ea59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e2644f92d0 0x55e26450b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:50438/0 (socket says 172.21.15.101:50438) 2023-07-18T03:13:28.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.353+0000 7f16ea59c700 1 -- 172.21.15.101:0/1738171962 learned_addr learned my addr 172.21.15.101:0/1738171962 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:13:28.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.353+0000 7f16ea59c700 1 -- 172.21.15.101:0/1738171962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e2644f9be0 msgr2=0x55e2645095c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:28.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.353+0000 7f16ea59c700 1 --2- 172.21.15.101:0/1738171962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e2644f9be0 0x55e2645095c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.353+0000 7f16ea59c700 1 -- 172.21.15.101:0/1738171962 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e264504ad0 msgr2=0x55e264509b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:28.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.353+0000 7f16ea59c700 1 --2- 172.21.15.101:0/1738171962 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e264504ad0 0x55e264509b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.353+0000 7f16ea59c700 1 -- 172.21.15.101:0/1738171962 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e264279de0 con 0x55e2644f92d0 2023-07-18T03:13:28.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.353+0000 7f16ead9d700 1 --2- 172.21.15.101:0/1738171962 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e264504ad0 0x55e264509b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:13:28.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.353+0000 7f16ea59c700 1 --2- 172.21.15.101:0/1738171962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e2644f92d0 0x55e26450b290 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f16dc005f50 tx=0x7f16dc009320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9bcc5394981832 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:13:28.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.353+0000 7f16d1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:13:28.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.354+0000 7f16d1ffb700 1 -- 172.21.15.101:0/1738171962 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16dc032070 con 0x55e2644f92d0 2023-07-18T03:13:28.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.354+0000 7f170452fb80 1 -- 172.21.15.101:0/1738171962 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f16dc01e040 con 0x55e2644f92d0 2023-07-18T03:13:28.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.354+0000 7f16d1ffb700 1 -- 172.21.15.101:0/1738171962 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f16dc0344e0 con 0x55e2644f92d0 2023-07-18T03:13:28.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.354+0000 7f16d1ffb700 1 -- 172.21.15.101:0/1738171962 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16dc01d040 con 0x55e2644f92d0 2023-07-18T03:13:28.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.354+0000 7f170452fb80 10 client.26756 fetch_fsmap learned FSMap version 26 2023-07-18T03:13:28.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.354+0000 7f170452fb80 1 -- 172.21.15.101:0/1738171962 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f16e4004050 con 0x55e2644f92d0 2023-07-18T03:13:28.355 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.354+0000 7f16d1ffb700 1 -- 172.21.15.101:0/1738171962 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16dc01e040 con 0x55e2644f92d0 2023-07-18T03:13:28.355 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.354+0000 7f16d1ffb700 1 -- 172.21.15.101:0/1738171962 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f16dc005510 con 0x55e2644f92d0 2023-07-18T03:13:28.355 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.354+0000 7f170452fb80 10 client.26756 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:13:28.355 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.354+0000 7f170452fb80 10 client.26756 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:13:28.355 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.354+0000 7f170452fb80 20 client.26756 populate_metadata read hostname 'smithi101' 2023-07-18T03:13:28.355 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.354+0000 7f170452fb80 1 --2- 172.21.15.101:0/1738171962 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55e2645691e0 0x55e26456b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.355 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.355+0000 7f170452fb80 4 client.26756 mds_command: new command op to 24541 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:13:28.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.355+0000 7f170452fb80 1 -- 172.21.15.101:0/1738171962 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e264407050 con 0x55e2645691e0 2023-07-18T03:13:28.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.355+0000 7f16e9d9b700 1 --2- 172.21.15.101:0/1738171962 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55e2645691e0 0x55e26456b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.355+0000 7f16e9d9b700 1 --2- 172.21.15.101:0/1738171962 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55e2645691e0 0x55e26456b5f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c03c4d1f6ce87da2 server_cookie=4bfa2317f644c178 in_seq=0 out_seq=0 2023-07-18T03:13:28.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.355+0000 7f16d1ffb700 10 client.26756 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:13:28.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.355+0000 7f16d1ffb700 1 -- 172.21.15.101:0/1738171962 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e264407050 con 0x55e2645691e0 2023-07-18T03:13:28.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.355+0000 7f16d1ffb700 10 client.26756 handle_command_reply: tid=0 2023-07-18T03:13:28.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.355+0000 7f170452fb80 1 client.26756 shutdown 2023-07-18T03:13:28.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.356+0000 7f170452fb80 1 -- 172.21.15.101:0/1738171962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e2644f92d0 msgr2=0x55e26450b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:28.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.356+0000 7f170452fb80 1 --2- 172.21.15.101:0/1738171962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e2644f92d0 0x55e26450b290 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f16dc005f50 tx=0x7f16dc009320 comp rx=0 tx=0).stop 2023-07-18T03:13:28.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.356+0000 7f170452fb80 1 -- 172.21.15.101:0/1738171962 shutdown_connections 2023-07-18T03:13:28.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.356+0000 7f170452fb80 1 --2- 172.21.15.101:0/1738171962 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55e2645691e0 0x55e26456b5f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.356+0000 7f170452fb80 1 --2- 172.21.15.101:0/1738171962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e2644f92d0 0x55e26450b290 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.356+0000 7f170452fb80 1 --2- 172.21.15.101:0/1738171962 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e264504ad0 0x55e264509b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.356+0000 7f16d1ffb700 0 client.26756 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:13:28.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.356+0000 7f170452fb80 1 --2- 172.21.15.101:0/1738171962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e2644f9be0 0x55e2645095c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.356+0000 7f170452fb80 1 -- 172.21.15.101:0/1738171962 >> 172.21.15.101:0/1738171962 conn(0x55e2644efda0 msgr2=0x55e2644fb3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:13:28.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.356+0000 7f170452fb80 1 -- 172.21.15.101:0/1738171962 shutdown_connections 2023-07-18T03:13:28.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.356+0000 7f170452fb80 1 -- 172.21.15.101:0/1738171962 wait complete. 2023-07-18T03:13:28.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.356+0000 7f170452fb80 20 client.26756 trim_cache size 0 max 16384 2023-07-18T03:13:28.359 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:13:28.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.356+0000 7f16f0bff700 1 -- 172.21.15.101:0/391087265 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f16d40a12c0 msgr2=0x7f16d40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:28.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.356+0000 7f16f0bff700 1 --2- 172.21.15.101:0/391087265 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f16d40a12c0 0x7f16d40a3770 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f16e40043a0 tx=0x7f16e4009920 comp rx=0 tx=0).stop 2023-07-18T03:13:28.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.356+0000 7f16f0bff700 1 -- 172.21.15.101:0/391087265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16ec100490 msgr2=0x7f16ec198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:28.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.356+0000 7f16f0bff700 1 --2- 172.21.15.101:0/391087265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16ec100490 0x7f16ec198330 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f16e000e9a0 tx=0x7f16e000ecb0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.357+0000 7f16f0bff700 1 -- 172.21.15.101:0/391087265 shutdown_connections 2023-07-18T03:13:28.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.357+0000 7f16f0bff700 1 --2- 172.21.15.101:0/391087265 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f16ec105af0 0x7f16ec19c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.357+0000 7f16f0bff700 1 --2- 172.21.15.101:0/391087265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16ec0ffae0 0x7f16ec197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.357+0000 7f16f0bff700 1 --2- 172.21.15.101:0/391087265 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f16d40a12c0 0x7f16d40a3770 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.361 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.357+0000 7f16f0bff700 1 --2- 172.21.15.101:0/391087265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16ec100490 0x7f16ec198330 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.361 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.357+0000 7f16f0bff700 1 -- 172.21.15.101:0/391087265 >> 172.21.15.101:0/391087265 conn(0x7f16ec0fb5e0 msgr2=0x7f16ec10aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:13:28.361 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.357+0000 7f16f0bff700 1 -- 172.21.15.101:0/391087265 shutdown_connections 2023-07-18T03:13:28.361 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.357+0000 7f16f0bff700 1 -- 172.21.15.101:0/391087265 wait complete. 2023-07-18T03:13:28.369 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-18T03:13:28.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.587+0000 7fea3a9ff700 1 -- 172.21.15.101:0/1528889144 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea340ffae0 msgr2=0x7fea340fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:28.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.587+0000 7fea3a9ff700 1 --2- 172.21.15.101:0/1528889144 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea340ffae0 0x7fea340fff50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fea28009990 tx=0x7fea28009ca0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.581 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.587+0000 7fea3a9ff700 1 -- 172.21.15.101:0/1528889144 shutdown_connections 2023-07-18T03:13:28.581 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.587+0000 7fea3a9ff700 1 --2- 172.21.15.101:0/1528889144 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea34100490 0x7fea3410c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.581 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.587+0000 7fea3a9ff700 1 --2- 172.21.15.101:0/1528889144 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fea34105af0 0x7fea34105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.581 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.587+0000 7fea3a9ff700 1 --2- 172.21.15.101:0/1528889144 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea340ffae0 0x7fea340fff50 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.581 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.587+0000 7fea3a9ff700 1 -- 172.21.15.101:0/1528889144 >> 172.21.15.101:0/1528889144 conn(0x7fea340fb5e0 msgr2=0x7fea340fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:13:28.581 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.587+0000 7fea3a9ff700 1 -- 172.21.15.101:0/1528889144 shutdown_connections 2023-07-18T03:13:28.581 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.587+0000 7fea3a9ff700 1 -- 172.21.15.101:0/1528889144 wait complete. 2023-07-18T03:13:28.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.588+0000 7fea3a9ff700 1 Processor -- start 2023-07-18T03:13:28.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.588+0000 7fea3a9ff700 1 -- start start 2023-07-18T03:13:28.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.589+0000 7fea3a9ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea340ffae0 0x7fea34072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.589+0000 7fea3a9ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fea34100490 0x7fea34072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.589+0000 7fea3a9ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea34105af0 0x7fea341a3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.589+0000 7fea3a9ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea34073580 con 0x7fea34100490 2023-07-18T03:13:28.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.589+0000 7fea3a9ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea341a4320 con 0x7fea34105af0 2023-07-18T03:13:28.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.589+0000 7fea3a9ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea341a4460 con 0x7fea340ffae0 2023-07-18T03:13:28.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.589+0000 7fea33fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea340ffae0 0x7fea34072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.589+0000 7fea337fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fea34100490 0x7fea34072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.589+0000 7fea337fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fea34100490 0x7fea34072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:41530/0 (socket says 172.21.15.101:41530) 2023-07-18T03:13:28.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.589+0000 7fea337fe700 1 -- 172.21.15.101:0/1495841191 learned_addr learned my addr 172.21.15.101:0/1495841191 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:13:28.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.589+0000 7fea33fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea340ffae0 0x7fea34072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:57218/0 (socket says 172.21.15.101:57218) 2023-07-18T03:13:28.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.589+0000 7fea38f9c700 1 --2- 172.21.15.101:0/1495841191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea34105af0 0x7fea341a3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.589+0000 7fea33fff700 1 -- 172.21.15.101:0/1495841191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea34105af0 msgr2=0x7fea341a3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:28.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.589+0000 7fea33fff700 1 --2- 172.21.15.101:0/1495841191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea34105af0 0x7fea341a3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.589+0000 7fea33fff700 1 -- 172.21.15.101:0/1495841191 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fea34100490 msgr2=0x7fea34072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:28.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.589+0000 7fea33fff700 1 --2- 172.21.15.101:0/1495841191 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fea34100490 0x7fea34072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.589+0000 7fea33fff700 1 -- 172.21.15.101:0/1495841191 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea280095f0 con 0x7fea340ffae0 2023-07-18T03:13:28.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.590+0000 7fea38f9c700 1 --2- 172.21.15.101:0/1495841191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea34105af0 0x7fea341a3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:13:28.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.590+0000 7fea337fe700 1 --2- 172.21.15.101:0/1495841191 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fea34100490 0x7fea34072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:13:28.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.590+0000 7fea33fff700 1 --2- 172.21.15.101:0/1495841191 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea340ffae0 0x7fea34072810 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fea2400e9f0 tx=0x7fea2400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:13:28.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.590+0000 7fea317fa700 1 -- 172.21.15.101:0/1495841191 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2400c9c0 con 0x7fea340ffae0 2023-07-18T03:13:28.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.590+0000 7fea317fa700 1 -- 172.21.15.101:0/1495841191 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea24010430 con 0x7fea340ffae0 2023-07-18T03:13:28.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.590+0000 7fea3a9ff700 1 -- 172.21.15.101:0/1495841191 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea341a46c0 con 0x7fea340ffae0 2023-07-18T03:13:28.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.590+0000 7fea317fa700 1 -- 172.21.15.101:0/1495841191 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea24018680 con 0x7fea340ffae0 2023-07-18T03:13:28.587 INFO:tasks.workunit.client.0.smithi101.stdout: 1 456915 8.59 MB/sec execute 173 sec latency 14.621 ms 2023-07-18T03:13:28.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.590+0000 7fea3a9ff700 1 -- 172.21.15.101:0/1495841191 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea341a4ba0 con 0x7fea340ffae0 2023-07-18T03:13:28.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.595+0000 7fea317fa700 1 -- 172.21.15.101:0/1495841191 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fea24010860 con 0x7fea340ffae0 2023-07-18T03:13:28.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.596+0000 7fea4e34ab80 1 Processor -- start 2023-07-18T03:13:28.589 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.596+0000 7fea4e34ab80 1 -- start start 2023-07-18T03:13:28.589 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.596+0000 7fea4e34ab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55de3f39f2d0 0x55de3f39f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.596+0000 7fea4e34ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de3f39fbe0 0x55de3f3aa590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.596+0000 7fea4e34ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de3f3aaad0 0x55de3f3aceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.596+0000 7fea4e34ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de3f2a5cb0 con 0x55de3f3aaad0 2023-07-18T03:13:28.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.596+0000 7fea4e34ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de3f2f8a40 con 0x55de3f39fbe0 2023-07-18T03:13:28.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.596+0000 7fea4e34ab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de3f2f1ed0 con 0x55de3f39f2d0 2023-07-18T03:13:28.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.596+0000 7fea38f9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55de3f39f2d0 0x55de3f39f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.596+0000 7fea38f9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55de3f39f2d0 0x55de3f39f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:57224/0 (socket says 172.21.15.101:57224) 2023-07-18T03:13:28.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.596+0000 7fea38f9c700 1 -- 172.21.15.101:0/120505958 learned_addr learned my addr 172.21.15.101:0/120505958 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:13:28.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.596+0000 7fea33fff700 1 --2- 172.21.15.101:0/120505958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de3f39fbe0 0x55de3f3aa590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.596+0000 7fea337fe700 1 --2- 172.21.15.101:0/120505958 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de3f3aaad0 0x55de3f3aceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.597+0000 7fea317fa700 1 --2- 172.21.15.101:0/1495841191 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fea1c0a12c0 0x7fea1c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.597+0000 7fea38f9c700 1 -- 172.21.15.101:0/120505958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de3f39fbe0 msgr2=0x55de3f3aa590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:28.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.597+0000 7fea38f9c700 1 --2- 172.21.15.101:0/120505958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de3f39fbe0 0x55de3f3aa590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.597+0000 7fea38f9c700 1 -- 172.21.15.101:0/120505958 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de3f3aaad0 msgr2=0x55de3f3aceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:28.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.597+0000 7fea38f9c700 1 --2- 172.21.15.101:0/120505958 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de3f3aaad0 0x55de3f3aceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.597+0000 7fea38f9c700 1 -- 172.21.15.101:0/120505958 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de3f11fde0 con 0x55de3f39f2d0 2023-07-18T03:13:28.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.597+0000 7fea337fe700 1 --2- 172.21.15.101:0/120505958 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de3f3aaad0 0x55de3f3aceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:13:28.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.597+0000 7fea317fa700 1 -- 172.21.15.101:0/1495841191 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fea24014070 con 0x7fea340ffae0 2023-07-18T03:13:28.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.597+0000 7fea38f9c700 1 --2- 172.21.15.101:0/120505958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55de3f39f2d0 0x55de3f39f6a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fea2c003e40 tx=0x7fea2c009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9caff2c9541bb703 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:13:28.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.597+0000 7fea38f9c700 1 --2- 172.21.15.101:0/1495841191 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fea1c0a12c0 0x7fea1c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.597+0000 7fea1a7fc700 1 -- 172.21.15.101:0/120505958 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2c0090d0 con 0x55de3f39f2d0 2023-07-18T03:13:28.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.597+0000 7fea1a7fc700 1 -- 172.21.15.101:0/120505958 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea2c00f040 con 0x55de3f39f2d0 2023-07-18T03:13:28.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.597+0000 7fea1a7fc700 1 -- 172.21.15.101:0/120505958 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2c00e8f0 con 0x55de3f39f2d0 2023-07-18T03:13:28.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.598+0000 7fea38f9c700 1 --2- 172.21.15.101:0/1495841191 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fea1c0a12c0 0x7fea1c0a3770 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fea2c007fd0 tx=0x7fea2c00eb10 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:13:28.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.598+0000 7fea4e34ab80 1 -- 172.21.15.101:0/120505958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55de3f39f2d0 msgr2=0x55de3f39f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:28.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.598+0000 7fea4e34ab80 1 --2- 172.21.15.101:0/120505958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55de3f39f2d0 0x55de3f39f6a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fea2c003e40 tx=0x7fea2c009920 comp rx=0 tx=0).stop 2023-07-18T03:13:28.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.598+0000 7fea4e34ab80 1 -- 172.21.15.101:0/120505958 shutdown_connections 2023-07-18T03:13:28.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.598+0000 7fea4e34ab80 1 --2- 172.21.15.101:0/120505958 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de3f3aaad0 0x55de3f3aceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.598+0000 7fea4e34ab80 1 --2- 172.21.15.101:0/120505958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55de3f39f2d0 0x55de3f39f6a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.598+0000 7fea4e34ab80 1 --2- 172.21.15.101:0/120505958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de3f39fbe0 0x55de3f3aa590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.598+0000 7fea4e34ab80 1 -- 172.21.15.101:0/120505958 >> 172.21.15.101:0/120505958 conn(0x55de3f395da0 msgr2=0x55de3f39b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:13:28.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.598+0000 7fea4e34ab80 1 -- 172.21.15.101:0/120505958 shutdown_connections 2023-07-18T03:13:28.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.599+0000 7fea4e34ab80 1 -- 172.21.15.101:0/120505958 wait complete. 2023-07-18T03:13:28.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.599+0000 7fea4e34ab80 1 -- start start 2023-07-18T03:13:28.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.599+0000 7fea4e34ab80 1 Processor -- start 2023-07-18T03:13:28.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.600+0000 7fea4e34ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de3f39f2d0 0x55de3f3aa540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.600+0000 7fea4e34ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de3f39fbe0 0x55de3f3a7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.600+0000 7fea4e34ab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55de3f3aaad0 0x55de3f3a7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.600+0000 7fea4e34ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de3f291ac0 con 0x55de3f39fbe0 2023-07-18T03:13:28.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.600+0000 7fea4e34ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de3f2917c0 con 0x55de3f39f2d0 2023-07-18T03:13:28.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.600+0000 7fea4e34ab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de3f175920 con 0x55de3f3aaad0 2023-07-18T03:13:28.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.600+0000 7fea4e34ab80 10 client.0 _ll_register_callbacks cb 0x55de3f28f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:13:28.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.600+0000 7fea33fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de3f39f2d0 0x55de3f3aa540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.600+0000 7fea38f9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55de3f3aaad0 0x55de3f3a7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.600+0000 7fea33fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de3f39f2d0 0x55de3f3aa540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:50478/0 (socket says 172.21.15.101:50478) 2023-07-18T03:13:28.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.600+0000 7fea33fff700 1 -- 172.21.15.101:0/3050636339 learned_addr learned my addr 172.21.15.101:0/3050636339 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:13:28.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.600+0000 7fea38f9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55de3f3aaad0 0x55de3f3a7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:57240/0 (socket says 172.21.15.101:57240) 2023-07-18T03:13:28.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.601+0000 7fea337fe700 1 --2- 172.21.15.101:0/3050636339 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de3f39fbe0 0x55de3f3a7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.601+0000 7fea33fff700 1 -- 172.21.15.101:0/3050636339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55de3f3aaad0 msgr2=0x55de3f3a7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:28.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.601+0000 7fea33fff700 1 --2- 172.21.15.101:0/3050636339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55de3f3aaad0 0x55de3f3a7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.601+0000 7fea33fff700 1 -- 172.21.15.101:0/3050636339 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de3f39fbe0 msgr2=0x55de3f3a7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:28.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.601+0000 7fea33fff700 1 --2- 172.21.15.101:0/3050636339 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de3f39fbe0 0x55de3f3a7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.601+0000 7fea33fff700 1 -- 172.21.15.101:0/3050636339 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea2c003b20 con 0x55de3f39f2d0 2023-07-18T03:13:28.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.601+0000 7fea38f9c700 1 --2- 172.21.15.101:0/3050636339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55de3f3aaad0 0x55de3f3a7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:13:28.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.601+0000 7fea33fff700 1 --2- 172.21.15.101:0/3050636339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de3f39f2d0 0x55de3f3aa540 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fea2400e9c0 tx=0x7fea240d0690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8672c78c194ca7ec server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:13:28.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.601+0000 7fea137fe700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:13:28.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.601+0000 7fea137fe700 1 -- 172.21.15.101:0/3050636339 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea24099a10 con 0x55de3f39f2d0 2023-07-18T03:13:28.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.601+0000 7fea137fe700 1 -- 172.21.15.101:0/3050636339 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fea24099ed0 con 0x55de3f39f2d0 2023-07-18T03:13:28.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.601+0000 7fea4e34ab80 1 -- 172.21.15.101:0/3050636339 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55de3f24b4f0 con 0x55de3f39f2d0 2023-07-18T03:13:28.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.602+0000 7fea137fe700 1 -- 172.21.15.101:0/3050636339 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea24099830 con 0x55de3f39f2d0 2023-07-18T03:13:28.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.602+0000 7fea4e34ab80 10 client.26768 fetch_fsmap learned FSMap version 26 2023-07-18T03:13:28.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.602+0000 7fea4e34ab80 1 -- 172.21.15.101:0/3050636339 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55de3f108c60 con 0x55de3f39f2d0 2023-07-18T03:13:28.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.602+0000 7fea137fe700 1 -- 172.21.15.101:0/3050636339 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea240b1070 con 0x55de3f39f2d0 2023-07-18T03:13:28.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.602+0000 7fea137fe700 1 -- 172.21.15.101:0/3050636339 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fea240993c0 con 0x55de3f39f2d0 2023-07-18T03:13:28.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.602+0000 7fea4e34ab80 10 client.26768 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:13:28.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.602+0000 7fea4e34ab80 10 client.26768 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:13:28.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.602+0000 7fea4e34ab80 20 client.26768 populate_metadata read hostname 'smithi101' 2023-07-18T03:13:28.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.602+0000 7fea4e34ab80 1 --2- 172.21.15.101:0/3050636339 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55de3f3ba510 0x55de3f3bc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.602+0000 7fea4e34ab80 4 client.26768 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:13:28.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.602+0000 7fea4e34ab80 1 -- 172.21.15.101:0/3050636339 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55de3f2eaeb0 con 0x55de3f3ba510 2023-07-18T03:13:28.604 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.602+0000 7fea337fe700 1 --2- 172.21.15.101:0/3050636339 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55de3f3ba510 0x55de3f3bc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.603+0000 7fea337fe700 1 --2- 172.21.15.101:0/3050636339 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55de3f3ba510 0x55de3f3bc920 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=221e6bdd6dfdeb2d server_cookie=9b2e62ae97ea7e4f in_seq=0 out_seq=0 2023-07-18T03:13:28.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.603+0000 7fea137fe700 10 client.26768 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:13:28.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.603+0000 7fea137fe700 1 -- 172.21.15.101:0/3050636339 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55de3f2eaeb0 con 0x55de3f3ba510 2023-07-18T03:13:28.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.603+0000 7fea137fe700 10 client.26768 handle_command_reply: tid=0 2023-07-18T03:13:28.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.604+0000 7fea4e34ab80 1 client.26768 shutdown 2023-07-18T03:13:28.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.604+0000 7fea4e34ab80 1 -- 172.21.15.101:0/3050636339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de3f39f2d0 msgr2=0x55de3f3aa540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:28.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.604+0000 7fea4e34ab80 1 --2- 172.21.15.101:0/3050636339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de3f39f2d0 0x55de3f3aa540 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fea2400e9c0 tx=0x7fea240d0690 comp rx=0 tx=0).stop 2023-07-18T03:13:28.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.604+0000 7fea4e34ab80 1 -- 172.21.15.101:0/3050636339 shutdown_connections 2023-07-18T03:13:28.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.604+0000 7fea4e34ab80 1 --2- 172.21.15.101:0/3050636339 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55de3f3ba510 0x55de3f3bc920 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.604+0000 7fea4e34ab80 1 --2- 172.21.15.101:0/3050636339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de3f39f2d0 0x55de3f3aa540 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.604+0000 7fea137fe700 0 client.26768 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:13:28.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.604+0000 7fea4e34ab80 1 --2- 172.21.15.101:0/3050636339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55de3f3aaad0 0x55de3f3a7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.604+0000 7fea4e34ab80 1 --2- 172.21.15.101:0/3050636339 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de3f39fbe0 0x55de3f3a7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.604+0000 7fea4e34ab80 1 -- 172.21.15.101:0/3050636339 >> 172.21.15.101:0/3050636339 conn(0x55de3f395da0 msgr2=0x55de3f39b230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:13:28.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.604+0000 7fea4e34ab80 1 -- 172.21.15.101:0/3050636339 shutdown_connections 2023-07-18T03:13:28.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.604+0000 7fea4e34ab80 1 -- 172.21.15.101:0/3050636339 wait complete. 2023-07-18T03:13:28.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.604+0000 7fea4e34ab80 20 client.26768 trim_cache size 0 max 16384 2023-07-18T03:13:28.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.618+0000 7fea4e34ab80 1 Processor -- start 2023-07-18T03:13:28.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.618+0000 7fea4e34ab80 1 -- start start 2023-07-18T03:13:28.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.618+0000 7fea4e34ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de3f39f2d0 0x55de3f3aa070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.618+0000 7fea4e34ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de3f39fbe0 0x55de3f3a6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.618+0000 7fea4e34ab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55de3f3aaad0 0x55de3f3a7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.618+0000 7fea4e34ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de3f392ef0 con 0x55de3f39fbe0 2023-07-18T03:13:28.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.618+0000 7fea4e34ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de3f3a7a70 con 0x55de3f39f2d0 2023-07-18T03:13:28.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.618+0000 7fea4e34ab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de3f39cc70 con 0x55de3f3aaad0 2023-07-18T03:13:28.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.618+0000 7fea33fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de3f39f2d0 0x55de3f3aa070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.618+0000 7fea33fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de3f39f2d0 0x55de3f3aa070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:50492/0 (socket says 172.21.15.101:50492) 2023-07-18T03:13:28.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.618+0000 7fea337fe700 1 --2- 172.21.15.101:0/1161218183 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de3f39fbe0 0x55de3f3a6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.618+0000 7fea33fff700 1 -- 172.21.15.101:0/1161218183 learned_addr learned my addr 172.21.15.101:0/1161218183 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:13:28.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.618+0000 7fea38f9c700 1 --2- 172.21.15.101:0/1161218183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55de3f3aaad0 0x55de3f3a7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.619+0000 7fea33fff700 1 -- 172.21.15.101:0/1161218183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55de3f3aaad0 msgr2=0x55de3f3a7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:28.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.619+0000 7fea33fff700 1 --2- 172.21.15.101:0/1161218183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55de3f3aaad0 0x55de3f3a7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.619+0000 7fea33fff700 1 -- 172.21.15.101:0/1161218183 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de3f39fbe0 msgr2=0x55de3f3a6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:28.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.619+0000 7fea33fff700 1 --2- 172.21.15.101:0/1161218183 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de3f39fbe0 0x55de3f3a6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.619+0000 7fea33fff700 1 -- 172.21.15.101:0/1161218183 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de3f146250 con 0x55de3f39f2d0 2023-07-18T03:13:28.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.619+0000 7fea33fff700 1 --2- 172.21.15.101:0/1161218183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de3f39f2d0 0x55de3f3aa070 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fea24099e60 tx=0x7fea240cd160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c46969ec37afbc2d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:13:28.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.619+0000 7fea1affd700 1 -- 172.21.15.101:0/1161218183 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea240d3ee0 con 0x55de3f39f2d0 2023-07-18T03:13:28.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.619+0000 7fea1affd700 1 -- 172.21.15.101:0/1161218183 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea240b1030 con 0x55de3f39f2d0 2023-07-18T03:13:28.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.619+0000 7fea1affd700 1 -- 172.21.15.101:0/1161218183 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2409a400 con 0x55de3f39f2d0 2023-07-18T03:13:28.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.620+0000 7fea4e34ab80 1 -- 172.21.15.101:0/1161218183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de3f39f2d0 msgr2=0x55de3f3aa070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:28.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.620+0000 7fea4e34ab80 1 --2- 172.21.15.101:0/1161218183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de3f39f2d0 0x55de3f3aa070 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fea24099e60 tx=0x7fea240cd160 comp rx=0 tx=0).stop 2023-07-18T03:13:28.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.620+0000 7fea4e34ab80 1 -- 172.21.15.101:0/1161218183 shutdown_connections 2023-07-18T03:13:28.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.620+0000 7fea4e34ab80 1 --2- 172.21.15.101:0/1161218183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de3f39f2d0 0x55de3f3aa070 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.620+0000 7fea4e34ab80 1 --2- 172.21.15.101:0/1161218183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55de3f3aaad0 0x55de3f3a7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.620+0000 7fea4e34ab80 1 --2- 172.21.15.101:0/1161218183 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de3f39fbe0 0x55de3f3a6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.620+0000 7fea4e34ab80 1 -- 172.21.15.101:0/1161218183 >> 172.21.15.101:0/1161218183 conn(0x55de3f395da0 msgr2=0x55de3f3a2a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:13:28.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.620+0000 7fea4e34ab80 1 -- 172.21.15.101:0/1161218183 shutdown_connections 2023-07-18T03:13:28.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.620+0000 7fea4e34ab80 1 -- 172.21.15.101:0/1161218183 wait complete. 2023-07-18T03:13:28.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.620+0000 7fea4e34ab80 1 -- start start 2023-07-18T03:13:28.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.621+0000 7fea4e34ab80 1 Processor -- start 2023-07-18T03:13:28.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.621+0000 7fea4e34ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de3f39f2d0 0x55de3f3af3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.621+0000 7fea4e34ab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55de3f39fbe0 0x55de3f3af8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.621+0000 7fea4e34ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de3f3aaad0 0x55de3f40d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.621+0000 7fea4e34ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de3f40d910 con 0x55de3f3aaad0 2023-07-18T03:13:28.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.621+0000 7fea4e34ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de3f40da50 con 0x55de3f39f2d0 2023-07-18T03:13:28.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.621+0000 7fea4e34ab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de3f40dbc0 con 0x55de3f39fbe0 2023-07-18T03:13:28.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.621+0000 7fea4e34ab80 10 client.0 _ll_register_callbacks cb 0x7fea24010b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:13:28.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.622+0000 7fea337fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55de3f39fbe0 0x55de3f3af8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.622+0000 7fea33fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de3f39f2d0 0x55de3f3af3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.622+0000 7fea337fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55de3f39fbe0 0x55de3f3af8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:57266/0 (socket says 172.21.15.101:57266) 2023-07-18T03:13:28.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.622+0000 7fea337fe700 1 -- 172.21.15.101:0/80810002 learned_addr learned my addr 172.21.15.101:0/80810002 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:13:28.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.622+0000 7fea38f9c700 1 --2- 172.21.15.101:0/80810002 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de3f3aaad0 0x55de3f40d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.622+0000 7fea33fff700 1 -- 172.21.15.101:0/80810002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55de3f39fbe0 msgr2=0x55de3f3af8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:28.628 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.622+0000 7fea33fff700 1 --2- 172.21.15.101:0/80810002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55de3f39fbe0 0x55de3f3af8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.628 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.622+0000 7fea33fff700 1 -- 172.21.15.101:0/80810002 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de3f3aaad0 msgr2=0x55de3f40d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:28.628 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.622+0000 7fea33fff700 1 --2- 172.21.15.101:0/80810002 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de3f3aaad0 0x55de3f40d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.628 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.622+0000 7fea33fff700 1 -- 172.21.15.101:0/80810002 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55de3f108c60 con 0x55de3f39f2d0 2023-07-18T03:13:28.628 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.622+0000 7fea337fe700 1 --2- 172.21.15.101:0/80810002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55de3f39fbe0 0x55de3f3af8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:13:28.629 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.622+0000 7fea33fff700 1 --2- 172.21.15.101:0/80810002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de3f39f2d0 0x55de3f3af3b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fea240cab50 tx=0x7fea2400c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3939921b0068cacd server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:13:28.629 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.622+0000 7fea137fe700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:13:28.629 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.622+0000 7fea137fe700 1 -- 172.21.15.101:0/80810002 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2409a400 con 0x55de3f39f2d0 2023-07-18T03:13:28.629 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.622+0000 7fea137fe700 1 -- 172.21.15.101:0/80810002 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fea240a2320 con 0x55de3f39f2d0 2023-07-18T03:13:28.629 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.622+0000 7fea4e34ab80 1 -- 172.21.15.101:0/80810002 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fea2c00f040 con 0x55de3f39f2d0 2023-07-18T03:13:28.629 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.623+0000 7fea137fe700 1 -- 172.21.15.101:0/80810002 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea24099ed0 con 0x55de3f39f2d0 2023-07-18T03:13:28.629 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.623+0000 7fea4e34ab80 10 client.26774 fetch_fsmap learned FSMap version 26 2023-07-18T03:13:28.629 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.623+0000 7fea4e34ab80 1 -- 172.21.15.101:0/80810002 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55de3f11e2b0 con 0x55de3f39f2d0 2023-07-18T03:13:28.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.623+0000 7fea137fe700 1 -- 172.21.15.101:0/80810002 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea2c00f040 con 0x55de3f39f2d0 2023-07-18T03:13:28.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.623+0000 7fea137fe700 1 -- 172.21.15.101:0/80810002 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fea240a1e90 con 0x55de3f39f2d0 2023-07-18T03:13:28.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.623+0000 7fea4e34ab80 10 client.26774 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:13:28.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.623+0000 7fea4e34ab80 10 client.26774 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:13:28.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.623+0000 7fea4e34ab80 20 client.26774 populate_metadata read hostname 'smithi101' 2023-07-18T03:13:28.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.623+0000 7fea4e34ab80 1 --2- 172.21.15.101:0/80810002 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55de3f40ee10 0x55de3f4111f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.623+0000 7fea4e34ab80 4 client.26774 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:13:28.631 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.623+0000 7fea4e34ab80 1 -- 172.21.15.101:0/80810002 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55de3f2ad050 con 0x55de3f40ee10 2023-07-18T03:13:28.631 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.623+0000 7fea337fe700 1 --2- 172.21.15.101:0/80810002 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55de3f40ee10 0x55de3f4111f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.631 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.624+0000 7fea337fe700 1 --2- 172.21.15.101:0/80810002 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55de3f40ee10 0x55de3f4111f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=52ab3fea8736d4ae server_cookie=677c8f30a748e7ea in_seq=0 out_seq=0 2023-07-18T03:13:28.631 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.624+0000 7fea137fe700 10 client.26774 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:13:28.631 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.624+0000 7fea137fe700 1 -- 172.21.15.101:0/80810002 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55de3f2ad050 con 0x55de3f40ee10 2023-07-18T03:13:28.631 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.624+0000 7fea137fe700 10 client.26774 handle_command_reply: tid=0 2023-07-18T03:13:28.631 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.624+0000 7fea4e34ab80 1 client.26774 shutdown 2023-07-18T03:13:28.631 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.624+0000 7fea4e34ab80 1 -- 172.21.15.101:0/80810002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de3f39f2d0 msgr2=0x55de3f3af3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:28.632 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.624+0000 7fea4e34ab80 1 --2- 172.21.15.101:0/80810002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de3f39f2d0 0x55de3f3af3b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fea240cab50 tx=0x7fea2400c6e0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.632 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.624+0000 7fea4e34ab80 1 -- 172.21.15.101:0/80810002 shutdown_connections 2023-07-18T03:13:28.632 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.624+0000 7fea4e34ab80 1 --2- 172.21.15.101:0/80810002 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55de3f40ee10 0x55de3f4111f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.632 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.624+0000 7fea4e34ab80 1 --2- 172.21.15.101:0/80810002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de3f39f2d0 0x55de3f3af3b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.632 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.624+0000 7fea4e34ab80 1 --2- 172.21.15.101:0/80810002 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de3f3aaad0 0x55de3f40d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.632 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.624+0000 7fea137fe700 0 client.26774 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:13:28.632 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.624+0000 7fea4e34ab80 1 --2- 172.21.15.101:0/80810002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55de3f39fbe0 0x55de3f3af8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.624+0000 7fea4e34ab80 1 -- 172.21.15.101:0/80810002 >> 172.21.15.101:0/80810002 conn(0x55de3f395da0 msgr2=0x55de3f3a6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:13:28.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.625+0000 7fea4e34ab80 1 -- 172.21.15.101:0/80810002 shutdown_connections 2023-07-18T03:13:28.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.625+0000 7fea4e34ab80 1 -- 172.21.15.101:0/80810002 wait complete. 2023-07-18T03:13:28.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.625+0000 7fea4e34ab80 20 client.26774 trim_cache size 0 max 16384 2023-07-18T03:13:28.633 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:13:28.634 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.625+0000 7fea3a9ff700 1 -- 172.21.15.101:0/1495841191 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fea1c0a12c0 msgr2=0x7fea1c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:28.634 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.625+0000 7fea3a9ff700 1 --2- 172.21.15.101:0/1495841191 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fea1c0a12c0 0x7fea1c0a3770 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fea2c007fd0 tx=0x7fea2c00eb10 comp rx=0 tx=0).stop 2023-07-18T03:13:28.634 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.625+0000 7fea3a9ff700 1 -- 172.21.15.101:0/1495841191 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea340ffae0 msgr2=0x7fea34072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:28.635 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.625+0000 7fea3a9ff700 1 --2- 172.21.15.101:0/1495841191 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea340ffae0 0x7fea34072810 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fea2400e9f0 tx=0x7fea2400ed00 comp rx=0 tx=0).stop 2023-07-18T03:13:28.635 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.625+0000 7fea3a9ff700 1 -- 172.21.15.101:0/1495841191 shutdown_connections 2023-07-18T03:13:28.635 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.625+0000 7fea3a9ff700 1 --2- 172.21.15.101:0/1495841191 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fea1c0a12c0 0x7fea1c0a3770 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.635 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.625+0000 7fea3a9ff700 1 --2- 172.21.15.101:0/1495841191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea34105af0 0x7fea341a3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.635 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.625+0000 7fea3a9ff700 1 --2- 172.21.15.101:0/1495841191 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea340ffae0 0x7fea34072810 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.635 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.625+0000 7fea3a9ff700 1 --2- 172.21.15.101:0/1495841191 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fea34100490 0x7fea34072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.635 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.625+0000 7fea3a9ff700 1 -- 172.21.15.101:0/1495841191 >> 172.21.15.101:0/1495841191 conn(0x7fea340fb5e0 msgr2=0x7fea340fbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:13:28.636 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.626+0000 7fea3a9ff700 1 -- 172.21.15.101:0/1495841191 shutdown_connections 2023-07-18T03:13:28.636 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.626+0000 7fea3a9ff700 1 -- 172.21.15.101:0/1495841191 wait complete. 2023-07-18T03:13:28.638 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-18T03:13:28.851 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.858+0000 7f24ecfff700 1 -- 172.21.15.101:0/213177791 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f24e81015b0 msgr2=0x7f24e8109aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:28.852 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.858+0000 7f24ecfff700 1 --2- 172.21.15.101:0/213177791 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f24e81015b0 0x7f24e8109aa0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f24d0009990 tx=0x7f24d0009ca0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.852 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.858+0000 7f24ecfff700 1 -- 172.21.15.101:0/213177791 shutdown_connections 2023-07-18T03:13:28.852 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.858+0000 7f24ecfff700 1 --2- 172.21.15.101:0/213177791 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24e810a070 0x7f24e810c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.852 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.858+0000 7f24ecfff700 1 --2- 172.21.15.101:0/213177791 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24e8100ca0 0x7f24e8101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.852 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.858+0000 7f24ecfff700 1 --2- 172.21.15.101:0/213177791 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f24e81015b0 0x7f24e8109aa0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.852 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.858+0000 7f24ecfff700 1 -- 172.21.15.101:0/213177791 >> 172.21.15.101:0/213177791 conn(0x7f24e80fa510 msgr2=0x7f24e80fc920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:13:28.853 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.858+0000 7f24ecfff700 1 -- 172.21.15.101:0/213177791 shutdown_connections 2023-07-18T03:13:28.853 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.858+0000 7f24ecfff700 1 -- 172.21.15.101:0/213177791 wait complete. 2023-07-18T03:13:28.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.859+0000 7f24ecfff700 1 Processor -- start 2023-07-18T03:13:28.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.860+0000 7f24ecfff700 1 -- start start 2023-07-18T03:13:28.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.860+0000 7f24ecfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f24e8100ca0 0x7f24e8197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.860+0000 7f24ecfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24e81015b0 0x7f24e81983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.860+0000 7f24ecfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24e810a070 0x7f24e819c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.860+0000 7f24ecfff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e819ce80 con 0x7f24e8100ca0 2023-07-18T03:13:28.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.860+0000 7f24ecfff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e819cff0 con 0x7f24e810a070 2023-07-18T03:13:28.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.860+0000 7f24ecfff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e819d1b0 con 0x7f24e81015b0 2023-07-18T03:13:28.856 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.860+0000 7f24dffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24e81015b0 0x7f24e81983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.856 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.860+0000 7f24e659c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f24e8100ca0 0x7f24e8197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.856 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.860+0000 7f24dffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24e81015b0 0x7f24e81983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:57276/0 (socket says 172.21.15.101:57276) 2023-07-18T03:13:28.856 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.860+0000 7f24dffff700 1 -- 172.21.15.101:0/3549645646 learned_addr learned my addr 172.21.15.101:0/3549645646 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:13:28.856 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.860+0000 7f24e6d9d700 1 --2- 172.21.15.101:0/3549645646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24e810a070 0x7f24e819c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.856 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.861+0000 7f24dffff700 1 -- 172.21.15.101:0/3549645646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24e810a070 msgr2=0x7f24e819c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:28.857 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.861+0000 7f24dffff700 1 --2- 172.21.15.101:0/3549645646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24e810a070 0x7f24e819c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.857 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.861+0000 7f24dffff700 1 -- 172.21.15.101:0/3549645646 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f24e8100ca0 msgr2=0x7f24e8197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:28.857 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.861+0000 7f24dffff700 1 --2- 172.21.15.101:0/3549645646 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f24e8100ca0 0x7f24e8197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.857 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.861+0000 7f24dffff700 1 -- 172.21.15.101:0/3549645646 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d00095f0 con 0x7f24e81015b0 2023-07-18T03:13:28.857 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.861+0000 7f24e659c700 1 --2- 172.21.15.101:0/3549645646 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f24e8100ca0 0x7f24e8197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:13:28.857 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.861+0000 7f24dffff700 1 --2- 172.21.15.101:0/3549645646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24e81015b0 0x7f24e81983e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f24d0005910 tx=0x7f24d0005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:13:28.857 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.861+0000 7f24df7fe700 1 -- 172.21.15.101:0/3549645646 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d001d070 con 0x7f24e81015b0 2023-07-18T03:13:28.857 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.861+0000 7f24ecfff700 1 -- 172.21.15.101:0/3549645646 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24e819d430 con 0x7f24e81015b0 2023-07-18T03:13:28.858 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.861+0000 7f24df7fe700 1 -- 172.21.15.101:0/3549645646 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24d0022470 con 0x7f24e81015b0 2023-07-18T03:13:28.858 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.861+0000 7f24df7fe700 1 -- 172.21.15.101:0/3549645646 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d0017650 con 0x7f24e81015b0 2023-07-18T03:13:28.858 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.861+0000 7f24ecfff700 1 -- 172.21.15.101:0/3549645646 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24e81a4cd0 con 0x7f24e81015b0 2023-07-18T03:13:28.858 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.865+0000 7f24df7fe700 1 -- 172.21.15.101:0/3549645646 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f24d000fb40 con 0x7f24e81015b0 2023-07-18T03:13:28.859 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.866+0000 7f250095eb80 1 Processor -- start 2023-07-18T03:13:28.859 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.867+0000 7f24df7fe700 1 --2- 172.21.15.101:0/3549645646 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f24d40a1340 0x7f24d40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.860 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.867+0000 7f250095eb80 1 -- start start 2023-07-18T03:13:28.861 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.867+0000 7f24df7fe700 1 -- 172.21.15.101:0/3549645646 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f24d00ce340 con 0x7f24e81015b0 2023-07-18T03:13:28.861 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.867+0000 7f24e6d9d700 1 --2- 172.21.15.101:0/3549645646 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f24d40a1340 0x7f24d40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.867+0000 7f250095eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a263112d0 0x560a263116a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.867+0000 7f250095eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a26311be0 0x560a2631c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.867+0000 7f250095eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a2631cad0 0x560a2631eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.867+0000 7f250095eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a26217cb0 con 0x560a2631cad0 2023-07-18T03:13:28.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.867+0000 7f250095eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a2626aa40 con 0x560a26311be0 2023-07-18T03:13:28.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.867+0000 7f250095eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a26263ed0 con 0x560a263112d0 2023-07-18T03:13:28.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.867+0000 7f24e659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a263112d0 0x560a263116a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.867+0000 7f24e659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a263112d0 0x560a263116a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:57278/0 (socket says 172.21.15.101:57278) 2023-07-18T03:13:28.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.867+0000 7f24e659c700 1 -- 172.21.15.101:0/94161522 learned_addr learned my addr 172.21.15.101:0/94161522 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:13:28.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.868+0000 7f24dffff700 1 --2- 172.21.15.101:0/94161522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a26311be0 0x560a2631c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.868+0000 7f24e6d9d700 1 --2- 172.21.15.101:0/94161522 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a2631cad0 0x560a2631eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.868+0000 7f24e659c700 1 -- 172.21.15.101:0/94161522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a26311be0 msgr2=0x560a2631c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:28.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.868+0000 7f24e659c700 1 --2- 172.21.15.101:0/94161522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a26311be0 0x560a2631c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.868+0000 7f24e659c700 1 -- 172.21.15.101:0/94161522 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a2631cad0 msgr2=0x560a2631eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:28.864 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.868+0000 7f24e659c700 1 --2- 172.21.15.101:0/94161522 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a2631cad0 0x560a2631eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.864 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.868+0000 7f24e659c700 1 -- 172.21.15.101:0/94161522 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a26091de0 con 0x560a263112d0 2023-07-18T03:13:28.864 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.868+0000 7f24e6d9d700 1 --2- 172.21.15.101:0/3549645646 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f24d40a1340 0x7f24d40a37f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f24e000f590 tx=0x7f24e0009920 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:13:28.864 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.868+0000 7f24e659c700 1 --2- 172.21.15.101:0/94161522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a263112d0 0x560a263116a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f24d80115f0 tx=0x7f24d8011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb3e7b92194f83b3 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:13:28.864 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.868+0000 7f24dcff9700 1 -- 172.21.15.101:0/94161522 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d8019410 con 0x560a263112d0 2023-07-18T03:13:28.864 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.868+0000 7f24dcff9700 1 -- 172.21.15.101:0/94161522 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24d8019a30 con 0x560a263112d0 2023-07-18T03:13:28.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.868+0000 7f24dcff9700 1 -- 172.21.15.101:0/94161522 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d8018ac0 con 0x560a263112d0 2023-07-18T03:13:28.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.869+0000 7f250095eb80 1 -- 172.21.15.101:0/94161522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a263112d0 msgr2=0x560a263116a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:28.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.869+0000 7f250095eb80 1 --2- 172.21.15.101:0/94161522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a263112d0 0x560a263116a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f24d80115f0 tx=0x7f24d8011900 comp rx=0 tx=0).stop 2023-07-18T03:13:28.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.869+0000 7f250095eb80 1 -- 172.21.15.101:0/94161522 shutdown_connections 2023-07-18T03:13:28.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.869+0000 7f250095eb80 1 --2- 172.21.15.101:0/94161522 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a2631cad0 0x560a2631eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.869+0000 7f250095eb80 1 --2- 172.21.15.101:0/94161522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a263112d0 0x560a263116a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.869+0000 7f250095eb80 1 --2- 172.21.15.101:0/94161522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a26311be0 0x560a2631c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.869+0000 7f250095eb80 1 -- 172.21.15.101:0/94161522 >> 172.21.15.101:0/94161522 conn(0x560a26307da0 msgr2=0x560a2630d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:13:28.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.869+0000 7f250095eb80 1 -- 172.21.15.101:0/94161522 shutdown_connections 2023-07-18T03:13:28.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.869+0000 7f250095eb80 1 -- 172.21.15.101:0/94161522 wait complete. 2023-07-18T03:13:28.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.870+0000 7f250095eb80 1 -- start start 2023-07-18T03:13:28.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.870+0000 7f250095eb80 1 Processor -- start 2023-07-18T03:13:28.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.871+0000 7f250095eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a263112d0 0x560a2631c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.871+0000 7f250095eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a26311be0 0x560a263196e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.871+0000 7f250095eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a2631cad0 0x560a26319cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.871+0000 7f250095eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a26203ac0 con 0x560a263112d0 2023-07-18T03:13:28.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.871+0000 7f250095eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a262037c0 con 0x560a2631cad0 2023-07-18T03:13:28.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.871+0000 7f250095eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a260e7920 con 0x560a26311be0 2023-07-18T03:13:28.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.871+0000 7f250095eb80 10 client.0 _ll_register_callbacks cb 0x560a26201760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:13:28.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.871+0000 7f24dffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a26311be0 0x560a263196e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.871+0000 7f24dffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a26311be0 0x560a263196e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:57282/0 (socket says 172.21.15.101:57282) 2023-07-18T03:13:28.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.871+0000 7f24dffff700 1 -- 172.21.15.101:0/1412988551 learned_addr learned my addr 172.21.15.101:0/1412988551 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:13:28.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.871+0000 7f24e6d9d700 1 --2- 172.21.15.101:0/1412988551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a2631cad0 0x560a26319cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.871+0000 7f24e659c700 1 --2- 172.21.15.101:0/1412988551 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a263112d0 0x560a2631c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.871+0000 7f24dffff700 1 -- 172.21.15.101:0/1412988551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a2631cad0 msgr2=0x560a26319cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:28.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.871+0000 7f24dffff700 1 --2- 172.21.15.101:0/1412988551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a2631cad0 0x560a26319cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.871+0000 7f24dffff700 1 -- 172.21.15.101:0/1412988551 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a263112d0 msgr2=0x560a2631c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:28.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.871+0000 7f24dffff700 1 --2- 172.21.15.101:0/1412988551 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a263112d0 0x560a2631c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.871+0000 7f24dffff700 1 -- 172.21.15.101:0/1412988551 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24d8009dd0 con 0x560a26311be0 2023-07-18T03:13:28.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.872+0000 7f24dffff700 1 --2- 172.21.15.101:0/1412988551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a26311be0 0x560a263196e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f24d000f740 tx=0x7f24d009acb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ad7491e6ca61b80 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:13:28.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.872+0000 7f24cdffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:13:28.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.872+0000 7f24cdffb700 1 -- 172.21.15.101:0/1412988551 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d009a7e0 con 0x560a26311be0 2023-07-18T03:13:28.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.872+0000 7f24cdffb700 1 -- 172.21.15.101:0/1412988551 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f24d009b0f0 con 0x560a26311be0 2023-07-18T03:13:28.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.872+0000 7f250095eb80 1 -- 172.21.15.101:0/1412988551 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560a261bd4f0 con 0x560a26311be0 2023-07-18T03:13:28.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.872+0000 7f24cdffb700 1 -- 172.21.15.101:0/1412988551 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24d009f120 con 0x560a26311be0 2023-07-18T03:13:28.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.872+0000 7f250095eb80 10 client.26815 fetch_fsmap learned FSMap version 26 2023-07-18T03:13:28.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.872+0000 7f250095eb80 1 -- 172.21.15.101:0/1412988551 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x560a2607ac60 con 0x560a26311be0 2023-07-18T03:13:28.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.872+0000 7f24cdffb700 1 -- 172.21.15.101:0/1412988551 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560a261bd4f0 con 0x560a26311be0 2023-07-18T03:13:28.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.872+0000 7f24cdffb700 1 -- 172.21.15.101:0/1412988551 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f24d00225e0 con 0x560a26311be0 2023-07-18T03:13:28.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.872+0000 7f250095eb80 10 client.26815 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:13:28.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.872+0000 7f250095eb80 10 client.26815 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:13:28.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.872+0000 7f250095eb80 20 client.26815 populate_metadata read hostname 'smithi101' 2023-07-18T03:13:28.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.872+0000 7f250095eb80 1 --2- 172.21.15.101:0/1412988551 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x560a2632bf60 0x560a2632e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.872+0000 7f250095eb80 4 client.26815 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:13:28.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.872+0000 7f250095eb80 1 -- 172.21.15.101:0/1412988551 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560a2625ceb0 con 0x560a2632bf60 2023-07-18T03:13:28.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.873+0000 7f24e659c700 1 --2- 172.21.15.101:0/1412988551 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x560a2632bf60 0x560a2632e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.873+0000 7f24cdffb700 10 client.26815 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:13:28.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.873+0000 7f24e659c700 1 --2- 172.21.15.101:0/1412988551 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x560a2632bf60 0x560a2632e340 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a804721ad7d0dcb server_cookie=15e9c90bf7759999 in_seq=0 out_seq=0 2023-07-18T03:13:28.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.874+0000 7f24cdffb700 1 -- 172.21.15.101:0/1412988551 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560a2625ceb0 con 0x560a2632bf60 2023-07-18T03:13:28.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.874+0000 7f24cdffb700 10 client.26815 handle_command_reply: tid=0 2023-07-18T03:13:28.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.874+0000 7f250095eb80 1 client.26815 shutdown 2023-07-18T03:13:28.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.874+0000 7f250095eb80 1 -- 172.21.15.101:0/1412988551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a26311be0 msgr2=0x560a263196e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:28.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.874+0000 7f250095eb80 1 --2- 172.21.15.101:0/1412988551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a26311be0 0x560a263196e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f24d000f740 tx=0x7f24d009acb0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.874+0000 7f250095eb80 1 -- 172.21.15.101:0/1412988551 shutdown_connections 2023-07-18T03:13:28.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.874+0000 7f250095eb80 1 --2- 172.21.15.101:0/1412988551 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x560a2632bf60 0x560a2632e340 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.874+0000 7f250095eb80 1 --2- 172.21.15.101:0/1412988551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a2631cad0 0x560a26319cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.874+0000 7f24cdffb700 0 client.26815 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:13:28.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.874+0000 7f250095eb80 1 --2- 172.21.15.101:0/1412988551 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a263112d0 0x560a2631c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.874+0000 7f250095eb80 1 --2- 172.21.15.101:0/1412988551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a26311be0 0x560a263196e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.874+0000 7f250095eb80 1 -- 172.21.15.101:0/1412988551 >> 172.21.15.101:0/1412988551 conn(0x560a26307da0 msgr2=0x560a2630d290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:13:28.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.875+0000 7f250095eb80 1 -- 172.21.15.101:0/1412988551 shutdown_connections 2023-07-18T03:13:28.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.875+0000 7f250095eb80 1 -- 172.21.15.101:0/1412988551 wait complete. 2023-07-18T03:13:28.877 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.875+0000 7f250095eb80 20 client.26815 trim_cache size 0 max 16384 2023-07-18T03:13:28.882 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.889+0000 7f250095eb80 1 Processor -- start 2023-07-18T03:13:28.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.890+0000 7f250095eb80 1 -- start start 2023-07-18T03:13:28.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.890+0000 7f250095eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a263112d0 0x560a2631c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.890+0000 7f250095eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a26311be0 0x560a26318ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.890+0000 7f250095eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a2631cad0 0x560a26319430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.890+0000 7f250095eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a26312d60 con 0x560a263112d0 2023-07-18T03:13:28.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.890+0000 7f250095eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a263131a0 con 0x560a26311be0 2023-07-18T03:13:28.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.890+0000 7f250095eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a263121c0 con 0x560a2631cad0 2023-07-18T03:13:28.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.890+0000 7f24e659c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a263112d0 0x560a2631c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.890+0000 7f24e659c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a263112d0 0x560a2631c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:41632/0 (socket says 172.21.15.101:41632) 2023-07-18T03:13:28.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.890+0000 7f24e659c700 1 -- 172.21.15.101:0/1723605191 learned_addr learned my addr 172.21.15.101:0/1723605191 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:13:28.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.890+0000 7f24dffff700 1 --2- 172.21.15.101:0/1723605191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a26311be0 0x560a26318ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.890+0000 7f24e6d9d700 1 --2- 172.21.15.101:0/1723605191 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a2631cad0 0x560a26319430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.891+0000 7f24dffff700 1 -- 172.21.15.101:0/1723605191 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a2631cad0 msgr2=0x560a26319430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:28.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.891+0000 7f24dffff700 1 --2- 172.21.15.101:0/1723605191 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a2631cad0 0x560a26319430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.891+0000 7f24dffff700 1 -- 172.21.15.101:0/1723605191 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a263112d0 msgr2=0x560a2631c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:28.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.891+0000 7f24dffff700 1 --2- 172.21.15.101:0/1723605191 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a263112d0 0x560a2631c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.891+0000 7f24dffff700 1 -- 172.21.15.101:0/1723605191 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a260b8250 con 0x560a26311be0 2023-07-18T03:13:28.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.891+0000 7f24e659c700 1 --2- 172.21.15.101:0/1723605191 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a263112d0 0x560a2631c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:13:28.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.891+0000 7f24dffff700 1 --2- 172.21.15.101:0/1723605191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a26311be0 0x560a26318ef0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f24d0099440 tx=0x7f24d00996a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d100e2bbde61330 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:13:28.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.891+0000 7f24dd7fa700 1 -- 172.21.15.101:0/1723605191 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d0017ea0 con 0x560a26311be0 2023-07-18T03:13:28.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.891+0000 7f24dd7fa700 1 -- 172.21.15.101:0/1723605191 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24d009b0f0 con 0x560a26311be0 2023-07-18T03:13:28.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.891+0000 7f24dd7fa700 1 -- 172.21.15.101:0/1723605191 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d009a820 con 0x560a26311be0 2023-07-18T03:13:28.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.892+0000 7f250095eb80 1 -- 172.21.15.101:0/1723605191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a26311be0 msgr2=0x560a26318ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:28.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.892+0000 7f250095eb80 1 --2- 172.21.15.101:0/1723605191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a26311be0 0x560a26318ef0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f24d0099440 tx=0x7f24d00996a0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.892+0000 7f250095eb80 1 -- 172.21.15.101:0/1723605191 shutdown_connections 2023-07-18T03:13:28.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.892+0000 7f250095eb80 1 --2- 172.21.15.101:0/1723605191 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a2631cad0 0x560a26319430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.892+0000 7f250095eb80 1 --2- 172.21.15.101:0/1723605191 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a263112d0 0x560a2631c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.892+0000 7f250095eb80 1 --2- 172.21.15.101:0/1723605191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a26311be0 0x560a26318ef0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.892+0000 7f250095eb80 1 -- 172.21.15.101:0/1723605191 >> 172.21.15.101:0/1723605191 conn(0x560a26307da0 msgr2=0x560a26314af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:13:28.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.892+0000 7f250095eb80 1 -- 172.21.15.101:0/1723605191 shutdown_connections 2023-07-18T03:13:28.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.892+0000 7f250095eb80 1 -- 172.21.15.101:0/1723605191 wait complete. 2023-07-18T03:13:28.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.893+0000 7f250095eb80 1 -- start start 2023-07-18T03:13:28.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.893+0000 7f250095eb80 1 Processor -- start 2023-07-18T03:13:28.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.893+0000 7f250095eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a263112d0 0x560a263216f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.893+0000 7f250095eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a26311be0 0x560a26321c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.894+0000 7f250095eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a2631cad0 0x560a2637edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.894+0000 7f250095eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a263223b0 con 0x560a26311be0 2023-07-18T03:13:28.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.894+0000 7f250095eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a2637f360 con 0x560a2631cad0 2023-07-18T03:13:28.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.894+0000 7f250095eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a2637f520 con 0x560a263112d0 2023-07-18T03:13:28.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.894+0000 7f250095eb80 10 client.0 _ll_register_callbacks cb 0x7f24d0097430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:13:28.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.894+0000 7f24e659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a263112d0 0x560a263216f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.894+0000 7f24e659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a263112d0 0x560a263216f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:57312/0 (socket says 172.21.15.101:57312) 2023-07-18T03:13:28.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.894+0000 7f24e659c700 1 -- 172.21.15.101:0/2371063056 learned_addr learned my addr 172.21.15.101:0/2371063056 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:13:28.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.894+0000 7f24dffff700 1 --2- 172.21.15.101:0/2371063056 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a26311be0 0x560a26321c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.894+0000 7f24e659c700 1 -- 172.21.15.101:0/2371063056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a2631cad0 msgr2=0x560a2637edf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-18T03:13:28.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.894+0000 7f24e659c700 1 --2- 172.21.15.101:0/2371063056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a2631cad0 0x560a2637edf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.894+0000 7f24e659c700 1 -- 172.21.15.101:0/2371063056 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a26311be0 msgr2=0x560a26321c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:28.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.894+0000 7f24e659c700 1 --2- 172.21.15.101:0/2371063056 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a26311be0 0x560a26321c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.894+0000 7f24e659c700 1 -- 172.21.15.101:0/2371063056 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560a26091de0 con 0x560a263112d0 2023-07-18T03:13:28.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.894+0000 7f24e659c700 1 --2- 172.21.15.101:0/2371063056 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a263112d0 0x560a263216f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f24d8011fd0 tx=0x7f24d8019d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0d6bc22605a79ff server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:13:28.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.894+0000 7f24cdffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:13:28.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.895+0000 7f24cdffb700 1 -- 172.21.15.101:0/2371063056 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d8019ea0 con 0x560a263112d0 2023-07-18T03:13:28.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.895+0000 7f250095eb80 1 -- 172.21.15.101:0/2371063056 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f24d8019a30 con 0x560a263112d0 2023-07-18T03:13:28.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.895+0000 7f24cdffb700 1 -- 172.21.15.101:0/2371063056 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f24d8005ce0 con 0x560a263112d0 2023-07-18T03:13:28.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.895+0000 7f24cdffb700 1 -- 172.21.15.101:0/2371063056 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24d8029da0 con 0x560a263112d0 2023-07-18T03:13:28.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.895+0000 7f250095eb80 10 client.26821 fetch_fsmap learned FSMap version 26 2023-07-18T03:13:28.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.895+0000 7f250095eb80 1 -- 172.21.15.101:0/2371063056 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x560a2607ac60 con 0x560a263112d0 2023-07-18T03:13:28.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.895+0000 7f24cdffb700 1 -- 172.21.15.101:0/2371063056 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24d8019a30 con 0x560a263112d0 2023-07-18T03:13:28.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.895+0000 7f24cdffb700 1 -- 172.21.15.101:0/2371063056 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f24d802f020 con 0x560a263112d0 2023-07-18T03:13:28.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.895+0000 7f250095eb80 10 client.26821 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:13:28.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.895+0000 7f250095eb80 10 client.26821 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:13:28.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.895+0000 7f250095eb80 20 client.26821 populate_metadata read hostname 'smithi101' 2023-07-18T03:13:28.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.895+0000 7f250095eb80 1 --2- 172.21.15.101:0/2371063056 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x560a26380bd0 0x560a26382fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:28.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.895+0000 7f250095eb80 4 client.26821 mds_command: new command op to 24562 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:13:28.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.895+0000 7f250095eb80 1 -- 172.21.15.101:0/2371063056 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560a2621f050 con 0x560a26380bd0 2023-07-18T03:13:28.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.896+0000 7f24dffff700 1 --2- 172.21.15.101:0/2371063056 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x560a26380bd0 0x560a26382fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:28.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.896+0000 7f24dffff700 1 --2- 172.21.15.101:0/2371063056 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x560a26380bd0 0x560a26382fb0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15939a33c49386d1 server_cookie=58813d8e37dd1813 in_seq=0 out_seq=0 2023-07-18T03:13:28.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.896+0000 7f24cdffb700 10 client.26821 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:13:28.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.897+0000 7f24cdffb700 1 -- 172.21.15.101:0/2371063056 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560a2621f050 con 0x560a26380bd0 2023-07-18T03:13:28.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.897+0000 7f24cdffb700 10 client.26821 handle_command_reply: tid=0 2023-07-18T03:13:28.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.897+0000 7f250095eb80 1 client.26821 shutdown 2023-07-18T03:13:28.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.897+0000 7f250095eb80 1 -- 172.21.15.101:0/2371063056 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a263112d0 msgr2=0x560a263216f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:28.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.897+0000 7f250095eb80 1 --2- 172.21.15.101:0/2371063056 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a263112d0 0x560a263216f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f24d8011fd0 tx=0x7f24d8019d30 comp rx=0 tx=0).stop 2023-07-18T03:13:28.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.897+0000 7f250095eb80 1 -- 172.21.15.101:0/2371063056 shutdown_connections 2023-07-18T03:13:28.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.897+0000 7f250095eb80 1 --2- 172.21.15.101:0/2371063056 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x560a26380bd0 0x560a26382fb0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.897+0000 7f250095eb80 1 --2- 172.21.15.101:0/2371063056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a2631cad0 0x560a2637edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.897+0000 7f250095eb80 1 --2- 172.21.15.101:0/2371063056 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a263112d0 0x560a263216f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.897+0000 7f24cdffb700 0 client.26821 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:13:28.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.897+0000 7f250095eb80 1 --2- 172.21.15.101:0/2371063056 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a26311be0 0x560a26321c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.897+0000 7f250095eb80 1 -- 172.21.15.101:0/2371063056 >> 172.21.15.101:0/2371063056 conn(0x560a26307da0 msgr2=0x560a263133e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:13:28.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.897+0000 7f250095eb80 1 -- 172.21.15.101:0/2371063056 shutdown_connections 2023-07-18T03:13:28.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.897+0000 7f250095eb80 1 -- 172.21.15.101:0/2371063056 wait complete. 2023-07-18T03:13:28.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.897+0000 7f250095eb80 20 client.26821 trim_cache size 0 max 16384 2023-07-18T03:13:28.898 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:13:28.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.898+0000 7f24ecfff700 1 -- 172.21.15.101:0/3549645646 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f24d40a1340 msgr2=0x7f24d40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:28.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.898+0000 7f24ecfff700 1 --2- 172.21.15.101:0/3549645646 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f24d40a1340 0x7f24d40a37f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f24e000f590 tx=0x7f24e0009920 comp rx=0 tx=0).stop 2023-07-18T03:13:28.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.898+0000 7f24ecfff700 1 -- 172.21.15.101:0/3549645646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24e81015b0 msgr2=0x7f24e81983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:28.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.898+0000 7f24ecfff700 1 --2- 172.21.15.101:0/3549645646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24e81015b0 0x7f24e81983e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f24d0005910 tx=0x7f24d0005720 comp rx=0 tx=0).stop 2023-07-18T03:13:28.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.898+0000 7f24ecfff700 1 -- 172.21.15.101:0/3549645646 shutdown_connections 2023-07-18T03:13:28.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.898+0000 7f24ecfff700 1 --2- 172.21.15.101:0/3549645646 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f24d40a1340 0x7f24d40a37f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.898+0000 7f24ecfff700 1 --2- 172.21.15.101:0/3549645646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24e810a070 0x7f24e819c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.898+0000 7f24ecfff700 1 --2- 172.21.15.101:0/3549645646 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f24e8100ca0 0x7f24e8197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.898+0000 7f24ecfff700 1 --2- 172.21.15.101:0/3549645646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24e81015b0 0x7f24e81983e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:28.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.898+0000 7f24ecfff700 1 -- 172.21.15.101:0/3549645646 >> 172.21.15.101:0/3549645646 conn(0x7f24e80fa510 msgr2=0x7f24e810c190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:13:28.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.898+0000 7f24ecfff700 1 -- 172.21.15.101:0/3549645646 shutdown_connections 2023-07-18T03:13:28.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:28.898+0000 7f24ecfff700 1 -- 172.21.15.101:0/3549645646 wait complete. 2023-07-18T03:13:29.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:29 smithi101 ceph-mon[126379]: pgmap v180: 129 pgs: 129 active+clean; 289 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 191 op/s 2023-07-18T03:13:29.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:29 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/608018886' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:13:29.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:29 smithi155 ceph-mon[128885]: pgmap v180: 129 pgs: 129 active+clean; 289 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 191 op/s 2023-07-18T03:13:29.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:29 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/608018886' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:13:29.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:29 smithi179 ceph-mon[129229]: pgmap v180: 129 pgs: 129 active+clean; 289 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 191 op/s 2023-07-18T03:13:29.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:29 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/608018886' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:13:29.583 INFO:tasks.workunit.client.0.smithi101.stdout: 1 457978 8.58 MB/sec execute 174 sec latency 55.996 ms 2023-07-18T03:13:29.912 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-18T03:13:29.912 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-18T03:13:30.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.130+0000 7f27defff700 1 -- 172.21.15.101:0/2105575557 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f27d8105ae0 msgr2=0x7f27d8105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:30.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.130+0000 7f27defff700 1 --2- 172.21.15.101:0/2105575557 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f27d8105ae0 0x7f27d8105eb0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f27cc009990 tx=0x7f27cc009ca0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.130+0000 7f27defff700 1 -- 172.21.15.101:0/2105575557 shutdown_connections 2023-07-18T03:13:30.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.130+0000 7f27defff700 1 --2- 172.21.15.101:0/2105575557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27d8100480 0x7f27d810c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.130+0000 7f27defff700 1 --2- 172.21.15.101:0/2105575557 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f27d8105ae0 0x7f27d8105eb0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.130+0000 7f27defff700 1 --2- 172.21.15.101:0/2105575557 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27d80ffad0 0x7f27d80fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.130+0000 7f27defff700 1 -- 172.21.15.101:0/2105575557 >> 172.21.15.101:0/2105575557 conn(0x7f27d80fb5f0 msgr2=0x7f27d80fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:13:30.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.130+0000 7f27defff700 1 -- 172.21.15.101:0/2105575557 shutdown_connections 2023-07-18T03:13:30.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.131+0000 7f27defff700 1 -- 172.21.15.101:0/2105575557 wait complete. 2023-07-18T03:13:30.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.131+0000 7f27defff700 1 Processor -- start 2023-07-18T03:13:30.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.132+0000 7f27defff700 1 -- start start 2023-07-18T03:13:30.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.132+0000 7f27defff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f27d80ffad0 0x7f27d8197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:30.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.132+0000 7f27defff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27d8100480 0x7f27d8198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:30.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.132+0000 7f27defff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27d8105ae0 0x7f27d819c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:30.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.132+0000 7f27defff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d819cdf0 con 0x7f27d80ffad0 2023-07-18T03:13:30.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.132+0000 7f27defff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d819cf60 con 0x7f27d8100480 2023-07-18T03:13:30.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.132+0000 7f27defff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d819d120 con 0x7f27d8105ae0 2023-07-18T03:13:30.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.132+0000 7f27dcd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f27d80ffad0 0x7f27d8197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:30.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.132+0000 7f27dd59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27d8105ae0 0x7f27d819c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:30.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.132+0000 7f27dcd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f27d80ffad0 0x7f27d8197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:41658/0 (socket says 172.21.15.101:41658) 2023-07-18T03:13:30.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.132+0000 7f27d7fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27d8100480 0x7f27d8198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:30.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.132+0000 7f27dcd9b700 1 -- 172.21.15.101:0/1233723008 learned_addr learned my addr 172.21.15.101:0/1233723008 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:13:30.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.133+0000 7f27dcd9b700 1 -- 172.21.15.101:0/1233723008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27d8105ae0 msgr2=0x7f27d819c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:30.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.133+0000 7f27dcd9b700 1 --2- 172.21.15.101:0/1233723008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27d8105ae0 0x7f27d819c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.133+0000 7f27dcd9b700 1 -- 172.21.15.101:0/1233723008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27d8100480 msgr2=0x7f27d8198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:30.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.133+0000 7f27dcd9b700 1 --2- 172.21.15.101:0/1233723008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27d8100480 0x7f27d8198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.133+0000 7f27dcd9b700 1 -- 172.21.15.101:0/1233723008 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27cc0095f0 con 0x7f27d80ffad0 2023-07-18T03:13:30.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.133+0000 7f27dcd9b700 1 --2- 172.21.15.101:0/1233723008 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f27d80ffad0 0x7f27d8197e10 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f27cc000c00 tx=0x7f27cc005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:13:30.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.133+0000 7f27d5ffb700 1 -- 172.21.15.101:0/1233723008 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27cc01d070 con 0x7f27d80ffad0 2023-07-18T03:13:30.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.133+0000 7f27d5ffb700 1 -- 172.21.15.101:0/1233723008 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27cc022470 con 0x7f27d80ffad0 2023-07-18T03:13:30.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.133+0000 7f27defff700 1 -- 172.21.15.101:0/1233723008 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27d819d3a0 con 0x7f27d80ffad0 2023-07-18T03:13:30.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.133+0000 7f27d5ffb700 1 -- 172.21.15.101:0/1233723008 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27cc017650 con 0x7f27d80ffad0 2023-07-18T03:13:30.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.133+0000 7f27defff700 1 -- 172.21.15.101:0/1233723008 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27d81a4c40 con 0x7f27d80ffad0 2023-07-18T03:13:30.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.136+0000 7f27d5ffb700 1 -- 172.21.15.101:0/1233723008 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f27cc0225e0 con 0x7f27d80ffad0 2023-07-18T03:13:30.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.138+0000 7f27f2912b80 1 Processor -- start 2023-07-18T03:13:30.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.138+0000 7f27d5ffb700 1 --2- 172.21.15.101:0/1233723008 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f27c80a1340 0x7f27c80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:30.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.138+0000 7f27f2912b80 1 -- start start 2023-07-18T03:13:30.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.138+0000 7f27d5ffb700 1 -- 172.21.15.101:0/1233723008 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f27cc0ce3a0 con 0x7f27d80ffad0 2023-07-18T03:13:30.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.138+0000 7f27dd59c700 1 --2- 172.21.15.101:0/1233723008 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f27c80a1340 0x7f27c80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:30.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.138+0000 7f27f2912b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561ef0ca6350 0x561ef0ca6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:30.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.138+0000 7f27f2912b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561ef0ca6c60 0x561ef0cb1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:30.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.138+0000 7f27f2912b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ef0cb1b50 0x561ef0cb3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:30.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.138+0000 7f27f2912b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ef0bacd30 con 0x561ef0ca6350 2023-07-18T03:13:30.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.138+0000 7f27f2912b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ef0bffac0 con 0x561ef0cb1b50 2023-07-18T03:13:30.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.138+0000 7f27f2912b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ef0bf8f50 con 0x561ef0ca6c60 2023-07-18T03:13:30.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.139+0000 7f27d7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561ef0ca6c60 0x561ef0cb1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:30.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.139+0000 7f27dcd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561ef0ca6350 0x561ef0ca6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:30.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.139+0000 7f27d7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561ef0ca6c60 0x561ef0cb1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:57346/0 (socket says 172.21.15.101:57346) 2023-07-18T03:13:30.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.139+0000 7f27d7fff700 1 -- 172.21.15.101:0/1810805926 learned_addr learned my addr 172.21.15.101:0/1810805926 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:13:30.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.139+0000 7f27dd59c700 1 --2- 172.21.15.101:0/1810805926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ef0cb1b50 0x561ef0cb3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:30.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.139+0000 7f27d7fff700 1 -- 172.21.15.101:0/1810805926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ef0cb1b50 msgr2=0x561ef0cb3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:30.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.139+0000 7f27d7fff700 1 --2- 172.21.15.101:0/1810805926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ef0cb1b50 0x561ef0cb3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.139+0000 7f27d7fff700 1 -- 172.21.15.101:0/1810805926 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561ef0ca6350 msgr2=0x561ef0ca6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:30.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.139+0000 7f27d7fff700 1 --2- 172.21.15.101:0/1810805926 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561ef0ca6350 0x561ef0ca6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.139+0000 7f27d7fff700 1 -- 172.21.15.101:0/1810805926 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ef0a26e60 con 0x561ef0ca6c60 2023-07-18T03:13:30.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.139+0000 7f27dd59c700 1 --2- 172.21.15.101:0/1233723008 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f27c80a1340 0x7f27c80a37f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f27d000f870 tx=0x7f27d000fc30 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:13:30.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.139+0000 7f27d7fff700 1 --2- 172.21.15.101:0/1810805926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561ef0ca6c60 0x561ef0cb1610 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f27c4009d20 tx=0x7f27c4011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=693ca5907445120 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:13:30.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.139+0000 7f27c2ffd700 1 -- 172.21.15.101:0/1810805926 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c4019630 con 0x561ef0ca6c60 2023-07-18T03:13:30.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.140+0000 7f27c2ffd700 1 -- 172.21.15.101:0/1810805926 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27c4019c50 con 0x561ef0ca6c60 2023-07-18T03:13:30.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.140+0000 7f27c2ffd700 1 -- 172.21.15.101:0/1810805926 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c400ebe0 con 0x561ef0ca6c60 2023-07-18T03:13:30.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.140+0000 7f27f2912b80 1 -- 172.21.15.101:0/1810805926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561ef0ca6c60 msgr2=0x561ef0cb1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:30.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.140+0000 7f27f2912b80 1 --2- 172.21.15.101:0/1810805926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561ef0ca6c60 0x561ef0cb1610 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f27c4009d20 tx=0x7f27c4011720 comp rx=0 tx=0).stop 2023-07-18T03:13:30.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.140+0000 7f27f2912b80 1 -- 172.21.15.101:0/1810805926 shutdown_connections 2023-07-18T03:13:30.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.140+0000 7f27f2912b80 1 --2- 172.21.15.101:0/1810805926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ef0cb1b50 0x561ef0cb3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.140+0000 7f27f2912b80 1 --2- 172.21.15.101:0/1810805926 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561ef0ca6350 0x561ef0ca6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.140+0000 7f27f2912b80 1 --2- 172.21.15.101:0/1810805926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561ef0ca6c60 0x561ef0cb1610 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.140+0000 7f27f2912b80 1 -- 172.21.15.101:0/1810805926 >> 172.21.15.101:0/1810805926 conn(0x561ef0c9ce20 msgr2=0x561ef0ca2720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:13:30.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.140+0000 7f27f2912b80 1 -- 172.21.15.101:0/1810805926 shutdown_connections 2023-07-18T03:13:30.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.140+0000 7f27f2912b80 1 -- 172.21.15.101:0/1810805926 wait complete. 2023-07-18T03:13:30.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.141+0000 7f27f2912b80 1 -- start start 2023-07-18T03:13:30.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.141+0000 7f27f2912b80 1 Processor -- start 2023-07-18T03:13:30.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.142+0000 7f27f2912b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561ef0ca6350 0x561ef0cb15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:30.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.142+0000 7f27f2912b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ef0ca6c60 0x561ef0cae7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:30.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.142+0000 7f27f2912b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561ef0cb1b50 0x561ef0caeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:30.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.142+0000 7f27f2912b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ef0b98b40 con 0x561ef0cb1b50 2023-07-18T03:13:30.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.142+0000 7f27f2912b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ef0b98840 con 0x561ef0ca6c60 2023-07-18T03:13:30.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.142+0000 7f27f2912b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ef0a7c9a0 con 0x561ef0ca6350 2023-07-18T03:13:30.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.142+0000 7f27f2912b80 10 client.0 _ll_register_callbacks cb 0x561ef0b967e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:13:30.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.142+0000 7f27dcd9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561ef0ca6350 0x561ef0cb15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:30.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.142+0000 7f27d7fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ef0ca6c60 0x561ef0cae7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:30.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.142+0000 7f27dd59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561ef0cb1b50 0x561ef0caeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:30.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.142+0000 7f27dcd9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561ef0ca6350 0x561ef0cb15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:57358/0 (socket says 172.21.15.101:57358) 2023-07-18T03:13:30.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.142+0000 7f27dcd9b700 1 -- 172.21.15.101:0/4039609009 learned_addr learned my addr 172.21.15.101:0/4039609009 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:13:30.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.142+0000 7f27d7fff700 1 -- 172.21.15.101:0/4039609009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561ef0ca6350 msgr2=0x561ef0cb15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:30.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.142+0000 7f27d7fff700 1 --2- 172.21.15.101:0/4039609009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561ef0ca6350 0x561ef0cb15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.142+0000 7f27d7fff700 1 -- 172.21.15.101:0/4039609009 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561ef0cb1b50 msgr2=0x561ef0caeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:30.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.142+0000 7f27d7fff700 1 --2- 172.21.15.101:0/4039609009 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561ef0cb1b50 0x561ef0caeef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.142+0000 7f27d7fff700 1 -- 172.21.15.101:0/4039609009 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27c4009980 con 0x561ef0ca6c60 2023-07-18T03:13:30.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.143+0000 7f27d7fff700 1 --2- 172.21.15.101:0/4039609009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ef0ca6c60 0x561ef0cae7b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f27c4000c00 tx=0x7f27c400edd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74ffb5a8abe40d0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:13:30.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.143+0000 7f27b7fff700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:13:30.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.143+0000 7f27b7fff700 1 -- 172.21.15.101:0/4039609009 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c4004230 con 0x561ef0ca6c60 2023-07-18T03:13:30.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.143+0000 7f27f2912b80 1 -- 172.21.15.101:0/4039609009 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561ef0b52570 con 0x561ef0ca6c60 2023-07-18T03:13:30.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.143+0000 7f27b7fff700 1 -- 172.21.15.101:0/4039609009 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f27c4005ce0 con 0x561ef0ca6c60 2023-07-18T03:13:30.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.143+0000 7f27b7fff700 1 -- 172.21.15.101:0/4039609009 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27c4004390 con 0x561ef0ca6c60 2023-07-18T03:13:30.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.143+0000 7f27f2912b80 10 client.26801 fetch_fsmap learned FSMap version 26 2023-07-18T03:13:30.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.143+0000 7f27f2912b80 1 -- 172.21.15.101:0/4039609009 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x561ef0a0fce0 con 0x561ef0ca6c60 2023-07-18T03:13:30.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.143+0000 7f27b7fff700 1 -- 172.21.15.101:0/4039609009 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561ef0b52570 con 0x561ef0ca6c60 2023-07-18T03:13:30.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.144+0000 7f27b7fff700 1 -- 172.21.15.101:0/4039609009 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f27c4005270 con 0x561ef0ca6c60 2023-07-18T03:13:30.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.144+0000 7f27f2912b80 10 client.26801 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:13:30.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.144+0000 7f27f2912b80 10 client.26801 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:13:30.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.144+0000 7f27f2912b80 20 client.26801 populate_metadata read hostname 'smithi101' 2023-07-18T03:13:30.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.144+0000 7f27f2912b80 1 --2- 172.21.15.101:0/4039609009 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x561ef0cc1360 0x561ef0cc3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:30.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.144+0000 7f27f2912b80 4 client.26801 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:13:30.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.144+0000 7f27f2912b80 1 -- 172.21.15.101:0/4039609009 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561ef0bf1f30 con 0x561ef0cc1360 2023-07-18T03:13:30.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.144+0000 7f27dcd9b700 1 --2- 172.21.15.101:0/4039609009 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x561ef0cc1360 0x561ef0cc3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:30.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.144+0000 7f27dcd9b700 1 --2- 172.21.15.101:0/4039609009 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x561ef0cc1360 0x561ef0cc3770 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea3dfa1a6ce99d92 server_cookie=9015b3ee415b6dc8 in_seq=0 out_seq=0 2023-07-18T03:13:30.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.144+0000 7f27b7fff700 10 client.26801 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:13:30.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.145+0000 7f27b7fff700 1 -- 172.21.15.101:0/4039609009 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561ef0bf1f30 con 0x561ef0cc1360 2023-07-18T03:13:30.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.145+0000 7f27b7fff700 10 client.26801 handle_command_reply: tid=0 2023-07-18T03:13:30.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.145+0000 7f27f2912b80 1 client.26801 shutdown 2023-07-18T03:13:30.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.145+0000 7f27f2912b80 1 -- 172.21.15.101:0/4039609009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ef0ca6c60 msgr2=0x561ef0cae7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:30.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.145+0000 7f27f2912b80 1 --2- 172.21.15.101:0/4039609009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ef0ca6c60 0x561ef0cae7b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f27c4000c00 tx=0x7f27c400edd0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.145+0000 7f27f2912b80 1 -- 172.21.15.101:0/4039609009 shutdown_connections 2023-07-18T03:13:30.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.145+0000 7f27f2912b80 1 --2- 172.21.15.101:0/4039609009 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x561ef0cc1360 0x561ef0cc3770 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.145+0000 7f27f2912b80 1 --2- 172.21.15.101:0/4039609009 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561ef0cb1b50 0x561ef0caeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.145+0000 7f27f2912b80 1 --2- 172.21.15.101:0/4039609009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561ef0ca6350 0x561ef0cb15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.145+0000 7f27f2912b80 1 --2- 172.21.15.101:0/4039609009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ef0ca6c60 0x561ef0cae7b0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.145+0000 7f27b7fff700 0 client.26801 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:13:30.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.145+0000 7f27f2912b80 1 -- 172.21.15.101:0/4039609009 >> 172.21.15.101:0/4039609009 conn(0x561ef0c9ce20 msgr2=0x561ef0ca22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:13:30.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.145+0000 7f27f2912b80 1 -- 172.21.15.101:0/4039609009 shutdown_connections 2023-07-18T03:13:30.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.145+0000 7f27f2912b80 1 -- 172.21.15.101:0/4039609009 wait complete. 2023-07-18T03:13:30.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.146+0000 7f27f2912b80 20 client.26801 trim_cache size 0 max 16384 2023-07-18T03:13:30.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.165+0000 7f27f2912b80 1 Processor -- start 2023-07-18T03:13:30.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.165+0000 7f27f2912b80 1 -- start start 2023-07-18T03:13:30.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.166+0000 7f27f2912b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561ef0ca6350 0x561ef0cbd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:30.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.166+0000 7f27f2912b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561ef0ca6c60 0x561ef0cb10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:30.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.166+0000 7f27f2912b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ef0cb1b50 0x561ef0caba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:30.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.166+0000 7f27f2912b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ef0ca7030 con 0x561ef0ca6350 2023-07-18T03:13:30.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.166+0000 7f27f2912b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ef0caf4f0 con 0x561ef0cb1b50 2023-07-18T03:13:30.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.166+0000 7f27f2912b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ef0caeb30 con 0x561ef0ca6c60 2023-07-18T03:13:30.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.166+0000 7f27dcd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561ef0ca6350 0x561ef0cbd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:30.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.166+0000 7f27dd59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ef0cb1b50 0x561ef0caba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:30.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.166+0000 7f27dcd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561ef0ca6350 0x561ef0cbd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:41680/0 (socket says 172.21.15.101:41680) 2023-07-18T03:13:30.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.166+0000 7f27dcd9b700 1 -- 172.21.15.101:0/1069804693 learned_addr learned my addr 172.21.15.101:0/1069804693 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:13:30.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.166+0000 7f27d7fff700 1 --2- 172.21.15.101:0/1069804693 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561ef0ca6c60 0x561ef0cb10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:30.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.166+0000 7f27dcd9b700 1 -- 172.21.15.101:0/1069804693 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561ef0ca6c60 msgr2=0x561ef0cb10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:30.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.166+0000 7f27dcd9b700 1 --2- 172.21.15.101:0/1069804693 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561ef0ca6c60 0x561ef0cb10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.166+0000 7f27dcd9b700 1 -- 172.21.15.101:0/1069804693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ef0cb1b50 msgr2=0x561ef0caba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:30.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.166+0000 7f27dcd9b700 1 --2- 172.21.15.101:0/1069804693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ef0cb1b50 0x561ef0caba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.166+0000 7f27dcd9b700 1 -- 172.21.15.101:0/1069804693 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ef0cae320 con 0x561ef0ca6350 2023-07-18T03:13:30.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.166+0000 7f27dd59c700 1 --2- 172.21.15.101:0/1069804693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ef0cb1b50 0x561ef0caba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:13:30.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.167+0000 7f27dcd9b700 1 --2- 172.21.15.101:0/1069804693 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561ef0ca6350 0x561ef0cbd840 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f27cc09b0e0 tx=0x7f27cc022c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9774c2fa0788b12a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:13:30.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.167+0000 7f27c37fe700 1 -- 172.21.15.101:0/1069804693 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27cc09f180 con 0x561ef0ca6350 2023-07-18T03:13:30.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.167+0000 7f27c37fe700 1 -- 172.21.15.101:0/1069804693 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27cc0cbe90 con 0x561ef0ca6350 2023-07-18T03:13:30.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.167+0000 7f27c37fe700 1 -- 172.21.15.101:0/1069804693 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27cc099610 con 0x561ef0ca6350 2023-07-18T03:13:30.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.167+0000 7f27f2912b80 1 -- 172.21.15.101:0/1069804693 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561ef0ca6350 msgr2=0x561ef0cbd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:30.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.167+0000 7f27f2912b80 1 --2- 172.21.15.101:0/1069804693 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561ef0ca6350 0x561ef0cbd840 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f27cc09b0e0 tx=0x7f27cc022c80 comp rx=0 tx=0).stop 2023-07-18T03:13:30.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.168+0000 7f27f2912b80 1 -- 172.21.15.101:0/1069804693 shutdown_connections 2023-07-18T03:13:30.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.168+0000 7f27f2912b80 1 --2- 172.21.15.101:0/1069804693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ef0cb1b50 0x561ef0caba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.168+0000 7f27f2912b80 1 --2- 172.21.15.101:0/1069804693 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561ef0ca6350 0x561ef0cbd840 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.168+0000 7f27f2912b80 1 --2- 172.21.15.101:0/1069804693 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561ef0ca6c60 0x561ef0cb10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.168+0000 7f27f2912b80 1 -- 172.21.15.101:0/1069804693 >> 172.21.15.101:0/1069804693 conn(0x561ef0c9ce20 msgr2=0x561ef0ca0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:13:30.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.168+0000 7f27f2912b80 1 -- 172.21.15.101:0/1069804693 shutdown_connections 2023-07-18T03:13:30.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.168+0000 7f27f2912b80 1 -- 172.21.15.101:0/1069804693 wait complete. 2023-07-18T03:13:30.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.168+0000 7f27f2912b80 1 -- start start 2023-07-18T03:13:30.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.168+0000 7f27f2912b80 1 Processor -- start 2023-07-18T03:13:30.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.169+0000 7f27f2912b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561ef0ca6350 0x561ef0cb6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:30.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.169+0000 7f27f2912b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ef0ca6c60 0x561ef0cb74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:30.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.169+0000 7f27f2912b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561ef0cb1b50 0x561ef0d151f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:30.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.169+0000 7f27f2912b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ef0cb7e00 con 0x561ef0cb1b50 2023-07-18T03:13:30.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.169+0000 7f27f2912b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ef0d15730 con 0x561ef0ca6c60 2023-07-18T03:13:30.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.169+0000 7f27f2912b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ef0d15870 con 0x561ef0ca6350 2023-07-18T03:13:30.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.169+0000 7f27f2912b80 10 client.0 _ll_register_callbacks cb 0x7f27cc0094b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:13:30.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.169+0000 7f27d7fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ef0ca6c60 0x561ef0cb74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:30.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.169+0000 7f27d7fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ef0ca6c60 0x561ef0cb74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:50574/0 (socket says 172.21.15.101:50574) 2023-07-18T03:13:30.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.169+0000 7f27d7fff700 1 -- 172.21.15.101:0/346924049 learned_addr learned my addr 172.21.15.101:0/346924049 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:13:30.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.169+0000 7f27dd59c700 1 --2- 172.21.15.101:0/346924049 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561ef0cb1b50 0x561ef0d151f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:30.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.169+0000 7f27dcd9b700 1 --2- 172.21.15.101:0/346924049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561ef0ca6350 0x561ef0cb6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:30.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.169+0000 7f27d7fff700 1 -- 172.21.15.101:0/346924049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561ef0ca6350 msgr2=0x561ef0cb6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:30.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.169+0000 7f27d7fff700 1 --2- 172.21.15.101:0/346924049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561ef0ca6350 0x561ef0cb6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.169+0000 7f27d7fff700 1 -- 172.21.15.101:0/346924049 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561ef0cb1b50 msgr2=0x561ef0d151f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:30.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.169+0000 7f27d7fff700 1 --2- 172.21.15.101:0/346924049 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561ef0cb1b50 0x561ef0d151f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.169+0000 7f27d7fff700 1 -- 172.21.15.101:0/346924049 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27d000fdb0 con 0x561ef0ca6c60 2023-07-18T03:13:30.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.170+0000 7f27d7fff700 1 --2- 172.21.15.101:0/346924049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ef0ca6c60 0x561ef0cb74f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f27c4029b50 tx=0x7f27c400e630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76d1f3bfc87049d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:13:30.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.170+0000 7f27b7fff700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:13:30.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.170+0000 7f27b7fff700 1 -- 172.21.15.101:0/346924049 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c40047a0 con 0x561ef0ca6c60 2023-07-18T03:13:30.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.170+0000 7f27f2912b80 1 -- 172.21.15.101:0/346924049 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f27c4019c50 con 0x561ef0ca6c60 2023-07-18T03:13:30.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.170+0000 7f27b7fff700 1 -- 172.21.15.101:0/346924049 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f27c4005ce0 con 0x561ef0ca6c60 2023-07-18T03:13:30.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.170+0000 7f27b7fff700 1 -- 172.21.15.101:0/346924049 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27c4004250 con 0x561ef0ca6c60 2023-07-18T03:13:30.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.170+0000 7f27f2912b80 10 client.26807 fetch_fsmap learned FSMap version 26 2023-07-18T03:13:30.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.170+0000 7f27f2912b80 1 -- 172.21.15.101:0/346924049 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f27d81a4c40 con 0x561ef0ca6c60 2023-07-18T03:13:30.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.170+0000 7f27b7fff700 1 -- 172.21.15.101:0/346924049 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27c4019c50 con 0x561ef0ca6c60 2023-07-18T03:13:30.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.171+0000 7f27b7fff700 1 -- 172.21.15.101:0/346924049 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f27c4019430 con 0x561ef0ca6c60 2023-07-18T03:13:30.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.171+0000 7f27f2912b80 10 client.26807 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:13:30.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.171+0000 7f27f2912b80 10 client.26807 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:13:30.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.171+0000 7f27f2912b80 20 client.26807 populate_metadata read hostname 'smithi101' 2023-07-18T03:13:30.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.171+0000 7f27f2912b80 1 --2- 172.21.15.101:0/346924049 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x561ef0d16b00 0x561ef0d18ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:30.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.171+0000 7f27f2912b80 4 client.26807 mds_command: new command op to 24562 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-18T03:13:30.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.171+0000 7f27f2912b80 1 -- 172.21.15.101:0/346924049 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x561ef0bb40d0 con 0x561ef0d16b00 2023-07-18T03:13:30.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.171+0000 7f27dcd9b700 1 --2- 172.21.15.101:0/346924049 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x561ef0d16b00 0x561ef0d18ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:30.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.171+0000 7f27b7fff700 10 client.26807 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:13:30.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.171+0000 7f27dcd9b700 1 --2- 172.21.15.101:0/346924049 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x561ef0d16b00 0x561ef0d18ee0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d4956bcfe972feb server_cookie=c62928aad7b85c89 in_seq=0 out_seq=0 2023-07-18T03:13:30.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.172+0000 7f27b7fff700 1 -- 172.21.15.101:0/346924049 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x561ef0bb40d0 con 0x561ef0d16b00 2023-07-18T03:13:30.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.172+0000 7f27b7fff700 10 client.26807 handle_command_reply: tid=0 2023-07-18T03:13:30.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.172+0000 7f27f2912b80 1 client.26807 shutdown 2023-07-18T03:13:30.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.173+0000 7f27f2912b80 1 -- 172.21.15.101:0/346924049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ef0ca6c60 msgr2=0x561ef0cb74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:30.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.173+0000 7f27f2912b80 1 --2- 172.21.15.101:0/346924049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ef0ca6c60 0x561ef0cb74f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f27c4029b50 tx=0x7f27c400e630 comp rx=0 tx=0).stop 2023-07-18T03:13:30.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.173+0000 7f27f2912b80 1 -- 172.21.15.101:0/346924049 shutdown_connections 2023-07-18T03:13:30.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.173+0000 7f27f2912b80 1 --2- 172.21.15.101:0/346924049 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x561ef0d16b00 0x561ef0d18ee0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.173+0000 7f27f2912b80 1 --2- 172.21.15.101:0/346924049 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561ef0cb1b50 0x561ef0d151f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.173+0000 7f27f2912b80 1 --2- 172.21.15.101:0/346924049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561ef0ca6350 0x561ef0cb6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.173+0000 7f27b7fff700 0 client.26807 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:13:30.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.173+0000 7f27f2912b80 1 --2- 172.21.15.101:0/346924049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ef0ca6c60 0x561ef0cb74f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.173+0000 7f27f2912b80 1 -- 172.21.15.101:0/346924049 >> 172.21.15.101:0/346924049 conn(0x561ef0c9ce20 msgr2=0x561ef0ca9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:13:30.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.173+0000 7f27f2912b80 1 -- 172.21.15.101:0/346924049 shutdown_connections 2023-07-18T03:13:30.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.173+0000 7f27f2912b80 1 -- 172.21.15.101:0/346924049 wait complete. 2023-07-18T03:13:30.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.173+0000 7f27f2912b80 20 client.26807 trim_cache size 0 max 16384 2023-07-18T03:13:30.174 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:13:30.175 INFO:teuthology.orchestra.run.smithi101.stdout: "return_code": 0, 2023-07-18T03:13:30.175 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_tag": "66465329-58c2-4ce5-9f24-8e2dd70a7af3", 2023-07-18T03:13:30.175 INFO:teuthology.orchestra.run.smithi101.stdout: "mode": "asynchronous" 2023-07-18T03:13:30.175 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:13:30.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.174+0000 7f27defff700 1 -- 172.21.15.101:0/1233723008 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f27c80a1340 msgr2=0x7f27c80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:30.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.174+0000 7f27defff700 1 --2- 172.21.15.101:0/1233723008 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f27c80a1340 0x7f27c80a37f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f27d000f870 tx=0x7f27d000fc30 comp rx=0 tx=0).stop 2023-07-18T03:13:30.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.174+0000 7f27defff700 1 -- 172.21.15.101:0/1233723008 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f27d80ffad0 msgr2=0x7f27d8197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:30.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.174+0000 7f27defff700 1 --2- 172.21.15.101:0/1233723008 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f27d80ffad0 0x7f27d8197e10 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f27cc000c00 tx=0x7f27cc005740 comp rx=0 tx=0).stop 2023-07-18T03:13:30.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.174+0000 7f27defff700 1 -- 172.21.15.101:0/1233723008 shutdown_connections 2023-07-18T03:13:30.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.174+0000 7f27defff700 1 --2- 172.21.15.101:0/1233723008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27d8105ae0 0x7f27d819c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.174+0000 7f27defff700 1 --2- 172.21.15.101:0/1233723008 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f27d80ffad0 0x7f27d8197e10 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.174+0000 7f27defff700 1 --2- 172.21.15.101:0/1233723008 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f27c80a1340 0x7f27c80a37f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.174+0000 7f27defff700 1 --2- 172.21.15.101:0/1233723008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27d8100480 0x7f27d8198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.174+0000 7f27defff700 1 -- 172.21.15.101:0/1233723008 >> 172.21.15.101:0/1233723008 conn(0x7f27d80fb5f0 msgr2=0x7f27d810ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:13:30.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.174+0000 7f27defff700 1 -- 172.21.15.101:0/1233723008 shutdown_connections 2023-07-18T03:13:30.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.174+0000 7f27defff700 1 -- 172.21.15.101:0/1233723008 wait complete. 2023-07-18T03:13:30.185 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:13:30.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.405+0000 7f076af9d700 1 -- 172.21.15.101:0/2170092965 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0764105a00 msgr2=0x7f0764105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:30.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.405+0000 7f076af9d700 1 --2- 172.21.15.101:0/2170092965 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0764105a00 0x7f0764105dd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0760009990 tx=0x7f0760009ca0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.405+0000 7f076af9d700 1 -- 172.21.15.101:0/2170092965 shutdown_connections 2023-07-18T03:13:30.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.405+0000 7f076af9d700 1 --2- 172.21.15.101:0/2170092965 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0764105a00 0x7f0764105dd0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.405+0000 7f076af9d700 1 --2- 172.21.15.101:0/2170092965 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f07641003a0 0x7f076410c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.405+0000 7f076af9d700 1 --2- 172.21.15.101:0/2170092965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07640ff9f0 0x7f07640ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.405+0000 7f076af9d700 1 -- 172.21.15.101:0/2170092965 >> 172.21.15.101:0/2170092965 conn(0x7f07640fb510 msgr2=0x7f07640fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:13:30.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.405+0000 7f076af9d700 1 -- 172.21.15.101:0/2170092965 shutdown_connections 2023-07-18T03:13:30.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.405+0000 7f076af9d700 1 -- 172.21.15.101:0/2170092965 wait complete. 2023-07-18T03:13:30.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.406+0000 7f076af9d700 1 Processor -- start 2023-07-18T03:13:30.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.406+0000 7f076af9d700 1 -- start start 2023-07-18T03:13:30.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.407+0000 7f076af9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07640ff9f0 0x7f076419d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:30.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.407+0000 7f076af9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07641003a0 0x7f076419de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:30.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.407+0000 7f076af9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0764105a00 0x7f0764197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:30.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.407+0000 7f076af9d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0764198000 con 0x7f0764105a00 2023-07-18T03:13:30.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.407+0000 7f076af9d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0764198170 con 0x7f07641003a0 2023-07-18T03:13:30.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.407+0000 7f076af9d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0764198330 con 0x7f07640ff9f0 2023-07-18T03:13:30.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.407+0000 7f076979a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07641003a0 0x7f076419de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:30.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.407+0000 7f0769f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07640ff9f0 0x7f076419d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:30.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.407+0000 7f076979a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07641003a0 0x7f076419de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:50590/0 (socket says 172.21.15.101:50590) 2023-07-18T03:13:30.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.407+0000 7f0769f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07640ff9f0 0x7f076419d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:57384/0 (socket says 172.21.15.101:57384) 2023-07-18T03:13:30.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.407+0000 7f076979a700 1 -- 172.21.15.101:0/1075436882 learned_addr learned my addr 172.21.15.101:0/1075436882 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:13:30.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.407+0000 7f076a79c700 1 --2- 172.21.15.101:0/1075436882 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0764105a00 0x7f0764197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:30.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.407+0000 7f076979a700 1 -- 172.21.15.101:0/1075436882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07640ff9f0 msgr2=0x7f076419d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:30.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.407+0000 7f076979a700 1 --2- 172.21.15.101:0/1075436882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07640ff9f0 0x7f076419d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.407+0000 7f076979a700 1 -- 172.21.15.101:0/1075436882 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0764105a00 msgr2=0x7f0764197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:30.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.407+0000 7f076979a700 1 --2- 172.21.15.101:0/1075436882 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0764105a00 0x7f0764197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.407+0000 7f076979a700 1 -- 172.21.15.101:0/1075436882 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07600095f0 con 0x7f07641003a0 2023-07-18T03:13:30.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.407+0000 7f076a79c700 1 --2- 172.21.15.101:0/1075436882 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0764105a00 0x7f0764197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:13:30.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.408+0000 7f076979a700 1 --2- 172.21.15.101:0/1075436882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07641003a0 0x7f076419de60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f075400e9a0 tx=0x7f075400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:13:30.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.408+0000 7f075affd700 1 -- 172.21.15.101:0/1075436882 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0754009790 con 0x7f07641003a0 2023-07-18T03:13:30.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.408+0000 7f075affd700 1 -- 172.21.15.101:0/1075436882 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f075400cbd0 con 0x7f07641003a0 2023-07-18T03:13:30.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.408+0000 7f076af9d700 1 -- 172.21.15.101:0/1075436882 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0764198610 con 0x7f07641003a0 2023-07-18T03:13:30.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.408+0000 7f075affd700 1 -- 172.21.15.101:0/1075436882 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0754018680 con 0x7f07641003a0 2023-07-18T03:13:30.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.408+0000 7f076af9d700 1 -- 172.21.15.101:0/1075436882 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07641a4910 con 0x7f07641003a0 2023-07-18T03:13:30.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.411+0000 7f075affd700 1 -- 172.21.15.101:0/1075436882 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f075400cd40 con 0x7f07641003a0 2023-07-18T03:13:30.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.412+0000 7f077faebb80 1 Processor -- start 2023-07-18T03:13:30.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.413+0000 7f077faebb80 1 -- start start 2023-07-18T03:13:30.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.413+0000 7f075affd700 1 --2- 172.21.15.101:0/1075436882 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f07500a13e0 0x7f07500a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:30.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.413+0000 7f077faebb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56389cf794f0 0x56389cf798c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:30.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.413+0000 7f077faebb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56389cf79e00 0x56389cf837f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:30.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.413+0000 7f077faebb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56389cf83d30 0x56389cf86110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:30.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.413+0000 7f077faebb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56389ce7ecb0 con 0x56389cf83d30 2023-07-18T03:13:30.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.413+0000 7f077faebb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56389ced1a40 con 0x56389cf79e00 2023-07-18T03:13:30.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.413+0000 7f077faebb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56389cecaed0 con 0x56389cf794f0 2023-07-18T03:13:30.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.413+0000 7f075affd700 1 -- 172.21.15.101:0/1075436882 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f0754014070 con 0x7f07641003a0 2023-07-18T03:13:30.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.413+0000 7f0769f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56389cf794f0 0x56389cf798c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:30.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.413+0000 7f0769f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56389cf794f0 0x56389cf798c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:57390/0 (socket says 172.21.15.101:57390) 2023-07-18T03:13:30.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.413+0000 7f0769f9b700 1 -- 172.21.15.101:0/3550020486 learned_addr learned my addr 172.21.15.101:0/3550020486 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:13:30.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.413+0000 7f076979a700 1 --2- 172.21.15.101:0/3550020486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56389cf79e00 0x56389cf837f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:30.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.413+0000 7f076a79c700 1 --2- 172.21.15.101:0/3550020486 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56389cf83d30 0x56389cf86110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:30.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.414+0000 7f076a79c700 1 --2- 172.21.15.101:0/1075436882 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f07500a13e0 0x7f07500a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:30.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.414+0000 7f0769f9b700 1 -- 172.21.15.101:0/3550020486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56389cf79e00 msgr2=0x56389cf837f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:30.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.414+0000 7f0769f9b700 1 --2- 172.21.15.101:0/3550020486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56389cf79e00 0x56389cf837f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.414+0000 7f0769f9b700 1 -- 172.21.15.101:0/3550020486 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56389cf83d30 msgr2=0x56389cf86110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:30.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.414+0000 7f0769f9b700 1 --2- 172.21.15.101:0/3550020486 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56389cf83d30 0x56389cf86110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.414+0000 7f0769f9b700 1 -- 172.21.15.101:0/3550020486 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56389ccf8de0 con 0x56389cf794f0 2023-07-18T03:13:30.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.414+0000 7f0769f9b700 1 --2- 172.21.15.101:0/3550020486 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56389cf794f0 0x56389cf798c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0760015000 tx=0x7f076000b610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db179d0bea33fab1 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:13:30.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.414+0000 7f076a79c700 1 --2- 172.21.15.101:0/1075436882 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f07500a13e0 0x7f07500a3890 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f075c009fd0 tx=0x7f075c00b040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:13:30.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.414+0000 7f074ffff700 1 -- 172.21.15.101:0/3550020486 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076001d070 con 0x56389cf794f0 2023-07-18T03:13:30.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.414+0000 7f074ffff700 1 -- 172.21.15.101:0/3550020486 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f076001e040 con 0x56389cf794f0 2023-07-18T03:13:30.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.414+0000 7f074ffff700 1 -- 172.21.15.101:0/3550020486 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07600048f0 con 0x56389cf794f0 2023-07-18T03:13:30.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.415+0000 7f077faebb80 1 -- 172.21.15.101:0/3550020486 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56389cf794f0 msgr2=0x56389cf798c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:30.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.415+0000 7f077faebb80 1 --2- 172.21.15.101:0/3550020486 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56389cf794f0 0x56389cf798c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0760015000 tx=0x7f076000b610 comp rx=0 tx=0).stop 2023-07-18T03:13:30.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.415+0000 7f077faebb80 1 -- 172.21.15.101:0/3550020486 shutdown_connections 2023-07-18T03:13:30.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.415+0000 7f077faebb80 1 --2- 172.21.15.101:0/3550020486 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56389cf83d30 0x56389cf86110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.415+0000 7f077faebb80 1 --2- 172.21.15.101:0/3550020486 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56389cf794f0 0x56389cf798c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.415+0000 7f077faebb80 1 --2- 172.21.15.101:0/3550020486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56389cf79e00 0x56389cf837f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.415+0000 7f077faebb80 1 -- 172.21.15.101:0/3550020486 >> 172.21.15.101:0/3550020486 conn(0x56389cf6eff0 msgr2=0x56389cf74930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:13:30.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.415+0000 7f077faebb80 1 -- 172.21.15.101:0/3550020486 shutdown_connections 2023-07-18T03:13:30.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.415+0000 7f077faebb80 1 -- 172.21.15.101:0/3550020486 wait complete. 2023-07-18T03:13:30.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.416+0000 7f077faebb80 1 -- start start 2023-07-18T03:13:30.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.416+0000 7f077faebb80 1 Processor -- start 2023-07-18T03:13:30.414 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.417+0000 7f077faebb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56389cf794f0 0x56389cf836f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:30.414 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.417+0000 7f077faebb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56389cf79e00 0x56389cf80970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:30.414 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.417+0000 7f077faebb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56389cf83d30 0x56389cf80f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:30.414 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.417+0000 7f077faebb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56389ce6aac0 con 0x56389cf794f0 2023-07-18T03:13:30.414 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.417+0000 7f077faebb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56389ce6a7c0 con 0x56389cf79e00 2023-07-18T03:13:30.414 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.417+0000 7f077faebb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56389cd4e920 con 0x56389cf83d30 2023-07-18T03:13:30.414 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.417+0000 7f077faebb80 10 client.0 _ll_register_callbacks cb 0x56389ce68760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:13:30.414 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.417+0000 7f076a79c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56389cf83d30 0x56389cf80f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:30.415 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.417+0000 7f076979a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56389cf79e00 0x56389cf80970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:30.415 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.417+0000 7f076a79c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56389cf83d30 0x56389cf80f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:57400/0 (socket says 172.21.15.101:57400) 2023-07-18T03:13:30.415 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.417+0000 7f076a79c700 1 -- 172.21.15.101:0/1480946635 learned_addr learned my addr 172.21.15.101:0/1480946635 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:13:30.415 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.417+0000 7f0769f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56389cf794f0 0x56389cf836f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:30.415 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.417+0000 7f076979a700 1 -- 172.21.15.101:0/1480946635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56389cf83d30 msgr2=0x56389cf80f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:30.415 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.417+0000 7f076979a700 1 --2- 172.21.15.101:0/1480946635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56389cf83d30 0x56389cf80f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.415 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.417+0000 7f076979a700 1 -- 172.21.15.101:0/1480946635 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56389cf794f0 msgr2=0x56389cf836f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:30.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.417+0000 7f076979a700 1 --2- 172.21.15.101:0/1480946635 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56389cf794f0 0x56389cf836f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.417+0000 7f076979a700 1 -- 172.21.15.101:0/1480946635 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f076410d130 con 0x56389cf79e00 2023-07-18T03:13:30.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.417+0000 7f076a79c700 1 --2- 172.21.15.101:0/1480946635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56389cf83d30 0x56389cf80f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:13:30.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.418+0000 7f076979a700 1 --2- 172.21.15.101:0/1480946635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56389cf79e00 0x56389cf80970 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f07540ca640 tx=0x7f0754055300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fbd5a5ad1fbc2e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:13:30.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.418+0000 7f074d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:13:30.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.418+0000 7f074d7fa700 1 -- 172.21.15.101:0/1480946635 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07540a2410 con 0x56389cf79e00 2023-07-18T03:13:30.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.418+0000 7f077faebb80 1 -- 172.21.15.101:0/1480946635 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56389ce244f0 con 0x56389cf79e00 2023-07-18T03:13:30.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.418+0000 7f074d7fa700 1 -- 172.21.15.101:0/1480946635 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f07540994b0 con 0x56389cf79e00 2023-07-18T03:13:30.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.418+0000 7f074d7fa700 1 -- 172.21.15.101:0/1480946635 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0754099670 con 0x56389cf79e00 2023-07-18T03:13:30.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.418+0000 7f077faebb80 10 client.26819 fetch_fsmap learned FSMap version 26 2023-07-18T03:13:30.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.418+0000 7f077faebb80 1 -- 172.21.15.101:0/1480946635 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56389cce1c60 con 0x56389cf79e00 2023-07-18T03:13:30.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.418+0000 7f074d7fa700 1 -- 172.21.15.101:0/1480946635 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56389ce244f0 con 0x56389cf79e00 2023-07-18T03:13:30.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.418+0000 7f074d7fa700 1 -- 172.21.15.101:0/1480946635 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0754098f20 con 0x56389cf79e00 2023-07-18T03:13:30.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.419+0000 7f077faebb80 10 client.26819 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:13:30.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.419+0000 7f077faebb80 10 client.26819 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:13:30.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.419+0000 7f077faebb80 20 client.26819 populate_metadata read hostname 'smithi101' 2023-07-18T03:13:30.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.419+0000 7f077faebb80 1 --2- 172.21.15.101:0/1480946635 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56389cf93630 0x56389cf95a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:30.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.419+0000 7f077faebb80 4 client.26819 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:13:30.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.419+0000 7f077faebb80 1 -- 172.21.15.101:0/1480946635 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56389cec3eb0 con 0x56389cf93630 2023-07-18T03:13:30.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.419+0000 7f0769f9b700 1 --2- 172.21.15.101:0/1480946635 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56389cf93630 0x56389cf95a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:30.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.419+0000 7f0769f9b700 1 --2- 172.21.15.101:0/1480946635 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56389cf93630 0x56389cf95a40 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0921533eb16d167 server_cookie=5e267313e401e6f4 in_seq=0 out_seq=0 2023-07-18T03:13:30.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.419+0000 7f074d7fa700 10 client.26819 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:13:30.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.420+0000 7f074d7fa700 1 -- 172.21.15.101:0/1480946635 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56389cec3eb0 con 0x56389cf93630 2023-07-18T03:13:30.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.420+0000 7f074d7fa700 10 client.26819 handle_command_reply: tid=0 2023-07-18T03:13:30.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.420+0000 7f077faebb80 1 client.26819 shutdown 2023-07-18T03:13:30.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.421+0000 7f077faebb80 1 -- 172.21.15.101:0/1480946635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56389cf79e00 msgr2=0x56389cf80970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:30.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.421+0000 7f077faebb80 1 --2- 172.21.15.101:0/1480946635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56389cf79e00 0x56389cf80970 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f07540ca640 tx=0x7f0754055300 comp rx=0 tx=0).stop 2023-07-18T03:13:30.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.421+0000 7f077faebb80 1 -- 172.21.15.101:0/1480946635 shutdown_connections 2023-07-18T03:13:30.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.421+0000 7f077faebb80 1 --2- 172.21.15.101:0/1480946635 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56389cf93630 0x56389cf95a40 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.421+0000 7f077faebb80 1 --2- 172.21.15.101:0/1480946635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56389cf83d30 0x56389cf80f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.421+0000 7f077faebb80 1 --2- 172.21.15.101:0/1480946635 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56389cf794f0 0x56389cf836f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.421+0000 7f077faebb80 1 --2- 172.21.15.101:0/1480946635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56389cf79e00 0x56389cf80970 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.421+0000 7f074d7fa700 0 client.26819 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:13:30.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.421+0000 7f077faebb80 1 -- 172.21.15.101:0/1480946635 >> 172.21.15.101:0/1480946635 conn(0x56389cf6eff0 msgr2=0x56389cf74520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:13:30.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.421+0000 7f077faebb80 1 -- 172.21.15.101:0/1480946635 shutdown_connections 2023-07-18T03:13:30.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.421+0000 7f077faebb80 1 -- 172.21.15.101:0/1480946635 wait complete. 2023-07-18T03:13:30.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.421+0000 7f077faebb80 20 client.26819 trim_cache size 0 max 16384 2023-07-18T03:13:30.432 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.439+0000 7f077faebb80 1 Processor -- start 2023-07-18T03:13:30.432 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.439+0000 7f077faebb80 1 -- start start 2023-07-18T03:13:30.432 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.439+0000 7f077faebb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56389cf794f0 0x56389cf8ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:30.433 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.439+0000 7f077faebb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56389cf79e00 0x56389cf83220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:30.433 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.439+0000 7f077faebb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56389cf83d30 0x56389cf80930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:30.433 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.439+0000 7f077faebb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56389cf90d80 con 0x56389cf794f0 2023-07-18T03:13:30.433 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.439+0000 7f077faebb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56389cf6ee30 con 0x56389cf79e00 2023-07-18T03:13:30.433 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.439+0000 7f077faebb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56389cf93410 con 0x56389cf83d30 2023-07-18T03:13:30.433 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.439+0000 7f0769f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56389cf794f0 0x56389cf8ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:30.433 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.439+0000 7f076a79c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56389cf83d30 0x56389cf80930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:30.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.439+0000 7f076a79c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56389cf83d30 0x56389cf80930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:57404/0 (socket says 172.21.15.101:57404) 2023-07-18T03:13:30.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.439+0000 7f076979a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56389cf79e00 0x56389cf83220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:30.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.439+0000 7f076a79c700 1 -- 172.21.15.101:0/2058890165 learned_addr learned my addr 172.21.15.101:0/2058890165 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:13:30.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.440+0000 7f076a79c700 1 -- 172.21.15.101:0/2058890165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56389cf79e00 msgr2=0x56389cf83220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:30.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.440+0000 7f076a79c700 1 --2- 172.21.15.101:0/2058890165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56389cf79e00 0x56389cf83220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.440+0000 7f076a79c700 1 -- 172.21.15.101:0/2058890165 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56389cf794f0 msgr2=0x56389cf8ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:30.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.440+0000 7f076a79c700 1 --2- 172.21.15.101:0/2058890165 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56389cf794f0 0x56389cf8ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.440+0000 7f076a79c700 1 -- 172.21.15.101:0/2058890165 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56389cd1f250 con 0x56389cf83d30 2023-07-18T03:13:30.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.440+0000 7f0769f9b700 1 --2- 172.21.15.101:0/2058890165 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56389cf794f0 0x56389cf8ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:13:30.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.440+0000 7f076a79c700 1 --2- 172.21.15.101:0/2058890165 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56389cf83d30 0x56389cf80930 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f075c009790 tx=0x7f075c004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23dfac3353c687d6 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:13:30.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.440+0000 7f0758ff9700 1 -- 172.21.15.101:0/2058890165 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075c00e040 con 0x56389cf83d30 2023-07-18T03:13:30.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.440+0000 7f0758ff9700 1 -- 172.21.15.101:0/2058890165 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f075c0243f0 con 0x56389cf83d30 2023-07-18T03:13:30.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.440+0000 7f0758ff9700 1 -- 172.21.15.101:0/2058890165 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075c023350 con 0x56389cf83d30 2023-07-18T03:13:30.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.441+0000 7f077faebb80 1 -- 172.21.15.101:0/2058890165 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56389cf83d30 msgr2=0x56389cf80930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:30.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.441+0000 7f077faebb80 1 --2- 172.21.15.101:0/2058890165 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56389cf83d30 0x56389cf80930 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f075c009790 tx=0x7f075c004ad0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.441+0000 7f077faebb80 1 -- 172.21.15.101:0/2058890165 shutdown_connections 2023-07-18T03:13:30.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.441+0000 7f077faebb80 1 --2- 172.21.15.101:0/2058890165 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56389cf83d30 0x56389cf80930 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.441+0000 7f077faebb80 1 --2- 172.21.15.101:0/2058890165 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56389cf794f0 0x56389cf8ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.441+0000 7f077faebb80 1 --2- 172.21.15.101:0/2058890165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56389cf79e00 0x56389cf83220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.441+0000 7f077faebb80 1 -- 172.21.15.101:0/2058890165 >> 172.21.15.101:0/2058890165 conn(0x56389cf6eff0 msgr2=0x56389cf7bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:13:30.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.441+0000 7f077faebb80 1 -- 172.21.15.101:0/2058890165 shutdown_connections 2023-07-18T03:13:30.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.441+0000 7f077faebb80 1 -- 172.21.15.101:0/2058890165 wait complete. 2023-07-18T03:13:30.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.441+0000 7f077faebb80 1 -- start start 2023-07-18T03:13:30.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.442+0000 7f077faebb80 1 Processor -- start 2023-07-18T03:13:30.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.442+0000 7f077faebb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56389cf794f0 0x56389cf8a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:30.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.442+0000 7f077faebb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56389cf79e00 0x56389cf88dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:30.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.442+0000 7f077faebb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56389cf83d30 0x56389cf89350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:30.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.442+0000 7f077faebb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56389cf89a60 con 0x56389cf794f0 2023-07-18T03:13:30.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.442+0000 7f077faebb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56389cf89bd0 con 0x56389cf83d30 2023-07-18T03:13:30.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.442+0000 7f077faebb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56389cfe7500 con 0x56389cf79e00 2023-07-18T03:13:30.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.442+0000 7f077faebb80 10 client.0 _ll_register_callbacks cb 0x7f0754099e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:13:30.440 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.442+0000 7f0769f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56389cf794f0 0x56389cf8a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:30.440 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.442+0000 7f0769f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56389cf794f0 0x56389cf8a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:41738/0 (socket says 172.21.15.101:41738) 2023-07-18T03:13:30.440 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.442+0000 7f0769f9b700 1 -- 172.21.15.101:0/1955360348 learned_addr learned my addr 172.21.15.101:0/1955360348 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:13:30.440 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.442+0000 7f076979a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56389cf79e00 0x56389cf88dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:30.440 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.442+0000 7f076a79c700 1 --2- 172.21.15.101:0/1955360348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56389cf83d30 0x56389cf89350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:30.440 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.443+0000 7f076979a700 1 -- 172.21.15.101:0/1955360348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56389cf83d30 msgr2=0x56389cf89350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:30.440 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.443+0000 7f076979a700 1 --2- 172.21.15.101:0/1955360348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56389cf83d30 0x56389cf89350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.443+0000 7f076979a700 1 -- 172.21.15.101:0/1955360348 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56389cf794f0 msgr2=0x56389cf8a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:13:30.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.443+0000 7f076979a700 1 --2- 172.21.15.101:0/1955360348 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56389cf794f0 0x56389cf8a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.443+0000 7f076979a700 1 -- 172.21.15.101:0/1955360348 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56389ccf8de0 con 0x56389cf79e00 2023-07-18T03:13:30.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.443+0000 7f076a79c700 1 --2- 172.21.15.101:0/1955360348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56389cf83d30 0x56389cf89350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:13:30.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.443+0000 7f0769f9b700 1 --2- 172.21.15.101:0/1955360348 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56389cf794f0 0x56389cf8a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:13:30.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.443+0000 7f076979a700 1 --2- 172.21.15.101:0/1955360348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56389cf79e00 0x56389cf88dd0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f07540964c0 tx=0x7f075400c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e920c300a74560e9 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:13:30.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.443+0000 7f074d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:13:30.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.443+0000 7f074d7fa700 1 -- 172.21.15.101:0/1955360348 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07540a2410 con 0x56389cf79e00 2023-07-18T03:13:30.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.443+0000 7f074d7fa700 1 -- 172.21.15.101:0/1955360348 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f0754094bd0 con 0x56389cf79e00 2023-07-18T03:13:30.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.443+0000 7f077faebb80 1 -- 172.21.15.101:0/1955360348 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f076001e040 con 0x56389cf79e00 2023-07-18T03:13:30.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.443+0000 7f074d7fa700 1 -- 172.21.15.101:0/1955360348 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0754094d90 con 0x56389cf79e00 2023-07-18T03:13:30.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.443+0000 7f077faebb80 10 client.26857 fetch_fsmap learned FSMap version 26 2023-07-18T03:13:30.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.443+0000 7f077faebb80 1 -- 172.21.15.101:0/1955360348 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f075c018940 con 0x56389cf79e00 2023-07-18T03:13:30.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.444+0000 7f074d7fa700 1 -- 172.21.15.101:0/1955360348 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f076001e040 con 0x56389cf79e00 2023-07-18T03:13:30.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.444+0000 7f074d7fa700 1 -- 172.21.15.101:0/1955360348 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 26) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f075409b000 con 0x56389cf79e00 2023-07-18T03:13:30.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.444+0000 7f077faebb80 10 client.26857 fetch_fsmap finished waiting for FSMap version 26 2023-07-18T03:13:30.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.444+0000 7f077faebb80 10 client.26857 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:13:30.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.444+0000 7f077faebb80 20 client.26857 populate_metadata read hostname 'smithi101' 2023-07-18T03:13:30.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.444+0000 7f077faebb80 1 --2- 172.21.15.101:0/1955360348 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56389cfe87b0 0x56389cfeabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:13:30.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.444+0000 7f077faebb80 4 client.26857 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:13:30.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.444+0000 7f077faebb80 1 -- 172.21.15.101:0/1955360348 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56389ce86050 con 0x56389cfe87b0 2023-07-18T03:13:30.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.444+0000 7f0769f9b700 1 --2- 172.21.15.101:0/1955360348 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56389cfe87b0 0x56389cfeabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:13:30.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.445+0000 7f0769f9b700 1 --2- 172.21.15.101:0/1955360348 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56389cfe87b0 0x56389cfeabc0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da926ff41da1eade server_cookie=1d2a2a0201af8a25 in_seq=0 out_seq=0 2023-07-18T03:13:30.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.445+0000 7f074d7fa700 10 client.26857 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:13:30.446 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.445+0000 7f074d7fa700 1 -- 172.21.15.101:0/1955360348 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56389ce86050 con 0x56389cfe87b0 2023-07-18T03:13:30.446 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.445+0000 7f074d7fa700 10 client.26857 handle_command_reply: tid=0 2023-07-18T03:13:30.446 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.445+0000 7f077faebb80 1 client.26857 shutdown 2023-07-18T03:13:30.446 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.445+0000 7f077faebb80 1 -- 172.21.15.101:0/1955360348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56389cf79e00 msgr2=0x56389cf88dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:30.446 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.445+0000 7f077faebb80 1 --2- 172.21.15.101:0/1955360348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56389cf79e00 0x56389cf88dd0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f07540964c0 tx=0x7f075400c6e0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.446 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.445+0000 7f077faebb80 1 -- 172.21.15.101:0/1955360348 shutdown_connections 2023-07-18T03:13:30.446 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.445+0000 7f077faebb80 1 --2- 172.21.15.101:0/1955360348 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56389cfe87b0 0x56389cfeabc0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.446 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.445+0000 7f077faebb80 1 --2- 172.21.15.101:0/1955360348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56389cf83d30 0x56389cf89350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.445+0000 7f077faebb80 1 --2- 172.21.15.101:0/1955360348 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56389cf794f0 0x56389cf8a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.445+0000 7f074d7fa700 0 client.26857 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:13:30.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.445+0000 7f077faebb80 1 --2- 172.21.15.101:0/1955360348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56389cf79e00 0x56389cf88dd0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.445+0000 7f077faebb80 1 -- 172.21.15.101:0/1955360348 >> 172.21.15.101:0/1955360348 conn(0x56389cf6eff0 msgr2=0x56389cf8ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:13:30.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.445+0000 7f077faebb80 1 -- 172.21.15.101:0/1955360348 shutdown_connections 2023-07-18T03:13:30.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.445+0000 7f077faebb80 1 -- 172.21.15.101:0/1955360348 wait complete. 2023-07-18T03:13:30.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.445+0000 7f077faebb80 20 client.26857 trim_cache size 0 max 16384 2023-07-18T03:13:30.448 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:13:30.448 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:13:30.448 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:13:30.448 INFO:teuthology.orchestra.run.smithi101.stdout: "66465329-58c2-4ce5-9f24-8e2dd70a7af3": { 2023-07-18T03:13:30.448 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:13:30.449 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "66465329-58c2-4ce5-9f24-8e2dd70a7af3", 2023-07-18T03:13:30.449 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:13:30.449 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:13:30.449 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:13:30.449 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:13:30.450 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.446+0000 7f076af9d700 1 -- 172.21.15.101:0/1075436882 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f07500a13e0 msgr2=0x7f07500a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:30.450 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.446+0000 7f076af9d700 1 --2- 172.21.15.101:0/1075436882 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f07500a13e0 0x7f07500a3890 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f075c009fd0 tx=0x7f075c00b040 comp rx=0 tx=0).stop 2023-07-18T03:13:30.450 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.446+0000 7f076af9d700 1 -- 172.21.15.101:0/1075436882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07641003a0 msgr2=0x7f076419de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:13:30.450 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.446+0000 7f076af9d700 1 --2- 172.21.15.101:0/1075436882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07641003a0 0x7f076419de60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f075400e9a0 tx=0x7f075400ecb0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.450 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.446+0000 7f076af9d700 1 -- 172.21.15.101:0/1075436882 shutdown_connections 2023-07-18T03:13:30.450 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.446+0000 7f076af9d700 1 --2- 172.21.15.101:0/1075436882 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0764105a00 0x7f0764197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.446+0000 7f076af9d700 1 --2- 172.21.15.101:0/1075436882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07640ff9f0 0x7f076419d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.446+0000 7f076af9d700 1 --2- 172.21.15.101:0/1075436882 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f07500a13e0 0x7f07500a3890 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.446+0000 7f076af9d700 1 --2- 172.21.15.101:0/1075436882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07641003a0 0x7f076419de60 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:13:30.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.446+0000 7f076af9d700 1 -- 172.21.15.101:0/1075436882 >> 172.21.15.101:0/1075436882 conn(0x7f07640fb510 msgr2=0x7f0764072740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:13:30.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.446+0000 7f076af9d700 1 -- 172.21.15.101:0/1075436882 shutdown_connections 2023-07-18T03:13:30.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:13:30.446+0000 7f076af9d700 1 -- 172.21.15.101:0/1075436882 wait complete. 2023-07-18T03:13:30.457 INFO:tasks.cephfs.filesystem:scrub status for tag:66465329-58c2-4ce5-9f24-8e2dd70a7af3 - {'path': '/', 'tag': '66465329-58c2-4ce5-9f24-8e2dd70a7af3', 'options': 'recursive,force'} 2023-07-18T03:13:30.583 INFO:tasks.workunit.client.0.smithi101.stdout: 1 458741 8.61 MB/sec execute 175 sec latency 77.553 ms 2023-07-18T03:13:31.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:31 smithi155 ceph-mon[128885]: pgmap v181: 129 pgs: 129 active+clean; 295 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 283 op/s 2023-07-18T03:13:31.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:31 smithi155 ceph-mon[128885]: scrub queued for path: / 2023-07-18T03:13:31.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:31 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:13:31.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:31 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:13:31.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:31 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:13:31.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:31 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:13:31.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:31 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:13:31.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:31 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:13:31.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:31 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:13:31.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:31 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:13:31.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:31 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:13:31.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:31 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:13:31.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:31 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:13:31.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:31 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:13:31.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:31 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:13:31.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:31 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:13:31.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:31 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:13:31.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:31 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:13:31.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:31 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:13:31.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:31 smithi179 ceph-mon[129229]: pgmap v181: 129 pgs: 129 active+clean; 295 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 283 op/s 2023-07-18T03:13:31.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:31 smithi179 ceph-mon[129229]: scrub queued for path: / 2023-07-18T03:13:31.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:31 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:13:31.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:31 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:13:31.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:31 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:13:31.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:31 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:13:31.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:31 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:13:31.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:31 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:13:31.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:31 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:13:31.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:31 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:13:31.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:31 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:13:31.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:31 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:13:31.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:31 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:13:31.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:31 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:13:31.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:31 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:13:31.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:31 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:13:31.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:31 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:13:31.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:31 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:13:31.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:31 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:13:31.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:31 smithi101 ceph-mon[126379]: pgmap v181: 129 pgs: 129 active+clean; 295 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 283 op/s 2023-07-18T03:13:31.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:31 smithi101 ceph-mon[126379]: scrub queued for path: / 2023-07-18T03:13:31.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:31 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:13:31.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:31 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:13:31.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:31 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:13:31.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:31 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:13:31.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:31 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:13:31.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:31 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:13:31.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:31 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:13:31.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:31 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:13:31.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:31 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:13:31.492 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:31 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:13:31.492 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:31 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:13:31.492 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:31 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:13:31.492 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:31 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:13:31.492 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:31 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:13:31.492 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:31 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:13:31.492 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:31 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:13:31.493 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:31 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:13:31.583 INFO:tasks.workunit.client.0.smithi101.stdout: 1 460044 8.63 MB/sec execute 176 sec latency 51.035 ms 2023-07-18T03:13:32.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:32 smithi155 ceph-mon[128885]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:13:32.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:32 smithi179 ceph-mon[129229]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:13:32.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:32 smithi101 ceph-mon[126379]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:13:32.583 INFO:tasks.workunit.client.0.smithi101.stdout: 1 461696 8.64 MB/sec execute 177 sec latency 236.765 ms 2023-07-18T03:13:33.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:33 smithi155 ceph-mon[128885]: pgmap v182: 129 pgs: 129 active+clean; 295 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 183 op/s 2023-07-18T03:13:33.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:33 smithi179 ceph-mon[129229]: pgmap v182: 129 pgs: 129 active+clean; 295 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 183 op/s 2023-07-18T03:13:33.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:33 smithi101 ceph-mon[126379]: pgmap v182: 129 pgs: 129 active+clean; 295 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 183 op/s 2023-07-18T03:13:33.584 INFO:tasks.workunit.client.0.smithi101.stdout: 1 461696 8.59 MB/sec execute 178 sec latency 1236.911 ms 2023-07-18T03:13:34.584 INFO:tasks.workunit.client.0.smithi101.stdout: 1 461696 8.54 MB/sec execute 179 sec latency 2237.046 ms 2023-07-18T03:13:35.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:35 smithi155 ceph-mon[128885]: pgmap v183: 129 pgs: 129 active+clean; 295 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 183 op/s 2023-07-18T03:13:35.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:35 smithi179 ceph-mon[129229]: pgmap v183: 129 pgs: 129 active+clean; 295 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 183 op/s 2023-07-18T03:13:35.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:35 smithi101 ceph-mon[126379]: pgmap v183: 129 pgs: 129 active+clean; 295 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 183 op/s 2023-07-18T03:13:35.584 INFO:tasks.workunit.client.0.smithi101.stdout: 1 461722 8.49 MB/sec execute 180 sec latency 2723.060 ms 2023-07-18T03:13:36.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:36 smithi179 ceph-mon[129229]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:13:36.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:36 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:13:36.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:36 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:13:36.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:36 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:13:36.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:36 smithi101 ceph-mon[126379]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:13:36.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:36 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:13:36.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:36 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:13:36.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:36 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:13:36.544 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:36 smithi155 ceph-mon[128885]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:13:36.545 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:36 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:13:36.545 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:36 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:13:36.545 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:36 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:13:36.584 INFO:tasks.workunit.client.0.smithi101.stdout: 1 462145 8.47 MB/sec execute 181 sec latency 549.549 ms 2023-07-18T03:13:37.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:37 smithi179 ceph-mon[129229]: pgmap v184: 129 pgs: 129 active+clean; 298 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 242 op/s 2023-07-18T03:13:37.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:37 smithi101 ceph-mon[126379]: pgmap v184: 129 pgs: 129 active+clean; 298 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 242 op/s 2023-07-18T03:13:37.544 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:37 smithi155 ceph-mon[128885]: pgmap v184: 129 pgs: 129 active+clean; 298 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 242 op/s 2023-07-18T03:13:37.584 INFO:tasks.workunit.client.0.smithi101.stdout: 1 462145 8.42 MB/sec execute 182 sec latency 1549.680 ms 2023-07-18T03:13:38.584 INFO:tasks.workunit.client.0.smithi101.stdout: 1 462145 8.38 MB/sec execute 183 sec latency 2549.726 ms 2023-07-18T03:13:39.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:39 smithi179 ceph-mon[129229]: pgmap v185: 129 pgs: 129 active+clean; 298 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 151 op/s 2023-07-18T03:13:39.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:39 smithi101 ceph-mon[126379]: pgmap v185: 129 pgs: 129 active+clean; 298 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 151 op/s 2023-07-18T03:13:39.544 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:39 smithi155 ceph-mon[128885]: pgmap v185: 129 pgs: 129 active+clean; 298 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 151 op/s 2023-07-18T03:13:39.584 INFO:tasks.workunit.client.0.smithi101.stdout: 1 462259 8.33 MB/sec execute 184 sec latency 3392.824 ms 2023-07-18T03:13:40.584 INFO:tasks.workunit.client.0.smithi101.stdout: 1 462288 8.29 MB/sec execute 185 sec latency 662.675 ms 2023-07-18T03:13:41.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:41 smithi179 ceph-mon[129229]: pgmap v186: 129 pgs: 129 active+clean; 298 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 177 op/s 2023-07-18T03:13:41.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:41 smithi101 ceph-mon[126379]: pgmap v186: 129 pgs: 129 active+clean; 298 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 177 op/s 2023-07-18T03:13:41.544 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:41 smithi155 ceph-mon[128885]: pgmap v186: 129 pgs: 129 active+clean; 298 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 177 op/s 2023-07-18T03:13:41.584 INFO:tasks.workunit.client.0.smithi101.stdout: 1 463328 8.27 MB/sec execute 186 sec latency 475.993 ms 2023-07-18T03:13:42.584 INFO:tasks.workunit.client.0.smithi101.stdout: 1 465251 8.28 MB/sec execute 187 sec latency 42.868 ms 2023-07-18T03:13:43.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:43 smithi179 ceph-mon[129229]: pgmap v187: 129 pgs: 129 active+clean; 298 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 1.2 MiB/s wr, 85 op/s 2023-07-18T03:13:43.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:43 smithi101 ceph-mon[126379]: pgmap v187: 129 pgs: 129 active+clean; 298 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 1.2 MiB/s wr, 85 op/s 2023-07-18T03:13:43.544 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:43 smithi155 ceph-mon[128885]: pgmap v187: 129 pgs: 129 active+clean; 298 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 1.2 MiB/s wr, 85 op/s 2023-07-18T03:13:43.584 INFO:tasks.workunit.client.0.smithi101.stdout: 1 466553 8.27 MB/sec execute 188 sec latency 12.283 ms 2023-07-18T03:13:44.584 INFO:tasks.workunit.client.0.smithi101.stdout: 1 468632 8.28 MB/sec execute 189 sec latency 17.840 ms 2023-07-18T03:13:45.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:45 smithi179 ceph-mon[129229]: pgmap v188: 129 pgs: 129 active+clean; 298 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 1.2 MiB/s wr, 85 op/s 2023-07-18T03:13:45.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:45 smithi101 ceph-mon[126379]: pgmap v188: 129 pgs: 129 active+clean; 298 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 1.2 MiB/s wr, 85 op/s 2023-07-18T03:13:45.544 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:45 smithi155 ceph-mon[128885]: pgmap v188: 129 pgs: 129 active+clean; 298 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 1.2 MiB/s wr, 85 op/s 2023-07-18T03:13:45.585 INFO:tasks.workunit.client.0.smithi101.stdout: 1 469907 8.28 MB/sec execute 190 sec latency 40.747 ms 2023-07-18T03:13:46.585 INFO:tasks.workunit.client.0.smithi101.stdout: 1 472009 8.29 MB/sec execute 191 sec latency 24.463 ms 2023-07-18T03:13:47.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:47 smithi179 ceph-mon[129229]: pgmap v189: 129 pgs: 129 active+clean; 302 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 2.2 MiB/s wr, 164 op/s 2023-07-18T03:13:47.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:47 smithi101 ceph-mon[126379]: pgmap v189: 129 pgs: 129 active+clean; 302 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 2.2 MiB/s wr, 164 op/s 2023-07-18T03:13:47.544 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:47 smithi155 ceph-mon[128885]: pgmap v189: 129 pgs: 129 active+clean; 302 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 2.2 MiB/s wr, 164 op/s 2023-07-18T03:13:47.585 INFO:tasks.workunit.client.0.smithi101.stdout: 1 473159 8.29 MB/sec execute 192 sec latency 96.800 ms 2023-07-18T03:13:48.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:48 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:13:48.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:48 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:13:48.544 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:48 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:13:48.585 INFO:tasks.workunit.client.0.smithi101.stdout: 1 474974 8.28 MB/sec execute 193 sec latency 16.944 ms 2023-07-18T03:13:49.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:49 smithi179 ceph-mon[129229]: pgmap v190: 129 pgs: 129 active+clean; 302 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 1.6 MiB/s wr, 105 op/s 2023-07-18T03:13:49.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:49 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:13:49.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:49 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:13:49.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:49 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:13:49.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:49 smithi101 ceph-mon[126379]: pgmap v190: 129 pgs: 129 active+clean; 302 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 1.6 MiB/s wr, 105 op/s 2023-07-18T03:13:49.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:49 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:13:49.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:49 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:13:49.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:49 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:13:49.544 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:49 smithi155 ceph-mon[128885]: pgmap v190: 129 pgs: 129 active+clean; 302 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 1.6 MiB/s wr, 105 op/s 2023-07-18T03:13:49.545 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:49 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:13:49.545 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:49 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:13:49.545 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:49 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:13:49.585 INFO:tasks.workunit.client.0.smithi101.stdout: 1 476667 8.30 MB/sec execute 194 sec latency 11.108 ms 2023-07-18T03:13:50.585 INFO:tasks.workunit.client.0.smithi101.stdout: 1 478606 8.30 MB/sec execute 195 sec latency 14.293 ms 2023-07-18T03:13:51.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:51 smithi179 ceph-mon[129229]: pgmap v191: 129 pgs: 129 active+clean; 308 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 2.9 MiB/s wr, 195 op/s 2023-07-18T03:13:51.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:51 smithi101 ceph-mon[126379]: pgmap v191: 129 pgs: 129 active+clean; 308 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 2.9 MiB/s wr, 195 op/s 2023-07-18T03:13:51.544 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:51 smithi155 ceph-mon[128885]: pgmap v191: 129 pgs: 129 active+clean; 308 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 2.9 MiB/s wr, 195 op/s 2023-07-18T03:13:51.585 INFO:tasks.workunit.client.0.smithi101.stdout: 1 480224 8.31 MB/sec execute 196 sec latency 34.436 ms 2023-07-18T03:13:52.585 INFO:tasks.workunit.client.0.smithi101.stdout: 1 481372 8.30 MB/sec execute 197 sec latency 198.323 ms 2023-07-18T03:13:53.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:53 smithi179 ceph-mon[129229]: pgmap v192: 129 pgs: 129 active+clean; 308 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 2.3 MiB/s wr, 168 op/s 2023-07-18T03:13:53.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:53 smithi101 ceph-mon[126379]: pgmap v192: 129 pgs: 129 active+clean; 308 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 2.3 MiB/s wr, 168 op/s 2023-07-18T03:13:53.544 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:53 smithi155 ceph-mon[128885]: pgmap v192: 129 pgs: 129 active+clean; 308 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 2.3 MiB/s wr, 168 op/s 2023-07-18T03:13:53.585 INFO:tasks.workunit.client.0.smithi101.stdout: 1 483537 8.32 MB/sec execute 198 sec latency 23.152 ms 2023-07-18T03:13:54.585 INFO:tasks.workunit.client.0.smithi101.stdout: 1 485113 8.31 MB/sec execute 199 sec latency 12.683 ms 2023-07-18T03:13:55.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:55 smithi179 ceph-mon[129229]: pgmap v193: 129 pgs: 129 active+clean; 308 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 2.3 MiB/s wr, 168 op/s 2023-07-18T03:13:55.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:55 smithi101 ceph-mon[126379]: pgmap v193: 129 pgs: 129 active+clean; 308 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 2.3 MiB/s wr, 168 op/s 2023-07-18T03:13:55.544 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:55 smithi155 ceph-mon[128885]: pgmap v193: 129 pgs: 129 active+clean; 308 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 2.3 MiB/s wr, 168 op/s 2023-07-18T03:13:55.585 INFO:tasks.workunit.client.0.smithi101.stdout: 1 487151 8.34 MB/sec execute 200 sec latency 12.057 ms 2023-07-18T03:13:56.585 INFO:tasks.workunit.client.0.smithi101.stdout: 1 488536 8.32 MB/sec execute 201 sec latency 39.514 ms 2023-07-18T03:13:57.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:57 smithi179 ceph-mon[129229]: pgmap v194: 129 pgs: 129 active+clean; 314 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 260 op/s 2023-07-18T03:13:57.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:57 smithi101 ceph-mon[126379]: pgmap v194: 129 pgs: 129 active+clean; 314 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 260 op/s 2023-07-18T03:13:57.544 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:57 smithi155 ceph-mon[128885]: pgmap v194: 129 pgs: 129 active+clean; 314 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 260 op/s 2023-07-18T03:13:57.585 INFO:tasks.workunit.client.0.smithi101.stdout: 1 490542 8.35 MB/sec execute 202 sec latency 36.080 ms 2023-07-18T03:13:58.586 INFO:tasks.workunit.client.0.smithi101.stdout: 1 491827 8.33 MB/sec execute 203 sec latency 31.510 ms 2023-07-18T03:13:59.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:13:59 smithi179 ceph-mon[129229]: pgmap v195: 129 pgs: 129 active+clean; 314 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 180 op/s 2023-07-18T03:13:59.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:13:59 smithi101 ceph-mon[126379]: pgmap v195: 129 pgs: 129 active+clean; 314 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 180 op/s 2023-07-18T03:13:59.544 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:13:59 smithi155 ceph-mon[128885]: pgmap v195: 129 pgs: 129 active+clean; 314 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 180 op/s 2023-07-18T03:13:59.586 INFO:tasks.workunit.client.0.smithi101.stdout: 1 493998 8.35 MB/sec execute 204 sec latency 14.817 ms 2023-07-18T03:14:00.458 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:14:00.586 INFO:tasks.workunit.client.0.smithi101.stdout: 1 495371 8.34 MB/sec execute 205 sec latency 11.165 ms 2023-07-18T03:14:00.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.674+0000 7fd4b49ff700 1 -- 172.21.15.101:0/606492810 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4b0100480 msgr2=0x7fd4b010c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:00.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.674+0000 7fd4b49ff700 1 --2- 172.21.15.101:0/606492810 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4b0100480 0x7fd4b010c910 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd4a800b0a0 tx=0x7fd4a800b3b0 comp rx=0 tx=0).stop 2023-07-18T03:14:00.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.674+0000 7fd4b49ff700 1 -- 172.21.15.101:0/606492810 shutdown_connections 2023-07-18T03:14:00.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.674+0000 7fd4b49ff700 1 --2- 172.21.15.101:0/606492810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4b0105ae0 0x7fd4b0105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:00.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.674+0000 7fd4b49ff700 1 --2- 172.21.15.101:0/606492810 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4b0100480 0x7fd4b010c910 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:00.668 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.674+0000 7fd4b49ff700 1 --2- 172.21.15.101:0/606492810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4b00ffad0 0x7fd4b00fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:00.668 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.674+0000 7fd4b49ff700 1 -- 172.21.15.101:0/606492810 >> 172.21.15.101:0/606492810 conn(0x7fd4b00fb5f0 msgr2=0x7fd4b00fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:00.668 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.674+0000 7fd4b49ff700 1 -- 172.21.15.101:0/606492810 shutdown_connections 2023-07-18T03:14:00.668 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.674+0000 7fd4b49ff700 1 -- 172.21.15.101:0/606492810 wait complete. 2023-07-18T03:14:00.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.675+0000 7fd4b49ff700 1 Processor -- start 2023-07-18T03:14:00.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.675+0000 7fd4b49ff700 1 -- start start 2023-07-18T03:14:00.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.675+0000 7fd4b49ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4b00ffad0 0x7fd4b0197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:00.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.675+0000 7fd4b49ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4b0100480 0x7fd4b0198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:00.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.675+0000 7fd4b49ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4b0105ae0 0x7fd4b019c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:00.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.675+0000 7fd4b49ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b019cec0 con 0x7fd4b0105ae0 2023-07-18T03:14:00.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.675+0000 7fd4b49ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b019d030 con 0x7fd4b0100480 2023-07-18T03:14:00.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.675+0000 7fd4b49ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b019d1f0 con 0x7fd4b00ffad0 2023-07-18T03:14:00.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.676+0000 7fd4ae59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4b00ffad0 0x7fd4b0197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:00.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.676+0000 7fd4aed9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4b0105ae0 0x7fd4b019c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:00.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.676+0000 7fd4ae59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4b00ffad0 0x7fd4b0197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:42926/0 (socket says 172.21.15.101:42926) 2023-07-18T03:14:00.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.676+0000 7fd4ae59c700 1 -- 172.21.15.101:0/3431271771 learned_addr learned my addr 172.21.15.101:0/3431271771 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:00.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.676+0000 7fd4add9b700 1 --2- 172.21.15.101:0/3431271771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4b0100480 0x7fd4b0198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:00.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.676+0000 7fd4aed9d700 1 -- 172.21.15.101:0/3431271771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4b00ffad0 msgr2=0x7fd4b0197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:00.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.676+0000 7fd4aed9d700 1 --2- 172.21.15.101:0/3431271771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4b00ffad0 0x7fd4b0197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:00.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.676+0000 7fd4aed9d700 1 -- 172.21.15.101:0/3431271771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4b0100480 msgr2=0x7fd4b0198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:00.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.676+0000 7fd4aed9d700 1 --2- 172.21.15.101:0/3431271771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4b0100480 0x7fd4b0198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:00.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.676+0000 7fd4aed9d700 1 -- 172.21.15.101:0/3431271771 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a8009d00 con 0x7fd4b0105ae0 2023-07-18T03:14:00.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.676+0000 7fd4add9b700 1 --2- 172.21.15.101:0/3431271771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4b0100480 0x7fd4b0198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:14:00.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.676+0000 7fd4aed9d700 1 --2- 172.21.15.101:0/3431271771 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4b0105ae0 0x7fd4b019c880 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8009fd0 tx=0x7fd4a8007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:00.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.676+0000 7fd49f7fe700 1 -- 172.21.15.101:0/3431271771 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a800e040 con 0x7fd4b0105ae0 2023-07-18T03:14:00.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.676+0000 7fd49f7fe700 1 -- 172.21.15.101:0/3431271771 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4a8004c50 con 0x7fd4b0105ae0 2023-07-18T03:14:00.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.676+0000 7fd4b49ff700 1 -- 172.21.15.101:0/3431271771 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4b019d470 con 0x7fd4b0105ae0 2023-07-18T03:14:00.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.677+0000 7fd49f7fe700 1 -- 172.21.15.101:0/3431271771 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a8005040 con 0x7fd4b0105ae0 2023-07-18T03:14:00.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.677+0000 7fd4b49ff700 1 -- 172.21.15.101:0/3431271771 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4b01a4d10 con 0x7fd4b0105ae0 2023-07-18T03:14:00.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.680+0000 7fd49f7fe700 1 -- 172.21.15.101:0/3431271771 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fd4a801d470 con 0x7fd4b0105ae0 2023-07-18T03:14:00.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.681+0000 7fd4c8315b80 1 Processor -- start 2023-07-18T03:14:00.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.681+0000 7fd4c8315b80 1 -- start start 2023-07-18T03:14:00.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.681+0000 7fd49f7fe700 1 --2- 172.21.15.101:0/3431271771 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd4980a1340 0x7fd4980a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:00.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.681+0000 7fd4c8315b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5629272252d0 0x5629272256a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:00.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.681+0000 7fd4c8315b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562927225be0 0x562927230590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:00.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.681+0000 7fd4c8315b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562927230ad0 0x562927232eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:00.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.681+0000 7fd4c8315b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56292712bcb0 con 0x5629272252d0 2023-07-18T03:14:00.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.681+0000 7fd4c8315b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56292717ea40 con 0x562927225be0 2023-07-18T03:14:00.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.681+0000 7fd4c8315b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562927177ed0 con 0x562927230ad0 2023-07-18T03:14:00.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.682+0000 7fd49f7fe700 1 -- 172.21.15.101:0/3431271771 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fd4a80ce610 con 0x7fd4b0105ae0 2023-07-18T03:14:00.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.682+0000 7fd4add9b700 1 --2- 172.21.15.101:0/3431271771 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd4980a1340 0x7fd4980a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:00.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.682+0000 7fd4ae59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5629272252d0 0x5629272256a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:00.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.682+0000 7fd4aed9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562927230ad0 0x562927232eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:00.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.682+0000 7fd4ae59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5629272252d0 0x5629272256a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:46510/0 (socket says 172.21.15.101:46510) 2023-07-18T03:14:00.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.682+0000 7fd4ae59c700 1 -- 172.21.15.101:0/432532067 learned_addr learned my addr 172.21.15.101:0/432532067 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:00.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.682+0000 7fd4add9b700 1 --2- 172.21.15.101:0/432532067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562927225be0 0x562927230590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:00.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.682+0000 7fd4aed9d700 1 -- 172.21.15.101:0/432532067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562927225be0 msgr2=0x562927230590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:00.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.682+0000 7fd4aed9d700 1 --2- 172.21.15.101:0/432532067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562927225be0 0x562927230590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:00.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.682+0000 7fd4aed9d700 1 -- 172.21.15.101:0/432532067 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5629272252d0 msgr2=0x5629272256a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:00.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.682+0000 7fd4aed9d700 1 --2- 172.21.15.101:0/432532067 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5629272252d0 0x5629272256a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:00.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.682+0000 7fd4aed9d700 1 -- 172.21.15.101:0/432532067 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562926fa5de0 con 0x562927230ad0 2023-07-18T03:14:00.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.682+0000 7fd4add9b700 1 --2- 172.21.15.101:0/432532067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562927225be0 0x562927230590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:14:00.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.682+0000 7fd4aed9d700 1 --2- 172.21.15.101:0/432532067 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562927230ad0 0x562927232eb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd4a80cb9b0 tx=0x7fd4a8007a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=deb37716b5d4a693 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:00.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.683+0000 7fd49cff9700 1 -- 172.21.15.101:0/432532067 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a802e350 con 0x562927230ad0 2023-07-18T03:14:00.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.683+0000 7fd49cff9700 1 -- 172.21.15.101:0/432532067 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4a809b3c0 con 0x562927230ad0 2023-07-18T03:14:00.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.683+0000 7fd49cff9700 1 -- 172.21.15.101:0/432532067 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a8097ad0 con 0x562927230ad0 2023-07-18T03:14:00.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.683+0000 7fd4c8315b80 1 -- 172.21.15.101:0/432532067 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562927230ad0 msgr2=0x562927232eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:00.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.683+0000 7fd4add9b700 1 --2- 172.21.15.101:0/3431271771 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd4980a1340 0x7fd4980a37f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd4b01994f0 tx=0x7fd4a4009890 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:00.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.683+0000 7fd4c8315b80 1 --2- 172.21.15.101:0/432532067 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562927230ad0 0x562927232eb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd4a80cb9b0 tx=0x7fd4a8007a90 comp rx=0 tx=0).stop 2023-07-18T03:14:00.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.683+0000 7fd4c8315b80 1 -- 172.21.15.101:0/432532067 shutdown_connections 2023-07-18T03:14:00.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.683+0000 7fd4c8315b80 1 --2- 172.21.15.101:0/432532067 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562927230ad0 0x562927232eb0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:00.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.683+0000 7fd4c8315b80 1 --2- 172.21.15.101:0/432532067 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5629272252d0 0x5629272256a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:00.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.683+0000 7fd4c8315b80 1 --2- 172.21.15.101:0/432532067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562927225be0 0x562927230590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:00.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.683+0000 7fd4c8315b80 1 -- 172.21.15.101:0/432532067 >> 172.21.15.101:0/432532067 conn(0x56292721bda0 msgr2=0x5629272216a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:00.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.683+0000 7fd4c8315b80 1 -- 172.21.15.101:0/432532067 shutdown_connections 2023-07-18T03:14:00.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.683+0000 7fd4c8315b80 1 -- 172.21.15.101:0/432532067 wait complete. 2023-07-18T03:14:00.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.684+0000 7fd4c8315b80 1 -- start start 2023-07-18T03:14:00.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.684+0000 7fd4c8315b80 1 Processor -- start 2023-07-18T03:14:00.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.685+0000 7fd4c8315b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629272252d0 0x562927230ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:00.690 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.685+0000 7fd4c8315b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562927225be0 0x5629272313e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:00.690 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.685+0000 7fd4c8315b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562927230ad0 0x56292723bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:00.690 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.685+0000 7fd4c8315b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562927117ac0 con 0x562927230ad0 2023-07-18T03:14:00.690 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.685+0000 7fd4c8315b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629271177c0 con 0x5629272252d0 2023-07-18T03:14:00.690 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.685+0000 7fd4c8315b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562926ffb920 con 0x562927225be0 2023-07-18T03:14:00.690 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.685+0000 7fd4c8315b80 10 client.0 _ll_register_callbacks cb 0x562927115760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:14:00.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.685+0000 7fd4aed9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562927230ad0 0x56292723bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:00.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.685+0000 7fd4ae59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629272252d0 0x562927230ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:00.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.685+0000 7fd4aed9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562927230ad0 0x56292723bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:46520/0 (socket says 172.21.15.101:46520) 2023-07-18T03:14:00.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.685+0000 7fd4ae59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629272252d0 0x562927230ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:55026/0 (socket says 172.21.15.101:55026) 2023-07-18T03:14:00.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.685+0000 7fd4aed9d700 1 -- 172.21.15.101:0/2864760497 learned_addr learned my addr 172.21.15.101:0/2864760497 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:00.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.685+0000 7fd4add9b700 1 --2- 172.21.15.101:0/2864760497 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562927225be0 0x5629272313e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:00.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.685+0000 7fd4aed9d700 1 -- 172.21.15.101:0/2864760497 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562927225be0 msgr2=0x5629272313e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:00.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.685+0000 7fd4aed9d700 1 --2- 172.21.15.101:0/2864760497 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562927225be0 0x5629272313e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:00.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.685+0000 7fd4aed9d700 1 -- 172.21.15.101:0/2864760497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629272252d0 msgr2=0x562927230ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:00.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.685+0000 7fd4aed9d700 1 --2- 172.21.15.101:0/2864760497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629272252d0 0x562927230ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:00.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.685+0000 7fd4aed9d700 1 -- 172.21.15.101:0/2864760497 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4b01a4d10 con 0x562927230ad0 2023-07-18T03:14:00.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.685+0000 7fd4add9b700 1 --2- 172.21.15.101:0/2864760497 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562927225be0 0x5629272313e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:14:00.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.685+0000 7fd4ae59c700 1 --2- 172.21.15.101:0/2864760497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629272252d0 0x562927230ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:14:00.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.686+0000 7fd4aed9d700 1 --2- 172.21.15.101:0/2864760497 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562927230ad0 0x56292723bf20 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8009820 tx=0x7fd4a80099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bef7994be70d2ca server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:00.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.686+0000 7fd495ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:14:00.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.686+0000 7fd495ffb700 1 -- 172.21.15.101:0/2864760497 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a80a12d0 con 0x562927230ad0 2023-07-18T03:14:00.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.686+0000 7fd4c8315b80 1 -- 172.21.15.101:0/2864760497 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5629270d14f0 con 0x562927230ad0 2023-07-18T03:14:00.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.686+0000 7fd495ffb700 1 -- 172.21.15.101:0/2864760497 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fd4a809f3f0 con 0x562927230ad0 2023-07-18T03:14:00.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.686+0000 7fd495ffb700 1 -- 172.21.15.101:0/2864760497 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4a80cbc30 con 0x562927230ad0 2023-07-18T03:14:00.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.686+0000 7fd4c8315b80 10 client.16794 fetch_fsmap learned FSMap version 28 2023-07-18T03:14:00.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.686+0000 7fd4c8315b80 1 -- 172.21.15.101:0/2864760497 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562926f8ec60 con 0x562927230ad0 2023-07-18T03:14:00.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.686+0000 7fd495ffb700 1 -- 172.21.15.101:0/2864760497 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5629270d14f0 con 0x562927230ad0 2023-07-18T03:14:00.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.686+0000 7fd495ffb700 1 -- 172.21.15.101:0/2864760497 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fd4a80a32f0 con 0x562927230ad0 2023-07-18T03:14:00.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.686+0000 7fd4c8315b80 10 client.16794 fetch_fsmap finished waiting for FSMap version 28 2023-07-18T03:14:00.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.686+0000 7fd4c8315b80 10 client.16794 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:14:00.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.686+0000 7fd4c8315b80 20 client.16794 populate_metadata read hostname 'smithi101' 2023-07-18T03:14:00.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.686+0000 7fd4c8315b80 1 --2- 172.21.15.101:0/2864760497 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56292723fcc0 0x5629272420a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:00.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.686+0000 7fd4c8315b80 4 client.16794 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:14:00.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.686+0000 7fd4c8315b80 1 -- 172.21.15.101:0/2864760497 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562927170eb0 con 0x56292723fcc0 2023-07-18T03:14:00.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.687+0000 7fd4ae59c700 1 --2- 172.21.15.101:0/2864760497 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56292723fcc0 0x5629272420a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:00.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.687+0000 7fd495ffb700 10 client.16794 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:14:00.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.687+0000 7fd4ae59c700 1 --2- 172.21.15.101:0/2864760497 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56292723fcc0 0x5629272420a0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf6ebac1ff9613a1 server_cookie=1aca4349bee6ae86 in_seq=0 out_seq=0 2023-07-18T03:14:00.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.688+0000 7fd495ffb700 1 -- 172.21.15.101:0/2864760497 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562927170eb0 con 0x56292723fcc0 2023-07-18T03:14:00.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.688+0000 7fd495ffb700 10 client.16794 handle_command_reply: tid=0 2023-07-18T03:14:00.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.688+0000 7fd4c8315b80 1 client.16794 shutdown 2023-07-18T03:14:00.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.688+0000 7fd4c8315b80 1 -- 172.21.15.101:0/2864760497 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562927230ad0 msgr2=0x56292723bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:00.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.688+0000 7fd4c8315b80 1 --2- 172.21.15.101:0/2864760497 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562927230ad0 0x56292723bf20 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8009820 tx=0x7fd4a80099e0 comp rx=0 tx=0).stop 2023-07-18T03:14:00.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.688+0000 7fd4c8315b80 1 -- 172.21.15.101:0/2864760497 shutdown_connections 2023-07-18T03:14:00.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.688+0000 7fd4c8315b80 1 --2- 172.21.15.101:0/2864760497 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56292723fcc0 0x5629272420a0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:00.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.688+0000 7fd4c8315b80 1 --2- 172.21.15.101:0/2864760497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5629272252d0 0x562927230ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:00.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.688+0000 7fd4c8315b80 1 --2- 172.21.15.101:0/2864760497 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562927230ad0 0x56292723bf20 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:00.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.688+0000 7fd495ffb700 0 client.16794 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:14:00.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.688+0000 7fd4c8315b80 1 --2- 172.21.15.101:0/2864760497 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562927225be0 0x5629272313e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:00.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.688+0000 7fd4c8315b80 1 -- 172.21.15.101:0/2864760497 >> 172.21.15.101:0/2864760497 conn(0x56292721bda0 msgr2=0x562927221230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:00.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.688+0000 7fd4c8315b80 1 -- 172.21.15.101:0/2864760497 shutdown_connections 2023-07-18T03:14:00.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.688+0000 7fd4c8315b80 1 -- 172.21.15.101:0/2864760497 wait complete. 2023-07-18T03:14:00.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.689+0000 7fd4c8315b80 20 client.16794 trim_cache size 0 max 16384 2023-07-18T03:14:00.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.705+0000 7fd4c8315b80 1 Processor -- start 2023-07-18T03:14:00.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.706+0000 7fd4c8315b80 1 -- start start 2023-07-18T03:14:00.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.706+0000 7fd4c8315b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629272252d0 0x562927235eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:00.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.706+0000 7fd4c8315b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562927225be0 0x56292722cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:00.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.706+0000 7fd4c8315b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562927230ad0 0x56292722d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:00.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.706+0000 7fd4c8315b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562927222ad0 con 0x562927230ad0 2023-07-18T03:14:00.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.706+0000 7fd4c8315b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562927226540 con 0x562927225be0 2023-07-18T03:14:00.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.706+0000 7fd4c8315b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562927225fb0 con 0x5629272252d0 2023-07-18T03:14:00.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.706+0000 7fd4aed9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562927230ad0 0x56292722d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:00.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.706+0000 7fd4aed9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562927230ad0 0x56292722d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:46534/0 (socket says 172.21.15.101:46534) 2023-07-18T03:14:00.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.706+0000 7fd4aed9d700 1 -- 172.21.15.101:0/3053971491 learned_addr learned my addr 172.21.15.101:0/3053971491 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:00.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.706+0000 7fd4ae59c700 1 --2- 172.21.15.101:0/3053971491 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629272252d0 0x562927235eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:00.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.706+0000 7fd4add9b700 1 --2- 172.21.15.101:0/3053971491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562927225be0 0x56292722cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:00.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.706+0000 7fd4aed9d700 1 -- 172.21.15.101:0/3053971491 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629272252d0 msgr2=0x562927235eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:00.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.706+0000 7fd4aed9d700 1 --2- 172.21.15.101:0/3053971491 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629272252d0 0x562927235eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:00.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.707+0000 7fd4aed9d700 1 -- 172.21.15.101:0/3053971491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562927225be0 msgr2=0x56292722cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:00.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.707+0000 7fd4aed9d700 1 --2- 172.21.15.101:0/3053971491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562927225be0 0x56292722cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:00.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.707+0000 7fd4aed9d700 1 -- 172.21.15.101:0/3053971491 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562926fcc250 con 0x562927230ad0 2023-07-18T03:14:00.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.707+0000 7fd4add9b700 1 --2- 172.21.15.101:0/3053971491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562927225be0 0x56292722cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:14:00.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.707+0000 7fd4aed9d700 1 --2- 172.21.15.101:0/3053971491 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562927230ad0 0x56292722d400 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd4a800b780 tx=0x7fd4a800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dc008b31df8279f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:00.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.707+0000 7fd49d7fa700 1 -- 172.21.15.101:0/3053971491 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a80a12d0 con 0x562927230ad0 2023-07-18T03:14:00.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.707+0000 7fd49d7fa700 1 -- 172.21.15.101:0/3053971491 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4a809f3f0 con 0x562927230ad0 2023-07-18T03:14:00.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.707+0000 7fd49d7fa700 1 -- 172.21.15.101:0/3053971491 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a80cbc30 con 0x562927230ad0 2023-07-18T03:14:00.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.707+0000 7fd4c8315b80 1 -- 172.21.15.101:0/3053971491 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562927230ad0 msgr2=0x56292722d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:00.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.708+0000 7fd4c8315b80 1 --2- 172.21.15.101:0/3053971491 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562927230ad0 0x56292722d400 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd4a800b780 tx=0x7fd4a800b040 comp rx=0 tx=0).stop 2023-07-18T03:14:00.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.708+0000 7fd4c8315b80 1 -- 172.21.15.101:0/3053971491 shutdown_connections 2023-07-18T03:14:00.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.708+0000 7fd4c8315b80 1 --2- 172.21.15.101:0/3053971491 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562927230ad0 0x56292722d400 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:00.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.708+0000 7fd4c8315b80 1 --2- 172.21.15.101:0/3053971491 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629272252d0 0x562927235eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:00.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.708+0000 7fd4c8315b80 1 --2- 172.21.15.101:0/3053971491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562927225be0 0x56292722cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:00.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.708+0000 7fd4c8315b80 1 -- 172.21.15.101:0/3053971491 >> 172.21.15.101:0/3053971491 conn(0x56292721bda0 msgr2=0x562927230ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:00.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.708+0000 7fd4c8315b80 1 -- 172.21.15.101:0/3053971491 shutdown_connections 2023-07-18T03:14:00.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.708+0000 7fd4c8315b80 1 -- 172.21.15.101:0/3053971491 wait complete. 2023-07-18T03:14:00.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.708+0000 7fd4c8315b80 1 -- start start 2023-07-18T03:14:00.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.708+0000 7fd4c8315b80 1 Processor -- start 2023-07-18T03:14:00.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.709+0000 7fd4c8315b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629272252d0 0x562927235b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:00.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.709+0000 7fd4c8315b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562927225be0 0x562927236060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:00.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.709+0000 7fd4c8315b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562927230ad0 0x562927234050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:00.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.709+0000 7fd4c8315b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562927234760 con 0x562927225be0 2023-07-18T03:14:00.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.709+0000 7fd4c8315b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629272348d0 con 0x562927230ad0 2023-07-18T03:14:00.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.709+0000 7fd4c8315b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562927234a90 con 0x5629272252d0 2023-07-18T03:14:00.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.709+0000 7fd4c8315b80 10 client.0 _ll_register_callbacks cb 0x7fd4a00095e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:14:00.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.709+0000 7fd4add9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562927225be0 0x562927236060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:00.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.709+0000 7fd4add9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562927225be0 0x562927236060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:46540/0 (socket says 172.21.15.101:46540) 2023-07-18T03:14:00.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.709+0000 7fd4add9b700 1 -- 172.21.15.101:0/209210402 learned_addr learned my addr 172.21.15.101:0/209210402 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:00.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.709+0000 7fd4ae59c700 1 --2- 172.21.15.101:0/209210402 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629272252d0 0x562927235b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:00.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.709+0000 7fd4aed9d700 1 --2- 172.21.15.101:0/209210402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562927230ad0 0x562927234050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:00.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.710+0000 7fd4aed9d700 1 -- 172.21.15.101:0/209210402 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629272252d0 msgr2=0x562927235b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:00.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.710+0000 7fd4aed9d700 1 --2- 172.21.15.101:0/209210402 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629272252d0 0x562927235b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:00.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.710+0000 7fd4aed9d700 1 -- 172.21.15.101:0/209210402 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562927225be0 msgr2=0x562927236060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:00.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.710+0000 7fd4aed9d700 1 --2- 172.21.15.101:0/209210402 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562927225be0 0x562927236060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:00.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.710+0000 7fd4aed9d700 1 -- 172.21.15.101:0/209210402 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562926f8ec60 con 0x562927230ad0 2023-07-18T03:14:00.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.710+0000 7fd4ae59c700 1 --2- 172.21.15.101:0/209210402 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629272252d0 0x562927235b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:14:00.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.710+0000 7fd4add9b700 1 --2- 172.21.15.101:0/209210402 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562927225be0 0x562927236060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:14:00.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.710+0000 7fd4aed9d700 1 --2- 172.21.15.101:0/209210402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562927230ad0 0x562927234050 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd4a800b780 tx=0x7fd4a800b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd3e880033af678c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:00.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.710+0000 7fd495ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:14:00.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.710+0000 7fd495ffb700 1 -- 172.21.15.101:0/209210402 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a80a12d0 con 0x562927230ad0 2023-07-18T03:14:00.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.710+0000 7fd4c8315b80 1 -- 172.21.15.101:0/209210402 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd4a809b3c0 con 0x562927230ad0 2023-07-18T03:14:00.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.710+0000 7fd495ffb700 1 -- 172.21.15.101:0/209210402 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fd4a8099500 con 0x562927230ad0 2023-07-18T03:14:00.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.710+0000 7fd495ffb700 1 -- 172.21.15.101:0/209210402 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4a80cbc30 con 0x562927230ad0 2023-07-18T03:14:00.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.710+0000 7fd4c8315b80 10 client.26840 fetch_fsmap learned FSMap version 28 2023-07-18T03:14:00.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.710+0000 7fd4c8315b80 1 -- 172.21.15.101:0/209210402 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562926fa42b0 con 0x562927230ad0 2023-07-18T03:14:00.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.710+0000 7fd495ffb700 1 -- 172.21.15.101:0/209210402 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4a809b3c0 con 0x562927230ad0 2023-07-18T03:14:00.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.710+0000 7fd495ffb700 1 -- 172.21.15.101:0/209210402 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 28) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fd4a80c5b60 con 0x562927230ad0 2023-07-18T03:14:00.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.711+0000 7fd4c8315b80 10 client.26840 fetch_fsmap finished waiting for FSMap version 28 2023-07-18T03:14:00.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.711+0000 7fd4c8315b80 10 client.26840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:14:00.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.711+0000 7fd4c8315b80 20 client.26840 populate_metadata read hostname 'smithi101' 2023-07-18T03:14:00.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.711+0000 7fd4c8315b80 1 --2- 172.21.15.101:0/209210402 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x562927294930 0x562927296d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:00.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.711+0000 7fd4c8315b80 4 client.26840 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:14:00.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.711+0000 7fd4c8315b80 1 -- 172.21.15.101:0/209210402 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562927133050 con 0x562927294930 2023-07-18T03:14:00.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.711+0000 7fd4ae59c700 1 --2- 172.21.15.101:0/209210402 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x562927294930 0x562927296d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:00.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.711+0000 7fd4ae59c700 1 --2- 172.21.15.101:0/209210402 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x562927294930 0x562927296d10 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe41542decf9b63c server_cookie=aa58c8e5ac7a3ec3 in_seq=0 out_seq=0 2023-07-18T03:14:00.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.711+0000 7fd495ffb700 10 client.26840 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:14:00.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.711+0000 7fd495ffb700 1 -- 172.21.15.101:0/209210402 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562927133050 con 0x562927294930 2023-07-18T03:14:00.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.711+0000 7fd495ffb700 10 client.26840 handle_command_reply: tid=0 2023-07-18T03:14:00.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.711+0000 7fd4c8315b80 1 client.26840 shutdown 2023-07-18T03:14:00.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.712+0000 7fd4c8315b80 1 -- 172.21.15.101:0/209210402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562927230ad0 msgr2=0x562927234050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:00.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.712+0000 7fd4c8315b80 1 --2- 172.21.15.101:0/209210402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562927230ad0 0x562927234050 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd4a800b780 tx=0x7fd4a800b3b0 comp rx=0 tx=0).stop 2023-07-18T03:14:00.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.712+0000 7fd4c8315b80 1 -- 172.21.15.101:0/209210402 shutdown_connections 2023-07-18T03:14:00.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.712+0000 7fd4c8315b80 1 --2- 172.21.15.101:0/209210402 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x562927294930 0x562927296d10 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:00.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.712+0000 7fd4c8315b80 1 --2- 172.21.15.101:0/209210402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562927230ad0 0x562927234050 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:00.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.712+0000 7fd4c8315b80 1 --2- 172.21.15.101:0/209210402 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629272252d0 0x562927235b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:00.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.712+0000 7fd4c8315b80 1 --2- 172.21.15.101:0/209210402 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562927225be0 0x562927236060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:00.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.712+0000 7fd495ffb700 0 client.26840 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:14:00.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.712+0000 7fd4c8315b80 1 -- 172.21.15.101:0/209210402 >> 172.21.15.101:0/209210402 conn(0x56292721bda0 msgr2=0x5629272273e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:00.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.712+0000 7fd4c8315b80 1 -- 172.21.15.101:0/209210402 shutdown_connections 2023-07-18T03:14:00.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.712+0000 7fd4c8315b80 1 -- 172.21.15.101:0/209210402 wait complete. 2023-07-18T03:14:00.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.712+0000 7fd4c8315b80 20 client.26840 trim_cache size 0 max 16384 2023-07-18T03:14:00.716 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:14:00.716 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "no active scrubs running", 2023-07-18T03:14:00.716 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": {} 2023-07-18T03:14:00.716 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:14:00.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.712+0000 7fd4b49ff700 1 -- 172.21.15.101:0/3431271771 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd4980a1340 msgr2=0x7fd4980a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:00.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.712+0000 7fd4b49ff700 1 --2- 172.21.15.101:0/3431271771 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd4980a1340 0x7fd4980a37f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd4b01994f0 tx=0x7fd4a4009890 comp rx=0 tx=0).stop 2023-07-18T03:14:00.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.713+0000 7fd4b49ff700 1 -- 172.21.15.101:0/3431271771 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4b0105ae0 msgr2=0x7fd4b019c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:00.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.713+0000 7fd4b49ff700 1 --2- 172.21.15.101:0/3431271771 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4b0105ae0 0x7fd4b019c880 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8009fd0 tx=0x7fd4a8007930 comp rx=0 tx=0).stop 2023-07-18T03:14:00.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.713+0000 7fd4b49ff700 1 -- 172.21.15.101:0/3431271771 shutdown_connections 2023-07-18T03:14:00.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.713+0000 7fd4b49ff700 1 --2- 172.21.15.101:0/3431271771 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4b0105ae0 0x7fd4b019c880 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:00.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.713+0000 7fd4b49ff700 1 --2- 172.21.15.101:0/3431271771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4b00ffad0 0x7fd4b0197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:00.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.713+0000 7fd4b49ff700 1 --2- 172.21.15.101:0/3431271771 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd4980a1340 0x7fd4980a37f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:00.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.713+0000 7fd4b49ff700 1 --2- 172.21.15.101:0/3431271771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4b0100480 0x7fd4b0198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:00.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.713+0000 7fd4b49ff700 1 -- 172.21.15.101:0/3431271771 >> 172.21.15.101:0/3431271771 conn(0x7fd4b00fb5f0 msgr2=0x7fd4b010ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:00.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.713+0000 7fd4b49ff700 1 -- 172.21.15.101:0/3431271771 shutdown_connections 2023-07-18T03:14:00.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.713+0000 7fd4b49ff700 1 -- 172.21.15.101:0/3431271771 wait complete. 2023-07-18T03:14:00.722 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-18T03:14:00.723 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:14:00.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.955+0000 7f7be359d700 1 -- 172.21.15.101:0/1730262417 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7bdc0ff9f0 msgr2=0x7f7bdc0ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:00.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.955+0000 7f7be359d700 1 --2- 172.21.15.101:0/1730262417 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7bdc0ff9f0 0x7f7bdc0ffe60 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc009990 tx=0x7f7bcc009ca0 comp rx=0 tx=0).stop 2023-07-18T03:14:00.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.956+0000 7f7be359d700 1 -- 172.21.15.101:0/1730262417 shutdown_connections 2023-07-18T03:14:00.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.956+0000 7f7be359d700 1 --2- 172.21.15.101:0/1730262417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bdc1003a0 0x7f7bdc10c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:00.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.956+0000 7f7be359d700 1 --2- 172.21.15.101:0/1730262417 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7bdc105a00 0x7f7bdc105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:00.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.956+0000 7f7be359d700 1 --2- 172.21.15.101:0/1730262417 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7bdc0ff9f0 0x7f7bdc0ffe60 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:00.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.956+0000 7f7be359d700 1 -- 172.21.15.101:0/1730262417 >> 172.21.15.101:0/1730262417 conn(0x7f7bdc0fb510 msgr2=0x7f7bdc0fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:00.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.956+0000 7f7be359d700 1 -- 172.21.15.101:0/1730262417 shutdown_connections 2023-07-18T03:14:00.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.956+0000 7f7be359d700 1 -- 172.21.15.101:0/1730262417 wait complete. 2023-07-18T03:14:00.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.957+0000 7f7be359d700 1 Processor -- start 2023-07-18T03:14:00.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.957+0000 7f7be359d700 1 -- start start 2023-07-18T03:14:00.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.958+0000 7f7be359d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bdc0ff9f0 0x7f7bdc19d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:00.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.958+0000 7f7be359d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7bdc1003a0 0x7f7bdc19ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:00.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.958+0000 7f7be359d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7bdc105a00 0x7f7bdc1978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:00.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.958+0000 7f7be359d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bdc197f50 con 0x7f7bdc105a00 2023-07-18T03:14:00.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.958+0000 7f7be359d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bdc1980c0 con 0x7f7bdc0ff9f0 2023-07-18T03:14:00.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.958+0000 7f7be359d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bdc198280 con 0x7f7bdc1003a0 2023-07-18T03:14:00.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.958+0000 7f7be1d9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7bdc1003a0 0x7f7bdc19ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:00.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.958+0000 7f7be1d9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7bdc1003a0 0x7f7bdc19ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:42986/0 (socket says 172.21.15.101:42986) 2023-07-18T03:14:00.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.958+0000 7f7be1d9a700 1 -- 172.21.15.101:0/3222873517 learned_addr learned my addr 172.21.15.101:0/3222873517 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:00.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.958+0000 7f7be2d9c700 1 --2- 172.21.15.101:0/3222873517 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7bdc105a00 0x7f7bdc1978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:00.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.958+0000 7f7be259b700 1 --2- 172.21.15.101:0/3222873517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bdc0ff9f0 0x7f7bdc19d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:00.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.958+0000 7f7be1d9a700 1 -- 172.21.15.101:0/3222873517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bdc0ff9f0 msgr2=0x7f7bdc19d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:00.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.958+0000 7f7be1d9a700 1 --2- 172.21.15.101:0/3222873517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bdc0ff9f0 0x7f7bdc19d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:00.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.958+0000 7f7be1d9a700 1 -- 172.21.15.101:0/3222873517 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7bdc105a00 msgr2=0x7f7bdc1978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:00.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.958+0000 7f7be1d9a700 1 --2- 172.21.15.101:0/3222873517 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7bdc105a00 0x7f7bdc1978e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:00.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.958+0000 7f7be1d9a700 1 -- 172.21.15.101:0/3222873517 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bcc0095f0 con 0x7f7bdc1003a0 2023-07-18T03:14:00.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.958+0000 7f7be259b700 1 --2- 172.21.15.101:0/3222873517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bdc0ff9f0 0x7f7bdc19d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:14:00.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.959+0000 7f7be1d9a700 1 --2- 172.21.15.101:0/3222873517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7bdc1003a0 0x7f7bdc19ddb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc005910 tx=0x7f7bcc005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:00.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.959+0000 7f7bd37fe700 1 -- 172.21.15.101:0/3222873517 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc01d070 con 0x7f7bdc1003a0 2023-07-18T03:14:00.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.959+0000 7f7bd37fe700 1 -- 172.21.15.101:0/3222873517 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bcc022470 con 0x7f7bdc1003a0 2023-07-18T03:14:00.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.959+0000 7f7be359d700 1 -- 172.21.15.101:0/3222873517 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bdc198500 con 0x7f7bdc1003a0 2023-07-18T03:14:00.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.959+0000 7f7bd37fe700 1 -- 172.21.15.101:0/3222873517 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc017650 con 0x7f7bdc1003a0 2023-07-18T03:14:00.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.959+0000 7f7be359d700 1 -- 172.21.15.101:0/3222873517 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bdc1a4860 con 0x7f7bdc1003a0 2023-07-18T03:14:00.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.960+0000 7f7be359d700 1 -- 172.21.15.101:0/3222873517 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bdc067220 con 0x7f7bdc1003a0 2023-07-18T03:14:00.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.962+0000 7f7bd37fe700 1 -- 172.21.15.101:0/3222873517 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f7bcc0225e0 con 0x7f7bdc1003a0 2023-07-18T03:14:00.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.964+0000 7f7bd37fe700 1 --2- 172.21.15.101:0/3222873517 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f7bc80a13b0 0x7f7bc80a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:00.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.964+0000 7f7bd37fe700 1 -- 172.21.15.101:0/3222873517 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f7bcc0ce5e0 con 0x7f7bdc1003a0 2023-07-18T03:14:00.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.964+0000 7f7be259b700 1 --2- 172.21.15.101:0/3222873517 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f7bc80a13b0 0x7f7bc80a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:00.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.964+0000 7f7be259b700 1 --2- 172.21.15.101:0/3222873517 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f7bc80a13b0 0x7f7bc80a3860 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc100b30 tx=0x7f7bd8009260 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:00.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:00.967+0000 7f7bd37fe700 1 -- 172.21.15.101:0/3222873517 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7bcc0976d0 con 0x7f7bdc1003a0 2023-07-18T03:14:01.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.299+0000 7f7be359d700 1 -- 172.21.15.101:0/3222873517 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7bdc198df0 con 0x7f7bdc1003a0 2023-07-18T03:14:01.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.300+0000 7f7bd37fe700 1 -- 172.21.15.101:0/3222873517 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10863 (secure 0 0 0) 0x7f7bcc09acf0 con 0x7f7bdc1003a0 2023-07-18T03:14:01.293 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:14:01.293 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/4192846769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":4192846769},{"type":"v1","addr":"172.21.15.101:6839","nonce":4192846769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6841/3824340175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6840","nonce":3824340175},{"type":"v1","addr":"172.21.15.101:6841","nonce":3824340175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:04:59.361185+0000","modified":"2023-07-18T03:14:01.125192+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.155:6839/2055462483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2055462483},{"type":"v1","addr":"172.21.15.155:6839","nonce":2055462483}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:14:01.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.303+0000 7f7be359d700 1 -- 172.21.15.101:0/3222873517 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f7bc80a13b0 msgr2=0x7f7bc80a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:01.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.303+0000 7f7be359d700 1 --2- 172.21.15.101:0/3222873517 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f7bc80a13b0 0x7f7bc80a3860 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc100b30 tx=0x7f7bd8009260 comp rx=0 tx=0).stop 2023-07-18T03:14:01.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.303+0000 7f7be359d700 1 -- 172.21.15.101:0/3222873517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7bdc1003a0 msgr2=0x7f7bdc19ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:01.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.303+0000 7f7be359d700 1 --2- 172.21.15.101:0/3222873517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7bdc1003a0 0x7f7bdc19ddb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc005910 tx=0x7f7bcc005720 comp rx=0 tx=0).stop 2023-07-18T03:14:01.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.303+0000 7f7be359d700 1 -- 172.21.15.101:0/3222873517 shutdown_connections 2023-07-18T03:14:01.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.303+0000 7f7be359d700 1 --2- 172.21.15.101:0/3222873517 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f7bc80a13b0 0x7f7bc80a3860 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.303+0000 7f7be359d700 1 --2- 172.21.15.101:0/3222873517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bdc0ff9f0 0x7f7bdc19d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.303+0000 7f7be359d700 1 --2- 172.21.15.101:0/3222873517 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7bdc105a00 0x7f7bdc1978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.303+0000 7f7be359d700 1 --2- 172.21.15.101:0/3222873517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7bdc1003a0 0x7f7bdc19ddb0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.303+0000 7f7be359d700 1 -- 172.21.15.101:0/3222873517 >> 172.21.15.101:0/3222873517 conn(0x7f7bdc0fb510 msgr2=0x7f7bdc10abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:01.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.303+0000 7f7be359d700 1 -- 172.21.15.101:0/3222873517 shutdown_connections 2023-07-18T03:14:01.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.303+0000 7f7be359d700 1 -- 172.21.15.101:0/3222873517 wait complete. 2023-07-18T03:14:01.298 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 29 2023-07-18T03:14:01.317 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-18T03:14:01.391 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:01 smithi101 ceph-mon[126379]: pgmap v196: 129 pgs: 129 active+clean; 320 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.8 MiB/s wr, 271 op/s 2023-07-18T03:14:01.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:01 smithi179 ceph-mon[129229]: pgmap v196: 129 pgs: 129 active+clean; 320 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.8 MiB/s wr, 271 op/s 2023-07-18T03:14:01.544 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:01 smithi155 ceph-mon[128885]: pgmap v196: 129 pgs: 129 active+clean; 320 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.8 MiB/s wr, 271 op/s 2023-07-18T03:14:01.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.591+0000 7fdd87fff700 1 -- 172.21.15.101:0/2648096314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd80100490 msgr2=0x7fdd8010c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:01.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.591+0000 7fdd87fff700 1 --2- 172.21.15.101:0/2648096314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd80100490 0x7fdd8010c920 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdd7800b0a0 tx=0x7fdd7800b3b0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.591+0000 7fdd87fff700 1 -- 172.21.15.101:0/2648096314 shutdown_connections 2023-07-18T03:14:01.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.591+0000 7fdd87fff700 1 --2- 172.21.15.101:0/2648096314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd80100490 0x7fdd8010c920 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.591+0000 7fdd87fff700 1 --2- 172.21.15.101:0/2648096314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd80105af0 0x7fdd80105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.591+0000 7fdd87fff700 1 --2- 172.21.15.101:0/2648096314 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdd800ffae0 0x7fdd800fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.591+0000 7fdd87fff700 1 -- 172.21.15.101:0/2648096314 >> 172.21.15.101:0/2648096314 conn(0x7fdd800fb5e0 msgr2=0x7fdd800fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:01.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.591+0000 7fdd87fff700 1 -- 172.21.15.101:0/2648096314 shutdown_connections 2023-07-18T03:14:01.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.591+0000 7fdd87fff700 1 -- 172.21.15.101:0/2648096314 wait complete. 2023-07-18T03:14:01.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.592+0000 7fdd87fff700 1 Processor -- start 2023-07-18T03:14:01.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.592+0000 7fdd87fff700 1 -- start start 2023-07-18T03:14:01.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.592+0000 7fdd87fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd800ffae0 0x7fdd801998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:01.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.592+0000 7fdd87fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdd80100490 0x7fdd80199e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:01.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.592+0000 7fdd87fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd80105af0 0x7fdd80193950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:01.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.592+0000 7fdd87fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd80193fc0 con 0x7fdd80100490 2023-07-18T03:14:01.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.592+0000 7fdd87fff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd80194130 con 0x7fdd800ffae0 2023-07-18T03:14:01.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.592+0000 7fdd87fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd801942f0 con 0x7fdd80105af0 2023-07-18T03:14:01.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.592+0000 7fdd85d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd800ffae0 0x7fdd801998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:01.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.592+0000 7fdd85d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd800ffae0 0x7fdd801998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:55084/0 (socket says 172.21.15.101:55084) 2023-07-18T03:14:01.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.592+0000 7fdd85d9b700 1 -- 172.21.15.101:0/2175028360 learned_addr learned my addr 172.21.15.101:0/2175028360 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:01.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.592+0000 7fdd8559a700 1 --2- 172.21.15.101:0/2175028360 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdd80100490 0x7fdd80199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:01.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.592+0000 7fdd8659c700 1 --2- 172.21.15.101:0/2175028360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd80105af0 0x7fdd80193950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:01.588 INFO:tasks.workunit.client.0.smithi101.stdout: 1 496140 8.31 MB/sec execute 206 sec latency 161.369 ms 2023-07-18T03:14:01.589 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.593+0000 7fdd85d9b700 1 -- 172.21.15.101:0/2175028360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd80105af0 msgr2=0x7fdd80193950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:01.589 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.593+0000 7fdd85d9b700 1 --2- 172.21.15.101:0/2175028360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd80105af0 0x7fdd80193950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.589 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.593+0000 7fdd85d9b700 1 -- 172.21.15.101:0/2175028360 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdd80100490 msgr2=0x7fdd80199e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:01.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.593+0000 7fdd85d9b700 1 --2- 172.21.15.101:0/2175028360 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdd80100490 0x7fdd80199e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.593+0000 7fdd85d9b700 1 -- 172.21.15.101:0/2175028360 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd78009d00 con 0x7fdd800ffae0 2023-07-18T03:14:01.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.593+0000 7fdd8559a700 1 --2- 172.21.15.101:0/2175028360 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdd80100490 0x7fdd80199e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:14:01.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.593+0000 7fdd85d9b700 1 --2- 172.21.15.101:0/2175028360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd800ffae0 0x7fdd801998e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c00e9a0 tx=0x7fdd7c00ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:01.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.593+0000 7fdd76ffd700 1 -- 172.21.15.101:0/2175028360 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c009790 con 0x7fdd800ffae0 2023-07-18T03:14:01.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.593+0000 7fdd76ffd700 1 -- 172.21.15.101:0/2175028360 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd7c00cbd0 con 0x7fdd800ffae0 2023-07-18T03:14:01.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.593+0000 7fdd87fff700 1 -- 172.21.15.101:0/2175028360 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd801945d0 con 0x7fdd800ffae0 2023-07-18T03:14:01.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.593+0000 7fdd76ffd700 1 -- 172.21.15.101:0/2175028360 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c018680 con 0x7fdd800ffae0 2023-07-18T03:14:01.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.594+0000 7fdd87fff700 1 -- 172.21.15.101:0/2175028360 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd801a08d0 con 0x7fdd800ffae0 2023-07-18T03:14:01.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.598+0000 7fdd76ffd700 1 -- 172.21.15.101:0/2175028360 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fdd7c00cd40 con 0x7fdd800ffae0 2023-07-18T03:14:01.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.600+0000 7fdd9b960b80 1 Processor -- start 2023-07-18T03:14:01.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.600+0000 7fdd76ffd700 1 --2- 172.21.15.101:0/2175028360 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fdd6c0a1340 0x7fdd6c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:01.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.600+0000 7fdd9b960b80 1 -- start start 2023-07-18T03:14:01.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.600+0000 7fdd76ffd700 1 -- 172.21.15.101:0/2175028360 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fdd7c014070 con 0x7fdd800ffae0 2023-07-18T03:14:01.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.601+0000 7fdd9b960b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635d4488ae0 0x5635d4488eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:01.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.601+0000 7fdd9b960b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635d44893f0 0x5635d4493da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:01.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.601+0000 7fdd9b960b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635d44942e0 0x5635d44966c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:01.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.601+0000 7fdd9b960b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635d4386cb0 con 0x5635d44942e0 2023-07-18T03:14:01.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.601+0000 7fdd9b960b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635d43d9a40 con 0x5635d44893f0 2023-07-18T03:14:01.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.601+0000 7fdd9b960b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635d43d2ed0 con 0x5635d4488ae0 2023-07-18T03:14:01.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.601+0000 7fdd8659c700 1 --2- 172.21.15.101:0/2175028360 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fdd6c0a1340 0x7fdd6c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:01.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.601+0000 7fdd8559a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635d44893f0 0x5635d4493da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:01.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.601+0000 7fdd85d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635d4488ae0 0x5635d4488eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:01.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.601+0000 7fdd8559a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635d44893f0 0x5635d4493da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:55096/0 (socket says 172.21.15.101:55096) 2023-07-18T03:14:01.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.601+0000 7fdd8559a700 1 -- 172.21.15.101:0/541504872 learned_addr learned my addr 172.21.15.101:0/541504872 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:01.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.602+0000 7fdd8559a700 1 -- 172.21.15.101:0/541504872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635d4488ae0 msgr2=0x5635d4488eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:01.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.602+0000 7fdd8559a700 1 --2- 172.21.15.101:0/541504872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635d4488ae0 0x5635d4488eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.602+0000 7fdd8559a700 1 -- 172.21.15.101:0/541504872 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635d44942e0 msgr2=0x5635d44966c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:01.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.602+0000 7fdd8559a700 1 --2- 172.21.15.101:0/541504872 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635d44942e0 0x5635d44966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.602+0000 7fdd8559a700 1 -- 172.21.15.101:0/541504872 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635d4200de0 con 0x5635d44893f0 2023-07-18T03:14:01.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.602+0000 7fdd85d9b700 1 --2- 172.21.15.101:0/541504872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635d4488ae0 0x5635d4488eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:14:01.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.602+0000 7fdd8559a700 1 --2- 172.21.15.101:0/541504872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635d44893f0 0x5635d4493da0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdd7000bd60 tx=0x7fdd7000f640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e30882b4fb5c2256 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:01.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.602+0000 7fdd8659c700 1 --2- 172.21.15.101:0/2175028360 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fdd6c0a1340 0x7fdd6c0a37f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdd780062a0 tx=0x7fdd7800bb10 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:01.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.602+0000 7fdd6bfff700 1 -- 172.21.15.101:0/541504872 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7000fda0 con 0x5635d44893f0 2023-07-18T03:14:01.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.602+0000 7fdd6bfff700 1 -- 172.21.15.101:0/541504872 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd70011900 con 0x5635d44893f0 2023-07-18T03:14:01.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.602+0000 7fdd6bfff700 1 -- 172.21.15.101:0/541504872 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd700106a0 con 0x5635d44893f0 2023-07-18T03:14:01.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.603+0000 7fdd9b960b80 1 -- 172.21.15.101:0/541504872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635d44893f0 msgr2=0x5635d4493da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:01.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.603+0000 7fdd9b960b80 1 --2- 172.21.15.101:0/541504872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635d44893f0 0x5635d4493da0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdd7000bd60 tx=0x7fdd7000f640 comp rx=0 tx=0).stop 2023-07-18T03:14:01.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.603+0000 7fdd9b960b80 1 -- 172.21.15.101:0/541504872 shutdown_connections 2023-07-18T03:14:01.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.603+0000 7fdd9b960b80 1 --2- 172.21.15.101:0/541504872 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635d44942e0 0x5635d44966c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.603+0000 7fdd9b960b80 1 --2- 172.21.15.101:0/541504872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635d4488ae0 0x5635d4488eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.603+0000 7fdd9b960b80 1 --2- 172.21.15.101:0/541504872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635d44893f0 0x5635d4493da0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.603+0000 7fdd9b960b80 1 -- 172.21.15.101:0/541504872 >> 172.21.15.101:0/541504872 conn(0x5635d4476da0 msgr2=0x5635d447c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:01.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.603+0000 7fdd9b960b80 1 -- 172.21.15.101:0/541504872 shutdown_connections 2023-07-18T03:14:01.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.603+0000 7fdd9b960b80 1 -- 172.21.15.101:0/541504872 wait complete. 2023-07-18T03:14:01.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.604+0000 7fdd9b960b80 1 -- start start 2023-07-18T03:14:01.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.604+0000 7fdd9b960b80 1 Processor -- start 2023-07-18T03:14:01.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.605+0000 7fdd9b960b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635d4488ae0 0x5635d4493cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:01.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.605+0000 7fdd9b960b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635d44893f0 0x5635d4490fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:01.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.605+0000 7fdd9b960b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635d44942e0 0x5635d4491680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:01.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.605+0000 7fdd9b960b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635d4372ac0 con 0x5635d44942e0 2023-07-18T03:14:01.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.605+0000 7fdd9b960b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635d43727c0 con 0x5635d4488ae0 2023-07-18T03:14:01.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.605+0000 7fdd9b960b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635d4256920 con 0x5635d44893f0 2023-07-18T03:14:01.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.605+0000 7fdd9b960b80 10 client.0 _ll_register_callbacks cb 0x5635d4370760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:14:01.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.605+0000 7fdd8659c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635d44942e0 0x5635d4491680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:01.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.606+0000 7fdd8659c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635d44942e0 0x5635d4491680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:46588/0 (socket says 172.21.15.101:46588) 2023-07-18T03:14:01.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.606+0000 7fdd8659c700 1 -- 172.21.15.101:0/1501661700 learned_addr learned my addr 172.21.15.101:0/1501661700 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:01.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.606+0000 7fdd85d9b700 1 --2- 172.21.15.101:0/1501661700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635d4488ae0 0x5635d4493cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:01.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.606+0000 7fdd8559a700 1 --2- 172.21.15.101:0/1501661700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635d44893f0 0x5635d4490fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:01.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.606+0000 7fdd85d9b700 1 -- 172.21.15.101:0/1501661700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635d44893f0 msgr2=0x5635d4490fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:01.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.606+0000 7fdd85d9b700 1 --2- 172.21.15.101:0/1501661700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635d44893f0 0x5635d4490fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.606+0000 7fdd85d9b700 1 -- 172.21.15.101:0/1501661700 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635d44942e0 msgr2=0x5635d4491680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:01.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.606+0000 7fdd85d9b700 1 --2- 172.21.15.101:0/1501661700 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635d44942e0 0x5635d4491680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.606+0000 7fdd85d9b700 1 -- 172.21.15.101:0/1501661700 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd801a08d0 con 0x5635d4488ae0 2023-07-18T03:14:01.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.606+0000 7fdd8659c700 1 --2- 172.21.15.101:0/1501661700 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635d44942e0 0x5635d4491680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:14:01.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.606+0000 7fdd85d9b700 1 --2- 172.21.15.101:0/1501661700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635d4488ae0 0x5635d4493cf0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c0ca720 tx=0x7fdd7c09e1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b25f717166a6a377 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:01.604 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.606+0000 7fdd697fa700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:14:01.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.607+0000 7fdd697fa700 1 -- 172.21.15.101:0/1501661700 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c0a24f0 con 0x5635d4488ae0 2023-07-18T03:14:01.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.607+0000 7fdd697fa700 1 -- 172.21.15.101:0/1501661700 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fdd7c0a2060 con 0x5635d4488ae0 2023-07-18T03:14:01.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.607+0000 7fdd9b960b80 1 -- 172.21.15.101:0/1501661700 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5635d432c4f0 con 0x5635d4488ae0 2023-07-18T03:14:01.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.607+0000 7fdd697fa700 1 -- 172.21.15.101:0/1501661700 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd7c0a2220 con 0x5635d4488ae0 2023-07-18T03:14:01.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.607+0000 7fdd9b960b80 10 client.26858 fetch_fsmap learned FSMap version 29 2023-07-18T03:14:01.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.607+0000 7fdd9b960b80 1 -- 172.21.15.101:0/1501661700 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fdd7000b9c0 con 0x5635d4488ae0 2023-07-18T03:14:01.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.607+0000 7fdd697fa700 1 -- 172.21.15.101:0/1501661700 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5635d432c4f0 con 0x5635d4488ae0 2023-07-18T03:14:01.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.608+0000 7fdd697fa700 1 -- 172.21.15.101:0/1501661700 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fdd7c09ae80 con 0x5635d4488ae0 2023-07-18T03:14:01.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.608+0000 7fdd9b960b80 10 client.26858 fetch_fsmap finished waiting for FSMap version 29 2023-07-18T03:14:01.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.608+0000 7fdd9b960b80 10 client.26858 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:14:01.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.608+0000 7fdd9b960b80 20 client.26858 populate_metadata read hostname 'smithi101' 2023-07-18T03:14:01.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.608+0000 7fdd9b960b80 1 --2- 172.21.15.101:0/1501661700 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5635d44a3ba0 0x5635d44a5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:01.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.608+0000 7fdd9b960b80 4 client.26858 mds_command: new command op to 24541 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:14:01.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.608+0000 7fdd9b960b80 1 -- 172.21.15.101:0/1501661700 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5635d43cbeb0 con 0x5635d44a3ba0 2023-07-18T03:14:01.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.608+0000 7fdd8559a700 1 --2- 172.21.15.101:0/1501661700 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5635d44a3ba0 0x5635d44a5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:01.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.609+0000 7fdd697fa700 10 client.26858 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:14:01.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.609+0000 7fdd8559a700 1 --2- 172.21.15.101:0/1501661700 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5635d44a3ba0 0x5635d44a5fb0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=70c910287d652124 server_cookie=2c91ce86fed5790a in_seq=0 out_seq=0 2023-07-18T03:14:01.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.610+0000 7fdd697fa700 1 -- 172.21.15.101:0/1501661700 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5635d43cbeb0 con 0x5635d44a3ba0 2023-07-18T03:14:01.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.610+0000 7fdd697fa700 10 client.26858 handle_command_reply: tid=0 2023-07-18T03:14:01.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.610+0000 7fdd9b960b80 1 client.26858 shutdown 2023-07-18T03:14:01.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.610+0000 7fdd9b960b80 1 -- 172.21.15.101:0/1501661700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635d4488ae0 msgr2=0x5635d4493cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:01.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.610+0000 7fdd9b960b80 1 --2- 172.21.15.101:0/1501661700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635d4488ae0 0x5635d4493cf0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c0ca720 tx=0x7fdd7c09e1c0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.610+0000 7fdd9b960b80 1 -- 172.21.15.101:0/1501661700 shutdown_connections 2023-07-18T03:14:01.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.610+0000 7fdd9b960b80 1 --2- 172.21.15.101:0/1501661700 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5635d44a3ba0 0x5635d44a5fb0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.610+0000 7fdd9b960b80 1 --2- 172.21.15.101:0/1501661700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635d4488ae0 0x5635d4493cf0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.610+0000 7fdd9b960b80 1 --2- 172.21.15.101:0/1501661700 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635d44942e0 0x5635d4491680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.610+0000 7fdd9b960b80 1 --2- 172.21.15.101:0/1501661700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635d44893f0 0x5635d4490fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.610+0000 7fdd9b960b80 1 -- 172.21.15.101:0/1501661700 >> 172.21.15.101:0/1501661700 conn(0x5635d4476da0 msgr2=0x5635d447c230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:01.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.610+0000 7fdd697fa700 0 client.26858 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:14:01.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.610+0000 7fdd9b960b80 1 -- 172.21.15.101:0/1501661700 shutdown_connections 2023-07-18T03:14:01.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.611+0000 7fdd9b960b80 1 -- 172.21.15.101:0/1501661700 wait complete. 2023-07-18T03:14:01.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.611+0000 7fdd9b960b80 20 client.26858 trim_cache size 0 max 16384 2023-07-18T03:14:01.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.627+0000 7fdd9b960b80 1 Processor -- start 2023-07-18T03:14:01.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.627+0000 7fdd9b960b80 1 -- start start 2023-07-18T03:14:01.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.627+0000 7fdd9b960b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635d4488ae0 0x5635d4493890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:01.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.627+0000 7fdd9b960b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635d44893f0 0x5635d44907a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:01.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.627+0000 7fdd9b960b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635d44942e0 0x5635d4490ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:01.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.627+0000 7fdd9b960b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635d44769a0 con 0x5635d4488ae0 2023-07-18T03:14:01.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.627+0000 7fdd9b960b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635d447d4d0 con 0x5635d44893f0 2023-07-18T03:14:01.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.627+0000 7fdd9b960b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635d447b490 con 0x5635d44942e0 2023-07-18T03:14:01.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.628+0000 7fdd8559a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635d44893f0 0x5635d44907a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:01.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.628+0000 7fdd8559a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635d44893f0 0x5635d44907a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:55124/0 (socket says 172.21.15.101:55124) 2023-07-18T03:14:01.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.628+0000 7fdd8559a700 1 -- 172.21.15.101:0/234052652 learned_addr learned my addr 172.21.15.101:0/234052652 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:01.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.628+0000 7fdd8659c700 1 --2- 172.21.15.101:0/234052652 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635d44942e0 0x5635d4490ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:01.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.628+0000 7fdd85d9b700 1 --2- 172.21.15.101:0/234052652 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635d4488ae0 0x5635d4493890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:01.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.628+0000 7fdd8559a700 1 -- 172.21.15.101:0/234052652 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635d44942e0 msgr2=0x5635d4490ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:01.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.628+0000 7fdd8559a700 1 --2- 172.21.15.101:0/234052652 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635d44942e0 0x5635d4490ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.628+0000 7fdd8559a700 1 -- 172.21.15.101:0/234052652 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635d4488ae0 msgr2=0x5635d4493890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:01.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.628+0000 7fdd8559a700 1 --2- 172.21.15.101:0/234052652 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635d4488ae0 0x5635d4493890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.628+0000 7fdd8559a700 1 -- 172.21.15.101:0/234052652 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635d41e9c60 con 0x5635d44893f0 2023-07-18T03:14:01.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.628+0000 7fdd85d9b700 1 --2- 172.21.15.101:0/234052652 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635d4488ae0 0x5635d4493890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:14:01.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.628+0000 7fdd8559a700 1 --2- 172.21.15.101:0/234052652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635d44893f0 0x5635d44907a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fdd7000bcb0 tx=0x7fdd7000b300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e17750ad3ad28c2 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:01.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.629+0000 7fdd74ff9700 1 -- 172.21.15.101:0/234052652 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7001f040 con 0x5635d44893f0 2023-07-18T03:14:01.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.629+0000 7fdd74ff9700 1 -- 172.21.15.101:0/234052652 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd7000b600 con 0x5635d44893f0 2023-07-18T03:14:01.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.629+0000 7fdd74ff9700 1 -- 172.21.15.101:0/234052652 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7002a170 con 0x5635d44893f0 2023-07-18T03:14:01.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.629+0000 7fdd9b960b80 1 -- 172.21.15.101:0/234052652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635d44893f0 msgr2=0x5635d44907a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:01.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.629+0000 7fdd9b960b80 1 --2- 172.21.15.101:0/234052652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635d44893f0 0x5635d44907a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fdd7000bcb0 tx=0x7fdd7000b300 comp rx=0 tx=0).stop 2023-07-18T03:14:01.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.629+0000 7fdd9b960b80 1 -- 172.21.15.101:0/234052652 shutdown_connections 2023-07-18T03:14:01.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.629+0000 7fdd9b960b80 1 --2- 172.21.15.101:0/234052652 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635d44942e0 0x5635d4490ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.629+0000 7fdd9b960b80 1 --2- 172.21.15.101:0/234052652 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635d4488ae0 0x5635d4493890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.629+0000 7fdd9b960b80 1 --2- 172.21.15.101:0/234052652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635d44893f0 0x5635d44907a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.629+0000 7fdd9b960b80 1 -- 172.21.15.101:0/234052652 >> 172.21.15.101:0/234052652 conn(0x5635d4476da0 msgr2=0x5635d448c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:01.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.630+0000 7fdd9b960b80 1 -- 172.21.15.101:0/234052652 shutdown_connections 2023-07-18T03:14:01.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.630+0000 7fdd9b960b80 1 -- 172.21.15.101:0/234052652 wait complete. 2023-07-18T03:14:01.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.630+0000 7fdd9b960b80 1 -- start start 2023-07-18T03:14:01.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.630+0000 7fdd9b960b80 1 Processor -- start 2023-07-18T03:14:01.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.631+0000 7fdd9b960b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635d4488ae0 0x5635d4498b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:01.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.631+0000 7fdd9b960b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635d44893f0 0x5635d4499060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:01.629 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.631+0000 7fdd9b960b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635d44942e0 0x5635d44f6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:01.629 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.631+0000 7fdd9b960b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635d44f6fa0 con 0x5635d4488ae0 2023-07-18T03:14:01.629 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.631+0000 7fdd9b960b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635d44f70e0 con 0x5635d44893f0 2023-07-18T03:14:01.629 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.631+0000 7fdd9b960b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635d44f7250 con 0x5635d44942e0 2023-07-18T03:14:01.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.631+0000 7fdd9b960b80 10 client.0 _ll_register_callbacks cb 0x7fdd7c09a0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:14:01.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.631+0000 7fdd8559a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635d44893f0 0x5635d4499060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:01.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.631+0000 7fdd8559a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635d44893f0 0x5635d4499060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:55134/0 (socket says 172.21.15.101:55134) 2023-07-18T03:14:01.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.631+0000 7fdd8559a700 1 -- 172.21.15.101:0/3757016957 learned_addr learned my addr 172.21.15.101:0/3757016957 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:01.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.631+0000 7fdd85d9b700 1 --2- 172.21.15.101:0/3757016957 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635d4488ae0 0x5635d4498b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:01.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.631+0000 7fdd8659c700 1 --2- 172.21.15.101:0/3757016957 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635d44942e0 0x5635d44f6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:01.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.631+0000 7fdd8559a700 1 -- 172.21.15.101:0/3757016957 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635d44942e0 msgr2=0x5635d44f6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:01.631 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.631+0000 7fdd8559a700 1 --2- 172.21.15.101:0/3757016957 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635d44942e0 0x5635d44f6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.631 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.631+0000 7fdd8559a700 1 -- 172.21.15.101:0/3757016957 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635d4488ae0 msgr2=0x5635d4498b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:01.631 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.631+0000 7fdd8559a700 1 --2- 172.21.15.101:0/3757016957 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635d4488ae0 0x5635d4498b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.631 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.631+0000 7fdd8559a700 1 -- 172.21.15.101:0/3757016957 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5635d4200de0 con 0x5635d44893f0 2023-07-18T03:14:01.631 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.632+0000 7fdd8559a700 1 --2- 172.21.15.101:0/3757016957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635d44893f0 0x5635d4499060 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fdd70000c00 tx=0x7fdd70005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=945f42d7a2bffc87 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:01.631 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.632+0000 7fdd697fa700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:14:01.631 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.632+0000 7fdd697fa700 1 -- 172.21.15.101:0/3757016957 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7001f040 con 0x5635d44893f0 2023-07-18T03:14:01.631 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.632+0000 7fdd9b960b80 1 -- 172.21.15.101:0/3757016957 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdd70011900 con 0x5635d44893f0 2023-07-18T03:14:01.632 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.632+0000 7fdd697fa700 1 -- 172.21.15.101:0/3757016957 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fdd70011440 con 0x5635d44893f0 2023-07-18T03:14:01.632 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.632+0000 7fdd697fa700 1 -- 172.21.15.101:0/3757016957 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd70011640 con 0x5635d44893f0 2023-07-18T03:14:01.632 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.632+0000 7fdd9b960b80 10 client.26864 fetch_fsmap learned FSMap version 29 2023-07-18T03:14:01.632 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.632+0000 7fdd9b960b80 1 -- 172.21.15.101:0/3757016957 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5635d4227250 con 0x5635d44893f0 2023-07-18T03:14:01.632 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.633+0000 7fdd697fa700 1 -- 172.21.15.101:0/3757016957 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd70011900 con 0x5635d44893f0 2023-07-18T03:14:01.632 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.633+0000 7fdd697fa700 1 -- 172.21.15.101:0/3757016957 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fdd700040d0 con 0x5635d44893f0 2023-07-18T03:14:01.632 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.633+0000 7fdd9b960b80 10 client.26864 fetch_fsmap finished waiting for FSMap version 29 2023-07-18T03:14:01.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.633+0000 7fdd9b960b80 10 client.26864 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:14:01.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.633+0000 7fdd9b960b80 20 client.26864 populate_metadata read hostname 'smithi101' 2023-07-18T03:14:01.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.633+0000 7fdd9b960b80 1 --2- 172.21.15.101:0/3757016957 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5635d44f8460 0x5635d44fa840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:01.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.633+0000 7fdd9b960b80 4 client.26864 mds_command: new command op to 24541 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:14:01.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.633+0000 7fdd9b960b80 1 -- 172.21.15.101:0/3757016957 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5635d438e050 con 0x5635d44f8460 2023-07-18T03:14:01.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.633+0000 7fdd85d9b700 1 --2- 172.21.15.101:0/3757016957 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5635d44f8460 0x5635d44fa840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:01.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.634+0000 7fdd85d9b700 1 --2- 172.21.15.101:0/3757016957 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5635d44f8460 0x5635d44fa840 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8afe20f43854bca5 server_cookie=c340c0cc9363b962 in_seq=0 out_seq=0 2023-07-18T03:14:01.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.634+0000 7fdd697fa700 10 client.26864 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:14:01.634 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.634+0000 7fdd697fa700 1 -- 172.21.15.101:0/3757016957 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5635d438e050 con 0x5635d44f8460 2023-07-18T03:14:01.634 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.634+0000 7fdd697fa700 10 client.26864 handle_command_reply: tid=0 2023-07-18T03:14:01.634 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.634+0000 7fdd9b960b80 1 client.26864 shutdown 2023-07-18T03:14:01.634 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.634+0000 7fdd9b960b80 1 -- 172.21.15.101:0/3757016957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635d44893f0 msgr2=0x5635d4499060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:01.634 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.634+0000 7fdd9b960b80 1 --2- 172.21.15.101:0/3757016957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635d44893f0 0x5635d4499060 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fdd70000c00 tx=0x7fdd70005880 comp rx=0 tx=0).stop 2023-07-18T03:14:01.634 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.635+0000 7fdd9b960b80 1 -- 172.21.15.101:0/3757016957 shutdown_connections 2023-07-18T03:14:01.635 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.635+0000 7fdd9b960b80 1 --2- 172.21.15.101:0/3757016957 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5635d44f8460 0x5635d44fa840 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.635 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.635+0000 7fdd9b960b80 1 --2- 172.21.15.101:0/3757016957 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635d44942e0 0x5635d44f6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.635 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.635+0000 7fdd697fa700 0 client.26864 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:14:01.635 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.635+0000 7fdd9b960b80 1 --2- 172.21.15.101:0/3757016957 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5635d4488ae0 0x5635d4498b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.635 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.635+0000 7fdd9b960b80 1 --2- 172.21.15.101:0/3757016957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5635d44893f0 0x5635d4499060 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.635 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.635+0000 7fdd9b960b80 1 -- 172.21.15.101:0/3757016957 >> 172.21.15.101:0/3757016957 conn(0x5635d4476da0 msgr2=0x5635d447b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:01.635 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.635+0000 7fdd9b960b80 1 -- 172.21.15.101:0/3757016957 shutdown_connections 2023-07-18T03:14:01.636 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.635+0000 7fdd9b960b80 1 -- 172.21.15.101:0/3757016957 wait complete. 2023-07-18T03:14:01.636 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.635+0000 7fdd9b960b80 20 client.26864 trim_cache size 0 max 16384 2023-07-18T03:14:01.636 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:14:01.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.636+0000 7fdd87fff700 1 -- 172.21.15.101:0/2175028360 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fdd6c0a1340 msgr2=0x7fdd6c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:01.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.636+0000 7fdd87fff700 1 --2- 172.21.15.101:0/2175028360 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fdd6c0a1340 0x7fdd6c0a37f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdd780062a0 tx=0x7fdd7800bb10 comp rx=0 tx=0).stop 2023-07-18T03:14:01.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.636+0000 7fdd87fff700 1 -- 172.21.15.101:0/2175028360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd800ffae0 msgr2=0x7fdd801998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:01.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.636+0000 7fdd87fff700 1 --2- 172.21.15.101:0/2175028360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd800ffae0 0x7fdd801998e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c00e9a0 tx=0x7fdd7c00ed60 comp rx=0 tx=0).stop 2023-07-18T03:14:01.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.636+0000 7fdd87fff700 1 -- 172.21.15.101:0/2175028360 shutdown_connections 2023-07-18T03:14:01.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.636+0000 7fdd87fff700 1 --2- 172.21.15.101:0/2175028360 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fdd6c0a1340 0x7fdd6c0a37f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.636+0000 7fdd87fff700 1 --2- 172.21.15.101:0/2175028360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd800ffae0 0x7fdd801998e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.638 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.636+0000 7fdd87fff700 1 --2- 172.21.15.101:0/2175028360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd80105af0 0x7fdd80193950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.638 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.636+0000 7fdd87fff700 1 --2- 172.21.15.101:0/2175028360 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdd80100490 0x7fdd80199e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.638 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.636+0000 7fdd87fff700 1 -- 172.21.15.101:0/2175028360 >> 172.21.15.101:0/2175028360 conn(0x7fdd800fb5e0 msgr2=0x7fdd801035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:01.638 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.636+0000 7fdd87fff700 1 -- 172.21.15.101:0/2175028360 shutdown_connections 2023-07-18T03:14:01.638 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.636+0000 7fdd87fff700 1 -- 172.21.15.101:0/2175028360 wait complete. 2023-07-18T03:14:01.650 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-18T03:14:01.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.893+0000 7f13a69ff700 1 -- 172.21.15.101:0/1434703061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13a0100490 msgr2=0x7f13a010c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:01.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.893+0000 7f13a69ff700 1 --2- 172.21.15.101:0/1434703061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13a0100490 0x7f13a010c920 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f139800b0a0 tx=0x7f139800b3b0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.893+0000 7f13a69ff700 1 -- 172.21.15.101:0/1434703061 shutdown_connections 2023-07-18T03:14:01.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.893+0000 7f13a69ff700 1 --2- 172.21.15.101:0/1434703061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13a0100490 0x7f13a010c920 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.893+0000 7f13a69ff700 1 --2- 172.21.15.101:0/1434703061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13a0105af0 0x7f13a0105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.893+0000 7f13a69ff700 1 --2- 172.21.15.101:0/1434703061 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f13a00ffae0 0x7f13a00fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.893+0000 7f13a69ff700 1 -- 172.21.15.101:0/1434703061 >> 172.21.15.101:0/1434703061 conn(0x7f13a00fb5e0 msgr2=0x7f13a00fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:01.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.893+0000 7f13a69ff700 1 -- 172.21.15.101:0/1434703061 shutdown_connections 2023-07-18T03:14:01.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.893+0000 7f13a69ff700 1 -- 172.21.15.101:0/1434703061 wait complete. 2023-07-18T03:14:01.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.894+0000 7f13a69ff700 1 Processor -- start 2023-07-18T03:14:01.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.895+0000 7f13a69ff700 1 -- start start 2023-07-18T03:14:01.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.895+0000 7f13a69ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f13a00ffae0 0x7f13a0197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:01.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.895+0000 7f13a69ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13a0100490 0x7f13a0198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:01.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.895+0000 7f13a69ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13a0105af0 0x7f13a019c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:01.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.895+0000 7f13a69ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a019ceb0 con 0x7f13a00ffae0 2023-07-18T03:14:01.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.895+0000 7f13a69ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a019d020 con 0x7f13a0100490 2023-07-18T03:14:01.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.895+0000 7f13a69ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a019d1e0 con 0x7f13a0105af0 2023-07-18T03:14:01.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.895+0000 7f139f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13a0100490 0x7f13a0198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:01.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.895+0000 7f139f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13a0100490 0x7f13a0198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:55144/0 (socket says 172.21.15.101:55144) 2023-07-18T03:14:01.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.895+0000 7f139f7fe700 1 -- 172.21.15.101:0/177448511 learned_addr learned my addr 172.21.15.101:0/177448511 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:01.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.895+0000 7f139ffff700 1 --2- 172.21.15.101:0/177448511 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f13a00ffae0 0x7f13a0197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:01.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.895+0000 7f13a4f9c700 1 --2- 172.21.15.101:0/177448511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13a0105af0 0x7f13a019c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:01.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.896+0000 7f139f7fe700 1 -- 172.21.15.101:0/177448511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13a0105af0 msgr2=0x7f13a019c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:01.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.896+0000 7f139f7fe700 1 --2- 172.21.15.101:0/177448511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13a0105af0 0x7f13a019c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.896+0000 7f139f7fe700 1 -- 172.21.15.101:0/177448511 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f13a00ffae0 msgr2=0x7f13a0197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:01.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.896+0000 7f139f7fe700 1 --2- 172.21.15.101:0/177448511 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f13a00ffae0 0x7f13a0197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.896+0000 7f139f7fe700 1 -- 172.21.15.101:0/177448511 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1398009d00 con 0x7f13a0100490 2023-07-18T03:14:01.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.896+0000 7f13a4f9c700 1 --2- 172.21.15.101:0/177448511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13a0105af0 0x7f13a019c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:14:01.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.896+0000 7f139f7fe700 1 --2- 172.21.15.101:0/177448511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13a0100490 0x7f13a0198410 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f139400e9a0 tx=0x7f139400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:01.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.896+0000 7f139d7fa700 1 -- 172.21.15.101:0/177448511 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1394009790 con 0x7f13a0100490 2023-07-18T03:14:01.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.896+0000 7f139d7fa700 1 -- 172.21.15.101:0/177448511 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f139400cbd0 con 0x7f13a0100490 2023-07-18T03:14:01.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.896+0000 7f139d7fa700 1 -- 172.21.15.101:0/177448511 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1394018680 con 0x7f13a0100490 2023-07-18T03:14:01.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.896+0000 7f13a69ff700 1 -- 172.21.15.101:0/177448511 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13a019d4c0 con 0x7f13a0100490 2023-07-18T03:14:01.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.897+0000 7f13a69ff700 1 -- 172.21.15.101:0/177448511 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13a0106ab0 con 0x7f13a0100490 2023-07-18T03:14:01.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.901+0000 7f139d7fa700 1 -- 172.21.15.101:0/177448511 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f139400cd40 con 0x7f13a0100490 2023-07-18T03:14:01.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.902+0000 7f13ba333b80 1 Processor -- start 2023-07-18T03:14:01.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.902+0000 7f13ba333b80 1 -- start start 2023-07-18T03:14:01.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.902+0000 7f139d7fa700 1 --2- 172.21.15.101:0/177448511 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f13880a1340 0x7f13880a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:01.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.902+0000 7f13ba333b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f2603dd2d0 0x55f2603dd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:01.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.902+0000 7f13ba333b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2603ddbe0 0x55f2603e8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:01.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.902+0000 7f13ba333b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2603e8ad0 0x55f2603eaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:01.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.902+0000 7f139d7fa700 1 -- 172.21.15.101:0/177448511 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f1394014070 con 0x7f13a0100490 2023-07-18T03:14:01.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.902+0000 7f13ba333b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2602e3cb0 con 0x55f2603e8ad0 2023-07-18T03:14:01.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.902+0000 7f13ba333b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f260336a40 con 0x55f2603ddbe0 2023-07-18T03:14:01.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.902+0000 7f13ba333b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f26032fed0 con 0x55f2603dd2d0 2023-07-18T03:14:01.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.903+0000 7f139ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f2603dd2d0 0x55f2603dd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:01.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.903+0000 7f13a4f9c700 1 --2- 172.21.15.101:0/177448511 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f13880a1340 0x7f13880a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:01.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.903+0000 7f139ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f2603dd2d0 0x55f2603dd6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:43064/0 (socket says 172.21.15.101:43064) 2023-07-18T03:14:01.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.903+0000 7f139ffff700 1 -- 172.21.15.101:0/191615284 learned_addr learned my addr 172.21.15.101:0/191615284 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:01.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.903+0000 7f139f7fe700 1 --2- 172.21.15.101:0/191615284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2603ddbe0 0x55f2603e8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:01.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.903+0000 7f13a4f9c700 1 --2- 172.21.15.101:0/191615284 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2603e8ad0 0x55f2603eaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:01.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.903+0000 7f139ffff700 1 -- 172.21.15.101:0/191615284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2603ddbe0 msgr2=0x55f2603e8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:01.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.903+0000 7f139ffff700 1 --2- 172.21.15.101:0/191615284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2603ddbe0 0x55f2603e8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.903+0000 7f139ffff700 1 -- 172.21.15.101:0/191615284 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2603e8ad0 msgr2=0x55f2603eaeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:01.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.903+0000 7f139ffff700 1 --2- 172.21.15.101:0/191615284 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2603e8ad0 0x55f2603eaeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.903+0000 7f139ffff700 1 -- 172.21.15.101:0/191615284 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f26015dde0 con 0x55f2603dd2d0 2023-07-18T03:14:01.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.904+0000 7f139ffff700 1 --2- 172.21.15.101:0/191615284 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f2603dd2d0 0x55f2603dd6a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1390004250 tx=0x7f1390009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62706217f6b05028 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:01.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.904+0000 7f13a4f9c700 1 --2- 172.21.15.101:0/177448511 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f13880a1340 0x7f13880a37f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1398009850 tx=0x7f13980097c0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:01.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.904+0000 7f13867fc700 1 -- 172.21.15.101:0/191615284 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139001b040 con 0x55f2603dd2d0 2023-07-18T03:14:01.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.904+0000 7f13867fc700 1 -- 172.21.15.101:0/191615284 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1390009da0 con 0x55f2603dd2d0 2023-07-18T03:14:01.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.904+0000 7f13867fc700 1 -- 172.21.15.101:0/191615284 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139001f980 con 0x55f2603dd2d0 2023-07-18T03:14:01.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.905+0000 7f13ba333b80 1 -- 172.21.15.101:0/191615284 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f2603dd2d0 msgr2=0x55f2603dd6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:01.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.905+0000 7f13ba333b80 1 --2- 172.21.15.101:0/191615284 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f2603dd2d0 0x55f2603dd6a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1390004250 tx=0x7f1390009260 comp rx=0 tx=0).stop 2023-07-18T03:14:01.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.905+0000 7f13ba333b80 1 -- 172.21.15.101:0/191615284 shutdown_connections 2023-07-18T03:14:01.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.905+0000 7f13ba333b80 1 --2- 172.21.15.101:0/191615284 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2603e8ad0 0x55f2603eaeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.905+0000 7f13ba333b80 1 --2- 172.21.15.101:0/191615284 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f2603dd2d0 0x55f2603dd6a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.905+0000 7f13ba333b80 1 --2- 172.21.15.101:0/191615284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2603ddbe0 0x55f2603e8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.905+0000 7f13ba333b80 1 -- 172.21.15.101:0/191615284 >> 172.21.15.101:0/191615284 conn(0x55f2603d3da0 msgr2=0x55f2603d96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:01.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.905+0000 7f13ba333b80 1 -- 172.21.15.101:0/191615284 shutdown_connections 2023-07-18T03:14:01.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.905+0000 7f13ba333b80 1 -- 172.21.15.101:0/191615284 wait complete. 2023-07-18T03:14:01.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.905+0000 7f13ba333b80 1 -- start start 2023-07-18T03:14:01.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.906+0000 7f13ba333b80 1 Processor -- start 2023-07-18T03:14:01.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.906+0000 7f13ba333b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f2603dd2d0 0x55f2603e84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:01.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.906+0000 7f13ba333b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2603ddbe0 0x55f2603e5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:01.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.906+0000 7f13ba333b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2603e8ad0 0x55f2603e5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:01.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.906+0000 7f13ba333b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2602cfac0 con 0x55f2603ddbe0 2023-07-18T03:14:01.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.906+0000 7f13ba333b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2602cf7c0 con 0x55f2603e8ad0 2023-07-18T03:14:01.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.906+0000 7f13ba333b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2601b3920 con 0x55f2603dd2d0 2023-07-18T03:14:01.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.906+0000 7f13ba333b80 10 client.0 _ll_register_callbacks cb 0x55f2602cd760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:14:01.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.906+0000 7f139f7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2603ddbe0 0x55f2603e5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:01.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.906+0000 7f13a4f9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2603e8ad0 0x55f2603e5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:01.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.906+0000 7f139ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f2603dd2d0 0x55f2603e84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:01.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.906+0000 7f139f7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2603ddbe0 0x55f2603e5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:46634/0 (socket says 172.21.15.101:46634) 2023-07-18T03:14:01.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.906+0000 7f13a4f9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2603e8ad0 0x55f2603e5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:55166/0 (socket says 172.21.15.101:55166) 2023-07-18T03:14:01.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.906+0000 7f139f7fe700 1 -- 172.21.15.101:0/1393088696 learned_addr learned my addr 172.21.15.101:0/1393088696 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:01.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.907+0000 7f139ffff700 1 -- 172.21.15.101:0/1393088696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2603e8ad0 msgr2=0x55f2603e5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:01.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.907+0000 7f139ffff700 1 --2- 172.21.15.101:0/1393088696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2603e8ad0 0x55f2603e5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.907+0000 7f139ffff700 1 -- 172.21.15.101:0/1393088696 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2603ddbe0 msgr2=0x55f2603e5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:01.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.907+0000 7f139ffff700 1 --2- 172.21.15.101:0/1393088696 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2603ddbe0 0x55f2603e5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.907+0000 7f139ffff700 1 -- 172.21.15.101:0/1393088696 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1390016e30 con 0x55f2603dd2d0 2023-07-18T03:14:01.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.907+0000 7f139ffff700 1 --2- 172.21.15.101:0/1393088696 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f2603dd2d0 0x55f2603e84f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f139001f460 tx=0x7f1390005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=638410acd8b6c5af server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:01.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.907+0000 7f137f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:14:01.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.907+0000 7f137f7fe700 1 -- 172.21.15.101:0/1393088696 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139001b030 con 0x55f2603dd2d0 2023-07-18T03:14:01.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.907+0000 7f13ba333b80 1 -- 172.21.15.101:0/1393088696 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f2602894f0 con 0x55f2603dd2d0 2023-07-18T03:14:01.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.907+0000 7f137f7fe700 1 -- 172.21.15.101:0/1393088696 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f1390005670 con 0x55f2603dd2d0 2023-07-18T03:14:01.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.908+0000 7f137f7fe700 1 -- 172.21.15.101:0/1393088696 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1390003680 con 0x55f2603dd2d0 2023-07-18T03:14:01.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.908+0000 7f13ba333b80 10 client.26911 fetch_fsmap learned FSMap version 29 2023-07-18T03:14:01.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.908+0000 7f13ba333b80 1 -- 172.21.15.101:0/1393088696 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f260146c60 con 0x55f2603dd2d0 2023-07-18T03:14:01.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.908+0000 7f137f7fe700 1 -- 172.21.15.101:0/1393088696 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f2602894f0 con 0x55f2603dd2d0 2023-07-18T03:14:01.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.908+0000 7f137f7fe700 1 -- 172.21.15.101:0/1393088696 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f1390004c80 con 0x55f2603dd2d0 2023-07-18T03:14:01.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.908+0000 7f13ba333b80 10 client.26911 fetch_fsmap finished waiting for FSMap version 29 2023-07-18T03:14:01.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.908+0000 7f13ba333b80 10 client.26911 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:14:01.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.908+0000 7f13ba333b80 20 client.26911 populate_metadata read hostname 'smithi101' 2023-07-18T03:14:01.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.908+0000 7f13ba333b80 1 --2- 172.21.15.101:0/1393088696 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55f2603f83a0 0x55f2603fa7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:01.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.908+0000 7f13ba333b80 4 client.26911 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:14:01.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.908+0000 7f13ba333b80 1 -- 172.21.15.101:0/1393088696 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f260328eb0 con 0x55f2603f83a0 2023-07-18T03:14:01.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.908+0000 7f139f7fe700 1 --2- 172.21.15.101:0/1393088696 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55f2603f83a0 0x55f2603fa7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:01.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.909+0000 7f139f7fe700 1 --2- 172.21.15.101:0/1393088696 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55f2603f83a0 0x55f2603fa7b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=648e27e2cf821e5c server_cookie=74729171c98d20a5 in_seq=0 out_seq=0 2023-07-18T03:14:01.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.909+0000 7f137f7fe700 10 client.26911 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:14:01.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.910+0000 7f137f7fe700 1 -- 172.21.15.101:0/1393088696 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f260328eb0 con 0x55f2603f83a0 2023-07-18T03:14:01.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.910+0000 7f137f7fe700 10 client.26911 handle_command_reply: tid=0 2023-07-18T03:14:01.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.910+0000 7f13ba333b80 1 client.26911 shutdown 2023-07-18T03:14:01.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.910+0000 7f13ba333b80 1 -- 172.21.15.101:0/1393088696 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f2603dd2d0 msgr2=0x55f2603e84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:01.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.910+0000 7f13ba333b80 1 --2- 172.21.15.101:0/1393088696 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f2603dd2d0 0x55f2603e84f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f139001f460 tx=0x7f1390005270 comp rx=0 tx=0).stop 2023-07-18T03:14:01.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.910+0000 7f13ba333b80 1 -- 172.21.15.101:0/1393088696 shutdown_connections 2023-07-18T03:14:01.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.910+0000 7f13ba333b80 1 --2- 172.21.15.101:0/1393088696 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55f2603f83a0 0x55f2603fa7b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.910+0000 7f13ba333b80 1 --2- 172.21.15.101:0/1393088696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2603e8ad0 0x55f2603e5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.910+0000 7f13ba333b80 1 --2- 172.21.15.101:0/1393088696 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f2603dd2d0 0x55f2603e84f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.910+0000 7f13ba333b80 1 --2- 172.21.15.101:0/1393088696 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2603ddbe0 0x55f2603e5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.910+0000 7f137f7fe700 0 client.26911 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:14:01.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.910+0000 7f13ba333b80 1 -- 172.21.15.101:0/1393088696 >> 172.21.15.101:0/1393088696 conn(0x55f2603d3da0 msgr2=0x55f2603d9230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:01.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.910+0000 7f13ba333b80 1 -- 172.21.15.101:0/1393088696 shutdown_connections 2023-07-18T03:14:01.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.910+0000 7f13ba333b80 1 -- 172.21.15.101:0/1393088696 wait complete. 2023-07-18T03:14:01.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.911+0000 7f13ba333b80 20 client.26911 trim_cache size 0 max 16384 2023-07-18T03:14:01.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.925+0000 7f13ba333b80 1 Processor -- start 2023-07-18T03:14:01.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.925+0000 7f13ba333b80 1 -- start start 2023-07-18T03:14:01.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.926+0000 7f13ba333b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2603dd2d0 0x55f2603e8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:01.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.926+0000 7f13ba333b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2603ddbe0 0x55f2603e4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:01.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.926+0000 7f13ba333b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f2603e8ad0 0x55f2603e5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:01.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.926+0000 7f13ba333b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2603de540 con 0x55f2603dd2d0 2023-07-18T03:14:01.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.926+0000 7f13ba333b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2603e6d50 con 0x55f2603ddbe0 2023-07-18T03:14:01.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.926+0000 7f13ba333b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2603f8180 con 0x55f2603e8ad0 2023-07-18T03:14:01.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.926+0000 7f139f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2603ddbe0 0x55f2603e4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:01.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.926+0000 7f13a4f9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f2603e8ad0 0x55f2603e5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:01.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.926+0000 7f139f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2603ddbe0 0x55f2603e4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:55176/0 (socket says 172.21.15.101:55176) 2023-07-18T03:14:01.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.926+0000 7f139f7fe700 1 -- 172.21.15.101:0/974327855 learned_addr learned my addr 172.21.15.101:0/974327855 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:01.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.926+0000 7f13a4f9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f2603e8ad0 0x55f2603e5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:43088/0 (socket says 172.21.15.101:43088) 2023-07-18T03:14:01.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.926+0000 7f139ffff700 1 --2- 172.21.15.101:0/974327855 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2603dd2d0 0x55f2603e8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:01.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.926+0000 7f139f7fe700 1 -- 172.21.15.101:0/974327855 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f2603e8ad0 msgr2=0x55f2603e5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:01.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.926+0000 7f139f7fe700 1 --2- 172.21.15.101:0/974327855 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f2603e8ad0 0x55f2603e5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.922 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.926+0000 7f139f7fe700 1 -- 172.21.15.101:0/974327855 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2603dd2d0 msgr2=0x55f2603e8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:01.922 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.926+0000 7f139f7fe700 1 --2- 172.21.15.101:0/974327855 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2603dd2d0 0x55f2603e8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.922 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.926+0000 7f139f7fe700 1 -- 172.21.15.101:0/974327855 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f260184250 con 0x55f2603ddbe0 2023-07-18T03:14:01.922 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.927+0000 7f139f7fe700 1 --2- 172.21.15.101:0/974327855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2603ddbe0 0x55f2603e4f30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f139400c5b0 tx=0x7f139401a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae6aa825fee2cd15 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:01.924 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.927+0000 7f1386ffd700 1 -- 172.21.15.101:0/974327855 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13940a2410 con 0x55f2603ddbe0 2023-07-18T03:14:01.925 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.927+0000 7f1386ffd700 1 -- 172.21.15.101:0/974327855 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1394098360 con 0x55f2603ddbe0 2023-07-18T03:14:01.925 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.927+0000 7f1386ffd700 1 -- 172.21.15.101:0/974327855 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1394094bd0 con 0x55f2603ddbe0 2023-07-18T03:14:01.925 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.928+0000 7f13ba333b80 1 -- 172.21.15.101:0/974327855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2603ddbe0 msgr2=0x55f2603e4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:01.925 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.928+0000 7f13ba333b80 1 --2- 172.21.15.101:0/974327855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2603ddbe0 0x55f2603e4f30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f139400c5b0 tx=0x7f139401a040 comp rx=0 tx=0).stop 2023-07-18T03:14:01.925 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.928+0000 7f13ba333b80 1 -- 172.21.15.101:0/974327855 shutdown_connections 2023-07-18T03:14:01.925 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.928+0000 7f13ba333b80 1 --2- 172.21.15.101:0/974327855 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f2603e8ad0 0x55f2603e5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.928+0000 7f13ba333b80 1 --2- 172.21.15.101:0/974327855 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2603dd2d0 0x55f2603e8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.928+0000 7f13ba333b80 1 --2- 172.21.15.101:0/974327855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2603ddbe0 0x55f2603e4f30 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.928+0000 7f13ba333b80 1 -- 172.21.15.101:0/974327855 >> 172.21.15.101:0/974327855 conn(0x55f2603d3da0 msgr2=0x55f2603f3c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:01.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.928+0000 7f13ba333b80 1 -- 172.21.15.101:0/974327855 shutdown_connections 2023-07-18T03:14:01.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.928+0000 7f13ba333b80 1 -- 172.21.15.101:0/974327855 wait complete. 2023-07-18T03:14:01.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.928+0000 7f13ba333b80 1 -- start start 2023-07-18T03:14:01.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.928+0000 7f13ba333b80 1 Processor -- start 2023-07-18T03:14:01.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.929+0000 7f13ba333b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2603dd2d0 0x55f2603ef6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:01.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.929+0000 7f13ba333b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2603ddbe0 0x55f2603edb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:01.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.929+0000 7f13ba333b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f2603e8ad0 0x55f2603ee250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:01.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.929+0000 7f13ba333b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2603ee920 con 0x55f2603ddbe0 2023-07-18T03:14:01.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.929+0000 7f13ba333b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f26044c290 con 0x55f2603dd2d0 2023-07-18T03:14:01.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.929+0000 7f13ba333b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f26044c3d0 con 0x55f2603e8ad0 2023-07-18T03:14:01.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.929+0000 7f13ba333b80 10 client.0 _ll_register_callbacks cb 0x7f1394010aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:14:01.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.929+0000 7f139f7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2603ddbe0 0x55f2603edb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:01.928 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.929+0000 7f139f7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2603ddbe0 0x55f2603edb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:46660/0 (socket says 172.21.15.101:46660) 2023-07-18T03:14:01.928 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.929+0000 7f139f7fe700 1 -- 172.21.15.101:0/234735752 learned_addr learned my addr 172.21.15.101:0/234735752 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:01.928 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.929+0000 7f139ffff700 1 --2- 172.21.15.101:0/234735752 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2603dd2d0 0x55f2603ef6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:01.928 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.929+0000 7f13a4f9c700 1 --2- 172.21.15.101:0/234735752 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f2603e8ad0 0x55f2603ee250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:01.928 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.929+0000 7f139f7fe700 1 -- 172.21.15.101:0/234735752 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f2603e8ad0 msgr2=0x55f2603ee250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:01.928 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.929+0000 7f139f7fe700 1 --2- 172.21.15.101:0/234735752 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f2603e8ad0 0x55f2603ee250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.928 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.929+0000 7f139f7fe700 1 -- 172.21.15.101:0/234735752 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2603dd2d0 msgr2=0x55f2603ef6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:01.929 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.930+0000 7f139f7fe700 1 --2- 172.21.15.101:0/234735752 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2603dd2d0 0x55f2603ef6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.929 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.930+0000 7f139f7fe700 1 -- 172.21.15.101:0/234735752 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13a0106ab0 con 0x55f2603ddbe0 2023-07-18T03:14:01.929 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.930+0000 7f139f7fe700 1 --2- 172.21.15.101:0/234735752 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2603ddbe0 0x55f2603edb10 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1394009230 tx=0x7f139409e150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b3d0b729c84166 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:01.929 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.930+0000 7f137f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:14:01.929 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.930+0000 7f137f7fe700 1 -- 172.21.15.101:0/234735752 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13940a2490 con 0x55f2603ddbe0 2023-07-18T03:14:01.929 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.930+0000 7f13ba333b80 1 -- 172.21.15.101:0/234735752 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1390009da0 con 0x55f2603ddbe0 2023-07-18T03:14:01.929 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.930+0000 7f137f7fe700 1 -- 172.21.15.101:0/234735752 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f13940b82b0 con 0x55f2603ddbe0 2023-07-18T03:14:01.929 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.930+0000 7f137f7fe700 1 -- 172.21.15.101:0/234735752 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f139409a220 con 0x55f2603ddbe0 2023-07-18T03:14:01.930 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.930+0000 7f13ba333b80 10 client.16836 fetch_fsmap learned FSMap version 29 2023-07-18T03:14:01.930 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.930+0000 7f13ba333b80 1 -- 172.21.15.101:0/234735752 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f26015c2b0 con 0x55f2603ddbe0 2023-07-18T03:14:01.930 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.930+0000 7f137f7fe700 1 -- 172.21.15.101:0/234735752 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1390009da0 con 0x55f2603ddbe0 2023-07-18T03:14:01.930 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.931+0000 7f137f7fe700 1 -- 172.21.15.101:0/234735752 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f13940a2010 con 0x55f2603ddbe0 2023-07-18T03:14:01.930 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.931+0000 7f13ba333b80 10 client.16836 fetch_fsmap finished waiting for FSMap version 29 2023-07-18T03:14:01.930 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.931+0000 7f13ba333b80 10 client.16836 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:14:01.930 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.931+0000 7f13ba333b80 20 client.16836 populate_metadata read hostname 'smithi101' 2023-07-18T03:14:01.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.931+0000 7f13ba333b80 1 --2- 172.21.15.101:0/234735752 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55f26044d610 0x55f26044fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:01.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.931+0000 7f13ba333b80 4 client.16836 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:14:01.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.931+0000 7f13ba333b80 1 -- 172.21.15.101:0/234735752 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f2602eb050 con 0x55f26044d610 2023-07-18T03:14:01.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.931+0000 7f139ffff700 1 --2- 172.21.15.101:0/234735752 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55f26044d610 0x55f26044fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:01.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.931+0000 7f139ffff700 1 --2- 172.21.15.101:0/234735752 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55f26044d610 0x55f26044fa20 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ae574b7410daeb8d server_cookie=311c17e2925884a5 in_seq=0 out_seq=0 2023-07-18T03:14:01.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.931+0000 7f137f7fe700 10 client.16836 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:14:01.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.932+0000 7f137f7fe700 1 -- 172.21.15.101:0/234735752 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f2602eb050 con 0x55f26044d610 2023-07-18T03:14:01.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.932+0000 7f137f7fe700 10 client.16836 handle_command_reply: tid=0 2023-07-18T03:14:01.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.932+0000 7f13ba333b80 1 client.16836 shutdown 2023-07-18T03:14:01.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.932+0000 7f13ba333b80 1 -- 172.21.15.101:0/234735752 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2603ddbe0 msgr2=0x55f2603edb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:01.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.932+0000 7f13ba333b80 1 --2- 172.21.15.101:0/234735752 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2603ddbe0 0x55f2603edb10 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1394009230 tx=0x7f139409e150 comp rx=0 tx=0).stop 2023-07-18T03:14:01.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.932+0000 7f13ba333b80 1 -- 172.21.15.101:0/234735752 shutdown_connections 2023-07-18T03:14:01.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.932+0000 7f13ba333b80 1 --2- 172.21.15.101:0/234735752 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55f26044d610 0x55f26044fa20 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.932+0000 7f13ba333b80 1 --2- 172.21.15.101:0/234735752 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f2603dd2d0 0x55f2603ef6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.932+0000 7f13ba333b80 1 --2- 172.21.15.101:0/234735752 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f2603e8ad0 0x55f2603ee250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.932+0000 7f13ba333b80 1 --2- 172.21.15.101:0/234735752 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2603ddbe0 0x55f2603edb10 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.934 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.932+0000 7f137f7fe700 0 client.16836 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:14:01.934 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.932+0000 7f13ba333b80 1 -- 172.21.15.101:0/234735752 >> 172.21.15.101:0/234735752 conn(0x55f2603d3da0 msgr2=0x55f2603df3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:01.934 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.932+0000 7f13ba333b80 1 -- 172.21.15.101:0/234735752 shutdown_connections 2023-07-18T03:14:01.934 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.932+0000 7f13ba333b80 1 -- 172.21.15.101:0/234735752 wait complete. 2023-07-18T03:14:01.934 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.933+0000 7f13ba333b80 20 client.16836 trim_cache size 0 max 16384 2023-07-18T03:14:01.934 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:14:01.935 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.933+0000 7f13a69ff700 1 -- 172.21.15.101:0/177448511 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f13880a1340 msgr2=0x7f13880a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:01.935 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.933+0000 7f13a69ff700 1 --2- 172.21.15.101:0/177448511 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f13880a1340 0x7f13880a37f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1398009850 tx=0x7f13980097c0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.933+0000 7f13a69ff700 1 -- 172.21.15.101:0/177448511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13a0100490 msgr2=0x7f13a0198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:01.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.933+0000 7f13a69ff700 1 --2- 172.21.15.101:0/177448511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13a0100490 0x7f13a0198410 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f139400e9a0 tx=0x7f139400ecb0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.933+0000 7f13a69ff700 1 -- 172.21.15.101:0/177448511 shutdown_connections 2023-07-18T03:14:01.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.933+0000 7f13a69ff700 1 --2- 172.21.15.101:0/177448511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13a0105af0 0x7f13a019c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.933+0000 7f13a69ff700 1 --2- 172.21.15.101:0/177448511 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f13a00ffae0 0x7f13a0197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.933+0000 7f13a69ff700 1 --2- 172.21.15.101:0/177448511 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f13880a1340 0x7f13880a37f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.933+0000 7f13a69ff700 1 --2- 172.21.15.101:0/177448511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13a0100490 0x7f13a0198410 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:01.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.933+0000 7f13a69ff700 1 -- 172.21.15.101:0/177448511 >> 172.21.15.101:0/177448511 conn(0x7f13a00fb5e0 msgr2=0x7f13a010ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:01.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.933+0000 7f13a69ff700 1 -- 172.21.15.101:0/177448511 shutdown_connections 2023-07-18T03:14:01.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:01.933+0000 7f13a69ff700 1 -- 172.21.15.101:0/177448511 wait complete. 2023-07-18T03:14:01.945 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-18T03:14:02.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.169+0000 7fddeffff700 1 -- 172.21.15.101:0/3678835582 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdde810a070 msgr2=0x7fdde810c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:02.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.169+0000 7fddeffff700 1 --2- 172.21.15.101:0/3678835582 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdde810a070 0x7fdde810c4b0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fdde000b0a0 tx=0x7fdde000b3b0 comp rx=0 tx=0).stop 2023-07-18T03:14:02.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.169+0000 7fddeffff700 1 -- 172.21.15.101:0/3678835582 shutdown_connections 2023-07-18T03:14:02.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.169+0000 7fddeffff700 1 --2- 172.21.15.101:0/3678835582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdde8100ca0 0x7fdde8101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:02.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.169+0000 7fddeffff700 1 --2- 172.21.15.101:0/3678835582 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdde810a070 0x7fdde810c4b0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:02.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.169+0000 7fddeffff700 1 --2- 172.21.15.101:0/3678835582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdde81015b0 0x7fdde8109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:02.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.169+0000 7fddeffff700 1 -- 172.21.15.101:0/3678835582 >> 172.21.15.101:0/3678835582 conn(0x7fdde80fa510 msgr2=0x7fdde80fc920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:02.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.169+0000 7fddeffff700 1 -- 172.21.15.101:0/3678835582 shutdown_connections 2023-07-18T03:14:02.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.169+0000 7fddeffff700 1 -- 172.21.15.101:0/3678835582 wait complete. 2023-07-18T03:14:02.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.170+0000 7fddeffff700 1 Processor -- start 2023-07-18T03:14:02.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.170+0000 7fddeffff700 1 -- start start 2023-07-18T03:14:02.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.171+0000 7fddeffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdde8100ca0 0x7fdde819bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:02.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.171+0000 7fddeffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdde81015b0 0x7fdde819c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:02.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.171+0000 7fddeffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdde810a070 0x7fdde8195b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:02.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.171+0000 7fddeffff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde81961b0 con 0x7fdde8100ca0 2023-07-18T03:14:02.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.171+0000 7fddeffff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde8196320 con 0x7fdde810a070 2023-07-18T03:14:02.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.171+0000 7fddeffff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde81964e0 con 0x7fdde81015b0 2023-07-18T03:14:02.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.171+0000 7fddedd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdde8100ca0 0x7fdde819bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:02.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.171+0000 7fddedd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdde8100ca0 0x7fdde819bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:46676/0 (socket says 172.21.15.101:46676) 2023-07-18T03:14:02.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.171+0000 7fddedd9b700 1 -- 172.21.15.101:0/2089172208 learned_addr learned my addr 172.21.15.101:0/2089172208 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:02.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.171+0000 7fdded59a700 1 --2- 172.21.15.101:0/2089172208 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdde81015b0 0x7fdde819c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:02.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.171+0000 7fddee59c700 1 --2- 172.21.15.101:0/2089172208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdde810a070 0x7fdde8195b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:02.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.171+0000 7fddedd9b700 1 -- 172.21.15.101:0/2089172208 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdde81015b0 msgr2=0x7fdde819c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:02.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.171+0000 7fddedd9b700 1 --2- 172.21.15.101:0/2089172208 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdde81015b0 0x7fdde819c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:02.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.171+0000 7fddedd9b700 1 -- 172.21.15.101:0/2089172208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdde810a070 msgr2=0x7fdde8195b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:02.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.171+0000 7fddedd9b700 1 --2- 172.21.15.101:0/2089172208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdde810a070 0x7fdde8195b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:02.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.171+0000 7fddedd9b700 1 -- 172.21.15.101:0/2089172208 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde0009d00 con 0x7fdde8100ca0 2023-07-18T03:14:02.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.171+0000 7fddee59c700 1 --2- 172.21.15.101:0/2089172208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdde810a070 0x7fdde8195b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:14:02.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.172+0000 7fddedd9b700 1 --2- 172.21.15.101:0/2089172208 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdde8100ca0 0x7fdde819bad0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fdde400e9a0 tx=0x7fdde400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:02.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.172+0000 7fdddeffd700 1 -- 172.21.15.101:0/2089172208 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde4009790 con 0x7fdde8100ca0 2023-07-18T03:14:02.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.172+0000 7fddeffff700 1 -- 172.21.15.101:0/2089172208 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdde81967c0 con 0x7fdde8100ca0 2023-07-18T03:14:02.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.172+0000 7fdddeffd700 1 -- 172.21.15.101:0/2089172208 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdde400cbd0 con 0x7fdde8100ca0 2023-07-18T03:14:02.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.172+0000 7fdddeffd700 1 -- 172.21.15.101:0/2089172208 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde4018680 con 0x7fdde8100ca0 2023-07-18T03:14:02.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.172+0000 7fddeffff700 1 -- 172.21.15.101:0/2089172208 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdde81a2ac0 con 0x7fdde8100ca0 2023-07-18T03:14:02.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.175+0000 7fdddeffd700 1 -- 172.21.15.101:0/2089172208 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fdde400cd40 con 0x7fdde8100ca0 2023-07-18T03:14:02.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.177+0000 7fde0385cb80 1 Processor -- start 2023-07-18T03:14:02.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.177+0000 7fdddeffd700 1 --2- 172.21.15.101:0/2089172208 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fddd40a12c0 0x7fddd40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:02.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.177+0000 7fde0385cb80 1 -- start start 2023-07-18T03:14:02.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.177+0000 7fdddeffd700 1 -- 172.21.15.101:0/2089172208 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fdde4014070 con 0x7fdde8100ca0 2023-07-18T03:14:02.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.177+0000 7fddee59c700 1 --2- 172.21.15.101:0/2089172208 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fddd40a12c0 0x7fddd40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:02.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.177+0000 7fde0385cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559872d062d0 0x559872d066a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:02.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.177+0000 7fde0385cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559872d06be0 0x559872d159a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:02.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.177+0000 7fde0385cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559872d15ee0 0x559872d182c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:02.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.178+0000 7fde0385cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559872c0ccb0 con 0x559872d06be0 2023-07-18T03:14:02.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.178+0000 7fde0385cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559872c5fa40 con 0x559872d15ee0 2023-07-18T03:14:02.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.178+0000 7fde0385cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x559872c58ed0 con 0x559872d062d0 2023-07-18T03:14:02.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.178+0000 7fddedd9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559872d062d0 0x559872d066a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:02.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.178+0000 7fddedd9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559872d062d0 0x559872d066a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:43128/0 (socket says 172.21.15.101:43128) 2023-07-18T03:14:02.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.178+0000 7fddedd9b700 1 -- 172.21.15.101:0/748928624 learned_addr learned my addr 172.21.15.101:0/748928624 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:02.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.178+0000 7fdded59a700 1 --2- 172.21.15.101:0/748928624 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559872d06be0 0x559872d159a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:02.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.178+0000 7fddedd9b700 1 -- 172.21.15.101:0/748928624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559872d15ee0 msgr2=0x559872d182c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:02.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.178+0000 7fddedd9b700 1 --2- 172.21.15.101:0/748928624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559872d15ee0 0x559872d182c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:02.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.178+0000 7fddedd9b700 1 -- 172.21.15.101:0/748928624 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559872d06be0 msgr2=0x559872d159a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:02.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.178+0000 7fddedd9b700 1 --2- 172.21.15.101:0/748928624 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559872d06be0 0x559872d159a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:02.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.178+0000 7fddedd9b700 1 -- 172.21.15.101:0/748928624 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559872a86de0 con 0x559872d062d0 2023-07-18T03:14:02.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.178+0000 7fdded59a700 1 --2- 172.21.15.101:0/748928624 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559872d06be0 0x559872d159a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:14:02.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.178+0000 7fddee59c700 1 --2- 172.21.15.101:0/2089172208 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fddd40a12c0 0x7fddd40a3770 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdde00062a0 tx=0x7fdde0006230 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:02.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.178+0000 7fddedd9b700 1 --2- 172.21.15.101:0/748928624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559872d062d0 0x559872d066a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fdde4096ab0 tx=0x7fdde4099a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21faba0f9fb05d22 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:02.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.178+0000 7fddd3fff700 1 -- 172.21.15.101:0/748928624 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde40cd160 con 0x559872d062d0 2023-07-18T03:14:02.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.178+0000 7fddd3fff700 1 -- 172.21.15.101:0/748928624 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdde409e070 con 0x559872d062d0 2023-07-18T03:14:02.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.179+0000 7fddd3fff700 1 -- 172.21.15.101:0/748928624 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde40a2260 con 0x559872d062d0 2023-07-18T03:14:02.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.179+0000 7fde0385cb80 1 -- 172.21.15.101:0/748928624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559872d062d0 msgr2=0x559872d066a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:02.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.179+0000 7fde0385cb80 1 --2- 172.21.15.101:0/748928624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559872d062d0 0x559872d066a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fdde4096ab0 tx=0x7fdde4099a20 comp rx=0 tx=0).stop 2023-07-18T03:14:02.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.179+0000 7fde0385cb80 1 -- 172.21.15.101:0/748928624 shutdown_connections 2023-07-18T03:14:02.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.179+0000 7fde0385cb80 1 --2- 172.21.15.101:0/748928624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559872d15ee0 0x559872d182c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:02.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.179+0000 7fde0385cb80 1 --2- 172.21.15.101:0/748928624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559872d062d0 0x559872d066a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:02.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.179+0000 7fde0385cb80 1 --2- 172.21.15.101:0/748928624 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559872d06be0 0x559872d159a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:02.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.179+0000 7fde0385cb80 1 -- 172.21.15.101:0/748928624 >> 172.21.15.101:0/748928624 conn(0x559872cfcda0 msgr2=0x559872d026a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:02.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.179+0000 7fde0385cb80 1 -- 172.21.15.101:0/748928624 shutdown_connections 2023-07-18T03:14:02.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.179+0000 7fde0385cb80 1 -- 172.21.15.101:0/748928624 wait complete. 2023-07-18T03:14:02.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.180+0000 7fde0385cb80 1 -- start start 2023-07-18T03:14:02.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.180+0000 7fde0385cb80 1 Processor -- start 2023-07-18T03:14:02.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.181+0000 7fde0385cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559872d062d0 0x559872d12c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:02.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.181+0000 7fde0385cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559872d06be0 0x559872d131b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:02.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.181+0000 7fde0385cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559872d15ee0 0x559872d136f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:02.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.181+0000 7fde0385cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559872bf8ac0 con 0x559872d06be0 2023-07-18T03:14:02.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.181+0000 7fde0385cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559872bf87c0 con 0x559872d15ee0 2023-07-18T03:14:02.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.181+0000 7fde0385cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x559872adc920 con 0x559872d062d0 2023-07-18T03:14:02.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.181+0000 7fde0385cb80 10 client.0 _ll_register_callbacks cb 0x559872bf6760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:14:02.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.181+0000 7fdded59a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559872d06be0 0x559872d131b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:02.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.181+0000 7fddedd9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559872d062d0 0x559872d12c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:02.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.181+0000 7fdded59a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559872d06be0 0x559872d131b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:46684/0 (socket says 172.21.15.101:46684) 2023-07-18T03:14:02.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.181+0000 7fdded59a700 1 -- 172.21.15.101:0/1376705745 learned_addr learned my addr 172.21.15.101:0/1376705745 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:02.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.181+0000 7fddedd9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559872d062d0 0x559872d12c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:43132/0 (socket says 172.21.15.101:43132) 2023-07-18T03:14:02.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.181+0000 7fddee59c700 1 --2- 172.21.15.101:0/1376705745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559872d15ee0 0x559872d136f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:02.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.182+0000 7fdded59a700 1 -- 172.21.15.101:0/1376705745 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559872d062d0 msgr2=0x559872d12c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:02.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.182+0000 7fdded59a700 1 --2- 172.21.15.101:0/1376705745 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559872d062d0 0x559872d12c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:02.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.182+0000 7fdded59a700 1 -- 172.21.15.101:0/1376705745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559872d15ee0 msgr2=0x559872d136f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:02.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.182+0000 7fdded59a700 1 --2- 172.21.15.101:0/1376705745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559872d15ee0 0x559872d136f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:02.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.182+0000 7fdded59a700 1 -- 172.21.15.101:0/1376705745 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdde81a2ac0 con 0x559872d06be0 2023-07-18T03:14:02.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.182+0000 7fddedd9b700 1 --2- 172.21.15.101:0/1376705745 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559872d062d0 0x559872d12c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:14:02.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.182+0000 7fddee59c700 1 --2- 172.21.15.101:0/1376705745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559872d15ee0 0x559872d136f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:14:02.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.182+0000 7fdded59a700 1 --2- 172.21.15.101:0/1376705745 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559872d06be0 0x559872d131b0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fddd8011740 tx=0x7fddd8011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76e8d14a02671f9b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:02.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.182+0000 7fddd17fa700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:14:02.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.182+0000 7fddd17fa700 1 -- 172.21.15.101:0/1376705745 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd8016070 con 0x559872d06be0 2023-07-18T03:14:02.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.182+0000 7fde0385cb80 1 -- 172.21.15.101:0/1376705745 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559872bb24f0 con 0x559872d06be0 2023-07-18T03:14:02.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.182+0000 7fddd17fa700 1 -- 172.21.15.101:0/1376705745 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fddd8004320 con 0x559872d06be0 2023-07-18T03:14:02.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.182+0000 7fddd17fa700 1 -- 172.21.15.101:0/1376705745 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fddd80044e0 con 0x559872d06be0 2023-07-18T03:14:02.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.182+0000 7fde0385cb80 10 client.16848 fetch_fsmap learned FSMap version 29 2023-07-18T03:14:02.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.182+0000 7fde0385cb80 1 -- 172.21.15.101:0/1376705745 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x559872a6fc60 con 0x559872d06be0 2023-07-18T03:14:02.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.183+0000 7fddd17fa700 1 -- 172.21.15.101:0/1376705745 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559872bb24f0 con 0x559872d06be0 2023-07-18T03:14:02.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.183+0000 7fddd17fa700 1 -- 172.21.15.101:0/1376705745 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fddd800f8b0 con 0x559872d06be0 2023-07-18T03:14:02.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.183+0000 7fde0385cb80 10 client.16848 fetch_fsmap finished waiting for FSMap version 29 2023-07-18T03:14:02.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.183+0000 7fde0385cb80 10 client.16848 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:14:02.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.183+0000 7fde0385cb80 20 client.16848 populate_metadata read hostname 'smithi101' 2023-07-18T03:14:02.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.183+0000 7fde0385cb80 1 --2- 172.21.15.101:0/1376705745 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x559872d259f0 0x559872d27e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:02.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.183+0000 7fde0385cb80 4 client.16848 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:14:02.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.183+0000 7fde0385cb80 1 -- 172.21.15.101:0/1376705745 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559872c51eb0 con 0x559872d259f0 2023-07-18T03:14:02.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.183+0000 7fddedd9b700 1 --2- 172.21.15.101:0/1376705745 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x559872d259f0 0x559872d27e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:02.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.184+0000 7fddd17fa700 10 client.16848 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:14:02.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.184+0000 7fddedd9b700 1 --2- 172.21.15.101:0/1376705745 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x559872d259f0 0x559872d27e00 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4027146a0abf6d0b server_cookie=bbe0b78f4ce64be3 in_seq=0 out_seq=0 2023-07-18T03:14:02.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.184+0000 7fddd17fa700 1 -- 172.21.15.101:0/1376705745 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559872c51eb0 con 0x559872d259f0 2023-07-18T03:14:02.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.184+0000 7fddd17fa700 10 client.16848 handle_command_reply: tid=0 2023-07-18T03:14:02.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.185+0000 7fde0385cb80 1 client.16848 shutdown 2023-07-18T03:14:02.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.185+0000 7fde0385cb80 1 -- 172.21.15.101:0/1376705745 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559872d06be0 msgr2=0x559872d131b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:02.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.185+0000 7fde0385cb80 1 --2- 172.21.15.101:0/1376705745 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559872d06be0 0x559872d131b0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fddd8011740 tx=0x7fddd8011a50 comp rx=0 tx=0).stop 2023-07-18T03:14:02.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.185+0000 7fde0385cb80 1 -- 172.21.15.101:0/1376705745 shutdown_connections 2023-07-18T03:14:02.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.185+0000 7fde0385cb80 1 --2- 172.21.15.101:0/1376705745 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x559872d259f0 0x559872d27e00 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:02.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.185+0000 7fde0385cb80 1 --2- 172.21.15.101:0/1376705745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559872d15ee0 0x559872d136f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:02.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.185+0000 7fde0385cb80 1 --2- 172.21.15.101:0/1376705745 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559872d062d0 0x559872d12c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:02.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.185+0000 7fddd17fa700 0 client.16848 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:14:02.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.185+0000 7fde0385cb80 1 --2- 172.21.15.101:0/1376705745 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559872d06be0 0x559872d131b0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:02.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.185+0000 7fde0385cb80 1 -- 172.21.15.101:0/1376705745 >> 172.21.15.101:0/1376705745 conn(0x559872cfcda0 msgr2=0x559872d02230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:02.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.185+0000 7fde0385cb80 1 -- 172.21.15.101:0/1376705745 shutdown_connections 2023-07-18T03:14:02.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.185+0000 7fde0385cb80 1 -- 172.21.15.101:0/1376705745 wait complete. 2023-07-18T03:14:02.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.185+0000 7fde0385cb80 20 client.16848 trim_cache size 0 max 16384 2023-07-18T03:14:02.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.198+0000 7fde0385cb80 1 Processor -- start 2023-07-18T03:14:02.192 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.198+0000 7fde0385cb80 1 -- start start 2023-07-18T03:14:02.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.199+0000 7fde0385cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559872d062d0 0x559872d1c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:02.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.199+0000 7fde0385cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559872d06be0 0x559872d12480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:02.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.199+0000 7fde0385cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559872d15ee0 0x559872d0ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:02.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.199+0000 7fde0385cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559872bb2520 con 0x559872d06be0 2023-07-18T03:14:02.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.199+0000 7fde0385cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559872cfeef0 con 0x559872d062d0 2023-07-18T03:14:02.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.199+0000 7fde0385cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x559872d081a0 con 0x559872d15ee0 2023-07-18T03:14:02.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.199+0000 7fddedd9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559872d062d0 0x559872d1c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:02.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.199+0000 7fddedd9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559872d062d0 0x559872d1c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:55232/0 (socket says 172.21.15.101:55232) 2023-07-18T03:14:02.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.199+0000 7fddedd9b700 1 -- 172.21.15.101:0/3188683197 learned_addr learned my addr 172.21.15.101:0/3188683197 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:02.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.199+0000 7fdded59a700 1 --2- 172.21.15.101:0/3188683197 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559872d06be0 0x559872d12480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:02.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.199+0000 7fddee59c700 1 --2- 172.21.15.101:0/3188683197 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559872d15ee0 0x559872d0ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:02.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.199+0000 7fddedd9b700 1 -- 172.21.15.101:0/3188683197 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559872d15ee0 msgr2=0x559872d0ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:02.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.199+0000 7fddedd9b700 1 --2- 172.21.15.101:0/3188683197 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559872d15ee0 0x559872d0ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:02.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.199+0000 7fddedd9b700 1 -- 172.21.15.101:0/3188683197 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559872d06be0 msgr2=0x559872d12480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:02.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.199+0000 7fddedd9b700 1 --2- 172.21.15.101:0/3188683197 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559872d06be0 0x559872d12480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:02.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.199+0000 7fddedd9b700 1 -- 172.21.15.101:0/3188683197 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559872aad250 con 0x559872d062d0 2023-07-18T03:14:02.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.200+0000 7fddedd9b700 1 --2- 172.21.15.101:0/3188683197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559872d062d0 0x559872d1c8f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdde4099fd0 tx=0x7fdde400eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d395daf6d879b83 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:02.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.200+0000 7fdddcff9700 1 -- 172.21.15.101:0/3188683197 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde40983e0 con 0x559872d062d0 2023-07-18T03:14:02.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.200+0000 7fdddcff9700 1 -- 172.21.15.101:0/3188683197 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdde4099530 con 0x559872d062d0 2023-07-18T03:14:02.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.200+0000 7fdddcff9700 1 -- 172.21.15.101:0/3188683197 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde4096090 con 0x559872d062d0 2023-07-18T03:14:02.198 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.200+0000 7fde0385cb80 1 -- 172.21.15.101:0/3188683197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559872d062d0 msgr2=0x559872d1c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:02.198 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.200+0000 7fde0385cb80 1 --2- 172.21.15.101:0/3188683197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559872d062d0 0x559872d1c8f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdde4099fd0 tx=0x7fdde400eef0 comp rx=0 tx=0).stop 2023-07-18T03:14:02.198 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.201+0000 7fde0385cb80 1 -- 172.21.15.101:0/3188683197 shutdown_connections 2023-07-18T03:14:02.198 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.201+0000 7fde0385cb80 1 --2- 172.21.15.101:0/3188683197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559872d062d0 0x559872d1c8f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:02.198 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.201+0000 7fde0385cb80 1 --2- 172.21.15.101:0/3188683197 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559872d15ee0 0x559872d0ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:02.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.201+0000 7fde0385cb80 1 --2- 172.21.15.101:0/3188683197 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559872d06be0 0x559872d12480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:02.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.201+0000 7fde0385cb80 1 -- 172.21.15.101:0/3188683197 >> 172.21.15.101:0/3188683197 conn(0x559872cfcda0 msgr2=0x559872d21c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:02.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.201+0000 7fde0385cb80 1 -- 172.21.15.101:0/3188683197 shutdown_connections 2023-07-18T03:14:02.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.201+0000 7fde0385cb80 1 -- 172.21.15.101:0/3188683197 wait complete. 2023-07-18T03:14:02.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.201+0000 7fde0385cb80 1 -- start start 2023-07-18T03:14:02.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.201+0000 7fde0385cb80 1 Processor -- start 2023-07-18T03:14:02.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.202+0000 7fde0385cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559872d062d0 0x559872d1a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:02.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.202+0000 7fde0385cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559872d06be0 0x559872d1ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:02.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.202+0000 7fde0385cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559872d15ee0 0x559872d788b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:02.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.202+0000 7fde0385cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559872d78df0 con 0x559872d15ee0 2023-07-18T03:14:02.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.202+0000 7fde0385cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559872d78f30 con 0x559872d06be0 2023-07-18T03:14:02.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.202+0000 7fde0385cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x559872d790f0 con 0x559872d062d0 2023-07-18T03:14:02.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.202+0000 7fde0385cb80 10 client.0 _ll_register_callbacks cb 0x7fddd8009da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:14:02.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.202+0000 7fdded59a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559872d06be0 0x559872d1ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:02.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.202+0000 7fddee59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559872d15ee0 0x559872d788b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:02.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.202+0000 7fddedd9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559872d062d0 0x559872d1a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:02.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.202+0000 7fdded59a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559872d06be0 0x559872d1ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:55244/0 (socket says 172.21.15.101:55244) 2023-07-18T03:14:02.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.202+0000 7fdded59a700 1 -- 172.21.15.101:0/911933919 learned_addr learned my addr 172.21.15.101:0/911933919 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:02.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.203+0000 7fdded59a700 1 -- 172.21.15.101:0/911933919 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559872d062d0 msgr2=0x559872d1a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:02.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.203+0000 7fdded59a700 1 --2- 172.21.15.101:0/911933919 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559872d062d0 0x559872d1a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:02.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.203+0000 7fdded59a700 1 -- 172.21.15.101:0/911933919 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559872d15ee0 msgr2=0x559872d788b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:02.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.203+0000 7fdded59a700 1 --2- 172.21.15.101:0/911933919 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559872d15ee0 0x559872d788b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:02.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.203+0000 7fdded59a700 1 -- 172.21.15.101:0/911933919 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdde40b1040 con 0x559872d06be0 2023-07-18T03:14:02.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.203+0000 7fdded59a700 1 --2- 172.21.15.101:0/911933919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559872d06be0 0x559872d1ac40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x559872cfcab0 tx=0x7fddd80055e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7830cd07b731ec9 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:02.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.203+0000 7fddd17fa700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:14:02.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.203+0000 7fddd17fa700 1 -- 172.21.15.101:0/911933919 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd8016070 con 0x559872d06be0 2023-07-18T03:14:02.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.203+0000 7fddd17fa700 1 -- 172.21.15.101:0/911933919 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fddd8000e40 con 0x559872d06be0 2023-07-18T03:14:02.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.203+0000 7fde0385cb80 1 -- 172.21.15.101:0/911933919 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdde409e070 con 0x559872d06be0 2023-07-18T03:14:02.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.203+0000 7fddd17fa700 1 -- 172.21.15.101:0/911933919 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fddd8011da0 con 0x559872d06be0 2023-07-18T03:14:02.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.203+0000 7fde0385cb80 10 client.26897 fetch_fsmap learned FSMap version 29 2023-07-18T03:14:02.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.203+0000 7fde0385cb80 1 -- 172.21.15.101:0/911933919 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x559872a852b0 con 0x559872d06be0 2023-07-18T03:14:02.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.204+0000 7fddd17fa700 1 -- 172.21.15.101:0/911933919 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdde409e070 con 0x559872d06be0 2023-07-18T03:14:02.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.204+0000 7fddd17fa700 1 -- 172.21.15.101:0/911933919 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fddd8011770 con 0x559872d06be0 2023-07-18T03:14:02.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.204+0000 7fde0385cb80 10 client.26897 fetch_fsmap finished waiting for FSMap version 29 2023-07-18T03:14:02.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.204+0000 7fde0385cb80 10 client.26897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:14:02.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.204+0000 7fde0385cb80 20 client.26897 populate_metadata read hostname 'smithi101' 2023-07-18T03:14:02.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.204+0000 7fde0385cb80 1 --2- 172.21.15.101:0/911933919 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x559872d7a860 0x559872d7cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:02.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.204+0000 7fde0385cb80 4 client.26897 mds_command: new command op to 24562 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:14:02.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.204+0000 7fde0385cb80 1 -- 172.21.15.101:0/911933919 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559872c14050 con 0x559872d7a860 2023-07-18T03:14:02.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.205+0000 7fddedd9b700 1 --2- 172.21.15.101:0/911933919 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x559872d7a860 0x559872d7cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:02.205 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.205+0000 7fddd17fa700 10 client.26897 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:14:02.205 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.205+0000 7fddedd9b700 1 --2- 172.21.15.101:0/911933919 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x559872d7a860 0x559872d7cc70 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0b12ce8bd35bdbb server_cookie=553958af7af6cf3f in_seq=0 out_seq=0 2023-07-18T03:14:02.206 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.206+0000 7fddd17fa700 1 -- 172.21.15.101:0/911933919 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559872c14050 con 0x559872d7a860 2023-07-18T03:14:02.206 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.206+0000 7fddd17fa700 10 client.26897 handle_command_reply: tid=0 2023-07-18T03:14:02.206 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.206+0000 7fde0385cb80 1 client.26897 shutdown 2023-07-18T03:14:02.206 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.206+0000 7fde0385cb80 1 -- 172.21.15.101:0/911933919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559872d06be0 msgr2=0x559872d1ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:02.206 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.206+0000 7fde0385cb80 1 --2- 172.21.15.101:0/911933919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559872d06be0 0x559872d1ac40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x559872cfcab0 tx=0x7fddd80055e0 comp rx=0 tx=0).stop 2023-07-18T03:14:02.206 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.206+0000 7fde0385cb80 1 -- 172.21.15.101:0/911933919 shutdown_connections 2023-07-18T03:14:02.206 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.206+0000 7fde0385cb80 1 --2- 172.21.15.101:0/911933919 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x559872d7a860 0x559872d7cc70 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:02.207 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.206+0000 7fde0385cb80 1 --2- 172.21.15.101:0/911933919 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559872d15ee0 0x559872d788b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:02.207 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.206+0000 7fddd17fa700 0 client.26897 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:14:02.207 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.206+0000 7fde0385cb80 1 --2- 172.21.15.101:0/911933919 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559872d062d0 0x559872d1a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:02.207 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.206+0000 7fde0385cb80 1 --2- 172.21.15.101:0/911933919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559872d06be0 0x559872d1ac40 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:02.207 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.206+0000 7fde0385cb80 1 -- 172.21.15.101:0/911933919 >> 172.21.15.101:0/911933919 conn(0x559872cfcda0 msgr2=0x559872d14580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:02.207 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.206+0000 7fde0385cb80 1 -- 172.21.15.101:0/911933919 shutdown_connections 2023-07-18T03:14:02.207 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.206+0000 7fde0385cb80 1 -- 172.21.15.101:0/911933919 wait complete. 2023-07-18T03:14:02.207 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.206+0000 7fde0385cb80 20 client.26897 trim_cache size 0 max 16384 2023-07-18T03:14:02.208 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:14:02.208 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.207+0000 7fddeffff700 1 -- 172.21.15.101:0/2089172208 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fddd40a12c0 msgr2=0x7fddd40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:02.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.207+0000 7fddeffff700 1 --2- 172.21.15.101:0/2089172208 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fddd40a12c0 0x7fddd40a3770 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdde00062a0 tx=0x7fdde0006230 comp rx=0 tx=0).stop 2023-07-18T03:14:02.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.207+0000 7fddeffff700 1 -- 172.21.15.101:0/2089172208 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdde8100ca0 msgr2=0x7fdde819bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:02.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.207+0000 7fddeffff700 1 --2- 172.21.15.101:0/2089172208 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdde8100ca0 0x7fdde819bad0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fdde400e9a0 tx=0x7fdde400ecb0 comp rx=0 tx=0).stop 2023-07-18T03:14:02.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.207+0000 7fddeffff700 1 -- 172.21.15.101:0/2089172208 shutdown_connections 2023-07-18T03:14:02.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.207+0000 7fddeffff700 1 --2- 172.21.15.101:0/2089172208 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fddd40a12c0 0x7fddd40a3770 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:02.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.207+0000 7fddeffff700 1 --2- 172.21.15.101:0/2089172208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdde810a070 0x7fdde8195b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:02.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.207+0000 7fddeffff700 1 --2- 172.21.15.101:0/2089172208 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdde8100ca0 0x7fdde819bad0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:02.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.207+0000 7fddeffff700 1 --2- 172.21.15.101:0/2089172208 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdde81015b0 0x7fdde819c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:02.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.207+0000 7fddeffff700 1 -- 172.21.15.101:0/2089172208 >> 172.21.15.101:0/2089172208 conn(0x7fdde80fa510 msgr2=0x7fdde8101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:02.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.207+0000 7fddeffff700 1 -- 172.21.15.101:0/2089172208 shutdown_connections 2023-07-18T03:14:02.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:02.208+0000 7fddeffff700 1 -- 172.21.15.101:0/2089172208 wait complete. 2023-07-18T03:14:02.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:02 smithi101 ceph-mon[126379]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:14:02.231 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:02 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3222873517' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:14:02.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:02 smithi179 ceph-mon[129229]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:14:02.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:02 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/3222873517' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:14:02.544 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:02 smithi155 ceph-mon[128885]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:14:02.545 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:02 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/3222873517' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:14:02.586 INFO:tasks.workunit.client.0.smithi101.stdout: 1 496610 8.29 MB/sec execute 207 sec latency 444.245 ms 2023-07-18T03:14:03.233 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-18T03:14:03.233 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-18T03:14:03.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.450+0000 7fab30bff700 1 -- 172.21.15.101:0/3145684121 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fab2c106fc0 msgr2=0x7fab2c10eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:03.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.450+0000 7fab30bff700 1 --2- 172.21.15.101:0/3145684121 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fab2c106fc0 0x7fab2c10eb90 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fab2400b0d0 tx=0x7fab2400b3e0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.450+0000 7fab30bff700 1 -- 172.21.15.101:0/3145684121 shutdown_connections 2023-07-18T03:14:03.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.450+0000 7fab30bff700 1 --2- 172.21.15.101:0/3145684121 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fab2c106fc0 0x7fab2c10eb90 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.450+0000 7fab30bff700 1 --2- 172.21.15.101:0/3145684121 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab2c101f80 0x7fab2c102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.450+0000 7fab30bff700 1 --2- 172.21.15.101:0/3145684121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab2c102920 0x7fab2c1068e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.450+0000 7fab30bff700 1 -- 172.21.15.101:0/3145684121 >> 172.21.15.101:0/3145684121 conn(0x7fab2c0fd830 msgr2=0x7fab2c0ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:03.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.450+0000 7fab30bff700 1 -- 172.21.15.101:0/3145684121 shutdown_connections 2023-07-18T03:14:03.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.450+0000 7fab30bff700 1 -- 172.21.15.101:0/3145684121 wait complete. 2023-07-18T03:14:03.446 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.451+0000 7fab30bff700 1 Processor -- start 2023-07-18T03:14:03.446 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.451+0000 7fab30bff700 1 -- start start 2023-07-18T03:14:03.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.452+0000 7fab30bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fab2c101f80 0x7fab2c197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:03.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.452+0000 7fab30bff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab2c102920 0x7fab2c198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:03.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.452+0000 7fab30bff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab2c106fc0 0x7fab2c19c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:03.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.452+0000 7fab30bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c19cfb0 con 0x7fab2c101f80 2023-07-18T03:14:03.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.452+0000 7fab30bff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c19d120 con 0x7fab2c102920 2023-07-18T03:14:03.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.452+0000 7fab30bff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c19d2e0 con 0x7fab2c106fc0 2023-07-18T03:14:03.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.452+0000 7fab2a59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fab2c101f80 0x7fab2c197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:03.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.452+0000 7fab2a59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fab2c101f80 0x7fab2c197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:46716/0 (socket says 172.21.15.101:46716) 2023-07-18T03:14:03.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.452+0000 7fab2a59c700 1 -- 172.21.15.101:0/3184219254 learned_addr learned my addr 172.21.15.101:0/3184219254 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:03.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.452+0000 7fab29d9b700 1 --2- 172.21.15.101:0/3184219254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab2c102920 0x7fab2c198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:03.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.452+0000 7fab2ad9d700 1 --2- 172.21.15.101:0/3184219254 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab2c106fc0 0x7fab2c19c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:03.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.452+0000 7fab2a59c700 1 -- 172.21.15.101:0/3184219254 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab2c106fc0 msgr2=0x7fab2c19c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:03.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.452+0000 7fab2a59c700 1 --2- 172.21.15.101:0/3184219254 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab2c106fc0 0x7fab2c19c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.452+0000 7fab2a59c700 1 -- 172.21.15.101:0/3184219254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab2c102920 msgr2=0x7fab2c198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:03.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.452+0000 7fab2a59c700 1 --2- 172.21.15.101:0/3184219254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab2c102920 0x7fab2c198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.452+0000 7fab2a59c700 1 -- 172.21.15.101:0/3184219254 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab24009d00 con 0x7fab2c101f80 2023-07-18T03:14:03.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.452+0000 7fab2ad9d700 1 --2- 172.21.15.101:0/3184219254 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab2c106fc0 0x7fab2c19c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:14:03.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.453+0000 7fab2a59c700 1 --2- 172.21.15.101:0/3184219254 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fab2c101f80 0x7fab2c197fd0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00d980 tx=0x7fab1c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:03.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.453+0000 7fab1b7fe700 1 -- 172.21.15.101:0/3184219254 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c004220 con 0x7fab2c101f80 2023-07-18T03:14:03.450 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.453+0000 7fab30bff700 1 -- 172.21.15.101:0/3184219254 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab2c19d5c0 con 0x7fab2c101f80 2023-07-18T03:14:03.450 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.453+0000 7fab1b7fe700 1 -- 172.21.15.101:0/3184219254 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab1c007640 con 0x7fab2c101f80 2023-07-18T03:14:03.450 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.453+0000 7fab1b7fe700 1 -- 172.21.15.101:0/3184219254 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c005610 con 0x7fab2c101f80 2023-07-18T03:14:03.450 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.453+0000 7fab30bff700 1 -- 172.21.15.101:0/3184219254 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab2c1a4e00 con 0x7fab2c101f80 2023-07-18T03:14:03.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:03 smithi179 ceph-mon[129229]: pgmap v197: 129 pgs: 129 active+clean; 320 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 2.5 MiB/s wr, 181 op/s 2023-07-18T03:14:03.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.456+0000 7fab1b7fe700 1 -- 172.21.15.101:0/3184219254 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fab1c004750 con 0x7fab2c101f80 2023-07-18T03:14:03.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.457+0000 7fab44558b80 1 Processor -- start 2023-07-18T03:14:03.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.457+0000 7fab1b7fe700 1 --2- 172.21.15.101:0/3184219254 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fab140a12c0 0x7fab140a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:03.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.458+0000 7fab44558b80 1 -- start start 2023-07-18T03:14:03.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.458+0000 7fab1b7fe700 1 -- 172.21.15.101:0/3184219254 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fab1c0ccf20 con 0x7fab2c101f80 2023-07-18T03:14:03.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.458+0000 7fab2ad9d700 1 --2- 172.21.15.101:0/3184219254 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fab140a12c0 0x7fab140a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:03.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.458+0000 7fab44558b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e438170350 0x55e438170720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:03.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.458+0000 7fab44558b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e438170c60 0x55e43817b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:03.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.458+0000 7fab44558b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e43817bb50 0x55e43817df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:03.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.458+0000 7fab44558b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e438076d30 con 0x55e43817bb50 2023-07-18T03:14:03.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.458+0000 7fab44558b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4380c9ac0 con 0x55e438170c60 2023-07-18T03:14:03.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.458+0000 7fab44558b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4380c2f50 con 0x55e438170350 2023-07-18T03:14:03.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.458+0000 7fab29d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e438170c60 0x55e43817b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:03.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.458+0000 7fab2ad9d700 1 --2- 172.21.15.101:0/3184219254 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fab140a12c0 0x7fab140a3770 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fab2400b770 tx=0x7fab2400b040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:03.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.458+0000 7fab2ad9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e43817bb50 0x55e43817df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:03.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.458+0000 7fab2a59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e438170350 0x55e438170720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:03.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.458+0000 7fab29d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e438170c60 0x55e43817b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:55268/0 (socket says 172.21.15.101:55268) 2023-07-18T03:14:03.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.459+0000 7fab29d9b700 1 -- 172.21.15.101:0/2505136485 learned_addr learned my addr 172.21.15.101:0/2505136485 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:03.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.459+0000 7fab29d9b700 1 -- 172.21.15.101:0/2505136485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e438170350 msgr2=0x55e438170720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:03.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.459+0000 7fab29d9b700 1 --2- 172.21.15.101:0/2505136485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e438170350 0x55e438170720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.459+0000 7fab29d9b700 1 -- 172.21.15.101:0/2505136485 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e43817bb50 msgr2=0x55e43817df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:03.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.459+0000 7fab29d9b700 1 --2- 172.21.15.101:0/2505136485 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e43817bb50 0x55e43817df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.459+0000 7fab29d9b700 1 -- 172.21.15.101:0/2505136485 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e437ef0e60 con 0x55e438170c60 2023-07-18T03:14:03.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.459+0000 7fab2ad9d700 1 --2- 172.21.15.101:0/2505136485 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e43817bb50 0x55e43817df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:14:03.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.459+0000 7fab29d9b700 1 --2- 172.21.15.101:0/2505136485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e438170c60 0x55e43817b610 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fab20009c30 tx=0x7fab20009f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f152c07f1a88c80 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:03.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.459+0000 7fab18ff9700 1 -- 172.21.15.101:0/2505136485 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab20011d10 con 0x55e438170c60 2023-07-18T03:14:03.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.460+0000 7fab18ff9700 1 -- 172.21.15.101:0/2505136485 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab20011e70 con 0x55e438170c60 2023-07-18T03:14:03.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.460+0000 7fab18ff9700 1 -- 172.21.15.101:0/2505136485 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab20005280 con 0x55e438170c60 2023-07-18T03:14:03.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.460+0000 7fab44558b80 1 -- 172.21.15.101:0/2505136485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e438170c60 msgr2=0x55e43817b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:03.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.460+0000 7fab44558b80 1 --2- 172.21.15.101:0/2505136485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e438170c60 0x55e43817b610 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fab20009c30 tx=0x7fab20009f40 comp rx=0 tx=0).stop 2023-07-18T03:14:03.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.460+0000 7fab44558b80 1 -- 172.21.15.101:0/2505136485 shutdown_connections 2023-07-18T03:14:03.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.460+0000 7fab44558b80 1 --2- 172.21.15.101:0/2505136485 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e43817bb50 0x55e43817df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.460+0000 7fab44558b80 1 --2- 172.21.15.101:0/2505136485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e438170350 0x55e438170720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.460+0000 7fab44558b80 1 --2- 172.21.15.101:0/2505136485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e438170c60 0x55e43817b610 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.460+0000 7fab44558b80 1 -- 172.21.15.101:0/2505136485 >> 172.21.15.101:0/2505136485 conn(0x55e438166e20 msgr2=0x55e43816c720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:03.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.460+0000 7fab44558b80 1 -- 172.21.15.101:0/2505136485 shutdown_connections 2023-07-18T03:14:03.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.460+0000 7fab44558b80 1 -- 172.21.15.101:0/2505136485 wait complete. 2023-07-18T03:14:03.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.461+0000 7fab44558b80 1 -- start start 2023-07-18T03:14:03.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.461+0000 7fab44558b80 1 Processor -- start 2023-07-18T03:14:03.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.462+0000 7fab44558b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e438170350 0x55e43817b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:03.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.462+0000 7fab44558b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e438170c60 0x55e438178770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:03.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.462+0000 7fab44558b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e43817bb50 0x55e438178da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:03.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.462+0000 7fab44558b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e438062b40 con 0x55e438170c60 2023-07-18T03:14:03.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.462+0000 7fab44558b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e438062840 con 0x55e438170350 2023-07-18T03:14:03.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.462+0000 7fab44558b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e437f469a0 con 0x55e43817bb50 2023-07-18T03:14:03.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.462+0000 7fab44558b80 10 client.0 _ll_register_callbacks cb 0x55e4380607e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:14:03.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.462+0000 7fab29d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e438170c60 0x55e438178770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:03.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.462+0000 7fab29d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e438170c60 0x55e438178770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:46740/0 (socket says 172.21.15.101:46740) 2023-07-18T03:14:03.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.462+0000 7fab29d9b700 1 -- 172.21.15.101:0/2137048545 learned_addr learned my addr 172.21.15.101:0/2137048545 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:03.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.462+0000 7fab2ad9d700 1 --2- 172.21.15.101:0/2137048545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e43817bb50 0x55e438178da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:03.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.462+0000 7fab2a59c700 1 --2- 172.21.15.101:0/2137048545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e438170350 0x55e43817b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:03.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.462+0000 7fab29d9b700 1 -- 172.21.15.101:0/2137048545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e43817bb50 msgr2=0x55e438178da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:03.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.462+0000 7fab29d9b700 1 --2- 172.21.15.101:0/2137048545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e43817bb50 0x55e438178da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.462+0000 7fab29d9b700 1 -- 172.21.15.101:0/2137048545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e438170350 msgr2=0x55e43817b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:03.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.462+0000 7fab29d9b700 1 --2- 172.21.15.101:0/2137048545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e438170350 0x55e43817b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.462+0000 7fab29d9b700 1 -- 172.21.15.101:0/2137048545 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab2c10f3d0 con 0x55e438170c60 2023-07-18T03:14:03.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.463+0000 7fab29d9b700 1 --2- 172.21.15.101:0/2137048545 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e438170c60 0x55e438178770 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fab20004b40 tx=0x7fab20005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d5b8f0bf6ac35e3 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:03.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.463+0000 7fab11ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:14:03.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.463+0000 7fab11ffb700 1 -- 172.21.15.101:0/2137048545 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab20004050 con 0x55e438170c60 2023-07-18T03:14:03.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.463+0000 7fab11ffb700 1 -- 172.21.15.101:0/2137048545 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fab200041b0 con 0x55e438170c60 2023-07-18T03:14:03.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.463+0000 7fab44558b80 1 -- 172.21.15.101:0/2137048545 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e43801c570 con 0x55e438170c60 2023-07-18T03:14:03.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.463+0000 7fab11ffb700 1 -- 172.21.15.101:0/2137048545 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab20004370 con 0x55e438170c60 2023-07-18T03:14:03.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.463+0000 7fab44558b80 10 client.16866 fetch_fsmap learned FSMap version 29 2023-07-18T03:14:03.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.463+0000 7fab44558b80 1 -- 172.21.15.101:0/2137048545 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fab200098e0 con 0x55e438170c60 2023-07-18T03:14:03.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.463+0000 7fab11ffb700 1 -- 172.21.15.101:0/2137048545 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e43801c570 con 0x55e438170c60 2023-07-18T03:14:03.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.463+0000 7fab11ffb700 1 -- 172.21.15.101:0/2137048545 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fab20005040 con 0x55e438170c60 2023-07-18T03:14:03.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.464+0000 7fab44558b80 10 client.16866 fetch_fsmap finished waiting for FSMap version 29 2023-07-18T03:14:03.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.464+0000 7fab44558b80 10 client.16866 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:14:03.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.464+0000 7fab44558b80 20 client.16866 populate_metadata read hostname 'smithi101' 2023-07-18T03:14:03.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.464+0000 7fab44558b80 1 --2- 172.21.15.101:0/2137048545 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55e43818b510 0x55e43818d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:03.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.464+0000 7fab44558b80 4 client.16866 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:14:03.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.464+0000 7fab44558b80 1 -- 172.21.15.101:0/2137048545 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e4380bbf30 con 0x55e43818b510 2023-07-18T03:14:03.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.464+0000 7fab2a59c700 1 --2- 172.21.15.101:0/2137048545 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55e43818b510 0x55e43818d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:03.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.464+0000 7fab2a59c700 1 --2- 172.21.15.101:0/2137048545 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55e43818b510 0x55e43818d920 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29b88aa47e4aef25 server_cookie=3c7b3e0c0c60d4ce in_seq=0 out_seq=0 2023-07-18T03:14:03.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.464+0000 7fab11ffb700 10 client.16866 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:14:03.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.465+0000 7fab11ffb700 1 -- 172.21.15.101:0/2137048545 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e4380bbf30 con 0x55e43818b510 2023-07-18T03:14:03.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.465+0000 7fab11ffb700 10 client.16866 handle_command_reply: tid=0 2023-07-18T03:14:03.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.465+0000 7fab44558b80 1 client.16866 shutdown 2023-07-18T03:14:03.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.465+0000 7fab44558b80 1 -- 172.21.15.101:0/2137048545 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e438170c60 msgr2=0x55e438178770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:03.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.465+0000 7fab44558b80 1 --2- 172.21.15.101:0/2137048545 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e438170c60 0x55e438178770 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fab20004b40 tx=0x7fab20005ce0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.465+0000 7fab44558b80 1 -- 172.21.15.101:0/2137048545 shutdown_connections 2023-07-18T03:14:03.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.465+0000 7fab44558b80 1 --2- 172.21.15.101:0/2137048545 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55e43818b510 0x55e43818d920 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.465+0000 7fab44558b80 1 --2- 172.21.15.101:0/2137048545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e438170350 0x55e43817b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.465+0000 7fab11ffb700 0 client.16866 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:14:03.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.465+0000 7fab44558b80 1 --2- 172.21.15.101:0/2137048545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e43817bb50 0x55e438178da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.465+0000 7fab44558b80 1 --2- 172.21.15.101:0/2137048545 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e438170c60 0x55e438178770 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.465+0000 7fab44558b80 1 -- 172.21.15.101:0/2137048545 >> 172.21.15.101:0/2137048545 conn(0x55e438166e20 msgr2=0x55e43816c310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:03.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.466+0000 7fab44558b80 1 -- 172.21.15.101:0/2137048545 shutdown_connections 2023-07-18T03:14:03.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.466+0000 7fab44558b80 1 -- 172.21.15.101:0/2137048545 wait complete. 2023-07-18T03:14:03.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.466+0000 7fab44558b80 20 client.16866 trim_cache size 0 max 16384 2023-07-18T03:14:03.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.483+0000 7fab44558b80 1 Processor -- start 2023-07-18T03:14:03.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.483+0000 7fab44558b80 1 -- start start 2023-07-18T03:14:03.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.483+0000 7fab44558b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e438170350 0x55e438173230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:03.480 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.483+0000 7fab44558b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e438170c60 0x55e438177f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:03.480 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.483+0000 7fab44558b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e43817bb50 0x55e438178470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:03.480 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.483+0000 7fab44558b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e43816dda0 con 0x55e438170350 2023-07-18T03:14:03.480 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.483+0000 7fab44558b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4381715c0 con 0x55e438170c60 2023-07-18T03:14:03.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.483+0000 7fab44558b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e438166a10 con 0x55e43817bb50 2023-07-18T03:14:03.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.484+0000 7fab2a59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e438170350 0x55e438173230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:03.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.484+0000 7fab2a59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e438170350 0x55e438173230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:46752/0 (socket says 172.21.15.101:46752) 2023-07-18T03:14:03.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.484+0000 7fab2a59c700 1 -- 172.21.15.101:0/218860181 learned_addr learned my addr 172.21.15.101:0/218860181 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:03.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.484+0000 7fab2ad9d700 1 --2- 172.21.15.101:0/218860181 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e43817bb50 0x55e438178470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:03.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.484+0000 7fab29d9b700 1 --2- 172.21.15.101:0/218860181 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e438170c60 0x55e438177f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:03.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.484+0000 7fab2a59c700 1 -- 172.21.15.101:0/218860181 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e43817bb50 msgr2=0x55e438178470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:03.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.484+0000 7fab2a59c700 1 --2- 172.21.15.101:0/218860181 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e43817bb50 0x55e438178470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.484+0000 7fab2a59c700 1 -- 172.21.15.101:0/218860181 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e438170c60 msgr2=0x55e438177f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:03.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.484+0000 7fab2a59c700 1 --2- 172.21.15.101:0/218860181 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e438170c60 0x55e438177f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.484+0000 7fab2a59c700 1 -- 172.21.15.101:0/218860181 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e43816b160 con 0x55e438170350 2023-07-18T03:14:03.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.484+0000 7fab2a59c700 1 --2- 172.21.15.101:0/218860181 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e438170350 0x55e438173230 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fab1c0ca320 tx=0x7fab1c099840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e140da8f4c9a1386 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:03.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.484+0000 7fab197fa700 1 -- 172.21.15.101:0/218860181 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c0b6070 con 0x55e438170350 2023-07-18T03:14:03.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.484+0000 7fab197fa700 1 -- 172.21.15.101:0/218860181 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab1c0a2090 con 0x55e438170350 2023-07-18T03:14:03.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.484+0000 7fab197fa700 1 -- 172.21.15.101:0/218860181 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c099ea0 con 0x55e438170350 2023-07-18T03:14:03.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.485+0000 7fab44558b80 1 -- 172.21.15.101:0/218860181 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e438170350 msgr2=0x55e438173230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:03.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.485+0000 7fab44558b80 1 --2- 172.21.15.101:0/218860181 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e438170350 0x55e438173230 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fab1c0ca320 tx=0x7fab1c099840 comp rx=0 tx=0).stop 2023-07-18T03:14:03.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.485+0000 7fab44558b80 1 -- 172.21.15.101:0/218860181 shutdown_connections 2023-07-18T03:14:03.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.485+0000 7fab44558b80 1 --2- 172.21.15.101:0/218860181 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e43817bb50 0x55e438178470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.485+0000 7fab44558b80 1 --2- 172.21.15.101:0/218860181 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e438170350 0x55e438173230 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.485+0000 7fab44558b80 1 --2- 172.21.15.101:0/218860181 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e438170c60 0x55e438177f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.485+0000 7fab44558b80 1 -- 172.21.15.101:0/218860181 >> 172.21.15.101:0/218860181 conn(0x55e438166e20 msgr2=0x55e438173ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:03.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.485+0000 7fab44558b80 1 -- 172.21.15.101:0/218860181 shutdown_connections 2023-07-18T03:14:03.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.485+0000 7fab44558b80 1 -- 172.21.15.101:0/218860181 wait complete. 2023-07-18T03:14:03.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.485+0000 7fab44558b80 1 -- start start 2023-07-18T03:14:03.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.486+0000 7fab44558b80 1 Processor -- start 2023-07-18T03:14:03.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.486+0000 7fab44558b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e438170350 0x55e438180d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:03.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.486+0000 7fab44558b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e438170c60 0x55e438181260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:03.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.486+0000 7fab44558b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e43817bb50 0x55e4381817a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:03.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.486+0000 7fab44558b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4381df430 con 0x55e438170350 2023-07-18T03:14:03.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.486+0000 7fab44558b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4381df5a0 con 0x55e43817bb50 2023-07-18T03:14:03.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.486+0000 7fab44558b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4381df760 con 0x55e438170c60 2023-07-18T03:14:03.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.486+0000 7fab44558b80 10 client.0 _ll_register_callbacks cb 0x7fab24009420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:14:03.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.486+0000 7fab2a59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e438170350 0x55e438180d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:03.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.486+0000 7fab2a59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e438170350 0x55e438180d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:46766/0 (socket says 172.21.15.101:46766) 2023-07-18T03:14:03.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.486+0000 7fab2a59c700 1 -- 172.21.15.101:0/3614698069 learned_addr learned my addr 172.21.15.101:0/3614698069 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:03.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.486+0000 7fab29d9b700 1 --2- 172.21.15.101:0/3614698069 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e438170c60 0x55e438181260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:03.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.487+0000 7fab2ad9d700 1 --2- 172.21.15.101:0/3614698069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e43817bb50 0x55e4381817a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:03.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.487+0000 7fab2a59c700 1 -- 172.21.15.101:0/3614698069 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e438170c60 msgr2=0x55e438181260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:03.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.487+0000 7fab2a59c700 1 --2- 172.21.15.101:0/3614698069 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e438170c60 0x55e438181260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.487+0000 7fab2a59c700 1 -- 172.21.15.101:0/3614698069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e43817bb50 msgr2=0x55e4381817a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:03.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.487+0000 7fab2a59c700 1 --2- 172.21.15.101:0/3614698069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e43817bb50 0x55e4381817a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.487+0000 7fab2a59c700 1 -- 172.21.15.101:0/3614698069 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab2c1a4e00 con 0x55e438170350 2023-07-18T03:14:03.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.487+0000 7fab2a59c700 1 --2- 172.21.15.101:0/3614698069 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e438170350 0x55e438180d20 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fab1c0a1b60 tx=0x7fab1c097fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48c93a4e030215ce server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:03.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.487+0000 7fab11ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:14:03.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.487+0000 7fab11ffb700 1 -- 172.21.15.101:0/3614698069 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c0b6070 con 0x55e438170350 2023-07-18T03:14:03.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.487+0000 7fab11ffb700 1 -- 172.21.15.101:0/3614698069 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fab1c0b2250 con 0x55e438170350 2023-07-18T03:14:03.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.487+0000 7fab44558b80 1 -- 172.21.15.101:0/3614698069 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fab20011e70 con 0x55e438170350 2023-07-18T03:14:03.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.487+0000 7fab11ffb700 1 -- 172.21.15.101:0/3614698069 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab1c0b2410 con 0x55e438170350 2023-07-18T03:14:03.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.487+0000 7fab44558b80 10 client.16872 fetch_fsmap learned FSMap version 29 2023-07-18T03:14:03.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.487+0000 7fab44558b80 1 -- 172.21.15.101:0/3614698069 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e437ed9ce0 con 0x55e438170350 2023-07-18T03:14:03.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.488+0000 7fab11ffb700 1 -- 172.21.15.101:0/3614698069 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab1c063080 con 0x55e438170350 2023-07-18T03:14:03.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.488+0000 7fab11ffb700 1 -- 172.21.15.101:0/3614698069 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fab1c05fab0 con 0x55e438170350 2023-07-18T03:14:03.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.488+0000 7fab44558b80 10 client.16872 fetch_fsmap finished waiting for FSMap version 29 2023-07-18T03:14:03.492 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.488+0000 7fab44558b80 10 client.16872 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:14:03.492 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.488+0000 7fab44558b80 20 client.16872 populate_metadata read hostname 'smithi101' 2023-07-18T03:14:03.492 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.488+0000 7fab44558b80 1 --2- 172.21.15.101:0/3614698069 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55e4381e0a90 0x55e4381e2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:03.492 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.488+0000 7fab44558b80 4 client.16872 mds_command: new command op to 24562 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-18T03:14:03.492 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.488+0000 7fab44558b80 1 -- 172.21.15.101:0/3614698069 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e43807e0d0 con 0x55e4381e0a90 2023-07-18T03:14:03.492 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.488+0000 7fab29d9b700 1 --2- 172.21.15.101:0/3614698069 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55e4381e0a90 0x55e4381e2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:03.492 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.489+0000 7fab29d9b700 1 --2- 172.21.15.101:0/3614698069 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55e4381e0a90 0x55e4381e2ea0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=124517c4a322c0b1 server_cookie=7b268d6a538426da in_seq=0 out_seq=0 2023-07-18T03:14:03.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.489+0000 7fab11ffb700 10 client.16872 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:14:03.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.490+0000 7fab11ffb700 1 -- 172.21.15.101:0/3614698069 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e43807e0d0 con 0x55e4381e0a90 2023-07-18T03:14:03.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.490+0000 7fab11ffb700 10 client.16872 handle_command_reply: tid=0 2023-07-18T03:14:03.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.490+0000 7fab44558b80 1 client.16872 shutdown 2023-07-18T03:14:03.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.490+0000 7fab44558b80 1 -- 172.21.15.101:0/3614698069 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e438170350 msgr2=0x55e438180d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:03.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.490+0000 7fab44558b80 1 --2- 172.21.15.101:0/3614698069 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e438170350 0x55e438180d20 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fab1c0a1b60 tx=0x7fab1c097fe0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.490+0000 7fab44558b80 1 -- 172.21.15.101:0/3614698069 shutdown_connections 2023-07-18T03:14:03.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.490+0000 7fab44558b80 1 --2- 172.21.15.101:0/3614698069 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55e4381e0a90 0x55e4381e2ea0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.490+0000 7fab44558b80 1 --2- 172.21.15.101:0/3614698069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e43817bb50 0x55e4381817a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.490+0000 7fab44558b80 1 --2- 172.21.15.101:0/3614698069 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e438170350 0x55e438180d20 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.490+0000 7fab44558b80 1 --2- 172.21.15.101:0/3614698069 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e438170c60 0x55e438181260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.490+0000 7fab11ffb700 0 client.16872 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:14:03.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.490+0000 7fab44558b80 1 -- 172.21.15.101:0/3614698069 >> 172.21.15.101:0/3614698069 conn(0x55e438166e20 msgr2=0x55e43817a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:03.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.490+0000 7fab44558b80 1 -- 172.21.15.101:0/3614698069 shutdown_connections 2023-07-18T03:14:03.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.490+0000 7fab44558b80 1 -- 172.21.15.101:0/3614698069 wait complete. 2023-07-18T03:14:03.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.491+0000 7fab44558b80 20 client.16872 trim_cache size 0 max 16384 2023-07-18T03:14:03.495 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:14:03.496 INFO:teuthology.orchestra.run.smithi101.stdout: "return_code": 0, 2023-07-18T03:14:03.496 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_tag": "c37b945e-a67b-4cc7-9a32-a53345e5f26a", 2023-07-18T03:14:03.496 INFO:teuthology.orchestra.run.smithi101.stdout: "mode": "asynchronous" 2023-07-18T03:14:03.496 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:14:03.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.491+0000 7fab30bff700 1 -- 172.21.15.101:0/3184219254 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fab140a12c0 msgr2=0x7fab140a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:03.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.491+0000 7fab30bff700 1 --2- 172.21.15.101:0/3184219254 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fab140a12c0 0x7fab140a3770 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fab2400b770 tx=0x7fab2400b040 comp rx=0 tx=0).stop 2023-07-18T03:14:03.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.491+0000 7fab30bff700 1 -- 172.21.15.101:0/3184219254 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fab2c101f80 msgr2=0x7fab2c197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:03.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.491+0000 7fab30bff700 1 --2- 172.21.15.101:0/3184219254 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fab2c101f80 0x7fab2c197fd0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00d980 tx=0x7fab1c00dc90 comp rx=0 tx=0).stop 2023-07-18T03:14:03.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.491+0000 7fab30bff700 1 -- 172.21.15.101:0/3184219254 shutdown_connections 2023-07-18T03:14:03.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.491+0000 7fab30bff700 1 --2- 172.21.15.101:0/3184219254 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab2c106fc0 0x7fab2c19c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.491+0000 7fab30bff700 1 --2- 172.21.15.101:0/3184219254 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fab2c101f80 0x7fab2c197fd0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.491+0000 7fab30bff700 1 --2- 172.21.15.101:0/3184219254 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fab140a12c0 0x7fab140a3770 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.491+0000 7fab30bff700 1 --2- 172.21.15.101:0/3184219254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab2c102920 0x7fab2c198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.491+0000 7fab30bff700 1 -- 172.21.15.101:0/3184219254 >> 172.21.15.101:0/3184219254 conn(0x7fab2c0fd830 msgr2=0x7fab2c0fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:03.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.491+0000 7fab30bff700 1 -- 172.21.15.101:0/3184219254 shutdown_connections 2023-07-18T03:14:03.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.491+0000 7fab30bff700 1 -- 172.21.15.101:0/3184219254 wait complete. 2023-07-18T03:14:03.499 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:03 smithi101 ceph-mon[126379]: pgmap v197: 129 pgs: 129 active+clean; 320 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 2.5 MiB/s wr, 181 op/s 2023-07-18T03:14:03.505 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:14:03.545 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:03 smithi155 ceph-mon[128885]: pgmap v197: 129 pgs: 129 active+clean; 320 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 2.5 MiB/s wr, 181 op/s 2023-07-18T03:14:03.586 INFO:tasks.workunit.client.0.smithi101.stdout: 1 496728 8.25 MB/sec execute 208 sec latency 1382.933 ms 2023-07-18T03:14:03.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.717+0000 7f86ef5ff700 1 -- 172.21.15.101:0/2970629433 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f86e8100490 msgr2=0x7f86e810c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:03.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.717+0000 7f86ef5ff700 1 --2- 172.21.15.101:0/2970629433 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f86e8100490 0x7f86e810c920 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f86e000b0a0 tx=0x7f86e000b3b0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.717+0000 7f86ef5ff700 1 -- 172.21.15.101:0/2970629433 shutdown_connections 2023-07-18T03:14:03.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.717+0000 7f86ef5ff700 1 --2- 172.21.15.101:0/2970629433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86e8105af0 0x7f86e8105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.717+0000 7f86ef5ff700 1 --2- 172.21.15.101:0/2970629433 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f86e8100490 0x7f86e810c920 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.717+0000 7f86ef5ff700 1 --2- 172.21.15.101:0/2970629433 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86e80ffae0 0x7f86e80fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.717+0000 7f86ef5ff700 1 -- 172.21.15.101:0/2970629433 >> 172.21.15.101:0/2970629433 conn(0x7f86e80fb5e0 msgr2=0x7f86e80fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:03.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.717+0000 7f86ef5ff700 1 -- 172.21.15.101:0/2970629433 shutdown_connections 2023-07-18T03:14:03.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.717+0000 7f86ef5ff700 1 -- 172.21.15.101:0/2970629433 wait complete. 2023-07-18T03:14:03.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.718+0000 7f86ef5ff700 1 Processor -- start 2023-07-18T03:14:03.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.719+0000 7f86ef5ff700 1 -- start start 2023-07-18T03:14:03.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.719+0000 7f86ef5ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86e80ffae0 0x7f86e8072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:03.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.719+0000 7f86ef5ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f86e8100490 0x7f86e8072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:03.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.719+0000 7f86ef5ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86e8105af0 0x7f86e81a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:03.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.719+0000 7f86ef5ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e8073580 con 0x7f86e8100490 2023-07-18T03:14:03.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.719+0000 7f86ef5ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e81a44c0 con 0x7f86e80ffae0 2023-07-18T03:14:03.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.719+0000 7f86ef5ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e81a4600 con 0x7f86e8105af0 2023-07-18T03:14:03.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.720+0000 7f86ecb9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f86e8100490 0x7f86e8072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:03.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.720+0000 7f86ed39b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86e80ffae0 0x7f86e8072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:03.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.720+0000 7f86ecb9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f86e8100490 0x7f86e8072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:46776/0 (socket says 172.21.15.101:46776) 2023-07-18T03:14:03.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.720+0000 7f86edb9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86e8105af0 0x7f86e81a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:03.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.720+0000 7f86ecb9a700 1 -- 172.21.15.101:0/1143224362 learned_addr learned my addr 172.21.15.101:0/1143224362 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:03.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.720+0000 7f86ecb9a700 1 -- 172.21.15.101:0/1143224362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86e8105af0 msgr2=0x7f86e81a3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:03.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.720+0000 7f86ecb9a700 1 --2- 172.21.15.101:0/1143224362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86e8105af0 0x7f86e81a3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.720+0000 7f86ecb9a700 1 -- 172.21.15.101:0/1143224362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86e80ffae0 msgr2=0x7f86e8072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:03.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.720+0000 7f86ecb9a700 1 --2- 172.21.15.101:0/1143224362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86e80ffae0 0x7f86e8072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.720+0000 7f86ecb9a700 1 -- 172.21.15.101:0/1143224362 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e0009d00 con 0x7f86e8100490 2023-07-18T03:14:03.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.720+0000 7f86edb9c700 1 --2- 172.21.15.101:0/1143224362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86e8105af0 0x7f86e81a3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:14:03.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.720+0000 7f86ecb9a700 1 --2- 172.21.15.101:0/1143224362 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f86e8100490 0x7f86e8072d50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f86d800e9a0 tx=0x7f86d800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:03.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.720+0000 7f86de7fc700 1 -- 172.21.15.101:0/1143224362 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d8009790 con 0x7f86e8100490 2023-07-18T03:14:03.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.720+0000 7f86ef5ff700 1 -- 172.21.15.101:0/1143224362 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86e81a4860 con 0x7f86e8100490 2023-07-18T03:14:03.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.720+0000 7f86de7fc700 1 -- 172.21.15.101:0/1143224362 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86d8010430 con 0x7f86e8100490 2023-07-18T03:14:03.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.721+0000 7f86de7fc700 1 -- 172.21.15.101:0/1143224362 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d8018680 con 0x7f86e8100490 2023-07-18T03:14:03.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.721+0000 7f86ef5ff700 1 -- 172.21.15.101:0/1143224362 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86e81a4d90 con 0x7f86e8100490 2023-07-18T03:14:03.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.724+0000 7f86de7fc700 1 -- 172.21.15.101:0/1143224362 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f86d80105a0 con 0x7f86e8100490 2023-07-18T03:14:03.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.725+0000 7f8702e33b80 1 Processor -- start 2023-07-18T03:14:03.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.725+0000 7f86de7fc700 1 --2- 172.21.15.101:0/1143224362 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f86d40a1340 0x7f86d40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:03.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.725+0000 7f8702e33b80 1 -- start start 2023-07-18T03:14:03.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.725+0000 7f86de7fc700 1 -- 172.21.15.101:0/1143224362 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f86d8014070 con 0x7f86e8100490 2023-07-18T03:14:03.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.726+0000 7f8702e33b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5631aad332d0 0x5631aad336a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:03.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.726+0000 7f8702e33b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5631aad33be0 0x5631aad3e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:03.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.726+0000 7f86edb9c700 1 --2- 172.21.15.101:0/1143224362 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f86d40a1340 0x7f86d40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:03.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.726+0000 7f8702e33b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631aad3ead0 0x5631aad40eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:03.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.726+0000 7f8702e33b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631aac39cb0 con 0x5631aad332d0 2023-07-18T03:14:03.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.726+0000 7f8702e33b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631aac8ca40 con 0x5631aad3ead0 2023-07-18T03:14:03.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.726+0000 7f8702e33b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631aac85ed0 con 0x5631aad33be0 2023-07-18T03:14:03.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.726+0000 7f86ed39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5631aad332d0 0x5631aad336a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:03.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.726+0000 7f86ed39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5631aad332d0 0x5631aad336a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:46778/0 (socket says 172.21.15.101:46778) 2023-07-18T03:14:03.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.726+0000 7f86ed39b700 1 -- 172.21.15.101:0/1905650480 learned_addr learned my addr 172.21.15.101:0/1905650480 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:03.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.726+0000 7f86edb9c700 1 --2- 172.21.15.101:0/1905650480 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631aad3ead0 0x5631aad40eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:03.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.726+0000 7f86ecb9a700 1 --2- 172.21.15.101:0/1905650480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5631aad33be0 0x5631aad3e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:03.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.726+0000 7f86ed39b700 1 -- 172.21.15.101:0/1905650480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5631aad33be0 msgr2=0x5631aad3e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:03.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.726+0000 7f86ed39b700 1 --2- 172.21.15.101:0/1905650480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5631aad33be0 0x5631aad3e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.726+0000 7f86ed39b700 1 -- 172.21.15.101:0/1905650480 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631aad3ead0 msgr2=0x5631aad40eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:03.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.726+0000 7f86ed39b700 1 --2- 172.21.15.101:0/1905650480 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631aad3ead0 0x5631aad40eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.726+0000 7f86ed39b700 1 -- 172.21.15.101:0/1905650480 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631aaab3de0 con 0x5631aad332d0 2023-07-18T03:14:03.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.727+0000 7f86edb9c700 1 --2- 172.21.15.101:0/1143224362 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f86d40a1340 0x7f86d40a37f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f86e8074920 tx=0x7f86e000bca0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:03.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.727+0000 7f86ed39b700 1 --2- 172.21.15.101:0/1905650480 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5631aad332d0 0x5631aad336a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f86e4007c70 tx=0x7f86e4007f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c05aadce013cf76 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:03.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.727+0000 7f86d37fe700 1 -- 172.21.15.101:0/1905650480 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e4010b00 con 0x5631aad332d0 2023-07-18T03:14:03.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.727+0000 7f86d37fe700 1 -- 172.21.15.101:0/1905650480 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86e4015070 con 0x5631aad332d0 2023-07-18T03:14:03.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.727+0000 7f86d37fe700 1 -- 172.21.15.101:0/1905650480 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e40193d0 con 0x5631aad332d0 2023-07-18T03:14:03.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.728+0000 7f8702e33b80 1 -- 172.21.15.101:0/1905650480 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5631aad332d0 msgr2=0x5631aad336a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:03.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.728+0000 7f8702e33b80 1 --2- 172.21.15.101:0/1905650480 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5631aad332d0 0x5631aad336a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f86e4007c70 tx=0x7f86e4007f80 comp rx=0 tx=0).stop 2023-07-18T03:14:03.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.728+0000 7f8702e33b80 1 -- 172.21.15.101:0/1905650480 shutdown_connections 2023-07-18T03:14:03.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.728+0000 7f8702e33b80 1 --2- 172.21.15.101:0/1905650480 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631aad3ead0 0x5631aad40eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.728+0000 7f8702e33b80 1 --2- 172.21.15.101:0/1905650480 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5631aad332d0 0x5631aad336a0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.728+0000 7f8702e33b80 1 --2- 172.21.15.101:0/1905650480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5631aad33be0 0x5631aad3e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.728+0000 7f8702e33b80 1 -- 172.21.15.101:0/1905650480 >> 172.21.15.101:0/1905650480 conn(0x5631aad29da0 msgr2=0x5631aad2f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:03.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.728+0000 7f8702e33b80 1 -- 172.21.15.101:0/1905650480 shutdown_connections 2023-07-18T03:14:03.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.728+0000 7f8702e33b80 1 -- 172.21.15.101:0/1905650480 wait complete. 2023-07-18T03:14:03.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.728+0000 7f8702e33b80 1 -- start start 2023-07-18T03:14:03.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.729+0000 7f8702e33b80 1 Processor -- start 2023-07-18T03:14:03.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.729+0000 7f8702e33b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5631aad332d0 0x5631aad3e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:03.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.729+0000 7f8702e33b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631aad33be0 0x5631aad3b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:03.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.729+0000 7f8702e33b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5631aad3ead0 0x5631aad3bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:03.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.729+0000 7f8702e33b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631aac25ac0 con 0x5631aad332d0 2023-07-18T03:14:03.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.729+0000 7f8702e33b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631aac257c0 con 0x5631aad33be0 2023-07-18T03:14:03.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.729+0000 7f8702e33b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631aab09920 con 0x5631aad3ead0 2023-07-18T03:14:03.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.730+0000 7f8702e33b80 10 client.0 _ll_register_callbacks cb 0x5631aac23760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:14:03.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.730+0000 7f86ed39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5631aad332d0 0x5631aad3e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:03.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.730+0000 7f86ed39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5631aad332d0 0x5631aad3e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:46786/0 (socket says 172.21.15.101:46786) 2023-07-18T03:14:03.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.730+0000 7f86ed39b700 1 -- 172.21.15.101:0/377773305 learned_addr learned my addr 172.21.15.101:0/377773305 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:03.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.730+0000 7f86ecb9a700 1 --2- 172.21.15.101:0/377773305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631aad33be0 0x5631aad3b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:03.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.730+0000 7f86edb9c700 1 --2- 172.21.15.101:0/377773305 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5631aad3ead0 0x5631aad3bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:03.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.730+0000 7f86ed39b700 1 -- 172.21.15.101:0/377773305 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5631aad3ead0 msgr2=0x5631aad3bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:03.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.730+0000 7f86ed39b700 1 --2- 172.21.15.101:0/377773305 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5631aad3ead0 0x5631aad3bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.730+0000 7f86ed39b700 1 -- 172.21.15.101:0/377773305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631aad33be0 msgr2=0x5631aad3b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:03.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.730+0000 7f86ed39b700 1 --2- 172.21.15.101:0/377773305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631aad33be0 0x5631aad3b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.730+0000 7f86ed39b700 1 -- 172.21.15.101:0/377773305 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86e40078a0 con 0x5631aad332d0 2023-07-18T03:14:03.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.730+0000 7f86ed39b700 1 --2- 172.21.15.101:0/377773305 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5631aad332d0 0x5631aad3e430 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f86e4011560 tx=0x7f86e4011590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dceb3cc03c20f41 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:03.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.730+0000 7f86d0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:14:03.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.731+0000 7f86d0ff9700 1 -- 172.21.15.101:0/377773305 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e4011860 con 0x5631aad332d0 2023-07-18T03:14:03.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.731+0000 7f8702e33b80 1 -- 172.21.15.101:0/377773305 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5631aabdf4f0 con 0x5631aad332d0 2023-07-18T03:14:03.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.731+0000 7f86d0ff9700 1 -- 172.21.15.101:0/377773305 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f86e4004020 con 0x5631aad332d0 2023-07-18T03:14:03.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.731+0000 7f86d0ff9700 1 -- 172.21.15.101:0/377773305 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86e40119c0 con 0x5631aad332d0 2023-07-18T03:14:03.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.731+0000 7f8702e33b80 10 client.16884 fetch_fsmap learned FSMap version 29 2023-07-18T03:14:03.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.731+0000 7f8702e33b80 1 -- 172.21.15.101:0/377773305 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5631aaa9cc60 con 0x5631aad332d0 2023-07-18T03:14:03.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.731+0000 7f86d0ff9700 1 -- 172.21.15.101:0/377773305 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5631aabdf4f0 con 0x5631aad332d0 2023-07-18T03:14:03.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.731+0000 7f86d0ff9700 1 -- 172.21.15.101:0/377773305 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f86e4005200 con 0x5631aad332d0 2023-07-18T03:14:03.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.731+0000 7f8702e33b80 10 client.16884 fetch_fsmap finished waiting for FSMap version 29 2023-07-18T03:14:03.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.731+0000 7f8702e33b80 10 client.16884 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:14:03.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.731+0000 7f8702e33b80 20 client.16884 populate_metadata read hostname 'smithi101' 2023-07-18T03:14:03.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.731+0000 7f8702e33b80 1 --2- 172.21.15.101:0/377773305 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5631aad4e300 0x5631aad50710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:03.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.731+0000 7f8702e33b80 4 client.16884 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:14:03.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.731+0000 7f8702e33b80 1 -- 172.21.15.101:0/377773305 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5631aac7eeb0 con 0x5631aad4e300 2023-07-18T03:14:03.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.732+0000 7f86ecb9a700 1 --2- 172.21.15.101:0/377773305 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5631aad4e300 0x5631aad50710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:03.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.732+0000 7f86ecb9a700 1 --2- 172.21.15.101:0/377773305 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5631aad4e300 0x5631aad50710 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93717035ab37af8 server_cookie=20abbee15452f4d9 in_seq=0 out_seq=0 2023-07-18T03:14:03.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.732+0000 7f86d0ff9700 10 client.16884 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:14:03.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.733+0000 7f86d0ff9700 1 -- 172.21.15.101:0/377773305 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5631aac7eeb0 con 0x5631aad4e300 2023-07-18T03:14:03.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.733+0000 7f86d0ff9700 10 client.16884 handle_command_reply: tid=0 2023-07-18T03:14:03.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.733+0000 7f8702e33b80 1 client.16884 shutdown 2023-07-18T03:14:03.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.733+0000 7f8702e33b80 1 -- 172.21.15.101:0/377773305 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5631aad332d0 msgr2=0x5631aad3e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:03.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.733+0000 7f8702e33b80 1 --2- 172.21.15.101:0/377773305 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5631aad332d0 0x5631aad3e430 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f86e4011560 tx=0x7f86e4011590 comp rx=0 tx=0).stop 2023-07-18T03:14:03.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.733+0000 7f8702e33b80 1 -- 172.21.15.101:0/377773305 shutdown_connections 2023-07-18T03:14:03.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.733+0000 7f8702e33b80 1 --2- 172.21.15.101:0/377773305 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5631aad4e300 0x5631aad50710 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.733+0000 7f8702e33b80 1 --2- 172.21.15.101:0/377773305 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5631aad3ead0 0x5631aad3bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.733+0000 7f8702e33b80 1 --2- 172.21.15.101:0/377773305 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5631aad332d0 0x5631aad3e430 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.733+0000 7f86d0ff9700 0 client.16884 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:14:03.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.733+0000 7f8702e33b80 1 --2- 172.21.15.101:0/377773305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631aad33be0 0x5631aad3b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.733+0000 7f8702e33b80 1 -- 172.21.15.101:0/377773305 >> 172.21.15.101:0/377773305 conn(0x5631aad29da0 msgr2=0x5631aad2f290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:03.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.733+0000 7f8702e33b80 1 -- 172.21.15.101:0/377773305 shutdown_connections 2023-07-18T03:14:03.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.733+0000 7f8702e33b80 1 -- 172.21.15.101:0/377773305 wait complete. 2023-07-18T03:14:03.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.734+0000 7f8702e33b80 20 client.16884 trim_cache size 0 max 16384 2023-07-18T03:14:03.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.751+0000 7f8702e33b80 1 Processor -- start 2023-07-18T03:14:03.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.751+0000 7f8702e33b80 1 -- start start 2023-07-18T03:14:03.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.751+0000 7f8702e33b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631aad332d0 0x5631aad454e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:03.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.751+0000 7f8702e33b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5631aad33be0 0x5631aad3dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:03.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.751+0000 7f8702e33b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5631aad3ead0 0x5631aad3ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:03.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.751+0000 7f8702e33b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631aad29970 con 0x5631aad3ead0 2023-07-18T03:14:03.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.751+0000 7f8702e33b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631aad3ca00 con 0x5631aad332d0 2023-07-18T03:14:03.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.751+0000 7f8702e33b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631aad45a20 con 0x5631aad33be0 2023-07-18T03:14:03.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.751+0000 7f86ecb9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5631aad33be0 0x5631aad3dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:03.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.751+0000 7f86edb9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5631aad3ead0 0x5631aad3ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:03.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.751+0000 7f86ed39b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631aad332d0 0x5631aad454e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:03.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.751+0000 7f86ecb9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5631aad33be0 0x5631aad3dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:43224/0 (socket says 172.21.15.101:43224) 2023-07-18T03:14:03.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.751+0000 7f86ecb9a700 1 -- 172.21.15.101:0/660513636 learned_addr learned my addr 172.21.15.101:0/660513636 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:03.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.752+0000 7f86ecb9a700 1 -- 172.21.15.101:0/660513636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631aad332d0 msgr2=0x5631aad454e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:03.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.752+0000 7f86ecb9a700 1 --2- 172.21.15.101:0/660513636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631aad332d0 0x5631aad454e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.752+0000 7f86ecb9a700 1 -- 172.21.15.101:0/660513636 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5631aad3ead0 msgr2=0x5631aad3ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:03.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.752+0000 7f86ecb9a700 1 --2- 172.21.15.101:0/660513636 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5631aad3ead0 0x5631aad3ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.752+0000 7f86ecb9a700 1 -- 172.21.15.101:0/660513636 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631aaada250 con 0x5631aad33be0 2023-07-18T03:14:03.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.752+0000 7f86ecb9a700 1 --2- 172.21.15.101:0/660513636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5631aad33be0 0x5631aad3dfd0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f86d8099c60 tx=0x7f86d80cc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cabc8113908daa95 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:03.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.752+0000 7f86d3fff700 1 -- 172.21.15.101:0/660513636 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d809df90 con 0x5631aad33be0 2023-07-18T03:14:03.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.752+0000 7f86d3fff700 1 -- 172.21.15.101:0/660513636 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86d80a23b0 con 0x5631aad33be0 2023-07-18T03:14:03.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.752+0000 7f86d3fff700 1 -- 172.21.15.101:0/660513636 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d809a420 con 0x5631aad33be0 2023-07-18T03:14:03.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.753+0000 7f8702e33b80 1 -- 172.21.15.101:0/660513636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5631aad33be0 msgr2=0x5631aad3dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:03.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.753+0000 7f8702e33b80 1 --2- 172.21.15.101:0/660513636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5631aad33be0 0x5631aad3dfd0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f86d8099c60 tx=0x7f86d80cc040 comp rx=0 tx=0).stop 2023-07-18T03:14:03.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.753+0000 7f8702e33b80 1 -- 172.21.15.101:0/660513636 shutdown_connections 2023-07-18T03:14:03.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.753+0000 7f8702e33b80 1 --2- 172.21.15.101:0/660513636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631aad332d0 0x5631aad454e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.753+0000 7f8702e33b80 1 --2- 172.21.15.101:0/660513636 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5631aad3ead0 0x5631aad3ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.753+0000 7f8702e33b80 1 --2- 172.21.15.101:0/660513636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5631aad33be0 0x5631aad3dfd0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.753+0000 7f8702e33b80 1 -- 172.21.15.101:0/660513636 >> 172.21.15.101:0/660513636 conn(0x5631aad29da0 msgr2=0x5631aad36990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:03.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.753+0000 7f8702e33b80 1 -- 172.21.15.101:0/660513636 shutdown_connections 2023-07-18T03:14:03.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.753+0000 7f8702e33b80 1 -- 172.21.15.101:0/660513636 wait complete. 2023-07-18T03:14:03.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.753+0000 7f8702e33b80 1 -- start start 2023-07-18T03:14:03.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.754+0000 7f8702e33b80 1 Processor -- start 2023-07-18T03:14:03.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.754+0000 7f8702e33b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5631aad332d0 0x5631aad45450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:03.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.754+0000 7f8702e33b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631aad33be0 0x5631aad41910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:03.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.754+0000 7f8702e33b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5631aad3ead0 0x5631aad42050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:03.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.754+0000 7f8702e33b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631aad42760 con 0x5631aad3ead0 2023-07-18T03:14:03.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.754+0000 7f8702e33b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631aada21a0 con 0x5631aad33be0 2023-07-18T03:14:03.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.754+0000 7f8702e33b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631aada22e0 con 0x5631aad332d0 2023-07-18T03:14:03.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.754+0000 7f8702e33b80 10 client.0 _ll_register_callbacks cb 0x7f86d8005760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:14:03.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.755+0000 7f86edb9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5631aad3ead0 0x5631aad42050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:03.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.755+0000 7f86ed39b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5631aad332d0 0x5631aad45450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:03.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.755+0000 7f86edb9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5631aad3ead0 0x5631aad42050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:46802/0 (socket says 172.21.15.101:46802) 2023-07-18T03:14:03.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.755+0000 7f86edb9c700 1 -- 172.21.15.101:0/2502931691 learned_addr learned my addr 172.21.15.101:0/2502931691 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:03.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.755+0000 7f86ed39b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5631aad332d0 0x5631aad45450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:43240/0 (socket says 172.21.15.101:43240) 2023-07-18T03:14:03.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.755+0000 7f86ecb9a700 1 --2- 172.21.15.101:0/2502931691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631aad33be0 0x5631aad41910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:03.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.755+0000 7f86edb9c700 1 -- 172.21.15.101:0/2502931691 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5631aad332d0 msgr2=0x5631aad45450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:03.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.755+0000 7f86edb9c700 1 --2- 172.21.15.101:0/2502931691 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5631aad332d0 0x5631aad45450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.755+0000 7f86edb9c700 1 -- 172.21.15.101:0/2502931691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631aad33be0 msgr2=0x5631aad41910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:03.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.755+0000 7f86edb9c700 1 --2- 172.21.15.101:0/2502931691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631aad33be0 0x5631aad41910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.755+0000 7f86edb9c700 1 -- 172.21.15.101:0/2502931691 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86e81a4d90 con 0x5631aad3ead0 2023-07-18T03:14:03.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.755+0000 7f86edb9c700 1 --2- 172.21.15.101:0/2502931691 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5631aad3ead0 0x5631aad42050 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f86e000b0a0 tx=0x7f86e0023040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a5b657e635c7559 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:03.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.755+0000 7f86d0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:14:03.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.755+0000 7f86d0ff9700 1 -- 172.21.15.101:0/2502931691 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e000e040 con 0x5631aad3ead0 2023-07-18T03:14:03.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.755+0000 7f86d0ff9700 1 -- 172.21.15.101:0/2502931691 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f86e002d690 con 0x5631aad3ead0 2023-07-18T03:14:03.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.755+0000 7f8702e33b80 1 -- 172.21.15.101:0/2502931691 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f86e4015070 con 0x5631aad3ead0 2023-07-18T03:14:03.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.756+0000 7f86d0ff9700 1 -- 172.21.15.101:0/2502931691 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86e002d850 con 0x5631aad3ead0 2023-07-18T03:14:03.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.756+0000 7f8702e33b80 10 client.16890 fetch_fsmap learned FSMap version 29 2023-07-18T03:14:03.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.756+0000 7f8702e33b80 1 -- 172.21.15.101:0/2502931691 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5631aaab22b0 con 0x5631aad3ead0 2023-07-18T03:14:03.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.756+0000 7f86d0ff9700 1 -- 172.21.15.101:0/2502931691 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86e4015070 con 0x5631aad3ead0 2023-07-18T03:14:03.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.756+0000 7f86d0ff9700 1 -- 172.21.15.101:0/2502931691 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f86e001fac0 con 0x5631aad3ead0 2023-07-18T03:14:03.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.756+0000 7f8702e33b80 10 client.16890 fetch_fsmap finished waiting for FSMap version 29 2023-07-18T03:14:03.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.756+0000 7f8702e33b80 10 client.16890 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:14:03.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.756+0000 7f8702e33b80 20 client.16890 populate_metadata read hostname 'smithi101' 2023-07-18T03:14:03.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.756+0000 7f8702e33b80 1 --2- 172.21.15.101:0/2502931691 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5631aada3ae0 0x5631aada5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:03.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.756+0000 7f8702e33b80 4 client.16890 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:14:03.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.756+0000 7f8702e33b80 1 -- 172.21.15.101:0/2502931691 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5631aac41050 con 0x5631aada3ae0 2023-07-18T03:14:03.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.756+0000 7f86ed39b700 1 --2- 172.21.15.101:0/2502931691 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5631aada3ae0 0x5631aada5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:03.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.757+0000 7f86ed39b700 1 --2- 172.21.15.101:0/2502931691 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5631aada3ae0 0x5631aada5ef0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efabb0a8230ad08a server_cookie=a212565fbb82898a in_seq=0 out_seq=0 2023-07-18T03:14:03.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.757+0000 7f86d0ff9700 10 client.16890 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:14:03.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.757+0000 7f86d0ff9700 1 -- 172.21.15.101:0/2502931691 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5631aac41050 con 0x5631aada3ae0 2023-07-18T03:14:03.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.757+0000 7f86d0ff9700 10 client.16890 handle_command_reply: tid=0 2023-07-18T03:14:03.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.757+0000 7f8702e33b80 1 client.16890 shutdown 2023-07-18T03:14:03.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.757+0000 7f8702e33b80 1 -- 172.21.15.101:0/2502931691 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5631aad3ead0 msgr2=0x5631aad42050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:03.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.757+0000 7f8702e33b80 1 --2- 172.21.15.101:0/2502931691 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5631aad3ead0 0x5631aad42050 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f86e000b0a0 tx=0x7f86e0023040 comp rx=0 tx=0).stop 2023-07-18T03:14:03.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.757+0000 7f8702e33b80 1 -- 172.21.15.101:0/2502931691 shutdown_connections 2023-07-18T03:14:03.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.757+0000 7f8702e33b80 1 --2- 172.21.15.101:0/2502931691 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5631aada3ae0 0x5631aada5ef0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.757+0000 7f8702e33b80 1 --2- 172.21.15.101:0/2502931691 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5631aad3ead0 0x5631aad42050 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.757+0000 7f8702e33b80 1 --2- 172.21.15.101:0/2502931691 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5631aad332d0 0x5631aad45450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.757+0000 7f86d0ff9700 0 client.16890 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:14:03.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.757+0000 7f8702e33b80 1 --2- 172.21.15.101:0/2502931691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631aad33be0 0x5631aad41910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.757+0000 7f8702e33b80 1 -- 172.21.15.101:0/2502931691 >> 172.21.15.101:0/2502931691 conn(0x5631aad29da0 msgr2=0x5631aad351a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:03.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.758+0000 7f8702e33b80 1 -- 172.21.15.101:0/2502931691 shutdown_connections 2023-07-18T03:14:03.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.758+0000 7f8702e33b80 1 -- 172.21.15.101:0/2502931691 wait complete. 2023-07-18T03:14:03.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.758+0000 7f8702e33b80 20 client.16890 trim_cache size 0 max 16384 2023-07-18T03:14:03.760 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:14:03.760 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-18T03:14:03.760 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:14:03.760 INFO:teuthology.orchestra.run.smithi101.stdout: "c37b945e-a67b-4cc7-9a32-a53345e5f26a": { 2023-07-18T03:14:03.760 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:14:03.760 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "c37b945e-a67b-4cc7-9a32-a53345e5f26a", 2023-07-18T03:14:03.761 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:14:03.761 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:14:03.761 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:14:03.761 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:14:03.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.758+0000 7f86ef5ff700 1 -- 172.21.15.101:0/1143224362 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f86d40a1340 msgr2=0x7f86d40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:03.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.758+0000 7f86ef5ff700 1 --2- 172.21.15.101:0/1143224362 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f86d40a1340 0x7f86d40a37f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f86e8074920 tx=0x7f86e000bca0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.758+0000 7f86ef5ff700 1 -- 172.21.15.101:0/1143224362 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f86e8100490 msgr2=0x7f86e8072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:03.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.758+0000 7f86ef5ff700 1 --2- 172.21.15.101:0/1143224362 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f86e8100490 0x7f86e8072d50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f86d800e9a0 tx=0x7f86d800ecb0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.758+0000 7f86ef5ff700 1 -- 172.21.15.101:0/1143224362 shutdown_connections 2023-07-18T03:14:03.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.758+0000 7f86ef5ff700 1 --2- 172.21.15.101:0/1143224362 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f86d40a1340 0x7f86d40a37f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.758+0000 7f86ef5ff700 1 --2- 172.21.15.101:0/1143224362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86e80ffae0 0x7f86e8072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.758+0000 7f86ef5ff700 1 --2- 172.21.15.101:0/1143224362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86e8105af0 0x7f86e81a3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.758+0000 7f86ef5ff700 1 --2- 172.21.15.101:0/1143224362 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f86e8100490 0x7f86e8072d50 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:03.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.758+0000 7f86ef5ff700 1 -- 172.21.15.101:0/1143224362 >> 172.21.15.101:0/1143224362 conn(0x7f86e80fb5e0 msgr2=0x7f86e8106940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:03.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.758+0000 7f86ef5ff700 1 -- 172.21.15.101:0/1143224362 shutdown_connections 2023-07-18T03:14:03.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:03.758+0000 7f86ef5ff700 1 -- 172.21.15.101:0/1143224362 wait complete. 2023-07-18T03:14:03.769 INFO:tasks.cephfs.filesystem:scrub status for tag:c37b945e-a67b-4cc7-9a32-a53345e5f26a - {'path': '/', 'tag': 'c37b945e-a67b-4cc7-9a32-a53345e5f26a', 'options': 'recursive,force'} 2023-07-18T03:14:04.449 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:04 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:14:04.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:04 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:14:04.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:04 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:14:04.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:04 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:14:04.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:04 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:14:04.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:04 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:14:04.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:04 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:14:04.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:04 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:14:04.545 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:04 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:14:04.545 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:04 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:14:04.545 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:04 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:14:04.545 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:04 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:14:04.586 INFO:tasks.workunit.client.0.smithi101.stdout: 1 496728 8.21 MB/sec execute 209 sec latency 1013.822 ms 2023-07-18T03:14:05.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:05 smithi179 ceph-mon[129229]: scrub queued for path: / 2023-07-18T03:14:05.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:05 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:14:05.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:05 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:14:05.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:05 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:14:05.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:05 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:14:05.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:05 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:14:05.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:05 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:14:05.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:05 smithi179 ceph-mon[129229]: pgmap v198: 129 pgs: 129 active+clean; 320 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 2.5 MiB/s wr, 181 op/s 2023-07-18T03:14:05.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:05 smithi101 ceph-mon[126379]: scrub queued for path: / 2023-07-18T03:14:05.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:05 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:14:05.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:05 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:14:05.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:05 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:14:05.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:05 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:14:05.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:05 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:14:05.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:05 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:14:05.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:05 smithi101 ceph-mon[126379]: pgmap v198: 129 pgs: 129 active+clean; 320 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 2.5 MiB/s wr, 181 op/s 2023-07-18T03:14:05.545 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:05 smithi155 ceph-mon[128885]: scrub queued for path: / 2023-07-18T03:14:05.545 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:05 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:14:05.545 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:05 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:14:05.545 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:05 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:14:05.546 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:05 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:14:05.546 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:05 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:14:05.546 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:05 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:14:05.546 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:05 smithi155 ceph-mon[128885]: pgmap v198: 129 pgs: 129 active+clean; 320 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 2.5 MiB/s wr, 181 op/s 2023-07-18T03:14:05.586 INFO:tasks.workunit.client.0.smithi101.stdout: 1 496728 8.18 MB/sec execute 210 sec latency 2013.897 ms 2023-07-18T03:14:06.586 INFO:tasks.workunit.client.0.smithi101.stdout: 1 496860 8.14 MB/sec execute 211 sec latency 2184.912 ms 2023-07-18T03:14:07.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:07 smithi179 ceph-mon[129229]: pgmap v199: 129 pgs: 129 active+clean; 342 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.0 MiB/s wr, 268 op/s 2023-07-18T03:14:07.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:07 smithi179 ceph-mon[129229]: osdmap e93: 12 total, 12 up, 12 in 2023-07-18T03:14:07.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:07 smithi101 ceph-mon[126379]: pgmap v199: 129 pgs: 129 active+clean; 342 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.0 MiB/s wr, 268 op/s 2023-07-18T03:14:07.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:07 smithi101 ceph-mon[126379]: osdmap e93: 12 total, 12 up, 12 in 2023-07-18T03:14:07.545 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:07 smithi155 ceph-mon[128885]: pgmap v199: 129 pgs: 129 active+clean; 342 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.0 MiB/s wr, 268 op/s 2023-07-18T03:14:07.545 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:07 smithi155 ceph-mon[128885]: osdmap e93: 12 total, 12 up, 12 in 2023-07-18T03:14:07.586 INFO:tasks.workunit.client.0.smithi101.stdout: 1 496860 8.10 MB/sec execute 212 sec latency 1695.557 ms 2023-07-18T03:14:08.587 INFO:tasks.workunit.client.0.smithi101.stdout: 1 496860 8.07 MB/sec execute 213 sec latency 2695.687 ms 2023-07-18T03:14:09.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:09 smithi179 ceph-mon[129229]: pgmap v201: 129 pgs: 129 active+clean; 342 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.4 MiB/s wr, 212 op/s 2023-07-18T03:14:09.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:09 smithi101 ceph-mon[126379]: pgmap v201: 129 pgs: 129 active+clean; 342 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.4 MiB/s wr, 212 op/s 2023-07-18T03:14:09.545 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:09 smithi155 ceph-mon[128885]: pgmap v201: 129 pgs: 129 active+clean; 342 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.4 MiB/s wr, 212 op/s 2023-07-18T03:14:09.587 INFO:tasks.workunit.client.0.smithi101.stdout: 1 496860 8.03 MB/sec execute 214 sec latency 3695.820 ms 2023-07-18T03:14:10.587 INFO:tasks.workunit.client.0.smithi101.stdout: 1 496860 7.99 MB/sec execute 215 sec latency 4695.968 ms 2023-07-18T03:14:11.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:11 smithi179 ceph-mon[129229]: pgmap v202: 129 pgs: 129 active+clean; 343 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 3.1 MiB/s wr, 116 op/s 2023-07-18T03:14:11.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:11 smithi101 ceph-mon[126379]: pgmap v202: 129 pgs: 129 active+clean; 343 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 3.1 MiB/s wr, 116 op/s 2023-07-18T03:14:11.545 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:11 smithi155 ceph-mon[128885]: pgmap v202: 129 pgs: 129 active+clean; 343 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 3.1 MiB/s wr, 116 op/s 2023-07-18T03:14:11.587 INFO:tasks.workunit.client.0.smithi101.stdout: 1 496872 7.95 MB/sec execute 216 sec latency 4871.136 ms 2023-07-18T03:14:12.587 INFO:tasks.workunit.client.0.smithi101.stdout: 1 496872 7.92 MB/sec execute 217 sec latency 1808.908 ms 2023-07-18T03:14:13.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:13 smithi179 ceph-mon[129229]: pgmap v203: 129 pgs: 129 active+clean; 343 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 3.1 MiB/s wr, 116 op/s 2023-07-18T03:14:13.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:13 smithi101 ceph-mon[126379]: pgmap v203: 129 pgs: 129 active+clean; 343 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 3.1 MiB/s wr, 116 op/s 2023-07-18T03:14:13.545 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:13 smithi155 ceph-mon[128885]: pgmap v203: 129 pgs: 129 active+clean; 343 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 3.1 MiB/s wr, 116 op/s 2023-07-18T03:14:13.587 INFO:tasks.workunit.client.0.smithi101.stdout: 1 496872 7.88 MB/sec execute 218 sec latency 2809.009 ms 2023-07-18T03:14:14.587 INFO:tasks.workunit.client.0.smithi101.stdout: 1 496872 7.85 MB/sec execute 219 sec latency 3809.105 ms 2023-07-18T03:14:15.200 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:14 smithi179 ceph-mon[129229]: pgmap v204: 129 pgs: 129 active+clean; 343 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 3.1 MiB/s wr, 116 op/s 2023-07-18T03:14:15.200 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:14 smithi179 ceph-mon[129229]: osdmap e94: 12 total, 12 up, 12 in 2023-07-18T03:14:15.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:14 smithi101 ceph-mon[126379]: pgmap v204: 129 pgs: 129 active+clean; 343 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 3.1 MiB/s wr, 116 op/s 2023-07-18T03:14:15.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:14 smithi101 ceph-mon[126379]: osdmap e94: 12 total, 12 up, 12 in 2023-07-18T03:14:15.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:14 smithi155 ceph-mon[128885]: pgmap v204: 129 pgs: 129 active+clean; 343 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 3.1 MiB/s wr, 116 op/s 2023-07-18T03:14:15.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:14 smithi155 ceph-mon[128885]: osdmap e94: 12 total, 12 up, 12 in 2023-07-18T03:14:15.587 INFO:tasks.workunit.client.0.smithi101.stdout: 1 497534 7.83 MB/sec execute 220 sec latency 4515.811 ms 2023-07-18T03:14:16.200 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:15 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:14:16.200 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:15 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:14:16.200 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:15 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:14:16.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:15 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:14:16.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:15 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:14:16.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:15 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:14:16.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:15 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:14:16.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:15 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:14:16.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:15 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:14:16.588 INFO:tasks.workunit.client.0.smithi101.stdout: 1 497723 7.80 MB/sec execute 221 sec latency 632.693 ms 2023-07-18T03:14:17.200 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:16 smithi179 ceph-mon[129229]: pgmap v206: 129 pgs: 129 active+clean; 344 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 386 KiB/s wr, 26 op/s 2023-07-18T03:14:17.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:16 smithi101 ceph-mon[126379]: pgmap v206: 129 pgs: 129 active+clean; 344 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 386 KiB/s wr, 26 op/s 2023-07-18T03:14:17.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:16 smithi155 ceph-mon[128885]: pgmap v206: 129 pgs: 129 active+clean; 344 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 386 KiB/s wr, 26 op/s 2023-07-18T03:14:17.588 INFO:tasks.workunit.client.0.smithi101.stdout: 1 497723 7.77 MB/sec execute 222 sec latency 1632.771 ms 2023-07-18T03:14:18.588 INFO:tasks.workunit.client.0.smithi101.stdout: 1 497723 7.73 MB/sec execute 223 sec latency 2632.917 ms 2023-07-18T03:14:19.200 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:18 smithi179 ceph-mon[129229]: pgmap v207: 129 pgs: 129 active+clean; 344 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 365 KiB/s wr, 24 op/s 2023-07-18T03:14:19.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:18 smithi101 ceph-mon[126379]: pgmap v207: 129 pgs: 129 active+clean; 344 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 365 KiB/s wr, 24 op/s 2023-07-18T03:14:19.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:18 smithi155 ceph-mon[128885]: pgmap v207: 129 pgs: 129 active+clean; 344 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 365 KiB/s wr, 24 op/s 2023-07-18T03:14:19.588 INFO:tasks.workunit.client.0.smithi101.stdout: 1 497723 7.70 MB/sec execute 224 sec latency 3633.009 ms 2023-07-18T03:14:20.588 INFO:tasks.workunit.client.0.smithi101.stdout: 1 497723 7.66 MB/sec execute 225 sec latency 4633.137 ms 2023-07-18T03:14:21.200 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:20 smithi179 ceph-mon[129229]: pgmap v208: 129 pgs: 129 active+clean; 347 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 1.0 MiB/s wr, 50 op/s 2023-07-18T03:14:21.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:20 smithi101 ceph-mon[126379]: pgmap v208: 129 pgs: 129 active+clean; 347 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 1.0 MiB/s wr, 50 op/s 2023-07-18T03:14:21.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:20 smithi155 ceph-mon[128885]: pgmap v208: 129 pgs: 129 active+clean; 347 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 1.0 MiB/s wr, 50 op/s 2023-07-18T03:14:21.588 INFO:tasks.workunit.client.0.smithi101.stdout: 1 497783 7.63 MB/sec execute 226 sec latency 4802.069 ms 2023-07-18T03:14:22.588 INFO:tasks.workunit.client.0.smithi101.stdout: 1 497783 7.60 MB/sec execute 227 sec latency 1752.479 ms 2023-07-18T03:14:23.200 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:22 smithi179 ceph-mon[129229]: pgmap v209: 129 pgs: 129 active+clean; 347 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 1.0 MiB/s wr, 50 op/s 2023-07-18T03:14:23.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:22 smithi101 ceph-mon[126379]: pgmap v209: 129 pgs: 129 active+clean; 347 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 1.0 MiB/s wr, 50 op/s 2023-07-18T03:14:23.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:22 smithi155 ceph-mon[128885]: pgmap v209: 129 pgs: 129 active+clean; 347 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 1.0 MiB/s wr, 50 op/s 2023-07-18T03:14:23.588 INFO:tasks.workunit.client.0.smithi101.stdout: 1 497783 7.56 MB/sec execute 228 sec latency 2752.590 ms 2023-07-18T03:14:24.589 INFO:tasks.workunit.client.0.smithi101.stdout: 1 497783 7.53 MB/sec execute 229 sec latency 3752.658 ms 2023-07-18T03:14:25.200 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:24 smithi179 ceph-mon[129229]: pgmap v210: 129 pgs: 129 active+clean; 347 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 1.0 MiB/s wr, 50 op/s 2023-07-18T03:14:25.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:24 smithi101 ceph-mon[126379]: pgmap v210: 129 pgs: 129 active+clean; 347 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 1.0 MiB/s wr, 50 op/s 2023-07-18T03:14:25.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:24 smithi155 ceph-mon[128885]: pgmap v210: 129 pgs: 129 active+clean; 347 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 1.0 MiB/s wr, 50 op/s 2023-07-18T03:14:25.589 INFO:tasks.workunit.client.0.smithi101.stdout: 1 497783 7.50 MB/sec execute 230 sec latency 4752.788 ms 2023-07-18T03:14:26.200 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:25 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:14:26.200 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:25 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:14:26.200 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:25 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:14:26.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:25 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:14:26.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:25 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:14:26.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:25 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:14:26.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:25 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:14:26.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:25 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:14:26.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:25 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:14:26.589 INFO:tasks.workunit.client.0.smithi101.stdout: 1 498826 7.49 MB/sec execute 231 sec latency 4926.030 ms 2023-07-18T03:14:27.199 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:26 smithi179 ceph-mon[129229]: pgmap v211: 129 pgs: 129 active+clean; 347 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 1.5 MiB/s wr, 71 op/s 2023-07-18T03:14:27.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:26 smithi101 ceph-mon[126379]: pgmap v211: 129 pgs: 129 active+clean; 347 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 1.5 MiB/s wr, 71 op/s 2023-07-18T03:14:27.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:26 smithi155 ceph-mon[128885]: pgmap v211: 129 pgs: 129 active+clean; 347 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 1.5 MiB/s wr, 71 op/s 2023-07-18T03:14:27.589 INFO:tasks.workunit.client.0.smithi101.stdout: 1 499092 7.46 MB/sec execute 232 sec latency 564.786 ms 2023-07-18T03:14:28.589 INFO:tasks.workunit.client.0.smithi101.stdout: 1 499092 7.42 MB/sec execute 233 sec latency 1564.847 ms 2023-07-18T03:14:29.200 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:28 smithi179 ceph-mon[129229]: pgmap v212: 129 pgs: 129 active+clean; 347 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 1.3 MiB/s wr, 58 op/s 2023-07-18T03:14:29.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:28 smithi101 ceph-mon[126379]: pgmap v212: 129 pgs: 129 active+clean; 347 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 1.3 MiB/s wr, 58 op/s 2023-07-18T03:14:29.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:28 smithi155 ceph-mon[128885]: pgmap v212: 129 pgs: 129 active+clean; 347 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 1.3 MiB/s wr, 58 op/s 2023-07-18T03:14:29.589 INFO:tasks.workunit.client.0.smithi101.stdout: 1 499209 7.39 MB/sec execute 234 sec latency 2430.749 ms 2023-07-18T03:14:30.589 INFO:tasks.workunit.client.0.smithi101.stdout: 1 500926 7.41 MB/sec execute 235 sec latency 21.325 ms 2023-07-18T03:14:31.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:30 smithi101 ceph-mon[126379]: pgmap v213: 129 pgs: 129 active+clean; 350 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 2.2 MiB/s wr, 106 op/s 2023-07-18T03:14:31.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:30 smithi155 ceph-mon[128885]: pgmap v213: 129 pgs: 129 active+clean; 350 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 2.2 MiB/s wr, 106 op/s 2023-07-18T03:14:31.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:30 smithi179 ceph-mon[129229]: pgmap v213: 129 pgs: 129 active+clean; 350 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 2.2 MiB/s wr, 106 op/s 2023-07-18T03:14:31.589 INFO:tasks.workunit.client.0.smithi101.stdout: 1 502000 7.40 MB/sec execute 236 sec latency 15.467 ms 2023-07-18T03:14:32.589 INFO:tasks.workunit.client.0.smithi101.stdout: 1 502633 7.38 MB/sec execute 237 sec latency 410.880 ms 2023-07-18T03:14:33.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:32 smithi101 ceph-mon[126379]: pgmap v214: 129 pgs: 129 active+clean; 350 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 1.6 MiB/s wr, 75 op/s 2023-07-18T03:14:33.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:32 smithi155 ceph-mon[128885]: pgmap v214: 129 pgs: 129 active+clean; 350 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 1.6 MiB/s wr, 75 op/s 2023-07-18T03:14:33.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:32 smithi179 ceph-mon[129229]: pgmap v214: 129 pgs: 129 active+clean; 350 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 1.6 MiB/s wr, 75 op/s 2023-07-18T03:14:33.589 INFO:tasks.workunit.client.0.smithi101.stdout: 1 503865 7.38 MB/sec execute 238 sec latency 807.393 ms 2023-07-18T03:14:33.771 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:14:33.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:33.994+0000 7fbe429ff700 1 -- 172.21.15.101:0/3161562441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe3c105af0 msgr2=0x7fbe3c105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:33.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:33.994+0000 7fbe429ff700 1 --2- 172.21.15.101:0/3161562441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe3c105af0 0x7fbe3c105ec0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c009990 tx=0x7fbe2c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:14:33.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:33.994+0000 7fbe429ff700 1 -- 172.21.15.101:0/3161562441 shutdown_connections 2023-07-18T03:14:33.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:33.994+0000 7fbe429ff700 1 --2- 172.21.15.101:0/3161562441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe3c105af0 0x7fbe3c105ec0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:33.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:33.994+0000 7fbe429ff700 1 --2- 172.21.15.101:0/3161562441 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbe3c100490 0x7fbe3c10c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:33.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:33.994+0000 7fbe429ff700 1 --2- 172.21.15.101:0/3161562441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe3c0ffae0 0x7fbe3c0fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:33.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:33.994+0000 7fbe429ff700 1 -- 172.21.15.101:0/3161562441 >> 172.21.15.101:0/3161562441 conn(0x7fbe3c0fb5e0 msgr2=0x7fbe3c0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:33.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:33.994+0000 7fbe429ff700 1 -- 172.21.15.101:0/3161562441 shutdown_connections 2023-07-18T03:14:33.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:33.994+0000 7fbe429ff700 1 -- 172.21.15.101:0/3161562441 wait complete. 2023-07-18T03:14:33.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:33.995+0000 7fbe429ff700 1 Processor -- start 2023-07-18T03:14:33.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:33.995+0000 7fbe429ff700 1 -- start start 2023-07-18T03:14:33.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:33.996+0000 7fbe429ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbe3c0ffae0 0x7fbe3c19ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:33.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:33.996+0000 7fbe429ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe3c100490 0x7fbe3c19bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:33.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:33.996+0000 7fbe429ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe3c105af0 0x7fbe3c195a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:33.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:33.996+0000 7fbe429ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c1961d0 con 0x7fbe3c0ffae0 2023-07-18T03:14:33.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:33.996+0000 7fbe429ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c196340 con 0x7fbe3c100490 2023-07-18T03:14:33.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:33.996+0000 7fbe429ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c196500 con 0x7fbe3c105af0 2023-07-18T03:14:33.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:33.996+0000 7fbe3bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbe3c0ffae0 0x7fbe3c19ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:33.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:33.996+0000 7fbe40f9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe3c105af0 0x7fbe3c195a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:33.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:33.996+0000 7fbe3bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbe3c0ffae0 0x7fbe3c19ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:58802/0 (socket says 172.21.15.101:58802) 2023-07-18T03:14:33.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:33.996+0000 7fbe40f9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe3c105af0 0x7fbe3c195a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:35190/0 (socket says 172.21.15.101:35190) 2023-07-18T03:14:33.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:33.996+0000 7fbe3bfff700 1 -- 172.21.15.101:0/3446077241 learned_addr learned my addr 172.21.15.101:0/3446077241 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:33.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:33.996+0000 7fbe3b7fe700 1 --2- 172.21.15.101:0/3446077241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe3c100490 0x7fbe3c19bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:33.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:33.997+0000 7fbe3bfff700 1 -- 172.21.15.101:0/3446077241 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe3c105af0 msgr2=0x7fbe3c195a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:33.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:33.997+0000 7fbe3bfff700 1 --2- 172.21.15.101:0/3446077241 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe3c105af0 0x7fbe3c195a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:33.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:33.997+0000 7fbe3bfff700 1 -- 172.21.15.101:0/3446077241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe3c100490 msgr2=0x7fbe3c19bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:33.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:33.997+0000 7fbe3bfff700 1 --2- 172.21.15.101:0/3446077241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe3c100490 0x7fbe3c19bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:33.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:33.997+0000 7fbe3bfff700 1 -- 172.21.15.101:0/3446077241 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe2c0095f0 con 0x7fbe3c0ffae0 2023-07-18T03:14:33.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:33.997+0000 7fbe3b7fe700 1 --2- 172.21.15.101:0/3446077241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe3c100490 0x7fbe3c19bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:14:33.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:33.997+0000 7fbe40f9c700 1 --2- 172.21.15.101:0/3446077241 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe3c105af0 0x7fbe3c195a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:14:33.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:33.997+0000 7fbe3bfff700 1 --2- 172.21.15.101:0/3446077241 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbe3c0ffae0 0x7fbe3c19ba20 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c005910 tx=0x7fbe2c005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:33.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:33.997+0000 7fbe397fa700 1 -- 172.21.15.101:0/3446077241 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c01d070 con 0x7fbe3c0ffae0 2023-07-18T03:14:33.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:33.997+0000 7fbe429ff700 1 -- 172.21.15.101:0/3446077241 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe3c196780 con 0x7fbe3c0ffae0 2023-07-18T03:14:33.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:33.997+0000 7fbe397fa700 1 -- 172.21.15.101:0/3446077241 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe2c022470 con 0x7fbe3c0ffae0 2023-07-18T03:14:33.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:33.997+0000 7fbe397fa700 1 -- 172.21.15.101:0/3446077241 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c017650 con 0x7fbe3c0ffae0 2023-07-18T03:14:33.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:33.997+0000 7fbe429ff700 1 -- 172.21.15.101:0/3446077241 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe3c072920 con 0x7fbe3c0ffae0 2023-07-18T03:14:33.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.000+0000 7fbe397fa700 1 -- 172.21.15.101:0/3446077241 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fbe2c0225e0 con 0x7fbe3c0ffae0 2023-07-18T03:14:33.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.002+0000 7fbe56360b80 1 Processor -- start 2023-07-18T03:14:33.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.002+0000 7fbe56360b80 1 -- start start 2023-07-18T03:14:33.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.003+0000 7fbe397fa700 1 --2- 172.21.15.101:0/3446077241 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fbe240a12c0 0x7fbe240a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:33.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.003+0000 7fbe56360b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605f18552d0 0x5605f18556a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:33.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.003+0000 7fbe56360b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605f1855be0 0x5605f1860590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:33.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.003+0000 7fbe56360b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605f1860ad0 0x5605f1862eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:33.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.003+0000 7fbe56360b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f175bcb0 con 0x5605f18552d0 2023-07-18T03:14:33.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.003+0000 7fbe56360b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f17aea40 con 0x5605f1855be0 2023-07-18T03:14:33.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.003+0000 7fbe56360b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f17a7ed0 con 0x5605f1860ad0 2023-07-18T03:14:33.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.003+0000 7fbe397fa700 1 -- 172.21.15.101:0/3446077241 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fbe2c0ce3b0 con 0x7fbe3c0ffae0 2023-07-18T03:14:33.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.003+0000 7fbe3bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605f18552d0 0x5605f18556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:33.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.003+0000 7fbe3b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605f1855be0 0x5605f1860590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:33.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.003+0000 7fbe3b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605f1855be0 0x5605f1860590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47834/0 (socket says 172.21.15.101:47834) 2023-07-18T03:14:33.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.003+0000 7fbe3b7fe700 1 -- 172.21.15.101:0/629983077 learned_addr learned my addr 172.21.15.101:0/629983077 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:33.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.003+0000 7fbe3bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605f18552d0 0x5605f18556a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:58816/0 (socket says 172.21.15.101:58816) 2023-07-18T03:14:33.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.003+0000 7fbe40f9c700 1 --2- 172.21.15.101:0/3446077241 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fbe240a12c0 0x7fbe240a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:33.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.003+0000 7fbe40f9c700 1 --2- 172.21.15.101:0/629983077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605f1860ad0 0x5605f1862eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:33.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.004+0000 7fbe3b7fe700 1 -- 172.21.15.101:0/629983077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605f1860ad0 msgr2=0x5605f1862eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:33.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.004+0000 7fbe3b7fe700 1 --2- 172.21.15.101:0/629983077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605f1860ad0 0x5605f1862eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:33.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.004+0000 7fbe3b7fe700 1 -- 172.21.15.101:0/629983077 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605f18552d0 msgr2=0x5605f18556a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:33.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.004+0000 7fbe3b7fe700 1 --2- 172.21.15.101:0/629983077 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605f18552d0 0x5605f18556a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:33.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.004+0000 7fbe3b7fe700 1 -- 172.21.15.101:0/629983077 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605f15d5de0 con 0x5605f1855be0 2023-07-18T03:14:33.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.004+0000 7fbe3bfff700 1 --2- 172.21.15.101:0/629983077 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605f18552d0 0x5605f18556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:14:34.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.004+0000 7fbe3b7fe700 1 --2- 172.21.15.101:0/629983077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605f1855be0 0x5605f1860590 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbe30011540 tx=0x7fbe30011850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=592c1a4cd8366353 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:34.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.004+0000 7fbe40f9c700 1 --2- 172.21.15.101:0/3446077241 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fbe240a12c0 0x7fbe240a3770 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbe3400c090 tx=0x7fbe34009920 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:34.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.004+0000 7fbe227fc700 1 -- 172.21.15.101:0/629983077 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe30004a30 con 0x5605f1855be0 2023-07-18T03:14:34.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.004+0000 7fbe227fc700 1 -- 172.21.15.101:0/629983077 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe300054e0 con 0x5605f1855be0 2023-07-18T03:14:34.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.004+0000 7fbe227fc700 1 -- 172.21.15.101:0/629983077 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3000e910 con 0x5605f1855be0 2023-07-18T03:14:34.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.005+0000 7fbe56360b80 1 -- 172.21.15.101:0/629983077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605f1855be0 msgr2=0x5605f1860590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:34.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.005+0000 7fbe56360b80 1 --2- 172.21.15.101:0/629983077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605f1855be0 0x5605f1860590 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbe30011540 tx=0x7fbe30011850 comp rx=0 tx=0).stop 2023-07-18T03:14:34.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.005+0000 7fbe56360b80 1 -- 172.21.15.101:0/629983077 shutdown_connections 2023-07-18T03:14:34.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.005+0000 7fbe56360b80 1 --2- 172.21.15.101:0/629983077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605f1860ad0 0x5605f1862eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.005+0000 7fbe56360b80 1 --2- 172.21.15.101:0/629983077 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605f18552d0 0x5605f18556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.005+0000 7fbe56360b80 1 --2- 172.21.15.101:0/629983077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605f1855be0 0x5605f1860590 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.005+0000 7fbe56360b80 1 -- 172.21.15.101:0/629983077 >> 172.21.15.101:0/629983077 conn(0x5605f184bda0 msgr2=0x5605f18516a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:34.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.005+0000 7fbe56360b80 1 -- 172.21.15.101:0/629983077 shutdown_connections 2023-07-18T03:14:34.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.005+0000 7fbe56360b80 1 -- 172.21.15.101:0/629983077 wait complete. 2023-07-18T03:14:34.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.006+0000 7fbe56360b80 1 -- start start 2023-07-18T03:14:34.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.006+0000 7fbe56360b80 1 Processor -- start 2023-07-18T03:14:34.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.006+0000 7fbe56360b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605f18552d0 0x5605f1860540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:34.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.006+0000 7fbe56360b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605f1855be0 0x5605f185d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:34.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.006+0000 7fbe56360b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605f1860ad0 0x5605f185deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:34.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.006+0000 7fbe56360b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f1747ac0 con 0x5605f1855be0 2023-07-18T03:14:34.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.006+0000 7fbe56360b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f17477c0 con 0x5605f18552d0 2023-07-18T03:14:34.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.006+0000 7fbe56360b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f162b920 con 0x5605f1860ad0 2023-07-18T03:14:34.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.007+0000 7fbe56360b80 10 client.0 _ll_register_callbacks cb 0x5605f1745760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:14:34.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.007+0000 7fbe40f9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605f1860ad0 0x5605f185deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:34.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.007+0000 7fbe40f9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605f1860ad0 0x5605f185deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:35202/0 (socket says 172.21.15.101:35202) 2023-07-18T03:14:34.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.007+0000 7fbe40f9c700 1 -- 172.21.15.101:0/2905769701 learned_addr learned my addr 172.21.15.101:0/2905769701 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:34.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.007+0000 7fbe3b7fe700 1 --2- 172.21.15.101:0/2905769701 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605f1855be0 0x5605f185d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:34.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.007+0000 7fbe3bfff700 1 --2- 172.21.15.101:0/2905769701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605f18552d0 0x5605f1860540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:34.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.007+0000 7fbe40f9c700 1 -- 172.21.15.101:0/2905769701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605f18552d0 msgr2=0x5605f1860540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:34.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.007+0000 7fbe40f9c700 1 --2- 172.21.15.101:0/2905769701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605f18552d0 0x5605f1860540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.007+0000 7fbe40f9c700 1 -- 172.21.15.101:0/2905769701 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605f1855be0 msgr2=0x5605f185d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:34.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.007+0000 7fbe40f9c700 1 --2- 172.21.15.101:0/2905769701 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605f1855be0 0x5605f185d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.007+0000 7fbe40f9c700 1 -- 172.21.15.101:0/2905769701 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe3c072920 con 0x5605f1860ad0 2023-07-18T03:14:34.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.008+0000 7fbe40f9c700 1 --2- 172.21.15.101:0/2905769701 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605f1860ad0 0x5605f185deb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fbe34020660 tx=0x7fbe34004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4090500682abbe34 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:34.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.008+0000 7fbe1b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:14:34.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.008+0000 7fbe1b7fe700 1 -- 172.21.15.101:0/2905769701 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3402a3f0 con 0x5605f1860ad0 2023-07-18T03:14:34.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.008+0000 7fbe56360b80 1 -- 172.21.15.101:0/2905769701 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5605f17014f0 con 0x5605f1860ad0 2023-07-18T03:14:34.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.008+0000 7fbe1b7fe700 1 -- 172.21.15.101:0/2905769701 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fbe3402a550 con 0x5605f1860ad0 2023-07-18T03:14:34.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.008+0000 7fbe1b7fe700 1 -- 172.21.15.101:0/2905769701 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe3402a710 con 0x5605f1860ad0 2023-07-18T03:14:34.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.008+0000 7fbe56360b80 10 client.26983 fetch_fsmap learned FSMap version 29 2023-07-18T03:14:34.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.008+0000 7fbe56360b80 1 -- 172.21.15.101:0/2905769701 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fbe30009d10 con 0x5605f1860ad0 2023-07-18T03:14:34.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.009+0000 7fbe1b7fe700 1 -- 172.21.15.101:0/2905769701 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5605f17014f0 con 0x5605f1860ad0 2023-07-18T03:14:34.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.009+0000 7fbe1b7fe700 1 -- 172.21.15.101:0/2905769701 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fbe34029070 con 0x5605f1860ad0 2023-07-18T03:14:34.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.009+0000 7fbe56360b80 10 client.26983 fetch_fsmap finished waiting for FSMap version 29 2023-07-18T03:14:34.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.009+0000 7fbe56360b80 10 client.26983 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:14:34.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.009+0000 7fbe56360b80 20 client.26983 populate_metadata read hostname 'smithi101' 2023-07-18T03:14:34.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.009+0000 7fbe56360b80 1 --2- 172.21.15.101:0/2905769701 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5605f1870440 0x5605f1872850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:34.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.009+0000 7fbe56360b80 4 client.26983 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:14:34.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.009+0000 7fbe56360b80 1 -- 172.21.15.101:0/2905769701 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5605f17a0eb0 con 0x5605f1870440 2023-07-18T03:14:34.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.009+0000 7fbe3bfff700 1 --2- 172.21.15.101:0/2905769701 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5605f1870440 0x5605f1872850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:34.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.010+0000 7fbe1b7fe700 10 client.26983 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:14:34.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.010+0000 7fbe3bfff700 1 --2- 172.21.15.101:0/2905769701 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5605f1870440 0x5605f1872850 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83e315c2d3c9248c server_cookie=b6753046b59711df in_seq=0 out_seq=0 2023-07-18T03:14:34.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.010+0000 7fbe1b7fe700 1 -- 172.21.15.101:0/2905769701 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5605f17a0eb0 con 0x5605f1870440 2023-07-18T03:14:34.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.010+0000 7fbe1b7fe700 10 client.26983 handle_command_reply: tid=0 2023-07-18T03:14:34.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.011+0000 7fbe56360b80 1 client.26983 shutdown 2023-07-18T03:14:34.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.011+0000 7fbe56360b80 1 -- 172.21.15.101:0/2905769701 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605f1860ad0 msgr2=0x5605f185deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:34.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.011+0000 7fbe56360b80 1 --2- 172.21.15.101:0/2905769701 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605f1860ad0 0x5605f185deb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fbe34020660 tx=0x7fbe34004b40 comp rx=0 tx=0).stop 2023-07-18T03:14:34.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.011+0000 7fbe56360b80 1 -- 172.21.15.101:0/2905769701 shutdown_connections 2023-07-18T03:14:34.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.011+0000 7fbe56360b80 1 --2- 172.21.15.101:0/2905769701 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5605f1870440 0x5605f1872850 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.011+0000 7fbe56360b80 1 --2- 172.21.15.101:0/2905769701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605f18552d0 0x5605f1860540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.011+0000 7fbe56360b80 1 --2- 172.21.15.101:0/2905769701 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605f1860ad0 0x5605f185deb0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.011+0000 7fbe56360b80 1 --2- 172.21.15.101:0/2905769701 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605f1855be0 0x5605f185d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.011+0000 7fbe1b7fe700 0 client.26983 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:14:34.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.011+0000 7fbe56360b80 1 -- 172.21.15.101:0/2905769701 >> 172.21.15.101:0/2905769701 conn(0x5605f184bda0 msgr2=0x5605f1851230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:34.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.011+0000 7fbe56360b80 1 -- 172.21.15.101:0/2905769701 shutdown_connections 2023-07-18T03:14:34.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.011+0000 7fbe56360b80 1 -- 172.21.15.101:0/2905769701 wait complete. 2023-07-18T03:14:34.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.011+0000 7fbe56360b80 20 client.26983 trim_cache size 0 max 16384 2023-07-18T03:14:34.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.030+0000 7fbe56360b80 1 Processor -- start 2023-07-18T03:14:34.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.030+0000 7fbe56360b80 1 -- start start 2023-07-18T03:14:34.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.030+0000 7fbe56360b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605f18552d0 0x5605f18582d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:34.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.030+0000 7fbe56360b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605f1855be0 0x5605f1858b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:34.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.030+0000 7fbe56360b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605f1860ad0 0x5605f1859070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:34.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.030+0000 7fbe56360b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f186cdd0 con 0x5605f18552d0 2023-07-18T03:14:34.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.030+0000 7fbe56360b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f1856160 con 0x5605f1860ad0 2023-07-18T03:14:34.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.030+0000 7fbe56360b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f1860880 con 0x5605f1855be0 2023-07-18T03:14:34.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.031+0000 7fbe40f9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605f1860ad0 0x5605f1859070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:34.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.031+0000 7fbe3b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605f1855be0 0x5605f1858b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:34.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.031+0000 7fbe40f9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605f1860ad0 0x5605f1859070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47848/0 (socket says 172.21.15.101:47848) 2023-07-18T03:14:34.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.031+0000 7fbe40f9c700 1 -- 172.21.15.101:0/271117838 learned_addr learned my addr 172.21.15.101:0/271117838 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:34.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.031+0000 7fbe3bfff700 1 --2- 172.21.15.101:0/271117838 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605f18552d0 0x5605f18582d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:34.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.031+0000 7fbe40f9c700 1 -- 172.21.15.101:0/271117838 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605f1855be0 msgr2=0x5605f1858b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:34.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.031+0000 7fbe40f9c700 1 --2- 172.21.15.101:0/271117838 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605f1855be0 0x5605f1858b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.031+0000 7fbe40f9c700 1 -- 172.21.15.101:0/271117838 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605f18552d0 msgr2=0x5605f18582d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:34.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.031+0000 7fbe40f9c700 1 --2- 172.21.15.101:0/271117838 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605f18552d0 0x5605f18582d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.031+0000 7fbe40f9c700 1 -- 172.21.15.101:0/271117838 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605f15bec60 con 0x5605f1860ad0 2023-07-18T03:14:34.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.031+0000 7fbe40f9c700 1 --2- 172.21.15.101:0/271117838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605f1860ad0 0x5605f1859070 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbe3401a040 tx=0x7fbe34008fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f13bca3cd3c97637 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:34.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.031+0000 7fbe22ffd700 1 -- 172.21.15.101:0/271117838 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe34040070 con 0x5605f1860ad0 2023-07-18T03:14:34.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.032+0000 7fbe22ffd700 1 -- 172.21.15.101:0/271117838 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe34026040 con 0x5605f1860ad0 2023-07-18T03:14:34.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.032+0000 7fbe22ffd700 1 -- 172.21.15.101:0/271117838 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3403ca90 con 0x5605f1860ad0 2023-07-18T03:14:34.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.032+0000 7fbe56360b80 1 -- 172.21.15.101:0/271117838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605f1860ad0 msgr2=0x5605f1859070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:34.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.032+0000 7fbe56360b80 1 --2- 172.21.15.101:0/271117838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605f1860ad0 0x5605f1859070 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbe3401a040 tx=0x7fbe34008fc0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.032+0000 7fbe56360b80 1 -- 172.21.15.101:0/271117838 shutdown_connections 2023-07-18T03:14:34.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.032+0000 7fbe56360b80 1 --2- 172.21.15.101:0/271117838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605f1860ad0 0x5605f1859070 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.032+0000 7fbe56360b80 1 --2- 172.21.15.101:0/271117838 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605f18552d0 0x5605f18582d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.032+0000 7fbe56360b80 1 --2- 172.21.15.101:0/271117838 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605f1855be0 0x5605f1858b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.032+0000 7fbe56360b80 1 -- 172.21.15.101:0/271117838 >> 172.21.15.101:0/271117838 conn(0x5605f184bda0 msgr2=0x5605f1860170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:34.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.032+0000 7fbe56360b80 1 -- 172.21.15.101:0/271117838 shutdown_connections 2023-07-18T03:14:34.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.032+0000 7fbe56360b80 1 -- 172.21.15.101:0/271117838 wait complete. 2023-07-18T03:14:34.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.033+0000 7fbe56360b80 1 -- start start 2023-07-18T03:14:34.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.033+0000 7fbe56360b80 1 Processor -- start 2023-07-18T03:14:34.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.034+0000 7fbe56360b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605f18552d0 0x5605f18675d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:34.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.034+0000 7fbe56360b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605f1855be0 0x5605f1865af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:34.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.034+0000 7fbe56360b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605f1860ad0 0x5605f1866090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:34.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.034+0000 7fbe56360b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f18667a0 con 0x5605f1855be0 2023-07-18T03:14:34.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.034+0000 7fbe56360b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f1866910 con 0x5605f1860ad0 2023-07-18T03:14:34.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.034+0000 7fbe56360b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f18c4310 con 0x5605f18552d0 2023-07-18T03:14:34.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.034+0000 7fbe56360b80 10 client.0 _ll_register_callbacks cb 0x7fbe34008f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:14:34.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.034+0000 7fbe3bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605f18552d0 0x5605f18675d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:34.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.034+0000 7fbe3bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605f18552d0 0x5605f18675d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:35224/0 (socket says 172.21.15.101:35224) 2023-07-18T03:14:34.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.034+0000 7fbe3bfff700 1 -- 172.21.15.101:0/1099386290 learned_addr learned my addr 172.21.15.101:0/1099386290 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:34.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.034+0000 7fbe3b7fe700 1 --2- 172.21.15.101:0/1099386290 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605f1855be0 0x5605f1865af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:34.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.034+0000 7fbe40f9c700 1 --2- 172.21.15.101:0/1099386290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605f1860ad0 0x5605f1866090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:34.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.034+0000 7fbe3bfff700 1 -- 172.21.15.101:0/1099386290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605f1860ad0 msgr2=0x5605f1866090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:34.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.034+0000 7fbe3bfff700 1 --2- 172.21.15.101:0/1099386290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605f1860ad0 0x5605f1866090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.034+0000 7fbe3bfff700 1 -- 172.21.15.101:0/1099386290 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605f1855be0 msgr2=0x5605f1865af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:34.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.034+0000 7fbe3bfff700 1 --2- 172.21.15.101:0/1099386290 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605f1855be0 0x5605f1865af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.034+0000 7fbe3bfff700 1 -- 172.21.15.101:0/1099386290 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe30009d10 con 0x5605f18552d0 2023-07-18T03:14:34.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.034+0000 7fbe3b7fe700 1 --2- 172.21.15.101:0/1099386290 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605f1855be0 0x5605f1865af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:14:34.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.034+0000 7fbe40f9c700 1 --2- 172.21.15.101:0/1099386290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605f1860ad0 0x5605f1866090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:14:34.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.035+0000 7fbe3bfff700 1 --2- 172.21.15.101:0/1099386290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605f18552d0 0x5605f18675d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c09b090 tx=0x7fbe2c022c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdf73139e7a8bf68 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:34.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.035+0000 7fbe1b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:14:34.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.035+0000 7fbe1b7fe700 1 -- 172.21.15.101:0/1099386290 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c09f130 con 0x5605f18552d0 2023-07-18T03:14:34.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.035+0000 7fbe1b7fe700 1 -- 172.21.15.101:0/1099386290 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fbe2c0cbe40 con 0x5605f18552d0 2023-07-18T03:14:34.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.035+0000 7fbe56360b80 1 -- 172.21.15.101:0/1099386290 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbe300054e0 con 0x5605f18552d0 2023-07-18T03:14:34.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.035+0000 7fbe1b7fe700 1 -- 172.21.15.101:0/1099386290 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe2c0a34c0 con 0x5605f18552d0 2023-07-18T03:14:34.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.035+0000 7fbe56360b80 10 client.26989 fetch_fsmap learned FSMap version 29 2023-07-18T03:14:34.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.035+0000 7fbe56360b80 1 -- 172.21.15.101:0/1099386290 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5605f15fc250 con 0x5605f18552d0 2023-07-18T03:14:34.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.035+0000 7fbe1b7fe700 1 -- 172.21.15.101:0/1099386290 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe300054e0 con 0x5605f18552d0 2023-07-18T03:14:34.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.036+0000 7fbe1b7fe700 1 -- 172.21.15.101:0/1099386290 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fbe2c09b360 con 0x5605f18552d0 2023-07-18T03:14:34.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.036+0000 7fbe56360b80 10 client.26989 fetch_fsmap finished waiting for FSMap version 29 2023-07-18T03:14:34.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.036+0000 7fbe56360b80 10 client.26989 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:14:34.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.036+0000 7fbe56360b80 20 client.26989 populate_metadata read hostname 'smithi101' 2023-07-18T03:14:34.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.036+0000 7fbe56360b80 1 --2- 172.21.15.101:0/1099386290 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5605f18c56e0 0x5605f18c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:34.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.036+0000 7fbe56360b80 4 client.26989 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:14:34.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.036+0000 7fbe56360b80 1 -- 172.21.15.101:0/1099386290 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5605f1763050 con 0x5605f18c56e0 2023-07-18T03:14:34.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.036+0000 7fbe3b7fe700 1 --2- 172.21.15.101:0/1099386290 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5605f18c56e0 0x5605f18c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:34.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.036+0000 7fbe3b7fe700 1 --2- 172.21.15.101:0/1099386290 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5605f18c56e0 0x5605f18c7af0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc648a87e7f671bf server_cookie=b898432a4664f453 in_seq=0 out_seq=0 2023-07-18T03:14:34.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.036+0000 7fbe1b7fe700 10 client.26989 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:14:34.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.037+0000 7fbe1b7fe700 1 -- 172.21.15.101:0/1099386290 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5605f1763050 con 0x5605f18c56e0 2023-07-18T03:14:34.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.037+0000 7fbe1b7fe700 10 client.26989 handle_command_reply: tid=0 2023-07-18T03:14:34.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.037+0000 7fbe56360b80 1 client.26989 shutdown 2023-07-18T03:14:34.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.037+0000 7fbe56360b80 1 -- 172.21.15.101:0/1099386290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605f18552d0 msgr2=0x5605f18675d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:34.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.037+0000 7fbe56360b80 1 --2- 172.21.15.101:0/1099386290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605f18552d0 0x5605f18675d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c09b090 tx=0x7fbe2c022c90 comp rx=0 tx=0).stop 2023-07-18T03:14:34.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.037+0000 7fbe56360b80 1 -- 172.21.15.101:0/1099386290 shutdown_connections 2023-07-18T03:14:34.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.037+0000 7fbe56360b80 1 --2- 172.21.15.101:0/1099386290 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5605f18c56e0 0x5605f18c7af0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.037+0000 7fbe56360b80 1 --2- 172.21.15.101:0/1099386290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605f1860ad0 0x5605f1866090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.037+0000 7fbe56360b80 1 --2- 172.21.15.101:0/1099386290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605f18552d0 0x5605f18675d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.037+0000 7fbe1b7fe700 0 client.26989 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:14:34.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.037+0000 7fbe56360b80 1 --2- 172.21.15.101:0/1099386290 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605f1855be0 0x5605f1865af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.037+0000 7fbe56360b80 1 -- 172.21.15.101:0/1099386290 >> 172.21.15.101:0/1099386290 conn(0x5605f184bda0 msgr2=0x5605f18508b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:34.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.037+0000 7fbe56360b80 1 -- 172.21.15.101:0/1099386290 shutdown_connections 2023-07-18T03:14:34.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.037+0000 7fbe56360b80 1 -- 172.21.15.101:0/1099386290 wait complete. 2023-07-18T03:14:34.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.037+0000 7fbe56360b80 20 client.26989 trim_cache size 0 max 16384 2023-07-18T03:14:34.039 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:14:34.039 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "no active scrubs running", 2023-07-18T03:14:34.039 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": {} 2023-07-18T03:14:34.039 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:14:34.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.038+0000 7fbe429ff700 1 -- 172.21.15.101:0/3446077241 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fbe240a12c0 msgr2=0x7fbe240a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:34.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.038+0000 7fbe429ff700 1 --2- 172.21.15.101:0/3446077241 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fbe240a12c0 0x7fbe240a3770 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbe3400c090 tx=0x7fbe34009920 comp rx=0 tx=0).stop 2023-07-18T03:14:34.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.038+0000 7fbe429ff700 1 -- 172.21.15.101:0/3446077241 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbe3c0ffae0 msgr2=0x7fbe3c19ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:34.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.038+0000 7fbe429ff700 1 --2- 172.21.15.101:0/3446077241 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbe3c0ffae0 0x7fbe3c19ba20 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c005910 tx=0x7fbe2c005720 comp rx=0 tx=0).stop 2023-07-18T03:14:34.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.038+0000 7fbe429ff700 1 -- 172.21.15.101:0/3446077241 shutdown_connections 2023-07-18T03:14:34.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.038+0000 7fbe429ff700 1 --2- 172.21.15.101:0/3446077241 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe3c105af0 0x7fbe3c195a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.038+0000 7fbe429ff700 1 --2- 172.21.15.101:0/3446077241 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbe3c0ffae0 0x7fbe3c19ba20 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.038+0000 7fbe429ff700 1 --2- 172.21.15.101:0/3446077241 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fbe240a12c0 0x7fbe240a3770 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.038+0000 7fbe429ff700 1 --2- 172.21.15.101:0/3446077241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe3c100490 0x7fbe3c19bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.038+0000 7fbe429ff700 1 -- 172.21.15.101:0/3446077241 >> 172.21.15.101:0/3446077241 conn(0x7fbe3c0fb5e0 msgr2=0x7fbe3c0fbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:34.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.038+0000 7fbe429ff700 1 -- 172.21.15.101:0/3446077241 shutdown_connections 2023-07-18T03:14:34.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.038+0000 7fbe429ff700 1 -- 172.21.15.101:0/3446077241 wait complete. 2023-07-18T03:14:34.050 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-18T03:14:34.050 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:14:34.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.270+0000 7f91525ff700 1 -- 172.21.15.101:0/907047413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f914c10c3d0 msgr2=0x7f914c10e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:34.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.270+0000 7f91525ff700 1 --2- 172.21.15.101:0/907047413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f914c10c3d0 0x7f914c10e7b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f914400b0a0 tx=0x7f914400b3b0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.270+0000 7f91525ff700 1 -- 172.21.15.101:0/907047413 shutdown_connections 2023-07-18T03:14:34.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.270+0000 7f91525ff700 1 --2- 172.21.15.101:0/907047413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f914c072850 0x7f914c072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.270+0000 7f91525ff700 1 --2- 172.21.15.101:0/907047413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f914c10c3d0 0x7f914c10e7b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.270+0000 7f91525ff700 1 --2- 172.21.15.101:0/907047413 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f914c073160 0x7f914c10be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.270+0000 7f91525ff700 1 -- 172.21.15.101:0/907047413 >> 172.21.15.101:0/907047413 conn(0x7f914c0fb650 msgr2=0x7f914c0fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:34.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.270+0000 7f91525ff700 1 -- 172.21.15.101:0/907047413 shutdown_connections 2023-07-18T03:14:34.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.271+0000 7f91525ff700 1 -- 172.21.15.101:0/907047413 wait complete. 2023-07-18T03:14:34.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.272+0000 7f91525ff700 1 Processor -- start 2023-07-18T03:14:34.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.272+0000 7f91525ff700 1 -- start start 2023-07-18T03:14:34.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.272+0000 7f91525ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f914c072850 0x7f914c197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:34.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.272+0000 7f91525ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f914c073160 0x7f914c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:34.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.272+0000 7f91525ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f914c10c3d0 0x7f914c19c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:34.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.272+0000 7f91525ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f914c19cf00 con 0x7f914c073160 2023-07-18T03:14:34.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.272+0000 7f91525ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f914c19d070 con 0x7f914c10c3d0 2023-07-18T03:14:34.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.272+0000 7f91525ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f914c19d230 con 0x7f914c072850 2023-07-18T03:14:34.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.272+0000 7f914bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f914c072850 0x7f914c197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:34.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.272+0000 7f914b7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f914c073160 0x7f914c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:34.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.272+0000 7f914bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f914c072850 0x7f914c197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:35244/0 (socket says 172.21.15.101:35244) 2023-07-18T03:14:34.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.272+0000 7f914bfff700 1 -- 172.21.15.101:0/3128452292 learned_addr learned my addr 172.21.15.101:0/3128452292 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:34.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.272+0000 7f9150b9c700 1 --2- 172.21.15.101:0/3128452292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f914c10c3d0 0x7f914c19c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:34.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.273+0000 7f914bfff700 1 -- 172.21.15.101:0/3128452292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f914c10c3d0 msgr2=0x7f914c19c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:34.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.273+0000 7f914bfff700 1 --2- 172.21.15.101:0/3128452292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f914c10c3d0 0x7f914c19c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.273+0000 7f914bfff700 1 -- 172.21.15.101:0/3128452292 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f914c073160 msgr2=0x7f914c198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:34.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.273+0000 7f914bfff700 1 --2- 172.21.15.101:0/3128452292 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f914c073160 0x7f914c198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.273+0000 7f914bfff700 1 -- 172.21.15.101:0/3128452292 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9144009d00 con 0x7f914c072850 2023-07-18T03:14:34.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.273+0000 7f914bfff700 1 --2- 172.21.15.101:0/3128452292 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f914c072850 0x7f914c197f20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f913c00e9a0 tx=0x7f913c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:34.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.273+0000 7f91497fa700 1 -- 172.21.15.101:0/3128452292 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913c009790 con 0x7f914c072850 2023-07-18T03:14:34.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.273+0000 7f91525ff700 1 -- 172.21.15.101:0/3128452292 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f914c19d510 con 0x7f914c072850 2023-07-18T03:14:34.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.273+0000 7f91497fa700 1 -- 172.21.15.101:0/3128452292 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f913c010430 con 0x7f914c072850 2023-07-18T03:14:34.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.273+0000 7f91497fa700 1 -- 172.21.15.101:0/3128452292 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913c018680 con 0x7f914c072850 2023-07-18T03:14:34.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.273+0000 7f91525ff700 1 -- 172.21.15.101:0/3128452292 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f914c1a4d50 con 0x7f914c072850 2023-07-18T03:14:34.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.274+0000 7f91525ff700 1 -- 172.21.15.101:0/3128452292 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f914c106cf0 con 0x7f914c072850 2023-07-18T03:14:34.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.277+0000 7f91497fa700 1 -- 172.21.15.101:0/3128452292 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f913c0105a0 con 0x7f914c072850 2023-07-18T03:14:34.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.278+0000 7f91497fa700 1 --2- 172.21.15.101:0/3128452292 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f91340a1300 0x7f91340a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:34.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.278+0000 7f91497fa700 1 -- 172.21.15.101:0/3128452292 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f913c014070 con 0x7f914c072850 2023-07-18T03:14:34.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.278+0000 7f914b7fe700 1 --2- 172.21.15.101:0/3128452292 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f91340a1300 0x7f91340a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:34.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.279+0000 7f914b7fe700 1 --2- 172.21.15.101:0/3128452292 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f91340a1300 0x7f91340a37b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f914c199530 tx=0x7f9140009260 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:34.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.280+0000 7f91497fa700 1 -- 172.21.15.101:0/3128452292 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f913c0964d0 con 0x7f914c072850 2023-07-18T03:14:34.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.545+0000 7f91525ff700 1 -- 172.21.15.101:0/3128452292 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f914c067300 con 0x7f914c072850 2023-07-18T03:14:34.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.545+0000 7f91497fa700 1 -- 172.21.15.101:0/3128452292 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10863 (secure 0 0 0) 0x7f913c026090 con 0x7f914c072850 2023-07-18T03:14:34.538 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:14:34.538 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/4192846769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":4192846769},{"type":"v1","addr":"172.21.15.101:6839","nonce":4192846769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6841/3824340175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6840","nonce":3824340175},{"type":"v1","addr":"172.21.15.101:6841","nonce":3824340175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:04:59.361185+0000","modified":"2023-07-18T03:14:01.125192+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.155:6839/2055462483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2055462483},{"type":"v1","addr":"172.21.15.155:6839","nonce":2055462483}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:14:34.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.548+0000 7f91525ff700 1 -- 172.21.15.101:0/3128452292 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f91340a1300 msgr2=0x7f91340a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:34.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.548+0000 7f91525ff700 1 --2- 172.21.15.101:0/3128452292 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f91340a1300 0x7f91340a37b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f914c199530 tx=0x7f9140009260 comp rx=0 tx=0).stop 2023-07-18T03:14:34.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.548+0000 7f91525ff700 1 -- 172.21.15.101:0/3128452292 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f914c072850 msgr2=0x7f914c197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:34.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.548+0000 7f91525ff700 1 --2- 172.21.15.101:0/3128452292 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f914c072850 0x7f914c197f20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f913c00e9a0 tx=0x7f913c00ecb0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.548+0000 7f91525ff700 1 -- 172.21.15.101:0/3128452292 shutdown_connections 2023-07-18T03:14:34.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.548+0000 7f91525ff700 1 --2- 172.21.15.101:0/3128452292 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f91340a1300 0x7f91340a37b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.548+0000 7f91525ff700 1 --2- 172.21.15.101:0/3128452292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f914c10c3d0 0x7f914c19c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.548+0000 7f91525ff700 1 --2- 172.21.15.101:0/3128452292 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f914c072850 0x7f914c197f20 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.548+0000 7f91525ff700 1 --2- 172.21.15.101:0/3128452292 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f914c073160 0x7f914c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.548+0000 7f91525ff700 1 -- 172.21.15.101:0/3128452292 >> 172.21.15.101:0/3128452292 conn(0x7f914c0fb650 msgr2=0x7f914c103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:34.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.548+0000 7f91525ff700 1 -- 172.21.15.101:0/3128452292 shutdown_connections 2023-07-18T03:14:34.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.548+0000 7f91525ff700 1 -- 172.21.15.101:0/3128452292 wait complete. 2023-07-18T03:14:34.543 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 29 2023-07-18T03:14:34.559 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-18T03:14:34.589 INFO:tasks.workunit.client.0.smithi101.stdout: 1 505111 7.38 MB/sec execute 239 sec latency 20.959 ms 2023-07-18T03:14:34.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.790+0000 7fb3fd7ff700 1 -- 172.21.15.101:0/820018003 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3f80ffae0 msgr2=0x7fb3f80fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:34.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.790+0000 7fb3fd7ff700 1 --2- 172.21.15.101:0/820018003 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3f80ffae0 0x7fb3f80fff50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0009990 tx=0x7fb3e0009ca0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.790+0000 7fb3fd7ff700 1 -- 172.21.15.101:0/820018003 shutdown_connections 2023-07-18T03:14:34.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.790+0000 7fb3fd7ff700 1 --2- 172.21.15.101:0/820018003 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb3f8100490 0x7fb3f810c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.790+0000 7fb3fd7ff700 1 --2- 172.21.15.101:0/820018003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3f8105af0 0x7fb3f8105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.790+0000 7fb3fd7ff700 1 --2- 172.21.15.101:0/820018003 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3f80ffae0 0x7fb3f80fff50 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.790+0000 7fb3fd7ff700 1 -- 172.21.15.101:0/820018003 >> 172.21.15.101:0/820018003 conn(0x7fb3f80fb5e0 msgr2=0x7fb3f80fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:34.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.790+0000 7fb3fd7ff700 1 -- 172.21.15.101:0/820018003 shutdown_connections 2023-07-18T03:14:34.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.790+0000 7fb3fd7ff700 1 -- 172.21.15.101:0/820018003 wait complete. 2023-07-18T03:14:34.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.791+0000 7fb3fd7ff700 1 Processor -- start 2023-07-18T03:14:34.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.791+0000 7fb3fd7ff700 1 -- start start 2023-07-18T03:14:34.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.792+0000 7fb3fd7ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb3f80ffae0 0x7fb3f8197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:34.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.792+0000 7fb3fd7ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3f8100490 0x7fb3f8198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:34.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.792+0000 7fb3fd7ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3f8105af0 0x7fb3f819c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:34.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.792+0000 7fb3fd7ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f819ce20 con 0x7fb3f80ffae0 2023-07-18T03:14:34.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.792+0000 7fb3fd7ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f819cf90 con 0x7fb3f8100490 2023-07-18T03:14:34.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.792+0000 7fb3fd7ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f819d150 con 0x7fb3f8105af0 2023-07-18T03:14:34.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.792+0000 7fb3f6ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb3f80ffae0 0x7fb3f8197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:34.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.792+0000 7fb3effff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3f8100490 0x7fb3f8198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:34.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.792+0000 7fb3effff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3f8100490 0x7fb3f8198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47888/0 (socket says 172.21.15.101:47888) 2023-07-18T03:14:34.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.792+0000 7fb3effff700 1 -- 172.21.15.101:0/3927556021 learned_addr learned my addr 172.21.15.101:0/3927556021 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:34.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.792+0000 7fb3f6ffd700 1 --2- 172.21.15.101:0/3927556021 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb3f80ffae0 0x7fb3f8197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:58880/0 (socket says 172.21.15.101:58880) 2023-07-18T03:14:34.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.792+0000 7fb3f77fe700 1 --2- 172.21.15.101:0/3927556021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3f8105af0 0x7fb3f819c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:34.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.792+0000 7fb3effff700 1 -- 172.21.15.101:0/3927556021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3f8105af0 msgr2=0x7fb3f819c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:34.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.792+0000 7fb3effff700 1 --2- 172.21.15.101:0/3927556021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3f8105af0 0x7fb3f819c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.792+0000 7fb3effff700 1 -- 172.21.15.101:0/3927556021 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb3f80ffae0 msgr2=0x7fb3f8197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:34.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.792+0000 7fb3effff700 1 --2- 172.21.15.101:0/3927556021 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb3f80ffae0 0x7fb3f8197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.792+0000 7fb3effff700 1 -- 172.21.15.101:0/3927556021 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e00095f0 con 0x7fb3f8100490 2023-07-18T03:14:34.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.792+0000 7fb3f6ffd700 1 --2- 172.21.15.101:0/3927556021 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb3f80ffae0 0x7fb3f8197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:14:34.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.793+0000 7fb3effff700 1 --2- 172.21.15.101:0/3927556021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3f8100490 0x7fb3f8198380 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0005910 tx=0x7fb3e0004710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:34.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.793+0000 7fb3f4ff9700 1 -- 172.21.15.101:0/3927556021 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e001d070 con 0x7fb3f8100490 2023-07-18T03:14:34.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.793+0000 7fb3f4ff9700 1 -- 172.21.15.101:0/3927556021 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3e0022470 con 0x7fb3f8100490 2023-07-18T03:14:34.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.793+0000 7fb3f4ff9700 1 -- 172.21.15.101:0/3927556021 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e0017650 con 0x7fb3f8100490 2023-07-18T03:14:34.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.793+0000 7fb3fd7ff700 1 -- 172.21.15.101:0/3927556021 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3f819d3d0 con 0x7fb3f8100490 2023-07-18T03:14:34.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.793+0000 7fb3fd7ff700 1 -- 172.21.15.101:0/3927556021 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f819d650 con 0x7fb3f8100490 2023-07-18T03:14:34.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.798+0000 7fb3f4ff9700 1 -- 172.21.15.101:0/3927556021 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fb3e00225e0 con 0x7fb3f8100490 2023-07-18T03:14:34.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.799+0000 7fb411062b80 1 Processor -- start 2023-07-18T03:14:34.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.799+0000 7fb411062b80 1 -- start start 2023-07-18T03:14:34.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.800+0000 7fb411062b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b1908c52d0 0x55b1908c56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:34.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.800+0000 7fb411062b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1908c5be0 0x55b1908d0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:34.793 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.800+0000 7fb411062b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b1908d0ad0 0x55b1908d2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:34.793 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.800+0000 7fb411062b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1907cbcb0 con 0x55b1908c52d0 2023-07-18T03:14:34.793 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.800+0000 7fb411062b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b19081ea40 con 0x55b1908c5be0 2023-07-18T03:14:34.793 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.800+0000 7fb411062b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b190817ed0 con 0x55b1908d0ad0 2023-07-18T03:14:34.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.800+0000 7fb3f4ff9700 1 --2- 172.21.15.101:0/3927556021 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fb3e40a1340 0x7fb3e40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:34.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.800+0000 7fb3effff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b1908d0ad0 0x55b1908d2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:34.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.800+0000 7fb3effff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b1908d0ad0 0x55b1908d2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:35276/0 (socket says 172.21.15.101:35276) 2023-07-18T03:14:34.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.800+0000 7fb3effff700 1 -- 172.21.15.101:0/1175013840 learned_addr learned my addr 172.21.15.101:0/1175013840 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:34.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.800+0000 7fb3f77fe700 1 --2- 172.21.15.101:0/1175013840 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b1908c52d0 0x55b1908c56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:34.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.800+0000 7fb3effff700 1 -- 172.21.15.101:0/1175013840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1908c5be0 msgr2=0x55b1908d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:34.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.800+0000 7fb3effff700 1 --2- 172.21.15.101:0/1175013840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1908c5be0 0x55b1908d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.800+0000 7fb3effff700 1 -- 172.21.15.101:0/1175013840 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b1908c52d0 msgr2=0x55b1908c56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:34.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.800+0000 7fb3effff700 1 --2- 172.21.15.101:0/1175013840 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b1908c52d0 0x55b1908c56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.800+0000 7fb3effff700 1 -- 172.21.15.101:0/1175013840 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b190645de0 con 0x55b1908d0ad0 2023-07-18T03:14:34.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.801+0000 7fb3f77fe700 1 --2- 172.21.15.101:0/3927556021 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fb3e40a1340 0x7fb3e40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:34.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.801+0000 7fb3f4ff9700 1 -- 172.21.15.101:0/3927556021 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fb3e00ce170 con 0x7fb3f8100490 2023-07-18T03:14:34.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.801+0000 7fb3effff700 1 --2- 172.21.15.101:0/1175013840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b1908d0ad0 0x55b1908d2eb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb3e00973b0 tx=0x7fb3e009ef30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3387ace9d2b471cc server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:34.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.801+0000 7fb3ed7fa700 1 -- 172.21.15.101:0/1175013840 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e00b3070 con 0x55b1908d0ad0 2023-07-18T03:14:34.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.801+0000 7fb3f77fe700 1 --2- 172.21.15.101:0/3927556021 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fb3e40a1340 0x7fb3e40a37f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0009fd0 tx=0x7fb3f000b040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:34.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.801+0000 7fb3ed7fa700 1 -- 172.21.15.101:0/1175013840 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3e0022900 con 0x55b1908d0ad0 2023-07-18T03:14:34.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.801+0000 7fb3ed7fa700 1 -- 172.21.15.101:0/1175013840 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e009b160 con 0x55b1908d0ad0 2023-07-18T03:14:34.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.802+0000 7fb411062b80 1 -- 172.21.15.101:0/1175013840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b1908d0ad0 msgr2=0x55b1908d2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:34.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.802+0000 7fb411062b80 1 --2- 172.21.15.101:0/1175013840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b1908d0ad0 0x55b1908d2eb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb3e00973b0 tx=0x7fb3e009ef30 comp rx=0 tx=0).stop 2023-07-18T03:14:34.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.802+0000 7fb411062b80 1 -- 172.21.15.101:0/1175013840 shutdown_connections 2023-07-18T03:14:34.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.802+0000 7fb411062b80 1 --2- 172.21.15.101:0/1175013840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b1908d0ad0 0x55b1908d2eb0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.802+0000 7fb411062b80 1 --2- 172.21.15.101:0/1175013840 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b1908c52d0 0x55b1908c56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.802+0000 7fb411062b80 1 --2- 172.21.15.101:0/1175013840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1908c5be0 0x55b1908d0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.802+0000 7fb411062b80 1 -- 172.21.15.101:0/1175013840 >> 172.21.15.101:0/1175013840 conn(0x55b1908bbda0 msgr2=0x55b1908c16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:34.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.802+0000 7fb411062b80 1 -- 172.21.15.101:0/1175013840 shutdown_connections 2023-07-18T03:14:34.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.802+0000 7fb411062b80 1 -- 172.21.15.101:0/1175013840 wait complete. 2023-07-18T03:14:34.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.803+0000 7fb411062b80 1 -- start start 2023-07-18T03:14:34.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.803+0000 7fb411062b80 1 Processor -- start 2023-07-18T03:14:34.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.804+0000 7fb411062b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b1908c52d0 0x55b1908d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:34.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.804+0000 7fb411062b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b1908c5be0 0x55b1908d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:34.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.804+0000 7fb411062b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1908d0ad0 0x55b1908d1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:34.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.804+0000 7fb411062b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1907b7ac0 con 0x55b1908c52d0 2023-07-18T03:14:34.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.804+0000 7fb411062b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1907b77c0 con 0x55b1908d0ad0 2023-07-18T03:14:34.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.804+0000 7fb411062b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b19069b920 con 0x55b1908c5be0 2023-07-18T03:14:34.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.804+0000 7fb411062b80 10 client.0 _ll_register_callbacks cb 0x55b1907b5760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:14:34.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.804+0000 7fb3f77fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1908d0ad0 0x55b1908d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:34.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.804+0000 7fb3effff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b1908c5be0 0x55b1908d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:34.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.804+0000 7fb3f77fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1908d0ad0 0x55b1908d1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47908/0 (socket says 172.21.15.101:47908) 2023-07-18T03:14:34.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.804+0000 7fb3f77fe700 1 -- 172.21.15.101:0/3389478047 learned_addr learned my addr 172.21.15.101:0/3389478047 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:34.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.804+0000 7fb3f6ffd700 1 --2- 172.21.15.101:0/3389478047 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b1908c52d0 0x55b1908d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:34.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.804+0000 7fb3f77fe700 1 -- 172.21.15.101:0/3389478047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b1908c5be0 msgr2=0x55b1908d13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:34.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.804+0000 7fb3f77fe700 1 --2- 172.21.15.101:0/3389478047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b1908c5be0 0x55b1908d13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.804+0000 7fb3f77fe700 1 -- 172.21.15.101:0/3389478047 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b1908c52d0 msgr2=0x55b1908d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:34.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.804+0000 7fb3f77fe700 1 --2- 172.21.15.101:0/3389478047 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b1908c52d0 0x55b1908d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.804+0000 7fb3f77fe700 1 -- 172.21.15.101:0/3389478047 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f819d3d0 con 0x55b1908d0ad0 2023-07-18T03:14:34.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.805+0000 7fb3f77fe700 1 --2- 172.21.15.101:0/3389478047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1908d0ad0 0x55b1908d1920 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0007bf0 tx=0x7fb3f0004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=770cf12e892a454d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:34.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.805+0000 7fb3de7fc700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:14:34.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.805+0000 7fb3de7fc700 1 -- 172.21.15.101:0/3389478047 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f000e040 con 0x55b1908d0ad0 2023-07-18T03:14:34.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.805+0000 7fb411062b80 1 -- 172.21.15.101:0/3389478047 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b1907714f0 con 0x55b1908d0ad0 2023-07-18T03:14:34.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.805+0000 7fb3de7fc700 1 -- 172.21.15.101:0/3389478047 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fb3f001d040 con 0x55b1908d0ad0 2023-07-18T03:14:34.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.805+0000 7fb3de7fc700 1 -- 172.21.15.101:0/3389478047 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3f0004db0 con 0x55b1908d0ad0 2023-07-18T03:14:34.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.805+0000 7fb411062b80 10 client.26966 fetch_fsmap learned FSMap version 29 2023-07-18T03:14:34.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.805+0000 7fb411062b80 1 -- 172.21.15.101:0/3389478047 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55b19062ec60 con 0x55b1908d0ad0 2023-07-18T03:14:34.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.805+0000 7fb3de7fc700 1 -- 172.21.15.101:0/3389478047 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b1907714f0 con 0x55b1908d0ad0 2023-07-18T03:14:34.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.806+0000 7fb3de7fc700 1 -- 172.21.15.101:0/3389478047 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fb3f0028020 con 0x55b1908d0ad0 2023-07-18T03:14:34.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.806+0000 7fb411062b80 10 client.26966 fetch_fsmap finished waiting for FSMap version 29 2023-07-18T03:14:34.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.806+0000 7fb411062b80 10 client.26966 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:14:34.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.806+0000 7fb411062b80 20 client.26966 populate_metadata read hostname 'smithi101' 2023-07-18T03:14:34.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.806+0000 7fb411062b80 1 --2- 172.21.15.101:0/3389478047 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55b1908e01d0 0x55b1908e25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:34.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.806+0000 7fb411062b80 4 client.26966 mds_command: new command op to 24541 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:14:34.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.806+0000 7fb411062b80 1 -- 172.21.15.101:0/3389478047 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b190810eb0 con 0x55b1908e01d0 2023-07-18T03:14:34.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.806+0000 7fb3f6ffd700 1 --2- 172.21.15.101:0/3389478047 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55b1908e01d0 0x55b1908e25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:34.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.807+0000 7fb3f6ffd700 1 --2- 172.21.15.101:0/3389478047 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55b1908e01d0 0x55b1908e25e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=44904806269130a9 server_cookie=bcbb726bf9bf2600 in_seq=0 out_seq=0 2023-07-18T03:14:34.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.807+0000 7fb3de7fc700 10 client.26966 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:14:34.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.807+0000 7fb3de7fc700 1 -- 172.21.15.101:0/3389478047 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b190810eb0 con 0x55b1908e01d0 2023-07-18T03:14:34.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.807+0000 7fb3de7fc700 10 client.26966 handle_command_reply: tid=0 2023-07-18T03:14:34.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.808+0000 7fb411062b80 1 client.26966 shutdown 2023-07-18T03:14:34.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.808+0000 7fb411062b80 1 -- 172.21.15.101:0/3389478047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1908d0ad0 msgr2=0x55b1908d1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:34.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.808+0000 7fb411062b80 1 --2- 172.21.15.101:0/3389478047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1908d0ad0 0x55b1908d1920 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0007bf0 tx=0x7fb3f0004ad0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.808+0000 7fb411062b80 1 -- 172.21.15.101:0/3389478047 shutdown_connections 2023-07-18T03:14:34.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.808+0000 7fb411062b80 1 --2- 172.21.15.101:0/3389478047 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55b1908e01d0 0x55b1908e25e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.808+0000 7fb411062b80 1 --2- 172.21.15.101:0/3389478047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1908d0ad0 0x55b1908d1920 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.808+0000 7fb411062b80 1 --2- 172.21.15.101:0/3389478047 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b1908c52d0 0x55b1908d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.808+0000 7fb411062b80 1 --2- 172.21.15.101:0/3389478047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b1908c5be0 0x55b1908d13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.808+0000 7fb411062b80 1 -- 172.21.15.101:0/3389478047 >> 172.21.15.101:0/3389478047 conn(0x55b1908bbda0 msgr2=0x55b1908c1290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:34.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.808+0000 7fb3de7fc700 0 client.26966 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:14:34.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.808+0000 7fb411062b80 1 -- 172.21.15.101:0/3389478047 shutdown_connections 2023-07-18T03:14:34.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.808+0000 7fb411062b80 1 -- 172.21.15.101:0/3389478047 wait complete. 2023-07-18T03:14:34.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.808+0000 7fb411062b80 20 client.26966 trim_cache size 0 max 16384 2023-07-18T03:14:34.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.823+0000 7fb411062b80 1 Processor -- start 2023-07-18T03:14:34.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.823+0000 7fb411062b80 1 -- start start 2023-07-18T03:14:34.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.823+0000 7fb411062b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b1908c52d0 0x55b1908d4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:34.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.823+0000 7fb411062b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b1908c5be0 0x55b1908ca400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:34.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.823+0000 7fb411062b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1908d0ad0 0x55b1908ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:34.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.823+0000 7fb411062b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1908c6e70 con 0x55b1908c5be0 2023-07-18T03:14:34.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.823+0000 7fb411062b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1908c6570 con 0x55b1908d0ad0 2023-07-18T03:14:34.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.823+0000 7fb411062b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1908bb9f0 con 0x55b1908c52d0 2023-07-18T03:14:34.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.823+0000 7fb3f6ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b1908c52d0 0x55b1908d4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:34.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.823+0000 7fb3effff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b1908c5be0 0x55b1908ca400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:34.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.823+0000 7fb3f6ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b1908c52d0 0x55b1908d4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:35288/0 (socket says 172.21.15.101:35288) 2023-07-18T03:14:34.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.823+0000 7fb3f6ffd700 1 -- 172.21.15.101:0/2717031394 learned_addr learned my addr 172.21.15.101:0/2717031394 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:34.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.823+0000 7fb3f77fe700 1 --2- 172.21.15.101:0/2717031394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1908d0ad0 0x55b1908ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:34.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.824+0000 7fb3f6ffd700 1 -- 172.21.15.101:0/2717031394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1908d0ad0 msgr2=0x55b1908ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:34.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.824+0000 7fb3f6ffd700 1 --2- 172.21.15.101:0/2717031394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1908d0ad0 0x55b1908ccdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.824+0000 7fb3f6ffd700 1 -- 172.21.15.101:0/2717031394 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b1908c5be0 msgr2=0x55b1908ca400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:34.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.824+0000 7fb3f6ffd700 1 --2- 172.21.15.101:0/2717031394 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b1908c5be0 0x55b1908ca400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.824+0000 7fb3f6ffd700 1 -- 172.21.15.101:0/2717031394 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1908cf6d0 con 0x55b1908c52d0 2023-07-18T03:14:34.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.824+0000 7fb3effff700 1 --2- 172.21.15.101:0/2717031394 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b1908c5be0 0x55b1908ca400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:14:34.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.824+0000 7fb3f6ffd700 1 --2- 172.21.15.101:0/2717031394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b1908c52d0 0x55b1908d4ed0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55b1908e2ec0 tx=0x7fb3e8011a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdbbc8826fe693a0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:34.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.824+0000 7fb3edffb700 1 -- 172.21.15.101:0/2717031394 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e8017040 con 0x55b1908c52d0 2023-07-18T03:14:34.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.824+0000 7fb3edffb700 1 -- 172.21.15.101:0/2717031394 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3e801c5b0 con 0x55b1908c52d0 2023-07-18T03:14:34.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.824+0000 7fb3edffb700 1 -- 172.21.15.101:0/2717031394 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e8005280 con 0x55b1908c52d0 2023-07-18T03:14:34.822 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.825+0000 7fb411062b80 1 -- 172.21.15.101:0/2717031394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b1908c52d0 msgr2=0x55b1908d4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:34.823 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.825+0000 7fb411062b80 1 --2- 172.21.15.101:0/2717031394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b1908c52d0 0x55b1908d4ed0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55b1908e2ec0 tx=0x7fb3e8011a10 comp rx=0 tx=0).stop 2023-07-18T03:14:34.823 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.825+0000 7fb411062b80 1 -- 172.21.15.101:0/2717031394 shutdown_connections 2023-07-18T03:14:34.823 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.825+0000 7fb411062b80 1 --2- 172.21.15.101:0/2717031394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1908d0ad0 0x55b1908ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.823 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.825+0000 7fb411062b80 1 --2- 172.21.15.101:0/2717031394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b1908c52d0 0x55b1908d4ed0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.823 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.825+0000 7fb411062b80 1 --2- 172.21.15.101:0/2717031394 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b1908c5be0 0x55b1908ca400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.823 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.825+0000 7fb411062b80 1 -- 172.21.15.101:0/2717031394 >> 172.21.15.101:0/2717031394 conn(0x55b1908bbda0 msgr2=0x55b1908dc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:34.824 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.825+0000 7fb411062b80 1 -- 172.21.15.101:0/2717031394 shutdown_connections 2023-07-18T03:14:34.824 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.825+0000 7fb411062b80 1 -- 172.21.15.101:0/2717031394 wait complete. 2023-07-18T03:14:34.824 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.826+0000 7fb411062b80 1 -- start start 2023-07-18T03:14:34.824 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.826+0000 7fb411062b80 1 Processor -- start 2023-07-18T03:14:34.824 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.826+0000 7fb411062b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1908c52d0 0x55b190928210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:34.824 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.826+0000 7fb411062b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b1908c5be0 0x55b190928750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:34.824 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.826+0000 7fb411062b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b1908d0ad0 0x55b19092d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:34.824 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.826+0000 7fb411062b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b19092d620 con 0x55b1908d0ad0 2023-07-18T03:14:34.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.826+0000 7fb411062b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b19092d790 con 0x55b1908c52d0 2023-07-18T03:14:34.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.826+0000 7fb411062b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b19092d950 con 0x55b1908c5be0 2023-07-18T03:14:34.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.826+0000 7fb411062b80 10 client.0 _ll_register_callbacks cb 0x7fb3f0007590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:14:34.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.826+0000 7fb3effff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b1908c5be0 0x55b190928750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:34.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.827+0000 7fb3f77fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b1908d0ad0 0x55b19092d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:34.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.827+0000 7fb3effff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b1908c5be0 0x55b190928750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:35294/0 (socket says 172.21.15.101:35294) 2023-07-18T03:14:34.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.827+0000 7fb3f6ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1908c52d0 0x55b190928210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:34.826 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.827+0000 7fb3effff700 1 -- 172.21.15.101:0/1739733217 learned_addr learned my addr 172.21.15.101:0/1739733217 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:34.826 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.827+0000 7fb3effff700 1 -- 172.21.15.101:0/1739733217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1908c52d0 msgr2=0x55b190928210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:34.826 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.827+0000 7fb3effff700 1 --2- 172.21.15.101:0/1739733217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1908c52d0 0x55b190928210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.826 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.827+0000 7fb3effff700 1 -- 172.21.15.101:0/1739733217 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b1908d0ad0 msgr2=0x55b19092d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:34.826 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.827+0000 7fb3effff700 1 --2- 172.21.15.101:0/1739733217 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b1908d0ad0 0x55b19092d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.826 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.827+0000 7fb3effff700 1 -- 172.21.15.101:0/1739733217 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b190645de0 con 0x55b1908c5be0 2023-07-18T03:14:34.826 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.827+0000 7fb3effff700 1 --2- 172.21.15.101:0/1739733217 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b1908c5be0 0x55b190928750 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0097400 tx=0x7fb3e0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b2b76e69869f7e4 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:34.826 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.827+0000 7fb3de7fc700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:14:34.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.827+0000 7fb3de7fc700 1 -- 172.21.15.101:0/1739733217 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e00b3040 con 0x55b1908c5be0 2023-07-18T03:14:34.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.827+0000 7fb411062b80 1 -- 172.21.15.101:0/1739733217 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb3e0022900 con 0x55b1908c5be0 2023-07-18T03:14:34.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.827+0000 7fb3de7fc700 1 -- 172.21.15.101:0/1739733217 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fb3e00a0040 con 0x55b1908c5be0 2023-07-18T03:14:34.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.827+0000 7fb3de7fc700 1 -- 172.21.15.101:0/1739733217 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3e009b160 con 0x55b1908c5be0 2023-07-18T03:14:34.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.828+0000 7fb411062b80 10 client.27013 fetch_fsmap learned FSMap version 29 2023-07-18T03:14:34.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.828+0000 7fb411062b80 1 -- 172.21.15.101:0/1739733217 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fb3e80114f0 con 0x55b1908c5be0 2023-07-18T03:14:34.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.828+0000 7fb3de7fc700 1 -- 172.21.15.101:0/1739733217 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3e0022900 con 0x55b1908c5be0 2023-07-18T03:14:34.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.828+0000 7fb3de7fc700 1 -- 172.21.15.101:0/1739733217 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fb3e0095b10 con 0x55b1908c5be0 2023-07-18T03:14:34.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.828+0000 7fb411062b80 10 client.27013 fetch_fsmap finished waiting for FSMap version 29 2023-07-18T03:14:34.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.828+0000 7fb411062b80 10 client.27013 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:14:34.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.828+0000 7fb411062b80 20 client.27013 populate_metadata read hostname 'smithi101' 2023-07-18T03:14:34.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.828+0000 7fb411062b80 1 --2- 172.21.15.101:0/1739733217 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55b190934da0 0x55b1909371b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:34.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.828+0000 7fb411062b80 4 client.27013 mds_command: new command op to 24541 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:14:34.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.828+0000 7fb411062b80 1 -- 172.21.15.101:0/1739733217 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b1907d3050 con 0x55b190934da0 2023-07-18T03:14:34.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.828+0000 7fb3f6ffd700 1 --2- 172.21.15.101:0/1739733217 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55b190934da0 0x55b1909371b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:34.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.829+0000 7fb3f6ffd700 1 --2- 172.21.15.101:0/1739733217 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55b190934da0 0x55b1909371b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ae37249693ac56e5 server_cookie=c872a9981df12c09 in_seq=0 out_seq=0 2023-07-18T03:14:34.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.829+0000 7fb3de7fc700 10 client.27013 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:14:34.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.829+0000 7fb3de7fc700 1 -- 172.21.15.101:0/1739733217 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b1907d3050 con 0x55b190934da0 2023-07-18T03:14:34.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.829+0000 7fb3de7fc700 10 client.27013 handle_command_reply: tid=0 2023-07-18T03:14:34.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.829+0000 7fb411062b80 1 client.27013 shutdown 2023-07-18T03:14:34.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.829+0000 7fb411062b80 1 -- 172.21.15.101:0/1739733217 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b1908c5be0 msgr2=0x55b190928750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:34.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.829+0000 7fb411062b80 1 --2- 172.21.15.101:0/1739733217 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b1908c5be0 0x55b190928750 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0097400 tx=0x7fb3e0000bc0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.829+0000 7fb411062b80 1 -- 172.21.15.101:0/1739733217 shutdown_connections 2023-07-18T03:14:34.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.829+0000 7fb411062b80 1 --2- 172.21.15.101:0/1739733217 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55b190934da0 0x55b1909371b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.829+0000 7fb411062b80 1 --2- 172.21.15.101:0/1739733217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1908c52d0 0x55b190928210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.829+0000 7fb411062b80 1 --2- 172.21.15.101:0/1739733217 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b1908d0ad0 0x55b19092d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.829+0000 7fb411062b80 1 --2- 172.21.15.101:0/1739733217 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b1908c5be0 0x55b190928750 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.829+0000 7fb3de7fc700 0 client.27013 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:14:34.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.829+0000 7fb411062b80 1 -- 172.21.15.101:0/1739733217 >> 172.21.15.101:0/1739733217 conn(0x55b1908bbda0 msgr2=0x55b1908ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:34.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.829+0000 7fb411062b80 1 -- 172.21.15.101:0/1739733217 shutdown_connections 2023-07-18T03:14:34.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.829+0000 7fb411062b80 1 -- 172.21.15.101:0/1739733217 wait complete. 2023-07-18T03:14:34.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.830+0000 7fb411062b80 20 client.27013 trim_cache size 0 max 16384 2023-07-18T03:14:34.831 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:14:34.832 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.830+0000 7fb3fd7ff700 1 -- 172.21.15.101:0/3927556021 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fb3e40a1340 msgr2=0x7fb3e40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:34.832 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.830+0000 7fb3fd7ff700 1 --2- 172.21.15.101:0/3927556021 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fb3e40a1340 0x7fb3e40a37f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0009fd0 tx=0x7fb3f000b040 comp rx=0 tx=0).stop 2023-07-18T03:14:34.832 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.830+0000 7fb3fd7ff700 1 -- 172.21.15.101:0/3927556021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3f8100490 msgr2=0x7fb3f8198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:34.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.830+0000 7fb3fd7ff700 1 --2- 172.21.15.101:0/3927556021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3f8100490 0x7fb3f8198380 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0005910 tx=0x7fb3e0004710 comp rx=0 tx=0).stop 2023-07-18T03:14:34.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.830+0000 7fb3fd7ff700 1 -- 172.21.15.101:0/3927556021 shutdown_connections 2023-07-18T03:14:34.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.830+0000 7fb3fd7ff700 1 --2- 172.21.15.101:0/3927556021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3f8105af0 0x7fb3f819c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.830+0000 7fb3fd7ff700 1 --2- 172.21.15.101:0/3927556021 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb3f80ffae0 0x7fb3f8197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.830+0000 7fb3fd7ff700 1 --2- 172.21.15.101:0/3927556021 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fb3e40a1340 0x7fb3e40a37f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.830+0000 7fb3fd7ff700 1 --2- 172.21.15.101:0/3927556021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3f8100490 0x7fb3f8198380 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:34.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.830+0000 7fb3fd7ff700 1 -- 172.21.15.101:0/3927556021 >> 172.21.15.101:0/3927556021 conn(0x7fb3f80fb5e0 msgr2=0x7fb3f810ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:34.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.830+0000 7fb3fd7ff700 1 -- 172.21.15.101:0/3927556021 shutdown_connections 2023-07-18T03:14:34.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:34.830+0000 7fb3fd7ff700 1 -- 172.21.15.101:0/3927556021 wait complete. 2023-07-18T03:14:34.841 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-18T03:14:35.094 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.101+0000 7efec939d700 1 -- 172.21.15.101:0/2455974958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efec41013b0 msgr2=0x7efec4109a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:35.094 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.101+0000 7efec939d700 1 --2- 172.21.15.101:0/2455974958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efec41013b0 0x7efec4109a80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7efeb80098f0 tx=0x7efeb8009c00 comp rx=0 tx=0).stop 2023-07-18T03:14:35.094 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.101+0000 7efec939d700 1 -- 172.21.15.101:0/2455974958 shutdown_connections 2023-07-18T03:14:35.094 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.101+0000 7efec939d700 1 --2- 172.21.15.101:0/2455974958 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efec4109fc0 0x7efec410c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.094 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.101+0000 7efec939d700 1 --2- 172.21.15.101:0/2455974958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efec4100aa0 0x7efec4100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.094 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.101+0000 7efec939d700 1 --2- 172.21.15.101:0/2455974958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efec41013b0 0x7efec4109a80 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.101+0000 7efec939d700 1 -- 172.21.15.101:0/2455974958 >> 172.21.15.101:0/2455974958 conn(0x7efec40745d0 msgr2=0x7efec40749d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:35.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.101+0000 7efec939d700 1 -- 172.21.15.101:0/2455974958 shutdown_connections 2023-07-18T03:14:35.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.101+0000 7efec939d700 1 -- 172.21.15.101:0/2455974958 wait complete. 2023-07-18T03:14:35.096 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.102+0000 7efec939d700 1 Processor -- start 2023-07-18T03:14:35.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.102+0000 7efec939d700 1 -- start start 2023-07-18T03:14:35.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.102+0000 7efec939d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efec4100aa0 0x7efec419da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:35.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.102+0000 7efec939d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efec41013b0 0x7efec419dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:35.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.102+0000 7efec939d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efec4109fc0 0x7efec4197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:35.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.102+0000 7efec939d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec4198280 con 0x7efec4109fc0 2023-07-18T03:14:35.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.102+0000 7efec939d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec41983f0 con 0x7efec41013b0 2023-07-18T03:14:35.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.102+0000 7efec939d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec41985b0 con 0x7efec4100aa0 2023-07-18T03:14:35.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.103+0000 7efec3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efec4100aa0 0x7efec419da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:35.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.103+0000 7efec3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efec4100aa0 0x7efec419da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:35312/0 (socket says 172.21.15.101:35312) 2023-07-18T03:14:35.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.103+0000 7efec3fff700 1 -- 172.21.15.101:0/3001496283 learned_addr learned my addr 172.21.15.101:0/3001496283 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:35.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.103+0000 7efec8b9c700 1 --2- 172.21.15.101:0/3001496283 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efec4109fc0 0x7efec4197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:35.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.103+0000 7efec37fe700 1 --2- 172.21.15.101:0/3001496283 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efec41013b0 0x7efec419dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:35.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.103+0000 7efec3fff700 1 -- 172.21.15.101:0/3001496283 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efec41013b0 msgr2=0x7efec419dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:35.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.103+0000 7efec3fff700 1 --2- 172.21.15.101:0/3001496283 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efec41013b0 0x7efec419dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.103+0000 7efec3fff700 1 -- 172.21.15.101:0/3001496283 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efec4109fc0 msgr2=0x7efec4197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:35.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.103+0000 7efec3fff700 1 --2- 172.21.15.101:0/3001496283 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efec4109fc0 0x7efec4197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.103+0000 7efec3fff700 1 -- 172.21.15.101:0/3001496283 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb40095f0 con 0x7efec4100aa0 2023-07-18T03:14:35.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.103+0000 7efec3fff700 1 --2- 172.21.15.101:0/3001496283 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efec4100aa0 0x7efec419da60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7efeb400ec10 tx=0x7efeb400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:35.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.104+0000 7efec17fa700 1 -- 172.21.15.101:0/3001496283 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb400cc90 con 0x7efec4100aa0 2023-07-18T03:14:35.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.104+0000 7efec17fa700 1 -- 172.21.15.101:0/3001496283 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efeb40044e0 con 0x7efec4100aa0 2023-07-18T03:14:35.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.104+0000 7efec939d700 1 -- 172.21.15.101:0/3001496283 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efebc009c40 con 0x7efec4100aa0 2023-07-18T03:14:35.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.104+0000 7efec17fa700 1 -- 172.21.15.101:0/3001496283 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb4010430 con 0x7efec4100aa0 2023-07-18T03:14:35.100 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.104+0000 7efec939d700 1 -- 172.21.15.101:0/3001496283 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb8009520 con 0x7efec4100aa0 2023-07-18T03:14:35.100 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.108+0000 7efec17fa700 1 -- 172.21.15.101:0/3001496283 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7efeb4003fb0 con 0x7efec4100aa0 2023-07-18T03:14:35.101 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.109+0000 7efeddfe5b80 1 Processor -- start 2023-07-18T03:14:35.101 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.109+0000 7efeddfe5b80 1 -- start start 2023-07-18T03:14:35.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.109+0000 7efec17fa700 1 --2- 172.21.15.101:0/3001496283 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7efeac0a13e0 0x7efeac0a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:35.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.109+0000 7efeddfe5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5573983824f0 0x5573983828c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:35.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.109+0000 7efeddfe5b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557398382e00 0x55739838c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:35.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.109+0000 7efeddfe5b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55739838cd30 0x55739838f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:35.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.109+0000 7efeddfe5b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557398287cb0 con 0x557398382e00 2023-07-18T03:14:35.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.109+0000 7efeddfe5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573982daa40 con 0x5573983824f0 2023-07-18T03:14:35.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.109+0000 7efeddfe5b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573982d3ed0 con 0x55739838cd30 2023-07-18T03:14:35.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.109+0000 7efec17fa700 1 -- 172.21.15.101:0/3001496283 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9461+0+0 (secure 0 0 0) 0x7efeb4014070 con 0x7efec4100aa0 2023-07-18T03:14:35.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.110+0000 7efec8b9c700 1 --2- 172.21.15.101:0/3001496283 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7efeac0a13e0 0x7efeac0a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:35.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.110+0000 7efec37fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557398382e00 0x55739838c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:35.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.110+0000 7efec37fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557398382e00 0x55739838c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:58936/0 (socket says 172.21.15.101:58936) 2023-07-18T03:14:35.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.110+0000 7efec37fe700 1 -- 172.21.15.101:0/3148329991 learned_addr learned my addr 172.21.15.101:0/3148329991 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:35.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.110+0000 7efec3fff700 1 --2- 172.21.15.101:0/3148329991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5573983824f0 0x5573983828c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:35.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.110+0000 7efec8b9c700 1 --2- 172.21.15.101:0/3148329991 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55739838cd30 0x55739838f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:35.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.110+0000 7efec37fe700 1 -- 172.21.15.101:0/3148329991 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55739838cd30 msgr2=0x55739838f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:35.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.110+0000 7efec37fe700 1 --2- 172.21.15.101:0/3148329991 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55739838cd30 0x55739838f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.110+0000 7efec37fe700 1 -- 172.21.15.101:0/3148329991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5573983824f0 msgr2=0x5573983828c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:35.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.110+0000 7efec37fe700 1 --2- 172.21.15.101:0/3148329991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5573983824f0 0x5573983828c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.110+0000 7efec37fe700 1 -- 172.21.15.101:0/3148329991 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557398101de0 con 0x557398382e00 2023-07-18T03:14:35.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.110+0000 7efec3fff700 1 --2- 172.21.15.101:0/3148329991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5573983824f0 0x5573983828c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:14:35.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.110+0000 7efec37fe700 1 --2- 172.21.15.101:0/3148329991 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557398382e00 0x55739838c7f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7efeb8000c00 tx=0x7efeb8005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a70643ccfc8438c6 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:35.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.111+0000 7efec8b9c700 1 --2- 172.21.15.101:0/3001496283 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7efeac0a13e0 0x7efeac0a3890 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7efebc009fd0 tx=0x7efebc009920 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:35.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.111+0000 7efeaa7fc700 1 -- 172.21.15.101:0/3148329991 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb800f720 con 0x557398382e00 2023-07-18T03:14:35.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.111+0000 7efeaa7fc700 1 -- 172.21.15.101:0/3148329991 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efeb801d040 con 0x557398382e00 2023-07-18T03:14:35.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.111+0000 7efeaa7fc700 1 -- 172.21.15.101:0/3148329991 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb8005540 con 0x557398382e00 2023-07-18T03:14:35.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.111+0000 7efeddfe5b80 1 -- 172.21.15.101:0/3148329991 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557398382e00 msgr2=0x55739838c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:35.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.111+0000 7efeddfe5b80 1 --2- 172.21.15.101:0/3148329991 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557398382e00 0x55739838c7f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7efeb8000c00 tx=0x7efeb8005dc0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.112+0000 7efeddfe5b80 1 -- 172.21.15.101:0/3148329991 shutdown_connections 2023-07-18T03:14:35.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.112+0000 7efeddfe5b80 1 --2- 172.21.15.101:0/3148329991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5573983824f0 0x5573983828c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.112+0000 7efeddfe5b80 1 --2- 172.21.15.101:0/3148329991 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55739838cd30 0x55739838f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.112+0000 7efeddfe5b80 1 --2- 172.21.15.101:0/3148329991 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557398382e00 0x55739838c7f0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.112+0000 7efeddfe5b80 1 -- 172.21.15.101:0/3148329991 >> 172.21.15.101:0/3148329991 conn(0x557398377ff0 msgr2=0x55739837d930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:35.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.112+0000 7efeddfe5b80 1 -- 172.21.15.101:0/3148329991 shutdown_connections 2023-07-18T03:14:35.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.112+0000 7efeddfe5b80 1 -- 172.21.15.101:0/3148329991 wait complete. 2023-07-18T03:14:35.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.112+0000 7efeddfe5b80 1 -- start start 2023-07-18T03:14:35.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.112+0000 7efeddfe5b80 1 Processor -- start 2023-07-18T03:14:35.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.113+0000 7efeddfe5b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5573983824f0 0x55739838c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:35.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.113+0000 7efeddfe5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557398382e00 0x557398389920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:35.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.113+0000 7efeddfe5b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55739838cd30 0x557398389f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:35.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.113+0000 7efeddfe5b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557398273ac0 con 0x5573983824f0 2023-07-18T03:14:35.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.113+0000 7efeddfe5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573982737c0 con 0x557398382e00 2023-07-18T03:14:35.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.113+0000 7efeddfe5b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x557398157920 con 0x55739838cd30 2023-07-18T03:14:35.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.113+0000 7efeddfe5b80 10 client.0 _ll_register_callbacks cb 0x557398271760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:14:35.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.113+0000 7efec37fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557398382e00 0x557398389920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:35.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.113+0000 7efec37fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557398382e00 0x557398389920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47962/0 (socket says 172.21.15.101:47962) 2023-07-18T03:14:35.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.113+0000 7efec37fe700 1 -- 172.21.15.101:0/4158825013 learned_addr learned my addr 172.21.15.101:0/4158825013 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:35.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.113+0000 7efec3fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5573983824f0 0x55739838c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:35.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.113+0000 7efec8b9c700 1 --2- 172.21.15.101:0/4158825013 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55739838cd30 0x557398389f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:35.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.114+0000 7efec37fe700 1 -- 172.21.15.101:0/4158825013 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55739838cd30 msgr2=0x557398389f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:35.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.114+0000 7efec37fe700 1 --2- 172.21.15.101:0/4158825013 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55739838cd30 0x557398389f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.114+0000 7efec37fe700 1 -- 172.21.15.101:0/4158825013 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5573983824f0 msgr2=0x55739838c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:35.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.114+0000 7efec37fe700 1 --2- 172.21.15.101:0/4158825013 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5573983824f0 0x55739838c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.114+0000 7efec37fe700 1 -- 172.21.15.101:0/4158825013 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb8009520 con 0x557398382e00 2023-07-18T03:14:35.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.114+0000 7efec37fe700 1 --2- 172.21.15.101:0/4158825013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557398382e00 0x557398389920 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7efeb8009fd0 tx=0x7efeb800b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1f1b628401876fd server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:35.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.114+0000 7efea37fe700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:14:35.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.114+0000 7efea37fe700 1 -- 172.21.15.101:0/4158825013 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb800f950 con 0x557398382e00 2023-07-18T03:14:35.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.114+0000 7efea37fe700 1 -- 172.21.15.101:0/4158825013 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9461+0+0 (secure 0 0 0) 0x7efeb8005240 con 0x557398382e00 2023-07-18T03:14:35.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.114+0000 7efeddfe5b80 1 -- 172.21.15.101:0/4158825013 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55739822d4f0 con 0x557398382e00 2023-07-18T03:14:35.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.114+0000 7efea37fe700 1 -- 172.21.15.101:0/4158825013 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7efeb800bbd0 con 0x557398382e00 2023-07-18T03:14:35.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.114+0000 7efeddfe5b80 10 client.26984 fetch_fsmap learned FSMap version 29 2023-07-18T03:14:35.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.114+0000 7efeddfe5b80 1 -- 172.21.15.101:0/4158825013 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7efec410d0f0 con 0x557398382e00 2023-07-18T03:14:35.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.115+0000 7efea37fe700 1 -- 172.21.15.101:0/4158825013 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55739822d4f0 con 0x557398382e00 2023-07-18T03:14:35.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.115+0000 7efea37fe700 1 -- 172.21.15.101:0/4158825013 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7efeb8015020 con 0x557398382e00 2023-07-18T03:14:35.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.115+0000 7efeddfe5b80 10 client.26984 fetch_fsmap finished waiting for FSMap version 29 2023-07-18T03:14:35.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.115+0000 7efeddfe5b80 10 client.26984 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:14:35.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.115+0000 7efeddfe5b80 20 client.26984 populate_metadata read hostname 'smithi101' 2023-07-18T03:14:35.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.115+0000 7efeddfe5b80 1 --2- 172.21.15.101:0/4158825013 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55739839c1a0 0x55739839e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:35.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.115+0000 7efeddfe5b80 4 client.26984 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:14:35.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.115+0000 7efeddfe5b80 1 -- 172.21.15.101:0/4158825013 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5573982cceb0 con 0x55739839c1a0 2023-07-18T03:14:35.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.115+0000 7efec3fff700 1 --2- 172.21.15.101:0/4158825013 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55739839c1a0 0x55739839e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:35.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.116+0000 7efec3fff700 1 --2- 172.21.15.101:0/4158825013 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55739839c1a0 0x55739839e580 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=be4bfcc36451eb66 server_cookie=4999243b23cffc25 in_seq=0 out_seq=0 2023-07-18T03:14:35.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.116+0000 7efea37fe700 10 client.26984 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:14:35.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.116+0000 7efea37fe700 1 -- 172.21.15.101:0/4158825013 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5573982cceb0 con 0x55739839c1a0 2023-07-18T03:14:35.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.116+0000 7efea37fe700 10 client.26984 handle_command_reply: tid=0 2023-07-18T03:14:35.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.117+0000 7efeddfe5b80 1 client.26984 shutdown 2023-07-18T03:14:35.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.117+0000 7efeddfe5b80 1 -- 172.21.15.101:0/4158825013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557398382e00 msgr2=0x557398389920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:35.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.117+0000 7efeddfe5b80 1 --2- 172.21.15.101:0/4158825013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557398382e00 0x557398389920 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7efeb8009fd0 tx=0x7efeb800b790 comp rx=0 tx=0).stop 2023-07-18T03:14:35.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.117+0000 7efeddfe5b80 1 -- 172.21.15.101:0/4158825013 shutdown_connections 2023-07-18T03:14:35.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.117+0000 7efeddfe5b80 1 --2- 172.21.15.101:0/4158825013 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55739839c1a0 0x55739839e580 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.117+0000 7efeddfe5b80 1 --2- 172.21.15.101:0/4158825013 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55739838cd30 0x557398389f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.117+0000 7efeddfe5b80 1 --2- 172.21.15.101:0/4158825013 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5573983824f0 0x55739838c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.117+0000 7efeddfe5b80 1 --2- 172.21.15.101:0/4158825013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557398382e00 0x557398389920 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.117+0000 7efeddfe5b80 1 -- 172.21.15.101:0/4158825013 >> 172.21.15.101:0/4158825013 conn(0x557398377ff0 msgr2=0x55739837d520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:35.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.117+0000 7efea37fe700 0 client.26984 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:14:35.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.117+0000 7efeddfe5b80 1 -- 172.21.15.101:0/4158825013 shutdown_connections 2023-07-18T03:14:35.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.117+0000 7efeddfe5b80 1 -- 172.21.15.101:0/4158825013 wait complete. 2023-07-18T03:14:35.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.117+0000 7efeddfe5b80 20 client.26984 trim_cache size 0 max 16384 2023-07-18T03:14:35.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.131+0000 7efeddfe5b80 1 Processor -- start 2023-07-18T03:14:35.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.131+0000 7efeddfe5b80 1 -- start start 2023-07-18T03:14:35.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.132+0000 7efeddfe5b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5573983824f0 0x55739838f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:35.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.132+0000 7efeddfe5b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557398382e00 0x55739838c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:35.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.132+0000 7efeddfe5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55739838cd30 0x557398384cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:35.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.132+0000 7efeddfe5b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557398383440 con 0x5573983824f0 2023-07-18T03:14:35.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.132+0000 7efeddfe5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573983831d0 con 0x55739838cd30 2023-07-18T03:14:35.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.132+0000 7efeddfe5b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x557398387ed0 con 0x557398382e00 2023-07-18T03:14:35.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.132+0000 7efec3fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5573983824f0 0x55739838f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:35.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.132+0000 7efec8b9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55739838cd30 0x557398384cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:35.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.132+0000 7efec3fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5573983824f0 0x55739838f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:58952/0 (socket says 172.21.15.101:58952) 2023-07-18T03:14:35.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.132+0000 7efec3fff700 1 -- 172.21.15.101:0/1264802817 learned_addr learned my addr 172.21.15.101:0/1264802817 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:35.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.132+0000 7efec37fe700 1 --2- 172.21.15.101:0/1264802817 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557398382e00 0x55739838c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:35.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.132+0000 7efec3fff700 1 -- 172.21.15.101:0/1264802817 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557398382e00 msgr2=0x55739838c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:35.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.132+0000 7efec3fff700 1 --2- 172.21.15.101:0/1264802817 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557398382e00 0x55739838c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.132+0000 7efec3fff700 1 -- 172.21.15.101:0/1264802817 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55739838cd30 msgr2=0x557398384cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:35.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.132+0000 7efec3fff700 1 --2- 172.21.15.101:0/1264802817 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55739838cd30 0x557398384cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.133+0000 7efec3fff700 1 -- 172.21.15.101:0/1264802817 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573980eac60 con 0x5573983824f0 2023-07-18T03:14:35.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.133+0000 7efec8b9c700 1 --2- 172.21.15.101:0/1264802817 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55739838cd30 0x557398384cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:14:35.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.133+0000 7efec37fe700 1 --2- 172.21.15.101:0/1264802817 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557398382e00 0x55739838c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:14:35.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.133+0000 7efec3fff700 1 --2- 172.21.15.101:0/1264802817 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5573983824f0 0x55739838f110 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7efeb40967d0 tx=0x7efeb4005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fece210722a39db server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:35.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.133+0000 7efeaaffd700 1 -- 172.21.15.101:0/1264802817 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb4096610 con 0x5573983824f0 2023-07-18T03:14:35.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.133+0000 7efeaaffd700 1 -- 172.21.15.101:0/1264802817 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efeb409a2d0 con 0x5573983824f0 2023-07-18T03:14:35.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.133+0000 7efeaaffd700 1 -- 172.21.15.101:0/1264802817 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb4004260 con 0x5573983824f0 2023-07-18T03:14:35.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.134+0000 7efeddfe5b80 1 -- 172.21.15.101:0/1264802817 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5573983824f0 msgr2=0x55739838f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:35.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.134+0000 7efeddfe5b80 1 --2- 172.21.15.101:0/1264802817 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5573983824f0 0x55739838f110 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7efeb40967d0 tx=0x7efeb4005d30 comp rx=0 tx=0).stop 2023-07-18T03:14:35.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.134+0000 7efeddfe5b80 1 -- 172.21.15.101:0/1264802817 shutdown_connections 2023-07-18T03:14:35.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.134+0000 7efeddfe5b80 1 --2- 172.21.15.101:0/1264802817 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55739838cd30 0x557398384cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.134+0000 7efeddfe5b80 1 --2- 172.21.15.101:0/1264802817 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5573983824f0 0x55739838f110 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.134+0000 7efeddfe5b80 1 --2- 172.21.15.101:0/1264802817 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557398382e00 0x55739838c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.134+0000 7efeddfe5b80 1 -- 172.21.15.101:0/1264802817 >> 172.21.15.101:0/1264802817 conn(0x557398377ff0 msgr2=0x557398398950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:35.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.134+0000 7efeddfe5b80 1 -- 172.21.15.101:0/1264802817 shutdown_connections 2023-07-18T03:14:35.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.134+0000 7efeddfe5b80 1 -- 172.21.15.101:0/1264802817 wait complete. 2023-07-18T03:14:35.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.134+0000 7efeddfe5b80 1 -- start start 2023-07-18T03:14:35.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.135+0000 7efeddfe5b80 1 Processor -- start 2023-07-18T03:14:35.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.135+0000 7efeddfe5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5573983824f0 0x557398393170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:35.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.135+0000 7efeddfe5b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557398382e00 0x5573983936b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:35.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.135+0000 7efeddfe5b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55739838cd30 0x557398391630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:35.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.135+0000 7efeddfe5b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557398391c00 con 0x55739838cd30 2023-07-18T03:14:35.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.135+0000 7efeddfe5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557398391d70 con 0x5573983824f0 2023-07-18T03:14:35.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.135+0000 7efeddfe5b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x557398391f30 con 0x557398382e00 2023-07-18T03:14:35.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.135+0000 7efeddfe5b80 10 client.0 _ll_register_callbacks cb 0x7efeb4009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:14:35.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.135+0000 7efec3fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5573983824f0 0x557398393170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:35.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.136+0000 7efec3fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5573983824f0 0x557398393170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47982/0 (socket says 172.21.15.101:47982) 2023-07-18T03:14:35.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.136+0000 7efec3fff700 1 -- 172.21.15.101:0/3261681312 learned_addr learned my addr 172.21.15.101:0/3261681312 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:35.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.136+0000 7efec8b9c700 1 --2- 172.21.15.101:0/3261681312 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55739838cd30 0x557398391630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:35.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.136+0000 7efec37fe700 1 --2- 172.21.15.101:0/3261681312 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557398382e00 0x5573983936b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:35.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.136+0000 7efec3fff700 1 -- 172.21.15.101:0/3261681312 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557398382e00 msgr2=0x5573983936b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:35.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.136+0000 7efec3fff700 1 --2- 172.21.15.101:0/3261681312 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557398382e00 0x5573983936b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.136+0000 7efec3fff700 1 -- 172.21.15.101:0/3261681312 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55739838cd30 msgr2=0x557398391630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:35.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.136+0000 7efec3fff700 1 --2- 172.21.15.101:0/3261681312 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55739838cd30 0x557398391630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.136+0000 7efec3fff700 1 -- 172.21.15.101:0/3261681312 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efec410d0f0 con 0x5573983824f0 2023-07-18T03:14:35.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.136+0000 7efec3fff700 1 --2- 172.21.15.101:0/3261681312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5573983824f0 0x557398393170 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7efeb409a260 tx=0x7efeb40850c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44137712a3902f46 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:35.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.136+0000 7efea37fe700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:14:35.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.136+0000 7efea37fe700 1 -- 172.21.15.101:0/3261681312 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb409e300 con 0x5573983824f0 2023-07-18T03:14:35.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.136+0000 7efea37fe700 1 -- 172.21.15.101:0/3261681312 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9461+0+0 (secure 0 0 0) 0x7efeb40a2720 con 0x5573983824f0 2023-07-18T03:14:35.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.136+0000 7efeddfe5b80 1 -- 172.21.15.101:0/3261681312 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efeb801d040 con 0x5573983824f0 2023-07-18T03:14:35.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.136+0000 7efea37fe700 1 -- 172.21.15.101:0/3261681312 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7efeb4004260 con 0x5573983824f0 2023-07-18T03:14:35.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.137+0000 7efeddfe5b80 10 client.26990 fetch_fsmap learned FSMap version 29 2023-07-18T03:14:35.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.137+0000 7efeddfe5b80 1 -- 172.21.15.101:0/3261681312 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7efebc008eb0 con 0x5573983824f0 2023-07-18T03:14:35.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.137+0000 7efea37fe700 1 -- 172.21.15.101:0/3261681312 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efeb801d040 con 0x5573983824f0 2023-07-18T03:14:35.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.137+0000 7efea37fe700 1 -- 172.21.15.101:0/3261681312 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7efeb40997c0 con 0x5573983824f0 2023-07-18T03:14:35.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.137+0000 7efeddfe5b80 10 client.26990 fetch_fsmap finished waiting for FSMap version 29 2023-07-18T03:14:35.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.137+0000 7efeddfe5b80 10 client.26990 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:14:35.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.137+0000 7efeddfe5b80 20 client.26990 populate_metadata read hostname 'smithi101' 2023-07-18T03:14:35.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.137+0000 7efeddfe5b80 1 --2- 172.21.15.101:0/3261681312 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5573983f0d30 0x5573983f3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:35.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.137+0000 7efeddfe5b80 4 client.26990 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:14:35.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.137+0000 7efeddfe5b80 1 -- 172.21.15.101:0/3261681312 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55739828f050 con 0x5573983f0d30 2023-07-18T03:14:35.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.137+0000 7efec37fe700 1 --2- 172.21.15.101:0/3261681312 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5573983f0d30 0x5573983f3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:35.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.138+0000 7efec37fe700 1 --2- 172.21.15.101:0/3261681312 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5573983f0d30 0x5573983f3140 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=faeda72ea543341d server_cookie=86602b330de9eb1f in_seq=0 out_seq=0 2023-07-18T03:14:35.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.138+0000 7efea37fe700 10 client.26990 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:14:35.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.138+0000 7efea37fe700 1 -- 172.21.15.101:0/3261681312 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55739828f050 con 0x5573983f0d30 2023-07-18T03:14:35.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.138+0000 7efea37fe700 10 client.26990 handle_command_reply: tid=0 2023-07-18T03:14:35.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.138+0000 7efeddfe5b80 1 client.26990 shutdown 2023-07-18T03:14:35.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.139+0000 7efeddfe5b80 1 -- 172.21.15.101:0/3261681312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5573983824f0 msgr2=0x557398393170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:35.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.139+0000 7efeddfe5b80 1 --2- 172.21.15.101:0/3261681312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5573983824f0 0x557398393170 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7efeb409a260 tx=0x7efeb40850c0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.139+0000 7efeddfe5b80 1 -- 172.21.15.101:0/3261681312 shutdown_connections 2023-07-18T03:14:35.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.139+0000 7efeddfe5b80 1 --2- 172.21.15.101:0/3261681312 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5573983f0d30 0x5573983f3140 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.139+0000 7efeddfe5b80 1 --2- 172.21.15.101:0/3261681312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5573983824f0 0x557398393170 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.139+0000 7efeddfe5b80 1 --2- 172.21.15.101:0/3261681312 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55739838cd30 0x557398391630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.139+0000 7efea37fe700 0 client.26990 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:14:35.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.139+0000 7efeddfe5b80 1 --2- 172.21.15.101:0/3261681312 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557398382e00 0x5573983936b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.139+0000 7efeddfe5b80 1 -- 172.21.15.101:0/3261681312 >> 172.21.15.101:0/3261681312 conn(0x557398377ff0 msgr2=0x557398384cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:35.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.139+0000 7efeddfe5b80 1 -- 172.21.15.101:0/3261681312 shutdown_connections 2023-07-18T03:14:35.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.139+0000 7efeddfe5b80 1 -- 172.21.15.101:0/3261681312 wait complete. 2023-07-18T03:14:35.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.139+0000 7efeddfe5b80 20 client.26990 trim_cache size 0 max 16384 2023-07-18T03:14:35.141 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:14:35.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.140+0000 7efec939d700 1 -- 172.21.15.101:0/3001496283 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7efeac0a13e0 msgr2=0x7efeac0a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:35.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.140+0000 7efec939d700 1 --2- 172.21.15.101:0/3001496283 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7efeac0a13e0 0x7efeac0a3890 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7efebc009fd0 tx=0x7efebc009920 comp rx=0 tx=0).stop 2023-07-18T03:14:35.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.140+0000 7efec939d700 1 -- 172.21.15.101:0/3001496283 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efec4100aa0 msgr2=0x7efec419da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:35.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.140+0000 7efec939d700 1 --2- 172.21.15.101:0/3001496283 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efec4100aa0 0x7efec419da60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7efeb400ec10 tx=0x7efeb400ef20 comp rx=0 tx=0).stop 2023-07-18T03:14:35.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.140+0000 7efec939d700 1 -- 172.21.15.101:0/3001496283 shutdown_connections 2023-07-18T03:14:35.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.140+0000 7efec939d700 1 --2- 172.21.15.101:0/3001496283 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efec4109fc0 0x7efec4197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.140+0000 7efec939d700 1 --2- 172.21.15.101:0/3001496283 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efec4100aa0 0x7efec419da60 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.140+0000 7efec939d700 1 --2- 172.21.15.101:0/3001496283 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7efeac0a13e0 0x7efeac0a3890 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.140+0000 7efec939d700 1 --2- 172.21.15.101:0/3001496283 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efec41013b0 0x7efec419dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.140+0000 7efec939d700 1 -- 172.21.15.101:0/3001496283 >> 172.21.15.101:0/3001496283 conn(0x7efec40745d0 msgr2=0x7efec40feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:35.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.140+0000 7efec939d700 1 -- 172.21.15.101:0/3001496283 shutdown_connections 2023-07-18T03:14:35.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.140+0000 7efec939d700 1 -- 172.21.15.101:0/3001496283 wait complete. 2023-07-18T03:14:35.148 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:34 smithi101 ceph-mon[126379]: pgmap v215: 129 pgs: 129 active+clean; 350 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 1.6 MiB/s wr, 75 op/s 2023-07-18T03:14:35.148 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:34 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3128452292' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:14:35.150 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-18T03:14:35.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:34 smithi155 ceph-mon[128885]: pgmap v215: 129 pgs: 129 active+clean; 350 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 1.6 MiB/s wr, 75 op/s 2023-07-18T03:14:35.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:34 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/3128452292' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:14:35.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.370+0000 7feaff3ff700 1 -- 172.21.15.101:0/4056525998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feaf8105ae0 msgr2=0x7feaf8105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:35.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.370+0000 7feaff3ff700 1 --2- 172.21.15.101:0/4056525998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feaf8105ae0 0x7feaf8105eb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7feaf4009990 tx=0x7feaf4009ca0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.370+0000 7feaff3ff700 1 -- 172.21.15.101:0/4056525998 shutdown_connections 2023-07-18T03:14:35.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.370+0000 7feaff3ff700 1 --2- 172.21.15.101:0/4056525998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feaf8105ae0 0x7feaf8105eb0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.370+0000 7feaff3ff700 1 --2- 172.21.15.101:0/4056525998 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7feaf8100480 0x7feaf810c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.370+0000 7feaff3ff700 1 --2- 172.21.15.101:0/4056525998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feaf80ffad0 0x7feaf80fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.370+0000 7feaff3ff700 1 -- 172.21.15.101:0/4056525998 >> 172.21.15.101:0/4056525998 conn(0x7feaf80fb5f0 msgr2=0x7feaf80fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:35.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.370+0000 7feaff3ff700 1 -- 172.21.15.101:0/4056525998 shutdown_connections 2023-07-18T03:14:35.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.370+0000 7feaff3ff700 1 -- 172.21.15.101:0/4056525998 wait complete. 2023-07-18T03:14:35.365 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.371+0000 7feaff3ff700 1 Processor -- start 2023-07-18T03:14:35.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.371+0000 7feaff3ff700 1 -- start start 2023-07-18T03:14:35.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.372+0000 7feaff3ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feaf80ffad0 0x7feaf8197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:35.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.372+0000 7feaff3ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feaf8100480 0x7feaf8198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:35.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.372+0000 7feaff3ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7feaf8105ae0 0x7feaf819c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:35.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.372+0000 7feaff3ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf819ce10 con 0x7feaf8105ae0 2023-07-18T03:14:35.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.372+0000 7feaff3ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf819cf80 con 0x7feaf8100480 2023-07-18T03:14:35.367 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.372+0000 7feaff3ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf819d140 con 0x7feaf80ffad0 2023-07-18T03:14:35.367 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.372+0000 7feafd99c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7feaf8105ae0 0x7feaf819c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:35.367 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.372+0000 7feafd19b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feaf80ffad0 0x7feaf8197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:35.367 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.372+0000 7feafd19b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feaf80ffad0 0x7feaf8197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:35352/0 (socket says 172.21.15.101:35352) 2023-07-18T03:14:35.367 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.372+0000 7feafd19b700 1 -- 172.21.15.101:0/1431246603 learned_addr learned my addr 172.21.15.101:0/1431246603 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:35.367 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.372+0000 7feafc99a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feaf8100480 0x7feaf8198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:35.367 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.372+0000 7feafd19b700 1 -- 172.21.15.101:0/1431246603 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feaf8100480 msgr2=0x7feaf8198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:35.367 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.372+0000 7feafd19b700 1 --2- 172.21.15.101:0/1431246603 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feaf8100480 0x7feaf8198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.368 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.372+0000 7feafd19b700 1 -- 172.21.15.101:0/1431246603 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7feaf8105ae0 msgr2=0x7feaf819c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:35.368 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.372+0000 7feafd19b700 1 --2- 172.21.15.101:0/1431246603 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7feaf8105ae0 0x7feaf819c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.368 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.372+0000 7feafd19b700 1 -- 172.21.15.101:0/1431246603 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf40095f0 con 0x7feaf80ffad0 2023-07-18T03:14:35.368 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.372+0000 7feafd19b700 1 --2- 172.21.15.101:0/1431246603 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feaf80ffad0 0x7feaf8197e30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7feaf4000c00 tx=0x7feaf4005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:35.368 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.373+0000 7feaee7fc700 1 -- 172.21.15.101:0/1431246603 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf401d070 con 0x7feaf80ffad0 2023-07-18T03:14:35.368 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.373+0000 7feaee7fc700 1 -- 172.21.15.101:0/1431246603 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feaf4022470 con 0x7feaf80ffad0 2023-07-18T03:14:35.368 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.373+0000 7feaff3ff700 1 -- 172.21.15.101:0/1431246603 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaf819d3c0 con 0x7feaf80ffad0 2023-07-18T03:14:35.369 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.373+0000 7feaee7fc700 1 -- 172.21.15.101:0/1431246603 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf4017630 con 0x7feaf80ffad0 2023-07-18T03:14:35.369 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.373+0000 7feaff3ff700 1 -- 172.21.15.101:0/1431246603 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf819d630 con 0x7feaf80ffad0 2023-07-18T03:14:35.369 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.376+0000 7feaee7fc700 1 -- 172.21.15.101:0/1431246603 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7feaf400fad0 con 0x7feaf80ffad0 2023-07-18T03:14:35.370 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.378+0000 7feb12da6b80 1 Processor -- start 2023-07-18T03:14:35.371 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.378+0000 7feb12da6b80 1 -- start start 2023-07-18T03:14:35.371 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.379+0000 7feaee7fc700 1 --2- 172.21.15.101:0/1431246603 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7feae40a12c0 0x7feae40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:35.371 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.379+0000 7feb12da6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d1c78a2d0 0x557d1c78a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:35.371 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.379+0000 7feb12da6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d1c78abe0 0x557d1c795590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:35.373 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.379+0000 7feb12da6b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557d1c795ad0 0x557d1c797eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:35.373 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.379+0000 7feaee7fc700 1 -- 172.21.15.101:0/1431246603 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9461+0+0 (secure 0 0 0) 0x7feaf40ce510 con 0x7feaf80ffad0 2023-07-18T03:14:35.374 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.379+0000 7feb12da6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d1c690cb0 con 0x557d1c78a2d0 2023-07-18T03:14:35.374 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.379+0000 7feb12da6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d1c6e3a40 con 0x557d1c78abe0 2023-07-18T03:14:35.374 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.379+0000 7feb12da6b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d1c6dced0 con 0x557d1c795ad0 2023-07-18T03:14:35.374 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.379+0000 7feafd99c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557d1c795ad0 0x557d1c797eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:35.374 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.379+0000 7feafc99a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d1c78abe0 0x557d1c795590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:35.374 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.379+0000 7feafd19b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d1c78a2d0 0x557d1c78a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:35.375 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.379+0000 7feafd99c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557d1c795ad0 0x557d1c797eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:35368/0 (socket says 172.21.15.101:35368) 2023-07-18T03:14:35.375 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.379+0000 7feafd99c700 1 -- 172.21.15.101:0/1166764875 learned_addr learned my addr 172.21.15.101:0/1166764875 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:35.375 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.379+0000 7feafd99c700 1 --2- 172.21.15.101:0/1431246603 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7feae40a12c0 0x7feae40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:35.375 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.379+0000 7feafc99a700 1 -- 172.21.15.101:0/1166764875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557d1c795ad0 msgr2=0x557d1c797eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:35.375 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.379+0000 7feafc99a700 1 --2- 172.21.15.101:0/1166764875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557d1c795ad0 0x557d1c797eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.375 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.379+0000 7feafc99a700 1 -- 172.21.15.101:0/1166764875 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d1c78a2d0 msgr2=0x557d1c78a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:35.376 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.379+0000 7feafc99a700 1 --2- 172.21.15.101:0/1166764875 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d1c78a2d0 0x557d1c78a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.376 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.379+0000 7feafc99a700 1 -- 172.21.15.101:0/1166764875 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d1c50ade0 con 0x557d1c78abe0 2023-07-18T03:14:35.376 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.379+0000 7feafd19b700 1 --2- 172.21.15.101:0/1166764875 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d1c78a2d0 0x557d1c78a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:14:35.376 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.380+0000 7feafc99a700 1 --2- 172.21.15.101:0/1166764875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d1c78abe0 0x557d1c795590 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7feae8009c80 tx=0x7feae8009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d35f1ef4c674669 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:35.376 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.380+0000 7feafd99c700 1 --2- 172.21.15.101:0/1431246603 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7feae40a12c0 0x7feae40a3770 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7feaf0009fd0 tx=0x7feaf000b040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:35.376 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.380+0000 7feae37fe700 1 -- 172.21.15.101:0/1166764875 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae8011cb0 con 0x557d1c78abe0 2023-07-18T03:14:35.376 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.380+0000 7feae37fe700 1 -- 172.21.15.101:0/1166764875 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feae8011e10 con 0x557d1c78abe0 2023-07-18T03:14:35.377 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.380+0000 7feae37fe700 1 -- 172.21.15.101:0/1166764875 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae8005280 con 0x557d1c78abe0 2023-07-18T03:14:35.377 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.381+0000 7feb12da6b80 1 -- 172.21.15.101:0/1166764875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d1c78abe0 msgr2=0x557d1c795590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:35.377 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.381+0000 7feb12da6b80 1 --2- 172.21.15.101:0/1166764875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d1c78abe0 0x557d1c795590 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7feae8009c80 tx=0x7feae8009f90 comp rx=0 tx=0).stop 2023-07-18T03:14:35.377 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.381+0000 7feb12da6b80 1 -- 172.21.15.101:0/1166764875 shutdown_connections 2023-07-18T03:14:35.377 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.381+0000 7feb12da6b80 1 --2- 172.21.15.101:0/1166764875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557d1c795ad0 0x557d1c797eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.377 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.381+0000 7feb12da6b80 1 --2- 172.21.15.101:0/1166764875 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d1c78a2d0 0x557d1c78a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.377 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.381+0000 7feb12da6b80 1 --2- 172.21.15.101:0/1166764875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d1c78abe0 0x557d1c795590 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.378 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.381+0000 7feb12da6b80 1 -- 172.21.15.101:0/1166764875 >> 172.21.15.101:0/1166764875 conn(0x557d1c780da0 msgr2=0x557d1c7866a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:35.378 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.381+0000 7feb12da6b80 1 -- 172.21.15.101:0/1166764875 shutdown_connections 2023-07-18T03:14:35.378 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.381+0000 7feb12da6b80 1 -- 172.21.15.101:0/1166764875 wait complete. 2023-07-18T03:14:35.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.381+0000 7feb12da6b80 1 -- start start 2023-07-18T03:14:35.381 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.382+0000 7feb12da6b80 1 Processor -- start 2023-07-18T03:14:35.381 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.382+0000 7feb12da6b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557d1c78a2d0 0x557d1c7954c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:35.381 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.382+0000 7feb12da6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d1c78abe0 0x557d1c792740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:35.381 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.382+0000 7feb12da6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d1c795ad0 0x557d1c792dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:35.381 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.382+0000 7feb12da6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d1c67cac0 con 0x557d1c78abe0 2023-07-18T03:14:35.381 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.382+0000 7feb12da6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d1c67c7c0 con 0x557d1c795ad0 2023-07-18T03:14:35.381 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.382+0000 7feb12da6b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d1c560920 con 0x557d1c78a2d0 2023-07-18T03:14:35.382 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.382+0000 7feb12da6b80 10 client.0 _ll_register_callbacks cb 0x557d1c67a760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:14:35.382 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.382+0000 7feafd19b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557d1c78a2d0 0x557d1c7954c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:35.382 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.382+0000 7feafc99a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d1c78abe0 0x557d1c792740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:35.382 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.382+0000 7feafd19b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557d1c78a2d0 0x557d1c7954c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:35372/0 (socket says 172.21.15.101:35372) 2023-07-18T03:14:35.382 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.382+0000 7feafd19b700 1 -- 172.21.15.101:0/1121407501 learned_addr learned my addr 172.21.15.101:0/1121407501 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:35.382 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.382+0000 7feafd99c700 1 --2- 172.21.15.101:0/1121407501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d1c795ad0 0x557d1c792dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:35.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.383+0000 7feafd19b700 1 -- 172.21.15.101:0/1121407501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d1c795ad0 msgr2=0x557d1c792dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:35.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.383+0000 7feafd19b700 1 --2- 172.21.15.101:0/1121407501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d1c795ad0 0x557d1c792dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.383+0000 7feafd19b700 1 -- 172.21.15.101:0/1121407501 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d1c78abe0 msgr2=0x557d1c792740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:35.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.383+0000 7feafd19b700 1 --2- 172.21.15.101:0/1121407501 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d1c78abe0 0x557d1c792740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.383+0000 7feafd19b700 1 -- 172.21.15.101:0/1121407501 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf4095e70 con 0x557d1c78a2d0 2023-07-18T03:14:35.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.383+0000 7feafc99a700 1 --2- 172.21.15.101:0/1121407501 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d1c78abe0 0x557d1c792740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:14:35.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.383+0000 7feafd19b700 1 --2- 172.21.15.101:0/1121407501 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557d1c78a2d0 0x557d1c7954c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7feaf40cb8b0 tx=0x7feaf40cbdb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=845826badd4ef4b4 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:35.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.383+0000 7feae0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:14:35.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.383+0000 7feae0ff9700 1 -- 172.21.15.101:0/1121407501 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf409abc0 con 0x557d1c78a2d0 2023-07-18T03:14:35.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.383+0000 7feb12da6b80 1 -- 172.21.15.101:0/1121407501 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557d1c6364f0 con 0x557d1c78a2d0 2023-07-18T03:14:35.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.383+0000 7feae0ff9700 1 -- 172.21.15.101:0/1121407501 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9461+0+0 (secure 0 0 0) 0x7feaf409b260 con 0x557d1c78a2d0 2023-07-18T03:14:35.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.383+0000 7feae0ff9700 1 -- 172.21.15.101:0/1121407501 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7feaf409f0d0 con 0x557d1c78a2d0 2023-07-18T03:14:35.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.383+0000 7feb12da6b80 10 client.27043 fetch_fsmap learned FSMap version 29 2023-07-18T03:14:35.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.383+0000 7feb12da6b80 1 -- 172.21.15.101:0/1121407501 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7feae80098e0 con 0x557d1c78a2d0 2023-07-18T03:14:35.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.384+0000 7feae0ff9700 1 -- 172.21.15.101:0/1121407501 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557d1c6364f0 con 0x557d1c78a2d0 2023-07-18T03:14:35.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.384+0000 7feae0ff9700 1 -- 172.21.15.101:0/1121407501 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7feaf40ce150 con 0x557d1c78a2d0 2023-07-18T03:14:35.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.384+0000 7feb12da6b80 10 client.27043 fetch_fsmap finished waiting for FSMap version 29 2023-07-18T03:14:35.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.384+0000 7feb12da6b80 10 client.27043 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:14:35.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.384+0000 7feb12da6b80 20 client.27043 populate_metadata read hostname 'smithi101' 2023-07-18T03:14:35.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.384+0000 7feb12da6b80 1 --2- 172.21.15.101:0/1121407501 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557d1c7a4ff0 0x557d1c7a7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:35.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.384+0000 7feb12da6b80 4 client.27043 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:14:35.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.384+0000 7feb12da6b80 1 -- 172.21.15.101:0/1121407501 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557d1c6d5eb0 con 0x557d1c7a4ff0 2023-07-18T03:14:35.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.384+0000 7feafc99a700 1 --2- 172.21.15.101:0/1121407501 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557d1c7a4ff0 0x557d1c7a7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:35.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.385+0000 7feae0ff9700 10 client.27043 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:14:35.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.385+0000 7feafc99a700 1 --2- 172.21.15.101:0/1121407501 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557d1c7a4ff0 0x557d1c7a7400 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cddf655b5374829d server_cookie=c7ecc5bd73a47ed7 in_seq=0 out_seq=0 2023-07-18T03:14:35.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.385+0000 7feae0ff9700 1 -- 172.21.15.101:0/1121407501 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557d1c6d5eb0 con 0x557d1c7a4ff0 2023-07-18T03:14:35.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.385+0000 7feae0ff9700 10 client.27043 handle_command_reply: tid=0 2023-07-18T03:14:35.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.386+0000 7feb12da6b80 1 client.27043 shutdown 2023-07-18T03:14:35.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.386+0000 7feb12da6b80 1 -- 172.21.15.101:0/1121407501 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557d1c78a2d0 msgr2=0x557d1c7954c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:35.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.386+0000 7feb12da6b80 1 --2- 172.21.15.101:0/1121407501 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557d1c78a2d0 0x557d1c7954c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7feaf40cb8b0 tx=0x7feaf40cbdb0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.386+0000 7feb12da6b80 1 -- 172.21.15.101:0/1121407501 shutdown_connections 2023-07-18T03:14:35.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.386+0000 7feb12da6b80 1 --2- 172.21.15.101:0/1121407501 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557d1c7a4ff0 0x557d1c7a7400 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.386+0000 7feb12da6b80 1 --2- 172.21.15.101:0/1121407501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d1c795ad0 0x557d1c792dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.386+0000 7feae0ff9700 0 client.27043 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:14:35.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.386+0000 7feb12da6b80 1 --2- 172.21.15.101:0/1121407501 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557d1c78a2d0 0x557d1c7954c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.386+0000 7feb12da6b80 1 --2- 172.21.15.101:0/1121407501 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d1c78abe0 0x557d1c792740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.386+0000 7feb12da6b80 1 -- 172.21.15.101:0/1121407501 >> 172.21.15.101:0/1121407501 conn(0x557d1c780da0 msgr2=0x557d1c786230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:35.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.386+0000 7feb12da6b80 1 -- 172.21.15.101:0/1121407501 shutdown_connections 2023-07-18T03:14:35.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.386+0000 7feb12da6b80 1 -- 172.21.15.101:0/1121407501 wait complete. 2023-07-18T03:14:35.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.386+0000 7feb12da6b80 20 client.27043 trim_cache size 0 max 16384 2023-07-18T03:14:35.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.400+0000 7feb12da6b80 1 Processor -- start 2023-07-18T03:14:35.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.400+0000 7feb12da6b80 1 -- start start 2023-07-18T03:14:35.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.400+0000 7feb12da6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d1c78a2d0 0x557d1c79c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:35.394 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.400+0000 7feb12da6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d1c78abe0 0x557d1c795060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:35.394 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.400+0000 7feb12da6b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557d1c795ad0 0x557d1c791f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:35.394 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.400+0000 7feb12da6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d1c790d00 con 0x557d1c78abe0 2023-07-18T03:14:35.394 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.400+0000 7feb12da6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d1c79c9e0 con 0x557d1c78a2d0 2023-07-18T03:14:35.394 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.400+0000 7feb12da6b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d1c7ac120 con 0x557d1c795ad0 2023-07-18T03:14:35.394 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.400+0000 7feafd19b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d1c78a2d0 0x557d1c79c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:35.394 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.400+0000 7feafd19b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d1c78a2d0 0x557d1c79c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:48038/0 (socket says 172.21.15.101:48038) 2023-07-18T03:14:35.395 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.400+0000 7feafd19b700 1 -- 172.21.15.101:0/1227072403 learned_addr learned my addr 172.21.15.101:0/1227072403 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:35.395 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.400+0000 7feafc99a700 1 --2- 172.21.15.101:0/1227072403 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d1c78abe0 0x557d1c795060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:35.395 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.400+0000 7feafd99c700 1 --2- 172.21.15.101:0/1227072403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557d1c795ad0 0x557d1c791f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:35.395 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.401+0000 7feafd19b700 1 -- 172.21.15.101:0/1227072403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557d1c795ad0 msgr2=0x557d1c791f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:35.395 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.401+0000 7feafd19b700 1 --2- 172.21.15.101:0/1227072403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557d1c795ad0 0x557d1c791f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.395 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.401+0000 7feafd19b700 1 -- 172.21.15.101:0/1227072403 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d1c78abe0 msgr2=0x557d1c795060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:35.396 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.401+0000 7feafd19b700 1 --2- 172.21.15.101:0/1227072403 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d1c78abe0 0x557d1c795060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.396 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.401+0000 7feafd19b700 1 -- 172.21.15.101:0/1227072403 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d1c4f3c60 con 0x557d1c78a2d0 2023-07-18T03:14:35.396 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.401+0000 7feafd99c700 1 --2- 172.21.15.101:0/1227072403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557d1c795ad0 0x557d1c791f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:14:35.396 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.401+0000 7feafd19b700 1 --2- 172.21.15.101:0/1227072403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d1c78a2d0 0x557d1c79c4a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7feaf4097760 tx=0x7feaf4022af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=834a6f3c0a722d8f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:35.396 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.401+0000 7feae3fff700 1 -- 172.21.15.101:0/1227072403 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf409f0d0 con 0x557d1c78a2d0 2023-07-18T03:14:35.396 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.401+0000 7feae3fff700 1 -- 172.21.15.101:0/1227072403 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feaf409abc0 con 0x557d1c78a2d0 2023-07-18T03:14:35.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.401+0000 7feae3fff700 1 -- 172.21.15.101:0/1227072403 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf40a3620 con 0x557d1c78a2d0 2023-07-18T03:14:35.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.402+0000 7feb12da6b80 1 -- 172.21.15.101:0/1227072403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d1c78a2d0 msgr2=0x557d1c79c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:35.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.402+0000 7feb12da6b80 1 --2- 172.21.15.101:0/1227072403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d1c78a2d0 0x557d1c79c4a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7feaf4097760 tx=0x7feaf4022af0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.402+0000 7feb12da6b80 1 -- 172.21.15.101:0/1227072403 shutdown_connections 2023-07-18T03:14:35.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.402+0000 7feb12da6b80 1 --2- 172.21.15.101:0/1227072403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d1c78a2d0 0x557d1c79c4a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.402+0000 7feb12da6b80 1 --2- 172.21.15.101:0/1227072403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557d1c795ad0 0x557d1c791f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.402+0000 7feb12da6b80 1 --2- 172.21.15.101:0/1227072403 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d1c78abe0 0x557d1c795060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.402+0000 7feb12da6b80 1 -- 172.21.15.101:0/1227072403 >> 172.21.15.101:0/1227072403 conn(0x557d1c780da0 msgr2=0x557d1c78dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:35.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.402+0000 7feb12da6b80 1 -- 172.21.15.101:0/1227072403 shutdown_connections 2023-07-18T03:14:35.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.402+0000 7feb12da6b80 1 -- 172.21.15.101:0/1227072403 wait complete. 2023-07-18T03:14:35.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.402+0000 7feb12da6b80 1 -- start start 2023-07-18T03:14:35.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.402+0000 7feb12da6b80 1 Processor -- start 2023-07-18T03:14:35.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.403+0000 7feb12da6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d1c78a2d0 0x557d1c78fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:35.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.403+0000 7feb12da6b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557d1c78abe0 0x557d1c7902b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:35.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.403+0000 7feb12da6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d1c795ad0 0x557d1c78ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:35.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.403+0000 7feb12da6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d1c78ef60 con 0x557d1c78a2d0 2023-07-18T03:14:35.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.403+0000 7feb12da6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d1c78f0d0 con 0x557d1c795ad0 2023-07-18T03:14:35.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.403+0000 7feb12da6b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d1c78f290 con 0x557d1c78abe0 2023-07-18T03:14:35.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.403+0000 7feb12da6b80 10 client.0 _ll_register_callbacks cb 0x7feaf40975a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:14:35.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.403+0000 7feafd19b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d1c78a2d0 0x557d1c78fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:35.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.403+0000 7feafc99a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557d1c78abe0 0x557d1c7902b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:35.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.403+0000 7feafd19b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d1c78a2d0 0x557d1c78fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:59020/0 (socket says 172.21.15.101:59020) 2023-07-18T03:14:35.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.403+0000 7feafd19b700 1 -- 172.21.15.101:0/3537811913 learned_addr learned my addr 172.21.15.101:0/3537811913 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:35.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.403+0000 7feafd99c700 1 --2- 172.21.15.101:0/3537811913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d1c795ad0 0x557d1c78ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:35.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.404+0000 7feafd19b700 1 -- 172.21.15.101:0/3537811913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557d1c78abe0 msgr2=0x557d1c7902b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:35.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.404+0000 7feafd19b700 1 --2- 172.21.15.101:0/3537811913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557d1c78abe0 0x557d1c7902b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.404+0000 7feafd19b700 1 -- 172.21.15.101:0/3537811913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d1c795ad0 msgr2=0x557d1c78ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:35.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.404+0000 7feafd19b700 1 --2- 172.21.15.101:0/3537811913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d1c795ad0 0x557d1c78ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.404+0000 7feafd19b700 1 -- 172.21.15.101:0/3537811913 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feae80098e0 con 0x557d1c78a2d0 2023-07-18T03:14:35.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.404+0000 7feafd99c700 1 --2- 172.21.15.101:0/3537811913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d1c795ad0 0x557d1c78ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:14:35.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.404+0000 7feafd19b700 1 --2- 172.21.15.101:0/3537811913 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d1c78a2d0 0x557d1c78fd70 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7feaf409af60 tx=0x7feaf40cbdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b05e34bf3054dfe server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:35.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.404+0000 7feae0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:14:35.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.404+0000 7feae0ff9700 1 -- 172.21.15.101:0/3537811913 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf40a1110 con 0x557d1c78a2d0 2023-07-18T03:14:35.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.404+0000 7feae0ff9700 1 -- 172.21.15.101:0/3537811913 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9461+0+0 (secure 0 0 0) 0x7feaf40a3620 con 0x557d1c78a2d0 2023-07-18T03:14:35.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.404+0000 7feb12da6b80 1 -- 172.21.15.101:0/3537811913 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feae8011e10 con 0x557d1c78a2d0 2023-07-18T03:14:35.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.404+0000 7feae0ff9700 1 -- 172.21.15.101:0/3537811913 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7feaf409f290 con 0x557d1c78a2d0 2023-07-18T03:14:35.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.404+0000 7feb12da6b80 10 client.16968 fetch_fsmap learned FSMap version 29 2023-07-18T03:14:35.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.404+0000 7feb12da6b80 1 -- 172.21.15.101:0/3537811913 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x557d1c531250 con 0x557d1c78a2d0 2023-07-18T03:14:35.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.405+0000 7feae0ff9700 1 -- 172.21.15.101:0/3537811913 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feae8011e10 con 0x557d1c78a2d0 2023-07-18T03:14:35.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.405+0000 7feae0ff9700 1 -- 172.21.15.101:0/3537811913 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7feaf40b3bb0 con 0x557d1c78a2d0 2023-07-18T03:14:35.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.405+0000 7feb12da6b80 10 client.16968 fetch_fsmap finished waiting for FSMap version 29 2023-07-18T03:14:35.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.405+0000 7feb12da6b80 10 client.16968 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:14:35.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.405+0000 7feb12da6b80 20 client.16968 populate_metadata read hostname 'smithi101' 2023-07-18T03:14:35.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.405+0000 7feb12da6b80 1 --2- 172.21.15.101:0/3537811913 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557d1c7f9840 0x557d1c7fbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:35.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.405+0000 7feb12da6b80 4 client.16968 mds_command: new command op to 24562 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:14:35.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.405+0000 7feb12da6b80 1 -- 172.21.15.101:0/3537811913 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557d1c698050 con 0x557d1c7f9840 2023-07-18T03:14:35.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.405+0000 7feafc99a700 1 --2- 172.21.15.101:0/3537811913 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557d1c7f9840 0x557d1c7fbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:35.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.406+0000 7feae0ff9700 10 client.16968 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:14:35.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.406+0000 7feafc99a700 1 --2- 172.21.15.101:0/3537811913 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557d1c7f9840 0x557d1c7fbc20 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=844bf4161bde239b server_cookie=f1ac46a603f72804 in_seq=0 out_seq=0 2023-07-18T03:14:35.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.406+0000 7feae0ff9700 1 -- 172.21.15.101:0/3537811913 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557d1c698050 con 0x557d1c7f9840 2023-07-18T03:14:35.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.406+0000 7feae0ff9700 10 client.16968 handle_command_reply: tid=0 2023-07-18T03:14:35.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.406+0000 7feb12da6b80 1 client.16968 shutdown 2023-07-18T03:14:35.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.406+0000 7feb12da6b80 1 -- 172.21.15.101:0/3537811913 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d1c78a2d0 msgr2=0x557d1c78fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:35.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.406+0000 7feb12da6b80 1 --2- 172.21.15.101:0/3537811913 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d1c78a2d0 0x557d1c78fd70 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7feaf409af60 tx=0x7feaf40cbdb0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.406+0000 7feb12da6b80 1 -- 172.21.15.101:0/3537811913 shutdown_connections 2023-07-18T03:14:35.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.406+0000 7feb12da6b80 1 --2- 172.21.15.101:0/3537811913 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557d1c7f9840 0x557d1c7fbc20 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.406+0000 7feb12da6b80 1 --2- 172.21.15.101:0/3537811913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d1c795ad0 0x557d1c78ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.406+0000 7feae0ff9700 0 client.16968 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:14:35.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.406+0000 7feb12da6b80 1 --2- 172.21.15.101:0/3537811913 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d1c78a2d0 0x557d1c78fd70 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.406+0000 7feb12da6b80 1 --2- 172.21.15.101:0/3537811913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557d1c78abe0 0x557d1c7902b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.406+0000 7feb12da6b80 1 -- 172.21.15.101:0/3537811913 >> 172.21.15.101:0/3537811913 conn(0x557d1c780da0 msgr2=0x557d1c7858b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:35.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.407+0000 7feb12da6b80 1 -- 172.21.15.101:0/3537811913 shutdown_connections 2023-07-18T03:14:35.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.407+0000 7feb12da6b80 1 -- 172.21.15.101:0/3537811913 wait complete. 2023-07-18T03:14:35.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.407+0000 7feb12da6b80 20 client.16968 trim_cache size 0 max 16384 2023-07-18T03:14:35.408 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:14:35.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.407+0000 7feaff3ff700 1 -- 172.21.15.101:0/1431246603 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7feae40a12c0 msgr2=0x7feae40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:35.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.407+0000 7feaff3ff700 1 --2- 172.21.15.101:0/1431246603 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7feae40a12c0 0x7feae40a3770 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7feaf0009fd0 tx=0x7feaf000b040 comp rx=0 tx=0).stop 2023-07-18T03:14:35.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.407+0000 7feaff3ff700 1 -- 172.21.15.101:0/1431246603 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feaf80ffad0 msgr2=0x7feaf8197e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:35.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.407+0000 7feaff3ff700 1 --2- 172.21.15.101:0/1431246603 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feaf80ffad0 0x7feaf8197e30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7feaf4000c00 tx=0x7feaf4005720 comp rx=0 tx=0).stop 2023-07-18T03:14:35.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.407+0000 7feaff3ff700 1 -- 172.21.15.101:0/1431246603 shutdown_connections 2023-07-18T03:14:35.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.407+0000 7feaff3ff700 1 --2- 172.21.15.101:0/1431246603 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7feaf8105ae0 0x7feaf819c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.407+0000 7feaff3ff700 1 --2- 172.21.15.101:0/1431246603 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feaf80ffad0 0x7feaf8197e30 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.407+0000 7feaff3ff700 1 --2- 172.21.15.101:0/1431246603 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7feae40a12c0 0x7feae40a3770 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.407+0000 7feaff3ff700 1 --2- 172.21.15.101:0/1431246603 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feaf8100480 0x7feaf8198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:35.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.407+0000 7feaff3ff700 1 -- 172.21.15.101:0/1431246603 >> 172.21.15.101:0/1431246603 conn(0x7feaf80fb5f0 msgr2=0x7feaf810aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:35.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.408+0000 7feaff3ff700 1 -- 172.21.15.101:0/1431246603 shutdown_connections 2023-07-18T03:14:35.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:35.408+0000 7feaff3ff700 1 -- 172.21.15.101:0/1431246603 wait complete. 2023-07-18T03:14:35.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:34 smithi179 ceph-mon[129229]: pgmap v215: 129 pgs: 129 active+clean; 350 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 1.6 MiB/s wr, 75 op/s 2023-07-18T03:14:35.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:34 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/3128452292' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:14:35.589 INFO:tasks.workunit.client.0.smithi101.stdout: 1 506150 7.37 MB/sec execute 240 sec latency 33.503 ms 2023-07-18T03:14:36.419 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-18T03:14:36.419 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-18T03:14:36.589 INFO:tasks.workunit.client.0.smithi101.stdout: 1 506183 7.34 MB/sec execute 241 sec latency 980.586 ms 2023-07-18T03:14:36.632 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.639+0000 7f60eb5ff700 1 -- 172.21.15.101:0/321550703 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f60e4100490 msgr2=0x7f60e410c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:36.632 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.639+0000 7f60eb5ff700 1 --2- 172.21.15.101:0/321550703 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f60e4100490 0x7f60e410c920 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f60dc00b0a0 tx=0x7f60dc00b3b0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.639+0000 7f60eb5ff700 1 -- 172.21.15.101:0/321550703 shutdown_connections 2023-07-18T03:14:36.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.639+0000 7f60eb5ff700 1 --2- 172.21.15.101:0/321550703 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f60e4100490 0x7f60e410c920 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.639+0000 7f60eb5ff700 1 --2- 172.21.15.101:0/321550703 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f60e4105af0 0x7f60e4105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.639+0000 7f60eb5ff700 1 --2- 172.21.15.101:0/321550703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60e40ffae0 0x7f60e40fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.639+0000 7f60eb5ff700 1 -- 172.21.15.101:0/321550703 >> 172.21.15.101:0/321550703 conn(0x7f60e40fb5e0 msgr2=0x7f60e40fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:36.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.639+0000 7f60eb5ff700 1 -- 172.21.15.101:0/321550703 shutdown_connections 2023-07-18T03:14:36.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.639+0000 7f60eb5ff700 1 -- 172.21.15.101:0/321550703 wait complete. 2023-07-18T03:14:36.635 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.640+0000 7f60eb5ff700 1 Processor -- start 2023-07-18T03:14:36.635 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.640+0000 7f60eb5ff700 1 -- start start 2023-07-18T03:14:36.635 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.641+0000 7f60eb5ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f60e40ffae0 0x7f60e4197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:36.636 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.641+0000 7f60eb5ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f60e4100490 0x7f60e41984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:36.636 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.641+0000 7f60eb5ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60e4105af0 0x7f60e419c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:36.636 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.641+0000 7f60eb5ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e419cf70 con 0x7f60e4100490 2023-07-18T03:14:36.636 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.641+0000 7f60eb5ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e419d0e0 con 0x7f60e4105af0 2023-07-18T03:14:36.636 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.641+0000 7f60eb5ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e419d2a0 con 0x7f60e40ffae0 2023-07-18T03:14:36.636 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.641+0000 7f60e8b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f60e4100490 0x7f60e41984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:36.636 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.641+0000 7f60e9b9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60e4105af0 0x7f60e419c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:36.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.641+0000 7f60e9b9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60e4105af0 0x7f60e419c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:49270/0 (socket says 172.21.15.101:49270) 2023-07-18T03:14:36.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.641+0000 7f60e9b9c700 1 -- 172.21.15.101:0/1593232164 learned_addr learned my addr 172.21.15.101:0/1593232164 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:36.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.641+0000 7f60e8b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f60e4100490 0x7f60e41984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:54754/0 (socket says 172.21.15.101:54754) 2023-07-18T03:14:36.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.641+0000 7f60e939b700 1 --2- 172.21.15.101:0/1593232164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f60e40ffae0 0x7f60e4197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:36.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.641+0000 7f60e8b9a700 1 -- 172.21.15.101:0/1593232164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f60e40ffae0 msgr2=0x7f60e4197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:36.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.641+0000 7f60e8b9a700 1 --2- 172.21.15.101:0/1593232164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f60e40ffae0 0x7f60e4197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.641+0000 7f60e8b9a700 1 -- 172.21.15.101:0/1593232164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60e4105af0 msgr2=0x7f60e419c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:36.638 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.641+0000 7f60e8b9a700 1 --2- 172.21.15.101:0/1593232164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60e4105af0 0x7f60e419c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.638 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.641+0000 7f60e8b9a700 1 -- 172.21.15.101:0/1593232164 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60dc009d00 con 0x7f60e4100490 2023-07-18T03:14:36.638 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.641+0000 7f60e9b9c700 1 --2- 172.21.15.101:0/1593232164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60e4105af0 0x7f60e419c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:14:36.638 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.642+0000 7f60e8b9a700 1 --2- 172.21.15.101:0/1593232164 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f60e4100490 0x7f60e41984d0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f60d400d950 tx=0x7f60d400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:36.638 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.642+0000 7f60da7fc700 1 -- 172.21.15.101:0/1593232164 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d400f890 con 0x7f60e4100490 2023-07-18T03:14:36.638 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.642+0000 7f60eb5ff700 1 -- 172.21.15.101:0/1593232164 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60e419d580 con 0x7f60e4100490 2023-07-18T03:14:36.638 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.642+0000 7f60da7fc700 1 -- 172.21.15.101:0/1593232164 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60d4010460 con 0x7f60e4100490 2023-07-18T03:14:36.639 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.642+0000 7f60da7fc700 1 -- 172.21.15.101:0/1593232164 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d400e610 con 0x7f60e4100490 2023-07-18T03:14:36.639 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.642+0000 7f60eb5ff700 1 -- 172.21.15.101:0/1593232164 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60e41a4dc0 con 0x7f60e4100490 2023-07-18T03:14:36.641 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.645+0000 7f60da7fc700 1 -- 172.21.15.101:0/1593232164 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f60d40108d0 con 0x7f60e4100490 2023-07-18T03:14:36.641 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.646+0000 7f60fef46b80 1 Processor -- start 2023-07-18T03:14:36.641 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.646+0000 7f60fef46b80 1 -- start start 2023-07-18T03:14:36.641 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.647+0000 7f60da7fc700 1 --2- 172.21.15.101:0/1593232164 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f60d00a12c0 0x7f60d00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:36.641 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.647+0000 7f60da7fc700 1 -- 172.21.15.101:0/1593232164 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f60d40ccda0 con 0x7f60e4100490 2023-07-18T03:14:36.641 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.647+0000 7f60fef46b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebebb1b350 0x55ebebb1b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:36.642 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.647+0000 7f60fef46b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebebb1bc60 0x55ebebb26610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:36.642 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.647+0000 7f60fef46b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ebebb26b50 0x55ebebb28f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:36.642 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.647+0000 7f60fef46b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebeba21d30 con 0x55ebebb26b50 2023-07-18T03:14:36.642 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.647+0000 7f60fef46b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebeba74ac0 con 0x55ebebb1b350 2023-07-18T03:14:36.642 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.647+0000 7f60fef46b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebeba6df50 con 0x55ebebb1bc60 2023-07-18T03:14:36.642 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.647+0000 7f60e9b9c700 1 --2- 172.21.15.101:0/1593232164 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f60d00a12c0 0x7f60d00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:36.642 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.647+0000 7f60e939b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebebb1b350 0x55ebebb1b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:36.643 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.647+0000 7f60e939b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebebb1b350 0x55ebebb1b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:49282/0 (socket says 172.21.15.101:49282) 2023-07-18T03:14:36.643 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.647+0000 7f60e939b700 1 -- 172.21.15.101:0/451682812 learned_addr learned my addr 172.21.15.101:0/451682812 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:36.643 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.647+0000 7f60e9b9c700 1 --2- 172.21.15.101:0/451682812 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ebebb26b50 0x55ebebb28f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:36.643 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.647+0000 7f60e8b9a700 1 --2- 172.21.15.101:0/451682812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebebb1bc60 0x55ebebb26610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:36.643 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.648+0000 7f60e939b700 1 -- 172.21.15.101:0/451682812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebebb1bc60 msgr2=0x55ebebb26610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:36.643 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.648+0000 7f60e939b700 1 --2- 172.21.15.101:0/451682812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebebb1bc60 0x55ebebb26610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.643 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.648+0000 7f60e939b700 1 -- 172.21.15.101:0/451682812 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ebebb26b50 msgr2=0x55ebebb28f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:36.644 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.648+0000 7f60e939b700 1 --2- 172.21.15.101:0/451682812 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ebebb26b50 0x55ebebb28f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.644 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.648+0000 7f60e939b700 1 -- 172.21.15.101:0/451682812 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ebeb89be60 con 0x55ebebb1b350 2023-07-18T03:14:36.644 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.648+0000 7f60e939b700 1 --2- 172.21.15.101:0/451682812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebebb1b350 0x55ebebb1b720 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f60e0007c70 tx=0x7f60e0007f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5c5820bfca672c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:36.644 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.648+0000 7f60e9b9c700 1 --2- 172.21.15.101:0/1593232164 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f60d00a12c0 0x7f60d00a3770 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f60dc009cd0 tx=0x7f60dc00cea0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:36.644 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.648+0000 7f60cf7fe700 1 -- 172.21.15.101:0/451682812 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e0010be0 con 0x55ebebb1b350 2023-07-18T03:14:36.644 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.648+0000 7f60cf7fe700 1 -- 172.21.15.101:0/451682812 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60e0015070 con 0x55ebebb1b350 2023-07-18T03:14:36.644 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.648+0000 7f60cf7fe700 1 -- 172.21.15.101:0/451682812 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e0019410 con 0x55ebebb1b350 2023-07-18T03:14:36.648 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.649+0000 7f60fef46b80 1 -- 172.21.15.101:0/451682812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebebb1b350 msgr2=0x55ebebb1b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:36.648 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.649+0000 7f60fef46b80 1 --2- 172.21.15.101:0/451682812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebebb1b350 0x55ebebb1b720 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f60e0007c70 tx=0x7f60e0007f80 comp rx=0 tx=0).stop 2023-07-18T03:14:36.648 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.649+0000 7f60fef46b80 1 -- 172.21.15.101:0/451682812 shutdown_connections 2023-07-18T03:14:36.648 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.649+0000 7f60fef46b80 1 --2- 172.21.15.101:0/451682812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebebb1b350 0x55ebebb1b720 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.649+0000 7f60fef46b80 1 --2- 172.21.15.101:0/451682812 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ebebb26b50 0x55ebebb28f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.649+0000 7f60fef46b80 1 --2- 172.21.15.101:0/451682812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebebb1bc60 0x55ebebb26610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.649+0000 7f60fef46b80 1 -- 172.21.15.101:0/451682812 >> 172.21.15.101:0/451682812 conn(0x55ebebb11e20 msgr2=0x55ebebb17720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:36.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.649+0000 7f60fef46b80 1 -- 172.21.15.101:0/451682812 shutdown_connections 2023-07-18T03:14:36.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.649+0000 7f60fef46b80 1 -- 172.21.15.101:0/451682812 wait complete. 2023-07-18T03:14:36.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.649+0000 7f60fef46b80 1 -- start start 2023-07-18T03:14:36.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.650+0000 7f60fef46b80 1 Processor -- start 2023-07-18T03:14:36.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.650+0000 7f60fef46b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebebb1b350 0x55ebebb26590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:36.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.650+0000 7f60fef46b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ebebb1bc60 0x55ebebb23810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:36.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.650+0000 7f60fef46b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebebb26b50 0x55ebebb23f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:36.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.650+0000 7f60fef46b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebeba0db40 con 0x55ebebb1bc60 2023-07-18T03:14:36.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.650+0000 7f60fef46b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebeba0d840 con 0x55ebebb1b350 2023-07-18T03:14:36.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.650+0000 7f60fef46b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebeb8f19a0 con 0x55ebebb26b50 2023-07-18T03:14:36.651 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.650+0000 7f60fef46b80 10 client.0 _ll_register_callbacks cb 0x55ebeba0b7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:14:36.651 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.650+0000 7f60e939b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebebb1b350 0x55ebebb26590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:36.651 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.651+0000 7f60e939b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebebb1b350 0x55ebebb26590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:49288/0 (socket says 172.21.15.101:49288) 2023-07-18T03:14:36.651 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.651+0000 7f60e939b700 1 -- 172.21.15.101:0/766949717 learned_addr learned my addr 172.21.15.101:0/766949717 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:36.651 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.651+0000 7f60e9b9c700 1 --2- 172.21.15.101:0/766949717 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebebb26b50 0x55ebebb23f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:36.651 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.651+0000 7f60e8b9a700 1 --2- 172.21.15.101:0/766949717 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ebebb1bc60 0x55ebebb23810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:36.651 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.651+0000 7f60e939b700 1 -- 172.21.15.101:0/766949717 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebebb26b50 msgr2=0x55ebebb23f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:36.651 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.651+0000 7f60e939b700 1 --2- 172.21.15.101:0/766949717 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebebb26b50 0x55ebebb23f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.651+0000 7f60e939b700 1 -- 172.21.15.101:0/766949717 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ebebb1bc60 msgr2=0x55ebebb23810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:36.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.651+0000 7f60e939b700 1 --2- 172.21.15.101:0/766949717 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ebebb1bc60 0x55ebebb23810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.651+0000 7f60e939b700 1 -- 172.21.15.101:0/766949717 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60e00078a0 con 0x55ebebb1b350 2023-07-18T03:14:36.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.651+0000 7f60e8b9a700 1 --2- 172.21.15.101:0/766949717 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ebebb1bc60 0x55ebebb23810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:14:36.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.651+0000 7f60e939b700 1 --2- 172.21.15.101:0/766949717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebebb1b350 0x55ebebb26590 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f60e0010a80 tx=0x7f60e0005f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea3bb511c2ee0df5 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:36.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.651+0000 7f60ccff9700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:14:36.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.651+0000 7f60ccff9700 1 -- 172.21.15.101:0/766949717 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e0004030 con 0x55ebebb1b350 2023-07-18T03:14:36.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.651+0000 7f60fef46b80 1 -- 172.21.15.101:0/766949717 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ebeb9c7570 con 0x55ebebb1b350 2023-07-18T03:14:36.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.651+0000 7f60ccff9700 1 -- 172.21.15.101:0/766949717 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f60e0004190 con 0x55ebebb1b350 2023-07-18T03:14:36.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.652+0000 7f60ccff9700 1 -- 172.21.15.101:0/766949717 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60e0004350 con 0x55ebebb1b350 2023-07-18T03:14:36.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.652+0000 7f60fef46b80 10 client.27020 fetch_fsmap learned FSMap version 29 2023-07-18T03:14:36.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.652+0000 7f60fef46b80 1 -- 172.21.15.101:0/766949717 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ebeb884ce0 con 0x55ebebb1b350 2023-07-18T03:14:36.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.652+0000 7f60ccff9700 1 -- 172.21.15.101:0/766949717 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ebeb9c7570 con 0x55ebebb1b350 2023-07-18T03:14:36.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.652+0000 7f60ccff9700 1 -- 172.21.15.101:0/766949717 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f60e002a060 con 0x55ebebb1b350 2023-07-18T03:14:36.654 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.652+0000 7f60fef46b80 10 client.27020 fetch_fsmap finished waiting for FSMap version 29 2023-07-18T03:14:36.654 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.652+0000 7f60fef46b80 10 client.27020 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:14:36.654 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.652+0000 7f60fef46b80 20 client.27020 populate_metadata read hostname 'smithi101' 2023-07-18T03:14:36.654 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.652+0000 7f60fef46b80 1 --2- 172.21.15.101:0/766949717 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ebebb363f0 0x55ebebb38800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:36.654 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.652+0000 7f60fef46b80 4 client.27020 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:14:36.654 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.652+0000 7f60fef46b80 1 -- 172.21.15.101:0/766949717 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ebeba66f30 con 0x55ebebb363f0 2023-07-18T03:14:36.654 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.653+0000 7f60e8b9a700 1 --2- 172.21.15.101:0/766949717 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ebebb363f0 0x55ebebb38800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:36.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.653+0000 7f60ccff9700 10 client.27020 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:14:36.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.653+0000 7f60e8b9a700 1 --2- 172.21.15.101:0/766949717 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ebebb363f0 0x55ebebb38800 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5836cff98c7fdea server_cookie=aa46d36c76ac5eb7 in_seq=0 out_seq=0 2023-07-18T03:14:36.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.654+0000 7f60ccff9700 1 -- 172.21.15.101:0/766949717 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ebeba66f30 con 0x55ebebb363f0 2023-07-18T03:14:36.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.654+0000 7f60ccff9700 10 client.27020 handle_command_reply: tid=0 2023-07-18T03:14:36.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.654+0000 7f60fef46b80 1 client.27020 shutdown 2023-07-18T03:14:36.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.654+0000 7f60fef46b80 1 -- 172.21.15.101:0/766949717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebebb1b350 msgr2=0x55ebebb26590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:36.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.654+0000 7f60fef46b80 1 --2- 172.21.15.101:0/766949717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebebb1b350 0x55ebebb26590 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f60e0010a80 tx=0x7f60e0005f60 comp rx=0 tx=0).stop 2023-07-18T03:14:36.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.654+0000 7f60fef46b80 1 -- 172.21.15.101:0/766949717 shutdown_connections 2023-07-18T03:14:36.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.654+0000 7f60fef46b80 1 --2- 172.21.15.101:0/766949717 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ebebb363f0 0x55ebebb38800 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.654+0000 7f60fef46b80 1 --2- 172.21.15.101:0/766949717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebebb1b350 0x55ebebb26590 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.654+0000 7f60fef46b80 1 --2- 172.21.15.101:0/766949717 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebebb26b50 0x55ebebb23f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.654+0000 7f60fef46b80 1 --2- 172.21.15.101:0/766949717 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ebebb1bc60 0x55ebebb23810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.654+0000 7f60ccff9700 0 client.27020 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:14:36.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.654+0000 7f60fef46b80 1 -- 172.21.15.101:0/766949717 >> 172.21.15.101:0/766949717 conn(0x55ebebb11e20 msgr2=0x55ebebb17310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:36.657 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.654+0000 7f60fef46b80 1 -- 172.21.15.101:0/766949717 shutdown_connections 2023-07-18T03:14:36.657 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.654+0000 7f60fef46b80 1 -- 172.21.15.101:0/766949717 wait complete. 2023-07-18T03:14:36.657 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.654+0000 7f60fef46b80 20 client.27020 trim_cache size 0 max 16384 2023-07-18T03:14:36.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.671+0000 7f60fef46b80 1 Processor -- start 2023-07-18T03:14:36.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.671+0000 7f60fef46b80 1 -- start start 2023-07-18T03:14:36.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.672+0000 7f60fef46b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebebb1b350 0x55ebebb237c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:36.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.672+0000 7f60fef46b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebebb1bc60 0x55ebebb22aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:36.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.672+0000 7f60fef46b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ebebb26b50 0x55ebebb260c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:36.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.672+0000 7f60fef46b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebebb1c5f0 con 0x55ebebb26b50 2023-07-18T03:14:36.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.672+0000 7f60fef46b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebebb183b0 con 0x55ebebb1bc60 2023-07-18T03:14:36.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.672+0000 7f60fef46b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebebb1f440 con 0x55ebebb1b350 2023-07-18T03:14:36.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.672+0000 7f60e939b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebebb1b350 0x55ebebb237c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:36.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.672+0000 7f60e939b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebebb1b350 0x55ebebb237c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:50756/0 (socket says 172.21.15.101:50756) 2023-07-18T03:14:36.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.672+0000 7f60e939b700 1 -- 172.21.15.101:0/1523674028 learned_addr learned my addr 172.21.15.101:0/1523674028 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:36.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.672+0000 7f60e9b9c700 1 --2- 172.21.15.101:0/1523674028 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ebebb26b50 0x55ebebb260c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:36.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.672+0000 7f60e8b9a700 1 --2- 172.21.15.101:0/1523674028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebebb1bc60 0x55ebebb22aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:36.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.672+0000 7f60e939b700 1 -- 172.21.15.101:0/1523674028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebebb1bc60 msgr2=0x55ebebb22aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:36.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.672+0000 7f60e939b700 1 --2- 172.21.15.101:0/1523674028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebebb1bc60 0x55ebebb22aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.668 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.672+0000 7f60e939b700 1 -- 172.21.15.101:0/1523674028 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ebebb26b50 msgr2=0x55ebebb260c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:36.668 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.672+0000 7f60e939b700 1 --2- 172.21.15.101:0/1523674028 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ebebb26b50 0x55ebebb260c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.668 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.672+0000 7f60e939b700 1 -- 172.21.15.101:0/1523674028 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ebebb13660 con 0x55ebebb1b350 2023-07-18T03:14:36.668 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.673+0000 7f60e939b700 1 --2- 172.21.15.101:0/1523674028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebebb1b350 0x55ebebb237c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f60e0005800 tx=0x7f60e0005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9731ba94a6dc94b4 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:36.668 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.673+0000 7f60cffff700 1 -- 172.21.15.101:0/1523674028 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e0005b00 con 0x55ebebb1b350 2023-07-18T03:14:36.668 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.673+0000 7f60cffff700 1 -- 172.21.15.101:0/1523674028 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60e0005000 con 0x55ebebb1b350 2023-07-18T03:14:36.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.673+0000 7f60cffff700 1 -- 172.21.15.101:0/1523674028 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e0004e80 con 0x55ebebb1b350 2023-07-18T03:14:36.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.673+0000 7f60fef46b80 1 -- 172.21.15.101:0/1523674028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebebb1b350 msgr2=0x55ebebb237c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:36.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.673+0000 7f60fef46b80 1 --2- 172.21.15.101:0/1523674028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebebb1b350 0x55ebebb237c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f60e0005800 tx=0x7f60e0005fd0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.674+0000 7f60fef46b80 1 -- 172.21.15.101:0/1523674028 shutdown_connections 2023-07-18T03:14:36.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.674+0000 7f60fef46b80 1 --2- 172.21.15.101:0/1523674028 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ebebb26b50 0x55ebebb260c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.674+0000 7f60fef46b80 1 --2- 172.21.15.101:0/1523674028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebebb1b350 0x55ebebb237c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.674+0000 7f60fef46b80 1 --2- 172.21.15.101:0/1523674028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebebb1bc60 0x55ebebb22aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.674+0000 7f60fef46b80 1 -- 172.21.15.101:0/1523674028 >> 172.21.15.101:0/1523674028 conn(0x55ebebb11e20 msgr2=0x55ebebb230b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:36.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.674+0000 7f60fef46b80 1 -- 172.21.15.101:0/1523674028 shutdown_connections 2023-07-18T03:14:36.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.674+0000 7f60fef46b80 1 -- 172.21.15.101:0/1523674028 wait complete. 2023-07-18T03:14:36.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.674+0000 7f60fef46b80 1 -- start start 2023-07-18T03:14:36.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.674+0000 7f60fef46b80 1 Processor -- start 2023-07-18T03:14:36.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.675+0000 7f60fef46b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebebb1b350 0x55ebebb2d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:36.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.675+0000 7f60fef46b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ebebb1bc60 0x55ebebb2b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:36.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.675+0000 7f60fef46b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebebb26b50 0x55ebebb2c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:36.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.675+0000 7f60fef46b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebebb2c800 con 0x55ebebb1bc60 2023-07-18T03:14:36.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.675+0000 7f60fef46b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebebb8a280 con 0x55ebebb1b350 2023-07-18T03:14:36.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.675+0000 7f60fef46b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebebb8a3c0 con 0x55ebebb26b50 2023-07-18T03:14:36.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.675+0000 7f60fef46b80 10 client.0 _ll_register_callbacks cb 0x7f60dc009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:14:36.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.675+0000 7f60e939b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebebb1b350 0x55ebebb2d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:36.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.675+0000 7f60e8b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ebebb1bc60 0x55ebebb2b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:36.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.675+0000 7f60e939b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebebb1b350 0x55ebebb2d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:49312/0 (socket says 172.21.15.101:49312) 2023-07-18T03:14:36.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.675+0000 7f60e939b700 1 -- 172.21.15.101:0/3393728780 learned_addr learned my addr 172.21.15.101:0/3393728780 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:36.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.675+0000 7f60e9b9c700 1 --2- 172.21.15.101:0/3393728780 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebebb26b50 0x55ebebb2c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:36.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.675+0000 7f60e939b700 1 -- 172.21.15.101:0/3393728780 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebebb26b50 msgr2=0x55ebebb2c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:36.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.675+0000 7f60e939b700 1 --2- 172.21.15.101:0/3393728780 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebebb26b50 0x55ebebb2c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.675+0000 7f60e939b700 1 -- 172.21.15.101:0/3393728780 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ebebb1bc60 msgr2=0x55ebebb2b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:36.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.675+0000 7f60e939b700 1 --2- 172.21.15.101:0/3393728780 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ebebb1bc60 0x55ebebb2b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.675+0000 7f60e939b700 1 -- 172.21.15.101:0/3393728780 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ebeb884ce0 con 0x55ebebb1b350 2023-07-18T03:14:36.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.675+0000 7f60e8b9a700 1 --2- 172.21.15.101:0/3393728780 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ebebb1bc60 0x55ebebb2b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:14:36.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.676+0000 7f60e939b700 1 --2- 172.21.15.101:0/3393728780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebebb1b350 0x55ebebb2d700 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f60e002e010 tx=0x7f60e0007220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a66d9db23364635b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:36.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.676+0000 7f60ccff9700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:14:36.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.676+0000 7f60ccff9700 1 -- 172.21.15.101:0/3393728780 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e0005170 con 0x55ebebb1b350 2023-07-18T03:14:36.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.676+0000 7f60fef46b80 1 -- 172.21.15.101:0/3393728780 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f60e0015070 con 0x55ebebb1b350 2023-07-18T03:14:36.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.676+0000 7f60ccff9700 1 -- 172.21.15.101:0/3393728780 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f60e0000e40 con 0x55ebebb1b350 2023-07-18T03:14:36.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.676+0000 7f60ccff9700 1 -- 172.21.15.101:0/3393728780 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60e00052d0 con 0x55ebebb1b350 2023-07-18T03:14:36.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.676+0000 7f60fef46b80 10 client.27026 fetch_fsmap learned FSMap version 29 2023-07-18T03:14:36.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.676+0000 7f60fef46b80 1 -- 172.21.15.101:0/3393728780 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ebeb8c22d0 con 0x55ebebb1b350 2023-07-18T03:14:36.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.676+0000 7f60ccff9700 1 -- 172.21.15.101:0/3393728780 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60e0015070 con 0x55ebebb1b350 2023-07-18T03:14:36.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.676+0000 7f60ccff9700 1 -- 172.21.15.101:0/3393728780 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f60e002a070 con 0x55ebebb1b350 2023-07-18T03:14:36.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.677+0000 7f60fef46b80 10 client.27026 fetch_fsmap finished waiting for FSMap version 29 2023-07-18T03:14:36.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.677+0000 7f60fef46b80 10 client.27026 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:14:36.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.677+0000 7f60fef46b80 20 client.27026 populate_metadata read hostname 'smithi101' 2023-07-18T03:14:36.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.677+0000 7f60fef46b80 1 --2- 172.21.15.101:0/3393728780 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ebebb8b5e0 0x55ebebb8d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:36.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.677+0000 7f60fef46b80 4 client.27026 mds_command: new command op to 24562 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-18T03:14:36.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.677+0000 7f60fef46b80 1 -- 172.21.15.101:0/3393728780 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55ebeba290d0 con 0x55ebebb8b5e0 2023-07-18T03:14:36.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.677+0000 7f60e8b9a700 1 --2- 172.21.15.101:0/3393728780 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ebebb8b5e0 0x55ebebb8d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:36.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.677+0000 7f60ccff9700 10 client.27026 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:14:36.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.677+0000 7f60e8b9a700 1 --2- 172.21.15.101:0/3393728780 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ebebb8b5e0 0x55ebebb8d9c0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f810ccc74da918c2 server_cookie=e5d0e304528fa42 in_seq=0 out_seq=0 2023-07-18T03:14:36.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.678+0000 7f60ccff9700 1 -- 172.21.15.101:0/3393728780 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ebeba290d0 con 0x55ebebb8b5e0 2023-07-18T03:14:36.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.678+0000 7f60ccff9700 10 client.27026 handle_command_reply: tid=0 2023-07-18T03:14:36.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.678+0000 7f60fef46b80 1 client.27026 shutdown 2023-07-18T03:14:36.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.678+0000 7f60fef46b80 1 -- 172.21.15.101:0/3393728780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebebb1b350 msgr2=0x55ebebb2d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:36.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.678+0000 7f60fef46b80 1 --2- 172.21.15.101:0/3393728780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebebb1b350 0x55ebebb2d700 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f60e002e010 tx=0x7f60e0007220 comp rx=0 tx=0).stop 2023-07-18T03:14:36.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.679+0000 7f60fef46b80 1 -- 172.21.15.101:0/3393728780 shutdown_connections 2023-07-18T03:14:36.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.679+0000 7f60fef46b80 1 --2- 172.21.15.101:0/3393728780 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ebebb8b5e0 0x55ebebb8d9c0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.679+0000 7f60fef46b80 1 --2- 172.21.15.101:0/3393728780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ebebb1b350 0x55ebebb2d700 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.679+0000 7f60fef46b80 1 --2- 172.21.15.101:0/3393728780 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebebb26b50 0x55ebebb2c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.679+0000 7f60ccff9700 0 client.27026 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:14:36.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.679+0000 7f60fef46b80 1 --2- 172.21.15.101:0/3393728780 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ebebb1bc60 0x55ebebb2b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.679+0000 7f60fef46b80 1 -- 172.21.15.101:0/3393728780 >> 172.21.15.101:0/3393728780 conn(0x55ebebb11e20 msgr2=0x55ebebb171c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:36.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.679+0000 7f60fef46b80 1 -- 172.21.15.101:0/3393728780 shutdown_connections 2023-07-18T03:14:36.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.679+0000 7f60fef46b80 1 -- 172.21.15.101:0/3393728780 wait complete. 2023-07-18T03:14:36.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.679+0000 7f60fef46b80 20 client.27026 trim_cache size 0 max 16384 2023-07-18T03:14:36.681 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:14:36.681 INFO:teuthology.orchestra.run.smithi101.stdout: "return_code": 0, 2023-07-18T03:14:36.681 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_tag": "7badda0a-877c-4699-829f-3d031d5afc15", 2023-07-18T03:14:36.681 INFO:teuthology.orchestra.run.smithi101.stdout: "mode": "asynchronous" 2023-07-18T03:14:36.682 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:14:36.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.679+0000 7f60eb5ff700 1 -- 172.21.15.101:0/1593232164 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f60d00a12c0 msgr2=0x7f60d00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:36.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.679+0000 7f60eb5ff700 1 --2- 172.21.15.101:0/1593232164 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f60d00a12c0 0x7f60d00a3770 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f60dc009cd0 tx=0x7f60dc00cea0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.679+0000 7f60eb5ff700 1 -- 172.21.15.101:0/1593232164 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f60e4100490 msgr2=0x7f60e41984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:36.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.679+0000 7f60eb5ff700 1 --2- 172.21.15.101:0/1593232164 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f60e4100490 0x7f60e41984d0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f60d400d950 tx=0x7f60d400dc60 comp rx=0 tx=0).stop 2023-07-18T03:14:36.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.680+0000 7f60eb5ff700 1 -- 172.21.15.101:0/1593232164 shutdown_connections 2023-07-18T03:14:36.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.680+0000 7f60eb5ff700 1 --2- 172.21.15.101:0/1593232164 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f60d00a12c0 0x7f60d00a3770 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.680+0000 7f60eb5ff700 1 --2- 172.21.15.101:0/1593232164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60e4105af0 0x7f60e419c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.680+0000 7f60eb5ff700 1 --2- 172.21.15.101:0/1593232164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f60e40ffae0 0x7f60e4197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.680+0000 7f60eb5ff700 1 --2- 172.21.15.101:0/1593232164 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f60e4100490 0x7f60e41984d0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.680+0000 7f60eb5ff700 1 -- 172.21.15.101:0/1593232164 >> 172.21.15.101:0/1593232164 conn(0x7f60e40fb5e0 msgr2=0x7f60e410ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:36.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.680+0000 7f60eb5ff700 1 -- 172.21.15.101:0/1593232164 shutdown_connections 2023-07-18T03:14:36.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.680+0000 7f60eb5ff700 1 -- 172.21.15.101:0/1593232164 wait complete. 2023-07-18T03:14:36.691 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:14:36.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.907+0000 7f50b3fff700 1 -- 172.21.15.101:0/933709487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f50ac10c310 msgr2=0x7f50ac10e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:36.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.907+0000 7f50b3fff700 1 --2- 172.21.15.101:0/933709487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f50ac10c310 0x7f50ac10e6f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f50a400b0a0 tx=0x7f50a400b3b0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.907+0000 7f50b3fff700 1 -- 172.21.15.101:0/933709487 shutdown_connections 2023-07-18T03:14:36.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.907+0000 7f50b3fff700 1 --2- 172.21.15.101:0/933709487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50ac0728b0 0x7f50ac072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.907+0000 7f50b3fff700 1 --2- 172.21.15.101:0/933709487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f50ac10c310 0x7f50ac10e6f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.907+0000 7f50b3fff700 1 --2- 172.21.15.101:0/933709487 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f50ac0731c0 0x7f50ac10bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.907+0000 7f50b3fff700 1 -- 172.21.15.101:0/933709487 >> 172.21.15.101:0/933709487 conn(0x7f50ac0fb5f0 msgr2=0x7f50ac0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:36.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.907+0000 7f50b3fff700 1 -- 172.21.15.101:0/933709487 shutdown_connections 2023-07-18T03:14:36.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.908+0000 7f50b3fff700 1 -- 172.21.15.101:0/933709487 wait complete. 2023-07-18T03:14:36.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.908+0000 7f50b3fff700 1 Processor -- start 2023-07-18T03:14:36.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.909+0000 7f50b3fff700 1 -- start start 2023-07-18T03:14:36.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.909+0000 7f50b3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f50ac0728b0 0x7f50ac197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:36.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.909+0000 7f50b3fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f50ac0731c0 0x7f50ac198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:36.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.909+0000 7f50b3fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50ac10c310 0x7f50ac19c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:36.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.909+0000 7f50b3fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac19cec0 con 0x7f50ac0731c0 2023-07-18T03:14:36.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.909+0000 7f50b3fff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac19d030 con 0x7f50ac10c310 2023-07-18T03:14:36.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.909+0000 7f50b3fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac19d1f0 con 0x7f50ac0728b0 2023-07-18T03:14:36.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.909+0000 7f50b1d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f50ac0728b0 0x7f50ac197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:36.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.909+0000 7f50b1d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f50ac0728b0 0x7f50ac197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:50784/0 (socket says 172.21.15.101:50784) 2023-07-18T03:14:36.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.909+0000 7f50b1d9b700 1 -- 172.21.15.101:0/2869514948 learned_addr learned my addr 172.21.15.101:0/2869514948 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:36.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.909+0000 7f50b159a700 1 --2- 172.21.15.101:0/2869514948 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f50ac0731c0 0x7f50ac198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:36.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.909+0000 7f50b259c700 1 --2- 172.21.15.101:0/2869514948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50ac10c310 0x7f50ac19c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:36.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.910+0000 7f50b1d9b700 1 -- 172.21.15.101:0/2869514948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50ac10c310 msgr2=0x7f50ac19c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:36.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.910+0000 7f50b1d9b700 1 --2- 172.21.15.101:0/2869514948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50ac10c310 0x7f50ac19c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.910+0000 7f50b1d9b700 1 -- 172.21.15.101:0/2869514948 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f50ac0731c0 msgr2=0x7f50ac198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:36.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.910+0000 7f50b1d9b700 1 --2- 172.21.15.101:0/2869514948 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f50ac0731c0 0x7f50ac198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.910+0000 7f50b1d9b700 1 -- 172.21.15.101:0/2869514948 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a4009d00 con 0x7f50ac0728b0 2023-07-18T03:14:36.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.910+0000 7f50b159a700 1 --2- 172.21.15.101:0/2869514948 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f50ac0731c0 0x7f50ac198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:14:36.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.910+0000 7f50b1d9b700 1 --2- 172.21.15.101:0/2869514948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f50ac0728b0 0x7f50ac197ee0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f50a800e9a0 tx=0x7f50a800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:36.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.910+0000 7f50a2ffd700 1 -- 172.21.15.101:0/2869514948 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a8009790 con 0x7f50ac0728b0 2023-07-18T03:14:36.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.910+0000 7f50a2ffd700 1 -- 172.21.15.101:0/2869514948 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50a8010430 con 0x7f50ac0728b0 2023-07-18T03:14:36.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.910+0000 7f50a2ffd700 1 -- 172.21.15.101:0/2869514948 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a8018680 con 0x7f50ac0728b0 2023-07-18T03:14:36.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.910+0000 7f50b3fff700 1 -- 172.21.15.101:0/2869514948 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50ac19d4d0 con 0x7f50ac0728b0 2023-07-18T03:14:36.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.910+0000 7f50b3fff700 1 -- 172.21.15.101:0/2869514948 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50ac1a4d10 con 0x7f50ac0728b0 2023-07-18T03:14:36.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.915+0000 7f50a2ffd700 1 -- 172.21.15.101:0/2869514948 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f50a80105a0 con 0x7f50ac0728b0 2023-07-18T03:14:36.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.916+0000 7f50c78abb80 1 Processor -- start 2023-07-18T03:14:36.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.916+0000 7f50c78abb80 1 -- start start 2023-07-18T03:14:36.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.916+0000 7f50c78abb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eb95dd12d0 0x55eb95dd16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:36.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.916+0000 7f50c78abb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eb95dd1be0 0x55eb95ddc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:36.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.916+0000 7f50c78abb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb95ddcad0 0x55eb95ddeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:36.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.916+0000 7f50c78abb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb95cd7cb0 con 0x55eb95dd12d0 2023-07-18T03:14:36.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.916+0000 7f50c78abb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb95d2aa40 con 0x55eb95ddcad0 2023-07-18T03:14:36.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.916+0000 7f50c78abb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb95d23ed0 con 0x55eb95dd1be0 2023-07-18T03:14:36.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.917+0000 7f50b1d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eb95dd1be0 0x55eb95ddc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:36.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.917+0000 7f50a2ffd700 1 --2- 172.21.15.101:0/2869514948 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f50980a12c0 0x7f50980a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:36.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.917+0000 7f50b159a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb95ddcad0 0x55eb95ddeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:36.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.917+0000 7f50b259c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eb95dd12d0 0x55eb95dd16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:36.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.917+0000 7f50b159a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb95ddcad0 0x55eb95ddeeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:49338/0 (socket says 172.21.15.101:49338) 2023-07-18T03:14:36.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.917+0000 7f50b159a700 1 -- 172.21.15.101:0/1204622730 learned_addr learned my addr 172.21.15.101:0/1204622730 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:36.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.917+0000 7f50a2ffd700 1 -- 172.21.15.101:0/2869514948 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f50a8014070 con 0x7f50ac0728b0 2023-07-18T03:14:36.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.917+0000 7f50b159a700 1 -- 172.21.15.101:0/1204622730 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eb95dd1be0 msgr2=0x55eb95ddc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:36.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.917+0000 7f50b159a700 1 --2- 172.21.15.101:0/1204622730 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eb95dd1be0 0x55eb95ddc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.917+0000 7f50b159a700 1 -- 172.21.15.101:0/1204622730 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eb95dd12d0 msgr2=0x55eb95dd16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:36.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.917+0000 7f50b159a700 1 --2- 172.21.15.101:0/1204622730 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eb95dd12d0 0x55eb95dd16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.917+0000 7f50b159a700 1 -- 172.21.15.101:0/1204622730 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb95b51de0 con 0x55eb95ddcad0 2023-07-18T03:14:36.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.917+0000 7f50b259c700 1 --2- 172.21.15.101:0/2869514948 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f50980a12c0 0x7f50980a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:36.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.917+0000 7f50b159a700 1 --2- 172.21.15.101:0/1204622730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb95ddcad0 0x55eb95ddeeb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f509c0114f0 tx=0x7f509c011800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c86540a23d5af2df server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:36.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.917+0000 7f5097fff700 1 -- 172.21.15.101:0/1204622730 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509c004230 con 0x55eb95ddcad0 2023-07-18T03:14:36.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.918+0000 7f5097fff700 1 -- 172.21.15.101:0/1204622730 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f509c004390 con 0x55eb95ddcad0 2023-07-18T03:14:36.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.918+0000 7f5097fff700 1 -- 172.21.15.101:0/1204622730 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509c00e8b0 con 0x55eb95ddcad0 2023-07-18T03:14:36.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.918+0000 7f50c78abb80 1 -- 172.21.15.101:0/1204622730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb95ddcad0 msgr2=0x55eb95ddeeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:36.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.918+0000 7f50b259c700 1 --2- 172.21.15.101:0/2869514948 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f50980a12c0 0x7f50980a3770 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f50a400b780 tx=0x7f50a40099e0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:36.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.918+0000 7f50c78abb80 1 --2- 172.21.15.101:0/1204622730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb95ddcad0 0x55eb95ddeeb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f509c0114f0 tx=0x7f509c011800 comp rx=0 tx=0).stop 2023-07-18T03:14:36.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.918+0000 7f50c78abb80 1 -- 172.21.15.101:0/1204622730 shutdown_connections 2023-07-18T03:14:36.916 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.918+0000 7f50c78abb80 1 --2- 172.21.15.101:0/1204622730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb95ddcad0 0x55eb95ddeeb0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.916 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.918+0000 7f50c78abb80 1 --2- 172.21.15.101:0/1204622730 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eb95dd12d0 0x55eb95dd16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.916 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.918+0000 7f50c78abb80 1 --2- 172.21.15.101:0/1204622730 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eb95dd1be0 0x55eb95ddc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.916 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.918+0000 7f50c78abb80 1 -- 172.21.15.101:0/1204622730 >> 172.21.15.101:0/1204622730 conn(0x55eb95dc7da0 msgr2=0x55eb95dcd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:36.916 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.918+0000 7f50c78abb80 1 -- 172.21.15.101:0/1204622730 shutdown_connections 2023-07-18T03:14:36.916 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.918+0000 7f50c78abb80 1 -- 172.21.15.101:0/1204622730 wait complete. 2023-07-18T03:14:36.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.919+0000 7f50c78abb80 1 -- start start 2023-07-18T03:14:36.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.919+0000 7f50c78abb80 1 Processor -- start 2023-07-18T03:14:36.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.920+0000 7f50c78abb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eb95dd12d0 0x55eb95ddc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:36.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.920+0000 7f50c78abb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb95dd1be0 0x55eb95dd9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:36.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.920+0000 7f50c78abb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eb95ddcad0 0x55eb95dd9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:36.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.920+0000 7f50c78abb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb95cc3ac0 con 0x55eb95dd12d0 2023-07-18T03:14:36.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.920+0000 7f50c78abb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb95cc37c0 con 0x55eb95dd1be0 2023-07-18T03:14:36.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.920+0000 7f50c78abb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb95ba7920 con 0x55eb95ddcad0 2023-07-18T03:14:36.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.920+0000 7f50c78abb80 10 client.0 _ll_register_callbacks cb 0x55eb95cc1760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:14:36.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.920+0000 7f50b159a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb95dd1be0 0x55eb95dd9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:36.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.920+0000 7f50b1d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eb95dd12d0 0x55eb95ddc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:36.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.920+0000 7f50b259c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eb95ddcad0 0x55eb95dd9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:36.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.920+0000 7f50b159a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb95dd1be0 0x55eb95dd9750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:49352/0 (socket says 172.21.15.101:49352) 2023-07-18T03:14:36.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.920+0000 7f50b159a700 1 -- 172.21.15.101:0/2698602607 learned_addr learned my addr 172.21.15.101:0/2698602607 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:36.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.921+0000 7f50b159a700 1 -- 172.21.15.101:0/2698602607 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eb95ddcad0 msgr2=0x55eb95dd9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:36.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.921+0000 7f50b159a700 1 --2- 172.21.15.101:0/2698602607 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eb95ddcad0 0x55eb95dd9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.921+0000 7f50b159a700 1 -- 172.21.15.101:0/2698602607 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eb95dd12d0 msgr2=0x55eb95ddc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:36.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.921+0000 7f50b159a700 1 --2- 172.21.15.101:0/2698602607 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eb95dd12d0 0x55eb95ddc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.921+0000 7f50b159a700 1 -- 172.21.15.101:0/2698602607 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f509c009cd0 con 0x55eb95dd1be0 2023-07-18T03:14:36.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.921+0000 7f50b159a700 1 --2- 172.21.15.101:0/2698602607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb95dd1be0 0x55eb95dd9750 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f509c011b50 tx=0x7f509c0052d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bf399dffc28db8b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:36.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.921+0000 7f50957fa700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:14:36.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.921+0000 7f50957fa700 1 -- 172.21.15.101:0/2698602607 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509c004950 con 0x55eb95dd1be0 2023-07-18T03:14:36.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.921+0000 7f50c78abb80 1 -- 172.21.15.101:0/2698602607 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eb95c7d4f0 con 0x55eb95dd1be0 2023-07-18T03:14:36.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.921+0000 7f50957fa700 1 -- 172.21.15.101:0/2698602607 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f509c004ab0 con 0x55eb95dd1be0 2023-07-18T03:14:36.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.921+0000 7f50957fa700 1 -- 172.21.15.101:0/2698602607 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f509c004c70 con 0x55eb95dd1be0 2023-07-18T03:14:36.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.922+0000 7f50c78abb80 10 client.27038 fetch_fsmap learned FSMap version 29 2023-07-18T03:14:36.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.922+0000 7f50c78abb80 1 -- 172.21.15.101:0/2698602607 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55eb95b3ac60 con 0x55eb95dd1be0 2023-07-18T03:14:36.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.922+0000 7f50957fa700 1 -- 172.21.15.101:0/2698602607 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55eb95c7d4f0 con 0x55eb95dd1be0 2023-07-18T03:14:36.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.922+0000 7f50957fa700 1 -- 172.21.15.101:0/2698602607 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f509c028020 con 0x55eb95dd1be0 2023-07-18T03:14:36.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.922+0000 7f50c78abb80 10 client.27038 fetch_fsmap finished waiting for FSMap version 29 2023-07-18T03:14:36.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.922+0000 7f50c78abb80 10 client.27038 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:14:36.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.922+0000 7f50c78abb80 20 client.27038 populate_metadata read hostname 'smithi101' 2023-07-18T03:14:36.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.922+0000 7f50c78abb80 1 --2- 172.21.15.101:0/2698602607 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55eb95dec2e0 0x55eb95dee6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:36.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.922+0000 7f50c78abb80 4 client.27038 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:14:36.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.922+0000 7f50c78abb80 1 -- 172.21.15.101:0/2698602607 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eb95d1ceb0 con 0x55eb95dec2e0 2023-07-18T03:14:36.922 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.922+0000 7f50b1d9b700 1 --2- 172.21.15.101:0/2698602607 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55eb95dec2e0 0x55eb95dee6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:36.922 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.923+0000 7f50957fa700 10 client.27038 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:14:36.922 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.923+0000 7f50b1d9b700 1 --2- 172.21.15.101:0/2698602607 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55eb95dec2e0 0x55eb95dee6f0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ef21ac655afb246 server_cookie=11ef842fbfe119ed in_seq=0 out_seq=0 2023-07-18T03:14:36.923 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.924+0000 7f50957fa700 1 -- 172.21.15.101:0/2698602607 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55eb95d1ceb0 con 0x55eb95dec2e0 2023-07-18T03:14:36.923 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.924+0000 7f50957fa700 10 client.27038 handle_command_reply: tid=0 2023-07-18T03:14:36.923 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.924+0000 7f50c78abb80 1 client.27038 shutdown 2023-07-18T03:14:36.923 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.924+0000 7f50c78abb80 1 -- 172.21.15.101:0/2698602607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb95dd1be0 msgr2=0x55eb95dd9750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:36.923 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.924+0000 7f50c78abb80 1 --2- 172.21.15.101:0/2698602607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb95dd1be0 0x55eb95dd9750 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f509c011b50 tx=0x7f509c0052d0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.923 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.924+0000 7f50c78abb80 1 -- 172.21.15.101:0/2698602607 shutdown_connections 2023-07-18T03:14:36.923 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.924+0000 7f50c78abb80 1 --2- 172.21.15.101:0/2698602607 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55eb95dec2e0 0x55eb95dee6f0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.924 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.924+0000 7f50c78abb80 1 --2- 172.21.15.101:0/2698602607 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eb95ddcad0 0x55eb95dd9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.924 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.924+0000 7f50c78abb80 1 --2- 172.21.15.101:0/2698602607 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eb95dd12d0 0x55eb95ddc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.924 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.924+0000 7f50c78abb80 1 --2- 172.21.15.101:0/2698602607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb95dd1be0 0x55eb95dd9750 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.924 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.924+0000 7f50c78abb80 1 -- 172.21.15.101:0/2698602607 >> 172.21.15.101:0/2698602607 conn(0x55eb95dc7da0 msgr2=0x55eb95dcd230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:36.924 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.924+0000 7f50957fa700 0 client.27038 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:14:36.924 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.925+0000 7f50c78abb80 1 -- 172.21.15.101:0/2698602607 shutdown_connections 2023-07-18T03:14:36.924 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.925+0000 7f50c78abb80 1 -- 172.21.15.101:0/2698602607 wait complete. 2023-07-18T03:14:36.925 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.925+0000 7f50c78abb80 20 client.27038 trim_cache size 0 max 16384 2023-07-18T03:14:36.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.944+0000 7f50c78abb80 1 Processor -- start 2023-07-18T03:14:36.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.944+0000 7f50c78abb80 1 -- start start 2023-07-18T03:14:36.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.944+0000 7f50c78abb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eb95dd12d0 0x55eb95ddc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:36.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.944+0000 7f50c78abb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb95dd1be0 0x55eb95dd4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:36.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.944+0000 7f50c78abb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eb95ddcad0 0x55eb95dd4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:36.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.944+0000 7f50c78abb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb95de9a60 con 0x55eb95ddcad0 2023-07-18T03:14:36.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.944+0000 7f50c78abb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb95dc9ec0 con 0x55eb95dd1be0 2023-07-18T03:14:36.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.944+0000 7f50c78abb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb95dced20 con 0x55eb95dd12d0 2023-07-18T03:14:36.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.945+0000 7f50b1d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eb95dd12d0 0x55eb95ddc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:36.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.945+0000 7f50b159a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb95dd1be0 0x55eb95dd4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:36.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.945+0000 7f50b259c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eb95ddcad0 0x55eb95dd4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:36.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.945+0000 7f50b1d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eb95dd12d0 0x55eb95ddc070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:50816/0 (socket says 172.21.15.101:50816) 2023-07-18T03:14:36.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.945+0000 7f50b159a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb95dd1be0 0x55eb95dd4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:49354/0 (socket says 172.21.15.101:49354) 2023-07-18T03:14:36.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.945+0000 7f50b1d9b700 1 -- 172.21.15.101:0/3171823517 learned_addr learned my addr 172.21.15.101:0/3171823517 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:36.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.945+0000 7f50b1d9b700 1 -- 172.21.15.101:0/3171823517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb95dd1be0 msgr2=0x55eb95dd4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:36.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.945+0000 7f50b1d9b700 1 --2- 172.21.15.101:0/3171823517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb95dd1be0 0x55eb95dd4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.945+0000 7f50b1d9b700 1 -- 172.21.15.101:0/3171823517 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eb95ddcad0 msgr2=0x55eb95dd4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:36.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.945+0000 7f50b1d9b700 1 --2- 172.21.15.101:0/3171823517 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eb95ddcad0 0x55eb95dd4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.945+0000 7f50b1d9b700 1 -- 172.21.15.101:0/3171823517 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb95b78250 con 0x55eb95dd12d0 2023-07-18T03:14:36.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.945+0000 7f50b159a700 1 --2- 172.21.15.101:0/3171823517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb95dd1be0 0x55eb95dd4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:14:36.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.945+0000 7f50b1d9b700 1 --2- 172.21.15.101:0/3171823517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eb95dd12d0 0x55eb95ddc070 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f50a80964b0 tx=0x7f50a8099bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4362ff8f2365580e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:36.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.946+0000 7f50a0ff9700 1 -- 172.21.15.101:0/3171823517 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a80cd170 con 0x55eb95dd12d0 2023-07-18T03:14:36.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.946+0000 7f50a0ff9700 1 -- 172.21.15.101:0/3171823517 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50a8099fb0 con 0x55eb95dd12d0 2023-07-18T03:14:36.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.946+0000 7f50a0ff9700 1 -- 172.21.15.101:0/3171823517 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a8099700 con 0x55eb95dd12d0 2023-07-18T03:14:36.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.946+0000 7f50c78abb80 1 -- 172.21.15.101:0/3171823517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eb95dd12d0 msgr2=0x55eb95ddc070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:36.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.946+0000 7f50c78abb80 1 --2- 172.21.15.101:0/3171823517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eb95dd12d0 0x55eb95ddc070 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f50a80964b0 tx=0x7f50a8099bf0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.946+0000 7f50c78abb80 1 -- 172.21.15.101:0/3171823517 shutdown_connections 2023-07-18T03:14:36.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.946+0000 7f50c78abb80 1 --2- 172.21.15.101:0/3171823517 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eb95ddcad0 0x55eb95dd4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.946+0000 7f50c78abb80 1 --2- 172.21.15.101:0/3171823517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eb95dd12d0 0x55eb95ddc070 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.946+0000 7f50c78abb80 1 --2- 172.21.15.101:0/3171823517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb95dd1be0 0x55eb95dd4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.946+0000 7f50c78abb80 1 -- 172.21.15.101:0/3171823517 >> 172.21.15.101:0/3171823517 conn(0x55eb95dc7da0 msgr2=0x55eb95de8700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:36.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.947+0000 7f50c78abb80 1 -- 172.21.15.101:0/3171823517 shutdown_connections 2023-07-18T03:14:36.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.947+0000 7f50c78abb80 1 -- 172.21.15.101:0/3171823517 wait complete. 2023-07-18T03:14:36.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.947+0000 7f50c78abb80 1 -- start start 2023-07-18T03:14:36.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.947+0000 7f50c78abb80 1 Processor -- start 2023-07-18T03:14:36.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.948+0000 7f50c78abb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eb95dd12d0 0x55eb95de3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:36.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.948+0000 7f50c78abb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb95dd1be0 0x55eb95de18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:36.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.948+0000 7f50c78abb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eb95ddcad0 0x55eb95de1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:36.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.948+0000 7f50c78abb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb95de2570 con 0x55eb95ddcad0 2023-07-18T03:14:36.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.948+0000 7f50c78abb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb95de26e0 con 0x55eb95dd1be0 2023-07-18T03:14:36.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.948+0000 7f50c78abb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb95e40170 con 0x55eb95dd12d0 2023-07-18T03:14:36.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.948+0000 7f50c78abb80 10 client.0 _ll_register_callbacks cb 0x7f50a80962f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:14:36.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.948+0000 7f50b1d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eb95dd12d0 0x55eb95de3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:36.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.948+0000 7f50b1d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eb95dd12d0 0x55eb95de3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:50832/0 (socket says 172.21.15.101:50832) 2023-07-18T03:14:36.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.948+0000 7f50b1d9b700 1 -- 172.21.15.101:0/823276907 learned_addr learned my addr 172.21.15.101:0/823276907 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:14:36.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.948+0000 7f50b259c700 1 --2- 172.21.15.101:0/823276907 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eb95ddcad0 0x55eb95de1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:36.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.948+0000 7f50b159a700 1 --2- 172.21.15.101:0/823276907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb95dd1be0 0x55eb95de18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:36.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.948+0000 7f50b1d9b700 1 -- 172.21.15.101:0/823276907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb95dd1be0 msgr2=0x55eb95de18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:36.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.948+0000 7f50b1d9b700 1 --2- 172.21.15.101:0/823276907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb95dd1be0 0x55eb95de18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.948+0000 7f50b1d9b700 1 -- 172.21.15.101:0/823276907 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eb95ddcad0 msgr2=0x55eb95de1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:14:36.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.948+0000 7f50b1d9b700 1 --2- 172.21.15.101:0/823276907 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eb95ddcad0 0x55eb95de1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.948+0000 7f50b1d9b700 1 -- 172.21.15.101:0/823276907 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f509c02f040 con 0x55eb95dd12d0 2023-07-18T03:14:36.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.948+0000 7f50b1d9b700 1 --2- 172.21.15.101:0/823276907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eb95dd12d0 0x55eb95de3570 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f50a8000c00 tx=0x7f50a8099a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b2066feec13a6c8 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:14:36.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.948+0000 7f50957fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:14:36.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.949+0000 7f50957fa700 1 -- 172.21.15.101:0/823276907 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a80cd170 con 0x55eb95dd12d0 2023-07-18T03:14:36.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.949+0000 7f50957fa700 1 -- 172.21.15.101:0/823276907 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f50a80a2400 con 0x55eb95dd12d0 2023-07-18T03:14:36.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.949+0000 7f50c78abb80 1 -- 172.21.15.101:0/823276907 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f509c004390 con 0x55eb95dd12d0 2023-07-18T03:14:36.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.949+0000 7f50957fa700 1 -- 172.21.15.101:0/823276907 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50a80994a0 con 0x55eb95dd12d0 2023-07-18T03:14:36.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.949+0000 7f50c78abb80 10 client.27082 fetch_fsmap learned FSMap version 29 2023-07-18T03:14:36.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.949+0000 7f50c78abb80 1 -- 172.21.15.101:0/823276907 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f50ac1a4d10 con 0x55eb95dd12d0 2023-07-18T03:14:36.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.949+0000 7f50957fa700 1 -- 172.21.15.101:0/823276907 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f509c004390 con 0x55eb95dd12d0 2023-07-18T03:14:36.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.949+0000 7f50957fa700 1 -- 172.21.15.101:0/823276907 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f50a8094c50 con 0x55eb95dd12d0 2023-07-18T03:14:36.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.949+0000 7f50c78abb80 10 client.27082 fetch_fsmap finished waiting for FSMap version 29 2023-07-18T03:14:36.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.949+0000 7f50c78abb80 10 client.27082 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:14:36.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.949+0000 7f50c78abb80 20 client.27082 populate_metadata read hostname 'smithi101' 2023-07-18T03:14:36.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.949+0000 7f50c78abb80 1 --2- 172.21.15.101:0/823276907 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55eb95e414a0 0x55eb95e438b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:14:36.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.949+0000 7f50c78abb80 4 client.27082 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:14:36.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.949+0000 7f50c78abb80 1 -- 172.21.15.101:0/823276907 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55eb95cdf050 con 0x55eb95e414a0 2023-07-18T03:14:36.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.949+0000 7f50b159a700 1 --2- 172.21.15.101:0/823276907 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55eb95e414a0 0x55eb95e438b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:14:36.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.950+0000 7f50957fa700 10 client.27082 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:14:36.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.950+0000 7f50b159a700 1 --2- 172.21.15.101:0/823276907 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55eb95e414a0 0x55eb95e438b0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f94e26d80e7b944c server_cookie=dd1082e159c2acee in_seq=0 out_seq=0 2023-07-18T03:14:36.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.950+0000 7f50957fa700 1 -- 172.21.15.101:0/823276907 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55eb95cdf050 con 0x55eb95e414a0 2023-07-18T03:14:36.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.950+0000 7f50957fa700 10 client.27082 handle_command_reply: tid=0 2023-07-18T03:14:36.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.950+0000 7f50c78abb80 1 client.27082 shutdown 2023-07-18T03:14:36.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.950+0000 7f50c78abb80 1 -- 172.21.15.101:0/823276907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eb95dd12d0 msgr2=0x55eb95de3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:36.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.950+0000 7f50c78abb80 1 --2- 172.21.15.101:0/823276907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eb95dd12d0 0x55eb95de3570 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f50a8000c00 tx=0x7f50a8099a40 comp rx=0 tx=0).stop 2023-07-18T03:14:36.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.950+0000 7f50c78abb80 1 -- 172.21.15.101:0/823276907 shutdown_connections 2023-07-18T03:14:36.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.950+0000 7f50c78abb80 1 --2- 172.21.15.101:0/823276907 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55eb95e414a0 0x55eb95e438b0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.950+0000 7f50c78abb80 1 --2- 172.21.15.101:0/823276907 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eb95ddcad0 0x55eb95de1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.951+0000 7f50c78abb80 1 --2- 172.21.15.101:0/823276907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eb95dd12d0 0x55eb95de3570 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.951+0000 7f50c78abb80 1 --2- 172.21.15.101:0/823276907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb95dd1be0 0x55eb95de18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.951+0000 7f50957fa700 0 client.27082 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:14:36.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.951+0000 7f50c78abb80 1 -- 172.21.15.101:0/823276907 >> 172.21.15.101:0/823276907 conn(0x55eb95dc7da0 msgr2=0x55eb95dd4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:36.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.951+0000 7f50c78abb80 1 -- 172.21.15.101:0/823276907 shutdown_connections 2023-07-18T03:14:36.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.951+0000 7f50c78abb80 1 -- 172.21.15.101:0/823276907 wait complete. 2023-07-18T03:14:36.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.951+0000 7f50c78abb80 20 client.27082 trim_cache size 0 max 16384 2023-07-18T03:14:36.952 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:14:36.952 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:14:36.953 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:14:36.953 INFO:teuthology.orchestra.run.smithi101.stdout: "7badda0a-877c-4699-829f-3d031d5afc15": { 2023-07-18T03:14:36.953 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:14:36.953 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "7badda0a-877c-4699-829f-3d031d5afc15", 2023-07-18T03:14:36.953 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:14:36.953 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:14:36.953 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:14:36.953 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:14:36.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.951+0000 7f50b3fff700 1 -- 172.21.15.101:0/2869514948 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f50980a12c0 msgr2=0x7f50980a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:36.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.951+0000 7f50b3fff700 1 --2- 172.21.15.101:0/2869514948 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f50980a12c0 0x7f50980a3770 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f50a400b780 tx=0x7f50a40099e0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.951+0000 7f50b3fff700 1 -- 172.21.15.101:0/2869514948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f50ac0728b0 msgr2=0x7f50ac197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:14:36.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.951+0000 7f50b3fff700 1 --2- 172.21.15.101:0/2869514948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f50ac0728b0 0x7f50ac197ee0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f50a800e9a0 tx=0x7f50a800ecb0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.951+0000 7f50b3fff700 1 -- 172.21.15.101:0/2869514948 shutdown_connections 2023-07-18T03:14:36.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.951+0000 7f50b3fff700 1 --2- 172.21.15.101:0/2869514948 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f50980a12c0 0x7f50980a3770 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.951+0000 7f50b3fff700 1 --2- 172.21.15.101:0/2869514948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50ac10c310 0x7f50ac19c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.951+0000 7f50b3fff700 1 --2- 172.21.15.101:0/2869514948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f50ac0728b0 0x7f50ac197ee0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.951+0000 7f50b3fff700 1 --2- 172.21.15.101:0/2869514948 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f50ac0731c0 0x7f50ac198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:14:36.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.951+0000 7f50b3fff700 1 -- 172.21.15.101:0/2869514948 >> 172.21.15.101:0/2869514948 conn(0x7f50ac0fb5f0 msgr2=0x7f50ac103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:14:36.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.952+0000 7f50b3fff700 1 -- 172.21.15.101:0/2869514948 shutdown_connections 2023-07-18T03:14:36.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:14:36.952+0000 7f50b3fff700 1 -- 172.21.15.101:0/2869514948 wait complete. 2023-07-18T03:14:36.962 INFO:tasks.cephfs.filesystem:scrub status for tag:7badda0a-877c-4699-829f-3d031d5afc15 - {'path': '/', 'tag': '7badda0a-877c-4699-829f-3d031d5afc15', 'options': 'recursive,force'} 2023-07-18T03:14:37.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:36 smithi101 ceph-mon[126379]: pgmap v216: 129 pgs: 129 active+clean; 350 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 161 op/s 2023-07-18T03:14:37.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:36 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:14:37.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:36 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:14:37.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:36 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:14:37.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:36 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:14:37.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:36 smithi155 ceph-mon[128885]: pgmap v216: 129 pgs: 129 active+clean; 350 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 161 op/s 2023-07-18T03:14:37.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:36 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:14:37.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:36 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:14:37.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:36 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:14:37.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:36 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:14:37.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:36 smithi179 ceph-mon[129229]: pgmap v216: 129 pgs: 129 active+clean; 350 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 161 op/s 2023-07-18T03:14:37.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:36 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:14:37.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:36 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:14:37.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:36 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:14:37.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:36 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:14:37.589 INFO:tasks.workunit.client.0.smithi101.stdout: 1 507697 7.34 MB/sec execute 242 sec latency 1087.203 ms 2023-07-18T03:14:38.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:37 smithi101 ceph-mon[126379]: scrub queued for path: / 2023-07-18T03:14:38.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:37 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:14:38.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:37 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:14:38.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:37 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:14:38.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:37 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:14:38.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:37 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:14:38.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:37 smithi155 ceph-mon[128885]: scrub queued for path: / 2023-07-18T03:14:38.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:37 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:14:38.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:37 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:14:38.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:37 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:14:38.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:37 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:14:38.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:37 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:14:38.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:37 smithi179 ceph-mon[129229]: scrub queued for path: / 2023-07-18T03:14:38.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:37 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:14:38.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:37 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:14:38.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:37 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:14:38.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:37 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:14:38.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:37 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:14:38.589 INFO:tasks.workunit.client.0.smithi101.stdout: 1 508835 7.34 MB/sec execute 243 sec latency 19.420 ms 2023-07-18T03:14:39.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:38 smithi101 ceph-mon[126379]: pgmap v217: 129 pgs: 129 active+clean; 350 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 133 op/s 2023-07-18T03:14:39.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:38 smithi155 ceph-mon[128885]: pgmap v217: 129 pgs: 129 active+clean; 350 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 133 op/s 2023-07-18T03:14:39.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:38 smithi179 ceph-mon[129229]: pgmap v217: 129 pgs: 129 active+clean; 350 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 133 op/s 2023-07-18T03:14:39.590 INFO:tasks.workunit.client.0.smithi101.stdout: 1 510176 7.33 MB/sec execute 244 sec latency 81.782 ms 2023-07-18T03:14:40.590 INFO:tasks.workunit.client.0.smithi101.stdout: 1 511890 7.35 MB/sec execute 245 sec latency 19.503 ms 2023-07-18T03:14:41.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:40 smithi101 ceph-mon[126379]: pgmap v218: 129 pgs: 129 active+clean; 355 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 195 op/s 2023-07-18T03:14:41.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:40 smithi155 ceph-mon[128885]: pgmap v218: 129 pgs: 129 active+clean; 355 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 195 op/s 2023-07-18T03:14:41.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:40 smithi179 ceph-mon[129229]: pgmap v218: 129 pgs: 129 active+clean; 355 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 195 op/s 2023-07-18T03:14:41.590 INFO:tasks.workunit.client.0.smithi101.stdout: 1 513169 7.34 MB/sec execute 246 sec latency 16.979 ms 2023-07-18T03:14:42.590 INFO:tasks.workunit.client.0.smithi101.stdout: 1 513272 7.31 MB/sec execute 247 sec latency 928.464 ms 2023-07-18T03:14:43.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:42 smithi101 ceph-mon[126379]: pgmap v219: 129 pgs: 129 active+clean; 355 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 147 op/s 2023-07-18T03:14:43.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:42 smithi155 ceph-mon[128885]: pgmap v219: 129 pgs: 129 active+clean; 355 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 147 op/s 2023-07-18T03:14:43.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:42 smithi179 ceph-mon[129229]: pgmap v219: 129 pgs: 129 active+clean; 355 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 147 op/s 2023-07-18T03:14:43.590 INFO:tasks.workunit.client.0.smithi101.stdout: 1 513272 7.28 MB/sec execute 248 sec latency 1928.591 ms 2023-07-18T03:14:44.590 INFO:tasks.workunit.client.0.smithi101.stdout: 1 513671 7.26 MB/sec execute 249 sec latency 2763.777 ms 2023-07-18T03:14:45.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:44 smithi101 ceph-mon[126379]: pgmap v220: 129 pgs: 129 active+clean; 355 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 147 op/s 2023-07-18T03:14:45.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:44 smithi155 ceph-mon[128885]: pgmap v220: 129 pgs: 129 active+clean; 355 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 147 op/s 2023-07-18T03:14:45.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:44 smithi179 ceph-mon[129229]: pgmap v220: 129 pgs: 129 active+clean; 355 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 147 op/s 2023-07-18T03:14:45.590 INFO:tasks.workunit.client.0.smithi101.stdout: 1 515420 7.28 MB/sec execute 250 sec latency 28.357 ms 2023-07-18T03:14:46.590 INFO:tasks.workunit.client.0.smithi101.stdout: 1 516630 7.27 MB/sec execute 251 sec latency 16.551 ms 2023-07-18T03:14:47.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:46 smithi101 ceph-mon[126379]: pgmap v221: 129 pgs: 129 active+clean; 358 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 189 op/s 2023-07-18T03:14:47.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:46 smithi155 ceph-mon[128885]: pgmap v221: 129 pgs: 129 active+clean; 358 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 189 op/s 2023-07-18T03:14:47.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:46 smithi179 ceph-mon[129229]: pgmap v221: 129 pgs: 129 active+clean; 358 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 189 op/s 2023-07-18T03:14:47.590 INFO:tasks.workunit.client.0.smithi101.stdout: 1 516816 7.24 MB/sec execute 252 sec latency 859.047 ms 2023-07-18T03:14:48.590 INFO:tasks.workunit.client.0.smithi101.stdout: 1 516816 7.22 MB/sec execute 253 sec latency 1859.104 ms 2023-07-18T03:14:49.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:48 smithi101 ceph-mon[126379]: pgmap v222: 129 pgs: 129 active+clean; 358 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 979 KiB/s wr, 104 op/s 2023-07-18T03:14:49.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:48 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:14:49.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:48 smithi155 ceph-mon[128885]: pgmap v222: 129 pgs: 129 active+clean; 358 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 979 KiB/s wr, 104 op/s 2023-07-18T03:14:49.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:48 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:14:49.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:48 smithi179 ceph-mon[129229]: pgmap v222: 129 pgs: 129 active+clean; 358 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 979 KiB/s wr, 104 op/s 2023-07-18T03:14:49.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:48 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:14:49.590 INFO:tasks.workunit.client.0.smithi101.stdout: 1 517265 7.19 MB/sec execute 254 sec latency 2694.604 ms 2023-07-18T03:14:50.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:49 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:14:50.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:49 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:14:50.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:49 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:14:50.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:49 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:14:50.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:49 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:14:50.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:49 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:14:50.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:49 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:14:50.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:49 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:14:50.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:49 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:14:50.590 INFO:tasks.workunit.client.0.smithi101.stdout: 1 518982 7.21 MB/sec execute 255 sec latency 20.614 ms 2023-07-18T03:14:51.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:50 smithi155 ceph-mon[128885]: pgmap v223: 129 pgs: 129 active+clean; 362 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 147 op/s 2023-07-18T03:14:51.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:50 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:14:51.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:50 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:14:51.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:50 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:14:51.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:50 smithi179 ceph-mon[129229]: pgmap v223: 129 pgs: 129 active+clean; 362 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 147 op/s 2023-07-18T03:14:51.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:50 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:14:51.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:50 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:14:51.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:50 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:14:51.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:50 smithi101 ceph-mon[126379]: pgmap v223: 129 pgs: 129 active+clean; 362 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 147 op/s 2023-07-18T03:14:51.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:50 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:14:51.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:50 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:14:51.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:50 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:14:51.590 INFO:tasks.workunit.client.0.smithi101.stdout: 1 520179 7.21 MB/sec execute 256 sec latency 21.221 ms 2023-07-18T03:14:52.590 INFO:tasks.workunit.client.0.smithi101.stdout: 1 520365 7.18 MB/sec execute 257 sec latency 853.979 ms 2023-07-18T03:14:53.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:52 smithi155 ceph-mon[128885]: pgmap v224: 129 pgs: 129 active+clean; 362 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 744 KiB/s wr, 85 op/s 2023-07-18T03:14:53.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:52 smithi179 ceph-mon[129229]: pgmap v224: 129 pgs: 129 active+clean; 362 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 744 KiB/s wr, 85 op/s 2023-07-18T03:14:53.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:52 smithi101 ceph-mon[126379]: pgmap v224: 129 pgs: 129 active+clean; 362 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 744 KiB/s wr, 85 op/s 2023-07-18T03:14:53.591 INFO:tasks.workunit.client.0.smithi101.stdout: 1 520365 7.15 MB/sec execute 258 sec latency 1854.115 ms 2023-07-18T03:14:54.591 INFO:tasks.workunit.client.0.smithi101.stdout: 1 520773 7.12 MB/sec execute 259 sec latency 2686.553 ms 2023-07-18T03:14:55.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:55 smithi155 ceph-mon[128885]: pgmap v225: 129 pgs: 129 active+clean; 362 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 744 KiB/s wr, 85 op/s 2023-07-18T03:14:55.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:55 smithi179 ceph-mon[129229]: pgmap v225: 129 pgs: 129 active+clean; 362 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 744 KiB/s wr, 85 op/s 2023-07-18T03:14:55.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:55 smithi101 ceph-mon[126379]: pgmap v225: 129 pgs: 129 active+clean; 362 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 744 KiB/s wr, 85 op/s 2023-07-18T03:14:55.591 INFO:tasks.workunit.client.0.smithi101.stdout: 1 522542 7.15 MB/sec execute 260 sec latency 15.643 ms 2023-07-18T03:14:56.591 INFO:tasks.workunit.client.0.smithi101.stdout: 1 523669 7.14 MB/sec execute 261 sec latency 16.318 ms 2023-07-18T03:14:57.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:57 smithi155 ceph-mon[128885]: pgmap v226: 129 pgs: 129 active+clean; 366 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.1 MiB/s wr, 127 op/s 2023-07-18T03:14:57.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:57 smithi179 ceph-mon[129229]: pgmap v226: 129 pgs: 129 active+clean; 366 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.1 MiB/s wr, 127 op/s 2023-07-18T03:14:57.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:57 smithi101 ceph-mon[126379]: pgmap v226: 129 pgs: 129 active+clean; 366 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.1 MiB/s wr, 127 op/s 2023-07-18T03:14:57.591 INFO:tasks.workunit.client.0.smithi101.stdout: 1 523906 7.11 MB/sec execute 262 sec latency 702.928 ms 2023-07-18T03:14:58.591 INFO:tasks.workunit.client.0.smithi101.stdout: 1 523906 7.09 MB/sec execute 263 sec latency 1703.030 ms 2023-07-18T03:14:59.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:14:59 smithi155 ceph-mon[128885]: pgmap v227: 129 pgs: 129 active+clean; 366 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 749 KiB/s wr, 85 op/s 2023-07-18T03:14:59.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:14:59 smithi179 ceph-mon[129229]: pgmap v227: 129 pgs: 129 active+clean; 366 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 749 KiB/s wr, 85 op/s 2023-07-18T03:14:59.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:14:59 smithi101 ceph-mon[126379]: pgmap v227: 129 pgs: 129 active+clean; 366 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 749 KiB/s wr, 85 op/s 2023-07-18T03:14:59.591 INFO:tasks.workunit.client.0.smithi101.stdout: 1 524193 7.06 MB/sec execute 264 sec latency 2558.804 ms 2023-07-18T03:15:00.591 INFO:tasks.workunit.client.0.smithi101.stdout: 1 526010 7.08 MB/sec execute 265 sec latency 15.708 ms 2023-07-18T03:15:01.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:01 smithi155 ceph-mon[128885]: pgmap v228: 129 pgs: 129 active+clean; 369 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 130 op/s 2023-07-18T03:15:01.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:01 smithi179 ceph-mon[129229]: pgmap v228: 129 pgs: 129 active+clean; 369 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 130 op/s 2023-07-18T03:15:01.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:01 smithi101 ceph-mon[126379]: pgmap v228: 129 pgs: 129 active+clean; 369 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 130 op/s 2023-07-18T03:15:01.591 INFO:tasks.workunit.client.0.smithi101.stdout: 1 526780 7.06 MB/sec execute 266 sec latency 52.822 ms 2023-07-18T03:15:02.591 INFO:tasks.workunit.client.0.smithi101.stdout: 1 527351 7.05 MB/sec execute 267 sec latency 62.633 ms 2023-07-18T03:15:03.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:03 smithi155 ceph-mon[128885]: pgmap v229: 129 pgs: 129 active+clean; 369 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 736 KiB/s wr, 87 op/s 2023-07-18T03:15:03.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:03 smithi179 ceph-mon[129229]: pgmap v229: 129 pgs: 129 active+clean; 369 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 736 KiB/s wr, 87 op/s 2023-07-18T03:15:03.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:03 smithi101 ceph-mon[126379]: pgmap v229: 129 pgs: 129 active+clean; 369 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 736 KiB/s wr, 87 op/s 2023-07-18T03:15:03.592 INFO:tasks.workunit.client.0.smithi101.stdout: 1 527456 7.03 MB/sec execute 268 sec latency 922.462 ms 2023-07-18T03:15:04.592 INFO:tasks.workunit.client.0.smithi101.stdout: 1 527743 7.00 MB/sec execute 269 sec latency 1761.432 ms 2023-07-18T03:15:05.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:05 smithi155 ceph-mon[128885]: pgmap v230: 129 pgs: 129 active+clean; 369 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 736 KiB/s wr, 87 op/s 2023-07-18T03:15:05.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:05 smithi179 ceph-mon[129229]: pgmap v230: 129 pgs: 129 active+clean; 369 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 736 KiB/s wr, 87 op/s 2023-07-18T03:15:05.487 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:05 smithi101 ceph-mon[126379]: pgmap v230: 129 pgs: 129 active+clean; 369 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 736 KiB/s wr, 87 op/s 2023-07-18T03:15:05.592 INFO:tasks.workunit.client.0.smithi101.stdout: 1 529544 7.02 MB/sec execute 270 sec latency 19.503 ms 2023-07-18T03:15:06.592 INFO:tasks.workunit.client.0.smithi101.stdout: 1 530476 7.01 MB/sec execute 271 sec latency 16.244 ms 2023-07-18T03:15:06.964 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:15:07.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.186+0000 7f795bfff700 1 -- 172.21.15.101:0/1745791502 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7954105af0 msgr2=0x7f7954105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:07.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.186+0000 7f795bfff700 1 --2- 172.21.15.101:0/1745791502 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7954105af0 0x7f7954105ec0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7950009990 tx=0x7f7950009ca0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.187+0000 7f795bfff700 1 -- 172.21.15.101:0/1745791502 shutdown_connections 2023-07-18T03:15:07.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.187+0000 7f795bfff700 1 --2- 172.21.15.101:0/1745791502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7954100490 0x7f795410c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.187+0000 7f795bfff700 1 --2- 172.21.15.101:0/1745791502 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7954105af0 0x7f7954105ec0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.187+0000 7f795bfff700 1 --2- 172.21.15.101:0/1745791502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79540ffae0 0x7f79540fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.187+0000 7f795bfff700 1 -- 172.21.15.101:0/1745791502 >> 172.21.15.101:0/1745791502 conn(0x7f79540fb5e0 msgr2=0x7f79540fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:07.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.187+0000 7f795bfff700 1 -- 172.21.15.101:0/1745791502 shutdown_connections 2023-07-18T03:15:07.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.187+0000 7f795bfff700 1 -- 172.21.15.101:0/1745791502 wait complete. 2023-07-18T03:15:07.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.188+0000 7f795bfff700 1 Processor -- start 2023-07-18T03:15:07.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.188+0000 7f795bfff700 1 -- start start 2023-07-18T03:15:07.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.188+0000 7f795bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79540ffae0 0x7f7954197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:07.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.188+0000 7f795bfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7954100490 0x7f79541983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:07.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.188+0000 7f795bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7954105af0 0x7f795419c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:07.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.188+0000 7f795bfff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795419ce60 con 0x7f7954105af0 2023-07-18T03:15:07.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.188+0000 7f795bfff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795419cfd0 con 0x7f7954100490 2023-07-18T03:15:07.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.188+0000 7f795bfff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795419d190 con 0x7f79540ffae0 2023-07-18T03:15:07.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.189+0000 7f795a59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7954105af0 0x7f795419c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:07.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.189+0000 7f7959d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79540ffae0 0x7f7954197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:07.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.189+0000 7f7959d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79540ffae0 0x7f7954197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:46820/0 (socket says 172.21.15.101:46820) 2023-07-18T03:15:07.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.189+0000 7f7959d9b700 1 -- 172.21.15.101:0/1665559305 learned_addr learned my addr 172.21.15.101:0/1665559305 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:07.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.189+0000 7f795959a700 1 --2- 172.21.15.101:0/1665559305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7954100490 0x7f79541983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:07.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.189+0000 7f7959d9b700 1 -- 172.21.15.101:0/1665559305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7954100490 msgr2=0x7f79541983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:07.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.189+0000 7f7959d9b700 1 --2- 172.21.15.101:0/1665559305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7954100490 0x7f79541983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.189+0000 7f7959d9b700 1 -- 172.21.15.101:0/1665559305 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7954105af0 msgr2=0x7f795419c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:07.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.189+0000 7f7959d9b700 1 --2- 172.21.15.101:0/1665559305 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7954105af0 0x7f795419c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.189+0000 7f7959d9b700 1 -- 172.21.15.101:0/1665559305 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79500095f0 con 0x7f79540ffae0 2023-07-18T03:15:07.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.189+0000 7f795a59c700 1 --2- 172.21.15.101:0/1665559305 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7954105af0 0x7f795419c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:15:07.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.189+0000 7f7959d9b700 1 --2- 172.21.15.101:0/1665559305 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79540ffae0 0x7f7954197e80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f79500095c0 tx=0x7f795000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:07.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.189+0000 7f794affd700 1 -- 172.21.15.101:0/1665559305 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795001d070 con 0x7f79540ffae0 2023-07-18T03:15:07.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.189+0000 7f794affd700 1 -- 172.21.15.101:0/1665559305 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7950005050 con 0x7f79540ffae0 2023-07-18T03:15:07.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.189+0000 7f794affd700 1 -- 172.21.15.101:0/1665559305 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7950004a90 con 0x7f79540ffae0 2023-07-18T03:15:07.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.190+0000 7f795bfff700 1 -- 172.21.15.101:0/1665559305 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f795419d410 con 0x7f79540ffae0 2023-07-18T03:15:07.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.190+0000 7f795bfff700 1 -- 172.21.15.101:0/1665559305 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f795419d690 con 0x7f79540ffae0 2023-07-18T03:15:07.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.193+0000 7f794affd700 1 -- 172.21.15.101:0/1665559305 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f7950005430 con 0x7f79540ffae0 2023-07-18T03:15:07.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.194+0000 7f796f9cbb80 1 Processor -- start 2023-07-18T03:15:07.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.194+0000 7f794affd700 1 --2- 172.21.15.101:0/1665559305 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f79400a12c0 0x7f79400a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:07.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.194+0000 7f796f9cbb80 1 -- start start 2023-07-18T03:15:07.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.194+0000 7f794affd700 1 -- 172.21.15.101:0/1665559305 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f79500cde20 con 0x7f79540ffae0 2023-07-18T03:15:07.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.194+0000 7f795a59c700 1 --2- 172.21.15.101:0/1665559305 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f79400a12c0 0x7f79400a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:07.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.194+0000 7f796f9cbb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55646bed92d0 0x55646bed96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:07.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.194+0000 7f796f9cbb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55646bed9be0 0x55646bee4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:07.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.194+0000 7f796f9cbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55646bee4ad0 0x55646bee6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:07.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.194+0000 7f796f9cbb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55646bddfcb0 con 0x55646bed9be0 2023-07-18T03:15:07.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.194+0000 7f796f9cbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55646be32a40 con 0x55646bee4ad0 2023-07-18T03:15:07.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.194+0000 7f796f9cbb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55646be2bed0 con 0x55646bed92d0 2023-07-18T03:15:07.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.195+0000 7f795959a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55646bed9be0 0x55646bee4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:07.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.195+0000 7f795959a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55646bed9be0 0x55646bee4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:39510/0 (socket says 172.21.15.101:39510) 2023-07-18T03:15:07.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.195+0000 7f795959a700 1 -- 172.21.15.101:0/913328697 learned_addr learned my addr 172.21.15.101:0/913328697 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:07.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.195+0000 7f7959d9b700 1 --2- 172.21.15.101:0/913328697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55646bed92d0 0x55646bed96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:07.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.195+0000 7f795a59c700 1 --2- 172.21.15.101:0/913328697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55646bee4ad0 0x55646bee6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:07.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.195+0000 7f7959d9b700 1 -- 172.21.15.101:0/913328697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55646bee4ad0 msgr2=0x55646bee6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:07.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.195+0000 7f7959d9b700 1 --2- 172.21.15.101:0/913328697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55646bee4ad0 0x55646bee6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.195+0000 7f7959d9b700 1 -- 172.21.15.101:0/913328697 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55646bed9be0 msgr2=0x55646bee4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:07.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.195+0000 7f7959d9b700 1 --2- 172.21.15.101:0/913328697 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55646bed9be0 0x55646bee4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.192 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.195+0000 7f7959d9b700 1 -- 172.21.15.101:0/913328697 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55646bc59de0 con 0x55646bed92d0 2023-07-18T03:15:07.192 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.195+0000 7f795959a700 1 --2- 172.21.15.101:0/913328697 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55646bed9be0 0x55646bee4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:15:07.192 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.195+0000 7f795a59c700 1 --2- 172.21.15.101:0/1665559305 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f79400a12c0 0x7f79400a3770 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f794c009550 tx=0x7f794c007e30 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:07.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.196+0000 7f7959d9b700 1 --2- 172.21.15.101:0/913328697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55646bed92d0 0x55646bed96a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f79500cb110 tx=0x7f795009a770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a18966bf814e64be server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:07.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.196+0000 7f793ffff700 1 -- 172.21.15.101:0/913328697 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795009eb50 con 0x55646bed92d0 2023-07-18T03:15:07.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.196+0000 7f793ffff700 1 -- 172.21.15.101:0/913328697 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79500a2ee0 con 0x55646bed92d0 2023-07-18T03:15:07.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.196+0000 7f793ffff700 1 -- 172.21.15.101:0/913328697 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795009af80 con 0x55646bed92d0 2023-07-18T03:15:07.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.196+0000 7f796f9cbb80 1 -- 172.21.15.101:0/913328697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55646bed92d0 msgr2=0x55646bed96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:07.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.196+0000 7f796f9cbb80 1 --2- 172.21.15.101:0/913328697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55646bed92d0 0x55646bed96a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f79500cb110 tx=0x7f795009a770 comp rx=0 tx=0).stop 2023-07-18T03:15:07.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.197+0000 7f796f9cbb80 1 -- 172.21.15.101:0/913328697 shutdown_connections 2023-07-18T03:15:07.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.197+0000 7f796f9cbb80 1 --2- 172.21.15.101:0/913328697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55646bee4ad0 0x55646bee6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.197+0000 7f796f9cbb80 1 --2- 172.21.15.101:0/913328697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55646bed92d0 0x55646bed96a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.197+0000 7f796f9cbb80 1 --2- 172.21.15.101:0/913328697 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55646bed9be0 0x55646bee4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.197+0000 7f796f9cbb80 1 -- 172.21.15.101:0/913328697 >> 172.21.15.101:0/913328697 conn(0x55646becfda0 msgr2=0x55646bed56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:07.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.197+0000 7f796f9cbb80 1 -- 172.21.15.101:0/913328697 shutdown_connections 2023-07-18T03:15:07.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.197+0000 7f796f9cbb80 1 -- 172.21.15.101:0/913328697 wait complete. 2023-07-18T03:15:07.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.197+0000 7f796f9cbb80 1 -- start start 2023-07-18T03:15:07.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.197+0000 7f796f9cbb80 1 Processor -- start 2023-07-18T03:15:07.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.198+0000 7f796f9cbb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55646bed92d0 0x55646bee4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:07.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.198+0000 7f796f9cbb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55646bed9be0 0x55646bee53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:07.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.198+0000 7f796f9cbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55646bee4ad0 0x55646bee5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:07.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.198+0000 7f796f9cbb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55646bdcbac0 con 0x55646bed92d0 2023-07-18T03:15:07.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.198+0000 7f796f9cbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55646bdcb7c0 con 0x55646bee4ad0 2023-07-18T03:15:07.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.198+0000 7f796f9cbb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55646bcaf920 con 0x55646bed9be0 2023-07-18T03:15:07.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.198+0000 7f796f9cbb80 10 client.0 _ll_register_callbacks cb 0x55646bdc9760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:15:07.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.198+0000 7f7959d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55646bed92d0 0x55646bee4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:07.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.198+0000 7f7959d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55646bed92d0 0x55646bee4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:39514/0 (socket says 172.21.15.101:39514) 2023-07-18T03:15:07.198 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.198+0000 7f7959d9b700 1 -- 172.21.15.101:0/3363403875 learned_addr learned my addr 172.21.15.101:0/3363403875 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:07.198 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.198+0000 7f795959a700 1 --2- 172.21.15.101:0/3363403875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55646bed9be0 0x55646bee53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:07.198 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.198+0000 7f795a59c700 1 --2- 172.21.15.101:0/3363403875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55646bee4ad0 0x55646bee5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:07.198 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.198+0000 7f7959d9b700 1 -- 172.21.15.101:0/3363403875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55646bed9be0 msgr2=0x55646bee53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:07.198 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.198+0000 7f7959d9b700 1 --2- 172.21.15.101:0/3363403875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55646bed9be0 0x55646bee53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.198 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.198+0000 7f7959d9b700 1 -- 172.21.15.101:0/3363403875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55646bee4ad0 msgr2=0x55646bee5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:07.198 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.198+0000 7f7959d9b700 1 --2- 172.21.15.101:0/3363403875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55646bee4ad0 0x55646bee5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.198 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.198+0000 7f7959d9b700 1 -- 172.21.15.101:0/3363403875 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79440078c0 con 0x55646bed92d0 2023-07-18T03:15:07.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.199+0000 7f7959d9b700 1 --2- 172.21.15.101:0/3363403875 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55646bed92d0 0x55646bee4ea0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f7950096fd0 tx=0x7f7950004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2ce78bb377a51d6 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:07.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.199+0000 7f793d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:15:07.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.199+0000 7f793d7fa700 1 -- 172.21.15.101:0/3363403875 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795009a010 con 0x55646bed92d0 2023-07-18T03:15:07.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.199+0000 7f793d7fa700 1 -- 172.21.15.101:0/3363403875 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f795009a170 con 0x55646bed92d0 2023-07-18T03:15:07.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.199+0000 7f796f9cbb80 1 -- 172.21.15.101:0/3363403875 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55646bd854f0 con 0x55646bed92d0 2023-07-18T03:15:07.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.199+0000 7f793d7fa700 1 -- 172.21.15.101:0/3363403875 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f795009ad80 con 0x55646bed92d0 2023-07-18T03:15:07.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.199+0000 7f796f9cbb80 10 client.17016 fetch_fsmap learned FSMap version 29 2023-07-18T03:15:07.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.199+0000 7f796f9cbb80 1 -- 172.21.15.101:0/3363403875 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f795419d410 con 0x55646bed92d0 2023-07-18T03:15:07.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.199+0000 7f793d7fa700 1 -- 172.21.15.101:0/3363403875 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7950064020 con 0x55646bed92d0 2023-07-18T03:15:07.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.200+0000 7f793d7fa700 1 -- 172.21.15.101:0/3363403875 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f795000fad0 con 0x55646bed92d0 2023-07-18T03:15:07.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.200+0000 7f796f9cbb80 10 client.17016 fetch_fsmap finished waiting for FSMap version 29 2023-07-18T03:15:07.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.200+0000 7f796f9cbb80 10 client.17016 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:15:07.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.200+0000 7f796f9cbb80 20 client.17016 populate_metadata read hostname 'smithi101' 2023-07-18T03:15:07.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.200+0000 7f796f9cbb80 1 --2- 172.21.15.101:0/3363403875 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55646bef3ce0 0x55646bef60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:07.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.200+0000 7f796f9cbb80 4 client.17016 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:15:07.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.200+0000 7f796f9cbb80 1 -- 172.21.15.101:0/3363403875 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55646be24eb0 con 0x55646bef3ce0 2023-07-18T03:15:07.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.200+0000 7f795959a700 1 --2- 172.21.15.101:0/3363403875 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55646bef3ce0 0x55646bef60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:07.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.200+0000 7f793d7fa700 10 client.17016 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:15:07.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.200+0000 7f795959a700 1 --2- 172.21.15.101:0/3363403875 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55646bef3ce0 0x55646bef60f0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=416aa8cdfd377f3f server_cookie=24c0edafd9662c3a in_seq=0 out_seq=0 2023-07-18T03:15:07.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.201+0000 7f793d7fa700 1 -- 172.21.15.101:0/3363403875 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55646be24eb0 con 0x55646bef3ce0 2023-07-18T03:15:07.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.201+0000 7f793d7fa700 10 client.17016 handle_command_reply: tid=0 2023-07-18T03:15:07.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.201+0000 7f796f9cbb80 1 client.17016 shutdown 2023-07-18T03:15:07.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.201+0000 7f796f9cbb80 1 -- 172.21.15.101:0/3363403875 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55646bed92d0 msgr2=0x55646bee4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:07.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.201+0000 7f796f9cbb80 1 --2- 172.21.15.101:0/3363403875 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55646bed92d0 0x55646bee4ea0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f7950096fd0 tx=0x7f7950004f90 comp rx=0 tx=0).stop 2023-07-18T03:15:07.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.201+0000 7f796f9cbb80 1 -- 172.21.15.101:0/3363403875 shutdown_connections 2023-07-18T03:15:07.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.201+0000 7f796f9cbb80 1 --2- 172.21.15.101:0/3363403875 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55646bef3ce0 0x55646bef60f0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.201+0000 7f796f9cbb80 1 --2- 172.21.15.101:0/3363403875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55646bee4ad0 0x55646bee5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.201+0000 7f796f9cbb80 1 --2- 172.21.15.101:0/3363403875 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55646bed92d0 0x55646bee4ea0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.201+0000 7f793d7fa700 0 client.17016 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:15:07.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.201+0000 7f796f9cbb80 1 --2- 172.21.15.101:0/3363403875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55646bed9be0 0x55646bee53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.201+0000 7f796f9cbb80 1 -- 172.21.15.101:0/3363403875 >> 172.21.15.101:0/3363403875 conn(0x55646becfda0 msgr2=0x55646bed5290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:07.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.202+0000 7f796f9cbb80 1 -- 172.21.15.101:0/3363403875 shutdown_connections 2023-07-18T03:15:07.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.202+0000 7f796f9cbb80 1 -- 172.21.15.101:0/3363403875 wait complete. 2023-07-18T03:15:07.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.202+0000 7f796f9cbb80 20 client.17016 trim_cache size 0 max 16384 2023-07-18T03:15:07.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.219+0000 7f796f9cbb80 1 Processor -- start 2023-07-18T03:15:07.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.219+0000 7f796f9cbb80 1 -- start start 2023-07-18T03:15:07.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.219+0000 7f796f9cbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55646bed92d0 0x55646bede540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:07.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.219+0000 7f796f9cbb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55646bed9be0 0x55646bef0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:07.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.220+0000 7f796f9cbb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55646bee4ad0 0x55646beef660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:07.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.220+0000 7f796f9cbb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55646bedc250 con 0x55646bee4ad0 2023-07-18T03:15:07.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.220+0000 7f796f9cbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55646beda540 con 0x55646bed92d0 2023-07-18T03:15:07.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.220+0000 7f796f9cbb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55646bed9fb0 con 0x55646bed9be0 2023-07-18T03:15:07.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.220+0000 7f795a59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55646bee4ad0 0x55646beef660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:07.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.220+0000 7f795959a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55646bed9be0 0x55646bef0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:07.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.220+0000 7f7959d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55646bed92d0 0x55646bede540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:07.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.220+0000 7f795959a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55646bed9be0 0x55646bef0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:46842/0 (socket says 172.21.15.101:46842) 2023-07-18T03:15:07.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.220+0000 7f795959a700 1 -- 172.21.15.101:0/401221280 learned_addr learned my addr 172.21.15.101:0/401221280 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:07.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.220+0000 7f7959d9b700 1 -- 172.21.15.101:0/401221280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55646bed9be0 msgr2=0x55646bef0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:07.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.220+0000 7f7959d9b700 1 --2- 172.21.15.101:0/401221280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55646bed9be0 0x55646bef0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.220+0000 7f7959d9b700 1 -- 172.21.15.101:0/401221280 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55646bee4ad0 msgr2=0x55646beef660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:07.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.220+0000 7f7959d9b700 1 --2- 172.21.15.101:0/401221280 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55646bee4ad0 0x55646beef660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.220+0000 7f7959d9b700 1 -- 172.21.15.101:0/401221280 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55646bc42c60 con 0x55646bed92d0 2023-07-18T03:15:07.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.220+0000 7f7959d9b700 1 --2- 172.21.15.101:0/401221280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55646bed92d0 0x55646bede540 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7950097590 tx=0x7f79500990d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5445a3b77ee99695 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:07.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.221+0000 7f7948ff9700 1 -- 172.21.15.101:0/401221280 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795000fad0 con 0x55646bed92d0 2023-07-18T03:15:07.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.221+0000 7f7948ff9700 1 -- 172.21.15.101:0/401221280 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55646bd854f0 con 0x55646bed92d0 2023-07-18T03:15:07.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.221+0000 7f7948ff9700 1 -- 172.21.15.101:0/401221280 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795009afe0 con 0x55646bed92d0 2023-07-18T03:15:07.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.221+0000 7f796f9cbb80 1 -- 172.21.15.101:0/401221280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55646bed92d0 msgr2=0x55646bede540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:07.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.221+0000 7f796f9cbb80 1 --2- 172.21.15.101:0/401221280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55646bed92d0 0x55646bede540 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7950097590 tx=0x7f79500990d0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.221+0000 7f796f9cbb80 1 -- 172.21.15.101:0/401221280 shutdown_connections 2023-07-18T03:15:07.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.221+0000 7f796f9cbb80 1 --2- 172.21.15.101:0/401221280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55646bed92d0 0x55646bede540 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.221+0000 7f796f9cbb80 1 --2- 172.21.15.101:0/401221280 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55646bee4ad0 0x55646beef660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.221+0000 7f796f9cbb80 1 --2- 172.21.15.101:0/401221280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55646bed9be0 0x55646bef0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.221+0000 7f796f9cbb80 1 -- 172.21.15.101:0/401221280 >> 172.21.15.101:0/401221280 conn(0x55646becfda0 msgr2=0x55646bee5730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:07.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.221+0000 7f796f9cbb80 1 -- 172.21.15.101:0/401221280 shutdown_connections 2023-07-18T03:15:07.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.221+0000 7f796f9cbb80 1 -- 172.21.15.101:0/401221280 wait complete. 2023-07-18T03:15:07.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.222+0000 7f796f9cbb80 1 -- start start 2023-07-18T03:15:07.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.222+0000 7f796f9cbb80 1 Processor -- start 2023-07-18T03:15:07.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.222+0000 7f796f9cbb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55646bed92d0 0x55646bee9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:07.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.222+0000 7f796f9cbb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55646bed9be0 0x55646bee62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:07.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.222+0000 7f796f9cbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55646bee4ad0 0x55646bee6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:07.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.222+0000 7f796f9cbb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55646bee6fa0 con 0x55646bed92d0 2023-07-18T03:15:07.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.222+0000 7f796f9cbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55646bee7110 con 0x55646bee4ad0 2023-07-18T03:15:07.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.222+0000 7f796f9cbb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55646bf47ba0 con 0x55646bed9be0 2023-07-18T03:15:07.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.223+0000 7f796f9cbb80 10 client.0 _ll_register_callbacks cb 0x7f79440077f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:15:07.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.223+0000 7f7959d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55646bed92d0 0x55646bee9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:07.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.223+0000 7f7959d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55646bed92d0 0x55646bee9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:39536/0 (socket says 172.21.15.101:39536) 2023-07-18T03:15:07.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.223+0000 7f7959d9b700 1 -- 172.21.15.101:0/2746819872 learned_addr learned my addr 172.21.15.101:0/2746819872 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:07.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.223+0000 7f795959a700 1 --2- 172.21.15.101:0/2746819872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55646bed9be0 0x55646bee62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:07.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.223+0000 7f795a59c700 1 --2- 172.21.15.101:0/2746819872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55646bee4ad0 0x55646bee6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:07.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.223+0000 7f7959d9b700 1 -- 172.21.15.101:0/2746819872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55646bed9be0 msgr2=0x55646bee62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:07.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.223+0000 7f7959d9b700 1 --2- 172.21.15.101:0/2746819872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55646bed9be0 0x55646bee62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.223+0000 7f7959d9b700 1 -- 172.21.15.101:0/2746819872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55646bee4ad0 msgr2=0x55646bee6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:07.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.223+0000 7f7959d9b700 1 --2- 172.21.15.101:0/2746819872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55646bee4ad0 0x55646bee6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.223+0000 7f7959d9b700 1 -- 172.21.15.101:0/2746819872 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f795419d410 con 0x55646bed92d0 2023-07-18T03:15:07.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.223+0000 7f795a59c700 1 --2- 172.21.15.101:0/2746819872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55646bee4ad0 0x55646bee6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:15:07.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.223+0000 7f7959d9b700 1 --2- 172.21.15.101:0/2746819872 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55646bed92d0 0x55646bee9f40 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f79500a29b0 tx=0x7f79500cb130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a09f05bbd06fc9 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:07.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.223+0000 7f793d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:15:07.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.224+0000 7f793d7fa700 1 -- 172.21.15.101:0/2746819872 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79500a2c50 con 0x55646bed92d0 2023-07-18T03:15:07.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.224+0000 7f793d7fa700 1 -- 172.21.15.101:0/2746819872 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f7950065080 con 0x55646bed92d0 2023-07-18T03:15:07.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.224+0000 7f796f9cbb80 1 -- 172.21.15.101:0/2746819872 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f79500a2ee0 con 0x55646bed92d0 2023-07-18T03:15:07.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.224+0000 7f793d7fa700 1 -- 172.21.15.101:0/2746819872 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7950099e00 con 0x55646bed92d0 2023-07-18T03:15:07.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.224+0000 7f796f9cbb80 10 client.17022 fetch_fsmap learned FSMap version 29 2023-07-18T03:15:07.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.224+0000 7f796f9cbb80 1 -- 172.21.15.101:0/2746819872 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55646bc80250 con 0x55646bed92d0 2023-07-18T03:15:07.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.224+0000 7f793d7fa700 1 -- 172.21.15.101:0/2746819872 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79500a2ee0 con 0x55646bed92d0 2023-07-18T03:15:07.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.224+0000 7f793d7fa700 1 -- 172.21.15.101:0/2746819872 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f795009a110 con 0x55646bed92d0 2023-07-18T03:15:07.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.224+0000 7f796f9cbb80 10 client.17022 fetch_fsmap finished waiting for FSMap version 29 2023-07-18T03:15:07.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.224+0000 7f796f9cbb80 10 client.17022 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:15:07.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.224+0000 7f796f9cbb80 20 client.17022 populate_metadata read hostname 'smithi101' 2023-07-18T03:15:07.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.224+0000 7f796f9cbb80 1 --2- 172.21.15.101:0/2746819872 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55646bf48de0 0x55646bf4b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:07.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.224+0000 7f796f9cbb80 4 client.17022 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:15:07.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.224+0000 7f796f9cbb80 1 -- 172.21.15.101:0/2746819872 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55646bde7050 con 0x55646bf48de0 2023-07-18T03:15:07.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.225+0000 7f795959a700 1 --2- 172.21.15.101:0/2746819872 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55646bf48de0 0x55646bf4b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:07.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.225+0000 7f793d7fa700 10 client.17022 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:15:07.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.225+0000 7f795959a700 1 --2- 172.21.15.101:0/2746819872 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55646bf48de0 0x55646bf4b1c0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6861c6aa35222d2 server_cookie=7c2d0052ec2e589c in_seq=0 out_seq=0 2023-07-18T03:15:07.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.225+0000 7f793d7fa700 1 -- 172.21.15.101:0/2746819872 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55646bde7050 con 0x55646bf48de0 2023-07-18T03:15:07.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.225+0000 7f793d7fa700 10 client.17022 handle_command_reply: tid=0 2023-07-18T03:15:07.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.225+0000 7f796f9cbb80 1 client.17022 shutdown 2023-07-18T03:15:07.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.226+0000 7f796f9cbb80 1 -- 172.21.15.101:0/2746819872 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55646bed92d0 msgr2=0x55646bee9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:07.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.226+0000 7f796f9cbb80 1 --2- 172.21.15.101:0/2746819872 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55646bed92d0 0x55646bee9f40 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f79500a29b0 tx=0x7f79500cb130 comp rx=0 tx=0).stop 2023-07-18T03:15:07.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.226+0000 7f796f9cbb80 1 -- 172.21.15.101:0/2746819872 shutdown_connections 2023-07-18T03:15:07.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.226+0000 7f796f9cbb80 1 --2- 172.21.15.101:0/2746819872 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55646bf48de0 0x55646bf4b1c0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.226+0000 7f796f9cbb80 1 --2- 172.21.15.101:0/2746819872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55646bee4ad0 0x55646bee6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.226+0000 7f796f9cbb80 1 --2- 172.21.15.101:0/2746819872 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55646bed92d0 0x55646bee9f40 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.226+0000 7f793d7fa700 0 client.17022 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:15:07.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.226+0000 7f796f9cbb80 1 --2- 172.21.15.101:0/2746819872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55646bed9be0 0x55646bee62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.226+0000 7f796f9cbb80 1 -- 172.21.15.101:0/2746819872 >> 172.21.15.101:0/2746819872 conn(0x55646becfda0 msgr2=0x55646bedba30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:07.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.226+0000 7f796f9cbb80 1 -- 172.21.15.101:0/2746819872 shutdown_connections 2023-07-18T03:15:07.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.226+0000 7f796f9cbb80 1 -- 172.21.15.101:0/2746819872 wait complete. 2023-07-18T03:15:07.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.226+0000 7f796f9cbb80 20 client.17022 trim_cache size 0 max 16384 2023-07-18T03:15:07.229 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:15:07.229 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "no active scrubs running", 2023-07-18T03:15:07.229 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": {} 2023-07-18T03:15:07.229 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:15:07.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.226+0000 7f795bfff700 1 -- 172.21.15.101:0/1665559305 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f79400a12c0 msgr2=0x7f79400a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:07.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.226+0000 7f795bfff700 1 --2- 172.21.15.101:0/1665559305 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f79400a12c0 0x7f79400a3770 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f794c009550 tx=0x7f794c007e30 comp rx=0 tx=0).stop 2023-07-18T03:15:07.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.226+0000 7f795bfff700 1 -- 172.21.15.101:0/1665559305 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79540ffae0 msgr2=0x7f7954197e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:07.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.226+0000 7f795bfff700 1 --2- 172.21.15.101:0/1665559305 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79540ffae0 0x7f7954197e80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f79500095c0 tx=0x7f795000f690 comp rx=0 tx=0).stop 2023-07-18T03:15:07.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.226+0000 7f795bfff700 1 -- 172.21.15.101:0/1665559305 shutdown_connections 2023-07-18T03:15:07.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.226+0000 7f795bfff700 1 --2- 172.21.15.101:0/1665559305 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7954105af0 0x7f795419c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.226+0000 7f795bfff700 1 --2- 172.21.15.101:0/1665559305 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79540ffae0 0x7f7954197e80 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.226+0000 7f795bfff700 1 --2- 172.21.15.101:0/1665559305 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f79400a12c0 0x7f79400a3770 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.226+0000 7f795bfff700 1 --2- 172.21.15.101:0/1665559305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7954100490 0x7f79541983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.226+0000 7f795bfff700 1 -- 172.21.15.101:0/1665559305 >> 172.21.15.101:0/1665559305 conn(0x7f79540fb5e0 msgr2=0x7f795410aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:07.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.227+0000 7f795bfff700 1 -- 172.21.15.101:0/1665559305 shutdown_connections 2023-07-18T03:15:07.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.227+0000 7f795bfff700 1 -- 172.21.15.101:0/1665559305 wait complete. 2023-07-18T03:15:07.238 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-18T03:15:07.238 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:15:07.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:07 smithi155 ceph-mon[128885]: pgmap v231: 129 pgs: 129 active+clean; 376 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 176 op/s 2023-07-18T03:15:07.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:07 smithi155 ceph-mon[128885]: osdmap e95: 12 total, 12 up, 12 in 2023-07-18T03:15:07.300 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:07 smithi101 ceph-mon[126379]: pgmap v231: 129 pgs: 129 active+clean; 376 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 176 op/s 2023-07-18T03:15:07.300 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:07 smithi101 ceph-mon[126379]: osdmap e95: 12 total, 12 up, 12 in 2023-07-18T03:15:07.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.443+0000 7f55c9bff700 1 -- 172.21.15.101:0/3258990165 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55c4105b00 msgr2=0x7f55c4105ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:07.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.443+0000 7f55c9bff700 1 --2- 172.21.15.101:0/3258990165 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55c4105b00 0x7f55c4105ed0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009990 tx=0x7f55b4009ca0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.443+0000 7f55c9bff700 1 -- 172.21.15.101:0/3258990165 shutdown_connections 2023-07-18T03:15:07.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.443+0000 7f55c9bff700 1 --2- 172.21.15.101:0/3258990165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55c41004a0 0x7f55c410c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.443+0000 7f55c9bff700 1 --2- 172.21.15.101:0/3258990165 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55c4105b00 0x7f55c4105ed0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.443+0000 7f55c9bff700 1 --2- 172.21.15.101:0/3258990165 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55c40ffaf0 0x7f55c40fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.443+0000 7f55c9bff700 1 -- 172.21.15.101:0/3258990165 >> 172.21.15.101:0/3258990165 conn(0x7f55c40fb630 msgr2=0x7f55c40fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:07.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.443+0000 7f55c9bff700 1 -- 172.21.15.101:0/3258990165 shutdown_connections 2023-07-18T03:15:07.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.443+0000 7f55c9bff700 1 -- 172.21.15.101:0/3258990165 wait complete. 2023-07-18T03:15:07.440 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.444+0000 7f55c9bff700 1 Processor -- start 2023-07-18T03:15:07.440 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.444+0000 7f55c9bff700 1 -- start start 2023-07-18T03:15:07.440 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.444+0000 7f55c9bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55c40ffaf0 0x7f55c4197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:07.440 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.444+0000 7f55c9bff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55c41004a0 0x7f55c4198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:07.440 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.444+0000 7f55c9bff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55c4105b00 0x7f55c419c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:07.440 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.444+0000 7f55c9bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c419cda0 con 0x7f55c40ffaf0 2023-07-18T03:15:07.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.444+0000 7f55c9bff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c419cf10 con 0x7f55c4105b00 2023-07-18T03:15:07.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.444+0000 7f55c9bff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c419d0d0 con 0x7f55c41004a0 2023-07-18T03:15:07.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.445+0000 7f55c37fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55c40ffaf0 0x7f55c4197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:07.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.445+0000 7f55c37fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55c40ffaf0 0x7f55c4197dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:39554/0 (socket says 172.21.15.101:39554) 2023-07-18T03:15:07.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.445+0000 7f55c37fe700 1 -- 172.21.15.101:0/834197619 learned_addr learned my addr 172.21.15.101:0/834197619 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:07.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.445+0000 7f55bbfff700 1 --2- 172.21.15.101:0/834197619 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55c41004a0 0x7f55c4198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:07.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.445+0000 7f55c3fff700 1 --2- 172.21.15.101:0/834197619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55c4105b00 0x7f55c419c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:07.442 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.445+0000 7f55c37fe700 1 -- 172.21.15.101:0/834197619 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55c41004a0 msgr2=0x7f55c4198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:07.442 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.445+0000 7f55c37fe700 1 --2- 172.21.15.101:0/834197619 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55c41004a0 0x7f55c4198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.442 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.445+0000 7f55c37fe700 1 -- 172.21.15.101:0/834197619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55c4105b00 msgr2=0x7f55c419c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:07.442 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.445+0000 7f55c37fe700 1 --2- 172.21.15.101:0/834197619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55c4105b00 0x7f55c419c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.442 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.445+0000 7f55c37fe700 1 -- 172.21.15.101:0/834197619 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55bc009c40 con 0x7f55c40ffaf0 2023-07-18T03:15:07.442 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.445+0000 7f55c37fe700 1 --2- 172.21.15.101:0/834197619 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55c40ffaf0 0x7f55c4197dc0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f55b4000c00 tx=0x7f55b4004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:07.442 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.445+0000 7f55c17fa700 1 -- 172.21.15.101:0/834197619 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b401d070 con 0x7f55c40ffaf0 2023-07-18T03:15:07.442 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.446+0000 7f55c9bff700 1 -- 172.21.15.101:0/834197619 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55b40095f0 con 0x7f55c40ffaf0 2023-07-18T03:15:07.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.446+0000 7f55c17fa700 1 -- 172.21.15.101:0/834197619 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55b4021410 con 0x7f55c40ffaf0 2023-07-18T03:15:07.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.446+0000 7f55c17fa700 1 -- 172.21.15.101:0/834197619 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b4017650 con 0x7f55c40ffaf0 2023-07-18T03:15:07.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.446+0000 7f55c9bff700 1 -- 172.21.15.101:0/834197619 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55c41a4a10 con 0x7f55c40ffaf0 2023-07-18T03:15:07.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.447+0000 7f55c9bff700 1 -- 172.21.15.101:0/834197619 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55c4067300 con 0x7f55c40ffaf0 2023-07-18T03:15:07.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.448+0000 7f55c17fa700 1 -- 172.21.15.101:0/834197619 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f55b4021580 con 0x7f55c40ffaf0 2023-07-18T03:15:07.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.450+0000 7f55c17fa700 1 --2- 172.21.15.101:0/834197619 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f55b00a1300 0x7f55b00a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:07.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.450+0000 7f55c17fa700 1 -- 172.21.15.101:0/834197619 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f55b40ce4a0 con 0x7f55c40ffaf0 2023-07-18T03:15:07.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.450+0000 7f55bbfff700 1 --2- 172.21.15.101:0/834197619 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f55b00a1300 0x7f55b00a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:07.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.451+0000 7f55bbfff700 1 --2- 172.21.15.101:0/834197619 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f55b00a1300 0x7f55b00a37b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f55c41993d0 tx=0x7f55ac009260 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:07.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.454+0000 7f55c17fa700 1 -- 172.21.15.101:0/834197619 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f55b40974e0 con 0x7f55c40ffaf0 2023-07-18T03:15:07.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:07 smithi179 ceph-mon[129229]: pgmap v231: 129 pgs: 129 active+clean; 376 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 176 op/s 2023-07-18T03:15:07.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:07 smithi179 ceph-mon[129229]: osdmap e95: 12 total, 12 up, 12 in 2023-07-18T03:15:07.592 INFO:tasks.workunit.client.0.smithi101.stdout: 1 530999 6.99 MB/sec execute 272 sec latency 461.976 ms 2023-07-18T03:15:07.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.711+0000 7f55c9bff700 1 -- 172.21.15.101:0/834197619 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f55c41a4c00 con 0x7f55c40ffaf0 2023-07-18T03:15:07.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.712+0000 7f55c17fa700 1 -- 172.21.15.101:0/834197619 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10863 (secure 0 0 0) 0x7f55b409ab00 con 0x7f55c40ffaf0 2023-07-18T03:15:07.707 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:15:07.707 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/4192846769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":4192846769},{"type":"v1","addr":"172.21.15.101:6839","nonce":4192846769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6841/3824340175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6840","nonce":3824340175},{"type":"v1","addr":"172.21.15.101:6841","nonce":3824340175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:04:59.361185+0000","modified":"2023-07-18T03:14:01.125192+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.155:6839/2055462483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2055462483},{"type":"v1","addr":"172.21.15.155:6839","nonce":2055462483}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:15:07.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.714+0000 7f55c9bff700 1 -- 172.21.15.101:0/834197619 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f55b00a1300 msgr2=0x7f55b00a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:07.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.714+0000 7f55c9bff700 1 --2- 172.21.15.101:0/834197619 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f55b00a1300 0x7f55b00a37b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f55c41993d0 tx=0x7f55ac009260 comp rx=0 tx=0).stop 2023-07-18T03:15:07.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.714+0000 7f55c9bff700 1 -- 172.21.15.101:0/834197619 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55c40ffaf0 msgr2=0x7f55c4197dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:07.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.714+0000 7f55c9bff700 1 --2- 172.21.15.101:0/834197619 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55c40ffaf0 0x7f55c4197dc0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f55b4000c00 tx=0x7f55b4004790 comp rx=0 tx=0).stop 2023-07-18T03:15:07.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.714+0000 7f55c9bff700 1 -- 172.21.15.101:0/834197619 shutdown_connections 2023-07-18T03:15:07.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.714+0000 7f55c9bff700 1 --2- 172.21.15.101:0/834197619 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f55b00a1300 0x7f55b00a37b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.714+0000 7f55c9bff700 1 --2- 172.21.15.101:0/834197619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55c4105b00 0x7f55c419c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.714+0000 7f55c9bff700 1 --2- 172.21.15.101:0/834197619 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55c40ffaf0 0x7f55c4197dc0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.714+0000 7f55c9bff700 1 --2- 172.21.15.101:0/834197619 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55c41004a0 0x7f55c4198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.714+0000 7f55c9bff700 1 -- 172.21.15.101:0/834197619 >> 172.21.15.101:0/834197619 conn(0x7f55c40fb630 msgr2=0x7f55c410add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:07.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.714+0000 7f55c9bff700 1 -- 172.21.15.101:0/834197619 shutdown_connections 2023-07-18T03:15:07.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.714+0000 7f55c9bff700 1 -- 172.21.15.101:0/834197619 wait complete. 2023-07-18T03:15:07.713 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 29 2023-07-18T03:15:07.727 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-18T03:15:07.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.937+0000 7f4af6bff700 1 -- 172.21.15.101:0/2448349264 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4af00ffae0 msgr2=0x7f4af00fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:07.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.937+0000 7f4af6bff700 1 --2- 172.21.15.101:0/2448349264 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4af00ffae0 0x7f4af00fff50 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f4adc009990 tx=0x7f4adc009ca0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.937+0000 7f4af6bff700 1 -- 172.21.15.101:0/2448349264 shutdown_connections 2023-07-18T03:15:07.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.937+0000 7f4af6bff700 1 --2- 172.21.15.101:0/2448349264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4af0105af0 0x7f4af0105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.937+0000 7f4af6bff700 1 --2- 172.21.15.101:0/2448349264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4af0100490 0x7f4af010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.937+0000 7f4af6bff700 1 --2- 172.21.15.101:0/2448349264 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4af00ffae0 0x7f4af00fff50 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.937+0000 7f4af6bff700 1 -- 172.21.15.101:0/2448349264 >> 172.21.15.101:0/2448349264 conn(0x7f4af00fb5e0 msgr2=0x7f4af00fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:07.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.937+0000 7f4af6bff700 1 -- 172.21.15.101:0/2448349264 shutdown_connections 2023-07-18T03:15:07.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.937+0000 7f4af6bff700 1 -- 172.21.15.101:0/2448349264 wait complete. 2023-07-18T03:15:07.934 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.938+0000 7f4af6bff700 1 Processor -- start 2023-07-18T03:15:07.934 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.938+0000 7f4af6bff700 1 -- start start 2023-07-18T03:15:07.934 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.939+0000 7f4af6bff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4af00ffae0 0x7f4af0072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:07.934 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.939+0000 7f4af6bff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4af0100490 0x7f4af0072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:07.935 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.939+0000 7f4af6bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4af0105af0 0x7f4af01a3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:07.935 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.939+0000 7f4af6bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af0073580 con 0x7f4af0105af0 2023-07-18T03:15:07.935 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.939+0000 7f4af6bff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af01a4340 con 0x7f4af0100490 2023-07-18T03:15:07.935 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.939+0000 7f4af6bff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af01a4480 con 0x7f4af00ffae0 2023-07-18T03:15:07.935 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.939+0000 7f4af519c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4af0105af0 0x7f4af01a3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:07.935 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.939+0000 7f4af519c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4af0105af0 0x7f4af01a3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:39560/0 (socket says 172.21.15.101:39560) 2023-07-18T03:15:07.935 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.939+0000 7f4aeffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4af0100490 0x7f4af0072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:07.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.939+0000 7f4af519c700 1 -- 172.21.15.101:0/2537155759 learned_addr learned my addr 172.21.15.101:0/2537155759 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:07.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.939+0000 7f4af499b700 1 --2- 172.21.15.101:0/2537155759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4af00ffae0 0x7f4af0072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:07.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.939+0000 7f4aeffff700 1 -- 172.21.15.101:0/2537155759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4af00ffae0 msgr2=0x7f4af0072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:07.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.939+0000 7f4aeffff700 1 --2- 172.21.15.101:0/2537155759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4af00ffae0 0x7f4af0072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.939+0000 7f4aeffff700 1 -- 172.21.15.101:0/2537155759 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4af0105af0 msgr2=0x7f4af01a3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:07.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.939+0000 7f4aeffff700 1 --2- 172.21.15.101:0/2537155759 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4af0105af0 0x7f4af01a3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.939+0000 7f4aeffff700 1 -- 172.21.15.101:0/2537155759 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4adc0095f0 con 0x7f4af0100490 2023-07-18T03:15:07.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.939+0000 7f4af519c700 1 --2- 172.21.15.101:0/2537155759 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4af0105af0 0x7f4af01a3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:15:07.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.940+0000 7f4aeffff700 1 --2- 172.21.15.101:0/2537155759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4af0100490 0x7f4af0072d50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4adc009990 tx=0x7f4adc009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:07.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.940+0000 7f4aedffb700 1 -- 172.21.15.101:0/2537155759 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4adc01d070 con 0x7f4af0100490 2023-07-18T03:15:07.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.940+0000 7f4aedffb700 1 -- 172.21.15.101:0/2537155759 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4adc021410 con 0x7f4af0100490 2023-07-18T03:15:07.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.940+0000 7f4af6bff700 1 -- 172.21.15.101:0/2537155759 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4af01a4680 con 0x7f4af0100490 2023-07-18T03:15:07.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.940+0000 7f4aedffb700 1 -- 172.21.15.101:0/2537155759 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4adc017650 con 0x7f4af0100490 2023-07-18T03:15:07.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.940+0000 7f4af6bff700 1 -- 172.21.15.101:0/2537155759 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4af01a4aa0 con 0x7f4af0100490 2023-07-18T03:15:07.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.943+0000 7f4aedffb700 1 -- 172.21.15.101:0/2537155759 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f4adc021810 con 0x7f4af0100490 2023-07-18T03:15:07.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.944+0000 7f4b0a524b80 1 Processor -- start 2023-07-18T03:15:07.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.944+0000 7f4b0a524b80 1 -- start start 2023-07-18T03:15:07.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.944+0000 7f4b0a524b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dcf82b22d0 0x55dcf82b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:07.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.945+0000 7f4b0a524b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcf82b2be0 0x55dcf82bd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:07.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.945+0000 7f4b0a524b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf82bdad0 0x55dcf82bfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:07.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.945+0000 7f4aedffb700 1 --2- 172.21.15.101:0/2537155759 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4ae00a12c0 0x7f4ae00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:07.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.945+0000 7f4b0a524b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf81b8cb0 con 0x55dcf82b22d0 2023-07-18T03:15:07.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.945+0000 7f4b0a524b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf820ba40 con 0x55dcf82bdad0 2023-07-18T03:15:07.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.945+0000 7f4b0a524b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf8204ed0 con 0x55dcf82b2be0 2023-07-18T03:15:07.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.945+0000 7f4aedffb700 1 -- 172.21.15.101:0/2537155759 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f4adc0ce420 con 0x7f4af0100490 2023-07-18T03:15:07.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.945+0000 7f4af499b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcf82b2be0 0x55dcf82bd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:07.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.945+0000 7f4aeffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf82bdad0 0x55dcf82bfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:07.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.945+0000 7f4aeffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf82bdad0 0x55dcf82bfeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:37180/0 (socket says 172.21.15.101:37180) 2023-07-18T03:15:07.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.945+0000 7f4af499b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcf82b2be0 0x55dcf82bd590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:46896/0 (socket says 172.21.15.101:46896) 2023-07-18T03:15:07.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.945+0000 7f4aeffff700 1 -- 172.21.15.101:0/3450103638 learned_addr learned my addr 172.21.15.101:0/3450103638 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:07.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.945+0000 7f4af519c700 1 --2- 172.21.15.101:0/3450103638 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dcf82b22d0 0x55dcf82b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:07.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.945+0000 7f4af519c700 1 --2- 172.21.15.101:0/2537155759 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4ae00a12c0 0x7f4ae00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:07.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.945+0000 7f4aeffff700 1 -- 172.21.15.101:0/3450103638 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcf82b2be0 msgr2=0x55dcf82bd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:07.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.945+0000 7f4aeffff700 1 --2- 172.21.15.101:0/3450103638 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcf82b2be0 0x55dcf82bd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.945+0000 7f4aeffff700 1 -- 172.21.15.101:0/3450103638 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dcf82b22d0 msgr2=0x55dcf82b26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:07.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.945+0000 7f4aeffff700 1 --2- 172.21.15.101:0/3450103638 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dcf82b22d0 0x55dcf82b26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.945+0000 7f4aeffff700 1 -- 172.21.15.101:0/3450103638 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dcf8032de0 con 0x55dcf82bdad0 2023-07-18T03:15:07.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.945+0000 7f4af519c700 1 --2- 172.21.15.101:0/3450103638 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dcf82b22d0 0x55dcf82b26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:15:07.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.945+0000 7f4af499b700 1 --2- 172.21.15.101:0/3450103638 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcf82b2be0 0x55dcf82bd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:15:07.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.946+0000 7f4aeffff700 1 --2- 172.21.15.101:0/3450103638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf82bdad0 0x55dcf82bfeb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4adc0098e0 tx=0x7f4adc097590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb9b3b208a247cb4 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:07.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.946+0000 7f4af519c700 1 --2- 172.21.15.101:0/2537155759 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4ae00a12c0 0x7f4ae00a3770 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8009fd0 tx=0x7f4ae800b040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:07.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.946+0000 7f4adaffd700 1 -- 172.21.15.101:0/3450103638 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4adc09b120 con 0x55dcf82bdad0 2023-07-18T03:15:07.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.946+0000 7f4adaffd700 1 -- 172.21.15.101:0/3450103638 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4adc09f150 con 0x55dcf82bdad0 2023-07-18T03:15:07.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.946+0000 7f4adaffd700 1 -- 172.21.15.101:0/3450103638 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4adc0cb990 con 0x55dcf82bdad0 2023-07-18T03:15:07.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.946+0000 7f4b0a524b80 1 -- 172.21.15.101:0/3450103638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf82bdad0 msgr2=0x55dcf82bfeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:07.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.946+0000 7f4b0a524b80 1 --2- 172.21.15.101:0/3450103638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf82bdad0 0x55dcf82bfeb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4adc0098e0 tx=0x7f4adc097590 comp rx=0 tx=0).stop 2023-07-18T03:15:07.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.947+0000 7f4b0a524b80 1 -- 172.21.15.101:0/3450103638 shutdown_connections 2023-07-18T03:15:07.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.947+0000 7f4b0a524b80 1 --2- 172.21.15.101:0/3450103638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf82bdad0 0x55dcf82bfeb0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.947+0000 7f4b0a524b80 1 --2- 172.21.15.101:0/3450103638 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dcf82b22d0 0x55dcf82b26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.947+0000 7f4b0a524b80 1 --2- 172.21.15.101:0/3450103638 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcf82b2be0 0x55dcf82bd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.947+0000 7f4b0a524b80 1 -- 172.21.15.101:0/3450103638 >> 172.21.15.101:0/3450103638 conn(0x55dcf82a8da0 msgr2=0x55dcf82ae6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:07.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.947+0000 7f4b0a524b80 1 -- 172.21.15.101:0/3450103638 shutdown_connections 2023-07-18T03:15:07.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.947+0000 7f4b0a524b80 1 -- 172.21.15.101:0/3450103638 wait complete. 2023-07-18T03:15:07.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.947+0000 7f4b0a524b80 1 -- start start 2023-07-18T03:15:07.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.947+0000 7f4b0a524b80 1 Processor -- start 2023-07-18T03:15:07.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.948+0000 7f4b0a524b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcf82b22d0 0x55dcf82bdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:07.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.948+0000 7f4b0a524b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dcf82b2be0 0x55dcf82be3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:07.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.948+0000 7f4b0a524b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf82bdad0 0x55dcf82be920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:07.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.948+0000 7f4b0a524b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf81a4ac0 con 0x55dcf82b2be0 2023-07-18T03:15:07.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.948+0000 7f4b0a524b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf81a47c0 con 0x55dcf82bdad0 2023-07-18T03:15:07.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.948+0000 7f4b0a524b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf8088920 con 0x55dcf82b22d0 2023-07-18T03:15:07.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.948+0000 7f4b0a524b80 10 client.0 _ll_register_callbacks cb 0x55dcf81a2760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:15:07.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.948+0000 7f4af499b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcf82b22d0 0x55dcf82bdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:07.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.948+0000 7f4aeffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dcf82b2be0 0x55dcf82be3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:07.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.948+0000 7f4af499b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcf82b22d0 0x55dcf82bdea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:46902/0 (socket says 172.21.15.101:46902) 2023-07-18T03:15:07.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.948+0000 7f4af499b700 1 -- 172.21.15.101:0/2093420740 learned_addr learned my addr 172.21.15.101:0/2093420740 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:07.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.948+0000 7f4af519c700 1 --2- 172.21.15.101:0/2093420740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf82bdad0 0x55dcf82be920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:07.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.948+0000 7f4af499b700 1 -- 172.21.15.101:0/2093420740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf82bdad0 msgr2=0x55dcf82be920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:07.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.948+0000 7f4af499b700 1 --2- 172.21.15.101:0/2093420740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf82bdad0 0x55dcf82be920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.948+0000 7f4af499b700 1 -- 172.21.15.101:0/2093420740 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dcf82b2be0 msgr2=0x55dcf82be3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:07.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.948+0000 7f4af499b700 1 --2- 172.21.15.101:0/2093420740 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dcf82b2be0 0x55dcf82be3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.948+0000 7f4af499b700 1 -- 172.21.15.101:0/2093420740 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ae40098e0 con 0x55dcf82b22d0 2023-07-18T03:15:07.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.949+0000 7f4af499b700 1 --2- 172.21.15.101:0/2093420740 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcf82b22d0 0x55dcf82bdea0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4ae40119d0 tx=0x7f4ae4011ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1352c1c4ef3c6a4 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:07.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.949+0000 7f4acffff700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:15:07.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.949+0000 7f4acffff700 1 -- 172.21.15.101:0/2093420740 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae4016070 con 0x55dcf82b22d0 2023-07-18T03:15:07.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.949+0000 7f4b0a524b80 1 -- 172.21.15.101:0/2093420740 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dcf815e4f0 con 0x55dcf82b22d0 2023-07-18T03:15:07.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.949+0000 7f4acffff700 1 -- 172.21.15.101:0/2093420740 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f4ae400fa40 con 0x55dcf82b22d0 2023-07-18T03:15:07.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.949+0000 7f4acffff700 1 -- 172.21.15.101:0/2093420740 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ae400fc00 con 0x55dcf82b22d0 2023-07-18T03:15:07.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.949+0000 7f4b0a524b80 10 client.27118 fetch_fsmap learned FSMap version 29 2023-07-18T03:15:07.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.949+0000 7f4b0a524b80 1 -- 172.21.15.101:0/2093420740 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f4af01a4aa0 con 0x55dcf82b22d0 2023-07-18T03:15:07.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.950+0000 7f4acffff700 1 -- 172.21.15.101:0/2093420740 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dcf815e4f0 con 0x55dcf82b22d0 2023-07-18T03:15:07.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.950+0000 7f4acffff700 1 -- 172.21.15.101:0/2093420740 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f4ae4019390 con 0x55dcf82b22d0 2023-07-18T03:15:07.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.950+0000 7f4b0a524b80 10 client.27118 fetch_fsmap finished waiting for FSMap version 29 2023-07-18T03:15:07.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.950+0000 7f4b0a524b80 10 client.27118 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:15:07.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.950+0000 7f4b0a524b80 20 client.27118 populate_metadata read hostname 'smithi101' 2023-07-18T03:15:07.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.950+0000 7f4b0a524b80 1 --2- 172.21.15.101:0/2093420740 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55dcf82cd190 0x55dcf82cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:07.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.950+0000 7f4b0a524b80 4 client.27118 mds_command: new command op to 24541 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:15:07.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.950+0000 7f4b0a524b80 1 -- 172.21.15.101:0/2093420740 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dcf81fdeb0 con 0x55dcf82cd190 2023-07-18T03:15:07.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.950+0000 7f4aeffff700 1 --2- 172.21.15.101:0/2093420740 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55dcf82cd190 0x55dcf82cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:07.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.950+0000 7f4aeffff700 1 --2- 172.21.15.101:0/2093420740 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55dcf82cd190 0x55dcf82cf5a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1f249bf8bc6939 server_cookie=a9d7e3e31207c406 in_seq=0 out_seq=0 2023-07-18T03:15:07.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.950+0000 7f4acffff700 10 client.27118 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:15:07.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.951+0000 7f4acffff700 1 -- 172.21.15.101:0/2093420740 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55dcf81fdeb0 con 0x55dcf82cd190 2023-07-18T03:15:07.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.951+0000 7f4acffff700 10 client.27118 handle_command_reply: tid=0 2023-07-18T03:15:07.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.951+0000 7f4b0a524b80 1 client.27118 shutdown 2023-07-18T03:15:07.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.951+0000 7f4b0a524b80 1 -- 172.21.15.101:0/2093420740 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcf82b22d0 msgr2=0x55dcf82bdea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:07.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.951+0000 7f4b0a524b80 1 --2- 172.21.15.101:0/2093420740 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcf82b22d0 0x55dcf82bdea0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4ae40119d0 tx=0x7f4ae4011ce0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.951+0000 7f4b0a524b80 1 -- 172.21.15.101:0/2093420740 shutdown_connections 2023-07-18T03:15:07.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.951+0000 7f4b0a524b80 1 --2- 172.21.15.101:0/2093420740 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55dcf82cd190 0x55dcf82cf5a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.951+0000 7f4b0a524b80 1 --2- 172.21.15.101:0/2093420740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf82bdad0 0x55dcf82be920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.951+0000 7f4acffff700 0 client.27118 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:15:07.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.951+0000 7f4b0a524b80 1 --2- 172.21.15.101:0/2093420740 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcf82b22d0 0x55dcf82bdea0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.951+0000 7f4b0a524b80 1 --2- 172.21.15.101:0/2093420740 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dcf82b2be0 0x55dcf82be3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.951+0000 7f4b0a524b80 1 -- 172.21.15.101:0/2093420740 >> 172.21.15.101:0/2093420740 conn(0x55dcf82a8da0 msgr2=0x55dcf82ae230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:07.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.952+0000 7f4b0a524b80 1 -- 172.21.15.101:0/2093420740 shutdown_connections 2023-07-18T03:15:07.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.952+0000 7f4b0a524b80 1 -- 172.21.15.101:0/2093420740 wait complete. 2023-07-18T03:15:07.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.952+0000 7f4b0a524b80 20 client.27118 trim_cache size 0 max 16384 2023-07-18T03:15:07.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.965+0000 7f4b0a524b80 1 Processor -- start 2023-07-18T03:15:07.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.965+0000 7f4b0a524b80 1 -- start start 2023-07-18T03:15:07.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.965+0000 7f4b0a524b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf82b22d0 0x55dcf82b74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:07.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.965+0000 7f4b0a524b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dcf82b2be0 0x55dcf82c1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:07.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.965+0000 7f4b0a524b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcf82bdad0 0x55dcf82bc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:07.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.965+0000 7f4b0a524b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf82b3580 con 0x55dcf82b2be0 2023-07-18T03:15:07.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.965+0000 7f4b0a524b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf82a8970 con 0x55dcf82b22d0 2023-07-18T03:15:07.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.965+0000 7f4b0a524b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf82bd0f0 con 0x55dcf82bdad0 2023-07-18T03:15:07.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.965+0000 7f4af499b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf82b22d0 0x55dcf82b74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:07.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.965+0000 7f4af519c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcf82bdad0 0x55dcf82bc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:07.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.965+0000 7f4af499b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf82b22d0 0x55dcf82b74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:37202/0 (socket says 172.21.15.101:37202) 2023-07-18T03:15:07.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.965+0000 7f4af499b700 1 -- 172.21.15.101:0/3110045149 learned_addr learned my addr 172.21.15.101:0/3110045149 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:07.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.965+0000 7f4aeffff700 1 --2- 172.21.15.101:0/3110045149 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dcf82b2be0 0x55dcf82c1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:07.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.966+0000 7f4af519c700 1 -- 172.21.15.101:0/3110045149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf82b22d0 msgr2=0x55dcf82b74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:07.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.966+0000 7f4af519c700 1 --2- 172.21.15.101:0/3110045149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf82b22d0 0x55dcf82b74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.966+0000 7f4af519c700 1 -- 172.21.15.101:0/3110045149 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dcf82b2be0 msgr2=0x55dcf82c1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:07.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.966+0000 7f4af519c700 1 --2- 172.21.15.101:0/3110045149 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dcf82b2be0 0x55dcf82c1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.966+0000 7f4af519c700 1 -- 172.21.15.101:0/3110045149 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dcf801bc60 con 0x55dcf82bdad0 2023-07-18T03:15:07.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.966+0000 7f4af499b700 1 --2- 172.21.15.101:0/3110045149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf82b22d0 0x55dcf82b74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:15:07.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.966+0000 7f4af519c700 1 --2- 172.21.15.101:0/3110045149 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcf82bdad0 0x55dcf82bc350 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8009690 tx=0x7f4ae8009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9056c3fd1f617dbb server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:07.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.966+0000 7f4adb7fe700 1 -- 172.21.15.101:0/3110045149 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae8028070 con 0x55dcf82bdad0 2023-07-18T03:15:07.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.966+0000 7f4adb7fe700 1 -- 172.21.15.101:0/3110045149 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ae800f040 con 0x55dcf82bdad0 2023-07-18T03:15:07.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.966+0000 7f4adb7fe700 1 -- 172.21.15.101:0/3110045149 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae8024980 con 0x55dcf82bdad0 2023-07-18T03:15:07.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.967+0000 7f4b0a524b80 1 -- 172.21.15.101:0/3110045149 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcf82bdad0 msgr2=0x55dcf82bc350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:07.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.967+0000 7f4b0a524b80 1 --2- 172.21.15.101:0/3110045149 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcf82bdad0 0x55dcf82bc350 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8009690 tx=0x7f4ae8009920 comp rx=0 tx=0).stop 2023-07-18T03:15:07.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.967+0000 7f4b0a524b80 1 -- 172.21.15.101:0/3110045149 shutdown_connections 2023-07-18T03:15:07.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.967+0000 7f4b0a524b80 1 --2- 172.21.15.101:0/3110045149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf82b22d0 0x55dcf82b74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.967+0000 7f4b0a524b80 1 --2- 172.21.15.101:0/3110045149 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcf82bdad0 0x55dcf82bc350 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.967+0000 7f4b0a524b80 1 --2- 172.21.15.101:0/3110045149 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dcf82b2be0 0x55dcf82c1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.967+0000 7f4b0a524b80 1 -- 172.21.15.101:0/3110045149 >> 172.21.15.101:0/3110045149 conn(0x55dcf82a8da0 msgr2=0x55dcf82b7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:07.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.967+0000 7f4b0a524b80 1 -- 172.21.15.101:0/3110045149 shutdown_connections 2023-07-18T03:15:07.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.967+0000 7f4b0a524b80 1 -- 172.21.15.101:0/3110045149 wait complete. 2023-07-18T03:15:07.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.967+0000 7f4b0a524b80 1 -- start start 2023-07-18T03:15:07.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.967+0000 7f4b0a524b80 1 Processor -- start 2023-07-18T03:15:07.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.968+0000 7f4b0a524b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf82b22d0 0x55dcf82c1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:07.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.968+0000 7f4b0a524b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dcf82b2be0 0x55dcf82c0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:07.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.968+0000 7f4b0a524b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcf82bdad0 0x55dcf82c0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:07.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.968+0000 7f4b0a524b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf82c0d00 con 0x55dcf82b2be0 2023-07-18T03:15:07.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.968+0000 7f4b0a524b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf82c0e70 con 0x55dcf82b22d0 2023-07-18T03:15:07.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.968+0000 7f4b0a524b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf8321060 con 0x55dcf82bdad0 2023-07-18T03:15:07.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.968+0000 7f4b0a524b80 10 client.0 _ll_register_callbacks cb 0x55dcf81a2760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:15:07.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.968+0000 7f4aeffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dcf82b2be0 0x55dcf82c0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:07.970 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.968+0000 7f4af499b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf82b22d0 0x55dcf82c1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:07.970 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.968+0000 7f4aeffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dcf82b2be0 0x55dcf82c0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:39582/0 (socket says 172.21.15.101:39582) 2023-07-18T03:15:07.970 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.968+0000 7f4aeffff700 1 -- 172.21.15.101:0/744528660 learned_addr learned my addr 172.21.15.101:0/744528660 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:07.970 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.968+0000 7f4af519c700 1 --2- 172.21.15.101:0/744528660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcf82bdad0 0x55dcf82c0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:07.970 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.968+0000 7f4aeffff700 1 -- 172.21.15.101:0/744528660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcf82bdad0 msgr2=0x55dcf82c0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:07.970 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.968+0000 7f4aeffff700 1 --2- 172.21.15.101:0/744528660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcf82bdad0 0x55dcf82c0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.970 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.969+0000 7f4aeffff700 1 -- 172.21.15.101:0/744528660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf82b22d0 msgr2=0x55dcf82c1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:07.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.969+0000 7f4aeffff700 1 --2- 172.21.15.101:0/744528660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf82b22d0 0x55dcf82c1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.969+0000 7f4aeffff700 1 -- 172.21.15.101:0/744528660 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4af01a4aa0 con 0x55dcf82b2be0 2023-07-18T03:15:07.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.969+0000 7f4af499b700 1 --2- 172.21.15.101:0/744528660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf82b22d0 0x55dcf82c1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:15:07.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.969+0000 7f4aeffff700 1 --2- 172.21.15.101:0/744528660 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dcf82b2be0 0x55dcf82c0020 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f4adc097b90 tx=0x7f4adc02efe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=930438cda84b730 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:07.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.969+0000 7f4acffff700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:15:07.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.969+0000 7f4acffff700 1 -- 172.21.15.101:0/744528660 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4adc09b120 con 0x55dcf82b2be0 2023-07-18T03:15:07.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.969+0000 7f4acffff700 1 -- 172.21.15.101:0/744528660 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f4adc0cb990 con 0x55dcf82b2be0 2023-07-18T03:15:07.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.969+0000 7f4b0a524b80 1 -- 172.21.15.101:0/744528660 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4adc09f150 con 0x55dcf82b2be0 2023-07-18T03:15:07.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.969+0000 7f4acffff700 1 -- 172.21.15.101:0/744528660 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4adc0cbb50 con 0x55dcf82b2be0 2023-07-18T03:15:07.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.969+0000 7f4b0a524b80 10 client.17046 fetch_fsmap learned FSMap version 29 2023-07-18T03:15:07.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.969+0000 7f4b0a524b80 1 -- 172.21.15.101:0/744528660 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f4ae8007630 con 0x55dcf82b2be0 2023-07-18T03:15:07.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.969+0000 7f4acffff700 1 -- 172.21.15.101:0/744528660 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4adc0c9080 con 0x55dcf82b2be0 2023-07-18T03:15:07.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.969+0000 7f4acffff700 1 -- 172.21.15.101:0/744528660 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f4adc09a610 con 0x55dcf82b2be0 2023-07-18T03:15:07.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.970+0000 7f4b0a524b80 10 client.17046 fetch_fsmap finished waiting for FSMap version 29 2023-07-18T03:15:07.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.970+0000 7f4b0a524b80 10 client.17046 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:15:07.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.970+0000 7f4b0a524b80 20 client.17046 populate_metadata read hostname 'smithi101' 2023-07-18T03:15:07.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.970+0000 7f4b0a524b80 1 --2- 172.21.15.101:0/744528660 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55dcf8322400 0x55dcf8324810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:07.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.970+0000 7f4b0a524b80 4 client.17046 mds_command: new command op to 24541 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:15:07.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.970+0000 7f4b0a524b80 1 -- 172.21.15.101:0/744528660 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dcf81c0050 con 0x55dcf8322400 2023-07-18T03:15:07.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.970+0000 7f4af499b700 1 --2- 172.21.15.101:0/744528660 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55dcf8322400 0x55dcf8324810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:07.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.970+0000 7f4acffff700 10 client.17046 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:15:07.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.970+0000 7f4af499b700 1 --2- 172.21.15.101:0/744528660 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55dcf8322400 0x55dcf8324810 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=72580333971604eb server_cookie=4f9605bc464e7639 in_seq=0 out_seq=0 2023-07-18T03:15:07.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.970+0000 7f4acffff700 1 -- 172.21.15.101:0/744528660 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dcf81c0050 con 0x55dcf8322400 2023-07-18T03:15:07.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.970+0000 7f4acffff700 10 client.17046 handle_command_reply: tid=0 2023-07-18T03:15:07.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.970+0000 7f4b0a524b80 1 client.17046 shutdown 2023-07-18T03:15:07.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.971+0000 7f4b0a524b80 1 -- 172.21.15.101:0/744528660 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dcf82b2be0 msgr2=0x55dcf82c0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:07.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.971+0000 7f4b0a524b80 1 --2- 172.21.15.101:0/744528660 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dcf82b2be0 0x55dcf82c0020 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f4adc097b90 tx=0x7f4adc02efe0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.971+0000 7f4b0a524b80 1 -- 172.21.15.101:0/744528660 shutdown_connections 2023-07-18T03:15:07.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.971+0000 7f4b0a524b80 1 --2- 172.21.15.101:0/744528660 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55dcf8322400 0x55dcf8324810 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.971+0000 7f4b0a524b80 1 --2- 172.21.15.101:0/744528660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcf82b22d0 0x55dcf82c1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.971+0000 7f4b0a524b80 1 --2- 172.21.15.101:0/744528660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcf82bdad0 0x55dcf82c0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.971+0000 7f4b0a524b80 1 --2- 172.21.15.101:0/744528660 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dcf82b2be0 0x55dcf82c0020 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.971+0000 7f4acffff700 0 client.17046 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:15:07.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.971+0000 7f4b0a524b80 1 -- 172.21.15.101:0/744528660 >> 172.21.15.101:0/744528660 conn(0x55dcf82a8da0 msgr2=0x55dcf82bc350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:07.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.971+0000 7f4b0a524b80 1 -- 172.21.15.101:0/744528660 shutdown_connections 2023-07-18T03:15:07.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.971+0000 7f4b0a524b80 1 -- 172.21.15.101:0/744528660 wait complete. 2023-07-18T03:15:07.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.971+0000 7f4b0a524b80 20 client.17046 trim_cache size 0 max 16384 2023-07-18T03:15:07.976 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:15:07.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.972+0000 7f4af6bff700 1 -- 172.21.15.101:0/2537155759 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4ae00a12c0 msgr2=0x7f4ae00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:07.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.972+0000 7f4af6bff700 1 --2- 172.21.15.101:0/2537155759 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4ae00a12c0 0x7f4ae00a3770 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8009fd0 tx=0x7f4ae800b040 comp rx=0 tx=0).stop 2023-07-18T03:15:07.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.972+0000 7f4af6bff700 1 -- 172.21.15.101:0/2537155759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4af0100490 msgr2=0x7f4af0072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:07.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.972+0000 7f4af6bff700 1 --2- 172.21.15.101:0/2537155759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4af0100490 0x7f4af0072d50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4adc009990 tx=0x7f4adc009580 comp rx=0 tx=0).stop 2023-07-18T03:15:07.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.972+0000 7f4af6bff700 1 -- 172.21.15.101:0/2537155759 shutdown_connections 2023-07-18T03:15:07.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.972+0000 7f4af6bff700 1 --2- 172.21.15.101:0/2537155759 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4af0105af0 0x7f4af01a3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.972+0000 7f4af6bff700 1 --2- 172.21.15.101:0/2537155759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4af00ffae0 0x7f4af0072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.972+0000 7f4af6bff700 1 --2- 172.21.15.101:0/2537155759 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4ae00a12c0 0x7f4ae00a3770 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.972+0000 7f4af6bff700 1 --2- 172.21.15.101:0/2537155759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4af0100490 0x7f4af0072d50 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:07.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.972+0000 7f4af6bff700 1 -- 172.21.15.101:0/2537155759 >> 172.21.15.101:0/2537155759 conn(0x7f4af00fb5e0 msgr2=0x7f4af00fbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:07.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.972+0000 7f4af6bff700 1 -- 172.21.15.101:0/2537155759 shutdown_connections 2023-07-18T03:15:07.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:07.972+0000 7f4af6bff700 1 -- 172.21.15.101:0/2537155759 wait complete. 2023-07-18T03:15:07.983 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-18T03:15:08.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.191+0000 7f8663fff700 1 -- 172.21.15.101:0/2467176302 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f865c0ffae0 msgr2=0x7f865c0fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:08.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.191+0000 7f8663fff700 1 --2- 172.21.15.101:0/2467176302 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f865c0ffae0 0x7f865c0fff50 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f864c009990 tx=0x7f864c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.191+0000 7f8663fff700 1 -- 172.21.15.101:0/2467176302 shutdown_connections 2023-07-18T03:15:08.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.191+0000 7f8663fff700 1 --2- 172.21.15.101:0/2467176302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f865c100490 0x7f865c10c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.191+0000 7f8663fff700 1 --2- 172.21.15.101:0/2467176302 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f865c105af0 0x7f865c105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.191+0000 7f8663fff700 1 --2- 172.21.15.101:0/2467176302 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f865c0ffae0 0x7f865c0fff50 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.191+0000 7f8663fff700 1 -- 172.21.15.101:0/2467176302 >> 172.21.15.101:0/2467176302 conn(0x7f865c0fb5e0 msgr2=0x7f865c0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:08.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.191+0000 7f8663fff700 1 -- 172.21.15.101:0/2467176302 shutdown_connections 2023-07-18T03:15:08.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.191+0000 7f8663fff700 1 -- 172.21.15.101:0/2467176302 wait complete. 2023-07-18T03:15:08.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.192+0000 7f8663fff700 1 Processor -- start 2023-07-18T03:15:08.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.192+0000 7f8663fff700 1 -- start start 2023-07-18T03:15:08.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.192+0000 7f8663fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f865c0ffae0 0x7f865c197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:08.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.193+0000 7f8663fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f865c100490 0x7f865c198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:08.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.193+0000 7f8663fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f865c105af0 0x7f865c19c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:08.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.193+0000 7f8663fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865c19cde0 con 0x7f865c0ffae0 2023-07-18T03:15:08.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.193+0000 7f8663fff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865c19cf50 con 0x7f865c100490 2023-07-18T03:15:08.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.193+0000 7f8663fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865c19d110 con 0x7f865c105af0 2023-07-18T03:15:08.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.193+0000 7f8661d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f865c0ffae0 0x7f865c197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:08.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.193+0000 7f8661d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f865c0ffae0 0x7f865c197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:39598/0 (socket says 172.21.15.101:39598) 2023-07-18T03:15:08.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.193+0000 7f8661d9b700 1 -- 172.21.15.101:0/538118105 learned_addr learned my addr 172.21.15.101:0/538118105 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:08.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.193+0000 7f866159a700 1 --2- 172.21.15.101:0/538118105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f865c100490 0x7f865c198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:08.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.193+0000 7f866259c700 1 --2- 172.21.15.101:0/538118105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f865c105af0 0x7f865c19c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:08.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.193+0000 7f8661d9b700 1 -- 172.21.15.101:0/538118105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f865c105af0 msgr2=0x7f865c19c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:08.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.193+0000 7f8661d9b700 1 --2- 172.21.15.101:0/538118105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f865c105af0 0x7f865c19c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.193+0000 7f8661d9b700 1 -- 172.21.15.101:0/538118105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f865c100490 msgr2=0x7f865c198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:08.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.193+0000 7f8661d9b700 1 --2- 172.21.15.101:0/538118105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f865c100490 0x7f865c198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.193+0000 7f8661d9b700 1 -- 172.21.15.101:0/538118105 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f864c0095f0 con 0x7f865c0ffae0 2023-07-18T03:15:08.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.193+0000 7f866159a700 1 --2- 172.21.15.101:0/538118105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f865c100490 0x7f865c198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:15:08.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.193+0000 7f8661d9b700 1 --2- 172.21.15.101:0/538118105 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f865c0ffae0 0x7f865c197e00 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f865800d980 tx=0x7f865800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:08.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.194+0000 7f8652ffd700 1 -- 172.21.15.101:0/538118105 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865800bb60 con 0x7f865c0ffae0 2023-07-18T03:15:08.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.194+0000 7f8652ffd700 1 -- 172.21.15.101:0/538118105 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8658007640 con 0x7f865c0ffae0 2023-07-18T03:15:08.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.194+0000 7f8663fff700 1 -- 172.21.15.101:0/538118105 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f865c19d3f0 con 0x7f865c0ffae0 2023-07-18T03:15:08.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.194+0000 7f8652ffd700 1 -- 172.21.15.101:0/538118105 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8658004e90 con 0x7f865c0ffae0 2023-07-18T03:15:08.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.194+0000 7f8663fff700 1 -- 172.21.15.101:0/538118105 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f865c1a4c30 con 0x7f865c0ffae0 2023-07-18T03:15:08.192 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.198+0000 7f8652ffd700 1 -- 172.21.15.101:0/538118105 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f865800bcc0 con 0x7f865c0ffae0 2023-07-18T03:15:08.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.199+0000 7f86778a3b80 1 Processor -- start 2023-07-18T03:15:08.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.200+0000 7f86778a3b80 1 -- start start 2023-07-18T03:15:08.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.200+0000 7f86778a3b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56046f1542d0 0x56046f1546a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:08.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.200+0000 7f86778a3b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56046f154be0 0x56046f15f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:08.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.200+0000 7f86778a3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56046f15fad0 0x56046f161eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:08.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.200+0000 7f86778a3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046f05acb0 con 0x56046f15fad0 2023-07-18T03:15:08.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.200+0000 7f86778a3b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046f0ada40 con 0x56046f154be0 2023-07-18T03:15:08.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.200+0000 7f86778a3b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046f0a6ed0 con 0x56046f1542d0 2023-07-18T03:15:08.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.200+0000 7f8652ffd700 1 --2- 172.21.15.101:0/538118105 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f86480a1340 0x7f86480a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:08.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.200+0000 7f8652ffd700 1 -- 172.21.15.101:0/538118105 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f86580ccf50 con 0x7f865c0ffae0 2023-07-18T03:15:08.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.200+0000 7f8661d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56046f154be0 0x56046f15f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:08.198 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.200+0000 7f8661d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56046f154be0 0x56046f15f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:37226/0 (socket says 172.21.15.101:37226) 2023-07-18T03:15:08.198 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.200+0000 7f8661d9b700 1 -- 172.21.15.101:0/2381610218 learned_addr learned my addr 172.21.15.101:0/2381610218 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:08.198 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.200+0000 7f866259c700 1 --2- 172.21.15.101:0/538118105 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f86480a1340 0x7f86480a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:08.198 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.200+0000 7f866159a700 1 --2- 172.21.15.101:0/2381610218 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56046f15fad0 0x56046f161eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:08.198 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.200+0000 7f866259c700 1 --2- 172.21.15.101:0/2381610218 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56046f1542d0 0x56046f1546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:08.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.201+0000 7f8661d9b700 1 -- 172.21.15.101:0/2381610218 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56046f1542d0 msgr2=0x56046f1546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:08.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.201+0000 7f8661d9b700 1 --2- 172.21.15.101:0/2381610218 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56046f1542d0 0x56046f1546a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.201+0000 7f8661d9b700 1 -- 172.21.15.101:0/2381610218 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56046f15fad0 msgr2=0x56046f161eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:08.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.201+0000 7f8661d9b700 1 --2- 172.21.15.101:0/2381610218 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56046f15fad0 0x56046f161eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.201+0000 7f8661d9b700 1 -- 172.21.15.101:0/2381610218 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56046eed4de0 con 0x56046f154be0 2023-07-18T03:15:08.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.201+0000 7f866159a700 1 --2- 172.21.15.101:0/2381610218 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56046f15fad0 0x56046f161eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:15:08.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.201+0000 7f8661d9b700 1 --2- 172.21.15.101:0/2381610218 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56046f154be0 0x56046f15f590 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f865800d950 tx=0x7f8658099520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=251567432c07cbb4 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:08.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.201+0000 7f866259c700 1 --2- 172.21.15.101:0/538118105 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f86480a1340 0x7f86480a37f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8654009fd0 tx=0x7f8654009920 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:08.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.201+0000 7f8647fff700 1 -- 172.21.15.101:0/2381610218 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8658099bc0 con 0x56046f154be0 2023-07-18T03:15:08.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.201+0000 7f8647fff700 1 -- 172.21.15.101:0/2381610218 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f865809dbf0 con 0x56046f154be0 2023-07-18T03:15:08.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.201+0000 7f8647fff700 1 -- 172.21.15.101:0/2381610218 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8658099310 con 0x56046f154be0 2023-07-18T03:15:08.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.202+0000 7f86778a3b80 1 -- 172.21.15.101:0/2381610218 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56046f154be0 msgr2=0x56046f15f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:08.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.202+0000 7f86778a3b80 1 --2- 172.21.15.101:0/2381610218 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56046f154be0 0x56046f15f590 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f865800d950 tx=0x7f8658099520 comp rx=0 tx=0).stop 2023-07-18T03:15:08.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.202+0000 7f86778a3b80 1 -- 172.21.15.101:0/2381610218 shutdown_connections 2023-07-18T03:15:08.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.202+0000 7f86778a3b80 1 --2- 172.21.15.101:0/2381610218 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56046f15fad0 0x56046f161eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.202+0000 7f86778a3b80 1 --2- 172.21.15.101:0/2381610218 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56046f1542d0 0x56046f1546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.202+0000 7f86778a3b80 1 --2- 172.21.15.101:0/2381610218 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56046f154be0 0x56046f15f590 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.202+0000 7f86778a3b80 1 -- 172.21.15.101:0/2381610218 >> 172.21.15.101:0/2381610218 conn(0x56046f14ada0 msgr2=0x56046f1506a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:08.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.202+0000 7f86778a3b80 1 -- 172.21.15.101:0/2381610218 shutdown_connections 2023-07-18T03:15:08.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.202+0000 7f86778a3b80 1 -- 172.21.15.101:0/2381610218 wait complete. 2023-07-18T03:15:08.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.202+0000 7f86778a3b80 1 -- start start 2023-07-18T03:15:08.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.203+0000 7f86778a3b80 1 Processor -- start 2023-07-18T03:15:08.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.203+0000 7f86778a3b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56046f1542d0 0x56046f15f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:08.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.203+0000 7f86778a3b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56046f154be0 0x56046f15c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:08.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.203+0000 7f86778a3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56046f15fad0 0x56046f15cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:08.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.203+0000 7f86778a3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046f046ac0 con 0x56046f15fad0 2023-07-18T03:15:08.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.203+0000 7f86778a3b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046f0467c0 con 0x56046f1542d0 2023-07-18T03:15:08.205 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.203+0000 7f86778a3b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046ef2a920 con 0x56046f154be0 2023-07-18T03:15:08.205 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.203+0000 7f86778a3b80 10 client.0 _ll_register_callbacks cb 0x56046f044760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:15:08.205 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.203+0000 7f8661d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56046f1542d0 0x56046f15f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:08.206 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.203+0000 7f8661d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56046f1542d0 0x56046f15f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:37234/0 (socket says 172.21.15.101:37234) 2023-07-18T03:15:08.206 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.203+0000 7f8661d9b700 1 -- 172.21.15.101:0/749463911 learned_addr learned my addr 172.21.15.101:0/749463911 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:08.206 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.203+0000 7f866259c700 1 --2- 172.21.15.101:0/749463911 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56046f15fad0 0x56046f15cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:08.206 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.203+0000 7f866159a700 1 --2- 172.21.15.101:0/749463911 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56046f154be0 0x56046f15c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:08.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.204+0000 7f8661d9b700 1 -- 172.21.15.101:0/749463911 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56046f154be0 msgr2=0x56046f15c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:08.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.204+0000 7f8661d9b700 1 --2- 172.21.15.101:0/749463911 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56046f154be0 0x56046f15c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.204+0000 7f8661d9b700 1 -- 172.21.15.101:0/749463911 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56046f15fad0 msgr2=0x56046f15cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:08.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.204+0000 7f8661d9b700 1 --2- 172.21.15.101:0/749463911 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56046f15fad0 0x56046f15cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.204+0000 7f8661d9b700 1 -- 172.21.15.101:0/749463911 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f865c1a4c30 con 0x56046f1542d0 2023-07-18T03:15:08.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.204+0000 7f866159a700 1 --2- 172.21.15.101:0/749463911 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56046f154be0 0x56046f15c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:15:08.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.204+0000 7f866259c700 1 --2- 172.21.15.101:0/749463911 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56046f15fad0 0x56046f15cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:15:08.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.204+0000 7f8661d9b700 1 --2- 172.21.15.101:0/749463911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56046f1542d0 0x56046f15f530 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8658099b00 tx=0x7f86580ca260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=165f40193555ec79 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:08.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.204+0000 7f86457fa700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:15:08.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.204+0000 7f86457fa700 1 -- 172.21.15.101:0/749463911 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8658081070 con 0x56046f1542d0 2023-07-18T03:15:08.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.204+0000 7f86457fa700 1 -- 172.21.15.101:0/749463911 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f86580b2230 con 0x56046f1542d0 2023-07-18T03:15:08.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.204+0000 7f86778a3b80 1 -- 172.21.15.101:0/749463911 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56046f0004f0 con 0x56046f1542d0 2023-07-18T03:15:08.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.204+0000 7f86457fa700 1 -- 172.21.15.101:0/749463911 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86580990b0 con 0x56046f1542d0 2023-07-18T03:15:08.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.204+0000 7f86778a3b80 10 client.27098 fetch_fsmap learned FSMap version 29 2023-07-18T03:15:08.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.204+0000 7f86778a3b80 1 -- 172.21.15.101:0/749463911 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56046eebdc60 con 0x56046f1542d0 2023-07-18T03:15:08.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.205+0000 7f86457fa700 1 -- 172.21.15.101:0/749463911 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86580c8080 con 0x56046f1542d0 2023-07-18T03:15:08.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.205+0000 7f86457fa700 1 -- 172.21.15.101:0/749463911 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f86580a1b80 con 0x56046f1542d0 2023-07-18T03:15:08.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.205+0000 7f86778a3b80 10 client.27098 fetch_fsmap finished waiting for FSMap version 29 2023-07-18T03:15:08.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.205+0000 7f86778a3b80 10 client.27098 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:15:08.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.205+0000 7f86778a3b80 20 client.27098 populate_metadata read hostname 'smithi101' 2023-07-18T03:15:08.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.205+0000 7f86778a3b80 1 --2- 172.21.15.101:0/749463911 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x56046f16eea0 0x56046f1712b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:08.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.205+0000 7f86778a3b80 4 client.27098 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:15:08.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.205+0000 7f86778a3b80 1 -- 172.21.15.101:0/749463911 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56046f09feb0 con 0x56046f16eea0 2023-07-18T03:15:08.218 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.205+0000 7f866159a700 1 --2- 172.21.15.101:0/749463911 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x56046f16eea0 0x56046f1712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:08.218 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.206+0000 7f86457fa700 10 client.27098 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:15:08.218 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.206+0000 7f866159a700 1 --2- 172.21.15.101:0/749463911 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x56046f16eea0 0x56046f1712b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8decf060c5813a84 server_cookie=be0bb2896bc0c860 in_seq=0 out_seq=0 2023-07-18T03:15:08.218 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.206+0000 7f86457fa700 1 -- 172.21.15.101:0/749463911 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56046f09feb0 con 0x56046f16eea0 2023-07-18T03:15:08.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.206+0000 7f86457fa700 10 client.27098 handle_command_reply: tid=0 2023-07-18T03:15:08.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.206+0000 7f86778a3b80 1 client.27098 shutdown 2023-07-18T03:15:08.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.206+0000 7f86778a3b80 1 -- 172.21.15.101:0/749463911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56046f1542d0 msgr2=0x56046f15f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:08.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.206+0000 7f86778a3b80 1 --2- 172.21.15.101:0/749463911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56046f1542d0 0x56046f15f530 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8658099b00 tx=0x7f86580ca260 comp rx=0 tx=0).stop 2023-07-18T03:15:08.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.206+0000 7f86778a3b80 1 -- 172.21.15.101:0/749463911 shutdown_connections 2023-07-18T03:15:08.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.206+0000 7f86778a3b80 1 --2- 172.21.15.101:0/749463911 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x56046f16eea0 0x56046f1712b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.206+0000 7f86778a3b80 1 --2- 172.21.15.101:0/749463911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56046f1542d0 0x56046f15f530 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.206+0000 7f86778a3b80 1 --2- 172.21.15.101:0/749463911 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56046f15fad0 0x56046f15cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.206+0000 7f86457fa700 0 client.27098 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:15:08.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.206+0000 7f86778a3b80 1 --2- 172.21.15.101:0/749463911 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56046f154be0 0x56046f15c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.206+0000 7f86778a3b80 1 -- 172.21.15.101:0/749463911 >> 172.21.15.101:0/749463911 conn(0x56046f14ada0 msgr2=0x56046f150230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:08.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.207+0000 7f86778a3b80 1 -- 172.21.15.101:0/749463911 shutdown_connections 2023-07-18T03:15:08.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.207+0000 7f86778a3b80 1 -- 172.21.15.101:0/749463911 wait complete. 2023-07-18T03:15:08.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.207+0000 7f86778a3b80 20 client.27098 trim_cache size 0 max 16384 2023-07-18T03:15:08.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.220+0000 7f86778a3b80 1 Processor -- start 2023-07-18T03:15:08.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.220+0000 7f86778a3b80 1 -- start start 2023-07-18T03:15:08.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.221+0000 7f86778a3b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56046f1542d0 0x56046f16af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:08.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.221+0000 7f86778a3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56046f154be0 0x56046f1571b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:08.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.221+0000 7f86778a3b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56046f15fad0 0x56046f163ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:08.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.221+0000 7f86778a3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046f147fa0 con 0x56046f154be0 2023-07-18T03:15:08.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.221+0000 7f86778a3b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046f15d730 con 0x56046f15fad0 2023-07-18T03:15:08.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.221+0000 7f86778a3b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046f15dd60 con 0x56046f1542d0 2023-07-18T03:15:08.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.221+0000 7f866159a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56046f154be0 0x56046f1571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:08.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.221+0000 7f866259c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56046f15fad0 0x56046f163ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:08.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.221+0000 7f866159a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56046f154be0 0x56046f1571b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:39626/0 (socket says 172.21.15.101:39626) 2023-07-18T03:15:08.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.221+0000 7f866159a700 1 -- 172.21.15.101:0/2409123434 learned_addr learned my addr 172.21.15.101:0/2409123434 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:08.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.221+0000 7f8661d9b700 1 --2- 172.21.15.101:0/2409123434 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56046f1542d0 0x56046f16af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:08.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.221+0000 7f866159a700 1 -- 172.21.15.101:0/2409123434 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56046f1542d0 msgr2=0x56046f16af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:08.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.221+0000 7f866159a700 1 --2- 172.21.15.101:0/2409123434 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56046f1542d0 0x56046f16af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.221+0000 7f866159a700 1 -- 172.21.15.101:0/2409123434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56046f15fad0 msgr2=0x56046f163ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:08.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.221+0000 7f866159a700 1 --2- 172.21.15.101:0/2409123434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56046f15fad0 0x56046f163ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.221+0000 7f866159a700 1 -- 172.21.15.101:0/2409123434 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56046eefb250 con 0x56046f154be0 2023-07-18T03:15:08.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.221+0000 7f866259c700 1 --2- 172.21.15.101:0/2409123434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56046f15fad0 0x56046f163ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:15:08.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.222+0000 7f866159a700 1 --2- 172.21.15.101:0/2409123434 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56046f154be0 0x56046f1571b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f864c009990 tx=0x7f864c0092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=573c64641ed590fc server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:08.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.222+0000 7f8650ff9700 1 -- 172.21.15.101:0/2409123434 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864c00b970 con 0x56046f154be0 2023-07-18T03:15:08.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.222+0000 7f8650ff9700 1 -- 172.21.15.101:0/2409123434 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f864c01e040 con 0x56046f154be0 2023-07-18T03:15:08.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.222+0000 7f8650ff9700 1 -- 172.21.15.101:0/2409123434 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864c0052a0 con 0x56046f154be0 2023-07-18T03:15:08.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.223+0000 7f86778a3b80 1 -- 172.21.15.101:0/2409123434 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56046f154be0 msgr2=0x56046f1571b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:08.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.223+0000 7f86778a3b80 1 --2- 172.21.15.101:0/2409123434 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56046f154be0 0x56046f1571b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f864c009990 tx=0x7f864c0092f0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.223+0000 7f86778a3b80 1 -- 172.21.15.101:0/2409123434 shutdown_connections 2023-07-18T03:15:08.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.223+0000 7f86778a3b80 1 --2- 172.21.15.101:0/2409123434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56046f15fad0 0x56046f163ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.223+0000 7f86778a3b80 1 --2- 172.21.15.101:0/2409123434 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56046f1542d0 0x56046f16af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.223+0000 7f86778a3b80 1 --2- 172.21.15.101:0/2409123434 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56046f154be0 0x56046f1571b0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.223+0000 7f86778a3b80 1 -- 172.21.15.101:0/2409123434 >> 172.21.15.101:0/2409123434 conn(0x56046f14ada0 msgr2=0x56046f14f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:08.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.223+0000 7f86778a3b80 1 -- 172.21.15.101:0/2409123434 shutdown_connections 2023-07-18T03:15:08.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.223+0000 7f86778a3b80 1 -- 172.21.15.101:0/2409123434 wait complete. 2023-07-18T03:15:08.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.223+0000 7f86778a3b80 1 -- start start 2023-07-18T03:15:08.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.223+0000 7f86778a3b80 1 Processor -- start 2023-07-18T03:15:08.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.224+0000 7f86778a3b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56046f1542d0 0x56046f163e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:08.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.224+0000 7f86778a3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56046f154be0 0x56046f162320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:08.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.224+0000 7f86778a3b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56046f15fad0 0x56046f162860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:08.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.224+0000 7f86778a3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046f162de0 con 0x56046f154be0 2023-07-18T03:15:08.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.224+0000 7f86778a3b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046f162f50 con 0x56046f1542d0 2023-07-18T03:15:08.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.224+0000 7f86778a3b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046f163110 con 0x56046f15fad0 2023-07-18T03:15:08.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.224+0000 7f86778a3b80 10 client.0 _ll_register_callbacks cb 0x56046f044760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:15:08.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.224+0000 7f866159a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56046f154be0 0x56046f162320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:08.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.224+0000 7f8661d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56046f1542d0 0x56046f163e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:08.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.224+0000 7f866159a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56046f154be0 0x56046f162320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:39630/0 (socket says 172.21.15.101:39630) 2023-07-18T03:15:08.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.224+0000 7f866159a700 1 -- 172.21.15.101:0/4120584495 learned_addr learned my addr 172.21.15.101:0/4120584495 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:08.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.224+0000 7f866259c700 1 --2- 172.21.15.101:0/4120584495 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56046f15fad0 0x56046f162860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:08.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.224+0000 7f8661d9b700 1 -- 172.21.15.101:0/4120584495 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56046f15fad0 msgr2=0x56046f162860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:08.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.224+0000 7f8661d9b700 1 --2- 172.21.15.101:0/4120584495 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56046f15fad0 0x56046f162860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.224+0000 7f8661d9b700 1 -- 172.21.15.101:0/4120584495 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56046f154be0 msgr2=0x56046f162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:08.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.224+0000 7f8661d9b700 1 --2- 172.21.15.101:0/4120584495 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56046f154be0 0x56046f162320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.224+0000 7f8661d9b700 1 -- 172.21.15.101:0/4120584495 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86540090d0 con 0x56046f1542d0 2023-07-18T03:15:08.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.224+0000 7f866159a700 1 --2- 172.21.15.101:0/4120584495 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56046f154be0 0x56046f162320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:15:08.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.225+0000 7f8661d9b700 1 --2- 172.21.15.101:0/4120584495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56046f1542d0 0x56046f163e00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f86580a1f10 tx=0x7f86580a1db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb930cee78239ca1 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:08.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.225+0000 7f86457fa700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:15:08.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.225+0000 7f86457fa700 1 -- 172.21.15.101:0/4120584495 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865809a020 con 0x56046f1542d0 2023-07-18T03:15:08.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.225+0000 7f86457fa700 1 -- 172.21.15.101:0/4120584495 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f86580990b0 con 0x56046f1542d0 2023-07-18T03:15:08.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.225+0000 7f86778a3b80 1 -- 172.21.15.101:0/4120584495 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f865809dbf0 con 0x56046f1542d0 2023-07-18T03:15:08.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.225+0000 7f86457fa700 1 -- 172.21.15.101:0/4120584495 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8658098b20 con 0x56046f1542d0 2023-07-18T03:15:08.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.225+0000 7f86778a3b80 10 client.27104 fetch_fsmap learned FSMap version 29 2023-07-18T03:15:08.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.225+0000 7f86778a3b80 1 -- 172.21.15.101:0/4120584495 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f865c10d220 con 0x56046f1542d0 2023-07-18T03:15:08.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.225+0000 7f86457fa700 1 -- 172.21.15.101:0/4120584495 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f865809dbf0 con 0x56046f1542d0 2023-07-18T03:15:08.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.225+0000 7f86457fa700 1 -- 172.21.15.101:0/4120584495 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f8658078a90 con 0x56046f1542d0 2023-07-18T03:15:08.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.225+0000 7f86778a3b80 10 client.27104 fetch_fsmap finished waiting for FSMap version 29 2023-07-18T03:15:08.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.225+0000 7f86778a3b80 10 client.27104 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:15:08.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.225+0000 7f86778a3b80 20 client.27104 populate_metadata read hostname 'smithi101' 2023-07-18T03:15:08.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.225+0000 7f86778a3b80 1 --2- 172.21.15.101:0/4120584495 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x56046f1c3dc0 0x56046f1c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:08.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.226+0000 7f86778a3b80 4 client.27104 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:15:08.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.226+0000 7f86778a3b80 1 -- 172.21.15.101:0/4120584495 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56046f062050 con 0x56046f1c3dc0 2023-07-18T03:15:08.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.226+0000 7f866159a700 1 --2- 172.21.15.101:0/4120584495 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x56046f1c3dc0 0x56046f1c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:08.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.226+0000 7f86457fa700 10 client.27104 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:15:08.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.226+0000 7f866159a700 1 --2- 172.21.15.101:0/4120584495 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x56046f1c3dc0 0x56046f1c61d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f94d0d3afe0ed5e9 server_cookie=36a1673662797497 in_seq=0 out_seq=0 2023-07-18T03:15:08.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.227+0000 7f86457fa700 1 -- 172.21.15.101:0/4120584495 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56046f062050 con 0x56046f1c3dc0 2023-07-18T03:15:08.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.227+0000 7f86457fa700 10 client.27104 handle_command_reply: tid=0 2023-07-18T03:15:08.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.227+0000 7f86778a3b80 1 client.27104 shutdown 2023-07-18T03:15:08.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.227+0000 7f86778a3b80 1 -- 172.21.15.101:0/4120584495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56046f1542d0 msgr2=0x56046f163e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:08.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.227+0000 7f86778a3b80 1 --2- 172.21.15.101:0/4120584495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56046f1542d0 0x56046f163e00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f86580a1f10 tx=0x7f86580a1db0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.227+0000 7f86778a3b80 1 -- 172.21.15.101:0/4120584495 shutdown_connections 2023-07-18T03:15:08.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.227+0000 7f86778a3b80 1 --2- 172.21.15.101:0/4120584495 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x56046f1c3dc0 0x56046f1c61d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.227+0000 7f86778a3b80 1 --2- 172.21.15.101:0/4120584495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56046f1542d0 0x56046f163e00 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.227+0000 7f86778a3b80 1 --2- 172.21.15.101:0/4120584495 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56046f15fad0 0x56046f162860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.227+0000 7f86778a3b80 1 --2- 172.21.15.101:0/4120584495 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56046f154be0 0x56046f162320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.227+0000 7f86457fa700 0 client.27104 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:15:08.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.227+0000 7f86778a3b80 1 -- 172.21.15.101:0/4120584495 >> 172.21.15.101:0/4120584495 conn(0x56046f14ada0 msgr2=0x56046f150140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:08.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.227+0000 7f86778a3b80 1 -- 172.21.15.101:0/4120584495 shutdown_connections 2023-07-18T03:15:08.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.227+0000 7f86778a3b80 1 -- 172.21.15.101:0/4120584495 wait complete. 2023-07-18T03:15:08.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.227+0000 7f86778a3b80 20 client.27104 trim_cache size 0 max 16384 2023-07-18T03:15:08.250 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:15:08.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.228+0000 7f8663fff700 1 -- 172.21.15.101:0/538118105 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f86480a1340 msgr2=0x7f86480a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:08.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.228+0000 7f8663fff700 1 --2- 172.21.15.101:0/538118105 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f86480a1340 0x7f86480a37f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8654009fd0 tx=0x7f8654009920 comp rx=0 tx=0).stop 2023-07-18T03:15:08.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.228+0000 7f8663fff700 1 -- 172.21.15.101:0/538118105 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f865c0ffae0 msgr2=0x7f865c197e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:08.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.228+0000 7f8663fff700 1 --2- 172.21.15.101:0/538118105 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f865c0ffae0 0x7f865c197e00 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f865800d980 tx=0x7f865800dd40 comp rx=0 tx=0).stop 2023-07-18T03:15:08.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.228+0000 7f8663fff700 1 -- 172.21.15.101:0/538118105 shutdown_connections 2023-07-18T03:15:08.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.228+0000 7f8663fff700 1 --2- 172.21.15.101:0/538118105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f865c105af0 0x7f865c19c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.228+0000 7f8663fff700 1 --2- 172.21.15.101:0/538118105 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f865c0ffae0 0x7f865c197e00 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.228+0000 7f8663fff700 1 --2- 172.21.15.101:0/538118105 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f86480a1340 0x7f86480a37f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.228+0000 7f8663fff700 1 --2- 172.21.15.101:0/538118105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f865c100490 0x7f865c198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.228+0000 7f8663fff700 1 -- 172.21.15.101:0/538118105 >> 172.21.15.101:0/538118105 conn(0x7f865c0fb5e0 msgr2=0x7f865c0fbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:08.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.228+0000 7f8663fff700 1 -- 172.21.15.101:0/538118105 shutdown_connections 2023-07-18T03:15:08.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.228+0000 7f8663fff700 1 -- 172.21.15.101:0/538118105 wait complete. 2023-07-18T03:15:08.255 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:08 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/834197619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:15:08.257 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-18T03:15:08.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:08 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/834197619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:15:08.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:08 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/834197619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:15:08.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.463+0000 7f897ebff700 1 -- 172.21.15.101:0/2381932403 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8978105af0 msgr2=0x7f8978105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:08.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.463+0000 7f897ebff700 1 --2- 172.21.15.101:0/2381932403 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8978105af0 0x7f8978105ec0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f896c009990 tx=0x7f896c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.463+0000 7f897ebff700 1 -- 172.21.15.101:0/2381932403 shutdown_connections 2023-07-18T03:15:08.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.463+0000 7f897ebff700 1 --2- 172.21.15.101:0/2381932403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8978100490 0x7f897810c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.463+0000 7f897ebff700 1 --2- 172.21.15.101:0/2381932403 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8978105af0 0x7f8978105ec0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.463+0000 7f897ebff700 1 --2- 172.21.15.101:0/2381932403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89780ffae0 0x7f89780fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.463+0000 7f897ebff700 1 -- 172.21.15.101:0/2381932403 >> 172.21.15.101:0/2381932403 conn(0x7f89780fb5e0 msgr2=0x7f89780fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:08.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.463+0000 7f897ebff700 1 -- 172.21.15.101:0/2381932403 shutdown_connections 2023-07-18T03:15:08.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.463+0000 7f897ebff700 1 -- 172.21.15.101:0/2381932403 wait complete. 2023-07-18T03:15:08.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.465+0000 7f897ebff700 1 Processor -- start 2023-07-18T03:15:08.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.465+0000 7f897ebff700 1 -- start start 2023-07-18T03:15:08.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.465+0000 7f897ebff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89780ffae0 0x7f897819ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:08.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.465+0000 7f897ebff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8978100490 0x7f897819bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:08.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.465+0000 7f897ebff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8978105af0 0x7f8978195aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:08.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.465+0000 7f897ebff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8978196110 con 0x7f8978105af0 2023-07-18T03:15:08.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.465+0000 7f897ebff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8978196280 con 0x7f8978100490 2023-07-18T03:15:08.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.465+0000 7f897ebff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8978196440 con 0x7f89780ffae0 2023-07-18T03:15:08.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.466+0000 7f897c99b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89780ffae0 0x7f897819ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:08.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.466+0000 7f897c99b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89780ffae0 0x7f897819ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:46984/0 (socket says 172.21.15.101:46984) 2023-07-18T03:15:08.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.466+0000 7f897c99b700 1 -- 172.21.15.101:0/3555444560 learned_addr learned my addr 172.21.15.101:0/3555444560 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:08.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.466+0000 7f897d19c700 1 --2- 172.21.15.101:0/3555444560 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8978105af0 0x7f8978195aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:08.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.466+0000 7f8977fff700 1 --2- 172.21.15.101:0/3555444560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8978100490 0x7f897819bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:08.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.466+0000 7f897c99b700 1 -- 172.21.15.101:0/3555444560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8978100490 msgr2=0x7f897819bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:08.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.466+0000 7f897c99b700 1 --2- 172.21.15.101:0/3555444560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8978100490 0x7f897819bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.466+0000 7f897c99b700 1 -- 172.21.15.101:0/3555444560 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8978105af0 msgr2=0x7f8978195aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:08.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.466+0000 7f897c99b700 1 --2- 172.21.15.101:0/3555444560 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8978105af0 0x7f8978195aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.466+0000 7f897c99b700 1 -- 172.21.15.101:0/3555444560 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8964009520 con 0x7f89780ffae0 2023-07-18T03:15:08.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.466+0000 7f8977fff700 1 --2- 172.21.15.101:0/3555444560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8978100490 0x7f897819bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:15:08.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.466+0000 7f897c99b700 1 --2- 172.21.15.101:0/3555444560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89780ffae0 0x7f897819ba30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f896c005910 tx=0x7f896c0057d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:08.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.467+0000 7f8975ffb700 1 -- 172.21.15.101:0/3555444560 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896c01d070 con 0x7f89780ffae0 2023-07-18T03:15:08.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.467+0000 7f897ebff700 1 -- 172.21.15.101:0/3555444560 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8970009c40 con 0x7f89780ffae0 2023-07-18T03:15:08.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.467+0000 7f8975ffb700 1 -- 172.21.15.101:0/3555444560 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f896c022470 con 0x7f89780ffae0 2023-07-18T03:15:08.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.467+0000 7f8975ffb700 1 -- 172.21.15.101:0/3555444560 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896c017650 con 0x7f89780ffae0 2023-07-18T03:15:08.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.467+0000 7f897ebff700 1 -- 172.21.15.101:0/3555444560 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f896c0095f0 con 0x7f89780ffae0 2023-07-18T03:15:08.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.470+0000 7f8975ffb700 1 -- 172.21.15.101:0/3555444560 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f896c0225e0 con 0x7f89780ffae0 2023-07-18T03:15:08.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.471+0000 7f89925d5b80 1 Processor -- start 2023-07-18T03:15:08.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.471+0000 7f89925d5b80 1 -- start start 2023-07-18T03:15:08.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.471+0000 7f8975ffb700 1 --2- 172.21.15.101:0/3555444560 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f89680a12c0 0x7f89680a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:08.471 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.472+0000 7f8975ffb700 1 -- 172.21.15.101:0/3555444560 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f896c0ce500 con 0x7f89780ffae0 2023-07-18T03:15:08.471 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.472+0000 7f89925d5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bab55652d0 0x55bab55656a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:08.471 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.472+0000 7f897d19c700 1 --2- 172.21.15.101:0/3555444560 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f89680a12c0 0x7f89680a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:08.471 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.472+0000 7f89925d5b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bab5565be0 0x55bab5570590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:08.471 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.472+0000 7f89925d5b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bab5570ad0 0x55bab5572eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:08.472 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.472+0000 7f89925d5b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bab546bcb0 con 0x55bab5565be0 2023-07-18T03:15:08.472 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.472+0000 7f89925d5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bab54bea40 con 0x55bab55652d0 2023-07-18T03:15:08.472 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.472+0000 7f89925d5b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bab54b7ed0 con 0x55bab5570ad0 2023-07-18T03:15:08.472 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.472+0000 7f897c99b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bab55652d0 0x55bab55656a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:08.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.472+0000 7f897c99b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bab55652d0 0x55bab55656a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:37278/0 (socket says 172.21.15.101:37278) 2023-07-18T03:15:08.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.472+0000 7f897c99b700 1 -- 172.21.15.101:0/2603718535 learned_addr learned my addr 172.21.15.101:0/2603718535 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:08.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.472+0000 7f8977fff700 1 --2- 172.21.15.101:0/2603718535 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bab5565be0 0x55bab5570590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:08.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.472+0000 7f897c99b700 1 -- 172.21.15.101:0/2603718535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bab5570ad0 msgr2=0x55bab5572eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:08.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.472+0000 7f897c99b700 1 --2- 172.21.15.101:0/2603718535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bab5570ad0 0x55bab5572eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.472+0000 7f897c99b700 1 -- 172.21.15.101:0/2603718535 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bab5565be0 msgr2=0x55bab5570590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:08.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.472+0000 7f897c99b700 1 --2- 172.21.15.101:0/2603718535 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bab5565be0 0x55bab5570590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.472+0000 7f897c99b700 1 -- 172.21.15.101:0/2603718535 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bab52e5de0 con 0x55bab55652d0 2023-07-18T03:15:08.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.473+0000 7f897d19c700 1 --2- 172.21.15.101:0/3555444560 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f89680a12c0 0x7f89680a3770 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f897819c730 tx=0x7f8970009920 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:08.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.473+0000 7f897c99b700 1 --2- 172.21.15.101:0/2603718535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bab55652d0 0x55bab55656a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f896c097c70 tx=0x7f896c009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d480a0f53fa3d43 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:08.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.473+0000 7f8962ffd700 1 -- 172.21.15.101:0/2603718535 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896c0a10b0 con 0x55bab55652d0 2023-07-18T03:15:08.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.473+0000 7f8962ffd700 1 -- 172.21.15.101:0/2603718535 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f896c09b200 con 0x55bab55652d0 2023-07-18T03:15:08.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.473+0000 7f8962ffd700 1 -- 172.21.15.101:0/2603718535 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896c0ce170 con 0x55bab55652d0 2023-07-18T03:15:08.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.474+0000 7f89925d5b80 1 -- 172.21.15.101:0/2603718535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bab55652d0 msgr2=0x55bab55656a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:08.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.474+0000 7f89925d5b80 1 --2- 172.21.15.101:0/2603718535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bab55652d0 0x55bab55656a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f896c097c70 tx=0x7f896c009310 comp rx=0 tx=0).stop 2023-07-18T03:15:08.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.474+0000 7f89925d5b80 1 -- 172.21.15.101:0/2603718535 shutdown_connections 2023-07-18T03:15:08.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.474+0000 7f89925d5b80 1 --2- 172.21.15.101:0/2603718535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bab55652d0 0x55bab55656a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.474+0000 7f89925d5b80 1 --2- 172.21.15.101:0/2603718535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bab5570ad0 0x55bab5572eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.474+0000 7f89925d5b80 1 --2- 172.21.15.101:0/2603718535 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bab5565be0 0x55bab5570590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.474+0000 7f89925d5b80 1 -- 172.21.15.101:0/2603718535 >> 172.21.15.101:0/2603718535 conn(0x55bab555bda0 msgr2=0x55bab55616a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:08.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.474+0000 7f89925d5b80 1 -- 172.21.15.101:0/2603718535 shutdown_connections 2023-07-18T03:15:08.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.474+0000 7f89925d5b80 1 -- 172.21.15.101:0/2603718535 wait complete. 2023-07-18T03:15:08.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.475+0000 7f89925d5b80 1 -- start start 2023-07-18T03:15:08.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.475+0000 7f89925d5b80 1 Processor -- start 2023-07-18T03:15:08.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.476+0000 7f89925d5b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bab55652d0 0x55bab5575b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:08.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.476+0000 7f89925d5b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bab5565be0 0x55bab5576050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:08.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.476+0000 7f89925d5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bab5570ad0 0x55bab556fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:08.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.476+0000 7f89925d5b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bab5457ac0 con 0x55bab5565be0 2023-07-18T03:15:08.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.476+0000 7f89925d5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bab54577c0 con 0x55bab5570ad0 2023-07-18T03:15:08.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.476+0000 7f89925d5b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bab533b920 con 0x55bab55652d0 2023-07-18T03:15:08.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.476+0000 7f89925d5b80 10 client.0 _ll_register_callbacks cb 0x55bab5455760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:15:08.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.476+0000 7f8977fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bab5565be0 0x55bab5576050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:08.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.476+0000 7f8977fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bab5565be0 0x55bab5576050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:39656/0 (socket says 172.21.15.101:39656) 2023-07-18T03:15:08.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.476+0000 7f8977fff700 1 -- 172.21.15.101:0/2217586655 learned_addr learned my addr 172.21.15.101:0/2217586655 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:08.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.476+0000 7f897c99b700 1 --2- 172.21.15.101:0/2217586655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bab55652d0 0x55bab5575b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:08.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.476+0000 7f897d19c700 1 --2- 172.21.15.101:0/2217586655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bab5570ad0 0x55bab556fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:08.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.476+0000 7f8977fff700 1 -- 172.21.15.101:0/2217586655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bab55652d0 msgr2=0x55bab5575b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:08.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.476+0000 7f8977fff700 1 --2- 172.21.15.101:0/2217586655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bab55652d0 0x55bab5575b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.476+0000 7f8977fff700 1 -- 172.21.15.101:0/2217586655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bab5570ad0 msgr2=0x55bab556fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:08.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.476+0000 7f8977fff700 1 --2- 172.21.15.101:0/2217586655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bab5570ad0 0x55bab556fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.476+0000 7f8977fff700 1 -- 172.21.15.101:0/2217586655 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f896c0095f0 con 0x55bab5565be0 2023-07-18T03:15:08.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.477+0000 7f8977fff700 1 --2- 172.21.15.101:0/2217586655 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bab5565be0 0x55bab5576050 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8964012a10 tx=0x7f8964012d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0ebb1185fc3cf50 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:08.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.477+0000 7f8957fff700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:15:08.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.477+0000 7f8957fff700 1 -- 172.21.15.101:0/2217586655 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8964017070 con 0x55bab5565be0 2023-07-18T03:15:08.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.477+0000 7f89925d5b80 1 -- 172.21.15.101:0/2217586655 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bab54114f0 con 0x55bab5565be0 2023-07-18T03:15:08.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.477+0000 7f8957fff700 1 -- 172.21.15.101:0/2217586655 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f8964011a40 con 0x55bab5565be0 2023-07-18T03:15:08.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.477+0000 7f8957fff700 1 -- 172.21.15.101:0/2217586655 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8964011c00 con 0x55bab5565be0 2023-07-18T03:15:08.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.477+0000 7f89925d5b80 10 client.17076 fetch_fsmap learned FSMap version 29 2023-07-18T03:15:08.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.477+0000 7f89925d5b80 1 -- 172.21.15.101:0/2217586655 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55bab52cec60 con 0x55bab5565be0 2023-07-18T03:15:08.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.477+0000 7f8957fff700 1 -- 172.21.15.101:0/2217586655 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8964024020 con 0x55bab5565be0 2023-07-18T03:15:08.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.478+0000 7f8957fff700 1 -- 172.21.15.101:0/2217586655 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f8964003a60 con 0x55bab5565be0 2023-07-18T03:15:08.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.478+0000 7f89925d5b80 10 client.17076 fetch_fsmap finished waiting for FSMap version 29 2023-07-18T03:15:08.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.478+0000 7f89925d5b80 10 client.17076 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:15:08.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.478+0000 7f89925d5b80 20 client.17076 populate_metadata read hostname 'smithi101' 2023-07-18T03:15:08.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.478+0000 7f89925d5b80 1 --2- 172.21.15.101:0/2217586655 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55bab55800e0 0x55bab55824f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:08.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.478+0000 7f89925d5b80 4 client.17076 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:15:08.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.478+0000 7f89925d5b80 1 -- 172.21.15.101:0/2217586655 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bab54b0eb0 con 0x55bab55800e0 2023-07-18T03:15:08.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.478+0000 7f897c99b700 1 --2- 172.21.15.101:0/2217586655 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55bab55800e0 0x55bab55824f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:08.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.479+0000 7f897c99b700 1 --2- 172.21.15.101:0/2217586655 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55bab55800e0 0x55bab55824f0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3b085ab45702b95 server_cookie=29512abe7a56843f in_seq=0 out_seq=0 2023-07-18T03:15:08.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.479+0000 7f8957fff700 10 client.17076 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:15:08.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.479+0000 7f8957fff700 1 -- 172.21.15.101:0/2217586655 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bab54b0eb0 con 0x55bab55800e0 2023-07-18T03:15:08.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.479+0000 7f8957fff700 10 client.17076 handle_command_reply: tid=0 2023-07-18T03:15:08.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.480+0000 7f89925d5b80 1 client.17076 shutdown 2023-07-18T03:15:08.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.480+0000 7f89925d5b80 1 -- 172.21.15.101:0/2217586655 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bab5565be0 msgr2=0x55bab5576050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:08.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.480+0000 7f89925d5b80 1 --2- 172.21.15.101:0/2217586655 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bab5565be0 0x55bab5576050 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8964012a10 tx=0x7f8964012d20 comp rx=0 tx=0).stop 2023-07-18T03:15:08.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.480+0000 7f89925d5b80 1 -- 172.21.15.101:0/2217586655 shutdown_connections 2023-07-18T03:15:08.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.480+0000 7f89925d5b80 1 --2- 172.21.15.101:0/2217586655 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55bab55800e0 0x55bab55824f0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.480+0000 7f89925d5b80 1 --2- 172.21.15.101:0/2217586655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bab5570ad0 0x55bab556fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.480+0000 7f89925d5b80 1 --2- 172.21.15.101:0/2217586655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bab55652d0 0x55bab5575b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.480+0000 7f89925d5b80 1 --2- 172.21.15.101:0/2217586655 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bab5565be0 0x55bab5576050 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.480+0000 7f8957fff700 0 client.17076 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:15:08.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.480+0000 7f89925d5b80 1 -- 172.21.15.101:0/2217586655 >> 172.21.15.101:0/2217586655 conn(0x55bab555bda0 msgr2=0x55bab5561230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:08.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.480+0000 7f89925d5b80 1 -- 172.21.15.101:0/2217586655 shutdown_connections 2023-07-18T03:15:08.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.480+0000 7f89925d5b80 1 -- 172.21.15.101:0/2217586655 wait complete. 2023-07-18T03:15:08.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.480+0000 7f89925d5b80 20 client.17076 trim_cache size 0 max 16384 2023-07-18T03:15:08.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.493+0000 7f89925d5b80 1 Processor -- start 2023-07-18T03:15:08.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.493+0000 7f89925d5b80 1 -- start start 2023-07-18T03:15:08.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.494+0000 7f89925d5b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bab55652d0 0x55bab556fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:08.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.494+0000 7f89925d5b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bab5565be0 0x55bab55756e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:08.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.494+0000 7f89925d5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bab5570ad0 0x55bab5575c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:08.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.494+0000 7f89925d5b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bab5566540 con 0x55bab55652d0 2023-07-18T03:15:08.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.494+0000 7f89925d5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bab555b990 con 0x55bab5570ad0 2023-07-18T03:15:08.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.494+0000 7f89925d5b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bab5566d60 con 0x55bab5565be0 2023-07-18T03:15:08.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.494+0000 7f897c99b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bab55652d0 0x55bab556fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:08.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.494+0000 7f897c99b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bab55652d0 0x55bab556fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:39672/0 (socket says 172.21.15.101:39672) 2023-07-18T03:15:08.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.494+0000 7f897c99b700 1 -- 172.21.15.101:0/887016102 learned_addr learned my addr 172.21.15.101:0/887016102 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:08.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.494+0000 7f897d19c700 1 --2- 172.21.15.101:0/887016102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bab5570ad0 0x55bab5575c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:08.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.494+0000 7f8977fff700 1 --2- 172.21.15.101:0/887016102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bab5565be0 0x55bab55756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:08.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.494+0000 7f897c99b700 1 -- 172.21.15.101:0/887016102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bab5565be0 msgr2=0x55bab55756e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:08.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.494+0000 7f897c99b700 1 --2- 172.21.15.101:0/887016102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bab5565be0 0x55bab55756e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.494+0000 7f897c99b700 1 -- 172.21.15.101:0/887016102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bab5570ad0 msgr2=0x55bab5575c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:08.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.494+0000 7f897c99b700 1 --2- 172.21.15.101:0/887016102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bab5570ad0 0x55bab5575c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.494+0000 7f897c99b700 1 -- 172.21.15.101:0/887016102 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bab530c250 con 0x55bab55652d0 2023-07-18T03:15:08.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.494+0000 7f8977fff700 1 --2- 172.21.15.101:0/887016102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bab5565be0 0x55bab55756e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:15:08.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.495+0000 7f897c99b700 1 --2- 172.21.15.101:0/887016102 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bab55652d0 0x55bab556fb70 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f896c097700 tx=0x7f896c01a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d171497d83a04a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:08.492 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.495+0000 7f89637fe700 1 -- 172.21.15.101:0/887016102 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896c0b2070 con 0x55bab55652d0 2023-07-18T03:15:08.492 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.495+0000 7f89637fe700 1 -- 172.21.15.101:0/887016102 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f896c0a10b0 con 0x55bab55652d0 2023-07-18T03:15:08.492 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.495+0000 7f89637fe700 1 -- 172.21.15.101:0/887016102 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896c09b460 con 0x55bab55652d0 2023-07-18T03:15:08.492 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.495+0000 7f89925d5b80 1 -- 172.21.15.101:0/887016102 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bab55652d0 msgr2=0x55bab556fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:08.492 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.495+0000 7f89925d5b80 1 --2- 172.21.15.101:0/887016102 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bab55652d0 0x55bab556fb70 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f896c097700 tx=0x7f896c01a040 comp rx=0 tx=0).stop 2023-07-18T03:15:08.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.495+0000 7f89925d5b80 1 -- 172.21.15.101:0/887016102 shutdown_connections 2023-07-18T03:15:08.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.495+0000 7f89925d5b80 1 --2- 172.21.15.101:0/887016102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bab5570ad0 0x55bab5575c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.495+0000 7f89925d5b80 1 --2- 172.21.15.101:0/887016102 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bab55652d0 0x55bab556fb70 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.495+0000 7f89925d5b80 1 --2- 172.21.15.101:0/887016102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bab5565be0 0x55bab55756e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.495+0000 7f89925d5b80 1 -- 172.21.15.101:0/887016102 >> 172.21.15.101:0/887016102 conn(0x55bab555bda0 msgr2=0x55bab5586e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:08.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.496+0000 7f89925d5b80 1 -- 172.21.15.101:0/887016102 shutdown_connections 2023-07-18T03:15:08.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.496+0000 7f89925d5b80 1 -- 172.21.15.101:0/887016102 wait complete. 2023-07-18T03:15:08.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.496+0000 7f89925d5b80 1 -- start start 2023-07-18T03:15:08.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.496+0000 7f89925d5b80 1 Processor -- start 2023-07-18T03:15:08.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.497+0000 7f89925d5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bab55652d0 0x55bab55cd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:08.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.497+0000 7f89925d5b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bab5565be0 0x55bab55cd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:08.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.497+0000 7f89925d5b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bab5570ad0 0x55bab55cbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:08.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.497+0000 7f89925d5b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bab55cc520 con 0x55bab5565be0 2023-07-18T03:15:08.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.497+0000 7f89925d5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bab55cc690 con 0x55bab55652d0 2023-07-18T03:15:08.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.497+0000 7f89925d5b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bab55cc850 con 0x55bab5570ad0 2023-07-18T03:15:08.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.497+0000 7f89925d5b80 10 client.0 _ll_register_callbacks cb 0x7f89640124e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:15:08.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.497+0000 7f8977fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bab5565be0 0x55bab55cd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:08.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.497+0000 7f897d19c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bab5570ad0 0x55bab55cbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:08.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.497+0000 7f897c99b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bab55652d0 0x55bab55cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:08.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.497+0000 7f8977fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bab5565be0 0x55bab55cd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:39674/0 (socket says 172.21.15.101:39674) 2023-07-18T03:15:08.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.497+0000 7f8977fff700 1 -- 172.21.15.101:0/2742500890 learned_addr learned my addr 172.21.15.101:0/2742500890 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:08.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.497+0000 7f897c99b700 1 -- 172.21.15.101:0/2742500890 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bab5570ad0 msgr2=0x55bab55cbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:08.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.497+0000 7f897c99b700 1 --2- 172.21.15.101:0/2742500890 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bab5570ad0 0x55bab55cbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.497+0000 7f897c99b700 1 -- 172.21.15.101:0/2742500890 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bab5565be0 msgr2=0x55bab55cd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:08.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.497+0000 7f897c99b700 1 --2- 172.21.15.101:0/2742500890 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bab5565be0 0x55bab55cd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.497+0000 7f897c99b700 1 -- 172.21.15.101:0/2742500890 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bab52e5de0 con 0x55bab55652d0 2023-07-18T03:15:08.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.497+0000 7f8977fff700 1 --2- 172.21.15.101:0/2742500890 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bab5565be0 0x55bab55cd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:15:08.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.498+0000 7f897c99b700 1 --2- 172.21.15.101:0/2742500890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bab55652d0 0x55bab55cd220 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f896c09af00 tx=0x7f896c005c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1de5c475b7f38c6e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:08.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.498+0000 7f8957fff700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:15:08.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.498+0000 7f8957fff700 1 -- 172.21.15.101:0/2742500890 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896c0b9070 con 0x55bab55652d0 2023-07-18T03:15:08.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.498+0000 7f89925d5b80 1 -- 172.21.15.101:0/2742500890 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f896c09b200 con 0x55bab55652d0 2023-07-18T03:15:08.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.498+0000 7f8957fff700 1 -- 172.21.15.101:0/2742500890 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f896c0ce170 con 0x55bab55652d0 2023-07-18T03:15:08.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.498+0000 7f8957fff700 1 -- 172.21.15.101:0/2742500890 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f896c0b2050 con 0x55bab55652d0 2023-07-18T03:15:08.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.498+0000 7f89925d5b80 10 client.27122 fetch_fsmap learned FSMap version 29 2023-07-18T03:15:08.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.498+0000 7f89925d5b80 1 -- 172.21.15.101:0/2742500890 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55bab52e42b0 con 0x55bab55652d0 2023-07-18T03:15:08.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.498+0000 7f8957fff700 1 -- 172.21.15.101:0/2742500890 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f896c064080 con 0x55bab55652d0 2023-07-18T03:15:08.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.498+0000 7f8957fff700 1 -- 172.21.15.101:0/2742500890 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f896c09b4d0 con 0x55bab55652d0 2023-07-18T03:15:08.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.498+0000 7f89925d5b80 10 client.27122 fetch_fsmap finished waiting for FSMap version 29 2023-07-18T03:15:08.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.498+0000 7f89925d5b80 10 client.27122 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:15:08.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.498+0000 7f89925d5b80 20 client.27122 populate_metadata read hostname 'smithi101' 2023-07-18T03:15:08.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.498+0000 7f89925d5b80 1 --2- 172.21.15.101:0/2742500890 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55bab55d4cd0 0x55bab55d70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:08.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.498+0000 7f89925d5b80 4 client.27122 mds_command: new command op to 24562 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:15:08.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.498+0000 7f89925d5b80 1 -- 172.21.15.101:0/2742500890 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bab5473050 con 0x55bab55d4cd0 2023-07-18T03:15:08.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.499+0000 7f8977fff700 1 --2- 172.21.15.101:0/2742500890 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55bab55d4cd0 0x55bab55d70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:08.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.499+0000 7f8957fff700 10 client.27122 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:15:08.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.499+0000 7f8977fff700 1 --2- 172.21.15.101:0/2742500890 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55bab55d4cd0 0x55bab55d70b0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2fc4f79b99de096 server_cookie=ca29038d6cb2a17 in_seq=0 out_seq=0 2023-07-18T03:15:08.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.500+0000 7f8957fff700 1 -- 172.21.15.101:0/2742500890 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bab5473050 con 0x55bab55d4cd0 2023-07-18T03:15:08.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.500+0000 7f8957fff700 10 client.27122 handle_command_reply: tid=0 2023-07-18T03:15:08.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.500+0000 7f89925d5b80 1 client.27122 shutdown 2023-07-18T03:15:08.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.500+0000 7f89925d5b80 1 -- 172.21.15.101:0/2742500890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bab55652d0 msgr2=0x55bab55cd220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:08.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.500+0000 7f89925d5b80 1 --2- 172.21.15.101:0/2742500890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bab55652d0 0x55bab55cd220 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f896c09af00 tx=0x7f896c005c80 comp rx=0 tx=0).stop 2023-07-18T03:15:08.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.500+0000 7f89925d5b80 1 -- 172.21.15.101:0/2742500890 shutdown_connections 2023-07-18T03:15:08.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.500+0000 7f89925d5b80 1 --2- 172.21.15.101:0/2742500890 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55bab55d4cd0 0x55bab55d70b0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.500+0000 7f89925d5b80 1 --2- 172.21.15.101:0/2742500890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bab55652d0 0x55bab55cd220 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.500+0000 7f89925d5b80 1 --2- 172.21.15.101:0/2742500890 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bab5570ad0 0x55bab55cbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.500+0000 7f89925d5b80 1 --2- 172.21.15.101:0/2742500890 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bab5565be0 0x55bab55cd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.500+0000 7f8957fff700 0 client.27122 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:15:08.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.500+0000 7f89925d5b80 1 -- 172.21.15.101:0/2742500890 >> 172.21.15.101:0/2742500890 conn(0x55bab555bda0 msgr2=0x55bab5567300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:08.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.500+0000 7f89925d5b80 1 -- 172.21.15.101:0/2742500890 shutdown_connections 2023-07-18T03:15:08.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.500+0000 7f89925d5b80 1 -- 172.21.15.101:0/2742500890 wait complete. 2023-07-18T03:15:08.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.500+0000 7f89925d5b80 20 client.27122 trim_cache size 0 max 16384 2023-07-18T03:15:08.505 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:15:08.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.500+0000 7f897ebff700 1 -- 172.21.15.101:0/3555444560 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f89680a12c0 msgr2=0x7f89680a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:08.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.500+0000 7f897ebff700 1 --2- 172.21.15.101:0/3555444560 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f89680a12c0 0x7f89680a3770 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f897819c730 tx=0x7f8970009920 comp rx=0 tx=0).stop 2023-07-18T03:15:08.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.501+0000 7f897ebff700 1 -- 172.21.15.101:0/3555444560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89780ffae0 msgr2=0x7f897819ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:08.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.501+0000 7f897ebff700 1 --2- 172.21.15.101:0/3555444560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89780ffae0 0x7f897819ba30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f896c005910 tx=0x7f896c0057d0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.501+0000 7f897ebff700 1 -- 172.21.15.101:0/3555444560 shutdown_connections 2023-07-18T03:15:08.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.501+0000 7f897ebff700 1 --2- 172.21.15.101:0/3555444560 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8978105af0 0x7f8978195aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.501+0000 7f897ebff700 1 --2- 172.21.15.101:0/3555444560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89780ffae0 0x7f897819ba30 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.501+0000 7f897ebff700 1 --2- 172.21.15.101:0/3555444560 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f89680a12c0 0x7f89680a3770 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.501+0000 7f897ebff700 1 --2- 172.21.15.101:0/3555444560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8978100490 0x7f897819bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:08.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.501+0000 7f897ebff700 1 -- 172.21.15.101:0/3555444560 >> 172.21.15.101:0/3555444560 conn(0x7f89780fb5e0 msgr2=0x7f89780fbbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:08.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.501+0000 7f897ebff700 1 -- 172.21.15.101:0/3555444560 shutdown_connections 2023-07-18T03:15:08.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:08.501+0000 7f897ebff700 1 -- 172.21.15.101:0/3555444560 wait complete. 2023-07-18T03:15:08.592 INFO:tasks.workunit.client.0.smithi101.stdout: 1 530999 6.97 MB/sec execute 273 sec latency 1462.098 ms 2023-07-18T03:15:09.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:09 smithi155 ceph-mon[128885]: pgmap v233: 129 pgs: 129 active+clean; 376 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 159 op/s 2023-07-18T03:15:09.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:09 smithi179 ceph-mon[129229]: pgmap v233: 129 pgs: 129 active+clean; 376 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 159 op/s 2023-07-18T03:15:09.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:09 smithi101 ceph-mon[126379]: pgmap v233: 129 pgs: 129 active+clean; 376 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 159 op/s 2023-07-18T03:15:09.512 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-18T03:15:09.513 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-18T03:15:09.592 INFO:tasks.workunit.client.0.smithi101.stdout: 1 531264 6.94 MB/sec execute 274 sec latency 2312.500 ms 2023-07-18T03:15:09.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.756+0000 7f19c05ff700 1 -- 172.21.15.101:0/1602958802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19b80fe8a0 msgr2=0x7f19b80fec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:09.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.756+0000 7f19c05ff700 1 --2- 172.21.15.101:0/1602958802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19b80fe8a0 0x7f19b80fec70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f19b4009990 tx=0x7f19b4009ca0 comp rx=0 tx=0).stop 2023-07-18T03:15:09.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.756+0000 7f19c05ff700 1 -- 172.21.15.101:0/1602958802 shutdown_connections 2023-07-18T03:15:09.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.756+0000 7f19c05ff700 1 --2- 172.21.15.101:0/1602958802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19b80fe8a0 0x7f19b80fec70 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:09.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.756+0000 7f19c05ff700 1 --2- 172.21.15.101:0/1602958802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f19b810c450 0x7f19b810e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:09.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.756+0000 7f19c05ff700 1 --2- 172.21.15.101:0/1602958802 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f19b80ff1b0 0x7f19b810bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:09.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.756+0000 7f19c05ff700 1 -- 172.21.15.101:0/1602958802 >> 172.21.15.101:0/1602958802 conn(0x7f19b80fa4b0 msgr2=0x7f19b80fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:09.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.756+0000 7f19c05ff700 1 -- 172.21.15.101:0/1602958802 shutdown_connections 2023-07-18T03:15:09.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.756+0000 7f19c05ff700 1 -- 172.21.15.101:0/1602958802 wait complete. 2023-07-18T03:15:09.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.758+0000 7f19c05ff700 1 Processor -- start 2023-07-18T03:15:09.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.758+0000 7f19c05ff700 1 -- start start 2023-07-18T03:15:09.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.758+0000 7f19c05ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19b80fe8a0 0x7f19b8197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:09.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.758+0000 7f19c05ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f19b80ff1b0 0x7f19b8198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:09.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.758+0000 7f19c05ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f19b810c450 0x7f19b819c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:09.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.758+0000 7f19c05ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b819ce30 con 0x7f19b80ff1b0 2023-07-18T03:15:09.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.758+0000 7f19c05ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b819cfa0 con 0x7f19b80fe8a0 2023-07-18T03:15:09.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.758+0000 7f19c05ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b819d160 con 0x7f19b810c450 2023-07-18T03:15:09.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.758+0000 7f19be39b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19b80fe8a0 0x7f19b8197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:09.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.758+0000 7f19bdb9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f19b80ff1b0 0x7f19b8198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:09.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.758+0000 7f19be39b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19b80fe8a0 0x7f19b8197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:37316/0 (socket says 172.21.15.101:37316) 2023-07-18T03:15:09.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.758+0000 7f19be39b700 1 -- 172.21.15.101:0/3830592223 learned_addr learned my addr 172.21.15.101:0/3830592223 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:09.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.758+0000 7f19beb9c700 1 --2- 172.21.15.101:0/3830592223 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f19b810c450 0x7f19b819c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:09.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.759+0000 7f19be39b700 1 -- 172.21.15.101:0/3830592223 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f19b810c450 msgr2=0x7f19b819c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:09.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.759+0000 7f19be39b700 1 --2- 172.21.15.101:0/3830592223 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f19b810c450 0x7f19b819c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:09.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.759+0000 7f19be39b700 1 -- 172.21.15.101:0/3830592223 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f19b80ff1b0 msgr2=0x7f19b8198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:09.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.759+0000 7f19be39b700 1 --2- 172.21.15.101:0/3830592223 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f19b80ff1b0 0x7f19b8198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:09.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.759+0000 7f19be39b700 1 -- 172.21.15.101:0/3830592223 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a8009520 con 0x7f19b80fe8a0 2023-07-18T03:15:09.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.759+0000 7f19be39b700 1 --2- 172.21.15.101:0/3830592223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19b80fe8a0 0x7f19b8197e50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f19b4005910 tx=0x7f19b4004710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:09.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.759+0000 7f19af7fe700 1 -- 172.21.15.101:0/3830592223 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b401d070 con 0x7f19b80fe8a0 2023-07-18T03:15:09.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.759+0000 7f19af7fe700 1 -- 172.21.15.101:0/3830592223 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19b4022470 con 0x7f19b80fe8a0 2023-07-18T03:15:09.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.759+0000 7f19c05ff700 1 -- 172.21.15.101:0/3830592223 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19b40095f0 con 0x7f19b80fe8a0 2023-07-18T03:15:09.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.759+0000 7f19af7fe700 1 -- 172.21.15.101:0/3830592223 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b4017650 con 0x7f19b80fe8a0 2023-07-18T03:15:09.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.759+0000 7f19c05ff700 1 -- 172.21.15.101:0/3830592223 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19b819d670 con 0x7f19b80fe8a0 2023-07-18T03:15:09.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.763+0000 7f19af7fe700 1 -- 172.21.15.101:0/3830592223 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f19b40225e0 con 0x7f19b80fe8a0 2023-07-18T03:15:09.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.764+0000 7f19d3fa5b80 1 Processor -- start 2023-07-18T03:15:09.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.764+0000 7f19d3fa5b80 1 -- start start 2023-07-18T03:15:09.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.764+0000 7f19af7fe700 1 --2- 172.21.15.101:0/3830592223 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f19a40a12c0 0x7f19a40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:09.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.764+0000 7f19d3fa5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562373ec8350 0x562373ec8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:09.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.764+0000 7f19d3fa5b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562373ec8c60 0x562373ed3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:09.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.764+0000 7f19af7fe700 1 -- 172.21.15.101:0/3830592223 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f19b40ce390 con 0x7f19b80fe8a0 2023-07-18T03:15:09.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.764+0000 7f19d3fa5b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562373ed3b50 0x562373ed5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:09.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.764+0000 7f19d3fa5b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562373dced30 con 0x562373ec8c60 2023-07-18T03:15:09.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.764+0000 7f19d3fa5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562373e21ac0 con 0x562373ec8350 2023-07-18T03:15:09.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.764+0000 7f19d3fa5b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562373e1af50 con 0x562373ed3b50 2023-07-18T03:15:09.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.765+0000 7f19beb9c700 1 --2- 172.21.15.101:0/3830592223 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f19a40a12c0 0x7f19a40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:09.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.765+0000 7f19be39b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562373ec8350 0x562373ec8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:09.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.765+0000 7f19beb9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562373ed3b50 0x562373ed5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:09.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.765+0000 7f19be39b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562373ec8350 0x562373ec8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:37332/0 (socket says 172.21.15.101:37332) 2023-07-18T03:15:09.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.765+0000 7f19be39b700 1 -- 172.21.15.101:0/1252337390 learned_addr learned my addr 172.21.15.101:0/1252337390 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:09.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.765+0000 7f19bdb9a700 1 --2- 172.21.15.101:0/1252337390 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562373ec8c60 0x562373ed3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:09.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.765+0000 7f19be39b700 1 -- 172.21.15.101:0/1252337390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562373ed3b50 msgr2=0x562373ed5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:09.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.765+0000 7f19be39b700 1 --2- 172.21.15.101:0/1252337390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562373ed3b50 0x562373ed5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:09.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.765+0000 7f19be39b700 1 -- 172.21.15.101:0/1252337390 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562373ec8c60 msgr2=0x562373ed3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:09.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.765+0000 7f19be39b700 1 --2- 172.21.15.101:0/1252337390 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562373ec8c60 0x562373ed3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:09.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.765+0000 7f19be39b700 1 -- 172.21.15.101:0/1252337390 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562373c48e60 con 0x562373ec8350 2023-07-18T03:15:09.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.765+0000 7f19be39b700 1 --2- 172.21.15.101:0/1252337390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562373ec8350 0x562373ec8720 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f19b4006010 tx=0x7f19b409ac50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65bc92dd67ab3b7b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:09.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.765+0000 7f19beb9c700 1 --2- 172.21.15.101:0/3830592223 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f19a40a12c0 0x7f19a40a3770 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f19b0009ec0 tx=0x7f19b0009920 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:09.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.765+0000 7f19acbfe700 1 -- 172.21.15.101:0/1252337390 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b40ce170 con 0x562373ec8350 2023-07-18T03:15:09.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.765+0000 7f19acbfe700 1 -- 172.21.15.101:0/1252337390 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19b409f0c0 con 0x562373ec8350 2023-07-18T03:15:09.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.765+0000 7f19acbfe700 1 -- 172.21.15.101:0/1252337390 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b409a7e0 con 0x562373ec8350 2023-07-18T03:15:09.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.766+0000 7f19d3fa5b80 1 -- 172.21.15.101:0/1252337390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562373ec8350 msgr2=0x562373ec8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:09.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.766+0000 7f19d3fa5b80 1 --2- 172.21.15.101:0/1252337390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562373ec8350 0x562373ec8720 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f19b4006010 tx=0x7f19b409ac50 comp rx=0 tx=0).stop 2023-07-18T03:15:09.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.766+0000 7f19d3fa5b80 1 -- 172.21.15.101:0/1252337390 shutdown_connections 2023-07-18T03:15:09.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.766+0000 7f19d3fa5b80 1 --2- 172.21.15.101:0/1252337390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562373ec8350 0x562373ec8720 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:09.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.766+0000 7f19d3fa5b80 1 --2- 172.21.15.101:0/1252337390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562373ed3b50 0x562373ed5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:09.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.766+0000 7f19d3fa5b80 1 --2- 172.21.15.101:0/1252337390 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562373ec8c60 0x562373ed3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:09.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.766+0000 7f19d3fa5b80 1 -- 172.21.15.101:0/1252337390 >> 172.21.15.101:0/1252337390 conn(0x562373ebee20 msgr2=0x562373ec4720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:09.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.766+0000 7f19d3fa5b80 1 -- 172.21.15.101:0/1252337390 shutdown_connections 2023-07-18T03:15:09.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.766+0000 7f19d3fa5b80 1 -- 172.21.15.101:0/1252337390 wait complete. 2023-07-18T03:15:09.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.767+0000 7f19d3fa5b80 1 -- start start 2023-07-18T03:15:09.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.767+0000 7f19d3fa5b80 1 Processor -- start 2023-07-18T03:15:09.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.768+0000 7f19d3fa5b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562373ec8350 0x562373ed8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:09.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.768+0000 7f19d3fa5b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562373ec8c60 0x562373ed9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:09.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.768+0000 7f19d3fa5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562373ed3b50 0x562373ed2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:09.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.768+0000 7f19d3fa5b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562373dbab40 con 0x562373ec8c60 2023-07-18T03:15:09.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.768+0000 7f19d3fa5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562373dba840 con 0x562373ed3b50 2023-07-18T03:15:09.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.768+0000 7f19d3fa5b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562373c9e9a0 con 0x562373ec8350 2023-07-18T03:15:09.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.768+0000 7f19d3fa5b80 10 client.0 _ll_register_callbacks cb 0x562373db87e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:15:09.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.768+0000 7f19be39b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562373ec8350 0x562373ed8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:09.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.768+0000 7f19be39b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562373ec8350 0x562373ed8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:47040/0 (socket says 172.21.15.101:47040) 2023-07-18T03:15:09.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.768+0000 7f19be39b700 1 -- 172.21.15.101:0/3246530914 learned_addr learned my addr 172.21.15.101:0/3246530914 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:09.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.768+0000 7f19bdb9a700 1 --2- 172.21.15.101:0/3246530914 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562373ec8c60 0x562373ed9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:09.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.768+0000 7f19beb9c700 1 --2- 172.21.15.101:0/3246530914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562373ed3b50 0x562373ed2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:09.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.769+0000 7f19be39b700 1 -- 172.21.15.101:0/3246530914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562373ed3b50 msgr2=0x562373ed2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:09.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.769+0000 7f19be39b700 1 --2- 172.21.15.101:0/3246530914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562373ed3b50 0x562373ed2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:09.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.769+0000 7f19be39b700 1 -- 172.21.15.101:0/3246530914 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562373ec8c60 msgr2=0x562373ed9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:09.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.769+0000 7f19be39b700 1 --2- 172.21.15.101:0/3246530914 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562373ec8c60 0x562373ed9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:09.769 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.769+0000 7f19be39b700 1 -- 172.21.15.101:0/3246530914 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19b819d670 con 0x562373ec8350 2023-07-18T03:15:09.769 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.769+0000 7f19be39b700 1 --2- 172.21.15.101:0/3246530914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562373ec8350 0x562373ed8be0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f19b4097590 tx=0x7f19b4022b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9e8c47179d23ce8 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:09.769 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.769+0000 7f19a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:15:09.769 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.769+0000 7f19a1ffb700 1 -- 172.21.15.101:0/3246530914 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b40ce170 con 0x562373ec8350 2023-07-18T03:15:09.769 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.769+0000 7f19a1ffb700 1 -- 172.21.15.101:0/3246530914 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f19b40a3450 con 0x562373ec8350 2023-07-18T03:15:09.769 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.769+0000 7f19d3fa5b80 1 -- 172.21.15.101:0/3246530914 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562373d74570 con 0x562373ec8350 2023-07-18T03:15:09.769 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.769+0000 7f19a1ffb700 1 -- 172.21.15.101:0/3246530914 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19b409b500 con 0x562373ec8350 2023-07-18T03:15:09.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.769+0000 7f19d3fa5b80 10 client.27169 fetch_fsmap learned FSMap version 29 2023-07-18T03:15:09.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.769+0000 7f19d3fa5b80 1 -- 172.21.15.101:0/3246530914 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x562373c31ce0 con 0x562373ec8350 2023-07-18T03:15:09.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.770+0000 7f19a1ffb700 1 -- 172.21.15.101:0/3246530914 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19b40cb080 con 0x562373ec8350 2023-07-18T03:15:09.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.770+0000 7f19a1ffb700 1 -- 172.21.15.101:0/3246530914 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f19b40a3050 con 0x562373ec8350 2023-07-18T03:15:09.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.770+0000 7f19d3fa5b80 10 client.27169 fetch_fsmap finished waiting for FSMap version 29 2023-07-18T03:15:09.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.770+0000 7f19d3fa5b80 10 client.27169 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:15:09.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.770+0000 7f19d3fa5b80 20 client.27169 populate_metadata read hostname 'smithi101' 2023-07-18T03:15:09.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.770+0000 7f19d3fa5b80 1 --2- 172.21.15.101:0/3246530914 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x562373ee2bb0 0x562373ee4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:09.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.770+0000 7f19d3fa5b80 4 client.27169 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:15:09.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.770+0000 7f19d3fa5b80 1 -- 172.21.15.101:0/3246530914 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562373e13f30 con 0x562373ee2bb0 2023-07-18T03:15:09.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.770+0000 7f19bdb9a700 1 --2- 172.21.15.101:0/3246530914 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x562373ee2bb0 0x562373ee4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:09.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.771+0000 7f19a1ffb700 10 client.27169 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:15:09.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.771+0000 7f19bdb9a700 1 --2- 172.21.15.101:0/3246530914 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x562373ee2bb0 0x562373ee4f90 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=179e816a700b422a server_cookie=a159705de85d0638 in_seq=0 out_seq=0 2023-07-18T03:15:09.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.771+0000 7f19a1ffb700 1 -- 172.21.15.101:0/3246530914 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562373e13f30 con 0x562373ee2bb0 2023-07-18T03:15:09.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.771+0000 7f19a1ffb700 10 client.27169 handle_command_reply: tid=0 2023-07-18T03:15:09.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.771+0000 7f19d3fa5b80 1 client.27169 shutdown 2023-07-18T03:15:09.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.771+0000 7f19d3fa5b80 1 -- 172.21.15.101:0/3246530914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562373ec8350 msgr2=0x562373ed8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:09.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.771+0000 7f19d3fa5b80 1 --2- 172.21.15.101:0/3246530914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562373ec8350 0x562373ed8be0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f19b4097590 tx=0x7f19b4022b30 comp rx=0 tx=0).stop 2023-07-18T03:15:09.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.771+0000 7f19d3fa5b80 1 -- 172.21.15.101:0/3246530914 shutdown_connections 2023-07-18T03:15:09.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.771+0000 7f19d3fa5b80 1 --2- 172.21.15.101:0/3246530914 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x562373ee2bb0 0x562373ee4f90 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:09.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.772+0000 7f19d3fa5b80 1 --2- 172.21.15.101:0/3246530914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562373ed3b50 0x562373ed2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:09.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.772+0000 7f19d3fa5b80 1 --2- 172.21.15.101:0/3246530914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562373ec8350 0x562373ed8be0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:09.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.772+0000 7f19d3fa5b80 1 --2- 172.21.15.101:0/3246530914 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562373ec8c60 0x562373ed9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:09.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.772+0000 7f19a1ffb700 0 client.27169 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:15:09.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.772+0000 7f19d3fa5b80 1 -- 172.21.15.101:0/3246530914 >> 172.21.15.101:0/3246530914 conn(0x562373ebee20 msgr2=0x562373ec4310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:09.774 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.772+0000 7f19d3fa5b80 1 -- 172.21.15.101:0/3246530914 shutdown_connections 2023-07-18T03:15:09.774 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.772+0000 7f19d3fa5b80 1 -- 172.21.15.101:0/3246530914 wait complete. 2023-07-18T03:15:09.774 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.772+0000 7f19d3fa5b80 20 client.27169 trim_cache size 0 max 16384 2023-07-18T03:15:09.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.790+0000 7f19d3fa5b80 1 Processor -- start 2023-07-18T03:15:09.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.790+0000 7f19d3fa5b80 1 -- start start 2023-07-18T03:15:09.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.790+0000 7f19d3fa5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562373ec8350 0x562373ed8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:09.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.790+0000 7f19d3fa5b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562373ec8c60 0x562373ed8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:09.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.790+0000 7f19d3fa5b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562373ed3b50 0x562373ecfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:09.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.790+0000 7f19d3fa5b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562373ec9ef0 con 0x562373ec8c60 2023-07-18T03:15:09.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.791+0000 7f19d3fa5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562373ebe9f0 con 0x562373ec8350 2023-07-18T03:15:09.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.791+0000 7f19d3fa5b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562373ed0ff0 con 0x562373ed3b50 2023-07-18T03:15:09.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.791+0000 7f19be39b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562373ec8350 0x562373ed8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:09.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.791+0000 7f19be39b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562373ec8350 0x562373ed8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:37352/0 (socket says 172.21.15.101:37352) 2023-07-18T03:15:09.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.791+0000 7f19be39b700 1 -- 172.21.15.101:0/3721701884 learned_addr learned my addr 172.21.15.101:0/3721701884 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:09.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.791+0000 7f19beb9c700 1 --2- 172.21.15.101:0/3721701884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562373ed3b50 0x562373ecfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:09.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.791+0000 7f19bdb9a700 1 --2- 172.21.15.101:0/3721701884 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562373ec8c60 0x562373ed8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:09.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.791+0000 7f19be39b700 1 -- 172.21.15.101:0/3721701884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562373ed3b50 msgr2=0x562373ecfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:09.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.791+0000 7f19be39b700 1 --2- 172.21.15.101:0/3721701884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562373ed3b50 0x562373ecfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:09.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.791+0000 7f19be39b700 1 -- 172.21.15.101:0/3721701884 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562373ec8c60 msgr2=0x562373ed8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:09.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.791+0000 7f19be39b700 1 --2- 172.21.15.101:0/3721701884 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562373ec8c60 0x562373ed8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:09.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.791+0000 7f19be39b700 1 -- 172.21.15.101:0/3721701884 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562373c6f2d0 con 0x562373ec8350 2023-07-18T03:15:09.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.791+0000 7f19be39b700 1 --2- 172.21.15.101:0/3721701884 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562373ec8350 0x562373ed8770 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f19b4022c80 tx=0x7f19b409ac50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f223208bf7d2d6fa server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:09.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.792+0000 7f19ad3ff700 1 -- 172.21.15.101:0/3721701884 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b409ef00 con 0x562373ec8350 2023-07-18T03:15:09.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.792+0000 7f19ad3ff700 1 -- 172.21.15.101:0/3721701884 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562373d74570 con 0x562373ec8350 2023-07-18T03:15:09.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.792+0000 7f19ad3ff700 1 -- 172.21.15.101:0/3721701884 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b40a2fc0 con 0x562373ec8350 2023-07-18T03:15:09.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.792+0000 7f19d3fa5b80 1 -- 172.21.15.101:0/3721701884 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562373ec8350 msgr2=0x562373ed8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:09.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.792+0000 7f19d3fa5b80 1 --2- 172.21.15.101:0/3721701884 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562373ec8350 0x562373ed8770 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f19b4022c80 tx=0x7f19b409ac50 comp rx=0 tx=0).stop 2023-07-18T03:15:09.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.792+0000 7f19d3fa5b80 1 -- 172.21.15.101:0/3721701884 shutdown_connections 2023-07-18T03:15:09.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.792+0000 7f19d3fa5b80 1 --2- 172.21.15.101:0/3721701884 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562373ec8350 0x562373ed8770 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:09.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.792+0000 7f19d3fa5b80 1 --2- 172.21.15.101:0/3721701884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562373ed3b50 0x562373ecfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:09.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.792+0000 7f19d3fa5b80 1 --2- 172.21.15.101:0/3721701884 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562373ec8c60 0x562373ed8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:09.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.792+0000 7f19d3fa5b80 1 -- 172.21.15.101:0/3721701884 >> 172.21.15.101:0/3721701884 conn(0x562373ebee20 msgr2=0x562373ed1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:09.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.792+0000 7f19d3fa5b80 1 -- 172.21.15.101:0/3721701884 shutdown_connections 2023-07-18T03:15:09.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.792+0000 7f19d3fa5b80 1 -- 172.21.15.101:0/3721701884 wait complete. 2023-07-18T03:15:09.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.793+0000 7f19d3fa5b80 1 -- start start 2023-07-18T03:15:09.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.793+0000 7f19d3fa5b80 1 Processor -- start 2023-07-18T03:15:09.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.793+0000 7f19d3fa5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562373ec8350 0x562373f307c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:09.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.793+0000 7f19d3fa5b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562373ec8c60 0x562373f30d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:09.793 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.793+0000 7f19d3fa5b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562373ed3b50 0x562373f2abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:09.793 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.793+0000 7f19d3fa5b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562373f2b190 con 0x562373ec8c60 2023-07-18T03:15:09.793 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.793+0000 7f19d3fa5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562373f2b300 con 0x562373ec8350 2023-07-18T03:15:09.793 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.793+0000 7f19d3fa5b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562373f2b4c0 con 0x562373ed3b50 2023-07-18T03:15:09.793 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.793+0000 7f19d3fa5b80 10 client.0 _ll_register_callbacks cb 0x7f19a80099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:15:09.793 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.793+0000 7f19be39b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562373ec8350 0x562373f307c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:09.793 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.794+0000 7f19be39b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562373ec8350 0x562373f307c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:37366/0 (socket says 172.21.15.101:37366) 2023-07-18T03:15:09.793 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.794+0000 7f19be39b700 1 -- 172.21.15.101:0/4183743913 learned_addr learned my addr 172.21.15.101:0/4183743913 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:09.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.794+0000 7f19beb9c700 1 --2- 172.21.15.101:0/4183743913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562373ed3b50 0x562373f2abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:09.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.794+0000 7f19bdb9a700 1 --2- 172.21.15.101:0/4183743913 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562373ec8c60 0x562373f30d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:09.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.794+0000 7f19be39b700 1 -- 172.21.15.101:0/4183743913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562373ed3b50 msgr2=0x562373f2abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:09.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.794+0000 7f19be39b700 1 --2- 172.21.15.101:0/4183743913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562373ed3b50 0x562373f2abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:09.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.794+0000 7f19be39b700 1 -- 172.21.15.101:0/4183743913 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562373ec8c60 msgr2=0x562373f30d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:09.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.794+0000 7f19be39b700 1 --2- 172.21.15.101:0/4183743913 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562373ec8c60 0x562373f30d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:09.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.794+0000 7f19be39b700 1 -- 172.21.15.101:0/4183743913 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562373c31ce0 con 0x562373ec8350 2023-07-18T03:15:09.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.794+0000 7f19beb9c700 1 --2- 172.21.15.101:0/4183743913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562373ed3b50 0x562373f2abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:15:09.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.794+0000 7f19be39b700 1 --2- 172.21.15.101:0/4183743913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562373ec8350 0x562373f307c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f19b400b6f0 tx=0x7f19b4022b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a287cd79f969872c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:09.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.794+0000 7f19a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:15:09.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.794+0000 7f19a1ffb700 1 -- 172.21.15.101:0/4183743913 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b409ef00 con 0x562373ec8350 2023-07-18T03:15:09.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.794+0000 7f19a1ffb700 1 -- 172.21.15.101:0/4183743913 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f19b40ce170 con 0x562373ec8350 2023-07-18T03:15:09.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.794+0000 7f19d3fa5b80 1 -- 172.21.15.101:0/4183743913 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f19b409f0c0 con 0x562373ec8350 2023-07-18T03:15:09.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.795+0000 7f19a1ffb700 1 -- 172.21.15.101:0/4183743913 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19b40a3450 con 0x562373ec8350 2023-07-18T03:15:09.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.795+0000 7f19d3fa5b80 10 client.27140 fetch_fsmap learned FSMap version 29 2023-07-18T03:15:09.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.795+0000 7f19d3fa5b80 1 -- 172.21.15.101:0/4183743913 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x562373c47330 con 0x562373ec8350 2023-07-18T03:15:09.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.795+0000 7f19a1ffb700 1 -- 172.21.15.101:0/4183743913 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19b409f0c0 con 0x562373ec8350 2023-07-18T03:15:09.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.795+0000 7f19a1ffb700 1 -- 172.21.15.101:0/4183743913 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f19b4099ff0 con 0x562373ec8350 2023-07-18T03:15:09.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.795+0000 7f19d3fa5b80 10 client.27140 fetch_fsmap finished waiting for FSMap version 29 2023-07-18T03:15:09.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.795+0000 7f19d3fa5b80 10 client.27140 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:15:09.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.795+0000 7f19d3fa5b80 20 client.27140 populate_metadata read hostname 'smithi101' 2023-07-18T03:15:09.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.795+0000 7f19d3fa5b80 1 --2- 172.21.15.101:0/4183743913 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x562373f38180 0x562373f3a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:09.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.795+0000 7f19d3fa5b80 4 client.27140 mds_command: new command op to 24562 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-18T03:15:09.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.795+0000 7f19d3fa5b80 1 -- 172.21.15.101:0/4183743913 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x562373dd60d0 con 0x562373f38180 2023-07-18T03:15:09.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.795+0000 7f19bdb9a700 1 --2- 172.21.15.101:0/4183743913 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x562373f38180 0x562373f3a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:09.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.796+0000 7f19bdb9a700 1 --2- 172.21.15.101:0/4183743913 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x562373f38180 0x562373f3a560 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22dae2961538e869 server_cookie=3beee0ae9006feb7 in_seq=0 out_seq=0 2023-07-18T03:15:09.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.796+0000 7f19a1ffb700 10 client.27140 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:15:09.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.797+0000 7f19a1ffb700 1 -- 172.21.15.101:0/4183743913 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x562373dd60d0 con 0x562373f38180 2023-07-18T03:15:09.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.797+0000 7f19a1ffb700 10 client.27140 handle_command_reply: tid=0 2023-07-18T03:15:09.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.797+0000 7f19d3fa5b80 1 client.27140 shutdown 2023-07-18T03:15:09.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.797+0000 7f19d3fa5b80 1 -- 172.21.15.101:0/4183743913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562373ec8350 msgr2=0x562373f307c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:09.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.797+0000 7f19d3fa5b80 1 --2- 172.21.15.101:0/4183743913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562373ec8350 0x562373f307c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f19b400b6f0 tx=0x7f19b4022b10 comp rx=0 tx=0).stop 2023-07-18T03:15:09.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.797+0000 7f19d3fa5b80 1 -- 172.21.15.101:0/4183743913 shutdown_connections 2023-07-18T03:15:09.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.797+0000 7f19d3fa5b80 1 --2- 172.21.15.101:0/4183743913 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x562373f38180 0x562373f3a560 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:09.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.797+0000 7f19d3fa5b80 1 --2- 172.21.15.101:0/4183743913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562373ec8350 0x562373f307c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:09.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.797+0000 7f19d3fa5b80 1 --2- 172.21.15.101:0/4183743913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562373ed3b50 0x562373f2abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:09.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.797+0000 7f19d3fa5b80 1 --2- 172.21.15.101:0/4183743913 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562373ec8c60 0x562373f30d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:09.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.797+0000 7f19a1ffb700 0 client.27140 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:15:09.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.797+0000 7f19d3fa5b80 1 -- 172.21.15.101:0/4183743913 >> 172.21.15.101:0/4183743913 conn(0x562373ebee20 msgr2=0x562373ec3930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:09.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.797+0000 7f19d3fa5b80 1 -- 172.21.15.101:0/4183743913 shutdown_connections 2023-07-18T03:15:09.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.797+0000 7f19d3fa5b80 1 -- 172.21.15.101:0/4183743913 wait complete. 2023-07-18T03:15:09.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.797+0000 7f19d3fa5b80 20 client.27140 trim_cache size 0 max 16384 2023-07-18T03:15:09.800 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:15:09.800 INFO:teuthology.orchestra.run.smithi101.stdout: "return_code": 0, 2023-07-18T03:15:09.800 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_tag": "23994a27-b341-4986-8822-5a01f3ba7acf", 2023-07-18T03:15:09.800 INFO:teuthology.orchestra.run.smithi101.stdout: "mode": "asynchronous" 2023-07-18T03:15:09.801 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:15:09.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.798+0000 7f19c05ff700 1 -- 172.21.15.101:0/3830592223 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f19a40a12c0 msgr2=0x7f19a40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:09.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.798+0000 7f19c05ff700 1 --2- 172.21.15.101:0/3830592223 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f19a40a12c0 0x7f19a40a3770 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f19b0009ec0 tx=0x7f19b0009920 comp rx=0 tx=0).stop 2023-07-18T03:15:09.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.798+0000 7f19c05ff700 1 -- 172.21.15.101:0/3830592223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19b80fe8a0 msgr2=0x7f19b8197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:09.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.798+0000 7f19c05ff700 1 --2- 172.21.15.101:0/3830592223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19b80fe8a0 0x7f19b8197e50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f19b4005910 tx=0x7f19b4004710 comp rx=0 tx=0).stop 2023-07-18T03:15:09.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.798+0000 7f19c05ff700 1 -- 172.21.15.101:0/3830592223 shutdown_connections 2023-07-18T03:15:09.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.798+0000 7f19c05ff700 1 --2- 172.21.15.101:0/3830592223 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f19a40a12c0 0x7f19a40a3770 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:09.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.798+0000 7f19c05ff700 1 --2- 172.21.15.101:0/3830592223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19b80fe8a0 0x7f19b8197e50 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:09.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.798+0000 7f19c05ff700 1 --2- 172.21.15.101:0/3830592223 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f19b810c450 0x7f19b819c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:09.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.798+0000 7f19c05ff700 1 --2- 172.21.15.101:0/3830592223 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f19b80ff1b0 0x7f19b8198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:09.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.798+0000 7f19c05ff700 1 -- 172.21.15.101:0/3830592223 >> 172.21.15.101:0/3830592223 conn(0x7f19b80fa4b0 msgr2=0x7f19b810d810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:09.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.798+0000 7f19c05ff700 1 -- 172.21.15.101:0/3830592223 shutdown_connections 2023-07-18T03:15:09.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:09.798+0000 7f19c05ff700 1 -- 172.21.15.101:0/3830592223 wait complete. 2023-07-18T03:15:09.811 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:15:10.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.032+0000 7fe8cf7ff700 1 -- 172.21.15.101:0/3329522806 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8c8105af0 msgr2=0x7fe8c8105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:10.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.032+0000 7fe8cf7ff700 1 --2- 172.21.15.101:0/3329522806 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8c8105af0 0x7fe8c8105ec0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4009990 tx=0x7fe8c4009ca0 comp rx=0 tx=0).stop 2023-07-18T03:15:10.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.032+0000 7fe8cf7ff700 1 -- 172.21.15.101:0/3329522806 shutdown_connections 2023-07-18T03:15:10.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.032+0000 7fe8cf7ff700 1 --2- 172.21.15.101:0/3329522806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8c8100490 0x7fe8c810c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:10.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.032+0000 7fe8cf7ff700 1 --2- 172.21.15.101:0/3329522806 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8c8105af0 0x7fe8c8105ec0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:10.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.032+0000 7fe8cf7ff700 1 --2- 172.21.15.101:0/3329522806 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe8c80ffae0 0x7fe8c80fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:10.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.032+0000 7fe8cf7ff700 1 -- 172.21.15.101:0/3329522806 >> 172.21.15.101:0/3329522806 conn(0x7fe8c80fb5e0 msgr2=0x7fe8c80fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:10.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.032+0000 7fe8cf7ff700 1 -- 172.21.15.101:0/3329522806 shutdown_connections 2023-07-18T03:15:10.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.032+0000 7fe8cf7ff700 1 -- 172.21.15.101:0/3329522806 wait complete. 2023-07-18T03:15:10.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.033+0000 7fe8cf7ff700 1 Processor -- start 2023-07-18T03:15:10.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.034+0000 7fe8cf7ff700 1 -- start start 2023-07-18T03:15:10.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.034+0000 7fe8cf7ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe8c80ffae0 0x7fe8c8072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:10.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.034+0000 7fe8cf7ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8c8100490 0x7fe8c8072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:10.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.034+0000 7fe8cf7ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8c8105af0 0x7fe8c81a3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:10.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.034+0000 7fe8cf7ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c8073580 con 0x7fe8c80ffae0 2023-07-18T03:15:10.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.034+0000 7fe8cf7ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c81a4340 con 0x7fe8c8100490 2023-07-18T03:15:10.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.034+0000 7fe8cf7ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c81a4480 con 0x7fe8c8105af0 2023-07-18T03:15:10.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.034+0000 7fe8cd59b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe8c80ffae0 0x7fe8c8072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:10.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.034+0000 7fe8ccd9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8c8100490 0x7fe8c8072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:10.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.034+0000 7fe8cd59b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe8c80ffae0 0x7fe8c8072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:39756/0 (socket says 172.21.15.101:39756) 2023-07-18T03:15:10.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.034+0000 7fe8cd59b700 1 -- 172.21.15.101:0/106749649 learned_addr learned my addr 172.21.15.101:0/106749649 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:10.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.034+0000 7fe8cdd9c700 1 --2- 172.21.15.101:0/106749649 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8c8105af0 0x7fe8c81a3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:10.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.035+0000 7fe8ccd9a700 1 -- 172.21.15.101:0/106749649 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8c8105af0 msgr2=0x7fe8c81a3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:10.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.035+0000 7fe8ccd9a700 1 --2- 172.21.15.101:0/106749649 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8c8105af0 0x7fe8c81a3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:10.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.035+0000 7fe8ccd9a700 1 -- 172.21.15.101:0/106749649 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe8c80ffae0 msgr2=0x7fe8c8072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:10.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.035+0000 7fe8ccd9a700 1 --2- 172.21.15.101:0/106749649 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe8c80ffae0 0x7fe8c8072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:10.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.035+0000 7fe8ccd9a700 1 -- 172.21.15.101:0/106749649 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c40095f0 con 0x7fe8c8100490 2023-07-18T03:15:10.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.035+0000 7fe8cd59b700 1 --2- 172.21.15.101:0/106749649 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe8c80ffae0 0x7fe8c8072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:15:10.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.035+0000 7fe8ccd9a700 1 --2- 172.21.15.101:0/106749649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8c8100490 0x7fe8c8072d50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe8b800e9f0 tx=0x7fe8b800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:10.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.035+0000 7fe8be7fc700 1 -- 172.21.15.101:0/106749649 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b800c9c0 con 0x7fe8c8100490 2023-07-18T03:15:10.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.035+0000 7fe8cf7ff700 1 -- 172.21.15.101:0/106749649 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8c81a46e0 con 0x7fe8c8100490 2023-07-18T03:15:10.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.035+0000 7fe8be7fc700 1 -- 172.21.15.101:0/106749649 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8b8010430 con 0x7fe8c8100490 2023-07-18T03:15:10.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.035+0000 7fe8be7fc700 1 -- 172.21.15.101:0/106749649 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8018680 con 0x7fe8c8100490 2023-07-18T03:15:10.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.035+0000 7fe8cf7ff700 1 -- 172.21.15.101:0/106749649 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c81a4c10 con 0x7fe8c8100490 2023-07-18T03:15:10.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.040+0000 7fe8be7fc700 1 -- 172.21.15.101:0/106749649 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fe8b8010860 con 0x7fe8c8100490 2023-07-18T03:15:10.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.040+0000 7fe8e30fab80 1 Processor -- start 2023-07-18T03:15:10.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.040+0000 7fe8e30fab80 1 -- start start 2023-07-18T03:15:10.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.041+0000 7fe8e30fab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfc0a52d0 0x564dfc0a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:10.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.041+0000 7fe8e30fab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564dfc0a5be0 0x564dfc0b0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:10.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.041+0000 7fe8e30fab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564dfc0b0ad0 0x564dfc0b2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:10.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.041+0000 7fe8e30fab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dfbfabcb0 con 0x564dfc0a5be0 2023-07-18T03:15:10.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.041+0000 7fe8e30fab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dfbffea40 con 0x564dfc0a52d0 2023-07-18T03:15:10.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.041+0000 7fe8e30fab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dfbff7ed0 con 0x564dfc0b0ad0 2023-07-18T03:15:10.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.041+0000 7fe8cdd9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfc0a52d0 0x564dfc0a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:10.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.041+0000 7fe8ccd9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564dfc0b0ad0 0x564dfc0b2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:10.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.041+0000 7fe8cdd9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfc0a52d0 0x564dfc0a56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:37396/0 (socket says 172.21.15.101:37396) 2023-07-18T03:15:10.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.041+0000 7fe8cdd9c700 1 -- 172.21.15.101:0/145117018 learned_addr learned my addr 172.21.15.101:0/145117018 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:10.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.041+0000 7fe8cd59b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564dfc0a5be0 0x564dfc0b0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:10.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.042+0000 7fe8cdd9c700 1 -- 172.21.15.101:0/145117018 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564dfc0b0ad0 msgr2=0x564dfc0b2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:10.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.042+0000 7fe8cdd9c700 1 --2- 172.21.15.101:0/145117018 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564dfc0b0ad0 0x564dfc0b2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:10.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.042+0000 7fe8cdd9c700 1 -- 172.21.15.101:0/145117018 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564dfc0a5be0 msgr2=0x564dfc0b0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:10.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.042+0000 7fe8cdd9c700 1 --2- 172.21.15.101:0/145117018 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564dfc0a5be0 0x564dfc0b0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:10.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.042+0000 7fe8cdd9c700 1 -- 172.21.15.101:0/145117018 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564dfbe25de0 con 0x564dfc0a52d0 2023-07-18T03:15:10.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.042+0000 7fe8cdd9c700 1 --2- 172.21.15.101:0/145117018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfc0a52d0 0x564dfc0a56a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0011600 tx=0x7fe8c0009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db56b8b447993327 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:10.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.042+0000 7fe8be7fc700 1 --2- 172.21.15.101:0/106749649 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fe8b40a12c0 0x7fe8b40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:10.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.042+0000 7fe8b37fe700 1 -- 172.21.15.101:0/145117018 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c000e040 con 0x564dfc0a52d0 2023-07-18T03:15:10.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.042+0000 7fe8b37fe700 1 -- 172.21.15.101:0/145117018 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8c000f040 con 0x564dfc0a52d0 2023-07-18T03:15:10.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.042+0000 7fe8b37fe700 1 -- 172.21.15.101:0/145117018 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c00047a0 con 0x564dfc0a52d0 2023-07-18T03:15:10.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.042+0000 7fe8be7fc700 1 -- 172.21.15.101:0/106749649 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9549+0+0 (secure 0 0 0) 0x7fe8b8014070 con 0x7fe8c8100490 2023-07-18T03:15:10.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.042+0000 7fe8cd59b700 1 --2- 172.21.15.101:0/106749649 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fe8b40a12c0 0x7fe8b40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:10.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.043+0000 7fe8e30fab80 1 -- 172.21.15.101:0/145117018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfc0a52d0 msgr2=0x564dfc0a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:10.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.043+0000 7fe8e30fab80 1 --2- 172.21.15.101:0/145117018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfc0a52d0 0x564dfc0a56a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0011600 tx=0x7fe8c0009920 comp rx=0 tx=0).stop 2023-07-18T03:15:10.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.043+0000 7fe8cd59b700 1 --2- 172.21.15.101:0/106749649 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fe8b40a12c0 0x7fe8b40a3770 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4006010 tx=0x7fe8c400b260 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:10.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.043+0000 7fe8e30fab80 1 -- 172.21.15.101:0/145117018 shutdown_connections 2023-07-18T03:15:10.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.043+0000 7fe8e30fab80 1 --2- 172.21.15.101:0/145117018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfc0a52d0 0x564dfc0a56a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:10.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.043+0000 7fe8e30fab80 1 --2- 172.21.15.101:0/145117018 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564dfc0b0ad0 0x564dfc0b2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:10.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.043+0000 7fe8e30fab80 1 --2- 172.21.15.101:0/145117018 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564dfc0a5be0 0x564dfc0b0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:10.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.043+0000 7fe8e30fab80 1 -- 172.21.15.101:0/145117018 >> 172.21.15.101:0/145117018 conn(0x564dfc09bda0 msgr2=0x564dfc0a16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:10.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.043+0000 7fe8e30fab80 1 -- 172.21.15.101:0/145117018 shutdown_connections 2023-07-18T03:15:10.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.043+0000 7fe8e30fab80 1 -- 172.21.15.101:0/145117018 wait complete. 2023-07-18T03:15:10.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.044+0000 7fe8e30fab80 1 -- start start 2023-07-18T03:15:10.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.044+0000 7fe8e30fab80 1 Processor -- start 2023-07-18T03:15:10.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.045+0000 7fe8e30fab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564dfc0a52d0 0x564dfc0b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:10.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.045+0000 7fe8e30fab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564dfc0a5be0 0x564dfc0ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:10.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.045+0000 7fe8e30fab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfc0b0ad0 0x564dfc0add90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:10.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.045+0000 7fe8e30fab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dfbf97ac0 con 0x564dfc0a5be0 2023-07-18T03:15:10.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.045+0000 7fe8e30fab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dfbf977c0 con 0x564dfc0b0ad0 2023-07-18T03:15:10.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.045+0000 7fe8e30fab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dfbe7b920 con 0x564dfc0a52d0 2023-07-18T03:15:10.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.045+0000 7fe8e30fab80 10 client.0 _ll_register_callbacks cb 0x564dfbf95760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:15:10.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.045+0000 7fe8cd59b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564dfc0a52d0 0x564dfc0b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:10.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.045+0000 7fe8cd59b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564dfc0a52d0 0x564dfc0b04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:47098/0 (socket says 172.21.15.101:47098) 2023-07-18T03:15:10.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.045+0000 7fe8ccd9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564dfc0a5be0 0x564dfc0ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:10.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.045+0000 7fe8cd59b700 1 -- 172.21.15.101:0/1209887838 learned_addr learned my addr 172.21.15.101:0/1209887838 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:10.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.046+0000 7fe8cdd9c700 1 --2- 172.21.15.101:0/1209887838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfc0b0ad0 0x564dfc0add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:10.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.046+0000 7fe8cd59b700 1 -- 172.21.15.101:0/1209887838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfc0b0ad0 msgr2=0x564dfc0add90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:10.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.046+0000 7fe8cd59b700 1 --2- 172.21.15.101:0/1209887838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfc0b0ad0 0x564dfc0add90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:10.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.046+0000 7fe8cd59b700 1 -- 172.21.15.101:0/1209887838 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564dfc0a5be0 msgr2=0x564dfc0ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:10.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.046+0000 7fe8cd59b700 1 --2- 172.21.15.101:0/1209887838 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564dfc0a5be0 0x564dfc0ad730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:10.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.046+0000 7fe8cd59b700 1 -- 172.21.15.101:0/1209887838 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c00112b0 con 0x564dfc0a52d0 2023-07-18T03:15:10.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.046+0000 7fe8cdd9c700 1 --2- 172.21.15.101:0/1209887838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfc0b0ad0 0x564dfc0add90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:15:10.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.046+0000 7fe8ccd9a700 1 --2- 172.21.15.101:0/1209887838 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564dfc0a5be0 0x564dfc0ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:15:10.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.046+0000 7fe8cd59b700 1 --2- 172.21.15.101:0/1209887838 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564dfc0a52d0 0x564dfc0b04b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40047b0 tx=0x7fe8c4009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab37d4f2101e20c5 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:10.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.046+0000 7fe8b0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:15:10.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.046+0000 7fe8b0ff9700 1 -- 172.21.15.101:0/1209887838 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4026660 con 0x564dfc0a52d0 2023-07-18T03:15:10.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.047+0000 7fe8b0ff9700 1 -- 172.21.15.101:0/1209887838 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9549+0+0 (secure 0 0 0) 0x7fe8c40267c0 con 0x564dfc0a52d0 2023-07-18T03:15:10.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.047+0000 7fe8e30fab80 1 -- 172.21.15.101:0/1209887838 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564dfbf514f0 con 0x564dfc0a52d0 2023-07-18T03:15:10.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.047+0000 7fe8b0ff9700 1 -- 172.21.15.101:0/1209887838 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8c4026980 con 0x564dfc0a52d0 2023-07-18T03:15:10.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.047+0000 7fe8e30fab80 10 client.27187 fetch_fsmap learned FSMap version 29 2023-07-18T03:15:10.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.047+0000 7fe8e30fab80 1 -- 172.21.15.101:0/1209887838 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x564dfbe0ec60 con 0x564dfc0a52d0 2023-07-18T03:15:10.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.047+0000 7fe8b0ff9700 1 -- 172.21.15.101:0/1209887838 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564dfbf514f0 con 0x564dfc0a52d0 2023-07-18T03:15:10.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.047+0000 7fe8b0ff9700 1 -- 172.21.15.101:0/1209887838 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fe8c400fc10 con 0x564dfc0a52d0 2023-07-18T03:15:10.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.047+0000 7fe8e30fab80 10 client.27187 fetch_fsmap finished waiting for FSMap version 29 2023-07-18T03:15:10.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.047+0000 7fe8e30fab80 10 client.27187 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:15:10.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.048+0000 7fe8e30fab80 20 client.27187 populate_metadata read hostname 'smithi101' 2023-07-18T03:15:10.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.048+0000 7fe8e30fab80 1 --2- 172.21.15.101:0/1209887838 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x564dfc0c0260 0x564dfc0c2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:10.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.048+0000 7fe8e30fab80 4 client.27187 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:15:10.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.048+0000 7fe8e30fab80 1 -- 172.21.15.101:0/1209887838 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564dfbff0eb0 con 0x564dfc0c0260 2023-07-18T03:15:10.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.048+0000 7fe8ccd9a700 1 --2- 172.21.15.101:0/1209887838 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x564dfc0c0260 0x564dfc0c2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:10.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.048+0000 7fe8ccd9a700 1 --2- 172.21.15.101:0/1209887838 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x564dfc0c0260 0x564dfc0c2670 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=323d2db35412c738 server_cookie=a73007b5e7d7154b in_seq=0 out_seq=0 2023-07-18T03:15:10.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.048+0000 7fe8b0ff9700 10 client.27187 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:15:10.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.049+0000 7fe8b0ff9700 1 -- 172.21.15.101:0/1209887838 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564dfbff0eb0 con 0x564dfc0c0260 2023-07-18T03:15:10.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.049+0000 7fe8b0ff9700 10 client.27187 handle_command_reply: tid=0 2023-07-18T03:15:10.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.049+0000 7fe8e30fab80 1 client.27187 shutdown 2023-07-18T03:15:10.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.049+0000 7fe8e30fab80 1 -- 172.21.15.101:0/1209887838 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564dfc0a52d0 msgr2=0x564dfc0b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:10.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.049+0000 7fe8e30fab80 1 --2- 172.21.15.101:0/1209887838 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564dfc0a52d0 0x564dfc0b04b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40047b0 tx=0x7fe8c4009310 comp rx=0 tx=0).stop 2023-07-18T03:15:10.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.050+0000 7fe8e30fab80 1 -- 172.21.15.101:0/1209887838 shutdown_connections 2023-07-18T03:15:10.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.050+0000 7fe8e30fab80 1 --2- 172.21.15.101:0/1209887838 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x564dfc0c0260 0x564dfc0c2670 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:10.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.050+0000 7fe8e30fab80 1 --2- 172.21.15.101:0/1209887838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfc0b0ad0 0x564dfc0add90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:10.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.050+0000 7fe8e30fab80 1 --2- 172.21.15.101:0/1209887838 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564dfc0a52d0 0x564dfc0b04b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:10.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.050+0000 7fe8b0ff9700 0 client.27187 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:15:10.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.050+0000 7fe8e30fab80 1 --2- 172.21.15.101:0/1209887838 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564dfc0a5be0 0x564dfc0ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:10.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.050+0000 7fe8e30fab80 1 -- 172.21.15.101:0/1209887838 >> 172.21.15.101:0/1209887838 conn(0x564dfc09bda0 msgr2=0x564dfc0a1230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:10.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.050+0000 7fe8e30fab80 1 -- 172.21.15.101:0/1209887838 shutdown_connections 2023-07-18T03:15:10.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.050+0000 7fe8e30fab80 1 -- 172.21.15.101:0/1209887838 wait complete. 2023-07-18T03:15:10.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.050+0000 7fe8e30fab80 20 client.27187 trim_cache size 0 max 16384 2023-07-18T03:15:10.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.068+0000 7fe8e30fab80 1 Processor -- start 2023-07-18T03:15:10.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.068+0000 7fe8e30fab80 1 -- start start 2023-07-18T03:15:10.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.068+0000 7fe8e30fab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564dfc0a52d0 0x564dfc0b74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:10.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.068+0000 7fe8e30fab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564dfc0a5be0 0x564dfc0a8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:10.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.068+0000 7fe8e30fab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfc0b0ad0 0x564dfc0a8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:10.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.068+0000 7fe8e30fab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dfc0a0fc0 con 0x564dfc0a52d0 2023-07-18T03:15:10.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.068+0000 7fe8e30fab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dfc0a82f0 con 0x564dfc0b0ad0 2023-07-18T03:15:10.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.068+0000 7fe8e30fab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dfc0aece0 con 0x564dfc0a5be0 2023-07-18T03:15:10.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.068+0000 7fe8ccd9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564dfc0a5be0 0x564dfc0a8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:10.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.069+0000 7fe8ccd9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564dfc0a5be0 0x564dfc0a8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:47104/0 (socket says 172.21.15.101:47104) 2023-07-18T03:15:10.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.069+0000 7fe8cd59b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564dfc0a52d0 0x564dfc0b74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:10.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.069+0000 7fe8ccd9a700 1 -- 172.21.15.101:0/2401338198 learned_addr learned my addr 172.21.15.101:0/2401338198 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:10.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.069+0000 7fe8cdd9c700 1 --2- 172.21.15.101:0/2401338198 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfc0b0ad0 0x564dfc0a8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:10.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.069+0000 7fe8ccd9a700 1 -- 172.21.15.101:0/2401338198 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfc0b0ad0 msgr2=0x564dfc0a8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:10.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.069+0000 7fe8ccd9a700 1 --2- 172.21.15.101:0/2401338198 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfc0b0ad0 0x564dfc0a8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:10.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.069+0000 7fe8ccd9a700 1 -- 172.21.15.101:0/2401338198 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564dfc0a52d0 msgr2=0x564dfc0b74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:10.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.069+0000 7fe8ccd9a700 1 --2- 172.21.15.101:0/2401338198 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564dfc0a52d0 0x564dfc0b74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:10.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.069+0000 7fe8ccd9a700 1 -- 172.21.15.101:0/2401338198 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564dfbe4c250 con 0x564dfc0a5be0 2023-07-18T03:15:10.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.069+0000 7fe8cdd9c700 1 --2- 172.21.15.101:0/2401338198 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfc0b0ad0 0x564dfc0a8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:15:10.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.069+0000 7fe8ccd9a700 1 --2- 172.21.15.101:0/2401338198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564dfc0a5be0 0x564dfc0a8a30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe8b80cab90 tx=0x7fe8b80d0b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33f27cbf19ee84cb server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:10.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.069+0000 7fe8b3fff700 1 -- 172.21.15.101:0/2401338198 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8099ce0 con 0x564dfc0a5be0 2023-07-18T03:15:10.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.069+0000 7fe8b3fff700 1 -- 172.21.15.101:0/2401338198 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8b8099a50 con 0x564dfc0a5be0 2023-07-18T03:15:10.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.069+0000 7fe8b3fff700 1 -- 172.21.15.101:0/2401338198 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b80a23f0 con 0x564dfc0a5be0 2023-07-18T03:15:10.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.070+0000 7fe8e30fab80 1 -- 172.21.15.101:0/2401338198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564dfc0a5be0 msgr2=0x564dfc0a8a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:10.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.070+0000 7fe8e30fab80 1 --2- 172.21.15.101:0/2401338198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564dfc0a5be0 0x564dfc0a8a30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe8b80cab90 tx=0x7fe8b80d0b90 comp rx=0 tx=0).stop 2023-07-18T03:15:10.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.070+0000 7fe8e30fab80 1 -- 172.21.15.101:0/2401338198 shutdown_connections 2023-07-18T03:15:10.070 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.070+0000 7fe8e30fab80 1 --2- 172.21.15.101:0/2401338198 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfc0b0ad0 0x564dfc0a8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:10.070 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.070+0000 7fe8e30fab80 1 --2- 172.21.15.101:0/2401338198 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564dfc0a52d0 0x564dfc0b74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:10.070 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.070+0000 7fe8e30fab80 1 --2- 172.21.15.101:0/2401338198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564dfc0a5be0 0x564dfc0a8a30 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:10.070 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.070+0000 7fe8e30fab80 1 -- 172.21.15.101:0/2401338198 >> 172.21.15.101:0/2401338198 conn(0x564dfc09bda0 msgr2=0x564dfc0a08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:10.070 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.070+0000 7fe8e30fab80 1 -- 172.21.15.101:0/2401338198 shutdown_connections 2023-07-18T03:15:10.070 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.070+0000 7fe8e30fab80 1 -- 172.21.15.101:0/2401338198 wait complete. 2023-07-18T03:15:10.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.071+0000 7fe8e30fab80 1 -- start start 2023-07-18T03:15:10.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.071+0000 7fe8e30fab80 1 Processor -- start 2023-07-18T03:15:10.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.071+0000 7fe8e30fab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfc0a52d0 0x564dfc0b73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:10.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.071+0000 7fe8e30fab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564dfc0a5be0 0x564dfc0b5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:10.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.071+0000 7fe8e30fab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564dfc0b0ad0 0x564dfc0b5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:10.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.071+0000 7fe8e30fab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dfc0b60a0 con 0x564dfc0a5be0 2023-07-18T03:15:10.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.071+0000 7fe8e30fab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dfc0b61e0 con 0x564dfc0a52d0 2023-07-18T03:15:10.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.071+0000 7fe8e30fab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dfc0b63a0 con 0x564dfc0b0ad0 2023-07-18T03:15:10.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.071+0000 7fe8e30fab80 10 client.0 _ll_register_callbacks cb 0x564dfbf95760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:15:10.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.072+0000 7fe8cdd9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564dfc0b0ad0 0x564dfc0b5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:10.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.072+0000 7fe8cd59b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfc0a52d0 0x564dfc0b73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:10.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.072+0000 7fe8ccd9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564dfc0a5be0 0x564dfc0b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:10.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.072+0000 7fe8cd59b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfc0a52d0 0x564dfc0b73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:37420/0 (socket says 172.21.15.101:37420) 2023-07-18T03:15:10.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.072+0000 7fe8cdd9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564dfc0b0ad0 0x564dfc0b5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:47116/0 (socket says 172.21.15.101:47116) 2023-07-18T03:15:10.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.072+0000 7fe8cd59b700 1 -- 172.21.15.101:0/521789732 learned_addr learned my addr 172.21.15.101:0/521789732 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:10.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.072+0000 7fe8cd59b700 1 -- 172.21.15.101:0/521789732 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564dfc0b0ad0 msgr2=0x564dfc0b5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:10.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.072+0000 7fe8cd59b700 1 --2- 172.21.15.101:0/521789732 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564dfc0b0ad0 0x564dfc0b5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:10.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.072+0000 7fe8cd59b700 1 -- 172.21.15.101:0/521789732 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564dfc0a5be0 msgr2=0x564dfc0b5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:10.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.072+0000 7fe8cd59b700 1 --2- 172.21.15.101:0/521789732 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564dfc0a5be0 0x564dfc0b5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:10.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.072+0000 7fe8cd59b700 1 -- 172.21.15.101:0/521789732 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c81a4c10 con 0x564dfc0a52d0 2023-07-18T03:15:10.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.072+0000 7fe8ccd9a700 1 --2- 172.21.15.101:0/521789732 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564dfc0a5be0 0x564dfc0b5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:15:10.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.072+0000 7fe8cd59b700 1 --2- 172.21.15.101:0/521789732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfc0a52d0 0x564dfc0b73a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40047b0 tx=0x7fe8c4024b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd86b4a7d76593df server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:10.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.072+0000 7fe8b0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:15:10.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.072+0000 7fe8b0ff9700 1 -- 172.21.15.101:0/521789732 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4045070 con 0x564dfc0a52d0 2023-07-18T03:15:10.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.072+0000 7fe8b0ff9700 1 -- 172.21.15.101:0/521789732 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9549+0+0 (secure 0 0 0) 0x7fe8c40246a0 con 0x564dfc0a52d0 2023-07-18T03:15:10.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.072+0000 7fe8e30fab80 1 -- 172.21.15.101:0/521789732 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe8c000f040 con 0x564dfc0a52d0 2023-07-18T03:15:10.075 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.073+0000 7fe8b0ff9700 1 -- 172.21.15.101:0/521789732 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8c400f460 con 0x564dfc0a52d0 2023-07-18T03:15:10.075 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.073+0000 7fe8e30fab80 10 client.27158 fetch_fsmap learned FSMap version 29 2023-07-18T03:15:10.075 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.073+0000 7fe8e30fab80 1 -- 172.21.15.101:0/521789732 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x564dfbe242b0 con 0x564dfc0a52d0 2023-07-18T03:15:10.075 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.073+0000 7fe8b0ff9700 1 -- 172.21.15.101:0/521789732 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8c000f040 con 0x564dfc0a52d0 2023-07-18T03:15:10.075 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.073+0000 7fe8b0ff9700 1 -- 172.21.15.101:0/521789732 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fe8c4048020 con 0x564dfc0a52d0 2023-07-18T03:15:10.075 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.073+0000 7fe8e30fab80 10 client.27158 fetch_fsmap finished waiting for FSMap version 29 2023-07-18T03:15:10.076 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.073+0000 7fe8e30fab80 10 client.27158 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:15:10.076 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.073+0000 7fe8e30fab80 20 client.27158 populate_metadata read hostname 'smithi101' 2023-07-18T03:15:10.076 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.073+0000 7fe8e30fab80 1 --2- 172.21.15.101:0/521789732 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x564dfc1151c0 0x564dfc1175d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:10.076 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.073+0000 7fe8e30fab80 4 client.27158 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:15:10.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.073+0000 7fe8e30fab80 1 -- 172.21.15.101:0/521789732 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564dfbfb3050 con 0x564dfc1151c0 2023-07-18T03:15:10.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.073+0000 7fe8ccd9a700 1 --2- 172.21.15.101:0/521789732 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x564dfc1151c0 0x564dfc1175d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:10.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.074+0000 7fe8ccd9a700 1 --2- 172.21.15.101:0/521789732 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x564dfc1151c0 0x564dfc1175d0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfe663225d56e68 server_cookie=c13172df2616a81e in_seq=0 out_seq=0 2023-07-18T03:15:10.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.074+0000 7fe8b0ff9700 10 client.27158 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:15:10.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.074+0000 7fe8b0ff9700 1 -- 172.21.15.101:0/521789732 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564dfbfb3050 con 0x564dfc1151c0 2023-07-18T03:15:10.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.074+0000 7fe8b0ff9700 10 client.27158 handle_command_reply: tid=0 2023-07-18T03:15:10.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.074+0000 7fe8e30fab80 1 client.27158 shutdown 2023-07-18T03:15:10.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.074+0000 7fe8e30fab80 1 -- 172.21.15.101:0/521789732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfc0a52d0 msgr2=0x564dfc0b73a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:10.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.074+0000 7fe8e30fab80 1 --2- 172.21.15.101:0/521789732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfc0a52d0 0x564dfc0b73a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40047b0 tx=0x7fe8c4024b10 comp rx=0 tx=0).stop 2023-07-18T03:15:10.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.074+0000 7fe8e30fab80 1 -- 172.21.15.101:0/521789732 shutdown_connections 2023-07-18T03:15:10.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.074+0000 7fe8e30fab80 1 --2- 172.21.15.101:0/521789732 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x564dfc1151c0 0x564dfc1175d0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:10.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.074+0000 7fe8e30fab80 1 --2- 172.21.15.101:0/521789732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dfc0a52d0 0x564dfc0b73a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:10.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.074+0000 7fe8e30fab80 1 --2- 172.21.15.101:0/521789732 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564dfc0b0ad0 0x564dfc0b5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:10.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.074+0000 7fe8e30fab80 1 --2- 172.21.15.101:0/521789732 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564dfc0a5be0 0x564dfc0b5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:10.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.074+0000 7fe8b0ff9700 0 client.27158 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:15:10.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.074+0000 7fe8e30fab80 1 -- 172.21.15.101:0/521789732 >> 172.21.15.101:0/521789732 conn(0x564dfc09bda0 msgr2=0x564dfc0a8a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:10.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.074+0000 7fe8e30fab80 1 -- 172.21.15.101:0/521789732 shutdown_connections 2023-07-18T03:15:10.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.075+0000 7fe8e30fab80 1 -- 172.21.15.101:0/521789732 wait complete. 2023-07-18T03:15:10.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.075+0000 7fe8e30fab80 20 client.27158 trim_cache size 0 max 16384 2023-07-18T03:15:10.080 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:15:10.080 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:15:10.080 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:15:10.080 INFO:teuthology.orchestra.run.smithi101.stdout: "23994a27-b341-4986-8822-5a01f3ba7acf": { 2023-07-18T03:15:10.080 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:15:10.080 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "23994a27-b341-4986-8822-5a01f3ba7acf", 2023-07-18T03:15:10.081 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:15:10.081 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:15:10.081 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:15:10.081 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:15:10.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.075+0000 7fe8cf7ff700 1 -- 172.21.15.101:0/106749649 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fe8b40a12c0 msgr2=0x7fe8b40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:10.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.075+0000 7fe8cf7ff700 1 --2- 172.21.15.101:0/106749649 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fe8b40a12c0 0x7fe8b40a3770 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4006010 tx=0x7fe8c400b260 comp rx=0 tx=0).stop 2023-07-18T03:15:10.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.075+0000 7fe8cf7ff700 1 -- 172.21.15.101:0/106749649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8c8100490 msgr2=0x7fe8c8072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:10.083 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.075+0000 7fe8cf7ff700 1 --2- 172.21.15.101:0/106749649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8c8100490 0x7fe8c8072d50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe8b800e9f0 tx=0x7fe8b800ed00 comp rx=0 tx=0).stop 2023-07-18T03:15:10.083 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.075+0000 7fe8cf7ff700 1 -- 172.21.15.101:0/106749649 shutdown_connections 2023-07-18T03:15:10.083 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.075+0000 7fe8cf7ff700 1 --2- 172.21.15.101:0/106749649 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8c8105af0 0x7fe8c81a3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:10.083 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.075+0000 7fe8cf7ff700 1 --2- 172.21.15.101:0/106749649 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe8c80ffae0 0x7fe8c8072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:10.083 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.075+0000 7fe8cf7ff700 1 --2- 172.21.15.101:0/106749649 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fe8b40a12c0 0x7fe8b40a3770 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:10.083 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.075+0000 7fe8cf7ff700 1 --2- 172.21.15.101:0/106749649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8c8100490 0x7fe8c8072d50 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:10.083 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.075+0000 7fe8cf7ff700 1 -- 172.21.15.101:0/106749649 >> 172.21.15.101:0/106749649 conn(0x7fe8c80fb5e0 msgr2=0x7fe8c80fbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:10.084 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.075+0000 7fe8cf7ff700 1 -- 172.21.15.101:0/106749649 shutdown_connections 2023-07-18T03:15:10.084 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:10.075+0000 7fe8cf7ff700 1 -- 172.21.15.101:0/106749649 wait complete. 2023-07-18T03:15:10.087 INFO:tasks.cephfs.filesystem:scrub status for tag:23994a27-b341-4986-8822-5a01f3ba7acf - {'path': '/', 'tag': '23994a27-b341-4986-8822-5a01f3ba7acf', 'options': 'recursive,force'} 2023-07-18T03:15:10.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:10 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:15:10.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:10 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:15:10.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:10 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:15:10.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:10 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:15:10.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:10 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:15:10.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:10 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:15:10.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:10 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:15:10.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:10 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:15:10.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:10 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:15:10.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:10 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:15:10.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:10 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:15:10.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:10 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:15:10.592 INFO:tasks.workunit.client.0.smithi101.stdout: 1 533123 6.97 MB/sec execute 275 sec latency 24.224 ms 2023-07-18T03:15:11.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:11 smithi155 ceph-mon[128885]: pgmap v234: 129 pgs: 129 active+clean; 375 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 172 op/s 2023-07-18T03:15:11.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:11 smithi155 ceph-mon[128885]: scrub queued for path: / 2023-07-18T03:15:11.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:11 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:15:11.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:11 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:15:11.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:11 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:15:11.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:11 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:15:11.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:11 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:15:11.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:11 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:15:11.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:11 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:15:11.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:11 smithi179 ceph-mon[129229]: pgmap v234: 129 pgs: 129 active+clean; 375 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 172 op/s 2023-07-18T03:15:11.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:11 smithi179 ceph-mon[129229]: scrub queued for path: / 2023-07-18T03:15:11.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:11 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:15:11.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:11 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:15:11.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:11 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:15:11.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:11 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:15:11.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:11 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:15:11.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:11 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:15:11.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:11 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:15:11.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:11 smithi101 ceph-mon[126379]: pgmap v234: 129 pgs: 129 active+clean; 375 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 172 op/s 2023-07-18T03:15:11.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:11 smithi101 ceph-mon[126379]: scrub queued for path: / 2023-07-18T03:15:11.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:11 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:15:11.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:11 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:15:11.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:11 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:15:11.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:11 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:15:11.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:11 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:15:11.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:11 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:15:11.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:11 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:15:11.592 INFO:tasks.workunit.client.0.smithi101.stdout: 1 534139 6.95 MB/sec execute 276 sec latency 13.878 ms 2023-07-18T03:15:12.592 INFO:tasks.workunit.client.0.smithi101.stdout: 1 534579 6.94 MB/sec execute 277 sec latency 531.183 ms 2023-07-18T03:15:13.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:13 smithi155 ceph-mon[128885]: pgmap v235: 129 pgs: 129 active+clean; 375 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 172 op/s 2023-07-18T03:15:13.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:13 smithi179 ceph-mon[129229]: pgmap v235: 129 pgs: 129 active+clean; 375 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 172 op/s 2023-07-18T03:15:13.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:13 smithi101 ceph-mon[126379]: pgmap v235: 129 pgs: 129 active+clean; 375 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 172 op/s 2023-07-18T03:15:13.592 INFO:tasks.workunit.client.0.smithi101.stdout: 1 534579 6.91 MB/sec execute 278 sec latency 1531.338 ms 2023-07-18T03:15:14.592 INFO:tasks.workunit.client.0.smithi101.stdout: 1 534781 6.89 MB/sec execute 279 sec latency 2409.943 ms 2023-07-18T03:15:15.200 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:14 smithi179 ceph-mon[129229]: pgmap v236: 129 pgs: 129 active+clean; 375 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 172 op/s 2023-07-18T03:15:15.201 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:14 smithi179 ceph-mon[129229]: osdmap e96: 12 total, 12 up, 12 in 2023-07-18T03:15:15.237 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:14 smithi101 ceph-mon[126379]: pgmap v236: 129 pgs: 129 active+clean; 375 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 172 op/s 2023-07-18T03:15:15.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:14 smithi101 ceph-mon[126379]: osdmap e96: 12 total, 12 up, 12 in 2023-07-18T03:15:15.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:14 smithi155 ceph-mon[128885]: pgmap v236: 129 pgs: 129 active+clean; 375 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 172 op/s 2023-07-18T03:15:15.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:14 smithi155 ceph-mon[128885]: osdmap e96: 12 total, 12 up, 12 in 2023-07-18T03:15:15.593 INFO:tasks.workunit.client.0.smithi101.stdout: 1 536665 6.91 MB/sec execute 280 sec latency 36.309 ms 2023-07-18T03:15:16.200 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:15 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:15:16.200 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:15 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:15:16.200 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:15 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:15:16.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:15 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:15:16.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:15 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:15:16.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:15 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:15:16.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:15 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:15:16.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:15 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:15:16.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:15 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:15:16.593 INFO:tasks.workunit.client.0.smithi101.stdout: 1 537652 6.89 MB/sec execute 281 sec latency 13.799 ms 2023-07-18T03:15:17.200 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:16 smithi179 ceph-mon[129229]: pgmap v238: 129 pgs: 129 active+clean; 380 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 136 op/s 2023-07-18T03:15:17.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:16 smithi101 ceph-mon[126379]: pgmap v238: 129 pgs: 129 active+clean; 380 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 136 op/s 2023-07-18T03:15:17.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:16 smithi155 ceph-mon[128885]: pgmap v238: 129 pgs: 129 active+clean; 380 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 136 op/s 2023-07-18T03:15:17.593 INFO:tasks.workunit.client.0.smithi101.stdout: 1 538122 6.88 MB/sec execute 282 sec latency 518.125 ms 2023-07-18T03:15:18.593 INFO:tasks.workunit.client.0.smithi101.stdout: 1 538122 6.86 MB/sec execute 283 sec latency 1518.256 ms 2023-07-18T03:15:19.200 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:18 smithi179 ceph-mon[129229]: pgmap v239: 129 pgs: 129 active+clean; 380 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 131 op/s 2023-07-18T03:15:19.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:18 smithi101 ceph-mon[126379]: pgmap v239: 129 pgs: 129 active+clean; 380 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 131 op/s 2023-07-18T03:15:19.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:18 smithi155 ceph-mon[128885]: pgmap v239: 129 pgs: 129 active+clean; 380 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 131 op/s 2023-07-18T03:15:19.593 INFO:tasks.workunit.client.0.smithi101.stdout: 1 538410 6.83 MB/sec execute 284 sec latency 2358.536 ms 2023-07-18T03:15:20.593 INFO:tasks.workunit.client.0.smithi101.stdout: 1 540203 6.86 MB/sec execute 285 sec latency 19.261 ms 2023-07-18T03:15:21.200 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:20 smithi179 ceph-mon[129229]: pgmap v240: 129 pgs: 129 active+clean; 386 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 131 op/s 2023-07-18T03:15:21.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:20 smithi101 ceph-mon[126379]: pgmap v240: 129 pgs: 129 active+clean; 386 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 131 op/s 2023-07-18T03:15:21.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:20 smithi155 ceph-mon[128885]: pgmap v240: 129 pgs: 129 active+clean; 386 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 131 op/s 2023-07-18T03:15:21.593 INFO:tasks.workunit.client.0.smithi101.stdout: 1 541201 6.84 MB/sec execute 286 sec latency 15.886 ms 2023-07-18T03:15:22.593 INFO:tasks.workunit.client.0.smithi101.stdout: 1 541671 6.83 MB/sec execute 287 sec latency 498.706 ms 2023-07-18T03:15:23.200 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:22 smithi179 ceph-mon[129229]: pgmap v241: 129 pgs: 129 active+clean; 386 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 131 op/s 2023-07-18T03:15:23.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:22 smithi101 ceph-mon[126379]: pgmap v241: 129 pgs: 129 active+clean; 386 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 131 op/s 2023-07-18T03:15:23.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:22 smithi155 ceph-mon[128885]: pgmap v241: 129 pgs: 129 active+clean; 386 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 131 op/s 2023-07-18T03:15:23.593 INFO:tasks.workunit.client.0.smithi101.stdout: 1 541671 6.80 MB/sec execute 288 sec latency 1498.820 ms 2023-07-18T03:15:24.594 INFO:tasks.workunit.client.0.smithi101.stdout: 1 541873 6.78 MB/sec execute 289 sec latency 2347.004 ms 2023-07-18T03:15:25.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:24 smithi101 ceph-mon[126379]: pgmap v242: 129 pgs: 129 active+clean; 386 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 131 op/s 2023-07-18T03:15:25.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:24 smithi155 ceph-mon[128885]: pgmap v242: 129 pgs: 129 active+clean; 386 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 131 op/s 2023-07-18T03:15:25.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:24 smithi179 ceph-mon[129229]: pgmap v242: 129 pgs: 129 active+clean; 386 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 131 op/s 2023-07-18T03:15:25.594 INFO:tasks.workunit.client.0.smithi101.stdout: 1 543644 6.80 MB/sec execute 290 sec latency 95.221 ms 2023-07-18T03:15:26.594 INFO:tasks.workunit.client.0.smithi101.stdout: 1 544589 6.79 MB/sec execute 291 sec latency 16.809 ms 2023-07-18T03:15:27.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:26 smithi101 ceph-mon[126379]: pgmap v243: 129 pgs: 129 active+clean; 390 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 164 op/s 2023-07-18T03:15:27.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:26 smithi155 ceph-mon[128885]: pgmap v243: 129 pgs: 129 active+clean; 390 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 164 op/s 2023-07-18T03:15:27.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:26 smithi179 ceph-mon[129229]: pgmap v243: 129 pgs: 129 active+clean; 390 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 164 op/s 2023-07-18T03:15:27.594 INFO:tasks.workunit.client.0.smithi101.stdout: 1 545212 6.78 MB/sec execute 292 sec latency 382.281 ms 2023-07-18T03:15:28.594 INFO:tasks.workunit.client.0.smithi101.stdout: 1 545212 6.75 MB/sec execute 293 sec latency 1382.359 ms 2023-07-18T03:15:29.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:28 smithi101 ceph-mon[126379]: pgmap v244: 129 pgs: 129 active+clean; 390 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 106 op/s 2023-07-18T03:15:29.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:28 smithi155 ceph-mon[128885]: pgmap v244: 129 pgs: 129 active+clean; 390 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 106 op/s 2023-07-18T03:15:29.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:28 smithi179 ceph-mon[129229]: pgmap v244: 129 pgs: 129 active+clean; 390 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 106 op/s 2023-07-18T03:15:29.594 INFO:tasks.workunit.client.0.smithi101.stdout: 1 545612 6.73 MB/sec execute 294 sec latency 2214.362 ms 2023-07-18T03:15:30.594 INFO:tasks.workunit.client.0.smithi101.stdout: 1 547354 6.75 MB/sec execute 295 sec latency 31.675 ms 2023-07-18T03:15:31.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:30 smithi101 ceph-mon[126379]: pgmap v245: 129 pgs: 129 active+clean; 392 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 155 op/s 2023-07-18T03:15:31.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:30 smithi155 ceph-mon[128885]: pgmap v245: 129 pgs: 129 active+clean; 392 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 155 op/s 2023-07-18T03:15:31.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:30 smithi179 ceph-mon[129229]: pgmap v245: 129 pgs: 129 active+clean; 392 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 155 op/s 2023-07-18T03:15:31.594 INFO:tasks.workunit.client.0.smithi101.stdout: 1 548523 6.75 MB/sec execute 296 sec latency 17.209 ms 2023-07-18T03:15:32.594 INFO:tasks.workunit.client.0.smithi101.stdout: 1 548762 6.73 MB/sec execute 297 sec latency 820.549 ms 2023-07-18T03:15:33.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:32 smithi101 ceph-mon[126379]: pgmap v246: 129 pgs: 129 active+clean; 392 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 102 op/s 2023-07-18T03:15:33.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:32 smithi155 ceph-mon[128885]: pgmap v246: 129 pgs: 129 active+clean; 392 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 102 op/s 2023-07-18T03:15:33.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:32 smithi179 ceph-mon[129229]: pgmap v246: 129 pgs: 129 active+clean; 392 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 102 op/s 2023-07-18T03:15:33.594 INFO:tasks.workunit.client.0.smithi101.stdout: 1 548762 6.70 MB/sec execute 298 sec latency 1820.685 ms 2023-07-18T03:15:34.594 INFO:tasks.workunit.client.0.smithi101.stdout: 1 549161 6.68 MB/sec execute 299 sec latency 2655.007 ms 2023-07-18T03:15:35.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:34 smithi101 ceph-mon[126379]: pgmap v247: 129 pgs: 129 active+clean; 392 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 102 op/s 2023-07-18T03:15:35.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:34 smithi155 ceph-mon[128885]: pgmap v247: 129 pgs: 129 active+clean; 392 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 102 op/s 2023-07-18T03:15:35.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:34 smithi179 ceph-mon[129229]: pgmap v247: 129 pgs: 129 active+clean; 392 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 102 op/s 2023-07-18T03:15:35.594 INFO:tasks.workunit.client.0.smithi101.stdout: 1 550886 6.70 MB/sec execute 300 sec latency 20.292 ms 2023-07-18T03:15:36.594 INFO:tasks.workunit.client.0.smithi101.stdout: 1 552002 6.70 MB/sec execute 301 sec latency 17.091 ms 2023-07-18T03:15:37.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:36 smithi101 ceph-mon[126379]: pgmap v248: 129 pgs: 129 active+clean; 394 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 150 op/s 2023-07-18T03:15:37.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:36 smithi155 ceph-mon[128885]: pgmap v248: 129 pgs: 129 active+clean; 394 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 150 op/s 2023-07-18T03:15:37.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:36 smithi179 ceph-mon[129229]: pgmap v248: 129 pgs: 129 active+clean; 394 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 150 op/s 2023-07-18T03:15:37.595 INFO:tasks.workunit.client.0.smithi101.stdout: 1 552305 6.68 MB/sec execute 302 sec latency 517.874 ms 2023-07-18T03:15:38.595 INFO:tasks.workunit.client.0.smithi101.stdout: 1 552305 6.66 MB/sec execute 303 sec latency 1517.941 ms 2023-07-18T03:15:39.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:38 smithi101 ceph-mon[126379]: pgmap v249: 129 pgs: 129 active+clean; 394 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1009 KiB/s wr, 99 op/s 2023-07-18T03:15:39.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:38 smithi155 ceph-mon[128885]: pgmap v249: 129 pgs: 129 active+clean; 394 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1009 KiB/s wr, 99 op/s 2023-07-18T03:15:39.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:38 smithi179 ceph-mon[129229]: pgmap v249: 129 pgs: 129 active+clean; 394 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1009 KiB/s wr, 99 op/s 2023-07-18T03:15:39.595 INFO:tasks.workunit.client.0.smithi101.stdout: 1 552484 6.63 MB/sec execute 304 sec latency 2349.335 ms 2023-07-18T03:15:40.088 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:15:40.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.318+0000 7f2cb0fff700 1 -- 172.21.15.101:0/786230716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cac0ffae0 msgr2=0x7f2cac0fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:40.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.318+0000 7f2cb0fff700 1 --2- 172.21.15.101:0/786230716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cac0ffae0 0x7f2cac0fff50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0009990 tx=0x7f2ca0009ca0 comp rx=0 tx=0).stop 2023-07-18T03:15:40.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.318+0000 7f2cb0fff700 1 -- 172.21.15.101:0/786230716 shutdown_connections 2023-07-18T03:15:40.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.318+0000 7f2cb0fff700 1 --2- 172.21.15.101:0/786230716 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cac100490 0x7f2cac10c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:40.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.318+0000 7f2cb0fff700 1 --2- 172.21.15.101:0/786230716 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2cac105af0 0x7f2cac105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:40.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.318+0000 7f2cb0fff700 1 --2- 172.21.15.101:0/786230716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cac0ffae0 0x7f2cac0fff50 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:40.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.318+0000 7f2cb0fff700 1 -- 172.21.15.101:0/786230716 >> 172.21.15.101:0/786230716 conn(0x7f2cac0fb5e0 msgr2=0x7f2cac0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:40.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.318+0000 7f2cb0fff700 1 -- 172.21.15.101:0/786230716 shutdown_connections 2023-07-18T03:15:40.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.319+0000 7f2cb0fff700 1 -- 172.21.15.101:0/786230716 wait complete. 2023-07-18T03:15:40.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.319+0000 7f2cb0fff700 1 Processor -- start 2023-07-18T03:15:40.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.320+0000 7f2cb0fff700 1 -- start start 2023-07-18T03:15:40.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.320+0000 7f2cb0fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2cac0ffae0 0x7f2cac197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:40.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.320+0000 7f2cb0fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cac100490 0x7f2cac198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:40.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.320+0000 7f2cb0fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cac105af0 0x7f2cac19c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:40.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.320+0000 7f2cb0fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac19cde0 con 0x7f2cac0ffae0 2023-07-18T03:15:40.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.320+0000 7f2cb0fff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac19cf50 con 0x7f2cac105af0 2023-07-18T03:15:40.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.320+0000 7f2cb0fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac19d110 con 0x7f2cac100490 2023-07-18T03:15:40.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.320+0000 7f2caa59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2cac0ffae0 0x7f2cac197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:40.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.320+0000 7f2caa59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2cac0ffae0 0x7f2cac197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:43812/0 (socket says 172.21.15.101:43812) 2023-07-18T03:15:40.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.320+0000 7f2caa59c700 1 -- 172.21.15.101:0/1417215693 learned_addr learned my addr 172.21.15.101:0/1417215693 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:40.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.320+0000 7f2ca9d9b700 1 --2- 172.21.15.101:0/1417215693 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cac100490 0x7f2cac198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:40.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.320+0000 7f2caad9d700 1 --2- 172.21.15.101:0/1417215693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cac105af0 0x7f2cac19c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:40.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.321+0000 7f2caa59c700 1 -- 172.21.15.101:0/1417215693 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cac100490 msgr2=0x7f2cac198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:40.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.321+0000 7f2caa59c700 1 --2- 172.21.15.101:0/1417215693 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cac100490 0x7f2cac198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:40.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.321+0000 7f2caa59c700 1 -- 172.21.15.101:0/1417215693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cac105af0 msgr2=0x7f2cac19c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:40.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.321+0000 7f2caa59c700 1 --2- 172.21.15.101:0/1417215693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cac105af0 0x7f2cac19c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:40.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.321+0000 7f2caa59c700 1 -- 172.21.15.101:0/1417215693 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c9c009520 con 0x7f2cac0ffae0 2023-07-18T03:15:40.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.321+0000 7f2caa59c700 1 --2- 172.21.15.101:0/1417215693 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2cac0ffae0 0x7f2cac197e00 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c00eb30 tx=0x7f2c9c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:40.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.321+0000 7f2c9b7fe700 1 -- 172.21.15.101:0/1417215693 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c009880 con 0x7f2cac0ffae0 2023-07-18T03:15:40.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.321+0000 7f2c9b7fe700 1 -- 172.21.15.101:0/1417215693 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c9c0044e0 con 0x7f2cac0ffae0 2023-07-18T03:15:40.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.321+0000 7f2cb0fff700 1 -- 172.21.15.101:0/1417215693 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ca00095f0 con 0x7f2cac0ffae0 2023-07-18T03:15:40.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.321+0000 7f2c9b7fe700 1 -- 172.21.15.101:0/1417215693 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c003d80 con 0x7f2cac0ffae0 2023-07-18T03:15:40.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.321+0000 7f2cb0fff700 1 -- 172.21.15.101:0/1417215693 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cac1a4a50 con 0x7f2cac0ffae0 2023-07-18T03:15:40.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.324+0000 7f2c9b7fe700 1 -- 172.21.15.101:0/1417215693 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f2c9c003f20 con 0x7f2cac0ffae0 2023-07-18T03:15:40.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.326+0000 7f2cc4861b80 1 Processor -- start 2023-07-18T03:15:40.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.326+0000 7f2c9b7fe700 1 --2- 172.21.15.101:0/1417215693 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2c940a12c0 0x7f2c940a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:40.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.326+0000 7f2c9b7fe700 1 -- 172.21.15.101:0/1417215693 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f2c9c014070 con 0x7f2cac0ffae0 2023-07-18T03:15:40.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.326+0000 7f2cc4861b80 1 -- start start 2023-07-18T03:15:40.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.326+0000 7f2caad9d700 1 --2- 172.21.15.101:0/1417215693 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2c940a12c0 0x7f2c940a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:40.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.326+0000 7f2cc4861b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559fd74d62d0 0x559fd74d66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:40.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.326+0000 7f2cc4861b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559fd74d6be0 0x559fd74e1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:40.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.326+0000 7f2cc4861b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559fd74e1ad0 0x559fd74e3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:40.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.326+0000 7f2cc4861b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fd73dccb0 con 0x559fd74d62d0 2023-07-18T03:15:40.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.326+0000 7f2cc4861b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fd742fa40 con 0x559fd74d6be0 2023-07-18T03:15:40.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.326+0000 7f2cc4861b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fd7428ed0 con 0x559fd74e1ad0 2023-07-18T03:15:40.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.327+0000 7f2ca9d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559fd74d6be0 0x559fd74e1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:40.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.327+0000 7f2caa59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559fd74d62d0 0x559fd74d66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:40.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.327+0000 7f2ca9d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559fd74d6be0 0x559fd74e1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:35358/0 (socket says 172.21.15.101:35358) 2023-07-18T03:15:40.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.327+0000 7f2ca9d9b700 1 -- 172.21.15.101:0/4012695750 learned_addr learned my addr 172.21.15.101:0/4012695750 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:40.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.327+0000 7f2caad9d700 1 --2- 172.21.15.101:0/1417215693 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2c940a12c0 0x7f2c940a3770 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2ca400f540 tx=0x7f2ca400b040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:40.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.327+0000 7f2caad9d700 1 --2- 172.21.15.101:0/4012695750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559fd74e1ad0 0x559fd74e3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:40.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.327+0000 7f2ca9d9b700 1 -- 172.21.15.101:0/4012695750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559fd74e1ad0 msgr2=0x559fd74e3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:40.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.327+0000 7f2ca9d9b700 1 --2- 172.21.15.101:0/4012695750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559fd74e1ad0 0x559fd74e3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:40.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.327+0000 7f2ca9d9b700 1 -- 172.21.15.101:0/4012695750 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559fd74d62d0 msgr2=0x559fd74d66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:40.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.327+0000 7f2ca9d9b700 1 --2- 172.21.15.101:0/4012695750 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559fd74d62d0 0x559fd74d66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:40.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.327+0000 7f2ca9d9b700 1 -- 172.21.15.101:0/4012695750 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559fd7256de0 con 0x559fd74d6be0 2023-07-18T03:15:40.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.327+0000 7f2caad9d700 1 --2- 172.21.15.101:0/4012695750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559fd74e1ad0 0x559fd74e3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:15:40.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.327+0000 7f2ca9d9b700 1 --2- 172.21.15.101:0/4012695750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559fd74d6be0 0x559fd74e1590 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0015000 tx=0x7f2ca0005ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51f2d1577151a473 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:40.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.328+0000 7f2c98ff9700 1 -- 172.21.15.101:0/4012695750 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca001d070 con 0x559fd74d6be0 2023-07-18T03:15:40.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.328+0000 7f2c98ff9700 1 -- 172.21.15.101:0/4012695750 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ca001e040 con 0x559fd74d6be0 2023-07-18T03:15:40.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.328+0000 7f2c98ff9700 1 -- 172.21.15.101:0/4012695750 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca00047f0 con 0x559fd74d6be0 2023-07-18T03:15:40.328 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.328+0000 7f2cc4861b80 1 -- 172.21.15.101:0/4012695750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559fd74d6be0 msgr2=0x559fd74e1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:40.328 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.328+0000 7f2cc4861b80 1 --2- 172.21.15.101:0/4012695750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559fd74d6be0 0x559fd74e1590 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0015000 tx=0x7f2ca0005ea0 comp rx=0 tx=0).stop 2023-07-18T03:15:40.328 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.329+0000 7f2cc4861b80 1 -- 172.21.15.101:0/4012695750 shutdown_connections 2023-07-18T03:15:40.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.329+0000 7f2cc4861b80 1 --2- 172.21.15.101:0/4012695750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559fd74e1ad0 0x559fd74e3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:40.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.329+0000 7f2cc4861b80 1 --2- 172.21.15.101:0/4012695750 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559fd74d62d0 0x559fd74d66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:40.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.329+0000 7f2cc4861b80 1 --2- 172.21.15.101:0/4012695750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559fd74d6be0 0x559fd74e1590 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:40.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.329+0000 7f2cc4861b80 1 -- 172.21.15.101:0/4012695750 >> 172.21.15.101:0/4012695750 conn(0x559fd74ccda0 msgr2=0x559fd74d26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:40.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.329+0000 7f2cc4861b80 1 -- 172.21.15.101:0/4012695750 shutdown_connections 2023-07-18T03:15:40.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.329+0000 7f2cc4861b80 1 -- 172.21.15.101:0/4012695750 wait complete. 2023-07-18T03:15:40.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.329+0000 7f2cc4861b80 1 -- start start 2023-07-18T03:15:40.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.329+0000 7f2cc4861b80 1 Processor -- start 2023-07-18T03:15:40.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.330+0000 7f2cc4861b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559fd74d62d0 0x559fd74e1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:40.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.330+0000 7f2cc4861b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559fd74d6be0 0x559fd74de7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:40.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.330+0000 7f2cc4861b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559fd74e1ad0 0x559fd74deee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:40.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.330+0000 7f2cc4861b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fd73c8ac0 con 0x559fd74d6be0 2023-07-18T03:15:40.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.330+0000 7f2cc4861b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fd73c87c0 con 0x559fd74d62d0 2023-07-18T03:15:40.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.330+0000 7f2cc4861b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fd72ac920 con 0x559fd74e1ad0 2023-07-18T03:15:40.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.330+0000 7f2cc4861b80 10 client.0 _ll_register_callbacks cb 0x559fd73c6760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:15:40.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.330+0000 7f2caad9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559fd74e1ad0 0x559fd74deee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:40.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.330+0000 7f2caad9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559fd74e1ad0 0x559fd74deee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:51072/0 (socket says 172.21.15.101:51072) 2023-07-18T03:15:40.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.330+0000 7f2caad9d700 1 -- 172.21.15.101:0/1536117979 learned_addr learned my addr 172.21.15.101:0/1536117979 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:40.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.330+0000 7f2ca9d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559fd74d6be0 0x559fd74de7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:40.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.330+0000 7f2caa59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559fd74d62d0 0x559fd74e1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:40.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.331+0000 7f2caad9d700 1 -- 172.21.15.101:0/1536117979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559fd74d62d0 msgr2=0x559fd74e1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:40.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.331+0000 7f2caad9d700 1 --2- 172.21.15.101:0/1536117979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559fd74d62d0 0x559fd74e1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:40.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.331+0000 7f2caad9d700 1 -- 172.21.15.101:0/1536117979 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559fd74d6be0 msgr2=0x559fd74de7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:40.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.331+0000 7f2caad9d700 1 --2- 172.21.15.101:0/1536117979 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559fd74d6be0 0x559fd74de7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:40.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.331+0000 7f2caad9d700 1 -- 172.21.15.101:0/1536117979 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cac10d220 con 0x559fd74e1ad0 2023-07-18T03:15:40.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.331+0000 7f2caa59c700 1 --2- 172.21.15.101:0/1536117979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559fd74d62d0 0x559fd74e1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:15:40.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.331+0000 7f2caad9d700 1 --2- 172.21.15.101:0/1536117979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559fd74e1ad0 0x559fd74deee0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4008b40 tx=0x7f2ca4004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19051d8c5999cb6e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:40.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.331+0000 7f2c91ffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:15:40.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.331+0000 7f2c91ffb700 1 -- 172.21.15.101:0/1536117979 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca401c040 con 0x559fd74e1ad0 2023-07-18T03:15:40.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.331+0000 7f2cc4861b80 1 -- 172.21.15.101:0/1536117979 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559fd73824f0 con 0x559fd74e1ad0 2023-07-18T03:15:40.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.331+0000 7f2c91ffb700 1 -- 172.21.15.101:0/1536117979 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f2ca4017ce0 con 0x559fd74e1ad0 2023-07-18T03:15:40.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.331+0000 7f2c91ffb700 1 -- 172.21.15.101:0/1536117979 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ca400e040 con 0x559fd74e1ad0 2023-07-18T03:15:40.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.331+0000 7f2cc4861b80 10 client.27205 fetch_fsmap learned FSMap version 29 2023-07-18T03:15:40.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.331+0000 7f2cc4861b80 1 -- 172.21.15.101:0/1536117979 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x559fd723fc60 con 0x559fd74e1ad0 2023-07-18T03:15:40.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.332+0000 7f2c91ffb700 1 -- 172.21.15.101:0/1536117979 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559fd73824f0 con 0x559fd74e1ad0 2023-07-18T03:15:40.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.332+0000 7f2c91ffb700 1 -- 172.21.15.101:0/1536117979 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f2ca4015410 con 0x559fd74e1ad0 2023-07-18T03:15:40.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.332+0000 7f2cc4861b80 10 client.27205 fetch_fsmap finished waiting for FSMap version 29 2023-07-18T03:15:40.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.332+0000 7f2cc4861b80 10 client.27205 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:15:40.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.332+0000 7f2cc4861b80 20 client.27205 populate_metadata read hostname 'smithi101' 2023-07-18T03:15:40.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.332+0000 7f2cc4861b80 1 --2- 172.21.15.101:0/1536117979 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x559fd74f13d0 0x559fd74f37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:40.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.332+0000 7f2cc4861b80 4 client.27205 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:15:40.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.332+0000 7f2cc4861b80 1 -- 172.21.15.101:0/1536117979 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559fd7421eb0 con 0x559fd74f13d0 2023-07-18T03:15:40.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.332+0000 7f2caa59c700 1 --2- 172.21.15.101:0/1536117979 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x559fd74f13d0 0x559fd74f37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:40.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.333+0000 7f2caa59c700 1 --2- 172.21.15.101:0/1536117979 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x559fd74f13d0 0x559fd74f37e0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aab40b39f9cca90e server_cookie=4efe1d475e7164cb in_seq=0 out_seq=0 2023-07-18T03:15:40.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.333+0000 7f2c91ffb700 10 client.27205 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:15:40.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.334+0000 7f2c91ffb700 1 -- 172.21.15.101:0/1536117979 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559fd7421eb0 con 0x559fd74f13d0 2023-07-18T03:15:40.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.334+0000 7f2c91ffb700 10 client.27205 handle_command_reply: tid=0 2023-07-18T03:15:40.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.334+0000 7f2cc4861b80 1 client.27205 shutdown 2023-07-18T03:15:40.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.334+0000 7f2cc4861b80 1 -- 172.21.15.101:0/1536117979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559fd74e1ad0 msgr2=0x559fd74deee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:40.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.334+0000 7f2cc4861b80 1 --2- 172.21.15.101:0/1536117979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559fd74e1ad0 0x559fd74deee0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4008b40 tx=0x7f2ca4004ad0 comp rx=0 tx=0).stop 2023-07-18T03:15:40.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.334+0000 7f2cc4861b80 1 -- 172.21.15.101:0/1536117979 shutdown_connections 2023-07-18T03:15:40.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.334+0000 7f2cc4861b80 1 --2- 172.21.15.101:0/1536117979 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x559fd74f13d0 0x559fd74f37e0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:40.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.334+0000 7f2cc4861b80 1 --2- 172.21.15.101:0/1536117979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559fd74d62d0 0x559fd74e1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:40.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.334+0000 7f2c91ffb700 0 client.27205 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:15:40.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.334+0000 7f2cc4861b80 1 --2- 172.21.15.101:0/1536117979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559fd74e1ad0 0x559fd74deee0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:40.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.334+0000 7f2cc4861b80 1 --2- 172.21.15.101:0/1536117979 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559fd74d6be0 0x559fd74de7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:40.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.334+0000 7f2cc4861b80 1 -- 172.21.15.101:0/1536117979 >> 172.21.15.101:0/1536117979 conn(0x559fd74ccda0 msgr2=0x559fd74d2290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:40.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.334+0000 7f2cc4861b80 1 -- 172.21.15.101:0/1536117979 shutdown_connections 2023-07-18T03:15:40.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.334+0000 7f2cc4861b80 1 -- 172.21.15.101:0/1536117979 wait complete. 2023-07-18T03:15:40.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.334+0000 7f2cc4861b80 20 client.27205 trim_cache size 0 max 16384 2023-07-18T03:15:40.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.352+0000 7f2cc4861b80 1 Processor -- start 2023-07-18T03:15:40.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.352+0000 7f2cc4861b80 1 -- start start 2023-07-18T03:15:40.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.352+0000 7f2cc4861b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559fd74d62d0 0x559fd74d66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:40.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.352+0000 7f2cc4861b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559fd74d6be0 0x559fd74d91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:40.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.352+0000 7f2cc4861b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559fd74e1ad0 0x559fd74e84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:40.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.352+0000 7f2cc4861b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fd74d3d20 con 0x559fd74e1ad0 2023-07-18T03:15:40.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.352+0000 7f2cc4861b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fd74dcd60 con 0x559fd74d6be0 2023-07-18T03:15:40.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.352+0000 7f2cc4861b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fd74d2a10 con 0x559fd74d62d0 2023-07-18T03:15:40.348 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.352+0000 7f2ca9d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559fd74d6be0 0x559fd74d91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:40.348 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.352+0000 7f2caa59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559fd74d62d0 0x559fd74d66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:40.349 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.352+0000 7f2caad9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559fd74e1ad0 0x559fd74e84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:40.349 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.352+0000 7f2caa59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559fd74d62d0 0x559fd74d66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:51082/0 (socket says 172.21.15.101:51082) 2023-07-18T03:15:40.349 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.352+0000 7f2caa59c700 1 -- 172.21.15.101:0/252970259 learned_addr learned my addr 172.21.15.101:0/252970259 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:40.349 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.353+0000 7f2caad9d700 1 -- 172.21.15.101:0/252970259 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559fd74d62d0 msgr2=0x559fd74d66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:40.349 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.353+0000 7f2caad9d700 1 --2- 172.21.15.101:0/252970259 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559fd74d62d0 0x559fd74d66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:40.349 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.353+0000 7f2caad9d700 1 -- 172.21.15.101:0/252970259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559fd74d6be0 msgr2=0x559fd74d91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:40.349 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.353+0000 7f2caad9d700 1 --2- 172.21.15.101:0/252970259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559fd74d6be0 0x559fd74d91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:40.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.353+0000 7f2caad9d700 1 -- 172.21.15.101:0/252970259 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559fd727d250 con 0x559fd74e1ad0 2023-07-18T03:15:40.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.353+0000 7f2caad9d700 1 --2- 172.21.15.101:0/252970259 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559fd74e1ad0 0x559fd74e84b0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4008b10 tx=0x7f2ca4009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ee478cf9808f34 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:40.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.353+0000 7f2c997fa700 1 -- 172.21.15.101:0/252970259 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca400e030 con 0x559fd74e1ad0 2023-07-18T03:15:40.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.353+0000 7f2c997fa700 1 -- 172.21.15.101:0/252970259 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ca400f040 con 0x559fd74e1ad0 2023-07-18T03:15:40.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.353+0000 7f2c997fa700 1 -- 172.21.15.101:0/252970259 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4003d70 con 0x559fd74e1ad0 2023-07-18T03:15:40.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.354+0000 7f2cc4861b80 1 -- 172.21.15.101:0/252970259 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559fd74e1ad0 msgr2=0x559fd74e84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:40.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.354+0000 7f2cc4861b80 1 --2- 172.21.15.101:0/252970259 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559fd74e1ad0 0x559fd74e84b0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4008b10 tx=0x7f2ca4009940 comp rx=0 tx=0).stop 2023-07-18T03:15:40.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.354+0000 7f2cc4861b80 1 -- 172.21.15.101:0/252970259 shutdown_connections 2023-07-18T03:15:40.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.354+0000 7f2cc4861b80 1 --2- 172.21.15.101:0/252970259 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559fd74e1ad0 0x559fd74e84b0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:40.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.354+0000 7f2cc4861b80 1 --2- 172.21.15.101:0/252970259 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559fd74d62d0 0x559fd74d66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:40.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.354+0000 7f2cc4861b80 1 --2- 172.21.15.101:0/252970259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559fd74d6be0 0x559fd74d91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:40.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.354+0000 7f2cc4861b80 1 -- 172.21.15.101:0/252970259 >> 172.21.15.101:0/252970259 conn(0x559fd74ccda0 msgr2=0x559fd74ed760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:40.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.354+0000 7f2cc4861b80 1 -- 172.21.15.101:0/252970259 shutdown_connections 2023-07-18T03:15:40.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.354+0000 7f2cc4861b80 1 -- 172.21.15.101:0/252970259 wait complete. 2023-07-18T03:15:40.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.354+0000 7f2cc4861b80 1 -- start start 2023-07-18T03:15:40.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.354+0000 7f2cc4861b80 1 Processor -- start 2023-07-18T03:15:40.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.355+0000 7f2cc4861b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559fd74d62d0 0x559fd74e6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:40.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.355+0000 7f2cc4861b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559fd74d6be0 0x559fd74e69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:40.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.355+0000 7f2cc4861b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559fd74e1ad0 0x559fd74e6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:40.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.355+0000 7f2cc4861b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fd7544460 con 0x559fd74d62d0 2023-07-18T03:15:40.355 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.355+0000 7f2cc4861b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fd75445d0 con 0x559fd74d6be0 2023-07-18T03:15:40.355 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.355+0000 7f2cc4861b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fd7544790 con 0x559fd74e1ad0 2023-07-18T03:15:40.355 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.355+0000 7f2cc4861b80 10 client.0 _ll_register_callbacks cb 0x7f2ca4009350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:15:40.355 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.355+0000 7f2ca9d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559fd74d6be0 0x559fd74e69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:40.355 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.355+0000 7f2caa59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559fd74d62d0 0x559fd74e6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:40.355 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.355+0000 7f2caad9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559fd74e1ad0 0x559fd74e6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:40.355 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.355+0000 7f2ca9d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559fd74d6be0 0x559fd74e69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:35386/0 (socket says 172.21.15.101:35386) 2023-07-18T03:15:40.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.355+0000 7f2ca9d9b700 1 -- 172.21.15.101:0/1518697352 learned_addr learned my addr 172.21.15.101:0/1518697352 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:40.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.356+0000 7f2ca9d9b700 1 -- 172.21.15.101:0/1518697352 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559fd74e1ad0 msgr2=0x559fd74e6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:40.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.356+0000 7f2ca9d9b700 1 --2- 172.21.15.101:0/1518697352 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559fd74e1ad0 0x559fd74e6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:40.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.356+0000 7f2ca9d9b700 1 -- 172.21.15.101:0/1518697352 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559fd74d62d0 msgr2=0x559fd74e6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:40.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.356+0000 7f2ca9d9b700 1 --2- 172.21.15.101:0/1518697352 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559fd74d62d0 0x559fd74e6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:40.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.356+0000 7f2ca9d9b700 1 -- 172.21.15.101:0/1518697352 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559fd723fc60 con 0x559fd74d6be0 2023-07-18T03:15:40.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.356+0000 7f2caa59c700 1 --2- 172.21.15.101:0/1518697352 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559fd74d62d0 0x559fd74e6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:15:40.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.356+0000 7f2ca9d9b700 1 --2- 172.21.15.101:0/1518697352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559fd74d6be0 0x559fd74e69d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2ca000b6f0 tx=0x7f2ca00057b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61e6462ba27a37bd server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:40.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.356+0000 7f2c91ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:15:40.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.356+0000 7f2c91ffb700 1 -- 172.21.15.101:0/1518697352 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0031070 con 0x559fd74d6be0 2023-07-18T03:15:40.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.356+0000 7f2cc4861b80 1 -- 172.21.15.101:0/1518697352 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2ca001e040 con 0x559fd74d6be0 2023-07-18T03:15:40.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.356+0000 7f2c91ffb700 1 -- 172.21.15.101:0/1518697352 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f2ca000bce0 con 0x559fd74d6be0 2023-07-18T03:15:40.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.356+0000 7f2c91ffb700 1 -- 172.21.15.101:0/1518697352 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ca001d040 con 0x559fd74d6be0 2023-07-18T03:15:40.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.356+0000 7f2cc4861b80 10 client.27176 fetch_fsmap learned FSMap version 29 2023-07-18T03:15:40.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.356+0000 7f2cc4861b80 1 -- 172.21.15.101:0/1518697352 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x559fd72552b0 con 0x559fd74d6be0 2023-07-18T03:15:40.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.357+0000 7f2c91ffb700 1 -- 172.21.15.101:0/1518697352 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ca001e040 con 0x559fd74d6be0 2023-07-18T03:15:40.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.357+0000 7f2c91ffb700 1 -- 172.21.15.101:0/1518697352 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f2ca000f560 con 0x559fd74d6be0 2023-07-18T03:15:40.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.357+0000 7f2cc4861b80 10 client.27176 fetch_fsmap finished waiting for FSMap version 29 2023-07-18T03:15:40.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.357+0000 7f2cc4861b80 10 client.27176 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:15:40.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.357+0000 7f2cc4861b80 20 client.27176 populate_metadata read hostname 'smithi101' 2023-07-18T03:15:40.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.357+0000 7f2cc4861b80 1 --2- 172.21.15.101:0/1518697352 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x559fd7545b90 0x559fd7547fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:40.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.357+0000 7f2cc4861b80 4 client.27176 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:15:40.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.357+0000 7f2cc4861b80 1 -- 172.21.15.101:0/1518697352 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559fd73e4050 con 0x559fd7545b90 2023-07-18T03:15:40.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.357+0000 7f2caa59c700 1 --2- 172.21.15.101:0/1518697352 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x559fd7545b90 0x559fd7547fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:40.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.357+0000 7f2caa59c700 1 --2- 172.21.15.101:0/1518697352 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x559fd7545b90 0x559fd7547fa0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2efe9b9bdf3a8299 server_cookie=6c4ea53b0d4aab65 in_seq=0 out_seq=0 2023-07-18T03:15:40.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.357+0000 7f2c91ffb700 10 client.27176 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:15:40.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.358+0000 7f2c91ffb700 1 -- 172.21.15.101:0/1518697352 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x559fd73e4050 con 0x559fd7545b90 2023-07-18T03:15:40.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.358+0000 7f2c91ffb700 10 client.27176 handle_command_reply: tid=0 2023-07-18T03:15:40.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.358+0000 7f2cc4861b80 1 client.27176 shutdown 2023-07-18T03:15:40.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.358+0000 7f2cc4861b80 1 -- 172.21.15.101:0/1518697352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559fd74d6be0 msgr2=0x559fd74e69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:40.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.358+0000 7f2cc4861b80 1 --2- 172.21.15.101:0/1518697352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559fd74d6be0 0x559fd74e69d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2ca000b6f0 tx=0x7f2ca00057b0 comp rx=0 tx=0).stop 2023-07-18T03:15:40.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.359+0000 7f2cc4861b80 1 -- 172.21.15.101:0/1518697352 shutdown_connections 2023-07-18T03:15:40.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.359+0000 7f2cc4861b80 1 --2- 172.21.15.101:0/1518697352 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x559fd7545b90 0x559fd7547fa0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:40.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.359+0000 7f2cc4861b80 1 --2- 172.21.15.101:0/1518697352 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559fd74e1ad0 0x559fd74e6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:40.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.359+0000 7f2c91ffb700 0 client.27176 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:15:40.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.359+0000 7f2cc4861b80 1 --2- 172.21.15.101:0/1518697352 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559fd74d62d0 0x559fd74e6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:40.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.359+0000 7f2cc4861b80 1 --2- 172.21.15.101:0/1518697352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559fd74d6be0 0x559fd74e69d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:40.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.359+0000 7f2cc4861b80 1 -- 172.21.15.101:0/1518697352 >> 172.21.15.101:0/1518697352 conn(0x559fd74ccda0 msgr2=0x559fd74ecc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:40.361 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.359+0000 7f2cc4861b80 1 -- 172.21.15.101:0/1518697352 shutdown_connections 2023-07-18T03:15:40.361 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.359+0000 7f2cc4861b80 1 -- 172.21.15.101:0/1518697352 wait complete. 2023-07-18T03:15:40.361 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.359+0000 7f2cc4861b80 20 client.27176 trim_cache size 0 max 16384 2023-07-18T03:15:40.361 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:15:40.361 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "no active scrubs running", 2023-07-18T03:15:40.362 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": {} 2023-07-18T03:15:40.362 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:15:40.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.359+0000 7f2cb0fff700 1 -- 172.21.15.101:0/1417215693 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2c940a12c0 msgr2=0x7f2c940a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:40.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.359+0000 7f2cb0fff700 1 --2- 172.21.15.101:0/1417215693 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2c940a12c0 0x7f2c940a3770 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2ca400f540 tx=0x7f2ca400b040 comp rx=0 tx=0).stop 2023-07-18T03:15:40.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.359+0000 7f2cb0fff700 1 -- 172.21.15.101:0/1417215693 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2cac0ffae0 msgr2=0x7f2cac197e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:40.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.359+0000 7f2cb0fff700 1 --2- 172.21.15.101:0/1417215693 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2cac0ffae0 0x7f2cac197e00 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c00eb30 tx=0x7f2c9c00ee40 comp rx=0 tx=0).stop 2023-07-18T03:15:40.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.360+0000 7f2cb0fff700 1 -- 172.21.15.101:0/1417215693 shutdown_connections 2023-07-18T03:15:40.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.360+0000 7f2cb0fff700 1 --2- 172.21.15.101:0/1417215693 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2c940a12c0 0x7f2c940a3770 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:40.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.360+0000 7f2cb0fff700 1 --2- 172.21.15.101:0/1417215693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cac105af0 0x7f2cac19c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:40.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.360+0000 7f2cb0fff700 1 --2- 172.21.15.101:0/1417215693 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2cac0ffae0 0x7f2cac197e00 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:40.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.360+0000 7f2cb0fff700 1 --2- 172.21.15.101:0/1417215693 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cac100490 0x7f2cac198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:40.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.360+0000 7f2cb0fff700 1 -- 172.21.15.101:0/1417215693 >> 172.21.15.101:0/1417215693 conn(0x7f2cac0fb5e0 msgr2=0x7f2cac10af00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:40.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.360+0000 7f2cb0fff700 1 -- 172.21.15.101:0/1417215693 shutdown_connections 2023-07-18T03:15:40.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.360+0000 7f2cb0fff700 1 -- 172.21.15.101:0/1417215693 wait complete. 2023-07-18T03:15:40.372 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-18T03:15:40.372 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:15:40.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.597+0000 7f4a3d7ff700 1 -- 172.21.15.101:0/3533080208 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4a38100d00 msgr2=0x7f4a381010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:40.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.597+0000 7f4a3d7ff700 1 --2- 172.21.15.101:0/3533080208 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4a38100d00 0x7f4a381010d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f4a280098f0 tx=0x7f4a28009c00 comp rx=0 tx=0).stop 2023-07-18T03:15:40.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.597+0000 7f4a3d7ff700 1 -- 172.21.15.101:0/3533080208 shutdown_connections 2023-07-18T03:15:40.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.597+0000 7f4a3d7ff700 1 --2- 172.21.15.101:0/3533080208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a3810a040 0x7f4a3810c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:40.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.597+0000 7f4a3d7ff700 1 --2- 172.21.15.101:0/3533080208 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4a38100d00 0x7f4a381010d0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:40.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.597+0000 7f4a3d7ff700 1 --2- 172.21.15.101:0/3533080208 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a38101610 0x7f4a38109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:40.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.597+0000 7f4a3d7ff700 1 -- 172.21.15.101:0/3533080208 >> 172.21.15.101:0/3533080208 conn(0x7f4a380fa490 msgr2=0x7f4a380fc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:40.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.597+0000 7f4a3d7ff700 1 -- 172.21.15.101:0/3533080208 shutdown_connections 2023-07-18T03:15:40.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.597+0000 7f4a3d7ff700 1 -- 172.21.15.101:0/3533080208 wait complete. 2023-07-18T03:15:40.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.598+0000 7f4a3d7ff700 1 Processor -- start 2023-07-18T03:15:40.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.598+0000 7f4a3d7ff700 1 -- start start 2023-07-18T03:15:40.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.598+0000 7f4a3d7ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a38100d00 0x7f4a38197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:40.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.598+0000 7f4a3d7ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a38101610 0x7f4a381983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:40.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.598+0000 7f4a3d7ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4a3810a040 0x7f4a3819c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:40.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.598+0000 7f4a3d7ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3819ce40 con 0x7f4a3810a040 2023-07-18T03:15:40.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.598+0000 7f4a3d7ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3819cfb0 con 0x7f4a38100d00 2023-07-18T03:15:40.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.598+0000 7f4a3d7ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3819d170 con 0x7f4a38101610 2023-07-18T03:15:40.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.599+0000 7f4a367fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a38101610 0x7f4a381983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:40.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.599+0000 7f4a367fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a38101610 0x7f4a381983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:51096/0 (socket says 172.21.15.101:51096) 2023-07-18T03:15:40.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.599+0000 7f4a367fc700 1 -- 172.21.15.101:0/1277916984 learned_addr learned my addr 172.21.15.101:0/1277916984 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:40.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.599+0000 7f4a377fe700 1 --2- 172.21.15.101:0/1277916984 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4a3810a040 0x7f4a3819c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:40.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.599+0000 7f4a36ffd700 1 --2- 172.21.15.101:0/1277916984 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a38100d00 0x7f4a38197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:40.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.599+0000 7f4a367fc700 1 -- 172.21.15.101:0/1277916984 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a38100d00 msgr2=0x7f4a38197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:40.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.599+0000 7f4a367fc700 1 --2- 172.21.15.101:0/1277916984 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a38100d00 0x7f4a38197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:40.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.599+0000 7f4a367fc700 1 -- 172.21.15.101:0/1277916984 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4a3810a040 msgr2=0x7f4a3819c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:40.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.599+0000 7f4a367fc700 1 --2- 172.21.15.101:0/1277916984 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4a3810a040 0x7f4a3819c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:40.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.599+0000 7f4a367fc700 1 -- 172.21.15.101:0/1277916984 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a2c009520 con 0x7f4a38101610 2023-07-18T03:15:40.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.599+0000 7f4a367fc700 1 --2- 172.21.15.101:0/1277916984 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a38101610 0x7f4a381983a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c00eb30 tx=0x7f4a2c00ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:40.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.599+0000 7f4a1ffff700 1 -- 172.21.15.101:0/1277916984 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c009880 con 0x7f4a38101610 2023-07-18T03:15:40.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.600+0000 7f4a1ffff700 1 -- 172.21.15.101:0/1277916984 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a2c0044e0 con 0x7f4a38101610 2023-07-18T03:15:40.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.600+0000 7f4a3d7ff700 1 -- 172.21.15.101:0/1277916984 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a30009d00 con 0x7f4a38101610 2023-07-18T03:15:40.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.600+0000 7f4a1ffff700 1 -- 172.21.15.101:0/1277916984 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c003d80 con 0x7f4a38101610 2023-07-18T03:15:40.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.600+0000 7f4a3d7ff700 1 -- 172.21.15.101:0/1277916984 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a28009520 con 0x7f4a38101610 2023-07-18T03:15:40.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.601+0000 7f4a3d7ff700 1 -- 172.21.15.101:0/1277916984 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a38067300 con 0x7f4a38101610 2023-07-18T03:15:40.599 INFO:tasks.workunit.client.0.smithi101.stdout: 1 554381 6.66 MB/sec execute 305 sec latency 23.044 ms 2023-07-18T03:15:40.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.603+0000 7f4a1ffff700 1 -- 172.21.15.101:0/1277916984 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f4a2c003f20 con 0x7f4a38101610 2023-07-18T03:15:40.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.604+0000 7f4a1ffff700 1 --2- 172.21.15.101:0/1277916984 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4a200a1300 0x7f4a200a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:40.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.604+0000 7f4a1ffff700 1 -- 172.21.15.101:0/1277916984 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f4a2c014070 con 0x7f4a38101610 2023-07-18T03:15:40.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.604+0000 7f4a36ffd700 1 --2- 172.21.15.101:0/1277916984 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4a200a1300 0x7f4a200a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:40.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.605+0000 7f4a36ffd700 1 --2- 172.21.15.101:0/1277916984 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4a200a1300 0x7f4a200a37b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4a28005e90 tx=0x7f4a28019040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:40.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.608+0000 7f4a1ffff700 1 -- 172.21.15.101:0/1277916984 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4a2c096670 con 0x7f4a38101610 2023-07-18T03:15:40.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.877+0000 7f4a3d7ff700 1 -- 172.21.15.101:0/1277916984 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a381991b0 con 0x7f4a38101610 2023-07-18T03:15:40.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.878+0000 7f4a1ffff700 1 -- 172.21.15.101:0/1277916984 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10863 (secure 0 0 0) 0x7f4a2c099c90 con 0x7f4a38101610 2023-07-18T03:15:40.872 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:15:40.873 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/4192846769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":4192846769},{"type":"v1","addr":"172.21.15.101:6839","nonce":4192846769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6841/3824340175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6840","nonce":3824340175},{"type":"v1","addr":"172.21.15.101:6841","nonce":3824340175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:04:59.361185+0000","modified":"2023-07-18T03:14:01.125192+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.155:6839/2055462483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2055462483},{"type":"v1","addr":"172.21.15.155:6839","nonce":2055462483}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:15:40.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.881+0000 7f4a3d7ff700 1 -- 172.21.15.101:0/1277916984 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4a200a1300 msgr2=0x7f4a200a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:40.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.881+0000 7f4a3d7ff700 1 --2- 172.21.15.101:0/1277916984 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4a200a1300 0x7f4a200a37b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4a28005e90 tx=0x7f4a28019040 comp rx=0 tx=0).stop 2023-07-18T03:15:40.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.881+0000 7f4a3d7ff700 1 -- 172.21.15.101:0/1277916984 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a38101610 msgr2=0x7f4a381983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:40.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.881+0000 7f4a3d7ff700 1 --2- 172.21.15.101:0/1277916984 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a38101610 0x7f4a381983a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c00eb30 tx=0x7f4a2c00ee40 comp rx=0 tx=0).stop 2023-07-18T03:15:40.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.881+0000 7f4a3d7ff700 1 -- 172.21.15.101:0/1277916984 shutdown_connections 2023-07-18T03:15:40.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.881+0000 7f4a3d7ff700 1 --2- 172.21.15.101:0/1277916984 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4a200a1300 0x7f4a200a37b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:40.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.881+0000 7f4a3d7ff700 1 --2- 172.21.15.101:0/1277916984 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a38100d00 0x7f4a38197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:40.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.881+0000 7f4a3d7ff700 1 --2- 172.21.15.101:0/1277916984 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4a3810a040 0x7f4a3819c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:40.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.881+0000 7f4a3d7ff700 1 --2- 172.21.15.101:0/1277916984 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a38101610 0x7f4a381983a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:40.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.881+0000 7f4a3d7ff700 1 -- 172.21.15.101:0/1277916984 >> 172.21.15.101:0/1277916984 conn(0x7f4a380fa490 msgr2=0x7f4a3810c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:40.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.881+0000 7f4a3d7ff700 1 -- 172.21.15.101:0/1277916984 shutdown_connections 2023-07-18T03:15:40.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:40.881+0000 7f4a3d7ff700 1 -- 172.21.15.101:0/1277916984 wait complete. 2023-07-18T03:15:40.877 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 29 2023-07-18T03:15:40.894 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-18T03:15:41.108 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.113+0000 7f6e7f9ff700 1 -- 172.21.15.101:0/2088885591 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e78100480 msgr2=0x7f6e7810c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:41.108 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.113+0000 7f6e7f9ff700 1 --2- 172.21.15.101:0/2088885591 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e78100480 0x7f6e7810c910 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6e7000b0a0 tx=0x7f6e7000b3b0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.108 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.113+0000 7f6e7f9ff700 1 -- 172.21.15.101:0/2088885591 shutdown_connections 2023-07-18T03:15:41.108 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.113+0000 7f6e7f9ff700 1 --2- 172.21.15.101:0/2088885591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e78105ae0 0x7f6e78105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.113+0000 7f6e7f9ff700 1 --2- 172.21.15.101:0/2088885591 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e78100480 0x7f6e7810c910 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.113+0000 7f6e7f9ff700 1 --2- 172.21.15.101:0/2088885591 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6e780ffad0 0x7f6e780fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.113+0000 7f6e7f9ff700 1 -- 172.21.15.101:0/2088885591 >> 172.21.15.101:0/2088885591 conn(0x7f6e780fb5f0 msgr2=0x7f6e780fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:41.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.113+0000 7f6e7f9ff700 1 -- 172.21.15.101:0/2088885591 shutdown_connections 2023-07-18T03:15:41.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.113+0000 7f6e7f9ff700 1 -- 172.21.15.101:0/2088885591 wait complete. 2023-07-18T03:15:41.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.114+0000 7f6e7f9ff700 1 Processor -- start 2023-07-18T03:15:41.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.115+0000 7f6e7f9ff700 1 -- start start 2023-07-18T03:15:41.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.115+0000 7f6e7f9ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e780ffad0 0x7f6e78197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.115+0000 7f6e7f9ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e78100480 0x7f6e78198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.115+0000 7f6e7f9ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6e78105ae0 0x7f6e7819c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.115+0000 7f6e7f9ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7819cef0 con 0x7f6e78105ae0 2023-07-18T03:15:41.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.115+0000 7f6e7f9ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7819d060 con 0x7f6e780ffad0 2023-07-18T03:15:41.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.115+0000 7f6e7f9ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7819d220 con 0x7f6e78100480 2023-07-18T03:15:41.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.115+0000 7f6e7cf9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e78100480 0x7f6e78198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.115+0000 7f6e7d79b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e780ffad0 0x7f6e78197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.115+0000 7f6e7cf9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e78100480 0x7f6e78198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:51112/0 (socket says 172.21.15.101:51112) 2023-07-18T03:15:41.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.115+0000 7f6e7cf9a700 1 -- 172.21.15.101:0/3612148180 learned_addr learned my addr 172.21.15.101:0/3612148180 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:41.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.115+0000 7f6e7d79b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e780ffad0 0x7f6e78197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:35416/0 (socket says 172.21.15.101:35416) 2023-07-18T03:15:41.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.115+0000 7f6e7df9c700 1 --2- 172.21.15.101:0/3612148180 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6e78105ae0 0x7f6e7819c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.115+0000 7f6e7cf9a700 1 -- 172.21.15.101:0/3612148180 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e780ffad0 msgr2=0x7f6e78197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:41.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.115+0000 7f6e7cf9a700 1 --2- 172.21.15.101:0/3612148180 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e780ffad0 0x7f6e78197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.115+0000 7f6e7cf9a700 1 -- 172.21.15.101:0/3612148180 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6e78105ae0 msgr2=0x7f6e7819c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:41.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.115+0000 7f6e7cf9a700 1 --2- 172.21.15.101:0/3612148180 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6e78105ae0 0x7f6e7819c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.115+0000 7f6e7cf9a700 1 -- 172.21.15.101:0/3612148180 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e70009d00 con 0x7f6e78100480 2023-07-18T03:15:41.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.116+0000 7f6e7d79b700 1 --2- 172.21.15.101:0/3612148180 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e780ffad0 0x7f6e78197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:15:41.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.116+0000 7f6e7df9c700 1 --2- 172.21.15.101:0/3612148180 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6e78105ae0 0x7f6e7819c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:15:41.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.116+0000 7f6e7cf9a700 1 --2- 172.21.15.101:0/3612148180 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e78100480 0x7f6e78198450 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6e6800e9a0 tx=0x7f6e6800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:41.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.116+0000 7f6e6e7fc700 1 -- 172.21.15.101:0/3612148180 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e68009790 con 0x7f6e78100480 2023-07-18T03:15:41.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.116+0000 7f6e6e7fc700 1 -- 172.21.15.101:0/3612148180 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e68010430 con 0x7f6e78100480 2023-07-18T03:15:41.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.116+0000 7f6e7f9ff700 1 -- 172.21.15.101:0/3612148180 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e7819d500 con 0x7f6e78100480 2023-07-18T03:15:41.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.116+0000 7f6e6e7fc700 1 -- 172.21.15.101:0/3612148180 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e68018680 con 0x7f6e78100480 2023-07-18T03:15:41.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.116+0000 7f6e7f9ff700 1 -- 172.21.15.101:0/3612148180 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e781a4d40 con 0x7f6e78100480 2023-07-18T03:15:41.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.120+0000 7f6e6e7fc700 1 -- 172.21.15.101:0/3612148180 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f6e680105a0 con 0x7f6e78100480 2023-07-18T03:15:41.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.121+0000 7f6e9331bb80 1 Processor -- start 2023-07-18T03:15:41.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.121+0000 7f6e9331bb80 1 -- start start 2023-07-18T03:15:41.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.121+0000 7f6e6e7fc700 1 --2- 172.21.15.101:0/3612148180 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6e640a12c0 0x7f6e640a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.121+0000 7f6e6e7fc700 1 -- 172.21.15.101:0/3612148180 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f6e68014070 con 0x7f6e78100480 2023-07-18T03:15:41.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.121+0000 7f6e9331bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601c25492d0 0x5601c25496a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.121+0000 7f6e9331bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5601c2549be0 0x5601c2554590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.121+0000 7f6e9331bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601c2554ad0 0x5601c2556eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.121+0000 7f6e9331bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601c244fcb0 con 0x5601c2549be0 2023-07-18T03:15:41.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.121+0000 7f6e9331bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601c24a2a40 con 0x5601c25492d0 2023-07-18T03:15:41.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.121+0000 7f6e9331bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601c249bed0 con 0x5601c2554ad0 2023-07-18T03:15:41.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.122+0000 7f6e7df9c700 1 --2- 172.21.15.101:0/3612148180 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6e640a12c0 0x7f6e640a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.122+0000 7f6e7d79b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601c25492d0 0x5601c25496a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.122+0000 7f6e7d79b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601c25492d0 0x5601c25496a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:35418/0 (socket says 172.21.15.101:35418) 2023-07-18T03:15:41.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.122+0000 7f6e7d79b700 1 -- 172.21.15.101:0/326310067 learned_addr learned my addr 172.21.15.101:0/326310067 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:41.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.122+0000 7f6e7df9c700 1 --2- 172.21.15.101:0/326310067 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601c2554ad0 0x5601c2556eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.122+0000 7f6e7cf9a700 1 --2- 172.21.15.101:0/326310067 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5601c2549be0 0x5601c2554590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.122+0000 7f6e7d79b700 1 -- 172.21.15.101:0/326310067 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601c2554ad0 msgr2=0x5601c2556eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:41.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.122+0000 7f6e7d79b700 1 --2- 172.21.15.101:0/326310067 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601c2554ad0 0x5601c2556eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.122+0000 7f6e7d79b700 1 -- 172.21.15.101:0/326310067 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5601c2549be0 msgr2=0x5601c2554590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:41.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.122+0000 7f6e7d79b700 1 --2- 172.21.15.101:0/326310067 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5601c2549be0 0x5601c2554590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.122+0000 7f6e7d79b700 1 -- 172.21.15.101:0/326310067 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601c22c9de0 con 0x5601c25492d0 2023-07-18T03:15:41.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.122+0000 7f6e7df9c700 1 --2- 172.21.15.101:0/3612148180 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6e640a12c0 0x7f6e640a3770 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6e7000b780 tx=0x7f6e70009480 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:41.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.122+0000 7f6e7d79b700 1 --2- 172.21.15.101:0/326310067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601c25492d0 0x5601c25496a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6e740115f0 tx=0x7f6e74011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88548c8e2c5fa26e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:41.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.123+0000 7f6e637fe700 1 -- 172.21.15.101:0/326310067 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e74019630 con 0x5601c25492d0 2023-07-18T03:15:41.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.123+0000 7f6e637fe700 1 -- 172.21.15.101:0/326310067 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e74019c50 con 0x5601c25492d0 2023-07-18T03:15:41.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.123+0000 7f6e637fe700 1 -- 172.21.15.101:0/326310067 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7400ebd0 con 0x5601c25492d0 2023-07-18T03:15:41.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.123+0000 7f6e9331bb80 1 -- 172.21.15.101:0/326310067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601c25492d0 msgr2=0x5601c25496a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:41.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.123+0000 7f6e9331bb80 1 --2- 172.21.15.101:0/326310067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601c25492d0 0x5601c25496a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6e740115f0 tx=0x7f6e74011900 comp rx=0 tx=0).stop 2023-07-18T03:15:41.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.124+0000 7f6e9331bb80 1 -- 172.21.15.101:0/326310067 shutdown_connections 2023-07-18T03:15:41.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.124+0000 7f6e9331bb80 1 --2- 172.21.15.101:0/326310067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601c25492d0 0x5601c25496a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.124+0000 7f6e9331bb80 1 --2- 172.21.15.101:0/326310067 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601c2554ad0 0x5601c2556eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.124+0000 7f6e9331bb80 1 --2- 172.21.15.101:0/326310067 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5601c2549be0 0x5601c2554590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.124+0000 7f6e9331bb80 1 -- 172.21.15.101:0/326310067 >> 172.21.15.101:0/326310067 conn(0x5601c253fda0 msgr2=0x5601c25456a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:41.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.124+0000 7f6e9331bb80 1 -- 172.21.15.101:0/326310067 shutdown_connections 2023-07-18T03:15:41.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.124+0000 7f6e9331bb80 1 -- 172.21.15.101:0/326310067 wait complete. 2023-07-18T03:15:41.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.124+0000 7f6e9331bb80 1 -- start start 2023-07-18T03:15:41.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.124+0000 7f6e9331bb80 1 Processor -- start 2023-07-18T03:15:41.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.125+0000 7f6e9331bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601c25492d0 0x5601c2554550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.125+0000 7f6e9331bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601c2549be0 0x5601c2551760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.125+0000 7f6e9331bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5601c2554ad0 0x5601c2551e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.125+0000 7f6e9331bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601c243bac0 con 0x5601c2554ad0 2023-07-18T03:15:41.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.125+0000 7f6e9331bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601c243b7c0 con 0x5601c25492d0 2023-07-18T03:15:41.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.125+0000 7f6e9331bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601c231f920 con 0x5601c2549be0 2023-07-18T03:15:41.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.125+0000 7f6e9331bb80 10 client.0 _ll_register_callbacks cb 0x5601c2439760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:15:41.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.125+0000 7f6e7d79b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601c25492d0 0x5601c2554550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.125+0000 7f6e7cf9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601c2549be0 0x5601c2551760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.125+0000 7f6e7df9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5601c2554ad0 0x5601c2551e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.125+0000 7f6e7d79b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601c25492d0 0x5601c2554550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:35428/0 (socket says 172.21.15.101:35428) 2023-07-18T03:15:41.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.125+0000 7f6e7d79b700 1 -- 172.21.15.101:0/3733972461 learned_addr learned my addr 172.21.15.101:0/3733972461 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:41.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.125+0000 7f6e7cf9a700 1 -- 172.21.15.101:0/3733972461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601c25492d0 msgr2=0x5601c2554550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:41.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.125+0000 7f6e7cf9a700 1 --2- 172.21.15.101:0/3733972461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601c25492d0 0x5601c2554550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.125+0000 7f6e7cf9a700 1 -- 172.21.15.101:0/3733972461 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5601c2554ad0 msgr2=0x5601c2551e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:41.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.125+0000 7f6e7cf9a700 1 --2- 172.21.15.101:0/3733972461 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5601c2554ad0 0x5601c2551e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.125+0000 7f6e7cf9a700 1 -- 172.21.15.101:0/3733972461 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e74009d50 con 0x5601c2549be0 2023-07-18T03:15:41.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.126+0000 7f6e7cf9a700 1 --2- 172.21.15.101:0/3733972461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601c2549be0 0x5601c2551760 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6e680ca630 tx=0x7f6e6809a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8329c5944a61a91 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:41.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.126+0000 7f6e60ff9700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:15:41.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.126+0000 7f6e60ff9700 1 -- 172.21.15.101:0/3733972461 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e680b1070 con 0x5601c2549be0 2023-07-18T03:15:41.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.126+0000 7f6e60ff9700 1 -- 172.21.15.101:0/3733972461 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f6e680a2400 con 0x5601c2549be0 2023-07-18T03:15:41.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.126+0000 7f6e9331bb80 1 -- 172.21.15.101:0/3733972461 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5601c23f54f0 con 0x5601c2549be0 2023-07-18T03:15:41.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.126+0000 7f6e60ff9700 1 -- 172.21.15.101:0/3733972461 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e680994a0 con 0x5601c2549be0 2023-07-18T03:15:41.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.126+0000 7f6e9331bb80 10 client.27229 fetch_fsmap learned FSMap version 29 2023-07-18T03:15:41.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.126+0000 7f6e9331bb80 1 -- 172.21.15.101:0/3733972461 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5601c22b2c60 con 0x5601c2549be0 2023-07-18T03:15:41.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.126+0000 7f6e60ff9700 1 -- 172.21.15.101:0/3733972461 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e680c8020 con 0x5601c2549be0 2023-07-18T03:15:41.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.126+0000 7f6e60ff9700 1 -- 172.21.15.101:0/3733972461 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f6e680b8220 con 0x5601c2549be0 2023-07-18T03:15:41.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.126+0000 7f6e9331bb80 10 client.27229 fetch_fsmap finished waiting for FSMap version 29 2023-07-18T03:15:41.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.126+0000 7f6e9331bb80 10 client.27229 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:15:41.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.127+0000 7f6e9331bb80 20 client.27229 populate_metadata read hostname 'smithi101' 2023-07-18T03:15:41.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.127+0000 7f6e9331bb80 1 --2- 172.21.15.101:0/3733972461 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5601c2564470 0x5601c2566880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.127+0000 7f6e9331bb80 4 client.27229 mds_command: new command op to 24541 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:15:41.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.127+0000 7f6e9331bb80 1 -- 172.21.15.101:0/3733972461 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5601c2494eb0 con 0x5601c2564470 2023-07-18T03:15:41.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.127+0000 7f6e7d79b700 1 --2- 172.21.15.101:0/3733972461 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5601c2564470 0x5601c2566880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.127+0000 7f6e7d79b700 1 --2- 172.21.15.101:0/3733972461 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5601c2564470 0x5601c2566880 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=15048466e30f51fc server_cookie=c95714b842c98433 in_seq=0 out_seq=0 2023-07-18T03:15:41.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.127+0000 7f6e60ff9700 10 client.27229 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:15:41.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.128+0000 7f6e60ff9700 1 -- 172.21.15.101:0/3733972461 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5601c2494eb0 con 0x5601c2564470 2023-07-18T03:15:41.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.128+0000 7f6e60ff9700 10 client.27229 handle_command_reply: tid=0 2023-07-18T03:15:41.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.128+0000 7f6e9331bb80 1 client.27229 shutdown 2023-07-18T03:15:41.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.128+0000 7f6e9331bb80 1 -- 172.21.15.101:0/3733972461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601c2549be0 msgr2=0x5601c2551760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:41.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.128+0000 7f6e9331bb80 1 --2- 172.21.15.101:0/3733972461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601c2549be0 0x5601c2551760 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6e680ca630 tx=0x7f6e6809a100 comp rx=0 tx=0).stop 2023-07-18T03:15:41.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.128+0000 7f6e9331bb80 1 -- 172.21.15.101:0/3733972461 shutdown_connections 2023-07-18T03:15:41.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.128+0000 7f6e9331bb80 1 --2- 172.21.15.101:0/3733972461 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5601c2564470 0x5601c2566880 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.128+0000 7f6e9331bb80 1 --2- 172.21.15.101:0/3733972461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601c25492d0 0x5601c2554550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.128+0000 7f6e60ff9700 0 client.27229 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:15:41.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.128+0000 7f6e9331bb80 1 --2- 172.21.15.101:0/3733972461 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5601c2554ad0 0x5601c2551e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.128+0000 7f6e9331bb80 1 --2- 172.21.15.101:0/3733972461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601c2549be0 0x5601c2551760 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.128+0000 7f6e9331bb80 1 -- 172.21.15.101:0/3733972461 >> 172.21.15.101:0/3733972461 conn(0x5601c253fda0 msgr2=0x5601c2545230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:41.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.129+0000 7f6e9331bb80 1 -- 172.21.15.101:0/3733972461 shutdown_connections 2023-07-18T03:15:41.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.129+0000 7f6e9331bb80 1 -- 172.21.15.101:0/3733972461 wait complete. 2023-07-18T03:15:41.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.129+0000 7f6e9331bb80 20 client.27229 trim_cache size 0 max 16384 2023-07-18T03:15:41.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.142+0000 7f6e9331bb80 1 Processor -- start 2023-07-18T03:15:41.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.142+0000 7f6e9331bb80 1 -- start start 2023-07-18T03:15:41.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.142+0000 7f6e9331bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5601c25492d0 0x5601c254c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.142+0000 7f6e9331bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601c2549be0 0x5601c2554080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.142+0000 7f6e9331bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601c2554ad0 0x5601c2551000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.142+0000 7f6e9331bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601c2561ad0 con 0x5601c25492d0 2023-07-18T03:15:41.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.142+0000 7f6e9331bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601c254a1c0 con 0x5601c2554ad0 2023-07-18T03:15:41.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.142+0000 7f6e9331bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601c2550cd0 con 0x5601c2549be0 2023-07-18T03:15:41.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.143+0000 7f6e7d79b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5601c25492d0 0x5601c254c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.143+0000 7f6e7df9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601c2554ad0 0x5601c2551000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.143+0000 7f6e7cf9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601c2549be0 0x5601c2554080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.143+0000 7f6e7d79b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5601c25492d0 0x5601c254c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:43914/0 (socket says 172.21.15.101:43914) 2023-07-18T03:15:41.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.143+0000 7f6e7d79b700 1 -- 172.21.15.101:0/2263217891 learned_addr learned my addr 172.21.15.101:0/2263217891 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:41.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.143+0000 7f6e7cf9a700 1 -- 172.21.15.101:0/2263217891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601c2554ad0 msgr2=0x5601c2551000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:41.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.143+0000 7f6e7cf9a700 1 --2- 172.21.15.101:0/2263217891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601c2554ad0 0x5601c2551000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.143+0000 7f6e7cf9a700 1 -- 172.21.15.101:0/2263217891 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5601c25492d0 msgr2=0x5601c254c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:41.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.143+0000 7f6e7cf9a700 1 --2- 172.21.15.101:0/2263217891 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5601c25492d0 0x5601c254c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.143+0000 7f6e7cf9a700 1 -- 172.21.15.101:0/2263217891 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601c22f0250 con 0x5601c2549be0 2023-07-18T03:15:41.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.143+0000 7f6e7cf9a700 1 --2- 172.21.15.101:0/2263217891 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601c2549be0 0x5601c2554080 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6e6800c2d0 tx=0x7f6e68010a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=debc002c16724753 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:41.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.143+0000 7f6e63fff700 1 -- 172.21.15.101:0/2263217891 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e680ca4b0 con 0x5601c2549be0 2023-07-18T03:15:41.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.143+0000 7f6e63fff700 1 -- 172.21.15.101:0/2263217891 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e680c8070 con 0x5601c2549be0 2023-07-18T03:15:41.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.143+0000 7f6e63fff700 1 -- 172.21.15.101:0/2263217891 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e680b8120 con 0x5601c2549be0 2023-07-18T03:15:41.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.144+0000 7f6e9331bb80 1 -- 172.21.15.101:0/2263217891 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601c2549be0 msgr2=0x5601c2554080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:41.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.144+0000 7f6e9331bb80 1 --2- 172.21.15.101:0/2263217891 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601c2549be0 0x5601c2554080 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6e6800c2d0 tx=0x7f6e68010a90 comp rx=0 tx=0).stop 2023-07-18T03:15:41.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.144+0000 7f6e9331bb80 1 -- 172.21.15.101:0/2263217891 shutdown_connections 2023-07-18T03:15:41.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.144+0000 7f6e9331bb80 1 --2- 172.21.15.101:0/2263217891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601c2554ad0 0x5601c2551000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.144+0000 7f6e9331bb80 1 --2- 172.21.15.101:0/2263217891 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5601c25492d0 0x5601c254c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.144+0000 7f6e9331bb80 1 --2- 172.21.15.101:0/2263217891 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601c2549be0 0x5601c2554080 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.144+0000 7f6e9331bb80 1 -- 172.21.15.101:0/2263217891 >> 172.21.15.101:0/2263217891 conn(0x5601c253fda0 msgr2=0x5601c254ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:41.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.144+0000 7f6e9331bb80 1 -- 172.21.15.101:0/2263217891 shutdown_connections 2023-07-18T03:15:41.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.144+0000 7f6e9331bb80 1 -- 172.21.15.101:0/2263217891 wait complete. 2023-07-18T03:15:41.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.144+0000 7f6e9331bb80 1 -- start start 2023-07-18T03:15:41.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.145+0000 7f6e9331bb80 1 Processor -- start 2023-07-18T03:15:41.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.145+0000 7f6e9331bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601c25492d0 0x5601c255b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.145+0000 7f6e9331bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601c2549be0 0x5601c2559670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.145+0000 7f6e9331bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5601c2554ad0 0x5601c2559bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.145+0000 7f6e9331bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601c255a0f0 con 0x5601c2554ad0 2023-07-18T03:15:41.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.145+0000 7f6e9331bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601c255a260 con 0x5601c25492d0 2023-07-18T03:15:41.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.145+0000 7f6e9331bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601c255a420 con 0x5601c2549be0 2023-07-18T03:15:41.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.145+0000 7f6e9331bb80 10 client.0 _ll_register_callbacks cb 0x7f6e68099a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:15:41.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.145+0000 7f6e7d79b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601c25492d0 0x5601c255b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.145+0000 7f6e7d79b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601c25492d0 0x5601c255b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:35448/0 (socket says 172.21.15.101:35448) 2023-07-18T03:15:41.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.145+0000 7f6e7d79b700 1 -- 172.21.15.101:0/842970574 learned_addr learned my addr 172.21.15.101:0/842970574 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:41.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.145+0000 7f6e7cf9a700 1 --2- 172.21.15.101:0/842970574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601c2549be0 0x5601c2559670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.145+0000 7f6e7df9c700 1 --2- 172.21.15.101:0/842970574 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5601c2554ad0 0x5601c2559bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.146+0000 7f6e7d79b700 1 -- 172.21.15.101:0/842970574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601c2549be0 msgr2=0x5601c2559670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:41.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.146+0000 7f6e7d79b700 1 --2- 172.21.15.101:0/842970574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601c2549be0 0x5601c2559670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.146+0000 7f6e7d79b700 1 -- 172.21.15.101:0/842970574 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5601c2554ad0 msgr2=0x5601c2559bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:41.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.146+0000 7f6e7d79b700 1 --2- 172.21.15.101:0/842970574 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5601c2554ad0 0x5601c2559bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.146+0000 7f6e7d79b700 1 -- 172.21.15.101:0/842970574 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5601c22b2c60 con 0x5601c25492d0 2023-07-18T03:15:41.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.146+0000 7f6e7cf9a700 1 --2- 172.21.15.101:0/842970574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601c2549be0 0x5601c2559670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:15:41.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.146+0000 7f6e7d79b700 1 --2- 172.21.15.101:0/842970574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601c25492d0 0x5601c255b310 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6e74011c50 tx=0x7f6e74009490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1c081f1c427de02 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:41.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.146+0000 7f6e60ff9700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:15:41.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.146+0000 7f6e60ff9700 1 -- 172.21.15.101:0/842970574 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e74029070 con 0x5601c25492d0 2023-07-18T03:15:41.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.146+0000 7f6e60ff9700 1 -- 172.21.15.101:0/842970574 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f6e74005040 con 0x5601c25492d0 2023-07-18T03:15:41.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.146+0000 7f6e9331bb80 1 -- 172.21.15.101:0/842970574 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6e74019c50 con 0x5601c25492d0 2023-07-18T03:15:41.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.146+0000 7f6e60ff9700 1 -- 172.21.15.101:0/842970574 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e74019ae0 con 0x5601c25492d0 2023-07-18T03:15:41.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.146+0000 7f6e9331bb80 10 client.27200 fetch_fsmap learned FSMap version 29 2023-07-18T03:15:41.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.146+0000 7f6e9331bb80 1 -- 172.21.15.101:0/842970574 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5601c22c82b0 con 0x5601c25492d0 2023-07-18T03:15:41.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.147+0000 7f6e60ff9700 1 -- 172.21.15.101:0/842970574 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e74019c50 con 0x5601c25492d0 2023-07-18T03:15:41.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.147+0000 7f6e60ff9700 1 -- 172.21.15.101:0/842970574 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f6e7400e4a0 con 0x5601c25492d0 2023-07-18T03:15:41.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.147+0000 7f6e9331bb80 10 client.27200 fetch_fsmap finished waiting for FSMap version 29 2023-07-18T03:15:41.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.147+0000 7f6e9331bb80 10 client.27200 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:15:41.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.147+0000 7f6e9331bb80 20 client.27200 populate_metadata read hostname 'smithi101' 2023-07-18T03:15:41.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.147+0000 7f6e9331bb80 1 --2- 172.21.15.101:0/842970574 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5601c25b9700 0x5601c25bbb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.147+0000 7f6e9331bb80 4 client.27200 mds_command: new command op to 24541 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:15:41.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.147+0000 7f6e9331bb80 1 -- 172.21.15.101:0/842970574 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5601c2457050 con 0x5601c25b9700 2023-07-18T03:15:41.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.147+0000 7f6e7cf9a700 1 --2- 172.21.15.101:0/842970574 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5601c25b9700 0x5601c25bbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.148+0000 7f6e7cf9a700 1 --2- 172.21.15.101:0/842970574 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5601c25b9700 0x5601c25bbb10 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b6f85b35fa79c87b server_cookie=37fcca248a83d8c7 in_seq=0 out_seq=0 2023-07-18T03:15:41.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.148+0000 7f6e60ff9700 10 client.27200 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:15:41.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.148+0000 7f6e60ff9700 1 -- 172.21.15.101:0/842970574 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5601c2457050 con 0x5601c25b9700 2023-07-18T03:15:41.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.148+0000 7f6e60ff9700 10 client.27200 handle_command_reply: tid=0 2023-07-18T03:15:41.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.148+0000 7f6e9331bb80 1 client.27200 shutdown 2023-07-18T03:15:41.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.148+0000 7f6e9331bb80 1 -- 172.21.15.101:0/842970574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601c25492d0 msgr2=0x5601c255b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:41.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.148+0000 7f6e9331bb80 1 --2- 172.21.15.101:0/842970574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601c25492d0 0x5601c255b310 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6e74011c50 tx=0x7f6e74009490 comp rx=0 tx=0).stop 2023-07-18T03:15:41.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.148+0000 7f6e9331bb80 1 -- 172.21.15.101:0/842970574 shutdown_connections 2023-07-18T03:15:41.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.148+0000 7f6e9331bb80 1 --2- 172.21.15.101:0/842970574 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5601c25b9700 0x5601c25bbb10 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.148+0000 7f6e9331bb80 1 --2- 172.21.15.101:0/842970574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5601c25492d0 0x5601c255b310 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.148+0000 7f6e60ff9700 0 client.27200 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:15:41.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.148+0000 7f6e9331bb80 1 --2- 172.21.15.101:0/842970574 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5601c2554ad0 0x5601c2559bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.148+0000 7f6e9331bb80 1 --2- 172.21.15.101:0/842970574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5601c2549be0 0x5601c2559670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.148+0000 7f6e9331bb80 1 -- 172.21.15.101:0/842970574 >> 172.21.15.101:0/842970574 conn(0x5601c253fda0 msgr2=0x5601c2550f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:41.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.148+0000 7f6e9331bb80 1 -- 172.21.15.101:0/842970574 shutdown_connections 2023-07-18T03:15:41.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.149+0000 7f6e9331bb80 1 -- 172.21.15.101:0/842970574 wait complete. 2023-07-18T03:15:41.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.149+0000 7f6e9331bb80 20 client.27200 trim_cache size 0 max 16384 2023-07-18T03:15:41.156 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:15:41.156 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.149+0000 7f6e7f9ff700 1 -- 172.21.15.101:0/3612148180 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6e640a12c0 msgr2=0x7f6e640a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:41.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.149+0000 7f6e7f9ff700 1 --2- 172.21.15.101:0/3612148180 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6e640a12c0 0x7f6e640a3770 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6e7000b780 tx=0x7f6e70009480 comp rx=0 tx=0).stop 2023-07-18T03:15:41.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.149+0000 7f6e7f9ff700 1 -- 172.21.15.101:0/3612148180 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e78100480 msgr2=0x7f6e78198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:41.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.149+0000 7f6e7f9ff700 1 --2- 172.21.15.101:0/3612148180 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e78100480 0x7f6e78198450 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6e6800e9a0 tx=0x7f6e6800ecb0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.149+0000 7f6e7f9ff700 1 -- 172.21.15.101:0/3612148180 shutdown_connections 2023-07-18T03:15:41.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.149+0000 7f6e7f9ff700 1 --2- 172.21.15.101:0/3612148180 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6e640a12c0 0x7f6e640a3770 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.149+0000 7f6e7f9ff700 1 --2- 172.21.15.101:0/3612148180 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e780ffad0 0x7f6e78197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.149+0000 7f6e7f9ff700 1 --2- 172.21.15.101:0/3612148180 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6e78105ae0 0x7f6e7819c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.149+0000 7f6e7f9ff700 1 --2- 172.21.15.101:0/3612148180 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e78100480 0x7f6e78198450 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.149+0000 7f6e7f9ff700 1 -- 172.21.15.101:0/3612148180 >> 172.21.15.101:0/3612148180 conn(0x7f6e780fb5f0 msgr2=0x7f6e7810aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:41.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.149+0000 7f6e7f9ff700 1 -- 172.21.15.101:0/3612148180 shutdown_connections 2023-07-18T03:15:41.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.150+0000 7f6e7f9ff700 1 -- 172.21.15.101:0/3612148180 wait complete. 2023-07-18T03:15:41.160 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:40 smithi101 ceph-mon[126379]: pgmap v250: 129 pgs: 129 active+clean; 398 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 140 op/s 2023-07-18T03:15:41.160 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:40 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1277916984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:15:41.162 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-18T03:15:41.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:40 smithi155 ceph-mon[128885]: pgmap v250: 129 pgs: 129 active+clean; 398 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 140 op/s 2023-07-18T03:15:41.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:40 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1277916984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:15:41.379 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.384+0000 7f662dfff700 1 -- 172.21.15.101:0/1979459712 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6628072850 msgr2=0x7f6628072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:41.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.384+0000 7f662dfff700 1 --2- 172.21.15.101:0/1979459712 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6628072850 0x7f6628072c20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f66180098f0 tx=0x7f6618009c00 comp rx=0 tx=0).stop 2023-07-18T03:15:41.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.384+0000 7f662dfff700 1 -- 172.21.15.101:0/1979459712 shutdown_connections 2023-07-18T03:15:41.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.384+0000 7f662dfff700 1 --2- 172.21.15.101:0/1979459712 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f662810c3a0 0x7f662810e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.381 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.384+0000 7f662dfff700 1 --2- 172.21.15.101:0/1979459712 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6628072850 0x7f6628072c20 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.381 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.384+0000 7f662dfff700 1 --2- 172.21.15.101:0/1979459712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6628073160 0x7f662810be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.381 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.384+0000 7f662dfff700 1 -- 172.21.15.101:0/1979459712 >> 172.21.15.101:0/1979459712 conn(0x7f66280fb600 msgr2=0x7f66280fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:41.382 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.384+0000 7f662dfff700 1 -- 172.21.15.101:0/1979459712 shutdown_connections 2023-07-18T03:15:41.382 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.384+0000 7f662dfff700 1 -- 172.21.15.101:0/1979459712 wait complete. 2023-07-18T03:15:41.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.385+0000 7f662dfff700 1 Processor -- start 2023-07-18T03:15:41.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.386+0000 7f662dfff700 1 -- start start 2023-07-18T03:15:41.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.386+0000 7f662dfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6628072850 0x7f6628197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.386+0000 7f662dfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6628073160 0x7f66281983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.386+0000 7f662dfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f662810c3a0 0x7f662819c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.386+0000 7f662dfff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662819ce70 con 0x7f6628072850 2023-07-18T03:15:41.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.386+0000 7f662dfff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662819cfe0 con 0x7f6628073160 2023-07-18T03:15:41.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.386+0000 7f662dfff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662819d1a0 con 0x7f662810c3a0 2023-07-18T03:15:41.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.386+0000 7f6626ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6628073160 0x7f66281983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.386+0000 7f6627fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f662810c3a0 0x7f662819c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.386+0000 7f66277fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6628072850 0x7f6628197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.387+0000 7f6627fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f662810c3a0 0x7f662819c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:51184/0 (socket says 172.21.15.101:51184) 2023-07-18T03:15:41.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.387+0000 7f6627fff700 1 -- 172.21.15.101:0/362088135 learned_addr learned my addr 172.21.15.101:0/362088135 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:41.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.387+0000 7f6627fff700 1 -- 172.21.15.101:0/362088135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6628073160 msgr2=0x7f66281983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:41.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.387+0000 7f6627fff700 1 --2- 172.21.15.101:0/362088135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6628073160 0x7f66281983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.387+0000 7f6627fff700 1 -- 172.21.15.101:0/362088135 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6628072850 msgr2=0x7f6628197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:41.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.387+0000 7f6627fff700 1 --2- 172.21.15.101:0/362088135 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6628072850 0x7f6628197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.387+0000 7f6627fff700 1 -- 172.21.15.101:0/362088135 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f661c0095f0 con 0x7f662810c3a0 2023-07-18T03:15:41.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.387+0000 7f6627fff700 1 --2- 172.21.15.101:0/362088135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f662810c3a0 0x7f662819c830 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6620009fd0 tx=0x7f662000ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:41.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.387+0000 7f6624ff9700 1 -- 172.21.15.101:0/362088135 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6620018070 con 0x7f662810c3a0 2023-07-18T03:15:41.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.387+0000 7f6624ff9700 1 -- 172.21.15.101:0/362088135 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f662000f040 con 0x7f662810c3a0 2023-07-18T03:15:41.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.387+0000 7f6624ff9700 1 -- 172.21.15.101:0/362088135 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662000cd00 con 0x7f662810c3a0 2023-07-18T03:15:41.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.387+0000 7f662dfff700 1 -- 172.21.15.101:0/362088135 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6618009520 con 0x7f662810c3a0 2023-07-18T03:15:41.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.388+0000 7f662dfff700 1 -- 172.21.15.101:0/362088135 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66281a4ae0 con 0x7f662810c3a0 2023-07-18T03:15:41.396 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.391+0000 7f6624ff9700 1 -- 172.21.15.101:0/362088135 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f6620004ad0 con 0x7f662810c3a0 2023-07-18T03:15:41.396 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.392+0000 7f6641873b80 1 Processor -- start 2023-07-18T03:15:41.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.392+0000 7f6624ff9700 1 --2- 172.21.15.101:0/362088135 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f66100a12c0 0x7f66100a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.392+0000 7f6641873b80 1 -- start start 2023-07-18T03:15:41.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.393+0000 7f6624ff9700 1 -- 172.21.15.101:0/362088135 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f66200ccab0 con 0x7f662810c3a0 2023-07-18T03:15:41.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.393+0000 7f6626ffd700 1 --2- 172.21.15.101:0/362088135 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f66100a12c0 0x7f66100a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.393+0000 7f6641873b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555ebd4b2d0 0x5555ebd4b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.393+0000 7f6641873b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555ebd4bbe0 0x5555ebd56590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.393+0000 7f6641873b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5555ebd56ad0 0x5555ebd58eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.393+0000 7f6641873b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555ebc51cb0 con 0x5555ebd56ad0 2023-07-18T03:15:41.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.393+0000 7f6641873b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555ebca4a40 con 0x5555ebd4b2d0 2023-07-18T03:15:41.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.393+0000 7f6641873b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555ebc9ded0 con 0x5555ebd4bbe0 2023-07-18T03:15:41.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.393+0000 7f66277fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555ebd4b2d0 0x5555ebd4b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.393+0000 7f6626ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555ebd4bbe0 0x5555ebd56590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.393+0000 7f66277fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555ebd4b2d0 0x5555ebd4b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:35484/0 (socket says 172.21.15.101:35484) 2023-07-18T03:15:41.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.393+0000 7f66277fe700 1 -- 172.21.15.101:0/1086980885 learned_addr learned my addr 172.21.15.101:0/1086980885 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:41.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.393+0000 7f6627fff700 1 --2- 172.21.15.101:0/1086980885 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5555ebd56ad0 0x5555ebd58eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.394+0000 7f66277fe700 1 -- 172.21.15.101:0/1086980885 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555ebd4bbe0 msgr2=0x5555ebd56590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:41.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.394+0000 7f66277fe700 1 --2- 172.21.15.101:0/1086980885 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555ebd4bbe0 0x5555ebd56590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.394+0000 7f66277fe700 1 -- 172.21.15.101:0/1086980885 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5555ebd56ad0 msgr2=0x5555ebd58eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:41.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.394+0000 7f66277fe700 1 --2- 172.21.15.101:0/1086980885 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5555ebd56ad0 0x5555ebd58eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.394+0000 7f66277fe700 1 -- 172.21.15.101:0/1086980885 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555ebacbde0 con 0x5555ebd4b2d0 2023-07-18T03:15:41.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.394+0000 7f6627fff700 1 --2- 172.21.15.101:0/1086980885 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5555ebd56ad0 0x5555ebd58eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:15:41.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.394+0000 7f6626ffd700 1 --2- 172.21.15.101:0/1086980885 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555ebd4bbe0 0x5555ebd56590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:15:41.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.394+0000 7f6626ffd700 1 --2- 172.21.15.101:0/362088135 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f66100a12c0 0x7f66100a3770 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f66281994a0 tx=0x7f661c0117b0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:41.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.394+0000 7f66277fe700 1 --2- 172.21.15.101:0/1086980885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555ebd4b2d0 0x5555ebd4b6a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f66180098f0 tx=0x7f6618005b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=860a889ca2f5e7c6 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:41.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.394+0000 7f660dffb700 1 -- 172.21.15.101:0/1086980885 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661800f720 con 0x5555ebd4b2d0 2023-07-18T03:15:41.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.394+0000 7f660dffb700 1 -- 172.21.15.101:0/1086980885 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f661801d040 con 0x5555ebd4b2d0 2023-07-18T03:15:41.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.394+0000 7f660dffb700 1 -- 172.21.15.101:0/1086980885 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66180054c0 con 0x5555ebd4b2d0 2023-07-18T03:15:41.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.395+0000 7f6641873b80 1 -- 172.21.15.101:0/1086980885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555ebd4b2d0 msgr2=0x5555ebd4b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:41.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.395+0000 7f6641873b80 1 --2- 172.21.15.101:0/1086980885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555ebd4b2d0 0x5555ebd4b6a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f66180098f0 tx=0x7f6618005b80 comp rx=0 tx=0).stop 2023-07-18T03:15:41.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.395+0000 7f6641873b80 1 -- 172.21.15.101:0/1086980885 shutdown_connections 2023-07-18T03:15:41.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.395+0000 7f6641873b80 1 --2- 172.21.15.101:0/1086980885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555ebd4b2d0 0x5555ebd4b6a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.395+0000 7f6641873b80 1 --2- 172.21.15.101:0/1086980885 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5555ebd56ad0 0x5555ebd58eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.395+0000 7f6641873b80 1 --2- 172.21.15.101:0/1086980885 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555ebd4bbe0 0x5555ebd56590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.395+0000 7f6641873b80 1 -- 172.21.15.101:0/1086980885 >> 172.21.15.101:0/1086980885 conn(0x5555ebd41da0 msgr2=0x5555ebd476a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:41.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.396+0000 7f6641873b80 1 -- 172.21.15.101:0/1086980885 shutdown_connections 2023-07-18T03:15:41.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.396+0000 7f6641873b80 1 -- 172.21.15.101:0/1086980885 wait complete. 2023-07-18T03:15:41.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.396+0000 7f6641873b80 1 -- start start 2023-07-18T03:15:41.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.396+0000 7f6641873b80 1 Processor -- start 2023-07-18T03:15:41.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.397+0000 7f6641873b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555ebd4b2d0 0x5555ebd56510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.397+0000 7f6641873b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5555ebd4bbe0 0x5555ebd53790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.397+0000 7f6641873b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555ebd56ad0 0x5555ebd53ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.397+0000 7f6641873b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555ebc3dac0 con 0x5555ebd4bbe0 2023-07-18T03:15:41.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.397+0000 7f6641873b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555ebc3d7c0 con 0x5555ebd4b2d0 2023-07-18T03:15:41.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.397+0000 7f6641873b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555ebb21920 con 0x5555ebd56ad0 2023-07-18T03:15:41.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.397+0000 7f6641873b80 10 client.0 _ll_register_callbacks cb 0x5555ebc3b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:15:41.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.397+0000 7f66277fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555ebd4b2d0 0x5555ebd56510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.397+0000 7f66277fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555ebd4b2d0 0x5555ebd56510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:35488/0 (socket says 172.21.15.101:35488) 2023-07-18T03:15:41.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.397+0000 7f66277fe700 1 -- 172.21.15.101:0/2259669812 learned_addr learned my addr 172.21.15.101:0/2259669812 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:41.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.398+0000 7f6627fff700 1 --2- 172.21.15.101:0/2259669812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555ebd56ad0 0x5555ebd53ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.398+0000 7f6626ffd700 1 --2- 172.21.15.101:0/2259669812 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5555ebd4bbe0 0x5555ebd53790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.398+0000 7f66277fe700 1 -- 172.21.15.101:0/2259669812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555ebd56ad0 msgr2=0x5555ebd53ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:41.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.398+0000 7f66277fe700 1 --2- 172.21.15.101:0/2259669812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555ebd56ad0 0x5555ebd53ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.398+0000 7f66277fe700 1 -- 172.21.15.101:0/2259669812 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5555ebd4bbe0 msgr2=0x5555ebd53790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:41.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.398+0000 7f66277fe700 1 --2- 172.21.15.101:0/2259669812 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5555ebd4bbe0 0x5555ebd53790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.398+0000 7f66277fe700 1 -- 172.21.15.101:0/2259669812 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f661c00f8b0 con 0x5555ebd4b2d0 2023-07-18T03:15:41.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.398+0000 7f6626ffd700 1 --2- 172.21.15.101:0/2259669812 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5555ebd4bbe0 0x5555ebd53790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:15:41.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.398+0000 7f66277fe700 1 --2- 172.21.15.101:0/2259669812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555ebd4b2d0 0x5555ebd56510 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6618009fd0 tx=0x7f6618005440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ac787cbd82ccdb server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:41.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.398+0000 7f6606ffd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:15:41.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.399+0000 7f6606ffd700 1 -- 172.21.15.101:0/2259669812 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661800b970 con 0x5555ebd4b2d0 2023-07-18T03:15:41.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.399+0000 7f6606ffd700 1 -- 172.21.15.101:0/2259669812 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f661800bad0 con 0x5555ebd4b2d0 2023-07-18T03:15:41.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.399+0000 7f6641873b80 1 -- 172.21.15.101:0/2259669812 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5555ebbf74f0 con 0x5555ebd4b2d0 2023-07-18T03:15:41.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.399+0000 7f6606ffd700 1 -- 172.21.15.101:0/2259669812 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66180056e0 con 0x5555ebd4b2d0 2023-07-18T03:15:41.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.399+0000 7f6641873b80 10 client.27212 fetch_fsmap learned FSMap version 29 2023-07-18T03:15:41.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.399+0000 7f6641873b80 1 -- 172.21.15.101:0/2259669812 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f662810f3c0 con 0x5555ebd4b2d0 2023-07-18T03:15:41.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.399+0000 7f6606ffd700 1 -- 172.21.15.101:0/2259669812 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5555ebbf74f0 con 0x5555ebd4b2d0 2023-07-18T03:15:41.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.399+0000 7f6606ffd700 1 -- 172.21.15.101:0/2259669812 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f6618015020 con 0x5555ebd4b2d0 2023-07-18T03:15:41.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.400+0000 7f6641873b80 10 client.27212 fetch_fsmap finished waiting for FSMap version 29 2023-07-18T03:15:41.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.400+0000 7f6641873b80 10 client.27212 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:15:41.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.400+0000 7f6641873b80 20 client.27212 populate_metadata read hostname 'smithi101' 2023-07-18T03:15:41.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.400+0000 7f6641873b80 1 --2- 172.21.15.101:0/2259669812 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5555ebd65f50 0x5555ebd68360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.400+0000 7f6641873b80 4 client.27212 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:15:41.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.400+0000 7f6641873b80 1 -- 172.21.15.101:0/2259669812 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5555ebc96eb0 con 0x5555ebd65f50 2023-07-18T03:15:41.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.400+0000 7f6626ffd700 1 --2- 172.21.15.101:0/2259669812 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5555ebd65f50 0x5555ebd68360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.400+0000 7f6606ffd700 10 client.27212 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:15:41.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.400+0000 7f6626ffd700 1 --2- 172.21.15.101:0/2259669812 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5555ebd65f50 0x5555ebd68360 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fdf9860819baee9c server_cookie=21db9fc52c3faaa in_seq=0 out_seq=0 2023-07-18T03:15:41.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.401+0000 7f6606ffd700 1 -- 172.21.15.101:0/2259669812 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5555ebc96eb0 con 0x5555ebd65f50 2023-07-18T03:15:41.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.401+0000 7f6606ffd700 10 client.27212 handle_command_reply: tid=0 2023-07-18T03:15:41.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.401+0000 7f6641873b80 1 client.27212 shutdown 2023-07-18T03:15:41.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.401+0000 7f6641873b80 1 -- 172.21.15.101:0/2259669812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555ebd4b2d0 msgr2=0x5555ebd56510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:41.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.401+0000 7f6641873b80 1 --2- 172.21.15.101:0/2259669812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555ebd4b2d0 0x5555ebd56510 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6618009fd0 tx=0x7f6618005440 comp rx=0 tx=0).stop 2023-07-18T03:15:41.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.402+0000 7f6641873b80 1 -- 172.21.15.101:0/2259669812 shutdown_connections 2023-07-18T03:15:41.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.402+0000 7f6641873b80 1 --2- 172.21.15.101:0/2259669812 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5555ebd65f50 0x5555ebd68360 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.402+0000 7f6641873b80 1 --2- 172.21.15.101:0/2259669812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555ebd4b2d0 0x5555ebd56510 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.402+0000 7f6641873b80 1 --2- 172.21.15.101:0/2259669812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555ebd56ad0 0x5555ebd53ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.402+0000 7f6641873b80 1 --2- 172.21.15.101:0/2259669812 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5555ebd4bbe0 0x5555ebd53790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.402+0000 7f6641873b80 1 -- 172.21.15.101:0/2259669812 >> 172.21.15.101:0/2259669812 conn(0x5555ebd41da0 msgr2=0x5555ebd47290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:41.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.402+0000 7f6606ffd700 0 client.27212 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:15:41.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.402+0000 7f6641873b80 1 -- 172.21.15.101:0/2259669812 shutdown_connections 2023-07-18T03:15:41.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.402+0000 7f6641873b80 1 -- 172.21.15.101:0/2259669812 wait complete. 2023-07-18T03:15:41.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.402+0000 7f6641873b80 20 client.27212 trim_cache size 0 max 16384 2023-07-18T03:15:41.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.417+0000 7f6641873b80 1 Processor -- start 2023-07-18T03:15:41.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.417+0000 7f6641873b80 1 -- start start 2023-07-18T03:15:41.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.418+0000 7f6641873b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555ebd4b2d0 0x5555ebd4e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.418+0000 7f6641873b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555ebd4bbe0 0x5555ebd5d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.418+0000 7f6641873b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5555ebd56ad0 0x5555ebd468b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.418+0000 7f6641873b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555ebd43ec0 con 0x5555ebd56ad0 2023-07-18T03:15:41.414 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.418+0000 7f6641873b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555ebd4c540 con 0x5555ebd4bbe0 2023-07-18T03:15:41.414 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.418+0000 7f6641873b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555ebd54da0 con 0x5555ebd4b2d0 2023-07-18T03:15:41.414 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.418+0000 7f6626ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555ebd4bbe0 0x5555ebd5d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.414 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.418+0000 7f6627fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5555ebd56ad0 0x5555ebd468b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.414 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.418+0000 7f6626ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555ebd4bbe0 0x5555ebd5d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:35498/0 (socket says 172.21.15.101:35498) 2023-07-18T03:15:41.414 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.418+0000 7f6626ffd700 1 -- 172.21.15.101:0/83733862 learned_addr learned my addr 172.21.15.101:0/83733862 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:41.414 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.418+0000 7f66277fe700 1 --2- 172.21.15.101:0/83733862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555ebd4b2d0 0x5555ebd4e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.415 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.418+0000 7f6626ffd700 1 -- 172.21.15.101:0/83733862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555ebd4b2d0 msgr2=0x5555ebd4e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:41.415 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.418+0000 7f6626ffd700 1 --2- 172.21.15.101:0/83733862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555ebd4b2d0 0x5555ebd4e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.415 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.418+0000 7f6626ffd700 1 -- 172.21.15.101:0/83733862 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5555ebd56ad0 msgr2=0x5555ebd468b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:41.415 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.418+0000 7f6626ffd700 1 --2- 172.21.15.101:0/83733862 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5555ebd56ad0 0x5555ebd468b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.415 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.418+0000 7f6626ffd700 1 -- 172.21.15.101:0/83733862 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555ebab4c60 con 0x5555ebd4bbe0 2023-07-18T03:15:41.415 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.418+0000 7f6626ffd700 1 --2- 172.21.15.101:0/83733862 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555ebd4bbe0 0x5555ebd5d4b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f661c011fd0 tx=0x7f661c021000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce399a2d968f3dad server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:41.415 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.419+0000 7f660e7fc700 1 -- 172.21.15.101:0/83733862 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c026070 con 0x5555ebd4bbe0 2023-07-18T03:15:41.415 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.419+0000 7f660e7fc700 1 -- 172.21.15.101:0/83733862 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f661c005bc0 con 0x5555ebd4bbe0 2023-07-18T03:15:41.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.419+0000 7f660e7fc700 1 -- 172.21.15.101:0/83733862 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c0285c0 con 0x5555ebd4bbe0 2023-07-18T03:15:41.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.419+0000 7f6641873b80 1 -- 172.21.15.101:0/83733862 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555ebd4bbe0 msgr2=0x5555ebd5d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:41.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.419+0000 7f6641873b80 1 --2- 172.21.15.101:0/83733862 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555ebd4bbe0 0x5555ebd5d4b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f661c011fd0 tx=0x7f661c021000 comp rx=0 tx=0).stop 2023-07-18T03:15:41.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.419+0000 7f6641873b80 1 -- 172.21.15.101:0/83733862 shutdown_connections 2023-07-18T03:15:41.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.419+0000 7f6641873b80 1 --2- 172.21.15.101:0/83733862 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5555ebd56ad0 0x5555ebd468b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.419+0000 7f6641873b80 1 --2- 172.21.15.101:0/83733862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555ebd4b2d0 0x5555ebd4e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.419+0000 7f6641873b80 1 --2- 172.21.15.101:0/83733862 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555ebd4bbe0 0x5555ebd5d4b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.419+0000 7f6641873b80 1 -- 172.21.15.101:0/83733862 >> 172.21.15.101:0/83733862 conn(0x5555ebd41da0 msgr2=0x5555ebd62700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:41.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.420+0000 7f6641873b80 1 -- 172.21.15.101:0/83733862 shutdown_connections 2023-07-18T03:15:41.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.420+0000 7f6641873b80 1 -- 172.21.15.101:0/83733862 wait complete. 2023-07-18T03:15:41.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.420+0000 7f6641873b80 1 -- start start 2023-07-18T03:15:41.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.420+0000 7f6641873b80 1 Processor -- start 2023-07-18T03:15:41.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.421+0000 7f6641873b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555ebd4b2d0 0x5555ebd5d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.421+0000 7f6641873b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5555ebd4bbe0 0x5555ebd5b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.421+0000 7f6641873b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555ebd56ad0 0x5555ebd5bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.421+0000 7f6641873b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555ebd5c0d0 con 0x5555ebd4bbe0 2023-07-18T03:15:41.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.421+0000 7f6641873b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555ebd5c240 con 0x5555ebd4b2d0 2023-07-18T03:15:41.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.421+0000 7f6641873b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555ebd5c400 con 0x5555ebd56ad0 2023-07-18T03:15:41.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.421+0000 7f6641873b80 10 client.0 _ll_register_callbacks cb 0x7f661c011680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:15:41.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.421+0000 7f66277fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555ebd4b2d0 0x5555ebd5d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.421+0000 7f66277fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555ebd4b2d0 0x5555ebd5d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:35508/0 (socket says 172.21.15.101:35508) 2023-07-18T03:15:41.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.421+0000 7f66277fe700 1 -- 172.21.15.101:0/2141294262 learned_addr learned my addr 172.21.15.101:0/2141294262 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:41.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.421+0000 7f6626ffd700 1 --2- 172.21.15.101:0/2141294262 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5555ebd4bbe0 0x5555ebd5b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.421+0000 7f6627fff700 1 --2- 172.21.15.101:0/2141294262 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555ebd56ad0 0x5555ebd5bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.421+0000 7f66277fe700 1 -- 172.21.15.101:0/2141294262 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555ebd56ad0 msgr2=0x5555ebd5bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:41.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.421+0000 7f66277fe700 1 --2- 172.21.15.101:0/2141294262 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555ebd56ad0 0x5555ebd5bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.421+0000 7f66277fe700 1 -- 172.21.15.101:0/2141294262 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5555ebd4bbe0 msgr2=0x5555ebd5b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:41.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.421+0000 7f66277fe700 1 --2- 172.21.15.101:0/2141294262 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5555ebd4bbe0 0x5555ebd5b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.421+0000 7f66277fe700 1 -- 172.21.15.101:0/2141294262 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f661c00fa40 con 0x5555ebd4b2d0 2023-07-18T03:15:41.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.422+0000 7f66277fe700 1 --2- 172.21.15.101:0/2141294262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555ebd4b2d0 0x5555ebd5d320 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f661800faa0 tx=0x7f661800bfc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=380c8ab49e9b3277 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:41.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.422+0000 7f6606ffd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:15:41.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.422+0000 7f6606ffd700 1 -- 172.21.15.101:0/2141294262 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6618005560 con 0x5555ebd4b2d0 2023-07-18T03:15:41.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.422+0000 7f6641873b80 1 -- 172.21.15.101:0/2141294262 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5555ebc59050 con 0x5555ebd4b2d0 2023-07-18T03:15:41.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.422+0000 7f6606ffd700 1 -- 172.21.15.101:0/2141294262 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f66180056c0 con 0x5555ebd4b2d0 2023-07-18T03:15:41.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.422+0000 7f6606ffd700 1 -- 172.21.15.101:0/2141294262 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f661800b770 con 0x5555ebd4b2d0 2023-07-18T03:15:41.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.422+0000 7f6641873b80 10 client.27218 fetch_fsmap learned FSMap version 29 2023-07-18T03:15:41.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.422+0000 7f6641873b80 1 -- 172.21.15.101:0/2141294262 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5555ebaf2250 con 0x5555ebd4b2d0 2023-07-18T03:15:41.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.422+0000 7f6606ffd700 1 -- 172.21.15.101:0/2141294262 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5555ebc59050 con 0x5555ebd4b2d0 2023-07-18T03:15:41.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.423+0000 7f6606ffd700 1 -- 172.21.15.101:0/2141294262 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f6618009920 con 0x5555ebd4b2d0 2023-07-18T03:15:41.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.423+0000 7f6641873b80 10 client.27218 fetch_fsmap finished waiting for FSMap version 29 2023-07-18T03:15:41.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.423+0000 7f6641873b80 10 client.27218 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:15:41.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.423+0000 7f6641873b80 20 client.27218 populate_metadata read hostname 'smithi101' 2023-07-18T03:15:41.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.423+0000 7f6641873b80 1 --2- 172.21.15.101:0/2141294262 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5555ebdbaee0 0x5555ebdbd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.423+0000 7f6641873b80 4 client.27218 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:15:41.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.423+0000 7f6641873b80 1 -- 172.21.15.101:0/2141294262 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5555ebaecf70 con 0x5555ebdbaee0 2023-07-18T03:15:41.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.423+0000 7f6626ffd700 1 --2- 172.21.15.101:0/2141294262 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5555ebdbaee0 0x5555ebdbd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.423+0000 7f6606ffd700 10 client.27218 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:15:41.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.423+0000 7f6626ffd700 1 --2- 172.21.15.101:0/2141294262 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5555ebdbaee0 0x5555ebdbd2f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3aa61d5d750c14fc server_cookie=ff810ee965f3a437 in_seq=0 out_seq=0 2023-07-18T03:15:41.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.424+0000 7f6606ffd700 1 -- 172.21.15.101:0/2141294262 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5555ebaecf70 con 0x5555ebdbaee0 2023-07-18T03:15:41.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.424+0000 7f6606ffd700 10 client.27218 handle_command_reply: tid=0 2023-07-18T03:15:41.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.424+0000 7f6641873b80 1 client.27218 shutdown 2023-07-18T03:15:41.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.424+0000 7f6641873b80 1 -- 172.21.15.101:0/2141294262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555ebd4b2d0 msgr2=0x5555ebd5d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:41.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.424+0000 7f6641873b80 1 --2- 172.21.15.101:0/2141294262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555ebd4b2d0 0x5555ebd5d320 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f661800faa0 tx=0x7f661800bfc0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.424+0000 7f6641873b80 1 -- 172.21.15.101:0/2141294262 shutdown_connections 2023-07-18T03:15:41.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.424+0000 7f6641873b80 1 --2- 172.21.15.101:0/2141294262 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5555ebdbaee0 0x5555ebdbd2f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.424+0000 7f6641873b80 1 --2- 172.21.15.101:0/2141294262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555ebd4b2d0 0x5555ebd5d320 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.424+0000 7f6641873b80 1 --2- 172.21.15.101:0/2141294262 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555ebd56ad0 0x5555ebd5bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.424+0000 7f6606ffd700 0 client.27218 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:15:41.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.424+0000 7f6641873b80 1 --2- 172.21.15.101:0/2141294262 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5555ebd4bbe0 0x5555ebd5b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.424+0000 7f6641873b80 1 -- 172.21.15.101:0/2141294262 >> 172.21.15.101:0/2141294262 conn(0x5555ebd41da0 msgr2=0x5555ebd468b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:41.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.424+0000 7f6641873b80 1 -- 172.21.15.101:0/2141294262 shutdown_connections 2023-07-18T03:15:41.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.425+0000 7f6641873b80 1 -- 172.21.15.101:0/2141294262 wait complete. 2023-07-18T03:15:41.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.425+0000 7f6641873b80 20 client.27218 trim_cache size 0 max 16384 2023-07-18T03:15:41.428 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:15:41.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.425+0000 7f662dfff700 1 -- 172.21.15.101:0/362088135 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f66100a12c0 msgr2=0x7f66100a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:41.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.425+0000 7f662dfff700 1 --2- 172.21.15.101:0/362088135 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f66100a12c0 0x7f66100a3770 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f66281994a0 tx=0x7f661c0117b0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.425+0000 7f662dfff700 1 -- 172.21.15.101:0/362088135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f662810c3a0 msgr2=0x7f662819c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:41.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.425+0000 7f662dfff700 1 --2- 172.21.15.101:0/362088135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f662810c3a0 0x7f662819c830 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6620009fd0 tx=0x7f662000ef90 comp rx=0 tx=0).stop 2023-07-18T03:15:41.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.425+0000 7f662dfff700 1 -- 172.21.15.101:0/362088135 shutdown_connections 2023-07-18T03:15:41.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.425+0000 7f662dfff700 1 --2- 172.21.15.101:0/362088135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f662810c3a0 0x7f662819c830 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.425+0000 7f662dfff700 1 --2- 172.21.15.101:0/362088135 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6628072850 0x7f6628197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.425+0000 7f662dfff700 1 --2- 172.21.15.101:0/362088135 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f66100a12c0 0x7f66100a3770 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.425+0000 7f662dfff700 1 --2- 172.21.15.101:0/362088135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6628073160 0x7f66281983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.425+0000 7f662dfff700 1 -- 172.21.15.101:0/362088135 >> 172.21.15.101:0/362088135 conn(0x7f66280fb600 msgr2=0x7f662810e480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:41.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.425+0000 7f662dfff700 1 -- 172.21.15.101:0/362088135 shutdown_connections 2023-07-18T03:15:41.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.425+0000 7f662dfff700 1 -- 172.21.15.101:0/362088135 wait complete. 2023-07-18T03:15:41.437 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-18T03:15:41.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:40 smithi179 ceph-mon[129229]: pgmap v250: 129 pgs: 129 active+clean; 398 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 140 op/s 2023-07-18T03:15:41.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:40 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1277916984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:15:41.595 INFO:tasks.workunit.client.0.smithi101.stdout: 1 555233 6.64 MB/sec execute 306 sec latency 28.734 ms 2023-07-18T03:15:41.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.675+0000 7fd54c7ff700 1 -- 172.21.15.101:0/2964988722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd544105af0 msgr2=0x7fd544105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:41.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.675+0000 7fd54c7ff700 1 --2- 172.21.15.101:0/2964988722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd544105af0 0x7fd544105ec0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd540009990 tx=0x7fd540009ca0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.675+0000 7fd54c7ff700 1 -- 172.21.15.101:0/2964988722 shutdown_connections 2023-07-18T03:15:41.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.675+0000 7fd54c7ff700 1 --2- 172.21.15.101:0/2964988722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd544105af0 0x7fd544105ec0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.675+0000 7fd54c7ff700 1 --2- 172.21.15.101:0/2964988722 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd544100490 0x7fd54410c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.675+0000 7fd54c7ff700 1 --2- 172.21.15.101:0/2964988722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5440ffae0 0x7fd5440fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.675+0000 7fd54c7ff700 1 -- 172.21.15.101:0/2964988722 >> 172.21.15.101:0/2964988722 conn(0x7fd5440fb5e0 msgr2=0x7fd5440fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:41.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.675+0000 7fd54c7ff700 1 -- 172.21.15.101:0/2964988722 shutdown_connections 2023-07-18T03:15:41.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.675+0000 7fd54c7ff700 1 -- 172.21.15.101:0/2964988722 wait complete. 2023-07-18T03:15:41.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.677+0000 7fd54c7ff700 1 Processor -- start 2023-07-18T03:15:41.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.677+0000 7fd54c7ff700 1 -- start start 2023-07-18T03:15:41.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.677+0000 7fd54c7ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5440ffae0 0x7fd544197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.677+0000 7fd54c7ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd544100490 0x7fd5441983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.677+0000 7fd54c7ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd544105af0 0x7fd54419c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.677+0000 7fd54c7ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54419cde0 con 0x7fd544100490 2023-07-18T03:15:41.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.677+0000 7fd54c7ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54419cf50 con 0x7fd544105af0 2023-07-18T03:15:41.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.677+0000 7fd54c7ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54419d110 con 0x7fd5440ffae0 2023-07-18T03:15:41.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.677+0000 7fd549d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd544100490 0x7fd5441983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.677+0000 7fd54a59b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5440ffae0 0x7fd544197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.677+0000 7fd549d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd544100490 0x7fd5441983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:43992/0 (socket says 172.21.15.101:43992) 2023-07-18T03:15:41.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.677+0000 7fd549d9a700 1 -- 172.21.15.101:0/3225015038 learned_addr learned my addr 172.21.15.101:0/3225015038 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:41.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.677+0000 7fd54ad9c700 1 --2- 172.21.15.101:0/3225015038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd544105af0 0x7fd54419c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.678+0000 7fd549d9a700 1 -- 172.21.15.101:0/3225015038 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5440ffae0 msgr2=0x7fd544197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:41.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.678+0000 7fd549d9a700 1 --2- 172.21.15.101:0/3225015038 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5440ffae0 0x7fd544197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.678+0000 7fd549d9a700 1 -- 172.21.15.101:0/3225015038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd544105af0 msgr2=0x7fd54419c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:41.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.678+0000 7fd549d9a700 1 --2- 172.21.15.101:0/3225015038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd544105af0 0x7fd54419c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.678+0000 7fd549d9a700 1 -- 172.21.15.101:0/3225015038 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5400095f0 con 0x7fd544100490 2023-07-18T03:15:41.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.678+0000 7fd549d9a700 1 --2- 172.21.15.101:0/3225015038 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd544100490 0x7fd5441983b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd53400e9a0 tx=0x7fd53400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:41.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.678+0000 7fd53b7fe700 1 -- 172.21.15.101:0/3225015038 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd534009790 con 0x7fd544100490 2023-07-18T03:15:41.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.678+0000 7fd54c7ff700 1 -- 172.21.15.101:0/3225015038 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd54419d3f0 con 0x7fd544100490 2023-07-18T03:15:41.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.678+0000 7fd53b7fe700 1 -- 172.21.15.101:0/3225015038 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd534010430 con 0x7fd544100490 2023-07-18T03:15:41.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.678+0000 7fd53b7fe700 1 -- 172.21.15.101:0/3225015038 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd534018680 con 0x7fd544100490 2023-07-18T03:15:41.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.678+0000 7fd54c7ff700 1 -- 172.21.15.101:0/3225015038 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5441a4c30 con 0x7fd544100490 2023-07-18T03:15:41.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.682+0000 7fd53b7fe700 1 -- 172.21.15.101:0/3225015038 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fd5340105a0 con 0x7fd544100490 2023-07-18T03:15:41.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.683+0000 7fd560159b80 1 Processor -- start 2023-07-18T03:15:41.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.683+0000 7fd560159b80 1 -- start start 2023-07-18T03:15:41.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.683+0000 7fd560159b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f9614ac2d0 0x55f9614ac6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.683+0000 7fd560159b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f9614acbe0 0x55f9614b7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.683+0000 7fd560159b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9614b7ad0 0x55f9614b9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.683+0000 7fd560159b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9613b2cb0 con 0x55f9614ac2d0 2023-07-18T03:15:41.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.683+0000 7fd560159b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f961405a40 con 0x55f9614b7ad0 2023-07-18T03:15:41.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.683+0000 7fd560159b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9613feed0 con 0x55f9614acbe0 2023-07-18T03:15:41.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.683+0000 7fd53b7fe700 1 --2- 172.21.15.101:0/3225015038 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd5300a12c0 0x7fd5300a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.683+0000 7fd53b7fe700 1 -- 172.21.15.101:0/3225015038 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fd534014070 con 0x7fd544100490 2023-07-18T03:15:41.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.683+0000 7fd54a59b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f9614acbe0 0x55f9614b7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.683+0000 7fd54a59b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f9614acbe0 0x55f9614b7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:51258/0 (socket says 172.21.15.101:51258) 2023-07-18T03:15:41.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.683+0000 7fd54a59b700 1 -- 172.21.15.101:0/2767164373 learned_addr learned my addr 172.21.15.101:0/2767164373 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:41.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.684+0000 7fd549d9a700 1 --2- 172.21.15.101:0/2767164373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9614b7ad0 0x55f9614b9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.684+0000 7fd54ad9c700 1 --2- 172.21.15.101:0/2767164373 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f9614ac2d0 0x55f9614ac6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.684+0000 7fd54ad9c700 1 --2- 172.21.15.101:0/3225015038 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd5300a12c0 0x7fd5300a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.684+0000 7fd54a59b700 1 -- 172.21.15.101:0/2767164373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9614b7ad0 msgr2=0x55f9614b9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:41.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.684+0000 7fd54a59b700 1 --2- 172.21.15.101:0/2767164373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9614b7ad0 0x55f9614b9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.684+0000 7fd54a59b700 1 -- 172.21.15.101:0/2767164373 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f9614ac2d0 msgr2=0x55f9614ac6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:41.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.684+0000 7fd54a59b700 1 --2- 172.21.15.101:0/2767164373 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f9614ac2d0 0x55f9614ac6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.684+0000 7fd54a59b700 1 -- 172.21.15.101:0/2767164373 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f96122cde0 con 0x55f9614acbe0 2023-07-18T03:15:41.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.684+0000 7fd549d9a700 1 --2- 172.21.15.101:0/2767164373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9614b7ad0 0x55f9614b9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:15:41.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.684+0000 7fd54a59b700 1 --2- 172.21.15.101:0/2767164373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f9614acbe0 0x55f9614b7590 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd540005e80 tx=0x7fd540005eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9b5969f5a949869 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:41.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.684+0000 7fd54ad9c700 1 --2- 172.21.15.101:0/3225015038 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd5300a12c0 0x7fd5300a3770 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd53c009ec0 tx=0x7fd53c00b040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:41.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.684+0000 7fd538ff9700 1 -- 172.21.15.101:0/2767164373 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54001d070 con 0x55f9614acbe0 2023-07-18T03:15:41.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.685+0000 7fd538ff9700 1 -- 172.21.15.101:0/2767164373 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd54001e040 con 0x55f9614acbe0 2023-07-18T03:15:41.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.685+0000 7fd538ff9700 1 -- 172.21.15.101:0/2767164373 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd540004900 con 0x55f9614acbe0 2023-07-18T03:15:41.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.685+0000 7fd560159b80 1 -- 172.21.15.101:0/2767164373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f9614acbe0 msgr2=0x55f9614b7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:41.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.685+0000 7fd560159b80 1 --2- 172.21.15.101:0/2767164373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f9614acbe0 0x55f9614b7590 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd540005e80 tx=0x7fd540005eb0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.685+0000 7fd560159b80 1 -- 172.21.15.101:0/2767164373 shutdown_connections 2023-07-18T03:15:41.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.685+0000 7fd560159b80 1 --2- 172.21.15.101:0/2767164373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9614b7ad0 0x55f9614b9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.685+0000 7fd560159b80 1 --2- 172.21.15.101:0/2767164373 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f9614ac2d0 0x55f9614ac6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.685+0000 7fd560159b80 1 --2- 172.21.15.101:0/2767164373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f9614acbe0 0x55f9614b7590 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.685+0000 7fd560159b80 1 -- 172.21.15.101:0/2767164373 >> 172.21.15.101:0/2767164373 conn(0x55f9614a2da0 msgr2=0x55f9614a86a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:41.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.686+0000 7fd560159b80 1 -- 172.21.15.101:0/2767164373 shutdown_connections 2023-07-18T03:15:41.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.686+0000 7fd560159b80 1 -- 172.21.15.101:0/2767164373 wait complete. 2023-07-18T03:15:41.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.686+0000 7fd560159b80 1 -- start start 2023-07-18T03:15:41.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.686+0000 7fd560159b80 1 Processor -- start 2023-07-18T03:15:41.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.687+0000 7fd560159b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9614ac2d0 0x55f9614b74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.687+0000 7fd560159b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f9614acbe0 0x55f9614b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.687+0000 7fd560159b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f9614b7ad0 0x55f9614b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.687+0000 7fd560159b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f96139eac0 con 0x55f9614acbe0 2023-07-18T03:15:41.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.687+0000 7fd560159b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f96139e7c0 con 0x55f9614ac2d0 2023-07-18T03:15:41.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.687+0000 7fd560159b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f961282920 con 0x55f9614b7ad0 2023-07-18T03:15:41.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.687+0000 7fd560159b80 10 client.0 _ll_register_callbacks cb 0x55f96139c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:15:41.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.687+0000 7fd54ad9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f9614b7ad0 0x55f9614b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.687+0000 7fd54a59b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9614ac2d0 0x55f9614b74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.687+0000 7fd54ad9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f9614b7ad0 0x55f9614b4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:51264/0 (socket says 172.21.15.101:51264) 2023-07-18T03:15:41.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.687+0000 7fd549d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f9614acbe0 0x55f9614b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.687+0000 7fd54ad9c700 1 -- 172.21.15.101:0/104813510 learned_addr learned my addr 172.21.15.101:0/104813510 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:41.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.687+0000 7fd54ad9c700 1 -- 172.21.15.101:0/104813510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9614ac2d0 msgr2=0x55f9614b74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:41.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.687+0000 7fd54ad9c700 1 --2- 172.21.15.101:0/104813510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9614ac2d0 0x55f9614b74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.687+0000 7fd54ad9c700 1 -- 172.21.15.101:0/104813510 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f9614acbe0 msgr2=0x55f9614b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:41.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.687+0000 7fd54ad9c700 1 --2- 172.21.15.101:0/104813510 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f9614acbe0 0x55f9614b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.687+0000 7fd54ad9c700 1 -- 172.21.15.101:0/104813510 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5441a4c30 con 0x55f9614b7ad0 2023-07-18T03:15:41.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.688+0000 7fd54ad9c700 1 --2- 172.21.15.101:0/104813510 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f9614b7ad0 0x55f9614b4e00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd53c018b50 tx=0x7fd53c004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a34fbd3502e7295 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:41.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.688+0000 7fd52dffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:15:41.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.688+0000 7fd52dffb700 1 -- 172.21.15.101:0/104813510 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53c00e070 con 0x55f9614b7ad0 2023-07-18T03:15:41.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.688+0000 7fd560159b80 1 -- 172.21.15.101:0/104813510 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f9613584f0 con 0x55f9614b7ad0 2023-07-18T03:15:41.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.688+0000 7fd52dffb700 1 -- 172.21.15.101:0/104813510 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fd53c016a10 con 0x55f9614b7ad0 2023-07-18T03:15:41.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.688+0000 7fd52dffb700 1 -- 172.21.15.101:0/104813510 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd53c018e60 con 0x55f9614b7ad0 2023-07-18T03:15:41.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.688+0000 7fd560159b80 10 client.27265 fetch_fsmap learned FSMap version 29 2023-07-18T03:15:41.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.688+0000 7fd560159b80 1 -- 172.21.15.101:0/104813510 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fd54410d220 con 0x55f9614b7ad0 2023-07-18T03:15:41.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.689+0000 7fd52dffb700 1 -- 172.21.15.101:0/104813510 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f9613584f0 con 0x55f9614b7ad0 2023-07-18T03:15:41.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.689+0000 7fd52dffb700 1 -- 172.21.15.101:0/104813510 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fd53c021300 con 0x55f9614b7ad0 2023-07-18T03:15:41.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.689+0000 7fd560159b80 10 client.27265 fetch_fsmap finished waiting for FSMap version 29 2023-07-18T03:15:41.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.689+0000 7fd560159b80 10 client.27265 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:15:41.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.689+0000 7fd560159b80 20 client.27265 populate_metadata read hostname 'smithi101' 2023-07-18T03:15:41.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.689+0000 7fd560159b80 1 --2- 172.21.15.101:0/104813510 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f9614c7430 0x55f9614c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.690 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.689+0000 7fd560159b80 4 client.27265 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:15:41.690 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.689+0000 7fd560159b80 1 -- 172.21.15.101:0/104813510 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f9613f7eb0 con 0x55f9614c7430 2023-07-18T03:15:41.690 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.689+0000 7fd54a59b700 1 --2- 172.21.15.101:0/104813510 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f9614c7430 0x55f9614c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.690+0000 7fd52dffb700 10 client.27265 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:15:41.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.690+0000 7fd54a59b700 1 --2- 172.21.15.101:0/104813510 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f9614c7430 0x55f9614c9840 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dac27154bc50575 server_cookie=667c49beb9ab7def in_seq=0 out_seq=0 2023-07-18T03:15:41.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.690+0000 7fd52dffb700 1 -- 172.21.15.101:0/104813510 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f9613f7eb0 con 0x55f9614c7430 2023-07-18T03:15:41.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.690+0000 7fd52dffb700 10 client.27265 handle_command_reply: tid=0 2023-07-18T03:15:41.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.690+0000 7fd560159b80 1 client.27265 shutdown 2023-07-18T03:15:41.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.691+0000 7fd560159b80 1 -- 172.21.15.101:0/104813510 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f9614b7ad0 msgr2=0x55f9614b4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:41.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.691+0000 7fd560159b80 1 --2- 172.21.15.101:0/104813510 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f9614b7ad0 0x55f9614b4e00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd53c018b50 tx=0x7fd53c004ad0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.691+0000 7fd560159b80 1 -- 172.21.15.101:0/104813510 shutdown_connections 2023-07-18T03:15:41.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.691+0000 7fd560159b80 1 --2- 172.21.15.101:0/104813510 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f9614c7430 0x55f9614c9840 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.691+0000 7fd560159b80 1 --2- 172.21.15.101:0/104813510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9614ac2d0 0x55f9614b74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.691+0000 7fd560159b80 1 --2- 172.21.15.101:0/104813510 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f9614b7ad0 0x55f9614b4e00 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.691+0000 7fd560159b80 1 --2- 172.21.15.101:0/104813510 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f9614acbe0 0x55f9614b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.691+0000 7fd52dffb700 0 client.27265 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:15:41.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.691+0000 7fd560159b80 1 -- 172.21.15.101:0/104813510 >> 172.21.15.101:0/104813510 conn(0x55f9614a2da0 msgr2=0x55f9614a8290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:41.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.691+0000 7fd560159b80 1 -- 172.21.15.101:0/104813510 shutdown_connections 2023-07-18T03:15:41.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.691+0000 7fd560159b80 1 -- 172.21.15.101:0/104813510 wait complete. 2023-07-18T03:15:41.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.691+0000 7fd560159b80 20 client.27265 trim_cache size 0 max 16384 2023-07-18T03:15:41.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.705+0000 7fd560159b80 1 Processor -- start 2023-07-18T03:15:41.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.705+0000 7fd560159b80 1 -- start start 2023-07-18T03:15:41.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.705+0000 7fd560159b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9614ac2d0 0x55f9614ac6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.705+0000 7fd560159b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f9614acbe0 0x55f9614c3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.705+0000 7fd560159b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f9614b7ad0 0x55f9614afaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.705+0000 7fd560159b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9614a2970 con 0x55f9614acbe0 2023-07-18T03:15:41.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.705+0000 7fd560159b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9614ae1a0 con 0x55f9614ac2d0 2023-07-18T03:15:41.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.705+0000 7fd560159b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9614c3c80 con 0x55f9614b7ad0 2023-07-18T03:15:41.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.705+0000 7fd54a59b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9614ac2d0 0x55f9614ac6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.705+0000 7fd54a59b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9614ac2d0 0x55f9614ac6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:35574/0 (socket says 172.21.15.101:35574) 2023-07-18T03:15:41.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.705+0000 7fd54a59b700 1 -- 172.21.15.101:0/1036658430 learned_addr learned my addr 172.21.15.101:0/1036658430 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:41.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.705+0000 7fd54ad9c700 1 --2- 172.21.15.101:0/1036658430 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f9614b7ad0 0x55f9614afaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.705+0000 7fd549d9a700 1 --2- 172.21.15.101:0/1036658430 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f9614acbe0 0x55f9614c3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.706+0000 7fd54a59b700 1 -- 172.21.15.101:0/1036658430 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f9614b7ad0 msgr2=0x55f9614afaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:41.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.706+0000 7fd54a59b700 1 --2- 172.21.15.101:0/1036658430 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f9614b7ad0 0x55f9614afaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.706+0000 7fd54a59b700 1 -- 172.21.15.101:0/1036658430 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f9614acbe0 msgr2=0x55f9614c3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:41.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.706+0000 7fd54a59b700 1 --2- 172.21.15.101:0/1036658430 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f9614acbe0 0x55f9614c3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.706+0000 7fd54a59b700 1 -- 172.21.15.101:0/1036658430 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f961215c60 con 0x55f9614ac2d0 2023-07-18T03:15:41.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.706+0000 7fd54ad9c700 1 --2- 172.21.15.101:0/1036658430 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f9614b7ad0 0x55f9614afaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:15:41.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.706+0000 7fd54a59b700 1 --2- 172.21.15.101:0/1036658430 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9614ac2d0 0x55f9614ac6a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd540000c00 tx=0x7fd540005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b11a8a9cc69d2b3d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:41.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.706+0000 7fd5397fa700 1 -- 172.21.15.101:0/1036658430 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54001d040 con 0x55f9614ac2d0 2023-07-18T03:15:41.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.706+0000 7fd5397fa700 1 -- 172.21.15.101:0/1036658430 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd540005bc0 con 0x55f9614ac2d0 2023-07-18T03:15:41.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.706+0000 7fd5397fa700 1 -- 172.21.15.101:0/1036658430 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd540033bc0 con 0x55f9614ac2d0 2023-07-18T03:15:41.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.707+0000 7fd560159b80 1 -- 172.21.15.101:0/1036658430 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9614ac2d0 msgr2=0x55f9614ac6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:41.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.707+0000 7fd560159b80 1 --2- 172.21.15.101:0/1036658430 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9614ac2d0 0x55f9614ac6a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd540000c00 tx=0x7fd540005fb0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.707+0000 7fd560159b80 1 -- 172.21.15.101:0/1036658430 shutdown_connections 2023-07-18T03:15:41.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.707+0000 7fd560159b80 1 --2- 172.21.15.101:0/1036658430 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9614ac2d0 0x55f9614ac6a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.707+0000 7fd560159b80 1 --2- 172.21.15.101:0/1036658430 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f9614b7ad0 0x55f9614afaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.707+0000 7fd560159b80 1 --2- 172.21.15.101:0/1036658430 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f9614acbe0 0x55f9614c3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.707+0000 7fd560159b80 1 -- 172.21.15.101:0/1036658430 >> 172.21.15.101:0/1036658430 conn(0x55f9614a2da0 msgr2=0x55f9614ce160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:41.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.707+0000 7fd560159b80 1 -- 172.21.15.101:0/1036658430 shutdown_connections 2023-07-18T03:15:41.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.707+0000 7fd560159b80 1 -- 172.21.15.101:0/1036658430 wait complete. 2023-07-18T03:15:41.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.707+0000 7fd560159b80 1 -- start start 2023-07-18T03:15:41.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.708+0000 7fd560159b80 1 Processor -- start 2023-07-18T03:15:41.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.708+0000 7fd560159b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9614ac2d0 0x55f9614bc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.708+0000 7fd560159b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f9614acbe0 0x55f9614bc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.708+0000 7fd560159b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f9614b7ad0 0x55f96151a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.708+0000 7fd560159b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f96151a7f0 con 0x55f9614b7ad0 2023-07-18T03:15:41.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.708+0000 7fd560159b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f96151a930 con 0x55f9614ac2d0 2023-07-18T03:15:41.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.708+0000 7fd560159b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f96151aa70 con 0x55f9614acbe0 2023-07-18T03:15:41.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.708+0000 7fd560159b80 10 client.0 _ll_register_callbacks cb 0x55f96139c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:15:41.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.709+0000 7fd54a59b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9614ac2d0 0x55f9614bc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.709+0000 7fd549d9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f9614acbe0 0x55f9614bc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.709+0000 7fd54a59b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9614ac2d0 0x55f9614bc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:35590/0 (socket says 172.21.15.101:35590) 2023-07-18T03:15:41.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.709+0000 7fd54a59b700 1 -- 172.21.15.101:0/928561814 learned_addr learned my addr 172.21.15.101:0/928561814 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:41.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.709+0000 7fd54ad9c700 1 --2- 172.21.15.101:0/928561814 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f9614b7ad0 0x55f96151a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.709+0000 7fd549d9a700 1 -- 172.21.15.101:0/928561814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9614ac2d0 msgr2=0x55f9614bc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:41.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.709+0000 7fd549d9a700 1 --2- 172.21.15.101:0/928561814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9614ac2d0 0x55f9614bc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.709+0000 7fd549d9a700 1 -- 172.21.15.101:0/928561814 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f9614b7ad0 msgr2=0x55f96151a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:41.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.709+0000 7fd549d9a700 1 --2- 172.21.15.101:0/928561814 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f9614b7ad0 0x55f96151a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.709+0000 7fd549d9a700 1 -- 172.21.15.101:0/928561814 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd53c019e70 con 0x55f9614acbe0 2023-07-18T03:15:41.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.709+0000 7fd54a59b700 1 --2- 172.21.15.101:0/928561814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9614ac2d0 0x55f9614bc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:15:41.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.709+0000 7fd549d9a700 1 --2- 172.21.15.101:0/928561814 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f9614acbe0 0x55f9614bc770 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd5340ca620 tx=0x7fd534096410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98e8891eb164e85a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:41.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.709+0000 7fd52dffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:15:41.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.709+0000 7fd52dffb700 1 -- 172.21.15.101:0/928561814 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53406a460 con 0x55f9614acbe0 2023-07-18T03:15:41.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.709+0000 7fd52dffb700 1 -- 172.21.15.101:0/928561814 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fd53409dfd0 con 0x55f9614acbe0 2023-07-18T03:15:41.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.709+0000 7fd560159b80 1 -- 172.21.15.101:0/928561814 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f9613ba050 con 0x55f9614acbe0 2023-07-18T03:15:41.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.710+0000 7fd52dffb700 1 -- 172.21.15.101:0/928561814 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5340a23f0 con 0x55f9614acbe0 2023-07-18T03:15:41.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.710+0000 7fd560159b80 10 client.27271 fetch_fsmap learned FSMap version 29 2023-07-18T03:15:41.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.710+0000 7fd560159b80 1 -- 172.21.15.101:0/928561814 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f96122cde0 con 0x55f9614acbe0 2023-07-18T03:15:41.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.710+0000 7fd52dffb700 1 -- 172.21.15.101:0/928561814 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5340b6020 con 0x55f9614acbe0 2023-07-18T03:15:41.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.710+0000 7fd52dffb700 1 -- 172.21.15.101:0/928561814 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fd534098f00 con 0x55f9614acbe0 2023-07-18T03:15:41.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.710+0000 7fd560159b80 10 client.27271 fetch_fsmap finished waiting for FSMap version 29 2023-07-18T03:15:41.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.710+0000 7fd560159b80 10 client.27271 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:15:41.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.710+0000 7fd560159b80 20 client.27271 populate_metadata read hostname 'smithi101' 2023-07-18T03:15:41.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.710+0000 7fd560159b80 1 --2- 172.21.15.101:0/928561814 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f96151c300 0x55f96151e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:41.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.710+0000 7fd560159b80 4 client.27271 mds_command: new command op to 24562 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:15:41.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.710+0000 7fd560159b80 1 -- 172.21.15.101:0/928561814 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f96124df70 con 0x55f96151c300 2023-07-18T03:15:41.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.710+0000 7fd54a59b700 1 --2- 172.21.15.101:0/928561814 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f96151c300 0x55f96151e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:41.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.711+0000 7fd54a59b700 1 --2- 172.21.15.101:0/928561814 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f96151c300 0x55f96151e710 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7956b20c702da1d2 server_cookie=c74f77fa4760f932 in_seq=0 out_seq=0 2023-07-18T03:15:41.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.711+0000 7fd52dffb700 10 client.27271 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:15:41.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.711+0000 7fd52dffb700 1 -- 172.21.15.101:0/928561814 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f96124df70 con 0x55f96151c300 2023-07-18T03:15:41.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.711+0000 7fd52dffb700 10 client.27271 handle_command_reply: tid=0 2023-07-18T03:15:41.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.711+0000 7fd560159b80 1 client.27271 shutdown 2023-07-18T03:15:41.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.711+0000 7fd560159b80 1 -- 172.21.15.101:0/928561814 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f9614acbe0 msgr2=0x55f9614bc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:41.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.711+0000 7fd560159b80 1 --2- 172.21.15.101:0/928561814 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f9614acbe0 0x55f9614bc770 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd5340ca620 tx=0x7fd534096410 comp rx=0 tx=0).stop 2023-07-18T03:15:41.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.711+0000 7fd560159b80 1 -- 172.21.15.101:0/928561814 shutdown_connections 2023-07-18T03:15:41.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.711+0000 7fd560159b80 1 --2- 172.21.15.101:0/928561814 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f96151c300 0x55f96151e710 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.711+0000 7fd560159b80 1 --2- 172.21.15.101:0/928561814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9614ac2d0 0x55f9614bc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.711+0000 7fd560159b80 1 --2- 172.21.15.101:0/928561814 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f9614b7ad0 0x55f96151a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.711+0000 7fd560159b80 1 --2- 172.21.15.101:0/928561814 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f9614acbe0 0x55f9614bc770 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.711+0000 7fd560159b80 1 -- 172.21.15.101:0/928561814 >> 172.21.15.101:0/928561814 conn(0x55f9614a2da0 msgr2=0x55f9614afaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:41.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.711+0000 7fd52dffb700 0 client.27271 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:15:41.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.711+0000 7fd560159b80 1 -- 172.21.15.101:0/928561814 shutdown_connections 2023-07-18T03:15:41.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.711+0000 7fd560159b80 1 -- 172.21.15.101:0/928561814 wait complete. 2023-07-18T03:15:41.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.712+0000 7fd560159b80 20 client.27271 trim_cache size 0 max 16384 2023-07-18T03:15:41.716 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:15:41.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.712+0000 7fd54c7ff700 1 -- 172.21.15.101:0/3225015038 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd5300a12c0 msgr2=0x7fd5300a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:41.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.712+0000 7fd54c7ff700 1 --2- 172.21.15.101:0/3225015038 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd5300a12c0 0x7fd5300a3770 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd53c009ec0 tx=0x7fd53c00b040 comp rx=0 tx=0).stop 2023-07-18T03:15:41.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.712+0000 7fd54c7ff700 1 -- 172.21.15.101:0/3225015038 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd544100490 msgr2=0x7fd5441983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:41.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.712+0000 7fd54c7ff700 1 --2- 172.21.15.101:0/3225015038 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd544100490 0x7fd5441983b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd53400e9a0 tx=0x7fd53400ecb0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.712+0000 7fd54c7ff700 1 -- 172.21.15.101:0/3225015038 shutdown_connections 2023-07-18T03:15:41.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.712+0000 7fd54c7ff700 1 --2- 172.21.15.101:0/3225015038 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd5300a12c0 0x7fd5300a3770 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.712+0000 7fd54c7ff700 1 --2- 172.21.15.101:0/3225015038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd544105af0 0x7fd54419c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.712+0000 7fd54c7ff700 1 --2- 172.21.15.101:0/3225015038 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5440ffae0 0x7fd544197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.712+0000 7fd54c7ff700 1 --2- 172.21.15.101:0/3225015038 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd544100490 0x7fd5441983b0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:41.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.712+0000 7fd54c7ff700 1 -- 172.21.15.101:0/3225015038 >> 172.21.15.101:0/3225015038 conn(0x7fd5440fb5e0 msgr2=0x7fd54410adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:41.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.713+0000 7fd54c7ff700 1 -- 172.21.15.101:0/3225015038 shutdown_connections 2023-07-18T03:15:41.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:41.713+0000 7fd54c7ff700 1 -- 172.21.15.101:0/3225015038 wait complete. 2023-07-18T03:15:42.595 INFO:tasks.workunit.client.0.smithi101.stdout: 1 555852 6.63 MB/sec execute 307 sec latency 602.231 ms 2023-07-18T03:15:42.725 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-18T03:15:42.726 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-18T03:15:42.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.944+0000 7f86569ff700 1 -- 172.21.15.101:0/3162341547 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f86500ffad0 msgr2=0x7f86500fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:42.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.944+0000 7f86569ff700 1 --2- 172.21.15.101:0/3162341547 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f86500ffad0 0x7f86500fff40 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f86380098f0 tx=0x7f8638009c00 comp rx=0 tx=0).stop 2023-07-18T03:15:42.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.944+0000 7f86569ff700 1 -- 172.21.15.101:0/3162341547 shutdown_connections 2023-07-18T03:15:42.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.944+0000 7f86569ff700 1 --2- 172.21.15.101:0/3162341547 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8650105ae0 0x7f8650105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:42.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.944+0000 7f86569ff700 1 --2- 172.21.15.101:0/3162341547 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8650100480 0x7f865010c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:42.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.944+0000 7f86569ff700 1 --2- 172.21.15.101:0/3162341547 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f86500ffad0 0x7f86500fff40 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:42.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.944+0000 7f86569ff700 1 -- 172.21.15.101:0/3162341547 >> 172.21.15.101:0/3162341547 conn(0x7f86500fb5f0 msgr2=0x7f86500fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:42.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.944+0000 7f86569ff700 1 -- 172.21.15.101:0/3162341547 shutdown_connections 2023-07-18T03:15:42.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.944+0000 7f86569ff700 1 -- 172.21.15.101:0/3162341547 wait complete. 2023-07-18T03:15:42.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.945+0000 7f86569ff700 1 Processor -- start 2023-07-18T03:15:42.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.945+0000 7f86569ff700 1 -- start start 2023-07-18T03:15:42.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.946+0000 7f86569ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f86500ffad0 0x7f8650072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:42.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.946+0000 7f86569ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8650100480 0x7f8650072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:42.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.946+0000 7f86569ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8650105ae0 0x7f86501a3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:42.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.946+0000 7f86569ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8650073580 con 0x7f86500ffad0 2023-07-18T03:15:42.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.946+0000 7f86569ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86501a43e0 con 0x7f8650100480 2023-07-18T03:15:42.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.946+0000 7f86569ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86501a4520 con 0x7f8650105ae0 2023-07-18T03:15:42.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.946+0000 7f864ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f86500ffad0 0x7f8650072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:42.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.946+0000 7f8647fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8650100480 0x7f8650072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:42.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.946+0000 7f8654f9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8650105ae0 0x7f86501a3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:42.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.946+0000 7f864ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f86500ffad0 0x7f8650072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:44032/0 (socket says 172.21.15.101:44032) 2023-07-18T03:15:42.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.946+0000 7f8647fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8650100480 0x7f8650072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:35614/0 (socket says 172.21.15.101:35614) 2023-07-18T03:15:42.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.946+0000 7f864ffff700 1 -- 172.21.15.101:0/3575614998 learned_addr learned my addr 172.21.15.101:0/3575614998 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:42.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.946+0000 7f8647fff700 1 -- 172.21.15.101:0/3575614998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8650105ae0 msgr2=0x7f86501a3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:42.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.946+0000 7f8647fff700 1 --2- 172.21.15.101:0/3575614998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8650105ae0 0x7f86501a3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:42.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.946+0000 7f8647fff700 1 -- 172.21.15.101:0/3575614998 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f86500ffad0 msgr2=0x7f8650072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:42.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.946+0000 7f8647fff700 1 --2- 172.21.15.101:0/3575614998 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f86500ffad0 0x7f8650072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:42.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.946+0000 7f8647fff700 1 -- 172.21.15.101:0/3575614998 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86400095f0 con 0x7f8650100480 2023-07-18T03:15:42.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.946+0000 7f864ffff700 1 --2- 172.21.15.101:0/3575614998 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f86500ffad0 0x7f8650072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:15:42.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.947+0000 7f8647fff700 1 --2- 172.21.15.101:0/3575614998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8650100480 0x7f8650072d50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8638009810 tx=0x7f863800fa50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:42.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.947+0000 7f864dffb700 1 -- 172.21.15.101:0/3575614998 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863801c070 con 0x7f8650100480 2023-07-18T03:15:42.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.947+0000 7f86569ff700 1 -- 172.21.15.101:0/3575614998 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8638009520 con 0x7f8650100480 2023-07-18T03:15:42.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.947+0000 7f86569ff700 1 -- 172.21.15.101:0/3575614998 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86501a4a10 con 0x7f8650100480 2023-07-18T03:15:42.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.947+0000 7f864dffb700 1 -- 172.21.15.101:0/3575614998 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f863800bba0 con 0x7f8650100480 2023-07-18T03:15:42.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.947+0000 7f864dffb700 1 -- 172.21.15.101:0/3575614998 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8638003970 con 0x7f8650100480 2023-07-18T03:15:42.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.950+0000 7f864dffb700 1 -- 172.21.15.101:0/3575614998 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f8638020030 con 0x7f8650100480 2023-07-18T03:15:42.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.951+0000 7f866a246b80 1 Processor -- start 2023-07-18T03:15:42.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.952+0000 7f866a246b80 1 -- start start 2023-07-18T03:15:42.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.952+0000 7f866a246b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee39003350 0x55ee39003720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:42.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.952+0000 7f866a246b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ee39003c60 0x55ee3900e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:42.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.952+0000 7f866a246b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee3900eb50 0x55ee39010f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:42.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.952+0000 7f864dffb700 1 --2- 172.21.15.101:0/3575614998 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f863c0a12c0 0x7f863c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:42.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.952+0000 7f866a246b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee38f09d30 con 0x55ee39003c60 2023-07-18T03:15:42.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.952+0000 7f866a246b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee38f5cac0 con 0x55ee3900eb50 2023-07-18T03:15:42.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.952+0000 7f866a246b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee38f55f50 con 0x55ee39003350 2023-07-18T03:15:42.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.952+0000 7f864dffb700 1 -- 172.21.15.101:0/3575614998 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f86380cdfc0 con 0x7f8650100480 2023-07-18T03:15:42.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.952+0000 7f8654f9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee39003350 0x55ee39003720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:42.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.952+0000 7f864ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ee39003c60 0x55ee3900e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:42.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.952+0000 7f8654f9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee39003350 0x55ee39003720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:51314/0 (socket says 172.21.15.101:51314) 2023-07-18T03:15:42.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.952+0000 7f8654f9c700 1 -- 172.21.15.101:0/4196994618 learned_addr learned my addr 172.21.15.101:0/4196994618 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:42.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.952+0000 7f8647fff700 1 --2- 172.21.15.101:0/3575614998 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f863c0a12c0 0x7f863c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:42.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.952+0000 7f8654f9c700 1 --2- 172.21.15.101:0/4196994618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee3900eb50 0x55ee39010f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:42.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.953+0000 7f8654f9c700 1 -- 172.21.15.101:0/4196994618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee3900eb50 msgr2=0x55ee39010f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:42.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.953+0000 7f8654f9c700 1 --2- 172.21.15.101:0/4196994618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee3900eb50 0x55ee39010f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:42.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.953+0000 7f8654f9c700 1 -- 172.21.15.101:0/4196994618 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ee39003c60 msgr2=0x55ee3900e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:42.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.953+0000 7f8654f9c700 1 --2- 172.21.15.101:0/4196994618 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ee39003c60 0x55ee3900e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:42.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.953+0000 7f8654f9c700 1 -- 172.21.15.101:0/4196994618 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee38d83e60 con 0x55ee39003350 2023-07-18T03:15:42.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.953+0000 7f8647fff700 1 --2- 172.21.15.101:0/3575614998 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f863c0a12c0 0x7f863c0a3770 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8638097210 tx=0x7f86380055c0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:42.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.953+0000 7f8654f9c700 1 --2- 172.21.15.101:0/4196994618 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee39003350 0x55ee39003720 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8648009790 tx=0x7f864800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c70a9fb5f8f4136 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:42.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.953+0000 7f86467fc700 1 -- 172.21.15.101:0/4196994618 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8648004bb0 con 0x55ee39003350 2023-07-18T03:15:42.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.953+0000 7f86467fc700 1 -- 172.21.15.101:0/4196994618 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f864800f070 con 0x55ee39003350 2023-07-18T03:15:42.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.953+0000 7f86467fc700 1 -- 172.21.15.101:0/4196994618 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8648017c40 con 0x55ee39003350 2023-07-18T03:15:42.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.954+0000 7f866a246b80 1 -- 172.21.15.101:0/4196994618 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee39003350 msgr2=0x55ee39003720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:42.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.954+0000 7f866a246b80 1 --2- 172.21.15.101:0/4196994618 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee39003350 0x55ee39003720 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8648009790 tx=0x7f864800b040 comp rx=0 tx=0).stop 2023-07-18T03:15:42.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.954+0000 7f866a246b80 1 -- 172.21.15.101:0/4196994618 shutdown_connections 2023-07-18T03:15:42.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.954+0000 7f866a246b80 1 --2- 172.21.15.101:0/4196994618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee3900eb50 0x55ee39010f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:42.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.954+0000 7f866a246b80 1 --2- 172.21.15.101:0/4196994618 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee39003350 0x55ee39003720 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:42.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.954+0000 7f866a246b80 1 --2- 172.21.15.101:0/4196994618 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ee39003c60 0x55ee3900e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:42.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.954+0000 7f866a246b80 1 -- 172.21.15.101:0/4196994618 >> 172.21.15.101:0/4196994618 conn(0x55ee38ff9e20 msgr2=0x55ee38fff720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:42.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.954+0000 7f866a246b80 1 -- 172.21.15.101:0/4196994618 shutdown_connections 2023-07-18T03:15:42.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.954+0000 7f866a246b80 1 -- 172.21.15.101:0/4196994618 wait complete. 2023-07-18T03:15:42.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.955+0000 7f866a246b80 1 -- start start 2023-07-18T03:15:42.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.955+0000 7f866a246b80 1 Processor -- start 2023-07-18T03:15:42.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.955+0000 7f866a246b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee39003350 0x55ee3900e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:42.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.955+0000 7f866a246b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee39003c60 0x55ee3900b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:42.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.955+0000 7f866a246b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ee3900eb50 0x55ee3900bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:42.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.955+0000 7f866a246b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee38ef5b40 con 0x55ee3900eb50 2023-07-18T03:15:42.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.955+0000 7f866a246b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee38ef5840 con 0x55ee39003c60 2023-07-18T03:15:42.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.955+0000 7f866a246b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee38dd99a0 con 0x55ee39003350 2023-07-18T03:15:42.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.955+0000 7f866a246b80 10 client.0 _ll_register_callbacks cb 0x55ee38ef37e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:15:42.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.956+0000 7f8654f9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee39003350 0x55ee3900e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:42.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.956+0000 7f8654f9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ee3900eb50 0x55ee3900bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:42.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.956+0000 7f864ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee39003c60 0x55ee3900b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:42.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.956+0000 7f8654f9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ee3900eb50 0x55ee3900bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:44048/0 (socket says 172.21.15.101:44048) 2023-07-18T03:15:42.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.956+0000 7f8654f9c700 1 -- 172.21.15.101:0/2685708695 learned_addr learned my addr 172.21.15.101:0/2685708695 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:42.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.956+0000 7f864ffff700 1 -- 172.21.15.101:0/2685708695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee39003350 msgr2=0x55ee3900e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:42.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.956+0000 7f864ffff700 1 --2- 172.21.15.101:0/2685708695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee39003350 0x55ee3900e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:42.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.956+0000 7f864ffff700 1 -- 172.21.15.101:0/2685708695 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ee3900eb50 msgr2=0x55ee3900bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:42.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.956+0000 7f864ffff700 1 --2- 172.21.15.101:0/2685708695 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ee3900eb50 0x55ee3900bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:42.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.956+0000 7f864ffff700 1 -- 172.21.15.101:0/2685708695 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8648011040 con 0x55ee39003c60 2023-07-18T03:15:42.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.956+0000 7f8654f9c700 1 --2- 172.21.15.101:0/2685708695 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ee3900eb50 0x55ee3900bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:15:42.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.956+0000 7f864ffff700 1 --2- 172.21.15.101:0/2685708695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee39003c60 0x55ee3900b770 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8640005d10 tx=0x7f864000f6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=417a9db3ac25ba26 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:42.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.956+0000 7f862f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:15:42.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.957+0000 7f862f7fe700 1 -- 172.21.15.101:0/2685708695 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8640017070 con 0x55ee39003c60 2023-07-18T03:15:42.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.957+0000 7f866a246b80 1 -- 172.21.15.101:0/2685708695 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ee38eaf570 con 0x55ee39003c60 2023-07-18T03:15:42.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.957+0000 7f862f7fe700 1 -- 172.21.15.101:0/2685708695 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f864000fb80 con 0x55ee39003c60 2023-07-18T03:15:42.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.957+0000 7f862f7fe700 1 -- 172.21.15.101:0/2685708695 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f864000fd40 con 0x55ee39003c60 2023-07-18T03:15:42.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.957+0000 7f866a246b80 10 client.27248 fetch_fsmap learned FSMap version 29 2023-07-18T03:15:42.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.957+0000 7f866a246b80 1 -- 172.21.15.101:0/2685708695 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ee38d6cce0 con 0x55ee39003c60 2023-07-18T03:15:42.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.957+0000 7f862f7fe700 1 -- 172.21.15.101:0/2685708695 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ee38eaf570 con 0x55ee39003c60 2023-07-18T03:15:42.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.957+0000 7f862f7fe700 1 -- 172.21.15.101:0/2685708695 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f8640003bf0 con 0x55ee39003c60 2023-07-18T03:15:42.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.957+0000 7f866a246b80 10 client.27248 fetch_fsmap finished waiting for FSMap version 29 2023-07-18T03:15:42.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.957+0000 7f866a246b80 10 client.27248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:15:42.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.957+0000 7f866a246b80 20 client.27248 populate_metadata read hostname 'smithi101' 2023-07-18T03:15:42.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.957+0000 7f866a246b80 1 --2- 172.21.15.101:0/2685708695 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ee3901e560 0x55ee39020970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:42.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.957+0000 7f866a246b80 4 client.27248 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:15:42.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.957+0000 7f866a246b80 1 -- 172.21.15.101:0/2685708695 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ee38f4ef30 con 0x55ee3901e560 2023-07-18T03:15:42.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.958+0000 7f8654f9c700 1 --2- 172.21.15.101:0/2685708695 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ee3901e560 0x55ee39020970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:42.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.958+0000 7f8654f9c700 1 --2- 172.21.15.101:0/2685708695 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ee3901e560 0x55ee39020970 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=520438985e2fc04f server_cookie=a320da62f7427e23 in_seq=0 out_seq=0 2023-07-18T03:15:42.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.958+0000 7f862f7fe700 10 client.27248 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:15:42.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.959+0000 7f862f7fe700 1 -- 172.21.15.101:0/2685708695 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ee38f4ef30 con 0x55ee3901e560 2023-07-18T03:15:42.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.959+0000 7f862f7fe700 10 client.27248 handle_command_reply: tid=0 2023-07-18T03:15:42.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.959+0000 7f866a246b80 1 client.27248 shutdown 2023-07-18T03:15:42.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.959+0000 7f866a246b80 1 -- 172.21.15.101:0/2685708695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee39003c60 msgr2=0x55ee3900b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:42.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.959+0000 7f866a246b80 1 --2- 172.21.15.101:0/2685708695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee39003c60 0x55ee3900b770 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8640005d10 tx=0x7f864000f6a0 comp rx=0 tx=0).stop 2023-07-18T03:15:42.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.959+0000 7f866a246b80 1 -- 172.21.15.101:0/2685708695 shutdown_connections 2023-07-18T03:15:42.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.959+0000 7f866a246b80 1 --2- 172.21.15.101:0/2685708695 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ee3901e560 0x55ee39020970 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:42.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.959+0000 7f866a246b80 1 --2- 172.21.15.101:0/2685708695 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ee3900eb50 0x55ee3900bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:42.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.959+0000 7f862f7fe700 0 client.27248 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:15:42.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.959+0000 7f866a246b80 1 --2- 172.21.15.101:0/2685708695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee39003350 0x55ee3900e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:42.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.959+0000 7f866a246b80 1 --2- 172.21.15.101:0/2685708695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee39003c60 0x55ee3900b770 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:42.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.959+0000 7f866a246b80 1 -- 172.21.15.101:0/2685708695 >> 172.21.15.101:0/2685708695 conn(0x55ee38ff9e20 msgr2=0x55ee38fff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:42.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.959+0000 7f866a246b80 1 -- 172.21.15.101:0/2685708695 shutdown_connections 2023-07-18T03:15:42.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.959+0000 7f866a246b80 1 -- 172.21.15.101:0/2685708695 wait complete. 2023-07-18T03:15:42.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.959+0000 7f866a246b80 20 client.27248 trim_cache size 0 max 16384 2023-07-18T03:15:42.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.976+0000 7f866a246b80 1 Processor -- start 2023-07-18T03:15:42.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.977+0000 7f866a246b80 1 -- start start 2023-07-18T03:15:42.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.977+0000 7f866a246b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee39003350 0x55ee39015570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:42.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.977+0000 7f866a246b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ee39003c60 0x55ee39005b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:42.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.977+0000 7f866a246b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee3900eb50 0x55ee3900e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:42.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.977+0000 7f866a246b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee38ff99f0 con 0x55ee39003c60 2023-07-18T03:15:42.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.977+0000 7f866a246b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee3900c380 con 0x55ee39003350 2023-07-18T03:15:42.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.977+0000 7f866a246b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee39005220 con 0x55ee3900eb50 2023-07-18T03:15:42.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.977+0000 7f864ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ee39003c60 0x55ee39005b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:42.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.977+0000 7f864ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ee39003c60 0x55ee39005b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:44056/0 (socket says 172.21.15.101:44056) 2023-07-18T03:15:42.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.977+0000 7f864ffff700 1 -- 172.21.15.101:0/2783711284 learned_addr learned my addr 172.21.15.101:0/2783711284 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:42.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.977+0000 7f8654f9c700 1 --2- 172.21.15.101:0/2783711284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee39003350 0x55ee39015570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:42.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.977+0000 7f8654f9c700 1 --2- 172.21.15.101:0/2783711284 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee3900eb50 0x55ee3900e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:42.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.977+0000 7f864ffff700 1 -- 172.21.15.101:0/2783711284 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee3900eb50 msgr2=0x55ee3900e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:42.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.977+0000 7f864ffff700 1 --2- 172.21.15.101:0/2783711284 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee3900eb50 0x55ee3900e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:42.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.977+0000 7f864ffff700 1 -- 172.21.15.101:0/2783711284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee39003350 msgr2=0x55ee39015570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:42.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.977+0000 7f864ffff700 1 --2- 172.21.15.101:0/2783711284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee39003350 0x55ee39015570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:42.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.977+0000 7f864ffff700 1 -- 172.21.15.101:0/2783711284 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee38daa2d0 con 0x55ee39003c60 2023-07-18T03:15:42.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.977+0000 7f8654f9c700 1 --2- 172.21.15.101:0/2783711284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee39003350 0x55ee39015570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:15:42.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.978+0000 7f864ffff700 1 --2- 172.21.15.101:0/2783711284 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ee39003c60 0x55ee39005b70 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f86400105a0 tx=0x7f8640012420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecaf18ef04634700 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:42.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.978+0000 7f8646ffd700 1 -- 172.21.15.101:0/2783711284 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8640017070 con 0x55ee39003c60 2023-07-18T03:15:42.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.978+0000 7f8646ffd700 1 -- 172.21.15.101:0/2783711284 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86400106f0 con 0x55ee39003c60 2023-07-18T03:15:42.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.978+0000 7f8646ffd700 1 -- 172.21.15.101:0/2783711284 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8640005260 con 0x55ee39003c60 2023-07-18T03:15:42.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.978+0000 7f866a246b80 1 -- 172.21.15.101:0/2783711284 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ee39003c60 msgr2=0x55ee39005b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:42.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.978+0000 7f866a246b80 1 --2- 172.21.15.101:0/2783711284 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ee39003c60 0x55ee39005b70 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f86400105a0 tx=0x7f8640012420 comp rx=0 tx=0).stop 2023-07-18T03:15:42.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.978+0000 7f866a246b80 1 -- 172.21.15.101:0/2783711284 shutdown_connections 2023-07-18T03:15:42.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.978+0000 7f866a246b80 1 --2- 172.21.15.101:0/2783711284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee39003350 0x55ee39015570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:42.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.978+0000 7f866a246b80 1 --2- 172.21.15.101:0/2783711284 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee3900eb50 0x55ee3900e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:42.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.978+0000 7f866a246b80 1 --2- 172.21.15.101:0/2783711284 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ee39003c60 0x55ee39005b70 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:42.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.978+0000 7f866a246b80 1 -- 172.21.15.101:0/2783711284 >> 172.21.15.101:0/2783711284 conn(0x55ee38ff9e20 msgr2=0x55ee39005460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:42.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.979+0000 7f866a246b80 1 -- 172.21.15.101:0/2783711284 shutdown_connections 2023-07-18T03:15:42.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.979+0000 7f866a246b80 1 -- 172.21.15.101:0/2783711284 wait complete. 2023-07-18T03:15:42.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.979+0000 7f866a246b80 1 -- start start 2023-07-18T03:15:42.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.979+0000 7f866a246b80 1 Processor -- start 2023-07-18T03:15:42.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.979+0000 7f866a246b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee39003350 0x55ee3906cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:42.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.980+0000 7f866a246b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee39003c60 0x55ee3906d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:42.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.980+0000 7f866a246b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ee3900eb50 0x55ee39014850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:42.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.980+0000 7f866a246b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee39014e60 con 0x55ee3900eb50 2023-07-18T03:15:42.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.980+0000 7f866a246b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee39014fd0 con 0x55ee39003c60 2023-07-18T03:15:42.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.980+0000 7f866a246b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee39015190 con 0x55ee39003350 2023-07-18T03:15:42.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.980+0000 7f866a246b80 10 client.0 _ll_register_callbacks cb 0x7f8640012830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:15:42.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.980+0000 7f8654f9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee39003350 0x55ee3906cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:42.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.980+0000 7f864ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee39003c60 0x55ee3906d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:42.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.980+0000 7f8654f9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee39003350 0x55ee3906cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:51326/0 (socket says 172.21.15.101:51326) 2023-07-18T03:15:42.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.980+0000 7f864ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee39003c60 0x55ee3906d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:35658/0 (socket says 172.21.15.101:35658) 2023-07-18T03:15:42.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.980+0000 7f8654f9c700 1 -- 172.21.15.101:0/286815767 learned_addr learned my addr 172.21.15.101:0/286815767 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:42.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.980+0000 7f8654f9c700 1 --2- 172.21.15.101:0/286815767 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ee3900eb50 0x55ee39014850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:42.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.980+0000 7f864ffff700 1 -- 172.21.15.101:0/286815767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee39003350 msgr2=0x55ee3906cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:42.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.980+0000 7f864ffff700 1 --2- 172.21.15.101:0/286815767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee39003350 0x55ee3906cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:42.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.980+0000 7f864ffff700 1 -- 172.21.15.101:0/286815767 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ee3900eb50 msgr2=0x55ee39014850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:42.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.980+0000 7f864ffff700 1 --2- 172.21.15.101:0/286815767 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ee3900eb50 0x55ee39014850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:42.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.980+0000 7f864ffff700 1 -- 172.21.15.101:0/286815767 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8640021040 con 0x55ee39003c60 2023-07-18T03:15:42.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.981+0000 7f864ffff700 1 --2- 172.21.15.101:0/286815767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee39003c60 0x55ee3906d0f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8640003f80 tx=0x7f864000fdb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b99ae387e9499fa server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:42.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.981+0000 7f862f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:15:42.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.981+0000 7f862f7fe700 1 -- 172.21.15.101:0/286815767 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8640017030 con 0x55ee39003c60 2023-07-18T03:15:42.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.981+0000 7f866a246b80 1 -- 172.21.15.101:0/286815767 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f864800f070 con 0x55ee39003c60 2023-07-18T03:15:42.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.981+0000 7f862f7fe700 1 -- 172.21.15.101:0/286815767 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f8640010b80 con 0x55ee39003c60 2023-07-18T03:15:42.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.985+0000 7f862f7fe700 1 -- 172.21.15.101:0/286815767 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8640010d80 con 0x55ee39003c60 2023-07-18T03:15:42.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.985+0000 7f866a246b80 10 client.27254 fetch_fsmap learned FSMap version 29 2023-07-18T03:15:42.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.985+0000 7f866a246b80 1 -- 172.21.15.101:0/286815767 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ee38d82330 con 0x55ee39003c60 2023-07-18T03:15:42.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.985+0000 7f862f7fe700 1 -- 172.21.15.101:0/286815767 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f864800f070 con 0x55ee39003c60 2023-07-18T03:15:42.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.985+0000 7f862f7fe700 1 -- 172.21.15.101:0/286815767 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f8640039050 con 0x55ee39003c60 2023-07-18T03:15:42.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.985+0000 7f866a246b80 10 client.27254 fetch_fsmap finished waiting for FSMap version 29 2023-07-18T03:15:42.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.985+0000 7f866a246b80 10 client.27254 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:15:42.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.985+0000 7f866a246b80 20 client.27254 populate_metadata read hostname 'smithi101' 2023-07-18T03:15:42.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.985+0000 7f866a246b80 1 --2- 172.21.15.101:0/286815767 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ee39074690 0x55ee39076aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:42.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.985+0000 7f866a246b80 4 client.27254 mds_command: new command op to 24562 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-18T03:15:42.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.985+0000 7f866a246b80 1 -- 172.21.15.101:0/286815767 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55ee38f110d0 con 0x55ee39074690 2023-07-18T03:15:42.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.986+0000 7f8654f9c700 1 --2- 172.21.15.101:0/286815767 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ee39074690 0x55ee39076aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:42.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.986+0000 7f862f7fe700 10 client.27254 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:15:42.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.986+0000 7f8654f9c700 1 --2- 172.21.15.101:0/286815767 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ee39074690 0x55ee39076aa0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83586fe338d3d2e3 server_cookie=947a0e5960a25637 in_seq=0 out_seq=0 2023-07-18T03:15:42.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.987+0000 7f862f7fe700 1 -- 172.21.15.101:0/286815767 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ee38f110d0 con 0x55ee39074690 2023-07-18T03:15:42.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.987+0000 7f862f7fe700 10 client.27254 handle_command_reply: tid=0 2023-07-18T03:15:42.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.987+0000 7f866a246b80 1 client.27254 shutdown 2023-07-18T03:15:42.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.987+0000 7f866a246b80 1 -- 172.21.15.101:0/286815767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee39003c60 msgr2=0x55ee3906d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:42.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.987+0000 7f866a246b80 1 --2- 172.21.15.101:0/286815767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee39003c60 0x55ee3906d0f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8640003f80 tx=0x7f864000fdb0 comp rx=0 tx=0).stop 2023-07-18T03:15:42.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.987+0000 7f866a246b80 1 -- 172.21.15.101:0/286815767 shutdown_connections 2023-07-18T03:15:42.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.987+0000 7f866a246b80 1 --2- 172.21.15.101:0/286815767 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ee39074690 0x55ee39076aa0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:42.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.987+0000 7f866a246b80 1 --2- 172.21.15.101:0/286815767 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ee3900eb50 0x55ee39014850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:42.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.988+0000 7f862f7fe700 0 client.27254 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:15:42.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.988+0000 7f866a246b80 1 --2- 172.21.15.101:0/286815767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ee39003350 0x55ee3906cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:42.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.988+0000 7f866a246b80 1 --2- 172.21.15.101:0/286815767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee39003c60 0x55ee3906d0f0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:42.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.988+0000 7f866a246b80 1 -- 172.21.15.101:0/286815767 >> 172.21.15.101:0/286815767 conn(0x55ee38ff9e20 msgr2=0x55ee3900a820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:42.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.988+0000 7f866a246b80 1 -- 172.21.15.101:0/286815767 shutdown_connections 2023-07-18T03:15:42.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.988+0000 7f866a246b80 1 -- 172.21.15.101:0/286815767 wait complete. 2023-07-18T03:15:42.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.988+0000 7f866a246b80 20 client.27254 trim_cache size 0 max 16384 2023-07-18T03:15:42.990 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:15:42.990 INFO:teuthology.orchestra.run.smithi101.stdout: "return_code": 0, 2023-07-18T03:15:42.990 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_tag": "d88cbea8-f9ad-4ec0-b018-08f6344c78a2", 2023-07-18T03:15:42.990 INFO:teuthology.orchestra.run.smithi101.stdout: "mode": "asynchronous" 2023-07-18T03:15:42.990 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:15:42.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.988+0000 7f86569ff700 1 -- 172.21.15.101:0/3575614998 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f863c0a12c0 msgr2=0x7f863c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:42.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.988+0000 7f86569ff700 1 --2- 172.21.15.101:0/3575614998 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f863c0a12c0 0x7f863c0a3770 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8638097210 tx=0x7f86380055c0 comp rx=0 tx=0).stop 2023-07-18T03:15:42.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.988+0000 7f86569ff700 1 -- 172.21.15.101:0/3575614998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8650100480 msgr2=0x7f8650072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:42.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.988+0000 7f86569ff700 1 --2- 172.21.15.101:0/3575614998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8650100480 0x7f8650072d50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8638009810 tx=0x7f863800fa50 comp rx=0 tx=0).stop 2023-07-18T03:15:42.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.989+0000 7f86569ff700 1 -- 172.21.15.101:0/3575614998 shutdown_connections 2023-07-18T03:15:42.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.989+0000 7f86569ff700 1 --2- 172.21.15.101:0/3575614998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8650105ae0 0x7f86501a3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:42.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.989+0000 7f86569ff700 1 --2- 172.21.15.101:0/3575614998 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f86500ffad0 0x7f8650072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:42.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.989+0000 7f86569ff700 1 --2- 172.21.15.101:0/3575614998 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f863c0a12c0 0x7f863c0a3770 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:42.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.989+0000 7f86569ff700 1 --2- 172.21.15.101:0/3575614998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8650100480 0x7f8650072d50 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:42.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.989+0000 7f86569ff700 1 -- 172.21.15.101:0/3575614998 >> 172.21.15.101:0/3575614998 conn(0x7f86500fb5f0 msgr2=0x7f86500fbca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:42.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.989+0000 7f86569ff700 1 -- 172.21.15.101:0/3575614998 shutdown_connections 2023-07-18T03:15:42.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:42.989+0000 7f86569ff700 1 -- 172.21.15.101:0/3575614998 wait complete. 2023-07-18T03:15:43.001 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:15:43.063 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:42 smithi101 ceph-mon[126379]: pgmap v251: 129 pgs: 129 active+clean; 400 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 102 op/s 2023-07-18T03:15:43.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.219+0000 7f7aeddff700 1 -- 172.21.15.101:0/2494591013 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ae8105ae0 msgr2=0x7f7ae8105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:43.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.219+0000 7f7aeddff700 1 --2- 172.21.15.101:0/2494591013 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ae8105ae0 0x7f7ae8105eb0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7ad80098f0 tx=0x7f7ad8009c00 comp rx=0 tx=0).stop 2023-07-18T03:15:43.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.219+0000 7f7aeddff700 1 -- 172.21.15.101:0/2494591013 shutdown_connections 2023-07-18T03:15:43.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.219+0000 7f7aeddff700 1 --2- 172.21.15.101:0/2494591013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ae8100480 0x7f7ae810c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:43.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.219+0000 7f7aeddff700 1 --2- 172.21.15.101:0/2494591013 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ae8105ae0 0x7f7ae8105eb0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:43.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.219+0000 7f7aeddff700 1 --2- 172.21.15.101:0/2494591013 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7ae80ffad0 0x7f7ae80fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:43.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.219+0000 7f7aeddff700 1 -- 172.21.15.101:0/2494591013 >> 172.21.15.101:0/2494591013 conn(0x7f7ae80fb5f0 msgr2=0x7f7ae80fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:43.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.219+0000 7f7aeddff700 1 -- 172.21.15.101:0/2494591013 shutdown_connections 2023-07-18T03:15:43.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.219+0000 7f7aeddff700 1 -- 172.21.15.101:0/2494591013 wait complete. 2023-07-18T03:15:43.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.220+0000 7f7aeddff700 1 Processor -- start 2023-07-18T03:15:43.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.220+0000 7f7aeddff700 1 -- start start 2023-07-18T03:15:43.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.221+0000 7f7aeddff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ae80ffad0 0x7f7ae8072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:43.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.221+0000 7f7aeddff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7ae8100480 0x7f7ae8072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:43.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.221+0000 7f7aeddff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ae8105ae0 0x7f7ae8073290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:43.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.221+0000 7f7aeddff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae81a40f0 con 0x7f7ae8100480 2023-07-18T03:15:43.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.221+0000 7f7aeddff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae81a4260 con 0x7f7ae8105ae0 2023-07-18T03:15:43.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.221+0000 7f7aeddff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae81a4420 con 0x7f7ae80ffad0 2023-07-18T03:15:43.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.221+0000 7f7ae77fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ae80ffad0 0x7f7ae8072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:43.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.221+0000 7f7ae77fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ae80ffad0 0x7f7ae8072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:51356/0 (socket says 172.21.15.101:51356) 2023-07-18T03:15:43.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.221+0000 7f7ae77fe700 1 -- 172.21.15.101:0/3889180465 learned_addr learned my addr 172.21.15.101:0/3889180465 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:43.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.221+0000 7f7ae7fff700 1 --2- 172.21.15.101:0/3889180465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ae8105ae0 0x7f7ae8073290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:43.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.221+0000 7f7ae6ffd700 1 --2- 172.21.15.101:0/3889180465 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7ae8100480 0x7f7ae8072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:43.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.221+0000 7f7ae77fe700 1 -- 172.21.15.101:0/3889180465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ae8105ae0 msgr2=0x7f7ae8073290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:43.218 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.221+0000 7f7ae77fe700 1 --2- 172.21.15.101:0/3889180465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ae8105ae0 0x7f7ae8073290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:43.218 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.221+0000 7f7ae77fe700 1 -- 172.21.15.101:0/3889180465 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7ae8100480 msgr2=0x7f7ae8072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:43.218 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.221+0000 7f7ae77fe700 1 --2- 172.21.15.101:0/3889180465 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7ae8100480 0x7f7ae8072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:43.218 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.221+0000 7f7ae77fe700 1 -- 172.21.15.101:0/3889180465 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad8009520 con 0x7f7ae80ffad0 2023-07-18T03:15:43.218 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.222+0000 7f7ae77fe700 1 --2- 172.21.15.101:0/3889180465 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ae80ffad0 0x7f7ae8072810 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8009810 tx=0x7f7ad800fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:43.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.222+0000 7f7ae4ff9700 1 -- 172.21.15.101:0/3889180465 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad801c070 con 0x7f7ae80ffad0 2023-07-18T03:15:43.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.222+0000 7f7aeddff700 1 -- 172.21.15.101:0/3889180465 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ae81a46a0 con 0x7f7ae80ffad0 2023-07-18T03:15:43.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.222+0000 7f7ae4ff9700 1 -- 172.21.15.101:0/3889180465 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ad800bb20 con 0x7f7ae80ffad0 2023-07-18T03:15:43.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.222+0000 7f7ae4ff9700 1 -- 172.21.15.101:0/3889180465 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8004930 con 0x7f7ae80ffad0 2023-07-18T03:15:43.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.222+0000 7f7aeddff700 1 -- 172.21.15.101:0/3889180465 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ae81a4ac0 con 0x7f7ae80ffad0 2023-07-18T03:15:43.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.227+0000 7f7ae4ff9700 1 -- 172.21.15.101:0/3889180465 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f7ad8020070 con 0x7f7ae80ffad0 2023-07-18T03:15:43.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.228+0000 7f7b01777b80 1 Processor -- start 2023-07-18T03:15:43.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.228+0000 7f7b01777b80 1 -- start start 2023-07-18T03:15:43.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.228+0000 7f7b01777b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596ddf2a2d0 0x5596ddf2a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:43.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.228+0000 7f7b01777b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596ddf2abe0 0x5596ddf35590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:43.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.228+0000 7f7b01777b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596ddf35ad0 0x5596ddf37eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:43.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.228+0000 7f7b01777b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596dde30cb0 con 0x5596ddf2abe0 2023-07-18T03:15:43.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.228+0000 7f7b01777b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596dde83a40 con 0x5596ddf35ad0 2023-07-18T03:15:43.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.228+0000 7f7b01777b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596dde7ced0 con 0x5596ddf2a2d0 2023-07-18T03:15:43.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.228+0000 7f7ae7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596ddf2a2d0 0x5596ddf2a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:43.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.229+0000 7f7ae4ff9700 1 --2- 172.21.15.101:0/3889180465 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f7ad00a12c0 0x7f7ad00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:43.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.229+0000 7f7ae77fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596ddf2abe0 0x5596ddf35590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:43.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.229+0000 7f7ae7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596ddf2a2d0 0x5596ddf2a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:51364/0 (socket says 172.21.15.101:51364) 2023-07-18T03:15:43.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.229+0000 7f7ae7fff700 1 -- 172.21.15.101:0/3634322216 learned_addr learned my addr 172.21.15.101:0/3634322216 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:43.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.229+0000 7f7ae7fff700 1 --2- 172.21.15.101:0/3889180465 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f7ad00a12c0 0x7f7ad00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:43.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.229+0000 7f7ae6ffd700 1 --2- 172.21.15.101:0/3634322216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596ddf35ad0 0x5596ddf37eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:43.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.229+0000 7f7ae4ff9700 1 -- 172.21.15.101:0/3889180465 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f7ad80cdd40 con 0x7f7ae80ffad0 2023-07-18T03:15:43.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.229+0000 7f7ae77fe700 1 -- 172.21.15.101:0/3634322216 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596ddf2a2d0 msgr2=0x5596ddf2a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:43.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.229+0000 7f7ae77fe700 1 --2- 172.21.15.101:0/3634322216 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596ddf2a2d0 0x5596ddf2a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:43.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.229+0000 7f7ae77fe700 1 -- 172.21.15.101:0/3634322216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596ddf35ad0 msgr2=0x5596ddf37eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:43.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.229+0000 7f7ae77fe700 1 --2- 172.21.15.101:0/3634322216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596ddf35ad0 0x5596ddf37eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:43.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.229+0000 7f7ae77fe700 1 -- 172.21.15.101:0/3634322216 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596ddcaade0 con 0x5596ddf2abe0 2023-07-18T03:15:43.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.229+0000 7f7ae7fff700 1 --2- 172.21.15.101:0/3634322216 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596ddf2a2d0 0x5596ddf2a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:15:43.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.229+0000 7f7ae6ffd700 1 --2- 172.21.15.101:0/3634322216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596ddf35ad0 0x5596ddf37eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:15:43.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.229+0000 7f7ae77fe700 1 --2- 172.21.15.101:0/3634322216 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596ddf2abe0 0x5596ddf35590 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f7ad80cb0e0 tx=0x7f7ad800fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=917755709d39a4c6 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:43.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.230+0000 7f7ae7fff700 1 --2- 172.21.15.101:0/3889180465 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f7ad00a12c0 0x7f7ad00a3770 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0006220 tx=0x7f7ae0004b40 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:43.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.230+0000 7f7acdffb700 1 -- 172.21.15.101:0/3634322216 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad80a2eb0 con 0x5596ddf2abe0 2023-07-18T03:15:43.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.230+0000 7f7acdffb700 1 -- 172.21.15.101:0/3634322216 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ad809aaf0 con 0x5596ddf2abe0 2023-07-18T03:15:43.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.230+0000 7f7acdffb700 1 -- 172.21.15.101:0/3634322216 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad80a2a20 con 0x5596ddf2abe0 2023-07-18T03:15:43.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.230+0000 7f7b01777b80 1 -- 172.21.15.101:0/3634322216 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596ddf2abe0 msgr2=0x5596ddf35590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:43.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.230+0000 7f7b01777b80 1 --2- 172.21.15.101:0/3634322216 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596ddf2abe0 0x5596ddf35590 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f7ad80cb0e0 tx=0x7f7ad800fa70 comp rx=0 tx=0).stop 2023-07-18T03:15:43.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.230+0000 7f7b01777b80 1 -- 172.21.15.101:0/3634322216 shutdown_connections 2023-07-18T03:15:43.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.230+0000 7f7b01777b80 1 --2- 172.21.15.101:0/3634322216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596ddf35ad0 0x5596ddf37eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:43.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.230+0000 7f7b01777b80 1 --2- 172.21.15.101:0/3634322216 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596ddf2a2d0 0x5596ddf2a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:43.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.230+0000 7f7b01777b80 1 --2- 172.21.15.101:0/3634322216 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596ddf2abe0 0x5596ddf35590 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:43.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.230+0000 7f7b01777b80 1 -- 172.21.15.101:0/3634322216 >> 172.21.15.101:0/3634322216 conn(0x5596ddf20da0 msgr2=0x5596ddf266a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:43.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.231+0000 7f7b01777b80 1 -- 172.21.15.101:0/3634322216 shutdown_connections 2023-07-18T03:15:43.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.231+0000 7f7b01777b80 1 -- 172.21.15.101:0/3634322216 wait complete. 2023-07-18T03:15:43.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.231+0000 7f7b01777b80 1 -- start start 2023-07-18T03:15:43.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.231+0000 7f7b01777b80 1 Processor -- start 2023-07-18T03:15:43.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.232+0000 7f7b01777b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596ddf2a2d0 0x5596ddf3abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:43.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.232+0000 7f7b01777b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596ddf2abe0 0x5596ddf3b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:43.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.232+0000 7f7b01777b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596ddf35ad0 0x5596ddf34a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:43.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.232+0000 7f7b01777b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596dde1cac0 con 0x5596ddf2a2d0 2023-07-18T03:15:43.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.232+0000 7f7b01777b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596dde1c7c0 con 0x5596ddf2abe0 2023-07-18T03:15:43.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.232+0000 7f7b01777b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596ddd00920 con 0x5596ddf35ad0 2023-07-18T03:15:43.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.232+0000 7f7b01777b80 10 client.0 _ll_register_callbacks cb 0x5596dde1a760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:15:43.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.232+0000 7f7ae77fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596ddf2a2d0 0x5596ddf3abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:43.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.232+0000 7f7ae77fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596ddf2a2d0 0x5596ddf3abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:44102/0 (socket says 172.21.15.101:44102) 2023-07-18T03:15:43.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.232+0000 7f7ae77fe700 1 -- 172.21.15.101:0/3681712922 learned_addr learned my addr 172.21.15.101:0/3681712922 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:43.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.232+0000 7f7ae6ffd700 1 --2- 172.21.15.101:0/3681712922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596ddf2abe0 0x5596ddf3b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:43.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.232+0000 7f7ae7fff700 1 --2- 172.21.15.101:0/3681712922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596ddf35ad0 0x5596ddf34a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:43.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.232+0000 7f7ae77fe700 1 -- 172.21.15.101:0/3681712922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596ddf35ad0 msgr2=0x5596ddf34a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:43.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.232+0000 7f7ae77fe700 1 --2- 172.21.15.101:0/3681712922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596ddf35ad0 0x5596ddf34a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:43.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.232+0000 7f7ae77fe700 1 -- 172.21.15.101:0/3681712922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596ddf2abe0 msgr2=0x5596ddf3b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:43.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.232+0000 7f7ae77fe700 1 --2- 172.21.15.101:0/3681712922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596ddf2abe0 0x5596ddf3b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:43.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.232+0000 7f7ae77fe700 1 -- 172.21.15.101:0/3681712922 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ae000f840 con 0x5596ddf2a2d0 2023-07-18T03:15:43.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.233+0000 7f7ae77fe700 1 --2- 172.21.15.101:0/3681712922 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596ddf2a2d0 0x5596ddf3abc0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8005c10 tx=0x7f7ad80a3000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ec555a1dff14a5 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:43.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.233+0000 7f7ac6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:15:43.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.233+0000 7f7ac6ffd700 1 -- 172.21.15.101:0/3681712922 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8098f10 con 0x5596ddf2a2d0 2023-07-18T03:15:43.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.233+0000 7f7b01777b80 1 -- 172.21.15.101:0/3681712922 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5596dddd64f0 con 0x5596ddf2a2d0 2023-07-18T03:15:43.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.233+0000 7f7ac6ffd700 1 -- 172.21.15.101:0/3681712922 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f7ad80a2a20 con 0x5596ddf2a2d0 2023-07-18T03:15:43.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.233+0000 7f7ac6ffd700 1 -- 172.21.15.101:0/3681712922 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ad80a2c20 con 0x5596ddf2a2d0 2023-07-18T03:15:43.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.233+0000 7f7b01777b80 10 client.17226 fetch_fsmap learned FSMap version 29 2023-07-18T03:15:43.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.233+0000 7f7b01777b80 1 -- 172.21.15.101:0/3681712922 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f7ae81a4ac0 con 0x5596ddf2a2d0 2023-07-18T03:15:43.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.233+0000 7f7ac6ffd700 1 -- 172.21.15.101:0/3681712922 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5596dddd64f0 con 0x5596ddf2a2d0 2023-07-18T03:15:43.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.234+0000 7f7ac6ffd700 1 -- 172.21.15.101:0/3681712922 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f7ad800b7a0 con 0x5596ddf2a2d0 2023-07-18T03:15:43.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.234+0000 7f7b01777b80 10 client.17226 fetch_fsmap finished waiting for FSMap version 29 2023-07-18T03:15:43.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.234+0000 7f7b01777b80 10 client.17226 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:15:43.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.234+0000 7f7b01777b80 20 client.17226 populate_metadata read hostname 'smithi101' 2023-07-18T03:15:43.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.234+0000 7f7b01777b80 1 --2- 172.21.15.101:0/3681712922 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5596ddf44ba0 0x5596ddf46f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:43.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.234+0000 7f7b01777b80 4 client.17226 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:15:43.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.234+0000 7f7b01777b80 1 -- 172.21.15.101:0/3681712922 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5596dde75eb0 con 0x5596ddf44ba0 2023-07-18T03:15:43.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.234+0000 7f7ae6ffd700 1 --2- 172.21.15.101:0/3681712922 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5596ddf44ba0 0x5596ddf46f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:43.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.235+0000 7f7ac6ffd700 10 client.17226 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:15:43.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.235+0000 7f7ae6ffd700 1 --2- 172.21.15.101:0/3681712922 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5596ddf44ba0 0x5596ddf46f80 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9343abee328cdf65 server_cookie=71c842985df9068a in_seq=0 out_seq=0 2023-07-18T03:15:43.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.235+0000 7f7ac6ffd700 1 -- 172.21.15.101:0/3681712922 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5596dde75eb0 con 0x5596ddf44ba0 2023-07-18T03:15:43.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.235+0000 7f7ac6ffd700 10 client.17226 handle_command_reply: tid=0 2023-07-18T03:15:43.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.235+0000 7f7b01777b80 1 client.17226 shutdown 2023-07-18T03:15:43.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.235+0000 7f7b01777b80 1 -- 172.21.15.101:0/3681712922 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596ddf2a2d0 msgr2=0x5596ddf3abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:43.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.235+0000 7f7b01777b80 1 --2- 172.21.15.101:0/3681712922 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596ddf2a2d0 0x5596ddf3abc0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8005c10 tx=0x7f7ad80a3000 comp rx=0 tx=0).stop 2023-07-18T03:15:43.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.235+0000 7f7b01777b80 1 -- 172.21.15.101:0/3681712922 shutdown_connections 2023-07-18T03:15:43.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.235+0000 7f7b01777b80 1 --2- 172.21.15.101:0/3681712922 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5596ddf44ba0 0x5596ddf46f80 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:43.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.235+0000 7f7b01777b80 1 --2- 172.21.15.101:0/3681712922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596ddf35ad0 0x5596ddf34a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:43.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.235+0000 7f7b01777b80 1 --2- 172.21.15.101:0/3681712922 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596ddf2a2d0 0x5596ddf3abc0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:43.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.235+0000 7f7b01777b80 1 --2- 172.21.15.101:0/3681712922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596ddf2abe0 0x5596ddf3b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:43.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.235+0000 7f7ac6ffd700 0 client.17226 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:15:43.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.235+0000 7f7b01777b80 1 -- 172.21.15.101:0/3681712922 >> 172.21.15.101:0/3681712922 conn(0x5596ddf20da0 msgr2=0x5596ddf26290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:43.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.236+0000 7f7b01777b80 1 -- 172.21.15.101:0/3681712922 shutdown_connections 2023-07-18T03:15:43.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.236+0000 7f7b01777b80 1 -- 172.21.15.101:0/3681712922 wait complete. 2023-07-18T03:15:43.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.236+0000 7f7b01777b80 20 client.17226 trim_cache size 0 max 16384 2023-07-18T03:15:43.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.253+0000 7f7b01777b80 1 Processor -- start 2023-07-18T03:15:43.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.254+0000 7f7b01777b80 1 -- start start 2023-07-18T03:15:43.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.254+0000 7f7b01777b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596ddf2a2d0 0x5596ddf46f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:43.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.254+0000 7f7b01777b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596ddf2abe0 0x5596ddf24e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:43.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.254+0000 7f7b01777b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596ddf35ad0 0x5596ddf2fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:43.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.254+0000 7f7b01777b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596ddf209a0 con 0x5596ddf2abe0 2023-07-18T03:15:43.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.254+0000 7f7b01777b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596ddf274d0 con 0x5596ddf2a2d0 2023-07-18T03:15:43.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.254+0000 7f7b01777b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596ddf25350 con 0x5596ddf35ad0 2023-07-18T03:15:43.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.254+0000 7f7ae77fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596ddf2a2d0 0x5596ddf46f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:43.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.254+0000 7f7ae77fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596ddf2a2d0 0x5596ddf46f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:35710/0 (socket says 172.21.15.101:35710) 2023-07-18T03:15:43.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.254+0000 7f7ae77fe700 1 -- 172.21.15.101:0/1315515552 learned_addr learned my addr 172.21.15.101:0/1315515552 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:43.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.254+0000 7f7ae6ffd700 1 --2- 172.21.15.101:0/1315515552 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596ddf2abe0 0x5596ddf24e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:43.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.254+0000 7f7ae7fff700 1 --2- 172.21.15.101:0/1315515552 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596ddf35ad0 0x5596ddf2fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:43.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.254+0000 7f7ae77fe700 1 -- 172.21.15.101:0/1315515552 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596ddf35ad0 msgr2=0x5596ddf2fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:43.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.254+0000 7f7ae77fe700 1 --2- 172.21.15.101:0/1315515552 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596ddf35ad0 0x5596ddf2fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:43.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.254+0000 7f7ae77fe700 1 -- 172.21.15.101:0/1315515552 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596ddf2abe0 msgr2=0x5596ddf24e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:43.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.254+0000 7f7ae77fe700 1 --2- 172.21.15.101:0/1315515552 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596ddf2abe0 0x5596ddf24e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:43.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.254+0000 7f7ae77fe700 1 -- 172.21.15.101:0/1315515552 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596ddc93c60 con 0x5596ddf2a2d0 2023-07-18T03:15:43.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.254+0000 7f7ae6ffd700 1 --2- 172.21.15.101:0/1315515552 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596ddf2abe0 0x5596ddf24e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:15:43.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.255+0000 7f7ae77fe700 1 --2- 172.21.15.101:0/1315515552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596ddf2a2d0 0x5596ddf46f80 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7ad809aa80 tx=0x7f7ad8005450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52664dd5ab681b55 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:43.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.255+0000 7f7ace7fc700 1 -- 172.21.15.101:0/1315515552 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad80b3260 con 0x5596ddf2a2d0 2023-07-18T03:15:43.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.255+0000 7f7ace7fc700 1 -- 172.21.15.101:0/1315515552 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ad809a450 con 0x5596ddf2a2d0 2023-07-18T03:15:43.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.255+0000 7f7ace7fc700 1 -- 172.21.15.101:0/1315515552 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8099fe0 con 0x5596ddf2a2d0 2023-07-18T03:15:43.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.256+0000 7f7b01777b80 1 -- 172.21.15.101:0/1315515552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596ddf2a2d0 msgr2=0x5596ddf46f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:43.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.256+0000 7f7b01777b80 1 --2- 172.21.15.101:0/1315515552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596ddf2a2d0 0x5596ddf46f80 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7ad809aa80 tx=0x7f7ad8005450 comp rx=0 tx=0).stop 2023-07-18T03:15:43.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.256+0000 7f7b01777b80 1 -- 172.21.15.101:0/1315515552 shutdown_connections 2023-07-18T03:15:43.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.256+0000 7f7b01777b80 1 --2- 172.21.15.101:0/1315515552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596ddf2a2d0 0x5596ddf46f80 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:43.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.256+0000 7f7b01777b80 1 --2- 172.21.15.101:0/1315515552 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596ddf35ad0 0x5596ddf2fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:43.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.256+0000 7f7b01777b80 1 --2- 172.21.15.101:0/1315515552 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596ddf2abe0 0x5596ddf24e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:43.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.256+0000 7f7b01777b80 1 -- 172.21.15.101:0/1315515552 >> 172.21.15.101:0/1315515552 conn(0x5596ddf20da0 msgr2=0x5596ddf4b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:43.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.256+0000 7f7b01777b80 1 -- 172.21.15.101:0/1315515552 shutdown_connections 2023-07-18T03:15:43.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.256+0000 7f7b01777b80 1 -- 172.21.15.101:0/1315515552 wait complete. 2023-07-18T03:15:43.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.256+0000 7f7b01777b80 1 -- start start 2023-07-18T03:15:43.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.256+0000 7f7b01777b80 1 Processor -- start 2023-07-18T03:15:43.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.257+0000 7f7b01777b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596ddf2a2d0 0x5596ddf8c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:43.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.257+0000 7f7b01777b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596ddf2abe0 0x5596ddf8ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:43.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.257+0000 7f7b01777b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596ddf35ad0 0x5596ddf8d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:43.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.257+0000 7f7b01777b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596ddf97bc0 con 0x5596ddf2abe0 2023-07-18T03:15:43.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.257+0000 7f7b01777b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596ddf97d30 con 0x5596ddf35ad0 2023-07-18T03:15:43.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.257+0000 7f7b01777b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596ddf97ef0 con 0x5596ddf2a2d0 2023-07-18T03:15:43.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.257+0000 7f7b01777b80 10 client.0 _ll_register_callbacks cb 0x7f7ae000f1b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:15:43.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.257+0000 7f7ae77fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596ddf2a2d0 0x5596ddf8c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:43.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.257+0000 7f7ae7fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596ddf35ad0 0x5596ddf8d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:43.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.257+0000 7f7ae6ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596ddf2abe0 0x5596ddf8ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:43.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.257+0000 7f7ae77fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596ddf2a2d0 0x5596ddf8c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:51386/0 (socket says 172.21.15.101:51386) 2023-07-18T03:15:43.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.257+0000 7f7ae77fe700 1 -- 172.21.15.101:0/2025211262 learned_addr learned my addr 172.21.15.101:0/2025211262 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:15:43.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.257+0000 7f7ae7fff700 1 -- 172.21.15.101:0/2025211262 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596ddf2a2d0 msgr2=0x5596ddf8c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:43.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.257+0000 7f7ae7fff700 1 --2- 172.21.15.101:0/2025211262 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596ddf2a2d0 0x5596ddf8c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:43.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.257+0000 7f7ae7fff700 1 -- 172.21.15.101:0/2025211262 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596ddf2abe0 msgr2=0x5596ddf8ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:15:43.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.257+0000 7f7ae7fff700 1 --2- 172.21.15.101:0/2025211262 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596ddf2abe0 0x5596ddf8ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:43.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.257+0000 7f7ae7fff700 1 -- 172.21.15.101:0/2025211262 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ae81a4ac0 con 0x5596ddf35ad0 2023-07-18T03:15:43.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.257+0000 7f7ae77fe700 1 --2- 172.21.15.101:0/2025211262 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596ddf2a2d0 0x5596ddf8c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:15:43.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.258+0000 7f7ae7fff700 1 --2- 172.21.15.101:0/2025211262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596ddf35ad0 0x5596ddf8d430 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x5596ddf255e0 tx=0x7f7ae0004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff8b97eb4d1b44d6 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:15:43.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.258+0000 7f7ac6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:15:43.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.258+0000 7f7ac6ffd700 1 -- 172.21.15.101:0/2025211262 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae000e040 con 0x5596ddf35ad0 2023-07-18T03:15:43.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.258+0000 7f7b01777b80 1 -- 172.21.15.101:0/2025211262 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7ad809aaf0 con 0x5596ddf35ad0 2023-07-18T03:15:43.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.258+0000 7f7ac6ffd700 1 -- 172.21.15.101:0/2025211262 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f7ae0028470 con 0x5596ddf35ad0 2023-07-18T03:15:43.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.258+0000 7f7ac6ffd700 1 -- 172.21.15.101:0/2025211262 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ae0028650 con 0x5596ddf35ad0 2023-07-18T03:15:43.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.258+0000 7f7b01777b80 10 client.27269 fetch_fsmap learned FSMap version 29 2023-07-18T03:15:43.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.258+0000 7f7b01777b80 1 -- 172.21.15.101:0/2025211262 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5596ddcd1250 con 0x5596ddf35ad0 2023-07-18T03:15:43.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.258+0000 7f7ac6ffd700 1 -- 172.21.15.101:0/2025211262 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ad809aaf0 con 0x5596ddf35ad0 2023-07-18T03:15:43.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.259+0000 7f7ac6ffd700 1 -- 172.21.15.101:0/2025211262 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f7ae001a740 con 0x5596ddf35ad0 2023-07-18T03:15:43.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.259+0000 7f7b01777b80 10 client.27269 fetch_fsmap finished waiting for FSMap version 29 2023-07-18T03:15:43.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.259+0000 7f7b01777b80 10 client.27269 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:15:43.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.259+0000 7f7b01777b80 20 client.27269 populate_metadata read hostname 'smithi101' 2023-07-18T03:15:43.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.259+0000 7f7b01777b80 1 --2- 172.21.15.101:0/2025211262 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5596ddf99700 0x5596ddf9bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:15:43.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.259+0000 7f7b01777b80 4 client.27269 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:15:43.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.259+0000 7f7b01777b80 1 -- 172.21.15.101:0/2025211262 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5596dde38050 con 0x5596ddf99700 2023-07-18T03:15:43.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.259+0000 7f7ae77fe700 1 --2- 172.21.15.101:0/2025211262 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5596ddf99700 0x5596ddf9bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:15:43.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.259+0000 7f7ae77fe700 1 --2- 172.21.15.101:0/2025211262 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5596ddf99700 0x5596ddf9bb10 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=354e1e0461f1b4c7 server_cookie=d2471146a6657a0f in_seq=0 out_seq=0 2023-07-18T03:15:43.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.259+0000 7f7ac6ffd700 10 client.27269 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:15:43.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.259+0000 7f7ac6ffd700 1 -- 172.21.15.101:0/2025211262 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5596dde38050 con 0x5596ddf99700 2023-07-18T03:15:43.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.259+0000 7f7ac6ffd700 10 client.27269 handle_command_reply: tid=0 2023-07-18T03:15:43.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.259+0000 7f7b01777b80 1 client.27269 shutdown 2023-07-18T03:15:43.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.260+0000 7f7b01777b80 1 -- 172.21.15.101:0/2025211262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596ddf35ad0 msgr2=0x5596ddf8d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:43.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.260+0000 7f7b01777b80 1 --2- 172.21.15.101:0/2025211262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596ddf35ad0 0x5596ddf8d430 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x5596ddf255e0 tx=0x7f7ae0004ad0 comp rx=0 tx=0).stop 2023-07-18T03:15:43.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.260+0000 7f7b01777b80 1 -- 172.21.15.101:0/2025211262 shutdown_connections 2023-07-18T03:15:43.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.260+0000 7f7b01777b80 1 --2- 172.21.15.101:0/2025211262 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5596ddf99700 0x5596ddf9bb10 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:43.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.260+0000 7f7b01777b80 1 --2- 172.21.15.101:0/2025211262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596ddf35ad0 0x5596ddf8d430 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:43.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.260+0000 7f7b01777b80 1 --2- 172.21.15.101:0/2025211262 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596ddf2a2d0 0x5596ddf8c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:43.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.260+0000 7f7b01777b80 1 --2- 172.21.15.101:0/2025211262 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596ddf2abe0 0x5596ddf8ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:43.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.260+0000 7f7b01777b80 1 -- 172.21.15.101:0/2025211262 >> 172.21.15.101:0/2025211262 conn(0x5596ddf20da0 msgr2=0x5596ddf2fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:43.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.260+0000 7f7ac6ffd700 0 client.27269 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:15:43.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.260+0000 7f7b01777b80 1 -- 172.21.15.101:0/2025211262 shutdown_connections 2023-07-18T03:15:43.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.260+0000 7f7b01777b80 1 -- 172.21.15.101:0/2025211262 wait complete. 2023-07-18T03:15:43.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.260+0000 7f7b01777b80 20 client.27269 trim_cache size 0 max 16384 2023-07-18T03:15:43.263 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:15:43.264 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:15:43.264 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:15:43.264 INFO:teuthology.orchestra.run.smithi101.stdout: "d88cbea8-f9ad-4ec0-b018-08f6344c78a2": { 2023-07-18T03:15:43.264 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:15:43.264 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "d88cbea8-f9ad-4ec0-b018-08f6344c78a2", 2023-07-18T03:15:43.264 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:15:43.264 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:15:43.265 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:15:43.265 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:15:43.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.261+0000 7f7aeddff700 1 -- 172.21.15.101:0/3889180465 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f7ad00a12c0 msgr2=0x7f7ad00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:43.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.261+0000 7f7aeddff700 1 --2- 172.21.15.101:0/3889180465 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f7ad00a12c0 0x7f7ad00a3770 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0006220 tx=0x7f7ae0004b40 comp rx=0 tx=0).stop 2023-07-18T03:15:43.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.261+0000 7f7aeddff700 1 -- 172.21.15.101:0/3889180465 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ae80ffad0 msgr2=0x7f7ae8072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:15:43.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.261+0000 7f7aeddff700 1 --2- 172.21.15.101:0/3889180465 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ae80ffad0 0x7f7ae8072810 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8009810 tx=0x7f7ad800fa00 comp rx=0 tx=0).stop 2023-07-18T03:15:43.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.261+0000 7f7aeddff700 1 -- 172.21.15.101:0/3889180465 shutdown_connections 2023-07-18T03:15:43.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.261+0000 7f7aeddff700 1 --2- 172.21.15.101:0/3889180465 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f7ad00a12c0 0x7f7ad00a3770 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:43.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.261+0000 7f7aeddff700 1 --2- 172.21.15.101:0/3889180465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ae8105ae0 0x7f7ae8073290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:43.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.261+0000 7f7aeddff700 1 --2- 172.21.15.101:0/3889180465 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ae80ffad0 0x7f7ae8072810 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:43.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.261+0000 7f7aeddff700 1 --2- 172.21.15.101:0/3889180465 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7ae8100480 0x7f7ae8072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:15:43.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.261+0000 7f7aeddff700 1 -- 172.21.15.101:0/3889180465 >> 172.21.15.101:0/3889180465 conn(0x7f7ae80fb5f0 msgr2=0x7f7ae80fbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:15:43.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.261+0000 7f7aeddff700 1 -- 172.21.15.101:0/3889180465 shutdown_connections 2023-07-18T03:15:43.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:15:43.261+0000 7f7aeddff700 1 -- 172.21.15.101:0/3889180465 wait complete. 2023-07-18T03:15:43.274 INFO:tasks.cephfs.filesystem:scrub status for tag:d88cbea8-f9ad-4ec0-b018-08f6344c78a2 - {'path': '/', 'tag': 'd88cbea8-f9ad-4ec0-b018-08f6344c78a2', 'options': 'recursive,force'} 2023-07-18T03:15:43.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:42 smithi155 ceph-mon[128885]: pgmap v251: 129 pgs: 129 active+clean; 400 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 102 op/s 2023-07-18T03:15:43.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:42 smithi179 ceph-mon[129229]: pgmap v251: 129 pgs: 129 active+clean; 400 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 102 op/s 2023-07-18T03:15:43.595 INFO:tasks.workunit.client.0.smithi101.stdout: 1 557060 6.63 MB/sec execute 308 sec latency 1010.982 ms 2023-07-18T03:15:44.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:43 smithi101 ceph-mon[126379]: scrub queued for path: / 2023-07-18T03:15:44.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:43 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:15:44.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:43 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:15:44.239 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:43 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:15:44.239 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:43 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:15:44.239 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:43 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:15:44.239 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:43 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:15:44.239 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:43 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:15:44.239 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:43 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:15:44.240 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:43 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:15:44.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:43 smithi155 ceph-mon[128885]: scrub queued for path: / 2023-07-18T03:15:44.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:43 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:15:44.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:43 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:15:44.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:43 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:15:44.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:43 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:15:44.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:43 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:15:44.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:43 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:15:44.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:43 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:15:44.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:43 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:15:44.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:43 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:15:44.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:43 smithi179 ceph-mon[129229]: scrub queued for path: / 2023-07-18T03:15:44.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:43 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:15:44.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:43 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:15:44.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:43 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:15:44.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:43 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:15:44.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:43 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:15:44.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:43 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:15:44.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:43 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:15:44.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:43 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:15:44.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:43 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:15:44.598 INFO:tasks.workunit.client.0.smithi101.stdout: 1 558299 6.63 MB/sec execute 309 sec latency 13.517 ms 2023-07-18T03:15:45.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:44 smithi101 ceph-mon[126379]: pgmap v252: 129 pgs: 129 active+clean; 400 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 98 op/s 2023-07-18T03:15:45.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:44 smithi155 ceph-mon[128885]: pgmap v252: 129 pgs: 129 active+clean; 400 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 98 op/s 2023-07-18T03:15:45.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:44 smithi179 ceph-mon[129229]: pgmap v252: 129 pgs: 129 active+clean; 400 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 98 op/s 2023-07-18T03:15:45.598 INFO:tasks.workunit.client.0.smithi101.stdout: 1 559397 6.63 MB/sec execute 310 sec latency 193.737 ms 2023-07-18T03:15:46.598 INFO:tasks.workunit.client.0.smithi101.stdout: 1 559397 6.61 MB/sec execute 311 sec latency 1193.800 ms 2023-07-18T03:15:47.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:46 smithi101 ceph-mon[126379]: pgmap v253: 129 pgs: 129 active+clean; 404 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 167 op/s 2023-07-18T03:15:47.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:46 smithi155 ceph-mon[128885]: pgmap v253: 129 pgs: 129 active+clean; 404 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 167 op/s 2023-07-18T03:15:47.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:46 smithi179 ceph-mon[129229]: pgmap v253: 129 pgs: 129 active+clean; 404 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 167 op/s 2023-07-18T03:15:47.598 INFO:tasks.workunit.client.0.smithi101.stdout: 1 559397 6.59 MB/sec execute 312 sec latency 2193.936 ms 2023-07-18T03:15:48.598 INFO:tasks.workunit.client.0.smithi101.stdout: 1 559397 6.57 MB/sec execute 313 sec latency 3194.043 ms 2023-07-18T03:15:49.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:48 smithi155 ceph-mon[128885]: pgmap v254: 129 pgs: 129 active+clean; 404 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 120 op/s 2023-07-18T03:15:49.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:48 smithi179 ceph-mon[129229]: pgmap v254: 129 pgs: 129 active+clean; 404 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 120 op/s 2023-07-18T03:15:49.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:48 smithi101 ceph-mon[126379]: pgmap v254: 129 pgs: 129 active+clean; 404 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 120 op/s 2023-07-18T03:15:49.598 INFO:tasks.workunit.client.0.smithi101.stdout: 1 559796 6.55 MB/sec execute 314 sec latency 4035.681 ms 2023-07-18T03:15:50.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:50 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:15:50.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:50 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:15:50.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:50 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:15:50.598 INFO:tasks.workunit.client.0.smithi101.stdout: 1 561407 6.56 MB/sec execute 315 sec latency 30.239 ms 2023-07-18T03:15:51.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:51 smithi155 ceph-mon[128885]: pgmap v255: 129 pgs: 129 active+clean; 405 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 143 op/s 2023-07-18T03:15:51.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:51 smithi155 ceph-mon[128885]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:15:51.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:51 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:15:51.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:51 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:15:51.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:51 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:15:51.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:51 smithi179 ceph-mon[129229]: pgmap v255: 129 pgs: 129 active+clean; 405 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 143 op/s 2023-07-18T03:15:51.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:51 smithi179 ceph-mon[129229]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:15:51.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:51 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:15:51.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:51 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:15:51.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:51 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:15:51.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:51 smithi101 ceph-mon[126379]: pgmap v255: 129 pgs: 129 active+clean; 405 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 143 op/s 2023-07-18T03:15:51.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:51 smithi101 ceph-mon[126379]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:15:51.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:51 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:15:51.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:51 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:15:51.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:51 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:15:51.598 INFO:tasks.workunit.client.0.smithi101.stdout: 1 562437 6.55 MB/sec execute 316 sec latency 13.444 ms 2023-07-18T03:15:52.598 INFO:tasks.workunit.client.0.smithi101.stdout: 1 562946 6.54 MB/sec execute 317 sec latency 666.684 ms 2023-07-18T03:15:53.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:53 smithi155 ceph-mon[128885]: pgmap v256: 129 pgs: 129 active+clean; 405 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 111 op/s 2023-07-18T03:15:53.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:53 smithi179 ceph-mon[129229]: pgmap v256: 129 pgs: 129 active+clean; 405 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 111 op/s 2023-07-18T03:15:53.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:53 smithi101 ceph-mon[126379]: pgmap v256: 129 pgs: 129 active+clean; 405 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 111 op/s 2023-07-18T03:15:53.598 INFO:tasks.workunit.client.0.smithi101.stdout: 1 562946 6.52 MB/sec execute 318 sec latency 1666.794 ms 2023-07-18T03:15:54.599 INFO:tasks.workunit.client.0.smithi101.stdout: 1 563346 6.50 MB/sec execute 319 sec latency 2497.762 ms 2023-07-18T03:15:55.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:55 smithi155 ceph-mon[128885]: pgmap v257: 129 pgs: 129 active+clean; 405 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 100 op/s 2023-07-18T03:15:55.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:55 smithi179 ceph-mon[129229]: pgmap v257: 129 pgs: 129 active+clean; 405 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 100 op/s 2023-07-18T03:15:55.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:55 smithi101 ceph-mon[126379]: pgmap v257: 129 pgs: 129 active+clean; 405 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 100 op/s 2023-07-18T03:15:55.599 INFO:tasks.workunit.client.0.smithi101.stdout: 1 565051 6.52 MB/sec execute 320 sec latency 37.319 ms 2023-07-18T03:15:56.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:56 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:15:56.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:56 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:15:56.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:56 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:15:56.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:56 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:15:56.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:56 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:15:56.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:56 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:15:56.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:56 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:15:56.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:56 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:15:56.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:56 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:15:56.599 INFO:tasks.workunit.client.0.smithi101.stdout: 1 566045 6.51 MB/sec execute 321 sec latency 48.071 ms 2023-07-18T03:15:57.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:57 smithi155 ceph-mon[128885]: pgmap v258: 129 pgs: 129 active+clean; 409 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 153 op/s 2023-07-18T03:15:57.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:57 smithi155 ceph-mon[128885]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:15:57.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:57 smithi179 ceph-mon[129229]: pgmap v258: 129 pgs: 129 active+clean; 409 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 153 op/s 2023-07-18T03:15:57.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:57 smithi179 ceph-mon[129229]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:15:57.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:57 smithi101 ceph-mon[126379]: pgmap v258: 129 pgs: 129 active+clean; 409 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 153 op/s 2023-07-18T03:15:57.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:57 smithi101 ceph-mon[126379]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:15:57.599 INFO:tasks.workunit.client.0.smithi101.stdout: 1 566486 6.50 MB/sec execute 322 sec latency 724.541 ms 2023-07-18T03:15:58.599 INFO:tasks.workunit.client.0.smithi101.stdout: 1 566486 6.48 MB/sec execute 323 sec latency 1724.641 ms 2023-07-18T03:15:59.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:15:59 smithi155 ceph-mon[128885]: pgmap v259: 129 pgs: 129 active+clean; 409 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 84 op/s 2023-07-18T03:15:59.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:15:59 smithi179 ceph-mon[129229]: pgmap v259: 129 pgs: 129 active+clean; 409 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 84 op/s 2023-07-18T03:15:59.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:15:59 smithi101 ceph-mon[126379]: pgmap v259: 129 pgs: 129 active+clean; 409 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 84 op/s 2023-07-18T03:15:59.599 INFO:tasks.workunit.client.0.smithi101.stdout: 1 566885 6.46 MB/sec execute 324 sec latency 2558.768 ms 2023-07-18T03:16:00.599 INFO:tasks.workunit.client.0.smithi101.stdout: 1 568561 6.48 MB/sec execute 325 sec latency 70.871 ms 2023-07-18T03:16:01.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:01 smithi155 ceph-mon[128885]: pgmap v260: 129 pgs: 129 active+clean; 412 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 122 op/s 2023-07-18T03:16:01.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:01 smithi179 ceph-mon[129229]: pgmap v260: 129 pgs: 129 active+clean; 412 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 122 op/s 2023-07-18T03:16:01.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:01 smithi101 ceph-mon[126379]: pgmap v260: 129 pgs: 129 active+clean; 412 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 122 op/s 2023-07-18T03:16:01.599 INFO:tasks.workunit.client.0.smithi101.stdout: 1 569359 6.47 MB/sec execute 326 sec latency 57.994 ms 2023-07-18T03:16:02.599 INFO:tasks.workunit.client.0.smithi101.stdout: 1 569874 6.46 MB/sec execute 327 sec latency 67.298 ms 2023-07-18T03:16:03.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:03 smithi155 ceph-mon[128885]: pgmap v261: 129 pgs: 129 active+clean; 413 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 113 op/s 2023-07-18T03:16:03.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:03 smithi179 ceph-mon[129229]: pgmap v261: 129 pgs: 129 active+clean; 413 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 113 op/s 2023-07-18T03:16:03.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:03 smithi101 ceph-mon[126379]: pgmap v261: 129 pgs: 129 active+clean; 413 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 113 op/s 2023-07-18T03:16:03.599 INFO:tasks.workunit.client.0.smithi101.stdout: 1 570036 6.44 MB/sec execute 328 sec latency 879.206 ms 2023-07-18T03:16:04.599 INFO:tasks.workunit.client.0.smithi101.stdout: 1 570435 6.42 MB/sec execute 329 sec latency 1710.981 ms 2023-07-18T03:16:05.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:05 smithi155 ceph-mon[128885]: pgmap v262: 129 pgs: 129 active+clean; 413 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 105 op/s 2023-07-18T03:16:05.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:05 smithi179 ceph-mon[129229]: pgmap v262: 129 pgs: 129 active+clean; 413 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 105 op/s 2023-07-18T03:16:05.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:05 smithi101 ceph-mon[126379]: pgmap v262: 129 pgs: 129 active+clean; 413 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 105 op/s 2023-07-18T03:16:05.600 INFO:tasks.workunit.client.0.smithi101.stdout: 1 572112 6.44 MB/sec execute 330 sec latency 21.836 ms 2023-07-18T03:16:06.600 INFO:tasks.workunit.client.0.smithi101.stdout: 1 573066 6.43 MB/sec execute 331 sec latency 14.448 ms 2023-07-18T03:16:07.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:07 smithi155 ceph-mon[128885]: pgmap v263: 129 pgs: 129 active+clean; 417 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 192 op/s 2023-07-18T03:16:07.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:07 smithi155 ceph-mon[128885]: osdmap e97: 12 total, 12 up, 12 in 2023-07-18T03:16:07.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:07 smithi179 ceph-mon[129229]: pgmap v263: 129 pgs: 129 active+clean; 417 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 192 op/s 2023-07-18T03:16:07.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:07 smithi179 ceph-mon[129229]: osdmap e97: 12 total, 12 up, 12 in 2023-07-18T03:16:07.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:07 smithi101 ceph-mon[126379]: pgmap v263: 129 pgs: 129 active+clean; 417 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 192 op/s 2023-07-18T03:16:07.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:07 smithi101 ceph-mon[126379]: osdmap e97: 12 total, 12 up, 12 in 2023-07-18T03:16:07.600 INFO:tasks.workunit.client.0.smithi101.stdout: 1 573579 6.42 MB/sec execute 332 sec latency 490.769 ms 2023-07-18T03:16:08.600 INFO:tasks.workunit.client.0.smithi101.stdout: 1 573579 6.40 MB/sec execute 333 sec latency 1490.935 ms 2023-07-18T03:16:09.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:09 smithi155 ceph-mon[128885]: pgmap v265: 129 pgs: 129 active+clean; 417 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 167 op/s 2023-07-18T03:16:09.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:09 smithi179 ceph-mon[129229]: pgmap v265: 129 pgs: 129 active+clean; 417 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 167 op/s 2023-07-18T03:16:09.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:09 smithi101 ceph-mon[126379]: pgmap v265: 129 pgs: 129 active+clean; 417 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 167 op/s 2023-07-18T03:16:09.600 INFO:tasks.workunit.client.0.smithi101.stdout: 1 574029 6.38 MB/sec execute 334 sec latency 2321.981 ms 2023-07-18T03:16:10.601 INFO:tasks.workunit.client.0.smithi101.stdout: 1 575695 6.40 MB/sec execute 335 sec latency 29.143 ms 2023-07-18T03:16:11.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:11 smithi155 ceph-mon[128885]: pgmap v266: 129 pgs: 129 active+clean; 417 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 173 op/s 2023-07-18T03:16:11.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:11 smithi179 ceph-mon[129229]: pgmap v266: 129 pgs: 129 active+clean; 417 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 173 op/s 2023-07-18T03:16:11.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:11 smithi101 ceph-mon[126379]: pgmap v266: 129 pgs: 129 active+clean; 417 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 173 op/s 2023-07-18T03:16:11.601 INFO:tasks.workunit.client.0.smithi101.stdout: 1 576836 6.40 MB/sec execute 336 sec latency 18.167 ms 2023-07-18T03:16:12.601 INFO:tasks.workunit.client.0.smithi101.stdout: 1 577126 6.38 MB/sec execute 337 sec latency 608.164 ms 2023-07-18T03:16:13.275 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:16:13.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:13 smithi155 ceph-mon[128885]: pgmap v267: 129 pgs: 129 active+clean; 418 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 168 op/s 2023-07-18T03:16:13.342 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:13 smithi101 ceph-mon[126379]: pgmap v267: 129 pgs: 129 active+clean; 418 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 168 op/s 2023-07-18T03:16:13.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:13 smithi179 ceph-mon[129229]: pgmap v267: 129 pgs: 129 active+clean; 418 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 168 op/s 2023-07-18T03:16:13.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.491+0000 7fd1a1fff700 1 -- 172.21.15.101:0/3415384056 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd19c0ffad0 msgr2=0x7fd19c0fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:13.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.491+0000 7fd1a1fff700 1 --2- 172.21.15.101:0/3415384056 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd19c0ffad0 0x7fd19c0fff40 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd190009990 tx=0x7fd190009ca0 comp rx=0 tx=0).stop 2023-07-18T03:16:13.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.491+0000 7fd1a1fff700 1 -- 172.21.15.101:0/3415384056 shutdown_connections 2023-07-18T03:16:13.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.491+0000 7fd1a1fff700 1 --2- 172.21.15.101:0/3415384056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd19c105ae0 0x7fd19c105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:13.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.491+0000 7fd1a1fff700 1 --2- 172.21.15.101:0/3415384056 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd19c100480 0x7fd19c10c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:13.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.491+0000 7fd1a1fff700 1 --2- 172.21.15.101:0/3415384056 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd19c0ffad0 0x7fd19c0fff40 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:13.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.491+0000 7fd1a1fff700 1 -- 172.21.15.101:0/3415384056 >> 172.21.15.101:0/3415384056 conn(0x7fd19c0fb5f0 msgr2=0x7fd19c0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:13.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.491+0000 7fd1a1fff700 1 -- 172.21.15.101:0/3415384056 shutdown_connections 2023-07-18T03:16:13.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.491+0000 7fd1a1fff700 1 -- 172.21.15.101:0/3415384056 wait complete. 2023-07-18T03:16:13.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.492+0000 7fd1a1fff700 1 Processor -- start 2023-07-18T03:16:13.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.493+0000 7fd1a1fff700 1 -- start start 2023-07-18T03:16:13.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.493+0000 7fd1a1fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd19c0ffad0 0x7fd19c197d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:13.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.493+0000 7fd1a1fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd19c100480 0x7fd19c1982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:13.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.493+0000 7fd1a1fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd19c105ae0 0x7fd19c19c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:13.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.493+0000 7fd1a1fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c19cd70 con 0x7fd19c0ffad0 2023-07-18T03:16:13.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.493+0000 7fd1a1fff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c19cee0 con 0x7fd19c100480 2023-07-18T03:16:13.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.493+0000 7fd1a1fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c19d0a0 con 0x7fd19c105ae0 2023-07-18T03:16:13.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.493+0000 7fd19b7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd19c0ffad0 0x7fd19c197d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:13.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.493+0000 7fd19b7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd19c0ffad0 0x7fd19c197d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:56506/0 (socket says 172.21.15.101:56506) 2023-07-18T03:16:13.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.493+0000 7fd19b7fe700 1 -- 172.21.15.101:0/1641584084 learned_addr learned my addr 172.21.15.101:0/1641584084 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:13.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.493+0000 7fd19bfff700 1 --2- 172.21.15.101:0/1641584084 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd19c105ae0 0x7fd19c19c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:13.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.493+0000 7fd19affd700 1 --2- 172.21.15.101:0/1641584084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd19c100480 0x7fd19c1982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:13.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.494+0000 7fd19b7fe700 1 -- 172.21.15.101:0/1641584084 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd19c105ae0 msgr2=0x7fd19c19c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:13.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.494+0000 7fd19b7fe700 1 --2- 172.21.15.101:0/1641584084 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd19c105ae0 0x7fd19c19c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:13.492 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.494+0000 7fd19b7fe700 1 -- 172.21.15.101:0/1641584084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd19c100480 msgr2=0x7fd19c1982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:13.492 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.494+0000 7fd19b7fe700 1 --2- 172.21.15.101:0/1641584084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd19c100480 0x7fd19c1982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:13.492 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.494+0000 7fd19b7fe700 1 -- 172.21.15.101:0/1641584084 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1900095f0 con 0x7fd19c0ffad0 2023-07-18T03:16:13.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.494+0000 7fd19b7fe700 1 --2- 172.21.15.101:0/1641584084 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd19c0ffad0 0x7fd19c197d90 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd18c00e9a0 tx=0x7fd18c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:13.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.494+0000 7fd198ff9700 1 -- 172.21.15.101:0/1641584084 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18c009790 con 0x7fd19c0ffad0 2023-07-18T03:16:13.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.494+0000 7fd198ff9700 1 -- 172.21.15.101:0/1641584084 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd18c010430 con 0x7fd19c0ffad0 2023-07-18T03:16:13.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.494+0000 7fd198ff9700 1 -- 172.21.15.101:0/1641584084 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18c018680 con 0x7fd19c0ffad0 2023-07-18T03:16:13.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.494+0000 7fd1a1fff700 1 -- 172.21.15.101:0/1641584084 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd19c19d380 con 0x7fd19c0ffad0 2023-07-18T03:16:13.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.494+0000 7fd1a1fff700 1 -- 172.21.15.101:0/1641584084 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd19c1a4bc0 con 0x7fd19c0ffad0 2023-07-18T03:16:13.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.497+0000 7fd198ff9700 1 -- 172.21.15.101:0/1641584084 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fd18c0105a0 con 0x7fd19c0ffad0 2023-07-18T03:16:13.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.499+0000 7fd1b5832b80 1 Processor -- start 2023-07-18T03:16:13.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.499+0000 7fd1b5832b80 1 -- start start 2023-07-18T03:16:13.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.500+0000 7fd198ff9700 1 --2- 172.21.15.101:0/1641584084 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd1840a1340 0x7fd1840a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:13.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.500+0000 7fd1b5832b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618c82b22d0 0x5618c82b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:13.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.500+0000 7fd1b5832b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5618c82b2be0 0x5618c82bd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:13.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.500+0000 7fd1b5832b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618c82bdad0 0x5618c82bfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:13.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.500+0000 7fd198ff9700 1 -- 172.21.15.101:0/1641584084 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9549+0+0 (secure 0 0 0) 0x7fd18c014070 con 0x7fd19c0ffad0 2023-07-18T03:16:13.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.500+0000 7fd1b5832b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618c81b8cb0 con 0x5618c82b2be0 2023-07-18T03:16:13.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.500+0000 7fd1b5832b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618c820ba40 con 0x5618c82b22d0 2023-07-18T03:16:13.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.500+0000 7fd1b5832b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618c8204ed0 con 0x5618c82bdad0 2023-07-18T03:16:13.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.500+0000 7fd19bfff700 1 --2- 172.21.15.101:0/1641584084 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd1840a1340 0x7fd1840a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:13.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.500+0000 7fd19affd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5618c82b2be0 0x5618c82bd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:13.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.500+0000 7fd19affd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5618c82b2be0 0x5618c82bd590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:56520/0 (socket says 172.21.15.101:56520) 2023-07-18T03:16:13.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.500+0000 7fd19affd700 1 -- 172.21.15.101:0/1001654878 learned_addr learned my addr 172.21.15.101:0/1001654878 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:13.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.500+0000 7fd19b7fe700 1 --2- 172.21.15.101:0/1001654878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618c82b22d0 0x5618c82b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:13.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.501+0000 7fd19bfff700 1 --2- 172.21.15.101:0/1001654878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618c82bdad0 0x5618c82bfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:13.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.501+0000 7fd19affd700 1 -- 172.21.15.101:0/1001654878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618c82bdad0 msgr2=0x5618c82bfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:13.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.501+0000 7fd19affd700 1 --2- 172.21.15.101:0/1001654878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618c82bdad0 0x5618c82bfeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:13.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.501+0000 7fd19affd700 1 -- 172.21.15.101:0/1001654878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618c82b22d0 msgr2=0x5618c82b26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:13.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.501+0000 7fd19affd700 1 --2- 172.21.15.101:0/1001654878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618c82b22d0 0x5618c82b26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:13.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.501+0000 7fd19affd700 1 -- 172.21.15.101:0/1001654878 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618c8032de0 con 0x5618c82b2be0 2023-07-18T03:16:13.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.501+0000 7fd19bfff700 1 --2- 172.21.15.101:0/1641584084 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd1840a1340 0x7fd1840a37f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd194003fd0 tx=0x7fd194004320 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:13.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.501+0000 7fd19affd700 1 --2- 172.21.15.101:0/1001654878 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5618c82b2be0 0x5618c82bd590 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fd190009990 tx=0x7fd190005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db10e72ef93351c8 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:13.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.501+0000 7fd181ffb700 1 -- 172.21.15.101:0/1001654878 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19001d070 con 0x5618c82b2be0 2023-07-18T03:16:13.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.501+0000 7fd181ffb700 1 -- 172.21.15.101:0/1001654878 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd19001e040 con 0x5618c82b2be0 2023-07-18T03:16:13.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.501+0000 7fd181ffb700 1 -- 172.21.15.101:0/1001654878 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd190004930 con 0x5618c82b2be0 2023-07-18T03:16:13.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.502+0000 7fd1b5832b80 1 -- 172.21.15.101:0/1001654878 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5618c82b2be0 msgr2=0x5618c82bd590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:13.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.502+0000 7fd1b5832b80 1 --2- 172.21.15.101:0/1001654878 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5618c82b2be0 0x5618c82bd590 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fd190009990 tx=0x7fd190005f00 comp rx=0 tx=0).stop 2023-07-18T03:16:13.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.502+0000 7fd1b5832b80 1 -- 172.21.15.101:0/1001654878 shutdown_connections 2023-07-18T03:16:13.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.502+0000 7fd1b5832b80 1 --2- 172.21.15.101:0/1001654878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618c82b22d0 0x5618c82b26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:13.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.502+0000 7fd1b5832b80 1 --2- 172.21.15.101:0/1001654878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618c82bdad0 0x5618c82bfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:13.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.502+0000 7fd1b5832b80 1 --2- 172.21.15.101:0/1001654878 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5618c82b2be0 0x5618c82bd590 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:13.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.502+0000 7fd1b5832b80 1 -- 172.21.15.101:0/1001654878 >> 172.21.15.101:0/1001654878 conn(0x5618c82a8da0 msgr2=0x5618c82ae6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:13.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.502+0000 7fd1b5832b80 1 -- 172.21.15.101:0/1001654878 shutdown_connections 2023-07-18T03:16:13.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.502+0000 7fd1b5832b80 1 -- 172.21.15.101:0/1001654878 wait complete. 2023-07-18T03:16:13.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.503+0000 7fd1b5832b80 1 -- start start 2023-07-18T03:16:13.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.503+0000 7fd1b5832b80 1 Processor -- start 2023-07-18T03:16:13.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.503+0000 7fd1b5832b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618c82b22d0 0x5618c82bd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:13.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.503+0000 7fd1b5832b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618c82b2be0 0x5618c82ba720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:13.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.503+0000 7fd1b5832b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5618c82bdad0 0x5618c82bae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:13.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.503+0000 7fd1b5832b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618c81a4ac0 con 0x5618c82bdad0 2023-07-18T03:16:13.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.503+0000 7fd1b5832b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618c81a47c0 con 0x5618c82b22d0 2023-07-18T03:16:13.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.503+0000 7fd1b5832b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618c8088920 con 0x5618c82b2be0 2023-07-18T03:16:13.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.504+0000 7fd1b5832b80 10 client.0 _ll_register_callbacks cb 0x5618c81a2760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:16:13.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.504+0000 7fd19bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5618c82bdad0 0x5618c82bae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:13.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.504+0000 7fd19b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618c82b22d0 0x5618c82bd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:13.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.504+0000 7fd19b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618c82b22d0 0x5618c82bd510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:52300/0 (socket says 172.21.15.101:52300) 2023-07-18T03:16:13.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.504+0000 7fd19bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5618c82bdad0 0x5618c82bae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:56526/0 (socket says 172.21.15.101:56526) 2023-07-18T03:16:13.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.504+0000 7fd19b7fe700 1 -- 172.21.15.101:0/4286119617 learned_addr learned my addr 172.21.15.101:0/4286119617 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:13.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.504+0000 7fd19affd700 1 --2- 172.21.15.101:0/4286119617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618c82b2be0 0x5618c82ba720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:13.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.504+0000 7fd19b7fe700 1 -- 172.21.15.101:0/4286119617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618c82b2be0 msgr2=0x5618c82ba720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:13.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.504+0000 7fd19b7fe700 1 --2- 172.21.15.101:0/4286119617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618c82b2be0 0x5618c82ba720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:13.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.504+0000 7fd19b7fe700 1 -- 172.21.15.101:0/4286119617 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5618c82bdad0 msgr2=0x5618c82bae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:13.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.504+0000 7fd19b7fe700 1 --2- 172.21.15.101:0/4286119617 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5618c82bdad0 0x5618c82bae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:13.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.504+0000 7fd19b7fe700 1 -- 172.21.15.101:0/4286119617 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd19c10d210 con 0x5618c82b22d0 2023-07-18T03:16:13.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.504+0000 7fd19bfff700 1 --2- 172.21.15.101:0/4286119617 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5618c82bdad0 0x5618c82bae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:16:13.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.504+0000 7fd19b7fe700 1 --2- 172.21.15.101:0/4286119617 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618c82b22d0 0x5618c82bd510 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd18c096980 tx=0x7fd18c051660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85b4e16235a1e95 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:13.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.504+0000 7fd17affd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:16:13.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.505+0000 7fd17affd700 1 -- 172.21.15.101:0/4286119617 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18c0a2360 con 0x5618c82b22d0 2023-07-18T03:16:13.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.505+0000 7fd17affd700 1 -- 172.21.15.101:0/4286119617 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9549+0+0 (secure 0 0 0) 0x7fd18c09a170 con 0x5618c82b22d0 2023-07-18T03:16:13.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.505+0000 7fd1b5832b80 1 -- 172.21.15.101:0/4286119617 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5618c815e4f0 con 0x5618c82b22d0 2023-07-18T03:16:13.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.505+0000 7fd17affd700 1 -- 172.21.15.101:0/4286119617 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd18c09a330 con 0x5618c82b22d0 2023-07-18T03:16:13.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.505+0000 7fd1b5832b80 10 client.27281 fetch_fsmap learned FSMap version 31 2023-07-18T03:16:13.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.505+0000 7fd1b5832b80 1 -- 172.21.15.101:0/4286119617 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5618c801bc60 con 0x5618c82b22d0 2023-07-18T03:16:13.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.505+0000 7fd17affd700 1 -- 172.21.15.101:0/4286119617 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5618c815e4f0 con 0x5618c82b22d0 2023-07-18T03:16:13.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.505+0000 7fd17affd700 1 -- 172.21.15.101:0/4286119617 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fd18c0a1fb0 con 0x5618c82b22d0 2023-07-18T03:16:13.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.505+0000 7fd1b5832b80 10 client.27281 fetch_fsmap finished waiting for FSMap version 31 2023-07-18T03:16:13.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.505+0000 7fd1b5832b80 10 client.27281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:16:13.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.505+0000 7fd1b5832b80 20 client.27281 populate_metadata read hostname 'smithi101' 2023-07-18T03:16:13.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.505+0000 7fd1b5832b80 1 --2- 172.21.15.101:0/4286119617 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5618c82cd440 0x5618c82cf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:13.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.505+0000 7fd1b5832b80 4 client.27281 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:16:13.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.505+0000 7fd1b5832b80 1 -- 172.21.15.101:0/4286119617 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5618c81fdeb0 con 0x5618c82cd440 2023-07-18T03:16:13.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.506+0000 7fd19affd700 1 --2- 172.21.15.101:0/4286119617 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5618c82cd440 0x5618c82cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:13.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.506+0000 7fd19affd700 1 --2- 172.21.15.101:0/4286119617 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5618c82cd440 0x5618c82cf850 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5546566b248f19ff server_cookie=49b7ed856db74b4e in_seq=0 out_seq=0 2023-07-18T03:16:13.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.506+0000 7fd17affd700 10 client.27281 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:16:13.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.507+0000 7fd17affd700 1 -- 172.21.15.101:0/4286119617 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5618c81fdeb0 con 0x5618c82cd440 2023-07-18T03:16:13.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.507+0000 7fd17affd700 10 client.27281 handle_command_reply: tid=0 2023-07-18T03:16:13.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.507+0000 7fd1b5832b80 1 client.27281 shutdown 2023-07-18T03:16:13.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.508+0000 7fd1b5832b80 1 -- 172.21.15.101:0/4286119617 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618c82b22d0 msgr2=0x5618c82bd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:13.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.508+0000 7fd1b5832b80 1 --2- 172.21.15.101:0/4286119617 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618c82b22d0 0x5618c82bd510 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd18c096980 tx=0x7fd18c051660 comp rx=0 tx=0).stop 2023-07-18T03:16:13.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.508+0000 7fd1b5832b80 1 -- 172.21.15.101:0/4286119617 shutdown_connections 2023-07-18T03:16:13.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.508+0000 7fd1b5832b80 1 --2- 172.21.15.101:0/4286119617 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5618c82cd440 0x5618c82cf850 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:13.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.508+0000 7fd1b5832b80 1 --2- 172.21.15.101:0/4286119617 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618c82b22d0 0x5618c82bd510 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:13.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.508+0000 7fd1b5832b80 1 --2- 172.21.15.101:0/4286119617 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5618c82bdad0 0x5618c82bae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:13.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.508+0000 7fd1b5832b80 1 --2- 172.21.15.101:0/4286119617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618c82b2be0 0x5618c82ba720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:13.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.508+0000 7fd17affd700 0 client.27281 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:16:13.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.508+0000 7fd1b5832b80 1 -- 172.21.15.101:0/4286119617 >> 172.21.15.101:0/4286119617 conn(0x5618c82a8da0 msgr2=0x5618c82ae290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:13.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.508+0000 7fd1b5832b80 1 -- 172.21.15.101:0/4286119617 shutdown_connections 2023-07-18T03:16:13.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.508+0000 7fd1b5832b80 1 -- 172.21.15.101:0/4286119617 wait complete. 2023-07-18T03:16:13.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.508+0000 7fd1b5832b80 20 client.27281 trim_cache size 0 max 16384 2023-07-18T03:16:13.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.525+0000 7fd1b5832b80 1 Processor -- start 2023-07-18T03:16:13.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.525+0000 7fd1b5832b80 1 -- start start 2023-07-18T03:16:13.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.526+0000 7fd1b5832b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618c82b22d0 0x5618c82bd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:13.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.526+0000 7fd1b5832b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5618c82b2be0 0x5618c82b9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:13.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.526+0000 7fd1b5832b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618c82bdad0 0x5618c82ba500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:13.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.526+0000 7fd1b5832b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618c82b3d60 con 0x5618c82b2be0 2023-07-18T03:16:13.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.526+0000 7fd1b5832b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618c82aebe0 con 0x5618c82bdad0 2023-07-18T03:16:13.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.526+0000 7fd1b5832b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618c82b41a0 con 0x5618c82b22d0 2023-07-18T03:16:13.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.526+0000 7fd19affd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5618c82b2be0 0x5618c82b9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:13.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.526+0000 7fd19affd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5618c82b2be0 0x5618c82b9fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:56542/0 (socket says 172.21.15.101:56542) 2023-07-18T03:16:13.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.526+0000 7fd19affd700 1 -- 172.21.15.101:0/171910933 learned_addr learned my addr 172.21.15.101:0/171910933 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:13.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.526+0000 7fd19b7fe700 1 --2- 172.21.15.101:0/171910933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618c82b22d0 0x5618c82bd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:13.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.526+0000 7fd19bfff700 1 --2- 172.21.15.101:0/171910933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618c82bdad0 0x5618c82ba500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:13.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.526+0000 7fd19affd700 1 -- 172.21.15.101:0/171910933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618c82b22d0 msgr2=0x5618c82bd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:13.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.526+0000 7fd19affd700 1 --2- 172.21.15.101:0/171910933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618c82b22d0 0x5618c82bd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:13.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.526+0000 7fd19affd700 1 -- 172.21.15.101:0/171910933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618c82bdad0 msgr2=0x5618c82ba500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:13.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.526+0000 7fd19affd700 1 --2- 172.21.15.101:0/171910933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618c82bdad0 0x5618c82ba500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:13.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.526+0000 7fd19affd700 1 -- 172.21.15.101:0/171910933 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618c8059250 con 0x5618c82b2be0 2023-07-18T03:16:13.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.527+0000 7fd19affd700 1 --2- 172.21.15.101:0/171910933 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5618c82b2be0 0x5618c82b9fc0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fd190026000 tx=0x7fd1900057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c64b1ccbc1976a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:13.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.527+0000 7fd1827fc700 1 -- 172.21.15.101:0/171910933 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19001d040 con 0x5618c82b2be0 2023-07-18T03:16:13.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.527+0000 7fd1827fc700 1 -- 172.21.15.101:0/171910933 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd190033040 con 0x5618c82b2be0 2023-07-18T03:16:13.524 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.527+0000 7fd1827fc700 1 -- 172.21.15.101:0/171910933 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd190035ad0 con 0x5618c82b2be0 2023-07-18T03:16:13.526 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.527+0000 7fd1b5832b80 1 -- 172.21.15.101:0/171910933 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5618c82b2be0 msgr2=0x5618c82b9fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:13.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.527+0000 7fd1b5832b80 1 --2- 172.21.15.101:0/171910933 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5618c82b2be0 0x5618c82b9fc0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fd190026000 tx=0x7fd1900057b0 comp rx=0 tx=0).stop 2023-07-18T03:16:13.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.527+0000 7fd1b5832b80 1 -- 172.21.15.101:0/171910933 shutdown_connections 2023-07-18T03:16:13.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.527+0000 7fd1b5832b80 1 --2- 172.21.15.101:0/171910933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618c82bdad0 0x5618c82ba500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:13.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.527+0000 7fd1b5832b80 1 --2- 172.21.15.101:0/171910933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618c82b22d0 0x5618c82bd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:13.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.527+0000 7fd1b5832b80 1 --2- 172.21.15.101:0/171910933 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5618c82b2be0 0x5618c82b9fc0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:13.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.527+0000 7fd1b5832b80 1 -- 172.21.15.101:0/171910933 >> 172.21.15.101:0/171910933 conn(0x5618c82a8da0 msgr2=0x5618c82c96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:13.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.528+0000 7fd1b5832b80 1 -- 172.21.15.101:0/171910933 shutdown_connections 2023-07-18T03:16:13.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.528+0000 7fd1b5832b80 1 -- 172.21.15.101:0/171910933 wait complete. 2023-07-18T03:16:13.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.528+0000 7fd1b5832b80 1 -- start start 2023-07-18T03:16:13.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.528+0000 7fd1b5832b80 1 Processor -- start 2023-07-18T03:16:13.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.529+0000 7fd1b5832b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618c82b22d0 0x5618c82c3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:13.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.529+0000 7fd1b5832b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5618c82b2be0 0x5618c82c4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:13.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.529+0000 7fd1b5832b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618c82bdad0 0x5618c82c2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:13.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.529+0000 7fd1b5832b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618c82c2860 con 0x5618c82b2be0 2023-07-18T03:16:13.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.529+0000 7fd1b5832b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618c82c29d0 con 0x5618c82bdad0 2023-07-18T03:16:13.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.529+0000 7fd1b5832b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618c82c2b90 con 0x5618c82b22d0 2023-07-18T03:16:13.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.529+0000 7fd1b5832b80 10 client.0 _ll_register_callbacks cb 0x7fd18c099a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:16:13.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.529+0000 7fd19affd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5618c82b2be0 0x5618c82c4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:13.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.529+0000 7fd19affd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5618c82b2be0 0x5618c82c4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:56558/0 (socket says 172.21.15.101:56558) 2023-07-18T03:16:13.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.529+0000 7fd19affd700 1 -- 172.21.15.101:0/3392360504 learned_addr learned my addr 172.21.15.101:0/3392360504 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:13.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.529+0000 7fd19b7fe700 1 --2- 172.21.15.101:0/3392360504 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618c82b22d0 0x5618c82c3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:13.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.529+0000 7fd19bfff700 1 --2- 172.21.15.101:0/3392360504 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618c82bdad0 0x5618c82c2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:13.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.529+0000 7fd19affd700 1 -- 172.21.15.101:0/3392360504 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618c82b22d0 msgr2=0x5618c82c3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:13.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.529+0000 7fd19affd700 1 --2- 172.21.15.101:0/3392360504 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618c82b22d0 0x5618c82c3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:13.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.529+0000 7fd19affd700 1 -- 172.21.15.101:0/3392360504 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618c82bdad0 msgr2=0x5618c82c2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:13.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.529+0000 7fd19affd700 1 --2- 172.21.15.101:0/3392360504 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618c82bdad0 0x5618c82c2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:13.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.529+0000 7fd19affd700 1 -- 172.21.15.101:0/3392360504 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5618c8032de0 con 0x5618c82b2be0 2023-07-18T03:16:13.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.529+0000 7fd19affd700 1 --2- 172.21.15.101:0/3392360504 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5618c82b2be0 0x5618c82c4300 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fd190035530 tx=0x7fd1900058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0f643be0bdc8138 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:13.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.529+0000 7fd17affd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:16:13.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.530+0000 7fd17affd700 1 -- 172.21.15.101:0/3392360504 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19001d040 con 0x5618c82b2be0 2023-07-18T03:16:13.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.530+0000 7fd1b5832b80 1 -- 172.21.15.101:0/3392360504 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd19001e040 con 0x5618c82b2be0 2023-07-18T03:16:13.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.530+0000 7fd17affd700 1 -- 172.21.15.101:0/3392360504 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9549+0+0 (secure 0 0 0) 0x7fd190035870 con 0x5618c82b2be0 2023-07-18T03:16:13.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.530+0000 7fd17affd700 1 -- 172.21.15.101:0/3392360504 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd190035a30 con 0x5618c82b2be0 2023-07-18T03:16:13.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.530+0000 7fd1b5832b80 10 client.17250 fetch_fsmap learned FSMap version 31 2023-07-18T03:16:13.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.530+0000 7fd1b5832b80 1 -- 172.21.15.101:0/3392360504 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5618c80312b0 con 0x5618c82b2be0 2023-07-18T03:16:13.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.530+0000 7fd17affd700 1 -- 172.21.15.101:0/3392360504 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd19001e040 con 0x5618c82b2be0 2023-07-18T03:16:13.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.530+0000 7fd17affd700 1 -- 172.21.15.101:0/3392360504 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fd19000fbf0 con 0x5618c82b2be0 2023-07-18T03:16:13.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.530+0000 7fd1b5832b80 10 client.17250 fetch_fsmap finished waiting for FSMap version 31 2023-07-18T03:16:13.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.530+0000 7fd1b5832b80 10 client.17250 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:16:13.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.530+0000 7fd1b5832b80 20 client.17250 populate_metadata read hostname 'smithi101' 2023-07-18T03:16:13.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.530+0000 7fd1b5832b80 1 --2- 172.21.15.101:0/3392360504 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5618c8321f70 0x5618c8324380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:13.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.530+0000 7fd1b5832b80 4 client.17250 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:16:13.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.530+0000 7fd1b5832b80 1 -- 172.21.15.101:0/3392360504 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5618c81c0050 con 0x5618c8321f70 2023-07-18T03:16:13.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.531+0000 7fd19b7fe700 1 --2- 172.21.15.101:0/3392360504 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5618c8321f70 0x5618c8324380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:13.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.531+0000 7fd17affd700 10 client.17250 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:16:13.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.531+0000 7fd19b7fe700 1 --2- 172.21.15.101:0/3392360504 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5618c8321f70 0x5618c8324380 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=738a7981c2366457 server_cookie=a40f9c7e699a6be5 in_seq=0 out_seq=0 2023-07-18T03:16:13.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.531+0000 7fd17affd700 1 -- 172.21.15.101:0/3392360504 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5618c81c0050 con 0x5618c8321f70 2023-07-18T03:16:13.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.531+0000 7fd17affd700 10 client.17250 handle_command_reply: tid=0 2023-07-18T03:16:13.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.531+0000 7fd1b5832b80 1 client.17250 shutdown 2023-07-18T03:16:13.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.532+0000 7fd1b5832b80 1 -- 172.21.15.101:0/3392360504 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5618c82b2be0 msgr2=0x5618c82c4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:13.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.532+0000 7fd1b5832b80 1 --2- 172.21.15.101:0/3392360504 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5618c82b2be0 0x5618c82c4300 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fd190035530 tx=0x7fd1900058e0 comp rx=0 tx=0).stop 2023-07-18T03:16:13.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.532+0000 7fd1b5832b80 1 -- 172.21.15.101:0/3392360504 shutdown_connections 2023-07-18T03:16:13.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.532+0000 7fd1b5832b80 1 --2- 172.21.15.101:0/3392360504 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5618c8321f70 0x5618c8324380 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:13.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.532+0000 7fd1b5832b80 1 --2- 172.21.15.101:0/3392360504 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618c82bdad0 0x5618c82c2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:13.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.532+0000 7fd17affd700 0 client.17250 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:16:13.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.532+0000 7fd1b5832b80 1 --2- 172.21.15.101:0/3392360504 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618c82b22d0 0x5618c82c3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:13.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.532+0000 7fd1b5832b80 1 --2- 172.21.15.101:0/3392360504 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5618c82b2be0 0x5618c82c4300 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:13.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.532+0000 7fd1b5832b80 1 -- 172.21.15.101:0/3392360504 >> 172.21.15.101:0/3392360504 conn(0x5618c82a8da0 msgr2=0x5618c82ad8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:13.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.532+0000 7fd1b5832b80 1 -- 172.21.15.101:0/3392360504 shutdown_connections 2023-07-18T03:16:13.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.532+0000 7fd1b5832b80 1 -- 172.21.15.101:0/3392360504 wait complete. 2023-07-18T03:16:13.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.532+0000 7fd1b5832b80 20 client.17250 trim_cache size 0 max 16384 2023-07-18T03:16:13.536 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:16:13.536 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "no active scrubs running", 2023-07-18T03:16:13.536 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": {} 2023-07-18T03:16:13.536 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:16:13.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.532+0000 7fd1a1fff700 1 -- 172.21.15.101:0/1641584084 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd1840a1340 msgr2=0x7fd1840a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:13.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.532+0000 7fd1a1fff700 1 --2- 172.21.15.101:0/1641584084 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd1840a1340 0x7fd1840a37f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd194003fd0 tx=0x7fd194004320 comp rx=0 tx=0).stop 2023-07-18T03:16:13.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.532+0000 7fd1a1fff700 1 -- 172.21.15.101:0/1641584084 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd19c0ffad0 msgr2=0x7fd19c197d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:13.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.532+0000 7fd1a1fff700 1 --2- 172.21.15.101:0/1641584084 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd19c0ffad0 0x7fd19c197d90 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd18c00e9a0 tx=0x7fd18c00ecb0 comp rx=0 tx=0).stop 2023-07-18T03:16:13.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.533+0000 7fd1a1fff700 1 -- 172.21.15.101:0/1641584084 shutdown_connections 2023-07-18T03:16:13.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.533+0000 7fd1a1fff700 1 --2- 172.21.15.101:0/1641584084 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd19c105ae0 0x7fd19c19c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:13.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.533+0000 7fd1a1fff700 1 --2- 172.21.15.101:0/1641584084 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd19c0ffad0 0x7fd19c197d90 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:13.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.533+0000 7fd1a1fff700 1 --2- 172.21.15.101:0/1641584084 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd1840a1340 0x7fd1840a37f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:13.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.533+0000 7fd1a1fff700 1 --2- 172.21.15.101:0/1641584084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd19c100480 0x7fd19c1982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:13.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.533+0000 7fd1a1fff700 1 -- 172.21.15.101:0/1641584084 >> 172.21.15.101:0/1641584084 conn(0x7fd19c0fb5f0 msgr2=0x7fd19c10aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:13.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.533+0000 7fd1a1fff700 1 -- 172.21.15.101:0/1641584084 shutdown_connections 2023-07-18T03:16:13.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.533+0000 7fd1a1fff700 1 -- 172.21.15.101:0/1641584084 wait complete. 2023-07-18T03:16:13.544 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-18T03:16:13.545 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:16:13.601 INFO:tasks.workunit.client.0.smithi101.stdout: 1 577126 6.36 MB/sec execute 338 sec latency 1608.206 ms 2023-07-18T03:16:13.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.748+0000 7ff1fedff700 1 -- 172.21.15.101:0/950283538 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff1f80731c0 msgr2=0x7ff1f810bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:13.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.748+0000 7ff1fedff700 1 --2- 172.21.15.101:0/950283538 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff1f80731c0 0x7ff1f810bdf0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4009990 tx=0x7ff1e4009ca0 comp rx=0 tx=0).stop 2023-07-18T03:16:13.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.749+0000 7ff1fedff700 1 -- 172.21.15.101:0/950283538 shutdown_connections 2023-07-18T03:16:13.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.749+0000 7ff1fedff700 1 --2- 172.21.15.101:0/950283538 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1f80728b0 0x7ff1f8072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:13.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.749+0000 7ff1fedff700 1 --2- 172.21.15.101:0/950283538 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff1f810c330 0x7ff1f810e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:13.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.749+0000 7ff1fedff700 1 --2- 172.21.15.101:0/950283538 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff1f80731c0 0x7ff1f810bdf0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:13.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.749+0000 7ff1fedff700 1 -- 172.21.15.101:0/950283538 >> 172.21.15.101:0/950283538 conn(0x7ff1f80fb630 msgr2=0x7ff1f80fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:13.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.749+0000 7ff1fedff700 1 -- 172.21.15.101:0/950283538 shutdown_connections 2023-07-18T03:16:13.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.749+0000 7ff1fedff700 1 -- 172.21.15.101:0/950283538 wait complete. 2023-07-18T03:16:13.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.750+0000 7ff1fedff700 1 Processor -- start 2023-07-18T03:16:13.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.750+0000 7ff1fedff700 1 -- start start 2023-07-18T03:16:13.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.750+0000 7ff1fedff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1f80728b0 0x7ff1f8197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:13.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.750+0000 7ff1fedff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff1f80731c0 0x7ff1f81982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:13.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.750+0000 7ff1fedff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff1f810c330 0x7ff1f819c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:13.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.750+0000 7ff1fedff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f819cd90 con 0x7ff1f810c330 2023-07-18T03:16:13.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.750+0000 7ff1fedff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f819cf00 con 0x7ff1f80728b0 2023-07-18T03:16:13.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.750+0000 7ff1fedff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f819d0c0 con 0x7ff1f80731c0 2023-07-18T03:16:13.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.750+0000 7ff1fd39c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff1f810c330 0x7ff1f819c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:13.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.750+0000 7ff1fd39c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff1f810c330 0x7ff1f819c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:56572/0 (socket says 172.21.15.101:56572) 2023-07-18T03:16:13.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.750+0000 7ff1fd39c700 1 -- 172.21.15.101:0/2997705627 learned_addr learned my addr 172.21.15.101:0/2997705627 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:13.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.750+0000 7ff1f7fff700 1 --2- 172.21.15.101:0/2997705627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff1f80731c0 0x7ff1f81982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:13.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.750+0000 7ff1fcb9b700 1 --2- 172.21.15.101:0/2997705627 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1f80728b0 0x7ff1f8197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:13.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.751+0000 7ff1fd39c700 1 -- 172.21.15.101:0/2997705627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff1f80731c0 msgr2=0x7ff1f81982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:13.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.751+0000 7ff1fd39c700 1 --2- 172.21.15.101:0/2997705627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff1f80731c0 0x7ff1f81982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:13.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.751+0000 7ff1fd39c700 1 -- 172.21.15.101:0/2997705627 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1f80728b0 msgr2=0x7ff1f8197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:13.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.751+0000 7ff1fd39c700 1 --2- 172.21.15.101:0/2997705627 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1f80728b0 0x7ff1f8197db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:13.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.751+0000 7ff1fd39c700 1 -- 172.21.15.101:0/2997705627 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e40095f0 con 0x7ff1f810c330 2023-07-18T03:16:13.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.751+0000 7ff1fd39c700 1 --2- 172.21.15.101:0/2997705627 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff1f810c330 0x7ff1f819c750 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ff1f000ef30 tx=0x7ff1f000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:13.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.751+0000 7ff1f5ffb700 1 -- 172.21.15.101:0/2997705627 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f0018070 con 0x7ff1f810c330 2023-07-18T03:16:13.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.751+0000 7ff1f5ffb700 1 -- 172.21.15.101:0/2997705627 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1f000f040 con 0x7ff1f810c330 2023-07-18T03:16:13.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.751+0000 7ff1fedff700 1 -- 172.21.15.101:0/2997705627 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1f819d340 con 0x7ff1f810c330 2023-07-18T03:16:13.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.751+0000 7ff1f5ffb700 1 -- 172.21.15.101:0/2997705627 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f000cd00 con 0x7ff1f810c330 2023-07-18T03:16:13.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.751+0000 7ff1fedff700 1 -- 172.21.15.101:0/2997705627 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1f81a4be0 con 0x7ff1f810c330 2023-07-18T03:16:13.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.752+0000 7ff1fedff700 1 -- 172.21.15.101:0/2997705627 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1f8107540 con 0x7ff1f810c330 2023-07-18T03:16:13.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.754+0000 7ff1f5ffb700 1 -- 172.21.15.101:0/2997705627 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7ff1f0003a00 con 0x7ff1f810c330 2023-07-18T03:16:13.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.756+0000 7ff1f5ffb700 1 --2- 172.21.15.101:0/2997705627 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff1e80a1380 0x7ff1e80a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:13.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.756+0000 7ff1f5ffb700 1 -- 172.21.15.101:0/2997705627 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9549+0+0 (secure 0 0 0) 0x7ff1f00ccb70 con 0x7ff1f810c330 2023-07-18T03:16:13.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.756+0000 7ff1fcb9b700 1 --2- 172.21.15.101:0/2997705627 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff1e80a1380 0x7ff1e80a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:13.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.756+0000 7ff1fcb9b700 1 --2- 172.21.15.101:0/2997705627 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff1e80a1380 0x7ff1e80a3830 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec005950 tx=0x7ff1ec00b410 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:13.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:13.759+0000 7ff1f5ffb700 1 -- 172.21.15.101:0/2997705627 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff1f0095bb0 con 0x7ff1f810c330 2023-07-18T03:16:13.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.004+0000 7ff1fedff700 1 -- 172.21.15.101:0/2997705627 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff1f8067300 con 0x7ff1f810c330 2023-07-18T03:16:13.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.005+0000 7ff1f5ffb700 1 -- 172.21.15.101:0/2997705627 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+10861 (secure 0 0 0) 0x7ff1f00991d0 con 0x7ff1f810c330 2023-07-18T03:16:14.000 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:16:14.000 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/4192846769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":4192846769},{"type":"v1","addr":"172.21.15.101:6839","nonce":4192846769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6841/3824340175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6840","nonce":3824340175},{"type":"v1","addr":"172.21.15.101:6841","nonce":3824340175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:04:59.361185+0000","modified":"2023-07-18T03:15:56.007271+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.155:6839/2055462483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2055462483},{"type":"v1","addr":"172.21.15.155:6839","nonce":2055462483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:16:14.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.007+0000 7ff1fedff700 1 -- 172.21.15.101:0/2997705627 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff1e80a1380 msgr2=0x7ff1e80a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:14.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.007+0000 7ff1fedff700 1 --2- 172.21.15.101:0/2997705627 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff1e80a1380 0x7ff1e80a3830 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec005950 tx=0x7ff1ec00b410 comp rx=0 tx=0).stop 2023-07-18T03:16:14.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.007+0000 7ff1fedff700 1 -- 172.21.15.101:0/2997705627 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff1f810c330 msgr2=0x7ff1f819c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:14.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.007+0000 7ff1fedff700 1 --2- 172.21.15.101:0/2997705627 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff1f810c330 0x7ff1f819c750 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ff1f000ef30 tx=0x7ff1f000c3a0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.007+0000 7ff1fedff700 1 -- 172.21.15.101:0/2997705627 shutdown_connections 2023-07-18T03:16:14.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.007+0000 7ff1fedff700 1 --2- 172.21.15.101:0/2997705627 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff1e80a1380 0x7ff1e80a3830 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.007+0000 7ff1fedff700 1 --2- 172.21.15.101:0/2997705627 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1f80728b0 0x7ff1f8197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.007+0000 7ff1fedff700 1 --2- 172.21.15.101:0/2997705627 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff1f810c330 0x7ff1f819c750 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.007+0000 7ff1fedff700 1 --2- 172.21.15.101:0/2997705627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff1f80731c0 0x7ff1f81982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.007+0000 7ff1fedff700 1 -- 172.21.15.101:0/2997705627 >> 172.21.15.101:0/2997705627 conn(0x7ff1f80fb630 msgr2=0x7ff1f810e400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:14.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.008+0000 7ff1fedff700 1 -- 172.21.15.101:0/2997705627 shutdown_connections 2023-07-18T03:16:14.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.008+0000 7ff1fedff700 1 -- 172.21.15.101:0/2997705627 wait complete. 2023-07-18T03:16:14.004 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 31 2023-07-18T03:16:14.020 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-18T03:16:14.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.231+0000 7f0e43bff700 1 -- 172.21.15.101:0/4015269186 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0e3c0ffae0 msgr2=0x7f0e3c0fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:14.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.231+0000 7f0e43bff700 1 --2- 172.21.15.101:0/4015269186 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0e3c0ffae0 0x7f0e3c0fff50 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c009990 tx=0x7f0e2c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.231+0000 7f0e43bff700 1 -- 172.21.15.101:0/4015269186 shutdown_connections 2023-07-18T03:16:14.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.231+0000 7f0e43bff700 1 --2- 172.21.15.101:0/4015269186 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e3c105af0 0x7f0e3c105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.231+0000 7f0e43bff700 1 --2- 172.21.15.101:0/4015269186 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e3c100490 0x7f0e3c10c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.231+0000 7f0e43bff700 1 --2- 172.21.15.101:0/4015269186 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0e3c0ffae0 0x7f0e3c0fff50 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.231+0000 7f0e43bff700 1 -- 172.21.15.101:0/4015269186 >> 172.21.15.101:0/4015269186 conn(0x7f0e3c0fb5e0 msgr2=0x7f0e3c0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:14.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.231+0000 7f0e43bff700 1 -- 172.21.15.101:0/4015269186 shutdown_connections 2023-07-18T03:16:14.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.231+0000 7f0e43bff700 1 -- 172.21.15.101:0/4015269186 wait complete. 2023-07-18T03:16:14.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.233+0000 7f0e43bff700 1 Processor -- start 2023-07-18T03:16:14.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.233+0000 7f0e43bff700 1 -- start start 2023-07-18T03:16:14.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.233+0000 7f0e43bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0e3c0ffae0 0x7f0e3c19b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.233+0000 7f0e43bff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e3c100490 0x7f0e3c19bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.233+0000 7f0e43bff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e3c105af0 0x7f0e3c195a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.233+0000 7f0e43bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c1960c0 con 0x7f0e3c0ffae0 2023-07-18T03:16:14.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.233+0000 7f0e43bff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c196230 con 0x7f0e3c100490 2023-07-18T03:16:14.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.233+0000 7f0e43bff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c1963f0 con 0x7f0e3c105af0 2023-07-18T03:16:14.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.234+0000 7f0e4199b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0e3c0ffae0 0x7f0e3c19b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.234+0000 7f0e4199b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0e3c0ffae0 0x7f0e3c19b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:56590/0 (socket says 172.21.15.101:56590) 2023-07-18T03:16:14.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.234+0000 7f0e4199b700 1 -- 172.21.15.101:0/1547268060 learned_addr learned my addr 172.21.15.101:0/1547268060 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:14.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.234+0000 7f0e4219c700 1 --2- 172.21.15.101:0/1547268060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e3c105af0 0x7f0e3c195a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.234+0000 7f0e4119a700 1 --2- 172.21.15.101:0/1547268060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e3c100490 0x7f0e3c19bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.234+0000 7f0e4199b700 1 -- 172.21.15.101:0/1547268060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e3c105af0 msgr2=0x7f0e3c195a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:14.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.234+0000 7f0e4199b700 1 --2- 172.21.15.101:0/1547268060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e3c105af0 0x7f0e3c195a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.234+0000 7f0e4199b700 1 -- 172.21.15.101:0/1547268060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e3c100490 msgr2=0x7f0e3c19bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:14.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.234+0000 7f0e4199b700 1 --2- 172.21.15.101:0/1547268060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e3c100490 0x7f0e3c19bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.234+0000 7f0e4199b700 1 -- 172.21.15.101:0/1547268060 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e2c0095f0 con 0x7f0e3c0ffae0 2023-07-18T03:16:14.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.234+0000 7f0e4119a700 1 --2- 172.21.15.101:0/1547268060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e3c100490 0x7f0e3c19bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:16:14.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.234+0000 7f0e4199b700 1 --2- 172.21.15.101:0/1547268060 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0e3c0ffae0 0x7f0e3c19b9e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f0e3800e9a0 tx=0x7f0e3800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:14.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.234+0000 7f0e32ffd700 1 -- 172.21.15.101:0/1547268060 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e38009790 con 0x7f0e3c0ffae0 2023-07-18T03:16:14.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.235+0000 7f0e43bff700 1 -- 172.21.15.101:0/1547268060 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e3c1966d0 con 0x7f0e3c0ffae0 2023-07-18T03:16:14.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.235+0000 7f0e32ffd700 1 -- 172.21.15.101:0/1547268060 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e3800cbd0 con 0x7f0e3c0ffae0 2023-07-18T03:16:14.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.235+0000 7f0e32ffd700 1 -- 172.21.15.101:0/1547268060 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e38018680 con 0x7f0e3c0ffae0 2023-07-18T03:16:14.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.235+0000 7f0e43bff700 1 -- 172.21.15.101:0/1547268060 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e3c1a29d0 con 0x7f0e3c0ffae0 2023-07-18T03:16:14.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.238+0000 7f0e32ffd700 1 -- 172.21.15.101:0/1547268060 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f0e3800cd40 con 0x7f0e3c0ffae0 2023-07-18T03:16:14.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.239+0000 7f0e5753ab80 1 Processor -- start 2023-07-18T03:16:14.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.239+0000 7f0e5753ab80 1 -- start start 2023-07-18T03:16:14.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.239+0000 7f0e32ffd700 1 --2- 172.21.15.101:0/1547268060 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0e280a12c0 0x7f0e280a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.239+0000 7f0e32ffd700 1 -- 172.21.15.101:0/1547268060 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f0e38014070 con 0x7f0e3c0ffae0 2023-07-18T03:16:14.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.239+0000 7f0e5753ab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561dccf2e2d0 0x561dccf2e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.239+0000 7f0e5753ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dccf2ebe0 0x561dccf39590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.239+0000 7f0e5753ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561dccf39ad0 0x561dccf3beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.239+0000 7f0e5753ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dcce34cb0 con 0x561dccf39ad0 2023-07-18T03:16:14.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.239+0000 7f0e5753ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dcce87a40 con 0x561dccf2ebe0 2023-07-18T03:16:14.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.239+0000 7f0e5753ab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dcce80ed0 con 0x561dccf2e2d0 2023-07-18T03:16:14.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.239+0000 7f0e4219c700 1 --2- 172.21.15.101:0/1547268060 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0e280a12c0 0x7f0e280a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.240+0000 7f0e4199b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561dccf2e2d0 0x561dccf2e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.240+0000 7f0e4119a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dccf2ebe0 0x561dccf39590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.240+0000 7f0e4199b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561dccf2e2d0 0x561dccf2e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:59436/0 (socket says 172.21.15.101:59436) 2023-07-18T03:16:14.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.240+0000 7f0e4199b700 1 -- 172.21.15.101:0/1076449757 learned_addr learned my addr 172.21.15.101:0/1076449757 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:14.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.240+0000 7f0e4219c700 1 --2- 172.21.15.101:0/1076449757 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561dccf39ad0 0x561dccf3beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.240+0000 7f0e4199b700 1 -- 172.21.15.101:0/1076449757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dccf2ebe0 msgr2=0x561dccf39590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:14.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.240+0000 7f0e4199b700 1 --2- 172.21.15.101:0/1076449757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dccf2ebe0 0x561dccf39590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.240+0000 7f0e4199b700 1 -- 172.21.15.101:0/1076449757 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561dccf39ad0 msgr2=0x561dccf3beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:14.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.240+0000 7f0e4199b700 1 --2- 172.21.15.101:0/1076449757 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561dccf39ad0 0x561dccf3beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.240+0000 7f0e4199b700 1 -- 172.21.15.101:0/1076449757 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561dcccaede0 con 0x561dccf2e2d0 2023-07-18T03:16:14.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.240+0000 7f0e4219c700 1 --2- 172.21.15.101:0/1547268060 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0e280a12c0 0x7f0e280a3770 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c19c6e0 tx=0x7f0e34004320 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:14.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.240+0000 7f0e4199b700 1 --2- 172.21.15.101:0/1076449757 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561dccf2e2d0 0x561dccf2e6a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0e38096a50 tx=0x7f0e38099fe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15f96429785888b0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:14.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.241+0000 7f0e27fff700 1 -- 172.21.15.101:0/1076449757 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e380cd160 con 0x561dccf2e2d0 2023-07-18T03:16:14.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.241+0000 7f0e27fff700 1 -- 172.21.15.101:0/1076449757 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e380a2430 con 0x561dccf2e2d0 2023-07-18T03:16:14.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.241+0000 7f0e27fff700 1 -- 172.21.15.101:0/1076449757 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e380a2200 con 0x561dccf2e2d0 2023-07-18T03:16:14.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.241+0000 7f0e5753ab80 1 -- 172.21.15.101:0/1076449757 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561dccf2e2d0 msgr2=0x561dccf2e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:14.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.241+0000 7f0e5753ab80 1 --2- 172.21.15.101:0/1076449757 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561dccf2e2d0 0x561dccf2e6a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0e38096a50 tx=0x7f0e38099fe0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.241+0000 7f0e5753ab80 1 -- 172.21.15.101:0/1076449757 shutdown_connections 2023-07-18T03:16:14.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.241+0000 7f0e5753ab80 1 --2- 172.21.15.101:0/1076449757 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561dccf39ad0 0x561dccf3beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.241+0000 7f0e5753ab80 1 --2- 172.21.15.101:0/1076449757 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561dccf2e2d0 0x561dccf2e6a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.241+0000 7f0e5753ab80 1 --2- 172.21.15.101:0/1076449757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dccf2ebe0 0x561dccf39590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.241+0000 7f0e5753ab80 1 -- 172.21.15.101:0/1076449757 >> 172.21.15.101:0/1076449757 conn(0x561dccf24da0 msgr2=0x561dccf2a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:14.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.241+0000 7f0e5753ab80 1 -- 172.21.15.101:0/1076449757 shutdown_connections 2023-07-18T03:16:14.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.242+0000 7f0e5753ab80 1 -- 172.21.15.101:0/1076449757 wait complete. 2023-07-18T03:16:14.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.242+0000 7f0e5753ab80 1 -- start start 2023-07-18T03:16:14.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.242+0000 7f0e5753ab80 1 Processor -- start 2023-07-18T03:16:14.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.243+0000 7f0e5753ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561dccf2e2d0 0x561dccf36840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.243+0000 7f0e5753ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dccf2ebe0 0x561dccf36d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.243+0000 7f0e5753ab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561dccf39ad0 0x561dccf372c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.243+0000 7f0e5753ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dcce20ac0 con 0x561dccf2e2d0 2023-07-18T03:16:14.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.243+0000 7f0e5753ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dcce207c0 con 0x561dccf2ebe0 2023-07-18T03:16:14.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.243+0000 7f0e5753ab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dccd04920 con 0x561dccf39ad0 2023-07-18T03:16:14.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.243+0000 7f0e5753ab80 10 client.0 _ll_register_callbacks cb 0x561dcce1e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:16:14.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.243+0000 7f0e4199b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561dccf2e2d0 0x561dccf36840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.243+0000 7f0e4199b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561dccf2e2d0 0x561dccf36840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:56620/0 (socket says 172.21.15.101:56620) 2023-07-18T03:16:14.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.243+0000 7f0e4199b700 1 -- 172.21.15.101:0/4050746251 learned_addr learned my addr 172.21.15.101:0/4050746251 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:14.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.243+0000 7f0e4219c700 1 --2- 172.21.15.101:0/4050746251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561dccf39ad0 0x561dccf372c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.244+0000 7f0e4119a700 1 --2- 172.21.15.101:0/4050746251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dccf2ebe0 0x561dccf36d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.244+0000 7f0e4199b700 1 -- 172.21.15.101:0/4050746251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561dccf39ad0 msgr2=0x561dccf372c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:14.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.244+0000 7f0e4199b700 1 --2- 172.21.15.101:0/4050746251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561dccf39ad0 0x561dccf372c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.244+0000 7f0e4199b700 1 -- 172.21.15.101:0/4050746251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dccf2ebe0 msgr2=0x561dccf36d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:14.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.244+0000 7f0e4199b700 1 --2- 172.21.15.101:0/4050746251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dccf2ebe0 0x561dccf36d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.244+0000 7f0e4199b700 1 -- 172.21.15.101:0/4050746251 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e3c1a29d0 con 0x561dccf2e2d0 2023-07-18T03:16:14.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.244+0000 7f0e4219c700 1 --2- 172.21.15.101:0/4050746251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561dccf39ad0 0x561dccf372c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:16:14.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.244+0000 7f0e4119a700 1 --2- 172.21.15.101:0/4050746251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dccf2ebe0 0x561dccf36d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:16:14.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.244+0000 7f0e4199b700 1 --2- 172.21.15.101:0/4050746251 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561dccf2e2d0 0x561dccf36840 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f0e380cabe0 tx=0x7f0e38099780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fedbff523f5e8ef server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:14.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.244+0000 7f0e257fa700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:16:14.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.244+0000 7f0e257fa700 1 -- 172.21.15.101:0/4050746251 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e380cd160 con 0x561dccf2e2d0 2023-07-18T03:16:14.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.244+0000 7f0e257fa700 1 -- 172.21.15.101:0/4050746251 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f0e380a21b0 con 0x561dccf2e2d0 2023-07-18T03:16:14.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.244+0000 7f0e5753ab80 1 -- 172.21.15.101:0/4050746251 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561dccdda4f0 con 0x561dccf2e2d0 2023-07-18T03:16:14.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.245+0000 7f0e257fa700 1 -- 172.21.15.101:0/4050746251 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e380b8270 con 0x561dccf2e2d0 2023-07-18T03:16:14.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.245+0000 7f0e5753ab80 10 client.17268 fetch_fsmap learned FSMap version 31 2023-07-18T03:16:14.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.245+0000 7f0e5753ab80 1 -- 172.21.15.101:0/4050746251 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x561dccc97c60 con 0x561dccf2e2d0 2023-07-18T03:16:14.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.245+0000 7f0e257fa700 1 -- 172.21.15.101:0/4050746251 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e38063020 con 0x561dccf2e2d0 2023-07-18T03:16:14.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.245+0000 7f0e257fa700 1 -- 172.21.15.101:0/4050746251 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f0e38098fd0 con 0x561dccf2e2d0 2023-07-18T03:16:14.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.245+0000 7f0e5753ab80 10 client.17268 fetch_fsmap finished waiting for FSMap version 31 2023-07-18T03:16:14.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.245+0000 7f0e5753ab80 10 client.17268 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:16:14.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.245+0000 7f0e5753ab80 20 client.17268 populate_metadata read hostname 'smithi101' 2023-07-18T03:16:14.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.245+0000 7f0e5753ab80 1 --2- 172.21.15.101:0/4050746251 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x561dccf490c0 0x561dccf4b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.245+0000 7f0e5753ab80 4 client.17268 mds_command: new command op to 24541 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:16:14.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.245+0000 7f0e5753ab80 1 -- 172.21.15.101:0/4050746251 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561dcce79eb0 con 0x561dccf490c0 2023-07-18T03:16:14.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.245+0000 7f0e4119a700 1 --2- 172.21.15.101:0/4050746251 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x561dccf490c0 0x561dccf4b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.246+0000 7f0e4119a700 1 --2- 172.21.15.101:0/4050746251 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x561dccf490c0 0x561dccf4b4a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=57f620115f8f9a62 server_cookie=4834fa66fb746770 in_seq=0 out_seq=0 2023-07-18T03:16:14.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.246+0000 7f0e257fa700 10 client.17268 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:16:14.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.247+0000 7f0e257fa700 1 -- 172.21.15.101:0/4050746251 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561dcce79eb0 con 0x561dccf490c0 2023-07-18T03:16:14.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.247+0000 7f0e257fa700 10 client.17268 handle_command_reply: tid=0 2023-07-18T03:16:14.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.247+0000 7f0e5753ab80 1 client.17268 shutdown 2023-07-18T03:16:14.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.247+0000 7f0e5753ab80 1 -- 172.21.15.101:0/4050746251 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561dccf2e2d0 msgr2=0x561dccf36840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:14.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.247+0000 7f0e5753ab80 1 --2- 172.21.15.101:0/4050746251 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561dccf2e2d0 0x561dccf36840 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f0e380cabe0 tx=0x7f0e38099780 comp rx=0 tx=0).stop 2023-07-18T03:16:14.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.247+0000 7f0e5753ab80 1 -- 172.21.15.101:0/4050746251 shutdown_connections 2023-07-18T03:16:14.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.247+0000 7f0e5753ab80 1 --2- 172.21.15.101:0/4050746251 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x561dccf490c0 0x561dccf4b4a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.247+0000 7f0e5753ab80 1 --2- 172.21.15.101:0/4050746251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561dccf39ad0 0x561dccf372c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.247+0000 7f0e5753ab80 1 --2- 172.21.15.101:0/4050746251 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561dccf2e2d0 0x561dccf36840 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.247+0000 7f0e5753ab80 1 --2- 172.21.15.101:0/4050746251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dccf2ebe0 0x561dccf36d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.247+0000 7f0e5753ab80 1 -- 172.21.15.101:0/4050746251 >> 172.21.15.101:0/4050746251 conn(0x561dccf24da0 msgr2=0x561dccf2a230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:14.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.247+0000 7f0e257fa700 0 client.17268 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:16:14.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.247+0000 7f0e5753ab80 1 -- 172.21.15.101:0/4050746251 shutdown_connections 2023-07-18T03:16:14.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.247+0000 7f0e5753ab80 1 -- 172.21.15.101:0/4050746251 wait complete. 2023-07-18T03:16:14.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.248+0000 7f0e5753ab80 20 client.17268 trim_cache size 0 max 16384 2023-07-18T03:16:14.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.261+0000 7f0e5753ab80 1 Processor -- start 2023-07-18T03:16:14.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.261+0000 7f0e5753ab80 1 -- start start 2023-07-18T03:16:14.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.261+0000 7f0e5753ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dccf2e2d0 0x561dccf404e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.261+0000 7f0e5753ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561dccf2ebe0 0x561dccf370f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.261+0000 7f0e5753ab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561dccf39ad0 0x561dccf33ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.261+0000 7f0e5753ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dccf2f160 con 0x561dccf2ebe0 2023-07-18T03:16:14.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.261+0000 7f0e5753ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dccf26ec0 con 0x561dccf2e2d0 2023-07-18T03:16:14.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.261+0000 7f0e5753ab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dccf31680 con 0x561dccf39ad0 2023-07-18T03:16:14.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.261+0000 7f0e4119a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561dccf2ebe0 0x561dccf370f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.261+0000 7f0e4199b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dccf2e2d0 0x561dccf404e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.262+0000 7f0e4219c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561dccf39ad0 0x561dccf33ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.262+0000 7f0e4199b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dccf2e2d0 0x561dccf404e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:52388/0 (socket says 172.21.15.101:52388) 2023-07-18T03:16:14.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.262+0000 7f0e4199b700 1 -- 172.21.15.101:0/1826138425 learned_addr learned my addr 172.21.15.101:0/1826138425 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:14.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.262+0000 7f0e4119a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561dccf2ebe0 0x561dccf370f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:56634/0 (socket says 172.21.15.101:56634) 2023-07-18T03:16:14.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.262+0000 7f0e4199b700 1 -- 172.21.15.101:0/1826138425 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561dccf39ad0 msgr2=0x561dccf33ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:14.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.262+0000 7f0e4199b700 1 --2- 172.21.15.101:0/1826138425 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561dccf39ad0 0x561dccf33ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.262+0000 7f0e4199b700 1 -- 172.21.15.101:0/1826138425 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561dccf2ebe0 msgr2=0x561dccf370f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:14.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.262+0000 7f0e4199b700 1 --2- 172.21.15.101:0/1826138425 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561dccf2ebe0 0x561dccf370f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.262+0000 7f0e4199b700 1 -- 172.21.15.101:0/1826138425 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561dccf265e0 con 0x561dccf2e2d0 2023-07-18T03:16:14.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.262+0000 7f0e4119a700 1 --2- 172.21.15.101:0/1826138425 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561dccf2ebe0 0x561dccf370f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:16:14.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.262+0000 7f0e4199b700 1 --2- 172.21.15.101:0/1826138425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dccf2e2d0 0x561dccf404e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0e38096a50 tx=0x7f0e38005b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=630a058941bc626f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:14.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.262+0000 7f0e30bff700 1 -- 172.21.15.101:0/1826138425 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e380cd160 con 0x561dccf2e2d0 2023-07-18T03:16:14.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.262+0000 7f0e30bff700 1 -- 172.21.15.101:0/1826138425 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e38063070 con 0x561dccf2e2d0 2023-07-18T03:16:14.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.262+0000 7f0e30bff700 1 -- 172.21.15.101:0/1826138425 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e380ca120 con 0x561dccf2e2d0 2023-07-18T03:16:14.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.263+0000 7f0e5753ab80 1 -- 172.21.15.101:0/1826138425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dccf2e2d0 msgr2=0x561dccf404e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:14.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.263+0000 7f0e5753ab80 1 --2- 172.21.15.101:0/1826138425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dccf2e2d0 0x561dccf404e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0e38096a50 tx=0x7f0e38005b40 comp rx=0 tx=0).stop 2023-07-18T03:16:14.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.263+0000 7f0e5753ab80 1 -- 172.21.15.101:0/1826138425 shutdown_connections 2023-07-18T03:16:14.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.263+0000 7f0e5753ab80 1 --2- 172.21.15.101:0/1826138425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dccf2e2d0 0x561dccf404e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.263+0000 7f0e5753ab80 1 --2- 172.21.15.101:0/1826138425 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561dccf39ad0 0x561dccf33ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.263+0000 7f0e5753ab80 1 --2- 172.21.15.101:0/1826138425 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561dccf2ebe0 0x561dccf370f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.263+0000 7f0e5753ab80 1 -- 172.21.15.101:0/1826138425 >> 172.21.15.101:0/1826138425 conn(0x561dccf24da0 msgr2=0x561dccf31b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:14.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.263+0000 7f0e5753ab80 1 -- 172.21.15.101:0/1826138425 shutdown_connections 2023-07-18T03:16:14.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.263+0000 7f0e5753ab80 1 -- 172.21.15.101:0/1826138425 wait complete. 2023-07-18T03:16:14.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.263+0000 7f0e5753ab80 1 -- start start 2023-07-18T03:16:14.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.264+0000 7f0e5753ab80 1 Processor -- start 2023-07-18T03:16:14.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.264+0000 7f0e5753ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dccf2e2d0 0x561dccf3dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.264+0000 7f0e5753ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561dccf2ebe0 0x561dccf3e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.264+0000 7f0e5753ab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561dccf39ad0 0x561dccf3e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.264+0000 7f0e5753ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dccf9bf80 con 0x561dccf2ebe0 2023-07-18T03:16:14.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.264+0000 7f0e5753ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dccf9c0f0 con 0x561dccf2e2d0 2023-07-18T03:16:14.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.264+0000 7f0e5753ab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dccf9c2b0 con 0x561dccf39ad0 2023-07-18T03:16:14.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.264+0000 7f0e5753ab80 10 client.0 _ll_register_callbacks cb 0x7f0e2c009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:16:14.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.264+0000 7f0e4119a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561dccf2ebe0 0x561dccf3e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.264+0000 7f0e4199b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dccf2e2d0 0x561dccf3dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.264+0000 7f0e4119a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561dccf2ebe0 0x561dccf3e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:56644/0 (socket says 172.21.15.101:56644) 2023-07-18T03:16:14.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.264+0000 7f0e4119a700 1 -- 172.21.15.101:0/1038619468 learned_addr learned my addr 172.21.15.101:0/1038619468 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:14.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.265+0000 7f0e4219c700 1 --2- 172.21.15.101:0/1038619468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561dccf39ad0 0x561dccf3e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.265+0000 7f0e4119a700 1 -- 172.21.15.101:0/1038619468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561dccf39ad0 msgr2=0x561dccf3e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:14.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.265+0000 7f0e4119a700 1 --2- 172.21.15.101:0/1038619468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561dccf39ad0 0x561dccf3e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.265+0000 7f0e4119a700 1 -- 172.21.15.101:0/1038619468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dccf2e2d0 msgr2=0x561dccf3dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:14.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.265+0000 7f0e4119a700 1 --2- 172.21.15.101:0/1038619468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dccf2e2d0 0x561dccf3dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.265+0000 7f0e4119a700 1 -- 172.21.15.101:0/1038619468 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e3c10d220 con 0x561dccf2ebe0 2023-07-18T03:16:14.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.265+0000 7f0e4119a700 1 --2- 172.21.15.101:0/1038619468 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561dccf2ebe0 0x561dccf3e220 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c009990 tx=0x7f0e2c005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f090f4dc3737247d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:14.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.265+0000 7f0e257fa700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:16:14.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.265+0000 7f0e257fa700 1 -- 172.21.15.101:0/1038619468 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c02b070 con 0x561dccf2ebe0 2023-07-18T03:16:14.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.265+0000 7f0e257fa700 1 -- 172.21.15.101:0/1038619468 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f0e2c00f670 con 0x561dccf2ebe0 2023-07-18T03:16:14.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.265+0000 7f0e5753ab80 1 -- 172.21.15.101:0/1038619468 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0e380a2430 con 0x561dccf2ebe0 2023-07-18T03:16:14.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.266+0000 7f0e257fa700 1 -- 172.21.15.101:0/1038619468 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e2c00bec0 con 0x561dccf2ebe0 2023-07-18T03:16:14.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.266+0000 7f0e5753ab80 10 client.17274 fetch_fsmap learned FSMap version 31 2023-07-18T03:16:14.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.266+0000 7f0e5753ab80 1 -- 172.21.15.101:0/1038619468 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x561dccc97c60 con 0x561dccf2ebe0 2023-07-18T03:16:14.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.266+0000 7f0e257fa700 1 -- 172.21.15.101:0/1038619468 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e380a2430 con 0x561dccf2ebe0 2023-07-18T03:16:14.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.266+0000 7f0e257fa700 1 -- 172.21.15.101:0/1038619468 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f0e2c02f930 con 0x561dccf2ebe0 2023-07-18T03:16:14.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.266+0000 7f0e5753ab80 10 client.17274 fetch_fsmap finished waiting for FSMap version 31 2023-07-18T03:16:14.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.266+0000 7f0e5753ab80 10 client.17274 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:16:14.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.266+0000 7f0e5753ab80 20 client.17274 populate_metadata read hostname 'smithi101' 2023-07-18T03:16:14.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.266+0000 7f0e5753ab80 1 --2- 172.21.15.101:0/1038619468 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x561dccf9da50 0x561dccf9fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.266+0000 7f0e5753ab80 4 client.17274 mds_command: new command op to 24541 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:16:14.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.266+0000 7f0e5753ab80 1 -- 172.21.15.101:0/1038619468 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561dcce3c050 con 0x561dccf9da50 2023-07-18T03:16:14.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.267+0000 7f0e4199b700 1 --2- 172.21.15.101:0/1038619468 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x561dccf9da50 0x561dccf9fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.267+0000 7f0e257fa700 10 client.17274 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:16:14.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.267+0000 7f0e4199b700 1 --2- 172.21.15.101:0/1038619468 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x561dccf9da50 0x561dccf9fe60 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=21819ef183ce2bfa server_cookie=e4c52808847da8c1 in_seq=0 out_seq=0 2023-07-18T03:16:14.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.267+0000 7f0e257fa700 1 -- 172.21.15.101:0/1038619468 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561dcce3c050 con 0x561dccf9da50 2023-07-18T03:16:14.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.267+0000 7f0e257fa700 10 client.17274 handle_command_reply: tid=0 2023-07-18T03:16:14.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.267+0000 7f0e5753ab80 1 client.17274 shutdown 2023-07-18T03:16:14.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.267+0000 7f0e5753ab80 1 -- 172.21.15.101:0/1038619468 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561dccf2ebe0 msgr2=0x561dccf3e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:14.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.267+0000 7f0e5753ab80 1 --2- 172.21.15.101:0/1038619468 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561dccf2ebe0 0x561dccf3e220 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c009990 tx=0x7f0e2c005ab0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.267+0000 7f0e5753ab80 1 -- 172.21.15.101:0/1038619468 shutdown_connections 2023-07-18T03:16:14.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.267+0000 7f0e5753ab80 1 --2- 172.21.15.101:0/1038619468 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x561dccf9da50 0x561dccf9fe60 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.267+0000 7f0e5753ab80 1 --2- 172.21.15.101:0/1038619468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561dccf2e2d0 0x561dccf3dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.267+0000 7f0e257fa700 0 client.17274 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:16:14.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.267+0000 7f0e5753ab80 1 --2- 172.21.15.101:0/1038619468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561dccf39ad0 0x561dccf3e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.267+0000 7f0e5753ab80 1 --2- 172.21.15.101:0/1038619468 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561dccf2ebe0 0x561dccf3e220 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.267+0000 7f0e5753ab80 1 -- 172.21.15.101:0/1038619468 >> 172.21.15.101:0/1038619468 conn(0x561dccf24da0 msgr2=0x561dccf34ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:14.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.268+0000 7f0e5753ab80 1 -- 172.21.15.101:0/1038619468 shutdown_connections 2023-07-18T03:16:14.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.268+0000 7f0e5753ab80 1 -- 172.21.15.101:0/1038619468 wait complete. 2023-07-18T03:16:14.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.268+0000 7f0e5753ab80 20 client.17274 trim_cache size 0 max 16384 2023-07-18T03:16:14.272 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:16:14.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.268+0000 7f0e43bff700 1 -- 172.21.15.101:0/1547268060 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0e280a12c0 msgr2=0x7f0e280a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:14.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.268+0000 7f0e43bff700 1 --2- 172.21.15.101:0/1547268060 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0e280a12c0 0x7f0e280a3770 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c19c6e0 tx=0x7f0e34004320 comp rx=0 tx=0).stop 2023-07-18T03:16:14.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.268+0000 7f0e43bff700 1 -- 172.21.15.101:0/1547268060 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0e3c0ffae0 msgr2=0x7f0e3c19b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:14.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.268+0000 7f0e43bff700 1 --2- 172.21.15.101:0/1547268060 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0e3c0ffae0 0x7f0e3c19b9e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f0e3800e9a0 tx=0x7f0e3800ecb0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.268+0000 7f0e43bff700 1 -- 172.21.15.101:0/1547268060 shutdown_connections 2023-07-18T03:16:14.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.268+0000 7f0e43bff700 1 --2- 172.21.15.101:0/1547268060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e3c105af0 0x7f0e3c195a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.268+0000 7f0e43bff700 1 --2- 172.21.15.101:0/1547268060 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0e3c0ffae0 0x7f0e3c19b9e0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.268+0000 7f0e43bff700 1 --2- 172.21.15.101:0/1547268060 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0e280a12c0 0x7f0e280a3770 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.268+0000 7f0e43bff700 1 --2- 172.21.15.101:0/1547268060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e3c100490 0x7f0e3c19bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.268+0000 7f0e43bff700 1 -- 172.21.15.101:0/1547268060 >> 172.21.15.101:0/1547268060 conn(0x7f0e3c0fb5e0 msgr2=0x7f0e3c0fbc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:14.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.268+0000 7f0e43bff700 1 -- 172.21.15.101:0/1547268060 shutdown_connections 2023-07-18T03:16:14.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.268+0000 7f0e43bff700 1 -- 172.21.15.101:0/1547268060 wait complete. 2023-07-18T03:16:14.280 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-18T03:16:14.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.500+0000 7effa81ff700 1 -- 172.21.15.101:0/3483525498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effa0101610 msgr2=0x7effa0109b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:14.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.500+0000 7effa81ff700 1 --2- 172.21.15.101:0/3483525498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effa0101610 0x7effa0109b00 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7eff90009990 tx=0x7eff90009ca0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.501+0000 7effa81ff700 1 -- 172.21.15.101:0/3483525498 shutdown_connections 2023-07-18T03:16:14.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.501+0000 7effa81ff700 1 --2- 172.21.15.101:0/3483525498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7effa010a040 0x7effa010c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.501+0000 7effa81ff700 1 --2- 172.21.15.101:0/3483525498 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7effa0100d00 0x7effa01010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.501+0000 7effa81ff700 1 --2- 172.21.15.101:0/3483525498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effa0101610 0x7effa0109b00 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.501+0000 7effa81ff700 1 -- 172.21.15.101:0/3483525498 >> 172.21.15.101:0/3483525498 conn(0x7effa00fa510 msgr2=0x7effa00fc920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:14.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.501+0000 7effa81ff700 1 -- 172.21.15.101:0/3483525498 shutdown_connections 2023-07-18T03:16:14.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.501+0000 7effa81ff700 1 -- 172.21.15.101:0/3483525498 wait complete. 2023-07-18T03:16:14.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.502+0000 7effa81ff700 1 Processor -- start 2023-07-18T03:16:14.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.502+0000 7effa81ff700 1 -- start start 2023-07-18T03:16:14.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.503+0000 7effa81ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7effa0100d00 0x7effa00fe950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.503+0000 7effa81ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effa0101610 0x7effa00fee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.503+0000 7effa81ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7effa010a040 0x7effa01a3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.503+0000 7effa81ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa00ff6c0 con 0x7effa010a040 2023-07-18T03:16:14.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.503+0000 7effa81ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa01a4300 con 0x7effa0101610 2023-07-18T03:16:14.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.503+0000 7effa81ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa01a4440 con 0x7effa0100d00 2023-07-18T03:16:14.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.503+0000 7effa5f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7effa0100d00 0x7effa00fe950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.503+0000 7effa579a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effa0101610 0x7effa00fee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.503+0000 7effa5f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7effa0100d00 0x7effa00fe950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:59482/0 (socket says 172.21.15.101:59482) 2023-07-18T03:16:14.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.503+0000 7effa5f9b700 1 -- 172.21.15.101:0/3578397 learned_addr learned my addr 172.21.15.101:0/3578397 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:14.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.503+0000 7effa679c700 1 --2- 172.21.15.101:0/3578397 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7effa010a040 0x7effa01a3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.504+0000 7effa579a700 1 -- 172.21.15.101:0/3578397 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7effa0100d00 msgr2=0x7effa00fe950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:14.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.504+0000 7effa579a700 1 --2- 172.21.15.101:0/3578397 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7effa0100d00 0x7effa00fe950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.504+0000 7effa579a700 1 -- 172.21.15.101:0/3578397 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7effa010a040 msgr2=0x7effa01a3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:14.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.504+0000 7effa579a700 1 --2- 172.21.15.101:0/3578397 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7effa010a040 0x7effa01a3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.504+0000 7effa579a700 1 -- 172.21.15.101:0/3578397 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff900095f0 con 0x7effa0101610 2023-07-18T03:16:14.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.504+0000 7effa5f9b700 1 --2- 172.21.15.101:0/3578397 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7effa0100d00 0x7effa00fe950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:16:14.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.504+0000 7effa579a700 1 --2- 172.21.15.101:0/3578397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effa0101610 0x7effa00fee90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7eff90009990 tx=0x7eff90009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:14.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.504+0000 7eff96ffd700 1 -- 172.21.15.101:0/3578397 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9001d070 con 0x7effa0101610 2023-07-18T03:16:14.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.504+0000 7effa81ff700 1 -- 172.21.15.101:0/3578397 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effa01a4640 con 0x7effa0101610 2023-07-18T03:16:14.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.504+0000 7eff96ffd700 1 -- 172.21.15.101:0/3578397 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff90021410 con 0x7effa0101610 2023-07-18T03:16:14.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.504+0000 7eff96ffd700 1 -- 172.21.15.101:0/3578397 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff90017650 con 0x7effa0101610 2023-07-18T03:16:14.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.505+0000 7effa81ff700 1 -- 172.21.15.101:0/3578397 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effa01a4b10 con 0x7effa0101610 2023-07-18T03:16:14.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.508+0000 7eff96ffd700 1 -- 172.21.15.101:0/3578397 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7eff90021810 con 0x7effa0101610 2023-07-18T03:16:14.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.509+0000 7effbba7cb80 1 Processor -- start 2023-07-18T03:16:14.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.509+0000 7effbba7cb80 1 -- start start 2023-07-18T03:16:14.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.510+0000 7effbba7cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55581492a2d0 0x55581492a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.510+0000 7effbba7cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55581492abe0 0x555814935590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.510+0000 7effbba7cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555814935ad0 0x555814937eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.510+0000 7effbba7cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555814830cb0 con 0x55581492abe0 2023-07-18T03:16:14.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.510+0000 7effbba7cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555814883a40 con 0x55581492a2d0 2023-07-18T03:16:14.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.510+0000 7effbba7cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55581487ced0 con 0x555814935ad0 2023-07-18T03:16:14.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.510+0000 7effa679c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55581492a2d0 0x55581492a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.510+0000 7effa5f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55581492abe0 0x555814935590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.510+0000 7effa679c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55581492a2d0 0x55581492a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:52422/0 (socket says 172.21.15.101:52422) 2023-07-18T03:16:14.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.510+0000 7effa679c700 1 -- 172.21.15.101:0/2685019543 learned_addr learned my addr 172.21.15.101:0/2685019543 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:14.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.510+0000 7effa579a700 1 --2- 172.21.15.101:0/2685019543 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555814935ad0 0x555814937eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.510+0000 7eff96ffd700 1 --2- 172.21.15.101:0/3578397 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7eff8c0a12c0 0x7eff8c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.511+0000 7effa5f9b700 1 -- 172.21.15.101:0/2685019543 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555814935ad0 msgr2=0x555814937eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:14.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.511+0000 7effa5f9b700 1 --2- 172.21.15.101:0/2685019543 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555814935ad0 0x555814937eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.511+0000 7effa5f9b700 1 -- 172.21.15.101:0/2685019543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55581492a2d0 msgr2=0x55581492a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:14.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.511+0000 7effa5f9b700 1 --2- 172.21.15.101:0/2685019543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55581492a2d0 0x55581492a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.511+0000 7eff96ffd700 1 -- 172.21.15.101:0/3578397 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9461+0+0 (secure 0 0 0) 0x7eff900ce3d0 con 0x7effa0101610 2023-07-18T03:16:14.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.511+0000 7effa5f9b700 1 -- 172.21.15.101:0/2685019543 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558146aade0 con 0x55581492abe0 2023-07-18T03:16:14.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.511+0000 7effa679c700 1 --2- 172.21.15.101:0/2685019543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55581492a2d0 0x55581492a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:16:14.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.511+0000 7effa679c700 1 --2- 172.21.15.101:0/3578397 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7eff8c0a12c0 0x7eff8c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.511+0000 7effa5f9b700 1 --2- 172.21.15.101:0/2685019543 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55581492abe0 0x555814935590 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7eff9c011570 tx=0x7eff9c011880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=612f5d49546efe08 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:14.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.511+0000 7eff8bfff700 1 -- 172.21.15.101:0/2685019543 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c011ea0 con 0x55581492abe0 2023-07-18T03:16:14.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.511+0000 7eff8bfff700 1 -- 172.21.15.101:0/2685019543 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff9c003680 con 0x55581492abe0 2023-07-18T03:16:14.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.511+0000 7eff8bfff700 1 -- 172.21.15.101:0/2685019543 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c0197e0 con 0x55581492abe0 2023-07-18T03:16:14.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.512+0000 7effbba7cb80 1 -- 172.21.15.101:0/2685019543 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55581492abe0 msgr2=0x555814935590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:14.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.512+0000 7effa679c700 1 --2- 172.21.15.101:0/3578397 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7eff8c0a12c0 0x7eff8c0a3770 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7eff98007620 tx=0x7eff98009040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:14.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.512+0000 7effbba7cb80 1 --2- 172.21.15.101:0/2685019543 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55581492abe0 0x555814935590 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7eff9c011570 tx=0x7eff9c011880 comp rx=0 tx=0).stop 2023-07-18T03:16:14.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.512+0000 7effbba7cb80 1 -- 172.21.15.101:0/2685019543 shutdown_connections 2023-07-18T03:16:14.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.512+0000 7effbba7cb80 1 --2- 172.21.15.101:0/2685019543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55581492a2d0 0x55581492a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.512+0000 7effbba7cb80 1 --2- 172.21.15.101:0/2685019543 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555814935ad0 0x555814937eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.512+0000 7effbba7cb80 1 --2- 172.21.15.101:0/2685019543 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55581492abe0 0x555814935590 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.512+0000 7effbba7cb80 1 -- 172.21.15.101:0/2685019543 >> 172.21.15.101:0/2685019543 conn(0x555814920da0 msgr2=0x5558149266a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:14.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.512+0000 7effbba7cb80 1 -- 172.21.15.101:0/2685019543 shutdown_connections 2023-07-18T03:16:14.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.512+0000 7effbba7cb80 1 -- 172.21.15.101:0/2685019543 wait complete. 2023-07-18T03:16:14.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.513+0000 7effbba7cb80 1 -- start start 2023-07-18T03:16:14.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.513+0000 7effbba7cb80 1 Processor -- start 2023-07-18T03:16:14.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.513+0000 7effbba7cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55581492a2d0 0x5558149354a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.513+0000 7effbba7cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55581492abe0 0x555814932720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.514+0000 7effbba7cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555814935ad0 0x555814932d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.514+0000 7effbba7cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55581481cac0 con 0x55581492a2d0 2023-07-18T03:16:14.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.514+0000 7effbba7cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55581481c7c0 con 0x555814935ad0 2023-07-18T03:16:14.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.514+0000 7effbba7cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555814700920 con 0x55581492abe0 2023-07-18T03:16:14.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.514+0000 7effbba7cb80 10 client.0 _ll_register_callbacks cb 0x55581481a760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:16:14.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.514+0000 7effa5f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55581492a2d0 0x5558149354a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.514+0000 7effa5f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55581492a2d0 0x5558149354a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:56688/0 (socket says 172.21.15.101:56688) 2023-07-18T03:16:14.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.514+0000 7effa5f9b700 1 -- 172.21.15.101:0/3434338792 learned_addr learned my addr 172.21.15.101:0/3434338792 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:14.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.514+0000 7effa579a700 1 --2- 172.21.15.101:0/3434338792 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55581492abe0 0x555814932720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.514+0000 7effa679c700 1 --2- 172.21.15.101:0/3434338792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555814935ad0 0x555814932d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.514+0000 7effa5f9b700 1 -- 172.21.15.101:0/3434338792 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55581492abe0 msgr2=0x555814932720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:14.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.514+0000 7effa5f9b700 1 --2- 172.21.15.101:0/3434338792 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55581492abe0 0x555814932720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.514+0000 7effa5f9b700 1 -- 172.21.15.101:0/3434338792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555814935ad0 msgr2=0x555814932d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:14.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.514+0000 7effa5f9b700 1 --2- 172.21.15.101:0/3434338792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555814935ad0 0x555814932d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.514+0000 7effa5f9b700 1 -- 172.21.15.101:0/3434338792 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff98011300 con 0x55581492a2d0 2023-07-18T03:16:14.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.514+0000 7effa5f9b700 1 --2- 172.21.15.101:0/3434338792 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55581492a2d0 0x5558149354a0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7eff9c011540 tx=0x7eff9c004300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac97165f344b5dc server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:14.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.514+0000 7eff897fa700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:16:14.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.515+0000 7eff897fa700 1 -- 172.21.15.101:0/3434338792 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c005450 con 0x55581492a2d0 2023-07-18T03:16:14.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.515+0000 7eff897fa700 1 -- 172.21.15.101:0/3434338792 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9461+0+0 (secure 0 0 0) 0x7eff9c005ce0 con 0x55581492a2d0 2023-07-18T03:16:14.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.515+0000 7effbba7cb80 1 -- 172.21.15.101:0/3434338792 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5558147d64f0 con 0x55581492a2d0 2023-07-18T03:16:14.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.515+0000 7eff897fa700 1 -- 172.21.15.101:0/3434338792 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff9c0045a0 con 0x55581492a2d0 2023-07-18T03:16:14.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.515+0000 7effbba7cb80 10 client.17286 fetch_fsmap learned FSMap version 31 2023-07-18T03:16:14.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.515+0000 7effbba7cb80 1 -- 172.21.15.101:0/3434338792 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7eff9c009d60 con 0x55581492a2d0 2023-07-18T03:16:14.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.515+0000 7eff897fa700 1 -- 172.21.15.101:0/3434338792 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5558147d64f0 con 0x55581492a2d0 2023-07-18T03:16:14.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.516+0000 7eff897fa700 1 -- 172.21.15.101:0/3434338792 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7eff9c021040 con 0x55581492a2d0 2023-07-18T03:16:14.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.516+0000 7effbba7cb80 10 client.17286 fetch_fsmap finished waiting for FSMap version 31 2023-07-18T03:16:14.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.516+0000 7effbba7cb80 10 client.17286 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:16:14.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.516+0000 7effbba7cb80 20 client.17286 populate_metadata read hostname 'smithi101' 2023-07-18T03:16:14.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.516+0000 7effbba7cb80 1 --2- 172.21.15.101:0/3434338792 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x555814945240 0x555814947650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.516+0000 7effbba7cb80 4 client.17286 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:16:14.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.516+0000 7effbba7cb80 1 -- 172.21.15.101:0/3434338792 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555814875eb0 con 0x555814945240 2023-07-18T03:16:14.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.516+0000 7effa579a700 1 --2- 172.21.15.101:0/3434338792 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x555814945240 0x555814947650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.516+0000 7effa579a700 1 --2- 172.21.15.101:0/3434338792 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x555814945240 0x555814947650 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=75153d9f36ae2ef3 server_cookie=5856ab52e2ba6034 in_seq=0 out_seq=0 2023-07-18T03:16:14.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.516+0000 7eff897fa700 10 client.17286 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:16:14.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.517+0000 7eff897fa700 1 -- 172.21.15.101:0/3434338792 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555814875eb0 con 0x555814945240 2023-07-18T03:16:14.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.517+0000 7eff897fa700 10 client.17286 handle_command_reply: tid=0 2023-07-18T03:16:14.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.517+0000 7effbba7cb80 1 client.17286 shutdown 2023-07-18T03:16:14.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.517+0000 7effbba7cb80 1 -- 172.21.15.101:0/3434338792 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55581492a2d0 msgr2=0x5558149354a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:14.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.517+0000 7effbba7cb80 1 --2- 172.21.15.101:0/3434338792 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55581492a2d0 0x5558149354a0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7eff9c011540 tx=0x7eff9c004300 comp rx=0 tx=0).stop 2023-07-18T03:16:14.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.517+0000 7effbba7cb80 1 -- 172.21.15.101:0/3434338792 shutdown_connections 2023-07-18T03:16:14.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.517+0000 7effbba7cb80 1 --2- 172.21.15.101:0/3434338792 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x555814945240 0x555814947650 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.517+0000 7effbba7cb80 1 --2- 172.21.15.101:0/3434338792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555814935ad0 0x555814932d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.517+0000 7eff897fa700 0 client.17286 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:16:14.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.517+0000 7effbba7cb80 1 --2- 172.21.15.101:0/3434338792 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55581492a2d0 0x5558149354a0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.517+0000 7effbba7cb80 1 --2- 172.21.15.101:0/3434338792 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55581492abe0 0x555814932720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.517+0000 7effbba7cb80 1 -- 172.21.15.101:0/3434338792 >> 172.21.15.101:0/3434338792 conn(0x555814920da0 msgr2=0x555814926230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:14.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.517+0000 7effbba7cb80 1 -- 172.21.15.101:0/3434338792 shutdown_connections 2023-07-18T03:16:14.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.518+0000 7effbba7cb80 1 -- 172.21.15.101:0/3434338792 wait complete. 2023-07-18T03:16:14.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.518+0000 7effbba7cb80 20 client.17286 trim_cache size 0 max 16384 2023-07-18T03:16:14.526 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.532+0000 7effbba7cb80 1 Processor -- start 2023-07-18T03:16:14.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.532+0000 7effbba7cb80 1 -- start start 2023-07-18T03:16:14.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.533+0000 7effbba7cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55581492a2d0 0x555814934fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.533+0000 7effbba7cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55581492abe0 0x555814931fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.533+0000 7effbba7cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555814935ad0 0x555814932500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.533+0000 7effbba7cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555814933330 con 0x555814935ad0 2023-07-18T03:16:14.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.533+0000 7effbba7cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555814932a40 con 0x55581492a2d0 2023-07-18T03:16:14.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.533+0000 7effbba7cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555814931c90 con 0x55581492abe0 2023-07-18T03:16:14.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.533+0000 7effa5f9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55581492a2d0 0x555814934fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.533+0000 7effa679c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555814935ad0 0x555814932500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.533+0000 7effa579a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55581492abe0 0x555814931fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.533+0000 7effa5f9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55581492a2d0 0x555814934fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:52450/0 (socket says 172.21.15.101:52450) 2023-07-18T03:16:14.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.533+0000 7effa5f9b700 1 -- 172.21.15.101:0/4032086722 learned_addr learned my addr 172.21.15.101:0/4032086722 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:14.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.533+0000 7effa5f9b700 1 -- 172.21.15.101:0/4032086722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55581492abe0 msgr2=0x555814931fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:14.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.533+0000 7effa5f9b700 1 --2- 172.21.15.101:0/4032086722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55581492abe0 0x555814931fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.533+0000 7effa5f9b700 1 -- 172.21.15.101:0/4032086722 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555814935ad0 msgr2=0x555814932500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:14.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.533+0000 7effa5f9b700 1 --2- 172.21.15.101:0/4032086722 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555814935ad0 0x555814932500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.533+0000 7effa5f9b700 1 -- 172.21.15.101:0/4032086722 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555814693c60 con 0x55581492a2d0 2023-07-18T03:16:14.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.534+0000 7effa5f9b700 1 --2- 172.21.15.101:0/4032086722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55581492a2d0 0x555814934fd0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7eff9c023510 tx=0x7eff9c0237b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddd614f3ac9e5e92 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:14.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.534+0000 7eff94ff9700 1 -- 172.21.15.101:0/4032086722 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c003c60 con 0x55581492a2d0 2023-07-18T03:16:14.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.534+0000 7eff94ff9700 1 -- 172.21.15.101:0/4032086722 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff9c003dc0 con 0x55581492a2d0 2023-07-18T03:16:14.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.534+0000 7eff94ff9700 1 -- 172.21.15.101:0/4032086722 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c0187a0 con 0x55581492a2d0 2023-07-18T03:16:14.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.534+0000 7effbba7cb80 1 -- 172.21.15.101:0/4032086722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55581492a2d0 msgr2=0x555814934fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:14.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.534+0000 7effbba7cb80 1 --2- 172.21.15.101:0/4032086722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55581492a2d0 0x555814934fd0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7eff9c023510 tx=0x7eff9c0237b0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.535+0000 7effbba7cb80 1 -- 172.21.15.101:0/4032086722 shutdown_connections 2023-07-18T03:16:14.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.535+0000 7effbba7cb80 1 --2- 172.21.15.101:0/4032086722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55581492a2d0 0x555814934fd0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.535+0000 7effbba7cb80 1 --2- 172.21.15.101:0/4032086722 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555814935ad0 0x555814932500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.535+0000 7effbba7cb80 1 --2- 172.21.15.101:0/4032086722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55581492abe0 0x555814931fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.535+0000 7effbba7cb80 1 -- 172.21.15.101:0/4032086722 >> 172.21.15.101:0/4032086722 conn(0x555814920da0 msgr2=0x55581492da50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:14.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.535+0000 7effbba7cb80 1 -- 172.21.15.101:0/4032086722 shutdown_connections 2023-07-18T03:16:14.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.535+0000 7effbba7cb80 1 -- 172.21.15.101:0/4032086722 wait complete. 2023-07-18T03:16:14.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.535+0000 7effbba7cb80 1 -- start start 2023-07-18T03:16:14.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.536+0000 7effbba7cb80 1 Processor -- start 2023-07-18T03:16:14.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.536+0000 7effbba7cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55581492a2d0 0x55581493bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.536+0000 7effbba7cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55581492abe0 0x55581493c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.536+0000 7effbba7cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555814935ad0 0x55581493a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.536+0000 7effbba7cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55581493ab50 con 0x555814935ad0 2023-07-18T03:16:14.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.536+0000 7effbba7cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55581493acc0 con 0x55581492abe0 2023-07-18T03:16:14.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.536+0000 7effbba7cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55581493ae80 con 0x55581492a2d0 2023-07-18T03:16:14.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.536+0000 7effbba7cb80 10 client.0 _ll_register_callbacks cb 0x7eff98009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:16:14.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.536+0000 7effa579a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55581492abe0 0x55581493c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.536+0000 7effa5f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55581492a2d0 0x55581493bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.537+0000 7effa579a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55581492abe0 0x55581493c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:52454/0 (socket says 172.21.15.101:52454) 2023-07-18T03:16:14.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.537+0000 7effa5f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55581492a2d0 0x55581493bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:59524/0 (socket says 172.21.15.101:59524) 2023-07-18T03:16:14.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.537+0000 7effa579a700 1 -- 172.21.15.101:0/3019121738 learned_addr learned my addr 172.21.15.101:0/3019121738 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:14.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.537+0000 7effa679c700 1 --2- 172.21.15.101:0/3019121738 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555814935ad0 0x55581493a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.537+0000 7effa579a700 1 -- 172.21.15.101:0/3019121738 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55581492a2d0 msgr2=0x55581493bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:14.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.537+0000 7effa579a700 1 --2- 172.21.15.101:0/3019121738 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55581492a2d0 0x55581493bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.537+0000 7effa579a700 1 -- 172.21.15.101:0/3019121738 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555814935ad0 msgr2=0x55581493a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:14.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.537+0000 7effa579a700 1 --2- 172.21.15.101:0/3019121738 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555814935ad0 0x55581493a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.537+0000 7effa579a700 1 -- 172.21.15.101:0/3019121738 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff9c009d60 con 0x55581492abe0 2023-07-18T03:16:14.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.537+0000 7effa579a700 1 --2- 172.21.15.101:0/3019121738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55581492abe0 0x55581493c500 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7eff900cb770 tx=0x7eff90097460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c0795027b71b8a0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:14.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.537+0000 7eff897fa700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:16:14.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.537+0000 7eff897fa700 1 -- 172.21.15.101:0/3019121738 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9009aef0 con 0x55581492abe0 2023-07-18T03:16:14.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.537+0000 7eff897fa700 1 -- 172.21.15.101:0/3019121738 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9461+0+0 (secure 0 0 0) 0x7eff9009b120 con 0x55581492abe0 2023-07-18T03:16:14.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.537+0000 7effbba7cb80 1 -- 172.21.15.101:0/3019121738 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff9c003680 con 0x55581492abe0 2023-07-18T03:16:14.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.538+0000 7eff897fa700 1 -- 172.21.15.101:0/3019121738 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff9009f150 con 0x55581492abe0 2023-07-18T03:16:14.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.538+0000 7effbba7cb80 10 client.27329 fetch_fsmap learned FSMap version 31 2023-07-18T03:16:14.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.538+0000 7effbba7cb80 1 -- 172.21.15.101:0/3019121738 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5558146d1250 con 0x55581492abe0 2023-07-18T03:16:14.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.538+0000 7eff897fa700 1 -- 172.21.15.101:0/3019121738 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff9c003680 con 0x55581492abe0 2023-07-18T03:16:14.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.538+0000 7eff897fa700 1 -- 172.21.15.101:0/3019121738 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7eff9009a610 con 0x55581492abe0 2023-07-18T03:16:14.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.538+0000 7effbba7cb80 10 client.27329 fetch_fsmap finished waiting for FSMap version 31 2023-07-18T03:16:14.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.538+0000 7effbba7cb80 10 client.27329 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:16:14.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.538+0000 7effbba7cb80 20 client.27329 populate_metadata read hostname 'smithi101' 2023-07-18T03:16:14.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.538+0000 7effbba7cb80 1 --2- 172.21.15.101:0/3019121738 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x555814999ff0 0x55581499c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.538+0000 7effbba7cb80 4 client.27329 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:16:14.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.538+0000 7effbba7cb80 1 -- 172.21.15.101:0/3019121738 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555814838050 con 0x555814999ff0 2023-07-18T03:16:14.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.538+0000 7effa5f9b700 1 --2- 172.21.15.101:0/3019121738 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x555814999ff0 0x55581499c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.539+0000 7eff897fa700 10 client.27329 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:16:14.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.539+0000 7effa5f9b700 1 --2- 172.21.15.101:0/3019121738 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x555814999ff0 0x55581499c400 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=47852250cd0c6cba server_cookie=b9171abad35e361a in_seq=0 out_seq=0 2023-07-18T03:16:14.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.539+0000 7eff897fa700 1 -- 172.21.15.101:0/3019121738 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555814838050 con 0x555814999ff0 2023-07-18T03:16:14.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.539+0000 7eff897fa700 10 client.27329 handle_command_reply: tid=0 2023-07-18T03:16:14.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.539+0000 7effbba7cb80 1 client.27329 shutdown 2023-07-18T03:16:14.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.540+0000 7effbba7cb80 1 -- 172.21.15.101:0/3019121738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55581492abe0 msgr2=0x55581493c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:14.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.540+0000 7effbba7cb80 1 --2- 172.21.15.101:0/3019121738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55581492abe0 0x55581493c500 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7eff900cb770 tx=0x7eff90097460 comp rx=0 tx=0).stop 2023-07-18T03:16:14.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.540+0000 7effbba7cb80 1 -- 172.21.15.101:0/3019121738 shutdown_connections 2023-07-18T03:16:14.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.540+0000 7effbba7cb80 1 --2- 172.21.15.101:0/3019121738 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x555814999ff0 0x55581499c400 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.540+0000 7effbba7cb80 1 --2- 172.21.15.101:0/3019121738 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555814935ad0 0x55581493a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.540+0000 7effbba7cb80 1 --2- 172.21.15.101:0/3019121738 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55581492a2d0 0x55581493bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.540+0000 7eff897fa700 0 client.27329 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:16:14.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.540+0000 7effbba7cb80 1 --2- 172.21.15.101:0/3019121738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55581492abe0 0x55581493c500 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.540+0000 7effbba7cb80 1 -- 172.21.15.101:0/3019121738 >> 172.21.15.101:0/3019121738 conn(0x555814920da0 msgr2=0x555814931f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:14.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.540+0000 7effbba7cb80 1 -- 172.21.15.101:0/3019121738 shutdown_connections 2023-07-18T03:16:14.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.540+0000 7effbba7cb80 1 -- 172.21.15.101:0/3019121738 wait complete. 2023-07-18T03:16:14.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.540+0000 7effbba7cb80 20 client.27329 trim_cache size 0 max 16384 2023-07-18T03:16:14.542 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:16:14.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.541+0000 7effa81ff700 1 -- 172.21.15.101:0/3578397 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7eff8c0a12c0 msgr2=0x7eff8c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:14.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.541+0000 7effa81ff700 1 --2- 172.21.15.101:0/3578397 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7eff8c0a12c0 0x7eff8c0a3770 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7eff98007620 tx=0x7eff98009040 comp rx=0 tx=0).stop 2023-07-18T03:16:14.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.541+0000 7effa81ff700 1 -- 172.21.15.101:0/3578397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effa0101610 msgr2=0x7effa00fee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:14.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.541+0000 7effa81ff700 1 --2- 172.21.15.101:0/3578397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effa0101610 0x7effa00fee90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7eff90009990 tx=0x7eff90009580 comp rx=0 tx=0).stop 2023-07-18T03:16:14.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.541+0000 7effa81ff700 1 -- 172.21.15.101:0/3578397 shutdown_connections 2023-07-18T03:16:14.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.541+0000 7effa81ff700 1 --2- 172.21.15.101:0/3578397 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7effa010a040 0x7effa01a3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.541+0000 7effa81ff700 1 --2- 172.21.15.101:0/3578397 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7effa0100d00 0x7effa00fe950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.541+0000 7effa81ff700 1 --2- 172.21.15.101:0/3578397 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7eff8c0a12c0 0x7eff8c0a3770 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.544 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.541+0000 7effa81ff700 1 --2- 172.21.15.101:0/3578397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effa0101610 0x7effa00fee90 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.544 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.541+0000 7effa81ff700 1 -- 172.21.15.101:0/3578397 >> 172.21.15.101:0/3578397 conn(0x7effa00fa510 msgr2=0x7effa010c170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:14.544 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.541+0000 7effa81ff700 1 -- 172.21.15.101:0/3578397 shutdown_connections 2023-07-18T03:16:14.544 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.541+0000 7effa81ff700 1 -- 172.21.15.101:0/3578397 wait complete. 2023-07-18T03:16:14.566 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-18T03:16:14.601 INFO:tasks.workunit.client.0.smithi101.stdout: 1 577413 6.34 MB/sec execute 339 sec latency 2455.896 ms 2023-07-18T03:16:14.793 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.798+0000 7ff055fff700 1 -- 172.21.15.101:0/645874114 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff050105af0 msgr2=0x7ff050105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:14.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.798+0000 7ff055fff700 1 --2- 172.21.15.101:0/645874114 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff050105af0 0x7ff050105ec0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff040009990 tx=0x7ff040009ca0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.799+0000 7ff055fff700 1 -- 172.21.15.101:0/645874114 shutdown_connections 2023-07-18T03:16:14.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.799+0000 7ff055fff700 1 --2- 172.21.15.101:0/645874114 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff050100490 0x7ff05010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.799+0000 7ff055fff700 1 --2- 172.21.15.101:0/645874114 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff050105af0 0x7ff050105ec0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.799+0000 7ff055fff700 1 --2- 172.21.15.101:0/645874114 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0500ffae0 0x7ff0500fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.799+0000 7ff055fff700 1 -- 172.21.15.101:0/645874114 >> 172.21.15.101:0/645874114 conn(0x7ff0500fb5e0 msgr2=0x7ff0500fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:14.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.799+0000 7ff055fff700 1 -- 172.21.15.101:0/645874114 shutdown_connections 2023-07-18T03:16:14.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.799+0000 7ff055fff700 1 -- 172.21.15.101:0/645874114 wait complete. 2023-07-18T03:16:14.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.800+0000 7ff055fff700 1 Processor -- start 2023-07-18T03:16:14.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.800+0000 7ff055fff700 1 -- start start 2023-07-18T03:16:14.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.800+0000 7ff055fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0500ffae0 0x7ff050197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.800+0000 7ff055fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff050100490 0x7ff050198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.800+0000 7ff055fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff050105af0 0x7ff05019c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.800+0000 7ff055fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05019cdd0 con 0x7ff050100490 2023-07-18T03:16:14.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.800+0000 7ff055fff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05019cf40 con 0x7ff0500ffae0 2023-07-18T03:16:14.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.800+0000 7ff055fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05019d100 con 0x7ff050105af0 2023-07-18T03:16:14.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.801+0000 7ff04f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0500ffae0 0x7ff050197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.801+0000 7ff04effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff050100490 0x7ff050198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.801+0000 7ff04f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0500ffae0 0x7ff050197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:52472/0 (socket says 172.21.15.101:52472) 2023-07-18T03:16:14.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.801+0000 7ff04f7fe700 1 -- 172.21.15.101:0/786575960 learned_addr learned my addr 172.21.15.101:0/786575960 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:14.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.801+0000 7ff04ffff700 1 --2- 172.21.15.101:0/786575960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff050105af0 0x7ff05019c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.801+0000 7ff04f7fe700 1 -- 172.21.15.101:0/786575960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff050105af0 msgr2=0x7ff05019c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:14.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.801+0000 7ff04f7fe700 1 --2- 172.21.15.101:0/786575960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff050105af0 0x7ff05019c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.801+0000 7ff04f7fe700 1 -- 172.21.15.101:0/786575960 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff050100490 msgr2=0x7ff050198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:14.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.801+0000 7ff04f7fe700 1 --2- 172.21.15.101:0/786575960 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff050100490 0x7ff050198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.801+0000 7ff04f7fe700 1 -- 172.21.15.101:0/786575960 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff044009520 con 0x7ff0500ffae0 2023-07-18T03:16:14.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.801+0000 7ff04f7fe700 1 --2- 172.21.15.101:0/786575960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0500ffae0 0x7ff050197df0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff040005910 tx=0x7ff040005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:14.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.802+0000 7ff04cff9700 1 -- 172.21.15.101:0/786575960 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04001d070 con 0x7ff0500ffae0 2023-07-18T03:16:14.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.802+0000 7ff04cff9700 1 -- 172.21.15.101:0/786575960 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff040022470 con 0x7ff0500ffae0 2023-07-18T03:16:14.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.802+0000 7ff055fff700 1 -- 172.21.15.101:0/786575960 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0400095f0 con 0x7ff0500ffae0 2023-07-18T03:16:14.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.802+0000 7ff04cff9700 1 -- 172.21.15.101:0/786575960 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff040017650 con 0x7ff0500ffae0 2023-07-18T03:16:14.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.802+0000 7ff055fff700 1 -- 172.21.15.101:0/786575960 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff05019d610 con 0x7ff0500ffae0 2023-07-18T03:16:14.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.806+0000 7ff04cff9700 1 -- 172.21.15.101:0/786575960 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7ff0400225e0 con 0x7ff0500ffae0 2023-07-18T03:16:14.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.807+0000 7ff0699a8b80 1 Processor -- start 2023-07-18T03:16:14.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.808+0000 7ff0699a8b80 1 -- start start 2023-07-18T03:16:14.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.808+0000 7ff0699a8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b895bd92d0 0x55b895bd96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.808+0000 7ff0699a8b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b895bd9be0 0x55b895be4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.808+0000 7ff0699a8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b895be4ad0 0x55b895be6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.808+0000 7ff0699a8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b895adfcb0 con 0x55b895be4ad0 2023-07-18T03:16:14.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.808+0000 7ff0699a8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b895b32a40 con 0x55b895bd92d0 2023-07-18T03:16:14.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.808+0000 7ff0699a8b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b895b2bed0 con 0x55b895bd9be0 2023-07-18T03:16:14.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.809+0000 7ff04f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b895bd9be0 0x55b895be4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.809+0000 7ff04f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b895bd9be0 0x55b895be4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:59562/0 (socket says 172.21.15.101:59562) 2023-07-18T03:16:14.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.809+0000 7ff04f7fe700 1 -- 172.21.15.101:0/1598848927 learned_addr learned my addr 172.21.15.101:0/1598848927 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:14.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.809+0000 7ff04ffff700 1 --2- 172.21.15.101:0/1598848927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b895bd92d0 0x55b895bd96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.809+0000 7ff04effd700 1 --2- 172.21.15.101:0/1598848927 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b895be4ad0 0x55b895be6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.809+0000 7ff04f7fe700 1 -- 172.21.15.101:0/1598848927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b895bd92d0 msgr2=0x55b895bd96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:14.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.809+0000 7ff04f7fe700 1 --2- 172.21.15.101:0/1598848927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b895bd92d0 0x55b895bd96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.809+0000 7ff04f7fe700 1 -- 172.21.15.101:0/1598848927 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b895be4ad0 msgr2=0x55b895be6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:14.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.809+0000 7ff04f7fe700 1 --2- 172.21.15.101:0/1598848927 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b895be4ad0 0x55b895be6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.809+0000 7ff04f7fe700 1 -- 172.21.15.101:0/1598848927 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b895959de0 con 0x55b895bd9be0 2023-07-18T03:16:14.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.810+0000 7ff04f7fe700 1 --2- 172.21.15.101:0/1598848927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b895bd9be0 0x55b895be4590 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff0400cb6e0 tx=0x7ff0400d66b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20485c8b5c468597 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:14.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.810+0000 7ff04cff9700 1 --2- 172.21.15.101:0/786575960 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff0380a12c0 0x7ff0380a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.810+0000 7ff04ffff700 1 --2- 172.21.15.101:0/786575960 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff0380a12c0 0x7ff0380a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.810+0000 7ff035ffb700 1 -- 172.21.15.101:0/1598848927 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0400974c0 con 0x55b895bd9be0 2023-07-18T03:16:14.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.810+0000 7ff04cff9700 1 -- 172.21.15.101:0/786575960 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9461+0+0 (secure 0 0 0) 0x7ff0400ce430 con 0x7ff0500ffae0 2023-07-18T03:16:14.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.810+0000 7ff035ffb700 1 -- 172.21.15.101:0/1598848927 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff04009af50 con 0x55b895bd9be0 2023-07-18T03:16:14.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.810+0000 7ff035ffb700 1 -- 172.21.15.101:0/1598848927 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04009acc0 con 0x55b895bd9be0 2023-07-18T03:16:14.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.811+0000 7ff0699a8b80 1 -- 172.21.15.101:0/1598848927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b895bd9be0 msgr2=0x55b895be4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:14.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.811+0000 7ff04ffff700 1 --2- 172.21.15.101:0/786575960 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff0380a12c0 0x7ff0380a3770 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff0480127f0 tx=0x7ff04800b040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:14.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.811+0000 7ff0699a8b80 1 --2- 172.21.15.101:0/1598848927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b895bd9be0 0x55b895be4590 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff0400cb6e0 tx=0x7ff0400d66b0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.811+0000 7ff0699a8b80 1 -- 172.21.15.101:0/1598848927 shutdown_connections 2023-07-18T03:16:14.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.811+0000 7ff0699a8b80 1 --2- 172.21.15.101:0/1598848927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b895bd92d0 0x55b895bd96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.811+0000 7ff0699a8b80 1 --2- 172.21.15.101:0/1598848927 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b895be4ad0 0x55b895be6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.811+0000 7ff0699a8b80 1 --2- 172.21.15.101:0/1598848927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b895bd9be0 0x55b895be4590 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.811+0000 7ff0699a8b80 1 -- 172.21.15.101:0/1598848927 >> 172.21.15.101:0/1598848927 conn(0x55b895bcfda0 msgr2=0x55b895bd56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:14.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.812+0000 7ff0699a8b80 1 -- 172.21.15.101:0/1598848927 shutdown_connections 2023-07-18T03:16:14.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.812+0000 7ff0699a8b80 1 -- 172.21.15.101:0/1598848927 wait complete. 2023-07-18T03:16:14.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.812+0000 7ff0699a8b80 1 -- start start 2023-07-18T03:16:14.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.812+0000 7ff0699a8b80 1 Processor -- start 2023-07-18T03:16:14.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.813+0000 7ff0699a8b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b895bd92d0 0x55b895be4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.813+0000 7ff0699a8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b895bd9be0 0x55b895be53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.813+0000 7ff0699a8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b895be4ad0 0x55b895be5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.813+0000 7ff0699a8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b895acbac0 con 0x55b895bd9be0 2023-07-18T03:16:14.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.813+0000 7ff0699a8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b895acb7c0 con 0x55b895be4ad0 2023-07-18T03:16:14.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.813+0000 7ff0699a8b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8959af920 con 0x55b895bd92d0 2023-07-18T03:16:14.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.813+0000 7ff0699a8b80 10 client.0 _ll_register_callbacks cb 0x55b895ac9760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:16:14.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.813+0000 7ff04f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b895bd92d0 0x55b895be4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.813+0000 7ff04f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b895bd92d0 0x55b895be4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:59564/0 (socket says 172.21.15.101:59564) 2023-07-18T03:16:14.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.813+0000 7ff04effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b895bd9be0 0x55b895be53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.813+0000 7ff04f7fe700 1 -- 172.21.15.101:0/200545405 learned_addr learned my addr 172.21.15.101:0/200545405 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:14.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.813+0000 7ff04ffff700 1 --2- 172.21.15.101:0/200545405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b895be4ad0 0x55b895be5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.813 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.813+0000 7ff04f7fe700 1 -- 172.21.15.101:0/200545405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b895be4ad0 msgr2=0x55b895be5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:14.813 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.813+0000 7ff04f7fe700 1 --2- 172.21.15.101:0/200545405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b895be4ad0 0x55b895be5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.813 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.813+0000 7ff04f7fe700 1 -- 172.21.15.101:0/200545405 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b895bd9be0 msgr2=0x55b895be53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:14.814 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.813+0000 7ff04f7fe700 1 --2- 172.21.15.101:0/200545405 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b895bd9be0 0x55b895be53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.814 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.813+0000 7ff04f7fe700 1 -- 172.21.15.101:0/200545405 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff05019d610 con 0x55b895bd92d0 2023-07-18T03:16:14.814 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.814+0000 7ff04f7fe700 1 --2- 172.21.15.101:0/200545405 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b895bd92d0 0x55b895be4ea0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff0400d6190 tx=0x7ff040022c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59b1142e0a84c1e5 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:14.814 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.814+0000 7ff02effd700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:16:14.814 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.814+0000 7ff02effd700 1 -- 172.21.15.101:0/200545405 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04009aae0 con 0x55b895bd92d0 2023-07-18T03:16:14.814 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.814+0000 7ff02effd700 1 -- 172.21.15.101:0/200545405 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9461+0+0 (secure 0 0 0) 0x7ff04009b180 con 0x55b895bd92d0 2023-07-18T03:16:14.814 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.814+0000 7ff0699a8b80 1 -- 172.21.15.101:0/200545405 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b895a854f0 con 0x55b895bd92d0 2023-07-18T03:16:14.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.814+0000 7ff02effd700 1 -- 172.21.15.101:0/200545405 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff04009acc0 con 0x55b895bd92d0 2023-07-18T03:16:14.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.814+0000 7ff0699a8b80 10 client.27379 fetch_fsmap learned FSMap version 31 2023-07-18T03:16:14.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.814+0000 7ff0699a8b80 1 -- 172.21.15.101:0/200545405 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55b895942c60 con 0x55b895bd92d0 2023-07-18T03:16:14.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.814+0000 7ff02effd700 1 -- 172.21.15.101:0/200545405 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b895a854f0 con 0x55b895bd92d0 2023-07-18T03:16:14.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.815+0000 7ff02effd700 1 -- 172.21.15.101:0/200545405 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff0400cb9f0 con 0x55b895bd92d0 2023-07-18T03:16:14.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.815+0000 7ff0699a8b80 10 client.27379 fetch_fsmap finished waiting for FSMap version 31 2023-07-18T03:16:14.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.815+0000 7ff0699a8b80 10 client.27379 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:16:14.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.815+0000 7ff0699a8b80 20 client.27379 populate_metadata read hostname 'smithi101' 2023-07-18T03:16:14.816 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.815+0000 7ff0699a8b80 1 --2- 172.21.15.101:0/200545405 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b895bf3cb0 0x55b895bf6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.816 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.815+0000 7ff0699a8b80 4 client.27379 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:16:14.816 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.815+0000 7ff0699a8b80 1 -- 172.21.15.101:0/200545405 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b895b24eb0 con 0x55b895bf3cb0 2023-07-18T03:16:14.816 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.815+0000 7ff04effd700 1 --2- 172.21.15.101:0/200545405 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b895bf3cb0 0x55b895bf6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.816 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.816+0000 7ff04effd700 1 --2- 172.21.15.101:0/200545405 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b895bf3cb0 0x55b895bf6090 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ab5b8b3760bfde8 server_cookie=2b48236fecfd3e12 in_seq=0 out_seq=0 2023-07-18T03:16:14.816 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.816+0000 7ff02effd700 10 client.27379 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:16:14.816 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.816+0000 7ff02effd700 1 -- 172.21.15.101:0/200545405 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b895b24eb0 con 0x55b895bf3cb0 2023-07-18T03:16:14.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.816+0000 7ff02effd700 10 client.27379 handle_command_reply: tid=0 2023-07-18T03:16:14.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.816+0000 7ff0699a8b80 1 client.27379 shutdown 2023-07-18T03:16:14.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.816+0000 7ff0699a8b80 1 -- 172.21.15.101:0/200545405 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b895bd92d0 msgr2=0x55b895be4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:14.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.817+0000 7ff0699a8b80 1 --2- 172.21.15.101:0/200545405 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b895bd92d0 0x55b895be4ea0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff0400d6190 tx=0x7ff040022c00 comp rx=0 tx=0).stop 2023-07-18T03:16:14.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.817+0000 7ff0699a8b80 1 -- 172.21.15.101:0/200545405 shutdown_connections 2023-07-18T03:16:14.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.817+0000 7ff0699a8b80 1 --2- 172.21.15.101:0/200545405 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b895bf3cb0 0x55b895bf6090 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.817+0000 7ff0699a8b80 1 --2- 172.21.15.101:0/200545405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b895be4ad0 0x55b895be5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.817+0000 7ff0699a8b80 1 --2- 172.21.15.101:0/200545405 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b895bd92d0 0x55b895be4ea0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.817+0000 7ff02effd700 0 client.27379 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:16:14.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.817+0000 7ff0699a8b80 1 --2- 172.21.15.101:0/200545405 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b895bd9be0 0x55b895be53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.817+0000 7ff0699a8b80 1 -- 172.21.15.101:0/200545405 >> 172.21.15.101:0/200545405 conn(0x55b895bcfda0 msgr2=0x55b895bd5230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:14.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.817+0000 7ff0699a8b80 1 -- 172.21.15.101:0/200545405 shutdown_connections 2023-07-18T03:16:14.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.817+0000 7ff0699a8b80 1 -- 172.21.15.101:0/200545405 wait complete. 2023-07-18T03:16:14.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.817+0000 7ff0699a8b80 20 client.27379 trim_cache size 0 max 16384 2023-07-18T03:16:14.826 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.832+0000 7ff0699a8b80 1 Processor -- start 2023-07-18T03:16:14.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.832+0000 7ff0699a8b80 1 -- start start 2023-07-18T03:16:14.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.833+0000 7ff0699a8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b895bd92d0 0x55b895bef680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.833+0000 7ff0699a8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b895bd9be0 0x55b895bdeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.833+0000 7ff0699a8b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b895be4ad0 0x55b895bdf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.833+0000 7ff0699a8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b895bcf970 con 0x55b895bd92d0 2023-07-18T03:16:14.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.833+0000 7ff0699a8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b895bd64d0 con 0x55b895bd9be0 2023-07-18T03:16:14.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.833+0000 7ff0699a8b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b895be4190 con 0x55b895be4ad0 2023-07-18T03:16:14.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.833+0000 7ff04effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b895bd9be0 0x55b895bdeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.833+0000 7ff04f7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b895bd92d0 0x55b895bef680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.833+0000 7ff04effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b895bd9be0 0x55b895bdeeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:52496/0 (socket says 172.21.15.101:52496) 2023-07-18T03:16:14.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.833+0000 7ff04effd700 1 -- 172.21.15.101:0/1725943801 learned_addr learned my addr 172.21.15.101:0/1725943801 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:14.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.833+0000 7ff04ffff700 1 --2- 172.21.15.101:0/1725943801 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b895be4ad0 0x55b895bdf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.833+0000 7ff04f7fe700 1 -- 172.21.15.101:0/1725943801 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b895be4ad0 msgr2=0x55b895bdf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:14.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.833+0000 7ff04f7fe700 1 --2- 172.21.15.101:0/1725943801 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b895be4ad0 0x55b895bdf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.833+0000 7ff04f7fe700 1 -- 172.21.15.101:0/1725943801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b895bd9be0 msgr2=0x55b895bdeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:14.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.833+0000 7ff04f7fe700 1 --2- 172.21.15.101:0/1725943801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b895bd9be0 0x55b895bdeeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.833+0000 7ff04f7fe700 1 -- 172.21.15.101:0/1725943801 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b895bd15e0 con 0x55b895bd92d0 2023-07-18T03:16:14.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.833+0000 7ff04effd700 1 --2- 172.21.15.101:0/1725943801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b895bd9be0 0x55b895bdeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:16:14.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.834+0000 7ff04f7fe700 1 --2- 172.21.15.101:0/1725943801 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b895bd92d0 0x55b895bef680 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff04009ae80 tx=0x7ff0400975d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e33b2ef7f82836e8 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:14.832 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.834+0000 7ff0367fc700 1 -- 172.21.15.101:0/1725943801 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0400d6a80 con 0x55b895bd92d0 2023-07-18T03:16:14.832 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.834+0000 7ff0367fc700 1 -- 172.21.15.101:0/1725943801 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff04009acc0 con 0x55b895bd92d0 2023-07-18T03:16:14.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.834+0000 7ff0367fc700 1 -- 172.21.15.101:0/1725943801 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04009aae0 con 0x55b895bd92d0 2023-07-18T03:16:14.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.834+0000 7ff0699a8b80 1 -- 172.21.15.101:0/1725943801 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b895bd92d0 msgr2=0x55b895bef680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:14.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.834+0000 7ff0699a8b80 1 --2- 172.21.15.101:0/1725943801 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b895bd92d0 0x55b895bef680 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff04009ae80 tx=0x7ff0400975d0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.835+0000 7ff0699a8b80 1 -- 172.21.15.101:0/1725943801 shutdown_connections 2023-07-18T03:16:14.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.835+0000 7ff0699a8b80 1 --2- 172.21.15.101:0/1725943801 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b895be4ad0 0x55b895bdf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.835+0000 7ff0699a8b80 1 --2- 172.21.15.101:0/1725943801 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b895bd92d0 0x55b895bef680 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.835+0000 7ff0699a8b80 1 --2- 172.21.15.101:0/1725943801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b895bd9be0 0x55b895bdeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.835+0000 7ff0699a8b80 1 -- 172.21.15.101:0/1725943801 >> 172.21.15.101:0/1725943801 conn(0x55b895bcfda0 msgr2=0x55b895be3420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:14.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.835+0000 7ff0699a8b80 1 -- 172.21.15.101:0/1725943801 shutdown_connections 2023-07-18T03:16:14.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.835+0000 7ff0699a8b80 1 -- 172.21.15.101:0/1725943801 wait complete. 2023-07-18T03:16:14.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.835+0000 7ff0699a8b80 1 -- start start 2023-07-18T03:16:14.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.835+0000 7ff0699a8b80 1 Processor -- start 2023-07-18T03:16:14.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.836+0000 7ff0699a8b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b895bd92d0 0x55b895be70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.836+0000 7ff0699a8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b895bd9be0 0x55b895be75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.836+0000 7ff0699a8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b895be4ad0 0x55b895c46b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.836+0000 7ff0699a8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b895c47080 con 0x55b895be4ad0 2023-07-18T03:16:14.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.836+0000 7ff0699a8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b895c471c0 con 0x55b895bd9be0 2023-07-18T03:16:14.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.836+0000 7ff0699a8b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b895c47300 con 0x55b895bd92d0 2023-07-18T03:16:14.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.836+0000 7ff0699a8b80 10 client.0 _ll_register_callbacks cb 0x7ff048009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:16:14.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.836+0000 7ff04ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b895be4ad0 0x55b895c46b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.836+0000 7ff04f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b895bd92d0 0x55b895be70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.836+0000 7ff04ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b895be4ad0 0x55b895c46b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:56776/0 (socket says 172.21.15.101:56776) 2023-07-18T03:16:14.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.836+0000 7ff04ffff700 1 -- 172.21.15.101:0/1097461810 learned_addr learned my addr 172.21.15.101:0/1097461810 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:14.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.836+0000 7ff04f7fe700 1 --2- 172.21.15.101:0/1097461810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b895bd92d0 0x55b895be70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:59576/0 (socket says 172.21.15.101:59576) 2023-07-18T03:16:14.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.836+0000 7ff04effd700 1 --2- 172.21.15.101:0/1097461810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b895bd9be0 0x55b895be75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.836+0000 7ff04f7fe700 1 -- 172.21.15.101:0/1097461810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b895bd9be0 msgr2=0x55b895be75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:14.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.836+0000 7ff04f7fe700 1 --2- 172.21.15.101:0/1097461810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b895bd9be0 0x55b895be75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.836+0000 7ff04f7fe700 1 -- 172.21.15.101:0/1097461810 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b895be4ad0 msgr2=0x55b895c46b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:14.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.836+0000 7ff04f7fe700 1 --2- 172.21.15.101:0/1097461810 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b895be4ad0 0x55b895c46b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.836+0000 7ff04f7fe700 1 -- 172.21.15.101:0/1097461810 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff044009c40 con 0x55b895bd92d0 2023-07-18T03:16:14.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.837+0000 7ff04f7fe700 1 --2- 172.21.15.101:0/1097461810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b895bd92d0 0x55b895be70a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff040000c00 tx=0x7ff0400a3660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31f2b27ca17e0405 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:14.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.837+0000 7ff02effd700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:16:14.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.837+0000 7ff02effd700 1 -- 172.21.15.101:0/1097461810 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0400a32b0 con 0x55b895bd92d0 2023-07-18T03:16:14.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.837+0000 7ff02effd700 1 -- 172.21.15.101:0/1097461810 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9461+0+0 (secure 0 0 0) 0x7ff04009b180 con 0x55b895bd92d0 2023-07-18T03:16:14.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.837+0000 7ff0699a8b80 1 -- 172.21.15.101:0/1097461810 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff04009af50 con 0x55b895bd92d0 2023-07-18T03:16:14.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.837+0000 7ff02effd700 1 -- 172.21.15.101:0/1097461810 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0400d6a80 con 0x55b895bd92d0 2023-07-18T03:16:14.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.837+0000 7ff0699a8b80 10 client.27385 fetch_fsmap learned FSMap version 31 2023-07-18T03:16:14.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.837+0000 7ff0699a8b80 1 -- 172.21.15.101:0/1097461810 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55b895942c60 con 0x55b895bd92d0 2023-07-18T03:16:14.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.837+0000 7ff02effd700 1 -- 172.21.15.101:0/1097461810 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff04009af50 con 0x55b895bd92d0 2023-07-18T03:16:14.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.837+0000 7ff02effd700 1 -- 172.21.15.101:0/1097461810 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff04009a670 con 0x55b895bd92d0 2023-07-18T03:16:14.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.837+0000 7ff0699a8b80 10 client.27385 fetch_fsmap finished waiting for FSMap version 31 2023-07-18T03:16:14.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.837+0000 7ff0699a8b80 10 client.27385 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:16:14.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.837+0000 7ff0699a8b80 20 client.27385 populate_metadata read hostname 'smithi101' 2023-07-18T03:16:14.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.838+0000 7ff0699a8b80 1 --2- 172.21.15.101:0/1097461810 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b895c48480 0x55b895c4a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:14.839 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.838+0000 7ff0699a8b80 4 client.27385 mds_command: new command op to 24562 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:16:14.839 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.838+0000 7ff0699a8b80 1 -- 172.21.15.101:0/1097461810 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b895ae7050 con 0x55b895c48480 2023-07-18T03:16:14.839 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.838+0000 7ff04effd700 1 --2- 172.21.15.101:0/1097461810 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b895c48480 0x55b895c4a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:14.840 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.838+0000 7ff04effd700 1 --2- 172.21.15.101:0/1097461810 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b895c48480 0x55b895c4a860 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc57e743ecc3740a server_cookie=7e712cef76b755ac in_seq=0 out_seq=0 2023-07-18T03:16:14.840 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.838+0000 7ff02effd700 10 client.27385 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:16:14.840 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.839+0000 7ff02effd700 1 -- 172.21.15.101:0/1097461810 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b895ae7050 con 0x55b895c48480 2023-07-18T03:16:14.840 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.839+0000 7ff02effd700 10 client.27385 handle_command_reply: tid=0 2023-07-18T03:16:14.840 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.839+0000 7ff0699a8b80 1 client.27385 shutdown 2023-07-18T03:16:14.841 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.839+0000 7ff0699a8b80 1 -- 172.21.15.101:0/1097461810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b895bd92d0 msgr2=0x55b895be70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:14.841 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.839+0000 7ff0699a8b80 1 --2- 172.21.15.101:0/1097461810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b895bd92d0 0x55b895be70a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff040000c00 tx=0x7ff0400a3660 comp rx=0 tx=0).stop 2023-07-18T03:16:14.841 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.839+0000 7ff0699a8b80 1 -- 172.21.15.101:0/1097461810 shutdown_connections 2023-07-18T03:16:14.841 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.839+0000 7ff0699a8b80 1 --2- 172.21.15.101:0/1097461810 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b895c48480 0x55b895c4a860 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.841 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.839+0000 7ff0699a8b80 1 --2- 172.21.15.101:0/1097461810 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b895be4ad0 0x55b895c46b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.841 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.839+0000 7ff0699a8b80 1 --2- 172.21.15.101:0/1097461810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b895bd92d0 0x55b895be70a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.841 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.839+0000 7ff0699a8b80 1 --2- 172.21.15.101:0/1097461810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b895bd9be0 0x55b895be75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.842 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.839+0000 7ff0699a8b80 1 -- 172.21.15.101:0/1097461810 >> 172.21.15.101:0/1097461810 conn(0x55b895bcfda0 msgr2=0x55b895bdeeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:14.842 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.839+0000 7ff02effd700 0 client.27385 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:16:14.842 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.839+0000 7ff0699a8b80 1 -- 172.21.15.101:0/1097461810 shutdown_connections 2023-07-18T03:16:14.842 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.839+0000 7ff0699a8b80 1 -- 172.21.15.101:0/1097461810 wait complete. 2023-07-18T03:16:14.842 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.840+0000 7ff0699a8b80 20 client.27385 trim_cache size 0 max 16384 2023-07-18T03:16:14.842 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:16:14.843 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.840+0000 7ff055fff700 1 -- 172.21.15.101:0/786575960 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff0380a12c0 msgr2=0x7ff0380a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:14.843 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.840+0000 7ff055fff700 1 --2- 172.21.15.101:0/786575960 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff0380a12c0 0x7ff0380a3770 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff0480127f0 tx=0x7ff04800b040 comp rx=0 tx=0).stop 2023-07-18T03:16:14.843 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.840+0000 7ff055fff700 1 -- 172.21.15.101:0/786575960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0500ffae0 msgr2=0x7ff050197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:14.843 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.840+0000 7ff055fff700 1 --2- 172.21.15.101:0/786575960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0500ffae0 0x7ff050197df0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff040005910 tx=0x7ff040005720 comp rx=0 tx=0).stop 2023-07-18T03:16:14.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.840+0000 7ff055fff700 1 -- 172.21.15.101:0/786575960 shutdown_connections 2023-07-18T03:16:14.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.840+0000 7ff055fff700 1 --2- 172.21.15.101:0/786575960 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff0380a12c0 0x7ff0380a3770 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.840+0000 7ff055fff700 1 --2- 172.21.15.101:0/786575960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0500ffae0 0x7ff050197df0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.840+0000 7ff055fff700 1 --2- 172.21.15.101:0/786575960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff050105af0 0x7ff05019c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.840+0000 7ff055fff700 1 --2- 172.21.15.101:0/786575960 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff050100490 0x7ff050198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:14.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.840+0000 7ff055fff700 1 -- 172.21.15.101:0/786575960 >> 172.21.15.101:0/786575960 conn(0x7ff0500fb5e0 msgr2=0x7ff05010ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:14.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.840+0000 7ff055fff700 1 -- 172.21.15.101:0/786575960 shutdown_connections 2023-07-18T03:16:14.845 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:14.840+0000 7ff055fff700 1 -- 172.21.15.101:0/786575960 wait complete. 2023-07-18T03:16:15.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:14 smithi101 ceph-mon[126379]: pgmap v268: 129 pgs: 129 active+clean; 418 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 168 op/s 2023-07-18T03:16:15.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:14 smithi101 ceph-mon[126379]: osdmap e98: 12 total, 12 up, 12 in 2023-07-18T03:16:15.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:14 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2997705627' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:16:15.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:14 smithi155 ceph-mon[128885]: pgmap v268: 129 pgs: 129 active+clean; 418 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 168 op/s 2023-07-18T03:16:15.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:14 smithi155 ceph-mon[128885]: osdmap e98: 12 total, 12 up, 12 in 2023-07-18T03:16:15.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:14 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/2997705627' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:16:15.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:14 smithi179 ceph-mon[129229]: pgmap v268: 129 pgs: 129 active+clean; 418 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 168 op/s 2023-07-18T03:16:15.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:14 smithi179 ceph-mon[129229]: osdmap e98: 12 total, 12 up, 12 in 2023-07-18T03:16:15.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:14 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/2997705627' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:16:15.601 INFO:tasks.workunit.client.0.smithi101.stdout: 1 579197 6.37 MB/sec execute 340 sec latency 25.053 ms 2023-07-18T03:16:15.854 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-18T03:16:15.855 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-18T03:16:16.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.083+0000 7f28995ff700 1 -- 172.21.15.101:0/2204161579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2894073160 msgr2=0x7f289410be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:16.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.083+0000 7f28995ff700 1 --2- 172.21.15.101:0/2204161579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2894073160 0x7f289410be60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f28880098f0 tx=0x7f2888009c00 comp rx=0 tx=0).stop 2023-07-18T03:16:16.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.083+0000 7f28995ff700 1 -- 172.21.15.101:0/2204161579 shutdown_connections 2023-07-18T03:16:16.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.084+0000 7f28995ff700 1 --2- 172.21.15.101:0/2204161579 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f289410c3a0 0x7f289410e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.084+0000 7f28995ff700 1 --2- 172.21.15.101:0/2204161579 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2894072850 0x7f2894072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.084+0000 7f28995ff700 1 --2- 172.21.15.101:0/2204161579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2894073160 0x7f289410be60 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.084+0000 7f28995ff700 1 -- 172.21.15.101:0/2204161579 >> 172.21.15.101:0/2204161579 conn(0x7f28940fb600 msgr2=0x7f28940fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:16.081 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.084+0000 7f28995ff700 1 -- 172.21.15.101:0/2204161579 shutdown_connections 2023-07-18T03:16:16.081 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.084+0000 7f28995ff700 1 -- 172.21.15.101:0/2204161579 wait complete. 2023-07-18T03:16:16.084 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.085+0000 7f28995ff700 1 Processor -- start 2023-07-18T03:16:16.084 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.085+0000 7f28995ff700 1 -- start start 2023-07-18T03:16:16.085 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.085+0000 7f28995ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2894072850 0x7f2894197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:16.085 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.085+0000 7f28995ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2894073160 0x7f28941983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:16.085 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.085+0000 7f28995ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f289410c3a0 0x7f289419c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:16.085 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.085+0000 7f28995ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289419ce70 con 0x7f2894073160 2023-07-18T03:16:16.085 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.085+0000 7f28995ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289419cfe0 con 0x7f289410c3a0 2023-07-18T03:16:16.086 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.085+0000 7f28995ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289419d1a0 con 0x7f2894072850 2023-07-18T03:16:16.086 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.086+0000 7f2892ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2894072850 0x7f2894197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:16.086 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.086+0000 7f2892ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2894072850 0x7f2894197e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:35742/0 (socket says 172.21.15.101:35742) 2023-07-18T03:16:16.086 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.086+0000 7f2892ffd700 1 -- 172.21.15.101:0/1152288738 learned_addr learned my addr 172.21.15.101:0/1152288738 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:16.087 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.086+0000 7f28927fc700 1 --2- 172.21.15.101:0/1152288738 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2894073160 0x7f28941983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:16.087 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.086+0000 7f28937fe700 1 --2- 172.21.15.101:0/1152288738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f289410c3a0 0x7f289419c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:16.087 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.086+0000 7f2892ffd700 1 -- 172.21.15.101:0/1152288738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f289410c3a0 msgr2=0x7f289419c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:16.088 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.086+0000 7f2892ffd700 1 --2- 172.21.15.101:0/1152288738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f289410c3a0 0x7f289419c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.088 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.086+0000 7f2892ffd700 1 -- 172.21.15.101:0/1152288738 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2894073160 msgr2=0x7f28941983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:16.088 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.086+0000 7f2892ffd700 1 --2- 172.21.15.101:0/1152288738 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2894073160 0x7f28941983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.088 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.086+0000 7f2892ffd700 1 -- 172.21.15.101:0/1152288738 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28840095f0 con 0x7f2894072850 2023-07-18T03:16:16.088 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.086+0000 7f2892ffd700 1 --2- 172.21.15.101:0/1152288738 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2894072850 0x7f2894197e90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f288400ec10 tx=0x7f288400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:16.088 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.086+0000 7f287bfff700 1 -- 172.21.15.101:0/1152288738 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288400cc90 con 0x7f2894072850 2023-07-18T03:16:16.089 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.086+0000 7f28995ff700 1 -- 172.21.15.101:0/1152288738 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2888009520 con 0x7f2894072850 2023-07-18T03:16:16.089 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.086+0000 7f287bfff700 1 -- 172.21.15.101:0/1152288738 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28840044e0 con 0x7f2894072850 2023-07-18T03:16:16.089 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.086+0000 7f287bfff700 1 -- 172.21.15.101:0/1152288738 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2884010430 con 0x7f2894072850 2023-07-18T03:16:16.089 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.087+0000 7f28995ff700 1 -- 172.21.15.101:0/1152288738 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28941a4ae0 con 0x7f2894072850 2023-07-18T03:16:16.089 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.090+0000 7f287bfff700 1 -- 172.21.15.101:0/1152288738 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f2884003f90 con 0x7f2894072850 2023-07-18T03:16:16.094 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.091+0000 7f28acf9eb80 1 Processor -- start 2023-07-18T03:16:16.094 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.091+0000 7f28acf9eb80 1 -- start start 2023-07-18T03:16:16.094 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.091+0000 7f287bfff700 1 --2- 172.21.15.101:0/1152288738 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f287c0a12c0 0x7f287c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:16.094 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.091+0000 7f287bfff700 1 -- 172.21.15.101:0/1152288738 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f2884014070 con 0x7f2894072850 2023-07-18T03:16:16.094 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.091+0000 7f28acf9eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a777f41350 0x55a777f41720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:16.094 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.091+0000 7f28acf9eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a777f41c60 0x55a777f4c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:16.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.091+0000 7f28acf9eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a777f4cb50 0x55a777f4ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:16.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.091+0000 7f28acf9eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a777e47d30 con 0x55a777f41c60 2023-07-18T03:16:16.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.091+0000 7f28acf9eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a777e9aac0 con 0x55a777f41350 2023-07-18T03:16:16.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.091+0000 7f28acf9eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a777e93f50 con 0x55a777f4cb50 2023-07-18T03:16:16.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.092+0000 7f28937fe700 1 --2- 172.21.15.101:0/1152288738 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f287c0a12c0 0x7f287c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:16.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.092+0000 7f28937fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a777f4cb50 0x55a777f4ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:16.096 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.092+0000 7f2892ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a777f41350 0x55a777f41720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:16.096 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.092+0000 7f28937fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a777f4cb50 0x55a777f4ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:35754/0 (socket says 172.21.15.101:35754) 2023-07-18T03:16:16.096 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.092+0000 7f28937fe700 1 -- 172.21.15.101:0/3665712166 learned_addr learned my addr 172.21.15.101:0/3665712166 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:16.096 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.092+0000 7f2892ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a777f41350 0x55a777f41720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:51170/0 (socket says 172.21.15.101:51170) 2023-07-18T03:16:16.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.092+0000 7f28927fc700 1 --2- 172.21.15.101:0/3665712166 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a777f41c60 0x55a777f4c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:16.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.092+0000 7f2892ffd700 1 -- 172.21.15.101:0/3665712166 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a777f4cb50 msgr2=0x55a777f4ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:16.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.092+0000 7f2892ffd700 1 --2- 172.21.15.101:0/3665712166 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a777f4cb50 0x55a777f4ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.092+0000 7f2892ffd700 1 -- 172.21.15.101:0/3665712166 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a777f41c60 msgr2=0x55a777f4c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:16.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.092+0000 7f2892ffd700 1 --2- 172.21.15.101:0/3665712166 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a777f41c60 0x55a777f4c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.092+0000 7f2892ffd700 1 -- 172.21.15.101:0/3665712166 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a777cc1e60 con 0x55a777f41350 2023-07-18T03:16:16.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.092+0000 7f28927fc700 1 --2- 172.21.15.101:0/3665712166 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a777f41c60 0x55a777f4c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:16:16.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.092+0000 7f28937fe700 1 --2- 172.21.15.101:0/3665712166 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a777f4cb50 0x55a777f4ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:16:16.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.092+0000 7f28937fe700 1 --2- 172.21.15.101:0/1152288738 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f287c0a12c0 0x7f287c0a3770 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f288c016dd0 tx=0x7f288c016450 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:16.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.092+0000 7f2892ffd700 1 --2- 172.21.15.101:0/3665712166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a777f41350 0x55a777f41720 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f28840ca990 tx=0x7f288409a1b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0069c8224037b82 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:16.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.093+0000 7f28797fa700 1 -- 172.21.15.101:0/3665712166 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288409e170 con 0x55a777f41350 2023-07-18T03:16:16.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.093+0000 7f28797fa700 1 -- 172.21.15.101:0/3665712166 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f288409e340 con 0x55a777f41350 2023-07-18T03:16:16.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.093+0000 7f28797fa700 1 -- 172.21.15.101:0/3665712166 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28840cd220 con 0x55a777f41350 2023-07-18T03:16:16.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.093+0000 7f28acf9eb80 1 -- 172.21.15.101:0/3665712166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a777f41350 msgr2=0x55a777f41720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:16.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.093+0000 7f28acf9eb80 1 --2- 172.21.15.101:0/3665712166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a777f41350 0x55a777f41720 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f28840ca990 tx=0x7f288409a1b0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.093+0000 7f28acf9eb80 1 -- 172.21.15.101:0/3665712166 shutdown_connections 2023-07-18T03:16:16.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.094+0000 7f28acf9eb80 1 --2- 172.21.15.101:0/3665712166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a777f41350 0x55a777f41720 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.094+0000 7f28acf9eb80 1 --2- 172.21.15.101:0/3665712166 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a777f4cb50 0x55a777f4ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.094+0000 7f28acf9eb80 1 --2- 172.21.15.101:0/3665712166 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a777f41c60 0x55a777f4c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.094+0000 7f28acf9eb80 1 -- 172.21.15.101:0/3665712166 >> 172.21.15.101:0/3665712166 conn(0x55a777f37e20 msgr2=0x55a777f3d720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:16.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.094+0000 7f28acf9eb80 1 -- 172.21.15.101:0/3665712166 shutdown_connections 2023-07-18T03:16:16.100 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.094+0000 7f28acf9eb80 1 -- 172.21.15.101:0/3665712166 wait complete. 2023-07-18T03:16:16.100 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.094+0000 7f28acf9eb80 1 -- start start 2023-07-18T03:16:16.100 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.094+0000 7f28acf9eb80 1 Processor -- start 2023-07-18T03:16:16.100 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.095+0000 7f28acf9eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a777f41350 0x55a777f498c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:16.100 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.095+0000 7f28acf9eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a777f41c60 0x55a777f49e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:16.100 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.095+0000 7f28acf9eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a777f4cb50 0x55a777f4a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:16.100 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.095+0000 7f28acf9eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a777e33b40 con 0x55a777f41350 2023-07-18T03:16:16.101 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.095+0000 7f28acf9eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a777e33840 con 0x55a777f4cb50 2023-07-18T03:16:16.101 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.095+0000 7f28acf9eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a777d179a0 con 0x55a777f41c60 2023-07-18T03:16:16.101 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.095+0000 7f28acf9eb80 10 client.0 _ll_register_callbacks cb 0x55a777e317e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:16:16.101 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.096+0000 7f2892ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a777f41350 0x55a777f498c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:16.101 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.096+0000 7f28927fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a777f41c60 0x55a777f49e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:16.101 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.096+0000 7f28927fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a777f41c60 0x55a777f49e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:35760/0 (socket says 172.21.15.101:35760) 2023-07-18T03:16:16.101 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.096+0000 7f2892ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a777f41350 0x55a777f498c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:46892/0 (socket says 172.21.15.101:46892) 2023-07-18T03:16:16.102 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.096+0000 7f28927fc700 1 -- 172.21.15.101:0/3765553494 learned_addr learned my addr 172.21.15.101:0/3765553494 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:16.102 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.096+0000 7f28937fe700 1 --2- 172.21.15.101:0/3765553494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a777f4cb50 0x55a777f4a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:16.102 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.096+0000 7f28927fc700 1 -- 172.21.15.101:0/3765553494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a777f4cb50 msgr2=0x55a777f4a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:16.102 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.096+0000 7f28927fc700 1 --2- 172.21.15.101:0/3765553494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a777f4cb50 0x55a777f4a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.102 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.096+0000 7f28927fc700 1 -- 172.21.15.101:0/3765553494 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a777f41350 msgr2=0x55a777f498c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:16.102 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.096+0000 7f28927fc700 1 --2- 172.21.15.101:0/3765553494 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a777f41350 0x55a777f498c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.102 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.096+0000 7f28927fc700 1 -- 172.21.15.101:0/3765553494 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f288c01abe0 con 0x55a777f41c60 2023-07-18T03:16:16.102 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.096+0000 7f28927fc700 1 --2- 172.21.15.101:0/3765553494 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a777f41c60 0x55a777f49e00 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2888000c00 tx=0x7f288800b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ff8253338c47dd5 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:16.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.096+0000 7f2876ff5700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:16:16.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.096+0000 7f2876ff5700 1 -- 172.21.15.101:0/3765553494 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288801c070 con 0x55a777f41c60 2023-07-18T03:16:16.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.096+0000 7f28acf9eb80 1 -- 172.21.15.101:0/3765553494 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a777ded570 con 0x55a777f41c60 2023-07-18T03:16:16.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.097+0000 7f2876ff5700 1 -- 172.21.15.101:0/3765553494 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f288800f8f0 con 0x55a777f41c60 2023-07-18T03:16:16.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.097+0000 7f2876ff5700 1 -- 172.21.15.101:0/3765553494 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f288800bb90 con 0x55a777f41c60 2023-07-18T03:16:16.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.097+0000 7f28acf9eb80 10 client.27397 fetch_fsmap learned FSMap version 31 2023-07-18T03:16:16.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.097+0000 7f28acf9eb80 1 -- 172.21.15.101:0/3765553494 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f28941a4ae0 con 0x55a777f41c60 2023-07-18T03:16:16.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.097+0000 7f2876ff5700 1 -- 172.21.15.101:0/3765553494 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a777ded570 con 0x55a777f41c60 2023-07-18T03:16:16.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.097+0000 7f2876ff5700 1 -- 172.21.15.101:0/3765553494 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f2888015020 con 0x55a777f41c60 2023-07-18T03:16:16.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.097+0000 7f28acf9eb80 10 client.27397 fetch_fsmap finished waiting for FSMap version 31 2023-07-18T03:16:16.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.097+0000 7f28acf9eb80 10 client.27397 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:16:16.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.097+0000 7f28acf9eb80 20 client.27397 populate_metadata read hostname 'smithi101' 2023-07-18T03:16:16.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.097+0000 7f28acf9eb80 1 --2- 172.21.15.101:0/3765553494 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a777f5c560 0x55a777f5e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:16.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.097+0000 7f28acf9eb80 4 client.27397 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:16:16.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.097+0000 7f28acf9eb80 1 -- 172.21.15.101:0/3765553494 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a777e8cf30 con 0x55a777f5c560 2023-07-18T03:16:16.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.098+0000 7f2892ffd700 1 --2- 172.21.15.101:0/3765553494 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a777f5c560 0x55a777f5e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:16.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.098+0000 7f2892ffd700 1 --2- 172.21.15.101:0/3765553494 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a777f5c560 0x55a777f5e970 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4b01e4c1a6d109f server_cookie=f8aa035d60e1a2bb in_seq=0 out_seq=0 2023-07-18T03:16:16.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.098+0000 7f2876ff5700 10 client.27397 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:16:16.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.098+0000 7f2876ff5700 1 -- 172.21.15.101:0/3765553494 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a777e8cf30 con 0x55a777f5c560 2023-07-18T03:16:16.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.098+0000 7f2876ff5700 10 client.27397 handle_command_reply: tid=0 2023-07-18T03:16:16.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.099+0000 7f28acf9eb80 1 client.27397 shutdown 2023-07-18T03:16:16.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.099+0000 7f28acf9eb80 1 -- 172.21.15.101:0/3765553494 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a777f41c60 msgr2=0x55a777f49e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:16.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.099+0000 7f28acf9eb80 1 --2- 172.21.15.101:0/3765553494 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a777f41c60 0x55a777f49e00 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2888000c00 tx=0x7f288800b790 comp rx=0 tx=0).stop 2023-07-18T03:16:16.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.099+0000 7f28acf9eb80 1 -- 172.21.15.101:0/3765553494 shutdown_connections 2023-07-18T03:16:16.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.099+0000 7f28acf9eb80 1 --2- 172.21.15.101:0/3765553494 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a777f5c560 0x55a777f5e970 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.099+0000 7f28acf9eb80 1 --2- 172.21.15.101:0/3765553494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a777f4cb50 0x55a777f4a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.099+0000 7f28acf9eb80 1 --2- 172.21.15.101:0/3765553494 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a777f41350 0x55a777f498c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.099+0000 7f2876ff5700 0 client.27397 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:16:16.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.099+0000 7f28acf9eb80 1 --2- 172.21.15.101:0/3765553494 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a777f41c60 0x55a777f49e00 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.099+0000 7f28acf9eb80 1 -- 172.21.15.101:0/3765553494 >> 172.21.15.101:0/3765553494 conn(0x55a777f37e20 msgr2=0x55a777f3d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:16.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.099+0000 7f28acf9eb80 1 -- 172.21.15.101:0/3765553494 shutdown_connections 2023-07-18T03:16:16.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.099+0000 7f28acf9eb80 1 -- 172.21.15.101:0/3765553494 wait complete. 2023-07-18T03:16:16.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.099+0000 7f28acf9eb80 20 client.27397 trim_cache size 0 max 16384 2023-07-18T03:16:16.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.117+0000 7f28acf9eb80 1 Processor -- start 2023-07-18T03:16:16.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.117+0000 7f28acf9eb80 1 -- start start 2023-07-18T03:16:16.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.117+0000 7f28acf9eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a777f41350 0x55a777f53530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:16.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.117+0000 7f28acf9eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a777f41c60 0x55a777f46b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:16.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.117+0000 7f28acf9eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a777f4cb50 0x55a777f47090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:16.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.117+0000 7f28acf9eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a777f425c0 con 0x55a777f41c60 2023-07-18T03:16:16.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.117+0000 7f28acf9eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a777f42de0 con 0x55a777f41350 2023-07-18T03:16:16.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.117+0000 7f28acf9eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a777f47e20 con 0x55a777f4cb50 2023-07-18T03:16:16.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.118+0000 7f2892ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a777f41350 0x55a777f53530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:16.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.118+0000 7f28927fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a777f41c60 0x55a777f46b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:16.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.118+0000 7f2892ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a777f41350 0x55a777f53530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:51192/0 (socket says 172.21.15.101:51192) 2023-07-18T03:16:16.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.118+0000 7f2892ffd700 1 -- 172.21.15.101:0/3741825518 learned_addr learned my addr 172.21.15.101:0/3741825518 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:16.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.118+0000 7f28937fe700 1 --2- 172.21.15.101:0/3741825518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a777f4cb50 0x55a777f47090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:16.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.118+0000 7f2892ffd700 1 -- 172.21.15.101:0/3741825518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a777f4cb50 msgr2=0x55a777f47090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:16.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.118+0000 7f2892ffd700 1 --2- 172.21.15.101:0/3741825518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a777f4cb50 0x55a777f47090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.118+0000 7f2892ffd700 1 -- 172.21.15.101:0/3741825518 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a777f41c60 msgr2=0x55a777f46b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:16.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.118+0000 7f2892ffd700 1 --2- 172.21.15.101:0/3741825518 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a777f41c60 0x55a777f46b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.118+0000 7f2892ffd700 1 -- 172.21.15.101:0/3741825518 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a777f39660 con 0x55a777f41350 2023-07-18T03:16:16.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.118+0000 7f28937fe700 1 --2- 172.21.15.101:0/3741825518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a777f4cb50 0x55a777f47090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:16:16.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.118+0000 7f2892ffd700 1 --2- 172.21.15.101:0/3741825518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a777f41350 0x55a777f53530 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f288409a2a0 tx=0x7f288400ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3e093acca7fb13e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:16.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.118+0000 7f2879ffb700 1 -- 172.21.15.101:0/3741825518 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28840b8040 con 0x55a777f41350 2023-07-18T03:16:16.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.118+0000 7f2879ffb700 1 -- 172.21.15.101:0/3741825518 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f288409a0e0 con 0x55a777f41350 2023-07-18T03:16:16.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.118+0000 7f2879ffb700 1 -- 172.21.15.101:0/3741825518 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28840cd420 con 0x55a777f41350 2023-07-18T03:16:16.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.119+0000 7f28acf9eb80 1 -- 172.21.15.101:0/3741825518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a777f41350 msgr2=0x55a777f53530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:16.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.119+0000 7f28acf9eb80 1 --2- 172.21.15.101:0/3741825518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a777f41350 0x55a777f53530 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f288409a2a0 tx=0x7f288400ef90 comp rx=0 tx=0).stop 2023-07-18T03:16:16.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.119+0000 7f28acf9eb80 1 -- 172.21.15.101:0/3741825518 shutdown_connections 2023-07-18T03:16:16.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.119+0000 7f28acf9eb80 1 --2- 172.21.15.101:0/3741825518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a777f41350 0x55a777f53530 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.119+0000 7f28acf9eb80 1 --2- 172.21.15.101:0/3741825518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a777f4cb50 0x55a777f47090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.119+0000 7f28acf9eb80 1 --2- 172.21.15.101:0/3741825518 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a777f41c60 0x55a777f46b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.119+0000 7f28acf9eb80 1 -- 172.21.15.101:0/3741825518 >> 172.21.15.101:0/3741825518 conn(0x55a777f37e20 msgr2=0x55a777f4ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:16.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.119+0000 7f28acf9eb80 1 -- 172.21.15.101:0/3741825518 shutdown_connections 2023-07-18T03:16:16.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.119+0000 7f28acf9eb80 1 -- 172.21.15.101:0/3741825518 wait complete. 2023-07-18T03:16:16.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.120+0000 7f28acf9eb80 1 -- start start 2023-07-18T03:16:16.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.120+0000 7f28acf9eb80 1 Processor -- start 2023-07-18T03:16:16.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.120+0000 7f28acf9eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a777f41350 0x55a777faa540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:16.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.120+0000 7f28acf9eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a777f41c60 0x55a777faaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:16.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.120+0000 7f28acf9eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a777f4cb50 0x55a777fa4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:16.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.120+0000 7f28acf9eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a777fa4f10 con 0x55a777f41c60 2023-07-18T03:16:16.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.120+0000 7f28acf9eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a777fa5080 con 0x55a777f4cb50 2023-07-18T03:16:16.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.120+0000 7f28acf9eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a777fa5240 con 0x55a777f41350 2023-07-18T03:16:16.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.120+0000 7f28acf9eb80 10 client.0 _ll_register_callbacks cb 0x7f2888009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:16:16.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.121+0000 7f2892ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a777f41350 0x55a777faa540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:16.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.121+0000 7f2892ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a777f41350 0x55a777faa540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:35788/0 (socket says 172.21.15.101:35788) 2023-07-18T03:16:16.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.121+0000 7f2892ffd700 1 -- 172.21.15.101:0/3944390267 learned_addr learned my addr 172.21.15.101:0/3944390267 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:16.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.121+0000 7f28937fe700 1 --2- 172.21.15.101:0/3944390267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a777f4cb50 0x55a777fa4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:16.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.121+0000 7f28927fc700 1 --2- 172.21.15.101:0/3944390267 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a777f41c60 0x55a777faaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:16.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.121+0000 7f28927fc700 1 -- 172.21.15.101:0/3944390267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a777f41350 msgr2=0x55a777faa540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:16.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.121+0000 7f28927fc700 1 --2- 172.21.15.101:0/3944390267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a777f41350 0x55a777faa540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.121+0000 7f28927fc700 1 -- 172.21.15.101:0/3944390267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a777f4cb50 msgr2=0x55a777fa4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:16.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.121+0000 7f28927fc700 1 --2- 172.21.15.101:0/3944390267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a777f4cb50 0x55a777fa4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.121+0000 7f28927fc700 1 -- 172.21.15.101:0/3944390267 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a777cc1e60 con 0x55a777f41c60 2023-07-18T03:16:16.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.121+0000 7f2892ffd700 1 --2- 172.21.15.101:0/3944390267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a777f41350 0x55a777faa540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:16:16.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.121+0000 7f28927fc700 1 --2- 172.21.15.101:0/3944390267 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a777f41c60 0x55a777faaa80 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f2888009810 tx=0x7f288800fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad9be7f0f6c5a7e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:16.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.121+0000 7f2876ff5700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:16:16.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.121+0000 7f2876ff5700 1 -- 172.21.15.101:0/3944390267 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288801c040 con 0x55a777f41c60 2023-07-18T03:16:16.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.122+0000 7f28acf9eb80 1 -- 172.21.15.101:0/3944390267 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f288409e340 con 0x55a777f41c60 2023-07-18T03:16:16.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.122+0000 7f2876ff5700 1 -- 172.21.15.101:0/3944390267 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f2888000e40 con 0x55a777f41c60 2023-07-18T03:16:16.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.122+0000 7f2876ff5700 1 -- 172.21.15.101:0/3944390267 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f288800b800 con 0x55a777f41c60 2023-07-18T03:16:16.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.122+0000 7f28acf9eb80 10 client.17328 fetch_fsmap learned FSMap version 31 2023-07-18T03:16:16.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.122+0000 7f28acf9eb80 1 -- 172.21.15.101:0/3944390267 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55a777caace0 con 0x55a777f41c60 2023-07-18T03:16:16.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.122+0000 7f2876ff5700 1 -- 172.21.15.101:0/3944390267 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f288409e340 con 0x55a777f41c60 2023-07-18T03:16:16.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.122+0000 7f2876ff5700 1 -- 172.21.15.101:0/3944390267 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f2888009c70 con 0x55a777f41c60 2023-07-18T03:16:16.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.122+0000 7f28acf9eb80 10 client.17328 fetch_fsmap finished waiting for FSMap version 31 2023-07-18T03:16:16.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.122+0000 7f28acf9eb80 10 client.17328 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:16:16.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.122+0000 7f28acf9eb80 20 client.17328 populate_metadata read hostname 'smithi101' 2023-07-18T03:16:16.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.122+0000 7f28acf9eb80 1 --2- 172.21.15.101:0/3944390267 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a777fb2090 0x55a777fb44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:16.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.122+0000 7f28acf9eb80 4 client.17328 mds_command: new command op to 24562 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-18T03:16:16.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.122+0000 7f28acf9eb80 1 -- 172.21.15.101:0/3944390267 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55a777e4f0d0 con 0x55a777fb2090 2023-07-18T03:16:16.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.123+0000 7f2892ffd700 1 --2- 172.21.15.101:0/3944390267 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a777fb2090 0x55a777fb44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:16.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.123+0000 7f2876ff5700 10 client.17328 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:16:16.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.123+0000 7f2892ffd700 1 --2- 172.21.15.101:0/3944390267 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a777fb2090 0x55a777fb44a0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=254a4b63cc0e5ecf server_cookie=a0cb850e9ff0660 in_seq=0 out_seq=0 2023-07-18T03:16:16.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.124+0000 7f2876ff5700 1 -- 172.21.15.101:0/3944390267 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a777e4f0d0 con 0x55a777fb2090 2023-07-18T03:16:16.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.124+0000 7f2876ff5700 10 client.17328 handle_command_reply: tid=0 2023-07-18T03:16:16.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.124+0000 7f28acf9eb80 1 client.17328 shutdown 2023-07-18T03:16:16.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.124+0000 7f28acf9eb80 1 -- 172.21.15.101:0/3944390267 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a777f41c60 msgr2=0x55a777faaa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:16.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.124+0000 7f28acf9eb80 1 --2- 172.21.15.101:0/3944390267 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a777f41c60 0x55a777faaa80 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f2888009810 tx=0x7f288800fa70 comp rx=0 tx=0).stop 2023-07-18T03:16:16.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.124+0000 7f28acf9eb80 1 -- 172.21.15.101:0/3944390267 shutdown_connections 2023-07-18T03:16:16.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.124+0000 7f28acf9eb80 1 --2- 172.21.15.101:0/3944390267 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a777fb2090 0x55a777fb44a0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.124+0000 7f28acf9eb80 1 --2- 172.21.15.101:0/3944390267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a777f4cb50 0x55a777fa4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.124+0000 7f28acf9eb80 1 --2- 172.21.15.101:0/3944390267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a777f41350 0x55a777faa540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.124+0000 7f28acf9eb80 1 --2- 172.21.15.101:0/3944390267 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a777f41c60 0x55a777faaa80 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.124+0000 7f28acf9eb80 1 -- 172.21.15.101:0/3944390267 >> 172.21.15.101:0/3944390267 conn(0x55a777f37e20 msgr2=0x55a777f3c930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:16.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.124+0000 7f2876ff5700 0 client.17328 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:16:16.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.124+0000 7f28acf9eb80 1 -- 172.21.15.101:0/3944390267 shutdown_connections 2023-07-18T03:16:16.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.124+0000 7f28acf9eb80 1 -- 172.21.15.101:0/3944390267 wait complete. 2023-07-18T03:16:16.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.125+0000 7f28acf9eb80 20 client.17328 trim_cache size 0 max 16384 2023-07-18T03:16:16.130 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:16:16.130 INFO:teuthology.orchestra.run.smithi101.stdout: "return_code": 0, 2023-07-18T03:16:16.130 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_tag": "b77fc20d-ebae-4c6a-8c7b-19dc5a9ce13d", 2023-07-18T03:16:16.131 INFO:teuthology.orchestra.run.smithi101.stdout: "mode": "asynchronous" 2023-07-18T03:16:16.131 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:16:16.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.125+0000 7f28995ff700 1 -- 172.21.15.101:0/1152288738 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f287c0a12c0 msgr2=0x7f287c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:16.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.125+0000 7f28995ff700 1 --2- 172.21.15.101:0/1152288738 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f287c0a12c0 0x7f287c0a3770 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f288c016dd0 tx=0x7f288c016450 comp rx=0 tx=0).stop 2023-07-18T03:16:16.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.125+0000 7f28995ff700 1 -- 172.21.15.101:0/1152288738 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2894072850 msgr2=0x7f2894197e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:16.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.125+0000 7f28995ff700 1 --2- 172.21.15.101:0/1152288738 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2894072850 0x7f2894197e90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f288400ec10 tx=0x7f288400ef20 comp rx=0 tx=0).stop 2023-07-18T03:16:16.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.125+0000 7f28995ff700 1 -- 172.21.15.101:0/1152288738 shutdown_connections 2023-07-18T03:16:16.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.125+0000 7f28995ff700 1 --2- 172.21.15.101:0/1152288738 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f287c0a12c0 0x7f287c0a3770 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.125+0000 7f28995ff700 1 --2- 172.21.15.101:0/1152288738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f289410c3a0 0x7f289419c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.125+0000 7f28995ff700 1 --2- 172.21.15.101:0/1152288738 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2894072850 0x7f2894197e90 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.125+0000 7f28995ff700 1 --2- 172.21.15.101:0/1152288738 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2894073160 0x7f28941983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.125+0000 7f28995ff700 1 -- 172.21.15.101:0/1152288738 >> 172.21.15.101:0/1152288738 conn(0x7f28940fb600 msgr2=0x7f289410e450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:16.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.125+0000 7f28995ff700 1 -- 172.21.15.101:0/1152288738 shutdown_connections 2023-07-18T03:16:16.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.125+0000 7f28995ff700 1 -- 172.21.15.101:0/1152288738 wait complete. 2023-07-18T03:16:16.141 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:16:16.369 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.374+0000 7f572a3ff700 1 -- 172.21.15.101:0/3416536561 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5724073160 msgr2=0x7f572410be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:16.369 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.374+0000 7f572a3ff700 1 --2- 172.21.15.101:0/3416536561 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5724073160 0x7f572410be40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5718009990 tx=0x7f5718009ca0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.369 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.374+0000 7f572a3ff700 1 -- 172.21.15.101:0/3416536561 shutdown_connections 2023-07-18T03:16:16.369 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.374+0000 7f572a3ff700 1 --2- 172.21.15.101:0/3416536561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5724072850 0x7f5724072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.370 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.374+0000 7f572a3ff700 1 --2- 172.21.15.101:0/3416536561 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f572410c380 0x7f572410e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.370 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.374+0000 7f572a3ff700 1 --2- 172.21.15.101:0/3416536561 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5724073160 0x7f572410be40 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.370 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.374+0000 7f572a3ff700 1 -- 172.21.15.101:0/3416536561 >> 172.21.15.101:0/3416536561 conn(0x7f57240fb5e0 msgr2=0x7f57240fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:16.370 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.374+0000 7f572a3ff700 1 -- 172.21.15.101:0/3416536561 shutdown_connections 2023-07-18T03:16:16.370 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.374+0000 7f572a3ff700 1 -- 172.21.15.101:0/3416536561 wait complete. 2023-07-18T03:16:16.372 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.375+0000 7f572a3ff700 1 Processor -- start 2023-07-18T03:16:16.372 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.376+0000 7f572a3ff700 1 -- start start 2023-07-18T03:16:16.372 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.376+0000 7f572a3ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5724072850 0x7f5724197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:16.372 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.376+0000 7f572a3ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5724073160 0x7f5724198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:16.372 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.376+0000 7f572a3ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f572410c380 0x7f572419c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:16.373 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.376+0000 7f572a3ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f572419ce10 con 0x7f572410c380 2023-07-18T03:16:16.373 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.376+0000 7f572a3ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f572419cf80 con 0x7f5724073160 2023-07-18T03:16:16.373 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.376+0000 7f572a3ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f572419d140 con 0x7f5724072850 2023-07-18T03:16:16.373 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.376+0000 7f5723fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5724072850 0x7f5724197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:16.373 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.376+0000 7f5723fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5724072850 0x7f5724197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:35804/0 (socket says 172.21.15.101:35804) 2023-07-18T03:16:16.373 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.376+0000 7f5723fff700 1 -- 172.21.15.101:0/1977130690 learned_addr learned my addr 172.21.15.101:0/1977130690 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:16.373 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.376+0000 7f57237fe700 1 --2- 172.21.15.101:0/1977130690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5724073160 0x7f5724198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:16.374 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.376+0000 7f572899c700 1 --2- 172.21.15.101:0/1977130690 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f572410c380 0x7f572419c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:16.374 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.377+0000 7f5723fff700 1 -- 172.21.15.101:0/1977130690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5724073160 msgr2=0x7f5724198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:16.374 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.377+0000 7f5723fff700 1 --2- 172.21.15.101:0/1977130690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5724073160 0x7f5724198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.374 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.377+0000 7f5723fff700 1 -- 172.21.15.101:0/1977130690 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f572410c380 msgr2=0x7f572419c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:16.374 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.377+0000 7f5723fff700 1 --2- 172.21.15.101:0/1977130690 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f572410c380 0x7f572419c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.374 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.377+0000 7f5723fff700 1 -- 172.21.15.101:0/1977130690 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57180095f0 con 0x7f5724072850 2023-07-18T03:16:16.374 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.377+0000 7f57237fe700 1 --2- 172.21.15.101:0/1977130690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5724073160 0x7f5724198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:16:16.374 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.377+0000 7f5723fff700 1 --2- 172.21.15.101:0/1977130690 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5724072850 0x7f5724197e30 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f571400e9a0 tx=0x7f571400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:16.375 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.377+0000 7f57217fa700 1 -- 172.21.15.101:0/1977130690 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5714009790 con 0x7f5724072850 2023-07-18T03:16:16.375 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.377+0000 7f57217fa700 1 -- 172.21.15.101:0/1977130690 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5714010430 con 0x7f5724072850 2023-07-18T03:16:16.375 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.377+0000 7f572a3ff700 1 -- 172.21.15.101:0/1977130690 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f572419d420 con 0x7f5724072850 2023-07-18T03:16:16.375 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.377+0000 7f57217fa700 1 -- 172.21.15.101:0/1977130690 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5714018680 con 0x7f5724072850 2023-07-18T03:16:16.375 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.377+0000 7f572a3ff700 1 -- 172.21.15.101:0/1977130690 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57241a4c60 con 0x7f5724072850 2023-07-18T03:16:16.375 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.380+0000 7f57217fa700 1 -- 172.21.15.101:0/1977130690 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f57140105a0 con 0x7f5724072850 2023-07-18T03:16:16.376 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.382+0000 7f57217fa700 1 --2- 172.21.15.101:0/1977130690 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f570c0a12c0 0x7f570c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:16.376 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.382+0000 7f57217fa700 1 -- 172.21.15.101:0/1977130690 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f5714014070 con 0x7f5724072850 2023-07-18T03:16:16.377 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.382+0000 7f573dd88b80 1 Processor -- start 2023-07-18T03:16:16.377 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.382+0000 7f573dd88b80 1 -- start start 2023-07-18T03:16:16.378 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.383+0000 7f573dd88b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4977122d0 0x55c4977126a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:16.378 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.383+0000 7f573dd88b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c497712be0 0x55c49771d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:16.378 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.383+0000 7f57237fe700 1 --2- 172.21.15.101:0/1977130690 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f570c0a12c0 0x7f570c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:16.379 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.383+0000 7f573dd88b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c49771dad0 0x55c49771feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:16.379 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.383+0000 7f573dd88b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c497618cb0 con 0x55c497712be0 2023-07-18T03:16:16.379 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.383+0000 7f573dd88b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c49766ba40 con 0x55c4977122d0 2023-07-18T03:16:16.379 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.383+0000 7f573dd88b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c497664ed0 con 0x55c49771dad0 2023-07-18T03:16:16.379 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.383+0000 7f5723fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4977122d0 0x55c4977126a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:16.379 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.383+0000 7f5723fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4977122d0 0x55c4977126a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:51228/0 (socket says 172.21.15.101:51228) 2023-07-18T03:16:16.379 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.383+0000 7f5723fff700 1 -- 172.21.15.101:0/841132894 learned_addr learned my addr 172.21.15.101:0/841132894 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:16.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.383+0000 7f572899c700 1 --2- 172.21.15.101:0/841132894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c49771dad0 0x55c49771feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:16.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.383+0000 7f57237fe700 1 --2- 172.21.15.101:0/841132894 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c497712be0 0x55c49771d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:16.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.383+0000 7f5723fff700 1 -- 172.21.15.101:0/841132894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c49771dad0 msgr2=0x55c49771feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:16.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.383+0000 7f5723fff700 1 --2- 172.21.15.101:0/841132894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c49771dad0 0x55c49771feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.383+0000 7f5723fff700 1 -- 172.21.15.101:0/841132894 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c497712be0 msgr2=0x55c49771d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:16.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.383+0000 7f5723fff700 1 --2- 172.21.15.101:0/841132894 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c497712be0 0x55c49771d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.384+0000 7f5723fff700 1 -- 172.21.15.101:0/841132894 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c497492de0 con 0x55c4977122d0 2023-07-18T03:16:16.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.384+0000 7f5723fff700 1 --2- 172.21.15.101:0/841132894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4977122d0 0x55c4977126a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f57140a1ed0 tx=0x7f5714099fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3212882b750a55bc server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:16.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.384+0000 7f57237fe700 1 --2- 172.21.15.101:0/1977130690 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f570c0a12c0 0x7f570c0a3770 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5724199440 tx=0x7f571800bf00 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:16.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.384+0000 7f570a7fc700 1 -- 172.21.15.101:0/841132894 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57140a2400 con 0x55c4977122d0 2023-07-18T03:16:16.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.384+0000 7f570a7fc700 1 -- 172.21.15.101:0/841132894 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5714010880 con 0x55c4977122d0 2023-07-18T03:16:16.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.384+0000 7f570a7fc700 1 -- 172.21.15.101:0/841132894 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5714099730 con 0x55c4977122d0 2023-07-18T03:16:16.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.385+0000 7f573dd88b80 1 -- 172.21.15.101:0/841132894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4977122d0 msgr2=0x55c4977126a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:16.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.385+0000 7f573dd88b80 1 --2- 172.21.15.101:0/841132894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4977122d0 0x55c4977126a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f57140a1ed0 tx=0x7f5714099fb0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.385+0000 7f573dd88b80 1 -- 172.21.15.101:0/841132894 shutdown_connections 2023-07-18T03:16:16.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.385+0000 7f573dd88b80 1 --2- 172.21.15.101:0/841132894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4977122d0 0x55c4977126a0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.385+0000 7f573dd88b80 1 --2- 172.21.15.101:0/841132894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c49771dad0 0x55c49771feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.385+0000 7f573dd88b80 1 --2- 172.21.15.101:0/841132894 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c497712be0 0x55c49771d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.385+0000 7f573dd88b80 1 -- 172.21.15.101:0/841132894 >> 172.21.15.101:0/841132894 conn(0x55c497708da0 msgr2=0x55c49770e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:16.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.385+0000 7f573dd88b80 1 -- 172.21.15.101:0/841132894 shutdown_connections 2023-07-18T03:16:16.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.385+0000 7f573dd88b80 1 -- 172.21.15.101:0/841132894 wait complete. 2023-07-18T03:16:16.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.386+0000 7f573dd88b80 1 -- start start 2023-07-18T03:16:16.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.386+0000 7f573dd88b80 1 Processor -- start 2023-07-18T03:16:16.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.386+0000 7f573dd88b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4977122d0 0x55c49771a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:16.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.386+0000 7f573dd88b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c497712be0 0x55c49771ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:16.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.386+0000 7f573dd88b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c49771dad0 0x55c49771b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:16.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.386+0000 7f573dd88b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c497604ac0 con 0x55c497712be0 2023-07-18T03:16:16.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.386+0000 7f573dd88b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4976047c0 con 0x55c49771dad0 2023-07-18T03:16:16.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.386+0000 7f573dd88b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4974e8920 con 0x55c4977122d0 2023-07-18T03:16:16.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.386+0000 7f573dd88b80 10 client.0 _ll_register_callbacks cb 0x55c497602760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:16:16.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.387+0000 7f5723fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4977122d0 0x55c49771a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:16.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.387+0000 7f5723fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4977122d0 0x55c49771a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:35824/0 (socket says 172.21.15.101:35824) 2023-07-18T03:16:16.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.387+0000 7f5723fff700 1 -- 172.21.15.101:0/1340256169 learned_addr learned my addr 172.21.15.101:0/1340256169 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:16.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.387+0000 7f572899c700 1 --2- 172.21.15.101:0/1340256169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c49771dad0 0x55c49771b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:16.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.387+0000 7f57237fe700 1 --2- 172.21.15.101:0/1340256169 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c497712be0 0x55c49771ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:16.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.387+0000 7f5723fff700 1 -- 172.21.15.101:0/1340256169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c49771dad0 msgr2=0x55c49771b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:16.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.387+0000 7f5723fff700 1 --2- 172.21.15.101:0/1340256169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c49771dad0 0x55c49771b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.387+0000 7f5723fff700 1 -- 172.21.15.101:0/1340256169 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c497712be0 msgr2=0x55c49771ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:16.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.387+0000 7f5723fff700 1 --2- 172.21.15.101:0/1340256169 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c497712be0 0x55c49771ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.387+0000 7f5723fff700 1 -- 172.21.15.101:0/1340256169 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57241a4c60 con 0x55c4977122d0 2023-07-18T03:16:16.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.387+0000 7f5723fff700 1 --2- 172.21.15.101:0/1340256169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4977122d0 0x55c49771a7e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f57140ca6e0 tx=0x7f5714099750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74830b65bcb83d3e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:16.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.387+0000 7f57037fe700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:16:16.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.387+0000 7f57037fe700 1 -- 172.21.15.101:0/1340256169 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57140a2400 con 0x55c4977122d0 2023-07-18T03:16:16.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.387+0000 7f573dd88b80 1 -- 172.21.15.101:0/1340256169 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c4975be4f0 con 0x55c4977122d0 2023-07-18T03:16:16.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.387+0000 7f57037fe700 1 -- 172.21.15.101:0/1340256169 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f57140a2180 con 0x55c4977122d0 2023-07-18T03:16:16.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.388+0000 7f57037fe700 1 -- 172.21.15.101:0/1340256169 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5714094bc0 con 0x55c4977122d0 2023-07-18T03:16:16.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.388+0000 7f573dd88b80 10 client.27415 fetch_fsmap learned FSMap version 31 2023-07-18T03:16:16.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.388+0000 7f573dd88b80 1 -- 172.21.15.101:0/1340256169 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55c49747bc60 con 0x55c4977122d0 2023-07-18T03:16:16.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.388+0000 7f57037fe700 1 -- 172.21.15.101:0/1340256169 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57140c8080 con 0x55c4977122d0 2023-07-18T03:16:16.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.388+0000 7f57037fe700 1 -- 172.21.15.101:0/1340256169 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f5714099110 con 0x55c4977122d0 2023-07-18T03:16:16.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.388+0000 7f573dd88b80 10 client.27415 fetch_fsmap finished waiting for FSMap version 31 2023-07-18T03:16:16.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.388+0000 7f573dd88b80 10 client.27415 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:16:16.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.388+0000 7f573dd88b80 20 client.27415 populate_metadata read hostname 'smithi101' 2023-07-18T03:16:16.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.388+0000 7f573dd88b80 1 --2- 172.21.15.101:0/1340256169 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c497735810 0x55c497737c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:16.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.388+0000 7f573dd88b80 4 client.27415 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:16:16.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.388+0000 7f573dd88b80 1 -- 172.21.15.101:0/1340256169 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c49765deb0 con 0x55c497735810 2023-07-18T03:16:16.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.388+0000 7f57237fe700 1 --2- 172.21.15.101:0/1340256169 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c497735810 0x55c497737c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:16.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.389+0000 7f57037fe700 10 client.27415 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:16:16.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.389+0000 7f57237fe700 1 --2- 172.21.15.101:0/1340256169 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c497735810 0x55c497737c20 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17c7c66e9a4a417c server_cookie=2a797512ee053f5c in_seq=0 out_seq=0 2023-07-18T03:16:16.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.390+0000 7f57037fe700 1 -- 172.21.15.101:0/1340256169 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c49765deb0 con 0x55c497735810 2023-07-18T03:16:16.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.390+0000 7f57037fe700 10 client.27415 handle_command_reply: tid=0 2023-07-18T03:16:16.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.390+0000 7f573dd88b80 1 client.27415 shutdown 2023-07-18T03:16:16.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.390+0000 7f573dd88b80 1 -- 172.21.15.101:0/1340256169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4977122d0 msgr2=0x55c49771a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:16.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.390+0000 7f573dd88b80 1 --2- 172.21.15.101:0/1340256169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4977122d0 0x55c49771a7e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f57140ca6e0 tx=0x7f5714099750 comp rx=0 tx=0).stop 2023-07-18T03:16:16.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.390+0000 7f573dd88b80 1 -- 172.21.15.101:0/1340256169 shutdown_connections 2023-07-18T03:16:16.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.390+0000 7f573dd88b80 1 --2- 172.21.15.101:0/1340256169 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c497735810 0x55c497737c20 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.390+0000 7f573dd88b80 1 --2- 172.21.15.101:0/1340256169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c49771dad0 0x55c49771b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.390+0000 7f573dd88b80 1 --2- 172.21.15.101:0/1340256169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4977122d0 0x55c49771a7e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.390+0000 7f573dd88b80 1 --2- 172.21.15.101:0/1340256169 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c497712be0 0x55c49771ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.390+0000 7f573dd88b80 1 -- 172.21.15.101:0/1340256169 >> 172.21.15.101:0/1340256169 conn(0x55c497708da0 msgr2=0x55c49770e230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:16.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.390+0000 7f57037fe700 0 client.27415 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:16:16.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.390+0000 7f573dd88b80 1 -- 172.21.15.101:0/1340256169 shutdown_connections 2023-07-18T03:16:16.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.390+0000 7f573dd88b80 1 -- 172.21.15.101:0/1340256169 wait complete. 2023-07-18T03:16:16.394 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.391+0000 7f573dd88b80 20 client.27415 trim_cache size 0 max 16384 2023-07-18T03:16:16.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.416+0000 7f573dd88b80 1 Processor -- start 2023-07-18T03:16:16.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.416+0000 7f573dd88b80 1 -- start start 2023-07-18T03:16:16.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.416+0000 7f573dd88b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4977122d0 0x55c4977244a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:16.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.416+0000 7f573dd88b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c497712be0 0x55c49771b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:16.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.416+0000 7f573dd88b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c49771dad0 0x55c497717a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:16.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.416+0000 7f573dd88b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c497715680 con 0x55c4977122d0 2023-07-18T03:16:16.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.416+0000 7f573dd88b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c497708970 con 0x55c497712be0 2023-07-18T03:16:16.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.416+0000 7f573dd88b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c497713160 con 0x55c49771dad0 2023-07-18T03:16:16.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.417+0000 7f572899c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c49771dad0 0x55c497717a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:16.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.417+0000 7f572899c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c49771dad0 0x55c497717a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:35840/0 (socket says 172.21.15.101:35840) 2023-07-18T03:16:16.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.417+0000 7f572899c700 1 -- 172.21.15.101:0/1814779768 learned_addr learned my addr 172.21.15.101:0/1814779768 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:16.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.417+0000 7f57237fe700 1 --2- 172.21.15.101:0/1814779768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c497712be0 0x55c49771b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:16.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.417+0000 7f5723fff700 1 --2- 172.21.15.101:0/1814779768 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4977122d0 0x55c4977244a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:16.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.417+0000 7f57237fe700 1 -- 172.21.15.101:0/1814779768 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c49771dad0 msgr2=0x55c497717a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:16.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.417+0000 7f57237fe700 1 --2- 172.21.15.101:0/1814779768 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c49771dad0 0x55c497717a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.417+0000 7f57237fe700 1 -- 172.21.15.101:0/1814779768 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4977122d0 msgr2=0x55c4977244a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:16.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.417+0000 7f57237fe700 1 --2- 172.21.15.101:0/1814779768 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4977122d0 0x55c4977244a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.417+0000 7f57237fe700 1 -- 172.21.15.101:0/1814779768 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c49770a5e0 con 0x55c497712be0 2023-07-18T03:16:16.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.417+0000 7f57237fe700 1 --2- 172.21.15.101:0/1814779768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c497712be0 0x55c49771b090 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f571800b2e0 tx=0x7f5718027000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9a08fb351f52ffc server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:16.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.418+0000 7f570affd700 1 -- 172.21.15.101:0/1814779768 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571801d070 con 0x55c497712be0 2023-07-18T03:16:16.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.418+0000 7f570affd700 1 -- 172.21.15.101:0/1814779768 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5718004e20 con 0x55c497712be0 2023-07-18T03:16:16.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.418+0000 7f570affd700 1 -- 172.21.15.101:0/1814779768 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57180363c0 con 0x55c497712be0 2023-07-18T03:16:16.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.418+0000 7f573dd88b80 1 -- 172.21.15.101:0/1814779768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c497712be0 msgr2=0x55c49771b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:16.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.418+0000 7f573dd88b80 1 --2- 172.21.15.101:0/1814779768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c497712be0 0x55c49771b090 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f571800b2e0 tx=0x7f5718027000 comp rx=0 tx=0).stop 2023-07-18T03:16:16.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.418+0000 7f573dd88b80 1 -- 172.21.15.101:0/1814779768 shutdown_connections 2023-07-18T03:16:16.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.418+0000 7f573dd88b80 1 --2- 172.21.15.101:0/1814779768 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c49771dad0 0x55c497717a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.418+0000 7f573dd88b80 1 --2- 172.21.15.101:0/1814779768 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4977122d0 0x55c4977244a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.418+0000 7f573dd88b80 1 --2- 172.21.15.101:0/1814779768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c497712be0 0x55c49771b090 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.419+0000 7f573dd88b80 1 -- 172.21.15.101:0/1814779768 >> 172.21.15.101:0/1814779768 conn(0x55c497708da0 msgr2=0x55c497715b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:16.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.419+0000 7f573dd88b80 1 -- 172.21.15.101:0/1814779768 shutdown_connections 2023-07-18T03:16:16.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.419+0000 7f573dd88b80 1 -- 172.21.15.101:0/1814779768 wait complete. 2023-07-18T03:16:16.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.419+0000 7f573dd88b80 1 -- start start 2023-07-18T03:16:16.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.419+0000 7f573dd88b80 1 Processor -- start 2023-07-18T03:16:16.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.420+0000 7f573dd88b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4977122d0 0x55c4977242e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:16.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.420+0000 7f573dd88b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c497712be0 0x55c497722610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:16.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.420+0000 7f573dd88b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c49771dad0 0x55c497722b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:16.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.420+0000 7f573dd88b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c497723090 con 0x55c497712be0 2023-07-18T03:16:16.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.420+0000 7f573dd88b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c497723200 con 0x55c49771dad0 2023-07-18T03:16:16.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.420+0000 7f573dd88b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4977233c0 con 0x55c4977122d0 2023-07-18T03:16:16.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.420+0000 7f573dd88b80 10 client.0 _ll_register_callbacks cb 0x55c497602760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:16:16.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.420+0000 7f5723fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4977122d0 0x55c4977242e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:16.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.420+0000 7f5723fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4977122d0 0x55c4977242e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:35842/0 (socket says 172.21.15.101:35842) 2023-07-18T03:16:16.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.420+0000 7f5723fff700 1 -- 172.21.15.101:0/2245949399 learned_addr learned my addr 172.21.15.101:0/2245949399 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:16.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.420+0000 7f572899c700 1 --2- 172.21.15.101:0/2245949399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c49771dad0 0x55c497722b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:16.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.420+0000 7f57237fe700 1 --2- 172.21.15.101:0/2245949399 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c497712be0 0x55c497722610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:16.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.420+0000 7f5723fff700 1 -- 172.21.15.101:0/2245949399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c49771dad0 msgr2=0x55c497722b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:16.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.420+0000 7f5723fff700 1 --2- 172.21.15.101:0/2245949399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c49771dad0 0x55c497722b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.420+0000 7f5723fff700 1 -- 172.21.15.101:0/2245949399 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c497712be0 msgr2=0x55c497722610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:16.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.420+0000 7f5723fff700 1 --2- 172.21.15.101:0/2245949399 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c497712be0 0x55c497722610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.420+0000 7f5723fff700 1 -- 172.21.15.101:0/2245949399 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5718020040 con 0x55c4977122d0 2023-07-18T03:16:16.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.420+0000 7f57237fe700 1 --2- 172.21.15.101:0/2245949399 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c497712be0 0x55c497722610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:16:16.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.421+0000 7f5723fff700 1 --2- 172.21.15.101:0/2245949399 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4977122d0 0x55c4977242e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f57140964b0 tx=0x7f5714099c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bbb134e9ef61478 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:16.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.421+0000 7f57037fe700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:16:16.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.421+0000 7f57037fe700 1 -- 172.21.15.101:0/2245949399 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57140ca180 con 0x55c4977122d0 2023-07-18T03:16:16.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.421+0000 7f57037fe700 1 -- 172.21.15.101:0/2245949399 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f57140a2400 con 0x55c4977122d0 2023-07-18T03:16:16.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.421+0000 7f573dd88b80 1 -- 172.21.15.101:0/2245949399 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5714010880 con 0x55c4977122d0 2023-07-18T03:16:16.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.421+0000 7f57037fe700 1 -- 172.21.15.101:0/2245949399 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57140962f0 con 0x55c4977122d0 2023-07-18T03:16:16.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.421+0000 7f573dd88b80 10 client.27421 fetch_fsmap learned FSMap version 31 2023-07-18T03:16:16.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.421+0000 7f573dd88b80 1 -- 172.21.15.101:0/2245949399 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55c4974b9250 con 0x55c4977122d0 2023-07-18T03:16:16.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.421+0000 7f57037fe700 1 -- 172.21.15.101:0/2245949399 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5714010880 con 0x55c4977122d0 2023-07-18T03:16:16.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.421+0000 7f57037fe700 1 -- 172.21.15.101:0/2245949399 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f571400ca10 con 0x55c4977122d0 2023-07-18T03:16:16.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.421+0000 7f573dd88b80 10 client.27421 fetch_fsmap finished waiting for FSMap version 31 2023-07-18T03:16:16.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.421+0000 7f573dd88b80 10 client.27421 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:16:16.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.421+0000 7f573dd88b80 20 client.27421 populate_metadata read hostname 'smithi101' 2023-07-18T03:16:16.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.422+0000 7f573dd88b80 1 --2- 172.21.15.101:0/2245949399 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c49778a6b0 0x55c49778cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:16.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.422+0000 7f573dd88b80 4 client.27421 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:16:16.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.422+0000 7f573dd88b80 1 -- 172.21.15.101:0/2245949399 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c497620050 con 0x55c49778a6b0 2023-07-18T03:16:16.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.422+0000 7f57237fe700 1 --2- 172.21.15.101:0/2245949399 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c49778a6b0 0x55c49778cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:16.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.422+0000 7f57037fe700 10 client.27421 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:16:16.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.422+0000 7f57237fe700 1 --2- 172.21.15.101:0/2245949399 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c49778a6b0 0x55c49778cac0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5218668bb78861a1 server_cookie=ad01b8be640ba138 in_seq=0 out_seq=0 2023-07-18T03:16:16.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.422+0000 7f57037fe700 1 -- 172.21.15.101:0/2245949399 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c497620050 con 0x55c49778a6b0 2023-07-18T03:16:16.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.422+0000 7f57037fe700 10 client.27421 handle_command_reply: tid=0 2023-07-18T03:16:16.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.422+0000 7f573dd88b80 1 client.27421 shutdown 2023-07-18T03:16:16.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.423+0000 7f573dd88b80 1 -- 172.21.15.101:0/2245949399 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4977122d0 msgr2=0x55c4977242e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:16.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.423+0000 7f573dd88b80 1 --2- 172.21.15.101:0/2245949399 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4977122d0 0x55c4977242e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f57140964b0 tx=0x7f5714099c20 comp rx=0 tx=0).stop 2023-07-18T03:16:16.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.423+0000 7f573dd88b80 1 -- 172.21.15.101:0/2245949399 shutdown_connections 2023-07-18T03:16:16.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.423+0000 7f573dd88b80 1 --2- 172.21.15.101:0/2245949399 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c49778a6b0 0x55c49778cac0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.423+0000 7f573dd88b80 1 --2- 172.21.15.101:0/2245949399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c49771dad0 0x55c497722b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.423+0000 7f573dd88b80 1 --2- 172.21.15.101:0/2245949399 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4977122d0 0x55c4977242e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.423+0000 7f573dd88b80 1 --2- 172.21.15.101:0/2245949399 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c497712be0 0x55c497722610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.423+0000 7f573dd88b80 1 -- 172.21.15.101:0/2245949399 >> 172.21.15.101:0/2245949399 conn(0x55c497708da0 msgr2=0x55c49771afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:16.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.423+0000 7f57037fe700 0 client.27421 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:16:16.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.423+0000 7f573dd88b80 1 -- 172.21.15.101:0/2245949399 shutdown_connections 2023-07-18T03:16:16.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.423+0000 7f573dd88b80 1 -- 172.21.15.101:0/2245949399 wait complete. 2023-07-18T03:16:16.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.423+0000 7f573dd88b80 20 client.27421 trim_cache size 0 max 16384 2023-07-18T03:16:16.426 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:16:16.426 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:16:16.426 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:16:16.427 INFO:teuthology.orchestra.run.smithi101.stdout: "b77fc20d-ebae-4c6a-8c7b-19dc5a9ce13d": { 2023-07-18T03:16:16.427 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:16:16.427 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "b77fc20d-ebae-4c6a-8c7b-19dc5a9ce13d", 2023-07-18T03:16:16.427 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:16:16.427 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:16:16.427 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:16:16.427 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:16:16.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.424+0000 7f572a3ff700 1 -- 172.21.15.101:0/1977130690 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f570c0a12c0 msgr2=0x7f570c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:16.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.424+0000 7f572a3ff700 1 --2- 172.21.15.101:0/1977130690 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f570c0a12c0 0x7f570c0a3770 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5724199440 tx=0x7f571800bf00 comp rx=0 tx=0).stop 2023-07-18T03:16:16.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.424+0000 7f572a3ff700 1 -- 172.21.15.101:0/1977130690 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5724072850 msgr2=0x7f5724197e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:16.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.424+0000 7f572a3ff700 1 --2- 172.21.15.101:0/1977130690 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5724072850 0x7f5724197e30 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f571400e9a0 tx=0x7f571400ecb0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.424+0000 7f572a3ff700 1 -- 172.21.15.101:0/1977130690 shutdown_connections 2023-07-18T03:16:16.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.424+0000 7f572a3ff700 1 --2- 172.21.15.101:0/1977130690 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f572410c380 0x7f572419c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.424+0000 7f572a3ff700 1 --2- 172.21.15.101:0/1977130690 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5724072850 0x7f5724197e30 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.424+0000 7f572a3ff700 1 --2- 172.21.15.101:0/1977130690 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f570c0a12c0 0x7f570c0a3770 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.424+0000 7f572a3ff700 1 --2- 172.21.15.101:0/1977130690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5724073160 0x7f5724198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:16.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.424+0000 7f572a3ff700 1 -- 172.21.15.101:0/1977130690 >> 172.21.15.101:0/1977130690 conn(0x7f57240fb5e0 msgr2=0x7f572410e420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:16.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.424+0000 7f572a3ff700 1 -- 172.21.15.101:0/1977130690 shutdown_connections 2023-07-18T03:16:16.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:16.424+0000 7f572a3ff700 1 -- 172.21.15.101:0/1977130690 wait complete. 2023-07-18T03:16:16.436 INFO:tasks.cephfs.filesystem:scrub status for tag:b77fc20d-ebae-4c6a-8c7b-19dc5a9ce13d - {'path': '/', 'tag': 'b77fc20d-ebae-4c6a-8c7b-19dc5a9ce13d', 'options': 'recursive,force'} 2023-07-18T03:16:16.601 INFO:tasks.workunit.client.0.smithi101.stdout: 1 580085 6.35 MB/sec execute 341 sec latency 21.978 ms 2023-07-18T03:16:17.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:16 smithi101 ceph-mon[126379]: pgmap v270: 129 pgs: 129 active+clean; 423 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 133 op/s 2023-07-18T03:16:17.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:16 smithi101 ceph-mon[126379]: scrub queued for path: / 2023-07-18T03:16:17.239 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:16 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:16:17.239 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:16 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:16:17.239 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:16 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:16:17.239 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:16 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:16:17.240 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:16 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:16:17.240 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:16 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:16:17.240 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:16 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:16:17.240 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:16 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:16:17.240 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:16 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:16:17.240 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:16 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:16:17.241 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:16 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:16:17.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:16 smithi155 ceph-mon[128885]: pgmap v270: 129 pgs: 129 active+clean; 423 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 133 op/s 2023-07-18T03:16:17.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:16 smithi155 ceph-mon[128885]: scrub queued for path: / 2023-07-18T03:16:17.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:16 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:16:17.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:16 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:16:17.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:16 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:16:17.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:16 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:16:17.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:16 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:16:17.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:16 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:16:17.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:16 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:16:17.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:16 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:16:17.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:16 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:16:17.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:16 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:16:17.298 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:16 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:16:17.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:16 smithi179 ceph-mon[129229]: pgmap v270: 129 pgs: 129 active+clean; 423 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 133 op/s 2023-07-18T03:16:17.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:16 smithi179 ceph-mon[129229]: scrub queued for path: / 2023-07-18T03:16:17.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:16 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:16:17.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:16 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:16:17.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:16 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:16:17.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:16 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:16:17.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:16 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:16:17.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:16 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:16:17.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:16 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:16:17.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:16 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:16:17.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:16 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:16:17.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:16 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:16:17.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:16 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:16:17.601 INFO:tasks.workunit.client.0.smithi101.stdout: 1 580669 6.34 MB/sec execute 342 sec latency 597.928 ms 2023-07-18T03:16:18.601 INFO:tasks.workunit.client.0.smithi101.stdout: 1 580669 6.33 MB/sec execute 343 sec latency 1598.085 ms 2023-07-18T03:16:19.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:18 smithi101 ceph-mon[126379]: pgmap v271: 129 pgs: 129 active+clean; 423 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 128 op/s 2023-07-18T03:16:19.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:18 smithi155 ceph-mon[128885]: pgmap v271: 129 pgs: 129 active+clean; 423 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 128 op/s 2023-07-18T03:16:19.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:18 smithi179 ceph-mon[129229]: pgmap v271: 129 pgs: 129 active+clean; 423 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 128 op/s 2023-07-18T03:16:19.601 INFO:tasks.workunit.client.0.smithi101.stdout: 1 581117 6.31 MB/sec execute 344 sec latency 2423.807 ms 2023-07-18T03:16:20.602 INFO:tasks.workunit.client.0.smithi101.stdout: 1 582766 6.33 MB/sec execute 345 sec latency 25.453 ms 2023-07-18T03:16:21.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:20 smithi101 ceph-mon[126379]: pgmap v272: 129 pgs: 129 active+clean; 428 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 134 op/s 2023-07-18T03:16:21.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:20 smithi155 ceph-mon[128885]: pgmap v272: 129 pgs: 129 active+clean; 428 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 134 op/s 2023-07-18T03:16:21.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:20 smithi179 ceph-mon[129229]: pgmap v272: 129 pgs: 129 active+clean; 428 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 134 op/s 2023-07-18T03:16:21.602 INFO:tasks.workunit.client.0.smithi101.stdout: 1 583911 6.33 MB/sec execute 346 sec latency 16.816 ms 2023-07-18T03:16:22.602 INFO:tasks.workunit.client.0.smithi101.stdout: 1 584222 6.31 MB/sec execute 347 sec latency 750.799 ms 2023-07-18T03:16:23.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:22 smithi101 ceph-mon[126379]: pgmap v273: 129 pgs: 129 active+clean; 429 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 133 op/s 2023-07-18T03:16:23.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:22 smithi155 ceph-mon[128885]: pgmap v273: 129 pgs: 129 active+clean; 429 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 133 op/s 2023-07-18T03:16:23.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:22 smithi179 ceph-mon[129229]: pgmap v273: 129 pgs: 129 active+clean; 429 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 133 op/s 2023-07-18T03:16:23.602 INFO:tasks.workunit.client.0.smithi101.stdout: 1 584222 6.29 MB/sec execute 348 sec latency 1750.937 ms 2023-07-18T03:16:24.602 INFO:tasks.workunit.client.0.smithi101.stdout: 1 584622 6.27 MB/sec execute 349 sec latency 2591.810 ms 2023-07-18T03:16:25.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:24 smithi101 ceph-mon[126379]: pgmap v274: 129 pgs: 129 active+clean; 429 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 133 op/s 2023-07-18T03:16:25.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:24 smithi155 ceph-mon[128885]: pgmap v274: 129 pgs: 129 active+clean; 429 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 133 op/s 2023-07-18T03:16:25.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:24 smithi179 ceph-mon[129229]: pgmap v274: 129 pgs: 129 active+clean; 429 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 133 op/s 2023-07-18T03:16:25.603 INFO:tasks.workunit.client.0.smithi101.stdout: 1 586288 6.29 MB/sec execute 350 sec latency 54.327 ms 2023-07-18T03:16:26.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:25 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:16:26.239 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:25 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:16:26.239 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:25 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:16:26.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:25 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:16:26.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:25 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:16:26.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:25 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:16:26.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:25 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:16:26.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:25 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:16:26.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:25 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:16:26.604 INFO:tasks.workunit.client.0.smithi101.stdout: 1 587115 6.28 MB/sec execute 351 sec latency 22.177 ms 2023-07-18T03:16:27.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:26 smithi101 ceph-mon[126379]: pgmap v275: 129 pgs: 129 active+clean; 434 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 165 op/s 2023-07-18T03:16:27.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:26 smithi155 ceph-mon[128885]: pgmap v275: 129 pgs: 129 active+clean; 434 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 165 op/s 2023-07-18T03:16:27.450 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:26 smithi179 ceph-mon[129229]: pgmap v275: 129 pgs: 129 active+clean; 434 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 165 op/s 2023-07-18T03:16:27.604 INFO:tasks.workunit.client.0.smithi101.stdout: 1 587763 6.27 MB/sec execute 352 sec latency 537.117 ms 2023-07-18T03:16:28.604 INFO:tasks.workunit.client.0.smithi101.stdout: 1 587763 6.25 MB/sec execute 353 sec latency 1537.237 ms 2023-07-18T03:16:29.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:28 smithi101 ceph-mon[126379]: pgmap v276: 129 pgs: 129 active+clean; 434 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 108 op/s 2023-07-18T03:16:29.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:28 smithi155 ceph-mon[128885]: pgmap v276: 129 pgs: 129 active+clean; 434 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 108 op/s 2023-07-18T03:16:29.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:28 smithi179 ceph-mon[129229]: pgmap v276: 129 pgs: 129 active+clean; 434 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 108 op/s 2023-07-18T03:16:29.604 INFO:tasks.workunit.client.0.smithi101.stdout: 1 588211 6.24 MB/sec execute 354 sec latency 2363.746 ms 2023-07-18T03:16:30.604 INFO:tasks.workunit.client.0.smithi101.stdout: 1 589834 6.26 MB/sec execute 355 sec latency 15.816 ms 2023-07-18T03:16:31.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:30 smithi101 ceph-mon[126379]: pgmap v277: 129 pgs: 129 active+clean; 437 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 158 op/s 2023-07-18T03:16:31.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:30 smithi155 ceph-mon[128885]: pgmap v277: 129 pgs: 129 active+clean; 437 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 158 op/s 2023-07-18T03:16:31.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:30 smithi179 ceph-mon[129229]: pgmap v277: 129 pgs: 129 active+clean; 437 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 158 op/s 2023-07-18T03:16:31.604 INFO:tasks.workunit.client.0.smithi101.stdout: 1 590692 6.25 MB/sec execute 356 sec latency 28.387 ms 2023-07-18T03:16:32.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:31 smithi155 ceph-mon[128885]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:16:32.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:31 smithi179 ceph-mon[129229]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:16:32.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:31 smithi101 ceph-mon[126379]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:16:32.604 INFO:tasks.workunit.client.0.smithi101.stdout: 1 591312 6.24 MB/sec execute 357 sec latency 565.832 ms 2023-07-18T03:16:33.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:32 smithi155 ceph-mon[128885]: pgmap v278: 129 pgs: 129 active+clean; 438 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 111 op/s 2023-07-18T03:16:33.493 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:32 smithi101 ceph-mon[126379]: pgmap v278: 129 pgs: 129 active+clean; 438 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 111 op/s 2023-07-18T03:16:33.494 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:32 smithi179 ceph-mon[129229]: pgmap v278: 129 pgs: 129 active+clean; 438 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 111 op/s 2023-07-18T03:16:33.604 INFO:tasks.workunit.client.0.smithi101.stdout: 1 591312 6.22 MB/sec execute 358 sec latency 1565.876 ms 2023-07-18T03:16:34.604 INFO:tasks.workunit.client.0.smithi101.stdout: 1 591762 6.20 MB/sec execute 359 sec latency 2387.691 ms 2023-07-18T03:16:35.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:35 smithi155 ceph-mon[128885]: pgmap v279: 129 pgs: 129 active+clean; 438 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 101 op/s 2023-07-18T03:16:35.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:35 smithi179 ceph-mon[129229]: pgmap v279: 129 pgs: 129 active+clean; 438 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 101 op/s 2023-07-18T03:16:35.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:35 smithi101 ceph-mon[126379]: pgmap v279: 129 pgs: 129 active+clean; 438 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 101 op/s 2023-07-18T03:16:35.604 INFO:tasks.workunit.client.0.smithi101.stdout: 1 593418 6.22 MB/sec execute 360 sec latency 15.864 ms 2023-07-18T03:16:36.605 INFO:tasks.workunit.client.0.smithi101.stdout: 1 594334 6.21 MB/sec execute 361 sec latency 38.458 ms 2023-07-18T03:16:37.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:37 smithi155 ceph-mon[128885]: pgmap v280: 129 pgs: 129 active+clean; 442 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 148 op/s 2023-07-18T03:16:37.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:37 smithi155 ceph-mon[128885]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:16:37.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:37 smithi179 ceph-mon[129229]: pgmap v280: 129 pgs: 129 active+clean; 442 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 148 op/s 2023-07-18T03:16:37.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:37 smithi179 ceph-mon[129229]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:16:37.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:37 smithi101 ceph-mon[126379]: pgmap v280: 129 pgs: 129 active+clean; 442 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 148 op/s 2023-07-18T03:16:37.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:37 smithi101 ceph-mon[126379]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:16:37.605 INFO:tasks.workunit.client.0.smithi101.stdout: 1 594856 6.21 MB/sec execute 362 sec latency 380.858 ms 2023-07-18T03:16:38.606 INFO:tasks.workunit.client.0.smithi101.stdout: 1 594856 6.19 MB/sec execute 363 sec latency 1380.939 ms 2023-07-18T03:16:39.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:39 smithi155 ceph-mon[128885]: pgmap v281: 129 pgs: 129 active+clean; 442 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 98 op/s 2023-07-18T03:16:39.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:39 smithi179 ceph-mon[129229]: pgmap v281: 129 pgs: 129 active+clean; 442 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 98 op/s 2023-07-18T03:16:39.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:39 smithi101 ceph-mon[126379]: pgmap v281: 129 pgs: 129 active+clean; 442 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 98 op/s 2023-07-18T03:16:39.605 INFO:tasks.workunit.client.0.smithi101.stdout: 1 595008 6.17 MB/sec execute 364 sec latency 2205.827 ms 2023-07-18T03:16:40.605 INFO:tasks.workunit.client.0.smithi101.stdout: 1 596921 6.19 MB/sec execute 365 sec latency 15.675 ms 2023-07-18T03:16:41.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:41 smithi155 ceph-mon[128885]: pgmap v282: 129 pgs: 129 active+clean; 446 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 147 op/s 2023-07-18T03:16:41.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:41 smithi179 ceph-mon[129229]: pgmap v282: 129 pgs: 129 active+clean; 446 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 147 op/s 2023-07-18T03:16:41.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:41 smithi101 ceph-mon[126379]: pgmap v282: 129 pgs: 129 active+clean; 446 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 147 op/s 2023-07-18T03:16:41.607 INFO:tasks.workunit.client.0.smithi101.stdout: 1 597867 6.18 MB/sec execute 366 sec latency 24.360 ms 2023-07-18T03:16:42.605 INFO:tasks.workunit.client.0.smithi101.stdout: 1 598403 6.17 MB/sec execute 367 sec latency 577.678 ms 2023-07-18T03:16:43.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:43 smithi155 ceph-mon[128885]: pgmap v283: 129 pgs: 129 active+clean; 445 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 98 op/s 2023-07-18T03:16:43.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:43 smithi179 ceph-mon[129229]: pgmap v283: 129 pgs: 129 active+clean; 445 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 98 op/s 2023-07-18T03:16:43.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:43 smithi101 ceph-mon[126379]: pgmap v283: 129 pgs: 129 active+clean; 445 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 98 op/s 2023-07-18T03:16:43.606 INFO:tasks.workunit.client.0.smithi101.stdout: 1 598403 6.16 MB/sec execute 368 sec latency 1577.797 ms 2023-07-18T03:16:44.606 INFO:tasks.workunit.client.0.smithi101.stdout: 1 598803 6.14 MB/sec execute 369 sec latency 2402.033 ms 2023-07-18T03:16:45.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:45 smithi155 ceph-mon[128885]: pgmap v284: 129 pgs: 129 active+clean; 445 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 96 op/s 2023-07-18T03:16:45.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:45 smithi179 ceph-mon[129229]: pgmap v284: 129 pgs: 129 active+clean; 445 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 96 op/s 2023-07-18T03:16:45.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:45 smithi101 ceph-mon[126379]: pgmap v284: 129 pgs: 129 active+clean; 445 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 96 op/s 2023-07-18T03:16:45.606 INFO:tasks.workunit.client.0.smithi101.stdout: 1 600557 6.16 MB/sec execute 370 sec latency 13.948 ms 2023-07-18T03:16:46.438 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:16:46.606 INFO:tasks.workunit.client.0.smithi101.stdout: 1 601640 6.16 MB/sec execute 371 sec latency 22.036 ms 2023-07-18T03:16:46.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.689+0000 7efd97bff700 1 -- 172.21.15.101:0/1098509529 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd90100d00 msgr2=0x7efd901010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:46.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.689+0000 7efd97bff700 1 --2- 172.21.15.101:0/1098509529 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd90100d00 0x7efd901010d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7efd8c009990 tx=0x7efd8c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:16:46.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.689+0000 7efd97bff700 1 -- 172.21.15.101:0/1098509529 shutdown_connections 2023-07-18T03:16:46.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.689+0000 7efd97bff700 1 --2- 172.21.15.101:0/1098509529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd9010a040 0x7efd9010c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:46.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.689+0000 7efd97bff700 1 --2- 172.21.15.101:0/1098509529 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd90100d00 0x7efd901010d0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:46.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.689+0000 7efd97bff700 1 --2- 172.21.15.101:0/1098509529 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efd90101610 0x7efd90109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:46.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.689+0000 7efd97bff700 1 -- 172.21.15.101:0/1098509529 >> 172.21.15.101:0/1098509529 conn(0x7efd900fa4b0 msgr2=0x7efd900fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:46.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.689+0000 7efd97bff700 1 -- 172.21.15.101:0/1098509529 shutdown_connections 2023-07-18T03:16:46.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.689+0000 7efd97bff700 1 -- 172.21.15.101:0/1098509529 wait complete. 2023-07-18T03:16:46.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.690+0000 7efd97bff700 1 Processor -- start 2023-07-18T03:16:46.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.690+0000 7efd97bff700 1 -- start start 2023-07-18T03:16:46.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.691+0000 7efd97bff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd90100d00 0x7efd9019ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:46.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.691+0000 7efd97bff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd90101610 0x7efd9019bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:46.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.691+0000 7efd97bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efd9010a040 0x7efd90195a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:46.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.691+0000 7efd97bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd901960e0 con 0x7efd9010a040 2023-07-18T03:16:46.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.691+0000 7efd97bff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd90196250 con 0x7efd90101610 2023-07-18T03:16:46.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.691+0000 7efd97bff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd90196410 con 0x7efd90100d00 2023-07-18T03:16:46.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.691+0000 7efd9599b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd90100d00 0x7efd9019ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:46.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.691+0000 7efd9599b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd90100d00 0x7efd9019ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:47384/0 (socket says 172.21.15.101:47384) 2023-07-18T03:16:46.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.691+0000 7efd9599b700 1 -- 172.21.15.101:0/2229745111 learned_addr learned my addr 172.21.15.101:0/2229745111 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:46.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.691+0000 7efd9619c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efd9010a040 0x7efd90195a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:46.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.691+0000 7efd9519a700 1 --2- 172.21.15.101:0/2229745111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd90101610 0x7efd9019bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:46.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.691+0000 7efd9599b700 1 -- 172.21.15.101:0/2229745111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd90101610 msgr2=0x7efd9019bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:46.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.691+0000 7efd9599b700 1 --2- 172.21.15.101:0/2229745111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd90101610 0x7efd9019bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:46.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.691+0000 7efd9599b700 1 -- 172.21.15.101:0/2229745111 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efd9010a040 msgr2=0x7efd90195a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:46.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.691+0000 7efd9599b700 1 --2- 172.21.15.101:0/2229745111 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efd9010a040 0x7efd90195a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:46.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.691+0000 7efd9599b700 1 -- 172.21.15.101:0/2229745111 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd8c0095f0 con 0x7efd90100d00 2023-07-18T03:16:46.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.692+0000 7efd9599b700 1 --2- 172.21.15.101:0/2229745111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd90100d00 0x7efd9019ba00 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7efd8c005910 tx=0x7efd8c005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:46.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.692+0000 7efd86ffd700 1 -- 172.21.15.101:0/2229745111 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8c01d070 con 0x7efd90100d00 2023-07-18T03:16:46.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.692+0000 7efd86ffd700 1 -- 172.21.15.101:0/2229745111 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd8c022470 con 0x7efd90100d00 2023-07-18T03:16:46.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.692+0000 7efd86ffd700 1 -- 172.21.15.101:0/2229745111 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8c017650 con 0x7efd90100d00 2023-07-18T03:16:46.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.692+0000 7efd97bff700 1 -- 172.21.15.101:0/2229745111 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd90196690 con 0x7efd90100d00 2023-07-18T03:16:46.690 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.692+0000 7efd97bff700 1 -- 172.21.15.101:0/2229745111 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd901a29f0 con 0x7efd90100d00 2023-07-18T03:16:46.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.695+0000 7efd86ffd700 1 -- 172.21.15.101:0/2229745111 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7efd8c0225e0 con 0x7efd90100d00 2023-07-18T03:16:46.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.696+0000 7efdab5e6b80 1 Processor -- start 2023-07-18T03:16:46.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.697+0000 7efdab5e6b80 1 -- start start 2023-07-18T03:16:46.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.697+0000 7efd86ffd700 1 --2- 172.21.15.101:0/2229745111 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7efd7c0a1340 0x7efd7c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:46.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.697+0000 7efd86ffd700 1 -- 172.21.15.101:0/2229745111 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9461+0+0 (secure 0 0 0) 0x7efd8c0ce380 con 0x7efd90100d00 2023-07-18T03:16:46.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.697+0000 7efd9619c700 1 --2- 172.21.15.101:0/2229745111 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7efd7c0a1340 0x7efd7c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:46.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.697+0000 7efdab5e6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56374518d2d0 0x56374518d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:46.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.697+0000 7efdab5e6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56374518dbe0 0x56374519c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:46.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.697+0000 7efdab5e6b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56374519cee0 0x56374519f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:46.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.697+0000 7efdab5e6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563745093cb0 con 0x56374518d2d0 2023-07-18T03:16:46.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.697+0000 7efdab5e6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637450e6a40 con 0x56374518dbe0 2023-07-18T03:16:46.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.697+0000 7efdab5e6b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637450dfed0 con 0x56374519cee0 2023-07-18T03:16:46.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.698+0000 7efd9599b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56374518d2d0 0x56374518d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:46.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.698+0000 7efd9519a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56374518dbe0 0x56374519c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:46.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.698+0000 7efd9519a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56374518dbe0 0x56374519c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60100/0 (socket says 172.21.15.101:60100) 2023-07-18T03:16:46.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.698+0000 7efd9519a700 1 -- 172.21.15.101:0/3880443009 learned_addr learned my addr 172.21.15.101:0/3880443009 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:46.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.698+0000 7efd9599b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56374518d2d0 0x56374518d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:50374/0 (socket says 172.21.15.101:50374) 2023-07-18T03:16:46.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.698+0000 7efd9519a700 1 -- 172.21.15.101:0/3880443009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56374519cee0 msgr2=0x56374519f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:46.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.698+0000 7efd9519a700 1 --2- 172.21.15.101:0/3880443009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56374519cee0 0x56374519f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:46.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.698+0000 7efd9519a700 1 -- 172.21.15.101:0/3880443009 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56374518d2d0 msgr2=0x56374518d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:46.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.698+0000 7efd9519a700 1 --2- 172.21.15.101:0/3880443009 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56374518d2d0 0x56374518d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:46.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.698+0000 7efd9619c700 1 --2- 172.21.15.101:0/2229745111 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7efd7c0a1340 0x7efd7c0a37f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efd9019c700 tx=0x7efd8800b040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:46.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.698+0000 7efd9519a700 1 -- 172.21.15.101:0/3880443009 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563744f0dde0 con 0x56374518dbe0 2023-07-18T03:16:46.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.698+0000 7efd9519a700 1 --2- 172.21.15.101:0/3880443009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56374518dbe0 0x56374519c9a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7efd80009d20 tx=0x7efd80011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5cbc3f9d2336287 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:46.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.698+0000 7efd7bfff700 1 -- 172.21.15.101:0/3880443009 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd80019630 con 0x56374518dbe0 2023-07-18T03:16:46.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.698+0000 7efd7bfff700 1 -- 172.21.15.101:0/3880443009 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd80019c50 con 0x56374518dbe0 2023-07-18T03:16:46.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.698+0000 7efd7bfff700 1 -- 172.21.15.101:0/3880443009 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8000ec40 con 0x56374518dbe0 2023-07-18T03:16:46.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.699+0000 7efdab5e6b80 1 -- 172.21.15.101:0/3880443009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56374518dbe0 msgr2=0x56374519c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:46.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.699+0000 7efdab5e6b80 1 --2- 172.21.15.101:0/3880443009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56374518dbe0 0x56374519c9a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7efd80009d20 tx=0x7efd80011720 comp rx=0 tx=0).stop 2023-07-18T03:16:46.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.699+0000 7efdab5e6b80 1 -- 172.21.15.101:0/3880443009 shutdown_connections 2023-07-18T03:16:46.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.699+0000 7efdab5e6b80 1 --2- 172.21.15.101:0/3880443009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56374519cee0 0x56374519f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:46.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.699+0000 7efdab5e6b80 1 --2- 172.21.15.101:0/3880443009 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56374518d2d0 0x56374518d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:46.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.699+0000 7efdab5e6b80 1 --2- 172.21.15.101:0/3880443009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56374518dbe0 0x56374519c9a0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:46.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.699+0000 7efdab5e6b80 1 -- 172.21.15.101:0/3880443009 >> 172.21.15.101:0/3880443009 conn(0x563745183da0 msgr2=0x5637451896a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:46.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.699+0000 7efdab5e6b80 1 -- 172.21.15.101:0/3880443009 shutdown_connections 2023-07-18T03:16:46.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.699+0000 7efdab5e6b80 1 -- 172.21.15.101:0/3880443009 wait complete. 2023-07-18T03:16:46.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.700+0000 7efdab5e6b80 1 -- start start 2023-07-18T03:16:46.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.700+0000 7efdab5e6b80 1 Processor -- start 2023-07-18T03:16:46.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.701+0000 7efdab5e6b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56374518d2d0 0x56374519c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:46.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.701+0000 7efdab5e6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56374518dbe0 0x563745199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:46.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.701+0000 7efdab5e6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56374519cee0 0x56374519a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:46.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.701+0000 7efdab5e6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56374507fac0 con 0x56374519cee0 2023-07-18T03:16:46.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.701+0000 7efdab5e6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56374507f7c0 con 0x56374518dbe0 2023-07-18T03:16:46.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.701+0000 7efdab5e6b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563744f63920 con 0x56374518d2d0 2023-07-18T03:16:46.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.701+0000 7efdab5e6b80 10 client.0 _ll_register_callbacks cb 0x56374507d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:16:46.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.701+0000 7efd9599b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56374518d2d0 0x56374519c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:46.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.701+0000 7efd9599b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56374518d2d0 0x56374519c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:47400/0 (socket says 172.21.15.101:47400) 2023-07-18T03:16:46.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.701+0000 7efd9599b700 1 -- 172.21.15.101:0/1639068361 learned_addr learned my addr 172.21.15.101:0/1639068361 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:46.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.701+0000 7efd9519a700 1 --2- 172.21.15.101:0/1639068361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56374518dbe0 0x563745199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:46.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.701+0000 7efd9619c700 1 --2- 172.21.15.101:0/1639068361 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56374519cee0 0x56374519a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:46.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.702+0000 7efd9599b700 1 -- 172.21.15.101:0/1639068361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56374518dbe0 msgr2=0x563745199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:46.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.702+0000 7efd9599b700 1 --2- 172.21.15.101:0/1639068361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56374518dbe0 0x563745199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:46.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.702+0000 7efd9599b700 1 -- 172.21.15.101:0/1639068361 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56374519cee0 msgr2=0x56374519a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:46.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.702+0000 7efd9599b700 1 --2- 172.21.15.101:0/1639068361 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56374519cee0 0x56374519a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:46.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.702+0000 7efd9599b700 1 -- 172.21.15.101:0/1639068361 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd80009980 con 0x56374518d2d0 2023-07-18T03:16:46.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.702+0000 7efd9519a700 1 --2- 172.21.15.101:0/1639068361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56374518dbe0 0x563745199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:16:46.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.702+0000 7efd9599b700 1 --2- 172.21.15.101:0/1639068361 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56374518d2d0 0x56374519c8e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7efd8c000c00 tx=0x7efd8c09ac10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff29a74f92ac24ff server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:46.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.702+0000 7efd797fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:16:46.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.702+0000 7efd797fa700 1 -- 172.21.15.101:0/1639068361 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8c0b2070 con 0x56374518d2d0 2023-07-18T03:16:46.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.702+0000 7efd797fa700 1 -- 172.21.15.101:0/1639068361 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9461+0+0 (secure 0 0 0) 0x7efd8c0ce150 con 0x56374518d2d0 2023-07-18T03:16:46.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.702+0000 7efdab5e6b80 1 -- 172.21.15.101:0/1639068361 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5637450394f0 con 0x56374518d2d0 2023-07-18T03:16:46.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.702+0000 7efd797fa700 1 -- 172.21.15.101:0/1639068361 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd8c09b0d0 con 0x56374518d2d0 2023-07-18T03:16:46.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.702+0000 7efdab5e6b80 10 client.27430 fetch_fsmap learned FSMap version 33 2023-07-18T03:16:46.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.702+0000 7efdab5e6b80 1 -- 172.21.15.101:0/1639068361 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x563744ef6c60 con 0x56374518d2d0 2023-07-18T03:16:46.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.702+0000 7efd797fa700 1 -- 172.21.15.101:0/1639068361 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd8c0c9020 con 0x56374518d2d0 2023-07-18T03:16:46.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.703+0000 7efd797fa700 1 -- 172.21.15.101:0/1639068361 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efd8c0a3000 con 0x56374518d2d0 2023-07-18T03:16:46.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.703+0000 7efdab5e6b80 10 client.27430 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:16:46.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.703+0000 7efdab5e6b80 10 client.27430 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:16:46.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.703+0000 7efdab5e6b80 20 client.27430 populate_metadata read hostname 'smithi101' 2023-07-18T03:16:46.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.703+0000 7efdab5e6b80 1 --2- 172.21.15.101:0/1639068361 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5637451ac4b0 0x5637451ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:46.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.703+0000 7efdab5e6b80 4 client.27430 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:16:46.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.703+0000 7efdab5e6b80 1 -- 172.21.15.101:0/1639068361 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5637450d8eb0 con 0x5637451ac4b0 2023-07-18T03:16:46.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.703+0000 7efd9519a700 1 --2- 172.21.15.101:0/1639068361 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5637451ac4b0 0x5637451ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:46.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.704+0000 7efd797fa700 10 client.27430 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:16:46.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.704+0000 7efd9519a700 1 --2- 172.21.15.101:0/1639068361 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5637451ac4b0 0x5637451ae8c0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=197aac6da3b27d96 server_cookie=a0dfdee30fc748d3 in_seq=0 out_seq=0 2023-07-18T03:16:46.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.704+0000 7efd797fa700 1 -- 172.21.15.101:0/1639068361 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5637450d8eb0 con 0x5637451ac4b0 2023-07-18T03:16:46.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.704+0000 7efd797fa700 10 client.27430 handle_command_reply: tid=0 2023-07-18T03:16:46.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.704+0000 7efdab5e6b80 1 client.27430 shutdown 2023-07-18T03:16:46.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.705+0000 7efdab5e6b80 1 -- 172.21.15.101:0/1639068361 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56374518d2d0 msgr2=0x56374519c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:46.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.705+0000 7efdab5e6b80 1 --2- 172.21.15.101:0/1639068361 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56374518d2d0 0x56374519c8e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7efd8c000c00 tx=0x7efd8c09ac10 comp rx=0 tx=0).stop 2023-07-18T03:16:46.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.705+0000 7efdab5e6b80 1 -- 172.21.15.101:0/1639068361 shutdown_connections 2023-07-18T03:16:46.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.705+0000 7efdab5e6b80 1 --2- 172.21.15.101:0/1639068361 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5637451ac4b0 0x5637451ae8c0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:46.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.705+0000 7efdab5e6b80 1 --2- 172.21.15.101:0/1639068361 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56374519cee0 0x56374519a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:46.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.705+0000 7efd797fa700 0 client.27430 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:16:46.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.705+0000 7efdab5e6b80 1 --2- 172.21.15.101:0/1639068361 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56374518d2d0 0x56374519c8e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:46.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.705+0000 7efdab5e6b80 1 --2- 172.21.15.101:0/1639068361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56374518dbe0 0x563745199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:46.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.705+0000 7efdab5e6b80 1 -- 172.21.15.101:0/1639068361 >> 172.21.15.101:0/1639068361 conn(0x563745183da0 msgr2=0x563745189290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:46.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.705+0000 7efdab5e6b80 1 -- 172.21.15.101:0/1639068361 shutdown_connections 2023-07-18T03:16:46.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.705+0000 7efdab5e6b80 1 -- 172.21.15.101:0/1639068361 wait complete. 2023-07-18T03:16:46.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.705+0000 7efdab5e6b80 20 client.27430 trim_cache size 0 max 16384 2023-07-18T03:16:46.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.726+0000 7efdab5e6b80 1 Processor -- start 2023-07-18T03:16:46.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.726+0000 7efdab5e6b80 1 -- start start 2023-07-18T03:16:46.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.726+0000 7efdab5e6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56374518d2d0 0x56374519c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:46.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.726+0000 7efdab5e6b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56374518dbe0 0x5637451993e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:46.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.727+0000 7efdab5e6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56374519cee0 0x563745199920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:46.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.727+0000 7efdab5e6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563745181010 con 0x56374519cee0 2023-07-18T03:16:46.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.727+0000 7efdab5e6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563745190280 con 0x56374518d2d0 2023-07-18T03:16:46.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.727+0000 7efdab5e6b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637451839a0 con 0x56374518dbe0 2023-07-18T03:16:46.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.727+0000 7efd9519a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56374518dbe0 0x5637451993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:46.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.727+0000 7efd9599b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56374518d2d0 0x56374519c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:46.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.727+0000 7efd9519a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56374518dbe0 0x5637451993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:47402/0 (socket says 172.21.15.101:47402) 2023-07-18T03:16:46.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.727+0000 7efd9599b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56374518d2d0 0x56374519c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60114/0 (socket says 172.21.15.101:60114) 2023-07-18T03:16:46.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.727+0000 7efd9519a700 1 -- 172.21.15.101:0/3080813388 learned_addr learned my addr 172.21.15.101:0/3080813388 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:46.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.727+0000 7efd9619c700 1 --2- 172.21.15.101:0/3080813388 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56374519cee0 0x563745199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:46.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.727+0000 7efd9519a700 1 -- 172.21.15.101:0/3080813388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56374518d2d0 msgr2=0x56374519c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:46.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.727+0000 7efd9519a700 1 --2- 172.21.15.101:0/3080813388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56374518d2d0 0x56374519c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:46.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.727+0000 7efd9519a700 1 -- 172.21.15.101:0/3080813388 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56374519cee0 msgr2=0x563745199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:46.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.727+0000 7efd9519a700 1 --2- 172.21.15.101:0/3080813388 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56374519cee0 0x563745199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:46.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.727+0000 7efd9519a700 1 -- 172.21.15.101:0/3080813388 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563744f34250 con 0x56374518dbe0 2023-07-18T03:16:46.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.727+0000 7efd9519a700 1 --2- 172.21.15.101:0/3080813388 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56374518dbe0 0x5637451993e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7efd80009d20 tx=0x7efd800058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d802586fa43db4 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:46.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.728+0000 7efd84bff700 1 -- 172.21.15.101:0/3080813388 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8000ec90 con 0x56374518dbe0 2023-07-18T03:16:46.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.728+0000 7efd84bff700 1 -- 172.21.15.101:0/3080813388 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd8002a040 con 0x56374518dbe0 2023-07-18T03:16:46.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.728+0000 7efd84bff700 1 -- 172.21.15.101:0/3080813388 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8002aca0 con 0x56374518dbe0 2023-07-18T03:16:46.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.728+0000 7efdab5e6b80 1 -- 172.21.15.101:0/3080813388 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56374518dbe0 msgr2=0x5637451993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:46.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.728+0000 7efdab5e6b80 1 --2- 172.21.15.101:0/3080813388 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56374518dbe0 0x5637451993e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7efd80009d20 tx=0x7efd800058e0 comp rx=0 tx=0).stop 2023-07-18T03:16:46.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.729+0000 7efdab5e6b80 1 -- 172.21.15.101:0/3080813388 shutdown_connections 2023-07-18T03:16:46.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.729+0000 7efdab5e6b80 1 --2- 172.21.15.101:0/3080813388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56374518d2d0 0x56374519c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:46.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.729+0000 7efdab5e6b80 1 --2- 172.21.15.101:0/3080813388 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56374519cee0 0x563745199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:46.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.729+0000 7efdab5e6b80 1 --2- 172.21.15.101:0/3080813388 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56374518dbe0 0x5637451993e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:46.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.729+0000 7efdab5e6b80 1 -- 172.21.15.101:0/3080813388 >> 172.21.15.101:0/3080813388 conn(0x563745183da0 msgr2=0x563745194f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:46.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.729+0000 7efdab5e6b80 1 -- 172.21.15.101:0/3080813388 shutdown_connections 2023-07-18T03:16:46.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.729+0000 7efdab5e6b80 1 -- 172.21.15.101:0/3080813388 wait complete. 2023-07-18T03:16:46.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.729+0000 7efdab5e6b80 1 -- start start 2023-07-18T03:16:46.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.729+0000 7efdab5e6b80 1 Processor -- start 2023-07-18T03:16:46.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.730+0000 7efdab5e6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56374518d2d0 0x5637451a31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:46.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.730+0000 7efdab5e6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56374518dbe0 0x5637451a3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:46.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.730+0000 7efdab5e6b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56374519cee0 0x5637451a1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:46.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.730+0000 7efdab5e6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637451a1c70 con 0x56374518dbe0 2023-07-18T03:16:46.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.730+0000 7efdab5e6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637451a1de0 con 0x56374518d2d0 2023-07-18T03:16:46.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.730+0000 7efdab5e6b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637451a1fa0 con 0x56374519cee0 2023-07-18T03:16:46.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.730+0000 7efdab5e6b80 10 client.0 _ll_register_callbacks cb 0x7efd8c097410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:16:46.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.730+0000 7efd9519a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56374518dbe0 0x5637451a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:46.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.730+0000 7efd9519a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56374518dbe0 0x5637451a3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:50408/0 (socket says 172.21.15.101:50408) 2023-07-18T03:16:46.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.730+0000 7efd9519a700 1 -- 172.21.15.101:0/1278285458 learned_addr learned my addr 172.21.15.101:0/1278285458 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:46.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.730+0000 7efd9619c700 1 --2- 172.21.15.101:0/1278285458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56374519cee0 0x5637451a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:46.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.730+0000 7efd9599b700 1 --2- 172.21.15.101:0/1278285458 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56374518d2d0 0x5637451a31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:46.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.730+0000 7efd9519a700 1 -- 172.21.15.101:0/1278285458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56374519cee0 msgr2=0x5637451a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:46.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.730+0000 7efd9519a700 1 --2- 172.21.15.101:0/1278285458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56374519cee0 0x5637451a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:46.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.730+0000 7efd9519a700 1 -- 172.21.15.101:0/1278285458 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56374518d2d0 msgr2=0x5637451a31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:46.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.730+0000 7efd9519a700 1 --2- 172.21.15.101:0/1278285458 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56374518d2d0 0x5637451a31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:46.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.730+0000 7efd9519a700 1 -- 172.21.15.101:0/1278285458 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563744f0dde0 con 0x56374518dbe0 2023-07-18T03:16:46.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.730+0000 7efd9599b700 1 --2- 172.21.15.101:0/1278285458 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56374518d2d0 0x5637451a31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:16:46.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.731+0000 7efd9519a700 1 --2- 172.21.15.101:0/1278285458 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56374518dbe0 0x5637451a3710 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7efd80000c00 tx=0x7efd80005790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=294e41b57c3f8ca2 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:46.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.731+0000 7efd797fa700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:16:46.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.731+0000 7efd797fa700 1 -- 172.21.15.101:0/1278285458 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8002a3b0 con 0x56374518dbe0 2023-07-18T03:16:46.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.731+0000 7efdab5e6b80 1 -- 172.21.15.101:0/1278285458 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efd80019c50 con 0x56374518dbe0 2023-07-18T03:16:46.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.731+0000 7efd797fa700 1 -- 172.21.15.101:0/1278285458 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9461+0+0 (secure 0 0 0) 0x7efd8000e6c0 con 0x56374518dbe0 2023-07-18T03:16:46.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.731+0000 7efd797fa700 1 -- 172.21.15.101:0/1278285458 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd8000eea0 con 0x56374518dbe0 2023-07-18T03:16:46.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.731+0000 7efdab5e6b80 10 client.17361 fetch_fsmap learned FSMap version 33 2023-07-18T03:16:46.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.731+0000 7efdab5e6b80 1 -- 172.21.15.101:0/1278285458 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x563744f0c2b0 con 0x56374518dbe0 2023-07-18T03:16:46.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.731+0000 7efd797fa700 1 -- 172.21.15.101:0/1278285458 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd80019c50 con 0x56374518dbe0 2023-07-18T03:16:46.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.732+0000 7efd797fa700 1 -- 172.21.15.101:0/1278285458 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efd8002a780 con 0x56374518dbe0 2023-07-18T03:16:46.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.732+0000 7efdab5e6b80 10 client.17361 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:16:46.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.732+0000 7efdab5e6b80 10 client.17361 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:16:46.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.732+0000 7efdab5e6b80 20 client.17361 populate_metadata read hostname 'smithi101' 2023-07-18T03:16:46.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.732+0000 7efdab5e6b80 1 --2- 172.21.15.101:0/1278285458 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563745201210 0x563745203620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:46.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.732+0000 7efdab5e6b80 4 client.17361 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:16:46.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.732+0000 7efdab5e6b80 1 -- 172.21.15.101:0/1278285458 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56374509b050 con 0x563745201210 2023-07-18T03:16:46.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.732+0000 7efd9599b700 1 --2- 172.21.15.101:0/1278285458 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563745201210 0x563745203620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:46.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.732+0000 7efd9599b700 1 --2- 172.21.15.101:0/1278285458 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563745201210 0x563745203620 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ee9507833a108d2 server_cookie=3836e649d7b72c5a in_seq=0 out_seq=0 2023-07-18T03:16:46.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.732+0000 7efd797fa700 10 client.17361 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:16:46.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.733+0000 7efd797fa700 1 -- 172.21.15.101:0/1278285458 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56374509b050 con 0x563745201210 2023-07-18T03:16:46.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.733+0000 7efd797fa700 10 client.17361 handle_command_reply: tid=0 2023-07-18T03:16:46.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.733+0000 7efdab5e6b80 1 client.17361 shutdown 2023-07-18T03:16:46.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.733+0000 7efdab5e6b80 1 -- 172.21.15.101:0/1278285458 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56374518dbe0 msgr2=0x5637451a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:46.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.733+0000 7efdab5e6b80 1 --2- 172.21.15.101:0/1278285458 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56374518dbe0 0x5637451a3710 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7efd80000c00 tx=0x7efd80005790 comp rx=0 tx=0).stop 2023-07-18T03:16:46.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.733+0000 7efdab5e6b80 1 -- 172.21.15.101:0/1278285458 shutdown_connections 2023-07-18T03:16:46.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.733+0000 7efdab5e6b80 1 --2- 172.21.15.101:0/1278285458 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563745201210 0x563745203620 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:46.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.733+0000 7efdab5e6b80 1 --2- 172.21.15.101:0/1278285458 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56374518d2d0 0x5637451a31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:46.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.733+0000 7efdab5e6b80 1 --2- 172.21.15.101:0/1278285458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56374519cee0 0x5637451a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:46.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.733+0000 7efdab5e6b80 1 --2- 172.21.15.101:0/1278285458 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56374518dbe0 0x5637451a3710 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:46.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.733+0000 7efd797fa700 0 client.17361 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:16:46.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.733+0000 7efdab5e6b80 1 -- 172.21.15.101:0/1278285458 >> 172.21.15.101:0/1278285458 conn(0x563745183da0 msgr2=0x56374518f240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:46.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.734+0000 7efdab5e6b80 1 -- 172.21.15.101:0/1278285458 shutdown_connections 2023-07-18T03:16:46.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.734+0000 7efdab5e6b80 1 -- 172.21.15.101:0/1278285458 wait complete. 2023-07-18T03:16:46.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.734+0000 7efdab5e6b80 20 client.17361 trim_cache size 0 max 16384 2023-07-18T03:16:46.737 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:16:46.737 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "no active scrubs running", 2023-07-18T03:16:46.737 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": {} 2023-07-18T03:16:46.737 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:16:46.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.734+0000 7efd97bff700 1 -- 172.21.15.101:0/2229745111 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7efd7c0a1340 msgr2=0x7efd7c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:46.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.734+0000 7efd97bff700 1 --2- 172.21.15.101:0/2229745111 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7efd7c0a1340 0x7efd7c0a37f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efd9019c700 tx=0x7efd8800b040 comp rx=0 tx=0).stop 2023-07-18T03:16:46.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.734+0000 7efd97bff700 1 -- 172.21.15.101:0/2229745111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd90100d00 msgr2=0x7efd9019ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:46.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.734+0000 7efd97bff700 1 --2- 172.21.15.101:0/2229745111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd90100d00 0x7efd9019ba00 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7efd8c005910 tx=0x7efd8c005720 comp rx=0 tx=0).stop 2023-07-18T03:16:46.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.734+0000 7efd97bff700 1 -- 172.21.15.101:0/2229745111 shutdown_connections 2023-07-18T03:16:46.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.734+0000 7efd97bff700 1 --2- 172.21.15.101:0/2229745111 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efd9010a040 0x7efd90195a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:46.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.734+0000 7efd97bff700 1 --2- 172.21.15.101:0/2229745111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd90100d00 0x7efd9019ba00 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:46.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.734+0000 7efd97bff700 1 --2- 172.21.15.101:0/2229745111 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7efd7c0a1340 0x7efd7c0a37f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:46.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.734+0000 7efd97bff700 1 --2- 172.21.15.101:0/2229745111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd90101610 0x7efd9019bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:46.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.734+0000 7efd97bff700 1 -- 172.21.15.101:0/2229745111 >> 172.21.15.101:0/2229745111 conn(0x7efd900fa4b0 msgr2=0x7efd9010c290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:46.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.735+0000 7efd97bff700 1 -- 172.21.15.101:0/2229745111 shutdown_connections 2023-07-18T03:16:46.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.735+0000 7efd97bff700 1 -- 172.21.15.101:0/2229745111 wait complete. 2023-07-18T03:16:46.746 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-18T03:16:46.747 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:16:46.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.966+0000 7fd5459ff700 1 -- 172.21.15.101:0/2399323893 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5400728b0 msgr2=0x7fd540072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:46.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.966+0000 7fd5459ff700 1 --2- 172.21.15.101:0/2399323893 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5400728b0 0x7fd540072c80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd530009990 tx=0x7fd530009ca0 comp rx=0 tx=0).stop 2023-07-18T03:16:46.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.966+0000 7fd5459ff700 1 -- 172.21.15.101:0/2399323893 shutdown_connections 2023-07-18T03:16:46.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.966+0000 7fd5459ff700 1 --2- 172.21.15.101:0/2399323893 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd54010c310 0x7fd54010e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:46.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.966+0000 7fd5459ff700 1 --2- 172.21.15.101:0/2399323893 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5400728b0 0x7fd540072c80 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:46.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.966+0000 7fd5459ff700 1 --2- 172.21.15.101:0/2399323893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5400731c0 0x7fd54010bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:46.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.966+0000 7fd5459ff700 1 -- 172.21.15.101:0/2399323893 >> 172.21.15.101:0/2399323893 conn(0x7fd5400fb590 msgr2=0x7fd5400fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:46.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.966+0000 7fd5459ff700 1 -- 172.21.15.101:0/2399323893 shutdown_connections 2023-07-18T03:16:46.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.966+0000 7fd5459ff700 1 -- 172.21.15.101:0/2399323893 wait complete. 2023-07-18T03:16:46.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.967+0000 7fd5459ff700 1 Processor -- start 2023-07-18T03:16:46.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.968+0000 7fd5459ff700 1 -- start start 2023-07-18T03:16:46.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.968+0000 7fd5459ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5400728b0 0x7fd540197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:46.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.968+0000 7fd5459ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd5400731c0 0x7fd540198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:46.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.968+0000 7fd5459ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd54010c310 0x7fd54019c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:46.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.968+0000 7fd5459ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54019cdf0 con 0x7fd5400731c0 2023-07-18T03:16:46.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.968+0000 7fd5459ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54019cf60 con 0x7fd54010c310 2023-07-18T03:16:46.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.968+0000 7fd5459ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54019d120 con 0x7fd5400728b0 2023-07-18T03:16:46.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.968+0000 7fd53effd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5400728b0 0x7fd540197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:46.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.968+0000 7fd53e7fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd5400731c0 0x7fd540198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:46.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.968+0000 7fd53effd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5400728b0 0x7fd540197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:47440/0 (socket says 172.21.15.101:47440) 2023-07-18T03:16:46.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.968+0000 7fd53effd700 1 -- 172.21.15.101:0/2048034676 learned_addr learned my addr 172.21.15.101:0/2048034676 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:46.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.968+0000 7fd53f7fe700 1 --2- 172.21.15.101:0/2048034676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd54010c310 0x7fd54019c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:46.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.969+0000 7fd53e7fc700 1 -- 172.21.15.101:0/2048034676 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5400728b0 msgr2=0x7fd540197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:46.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.969+0000 7fd53e7fc700 1 --2- 172.21.15.101:0/2048034676 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5400728b0 0x7fd540197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:46.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.969+0000 7fd53e7fc700 1 -- 172.21.15.101:0/2048034676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd54010c310 msgr2=0x7fd54019c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:46.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.969+0000 7fd53e7fc700 1 --2- 172.21.15.101:0/2048034676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd54010c310 0x7fd54019c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:46.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.969+0000 7fd53e7fc700 1 -- 172.21.15.101:0/2048034676 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5300095f0 con 0x7fd5400731c0 2023-07-18T03:16:46.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.969+0000 7fd53f7fe700 1 --2- 172.21.15.101:0/2048034676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd54010c310 0x7fd54019c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:16:46.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.969+0000 7fd53effd700 1 --2- 172.21.15.101:0/2048034676 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5400728b0 0x7fd540197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:16:46.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.969+0000 7fd53e7fc700 1 --2- 172.21.15.101:0/2048034676 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd5400731c0 0x7fd540198350 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd53400e9f0 tx=0x7fd53400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:46.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.969+0000 7fd5449fd700 1 -- 172.21.15.101:0/2048034676 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53400c9c0 con 0x7fd5400731c0 2023-07-18T03:16:46.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.969+0000 7fd5459ff700 1 -- 172.21.15.101:0/2048034676 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd54019d400 con 0x7fd5400731c0 2023-07-18T03:16:46.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.969+0000 7fd5449fd700 1 -- 172.21.15.101:0/2048034676 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd534010430 con 0x7fd5400731c0 2023-07-18T03:16:46.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.969+0000 7fd5449fd700 1 -- 172.21.15.101:0/2048034676 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd534018680 con 0x7fd5400731c0 2023-07-18T03:16:46.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.969+0000 7fd5459ff700 1 -- 172.21.15.101:0/2048034676 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5401a4c40 con 0x7fd5400731c0 2023-07-18T03:16:46.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.970+0000 7fd5459ff700 1 -- 172.21.15.101:0/2048034676 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd540106c30 con 0x7fd5400731c0 2023-07-18T03:16:46.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.972+0000 7fd5449fd700 1 -- 172.21.15.101:0/2048034676 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fd534010860 con 0x7fd5400731c0 2023-07-18T03:16:46.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.974+0000 7fd5449fd700 1 --2- 172.21.15.101:0/2048034676 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd5280a1300 0x7fd5280a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:46.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.974+0000 7fd5449fd700 1 -- 172.21.15.101:0/2048034676 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fd534014070 con 0x7fd5400731c0 2023-07-18T03:16:46.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.974+0000 7fd53effd700 1 --2- 172.21.15.101:0/2048034676 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd5280a1300 0x7fd5280a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:46.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.975+0000 7fd53effd700 1 --2- 172.21.15.101:0/2048034676 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd5280a1300 0x7fd5280a37b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd530006010 tx=0x7fd53000b260 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:46.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:46.978+0000 7fd5449fd700 1 -- 172.21.15.101:0/2048034676 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd534051790 con 0x7fd5400731c0 2023-07-18T03:16:47.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.219+0000 7fd5459ff700 1 -- 172.21.15.101:0/2048034676 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd540067300 con 0x7fd5400731c0 2023-07-18T03:16:47.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.220+0000 7fd5449fd700 1 -- 172.21.15.101:0/2048034676 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+10858 (secure 0 0 0) 0x7fd534096250 con 0x7fd5400731c0 2023-07-18T03:16:47.214 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:16:47.215 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/4192846769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":4192846769},{"type":"v1","addr":"172.21.15.101:6839","nonce":4192846769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6841/3824340175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6840","nonce":3824340175},{"type":"v1","addr":"172.21.15.101:6841","nonce":3824340175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:04:59.361185+0000","modified":"2023-07-18T03:16:36.002358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.155:6839/2055462483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2055462483},{"type":"v1","addr":"172.21.15.155:6839","nonce":2055462483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:16:47.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.222+0000 7fd5459ff700 1 -- 172.21.15.101:0/2048034676 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd5280a1300 msgr2=0x7fd5280a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:47.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.222+0000 7fd5459ff700 1 --2- 172.21.15.101:0/2048034676 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd5280a1300 0x7fd5280a37b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd530006010 tx=0x7fd53000b260 comp rx=0 tx=0).stop 2023-07-18T03:16:47.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.223+0000 7fd5459ff700 1 -- 172.21.15.101:0/2048034676 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd5400731c0 msgr2=0x7fd540198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:47.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.223+0000 7fd5459ff700 1 --2- 172.21.15.101:0/2048034676 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd5400731c0 0x7fd540198350 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd53400e9f0 tx=0x7fd53400ed00 comp rx=0 tx=0).stop 2023-07-18T03:16:47.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.223+0000 7fd5459ff700 1 -- 172.21.15.101:0/2048034676 shutdown_connections 2023-07-18T03:16:47.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.223+0000 7fd5459ff700 1 --2- 172.21.15.101:0/2048034676 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd5280a1300 0x7fd5280a37b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.223+0000 7fd5459ff700 1 --2- 172.21.15.101:0/2048034676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd54010c310 0x7fd54019c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.223+0000 7fd5459ff700 1 --2- 172.21.15.101:0/2048034676 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5400728b0 0x7fd540197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.223+0000 7fd5459ff700 1 --2- 172.21.15.101:0/2048034676 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd5400731c0 0x7fd540198350 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.218 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.223+0000 7fd5459ff700 1 -- 172.21.15.101:0/2048034676 >> 172.21.15.101:0/2048034676 conn(0x7fd5400fb590 msgr2=0x7fd540103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:47.218 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.223+0000 7fd5459ff700 1 -- 172.21.15.101:0/2048034676 shutdown_connections 2023-07-18T03:16:47.218 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.223+0000 7fd5459ff700 1 -- 172.21.15.101:0/2048034676 wait complete. 2023-07-18T03:16:47.218 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 33 2023-07-18T03:16:47.232 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:47 smithi101 ceph-mon[126379]: pgmap v285: 129 pgs: 129 active+clean; 450 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 150 op/s 2023-07-18T03:16:47.235 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-18T03:16:47.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:47 smithi155 ceph-mon[128885]: pgmap v285: 129 pgs: 129 active+clean; 450 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 150 op/s 2023-07-18T03:16:47.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.440+0000 7f04b75ff700 1 -- 172.21.15.101:0/9557260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04b00728b0 msgr2=0x7f04b0072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:47.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.440+0000 7f04b75ff700 1 --2- 172.21.15.101:0/9557260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04b00728b0 0x7f04b0072c80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f04ac0098f0 tx=0x7f04ac009c00 comp rx=0 tx=0).stop 2023-07-18T03:16:47.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.440+0000 7f04b75ff700 1 -- 172.21.15.101:0/9557260 shutdown_connections 2023-07-18T03:16:47.435 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.440+0000 7f04b75ff700 1 --2- 172.21.15.101:0/9557260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04b00728b0 0x7f04b0072c80 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.435 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.440+0000 7f04b75ff700 1 --2- 172.21.15.101:0/9557260 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f04b010c320 0x7f04b010e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.435 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.440+0000 7f04b75ff700 1 --2- 172.21.15.101:0/9557260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04b00731c0 0x7f04b010bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.435 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.440+0000 7f04b75ff700 1 -- 172.21.15.101:0/9557260 >> 172.21.15.101:0/9557260 conn(0x7f04b00fb5e0 msgr2=0x7f04b00fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:47.435 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.440+0000 7f04b75ff700 1 -- 172.21.15.101:0/9557260 shutdown_connections 2023-07-18T03:16:47.435 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.440+0000 7f04b75ff700 1 -- 172.21.15.101:0/9557260 wait complete. 2023-07-18T03:16:47.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.442+0000 7f04b75ff700 1 Processor -- start 2023-07-18T03:16:47.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.442+0000 7f04b75ff700 1 -- start start 2023-07-18T03:16:47.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.442+0000 7f04b75ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f04b00728b0 0x7f04b0197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.442+0000 7f04b75ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04b00731c0 0x7f04b01983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.443+0000 7f04b75ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04b010c320 0x7f04b019c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.443+0000 7f04b75ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b019ce70 con 0x7f04b00728b0 2023-07-18T03:16:47.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.443+0000 7f04b75ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b019cfe0 con 0x7f04b010c320 2023-07-18T03:16:47.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.443+0000 7f04b75ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b019d1a0 con 0x7f04b00731c0 2023-07-18T03:16:47.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.443+0000 7f04b539b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f04b00728b0 0x7f04b0197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.443+0000 7f04b539b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f04b00728b0 0x7f04b0197e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:50446/0 (socket says 172.21.15.101:50446) 2023-07-18T03:16:47.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.443+0000 7f04b4b9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04b00731c0 0x7f04b01983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.443+0000 7f04b539b700 1 -- 172.21.15.101:0/1500516664 learned_addr learned my addr 172.21.15.101:0/1500516664 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:47.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.443+0000 7f04b5b9c700 1 --2- 172.21.15.101:0/1500516664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04b010c320 0x7f04b019c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.443+0000 7f04b539b700 1 -- 172.21.15.101:0/1500516664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04b00731c0 msgr2=0x7f04b01983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:47.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.443+0000 7f04b539b700 1 --2- 172.21.15.101:0/1500516664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04b00731c0 0x7f04b01983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.443+0000 7f04b539b700 1 -- 172.21.15.101:0/1500516664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04b010c320 msgr2=0x7f04b019c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:47.440 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.443+0000 7f04b539b700 1 --2- 172.21.15.101:0/1500516664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04b010c320 0x7f04b019c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.440 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.443+0000 7f04b539b700 1 -- 172.21.15.101:0/1500516664 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a8009d00 con 0x7f04b00728b0 2023-07-18T03:16:47.440 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.443+0000 7f04b4b9a700 1 --2- 172.21.15.101:0/1500516664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04b00731c0 0x7f04b01983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:16:47.440 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.443+0000 7f04b539b700 1 --2- 172.21.15.101:0/1500516664 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f04b00728b0 0x7f04b0197e90 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f04ac009810 tx=0x7f04ac00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:47.440 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.444+0000 7f04a67fc700 1 -- 172.21.15.101:0/1500516664 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ac01c070 con 0x7f04b00728b0 2023-07-18T03:16:47.440 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.444+0000 7f04b75ff700 1 -- 172.21.15.101:0/1500516664 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04ac009520 con 0x7f04b00728b0 2023-07-18T03:16:47.440 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.444+0000 7f04a67fc700 1 -- 172.21.15.101:0/1500516664 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04ac00bb20 con 0x7f04b00728b0 2023-07-18T03:16:47.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.444+0000 7f04a67fc700 1 -- 172.21.15.101:0/1500516664 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ac004930 con 0x7f04b00728b0 2023-07-18T03:16:47.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.444+0000 7f04b75ff700 1 -- 172.21.15.101:0/1500516664 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04b01a4ae0 con 0x7f04b00728b0 2023-07-18T03:16:47.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.448+0000 7f04a67fc700 1 -- 172.21.15.101:0/1500516664 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f04ac020070 con 0x7f04b00728b0 2023-07-18T03:16:47.442 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.449+0000 7f04cae43b80 1 Processor -- start 2023-07-18T03:16:47.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.449+0000 7f04cae43b80 1 -- start start 2023-07-18T03:16:47.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.450+0000 7f04cae43b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559a4a8532d0 0x559a4a8536a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.450+0000 7f04cae43b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a4a853be0 0x559a4a85e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.450+0000 7f04cae43b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a4a85ead0 0x559a4a860eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.450+0000 7f04cae43b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a4a759cb0 con 0x559a4a8532d0 2023-07-18T03:16:47.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.450+0000 7f04cae43b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a4a7aca40 con 0x559a4a853be0 2023-07-18T03:16:47.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.450+0000 7f04cae43b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a4a7a5ed0 con 0x559a4a85ead0 2023-07-18T03:16:47.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.450+0000 7f04a67fc700 1 --2- 172.21.15.101:0/1500516664 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f049c0a12c0 0x7f049c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.450+0000 7f04a67fc700 1 -- 172.21.15.101:0/1500516664 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f04ac0cdde0 con 0x7f04b00728b0 2023-07-18T03:16:47.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.450+0000 7f04b5b9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559a4a8532d0 0x559a4a8536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.450+0000 7f04b5b9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559a4a8532d0 0x559a4a8536a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:50458/0 (socket says 172.21.15.101:50458) 2023-07-18T03:16:47.446 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.450+0000 7f04b5b9c700 1 -- 172.21.15.101:0/3424819588 learned_addr learned my addr 172.21.15.101:0/3424819588 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:47.446 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.450+0000 7f04b539b700 1 --2- 172.21.15.101:0/3424819588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a4a853be0 0x559a4a85e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.446 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.450+0000 7f04b4b9a700 1 --2- 172.21.15.101:0/3424819588 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a4a85ead0 0x559a4a860eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.446 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.450+0000 7f04b5b9c700 1 --2- 172.21.15.101:0/1500516664 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f049c0a12c0 0x7f049c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.446 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.450+0000 7f04b539b700 1 -- 172.21.15.101:0/3424819588 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a4a85ead0 msgr2=0x559a4a860eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:47.446 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.450+0000 7f04b539b700 1 --2- 172.21.15.101:0/3424819588 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a4a85ead0 0x559a4a860eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.446 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.450+0000 7f04b539b700 1 -- 172.21.15.101:0/3424819588 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559a4a8532d0 msgr2=0x559a4a8536a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:47.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.450+0000 7f04b539b700 1 --2- 172.21.15.101:0/3424819588 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559a4a8532d0 0x559a4a8536a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.451+0000 7f04b539b700 1 -- 172.21.15.101:0/3424819588 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a4a5d3de0 con 0x559a4a853be0 2023-07-18T03:16:47.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.451+0000 7f04b5b9c700 1 --2- 172.21.15.101:0/3424819588 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559a4a8532d0 0x559a4a8536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:16:47.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.451+0000 7f04b4b9a700 1 --2- 172.21.15.101:0/3424819588 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a4a85ead0 0x559a4a860eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:16:47.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.451+0000 7f04b539b700 1 --2- 172.21.15.101:0/3424819588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a4a853be0 0x559a4a85e590 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f04ac0cb120 tx=0x7f04ac009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e7ca0c016495a12 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:47.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.451+0000 7f049b7fe700 1 -- 172.21.15.101:0/3424819588 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ac09ab30 con 0x559a4a853be0 2023-07-18T03:16:47.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.451+0000 7f04b5b9c700 1 --2- 172.21.15.101:0/1500516664 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f049c0a12c0 0x7f049c0a3770 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f04a8008ba0 tx=0x7f04a800b040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:47.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.451+0000 7f049b7fe700 1 -- 172.21.15.101:0/3424819588 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04ac09eb60 con 0x559a4a853be0 2023-07-18T03:16:47.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.451+0000 7f049b7fe700 1 -- 172.21.15.101:0/3424819588 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ac09aff0 con 0x559a4a853be0 2023-07-18T03:16:47.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.452+0000 7f04cae43b80 1 -- 172.21.15.101:0/3424819588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a4a853be0 msgr2=0x559a4a85e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:47.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.452+0000 7f04cae43b80 1 --2- 172.21.15.101:0/3424819588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a4a853be0 0x559a4a85e590 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f04ac0cb120 tx=0x7f04ac009260 comp rx=0 tx=0).stop 2023-07-18T03:16:47.452 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.452+0000 7f04cae43b80 1 -- 172.21.15.101:0/3424819588 shutdown_connections 2023-07-18T03:16:47.452 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.452+0000 7f04cae43b80 1 --2- 172.21.15.101:0/3424819588 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a4a85ead0 0x559a4a860eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.452 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.452+0000 7f04cae43b80 1 --2- 172.21.15.101:0/3424819588 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559a4a8532d0 0x559a4a8536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.452 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.452+0000 7f04cae43b80 1 --2- 172.21.15.101:0/3424819588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a4a853be0 0x559a4a85e590 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.452 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.452+0000 7f04cae43b80 1 -- 172.21.15.101:0/3424819588 >> 172.21.15.101:0/3424819588 conn(0x559a4a849da0 msgr2=0x559a4a84f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:47.452 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.452+0000 7f04cae43b80 1 -- 172.21.15.101:0/3424819588 shutdown_connections 2023-07-18T03:16:47.452 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.452+0000 7f04cae43b80 1 -- 172.21.15.101:0/3424819588 wait complete. 2023-07-18T03:16:47.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.452+0000 7f04cae43b80 1 -- start start 2023-07-18T03:16:47.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.452+0000 7f04cae43b80 1 Processor -- start 2023-07-18T03:16:47.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.453+0000 7f04cae43b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a4a8532d0 0x559a4a862a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.453+0000 7f04cae43b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a4a853be0 0x559a4a862f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.453+0000 7f04cae43b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559a4a85ead0 0x559a4a85d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.453+0000 7f04cae43b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a4a745ac0 con 0x559a4a85ead0 2023-07-18T03:16:47.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.453+0000 7f04cae43b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a4a7457c0 con 0x559a4a8532d0 2023-07-18T03:16:47.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.453+0000 7f04cae43b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a4a629920 con 0x559a4a853be0 2023-07-18T03:16:47.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.453+0000 7f04cae43b80 10 client.0 _ll_register_callbacks cb 0x559a4a743760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:16:47.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.453+0000 7f04b539b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a4a8532d0 0x559a4a862a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.453+0000 7f04b539b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a4a8532d0 0x559a4a862a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60186/0 (socket says 172.21.15.101:60186) 2023-07-18T03:16:47.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.453+0000 7f04b539b700 1 -- 172.21.15.101:0/4211996478 learned_addr learned my addr 172.21.15.101:0/4211996478 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:47.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.453+0000 7f04b5b9c700 1 --2- 172.21.15.101:0/4211996478 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559a4a85ead0 0x559a4a85d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.453+0000 7f04b4b9a700 1 --2- 172.21.15.101:0/4211996478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a4a853be0 0x559a4a862f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.454+0000 7f04b539b700 1 -- 172.21.15.101:0/4211996478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a4a853be0 msgr2=0x559a4a862f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:47.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.454+0000 7f04b539b700 1 --2- 172.21.15.101:0/4211996478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a4a853be0 0x559a4a862f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.454+0000 7f04b539b700 1 -- 172.21.15.101:0/4211996478 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559a4a85ead0 msgr2=0x559a4a85d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:47.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.454+0000 7f04b539b700 1 --2- 172.21.15.101:0/4211996478 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559a4a85ead0 0x559a4a85d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.454+0000 7f04b539b700 1 -- 172.21.15.101:0/4211996478 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04a8008710 con 0x559a4a8532d0 2023-07-18T03:16:47.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.454+0000 7f04b539b700 1 --2- 172.21.15.101:0/4211996478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a4a8532d0 0x559a4a862a20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f04ac005e90 tx=0x7f04ac09a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d974d5dea53a5f02 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:47.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.454+0000 7f0498ff9700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:16:47.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.454+0000 7f0498ff9700 1 -- 172.21.15.101:0/4211996478 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ac0a2ef0 con 0x559a4a8532d0 2023-07-18T03:16:47.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.454+0000 7f0498ff9700 1 -- 172.21.15.101:0/4211996478 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f04ac09ab30 con 0x559a4a8532d0 2023-07-18T03:16:47.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.454+0000 7f04cae43b80 1 -- 172.21.15.101:0/4211996478 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559a4a6ff4f0 con 0x559a4a8532d0 2023-07-18T03:16:47.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.454+0000 7f0498ff9700 1 -- 172.21.15.101:0/4211996478 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04ac0b3260 con 0x559a4a8532d0 2023-07-18T03:16:47.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.454+0000 7f04cae43b80 10 client.27419 fetch_fsmap learned FSMap version 33 2023-07-18T03:16:47.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.454+0000 7f04cae43b80 1 -- 172.21.15.101:0/4211996478 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f04a0009c90 con 0x559a4a8532d0 2023-07-18T03:16:47.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.454+0000 7f0498ff9700 1 -- 172.21.15.101:0/4211996478 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559a4a6ff4f0 con 0x559a4a8532d0 2023-07-18T03:16:47.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.455+0000 7f0498ff9700 1 -- 172.21.15.101:0/4211996478 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f04ac096a20 con 0x559a4a8532d0 2023-07-18T03:16:47.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.455+0000 7f04cae43b80 10 client.27419 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:16:47.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.455+0000 7f04cae43b80 10 client.27419 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:16:47.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.455+0000 7f04cae43b80 20 client.27419 populate_metadata read hostname 'smithi101' 2023-07-18T03:16:47.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.455+0000 7f04cae43b80 1 --2- 172.21.15.101:0/4211996478 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x559a4a86dbd0 0x559a4a86ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.455+0000 7f04cae43b80 4 client.27419 mds_command: new command op to 24541 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:16:47.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.455+0000 7f04cae43b80 1 -- 172.21.15.101:0/4211996478 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559a4a79eeb0 con 0x559a4a86dbd0 2023-07-18T03:16:47.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.455+0000 7f04b4b9a700 1 --2- 172.21.15.101:0/4211996478 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x559a4a86dbd0 0x559a4a86ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.455+0000 7f0498ff9700 10 client.27419 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:16:47.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.455+0000 7f04b4b9a700 1 --2- 172.21.15.101:0/4211996478 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x559a4a86dbd0 0x559a4a86ffe0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5ccb695569b498ff server_cookie=30e680b4967e4722 in_seq=0 out_seq=0 2023-07-18T03:16:47.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.456+0000 7f0498ff9700 1 -- 172.21.15.101:0/4211996478 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559a4a79eeb0 con 0x559a4a86dbd0 2023-07-18T03:16:47.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.456+0000 7f0498ff9700 10 client.27419 handle_command_reply: tid=0 2023-07-18T03:16:47.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.456+0000 7f04cae43b80 1 client.27419 shutdown 2023-07-18T03:16:47.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.456+0000 7f04cae43b80 1 -- 172.21.15.101:0/4211996478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a4a8532d0 msgr2=0x559a4a862a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:47.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.456+0000 7f04cae43b80 1 --2- 172.21.15.101:0/4211996478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a4a8532d0 0x559a4a862a20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f04ac005e90 tx=0x7f04ac09a250 comp rx=0 tx=0).stop 2023-07-18T03:16:47.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.456+0000 7f04cae43b80 1 -- 172.21.15.101:0/4211996478 shutdown_connections 2023-07-18T03:16:47.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.456+0000 7f04cae43b80 1 --2- 172.21.15.101:0/4211996478 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x559a4a86dbd0 0x559a4a86ffe0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.456+0000 7f04cae43b80 1 --2- 172.21.15.101:0/4211996478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a4a8532d0 0x559a4a862a20 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.456+0000 7f0498ff9700 0 client.27419 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:16:47.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.456+0000 7f04cae43b80 1 --2- 172.21.15.101:0/4211996478 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559a4a85ead0 0x559a4a85d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.456+0000 7f04cae43b80 1 --2- 172.21.15.101:0/4211996478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a4a853be0 0x559a4a862f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.456+0000 7f04cae43b80 1 -- 172.21.15.101:0/4211996478 >> 172.21.15.101:0/4211996478 conn(0x559a4a849da0 msgr2=0x559a4a84f230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:47.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.457+0000 7f04cae43b80 1 -- 172.21.15.101:0/4211996478 shutdown_connections 2023-07-18T03:16:47.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.457+0000 7f04cae43b80 1 -- 172.21.15.101:0/4211996478 wait complete. 2023-07-18T03:16:47.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.457+0000 7f04cae43b80 20 client.27419 trim_cache size 0 max 16384 2023-07-18T03:16:47.461 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:47 smithi179 ceph-mon[129229]: pgmap v285: 129 pgs: 129 active+clean; 450 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 150 op/s 2023-07-18T03:16:47.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.470+0000 7f04cae43b80 1 Processor -- start 2023-07-18T03:16:47.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.470+0000 7f04cae43b80 1 -- start start 2023-07-18T03:16:47.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.470+0000 7f04cae43b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559a4a8532d0 0x559a4a8626e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.470+0000 7f04cae43b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a4a853be0 0x559a4a862c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.470+0000 7f04cae43b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a4a85ead0 0x559a4a85ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.470+0000 7f04cae43b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a4a79eeb0 con 0x559a4a8532d0 2023-07-18T03:16:47.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.470+0000 7f04cae43b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a4a84fc20 con 0x559a4a85ead0 2023-07-18T03:16:47.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.470+0000 7f04cae43b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a4a850d20 con 0x559a4a853be0 2023-07-18T03:16:47.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.470+0000 7f04b539b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559a4a8532d0 0x559a4a8626e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.470+0000 7f04b539b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559a4a8532d0 0x559a4a8626e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:50490/0 (socket says 172.21.15.101:50490) 2023-07-18T03:16:47.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.470+0000 7f04b539b700 1 -- 172.21.15.101:0/1667029970 learned_addr learned my addr 172.21.15.101:0/1667029970 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:47.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.471+0000 7f04b5b9c700 1 --2- 172.21.15.101:0/1667029970 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a4a85ead0 0x559a4a85ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.471+0000 7f04b4b9a700 1 --2- 172.21.15.101:0/1667029970 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a4a853be0 0x559a4a862c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.471+0000 7f04b539b700 1 -- 172.21.15.101:0/1667029970 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a4a853be0 msgr2=0x559a4a862c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:47.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.471+0000 7f04b539b700 1 --2- 172.21.15.101:0/1667029970 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a4a853be0 0x559a4a862c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.471+0000 7f04b539b700 1 -- 172.21.15.101:0/1667029970 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a4a85ead0 msgr2=0x559a4a85ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:47.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.471+0000 7f04b539b700 1 --2- 172.21.15.101:0/1667029970 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a4a85ead0 0x559a4a85ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.471+0000 7f04b539b700 1 -- 172.21.15.101:0/1667029970 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b01a4ae0 con 0x559a4a8532d0 2023-07-18T03:16:47.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.471+0000 7f04b5b9c700 1 --2- 172.21.15.101:0/1667029970 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a4a85ead0 0x559a4a85ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:16:47.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.471+0000 7f04b539b700 1 --2- 172.21.15.101:0/1667029970 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559a4a8532d0 0x559a4a8626e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f04ac09aac0 tx=0x7f04ac00bc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c460760ca04ddf09 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:47.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.471+0000 7f049bfff700 1 -- 172.21.15.101:0/1667029970 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ac0b3260 con 0x559a4a8532d0 2023-07-18T03:16:47.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.471+0000 7f049bfff700 1 -- 172.21.15.101:0/1667029970 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04ac09ab30 con 0x559a4a8532d0 2023-07-18T03:16:47.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.471+0000 7f049bfff700 1 -- 172.21.15.101:0/1667029970 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ac0a2ca0 con 0x559a4a8532d0 2023-07-18T03:16:47.471 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.472+0000 7f04cae43b80 1 -- 172.21.15.101:0/1667029970 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559a4a8532d0 msgr2=0x559a4a8626e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:47.471 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.472+0000 7f04cae43b80 1 --2- 172.21.15.101:0/1667029970 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559a4a8532d0 0x559a4a8626e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f04ac09aac0 tx=0x7f04ac00bc90 comp rx=0 tx=0).stop 2023-07-18T03:16:47.471 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.472+0000 7f04cae43b80 1 -- 172.21.15.101:0/1667029970 shutdown_connections 2023-07-18T03:16:47.471 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.472+0000 7f04cae43b80 1 --2- 172.21.15.101:0/1667029970 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a4a85ead0 0x559a4a85ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.471 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.472+0000 7f04cae43b80 1 --2- 172.21.15.101:0/1667029970 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559a4a8532d0 0x559a4a8626e0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.471 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.472+0000 7f04cae43b80 1 --2- 172.21.15.101:0/1667029970 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a4a853be0 0x559a4a862c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.472 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.472+0000 7f04cae43b80 1 -- 172.21.15.101:0/1667029970 >> 172.21.15.101:0/1667029970 conn(0x559a4a849da0 msgr2=0x559a4a863f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:47.472 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.472+0000 7f04cae43b80 1 -- 172.21.15.101:0/1667029970 shutdown_connections 2023-07-18T03:16:47.472 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.472+0000 7f04cae43b80 1 -- 172.21.15.101:0/1667029970 wait complete. 2023-07-18T03:16:47.472 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.472+0000 7f04cae43b80 1 -- start start 2023-07-18T03:16:47.472 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.472+0000 7f04cae43b80 1 Processor -- start 2023-07-18T03:16:47.472 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.473+0000 7f04cae43b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559a4a8532d0 0x559a4a8669f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.472 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.473+0000 7f04cae43b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a4a853be0 0x559a4a866f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.472 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.473+0000 7f04cae43b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a4a85ead0 0x559a4a8c0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.473+0000 7f04cae43b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a4a8c0fc0 con 0x559a4a8532d0 2023-07-18T03:16:47.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.473+0000 7f04cae43b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a4a8c1100 con 0x559a4a853be0 2023-07-18T03:16:47.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.473+0000 7f04cae43b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a4a8c1270 con 0x559a4a85ead0 2023-07-18T03:16:47.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.473+0000 7f04cae43b80 10 client.0 _ll_register_callbacks cb 0x559a4a743760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:16:47.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.473+0000 7f04b5b9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a4a85ead0 0x559a4a8c0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.473+0000 7f04b4b9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a4a853be0 0x559a4a866f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.473+0000 7f04b5b9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a4a85ead0 0x559a4a8c0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:47496/0 (socket says 172.21.15.101:47496) 2023-07-18T03:16:47.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.473+0000 7f04b5b9c700 1 -- 172.21.15.101:0/2173580013 learned_addr learned my addr 172.21.15.101:0/2173580013 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:47.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.473+0000 7f04b539b700 1 --2- 172.21.15.101:0/2173580013 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559a4a8532d0 0x559a4a8669f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.474+0000 7f04b5b9c700 1 -- 172.21.15.101:0/2173580013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a4a853be0 msgr2=0x559a4a866f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:47.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.474+0000 7f04b5b9c700 1 --2- 172.21.15.101:0/2173580013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a4a853be0 0x559a4a866f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.474+0000 7f04b5b9c700 1 -- 172.21.15.101:0/2173580013 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559a4a8532d0 msgr2=0x559a4a8669f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:47.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.474+0000 7f04b5b9c700 1 --2- 172.21.15.101:0/2173580013 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559a4a8532d0 0x559a4a8669f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.474+0000 7f04b5b9c700 1 -- 172.21.15.101:0/2173580013 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04a0009c90 con 0x559a4a85ead0 2023-07-18T03:16:47.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.474+0000 7f04b5b9c700 1 --2- 172.21.15.101:0/2173580013 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a4a85ead0 0x559a4a8c0a80 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f04a8018a70 tx=0x7f04a8004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdb5032996f489fd server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:47.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.474+0000 7f0498ff9700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:16:47.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.474+0000 7f0498ff9700 1 -- 172.21.15.101:0/2173580013 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a8022040 con 0x559a4a85ead0 2023-07-18T03:16:47.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.474+0000 7f0498ff9700 1 -- 172.21.15.101:0/2173580013 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f04a8018e40 con 0x559a4a85ead0 2023-07-18T03:16:47.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.474+0000 7f04cae43b80 1 -- 172.21.15.101:0/2173580013 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f04ac09eb60 con 0x559a4a85ead0 2023-07-18T03:16:47.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.474+0000 7f0498ff9700 1 -- 172.21.15.101:0/2173580013 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04a80046e0 con 0x559a4a85ead0 2023-07-18T03:16:47.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.474+0000 7f04cae43b80 10 client.27460 fetch_fsmap learned FSMap version 33 2023-07-18T03:16:47.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.474+0000 7f04cae43b80 1 -- 172.21.15.101:0/2173580013 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x559a4a5bcc60 con 0x559a4a85ead0 2023-07-18T03:16:47.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.475+0000 7f0498ff9700 1 -- 172.21.15.101:0/2173580013 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04ac09eb60 con 0x559a4a85ead0 2023-07-18T03:16:47.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.475+0000 7f0498ff9700 1 -- 172.21.15.101:0/2173580013 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f04a8028430 con 0x559a4a85ead0 2023-07-18T03:16:47.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.475+0000 7f04cae43b80 10 client.27460 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:16:47.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.475+0000 7f04cae43b80 10 client.27460 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:16:47.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.475+0000 7f04cae43b80 20 client.27460 populate_metadata read hostname 'smithi101' 2023-07-18T03:16:47.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.475+0000 7f04cae43b80 1 --2- 172.21.15.101:0/2173580013 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x559a4a8c2970 0x559a4a8c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.475+0000 7f04cae43b80 4 client.27460 mds_command: new command op to 24541 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:16:47.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.475+0000 7f04cae43b80 1 -- 172.21.15.101:0/2173580013 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559a4a761050 con 0x559a4a8c2970 2023-07-18T03:16:47.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.475+0000 7f04b539b700 1 --2- 172.21.15.101:0/2173580013 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x559a4a8c2970 0x559a4a8c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.475+0000 7f0498ff9700 10 client.27460 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:16:47.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.475+0000 7f04b539b700 1 --2- 172.21.15.101:0/2173580013 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x559a4a8c2970 0x559a4a8c4d80 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=694fc2a9c1acb5c3 server_cookie=dda0d6395e2191f in_seq=0 out_seq=0 2023-07-18T03:16:47.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.476+0000 7f0498ff9700 1 -- 172.21.15.101:0/2173580013 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559a4a761050 con 0x559a4a8c2970 2023-07-18T03:16:47.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.476+0000 7f0498ff9700 10 client.27460 handle_command_reply: tid=0 2023-07-18T03:16:47.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.476+0000 7f04cae43b80 1 client.27460 shutdown 2023-07-18T03:16:47.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.476+0000 7f04cae43b80 1 -- 172.21.15.101:0/2173580013 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a4a85ead0 msgr2=0x559a4a8c0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:47.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.476+0000 7f04cae43b80 1 --2- 172.21.15.101:0/2173580013 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a4a85ead0 0x559a4a8c0a80 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f04a8018a70 tx=0x7f04a8004b40 comp rx=0 tx=0).stop 2023-07-18T03:16:47.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.476+0000 7f04cae43b80 1 -- 172.21.15.101:0/2173580013 shutdown_connections 2023-07-18T03:16:47.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.476+0000 7f04cae43b80 1 --2- 172.21.15.101:0/2173580013 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x559a4a8c2970 0x559a4a8c4d80 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.476+0000 7f04cae43b80 1 --2- 172.21.15.101:0/2173580013 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a4a85ead0 0x559a4a8c0a80 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.476+0000 7f04cae43b80 1 --2- 172.21.15.101:0/2173580013 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559a4a8532d0 0x559a4a8669f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.476+0000 7f0498ff9700 0 client.27460 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:16:47.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.476+0000 7f04cae43b80 1 --2- 172.21.15.101:0/2173580013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a4a853be0 0x559a4a866f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.476+0000 7f04cae43b80 1 -- 172.21.15.101:0/2173580013 >> 172.21.15.101:0/2173580013 conn(0x559a4a849da0 msgr2=0x559a4a84e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:47.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.476+0000 7f04cae43b80 1 -- 172.21.15.101:0/2173580013 shutdown_connections 2023-07-18T03:16:47.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.476+0000 7f04cae43b80 1 -- 172.21.15.101:0/2173580013 wait complete. 2023-07-18T03:16:47.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.476+0000 7f04cae43b80 20 client.27460 trim_cache size 0 max 16384 2023-07-18T03:16:47.479 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:16:47.480 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.477+0000 7f04b75ff700 1 -- 172.21.15.101:0/1500516664 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f049c0a12c0 msgr2=0x7f049c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:47.480 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.477+0000 7f04b75ff700 1 --2- 172.21.15.101:0/1500516664 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f049c0a12c0 0x7f049c0a3770 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f04a8008ba0 tx=0x7f04a800b040 comp rx=0 tx=0).stop 2023-07-18T03:16:47.480 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.477+0000 7f04b75ff700 1 -- 172.21.15.101:0/1500516664 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f04b00728b0 msgr2=0x7f04b0197e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:47.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.477+0000 7f04b75ff700 1 --2- 172.21.15.101:0/1500516664 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f04b00728b0 0x7f04b0197e90 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f04ac009810 tx=0x7f04ac00fa00 comp rx=0 tx=0).stop 2023-07-18T03:16:47.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.477+0000 7f04b75ff700 1 -- 172.21.15.101:0/1500516664 shutdown_connections 2023-07-18T03:16:47.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.477+0000 7f04b75ff700 1 --2- 172.21.15.101:0/1500516664 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f049c0a12c0 0x7f049c0a3770 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.477+0000 7f04b75ff700 1 --2- 172.21.15.101:0/1500516664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04b010c320 0x7f04b019c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.477+0000 7f04b75ff700 1 --2- 172.21.15.101:0/1500516664 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f04b00728b0 0x7f04b0197e90 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.477+0000 7f04b75ff700 1 --2- 172.21.15.101:0/1500516664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04b00731c0 0x7f04b01983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.477+0000 7f04b75ff700 1 -- 172.21.15.101:0/1500516664 >> 172.21.15.101:0/1500516664 conn(0x7f04b00fb5e0 msgr2=0x7f04b010e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:47.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.477+0000 7f04b75ff700 1 -- 172.21.15.101:0/1500516664 shutdown_connections 2023-07-18T03:16:47.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.477+0000 7f04b75ff700 1 -- 172.21.15.101:0/1500516664 wait complete. 2023-07-18T03:16:47.488 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-18T03:16:47.606 INFO:tasks.workunit.client.0.smithi101.stdout: 1 601947 6.14 MB/sec execute 372 sec latency 697.052 ms 2023-07-18T03:16:47.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.700+0000 7fd9f23ff700 1 -- 172.21.15.101:0/1769688508 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd9ec105af0 msgr2=0x7fd9ec105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:47.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.700+0000 7fd9f23ff700 1 --2- 172.21.15.101:0/1769688508 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd9ec105af0 0x7fd9ec105ec0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc009990 tx=0x7fd9dc009ca0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.700+0000 7fd9f23ff700 1 -- 172.21.15.101:0/1769688508 shutdown_connections 2023-07-18T03:16:47.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.700+0000 7fd9f23ff700 1 --2- 172.21.15.101:0/1769688508 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9ec100490 0x7fd9ec10c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.700+0000 7fd9f23ff700 1 --2- 172.21.15.101:0/1769688508 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd9ec105af0 0x7fd9ec105ec0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.700+0000 7fd9f23ff700 1 --2- 172.21.15.101:0/1769688508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9ec0ffae0 0x7fd9ec0fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.700+0000 7fd9f23ff700 1 -- 172.21.15.101:0/1769688508 >> 172.21.15.101:0/1769688508 conn(0x7fd9ec0fb5e0 msgr2=0x7fd9ec0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:47.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.700+0000 7fd9f23ff700 1 -- 172.21.15.101:0/1769688508 shutdown_connections 2023-07-18T03:16:47.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.700+0000 7fd9f23ff700 1 -- 172.21.15.101:0/1769688508 wait complete. 2023-07-18T03:16:47.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.701+0000 7fd9f23ff700 1 Processor -- start 2023-07-18T03:16:47.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.702+0000 7fd9f23ff700 1 -- start start 2023-07-18T03:16:47.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.702+0000 7fd9f23ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd9ec0ffae0 0x7fd9ec072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.702+0000 7fd9f23ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9ec100490 0x7fd9ec072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.702+0000 7fd9f23ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9ec105af0 0x7fd9ec1a3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.702+0000 7fd9f23ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec073580 con 0x7fd9ec0ffae0 2023-07-18T03:16:47.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.702+0000 7fd9f23ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec1a4320 con 0x7fd9ec100490 2023-07-18T03:16:47.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.702+0000 7fd9f23ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec1a4460 con 0x7fd9ec105af0 2023-07-18T03:16:47.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.702+0000 7fd9ebfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd9ec0ffae0 0x7fd9ec072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.702+0000 7fd9ebfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd9ec0ffae0 0x7fd9ec072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:50520/0 (socket says 172.21.15.101:50520) 2023-07-18T03:16:47.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.702+0000 7fd9ebfff700 1 -- 172.21.15.101:0/2208648727 learned_addr learned my addr 172.21.15.101:0/2208648727 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:47.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.702+0000 7fd9eb7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9ec100490 0x7fd9ec072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.702+0000 7fd9f099c700 1 --2- 172.21.15.101:0/2208648727 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9ec105af0 0x7fd9ec1a3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.703+0000 7fd9ebfff700 1 -- 172.21.15.101:0/2208648727 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9ec105af0 msgr2=0x7fd9ec1a3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:47.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.703+0000 7fd9ebfff700 1 --2- 172.21.15.101:0/2208648727 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9ec105af0 0x7fd9ec1a3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.703+0000 7fd9ebfff700 1 -- 172.21.15.101:0/2208648727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9ec100490 msgr2=0x7fd9ec072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:47.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.703+0000 7fd9ebfff700 1 --2- 172.21.15.101:0/2208648727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9ec100490 0x7fd9ec072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.703+0000 7fd9ebfff700 1 -- 172.21.15.101:0/2208648727 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9dc0095f0 con 0x7fd9ec0ffae0 2023-07-18T03:16:47.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.703+0000 7fd9ebfff700 1 --2- 172.21.15.101:0/2208648727 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd9ec0ffae0 0x7fd9ec072810 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc005910 tx=0x7fd9dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:47.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.703+0000 7fd9e97fa700 1 -- 172.21.15.101:0/2208648727 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc01d070 con 0x7fd9ec0ffae0 2023-07-18T03:16:47.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.703+0000 7fd9e97fa700 1 -- 172.21.15.101:0/2208648727 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9dc022470 con 0x7fd9ec0ffae0 2023-07-18T03:16:47.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.703+0000 7fd9f23ff700 1 -- 172.21.15.101:0/2208648727 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9ec1a4660 con 0x7fd9ec0ffae0 2023-07-18T03:16:47.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.703+0000 7fd9e97fa700 1 -- 172.21.15.101:0/2208648727 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc017650 con 0x7fd9ec0ffae0 2023-07-18T03:16:47.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.703+0000 7fd9f23ff700 1 -- 172.21.15.101:0/2208648727 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9ec1a4a80 con 0x7fd9ec0ffae0 2023-07-18T03:16:47.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.707+0000 7fd9e97fa700 1 -- 172.21.15.101:0/2208648727 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fd9dc00fad0 con 0x7fd9ec0ffae0 2023-07-18T03:16:47.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.708+0000 7fda05c56b80 1 Processor -- start 2023-07-18T03:16:47.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.709+0000 7fda05c56b80 1 -- start start 2023-07-18T03:16:47.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.709+0000 7fda05c56b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea5da42d0 0x55bea5da46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.709+0000 7fda05c56b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bea5da4be0 0x55bea5daf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.709+0000 7fda05c56b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bea5dafad0 0x55bea5db1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.709+0000 7fda05c56b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bea5caacb0 con 0x55bea5da4be0 2023-07-18T03:16:47.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.709+0000 7fda05c56b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bea5cfda40 con 0x55bea5da42d0 2023-07-18T03:16:47.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.709+0000 7fda05c56b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bea5cf6ed0 con 0x55bea5dafad0 2023-07-18T03:16:47.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.709+0000 7fd9e97fa700 1 --2- 172.21.15.101:0/2208648727 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd9d40a12c0 0x7fd9d40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.709+0000 7fd9ebfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bea5da4be0 0x55bea5daf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.709+0000 7fd9eb7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bea5dafad0 0x55bea5db1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.709+0000 7fd9eb7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bea5dafad0 0x55bea5db1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:47520/0 (socket says 172.21.15.101:47520) 2023-07-18T03:16:47.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.709+0000 7fd9ebfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bea5da4be0 0x55bea5daf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:50530/0 (socket says 172.21.15.101:50530) 2023-07-18T03:16:47.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.709+0000 7fd9eb7fe700 1 -- 172.21.15.101:0/3167306643 learned_addr learned my addr 172.21.15.101:0/3167306643 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:47.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.710+0000 7fd9f099c700 1 --2- 172.21.15.101:0/3167306643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea5da42d0 0x55bea5da46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.710+0000 7fd9e97fa700 1 -- 172.21.15.101:0/2208648727 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fd9dc0ce560 con 0x7fd9ec0ffae0 2023-07-18T03:16:47.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.710+0000 7fd9f099c700 1 --2- 172.21.15.101:0/2208648727 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd9d40a12c0 0x7fd9d40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.710+0000 7fd9eb7fe700 1 -- 172.21.15.101:0/3167306643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea5da42d0 msgr2=0x55bea5da46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:47.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.710+0000 7fd9eb7fe700 1 --2- 172.21.15.101:0/3167306643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea5da42d0 0x55bea5da46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.710+0000 7fd9eb7fe700 1 -- 172.21.15.101:0/3167306643 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bea5da4be0 msgr2=0x55bea5daf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:47.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.710+0000 7fd9eb7fe700 1 --2- 172.21.15.101:0/3167306643 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bea5da4be0 0x55bea5daf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.710+0000 7fd9eb7fe700 1 -- 172.21.15.101:0/3167306643 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bea5b24de0 con 0x55bea5dafad0 2023-07-18T03:16:47.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.710+0000 7fd9eb7fe700 1 --2- 172.21.15.101:0/3167306643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bea5dafad0 0x55bea5db1eb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd9e000bd30 tx=0x7fd9e00106b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=519c5f069bd525b0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:47.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.710+0000 7fd9f099c700 1 --2- 172.21.15.101:0/2208648727 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd9d40a12c0 0x7fd9d40a3770 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4009ec0 tx=0x7fd9e400b040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:47.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.710+0000 7fd9d27fc700 1 -- 172.21.15.101:0/3167306643 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e0010db0 con 0x55bea5dafad0 2023-07-18T03:16:47.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.710+0000 7fd9d27fc700 1 -- 172.21.15.101:0/3167306643 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9e0015070 con 0x55bea5dafad0 2023-07-18T03:16:47.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.710+0000 7fd9d27fc700 1 -- 172.21.15.101:0/3167306643 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e0003d20 con 0x55bea5dafad0 2023-07-18T03:16:47.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.711+0000 7fda05c56b80 1 -- 172.21.15.101:0/3167306643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bea5dafad0 msgr2=0x55bea5db1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:47.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.711+0000 7fda05c56b80 1 --2- 172.21.15.101:0/3167306643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bea5dafad0 0x55bea5db1eb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd9e000bd30 tx=0x7fd9e00106b0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.711+0000 7fda05c56b80 1 -- 172.21.15.101:0/3167306643 shutdown_connections 2023-07-18T03:16:47.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.711+0000 7fda05c56b80 1 --2- 172.21.15.101:0/3167306643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea5da42d0 0x55bea5da46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.711+0000 7fda05c56b80 1 --2- 172.21.15.101:0/3167306643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bea5dafad0 0x55bea5db1eb0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.711+0000 7fda05c56b80 1 --2- 172.21.15.101:0/3167306643 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bea5da4be0 0x55bea5daf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.711+0000 7fda05c56b80 1 -- 172.21.15.101:0/3167306643 >> 172.21.15.101:0/3167306643 conn(0x55bea5d9ada0 msgr2=0x55bea5da06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:47.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.711+0000 7fda05c56b80 1 -- 172.21.15.101:0/3167306643 shutdown_connections 2023-07-18T03:16:47.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.711+0000 7fda05c56b80 1 -- 172.21.15.101:0/3167306643 wait complete. 2023-07-18T03:16:47.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.712+0000 7fda05c56b80 1 -- start start 2023-07-18T03:16:47.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.712+0000 7fda05c56b80 1 Processor -- start 2023-07-18T03:16:47.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.712+0000 7fda05c56b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea5da42d0 0x55bea5daf520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.712+0000 7fda05c56b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bea5da4be0 0x55bea5dac810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.712+0000 7fda05c56b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bea5dafad0 0x55bea5dacf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.712+0000 7fda05c56b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bea5c96ac0 con 0x55bea5dafad0 2023-07-18T03:16:47.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.712+0000 7fda05c56b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bea5c967c0 con 0x55bea5da42d0 2023-07-18T03:16:47.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.712+0000 7fda05c56b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bea5b7a920 con 0x55bea5da4be0 2023-07-18T03:16:47.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.713+0000 7fda05c56b80 10 client.0 _ll_register_callbacks cb 0x55bea5c94760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:16:47.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.713+0000 7fd9eb7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bea5da4be0 0x55bea5dac810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.713+0000 7fd9eb7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bea5da4be0 0x55bea5dac810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:47534/0 (socket says 172.21.15.101:47534) 2023-07-18T03:16:47.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.713+0000 7fd9eb7fe700 1 -- 172.21.15.101:0/4071649565 learned_addr learned my addr 172.21.15.101:0/4071649565 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:47.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.713+0000 7fd9f099c700 1 --2- 172.21.15.101:0/4071649565 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bea5dafad0 0x55bea5dacf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.713+0000 7fd9ebfff700 1 --2- 172.21.15.101:0/4071649565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea5da42d0 0x55bea5daf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.713+0000 7fd9eb7fe700 1 -- 172.21.15.101:0/4071649565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea5da42d0 msgr2=0x55bea5daf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:47.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.713+0000 7fd9eb7fe700 1 --2- 172.21.15.101:0/4071649565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea5da42d0 0x55bea5daf520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.713+0000 7fd9eb7fe700 1 -- 172.21.15.101:0/4071649565 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bea5dafad0 msgr2=0x55bea5dacf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:47.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.713+0000 7fd9eb7fe700 1 --2- 172.21.15.101:0/4071649565 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bea5dafad0 0x55bea5dacf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.713+0000 7fd9eb7fe700 1 -- 172.21.15.101:0/4071649565 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9e000b960 con 0x55bea5da4be0 2023-07-18T03:16:47.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.713+0000 7fd9eb7fe700 1 --2- 172.21.15.101:0/4071649565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bea5da4be0 0x55bea5dac810 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd9e000b900 tx=0x7fd9e0010de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65290f3d40ac4ea4 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:47.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.713+0000 7fd9cb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:16:47.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.713+0000 7fd9cb7fe700 1 -- 172.21.15.101:0/4071649565 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e0020070 con 0x55bea5da4be0 2023-07-18T03:16:47.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.714+0000 7fd9cb7fe700 1 -- 172.21.15.101:0/4071649565 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fd9e0017070 con 0x55bea5da4be0 2023-07-18T03:16:47.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.714+0000 7fda05c56b80 1 -- 172.21.15.101:0/4071649565 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bea5c504f0 con 0x55bea5da4be0 2023-07-18T03:16:47.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.714+0000 7fd9cb7fe700 1 -- 172.21.15.101:0/4071649565 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9e0003680 con 0x55bea5da4be0 2023-07-18T03:16:47.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.714+0000 7fda05c56b80 10 client.27472 fetch_fsmap learned FSMap version 33 2023-07-18T03:16:47.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.714+0000 7fda05c56b80 1 -- 172.21.15.101:0/4071649565 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55bea5b0dc60 con 0x55bea5da4be0 2023-07-18T03:16:47.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.714+0000 7fd9cb7fe700 1 -- 172.21.15.101:0/4071649565 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bea5c504f0 con 0x55bea5da4be0 2023-07-18T03:16:47.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.714+0000 7fd9cb7fe700 1 -- 172.21.15.101:0/4071649565 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd9e0011440 con 0x55bea5da4be0 2023-07-18T03:16:47.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.714+0000 7fda05c56b80 10 client.27472 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:16:47.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.714+0000 7fda05c56b80 10 client.27472 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:16:47.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.714+0000 7fda05c56b80 20 client.27472 populate_metadata read hostname 'smithi101' 2023-07-18T03:16:47.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.714+0000 7fda05c56b80 1 --2- 172.21.15.101:0/4071649565 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55bea5dbf480 0x55bea5dc1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.714+0000 7fda05c56b80 4 client.27472 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:16:47.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.714+0000 7fda05c56b80 1 -- 172.21.15.101:0/4071649565 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bea5cefeb0 con 0x55bea5dbf480 2023-07-18T03:16:47.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.715+0000 7fd9ebfff700 1 --2- 172.21.15.101:0/4071649565 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55bea5dbf480 0x55bea5dc1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.715+0000 7fd9ebfff700 1 --2- 172.21.15.101:0/4071649565 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55bea5dbf480 0x55bea5dc1890 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=65f4adc1ada7fc8 server_cookie=b4f1eceb1a2d83b9 in_seq=0 out_seq=0 2023-07-18T03:16:47.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.715+0000 7fd9cb7fe700 10 client.27472 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:16:47.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.716+0000 7fd9cb7fe700 1 -- 172.21.15.101:0/4071649565 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bea5cefeb0 con 0x55bea5dbf480 2023-07-18T03:16:47.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.716+0000 7fd9cb7fe700 10 client.27472 handle_command_reply: tid=0 2023-07-18T03:16:47.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.716+0000 7fda05c56b80 1 client.27472 shutdown 2023-07-18T03:16:47.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.716+0000 7fda05c56b80 1 -- 172.21.15.101:0/4071649565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bea5da4be0 msgr2=0x55bea5dac810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:47.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.716+0000 7fda05c56b80 1 --2- 172.21.15.101:0/4071649565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bea5da4be0 0x55bea5dac810 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd9e000b900 tx=0x7fd9e0010de0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.716+0000 7fda05c56b80 1 -- 172.21.15.101:0/4071649565 shutdown_connections 2023-07-18T03:16:47.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.716+0000 7fda05c56b80 1 --2- 172.21.15.101:0/4071649565 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55bea5dbf480 0x55bea5dc1890 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.716+0000 7fda05c56b80 1 --2- 172.21.15.101:0/4071649565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea5da42d0 0x55bea5daf520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.716+0000 7fda05c56b80 1 --2- 172.21.15.101:0/4071649565 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bea5dafad0 0x55bea5dacf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.716+0000 7fda05c56b80 1 --2- 172.21.15.101:0/4071649565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bea5da4be0 0x55bea5dac810 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.716+0000 7fd9cb7fe700 0 client.27472 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:16:47.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.716+0000 7fda05c56b80 1 -- 172.21.15.101:0/4071649565 >> 172.21.15.101:0/4071649565 conn(0x55bea5d9ada0 msgr2=0x55bea5da0230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:47.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.716+0000 7fda05c56b80 1 -- 172.21.15.101:0/4071649565 shutdown_connections 2023-07-18T03:16:47.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.716+0000 7fda05c56b80 1 -- 172.21.15.101:0/4071649565 wait complete. 2023-07-18T03:16:47.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.717+0000 7fda05c56b80 20 client.27472 trim_cache size 0 max 16384 2023-07-18T03:16:47.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.730+0000 7fda05c56b80 1 Processor -- start 2023-07-18T03:16:47.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.730+0000 7fda05c56b80 1 -- start start 2023-07-18T03:16:47.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.731+0000 7fda05c56b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea5da42d0 0x55bea5dc1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.731+0000 7fda05c56b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bea5da4be0 0x55bea5daf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.731+0000 7fda05c56b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bea5dafad0 0x55bea5da7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.731+0000 7fda05c56b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bea5da76f0 con 0x55bea5da4be0 2023-07-18T03:16:47.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.731+0000 7fda05c56b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bea5d97fd0 con 0x55bea5da42d0 2023-07-18T03:16:47.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.731+0000 7fda05c56b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bea5d9a9a0 con 0x55bea5dafad0 2023-07-18T03:16:47.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.731+0000 7fd9ebfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea5da42d0 0x55bea5dc1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.731+0000 7fd9eb7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bea5da4be0 0x55bea5daf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.731+0000 7fd9ebfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea5da42d0 0x55bea5dc1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60238/0 (socket says 172.21.15.101:60238) 2023-07-18T03:16:47.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.731+0000 7fd9ebfff700 1 -- 172.21.15.101:0/2872059085 learned_addr learned my addr 172.21.15.101:0/2872059085 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:47.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.731+0000 7fd9f099c700 1 --2- 172.21.15.101:0/2872059085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bea5dafad0 0x55bea5da7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.731+0000 7fd9eb7fe700 1 -- 172.21.15.101:0/2872059085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bea5dafad0 msgr2=0x55bea5da7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:47.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.731+0000 7fd9eb7fe700 1 --2- 172.21.15.101:0/2872059085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bea5dafad0 0x55bea5da7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.731+0000 7fd9eb7fe700 1 -- 172.21.15.101:0/2872059085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea5da42d0 msgr2=0x55bea5dc1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:47.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.731+0000 7fd9eb7fe700 1 --2- 172.21.15.101:0/2872059085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea5da42d0 0x55bea5dc1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.731+0000 7fd9eb7fe700 1 -- 172.21.15.101:0/2872059085 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bea5b4b250 con 0x55bea5da4be0 2023-07-18T03:16:47.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.731+0000 7fd9eb7fe700 1 --2- 172.21.15.101:0/2872059085 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bea5da4be0 0x55bea5daf150 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fd9e00117f0 tx=0x7fd9e0005f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdc07614761202f7 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:47.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.732+0000 7fd9d2ffd700 1 -- 172.21.15.101:0/2872059085 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e0020040 con 0x55bea5da4be0 2023-07-18T03:16:47.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.732+0000 7fd9d2ffd700 1 -- 172.21.15.101:0/2872059085 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9e0005710 con 0x55bea5da4be0 2023-07-18T03:16:47.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.732+0000 7fd9d2ffd700 1 -- 172.21.15.101:0/2872059085 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e0003cc0 con 0x55bea5da4be0 2023-07-18T03:16:47.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.732+0000 7fda05c56b80 1 -- 172.21.15.101:0/2872059085 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bea5da4be0 msgr2=0x55bea5daf150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:47.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.732+0000 7fda05c56b80 1 --2- 172.21.15.101:0/2872059085 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bea5da4be0 0x55bea5daf150 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fd9e00117f0 tx=0x7fd9e0005f60 comp rx=0 tx=0).stop 2023-07-18T03:16:47.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.732+0000 7fda05c56b80 1 -- 172.21.15.101:0/2872059085 shutdown_connections 2023-07-18T03:16:47.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.732+0000 7fda05c56b80 1 --2- 172.21.15.101:0/2872059085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea5da42d0 0x55bea5dc1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.732+0000 7fda05c56b80 1 --2- 172.21.15.101:0/2872059085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bea5dafad0 0x55bea5da7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.732+0000 7fda05c56b80 1 --2- 172.21.15.101:0/2872059085 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bea5da4be0 0x55bea5daf150 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.732+0000 7fda05c56b80 1 -- 172.21.15.101:0/2872059085 >> 172.21.15.101:0/2872059085 conn(0x55bea5d9ada0 msgr2=0x55bea5dc61b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:47.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.732+0000 7fda05c56b80 1 -- 172.21.15.101:0/2872059085 shutdown_connections 2023-07-18T03:16:47.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.732+0000 7fda05c56b80 1 -- 172.21.15.101:0/2872059085 wait complete. 2023-07-18T03:16:47.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.733+0000 7fda05c56b80 1 -- start start 2023-07-18T03:16:47.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.733+0000 7fda05c56b80 1 Processor -- start 2023-07-18T03:16:47.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.733+0000 7fda05c56b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea5da42d0 0x55bea5db3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.733+0000 7fda05c56b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bea5da4be0 0x55bea5db40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.733+0000 7fda05c56b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bea5dafad0 0x55bea5db45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.733+0000 7fda05c56b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bea5e12300 con 0x55bea5da4be0 2023-07-18T03:16:47.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.733+0000 7fda05c56b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bea5e12440 con 0x55bea5da42d0 2023-07-18T03:16:47.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.733+0000 7fda05c56b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bea5e12580 con 0x55bea5dafad0 2023-07-18T03:16:47.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.733+0000 7fda05c56b80 10 client.0 _ll_register_callbacks cb 0x7fd9dc005c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:16:47.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.734+0000 7fd9f099c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bea5dafad0 0x55bea5db45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.734+0000 7fd9f099c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bea5dafad0 0x55bea5db45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:47564/0 (socket says 172.21.15.101:47564) 2023-07-18T03:16:47.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.734+0000 7fd9f099c700 1 -- 172.21.15.101:0/2710502393 learned_addr learned my addr 172.21.15.101:0/2710502393 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:47.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.734+0000 7fd9ebfff700 1 --2- 172.21.15.101:0/2710502393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea5da42d0 0x55bea5db3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.734+0000 7fd9eb7fe700 1 --2- 172.21.15.101:0/2710502393 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bea5da4be0 0x55bea5db40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.734+0000 7fd9eb7fe700 1 -- 172.21.15.101:0/2710502393 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bea5dafad0 msgr2=0x55bea5db45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:47.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.734+0000 7fd9eb7fe700 1 --2- 172.21.15.101:0/2710502393 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bea5dafad0 0x55bea5db45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.734+0000 7fd9eb7fe700 1 -- 172.21.15.101:0/2710502393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea5da42d0 msgr2=0x55bea5db3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:47.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.734+0000 7fd9eb7fe700 1 --2- 172.21.15.101:0/2710502393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea5da42d0 0x55bea5db3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.734+0000 7fd9eb7fe700 1 -- 172.21.15.101:0/2710502393 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bea5b0dc60 con 0x55bea5da4be0 2023-07-18T03:16:47.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.734+0000 7fd9ebfff700 1 --2- 172.21.15.101:0/2710502393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea5da42d0 0x55bea5db3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:16:47.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.734+0000 7fd9eb7fe700 1 --2- 172.21.15.101:0/2710502393 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bea5da4be0 0x55bea5db40a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x55bea5da73b0 tx=0x7fd9e00106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a990c8721a8699c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:47.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.734+0000 7fd9cb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:16:47.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.735+0000 7fd9cb7fe700 1 -- 172.21.15.101:0/2710502393 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e0020040 con 0x55bea5da4be0 2023-07-18T03:16:47.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.735+0000 7fda05c56b80 1 -- 172.21.15.101:0/2710502393 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd9e0015070 con 0x55bea5da4be0 2023-07-18T03:16:47.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.735+0000 7fd9cb7fe700 1 -- 172.21.15.101:0/2710502393 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fd9e0017070 con 0x55bea5da4be0 2023-07-18T03:16:47.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.735+0000 7fd9cb7fe700 1 -- 172.21.15.101:0/2710502393 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9e0003900 con 0x55bea5da4be0 2023-07-18T03:16:47.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.735+0000 7fda05c56b80 10 client.17403 fetch_fsmap learned FSMap version 33 2023-07-18T03:16:47.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.735+0000 7fda05c56b80 1 -- 172.21.15.101:0/2710502393 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55bea5b232b0 con 0x55bea5da4be0 2023-07-18T03:16:47.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.735+0000 7fd9cb7fe700 1 -- 172.21.15.101:0/2710502393 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9e0015070 con 0x55bea5da4be0 2023-07-18T03:16:47.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.735+0000 7fd9cb7fe700 1 -- 172.21.15.101:0/2710502393 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd9e003da00 con 0x55bea5da4be0 2023-07-18T03:16:47.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.735+0000 7fda05c56b80 10 client.17403 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:16:47.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.735+0000 7fda05c56b80 10 client.17403 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:16:47.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.735+0000 7fda05c56b80 20 client.17403 populate_metadata read hostname 'smithi101' 2023-07-18T03:16:47.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.735+0000 7fda05c56b80 1 --2- 172.21.15.101:0/2710502393 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55bea5e13880 0x55bea5e15c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.735+0000 7fda05c56b80 4 client.17403 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:16:47.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.735+0000 7fda05c56b80 1 -- 172.21.15.101:0/2710502393 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bea5cb2050 con 0x55bea5e13880 2023-07-18T03:16:47.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.736+0000 7fd9ebfff700 1 --2- 172.21.15.101:0/2710502393 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55bea5e13880 0x55bea5e15c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.736+0000 7fd9ebfff700 1 --2- 172.21.15.101:0/2710502393 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55bea5e13880 0x55bea5e15c90 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=82ebf98998e9d015 server_cookie=95ef843ce7e9946 in_seq=0 out_seq=0 2023-07-18T03:16:47.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.736+0000 7fd9cb7fe700 10 client.17403 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:16:47.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.736+0000 7fd9cb7fe700 1 -- 172.21.15.101:0/2710502393 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bea5cb2050 con 0x55bea5e13880 2023-07-18T03:16:47.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.736+0000 7fd9cb7fe700 10 client.17403 handle_command_reply: tid=0 2023-07-18T03:16:47.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.736+0000 7fda05c56b80 1 client.17403 shutdown 2023-07-18T03:16:47.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.737+0000 7fda05c56b80 1 -- 172.21.15.101:0/2710502393 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bea5da4be0 msgr2=0x55bea5db40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:47.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.737+0000 7fda05c56b80 1 --2- 172.21.15.101:0/2710502393 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bea5da4be0 0x55bea5db40a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x55bea5da73b0 tx=0x7fd9e00106b0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.737+0000 7fda05c56b80 1 -- 172.21.15.101:0/2710502393 shutdown_connections 2023-07-18T03:16:47.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.737+0000 7fda05c56b80 1 --2- 172.21.15.101:0/2710502393 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55bea5e13880 0x55bea5e15c90 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.737+0000 7fda05c56b80 1 --2- 172.21.15.101:0/2710502393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea5da42d0 0x55bea5db3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.737+0000 7fda05c56b80 1 --2- 172.21.15.101:0/2710502393 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bea5dafad0 0x55bea5db45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.737+0000 7fda05c56b80 1 --2- 172.21.15.101:0/2710502393 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bea5da4be0 0x55bea5db40a0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.737+0000 7fd9cb7fe700 0 client.17403 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:16:47.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.737+0000 7fda05c56b80 1 -- 172.21.15.101:0/2710502393 >> 172.21.15.101:0/2710502393 conn(0x55bea5d9ada0 msgr2=0x55bea5da7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:47.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.737+0000 7fda05c56b80 1 -- 172.21.15.101:0/2710502393 shutdown_connections 2023-07-18T03:16:47.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.737+0000 7fda05c56b80 1 -- 172.21.15.101:0/2710502393 wait complete. 2023-07-18T03:16:47.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.737+0000 7fda05c56b80 20 client.17403 trim_cache size 0 max 16384 2023-07-18T03:16:47.739 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:16:47.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.738+0000 7fd9f23ff700 1 -- 172.21.15.101:0/2208648727 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd9d40a12c0 msgr2=0x7fd9d40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:47.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.738+0000 7fd9f23ff700 1 --2- 172.21.15.101:0/2208648727 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd9d40a12c0 0x7fd9d40a3770 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4009ec0 tx=0x7fd9e400b040 comp rx=0 tx=0).stop 2023-07-18T03:16:47.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.738+0000 7fd9f23ff700 1 -- 172.21.15.101:0/2208648727 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd9ec0ffae0 msgr2=0x7fd9ec072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:47.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.738+0000 7fd9f23ff700 1 --2- 172.21.15.101:0/2208648727 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd9ec0ffae0 0x7fd9ec072810 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc005910 tx=0x7fd9dc004730 comp rx=0 tx=0).stop 2023-07-18T03:16:47.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.738+0000 7fd9f23ff700 1 -- 172.21.15.101:0/2208648727 shutdown_connections 2023-07-18T03:16:47.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.738+0000 7fd9f23ff700 1 --2- 172.21.15.101:0/2208648727 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9ec105af0 0x7fd9ec1a3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.738+0000 7fd9f23ff700 1 --2- 172.21.15.101:0/2208648727 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd9ec0ffae0 0x7fd9ec072810 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.738+0000 7fd9f23ff700 1 --2- 172.21.15.101:0/2208648727 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd9d40a12c0 0x7fd9d40a3770 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.738+0000 7fd9f23ff700 1 --2- 172.21.15.101:0/2208648727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9ec100490 0x7fd9ec072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.738+0000 7fd9f23ff700 1 -- 172.21.15.101:0/2208648727 >> 172.21.15.101:0/2208648727 conn(0x7fd9ec0fb5e0 msgr2=0x7fd9ec1035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:47.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.738+0000 7fd9f23ff700 1 -- 172.21.15.101:0/2208648727 shutdown_connections 2023-07-18T03:16:47.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.738+0000 7fd9f23ff700 1 -- 172.21.15.101:0/2208648727 wait complete. 2023-07-18T03:16:47.749 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-18T03:16:47.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.958+0000 7f3d523ff700 1 -- 172.21.15.101:0/3338823809 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3d4c105ae0 msgr2=0x7f3d4c105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:47.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.958+0000 7f3d523ff700 1 --2- 172.21.15.101:0/3338823809 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3d4c105ae0 0x7f3d4c105eb0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c009990 tx=0x7f3d3c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.958+0000 7f3d523ff700 1 -- 172.21.15.101:0/3338823809 shutdown_connections 2023-07-18T03:16:47.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.958+0000 7f3d523ff700 1 --2- 172.21.15.101:0/3338823809 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d4c100480 0x7f3d4c10c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.958+0000 7f3d523ff700 1 --2- 172.21.15.101:0/3338823809 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3d4c105ae0 0x7f3d4c105eb0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.958+0000 7f3d523ff700 1 --2- 172.21.15.101:0/3338823809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d4c0ffad0 0x7f3d4c0fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.958+0000 7f3d523ff700 1 -- 172.21.15.101:0/3338823809 >> 172.21.15.101:0/3338823809 conn(0x7f3d4c0fb5f0 msgr2=0x7f3d4c0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:47.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.958+0000 7f3d523ff700 1 -- 172.21.15.101:0/3338823809 shutdown_connections 2023-07-18T03:16:47.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.958+0000 7f3d523ff700 1 -- 172.21.15.101:0/3338823809 wait complete. 2023-07-18T03:16:47.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.959+0000 7f3d523ff700 1 Processor -- start 2023-07-18T03:16:47.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.959+0000 7f3d523ff700 1 -- start start 2023-07-18T03:16:47.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.960+0000 7f3d523ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3d4c0ffad0 0x7f3d4c19b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.960+0000 7f3d523ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d4c100480 0x7f3d4c19beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.960+0000 7f3d523ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d4c105ae0 0x7f3d4c1959e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.960+0000 7f3d523ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4c196050 con 0x7f3d4c0ffad0 2023-07-18T03:16:47.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.960+0000 7f3d523ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4c1961c0 con 0x7f3d4c100480 2023-07-18T03:16:47.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.960+0000 7f3d523ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4c196380 con 0x7f3d4c105ae0 2023-07-18T03:16:47.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.960+0000 7f3d4bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3d4c0ffad0 0x7f3d4c19b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.960+0000 7f3d4bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3d4c0ffad0 0x7f3d4c19b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:50566/0 (socket says 172.21.15.101:50566) 2023-07-18T03:16:47.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.960+0000 7f3d4bfff700 1 -- 172.21.15.101:0/3618675550 learned_addr learned my addr 172.21.15.101:0/3618675550 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:47.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.960+0000 7f3d5099c700 1 --2- 172.21.15.101:0/3618675550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d4c105ae0 0x7f3d4c1959e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.960+0000 7f3d43fff700 1 --2- 172.21.15.101:0/3618675550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d4c100480 0x7f3d4c19beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.960+0000 7f3d4bfff700 1 -- 172.21.15.101:0/3618675550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d4c105ae0 msgr2=0x7f3d4c1959e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:47.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.960+0000 7f3d4bfff700 1 --2- 172.21.15.101:0/3618675550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d4c105ae0 0x7f3d4c1959e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.960+0000 7f3d4bfff700 1 -- 172.21.15.101:0/3618675550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d4c100480 msgr2=0x7f3d4c19beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:47.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.960+0000 7f3d4bfff700 1 --2- 172.21.15.101:0/3618675550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d4c100480 0x7f3d4c19beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.960+0000 7f3d4bfff700 1 -- 172.21.15.101:0/3618675550 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d3c0095f0 con 0x7f3d4c0ffad0 2023-07-18T03:16:47.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.960+0000 7f3d4bfff700 1 --2- 172.21.15.101:0/3618675550 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3d4c0ffad0 0x7f3d4c19b970 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c000c00 tx=0x7f3d3c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:47.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.961+0000 7f3d49ffb700 1 -- 172.21.15.101:0/3618675550 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c01d070 con 0x7f3d4c0ffad0 2023-07-18T03:16:47.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.961+0000 7f3d49ffb700 1 -- 172.21.15.101:0/3618675550 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d3c00fb80 con 0x7f3d4c0ffad0 2023-07-18T03:16:47.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.961+0000 7f3d49ffb700 1 -- 172.21.15.101:0/3618675550 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c0173f0 con 0x7f3d4c0ffad0 2023-07-18T03:16:47.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.961+0000 7f3d523ff700 1 -- 172.21.15.101:0/3618675550 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d4c196600 con 0x7f3d4c0ffad0 2023-07-18T03:16:47.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.961+0000 7f3d523ff700 1 -- 172.21.15.101:0/3618675550 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d4c1a2960 con 0x7f3d4c0ffad0 2023-07-18T03:16:47.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.964+0000 7f3d49ffb700 1 -- 172.21.15.101:0/3618675550 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f3d3c00fcf0 con 0x7f3d4c0ffad0 2023-07-18T03:16:47.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.966+0000 7f3d65da3b80 1 Processor -- start 2023-07-18T03:16:47.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.966+0000 7f3d65da3b80 1 -- start start 2023-07-18T03:16:47.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.966+0000 7f3d65da3b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589711e72d0 0x5589711e76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.967+0000 7f3d65da3b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5589711e7be0 0x5589711f2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.967+0000 7f3d65da3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5589711f2ad0 0x5589711f4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.967+0000 7f3d49ffb700 1 --2- 172.21.15.101:0/3618675550 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f3d380a1340 0x7f3d380a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.967+0000 7f3d65da3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589710edcb0 con 0x5589711f2ad0 2023-07-18T03:16:47.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.967+0000 7f3d65da3b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558971140a40 con 0x5589711e72d0 2023-07-18T03:16:47.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.967+0000 7f3d65da3b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x558971139ed0 con 0x5589711e7be0 2023-07-18T03:16:47.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.967+0000 7f3d49ffb700 1 -- 172.21.15.101:0/3618675550 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f3d3c0ce270 con 0x7f3d4c0ffad0 2023-07-18T03:16:47.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.967+0000 7f3d43fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5589711f2ad0 0x5589711f4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.967+0000 7f3d5099c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589711e72d0 0x5589711e76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.967+0000 7f3d43fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5589711f2ad0 0x5589711f4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:50582/0 (socket says 172.21.15.101:50582) 2023-07-18T03:16:47.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.967+0000 7f3d43fff700 1 -- 172.21.15.101:0/3331458538 learned_addr learned my addr 172.21.15.101:0/3331458538 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:47.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.967+0000 7f3d4bfff700 1 --2- 172.21.15.101:0/3331458538 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5589711e7be0 0x5589711f2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.967+0000 7f3d5099c700 1 --2- 172.21.15.101:0/3618675550 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f3d380a1340 0x7f3d380a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.967+0000 7f3d4bfff700 1 -- 172.21.15.101:0/3331458538 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589711e72d0 msgr2=0x5589711e76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:47.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.967+0000 7f3d4bfff700 1 --2- 172.21.15.101:0/3331458538 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589711e72d0 0x5589711e76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.967+0000 7f3d4bfff700 1 -- 172.21.15.101:0/3331458538 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5589711f2ad0 msgr2=0x5589711f4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:47.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.967+0000 7f3d4bfff700 1 --2- 172.21.15.101:0/3331458538 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5589711f2ad0 0x5589711f4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.967+0000 7f3d4bfff700 1 -- 172.21.15.101:0/3331458538 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558970f67de0 con 0x5589711e7be0 2023-07-18T03:16:47.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.968+0000 7f3d4bfff700 1 --2- 172.21.15.101:0/3331458538 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5589711e7be0 0x5589711f2590 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c0cb5b0 tx=0x7f3d3c005ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e78ce0a605cb882 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:47.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.968+0000 7f3d5099c700 1 --2- 172.21.15.101:0/3618675550 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f3d380a1340 0x7f3d380a37f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3d44009550 tx=0x7f3d4400b040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:47.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.968+0000 7f3d427fc700 1 -- 172.21.15.101:0/3331458538 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c09f110 con 0x5589711e7be0 2023-07-18T03:16:47.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.968+0000 7f3d427fc700 1 -- 172.21.15.101:0/3331458538 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d3c0a34a0 con 0x5589711e7be0 2023-07-18T03:16:47.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.968+0000 7f3d427fc700 1 -- 172.21.15.101:0/3331458538 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c09a5d0 con 0x5589711e7be0 2023-07-18T03:16:47.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.968+0000 7f3d65da3b80 1 -- 172.21.15.101:0/3331458538 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5589711e7be0 msgr2=0x5589711f2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:47.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.968+0000 7f3d65da3b80 1 --2- 172.21.15.101:0/3331458538 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5589711e7be0 0x5589711f2590 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c0cb5b0 tx=0x7f3d3c005ad0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.969+0000 7f3d65da3b80 1 -- 172.21.15.101:0/3331458538 shutdown_connections 2023-07-18T03:16:47.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.969+0000 7f3d65da3b80 1 --2- 172.21.15.101:0/3331458538 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589711e72d0 0x5589711e76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.969+0000 7f3d65da3b80 1 --2- 172.21.15.101:0/3331458538 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5589711f2ad0 0x5589711f4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.969+0000 7f3d65da3b80 1 --2- 172.21.15.101:0/3331458538 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5589711e7be0 0x5589711f2590 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.969+0000 7f3d65da3b80 1 -- 172.21.15.101:0/3331458538 >> 172.21.15.101:0/3331458538 conn(0x5589711ddda0 msgr2=0x5589711e36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:47.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.969+0000 7f3d65da3b80 1 -- 172.21.15.101:0/3331458538 shutdown_connections 2023-07-18T03:16:47.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.969+0000 7f3d65da3b80 1 -- 172.21.15.101:0/3331458538 wait complete. 2023-07-18T03:16:47.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.969+0000 7f3d65da3b80 1 -- start start 2023-07-18T03:16:47.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.969+0000 7f3d65da3b80 1 Processor -- start 2023-07-18T03:16:47.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.970+0000 7f3d65da3b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589711e72d0 0x5589711f7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.970+0000 7f3d65da3b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5589711e7be0 0x5589711f8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.970+0000 7f3d65da3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5589711f2ad0 0x5589711f1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.970+0000 7f3d65da3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589710d9ac0 con 0x5589711f2ad0 2023-07-18T03:16:47.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.970+0000 7f3d65da3b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589710d97c0 con 0x5589711e72d0 2023-07-18T03:16:47.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.970+0000 7f3d65da3b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x558970fbd920 con 0x5589711e7be0 2023-07-18T03:16:47.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.970+0000 7f3d65da3b80 10 client.0 _ll_register_callbacks cb 0x5589710d7760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:16:47.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.970+0000 7f3d4bfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589711e72d0 0x5589711f7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.970+0000 7f3d5099c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5589711f2ad0 0x5589711f1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.970+0000 7f3d4bfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589711e72d0 0x5589711f7c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60278/0 (socket says 172.21.15.101:60278) 2023-07-18T03:16:47.970 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.970+0000 7f3d43fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5589711e7be0 0x5589711f8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.970 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.970+0000 7f3d4bfff700 1 -- 172.21.15.101:0/3164690182 learned_addr learned my addr 172.21.15.101:0/3164690182 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:47.970 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.971+0000 7f3d4bfff700 1 -- 172.21.15.101:0/3164690182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5589711e7be0 msgr2=0x5589711f8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:47.970 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.971+0000 7f3d4bfff700 1 --2- 172.21.15.101:0/3164690182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5589711e7be0 0x5589711f8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.970 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.971+0000 7f3d4bfff700 1 -- 172.21.15.101:0/3164690182 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5589711f2ad0 msgr2=0x5589711f1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:47.970 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.971+0000 7f3d4bfff700 1 --2- 172.21.15.101:0/3164690182 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5589711f2ad0 0x5589711f1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.971+0000 7f3d4bfff700 1 -- 172.21.15.101:0/3164690182 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d4c1a2960 con 0x5589711e72d0 2023-07-18T03:16:47.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.971+0000 7f3d43fff700 1 --2- 172.21.15.101:0/3164690182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5589711e7be0 0x5589711f8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:16:47.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.971+0000 7f3d4bfff700 1 --2- 172.21.15.101:0/3164690182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589711e72d0 0x5589711f7c20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c0cb4c0 tx=0x7f3d3c009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40975e5521080c6c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:47.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.971+0000 7f3d2b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:16:47.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.971+0000 7f3d2b7fe700 1 -- 172.21.15.101:0/3164690182 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c09f110 con 0x5589711e72d0 2023-07-18T03:16:47.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.971+0000 7f3d2b7fe700 1 -- 172.21.15.101:0/3164690182 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f3d3c09b550 con 0x5589711e72d0 2023-07-18T03:16:47.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.971+0000 7f3d65da3b80 1 -- 172.21.15.101:0/3164690182 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5589710934f0 con 0x5589711e72d0 2023-07-18T03:16:47.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.971+0000 7f3d2b7fe700 1 -- 172.21.15.101:0/3164690182 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d3c0b92d0 con 0x5589711e72d0 2023-07-18T03:16:47.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.971+0000 7f3d65da3b80 10 client.27455 fetch_fsmap learned FSMap version 33 2023-07-18T03:16:47.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.971+0000 7f3d65da3b80 1 -- 172.21.15.101:0/3164690182 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x558970f50c60 con 0x5589711e72d0 2023-07-18T03:16:47.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.972+0000 7f3d2b7fe700 1 -- 172.21.15.101:0/3164690182 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5589710934f0 con 0x5589711e72d0 2023-07-18T03:16:47.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.972+0000 7f3d2b7fe700 1 -- 172.21.15.101:0/3164690182 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3d3c09a040 con 0x5589711e72d0 2023-07-18T03:16:47.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.972+0000 7f3d65da3b80 10 client.27455 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:16:47.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.972+0000 7f3d65da3b80 10 client.27455 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:16:47.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.972+0000 7f3d65da3b80 20 client.27455 populate_metadata read hostname 'smithi101' 2023-07-18T03:16:47.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.972+0000 7f3d65da3b80 1 --2- 172.21.15.101:0/3164690182 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x558971201ba0 0x558971203f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.972+0000 7f3d65da3b80 4 client.27455 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:16:47.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.972+0000 7f3d65da3b80 1 -- 172.21.15.101:0/3164690182 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558971132eb0 con 0x558971201ba0 2023-07-18T03:16:47.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.972+0000 7f3d43fff700 1 --2- 172.21.15.101:0/3164690182 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x558971201ba0 0x558971203f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.973+0000 7f3d2b7fe700 10 client.27455 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:16:47.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.973+0000 7f3d43fff700 1 --2- 172.21.15.101:0/3164690182 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x558971201ba0 0x558971203f80 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=689edde48979c51c server_cookie=735a70ae9fcf9c82 in_seq=0 out_seq=0 2023-07-18T03:16:47.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.973+0000 7f3d2b7fe700 1 -- 172.21.15.101:0/3164690182 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558971132eb0 con 0x558971201ba0 2023-07-18T03:16:47.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.973+0000 7f3d2b7fe700 10 client.27455 handle_command_reply: tid=0 2023-07-18T03:16:47.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.973+0000 7f3d65da3b80 1 client.27455 shutdown 2023-07-18T03:16:47.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.974+0000 7f3d65da3b80 1 -- 172.21.15.101:0/3164690182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589711e72d0 msgr2=0x5589711f7c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:47.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.974+0000 7f3d65da3b80 1 --2- 172.21.15.101:0/3164690182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589711e72d0 0x5589711f7c20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c0cb4c0 tx=0x7f3d3c009310 comp rx=0 tx=0).stop 2023-07-18T03:16:47.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.974+0000 7f3d65da3b80 1 -- 172.21.15.101:0/3164690182 shutdown_connections 2023-07-18T03:16:47.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.974+0000 7f3d65da3b80 1 --2- 172.21.15.101:0/3164690182 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x558971201ba0 0x558971203f80 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.974+0000 7f3d65da3b80 1 --2- 172.21.15.101:0/3164690182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589711e72d0 0x5589711f7c20 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.974+0000 7f3d65da3b80 1 --2- 172.21.15.101:0/3164690182 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5589711f2ad0 0x5589711f1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.974+0000 7f3d65da3b80 1 --2- 172.21.15.101:0/3164690182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5589711e7be0 0x5589711f8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.974+0000 7f3d2b7fe700 0 client.27455 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:16:47.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.974+0000 7f3d65da3b80 1 -- 172.21.15.101:0/3164690182 >> 172.21.15.101:0/3164690182 conn(0x5589711ddda0 msgr2=0x5589711e3290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:47.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.974+0000 7f3d65da3b80 1 -- 172.21.15.101:0/3164690182 shutdown_connections 2023-07-18T03:16:47.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.974+0000 7f3d65da3b80 1 -- 172.21.15.101:0/3164690182 wait complete. 2023-07-18T03:16:47.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.974+0000 7f3d65da3b80 20 client.27455 trim_cache size 0 max 16384 2023-07-18T03:16:47.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.987+0000 7f3d65da3b80 1 Processor -- start 2023-07-18T03:16:47.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.987+0000 7f3d65da3b80 1 -- start start 2023-07-18T03:16:47.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.987+0000 7f3d65da3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5589711e72d0 0x5589711f77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.987+0000 7f3d65da3b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589711e7be0 0x5589711f7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.987+0000 7f3d65da3b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5589711f2ad0 0x5589711eed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.987+0000 7f3d65da3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589711e8540 con 0x5589711e72d0 2023-07-18T03:16:47.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.988+0000 7f3d65da3b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589711f3450 con 0x5589711e7be0 2023-07-18T03:16:47.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.988+0000 7f3d65da3b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589711e8d60 con 0x5589711f2ad0 2023-07-18T03:16:47.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.988+0000 7f3d4bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5589711e72d0 0x5589711f77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.988+0000 7f3d4bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5589711e72d0 0x5589711f77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:50598/0 (socket says 172.21.15.101:50598) 2023-07-18T03:16:47.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.988+0000 7f3d4bfff700 1 -- 172.21.15.101:0/705085647 learned_addr learned my addr 172.21.15.101:0/705085647 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:47.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.988+0000 7f3d5099c700 1 --2- 172.21.15.101:0/705085647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5589711f2ad0 0x5589711eed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.988+0000 7f3d43fff700 1 --2- 172.21.15.101:0/705085647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589711e7be0 0x5589711f7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.988+0000 7f3d4bfff700 1 -- 172.21.15.101:0/705085647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5589711f2ad0 msgr2=0x5589711eed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:47.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.988+0000 7f3d4bfff700 1 --2- 172.21.15.101:0/705085647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5589711f2ad0 0x5589711eed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.988+0000 7f3d4bfff700 1 -- 172.21.15.101:0/705085647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589711e7be0 msgr2=0x5589711f7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:47.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.988+0000 7f3d4bfff700 1 --2- 172.21.15.101:0/705085647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589711e7be0 0x5589711f7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.988+0000 7f3d4bfff700 1 -- 172.21.15.101:0/705085647 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558970f8e250 con 0x5589711e72d0 2023-07-18T03:16:47.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.988+0000 7f3d5099c700 1 --2- 172.21.15.101:0/705085647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5589711f2ad0 0x5589711eed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:16:47.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.988+0000 7f3d4bfff700 1 --2- 172.21.15.101:0/705085647 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5589711e72d0 0x5589711f77b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c0c6c50 tx=0x7f3d3c005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64c65c1722781343 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:47.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.988+0000 7f3d42ffd700 1 -- 172.21.15.101:0/705085647 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c07c070 con 0x5589711e72d0 2023-07-18T03:16:47.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.989+0000 7f3d42ffd700 1 -- 172.21.15.101:0/705085647 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d3c0cb4c0 con 0x5589711e72d0 2023-07-18T03:16:47.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.989+0000 7f3d42ffd700 1 -- 172.21.15.101:0/705085647 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c0b92d0 con 0x5589711e72d0 2023-07-18T03:16:47.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.989+0000 7f3d65da3b80 1 -- 172.21.15.101:0/705085647 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5589711e72d0 msgr2=0x5589711f77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:47.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.989+0000 7f3d65da3b80 1 --2- 172.21.15.101:0/705085647 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5589711e72d0 0x5589711f77b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c0c6c50 tx=0x7f3d3c005ad0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.989+0000 7f3d65da3b80 1 -- 172.21.15.101:0/705085647 shutdown_connections 2023-07-18T03:16:47.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.989+0000 7f3d65da3b80 1 --2- 172.21.15.101:0/705085647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5589711f2ad0 0x5589711eed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.989+0000 7f3d65da3b80 1 --2- 172.21.15.101:0/705085647 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5589711e72d0 0x5589711f77b0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.989+0000 7f3d65da3b80 1 --2- 172.21.15.101:0/705085647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589711e7be0 0x5589711f7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.989+0000 7f3d65da3b80 1 -- 172.21.15.101:0/705085647 >> 172.21.15.101:0/705085647 conn(0x5589711ddda0 msgr2=0x5589711e9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:47.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.989+0000 7f3d65da3b80 1 -- 172.21.15.101:0/705085647 shutdown_connections 2023-07-18T03:16:47.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.989+0000 7f3d65da3b80 1 -- 172.21.15.101:0/705085647 wait complete. 2023-07-18T03:16:47.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.990+0000 7f3d65da3b80 1 -- start start 2023-07-18T03:16:47.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.990+0000 7f3d65da3b80 1 Processor -- start 2023-07-18T03:16:47.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.990+0000 7f3d65da3b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589711e72d0 0x55897124ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.990+0000 7f3d65da3b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5589711e7be0 0x55897124f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.990+0000 7f3d65da3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5589711f2ad0 0x55897124d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.990+0000 7f3d65da3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55897124dea0 con 0x5589711f2ad0 2023-07-18T03:16:47.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.990+0000 7f3d65da3b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55897124e010 con 0x5589711e72d0 2023-07-18T03:16:47.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.990+0000 7f3d65da3b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55897124e1d0 con 0x5589711e7be0 2023-07-18T03:16:47.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.990+0000 7f3d65da3b80 10 client.0 _ll_register_callbacks cb 0x7f3d3400b4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:16:47.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.991+0000 7f3d5099c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5589711f2ad0 0x55897124d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.991+0000 7f3d43fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5589711e7be0 0x55897124f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.991+0000 7f3d43fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5589711e7be0 0x55897124f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:47636/0 (socket says 172.21.15.101:47636) 2023-07-18T03:16:47.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.991+0000 7f3d5099c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5589711f2ad0 0x55897124d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:50600/0 (socket says 172.21.15.101:50600) 2023-07-18T03:16:47.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.991+0000 7f3d43fff700 1 -- 172.21.15.101:0/915331752 learned_addr learned my addr 172.21.15.101:0/915331752 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:47.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.991+0000 7f3d4bfff700 1 --2- 172.21.15.101:0/915331752 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589711e72d0 0x55897124ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.991+0000 7f3d43fff700 1 -- 172.21.15.101:0/915331752 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589711e72d0 msgr2=0x55897124ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:47.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.991+0000 7f3d43fff700 1 --2- 172.21.15.101:0/915331752 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589711e72d0 0x55897124ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.991+0000 7f3d43fff700 1 -- 172.21.15.101:0/915331752 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5589711f2ad0 msgr2=0x55897124d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:47.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.991+0000 7f3d43fff700 1 --2- 172.21.15.101:0/915331752 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5589711f2ad0 0x55897124d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.991+0000 7f3d43fff700 1 -- 172.21.15.101:0/915331752 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558970f50c60 con 0x5589711e7be0 2023-07-18T03:16:47.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.991+0000 7f3d43fff700 1 --2- 172.21.15.101:0/915331752 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5589711e7be0 0x55897124f130 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x558971204790 tx=0x7f3d3401c5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4163ba70ef70ce6 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:47.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.991+0000 7f3d2b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:16:47.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.991+0000 7f3d2b7fe700 1 -- 172.21.15.101:0/915331752 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d34017040 con 0x5589711e7be0 2023-07-18T03:16:47.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.991+0000 7f3d65da3b80 1 -- 172.21.15.101:0/915331752 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5589710f5050 con 0x5589711e7be0 2023-07-18T03:16:47.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.991+0000 7f3d2b7fe700 1 -- 172.21.15.101:0/915331752 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f3d3401ce20 con 0x5589711e7be0 2023-07-18T03:16:47.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.992+0000 7f3d2b7fe700 1 -- 172.21.15.101:0/915331752 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d34004030 con 0x5589711e7be0 2023-07-18T03:16:47.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.992+0000 7f3d65da3b80 10 client.27496 fetch_fsmap learned FSMap version 33 2023-07-18T03:16:47.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.992+0000 7f3d65da3b80 1 -- 172.21.15.101:0/915331752 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x558970f662b0 con 0x5589711e7be0 2023-07-18T03:16:47.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.992+0000 7f3d2b7fe700 1 -- 172.21.15.101:0/915331752 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5589710f5050 con 0x5589711e7be0 2023-07-18T03:16:47.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.992+0000 7f3d2b7fe700 1 -- 172.21.15.101:0/915331752 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3d34003680 con 0x5589711e7be0 2023-07-18T03:16:47.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.992+0000 7f3d65da3b80 10 client.27496 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:16:47.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.992+0000 7f3d65da3b80 10 client.27496 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:16:47.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.992+0000 7f3d65da3b80 20 client.27496 populate_metadata read hostname 'smithi101' 2023-07-18T03:16:47.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.992+0000 7f3d65da3b80 1 --2- 172.21.15.101:0/915331752 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x558971256c60 0x558971259070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:47.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.992+0000 7f3d65da3b80 4 client.27496 mds_command: new command op to 24562 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:16:47.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.992+0000 7f3d65da3b80 1 -- 172.21.15.101:0/915331752 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558970f88f70 con 0x558971256c60 2023-07-18T03:16:47.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.993+0000 7f3d4bfff700 1 --2- 172.21.15.101:0/915331752 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x558971256c60 0x558971259070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:47.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.993+0000 7f3d4bfff700 1 --2- 172.21.15.101:0/915331752 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x558971256c60 0x558971259070 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e0f749a39243ba7 server_cookie=d110cd9a0814a374 in_seq=0 out_seq=0 2023-07-18T03:16:47.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.993+0000 7f3d2b7fe700 10 client.27496 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:16:47.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.993+0000 7f3d2b7fe700 1 -- 172.21.15.101:0/915331752 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558970f88f70 con 0x558971256c60 2023-07-18T03:16:47.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.993+0000 7f3d2b7fe700 10 client.27496 handle_command_reply: tid=0 2023-07-18T03:16:47.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.993+0000 7f3d65da3b80 1 client.27496 shutdown 2023-07-18T03:16:47.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.993+0000 7f3d65da3b80 1 -- 172.21.15.101:0/915331752 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5589711e7be0 msgr2=0x55897124f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:47.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.993+0000 7f3d65da3b80 1 --2- 172.21.15.101:0/915331752 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5589711e7be0 0x55897124f130 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x558971204790 tx=0x7f3d3401c5f0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.993+0000 7f3d65da3b80 1 -- 172.21.15.101:0/915331752 shutdown_connections 2023-07-18T03:16:47.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.993+0000 7f3d65da3b80 1 --2- 172.21.15.101:0/915331752 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x558971256c60 0x558971259070 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.993+0000 7f3d65da3b80 1 --2- 172.21.15.101:0/915331752 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589711e72d0 0x55897124ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.993+0000 7f3d65da3b80 1 --2- 172.21.15.101:0/915331752 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5589711f2ad0 0x55897124d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.993+0000 7f3d65da3b80 1 --2- 172.21.15.101:0/915331752 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5589711e7be0 0x55897124f130 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.993+0000 7f3d2b7fe700 0 client.27496 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:16:47.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.993+0000 7f3d65da3b80 1 -- 172.21.15.101:0/915331752 >> 172.21.15.101:0/915331752 conn(0x5589711ddda0 msgr2=0x5589711e28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:47.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.994+0000 7f3d65da3b80 1 -- 172.21.15.101:0/915331752 shutdown_connections 2023-07-18T03:16:47.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.994+0000 7f3d65da3b80 1 -- 172.21.15.101:0/915331752 wait complete. 2023-07-18T03:16:47.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.994+0000 7f3d65da3b80 20 client.27496 trim_cache size 0 max 16384 2023-07-18T03:16:47.996 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:16:47.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.994+0000 7f3d523ff700 1 -- 172.21.15.101:0/3618675550 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f3d380a1340 msgr2=0x7f3d380a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:47.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.994+0000 7f3d523ff700 1 --2- 172.21.15.101:0/3618675550 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f3d380a1340 0x7f3d380a37f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3d44009550 tx=0x7f3d4400b040 comp rx=0 tx=0).stop 2023-07-18T03:16:47.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.994+0000 7f3d523ff700 1 -- 172.21.15.101:0/3618675550 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3d4c0ffad0 msgr2=0x7f3d4c19b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:47.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.994+0000 7f3d523ff700 1 --2- 172.21.15.101:0/3618675550 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3d4c0ffad0 0x7f3d4c19b970 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c000c00 tx=0x7f3d3c0047b0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.994+0000 7f3d523ff700 1 -- 172.21.15.101:0/3618675550 shutdown_connections 2023-07-18T03:16:47.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.994+0000 7f3d523ff700 1 --2- 172.21.15.101:0/3618675550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d4c105ae0 0x7f3d4c1959e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.994+0000 7f3d523ff700 1 --2- 172.21.15.101:0/3618675550 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3d4c0ffad0 0x7f3d4c19b970 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.994+0000 7f3d523ff700 1 --2- 172.21.15.101:0/3618675550 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f3d380a1340 0x7f3d380a37f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.994+0000 7f3d523ff700 1 --2- 172.21.15.101:0/3618675550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d4c100480 0x7f3d4c19beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:47.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.994+0000 7f3d523ff700 1 -- 172.21.15.101:0/3618675550 >> 172.21.15.101:0/3618675550 conn(0x7f3d4c0fb5f0 msgr2=0x7f3d4c0fbb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:47.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.994+0000 7f3d523ff700 1 -- 172.21.15.101:0/3618675550 shutdown_connections 2023-07-18T03:16:47.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:47.994+0000 7f3d523ff700 1 -- 172.21.15.101:0/3618675550 wait complete. 2023-07-18T03:16:48.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:48 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/2048034676' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:16:48.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:48 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/2048034676' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:16:48.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:48 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2048034676' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:16:48.606 INFO:tasks.workunit.client.0.smithi101.stdout: 1 601947 6.12 MB/sec execute 373 sec latency 1697.158 ms 2023-07-18T03:16:49.006 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-18T03:16:49.006 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-18T03:16:49.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.225+0000 7f5eb95ff700 1 -- 172.21.15.101:0/399046018 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5eb40ffad0 msgr2=0x7f5eb40fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:49.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.225+0000 7f5eb95ff700 1 --2- 172.21.15.101:0/399046018 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5eb40ffad0 0x7f5eb40fff40 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8009990 tx=0x7f5ea8009ca0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.225+0000 7f5eb95ff700 1 -- 172.21.15.101:0/399046018 shutdown_connections 2023-07-18T03:16:49.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.225+0000 7f5eb95ff700 1 --2- 172.21.15.101:0/399046018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5eb4105ae0 0x7f5eb4105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.225+0000 7f5eb95ff700 1 --2- 172.21.15.101:0/399046018 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5eb4100480 0x7f5eb410c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.225+0000 7f5eb95ff700 1 --2- 172.21.15.101:0/399046018 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5eb40ffad0 0x7f5eb40fff40 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.225+0000 7f5eb95ff700 1 -- 172.21.15.101:0/399046018 >> 172.21.15.101:0/399046018 conn(0x7f5eb40fb5f0 msgr2=0x7f5eb40fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:49.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.225+0000 7f5eb95ff700 1 -- 172.21.15.101:0/399046018 shutdown_connections 2023-07-18T03:16:49.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.225+0000 7f5eb95ff700 1 -- 172.21.15.101:0/399046018 wait complete. 2023-07-18T03:16:49.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.227+0000 7f5eb95ff700 1 Processor -- start 2023-07-18T03:16:49.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.227+0000 7f5eb95ff700 1 -- start start 2023-07-18T03:16:49.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.228+0000 7f5eb95ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5eb40ffad0 0x7f5eb419b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:49.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.228+0000 7f5eb95ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5eb4100480 0x7f5eb419bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:49.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.228+0000 7f5eb95ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5eb4105ae0 0x7f5eb4195a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:49.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.228+0000 7f5eb95ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb41960b0 con 0x7f5eb4100480 2023-07-18T03:16:49.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.228+0000 7f5eb95ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb4196220 con 0x7f5eb40ffad0 2023-07-18T03:16:49.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.228+0000 7f5eb95ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb41963e0 con 0x7f5eb4105ae0 2023-07-18T03:16:49.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.228+0000 7f5eb27fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5eb4100480 0x7f5eb419bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:49.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.228+0000 7f5eb27fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5eb4100480 0x7f5eb419bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:50614/0 (socket says 172.21.15.101:50614) 2023-07-18T03:16:49.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.228+0000 7f5eb27fc700 1 -- 172.21.15.101:0/479980673 learned_addr learned my addr 172.21.15.101:0/479980673 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:49.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.228+0000 7f5eb2ffd700 1 --2- 172.21.15.101:0/479980673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5eb40ffad0 0x7f5eb419b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:49.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.228+0000 7f5eb37fe700 1 --2- 172.21.15.101:0/479980673 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5eb4105ae0 0x7f5eb4195a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:49.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.228+0000 7f5eb27fc700 1 -- 172.21.15.101:0/479980673 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5eb4105ae0 msgr2=0x7f5eb4195a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:49.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.228+0000 7f5eb27fc700 1 --2- 172.21.15.101:0/479980673 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5eb4105ae0 0x7f5eb4195a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.228+0000 7f5eb27fc700 1 -- 172.21.15.101:0/479980673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5eb40ffad0 msgr2=0x7f5eb419b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:49.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.228+0000 7f5eb27fc700 1 --2- 172.21.15.101:0/479980673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5eb40ffad0 0x7f5eb419b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.228+0000 7f5eb27fc700 1 -- 172.21.15.101:0/479980673 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea80095f0 con 0x7f5eb4100480 2023-07-18T03:16:49.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.228+0000 7f5eb37fe700 1 --2- 172.21.15.101:0/479980673 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5eb4105ae0 0x7f5eb4195a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:16:49.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.229+0000 7f5eb27fc700 1 --2- 172.21.15.101:0/479980673 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5eb4100480 0x7f5eb419bf10 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8005910 tx=0x7f5ea8005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:49.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.229+0000 7f5e9bfff700 1 -- 172.21.15.101:0/479980673 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea801d070 con 0x7f5eb4100480 2023-07-18T03:16:49.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.229+0000 7f5e9bfff700 1 -- 172.21.15.101:0/479980673 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ea8022470 con 0x7f5eb4100480 2023-07-18T03:16:49.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.229+0000 7f5eb95ff700 1 -- 172.21.15.101:0/479980673 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eb4196660 con 0x7f5eb4100480 2023-07-18T03:16:49.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.229+0000 7f5e9bfff700 1 -- 172.21.15.101:0/479980673 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8017650 con 0x7f5eb4100480 2023-07-18T03:16:49.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.229+0000 7f5eb95ff700 1 -- 172.21.15.101:0/479980673 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eb41a29c0 con 0x7f5eb4100480 2023-07-18T03:16:49.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.232+0000 7f5e9bfff700 1 -- 172.21.15.101:0/479980673 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f5ea80225e0 con 0x7f5eb4100480 2023-07-18T03:16:49.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.233+0000 7f5eccf2bb80 1 Processor -- start 2023-07-18T03:16:49.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.233+0000 7f5eccf2bb80 1 -- start start 2023-07-18T03:16:49.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.233+0000 7f5e9bfff700 1 --2- 172.21.15.101:0/479980673 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f5e9c0a1340 0x7f5e9c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:49.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.234+0000 7f5eccf2bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af5f10e350 0x55af5f10e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:49.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.234+0000 7f5e9bfff700 1 -- 172.21.15.101:0/479980673 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f5ea80ce3f0 con 0x7f5eb4100480 2023-07-18T03:16:49.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.234+0000 7f5eccf2bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af5f10ec60 0x55af5f119610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:49.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.234+0000 7f5eccf2bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af5f119b50 0x55af5f11bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:49.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.234+0000 7f5eccf2bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af5f014d30 con 0x55af5f119b50 2023-07-18T03:16:49.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.234+0000 7f5eccf2bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af5f067ac0 con 0x55af5f10ec60 2023-07-18T03:16:49.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.234+0000 7f5eccf2bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af5f060f50 con 0x55af5f10e350 2023-07-18T03:16:49.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.234+0000 7f5eb37fe700 1 --2- 172.21.15.101:0/479980673 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f5e9c0a1340 0x7f5e9c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:49.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.234+0000 7f5eb2ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af5f10e350 0x55af5f10e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:49.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.234+0000 7f5eb37fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af5f119b50 0x55af5f11bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:49.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.234+0000 7f5eb2ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af5f10e350 0x55af5f10e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:47666/0 (socket says 172.21.15.101:47666) 2023-07-18T03:16:49.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.234+0000 7f5eb2ffd700 1 -- 172.21.15.101:0/3669344389 learned_addr learned my addr 172.21.15.101:0/3669344389 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:49.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.234+0000 7f5eb27fc700 1 --2- 172.21.15.101:0/3669344389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af5f10ec60 0x55af5f119610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:49.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.235+0000 7f5eb2ffd700 1 -- 172.21.15.101:0/3669344389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af5f10ec60 msgr2=0x55af5f119610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:49.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.235+0000 7f5eb2ffd700 1 --2- 172.21.15.101:0/3669344389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af5f10ec60 0x55af5f119610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.235+0000 7f5eb2ffd700 1 -- 172.21.15.101:0/3669344389 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af5f119b50 msgr2=0x55af5f11bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:49.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.235+0000 7f5eb2ffd700 1 --2- 172.21.15.101:0/3669344389 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af5f119b50 0x55af5f11bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.235+0000 7f5eb2ffd700 1 -- 172.21.15.101:0/3669344389 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af5ee8ee60 con 0x55af5f10e350 2023-07-18T03:16:49.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.235+0000 7f5eb27fc700 1 --2- 172.21.15.101:0/3669344389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af5f10ec60 0x55af5f119610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:16:49.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.235+0000 7f5eb37fe700 1 --2- 172.21.15.101:0/479980673 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f5e9c0a1340 0x7f5e9c0a37f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5eac009fd0 tx=0x7f5eac007b00 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:49.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.235+0000 7f5eb2ffd700 1 --2- 172.21.15.101:0/3669344389 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af5f10e350 0x55af5f10e720 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4009d00 tx=0x7f5ea4011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b59ea06eabfda7 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:49.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.235+0000 7f5e997fa700 1 -- 172.21.15.101:0/3669344389 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4011da0 con 0x55af5f10e350 2023-07-18T03:16:49.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.235+0000 7f5e997fa700 1 -- 172.21.15.101:0/3669344389 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ea4009e50 con 0x55af5f10e350 2023-07-18T03:16:49.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.235+0000 7f5e997fa700 1 -- 172.21.15.101:0/3669344389 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4019420 con 0x55af5f10e350 2023-07-18T03:16:49.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.236+0000 7f5eccf2bb80 1 -- 172.21.15.101:0/3669344389 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af5f10e350 msgr2=0x55af5f10e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:49.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.236+0000 7f5eccf2bb80 1 --2- 172.21.15.101:0/3669344389 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af5f10e350 0x55af5f10e720 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4009d00 tx=0x7f5ea4011720 comp rx=0 tx=0).stop 2023-07-18T03:16:49.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.236+0000 7f5eccf2bb80 1 -- 172.21.15.101:0/3669344389 shutdown_connections 2023-07-18T03:16:49.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.236+0000 7f5eccf2bb80 1 --2- 172.21.15.101:0/3669344389 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af5f119b50 0x55af5f11bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.236+0000 7f5eccf2bb80 1 --2- 172.21.15.101:0/3669344389 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af5f10e350 0x55af5f10e720 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.236+0000 7f5eccf2bb80 1 --2- 172.21.15.101:0/3669344389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af5f10ec60 0x55af5f119610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.236+0000 7f5eccf2bb80 1 -- 172.21.15.101:0/3669344389 >> 172.21.15.101:0/3669344389 conn(0x55af5f104e20 msgr2=0x55af5f10a720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:49.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.236+0000 7f5eccf2bb80 1 -- 172.21.15.101:0/3669344389 shutdown_connections 2023-07-18T03:16:49.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.236+0000 7f5eccf2bb80 1 -- 172.21.15.101:0/3669344389 wait complete. 2023-07-18T03:16:49.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.237+0000 7f5eccf2bb80 1 -- start start 2023-07-18T03:16:49.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.237+0000 7f5eccf2bb80 1 Processor -- start 2023-07-18T03:16:49.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.237+0000 7f5eccf2bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af5f10e350 0x55af5f1195e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:49.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.237+0000 7f5eccf2bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af5f10ec60 0x55af5f116860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:49.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.237+0000 7f5eccf2bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af5f119b50 0x55af5f116fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:49.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.237+0000 7f5eccf2bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af5f000b40 con 0x55af5f10e350 2023-07-18T03:16:49.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.237+0000 7f5eccf2bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af5f000840 con 0x55af5f119b50 2023-07-18T03:16:49.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.237+0000 7f5eccf2bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af5eee49a0 con 0x55af5f10ec60 2023-07-18T03:16:49.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.237+0000 7f5eccf2bb80 10 client.0 _ll_register_callbacks cb 0x55af5effe7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:16:49.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.238+0000 7f5eb2ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af5f10e350 0x55af5f1195e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:49.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.238+0000 7f5eb2ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af5f10e350 0x55af5f1195e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:50626/0 (socket says 172.21.15.101:50626) 2023-07-18T03:16:49.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.238+0000 7f5eb2ffd700 1 -- 172.21.15.101:0/1821425122 learned_addr learned my addr 172.21.15.101:0/1821425122 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:49.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.238+0000 7f5eb27fc700 1 --2- 172.21.15.101:0/1821425122 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af5f10ec60 0x55af5f116860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:49.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.238+0000 7f5eb37fe700 1 --2- 172.21.15.101:0/1821425122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af5f119b50 0x55af5f116fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:49.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.238+0000 7f5eb2ffd700 1 -- 172.21.15.101:0/1821425122 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af5f10ec60 msgr2=0x55af5f116860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:49.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.238+0000 7f5eb2ffd700 1 --2- 172.21.15.101:0/1821425122 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af5f10ec60 0x55af5f116860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.238+0000 7f5eb2ffd700 1 -- 172.21.15.101:0/1821425122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af5f119b50 msgr2=0x55af5f116fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:49.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.238+0000 7f5eb2ffd700 1 --2- 172.21.15.101:0/1821425122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af5f119b50 0x55af5f116fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.238+0000 7f5eb2ffd700 1 -- 172.21.15.101:0/1821425122 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eb41a29c0 con 0x55af5f10e350 2023-07-18T03:16:49.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.238+0000 7f5eb2ffd700 1 --2- 172.21.15.101:0/1821425122 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af5f10e350 0x55af5f1195e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4006010 tx=0x7f5ea4005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c51b6c80027637ac server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:49.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.238+0000 7f5e96ff5700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:16:49.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.239+0000 7f5e96ff5700 1 -- 172.21.15.101:0/1821425122 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4004950 con 0x55af5f10e350 2023-07-18T03:16:49.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.239+0000 7f5eccf2bb80 1 -- 172.21.15.101:0/1821425122 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55af5efba570 con 0x55af5f10e350 2023-07-18T03:16:49.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.239+0000 7f5e96ff5700 1 -- 172.21.15.101:0/1821425122 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f5ea4005ce0 con 0x55af5f10e350 2023-07-18T03:16:49.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.239+0000 7f5e96ff5700 1 -- 172.21.15.101:0/1821425122 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ea40056a0 con 0x55af5f10e350 2023-07-18T03:16:49.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.239+0000 7f5eccf2bb80 10 client.17433 fetch_fsmap learned FSMap version 33 2023-07-18T03:16:49.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.239+0000 7f5eccf2bb80 1 -- 172.21.15.101:0/1821425122 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f5ea4009960 con 0x55af5f10e350 2023-07-18T03:16:49.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.239+0000 7f5e96ff5700 1 -- 172.21.15.101:0/1821425122 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55af5efba570 con 0x55af5f10e350 2023-07-18T03:16:49.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.239+0000 7f5e96ff5700 1 -- 172.21.15.101:0/1821425122 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5ea4028020 con 0x55af5f10e350 2023-07-18T03:16:49.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.240+0000 7f5eccf2bb80 10 client.17433 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:16:49.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.240+0000 7f5eccf2bb80 10 client.17433 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:16:49.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.240+0000 7f5eccf2bb80 20 client.17433 populate_metadata read hostname 'smithi101' 2023-07-18T03:16:49.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.240+0000 7f5eccf2bb80 1 --2- 172.21.15.101:0/1821425122 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55af5f1293f0 0x55af5f12b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:49.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.240+0000 7f5eccf2bb80 4 client.17433 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:16:49.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.240+0000 7f5eccf2bb80 1 -- 172.21.15.101:0/1821425122 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55af5f059f30 con 0x55af5f1293f0 2023-07-18T03:16:49.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.240+0000 7f5eb27fc700 1 --2- 172.21.15.101:0/1821425122 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55af5f1293f0 0x55af5f12b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:49.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.240+0000 7f5eb27fc700 1 --2- 172.21.15.101:0/1821425122 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55af5f1293f0 0x55af5f12b7d0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbd7669a25c7d818 server_cookie=7c06ba52cb4c1c75 in_seq=0 out_seq=0 2023-07-18T03:16:49.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.240+0000 7f5e96ff5700 10 client.17433 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:16:49.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.241+0000 7f5e96ff5700 1 -- 172.21.15.101:0/1821425122 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55af5f059f30 con 0x55af5f1293f0 2023-07-18T03:16:49.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.241+0000 7f5e96ff5700 10 client.17433 handle_command_reply: tid=0 2023-07-18T03:16:49.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.241+0000 7f5eccf2bb80 1 client.17433 shutdown 2023-07-18T03:16:49.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.241+0000 7f5eccf2bb80 1 -- 172.21.15.101:0/1821425122 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af5f10e350 msgr2=0x55af5f1195e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:49.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.241+0000 7f5eccf2bb80 1 --2- 172.21.15.101:0/1821425122 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af5f10e350 0x55af5f1195e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4006010 tx=0x7f5ea4005270 comp rx=0 tx=0).stop 2023-07-18T03:16:49.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.241+0000 7f5eccf2bb80 1 -- 172.21.15.101:0/1821425122 shutdown_connections 2023-07-18T03:16:49.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.241+0000 7f5eccf2bb80 1 --2- 172.21.15.101:0/1821425122 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55af5f1293f0 0x55af5f12b7d0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.241+0000 7f5eccf2bb80 1 --2- 172.21.15.101:0/1821425122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af5f119b50 0x55af5f116fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.241+0000 7f5eccf2bb80 1 --2- 172.21.15.101:0/1821425122 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af5f10e350 0x55af5f1195e0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.241+0000 7f5e96ff5700 0 client.17433 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:16:49.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.241+0000 7f5eccf2bb80 1 --2- 172.21.15.101:0/1821425122 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af5f10ec60 0x55af5f116860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.241+0000 7f5eccf2bb80 1 -- 172.21.15.101:0/1821425122 >> 172.21.15.101:0/1821425122 conn(0x55af5f104e20 msgr2=0x55af5f10a310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:49.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.242+0000 7f5eccf2bb80 1 -- 172.21.15.101:0/1821425122 shutdown_connections 2023-07-18T03:16:49.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.242+0000 7f5eccf2bb80 1 -- 172.21.15.101:0/1821425122 wait complete. 2023-07-18T03:16:49.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.242+0000 7f5eccf2bb80 20 client.17433 trim_cache size 0 max 16384 2023-07-18T03:16:49.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.259+0000 7f5eccf2bb80 1 Processor -- start 2023-07-18T03:16:49.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.259+0000 7f5eccf2bb80 1 -- start start 2023-07-18T03:16:49.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.259+0000 7f5eccf2bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af5f10e350 0x55af5f119110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:49.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.259+0000 7f5eccf2bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af5f10ec60 0x55af5f1154f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:49.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.259+0000 7f5eccf2bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af5f119b50 0x55af5f115a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:49.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.259+0000 7f5eccf2bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af5f10f5c0 con 0x55af5f10ec60 2023-07-18T03:16:49.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.259+0000 7f5eccf2bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af5f110220 con 0x55af5f119b50 2023-07-18T03:16:49.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.259+0000 7f5eccf2bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af5f116990 con 0x55af5f10e350 2023-07-18T03:16:49.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.260+0000 7f5eb27fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af5f10ec60 0x55af5f1154f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:49.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.260+0000 7f5eb27fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af5f10ec60 0x55af5f1154f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:50642/0 (socket says 172.21.15.101:50642) 2023-07-18T03:16:49.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.260+0000 7f5eb27fc700 1 -- 172.21.15.101:0/1255125155 learned_addr learned my addr 172.21.15.101:0/1255125155 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:49.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.260+0000 7f5eb37fe700 1 --2- 172.21.15.101:0/1255125155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af5f119b50 0x55af5f115a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:49.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.260+0000 7f5eb2ffd700 1 --2- 172.21.15.101:0/1255125155 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af5f10e350 0x55af5f119110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:49.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.260+0000 7f5eb27fc700 1 -- 172.21.15.101:0/1255125155 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af5f10e350 msgr2=0x55af5f119110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:49.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.260+0000 7f5eb27fc700 1 --2- 172.21.15.101:0/1255125155 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af5f10e350 0x55af5f119110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.260+0000 7f5eb27fc700 1 -- 172.21.15.101:0/1255125155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af5f119b50 msgr2=0x55af5f115a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:49.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.260+0000 7f5eb27fc700 1 --2- 172.21.15.101:0/1255125155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af5f119b50 0x55af5f115a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.260+0000 7f5eb27fc700 1 -- 172.21.15.101:0/1255125155 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af5ee77ce0 con 0x55af5f10ec60 2023-07-18T03:16:49.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.260+0000 7f5eb2ffd700 1 --2- 172.21.15.101:0/1255125155 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af5f10e350 0x55af5f119110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:16:49.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.260+0000 7f5eb37fe700 1 --2- 172.21.15.101:0/1255125155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af5f119b50 0x55af5f115a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:16:49.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.260+0000 7f5eb27fc700 1 --2- 172.21.15.101:0/1255125155 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af5f10ec60 0x55af5f1154f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f5ea80cb7f0 tx=0x7f5ea801a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26079b2c629b0906 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:49.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.260+0000 7f5e99ffb700 1 -- 172.21.15.101:0/1255125155 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea809ab00 con 0x55af5f10ec60 2023-07-18T03:16:49.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.260+0000 7f5e99ffb700 1 -- 172.21.15.101:0/1255125155 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ea809b1a0 con 0x55af5f10ec60 2023-07-18T03:16:49.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.260+0000 7f5e99ffb700 1 -- 172.21.15.101:0/1255125155 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea80229e0 con 0x55af5f10ec60 2023-07-18T03:16:49.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.261+0000 7f5eccf2bb80 1 -- 172.21.15.101:0/1255125155 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af5f10ec60 msgr2=0x55af5f1154f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:49.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.261+0000 7f5eccf2bb80 1 --2- 172.21.15.101:0/1255125155 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af5f10ec60 0x55af5f1154f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f5ea80cb7f0 tx=0x7f5ea801a040 comp rx=0 tx=0).stop 2023-07-18T03:16:49.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.261+0000 7f5eccf2bb80 1 -- 172.21.15.101:0/1255125155 shutdown_connections 2023-07-18T03:16:49.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.261+0000 7f5eccf2bb80 1 --2- 172.21.15.101:0/1255125155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af5f119b50 0x55af5f115a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.261+0000 7f5eccf2bb80 1 --2- 172.21.15.101:0/1255125155 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af5f10e350 0x55af5f119110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.261+0000 7f5eccf2bb80 1 --2- 172.21.15.101:0/1255125155 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af5f10ec60 0x55af5f1154f0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.261+0000 7f5eccf2bb80 1 -- 172.21.15.101:0/1255125155 >> 172.21.15.101:0/1255125155 conn(0x55af5f104e20 msgr2=0x55af5f114de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:49.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.261+0000 7f5eccf2bb80 1 -- 172.21.15.101:0/1255125155 shutdown_connections 2023-07-18T03:16:49.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.261+0000 7f5eccf2bb80 1 -- 172.21.15.101:0/1255125155 wait complete. 2023-07-18T03:16:49.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.261+0000 7f5eccf2bb80 1 -- start start 2023-07-18T03:16:49.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.262+0000 7f5eccf2bb80 1 Processor -- start 2023-07-18T03:16:49.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.262+0000 7f5eccf2bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af5f10e350 0x55af5f11e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:49.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.262+0000 7f5eccf2bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af5f10ec60 0x55af5f11ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:49.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.262+0000 7f5eccf2bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af5f119b50 0x55af5f11efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:49.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.262+0000 7f5eccf2bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af5f17e260 con 0x55af5f10ec60 2023-07-18T03:16:49.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.262+0000 7f5eccf2bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af5f17e3a0 con 0x55af5f10e350 2023-07-18T03:16:49.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.262+0000 7f5eccf2bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af5f17e4e0 con 0x55af5f119b50 2023-07-18T03:16:49.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.262+0000 7f5eccf2bb80 10 client.0 _ll_register_callbacks cb 0x7f5ea8005c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:16:49.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.262+0000 7f5eb27fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af5f10ec60 0x55af5f11ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:49.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.262+0000 7f5eb27fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af5f10ec60 0x55af5f11ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:50652/0 (socket says 172.21.15.101:50652) 2023-07-18T03:16:49.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.262+0000 7f5eb27fc700 1 -- 172.21.15.101:0/3447213964 learned_addr learned my addr 172.21.15.101:0/3447213964 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:49.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.262+0000 7f5eb2ffd700 1 --2- 172.21.15.101:0/3447213964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af5f10e350 0x55af5f11e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:49.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.263+0000 7f5eb37fe700 1 --2- 172.21.15.101:0/3447213964 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af5f119b50 0x55af5f11efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:49.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.263+0000 7f5eb27fc700 1 -- 172.21.15.101:0/3447213964 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af5f119b50 msgr2=0x55af5f11efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:49.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.263+0000 7f5eb27fc700 1 --2- 172.21.15.101:0/3447213964 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af5f119b50 0x55af5f11efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.263+0000 7f5eb27fc700 1 -- 172.21.15.101:0/3447213964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af5f10e350 msgr2=0x55af5f11e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:49.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.263+0000 7f5eb27fc700 1 --2- 172.21.15.101:0/3447213964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af5f10e350 0x55af5f11e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.263+0000 7f5eb27fc700 1 -- 172.21.15.101:0/3447213964 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eb4196660 con 0x55af5f10ec60 2023-07-18T03:16:49.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.263+0000 7f5eb2ffd700 1 --2- 172.21.15.101:0/3447213964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af5f10e350 0x55af5f11e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:16:49.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.263+0000 7f5eb27fc700 1 --2- 172.21.15.101:0/3447213964 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af5f10ec60 0x55af5f11ea90 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f5ea809aea0 tx=0x7f5ea8022c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8712b4779dc3a723 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:49.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.263+0000 7f5e96ff5700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:16:49.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.263+0000 7f5e96ff5700 1 -- 172.21.15.101:0/3447213964 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea80a3560 con 0x55af5f10ec60 2023-07-18T03:16:49.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.263+0000 7f5eccf2bb80 1 -- 172.21.15.101:0/3447213964 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ea4009e50 con 0x55af5f10ec60 2023-07-18T03:16:49.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.263+0000 7f5e96ff5700 1 -- 172.21.15.101:0/3447213964 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f5ea80978b0 con 0x55af5f10ec60 2023-07-18T03:16:49.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.263+0000 7f5e96ff5700 1 -- 172.21.15.101:0/3447213964 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ea809b400 con 0x55af5f10ec60 2023-07-18T03:16:49.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.263+0000 7f5eccf2bb80 10 client.17439 fetch_fsmap learned FSMap version 33 2023-07-18T03:16:49.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.263+0000 7f5eccf2bb80 1 -- 172.21.15.101:0/3447213964 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55af5eeb52d0 con 0x55af5f10ec60 2023-07-18T03:16:49.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.264+0000 7f5e96ff5700 1 -- 172.21.15.101:0/3447213964 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ea4009e50 con 0x55af5f10ec60 2023-07-18T03:16:49.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.264+0000 7f5e96ff5700 1 -- 172.21.15.101:0/3447213964 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5ea80a30d0 con 0x55af5f10ec60 2023-07-18T03:16:49.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.264+0000 7f5eccf2bb80 10 client.17439 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:16:49.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.264+0000 7f5eccf2bb80 10 client.17439 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:16:49.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.264+0000 7f5eccf2bb80 20 client.17439 populate_metadata read hostname 'smithi101' 2023-07-18T03:16:49.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.264+0000 7f5eccf2bb80 1 --2- 172.21.15.101:0/3447213964 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55af5f17f7d0 0x55af5f181be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:49.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.264+0000 7f5eccf2bb80 4 client.17439 mds_command: new command op to 24562 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-18T03:16:49.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.264+0000 7f5eccf2bb80 1 -- 172.21.15.101:0/3447213964 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55af5f01c0d0 con 0x55af5f17f7d0 2023-07-18T03:16:49.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.264+0000 7f5eb2ffd700 1 --2- 172.21.15.101:0/3447213964 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55af5f17f7d0 0x55af5f181be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:49.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.265+0000 7f5eb2ffd700 1 --2- 172.21.15.101:0/3447213964 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55af5f17f7d0 0x55af5f181be0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ece7852ef4191d5 server_cookie=2d0efd4ab1f3088d in_seq=0 out_seq=0 2023-07-18T03:16:49.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.265+0000 7f5e96ff5700 10 client.17439 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:16:49.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.266+0000 7f5e96ff5700 1 -- 172.21.15.101:0/3447213964 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55af5f01c0d0 con 0x55af5f17f7d0 2023-07-18T03:16:49.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.266+0000 7f5e96ff5700 10 client.17439 handle_command_reply: tid=0 2023-07-18T03:16:49.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.266+0000 7f5eccf2bb80 1 client.17439 shutdown 2023-07-18T03:16:49.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.266+0000 7f5eccf2bb80 1 -- 172.21.15.101:0/3447213964 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af5f10ec60 msgr2=0x55af5f11ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:49.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.266+0000 7f5eccf2bb80 1 --2- 172.21.15.101:0/3447213964 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af5f10ec60 0x55af5f11ea90 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f5ea809aea0 tx=0x7f5ea8022c00 comp rx=0 tx=0).stop 2023-07-18T03:16:49.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.266+0000 7f5eccf2bb80 1 -- 172.21.15.101:0/3447213964 shutdown_connections 2023-07-18T03:16:49.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.266+0000 7f5eccf2bb80 1 --2- 172.21.15.101:0/3447213964 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55af5f17f7d0 0x55af5f181be0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.266+0000 7f5eccf2bb80 1 --2- 172.21.15.101:0/3447213964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af5f10e350 0x55af5f11e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.266+0000 7f5eccf2bb80 1 --2- 172.21.15.101:0/3447213964 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af5f119b50 0x55af5f11efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.266+0000 7f5eccf2bb80 1 --2- 172.21.15.101:0/3447213964 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af5f10ec60 0x55af5f11ea90 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.266+0000 7f5eccf2bb80 1 -- 172.21.15.101:0/3447213964 >> 172.21.15.101:0/3447213964 conn(0x55af5f104e20 msgr2=0x55af5f115670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:49.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.266+0000 7f5e96ff5700 0 client.17439 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:16:49.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.267+0000 7f5eccf2bb80 1 -- 172.21.15.101:0/3447213964 shutdown_connections 2023-07-18T03:16:49.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.267+0000 7f5eccf2bb80 1 -- 172.21.15.101:0/3447213964 wait complete. 2023-07-18T03:16:49.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.267+0000 7f5eccf2bb80 20 client.17439 trim_cache size 0 max 16384 2023-07-18T03:16:49.269 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:16:49.269 INFO:teuthology.orchestra.run.smithi101.stdout: "return_code": 0, 2023-07-18T03:16:49.269 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_tag": "f05717c7-804b-4547-982a-1724b5dada5e", 2023-07-18T03:16:49.269 INFO:teuthology.orchestra.run.smithi101.stdout: "mode": "asynchronous" 2023-07-18T03:16:49.269 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:16:49.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.267+0000 7f5eb95ff700 1 -- 172.21.15.101:0/479980673 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f5e9c0a1340 msgr2=0x7f5e9c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:49.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.267+0000 7f5eb95ff700 1 --2- 172.21.15.101:0/479980673 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f5e9c0a1340 0x7f5e9c0a37f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5eac009fd0 tx=0x7f5eac007b00 comp rx=0 tx=0).stop 2023-07-18T03:16:49.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.267+0000 7f5eb95ff700 1 -- 172.21.15.101:0/479980673 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5eb4100480 msgr2=0x7f5eb419bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:49.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.267+0000 7f5eb95ff700 1 --2- 172.21.15.101:0/479980673 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5eb4100480 0x7f5eb419bf10 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8005910 tx=0x7f5ea8005720 comp rx=0 tx=0).stop 2023-07-18T03:16:49.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.267+0000 7f5eb95ff700 1 -- 172.21.15.101:0/479980673 shutdown_connections 2023-07-18T03:16:49.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.267+0000 7f5eb95ff700 1 --2- 172.21.15.101:0/479980673 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f5e9c0a1340 0x7f5e9c0a37f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.267+0000 7f5eb95ff700 1 --2- 172.21.15.101:0/479980673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5eb40ffad0 0x7f5eb419b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.267+0000 7f5eb95ff700 1 --2- 172.21.15.101:0/479980673 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5eb4105ae0 0x7f5eb4195a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.267+0000 7f5eb95ff700 1 --2- 172.21.15.101:0/479980673 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5eb4100480 0x7f5eb419bf10 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.267+0000 7f5eb95ff700 1 -- 172.21.15.101:0/479980673 >> 172.21.15.101:0/479980673 conn(0x7f5eb40fb5f0 msgr2=0x7f5eb41035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:49.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.267+0000 7f5eb95ff700 1 -- 172.21.15.101:0/479980673 shutdown_connections 2023-07-18T03:16:49.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.268+0000 7f5eb95ff700 1 -- 172.21.15.101:0/479980673 wait complete. 2023-07-18T03:16:49.277 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:49 smithi101 ceph-mon[126379]: pgmap v286: 129 pgs: 129 active+clean; 450 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 103 op/s 2023-07-18T03:16:49.280 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:16:49.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:49 smithi155 ceph-mon[128885]: pgmap v286: 129 pgs: 129 active+clean; 450 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 103 op/s 2023-07-18T03:16:49.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:49 smithi179 ceph-mon[129229]: pgmap v286: 129 pgs: 129 active+clean; 450 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 103 op/s 2023-07-18T03:16:49.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.499+0000 7fd617bff700 1 -- 172.21.15.101:0/3112451307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd610105af0 msgr2=0x7fd610105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:49.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.499+0000 7fd617bff700 1 --2- 172.21.15.101:0/3112451307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd610105af0 0x7fd610105ec0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd60c009990 tx=0x7fd60c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.499+0000 7fd617bff700 1 -- 172.21.15.101:0/3112451307 shutdown_connections 2023-07-18T03:16:49.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.499+0000 7fd617bff700 1 --2- 172.21.15.101:0/3112451307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd610105af0 0x7fd610105ec0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.499+0000 7fd617bff700 1 --2- 172.21.15.101:0/3112451307 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd610100490 0x7fd61010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.499+0000 7fd617bff700 1 --2- 172.21.15.101:0/3112451307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6100ffae0 0x7fd6100fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.499+0000 7fd617bff700 1 -- 172.21.15.101:0/3112451307 >> 172.21.15.101:0/3112451307 conn(0x7fd6100fb5e0 msgr2=0x7fd6100fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:49.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.499+0000 7fd617bff700 1 -- 172.21.15.101:0/3112451307 shutdown_connections 2023-07-18T03:16:49.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.499+0000 7fd617bff700 1 -- 172.21.15.101:0/3112451307 wait complete. 2023-07-18T03:16:49.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.500+0000 7fd617bff700 1 Processor -- start 2023-07-18T03:16:49.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.500+0000 7fd617bff700 1 -- start start 2023-07-18T03:16:49.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.501+0000 7fd617bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd6100ffae0 0x7fd610197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:49.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.501+0000 7fd617bff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd610100490 0x7fd610198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:49.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.501+0000 7fd617bff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd610105af0 0x7fd61019c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:49.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.501+0000 7fd617bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61019cdd0 con 0x7fd6100ffae0 2023-07-18T03:16:49.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.501+0000 7fd617bff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61019cf40 con 0x7fd610100490 2023-07-18T03:16:49.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.501+0000 7fd617bff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61019d100 con 0x7fd610105af0 2023-07-18T03:16:49.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.501+0000 7fd61599b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd6100ffae0 0x7fd610197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:49.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.501+0000 7fd61599b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd6100ffae0 0x7fd610197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:50662/0 (socket says 172.21.15.101:50662) 2023-07-18T03:16:49.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.501+0000 7fd61599b700 1 -- 172.21.15.101:0/281055442 learned_addr learned my addr 172.21.15.101:0/281055442 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:49.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.501+0000 7fd61619c700 1 --2- 172.21.15.101:0/281055442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd610105af0 0x7fd61019c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:49.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.501+0000 7fd61519a700 1 --2- 172.21.15.101:0/281055442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd610100490 0x7fd610198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:49.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.502+0000 7fd61599b700 1 -- 172.21.15.101:0/281055442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd610105af0 msgr2=0x7fd61019c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:49.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.502+0000 7fd61599b700 1 --2- 172.21.15.101:0/281055442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd610105af0 0x7fd61019c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.502+0000 7fd61599b700 1 -- 172.21.15.101:0/281055442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd610100490 msgr2=0x7fd610198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:49.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.502+0000 7fd61599b700 1 --2- 172.21.15.101:0/281055442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd610100490 0x7fd610198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.502+0000 7fd61599b700 1 -- 172.21.15.101:0/281055442 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd60c0095f0 con 0x7fd6100ffae0 2023-07-18T03:16:49.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.502+0000 7fd61599b700 1 --2- 172.21.15.101:0/281055442 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd6100ffae0 0x7fd610197df0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fd60c005910 tx=0x7fd60c005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:49.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.502+0000 7fd606ffd700 1 -- 172.21.15.101:0/281055442 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c01d070 con 0x7fd6100ffae0 2023-07-18T03:16:49.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.502+0000 7fd617bff700 1 -- 172.21.15.101:0/281055442 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd61019d380 con 0x7fd6100ffae0 2023-07-18T03:16:49.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.502+0000 7fd606ffd700 1 -- 172.21.15.101:0/281055442 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd60c022470 con 0x7fd6100ffae0 2023-07-18T03:16:49.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.502+0000 7fd606ffd700 1 -- 172.21.15.101:0/281055442 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c017650 con 0x7fd6100ffae0 2023-07-18T03:16:49.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.502+0000 7fd617bff700 1 -- 172.21.15.101:0/281055442 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6101a4c20 con 0x7fd6100ffae0 2023-07-18T03:16:49.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.506+0000 7fd606ffd700 1 -- 172.21.15.101:0/281055442 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fd60c0225e0 con 0x7fd6100ffae0 2023-07-18T03:16:49.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.507+0000 7fd62b59fb80 1 Processor -- start 2023-07-18T03:16:49.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.507+0000 7fd62b59fb80 1 -- start start 2023-07-18T03:16:49.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.508+0000 7fd62b59fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4f91ed2d0 0x55f4f91ed6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:49.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.508+0000 7fd62b59fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f4f91edbe0 0x55f4f91f8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:49.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.508+0000 7fd62b59fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4f91f8ad0 0x55f4f91faeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:49.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.508+0000 7fd62b59fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f90f3cb0 con 0x55f4f91ed2d0 2023-07-18T03:16:49.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.508+0000 7fd62b59fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f9146a40 con 0x55f4f91f8ad0 2023-07-18T03:16:49.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.508+0000 7fd62b59fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f913fed0 con 0x55f4f91edbe0 2023-07-18T03:16:49.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.508+0000 7fd61619c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4f91ed2d0 0x55f4f91ed6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:49.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.508+0000 7fd61619c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4f91ed2d0 0x55f4f91ed6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:50664/0 (socket says 172.21.15.101:50664) 2023-07-18T03:16:49.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.508+0000 7fd61619c700 1 -- 172.21.15.101:0/2401579101 learned_addr learned my addr 172.21.15.101:0/2401579101 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:49.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.508+0000 7fd61519a700 1 --2- 172.21.15.101:0/2401579101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4f91f8ad0 0x55f4f91faeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:49.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.508+0000 7fd606ffd700 1 --2- 172.21.15.101:0/281055442 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd5fc0a12c0 0x7fd5fc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:49.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.508+0000 7fd61599b700 1 --2- 172.21.15.101:0/2401579101 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f4f91edbe0 0x55f4f91f8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:49.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.509+0000 7fd61619c700 1 -- 172.21.15.101:0/2401579101 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f4f91edbe0 msgr2=0x55f4f91f8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:49.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.509+0000 7fd61619c700 1 --2- 172.21.15.101:0/2401579101 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f4f91edbe0 0x55f4f91f8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.509+0000 7fd61619c700 1 -- 172.21.15.101:0/2401579101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4f91f8ad0 msgr2=0x55f4f91faeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:49.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.509+0000 7fd606ffd700 1 -- 172.21.15.101:0/281055442 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fd60c0ce430 con 0x7fd6100ffae0 2023-07-18T03:16:49.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.509+0000 7fd61619c700 1 --2- 172.21.15.101:0/2401579101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4f91f8ad0 0x55f4f91faeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.509+0000 7fd61619c700 1 -- 172.21.15.101:0/2401579101 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4f8f6dde0 con 0x55f4f91ed2d0 2023-07-18T03:16:49.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.509+0000 7fd61519a700 1 --2- 172.21.15.101:0/2401579101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4f91f8ad0 0x55f4f91faeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:16:49.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.509+0000 7fd61619c700 1 --2- 172.21.15.101:0/281055442 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd5fc0a12c0 0x7fd5fc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:49.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.509+0000 7fd61619c700 1 --2- 172.21.15.101:0/2401579101 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4f91ed2d0 0x55f4f91ed6a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fd608011790 tx=0x7fd608011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b49ba6545b1e1860 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:49.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.509+0000 7fd5fbfff700 1 -- 172.21.15.101:0/2401579101 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60800e040 con 0x55f4f91ed2d0 2023-07-18T03:16:49.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.509+0000 7fd5fbfff700 1 -- 172.21.15.101:0/2401579101 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd60800f040 con 0x55f4f91ed2d0 2023-07-18T03:16:49.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.509+0000 7fd5fbfff700 1 -- 172.21.15.101:0/2401579101 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd608023880 con 0x55f4f91ed2d0 2023-07-18T03:16:49.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.509+0000 7fd61619c700 1 --2- 172.21.15.101:0/281055442 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd5fc0a12c0 0x7fd5fc0a3770 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd608022000 tx=0x7fd60801f000 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:49.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.510+0000 7fd62b59fb80 1 -- 172.21.15.101:0/2401579101 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4f91ed2d0 msgr2=0x55f4f91ed6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:49.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.510+0000 7fd62b59fb80 1 --2- 172.21.15.101:0/2401579101 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4f91ed2d0 0x55f4f91ed6a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fd608011790 tx=0x7fd608011aa0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.510+0000 7fd62b59fb80 1 -- 172.21.15.101:0/2401579101 shutdown_connections 2023-07-18T03:16:49.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.510+0000 7fd62b59fb80 1 --2- 172.21.15.101:0/2401579101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4f91f8ad0 0x55f4f91faeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.510+0000 7fd62b59fb80 1 --2- 172.21.15.101:0/2401579101 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4f91ed2d0 0x55f4f91ed6a0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.510+0000 7fd62b59fb80 1 --2- 172.21.15.101:0/2401579101 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f4f91edbe0 0x55f4f91f8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.510+0000 7fd62b59fb80 1 -- 172.21.15.101:0/2401579101 >> 172.21.15.101:0/2401579101 conn(0x55f4f91e3da0 msgr2=0x55f4f91e96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:49.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.510+0000 7fd62b59fb80 1 -- 172.21.15.101:0/2401579101 shutdown_connections 2023-07-18T03:16:49.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.510+0000 7fd62b59fb80 1 -- 172.21.15.101:0/2401579101 wait complete. 2023-07-18T03:16:49.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.511+0000 7fd62b59fb80 1 -- start start 2023-07-18T03:16:49.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.511+0000 7fd62b59fb80 1 Processor -- start 2023-07-18T03:16:49.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.512+0000 7fd62b59fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4f91ed2d0 0x55f4f91f8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:49.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.512+0000 7fd62b59fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4f91edbe0 0x55f4f91f57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:49.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.512+0000 7fd62b59fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f4f91f8ad0 0x55f4f91f5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:49.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.512+0000 7fd62b59fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f90dfac0 con 0x55f4f91edbe0 2023-07-18T03:16:49.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.512+0000 7fd62b59fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f90df7c0 con 0x55f4f91ed2d0 2023-07-18T03:16:49.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.512+0000 7fd62b59fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f8fc3920 con 0x55f4f91f8ad0 2023-07-18T03:16:49.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.512+0000 7fd61599b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4f91ed2d0 0x55f4f91f8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:49.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.512+0000 7fd62b59fb80 10 client.0 _ll_register_callbacks cb 0x55f4f90dd760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:16:49.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.512+0000 7fd61599b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4f91ed2d0 0x55f4f91f8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60374/0 (socket says 172.21.15.101:60374) 2023-07-18T03:16:49.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.512+0000 7fd61599b700 1 -- 172.21.15.101:0/115713746 learned_addr learned my addr 172.21.15.101:0/115713746 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:49.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.512+0000 7fd61519a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4f91edbe0 0x55f4f91f57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:49.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.512+0000 7fd61619c700 1 --2- 172.21.15.101:0/115713746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f4f91f8ad0 0x55f4f91f5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:49.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.513+0000 7fd61519a700 1 -- 172.21.15.101:0/115713746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f4f91f8ad0 msgr2=0x55f4f91f5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:49.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.513+0000 7fd61519a700 1 --2- 172.21.15.101:0/115713746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f4f91f8ad0 0x55f4f91f5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.513+0000 7fd61519a700 1 -- 172.21.15.101:0/115713746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4f91ed2d0 msgr2=0x55f4f91f8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:49.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.513+0000 7fd61519a700 1 --2- 172.21.15.101:0/115713746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4f91ed2d0 0x55f4f91f8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.513+0000 7fd61519a700 1 -- 172.21.15.101:0/115713746 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6000098e0 con 0x55f4f91edbe0 2023-07-18T03:16:49.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.513+0000 7fd61519a700 1 --2- 172.21.15.101:0/115713746 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4f91edbe0 0x55f4f91f57c0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fd600011940 tx=0x7fd600011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c7ddd0a8481653 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:49.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.513+0000 7fd5f97fa700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:16:49.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.513+0000 7fd5f97fa700 1 -- 172.21.15.101:0/115713746 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd600016070 con 0x55f4f91edbe0 2023-07-18T03:16:49.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.513+0000 7fd62b59fb80 1 -- 172.21.15.101:0/115713746 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f4f90994f0 con 0x55f4f91edbe0 2023-07-18T03:16:49.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.513+0000 7fd5f97fa700 1 -- 172.21.15.101:0/115713746 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fd60000fa20 con 0x55f4f91edbe0 2023-07-18T03:16:49.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.513+0000 7fd5f97fa700 1 -- 172.21.15.101:0/115713746 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd60000fbe0 con 0x55f4f91edbe0 2023-07-18T03:16:49.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.514+0000 7fd62b59fb80 10 client.17451 fetch_fsmap learned FSMap version 33 2023-07-18T03:16:49.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.514+0000 7fd62b59fb80 1 -- 172.21.15.101:0/115713746 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7fd608011380 con 0x55f4f91edbe0 2023-07-18T03:16:49.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.514+0000 7fd5f97fa700 1 -- 172.21.15.101:0/115713746 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f4f90994f0 con 0x55f4f91edbe0 2023-07-18T03:16:49.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.514+0000 7fd5f97fa700 1 -- 172.21.15.101:0/115713746 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd600003af0 con 0x55f4f91edbe0 2023-07-18T03:16:49.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.514+0000 7fd62b59fb80 10 client.17451 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:16:49.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.514+0000 7fd62b59fb80 10 client.17451 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:16:49.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.514+0000 7fd62b59fb80 20 client.17451 populate_metadata read hostname 'smithi101' 2023-07-18T03:16:49.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.514+0000 7fd62b59fb80 1 --2- 172.21.15.101:0/115713746 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f4f92084c0 0x55f4f920a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:49.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.514+0000 7fd62b59fb80 4 client.17451 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:16:49.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.514+0000 7fd62b59fb80 1 -- 172.21.15.101:0/115713746 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f4f9138eb0 con 0x55f4f92084c0 2023-07-18T03:16:49.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.514+0000 7fd61599b700 1 --2- 172.21.15.101:0/115713746 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f4f92084c0 0x55f4f920a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:49.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.515+0000 7fd61599b700 1 --2- 172.21.15.101:0/115713746 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f4f92084c0 0x55f4f920a8d0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=751ec1c3180c8bbf server_cookie=6e7d53ea122e84d in_seq=0 out_seq=0 2023-07-18T03:16:49.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.515+0000 7fd5f97fa700 10 client.17451 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:16:49.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.515+0000 7fd5f97fa700 1 -- 172.21.15.101:0/115713746 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f4f9138eb0 con 0x55f4f92084c0 2023-07-18T03:16:49.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.515+0000 7fd5f97fa700 10 client.17451 handle_command_reply: tid=0 2023-07-18T03:16:49.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.515+0000 7fd62b59fb80 1 client.17451 shutdown 2023-07-18T03:16:49.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.516+0000 7fd62b59fb80 1 -- 172.21.15.101:0/115713746 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4f91edbe0 msgr2=0x55f4f91f57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:49.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.516+0000 7fd62b59fb80 1 --2- 172.21.15.101:0/115713746 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4f91edbe0 0x55f4f91f57c0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fd600011940 tx=0x7fd600011c50 comp rx=0 tx=0).stop 2023-07-18T03:16:49.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.516+0000 7fd62b59fb80 1 -- 172.21.15.101:0/115713746 shutdown_connections 2023-07-18T03:16:49.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.516+0000 7fd62b59fb80 1 --2- 172.21.15.101:0/115713746 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f4f92084c0 0x55f4f920a8d0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.516+0000 7fd62b59fb80 1 --2- 172.21.15.101:0/115713746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4f91ed2d0 0x55f4f91f8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.516+0000 7fd62b59fb80 1 --2- 172.21.15.101:0/115713746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f4f91f8ad0 0x55f4f91f5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.516+0000 7fd5f97fa700 0 client.17451 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:16:49.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.516+0000 7fd62b59fb80 1 --2- 172.21.15.101:0/115713746 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4f91edbe0 0x55f4f91f57c0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.516+0000 7fd62b59fb80 1 -- 172.21.15.101:0/115713746 >> 172.21.15.101:0/115713746 conn(0x55f4f91e3da0 msgr2=0x55f4f91e9230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:49.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.516+0000 7fd62b59fb80 1 -- 172.21.15.101:0/115713746 shutdown_connections 2023-07-18T03:16:49.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.516+0000 7fd62b59fb80 1 -- 172.21.15.101:0/115713746 wait complete. 2023-07-18T03:16:49.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.516+0000 7fd62b59fb80 20 client.17451 trim_cache size 0 max 16384 2023-07-18T03:16:49.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.535+0000 7fd62b59fb80 1 Processor -- start 2023-07-18T03:16:49.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.535+0000 7fd62b59fb80 1 -- start start 2023-07-18T03:16:49.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.536+0000 7fd62b59fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4f91ed2d0 0x55f4f91ff4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:49.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.536+0000 7fd62b59fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4f91edbe0 0x55f4f9203c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:49.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.536+0000 7fd62b59fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f4f91f8ad0 0x55f4f91f8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:49.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.536+0000 7fd62b59fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f91f68d0 con 0x55f4f91ed2d0 2023-07-18T03:16:49.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.536+0000 7fd62b59fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f91ffa30 con 0x55f4f91edbe0 2023-07-18T03:16:49.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.536+0000 7fd62b59fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f91f5eb0 con 0x55f4f91f8ad0 2023-07-18T03:16:49.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.536+0000 7fd61519a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4f91edbe0 0x55f4f9203c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:49.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.536+0000 7fd61519a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4f91edbe0 0x55f4f9203c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60390/0 (socket says 172.21.15.101:60390) 2023-07-18T03:16:49.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.536+0000 7fd61519a700 1 -- 172.21.15.101:0/3378247148 learned_addr learned my addr 172.21.15.101:0/3378247148 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:49.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.536+0000 7fd61599b700 1 --2- 172.21.15.101:0/3378247148 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4f91ed2d0 0x55f4f91ff4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:49.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.536+0000 7fd61619c700 1 --2- 172.21.15.101:0/3378247148 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f4f91f8ad0 0x55f4f91f8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:49.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.536+0000 7fd61519a700 1 -- 172.21.15.101:0/3378247148 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f4f91f8ad0 msgr2=0x55f4f91f8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:49.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.536+0000 7fd61519a700 1 --2- 172.21.15.101:0/3378247148 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f4f91f8ad0 0x55f4f91f8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.536+0000 7fd61519a700 1 -- 172.21.15.101:0/3378247148 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4f91ed2d0 msgr2=0x55f4f91ff4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:49.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.536+0000 7fd61519a700 1 --2- 172.21.15.101:0/3378247148 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4f91ed2d0 0x55f4f91ff4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.536+0000 7fd61519a700 1 -- 172.21.15.101:0/3378247148 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4f8f56c60 con 0x55f4f91edbe0 2023-07-18T03:16:49.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.537+0000 7fd61519a700 1 --2- 172.21.15.101:0/3378247148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4f91edbe0 0x55f4f9203c90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd600009340 tx=0x7fd6000096d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3efb3f56460a5668 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:49.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.537+0000 7fd604bff700 1 -- 172.21.15.101:0/3378247148 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd600016070 con 0x55f4f91edbe0 2023-07-18T03:16:49.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.537+0000 7fd604bff700 1 -- 172.21.15.101:0/3378247148 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd600003680 con 0x55f4f91edbe0 2023-07-18T03:16:49.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.537+0000 7fd604bff700 1 -- 172.21.15.101:0/3378247148 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd600005400 con 0x55f4f91edbe0 2023-07-18T03:16:49.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.538+0000 7fd62b59fb80 1 -- 172.21.15.101:0/3378247148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4f91edbe0 msgr2=0x55f4f9203c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:49.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.538+0000 7fd62b59fb80 1 --2- 172.21.15.101:0/3378247148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4f91edbe0 0x55f4f9203c90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd600009340 tx=0x7fd6000096d0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.538+0000 7fd62b59fb80 1 -- 172.21.15.101:0/3378247148 shutdown_connections 2023-07-18T03:16:49.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.538+0000 7fd62b59fb80 1 --2- 172.21.15.101:0/3378247148 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f4f91f8ad0 0x55f4f91f8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.538+0000 7fd62b59fb80 1 --2- 172.21.15.101:0/3378247148 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4f91ed2d0 0x55f4f91ff4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.538+0000 7fd62b59fb80 1 --2- 172.21.15.101:0/3378247148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4f91edbe0 0x55f4f9203c90 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.538+0000 7fd62b59fb80 1 -- 172.21.15.101:0/3378247148 >> 172.21.15.101:0/3378247148 conn(0x55f4f91e3da0 msgr2=0x55f4f920f230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:49.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.538+0000 7fd62b59fb80 1 -- 172.21.15.101:0/3378247148 shutdown_connections 2023-07-18T03:16:49.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.538+0000 7fd62b59fb80 1 -- 172.21.15.101:0/3378247148 wait complete. 2023-07-18T03:16:49.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.538+0000 7fd62b59fb80 1 -- start start 2023-07-18T03:16:49.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.538+0000 7fd62b59fb80 1 Processor -- start 2023-07-18T03:16:49.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.539+0000 7fd62b59fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f4f91ed2d0 0x55f4f91ff4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:49.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.539+0000 7fd62b59fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4f91edbe0 0x55f4f91fd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:49.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.539+0000 7fd62b59fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4f91f8ad0 0x55f4f91fdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:49.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.539+0000 7fd62b59fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f91fe5d0 con 0x55f4f91f8ad0 2023-07-18T03:16:49.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.539+0000 7fd62b59fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f925c390 con 0x55f4f91edbe0 2023-07-18T03:16:49.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.539+0000 7fd62b59fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f925c4d0 con 0x55f4f91ed2d0 2023-07-18T03:16:49.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.539+0000 7fd62b59fb80 10 client.0 _ll_register_callbacks cb 0x7fd600011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:16:49.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.539+0000 7fd61599b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f4f91ed2d0 0x55f4f91ff4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:49.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.539+0000 7fd61519a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4f91edbe0 0x55f4f91fd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:49.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.539+0000 7fd61599b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f4f91ed2d0 0x55f4f91ff4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:47780/0 (socket says 172.21.15.101:47780) 2023-07-18T03:16:49.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.539+0000 7fd61619c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4f91f8ad0 0x55f4f91fdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:49.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.539+0000 7fd61599b700 1 -- 172.21.15.101:0/3118524506 learned_addr learned my addr 172.21.15.101:0/3118524506 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:16:49.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.540+0000 7fd61599b700 1 -- 172.21.15.101:0/3118524506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4f91edbe0 msgr2=0x55f4f91fd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:49.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.540+0000 7fd61599b700 1 --2- 172.21.15.101:0/3118524506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4f91edbe0 0x55f4f91fd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.540+0000 7fd61599b700 1 -- 172.21.15.101:0/3118524506 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4f91f8ad0 msgr2=0x55f4f91fdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:16:49.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.540+0000 7fd61599b700 1 --2- 172.21.15.101:0/3118524506 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4f91f8ad0 0x55f4f91fdec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.540+0000 7fd61599b700 1 -- 172.21.15.101:0/3118524506 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd608011380 con 0x55f4f91ed2d0 2023-07-18T03:16:49.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.540+0000 7fd61519a700 1 --2- 172.21.15.101:0/3118524506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4f91edbe0 0x55f4f91fd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:16:49.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.540+0000 7fd61619c700 1 --2- 172.21.15.101:0/3118524506 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4f91f8ad0 0x55f4f91fdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:16:49.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.540+0000 7fd61599b700 1 --2- 172.21.15.101:0/3118524506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f4f91ed2d0 0x55f4f91ff4a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd60c09b110 tx=0x7fd60c0d0840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23a7a70318ed77bd server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:16:49.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.540+0000 7fd5f97fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:16:49.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.540+0000 7fd5f97fa700 1 -- 172.21.15.101:0/3118524506 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c09aae0 con 0x55f4f91ed2d0 2023-07-18T03:16:49.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.540+0000 7fd5f97fa700 1 -- 172.21.15.101:0/3118524506 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fd60c09f1b0 con 0x55f4f91ed2d0 2023-07-18T03:16:49.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.540+0000 7fd62b59fb80 1 -- 172.21.15.101:0/3118524506 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd60800f040 con 0x55f4f91ed2d0 2023-07-18T03:16:49.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.540+0000 7fd5f97fa700 1 -- 172.21.15.101:0/3118524506 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd60c0974c0 con 0x55f4f91ed2d0 2023-07-18T03:16:49.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.540+0000 7fd62b59fb80 10 client.27532 fetch_fsmap learned FSMap version 33 2023-07-18T03:16:49.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.540+0000 7fd62b59fb80 1 -- 172.21.15.101:0/3118524506 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55f4f8f94250 con 0x55f4f91ed2d0 2023-07-18T03:16:49.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.540+0000 7fd5f97fa700 1 -- 172.21.15.101:0/3118524506 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd60800f040 con 0x55f4f91ed2d0 2023-07-18T03:16:49.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.541+0000 7fd5f97fa700 1 -- 172.21.15.101:0/3118524506 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd60c09a670 con 0x55f4f91ed2d0 2023-07-18T03:16:49.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.541+0000 7fd62b59fb80 10 client.27532 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:16:49.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.541+0000 7fd62b59fb80 10 client.27532 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:16:49.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.541+0000 7fd62b59fb80 20 client.27532 populate_metadata read hostname 'smithi101' 2023-07-18T03:16:49.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.541+0000 7fd62b59fb80 1 --2- 172.21.15.101:0/3118524506 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f4f925d850 0x55f4f925fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:16:49.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.541+0000 7fd62b59fb80 4 client.27532 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:16:49.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.541+0000 7fd62b59fb80 1 -- 172.21.15.101:0/3118524506 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f4f90fb050 con 0x55f4f925d850 2023-07-18T03:16:49.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.541+0000 7fd61519a700 1 --2- 172.21.15.101:0/3118524506 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f4f925d850 0x55f4f925fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:16:49.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.541+0000 7fd5f97fa700 10 client.27532 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:16:49.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.541+0000 7fd61519a700 1 --2- 172.21.15.101:0/3118524506 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f4f925d850 0x55f4f925fc30 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=441154a20b52a93f server_cookie=4be72aeb3bc9fd62 in_seq=0 out_seq=0 2023-07-18T03:16:49.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.542+0000 7fd5f97fa700 1 -- 172.21.15.101:0/3118524506 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55f4f90fb050 con 0x55f4f925d850 2023-07-18T03:16:49.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.542+0000 7fd5f97fa700 10 client.27532 handle_command_reply: tid=0 2023-07-18T03:16:49.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.542+0000 7fd62b59fb80 1 client.27532 shutdown 2023-07-18T03:16:49.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.542+0000 7fd62b59fb80 1 -- 172.21.15.101:0/3118524506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f4f91ed2d0 msgr2=0x55f4f91ff4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:49.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.542+0000 7fd62b59fb80 1 --2- 172.21.15.101:0/3118524506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f4f91ed2d0 0x55f4f91ff4a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd60c09b110 tx=0x7fd60c0d0840 comp rx=0 tx=0).stop 2023-07-18T03:16:49.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.542+0000 7fd62b59fb80 1 -- 172.21.15.101:0/3118524506 shutdown_connections 2023-07-18T03:16:49.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.542+0000 7fd62b59fb80 1 --2- 172.21.15.101:0/3118524506 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f4f925d850 0x55f4f925fc30 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.544 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.542+0000 7fd62b59fb80 1 --2- 172.21.15.101:0/3118524506 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4f91f8ad0 0x55f4f91fdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.544 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.542+0000 7fd62b59fb80 1 --2- 172.21.15.101:0/3118524506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f4f91ed2d0 0x55f4f91ff4a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.544 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.542+0000 7fd62b59fb80 1 --2- 172.21.15.101:0/3118524506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f4f91edbe0 0x55f4f91fd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.544 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.542+0000 7fd5f97fa700 0 client.27532 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:16:49.544 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.542+0000 7fd62b59fb80 1 -- 172.21.15.101:0/3118524506 >> 172.21.15.101:0/3118524506 conn(0x55f4f91e3da0 msgr2=0x55f4f91ef3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:49.544 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.542+0000 7fd62b59fb80 1 -- 172.21.15.101:0/3118524506 shutdown_connections 2023-07-18T03:16:49.544 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.542+0000 7fd62b59fb80 1 -- 172.21.15.101:0/3118524506 wait complete. 2023-07-18T03:16:49.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.542+0000 7fd62b59fb80 20 client.27532 trim_cache size 0 max 16384 2023-07-18T03:16:49.545 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:16:49.545 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:16:49.545 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:16:49.545 INFO:teuthology.orchestra.run.smithi101.stdout: "f05717c7-804b-4547-982a-1724b5dada5e": { 2023-07-18T03:16:49.545 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:16:49.546 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "f05717c7-804b-4547-982a-1724b5dada5e", 2023-07-18T03:16:49.546 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:16:49.546 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:16:49.546 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:16:49.546 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:16:49.547 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.543+0000 7fd617bff700 1 -- 172.21.15.101:0/281055442 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd5fc0a12c0 msgr2=0x7fd5fc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:49.547 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.543+0000 7fd617bff700 1 --2- 172.21.15.101:0/281055442 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd5fc0a12c0 0x7fd5fc0a3770 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd608022000 tx=0x7fd60801f000 comp rx=0 tx=0).stop 2023-07-18T03:16:49.547 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.543+0000 7fd617bff700 1 -- 172.21.15.101:0/281055442 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd6100ffae0 msgr2=0x7fd610197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:16:49.547 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.543+0000 7fd617bff700 1 --2- 172.21.15.101:0/281055442 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd6100ffae0 0x7fd610197df0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fd60c005910 tx=0x7fd60c005720 comp rx=0 tx=0).stop 2023-07-18T03:16:49.547 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.543+0000 7fd617bff700 1 -- 172.21.15.101:0/281055442 shutdown_connections 2023-07-18T03:16:49.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.543+0000 7fd617bff700 1 --2- 172.21.15.101:0/281055442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd610105af0 0x7fd61019c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.543+0000 7fd617bff700 1 --2- 172.21.15.101:0/281055442 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd6100ffae0 0x7fd610197df0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.543+0000 7fd617bff700 1 --2- 172.21.15.101:0/281055442 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd5fc0a12c0 0x7fd5fc0a3770 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.543+0000 7fd617bff700 1 --2- 172.21.15.101:0/281055442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd610100490 0x7fd610198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:16:49.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.543+0000 7fd617bff700 1 -- 172.21.15.101:0/281055442 >> 172.21.15.101:0/281055442 conn(0x7fd6100fb5e0 msgr2=0x7fd61010aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:16:49.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.543+0000 7fd617bff700 1 -- 172.21.15.101:0/281055442 shutdown_connections 2023-07-18T03:16:49.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:16:49.543+0000 7fd617bff700 1 -- 172.21.15.101:0/281055442 wait complete. 2023-07-18T03:16:49.556 INFO:tasks.cephfs.filesystem:scrub status for tag:f05717c7-804b-4547-982a-1724b5dada5e - {'path': '/', 'tag': 'f05717c7-804b-4547-982a-1724b5dada5e', 'options': 'recursive,force'} 2023-07-18T03:16:49.606 INFO:tasks.workunit.client.0.smithi101.stdout: 1 602684 6.11 MB/sec execute 374 sec latency 2381.173 ms 2023-07-18T03:16:50.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:50 smithi155 ceph-mon[128885]: scrub queued for path: / 2023-07-18T03:16:50.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:50 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:16:50.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:50 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:16:50.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:50 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:16:50.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:50 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:16:50.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:50 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:16:50.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:50 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:16:50.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:50 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:16:50.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:50 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:16:50.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:50 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:16:50.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:50 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:16:50.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:50 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:16:50.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:50 smithi179 ceph-mon[129229]: scrub queued for path: / 2023-07-18T03:16:50.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:50 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:16:50.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:50 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:16:50.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:50 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:16:50.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:50 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:16:50.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:50 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:16:50.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:50 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:16:50.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:50 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:16:50.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:50 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:16:50.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:50 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:16:50.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:50 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:16:50.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:50 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:16:50.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:50 smithi101 ceph-mon[126379]: scrub queued for path: / 2023-07-18T03:16:50.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:50 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:16:50.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:50 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:16:50.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:50 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:16:50.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:50 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:16:50.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:50 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:16:50.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:50 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:16:50.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:50 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:16:50.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:50 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:16:50.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:50 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:16:50.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:50 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:16:50.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:50 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:16:50.606 INFO:tasks.workunit.client.0.smithi101.stdout: 1 604217 6.13 MB/sec execute 375 sec latency 11.026 ms 2023-07-18T03:16:51.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:51 smithi155 ceph-mon[128885]: pgmap v287: 129 pgs: 129 active+clean; 454 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 157 op/s 2023-07-18T03:16:51.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:51 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:16:51.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:51 smithi179 ceph-mon[129229]: pgmap v287: 129 pgs: 129 active+clean; 454 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 157 op/s 2023-07-18T03:16:51.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:51 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:16:51.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:51 smithi101 ceph-mon[126379]: pgmap v287: 129 pgs: 129 active+clean; 454 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 157 op/s 2023-07-18T03:16:51.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:51 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:16:51.606 INFO:tasks.workunit.client.0.smithi101.stdout: 1 605322 6.13 MB/sec execute 376 sec latency 41.705 ms 2023-07-18T03:16:52.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:52 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:16:52.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:52 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:16:52.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:52 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:16:52.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:52 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:16:52.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:52 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:16:52.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:52 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:16:52.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:52 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:16:52.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:52 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:16:52.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:52 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:16:52.606 INFO:tasks.workunit.client.0.smithi101.stdout: 1 605495 6.11 MB/sec execute 377 sec latency 826.003 ms 2023-07-18T03:16:53.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:53 smithi155 ceph-mon[128885]: pgmap v288: 129 pgs: 129 active+clean; 454 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 109 op/s 2023-07-18T03:16:53.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:53 smithi179 ceph-mon[129229]: pgmap v288: 129 pgs: 129 active+clean; 454 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 109 op/s 2023-07-18T03:16:53.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:53 smithi101 ceph-mon[126379]: pgmap v288: 129 pgs: 129 active+clean; 454 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 109 op/s 2023-07-18T03:16:53.607 INFO:tasks.workunit.client.0.smithi101.stdout: 1 605495 6.09 MB/sec execute 378 sec latency 1826.142 ms 2023-07-18T03:16:54.607 INFO:tasks.workunit.client.0.smithi101.stdout: 1 605946 6.08 MB/sec execute 379 sec latency 2647.621 ms 2023-07-18T03:16:55.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:55 smithi155 ceph-mon[128885]: pgmap v289: 129 pgs: 129 active+clean; 454 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 108 op/s 2023-07-18T03:16:55.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:55 smithi179 ceph-mon[129229]: pgmap v289: 129 pgs: 129 active+clean; 454 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 108 op/s 2023-07-18T03:16:55.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:55 smithi101 ceph-mon[126379]: pgmap v289: 129 pgs: 129 active+clean; 454 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 108 op/s 2023-07-18T03:16:55.607 INFO:tasks.workunit.client.0.smithi101.stdout: 1 607625 6.10 MB/sec execute 380 sec latency 13.554 ms 2023-07-18T03:16:56.607 INFO:tasks.workunit.client.0.smithi101.stdout: 1 608595 6.09 MB/sec execute 381 sec latency 23.884 ms 2023-07-18T03:16:57.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:57 smithi155 ceph-mon[128885]: pgmap v290: 129 pgs: 129 active+clean; 458 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 156 op/s 2023-07-18T03:16:57.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:57 smithi179 ceph-mon[129229]: pgmap v290: 129 pgs: 129 active+clean; 458 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 156 op/s 2023-07-18T03:16:57.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:57 smithi101 ceph-mon[126379]: pgmap v290: 129 pgs: 129 active+clean; 458 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 156 op/s 2023-07-18T03:16:57.608 INFO:tasks.workunit.client.0.smithi101.stdout: 1 609036 6.08 MB/sec execute 382 sec latency 691.759 ms 2023-07-18T03:16:58.608 INFO:tasks.workunit.client.0.smithi101.stdout: 1 609036 6.06 MB/sec execute 383 sec latency 1691.884 ms 2023-07-18T03:16:59.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:16:59 smithi179 ceph-mon[129229]: pgmap v291: 129 pgs: 129 active+clean; 458 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 103 op/s 2023-07-18T03:16:59.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:16:59 smithi101 ceph-mon[126379]: pgmap v291: 129 pgs: 129 active+clean; 458 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 103 op/s 2023-07-18T03:16:59.546 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:16:59 smithi155 ceph-mon[128885]: pgmap v291: 129 pgs: 129 active+clean; 458 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 103 op/s 2023-07-18T03:16:59.608 INFO:tasks.workunit.client.0.smithi101.stdout: 1 609435 6.05 MB/sec execute 384 sec latency 2528.038 ms 2023-07-18T03:17:00.608 INFO:tasks.workunit.client.0.smithi101.stdout: 1 611162 6.07 MB/sec execute 385 sec latency 18.101 ms 2023-07-18T03:17:01.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:01 smithi179 ceph-mon[129229]: pgmap v292: 129 pgs: 129 active+clean; 462 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 152 op/s 2023-07-18T03:17:01.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:01 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:17:01.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:01 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:17:01.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:01 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:17:01.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:01 smithi101 ceph-mon[126379]: pgmap v292: 129 pgs: 129 active+clean; 462 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 152 op/s 2023-07-18T03:17:01.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:01 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:17:01.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:01 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:17:01.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:01 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:17:01.546 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:01 smithi155 ceph-mon[128885]: pgmap v292: 129 pgs: 129 active+clean; 462 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 152 op/s 2023-07-18T03:17:01.546 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:01 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:17:01.546 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:01 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:17:01.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:01 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:17:01.608 INFO:tasks.workunit.client.0.smithi101.stdout: 1 611909 6.06 MB/sec execute 386 sec latency 61.795 ms 2023-07-18T03:17:02.608 INFO:tasks.workunit.client.0.smithi101.stdout: 1 612387 6.05 MB/sec execute 387 sec latency 90.027 ms 2023-07-18T03:17:03.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:03 smithi179 ceph-mon[129229]: pgmap v293: 129 pgs: 129 active+clean; 462 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 100 op/s 2023-07-18T03:17:03.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:03 smithi101 ceph-mon[126379]: pgmap v293: 129 pgs: 129 active+clean; 462 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 100 op/s 2023-07-18T03:17:03.546 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:03 smithi155 ceph-mon[128885]: pgmap v293: 129 pgs: 129 active+clean; 462 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 100 op/s 2023-07-18T03:17:03.609 INFO:tasks.workunit.client.0.smithi101.stdout: 1 612586 6.04 MB/sec execute 388 sec latency 818.980 ms 2023-07-18T03:17:04.609 INFO:tasks.workunit.client.0.smithi101.stdout: 1 612873 6.02 MB/sec execute 389 sec latency 1646.566 ms 2023-07-18T03:17:05.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:05 smithi179 ceph-mon[129229]: pgmap v294: 129 pgs: 129 active+clean; 462 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 99 op/s 2023-07-18T03:17:05.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:05 smithi101 ceph-mon[126379]: pgmap v294: 129 pgs: 129 active+clean; 462 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 99 op/s 2023-07-18T03:17:05.546 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:05 smithi155 ceph-mon[128885]: pgmap v294: 129 pgs: 129 active+clean; 462 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 99 op/s 2023-07-18T03:17:05.609 INFO:tasks.workunit.client.0.smithi101.stdout: 1 614654 6.04 MB/sec execute 390 sec latency 16.403 ms 2023-07-18T03:17:06.609 INFO:tasks.workunit.client.0.smithi101.stdout: 1 615349 6.03 MB/sec execute 391 sec latency 67.826 ms 2023-07-18T03:17:07.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:07 smithi179 ceph-mon[129229]: pgmap v295: 129 pgs: 129 active+clean; 466 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 184 op/s 2023-07-18T03:17:07.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:07 smithi179 ceph-mon[129229]: osdmap e99: 12 total, 12 up, 12 in 2023-07-18T03:17:07.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:07 smithi101 ceph-mon[126379]: pgmap v295: 129 pgs: 129 active+clean; 466 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 184 op/s 2023-07-18T03:17:07.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:07 smithi101 ceph-mon[126379]: osdmap e99: 12 total, 12 up, 12 in 2023-07-18T03:17:07.546 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:07 smithi155 ceph-mon[128885]: pgmap v295: 129 pgs: 129 active+clean; 466 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 184 op/s 2023-07-18T03:17:07.546 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:07 smithi155 ceph-mon[128885]: osdmap e99: 12 total, 12 up, 12 in 2023-07-18T03:17:07.609 INFO:tasks.workunit.client.0.smithi101.stdout: 1 616129 6.02 MB/sec execute 392 sec latency 440.661 ms 2023-07-18T03:17:08.609 INFO:tasks.workunit.client.0.smithi101.stdout: 1 616129 6.01 MB/sec execute 393 sec latency 1440.791 ms 2023-07-18T03:17:09.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:09 smithi179 ceph-mon[129229]: pgmap v297: 129 pgs: 129 active+clean; 466 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 163 op/s 2023-07-18T03:17:09.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:09 smithi101 ceph-mon[126379]: pgmap v297: 129 pgs: 129 active+clean; 466 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 163 op/s 2023-07-18T03:17:09.546 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:09 smithi155 ceph-mon[128885]: pgmap v297: 129 pgs: 129 active+clean; 466 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 163 op/s 2023-07-18T03:17:09.610 INFO:tasks.workunit.client.0.smithi101.stdout: 1 616578 5.99 MB/sec execute 394 sec latency 2264.029 ms 2023-07-18T03:17:10.609 INFO:tasks.workunit.client.0.smithi101.stdout: 1 618245 6.01 MB/sec execute 395 sec latency 14.443 ms 2023-07-18T03:17:11.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:11 smithi179 ceph-mon[129229]: pgmap v298: 129 pgs: 129 active+clean; 467 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 165 op/s 2023-07-18T03:17:11.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:11 smithi101 ceph-mon[126379]: pgmap v298: 129 pgs: 129 active+clean; 467 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 165 op/s 2023-07-18T03:17:11.546 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:11 smithi155 ceph-mon[128885]: pgmap v298: 129 pgs: 129 active+clean; 467 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 165 op/s 2023-07-18T03:17:11.609 INFO:tasks.workunit.client.0.smithi101.stdout: 1 619372 6.01 MB/sec execute 396 sec latency 18.054 ms 2023-07-18T03:17:12.610 INFO:tasks.workunit.client.0.smithi101.stdout: 1 619675 6.00 MB/sec execute 397 sec latency 649.272 ms 2023-07-18T03:17:13.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:13 smithi179 ceph-mon[129229]: pgmap v299: 129 pgs: 129 active+clean; 467 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 165 op/s 2023-07-18T03:17:13.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:13 smithi101 ceph-mon[126379]: pgmap v299: 129 pgs: 129 active+clean; 467 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 165 op/s 2023-07-18T03:17:13.546 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:13 smithi155 ceph-mon[128885]: pgmap v299: 129 pgs: 129 active+clean; 467 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 165 op/s 2023-07-18T03:17:13.610 INFO:tasks.workunit.client.0.smithi101.stdout: 1 619675 5.98 MB/sec execute 398 sec latency 1649.334 ms 2023-07-18T03:17:14.610 INFO:tasks.workunit.client.0.smithi101.stdout: 1 620074 5.97 MB/sec execute 399 sec latency 2494.100 ms 2023-07-18T03:17:15.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:14 smithi101 ceph-mon[126379]: pgmap v300: 129 pgs: 129 active+clean; 467 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 165 op/s 2023-07-18T03:17:15.239 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:14 smithi101 ceph-mon[126379]: osdmap e100: 12 total, 12 up, 12 in 2023-07-18T03:17:15.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:14 smithi155 ceph-mon[128885]: pgmap v300: 129 pgs: 129 active+clean; 467 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 165 op/s 2023-07-18T03:17:15.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:14 smithi155 ceph-mon[128885]: osdmap e100: 12 total, 12 up, 12 in 2023-07-18T03:17:15.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:14 smithi179 ceph-mon[129229]: pgmap v300: 129 pgs: 129 active+clean; 467 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 165 op/s 2023-07-18T03:17:15.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:14 smithi179 ceph-mon[129229]: osdmap e100: 12 total, 12 up, 12 in 2023-07-18T03:17:15.610 INFO:tasks.workunit.client.0.smithi101.stdout: 1 621791 5.98 MB/sec execute 400 sec latency 18.712 ms 2023-07-18T03:17:16.610 INFO:tasks.workunit.client.0.smithi101.stdout: 1 622952 5.98 MB/sec execute 401 sec latency 20.847 ms 2023-07-18T03:17:17.238 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:16 smithi101 ceph-mon[126379]: pgmap v302: 129 pgs: 129 active+clean; 473 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 130 op/s 2023-07-18T03:17:17.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:16 smithi155 ceph-mon[128885]: pgmap v302: 129 pgs: 129 active+clean; 473 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 130 op/s 2023-07-18T03:17:17.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:16 smithi179 ceph-mon[129229]: pgmap v302: 129 pgs: 129 active+clean; 473 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 130 op/s 2023-07-18T03:17:17.611 INFO:tasks.workunit.client.0.smithi101.stdout: 1 623219 5.97 MB/sec execute 402 sec latency 658.294 ms 2023-07-18T03:17:18.611 INFO:tasks.workunit.client.0.smithi101.stdout: 1 623219 5.95 MB/sec execute 403 sec latency 1658.442 ms 2023-07-18T03:17:19.239 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:18 smithi101 ceph-mon[126379]: pgmap v303: 129 pgs: 129 active+clean; 473 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 125 op/s 2023-07-18T03:17:19.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:18 smithi155 ceph-mon[128885]: pgmap v303: 129 pgs: 129 active+clean; 473 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 125 op/s 2023-07-18T03:17:19.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:18 smithi179 ceph-mon[129229]: pgmap v303: 129 pgs: 129 active+clean; 473 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 125 op/s 2023-07-18T03:17:19.557 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:17:19.611 INFO:tasks.workunit.client.0.smithi101.stdout: 1 623668 5.94 MB/sec execute 404 sec latency 2481.606 ms 2023-07-18T03:17:19.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.795+0000 7f27f6fff700 1 -- 172.21.15.101:0/160819875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27f0105b10 msgr2=0x7f27f0105ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:19.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.795+0000 7f27f6fff700 1 --2- 172.21.15.101:0/160819875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27f0105b10 0x7f27f0105ee0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f27e4009990 tx=0x7f27e4009ca0 comp rx=0 tx=0).stop 2023-07-18T03:17:19.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.796+0000 7f27f6fff700 1 -- 172.21.15.101:0/160819875 shutdown_connections 2023-07-18T03:17:19.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.796+0000 7f27f6fff700 1 --2- 172.21.15.101:0/160819875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27f01004b0 0x7f27f010c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:19.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.796+0000 7f27f6fff700 1 --2- 172.21.15.101:0/160819875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27f0105b10 0x7f27f0105ee0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:19.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.796+0000 7f27f6fff700 1 --2- 172.21.15.101:0/160819875 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f27f00ffb00 0x7f27f00fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:19.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.796+0000 7f27f6fff700 1 -- 172.21.15.101:0/160819875 >> 172.21.15.101:0/160819875 conn(0x7f27f00fb600 msgr2=0x7f27f00fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:19.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.796+0000 7f27f6fff700 1 -- 172.21.15.101:0/160819875 shutdown_connections 2023-07-18T03:17:19.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.796+0000 7f27f6fff700 1 -- 172.21.15.101:0/160819875 wait complete. 2023-07-18T03:17:19.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.797+0000 7f27f6fff700 1 Processor -- start 2023-07-18T03:17:19.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.797+0000 7f27f6fff700 1 -- start start 2023-07-18T03:17:19.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.798+0000 7f27f6fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f27f00ffb00 0x7f27f019b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:19.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.798+0000 7f27f6fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27f01004b0 0x7f27f019bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:19.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.798+0000 7f27f6fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27f0105b10 0x7f27f0195a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:19.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.798+0000 7f27f6fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f01960d0 con 0x7f27f00ffb00 2023-07-18T03:17:19.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.798+0000 7f27f6fff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f0196240 con 0x7f27f01004b0 2023-07-18T03:17:19.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.798+0000 7f27f6fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f0196400 con 0x7f27f0105b10 2023-07-18T03:17:19.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.798+0000 7f27effff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27f01004b0 0x7f27f019bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:19.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.798+0000 7f27f4d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f27f00ffb00 0x7f27f019b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:19.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.798+0000 7f27effff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27f01004b0 0x7f27f019bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60426/0 (socket says 172.21.15.101:60426) 2023-07-18T03:17:19.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.798+0000 7f27effff700 1 -- 172.21.15.101:0/2852109453 learned_addr learned my addr 172.21.15.101:0/2852109453 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:19.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.798+0000 7f27f559c700 1 --2- 172.21.15.101:0/2852109453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27f0105b10 0x7f27f0195a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:19.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.798+0000 7f27effff700 1 -- 172.21.15.101:0/2852109453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27f0105b10 msgr2=0x7f27f0195a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:19.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.798+0000 7f27effff700 1 --2- 172.21.15.101:0/2852109453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27f0105b10 0x7f27f0195a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:19.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.798+0000 7f27effff700 1 -- 172.21.15.101:0/2852109453 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f27f00ffb00 msgr2=0x7f27f019b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:19.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.798+0000 7f27effff700 1 --2- 172.21.15.101:0/2852109453 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f27f00ffb00 0x7f27f019b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:19.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.798+0000 7f27effff700 1 -- 172.21.15.101:0/2852109453 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e40095f0 con 0x7f27f01004b0 2023-07-18T03:17:19.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.798+0000 7f27f559c700 1 --2- 172.21.15.101:0/2852109453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27f0105b10 0x7f27f0195a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:17:19.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.799+0000 7f27effff700 1 --2- 172.21.15.101:0/2852109453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27f01004b0 0x7f27f019bf30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f27dc00e9a0 tx=0x7f27dc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:19.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.799+0000 7f27edffb700 1 -- 172.21.15.101:0/2852109453 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27dc009790 con 0x7f27f01004b0 2023-07-18T03:17:19.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.799+0000 7f27f6fff700 1 -- 172.21.15.101:0/2852109453 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27f01966e0 con 0x7f27f01004b0 2023-07-18T03:17:19.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.799+0000 7f27f6fff700 1 -- 172.21.15.101:0/2852109453 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27f0072920 con 0x7f27f01004b0 2023-07-18T03:17:19.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.799+0000 7f27edffb700 1 -- 172.21.15.101:0/2852109453 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27dc00cbd0 con 0x7f27f01004b0 2023-07-18T03:17:19.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.799+0000 7f27edffb700 1 -- 172.21.15.101:0/2852109453 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27dc018680 con 0x7f27f01004b0 2023-07-18T03:17:19.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.803+0000 7f27edffb700 1 -- 172.21.15.101:0/2852109453 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f27dc010430 con 0x7f27f01004b0 2023-07-18T03:17:19.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.804+0000 7f280a84bb80 1 Processor -- start 2023-07-18T03:17:19.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.804+0000 7f280a84bb80 1 -- start start 2023-07-18T03:17:19.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.804+0000 7f280a84bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555d8df012d0 0x555d8df016a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:19.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.804+0000 7f280a84bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d8df01be0 0x555d8df0c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:19.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.804+0000 7f280a84bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d8df0cad0 0x555d8df0eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:19.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.804+0000 7f27edffb700 1 --2- 172.21.15.101:0/2852109453 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f27e00a1340 0x7f27e00a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:19.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.804+0000 7f280a84bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d8de07cb0 con 0x555d8df01be0 2023-07-18T03:17:19.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.804+0000 7f280a84bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d8de5aa40 con 0x555d8df0cad0 2023-07-18T03:17:19.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.804+0000 7f280a84bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d8de53ed0 con 0x555d8df012d0 2023-07-18T03:17:19.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.804+0000 7f27edffb700 1 -- 172.21.15.101:0/2852109453 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f27dc014070 con 0x7f27f01004b0 2023-07-18T03:17:19.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.804+0000 7f27f559c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555d8df012d0 0x555d8df016a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:19.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.804+0000 7f27effff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d8df0cad0 0x555d8df0eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:19.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.805+0000 7f27effff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d8df0cad0 0x555d8df0eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60438/0 (socket says 172.21.15.101:60438) 2023-07-18T03:17:19.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.805+0000 7f27f559c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555d8df012d0 0x555d8df016a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:44016/0 (socket says 172.21.15.101:44016) 2023-07-18T03:17:19.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.805+0000 7f27effff700 1 -- 172.21.15.101:0/1499740985 learned_addr learned my addr 172.21.15.101:0/1499740985 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:19.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.805+0000 7f27f4d9b700 1 --2- 172.21.15.101:0/1499740985 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d8df01be0 0x555d8df0c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:19.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.805+0000 7f27f559c700 1 --2- 172.21.15.101:0/2852109453 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f27e00a1340 0x7f27e00a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:19.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.805+0000 7f27effff700 1 -- 172.21.15.101:0/1499740985 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555d8df012d0 msgr2=0x555d8df016a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:19.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.805+0000 7f27effff700 1 --2- 172.21.15.101:0/1499740985 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555d8df012d0 0x555d8df016a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:19.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.805+0000 7f27effff700 1 -- 172.21.15.101:0/1499740985 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d8df01be0 msgr2=0x555d8df0c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:19.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.805+0000 7f27effff700 1 --2- 172.21.15.101:0/1499740985 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d8df01be0 0x555d8df0c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:19.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.805+0000 7f27effff700 1 -- 172.21.15.101:0/1499740985 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d8dc81de0 con 0x555d8df0cad0 2023-07-18T03:17:19.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.805+0000 7f27effff700 1 --2- 172.21.15.101:0/1499740985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d8df0cad0 0x555d8df0eeb0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f27dc096a60 tx=0x7f27dc011f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b88081cf17953d47 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:19.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.805+0000 7f27f559c700 1 --2- 172.21.15.101:0/2852109453 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f27e00a1340 0x7f27e00a37f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f27f019c6f0 tx=0x7f27e800b040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:19.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.805+0000 7f27daffd700 1 -- 172.21.15.101:0/1499740985 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27dc0cd160 con 0x555d8df0cad0 2023-07-18T03:17:19.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.805+0000 7f27daffd700 1 -- 172.21.15.101:0/1499740985 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27dc099ff0 con 0x555d8df0cad0 2023-07-18T03:17:19.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.805+0000 7f27daffd700 1 -- 172.21.15.101:0/1499740985 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27dc09ff50 con 0x555d8df0cad0 2023-07-18T03:17:19.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.806+0000 7f280a84bb80 1 -- 172.21.15.101:0/1499740985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d8df0cad0 msgr2=0x555d8df0eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:19.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.806+0000 7f280a84bb80 1 --2- 172.21.15.101:0/1499740985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d8df0cad0 0x555d8df0eeb0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f27dc096a60 tx=0x7f27dc011f90 comp rx=0 tx=0).stop 2023-07-18T03:17:19.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.806+0000 7f280a84bb80 1 -- 172.21.15.101:0/1499740985 shutdown_connections 2023-07-18T03:17:19.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.806+0000 7f280a84bb80 1 --2- 172.21.15.101:0/1499740985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d8df0cad0 0x555d8df0eeb0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:19.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.806+0000 7f280a84bb80 1 --2- 172.21.15.101:0/1499740985 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555d8df012d0 0x555d8df016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:19.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.806+0000 7f280a84bb80 1 --2- 172.21.15.101:0/1499740985 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d8df01be0 0x555d8df0c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:19.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.806+0000 7f280a84bb80 1 -- 172.21.15.101:0/1499740985 >> 172.21.15.101:0/1499740985 conn(0x555d8def7da0 msgr2=0x555d8defd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:19.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.806+0000 7f280a84bb80 1 -- 172.21.15.101:0/1499740985 shutdown_connections 2023-07-18T03:17:19.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.806+0000 7f280a84bb80 1 -- 172.21.15.101:0/1499740985 wait complete. 2023-07-18T03:17:19.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.807+0000 7f280a84bb80 1 -- start start 2023-07-18T03:17:19.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.807+0000 7f280a84bb80 1 Processor -- start 2023-07-18T03:17:19.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.808+0000 7f280a84bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555d8df012d0 0x555d8df0c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:19.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.808+0000 7f280a84bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d8df01be0 0x555d8df096d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:19.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.808+0000 7f280a84bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d8df0cad0 0x555d8df09d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:19.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.808+0000 7f280a84bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d8ddf3ac0 con 0x555d8df01be0 2023-07-18T03:17:19.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.808+0000 7f280a84bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d8ddf37c0 con 0x555d8df0cad0 2023-07-18T03:17:19.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.808+0000 7f280a84bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d8dcd7920 con 0x555d8df012d0 2023-07-18T03:17:19.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.808+0000 7f280a84bb80 10 client.0 _ll_register_callbacks cb 0x555d8ddf1760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:17:19.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.808+0000 7f27f4d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555d8df012d0 0x555d8df0c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:19.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.808+0000 7f27effff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d8df01be0 0x555d8df096d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:19.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.808+0000 7f27f4d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555d8df012d0 0x555d8df0c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:44030/0 (socket says 172.21.15.101:44030) 2023-07-18T03:17:19.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.808+0000 7f27f4d9b700 1 -- 172.21.15.101:0/815128092 learned_addr learned my addr 172.21.15.101:0/815128092 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:19.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.808+0000 7f27f559c700 1 --2- 172.21.15.101:0/815128092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d8df0cad0 0x555d8df09d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:19.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.808+0000 7f27f4d9b700 1 -- 172.21.15.101:0/815128092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d8df0cad0 msgr2=0x555d8df09d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:19.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.808+0000 7f27f4d9b700 1 --2- 172.21.15.101:0/815128092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d8df0cad0 0x555d8df09d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:19.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.808+0000 7f27f4d9b700 1 -- 172.21.15.101:0/815128092 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d8df01be0 msgr2=0x555d8df096d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:19.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.808+0000 7f27f4d9b700 1 --2- 172.21.15.101:0/815128092 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d8df01be0 0x555d8df096d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:19.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.808+0000 7f27f4d9b700 1 -- 172.21.15.101:0/815128092 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27f0072920 con 0x555d8df012d0 2023-07-18T03:17:19.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.808+0000 7f27f559c700 1 --2- 172.21.15.101:0/815128092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d8df0cad0 0x555d8df09d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:17:19.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.809+0000 7f27f4d9b700 1 --2- 172.21.15.101:0/815128092 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555d8df012d0 0x555d8df0c450 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f27e400c010 tx=0x7f27e4005ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f7ff751f68ed271 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:19.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.809+0000 7f27cffff700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:17:19.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.809+0000 7f27cffff700 1 -- 172.21.15.101:0/815128092 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e401d070 con 0x555d8df012d0 2023-07-18T03:17:19.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.809+0000 7f280a84bb80 1 -- 172.21.15.101:0/815128092 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555d8ddad4f0 con 0x555d8df012d0 2023-07-18T03:17:19.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.809+0000 7f27cffff700 1 -- 172.21.15.101:0/815128092 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f27e4004500 con 0x555d8df012d0 2023-07-18T03:17:19.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.809+0000 7f27cffff700 1 -- 172.21.15.101:0/815128092 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27e40046c0 con 0x555d8df012d0 2023-07-18T03:17:19.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.809+0000 7f280a84bb80 10 client.27544 fetch_fsmap learned FSMap version 33 2023-07-18T03:17:19.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.809+0000 7f280a84bb80 1 -- 172.21.15.101:0/815128092 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x555d8dc6ac60 con 0x555d8df012d0 2023-07-18T03:17:19.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.809+0000 7f27cffff700 1 -- 172.21.15.101:0/815128092 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555d8ddad4f0 con 0x555d8df012d0 2023-07-18T03:17:19.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.810+0000 7f27cffff700 1 -- 172.21.15.101:0/815128092 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f27e400f980 con 0x555d8df012d0 2023-07-18T03:17:19.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.810+0000 7f280a84bb80 10 client.27544 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:17:19.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.810+0000 7f280a84bb80 10 client.27544 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:17:19.813 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.810+0000 7f280a84bb80 20 client.27544 populate_metadata read hostname 'smithi101' 2023-07-18T03:17:19.813 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.810+0000 7f280a84bb80 1 --2- 172.21.15.101:0/815128092 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x555d8df1c370 0x555d8df1e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:19.813 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.810+0000 7f280a84bb80 4 client.27544 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:17:19.813 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.810+0000 7f280a84bb80 1 -- 172.21.15.101:0/815128092 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555d8de4ceb0 con 0x555d8df1c370 2023-07-18T03:17:19.813 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.810+0000 7f27effff700 1 --2- 172.21.15.101:0/815128092 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x555d8df1c370 0x555d8df1e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:19.813 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.810+0000 7f27cffff700 10 client.27544 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:17:19.813 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.810+0000 7f27effff700 1 --2- 172.21.15.101:0/815128092 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x555d8df1c370 0x555d8df1e780 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a4b189d074ed799 server_cookie=708fea12ae62dc8d in_seq=0 out_seq=0 2023-07-18T03:17:19.814 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.811+0000 7f27cffff700 1 -- 172.21.15.101:0/815128092 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555d8de4ceb0 con 0x555d8df1c370 2023-07-18T03:17:19.814 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.811+0000 7f27cffff700 10 client.27544 handle_command_reply: tid=0 2023-07-18T03:17:19.814 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.811+0000 7f280a84bb80 1 client.27544 shutdown 2023-07-18T03:17:19.814 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.811+0000 7f280a84bb80 1 -- 172.21.15.101:0/815128092 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555d8df012d0 msgr2=0x555d8df0c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:19.814 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.811+0000 7f280a84bb80 1 --2- 172.21.15.101:0/815128092 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555d8df012d0 0x555d8df0c450 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f27e400c010 tx=0x7f27e4005ad0 comp rx=0 tx=0).stop 2023-07-18T03:17:19.814 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.811+0000 7f280a84bb80 1 -- 172.21.15.101:0/815128092 shutdown_connections 2023-07-18T03:17:19.814 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.811+0000 7f280a84bb80 1 --2- 172.21.15.101:0/815128092 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x555d8df1c370 0x555d8df1e780 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:19.814 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.811+0000 7f280a84bb80 1 --2- 172.21.15.101:0/815128092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d8df0cad0 0x555d8df09d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:19.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.811+0000 7f27cffff700 0 client.27544 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:17:19.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.811+0000 7f280a84bb80 1 --2- 172.21.15.101:0/815128092 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555d8df012d0 0x555d8df0c450 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:19.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.811+0000 7f280a84bb80 1 --2- 172.21.15.101:0/815128092 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d8df01be0 0x555d8df096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:19.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.811+0000 7f280a84bb80 1 -- 172.21.15.101:0/815128092 >> 172.21.15.101:0/815128092 conn(0x555d8def7da0 msgr2=0x555d8defd230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:19.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.812+0000 7f280a84bb80 1 -- 172.21.15.101:0/815128092 shutdown_connections 2023-07-18T03:17:19.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.812+0000 7f280a84bb80 1 -- 172.21.15.101:0/815128092 wait complete. 2023-07-18T03:17:19.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.812+0000 7f280a84bb80 20 client.27544 trim_cache size 0 max 16384 2023-07-18T03:17:19.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.830+0000 7f280a84bb80 1 Processor -- start 2023-07-18T03:17:19.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.830+0000 7f280a84bb80 1 -- start start 2023-07-18T03:17:19.826 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.831+0000 7f280a84bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d8df012d0 0x555d8df0bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:19.826 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.831+0000 7f280a84bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d8df01be0 0x555d8df08eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:19.826 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.831+0000 7f280a84bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555d8df0cad0 0x555d8df093f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:19.826 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.831+0000 7f280a84bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d8df0acb0 con 0x555d8df012d0 2023-07-18T03:17:19.826 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.831+0000 7f280a84bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d8df07c50 con 0x555d8df01be0 2023-07-18T03:17:19.826 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.831+0000 7f280a84bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d8df016a0 con 0x555d8df0cad0 2023-07-18T03:17:19.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.831+0000 7f27f4d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d8df012d0 0x555d8df0bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:19.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.831+0000 7f27f559c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555d8df0cad0 0x555d8df093f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:19.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.831+0000 7f27f4d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d8df012d0 0x555d8df0bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:45832/0 (socket says 172.21.15.101:45832) 2023-07-18T03:17:19.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.831+0000 7f27f559c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555d8df0cad0 0x555d8df093f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:44046/0 (socket says 172.21.15.101:44046) 2023-07-18T03:17:19.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.831+0000 7f27f4d9b700 1 -- 172.21.15.101:0/3882341362 learned_addr learned my addr 172.21.15.101:0/3882341362 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:19.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.831+0000 7f27effff700 1 --2- 172.21.15.101:0/3882341362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d8df01be0 0x555d8df08eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:19.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.831+0000 7f27f4d9b700 1 -- 172.21.15.101:0/3882341362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555d8df0cad0 msgr2=0x555d8df093f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:19.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.831+0000 7f27f4d9b700 1 --2- 172.21.15.101:0/3882341362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555d8df0cad0 0x555d8df093f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:19.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.831+0000 7f27f4d9b700 1 -- 172.21.15.101:0/3882341362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d8df01be0 msgr2=0x555d8df08eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:19.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.831+0000 7f27f4d9b700 1 --2- 172.21.15.101:0/3882341362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d8df01be0 0x555d8df08eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:19.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.831+0000 7f27f4d9b700 1 -- 172.21.15.101:0/3882341362 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d8dca8250 con 0x555d8df012d0 2023-07-18T03:17:19.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.831+0000 7f27effff700 1 --2- 172.21.15.101:0/3882341362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d8df01be0 0x555d8df08eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:17:19.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.832+0000 7f27f4d9b700 1 --2- 172.21.15.101:0/3882341362 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d8df012d0 0x555d8df0bff0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f27e4001eb0 tx=0x7f27e40053a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da04b725ef18d31b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:19.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.832+0000 7f27db7fe700 1 -- 172.21.15.101:0/3882341362 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e401d050 con 0x555d8df012d0 2023-07-18T03:17:19.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.832+0000 7f27db7fe700 1 -- 172.21.15.101:0/3882341362 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27e401e040 con 0x555d8df012d0 2023-07-18T03:17:19.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.832+0000 7f27db7fe700 1 -- 172.21.15.101:0/3882341362 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e4023710 con 0x555d8df012d0 2023-07-18T03:17:19.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.833+0000 7f280a84bb80 1 -- 172.21.15.101:0/3882341362 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d8df012d0 msgr2=0x555d8df0bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:19.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.833+0000 7f280a84bb80 1 --2- 172.21.15.101:0/3882341362 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d8df012d0 0x555d8df0bff0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f27e4001eb0 tx=0x7f27e40053a0 comp rx=0 tx=0).stop 2023-07-18T03:17:19.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.833+0000 7f280a84bb80 1 -- 172.21.15.101:0/3882341362 shutdown_connections 2023-07-18T03:17:19.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.833+0000 7f280a84bb80 1 --2- 172.21.15.101:0/3882341362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555d8df0cad0 0x555d8df093f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:19.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.833+0000 7f280a84bb80 1 --2- 172.21.15.101:0/3882341362 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d8df012d0 0x555d8df0bff0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:19.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.833+0000 7f280a84bb80 1 --2- 172.21.15.101:0/3882341362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d8df01be0 0x555d8df08eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:19.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.833+0000 7f280a84bb80 1 -- 172.21.15.101:0/3882341362 >> 172.21.15.101:0/3882341362 conn(0x555d8def7da0 msgr2=0x555d8df17c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:19.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.833+0000 7f280a84bb80 1 -- 172.21.15.101:0/3882341362 shutdown_connections 2023-07-18T03:17:19.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.833+0000 7f280a84bb80 1 -- 172.21.15.101:0/3882341362 wait complete. 2023-07-18T03:17:19.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.833+0000 7f280a84bb80 1 -- start start 2023-07-18T03:17:19.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.833+0000 7f280a84bb80 1 Processor -- start 2023-07-18T03:17:19.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.834+0000 7f280a84bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555d8df012d0 0x555d8df13310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:19.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.834+0000 7f280a84bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d8df01be0 0x555d8df11670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:19.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.834+0000 7f280a84bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d8df0cad0 0x555d8df11bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:19.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.834+0000 7f280a84bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d8df120f0 con 0x555d8df01be0 2023-07-18T03:17:19.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.834+0000 7f280a84bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d8df12260 con 0x555d8df0cad0 2023-07-18T03:17:19.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.834+0000 7f280a84bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d8df123d0 con 0x555d8df012d0 2023-07-18T03:17:19.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.834+0000 7f280a84bb80 10 client.0 _ll_register_callbacks cb 0x7f27e400bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:17:19.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.834+0000 7f27effff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d8df01be0 0x555d8df11670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:19.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.834+0000 7f27f559c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d8df0cad0 0x555d8df11bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:19.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.834+0000 7f27f559c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d8df0cad0 0x555d8df11bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60458/0 (socket says 172.21.15.101:60458) 2023-07-18T03:17:19.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.834+0000 7f27f559c700 1 -- 172.21.15.101:0/4279225089 learned_addr learned my addr 172.21.15.101:0/4279225089 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:19.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.834+0000 7f27effff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d8df01be0 0x555d8df11670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:45846/0 (socket says 172.21.15.101:45846) 2023-07-18T03:17:19.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.834+0000 7f27f4d9b700 1 --2- 172.21.15.101:0/4279225089 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555d8df012d0 0x555d8df13310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:19.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.835+0000 7f27f559c700 1 -- 172.21.15.101:0/4279225089 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555d8df012d0 msgr2=0x555d8df13310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:19.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.835+0000 7f27f559c700 1 --2- 172.21.15.101:0/4279225089 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555d8df012d0 0x555d8df13310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:19.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.835+0000 7f27f559c700 1 -- 172.21.15.101:0/4279225089 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d8df01be0 msgr2=0x555d8df11670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:19.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.835+0000 7f27f559c700 1 --2- 172.21.15.101:0/4279225089 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d8df01be0 0x555d8df11670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:19.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.835+0000 7f27f559c700 1 -- 172.21.15.101:0/4279225089 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555d8dc81de0 con 0x555d8df0cad0 2023-07-18T03:17:19.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.835+0000 7f27f559c700 1 --2- 172.21.15.101:0/4279225089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d8df0cad0 0x555d8df11bb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f27e8019fd0 tx=0x7f27e8004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4ee77f2d452cc06 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:19.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.835+0000 7f27cffff700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:17:19.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.835+0000 7f27cffff700 1 -- 172.21.15.101:0/4279225089 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e800e040 con 0x555d8df0cad0 2023-07-18T03:17:19.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.835+0000 7f27cffff700 1 -- 172.21.15.101:0/4279225089 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f27e8003c00 con 0x555d8df0cad0 2023-07-18T03:17:19.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.835+0000 7f280a84bb80 1 -- 172.21.15.101:0/4279225089 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f27dc099ff0 con 0x555d8df0cad0 2023-07-18T03:17:19.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.836+0000 7f27cffff700 1 -- 172.21.15.101:0/4279225089 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27e8003dc0 con 0x555d8df0cad0 2023-07-18T03:17:19.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.836+0000 7f280a84bb80 10 client.27515 fetch_fsmap learned FSMap version 33 2023-07-18T03:17:19.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.836+0000 7f280a84bb80 1 -- 172.21.15.101:0/4279225089 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x555d8dc6ac60 con 0x555d8df0cad0 2023-07-18T03:17:19.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.836+0000 7f27cffff700 1 -- 172.21.15.101:0/4279225089 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27dc099ff0 con 0x555d8df0cad0 2023-07-18T03:17:19.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.836+0000 7f27cffff700 1 -- 172.21.15.101:0/4279225089 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f27e80285a0 con 0x555d8df0cad0 2023-07-18T03:17:19.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.836+0000 7f280a84bb80 10 client.27515 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:17:19.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.836+0000 7f280a84bb80 10 client.27515 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:17:19.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.836+0000 7f280a84bb80 20 client.27515 populate_metadata read hostname 'smithi101' 2023-07-18T03:17:19.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.836+0000 7f280a84bb80 1 --2- 172.21.15.101:0/4279225089 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x555d8df716f0 0x555d8df73b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:19.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.836+0000 7f280a84bb80 4 client.27515 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:17:19.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.836+0000 7f280a84bb80 1 -- 172.21.15.101:0/4279225089 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555d8de0f050 con 0x555d8df716f0 2023-07-18T03:17:19.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.837+0000 7f27f4d9b700 1 --2- 172.21.15.101:0/4279225089 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x555d8df716f0 0x555d8df73b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:19.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.837+0000 7f27f4d9b700 1 --2- 172.21.15.101:0/4279225089 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x555d8df716f0 0x555d8df73b00 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da968829b7344ad9 server_cookie=a553fd1944cdab00 in_seq=0 out_seq=0 2023-07-18T03:17:19.839 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.837+0000 7f27cffff700 10 client.27515 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:17:19.839 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.838+0000 7f27cffff700 1 -- 172.21.15.101:0/4279225089 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x555d8de0f050 con 0x555d8df716f0 2023-07-18T03:17:19.839 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.838+0000 7f27cffff700 10 client.27515 handle_command_reply: tid=0 2023-07-18T03:17:19.839 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.838+0000 7f280a84bb80 1 client.27515 shutdown 2023-07-18T03:17:19.839 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.838+0000 7f280a84bb80 1 -- 172.21.15.101:0/4279225089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d8df0cad0 msgr2=0x555d8df11bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:19.839 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.838+0000 7f280a84bb80 1 --2- 172.21.15.101:0/4279225089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d8df0cad0 0x555d8df11bb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f27e8019fd0 tx=0x7f27e8004b40 comp rx=0 tx=0).stop 2023-07-18T03:17:19.839 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.838+0000 7f280a84bb80 1 -- 172.21.15.101:0/4279225089 shutdown_connections 2023-07-18T03:17:19.840 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.838+0000 7f280a84bb80 1 --2- 172.21.15.101:0/4279225089 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x555d8df716f0 0x555d8df73b00 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:19.840 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.838+0000 7f280a84bb80 1 --2- 172.21.15.101:0/4279225089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d8df0cad0 0x555d8df11bb0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:19.840 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.838+0000 7f280a84bb80 1 --2- 172.21.15.101:0/4279225089 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555d8df012d0 0x555d8df13310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:19.840 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.838+0000 7f27cffff700 0 client.27515 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:17:19.840 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.838+0000 7f280a84bb80 1 --2- 172.21.15.101:0/4279225089 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d8df01be0 0x555d8df11670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:19.840 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.838+0000 7f280a84bb80 1 -- 172.21.15.101:0/4279225089 >> 172.21.15.101:0/4279225089 conn(0x555d8def7da0 msgr2=0x555d8df032e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:19.840 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.838+0000 7f280a84bb80 1 -- 172.21.15.101:0/4279225089 shutdown_connections 2023-07-18T03:17:19.841 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.838+0000 7f280a84bb80 1 -- 172.21.15.101:0/4279225089 wait complete. 2023-07-18T03:17:19.841 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.838+0000 7f280a84bb80 20 client.27515 trim_cache size 0 max 16384 2023-07-18T03:17:19.841 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:17:19.841 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "no active scrubs running", 2023-07-18T03:17:19.841 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": {} 2023-07-18T03:17:19.842 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:17:19.843 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.839+0000 7f27f6fff700 1 -- 172.21.15.101:0/2852109453 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f27e00a1340 msgr2=0x7f27e00a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:19.843 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.839+0000 7f27f6fff700 1 --2- 172.21.15.101:0/2852109453 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f27e00a1340 0x7f27e00a37f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f27f019c6f0 tx=0x7f27e800b040 comp rx=0 tx=0).stop 2023-07-18T03:17:19.843 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.839+0000 7f27f6fff700 1 -- 172.21.15.101:0/2852109453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27f01004b0 msgr2=0x7f27f019bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:19.843 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.839+0000 7f27f6fff700 1 --2- 172.21.15.101:0/2852109453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27f01004b0 0x7f27f019bf30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f27dc00e9a0 tx=0x7f27dc00ecb0 comp rx=0 tx=0).stop 2023-07-18T03:17:19.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.839+0000 7f27f6fff700 1 -- 172.21.15.101:0/2852109453 shutdown_connections 2023-07-18T03:17:19.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.839+0000 7f27f6fff700 1 --2- 172.21.15.101:0/2852109453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27f0105b10 0x7f27f0195a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:19.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.839+0000 7f27f6fff700 1 --2- 172.21.15.101:0/2852109453 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f27f00ffb00 0x7f27f019b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:19.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.839+0000 7f27f6fff700 1 --2- 172.21.15.101:0/2852109453 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f27e00a1340 0x7f27e00a37f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:19.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.839+0000 7f27f6fff700 1 --2- 172.21.15.101:0/2852109453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27f01004b0 0x7f27f019bf30 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:19.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.839+0000 7f27f6fff700 1 -- 172.21.15.101:0/2852109453 >> 172.21.15.101:0/2852109453 conn(0x7f27f00fb600 msgr2=0x7f27f00fbc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:19.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.839+0000 7f27f6fff700 1 -- 172.21.15.101:0/2852109453 shutdown_connections 2023-07-18T03:17:19.845 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:19.839+0000 7f27f6fff700 1 -- 172.21.15.101:0/2852109453 wait complete. 2023-07-18T03:17:19.852 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-18T03:17:19.853 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:17:20.076 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.078+0000 7fa91bfff700 1 -- 172.21.15.101:0/3917831098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9141004a0 msgr2=0x7fa91410c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:20.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.078+0000 7fa91bfff700 1 --2- 172.21.15.101:0/3917831098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9141004a0 0x7fa91410c930 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa90c00b0a0 tx=0x7fa90c00b3b0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.078+0000 7fa91bfff700 1 -- 172.21.15.101:0/3917831098 shutdown_connections 2023-07-18T03:17:20.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.078+0000 7fa91bfff700 1 --2- 172.21.15.101:0/3917831098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9141004a0 0x7fa91410c930 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.078+0000 7fa91bfff700 1 --2- 172.21.15.101:0/3917831098 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa914105b00 0x7fa914105ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.078+0000 7fa91bfff700 1 --2- 172.21.15.101:0/3917831098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9140ffaf0 0x7fa9140fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.078+0000 7fa91bfff700 1 -- 172.21.15.101:0/3917831098 >> 172.21.15.101:0/3917831098 conn(0x7fa9140fb630 msgr2=0x7fa9140fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:20.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.078+0000 7fa91bfff700 1 -- 172.21.15.101:0/3917831098 shutdown_connections 2023-07-18T03:17:20.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.078+0000 7fa91bfff700 1 -- 172.21.15.101:0/3917831098 wait complete. 2023-07-18T03:17:20.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.079+0000 7fa91bfff700 1 Processor -- start 2023-07-18T03:17:20.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.079+0000 7fa91bfff700 1 -- start start 2023-07-18T03:17:20.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.080+0000 7fa91bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9140ffaf0 0x7fa914197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:20.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.080+0000 7fa91bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa9141004a0 0x7fa914198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:20.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.080+0000 7fa91bfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa914105b00 0x7fa91419c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:20.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.080+0000 7fa91bfff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91419ced0 con 0x7fa9141004a0 2023-07-18T03:17:20.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.080+0000 7fa91bfff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91419d040 con 0x7fa914105b00 2023-07-18T03:17:20.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.080+0000 7fa91bfff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91419d200 con 0x7fa9140ffaf0 2023-07-18T03:17:20.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.080+0000 7fa91959a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa9141004a0 0x7fa914198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:20.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.080+0000 7fa91959a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa9141004a0 0x7fa914198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:45860/0 (socket says 172.21.15.101:45860) 2023-07-18T03:17:20.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.080+0000 7fa91959a700 1 -- 172.21.15.101:0/2146570879 learned_addr learned my addr 172.21.15.101:0/2146570879 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:20.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.080+0000 7fa91959a700 1 -- 172.21.15.101:0/2146570879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9140ffaf0 msgr2=0x7fa914197ef0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-18T03:17:20.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.080+0000 7fa91959a700 1 --2- 172.21.15.101:0/2146570879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9140ffaf0 0x7fa914197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.080+0000 7fa91959a700 1 -- 172.21.15.101:0/2146570879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa914105b00 msgr2=0x7fa91419c890 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-18T03:17:20.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.080+0000 7fa91959a700 1 --2- 172.21.15.101:0/2146570879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa914105b00 0x7fa91419c890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.080+0000 7fa91959a700 1 -- 172.21.15.101:0/2146570879 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa90c009d00 con 0x7fa9141004a0 2023-07-18T03:17:20.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.081+0000 7fa91959a700 1 --2- 172.21.15.101:0/2146570879 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa9141004a0 0x7fa914198430 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa90400e9a0 tx=0x7fa90400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:20.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.081+0000 7fa90affd700 1 -- 172.21.15.101:0/2146570879 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa904009790 con 0x7fa9141004a0 2023-07-18T03:17:20.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.081+0000 7fa91bfff700 1 -- 172.21.15.101:0/2146570879 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa91419d4e0 con 0x7fa9141004a0 2023-07-18T03:17:20.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.081+0000 7fa90affd700 1 -- 172.21.15.101:0/2146570879 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa90400cbd0 con 0x7fa9141004a0 2023-07-18T03:17:20.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.081+0000 7fa90affd700 1 -- 172.21.15.101:0/2146570879 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa904018680 con 0x7fa9141004a0 2023-07-18T03:17:20.081 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.081+0000 7fa91bfff700 1 -- 172.21.15.101:0/2146570879 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9141a4d20 con 0x7fa9141004a0 2023-07-18T03:17:20.081 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.082+0000 7fa91bfff700 1 -- 172.21.15.101:0/2146570879 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa914067300 con 0x7fa9141004a0 2023-07-18T03:17:20.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.084+0000 7fa90affd700 1 -- 172.21.15.101:0/2146570879 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fa90400cd40 con 0x7fa9141004a0 2023-07-18T03:17:20.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.085+0000 7fa90affd700 1 --2- 172.21.15.101:0/2146570879 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa9000a1300 0x7fa9000a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:20.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.085+0000 7fa90affd700 1 -- 172.21.15.101:0/2146570879 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fa904014070 con 0x7fa9141004a0 2023-07-18T03:17:20.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.086+0000 7fa919d9b700 1 --2- 172.21.15.101:0/2146570879 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa9000a1300 0x7fa9000a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:20.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.086+0000 7fa919d9b700 1 --2- 172.21.15.101:0/2146570879 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa9000a1300 0x7fa9000a37b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa910009b60 tx=0x7fa910009620 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:20.085 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.090+0000 7fa90affd700 1 -- 172.21.15.101:0/2146570879 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa904051790 con 0x7fa9141004a0 2023-07-18T03:17:20.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.371+0000 7fa91bfff700 1 -- 172.21.15.101:0/2146570879 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa914199240 con 0x7fa9141004a0 2023-07-18T03:17:20.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.372+0000 7fa90affd700 1 -- 172.21.15.101:0/2146570879 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+10858 (secure 0 0 0) 0x7fa904099f40 con 0x7fa9141004a0 2023-07-18T03:17:20.367 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:17:20.367 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/4192846769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":4192846769},{"type":"v1","addr":"172.21.15.101:6839","nonce":4192846769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6841/3824340175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6840","nonce":3824340175},{"type":"v1","addr":"172.21.15.101:6841","nonce":3824340175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:04:59.361185+0000","modified":"2023-07-18T03:16:36.002358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.155:6839/2055462483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2055462483},{"type":"v1","addr":"172.21.15.155:6839","nonce":2055462483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:17:20.369 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.374+0000 7fa91bfff700 1 -- 172.21.15.101:0/2146570879 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa9000a1300 msgr2=0x7fa9000a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:20.369 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.374+0000 7fa91bfff700 1 --2- 172.21.15.101:0/2146570879 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa9000a1300 0x7fa9000a37b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa910009b60 tx=0x7fa910009620 comp rx=0 tx=0).stop 2023-07-18T03:17:20.369 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.374+0000 7fa91bfff700 1 -- 172.21.15.101:0/2146570879 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa9141004a0 msgr2=0x7fa914198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:20.370 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.374+0000 7fa91bfff700 1 --2- 172.21.15.101:0/2146570879 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa9141004a0 0x7fa914198430 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa90400e9a0 tx=0x7fa90400ed60 comp rx=0 tx=0).stop 2023-07-18T03:17:20.370 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.374+0000 7fa91bfff700 1 -- 172.21.15.101:0/2146570879 shutdown_connections 2023-07-18T03:17:20.370 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.374+0000 7fa91bfff700 1 --2- 172.21.15.101:0/2146570879 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa9000a1300 0x7fa9000a37b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.370 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.374+0000 7fa91bfff700 1 --2- 172.21.15.101:0/2146570879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa914105b00 0x7fa91419c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.370 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.374+0000 7fa91bfff700 1 --2- 172.21.15.101:0/2146570879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9140ffaf0 0x7fa914197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.370 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.374+0000 7fa91bfff700 1 --2- 172.21.15.101:0/2146570879 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa9141004a0 0x7fa914198430 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.370 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.374+0000 7fa91bfff700 1 -- 172.21.15.101:0/2146570879 >> 172.21.15.101:0/2146570879 conn(0x7fa9140fb630 msgr2=0x7fa91410aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:20.370 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.374+0000 7fa91bfff700 1 -- 172.21.15.101:0/2146570879 shutdown_connections 2023-07-18T03:17:20.371 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.374+0000 7fa91bfff700 1 -- 172.21.15.101:0/2146570879 wait complete. 2023-07-18T03:17:20.371 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 33 2023-07-18T03:17:20.389 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-18T03:17:20.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.615+0000 7f82d51ff700 1 -- 172.21.15.101:0/530666097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82d0101f70 msgr2=0x7f82d0102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:20.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.615+0000 7f82d51ff700 1 --2- 172.21.15.101:0/530666097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82d0101f70 0x7f82d0102340 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f82c00099c0 tx=0x7f82c0009cd0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.615+0000 7f82d51ff700 1 -- 172.21.15.101:0/530666097 shutdown_connections 2023-07-18T03:17:20.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.615+0000 7f82d51ff700 1 --2- 172.21.15.101:0/530666097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82d0106fb0 0x7f82d010eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.615+0000 7f82d51ff700 1 --2- 172.21.15.101:0/530666097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82d0101f70 0x7f82d0102340 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.615+0000 7f82d51ff700 1 --2- 172.21.15.101:0/530666097 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f82d0102910 0x7f82d01068d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.615+0000 7f82d51ff700 1 -- 172.21.15.101:0/530666097 >> 172.21.15.101:0/530666097 conn(0x7f82d00fd7e0 msgr2=0x7f82d00ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:20.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.615+0000 7f82d51ff700 1 -- 172.21.15.101:0/530666097 shutdown_connections 2023-07-18T03:17:20.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.615+0000 7f82d51ff700 1 -- 172.21.15.101:0/530666097 wait complete. 2023-07-18T03:17:20.612 INFO:tasks.workunit.client.0.smithi101.stdout: 1 625351 5.96 MB/sec execute 405 sec latency 14.489 ms 2023-07-18T03:17:20.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.616+0000 7f82d51ff700 1 Processor -- start 2023-07-18T03:17:20.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.616+0000 7f82d51ff700 1 -- start start 2023-07-18T03:17:20.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.617+0000 7f82d51ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82d0101f70 0x7f82d0197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:20.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.617+0000 7f82d51ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f82d0102910 0x7f82d0198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:20.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.617+0000 7f82d51ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82d0106fb0 0x7f82d019c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:20.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.617+0000 7f82d51ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d019ce10 con 0x7f82d0102910 2023-07-18T03:17:20.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.617+0000 7f82d51ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d019cf80 con 0x7f82d0101f70 2023-07-18T03:17:20.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.617+0000 7f82d51ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d019d140 con 0x7f82d0106fb0 2023-07-18T03:17:20.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.617+0000 7f82ced9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82d0101f70 0x7f82d0197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:20.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.617+0000 7f82ced9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82d0101f70 0x7f82d0197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60494/0 (socket says 172.21.15.101:60494) 2023-07-18T03:17:20.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.617+0000 7f82ce59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f82d0102910 0x7f82d0198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:20.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.617+0000 7f82ced9d700 1 -- 172.21.15.101:0/1482741070 learned_addr learned my addr 172.21.15.101:0/1482741070 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:20.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.617+0000 7f82cf59e700 1 --2- 172.21.15.101:0/1482741070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82d0106fb0 0x7f82d019c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:20.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.617+0000 7f82ced9d700 1 -- 172.21.15.101:0/1482741070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82d0106fb0 msgr2=0x7f82d019c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:20.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.617+0000 7f82ced9d700 1 --2- 172.21.15.101:0/1482741070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82d0106fb0 0x7f82d019c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.617+0000 7f82ced9d700 1 -- 172.21.15.101:0/1482741070 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f82d0102910 msgr2=0x7f82d0198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:20.616 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.617+0000 7f82ced9d700 1 --2- 172.21.15.101:0/1482741070 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f82d0102910 0x7f82d0198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.616 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.617+0000 7f82ced9d700 1 -- 172.21.15.101:0/1482741070 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c4009520 con 0x7f82d0101f70 2023-07-18T03:17:20.616 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.618+0000 7f82ced9d700 1 --2- 172.21.15.101:0/1482741070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82d0101f70 0x7f82d0197e30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f82c00098e0 tx=0x7f82c000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:20.616 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.618+0000 7f82b7fff700 1 -- 172.21.15.101:0/1482741070 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c001d070 con 0x7f82d0101f70 2023-07-18T03:17:20.616 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.618+0000 7f82d51ff700 1 -- 172.21.15.101:0/1482741070 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82c00095f0 con 0x7f82d0101f70 2023-07-18T03:17:20.616 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.618+0000 7f82b7fff700 1 -- 172.21.15.101:0/1482741070 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82c0022470 con 0x7f82d0101f70 2023-07-18T03:17:20.616 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.618+0000 7f82b7fff700 1 -- 172.21.15.101:0/1482741070 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c0017650 con 0x7f82d0101f70 2023-07-18T03:17:20.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.618+0000 7f82d51ff700 1 -- 172.21.15.101:0/1482741070 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82d01a4a80 con 0x7f82d0101f70 2023-07-18T03:17:20.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.621+0000 7f82b7fff700 1 -- 172.21.15.101:0/1482741070 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f82c0022850 con 0x7f82d0101f70 2023-07-18T03:17:20.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.623+0000 7f82e8aa2b80 1 Processor -- start 2023-07-18T03:17:20.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.623+0000 7f82e8aa2b80 1 -- start start 2023-07-18T03:17:20.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.624+0000 7f82e8aa2b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562406cc62d0 0x562406cc66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:20.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.624+0000 7f82e8aa2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562406cc6be0 0x562406cd1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:20.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.624+0000 7f82b7fff700 1 --2- 172.21.15.101:0/1482741070 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f82b80a12c0 0x7f82b80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:20.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.624+0000 7f82e8aa2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562406cd1ad0 0x562406cd3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:20.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.624+0000 7f82e8aa2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562406bcccb0 con 0x562406cd1ad0 2023-07-18T03:17:20.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.624+0000 7f82e8aa2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562406c1fa40 con 0x562406cc6be0 2023-07-18T03:17:20.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.624+0000 7f82e8aa2b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562406c18ed0 con 0x562406cc62d0 2023-07-18T03:17:20.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.624+0000 7f82b7fff700 1 -- 172.21.15.101:0/1482741070 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f82c00cdb70 con 0x7f82d0101f70 2023-07-18T03:17:20.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.624+0000 7f82cf59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562406cc62d0 0x562406cc66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:20.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.624+0000 7f82cf59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562406cc62d0 0x562406cc66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:44124/0 (socket says 172.21.15.101:44124) 2023-07-18T03:17:20.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.624+0000 7f82cf59e700 1 -- 172.21.15.101:0/1702722063 learned_addr learned my addr 172.21.15.101:0/1702722063 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:20.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.624+0000 7f82ced9d700 1 --2- 172.21.15.101:0/1482741070 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f82b80a12c0 0x7f82b80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:20.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.625+0000 7f82cf59e700 1 --2- 172.21.15.101:0/1702722063 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562406cd1ad0 0x562406cd3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:20.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.625+0000 7f82ce59c700 1 --2- 172.21.15.101:0/1702722063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562406cc6be0 0x562406cd1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:20.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.625+0000 7f82cf59e700 1 -- 172.21.15.101:0/1702722063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562406cc6be0 msgr2=0x562406cd1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:20.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.625+0000 7f82cf59e700 1 --2- 172.21.15.101:0/1702722063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562406cc6be0 0x562406cd1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.625+0000 7f82cf59e700 1 -- 172.21.15.101:0/1702722063 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562406cd1ad0 msgr2=0x562406cd3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:20.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.625+0000 7f82cf59e700 1 --2- 172.21.15.101:0/1702722063 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562406cd1ad0 0x562406cd3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.625+0000 7f82cf59e700 1 -- 172.21.15.101:0/1702722063 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562406a46de0 con 0x562406cc62d0 2023-07-18T03:17:20.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.625+0000 7f82ce59c700 1 --2- 172.21.15.101:0/1702722063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562406cc6be0 0x562406cd1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:17:20.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.625+0000 7f82cf59e700 1 --2- 172.21.15.101:0/1702722063 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562406cc62d0 0x562406cc66a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f82c8008920 tx=0x7f82c8009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55dd0fe07107d711 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:20.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.625+0000 7f82ced9d700 1 --2- 172.21.15.101:0/1482741070 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f82b80a12c0 0x7f82b80a3770 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f82c0097440 tx=0x7f82c0022b00 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:20.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.625+0000 7f82b57fa700 1 -- 172.21.15.101:0/1702722063 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c800e070 con 0x562406cc62d0 2023-07-18T03:17:20.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.625+0000 7f82b57fa700 1 -- 172.21.15.101:0/1702722063 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82c800f040 con 0x562406cc62d0 2023-07-18T03:17:20.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.625+0000 7f82b57fa700 1 -- 172.21.15.101:0/1702722063 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c8004020 con 0x562406cc62d0 2023-07-18T03:17:20.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.626+0000 7f82e8aa2b80 1 -- 172.21.15.101:0/1702722063 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562406cc62d0 msgr2=0x562406cc66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:20.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.626+0000 7f82e8aa2b80 1 --2- 172.21.15.101:0/1702722063 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562406cc62d0 0x562406cc66a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f82c8008920 tx=0x7f82c8009920 comp rx=0 tx=0).stop 2023-07-18T03:17:20.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.626+0000 7f82e8aa2b80 1 -- 172.21.15.101:0/1702722063 shutdown_connections 2023-07-18T03:17:20.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.626+0000 7f82e8aa2b80 1 --2- 172.21.15.101:0/1702722063 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562406cd1ad0 0x562406cd3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.626+0000 7f82e8aa2b80 1 --2- 172.21.15.101:0/1702722063 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562406cc62d0 0x562406cc66a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.626+0000 7f82e8aa2b80 1 --2- 172.21.15.101:0/1702722063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562406cc6be0 0x562406cd1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.626+0000 7f82e8aa2b80 1 -- 172.21.15.101:0/1702722063 >> 172.21.15.101:0/1702722063 conn(0x562406cbcda0 msgr2=0x562406cc26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:20.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.626+0000 7f82e8aa2b80 1 -- 172.21.15.101:0/1702722063 shutdown_connections 2023-07-18T03:17:20.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.626+0000 7f82e8aa2b80 1 -- 172.21.15.101:0/1702722063 wait complete. 2023-07-18T03:17:20.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.627+0000 7f82e8aa2b80 1 -- start start 2023-07-18T03:17:20.631 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.627+0000 7f82e8aa2b80 1 Processor -- start 2023-07-18T03:17:20.631 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.628+0000 7f82e8aa2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562406cc62d0 0x562406cd1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:20.631 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.628+0000 7f82e8aa2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562406cc6be0 0x562406cce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:20.631 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.628+0000 7f82e8aa2b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562406cd1ad0 0x562406cceec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:20.631 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.628+0000 7f82e8aa2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562406bb8ac0 con 0x562406cc6be0 2023-07-18T03:17:20.631 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.628+0000 7f82e8aa2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562406bb87c0 con 0x562406cc62d0 2023-07-18T03:17:20.632 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.628+0000 7f82e8aa2b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562406a9c920 con 0x562406cd1ad0 2023-07-18T03:17:20.632 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.628+0000 7f82e8aa2b80 10 client.0 _ll_register_callbacks cb 0x562406bb6760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:17:20.632 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.628+0000 7f82cf59e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562406cc62d0 0x562406cd1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:20.632 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.628+0000 7f82cf59e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562406cc62d0 0x562406cd1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60516/0 (socket says 172.21.15.101:60516) 2023-07-18T03:17:20.632 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.628+0000 7f82cf59e700 1 -- 172.21.15.101:0/1621115436 learned_addr learned my addr 172.21.15.101:0/1621115436 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:20.632 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.628+0000 7f82ce59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562406cc6be0 0x562406cce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:20.632 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.628+0000 7f82cf59e700 1 --2- 172.21.15.101:0/1621115436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562406cd1ad0 0x562406cceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:20.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.628+0000 7f82cf59e700 1 -- 172.21.15.101:0/1621115436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562406cd1ad0 msgr2=0x562406cceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:20.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.628+0000 7f82cf59e700 1 --2- 172.21.15.101:0/1621115436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562406cd1ad0 0x562406cceec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.628+0000 7f82cf59e700 1 -- 172.21.15.101:0/1621115436 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562406cc6be0 msgr2=0x562406cce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:20.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.628+0000 7f82cf59e700 1 --2- 172.21.15.101:0/1621115436 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562406cc6be0 0x562406cce7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.628+0000 7f82cf59e700 1 -- 172.21.15.101:0/1621115436 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c8008530 con 0x562406cc62d0 2023-07-18T03:17:20.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.629+0000 7f82cf59e700 1 --2- 172.21.15.101:0/1621115436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562406cc62d0 0x562406cd1550 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f82c80189e0 tx=0x7f82c8018b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5594b2e9548a9261 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:20.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.629+0000 7f82b2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:17:20.634 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.629+0000 7f82b2ff5700 1 -- 172.21.15.101:0/1621115436 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c800e030 con 0x562406cc62d0 2023-07-18T03:17:20.634 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.629+0000 7f82e8aa2b80 1 -- 172.21.15.101:0/1621115436 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562406b724f0 con 0x562406cc62d0 2023-07-18T03:17:20.634 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.629+0000 7f82b2ff5700 1 -- 172.21.15.101:0/1621115436 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f82c8003a00 con 0x562406cc62d0 2023-07-18T03:17:20.634 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.629+0000 7f82b2ff5700 1 -- 172.21.15.101:0/1621115436 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82c8003bc0 con 0x562406cc62d0 2023-07-18T03:17:20.634 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.629+0000 7f82e8aa2b80 10 client.27530 fetch_fsmap learned FSMap version 33 2023-07-18T03:17:20.634 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.629+0000 7f82e8aa2b80 1 -- 172.21.15.101:0/1621115436 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x562406a2fc60 con 0x562406cc62d0 2023-07-18T03:17:20.634 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.629+0000 7f82b2ff5700 1 -- 172.21.15.101:0/1621115436 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562406b724f0 con 0x562406cc62d0 2023-07-18T03:17:20.634 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.629+0000 7f82b2ff5700 1 -- 172.21.15.101:0/1621115436 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f82c8029450 con 0x562406cc62d0 2023-07-18T03:17:20.635 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.630+0000 7f82e8aa2b80 10 client.27530 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:17:20.635 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.630+0000 7f82e8aa2b80 10 client.27530 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:17:20.635 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.630+0000 7f82e8aa2b80 20 client.27530 populate_metadata read hostname 'smithi101' 2023-07-18T03:17:20.635 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.630+0000 7f82e8aa2b80 1 --2- 172.21.15.101:0/1621115436 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x562406ce1410 0x562406ce3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:20.635 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.630+0000 7f82e8aa2b80 4 client.27530 mds_command: new command op to 24541 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:17:20.635 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.630+0000 7f82e8aa2b80 1 -- 172.21.15.101:0/1621115436 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562406c11eb0 con 0x562406ce1410 2023-07-18T03:17:20.635 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.630+0000 7f82ce59c700 1 --2- 172.21.15.101:0/1621115436 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x562406ce1410 0x562406ce3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:20.635 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.630+0000 7f82ce59c700 1 --2- 172.21.15.101:0/1621115436 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x562406ce1410 0x562406ce3820 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4d44985f6fce3ff6 server_cookie=9bc7f43dd9cabf42 in_seq=0 out_seq=0 2023-07-18T03:17:20.636 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.630+0000 7f82b2ff5700 10 client.27530 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:17:20.636 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.631+0000 7f82b2ff5700 1 -- 172.21.15.101:0/1621115436 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562406c11eb0 con 0x562406ce1410 2023-07-18T03:17:20.636 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.631+0000 7f82b2ff5700 10 client.27530 handle_command_reply: tid=0 2023-07-18T03:17:20.636 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.631+0000 7f82e8aa2b80 1 client.27530 shutdown 2023-07-18T03:17:20.636 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.631+0000 7f82e8aa2b80 1 -- 172.21.15.101:0/1621115436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562406cc62d0 msgr2=0x562406cd1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:20.636 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.631+0000 7f82e8aa2b80 1 --2- 172.21.15.101:0/1621115436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562406cc62d0 0x562406cd1550 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f82c80189e0 tx=0x7f82c8018b40 comp rx=0 tx=0).stop 2023-07-18T03:17:20.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.631+0000 7f82e8aa2b80 1 -- 172.21.15.101:0/1621115436 shutdown_connections 2023-07-18T03:17:20.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.631+0000 7f82e8aa2b80 1 --2- 172.21.15.101:0/1621115436 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x562406ce1410 0x562406ce3820 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.631+0000 7f82e8aa2b80 1 --2- 172.21.15.101:0/1621115436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562406cc62d0 0x562406cd1550 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.631+0000 7f82e8aa2b80 1 --2- 172.21.15.101:0/1621115436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562406cd1ad0 0x562406cceec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.631+0000 7f82b2ff5700 0 client.27530 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:17:20.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.631+0000 7f82e8aa2b80 1 --2- 172.21.15.101:0/1621115436 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562406cc6be0 0x562406cce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.631+0000 7f82e8aa2b80 1 -- 172.21.15.101:0/1621115436 >> 172.21.15.101:0/1621115436 conn(0x562406cbcda0 msgr2=0x562406cc2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:20.638 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.632+0000 7f82e8aa2b80 1 -- 172.21.15.101:0/1621115436 shutdown_connections 2023-07-18T03:17:20.638 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.632+0000 7f82e8aa2b80 1 -- 172.21.15.101:0/1621115436 wait complete. 2023-07-18T03:17:20.638 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.632+0000 7f82e8aa2b80 20 client.27530 trim_cache size 0 max 16384 2023-07-18T03:17:20.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.651+0000 7f82e8aa2b80 1 Processor -- start 2023-07-18T03:17:20.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.651+0000 7f82e8aa2b80 1 -- start start 2023-07-18T03:17:20.647 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.651+0000 7f82e8aa2b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562406cc62d0 0x562406cd10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:20.647 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.651+0000 7f82e8aa2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562406cc6be0 0x562406cc9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:20.647 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.651+0000 7f82e8aa2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562406cd1ad0 0x562406cca070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:20.647 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.651+0000 7f82e8aa2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562406cc7540 con 0x562406cc6be0 2023-07-18T03:17:20.648 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.651+0000 7f82e8aa2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562406ccf8e0 con 0x562406cd1ad0 2023-07-18T03:17:20.648 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.651+0000 7f82e8aa2b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562406cc3ce0 con 0x562406cc62d0 2023-07-18T03:17:20.648 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.652+0000 7f82cf59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562406cc62d0 0x562406cd10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:20.648 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.652+0000 7f82cf59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562406cc62d0 0x562406cd10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:44138/0 (socket says 172.21.15.101:44138) 2023-07-18T03:17:20.648 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.652+0000 7f82cf59e700 1 -- 172.21.15.101:0/1719038837 learned_addr learned my addr 172.21.15.101:0/1719038837 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:20.648 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.652+0000 7f82ce59c700 1 --2- 172.21.15.101:0/1719038837 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562406cc6be0 0x562406cc9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:20.648 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.652+0000 7f82cf59e700 1 --2- 172.21.15.101:0/1719038837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562406cd1ad0 0x562406cca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:20.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.652+0000 7f82cf59e700 1 -- 172.21.15.101:0/1719038837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562406cd1ad0 msgr2=0x562406cca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:20.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.652+0000 7f82cf59e700 1 --2- 172.21.15.101:0/1719038837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562406cd1ad0 0x562406cca070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.652+0000 7f82cf59e700 1 -- 172.21.15.101:0/1719038837 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562406cc6be0 msgr2=0x562406cc9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:20.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.652+0000 7f82cf59e700 1 --2- 172.21.15.101:0/1719038837 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562406cc6be0 0x562406cc9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.652+0000 7f82cf59e700 1 -- 172.21.15.101:0/1719038837 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562406a6d250 con 0x562406cc62d0 2023-07-18T03:17:20.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.652+0000 7f82ce59c700 1 --2- 172.21.15.101:0/1719038837 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562406cc6be0 0x562406cc9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:17:20.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.652+0000 7f82cf59e700 1 --2- 172.21.15.101:0/1719038837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562406cc62d0 0x562406cd10f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f82c8008920 tx=0x7f82c80088b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7e1668cfea49590 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:20.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.652+0000 7f82b5ffb700 1 -- 172.21.15.101:0/1719038837 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c800e070 con 0x562406cc62d0 2023-07-18T03:17:20.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.653+0000 7f82b5ffb700 1 -- 172.21.15.101:0/1719038837 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82c8009230 con 0x562406cc62d0 2023-07-18T03:17:20.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.653+0000 7f82b5ffb700 1 -- 172.21.15.101:0/1719038837 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c8029670 con 0x562406cc62d0 2023-07-18T03:17:20.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.653+0000 7f82e8aa2b80 1 -- 172.21.15.101:0/1719038837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562406cc62d0 msgr2=0x562406cd10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:20.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.653+0000 7f82e8aa2b80 1 --2- 172.21.15.101:0/1719038837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562406cc62d0 0x562406cd10f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f82c8008920 tx=0x7f82c80088b0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.653+0000 7f82e8aa2b80 1 -- 172.21.15.101:0/1719038837 shutdown_connections 2023-07-18T03:17:20.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.653+0000 7f82e8aa2b80 1 --2- 172.21.15.101:0/1719038837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562406cd1ad0 0x562406cca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.653+0000 7f82e8aa2b80 1 --2- 172.21.15.101:0/1719038837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562406cc62d0 0x562406cd10f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.653+0000 7f82e8aa2b80 1 --2- 172.21.15.101:0/1719038837 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562406cc6be0 0x562406cc9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.653+0000 7f82e8aa2b80 1 -- 172.21.15.101:0/1719038837 >> 172.21.15.101:0/1719038837 conn(0x562406cbcda0 msgr2=0x562406cdd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:20.654 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.653+0000 7f82e8aa2b80 1 -- 172.21.15.101:0/1719038837 shutdown_connections 2023-07-18T03:17:20.654 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.654+0000 7f82e8aa2b80 1 -- 172.21.15.101:0/1719038837 wait complete. 2023-07-18T03:17:20.654 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.654+0000 7f82e8aa2b80 1 -- start start 2023-07-18T03:17:20.654 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.654+0000 7f82e8aa2b80 1 Processor -- start 2023-07-18T03:17:20.654 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.655+0000 7f82e8aa2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562406cc62d0 0x562406cd7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:20.654 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.655+0000 7f82e8aa2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562406cc6be0 0x562406cd7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:20.654 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.655+0000 7f82e8aa2b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562406cd1ad0 0x562406d34290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:20.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.655+0000 7f82e8aa2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562406cd8310 con 0x562406cc62d0 2023-07-18T03:17:20.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.655+0000 7f82e8aa2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562406cd8480 con 0x562406cc6be0 2023-07-18T03:17:20.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.655+0000 7f82e8aa2b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562406d347d0 con 0x562406cd1ad0 2023-07-18T03:17:20.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.655+0000 7f82e8aa2b80 10 client.0 _ll_register_callbacks cb 0x7f82c8018670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:17:20.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.655+0000 7f82cf59e700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562406cc62d0 0x562406cd7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:20.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.655+0000 7f82cf59e700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562406cc62d0 0x562406cd7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:45924/0 (socket says 172.21.15.101:45924) 2023-07-18T03:17:20.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.655+0000 7f82cf59e700 1 -- 172.21.15.101:0/1066681569 learned_addr learned my addr 172.21.15.101:0/1066681569 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:20.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.655+0000 7f82ce59c700 1 --2- 172.21.15.101:0/1066681569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562406cc6be0 0x562406cd7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:20.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.655+0000 7f82cf59e700 1 --2- 172.21.15.101:0/1066681569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562406cd1ad0 0x562406d34290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:20.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.655+0000 7f82cf59e700 1 -- 172.21.15.101:0/1066681569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562406cd1ad0 msgr2=0x562406d34290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:20.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.655+0000 7f82cf59e700 1 --2- 172.21.15.101:0/1066681569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562406cd1ad0 0x562406d34290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.655+0000 7f82cf59e700 1 -- 172.21.15.101:0/1066681569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562406cc6be0 msgr2=0x562406cd7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:20.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.655+0000 7f82cf59e700 1 --2- 172.21.15.101:0/1066681569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562406cc6be0 0x562406cd7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.657 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.655+0000 7f82cf59e700 1 -- 172.21.15.101:0/1066681569 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562406a2fc60 con 0x562406cc62d0 2023-07-18T03:17:20.657 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.656+0000 7f82cf59e700 1 --2- 172.21.15.101:0/1066681569 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562406cc62d0 0x562406cd7660 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f82c8009550 tx=0x7f82c8018740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=518c293a4ba0004 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:20.657 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.656+0000 7f82b2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:17:20.657 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.656+0000 7f82b2ff5700 1 -- 172.21.15.101:0/1066681569 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c800e040 con 0x562406cc62d0 2023-07-18T03:17:20.658 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.656+0000 7f82b2ff5700 1 -- 172.21.15.101:0/1066681569 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f82c8029450 con 0x562406cc62d0 2023-07-18T03:17:20.658 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.656+0000 7f82e8aa2b80 1 -- 172.21.15.101:0/1066681569 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f82c800f040 con 0x562406cc62d0 2023-07-18T03:17:20.658 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.656+0000 7f82b2ff5700 1 -- 172.21.15.101:0/1066681569 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82c8029610 con 0x562406cc62d0 2023-07-18T03:17:20.658 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.656+0000 7f82e8aa2b80 10 client.17499 fetch_fsmap learned FSMap version 33 2023-07-18T03:17:20.659 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.656+0000 7f82e8aa2b80 1 -- 172.21.15.101:0/1066681569 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x562406a452b0 con 0x562406cc62d0 2023-07-18T03:17:20.659 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.656+0000 7f82b2ff5700 1 -- 172.21.15.101:0/1066681569 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82c800f040 con 0x562406cc62d0 2023-07-18T03:17:20.659 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.656+0000 7f82b2ff5700 1 -- 172.21.15.101:0/1066681569 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f82c80196a0 con 0x562406cc62d0 2023-07-18T03:17:20.659 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.657+0000 7f82e8aa2b80 10 client.17499 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:17:20.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.657+0000 7f82e8aa2b80 10 client.17499 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:17:20.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.657+0000 7f82e8aa2b80 20 client.17499 populate_metadata read hostname 'smithi101' 2023-07-18T03:17:20.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.657+0000 7f82e8aa2b80 1 --2- 172.21.15.101:0/1066681569 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x562406d35a40 0x562406d37e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:20.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.657+0000 7f82e8aa2b80 4 client.17499 mds_command: new command op to 24541 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:17:20.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.657+0000 7f82e8aa2b80 1 -- 172.21.15.101:0/1066681569 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562406bd4050 con 0x562406d35a40 2023-07-18T03:17:20.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.657+0000 7f82ce59c700 1 --2- 172.21.15.101:0/1066681569 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x562406d35a40 0x562406d37e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:20.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.657+0000 7f82ce59c700 1 --2- 172.21.15.101:0/1066681569 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x562406d35a40 0x562406d37e20 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1816775ef59478d2 server_cookie=45220ef1697ef22c in_seq=0 out_seq=0 2023-07-18T03:17:20.662 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.657+0000 7f82b2ff5700 10 client.17499 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:17:20.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.659+0000 7f82b2ff5700 1 -- 172.21.15.101:0/1066681569 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562406bd4050 con 0x562406d35a40 2023-07-18T03:17:20.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.659+0000 7f82b2ff5700 10 client.17499 handle_command_reply: tid=0 2023-07-18T03:17:20.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.659+0000 7f82e8aa2b80 1 client.17499 shutdown 2023-07-18T03:17:20.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.659+0000 7f82e8aa2b80 1 -- 172.21.15.101:0/1066681569 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562406cc62d0 msgr2=0x562406cd7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:20.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.659+0000 7f82e8aa2b80 1 --2- 172.21.15.101:0/1066681569 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562406cc62d0 0x562406cd7660 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f82c8009550 tx=0x7f82c8018740 comp rx=0 tx=0).stop 2023-07-18T03:17:20.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.659+0000 7f82e8aa2b80 1 -- 172.21.15.101:0/1066681569 shutdown_connections 2023-07-18T03:17:20.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.659+0000 7f82e8aa2b80 1 --2- 172.21.15.101:0/1066681569 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x562406d35a40 0x562406d37e20 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.659+0000 7f82e8aa2b80 1 --2- 172.21.15.101:0/1066681569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562406cd1ad0 0x562406d34290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.659+0000 7f82e8aa2b80 1 --2- 172.21.15.101:0/1066681569 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562406cc62d0 0x562406cd7660 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.659+0000 7f82e8aa2b80 1 --2- 172.21.15.101:0/1066681569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562406cc6be0 0x562406cd7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.659+0000 7f82b2ff5700 0 client.17499 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:17:20.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.659+0000 7f82e8aa2b80 1 -- 172.21.15.101:0/1066681569 >> 172.21.15.101:0/1066681569 conn(0x562406cbcda0 msgr2=0x562406cc9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:20.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.659+0000 7f82e8aa2b80 1 -- 172.21.15.101:0/1066681569 shutdown_connections 2023-07-18T03:17:20.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.659+0000 7f82e8aa2b80 1 -- 172.21.15.101:0/1066681569 wait complete. 2023-07-18T03:17:20.668 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.660+0000 7f82e8aa2b80 20 client.17499 trim_cache size 0 max 16384 2023-07-18T03:17:20.668 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:17:20.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.660+0000 7f82d51ff700 1 -- 172.21.15.101:0/1482741070 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f82b80a12c0 msgr2=0x7f82b80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:20.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.660+0000 7f82d51ff700 1 --2- 172.21.15.101:0/1482741070 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f82b80a12c0 0x7f82b80a3770 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f82c0097440 tx=0x7f82c0022b00 comp rx=0 tx=0).stop 2023-07-18T03:17:20.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.660+0000 7f82d51ff700 1 -- 172.21.15.101:0/1482741070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82d0101f70 msgr2=0x7f82d0197e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:20.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.660+0000 7f82d51ff700 1 --2- 172.21.15.101:0/1482741070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82d0101f70 0x7f82d0197e30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f82c00098e0 tx=0x7f82c000f690 comp rx=0 tx=0).stop 2023-07-18T03:17:20.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.660+0000 7f82d51ff700 1 -- 172.21.15.101:0/1482741070 shutdown_connections 2023-07-18T03:17:20.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.660+0000 7f82d51ff700 1 --2- 172.21.15.101:0/1482741070 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f82b80a12c0 0x7f82b80a3770 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.660+0000 7f82d51ff700 1 --2- 172.21.15.101:0/1482741070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82d0101f70 0x7f82d0197e30 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.660+0000 7f82d51ff700 1 --2- 172.21.15.101:0/1482741070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82d0106fb0 0x7f82d019c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.660+0000 7f82d51ff700 1 --2- 172.21.15.101:0/1482741070 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f82d0102910 0x7f82d0198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.660+0000 7f82d51ff700 1 -- 172.21.15.101:0/1482741070 >> 172.21.15.101:0/1482741070 conn(0x7f82d00fd7e0 msgr2=0x7f82d0105910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:20.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.661+0000 7f82d51ff700 1 -- 172.21.15.101:0/1482741070 shutdown_connections 2023-07-18T03:17:20.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.661+0000 7f82d51ff700 1 -- 172.21.15.101:0/1482741070 wait complete. 2023-07-18T03:17:20.674 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-18T03:17:20.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.899+0000 7f8ee39ff700 1 -- 172.21.15.101:0/3284887521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8edc100ce0 msgr2=0x7f8edc1010b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:20.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.899+0000 7f8ee39ff700 1 --2- 172.21.15.101:0/3284887521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8edc100ce0 0x7f8edc1010b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8009990 tx=0x7f8ed8009ca0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.899+0000 7f8ee39ff700 1 -- 172.21.15.101:0/3284887521 shutdown_connections 2023-07-18T03:17:20.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.899+0000 7f8ee39ff700 1 --2- 172.21.15.101:0/3284887521 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8edc10a020 0x7f8edc10c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.899+0000 7f8ee39ff700 1 --2- 172.21.15.101:0/3284887521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8edc100ce0 0x7f8edc1010b0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.899+0000 7f8ee39ff700 1 --2- 172.21.15.101:0/3284887521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8edc1015f0 0x7f8edc109ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.899+0000 7f8ee39ff700 1 -- 172.21.15.101:0/3284887521 >> 172.21.15.101:0/3284887521 conn(0x7f8edc0fa4b0 msgr2=0x7f8edc0fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:20.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.899+0000 7f8ee39ff700 1 -- 172.21.15.101:0/3284887521 shutdown_connections 2023-07-18T03:17:20.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.899+0000 7f8ee39ff700 1 -- 172.21.15.101:0/3284887521 wait complete. 2023-07-18T03:17:20.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.901+0000 7f8ee39ff700 1 Processor -- start 2023-07-18T03:17:20.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.901+0000 7f8ee39ff700 1 -- start start 2023-07-18T03:17:20.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.902+0000 7f8ee39ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8edc100ce0 0x7f8edc0fe930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:20.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.902+0000 7f8ee39ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8edc1015f0 0x7f8edc0fee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:20.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.902+0000 7f8ee39ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8edc10a020 0x7f8edc1a3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:20.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.902+0000 7f8ee39ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc0ff6a0 con 0x7f8edc100ce0 2023-07-18T03:17:20.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.902+0000 7f8ee39ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc1a43a0 con 0x7f8edc10a020 2023-07-18T03:17:20.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.902+0000 7f8ee39ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc1a44e0 con 0x7f8edc1015f0 2023-07-18T03:17:20.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.902+0000 7f8ee179b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8edc100ce0 0x7f8edc0fe930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:20.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.902+0000 7f8ee0f9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8edc1015f0 0x7f8edc0fee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:20.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.902+0000 7f8ee0f9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8edc1015f0 0x7f8edc0fee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:44162/0 (socket says 172.21.15.101:44162) 2023-07-18T03:17:20.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.902+0000 7f8ee0f9a700 1 -- 172.21.15.101:0/2985886448 learned_addr learned my addr 172.21.15.101:0/2985886448 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:20.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.902+0000 7f8ee179b700 1 --2- 172.21.15.101:0/2985886448 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8edc100ce0 0x7f8edc0fe930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:45940/0 (socket says 172.21.15.101:45940) 2023-07-18T03:17:20.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.902+0000 7f8ee1f9c700 1 --2- 172.21.15.101:0/2985886448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8edc10a020 0x7f8edc1a3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:20.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.902+0000 7f8ee0f9a700 1 -- 172.21.15.101:0/2985886448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8edc10a020 msgr2=0x7f8edc1a3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:20.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.902+0000 7f8ee0f9a700 1 --2- 172.21.15.101:0/2985886448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8edc10a020 0x7f8edc1a3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.902+0000 7f8ee0f9a700 1 -- 172.21.15.101:0/2985886448 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8edc100ce0 msgr2=0x7f8edc0fe930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:20.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.902+0000 7f8ee0f9a700 1 --2- 172.21.15.101:0/2985886448 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8edc100ce0 0x7f8edc0fe930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.902+0000 7f8ee0f9a700 1 -- 172.21.15.101:0/2985886448 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed80095f0 con 0x7f8edc1015f0 2023-07-18T03:17:20.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.902+0000 7f8ee179b700 1 --2- 172.21.15.101:0/2985886448 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8edc100ce0 0x7f8edc0fe930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:17:20.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.903+0000 7f8ee0f9a700 1 --2- 172.21.15.101:0/2985886448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8edc1015f0 0x7f8edc0fee70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc00e9a0 tx=0x7f8ecc00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:20.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.903+0000 7f8ed27fc700 1 -- 172.21.15.101:0/2985886448 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc009790 con 0x7f8edc1015f0 2023-07-18T03:17:20.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.903+0000 7f8ed27fc700 1 -- 172.21.15.101:0/2985886448 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ecc010430 con 0x7f8edc1015f0 2023-07-18T03:17:20.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.903+0000 7f8ee39ff700 1 -- 172.21.15.101:0/2985886448 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8edc1a4740 con 0x7f8edc1015f0 2023-07-18T03:17:20.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.903+0000 7f8ed27fc700 1 -- 172.21.15.101:0/2985886448 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc018680 con 0x7f8edc1015f0 2023-07-18T03:17:20.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.903+0000 7f8ee39ff700 1 -- 172.21.15.101:0/2985886448 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8edc1a4c70 con 0x7f8edc1015f0 2023-07-18T03:17:20.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.906+0000 7f8ed27fc700 1 -- 172.21.15.101:0/2985886448 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f8ecc0105a0 con 0x7f8edc1015f0 2023-07-18T03:17:20.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.908+0000 7f8ef7246b80 1 Processor -- start 2023-07-18T03:17:20.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.908+0000 7f8ed27fc700 1 --2- 172.21.15.101:0/2985886448 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f8ec80a1340 0x7f8ec80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:20.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.908+0000 7f8ef7246b80 1 -- start start 2023-07-18T03:17:20.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.908+0000 7f8ed27fc700 1 -- 172.21.15.101:0/2985886448 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f8ecc014070 con 0x7f8edc1015f0 2023-07-18T03:17:20.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.908+0000 7f8ee1f9c700 1 --2- 172.21.15.101:0/2985886448 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f8ec80a1340 0x7f8ec80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:20.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.908+0000 7f8ef7246b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ebec612d0 0x555ebec616a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:20.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.908+0000 7f8ef7246b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ebec61be0 0x555ebec6c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:20.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.908+0000 7f8ef7246b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ebec6cad0 0x555ebec6eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:20.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.909+0000 7f8ef7246b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ebeb67cb0 con 0x555ebec61be0 2023-07-18T03:17:20.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.909+0000 7f8ef7246b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ebebbaa40 con 0x555ebec612d0 2023-07-18T03:17:20.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.909+0000 7f8ef7246b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ebebb3ed0 con 0x555ebec6cad0 2023-07-18T03:17:20.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.909+0000 7f8ee179b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ebec612d0 0x555ebec616a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:20.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.909+0000 7f8ee179b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ebec612d0 0x555ebec616a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60570/0 (socket says 172.21.15.101:60570) 2023-07-18T03:17:20.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.909+0000 7f8ee179b700 1 -- 172.21.15.101:0/1357563632 learned_addr learned my addr 172.21.15.101:0/1357563632 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:20.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.909+0000 7f8ee1f9c700 1 --2- 172.21.15.101:0/2985886448 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f8ec80a1340 0x7f8ec80a37f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8edc100a40 tx=0x7f8ed4009100 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:20.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.909+0000 7f8ee1f9c700 1 --2- 172.21.15.101:0/1357563632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ebec6cad0 0x555ebec6eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:20.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.909+0000 7f8ee0f9a700 1 --2- 172.21.15.101:0/1357563632 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ebec61be0 0x555ebec6c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:20.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.909+0000 7f8ee179b700 1 -- 172.21.15.101:0/1357563632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ebec6cad0 msgr2=0x555ebec6eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:20.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.909+0000 7f8ee179b700 1 --2- 172.21.15.101:0/1357563632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ebec6cad0 0x555ebec6eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.909+0000 7f8ee179b700 1 -- 172.21.15.101:0/1357563632 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ebec61be0 msgr2=0x555ebec6c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:20.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.909+0000 7f8ee179b700 1 --2- 172.21.15.101:0/1357563632 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ebec61be0 0x555ebec6c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.909+0000 7f8ee179b700 1 -- 172.21.15.101:0/1357563632 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ebe9e1de0 con 0x555ebec612d0 2023-07-18T03:17:20.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.909+0000 7f8ee179b700 1 --2- 172.21.15.101:0/1357563632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ebec612d0 0x555ebec616a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8ed800b6c0 tx=0x7f8ed8005ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3da4d5462ac97c69 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:20.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.910+0000 7f8ec77fe700 1 -- 172.21.15.101:0/1357563632 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed801d070 con 0x555ebec612d0 2023-07-18T03:17:20.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.910+0000 7f8ec77fe700 1 -- 172.21.15.101:0/1357563632 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ed801e040 con 0x555ebec612d0 2023-07-18T03:17:20.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.910+0000 7f8ec77fe700 1 -- 172.21.15.101:0/1357563632 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed800f460 con 0x555ebec612d0 2023-07-18T03:17:20.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.910+0000 7f8ef7246b80 1 -- 172.21.15.101:0/1357563632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ebec612d0 msgr2=0x555ebec616a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:20.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.910+0000 7f8ef7246b80 1 --2- 172.21.15.101:0/1357563632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ebec612d0 0x555ebec616a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8ed800b6c0 tx=0x7f8ed8005ee0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.911+0000 7f8ef7246b80 1 -- 172.21.15.101:0/1357563632 shutdown_connections 2023-07-18T03:17:20.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.911+0000 7f8ef7246b80 1 --2- 172.21.15.101:0/1357563632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ebec612d0 0x555ebec616a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.911+0000 7f8ef7246b80 1 --2- 172.21.15.101:0/1357563632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ebec6cad0 0x555ebec6eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.911+0000 7f8ef7246b80 1 --2- 172.21.15.101:0/1357563632 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ebec61be0 0x555ebec6c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.911+0000 7f8ef7246b80 1 -- 172.21.15.101:0/1357563632 >> 172.21.15.101:0/1357563632 conn(0x555ebec57da0 msgr2=0x555ebec5d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:20.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.911+0000 7f8ef7246b80 1 -- 172.21.15.101:0/1357563632 shutdown_connections 2023-07-18T03:17:20.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.911+0000 7f8ef7246b80 1 -- 172.21.15.101:0/1357563632 wait complete. 2023-07-18T03:17:20.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.911+0000 7f8ef7246b80 1 -- start start 2023-07-18T03:17:20.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.912+0000 7f8ef7246b80 1 Processor -- start 2023-07-18T03:17:20.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.912+0000 7f8ef7246b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ebec612d0 0x555ebec6c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:20.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.912+0000 7f8ef7246b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ebec61be0 0x555ebec69750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:20.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.912+0000 7f8ef7246b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ebec6cad0 0x555ebec69e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:20.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.912+0000 7f8ef7246b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ebeb53ac0 con 0x555ebec6cad0 2023-07-18T03:17:20.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.912+0000 7f8ef7246b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ebeb537c0 con 0x555ebec612d0 2023-07-18T03:17:20.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.912+0000 7f8ef7246b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ebea37920 con 0x555ebec61be0 2023-07-18T03:17:20.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.912+0000 7f8ef7246b80 10 client.0 _ll_register_callbacks cb 0x555ebeb51760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:17:20.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.912+0000 7f8ee179b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ebec612d0 0x555ebec6c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:20.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.912+0000 7f8ee0f9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ebec61be0 0x555ebec69750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:20.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.912+0000 7f8ee179b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ebec612d0 0x555ebec6c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60580/0 (socket says 172.21.15.101:60580) 2023-07-18T03:17:20.916 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.912+0000 7f8ee1f9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ebec6cad0 0x555ebec69e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:20.916 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.912+0000 7f8ee179b700 1 -- 172.21.15.101:0/99662303 learned_addr learned my addr 172.21.15.101:0/99662303 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:20.916 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.913+0000 7f8ee0f9a700 1 -- 172.21.15.101:0/99662303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ebec612d0 msgr2=0x555ebec6c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:20.916 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.913+0000 7f8ee0f9a700 1 --2- 172.21.15.101:0/99662303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ebec612d0 0x555ebec6c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.916 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.913+0000 7f8ee0f9a700 1 -- 172.21.15.101:0/99662303 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ebec6cad0 msgr2=0x555ebec69e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:20.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.913+0000 7f8ee0f9a700 1 --2- 172.21.15.101:0/99662303 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ebec6cad0 0x555ebec69e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.913+0000 7f8ee0f9a700 1 -- 172.21.15.101:0/99662303 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8edc10d170 con 0x555ebec61be0 2023-07-18T03:17:20.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.913+0000 7f8ee0f9a700 1 --2- 172.21.15.101:0/99662303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ebec61be0 0x555ebec69750 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc0ca5e0 tx=0x7f8ecc099930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=936968cf0d1c9b96 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:20.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.913+0000 7f8ec4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:17:20.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.913+0000 7f8ec4ff9700 1 -- 172.21.15.101:0/99662303 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc099650 con 0x555ebec61be0 2023-07-18T03:17:20.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.913+0000 7f8ec4ff9700 1 -- 172.21.15.101:0/99662303 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f8ecc09df90 con 0x555ebec61be0 2023-07-18T03:17:20.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.913+0000 7f8ef7246b80 1 -- 172.21.15.101:0/99662303 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555ebeb0d4f0 con 0x555ebec61be0 2023-07-18T03:17:20.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.913+0000 7f8ec4ff9700 1 -- 172.21.15.101:0/99662303 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ecc0a23b0 con 0x555ebec61be0 2023-07-18T03:17:20.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.913+0000 7f8ef7246b80 10 client.27583 fetch_fsmap learned FSMap version 33 2023-07-18T03:17:20.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.913+0000 7f8ef7246b80 1 -- 172.21.15.101:0/99662303 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x555ebe9cac60 con 0x555ebec61be0 2023-07-18T03:17:20.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.914+0000 7f8ec4ff9700 1 -- 172.21.15.101:0/99662303 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ecc0ca080 con 0x555ebec61be0 2023-07-18T03:17:20.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.914+0000 7f8ec4ff9700 1 -- 172.21.15.101:0/99662303 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8ecc0a2040 con 0x555ebec61be0 2023-07-18T03:17:20.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.914+0000 7f8ef7246b80 10 client.27583 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:17:20.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.914+0000 7f8ef7246b80 10 client.27583 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:17:20.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.914+0000 7f8ef7246b80 20 client.27583 populate_metadata read hostname 'smithi101' 2023-07-18T03:17:20.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.914+0000 7f8ef7246b80 1 --2- 172.21.15.101:0/99662303 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x555ebec7c310 0x555ebec7e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:20.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.914+0000 7f8ef7246b80 4 client.27583 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:17:20.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.914+0000 7f8ef7246b80 1 -- 172.21.15.101:0/99662303 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555ebebaceb0 con 0x555ebec7c310 2023-07-18T03:17:20.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.914+0000 7f8ee179b700 1 --2- 172.21.15.101:0/99662303 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x555ebec7c310 0x555ebec7e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:20.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.915+0000 7f8ec4ff9700 10 client.27583 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:17:20.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.915+0000 7f8ee179b700 1 --2- 172.21.15.101:0/99662303 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x555ebec7c310 0x555ebec7e720 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6591de9781138eb9 server_cookie=bffece3fe33530a1 in_seq=0 out_seq=0 2023-07-18T03:17:20.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.915+0000 7f8ec4ff9700 1 -- 172.21.15.101:0/99662303 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555ebebaceb0 con 0x555ebec7c310 2023-07-18T03:17:20.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.915+0000 7f8ec4ff9700 10 client.27583 handle_command_reply: tid=0 2023-07-18T03:17:20.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.915+0000 7f8ef7246b80 1 client.27583 shutdown 2023-07-18T03:17:20.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.916+0000 7f8ef7246b80 1 -- 172.21.15.101:0/99662303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ebec61be0 msgr2=0x555ebec69750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:20.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.916+0000 7f8ef7246b80 1 --2- 172.21.15.101:0/99662303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ebec61be0 0x555ebec69750 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc0ca5e0 tx=0x7f8ecc099930 comp rx=0 tx=0).stop 2023-07-18T03:17:20.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.916+0000 7f8ef7246b80 1 -- 172.21.15.101:0/99662303 shutdown_connections 2023-07-18T03:17:20.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.916+0000 7f8ef7246b80 1 --2- 172.21.15.101:0/99662303 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x555ebec7c310 0x555ebec7e720 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.916+0000 7f8ef7246b80 1 --2- 172.21.15.101:0/99662303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ebec612d0 0x555ebec6c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.916+0000 7f8ec4ff9700 0 client.27583 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:17:20.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.916+0000 7f8ef7246b80 1 --2- 172.21.15.101:0/99662303 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ebec6cad0 0x555ebec69e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.922 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.916+0000 7f8ef7246b80 1 --2- 172.21.15.101:0/99662303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ebec61be0 0x555ebec69750 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.922 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.916+0000 7f8ef7246b80 1 -- 172.21.15.101:0/99662303 >> 172.21.15.101:0/99662303 conn(0x555ebec57da0 msgr2=0x555ebec5d230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:20.922 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.916+0000 7f8ef7246b80 1 -- 172.21.15.101:0/99662303 shutdown_connections 2023-07-18T03:17:20.922 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.916+0000 7f8ef7246b80 1 -- 172.21.15.101:0/99662303 wait complete. 2023-07-18T03:17:20.922 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.916+0000 7f8ef7246b80 20 client.27583 trim_cache size 0 max 16384 2023-07-18T03:17:20.925 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.930+0000 7f8ef7246b80 1 Processor -- start 2023-07-18T03:17:20.925 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.930+0000 7f8ef7246b80 1 -- start start 2023-07-18T03:17:20.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.931+0000 7f8ef7246b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ebec612d0 0x555ebec69d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:20.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.931+0000 7f8ef7246b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ebec61be0 0x555ebec83080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:20.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.931+0000 7f8ef7246b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ebec6cad0 0x555ebec734b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:20.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.931+0000 7f8ef7246b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ebec835c0 con 0x555ebec61be0 2023-07-18T03:17:20.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.931+0000 7f8ef7246b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ebec739f0 con 0x555ebec612d0 2023-07-18T03:17:20.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.931+0000 7f8ef7246b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ebec68cc0 con 0x555ebec6cad0 2023-07-18T03:17:20.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.931+0000 7f8ee0f9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ebec61be0 0x555ebec83080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:20.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.931+0000 7f8ee1f9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ebec6cad0 0x555ebec734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:20.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.931+0000 7f8ee1f9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ebec6cad0 0x555ebec734b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:44198/0 (socket says 172.21.15.101:44198) 2023-07-18T03:17:20.928 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.931+0000 7f8ee1f9c700 1 -- 172.21.15.101:0/2814658086 learned_addr learned my addr 172.21.15.101:0/2814658086 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:20.928 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.931+0000 7f8ee179b700 1 --2- 172.21.15.101:0/2814658086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ebec612d0 0x555ebec69d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:20.928 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.931+0000 7f8ee1f9c700 1 -- 172.21.15.101:0/2814658086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ebec612d0 msgr2=0x555ebec69d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:20.928 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.931+0000 7f8ee1f9c700 1 --2- 172.21.15.101:0/2814658086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ebec612d0 0x555ebec69d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.928 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.931+0000 7f8ee1f9c700 1 -- 172.21.15.101:0/2814658086 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ebec61be0 msgr2=0x555ebec83080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:20.928 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.931+0000 7f8ee1f9c700 1 --2- 172.21.15.101:0/2814658086 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ebec61be0 0x555ebec83080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.928 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.931+0000 7f8ee1f9c700 1 -- 172.21.15.101:0/2814658086 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ebea08250 con 0x555ebec6cad0 2023-07-18T03:17:20.929 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.932+0000 7f8ee1f9c700 1 --2- 172.21.15.101:0/2814658086 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ebec6cad0 0x555ebec734b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8ed4004810 tx=0x7f8ed4013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4da9e2c662287724 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:20.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.932+0000 7f8ec7fff700 1 -- 172.21.15.101:0/2814658086 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4023460 con 0x555ebec6cad0 2023-07-18T03:17:20.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.932+0000 7f8ec7fff700 1 -- 172.21.15.101:0/2814658086 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ed400f040 con 0x555ebec6cad0 2023-07-18T03:17:20.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.932+0000 7f8ec7fff700 1 -- 172.21.15.101:0/2814658086 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4015a00 con 0x555ebec6cad0 2023-07-18T03:17:20.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.933+0000 7f8ef7246b80 1 -- 172.21.15.101:0/2814658086 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ebec6cad0 msgr2=0x555ebec734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:20.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.933+0000 7f8ef7246b80 1 --2- 172.21.15.101:0/2814658086 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ebec6cad0 0x555ebec734b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8ed4004810 tx=0x7f8ed4013000 comp rx=0 tx=0).stop 2023-07-18T03:17:20.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.933+0000 7f8ef7246b80 1 -- 172.21.15.101:0/2814658086 shutdown_connections 2023-07-18T03:17:20.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.933+0000 7f8ef7246b80 1 --2- 172.21.15.101:0/2814658086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ebec612d0 0x555ebec69d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.933+0000 7f8ef7246b80 1 --2- 172.21.15.101:0/2814658086 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ebec6cad0 0x555ebec734b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.933+0000 7f8ef7246b80 1 --2- 172.21.15.101:0/2814658086 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ebec61be0 0x555ebec83080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.933+0000 7f8ef7246b80 1 -- 172.21.15.101:0/2814658086 >> 172.21.15.101:0/2814658086 conn(0x555ebec57da0 msgr2=0x555ebec59310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:20.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.933+0000 7f8ef7246b80 1 -- 172.21.15.101:0/2814658086 shutdown_connections 2023-07-18T03:17:20.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.933+0000 7f8ef7246b80 1 -- 172.21.15.101:0/2814658086 wait complete. 2023-07-18T03:17:20.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.933+0000 7f8ef7246b80 1 -- start start 2023-07-18T03:17:20.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.934+0000 7f8ef7246b80 1 Processor -- start 2023-07-18T03:17:20.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.934+0000 7f8ef7246b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ebec612d0 0x555ebec73250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:20.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.934+0000 7f8ef7246b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ebec61be0 0x555ebec71580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:20.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.934+0000 7f8ef7246b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ebec6cad0 0x555ebec71ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:20.934 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.934+0000 7f8ef7246b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ebec721d0 con 0x555ebec6cad0 2023-07-18T03:17:20.934 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.934+0000 7f8ef7246b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ebec72340 con 0x555ebec612d0 2023-07-18T03:17:20.934 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.934+0000 7f8ef7246b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ebecd01e0 con 0x555ebec61be0 2023-07-18T03:17:20.934 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.935+0000 7f8ef7246b80 10 client.0 _ll_register_callbacks cb 0x7f8ecc0cd140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:17:20.934 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.935+0000 7f8ee179b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ebec612d0 0x555ebec73250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:20.934 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.935+0000 7f8ee179b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ebec612d0 0x555ebec73250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60600/0 (socket says 172.21.15.101:60600) 2023-07-18T03:17:20.934 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.935+0000 7f8ee0f9a700 1 --2- 172.21.15.101:0/3070203474 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ebec61be0 0x555ebec71580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:20.935 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.935+0000 7f8ee179b700 1 -- 172.21.15.101:0/3070203474 learned_addr learned my addr 172.21.15.101:0/3070203474 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:20.935 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.935+0000 7f8ee1f9c700 1 --2- 172.21.15.101:0/3070203474 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ebec6cad0 0x555ebec71ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:20.935 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.935+0000 7f8ee179b700 1 -- 172.21.15.101:0/3070203474 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ebec61be0 msgr2=0x555ebec71580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:20.935 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.935+0000 7f8ee179b700 1 --2- 172.21.15.101:0/3070203474 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ebec61be0 0x555ebec71580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.935 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.935+0000 7f8ee179b700 1 -- 172.21.15.101:0/3070203474 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ebec6cad0 msgr2=0x555ebec71ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:20.935 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.935+0000 7f8ee179b700 1 --2- 172.21.15.101:0/3070203474 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ebec6cad0 0x555ebec71ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.935+0000 7f8ee179b700 1 -- 172.21.15.101:0/3070203474 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ed40044c0 con 0x555ebec612d0 2023-07-18T03:17:20.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.935+0000 7f8ee1f9c700 1 --2- 172.21.15.101:0/3070203474 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ebec6cad0 0x555ebec71ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:17:20.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.935+0000 7f8ee179b700 1 --2- 172.21.15.101:0/3070203474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ebec612d0 0x555ebec73250 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8009ce0 tx=0x7f8ed8004500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a38824a1f0235ff server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:20.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.935+0000 7f8ec4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:17:20.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.936+0000 7f8ec4ff9700 1 -- 172.21.15.101:0/3070203474 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed8034070 con 0x555ebec612d0 2023-07-18T03:17:20.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.936+0000 7f8ec4ff9700 1 -- 172.21.15.101:0/3070203474 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f8ed80341d0 con 0x555ebec612d0 2023-07-18T03:17:20.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.936+0000 7f8ef7246b80 1 -- 172.21.15.101:0/3070203474 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ed801e040 con 0x555ebec612d0 2023-07-18T03:17:20.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.936+0000 7f8ec4ff9700 1 -- 172.21.15.101:0/3070203474 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ed801d040 con 0x555ebec612d0 2023-07-18T03:17:20.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.936+0000 7f8ef7246b80 10 client.27554 fetch_fsmap learned FSMap version 33 2023-07-18T03:17:20.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.936+0000 7f8ef7246b80 1 -- 172.21.15.101:0/3070203474 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x555ebe9e02b0 con 0x555ebec612d0 2023-07-18T03:17:20.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.936+0000 7f8ec4ff9700 1 -- 172.21.15.101:0/3070203474 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ed801e040 con 0x555ebec612d0 2023-07-18T03:17:20.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.936+0000 7f8ec4ff9700 1 -- 172.21.15.101:0/3070203474 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8ed800f4d0 con 0x555ebec612d0 2023-07-18T03:17:20.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.936+0000 7f8ef7246b80 10 client.27554 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:17:20.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.936+0000 7f8ef7246b80 10 client.27554 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:17:20.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.936+0000 7f8ef7246b80 20 client.27554 populate_metadata read hostname 'smithi101' 2023-07-18T03:17:20.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.936+0000 7f8ef7246b80 1 --2- 172.21.15.101:0/3070203474 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x555ebecd13a0 0x555ebecd37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:20.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.936+0000 7f8ef7246b80 4 client.27554 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:17:20.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.936+0000 7f8ef7246b80 1 -- 172.21.15.101:0/3070203474 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555ebeb6f050 con 0x555ebecd13a0 2023-07-18T03:17:20.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.937+0000 7f8ee0f9a700 1 --2- 172.21.15.101:0/3070203474 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x555ebecd13a0 0x555ebecd37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:20.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.937+0000 7f8ec4ff9700 10 client.27554 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:17:20.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.937+0000 7f8ee0f9a700 1 --2- 172.21.15.101:0/3070203474 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x555ebecd13a0 0x555ebecd37b0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=21f298f67466bf3a server_cookie=16254ae87c1cf913 in_seq=0 out_seq=0 2023-07-18T03:17:20.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.937+0000 7f8ec4ff9700 1 -- 172.21.15.101:0/3070203474 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555ebeb6f050 con 0x555ebecd13a0 2023-07-18T03:17:20.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.937+0000 7f8ec4ff9700 10 client.27554 handle_command_reply: tid=0 2023-07-18T03:17:20.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.937+0000 7f8ef7246b80 1 client.27554 shutdown 2023-07-18T03:17:20.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.937+0000 7f8ef7246b80 1 -- 172.21.15.101:0/3070203474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ebec612d0 msgr2=0x555ebec73250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:20.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.937+0000 7f8ef7246b80 1 --2- 172.21.15.101:0/3070203474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ebec612d0 0x555ebec73250 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8009ce0 tx=0x7f8ed8004500 comp rx=0 tx=0).stop 2023-07-18T03:17:20.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.937+0000 7f8ef7246b80 1 -- 172.21.15.101:0/3070203474 shutdown_connections 2023-07-18T03:17:20.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.937+0000 7f8ef7246b80 1 --2- 172.21.15.101:0/3070203474 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x555ebecd13a0 0x555ebecd37b0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.937+0000 7f8ef7246b80 1 --2- 172.21.15.101:0/3070203474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ebec612d0 0x555ebec73250 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.937+0000 7f8ef7246b80 1 --2- 172.21.15.101:0/3070203474 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555ebec6cad0 0x555ebec71ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.937+0000 7f8ef7246b80 1 --2- 172.21.15.101:0/3070203474 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555ebec61be0 0x555ebec71580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.937+0000 7f8ef7246b80 1 -- 172.21.15.101:0/3070203474 >> 172.21.15.101:0/3070203474 conn(0x555ebec57da0 msgr2=0x555ebec786c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:20.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.938+0000 7f8ec4ff9700 0 client.27554 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:17:20.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.938+0000 7f8ef7246b80 1 -- 172.21.15.101:0/3070203474 shutdown_connections 2023-07-18T03:17:20.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.938+0000 7f8ef7246b80 1 -- 172.21.15.101:0/3070203474 wait complete. 2023-07-18T03:17:20.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.938+0000 7f8ef7246b80 20 client.27554 trim_cache size 0 max 16384 2023-07-18T03:17:20.942 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:17:20.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.939+0000 7f8ee39ff700 1 -- 172.21.15.101:0/2985886448 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f8ec80a1340 msgr2=0x7f8ec80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:20.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.939+0000 7f8ee39ff700 1 --2- 172.21.15.101:0/2985886448 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f8ec80a1340 0x7f8ec80a37f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8edc100a40 tx=0x7f8ed4009100 comp rx=0 tx=0).stop 2023-07-18T03:17:20.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.939+0000 7f8ee39ff700 1 -- 172.21.15.101:0/2985886448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8edc1015f0 msgr2=0x7f8edc0fee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:20.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.939+0000 7f8ee39ff700 1 --2- 172.21.15.101:0/2985886448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8edc1015f0 0x7f8edc0fee70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc00e9a0 tx=0x7f8ecc00ecb0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.939+0000 7f8ee39ff700 1 -- 172.21.15.101:0/2985886448 shutdown_connections 2023-07-18T03:17:20.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.939+0000 7f8ee39ff700 1 --2- 172.21.15.101:0/2985886448 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f8ec80a1340 0x7f8ec80a37f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.939+0000 7f8ee39ff700 1 --2- 172.21.15.101:0/2985886448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8edc10a020 0x7f8edc1a3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.939+0000 7f8ee39ff700 1 --2- 172.21.15.101:0/2985886448 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8edc100ce0 0x7f8edc0fe930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.939+0000 7f8ee39ff700 1 --2- 172.21.15.101:0/2985886448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8edc1015f0 0x7f8edc0fee70 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:20.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.939+0000 7f8ee39ff700 1 -- 172.21.15.101:0/2985886448 >> 172.21.15.101:0/2985886448 conn(0x7f8edc0fa4b0 msgr2=0x7f8edc10c210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:20.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.939+0000 7f8ee39ff700 1 -- 172.21.15.101:0/2985886448 shutdown_connections 2023-07-18T03:17:20.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:20.939+0000 7f8ee39ff700 1 -- 172.21.15.101:0/2985886448 wait complete. 2023-07-18T03:17:20.953 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-18T03:17:21.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.195+0000 7f9a555ff700 1 -- 172.21.15.101:0/3093172527 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9a50100490 msgr2=0x7f9a5010c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:21.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.195+0000 7f9a555ff700 1 --2- 172.21.15.101:0/3093172527 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9a50100490 0x7f9a5010c920 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f9a4800b0a0 tx=0x7f9a4800b3b0 comp rx=0 tx=0).stop 2023-07-18T03:17:21.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.196+0000 7f9a555ff700 1 -- 172.21.15.101:0/3093172527 shutdown_connections 2023-07-18T03:17:21.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.196+0000 7f9a555ff700 1 --2- 172.21.15.101:0/3093172527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a50105af0 0x7f9a50105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:21.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.196+0000 7f9a555ff700 1 --2- 172.21.15.101:0/3093172527 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9a50100490 0x7f9a5010c920 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:21.192 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.196+0000 7f9a555ff700 1 --2- 172.21.15.101:0/3093172527 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a500ffae0 0x7f9a500fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:21.192 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.196+0000 7f9a555ff700 1 -- 172.21.15.101:0/3093172527 >> 172.21.15.101:0/3093172527 conn(0x7f9a500fb5e0 msgr2=0x7f9a500fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:21.192 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.196+0000 7f9a555ff700 1 -- 172.21.15.101:0/3093172527 shutdown_connections 2023-07-18T03:17:21.192 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.196+0000 7f9a555ff700 1 -- 172.21.15.101:0/3093172527 wait complete. 2023-07-18T03:17:21.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.197+0000 7f9a555ff700 1 Processor -- start 2023-07-18T03:17:21.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.197+0000 7f9a555ff700 1 -- start start 2023-07-18T03:17:21.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.198+0000 7f9a555ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a500ffae0 0x7f9a50197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:21.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.198+0000 7f9a555ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9a50100490 0x7f9a50198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:21.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.198+0000 7f9a555ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a50105af0 0x7f9a5019c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:21.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.198+0000 7f9a555ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5019cf30 con 0x7f9a50100490 2023-07-18T03:17:21.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.198+0000 7f9a555ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5019d0a0 con 0x7f9a500ffae0 2023-07-18T03:17:21.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.198+0000 7f9a555ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5019d260 con 0x7f9a50105af0 2023-07-18T03:17:21.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.198+0000 7f9a4effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a500ffae0 0x7f9a50197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:21.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.198+0000 7f9a4e7fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9a50100490 0x7f9a50198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:21.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.198+0000 7f9a4effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a500ffae0 0x7f9a50197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60626/0 (socket says 172.21.15.101:60626) 2023-07-18T03:17:21.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.198+0000 7f9a4effd700 1 -- 172.21.15.101:0/3295486070 learned_addr learned my addr 172.21.15.101:0/3295486070 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:21.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.198+0000 7f9a4f7fe700 1 --2- 172.21.15.101:0/3295486070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a50105af0 0x7f9a5019c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:21.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.198+0000 7f9a4effd700 1 -- 172.21.15.101:0/3295486070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a50105af0 msgr2=0x7f9a5019c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:21.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.198+0000 7f9a4effd700 1 --2- 172.21.15.101:0/3295486070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a50105af0 0x7f9a5019c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:21.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.198+0000 7f9a4effd700 1 -- 172.21.15.101:0/3295486070 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9a50100490 msgr2=0x7f9a50198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:21.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.198+0000 7f9a4effd700 1 --2- 172.21.15.101:0/3295486070 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9a50100490 0x7f9a50198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:21.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.198+0000 7f9a4effd700 1 -- 172.21.15.101:0/3295486070 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a48009d00 con 0x7f9a500ffae0 2023-07-18T03:17:21.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.199+0000 7f9a4effd700 1 --2- 172.21.15.101:0/3295486070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a500ffae0 0x7f9a50197f50 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9a4000e9a0 tx=0x7f9a4000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:21.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.199+0000 7f9a37fff700 1 -- 172.21.15.101:0/3295486070 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a40009790 con 0x7f9a500ffae0 2023-07-18T03:17:21.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.199+0000 7f9a555ff700 1 -- 172.21.15.101:0/3295486070 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a5019d540 con 0x7f9a500ffae0 2023-07-18T03:17:21.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.199+0000 7f9a37fff700 1 -- 172.21.15.101:0/3295486070 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a40010430 con 0x7f9a500ffae0 2023-07-18T03:17:21.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.199+0000 7f9a37fff700 1 -- 172.21.15.101:0/3295486070 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a40018680 con 0x7f9a500ffae0 2023-07-18T03:17:21.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.199+0000 7f9a555ff700 1 -- 172.21.15.101:0/3295486070 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a501a4d80 con 0x7f9a500ffae0 2023-07-18T03:17:21.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.203+0000 7f9a37fff700 1 -- 172.21.15.101:0/3295486070 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f9a400105a0 con 0x7f9a500ffae0 2023-07-18T03:17:21.198 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.203+0000 7f9a68e9cb80 1 Processor -- start 2023-07-18T03:17:21.198 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.204+0000 7f9a68e9cb80 1 -- start start 2023-07-18T03:17:21.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.204+0000 7f9a68e9cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575b7b632d0 0x5575b7b636a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:21.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.204+0000 7f9a68e9cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b7b63be0 0x5575b7b6e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:21.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.204+0000 7f9a68e9cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5575b7b6ead0 0x5575b7b70eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:21.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.204+0000 7f9a68e9cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b7a69cb0 con 0x5575b7b6ead0 2023-07-18T03:17:21.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.204+0000 7f9a68e9cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b7abca40 con 0x5575b7b63be0 2023-07-18T03:17:21.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.204+0000 7f9a68e9cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b7ab5ed0 con 0x5575b7b632d0 2023-07-18T03:17:21.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.205+0000 7f9a4effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b7b63be0 0x5575b7b6e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:21.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.205+0000 7f9a4f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575b7b632d0 0x5575b7b636a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:21.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.205+0000 7f9a4f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575b7b632d0 0x5575b7b636a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:44244/0 (socket says 172.21.15.101:44244) 2023-07-18T03:17:21.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.205+0000 7f9a4f7fe700 1 -- 172.21.15.101:0/4083418965 learned_addr learned my addr 172.21.15.101:0/4083418965 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:21.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.205+0000 7f9a4e7fc700 1 --2- 172.21.15.101:0/4083418965 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5575b7b6ead0 0x5575b7b70eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:21.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.205+0000 7f9a37fff700 1 --2- 172.21.15.101:0/3295486070 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9a380a12c0 0x7f9a380a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:21.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.205+0000 7f9a4effd700 1 -- 172.21.15.101:0/4083418965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575b7b632d0 msgr2=0x5575b7b636a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:21.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.205+0000 7f9a4effd700 1 --2- 172.21.15.101:0/4083418965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575b7b632d0 0x5575b7b636a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:21.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.205+0000 7f9a4effd700 1 -- 172.21.15.101:0/4083418965 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5575b7b6ead0 msgr2=0x5575b7b70eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:21.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.205+0000 7f9a4effd700 1 --2- 172.21.15.101:0/4083418965 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5575b7b6ead0 0x5575b7b70eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:21.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.205+0000 7f9a4effd700 1 -- 172.21.15.101:0/4083418965 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575b78e3de0 con 0x5575b7b63be0 2023-07-18T03:17:21.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.205+0000 7f9a4e7fc700 1 --2- 172.21.15.101:0/4083418965 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5575b7b6ead0 0x5575b7b70eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:17:21.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.205+0000 7f9a37fff700 1 -- 172.21.15.101:0/3295486070 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f9a40014070 con 0x7f9a500ffae0 2023-07-18T03:17:21.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.205+0000 7f9a4f7fe700 1 --2- 172.21.15.101:0/3295486070 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9a380a12c0 0x7f9a380a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:21.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.205+0000 7f9a4effd700 1 --2- 172.21.15.101:0/4083418965 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b7b63be0 0x5575b7b6e590 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9a400ca5a0 tx=0x7f9a40024000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4464697e2703452c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:21.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.206+0000 7f9a357fa700 1 -- 172.21.15.101:0/4083418965 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a40099a60 con 0x5575b7b63be0 2023-07-18T03:17:21.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.206+0000 7f9a357fa700 1 -- 172.21.15.101:0/4083418965 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a40099880 con 0x5575b7b63be0 2023-07-18T03:17:21.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.206+0000 7f9a357fa700 1 -- 172.21.15.101:0/4083418965 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a400982c0 con 0x5575b7b63be0 2023-07-18T03:17:21.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.206+0000 7f9a4f7fe700 1 --2- 172.21.15.101:0/3295486070 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9a380a12c0 0x7f9a380a3770 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9a4800b0a0 tx=0x7f9a480099e0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:21.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.206+0000 7f9a68e9cb80 1 -- 172.21.15.101:0/4083418965 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b7b63be0 msgr2=0x5575b7b6e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:21.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.206+0000 7f9a68e9cb80 1 --2- 172.21.15.101:0/4083418965 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b7b63be0 0x5575b7b6e590 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9a400ca5a0 tx=0x7f9a40024000 comp rx=0 tx=0).stop 2023-07-18T03:17:21.205 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.206+0000 7f9a68e9cb80 1 -- 172.21.15.101:0/4083418965 shutdown_connections 2023-07-18T03:17:21.205 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.206+0000 7f9a68e9cb80 1 --2- 172.21.15.101:0/4083418965 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5575b7b6ead0 0x5575b7b70eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:21.205 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.206+0000 7f9a68e9cb80 1 --2- 172.21.15.101:0/4083418965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575b7b632d0 0x5575b7b636a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:21.205 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.206+0000 7f9a68e9cb80 1 --2- 172.21.15.101:0/4083418965 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b7b63be0 0x5575b7b6e590 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:21.205 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.207+0000 7f9a68e9cb80 1 -- 172.21.15.101:0/4083418965 >> 172.21.15.101:0/4083418965 conn(0x5575b7b59da0 msgr2=0x5575b7b5f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:21.205 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.207+0000 7f9a68e9cb80 1 -- 172.21.15.101:0/4083418965 shutdown_connections 2023-07-18T03:17:21.205 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.207+0000 7f9a68e9cb80 1 -- 172.21.15.101:0/4083418965 wait complete. 2023-07-18T03:17:21.206 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.207+0000 7f9a68e9cb80 1 -- start start 2023-07-18T03:17:21.206 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.207+0000 7f9a68e9cb80 1 Processor -- start 2023-07-18T03:17:21.208 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.208+0000 7f9a68e9cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575b7b632d0 0x5575b7b6e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:21.208 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.208+0000 7f9a68e9cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5575b7b63be0 0x5575b7b6b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:21.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.208+0000 7f9a68e9cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b7b6ead0 0x5575b7b6be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:21.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.208+0000 7f9a68e9cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b7a55ac0 con 0x5575b7b63be0 2023-07-18T03:17:21.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.208+0000 7f9a68e9cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b7a557c0 con 0x5575b7b6ead0 2023-07-18T03:17:21.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.208+0000 7f9a68e9cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b7939920 con 0x5575b7b632d0 2023-07-18T03:17:21.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.208+0000 7f9a68e9cb80 10 client.0 _ll_register_callbacks cb 0x5575b7a53760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:17:21.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.208+0000 7f9a4effd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575b7b632d0 0x5575b7b6e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:21.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.208+0000 7f9a4effd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575b7b632d0 0x5575b7b6e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:44252/0 (socket says 172.21.15.101:44252) 2023-07-18T03:17:21.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.208+0000 7f9a4effd700 1 -- 172.21.15.101:0/2994613474 learned_addr learned my addr 172.21.15.101:0/2994613474 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:21.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.208+0000 7f9a4e7fc700 1 --2- 172.21.15.101:0/2994613474 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5575b7b63be0 0x5575b7b6b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:21.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.209+0000 7f9a4f7fe700 1 --2- 172.21.15.101:0/2994613474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b7b6ead0 0x5575b7b6be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:21.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.209+0000 7f9a4effd700 1 -- 172.21.15.101:0/2994613474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b7b6ead0 msgr2=0x5575b7b6be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:21.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.209+0000 7f9a4effd700 1 --2- 172.21.15.101:0/2994613474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b7b6ead0 0x5575b7b6be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:21.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.209+0000 7f9a4effd700 1 -- 172.21.15.101:0/2994613474 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5575b7b63be0 msgr2=0x5575b7b6b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:21.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.209+0000 7f9a4effd700 1 --2- 172.21.15.101:0/2994613474 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5575b7b63be0 0x5575b7b6b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:21.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.209+0000 7f9a4effd700 1 -- 172.21.15.101:0/2994613474 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a440098e0 con 0x5575b7b632d0 2023-07-18T03:17:21.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.209+0000 7f9a4e7fc700 1 --2- 172.21.15.101:0/2994613474 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5575b7b63be0 0x5575b7b6b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:17:21.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.209+0000 7f9a4effd700 1 --2- 172.21.15.101:0/2994613474 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575b7b632d0 0x5575b7b6e520 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9a400ca5d0 tx=0x7f9a40099e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78430cab92df2fad server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:21.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.209+0000 7f9a32ff5700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:17:21.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.209+0000 7f9a32ff5700 1 -- 172.21.15.101:0/2994613474 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a400a2370 con 0x5575b7b632d0 2023-07-18T03:17:21.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.209+0000 7f9a32ff5700 1 -- 172.21.15.101:0/2994613474 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f9a40095d10 con 0x5575b7b632d0 2023-07-18T03:17:21.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.209+0000 7f9a68e9cb80 1 -- 172.21.15.101:0/2994613474 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5575b7a0f4f0 con 0x5575b7b632d0 2023-07-18T03:17:21.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.209+0000 7f9a32ff5700 1 -- 172.21.15.101:0/2994613474 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a40050120 con 0x5575b7b632d0 2023-07-18T03:17:21.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.209+0000 7f9a68e9cb80 10 client.27601 fetch_fsmap learned FSMap version 33 2023-07-18T03:17:21.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.209+0000 7f9a68e9cb80 1 -- 172.21.15.101:0/2994613474 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f9a501a4d80 con 0x5575b7b632d0 2023-07-18T03:17:21.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.210+0000 7f9a32ff5700 1 -- 172.21.15.101:0/2994613474 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5575b7a0f4f0 con 0x5575b7b632d0 2023-07-18T03:17:21.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.210+0000 7f9a32ff5700 1 -- 172.21.15.101:0/2994613474 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9a40099410 con 0x5575b7b632d0 2023-07-18T03:17:21.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.210+0000 7f9a68e9cb80 10 client.27601 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:17:21.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.210+0000 7f9a68e9cb80 10 client.27601 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:17:21.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.210+0000 7f9a68e9cb80 20 client.27601 populate_metadata read hostname 'smithi101' 2023-07-18T03:17:21.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.210+0000 7f9a68e9cb80 1 --2- 172.21.15.101:0/2994613474 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5575b7b7de30 0x5575b7b80240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:21.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.210+0000 7f9a68e9cb80 4 client.27601 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:17:21.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.210+0000 7f9a68e9cb80 1 -- 172.21.15.101:0/2994613474 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5575b7aaeeb0 con 0x5575b7b7de30 2023-07-18T03:17:21.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.210+0000 7f9a4e7fc700 1 --2- 172.21.15.101:0/2994613474 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5575b7b7de30 0x5575b7b80240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:21.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.211+0000 7f9a32ff5700 10 client.27601 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:17:21.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.211+0000 7f9a4e7fc700 1 --2- 172.21.15.101:0/2994613474 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5575b7b7de30 0x5575b7b80240 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=697380e316591f07 server_cookie=dee844b9ca5280e7 in_seq=0 out_seq=0 2023-07-18T03:17:21.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.211+0000 7f9a32ff5700 1 -- 172.21.15.101:0/2994613474 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5575b7aaeeb0 con 0x5575b7b7de30 2023-07-18T03:17:21.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.211+0000 7f9a32ff5700 10 client.27601 handle_command_reply: tid=0 2023-07-18T03:17:21.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.211+0000 7f9a68e9cb80 1 client.27601 shutdown 2023-07-18T03:17:21.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.212+0000 7f9a68e9cb80 1 -- 172.21.15.101:0/2994613474 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575b7b632d0 msgr2=0x5575b7b6e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:21.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.212+0000 7f9a68e9cb80 1 --2- 172.21.15.101:0/2994613474 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575b7b632d0 0x5575b7b6e520 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9a400ca5d0 tx=0x7f9a40099e20 comp rx=0 tx=0).stop 2023-07-18T03:17:21.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.212+0000 7f9a68e9cb80 1 -- 172.21.15.101:0/2994613474 shutdown_connections 2023-07-18T03:17:21.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.212+0000 7f9a68e9cb80 1 --2- 172.21.15.101:0/2994613474 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5575b7b7de30 0x5575b7b80240 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:21.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.212+0000 7f9a68e9cb80 1 --2- 172.21.15.101:0/2994613474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b7b6ead0 0x5575b7b6be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:21.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.212+0000 7f9a68e9cb80 1 --2- 172.21.15.101:0/2994613474 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575b7b632d0 0x5575b7b6e520 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:21.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.212+0000 7f9a32ff5700 0 client.27601 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:17:21.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.212+0000 7f9a68e9cb80 1 --2- 172.21.15.101:0/2994613474 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5575b7b63be0 0x5575b7b6b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:21.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.212+0000 7f9a68e9cb80 1 -- 172.21.15.101:0/2994613474 >> 172.21.15.101:0/2994613474 conn(0x5575b7b59da0 msgr2=0x5575b7b5f230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:21.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.212+0000 7f9a68e9cb80 1 -- 172.21.15.101:0/2994613474 shutdown_connections 2023-07-18T03:17:21.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.212+0000 7f9a68e9cb80 1 -- 172.21.15.101:0/2994613474 wait complete. 2023-07-18T03:17:21.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.212+0000 7f9a68e9cb80 20 client.27601 trim_cache size 0 max 16384 2023-07-18T03:17:21.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.227+0000 7f9a68e9cb80 1 Processor -- start 2023-07-18T03:17:21.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.227+0000 7f9a68e9cb80 1 -- start start 2023-07-18T03:17:21.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.227+0000 7f9a68e9cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b7b632d0 0x5575b7b6e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:21.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.227+0000 7f9a68e9cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5575b7b63be0 0x5575b7b6afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:21.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.227+0000 7f9a68e9cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575b7b6ead0 0x5575b7b6b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:21.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.227+0000 7f9a68e9cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b7b64540 con 0x5575b7b63be0 2023-07-18T03:17:21.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.227+0000 7f9a68e9cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b7b63a20 con 0x5575b7b632d0 2023-07-18T03:17:21.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.227+0000 7f9a68e9cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b7b64160 con 0x5575b7b6ead0 2023-07-18T03:17:21.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.227+0000 7f9a4effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b7b632d0 0x5575b7b6e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:21.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.228+0000 7f9a4effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b7b632d0 0x5575b7b6e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60654/0 (socket says 172.21.15.101:60654) 2023-07-18T03:17:21.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.228+0000 7f9a4effd700 1 -- 172.21.15.101:0/3445727261 learned_addr learned my addr 172.21.15.101:0/3445727261 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:21.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.228+0000 7f9a4e7fc700 1 --2- 172.21.15.101:0/3445727261 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5575b7b63be0 0x5575b7b6afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:21.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.228+0000 7f9a4f7fe700 1 --2- 172.21.15.101:0/3445727261 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575b7b6ead0 0x5575b7b6b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:21.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.228+0000 7f9a4effd700 1 -- 172.21.15.101:0/3445727261 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575b7b6ead0 msgr2=0x5575b7b6b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:21.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.228+0000 7f9a4effd700 1 --2- 172.21.15.101:0/3445727261 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575b7b6ead0 0x5575b7b6b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:21.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.228+0000 7f9a4effd700 1 -- 172.21.15.101:0/3445727261 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5575b7b63be0 msgr2=0x5575b7b6afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:21.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.228+0000 7f9a4effd700 1 --2- 172.21.15.101:0/3445727261 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5575b7b63be0 0x5575b7b6afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:21.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.228+0000 7f9a4effd700 1 -- 172.21.15.101:0/3445727261 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575b78ccc60 con 0x5575b7b632d0 2023-07-18T03:17:21.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.228+0000 7f9a4f7fe700 1 --2- 172.21.15.101:0/3445727261 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575b7b6ead0 0x5575b7b6b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:17:21.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.228+0000 7f9a4effd700 1 --2- 172.21.15.101:0/3445727261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b7b632d0 0x5575b7b6e050 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9a400999f0 tx=0x7f9a400984d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb972f6316f66ae0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:21.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.228+0000 7f9a35ffb700 1 -- 172.21.15.101:0/3445727261 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4009a180 con 0x5575b7b632d0 2023-07-18T03:17:21.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.228+0000 7f9a35ffb700 1 -- 172.21.15.101:0/3445727261 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a4009a3e0 con 0x5575b7b632d0 2023-07-18T03:17:21.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.228+0000 7f9a35ffb700 1 -- 172.21.15.101:0/3445727261 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a400ca820 con 0x5575b7b632d0 2023-07-18T03:17:21.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.229+0000 7f9a68e9cb80 1 -- 172.21.15.101:0/3445727261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b7b632d0 msgr2=0x5575b7b6e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:21.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.229+0000 7f9a68e9cb80 1 --2- 172.21.15.101:0/3445727261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b7b632d0 0x5575b7b6e050 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9a400999f0 tx=0x7f9a400984d0 comp rx=0 tx=0).stop 2023-07-18T03:17:21.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.229+0000 7f9a68e9cb80 1 -- 172.21.15.101:0/3445727261 shutdown_connections 2023-07-18T03:17:21.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.229+0000 7f9a68e9cb80 1 --2- 172.21.15.101:0/3445727261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b7b632d0 0x5575b7b6e050 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:21.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.229+0000 7f9a68e9cb80 1 --2- 172.21.15.101:0/3445727261 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575b7b6ead0 0x5575b7b6b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:21.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.229+0000 7f9a68e9cb80 1 --2- 172.21.15.101:0/3445727261 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5575b7b63be0 0x5575b7b6afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:21.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.229+0000 7f9a68e9cb80 1 -- 172.21.15.101:0/3445727261 >> 172.21.15.101:0/3445727261 conn(0x5575b7b59da0 msgr2=0x5575b7b7a700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:21.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.229+0000 7f9a68e9cb80 1 -- 172.21.15.101:0/3445727261 shutdown_connections 2023-07-18T03:17:21.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.229+0000 7f9a68e9cb80 1 -- 172.21.15.101:0/3445727261 wait complete. 2023-07-18T03:17:21.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.230+0000 7f9a68e9cb80 1 -- start start 2023-07-18T03:17:21.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.230+0000 7f9a68e9cb80 1 Processor -- start 2023-07-18T03:17:21.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.230+0000 7f9a68e9cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b7b632d0 0x5575b7b74e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:21.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.230+0000 7f9a68e9cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575b7b63be0 0x5575b7b75340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:21.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.230+0000 7f9a68e9cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5575b7b6ead0 0x5575b7b73490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:21.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.230+0000 7f9a68e9cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b7b73a20 con 0x5575b7b6ead0 2023-07-18T03:17:21.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.230+0000 7f9a68e9cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b7b73b90 con 0x5575b7b632d0 2023-07-18T03:17:21.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.230+0000 7f9a68e9cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b7b73d50 con 0x5575b7b63be0 2023-07-18T03:17:21.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.230+0000 7f9a68e9cb80 10 client.0 _ll_register_callbacks cb 0x7f9a44009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:17:21.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.231+0000 7f9a4e7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575b7b63be0 0x5575b7b75340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:21.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.231+0000 7f9a4effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b7b632d0 0x5575b7b74e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:21.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.231+0000 7f9a4effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b7b632d0 0x5575b7b74e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60670/0 (socket says 172.21.15.101:60670) 2023-07-18T03:17:21.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.231+0000 7f9a4e7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575b7b63be0 0x5575b7b75340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:44266/0 (socket says 172.21.15.101:44266) 2023-07-18T03:17:21.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.231+0000 7f9a4effd700 1 -- 172.21.15.101:0/4246215954 learned_addr learned my addr 172.21.15.101:0/4246215954 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:21.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.231+0000 7f9a4f7fe700 1 --2- 172.21.15.101:0/4246215954 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5575b7b6ead0 0x5575b7b73490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:21.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.231+0000 7f9a4e7fc700 1 -- 172.21.15.101:0/4246215954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b7b632d0 msgr2=0x5575b7b74e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:21.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.231+0000 7f9a4e7fc700 1 --2- 172.21.15.101:0/4246215954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b7b632d0 0x5575b7b74e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:21.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.231+0000 7f9a4e7fc700 1 -- 172.21.15.101:0/4246215954 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5575b7b6ead0 msgr2=0x5575b7b73490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:21.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.231+0000 7f9a4e7fc700 1 --2- 172.21.15.101:0/4246215954 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5575b7b6ead0 0x5575b7b73490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:21.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.231+0000 7f9a4e7fc700 1 -- 172.21.15.101:0/4246215954 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a501a4d80 con 0x5575b7b63be0 2023-07-18T03:17:21.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.231+0000 7f9a4effd700 1 --2- 172.21.15.101:0/4246215954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b7b632d0 0x5575b7b74e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:17:21.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.231+0000 7f9a4e7fc700 1 --2- 172.21.15.101:0/4246215954 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575b7b63be0 0x5575b7b75340 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x5575b7b80ad0 tx=0x7f9a4401fd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc3a746c527d01b9 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:21.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.231+0000 7f9a32ff5700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:17:21.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.232+0000 7f9a32ff5700 1 -- 172.21.15.101:0/4246215954 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a44010ce0 con 0x5575b7b63be0 2023-07-18T03:17:21.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.232+0000 7f9a68e9cb80 1 -- 172.21.15.101:0/4246215954 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9a40099880 con 0x5575b7b63be0 2023-07-18T03:17:21.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.232+0000 7f9a32ff5700 1 -- 172.21.15.101:0/4246215954 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f9a44010e40 con 0x5575b7b63be0 2023-07-18T03:17:21.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.232+0000 7f9a32ff5700 1 -- 172.21.15.101:0/4246215954 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a44004830 con 0x5575b7b63be0 2023-07-18T03:17:21.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.232+0000 7f9a68e9cb80 10 client.27607 fetch_fsmap learned FSMap version 33 2023-07-18T03:17:21.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.232+0000 7f9a68e9cb80 1 -- 172.21.15.101:0/4246215954 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5575b790a250 con 0x5575b7b63be0 2023-07-18T03:17:21.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.232+0000 7f9a32ff5700 1 -- 172.21.15.101:0/4246215954 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a40099880 con 0x5575b7b63be0 2023-07-18T03:17:21.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.232+0000 7f9a32ff5700 1 -- 172.21.15.101:0/4246215954 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9a4401d480 con 0x5575b7b63be0 2023-07-18T03:17:21.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.232+0000 7f9a68e9cb80 10 client.27607 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:17:21.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.233+0000 7f9a68e9cb80 10 client.27607 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:17:21.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.233+0000 7f9a68e9cb80 20 client.27607 populate_metadata read hostname 'smithi101' 2023-07-18T03:17:21.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.233+0000 7f9a68e9cb80 1 --2- 172.21.15.101:0/4246215954 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5575b7bd2f20 0x5575b7bd5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:21.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.233+0000 7f9a68e9cb80 4 client.27607 mds_command: new command op to 24562 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:17:21.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.233+0000 7f9a68e9cb80 1 -- 172.21.15.101:0/4246215954 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5575b7a71050 con 0x5575b7bd2f20 2023-07-18T03:17:21.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.233+0000 7f9a4effd700 1 --2- 172.21.15.101:0/4246215954 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5575b7bd2f20 0x5575b7bd5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:21.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.233+0000 7f9a4effd700 1 --2- 172.21.15.101:0/4246215954 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5575b7bd2f20 0x5575b7bd5330 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23fb49f27d7b1fce server_cookie=8653843110cb9e7e in_seq=0 out_seq=0 2023-07-18T03:17:21.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.233+0000 7f9a32ff5700 10 client.27607 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:17:21.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.233+0000 7f9a32ff5700 1 -- 172.21.15.101:0/4246215954 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5575b7a71050 con 0x5575b7bd2f20 2023-07-18T03:17:21.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.233+0000 7f9a32ff5700 10 client.27607 handle_command_reply: tid=0 2023-07-18T03:17:21.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.233+0000 7f9a68e9cb80 1 client.27607 shutdown 2023-07-18T03:17:21.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.233+0000 7f9a68e9cb80 1 -- 172.21.15.101:0/4246215954 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575b7b63be0 msgr2=0x5575b7b75340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:21.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.234+0000 7f9a68e9cb80 1 --2- 172.21.15.101:0/4246215954 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575b7b63be0 0x5575b7b75340 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x5575b7b80ad0 tx=0x7f9a4401fd20 comp rx=0 tx=0).stop 2023-07-18T03:17:21.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.234+0000 7f9a68e9cb80 1 -- 172.21.15.101:0/4246215954 shutdown_connections 2023-07-18T03:17:21.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.234+0000 7f9a68e9cb80 1 --2- 172.21.15.101:0/4246215954 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5575b7bd2f20 0x5575b7bd5330 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:21.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.234+0000 7f9a68e9cb80 1 --2- 172.21.15.101:0/4246215954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5575b7b632d0 0x5575b7b74e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:21.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.234+0000 7f9a68e9cb80 1 --2- 172.21.15.101:0/4246215954 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5575b7b6ead0 0x5575b7b73490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:21.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.234+0000 7f9a68e9cb80 1 --2- 172.21.15.101:0/4246215954 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5575b7b63be0 0x5575b7b75340 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:21.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.234+0000 7f9a32ff5700 0 client.27607 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:17:21.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.234+0000 7f9a68e9cb80 1 -- 172.21.15.101:0/4246215954 >> 172.21.15.101:0/4246215954 conn(0x5575b7b59da0 msgr2=0x5575b7b651a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:21.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.234+0000 7f9a68e9cb80 1 -- 172.21.15.101:0/4246215954 shutdown_connections 2023-07-18T03:17:21.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.234+0000 7f9a68e9cb80 1 -- 172.21.15.101:0/4246215954 wait complete. 2023-07-18T03:17:21.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.234+0000 7f9a68e9cb80 20 client.27607 trim_cache size 0 max 16384 2023-07-18T03:17:21.238 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:17:21.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.234+0000 7f9a555ff700 1 -- 172.21.15.101:0/3295486070 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9a380a12c0 msgr2=0x7f9a380a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:21.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.234+0000 7f9a555ff700 1 --2- 172.21.15.101:0/3295486070 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9a380a12c0 0x7f9a380a3770 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9a4800b0a0 tx=0x7f9a480099e0 comp rx=0 tx=0).stop 2023-07-18T03:17:21.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.234+0000 7f9a555ff700 1 -- 172.21.15.101:0/3295486070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a500ffae0 msgr2=0x7f9a50197f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:21.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.234+0000 7f9a555ff700 1 --2- 172.21.15.101:0/3295486070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a500ffae0 0x7f9a50197f50 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9a4000e9a0 tx=0x7f9a4000ecb0 comp rx=0 tx=0).stop 2023-07-18T03:17:21.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.235+0000 7f9a555ff700 1 -- 172.21.15.101:0/3295486070 shutdown_connections 2023-07-18T03:17:21.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.235+0000 7f9a555ff700 1 --2- 172.21.15.101:0/3295486070 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9a380a12c0 0x7f9a380a3770 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:21.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.235+0000 7f9a555ff700 1 --2- 172.21.15.101:0/3295486070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a500ffae0 0x7f9a50197f50 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:21.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.235+0000 7f9a555ff700 1 --2- 172.21.15.101:0/3295486070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a50105af0 0x7f9a5019c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:21.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.235+0000 7f9a555ff700 1 --2- 172.21.15.101:0/3295486070 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9a50100490 0x7f9a50198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:21.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.235+0000 7f9a555ff700 1 -- 172.21.15.101:0/3295486070 >> 172.21.15.101:0/3295486070 conn(0x7f9a500fb5e0 msgr2=0x7f9a5010ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:21.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.235+0000 7f9a555ff700 1 -- 172.21.15.101:0/3295486070 shutdown_connections 2023-07-18T03:17:21.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:21.235+0000 7f9a555ff700 1 -- 172.21.15.101:0/3295486070 wait complete. 2023-07-18T03:17:21.241 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:20 smithi101 ceph-mon[126379]: pgmap v304: 129 pgs: 129 active+clean; 480 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 133 op/s 2023-07-18T03:17:21.241 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:20 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2146570879' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:17:21.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:20 smithi155 ceph-mon[128885]: pgmap v304: 129 pgs: 129 active+clean; 480 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 133 op/s 2023-07-18T03:17:21.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:20 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/2146570879' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:17:21.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:20 smithi179 ceph-mon[129229]: pgmap v304: 129 pgs: 129 active+clean; 480 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 133 op/s 2023-07-18T03:17:21.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:20 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/2146570879' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:17:21.611 INFO:tasks.workunit.client.0.smithi101.stdout: 1 626511 5.96 MB/sec execute 406 sec latency 19.401 ms 2023-07-18T03:17:22.254 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-18T03:17:22.255 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-18T03:17:22.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.465+0000 7fd5dfbff700 1 -- 172.21.15.101:0/2099682647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5d8100480 msgr2=0x7fd5d810c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:22.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.465+0000 7fd5dfbff700 1 --2- 172.21.15.101:0/2099682647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5d8100480 0x7fd5d810c910 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd5d000b0a0 tx=0x7fd5d000b3b0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.465+0000 7fd5dfbff700 1 -- 172.21.15.101:0/2099682647 shutdown_connections 2023-07-18T03:17:22.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.465+0000 7fd5dfbff700 1 --2- 172.21.15.101:0/2099682647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5d8105ae0 0x7fd5d8105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.465+0000 7fd5dfbff700 1 --2- 172.21.15.101:0/2099682647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5d8100480 0x7fd5d810c910 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.465+0000 7fd5dfbff700 1 --2- 172.21.15.101:0/2099682647 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd5d80ffad0 0x7fd5d80fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.465+0000 7fd5dfbff700 1 -- 172.21.15.101:0/2099682647 >> 172.21.15.101:0/2099682647 conn(0x7fd5d80fb5f0 msgr2=0x7fd5d80fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:22.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.465+0000 7fd5dfbff700 1 -- 172.21.15.101:0/2099682647 shutdown_connections 2023-07-18T03:17:22.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.466+0000 7fd5dfbff700 1 -- 172.21.15.101:0/2099682647 wait complete. 2023-07-18T03:17:22.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.467+0000 7fd5dfbff700 1 Processor -- start 2023-07-18T03:17:22.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.467+0000 7fd5dfbff700 1 -- start start 2023-07-18T03:17:22.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.468+0000 7fd5dfbff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd5d80ffad0 0x7fd5d819bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:22.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.468+0000 7fd5dfbff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5d8100480 0x7fd5d819c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:22.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.468+0000 7fd5dfbff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5d8105ae0 0x7fd5d8195b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:22.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.468+0000 7fd5dfbff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d81961c0 con 0x7fd5d80ffad0 2023-07-18T03:17:22.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.468+0000 7fd5dfbff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d8196330 con 0x7fd5d8105ae0 2023-07-18T03:17:22.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.468+0000 7fd5dfbff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d81964f0 con 0x7fd5d8100480 2023-07-18T03:17:22.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.468+0000 7fd5dd99b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd5d80ffad0 0x7fd5d819bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:22.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.468+0000 7fd5dd99b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd5d80ffad0 0x7fd5d819bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:46058/0 (socket says 172.21.15.101:46058) 2023-07-18T03:17:22.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.468+0000 7fd5dd99b700 1 -- 172.21.15.101:0/4202171362 learned_addr learned my addr 172.21.15.101:0/4202171362 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:22.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.468+0000 7fd5dd19a700 1 --2- 172.21.15.101:0/4202171362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5d8100480 0x7fd5d819c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:22.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.468+0000 7fd5de19c700 1 --2- 172.21.15.101:0/4202171362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5d8105ae0 0x7fd5d8195b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:22.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.468+0000 7fd5dd99b700 1 -- 172.21.15.101:0/4202171362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5d8100480 msgr2=0x7fd5d819c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:22.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.468+0000 7fd5dd99b700 1 --2- 172.21.15.101:0/4202171362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5d8100480 0x7fd5d819c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.468+0000 7fd5dd99b700 1 -- 172.21.15.101:0/4202171362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5d8105ae0 msgr2=0x7fd5d8195b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:22.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.468+0000 7fd5dd99b700 1 --2- 172.21.15.101:0/4202171362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5d8105ae0 0x7fd5d8195b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.468+0000 7fd5dd99b700 1 -- 172.21.15.101:0/4202171362 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c8009520 con 0x7fd5d80ffad0 2023-07-18T03:17:22.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.469+0000 7fd5dd99b700 1 --2- 172.21.15.101:0/4202171362 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd5d80ffad0 0x7fd5d819bae0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fd5d400e9a0 tx=0x7fd5d400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:22.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.469+0000 7fd5ceffd700 1 -- 172.21.15.101:0/4202171362 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d4009790 con 0x7fd5d80ffad0 2023-07-18T03:17:22.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.469+0000 7fd5dfbff700 1 -- 172.21.15.101:0/4202171362 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5d0009d00 con 0x7fd5d80ffad0 2023-07-18T03:17:22.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.469+0000 7fd5ceffd700 1 -- 172.21.15.101:0/4202171362 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5d4010430 con 0x7fd5d80ffad0 2023-07-18T03:17:22.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.469+0000 7fd5ceffd700 1 -- 172.21.15.101:0/4202171362 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d4018680 con 0x7fd5d80ffad0 2023-07-18T03:17:22.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.469+0000 7fd5dfbff700 1 -- 172.21.15.101:0/4202171362 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5d81969c0 con 0x7fd5d80ffad0 2023-07-18T03:17:22.471 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.472+0000 7fd5ceffd700 1 -- 172.21.15.101:0/4202171362 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fd5d40105a0 con 0x7fd5d80ffad0 2023-07-18T03:17:22.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.473+0000 7fd5f345fb80 1 Processor -- start 2023-07-18T03:17:22.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.473+0000 7fd5ceffd700 1 --2- 172.21.15.101:0/4202171362 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd5c40a12c0 0x7fd5c40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:22.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.473+0000 7fd5f345fb80 1 -- start start 2023-07-18T03:17:22.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.473+0000 7fd5ceffd700 1 -- 172.21.15.101:0/4202171362 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fd5d4014070 con 0x7fd5d80ffad0 2023-07-18T03:17:22.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.473+0000 7fd5de19c700 1 --2- 172.21.15.101:0/4202171362 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd5c40a12c0 0x7fd5c40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:22.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.474+0000 7fd5f345fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff7d8c2350 0x55ff7d8c2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:22.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.474+0000 7fd5f345fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff7d8c2c60 0x55ff7d8d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:22.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.474+0000 7fd5f345fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff7d8d1f60 0x55ff7d8d4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:22.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.474+0000 7fd5f345fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff7d7c8d30 con 0x55ff7d8d1f60 2023-07-18T03:17:22.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.474+0000 7fd5f345fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff7d81bac0 con 0x55ff7d8c2350 2023-07-18T03:17:22.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.474+0000 7fd5f345fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff7d814f50 con 0x55ff7d8c2c60 2023-07-18T03:17:22.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.474+0000 7fd5dd99b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff7d8c2350 0x55ff7d8c2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:22.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.474+0000 7fd5dd99b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff7d8c2350 0x55ff7d8c2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60698/0 (socket says 172.21.15.101:60698) 2023-07-18T03:17:22.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.474+0000 7fd5dd99b700 1 -- 172.21.15.101:0/2975429473 learned_addr learned my addr 172.21.15.101:0/2975429473 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:22.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.474+0000 7fd5dd19a700 1 --2- 172.21.15.101:0/2975429473 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff7d8c2c60 0x55ff7d8d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:22.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.474+0000 7fd5de19c700 1 --2- 172.21.15.101:0/2975429473 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff7d8d1f60 0x55ff7d8d4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:22.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.474+0000 7fd5dd99b700 1 -- 172.21.15.101:0/2975429473 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff7d8c2c60 msgr2=0x55ff7d8d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:22.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.474+0000 7fd5dd99b700 1 --2- 172.21.15.101:0/2975429473 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff7d8c2c60 0x55ff7d8d1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.480 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.474+0000 7fd5dd99b700 1 -- 172.21.15.101:0/2975429473 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff7d8d1f60 msgr2=0x55ff7d8d4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:22.480 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.474+0000 7fd5dd99b700 1 --2- 172.21.15.101:0/2975429473 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff7d8d1f60 0x55ff7d8d4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.480 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.474+0000 7fd5dd99b700 1 -- 172.21.15.101:0/2975429473 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff7d642e60 con 0x55ff7d8c2350 2023-07-18T03:17:22.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.475+0000 7fd5de19c700 1 --2- 172.21.15.101:0/4202171362 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd5c40a12c0 0x7fd5c40a3770 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd5d000b780 tx=0x7fd5d00095c0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:22.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.475+0000 7fd5dd99b700 1 --2- 172.21.15.101:0/2975429473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff7d8c2350 0x55ff7d8c2720 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4096800 tx=0x7fd5d40997e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e6ec0d178af51ed server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:22.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.475+0000 7fd5c3fff700 1 -- 172.21.15.101:0/2975429473 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d40a21e0 con 0x55ff7d8c2350 2023-07-18T03:17:22.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.475+0000 7fd5c3fff700 1 -- 172.21.15.101:0/2975429473 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5d4010880 con 0x55ff7d8c2350 2023-07-18T03:17:22.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.475+0000 7fd5c3fff700 1 -- 172.21.15.101:0/2975429473 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d40a1d50 con 0x55ff7d8c2350 2023-07-18T03:17:22.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.476+0000 7fd5f345fb80 1 -- 172.21.15.101:0/2975429473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff7d8c2350 msgr2=0x55ff7d8c2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:22.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.476+0000 7fd5f345fb80 1 --2- 172.21.15.101:0/2975429473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff7d8c2350 0x55ff7d8c2720 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4096800 tx=0x7fd5d40997e0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.476+0000 7fd5f345fb80 1 -- 172.21.15.101:0/2975429473 shutdown_connections 2023-07-18T03:17:22.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.476+0000 7fd5f345fb80 1 --2- 172.21.15.101:0/2975429473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff7d8c2350 0x55ff7d8c2720 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.476+0000 7fd5f345fb80 1 --2- 172.21.15.101:0/2975429473 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff7d8d1f60 0x55ff7d8d4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.476+0000 7fd5f345fb80 1 --2- 172.21.15.101:0/2975429473 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff7d8c2c60 0x55ff7d8d1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.476+0000 7fd5f345fb80 1 -- 172.21.15.101:0/2975429473 >> 172.21.15.101:0/2975429473 conn(0x55ff7d8b8e20 msgr2=0x55ff7d8be720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:22.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.476+0000 7fd5f345fb80 1 -- 172.21.15.101:0/2975429473 shutdown_connections 2023-07-18T03:17:22.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.476+0000 7fd5f345fb80 1 -- 172.21.15.101:0/2975429473 wait complete. 2023-07-18T03:17:22.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.476+0000 7fd5f345fb80 1 -- start start 2023-07-18T03:17:22.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.476+0000 7fd5f345fb80 1 Processor -- start 2023-07-18T03:17:22.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.477+0000 7fd5f345fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff7d8c2350 0x55ff7d8ced20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:22.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.477+0000 7fd5f345fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff7d8c2c60 0x55ff7d8cf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:22.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.477+0000 7fd5f345fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff7d8d1f60 0x55ff7d8cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:22.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.477+0000 7fd5f345fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff7d7b4b40 con 0x55ff7d8d1f60 2023-07-18T03:17:22.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.477+0000 7fd5f345fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff7d7b4840 con 0x55ff7d8c2c60 2023-07-18T03:17:22.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.477+0000 7fd5f345fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff7d6989a0 con 0x55ff7d8c2350 2023-07-18T03:17:22.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.477+0000 7fd5f345fb80 10 client.0 _ll_register_callbacks cb 0x55ff7d7b27e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:17:22.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.477+0000 7fd5de19c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff7d8d1f60 0x55ff7d8cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:22.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.477+0000 7fd5de19c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff7d8d1f60 0x55ff7d8cf7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:46076/0 (socket says 172.21.15.101:46076) 2023-07-18T03:17:22.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.477+0000 7fd5de19c700 1 -- 172.21.15.101:0/3371378262 learned_addr learned my addr 172.21.15.101:0/3371378262 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:22.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.477+0000 7fd5dd19a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff7d8c2c60 0x55ff7d8cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:22.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.477+0000 7fd5dd99b700 1 --2- 172.21.15.101:0/3371378262 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff7d8c2350 0x55ff7d8ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:22.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.478+0000 7fd5dd19a700 1 -- 172.21.15.101:0/3371378262 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff7d8c2350 msgr2=0x55ff7d8ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:22.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.478+0000 7fd5dd19a700 1 --2- 172.21.15.101:0/3371378262 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff7d8c2350 0x55ff7d8ced20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.478+0000 7fd5dd19a700 1 -- 172.21.15.101:0/3371378262 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff7d8d1f60 msgr2=0x55ff7d8cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:22.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.478+0000 7fd5dd19a700 1 --2- 172.21.15.101:0/3371378262 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff7d8d1f60 0x55ff7d8cf7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.478+0000 7fd5dd19a700 1 -- 172.21.15.101:0/3371378262 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5d0009d00 con 0x55ff7d8c2c60 2023-07-18T03:17:22.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.478+0000 7fd5dd19a700 1 --2- 172.21.15.101:0/3371378262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff7d8c2c60 0x55ff7d8cf260 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd5c80127f0 tx=0x7fd5c8012b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df1e1385e2b2ac16 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:22.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.478+0000 7fd5c17fa700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:17:22.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.478+0000 7fd5c17fa700 1 -- 172.21.15.101:0/3371378262 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c8017070 con 0x55ff7d8c2c60 2023-07-18T03:17:22.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.478+0000 7fd5f345fb80 1 -- 172.21.15.101:0/3371378262 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ff7d76e570 con 0x55ff7d8c2c60 2023-07-18T03:17:22.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.478+0000 7fd5c17fa700 1 -- 172.21.15.101:0/3371378262 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fd5c800f4e0 con 0x55ff7d8c2c60 2023-07-18T03:17:22.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.478+0000 7fd5c17fa700 1 -- 172.21.15.101:0/3371378262 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5c800f6a0 con 0x55ff7d8c2c60 2023-07-18T03:17:22.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.478+0000 7fd5f345fb80 10 client.27584 fetch_fsmap learned FSMap version 33 2023-07-18T03:17:22.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.478+0000 7fd5f345fb80 1 -- 172.21.15.101:0/3371378262 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55ff7d62bce0 con 0x55ff7d8c2c60 2023-07-18T03:17:22.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.479+0000 7fd5c17fa700 1 -- 172.21.15.101:0/3371378262 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ff7d76e570 con 0x55ff7d8c2c60 2023-07-18T03:17:22.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.479+0000 7fd5c17fa700 1 -- 172.21.15.101:0/3371378262 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd5c8010820 con 0x55ff7d8c2c60 2023-07-18T03:17:22.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.479+0000 7fd5f345fb80 10 client.27584 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:17:22.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.479+0000 7fd5f345fb80 10 client.27584 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:17:22.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.479+0000 7fd5f345fb80 20 client.27584 populate_metadata read hostname 'smithi101' 2023-07-18T03:17:22.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.479+0000 7fd5f345fb80 1 --2- 172.21.15.101:0/3371378262 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ff7d8e19e0 0x55ff7d8e3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:22.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.479+0000 7fd5f345fb80 4 client.27584 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:17:22.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.479+0000 7fd5f345fb80 1 -- 172.21.15.101:0/3371378262 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff7d80df30 con 0x55ff7d8e19e0 2023-07-18T03:17:22.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.480+0000 7fd5dd99b700 1 --2- 172.21.15.101:0/3371378262 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ff7d8e19e0 0x55ff7d8e3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:22.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.480+0000 7fd5c17fa700 10 client.27584 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:17:22.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.480+0000 7fd5dd99b700 1 --2- 172.21.15.101:0/3371378262 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ff7d8e19e0 0x55ff7d8e3df0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a4f536e32a21345 server_cookie=7187b4ee42d21524 in_seq=0 out_seq=0 2023-07-18T03:17:22.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.481+0000 7fd5c17fa700 1 -- 172.21.15.101:0/3371378262 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ff7d80df30 con 0x55ff7d8e19e0 2023-07-18T03:17:22.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.481+0000 7fd5c17fa700 10 client.27584 handle_command_reply: tid=0 2023-07-18T03:17:22.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.481+0000 7fd5f345fb80 1 client.27584 shutdown 2023-07-18T03:17:22.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.481+0000 7fd5f345fb80 1 -- 172.21.15.101:0/3371378262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff7d8c2c60 msgr2=0x55ff7d8cf260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:22.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.481+0000 7fd5f345fb80 1 --2- 172.21.15.101:0/3371378262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff7d8c2c60 0x55ff7d8cf260 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd5c80127f0 tx=0x7fd5c8012b00 comp rx=0 tx=0).stop 2023-07-18T03:17:22.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.481+0000 7fd5f345fb80 1 -- 172.21.15.101:0/3371378262 shutdown_connections 2023-07-18T03:17:22.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.481+0000 7fd5f345fb80 1 --2- 172.21.15.101:0/3371378262 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ff7d8e19e0 0x55ff7d8e3df0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.481+0000 7fd5f345fb80 1 --2- 172.21.15.101:0/3371378262 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff7d8d1f60 0x55ff7d8cf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.481+0000 7fd5f345fb80 1 --2- 172.21.15.101:0/3371378262 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff7d8c2350 0x55ff7d8ced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.481+0000 7fd5f345fb80 1 --2- 172.21.15.101:0/3371378262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff7d8c2c60 0x55ff7d8cf260 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.481+0000 7fd5c17fa700 0 client.27584 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:17:22.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.481+0000 7fd5f345fb80 1 -- 172.21.15.101:0/3371378262 >> 172.21.15.101:0/3371378262 conn(0x55ff7d8b8e20 msgr2=0x55ff7d8be2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:22.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.481+0000 7fd5f345fb80 1 -- 172.21.15.101:0/3371378262 shutdown_connections 2023-07-18T03:17:22.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.481+0000 7fd5f345fb80 1 -- 172.21.15.101:0/3371378262 wait complete. 2023-07-18T03:17:22.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.481+0000 7fd5f345fb80 20 client.27584 trim_cache size 0 max 16384 2023-07-18T03:17:22.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.498+0000 7fd5f345fb80 1 Processor -- start 2023-07-18T03:17:22.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.499+0000 7fd5f345fb80 1 -- start start 2023-07-18T03:17:22.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.499+0000 7fd5f345fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff7d8c2350 0x55ff7d8c4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:22.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.499+0000 7fd5f345fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff7d8c2c60 0x55ff7d8cbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:22.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.499+0000 7fd5f345fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff7d8d1f60 0x55ff7d8cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:22.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.499+0000 7fd5f345fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff7d76e570 con 0x55ff7d8c2350 2023-07-18T03:17:22.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.499+0000 7fd5f345fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff7d8bf360 con 0x55ff7d8c2c60 2023-07-18T03:17:22.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.499+0000 7fd5f345fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff7d8baf40 con 0x55ff7d8d1f60 2023-07-18T03:17:22.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.499+0000 7fd5dd99b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff7d8c2350 0x55ff7d8c4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:22.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.499+0000 7fd5dd99b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff7d8c2350 0x55ff7d8c4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:46082/0 (socket says 172.21.15.101:46082) 2023-07-18T03:17:22.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.499+0000 7fd5dd99b700 1 -- 172.21.15.101:0/1359411231 learned_addr learned my addr 172.21.15.101:0/1359411231 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:22.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.499+0000 7fd5de19c700 1 --2- 172.21.15.101:0/1359411231 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff7d8d1f60 0x55ff7d8cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:22.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.499+0000 7fd5dd19a700 1 --2- 172.21.15.101:0/1359411231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff7d8c2c60 0x55ff7d8cbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:22.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.499+0000 7fd5dd99b700 1 -- 172.21.15.101:0/1359411231 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff7d8d1f60 msgr2=0x55ff7d8cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:22.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.499+0000 7fd5dd99b700 1 --2- 172.21.15.101:0/1359411231 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff7d8d1f60 0x55ff7d8cc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.499+0000 7fd5dd99b700 1 -- 172.21.15.101:0/1359411231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff7d8c2c60 msgr2=0x55ff7d8cbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:22.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.499+0000 7fd5dd99b700 1 --2- 172.21.15.101:0/1359411231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff7d8c2c60 0x55ff7d8cbf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.499+0000 7fd5dd99b700 1 -- 172.21.15.101:0/1359411231 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff7d8ce890 con 0x55ff7d8c2350 2023-07-18T03:17:22.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.500+0000 7fd5dd99b700 1 --2- 172.21.15.101:0/1359411231 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff7d8c2350 0x55ff7d8c4a70 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd5d400e970 tx=0x7fd5d409df40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a97de2c3e97093b4 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:22.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.500+0000 7fd5ccbff700 1 -- 172.21.15.101:0/1359411231 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d40b81c0 con 0x55ff7d8c2350 2023-07-18T03:17:22.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.500+0000 7fd5ccbff700 1 -- 172.21.15.101:0/1359411231 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5d4099ff0 con 0x55ff7d8c2350 2023-07-18T03:17:22.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.500+0000 7fd5ccbff700 1 -- 172.21.15.101:0/1359411231 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d40cab60 con 0x55ff7d8c2350 2023-07-18T03:17:22.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.500+0000 7fd5f345fb80 1 -- 172.21.15.101:0/1359411231 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff7d8c2350 msgr2=0x55ff7d8c4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:22.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.500+0000 7fd5f345fb80 1 --2- 172.21.15.101:0/1359411231 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff7d8c2350 0x55ff7d8c4a70 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd5d400e970 tx=0x7fd5d409df40 comp rx=0 tx=0).stop 2023-07-18T03:17:22.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.501+0000 7fd5f345fb80 1 -- 172.21.15.101:0/1359411231 shutdown_connections 2023-07-18T03:17:22.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.501+0000 7fd5f345fb80 1 --2- 172.21.15.101:0/1359411231 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff7d8d1f60 0x55ff7d8cc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.501+0000 7fd5f345fb80 1 --2- 172.21.15.101:0/1359411231 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff7d8c2350 0x55ff7d8c4a70 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.501+0000 7fd5f345fb80 1 --2- 172.21.15.101:0/1359411231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff7d8c2c60 0x55ff7d8cbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.501+0000 7fd5f345fb80 1 -- 172.21.15.101:0/1359411231 >> 172.21.15.101:0/1359411231 conn(0x55ff7d8b8e20 msgr2=0x55ff7d8c4360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:22.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.501+0000 7fd5f345fb80 1 -- 172.21.15.101:0/1359411231 shutdown_connections 2023-07-18T03:17:22.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.501+0000 7fd5f345fb80 1 -- 172.21.15.101:0/1359411231 wait complete. 2023-07-18T03:17:22.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.501+0000 7fd5f345fb80 1 -- start start 2023-07-18T03:17:22.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.501+0000 7fd5f345fb80 1 Processor -- start 2023-07-18T03:17:22.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.502+0000 7fd5f345fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff7d8c2350 0x55ff7d8d8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:22.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.502+0000 7fd5f345fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff7d8c2c60 0x55ff7d8d6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:22.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.502+0000 7fd5f345fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff7d8d1f60 0x55ff7d8d7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:22.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.502+0000 7fd5f345fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff7d8d7a70 con 0x55ff7d8d1f60 2023-07-18T03:17:22.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.502+0000 7fd5f345fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff7d8d7be0 con 0x55ff7d8c2c60 2023-07-18T03:17:22.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.502+0000 7fd5f345fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff7d935870 con 0x55ff7d8c2350 2023-07-18T03:17:22.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.502+0000 7fd5f345fb80 10 client.0 _ll_register_callbacks cb 0x7fd5c8009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:17:22.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.502+0000 7fd5dd99b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff7d8c2350 0x55ff7d8d8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:22.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.502+0000 7fd5dd99b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff7d8c2350 0x55ff7d8d8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:44298/0 (socket says 172.21.15.101:44298) 2023-07-18T03:17:22.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.502+0000 7fd5dd99b700 1 -- 172.21.15.101:0/2995425818 learned_addr learned my addr 172.21.15.101:0/2995425818 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:22.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.502+0000 7fd5de19c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff7d8d1f60 0x55ff7d8d7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:22.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.502+0000 7fd5dd19a700 1 --2- 172.21.15.101:0/2995425818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff7d8c2c60 0x55ff7d8d6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:22.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.502+0000 7fd5dd99b700 1 -- 172.21.15.101:0/2995425818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff7d8c2c60 msgr2=0x55ff7d8d6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:22.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.503+0000 7fd5dd99b700 1 --2- 172.21.15.101:0/2995425818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff7d8c2c60 0x55ff7d8d6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.503+0000 7fd5dd99b700 1 -- 172.21.15.101:0/2995425818 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff7d8d1f60 msgr2=0x55ff7d8d7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:22.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.503+0000 7fd5dd99b700 1 --2- 172.21.15.101:0/2995425818 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff7d8d1f60 0x55ff7d8d7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.503+0000 7fd5dd99b700 1 -- 172.21.15.101:0/2995425818 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff7d642e60 con 0x55ff7d8c2350 2023-07-18T03:17:22.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.503+0000 7fd5dd99b700 1 --2- 172.21.15.101:0/2995425818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff7d8c2350 0x55ff7d8d8a70 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd5d40a1cb0 tx=0x7fd5d4099500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90f48df70b58cea1 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:22.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.503+0000 7fd5c17fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:17:22.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.503+0000 7fd5c17fa700 1 -- 172.21.15.101:0/2995425818 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d409a160 con 0x55ff7d8c2350 2023-07-18T03:17:22.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.503+0000 7fd5c17fa700 1 -- 172.21.15.101:0/2995425818 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fd5d40a1d50 con 0x55ff7d8c2350 2023-07-18T03:17:22.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.503+0000 7fd5f345fb80 1 -- 172.21.15.101:0/2995425818 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd5d4010880 con 0x55ff7d8c2350 2023-07-18T03:17:22.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.503+0000 7fd5c17fa700 1 -- 172.21.15.101:0/2995425818 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5d409a2c0 con 0x55ff7d8c2350 2023-07-18T03:17:22.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.503+0000 7fd5f345fb80 10 client.27625 fetch_fsmap learned FSMap version 33 2023-07-18T03:17:22.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.503+0000 7fd5f345fb80 1 -- 172.21.15.101:0/2995425818 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55ff7d6692d0 con 0x55ff7d8c2350 2023-07-18T03:17:22.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.503+0000 7fd5c17fa700 1 -- 172.21.15.101:0/2995425818 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5d407c020 con 0x55ff7d8c2350 2023-07-18T03:17:22.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.503+0000 7fd5c17fa700 1 -- 172.21.15.101:0/2995425818 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd5d4095b80 con 0x55ff7d8c2350 2023-07-18T03:17:22.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.504+0000 7fd5f345fb80 10 client.27625 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:17:22.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.504+0000 7fd5f345fb80 10 client.27625 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:17:22.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.504+0000 7fd5f345fb80 20 client.27625 populate_metadata read hostname 'smithi101' 2023-07-18T03:17:22.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.504+0000 7fd5f345fb80 1 --2- 172.21.15.101:0/2995425818 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ff7d936b00 0x55ff7d938ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:22.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.504+0000 7fd5f345fb80 4 client.27625 mds_command: new command op to 24562 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-18T03:17:22.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.504+0000 7fd5f345fb80 1 -- 172.21.15.101:0/2995425818 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55ff7d7d00d0 con 0x55ff7d936b00 2023-07-18T03:17:22.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.504+0000 7fd5dd19a700 1 --2- 172.21.15.101:0/2995425818 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ff7d936b00 0x55ff7d938ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:22.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.504+0000 7fd5c17fa700 10 client.27625 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:17:22.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.504+0000 7fd5dd19a700 1 --2- 172.21.15.101:0/2995425818 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ff7d936b00 0x55ff7d938ee0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f1e903cc2a6e49 server_cookie=37a9fd5093c4272a in_seq=0 out_seq=0 2023-07-18T03:17:22.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.506+0000 7fd5c17fa700 1 -- 172.21.15.101:0/2995425818 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ff7d7d00d0 con 0x55ff7d936b00 2023-07-18T03:17:22.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.506+0000 7fd5c17fa700 10 client.27625 handle_command_reply: tid=0 2023-07-18T03:17:22.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.506+0000 7fd5f345fb80 1 client.27625 shutdown 2023-07-18T03:17:22.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.506+0000 7fd5f345fb80 1 -- 172.21.15.101:0/2995425818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff7d8c2350 msgr2=0x55ff7d8d8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:22.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.506+0000 7fd5f345fb80 1 --2- 172.21.15.101:0/2995425818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff7d8c2350 0x55ff7d8d8a70 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd5d40a1cb0 tx=0x7fd5d4099500 comp rx=0 tx=0).stop 2023-07-18T03:17:22.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.506+0000 7fd5f345fb80 1 -- 172.21.15.101:0/2995425818 shutdown_connections 2023-07-18T03:17:22.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.506+0000 7fd5f345fb80 1 --2- 172.21.15.101:0/2995425818 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ff7d936b00 0x55ff7d938ee0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.506+0000 7fd5f345fb80 1 --2- 172.21.15.101:0/2995425818 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff7d8d1f60 0x55ff7d8d7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.506+0000 7fd5f345fb80 1 --2- 172.21.15.101:0/2995425818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff7d8c2350 0x55ff7d8d8a70 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.506+0000 7fd5f345fb80 1 --2- 172.21.15.101:0/2995425818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff7d8c2c60 0x55ff7d8d6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.506+0000 7fd5c17fa700 0 client.27625 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:17:22.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.506+0000 7fd5f345fb80 1 -- 172.21.15.101:0/2995425818 >> 172.21.15.101:0/2995425818 conn(0x55ff7d8b8e20 msgr2=0x55ff7d8cf500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:22.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.506+0000 7fd5f345fb80 1 -- 172.21.15.101:0/2995425818 shutdown_connections 2023-07-18T03:17:22.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.506+0000 7fd5f345fb80 1 -- 172.21.15.101:0/2995425818 wait complete. 2023-07-18T03:17:22.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.506+0000 7fd5f345fb80 20 client.27625 trim_cache size 0 max 16384 2023-07-18T03:17:22.513 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:17:22.514 INFO:teuthology.orchestra.run.smithi101.stdout: "return_code": 0, 2023-07-18T03:17:22.514 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_tag": "8bcf0ed4-ebb7-4ffb-bee4-79682f1524c3", 2023-07-18T03:17:22.514 INFO:teuthology.orchestra.run.smithi101.stdout: "mode": "asynchronous" 2023-07-18T03:17:22.514 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:17:22.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.507+0000 7fd5dfbff700 1 -- 172.21.15.101:0/4202171362 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd5c40a12c0 msgr2=0x7fd5c40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:22.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.507+0000 7fd5dfbff700 1 --2- 172.21.15.101:0/4202171362 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd5c40a12c0 0x7fd5c40a3770 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd5d000b780 tx=0x7fd5d00095c0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.507+0000 7fd5dfbff700 1 -- 172.21.15.101:0/4202171362 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd5d80ffad0 msgr2=0x7fd5d819bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:22.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.507+0000 7fd5dfbff700 1 --2- 172.21.15.101:0/4202171362 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd5d80ffad0 0x7fd5d819bae0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fd5d400e9a0 tx=0x7fd5d400ecb0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.507+0000 7fd5dfbff700 1 -- 172.21.15.101:0/4202171362 shutdown_connections 2023-07-18T03:17:22.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.507+0000 7fd5dfbff700 1 --2- 172.21.15.101:0/4202171362 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd5c40a12c0 0x7fd5c40a3770 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.507+0000 7fd5dfbff700 1 --2- 172.21.15.101:0/4202171362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5d8105ae0 0x7fd5d8195b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.507+0000 7fd5dfbff700 1 --2- 172.21.15.101:0/4202171362 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd5d80ffad0 0x7fd5d819bae0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.507+0000 7fd5dfbff700 1 --2- 172.21.15.101:0/4202171362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5d8100480 0x7fd5d819c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.507+0000 7fd5dfbff700 1 -- 172.21.15.101:0/4202171362 >> 172.21.15.101:0/4202171362 conn(0x7fd5d80fb5f0 msgr2=0x7fd5d80fbb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:22.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.507+0000 7fd5dfbff700 1 -- 172.21.15.101:0/4202171362 shutdown_connections 2023-07-18T03:17:22.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.507+0000 7fd5dfbff700 1 -- 172.21.15.101:0/4202171362 wait complete. 2023-07-18T03:17:22.522 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:17:22.611 INFO:tasks.workunit.client.0.smithi101.stdout: 1 626903 5.94 MB/sec execute 407 sec latency 709.507 ms 2023-07-18T03:17:22.780 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.785+0000 7f0f33dff700 1 -- 172.21.15.101:0/3180000024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f2c0728b0 msgr2=0x7f0f2c072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:22.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.785+0000 7f0f33dff700 1 --2- 172.21.15.101:0/3180000024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f2c0728b0 0x7f0f2c072c80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0f28009990 tx=0x7f0f28009ca0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.785+0000 7f0f33dff700 1 -- 172.21.15.101:0/3180000024 shutdown_connections 2023-07-18T03:17:22.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.785+0000 7f0f33dff700 1 --2- 172.21.15.101:0/3180000024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f2c0728b0 0x7f0f2c072c80 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.785+0000 7f0f33dff700 1 --2- 172.21.15.101:0/3180000024 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0f2c10c320 0x7f0f2c10e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.785+0000 7f0f33dff700 1 --2- 172.21.15.101:0/3180000024 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f2c0731c0 0x7f0f2c10bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.785+0000 7f0f33dff700 1 -- 172.21.15.101:0/3180000024 >> 172.21.15.101:0/3180000024 conn(0x7f0f2c0fb5e0 msgr2=0x7f0f2c0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:22.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.785+0000 7f0f33dff700 1 -- 172.21.15.101:0/3180000024 shutdown_connections 2023-07-18T03:17:22.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.785+0000 7f0f33dff700 1 -- 172.21.15.101:0/3180000024 wait complete. 2023-07-18T03:17:22.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.786+0000 7f0f33dff700 1 Processor -- start 2023-07-18T03:17:22.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.786+0000 7f0f33dff700 1 -- start start 2023-07-18T03:17:22.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.787+0000 7f0f33dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0f2c0728b0 0x7f0f2c197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:22.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.787+0000 7f0f33dff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f2c0731c0 0x7f0f2c1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:22.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.787+0000 7f0f33dff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f2c10c320 0x7f0f2c19c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:22.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.787+0000 7f0f33dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2c19ce80 con 0x7f0f2c0728b0 2023-07-18T03:17:22.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.787+0000 7f0f33dff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2c19cff0 con 0x7f0f2c10c320 2023-07-18T03:17:22.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.787+0000 7f0f33dff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2c19d1b0 con 0x7f0f2c0731c0 2023-07-18T03:17:22.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.787+0000 7f0f31b9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0f2c0728b0 0x7f0f2c197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:22.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.787+0000 7f0f31b9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0f2c0728b0 0x7f0f2c197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:46110/0 (socket says 172.21.15.101:46110) 2023-07-18T03:17:22.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.787+0000 7f0f31b9b700 1 -- 172.21.15.101:0/4263758837 learned_addr learned my addr 172.21.15.101:0/4263758837 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:22.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.787+0000 7f0f3239c700 1 --2- 172.21.15.101:0/4263758837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f2c10c320 0x7f0f2c19c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:22.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.787+0000 7f0f31b9b700 1 -- 172.21.15.101:0/4263758837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f2c0731c0 msgr2=0x7f0f2c1983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:22.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.787+0000 7f0f31b9b700 1 --2- 172.21.15.101:0/4263758837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f2c0731c0 0x7f0f2c1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.787+0000 7f0f31b9b700 1 -- 172.21.15.101:0/4263758837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f2c10c320 msgr2=0x7f0f2c19c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:22.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.787+0000 7f0f31b9b700 1 --2- 172.21.15.101:0/4263758837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f2c10c320 0x7f0f2c19c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.787+0000 7f0f31b9b700 1 -- 172.21.15.101:0/4263758837 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f280095f0 con 0x7f0f2c0728b0 2023-07-18T03:17:22.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.788+0000 7f0f31b9b700 1 --2- 172.21.15.101:0/4263758837 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0f2c0728b0 0x7f0f2c197ea0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f0f28005910 tx=0x7f0f28005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:22.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.788+0000 7f0f22ffd700 1 -- 172.21.15.101:0/4263758837 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2801d070 con 0x7f0f2c0728b0 2023-07-18T03:17:22.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.788+0000 7f0f33dff700 1 -- 172.21.15.101:0/4263758837 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f2c19d430 con 0x7f0f2c0728b0 2023-07-18T03:17:22.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.788+0000 7f0f22ffd700 1 -- 172.21.15.101:0/4263758837 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f28022470 con 0x7f0f2c0728b0 2023-07-18T03:17:22.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.788+0000 7f0f22ffd700 1 -- 172.21.15.101:0/4263758837 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f28017650 con 0x7f0f2c0728b0 2023-07-18T03:17:22.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.788+0000 7f0f33dff700 1 -- 172.21.15.101:0/4263758837 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f2c1a4cd0 con 0x7f0f2c0728b0 2023-07-18T03:17:22.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.792+0000 7f0f22ffd700 1 -- 172.21.15.101:0/4263758837 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f0f280225e0 con 0x7f0f2c0728b0 2023-07-18T03:17:22.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.793+0000 7f0f4765db80 1 Processor -- start 2023-07-18T03:17:22.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.793+0000 7f0f4765db80 1 -- start start 2023-07-18T03:17:22.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.793+0000 7f0f22ffd700 1 --2- 172.21.15.101:0/4263758837 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0f180a12c0 0x7f0f180a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:22.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.793+0000 7f0f4765db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630ed96b2d0 0x5630ed96b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:22.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.794+0000 7f0f4765db80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5630ed96bbe0 0x5630ed976590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:22.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.794+0000 7f0f4765db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5630ed976ad0 0x5630ed978eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:22.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.794+0000 7f0f4765db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ed871cb0 con 0x5630ed96b2d0 2023-07-18T03:17:22.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.794+0000 7f0f4765db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ed8c4a40 con 0x5630ed976ad0 2023-07-18T03:17:22.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.794+0000 7f0f4765db80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ed8bded0 con 0x5630ed96bbe0 2023-07-18T03:17:22.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.794+0000 7f0f22ffd700 1 -- 172.21.15.101:0/4263758837 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f0f280ce350 con 0x7f0f2c0728b0 2023-07-18T03:17:22.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.794+0000 7f0f31b9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630ed96b2d0 0x5630ed96b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:22.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.794+0000 7f0f31b9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630ed96b2d0 0x5630ed96b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:46112/0 (socket says 172.21.15.101:46112) 2023-07-18T03:17:22.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.794+0000 7f0f31b9b700 1 -- 172.21.15.101:0/231452476 learned_addr learned my addr 172.21.15.101:0/231452476 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:22.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.794+0000 7f0f3239c700 1 --2- 172.21.15.101:0/4263758837 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0f180a12c0 0x7f0f180a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:22.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.794+0000 7f0f3139a700 1 --2- 172.21.15.101:0/231452476 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5630ed96bbe0 0x5630ed976590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:22.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.794+0000 7f0f3239c700 1 --2- 172.21.15.101:0/231452476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5630ed976ad0 0x5630ed978eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:22.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.794+0000 7f0f31b9b700 1 -- 172.21.15.101:0/231452476 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5630ed96bbe0 msgr2=0x5630ed976590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:22.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.794+0000 7f0f31b9b700 1 --2- 172.21.15.101:0/231452476 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5630ed96bbe0 0x5630ed976590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.794+0000 7f0f31b9b700 1 -- 172.21.15.101:0/231452476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5630ed976ad0 msgr2=0x5630ed978eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:22.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.794+0000 7f0f31b9b700 1 --2- 172.21.15.101:0/231452476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5630ed976ad0 0x5630ed978eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.794+0000 7f0f31b9b700 1 -- 172.21.15.101:0/231452476 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5630ed6ebde0 con 0x5630ed96b2d0 2023-07-18T03:17:22.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.794+0000 7f0f3239c700 1 --2- 172.21.15.101:0/231452476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5630ed976ad0 0x5630ed978eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:17:22.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.795+0000 7f0f31b9b700 1 --2- 172.21.15.101:0/231452476 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630ed96b2d0 0x5630ed96b6a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0f280a2f30 tx=0x7f0f28009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24e9d3d8d2ffaf3e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:22.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.795+0000 7f0f3239c700 1 --2- 172.21.15.101:0/4263758837 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0f180a12c0 0x7f0f180a3770 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0f24009ec0 tx=0x7f0f24009920 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:22.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.795+0000 7f0f17fff700 1 -- 172.21.15.101:0/231452476 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2809abe0 con 0x5630ed96b2d0 2023-07-18T03:17:22.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.795+0000 7f0f17fff700 1 -- 172.21.15.101:0/231452476 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f2809aa00 con 0x5630ed96b2d0 2023-07-18T03:17:22.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.795+0000 7f0f17fff700 1 -- 172.21.15.101:0/231452476 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f280228c0 con 0x5630ed96b2d0 2023-07-18T03:17:22.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.796+0000 7f0f4765db80 1 -- 172.21.15.101:0/231452476 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630ed96b2d0 msgr2=0x5630ed96b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:22.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.796+0000 7f0f4765db80 1 --2- 172.21.15.101:0/231452476 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630ed96b2d0 0x5630ed96b6a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0f280a2f30 tx=0x7f0f28009310 comp rx=0 tx=0).stop 2023-07-18T03:17:22.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.796+0000 7f0f4765db80 1 -- 172.21.15.101:0/231452476 shutdown_connections 2023-07-18T03:17:22.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.796+0000 7f0f4765db80 1 --2- 172.21.15.101:0/231452476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5630ed976ad0 0x5630ed978eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.796+0000 7f0f4765db80 1 --2- 172.21.15.101:0/231452476 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630ed96b2d0 0x5630ed96b6a0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.796+0000 7f0f4765db80 1 --2- 172.21.15.101:0/231452476 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5630ed96bbe0 0x5630ed976590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.796+0000 7f0f4765db80 1 -- 172.21.15.101:0/231452476 >> 172.21.15.101:0/231452476 conn(0x5630ed961da0 msgr2=0x5630ed9676a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:22.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.796+0000 7f0f4765db80 1 -- 172.21.15.101:0/231452476 shutdown_connections 2023-07-18T03:17:22.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.796+0000 7f0f4765db80 1 -- 172.21.15.101:0/231452476 wait complete. 2023-07-18T03:17:22.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.796+0000 7f0f4765db80 1 -- start start 2023-07-18T03:17:22.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.797+0000 7f0f4765db80 1 Processor -- start 2023-07-18T03:17:22.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.797+0000 7f0f4765db80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5630ed96b2d0 0x5630ed97bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:22.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.797+0000 7f0f4765db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630ed96bbe0 0x5630ed97c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:22.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.797+0000 7f0f4765db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5630ed976ad0 0x5630ed975a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:22.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.797+0000 7f0f4765db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ed85dac0 con 0x5630ed96bbe0 2023-07-18T03:17:22.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.797+0000 7f0f4765db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ed85d7c0 con 0x5630ed976ad0 2023-07-18T03:17:22.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.797+0000 7f0f4765db80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ed741920 con 0x5630ed96b2d0 2023-07-18T03:17:22.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.797+0000 7f0f4765db80 10 client.0 _ll_register_callbacks cb 0x5630ed85b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:17:22.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.797+0000 7f0f31b9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5630ed96b2d0 0x5630ed97bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:22.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.797+0000 7f0f3139a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630ed96bbe0 0x5630ed97c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:22.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.797+0000 7f0f31b9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5630ed96b2d0 0x5630ed97bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:44330/0 (socket says 172.21.15.101:44330) 2023-07-18T03:17:22.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.797+0000 7f0f31b9b700 1 -- 172.21.15.101:0/4081014140 learned_addr learned my addr 172.21.15.101:0/4081014140 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:22.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.798+0000 7f0f3239c700 1 --2- 172.21.15.101:0/4081014140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5630ed976ad0 0x5630ed975a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:22.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.798+0000 7f0f31b9b700 1 -- 172.21.15.101:0/4081014140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5630ed976ad0 msgr2=0x5630ed975a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:22.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.798+0000 7f0f31b9b700 1 --2- 172.21.15.101:0/4081014140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5630ed976ad0 0x5630ed975a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.798+0000 7f0f31b9b700 1 -- 172.21.15.101:0/4081014140 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630ed96bbe0 msgr2=0x5630ed97c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:22.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.798+0000 7f0f31b9b700 1 --2- 172.21.15.101:0/4081014140 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630ed96bbe0 0x5630ed97c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.798+0000 7f0f31b9b700 1 -- 172.21.15.101:0/4081014140 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f2c1a4cd0 con 0x5630ed96b2d0 2023-07-18T03:17:22.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.798+0000 7f0f3139a700 1 --2- 172.21.15.101:0/4081014140 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630ed96bbe0 0x5630ed97c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:17:22.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.798+0000 7f0f31b9b700 1 --2- 172.21.15.101:0/4081014140 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5630ed96b2d0 0x5630ed97bc00 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0f2809b030 tx=0x7f0f280cbc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e42d94fb487844a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:22.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.798+0000 7f0f157fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:17:22.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.798+0000 7f0f157fa700 1 -- 172.21.15.101:0/4081014140 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2809f0d0 con 0x5630ed96b2d0 2023-07-18T03:17:22.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.798+0000 7f0f4765db80 1 -- 172.21.15.101:0/4081014140 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5630ed8174f0 con 0x5630ed96b2d0 2023-07-18T03:17:22.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.798+0000 7f0f157fa700 1 -- 172.21.15.101:0/4081014140 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f0f280a3460 con 0x5630ed96b2d0 2023-07-18T03:17:22.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.798+0000 7f0f157fa700 1 -- 172.21.15.101:0/4081014140 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f2809b300 con 0x5630ed96b2d0 2023-07-18T03:17:22.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.798+0000 7f0f4765db80 10 client.27634 fetch_fsmap learned FSMap version 33 2023-07-18T03:17:22.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.798+0000 7f0f4765db80 1 -- 172.21.15.101:0/4081014140 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5630ed6d4c60 con 0x5630ed96b2d0 2023-07-18T03:17:22.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.799+0000 7f0f157fa700 1 -- 172.21.15.101:0/4081014140 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f280be080 con 0x5630ed96b2d0 2023-07-18T03:17:22.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.799+0000 7f0f157fa700 1 -- 172.21.15.101:0/4081014140 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0f280a2fd0 con 0x5630ed96b2d0 2023-07-18T03:17:22.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.799+0000 7f0f4765db80 10 client.27634 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:17:22.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.799+0000 7f0f4765db80 10 client.27634 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:17:22.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.799+0000 7f0f4765db80 20 client.27634 populate_metadata read hostname 'smithi101' 2023-07-18T03:17:22.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.799+0000 7f0f4765db80 1 --2- 172.21.15.101:0/4081014140 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5630ed985ba0 0x5630ed987f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:22.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.799+0000 7f0f4765db80 4 client.27634 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:17:22.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.799+0000 7f0f4765db80 1 -- 172.21.15.101:0/4081014140 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5630ed8b6eb0 con 0x5630ed985ba0 2023-07-18T03:17:22.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.799+0000 7f0f3139a700 1 --2- 172.21.15.101:0/4081014140 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5630ed985ba0 0x5630ed987f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:22.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.800+0000 7f0f157fa700 10 client.27634 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:17:22.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.800+0000 7f0f3139a700 1 --2- 172.21.15.101:0/4081014140 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5630ed985ba0 0x5630ed987f80 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c31e395a5e18faa5 server_cookie=349c5a443763de91 in_seq=0 out_seq=0 2023-07-18T03:17:22.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.801+0000 7f0f157fa700 1 -- 172.21.15.101:0/4081014140 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5630ed8b6eb0 con 0x5630ed985ba0 2023-07-18T03:17:22.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.801+0000 7f0f157fa700 10 client.27634 handle_command_reply: tid=0 2023-07-18T03:17:22.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.801+0000 7f0f4765db80 1 client.27634 shutdown 2023-07-18T03:17:22.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.801+0000 7f0f4765db80 1 -- 172.21.15.101:0/4081014140 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5630ed96b2d0 msgr2=0x5630ed97bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:22.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.801+0000 7f0f4765db80 1 --2- 172.21.15.101:0/4081014140 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5630ed96b2d0 0x5630ed97bc00 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0f2809b030 tx=0x7f0f280cbc40 comp rx=0 tx=0).stop 2023-07-18T03:17:22.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.801+0000 7f0f4765db80 1 -- 172.21.15.101:0/4081014140 shutdown_connections 2023-07-18T03:17:22.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.801+0000 7f0f4765db80 1 --2- 172.21.15.101:0/4081014140 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5630ed985ba0 0x5630ed987f80 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.801+0000 7f0f4765db80 1 --2- 172.21.15.101:0/4081014140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5630ed976ad0 0x5630ed975a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.801+0000 7f0f157fa700 0 client.27634 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:17:22.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.801+0000 7f0f4765db80 1 --2- 172.21.15.101:0/4081014140 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5630ed96b2d0 0x5630ed97bc00 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.801+0000 7f0f4765db80 1 --2- 172.21.15.101:0/4081014140 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630ed96bbe0 0x5630ed97c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.801+0000 7f0f4765db80 1 -- 172.21.15.101:0/4081014140 >> 172.21.15.101:0/4081014140 conn(0x5630ed961da0 msgr2=0x5630ed967290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:22.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.801+0000 7f0f4765db80 1 -- 172.21.15.101:0/4081014140 shutdown_connections 2023-07-18T03:17:22.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.801+0000 7f0f4765db80 1 -- 172.21.15.101:0/4081014140 wait complete. 2023-07-18T03:17:22.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.801+0000 7f0f4765db80 20 client.27634 trim_cache size 0 max 16384 2023-07-18T03:17:22.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.822+0000 7f0f4765db80 1 Processor -- start 2023-07-18T03:17:22.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.822+0000 7f0f4765db80 1 -- start start 2023-07-18T03:17:22.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.822+0000 7f0f4765db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630ed96b2d0 0x5630ed975bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:22.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.822+0000 7f0f4765db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5630ed96bbe0 0x5630ed97b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:22.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.822+0000 7f0f4765db80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5630ed976ad0 0x5630ed97bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:22.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.822+0000 7f0f4765db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ed96ce70 con 0x5630ed96b2d0 2023-07-18T03:17:22.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.822+0000 7f0f4765db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ed963ef0 con 0x5630ed96bbe0 2023-07-18T03:17:22.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.822+0000 7f0f4765db80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ed977300 con 0x5630ed976ad0 2023-07-18T03:17:22.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.822+0000 7f0f31b9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630ed96b2d0 0x5630ed975bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:22.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.822+0000 7f0f3139a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5630ed96bbe0 0x5630ed97b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:22.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.822+0000 7f0f31b9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630ed96b2d0 0x5630ed975bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:46134/0 (socket says 172.21.15.101:46134) 2023-07-18T03:17:22.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.822+0000 7f0f31b9b700 1 -- 172.21.15.101:0/3968440656 learned_addr learned my addr 172.21.15.101:0/3968440656 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:22.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.822+0000 7f0f3139a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5630ed96bbe0 0x5630ed97b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60780/0 (socket says 172.21.15.101:60780) 2023-07-18T03:17:22.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.822+0000 7f0f3239c700 1 --2- 172.21.15.101:0/3968440656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5630ed976ad0 0x5630ed97bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:22.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.823+0000 7f0f3139a700 1 -- 172.21.15.101:0/3968440656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5630ed976ad0 msgr2=0x5630ed97bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:22.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.823+0000 7f0f3139a700 1 --2- 172.21.15.101:0/3968440656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5630ed976ad0 0x5630ed97bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.823+0000 7f0f3139a700 1 -- 172.21.15.101:0/3968440656 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630ed96b2d0 msgr2=0x5630ed975bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:22.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.823+0000 7f0f3139a700 1 --2- 172.21.15.101:0/3968440656 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630ed96b2d0 0x5630ed975bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.823+0000 7f0f3139a700 1 -- 172.21.15.101:0/3968440656 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5630ed712250 con 0x5630ed96bbe0 2023-07-18T03:17:22.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.823+0000 7f0f31b9b700 1 --2- 172.21.15.101:0/3968440656 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630ed96b2d0 0x5630ed975bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:17:22.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.823+0000 7f0f3239c700 1 --2- 172.21.15.101:0/3968440656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5630ed976ad0 0x5630ed97bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:17:22.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.823+0000 7f0f3139a700 1 --2- 172.21.15.101:0/3968440656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5630ed96bbe0 0x5630ed97b7b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x5630ed988830 tx=0x7f0f1c01ec40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=536ac7ad410cdcd1 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:22.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.823+0000 7f0f20bff700 1 -- 172.21.15.101:0/3968440656 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c017040 con 0x5630ed96bbe0 2023-07-18T03:17:22.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.823+0000 7f0f20bff700 1 -- 172.21.15.101:0/3968440656 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f1c019040 con 0x5630ed96bbe0 2023-07-18T03:17:22.822 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.823+0000 7f0f20bff700 1 -- 172.21.15.101:0/3968440656 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c005600 con 0x5630ed96bbe0 2023-07-18T03:17:22.822 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.824+0000 7f0f4765db80 1 -- 172.21.15.101:0/3968440656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5630ed96bbe0 msgr2=0x5630ed97b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:22.822 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.824+0000 7f0f4765db80 1 --2- 172.21.15.101:0/3968440656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5630ed96bbe0 0x5630ed97b7b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x5630ed988830 tx=0x7f0f1c01ec40 comp rx=0 tx=0).stop 2023-07-18T03:17:22.822 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.824+0000 7f0f4765db80 1 -- 172.21.15.101:0/3968440656 shutdown_connections 2023-07-18T03:17:22.822 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.824+0000 7f0f4765db80 1 --2- 172.21.15.101:0/3968440656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5630ed976ad0 0x5630ed97bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.822 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.824+0000 7f0f4765db80 1 --2- 172.21.15.101:0/3968440656 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630ed96b2d0 0x5630ed975bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.822 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.824+0000 7f0f4765db80 1 --2- 172.21.15.101:0/3968440656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5630ed96bbe0 0x5630ed97b7b0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.823 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.824+0000 7f0f4765db80 1 -- 172.21.15.101:0/3968440656 >> 172.21.15.101:0/3968440656 conn(0x5630ed961da0 msgr2=0x5630ed965e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:22.823 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.824+0000 7f0f4765db80 1 -- 172.21.15.101:0/3968440656 shutdown_connections 2023-07-18T03:17:22.823 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.824+0000 7f0f4765db80 1 -- 172.21.15.101:0/3968440656 wait complete. 2023-07-18T03:17:22.826 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.825+0000 7f0f4765db80 1 -- start start 2023-07-18T03:17:22.826 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.825+0000 7f0f4765db80 1 Processor -- start 2023-07-18T03:17:22.826 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.825+0000 7f0f4765db80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5630ed96b2d0 0x5630ed9d3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:22.826 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.825+0000 7f0f4765db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630ed96bbe0 0x5630ed9d37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:22.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.826+0000 7f0f4765db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5630ed976ad0 0x5630ed9cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:22.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.826+0000 7f0f4765db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ed9cdfa0 con 0x5630ed96bbe0 2023-07-18T03:17:22.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.826+0000 7f0f4765db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ed9ce110 con 0x5630ed976ad0 2023-07-18T03:17:22.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.826+0000 7f0f4765db80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ed9ce2d0 con 0x5630ed96b2d0 2023-07-18T03:17:22.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.826+0000 7f0f4765db80 10 client.0 _ll_register_callbacks cb 0x7f0f1c009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:17:22.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.826+0000 7f0f3139a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630ed96bbe0 0x5630ed9d37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:22.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.826+0000 7f0f31b9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5630ed96b2d0 0x5630ed9d3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:22.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.826+0000 7f0f3139a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630ed96bbe0 0x5630ed9d37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:46140/0 (socket says 172.21.15.101:46140) 2023-07-18T03:17:22.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.826+0000 7f0f3239c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5630ed976ad0 0x5630ed9cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:22.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.826+0000 7f0f3139a700 1 -- 172.21.15.101:0/2051098335 learned_addr learned my addr 172.21.15.101:0/2051098335 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:22.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.826+0000 7f0f31b9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5630ed96b2d0 0x5630ed9d3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:44340/0 (socket says 172.21.15.101:44340) 2023-07-18T03:17:22.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.826+0000 7f0f3139a700 1 -- 172.21.15.101:0/2051098335 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5630ed96b2d0 msgr2=0x5630ed9d3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:22.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.826+0000 7f0f3139a700 1 --2- 172.21.15.101:0/2051098335 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5630ed96b2d0 0x5630ed9d3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.826+0000 7f0f3139a700 1 -- 172.21.15.101:0/2051098335 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5630ed976ad0 msgr2=0x5630ed9cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:22.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.826+0000 7f0f3139a700 1 --2- 172.21.15.101:0/2051098335 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5630ed976ad0 0x5630ed9cd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.826+0000 7f0f3139a700 1 -- 172.21.15.101:0/2051098335 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5630ed6d4c60 con 0x5630ed96bbe0 2023-07-18T03:17:22.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.826+0000 7f0f3139a700 1 --2- 172.21.15.101:0/2051098335 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630ed96bbe0 0x5630ed9d37d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c004790 tx=0x7f0f1c01e520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9295cd53076134e5 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:22.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.826+0000 7f0f157fa700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:17:22.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.827+0000 7f0f157fa700 1 -- 172.21.15.101:0/2051098335 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c017040 con 0x5630ed96bbe0 2023-07-18T03:17:22.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.827+0000 7f0f4765db80 1 -- 172.21.15.101:0/2051098335 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0f2809aa00 con 0x5630ed96bbe0 2023-07-18T03:17:22.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.827+0000 7f0f157fa700 1 -- 172.21.15.101:0/2051098335 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f0f1c005420 con 0x5630ed96bbe0 2023-07-18T03:17:22.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.827+0000 7f0f157fa700 1 -- 172.21.15.101:0/2051098335 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f1c0055e0 con 0x5630ed96bbe0 2023-07-18T03:17:22.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.827+0000 7f0f4765db80 10 client.17571 fetch_fsmap learned FSMap version 33 2023-07-18T03:17:22.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.827+0000 7f0f4765db80 1 -- 172.21.15.101:0/2051098335 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f0f1c012050 con 0x5630ed96bbe0 2023-07-18T03:17:22.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.827+0000 7f0f157fa700 1 -- 172.21.15.101:0/2051098335 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f2809aa00 con 0x5630ed96bbe0 2023-07-18T03:17:22.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.827+0000 7f0f157fa700 1 -- 172.21.15.101:0/2051098335 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0f1c0128a0 con 0x5630ed96bbe0 2023-07-18T03:17:22.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.828+0000 7f0f4765db80 10 client.17571 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:17:22.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.828+0000 7f0f4765db80 10 client.17571 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:17:22.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.828+0000 7f0f4765db80 20 client.17571 populate_metadata read hostname 'smithi101' 2023-07-18T03:17:22.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.828+0000 7f0f4765db80 1 --2- 172.21.15.101:0/2051098335 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5630ed9db130 0x5630ed9dd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:22.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.828+0000 7f0f4765db80 4 client.17571 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:17:22.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.828+0000 7f0f4765db80 1 -- 172.21.15.101:0/2051098335 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5630ed879050 con 0x5630ed9db130 2023-07-18T03:17:22.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.828+0000 7f0f31b9b700 1 --2- 172.21.15.101:0/2051098335 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5630ed9db130 0x5630ed9dd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:22.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.828+0000 7f0f31b9b700 1 --2- 172.21.15.101:0/2051098335 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5630ed9db130 0x5630ed9dd540 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b992f315d6e151e7 server_cookie=249e79312dd3c406 in_seq=0 out_seq=0 2023-07-18T03:17:22.832 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.828+0000 7f0f157fa700 10 client.17571 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:17:22.832 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.828+0000 7f0f157fa700 1 -- 172.21.15.101:0/2051098335 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5630ed879050 con 0x5630ed9db130 2023-07-18T03:17:22.832 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.828+0000 7f0f157fa700 10 client.17571 handle_command_reply: tid=0 2023-07-18T03:17:22.832 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.828+0000 7f0f4765db80 1 client.17571 shutdown 2023-07-18T03:17:22.832 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.829+0000 7f0f4765db80 1 -- 172.21.15.101:0/2051098335 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630ed96bbe0 msgr2=0x5630ed9d37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:22.832 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.829+0000 7f0f4765db80 1 --2- 172.21.15.101:0/2051098335 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630ed96bbe0 0x5630ed9d37d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c004790 tx=0x7f0f1c01e520 comp rx=0 tx=0).stop 2023-07-18T03:17:22.832 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.829+0000 7f0f4765db80 1 -- 172.21.15.101:0/2051098335 shutdown_connections 2023-07-18T03:17:22.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.829+0000 7f0f4765db80 1 --2- 172.21.15.101:0/2051098335 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5630ed9db130 0x5630ed9dd540 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.829+0000 7f0f4765db80 1 --2- 172.21.15.101:0/2051098335 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5630ed976ad0 0x5630ed9cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.829+0000 7f0f4765db80 1 --2- 172.21.15.101:0/2051098335 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5630ed96b2d0 0x5630ed9d3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.829+0000 7f0f4765db80 1 --2- 172.21.15.101:0/2051098335 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630ed96bbe0 0x5630ed9d37d0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.829+0000 7f0f157fa700 0 client.17571 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:17:22.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.829+0000 7f0f4765db80 1 -- 172.21.15.101:0/2051098335 >> 172.21.15.101:0/2051098335 conn(0x5630ed961da0 msgr2=0x5630ed974240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:22.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.829+0000 7f0f4765db80 1 -- 172.21.15.101:0/2051098335 shutdown_connections 2023-07-18T03:17:22.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.829+0000 7f0f4765db80 1 -- 172.21.15.101:0/2051098335 wait complete. 2023-07-18T03:17:22.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.829+0000 7f0f4765db80 20 client.17571 trim_cache size 0 max 16384 2023-07-18T03:17:22.834 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:17:22.834 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:17:22.834 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:17:22.835 INFO:teuthology.orchestra.run.smithi101.stdout: "8bcf0ed4-ebb7-4ffb-bee4-79682f1524c3": { 2023-07-18T03:17:22.835 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:17:22.835 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "8bcf0ed4-ebb7-4ffb-bee4-79682f1524c3", 2023-07-18T03:17:22.835 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:17:22.835 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:17:22.835 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:17:22.835 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:17:22.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.829+0000 7f0f33dff700 1 -- 172.21.15.101:0/4263758837 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0f180a12c0 msgr2=0x7f0f180a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:22.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.829+0000 7f0f33dff700 1 --2- 172.21.15.101:0/4263758837 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0f180a12c0 0x7f0f180a3770 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0f24009ec0 tx=0x7f0f24009920 comp rx=0 tx=0).stop 2023-07-18T03:17:22.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.829+0000 7f0f33dff700 1 -- 172.21.15.101:0/4263758837 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0f2c0728b0 msgr2=0x7f0f2c197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:22.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.829+0000 7f0f33dff700 1 --2- 172.21.15.101:0/4263758837 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0f2c0728b0 0x7f0f2c197ea0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f0f28005910 tx=0x7f0f28005720 comp rx=0 tx=0).stop 2023-07-18T03:17:22.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.830+0000 7f0f33dff700 1 -- 172.21.15.101:0/4263758837 shutdown_connections 2023-07-18T03:17:22.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.830+0000 7f0f33dff700 1 --2- 172.21.15.101:0/4263758837 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0f180a12c0 0x7f0f180a3770 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.830+0000 7f0f33dff700 1 --2- 172.21.15.101:0/4263758837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f2c10c320 0x7f0f2c19c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.830+0000 7f0f33dff700 1 --2- 172.21.15.101:0/4263758837 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0f2c0728b0 0x7f0f2c197ea0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.830+0000 7f0f33dff700 1 --2- 172.21.15.101:0/4263758837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f2c0731c0 0x7f0f2c1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:22.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.830+0000 7f0f33dff700 1 -- 172.21.15.101:0/4263758837 >> 172.21.15.101:0/4263758837 conn(0x7f0f2c0fb5e0 msgr2=0x7f0f2c10e460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:22.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.830+0000 7f0f33dff700 1 -- 172.21.15.101:0/4263758837 shutdown_connections 2023-07-18T03:17:22.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:22.830+0000 7f0f33dff700 1 -- 172.21.15.101:0/4263758837 wait complete. 2023-07-18T03:17:22.843 INFO:tasks.cephfs.filesystem:scrub status for tag:8bcf0ed4-ebb7-4ffb-bee4-79682f1524c3 - {'path': '/', 'tag': '8bcf0ed4-ebb7-4ffb-bee4-79682f1524c3', 'options': 'recursive,force'} 2023-07-18T03:17:23.239 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:22 smithi101 ceph-mon[126379]: pgmap v305: 129 pgs: 129 active+clean; 480 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 130 op/s 2023-07-18T03:17:23.239 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:22 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:17:23.239 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:22 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:17:23.239 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:22 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:17:23.239 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:22 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:17:23.239 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:22 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:17:23.240 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:22 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:17:23.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:22 smithi155 ceph-mon[128885]: pgmap v305: 129 pgs: 129 active+clean; 480 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 130 op/s 2023-07-18T03:17:23.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:22 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:17:23.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:22 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:17:23.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:22 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:17:23.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:22 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:17:23.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:22 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:17:23.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:22 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:17:23.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:22 smithi179 ceph-mon[129229]: pgmap v305: 129 pgs: 129 active+clean; 480 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 130 op/s 2023-07-18T03:17:23.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:22 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:17:23.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:22 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:17:23.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:22 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:17:23.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:22 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:17:23.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:22 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:17:23.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:22 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:17:23.611 INFO:tasks.workunit.client.0.smithi101.stdout: 1 628833 5.96 MB/sec execute 408 sec latency 41.873 ms 2023-07-18T03:17:24.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:23 smithi155 ceph-mon[128885]: scrub queued for path: / 2023-07-18T03:17:24.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:23 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:17:24.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:23 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:17:24.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:23 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:17:24.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:23 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:17:24.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:23 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:17:24.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:23 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:17:24.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:23 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:17:24.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:23 smithi179 ceph-mon[129229]: scrub queued for path: / 2023-07-18T03:17:24.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:23 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:17:24.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:23 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:17:24.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:23 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:17:24.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:23 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:17:24.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:23 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:17:24.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:23 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:17:24.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:23 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:17:24.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:23 smithi101 ceph-mon[126379]: scrub queued for path: / 2023-07-18T03:17:24.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:23 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:17:24.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:23 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:17:24.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:23 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:17:24.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:23 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:17:24.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:23 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:17:24.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:23 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:17:24.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:23 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:17:24.611 INFO:tasks.workunit.client.0.smithi101.stdout: 1 629796 5.95 MB/sec execute 409 sec latency 16.313 ms 2023-07-18T03:17:25.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:24 smithi155 ceph-mon[128885]: pgmap v306: 129 pgs: 129 active+clean; 480 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 130 op/s 2023-07-18T03:17:25.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:24 smithi179 ceph-mon[129229]: pgmap v306: 129 pgs: 129 active+clean; 480 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 130 op/s 2023-07-18T03:17:25.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:24 smithi101 ceph-mon[126379]: pgmap v306: 129 pgs: 129 active+clean; 480 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 130 op/s 2023-07-18T03:17:25.612 INFO:tasks.workunit.client.0.smithi101.stdout: 1 630309 5.94 MB/sec execute 410 sec latency 485.016 ms 2023-07-18T03:17:26.612 INFO:tasks.workunit.client.0.smithi101.stdout: 1 630309 5.93 MB/sec execute 411 sec latency 1485.171 ms 2023-07-18T03:17:27.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:27 smithi155 ceph-mon[128885]: pgmap v307: 129 pgs: 129 active+clean; 488 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 193 op/s 2023-07-18T03:17:27.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:27 smithi179 ceph-mon[129229]: pgmap v307: 129 pgs: 129 active+clean; 488 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 193 op/s 2023-07-18T03:17:27.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:27 smithi101 ceph-mon[126379]: pgmap v307: 129 pgs: 129 active+clean; 488 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 193 op/s 2023-07-18T03:17:27.613 INFO:tasks.workunit.client.0.smithi101.stdout: 1 630309 5.92 MB/sec execute 412 sec latency 2485.276 ms 2023-07-18T03:17:28.612 INFO:tasks.workunit.client.0.smithi101.stdout: 1 630309 5.90 MB/sec execute 413 sec latency 3485.404 ms 2023-07-18T03:17:29.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:29 smithi155 ceph-mon[128885]: pgmap v308: 129 pgs: 129 active+clean; 488 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 137 op/s 2023-07-18T03:17:29.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:29 smithi179 ceph-mon[129229]: pgmap v308: 129 pgs: 129 active+clean; 488 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 137 op/s 2023-07-18T03:17:29.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:29 smithi101 ceph-mon[126379]: pgmap v308: 129 pgs: 129 active+clean; 488 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 137 op/s 2023-07-18T03:17:29.612 INFO:tasks.workunit.client.0.smithi101.stdout: 1 630507 5.89 MB/sec execute 414 sec latency 4336.161 ms 2023-07-18T03:17:30.612 INFO:tasks.workunit.client.0.smithi101.stdout: 1 632404 5.91 MB/sec execute 415 sec latency 28.753 ms 2023-07-18T03:17:31.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:31 smithi155 ceph-mon[128885]: pgmap v309: 129 pgs: 129 active+clean; 492 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 157 op/s 2023-07-18T03:17:31.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:31 smithi179 ceph-mon[129229]: pgmap v309: 129 pgs: 129 active+clean; 492 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 157 op/s 2023-07-18T03:17:31.488 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:31 smithi101 ceph-mon[126379]: pgmap v309: 129 pgs: 129 active+clean; 492 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 157 op/s 2023-07-18T03:17:31.613 INFO:tasks.workunit.client.0.smithi101.stdout: 1 633354 5.90 MB/sec execute 416 sec latency 51.742 ms 2023-07-18T03:17:32.613 INFO:tasks.workunit.client.0.smithi101.stdout: 1 633858 5.89 MB/sec execute 417 sec latency 665.506 ms 2023-07-18T03:17:33.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:33 smithi155 ceph-mon[128885]: pgmap v310: 129 pgs: 129 active+clean; 492 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 101 op/s 2023-07-18T03:17:33.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:33 smithi179 ceph-mon[129229]: pgmap v310: 129 pgs: 129 active+clean; 492 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 101 op/s 2023-07-18T03:17:33.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:33 smithi101 ceph-mon[126379]: pgmap v310: 129 pgs: 129 active+clean; 492 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 101 op/s 2023-07-18T03:17:33.613 INFO:tasks.workunit.client.0.smithi101.stdout: 1 633858 5.88 MB/sec execute 418 sec latency 1665.680 ms 2023-07-18T03:17:34.614 INFO:tasks.workunit.client.0.smithi101.stdout: 1 634307 5.86 MB/sec execute 419 sec latency 2494.561 ms 2023-07-18T03:17:35.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:35 smithi155 ceph-mon[128885]: pgmap v311: 129 pgs: 129 active+clean; 492 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 101 op/s 2023-07-18T03:17:35.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:35 smithi179 ceph-mon[129229]: pgmap v311: 129 pgs: 129 active+clean; 492 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 101 op/s 2023-07-18T03:17:35.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:35 smithi101 ceph-mon[126379]: pgmap v311: 129 pgs: 129 active+clean; 492 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 101 op/s 2023-07-18T03:17:35.613 INFO:tasks.workunit.client.0.smithi101.stdout: 1 636034 5.88 MB/sec execute 420 sec latency 13.603 ms 2023-07-18T03:17:36.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:36 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:17:36.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:36 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:17:36.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:36 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:17:36.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:36 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:17:36.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:36 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:17:36.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:36 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:17:36.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:36 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:17:36.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:36 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:17:36.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:36 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:17:36.614 INFO:tasks.workunit.client.0.smithi101.stdout: 1 637177 5.88 MB/sec execute 421 sec latency 19.496 ms 2023-07-18T03:17:37.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:37 smithi155 ceph-mon[128885]: pgmap v312: 129 pgs: 129 active+clean; 494 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 150 op/s 2023-07-18T03:17:37.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:37 smithi179 ceph-mon[129229]: pgmap v312: 129 pgs: 129 active+clean; 494 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 150 op/s 2023-07-18T03:17:37.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:37 smithi101 ceph-mon[126379]: pgmap v312: 129 pgs: 129 active+clean; 494 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 150 op/s 2023-07-18T03:17:37.614 INFO:tasks.workunit.client.0.smithi101.stdout: 1 638105 5.87 MB/sec execute 422 sec latency 382.403 ms 2023-07-18T03:17:38.614 INFO:tasks.workunit.client.0.smithi101.stdout: 1 639576 5.88 MB/sec execute 423 sec latency 60.642 ms 2023-07-18T03:17:39.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:39 smithi155 ceph-mon[128885]: pgmap v313: 129 pgs: 129 active+clean; 494 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 985 KiB/s wr, 69 op/s 2023-07-18T03:17:39.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:39 smithi179 ceph-mon[129229]: pgmap v313: 129 pgs: 129 active+clean; 494 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 985 KiB/s wr, 69 op/s 2023-07-18T03:17:39.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:39 smithi101 ceph-mon[126379]: pgmap v313: 129 pgs: 129 active+clean; 494 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 985 KiB/s wr, 69 op/s 2023-07-18T03:17:39.614 INFO:tasks.workunit.client.0.smithi101.stdout: 1 640709 5.88 MB/sec execute 424 sec latency 17.484 ms 2023-07-18T03:17:40.614 INFO:tasks.workunit.client.0.smithi101.stdout: 1 640948 5.87 MB/sec execute 425 sec latency 690.357 ms 2023-07-18T03:17:41.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:41 smithi155 ceph-mon[128885]: pgmap v314: 129 pgs: 129 active+clean; 502 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 156 op/s 2023-07-18T03:17:41.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:41 smithi179 ceph-mon[129229]: pgmap v314: 129 pgs: 129 active+clean; 502 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 156 op/s 2023-07-18T03:17:41.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:41 smithi101 ceph-mon[126379]: pgmap v314: 129 pgs: 129 active+clean; 502 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 156 op/s 2023-07-18T03:17:41.614 INFO:tasks.workunit.client.0.smithi101.stdout: 1 640948 5.86 MB/sec execute 426 sec latency 1690.417 ms 2023-07-18T03:17:42.615 INFO:tasks.workunit.client.0.smithi101.stdout: 1 640948 5.84 MB/sec execute 427 sec latency 2690.565 ms 2023-07-18T03:17:43.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:43 smithi155 ceph-mon[128885]: pgmap v315: 129 pgs: 129 active+clean; 502 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 135 op/s 2023-07-18T03:17:43.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:43 smithi179 ceph-mon[129229]: pgmap v315: 129 pgs: 129 active+clean; 502 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 135 op/s 2023-07-18T03:17:43.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:43 smithi101 ceph-mon[126379]: pgmap v315: 129 pgs: 129 active+clean; 502 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 135 op/s 2023-07-18T03:17:43.615 INFO:tasks.workunit.client.0.smithi101.stdout: 1 640948 5.83 MB/sec execute 428 sec latency 3690.635 ms 2023-07-18T03:17:44.615 INFO:tasks.workunit.client.0.smithi101.stdout: 1 641397 5.82 MB/sec execute 429 sec latency 4508.481 ms 2023-07-18T03:17:45.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:45 smithi155 ceph-mon[128885]: pgmap v316: 129 pgs: 129 active+clean; 502 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 135 op/s 2023-07-18T03:17:45.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:45 smithi179 ceph-mon[129229]: pgmap v316: 129 pgs: 129 active+clean; 502 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 135 op/s 2023-07-18T03:17:45.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:45 smithi101 ceph-mon[126379]: pgmap v316: 129 pgs: 129 active+clean; 502 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 135 op/s 2023-07-18T03:17:45.615 INFO:tasks.workunit.client.0.smithi101.stdout: 1 643095 5.83 MB/sec execute 430 sec latency 20.493 ms 2023-07-18T03:17:46.615 INFO:tasks.workunit.client.0.smithi101.stdout: 1 644225 5.83 MB/sec execute 431 sec latency 17.616 ms 2023-07-18T03:17:47.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:47 smithi155 ceph-mon[128885]: pgmap v317: 129 pgs: 129 active+clean; 505 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 154 op/s 2023-07-18T03:17:47.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:47 smithi179 ceph-mon[129229]: pgmap v317: 129 pgs: 129 active+clean; 505 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 154 op/s 2023-07-18T03:17:47.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:47 smithi101 ceph-mon[126379]: pgmap v317: 129 pgs: 129 active+clean; 505 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 154 op/s 2023-07-18T03:17:47.615 INFO:tasks.workunit.client.0.smithi101.stdout: 1 644492 5.82 MB/sec execute 432 sec latency 636.658 ms 2023-07-18T03:17:48.615 INFO:tasks.workunit.client.0.smithi101.stdout: 1 644492 5.81 MB/sec execute 433 sec latency 1636.704 ms 2023-07-18T03:17:49.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:49 smithi155 ceph-mon[128885]: pgmap v318: 129 pgs: 129 active+clean; 505 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 105 op/s 2023-07-18T03:17:49.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:49 smithi179 ceph-mon[129229]: pgmap v318: 129 pgs: 129 active+clean; 505 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 105 op/s 2023-07-18T03:17:49.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:49 smithi101 ceph-mon[126379]: pgmap v318: 129 pgs: 129 active+clean; 505 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 105 op/s 2023-07-18T03:17:49.615 INFO:tasks.workunit.client.0.smithi101.stdout: 1 644941 5.79 MB/sec execute 434 sec latency 2454.800 ms 2023-07-18T03:17:50.615 INFO:tasks.workunit.client.0.smithi101.stdout: 1 646617 5.81 MB/sec execute 435 sec latency 17.191 ms 2023-07-18T03:17:51.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:51 smithi155 ceph-mon[128885]: pgmap v319: 129 pgs: 129 active+clean; 509 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 156 op/s 2023-07-18T03:17:51.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:51 smithi179 ceph-mon[129229]: pgmap v319: 129 pgs: 129 active+clean; 509 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 156 op/s 2023-07-18T03:17:51.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:51 smithi101 ceph-mon[126379]: pgmap v319: 129 pgs: 129 active+clean; 509 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 156 op/s 2023-07-18T03:17:51.615 INFO:tasks.workunit.client.0.smithi101.stdout: 1 647752 5.81 MB/sec execute 436 sec latency 15.324 ms 2023-07-18T03:17:52.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:52 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:17:52.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:52 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:17:52.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:52 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:17:52.617 INFO:tasks.workunit.client.0.smithi101.stdout: 1 648042 5.80 MB/sec execute 437 sec latency 584.322 ms 2023-07-18T03:17:52.843 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:17:53.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.073+0000 7efd735ff700 1 -- 172.21.15.101:0/3693106908 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd6c0ff380 msgr2=0x7efd6c10b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:53.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.073+0000 7efd735ff700 1 --2- 172.21.15.101:0/3693106908 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd6c0ff380 0x7efd6c10b810 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7efd64009f90 tx=0x7efd6400b330 comp rx=0 tx=0).stop 2023-07-18T03:17:53.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.073+0000 7efd735ff700 1 -- 172.21.15.101:0/3693106908 shutdown_connections 2023-07-18T03:17:53.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.073+0000 7efd735ff700 1 --2- 172.21.15.101:0/3693106908 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd6c1049e0 0x7efd6c104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.073+0000 7efd735ff700 1 --2- 172.21.15.101:0/3693106908 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd6c0ff380 0x7efd6c10b810 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.073+0000 7efd735ff700 1 --2- 172.21.15.101:0/3693106908 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efd6c0fe9d0 0x7efd6c0fee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.073+0000 7efd735ff700 1 -- 172.21.15.101:0/3693106908 >> 172.21.15.101:0/3693106908 conn(0x7efd6c0fa4b0 msgr2=0x7efd6c0fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:53.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.073+0000 7efd735ff700 1 -- 172.21.15.101:0/3693106908 shutdown_connections 2023-07-18T03:17:53.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.073+0000 7efd735ff700 1 -- 172.21.15.101:0/3693106908 wait complete. 2023-07-18T03:17:53.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.074+0000 7efd735ff700 1 Processor -- start 2023-07-18T03:17:53.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.074+0000 7efd735ff700 1 -- start start 2023-07-18T03:17:53.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.075+0000 7efd735ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd6c0fe9d0 0x7efd6c197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:53.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.075+0000 7efd735ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efd6c0ff380 0x7efd6c198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:53.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.075+0000 7efd735ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd6c1049e0 0x7efd6c19c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:53.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.075+0000 7efd735ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c19ceb0 con 0x7efd6c0ff380 2023-07-18T03:17:53.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.075+0000 7efd735ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c19d020 con 0x7efd6c1049e0 2023-07-18T03:17:53.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.075+0000 7efd735ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c19d1e0 con 0x7efd6c0fe9d0 2023-07-18T03:17:53.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.075+0000 7efd7139b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd6c0fe9d0 0x7efd6c197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:53.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.075+0000 7efd70b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efd6c0ff380 0x7efd6c198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:53.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.075+0000 7efd7139b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd6c0fe9d0 0x7efd6c197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:39484/0 (socket says 172.21.15.101:39484) 2023-07-18T03:17:53.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.075+0000 7efd7139b700 1 -- 172.21.15.101:0/3454941036 learned_addr learned my addr 172.21.15.101:0/3454941036 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:53.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.075+0000 7efd71b9c700 1 --2- 172.21.15.101:0/3454941036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd6c1049e0 0x7efd6c19c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:53.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.076+0000 7efd7139b700 1 -- 172.21.15.101:0/3454941036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd6c1049e0 msgr2=0x7efd6c19c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:53.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.076+0000 7efd7139b700 1 --2- 172.21.15.101:0/3454941036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd6c1049e0 0x7efd6c19c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.076+0000 7efd7139b700 1 -- 172.21.15.101:0/3454941036 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efd6c0ff380 msgr2=0x7efd6c198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:53.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.076+0000 7efd7139b700 1 --2- 172.21.15.101:0/3454941036 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efd6c0ff380 0x7efd6c198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.076+0000 7efd7139b700 1 -- 172.21.15.101:0/3454941036 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd680095f0 con 0x7efd6c0fe9d0 2023-07-18T03:17:53.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.076+0000 7efd7139b700 1 --2- 172.21.15.101:0/3454941036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd6c0fe9d0 0x7efd6c197ed0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7efd6800ebf0 tx=0x7efd6800ef00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:53.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.076+0000 7efd627fc700 1 -- 172.21.15.101:0/3454941036 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd68009a10 con 0x7efd6c0fe9d0 2023-07-18T03:17:53.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.076+0000 7efd627fc700 1 -- 172.21.15.101:0/3454941036 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd6800cc90 con 0x7efd6c0fe9d0 2023-07-18T03:17:53.075 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.076+0000 7efd627fc700 1 -- 172.21.15.101:0/3454941036 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd68018750 con 0x7efd6c0fe9d0 2023-07-18T03:17:53.075 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.076+0000 7efd735ff700 1 -- 172.21.15.101:0/3454941036 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd64009c40 con 0x7efd6c0fe9d0 2023-07-18T03:17:53.075 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.076+0000 7efd735ff700 1 -- 172.21.15.101:0/3454941036 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd6c1a4b20 con 0x7efd6c0fe9d0 2023-07-18T03:17:53.075 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.079+0000 7efd627fc700 1 -- 172.21.15.101:0/3454941036 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7efd680188b0 con 0x7efd6c0fe9d0 2023-07-18T03:17:53.076 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.081+0000 7efd627fc700 1 --2- 172.21.15.101:0/3454941036 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7efd580a1340 0x7efd580a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:53.076 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.081+0000 7efd86e60b80 1 Processor -- start 2023-07-18T03:17:53.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.081+0000 7efd627fc700 1 -- 172.21.15.101:0/3454941036 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7efd68014070 con 0x7efd6c0fe9d0 2023-07-18T03:17:53.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.081+0000 7efd70b9a700 1 --2- 172.21.15.101:0/3454941036 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7efd580a1340 0x7efd580a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:53.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.082+0000 7efd86e60b80 1 -- start start 2023-07-18T03:17:53.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.082+0000 7efd86e60b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612495f22d0 0x5612495f26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:53.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.082+0000 7efd86e60b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612495f2be0 0x5612495fd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:53.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.082+0000 7efd86e60b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5612495fdad0 0x5612495ffeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:53.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.082+0000 7efd86e60b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612494f8cb0 con 0x5612495fdad0 2023-07-18T03:17:53.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.082+0000 7efd86e60b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56124954ba40 con 0x5612495f22d0 2023-07-18T03:17:53.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.082+0000 7efd86e60b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x561249544ed0 con 0x5612495f2be0 2023-07-18T03:17:53.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.082+0000 7efd70b9a700 1 --2- 172.21.15.101:0/3454941036 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7efd580a1340 0x7efd580a37f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efd6c1994e0 tx=0x7efd5c009260 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:53.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.082+0000 7efd70b9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612495f2be0 0x5612495fd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:53.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.082+0000 7efd7139b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612495f22d0 0x5612495f26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:53.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.082+0000 7efd70b9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612495f2be0 0x5612495fd590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:39496/0 (socket says 172.21.15.101:39496) 2023-07-18T03:17:53.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.082+0000 7efd7139b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612495f22d0 0x5612495f26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:35926/0 (socket says 172.21.15.101:35926) 2023-07-18T03:17:53.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.082+0000 7efd70b9a700 1 -- 172.21.15.101:0/1606660288 learned_addr learned my addr 172.21.15.101:0/1606660288 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:53.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.082+0000 7efd71b9c700 1 --2- 172.21.15.101:0/1606660288 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5612495fdad0 0x5612495ffeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:53.081 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.083+0000 7efd7139b700 1 -- 172.21.15.101:0/1606660288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612495f2be0 msgr2=0x5612495fd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:53.081 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.083+0000 7efd7139b700 1 --2- 172.21.15.101:0/1606660288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612495f2be0 0x5612495fd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.081 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.083+0000 7efd7139b700 1 -- 172.21.15.101:0/1606660288 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5612495fdad0 msgr2=0x5612495ffeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:53.081 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.083+0000 7efd7139b700 1 --2- 172.21.15.101:0/1606660288 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5612495fdad0 0x5612495ffeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.081 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.083+0000 7efd7139b700 1 -- 172.21.15.101:0/1606660288 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561249372de0 con 0x5612495f22d0 2023-07-18T03:17:53.081 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.083+0000 7efd70b9a700 1 --2- 172.21.15.101:0/1606660288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612495f2be0 0x5612495fd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:17:53.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.083+0000 7efd71b9c700 1 --2- 172.21.15.101:0/1606660288 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5612495fdad0 0x5612495ffeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:17:53.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.083+0000 7efd7139b700 1 --2- 172.21.15.101:0/1606660288 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612495f22d0 0x5612495f26a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7efd680ca9d0 tx=0x7efd6800ce80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2461e8931e511c1c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:53.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.083+0000 7efd577fe700 1 -- 172.21.15.101:0/1606660288 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd680a27a0 con 0x5612495f22d0 2023-07-18T03:17:53.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.083+0000 7efd577fe700 1 -- 172.21.15.101:0/1606660288 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd680986f0 con 0x5612495f22d0 2023-07-18T03:17:53.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.083+0000 7efd577fe700 1 -- 172.21.15.101:0/1606660288 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd68099840 con 0x5612495f22d0 2023-07-18T03:17:53.085 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.084+0000 7efd86e60b80 1 -- 172.21.15.101:0/1606660288 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612495f22d0 msgr2=0x5612495f26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:53.086 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.084+0000 7efd86e60b80 1 --2- 172.21.15.101:0/1606660288 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612495f22d0 0x5612495f26a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7efd680ca9d0 tx=0x7efd6800ce80 comp rx=0 tx=0).stop 2023-07-18T03:17:53.086 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.084+0000 7efd86e60b80 1 -- 172.21.15.101:0/1606660288 shutdown_connections 2023-07-18T03:17:53.086 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.084+0000 7efd86e60b80 1 --2- 172.21.15.101:0/1606660288 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612495f22d0 0x5612495f26a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.086 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.084+0000 7efd86e60b80 1 --2- 172.21.15.101:0/1606660288 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5612495fdad0 0x5612495ffeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.086 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.084+0000 7efd86e60b80 1 --2- 172.21.15.101:0/1606660288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612495f2be0 0x5612495fd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.086 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.084+0000 7efd86e60b80 1 -- 172.21.15.101:0/1606660288 >> 172.21.15.101:0/1606660288 conn(0x5612495e8da0 msgr2=0x5612495ee6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:53.086 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.084+0000 7efd86e60b80 1 -- 172.21.15.101:0/1606660288 shutdown_connections 2023-07-18T03:17:53.087 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.084+0000 7efd86e60b80 1 -- 172.21.15.101:0/1606660288 wait complete. 2023-07-18T03:17:53.087 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.085+0000 7efd86e60b80 1 -- start start 2023-07-18T03:17:53.087 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.085+0000 7efd86e60b80 1 Processor -- start 2023-07-18T03:17:53.087 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.085+0000 7efd86e60b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612495f22d0 0x5612495fd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:53.087 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.085+0000 7efd86e60b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612495f2be0 0x5612495fa860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:53.087 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.085+0000 7efd86e60b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5612495fdad0 0x5612495fafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:53.087 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.085+0000 7efd86e60b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612494e4ac0 con 0x5612495fdad0 2023-07-18T03:17:53.088 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.085+0000 7efd86e60b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612494e47c0 con 0x5612495f2be0 2023-07-18T03:17:53.088 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.085+0000 7efd86e60b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612493c8920 con 0x5612495f22d0 2023-07-18T03:17:53.088 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.085+0000 7efd86e60b80 10 client.0 _ll_register_callbacks cb 0x5612494e2760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:17:53.088 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.086+0000 7efd7139b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612495f22d0 0x5612495fd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:53.089 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.086+0000 7efd70b9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612495f2be0 0x5612495fa860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:53.089 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.086+0000 7efd7139b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612495f22d0 0x5612495fd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:39512/0 (socket says 172.21.15.101:39512) 2023-07-18T03:17:53.089 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.086+0000 7efd7139b700 1 -- 172.21.15.101:0/2707002651 learned_addr learned my addr 172.21.15.101:0/2707002651 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:53.089 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.086+0000 7efd71b9c700 1 --2- 172.21.15.101:0/2707002651 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5612495fdad0 0x5612495fafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:53.090 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.086+0000 7efd7139b700 1 -- 172.21.15.101:0/2707002651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612495f2be0 msgr2=0x5612495fa860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:53.090 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.086+0000 7efd7139b700 1 --2- 172.21.15.101:0/2707002651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612495f2be0 0x5612495fa860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.090 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.086+0000 7efd7139b700 1 -- 172.21.15.101:0/2707002651 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5612495fdad0 msgr2=0x5612495fafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:53.090 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.086+0000 7efd7139b700 1 --2- 172.21.15.101:0/2707002651 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5612495fdad0 0x5612495fafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.091 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.086+0000 7efd7139b700 1 -- 172.21.15.101:0/2707002651 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd5c017070 con 0x5612495f22d0 2023-07-18T03:17:53.091 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.086+0000 7efd70b9a700 1 --2- 172.21.15.101:0/2707002651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612495f2be0 0x5612495fa860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:17:53.091 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.086+0000 7efd7139b700 1 --2- 172.21.15.101:0/2707002651 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612495f22d0 0x5612495fd570 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7efd6809a2e0 tx=0x7efd68099a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7fb19412c849b58 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:53.091 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.086+0000 7efd54ff9700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:17:53.091 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.086+0000 7efd54ff9700 1 -- 172.21.15.101:0/2707002651 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd680a27a0 con 0x5612495f22d0 2023-07-18T03:17:53.091 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.086+0000 7efd54ff9700 1 -- 172.21.15.101:0/2707002651 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7efd6809e120 con 0x5612495f22d0 2023-07-18T03:17:53.092 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.086+0000 7efd86e60b80 1 -- 172.21.15.101:0/2707002651 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56124949e4f0 con 0x5612495f22d0 2023-07-18T03:17:53.092 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.087+0000 7efd54ff9700 1 -- 172.21.15.101:0/2707002651 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd6809a7c0 con 0x5612495f22d0 2023-07-18T03:17:53.092 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.087+0000 7efd86e60b80 10 client.27652 fetch_fsmap learned FSMap version 33 2023-07-18T03:17:53.092 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.087+0000 7efd86e60b80 1 -- 172.21.15.101:0/2707002651 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7efd6c1a4b20 con 0x5612495f22d0 2023-07-18T03:17:53.092 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.087+0000 7efd54ff9700 1 -- 172.21.15.101:0/2707002651 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56124949e4f0 con 0x5612495f22d0 2023-07-18T03:17:53.092 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.087+0000 7efd54ff9700 1 -- 172.21.15.101:0/2707002651 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efd68018b60 con 0x5612495f22d0 2023-07-18T03:17:53.092 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.087+0000 7efd86e60b80 10 client.27652 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:17:53.093 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.087+0000 7efd86e60b80 10 client.27652 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:17:53.093 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.087+0000 7efd86e60b80 20 client.27652 populate_metadata read hostname 'smithi101' 2023-07-18T03:17:53.093 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.087+0000 7efd86e60b80 1 --2- 172.21.15.101:0/2707002651 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56124960cf70 0x56124960f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:53.093 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.087+0000 7efd86e60b80 4 client.27652 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:17:53.093 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.087+0000 7efd86e60b80 1 -- 172.21.15.101:0/2707002651 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56124953deb0 con 0x56124960cf70 2023-07-18T03:17:53.093 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.087+0000 7efd70b9a700 1 --2- 172.21.15.101:0/2707002651 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56124960cf70 0x56124960f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:53.093 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.088+0000 7efd70b9a700 1 --2- 172.21.15.101:0/2707002651 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56124960cf70 0x56124960f380 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6131a3987f11ff86 server_cookie=422bba4980f8c038 in_seq=0 out_seq=0 2023-07-18T03:17:53.094 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.088+0000 7efd54ff9700 10 client.27652 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:17:53.094 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.088+0000 7efd54ff9700 1 -- 172.21.15.101:0/2707002651 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56124953deb0 con 0x56124960cf70 2023-07-18T03:17:53.094 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.089+0000 7efd54ff9700 10 client.27652 handle_command_reply: tid=0 2023-07-18T03:17:53.094 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.090+0000 7efd86e60b80 1 client.27652 shutdown 2023-07-18T03:17:53.094 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.090+0000 7efd86e60b80 1 -- 172.21.15.101:0/2707002651 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612495f22d0 msgr2=0x5612495fd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:53.094 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.090+0000 7efd86e60b80 1 --2- 172.21.15.101:0/2707002651 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612495f22d0 0x5612495fd570 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7efd6809a2e0 tx=0x7efd68099a70 comp rx=0 tx=0).stop 2023-07-18T03:17:53.094 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.090+0000 7efd86e60b80 1 -- 172.21.15.101:0/2707002651 shutdown_connections 2023-07-18T03:17:53.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.090+0000 7efd86e60b80 1 --2- 172.21.15.101:0/2707002651 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56124960cf70 0x56124960f380 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.090+0000 7efd86e60b80 1 --2- 172.21.15.101:0/2707002651 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5612495fdad0 0x5612495fafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.090+0000 7efd86e60b80 1 --2- 172.21.15.101:0/2707002651 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612495f22d0 0x5612495fd570 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.090+0000 7efd86e60b80 1 --2- 172.21.15.101:0/2707002651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612495f2be0 0x5612495fa860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.090+0000 7efd54ff9700 0 client.27652 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:17:53.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.090+0000 7efd86e60b80 1 -- 172.21.15.101:0/2707002651 >> 172.21.15.101:0/2707002651 conn(0x5612495e8da0 msgr2=0x5612495ee230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:53.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.090+0000 7efd86e60b80 1 -- 172.21.15.101:0/2707002651 shutdown_connections 2023-07-18T03:17:53.096 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.090+0000 7efd86e60b80 1 -- 172.21.15.101:0/2707002651 wait complete. 2023-07-18T03:17:53.096 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.091+0000 7efd86e60b80 20 client.27652 trim_cache size 0 max 16384 2023-07-18T03:17:53.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.108+0000 7efd86e60b80 1 Processor -- start 2023-07-18T03:17:53.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.108+0000 7efd86e60b80 1 -- start start 2023-07-18T03:17:53.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.108+0000 7efd86e60b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5612495f22d0 0x5612495f5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:53.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.108+0000 7efd86e60b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612495f2be0 0x5612495f60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:53.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.108+0000 7efd86e60b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612495fdad0 0x5612495fa020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:53.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.109+0000 7efd86e60b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612495fb7e0 con 0x5612495f22d0 2023-07-18T03:17:53.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.109+0000 7efd86e60b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612495efce0 con 0x5612495fdad0 2023-07-18T03:17:53.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.109+0000 7efd86e60b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612495e8970 con 0x5612495f2be0 2023-07-18T03:17:53.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.109+0000 7efd7139b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5612495f22d0 0x5612495f5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:53.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.109+0000 7efd71b9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612495fdad0 0x5612495fa020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:53.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.109+0000 7efd70b9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612495f2be0 0x5612495f60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:53.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.109+0000 7efd7139b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5612495f22d0 0x5612495f5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:57812/0 (socket says 172.21.15.101:57812) 2023-07-18T03:17:53.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.109+0000 7efd7139b700 1 -- 172.21.15.101:0/2056396532 learned_addr learned my addr 172.21.15.101:0/2056396532 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:53.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.109+0000 7efd71b9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612495fdad0 0x5612495fa020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:35948/0 (socket says 172.21.15.101:35948) 2023-07-18T03:17:53.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.109+0000 7efd70b9a700 1 -- 172.21.15.101:0/2056396532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612495fdad0 msgr2=0x5612495fa020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:53.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.109+0000 7efd70b9a700 1 --2- 172.21.15.101:0/2056396532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612495fdad0 0x5612495fa020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.109+0000 7efd70b9a700 1 -- 172.21.15.101:0/2056396532 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5612495f22d0 msgr2=0x5612495f5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:53.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.109+0000 7efd70b9a700 1 --2- 172.21.15.101:0/2056396532 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5612495f22d0 0x5612495f5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.109+0000 7efd70b9a700 1 -- 172.21.15.101:0/2056396532 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56124935bc60 con 0x5612495f2be0 2023-07-18T03:17:53.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.109+0000 7efd71b9c700 1 --2- 172.21.15.101:0/2056396532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612495fdad0 0x5612495fa020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:17:53.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.110+0000 7efd70b9a700 1 --2- 172.21.15.101:0/2056396532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612495f2be0 0x5612495f60b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x56124960fc90 tx=0x7efd5c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9a1528fb5e55fd2 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:53.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.110+0000 7efd57fff700 1 -- 172.21.15.101:0/2056396532 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5c022030 con 0x5612495f2be0 2023-07-18T03:17:53.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.110+0000 7efd57fff700 1 -- 172.21.15.101:0/2056396532 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd5c023040 con 0x5612495f2be0 2023-07-18T03:17:53.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.110+0000 7efd57fff700 1 -- 172.21.15.101:0/2056396532 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5c01d670 con 0x5612495f2be0 2023-07-18T03:17:53.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.111+0000 7efd86e60b80 1 -- 172.21.15.101:0/2056396532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612495f2be0 msgr2=0x5612495f60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:53.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.111+0000 7efd86e60b80 1 --2- 172.21.15.101:0/2056396532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612495f2be0 0x5612495f60b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x56124960fc90 tx=0x7efd5c014000 comp rx=0 tx=0).stop 2023-07-18T03:17:53.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.111+0000 7efd86e60b80 1 -- 172.21.15.101:0/2056396532 shutdown_connections 2023-07-18T03:17:53.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.111+0000 7efd86e60b80 1 --2- 172.21.15.101:0/2056396532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612495fdad0 0x5612495fa020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.111+0000 7efd86e60b80 1 --2- 172.21.15.101:0/2056396532 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5612495f22d0 0x5612495f5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.111+0000 7efd86e60b80 1 --2- 172.21.15.101:0/2056396532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612495f2be0 0x5612495f60b0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.111+0000 7efd86e60b80 1 -- 172.21.15.101:0/2056396532 >> 172.21.15.101:0/2056396532 conn(0x5612495e8da0 msgr2=0x561249609700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:53.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.111+0000 7efd86e60b80 1 -- 172.21.15.101:0/2056396532 shutdown_connections 2023-07-18T03:17:53.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.111+0000 7efd86e60b80 1 -- 172.21.15.101:0/2056396532 wait complete. 2023-07-18T03:17:53.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.111+0000 7efd86e60b80 1 -- start start 2023-07-18T03:17:53.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.111+0000 7efd86e60b80 1 Processor -- start 2023-07-18T03:17:53.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.112+0000 7efd86e60b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612495f22d0 0x561249604480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:53.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.112+0000 7efd86e60b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5612495f2be0 0x561249602740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:53.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.112+0000 7efd86e60b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612495fdad0 0x561249602c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:53.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.112+0000 7efd86e60b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561249603390 con 0x5612495f2be0 2023-07-18T03:17:53.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.112+0000 7efd86e60b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561249603500 con 0x5612495fdad0 2023-07-18T03:17:53.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.112+0000 7efd86e60b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x561249660e40 con 0x5612495f22d0 2023-07-18T03:17:53.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.112+0000 7efd86e60b80 10 client.0 _ll_register_callbacks cb 0x7efd5c011a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:17:53.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.112+0000 7efd7139b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612495f22d0 0x561249604480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:53.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.112+0000 7efd7139b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612495f22d0 0x561249604480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:39544/0 (socket says 172.21.15.101:39544) 2023-07-18T03:17:53.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.112+0000 7efd7139b700 1 -- 172.21.15.101:0/1216669183 learned_addr learned my addr 172.21.15.101:0/1216669183 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:53.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.112+0000 7efd70b9a700 1 --2- 172.21.15.101:0/1216669183 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5612495f2be0 0x561249602740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:53.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.112+0000 7efd71b9c700 1 --2- 172.21.15.101:0/1216669183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612495fdad0 0x561249602c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:53.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.112+0000 7efd7139b700 1 -- 172.21.15.101:0/1216669183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612495fdad0 msgr2=0x561249602c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:53.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.112+0000 7efd7139b700 1 --2- 172.21.15.101:0/1216669183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612495fdad0 0x561249602c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.112+0000 7efd7139b700 1 -- 172.21.15.101:0/1216669183 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5612495f2be0 msgr2=0x561249602740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:53.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.112+0000 7efd7139b700 1 --2- 172.21.15.101:0/1216669183 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5612495f2be0 0x561249602740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.112+0000 7efd7139b700 1 -- 172.21.15.101:0/1216669183 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd6c10c110 con 0x5612495f22d0 2023-07-18T03:17:53.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.113+0000 7efd7139b700 1 --2- 172.21.15.101:0/1216669183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612495f22d0 0x561249604480 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7efd68010d10 tx=0x7efd680cd550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=534e1f4a0e994570 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:53.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.113+0000 7efd54ff9700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:17:53.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.113+0000 7efd54ff9700 1 -- 172.21.15.101:0/1216669183 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd68099ae0 con 0x5612495f22d0 2023-07-18T03:17:53.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.113+0000 7efd54ff9700 1 -- 172.21.15.101:0/1216669183 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7efd680a27a0 con 0x5612495f22d0 2023-07-18T03:17:53.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.113+0000 7efd86e60b80 1 -- 172.21.15.101:0/1216669183 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efd680986f0 con 0x5612495f22d0 2023-07-18T03:17:53.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.113+0000 7efd54ff9700 1 -- 172.21.15.101:0/1216669183 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd68096690 con 0x5612495f22d0 2023-07-18T03:17:53.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.113+0000 7efd86e60b80 10 client.27658 fetch_fsmap learned FSMap version 33 2023-07-18T03:17:53.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.113+0000 7efd86e60b80 1 -- 172.21.15.101:0/1216669183 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7efd5c004360 con 0x5612495f22d0 2023-07-18T03:17:53.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.113+0000 7efd54ff9700 1 -- 172.21.15.101:0/1216669183 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd680986f0 con 0x5612495f22d0 2023-07-18T03:17:53.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.113+0000 7efd54ff9700 1 -- 172.21.15.101:0/1216669183 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efd680a2310 con 0x5612495f22d0 2023-07-18T03:17:53.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.113+0000 7efd86e60b80 10 client.27658 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:17:53.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.113+0000 7efd86e60b80 10 client.27658 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:17:53.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.113+0000 7efd86e60b80 20 client.27658 populate_metadata read hostname 'smithi101' 2023-07-18T03:17:53.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.113+0000 7efd86e60b80 1 --2- 172.21.15.101:0/1216669183 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x561249661f20 0x561249664300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:53.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.113+0000 7efd86e60b80 4 client.27658 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:17:53.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.113+0000 7efd86e60b80 1 -- 172.21.15.101:0/1216669183 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561249500050 con 0x561249661f20 2023-07-18T03:17:53.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.114+0000 7efd70b9a700 1 --2- 172.21.15.101:0/1216669183 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x561249661f20 0x561249664300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:53.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.114+0000 7efd54ff9700 10 client.27658 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:17:53.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.114+0000 7efd70b9a700 1 --2- 172.21.15.101:0/1216669183 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x561249661f20 0x561249664300 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6857c147117136d server_cookie=e633e39098767789 in_seq=0 out_seq=0 2023-07-18T03:17:53.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.114+0000 7efd54ff9700 1 -- 172.21.15.101:0/1216669183 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561249500050 con 0x561249661f20 2023-07-18T03:17:53.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.114+0000 7efd54ff9700 10 client.27658 handle_command_reply: tid=0 2023-07-18T03:17:53.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.114+0000 7efd86e60b80 1 client.27658 shutdown 2023-07-18T03:17:53.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.115+0000 7efd86e60b80 1 -- 172.21.15.101:0/1216669183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612495f22d0 msgr2=0x561249604480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:53.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.115+0000 7efd86e60b80 1 --2- 172.21.15.101:0/1216669183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612495f22d0 0x561249604480 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7efd68010d10 tx=0x7efd680cd550 comp rx=0 tx=0).stop 2023-07-18T03:17:53.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.115+0000 7efd86e60b80 1 -- 172.21.15.101:0/1216669183 shutdown_connections 2023-07-18T03:17:53.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.115+0000 7efd86e60b80 1 --2- 172.21.15.101:0/1216669183 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x561249661f20 0x561249664300 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.115+0000 7efd86e60b80 1 --2- 172.21.15.101:0/1216669183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612495fdad0 0x561249602c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.115+0000 7efd86e60b80 1 --2- 172.21.15.101:0/1216669183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612495f22d0 0x561249604480 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.115+0000 7efd54ff9700 0 client.27658 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:17:53.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.115+0000 7efd86e60b80 1 --2- 172.21.15.101:0/1216669183 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5612495f2be0 0x561249602740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.115+0000 7efd86e60b80 1 -- 172.21.15.101:0/1216669183 >> 172.21.15.101:0/1216669183 conn(0x5612495e8da0 msgr2=0x5612495f5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:53.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.115+0000 7efd86e60b80 1 -- 172.21.15.101:0/1216669183 shutdown_connections 2023-07-18T03:17:53.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.115+0000 7efd86e60b80 1 -- 172.21.15.101:0/1216669183 wait complete. 2023-07-18T03:17:53.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.115+0000 7efd86e60b80 20 client.27658 trim_cache size 0 max 16384 2023-07-18T03:17:53.125 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:17:53.125 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "no active scrubs running", 2023-07-18T03:17:53.125 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": {} 2023-07-18T03:17:53.126 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:17:53.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.115+0000 7efd735ff700 1 -- 172.21.15.101:0/3454941036 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7efd580a1340 msgr2=0x7efd580a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:53.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.115+0000 7efd735ff700 1 --2- 172.21.15.101:0/3454941036 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7efd580a1340 0x7efd580a37f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efd6c1994e0 tx=0x7efd5c009260 comp rx=0 tx=0).stop 2023-07-18T03:17:53.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.115+0000 7efd735ff700 1 -- 172.21.15.101:0/3454941036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd6c0fe9d0 msgr2=0x7efd6c197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:53.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.115+0000 7efd735ff700 1 --2- 172.21.15.101:0/3454941036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd6c0fe9d0 0x7efd6c197ed0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7efd6800ebf0 tx=0x7efd6800ef00 comp rx=0 tx=0).stop 2023-07-18T03:17:53.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.116+0000 7efd735ff700 1 -- 172.21.15.101:0/3454941036 shutdown_connections 2023-07-18T03:17:53.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.116+0000 7efd735ff700 1 --2- 172.21.15.101:0/3454941036 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7efd580a1340 0x7efd580a37f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.116+0000 7efd735ff700 1 --2- 172.21.15.101:0/3454941036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd6c1049e0 0x7efd6c19c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.116+0000 7efd735ff700 1 --2- 172.21.15.101:0/3454941036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd6c0fe9d0 0x7efd6c197ed0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.116+0000 7efd735ff700 1 --2- 172.21.15.101:0/3454941036 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efd6c0ff380 0x7efd6c198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.116+0000 7efd735ff700 1 -- 172.21.15.101:0/3454941036 >> 172.21.15.101:0/3454941036 conn(0x7efd6c0fa4b0 msgr2=0x7efd6c067690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:53.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.116+0000 7efd735ff700 1 -- 172.21.15.101:0/3454941036 shutdown_connections 2023-07-18T03:17:53.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.116+0000 7efd735ff700 1 -- 172.21.15.101:0/3454941036 wait complete. 2023-07-18T03:17:53.130 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-18T03:17:53.130 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:17:53.192 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:53 smithi101 ceph-mon[126379]: pgmap v320: 129 pgs: 129 active+clean; 509 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 69 op/s 2023-07-18T03:17:53.300 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:53 smithi155 ceph-mon[128885]: pgmap v320: 129 pgs: 129 active+clean; 509 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 69 op/s 2023-07-18T03:17:53.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.341+0000 7fcb28bff700 1 -- 172.21.15.101:0/1597359706 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcb240728b0 msgr2=0x7fcb24072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:53.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.341+0000 7fcb28bff700 1 --2- 172.21.15.101:0/1597359706 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcb240728b0 0x7fcb24072c80 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fcb14009990 tx=0x7fcb14009ca0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.341+0000 7fcb28bff700 1 -- 172.21.15.101:0/1597359706 shutdown_connections 2023-07-18T03:17:53.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.341+0000 7fcb28bff700 1 --2- 172.21.15.101:0/1597359706 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb2410c330 0x7fcb2410e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.338 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.341+0000 7fcb28bff700 1 --2- 172.21.15.101:0/1597359706 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcb240728b0 0x7fcb24072c80 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.338 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.341+0000 7fcb28bff700 1 --2- 172.21.15.101:0/1597359706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb240731c0 0x7fcb2410bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.338 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.341+0000 7fcb28bff700 1 -- 172.21.15.101:0/1597359706 >> 172.21.15.101:0/1597359706 conn(0x7fcb240fb630 msgr2=0x7fcb240fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:53.338 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.341+0000 7fcb28bff700 1 -- 172.21.15.101:0/1597359706 shutdown_connections 2023-07-18T03:17:53.338 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.342+0000 7fcb28bff700 1 -- 172.21.15.101:0/1597359706 wait complete. 2023-07-18T03:17:53.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.342+0000 7fcb28bff700 1 Processor -- start 2023-07-18T03:17:53.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.343+0000 7fcb28bff700 1 -- start start 2023-07-18T03:17:53.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.343+0000 7fcb28bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcb240728b0 0x7fcb24197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:53.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.343+0000 7fcb28bff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb240731c0 0x7fcb24198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:53.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.343+0000 7fcb28bff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb2410c330 0x7fcb2419c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:53.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.343+0000 7fcb28bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2419ce00 con 0x7fcb240728b0 2023-07-18T03:17:53.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.343+0000 7fcb28bff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2419cf70 con 0x7fcb240731c0 2023-07-18T03:17:53.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.343+0000 7fcb28bff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2419d130 con 0x7fcb2410c330 2023-07-18T03:17:53.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.343+0000 7fcb21d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb240731c0 0x7fcb24198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:53.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.343+0000 7fcb2259c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcb240728b0 0x7fcb24197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:53.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.343+0000 7fcb21d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb240731c0 0x7fcb24198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:35974/0 (socket says 172.21.15.101:35974) 2023-07-18T03:17:53.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.343+0000 7fcb21d9b700 1 -- 172.21.15.101:0/1578441379 learned_addr learned my addr 172.21.15.101:0/1578441379 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:53.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.343+0000 7fcb22d9d700 1 --2- 172.21.15.101:0/1578441379 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb2410c330 0x7fcb2419c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:53.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.344+0000 7fcb21d9b700 1 -- 172.21.15.101:0/1578441379 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb2410c330 msgr2=0x7fcb2419c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:53.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.344+0000 7fcb21d9b700 1 --2- 172.21.15.101:0/1578441379 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb2410c330 0x7fcb2419c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.344+0000 7fcb21d9b700 1 -- 172.21.15.101:0/1578441379 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcb240728b0 msgr2=0x7fcb24197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:53.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.344+0000 7fcb21d9b700 1 --2- 172.21.15.101:0/1578441379 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcb240728b0 0x7fcb24197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.344+0000 7fcb21d9b700 1 -- 172.21.15.101:0/1578441379 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb140095f0 con 0x7fcb240731c0 2023-07-18T03:17:53.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.344+0000 7fcb2259c700 1 --2- 172.21.15.101:0/1578441379 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcb240728b0 0x7fcb24197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:17:53.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.344+0000 7fcb21d9b700 1 --2- 172.21.15.101:0/1578441379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb240731c0 0x7fcb24198360 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fcb1800e9a0 tx=0x7fcb1800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:53.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.344+0000 7fcb137fe700 1 -- 172.21.15.101:0/1578441379 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb18009790 con 0x7fcb240731c0 2023-07-18T03:17:53.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.344+0000 7fcb28bff700 1 -- 172.21.15.101:0/1578441379 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb2419d410 con 0x7fcb240731c0 2023-07-18T03:17:53.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.344+0000 7fcb137fe700 1 -- 172.21.15.101:0/1578441379 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb18010430 con 0x7fcb240731c0 2023-07-18T03:17:53.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.344+0000 7fcb137fe700 1 -- 172.21.15.101:0/1578441379 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb18018680 con 0x7fcb240731c0 2023-07-18T03:17:53.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.344+0000 7fcb28bff700 1 -- 172.21.15.101:0/1578441379 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb241a4c50 con 0x7fcb240731c0 2023-07-18T03:17:53.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.345+0000 7fcb28bff700 1 -- 172.21.15.101:0/1578441379 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb24067300 con 0x7fcb240731c0 2023-07-18T03:17:53.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.348+0000 7fcb137fe700 1 -- 172.21.15.101:0/1578441379 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fcb180105a0 con 0x7fcb240731c0 2023-07-18T03:17:53.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.349+0000 7fcb137fe700 1 --2- 172.21.15.101:0/1578441379 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fcb0c0a1300 0x7fcb0c0a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:53.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.349+0000 7fcb137fe700 1 -- 172.21.15.101:0/1578441379 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fcb18014070 con 0x7fcb240731c0 2023-07-18T03:17:53.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.350+0000 7fcb2259c700 1 --2- 172.21.15.101:0/1578441379 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fcb0c0a1300 0x7fcb0c0a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:53.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.350+0000 7fcb2259c700 1 --2- 172.21.15.101:0/1578441379 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fcb0c0a1300 0x7fcb0c0a37b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcb1400b2e0 tx=0x7fcb14005fb0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:53.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.351+0000 7fcb137fe700 1 -- 172.21.15.101:0/1578441379 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fcb180962a0 con 0x7fcb240731c0 2023-07-18T03:17:53.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:53 smithi179 ceph-mon[129229]: pgmap v320: 129 pgs: 129 active+clean; 509 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 69 op/s 2023-07-18T03:17:53.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.620+0000 7fcb28bff700 1 -- 172.21.15.101:0/1578441379 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb24199170 con 0x7fcb240731c0 2023-07-18T03:17:53.616 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.621+0000 7fcb137fe700 1 -- 172.21.15.101:0/1578441379 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+10858 (secure 0 0 0) 0x7fcb180998c0 con 0x7fcb240731c0 2023-07-18T03:17:53.616 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:17:53.616 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/4192846769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":4192846769},{"type":"v1","addr":"172.21.15.101:6839","nonce":4192846769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6841/3824340175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6840","nonce":3824340175},{"type":"v1","addr":"172.21.15.101:6841","nonce":3824340175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:04:59.361185+0000","modified":"2023-07-18T03:16:36.002358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.155:6839/2055462483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2055462483},{"type":"v1","addr":"172.21.15.155:6839","nonce":2055462483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:17:53.617 INFO:tasks.workunit.client.0.smithi101.stdout: 1 648042 5.78 MB/sec execute 438 sec latency 1584.381 ms 2023-07-18T03:17:53.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.623+0000 7fcb28bff700 1 -- 172.21.15.101:0/1578441379 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fcb0c0a1300 msgr2=0x7fcb0c0a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:53.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.623+0000 7fcb28bff700 1 --2- 172.21.15.101:0/1578441379 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fcb0c0a1300 0x7fcb0c0a37b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcb1400b2e0 tx=0x7fcb14005fb0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.623+0000 7fcb28bff700 1 -- 172.21.15.101:0/1578441379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb240731c0 msgr2=0x7fcb24198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:53.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.623+0000 7fcb28bff700 1 --2- 172.21.15.101:0/1578441379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb240731c0 0x7fcb24198360 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fcb1800e9a0 tx=0x7fcb1800ecb0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.623+0000 7fcb28bff700 1 -- 172.21.15.101:0/1578441379 shutdown_connections 2023-07-18T03:17:53.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.623+0000 7fcb28bff700 1 --2- 172.21.15.101:0/1578441379 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb2410c330 0x7fcb2419c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.623+0000 7fcb28bff700 1 --2- 172.21.15.101:0/1578441379 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcb240728b0 0x7fcb24197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.623+0000 7fcb28bff700 1 --2- 172.21.15.101:0/1578441379 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fcb0c0a1300 0x7fcb0c0a37b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.623+0000 7fcb28bff700 1 --2- 172.21.15.101:0/1578441379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb240731c0 0x7fcb24198360 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.623+0000 7fcb28bff700 1 -- 172.21.15.101:0/1578441379 >> 172.21.15.101:0/1578441379 conn(0x7fcb240fb630 msgr2=0x7fcb2410e400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:53.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.623+0000 7fcb28bff700 1 -- 172.21.15.101:0/1578441379 shutdown_connections 2023-07-18T03:17:53.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.623+0000 7fcb28bff700 1 -- 172.21.15.101:0/1578441379 wait complete. 2023-07-18T03:17:53.620 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 33 2023-07-18T03:17:53.638 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-18T03:17:53.853 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.857+0000 7f6c05dff700 1 -- 172.21.15.101:0/473330283 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c000ffad0 msgr2=0x7f6c000fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:53.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.857+0000 7f6c05dff700 1 --2- 172.21.15.101:0/473330283 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c000ffad0 0x7f6c000fff40 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f6bf40098f0 tx=0x7f6bf4009c00 comp rx=0 tx=0).stop 2023-07-18T03:17:53.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.858+0000 7f6c05dff700 1 -- 172.21.15.101:0/473330283 shutdown_connections 2023-07-18T03:17:53.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.858+0000 7f6c05dff700 1 --2- 172.21.15.101:0/473330283 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c00105ae0 0x7f6c00105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.858+0000 7f6c05dff700 1 --2- 172.21.15.101:0/473330283 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c00100480 0x7f6c0010c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.858+0000 7f6c05dff700 1 --2- 172.21.15.101:0/473330283 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c000ffad0 0x7f6c000fff40 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.858+0000 7f6c05dff700 1 -- 172.21.15.101:0/473330283 >> 172.21.15.101:0/473330283 conn(0x7f6c000fb5f0 msgr2=0x7f6c000fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:53.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.858+0000 7f6c05dff700 1 -- 172.21.15.101:0/473330283 shutdown_connections 2023-07-18T03:17:53.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.858+0000 7f6c05dff700 1 -- 172.21.15.101:0/473330283 wait complete. 2023-07-18T03:17:53.857 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.859+0000 7f6c05dff700 1 Processor -- start 2023-07-18T03:17:53.857 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.859+0000 7f6c05dff700 1 -- start start 2023-07-18T03:17:53.857 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.859+0000 7f6c05dff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c000ffad0 0x7f6c00197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:53.857 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.859+0000 7f6c05dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c00100480 0x7f6c00198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:53.858 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.859+0000 7f6c05dff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c00105ae0 0x7f6c0019c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:53.858 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.859+0000 7f6c05dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0019ce30 con 0x7f6c00100480 2023-07-18T03:17:53.858 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.859+0000 7f6c05dff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0019cfa0 con 0x7f6c00105ae0 2023-07-18T03:17:53.859 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.859+0000 7f6c05dff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0019d160 con 0x7f6c000ffad0 2023-07-18T03:17:53.859 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.860+0000 7f6bfeffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c00100480 0x7f6c00198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:53.859 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.860+0000 7f6bfeffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c00100480 0x7f6c00198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:57852/0 (socket says 172.21.15.101:57852) 2023-07-18T03:17:53.859 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.860+0000 7f6bfeffd700 1 -- 172.21.15.101:0/3081269760 learned_addr learned my addr 172.21.15.101:0/3081269760 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:53.860 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.860+0000 7f6bff7fe700 1 --2- 172.21.15.101:0/3081269760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c000ffad0 0x7f6c00197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:53.860 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.860+0000 7f6bfffff700 1 --2- 172.21.15.101:0/3081269760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c00105ae0 0x7f6c0019c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:53.860 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.860+0000 7f6bfeffd700 1 -- 172.21.15.101:0/3081269760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c000ffad0 msgr2=0x7f6c00197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:53.861 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.860+0000 7f6bfeffd700 1 --2- 172.21.15.101:0/3081269760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c000ffad0 0x7f6c00197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.861 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.860+0000 7f6bfeffd700 1 -- 172.21.15.101:0/3081269760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c00105ae0 msgr2=0x7f6c0019c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:53.861 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.860+0000 7f6bfeffd700 1 --2- 172.21.15.101:0/3081269760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c00105ae0 0x7f6c0019c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.860+0000 7f6bfeffd700 1 -- 172.21.15.101:0/3081269760 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf0009d00 con 0x7f6c00100480 2023-07-18T03:17:53.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.860+0000 7f6bfffff700 1 --2- 172.21.15.101:0/3081269760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c00105ae0 0x7f6c0019c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:17:53.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.860+0000 7f6bfeffd700 1 --2- 172.21.15.101:0/3081269760 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c00100480 0x7f6c00198390 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4009810 tx=0x7f6bf400fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:53.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.861+0000 7f6bfcff9700 1 -- 172.21.15.101:0/3081269760 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf401c070 con 0x7f6c00100480 2023-07-18T03:17:53.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.861+0000 7f6bfcff9700 1 -- 172.21.15.101:0/3081269760 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6bf400bb20 con 0x7f6c00100480 2023-07-18T03:17:53.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.861+0000 7f6c05dff700 1 -- 172.21.15.101:0/3081269760 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bf4009520 con 0x7f6c00100480 2023-07-18T03:17:53.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.861+0000 7f6bfcff9700 1 -- 172.21.15.101:0/3081269760 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf4004930 con 0x7f6c00100480 2023-07-18T03:17:53.864 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.861+0000 7f6c05dff700 1 -- 172.21.15.101:0/3081269760 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c0019d670 con 0x7f6c00100480 2023-07-18T03:17:53.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.864+0000 7f6bfcff9700 1 -- 172.21.15.101:0/3081269760 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f6bf4020070 con 0x7f6c00100480 2023-07-18T03:17:53.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.865+0000 7f6bfcff9700 1 --2- 172.21.15.101:0/3081269760 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6be80a12c0 0x7f6be80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:53.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.865+0000 7f6bfcff9700 1 -- 172.21.15.101:0/3081269760 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f6bf40cdda0 con 0x7f6c00100480 2023-07-18T03:17:53.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.865+0000 7f6c19740b80 1 Processor -- start 2023-07-18T03:17:53.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.865+0000 7f6bff7fe700 1 --2- 172.21.15.101:0/3081269760 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6be80a12c0 0x7f6be80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:53.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.866+0000 7f6c19740b80 1 -- start start 2023-07-18T03:17:53.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.866+0000 7f6c19740b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561bb6b232d0 0x561bb6b236a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:53.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.866+0000 7f6c19740b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561bb6b23be0 0x561bb6b2e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:53.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.866+0000 7f6c19740b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561bb6b2ead0 0x561bb6b30eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:53.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.866+0000 7f6c19740b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bb6a29cb0 con 0x561bb6b2ead0 2023-07-18T03:17:53.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.866+0000 7f6c19740b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bb6a7ca40 con 0x561bb6b23be0 2023-07-18T03:17:53.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.866+0000 7f6c19740b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bb6a75ed0 con 0x561bb6b232d0 2023-07-18T03:17:53.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.867+0000 7f6bfeffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561bb6b23be0 0x561bb6b2e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:53.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.867+0000 7f6bfeffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561bb6b23be0 0x561bb6b2e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:36004/0 (socket says 172.21.15.101:36004) 2023-07-18T03:17:53.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.867+0000 7f6bfeffd700 1 -- 172.21.15.101:0/2641797742 learned_addr learned my addr 172.21.15.101:0/2641797742 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:53.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.867+0000 7f6bfffff700 1 --2- 172.21.15.101:0/2641797742 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561bb6b2ead0 0x561bb6b30eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:53.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.867+0000 7f6bfeffd700 1 -- 172.21.15.101:0/2641797742 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561bb6b232d0 msgr2=0x561bb6b236a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:53.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.867+0000 7f6bfeffd700 1 --2- 172.21.15.101:0/2641797742 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561bb6b232d0 0x561bb6b236a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.867+0000 7f6bfeffd700 1 -- 172.21.15.101:0/2641797742 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561bb6b2ead0 msgr2=0x561bb6b30eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:53.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.867+0000 7f6bfeffd700 1 --2- 172.21.15.101:0/2641797742 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561bb6b2ead0 0x561bb6b30eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.867+0000 7f6bfeffd700 1 -- 172.21.15.101:0/2641797742 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bb68a3de0 con 0x561bb6b23be0 2023-07-18T03:17:53.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.867+0000 7f6bfffff700 1 --2- 172.21.15.101:0/2641797742 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561bb6b2ead0 0x561bb6b30eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:17:53.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.867+0000 7f6bfeffd700 1 --2- 172.21.15.101:0/2641797742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561bb6b23be0 0x561bb6b2e590 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4097560 tx=0x7f6bf40054e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=196430459baae6f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:53.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.868+0000 7f6be5ffb700 1 -- 172.21.15.101:0/2641797742 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf40a2eb0 con 0x561bb6b23be0 2023-07-18T03:17:53.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.868+0000 7f6be5ffb700 1 -- 172.21.15.101:0/2641797742 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6bf4097200 con 0x561bb6b23be0 2023-07-18T03:17:53.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.868+0000 7f6be5ffb700 1 -- 172.21.15.101:0/2641797742 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf4099fe0 con 0x561bb6b23be0 2023-07-18T03:17:53.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.869+0000 7f6c19740b80 1 -- 172.21.15.101:0/2641797742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561bb6b23be0 msgr2=0x561bb6b2e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:53.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.869+0000 7f6bff7fe700 1 --2- 172.21.15.101:0/3081269760 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6be80a12c0 0x7f6be80a3770 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0009cd0 tx=0x7f6bf000b040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:53.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.869+0000 7f6c19740b80 1 --2- 172.21.15.101:0/2641797742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561bb6b23be0 0x561bb6b2e590 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4097560 tx=0x7f6bf40054e0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.869+0000 7f6c19740b80 1 -- 172.21.15.101:0/2641797742 shutdown_connections 2023-07-18T03:17:53.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.869+0000 7f6c19740b80 1 --2- 172.21.15.101:0/2641797742 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561bb6b2ead0 0x561bb6b30eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.869+0000 7f6c19740b80 1 --2- 172.21.15.101:0/2641797742 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561bb6b232d0 0x561bb6b236a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.869+0000 7f6c19740b80 1 --2- 172.21.15.101:0/2641797742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561bb6b23be0 0x561bb6b2e590 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.869+0000 7f6c19740b80 1 -- 172.21.15.101:0/2641797742 >> 172.21.15.101:0/2641797742 conn(0x561bb6b19da0 msgr2=0x561bb6b1f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:53.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.869+0000 7f6c19740b80 1 -- 172.21.15.101:0/2641797742 shutdown_connections 2023-07-18T03:17:53.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.869+0000 7f6c19740b80 1 -- 172.21.15.101:0/2641797742 wait complete. 2023-07-18T03:17:53.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.870+0000 7f6c19740b80 1 -- start start 2023-07-18T03:17:53.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.870+0000 7f6c19740b80 1 Processor -- start 2023-07-18T03:17:53.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.871+0000 7f6c19740b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561bb6b232d0 0x561bb6b2eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:53.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.871+0000 7f6c19740b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561bb6b23be0 0x561bb6b2f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:53.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.871+0000 7f6c19740b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561bb6b2ead0 0x561bb6b2f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:53.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.871+0000 7f6c19740b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bb6a15ac0 con 0x561bb6b2ead0 2023-07-18T03:17:53.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.871+0000 7f6c19740b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bb6a157c0 con 0x561bb6b23be0 2023-07-18T03:17:53.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.871+0000 7f6c19740b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bb68f9920 con 0x561bb6b232d0 2023-07-18T03:17:53.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.871+0000 7f6c19740b80 10 client.0 _ll_register_callbacks cb 0x561bb6a13760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:17:53.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.871+0000 7f6bff7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561bb6b232d0 0x561bb6b2eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:53.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.871+0000 7f6bfeffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561bb6b23be0 0x561bb6b2f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:53.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.871+0000 7f6bfeffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561bb6b23be0 0x561bb6b2f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:36020/0 (socket says 172.21.15.101:36020) 2023-07-18T03:17:53.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.871+0000 7f6bff7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561bb6b232d0 0x561bb6b2eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:39592/0 (socket says 172.21.15.101:39592) 2023-07-18T03:17:53.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.871+0000 7f6bfeffd700 1 -- 172.21.15.101:0/2089885327 learned_addr learned my addr 172.21.15.101:0/2089885327 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:53.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.871+0000 7f6bfffff700 1 --2- 172.21.15.101:0/2089885327 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561bb6b2ead0 0x561bb6b2f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:53.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.871+0000 7f6bfeffd700 1 -- 172.21.15.101:0/2089885327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561bb6b232d0 msgr2=0x561bb6b2eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:53.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.871+0000 7f6bfeffd700 1 --2- 172.21.15.101:0/2089885327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561bb6b232d0 0x561bb6b2eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.871+0000 7f6bfeffd700 1 -- 172.21.15.101:0/2089885327 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561bb6b2ead0 msgr2=0x561bb6b2f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:53.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.871+0000 7f6bfeffd700 1 --2- 172.21.15.101:0/2089885327 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561bb6b2ead0 0x561bb6b2f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.871+0000 7f6bfeffd700 1 -- 172.21.15.101:0/2089885327 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c0019d670 con 0x561bb6b23be0 2023-07-18T03:17:53.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.872+0000 7f6bfeffd700 1 --2- 172.21.15.101:0/2089885327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561bb6b23be0 0x561bb6b2f3e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4005c10 tx=0x7f6bf409a1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=928bf2ecf9c5e15f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:53.877 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.872+0000 7f6bdeffd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:17:53.877 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.872+0000 7f6bdeffd700 1 -- 172.21.15.101:0/2089885327 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf40a2eb0 con 0x561bb6b23be0 2023-07-18T03:17:53.877 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.872+0000 7f6bdeffd700 1 -- 172.21.15.101:0/2089885327 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f6bf409ad50 con 0x561bb6b23be0 2023-07-18T03:17:53.877 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.872+0000 7f6c19740b80 1 -- 172.21.15.101:0/2089885327 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561bb69cf4f0 con 0x561bb6b23be0 2023-07-18T03:17:53.877 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.872+0000 7f6bdeffd700 1 -- 172.21.15.101:0/2089885327 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6bf409af10 con 0x561bb6b23be0 2023-07-18T03:17:53.877 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.872+0000 7f6c19740b80 10 client.27644 fetch_fsmap learned FSMap version 33 2023-07-18T03:17:53.877 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.872+0000 7f6c19740b80 1 -- 172.21.15.101:0/2089885327 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x561bb688cc60 con 0x561bb6b23be0 2023-07-18T03:17:53.877 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.872+0000 7f6bdeffd700 1 -- 172.21.15.101:0/2089885327 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561bb69cf4f0 con 0x561bb6b23be0 2023-07-18T03:17:53.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.873+0000 7f6bdeffd700 1 -- 172.21.15.101:0/2089885327 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6bf40968e0 con 0x561bb6b23be0 2023-07-18T03:17:53.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.873+0000 7f6c19740b80 10 client.27644 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:17:53.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.873+0000 7f6c19740b80 10 client.27644 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:17:53.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.873+0000 7f6c19740b80 20 client.27644 populate_metadata read hostname 'smithi101' 2023-07-18T03:17:53.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.873+0000 7f6c19740b80 1 --2- 172.21.15.101:0/2089885327 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x561bb6b3dbb0 0x561bb6b3ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:53.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.873+0000 7f6c19740b80 4 client.27644 mds_command: new command op to 24541 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:17:53.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.873+0000 7f6c19740b80 1 -- 172.21.15.101:0/2089885327 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561bb6a6eeb0 con 0x561bb6b3dbb0 2023-07-18T03:17:53.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.873+0000 7f6bff7fe700 1 --2- 172.21.15.101:0/2089885327 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x561bb6b3dbb0 0x561bb6b3ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:53.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.874+0000 7f6bff7fe700 1 --2- 172.21.15.101:0/2089885327 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x561bb6b3dbb0 0x561bb6b3ff90 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4615d6916f1dc90c server_cookie=3fdbaf3c602d61ca in_seq=0 out_seq=0 2023-07-18T03:17:53.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.874+0000 7f6bdeffd700 10 client.27644 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:17:53.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.874+0000 7f6bdeffd700 1 -- 172.21.15.101:0/2089885327 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561bb6a6eeb0 con 0x561bb6b3dbb0 2023-07-18T03:17:53.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.874+0000 7f6bdeffd700 10 client.27644 handle_command_reply: tid=0 2023-07-18T03:17:53.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.874+0000 7f6c19740b80 1 client.27644 shutdown 2023-07-18T03:17:53.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.874+0000 7f6c19740b80 1 -- 172.21.15.101:0/2089885327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561bb6b23be0 msgr2=0x561bb6b2f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:53.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.874+0000 7f6c19740b80 1 --2- 172.21.15.101:0/2089885327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561bb6b23be0 0x561bb6b2f3e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4005c10 tx=0x7f6bf409a1f0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.875+0000 7f6c19740b80 1 -- 172.21.15.101:0/2089885327 shutdown_connections 2023-07-18T03:17:53.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.875+0000 7f6c19740b80 1 --2- 172.21.15.101:0/2089885327 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x561bb6b3dbb0 0x561bb6b3ff90 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.875+0000 7f6c19740b80 1 --2- 172.21.15.101:0/2089885327 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561bb6b2ead0 0x561bb6b2f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.875+0000 7f6c19740b80 1 --2- 172.21.15.101:0/2089885327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561bb6b232d0 0x561bb6b2eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.875+0000 7f6bdeffd700 0 client.27644 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:17:53.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.875+0000 7f6c19740b80 1 --2- 172.21.15.101:0/2089885327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561bb6b23be0 0x561bb6b2f3e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.875+0000 7f6c19740b80 1 -- 172.21.15.101:0/2089885327 >> 172.21.15.101:0/2089885327 conn(0x561bb6b19da0 msgr2=0x561bb6b1f290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:53.882 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.875+0000 7f6c19740b80 1 -- 172.21.15.101:0/2089885327 shutdown_connections 2023-07-18T03:17:53.882 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.875+0000 7f6c19740b80 1 -- 172.21.15.101:0/2089885327 wait complete. 2023-07-18T03:17:53.882 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.875+0000 7f6c19740b80 20 client.27644 trim_cache size 0 max 16384 2023-07-18T03:17:53.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.888+0000 7f6c19740b80 1 Processor -- start 2023-07-18T03:17:53.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.888+0000 7f6c19740b80 1 -- start start 2023-07-18T03:17:53.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.889+0000 7f6c19740b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561bb6b19da0 0x561bb6b32ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:53.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.889+0000 7f6c19740b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561bb6b232d0 0x561bb6b281c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:53.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.889+0000 7f6c19740b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561bb6b23be0 0x561bb6b2adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:53.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.889+0000 7f6c19740b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bb6b405c0 con 0x561bb6b23be0 2023-07-18T03:17:53.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.889+0000 7f6c19740b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bb6b2c0e0 con 0x561bb6b19da0 2023-07-18T03:17:53.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.889+0000 7f6c19740b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bb6b2dd40 con 0x561bb6b232d0 2023-07-18T03:17:53.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.889+0000 7f6bff7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561bb6b19da0 0x561bb6b32ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:53.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.889+0000 7f6bff7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561bb6b19da0 0x561bb6b32ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:36028/0 (socket says 172.21.15.101:36028) 2023-07-18T03:17:53.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.889+0000 7f6bfffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561bb6b23be0 0x561bb6b2adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:53.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.889+0000 7f6bfeffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561bb6b232d0 0x561bb6b281c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:53.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.889+0000 7f6bff7fe700 1 -- 172.21.15.101:0/601365996 learned_addr learned my addr 172.21.15.101:0/601365996 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:53.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.890+0000 7f6bff7fe700 1 -- 172.21.15.101:0/601365996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561bb6b232d0 msgr2=0x561bb6b281c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:53.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.890+0000 7f6bff7fe700 1 --2- 172.21.15.101:0/601365996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561bb6b232d0 0x561bb6b281c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.890+0000 7f6bff7fe700 1 -- 172.21.15.101:0/601365996 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561bb6b23be0 msgr2=0x561bb6b2adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:53.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.890+0000 7f6bff7fe700 1 --2- 172.21.15.101:0/601365996 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561bb6b23be0 0x561bb6b2adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.890+0000 7f6bff7fe700 1 -- 172.21.15.101:0/601365996 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bb68ca250 con 0x561bb6b19da0 2023-07-18T03:17:53.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.890+0000 7f6bff7fe700 1 --2- 172.21.15.101:0/601365996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561bb6b19da0 0x561bb6b32ed0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6bf00093c0 tx=0x7f6bf0009050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5296c5fa86e5c539 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:53.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.890+0000 7f6be67fc700 1 -- 172.21.15.101:0/601365996 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf001c070 con 0x561bb6b19da0 2023-07-18T03:17:53.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.890+0000 7f6be67fc700 1 -- 172.21.15.101:0/601365996 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6bf000f040 con 0x561bb6b19da0 2023-07-18T03:17:53.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.890+0000 7f6be67fc700 1 -- 172.21.15.101:0/601365996 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf000e030 con 0x561bb6b19da0 2023-07-18T03:17:53.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.891+0000 7f6c19740b80 1 -- 172.21.15.101:0/601365996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561bb6b19da0 msgr2=0x561bb6b32ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:53.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.891+0000 7f6c19740b80 1 --2- 172.21.15.101:0/601365996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561bb6b19da0 0x561bb6b32ed0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6bf00093c0 tx=0x7f6bf0009050 comp rx=0 tx=0).stop 2023-07-18T03:17:53.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.891+0000 7f6c19740b80 1 -- 172.21.15.101:0/601365996 shutdown_connections 2023-07-18T03:17:53.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.891+0000 7f6c19740b80 1 --2- 172.21.15.101:0/601365996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561bb6b19da0 0x561bb6b32ed0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.891+0000 7f6c19740b80 1 --2- 172.21.15.101:0/601365996 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561bb6b23be0 0x561bb6b2adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.891+0000 7f6c19740b80 1 --2- 172.21.15.101:0/601365996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561bb6b232d0 0x561bb6b281c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.891+0000 7f6c19740b80 1 -- 172.21.15.101:0/601365996 >> 172.21.15.101:0/601365996 conn(0x561bb6b1b600 msgr2=0x561bb6b28e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:53.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.891+0000 7f6c19740b80 1 -- 172.21.15.101:0/601365996 shutdown_connections 2023-07-18T03:17:53.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.891+0000 7f6c19740b80 1 -- 172.21.15.101:0/601365996 wait complete. 2023-07-18T03:17:53.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.891+0000 7f6c19740b80 1 -- start start 2023-07-18T03:17:53.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.891+0000 7f6c19740b80 1 Processor -- start 2023-07-18T03:17:53.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.892+0000 7f6c19740b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561bb6b19da0 0x561bb6b8b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:53.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.892+0000 7f6c19740b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561bb6b232d0 0x561bb6b8bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:53.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.892+0000 7f6c19740b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561bb6b23be0 0x561bb6b85960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:53.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.892+0000 7f6c19740b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bb6b85f40 con 0x561bb6b19da0 2023-07-18T03:17:53.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.892+0000 7f6c19740b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bb6b860b0 con 0x561bb6b23be0 2023-07-18T03:17:53.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.892+0000 7f6c19740b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bb6b86270 con 0x561bb6b232d0 2023-07-18T03:17:53.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.892+0000 7f6c19740b80 10 client.0 _ll_register_callbacks cb 0x7f6bf409a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:17:53.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.892+0000 7f6bff7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561bb6b19da0 0x561bb6b8b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:53.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.892+0000 7f6bfffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561bb6b23be0 0x561bb6b85960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:53.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.892+0000 7f6bfeffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561bb6b232d0 0x561bb6b8bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:53.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.892+0000 7f6bff7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561bb6b19da0 0x561bb6b8b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:57898/0 (socket says 172.21.15.101:57898) 2023-07-18T03:17:53.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.892+0000 7f6bff7fe700 1 -- 172.21.15.101:0/3550877998 learned_addr learned my addr 172.21.15.101:0/3550877998 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:53.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.892+0000 7f6bff7fe700 1 -- 172.21.15.101:0/3550877998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561bb6b232d0 msgr2=0x561bb6b8bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:53.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.892+0000 7f6bff7fe700 1 --2- 172.21.15.101:0/3550877998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561bb6b232d0 0x561bb6b8bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.892+0000 7f6bff7fe700 1 -- 172.21.15.101:0/3550877998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561bb6b23be0 msgr2=0x561bb6b85960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:53.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.892+0000 7f6bff7fe700 1 --2- 172.21.15.101:0/3550877998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561bb6b23be0 0x561bb6b85960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.892+0000 7f6bff7fe700 1 -- 172.21.15.101:0/3550877998 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bf00249c0 con 0x561bb6b19da0 2023-07-18T03:17:53.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.893+0000 7f6bff7fe700 1 --2- 172.21.15.101:0/3550877998 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561bb6b19da0 0x561bb6b8b590 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0005c30 tx=0x7f6bf00246b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2caad1e8d2cb9978 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:53.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.893+0000 7f6bdeffd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:17:53.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.893+0000 7f6bdeffd700 1 -- 172.21.15.101:0/3550877998 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf001c040 con 0x561bb6b19da0 2023-07-18T03:17:53.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.893+0000 7f6bdeffd700 1 -- 172.21.15.101:0/3550877998 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f6bf0030660 con 0x561bb6b19da0 2023-07-18T03:17:53.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.893+0000 7f6c19740b80 1 -- 172.21.15.101:0/3550877998 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6bf4097200 con 0x561bb6b19da0 2023-07-18T03:17:53.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.893+0000 7f6bdeffd700 1 -- 172.21.15.101:0/3550877998 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6bf001e040 con 0x561bb6b19da0 2023-07-18T03:17:53.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.893+0000 7f6c19740b80 10 client.17613 fetch_fsmap learned FSMap version 33 2023-07-18T03:17:53.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.893+0000 7f6c19740b80 1 -- 172.21.15.101:0/3550877998 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x561bb68a22b0 con 0x561bb6b19da0 2023-07-18T03:17:53.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.893+0000 7f6bdeffd700 1 -- 172.21.15.101:0/3550877998 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6bf4097200 con 0x561bb6b19da0 2023-07-18T03:17:53.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.894+0000 7f6bdeffd700 1 -- 172.21.15.101:0/3550877998 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6bf0021070 con 0x561bb6b19da0 2023-07-18T03:17:53.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.894+0000 7f6c19740b80 10 client.17613 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:17:53.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.894+0000 7f6c19740b80 10 client.17613 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:17:53.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.894+0000 7f6c19740b80 20 client.17613 populate_metadata read hostname 'smithi101' 2023-07-18T03:17:53.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.894+0000 7f6c19740b80 1 --2- 172.21.15.101:0/3550877998 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x561bb6b2ead0 0x561bb6b95370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:53.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.894+0000 7f6c19740b80 4 client.17613 mds_command: new command op to 24541 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:17:53.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.894+0000 7f6c19740b80 1 -- 172.21.15.101:0/3550877998 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561bb6a31050 con 0x561bb6b2ead0 2023-07-18T03:17:53.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.894+0000 7f6bfeffd700 1 --2- 172.21.15.101:0/3550877998 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x561bb6b2ead0 0x561bb6b95370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:53.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.894+0000 7f6bdeffd700 10 client.17613 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:17:53.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.894+0000 7f6bfeffd700 1 --2- 172.21.15.101:0/3550877998 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x561bb6b2ead0 0x561bb6b95370 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5510af74a09cb307 server_cookie=2bc2dff15ef4ee in_seq=0 out_seq=0 2023-07-18T03:17:53.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.895+0000 7f6bdeffd700 1 -- 172.21.15.101:0/3550877998 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561bb6a31050 con 0x561bb6b2ead0 2023-07-18T03:17:53.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.895+0000 7f6bdeffd700 10 client.17613 handle_command_reply: tid=0 2023-07-18T03:17:53.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.895+0000 7f6c19740b80 1 client.17613 shutdown 2023-07-18T03:17:53.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.895+0000 7f6c19740b80 1 -- 172.21.15.101:0/3550877998 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561bb6b19da0 msgr2=0x561bb6b8b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:53.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.895+0000 7f6c19740b80 1 --2- 172.21.15.101:0/3550877998 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561bb6b19da0 0x561bb6b8b590 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0005c30 tx=0x7f6bf00246b0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.895+0000 7f6c19740b80 1 -- 172.21.15.101:0/3550877998 shutdown_connections 2023-07-18T03:17:53.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.895+0000 7f6c19740b80 1 --2- 172.21.15.101:0/3550877998 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x561bb6b2ead0 0x561bb6b95370 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.895+0000 7f6c19740b80 1 --2- 172.21.15.101:0/3550877998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561bb6b23be0 0x561bb6b85960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.895+0000 7f6c19740b80 1 --2- 172.21.15.101:0/3550877998 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561bb6b19da0 0x561bb6b8b590 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.895+0000 7f6c19740b80 1 --2- 172.21.15.101:0/3550877998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561bb6b232d0 0x561bb6b8bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.895+0000 7f6c19740b80 1 -- 172.21.15.101:0/3550877998 >> 172.21.15.101:0/3550877998 conn(0x561bb6b1b600 msgr2=0x561bb6b2c330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:53.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.895+0000 7f6bdeffd700 0 client.17613 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:17:53.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.895+0000 7f6c19740b80 1 -- 172.21.15.101:0/3550877998 shutdown_connections 2023-07-18T03:17:53.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.895+0000 7f6c19740b80 1 -- 172.21.15.101:0/3550877998 wait complete. 2023-07-18T03:17:53.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.895+0000 7f6c19740b80 20 client.17613 trim_cache size 0 max 16384 2023-07-18T03:17:53.902 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:17:53.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.896+0000 7f6c05dff700 1 -- 172.21.15.101:0/3081269760 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6be80a12c0 msgr2=0x7f6be80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:53.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.896+0000 7f6c05dff700 1 --2- 172.21.15.101:0/3081269760 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6be80a12c0 0x7f6be80a3770 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0009cd0 tx=0x7f6bf000b040 comp rx=0 tx=0).stop 2023-07-18T03:17:53.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.896+0000 7f6c05dff700 1 -- 172.21.15.101:0/3081269760 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c00100480 msgr2=0x7f6c00198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:53.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.896+0000 7f6c05dff700 1 --2- 172.21.15.101:0/3081269760 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c00100480 0x7f6c00198390 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4009810 tx=0x7f6bf400fa00 comp rx=0 tx=0).stop 2023-07-18T03:17:53.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.896+0000 7f6c05dff700 1 -- 172.21.15.101:0/3081269760 shutdown_connections 2023-07-18T03:17:53.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.896+0000 7f6c05dff700 1 --2- 172.21.15.101:0/3081269760 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6be80a12c0 0x7f6be80a3770 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.896+0000 7f6c05dff700 1 --2- 172.21.15.101:0/3081269760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c00105ae0 0x7f6c0019c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.896+0000 7f6c05dff700 1 --2- 172.21.15.101:0/3081269760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c000ffad0 0x7f6c00197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.896+0000 7f6c05dff700 1 --2- 172.21.15.101:0/3081269760 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c00100480 0x7f6c00198390 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:53.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.896+0000 7f6c05dff700 1 -- 172.21.15.101:0/3081269760 >> 172.21.15.101:0/3081269760 conn(0x7f6c000fb5f0 msgr2=0x7f6c0010ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:53.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.896+0000 7f6c05dff700 1 -- 172.21.15.101:0/3081269760 shutdown_connections 2023-07-18T03:17:53.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:53.896+0000 7f6c05dff700 1 -- 172.21.15.101:0/3081269760 wait complete. 2023-07-18T03:17:53.909 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-18T03:17:54.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.124+0000 7f665c9ff700 1 -- 172.21.15.101:0/2478159426 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6658105ae0 msgr2=0x7f6658105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:54.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.124+0000 7f665c9ff700 1 --2- 172.21.15.101:0/2478159426 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6658105ae0 0x7f6658105eb0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f6648009990 tx=0x7f6648009ca0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.124+0000 7f665c9ff700 1 -- 172.21.15.101:0/2478159426 shutdown_connections 2023-07-18T03:17:54.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.124+0000 7f665c9ff700 1 --2- 172.21.15.101:0/2478159426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6658100480 0x7f665810c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.124+0000 7f665c9ff700 1 --2- 172.21.15.101:0/2478159426 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6658105ae0 0x7f6658105eb0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.124+0000 7f665c9ff700 1 --2- 172.21.15.101:0/2478159426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66580ffad0 0x7f66580fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.124+0000 7f665c9ff700 1 -- 172.21.15.101:0/2478159426 >> 172.21.15.101:0/2478159426 conn(0x7f66580fb5f0 msgr2=0x7f66580fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:54.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.124+0000 7f665c9ff700 1 -- 172.21.15.101:0/2478159426 shutdown_connections 2023-07-18T03:17:54.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.124+0000 7f665c9ff700 1 -- 172.21.15.101:0/2478159426 wait complete. 2023-07-18T03:17:54.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.125+0000 7f665c9ff700 1 Processor -- start 2023-07-18T03:17:54.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.125+0000 7f665c9ff700 1 -- start start 2023-07-18T03:17:54.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.125+0000 7f665c9ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f66580ffad0 0x7f6658072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:54.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.125+0000 7f665c9ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6658100480 0x7f6658072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:54.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.125+0000 7f665c9ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6658105ae0 0x7f66581a3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:54.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.125+0000 7f665c9ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6658073580 con 0x7f66580ffad0 2023-07-18T03:17:54.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.125+0000 7f665c9ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66581a4300 con 0x7f6658105ae0 2023-07-18T03:17:54.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.125+0000 7f665c9ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66581a4440 con 0x7f6658100480 2023-07-18T03:17:54.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.125+0000 7f665659c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f66580ffad0 0x7f6658072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:54.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.126+0000 7f665659c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f66580ffad0 0x7f6658072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:57914/0 (socket says 172.21.15.101:57914) 2023-07-18T03:17:54.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.126+0000 7f665659c700 1 -- 172.21.15.101:0/2719944038 learned_addr learned my addr 172.21.15.101:0/2719944038 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:54.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.126+0000 7f6656d9d700 1 --2- 172.21.15.101:0/2719944038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6658105ae0 0x7f66581a3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:54.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.126+0000 7f6655d9b700 1 --2- 172.21.15.101:0/2719944038 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6658100480 0x7f6658072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:54.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.126+0000 7f665659c700 1 -- 172.21.15.101:0/2719944038 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6658100480 msgr2=0x7f6658072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:54.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.126+0000 7f665659c700 1 --2- 172.21.15.101:0/2719944038 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6658100480 0x7f6658072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.126+0000 7f665659c700 1 -- 172.21.15.101:0/2719944038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6658105ae0 msgr2=0x7f66581a3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:54.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.126+0000 7f665659c700 1 --2- 172.21.15.101:0/2719944038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6658105ae0 0x7f66581a3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.126+0000 7f665659c700 1 -- 172.21.15.101:0/2719944038 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66480095f0 con 0x7f66580ffad0 2023-07-18T03:17:54.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.126+0000 7f6655d9b700 1 --2- 172.21.15.101:0/2719944038 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6658100480 0x7f6658072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:17:54.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.126+0000 7f665659c700 1 --2- 172.21.15.101:0/2719944038 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f66580ffad0 0x7f6658072810 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f6648005910 tx=0x7f6648004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:54.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.126+0000 7f66477fe700 1 -- 172.21.15.101:0/2719944038 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664801d070 con 0x7f66580ffad0 2023-07-18T03:17:54.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.126+0000 7f66477fe700 1 -- 172.21.15.101:0/2719944038 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6648022470 con 0x7f66580ffad0 2023-07-18T03:17:54.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.126+0000 7f665c9ff700 1 -- 172.21.15.101:0/2719944038 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66581a4640 con 0x7f66580ffad0 2023-07-18T03:17:54.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.126+0000 7f66477fe700 1 -- 172.21.15.101:0/2719944038 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6648017650 con 0x7f66580ffad0 2023-07-18T03:17:54.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.127+0000 7f665c9ff700 1 -- 172.21.15.101:0/2719944038 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66581a4ac0 con 0x7f66580ffad0 2023-07-18T03:17:54.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.130+0000 7f66477fe700 1 -- 172.21.15.101:0/2719944038 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f664800fad0 con 0x7f66580ffad0 2023-07-18T03:17:54.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.132+0000 7f6670281b80 1 Processor -- start 2023-07-18T03:17:54.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.132+0000 7f6670281b80 1 -- start start 2023-07-18T03:17:54.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.132+0000 7f66477fe700 1 --2- 172.21.15.101:0/2719944038 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f66400a1340 0x7f66400a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:54.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.132+0000 7f66477fe700 1 -- 172.21.15.101:0/2719944038 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f66480ce510 con 0x7f66580ffad0 2023-07-18T03:17:54.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.132+0000 7f6670281b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff718052d0 0x55ff718056a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:54.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.132+0000 7f6656d9d700 1 --2- 172.21.15.101:0/2719944038 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f66400a1340 0x7f66400a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:54.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.132+0000 7f6670281b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff71805be0 0x55ff71810590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:54.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.133+0000 7f6670281b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff71810ad0 0x55ff71812eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:54.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.133+0000 7f6670281b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff7170bcb0 con 0x55ff718052d0 2023-07-18T03:17:54.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.133+0000 7f6670281b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff7175ea40 con 0x55ff71810ad0 2023-07-18T03:17:54.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.133+0000 7f6670281b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff71757ed0 con 0x55ff71805be0 2023-07-18T03:17:54.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.133+0000 7f665659c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff718052d0 0x55ff718056a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:54.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.133+0000 7f665659c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff718052d0 0x55ff718056a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:57928/0 (socket says 172.21.15.101:57928) 2023-07-18T03:17:54.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.133+0000 7f665659c700 1 -- 172.21.15.101:0/2482724447 learned_addr learned my addr 172.21.15.101:0/2482724447 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:54.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.133+0000 7f6655d9b700 1 --2- 172.21.15.101:0/2482724447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff71805be0 0x55ff71810590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:54.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.133+0000 7f665659c700 1 -- 172.21.15.101:0/2482724447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff71805be0 msgr2=0x55ff71810590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:54.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.133+0000 7f665659c700 1 --2- 172.21.15.101:0/2482724447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff71805be0 0x55ff71810590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.133+0000 7f665659c700 1 -- 172.21.15.101:0/2482724447 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff71810ad0 msgr2=0x55ff71812eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:54.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.133+0000 7f665659c700 1 --2- 172.21.15.101:0/2482724447 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff71810ad0 0x55ff71812eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.133+0000 7f665659c700 1 -- 172.21.15.101:0/2482724447 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff71585de0 con 0x55ff718052d0 2023-07-18T03:17:54.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.133+0000 7f6656d9d700 1 --2- 172.21.15.101:0/2719944038 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f66400a1340 0x7f66400a37f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6658074920 tx=0x7f6650009ba0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:54.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.133+0000 7f665659c700 1 --2- 172.21.15.101:0/2482724447 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff718052d0 0x55ff718056a0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f6648097cd0 tx=0x7f6648009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d2bb1c7ca9c21fb server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:54.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.134+0000 7f6644ff9700 1 -- 172.21.15.101:0/2482724447 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664809f290 con 0x55ff718052d0 2023-07-18T03:17:54.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.134+0000 7f6644ff9700 1 -- 172.21.15.101:0/2482724447 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66480a3620 con 0x55ff718052d0 2023-07-18T03:17:54.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.134+0000 7f6644ff9700 1 -- 172.21.15.101:0/2482724447 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66480ce130 con 0x55ff718052d0 2023-07-18T03:17:54.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.134+0000 7f6670281b80 1 -- 172.21.15.101:0/2482724447 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff718052d0 msgr2=0x55ff718056a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:54.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.134+0000 7f6670281b80 1 --2- 172.21.15.101:0/2482724447 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff718052d0 0x55ff718056a0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f6648097cd0 tx=0x7f6648009ca0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.134+0000 7f6670281b80 1 -- 172.21.15.101:0/2482724447 shutdown_connections 2023-07-18T03:17:54.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.134+0000 7f6670281b80 1 --2- 172.21.15.101:0/2482724447 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff71810ad0 0x55ff71812eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.134+0000 7f6670281b80 1 --2- 172.21.15.101:0/2482724447 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff718052d0 0x55ff718056a0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.134+0000 7f6670281b80 1 --2- 172.21.15.101:0/2482724447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff71805be0 0x55ff71810590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.134+0000 7f6670281b80 1 -- 172.21.15.101:0/2482724447 >> 172.21.15.101:0/2482724447 conn(0x55ff717fbda0 msgr2=0x55ff718016a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:54.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.134+0000 7f6670281b80 1 -- 172.21.15.101:0/2482724447 shutdown_connections 2023-07-18T03:17:54.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.134+0000 7f6670281b80 1 -- 172.21.15.101:0/2482724447 wait complete. 2023-07-18T03:17:54.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.135+0000 7f6670281b80 1 -- start start 2023-07-18T03:17:54.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.135+0000 7f6670281b80 1 Processor -- start 2023-07-18T03:17:54.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.136+0000 7f6670281b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff718052d0 0x55ff71810ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:54.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.136+0000 7f6670281b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff71805be0 0x55ff718113e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:54.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.136+0000 7f6670281b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff71810ad0 0x55ff71811920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:54.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.136+0000 7f6670281b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff716f7ac0 con 0x55ff71805be0 2023-07-18T03:17:54.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.136+0000 7f6670281b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff716f77c0 con 0x55ff718052d0 2023-07-18T03:17:54.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.136+0000 7f6670281b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff715db920 con 0x55ff71810ad0 2023-07-18T03:17:54.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.136+0000 7f6670281b80 10 client.0 _ll_register_callbacks cb 0x55ff716f5760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:17:54.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.136+0000 7f665659c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff718052d0 0x55ff71810ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:54.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.136+0000 7f665659c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff718052d0 0x55ff71810ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:36072/0 (socket says 172.21.15.101:36072) 2023-07-18T03:17:54.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.136+0000 7f665659c700 1 -- 172.21.15.101:0/875493527 learned_addr learned my addr 172.21.15.101:0/875493527 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:54.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.136+0000 7f6655d9b700 1 --2- 172.21.15.101:0/875493527 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff71805be0 0x55ff718113e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:54.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.136+0000 7f6656d9d700 1 --2- 172.21.15.101:0/875493527 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff71810ad0 0x55ff71811920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:54.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.136+0000 7f665659c700 1 -- 172.21.15.101:0/875493527 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff71810ad0 msgr2=0x55ff71811920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:54.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.136+0000 7f665659c700 1 --2- 172.21.15.101:0/875493527 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff71810ad0 0x55ff71811920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.136+0000 7f665659c700 1 -- 172.21.15.101:0/875493527 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff71805be0 msgr2=0x55ff718113e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:54.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.136+0000 7f665659c700 1 --2- 172.21.15.101:0/875493527 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff71805be0 0x55ff718113e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.136+0000 7f665659c700 1 -- 172.21.15.101:0/875493527 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66581a4ac0 con 0x55ff718052d0 2023-07-18T03:17:54.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.136+0000 7f665659c700 1 --2- 172.21.15.101:0/875493527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff718052d0 0x55ff71810ea0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6648096fe0 tx=0x7f6648005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6ab75f9caf274f4 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:54.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.137+0000 7f663dffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:17:54.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.137+0000 7f663dffb700 1 -- 172.21.15.101:0/875493527 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664809f040 con 0x55ff718052d0 2023-07-18T03:17:54.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.137+0000 7f663dffb700 1 -- 172.21.15.101:0/875493527 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f66480225e0 con 0x55ff718052d0 2023-07-18T03:17:54.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.137+0000 7f6670281b80 1 -- 172.21.15.101:0/875493527 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ff716b14f0 con 0x55ff718052d0 2023-07-18T03:17:54.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.137+0000 7f663dffb700 1 -- 172.21.15.101:0/875493527 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6648022890 con 0x55ff718052d0 2023-07-18T03:17:54.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.137+0000 7f6670281b80 10 client.27659 fetch_fsmap learned FSMap version 33 2023-07-18T03:17:54.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.137+0000 7f6670281b80 1 -- 172.21.15.101:0/875493527 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55ff7156ec60 con 0x55ff718052d0 2023-07-18T03:17:54.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.137+0000 7f663dffb700 1 -- 172.21.15.101:0/875493527 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66480c9080 con 0x55ff718052d0 2023-07-18T03:17:54.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.137+0000 7f663dffb700 1 -- 172.21.15.101:0/875493527 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f664809a750 con 0x55ff718052d0 2023-07-18T03:17:54.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.137+0000 7f6670281b80 10 client.27659 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:17:54.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.137+0000 7f6670281b80 10 client.27659 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:17:54.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.137+0000 7f6670281b80 20 client.27659 populate_metadata read hostname 'smithi101' 2023-07-18T03:17:54.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.137+0000 7f6670281b80 1 --2- 172.21.15.101:0/875493527 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55ff7181fd40 0x55ff71822150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:54.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.137+0000 7f6670281b80 4 client.27659 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:17:54.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.137+0000 7f6670281b80 1 -- 172.21.15.101:0/875493527 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff71750eb0 con 0x55ff7181fd40 2023-07-18T03:17:54.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.138+0000 7f6655d9b700 1 --2- 172.21.15.101:0/875493527 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55ff7181fd40 0x55ff71822150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:54.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.138+0000 7f663dffb700 10 client.27659 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:17:54.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.138+0000 7f6655d9b700 1 --2- 172.21.15.101:0/875493527 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55ff7181fd40 0x55ff71822150 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2c8e99b0d9deb12b server_cookie=ede2031ceaf08387 in_seq=0 out_seq=0 2023-07-18T03:17:54.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.139+0000 7f663dffb700 1 -- 172.21.15.101:0/875493527 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ff71750eb0 con 0x55ff7181fd40 2023-07-18T03:17:54.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.139+0000 7f663dffb700 10 client.27659 handle_command_reply: tid=0 2023-07-18T03:17:54.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.139+0000 7f6670281b80 1 client.27659 shutdown 2023-07-18T03:17:54.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.139+0000 7f6670281b80 1 -- 172.21.15.101:0/875493527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff718052d0 msgr2=0x55ff71810ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:54.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.139+0000 7f6670281b80 1 --2- 172.21.15.101:0/875493527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff718052d0 0x55ff71810ea0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6648096fe0 tx=0x7f6648005fd0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.139+0000 7f6670281b80 1 -- 172.21.15.101:0/875493527 shutdown_connections 2023-07-18T03:17:54.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.139+0000 7f6670281b80 1 --2- 172.21.15.101:0/875493527 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55ff7181fd40 0x55ff71822150 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.139+0000 7f6670281b80 1 --2- 172.21.15.101:0/875493527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff718052d0 0x55ff71810ea0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.139+0000 7f663dffb700 0 client.27659 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:17:54.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.139+0000 7f6670281b80 1 --2- 172.21.15.101:0/875493527 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff71810ad0 0x55ff71811920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.139+0000 7f6670281b80 1 --2- 172.21.15.101:0/875493527 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff71805be0 0x55ff718113e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.139+0000 7f6670281b80 1 -- 172.21.15.101:0/875493527 >> 172.21.15.101:0/875493527 conn(0x55ff717fbda0 msgr2=0x55ff71801230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:54.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.139+0000 7f6670281b80 1 -- 172.21.15.101:0/875493527 shutdown_connections 2023-07-18T03:17:54.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.139+0000 7f6670281b80 1 -- 172.21.15.101:0/875493527 wait complete. 2023-07-18T03:17:54.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.139+0000 7f6670281b80 20 client.27659 trim_cache size 0 max 16384 2023-07-18T03:17:54.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.152+0000 7f6670281b80 1 Processor -- start 2023-07-18T03:17:54.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.153+0000 7f6670281b80 1 -- start start 2023-07-18T03:17:54.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.153+0000 7f6670281b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff718052d0 0x55ff71814ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:54.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.153+0000 7f6670281b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff71805be0 0x55ff7181c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:54.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.153+0000 7f6670281b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff71810ad0 0x55ff7180cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:54.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.153+0000 7f6670281b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff71805fb0 con 0x55ff71805be0 2023-07-18T03:17:54.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.153+0000 7f6670281b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff71802310 con 0x55ff718052d0 2023-07-18T03:17:54.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.153+0000 7f6670281b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff717fb970 con 0x55ff71810ad0 2023-07-18T03:17:54.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.153+0000 7f6655d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff71805be0 0x55ff7181c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:54.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.153+0000 7f6655d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff71805be0 0x55ff7181c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:57952/0 (socket says 172.21.15.101:57952) 2023-07-18T03:17:54.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.153+0000 7f6655d9b700 1 -- 172.21.15.101:0/556764706 learned_addr learned my addr 172.21.15.101:0/556764706 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:54.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.153+0000 7f665659c700 1 --2- 172.21.15.101:0/556764706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff718052d0 0x55ff71814ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:54.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.153+0000 7f6656d9d700 1 --2- 172.21.15.101:0/556764706 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff71810ad0 0x55ff7180cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:54.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.154+0000 7f6655d9b700 1 -- 172.21.15.101:0/556764706 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff71810ad0 msgr2=0x55ff7180cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:54.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.154+0000 7f6655d9b700 1 --2- 172.21.15.101:0/556764706 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff71810ad0 0x55ff7180cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.154+0000 7f6655d9b700 1 -- 172.21.15.101:0/556764706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff718052d0 msgr2=0x55ff71814ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:54.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.154+0000 7f6655d9b700 1 --2- 172.21.15.101:0/556764706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff718052d0 0x55ff71814ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.154+0000 7f6655d9b700 1 -- 172.21.15.101:0/556764706 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff715ac250 con 0x55ff71805be0 2023-07-18T03:17:54.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.154+0000 7f6655d9b700 1 --2- 172.21.15.101:0/556764706 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff71805be0 0x55ff7181c450 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x55ff71810120 tx=0x7f664c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a373b81d76d234f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:54.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.154+0000 7f66457fa700 1 -- 172.21.15.101:0/556764706 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664c017070 con 0x55ff71805be0 2023-07-18T03:17:54.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.154+0000 7f66457fa700 1 -- 172.21.15.101:0/556764706 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f664c018040 con 0x55ff71805be0 2023-07-18T03:17:54.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.154+0000 7f66457fa700 1 -- 172.21.15.101:0/556764706 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664c004b20 con 0x55ff71805be0 2023-07-18T03:17:54.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.155+0000 7f6670281b80 1 -- 172.21.15.101:0/556764706 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff71805be0 msgr2=0x55ff7181c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:54.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.155+0000 7f6670281b80 1 --2- 172.21.15.101:0/556764706 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff71805be0 0x55ff7181c450 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x55ff71810120 tx=0x7f664c009260 comp rx=0 tx=0).stop 2023-07-18T03:17:54.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.155+0000 7f6670281b80 1 -- 172.21.15.101:0/556764706 shutdown_connections 2023-07-18T03:17:54.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.155+0000 7f6670281b80 1 --2- 172.21.15.101:0/556764706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff718052d0 0x55ff71814ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.155+0000 7f6670281b80 1 --2- 172.21.15.101:0/556764706 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff71810ad0 0x55ff7180cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.155+0000 7f6670281b80 1 --2- 172.21.15.101:0/556764706 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff71805be0 0x55ff7181c450 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.155+0000 7f6670281b80 1 -- 172.21.15.101:0/556764706 >> 172.21.15.101:0/556764706 conn(0x55ff717fbda0 msgr2=0x55ff7181b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:54.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.155+0000 7f6670281b80 1 -- 172.21.15.101:0/556764706 shutdown_connections 2023-07-18T03:17:54.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.155+0000 7f6670281b80 1 -- 172.21.15.101:0/556764706 wait complete. 2023-07-18T03:17:54.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.155+0000 7f6670281b80 1 -- start start 2023-07-18T03:17:54.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.156+0000 7f6670281b80 1 Processor -- start 2023-07-18T03:17:54.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.156+0000 7f6670281b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff718052d0 0x55ff7186d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:54.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.156+0000 7f6670281b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff71805be0 0x55ff7186dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:54.156 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.156+0000 7f6670281b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff71810ad0 0x55ff71867b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:54.156 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.156+0000 7f6670281b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff71868160 con 0x55ff71805be0 2023-07-18T03:17:54.156 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.156+0000 7f6670281b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff718682d0 con 0x55ff718052d0 2023-07-18T03:17:54.156 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.156+0000 7f6670281b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff71868490 con 0x55ff71810ad0 2023-07-18T03:17:54.156 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.156+0000 7f6670281b80 10 client.0 _ll_register_callbacks cb 0x7f664809b030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:17:54.156 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.156+0000 7f665659c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff718052d0 0x55ff7186d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:54.156 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.156+0000 7f665659c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff718052d0 0x55ff7186d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:36082/0 (socket says 172.21.15.101:36082) 2023-07-18T03:17:54.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.156+0000 7f665659c700 1 -- 172.21.15.101:0/3811172946 learned_addr learned my addr 172.21.15.101:0/3811172946 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:54.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.156+0000 7f6655d9b700 1 --2- 172.21.15.101:0/3811172946 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff71805be0 0x55ff7186dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:54.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.156+0000 7f6656d9d700 1 --2- 172.21.15.101:0/3811172946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff71810ad0 0x55ff71867b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:54.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.157+0000 7f665659c700 1 -- 172.21.15.101:0/3811172946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff71810ad0 msgr2=0x55ff71867b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:54.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.157+0000 7f665659c700 1 --2- 172.21.15.101:0/3811172946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff71810ad0 0x55ff71867b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.157+0000 7f665659c700 1 -- 172.21.15.101:0/3811172946 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff71805be0 msgr2=0x55ff7186dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:54.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.157+0000 7f665659c700 1 --2- 172.21.15.101:0/3811172946 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff71805be0 0x55ff7186dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.157+0000 7f665659c700 1 -- 172.21.15.101:0/3811172946 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f664c012170 con 0x55ff718052d0 2023-07-18T03:17:54.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.157+0000 7f6655d9b700 1 --2- 172.21.15.101:0/3811172946 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff71805be0 0x55ff7186dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:17:54.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.157+0000 7f665659c700 1 --2- 172.21.15.101:0/3811172946 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff718052d0 0x55ff7186d680 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f66480b9e80 tx=0x7f66480cb930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6669bec7b732ed76 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:54.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.157+0000 7f663dffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:17:54.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.157+0000 7f663dffb700 1 -- 172.21.15.101:0/3811172946 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66480a3230 con 0x55ff718052d0 2023-07-18T03:17:54.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.157+0000 7f663dffb700 1 -- 172.21.15.101:0/3811172946 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f66480a33f0 con 0x55ff718052d0 2023-07-18T03:17:54.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.157+0000 7f6670281b80 1 -- 172.21.15.101:0/3811172946 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ff7180a960 con 0x55ff718052d0 2023-07-18T03:17:54.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.157+0000 7f663dffb700 1 -- 172.21.15.101:0/3811172946 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f664807b050 con 0x55ff718052d0 2023-07-18T03:17:54.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.157+0000 7f6670281b80 10 client.27665 fetch_fsmap learned FSMap version 33 2023-07-18T03:17:54.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.157+0000 7f6670281b80 1 -- 172.21.15.101:0/3811172946 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55ff715842b0 con 0x55ff718052d0 2023-07-18T03:17:54.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.157+0000 7f663dffb700 1 -- 172.21.15.101:0/3811172946 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ff7180a960 con 0x55ff718052d0 2023-07-18T03:17:54.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.158+0000 7f663dffb700 1 -- 172.21.15.101:0/3811172946 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6648097050 con 0x55ff718052d0 2023-07-18T03:17:54.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.158+0000 7f6670281b80 10 client.27665 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:17:54.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.158+0000 7f6670281b80 10 client.27665 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:17:54.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.158+0000 7f6670281b80 20 client.27665 populate_metadata read hostname 'smithi101' 2023-07-18T03:17:54.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.158+0000 7f6670281b80 1 --2- 172.21.15.101:0/3811172946 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55ff71874fa0 0x55ff71877380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:54.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.158+0000 7f6670281b80 4 client.27665 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:17:54.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.158+0000 7f6670281b80 1 -- 172.21.15.101:0/3811172946 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f66480a3620 con 0x55ff71874fa0 2023-07-18T03:17:54.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.158+0000 7f6655d9b700 1 --2- 172.21.15.101:0/3811172946 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55ff71874fa0 0x55ff71877380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:54.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.158+0000 7f663dffb700 10 client.27665 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:17:54.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.158+0000 7f6655d9b700 1 --2- 172.21.15.101:0/3811172946 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55ff71874fa0 0x55ff71877380 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a5c45e0066dfe339 server_cookie=201832f429123ac0 in_seq=0 out_seq=0 2023-07-18T03:17:54.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.159+0000 7f663dffb700 1 -- 172.21.15.101:0/3811172946 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f66480a3620 con 0x55ff71874fa0 2023-07-18T03:17:54.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.159+0000 7f663dffb700 10 client.27665 handle_command_reply: tid=0 2023-07-18T03:17:54.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.159+0000 7f6670281b80 1 client.27665 shutdown 2023-07-18T03:17:54.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.159+0000 7f6670281b80 1 -- 172.21.15.101:0/3811172946 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff718052d0 msgr2=0x55ff7186d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:54.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.159+0000 7f6670281b80 1 --2- 172.21.15.101:0/3811172946 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff718052d0 0x55ff7186d680 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f66480b9e80 tx=0x7f66480cb930 comp rx=0 tx=0).stop 2023-07-18T03:17:54.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.159+0000 7f6670281b80 1 -- 172.21.15.101:0/3811172946 shutdown_connections 2023-07-18T03:17:54.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.159+0000 7f6670281b80 1 --2- 172.21.15.101:0/3811172946 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55ff71874fa0 0x55ff71877380 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.159+0000 7f6670281b80 1 --2- 172.21.15.101:0/3811172946 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff718052d0 0x55ff7186d680 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.159+0000 7f6670281b80 1 --2- 172.21.15.101:0/3811172946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff71810ad0 0x55ff71867b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.159+0000 7f6670281b80 1 --2- 172.21.15.101:0/3811172946 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ff71805be0 0x55ff7186dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.159+0000 7f663dffb700 0 client.27665 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:17:54.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.159+0000 7f6670281b80 1 -- 172.21.15.101:0/3811172946 >> 172.21.15.101:0/3811172946 conn(0x55ff717fbda0 msgr2=0x55ff718073e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:54.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.159+0000 7f6670281b80 1 -- 172.21.15.101:0/3811172946 shutdown_connections 2023-07-18T03:17:54.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.159+0000 7f6670281b80 1 -- 172.21.15.101:0/3811172946 wait complete. 2023-07-18T03:17:54.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.159+0000 7f6670281b80 20 client.27665 trim_cache size 0 max 16384 2023-07-18T03:17:54.164 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:17:54.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.160+0000 7f665c9ff700 1 -- 172.21.15.101:0/2719944038 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f66400a1340 msgr2=0x7f66400a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:54.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.160+0000 7f665c9ff700 1 --2- 172.21.15.101:0/2719944038 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f66400a1340 0x7f66400a37f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6658074920 tx=0x7f6650009ba0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.160+0000 7f665c9ff700 1 -- 172.21.15.101:0/2719944038 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f66580ffad0 msgr2=0x7f6658072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:54.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.160+0000 7f665c9ff700 1 --2- 172.21.15.101:0/2719944038 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f66580ffad0 0x7f6658072810 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f6648005910 tx=0x7f6648004730 comp rx=0 tx=0).stop 2023-07-18T03:17:54.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.160+0000 7f665c9ff700 1 -- 172.21.15.101:0/2719944038 shutdown_connections 2023-07-18T03:17:54.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.160+0000 7f665c9ff700 1 --2- 172.21.15.101:0/2719944038 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f66400a1340 0x7f66400a37f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.160+0000 7f665c9ff700 1 --2- 172.21.15.101:0/2719944038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6658105ae0 0x7f66581a3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.160+0000 7f665c9ff700 1 --2- 172.21.15.101:0/2719944038 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f66580ffad0 0x7f6658072810 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.160+0000 7f665c9ff700 1 --2- 172.21.15.101:0/2719944038 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6658100480 0x7f6658072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.160+0000 7f665c9ff700 1 -- 172.21.15.101:0/2719944038 >> 172.21.15.101:0/2719944038 conn(0x7f66580fb5f0 msgr2=0x7f66580fbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:54.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.160+0000 7f665c9ff700 1 -- 172.21.15.101:0/2719944038 shutdown_connections 2023-07-18T03:17:54.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.160+0000 7f665c9ff700 1 -- 172.21.15.101:0/2719944038 wait complete. 2023-07-18T03:17:54.170 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:54 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1578441379' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:17:54.172 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-18T03:17:54.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.389+0000 7ff69c1ff700 1 -- 172.21.15.101:0/2153262927 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff6940731c0 msgr2=0x7ff69410bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:54.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.389+0000 7ff69c1ff700 1 --2- 172.21.15.101:0/2153262927 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff6940731c0 0x7ff69410bde0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ff684009990 tx=0x7ff684009ca0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.389+0000 7ff69c1ff700 1 -- 172.21.15.101:0/2153262927 shutdown_connections 2023-07-18T03:17:54.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.389+0000 7ff69c1ff700 1 --2- 172.21.15.101:0/2153262927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6940728b0 0x7ff694072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.389+0000 7ff69c1ff700 1 --2- 172.21.15.101:0/2153262927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff69410c320 0x7ff69410e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.389+0000 7ff69c1ff700 1 --2- 172.21.15.101:0/2153262927 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff6940731c0 0x7ff69410bde0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.389+0000 7ff69c1ff700 1 -- 172.21.15.101:0/2153262927 >> 172.21.15.101:0/2153262927 conn(0x7ff6940fb5e0 msgr2=0x7ff6940fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:54.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.389+0000 7ff69c1ff700 1 -- 172.21.15.101:0/2153262927 shutdown_connections 2023-07-18T03:17:54.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.389+0000 7ff69c1ff700 1 -- 172.21.15.101:0/2153262927 wait complete. 2023-07-18T03:17:54.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.391+0000 7ff69c1ff700 1 Processor -- start 2023-07-18T03:17:54.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.391+0000 7ff69c1ff700 1 -- start start 2023-07-18T03:17:54.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.392+0000 7ff69c1ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6940728b0 0x7ff694197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:54.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.392+0000 7ff69c1ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6940731c0 0x7ff694198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:54.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.392+0000 7ff69c1ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff69410c320 0x7ff69419c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:54.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.392+0000 7ff69c1ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69419cdb0 con 0x7ff69410c320 2023-07-18T03:17:54.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.392+0000 7ff69c1ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69419cf20 con 0x7ff6940731c0 2023-07-18T03:17:54.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.392+0000 7ff69c1ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69419d0e0 con 0x7ff6940728b0 2023-07-18T03:17:54.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.392+0000 7ff699f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6940728b0 0x7ff694197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:54.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.392+0000 7ff699f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6940728b0 0x7ff694197dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:39678/0 (socket says 172.21.15.101:39678) 2023-07-18T03:17:54.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.392+0000 7ff69a79c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff69410c320 0x7ff69419c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:54.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.392+0000 7ff699f9b700 1 -- 172.21.15.101:0/2990103585 learned_addr learned my addr 172.21.15.101:0/2990103585 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:54.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.392+0000 7ff69979a700 1 --2- 172.21.15.101:0/2990103585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6940731c0 0x7ff694198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:54.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.392+0000 7ff699f9b700 1 -- 172.21.15.101:0/2990103585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6940731c0 msgr2=0x7ff694198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:54.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.392+0000 7ff699f9b700 1 --2- 172.21.15.101:0/2990103585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6940731c0 0x7ff694198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.392+0000 7ff699f9b700 1 -- 172.21.15.101:0/2990103585 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff69410c320 msgr2=0x7ff69419c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:54.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.392+0000 7ff699f9b700 1 --2- 172.21.15.101:0/2990103585 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff69410c320 0x7ff69419c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.392+0000 7ff699f9b700 1 -- 172.21.15.101:0/2990103585 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6840095f0 con 0x7ff6940728b0 2023-07-18T03:17:54.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.393+0000 7ff699f9b700 1 --2- 172.21.15.101:0/2990103585 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6940728b0 0x7ff694197dd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff69000e9a0 tx=0x7ff69000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:54.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.393+0000 7ff68affd700 1 -- 172.21.15.101:0/2990103585 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff690009790 con 0x7ff6940728b0 2023-07-18T03:17:54.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.393+0000 7ff68affd700 1 -- 172.21.15.101:0/2990103585 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff690010430 con 0x7ff6940728b0 2023-07-18T03:17:54.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.393+0000 7ff69c1ff700 1 -- 172.21.15.101:0/2990103585 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff69419d3c0 con 0x7ff6940728b0 2023-07-18T03:17:54.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.393+0000 7ff68affd700 1 -- 172.21.15.101:0/2990103585 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff690018680 con 0x7ff6940728b0 2023-07-18T03:17:54.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.393+0000 7ff69c1ff700 1 -- 172.21.15.101:0/2990103585 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6941a4c00 con 0x7ff6940728b0 2023-07-18T03:17:54.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.396+0000 7ff68affd700 1 -- 172.21.15.101:0/2990103585 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7ff6900105a0 con 0x7ff6940728b0 2023-07-18T03:17:54.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.398+0000 7ff68affd700 1 --2- 172.21.15.101:0/2990103585 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff6800a12c0 0x7ff6800a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:54.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.398+0000 7ff68affd700 1 -- 172.21.15.101:0/2990103585 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7ff690014070 con 0x7ff6940728b0 2023-07-18T03:17:54.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.398+0000 7ff69979a700 1 --2- 172.21.15.101:0/2990103585 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff6800a12c0 0x7ff6800a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:54.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.398+0000 7ff69979a700 1 --2- 172.21.15.101:0/2990103585 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff6800a12c0 0x7ff6800a3770 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff6941993e0 tx=0x7ff684005fd0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:54.394 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.399+0000 7ff6afb90b80 1 Processor -- start 2023-07-18T03:17:54.394 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.399+0000 7ff6afb90b80 1 -- start start 2023-07-18T03:17:54.395 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.400+0000 7ff6afb90b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8541292d0 0x55f8541296a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:54.395 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.400+0000 7ff6afb90b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f854129be0 0x55f854134590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:54.395 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.400+0000 7ff6afb90b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f854134ad0 0x55f854136eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:54.396 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.400+0000 7ff6afb90b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f85402fcb0 con 0x55f854129be0 2023-07-18T03:17:54.396 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.400+0000 7ff6afb90b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f854082a40 con 0x55f854134ad0 2023-07-18T03:17:54.396 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.400+0000 7ff6afb90b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f85407bed0 con 0x55f8541292d0 2023-07-18T03:17:54.396 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.400+0000 7ff699f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8541292d0 0x55f8541296a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:54.396 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.400+0000 7ff699f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8541292d0 0x55f8541296a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:39686/0 (socket says 172.21.15.101:39686) 2023-07-18T03:17:54.396 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.400+0000 7ff699f9b700 1 -- 172.21.15.101:0/2925339671 learned_addr learned my addr 172.21.15.101:0/2925339671 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:54.396 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.400+0000 7ff69979a700 1 --2- 172.21.15.101:0/2925339671 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f854129be0 0x55f854134590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:54.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.400+0000 7ff69a79c700 1 --2- 172.21.15.101:0/2925339671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f854134ad0 0x55f854136eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:54.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.400+0000 7ff699f9b700 1 -- 172.21.15.101:0/2925339671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f854134ad0 msgr2=0x55f854136eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:54.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.400+0000 7ff699f9b700 1 --2- 172.21.15.101:0/2925339671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f854134ad0 0x55f854136eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.400+0000 7ff699f9b700 1 -- 172.21.15.101:0/2925339671 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f854129be0 msgr2=0x55f854134590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:54.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.400+0000 7ff699f9b700 1 --2- 172.21.15.101:0/2925339671 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f854129be0 0x55f854134590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.400+0000 7ff699f9b700 1 -- 172.21.15.101:0/2925339671 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f853ea9de0 con 0x55f8541292d0 2023-07-18T03:17:54.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.400+0000 7ff69a79c700 1 --2- 172.21.15.101:0/2925339671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f854134ad0 0x55f854136eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:17:54.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.401+0000 7ff699f9b700 1 --2- 172.21.15.101:0/2925339671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8541292d0 0x55f8541296a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff690096a20 tx=0x7ff690099c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85df8da8184997 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:54.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.401+0000 7ff67ffff700 1 -- 172.21.15.101:0/2925339671 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6900a2400 con 0x55f8541292d0 2023-07-18T03:17:54.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.401+0000 7ff67ffff700 1 -- 172.21.15.101:0/2925339671 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff690010880 con 0x55f8541292d0 2023-07-18T03:17:54.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.401+0000 7ff67ffff700 1 -- 172.21.15.101:0/2925339671 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6900996e0 con 0x55f8541292d0 2023-07-18T03:17:54.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.402+0000 7ff6afb90b80 1 -- 172.21.15.101:0/2925339671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8541292d0 msgr2=0x55f8541296a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:54.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.402+0000 7ff6afb90b80 1 --2- 172.21.15.101:0/2925339671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8541292d0 0x55f8541296a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff690096a20 tx=0x7ff690099c20 comp rx=0 tx=0).stop 2023-07-18T03:17:54.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.402+0000 7ff6afb90b80 1 -- 172.21.15.101:0/2925339671 shutdown_connections 2023-07-18T03:17:54.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.402+0000 7ff6afb90b80 1 --2- 172.21.15.101:0/2925339671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f854134ad0 0x55f854136eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.402+0000 7ff6afb90b80 1 --2- 172.21.15.101:0/2925339671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8541292d0 0x55f8541296a0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.402+0000 7ff6afb90b80 1 --2- 172.21.15.101:0/2925339671 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f854129be0 0x55f854134590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.402+0000 7ff6afb90b80 1 -- 172.21.15.101:0/2925339671 >> 172.21.15.101:0/2925339671 conn(0x55f85411fda0 msgr2=0x55f8541256a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:54.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.402+0000 7ff6afb90b80 1 -- 172.21.15.101:0/2925339671 shutdown_connections 2023-07-18T03:17:54.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.402+0000 7ff6afb90b80 1 -- 172.21.15.101:0/2925339671 wait complete. 2023-07-18T03:17:54.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.402+0000 7ff6afb90b80 1 -- start start 2023-07-18T03:17:54.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.403+0000 7ff6afb90b80 1 Processor -- start 2023-07-18T03:17:54.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.403+0000 7ff6afb90b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8541292d0 0x55f854131850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:54.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.403+0000 7ff6afb90b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f854129be0 0x55f854131d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:54.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.403+0000 7ff6afb90b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f854134ad0 0x55f8541322d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:54.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.403+0000 7ff6afb90b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f85401bac0 con 0x55f854129be0 2023-07-18T03:17:54.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.403+0000 7ff6afb90b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f85401b7c0 con 0x55f854134ad0 2023-07-18T03:17:54.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.403+0000 7ff6afb90b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f853eff920 con 0x55f8541292d0 2023-07-18T03:17:54.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.403+0000 7ff6afb90b80 10 client.0 _ll_register_callbacks cb 0x55f854019760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:17:54.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.403+0000 7ff69979a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f854129be0 0x55f854131d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:54.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.403+0000 7ff699f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8541292d0 0x55f854131850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:54.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.403+0000 7ff699f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8541292d0 0x55f854131850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:39692/0 (socket says 172.21.15.101:39692) 2023-07-18T03:17:54.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.403+0000 7ff699f9b700 1 -- 172.21.15.101:0/1221067012 learned_addr learned my addr 172.21.15.101:0/1221067012 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:54.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.403+0000 7ff69a79c700 1 --2- 172.21.15.101:0/1221067012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f854134ad0 0x55f8541322d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:54.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.404+0000 7ff699f9b700 1 -- 172.21.15.101:0/1221067012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f854134ad0 msgr2=0x55f8541322d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:54.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.404+0000 7ff699f9b700 1 --2- 172.21.15.101:0/1221067012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f854134ad0 0x55f8541322d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.404+0000 7ff699f9b700 1 -- 172.21.15.101:0/1221067012 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f854129be0 msgr2=0x55f854131d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:54.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.404+0000 7ff699f9b700 1 --2- 172.21.15.101:0/1221067012 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f854129be0 0x55f854131d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.404+0000 7ff699f9b700 1 -- 172.21.15.101:0/1221067012 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6941a4c00 con 0x55f8541292d0 2023-07-18T03:17:54.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.404+0000 7ff699f9b700 1 --2- 172.21.15.101:0/1221067012 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8541292d0 0x55f854131850 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff690099b30 tx=0x7ff69009a440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a810ae9e9e304ee6 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:54.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.404+0000 7ff67d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:17:54.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.404+0000 7ff67d7fa700 1 -- 172.21.15.101:0/1221067012 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6900a2400 con 0x55f8541292d0 2023-07-18T03:17:54.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.404+0000 7ff67d7fa700 1 -- 172.21.15.101:0/1221067012 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7ff6900a2180 con 0x55f8541292d0 2023-07-18T03:17:54.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.404+0000 7ff6afb90b80 1 -- 172.21.15.101:0/1221067012 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f853fd54f0 con 0x55f8541292d0 2023-07-18T03:17:54.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.404+0000 7ff67d7fa700 1 -- 172.21.15.101:0/1221067012 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff690094bc0 con 0x55f8541292d0 2023-07-18T03:17:54.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.404+0000 7ff6afb90b80 10 client.27709 fetch_fsmap learned FSMap version 33 2023-07-18T03:17:54.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.404+0000 7ff6afb90b80 1 -- 172.21.15.101:0/1221067012 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55f853e92c60 con 0x55f8541292d0 2023-07-18T03:17:54.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.405+0000 7ff67d7fa700 1 -- 172.21.15.101:0/1221067012 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f853fd54f0 con 0x55f8541292d0 2023-07-18T03:17:54.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.405+0000 7ff67d7fa700 1 -- 172.21.15.101:0/1221067012 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff690099030 con 0x55f8541292d0 2023-07-18T03:17:54.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.405+0000 7ff6afb90b80 10 client.27709 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:17:54.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.405+0000 7ff6afb90b80 10 client.27709 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:17:54.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.405+0000 7ff6afb90b80 20 client.27709 populate_metadata read hostname 'smithi101' 2023-07-18T03:17:54.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.405+0000 7ff6afb90b80 1 --2- 172.21.15.101:0/1221067012 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f8541440f0 0x55f8541464d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:54.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.405+0000 7ff6afb90b80 4 client.27709 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:17:54.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.405+0000 7ff6afb90b80 1 -- 172.21.15.101:0/1221067012 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f854074eb0 con 0x55f8541440f0 2023-07-18T03:17:54.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.405+0000 7ff69979a700 1 --2- 172.21.15.101:0/1221067012 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f8541440f0 0x55f8541464d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:54.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.406+0000 7ff67d7fa700 10 client.27709 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:17:54.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.406+0000 7ff69979a700 1 --2- 172.21.15.101:0/1221067012 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f8541440f0 0x55f8541464d0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d7d78b3bb0a6687 server_cookie=2064c6f8eef351db in_seq=0 out_seq=0 2023-07-18T03:17:54.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.406+0000 7ff67d7fa700 1 -- 172.21.15.101:0/1221067012 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f854074eb0 con 0x55f8541440f0 2023-07-18T03:17:54.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.406+0000 7ff67d7fa700 10 client.27709 handle_command_reply: tid=0 2023-07-18T03:17:54.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.406+0000 7ff6afb90b80 1 client.27709 shutdown 2023-07-18T03:17:54.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.406+0000 7ff6afb90b80 1 -- 172.21.15.101:0/1221067012 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8541292d0 msgr2=0x55f854131850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:54.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.406+0000 7ff6afb90b80 1 --2- 172.21.15.101:0/1221067012 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8541292d0 0x55f854131850 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff690099b30 tx=0x7ff69009a440 comp rx=0 tx=0).stop 2023-07-18T03:17:54.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.406+0000 7ff6afb90b80 1 -- 172.21.15.101:0/1221067012 shutdown_connections 2023-07-18T03:17:54.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.406+0000 7ff6afb90b80 1 --2- 172.21.15.101:0/1221067012 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f8541440f0 0x55f8541464d0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.406+0000 7ff6afb90b80 1 --2- 172.21.15.101:0/1221067012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f854134ad0 0x55f8541322d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.406+0000 7ff6afb90b80 1 --2- 172.21.15.101:0/1221067012 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8541292d0 0x55f854131850 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.406+0000 7ff6afb90b80 1 --2- 172.21.15.101:0/1221067012 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f854129be0 0x55f854131d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.406+0000 7ff67d7fa700 0 client.27709 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:17:54.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.406+0000 7ff6afb90b80 1 -- 172.21.15.101:0/1221067012 >> 172.21.15.101:0/1221067012 conn(0x55f85411fda0 msgr2=0x55f854125230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:54.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.407+0000 7ff6afb90b80 1 -- 172.21.15.101:0/1221067012 shutdown_connections 2023-07-18T03:17:54.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.407+0000 7ff6afb90b80 1 -- 172.21.15.101:0/1221067012 wait complete. 2023-07-18T03:17:54.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.407+0000 7ff6afb90b80 20 client.27709 trim_cache size 0 max 16384 2023-07-18T03:17:54.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.425+0000 7ff6afb90b80 1 Processor -- start 2023-07-18T03:17:54.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.425+0000 7ff6afb90b80 1 -- start start 2023-07-18T03:17:54.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.426+0000 7ff6afb90b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8541292d0 0x55f85413b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:54.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.426+0000 7ff6afb90b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f854129be0 0x55f854132030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:54.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.426+0000 7ff6afb90b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f854134ad0 0x55f854132570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:54.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.426+0000 7ff6afb90b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f85412ad60 con 0x55f854129be0 2023-07-18T03:17:54.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.426+0000 7ff6afb90b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f85411cf80 con 0x55f854134ad0 2023-07-18T03:17:54.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.426+0000 7ff6afb90b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f85412b1a0 con 0x55f8541292d0 2023-07-18T03:17:54.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.426+0000 7ff699f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8541292d0 0x55f85413b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:54.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.426+0000 7ff699f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8541292d0 0x55f85413b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:39702/0 (socket says 172.21.15.101:39702) 2023-07-18T03:17:54.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.426+0000 7ff699f9b700 1 -- 172.21.15.101:0/1004879086 learned_addr learned my addr 172.21.15.101:0/1004879086 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:54.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.426+0000 7ff69a79c700 1 --2- 172.21.15.101:0/1004879086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f854134ad0 0x55f854132570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:54.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.426+0000 7ff699f9b700 1 -- 172.21.15.101:0/1004879086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f854134ad0 msgr2=0x55f854132570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:54.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.426+0000 7ff699f9b700 1 --2- 172.21.15.101:0/1004879086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f854134ad0 0x55f854132570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.426+0000 7ff69979a700 1 --2- 172.21.15.101:0/1004879086 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f854129be0 0x55f854132030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:54.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.426+0000 7ff699f9b700 1 -- 172.21.15.101:0/1004879086 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f854129be0 msgr2=0x55f854132030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:54.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.426+0000 7ff699f9b700 1 --2- 172.21.15.101:0/1004879086 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f854129be0 0x55f854132030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.426+0000 7ff699f9b700 1 -- 172.21.15.101:0/1004879086 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8541215e0 con 0x55f8541292d0 2023-07-18T03:17:54.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.427+0000 7ff69979a700 1 --2- 172.21.15.101:0/1004879086 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f854129be0 0x55f854132030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:17:54.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.427+0000 7ff699f9b700 1 --2- 172.21.15.101:0/1004879086 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8541292d0 0x55f85413b4f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff690099f40 tx=0x7ff690099c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57256c0557196a14 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:54.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.427+0000 7ff688ff9700 1 -- 172.21.15.101:0/1004879086 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69009a030 con 0x55f8541292d0 2023-07-18T03:17:54.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.428+0000 7ff688ff9700 1 -- 172.21.15.101:0/1004879086 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6900b8030 con 0x55f8541292d0 2023-07-18T03:17:54.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.428+0000 7ff688ff9700 1 -- 172.21.15.101:0/1004879086 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff690094bc0 con 0x55f8541292d0 2023-07-18T03:17:54.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.429+0000 7ff6afb90b80 1 -- 172.21.15.101:0/1004879086 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8541292d0 msgr2=0x55f85413b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:54.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.429+0000 7ff6afb90b80 1 --2- 172.21.15.101:0/1004879086 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8541292d0 0x55f85413b4f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff690099f40 tx=0x7ff690099c20 comp rx=0 tx=0).stop 2023-07-18T03:17:54.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.429+0000 7ff6afb90b80 1 -- 172.21.15.101:0/1004879086 shutdown_connections 2023-07-18T03:17:54.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.429+0000 7ff6afb90b80 1 --2- 172.21.15.101:0/1004879086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f854134ad0 0x55f854132570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.429+0000 7ff6afb90b80 1 --2- 172.21.15.101:0/1004879086 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8541292d0 0x55f85413b4f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.429+0000 7ff6afb90b80 1 --2- 172.21.15.101:0/1004879086 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f854129be0 0x55f854132030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.429+0000 7ff6afb90b80 1 -- 172.21.15.101:0/1004879086 >> 172.21.15.101:0/1004879086 conn(0x55f85411fda0 msgr2=0x55f85412cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:54.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.429+0000 7ff6afb90b80 1 -- 172.21.15.101:0/1004879086 shutdown_connections 2023-07-18T03:17:54.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.429+0000 7ff6afb90b80 1 -- 172.21.15.101:0/1004879086 wait complete. 2023-07-18T03:17:54.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.430+0000 7ff6afb90b80 1 -- start start 2023-07-18T03:17:54.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.430+0000 7ff6afb90b80 1 Processor -- start 2023-07-18T03:17:54.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.431+0000 7ff6afb90b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8541292d0 0x55f85418beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:54.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.431+0000 7ff6afb90b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f854129be0 0x55f85418c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:54.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.431+0000 7ff6afb90b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f854134ad0 0x55f85418c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:54.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.431+0000 7ff6afb90b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f854197010 con 0x55f854129be0 2023-07-18T03:17:54.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.431+0000 7ff6afb90b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f854197150 con 0x55f854134ad0 2023-07-18T03:17:54.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.431+0000 7ff6afb90b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f854197310 con 0x55f8541292d0 2023-07-18T03:17:54.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.431+0000 7ff6afb90b80 10 client.0 _ll_register_callbacks cb 0x7ff684005e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:17:54.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.431+0000 7ff699f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8541292d0 0x55f85418beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:54.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.431+0000 7ff699f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8541292d0 0x55f85418beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:39706/0 (socket says 172.21.15.101:39706) 2023-07-18T03:17:54.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.431+0000 7ff699f9b700 1 -- 172.21.15.101:0/2598456662 learned_addr learned my addr 172.21.15.101:0/2598456662 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:54.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.431+0000 7ff69a79c700 1 --2- 172.21.15.101:0/2598456662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f854134ad0 0x55f85418c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:54.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.431+0000 7ff69979a700 1 --2- 172.21.15.101:0/2598456662 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f854129be0 0x55f85418c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:54.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.431+0000 7ff699f9b700 1 -- 172.21.15.101:0/2598456662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f854134ad0 msgr2=0x55f85418c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:54.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.431+0000 7ff699f9b700 1 --2- 172.21.15.101:0/2598456662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f854134ad0 0x55f85418c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.431+0000 7ff699f9b700 1 -- 172.21.15.101:0/2598456662 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f854129be0 msgr2=0x55f85418c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:54.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.431+0000 7ff699f9b700 1 --2- 172.21.15.101:0/2598456662 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f854129be0 0x55f85418c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.431+0000 7ff699f9b700 1 -- 172.21.15.101:0/2598456662 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f853e92c60 con 0x55f8541292d0 2023-07-18T03:17:54.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.432+0000 7ff699f9b700 1 --2- 172.21.15.101:0/2598456662 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8541292d0 0x55f85418beb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff690000c00 tx=0x7ff6900999c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a07ca32dab8a2576 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:54.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.432+0000 7ff67d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:17:54.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.432+0000 7ff67d7fa700 1 -- 172.21.15.101:0/2598456662 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6900a2400 con 0x55f8541292d0 2023-07-18T03:17:54.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.432+0000 7ff67d7fa700 1 -- 172.21.15.101:0/2598456662 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7ff690094bc0 con 0x55f8541292d0 2023-07-18T03:17:54.432 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.432+0000 7ff6afb90b80 1 -- 172.21.15.101:0/2598456662 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff690010880 con 0x55f8541292d0 2023-07-18T03:17:54.432 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.432+0000 7ff67d7fa700 1 -- 172.21.15.101:0/2598456662 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff690094d80 con 0x55f8541292d0 2023-07-18T03:17:54.432 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.432+0000 7ff6afb90b80 10 client.27715 fetch_fsmap learned FSMap version 33 2023-07-18T03:17:54.432 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.432+0000 7ff6afb90b80 1 -- 172.21.15.101:0/2598456662 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55f853ed0250 con 0x55f8541292d0 2023-07-18T03:17:54.432 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.432+0000 7ff67d7fa700 1 -- 172.21.15.101:0/2598456662 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff690010880 con 0x55f8541292d0 2023-07-18T03:17:54.432 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.432+0000 7ff67d7fa700 1 -- 172.21.15.101:0/2598456662 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff69009aeb0 con 0x55f8541292d0 2023-07-18T03:17:54.432 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.433+0000 7ff6afb90b80 10 client.27715 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:17:54.433 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.433+0000 7ff6afb90b80 10 client.27715 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:17:54.433 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.433+0000 7ff6afb90b80 20 client.27715 populate_metadata read hostname 'smithi101' 2023-07-18T03:17:54.433 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.433+0000 7ff6afb90b80 1 --2- 172.21.15.101:0/2598456662 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f854198520 0x55f85419a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:54.433 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.433+0000 7ff6afb90b80 4 client.27715 mds_command: new command op to 24562 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:17:54.433 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.433+0000 7ff6afb90b80 1 -- 172.21.15.101:0/2598456662 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f854037050 con 0x55f854198520 2023-07-18T03:17:54.433 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.433+0000 7ff69979a700 1 --2- 172.21.15.101:0/2598456662 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f854198520 0x55f85419a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:54.433 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.433+0000 7ff69979a700 1 --2- 172.21.15.101:0/2598456662 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f854198520 0x55f85419a930 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d33dc6ffacb67fb server_cookie=ff0dc8adc7b332f in_seq=0 out_seq=0 2023-07-18T03:17:54.433 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.433+0000 7ff67d7fa700 10 client.27715 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:17:54.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.434+0000 7ff67d7fa700 1 -- 172.21.15.101:0/2598456662 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f854037050 con 0x55f854198520 2023-07-18T03:17:54.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.434+0000 7ff67d7fa700 10 client.27715 handle_command_reply: tid=0 2023-07-18T03:17:54.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.434+0000 7ff6afb90b80 1 client.27715 shutdown 2023-07-18T03:17:54.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.434+0000 7ff6afb90b80 1 -- 172.21.15.101:0/2598456662 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8541292d0 msgr2=0x55f85418beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:54.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.434+0000 7ff6afb90b80 1 --2- 172.21.15.101:0/2598456662 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8541292d0 0x55f85418beb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff690000c00 tx=0x7ff6900999c0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.435 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.434+0000 7ff6afb90b80 1 -- 172.21.15.101:0/2598456662 shutdown_connections 2023-07-18T03:17:54.435 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.434+0000 7ff6afb90b80 1 --2- 172.21.15.101:0/2598456662 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f854198520 0x55f85419a930 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.435 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.434+0000 7ff6afb90b80 1 --2- 172.21.15.101:0/2598456662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f854134ad0 0x55f85418c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.435 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.434+0000 7ff6afb90b80 1 --2- 172.21.15.101:0/2598456662 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8541292d0 0x55f85418beb0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.434+0000 7ff6afb90b80 1 --2- 172.21.15.101:0/2598456662 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f854129be0 0x55f85418c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.434+0000 7ff6afb90b80 1 -- 172.21.15.101:0/2598456662 >> 172.21.15.101:0/2598456662 conn(0x55f85411fda0 msgr2=0x55f854132030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:54.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.434+0000 7ff67d7fa700 0 client.27715 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:17:54.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.434+0000 7ff6afb90b80 1 -- 172.21.15.101:0/2598456662 shutdown_connections 2023-07-18T03:17:54.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.434+0000 7ff6afb90b80 1 -- 172.21.15.101:0/2598456662 wait complete. 2023-07-18T03:17:54.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.435+0000 7ff6afb90b80 20 client.27715 trim_cache size 0 max 16384 2023-07-18T03:17:54.437 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:17:54.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.435+0000 7ff69c1ff700 1 -- 172.21.15.101:0/2990103585 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff6800a12c0 msgr2=0x7ff6800a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:54.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.435+0000 7ff69c1ff700 1 --2- 172.21.15.101:0/2990103585 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff6800a12c0 0x7ff6800a3770 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff6941993e0 tx=0x7ff684005fd0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.435+0000 7ff69c1ff700 1 -- 172.21.15.101:0/2990103585 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6940728b0 msgr2=0x7ff694197dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:54.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.435+0000 7ff69c1ff700 1 --2- 172.21.15.101:0/2990103585 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6940728b0 0x7ff694197dd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff69000e9a0 tx=0x7ff69000ecb0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.435+0000 7ff69c1ff700 1 -- 172.21.15.101:0/2990103585 shutdown_connections 2023-07-18T03:17:54.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.435+0000 7ff69c1ff700 1 --2- 172.21.15.101:0/2990103585 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff69410c320 0x7ff69419c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.435+0000 7ff69c1ff700 1 --2- 172.21.15.101:0/2990103585 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6940728b0 0x7ff694197dd0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.435+0000 7ff69c1ff700 1 --2- 172.21.15.101:0/2990103585 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff6800a12c0 0x7ff6800a3770 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.435+0000 7ff69c1ff700 1 --2- 172.21.15.101:0/2990103585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6940731c0 0x7ff694198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:54.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.435+0000 7ff69c1ff700 1 -- 172.21.15.101:0/2990103585 >> 172.21.15.101:0/2990103585 conn(0x7ff6940fb5e0 msgr2=0x7ff69410e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:54.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.436+0000 7ff69c1ff700 1 -- 172.21.15.101:0/2990103585 shutdown_connections 2023-07-18T03:17:54.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:54.436+0000 7ff69c1ff700 1 -- 172.21.15.101:0/2990103585 wait complete. 2023-07-18T03:17:54.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:54 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1578441379' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:17:54.546 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:54 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1578441379' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:17:54.617 INFO:tasks.workunit.client.0.smithi101.stdout: 1 648441 5.77 MB/sec execute 439 sec latency 2398.269 ms 2023-07-18T03:17:55.451 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-18T03:17:55.451 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-18T03:17:55.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:55 smithi179 ceph-mon[129229]: pgmap v321: 129 pgs: 129 active+clean; 509 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 69 op/s 2023-07-18T03:17:55.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:55 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:17:55.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:55 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:17:55.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:55 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:17:55.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:55 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:17:55.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:55 smithi101 ceph-mon[126379]: pgmap v321: 129 pgs: 129 active+clean; 509 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 69 op/s 2023-07-18T03:17:55.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:55 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:17:55.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:55 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:17:55.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:55 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:17:55.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:55 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:17:55.546 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:55 smithi155 ceph-mon[128885]: pgmap v321: 129 pgs: 129 active+clean; 509 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 69 op/s 2023-07-18T03:17:55.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:55 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:17:55.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:55 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:17:55.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:55 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:17:55.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:55 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:17:55.617 INFO:tasks.workunit.client.0.smithi101.stdout: 1 650095 5.79 MB/sec execute 440 sec latency 31.754 ms 2023-07-18T03:17:55.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.685+0000 7fc3d4bff700 1 -- 172.21.15.101:0/2140426537 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3d0100490 msgr2=0x7fc3d010c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:55.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.685+0000 7fc3d4bff700 1 --2- 172.21.15.101:0/2140426537 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3d0100490 0x7fc3d010c920 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009f90 tx=0x7fc3c800b330 comp rx=0 tx=0).stop 2023-07-18T03:17:55.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.685+0000 7fc3d4bff700 1 -- 172.21.15.101:0/2140426537 shutdown_connections 2023-07-18T03:17:55.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.685+0000 7fc3d4bff700 1 --2- 172.21.15.101:0/2140426537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3d0105af0 0x7fc3d0105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:55.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.685+0000 7fc3d4bff700 1 --2- 172.21.15.101:0/2140426537 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3d0100490 0x7fc3d010c920 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:55.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.685+0000 7fc3d4bff700 1 --2- 172.21.15.101:0/2140426537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3d00ffae0 0x7fc3d00fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:55.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.685+0000 7fc3d4bff700 1 -- 172.21.15.101:0/2140426537 >> 172.21.15.101:0/2140426537 conn(0x7fc3d00fb5e0 msgr2=0x7fc3d00fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:55.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.686+0000 7fc3d4bff700 1 -- 172.21.15.101:0/2140426537 shutdown_connections 2023-07-18T03:17:55.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.686+0000 7fc3d4bff700 1 -- 172.21.15.101:0/2140426537 wait complete. 2023-07-18T03:17:55.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.687+0000 7fc3d4bff700 1 Processor -- start 2023-07-18T03:17:55.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.687+0000 7fc3d4bff700 1 -- start start 2023-07-18T03:17:55.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.687+0000 7fc3d4bff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3d00ffae0 0x7fc3d0072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:55.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.687+0000 7fc3d4bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3d0100490 0x7fc3d0072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:55.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.688+0000 7fc3d4bff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3d0105af0 0x7fc3d01a3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:55.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.688+0000 7fc3d4bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d0073580 con 0x7fc3d0100490 2023-07-18T03:17:55.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.688+0000 7fc3d4bff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d01a4420 con 0x7fc3d0105af0 2023-07-18T03:17:55.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.688+0000 7fc3d4bff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d01a4560 con 0x7fc3d00ffae0 2023-07-18T03:17:55.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.688+0000 7fc3cdd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3d0100490 0x7fc3d0072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:55.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.688+0000 7fc3ce59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3d00ffae0 0x7fc3d0072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:55.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.688+0000 7fc3cdd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3d0100490 0x7fc3d0072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:58028/0 (socket says 172.21.15.101:58028) 2023-07-18T03:17:55.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.688+0000 7fc3ced9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3d0105af0 0x7fc3d01a3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:55.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.688+0000 7fc3cdd9b700 1 -- 172.21.15.101:0/3989741681 learned_addr learned my addr 172.21.15.101:0/3989741681 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:55.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.688+0000 7fc3ced9d700 1 -- 172.21.15.101:0/3989741681 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3d00ffae0 msgr2=0x7fc3d0072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:55.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.688+0000 7fc3ced9d700 1 --2- 172.21.15.101:0/3989741681 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3d00ffae0 0x7fc3d0072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:55.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.688+0000 7fc3ced9d700 1 -- 172.21.15.101:0/3989741681 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3d0100490 msgr2=0x7fc3d0072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:55.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.688+0000 7fc3ced9d700 1 --2- 172.21.15.101:0/3989741681 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3d0100490 0x7fc3d0072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:55.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.688+0000 7fc3ced9d700 1 -- 172.21.15.101:0/3989741681 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c00095f0 con 0x7fc3d0105af0 2023-07-18T03:17:55.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.689+0000 7fc3ced9d700 1 --2- 172.21.15.101:0/3989741681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3d0105af0 0x7fc3d01a3ee0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc3c800b6c0 tx=0x7fc3c80094b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:55.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.690+0000 7fc3bf7fe700 1 -- 172.21.15.101:0/3989741681 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c800e040 con 0x7fc3d0105af0 2023-07-18T03:17:55.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.691+0000 7fc3bf7fe700 1 -- 172.21.15.101:0/3989741681 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3c801d470 con 0x7fc3d0105af0 2023-07-18T03:17:55.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.691+0000 7fc3bf7fe700 1 -- 172.21.15.101:0/3989741681 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c80052a0 con 0x7fc3d0105af0 2023-07-18T03:17:55.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.691+0000 7fc3d4bff700 1 -- 172.21.15.101:0/3989741681 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3c8009c40 con 0x7fc3d0105af0 2023-07-18T03:17:55.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.691+0000 7fc3d4bff700 1 -- 172.21.15.101:0/3989741681 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3d01a4a00 con 0x7fc3d0105af0 2023-07-18T03:17:55.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.694+0000 7fc3bf7fe700 1 -- 172.21.15.101:0/3989741681 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fc3c800b9f0 con 0x7fc3d0105af0 2023-07-18T03:17:55.690 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.695+0000 7fc3bf7fe700 1 --2- 172.21.15.101:0/3989741681 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc3b80a1340 0x7fc3b80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:55.690 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.695+0000 7fc3bf7fe700 1 -- 172.21.15.101:0/3989741681 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fc3c80ce380 con 0x7fc3d0105af0 2023-07-18T03:17:55.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.696+0000 7fc3ce59c700 1 --2- 172.21.15.101:0/3989741681 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc3b80a1340 0x7fc3b80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:55.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.696+0000 7fc3e8494b80 1 Processor -- start 2023-07-18T03:17:55.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.696+0000 7fc3e8494b80 1 -- start start 2023-07-18T03:17:55.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.696+0000 7fc3ce59c700 1 --2- 172.21.15.101:0/3989741681 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc3b80a1340 0x7fc3b80a37f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0005fd0 tx=0x7fc3c0009310 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:55.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.697+0000 7fc3e8494b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dcd62a4350 0x55dcd62a4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:55.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.697+0000 7fc3e8494b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcd62a4c60 0x55dcd62af610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:55.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.697+0000 7fc3e8494b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcd62afb50 0x55dcd62b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:55.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.697+0000 7fc3e8494b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcd61aad30 con 0x55dcd62a4350 2023-07-18T03:17:55.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.697+0000 7fc3e8494b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcd61fdac0 con 0x55dcd62a4c60 2023-07-18T03:17:55.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.697+0000 7fc3e8494b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcd61f6f50 con 0x55dcd62afb50 2023-07-18T03:17:55.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.697+0000 7fc3ce59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dcd62a4350 0x55dcd62a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:55.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.697+0000 7fc3cdd9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcd62a4c60 0x55dcd62af610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:55.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.697+0000 7fc3ce59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dcd62a4350 0x55dcd62a4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:58032/0 (socket says 172.21.15.101:58032) 2023-07-18T03:17:55.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.697+0000 7fc3ce59c700 1 -- 172.21.15.101:0/2859740879 learned_addr learned my addr 172.21.15.101:0/2859740879 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:55.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.697+0000 7fc3ced9d700 1 --2- 172.21.15.101:0/2859740879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcd62afb50 0x55dcd62b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:55.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.697+0000 7fc3cdd9b700 1 -- 172.21.15.101:0/2859740879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcd62afb50 msgr2=0x55dcd62b1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:55.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.697+0000 7fc3cdd9b700 1 --2- 172.21.15.101:0/2859740879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcd62afb50 0x55dcd62b1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:55.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.697+0000 7fc3cdd9b700 1 -- 172.21.15.101:0/2859740879 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dcd62a4350 msgr2=0x55dcd62a4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:55.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.697+0000 7fc3cdd9b700 1 --2- 172.21.15.101:0/2859740879 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dcd62a4350 0x55dcd62a4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:55.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.697+0000 7fc3cdd9b700 1 -- 172.21.15.101:0/2859740879 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dcd6024e60 con 0x55dcd62a4c60 2023-07-18T03:17:55.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.698+0000 7fc3cdd9b700 1 --2- 172.21.15.101:0/2859740879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcd62a4c60 0x55dcd62af610 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0074000 tx=0x7fc3c4011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a44e0d52ef6408b8 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:55.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.698+0000 7fc3bcff9700 1 -- 172.21.15.101:0/2859740879 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c4011cc0 con 0x55dcd62a4c60 2023-07-18T03:17:55.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.698+0000 7fc3bcff9700 1 -- 172.21.15.101:0/2859740879 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3c4011e20 con 0x55dcd62a4c60 2023-07-18T03:17:55.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.698+0000 7fc3bcff9700 1 -- 172.21.15.101:0/2859740879 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c4005280 con 0x55dcd62a4c60 2023-07-18T03:17:55.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.699+0000 7fc3e8494b80 1 -- 172.21.15.101:0/2859740879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcd62a4c60 msgr2=0x55dcd62af610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:55.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.699+0000 7fc3e8494b80 1 --2- 172.21.15.101:0/2859740879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcd62a4c60 0x55dcd62af610 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0074000 tx=0x7fc3c4011440 comp rx=0 tx=0).stop 2023-07-18T03:17:55.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.699+0000 7fc3e8494b80 1 -- 172.21.15.101:0/2859740879 shutdown_connections 2023-07-18T03:17:55.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.699+0000 7fc3e8494b80 1 --2- 172.21.15.101:0/2859740879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcd62afb50 0x55dcd62b1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:55.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.699+0000 7fc3e8494b80 1 --2- 172.21.15.101:0/2859740879 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dcd62a4350 0x55dcd62a4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:55.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.699+0000 7fc3e8494b80 1 --2- 172.21.15.101:0/2859740879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcd62a4c60 0x55dcd62af610 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:55.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.699+0000 7fc3e8494b80 1 -- 172.21.15.101:0/2859740879 >> 172.21.15.101:0/2859740879 conn(0x55dcd629ae20 msgr2=0x55dcd62a0720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:55.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.699+0000 7fc3e8494b80 1 -- 172.21.15.101:0/2859740879 shutdown_connections 2023-07-18T03:17:55.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.699+0000 7fc3e8494b80 1 -- 172.21.15.101:0/2859740879 wait complete. 2023-07-18T03:17:55.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.700+0000 7fc3e8494b80 1 -- start start 2023-07-18T03:17:55.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.700+0000 7fc3e8494b80 1 Processor -- start 2023-07-18T03:17:55.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.700+0000 7fc3e8494b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcd62a4350 0x55dcd62af550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:55.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.700+0000 7fc3e8494b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dcd62a4c60 0x55dcd62ac840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:55.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.700+0000 7fc3e8494b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcd62afb50 0x55dcd62acec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:55.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.700+0000 7fc3e8494b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcd6196b40 con 0x55dcd62a4c60 2023-07-18T03:17:55.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.700+0000 7fc3e8494b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcd6196840 con 0x55dcd62afb50 2023-07-18T03:17:55.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.700+0000 7fc3e8494b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcd607a9a0 con 0x55dcd62a4350 2023-07-18T03:17:55.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.701+0000 7fc3e8494b80 10 client.0 _ll_register_callbacks cb 0x55dcd61947e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:17:55.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.701+0000 7fc3ce59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcd62a4350 0x55dcd62af550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:55.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.701+0000 7fc3ce59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcd62a4350 0x55dcd62af550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:39750/0 (socket says 172.21.15.101:39750) 2023-07-18T03:17:55.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.701+0000 7fc3ce59c700 1 -- 172.21.15.101:0/2156688983 learned_addr learned my addr 172.21.15.101:0/2156688983 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:55.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.701+0000 7fc3cdd9b700 1 --2- 172.21.15.101:0/2156688983 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dcd62a4c60 0x55dcd62ac840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:55.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.701+0000 7fc3ced9d700 1 --2- 172.21.15.101:0/2156688983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcd62afb50 0x55dcd62acec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:55.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.701+0000 7fc3ce59c700 1 -- 172.21.15.101:0/2156688983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcd62afb50 msgr2=0x55dcd62acec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:55.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.701+0000 7fc3ce59c700 1 --2- 172.21.15.101:0/2156688983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcd62afb50 0x55dcd62acec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:55.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.701+0000 7fc3ce59c700 1 -- 172.21.15.101:0/2156688983 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dcd62a4c60 msgr2=0x55dcd62ac840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:55.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.701+0000 7fc3ce59c700 1 --2- 172.21.15.101:0/2156688983 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dcd62a4c60 0x55dcd62ac840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:55.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.701+0000 7fc3ce59c700 1 -- 172.21.15.101:0/2156688983 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3c4009970 con 0x55dcd62a4350 2023-07-18T03:17:55.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.701+0000 7fc3cdd9b700 1 --2- 172.21.15.101:0/2156688983 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dcd62a4c60 0x55dcd62ac840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:17:55.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.702+0000 7fc3ce59c700 1 --2- 172.21.15.101:0/2156688983 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcd62a4350 0x55dcd62af550 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0003900 tx=0x7fc3c00047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=709f90b6b52ef405 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:55.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.702+0000 7fc3b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:17:55.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.702+0000 7fc3b5ffb700 1 -- 172.21.15.101:0/2156688983 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c0022070 con 0x55dcd62a4350 2023-07-18T03:17:55.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.702+0000 7fc3e8494b80 1 -- 172.21.15.101:0/2156688983 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dcd6150570 con 0x55dcd62a4350 2023-07-18T03:17:55.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.702+0000 7fc3b5ffb700 1 -- 172.21.15.101:0/2156688983 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fc3c000e8a0 con 0x55dcd62a4350 2023-07-18T03:17:55.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.702+0000 7fc3b5ffb700 1 -- 172.21.15.101:0/2156688983 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3c000ea60 con 0x55dcd62a4350 2023-07-18T03:17:55.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.702+0000 7fc3e8494b80 10 client.27727 fetch_fsmap learned FSMap version 33 2023-07-18T03:17:55.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.702+0000 7fc3e8494b80 1 -- 172.21.15.101:0/2156688983 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55dcd600dce0 con 0x55dcd62a4350 2023-07-18T03:17:55.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.702+0000 7fc3b5ffb700 1 -- 172.21.15.101:0/2156688983 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dcd6150570 con 0x55dcd62a4350 2023-07-18T03:17:55.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.703+0000 7fc3b5ffb700 1 -- 172.21.15.101:0/2156688983 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc3c001d410 con 0x55dcd62a4350 2023-07-18T03:17:55.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.703+0000 7fc3e8494b80 10 client.27727 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:17:55.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.703+0000 7fc3e8494b80 10 client.27727 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:17:55.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.703+0000 7fc3e8494b80 20 client.27727 populate_metadata read hostname 'smithi101' 2023-07-18T03:17:55.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.703+0000 7fc3e8494b80 1 --2- 172.21.15.101:0/2156688983 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55dcd62bf460 0x55dcd62c1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:55.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.703+0000 7fc3e8494b80 4 client.27727 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:17:55.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.703+0000 7fc3e8494b80 1 -- 172.21.15.101:0/2156688983 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dcd61eff30 con 0x55dcd62bf460 2023-07-18T03:17:55.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.703+0000 7fc3cdd9b700 1 --2- 172.21.15.101:0/2156688983 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55dcd62bf460 0x55dcd62c1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:55.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.704+0000 7fc3b5ffb700 10 client.27727 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:17:55.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.704+0000 7fc3cdd9b700 1 --2- 172.21.15.101:0/2156688983 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55dcd62bf460 0x55dcd62c1870 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c771236a8bdc7c0 server_cookie=db1cb18385250bc1 in_seq=0 out_seq=0 2023-07-18T03:17:55.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.704+0000 7fc3b5ffb700 1 -- 172.21.15.101:0/2156688983 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55dcd61eff30 con 0x55dcd62bf460 2023-07-18T03:17:55.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.704+0000 7fc3b5ffb700 10 client.27727 handle_command_reply: tid=0 2023-07-18T03:17:55.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.704+0000 7fc3e8494b80 1 client.27727 shutdown 2023-07-18T03:17:55.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.705+0000 7fc3e8494b80 1 -- 172.21.15.101:0/2156688983 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcd62a4350 msgr2=0x55dcd62af550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:55.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.705+0000 7fc3e8494b80 1 --2- 172.21.15.101:0/2156688983 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcd62a4350 0x55dcd62af550 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0003900 tx=0x7fc3c00047c0 comp rx=0 tx=0).stop 2023-07-18T03:17:55.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.705+0000 7fc3e8494b80 1 -- 172.21.15.101:0/2156688983 shutdown_connections 2023-07-18T03:17:55.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.705+0000 7fc3e8494b80 1 --2- 172.21.15.101:0/2156688983 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55dcd62bf460 0x55dcd62c1870 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:55.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.705+0000 7fc3e8494b80 1 --2- 172.21.15.101:0/2156688983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcd62afb50 0x55dcd62acec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:55.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.705+0000 7fc3e8494b80 1 --2- 172.21.15.101:0/2156688983 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcd62a4350 0x55dcd62af550 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:55.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.705+0000 7fc3b5ffb700 0 client.27727 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:17:55.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.705+0000 7fc3e8494b80 1 --2- 172.21.15.101:0/2156688983 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dcd62a4c60 0x55dcd62ac840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:55.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.705+0000 7fc3e8494b80 1 -- 172.21.15.101:0/2156688983 >> 172.21.15.101:0/2156688983 conn(0x55dcd629ae20 msgr2=0x55dcd62a0310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:55.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.705+0000 7fc3e8494b80 1 -- 172.21.15.101:0/2156688983 shutdown_connections 2023-07-18T03:17:55.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.705+0000 7fc3e8494b80 1 -- 172.21.15.101:0/2156688983 wait complete. 2023-07-18T03:17:55.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.705+0000 7fc3e8494b80 20 client.27727 trim_cache size 0 max 16384 2023-07-18T03:17:55.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.724+0000 7fc3e8494b80 1 Processor -- start 2023-07-18T03:17:55.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.724+0000 7fc3e8494b80 1 -- start start 2023-07-18T03:17:55.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.724+0000 7fc3e8494b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcd62a4350 0x55dcd62b6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:55.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.724+0000 7fc3e8494b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dcd62a4c60 0x55dcd62a6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:55.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.724+0000 7fc3e8494b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcd62afb50 0x55dcd62a6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:55.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.724+0000 7fc3e8494b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcd62a5de0 con 0x55dcd62a4c60 2023-07-18T03:17:55.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.724+0000 7fc3e8494b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcd6298020 con 0x55dcd62afb50 2023-07-18T03:17:55.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.724+0000 7fc3e8494b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcd629a9f0 con 0x55dcd62a4350 2023-07-18T03:17:55.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.725+0000 7fc3ce59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcd62a4350 0x55dcd62b6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:55.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.725+0000 7fc3ced9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcd62afb50 0x55dcd62a6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:55.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.725+0000 7fc3ce59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcd62a4350 0x55dcd62b6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:39754/0 (socket says 172.21.15.101:39754) 2023-07-18T03:17:55.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.725+0000 7fc3ce59c700 1 -- 172.21.15.101:0/220113681 learned_addr learned my addr 172.21.15.101:0/220113681 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:55.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.725+0000 7fc3cdd9b700 1 --2- 172.21.15.101:0/220113681 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dcd62a4c60 0x55dcd62a6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:55.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.725+0000 7fc3ced9d700 1 -- 172.21.15.101:0/220113681 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcd62a4350 msgr2=0x55dcd62b6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:55.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.725+0000 7fc3ced9d700 1 --2- 172.21.15.101:0/220113681 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcd62a4350 0x55dcd62b6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:55.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.725+0000 7fc3ced9d700 1 -- 172.21.15.101:0/220113681 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dcd62a4c60 msgr2=0x55dcd62a6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:55.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.725+0000 7fc3ced9d700 1 --2- 172.21.15.101:0/220113681 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dcd62a4c60 0x55dcd62a6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:55.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.725+0000 7fc3ced9d700 1 -- 172.21.15.101:0/220113681 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dcd629c660 con 0x55dcd62afb50 2023-07-18T03:17:55.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.725+0000 7fc3ce59c700 1 --2- 172.21.15.101:0/220113681 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcd62a4350 0x55dcd62b6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:17:55.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.725+0000 7fc3ced9d700 1 --2- 172.21.15.101:0/220113681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcd62afb50 0x55dcd62a6e70 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc3c80cb6c0 tx=0x7fc3c801dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30589e6e52b10c55 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:55.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.726+0000 7fc3bd7fa700 1 -- 172.21.15.101:0/220113681 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c809b0d0 con 0x55dcd62afb50 2023-07-18T03:17:55.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.726+0000 7fc3bd7fa700 1 -- 172.21.15.101:0/220113681 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3c809f100 con 0x55dcd62afb50 2023-07-18T03:17:55.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.726+0000 7fc3bd7fa700 1 -- 172.21.15.101:0/220113681 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c801d730 con 0x55dcd62afb50 2023-07-18T03:17:55.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.726+0000 7fc3e8494b80 1 -- 172.21.15.101:0/220113681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcd62afb50 msgr2=0x55dcd62a6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:55.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.726+0000 7fc3e8494b80 1 --2- 172.21.15.101:0/220113681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcd62afb50 0x55dcd62a6e70 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc3c80cb6c0 tx=0x7fc3c801dc90 comp rx=0 tx=0).stop 2023-07-18T03:17:55.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.726+0000 7fc3e8494b80 1 -- 172.21.15.101:0/220113681 shutdown_connections 2023-07-18T03:17:55.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.726+0000 7fc3e8494b80 1 --2- 172.21.15.101:0/220113681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcd62afb50 0x55dcd62a6e70 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:55.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.726+0000 7fc3e8494b80 1 --2- 172.21.15.101:0/220113681 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcd62a4350 0x55dcd62b6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:55.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.726+0000 7fc3e8494b80 1 --2- 172.21.15.101:0/220113681 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dcd62a4c60 0x55dcd62a6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:55.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.726+0000 7fc3e8494b80 1 -- 172.21.15.101:0/220113681 >> 172.21.15.101:0/220113681 conn(0x55dcd629ae20 msgr2=0x55dcd62a6220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:55.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.727+0000 7fc3e8494b80 1 -- 172.21.15.101:0/220113681 shutdown_connections 2023-07-18T03:17:55.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.727+0000 7fc3e8494b80 1 -- 172.21.15.101:0/220113681 wait complete. 2023-07-18T03:17:55.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.727+0000 7fc3e8494b80 1 -- start start 2023-07-18T03:17:55.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.727+0000 7fc3e8494b80 1 Processor -- start 2023-07-18T03:17:55.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.728+0000 7fc3e8494b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcd62a4350 0x55dcd62b4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:55.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.728+0000 7fc3e8494b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcd62a4c60 0x55dcd62b5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:55.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.728+0000 7fc3e8494b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dcd62afb50 0x55dcd62b5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:55.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.728+0000 7fc3e8494b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcd6313330 con 0x55dcd62afb50 2023-07-18T03:17:55.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.728+0000 7fc3e8494b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcd6313470 con 0x55dcd62a4350 2023-07-18T03:17:55.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.728+0000 7fc3e8494b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcd63135e0 con 0x55dcd62a4c60 2023-07-18T03:17:55.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.728+0000 7fc3e8494b80 10 client.0 _ll_register_callbacks cb 0x7fc3c0004500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:17:55.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.728+0000 7fc3ce59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcd62a4350 0x55dcd62b4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:55.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.728+0000 7fc3cdd9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcd62a4c60 0x55dcd62b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:55.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.728+0000 7fc3cdd9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcd62a4c60 0x55dcd62b5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:39764/0 (socket says 172.21.15.101:39764) 2023-07-18T03:17:55.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.728+0000 7fc3cdd9b700 1 -- 172.21.15.101:0/2209053249 learned_addr learned my addr 172.21.15.101:0/2209053249 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:55.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.728+0000 7fc3ced9d700 1 --2- 172.21.15.101:0/2209053249 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dcd62afb50 0x55dcd62b5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:55.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.729+0000 7fc3cdd9b700 1 -- 172.21.15.101:0/2209053249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcd62a4350 msgr2=0x55dcd62b4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:55.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.729+0000 7fc3cdd9b700 1 --2- 172.21.15.101:0/2209053249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcd62a4350 0x55dcd62b4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:55.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.729+0000 7fc3cdd9b700 1 -- 172.21.15.101:0/2209053249 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dcd62afb50 msgr2=0x55dcd62b5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:55.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.729+0000 7fc3cdd9b700 1 --2- 172.21.15.101:0/2209053249 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dcd62afb50 0x55dcd62b5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:55.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.729+0000 7fc3cdd9b700 1 -- 172.21.15.101:0/2209053249 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dcd600dce0 con 0x55dcd62a4c60 2023-07-18T03:17:55.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.729+0000 7fc3ce59c700 1 --2- 172.21.15.101:0/2209053249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcd62a4350 0x55dcd62b4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:17:55.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.729+0000 7fc3cdd9b700 1 --2- 172.21.15.101:0/2209053249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcd62a4c60 0x55dcd62b5100 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0074000 tx=0x7fc3c4009790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2136ab1213584e92 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:55.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.729+0000 7fc3b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:17:55.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.729+0000 7fc3b5ffb700 1 -- 172.21.15.101:0/2209053249 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c4011c40 con 0x55dcd62a4c60 2023-07-18T03:17:55.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.729+0000 7fc3e8494b80 1 -- 172.21.15.101:0/2209053249 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc3c4011e20 con 0x55dcd62a4c60 2023-07-18T03:17:55.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.729+0000 7fc3b5ffb700 1 -- 172.21.15.101:0/2209053249 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fc3c4028110 con 0x55dcd62a4c60 2023-07-18T03:17:55.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.730+0000 7fc3b5ffb700 1 -- 172.21.15.101:0/2209053249 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3c40282d0 con 0x55dcd62a4c60 2023-07-18T03:17:55.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.730+0000 7fc3e8494b80 10 client.27733 fetch_fsmap learned FSMap version 33 2023-07-18T03:17:55.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.730+0000 7fc3e8494b80 1 -- 172.21.15.101:0/2209053249 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7fc3d01a4a00 con 0x55dcd62a4c60 2023-07-18T03:17:55.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.730+0000 7fc3b5ffb700 1 -- 172.21.15.101:0/2209053249 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3c4011e20 con 0x55dcd62a4c60 2023-07-18T03:17:55.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.730+0000 7fc3b5ffb700 1 -- 172.21.15.101:0/2209053249 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc3c4003680 con 0x55dcd62a4c60 2023-07-18T03:17:55.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.730+0000 7fc3e8494b80 10 client.27733 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:17:55.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.730+0000 7fc3e8494b80 10 client.27733 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:17:55.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.730+0000 7fc3e8494b80 20 client.27733 populate_metadata read hostname 'smithi101' 2023-07-18T03:17:55.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.730+0000 7fc3e8494b80 1 --2- 172.21.15.101:0/2209053249 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55dcd6314cd0 0x55dcd63170e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:55.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.730+0000 7fc3e8494b80 4 client.27733 mds_command: new command op to 24562 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-18T03:17:55.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.730+0000 7fc3e8494b80 1 -- 172.21.15.101:0/2209053249 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55dcd61b20d0 con 0x55dcd6314cd0 2023-07-18T03:17:55.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.730+0000 7fc3ce59c700 1 --2- 172.21.15.101:0/2209053249 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55dcd6314cd0 0x55dcd63170e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:55.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.731+0000 7fc3ce59c700 1 --2- 172.21.15.101:0/2209053249 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55dcd6314cd0 0x55dcd63170e0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=177105299c501431 server_cookie=40bda2c1c11b5970 in_seq=0 out_seq=0 2023-07-18T03:17:55.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.731+0000 7fc3b5ffb700 10 client.27733 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:17:55.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.732+0000 7fc3b5ffb700 1 -- 172.21.15.101:0/2209053249 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55dcd61b20d0 con 0x55dcd6314cd0 2023-07-18T03:17:55.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.732+0000 7fc3b5ffb700 10 client.27733 handle_command_reply: tid=0 2023-07-18T03:17:55.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.732+0000 7fc3e8494b80 1 client.27733 shutdown 2023-07-18T03:17:55.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.732+0000 7fc3e8494b80 1 -- 172.21.15.101:0/2209053249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcd62a4c60 msgr2=0x55dcd62b5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:55.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.732+0000 7fc3e8494b80 1 --2- 172.21.15.101:0/2209053249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcd62a4c60 0x55dcd62b5100 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0074000 tx=0x7fc3c4009790 comp rx=0 tx=0).stop 2023-07-18T03:17:55.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.732+0000 7fc3e8494b80 1 -- 172.21.15.101:0/2209053249 shutdown_connections 2023-07-18T03:17:55.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.732+0000 7fc3e8494b80 1 --2- 172.21.15.101:0/2209053249 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55dcd6314cd0 0x55dcd63170e0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:55.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.732+0000 7fc3e8494b80 1 --2- 172.21.15.101:0/2209053249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcd62a4350 0x55dcd62b4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:55.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.732+0000 7fc3e8494b80 1 --2- 172.21.15.101:0/2209053249 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dcd62afb50 0x55dcd62b5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:55.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.732+0000 7fc3e8494b80 1 --2- 172.21.15.101:0/2209053249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dcd62a4c60 0x55dcd62b5100 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:55.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.732+0000 7fc3b5ffb700 0 client.27733 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:17:55.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.732+0000 7fc3e8494b80 1 -- 172.21.15.101:0/2209053249 >> 172.21.15.101:0/2209053249 conn(0x55dcd629ae20 msgr2=0x55dcd62a6ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:55.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.732+0000 7fc3e8494b80 1 -- 172.21.15.101:0/2209053249 shutdown_connections 2023-07-18T03:17:55.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.732+0000 7fc3e8494b80 1 -- 172.21.15.101:0/2209053249 wait complete. 2023-07-18T03:17:55.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.733+0000 7fc3e8494b80 20 client.27733 trim_cache size 0 max 16384 2023-07-18T03:17:55.735 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:17:55.735 INFO:teuthology.orchestra.run.smithi101.stdout: "return_code": 0, 2023-07-18T03:17:55.735 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_tag": "2f35c27f-b228-4708-a2e6-b03ffc6edf56", 2023-07-18T03:17:55.735 INFO:teuthology.orchestra.run.smithi101.stdout: "mode": "asynchronous" 2023-07-18T03:17:55.735 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:17:55.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.733+0000 7fc3d4bff700 1 -- 172.21.15.101:0/3989741681 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc3b80a1340 msgr2=0x7fc3b80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:55.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.733+0000 7fc3d4bff700 1 --2- 172.21.15.101:0/3989741681 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc3b80a1340 0x7fc3b80a37f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0005fd0 tx=0x7fc3c0009310 comp rx=0 tx=0).stop 2023-07-18T03:17:55.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.733+0000 7fc3d4bff700 1 -- 172.21.15.101:0/3989741681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3d0105af0 msgr2=0x7fc3d01a3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:55.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.733+0000 7fc3d4bff700 1 --2- 172.21.15.101:0/3989741681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3d0105af0 0x7fc3d01a3ee0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc3c800b6c0 tx=0x7fc3c80094b0 comp rx=0 tx=0).stop 2023-07-18T03:17:55.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.733+0000 7fc3d4bff700 1 -- 172.21.15.101:0/3989741681 shutdown_connections 2023-07-18T03:17:55.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.733+0000 7fc3d4bff700 1 --2- 172.21.15.101:0/3989741681 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc3b80a1340 0x7fc3b80a37f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:55.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.733+0000 7fc3d4bff700 1 --2- 172.21.15.101:0/3989741681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3d0105af0 0x7fc3d01a3ee0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:55.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.733+0000 7fc3d4bff700 1 --2- 172.21.15.101:0/3989741681 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3d00ffae0 0x7fc3d0072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:55.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.733+0000 7fc3d4bff700 1 --2- 172.21.15.101:0/3989741681 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3d0100490 0x7fc3d0072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:55.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.733+0000 7fc3d4bff700 1 -- 172.21.15.101:0/3989741681 >> 172.21.15.101:0/3989741681 conn(0x7fc3d00fb5e0 msgr2=0x7fc3d00fbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:55.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.734+0000 7fc3d4bff700 1 -- 172.21.15.101:0/3989741681 shutdown_connections 2023-07-18T03:17:55.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:55.734+0000 7fc3d4bff700 1 -- 172.21.15.101:0/3989741681 wait complete. 2023-07-18T03:17:55.753 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:17:56.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.018+0000 7f6d9e9ff700 1 -- 172.21.15.101:0/903305668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d98106fb0 msgr2=0x7f6d9810eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:56.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.018+0000 7f6d9e9ff700 1 --2- 172.21.15.101:0/903305668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d98106fb0 0x7f6d9810eb80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6d9000b0a0 tx=0x7f6d9000b3b0 comp rx=0 tx=0).stop 2023-07-18T03:17:56.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.018+0000 7f6d9e9ff700 1 -- 172.21.15.101:0/903305668 shutdown_connections 2023-07-18T03:17:56.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.018+0000 7f6d9e9ff700 1 --2- 172.21.15.101:0/903305668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d98106fb0 0x7f6d9810eb80 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:56.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.018+0000 7f6d9e9ff700 1 --2- 172.21.15.101:0/903305668 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d98101f70 0x7f6d98102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:56.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.018+0000 7f6d9e9ff700 1 --2- 172.21.15.101:0/903305668 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6d98102910 0x7f6d981068d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:56.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.018+0000 7f6d9e9ff700 1 -- 172.21.15.101:0/903305668 >> 172.21.15.101:0/903305668 conn(0x7f6d980fd7e0 msgr2=0x7f6d980ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:56.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.018+0000 7f6d9e9ff700 1 -- 172.21.15.101:0/903305668 shutdown_connections 2023-07-18T03:17:56.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.018+0000 7f6d9e9ff700 1 -- 172.21.15.101:0/903305668 wait complete. 2023-07-18T03:17:56.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.019+0000 7f6d9e9ff700 1 Processor -- start 2023-07-18T03:17:56.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.019+0000 7f6d9e9ff700 1 -- start start 2023-07-18T03:17:56.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.020+0000 7f6d9e9ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6d98101f70 0x7f6d98197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:56.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.020+0000 7f6d9e9ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d98102910 0x7f6d98198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:56.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.020+0000 7f6d9e9ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d98106fb0 0x7f6d9819c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:56.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.020+0000 7f6d9e9ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9819cf10 con 0x7f6d98101f70 2023-07-18T03:17:56.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.020+0000 7f6d9e9ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9819d080 con 0x7f6d98102910 2023-07-18T03:17:56.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.020+0000 7f6d9e9ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9819d240 con 0x7f6d98106fb0 2023-07-18T03:17:56.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.020+0000 7f6d977fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d98102910 0x7f6d98198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:56.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.020+0000 7f6d97fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6d98101f70 0x7f6d98197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:56.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.020+0000 7f6d977fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d98102910 0x7f6d98198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:52042/0 (socket says 172.21.15.101:52042) 2023-07-18T03:17:56.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.020+0000 7f6d977fe700 1 -- 172.21.15.101:0/919950658 learned_addr learned my addr 172.21.15.101:0/919950658 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:56.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.020+0000 7f6d9cf9c700 1 --2- 172.21.15.101:0/919950658 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d98106fb0 0x7f6d9819c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:56.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.020+0000 7f6d977fe700 1 -- 172.21.15.101:0/919950658 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d98106fb0 msgr2=0x7f6d9819c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:56.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.020+0000 7f6d977fe700 1 --2- 172.21.15.101:0/919950658 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d98106fb0 0x7f6d9819c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:56.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.020+0000 7f6d977fe700 1 -- 172.21.15.101:0/919950658 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6d98101f70 msgr2=0x7f6d98197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:56.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.020+0000 7f6d977fe700 1 --2- 172.21.15.101:0/919950658 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6d98101f70 0x7f6d98197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:56.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.020+0000 7f6d977fe700 1 -- 172.21.15.101:0/919950658 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d90009d00 con 0x7f6d98102910 2023-07-18T03:17:56.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.021+0000 7f6d977fe700 1 --2- 172.21.15.101:0/919950658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d98102910 0x7f6d98198470 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c00e940 tx=0x7f6d8c00ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:56.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.021+0000 7f6d957fa700 1 -- 172.21.15.101:0/919950658 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c009730 con 0x7f6d98102910 2023-07-18T03:17:56.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.021+0000 7f6d957fa700 1 -- 172.21.15.101:0/919950658 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d8c010430 con 0x7f6d98102910 2023-07-18T03:17:56.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.021+0000 7f6d957fa700 1 -- 172.21.15.101:0/919950658 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c00cea0 con 0x7f6d98102910 2023-07-18T03:17:56.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.021+0000 7f6d9e9ff700 1 -- 172.21.15.101:0/919950658 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d9819d520 con 0x7f6d98102910 2023-07-18T03:17:56.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.021+0000 7f6d9e9ff700 1 -- 172.21.15.101:0/919950658 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d981a4d60 con 0x7f6d98102910 2023-07-18T03:17:56.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.026+0000 7f6d957fa700 1 -- 172.21.15.101:0/919950658 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f6d8c010810 con 0x7f6d98102910 2023-07-18T03:17:56.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.027+0000 7f6db22cfb80 1 Processor -- start 2023-07-18T03:17:56.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.027+0000 7f6db22cfb80 1 -- start start 2023-07-18T03:17:56.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.027+0000 7f6db22cfb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aaaf1012d0 0x55aaaf1016a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:56.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.027+0000 7f6db22cfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aaaf101be0 0x55aaaf10c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:56.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.027+0000 7f6db22cfb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aaaf10cad0 0x55aaaf10eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:56.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.027+0000 7f6db22cfb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaaf007cb0 con 0x55aaaf1012d0 2023-07-18T03:17:56.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.027+0000 7f6db22cfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaaf05aa40 con 0x55aaaf101be0 2023-07-18T03:17:56.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.027+0000 7f6db22cfb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaaf053ed0 con 0x55aaaf10cad0 2023-07-18T03:17:56.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.028+0000 7f6d977fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aaaf10cad0 0x55aaaf10eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:56.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.028+0000 7f6d9cf9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aaaf1012d0 0x55aaaf1016a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:56.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.028+0000 7f6d977fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aaaf10cad0 0x55aaaf10eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:34370/0 (socket says 172.21.15.101:34370) 2023-07-18T03:17:56.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.028+0000 7f6d977fe700 1 -- 172.21.15.101:0/840847293 learned_addr learned my addr 172.21.15.101:0/840847293 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:56.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.028+0000 7f6d97fff700 1 --2- 172.21.15.101:0/840847293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aaaf101be0 0x55aaaf10c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:56.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.028+0000 7f6d957fa700 1 --2- 172.21.15.101:0/919950658 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6d800a1340 0x7f6d800a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:56.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.028+0000 7f6d977fe700 1 -- 172.21.15.101:0/840847293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aaaf101be0 msgr2=0x55aaaf10c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:56.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.028+0000 7f6d977fe700 1 --2- 172.21.15.101:0/840847293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aaaf101be0 0x55aaaf10c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:56.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.028+0000 7f6d977fe700 1 -- 172.21.15.101:0/840847293 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aaaf1012d0 msgr2=0x55aaaf1016a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:56.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.028+0000 7f6d977fe700 1 --2- 172.21.15.101:0/840847293 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aaaf1012d0 0x55aaaf1016a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:56.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.028+0000 7f6d957fa700 1 -- 172.21.15.101:0/919950658 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f6d8c014070 con 0x7f6d98102910 2023-07-18T03:17:56.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.028+0000 7f6d977fe700 1 -- 172.21.15.101:0/840847293 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aaaee81de0 con 0x55aaaf10cad0 2023-07-18T03:17:56.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.029+0000 7f6d9cf9c700 1 --2- 172.21.15.101:0/919950658 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6d800a1340 0x7f6d800a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:56.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.029+0000 7f6d977fe700 1 --2- 172.21.15.101:0/840847293 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aaaf10cad0 0x55aaaf10eeb0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c0ca430 tx=0x7f6d8c010c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1feb4c08a222909 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:56.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.029+0000 7f6d7e7fc700 1 -- 172.21.15.101:0/840847293 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c0d0ba0 con 0x55aaaf10cad0 2023-07-18T03:17:56.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.029+0000 7f6d7e7fc700 1 -- 172.21.15.101:0/840847293 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d8c099980 con 0x55aaaf10cad0 2023-07-18T03:17:56.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.029+0000 7f6d7e7fc700 1 -- 172.21.15.101:0/840847293 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c098270 con 0x55aaaf10cad0 2023-07-18T03:17:56.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.030+0000 7f6db22cfb80 1 -- 172.21.15.101:0/840847293 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aaaf10cad0 msgr2=0x55aaaf10eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:56.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.030+0000 7f6d9cf9c700 1 --2- 172.21.15.101:0/919950658 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6d800a1340 0x7f6d800a37f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6d9000b780 tx=0x7f6d900099e0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:56.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.030+0000 7f6db22cfb80 1 --2- 172.21.15.101:0/840847293 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aaaf10cad0 0x55aaaf10eeb0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c0ca430 tx=0x7f6d8c010c10 comp rx=0 tx=0).stop 2023-07-18T03:17:56.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.030+0000 7f6db22cfb80 1 -- 172.21.15.101:0/840847293 shutdown_connections 2023-07-18T03:17:56.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.030+0000 7f6db22cfb80 1 --2- 172.21.15.101:0/840847293 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aaaf10cad0 0x55aaaf10eeb0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:56.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.030+0000 7f6db22cfb80 1 --2- 172.21.15.101:0/840847293 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aaaf1012d0 0x55aaaf1016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:56.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.030+0000 7f6db22cfb80 1 --2- 172.21.15.101:0/840847293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aaaf101be0 0x55aaaf10c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:56.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.030+0000 7f6db22cfb80 1 -- 172.21.15.101:0/840847293 >> 172.21.15.101:0/840847293 conn(0x55aaaf0f7da0 msgr2=0x55aaaf0fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:56.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.030+0000 7f6db22cfb80 1 -- 172.21.15.101:0/840847293 shutdown_connections 2023-07-18T03:17:56.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.030+0000 7f6db22cfb80 1 -- 172.21.15.101:0/840847293 wait complete. 2023-07-18T03:17:56.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.031+0000 7f6db22cfb80 1 -- start start 2023-07-18T03:17:56.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.031+0000 7f6db22cfb80 1 Processor -- start 2023-07-18T03:17:56.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.031+0000 7f6db22cfb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aaaf1012d0 0x55aaaf10c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:56.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.031+0000 7f6db22cfb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aaaf101be0 0x55aaaf1097b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:56.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.031+0000 7f6db22cfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aaaf10cad0 0x55aaaf109e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:56.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.031+0000 7f6db22cfb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaaeff3ac0 con 0x55aaaf101be0 2023-07-18T03:17:56.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.031+0000 7f6db22cfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaaeff37c0 con 0x55aaaf10cad0 2023-07-18T03:17:56.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.031+0000 7f6db22cfb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaaeed7920 con 0x55aaaf1012d0 2023-07-18T03:17:56.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.031+0000 7f6db22cfb80 10 client.0 _ll_register_callbacks cb 0x55aaaeff1760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:17:56.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.031+0000 7f6d97fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aaaf1012d0 0x55aaaf10c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:56.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.032+0000 7f6d97fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aaaf1012d0 0x55aaaf10c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:34384/0 (socket says 172.21.15.101:34384) 2023-07-18T03:17:56.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.032+0000 7f6d97fff700 1 -- 172.21.15.101:0/1531745601 learned_addr learned my addr 172.21.15.101:0/1531745601 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:56.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.032+0000 7f6d977fe700 1 --2- 172.21.15.101:0/1531745601 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aaaf101be0 0x55aaaf1097b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:56.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.032+0000 7f6d9cf9c700 1 --2- 172.21.15.101:0/1531745601 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aaaf10cad0 0x55aaaf109e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:56.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.032+0000 7f6d97fff700 1 -- 172.21.15.101:0/1531745601 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aaaf10cad0 msgr2=0x55aaaf109e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:56.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.032+0000 7f6d97fff700 1 --2- 172.21.15.101:0/1531745601 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aaaf10cad0 0x55aaaf109e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:56.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.032+0000 7f6d97fff700 1 -- 172.21.15.101:0/1531745601 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aaaf101be0 msgr2=0x55aaaf1097b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:56.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.032+0000 7f6d97fff700 1 --2- 172.21.15.101:0/1531745601 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aaaf101be0 0x55aaaf1097b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:56.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.032+0000 7f6d97fff700 1 -- 172.21.15.101:0/1531745601 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d981a4d60 con 0x55aaaf1012d0 2023-07-18T03:17:56.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.032+0000 7f6d977fe700 1 --2- 172.21.15.101:0/1531745601 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aaaf101be0 0x55aaaf1097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:17:56.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.032+0000 7f6d97fff700 1 --2- 172.21.15.101:0/1531745601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aaaf1012d0 0x55aaaf10c4c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6d88012760 tx=0x7f6d88012a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c157c57dcd8c0f6 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:56.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.032+0000 7f6d777fe700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:17:56.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.033+0000 7f6d777fe700 1 -- 172.21.15.101:0/1531745601 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d88017070 con 0x55aaaf1012d0 2023-07-18T03:17:56.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.033+0000 7f6db22cfb80 1 -- 172.21.15.101:0/1531745601 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aaaefad4f0 con 0x55aaaf1012d0 2023-07-18T03:17:56.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.033+0000 7f6d777fe700 1 -- 172.21.15.101:0/1531745601 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f6d8800f480 con 0x55aaaf1012d0 2023-07-18T03:17:56.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.033+0000 7f6d777fe700 1 -- 172.21.15.101:0/1531745601 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d880122e0 con 0x55aaaf1012d0 2023-07-18T03:17:56.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.033+0000 7f6db22cfb80 10 client.27745 fetch_fsmap learned FSMap version 33 2023-07-18T03:17:56.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.033+0000 7f6db22cfb80 1 -- 172.21.15.101:0/1531745601 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55aaaee6ac60 con 0x55aaaf1012d0 2023-07-18T03:17:56.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.033+0000 7f6d777fe700 1 -- 172.21.15.101:0/1531745601 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55aaaefad4f0 con 0x55aaaf1012d0 2023-07-18T03:17:56.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.033+0000 7f6d777fe700 1 -- 172.21.15.101:0/1531745601 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6d880108a0 con 0x55aaaf1012d0 2023-07-18T03:17:56.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.033+0000 7f6db22cfb80 10 client.27745 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:17:56.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.033+0000 7f6db22cfb80 10 client.27745 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:17:56.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.033+0000 7f6db22cfb80 20 client.27745 populate_metadata read hostname 'smithi101' 2023-07-18T03:17:56.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.033+0000 7f6db22cfb80 1 --2- 172.21.15.101:0/1531745601 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55aaaf11c400 0x55aaaf11e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:56.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.033+0000 7f6db22cfb80 4 client.27745 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:17:56.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.033+0000 7f6db22cfb80 1 -- 172.21.15.101:0/1531745601 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aaaf04ceb0 con 0x55aaaf11c400 2023-07-18T03:17:56.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.034+0000 7f6d977fe700 1 --2- 172.21.15.101:0/1531745601 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55aaaf11c400 0x55aaaf11e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:56.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.034+0000 7f6d777fe700 10 client.27745 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:17:56.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.034+0000 7f6d977fe700 1 --2- 172.21.15.101:0/1531745601 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55aaaf11c400 0x55aaaf11e810 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb4cb0f7c2bd9189 server_cookie=5626f93a58ce5961 in_seq=0 out_seq=0 2023-07-18T03:17:56.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.035+0000 7f6d777fe700 1 -- 172.21.15.101:0/1531745601 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55aaaf04ceb0 con 0x55aaaf11c400 2023-07-18T03:17:56.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.035+0000 7f6d777fe700 10 client.27745 handle_command_reply: tid=0 2023-07-18T03:17:56.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.035+0000 7f6db22cfb80 1 client.27745 shutdown 2023-07-18T03:17:56.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.035+0000 7f6db22cfb80 1 -- 172.21.15.101:0/1531745601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aaaf1012d0 msgr2=0x55aaaf10c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:56.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.035+0000 7f6db22cfb80 1 --2- 172.21.15.101:0/1531745601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aaaf1012d0 0x55aaaf10c4c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6d88012760 tx=0x7f6d88012a70 comp rx=0 tx=0).stop 2023-07-18T03:17:56.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.035+0000 7f6db22cfb80 1 -- 172.21.15.101:0/1531745601 shutdown_connections 2023-07-18T03:17:56.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.035+0000 7f6db22cfb80 1 --2- 172.21.15.101:0/1531745601 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55aaaf11c400 0x55aaaf11e810 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:56.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.035+0000 7f6db22cfb80 1 --2- 172.21.15.101:0/1531745601 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aaaf10cad0 0x55aaaf109e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:56.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.035+0000 7f6db22cfb80 1 --2- 172.21.15.101:0/1531745601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aaaf1012d0 0x55aaaf10c4c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:56.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.035+0000 7f6d777fe700 0 client.27745 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:17:56.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.035+0000 7f6db22cfb80 1 --2- 172.21.15.101:0/1531745601 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aaaf101be0 0x55aaaf1097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:56.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.035+0000 7f6db22cfb80 1 -- 172.21.15.101:0/1531745601 >> 172.21.15.101:0/1531745601 conn(0x55aaaf0f7da0 msgr2=0x55aaaf0fd230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:56.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.035+0000 7f6db22cfb80 1 -- 172.21.15.101:0/1531745601 shutdown_connections 2023-07-18T03:17:56.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.035+0000 7f6db22cfb80 1 -- 172.21.15.101:0/1531745601 wait complete. 2023-07-18T03:17:56.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.036+0000 7f6db22cfb80 20 client.27745 trim_cache size 0 max 16384 2023-07-18T03:17:56.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.055+0000 7f6db22cfb80 1 Processor -- start 2023-07-18T03:17:56.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.056+0000 7f6db22cfb80 1 -- start start 2023-07-18T03:17:56.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.056+0000 7f6db22cfb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aaaf1012d0 0x55aaaf10c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:56.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.056+0000 7f6db22cfb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aaaf101be0 0x55aaaf108f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:56.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.056+0000 7f6db22cfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aaaf10cad0 0x55aaaf1094b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:56.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.056+0000 7f6db22cfb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaaf10ad20 con 0x55aaaf101be0 2023-07-18T03:17:56.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.056+0000 7f6db22cfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaaf0f4fa0 con 0x55aaaf10cad0 2023-07-18T03:17:56.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.056+0000 7f6db22cfb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaaf0f7970 con 0x55aaaf1012d0 2023-07-18T03:17:56.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.056+0000 7f6d97fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aaaf1012d0 0x55aaaf10c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:56.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.056+0000 7f6d977fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aaaf101be0 0x55aaaf108f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:56.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.056+0000 7f6d97fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aaaf1012d0 0x55aaaf10c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:34388/0 (socket says 172.21.15.101:34388) 2023-07-18T03:17:56.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.056+0000 7f6d97fff700 1 -- 172.21.15.101:0/209624145 learned_addr learned my addr 172.21.15.101:0/209624145 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:56.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.056+0000 7f6d9cf9c700 1 --2- 172.21.15.101:0/209624145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aaaf10cad0 0x55aaaf1094b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:56.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.057+0000 7f6d97fff700 1 -- 172.21.15.101:0/209624145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aaaf10cad0 msgr2=0x55aaaf1094b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:56.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.057+0000 7f6d97fff700 1 --2- 172.21.15.101:0/209624145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aaaf10cad0 0x55aaaf1094b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:56.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.057+0000 7f6d97fff700 1 -- 172.21.15.101:0/209624145 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aaaf101be0 msgr2=0x55aaaf108f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:56.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.057+0000 7f6d97fff700 1 --2- 172.21.15.101:0/209624145 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aaaf101be0 0x55aaaf108f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:56.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.057+0000 7f6d97fff700 1 -- 172.21.15.101:0/209624145 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aaaeea8250 con 0x55aaaf1012d0 2023-07-18T03:17:56.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.057+0000 7f6d97fff700 1 --2- 172.21.15.101:0/209624145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aaaf1012d0 0x55aaaf10c060 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6d8800ffd0 tx=0x7f6d88009770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6e187f205c58edb server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:56.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.057+0000 7f6d7effd700 1 -- 172.21.15.101:0/209624145 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d88017040 con 0x55aaaf1012d0 2023-07-18T03:17:56.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.057+0000 7f6d7effd700 1 -- 172.21.15.101:0/209624145 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d880050e0 con 0x55aaaf1012d0 2023-07-18T03:17:56.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.057+0000 7f6d7effd700 1 -- 172.21.15.101:0/209624145 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d88003970 con 0x55aaaf1012d0 2023-07-18T03:17:56.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.058+0000 7f6db22cfb80 1 -- 172.21.15.101:0/209624145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aaaf1012d0 msgr2=0x55aaaf10c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:56.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.058+0000 7f6db22cfb80 1 --2- 172.21.15.101:0/209624145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aaaf1012d0 0x55aaaf10c060 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6d8800ffd0 tx=0x7f6d88009770 comp rx=0 tx=0).stop 2023-07-18T03:17:56.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.058+0000 7f6db22cfb80 1 -- 172.21.15.101:0/209624145 shutdown_connections 2023-07-18T03:17:56.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.058+0000 7f6db22cfb80 1 --2- 172.21.15.101:0/209624145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aaaf10cad0 0x55aaaf1094b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:56.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.058+0000 7f6db22cfb80 1 --2- 172.21.15.101:0/209624145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aaaf1012d0 0x55aaaf10c060 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:56.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.058+0000 7f6db22cfb80 1 --2- 172.21.15.101:0/209624145 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aaaf101be0 0x55aaaf108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:56.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.058+0000 7f6db22cfb80 1 -- 172.21.15.101:0/209624145 >> 172.21.15.101:0/209624145 conn(0x55aaaf0f7da0 msgr2=0x55aaaf104a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:56.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.058+0000 7f6db22cfb80 1 -- 172.21.15.101:0/209624145 shutdown_connections 2023-07-18T03:17:56.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.058+0000 7f6db22cfb80 1 -- 172.21.15.101:0/209624145 wait complete. 2023-07-18T03:17:56.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.058+0000 7f6db22cfb80 1 -- start start 2023-07-18T03:17:56.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.059+0000 7f6db22cfb80 1 Processor -- start 2023-07-18T03:17:56.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.059+0000 7f6db22cfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aaaf1012d0 0x55aaaf111220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:56.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.059+0000 7f6db22cfb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aaaf101be0 0x55aaaf111760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:56.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.059+0000 7f6db22cfb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aaaf10cad0 0x55aaaf16f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:56.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.059+0000 7f6db22cfb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaaf16f800 con 0x55aaaf10cad0 2023-07-18T03:17:56.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.059+0000 7f6db22cfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaaf16f940 con 0x55aaaf1012d0 2023-07-18T03:17:56.059 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.059+0000 7f6db22cfb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaaf16fa80 con 0x55aaaf101be0 2023-07-18T03:17:56.059 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.059+0000 7f6db22cfb80 10 client.0 _ll_register_callbacks cb 0x7f6d88009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:17:56.059 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.060+0000 7f6d97fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aaaf1012d0 0x55aaaf111220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:56.059 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.060+0000 7f6d9cf9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aaaf10cad0 0x55aaaf16f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:56.059 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.060+0000 7f6d97fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aaaf1012d0 0x55aaaf111220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:52080/0 (socket says 172.21.15.101:52080) 2023-07-18T03:17:56.059 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.060+0000 7f6d97fff700 1 -- 172.21.15.101:0/3864856764 learned_addr learned my addr 172.21.15.101:0/3864856764 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:17:56.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.060+0000 7f6d977fe700 1 --2- 172.21.15.101:0/3864856764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aaaf101be0 0x55aaaf111760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:56.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.060+0000 7f6d9cf9c700 1 -- 172.21.15.101:0/3864856764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aaaf101be0 msgr2=0x55aaaf111760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:56.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.060+0000 7f6d9cf9c700 1 --2- 172.21.15.101:0/3864856764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aaaf101be0 0x55aaaf111760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:56.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.060+0000 7f6d9cf9c700 1 -- 172.21.15.101:0/3864856764 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aaaf1012d0 msgr2=0x55aaaf111220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:17:56.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.060+0000 7f6d9cf9c700 1 --2- 172.21.15.101:0/3864856764 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aaaf1012d0 0x55aaaf111220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:56.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.060+0000 7f6d9cf9c700 1 -- 172.21.15.101:0/3864856764 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aaaee6ac60 con 0x55aaaf10cad0 2023-07-18T03:17:56.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.060+0000 7f6d97fff700 1 --2- 172.21.15.101:0/3864856764 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aaaf1012d0 0x55aaaf111220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:17:56.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.060+0000 7f6d9cf9c700 1 --2- 172.21.15.101:0/3864856764 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aaaf10cad0 0x55aaaf16f2c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f6d900062a0 tx=0x7f6d90006230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83114e0f3df48f52 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:17:56.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.061+0000 7f6d777fe700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:17:56.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.063+0000 7f6d777fe700 1 -- 172.21.15.101:0/3864856764 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9000e040 con 0x55aaaf10cad0 2023-07-18T03:17:56.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.063+0000 7f6db22cfb80 1 -- 172.21.15.101:0/3864856764 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d8c099980 con 0x55aaaf10cad0 2023-07-18T03:17:56.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.063+0000 7f6d777fe700 1 -- 172.21.15.101:0/3864856764 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f6d90005450 con 0x55aaaf10cad0 2023-07-18T03:17:56.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.064+0000 7f6d777fe700 1 -- 172.21.15.101:0/3864856764 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d90007990 con 0x55aaaf10cad0 2023-07-18T03:17:56.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.064+0000 7f6db22cfb80 10 client.17682 fetch_fsmap learned FSMap version 33 2023-07-18T03:17:56.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.064+0000 7f6db22cfb80 1 -- 172.21.15.101:0/3864856764 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55aaaee802b0 con 0x55aaaf10cad0 2023-07-18T03:17:56.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.064+0000 7f6d777fe700 1 -- 172.21.15.101:0/3864856764 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d8c099980 con 0x55aaaf10cad0 2023-07-18T03:17:56.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.064+0000 7f6d777fe700 1 -- 172.21.15.101:0/3864856764 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6d9001ea50 con 0x55aaaf10cad0 2023-07-18T03:17:56.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.064+0000 7f6db22cfb80 10 client.17682 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:17:56.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.064+0000 7f6db22cfb80 10 client.17682 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:17:56.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.064+0000 7f6db22cfb80 20 client.17682 populate_metadata read hostname 'smithi101' 2023-07-18T03:17:56.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.064+0000 7f6db22cfb80 1 --2- 172.21.15.101:0/3864856764 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55aaaf171120 0x55aaaf173530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:17:56.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.064+0000 7f6db22cfb80 4 client.17682 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:17:56.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.064+0000 7f6db22cfb80 1 -- 172.21.15.101:0/3864856764 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55aaaf00f050 con 0x55aaaf171120 2023-07-18T03:17:56.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.064+0000 7f6d97fff700 1 --2- 172.21.15.101:0/3864856764 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55aaaf171120 0x55aaaf173530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:17:56.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.065+0000 7f6d97fff700 1 --2- 172.21.15.101:0/3864856764 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55aaaf171120 0x55aaaf173530 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0d1668a0403a893 server_cookie=91d645404d626748 in_seq=0 out_seq=0 2023-07-18T03:17:56.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.065+0000 7f6d777fe700 10 client.17682 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:17:56.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.065+0000 7f6d777fe700 1 -- 172.21.15.101:0/3864856764 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55aaaf00f050 con 0x55aaaf171120 2023-07-18T03:17:56.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.065+0000 7f6d777fe700 10 client.17682 handle_command_reply: tid=0 2023-07-18T03:17:56.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.065+0000 7f6db22cfb80 1 client.17682 shutdown 2023-07-18T03:17:56.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.065+0000 7f6db22cfb80 1 -- 172.21.15.101:0/3864856764 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aaaf10cad0 msgr2=0x55aaaf16f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:56.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.065+0000 7f6db22cfb80 1 --2- 172.21.15.101:0/3864856764 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aaaf10cad0 0x55aaaf16f2c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f6d900062a0 tx=0x7f6d90006230 comp rx=0 tx=0).stop 2023-07-18T03:17:56.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.066+0000 7f6db22cfb80 1 -- 172.21.15.101:0/3864856764 shutdown_connections 2023-07-18T03:17:56.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.066+0000 7f6db22cfb80 1 --2- 172.21.15.101:0/3864856764 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55aaaf171120 0x55aaaf173530 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:56.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.066+0000 7f6db22cfb80 1 --2- 172.21.15.101:0/3864856764 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aaaf1012d0 0x55aaaf111220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:56.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.066+0000 7f6d777fe700 0 client.17682 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:17:56.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.066+0000 7f6db22cfb80 1 --2- 172.21.15.101:0/3864856764 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aaaf10cad0 0x55aaaf16f2c0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:56.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.066+0000 7f6db22cfb80 1 --2- 172.21.15.101:0/3864856764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aaaf101be0 0x55aaaf111760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:56.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.066+0000 7f6db22cfb80 1 -- 172.21.15.101:0/3864856764 >> 172.21.15.101:0/3864856764 conn(0x55aaaf0f7da0 msgr2=0x55aaaf1033e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:56.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.066+0000 7f6db22cfb80 1 -- 172.21.15.101:0/3864856764 shutdown_connections 2023-07-18T03:17:56.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.066+0000 7f6db22cfb80 1 -- 172.21.15.101:0/3864856764 wait complete. 2023-07-18T03:17:56.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.066+0000 7f6db22cfb80 20 client.17682 trim_cache size 0 max 16384 2023-07-18T03:17:56.068 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:17:56.068 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:17:56.069 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:17:56.069 INFO:teuthology.orchestra.run.smithi101.stdout: "2f35c27f-b228-4708-a2e6-b03ffc6edf56": { 2023-07-18T03:17:56.069 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:17:56.069 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "2f35c27f-b228-4708-a2e6-b03ffc6edf56", 2023-07-18T03:17:56.069 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:17:56.070 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:17:56.070 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:17:56.070 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:17:56.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.067+0000 7f6d9e9ff700 1 -- 172.21.15.101:0/919950658 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6d800a1340 msgr2=0x7f6d800a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:56.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.067+0000 7f6d9e9ff700 1 --2- 172.21.15.101:0/919950658 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6d800a1340 0x7f6d800a37f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6d9000b780 tx=0x7f6d900099e0 comp rx=0 tx=0).stop 2023-07-18T03:17:56.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.067+0000 7f6d9e9ff700 1 -- 172.21.15.101:0/919950658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d98102910 msgr2=0x7f6d98198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:17:56.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.067+0000 7f6d9e9ff700 1 --2- 172.21.15.101:0/919950658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d98102910 0x7f6d98198470 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c00e940 tx=0x7f6d8c00ec50 comp rx=0 tx=0).stop 2023-07-18T03:17:56.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.067+0000 7f6d9e9ff700 1 -- 172.21.15.101:0/919950658 shutdown_connections 2023-07-18T03:17:56.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.067+0000 7f6d9e9ff700 1 --2- 172.21.15.101:0/919950658 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d98106fb0 0x7f6d9819c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:56.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.067+0000 7f6d9e9ff700 1 --2- 172.21.15.101:0/919950658 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6d98101f70 0x7f6d98197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:56.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.067+0000 7f6d9e9ff700 1 --2- 172.21.15.101:0/919950658 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6d800a1340 0x7f6d800a37f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:56.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.067+0000 7f6d9e9ff700 1 --2- 172.21.15.101:0/919950658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d98102910 0x7f6d98198470 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:17:56.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.067+0000 7f6d9e9ff700 1 -- 172.21.15.101:0/919950658 >> 172.21.15.101:0/919950658 conn(0x7f6d980fd7e0 msgr2=0x7f6d980fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:17:56.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.067+0000 7f6d9e9ff700 1 -- 172.21.15.101:0/919950658 shutdown_connections 2023-07-18T03:17:56.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:17:56.068+0000 7f6d9e9ff700 1 -- 172.21.15.101:0/919950658 wait complete. 2023-07-18T03:17:56.086 INFO:tasks.cephfs.filesystem:scrub status for tag:2f35c27f-b228-4708-a2e6-b03ffc6edf56 - {'path': '/', 'tag': '2f35c27f-b228-4708-a2e6-b03ffc6edf56', 'options': 'recursive,force'} 2023-07-18T03:17:56.239 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:56 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:17:56.239 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:56 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:17:56.240 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:56 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:17:56.240 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:56 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:17:56.240 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:56 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:17:56.240 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:56 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:17:56.240 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:56 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:17:56.240 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:56 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:17:56.241 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:56 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:17:56.241 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:56 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:17:56.241 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:56 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:17:56.241 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:56 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:17:56.241 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:56 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:17:56.241 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:56 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:17:56.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:56 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:17:56.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:56 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:17:56.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:56 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:17:56.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:56 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:17:56.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:56 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:17:56.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:56 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:17:56.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:56 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:17:56.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:56 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:17:56.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:56 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:17:56.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:56 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:17:56.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:56 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:17:56.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:56 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:17:56.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:56 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:17:56.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:56 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:17:56.546 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:56 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:17:56.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:56 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:17:56.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:56 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:17:56.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:56 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:17:56.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:56 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:17:56.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:56 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:17:56.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:56 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:17:56.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:56 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:17:56.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:56 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:17:56.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:56 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:17:56.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:56 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:17:56.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:56 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:17:56.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:56 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:17:56.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:56 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:17:56.617 INFO:tasks.workunit.client.0.smithi101.stdout: 1 650928 5.78 MB/sec execute 441 sec latency 20.454 ms 2023-07-18T03:17:57.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:57 smithi179 ceph-mon[129229]: Adjusting osd_memory_target on smithi155 to 196.2M 2023-07-18T03:17:57.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:57 smithi179 ceph-mon[129229]: Unable to set osd_memory_target on smithi155 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-07-18T03:17:57.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:57 smithi179 ceph-mon[129229]: Adjusting osd_memory_target on smithi179 to 196.2M 2023-07-18T03:17:57.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:57 smithi179 ceph-mon[129229]: Unable to set osd_memory_target on smithi179 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-07-18T03:17:57.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:57 smithi179 ceph-mon[129229]: pgmap v322: 129 pgs: 129 active+clean; 514 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 120 op/s 2023-07-18T03:17:57.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:57 smithi179 ceph-mon[129229]: scrub queued for path: / 2023-07-18T03:17:57.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:57 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:17:57.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:57 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:17:57.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:57 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:17:57.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:57 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:17:57.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:57 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:17:57.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:57 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:17:57.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:57 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:17:57.454 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:57 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:17:57.454 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:57 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:17:57.454 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:57 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:17:57.454 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:57 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:17:57.454 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:57 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:17:57.454 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:57 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:17:57.454 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:57 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:17:57.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:57 smithi101 ceph-mon[126379]: Adjusting osd_memory_target on smithi155 to 196.2M 2023-07-18T03:17:57.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:57 smithi101 ceph-mon[126379]: Unable to set osd_memory_target on smithi155 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-07-18T03:17:57.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:57 smithi101 ceph-mon[126379]: Adjusting osd_memory_target on smithi179 to 196.2M 2023-07-18T03:17:57.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:57 smithi101 ceph-mon[126379]: Unable to set osd_memory_target on smithi179 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-07-18T03:17:57.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:57 smithi101 ceph-mon[126379]: pgmap v322: 129 pgs: 129 active+clean; 514 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 120 op/s 2023-07-18T03:17:57.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:57 smithi101 ceph-mon[126379]: scrub queued for path: / 2023-07-18T03:17:57.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:57 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:17:57.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:57 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:17:57.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:57 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:17:57.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:57 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:17:57.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:57 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:17:57.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:57 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:17:57.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:57 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:17:57.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:57 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:17:57.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:57 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:17:57.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:57 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:17:57.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:57 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:17:57.492 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:57 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:17:57.492 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:57 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:17:57.492 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:57 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:17:57.546 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:57 smithi155 ceph-mon[128885]: Adjusting osd_memory_target on smithi155 to 196.2M 2023-07-18T03:17:57.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:57 smithi155 ceph-mon[128885]: Unable to set osd_memory_target on smithi155 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-07-18T03:17:57.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:57 smithi155 ceph-mon[128885]: Adjusting osd_memory_target on smithi179 to 196.2M 2023-07-18T03:17:57.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:57 smithi155 ceph-mon[128885]: Unable to set osd_memory_target on smithi179 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-07-18T03:17:57.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:57 smithi155 ceph-mon[128885]: pgmap v322: 129 pgs: 129 active+clean; 514 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 120 op/s 2023-07-18T03:17:57.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:57 smithi155 ceph-mon[128885]: scrub queued for path: / 2023-07-18T03:17:57.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:57 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:17:57.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:57 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:17:57.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:57 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:17:57.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:57 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:17:57.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:57 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:17:57.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:57 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:17:57.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:57 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:17:57.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:57 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:17:57.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:57 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:17:57.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:57 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:17:57.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:57 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:17:57.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:57 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:17:57.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:57 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:17:57.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:57 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:17:57.617 INFO:tasks.workunit.client.0.smithi101.stdout: 1 651583 5.77 MB/sec execute 442 sec latency 358.269 ms 2023-07-18T03:17:58.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:58 smithi179 ceph-mon[129229]: Adjusting osd_memory_target on smithi101 to 220.9M 2023-07-18T03:17:58.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:58 smithi179 ceph-mon[129229]: Unable to set osd_memory_target on smithi101 to 231653990: error parsing value: Value '231653990' is below minimum 939524096 2023-07-18T03:17:58.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:58 smithi101 ceph-mon[126379]: Adjusting osd_memory_target on smithi101 to 220.9M 2023-07-18T03:17:58.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:58 smithi101 ceph-mon[126379]: Unable to set osd_memory_target on smithi101 to 231653990: error parsing value: Value '231653990' is below minimum 939524096 2023-07-18T03:17:58.546 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:58 smithi155 ceph-mon[128885]: Adjusting osd_memory_target on smithi101 to 220.9M 2023-07-18T03:17:58.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:58 smithi155 ceph-mon[128885]: Unable to set osd_memory_target on smithi101 to 231653990: error parsing value: Value '231653990' is below minimum 939524096 2023-07-18T03:17:58.618 INFO:tasks.workunit.client.0.smithi101.stdout: 1 651583 5.76 MB/sec execute 443 sec latency 1358.412 ms 2023-07-18T03:17:59.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:17:59 smithi179 ceph-mon[129229]: pgmap v323: 129 pgs: 129 active+clean; 514 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 101 op/s 2023-07-18T03:17:59.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:17:59 smithi101 ceph-mon[126379]: pgmap v323: 129 pgs: 129 active+clean; 514 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 101 op/s 2023-07-18T03:17:59.546 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:17:59 smithi155 ceph-mon[128885]: pgmap v323: 129 pgs: 129 active+clean; 514 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 101 op/s 2023-07-18T03:17:59.618 INFO:tasks.workunit.client.0.smithi101.stdout: 1 652032 5.75 MB/sec execute 444 sec latency 2181.938 ms 2023-07-18T03:18:00.618 INFO:tasks.workunit.client.0.smithi101.stdout: 1 652790 5.75 MB/sec execute 445 sec latency 589.869 ms 2023-07-18T03:18:01.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:01 smithi179 ceph-mon[129229]: pgmap v324: 129 pgs: 129 active+clean; 518 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 155 op/s 2023-07-18T03:18:01.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:01 smithi101 ceph-mon[126379]: pgmap v324: 129 pgs: 129 active+clean; 518 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 155 op/s 2023-07-18T03:18:01.546 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:01 smithi155 ceph-mon[128885]: pgmap v324: 129 pgs: 129 active+clean; 518 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 155 op/s 2023-07-18T03:18:01.618 INFO:tasks.workunit.client.0.smithi101.stdout: 1 653149 5.74 MB/sec execute 446 sec latency 739.271 ms 2023-07-18T03:18:02.618 INFO:tasks.workunit.client.0.smithi101.stdout: 1 654007 5.74 MB/sec execute 447 sec latency 58.968 ms 2023-07-18T03:18:03.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:03 smithi179 ceph-mon[129229]: pgmap v325: 129 pgs: 129 active+clean; 518 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 103 op/s 2023-07-18T03:18:03.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:03 smithi101 ceph-mon[126379]: pgmap v325: 129 pgs: 129 active+clean; 518 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 103 op/s 2023-07-18T03:18:03.546 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:03 smithi155 ceph-mon[128885]: pgmap v325: 129 pgs: 129 active+clean; 518 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 103 op/s 2023-07-18T03:18:03.618 INFO:tasks.workunit.client.0.smithi101.stdout: 1 655132 5.74 MB/sec execute 448 sec latency 89.540 ms 2023-07-18T03:18:04.618 INFO:tasks.workunit.client.0.smithi101.stdout: 1 655624 5.73 MB/sec execute 449 sec latency 899.140 ms 2023-07-18T03:18:05.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:05 smithi179 ceph-mon[129229]: pgmap v326: 129 pgs: 129 active+clean; 518 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 103 op/s 2023-07-18T03:18:05.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:05 smithi101 ceph-mon[126379]: pgmap v326: 129 pgs: 129 active+clean; 518 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 103 op/s 2023-07-18T03:18:05.546 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:05 smithi155 ceph-mon[128885]: pgmap v326: 129 pgs: 129 active+clean; 518 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 103 op/s 2023-07-18T03:18:05.618 INFO:tasks.workunit.client.0.smithi101.stdout: 1 657215 5.74 MB/sec execute 450 sec latency 78.315 ms 2023-07-18T03:18:06.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:06 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:18:06.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:06 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:18:06.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:06 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:18:06.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:06 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:18:06.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:06 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:18:06.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:06 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:18:06.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:06 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:18:06.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:06 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:18:06.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:06 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:18:06.618 INFO:tasks.workunit.client.0.smithi101.stdout: 1 658041 5.74 MB/sec execute 451 sec latency 46.117 ms 2023-07-18T03:18:07.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:07 smithi179 ceph-mon[129229]: pgmap v327: 129 pgs: 129 active+clean; 520 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 193 op/s 2023-07-18T03:18:07.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:07 smithi179 ceph-mon[129229]: osdmap e101: 12 total, 12 up, 12 in 2023-07-18T03:18:07.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:07 smithi101 ceph-mon[126379]: pgmap v327: 129 pgs: 129 active+clean; 520 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 193 op/s 2023-07-18T03:18:07.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:07 smithi101 ceph-mon[126379]: osdmap e101: 12 total, 12 up, 12 in 2023-07-18T03:18:07.546 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:07 smithi155 ceph-mon[128885]: pgmap v327: 129 pgs: 129 active+clean; 520 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 193 op/s 2023-07-18T03:18:07.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:07 smithi155 ceph-mon[128885]: osdmap e101: 12 total, 12 up, 12 in 2023-07-18T03:18:07.618 INFO:tasks.workunit.client.0.smithi101.stdout: 1 658675 5.73 MB/sec execute 452 sec latency 402.091 ms 2023-07-18T03:18:08.619 INFO:tasks.workunit.client.0.smithi101.stdout: 1 658675 5.72 MB/sec execute 453 sec latency 1402.135 ms 2023-07-18T03:18:09.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:09 smithi179 ceph-mon[129229]: pgmap v329: 129 pgs: 129 active+clean; 520 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 171 op/s 2023-07-18T03:18:09.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:09 smithi101 ceph-mon[126379]: pgmap v329: 129 pgs: 129 active+clean; 520 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 171 op/s 2023-07-18T03:18:09.546 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:09 smithi155 ceph-mon[128885]: pgmap v329: 129 pgs: 129 active+clean; 520 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 171 op/s 2023-07-18T03:18:09.619 INFO:tasks.workunit.client.0.smithi101.stdout: 1 659075 5.71 MB/sec execute 454 sec latency 2227.882 ms 2023-07-18T03:18:10.619 INFO:tasks.workunit.client.0.smithi101.stdout: 1 660763 5.72 MB/sec execute 455 sec latency 18.346 ms 2023-07-18T03:18:11.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:11 smithi179 ceph-mon[129229]: pgmap v330: 129 pgs: 129 active+clean; 529 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 169 op/s 2023-07-18T03:18:11.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:11 smithi101 ceph-mon[126379]: pgmap v330: 129 pgs: 129 active+clean; 529 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 169 op/s 2023-07-18T03:18:11.546 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:11 smithi155 ceph-mon[128885]: pgmap v330: 129 pgs: 129 active+clean; 529 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 169 op/s 2023-07-18T03:18:11.619 INFO:tasks.workunit.client.0.smithi101.stdout: 1 661767 5.72 MB/sec execute 456 sec latency 27.790 ms 2023-07-18T03:18:12.619 INFO:tasks.workunit.client.0.smithi101.stdout: 1 662222 5.71 MB/sec execute 457 sec latency 635.358 ms 2023-07-18T03:18:13.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:13 smithi179 ceph-mon[129229]: pgmap v331: 129 pgs: 129 active+clean; 529 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 169 op/s 2023-07-18T03:18:13.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:13 smithi101 ceph-mon[126379]: pgmap v331: 129 pgs: 129 active+clean; 529 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 169 op/s 2023-07-18T03:18:13.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:13 smithi155 ceph-mon[128885]: pgmap v331: 129 pgs: 129 active+clean; 529 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 169 op/s 2023-07-18T03:18:13.619 INFO:tasks.workunit.client.0.smithi101.stdout: 1 662222 5.70 MB/sec execute 458 sec latency 1635.463 ms 2023-07-18T03:18:14.619 INFO:tasks.workunit.client.0.smithi101.stdout: 1 662671 5.69 MB/sec execute 459 sec latency 2460.688 ms 2023-07-18T03:18:15.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:14 smithi155 ceph-mon[128885]: pgmap v332: 129 pgs: 129 active+clean; 529 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 169 op/s 2023-07-18T03:18:15.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:14 smithi155 ceph-mon[128885]: osdmap e102: 12 total, 12 up, 12 in 2023-07-18T03:18:15.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:14 smithi179 ceph-mon[129229]: pgmap v332: 129 pgs: 129 active+clean; 529 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 169 op/s 2023-07-18T03:18:15.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:14 smithi179 ceph-mon[129229]: osdmap e102: 12 total, 12 up, 12 in 2023-07-18T03:18:15.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:14 smithi101 ceph-mon[126379]: pgmap v332: 129 pgs: 129 active+clean; 529 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 169 op/s 2023-07-18T03:18:15.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:14 smithi101 ceph-mon[126379]: osdmap e102: 12 total, 12 up, 12 in 2023-07-18T03:18:15.619 INFO:tasks.workunit.client.0.smithi101.stdout: 1 664314 5.70 MB/sec execute 460 sec latency 28.485 ms 2023-07-18T03:18:16.619 INFO:tasks.workunit.client.0.smithi101.stdout: 1 665253 5.69 MB/sec execute 461 sec latency 12.991 ms 2023-07-18T03:18:17.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:16 smithi155 ceph-mon[128885]: pgmap v334: 129 pgs: 129 active+clean; 531 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 127 op/s 2023-07-18T03:18:17.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:16 smithi179 ceph-mon[129229]: pgmap v334: 129 pgs: 129 active+clean; 531 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 127 op/s 2023-07-18T03:18:17.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:16 smithi101 ceph-mon[126379]: pgmap v334: 129 pgs: 129 active+clean; 531 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 127 op/s 2023-07-18T03:18:17.619 INFO:tasks.workunit.client.0.smithi101.stdout: 1 665765 5.69 MB/sec execute 462 sec latency 642.314 ms 2023-07-18T03:18:18.619 INFO:tasks.workunit.client.0.smithi101.stdout: 1 665765 5.68 MB/sec execute 463 sec latency 1642.387 ms 2023-07-18T03:18:19.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:19 smithi155 ceph-mon[128885]: pgmap v335: 129 pgs: 129 active+clean; 531 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 122 op/s 2023-07-18T03:18:19.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:19 smithi179 ceph-mon[129229]: pgmap v335: 129 pgs: 129 active+clean; 531 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 122 op/s 2023-07-18T03:18:19.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:19 smithi101 ceph-mon[126379]: pgmap v335: 129 pgs: 129 active+clean; 531 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 122 op/s 2023-07-18T03:18:19.619 INFO:tasks.workunit.client.0.smithi101.stdout: 1 666052 5.67 MB/sec execute 464 sec latency 2460.474 ms 2023-07-18T03:18:20.619 INFO:tasks.workunit.client.0.smithi101.stdout: 1 667835 5.68 MB/sec execute 465 sec latency 31.174 ms 2023-07-18T03:18:21.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:21 smithi155 ceph-mon[128885]: pgmap v336: 129 pgs: 129 active+clean; 539 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 126 op/s 2023-07-18T03:18:21.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:21 smithi179 ceph-mon[129229]: pgmap v336: 129 pgs: 129 active+clean; 539 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 126 op/s 2023-07-18T03:18:21.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:21 smithi101 ceph-mon[126379]: pgmap v336: 129 pgs: 129 active+clean; 539 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 126 op/s 2023-07-18T03:18:21.620 INFO:tasks.workunit.client.0.smithi101.stdout: 1 668778 5.67 MB/sec execute 466 sec latency 26.909 ms 2023-07-18T03:18:22.620 INFO:tasks.workunit.client.0.smithi101.stdout: 1 669314 5.67 MB/sec execute 467 sec latency 607.695 ms 2023-07-18T03:18:23.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:23 smithi155 ceph-mon[128885]: pgmap v337: 129 pgs: 129 active+clean; 539 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 126 op/s 2023-07-18T03:18:23.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:23 smithi179 ceph-mon[129229]: pgmap v337: 129 pgs: 129 active+clean; 539 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 126 op/s 2023-07-18T03:18:23.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:23 smithi101 ceph-mon[126379]: pgmap v337: 129 pgs: 129 active+clean; 539 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 126 op/s 2023-07-18T03:18:23.620 INFO:tasks.workunit.client.0.smithi101.stdout: 1 669314 5.66 MB/sec execute 468 sec latency 1607.814 ms 2023-07-18T03:18:24.620 INFO:tasks.workunit.client.0.smithi101.stdout: 1 669764 5.65 MB/sec execute 469 sec latency 2422.765 ms 2023-07-18T03:18:25.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:25 smithi155 ceph-mon[128885]: pgmap v338: 129 pgs: 129 active+clean; 539 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 126 op/s 2023-07-18T03:18:25.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:25 smithi179 ceph-mon[129229]: pgmap v338: 129 pgs: 129 active+clean; 539 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 126 op/s 2023-07-18T03:18:25.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:25 smithi101 ceph-mon[126379]: pgmap v338: 129 pgs: 129 active+clean; 539 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 126 op/s 2023-07-18T03:18:25.620 INFO:tasks.workunit.client.0.smithi101.stdout: 1 671408 5.66 MB/sec execute 470 sec latency 17.062 ms 2023-07-18T03:18:26.087 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:18:26.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.315+0000 7f384e99d700 1 -- 172.21.15.101:0/2121143668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38480ff9f0 msgr2=0x7f38480ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:26.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.315+0000 7f384e99d700 1 --2- 172.21.15.101:0/2121143668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38480ff9f0 0x7f38480ffe60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3838009990 tx=0x7f3838009ca0 comp rx=0 tx=0).stop 2023-07-18T03:18:26.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.316+0000 7f384e99d700 1 -- 172.21.15.101:0/2121143668 shutdown_connections 2023-07-18T03:18:26.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.316+0000 7f384e99d700 1 --2- 172.21.15.101:0/2121143668 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38481003a0 0x7f384810c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:26.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.316+0000 7f384e99d700 1 --2- 172.21.15.101:0/2121143668 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3848105a00 0x7f3848105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:26.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.316+0000 7f384e99d700 1 --2- 172.21.15.101:0/2121143668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38480ff9f0 0x7f38480ffe60 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:26.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.316+0000 7f384e99d700 1 -- 172.21.15.101:0/2121143668 >> 172.21.15.101:0/2121143668 conn(0x7f38480fb510 msgr2=0x7f38480fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:18:26.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.316+0000 7f384e99d700 1 -- 172.21.15.101:0/2121143668 shutdown_connections 2023-07-18T03:18:26.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.316+0000 7f384e99d700 1 -- 172.21.15.101:0/2121143668 wait complete. 2023-07-18T03:18:26.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.317+0000 7f384e99d700 1 Processor -- start 2023-07-18T03:18:26.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.317+0000 7f384e99d700 1 -- start start 2023-07-18T03:18:26.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.317+0000 7f384e99d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38480ff9f0 0x7f3848072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:26.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.317+0000 7f384e99d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f38481003a0 0x7f3848072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:26.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.317+0000 7f384e99d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3848105a00 0x7f38481a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:26.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.317+0000 7f384e99d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3848073580 con 0x7f38481003a0 2023-07-18T03:18:26.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.317+0000 7f384e99d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38481a4090 con 0x7f38480ff9f0 2023-07-18T03:18:26.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.317+0000 7f384e99d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38481a41d0 con 0x7f3848105a00 2023-07-18T03:18:26.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.317+0000 7f384d99b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38480ff9f0 0x7f3848072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:26.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.317+0000 7f384e19c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3848105a00 0x7f38481a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:26.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.317+0000 7f384d19a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f38481003a0 0x7f3848072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:26.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.317+0000 7f384d99b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38480ff9f0 0x7f3848072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:38884/0 (socket says 172.21.15.101:38884) 2023-07-18T03:18:26.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.317+0000 7f384d99b700 1 -- 172.21.15.101:0/772600875 learned_addr learned my addr 172.21.15.101:0/772600875 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:18:26.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.318+0000 7f384e19c700 1 -- 172.21.15.101:0/772600875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38480ff9f0 msgr2=0x7f3848072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:26.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.318+0000 7f384e19c700 1 --2- 172.21.15.101:0/772600875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38480ff9f0 0x7f3848072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:26.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.318+0000 7f384e19c700 1 -- 172.21.15.101:0/772600875 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f38481003a0 msgr2=0x7f3848072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:26.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.318+0000 7f384e19c700 1 --2- 172.21.15.101:0/772600875 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f38481003a0 0x7f3848072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:26.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.318+0000 7f384e19c700 1 -- 172.21.15.101:0/772600875 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38380095f0 con 0x7f3848105a00 2023-07-18T03:18:26.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.318+0000 7f384e19c700 1 --2- 172.21.15.101:0/772600875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3848105a00 0x7f38481a3b50 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f384000efc0 tx=0x7f384000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:18:26.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.318+0000 7f383effd700 1 -- 172.21.15.101:0/772600875 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3840018070 con 0x7f3848105a00 2023-07-18T03:18:26.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.319+0000 7f384e99d700 1 -- 172.21.15.101:0/772600875 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38481a43d0 con 0x7f3848105a00 2023-07-18T03:18:26.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.319+0000 7f383effd700 1 -- 172.21.15.101:0/772600875 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f384000f040 con 0x7f3848105a00 2023-07-18T03:18:26.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.319+0000 7f383effd700 1 -- 172.21.15.101:0/772600875 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384000cd40 con 0x7f3848105a00 2023-07-18T03:18:26.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.319+0000 7f384e99d700 1 -- 172.21.15.101:0/772600875 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38481a48b0 con 0x7f3848105a00 2023-07-18T03:18:26.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.323+0000 7f383effd700 1 -- 172.21.15.101:0/772600875 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f3840003a00 con 0x7f3848105a00 2023-07-18T03:18:26.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.325+0000 7f3863594b80 1 Processor -- start 2023-07-18T03:18:26.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.325+0000 7f3863594b80 1 -- start start 2023-07-18T03:18:26.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.325+0000 7f3863594b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a534dd4f0 0x563a534dd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:26.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.325+0000 7f3863594b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a534dde00 0x563a534e77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:26.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.325+0000 7f3863594b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a534e7d30 0x563a534ea110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:26.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.325+0000 7f3863594b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a533e2cb0 con 0x563a534dde00 2023-07-18T03:18:26.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.325+0000 7f3863594b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a53435a40 con 0x563a534e7d30 2023-07-18T03:18:26.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.325+0000 7f3863594b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a5342eed0 con 0x563a534dd4f0 2023-07-18T03:18:26.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.325+0000 7f383effd700 1 --2- 172.21.15.101:0/772600875 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f38340a1360 0x7f38340a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:26.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.326+0000 7f384d19a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a534dd4f0 0x563a534dd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:26.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.326+0000 7f384d19a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a534dd4f0 0x563a534dd8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:35412/0 (socket says 172.21.15.101:35412) 2023-07-18T03:18:26.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.326+0000 7f384d19a700 1 -- 172.21.15.101:0/2849213731 learned_addr learned my addr 172.21.15.101:0/2849213731 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:18:26.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.326+0000 7f384d99b700 1 --2- 172.21.15.101:0/2849213731 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a534dde00 0x563a534e77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:26.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.326+0000 7f384d19a700 1 --2- 172.21.15.101:0/2849213731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a534e7d30 0x563a534ea110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:26.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.326+0000 7f383effd700 1 -- 172.21.15.101:0/772600875 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f38400cccf0 con 0x7f3848105a00 2023-07-18T03:18:26.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.326+0000 7f384e19c700 1 --2- 172.21.15.101:0/772600875 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f38340a1360 0x7f38340a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:26.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.326+0000 7f384d19a700 1 -- 172.21.15.101:0/2849213731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a534e7d30 msgr2=0x563a534ea110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:26.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.326+0000 7f384d19a700 1 --2- 172.21.15.101:0/2849213731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a534e7d30 0x563a534ea110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:26.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.326+0000 7f384d19a700 1 -- 172.21.15.101:0/2849213731 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a534dde00 msgr2=0x563a534e77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:26.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.326+0000 7f384d19a700 1 --2- 172.21.15.101:0/2849213731 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a534dde00 0x563a534e77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:26.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.326+0000 7f384d19a700 1 -- 172.21.15.101:0/2849213731 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a5325cde0 con 0x563a534dd4f0 2023-07-18T03:18:26.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.326+0000 7f384d99b700 1 --2- 172.21.15.101:0/2849213731 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a534dde00 0x563a534e77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:18:26.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.326+0000 7f384d19a700 1 --2- 172.21.15.101:0/2849213731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a534dd4f0 0x563a534dd8c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3838015000 tx=0x7f383801a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cec2a8c060a2adb8 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:18:26.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.327+0000 7f384e19c700 1 --2- 172.21.15.101:0/772600875 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f38340a1360 0x7f38340a3810 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f384000c6f0 tx=0x7f38400092e0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:18:26.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.327+0000 7f3833fff700 1 -- 172.21.15.101:0/2849213731 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383800bb60 con 0x563a534dd4f0 2023-07-18T03:18:26.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.327+0000 7f3833fff700 1 -- 172.21.15.101:0/2849213731 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f383800bcc0 con 0x563a534dd4f0 2023-07-18T03:18:26.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.327+0000 7f3833fff700 1 -- 172.21.15.101:0/2849213731 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3838004bd0 con 0x563a534dd4f0 2023-07-18T03:18:26.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.327+0000 7f3863594b80 1 -- 172.21.15.101:0/2849213731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a534dd4f0 msgr2=0x563a534dd8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:26.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.327+0000 7f3863594b80 1 --2- 172.21.15.101:0/2849213731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a534dd4f0 0x563a534dd8c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3838015000 tx=0x7f383801a040 comp rx=0 tx=0).stop 2023-07-18T03:18:26.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.328+0000 7f3863594b80 1 -- 172.21.15.101:0/2849213731 shutdown_connections 2023-07-18T03:18:26.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.328+0000 7f3863594b80 1 --2- 172.21.15.101:0/2849213731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a534e7d30 0x563a534ea110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:26.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.328+0000 7f3863594b80 1 --2- 172.21.15.101:0/2849213731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a534dd4f0 0x563a534dd8c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:26.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.328+0000 7f3863594b80 1 --2- 172.21.15.101:0/2849213731 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a534dde00 0x563a534e77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:26.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.328+0000 7f3863594b80 1 -- 172.21.15.101:0/2849213731 >> 172.21.15.101:0/2849213731 conn(0x563a534d2ff0 msgr2=0x563a534d8930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:18:26.327 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.328+0000 7f3863594b80 1 -- 172.21.15.101:0/2849213731 shutdown_connections 2023-07-18T03:18:26.327 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.328+0000 7f3863594b80 1 -- 172.21.15.101:0/2849213731 wait complete. 2023-07-18T03:18:26.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.328+0000 7f3863594b80 1 -- start start 2023-07-18T03:18:26.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.328+0000 7f3863594b80 1 Processor -- start 2023-07-18T03:18:26.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.329+0000 7f3863594b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a534dd4f0 0x563a534e77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:26.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.329+0000 7f3863594b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a534dde00 0x563a534e49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:26.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.329+0000 7f3863594b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a534e7d30 0x563a534e5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:26.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.329+0000 7f3863594b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a533ceac0 con 0x563a534dde00 2023-07-18T03:18:26.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.329+0000 7f3863594b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a533ce7c0 con 0x563a534e7d30 2023-07-18T03:18:26.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.329+0000 7f3863594b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a532b2920 con 0x563a534dd4f0 2023-07-18T03:18:26.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.329+0000 7f3863594b80 10 client.0 _ll_register_callbacks cb 0x563a533cc760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:18:26.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.329+0000 7f384d19a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a534dd4f0 0x563a534e77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:26.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.329+0000 7f384d19a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a534dd4f0 0x563a534e77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:35418/0 (socket says 172.21.15.101:35418) 2023-07-18T03:18:26.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.329+0000 7f384d19a700 1 -- 172.21.15.101:0/1891262861 learned_addr learned my addr 172.21.15.101:0/1891262861 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:18:26.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.329+0000 7f384d19a700 1 --2- 172.21.15.101:0/1891262861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a534e7d30 0x563a534e5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:26.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.329+0000 7f384d99b700 1 --2- 172.21.15.101:0/1891262861 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a534dde00 0x563a534e49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:26.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.330+0000 7f384d19a700 1 -- 172.21.15.101:0/1891262861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a534e7d30 msgr2=0x563a534e5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:26.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.330+0000 7f384d19a700 1 --2- 172.21.15.101:0/1891262861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a534e7d30 0x563a534e5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:26.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.330+0000 7f384d19a700 1 -- 172.21.15.101:0/1891262861 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a534dde00 msgr2=0x563a534e49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:26.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.330+0000 7f384d19a700 1 --2- 172.21.15.101:0/1891262861 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a534dde00 0x563a534e49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:26.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.330+0000 7f384d19a700 1 -- 172.21.15.101:0/1891262861 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38440098f0 con 0x563a534dd4f0 2023-07-18T03:18:26.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.330+0000 7f384d99b700 1 --2- 172.21.15.101:0/1891262861 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a534dde00 0x563a534e49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:18:26.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.330+0000 7f384d19a700 1 --2- 172.21.15.101:0/1891262861 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a534dd4f0 0x563a534e77c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f383800b3e0 tx=0x7f3838004710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2da897507855f78 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:18:26.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.330+0000 7f38317fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:18:26.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.330+0000 7f38317fa700 1 -- 172.21.15.101:0/1891262861 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3838004500 con 0x563a534dd4f0 2023-07-18T03:18:26.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.330+0000 7f38317fa700 1 -- 172.21.15.101:0/1891262861 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f38380049b0 con 0x563a534dd4f0 2023-07-18T03:18:26.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.330+0000 7f3863594b80 1 -- 172.21.15.101:0/1891262861 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563a533884f0 con 0x563a534dd4f0 2023-07-18T03:18:26.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.330+0000 7f38317fa700 1 -- 172.21.15.101:0/1891262861 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3838004b70 con 0x563a534dd4f0 2023-07-18T03:18:26.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.330+0000 7f3863594b80 10 client.27763 fetch_fsmap learned FSMap version 33 2023-07-18T03:18:26.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.330+0000 7f3863594b80 1 -- 172.21.15.101:0/1891262861 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f3838020040 con 0x563a534dd4f0 2023-07-18T03:18:26.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.330+0000 7f38317fa700 1 -- 172.21.15.101:0/1891262861 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563a533884f0 con 0x563a534dd4f0 2023-07-18T03:18:26.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.331+0000 7f38317fa700 1 -- 172.21.15.101:0/1891262861 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f383800f950 con 0x563a534dd4f0 2023-07-18T03:18:26.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.331+0000 7f3863594b80 10 client.27763 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:18:26.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.331+0000 7f3863594b80 10 client.27763 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:18:26.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.331+0000 7f3863594b80 20 client.27763 populate_metadata read hostname 'smithi101' 2023-07-18T03:18:26.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.331+0000 7f3863594b80 1 --2- 172.21.15.101:0/1891262861 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563a534f71e0 0x563a534f95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:26.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.331+0000 7f3863594b80 4 client.27763 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:18:26.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.331+0000 7f3863594b80 1 -- 172.21.15.101:0/1891262861 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563a53427eb0 con 0x563a534f71e0 2023-07-18T03:18:26.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.331+0000 7f384d99b700 1 --2- 172.21.15.101:0/1891262861 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563a534f71e0 0x563a534f95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:26.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.331+0000 7f384d99b700 1 --2- 172.21.15.101:0/1891262861 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563a534f71e0 0x563a534f95c0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d9019b248ce2513 server_cookie=46f85fdb9cda5758 in_seq=0 out_seq=0 2023-07-18T03:18:26.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.331+0000 7f38317fa700 10 client.27763 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:18:26.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.333+0000 7f38317fa700 1 -- 172.21.15.101:0/1891262861 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563a53427eb0 con 0x563a534f71e0 2023-07-18T03:18:26.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.333+0000 7f38317fa700 10 client.27763 handle_command_reply: tid=0 2023-07-18T03:18:26.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.333+0000 7f3863594b80 1 client.27763 shutdown 2023-07-18T03:18:26.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.333+0000 7f3863594b80 1 -- 172.21.15.101:0/1891262861 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a534dd4f0 msgr2=0x563a534e77c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:26.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.333+0000 7f3863594b80 1 --2- 172.21.15.101:0/1891262861 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a534dd4f0 0x563a534e77c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f383800b3e0 tx=0x7f3838004710 comp rx=0 tx=0).stop 2023-07-18T03:18:26.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.333+0000 7f3863594b80 1 -- 172.21.15.101:0/1891262861 shutdown_connections 2023-07-18T03:18:26.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.333+0000 7f3863594b80 1 --2- 172.21.15.101:0/1891262861 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563a534f71e0 0x563a534f95c0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:26.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.333+0000 7f3863594b80 1 --2- 172.21.15.101:0/1891262861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a534e7d30 0x563a534e5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:26.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.333+0000 7f3863594b80 1 --2- 172.21.15.101:0/1891262861 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a534dd4f0 0x563a534e77c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:26.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.333+0000 7f3863594b80 1 --2- 172.21.15.101:0/1891262861 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a534dde00 0x563a534e49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:26.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.333+0000 7f3863594b80 1 -- 172.21.15.101:0/1891262861 >> 172.21.15.101:0/1891262861 conn(0x563a534d2ff0 msgr2=0x563a534d8520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:18:26.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.333+0000 7f38317fa700 0 client.27763 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:18:26.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.334+0000 7f3863594b80 1 -- 172.21.15.101:0/1891262861 shutdown_connections 2023-07-18T03:18:26.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.334+0000 7f3863594b80 1 -- 172.21.15.101:0/1891262861 wait complete. 2023-07-18T03:18:26.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.334+0000 7f3863594b80 20 client.27763 trim_cache size 0 max 16384 2023-07-18T03:18:26.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.355+0000 7f3863594b80 1 Processor -- start 2023-07-18T03:18:26.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.355+0000 7f3863594b80 1 -- start start 2023-07-18T03:18:26.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.355+0000 7f3863594b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a534dd4f0 0x563a534ee7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:26.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.355+0000 7f3863594b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a534dde00 0x563a534dfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:26.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.355+0000 7f3863594b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a534e7d30 0x563a534e72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:26.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.355+0000 7f3863594b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a534d0210 con 0x563a534dde00 2023-07-18T03:18:26.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.355+0000 7f3863594b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a534df590 con 0x563a534e7d30 2023-07-18T03:18:26.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.355+0000 7f3863594b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a534df1a0 con 0x563a534dd4f0 2023-07-18T03:18:26.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.356+0000 7f384d19a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a534dd4f0 0x563a534ee7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:26.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.356+0000 7f384d99b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a534dde00 0x563a534dfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:26.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.356+0000 7f384d19a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a534dd4f0 0x563a534ee7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:35424/0 (socket says 172.21.15.101:35424) 2023-07-18T03:18:26.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.356+0000 7f384d19a700 1 -- 172.21.15.101:0/4145061555 learned_addr learned my addr 172.21.15.101:0/4145061555 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:18:26.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.356+0000 7f384d19a700 1 --2- 172.21.15.101:0/4145061555 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a534e7d30 0x563a534e72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:26.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.356+0000 7f384d99b700 1 -- 172.21.15.101:0/4145061555 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a534dd4f0 msgr2=0x563a534ee7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:26.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.356+0000 7f384d99b700 1 --2- 172.21.15.101:0/4145061555 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a534dd4f0 0x563a534ee7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:26.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.356+0000 7f384d99b700 1 -- 172.21.15.101:0/4145061555 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a534e7d30 msgr2=0x563a534e72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:26.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.356+0000 7f384d99b700 1 --2- 172.21.15.101:0/4145061555 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a534e7d30 0x563a534e72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:26.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.356+0000 7f384d99b700 1 -- 172.21.15.101:0/4145061555 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a53245c60 con 0x563a534dde00 2023-07-18T03:18:26.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.356+0000 7f384d19a700 1 --2- 172.21.15.101:0/4145061555 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a534e7d30 0x563a534e72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:18:26.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.356+0000 7f384d99b700 1 --2- 172.21.15.101:0/4145061555 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a534dde00 0x563a534dfd30 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x563a534f9d10 tx=0x7f3844004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3219cbf9aed92b8 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:18:26.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.356+0000 7f383cbff700 1 -- 172.21.15.101:0/4145061555 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3844016070 con 0x563a534dde00 2023-07-18T03:18:26.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.356+0000 7f383cbff700 1 -- 172.21.15.101:0/4145061555 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3844017040 con 0x563a534dde00 2023-07-18T03:18:26.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.356+0000 7f383cbff700 1 -- 172.21.15.101:0/4145061555 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3844004bb0 con 0x563a534dde00 2023-07-18T03:18:26.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.357+0000 7f3863594b80 1 -- 172.21.15.101:0/4145061555 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a534dde00 msgr2=0x563a534dfd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:26.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.357+0000 7f3863594b80 1 --2- 172.21.15.101:0/4145061555 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a534dde00 0x563a534dfd30 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x563a534f9d10 tx=0x7f3844004750 comp rx=0 tx=0).stop 2023-07-18T03:18:26.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.357+0000 7f3863594b80 1 -- 172.21.15.101:0/4145061555 shutdown_connections 2023-07-18T03:18:26.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.357+0000 7f3863594b80 1 --2- 172.21.15.101:0/4145061555 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a534e7d30 0x563a534e72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:26.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.357+0000 7f3863594b80 1 --2- 172.21.15.101:0/4145061555 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a534dd4f0 0x563a534ee7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:26.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.357+0000 7f3863594b80 1 --2- 172.21.15.101:0/4145061555 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a534dde00 0x563a534dfd30 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:26.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.357+0000 7f3863594b80 1 -- 172.21.15.101:0/4145061555 >> 172.21.15.101:0/4145061555 conn(0x563a534d2ff0 msgr2=0x563a534f3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:18:26.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.357+0000 7f3863594b80 1 -- 172.21.15.101:0/4145061555 shutdown_connections 2023-07-18T03:18:26.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.357+0000 7f3863594b80 1 -- 172.21.15.101:0/4145061555 wait complete. 2023-07-18T03:18:26.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.358+0000 7f3863594b80 1 -- start start 2023-07-18T03:18:26.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.358+0000 7f3863594b80 1 Processor -- start 2023-07-18T03:18:26.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.358+0000 7f3863594b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a534dd4f0 0x563a534ee1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:26.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.358+0000 7f3863594b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a534dde00 0x563a534ee730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:26.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.358+0000 7f3863594b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a534e7d30 0x563a534ec790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:26.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.359+0000 7f3863594b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a534ecd70 con 0x563a534dde00 2023-07-18T03:18:26.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.359+0000 7f3863594b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a534ecee0 con 0x563a534e7d30 2023-07-18T03:18:26.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.359+0000 7f3863594b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a534ed0a0 con 0x563a534dd4f0 2023-07-18T03:18:26.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.359+0000 7f3863594b80 10 client.0 _ll_register_callbacks cb 0x7f3838005770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:18:26.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.359+0000 7f384d19a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a534dd4f0 0x563a534ee1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:26.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.359+0000 7f384d19a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a534dd4f0 0x563a534ee1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:35436/0 (socket says 172.21.15.101:35436) 2023-07-18T03:18:26.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.359+0000 7f384d19a700 1 -- 172.21.15.101:0/1587372387 learned_addr learned my addr 172.21.15.101:0/1587372387 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:18:26.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.359+0000 7f384d99b700 1 --2- 172.21.15.101:0/1587372387 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a534dde00 0x563a534ee730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:26.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.359+0000 7f384d19a700 1 --2- 172.21.15.101:0/1587372387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a534e7d30 0x563a534ec790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:26.361 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.359+0000 7f384d99b700 1 -- 172.21.15.101:0/1587372387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a534dd4f0 msgr2=0x563a534ee1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:26.361 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.359+0000 7f384d99b700 1 --2- 172.21.15.101:0/1587372387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a534dd4f0 0x563a534ee1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:26.361 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.359+0000 7f384d99b700 1 -- 172.21.15.101:0/1587372387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a534e7d30 msgr2=0x563a534ec790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:26.361 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.359+0000 7f384d99b700 1 --2- 172.21.15.101:0/1587372387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a534e7d30 0x563a534ec790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:26.361 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.359+0000 7f384d99b700 1 -- 172.21.15.101:0/1587372387 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3844005ce0 con 0x563a534dde00 2023-07-18T03:18:26.361 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.359+0000 7f384d99b700 1 --2- 172.21.15.101:0/1587372387 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a534dde00 0x563a534ee730 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x563a534f4eb0 tx=0x7f3844004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beed7b31b6b3037d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:18:26.361 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.359+0000 7f38317fa700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:18:26.361 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.360+0000 7f38317fa700 1 -- 172.21.15.101:0/1587372387 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3844016070 con 0x563a534dde00 2023-07-18T03:18:26.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.360+0000 7f3863594b80 1 -- 172.21.15.101:0/1587372387 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f383800bcc0 con 0x563a534dde00 2023-07-18T03:18:26.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.360+0000 7f38317fa700 1 -- 172.21.15.101:0/1587372387 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f3844010930 con 0x563a534dde00 2023-07-18T03:18:26.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.360+0000 7f38317fa700 1 -- 172.21.15.101:0/1587372387 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3844010af0 con 0x563a534dde00 2023-07-18T03:18:26.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.360+0000 7f3863594b80 10 client.17700 fetch_fsmap learned FSMap version 33 2023-07-18T03:18:26.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.360+0000 7f3863594b80 1 -- 172.21.15.101:0/1587372387 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x563a53283250 con 0x563a534dde00 2023-07-18T03:18:26.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.360+0000 7f38317fa700 1 -- 172.21.15.101:0/1587372387 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f383800bcc0 con 0x563a534dde00 2023-07-18T03:18:26.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.360+0000 7f38317fa700 1 -- 172.21.15.101:0/1587372387 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3844005280 con 0x563a534dde00 2023-07-18T03:18:26.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.360+0000 7f3863594b80 10 client.17700 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:18:26.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.360+0000 7f3863594b80 10 client.17700 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:18:26.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.360+0000 7f3863594b80 20 client.17700 populate_metadata read hostname 'smithi101' 2023-07-18T03:18:26.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.360+0000 7f3863594b80 1 --2- 172.21.15.101:0/1587372387 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563a5354c090 0x563a5354e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:26.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.360+0000 7f3863594b80 4 client.17700 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:18:26.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.360+0000 7f3863594b80 1 -- 172.21.15.101:0/1587372387 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563a533ea050 con 0x563a5354c090 2023-07-18T03:18:26.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.361+0000 7f384d19a700 1 --2- 172.21.15.101:0/1587372387 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563a5354c090 0x563a5354e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:26.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.361+0000 7f38317fa700 10 client.17700 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:18:26.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.361+0000 7f384d19a700 1 --2- 172.21.15.101:0/1587372387 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563a5354c090 0x563a5354e4a0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c2cfbf329f0dab server_cookie=95e8b392692a4529 in_seq=0 out_seq=0 2023-07-18T03:18:26.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.361+0000 7f38317fa700 1 -- 172.21.15.101:0/1587372387 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x563a533ea050 con 0x563a5354c090 2023-07-18T03:18:26.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.361+0000 7f38317fa700 10 client.17700 handle_command_reply: tid=0 2023-07-18T03:18:26.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.361+0000 7f3863594b80 1 client.17700 shutdown 2023-07-18T03:18:26.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.362+0000 7f3863594b80 1 -- 172.21.15.101:0/1587372387 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a534dde00 msgr2=0x563a534ee730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:26.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.362+0000 7f3863594b80 1 --2- 172.21.15.101:0/1587372387 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a534dde00 0x563a534ee730 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x563a534f4eb0 tx=0x7f3844004280 comp rx=0 tx=0).stop 2023-07-18T03:18:26.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.362+0000 7f3863594b80 1 -- 172.21.15.101:0/1587372387 shutdown_connections 2023-07-18T03:18:26.365 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.362+0000 7f3863594b80 1 --2- 172.21.15.101:0/1587372387 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563a5354c090 0x563a5354e4a0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:26.365 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.362+0000 7f3863594b80 1 --2- 172.21.15.101:0/1587372387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a534e7d30 0x563a534ec790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:26.365 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.362+0000 7f3863594b80 1 --2- 172.21.15.101:0/1587372387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a534dd4f0 0x563a534ee1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:26.365 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.362+0000 7f3863594b80 1 --2- 172.21.15.101:0/1587372387 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a534dde00 0x563a534ee730 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:26.365 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.362+0000 7f3863594b80 1 -- 172.21.15.101:0/1587372387 >> 172.21.15.101:0/1587372387 conn(0x563a534d2ff0 msgr2=0x563a534f91f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:18:26.365 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.362+0000 7f38317fa700 0 client.17700 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:18:26.365 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.362+0000 7f3863594b80 1 -- 172.21.15.101:0/1587372387 shutdown_connections 2023-07-18T03:18:26.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.362+0000 7f3863594b80 1 -- 172.21.15.101:0/1587372387 wait complete. 2023-07-18T03:18:26.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.362+0000 7f3863594b80 20 client.17700 trim_cache size 0 max 16384 2023-07-18T03:18:26.366 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:18:26.366 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "no active scrubs running", 2023-07-18T03:18:26.366 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": {} 2023-07-18T03:18:26.367 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:18:26.367 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.362+0000 7f384e99d700 1 -- 172.21.15.101:0/772600875 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f38340a1360 msgr2=0x7f38340a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:26.367 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.362+0000 7f384e99d700 1 --2- 172.21.15.101:0/772600875 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f38340a1360 0x7f38340a3810 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f384000c6f0 tx=0x7f38400092e0 comp rx=0 tx=0).stop 2023-07-18T03:18:26.368 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.362+0000 7f384e99d700 1 -- 172.21.15.101:0/772600875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3848105a00 msgr2=0x7f38481a3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:26.368 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.362+0000 7f384e99d700 1 --2- 172.21.15.101:0/772600875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3848105a00 0x7f38481a3b50 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f384000efc0 tx=0x7f384000c3a0 comp rx=0 tx=0).stop 2023-07-18T03:18:26.368 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.363+0000 7f384e99d700 1 -- 172.21.15.101:0/772600875 shutdown_connections 2023-07-18T03:18:26.368 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.363+0000 7f384e99d700 1 --2- 172.21.15.101:0/772600875 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f38340a1360 0x7f38340a3810 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:26.368 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.363+0000 7f384e99d700 1 --2- 172.21.15.101:0/772600875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38480ff9f0 0x7f3848072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:26.368 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.363+0000 7f384e99d700 1 --2- 172.21.15.101:0/772600875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3848105a00 0x7f38481a3b50 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:26.368 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.363+0000 7f384e99d700 1 --2- 172.21.15.101:0/772600875 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f38481003a0 0x7f3848072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:26.369 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.363+0000 7f384e99d700 1 -- 172.21.15.101:0/772600875 >> 172.21.15.101:0/772600875 conn(0x7f38480fb510 msgr2=0x7f38480fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:18:26.369 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.363+0000 7f384e99d700 1 -- 172.21.15.101:0/772600875 shutdown_connections 2023-07-18T03:18:26.369 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.363+0000 7f384e99d700 1 -- 172.21.15.101:0/772600875 wait complete. 2023-07-18T03:18:26.376 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-18T03:18:26.377 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:18:26.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.603+0000 7f0b585ff700 1 -- 172.21.15.101:0/3320778071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b50072850 msgr2=0x7f0b50072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:26.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.603+0000 7f0b585ff700 1 --2- 172.21.15.101:0/3320778071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b50072850 0x7f0b50072c20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c00b0a0 tx=0x7f0b4c00b3b0 comp rx=0 tx=0).stop 2023-07-18T03:18:26.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.603+0000 7f0b585ff700 1 -- 172.21.15.101:0/3320778071 shutdown_connections 2023-07-18T03:18:26.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.603+0000 7f0b585ff700 1 --2- 172.21.15.101:0/3320778071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b50072850 0x7f0b50072c20 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:26.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.603+0000 7f0b585ff700 1 --2- 172.21.15.101:0/3320778071 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0b5010c390 0x7f0b5010e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:26.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.603+0000 7f0b585ff700 1 --2- 172.21.15.101:0/3320778071 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b50073160 0x7f0b5010be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:26.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.603+0000 7f0b585ff700 1 -- 172.21.15.101:0/3320778071 >> 172.21.15.101:0/3320778071 conn(0x7f0b500fb630 msgr2=0x7f0b500fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:18:26.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.603+0000 7f0b585ff700 1 -- 172.21.15.101:0/3320778071 shutdown_connections 2023-07-18T03:18:26.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.603+0000 7f0b585ff700 1 -- 172.21.15.101:0/3320778071 wait complete. 2023-07-18T03:18:26.604 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.604+0000 7f0b585ff700 1 Processor -- start 2023-07-18T03:18:26.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.604+0000 7f0b585ff700 1 -- start start 2023-07-18T03:18:26.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.605+0000 7f0b585ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0b50072850 0x7f0b500ffab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:26.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.605+0000 7f0b585ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b50073160 0x7f0b500ffff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:26.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.605+0000 7f0b585ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b5010c390 0x7f0b50100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:26.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.605+0000 7f0b585ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b501a8670 con 0x7f0b50072850 2023-07-18T03:18:26.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.605+0000 7f0b585ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b501a87e0 con 0x7f0b50073160 2023-07-18T03:18:26.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.605+0000 7f0b585ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b501a89a0 con 0x7f0b5010c390 2023-07-18T03:18:26.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.606+0000 7f0b55b9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b50073160 0x7f0b500ffff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:26.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.606+0000 7f0b55b9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b50073160 0x7f0b500ffff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:38940/0 (socket says 172.21.15.101:38940) 2023-07-18T03:18:26.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.606+0000 7f0b55b9a700 1 -- 172.21.15.101:0/2071242366 learned_addr learned my addr 172.21.15.101:0/2071242366 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:18:26.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.606+0000 7f0b5639b700 1 --2- 172.21.15.101:0/2071242366 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0b50072850 0x7f0b500ffab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:26.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.606+0000 7f0b56b9c700 1 --2- 172.21.15.101:0/2071242366 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b5010c390 0x7f0b50100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:26.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.606+0000 7f0b55b9a700 1 -- 172.21.15.101:0/2071242366 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b5010c390 msgr2=0x7f0b50100530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:26.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.606+0000 7f0b55b9a700 1 --2- 172.21.15.101:0/2071242366 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b5010c390 0x7f0b50100530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:26.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.606+0000 7f0b55b9a700 1 -- 172.21.15.101:0/2071242366 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0b50072850 msgr2=0x7f0b500ffab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:26.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.606+0000 7f0b55b9a700 1 --2- 172.21.15.101:0/2071242366 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0b50072850 0x7f0b500ffab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:26.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.606+0000 7f0b55b9a700 1 -- 172.21.15.101:0/2071242366 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b4c009d00 con 0x7f0b50073160 2023-07-18T03:18:26.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.606+0000 7f0b55b9a700 1 --2- 172.21.15.101:0/2071242366 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b50073160 0x7f0b500ffff0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0b4000e9a0 tx=0x7f0b4000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:18:26.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.607+0000 7f0b477fe700 1 -- 172.21.15.101:0/2071242366 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b40009790 con 0x7f0b50073160 2023-07-18T03:18:26.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.607+0000 7f0b477fe700 1 -- 172.21.15.101:0/2071242366 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b40010430 con 0x7f0b50073160 2023-07-18T03:18:26.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.607+0000 7f0b585ff700 1 -- 172.21.15.101:0/2071242366 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b501a8c80 con 0x7f0b50073160 2023-07-18T03:18:26.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.607+0000 7f0b477fe700 1 -- 172.21.15.101:0/2071242366 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b40018680 con 0x7f0b50073160 2023-07-18T03:18:26.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.607+0000 7f0b585ff700 1 -- 172.21.15.101:0/2071242366 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b501a9190 con 0x7f0b50073160 2023-07-18T03:18:26.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.608+0000 7f0b585ff700 1 -- 172.21.15.101:0/2071242366 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b50067300 con 0x7f0b50073160 2023-07-18T03:18:26.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.610+0000 7f0b477fe700 1 -- 172.21.15.101:0/2071242366 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f0b400105a0 con 0x7f0b50073160 2023-07-18T03:18:26.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.611+0000 7f0b477fe700 1 --2- 172.21.15.101:0/2071242366 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0b3c0a1380 0x7f0b3c0a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:26.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.611+0000 7f0b477fe700 1 -- 172.21.15.101:0/2071242366 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f0b40014070 con 0x7f0b50073160 2023-07-18T03:18:26.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.611+0000 7f0b5639b700 1 --2- 172.21.15.101:0/2071242366 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0b3c0a1380 0x7f0b3c0a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:26.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.612+0000 7f0b5639b700 1 --2- 172.21.15.101:0/2071242366 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0b3c0a1380 0x7f0b3c0a3830 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c009fd0 tx=0x7f0b4c006210 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:18:26.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.613+0000 7f0b477fe700 1 -- 172.21.15.101:0/2071242366 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0b40096270 con 0x7f0b50073160 2023-07-18T03:18:26.620 INFO:tasks.workunit.client.0.smithi101.stdout: 1 672447 5.66 MB/sec execute 471 sec latency 24.894 ms 2023-07-18T03:18:26.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.887+0000 7f0b585ff700 1 -- 172.21.15.101:0/2071242366 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0b50100fe0 con 0x7f0b50073160 2023-07-18T03:18:26.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.889+0000 7f0b477fe700 1 -- 172.21.15.101:0/2071242366 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+10858 (secure 0 0 0) 0x7f0b40099890 con 0x7f0b50073160 2023-07-18T03:18:26.884 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:18:26.884 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/4192846769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":4192846769},{"type":"v1","addr":"172.21.15.101:6839","nonce":4192846769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6841/3824340175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6840","nonce":3824340175},{"type":"v1","addr":"172.21.15.101:6841","nonce":3824340175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:04:59.361185+0000","modified":"2023-07-18T03:16:36.002358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.155:6839/2055462483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2055462483},{"type":"v1","addr":"172.21.15.155:6839","nonce":2055462483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:18:26.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.891+0000 7f0b585ff700 1 -- 172.21.15.101:0/2071242366 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0b3c0a1380 msgr2=0x7f0b3c0a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:26.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.891+0000 7f0b585ff700 1 --2- 172.21.15.101:0/2071242366 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0b3c0a1380 0x7f0b3c0a3830 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c009fd0 tx=0x7f0b4c006210 comp rx=0 tx=0).stop 2023-07-18T03:18:26.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.891+0000 7f0b585ff700 1 -- 172.21.15.101:0/2071242366 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b50073160 msgr2=0x7f0b500ffff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:26.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.891+0000 7f0b585ff700 1 --2- 172.21.15.101:0/2071242366 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b50073160 0x7f0b500ffff0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0b4000e9a0 tx=0x7f0b4000ecb0 comp rx=0 tx=0).stop 2023-07-18T03:18:26.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.891+0000 7f0b585ff700 1 -- 172.21.15.101:0/2071242366 shutdown_connections 2023-07-18T03:18:26.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.891+0000 7f0b585ff700 1 --2- 172.21.15.101:0/2071242366 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b5010c390 0x7f0b50100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:26.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.891+0000 7f0b585ff700 1 --2- 172.21.15.101:0/2071242366 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0b50072850 0x7f0b500ffab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:26.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.891+0000 7f0b585ff700 1 --2- 172.21.15.101:0/2071242366 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0b3c0a1380 0x7f0b3c0a3830 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:26.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.891+0000 7f0b585ff700 1 --2- 172.21.15.101:0/2071242366 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b50073160 0x7f0b500ffff0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:26.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.891+0000 7f0b585ff700 1 -- 172.21.15.101:0/2071242366 >> 172.21.15.101:0/2071242366 conn(0x7f0b500fb630 msgr2=0x7f0b5010e470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:18:26.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.891+0000 7f0b585ff700 1 -- 172.21.15.101:0/2071242366 shutdown_connections 2023-07-18T03:18:26.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:26.891+0000 7f0b585ff700 1 -- 172.21.15.101:0/2071242366 wait complete. 2023-07-18T03:18:26.889 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 33 2023-07-18T03:18:26.905 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-18T03:18:27.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.132+0000 7fcee5bff700 1 -- 172.21.15.101:0/3058433716 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcee0100490 msgr2=0x7fcee010c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:27.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.132+0000 7fcee5bff700 1 --2- 172.21.15.101:0/3058433716 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcee0100490 0x7fcee010c920 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fced800b0a0 tx=0x7fced800b3b0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.132+0000 7fcee5bff700 1 -- 172.21.15.101:0/3058433716 shutdown_connections 2023-07-18T03:18:27.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.132+0000 7fcee5bff700 1 --2- 172.21.15.101:0/3058433716 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcee0100490 0x7fcee010c920 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.132+0000 7fcee5bff700 1 --2- 172.21.15.101:0/3058433716 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcee0105af0 0x7fcee0105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.132+0000 7fcee5bff700 1 --2- 172.21.15.101:0/3058433716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcee00ffae0 0x7fcee00fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.132+0000 7fcee5bff700 1 -- 172.21.15.101:0/3058433716 >> 172.21.15.101:0/3058433716 conn(0x7fcee00fb5e0 msgr2=0x7fcee00fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:18:27.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.132+0000 7fcee5bff700 1 -- 172.21.15.101:0/3058433716 shutdown_connections 2023-07-18T03:18:27.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.132+0000 7fcee5bff700 1 -- 172.21.15.101:0/3058433716 wait complete. 2023-07-18T03:18:27.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.133+0000 7fcee5bff700 1 Processor -- start 2023-07-18T03:18:27.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.133+0000 7fcee5bff700 1 -- start start 2023-07-18T03:18:27.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.133+0000 7fcee5bff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcee00ffae0 0x7fcee0197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.133+0000 7fcee5bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcee0100490 0x7fcee0198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.133+0000 7fcee5bff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcee0105af0 0x7fcee019c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.133+0000 7fcee5bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee019cec0 con 0x7fcee0100490 2023-07-18T03:18:27.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.133+0000 7fcee5bff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee019d030 con 0x7fcee0105af0 2023-07-18T03:18:27.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.133+0000 7fcee5bff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee019d1f0 con 0x7fcee00ffae0 2023-07-18T03:18:27.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.134+0000 7fcedf7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcee00ffae0 0x7fcee0197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.134+0000 7fcedeffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcee0100490 0x7fcee0198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.134+0000 7fcedf7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcee00ffae0 0x7fcee0197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:35470/0 (socket says 172.21.15.101:35470) 2023-07-18T03:18:27.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.134+0000 7fcedf7fe700 1 -- 172.21.15.101:0/265250277 learned_addr learned my addr 172.21.15.101:0/265250277 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:18:27.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.134+0000 7fcedffff700 1 --2- 172.21.15.101:0/265250277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcee0105af0 0x7fcee019c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.134+0000 7fcedf7fe700 1 -- 172.21.15.101:0/265250277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcee0105af0 msgr2=0x7fcee019c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:27.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.134+0000 7fcedf7fe700 1 --2- 172.21.15.101:0/265250277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcee0105af0 0x7fcee019c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.134+0000 7fcedf7fe700 1 -- 172.21.15.101:0/265250277 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcee0100490 msgr2=0x7fcee0198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:27.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.134+0000 7fcedf7fe700 1 --2- 172.21.15.101:0/265250277 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcee0100490 0x7fcee0198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.134+0000 7fcedf7fe700 1 -- 172.21.15.101:0/265250277 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced8009d00 con 0x7fcee00ffae0 2023-07-18T03:18:27.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.134+0000 7fcedf7fe700 1 --2- 172.21.15.101:0/265250277 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcee00ffae0 0x7fcee0197ee0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fced000e9a0 tx=0x7fced000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:18:27.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.135+0000 7fcedcff9700 1 -- 172.21.15.101:0/265250277 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced0009790 con 0x7fcee00ffae0 2023-07-18T03:18:27.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.135+0000 7fcedcff9700 1 -- 172.21.15.101:0/265250277 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fced000cbd0 con 0x7fcee00ffae0 2023-07-18T03:18:27.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.135+0000 7fcee5bff700 1 -- 172.21.15.101:0/265250277 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcee019d4d0 con 0x7fcee00ffae0 2023-07-18T03:18:27.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.135+0000 7fcedcff9700 1 -- 172.21.15.101:0/265250277 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced0018680 con 0x7fcee00ffae0 2023-07-18T03:18:27.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.135+0000 7fcee5bff700 1 -- 172.21.15.101:0/265250277 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee01a4d10 con 0x7fcee00ffae0 2023-07-18T03:18:27.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.139+0000 7fcedcff9700 1 -- 172.21.15.101:0/265250277 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fced000cd40 con 0x7fcee00ffae0 2023-07-18T03:18:27.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.140+0000 7fcef956fb80 1 Processor -- start 2023-07-18T03:18:27.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.140+0000 7fcef956fb80 1 -- start start 2023-07-18T03:18:27.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.141+0000 7fcef956fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe5feea2d0 0x55fe5feea6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.141+0000 7fcef956fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe5feeabe0 0x55fe5fef5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.141+0000 7fcef956fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fe5fef5ad0 0x55fe5fef7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.141+0000 7fcef956fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe5fdf0cb0 con 0x55fe5fef5ad0 2023-07-18T03:18:27.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.141+0000 7fcef956fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe5fe43a40 con 0x55fe5feeabe0 2023-07-18T03:18:27.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.141+0000 7fcef956fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe5fe3ced0 con 0x55fe5feea2d0 2023-07-18T03:18:27.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.141+0000 7fcedffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe5feea2d0 0x55fe5feea6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.141+0000 7fcedffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe5feea2d0 0x55fe5feea6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:35486/0 (socket says 172.21.15.101:35486) 2023-07-18T03:18:27.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.141+0000 7fcedffff700 1 -- 172.21.15.101:0/250384090 learned_addr learned my addr 172.21.15.101:0/250384090 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:18:27.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.141+0000 7fcedeffd700 1 --2- 172.21.15.101:0/250384090 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fe5fef5ad0 0x55fe5fef7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.141+0000 7fcedf7fe700 1 --2- 172.21.15.101:0/250384090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe5feeabe0 0x55fe5fef5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.141+0000 7fcedcff9700 1 --2- 172.21.15.101:0/265250277 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fcec80a12c0 0x7fcec80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.141+0000 7fcedffff700 1 -- 172.21.15.101:0/250384090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe5feeabe0 msgr2=0x55fe5fef5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:27.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.141+0000 7fcedffff700 1 --2- 172.21.15.101:0/250384090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe5feeabe0 0x55fe5fef5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.141+0000 7fcedcff9700 1 -- 172.21.15.101:0/265250277 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fced0014070 con 0x7fcee00ffae0 2023-07-18T03:18:27.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.141+0000 7fcedffff700 1 -- 172.21.15.101:0/250384090 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fe5fef5ad0 msgr2=0x55fe5fef7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:27.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.141+0000 7fcedffff700 1 --2- 172.21.15.101:0/250384090 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fe5fef5ad0 0x55fe5fef7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.141+0000 7fcedffff700 1 -- 172.21.15.101:0/250384090 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe5fc6ade0 con 0x55fe5feea2d0 2023-07-18T03:18:27.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.142+0000 7fcedffff700 1 --2- 172.21.15.101:0/265250277 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fcec80a12c0 0x7fcec80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.142+0000 7fcedffff700 1 --2- 172.21.15.101:0/250384090 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe5feea2d0 0x55fe5feea6a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fced8015000 tx=0x7fced80092e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44d889ef352e794d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:18:27.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.142+0000 7fcec5ffb700 1 -- 172.21.15.101:0/250384090 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced800e050 con 0x55fe5feea2d0 2023-07-18T03:18:27.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.142+0000 7fcec5ffb700 1 -- 172.21.15.101:0/250384090 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fced8009540 con 0x55fe5feea2d0 2023-07-18T03:18:27.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.142+0000 7fcec5ffb700 1 -- 172.21.15.101:0/250384090 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced801db80 con 0x55fe5feea2d0 2023-07-18T03:18:27.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.143+0000 7fcef956fb80 1 -- 172.21.15.101:0/250384090 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe5feea2d0 msgr2=0x55fe5feea6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:27.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.143+0000 7fcedffff700 1 --2- 172.21.15.101:0/265250277 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fcec80a12c0 0x7fcec80a3770 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fced8029000 tx=0x7fced8026000 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:18:27.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.143+0000 7fcef956fb80 1 --2- 172.21.15.101:0/250384090 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe5feea2d0 0x55fe5feea6a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fced8015000 tx=0x7fced80092e0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.143+0000 7fcef956fb80 1 -- 172.21.15.101:0/250384090 shutdown_connections 2023-07-18T03:18:27.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.143+0000 7fcef956fb80 1 --2- 172.21.15.101:0/250384090 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fe5fef5ad0 0x55fe5fef7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.143+0000 7fcef956fb80 1 --2- 172.21.15.101:0/250384090 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe5feea2d0 0x55fe5feea6a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.143+0000 7fcef956fb80 1 --2- 172.21.15.101:0/250384090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe5feeabe0 0x55fe5fef5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.143+0000 7fcef956fb80 1 -- 172.21.15.101:0/250384090 >> 172.21.15.101:0/250384090 conn(0x55fe5fee0da0 msgr2=0x55fe5fee66a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:18:27.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.143+0000 7fcef956fb80 1 -- 172.21.15.101:0/250384090 shutdown_connections 2023-07-18T03:18:27.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.143+0000 7fcef956fb80 1 -- 172.21.15.101:0/250384090 wait complete. 2023-07-18T03:18:27.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.143+0000 7fcef956fb80 1 -- start start 2023-07-18T03:18:27.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.144+0000 7fcef956fb80 1 Processor -- start 2023-07-18T03:18:27.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.144+0000 7fcef956fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe5feea2d0 0x55fe5fef5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.144+0000 7fcef956fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fe5feeabe0 0x55fe5fef2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.144+0000 7fcef956fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe5fef5ad0 0x55fe5fef2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.144+0000 7fcef956fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe5fddcac0 con 0x55fe5feeabe0 2023-07-18T03:18:27.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.144+0000 7fcef956fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe5fddc7c0 con 0x55fe5feea2d0 2023-07-18T03:18:27.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.144+0000 7fcef956fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe5fcc0920 con 0x55fe5fef5ad0 2023-07-18T03:18:27.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.144+0000 7fcef956fb80 10 client.0 _ll_register_callbacks cb 0x55fe5fdda760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:18:27.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.144+0000 7fcedeffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fe5feeabe0 0x55fe5fef2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.144+0000 7fcedf7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe5feea2d0 0x55fe5fef5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.144+0000 7fcedeffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fe5feeabe0 0x55fe5fef2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:42302/0 (socket says 172.21.15.101:42302) 2023-07-18T03:18:27.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.144+0000 7fcedeffd700 1 -- 172.21.15.101:0/4187200537 learned_addr learned my addr 172.21.15.101:0/4187200537 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:18:27.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.145+0000 7fcedffff700 1 --2- 172.21.15.101:0/4187200537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe5fef5ad0 0x55fe5fef2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.145+0000 7fcedeffd700 1 -- 172.21.15.101:0/4187200537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe5fef5ad0 msgr2=0x55fe5fef2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:27.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.145+0000 7fcedeffd700 1 --2- 172.21.15.101:0/4187200537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe5fef5ad0 0x55fe5fef2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.145+0000 7fcedeffd700 1 -- 172.21.15.101:0/4187200537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe5feea2d0 msgr2=0x55fe5fef5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:27.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.145+0000 7fcedeffd700 1 --2- 172.21.15.101:0/4187200537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe5feea2d0 0x55fe5fef5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.145+0000 7fcedeffd700 1 -- 172.21.15.101:0/4187200537 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee010d220 con 0x55fe5feeabe0 2023-07-18T03:18:27.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.145+0000 7fcedeffd700 1 --2- 172.21.15.101:0/4187200537 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fe5feeabe0 0x55fe5fef2790 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fced40127f0 tx=0x7fced4012b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=157b2471393ef4ff server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:18:27.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.145+0000 7fcebeffd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:18:27.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.145+0000 7fcebeffd700 1 -- 172.21.15.101:0/4187200537 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced4017070 con 0x55fe5feeabe0 2023-07-18T03:18:27.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.145+0000 7fcef956fb80 1 -- 172.21.15.101:0/4187200537 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fe5fe35eb0 con 0x55fe5feeabe0 2023-07-18T03:18:27.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.145+0000 7fcebeffd700 1 -- 172.21.15.101:0/4187200537 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fced400f4e0 con 0x55fe5feeabe0 2023-07-18T03:18:27.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.146+0000 7fcebeffd700 1 -- 172.21.15.101:0/4187200537 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fced400f6a0 con 0x55fe5feeabe0 2023-07-18T03:18:27.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.146+0000 7fcef956fb80 10 client.17718 fetch_fsmap learned FSMap version 33 2023-07-18T03:18:27.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.146+0000 7fcef956fb80 1 -- 172.21.15.101:0/4187200537 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55fe5fc53c60 con 0x55fe5feeabe0 2023-07-18T03:18:27.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.146+0000 7fcebeffd700 1 -- 172.21.15.101:0/4187200537 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fe5fe35eb0 con 0x55fe5feeabe0 2023-07-18T03:18:27.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.146+0000 7fcebeffd700 1 -- 172.21.15.101:0/4187200537 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fced4010800 con 0x55fe5feeabe0 2023-07-18T03:18:27.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.146+0000 7fcef956fb80 10 client.17718 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:18:27.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.146+0000 7fcef956fb80 10 client.17718 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:18:27.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.146+0000 7fcef956fb80 20 client.17718 populate_metadata read hostname 'smithi101' 2023-07-18T03:18:27.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.146+0000 7fcef956fb80 1 --2- 172.21.15.101:0/4187200537 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55fe5ff05510 0x55fe5ff07920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.146+0000 7fcef956fb80 4 client.17718 mds_command: new command op to 24541 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:18:27.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.146+0000 7fcef956fb80 1 -- 172.21.15.101:0/4187200537 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe5fdf8050 con 0x55fe5ff05510 2023-07-18T03:18:27.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.146+0000 7fcedf7fe700 1 --2- 172.21.15.101:0/4187200537 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55fe5ff05510 0x55fe5ff07920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.147+0000 7fcebeffd700 10 client.17718 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:18:27.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.147+0000 7fcedf7fe700 1 --2- 172.21.15.101:0/4187200537 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55fe5ff05510 0x55fe5ff07920 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c4109e4dadd2c481 server_cookie=ab7e664140df8ee3 in_seq=0 out_seq=0 2023-07-18T03:18:27.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.147+0000 7fcebeffd700 1 -- 172.21.15.101:0/4187200537 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55fe5fdf8050 con 0x55fe5ff05510 2023-07-18T03:18:27.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.147+0000 7fcebeffd700 10 client.17718 handle_command_reply: tid=0 2023-07-18T03:18:27.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.148+0000 7fcef956fb80 1 client.17718 shutdown 2023-07-18T03:18:27.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.148+0000 7fcef956fb80 1 -- 172.21.15.101:0/4187200537 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fe5feeabe0 msgr2=0x55fe5fef2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:27.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.148+0000 7fcef956fb80 1 --2- 172.21.15.101:0/4187200537 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fe5feeabe0 0x55fe5fef2790 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fced40127f0 tx=0x7fced4012b00 comp rx=0 tx=0).stop 2023-07-18T03:18:27.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.148+0000 7fcef956fb80 1 -- 172.21.15.101:0/4187200537 shutdown_connections 2023-07-18T03:18:27.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.148+0000 7fcef956fb80 1 --2- 172.21.15.101:0/4187200537 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55fe5ff05510 0x55fe5ff07920 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.148+0000 7fcef956fb80 1 --2- 172.21.15.101:0/4187200537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe5feea2d0 0x55fe5fef5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.148+0000 7fcef956fb80 1 --2- 172.21.15.101:0/4187200537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe5fef5ad0 0x55fe5fef2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.148+0000 7fcef956fb80 1 --2- 172.21.15.101:0/4187200537 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fe5feeabe0 0x55fe5fef2790 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.148+0000 7fcebeffd700 0 client.17718 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:18:27.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.148+0000 7fcef956fb80 1 -- 172.21.15.101:0/4187200537 >> 172.21.15.101:0/4187200537 conn(0x55fe5fee0da0 msgr2=0x55fe5fee6230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:18:27.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.148+0000 7fcef956fb80 1 -- 172.21.15.101:0/4187200537 shutdown_connections 2023-07-18T03:18:27.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.148+0000 7fcef956fb80 1 -- 172.21.15.101:0/4187200537 wait complete. 2023-07-18T03:18:27.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.148+0000 7fcef956fb80 20 client.17718 trim_cache size 0 max 16384 2023-07-18T03:18:27.156 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.161+0000 7fcef956fb80 1 Processor -- start 2023-07-18T03:18:27.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.162+0000 7fcef956fb80 1 -- start start 2023-07-18T03:18:27.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.162+0000 7fcef956fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe5feea2d0 0x55fe5fef7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.162+0000 7fcef956fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fe5feeabe0 0x55fe5fef50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.162+0000 7fcef956fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe5fef5ad0 0x55fe5feeda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.162+0000 7fcef956fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe5fef38a0 con 0x55fe5feeabe0 2023-07-18T03:18:27.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.162+0000 7fcef956fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe5feec1a0 con 0x55fe5feea2d0 2023-07-18T03:18:27.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.162+0000 7fcef956fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe5ff07b40 con 0x55fe5fef5ad0 2023-07-18T03:18:27.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.162+0000 7fcedeffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fe5feeabe0 0x55fe5fef50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.162+0000 7fcedeffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fe5feeabe0 0x55fe5fef50b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:42314/0 (socket says 172.21.15.101:42314) 2023-07-18T03:18:27.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.162+0000 7fcedeffd700 1 -- 172.21.15.101:0/1788166393 learned_addr learned my addr 172.21.15.101:0/1788166393 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:18:27.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.162+0000 7fcedffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe5fef5ad0 0x55fe5feeda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.162+0000 7fcedf7fe700 1 --2- 172.21.15.101:0/1788166393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe5feea2d0 0x55fe5fef7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.163+0000 7fcedeffd700 1 -- 172.21.15.101:0/1788166393 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe5fef5ad0 msgr2=0x55fe5feeda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:27.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.163+0000 7fcedeffd700 1 --2- 172.21.15.101:0/1788166393 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe5fef5ad0 0x55fe5feeda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.163+0000 7fcedeffd700 1 -- 172.21.15.101:0/1788166393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe5feea2d0 msgr2=0x55fe5fef7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:27.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.163+0000 7fcedeffd700 1 --2- 172.21.15.101:0/1788166393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe5feea2d0 0x55fe5fef7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.163+0000 7fcedeffd700 1 -- 172.21.15.101:0/1788166393 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe5fc91250 con 0x55fe5feeabe0 2023-07-18T03:18:27.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.163+0000 7fcedeffd700 1 --2- 172.21.15.101:0/1788166393 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fe5feeabe0 0x55fe5fef50b0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fced40127c0 tx=0x7fced4012e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59055792775ca9c2 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:18:27.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.163+0000 7fcec67fc700 1 -- 172.21.15.101:0/1788166393 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced4017070 con 0x55fe5feeabe0 2023-07-18T03:18:27.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.163+0000 7fcec67fc700 1 -- 172.21.15.101:0/1788166393 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fced4018040 con 0x55fe5feeabe0 2023-07-18T03:18:27.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.163+0000 7fcec67fc700 1 -- 172.21.15.101:0/1788166393 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced4003680 con 0x55fe5feeabe0 2023-07-18T03:18:27.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.164+0000 7fcef956fb80 1 -- 172.21.15.101:0/1788166393 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fe5feeabe0 msgr2=0x55fe5fef50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:27.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.164+0000 7fcef956fb80 1 --2- 172.21.15.101:0/1788166393 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fe5feeabe0 0x55fe5fef50b0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fced40127c0 tx=0x7fced4012e50 comp rx=0 tx=0).stop 2023-07-18T03:18:27.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.164+0000 7fcef956fb80 1 -- 172.21.15.101:0/1788166393 shutdown_connections 2023-07-18T03:18:27.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.164+0000 7fcef956fb80 1 --2- 172.21.15.101:0/1788166393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe5feea2d0 0x55fe5fef7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.164+0000 7fcef956fb80 1 --2- 172.21.15.101:0/1788166393 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe5fef5ad0 0x55fe5feeda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.164+0000 7fcef956fb80 1 --2- 172.21.15.101:0/1788166393 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fe5feeabe0 0x55fe5fef50b0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.164+0000 7fcef956fb80 1 -- 172.21.15.101:0/1788166393 >> 172.21.15.101:0/1788166393 conn(0x55fe5fee0da0 msgr2=0x55fe5ff0c280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:18:27.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.164+0000 7fcef956fb80 1 -- 172.21.15.101:0/1788166393 shutdown_connections 2023-07-18T03:18:27.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.164+0000 7fcef956fb80 1 -- 172.21.15.101:0/1788166393 wait complete. 2023-07-18T03:18:27.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.164+0000 7fcef956fb80 1 -- start start 2023-07-18T03:18:27.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.164+0000 7fcef956fb80 1 Processor -- start 2023-07-18T03:18:27.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.165+0000 7fcef956fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe5feea2d0 0x55fe5fefa6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.165+0000 7fcef956fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe5feeabe0 0x55fe5fefac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.165+0000 7fcef956fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fe5fef5ad0 0x55fe5ff583d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.165+0000 7fcef956fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe5ff58910 con 0x55fe5fef5ad0 2023-07-18T03:18:27.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.165+0000 7fcef956fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe5ff58a50 con 0x55fe5feeabe0 2023-07-18T03:18:27.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.165+0000 7fcef956fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe5ff58c10 con 0x55fe5feea2d0 2023-07-18T03:18:27.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.165+0000 7fcef956fb80 10 client.0 _ll_register_callbacks cb 0x7fced4009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:18:27.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.165+0000 7fcedf7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe5feea2d0 0x55fe5fefa6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.165+0000 7fcedf7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe5feea2d0 0x55fe5fefa6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:35504/0 (socket says 172.21.15.101:35504) 2023-07-18T03:18:27.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.165+0000 7fcedf7fe700 1 -- 172.21.15.101:0/1499429404 learned_addr learned my addr 172.21.15.101:0/1499429404 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:18:27.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.165+0000 7fcedeffd700 1 --2- 172.21.15.101:0/1499429404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe5feeabe0 0x55fe5fefac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.165+0000 7fcedffff700 1 --2- 172.21.15.101:0/1499429404 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fe5fef5ad0 0x55fe5ff583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.165+0000 7fcedf7fe700 1 -- 172.21.15.101:0/1499429404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe5feeabe0 msgr2=0x55fe5fefac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:27.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.165+0000 7fcedf7fe700 1 --2- 172.21.15.101:0/1499429404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe5feeabe0 0x55fe5fefac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.165+0000 7fcedf7fe700 1 -- 172.21.15.101:0/1499429404 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fe5fef5ad0 msgr2=0x55fe5ff583d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:27.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.165+0000 7fcedf7fe700 1 --2- 172.21.15.101:0/1499429404 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fe5fef5ad0 0x55fe5ff583d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.165+0000 7fcedf7fe700 1 -- 172.21.15.101:0/1499429404 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe5fc53c60 con 0x55fe5feea2d0 2023-07-18T03:18:27.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.165+0000 7fcedffff700 1 --2- 172.21.15.101:0/1499429404 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fe5fef5ad0 0x55fe5ff583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:18:27.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.166+0000 7fcedf7fe700 1 --2- 172.21.15.101:0/1499429404 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe5feea2d0 0x55fe5fefa6c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fced0099ce0 tx=0x7fced0099ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=741d35b402ee6161 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:18:27.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.166+0000 7fcebeffd700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:18:27.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.166+0000 7fcebeffd700 1 -- 172.21.15.101:0/1499429404 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced009a440 con 0x55fe5feea2d0 2023-07-18T03:18:27.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.166+0000 7fcebeffd700 1 -- 172.21.15.101:0/1499429404 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fced0099fe0 con 0x55fe5feea2d0 2023-07-18T03:18:27.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.166+0000 7fcef956fb80 1 -- 172.21.15.101:0/1499429404 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fe5fd964f0 con 0x55fe5feea2d0 2023-07-18T03:18:27.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.166+0000 7fcebeffd700 1 -- 172.21.15.101:0/1499429404 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fced009e010 con 0x55fe5feea2d0 2023-07-18T03:18:27.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.166+0000 7fcef956fb80 10 client.27793 fetch_fsmap learned FSMap version 33 2023-07-18T03:18:27.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.166+0000 7fcef956fb80 1 -- 172.21.15.101:0/1499429404 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55fe5fc692b0 con 0x55fe5feea2d0 2023-07-18T03:18:27.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.166+0000 7fcebeffd700 1 -- 172.21.15.101:0/1499429404 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fced00b8020 con 0x55fe5feea2d0 2023-07-18T03:18:27.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.166+0000 7fcebeffd700 1 -- 172.21.15.101:0/1499429404 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fced00995f0 con 0x55fe5feea2d0 2023-07-18T03:18:27.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.166+0000 7fcef956fb80 10 client.27793 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:18:27.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.166+0000 7fcef956fb80 10 client.27793 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:18:27.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.166+0000 7fcef956fb80 20 client.27793 populate_metadata read hostname 'smithi101' 2023-07-18T03:18:27.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.166+0000 7fcef956fb80 1 --2- 172.21.15.101:0/1499429404 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55fe5ff5a3a0 0x55fe5ff5c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.166+0000 7fcef956fb80 4 client.27793 mds_command: new command op to 24541 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:18:27.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.166+0000 7fcef956fb80 1 -- 172.21.15.101:0/1499429404 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fe5fc8bf70 con 0x55fe5ff5a3a0 2023-07-18T03:18:27.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.167+0000 7fcedeffd700 1 --2- 172.21.15.101:0/1499429404 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55fe5ff5a3a0 0x55fe5ff5c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.167+0000 7fcebeffd700 10 client.27793 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:18:27.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.167+0000 7fcedeffd700 1 --2- 172.21.15.101:0/1499429404 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55fe5ff5a3a0 0x55fe5ff5c7b0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f7b9d08d317281a5 server_cookie=73de049c1a8ae830 in_seq=0 out_seq=0 2023-07-18T03:18:27.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.167+0000 7fcebeffd700 1 -- 172.21.15.101:0/1499429404 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fe5fc8bf70 con 0x55fe5ff5a3a0 2023-07-18T03:18:27.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.167+0000 7fcebeffd700 10 client.27793 handle_command_reply: tid=0 2023-07-18T03:18:27.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.167+0000 7fcef956fb80 1 client.27793 shutdown 2023-07-18T03:18:27.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.167+0000 7fcef956fb80 1 -- 172.21.15.101:0/1499429404 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe5feea2d0 msgr2=0x55fe5fefa6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:27.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.167+0000 7fcef956fb80 1 --2- 172.21.15.101:0/1499429404 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe5feea2d0 0x55fe5fefa6c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fced0099ce0 tx=0x7fced0099ee0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.168+0000 7fcef956fb80 1 -- 172.21.15.101:0/1499429404 shutdown_connections 2023-07-18T03:18:27.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.168+0000 7fcef956fb80 1 --2- 172.21.15.101:0/1499429404 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55fe5ff5a3a0 0x55fe5ff5c7b0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.168+0000 7fcef956fb80 1 --2- 172.21.15.101:0/1499429404 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fe5fef5ad0 0x55fe5ff583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.168+0000 7fcebeffd700 0 client.27793 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:18:27.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.168+0000 7fcef956fb80 1 --2- 172.21.15.101:0/1499429404 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe5feea2d0 0x55fe5fefa6c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.168+0000 7fcef956fb80 1 --2- 172.21.15.101:0/1499429404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe5feeabe0 0x55fe5fefac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.168+0000 7fcef956fb80 1 -- 172.21.15.101:0/1499429404 >> 172.21.15.101:0/1499429404 conn(0x55fe5fee0da0 msgr2=0x55fe5feeda90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:18:27.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.168+0000 7fcef956fb80 1 -- 172.21.15.101:0/1499429404 shutdown_connections 2023-07-18T03:18:27.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.168+0000 7fcef956fb80 1 -- 172.21.15.101:0/1499429404 wait complete. 2023-07-18T03:18:27.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.168+0000 7fcef956fb80 20 client.27793 trim_cache size 0 max 16384 2023-07-18T03:18:27.173 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:18:27.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.168+0000 7fcee5bff700 1 -- 172.21.15.101:0/265250277 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fcec80a12c0 msgr2=0x7fcec80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:27.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.168+0000 7fcee5bff700 1 --2- 172.21.15.101:0/265250277 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fcec80a12c0 0x7fcec80a3770 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fced8029000 tx=0x7fced8026000 comp rx=0 tx=0).stop 2023-07-18T03:18:27.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.168+0000 7fcee5bff700 1 -- 172.21.15.101:0/265250277 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcee00ffae0 msgr2=0x7fcee0197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:27.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.168+0000 7fcee5bff700 1 --2- 172.21.15.101:0/265250277 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcee00ffae0 0x7fcee0197ee0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fced000e9a0 tx=0x7fced000ecb0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.168+0000 7fcee5bff700 1 -- 172.21.15.101:0/265250277 shutdown_connections 2023-07-18T03:18:27.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.168+0000 7fcee5bff700 1 --2- 172.21.15.101:0/265250277 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fcec80a12c0 0x7fcec80a3770 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.168+0000 7fcee5bff700 1 --2- 172.21.15.101:0/265250277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcee0105af0 0x7fcee019c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.168+0000 7fcee5bff700 1 --2- 172.21.15.101:0/265250277 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcee00ffae0 0x7fcee0197ee0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.168+0000 7fcee5bff700 1 --2- 172.21.15.101:0/265250277 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcee0100490 0x7fcee0198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.168+0000 7fcee5bff700 1 -- 172.21.15.101:0/265250277 >> 172.21.15.101:0/265250277 conn(0x7fcee00fb5e0 msgr2=0x7fcee010aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:18:27.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.169+0000 7fcee5bff700 1 -- 172.21.15.101:0/265250277 shutdown_connections 2023-07-18T03:18:27.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.169+0000 7fcee5bff700 1 -- 172.21.15.101:0/265250277 wait complete. 2023-07-18T03:18:27.179 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:27 smithi101 ceph-mon[126379]: pgmap v339: 129 pgs: 129 active+clean; 545 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 161 op/s 2023-07-18T03:18:27.180 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:27 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2071242366' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:18:27.181 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-18T03:18:27.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:27 smithi155 ceph-mon[128885]: pgmap v339: 129 pgs: 129 active+clean; 545 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 161 op/s 2023-07-18T03:18:27.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:27 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/2071242366' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:18:27.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.388+0000 7fda133ff700 1 -- 172.21.15.101:0/855481337 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda0c0678e0 msgr2=0x7fda0c067cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:27.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.388+0000 7fda133ff700 1 --2- 172.21.15.101:0/855481337 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda0c0678e0 0x7fda0c067cb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fda08009990 tx=0x7fda08009ca0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.388+0000 7fda133ff700 1 -- 172.21.15.101:0/855481337 shutdown_connections 2023-07-18T03:18:27.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.388+0000 7fda133ff700 1 --2- 172.21.15.101:0/855481337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda0c104e00 0x7fda0c10c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.388+0000 7fda133ff700 1 --2- 172.21.15.101:0/855481337 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda0c0678e0 0x7fda0c067cb0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.388+0000 7fda133ff700 1 --2- 172.21.15.101:0/855481337 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda0c068280 0x7fda0c104720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.388+0000 7fda133ff700 1 -- 172.21.15.101:0/855481337 >> 172.21.15.101:0/855481337 conn(0x7fda0c074b10 msgr2=0x7fda0c074f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:18:27.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.388+0000 7fda133ff700 1 -- 172.21.15.101:0/855481337 shutdown_connections 2023-07-18T03:18:27.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.388+0000 7fda133ff700 1 -- 172.21.15.101:0/855481337 wait complete. 2023-07-18T03:18:27.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.389+0000 7fda133ff700 1 Processor -- start 2023-07-18T03:18:27.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.389+0000 7fda133ff700 1 -- start start 2023-07-18T03:18:27.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.390+0000 7fda133ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda0c0678e0 0x7fda0c19ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.390+0000 7fda133ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda0c068280 0x7fda0c19bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.390+0000 7fda133ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda0c104e00 0x7fda0c195aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.390+0000 7fda133ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0c196110 con 0x7fda0c068280 2023-07-18T03:18:27.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.390+0000 7fda133ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0c196280 con 0x7fda0c104e00 2023-07-18T03:18:27.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.390+0000 7fda133ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0c196440 con 0x7fda0c0678e0 2023-07-18T03:18:27.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.390+0000 7fda1099a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda0c068280 0x7fda0c19bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.390+0000 7fda1119b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda0c0678e0 0x7fda0c19ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.390+0000 7fda1099a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda0c068280 0x7fda0c19bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:42340/0 (socket says 172.21.15.101:42340) 2023-07-18T03:18:27.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.390+0000 7fda1119b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda0c0678e0 0x7fda0c19ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:35522/0 (socket says 172.21.15.101:35522) 2023-07-18T03:18:27.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.390+0000 7fda1099a700 1 -- 172.21.15.101:0/1641777362 learned_addr learned my addr 172.21.15.101:0/1641777362 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:18:27.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.390+0000 7fda1199c700 1 --2- 172.21.15.101:0/1641777362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda0c104e00 0x7fda0c195aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.390+0000 7fda1119b700 1 -- 172.21.15.101:0/1641777362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda0c104e00 msgr2=0x7fda0c195aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:27.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.390+0000 7fda1119b700 1 --2- 172.21.15.101:0/1641777362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda0c104e00 0x7fda0c195aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.390+0000 7fda1119b700 1 -- 172.21.15.101:0/1641777362 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda0c068280 msgr2=0x7fda0c19bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:27.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.390+0000 7fda1119b700 1 --2- 172.21.15.101:0/1641777362 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda0c068280 0x7fda0c19bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.390+0000 7fda1119b700 1 -- 172.21.15.101:0/1641777362 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9fc009520 con 0x7fda0c0678e0 2023-07-18T03:18:27.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.390+0000 7fda1099a700 1 --2- 172.21.15.101:0/1641777362 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda0c068280 0x7fda0c19bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:18:27.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.391+0000 7fda1119b700 1 --2- 172.21.15.101:0/1641777362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda0c0678e0 0x7fda0c19ba30 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fda08000c00 tx=0x7fda08009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:18:27.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.391+0000 7fda027fc700 1 -- 172.21.15.101:0/1641777362 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0801d070 con 0x7fda0c0678e0 2023-07-18T03:18:27.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.391+0000 7fda027fc700 1 -- 172.21.15.101:0/1641777362 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda08022470 con 0x7fda0c0678e0 2023-07-18T03:18:27.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.391+0000 7fda027fc700 1 -- 172.21.15.101:0/1641777362 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda08017650 con 0x7fda0c0678e0 2023-07-18T03:18:27.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.391+0000 7fda133ff700 1 -- 172.21.15.101:0/1641777362 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda080095f0 con 0x7fda0c0678e0 2023-07-18T03:18:27.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.391+0000 7fda133ff700 1 -- 172.21.15.101:0/1641777362 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda0c196900 con 0x7fda0c0678e0 2023-07-18T03:18:27.394 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.394+0000 7fda027fc700 1 -- 172.21.15.101:0/1641777362 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fda080225e0 con 0x7fda0c0678e0 2023-07-18T03:18:27.395 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.396+0000 7fda26d40b80 1 Processor -- start 2023-07-18T03:18:27.395 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.396+0000 7fda26d40b80 1 -- start start 2023-07-18T03:18:27.395 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.396+0000 7fda027fc700 1 --2- 172.21.15.101:0/1641777362 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd9f80a12c0 0x7fd9f80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.395 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.396+0000 7fda26d40b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6e5c382d0 0x55c6e5c386a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.395 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.396+0000 7fda26d40b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6e5c38be0 0x55c6e5c479a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.395 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.396+0000 7fda26d40b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c6e5c47ee0 0x55c6e5c4a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.395 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.396+0000 7fda26d40b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6e5b3ecb0 con 0x55c6e5c47ee0 2023-07-18T03:18:27.396 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.396+0000 7fda26d40b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6e5b91a40 con 0x55c6e5c38be0 2023-07-18T03:18:27.396 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.396+0000 7fda26d40b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6e5b8aed0 con 0x55c6e5c382d0 2023-07-18T03:18:27.396 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.397+0000 7fda027fc700 1 -- 172.21.15.101:0/1641777362 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fda080ce250 con 0x7fda0c0678e0 2023-07-18T03:18:27.396 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.397+0000 7fda1199c700 1 --2- 172.21.15.101:0/1641777362 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd9f80a12c0 0x7fd9f80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.396 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.397+0000 7fda1119b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6e5c382d0 0x55c6e5c386a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.396 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.397+0000 7fda1119b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6e5c382d0 0x55c6e5c386a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:35528/0 (socket says 172.21.15.101:35528) 2023-07-18T03:18:27.396 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.397+0000 7fda1119b700 1 -- 172.21.15.101:0/143847417 learned_addr learned my addr 172.21.15.101:0/143847417 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:18:27.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.397+0000 7fda1199c700 1 --2- 172.21.15.101:0/143847417 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c6e5c47ee0 0x55c6e5c4a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.397+0000 7fda1099a700 1 --2- 172.21.15.101:0/143847417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6e5c38be0 0x55c6e5c479a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.397+0000 7fda1119b700 1 -- 172.21.15.101:0/143847417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6e5c38be0 msgr2=0x55c6e5c479a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:27.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.397+0000 7fda1119b700 1 --2- 172.21.15.101:0/143847417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6e5c38be0 0x55c6e5c479a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.397+0000 7fda1119b700 1 -- 172.21.15.101:0/143847417 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c6e5c47ee0 msgr2=0x55c6e5c4a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:27.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.397+0000 7fda1119b700 1 --2- 172.21.15.101:0/143847417 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c6e5c47ee0 0x55c6e5c4a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.397+0000 7fda1119b700 1 -- 172.21.15.101:0/143847417 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6e59b8de0 con 0x55c6e5c382d0 2023-07-18T03:18:27.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.397+0000 7fda1199c700 1 --2- 172.21.15.101:0/143847417 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c6e5c47ee0 0x55c6e5c4a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:18:27.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.397+0000 7fda1099a700 1 --2- 172.21.15.101:0/143847417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6e5c38be0 0x55c6e5c479a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:18:27.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.397+0000 7fda1119b700 1 --2- 172.21.15.101:0/143847417 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6e5c382d0 0x55c6e5c386a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fda080974a0 tx=0x7fda0809ae60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ffc28e107cffa9e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:18:27.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.397+0000 7fd9f77fe700 1 -- 172.21.15.101:0/143847417 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda080a4220 con 0x55c6e5c382d0 2023-07-18T03:18:27.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.398+0000 7fd9f77fe700 1 -- 172.21.15.101:0/143847417 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda0809afa0 con 0x55c6e5c382d0 2023-07-18T03:18:27.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.398+0000 7fd9f77fe700 1 -- 172.21.15.101:0/143847417 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda080cba50 con 0x55c6e5c382d0 2023-07-18T03:18:27.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.398+0000 7fda1199c700 1 --2- 172.21.15.101:0/1641777362 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd9f80a12c0 0x7fd9f80a3770 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fda04009550 tx=0x7fda04009920 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:18:27.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.398+0000 7fda26d40b80 1 -- 172.21.15.101:0/143847417 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6e5c382d0 msgr2=0x55c6e5c386a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:27.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.398+0000 7fda26d40b80 1 --2- 172.21.15.101:0/143847417 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6e5c382d0 0x55c6e5c386a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fda080974a0 tx=0x7fda0809ae60 comp rx=0 tx=0).stop 2023-07-18T03:18:27.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.398+0000 7fda26d40b80 1 -- 172.21.15.101:0/143847417 shutdown_connections 2023-07-18T03:18:27.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.398+0000 7fda26d40b80 1 --2- 172.21.15.101:0/143847417 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c6e5c47ee0 0x55c6e5c4a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.398+0000 7fda26d40b80 1 --2- 172.21.15.101:0/143847417 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6e5c382d0 0x55c6e5c386a0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.398+0000 7fda26d40b80 1 --2- 172.21.15.101:0/143847417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6e5c38be0 0x55c6e5c479a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.398+0000 7fda26d40b80 1 -- 172.21.15.101:0/143847417 >> 172.21.15.101:0/143847417 conn(0x55c6e5c2eda0 msgr2=0x55c6e5c346a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:18:27.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.398+0000 7fda26d40b80 1 -- 172.21.15.101:0/143847417 shutdown_connections 2023-07-18T03:18:27.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.398+0000 7fda26d40b80 1 -- 172.21.15.101:0/143847417 wait complete. 2023-07-18T03:18:27.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.399+0000 7fda26d40b80 1 -- start start 2023-07-18T03:18:27.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.399+0000 7fda26d40b80 1 Processor -- start 2023-07-18T03:18:27.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.400+0000 7fda26d40b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6e5c382d0 0x55c6e5c4d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.400+0000 7fda26d40b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c6e5c38be0 0x55c6e5c4d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.400+0000 7fda26d40b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6e5c47ee0 0x55c6e5c46ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.400+0000 7fda26d40b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6e5b2aac0 con 0x55c6e5c38be0 2023-07-18T03:18:27.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.400+0000 7fda26d40b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6e5b2a7c0 con 0x55c6e5c382d0 2023-07-18T03:18:27.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.400+0000 7fda26d40b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6e5a0e920 con 0x55c6e5c47ee0 2023-07-18T03:18:27.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.400+0000 7fda26d40b80 10 client.0 _ll_register_callbacks cb 0x55c6e5b28760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:18:27.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.400+0000 7fda1099a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c6e5c38be0 0x55c6e5c4d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.400+0000 7fda1199c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6e5c47ee0 0x55c6e5c46ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.400+0000 7fda1199c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6e5c47ee0 0x55c6e5c46ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:35536/0 (socket says 172.21.15.101:35536) 2023-07-18T03:18:27.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.400+0000 7fda1199c700 1 -- 172.21.15.101:0/2819979067 learned_addr learned my addr 172.21.15.101:0/2819979067 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:18:27.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.400+0000 7fda1119b700 1 --2- 172.21.15.101:0/2819979067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6e5c382d0 0x55c6e5c4d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.400+0000 7fda1099a700 1 -- 172.21.15.101:0/2819979067 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6e5c47ee0 msgr2=0x55c6e5c46ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:27.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.400+0000 7fda1099a700 1 --2- 172.21.15.101:0/2819979067 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6e5c47ee0 0x55c6e5c46ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.400+0000 7fda1099a700 1 -- 172.21.15.101:0/2819979067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6e5c382d0 msgr2=0x55c6e5c4d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:27.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.400+0000 7fda1099a700 1 --2- 172.21.15.101:0/2819979067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6e5c382d0 0x55c6e5c4d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.400+0000 7fda1099a700 1 -- 172.21.15.101:0/2819979067 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda0c196900 con 0x55c6e5c38be0 2023-07-18T03:18:27.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.400+0000 7fda1199c700 1 --2- 172.21.15.101:0/2819979067 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6e5c47ee0 0x55c6e5c46ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:18:27.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.401+0000 7fda1099a700 1 --2- 172.21.15.101:0/2819979067 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c6e5c38be0 0x55c6e5c4d550 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fda0c1061e0 tx=0x7fd9fc009dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8468dd7f6faa98b8 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:18:27.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.401+0000 7fd9f4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:18:27.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.401+0000 7fd9f4ff9700 1 -- 172.21.15.101:0/2819979067 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc017070 con 0x55c6e5c38be0 2023-07-18T03:18:27.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.401+0000 7fda26d40b80 1 -- 172.21.15.101:0/2819979067 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c6e5ae44f0 con 0x55c6e5c38be0 2023-07-18T03:18:27.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.401+0000 7fd9f4ff9700 1 -- 172.21.15.101:0/2819979067 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fd9fc00f880 con 0x55c6e5c38be0 2023-07-18T03:18:27.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.401+0000 7fd9f4ff9700 1 -- 172.21.15.101:0/2819979067 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9fc00fa40 con 0x55c6e5c38be0 2023-07-18T03:18:27.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.401+0000 7fda26d40b80 10 client.17736 fetch_fsmap learned FSMap version 33 2023-07-18T03:18:27.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.401+0000 7fda26d40b80 1 -- 172.21.15.101:0/2819979067 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55c6e59a1c60 con 0x55c6e5c38be0 2023-07-18T03:18:27.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.401+0000 7fd9f4ff9700 1 -- 172.21.15.101:0/2819979067 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9fc024080 con 0x55c6e5c38be0 2023-07-18T03:18:27.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.402+0000 7fd9f4ff9700 1 -- 172.21.15.101:0/2819979067 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd9fc005020 con 0x55c6e5c38be0 2023-07-18T03:18:27.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.402+0000 7fda26d40b80 10 client.17736 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:18:27.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.402+0000 7fda26d40b80 10 client.17736 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:18:27.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.402+0000 7fda26d40b80 20 client.17736 populate_metadata read hostname 'smithi101' 2023-07-18T03:18:27.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.402+0000 7fda26d40b80 1 --2- 172.21.15.101:0/2819979067 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55c6e5c574f0 0x55c6e5c59900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.402+0000 7fda26d40b80 4 client.17736 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:18:27.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.402+0000 7fda26d40b80 1 -- 172.21.15.101:0/2819979067 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6e5b83eb0 con 0x55c6e5c574f0 2023-07-18T03:18:27.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.402+0000 7fda1119b700 1 --2- 172.21.15.101:0/2819979067 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55c6e5c574f0 0x55c6e5c59900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.403+0000 7fd9f4ff9700 10 client.17736 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:18:27.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.403+0000 7fda1119b700 1 --2- 172.21.15.101:0/2819979067 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55c6e5c574f0 0x55c6e5c59900 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a5e44159416e2614 server_cookie=9c2301d00c0e61d3 in_seq=0 out_seq=0 2023-07-18T03:18:27.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.403+0000 7fd9f4ff9700 1 -- 172.21.15.101:0/2819979067 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c6e5b83eb0 con 0x55c6e5c574f0 2023-07-18T03:18:27.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.403+0000 7fd9f4ff9700 10 client.17736 handle_command_reply: tid=0 2023-07-18T03:18:27.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.403+0000 7fda26d40b80 1 client.17736 shutdown 2023-07-18T03:18:27.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.403+0000 7fda26d40b80 1 -- 172.21.15.101:0/2819979067 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c6e5c38be0 msgr2=0x55c6e5c4d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:27.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.403+0000 7fda26d40b80 1 --2- 172.21.15.101:0/2819979067 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c6e5c38be0 0x55c6e5c4d550 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fda0c1061e0 tx=0x7fd9fc009dd0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.404+0000 7fda26d40b80 1 -- 172.21.15.101:0/2819979067 shutdown_connections 2023-07-18T03:18:27.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.404+0000 7fda26d40b80 1 --2- 172.21.15.101:0/2819979067 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55c6e5c574f0 0x55c6e5c59900 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.404+0000 7fda26d40b80 1 --2- 172.21.15.101:0/2819979067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6e5c382d0 0x55c6e5c4d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.404+0000 7fda26d40b80 1 --2- 172.21.15.101:0/2819979067 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6e5c47ee0 0x55c6e5c46ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.404+0000 7fda26d40b80 1 --2- 172.21.15.101:0/2819979067 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c6e5c38be0 0x55c6e5c4d550 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.404+0000 7fda26d40b80 1 -- 172.21.15.101:0/2819979067 >> 172.21.15.101:0/2819979067 conn(0x55c6e5c2eda0 msgr2=0x55c6e5c34230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:18:27.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.404+0000 7fd9f4ff9700 0 client.17736 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:18:27.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.404+0000 7fda26d40b80 1 -- 172.21.15.101:0/2819979067 shutdown_connections 2023-07-18T03:18:27.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.404+0000 7fda26d40b80 1 -- 172.21.15.101:0/2819979067 wait complete. 2023-07-18T03:18:27.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.404+0000 7fda26d40b80 20 client.17736 trim_cache size 0 max 16384 2023-07-18T03:18:27.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.417+0000 7fda26d40b80 1 Processor -- start 2023-07-18T03:18:27.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.417+0000 7fda26d40b80 1 -- start start 2023-07-18T03:18:27.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.417+0000 7fda26d40b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c6e5c382d0 0x55c6e5c46fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.417+0000 7fda26d40b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6e5c38be0 0x55c6e5c3aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.417+0000 7fda26d40b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6e5c47ee0 0x55c6e5c4cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.417+0000 7fda26d40b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6e5c2e970 con 0x55c6e5c382d0 2023-07-18T03:18:27.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.417+0000 7fda26d40b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6e5c54bf0 con 0x55c6e5c38be0 2023-07-18T03:18:27.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.417+0000 7fda26d40b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6e5c42aa0 con 0x55c6e5c47ee0 2023-07-18T03:18:27.415 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.418+0000 7fda1119b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c6e5c382d0 0x55c6e5c46fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.415 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.418+0000 7fda1119b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c6e5c382d0 0x55c6e5c46fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:42358/0 (socket says 172.21.15.101:42358) 2023-07-18T03:18:27.415 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.418+0000 7fda1119b700 1 -- 172.21.15.101:0/2560070337 learned_addr learned my addr 172.21.15.101:0/2560070337 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:18:27.415 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.418+0000 7fda1099a700 1 --2- 172.21.15.101:0/2560070337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6e5c38be0 0x55c6e5c3aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.415 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.418+0000 7fda1199c700 1 --2- 172.21.15.101:0/2560070337 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6e5c47ee0 0x55c6e5c4cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.418+0000 7fda1119b700 1 -- 172.21.15.101:0/2560070337 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6e5c47ee0 msgr2=0x55c6e5c4cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:27.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.418+0000 7fda1119b700 1 --2- 172.21.15.101:0/2560070337 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6e5c47ee0 0x55c6e5c4cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.418+0000 7fda1119b700 1 -- 172.21.15.101:0/2560070337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6e5c38be0 msgr2=0x55c6e5c3aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:27.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.418+0000 7fda1119b700 1 --2- 172.21.15.101:0/2560070337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6e5c38be0 0x55c6e5c3aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.418+0000 7fda1119b700 1 -- 172.21.15.101:0/2560070337 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6e59df250 con 0x55c6e5c382d0 2023-07-18T03:18:27.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.418+0000 7fda1119b700 1 --2- 172.21.15.101:0/2560070337 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c6e5c382d0 0x55c6e5c46fd0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fda08005cb0 tx=0x7fda0801a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4594527f031823ae server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:18:27.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.418+0000 7fd9f7fff700 1 -- 172.21.15.101:0/2560070337 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda080a33e0 con 0x55c6e5c382d0 2023-07-18T03:18:27.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.418+0000 7fd9f7fff700 1 -- 172.21.15.101:0/2560070337 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda08099340 con 0x55c6e5c382d0 2023-07-18T03:18:27.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.418+0000 7fd9f7fff700 1 -- 172.21.15.101:0/2560070337 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0809a490 con 0x55c6e5c382d0 2023-07-18T03:18:27.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.419+0000 7fda26d40b80 1 -- 172.21.15.101:0/2560070337 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c6e5c382d0 msgr2=0x55c6e5c46fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:27.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.419+0000 7fda26d40b80 1 --2- 172.21.15.101:0/2560070337 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c6e5c382d0 0x55c6e5c46fd0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fda08005cb0 tx=0x7fda0801a040 comp rx=0 tx=0).stop 2023-07-18T03:18:27.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.419+0000 7fda26d40b80 1 -- 172.21.15.101:0/2560070337 shutdown_connections 2023-07-18T03:18:27.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.419+0000 7fda26d40b80 1 --2- 172.21.15.101:0/2560070337 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6e5c47ee0 0x55c6e5c4cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.419+0000 7fda26d40b80 1 --2- 172.21.15.101:0/2560070337 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c6e5c382d0 0x55c6e5c46fd0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.419+0000 7fda26d40b80 1 --2- 172.21.15.101:0/2560070337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6e5c38be0 0x55c6e5c3aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.419+0000 7fda26d40b80 1 -- 172.21.15.101:0/2560070337 >> 172.21.15.101:0/2560070337 conn(0x55c6e5c2eda0 msgr2=0x55c6e5c3a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:18:27.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.419+0000 7fda26d40b80 1 -- 172.21.15.101:0/2560070337 shutdown_connections 2023-07-18T03:18:27.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.419+0000 7fda26d40b80 1 -- 172.21.15.101:0/2560070337 wait complete. 2023-07-18T03:18:27.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.419+0000 7fda26d40b80 1 -- start start 2023-07-18T03:18:27.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.420+0000 7fda26d40b80 1 Processor -- start 2023-07-18T03:18:27.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.420+0000 7fda26d40b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c6e5c382d0 0x55c6e5c9f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.420+0000 7fda26d40b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6e5c38be0 0x55c6e5c9fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.420+0000 7fda26d40b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6e5c47ee0 0x55c6e5caa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.420+0000 7fda26d40b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6e5caa900 con 0x55c6e5c382d0 2023-07-18T03:18:27.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.420+0000 7fda26d40b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6e5caaa40 con 0x55c6e5c47ee0 2023-07-18T03:18:27.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.420+0000 7fda26d40b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6e5caabb0 con 0x55c6e5c38be0 2023-07-18T03:18:27.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.420+0000 7fda26d40b80 10 client.0 _ll_register_callbacks cb 0x7fd9fc0127f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:18:27.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.420+0000 7fda1119b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c6e5c382d0 0x55c6e5c9f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.420+0000 7fda1099a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6e5c38be0 0x55c6e5c9fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.420+0000 7fda1199c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6e5c47ee0 0x55c6e5caa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.420+0000 7fda1119b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c6e5c382d0 0x55c6e5c9f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:42372/0 (socket says 172.21.15.101:42372) 2023-07-18T03:18:27.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.420+0000 7fda1119b700 1 -- 172.21.15.101:0/1211215853 learned_addr learned my addr 172.21.15.101:0/1211215853 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:18:27.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.421+0000 7fda1119b700 1 -- 172.21.15.101:0/1211215853 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6e5c38be0 msgr2=0x55c6e5c9fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:27.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.421+0000 7fda1119b700 1 --2- 172.21.15.101:0/1211215853 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6e5c38be0 0x55c6e5c9fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.421+0000 7fda1119b700 1 -- 172.21.15.101:0/1211215853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6e5c47ee0 msgr2=0x55c6e5caa3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:27.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.421+0000 7fda1119b700 1 --2- 172.21.15.101:0/1211215853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6e5c47ee0 0x55c6e5caa3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.421+0000 7fda1119b700 1 -- 172.21.15.101:0/1211215853 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c6e59b8de0 con 0x55c6e5c382d0 2023-07-18T03:18:27.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.421+0000 7fda1099a700 1 --2- 172.21.15.101:0/1211215853 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6e5c38be0 0x55c6e5c9fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:18:27.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.421+0000 7fda1119b700 1 --2- 172.21.15.101:0/1211215853 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c6e5c382d0 0x55c6e5c9f530 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fda0800b2e0 tx=0x7fda0809efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44647ab32fd65d7e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:18:27.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.421+0000 7fd9f4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:18:27.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.421+0000 7fd9f4ff9700 1 -- 172.21.15.101:0/1211215853 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda080aa220 con 0x55c6e5c382d0 2023-07-18T03:18:27.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.421+0000 7fda26d40b80 1 -- 172.21.15.101:0/1211215853 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda0809afa0 con 0x55c6e5c382d0 2023-07-18T03:18:27.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.421+0000 7fd9f4ff9700 1 -- 172.21.15.101:0/1211215853 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fda0809a490 con 0x55c6e5c382d0 2023-07-18T03:18:27.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.421+0000 7fd9f4ff9700 1 -- 172.21.15.101:0/1211215853 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda080a33e0 con 0x55c6e5c382d0 2023-07-18T03:18:27.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.421+0000 7fda26d40b80 10 client.17742 fetch_fsmap learned FSMap version 33 2023-07-18T03:18:27.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.422+0000 7fda26d40b80 1 -- 172.21.15.101:0/1211215853 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55c6e59b72b0 con 0x55c6e5c382d0 2023-07-18T03:18:27.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.422+0000 7fd9f4ff9700 1 -- 172.21.15.101:0/1211215853 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda0809afa0 con 0x55c6e5c382d0 2023-07-18T03:18:27.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.422+0000 7fd9f4ff9700 1 -- 172.21.15.101:0/1211215853 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fda08095bb0 con 0x55c6e5c382d0 2023-07-18T03:18:27.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.422+0000 7fda26d40b80 10 client.17742 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:18:27.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.422+0000 7fda26d40b80 10 client.17742 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:18:27.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.422+0000 7fda26d40b80 20 client.17742 populate_metadata read hostname 'smithi101' 2023-07-18T03:18:27.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.422+0000 7fda26d40b80 1 --2- 172.21.15.101:0/1211215853 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55c6e5cabe00 0x55c6e5cae1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.422+0000 7fda26d40b80 4 client.17742 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:18:27.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.422+0000 7fda26d40b80 1 -- 172.21.15.101:0/1211215853 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c6e5b46050 con 0x55c6e5cabe00 2023-07-18T03:18:27.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.422+0000 7fda1099a700 1 --2- 172.21.15.101:0/1211215853 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55c6e5cabe00 0x55c6e5cae1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.423+0000 7fd9f4ff9700 10 client.17742 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:18:27.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.423+0000 7fda1099a700 1 --2- 172.21.15.101:0/1211215853 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55c6e5cabe00 0x55c6e5cae1e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=47680b13e1bf7dfa server_cookie=ae06c72145de323c in_seq=0 out_seq=0 2023-07-18T03:18:27.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.423+0000 7fd9f4ff9700 1 -- 172.21.15.101:0/1211215853 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c6e5b46050 con 0x55c6e5cabe00 2023-07-18T03:18:27.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.423+0000 7fd9f4ff9700 10 client.17742 handle_command_reply: tid=0 2023-07-18T03:18:27.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.423+0000 7fda26d40b80 1 client.17742 shutdown 2023-07-18T03:18:27.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.423+0000 7fda26d40b80 1 -- 172.21.15.101:0/1211215853 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c6e5c382d0 msgr2=0x55c6e5c9f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:27.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.423+0000 7fda26d40b80 1 --2- 172.21.15.101:0/1211215853 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c6e5c382d0 0x55c6e5c9f530 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fda0800b2e0 tx=0x7fda0809efd0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.423+0000 7fda26d40b80 1 -- 172.21.15.101:0/1211215853 shutdown_connections 2023-07-18T03:18:27.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.423+0000 7fda26d40b80 1 --2- 172.21.15.101:0/1211215853 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55c6e5cabe00 0x55c6e5cae1e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.423+0000 7fda26d40b80 1 --2- 172.21.15.101:0/1211215853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6e5c47ee0 0x55c6e5caa3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.423+0000 7fda26d40b80 1 --2- 172.21.15.101:0/1211215853 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c6e5c382d0 0x55c6e5c9f530 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.423+0000 7fd9f4ff9700 0 client.17742 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:18:27.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.423+0000 7fda26d40b80 1 --2- 172.21.15.101:0/1211215853 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6e5c38be0 0x55c6e5c9fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.423+0000 7fda26d40b80 1 -- 172.21.15.101:0/1211215853 >> 172.21.15.101:0/1211215853 conn(0x55c6e5c2eda0 msgr2=0x55c6e5c46df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:18:27.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.423+0000 7fda26d40b80 1 -- 172.21.15.101:0/1211215853 shutdown_connections 2023-07-18T03:18:27.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.423+0000 7fda26d40b80 1 -- 172.21.15.101:0/1211215853 wait complete. 2023-07-18T03:18:27.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.424+0000 7fda26d40b80 20 client.17742 trim_cache size 0 max 16384 2023-07-18T03:18:27.429 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:18:27.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.424+0000 7fda133ff700 1 -- 172.21.15.101:0/1641777362 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd9f80a12c0 msgr2=0x7fd9f80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:27.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.424+0000 7fda133ff700 1 --2- 172.21.15.101:0/1641777362 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd9f80a12c0 0x7fd9f80a3770 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fda04009550 tx=0x7fda04009920 comp rx=0 tx=0).stop 2023-07-18T03:18:27.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.424+0000 7fda133ff700 1 -- 172.21.15.101:0/1641777362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda0c0678e0 msgr2=0x7fda0c19ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:27.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.424+0000 7fda133ff700 1 --2- 172.21.15.101:0/1641777362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda0c0678e0 0x7fda0c19ba30 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fda08000c00 tx=0x7fda08009580 comp rx=0 tx=0).stop 2023-07-18T03:18:27.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.424+0000 7fda133ff700 1 -- 172.21.15.101:0/1641777362 shutdown_connections 2023-07-18T03:18:27.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.424+0000 7fda133ff700 1 --2- 172.21.15.101:0/1641777362 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd9f80a12c0 0x7fd9f80a3770 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.424+0000 7fda133ff700 1 --2- 172.21.15.101:0/1641777362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda0c104e00 0x7fda0c195aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.424+0000 7fda133ff700 1 --2- 172.21.15.101:0/1641777362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda0c0678e0 0x7fda0c19ba30 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.424+0000 7fda133ff700 1 --2- 172.21.15.101:0/1641777362 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda0c068280 0x7fda0c19bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.424+0000 7fda133ff700 1 -- 172.21.15.101:0/1641777362 >> 172.21.15.101:0/1641777362 conn(0x7fda0c074b10 msgr2=0x7fda0c0fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:18:27.432 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.424+0000 7fda133ff700 1 -- 172.21.15.101:0/1641777362 shutdown_connections 2023-07-18T03:18:27.432 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.424+0000 7fda133ff700 1 -- 172.21.15.101:0/1641777362 wait complete. 2023-07-18T03:18:27.438 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-18T03:18:27.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:27 smithi179 ceph-mon[129229]: pgmap v339: 129 pgs: 129 active+clean; 545 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 161 op/s 2023-07-18T03:18:27.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:27 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/2071242366' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:18:27.620 INFO:tasks.workunit.client.0.smithi101.stdout: 1 672856 5.65 MB/sec execute 472 sec latency 576.672 ms 2023-07-18T03:18:27.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.642+0000 7f6319dff700 1 -- 172.21.15.101:0/1114382266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6314105ae0 msgr2=0x7f6314105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:27.638 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.642+0000 7f6319dff700 1 --2- 172.21.15.101:0/1114382266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6314105ae0 0x7f6314105eb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f63040098f0 tx=0x7f6304009c00 comp rx=0 tx=0).stop 2023-07-18T03:18:27.638 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.642+0000 7f6319dff700 1 -- 172.21.15.101:0/1114382266 shutdown_connections 2023-07-18T03:18:27.638 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.642+0000 7f6319dff700 1 --2- 172.21.15.101:0/1114382266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6314105ae0 0x7f6314105eb0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.638 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.642+0000 7f6319dff700 1 --2- 172.21.15.101:0/1114382266 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6314100480 0x7f631410c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.638 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.642+0000 7f6319dff700 1 --2- 172.21.15.101:0/1114382266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63140ffad0 0x7f63140fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.638 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.642+0000 7f6319dff700 1 -- 172.21.15.101:0/1114382266 >> 172.21.15.101:0/1114382266 conn(0x7f63140fb5f0 msgr2=0x7f63140fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:18:27.638 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.642+0000 7f6319dff700 1 -- 172.21.15.101:0/1114382266 shutdown_connections 2023-07-18T03:18:27.639 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.642+0000 7f6319dff700 1 -- 172.21.15.101:0/1114382266 wait complete. 2023-07-18T03:18:27.640 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.643+0000 7f6319dff700 1 Processor -- start 2023-07-18T03:18:27.640 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.643+0000 7f6319dff700 1 -- start start 2023-07-18T03:18:27.641 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.643+0000 7f6319dff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63140ffad0 0x7f631419ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.641 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.643+0000 7f6319dff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6314100480 0x7f631419bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.641 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.643+0000 7f6319dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6314105ae0 0x7f6314195a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.641 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.643+0000 7f6319dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6314196100 con 0x7f6314105ae0 2023-07-18T03:18:27.641 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.643+0000 7f6319dff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6314196270 con 0x7f63140ffad0 2023-07-18T03:18:27.641 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.643+0000 7f6319dff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6314196430 con 0x7f6314100480 2023-07-18T03:18:27.641 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.644+0000 7f6313fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6314105ae0 0x7f6314195a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.642 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.644+0000 7f63137fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63140ffad0 0x7f631419ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.642 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.644+0000 7f6313fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6314105ae0 0x7f6314195a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:42386/0 (socket says 172.21.15.101:42386) 2023-07-18T03:18:27.642 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.644+0000 7f6313fff700 1 -- 172.21.15.101:0/1037294692 learned_addr learned my addr 172.21.15.101:0/1037294692 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:18:27.642 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.644+0000 7f6312ffd700 1 --2- 172.21.15.101:0/1037294692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6314100480 0x7f631419bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.642 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.644+0000 7f6313fff700 1 -- 172.21.15.101:0/1037294692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6314100480 msgr2=0x7f631419bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:27.642 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.644+0000 7f6313fff700 1 --2- 172.21.15.101:0/1037294692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6314100480 0x7f631419bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.642 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.644+0000 7f6313fff700 1 -- 172.21.15.101:0/1037294692 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63140ffad0 msgr2=0x7f631419ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:27.643 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.644+0000 7f6313fff700 1 --2- 172.21.15.101:0/1037294692 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63140ffad0 0x7f631419ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.643 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.644+0000 7f6313fff700 1 -- 172.21.15.101:0/1037294692 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6304009520 con 0x7f6314105ae0 2023-07-18T03:18:27.643 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.644+0000 7f6313fff700 1 --2- 172.21.15.101:0/1037294692 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6314105ae0 0x7f6314195a90 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f630c00c110 tx=0x7f630c00c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:18:27.643 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.644+0000 7f6310ff9700 1 -- 172.21.15.101:0/1037294692 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630c017070 con 0x7f6314105ae0 2023-07-18T03:18:27.643 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.645+0000 7f6319dff700 1 -- 172.21.15.101:0/1037294692 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63141966b0 con 0x7f6314105ae0 2023-07-18T03:18:27.643 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.645+0000 7f6310ff9700 1 -- 172.21.15.101:0/1037294692 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f630c00e050 con 0x7f6314105ae0 2023-07-18T03:18:27.643 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.645+0000 7f6310ff9700 1 -- 172.21.15.101:0/1037294692 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630c0078f0 con 0x7f6314105ae0 2023-07-18T03:18:27.643 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.645+0000 7f6319dff700 1 -- 172.21.15.101:0/1037294692 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63141a2a10 con 0x7f6314105ae0 2023-07-18T03:18:27.644 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.648+0000 7f6310ff9700 1 -- 172.21.15.101:0/1037294692 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f630c007a50 con 0x7f6314105ae0 2023-07-18T03:18:27.644 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.649+0000 7f632d7c7b80 1 Processor -- start 2023-07-18T03:18:27.645 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.649+0000 7f632d7c7b80 1 -- start start 2023-07-18T03:18:27.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.649+0000 7f6310ff9700 1 --2- 172.21.15.101:0/1037294692 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f62fc0a12c0 0x7f62fc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.650+0000 7f6310ff9700 1 -- 172.21.15.101:0/1037294692 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f630c0ccac0 con 0x7f6314105ae0 2023-07-18T03:18:27.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.650+0000 7f632d7c7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561d65cdb2d0 0x561d65cdb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.650+0000 7f632d7c7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d65cdbbe0 0x561d65ce6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.650+0000 7f632d7c7b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d65ce6ad0 0x561d65ce8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.650+0000 7f632d7c7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d65be1cb0 con 0x561d65cdb2d0 2023-07-18T03:18:27.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.650+0000 7f632d7c7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d65c34a40 con 0x561d65cdbbe0 2023-07-18T03:18:27.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.650+0000 7f632d7c7b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d65c2ded0 con 0x561d65ce6ad0 2023-07-18T03:18:27.647 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.650+0000 7f6312ffd700 1 --2- 172.21.15.101:0/1037294692 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f62fc0a12c0 0x7f62fc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.647 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.650+0000 7f63137fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561d65cdb2d0 0x561d65cdb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.647 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.650+0000 7f63137fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561d65cdb2d0 0x561d65cdb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:42394/0 (socket says 172.21.15.101:42394) 2023-07-18T03:18:27.647 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.650+0000 7f63137fe700 1 -- 172.21.15.101:0/4150669409 learned_addr learned my addr 172.21.15.101:0/4150669409 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:18:27.647 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.650+0000 7f6313fff700 1 --2- 172.21.15.101:0/4150669409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d65ce6ad0 0x561d65ce8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.647 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.650+0000 7f6312ffd700 1 --2- 172.21.15.101:0/4150669409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d65cdbbe0 0x561d65ce6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.647 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.650+0000 7f63137fe700 1 -- 172.21.15.101:0/4150669409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d65ce6ad0 msgr2=0x561d65ce8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:27.648 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.650+0000 7f63137fe700 1 --2- 172.21.15.101:0/4150669409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d65ce6ad0 0x561d65ce8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.648 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.650+0000 7f63137fe700 1 -- 172.21.15.101:0/4150669409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d65cdbbe0 msgr2=0x561d65ce6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:27.648 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.650+0000 7f63137fe700 1 --2- 172.21.15.101:0/4150669409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d65cdbbe0 0x561d65ce6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.648 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.650+0000 7f63137fe700 1 -- 172.21.15.101:0/4150669409 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d65a5bde0 con 0x561d65cdb2d0 2023-07-18T03:18:27.648 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.650+0000 7f6312ffd700 1 --2- 172.21.15.101:0/4150669409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d65cdbbe0 0x561d65ce6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:18:27.651 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.651+0000 7f6312ffd700 1 --2- 172.21.15.101:0/1037294692 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f62fc0a12c0 0x7f62fc0a3770 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f63140fdef0 tx=0x7f6308014910 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:18:27.651 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.651+0000 7f63137fe700 1 --2- 172.21.15.101:0/4150669409 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561d65cdb2d0 0x561d65cdb6a0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f6304000c00 tx=0x7f6304005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce14e9d145b4ed4a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:18:27.651 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.651+0000 7f62f9ffb700 1 -- 172.21.15.101:0/4150669409 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630400f890 con 0x561d65cdb2d0 2023-07-18T03:18:27.651 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.651+0000 7f62f9ffb700 1 -- 172.21.15.101:0/4150669409 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f630401d040 con 0x561d65cdb2d0 2023-07-18T03:18:27.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.651+0000 7f62f9ffb700 1 -- 172.21.15.101:0/4150669409 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63040054f0 con 0x561d65cdb2d0 2023-07-18T03:18:27.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.652+0000 7f632d7c7b80 1 -- 172.21.15.101:0/4150669409 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561d65cdb2d0 msgr2=0x561d65cdb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:27.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.652+0000 7f632d7c7b80 1 --2- 172.21.15.101:0/4150669409 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561d65cdb2d0 0x561d65cdb6a0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f6304000c00 tx=0x7f6304005dc0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.652+0000 7f632d7c7b80 1 -- 172.21.15.101:0/4150669409 shutdown_connections 2023-07-18T03:18:27.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.652+0000 7f632d7c7b80 1 --2- 172.21.15.101:0/4150669409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d65ce6ad0 0x561d65ce8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.652+0000 7f632d7c7b80 1 --2- 172.21.15.101:0/4150669409 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561d65cdb2d0 0x561d65cdb6a0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.652+0000 7f632d7c7b80 1 --2- 172.21.15.101:0/4150669409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d65cdbbe0 0x561d65ce6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.652+0000 7f632d7c7b80 1 -- 172.21.15.101:0/4150669409 >> 172.21.15.101:0/4150669409 conn(0x561d65cd1da0 msgr2=0x561d65cd76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:18:27.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.652+0000 7f632d7c7b80 1 -- 172.21.15.101:0/4150669409 shutdown_connections 2023-07-18T03:18:27.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.652+0000 7f632d7c7b80 1 -- 172.21.15.101:0/4150669409 wait complete. 2023-07-18T03:18:27.654 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.652+0000 7f632d7c7b80 1 -- start start 2023-07-18T03:18:27.654 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.653+0000 7f632d7c7b80 1 Processor -- start 2023-07-18T03:18:27.654 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.653+0000 7f632d7c7b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d65cdb2d0 0x561d65ce6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.654 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.653+0000 7f632d7c7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561d65cdbbe0 0x561d65ce36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.654 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.654+0000 7f632d7c7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d65ce6ad0 0x561d65ce3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.654 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.654+0000 7f632d7c7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d65bcdac0 con 0x561d65cdbbe0 2023-07-18T03:18:27.654 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.654+0000 7f632d7c7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d65bcd7c0 con 0x561d65ce6ad0 2023-07-18T03:18:27.654 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.654+0000 7f632d7c7b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d65ab1920 con 0x561d65cdb2d0 2023-07-18T03:18:27.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.654+0000 7f632d7c7b80 10 client.0 _ll_register_callbacks cb 0x561d65bcb760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:18:27.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.654+0000 7f63137fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d65cdb2d0 0x561d65ce6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.654+0000 7f63137fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d65cdb2d0 0x561d65ce6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:35610/0 (socket says 172.21.15.101:35610) 2023-07-18T03:18:27.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.654+0000 7f63137fe700 1 -- 172.21.15.101:0/3867305824 learned_addr learned my addr 172.21.15.101:0/3867305824 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:18:27.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.654+0000 7f6312ffd700 1 --2- 172.21.15.101:0/3867305824 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561d65cdbbe0 0x561d65ce36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.654+0000 7f6313fff700 1 --2- 172.21.15.101:0/3867305824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d65ce6ad0 0x561d65ce3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.654+0000 7f63137fe700 1 -- 172.21.15.101:0/3867305824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d65ce6ad0 msgr2=0x561d65ce3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:27.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.654+0000 7f63137fe700 1 --2- 172.21.15.101:0/3867305824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d65ce6ad0 0x561d65ce3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.654+0000 7f63137fe700 1 -- 172.21.15.101:0/3867305824 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561d65cdbbe0 msgr2=0x561d65ce36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:27.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.654+0000 7f63137fe700 1 --2- 172.21.15.101:0/3867305824 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561d65cdbbe0 0x561d65ce36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.654+0000 7f63137fe700 1 -- 172.21.15.101:0/3867305824 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f631410d210 con 0x561d65cdb2d0 2023-07-18T03:18:27.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.654+0000 7f6313fff700 1 --2- 172.21.15.101:0/3867305824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d65ce6ad0 0x561d65ce3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:18:27.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.654+0000 7f6312ffd700 1 --2- 172.21.15.101:0/3867305824 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561d65cdbbe0 0x561d65ce36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:18:27.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.655+0000 7f63137fe700 1 --2- 172.21.15.101:0/3867305824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d65cdb2d0 0x561d65ce6460 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6304009fd0 tx=0x7f630400b770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f9a878ab609be70 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:18:27.657 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.655+0000 7f62f2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:18:27.657 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.655+0000 7f62f2ffd700 1 -- 172.21.15.101:0/3867305824 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63040054e0 con 0x561d65cdb2d0 2023-07-18T03:18:27.657 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.655+0000 7f632d7c7b80 1 -- 172.21.15.101:0/3867305824 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561d65b874f0 con 0x561d65cdb2d0 2023-07-18T03:18:27.657 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.655+0000 7f62f2ffd700 1 -- 172.21.15.101:0/3867305824 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f6304005640 con 0x561d65cdb2d0 2023-07-18T03:18:27.657 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.655+0000 7f62f2ffd700 1 -- 172.21.15.101:0/3867305824 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f630400bc10 con 0x561d65cdb2d0 2023-07-18T03:18:27.657 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.655+0000 7f632d7c7b80 10 client.27823 fetch_fsmap learned FSMap version 33 2023-07-18T03:18:27.657 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.655+0000 7f632d7c7b80 1 -- 172.21.15.101:0/3867305824 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x561d65a44c60 con 0x561d65cdb2d0 2023-07-18T03:18:27.657 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.655+0000 7f62f2ffd700 1 -- 172.21.15.101:0/3867305824 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561d65b874f0 con 0x561d65cdb2d0 2023-07-18T03:18:27.658 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.655+0000 7f62f2ffd700 1 -- 172.21.15.101:0/3867305824 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6304015020 con 0x561d65cdb2d0 2023-07-18T03:18:27.658 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.656+0000 7f632d7c7b80 10 client.27823 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:18:27.658 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.656+0000 7f632d7c7b80 10 client.27823 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:18:27.658 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.656+0000 7f632d7c7b80 20 client.27823 populate_metadata read hostname 'smithi101' 2023-07-18T03:18:27.658 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.656+0000 7f632d7c7b80 1 --2- 172.21.15.101:0/3867305824 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x561d65cfa2f0 0x561d65cfc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.658 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.656+0000 7f632d7c7b80 4 client.27823 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:18:27.659 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.656+0000 7f632d7c7b80 1 -- 172.21.15.101:0/3867305824 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561d65c26eb0 con 0x561d65cfa2f0 2023-07-18T03:18:27.659 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.656+0000 7f6312ffd700 1 --2- 172.21.15.101:0/3867305824 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x561d65cfa2f0 0x561d65cfc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.657+0000 7f6312ffd700 1 --2- 172.21.15.101:0/3867305824 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x561d65cfa2f0 0x561d65cfc700 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5138322cbbece1eb server_cookie=fc68a5ef832d5ac6 in_seq=0 out_seq=0 2023-07-18T03:18:27.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.657+0000 7f62f2ffd700 10 client.27823 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:18:27.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.657+0000 7f62f2ffd700 1 -- 172.21.15.101:0/3867305824 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561d65c26eb0 con 0x561d65cfa2f0 2023-07-18T03:18:27.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.657+0000 7f62f2ffd700 10 client.27823 handle_command_reply: tid=0 2023-07-18T03:18:27.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.657+0000 7f632d7c7b80 1 client.27823 shutdown 2023-07-18T03:18:27.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.661+0000 7f632d7c7b80 1 -- 172.21.15.101:0/3867305824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d65cdb2d0 msgr2=0x561d65ce6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:27.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.661+0000 7f632d7c7b80 1 --2- 172.21.15.101:0/3867305824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d65cdb2d0 0x561d65ce6460 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6304009fd0 tx=0x7f630400b770 comp rx=0 tx=0).stop 2023-07-18T03:18:27.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.661+0000 7f632d7c7b80 1 -- 172.21.15.101:0/3867305824 shutdown_connections 2023-07-18T03:18:27.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.661+0000 7f632d7c7b80 1 --2- 172.21.15.101:0/3867305824 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x561d65cfa2f0 0x561d65cfc700 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.661+0000 7f632d7c7b80 1 --2- 172.21.15.101:0/3867305824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d65ce6ad0 0x561d65ce3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.661+0000 7f632d7c7b80 1 --2- 172.21.15.101:0/3867305824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d65cdb2d0 0x561d65ce6460 secure :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6304009fd0 tx=0x7f630400b770 comp rx=0 tx=0).stop 2023-07-18T03:18:27.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.661+0000 7f632d7c7b80 1 --2- 172.21.15.101:0/3867305824 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561d65cdbbe0 0x561d65ce36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.662 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.661+0000 7f632d7c7b80 1 -- 172.21.15.101:0/3867305824 >> 172.21.15.101:0/3867305824 conn(0x561d65cd1da0 msgr2=0x561d65cd7230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:18:27.662 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.661+0000 7f62f2ffd700 0 client.27823 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:18:27.662 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.662+0000 7f632d7c7b80 1 -- 172.21.15.101:0/3867305824 shutdown_connections 2023-07-18T03:18:27.662 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.662+0000 7f632d7c7b80 1 -- 172.21.15.101:0/3867305824 wait complete. 2023-07-18T03:18:27.662 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.662+0000 7f632d7c7b80 20 client.27823 trim_cache size 0 max 16384 2023-07-18T03:18:27.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.682+0000 7f632d7c7b80 1 Processor -- start 2023-07-18T03:18:27.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.682+0000 7f632d7c7b80 1 -- start start 2023-07-18T03:18:27.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.682+0000 7f632d7c7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561d65cdbbe0 0x561d65ce36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.682+0000 7f632d7c7b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d65ce6ad0 0x561d65ce6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.682+0000 7f632d7c7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d65ce6540 0x561d65ce0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.682+0000 7f632d7c7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d65cd8530 con 0x561d65cdbbe0 2023-07-18T03:18:27.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.682+0000 7f632d7c7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d65cd8d20 con 0x561d65ce6540 2023-07-18T03:18:27.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.682+0000 7f632d7c7b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d65cf71b0 con 0x561d65ce6ad0 2023-07-18T03:18:27.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.682+0000 7f63137fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561d65cdbbe0 0x561d65ce36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.682+0000 7f6312ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d65ce6ad0 0x561d65ce6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.682+0000 7f6313fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d65ce6540 0x561d65ce0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.682+0000 7f6312ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d65ce6ad0 0x561d65ce6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:35620/0 (socket says 172.21.15.101:35620) 2023-07-18T03:18:27.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.682+0000 7f6312ffd700 1 -- 172.21.15.101:0/221157743 learned_addr learned my addr 172.21.15.101:0/221157743 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:18:27.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.683+0000 7f6312ffd700 1 -- 172.21.15.101:0/221157743 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d65ce6540 msgr2=0x561d65ce0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:27.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.683+0000 7f6312ffd700 1 --2- 172.21.15.101:0/221157743 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d65ce6540 0x561d65ce0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.683+0000 7f6312ffd700 1 -- 172.21.15.101:0/221157743 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561d65cdbbe0 msgr2=0x561d65ce36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:27.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.683+0000 7f6312ffd700 1 --2- 172.21.15.101:0/221157743 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561d65cdbbe0 0x561d65ce36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.683+0000 7f6312ffd700 1 -- 172.21.15.101:0/221157743 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d65a82250 con 0x561d65ce6ad0 2023-07-18T03:18:27.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.683+0000 7f6312ffd700 1 --2- 172.21.15.101:0/221157743 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d65ce6ad0 0x561d65ce6000 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6308014d10 tx=0x7f63080073d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f185df1fd1e3a921 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:18:27.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.683+0000 7f62fa7fc700 1 -- 172.21.15.101:0/221157743 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6308026040 con 0x561d65ce6ad0 2023-07-18T03:18:27.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.683+0000 7f62fa7fc700 1 -- 172.21.15.101:0/221157743 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f630801a070 con 0x561d65ce6ad0 2023-07-18T03:18:27.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.683+0000 7f62fa7fc700 1 -- 172.21.15.101:0/221157743 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63080045f0 con 0x561d65ce6ad0 2023-07-18T03:18:27.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.684+0000 7f632d7c7b80 1 -- 172.21.15.101:0/221157743 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d65ce6ad0 msgr2=0x561d65ce6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:27.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.684+0000 7f632d7c7b80 1 --2- 172.21.15.101:0/221157743 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d65ce6ad0 0x561d65ce6000 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6308014d10 tx=0x7f63080073d0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.684+0000 7f632d7c7b80 1 -- 172.21.15.101:0/221157743 shutdown_connections 2023-07-18T03:18:27.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.684+0000 7f632d7c7b80 1 --2- 172.21.15.101:0/221157743 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d65ce6540 0x561d65ce0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.684+0000 7f632d7c7b80 1 --2- 172.21.15.101:0/221157743 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561d65cdbbe0 0x561d65ce36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.684+0000 7f632d7c7b80 1 --2- 172.21.15.101:0/221157743 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d65ce6ad0 0x561d65ce6000 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.684+0000 7f632d7c7b80 1 -- 172.21.15.101:0/221157743 >> 172.21.15.101:0/221157743 conn(0x561d65cd1da0 msgr2=0x561d65ce2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:18:27.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.684+0000 7f632d7c7b80 1 -- 172.21.15.101:0/221157743 shutdown_connections 2023-07-18T03:18:27.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.684+0000 7f632d7c7b80 1 -- 172.21.15.101:0/221157743 wait complete. 2023-07-18T03:18:27.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.684+0000 7f632d7c7b80 1 -- start start 2023-07-18T03:18:27.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.685+0000 7f632d7c7b80 1 Processor -- start 2023-07-18T03:18:27.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.685+0000 7f632d7c7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d65cdbbe0 0x561d65cf1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.685+0000 7f632d7c7b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d65ce6540 0x561d65cefd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.685+0000 7f632d7c7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561d65ce6ad0 0x561d65cf0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.685+0000 7f632d7c7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d65cf09a0 con 0x561d65ce6ad0 2023-07-18T03:18:27.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.685+0000 7f632d7c7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d65cf0b10 con 0x561d65cdbbe0 2023-07-18T03:18:27.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.685+0000 7f632d7c7b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d65d4e180 con 0x561d65ce6540 2023-07-18T03:18:27.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.685+0000 7f632d7c7b80 10 client.0 _ll_register_callbacks cb 0x7f630400bef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:18:27.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.685+0000 7f63137fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d65cdbbe0 0x561d65cf1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.685+0000 7f6313fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561d65ce6ad0 0x561d65cf0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.690 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.685+0000 7f63137fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d65cdbbe0 0x561d65cf1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:39080/0 (socket says 172.21.15.101:39080) 2023-07-18T03:18:27.690 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.685+0000 7f63137fe700 1 -- 172.21.15.101:0/1884233819 learned_addr learned my addr 172.21.15.101:0/1884233819 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:18:27.690 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.685+0000 7f6312ffd700 1 --2- 172.21.15.101:0/1884233819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d65ce6540 0x561d65cefd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.686+0000 7f63137fe700 1 -- 172.21.15.101:0/1884233819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d65ce6540 msgr2=0x561d65cefd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:27.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.686+0000 7f63137fe700 1 --2- 172.21.15.101:0/1884233819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d65ce6540 0x561d65cefd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.686+0000 7f63137fe700 1 -- 172.21.15.101:0/1884233819 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561d65ce6ad0 msgr2=0x561d65cf0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:27.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.686+0000 7f63137fe700 1 --2- 172.21.15.101:0/1884233819 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561d65ce6ad0 0x561d65cf0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.686+0000 7f63137fe700 1 -- 172.21.15.101:0/1884233819 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63080140c0 con 0x561d65cdbbe0 2023-07-18T03:18:27.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.686+0000 7f6313fff700 1 --2- 172.21.15.101:0/1884233819 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561d65ce6ad0 0x561d65cf0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:18:27.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.686+0000 7f63137fe700 1 --2- 172.21.15.101:0/1884233819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d65cdbbe0 0x561d65cf1960 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6304021540 tx=0x7f630400edd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c033128bb1595d5 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:18:27.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.686+0000 7f62f2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:18:27.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.686+0000 7f62f2ffd700 1 -- 172.21.15.101:0/1884233819 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6304005920 con 0x561d65cdbbe0 2023-07-18T03:18:27.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.686+0000 7f632d7c7b80 1 -- 172.21.15.101:0/1884233819 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f630401d040 con 0x561d65cdbbe0 2023-07-18T03:18:27.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.686+0000 7f62f2ffd700 1 -- 172.21.15.101:0/1884233819 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f6304000e40 con 0x561d65cdbbe0 2023-07-18T03:18:27.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.687+0000 7f62f2ffd700 1 -- 172.21.15.101:0/1884233819 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63040217b0 con 0x561d65cdbbe0 2023-07-18T03:18:27.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.687+0000 7f632d7c7b80 10 client.27797 fetch_fsmap learned FSMap version 33 2023-07-18T03:18:27.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.687+0000 7f632d7c7b80 1 -- 172.21.15.101:0/1884233819 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x561d65a5a2b0 con 0x561d65cdbbe0 2023-07-18T03:18:27.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.687+0000 7f62f2ffd700 1 -- 172.21.15.101:0/1884233819 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f630401d040 con 0x561d65cdbbe0 2023-07-18T03:18:27.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.687+0000 7f62f2ffd700 1 -- 172.21.15.101:0/1884233819 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6304015070 con 0x561d65cdbbe0 2023-07-18T03:18:27.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.687+0000 7f632d7c7b80 10 client.27797 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:18:27.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.687+0000 7f632d7c7b80 10 client.27797 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:18:27.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.687+0000 7f632d7c7b80 20 client.27797 populate_metadata read hostname 'smithi101' 2023-07-18T03:18:27.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.687+0000 7f632d7c7b80 1 --2- 172.21.15.101:0/1884233819 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x561d65d4f380 0x561d65d51790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:27.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.687+0000 7f632d7c7b80 4 client.27797 mds_command: new command op to 24562 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:18:27.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.687+0000 7f632d7c7b80 1 -- 172.21.15.101:0/1884233819 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561d65be9050 con 0x561d65d4f380 2023-07-18T03:18:27.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.687+0000 7f6312ffd700 1 --2- 172.21.15.101:0/1884233819 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x561d65d4f380 0x561d65d51790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:27.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.688+0000 7f6312ffd700 1 --2- 172.21.15.101:0/1884233819 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x561d65d4f380 0x561d65d51790 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=712d56fbea9fd52e server_cookie=510f79ba59c6c4c in_seq=0 out_seq=0 2023-07-18T03:18:27.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.688+0000 7f62f2ffd700 10 client.27797 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:18:27.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.688+0000 7f62f2ffd700 1 -- 172.21.15.101:0/1884233819 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561d65be9050 con 0x561d65d4f380 2023-07-18T03:18:27.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.688+0000 7f62f2ffd700 10 client.27797 handle_command_reply: tid=0 2023-07-18T03:18:27.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.688+0000 7f632d7c7b80 1 client.27797 shutdown 2023-07-18T03:18:27.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.688+0000 7f632d7c7b80 1 -- 172.21.15.101:0/1884233819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d65cdbbe0 msgr2=0x561d65cf1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:27.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.688+0000 7f632d7c7b80 1 --2- 172.21.15.101:0/1884233819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d65cdbbe0 0x561d65cf1960 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6304021540 tx=0x7f630400edd0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.688+0000 7f632d7c7b80 1 -- 172.21.15.101:0/1884233819 shutdown_connections 2023-07-18T03:18:27.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.688+0000 7f632d7c7b80 1 --2- 172.21.15.101:0/1884233819 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x561d65d4f380 0x561d65d51790 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.688+0000 7f632d7c7b80 1 --2- 172.21.15.101:0/1884233819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d65cdbbe0 0x561d65cf1960 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.688+0000 7f632d7c7b80 1 --2- 172.21.15.101:0/1884233819 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561d65ce6ad0 0x561d65cf0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.688+0000 7f632d7c7b80 1 --2- 172.21.15.101:0/1884233819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d65ce6540 0x561d65cefd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.688+0000 7f62f2ffd700 0 client.27797 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:18:27.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.688+0000 7f632d7c7b80 1 -- 172.21.15.101:0/1884233819 >> 172.21.15.101:0/1884233819 conn(0x561d65cd1da0 msgr2=0x561d65ce1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:18:27.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.689+0000 7f632d7c7b80 1 -- 172.21.15.101:0/1884233819 shutdown_connections 2023-07-18T03:18:27.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.689+0000 7f632d7c7b80 1 -- 172.21.15.101:0/1884233819 wait complete. 2023-07-18T03:18:27.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.689+0000 7f632d7c7b80 20 client.27797 trim_cache size 0 max 16384 2023-07-18T03:18:27.703 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:18:27.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.689+0000 7f6319dff700 1 -- 172.21.15.101:0/1037294692 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f62fc0a12c0 msgr2=0x7f62fc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:27.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.689+0000 7f6319dff700 1 --2- 172.21.15.101:0/1037294692 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f62fc0a12c0 0x7f62fc0a3770 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f63140fdef0 tx=0x7f6308014910 comp rx=0 tx=0).stop 2023-07-18T03:18:27.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.689+0000 7f6319dff700 1 -- 172.21.15.101:0/1037294692 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6314105ae0 msgr2=0x7f6314195a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:27.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.689+0000 7f6319dff700 1 --2- 172.21.15.101:0/1037294692 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6314105ae0 0x7f6314195a90 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f630c00c110 tx=0x7f630c00c420 comp rx=0 tx=0).stop 2023-07-18T03:18:27.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.689+0000 7f6319dff700 1 -- 172.21.15.101:0/1037294692 shutdown_connections 2023-07-18T03:18:27.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.689+0000 7f6319dff700 1 --2- 172.21.15.101:0/1037294692 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f62fc0a12c0 0x7f62fc0a3770 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.689+0000 7f6319dff700 1 --2- 172.21.15.101:0/1037294692 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63140ffad0 0x7f631419ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.689+0000 7f6319dff700 1 --2- 172.21.15.101:0/1037294692 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6314105ae0 0x7f6314195a90 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.689+0000 7f6319dff700 1 --2- 172.21.15.101:0/1037294692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6314100480 0x7f631419bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:27.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.689+0000 7f6319dff700 1 -- 172.21.15.101:0/1037294692 >> 172.21.15.101:0/1037294692 conn(0x7f63140fb5f0 msgr2=0x7f63140fbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:18:27.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.689+0000 7f6319dff700 1 -- 172.21.15.101:0/1037294692 shutdown_connections 2023-07-18T03:18:27.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:27.689+0000 7f6319dff700 1 -- 172.21.15.101:0/1037294692 wait complete. 2023-07-18T03:18:28.620 INFO:tasks.workunit.client.0.smithi101.stdout: 1 672856 5.64 MB/sec execute 473 sec latency 1576.806 ms 2023-07-18T03:18:28.710 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-18T03:18:28.710 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-18T03:18:28.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.919+0000 7fc2f7bff700 1 -- 172.21.15.101:0/3694489820 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc2f0105af0 msgr2=0x7fc2f0105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:28.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.919+0000 7fc2f7bff700 1 --2- 172.21.15.101:0/3694489820 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc2f0105af0 0x7fc2f0105ec0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec009990 tx=0x7fc2ec009ca0 comp rx=0 tx=0).stop 2023-07-18T03:18:28.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.919+0000 7fc2f7bff700 1 -- 172.21.15.101:0/3694489820 shutdown_connections 2023-07-18T03:18:28.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.919+0000 7fc2f7bff700 1 --2- 172.21.15.101:0/3694489820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2f0100490 0x7fc2f010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:28.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.919+0000 7fc2f7bff700 1 --2- 172.21.15.101:0/3694489820 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc2f0105af0 0x7fc2f0105ec0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:28.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.919+0000 7fc2f7bff700 1 --2- 172.21.15.101:0/3694489820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2f00ffae0 0x7fc2f00fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:28.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.919+0000 7fc2f7bff700 1 -- 172.21.15.101:0/3694489820 >> 172.21.15.101:0/3694489820 conn(0x7fc2f00fb5e0 msgr2=0x7fc2f00fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:18:28.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.919+0000 7fc2f7bff700 1 -- 172.21.15.101:0/3694489820 shutdown_connections 2023-07-18T03:18:28.916 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.919+0000 7fc2f7bff700 1 -- 172.21.15.101:0/3694489820 wait complete. 2023-07-18T03:18:28.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.920+0000 7fc2f7bff700 1 Processor -- start 2023-07-18T03:18:28.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.920+0000 7fc2f7bff700 1 -- start start 2023-07-18T03:18:28.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.920+0000 7fc2f7bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc2f00ffae0 0x7fc2f019b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:28.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.920+0000 7fc2f7bff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2f0100490 0x7fc2f019bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:28.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.920+0000 7fc2f7bff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2f0105af0 0x7fc2f0195a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:28.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.920+0000 7fc2f7bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f01960b0 con 0x7fc2f00ffae0 2023-07-18T03:18:28.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.920+0000 7fc2f7bff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f0196220 con 0x7fc2f0105af0 2023-07-18T03:18:28.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.920+0000 7fc2f7bff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f01963e0 con 0x7fc2f0100490 2023-07-18T03:18:28.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.921+0000 7fc2f599b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc2f00ffae0 0x7fc2f019b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:28.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.921+0000 7fc2f599b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc2f00ffae0 0x7fc2f019b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:42442/0 (socket says 172.21.15.101:42442) 2023-07-18T03:18:28.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.921+0000 7fc2f599b700 1 -- 172.21.15.101:0/3265365594 learned_addr learned my addr 172.21.15.101:0/3265365594 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:18:28.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.921+0000 7fc2f519a700 1 --2- 172.21.15.101:0/3265365594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2f0100490 0x7fc2f019bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:28.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.921+0000 7fc2f619c700 1 --2- 172.21.15.101:0/3265365594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2f0105af0 0x7fc2f0195a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:28.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.921+0000 7fc2f599b700 1 -- 172.21.15.101:0/3265365594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2f0100490 msgr2=0x7fc2f019bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:28.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.921+0000 7fc2f599b700 1 --2- 172.21.15.101:0/3265365594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2f0100490 0x7fc2f019bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:28.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.921+0000 7fc2f599b700 1 -- 172.21.15.101:0/3265365594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2f0105af0 msgr2=0x7fc2f0195a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:28.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.921+0000 7fc2f599b700 1 --2- 172.21.15.101:0/3265365594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2f0105af0 0x7fc2f0195a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:28.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.921+0000 7fc2f599b700 1 -- 172.21.15.101:0/3265365594 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2ec0095f0 con 0x7fc2f00ffae0 2023-07-18T03:18:28.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.921+0000 7fc2f619c700 1 --2- 172.21.15.101:0/3265365594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2f0105af0 0x7fc2f0195a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:18:28.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.921+0000 7fc2f599b700 1 --2- 172.21.15.101:0/3265365594 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc2f00ffae0 0x7fc2f019b9d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec005910 tx=0x7fc2ec005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:18:28.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.921+0000 7fc2e6ffd700 1 -- 172.21.15.101:0/3265365594 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec01d070 con 0x7fc2f00ffae0 2023-07-18T03:18:28.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.922+0000 7fc2f7bff700 1 -- 172.21.15.101:0/3265365594 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2f0196660 con 0x7fc2f00ffae0 2023-07-18T03:18:28.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.922+0000 7fc2e6ffd700 1 -- 172.21.15.101:0/3265365594 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2ec022470 con 0x7fc2f00ffae0 2023-07-18T03:18:28.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.922+0000 7fc2e6ffd700 1 -- 172.21.15.101:0/3265365594 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec017650 con 0x7fc2f00ffae0 2023-07-18T03:18:28.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.922+0000 7fc2f7bff700 1 -- 172.21.15.101:0/3265365594 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2f01a29c0 con 0x7fc2f00ffae0 2023-07-18T03:18:28.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.926+0000 7fc2e6ffd700 1 -- 172.21.15.101:0/3265365594 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fc2ec0225e0 con 0x7fc2f00ffae0 2023-07-18T03:18:28.922 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.927+0000 7fc30b461b80 1 Processor -- start 2023-07-18T03:18:28.922 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.927+0000 7fc30b461b80 1 -- start start 2023-07-18T03:18:28.924 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.928+0000 7fc30b461b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56466bfc1350 0x56466bfc1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:28.924 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.928+0000 7fc30b461b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56466bfc1c60 0x56466bfd0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:28.925 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.928+0000 7fc30b461b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56466bfd0f60 0x56466bfd3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:28.925 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.928+0000 7fc30b461b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56466bec7d30 con 0x56466bfc1c60 2023-07-18T03:18:28.925 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.928+0000 7fc30b461b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56466bf1aac0 con 0x56466bfd0f60 2023-07-18T03:18:28.925 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.928+0000 7fc30b461b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56466bf13f50 con 0x56466bfc1350 2023-07-18T03:18:28.925 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.928+0000 7fc2e6ffd700 1 --2- 172.21.15.101:0/3265365594 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc2dc0a1340 0x7fc2dc0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:28.925 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.928+0000 7fc2f599b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56466bfc1c60 0x56466bfd0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:28.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.928+0000 7fc2e6ffd700 1 -- 172.21.15.101:0/3265365594 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fc2ec0ce470 con 0x7fc2f00ffae0 2023-07-18T03:18:28.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.928+0000 7fc2f619c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56466bfc1350 0x56466bfc1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:28.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.928+0000 7fc2f599b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56466bfc1c60 0x56466bfd0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:42456/0 (socket says 172.21.15.101:42456) 2023-07-18T03:18:28.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.928+0000 7fc2f599b700 1 -- 172.21.15.101:0/1916680782 learned_addr learned my addr 172.21.15.101:0/1916680782 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:18:28.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.928+0000 7fc2f519a700 1 --2- 172.21.15.101:0/1916680782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56466bfd0f60 0x56466bfd3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:28.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.928+0000 7fc2f619c700 1 --2- 172.21.15.101:0/3265365594 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc2dc0a1340 0x7fc2dc0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:28.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.929+0000 7fc2f599b700 1 -- 172.21.15.101:0/1916680782 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56466bfc1350 msgr2=0x56466bfc1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:28.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.929+0000 7fc2f599b700 1 --2- 172.21.15.101:0/1916680782 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56466bfc1350 0x56466bfc1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:28.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.929+0000 7fc2f599b700 1 -- 172.21.15.101:0/1916680782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56466bfd0f60 msgr2=0x56466bfd3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:28.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.929+0000 7fc2f599b700 1 --2- 172.21.15.101:0/1916680782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56466bfd0f60 0x56466bfd3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:28.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.929+0000 7fc2f599b700 1 -- 172.21.15.101:0/1916680782 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56466bd41e60 con 0x56466bfc1c60 2023-07-18T03:18:28.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.929+0000 7fc2f599b700 1 --2- 172.21.15.101:0/1916680782 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56466bfc1c60 0x56466bfd0a20 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec0976c0 tx=0x7fc2ec01fee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4c38b3930c82644 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:18:28.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.929+0000 7fc2f619c700 1 --2- 172.21.15.101:0/3265365594 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc2dc0a1340 0x7fc2dc0a37f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8009fd0 tx=0x7fc2e800b040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:18:28.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.929+0000 7fc2dbfff700 1 -- 172.21.15.101:0/1916680782 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec097500 con 0x56466bfc1c60 2023-07-18T03:18:28.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.929+0000 7fc2dbfff700 1 -- 172.21.15.101:0/1916680782 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2ec09af90 con 0x56466bfc1c60 2023-07-18T03:18:28.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.929+0000 7fc2dbfff700 1 -- 172.21.15.101:0/1916680782 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec09f260 con 0x56466bfc1c60 2023-07-18T03:18:28.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.930+0000 7fc30b461b80 1 -- 172.21.15.101:0/1916680782 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56466bfc1c60 msgr2=0x56466bfd0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:28.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.930+0000 7fc30b461b80 1 --2- 172.21.15.101:0/1916680782 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56466bfc1c60 0x56466bfd0a20 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec0976c0 tx=0x7fc2ec01fee0 comp rx=0 tx=0).stop 2023-07-18T03:18:28.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.930+0000 7fc30b461b80 1 -- 172.21.15.101:0/1916680782 shutdown_connections 2023-07-18T03:18:28.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.930+0000 7fc30b461b80 1 --2- 172.21.15.101:0/1916680782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56466bfd0f60 0x56466bfd3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:28.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.930+0000 7fc30b461b80 1 --2- 172.21.15.101:0/1916680782 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56466bfc1350 0x56466bfc1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:28.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.930+0000 7fc30b461b80 1 --2- 172.21.15.101:0/1916680782 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56466bfc1c60 0x56466bfd0a20 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:28.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.930+0000 7fc30b461b80 1 -- 172.21.15.101:0/1916680782 >> 172.21.15.101:0/1916680782 conn(0x56466bfb7e20 msgr2=0x56466bfbd720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:18:28.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.930+0000 7fc30b461b80 1 -- 172.21.15.101:0/1916680782 shutdown_connections 2023-07-18T03:18:28.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.930+0000 7fc30b461b80 1 -- 172.21.15.101:0/1916680782 wait complete. 2023-07-18T03:18:28.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.931+0000 7fc30b461b80 1 -- start start 2023-07-18T03:18:28.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.931+0000 7fc30b461b80 1 Processor -- start 2023-07-18T03:18:28.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.931+0000 7fc30b461b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56466bfc1350 0x56466bfd6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:28.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.931+0000 7fc30b461b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56466bfc1c60 0x56466bfd6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:28.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.931+0000 7fc30b461b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56466bfd0f60 0x56466bfcff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:28.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.931+0000 7fc30b461b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56466beb3b40 con 0x56466bfd0f60 2023-07-18T03:18:28.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.931+0000 7fc30b461b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56466beb3840 con 0x56466bfc1350 2023-07-18T03:18:28.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.931+0000 7fc30b461b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56466bd979a0 con 0x56466bfc1c60 2023-07-18T03:18:28.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.931+0000 7fc30b461b80 10 client.0 _ll_register_callbacks cb 0x56466beb17e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:18:28.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.932+0000 7fc2f619c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56466bfd0f60 0x56466bfcff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:28.934 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.932+0000 7fc2f619c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56466bfd0f60 0x56466bfcff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:42472/0 (socket says 172.21.15.101:42472) 2023-07-18T03:18:28.934 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.932+0000 7fc2f619c700 1 -- 172.21.15.101:0/1109131715 learned_addr learned my addr 172.21.15.101:0/1109131715 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:18:28.934 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.932+0000 7fc2f599b700 1 --2- 172.21.15.101:0/1109131715 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56466bfc1350 0x56466bfd6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:28.934 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.932+0000 7fc2f519a700 1 --2- 172.21.15.101:0/1109131715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56466bfc1c60 0x56466bfd6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:28.934 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.932+0000 7fc2f619c700 1 -- 172.21.15.101:0/1109131715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56466bfc1c60 msgr2=0x56466bfd6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:28.934 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.932+0000 7fc2f619c700 1 --2- 172.21.15.101:0/1109131715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56466bfc1c60 0x56466bfd6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:28.934 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.932+0000 7fc2f619c700 1 -- 172.21.15.101:0/1109131715 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56466bfc1350 msgr2=0x56466bfd6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:28.935 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.932+0000 7fc2f619c700 1 --2- 172.21.15.101:0/1109131715 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56466bfc1350 0x56466bfd6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:28.935 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.932+0000 7fc2f619c700 1 -- 172.21.15.101:0/1109131715 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2f01a29c0 con 0x56466bfd0f60 2023-07-18T03:18:28.935 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.932+0000 7fc2f619c700 1 --2- 172.21.15.101:0/1109131715 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56466bfd0f60 0x56466bfcff00 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8012010 tx=0x7fc2e8004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf01511c82e66c90 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:18:28.935 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.932+0000 7fc2d97fa700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:18:28.935 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.933+0000 7fc2d97fa700 1 -- 172.21.15.101:0/1109131715 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e8020070 con 0x56466bfd0f60 2023-07-18T03:18:28.935 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.933+0000 7fc30b461b80 1 -- 172.21.15.101:0/1109131715 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56466be6d570 con 0x56466bfd0f60 2023-07-18T03:18:28.935 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.933+0000 7fc2d97fa700 1 -- 172.21.15.101:0/1109131715 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fc2e8007500 con 0x56466bfd0f60 2023-07-18T03:18:28.935 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.933+0000 7fc2d97fa700 1 -- 172.21.15.101:0/1109131715 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2e80093d0 con 0x56466bfd0f60 2023-07-18T03:18:28.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.933+0000 7fc30b461b80 10 client.17772 fetch_fsmap learned FSMap version 33 2023-07-18T03:18:28.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.933+0000 7fc30b461b80 1 -- 172.21.15.101:0/1109131715 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x56466bd2ace0 con 0x56466bfd0f60 2023-07-18T03:18:28.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.933+0000 7fc2d97fa700 1 -- 172.21.15.101:0/1109131715 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56466be6d570 con 0x56466bfd0f60 2023-07-18T03:18:28.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.933+0000 7fc2d97fa700 1 -- 172.21.15.101:0/1109131715 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc2e80258e0 con 0x56466bfd0f60 2023-07-18T03:18:28.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.933+0000 7fc30b461b80 10 client.17772 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:18:28.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.933+0000 7fc30b461b80 10 client.17772 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:18:28.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.933+0000 7fc30b461b80 20 client.17772 populate_metadata read hostname 'smithi101' 2023-07-18T03:18:28.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.933+0000 7fc30b461b80 1 --2- 172.21.15.101:0/1109131715 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56466bfe05e0 0x56466bfe29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:28.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.933+0000 7fc30b461b80 4 client.17772 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:18:28.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.933+0000 7fc30b461b80 1 -- 172.21.15.101:0/1109131715 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56466bf0cf30 con 0x56466bfe05e0 2023-07-18T03:18:28.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.934+0000 7fc2f599b700 1 --2- 172.21.15.101:0/1109131715 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56466bfe05e0 0x56466bfe29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:28.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.934+0000 7fc2d97fa700 10 client.17772 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:18:28.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.934+0000 7fc2f599b700 1 --2- 172.21.15.101:0/1109131715 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56466bfe05e0 0x56466bfe29f0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8e60870c8d36094 server_cookie=b616b204bfaacc6a in_seq=0 out_seq=0 2023-07-18T03:18:28.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.935+0000 7fc2d97fa700 1 -- 172.21.15.101:0/1109131715 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56466bf0cf30 con 0x56466bfe05e0 2023-07-18T03:18:28.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.935+0000 7fc2d97fa700 10 client.17772 handle_command_reply: tid=0 2023-07-18T03:18:28.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.935+0000 7fc30b461b80 1 client.17772 shutdown 2023-07-18T03:18:28.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.935+0000 7fc30b461b80 1 -- 172.21.15.101:0/1109131715 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56466bfd0f60 msgr2=0x56466bfcff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:28.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.935+0000 7fc30b461b80 1 --2- 172.21.15.101:0/1109131715 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56466bfd0f60 0x56466bfcff00 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8012010 tx=0x7fc2e8004b40 comp rx=0 tx=0).stop 2023-07-18T03:18:28.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.935+0000 7fc30b461b80 1 -- 172.21.15.101:0/1109131715 shutdown_connections 2023-07-18T03:18:28.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.935+0000 7fc30b461b80 1 --2- 172.21.15.101:0/1109131715 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56466bfe05e0 0x56466bfe29f0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:28.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.935+0000 7fc30b461b80 1 --2- 172.21.15.101:0/1109131715 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56466bfc1350 0x56466bfd6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:28.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.935+0000 7fc30b461b80 1 --2- 172.21.15.101:0/1109131715 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56466bfd0f60 0x56466bfcff00 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:28.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.935+0000 7fc30b461b80 1 --2- 172.21.15.101:0/1109131715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56466bfc1c60 0x56466bfd6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:28.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.935+0000 7fc2d97fa700 0 client.17772 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:18:28.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.935+0000 7fc30b461b80 1 -- 172.21.15.101:0/1109131715 >> 172.21.15.101:0/1109131715 conn(0x56466bfb7e20 msgr2=0x56466bfbd310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:18:28.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.936+0000 7fc30b461b80 1 -- 172.21.15.101:0/1109131715 shutdown_connections 2023-07-18T03:18:28.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.936+0000 7fc30b461b80 1 -- 172.21.15.101:0/1109131715 wait complete. 2023-07-18T03:18:28.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.936+0000 7fc30b461b80 20 client.17772 trim_cache size 0 max 16384 2023-07-18T03:18:28.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.953+0000 7fc30b461b80 1 Processor -- start 2023-07-18T03:18:28.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.953+0000 7fc30b461b80 1 -- start start 2023-07-18T03:18:28.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.953+0000 7fc30b461b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56466bfc1350 0x56466bfcef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:28.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.953+0000 7fc30b461b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56466bfc1c60 0x56466bfd5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:28.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.953+0000 7fc30b461b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56466bfd0f60 0x56466bfd6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:28.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.953+0000 7fc30b461b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56466bfc2ef0 con 0x56466bfc1c60 2023-07-18T03:18:28.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.953+0000 7fc30b461b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56466be6d570 con 0x56466bfc1350 2023-07-18T03:18:28.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.953+0000 7fc30b461b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56466bfd1460 con 0x56466bfd0f60 2023-07-18T03:18:28.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.954+0000 7fc2f519a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56466bfc1c60 0x56466bfd5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:28.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.954+0000 7fc2f519a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56466bfc1c60 0x56466bfd5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:42476/0 (socket says 172.21.15.101:42476) 2023-07-18T03:18:28.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.954+0000 7fc2f599b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56466bfc1350 0x56466bfcef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:28.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.954+0000 7fc2f519a700 1 -- 172.21.15.101:0/2295352655 learned_addr learned my addr 172.21.15.101:0/2295352655 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:18:28.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.954+0000 7fc2f619c700 1 --2- 172.21.15.101:0/2295352655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56466bfd0f60 0x56466bfd6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:28.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.954+0000 7fc2f519a700 1 -- 172.21.15.101:0/2295352655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56466bfd0f60 msgr2=0x56466bfd6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:28.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.954+0000 7fc2f519a700 1 --2- 172.21.15.101:0/2295352655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56466bfd0f60 0x56466bfd6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:28.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.954+0000 7fc2f519a700 1 -- 172.21.15.101:0/2295352655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56466bfc1350 msgr2=0x56466bfcef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:28.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.954+0000 7fc2f519a700 1 --2- 172.21.15.101:0/2295352655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56466bfc1350 0x56466bfcef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:28.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.954+0000 7fc2f519a700 1 -- 172.21.15.101:0/2295352655 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56466bfd0b60 con 0x56466bfc1c60 2023-07-18T03:18:28.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.954+0000 7fc2f599b700 1 --2- 172.21.15.101:0/2295352655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56466bfc1350 0x56466bfcef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:18:28.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.954+0000 7fc2f519a700 1 --2- 172.21.15.101:0/2295352655 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56466bfc1c60 0x56466bfd5c10 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0009e70 tx=0x7fc2e0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e47eac58def9d23 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:18:28.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.954+0000 7fc2e4bff700 1 -- 172.21.15.101:0/2295352655 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e0017070 con 0x56466bfc1c60 2023-07-18T03:18:28.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.955+0000 7fc2e4bff700 1 -- 172.21.15.101:0/2295352655 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2e0018040 con 0x56466bfc1c60 2023-07-18T03:18:28.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.955+0000 7fc2e4bff700 1 -- 172.21.15.101:0/2295352655 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e0003aa0 con 0x56466bfc1c60 2023-07-18T03:18:28.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.955+0000 7fc30b461b80 1 -- 172.21.15.101:0/2295352655 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56466bfc1c60 msgr2=0x56466bfd5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:28.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.955+0000 7fc30b461b80 1 --2- 172.21.15.101:0/2295352655 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56466bfc1c60 0x56466bfd5c10 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0009e70 tx=0x7fc2e0009260 comp rx=0 tx=0).stop 2023-07-18T03:18:28.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.955+0000 7fc30b461b80 1 -- 172.21.15.101:0/2295352655 shutdown_connections 2023-07-18T03:18:28.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.955+0000 7fc30b461b80 1 --2- 172.21.15.101:0/2295352655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56466bfc1350 0x56466bfcef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:28.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.955+0000 7fc30b461b80 1 --2- 172.21.15.101:0/2295352655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56466bfd0f60 0x56466bfd6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:28.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.955+0000 7fc30b461b80 1 --2- 172.21.15.101:0/2295352655 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56466bfc1c60 0x56466bfd5c10 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:28.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.955+0000 7fc30b461b80 1 -- 172.21.15.101:0/2295352655 >> 172.21.15.101:0/2295352655 conn(0x56466bfb7e20 msgr2=0x56466bfcfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:18:28.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.955+0000 7fc30b461b80 1 -- 172.21.15.101:0/2295352655 shutdown_connections 2023-07-18T03:18:28.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.955+0000 7fc30b461b80 1 -- 172.21.15.101:0/2295352655 wait complete. 2023-07-18T03:18:28.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.956+0000 7fc30b461b80 1 -- start start 2023-07-18T03:18:28.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.956+0000 7fc30b461b80 1 Processor -- start 2023-07-18T03:18:28.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.956+0000 7fc30b461b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56466bfc1350 0x56466c02eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:28.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.956+0000 7fc30b461b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56466bfc1c60 0x56466c02f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:28.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.956+0000 7fc30b461b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56466bfd0f60 0x56466c028e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:28.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.956+0000 7fc30b461b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56466c029420 con 0x56466bfc1350 2023-07-18T03:18:28.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.956+0000 7fc30b461b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56466c029590 con 0x56466bfd0f60 2023-07-18T03:18:28.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.956+0000 7fc30b461b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56466c029750 con 0x56466bfc1c60 2023-07-18T03:18:28.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.956+0000 7fc30b461b80 10 client.0 _ll_register_callbacks cb 0x56466beb17e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:18:28.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.957+0000 7fc2f599b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56466bfc1350 0x56466c02eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:28.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.957+0000 7fc2f599b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56466bfc1350 0x56466c02eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:42480/0 (socket says 172.21.15.101:42480) 2023-07-18T03:18:28.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.957+0000 7fc2f599b700 1 -- 172.21.15.101:0/4029638657 learned_addr learned my addr 172.21.15.101:0/4029638657 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:18:28.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.957+0000 7fc2f519a700 1 --2- 172.21.15.101:0/4029638657 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56466bfc1c60 0x56466c02f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:28.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.957+0000 7fc2f619c700 1 --2- 172.21.15.101:0/4029638657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56466bfd0f60 0x56466c028e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:28.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.957+0000 7fc2f599b700 1 -- 172.21.15.101:0/4029638657 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56466bfc1c60 msgr2=0x56466c02f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:28.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.957+0000 7fc2f599b700 1 --2- 172.21.15.101:0/4029638657 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56466bfc1c60 0x56466c02f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:28.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.957+0000 7fc2f599b700 1 -- 172.21.15.101:0/4029638657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56466bfd0f60 msgr2=0x56466c028e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:28.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.957+0000 7fc2f599b700 1 --2- 172.21.15.101:0/4029638657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56466bfd0f60 0x56466c028e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:28.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.957+0000 7fc2f599b700 1 -- 172.21.15.101:0/4029638657 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56466bd41e60 con 0x56466bfc1350 2023-07-18T03:18:28.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.957+0000 7fc2f599b700 1 --2- 172.21.15.101:0/4029638657 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56466bfc1350 0x56466c02eae0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec00f740 tx=0x7fc2ec01a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=223f5009cc42a5ac server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:18:28.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.957+0000 7fc2d97fa700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:18:28.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.957+0000 7fc2d97fa700 1 -- 172.21.15.101:0/4029638657 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec09ad00 con 0x56466bfc1350 2023-07-18T03:18:28.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.957+0000 7fc2d97fa700 1 -- 172.21.15.101:0/4029638657 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fc2ec09f1f0 con 0x56466bfc1350 2023-07-18T03:18:28.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.957+0000 7fc30b461b80 1 -- 172.21.15.101:0/4029638657 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc2ec09af90 con 0x56466bfc1350 2023-07-18T03:18:28.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.957+0000 7fc2d97fa700 1 -- 172.21.15.101:0/4029638657 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2ec09ab20 con 0x56466bfc1350 2023-07-18T03:18:28.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.957+0000 7fc30b461b80 10 client.17778 fetch_fsmap learned FSMap version 33 2023-07-18T03:18:28.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.958+0000 7fc30b461b80 1 -- 172.21.15.101:0/4029638657 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7fc2e0009b20 con 0x56466bfc1350 2023-07-18T03:18:28.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.958+0000 7fc2d97fa700 1 -- 172.21.15.101:0/4029638657 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2ec09af90 con 0x56466bfc1350 2023-07-18T03:18:28.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.958+0000 7fc2d97fa700 1 -- 172.21.15.101:0/4029638657 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc2ec0a30f0 con 0x56466bfc1350 2023-07-18T03:18:28.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.958+0000 7fc30b461b80 10 client.17778 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:18:28.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.958+0000 7fc30b461b80 10 client.17778 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:18:28.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.958+0000 7fc30b461b80 20 client.17778 populate_metadata read hostname 'smithi101' 2023-07-18T03:18:28.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.958+0000 7fc30b461b80 1 --2- 172.21.15.101:0/4029638657 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56466c0364b0 0x56466c0388c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:28.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.958+0000 7fc30b461b80 4 client.17778 mds_command: new command op to 24562 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-18T03:18:28.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.958+0000 7fc30b461b80 1 -- 172.21.15.101:0/4029638657 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56466becf0d0 con 0x56466c0364b0 2023-07-18T03:18:28.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.958+0000 7fc2f519a700 1 --2- 172.21.15.101:0/4029638657 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56466c0364b0 0x56466c0388c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:28.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.959+0000 7fc2d97fa700 10 client.17778 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:18:28.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.959+0000 7fc2f519a700 1 --2- 172.21.15.101:0/4029638657 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56466c0364b0 0x56466c0388c0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=354f1430e031a325 server_cookie=6e0457e4063d920 in_seq=0 out_seq=0 2023-07-18T03:18:28.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.960+0000 7fc2d97fa700 1 -- 172.21.15.101:0/4029638657 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56466becf0d0 con 0x56466c0364b0 2023-07-18T03:18:28.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.960+0000 7fc2d97fa700 10 client.17778 handle_command_reply: tid=0 2023-07-18T03:18:28.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.960+0000 7fc30b461b80 1 client.17778 shutdown 2023-07-18T03:18:28.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.960+0000 7fc30b461b80 1 -- 172.21.15.101:0/4029638657 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56466bfc1350 msgr2=0x56466c02eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:28.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.960+0000 7fc30b461b80 1 --2- 172.21.15.101:0/4029638657 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56466bfc1350 0x56466c02eae0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec00f740 tx=0x7fc2ec01a040 comp rx=0 tx=0).stop 2023-07-18T03:18:28.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.960+0000 7fc30b461b80 1 -- 172.21.15.101:0/4029638657 shutdown_connections 2023-07-18T03:18:28.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.960+0000 7fc30b461b80 1 --2- 172.21.15.101:0/4029638657 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56466c0364b0 0x56466c0388c0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:28.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.960+0000 7fc30b461b80 1 --2- 172.21.15.101:0/4029638657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56466bfd0f60 0x56466c028e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:28.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.960+0000 7fc30b461b80 1 --2- 172.21.15.101:0/4029638657 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56466bfc1350 0x56466c02eae0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:28.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.960+0000 7fc30b461b80 1 --2- 172.21.15.101:0/4029638657 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56466bfc1c60 0x56466c02f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:28.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.960+0000 7fc2d97fa700 0 client.17778 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:18:28.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.960+0000 7fc30b461b80 1 -- 172.21.15.101:0/4029638657 >> 172.21.15.101:0/4029638657 conn(0x56466bfb7e20 msgr2=0x56466bfc3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:18:28.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.960+0000 7fc30b461b80 1 -- 172.21.15.101:0/4029638657 shutdown_connections 2023-07-18T03:18:28.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.960+0000 7fc30b461b80 1 -- 172.21.15.101:0/4029638657 wait complete. 2023-07-18T03:18:28.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.961+0000 7fc30b461b80 20 client.17778 trim_cache size 0 max 16384 2023-07-18T03:18:28.964 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:18:28.964 INFO:teuthology.orchestra.run.smithi101.stdout: "return_code": 0, 2023-07-18T03:18:28.964 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_tag": "00f9ad7e-1ca8-49f0-a47c-4a43f0c4ac1a", 2023-07-18T03:18:28.964 INFO:teuthology.orchestra.run.smithi101.stdout: "mode": "asynchronous" 2023-07-18T03:18:28.964 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:18:28.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.961+0000 7fc2f7bff700 1 -- 172.21.15.101:0/3265365594 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc2dc0a1340 msgr2=0x7fc2dc0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:28.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.961+0000 7fc2f7bff700 1 --2- 172.21.15.101:0/3265365594 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc2dc0a1340 0x7fc2dc0a37f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8009fd0 tx=0x7fc2e800b040 comp rx=0 tx=0).stop 2023-07-18T03:18:28.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.961+0000 7fc2f7bff700 1 -- 172.21.15.101:0/3265365594 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc2f00ffae0 msgr2=0x7fc2f019b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:28.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.961+0000 7fc2f7bff700 1 --2- 172.21.15.101:0/3265365594 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc2f00ffae0 0x7fc2f019b9d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec005910 tx=0x7fc2ec005720 comp rx=0 tx=0).stop 2023-07-18T03:18:28.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.961+0000 7fc2f7bff700 1 -- 172.21.15.101:0/3265365594 shutdown_connections 2023-07-18T03:18:28.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.961+0000 7fc2f7bff700 1 --2- 172.21.15.101:0/3265365594 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc2dc0a1340 0x7fc2dc0a37f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:28.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.961+0000 7fc2f7bff700 1 --2- 172.21.15.101:0/3265365594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2f0105af0 0x7fc2f0195a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:28.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.961+0000 7fc2f7bff700 1 --2- 172.21.15.101:0/3265365594 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc2f00ffae0 0x7fc2f019b9d0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:28.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.961+0000 7fc2f7bff700 1 --2- 172.21.15.101:0/3265365594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2f0100490 0x7fc2f019bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:28.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.961+0000 7fc2f7bff700 1 -- 172.21.15.101:0/3265365594 >> 172.21.15.101:0/3265365594 conn(0x7fc2f00fb5e0 msgr2=0x7fc2f00fbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:18:28.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.961+0000 7fc2f7bff700 1 -- 172.21.15.101:0/3265365594 shutdown_connections 2023-07-18T03:18:28.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:28.961+0000 7fc2f7bff700 1 -- 172.21.15.101:0/3265365594 wait complete. 2023-07-18T03:18:28.977 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:18:29.049 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:29 smithi101 ceph-mon[126379]: pgmap v340: 129 pgs: 129 active+clean; 545 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 106 op/s 2023-07-18T03:18:29.205 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.210+0000 7f637bbff700 1 -- 172.21.15.101:0/3803089714 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6374105af0 msgr2=0x7f6374105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:29.206 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.210+0000 7f637bbff700 1 --2- 172.21.15.101:0/3803089714 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6374105af0 0x7f6374105ec0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6370009990 tx=0x7f6370009ca0 comp rx=0 tx=0).stop 2023-07-18T03:18:29.206 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.210+0000 7f637bbff700 1 -- 172.21.15.101:0/3803089714 shutdown_connections 2023-07-18T03:18:29.206 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.210+0000 7f637bbff700 1 --2- 172.21.15.101:0/3803089714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6374100490 0x7f637410c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:29.206 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.210+0000 7f637bbff700 1 --2- 172.21.15.101:0/3803089714 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6374105af0 0x7f6374105ec0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:29.206 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.210+0000 7f637bbff700 1 --2- 172.21.15.101:0/3803089714 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63740ffae0 0x7f63740fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:29.206 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.210+0000 7f637bbff700 1 -- 172.21.15.101:0/3803089714 >> 172.21.15.101:0/3803089714 conn(0x7f63740fb5e0 msgr2=0x7f63740fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:18:29.207 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.210+0000 7f637bbff700 1 -- 172.21.15.101:0/3803089714 shutdown_connections 2023-07-18T03:18:29.207 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.210+0000 7f637bbff700 1 -- 172.21.15.101:0/3803089714 wait complete. 2023-07-18T03:18:29.208 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.211+0000 7f637bbff700 1 Processor -- start 2023-07-18T03:18:29.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.212+0000 7f637bbff700 1 -- start start 2023-07-18T03:18:29.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.212+0000 7f637bbff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63740ffae0 0x7f6374072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:29.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.212+0000 7f637bbff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6374100490 0x7f6374072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:29.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.212+0000 7f637bbff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6374105af0 0x7f63741a3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:29.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.212+0000 7f637bbff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6374073580 con 0x7f63740ffae0 2023-07-18T03:18:29.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.212+0000 7f637bbff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63741a4340 con 0x7f6374100490 2023-07-18T03:18:29.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.212+0000 7f637bbff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63741a4480 con 0x7f6374105af0 2023-07-18T03:18:29.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.213+0000 7f637999b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63740ffae0 0x7f6374072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:29.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.213+0000 7f637a19c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6374105af0 0x7f63741a3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:29.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.213+0000 7f637a19c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6374105af0 0x7f63741a3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:35702/0 (socket says 172.21.15.101:35702) 2023-07-18T03:18:29.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.213+0000 7f637919a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6374100490 0x7f6374072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:29.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.213+0000 7f637a19c700 1 -- 172.21.15.101:0/750560437 learned_addr learned my addr 172.21.15.101:0/750560437 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:18:29.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.213+0000 7f637999b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63740ffae0 0x7f6374072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:42500/0 (socket says 172.21.15.101:42500) 2023-07-18T03:18:29.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.213+0000 7f637919a700 1 -- 172.21.15.101:0/750560437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6374105af0 msgr2=0x7f63741a3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:29.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.213+0000 7f637919a700 1 --2- 172.21.15.101:0/750560437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6374105af0 0x7f63741a3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:29.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.213+0000 7f637919a700 1 -- 172.21.15.101:0/750560437 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63740ffae0 msgr2=0x7f6374072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:29.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.213+0000 7f637919a700 1 --2- 172.21.15.101:0/750560437 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63740ffae0 0x7f6374072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:29.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.213+0000 7f637919a700 1 -- 172.21.15.101:0/750560437 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63700095f0 con 0x7f6374100490 2023-07-18T03:18:29.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.213+0000 7f637a19c700 1 --2- 172.21.15.101:0/750560437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6374105af0 0x7f63741a3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:18:29.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.213+0000 7f637999b700 1 --2- 172.21.15.101:0/750560437 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63740ffae0 0x7f6374072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:18:29.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.213+0000 7f637919a700 1 --2- 172.21.15.101:0/750560437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6374100490 0x7f6374072d50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f636400e9f0 tx=0x7f636400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:18:29.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.214+0000 7f636affd700 1 -- 172.21.15.101:0/750560437 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636400c9c0 con 0x7f6374100490 2023-07-18T03:18:29.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.214+0000 7f637bbff700 1 -- 172.21.15.101:0/750560437 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63741a46e0 con 0x7f6374100490 2023-07-18T03:18:29.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.214+0000 7f636affd700 1 -- 172.21.15.101:0/750560437 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6364010430 con 0x7f6374100490 2023-07-18T03:18:29.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.214+0000 7f636affd700 1 -- 172.21.15.101:0/750560437 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6364018680 con 0x7f6374100490 2023-07-18T03:18:29.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.214+0000 7f637bbff700 1 -- 172.21.15.101:0/750560437 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63741a4c10 con 0x7f6374100490 2023-07-18T03:18:29.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.217+0000 7f636affd700 1 -- 172.21.15.101:0/750560437 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f6364010860 con 0x7f6374100490 2023-07-18T03:18:29.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.218+0000 7f638f551b80 1 Processor -- start 2023-07-18T03:18:29.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.218+0000 7f638f551b80 1 -- start start 2023-07-18T03:18:29.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.219+0000 7f638f551b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639877d22d0 0x5639877d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:29.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.219+0000 7f638f551b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5639877d2be0 0x5639877dd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:29.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.219+0000 7f638f551b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639877ddad0 0x5639877dfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:29.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.219+0000 7f638f551b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639876d8cb0 con 0x5639877d2be0 2023-07-18T03:18:29.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.219+0000 7f638f551b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56398772ba40 con 0x5639877ddad0 2023-07-18T03:18:29.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.219+0000 7f638f551b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563987724ed0 con 0x5639877d22d0 2023-07-18T03:18:29.218 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.219+0000 7f637a19c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639877d22d0 0x5639877d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:29.218 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.219+0000 7f637999b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5639877d2be0 0x5639877dd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:29.218 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.219+0000 7f637a19c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639877d22d0 0x5639877d26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:35708/0 (socket says 172.21.15.101:35708) 2023-07-18T03:18:29.218 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.219+0000 7f637a19c700 1 -- 172.21.15.101:0/129620134 learned_addr learned my addr 172.21.15.101:0/129620134 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:18:29.218 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.219+0000 7f637919a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639877ddad0 0x5639877dfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:29.218 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.219+0000 7f636affd700 1 --2- 172.21.15.101:0/750560437 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f63600a12c0 0x7f63600a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:29.218 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.219+0000 7f637999b700 1 -- 172.21.15.101:0/129620134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639877d22d0 msgr2=0x5639877d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:29.218 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.219+0000 7f637999b700 1 --2- 172.21.15.101:0/129620134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639877d22d0 0x5639877d26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:29.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.219+0000 7f637999b700 1 -- 172.21.15.101:0/129620134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639877ddad0 msgr2=0x5639877dfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:29.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.219+0000 7f637999b700 1 --2- 172.21.15.101:0/129620134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639877ddad0 0x5639877dfeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:29.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.219+0000 7f637999b700 1 -- 172.21.15.101:0/129620134 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563987552de0 con 0x5639877d2be0 2023-07-18T03:18:29.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.220+0000 7f636affd700 1 -- 172.21.15.101:0/750560437 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f6364014070 con 0x7f6374100490 2023-07-18T03:18:29.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.220+0000 7f637a19c700 1 --2- 172.21.15.101:0/750560437 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f63600a12c0 0x7f63600a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:29.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.220+0000 7f637999b700 1 --2- 172.21.15.101:0/129620134 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5639877d2be0 0x5639877dd590 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f6370000c00 tx=0x7f6370005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c38300108cf13f2 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:18:29.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.220+0000 7f635ffff700 1 -- 172.21.15.101:0/129620134 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637001d070 con 0x5639877d2be0 2023-07-18T03:18:29.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.220+0000 7f635ffff700 1 -- 172.21.15.101:0/129620134 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f637001e040 con 0x5639877d2be0 2023-07-18T03:18:29.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.220+0000 7f635ffff700 1 -- 172.21.15.101:0/129620134 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6370004d40 con 0x5639877d2be0 2023-07-18T03:18:29.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.221+0000 7f638f551b80 1 -- 172.21.15.101:0/129620134 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5639877d2be0 msgr2=0x5639877dd590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:29.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.221+0000 7f637a19c700 1 --2- 172.21.15.101:0/750560437 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f63600a12c0 0x7f63600a3770 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f636c0077a0 tx=0x7f636c004ad0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:18:29.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.221+0000 7f638f551b80 1 --2- 172.21.15.101:0/129620134 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5639877d2be0 0x5639877dd590 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f6370000c00 tx=0x7f6370005ee0 comp rx=0 tx=0).stop 2023-07-18T03:18:29.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.221+0000 7f638f551b80 1 -- 172.21.15.101:0/129620134 shutdown_connections 2023-07-18T03:18:29.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.221+0000 7f638f551b80 1 --2- 172.21.15.101:0/129620134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639877ddad0 0x5639877dfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:29.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.221+0000 7f638f551b80 1 --2- 172.21.15.101:0/129620134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639877d22d0 0x5639877d26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:29.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.221+0000 7f638f551b80 1 --2- 172.21.15.101:0/129620134 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5639877d2be0 0x5639877dd590 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:29.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.221+0000 7f638f551b80 1 -- 172.21.15.101:0/129620134 >> 172.21.15.101:0/129620134 conn(0x5639877c8da0 msgr2=0x5639877ce6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:18:29.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.221+0000 7f638f551b80 1 -- 172.21.15.101:0/129620134 shutdown_connections 2023-07-18T03:18:29.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.221+0000 7f638f551b80 1 -- 172.21.15.101:0/129620134 wait complete. 2023-07-18T03:18:29.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.222+0000 7f638f551b80 1 -- start start 2023-07-18T03:18:29.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.222+0000 7f638f551b80 1 Processor -- start 2023-07-18T03:18:29.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.223+0000 7f638f551b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639877d22d0 0x5639877dd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:29.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.223+0000 7f638f551b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639877d2be0 0x5639877da790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:29.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.223+0000 7f638f551b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5639877ddad0 0x5639877daed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:29.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.223+0000 7f638f551b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639876c4ac0 con 0x5639877ddad0 2023-07-18T03:18:29.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.223+0000 7f638f551b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639876c47c0 con 0x5639877d22d0 2023-07-18T03:18:29.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.223+0000 7f638f551b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639875a8920 con 0x5639877d2be0 2023-07-18T03:18:29.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.223+0000 7f638f551b80 10 client.0 _ll_register_callbacks cb 0x5639876c2760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:18:29.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.223+0000 7f637999b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639877d22d0 0x5639877dd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:29.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.223+0000 7f637919a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639877d2be0 0x5639877da790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:29.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.223+0000 7f637999b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639877d22d0 0x5639877dd510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:39166/0 (socket says 172.21.15.101:39166) 2023-07-18T03:18:29.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.223+0000 7f637919a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639877d2be0 0x5639877da790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:35724/0 (socket says 172.21.15.101:35724) 2023-07-18T03:18:29.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.223+0000 7f637999b700 1 -- 172.21.15.101:0/527065210 learned_addr learned my addr 172.21.15.101:0/527065210 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:18:29.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.223+0000 7f637a19c700 1 --2- 172.21.15.101:0/527065210 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5639877ddad0 0x5639877daed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:29.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.223+0000 7f637999b700 1 -- 172.21.15.101:0/527065210 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639877d2be0 msgr2=0x5639877da790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:29.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.223+0000 7f637999b700 1 --2- 172.21.15.101:0/527065210 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639877d2be0 0x5639877da790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:29.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.223+0000 7f637999b700 1 -- 172.21.15.101:0/527065210 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5639877ddad0 msgr2=0x5639877daed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:29.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.223+0000 7f637999b700 1 --2- 172.21.15.101:0/527065210 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5639877ddad0 0x5639877daed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:29.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.223+0000 7f637999b700 1 -- 172.21.15.101:0/527065210 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f637410d220 con 0x5639877d22d0 2023-07-18T03:18:29.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.223+0000 7f637919a700 1 --2- 172.21.15.101:0/527065210 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639877d2be0 0x5639877da790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:18:29.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.224+0000 7f637999b700 1 --2- 172.21.15.101:0/527065210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639877d22d0 0x5639877dd510 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f637000c010 tx=0x7f6370022630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c681de415648dde5 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:18:29.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.224+0000 7f635d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:18:29.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.224+0000 7f635d7fa700 1 -- 172.21.15.101:0/527065210 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637001d030 con 0x5639877d22d0 2023-07-18T03:18:29.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.224+0000 7f635d7fa700 1 -- 172.21.15.101:0/527065210 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f63700058e0 con 0x5639877d22d0 2023-07-18T03:18:29.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.224+0000 7f638f551b80 1 -- 172.21.15.101:0/527065210 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56398767e4f0 con 0x5639877d22d0 2023-07-18T03:18:29.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.224+0000 7f635d7fa700 1 -- 172.21.15.101:0/527065210 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6370022ca0 con 0x5639877d22d0 2023-07-18T03:18:29.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.224+0000 7f638f551b80 10 client.27824 fetch_fsmap learned FSMap version 33 2023-07-18T03:18:29.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.224+0000 7f638f551b80 1 -- 172.21.15.101:0/527065210 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x56398753bc60 con 0x5639877d22d0 2023-07-18T03:18:29.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.224+0000 7f635d7fa700 1 -- 172.21.15.101:0/527065210 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56398767e4f0 con 0x5639877d22d0 2023-07-18T03:18:29.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.225+0000 7f635d7fa700 1 -- 172.21.15.101:0/527065210 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f637002c400 con 0x5639877d22d0 2023-07-18T03:18:29.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.225+0000 7f638f551b80 10 client.27824 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:18:29.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.225+0000 7f638f551b80 10 client.27824 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:18:29.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.225+0000 7f638f551b80 20 client.27824 populate_metadata read hostname 'smithi101' 2023-07-18T03:18:29.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.225+0000 7f638f551b80 1 --2- 172.21.15.101:0/527065210 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5639877ecee0 0x5639877ef2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:29.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.225+0000 7f638f551b80 4 client.27824 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:18:29.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.225+0000 7f638f551b80 1 -- 172.21.15.101:0/527065210 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56398771deb0 con 0x5639877ecee0 2023-07-18T03:18:29.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.225+0000 7f637919a700 1 --2- 172.21.15.101:0/527065210 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5639877ecee0 0x5639877ef2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:29.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.226+0000 7f637919a700 1 --2- 172.21.15.101:0/527065210 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5639877ecee0 0x5639877ef2c0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0eb2bb45b5b76e0 server_cookie=256b7844cf9c7afb in_seq=0 out_seq=0 2023-07-18T03:18:29.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.226+0000 7f635d7fa700 10 client.27824 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:18:29.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.226+0000 7f635d7fa700 1 -- 172.21.15.101:0/527065210 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56398771deb0 con 0x5639877ecee0 2023-07-18T03:18:29.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.226+0000 7f635d7fa700 10 client.27824 handle_command_reply: tid=0 2023-07-18T03:18:29.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.226+0000 7f638f551b80 1 client.27824 shutdown 2023-07-18T03:18:29.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.226+0000 7f638f551b80 1 -- 172.21.15.101:0/527065210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639877d22d0 msgr2=0x5639877dd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:29.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.226+0000 7f638f551b80 1 --2- 172.21.15.101:0/527065210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639877d22d0 0x5639877dd510 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f637000c010 tx=0x7f6370022630 comp rx=0 tx=0).stop 2023-07-18T03:18:29.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.227+0000 7f638f551b80 1 -- 172.21.15.101:0/527065210 shutdown_connections 2023-07-18T03:18:29.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.227+0000 7f638f551b80 1 --2- 172.21.15.101:0/527065210 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5639877ecee0 0x5639877ef2c0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:29.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.227+0000 7f638f551b80 1 --2- 172.21.15.101:0/527065210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639877d22d0 0x5639877dd510 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:29.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.227+0000 7f638f551b80 1 --2- 172.21.15.101:0/527065210 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5639877ddad0 0x5639877daed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:29.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.227+0000 7f638f551b80 1 --2- 172.21.15.101:0/527065210 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639877d2be0 0x5639877da790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:29.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.227+0000 7f635d7fa700 0 client.27824 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:18:29.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.227+0000 7f638f551b80 1 -- 172.21.15.101:0/527065210 >> 172.21.15.101:0/527065210 conn(0x5639877c8da0 msgr2=0x5639877ce230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:18:29.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.227+0000 7f638f551b80 1 -- 172.21.15.101:0/527065210 shutdown_connections 2023-07-18T03:18:29.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.227+0000 7f638f551b80 1 -- 172.21.15.101:0/527065210 wait complete. 2023-07-18T03:18:29.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.227+0000 7f638f551b80 20 client.27824 trim_cache size 0 max 16384 2023-07-18T03:18:29.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.248+0000 7f638f551b80 1 Processor -- start 2023-07-18T03:18:29.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.248+0000 7f638f551b80 1 -- start start 2023-07-18T03:18:29.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.248+0000 7f638f551b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639877d22d0 0x5639877e44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:29.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.248+0000 7f638f551b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5639877d2be0 0x5639877dd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:29.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.248+0000 7f638f551b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639877ddad0 0x5639877d7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:29.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.248+0000 7f638f551b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639877cfd20 con 0x5639877d2be0 2023-07-18T03:18:29.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.248+0000 7f638f551b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639877d3540 con 0x5639877d22d0 2023-07-18T03:18:29.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.248+0000 7f638f551b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639877db4d0 con 0x5639877ddad0 2023-07-18T03:18:29.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.249+0000 7f637999b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639877d22d0 0x5639877e44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:29.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.249+0000 7f637a19c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639877ddad0 0x5639877d7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:29.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.249+0000 7f637999b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639877d22d0 0x5639877e44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:39182/0 (socket says 172.21.15.101:39182) 2023-07-18T03:18:29.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.249+0000 7f637a19c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639877ddad0 0x5639877d7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:35736/0 (socket says 172.21.15.101:35736) 2023-07-18T03:18:29.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.249+0000 7f637999b700 1 -- 172.21.15.101:0/137949049 learned_addr learned my addr 172.21.15.101:0/137949049 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:18:29.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.249+0000 7f637919a700 1 --2- 172.21.15.101:0/137949049 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5639877d2be0 0x5639877dd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:29.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.249+0000 7f637999b700 1 -- 172.21.15.101:0/137949049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639877ddad0 msgr2=0x5639877d7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:29.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.249+0000 7f637999b700 1 --2- 172.21.15.101:0/137949049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639877ddad0 0x5639877d7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:29.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.249+0000 7f637999b700 1 -- 172.21.15.101:0/137949049 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5639877d2be0 msgr2=0x5639877dd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:29.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.249+0000 7f637999b700 1 --2- 172.21.15.101:0/137949049 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5639877d2be0 0x5639877dd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:29.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.249+0000 7f637999b700 1 -- 172.21.15.101:0/137949049 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563987579250 con 0x5639877d22d0 2023-07-18T03:18:29.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.249+0000 7f637999b700 1 --2- 172.21.15.101:0/137949049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639877d22d0 0x5639877e44a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6370009990 tx=0x7f6370005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5d84ca9bfc4f574 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:18:29.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.250+0000 7f6368bff700 1 -- 172.21.15.101:0/137949049 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637001d070 con 0x5639877d22d0 2023-07-18T03:18:29.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.250+0000 7f6368bff700 1 -- 172.21.15.101:0/137949049 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63700058e0 con 0x5639877d22d0 2023-07-18T03:18:29.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.250+0000 7f6368bff700 1 -- 172.21.15.101:0/137949049 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637002ba00 con 0x5639877d22d0 2023-07-18T03:18:29.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.250+0000 7f638f551b80 1 -- 172.21.15.101:0/137949049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639877d22d0 msgr2=0x5639877e44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:29.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.250+0000 7f638f551b80 1 --2- 172.21.15.101:0/137949049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639877d22d0 0x5639877e44a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6370009990 tx=0x7f6370005e80 comp rx=0 tx=0).stop 2023-07-18T03:18:29.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.250+0000 7f638f551b80 1 -- 172.21.15.101:0/137949049 shutdown_connections 2023-07-18T03:18:29.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.250+0000 7f638f551b80 1 --2- 172.21.15.101:0/137949049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639877d22d0 0x5639877e44a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:29.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.250+0000 7f638f551b80 1 --2- 172.21.15.101:0/137949049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639877ddad0 0x5639877d7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:29.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.250+0000 7f638f551b80 1 --2- 172.21.15.101:0/137949049 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5639877d2be0 0x5639877dd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:29.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.250+0000 7f638f551b80 1 -- 172.21.15.101:0/137949049 >> 172.21.15.101:0/137949049 conn(0x5639877c8da0 msgr2=0x5639877d5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:18:29.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.250+0000 7f638f551b80 1 -- 172.21.15.101:0/137949049 shutdown_connections 2023-07-18T03:18:29.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.250+0000 7f638f551b80 1 -- 172.21.15.101:0/137949049 wait complete. 2023-07-18T03:18:29.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.251+0000 7f638f551b80 1 -- start start 2023-07-18T03:18:29.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.251+0000 7f638f551b80 1 Processor -- start 2023-07-18T03:18:29.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.252+0000 7f638f551b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639877d22d0 0x5639877d7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:29.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.252+0000 7f638f551b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639877d2be0 0x5639877d8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:29.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.252+0000 7f638f551b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5639877ddad0 0x5639877d6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:29.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.252+0000 7f638f551b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639877d70e0 con 0x5639877ddad0 2023-07-18T03:18:29.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.252+0000 7f638f551b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639877d7250 con 0x5639877d2be0 2023-07-18T03:18:29.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.252+0000 7f638f551b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639877d7410 con 0x5639877d22d0 2023-07-18T03:18:29.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.252+0000 7f638f551b80 10 client.0 _ll_register_callbacks cb 0x7f63640d09d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:18:29.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.252+0000 7f637999b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639877d22d0 0x5639877d7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:29.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.252+0000 7f637999b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639877d22d0 0x5639877d7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:35752/0 (socket says 172.21.15.101:35752) 2023-07-18T03:18:29.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.252+0000 7f637999b700 1 -- 172.21.15.101:0/653876319 learned_addr learned my addr 172.21.15.101:0/653876319 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:18:29.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.252+0000 7f637919a700 1 --2- 172.21.15.101:0/653876319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639877d2be0 0x5639877d8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:29.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.253+0000 7f637999b700 1 -- 172.21.15.101:0/653876319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639877d2be0 msgr2=0x5639877d8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:29.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.253+0000 7f637999b700 1 --2- 172.21.15.101:0/653876319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639877d2be0 0x5639877d8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:29.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.253+0000 7f637999b700 1 -- 172.21.15.101:0/653876319 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5639877ddad0 msgr2=0x5639877d6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:29.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.253+0000 7f637999b700 1 --2- 172.21.15.101:0/653876319 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5639877ddad0 0x5639877d6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:29.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.253+0000 7f637999b700 1 -- 172.21.15.101:0/653876319 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56398753bc60 con 0x5639877d22d0 2023-07-18T03:18:29.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.253+0000 7f637919a700 1 --2- 172.21.15.101:0/653876319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639877d2be0 0x5639877d8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:18:29.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.253+0000 7f637999b700 1 --2- 172.21.15.101:0/653876319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639877d22d0 0x5639877d7f30 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6370009ce0 tx=0x7f637000bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=363ad73949255e0c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:18:29.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.253+0000 7f635d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:18:29.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.253+0000 7f635d7fa700 1 -- 172.21.15.101:0/653876319 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637001d070 con 0x5639877d22d0 2023-07-18T03:18:29.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.253+0000 7f635d7fa700 1 -- 172.21.15.101:0/653876319 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f6370022aa0 con 0x5639877d22d0 2023-07-18T03:18:29.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.253+0000 7f638f551b80 1 -- 172.21.15.101:0/653876319 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f637001e040 con 0x5639877d22d0 2023-07-18T03:18:29.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.254+0000 7f635d7fa700 1 -- 172.21.15.101:0/653876319 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6370022c60 con 0x5639877d22d0 2023-07-18T03:18:29.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.254+0000 7f638f551b80 10 client.27865 fetch_fsmap learned FSMap version 33 2023-07-18T03:18:29.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.254+0000 7f638f551b80 1 -- 172.21.15.101:0/653876319 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5639875512b0 con 0x5639877d22d0 2023-07-18T03:18:29.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.254+0000 7f635d7fa700 1 -- 172.21.15.101:0/653876319 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f637001e040 con 0x5639877d22d0 2023-07-18T03:18:29.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.254+0000 7f635d7fa700 1 -- 172.21.15.101:0/653876319 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f637000f730 con 0x5639877d22d0 2023-07-18T03:18:29.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.254+0000 7f638f551b80 10 client.27865 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:18:29.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.254+0000 7f638f551b80 10 client.27865 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:18:29.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.254+0000 7f638f551b80 20 client.27865 populate_metadata read hostname 'smithi101' 2023-07-18T03:18:29.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.254+0000 7f638f551b80 1 --2- 172.21.15.101:0/653876319 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563987841890 0x563987843c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:29.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.254+0000 7f638f551b80 4 client.27865 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:18:29.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.254+0000 7f638f551b80 1 -- 172.21.15.101:0/653876319 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5639876e0050 con 0x563987841890 2023-07-18T03:18:29.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.254+0000 7f637919a700 1 --2- 172.21.15.101:0/653876319 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563987841890 0x563987843c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:29.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.255+0000 7f635d7fa700 10 client.27865 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:18:29.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.255+0000 7f637919a700 1 --2- 172.21.15.101:0/653876319 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563987841890 0x563987843c70 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc8e9ceb7b89b3c5 server_cookie=22c9e278dd3fc750 in_seq=0 out_seq=0 2023-07-18T03:18:29.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.255+0000 7f635d7fa700 1 -- 172.21.15.101:0/653876319 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5639876e0050 con 0x563987841890 2023-07-18T03:18:29.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.255+0000 7f635d7fa700 10 client.27865 handle_command_reply: tid=0 2023-07-18T03:18:29.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.255+0000 7f638f551b80 1 client.27865 shutdown 2023-07-18T03:18:29.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.255+0000 7f638f551b80 1 -- 172.21.15.101:0/653876319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639877d22d0 msgr2=0x5639877d7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:29.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.255+0000 7f638f551b80 1 --2- 172.21.15.101:0/653876319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639877d22d0 0x5639877d7f30 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6370009ce0 tx=0x7f637000bfd0 comp rx=0 tx=0).stop 2023-07-18T03:18:29.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.255+0000 7f638f551b80 1 -- 172.21.15.101:0/653876319 shutdown_connections 2023-07-18T03:18:29.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.255+0000 7f638f551b80 1 --2- 172.21.15.101:0/653876319 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563987841890 0x563987843c70 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:29.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.255+0000 7f638f551b80 1 --2- 172.21.15.101:0/653876319 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5639877ddad0 0x5639877d6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:29.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.255+0000 7f638f551b80 1 --2- 172.21.15.101:0/653876319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639877d22d0 0x5639877d7f30 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:29.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.255+0000 7f635d7fa700 0 client.27865 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:18:29.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.255+0000 7f638f551b80 1 --2- 172.21.15.101:0/653876319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5639877d2be0 0x5639877d8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:29.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.255+0000 7f638f551b80 1 -- 172.21.15.101:0/653876319 >> 172.21.15.101:0/653876319 conn(0x5639877c8da0 msgr2=0x5639877d43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:18:29.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.256+0000 7f638f551b80 1 -- 172.21.15.101:0/653876319 shutdown_connections 2023-07-18T03:18:29.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.256+0000 7f638f551b80 1 -- 172.21.15.101:0/653876319 wait complete. 2023-07-18T03:18:29.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.256+0000 7f638f551b80 20 client.27865 trim_cache size 0 max 16384 2023-07-18T03:18:29.259 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:18:29.259 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:18:29.259 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:18:29.259 INFO:teuthology.orchestra.run.smithi101.stdout: "00f9ad7e-1ca8-49f0-a47c-4a43f0c4ac1a": { 2023-07-18T03:18:29.259 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:18:29.259 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "00f9ad7e-1ca8-49f0-a47c-4a43f0c4ac1a", 2023-07-18T03:18:29.260 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:18:29.260 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:18:29.260 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:18:29.260 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:18:29.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.256+0000 7f637bbff700 1 -- 172.21.15.101:0/750560437 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f63600a12c0 msgr2=0x7f63600a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:29.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.256+0000 7f637bbff700 1 --2- 172.21.15.101:0/750560437 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f63600a12c0 0x7f63600a3770 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f636c0077a0 tx=0x7f636c004ad0 comp rx=0 tx=0).stop 2023-07-18T03:18:29.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.256+0000 7f637bbff700 1 -- 172.21.15.101:0/750560437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6374100490 msgr2=0x7f6374072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:29.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.256+0000 7f637bbff700 1 --2- 172.21.15.101:0/750560437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6374100490 0x7f6374072d50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f636400e9f0 tx=0x7f636400ed00 comp rx=0 tx=0).stop 2023-07-18T03:18:29.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.256+0000 7f637bbff700 1 -- 172.21.15.101:0/750560437 shutdown_connections 2023-07-18T03:18:29.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.256+0000 7f637bbff700 1 --2- 172.21.15.101:0/750560437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6374105af0 0x7f63741a3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:29.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.256+0000 7f637bbff700 1 --2- 172.21.15.101:0/750560437 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63740ffae0 0x7f6374072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:29.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.256+0000 7f637bbff700 1 --2- 172.21.15.101:0/750560437 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f63600a12c0 0x7f63600a3770 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:29.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.256+0000 7f637bbff700 1 --2- 172.21.15.101:0/750560437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6374100490 0x7f6374072d50 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:29.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.256+0000 7f637bbff700 1 -- 172.21.15.101:0/750560437 >> 172.21.15.101:0/750560437 conn(0x7f63740fb5e0 msgr2=0x7f63740fbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:18:29.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.257+0000 7f637bbff700 1 -- 172.21.15.101:0/750560437 shutdown_connections 2023-07-18T03:18:29.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:29.257+0000 7f637bbff700 1 -- 172.21.15.101:0/750560437 wait complete. 2023-07-18T03:18:29.270 INFO:tasks.cephfs.filesystem:scrub status for tag:00f9ad7e-1ca8-49f0-a47c-4a43f0c4ac1a - {'path': '/', 'tag': '00f9ad7e-1ca8-49f0-a47c-4a43f0c4ac1a', 'options': 'recursive,force'} 2023-07-18T03:18:29.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:29 smithi155 ceph-mon[128885]: pgmap v340: 129 pgs: 129 active+clean; 545 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 106 op/s 2023-07-18T03:18:29.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:29 smithi179 ceph-mon[129229]: pgmap v340: 129 pgs: 129 active+clean; 545 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 106 op/s 2023-07-18T03:18:29.620 INFO:tasks.workunit.client.0.smithi101.stdout: 1 673827 5.63 MB/sec execute 474 sec latency 1956.617 ms 2023-07-18T03:18:30.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:30 smithi155 ceph-mon[128885]: scrub queued for path: / 2023-07-18T03:18:30.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:30 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:18:30.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:30 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:18:30.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:30 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:18:30.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:30 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:18:30.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:30 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:18:30.298 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:30 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:18:30.298 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:30 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:18:30.298 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:30 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:18:30.298 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:30 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:18:30.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:30 smithi179 ceph-mon[129229]: scrub queued for path: / 2023-07-18T03:18:30.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:30 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:18:30.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:30 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:18:30.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:30 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:18:30.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:30 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:18:30.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:30 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:18:30.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:30 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:18:30.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:30 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:18:30.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:30 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:18:30.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:30 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:18:30.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:30 smithi101 ceph-mon[126379]: scrub queued for path: / 2023-07-18T03:18:30.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:30 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:18:30.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:30 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:18:30.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:30 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:18:30.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:30 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:18:30.492 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:30 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:18:30.492 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:30 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:18:30.492 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:30 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:18:30.493 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:30 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:18:30.493 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:30 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:18:30.620 INFO:tasks.workunit.client.0.smithi101.stdout: 1 675217 5.64 MB/sec execute 475 sec latency 20.600 ms 2023-07-18T03:18:31.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:31 smithi155 ceph-mon[128885]: pgmap v341: 129 pgs: 129 active+clean; 550 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 168 op/s 2023-07-18T03:18:31.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:31 smithi179 ceph-mon[129229]: pgmap v341: 129 pgs: 129 active+clean; 550 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 168 op/s 2023-07-18T03:18:31.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:31 smithi101 ceph-mon[126379]: pgmap v341: 129 pgs: 129 active+clean; 550 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 168 op/s 2023-07-18T03:18:31.620 INFO:tasks.workunit.client.0.smithi101.stdout: 1 676296 5.64 MB/sec execute 476 sec latency 22.704 ms 2023-07-18T03:18:32.621 INFO:tasks.workunit.client.0.smithi101.stdout: 1 676458 5.63 MB/sec execute 477 sec latency 800.943 ms 2023-07-18T03:18:33.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:33 smithi155 ceph-mon[128885]: pgmap v342: 129 pgs: 129 active+clean; 550 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 113 op/s 2023-07-18T03:18:33.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:33 smithi179 ceph-mon[129229]: pgmap v342: 129 pgs: 129 active+clean; 550 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 113 op/s 2023-07-18T03:18:33.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:33 smithi101 ceph-mon[126379]: pgmap v342: 129 pgs: 129 active+clean; 550 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 113 op/s 2023-07-18T03:18:33.621 INFO:tasks.workunit.client.0.smithi101.stdout: 1 676458 5.62 MB/sec execute 478 sec latency 1801.043 ms 2023-07-18T03:18:34.621 INFO:tasks.workunit.client.0.smithi101.stdout: 1 676857 5.61 MB/sec execute 479 sec latency 2622.879 ms 2023-07-18T03:18:35.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:35 smithi155 ceph-mon[128885]: pgmap v343: 129 pgs: 129 active+clean; 550 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 113 op/s 2023-07-18T03:18:35.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:35 smithi179 ceph-mon[129229]: pgmap v343: 129 pgs: 129 active+clean; 550 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 113 op/s 2023-07-18T03:18:35.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:35 smithi101 ceph-mon[126379]: pgmap v343: 129 pgs: 129 active+clean; 550 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 113 op/s 2023-07-18T03:18:35.628 INFO:tasks.workunit.client.0.smithi101.stdout: 1 678442 5.62 MB/sec execute 480 sec latency 79.790 ms 2023-07-18T03:18:36.622 INFO:tasks.workunit.client.0.smithi101.stdout: 1 679288 5.61 MB/sec execute 481 sec latency 17.807 ms 2023-07-18T03:18:37.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:37 smithi155 ceph-mon[128885]: pgmap v344: 129 pgs: 129 active+clean; 553 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 156 op/s 2023-07-18T03:18:37.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:37 smithi179 ceph-mon[129229]: pgmap v344: 129 pgs: 129 active+clean; 553 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 156 op/s 2023-07-18T03:18:37.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:37 smithi101 ceph-mon[126379]: pgmap v344: 129 pgs: 129 active+clean; 553 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 156 op/s 2023-07-18T03:18:37.621 INFO:tasks.workunit.client.0.smithi101.stdout: 1 680001 5.61 MB/sec execute 482 sec latency 347.749 ms 2023-07-18T03:18:38.621 INFO:tasks.workunit.client.0.smithi101.stdout: 1 680001 5.60 MB/sec execute 483 sec latency 1347.809 ms 2023-07-18T03:18:39.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:39 smithi155 ceph-mon[128885]: pgmap v345: 129 pgs: 129 active+clean; 553 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 104 op/s 2023-07-18T03:18:39.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:39 smithi179 ceph-mon[129229]: pgmap v345: 129 pgs: 129 active+clean; 553 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 104 op/s 2023-07-18T03:18:39.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:39 smithi101 ceph-mon[126379]: pgmap v345: 129 pgs: 129 active+clean; 553 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 104 op/s 2023-07-18T03:18:39.621 INFO:tasks.workunit.client.0.smithi101.stdout: 1 680401 5.59 MB/sec execute 484 sec latency 2168.761 ms 2023-07-18T03:18:40.621 INFO:tasks.workunit.client.0.smithi101.stdout: 1 682099 5.61 MB/sec execute 485 sec latency 17.890 ms 2023-07-18T03:18:41.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:41 smithi155 ceph-mon[128885]: pgmap v346: 129 pgs: 129 active+clean; 559 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 156 op/s 2023-07-18T03:18:41.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:41 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:18:41.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:41 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:18:41.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:41 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:18:41.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:41 smithi179 ceph-mon[129229]: pgmap v346: 129 pgs: 129 active+clean; 559 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 156 op/s 2023-07-18T03:18:41.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:41 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:18:41.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:41 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:18:41.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:41 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:18:41.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:41 smithi101 ceph-mon[126379]: pgmap v346: 129 pgs: 129 active+clean; 559 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 156 op/s 2023-07-18T03:18:41.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:41 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:18:41.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:41 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:18:41.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:41 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:18:41.622 INFO:tasks.workunit.client.0.smithi101.stdout: 1 683108 5.60 MB/sec execute 486 sec latency 15.075 ms 2023-07-18T03:18:42.621 INFO:tasks.workunit.client.0.smithi101.stdout: 1 683549 5.59 MB/sec execute 487 sec latency 507.439 ms 2023-07-18T03:18:43.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:43 smithi155 ceph-mon[128885]: pgmap v347: 129 pgs: 129 active+clean; 559 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 1.3 MiB/s wr, 94 op/s 2023-07-18T03:18:43.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:43 smithi179 ceph-mon[129229]: pgmap v347: 129 pgs: 129 active+clean; 559 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 1.3 MiB/s wr, 94 op/s 2023-07-18T03:18:43.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:43 smithi101 ceph-mon[126379]: pgmap v347: 129 pgs: 129 active+clean; 559 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 1.3 MiB/s wr, 94 op/s 2023-07-18T03:18:43.622 INFO:tasks.workunit.client.0.smithi101.stdout: 1 683549 5.58 MB/sec execute 488 sec latency 1507.565 ms 2023-07-18T03:18:44.622 INFO:tasks.workunit.client.0.smithi101.stdout: 1 683889 5.57 MB/sec execute 489 sec latency 2352.083 ms 2023-07-18T03:18:45.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:45 smithi155 ceph-mon[128885]: pgmap v348: 129 pgs: 129 active+clean; 559 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 1.3 MiB/s wr, 94 op/s 2023-07-18T03:18:45.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:45 smithi179 ceph-mon[129229]: pgmap v348: 129 pgs: 129 active+clean; 559 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 1.3 MiB/s wr, 94 op/s 2023-07-18T03:18:45.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:45 smithi101 ceph-mon[126379]: pgmap v348: 129 pgs: 129 active+clean; 559 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 1.3 MiB/s wr, 94 op/s 2023-07-18T03:18:45.622 INFO:tasks.workunit.client.0.smithi101.stdout: 1 685623 5.59 MB/sec execute 490 sec latency 19.936 ms 2023-07-18T03:18:46.622 INFO:tasks.workunit.client.0.smithi101.stdout: 1 686459 5.58 MB/sec execute 491 sec latency 85.547 ms 2023-07-18T03:18:47.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:47 smithi155 ceph-mon[128885]: pgmap v349: 129 pgs: 129 active+clean; 564 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 145 op/s 2023-07-18T03:18:47.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:47 smithi179 ceph-mon[129229]: pgmap v349: 129 pgs: 129 active+clean; 564 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 145 op/s 2023-07-18T03:18:47.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:47 smithi101 ceph-mon[126379]: pgmap v349: 129 pgs: 129 active+clean; 564 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 145 op/s 2023-07-18T03:18:47.622 INFO:tasks.workunit.client.0.smithi101.stdout: 1 687092 5.58 MB/sec execute 492 sec latency 349.828 ms 2023-07-18T03:18:48.622 INFO:tasks.workunit.client.0.smithi101.stdout: 1 687092 5.56 MB/sec execute 493 sec latency 1349.915 ms 2023-07-18T03:18:49.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:49 smithi155 ceph-mon[128885]: pgmap v350: 129 pgs: 129 active+clean; 564 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 1.5 MiB/s wr, 103 op/s 2023-07-18T03:18:49.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:49 smithi179 ceph-mon[129229]: pgmap v350: 129 pgs: 129 active+clean; 564 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 1.5 MiB/s wr, 103 op/s 2023-07-18T03:18:49.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:49 smithi101 ceph-mon[126379]: pgmap v350: 129 pgs: 129 active+clean; 564 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 1.5 MiB/s wr, 103 op/s 2023-07-18T03:18:49.622 INFO:tasks.workunit.client.0.smithi101.stdout: 1 687541 5.55 MB/sec execute 494 sec latency 2169.817 ms 2023-07-18T03:18:50.622 INFO:tasks.workunit.client.0.smithi101.stdout: 1 689185 5.57 MB/sec execute 495 sec latency 16.513 ms 2023-07-18T03:18:51.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:51 smithi155 ceph-mon[128885]: pgmap v351: 129 pgs: 129 active+clean; 569 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 154 op/s 2023-07-18T03:18:51.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:51 smithi179 ceph-mon[129229]: pgmap v351: 129 pgs: 129 active+clean; 569 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 154 op/s 2023-07-18T03:18:51.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:51 smithi101 ceph-mon[126379]: pgmap v351: 129 pgs: 129 active+clean; 569 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 154 op/s 2023-07-18T03:18:51.622 INFO:tasks.workunit.client.0.smithi101.stdout: 1 690337 5.57 MB/sec execute 496 sec latency 19.498 ms 2023-07-18T03:18:52.623 INFO:tasks.workunit.client.0.smithi101.stdout: 1 690640 5.56 MB/sec execute 497 sec latency 470.101 ms 2023-07-18T03:18:53.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:53 smithi179 ceph-mon[129229]: pgmap v352: 129 pgs: 129 active+clean; 569 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 102 op/s 2023-07-18T03:18:53.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:53 smithi101 ceph-mon[126379]: pgmap v352: 129 pgs: 129 active+clean; 569 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 102 op/s 2023-07-18T03:18:53.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:53 smithi155 ceph-mon[128885]: pgmap v352: 129 pgs: 129 active+clean; 569 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 102 op/s 2023-07-18T03:18:53.622 INFO:tasks.workunit.client.0.smithi101.stdout: 1 690640 5.55 MB/sec execute 498 sec latency 1470.187 ms 2023-07-18T03:18:54.623 INFO:tasks.workunit.client.0.smithi101.stdout: 1 691040 5.54 MB/sec execute 499 sec latency 2285.125 ms 2023-07-18T03:18:55.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:55 smithi179 ceph-mon[129229]: pgmap v353: 129 pgs: 129 active+clean; 569 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 102 op/s 2023-07-18T03:18:55.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:55 smithi101 ceph-mon[126379]: pgmap v353: 129 pgs: 129 active+clean; 569 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 102 op/s 2023-07-18T03:18:55.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:55 smithi155 ceph-mon[128885]: pgmap v353: 129 pgs: 129 active+clean; 569 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 102 op/s 2023-07-18T03:18:55.623 INFO:tasks.workunit.client.0.smithi101.stdout: 1 692736 5.55 MB/sec execute 500 sec latency 49.956 ms 2023-07-18T03:18:56.623 INFO:tasks.workunit.client.0.smithi101.stdout: 1 693926 5.55 MB/sec execute 501 sec latency 20.904 ms 2023-07-18T03:18:57.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:57 smithi179 ceph-mon[129229]: pgmap v354: 129 pgs: 129 active+clean; 574 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 151 op/s 2023-07-18T03:18:57.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:57 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:18:57.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:57 smithi101 ceph-mon[126379]: pgmap v354: 129 pgs: 129 active+clean; 574 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 151 op/s 2023-07-18T03:18:57.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:57 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:18:57.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:57 smithi155 ceph-mon[128885]: pgmap v354: 129 pgs: 129 active+clean; 574 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 151 op/s 2023-07-18T03:18:57.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:57 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:18:57.623 INFO:tasks.workunit.client.0.smithi101.stdout: 1 694183 5.54 MB/sec execute 502 sec latency 791.542 ms 2023-07-18T03:18:58.623 INFO:tasks.workunit.client.0.smithi101.stdout: 1 694183 5.53 MB/sec execute 503 sec latency 1791.710 ms 2023-07-18T03:18:58.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:58 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:18:58.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:58 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:18:58.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:58 smithi179 ceph-mon[129229]: pgmap v355: 129 pgs: 129 active+clean; 574 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 100 op/s 2023-07-18T03:18:58.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:58 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:18:58.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:58 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:18:58.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:18:58 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:18:58.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:58 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:18:58.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:58 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:18:58.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:58 smithi101 ceph-mon[126379]: pgmap v355: 129 pgs: 129 active+clean; 574 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 100 op/s 2023-07-18T03:18:58.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:58 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:18:58.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:58 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:18:58.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:18:58 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:18:58.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:58 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:18:58.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:58 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:18:58.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:58 smithi155 ceph-mon[128885]: pgmap v355: 129 pgs: 129 active+clean; 574 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 100 op/s 2023-07-18T03:18:58.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:58 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:18:58.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:58 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:18:58.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:18:58 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:18:59.271 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:18:59.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.490+0000 7f44555ff700 1 -- 172.21.15.101:0/2583475897 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44500ffad0 msgr2=0x7f44500fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:59.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.490+0000 7f44555ff700 1 --2- 172.21.15.101:0/2583475897 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44500ffad0 0x7f44500fff40 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4444009990 tx=0x7f4444009ca0 comp rx=0 tx=0).stop 2023-07-18T03:18:59.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.490+0000 7f44555ff700 1 -- 172.21.15.101:0/2583475897 shutdown_connections 2023-07-18T03:18:59.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.490+0000 7f44555ff700 1 --2- 172.21.15.101:0/2583475897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4450105ae0 0x7f4450105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:59.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.490+0000 7f44555ff700 1 --2- 172.21.15.101:0/2583475897 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4450100480 0x7f445010c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:59.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.490+0000 7f44555ff700 1 --2- 172.21.15.101:0/2583475897 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44500ffad0 0x7f44500fff40 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:59.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.490+0000 7f44555ff700 1 -- 172.21.15.101:0/2583475897 >> 172.21.15.101:0/2583475897 conn(0x7f44500fb5f0 msgr2=0x7f44500fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:18:59.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.490+0000 7f44555ff700 1 -- 172.21.15.101:0/2583475897 shutdown_connections 2023-07-18T03:18:59.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.490+0000 7f44555ff700 1 -- 172.21.15.101:0/2583475897 wait complete. 2023-07-18T03:18:59.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.491+0000 7f44555ff700 1 Processor -- start 2023-07-18T03:18:59.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.491+0000 7f44555ff700 1 -- start start 2023-07-18T03:18:59.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.492+0000 7f44555ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f44500ffad0 0x7f4450072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:59.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.492+0000 7f44555ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4450100480 0x7f4450072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:59.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.492+0000 7f44555ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4450105ae0 0x7f44501a3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:59.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.492+0000 7f44555ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4450073580 con 0x7f44500ffad0 2023-07-18T03:18:59.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.492+0000 7f44555ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44501a4280 con 0x7f4450100480 2023-07-18T03:18:59.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.492+0000 7f44555ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44501a43c0 con 0x7f4450105ae0 2023-07-18T03:18:59.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.492+0000 7f444effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f44500ffad0 0x7f4450072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:59.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.492+0000 7f444e7fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4450100480 0x7f4450072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:59.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.492+0000 7f444effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f44500ffad0 0x7f4450072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:59660/0 (socket says 172.21.15.101:59660) 2023-07-18T03:18:59.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.492+0000 7f444effd700 1 -- 172.21.15.101:0/433163865 learned_addr learned my addr 172.21.15.101:0/433163865 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:18:59.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.492+0000 7f444f7fe700 1 --2- 172.21.15.101:0/433163865 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4450105ae0 0x7f44501a3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:59.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.492+0000 7f444effd700 1 -- 172.21.15.101:0/433163865 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4450105ae0 msgr2=0x7f44501a3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:59.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.492+0000 7f444effd700 1 --2- 172.21.15.101:0/433163865 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4450105ae0 0x7f44501a3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:59.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.492+0000 7f444effd700 1 -- 172.21.15.101:0/433163865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4450100480 msgr2=0x7f4450072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:59.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.492+0000 7f444effd700 1 --2- 172.21.15.101:0/433163865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4450100480 0x7f4450072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:59.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.492+0000 7f444effd700 1 -- 172.21.15.101:0/433163865 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44440095f0 con 0x7f44500ffad0 2023-07-18T03:18:59.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.492+0000 7f444e7fc700 1 --2- 172.21.15.101:0/433163865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4450100480 0x7f4450072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:18:59.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.493+0000 7f444effd700 1 --2- 172.21.15.101:0/433163865 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f44500ffad0 0x7f4450072810 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f444000e9a0 tx=0x7f444000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:18:59.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.493+0000 7f4437fff700 1 -- 172.21.15.101:0/433163865 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4440009790 con 0x7f44500ffad0 2023-07-18T03:18:59.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.493+0000 7f44555ff700 1 -- 172.21.15.101:0/433163865 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44501a4620 con 0x7f44500ffad0 2023-07-18T03:18:59.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.493+0000 7f4437fff700 1 -- 172.21.15.101:0/433163865 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4440010430 con 0x7f44500ffad0 2023-07-18T03:18:59.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.493+0000 7f4437fff700 1 -- 172.21.15.101:0/433163865 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4440018680 con 0x7f44500ffad0 2023-07-18T03:18:59.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.493+0000 7f44555ff700 1 -- 172.21.15.101:0/433163865 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44501a4b00 con 0x7f44500ffad0 2023-07-18T03:18:59.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.496+0000 7f4437fff700 1 -- 172.21.15.101:0/433163865 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f44400105a0 con 0x7f44500ffad0 2023-07-18T03:18:59.492 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.497+0000 7f4468fb5b80 1 Processor -- start 2023-07-18T03:18:59.492 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.498+0000 7f4468fb5b80 1 -- start start 2023-07-18T03:18:59.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.498+0000 7f4468fb5b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b76bbbf2d0 0x55b76bbbf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:59.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.498+0000 7f4468fb5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b76bbbfbe0 0x55b76bbca590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:59.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.498+0000 7f4468fb5b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b76bbcaad0 0x55b76bbcceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:59.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.498+0000 7f4468fb5b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b76bac5cb0 con 0x55b76bbcaad0 2023-07-18T03:18:59.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.498+0000 7f4468fb5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b76bb18a40 con 0x55b76bbbfbe0 2023-07-18T03:18:59.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.498+0000 7f4468fb5b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b76bb11ed0 con 0x55b76bbbf2d0 2023-07-18T03:18:59.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.498+0000 7f4437fff700 1 --2- 172.21.15.101:0/433163865 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f44380a12c0 0x7f44380a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:59.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.498+0000 7f444effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b76bbbfbe0 0x55b76bbca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:59.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.498+0000 7f444f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b76bbbf2d0 0x55b76bbbf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:59.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.498+0000 7f444f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b76bbbf2d0 0x55b76bbbf6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41440/0 (socket says 172.21.15.101:41440) 2023-07-18T03:18:59.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.498+0000 7f444f7fe700 1 -- 172.21.15.101:0/44349532 learned_addr learned my addr 172.21.15.101:0/44349532 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:18:59.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.498+0000 7f444f7fe700 1 --2- 172.21.15.101:0/433163865 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f44380a12c0 0x7f44380a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:59.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.498+0000 7f444e7fc700 1 --2- 172.21.15.101:0/44349532 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b76bbcaad0 0x55b76bbcceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:59.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.499+0000 7f4437fff700 1 -- 172.21.15.101:0/433163865 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f4440014070 con 0x7f44500ffad0 2023-07-18T03:18:59.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.499+0000 7f444effd700 1 -- 172.21.15.101:0/44349532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b76bbbf2d0 msgr2=0x55b76bbbf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:59.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.499+0000 7f444effd700 1 --2- 172.21.15.101:0/44349532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b76bbbf2d0 0x55b76bbbf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:59.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.499+0000 7f444effd700 1 -- 172.21.15.101:0/44349532 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b76bbcaad0 msgr2=0x55b76bbcceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:59.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.499+0000 7f444effd700 1 --2- 172.21.15.101:0/44349532 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b76bbcaad0 0x55b76bbcceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:59.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.499+0000 7f444effd700 1 -- 172.21.15.101:0/44349532 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b76b93fde0 con 0x55b76bbbfbe0 2023-07-18T03:18:59.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.499+0000 7f444effd700 1 --2- 172.21.15.101:0/44349532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b76bbbfbe0 0x55b76bbca590 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f44400ca5a0 tx=0x7f4440065370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c2dbcc74cd783a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:18:59.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.499+0000 7f444f7fe700 1 --2- 172.21.15.101:0/433163865 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f44380a12c0 0x7f44380a3770 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4448009fd0 tx=0x7f444800b040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:18:59.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.499+0000 7f44357fa700 1 -- 172.21.15.101:0/44349532 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4440099880 con 0x55b76bbbfbe0 2023-07-18T03:18:59.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.499+0000 7f44357fa700 1 -- 172.21.15.101:0/44349532 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f444005e850 con 0x55b76bbbfbe0 2023-07-18T03:18:59.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.499+0000 7f44357fa700 1 -- 172.21.15.101:0/44349532 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444009a180 con 0x55b76bbbfbe0 2023-07-18T03:18:59.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.500+0000 7f4468fb5b80 1 -- 172.21.15.101:0/44349532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b76bbbfbe0 msgr2=0x55b76bbca590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:59.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.500+0000 7f4468fb5b80 1 --2- 172.21.15.101:0/44349532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b76bbbfbe0 0x55b76bbca590 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f44400ca5a0 tx=0x7f4440065370 comp rx=0 tx=0).stop 2023-07-18T03:18:59.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.500+0000 7f4468fb5b80 1 -- 172.21.15.101:0/44349532 shutdown_connections 2023-07-18T03:18:59.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.500+0000 7f4468fb5b80 1 --2- 172.21.15.101:0/44349532 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b76bbcaad0 0x55b76bbcceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:59.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.500+0000 7f4468fb5b80 1 --2- 172.21.15.101:0/44349532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b76bbbf2d0 0x55b76bbbf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:59.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.500+0000 7f4468fb5b80 1 --2- 172.21.15.101:0/44349532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b76bbbfbe0 0x55b76bbca590 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:59.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.500+0000 7f4468fb5b80 1 -- 172.21.15.101:0/44349532 >> 172.21.15.101:0/44349532 conn(0x55b76bbb5da0 msgr2=0x55b76bbbb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:18:59.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.500+0000 7f4468fb5b80 1 -- 172.21.15.101:0/44349532 shutdown_connections 2023-07-18T03:18:59.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.500+0000 7f4468fb5b80 1 -- 172.21.15.101:0/44349532 wait complete. 2023-07-18T03:18:59.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.501+0000 7f4468fb5b80 1 -- start start 2023-07-18T03:18:59.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.501+0000 7f4468fb5b80 1 Processor -- start 2023-07-18T03:18:59.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.501+0000 7f4468fb5b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b76bbbf2d0 0x55b76bbca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:59.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.501+0000 7f4468fb5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b76bbbfbe0 0x55b76bbc7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:59.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.501+0000 7f4468fb5b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b76bbcaad0 0x55b76bbc7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:59.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.501+0000 7f4468fb5b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b76bab1ac0 con 0x55b76bbbf2d0 2023-07-18T03:18:59.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.501+0000 7f4468fb5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b76bab17c0 con 0x55b76bbbfbe0 2023-07-18T03:18:59.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.501+0000 7f4468fb5b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b76b995920 con 0x55b76bbcaad0 2023-07-18T03:18:59.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.501+0000 7f4468fb5b80 10 client.0 _ll_register_callbacks cb 0x55b76baaf760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:18:59.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.502+0000 7f444e7fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b76bbbfbe0 0x55b76bbc7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:59.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.502+0000 7f444e7fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b76bbbfbe0 0x55b76bbc7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:44850/0 (socket says 172.21.15.101:44850) 2023-07-18T03:18:59.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.502+0000 7f444e7fc700 1 -- 172.21.15.101:0/2545672812 learned_addr learned my addr 172.21.15.101:0/2545672812 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:18:59.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.502+0000 7f444f7fe700 1 --2- 172.21.15.101:0/2545672812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b76bbcaad0 0x55b76bbc7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:59.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.502+0000 7f444effd700 1 --2- 172.21.15.101:0/2545672812 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b76bbbf2d0 0x55b76bbca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:59.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.502+0000 7f444e7fc700 1 -- 172.21.15.101:0/2545672812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b76bbcaad0 msgr2=0x55b76bbc7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:59.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.502+0000 7f444e7fc700 1 --2- 172.21.15.101:0/2545672812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b76bbcaad0 0x55b76bbc7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:59.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.502+0000 7f444e7fc700 1 -- 172.21.15.101:0/2545672812 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b76bbbf2d0 msgr2=0x55b76bbca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:59.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.502+0000 7f444e7fc700 1 --2- 172.21.15.101:0/2545672812 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b76bbbf2d0 0x55b76bbca4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:59.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.502+0000 7f444e7fc700 1 -- 172.21.15.101:0/2545672812 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44501a4b00 con 0x55b76bbbfbe0 2023-07-18T03:18:59.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.502+0000 7f444f7fe700 1 --2- 172.21.15.101:0/2545672812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b76bbcaad0 0x55b76bbc7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:18:59.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.502+0000 7f444e7fc700 1 --2- 172.21.15.101:0/2545672812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b76bbbfbe0 0x55b76bbc7730 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4444009990 tx=0x7f4444005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6746647edffc4de server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:18:59.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.502+0000 7f4432ff5700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:18:59.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.503+0000 7f4432ff5700 1 -- 172.21.15.101:0/2545672812 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444401d070 con 0x55b76bbbfbe0 2023-07-18T03:18:59.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.503+0000 7f4468fb5b80 1 -- 172.21.15.101:0/2545672812 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b76ba6b4f0 con 0x55b76bbbfbe0 2023-07-18T03:18:59.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.503+0000 7f4432ff5700 1 -- 172.21.15.101:0/2545672812 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f4444004910 con 0x55b76bbbfbe0 2023-07-18T03:18:59.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.503+0000 7f4432ff5700 1 -- 172.21.15.101:0/2545672812 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4444004ad0 con 0x55b76bbbfbe0 2023-07-18T03:18:59.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.503+0000 7f4468fb5b80 10 client.27842 fetch_fsmap learned FSMap version 33 2023-07-18T03:18:59.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.503+0000 7f4468fb5b80 1 -- 172.21.15.101:0/2545672812 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55b76b928c60 con 0x55b76bbbfbe0 2023-07-18T03:18:59.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.503+0000 7f4432ff5700 1 -- 172.21.15.101:0/2545672812 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b76ba6b4f0 con 0x55b76bbbfbe0 2023-07-18T03:18:59.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.503+0000 7f4432ff5700 1 -- 172.21.15.101:0/2545672812 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f444400fbd0 con 0x55b76bbbfbe0 2023-07-18T03:18:59.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.503+0000 7f4468fb5b80 10 client.27842 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:18:59.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.503+0000 7f4468fb5b80 10 client.27842 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:18:59.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.503+0000 7f4468fb5b80 20 client.27842 populate_metadata read hostname 'smithi101' 2023-07-18T03:18:59.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.503+0000 7f4468fb5b80 1 --2- 172.21.15.101:0/2545672812 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b76bbda370 0x55b76bbdc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:59.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.503+0000 7f4468fb5b80 4 client.27842 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:18:59.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.503+0000 7f4468fb5b80 1 -- 172.21.15.101:0/2545672812 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b76bb0aeb0 con 0x55b76bbda370 2023-07-18T03:18:59.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.504+0000 7f444effd700 1 --2- 172.21.15.101:0/2545672812 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b76bbda370 0x55b76bbdc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:59.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.504+0000 7f4432ff5700 10 client.27842 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:18:59.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.504+0000 7f444effd700 1 --2- 172.21.15.101:0/2545672812 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b76bbda370 0x55b76bbdc780 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13e954a2a7c1f48c server_cookie=f06e706d837850e9 in_seq=0 out_seq=0 2023-07-18T03:18:59.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.505+0000 7f4432ff5700 1 -- 172.21.15.101:0/2545672812 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b76bb0aeb0 con 0x55b76bbda370 2023-07-18T03:18:59.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.505+0000 7f4432ff5700 10 client.27842 handle_command_reply: tid=0 2023-07-18T03:18:59.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.505+0000 7f4468fb5b80 1 client.27842 shutdown 2023-07-18T03:18:59.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.505+0000 7f4468fb5b80 1 -- 172.21.15.101:0/2545672812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b76bbbfbe0 msgr2=0x55b76bbc7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:59.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.505+0000 7f4468fb5b80 1 --2- 172.21.15.101:0/2545672812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b76bbbfbe0 0x55b76bbc7730 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4444009990 tx=0x7f4444005270 comp rx=0 tx=0).stop 2023-07-18T03:18:59.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.505+0000 7f4468fb5b80 1 -- 172.21.15.101:0/2545672812 shutdown_connections 2023-07-18T03:18:59.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.505+0000 7f4468fb5b80 1 --2- 172.21.15.101:0/2545672812 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b76bbda370 0x55b76bbdc780 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:59.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.505+0000 7f4468fb5b80 1 --2- 172.21.15.101:0/2545672812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b76bbcaad0 0x55b76bbc7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:59.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.505+0000 7f4468fb5b80 1 --2- 172.21.15.101:0/2545672812 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b76bbbf2d0 0x55b76bbca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:59.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.505+0000 7f4432ff5700 0 client.27842 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:18:59.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.505+0000 7f4468fb5b80 1 --2- 172.21.15.101:0/2545672812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b76bbbfbe0 0x55b76bbc7730 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:59.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.505+0000 7f4468fb5b80 1 -- 172.21.15.101:0/2545672812 >> 172.21.15.101:0/2545672812 conn(0x55b76bbb5da0 msgr2=0x55b76bbbb230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:18:59.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.505+0000 7f4468fb5b80 1 -- 172.21.15.101:0/2545672812 shutdown_connections 2023-07-18T03:18:59.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.505+0000 7f4468fb5b80 1 -- 172.21.15.101:0/2545672812 wait complete. 2023-07-18T03:18:59.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.506+0000 7f4468fb5b80 20 client.27842 trim_cache size 0 max 16384 2023-07-18T03:18:59.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.525+0000 7f4468fb5b80 1 Processor -- start 2023-07-18T03:18:59.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.525+0000 7f4468fb5b80 1 -- start start 2023-07-18T03:18:59.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.525+0000 7f4468fb5b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b76bbbf2d0 0x55b76bbb9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:59.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.525+0000 7f4468fb5b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b76bbbfbe0 0x55b76bbc2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:59.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.525+0000 7f4468fb5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b76bbcaad0 0x55b76bbc6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:59.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.525+0000 7f4468fb5b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b76bbc0d60 con 0x55b76bbbf2d0 2023-07-18T03:18:59.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.525+0000 7f4468fb5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b76bbba350 con 0x55b76bbcaad0 2023-07-18T03:18:59.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.525+0000 7f4468fb5b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b76bbba510 con 0x55b76bbbfbe0 2023-07-18T03:18:59.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.526+0000 7f444effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b76bbbf2d0 0x55b76bbb9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:59.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.526+0000 7f444e7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b76bbbfbe0 0x55b76bbc2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:59.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.526+0000 7f444e7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b76bbbfbe0 0x55b76bbc2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41454/0 (socket says 172.21.15.101:41454) 2023-07-18T03:18:59.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.526+0000 7f444e7fc700 1 -- 172.21.15.101:0/2105213476 learned_addr learned my addr 172.21.15.101:0/2105213476 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:18:59.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.526+0000 7f444f7fe700 1 --2- 172.21.15.101:0/2105213476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b76bbcaad0 0x55b76bbc6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:59.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.526+0000 7f444e7fc700 1 -- 172.21.15.101:0/2105213476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b76bbcaad0 msgr2=0x55b76bbc6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:59.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.526+0000 7f444e7fc700 1 --2- 172.21.15.101:0/2105213476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b76bbcaad0 0x55b76bbc6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:59.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.526+0000 7f444e7fc700 1 -- 172.21.15.101:0/2105213476 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b76bbbf2d0 msgr2=0x55b76bbb9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:59.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.526+0000 7f444e7fc700 1 --2- 172.21.15.101:0/2105213476 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b76bbbf2d0 0x55b76bbb9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:59.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.526+0000 7f444e7fc700 1 -- 172.21.15.101:0/2105213476 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b76b966250 con 0x55b76bbbfbe0 2023-07-18T03:18:59.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.526+0000 7f444e7fc700 1 --2- 172.21.15.101:0/2105213476 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b76bbbfbe0 0x55b76bbc2a90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4444000c00 tx=0x7f4444024640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e74f952b1659ded5 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:18:59.525 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.527+0000 7f4435ffb700 1 -- 172.21.15.101:0/2105213476 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444401d070 con 0x55b76bbbfbe0 2023-07-18T03:18:59.525 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.527+0000 7f4435ffb700 1 -- 172.21.15.101:0/2105213476 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f444401e040 con 0x55b76bbbfbe0 2023-07-18T03:18:59.525 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.527+0000 7f4435ffb700 1 -- 172.21.15.101:0/2105213476 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4444004b00 con 0x55b76bbbfbe0 2023-07-18T03:18:59.526 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.527+0000 7f4468fb5b80 1 -- 172.21.15.101:0/2105213476 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b76bbbfbe0 msgr2=0x55b76bbc2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:59.526 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.527+0000 7f4468fb5b80 1 --2- 172.21.15.101:0/2105213476 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b76bbbfbe0 0x55b76bbc2a90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4444000c00 tx=0x7f4444024640 comp rx=0 tx=0).stop 2023-07-18T03:18:59.526 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.527+0000 7f4468fb5b80 1 -- 172.21.15.101:0/2105213476 shutdown_connections 2023-07-18T03:18:59.526 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.527+0000 7f4468fb5b80 1 --2- 172.21.15.101:0/2105213476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b76bbcaad0 0x55b76bbc6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:59.526 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.527+0000 7f4468fb5b80 1 --2- 172.21.15.101:0/2105213476 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b76bbbf2d0 0x55b76bbb9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:59.526 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.527+0000 7f4468fb5b80 1 --2- 172.21.15.101:0/2105213476 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b76bbbfbe0 0x55b76bbc2a90 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:59.526 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.527+0000 7f4468fb5b80 1 -- 172.21.15.101:0/2105213476 >> 172.21.15.101:0/2105213476 conn(0x55b76bbb5da0 msgr2=0x55b76bbd66c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:18:59.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.527+0000 7f4468fb5b80 1 -- 172.21.15.101:0/2105213476 shutdown_connections 2023-07-18T03:18:59.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.527+0000 7f4468fb5b80 1 -- 172.21.15.101:0/2105213476 wait complete. 2023-07-18T03:18:59.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.528+0000 7f4468fb5b80 1 -- start start 2023-07-18T03:18:59.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.528+0000 7f4468fb5b80 1 Processor -- start 2023-07-18T03:18:59.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.529+0000 7f4468fb5b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b76bbbf2d0 0x55b76bbd0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:59.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.529+0000 7f4468fb5b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b76bbbfbe0 0x55b76bbd13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:59.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.529+0000 7f4468fb5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b76bbcaad0 0x55b76bbcf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:59.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.529+0000 7f4468fb5b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b76bbcfa60 con 0x55b76bbbfbe0 2023-07-18T03:18:59.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.529+0000 7f4468fb5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b76bbcfbd0 con 0x55b76bbcaad0 2023-07-18T03:18:59.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.529+0000 7f4468fb5b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b76bbcfd90 con 0x55b76bbbf2d0 2023-07-18T03:18:59.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.529+0000 7f4468fb5b80 10 client.0 _ll_register_callbacks cb 0x7f444400bf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:18:59.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.529+0000 7f444effd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b76bbbf2d0 0x55b76bbd0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:59.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.529+0000 7f444effd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b76bbbf2d0 0x55b76bbd0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41464/0 (socket says 172.21.15.101:41464) 2023-07-18T03:18:59.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.529+0000 7f444effd700 1 -- 172.21.15.101:0/2767406166 learned_addr learned my addr 172.21.15.101:0/2767406166 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:18:59.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.529+0000 7f444e7fc700 1 --2- 172.21.15.101:0/2767406166 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b76bbbfbe0 0x55b76bbd13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:59.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.529+0000 7f444f7fe700 1 --2- 172.21.15.101:0/2767406166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b76bbcaad0 0x55b76bbcf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:59.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.529+0000 7f444effd700 1 -- 172.21.15.101:0/2767406166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b76bbcaad0 msgr2=0x55b76bbcf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:59.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.529+0000 7f444effd700 1 --2- 172.21.15.101:0/2767406166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b76bbcaad0 0x55b76bbcf490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:59.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.530+0000 7f444effd700 1 -- 172.21.15.101:0/2767406166 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b76bbbfbe0 msgr2=0x55b76bbd13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:18:59.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.530+0000 7f444effd700 1 --2- 172.21.15.101:0/2767406166 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b76bbbfbe0 0x55b76bbd13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:59.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.530+0000 7f444effd700 1 -- 172.21.15.101:0/2767406166 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b76b928c60 con 0x55b76bbbf2d0 2023-07-18T03:18:59.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.530+0000 7f444effd700 1 --2- 172.21.15.101:0/2767406166 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b76bbbf2d0 0x55b76bbd0e80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f44400cd1d0 tx=0x7f4440099cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd6fdefec1766657 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:18:59.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.530+0000 7f4432ff5700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:18:59.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.530+0000 7f4432ff5700 1 -- 172.21.15.101:0/2767406166 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44400a2370 con 0x55b76bbbf2d0 2023-07-18T03:18:59.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.530+0000 7f4432ff5700 1 -- 172.21.15.101:0/2767406166 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f4440099410 con 0x55b76bbbf2d0 2023-07-18T03:18:59.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.530+0000 7f4468fb5b80 1 -- 172.21.15.101:0/2767406166 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f444005e850 con 0x55b76bbbf2d0 2023-07-18T03:18:59.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.530+0000 7f4432ff5700 1 -- 172.21.15.101:0/2767406166 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4440065150 con 0x55b76bbbf2d0 2023-07-18T03:18:59.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.530+0000 7f4468fb5b80 10 client.27883 fetch_fsmap learned FSMap version 33 2023-07-18T03:18:59.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.530+0000 7f4468fb5b80 1 -- 172.21.15.101:0/2767406166 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55b76b93e2b0 con 0x55b76bbbf2d0 2023-07-18T03:18:59.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.530+0000 7f4432ff5700 1 -- 172.21.15.101:0/2767406166 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f444005e850 con 0x55b76bbbf2d0 2023-07-18T03:18:59.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.531+0000 7f4432ff5700 1 -- 172.21.15.101:0/2767406166 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f44400a1ee0 con 0x55b76bbbf2d0 2023-07-18T03:18:59.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.531+0000 7f4468fb5b80 10 client.27883 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:18:59.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.531+0000 7f4468fb5b80 10 client.27883 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:18:59.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.531+0000 7f4468fb5b80 20 client.27883 populate_metadata read hostname 'smithi101' 2023-07-18T03:18:59.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.531+0000 7f4468fb5b80 1 --2- 172.21.15.101:0/2767406166 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b76bc2efa0 0x55b76bc313b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:59.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.531+0000 7f4468fb5b80 4 client.27883 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:18:59.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.531+0000 7f4468fb5b80 1 -- 172.21.15.101:0/2767406166 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b76bacd050 con 0x55b76bc2efa0 2023-07-18T03:18:59.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.531+0000 7f444e7fc700 1 --2- 172.21.15.101:0/2767406166 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b76bc2efa0 0x55b76bc313b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:59.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.531+0000 7f4432ff5700 10 client.27883 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:18:59.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.531+0000 7f444e7fc700 1 --2- 172.21.15.101:0/2767406166 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b76bc2efa0 0x55b76bc313b0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b00a69d637381d9 server_cookie=f90b6c5bd5c43830 in_seq=0 out_seq=0 2023-07-18T03:18:59.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.532+0000 7f4432ff5700 1 -- 172.21.15.101:0/2767406166 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55b76bacd050 con 0x55b76bc2efa0 2023-07-18T03:18:59.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.532+0000 7f4432ff5700 10 client.27883 handle_command_reply: tid=0 2023-07-18T03:18:59.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.532+0000 7f4468fb5b80 1 client.27883 shutdown 2023-07-18T03:18:59.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.532+0000 7f4468fb5b80 1 -- 172.21.15.101:0/2767406166 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b76bbbf2d0 msgr2=0x55b76bbd0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:59.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.532+0000 7f4468fb5b80 1 --2- 172.21.15.101:0/2767406166 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b76bbbf2d0 0x55b76bbd0e80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f44400cd1d0 tx=0x7f4440099cf0 comp rx=0 tx=0).stop 2023-07-18T03:18:59.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.532+0000 7f4468fb5b80 1 -- 172.21.15.101:0/2767406166 shutdown_connections 2023-07-18T03:18:59.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.532+0000 7f4468fb5b80 1 --2- 172.21.15.101:0/2767406166 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b76bc2efa0 0x55b76bc313b0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:59.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.532+0000 7f4468fb5b80 1 --2- 172.21.15.101:0/2767406166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b76bbcaad0 0x55b76bbcf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:59.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.532+0000 7f4468fb5b80 1 --2- 172.21.15.101:0/2767406166 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b76bbbf2d0 0x55b76bbd0e80 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:59.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.532+0000 7f4468fb5b80 1 --2- 172.21.15.101:0/2767406166 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b76bbbfbe0 0x55b76bbd13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:59.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.532+0000 7f4432ff5700 0 client.27883 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:18:59.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.532+0000 7f4468fb5b80 1 -- 172.21.15.101:0/2767406166 >> 172.21.15.101:0/2767406166 conn(0x55b76bbb5da0 msgr2=0x55b76bbd5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:18:59.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.532+0000 7f4468fb5b80 1 -- 172.21.15.101:0/2767406166 shutdown_connections 2023-07-18T03:18:59.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.532+0000 7f4468fb5b80 1 -- 172.21.15.101:0/2767406166 wait complete. 2023-07-18T03:18:59.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.532+0000 7f4468fb5b80 20 client.27883 trim_cache size 0 max 16384 2023-07-18T03:18:59.535 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:18:59.535 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "no active scrubs running", 2023-07-18T03:18:59.536 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": {} 2023-07-18T03:18:59.536 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:18:59.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.533+0000 7f44555ff700 1 -- 172.21.15.101:0/433163865 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f44380a12c0 msgr2=0x7f44380a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:59.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.533+0000 7f44555ff700 1 --2- 172.21.15.101:0/433163865 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f44380a12c0 0x7f44380a3770 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4448009fd0 tx=0x7f444800b040 comp rx=0 tx=0).stop 2023-07-18T03:18:59.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.533+0000 7f44555ff700 1 -- 172.21.15.101:0/433163865 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f44500ffad0 msgr2=0x7f4450072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:59.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.533+0000 7f44555ff700 1 --2- 172.21.15.101:0/433163865 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f44500ffad0 0x7f4450072810 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f444000e9a0 tx=0x7f444000ecb0 comp rx=0 tx=0).stop 2023-07-18T03:18:59.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.533+0000 7f44555ff700 1 -- 172.21.15.101:0/433163865 shutdown_connections 2023-07-18T03:18:59.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.533+0000 7f44555ff700 1 --2- 172.21.15.101:0/433163865 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4450105ae0 0x7f44501a3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:59.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.533+0000 7f44555ff700 1 --2- 172.21.15.101:0/433163865 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f44500ffad0 0x7f4450072810 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:59.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.533+0000 7f44555ff700 1 --2- 172.21.15.101:0/433163865 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f44380a12c0 0x7f44380a3770 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:59.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.533+0000 7f44555ff700 1 --2- 172.21.15.101:0/433163865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4450100480 0x7f4450072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:59.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.533+0000 7f44555ff700 1 -- 172.21.15.101:0/433163865 >> 172.21.15.101:0/433163865 conn(0x7f44500fb5f0 msgr2=0x7f44501035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:18:59.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.533+0000 7f44555ff700 1 -- 172.21.15.101:0/433163865 shutdown_connections 2023-07-18T03:18:59.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.533+0000 7f44555ff700 1 -- 172.21.15.101:0/433163865 wait complete. 2023-07-18T03:18:59.547 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-18T03:18:59.547 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:18:59.623 INFO:tasks.workunit.client.0.smithi101.stdout: 1 694631 5.52 MB/sec execute 504 sec latency 2607.854 ms 2023-07-18T03:18:59.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.793+0000 7fda7d7ff700 1 -- 172.21.15.101:0/587899774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda78105b00 msgr2=0x7fda78105ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:59.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.793+0000 7fda7d7ff700 1 --2- 172.21.15.101:0/587899774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda78105b00 0x7fda78105ed0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fda68009990 tx=0x7fda68009ca0 comp rx=0 tx=0).stop 2023-07-18T03:18:59.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.793+0000 7fda7d7ff700 1 -- 172.21.15.101:0/587899774 shutdown_connections 2023-07-18T03:18:59.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.793+0000 7fda7d7ff700 1 --2- 172.21.15.101:0/587899774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda781004a0 0x7fda7810c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:59.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.793+0000 7fda7d7ff700 1 --2- 172.21.15.101:0/587899774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda78105b00 0x7fda78105ed0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:59.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.793+0000 7fda7d7ff700 1 --2- 172.21.15.101:0/587899774 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda780ffaf0 0x7fda780fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:59.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.793+0000 7fda7d7ff700 1 -- 172.21.15.101:0/587899774 >> 172.21.15.101:0/587899774 conn(0x7fda780fb630 msgr2=0x7fda780fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:18:59.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.793+0000 7fda7d7ff700 1 -- 172.21.15.101:0/587899774 shutdown_connections 2023-07-18T03:18:59.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.793+0000 7fda7d7ff700 1 -- 172.21.15.101:0/587899774 wait complete. 2023-07-18T03:18:59.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.794+0000 7fda7d7ff700 1 Processor -- start 2023-07-18T03:18:59.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.794+0000 7fda7d7ff700 1 -- start start 2023-07-18T03:18:59.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.794+0000 7fda7d7ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda780ffaf0 0x7fda7819b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:59.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.794+0000 7fda7d7ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda781004a0 0x7fda7819bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:59.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.794+0000 7fda7d7ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda78105b00 0x7fda78195a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:59.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.795+0000 7fda7d7ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda781960b0 con 0x7fda781004a0 2023-07-18T03:18:59.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.795+0000 7fda7d7ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda78196220 con 0x7fda780ffaf0 2023-07-18T03:18:59.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.795+0000 7fda7d7ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda781963e0 con 0x7fda78105b00 2023-07-18T03:18:59.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.795+0000 7fda76ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda780ffaf0 0x7fda7819b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:59.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.795+0000 7fda767fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda781004a0 0x7fda7819bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:59.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.795+0000 7fda76ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda780ffaf0 0x7fda7819b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:44888/0 (socket says 172.21.15.101:44888) 2023-07-18T03:18:59.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.795+0000 7fda76ffd700 1 -- 172.21.15.101:0/1729545640 learned_addr learned my addr 172.21.15.101:0/1729545640 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:18:59.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.795+0000 7fda777fe700 1 --2- 172.21.15.101:0/1729545640 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda78105b00 0x7fda78195a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:59.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.795+0000 7fda76ffd700 1 -- 172.21.15.101:0/1729545640 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda78105b00 msgr2=0x7fda78195a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:59.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.795+0000 7fda76ffd700 1 --2- 172.21.15.101:0/1729545640 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda78105b00 0x7fda78195a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:59.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.795+0000 7fda76ffd700 1 -- 172.21.15.101:0/1729545640 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda781004a0 msgr2=0x7fda7819bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:18:59.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.795+0000 7fda76ffd700 1 --2- 172.21.15.101:0/1729545640 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda781004a0 0x7fda7819bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:18:59.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.795+0000 7fda76ffd700 1 -- 172.21.15.101:0/1729545640 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda6c009520 con 0x7fda780ffaf0 2023-07-18T03:18:59.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.795+0000 7fda767fc700 1 --2- 172.21.15.101:0/1729545640 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda781004a0 0x7fda7819bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:18:59.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.795+0000 7fda777fe700 1 --2- 172.21.15.101:0/1729545640 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda78105b00 0x7fda78195a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:18:59.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.795+0000 7fda76ffd700 1 --2- 172.21.15.101:0/1729545640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda780ffaf0 0x7fda7819b9d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fda68005910 tx=0x7fda68004710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:18:59.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.796+0000 7fda5ffff700 1 -- 172.21.15.101:0/1729545640 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6801d070 con 0x7fda780ffaf0 2023-07-18T03:18:59.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.796+0000 7fda7d7ff700 1 -- 172.21.15.101:0/1729545640 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda70009c40 con 0x7fda780ffaf0 2023-07-18T03:18:59.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.796+0000 7fda5ffff700 1 -- 172.21.15.101:0/1729545640 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda68022470 con 0x7fda780ffaf0 2023-07-18T03:18:59.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.796+0000 7fda5ffff700 1 -- 172.21.15.101:0/1729545640 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda68017650 con 0x7fda780ffaf0 2023-07-18T03:18:59.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.796+0000 7fda7d7ff700 1 -- 172.21.15.101:0/1729545640 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda680095f0 con 0x7fda780ffaf0 2023-07-18T03:18:59.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.797+0000 7fda7d7ff700 1 -- 172.21.15.101:0/1729545640 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda78067300 con 0x7fda780ffaf0 2023-07-18T03:18:59.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.799+0000 7fda5ffff700 1 -- 172.21.15.101:0/1729545640 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fda680225e0 con 0x7fda780ffaf0 2023-07-18T03:18:59.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.800+0000 7fda5ffff700 1 --2- 172.21.15.101:0/1729545640 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fda600a1300 0x7fda600a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:18:59.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.800+0000 7fda5ffff700 1 -- 172.21.15.101:0/1729545640 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fda680ce320 con 0x7fda780ffaf0 2023-07-18T03:18:59.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.800+0000 7fda767fc700 1 --2- 172.21.15.101:0/1729545640 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fda600a1300 0x7fda600a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:18:59.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.801+0000 7fda767fc700 1 --2- 172.21.15.101:0/1729545640 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fda600a1300 0x7fda600a37b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fda6c009fd0 tx=0x7fda6c009260 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:18:59.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:18:59.805+0000 7fda5ffff700 1 -- 172.21.15.101:0/1729545640 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fda680973b0 con 0x7fda780ffaf0 2023-07-18T03:19:00.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.072+0000 7fda7d7ff700 1 -- 172.21.15.101:0/1729545640 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda781a27e0 con 0x7fda780ffaf0 2023-07-18T03:19:00.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.073+0000 7fda5ffff700 1 -- 172.21.15.101:0/1729545640 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+10858 (secure 0 0 0) 0x7fda6809a9d0 con 0x7fda780ffaf0 2023-07-18T03:19:00.068 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:19:00.068 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/4192846769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":4192846769},{"type":"v1","addr":"172.21.15.101:6839","nonce":4192846769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6841/3824340175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6840","nonce":3824340175},{"type":"v1","addr":"172.21.15.101:6841","nonce":3824340175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:04:59.361185+0000","modified":"2023-07-18T03:16:36.002358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.155:6839/2055462483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2055462483},{"type":"v1","addr":"172.21.15.155:6839","nonce":2055462483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:19:00.070 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.076+0000 7fda7d7ff700 1 -- 172.21.15.101:0/1729545640 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fda600a1300 msgr2=0x7fda600a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:00.070 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.076+0000 7fda7d7ff700 1 --2- 172.21.15.101:0/1729545640 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fda600a1300 0x7fda600a37b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fda6c009fd0 tx=0x7fda6c009260 comp rx=0 tx=0).stop 2023-07-18T03:19:00.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.076+0000 7fda7d7ff700 1 -- 172.21.15.101:0/1729545640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda780ffaf0 msgr2=0x7fda7819b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:00.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.076+0000 7fda7d7ff700 1 --2- 172.21.15.101:0/1729545640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda780ffaf0 0x7fda7819b9d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fda68005910 tx=0x7fda68004710 comp rx=0 tx=0).stop 2023-07-18T03:19:00.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.076+0000 7fda7d7ff700 1 -- 172.21.15.101:0/1729545640 shutdown_connections 2023-07-18T03:19:00.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.076+0000 7fda7d7ff700 1 --2- 172.21.15.101:0/1729545640 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fda600a1300 0x7fda600a37b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.076+0000 7fda7d7ff700 1 --2- 172.21.15.101:0/1729545640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda780ffaf0 0x7fda7819b9d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.076+0000 7fda7d7ff700 1 --2- 172.21.15.101:0/1729545640 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda78105b00 0x7fda78195a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.076+0000 7fda7d7ff700 1 --2- 172.21.15.101:0/1729545640 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda781004a0 0x7fda7819bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.076+0000 7fda7d7ff700 1 -- 172.21.15.101:0/1729545640 >> 172.21.15.101:0/1729545640 conn(0x7fda780fb630 msgr2=0x7fda780fbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:00.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.076+0000 7fda7d7ff700 1 -- 172.21.15.101:0/1729545640 shutdown_connections 2023-07-18T03:19:00.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.076+0000 7fda7d7ff700 1 -- 172.21.15.101:0/1729545640 wait complete. 2023-07-18T03:19:00.072 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 33 2023-07-18T03:19:00.090 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-18T03:19:00.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.312+0000 7f98a49ff700 1 -- 172.21.15.101:0/1562699763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98a00ffad0 msgr2=0x7f98a00fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:00.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.312+0000 7f98a49ff700 1 --2- 172.21.15.101:0/1562699763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98a00ffad0 0x7f98a00fff40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9894009990 tx=0x7f9894009ca0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.312+0000 7f98a49ff700 1 -- 172.21.15.101:0/1562699763 shutdown_connections 2023-07-18T03:19:00.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.312+0000 7f98a49ff700 1 --2- 172.21.15.101:0/1562699763 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f98a0100480 0x7f98a010c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.312+0000 7f98a49ff700 1 --2- 172.21.15.101:0/1562699763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98a0105ae0 0x7f98a0105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.312+0000 7f98a49ff700 1 --2- 172.21.15.101:0/1562699763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98a00ffad0 0x7f98a00fff40 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.312+0000 7f98a49ff700 1 -- 172.21.15.101:0/1562699763 >> 172.21.15.101:0/1562699763 conn(0x7f98a00fb5f0 msgr2=0x7f98a00fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:00.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.312+0000 7f98a49ff700 1 -- 172.21.15.101:0/1562699763 shutdown_connections 2023-07-18T03:19:00.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.312+0000 7f98a49ff700 1 -- 172.21.15.101:0/1562699763 wait complete. 2023-07-18T03:19:00.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.313+0000 7f98a49ff700 1 Processor -- start 2023-07-18T03:19:00.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.313+0000 7f98a49ff700 1 -- start start 2023-07-18T03:19:00.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.314+0000 7f98a49ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98a00ffad0 0x7f98a0197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.314+0000 7f98a49ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98a0100480 0x7f98a0198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.314+0000 7f98a49ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f98a0105ae0 0x7f98a019c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.314+0000 7f98a49ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a019cdf0 con 0x7f98a0105ae0 2023-07-18T03:19:00.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.314+0000 7f98a49ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a019cf60 con 0x7f98a0100480 2023-07-18T03:19:00.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.314+0000 7f98a49ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a019d120 con 0x7f98a00ffad0 2023-07-18T03:19:00.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.314+0000 7f989dd9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98a0100480 0x7f98a0198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.314+0000 7f989e59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98a00ffad0 0x7f98a0197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.314+0000 7f989dd9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98a0100480 0x7f98a0198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:44894/0 (socket says 172.21.15.101:44894) 2023-07-18T03:19:00.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.314+0000 7f989dd9b700 1 -- 172.21.15.101:0/2871706521 learned_addr learned my addr 172.21.15.101:0/2871706521 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:00.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.314+0000 7f989ed9d700 1 --2- 172.21.15.101:0/2871706521 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f98a0105ae0 0x7f98a019c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.315+0000 7f989e59c700 1 -- 172.21.15.101:0/2871706521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98a0100480 msgr2=0x7f98a0198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:00.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.315+0000 7f989e59c700 1 --2- 172.21.15.101:0/2871706521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98a0100480 0x7f98a0198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.315+0000 7f989e59c700 1 -- 172.21.15.101:0/2871706521 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f98a0105ae0 msgr2=0x7f98a019c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:00.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.315+0000 7f989e59c700 1 --2- 172.21.15.101:0/2871706521 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f98a0105ae0 0x7f98a019c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.315+0000 7f989e59c700 1 -- 172.21.15.101:0/2871706521 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98940095f0 con 0x7f98a00ffad0 2023-07-18T03:19:00.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.315+0000 7f989e59c700 1 --2- 172.21.15.101:0/2871706521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98a00ffad0 0x7f98a0197e10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f989000e9f0 tx=0x7f989000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:00.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.315+0000 7f988f7fe700 1 -- 172.21.15.101:0/2871706521 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989000c9c0 con 0x7f98a00ffad0 2023-07-18T03:19:00.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.315+0000 7f988f7fe700 1 -- 172.21.15.101:0/2871706521 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9890010430 con 0x7f98a00ffad0 2023-07-18T03:19:00.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.315+0000 7f98a49ff700 1 -- 172.21.15.101:0/2871706521 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98a019d400 con 0x7f98a00ffad0 2023-07-18T03:19:00.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.315+0000 7f988f7fe700 1 -- 172.21.15.101:0/2871706521 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9890018680 con 0x7f98a00ffad0 2023-07-18T03:19:00.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.315+0000 7f98a49ff700 1 -- 172.21.15.101:0/2871706521 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a01a4c40 con 0x7f98a00ffad0 2023-07-18T03:19:00.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.319+0000 7f988f7fe700 1 -- 172.21.15.101:0/2871706521 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f9890010860 con 0x7f98a00ffad0 2023-07-18T03:19:00.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.321+0000 7f98b8282b80 1 Processor -- start 2023-07-18T03:19:00.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.321+0000 7f98b8282b80 1 -- start start 2023-07-18T03:19:00.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.321+0000 7f988f7fe700 1 --2- 172.21.15.101:0/2871706521 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f98880a12c0 0x7f98880a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.322+0000 7f98b8282b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56120f77b2d0 0x56120f77b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.322+0000 7f98b8282b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56120f77bbe0 0x56120f786590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.322+0000 7f988f7fe700 1 -- 172.21.15.101:0/2871706521 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f9890014070 con 0x7f98a00ffad0 2023-07-18T03:19:00.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.322+0000 7f98b8282b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56120f786ad0 0x56120f788eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.322+0000 7f98b8282b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56120f681cb0 con 0x56120f786ad0 2023-07-18T03:19:00.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.322+0000 7f98b8282b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56120f6d4a40 con 0x56120f77b2d0 2023-07-18T03:19:00.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.322+0000 7f98b8282b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56120f6cded0 con 0x56120f77bbe0 2023-07-18T03:19:00.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.322+0000 7f989ed9d700 1 --2- 172.21.15.101:0/2871706521 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f98880a12c0 0x7f98880a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.322+0000 7f989ed9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56120f786ad0 0x56120f788eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.322+0000 7f989dd9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56120f77bbe0 0x56120f786590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.322+0000 7f989dd9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56120f77bbe0 0x56120f786590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41516/0 (socket says 172.21.15.101:41516) 2023-07-18T03:19:00.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.322+0000 7f989ed9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56120f786ad0 0x56120f788eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:59754/0 (socket says 172.21.15.101:59754) 2023-07-18T03:19:00.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.322+0000 7f989dd9b700 1 -- 172.21.15.101:0/3816209703 learned_addr learned my addr 172.21.15.101:0/3816209703 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:00.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.322+0000 7f989e59c700 1 --2- 172.21.15.101:0/3816209703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56120f77b2d0 0x56120f77b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.323+0000 7f989dd9b700 1 -- 172.21.15.101:0/3816209703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56120f77b2d0 msgr2=0x56120f77b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:00.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.323+0000 7f989dd9b700 1 --2- 172.21.15.101:0/3816209703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56120f77b2d0 0x56120f77b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.323+0000 7f989dd9b700 1 -- 172.21.15.101:0/3816209703 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56120f786ad0 msgr2=0x56120f788eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:00.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.323+0000 7f989dd9b700 1 --2- 172.21.15.101:0/3816209703 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56120f786ad0 0x56120f788eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.323+0000 7f989dd9b700 1 -- 172.21.15.101:0/3816209703 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56120f4fbde0 con 0x56120f77bbe0 2023-07-18T03:19:00.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.323+0000 7f989dd9b700 1 --2- 172.21.15.101:0/3816209703 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56120f77bbe0 0x56120f786590 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9894005f30 tx=0x7f989400b610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8695387d430c5f2 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:00.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.323+0000 7f989ed9d700 1 --2- 172.21.15.101:0/2871706521 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f98880a12c0 0x7f98880a3770 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9898009ec0 tx=0x7f98980092f0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:00.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.323+0000 7f988cff9700 1 -- 172.21.15.101:0/3816209703 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989401d070 con 0x56120f77bbe0 2023-07-18T03:19:00.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.323+0000 7f988cff9700 1 -- 172.21.15.101:0/3816209703 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f989401e040 con 0x56120f77bbe0 2023-07-18T03:19:00.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.323+0000 7f988cff9700 1 -- 172.21.15.101:0/3816209703 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98940048a0 con 0x56120f77bbe0 2023-07-18T03:19:00.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.324+0000 7f98b8282b80 1 -- 172.21.15.101:0/3816209703 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56120f77bbe0 msgr2=0x56120f786590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:00.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.324+0000 7f98b8282b80 1 --2- 172.21.15.101:0/3816209703 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56120f77bbe0 0x56120f786590 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9894005f30 tx=0x7f989400b610 comp rx=0 tx=0).stop 2023-07-18T03:19:00.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.324+0000 7f98b8282b80 1 -- 172.21.15.101:0/3816209703 shutdown_connections 2023-07-18T03:19:00.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.324+0000 7f98b8282b80 1 --2- 172.21.15.101:0/3816209703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56120f77b2d0 0x56120f77b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.324+0000 7f98b8282b80 1 --2- 172.21.15.101:0/3816209703 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56120f786ad0 0x56120f788eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.324+0000 7f98b8282b80 1 --2- 172.21.15.101:0/3816209703 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56120f77bbe0 0x56120f786590 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.324+0000 7f98b8282b80 1 -- 172.21.15.101:0/3816209703 >> 172.21.15.101:0/3816209703 conn(0x56120f771da0 msgr2=0x56120f7776a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:00.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.324+0000 7f98b8282b80 1 -- 172.21.15.101:0/3816209703 shutdown_connections 2023-07-18T03:19:00.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.324+0000 7f98b8282b80 1 -- 172.21.15.101:0/3816209703 wait complete. 2023-07-18T03:19:00.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.325+0000 7f98b8282b80 1 -- start start 2023-07-18T03:19:00.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.325+0000 7f98b8282b80 1 Processor -- start 2023-07-18T03:19:00.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.326+0000 7f98b8282b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56120f77b2d0 0x56120f786460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.326+0000 7f98b8282b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56120f77bbe0 0x56120f7836e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.326+0000 7f98b8282b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56120f786ad0 0x56120f783c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.326+0000 7f98b8282b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56120f66dac0 con 0x56120f786ad0 2023-07-18T03:19:00.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.326+0000 7f98b8282b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56120f66d7c0 con 0x56120f77bbe0 2023-07-18T03:19:00.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.326+0000 7f98b8282b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56120f551920 con 0x56120f77b2d0 2023-07-18T03:19:00.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.326+0000 7f98b8282b80 10 client.0 _ll_register_callbacks cb 0x56120f66b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:19:00.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.326+0000 7f989e59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56120f77b2d0 0x56120f786460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.326+0000 7f989e59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56120f77b2d0 0x56120f786460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41526/0 (socket says 172.21.15.101:41526) 2023-07-18T03:19:00.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.326+0000 7f989e59c700 1 -- 172.21.15.101:0/4101457127 learned_addr learned my addr 172.21.15.101:0/4101457127 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:00.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.326+0000 7f989dd9b700 1 --2- 172.21.15.101:0/4101457127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56120f77bbe0 0x56120f7836e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.326+0000 7f989ed9d700 1 --2- 172.21.15.101:0/4101457127 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56120f786ad0 0x56120f783c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.326+0000 7f989dd9b700 1 -- 172.21.15.101:0/4101457127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56120f77b2d0 msgr2=0x56120f786460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:00.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.326+0000 7f989dd9b700 1 --2- 172.21.15.101:0/4101457127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56120f77b2d0 0x56120f786460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.326+0000 7f989dd9b700 1 -- 172.21.15.101:0/4101457127 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56120f786ad0 msgr2=0x56120f783c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:00.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.327+0000 7f989dd9b700 1 --2- 172.21.15.101:0/4101457127 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56120f786ad0 0x56120f783c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.327+0000 7f989dd9b700 1 -- 172.21.15.101:0/4101457127 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a010d210 con 0x56120f77bbe0 2023-07-18T03:19:00.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.327+0000 7f989e59c700 1 --2- 172.21.15.101:0/4101457127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56120f77b2d0 0x56120f786460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:19:00.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.327+0000 7f989dd9b700 1 --2- 172.21.15.101:0/4101457127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56120f77bbe0 0x56120f7836e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9894006010 tx=0x7f989400faf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da728fcf44b8310b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:00.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.327+0000 7f9885ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:19:00.327 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.327+0000 7f9885ffb700 1 -- 172.21.15.101:0/4101457127 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989401d040 con 0x56120f77bbe0 2023-07-18T03:19:00.327 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.327+0000 7f98b8282b80 1 -- 172.21.15.101:0/4101457127 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56120f6274f0 con 0x56120f77bbe0 2023-07-18T03:19:00.327 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.327+0000 7f9885ffb700 1 -- 172.21.15.101:0/4101457127 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f9894005910 con 0x56120f77bbe0 2023-07-18T03:19:00.328 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.327+0000 7f9885ffb700 1 -- 172.21.15.101:0/4101457127 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9894005030 con 0x56120f77bbe0 2023-07-18T03:19:00.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.327+0000 7f98b8282b80 10 client.27866 fetch_fsmap learned FSMap version 33 2023-07-18T03:19:00.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.327+0000 7f98b8282b80 1 -- 172.21.15.101:0/4101457127 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x56120f4e4c60 con 0x56120f77bbe0 2023-07-18T03:19:00.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.328+0000 7f9885ffb700 1 -- 172.21.15.101:0/4101457127 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56120f6274f0 con 0x56120f77bbe0 2023-07-18T03:19:00.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.328+0000 7f9885ffb700 1 -- 172.21.15.101:0/4101457127 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9894004b00 con 0x56120f77bbe0 2023-07-18T03:19:00.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.328+0000 7f98b8282b80 10 client.27866 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:19:00.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.328+0000 7f98b8282b80 10 client.27866 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:19:00.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.328+0000 7f98b8282b80 20 client.27866 populate_metadata read hostname 'smithi101' 2023-07-18T03:19:00.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.328+0000 7f98b8282b80 1 --2- 172.21.15.101:0/4101457127 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x56120f79e6d0 0x56120f7a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.328+0000 7f98b8282b80 4 client.27866 mds_command: new command op to 24541 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:19:00.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.328+0000 7f98b8282b80 1 -- 172.21.15.101:0/4101457127 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56120f6c6eb0 con 0x56120f79e6d0 2023-07-18T03:19:00.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.328+0000 7f989e59c700 1 --2- 172.21.15.101:0/4101457127 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x56120f79e6d0 0x56120f7a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.329+0000 7f989e59c700 1 --2- 172.21.15.101:0/4101457127 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x56120f79e6d0 0x56120f7a0ae0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=77f234cd1d09ed89 server_cookie=461d198a287c4e68 in_seq=0 out_seq=0 2023-07-18T03:19:00.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.329+0000 7f9885ffb700 10 client.27866 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:19:00.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.329+0000 7f9885ffb700 1 -- 172.21.15.101:0/4101457127 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56120f6c6eb0 con 0x56120f79e6d0 2023-07-18T03:19:00.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.329+0000 7f9885ffb700 10 client.27866 handle_command_reply: tid=0 2023-07-18T03:19:00.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.329+0000 7f98b8282b80 1 client.27866 shutdown 2023-07-18T03:19:00.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.330+0000 7f98b8282b80 1 -- 172.21.15.101:0/4101457127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56120f77bbe0 msgr2=0x56120f7836e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:00.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.330+0000 7f98b8282b80 1 --2- 172.21.15.101:0/4101457127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56120f77bbe0 0x56120f7836e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9894006010 tx=0x7f989400faf0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.330+0000 7f98b8282b80 1 -- 172.21.15.101:0/4101457127 shutdown_connections 2023-07-18T03:19:00.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.330+0000 7f98b8282b80 1 --2- 172.21.15.101:0/4101457127 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x56120f79e6d0 0x56120f7a0ae0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.330+0000 7f98b8282b80 1 --2- 172.21.15.101:0/4101457127 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56120f786ad0 0x56120f783c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.330+0000 7f98b8282b80 1 --2- 172.21.15.101:0/4101457127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56120f77b2d0 0x56120f786460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.330+0000 7f98b8282b80 1 --2- 172.21.15.101:0/4101457127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56120f77bbe0 0x56120f7836e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.330+0000 7f98b8282b80 1 -- 172.21.15.101:0/4101457127 >> 172.21.15.101:0/4101457127 conn(0x56120f771da0 msgr2=0x56120f777230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:00.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.330+0000 7f9885ffb700 0 client.27866 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:19:00.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.330+0000 7f98b8282b80 1 -- 172.21.15.101:0/4101457127 shutdown_connections 2023-07-18T03:19:00.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.330+0000 7f98b8282b80 1 -- 172.21.15.101:0/4101457127 wait complete. 2023-07-18T03:19:00.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.330+0000 7f98b8282b80 20 client.27866 trim_cache size 0 max 16384 2023-07-18T03:19:00.338 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.343+0000 7f98b8282b80 1 Processor -- start 2023-07-18T03:19:00.339 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.344+0000 7f98b8282b80 1 -- start start 2023-07-18T03:19:00.339 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.344+0000 7f98b8282b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56120f77b2d0 0x56120f783c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.339 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.344+0000 7f98b8282b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56120f77bbe0 0x56120f782f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.339 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.344+0000 7f98b8282b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56120f786ad0 0x56120f795d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.339 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.344+0000 7f98b8282b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56120f7834c0 con 0x56120f786ad0 2023-07-18T03:19:00.339 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.344+0000 7f98b8282b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56120f77d1a0 con 0x56120f77b2d0 2023-07-18T03:19:00.339 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.344+0000 7f98b8282b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56120f7962b0 con 0x56120f77bbe0 2023-07-18T03:19:00.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.344+0000 7f989e59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56120f77b2d0 0x56120f783c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.344+0000 7f989ed9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56120f786ad0 0x56120f795d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.344+0000 7f989e59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56120f77b2d0 0x56120f783c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:44924/0 (socket says 172.21.15.101:44924) 2023-07-18T03:19:00.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.344+0000 7f989e59c700 1 -- 172.21.15.101:0/3833630981 learned_addr learned my addr 172.21.15.101:0/3833630981 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:00.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.344+0000 7f989dd9b700 1 --2- 172.21.15.101:0/3833630981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56120f77bbe0 0x56120f782f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.344+0000 7f989ed9d700 1 -- 172.21.15.101:0/3833630981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56120f77bbe0 msgr2=0x56120f782f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:00.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.344+0000 7f989ed9d700 1 --2- 172.21.15.101:0/3833630981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56120f77bbe0 0x56120f782f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.344+0000 7f989ed9d700 1 -- 172.21.15.101:0/3833630981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56120f77b2d0 msgr2=0x56120f783c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:00.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.344+0000 7f989ed9d700 1 --2- 172.21.15.101:0/3833630981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56120f77b2d0 0x56120f783c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.344+0000 7f989ed9d700 1 -- 172.21.15.101:0/3833630981 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56120f522250 con 0x56120f786ad0 2023-07-18T03:19:00.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.344+0000 7f989e59c700 1 --2- 172.21.15.101:0/3833630981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56120f77b2d0 0x56120f783c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:19:00.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.345+0000 7f989ed9d700 1 --2- 172.21.15.101:0/3833630981 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56120f786ad0 0x56120f795d70 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9898009fd0 tx=0x7f989801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed942a027e57320e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:00.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.345+0000 7f988d7fa700 1 -- 172.21.15.101:0/3833630981 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989800e070 con 0x56120f786ad0 2023-07-18T03:19:00.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.345+0000 7f988d7fa700 1 -- 172.21.15.101:0/3833630981 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f989800f040 con 0x56120f786ad0 2023-07-18T03:19:00.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.345+0000 7f988d7fa700 1 -- 172.21.15.101:0/3833630981 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9898004020 con 0x56120f786ad0 2023-07-18T03:19:00.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.346+0000 7f98b8282b80 1 -- 172.21.15.101:0/3833630981 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56120f786ad0 msgr2=0x56120f795d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:00.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.346+0000 7f98b8282b80 1 --2- 172.21.15.101:0/3833630981 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56120f786ad0 0x56120f795d70 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9898009fd0 tx=0x7f989801d000 comp rx=0 tx=0).stop 2023-07-18T03:19:00.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.346+0000 7f98b8282b80 1 -- 172.21.15.101:0/3833630981 shutdown_connections 2023-07-18T03:19:00.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.346+0000 7f98b8282b80 1 --2- 172.21.15.101:0/3833630981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56120f77b2d0 0x56120f783c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.346+0000 7f98b8282b80 1 --2- 172.21.15.101:0/3833630981 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56120f786ad0 0x56120f795d70 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.346+0000 7f98b8282b80 1 --2- 172.21.15.101:0/3833630981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56120f77bbe0 0x56120f782f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.346+0000 7f98b8282b80 1 -- 172.21.15.101:0/3833630981 >> 172.21.15.101:0/3833630981 conn(0x56120f771da0 msgr2=0x56120f7a5400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:00.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.346+0000 7f98b8282b80 1 -- 172.21.15.101:0/3833630981 shutdown_connections 2023-07-18T03:19:00.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.346+0000 7f98b8282b80 1 -- 172.21.15.101:0/3833630981 wait complete. 2023-07-18T03:19:00.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.346+0000 7f98b8282b80 1 -- start start 2023-07-18T03:19:00.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.346+0000 7f98b8282b80 1 Processor -- start 2023-07-18T03:19:00.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.347+0000 7f98b8282b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56120f77b2d0 0x56120f780d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.347+0000 7f98b8282b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56120f77bbe0 0x56120f781290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.347+0000 7f98b8282b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56120f786ad0 0x56120f77f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.347+0000 7f98b8282b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56120f780090 con 0x56120f77b2d0 2023-07-18T03:19:00.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.347+0000 7f98b8282b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56120f780200 con 0x56120f77bbe0 2023-07-18T03:19:00.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.347+0000 7f98b8282b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56120f7803c0 con 0x56120f786ad0 2023-07-18T03:19:00.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.347+0000 7f98b8282b80 10 client.0 _ll_register_callbacks cb 0x7f9898008750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:19:00.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.347+0000 7f989dd9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56120f77bbe0 0x56120f781290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.347+0000 7f989ed9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56120f786ad0 0x56120f77f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.347+0000 7f989e59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56120f77b2d0 0x56120f780d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.347+0000 7f989dd9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56120f77bbe0 0x56120f781290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:44938/0 (socket says 172.21.15.101:44938) 2023-07-18T03:19:00.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.347+0000 7f989dd9b700 1 -- 172.21.15.101:0/3334924033 learned_addr learned my addr 172.21.15.101:0/3334924033 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:00.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.347+0000 7f989ed9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56120f786ad0 0x56120f77f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41550/0 (socket says 172.21.15.101:41550) 2023-07-18T03:19:00.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.347+0000 7f989dd9b700 1 -- 172.21.15.101:0/3334924033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56120f786ad0 msgr2=0x56120f77f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:00.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.347+0000 7f989dd9b700 1 --2- 172.21.15.101:0/3334924033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56120f786ad0 0x56120f77f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.347+0000 7f989dd9b700 1 -- 172.21.15.101:0/3334924033 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56120f77b2d0 msgr2=0x56120f780d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:00.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.347+0000 7f989dd9b700 1 --2- 172.21.15.101:0/3334924033 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56120f77b2d0 0x56120f780d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.347+0000 7f989dd9b700 1 -- 172.21.15.101:0/3334924033 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a01a4c40 con 0x56120f77bbe0 2023-07-18T03:19:00.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.347+0000 7f989ed9d700 1 --2- 172.21.15.101:0/3334924033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56120f786ad0 0x56120f77f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:19:00.348 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.348+0000 7f989dd9b700 1 --2- 172.21.15.101:0/3334924033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56120f77bbe0 0x56120f781290 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9894000c00 tx=0x7f989400b480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=377bcf3564d3022d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:00.348 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.348+0000 7f9885ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:19:00.348 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.348+0000 7f9885ffb700 1 -- 172.21.15.101:0/3334924033 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989401d030 con 0x56120f77bbe0 2023-07-18T03:19:00.348 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.348+0000 7f98b8282b80 1 -- 172.21.15.101:0/3334924033 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f989401e040 con 0x56120f77bbe0 2023-07-18T03:19:00.348 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.348+0000 7f9885ffb700 1 -- 172.21.15.101:0/3334924033 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f9894022550 con 0x56120f77bbe0 2023-07-18T03:19:00.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.348+0000 7f9885ffb700 1 -- 172.21.15.101:0/3334924033 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f989400bb30 con 0x56120f77bbe0 2023-07-18T03:19:00.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.348+0000 7f98b8282b80 10 client.27872 fetch_fsmap learned FSMap version 33 2023-07-18T03:19:00.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.348+0000 7f98b8282b80 1 -- 172.21.15.101:0/3334924033 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f9898008570 con 0x56120f77bbe0 2023-07-18T03:19:00.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.348+0000 7f9885ffb700 1 -- 172.21.15.101:0/3334924033 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f989401e040 con 0x56120f77bbe0 2023-07-18T03:19:00.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.349+0000 7f9885ffb700 1 -- 172.21.15.101:0/3334924033 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9894022b20 con 0x56120f77bbe0 2023-07-18T03:19:00.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.349+0000 7f98b8282b80 10 client.27872 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:19:00.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.349+0000 7f98b8282b80 10 client.27872 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:19:00.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.349+0000 7f98b8282b80 20 client.27872 populate_metadata read hostname 'smithi101' 2023-07-18T03:19:00.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.349+0000 7f98b8282b80 1 --2- 172.21.15.101:0/3334924033 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x56120f7f3220 0x56120f7f5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.349+0000 7f98b8282b80 4 client.27872 mds_command: new command op to 24541 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:19:00.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.349+0000 7f98b8282b80 1 -- 172.21.15.101:0/3334924033 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56120f689050 con 0x56120f7f3220 2023-07-18T03:19:00.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.349+0000 7f989e59c700 1 --2- 172.21.15.101:0/3334924033 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x56120f7f3220 0x56120f7f5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.349+0000 7f9885ffb700 10 client.27872 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:19:00.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.349+0000 7f989e59c700 1 --2- 172.21.15.101:0/3334924033 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x56120f7f3220 0x56120f7f5600 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=de38c38b487c5d42 server_cookie=2130be52097b9f0c in_seq=0 out_seq=0 2023-07-18T03:19:00.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.350+0000 7f9885ffb700 1 -- 172.21.15.101:0/3334924033 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56120f689050 con 0x56120f7f3220 2023-07-18T03:19:00.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.350+0000 7f9885ffb700 10 client.27872 handle_command_reply: tid=0 2023-07-18T03:19:00.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.350+0000 7f98b8282b80 1 client.27872 shutdown 2023-07-18T03:19:00.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.350+0000 7f98b8282b80 1 -- 172.21.15.101:0/3334924033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56120f77bbe0 msgr2=0x56120f781290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:00.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.350+0000 7f98b8282b80 1 --2- 172.21.15.101:0/3334924033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56120f77bbe0 0x56120f781290 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9894000c00 tx=0x7f989400b480 comp rx=0 tx=0).stop 2023-07-18T03:19:00.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.350+0000 7f98b8282b80 1 -- 172.21.15.101:0/3334924033 shutdown_connections 2023-07-18T03:19:00.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.350+0000 7f98b8282b80 1 --2- 172.21.15.101:0/3334924033 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x56120f7f3220 0x56120f7f5600 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.350+0000 7f98b8282b80 1 --2- 172.21.15.101:0/3334924033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56120f786ad0 0x56120f77f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.350+0000 7f98b8282b80 1 --2- 172.21.15.101:0/3334924033 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56120f77b2d0 0x56120f780d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.350+0000 7f98b8282b80 1 --2- 172.21.15.101:0/3334924033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56120f77bbe0 0x56120f781290 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.350+0000 7f98b8282b80 1 -- 172.21.15.101:0/3334924033 >> 172.21.15.101:0/3334924033 conn(0x56120f771da0 msgr2=0x56120f79a520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:00.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.350+0000 7f9885ffb700 0 client.27872 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:19:00.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.350+0000 7f98b8282b80 1 -- 172.21.15.101:0/3334924033 shutdown_connections 2023-07-18T03:19:00.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.350+0000 7f98b8282b80 1 -- 172.21.15.101:0/3334924033 wait complete. 2023-07-18T03:19:00.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.350+0000 7f98b8282b80 20 client.27872 trim_cache size 0 max 16384 2023-07-18T03:19:00.354 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:19:00.355 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.351+0000 7f98a49ff700 1 -- 172.21.15.101:0/2871706521 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f98880a12c0 msgr2=0x7f98880a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:00.355 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.351+0000 7f98a49ff700 1 --2- 172.21.15.101:0/2871706521 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f98880a12c0 0x7f98880a3770 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9898009ec0 tx=0x7f98980092f0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.355 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.351+0000 7f98a49ff700 1 -- 172.21.15.101:0/2871706521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98a00ffad0 msgr2=0x7f98a0197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:00.355 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.351+0000 7f98a49ff700 1 --2- 172.21.15.101:0/2871706521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98a00ffad0 0x7f98a0197e10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f989000e9f0 tx=0x7f989000ed00 comp rx=0 tx=0).stop 2023-07-18T03:19:00.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.351+0000 7f98a49ff700 1 -- 172.21.15.101:0/2871706521 shutdown_connections 2023-07-18T03:19:00.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.351+0000 7f98a49ff700 1 --2- 172.21.15.101:0/2871706521 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f98a0105ae0 0x7f98a019c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.351+0000 7f98a49ff700 1 --2- 172.21.15.101:0/2871706521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98a00ffad0 0x7f98a0197e10 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.351+0000 7f98a49ff700 1 --2- 172.21.15.101:0/2871706521 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f98880a12c0 0x7f98880a3770 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.351+0000 7f98a49ff700 1 --2- 172.21.15.101:0/2871706521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98a0100480 0x7f98a0198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.351+0000 7f98a49ff700 1 -- 172.21.15.101:0/2871706521 >> 172.21.15.101:0/2871706521 conn(0x7f98a00fb5f0 msgr2=0x7f98a010ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:00.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.351+0000 7f98a49ff700 1 -- 172.21.15.101:0/2871706521 shutdown_connections 2023-07-18T03:19:00.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.351+0000 7f98a49ff700 1 -- 172.21.15.101:0/2871706521 wait complete. 2023-07-18T03:19:00.363 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-18T03:19:00.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.584+0000 7feb183ff700 1 -- 172.21.15.101:0/1116184238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb100ffad0 msgr2=0x7feb100fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:00.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.584+0000 7feb183ff700 1 --2- 172.21.15.101:0/1116184238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb100ffad0 0x7feb100fff40 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7feb000098f0 tx=0x7feb00009c00 comp rx=0 tx=0).stop 2023-07-18T03:19:00.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.584+0000 7feb183ff700 1 -- 172.21.15.101:0/1116184238 shutdown_connections 2023-07-18T03:19:00.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.584+0000 7feb183ff700 1 --2- 172.21.15.101:0/1116184238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb10100480 0x7feb1010c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.584+0000 7feb183ff700 1 --2- 172.21.15.101:0/1116184238 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7feb10105ae0 0x7feb10105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.584+0000 7feb183ff700 1 --2- 172.21.15.101:0/1116184238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb100ffad0 0x7feb100fff40 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.584+0000 7feb183ff700 1 -- 172.21.15.101:0/1116184238 >> 172.21.15.101:0/1116184238 conn(0x7feb100fb5f0 msgr2=0x7feb100fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:00.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.585+0000 7feb183ff700 1 -- 172.21.15.101:0/1116184238 shutdown_connections 2023-07-18T03:19:00.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.585+0000 7feb183ff700 1 -- 172.21.15.101:0/1116184238 wait complete. 2023-07-18T03:19:00.582 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.586+0000 7feb183ff700 1 Processor -- start 2023-07-18T03:19:00.582 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.586+0000 7feb183ff700 1 -- start start 2023-07-18T03:19:00.582 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.586+0000 7feb183ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb100ffad0 0x7feb10072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.582 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.586+0000 7feb183ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7feb10100480 0x7feb10072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.582 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.586+0000 7feb183ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb10105ae0 0x7feb101a3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.586+0000 7feb183ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb10073580 con 0x7feb10100480 2023-07-18T03:19:00.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.586+0000 7feb183ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb101a4320 con 0x7feb10105ae0 2023-07-18T03:19:00.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.586+0000 7feb183ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb101a4460 con 0x7feb100ffad0 2023-07-18T03:19:00.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.586+0000 7feb1599a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7feb10100480 0x7feb10072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.586+0000 7feb1619b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb100ffad0 0x7feb10072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.586+0000 7feb1599a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7feb10100480 0x7feb10072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:59792/0 (socket says 172.21.15.101:59792) 2023-07-18T03:19:00.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.586+0000 7feb1599a700 1 -- 172.21.15.101:0/3255996854 learned_addr learned my addr 172.21.15.101:0/3255996854 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:00.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.586+0000 7feb1699c700 1 --2- 172.21.15.101:0/3255996854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb10105ae0 0x7feb101a3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.587+0000 7feb1619b700 1 -- 172.21.15.101:0/3255996854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb10105ae0 msgr2=0x7feb101a3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:00.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.587+0000 7feb1619b700 1 --2- 172.21.15.101:0/3255996854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb10105ae0 0x7feb101a3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.587+0000 7feb1619b700 1 -- 172.21.15.101:0/3255996854 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7feb10100480 msgr2=0x7feb10072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:00.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.587+0000 7feb1619b700 1 --2- 172.21.15.101:0/3255996854 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7feb10100480 0x7feb10072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.587+0000 7feb1619b700 1 -- 172.21.15.101:0/3255996854 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb00009520 con 0x7feb100ffad0 2023-07-18T03:19:00.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.587+0000 7feb1599a700 1 --2- 172.21.15.101:0/3255996854 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7feb10100480 0x7feb10072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:19:00.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.587+0000 7feb1619b700 1 --2- 172.21.15.101:0/3255996854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb100ffad0 0x7feb10072810 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7feb0c00e9f0 tx=0x7feb0c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:00.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.587+0000 7feb077fe700 1 -- 172.21.15.101:0/3255996854 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0c00c9c0 con 0x7feb100ffad0 2023-07-18T03:19:00.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.587+0000 7feb077fe700 1 -- 172.21.15.101:0/3255996854 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb0c010430 con 0x7feb100ffad0 2023-07-18T03:19:00.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.587+0000 7feb077fe700 1 -- 172.21.15.101:0/3255996854 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0c018680 con 0x7feb100ffad0 2023-07-18T03:19:00.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.588+0000 7feb183ff700 1 -- 172.21.15.101:0/3255996854 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb101a46c0 con 0x7feb100ffad0 2023-07-18T03:19:00.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.588+0000 7feb183ff700 1 -- 172.21.15.101:0/3255996854 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb101a4b50 con 0x7feb100ffad0 2023-07-18T03:19:00.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.591+0000 7feb077fe700 1 -- 172.21.15.101:0/3255996854 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7feb0c010860 con 0x7feb100ffad0 2023-07-18T03:19:00.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.592+0000 7feb2bd70b80 1 Processor -- start 2023-07-18T03:19:00.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.592+0000 7feb077fe700 1 --2- 172.21.15.101:0/3255996854 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7feafc0a12c0 0x7feafc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.592+0000 7feb2bd70b80 1 -- start start 2023-07-18T03:19:00.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.593+0000 7feb077fe700 1 -- 172.21.15.101:0/3255996854 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7feb0c014070 con 0x7feb100ffad0 2023-07-18T03:19:00.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.593+0000 7feb1699c700 1 --2- 172.21.15.101:0/3255996854 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7feafc0a12c0 0x7feafc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.593+0000 7feb2bd70b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a9df4dd2d0 0x55a9df4dd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.593+0000 7feb2bd70b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9df4ddbe0 0x55a9df4e8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.593+0000 7feb2bd70b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9df4e8ad0 0x55a9df4eaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.593+0000 7feb2bd70b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9df3e3cb0 con 0x55a9df4ddbe0 2023-07-18T03:19:00.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.593+0000 7feb2bd70b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9df436a40 con 0x55a9df4e8ad0 2023-07-18T03:19:00.589 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.593+0000 7feb2bd70b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9df42fed0 con 0x55a9df4dd2d0 2023-07-18T03:19:00.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.593+0000 7feb1699c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9df4e8ad0 0x55a9df4eaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.593+0000 7feb1599a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9df4ddbe0 0x55a9df4e8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.593+0000 7feb1619b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a9df4dd2d0 0x55a9df4dd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.593+0000 7feb1619b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a9df4dd2d0 0x55a9df4dd6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41572/0 (socket says 172.21.15.101:41572) 2023-07-18T03:19:00.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.593+0000 7feb1699c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9df4e8ad0 0x55a9df4eaeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:44966/0 (socket says 172.21.15.101:44966) 2023-07-18T03:19:00.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.593+0000 7feb1699c700 1 -- 172.21.15.101:0/3738419962 learned_addr learned my addr 172.21.15.101:0/3738419962 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:00.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.593+0000 7feb1599a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9df4ddbe0 0x55a9df4e8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:59800/0 (socket says 172.21.15.101:59800) 2023-07-18T03:19:00.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.594+0000 7feb1699c700 1 --2- 172.21.15.101:0/3255996854 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7feafc0a12c0 0x7feafc0a3770 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7feb10074920 tx=0x7feb0800b040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:00.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.594+0000 7feb1619b700 1 -- 172.21.15.101:0/3738419962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9df4e8ad0 msgr2=0x55a9df4eaeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:00.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.594+0000 7feb1619b700 1 --2- 172.21.15.101:0/3738419962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9df4e8ad0 0x55a9df4eaeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.594+0000 7feb1619b700 1 -- 172.21.15.101:0/3738419962 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9df4ddbe0 msgr2=0x55a9df4e8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:00.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.594+0000 7feb1619b700 1 --2- 172.21.15.101:0/3738419962 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9df4ddbe0 0x55a9df4e8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.594+0000 7feb1619b700 1 -- 172.21.15.101:0/3738419962 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9df25dde0 con 0x55a9df4dd2d0 2023-07-18T03:19:00.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.594+0000 7feb1619b700 1 --2- 172.21.15.101:0/3738419962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a9df4dd2d0 0x55a9df4dd6a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7feb0c0ca550 tx=0x7feb0c099dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff71f54c277ae5c6 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:00.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.594+0000 7feb04bfe700 1 -- 172.21.15.101:0/3738419962 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0c0a2320 con 0x55a9df4dd2d0 2023-07-18T03:19:00.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.594+0000 7feb04bfe700 1 -- 172.21.15.101:0/3738419962 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb0c0105a0 con 0x55a9df4dd2d0 2023-07-18T03:19:00.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.594+0000 7feb04bfe700 1 -- 172.21.15.101:0/3738419962 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0c0a1e90 con 0x55a9df4dd2d0 2023-07-18T03:19:00.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.595+0000 7feb2bd70b80 1 -- 172.21.15.101:0/3738419962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a9df4dd2d0 msgr2=0x55a9df4dd6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:00.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.595+0000 7feb2bd70b80 1 --2- 172.21.15.101:0/3738419962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a9df4dd2d0 0x55a9df4dd6a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7feb0c0ca550 tx=0x7feb0c099dd0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.595+0000 7feb2bd70b80 1 -- 172.21.15.101:0/3738419962 shutdown_connections 2023-07-18T03:19:00.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.595+0000 7feb2bd70b80 1 --2- 172.21.15.101:0/3738419962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9df4e8ad0 0x55a9df4eaeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.595+0000 7feb2bd70b80 1 --2- 172.21.15.101:0/3738419962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a9df4dd2d0 0x55a9df4dd6a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.595+0000 7feb2bd70b80 1 --2- 172.21.15.101:0/3738419962 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9df4ddbe0 0x55a9df4e8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.595+0000 7feb2bd70b80 1 -- 172.21.15.101:0/3738419962 >> 172.21.15.101:0/3738419962 conn(0x55a9df4d3da0 msgr2=0x55a9df4d96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:00.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.595+0000 7feb2bd70b80 1 -- 172.21.15.101:0/3738419962 shutdown_connections 2023-07-18T03:19:00.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.595+0000 7feb2bd70b80 1 -- 172.21.15.101:0/3738419962 wait complete. 2023-07-18T03:19:00.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.596+0000 7feb2bd70b80 1 -- start start 2023-07-18T03:19:00.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.596+0000 7feb2bd70b80 1 Processor -- start 2023-07-18T03:19:00.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.596+0000 7feb2bd70b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9df4dd2d0 0x55a9df4e8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.596+0000 7feb2bd70b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a9df4ddbe0 0x55a9df4e56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.596+0000 7feb2bd70b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9df4e8ad0 0x55a9df4e5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.596+0000 7feb2bd70b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9df3cfac0 con 0x55a9df4e8ad0 2023-07-18T03:19:00.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.597+0000 7feb2bd70b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9df3cf7c0 con 0x55a9df4dd2d0 2023-07-18T03:19:00.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.597+0000 7feb2bd70b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9df2b3920 con 0x55a9df4ddbe0 2023-07-18T03:19:00.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.597+0000 7feb2bd70b80 10 client.0 _ll_register_callbacks cb 0x55a9df3cd760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:19:00.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.597+0000 7feb1699c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9df4e8ad0 0x55a9df4e5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.597+0000 7feb1699c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9df4e8ad0 0x55a9df4e5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:59804/0 (socket says 172.21.15.101:59804) 2023-07-18T03:19:00.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.597+0000 7feb1699c700 1 -- 172.21.15.101:0/3431929314 learned_addr learned my addr 172.21.15.101:0/3431929314 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:00.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.597+0000 7feb1619b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9df4dd2d0 0x55a9df4e8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.597+0000 7feb1599a700 1 --2- 172.21.15.101:0/3431929314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a9df4ddbe0 0x55a9df4e56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.597+0000 7feb1699c700 1 -- 172.21.15.101:0/3431929314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a9df4ddbe0 msgr2=0x55a9df4e56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:00.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.597+0000 7feb1699c700 1 --2- 172.21.15.101:0/3431929314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a9df4ddbe0 0x55a9df4e56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.597+0000 7feb1699c700 1 -- 172.21.15.101:0/3431929314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9df4dd2d0 msgr2=0x55a9df4e8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:00.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.597+0000 7feb1699c700 1 --2- 172.21.15.101:0/3431929314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9df4dd2d0 0x55a9df4e8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.597+0000 7feb1699c700 1 -- 172.21.15.101:0/3431929314 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb101a4b50 con 0x55a9df4e8ad0 2023-07-18T03:19:00.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.597+0000 7feb1619b700 1 --2- 172.21.15.101:0/3431929314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9df4dd2d0 0x55a9df4e8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:19:00.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.597+0000 7feb1699c700 1 --2- 172.21.15.101:0/3431929314 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9df4e8ad0 0x55a9df4e5cf0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7feb08008b80 tx=0x7feb08004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ee9ada3c379d4b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:00.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.597+0000 7feaf9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:19:00.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.598+0000 7feaf9ffb700 1 -- 172.21.15.101:0/3431929314 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0800e040 con 0x55a9df4e8ad0 2023-07-18T03:19:00.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.598+0000 7feb2bd70b80 1 -- 172.21.15.101:0/3431929314 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a9df3894f0 con 0x55a9df4e8ad0 2023-07-18T03:19:00.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.598+0000 7feaf9ffb700 1 -- 172.21.15.101:0/3431929314 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7feb08017ce0 con 0x55a9df4e8ad0 2023-07-18T03:19:00.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.598+0000 7feaf9ffb700 1 -- 172.21.15.101:0/3431929314 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb08017ea0 con 0x55a9df4e8ad0 2023-07-18T03:19:00.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.598+0000 7feb2bd70b80 10 client.17847 fetch_fsmap learned FSMap version 33 2023-07-18T03:19:00.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.598+0000 7feb2bd70b80 1 -- 172.21.15.101:0/3431929314 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55a9df246c60 con 0x55a9df4e8ad0 2023-07-18T03:19:00.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.598+0000 7feaf9ffb700 1 -- 172.21.15.101:0/3431929314 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a9df3894f0 con 0x55a9df4e8ad0 2023-07-18T03:19:00.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.598+0000 7feaf9ffb700 1 -- 172.21.15.101:0/3431929314 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7feb08004660 con 0x55a9df4e8ad0 2023-07-18T03:19:00.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.599+0000 7feb2bd70b80 10 client.17847 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:19:00.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.599+0000 7feb2bd70b80 10 client.17847 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:19:00.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.599+0000 7feb2bd70b80 20 client.17847 populate_metadata read hostname 'smithi101' 2023-07-18T03:19:00.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.599+0000 7feb2bd70b80 1 --2- 172.21.15.101:0/3431929314 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55a9df4f84b0 0x55a9df4fa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.599+0000 7feb2bd70b80 4 client.17847 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:19:00.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.599+0000 7feb2bd70b80 1 -- 172.21.15.101:0/3431929314 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a9df428eb0 con 0x55a9df4f84b0 2023-07-18T03:19:00.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.599+0000 7feb1619b700 1 --2- 172.21.15.101:0/3431929314 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55a9df4f84b0 0x55a9df4fa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.599+0000 7feb1619b700 1 --2- 172.21.15.101:0/3431929314 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55a9df4f84b0 0x55a9df4fa8c0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cbca17a8528971bc server_cookie=8698c5a89550b1d0 in_seq=0 out_seq=0 2023-07-18T03:19:00.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.599+0000 7feaf9ffb700 10 client.17847 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:19:00.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.600+0000 7feaf9ffb700 1 -- 172.21.15.101:0/3431929314 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a9df428eb0 con 0x55a9df4f84b0 2023-07-18T03:19:00.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.600+0000 7feaf9ffb700 10 client.17847 handle_command_reply: tid=0 2023-07-18T03:19:00.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.600+0000 7feb2bd70b80 1 client.17847 shutdown 2023-07-18T03:19:00.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.600+0000 7feb2bd70b80 1 -- 172.21.15.101:0/3431929314 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9df4e8ad0 msgr2=0x55a9df4e5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:00.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.600+0000 7feb2bd70b80 1 --2- 172.21.15.101:0/3431929314 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9df4e8ad0 0x55a9df4e5cf0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7feb08008b80 tx=0x7feb08004ad0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.600+0000 7feb2bd70b80 1 -- 172.21.15.101:0/3431929314 shutdown_connections 2023-07-18T03:19:00.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.600+0000 7feb2bd70b80 1 --2- 172.21.15.101:0/3431929314 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55a9df4f84b0 0x55a9df4fa8c0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.600+0000 7feb2bd70b80 1 --2- 172.21.15.101:0/3431929314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9df4dd2d0 0x55a9df4e8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.600+0000 7feb2bd70b80 1 --2- 172.21.15.101:0/3431929314 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9df4e8ad0 0x55a9df4e5cf0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.600+0000 7feaf9ffb700 0 client.17847 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:19:00.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.600+0000 7feb2bd70b80 1 --2- 172.21.15.101:0/3431929314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a9df4ddbe0 0x55a9df4e56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.600+0000 7feb2bd70b80 1 -- 172.21.15.101:0/3431929314 >> 172.21.15.101:0/3431929314 conn(0x55a9df4d3da0 msgr2=0x55a9df4d9230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:00.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.600+0000 7feb2bd70b80 1 -- 172.21.15.101:0/3431929314 shutdown_connections 2023-07-18T03:19:00.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.600+0000 7feb2bd70b80 1 -- 172.21.15.101:0/3431929314 wait complete. 2023-07-18T03:19:00.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.601+0000 7feb2bd70b80 20 client.17847 trim_cache size 0 max 16384 2023-07-18T03:19:00.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.614+0000 7feb2bd70b80 1 Processor -- start 2023-07-18T03:19:00.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.614+0000 7feb2bd70b80 1 -- start start 2023-07-18T03:19:00.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.614+0000 7feb2bd70b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9df4dd2d0 0x55a9df4ef550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.614+0000 7feb2bd70b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9df4ddbe0 0x55a9df4e09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.614+0000 7feb2bd70b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a9df4e8ad0 0x55a9df4e0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.614+0000 7feb2bd70b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9df4d5ec0 con 0x55a9df4ddbe0 2023-07-18T03:19:00.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.614+0000 7feb2bd70b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9df4df1a0 con 0x55a9df4dd2d0 2023-07-18T03:19:00.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.614+0000 7feb2bd70b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9df4dad20 con 0x55a9df4e8ad0 2023-07-18T03:19:00.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.614+0000 7feb1699c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a9df4e8ad0 0x55a9df4e0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.614+0000 7feb1599a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9df4ddbe0 0x55a9df4e09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.614+0000 7feb1699c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a9df4e8ad0 0x55a9df4e0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41582/0 (socket says 172.21.15.101:41582) 2023-07-18T03:19:00.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.614+0000 7feb1619b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9df4dd2d0 0x55a9df4ef550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.614+0000 7feb1699c700 1 -- 172.21.15.101:0/1072659796 learned_addr learned my addr 172.21.15.101:0/1072659796 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:00.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.614+0000 7feb1599a700 1 --2- 172.21.15.101:0/1072659796 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9df4ddbe0 0x55a9df4e09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:59812/0 (socket says 172.21.15.101:59812) 2023-07-18T03:19:00.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.615+0000 7feb1699c700 1 -- 172.21.15.101:0/1072659796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9df4dd2d0 msgr2=0x55a9df4ef550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:00.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.615+0000 7feb1699c700 1 --2- 172.21.15.101:0/1072659796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9df4dd2d0 0x55a9df4ef550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.615+0000 7feb1699c700 1 -- 172.21.15.101:0/1072659796 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9df4ddbe0 msgr2=0x55a9df4e09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:00.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.615+0000 7feb1699c700 1 --2- 172.21.15.101:0/1072659796 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9df4ddbe0 0x55a9df4e09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.615+0000 7feb1699c700 1 -- 172.21.15.101:0/1072659796 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9df284250 con 0x55a9df4e8ad0 2023-07-18T03:19:00.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.615+0000 7feb1599a700 1 --2- 172.21.15.101:0/1072659796 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9df4ddbe0 0x55a9df4e09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:19:00.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.615+0000 7feb1699c700 1 --2- 172.21.15.101:0/1072659796 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a9df4e8ad0 0x55a9df4e0f30 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7feb08004a20 tx=0x7feb080099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a719407db94f955e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:00.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.615+0000 7feb053ff700 1 -- 172.21.15.101:0/1072659796 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0800e030 con 0x55a9df4e8ad0 2023-07-18T03:19:00.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.615+0000 7feb053ff700 1 -- 172.21.15.101:0/1072659796 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb0800f040 con 0x55a9df4e8ad0 2023-07-18T03:19:00.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.615+0000 7feb053ff700 1 -- 172.21.15.101:0/1072659796 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb08029420 con 0x55a9df4e8ad0 2023-07-18T03:19:00.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.616+0000 7feb2bd70b80 1 -- 172.21.15.101:0/1072659796 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a9df4e8ad0 msgr2=0x55a9df4e0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:00.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.616+0000 7feb2bd70b80 1 --2- 172.21.15.101:0/1072659796 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a9df4e8ad0 0x55a9df4e0f30 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7feb08004a20 tx=0x7feb080099e0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.616+0000 7feb2bd70b80 1 -- 172.21.15.101:0/1072659796 shutdown_connections 2023-07-18T03:19:00.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.616+0000 7feb2bd70b80 1 --2- 172.21.15.101:0/1072659796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9df4dd2d0 0x55a9df4ef550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.616+0000 7feb2bd70b80 1 --2- 172.21.15.101:0/1072659796 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a9df4e8ad0 0x55a9df4e0f30 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.616+0000 7feb2bd70b80 1 --2- 172.21.15.101:0/1072659796 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9df4ddbe0 0x55a9df4e09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.616+0000 7feb2bd70b80 1 -- 172.21.15.101:0/1072659796 >> 172.21.15.101:0/1072659796 conn(0x55a9df4d3da0 msgr2=0x55a9df4f3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:00.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.616+0000 7feb2bd70b80 1 -- 172.21.15.101:0/1072659796 shutdown_connections 2023-07-18T03:19:00.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.616+0000 7feb2bd70b80 1 -- 172.21.15.101:0/1072659796 wait complete. 2023-07-18T03:19:00.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.616+0000 7feb2bd70b80 1 -- start start 2023-07-18T03:19:00.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.617+0000 7feb2bd70b80 1 Processor -- start 2023-07-18T03:19:00.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.617+0000 7feb2bd70b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9df4dd2d0 0x55a9df5404f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.617+0000 7feb2bd70b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9df4ddbe0 0x55a9df540a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.617+0000 7feb2bd70b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a9df4e8ad0 0x55a9df54b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.617+0000 7feb2bd70b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9df54b8c0 con 0x55a9df4ddbe0 2023-07-18T03:19:00.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.617+0000 7feb2bd70b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9df54ba00 con 0x55a9df4dd2d0 2023-07-18T03:19:00.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.617+0000 7feb2bd70b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9df54bb40 con 0x55a9df4e8ad0 2023-07-18T03:19:00.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.617+0000 7feb2bd70b80 10 client.0 _ll_register_callbacks cb 0x7feb08009500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:19:00.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.618+0000 7feb1599a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9df4ddbe0 0x55a9df540a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.618+0000 7feb1699c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a9df4e8ad0 0x55a9df54b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.618+0000 7feb1699c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a9df4e8ad0 0x55a9df54b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41592/0 (socket says 172.21.15.101:41592) 2023-07-18T03:19:00.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.618+0000 7feb1699c700 1 -- 172.21.15.101:0/2731699961 learned_addr learned my addr 172.21.15.101:0/2731699961 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:00.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.618+0000 7feb1619b700 1 --2- 172.21.15.101:0/2731699961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9df4dd2d0 0x55a9df5404f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.618+0000 7feb1699c700 1 -- 172.21.15.101:0/2731699961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9df4dd2d0 msgr2=0x55a9df5404f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:00.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.618+0000 7feb1699c700 1 --2- 172.21.15.101:0/2731699961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9df4dd2d0 0x55a9df5404f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.618+0000 7feb1699c700 1 -- 172.21.15.101:0/2731699961 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9df4ddbe0 msgr2=0x55a9df540a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:00.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.618+0000 7feb1699c700 1 --2- 172.21.15.101:0/2731699961 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9df4ddbe0 0x55a9df540a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.618+0000 7feb1699c700 1 -- 172.21.15.101:0/2731699961 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb1010d210 con 0x55a9df4e8ad0 2023-07-18T03:19:00.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.618+0000 7feb1599a700 1 --2- 172.21.15.101:0/2731699961 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9df4ddbe0 0x55a9df540a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:19:00.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.618+0000 7feb1699c700 1 --2- 172.21.15.101:0/2731699961 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a9df4e8ad0 0x55a9df54b380 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7feb0800f450 tx=0x7feb08017720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=549a058ac55d188f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:00.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.618+0000 7feaf9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:19:00.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.619+0000 7feaf9ffb700 1 -- 172.21.15.101:0/2731699961 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0800e070 con 0x55a9df4e8ad0 2023-07-18T03:19:00.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.619+0000 7feb2bd70b80 1 -- 172.21.15.101:0/2731699961 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feb0c0105a0 con 0x55a9df4e8ad0 2023-07-18T03:19:00.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.619+0000 7feaf9ffb700 1 -- 172.21.15.101:0/2731699961 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7feb08017ce0 con 0x55a9df4e8ad0 2023-07-18T03:19:00.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.619+0000 7feaf9ffb700 1 -- 172.21.15.101:0/2731699961 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb08015a30 con 0x55a9df4e8ad0 2023-07-18T03:19:00.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.619+0000 7feb2bd70b80 10 client.27925 fetch_fsmap learned FSMap version 33 2023-07-18T03:19:00.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.619+0000 7feb2bd70b80 1 -- 172.21.15.101:0/2731699961 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55a9df246c60 con 0x55a9df4e8ad0 2023-07-18T03:19:00.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.619+0000 7feaf9ffb700 1 -- 172.21.15.101:0/2731699961 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb0c0105a0 con 0x55a9df4e8ad0 2023-07-18T03:19:00.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.619+0000 7feaf9ffb700 1 -- 172.21.15.101:0/2731699961 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7feb08005840 con 0x55a9df4e8ad0 2023-07-18T03:19:00.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.619+0000 7feb2bd70b80 10 client.27925 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:19:00.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.619+0000 7feb2bd70b80 10 client.27925 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:19:00.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.619+0000 7feb2bd70b80 20 client.27925 populate_metadata read hostname 'smithi101' 2023-07-18T03:19:00.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.619+0000 7feb2bd70b80 1 --2- 172.21.15.101:0/2731699961 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55a9df54cd10 0x55a9df54f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.619+0000 7feb2bd70b80 4 client.27925 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:19:00.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.619+0000 7feb2bd70b80 1 -- 172.21.15.101:0/2731699961 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a9df3eb050 con 0x55a9df54cd10 2023-07-18T03:19:00.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.620+0000 7feb1619b700 1 --2- 172.21.15.101:0/2731699961 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55a9df54cd10 0x55a9df54f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.620+0000 7feb1619b700 1 --2- 172.21.15.101:0/2731699961 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55a9df54cd10 0x55a9df54f0f0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a9d69c29e9f6643 server_cookie=aff6d734ad7cd45c in_seq=0 out_seq=0 2023-07-18T03:19:00.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.620+0000 7feaf9ffb700 10 client.27925 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:19:00.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.620+0000 7feaf9ffb700 1 -- 172.21.15.101:0/2731699961 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a9df3eb050 con 0x55a9df54cd10 2023-07-18T03:19:00.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.620+0000 7feaf9ffb700 10 client.27925 handle_command_reply: tid=0 2023-07-18T03:19:00.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.620+0000 7feb2bd70b80 1 client.27925 shutdown 2023-07-18T03:19:00.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.621+0000 7feb2bd70b80 1 -- 172.21.15.101:0/2731699961 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a9df4e8ad0 msgr2=0x55a9df54b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:00.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.621+0000 7feb2bd70b80 1 --2- 172.21.15.101:0/2731699961 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a9df4e8ad0 0x55a9df54b380 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7feb0800f450 tx=0x7feb08017720 comp rx=0 tx=0).stop 2023-07-18T03:19:00.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.621+0000 7feb2bd70b80 1 -- 172.21.15.101:0/2731699961 shutdown_connections 2023-07-18T03:19:00.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.621+0000 7feb2bd70b80 1 --2- 172.21.15.101:0/2731699961 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55a9df54cd10 0x55a9df54f0f0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.621+0000 7feb2bd70b80 1 --2- 172.21.15.101:0/2731699961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9df4dd2d0 0x55a9df5404f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.621+0000 7feaf9ffb700 0 client.27925 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:19:00.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.621+0000 7feb2bd70b80 1 --2- 172.21.15.101:0/2731699961 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a9df4e8ad0 0x55a9df54b380 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.621+0000 7feb2bd70b80 1 --2- 172.21.15.101:0/2731699961 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9df4ddbe0 0x55a9df540a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.621+0000 7feb2bd70b80 1 -- 172.21.15.101:0/2731699961 >> 172.21.15.101:0/2731699961 conn(0x55a9df4d3da0 msgr2=0x55a9df4e09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:00.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.621+0000 7feb2bd70b80 1 -- 172.21.15.101:0/2731699961 shutdown_connections 2023-07-18T03:19:00.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.621+0000 7feb2bd70b80 1 -- 172.21.15.101:0/2731699961 wait complete. 2023-07-18T03:19:00.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.621+0000 7feb2bd70b80 20 client.27925 trim_cache size 0 max 16384 2023-07-18T03:19:00.624 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:19:00.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.622+0000 7feb183ff700 1 -- 172.21.15.101:0/3255996854 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7feafc0a12c0 msgr2=0x7feafc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:00.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.622+0000 7feb183ff700 1 --2- 172.21.15.101:0/3255996854 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7feafc0a12c0 0x7feafc0a3770 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7feb10074920 tx=0x7feb0800b040 comp rx=0 tx=0).stop 2023-07-18T03:19:00.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.622+0000 7feb183ff700 1 -- 172.21.15.101:0/3255996854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb100ffad0 msgr2=0x7feb10072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:00.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.622+0000 7feb183ff700 1 --2- 172.21.15.101:0/3255996854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb100ffad0 0x7feb10072810 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7feb0c00e9f0 tx=0x7feb0c00ed00 comp rx=0 tx=0).stop 2023-07-18T03:19:00.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.622+0000 7feb183ff700 1 -- 172.21.15.101:0/3255996854 shutdown_connections 2023-07-18T03:19:00.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.622+0000 7feb183ff700 1 --2- 172.21.15.101:0/3255996854 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7feafc0a12c0 0x7feafc0a3770 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.622+0000 7feb183ff700 1 --2- 172.21.15.101:0/3255996854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb10105ae0 0x7feb101a3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.622+0000 7feb183ff700 1 --2- 172.21.15.101:0/3255996854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb100ffad0 0x7feb10072810 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.622+0000 7feb183ff700 1 --2- 172.21.15.101:0/3255996854 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7feb10100480 0x7feb10072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.622+0000 7feb183ff700 1 -- 172.21.15.101:0/3255996854 >> 172.21.15.101:0/3255996854 conn(0x7feb100fb5f0 msgr2=0x7feb101035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:00.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.622+0000 7feb183ff700 1 -- 172.21.15.101:0/3255996854 shutdown_connections 2023-07-18T03:19:00.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.622+0000 7feb183ff700 1 -- 172.21.15.101:0/3255996854 wait complete. 2023-07-18T03:19:00.627 INFO:tasks.workunit.client.0.smithi101.stdout: 1 696254 5.53 MB/sec execute 505 sec latency 23.076 ms 2023-07-18T03:19:00.634 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-18T03:19:00.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.854+0000 7f37af3ff700 1 -- 172.21.15.101:0/2493679392 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37a8072850 msgr2=0x7f37a8072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:00.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.854+0000 7f37af3ff700 1 --2- 172.21.15.101:0/2493679392 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37a8072850 0x7f37a8072c20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f37a4009990 tx=0x7f37a4009ca0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.854+0000 7f37af3ff700 1 -- 172.21.15.101:0/2493679392 shutdown_connections 2023-07-18T03:19:00.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.854+0000 7f37af3ff700 1 --2- 172.21.15.101:0/2493679392 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f37a810c3a0 0x7f37a810e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.854+0000 7f37af3ff700 1 --2- 172.21.15.101:0/2493679392 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37a8072850 0x7f37a8072c20 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.850 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.854+0000 7f37af3ff700 1 --2- 172.21.15.101:0/2493679392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37a8073160 0x7f37a810be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.850 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.854+0000 7f37af3ff700 1 -- 172.21.15.101:0/2493679392 >> 172.21.15.101:0/2493679392 conn(0x7f37a80fb600 msgr2=0x7f37a80fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:00.850 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.854+0000 7f37af3ff700 1 -- 172.21.15.101:0/2493679392 shutdown_connections 2023-07-18T03:19:00.850 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.854+0000 7f37af3ff700 1 -- 172.21.15.101:0/2493679392 wait complete. 2023-07-18T03:19:00.851 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.856+0000 7f37af3ff700 1 Processor -- start 2023-07-18T03:19:00.851 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.856+0000 7f37af3ff700 1 -- start start 2023-07-18T03:19:00.851 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.856+0000 7f37af3ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f37a8072850 0x7f37a8197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.851 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.856+0000 7f37af3ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37a8073160 0x7f37a81983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.851 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.856+0000 7f37af3ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37a810c3a0 0x7f37a819c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.851 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.856+0000 7f37af3ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a819ce80 con 0x7f37a8072850 2023-07-18T03:19:00.852 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.856+0000 7f37af3ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a819cff0 con 0x7f37a810c3a0 2023-07-18T03:19:00.852 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.856+0000 7f37af3ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a819d1b0 con 0x7f37a8073160 2023-07-18T03:19:00.853 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.857+0000 7f37ad19b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f37a8072850 0x7f37a8197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.853 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.857+0000 7f37ad99c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37a810c3a0 0x7f37a819c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.853 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.857+0000 7f37ad99c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37a810c3a0 0x7f37a819c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:45014/0 (socket says 172.21.15.101:45014) 2023-07-18T03:19:00.853 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.857+0000 7f37ad99c700 1 -- 172.21.15.101:0/3003109835 learned_addr learned my addr 172.21.15.101:0/3003109835 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:00.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.857+0000 7f37ac99a700 1 --2- 172.21.15.101:0/3003109835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37a8073160 0x7f37a81983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.857+0000 7f37ad99c700 1 -- 172.21.15.101:0/3003109835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37a8073160 msgr2=0x7f37a81983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:00.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.857+0000 7f37ad99c700 1 --2- 172.21.15.101:0/3003109835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37a8073160 0x7f37a81983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.857+0000 7f37ad99c700 1 -- 172.21.15.101:0/3003109835 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f37a8072850 msgr2=0x7f37a8197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:00.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.857+0000 7f37ad99c700 1 --2- 172.21.15.101:0/3003109835 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f37a8072850 0x7f37a8197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.857+0000 7f37ad99c700 1 -- 172.21.15.101:0/3003109835 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a40095f0 con 0x7f37a810c3a0 2023-07-18T03:19:00.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.857+0000 7f37ac99a700 1 --2- 172.21.15.101:0/3003109835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37a8073160 0x7f37a81983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:19:00.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.858+0000 7f37ad99c700 1 --2- 172.21.15.101:0/3003109835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37a810c3a0 0x7f37a819c840 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f37a0007ee0 tx=0x7f37a000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:00.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.858+0000 7f379e7fc700 1 -- 172.21.15.101:0/3003109835 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a0017070 con 0x7f37a810c3a0 2023-07-18T03:19:00.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.858+0000 7f379e7fc700 1 -- 172.21.15.101:0/3003109835 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37a000f040 con 0x7f37a810c3a0 2023-07-18T03:19:00.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.858+0000 7f37af3ff700 1 -- 172.21.15.101:0/3003109835 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37a819d490 con 0x7f37a810c3a0 2023-07-18T03:19:00.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.858+0000 7f379e7fc700 1 -- 172.21.15.101:0/3003109835 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a00043a0 con 0x7f37a810c3a0 2023-07-18T03:19:00.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.858+0000 7f37af3ff700 1 -- 172.21.15.101:0/3003109835 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37a81a4cd0 con 0x7f37a810c3a0 2023-07-18T03:19:00.856 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.862+0000 7f379e7fc700 1 -- 172.21.15.101:0/3003109835 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f37a0003a00 con 0x7f37a810c3a0 2023-07-18T03:19:00.858 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.864+0000 7f37c2c73b80 1 Processor -- start 2023-07-18T03:19:00.858 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.864+0000 7f37c2c73b80 1 -- start start 2023-07-18T03:19:00.859 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.864+0000 7f37c2c73b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563482bac2d0 0x563482bac6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.859 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.864+0000 7f379e7fc700 1 --2- 172.21.15.101:0/3003109835 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f37940a12c0 0x7f37940a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.859 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.864+0000 7f37c2c73b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482bacbe0 0x563482bb7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.859 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.864+0000 7f37c2c73b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563482bb7ad0 0x563482bb9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.860 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.864+0000 7f37c2c73b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563482ab2cb0 con 0x563482bac2d0 2023-07-18T03:19:00.860 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.864+0000 7f37c2c73b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563482b05a40 con 0x563482bacbe0 2023-07-18T03:19:00.860 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.864+0000 7f37c2c73b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563482afeed0 con 0x563482bb7ad0 2023-07-18T03:19:00.860 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.865+0000 7f379e7fc700 1 -- 172.21.15.101:0/3003109835 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f37a00cc770 con 0x7f37a810c3a0 2023-07-18T03:19:00.861 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.865+0000 7f37ac99a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563482bac2d0 0x563482bac6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.861 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.865+0000 7f37ac99a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563482bac2d0 0x563482bac6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:59828/0 (socket says 172.21.15.101:59828) 2023-07-18T03:19:00.861 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.865+0000 7f37ac99a700 1 -- 172.21.15.101:0/2635171742 learned_addr learned my addr 172.21.15.101:0/2635171742 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:00.861 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.865+0000 7f37ac99a700 1 --2- 172.21.15.101:0/2635171742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482bacbe0 0x563482bb7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.865+0000 7f37ad99c700 1 --2- 172.21.15.101:0/2635171742 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563482bb7ad0 0x563482bb9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.865+0000 7f37ad19b700 1 --2- 172.21.15.101:0/3003109835 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f37940a12c0 0x7f37940a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.866+0000 7f37ac99a700 1 -- 172.21.15.101:0/2635171742 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563482bb7ad0 msgr2=0x563482bb9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:00.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.866+0000 7f37ac99a700 1 --2- 172.21.15.101:0/2635171742 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563482bb7ad0 0x563482bb9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.866+0000 7f37ac99a700 1 -- 172.21.15.101:0/2635171742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482bacbe0 msgr2=0x563482bb7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:00.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.866+0000 7f37ac99a700 1 --2- 172.21.15.101:0/2635171742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482bacbe0 0x563482bb7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.866+0000 7f37ac99a700 1 -- 172.21.15.101:0/2635171742 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56348292cde0 con 0x563482bac2d0 2023-07-18T03:19:00.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.866+0000 7f37ac99a700 1 --2- 172.21.15.101:0/2635171742 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563482bac2d0 0x563482bac6a0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f3798012c30 tx=0x7f379800bba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8306e79fe887e48c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:00.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.866+0000 7f37937fe700 1 -- 172.21.15.101:0/2635171742 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37980120d0 con 0x563482bac2d0 2023-07-18T03:19:00.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.866+0000 7f37ad19b700 1 --2- 172.21.15.101:0/3003109835 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f37940a12c0 0x7f37940a3770 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f37a400b5c0 tx=0x7f37a4005fb0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:00.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.866+0000 7f37937fe700 1 -- 172.21.15.101:0/2635171742 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3798012230 con 0x563482bac2d0 2023-07-18T03:19:00.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.866+0000 7f37937fe700 1 -- 172.21.15.101:0/2635171742 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3798005020 con 0x563482bac2d0 2023-07-18T03:19:00.864 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.867+0000 7f37c2c73b80 1 -- 172.21.15.101:0/2635171742 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563482bac2d0 msgr2=0x563482bac6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:00.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.867+0000 7f37c2c73b80 1 --2- 172.21.15.101:0/2635171742 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563482bac2d0 0x563482bac6a0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f3798012c30 tx=0x7f379800bba0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.868+0000 7f37c2c73b80 1 -- 172.21.15.101:0/2635171742 shutdown_connections 2023-07-18T03:19:00.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.868+0000 7f37c2c73b80 1 --2- 172.21.15.101:0/2635171742 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563482bb7ad0 0x563482bb9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.868+0000 7f37c2c73b80 1 --2- 172.21.15.101:0/2635171742 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563482bac2d0 0x563482bac6a0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.868+0000 7f37c2c73b80 1 --2- 172.21.15.101:0/2635171742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482bacbe0 0x563482bb7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.868+0000 7f37c2c73b80 1 -- 172.21.15.101:0/2635171742 >> 172.21.15.101:0/2635171742 conn(0x563482ba2da0 msgr2=0x563482ba86a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:00.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.868+0000 7f37c2c73b80 1 -- 172.21.15.101:0/2635171742 shutdown_connections 2023-07-18T03:19:00.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.868+0000 7f37c2c73b80 1 -- 172.21.15.101:0/2635171742 wait complete. 2023-07-18T03:19:00.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.868+0000 7f37c2c73b80 1 -- start start 2023-07-18T03:19:00.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.869+0000 7f37c2c73b80 1 Processor -- start 2023-07-18T03:19:00.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.870+0000 7f37c2c73b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482bac2d0 0x563482bb7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.870+0000 7f37c2c73b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563482bacbe0 0x563482bb47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.870+0000 7f37c2c73b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563482bb7ad0 0x563482bb4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.870+0000 7f37c2c73b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563482a9eac0 con 0x563482bacbe0 2023-07-18T03:19:00.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.870+0000 7f37c2c73b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563482a9e7c0 con 0x563482bac2d0 2023-07-18T03:19:00.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.870+0000 7f37c2c73b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563482982920 con 0x563482bb7ad0 2023-07-18T03:19:00.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.870+0000 7f37c2c73b80 10 client.0 _ll_register_callbacks cb 0x563482a9c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:19:00.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.870+0000 7f37ad19b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482bac2d0 0x563482bb7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.870+0000 7f37ad19b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482bac2d0 0x563482bb7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:45026/0 (socket says 172.21.15.101:45026) 2023-07-18T03:19:00.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.870+0000 7f37ad19b700 1 -- 172.21.15.101:0/3954587770 learned_addr learned my addr 172.21.15.101:0/3954587770 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:00.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.871+0000 7f37ac99a700 1 --2- 172.21.15.101:0/3954587770 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563482bacbe0 0x563482bb47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.871+0000 7f37ad19b700 1 -- 172.21.15.101:0/3954587770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563482bb7ad0 msgr2=0x563482bb4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:00.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.871+0000 7f37ad19b700 1 --2- 172.21.15.101:0/3954587770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563482bb7ad0 0x563482bb4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.871+0000 7f37ad19b700 1 -- 172.21.15.101:0/3954587770 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563482bacbe0 msgr2=0x563482bb47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:00.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.871+0000 7f37ad19b700 1 --2- 172.21.15.101:0/3954587770 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563482bacbe0 0x563482bb47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.871+0000 7f37ad19b700 1 -- 172.21.15.101:0/3954587770 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f379801a040 con 0x563482bac2d0 2023-07-18T03:19:00.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.871+0000 7f37ac99a700 1 --2- 172.21.15.101:0/3954587770 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563482bacbe0 0x563482bb47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:19:00.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.871+0000 7f37ad19b700 1 --2- 172.21.15.101:0/3954587770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482bac2d0 0x563482bb7560 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f37a4003730 tx=0x7f37a4009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4deb26c4dad15a78 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:00.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.871+0000 7f3790ff9700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:19:00.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.871+0000 7f3790ff9700 1 -- 172.21.15.101:0/3954587770 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a402a070 con 0x563482bac2d0 2023-07-18T03:19:00.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.871+0000 7f37c2c73b80 1 -- 172.21.15.101:0/3954587770 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563482a584f0 con 0x563482bac2d0 2023-07-18T03:19:00.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.871+0000 7f3790ff9700 1 -- 172.21.15.101:0/3954587770 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f37a4005610 con 0x563482bac2d0 2023-07-18T03:19:00.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.871+0000 7f3790ff9700 1 -- 172.21.15.101:0/3954587770 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37a4023460 con 0x563482bac2d0 2023-07-18T03:19:00.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.872+0000 7f37c2c73b80 10 client.27902 fetch_fsmap learned FSMap version 33 2023-07-18T03:19:00.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.872+0000 7f37c2c73b80 1 -- 172.21.15.101:0/3954587770 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x563482915c60 con 0x563482bac2d0 2023-07-18T03:19:00.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.872+0000 7f3790ff9700 1 -- 172.21.15.101:0/3954587770 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563482a584f0 con 0x563482bac2d0 2023-07-18T03:19:00.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.872+0000 7f3790ff9700 1 -- 172.21.15.101:0/3954587770 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f37a400f5a0 con 0x563482bac2d0 2023-07-18T03:19:00.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.872+0000 7f37c2c73b80 10 client.27902 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:19:00.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.872+0000 7f37c2c73b80 10 client.27902 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:19:00.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.872+0000 7f37c2c73b80 20 client.27902 populate_metadata read hostname 'smithi101' 2023-07-18T03:19:00.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.872+0000 7f37c2c73b80 1 --2- 172.21.15.101:0/3954587770 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563482bc7440 0x563482bc9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.872+0000 7f37c2c73b80 4 client.27902 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:19:00.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.872+0000 7f37c2c73b80 1 -- 172.21.15.101:0/3954587770 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563482af7eb0 con 0x563482bc7440 2023-07-18T03:19:00.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.873+0000 7f37ac99a700 1 --2- 172.21.15.101:0/3954587770 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563482bc7440 0x563482bc9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.874+0000 7f37ac99a700 1 --2- 172.21.15.101:0/3954587770 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563482bc7440 0x563482bc9850 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d020fb1d0a1f50b server_cookie=8d60d08de8b74f9c in_seq=0 out_seq=0 2023-07-18T03:19:00.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.874+0000 7f3790ff9700 10 client.27902 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:19:00.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.874+0000 7f3790ff9700 1 -- 172.21.15.101:0/3954587770 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563482af7eb0 con 0x563482bc7440 2023-07-18T03:19:00.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.874+0000 7f3790ff9700 10 client.27902 handle_command_reply: tid=0 2023-07-18T03:19:00.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.874+0000 7f37c2c73b80 1 client.27902 shutdown 2023-07-18T03:19:00.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.875+0000 7f37c2c73b80 1 -- 172.21.15.101:0/3954587770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482bac2d0 msgr2=0x563482bb7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:00.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.875+0000 7f37c2c73b80 1 --2- 172.21.15.101:0/3954587770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482bac2d0 0x563482bb7560 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f37a4003730 tx=0x7f37a4009310 comp rx=0 tx=0).stop 2023-07-18T03:19:00.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.875+0000 7f37c2c73b80 1 -- 172.21.15.101:0/3954587770 shutdown_connections 2023-07-18T03:19:00.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.875+0000 7f37c2c73b80 1 --2- 172.21.15.101:0/3954587770 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563482bc7440 0x563482bc9850 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.875+0000 7f37c2c73b80 1 --2- 172.21.15.101:0/3954587770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482bac2d0 0x563482bb7560 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.875+0000 7f37c2c73b80 1 --2- 172.21.15.101:0/3954587770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563482bb7ad0 0x563482bb4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.875+0000 7f3790ff9700 0 client.27902 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:19:00.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.875+0000 7f37c2c73b80 1 --2- 172.21.15.101:0/3954587770 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563482bacbe0 0x563482bb47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.877 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.875+0000 7f37c2c73b80 1 -- 172.21.15.101:0/3954587770 >> 172.21.15.101:0/3954587770 conn(0x563482ba2da0 msgr2=0x563482ba8290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:00.877 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.875+0000 7f37c2c73b80 1 -- 172.21.15.101:0/3954587770 shutdown_connections 2023-07-18T03:19:00.877 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.875+0000 7f37c2c73b80 1 -- 172.21.15.101:0/3954587770 wait complete. 2023-07-18T03:19:00.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.875+0000 7f37c2c73b80 20 client.27902 trim_cache size 0 max 16384 2023-07-18T03:19:00.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.893+0000 7f37c2c73b80 1 Processor -- start 2023-07-18T03:19:00.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.893+0000 7f37c2c73b80 1 -- start start 2023-07-18T03:19:00.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.893+0000 7f37c2c73b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482bac2d0 0x563482bb7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.893+0000 7f37c2c73b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563482bacbe0 0x563482bb3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.893+0000 7f37c2c73b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563482bb7ad0 0x563482bb4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.893+0000 7f37c2c73b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563482ba9d00 con 0x563482bacbe0 2023-07-18T03:19:00.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.893+0000 7f37c2c73b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563482bad540 con 0x563482bac2d0 2023-07-18T03:19:00.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.893+0000 7f37c2c73b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563482bb5dc0 con 0x563482bb7ad0 2023-07-18T03:19:00.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.893+0000 7f37ad99c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563482bb7ad0 0x563482bb4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.893+0000 7f37ad99c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563482bb7ad0 0x563482bb4500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41642/0 (socket says 172.21.15.101:41642) 2023-07-18T03:19:00.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.893+0000 7f37ad19b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482bac2d0 0x563482bb7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.893+0000 7f37ad99c700 1 -- 172.21.15.101:0/2264618960 learned_addr learned my addr 172.21.15.101:0/2264618960 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:00.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.894+0000 7f37ad99c700 1 -- 172.21.15.101:0/2264618960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482bac2d0 msgr2=0x563482bb7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:00.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.894+0000 7f37ad99c700 1 --2- 172.21.15.101:0/2264618960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482bac2d0 0x563482bb7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.894+0000 7f37ad99c700 1 -- 172.21.15.101:0/2264618960 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563482bacbe0 msgr2=0x563482bb3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:00.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.894+0000 7f37ad99c700 1 --2- 172.21.15.101:0/2264618960 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563482bacbe0 0x563482bb3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.894+0000 7f37ad99c700 1 -- 172.21.15.101:0/2264618960 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563482953250 con 0x563482bb7ad0 2023-07-18T03:19:00.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.894+0000 7f37ad19b700 1 --2- 172.21.15.101:0/2264618960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482bac2d0 0x563482bb7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:19:00.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.894+0000 7f37ad99c700 1 --2- 172.21.15.101:0/2264618960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563482bb7ad0 0x563482bb4500 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f37a00060b0 tx=0x7f37a0009410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a152e4c48d0960f8 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:00.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.894+0000 7f3793fff700 1 -- 172.21.15.101:0/2264618960 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a00ae020 con 0x563482bb7ad0 2023-07-18T03:19:00.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.895+0000 7f3793fff700 1 -- 172.21.15.101:0/2264618960 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37a0099430 con 0x563482bb7ad0 2023-07-18T03:19:00.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.895+0000 7f3793fff700 1 -- 172.21.15.101:0/2264618960 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a00cc2f0 con 0x563482bb7ad0 2023-07-18T03:19:00.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.895+0000 7f37c2c73b80 1 -- 172.21.15.101:0/2264618960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563482bb7ad0 msgr2=0x563482bb4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:00.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.895+0000 7f37c2c73b80 1 --2- 172.21.15.101:0/2264618960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563482bb7ad0 0x563482bb4500 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f37a00060b0 tx=0x7f37a0009410 comp rx=0 tx=0).stop 2023-07-18T03:19:00.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.895+0000 7f37c2c73b80 1 -- 172.21.15.101:0/2264618960 shutdown_connections 2023-07-18T03:19:00.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.895+0000 7f37c2c73b80 1 --2- 172.21.15.101:0/2264618960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482bac2d0 0x563482bb7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.895+0000 7f37c2c73b80 1 --2- 172.21.15.101:0/2264618960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563482bb7ad0 0x563482bb4500 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.895+0000 7f37c2c73b80 1 --2- 172.21.15.101:0/2264618960 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563482bacbe0 0x563482bb3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.896+0000 7f37c2c73b80 1 -- 172.21.15.101:0/2264618960 >> 172.21.15.101:0/2264618960 conn(0x563482ba2da0 msgr2=0x563482bc37a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:00.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.896+0000 7f37c2c73b80 1 -- 172.21.15.101:0/2264618960 shutdown_connections 2023-07-18T03:19:00.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.896+0000 7f37c2c73b80 1 -- 172.21.15.101:0/2264618960 wait complete. 2023-07-18T03:19:00.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.896+0000 7f37c2c73b80 1 -- start start 2023-07-18T03:19:00.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.897+0000 7f37c2c73b80 1 Processor -- start 2023-07-18T03:19:00.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.897+0000 7f37c2c73b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482bac2d0 0x563482bbe4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.897+0000 7f37c2c73b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563482bacbe0 0x563482bbc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.897+0000 7f37c2c73b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563482bb7ad0 0x563482bbce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.897+0000 7f37c2c73b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563482bbd4e0 con 0x563482bacbe0 2023-07-18T03:19:00.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.897+0000 7f37c2c73b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563482bbd650 con 0x563482bac2d0 2023-07-18T03:19:00.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.897+0000 7f37c2c73b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563482c1b2f0 con 0x563482bb7ad0 2023-07-18T03:19:00.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.897+0000 7f37c2c73b80 10 client.0 _ll_register_callbacks cb 0x7f37a400bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:19:00.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.897+0000 7f37ad99c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563482bb7ad0 0x563482bbce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.898+0000 7f37ad99c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563482bb7ad0 0x563482bbce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41646/0 (socket says 172.21.15.101:41646) 2023-07-18T03:19:00.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.898+0000 7f37ad99c700 1 -- 172.21.15.101:0/3501566175 learned_addr learned my addr 172.21.15.101:0/3501566175 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:00.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.898+0000 7f37ad19b700 1 --2- 172.21.15.101:0/3501566175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482bac2d0 0x563482bbe4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.898+0000 7f37ac99a700 1 --2- 172.21.15.101:0/3501566175 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563482bacbe0 0x563482bbc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.898+0000 7f37ad99c700 1 -- 172.21.15.101:0/3501566175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482bac2d0 msgr2=0x563482bbe4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:00.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.898+0000 7f37ad99c700 1 --2- 172.21.15.101:0/3501566175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482bac2d0 0x563482bbe4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.898+0000 7f37ad99c700 1 -- 172.21.15.101:0/3501566175 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563482bacbe0 msgr2=0x563482bbc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:00.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.898+0000 7f37ad99c700 1 --2- 172.21.15.101:0/3501566175 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563482bacbe0 0x563482bbc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.898+0000 7f37ad99c700 1 -- 172.21.15.101:0/3501566175 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563482915c60 con 0x563482bb7ad0 2023-07-18T03:19:00.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.898+0000 7f37ad19b700 1 --2- 172.21.15.101:0/3501566175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482bac2d0 0x563482bbe4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:19:00.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.898+0000 7f37ad99c700 1 --2- 172.21.15.101:0/3501566175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563482bb7ad0 0x563482bbce40 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f37a0095e50 tx=0x7f37a00957b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d9b3f4d60464b1b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:00.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.898+0000 7f3790ff9700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:19:00.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.898+0000 7f3790ff9700 1 -- 172.21.15.101:0/3501566175 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a00ae040 con 0x563482bb7ad0 2023-07-18T03:19:00.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.898+0000 7f3790ff9700 1 -- 172.21.15.101:0/3501566175 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f37a009d460 con 0x563482bb7ad0 2023-07-18T03:19:00.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.899+0000 7f37c2c73b80 1 -- 172.21.15.101:0/3501566175 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3798012230 con 0x563482bb7ad0 2023-07-18T03:19:00.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.899+0000 7f3790ff9700 1 -- 172.21.15.101:0/3501566175 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37a00a1880 con 0x563482bb7ad0 2023-07-18T03:19:00.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.899+0000 7f37c2c73b80 10 client.27940 fetch_fsmap learned FSMap version 33 2023-07-18T03:19:00.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.899+0000 7f37c2c73b80 1 -- 172.21.15.101:0/3501566175 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f37a81a4cd0 con 0x563482bb7ad0 2023-07-18T03:19:00.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.899+0000 7f3790ff9700 1 -- 172.21.15.101:0/3501566175 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3798012230 con 0x563482bb7ad0 2023-07-18T03:19:00.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.899+0000 7f3790ff9700 1 -- 172.21.15.101:0/3501566175 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f37a0098a20 con 0x563482bb7ad0 2023-07-18T03:19:00.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.899+0000 7f37c2c73b80 10 client.27940 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:19:00.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.899+0000 7f37c2c73b80 10 client.27940 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:19:00.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.899+0000 7f37c2c73b80 20 client.27940 populate_metadata read hostname 'smithi101' 2023-07-18T03:19:00.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.899+0000 7f37c2c73b80 1 --2- 172.21.15.101:0/3501566175 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563482c1c5b0 0x563482c1e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:00.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.899+0000 7f37c2c73b80 4 client.27940 mds_command: new command op to 24562 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:19:00.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.899+0000 7f37c2c73b80 1 -- 172.21.15.101:0/3501566175 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563482aba050 con 0x563482c1c5b0 2023-07-18T03:19:00.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.899+0000 7f37ad19b700 1 --2- 172.21.15.101:0/3501566175 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563482c1c5b0 0x563482c1e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:00.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.900+0000 7f3790ff9700 10 client.27940 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:19:00.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.900+0000 7f37ad19b700 1 --2- 172.21.15.101:0/3501566175 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563482c1c5b0 0x563482c1e990 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e7cb650c1599e58 server_cookie=36088915fc9e9d15 in_seq=0 out_seq=0 2023-07-18T03:19:00.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.900+0000 7f3790ff9700 1 -- 172.21.15.101:0/3501566175 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563482aba050 con 0x563482c1c5b0 2023-07-18T03:19:00.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.900+0000 7f3790ff9700 10 client.27940 handle_command_reply: tid=0 2023-07-18T03:19:00.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.900+0000 7f37c2c73b80 1 client.27940 shutdown 2023-07-18T03:19:00.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.901+0000 7f37c2c73b80 1 -- 172.21.15.101:0/3501566175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563482bb7ad0 msgr2=0x563482bbce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:00.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.901+0000 7f37c2c73b80 1 --2- 172.21.15.101:0/3501566175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563482bb7ad0 0x563482bbce40 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f37a0095e50 tx=0x7f37a00957b0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.901+0000 7f37c2c73b80 1 -- 172.21.15.101:0/3501566175 shutdown_connections 2023-07-18T03:19:00.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.901+0000 7f37c2c73b80 1 --2- 172.21.15.101:0/3501566175 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563482c1c5b0 0x563482c1e990 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.901+0000 7f37c2c73b80 1 --2- 172.21.15.101:0/3501566175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563482bac2d0 0x563482bbe4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.901+0000 7f37c2c73b80 1 --2- 172.21.15.101:0/3501566175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563482bb7ad0 0x563482bbce40 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.901+0000 7f3790ff9700 0 client.27940 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:19:00.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.901+0000 7f37c2c73b80 1 --2- 172.21.15.101:0/3501566175 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563482bacbe0 0x563482bbc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.901+0000 7f37c2c73b80 1 -- 172.21.15.101:0/3501566175 >> 172.21.15.101:0/3501566175 conn(0x563482ba2da0 msgr2=0x563482bae3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:00.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.901+0000 7f37c2c73b80 1 -- 172.21.15.101:0/3501566175 shutdown_connections 2023-07-18T03:19:00.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.901+0000 7f37c2c73b80 1 -- 172.21.15.101:0/3501566175 wait complete. 2023-07-18T03:19:00.911 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:19:00.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.901+0000 7f37c2c73b80 20 client.27940 trim_cache size 0 max 16384 2023-07-18T03:19:00.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.902+0000 7f37af3ff700 1 -- 172.21.15.101:0/3003109835 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f37940a12c0 msgr2=0x7f37940a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:00.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.902+0000 7f37af3ff700 1 --2- 172.21.15.101:0/3003109835 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f37940a12c0 0x7f37940a3770 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f37a400b5c0 tx=0x7f37a4005fb0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.902+0000 7f37af3ff700 1 -- 172.21.15.101:0/3003109835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37a810c3a0 msgr2=0x7f37a819c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:00.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.902+0000 7f37af3ff700 1 --2- 172.21.15.101:0/3003109835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37a810c3a0 0x7f37a819c840 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f37a0007ee0 tx=0x7f37a000d3b0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.902+0000 7f37af3ff700 1 -- 172.21.15.101:0/3003109835 shutdown_connections 2023-07-18T03:19:00.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.902+0000 7f37af3ff700 1 --2- 172.21.15.101:0/3003109835 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f37940a12c0 0x7f37940a3770 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.902+0000 7f37af3ff700 1 --2- 172.21.15.101:0/3003109835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37a810c3a0 0x7f37a819c840 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.902+0000 7f37af3ff700 1 --2- 172.21.15.101:0/3003109835 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f37a8072850 0x7f37a8197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.902+0000 7f37af3ff700 1 --2- 172.21.15.101:0/3003109835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37a8073160 0x7f37a81983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:00.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.902+0000 7f37af3ff700 1 -- 172.21.15.101:0/3003109835 >> 172.21.15.101:0/3003109835 conn(0x7f37a80fb600 msgr2=0x7f37a810e440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:00.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.902+0000 7f37af3ff700 1 -- 172.21.15.101:0/3003109835 shutdown_connections 2023-07-18T03:19:00.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:00.902+0000 7f37af3ff700 1 -- 172.21.15.101:0/3003109835 wait complete. 2023-07-18T03:19:01.239 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:01 smithi101 ceph-mon[126379]: pgmap v356: 129 pgs: 129 active+clean; 579 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 151 op/s 2023-07-18T03:19:01.239 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:01 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1729545640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:19:01.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:01 smithi155 ceph-mon[128885]: pgmap v356: 129 pgs: 129 active+clean; 579 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 151 op/s 2023-07-18T03:19:01.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:01 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1729545640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:19:01.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:01 smithi179 ceph-mon[129229]: pgmap v356: 129 pgs: 129 active+clean; 579 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 151 op/s 2023-07-18T03:19:01.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:01 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1729545640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:19:01.623 INFO:tasks.workunit.client.0.smithi101.stdout: 1 697048 5.53 MB/sec execute 506 sec latency 48.651 ms 2023-07-18T03:19:01.920 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-18T03:19:01.920 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-18T03:19:02.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.189+0000 7ffa5ffff700 1 -- 172.21.15.101:0/2807911792 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa58072870 msgr2=0x7ffa58072c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:02.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.189+0000 7ffa5ffff700 1 --2- 172.21.15.101:0/2807911792 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa58072870 0x7ffa58072c40 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ffa54009990 tx=0x7ffa54009ca0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.189+0000 7ffa5ffff700 1 -- 172.21.15.101:0/2807911792 shutdown_connections 2023-07-18T03:19:02.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.189+0000 7ffa5ffff700 1 --2- 172.21.15.101:0/2807911792 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffa5810c380 0x7ffa5810e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.189+0000 7ffa5ffff700 1 --2- 172.21.15.101:0/2807911792 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa58072870 0x7ffa58072c40 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.189+0000 7ffa5ffff700 1 --2- 172.21.15.101:0/2807911792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa58073180 0x7ffa5810be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.189+0000 7ffa5ffff700 1 -- 172.21.15.101:0/2807911792 >> 172.21.15.101:0/2807911792 conn(0x7ffa580fb600 msgr2=0x7ffa580fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:02.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.189+0000 7ffa5ffff700 1 -- 172.21.15.101:0/2807911792 shutdown_connections 2023-07-18T03:19:02.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.189+0000 7ffa5ffff700 1 -- 172.21.15.101:0/2807911792 wait complete. 2023-07-18T03:19:02.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.190+0000 7ffa5ffff700 1 Processor -- start 2023-07-18T03:19:02.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.190+0000 7ffa5ffff700 1 -- start start 2023-07-18T03:19:02.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.191+0000 7ffa5ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffa58072870 0x7ffa58197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:02.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.191+0000 7ffa5ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa58073180 0x7ffa581983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:02.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.191+0000 7ffa5ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa5810c380 0x7ffa5819c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:02.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.191+0000 7ffa5ffff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5819ce40 con 0x7ffa58072870 2023-07-18T03:19:02.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.191+0000 7ffa5ffff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5819cfb0 con 0x7ffa58073180 2023-07-18T03:19:02.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.191+0000 7ffa5ffff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5819d170 con 0x7ffa5810c380 2023-07-18T03:19:02.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.191+0000 7ffa5d59a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa58073180 0x7ffa581983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:02.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.191+0000 7ffa5d59a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa58073180 0x7ffa581983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:45068/0 (socket says 172.21.15.101:45068) 2023-07-18T03:19:02.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.191+0000 7ffa5d59a700 1 -- 172.21.15.101:0/1671134155 learned_addr learned my addr 172.21.15.101:0/1671134155 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:02.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.191+0000 7ffa5e59c700 1 --2- 172.21.15.101:0/1671134155 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa5810c380 0x7ffa5819c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:02.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.191+0000 7ffa5dd9b700 1 --2- 172.21.15.101:0/1671134155 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffa58072870 0x7ffa58197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:02.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.192+0000 7ffa5d59a700 1 -- 172.21.15.101:0/1671134155 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa5810c380 msgr2=0x7ffa5819c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:02.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.192+0000 7ffa5d59a700 1 --2- 172.21.15.101:0/1671134155 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa5810c380 0x7ffa5819c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.192+0000 7ffa5d59a700 1 -- 172.21.15.101:0/1671134155 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffa58072870 msgr2=0x7ffa58197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:02.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.192+0000 7ffa5d59a700 1 --2- 172.21.15.101:0/1671134155 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffa58072870 0x7ffa58197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.192+0000 7ffa5d59a700 1 -- 172.21.15.101:0/1671134155 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa540095f0 con 0x7ffa58073180 2023-07-18T03:19:02.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.192+0000 7ffa5d59a700 1 --2- 172.21.15.101:0/1671134155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa58073180 0x7ffa581983a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ffa4800e9a0 tx=0x7ffa4800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:02.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.192+0000 7ffa4effd700 1 -- 172.21.15.101:0/1671134155 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa48009790 con 0x7ffa58073180 2023-07-18T03:19:02.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.192+0000 7ffa4effd700 1 -- 172.21.15.101:0/1671134155 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa4800cbd0 con 0x7ffa58073180 2023-07-18T03:19:02.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.192+0000 7ffa5ffff700 1 -- 172.21.15.101:0/1671134155 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa5819d450 con 0x7ffa58073180 2023-07-18T03:19:02.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.192+0000 7ffa4effd700 1 -- 172.21.15.101:0/1671134155 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa48018680 con 0x7ffa58073180 2023-07-18T03:19:02.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.193+0000 7ffa5ffff700 1 -- 172.21.15.101:0/1671134155 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa581a4c90 con 0x7ffa58073180 2023-07-18T03:19:02.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.197+0000 7ffa4effd700 1 -- 172.21.15.101:0/1671134155 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7ffa4800cd40 con 0x7ffa58073180 2023-07-18T03:19:02.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.199+0000 7ffa738b6b80 1 Processor -- start 2023-07-18T03:19:02.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.199+0000 7ffa738b6b80 1 -- start start 2023-07-18T03:19:02.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.200+0000 7ffa738b6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c0908fb350 0x55c0908fb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:02.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.200+0000 7ffa738b6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0908fbc60 0x55c09090ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:02.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.200+0000 7ffa738b6b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c09090f380 0x55c090911760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:02.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.200+0000 7ffa738b6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c090801d30 con 0x55c0908fb350 2023-07-18T03:19:02.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.200+0000 7ffa738b6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c090854ac0 con 0x55c0908fbc60 2023-07-18T03:19:02.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.200+0000 7ffa738b6b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c09084df50 con 0x55c09090f380 2023-07-18T03:19:02.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.200+0000 7ffa4effd700 1 --2- 172.21.15.101:0/1671134155 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ffa440a1340 0x7ffa440a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:02.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.200+0000 7ffa4effd700 1 -- 172.21.15.101:0/1671134155 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7ffa48014070 con 0x7ffa58073180 2023-07-18T03:19:02.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.200+0000 7ffa5dd9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0908fbc60 0x55c09090ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:02.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.200+0000 7ffa5dd9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0908fbc60 0x55c09090ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:45072/0 (socket says 172.21.15.101:45072) 2023-07-18T03:19:02.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.200+0000 7ffa5dd9b700 1 -- 172.21.15.101:0/2831253654 learned_addr learned my addr 172.21.15.101:0/2831253654 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:02.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.200+0000 7ffa5d59a700 1 --2- 172.21.15.101:0/2831253654 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c09090f380 0x55c090911760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:02.198 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.200+0000 7ffa5e59c700 1 --2- 172.21.15.101:0/2831253654 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c0908fb350 0x55c0908fb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:02.198 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.200+0000 7ffa5dd9b700 1 -- 172.21.15.101:0/2831253654 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c09090f380 msgr2=0x55c090911760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:02.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.200+0000 7ffa5dd9b700 1 --2- 172.21.15.101:0/2831253654 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c09090f380 0x55c090911760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.200+0000 7ffa5dd9b700 1 -- 172.21.15.101:0/2831253654 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c0908fb350 msgr2=0x55c0908fb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:02.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.200+0000 7ffa5dd9b700 1 --2- 172.21.15.101:0/2831253654 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c0908fb350 0x55c0908fb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.200+0000 7ffa5dd9b700 1 -- 172.21.15.101:0/2831253654 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c09067be60 con 0x55c0908fbc60 2023-07-18T03:19:02.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.200+0000 7ffa5e59c700 1 --2- 172.21.15.101:0/1671134155 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ffa440a1340 0x7ffa440a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:02.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.201+0000 7ffa5dd9b700 1 --2- 172.21.15.101:0/2831253654 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0908fbc60 0x55c09090ee40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ffa54005ea0 tx=0x7ffa54005ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c22eeed425d5a23 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:02.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.201+0000 7ffa43fff700 1 -- 172.21.15.101:0/2831253654 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5401d070 con 0x55c0908fbc60 2023-07-18T03:19:02.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.201+0000 7ffa43fff700 1 -- 172.21.15.101:0/2831253654 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa5401e040 con 0x55c0908fbc60 2023-07-18T03:19:02.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.201+0000 7ffa43fff700 1 -- 172.21.15.101:0/2831253654 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa54004920 con 0x55c0908fbc60 2023-07-18T03:19:02.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.202+0000 7ffa738b6b80 1 -- 172.21.15.101:0/2831253654 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0908fbc60 msgr2=0x55c09090ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:02.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.202+0000 7ffa738b6b80 1 --2- 172.21.15.101:0/2831253654 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0908fbc60 0x55c09090ee40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ffa54005ea0 tx=0x7ffa54005ed0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.202+0000 7ffa5e59c700 1 --2- 172.21.15.101:0/1671134155 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ffa440a1340 0x7ffa440a37f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ffa50009fd0 tx=0x7ffa5000b040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:02.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.202+0000 7ffa738b6b80 1 -- 172.21.15.101:0/2831253654 shutdown_connections 2023-07-18T03:19:02.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.202+0000 7ffa738b6b80 1 --2- 172.21.15.101:0/2831253654 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c09090f380 0x55c090911760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.202+0000 7ffa738b6b80 1 --2- 172.21.15.101:0/2831253654 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c0908fb350 0x55c0908fb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.202+0000 7ffa738b6b80 1 --2- 172.21.15.101:0/2831253654 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0908fbc60 0x55c09090ee40 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.202+0000 7ffa738b6b80 1 -- 172.21.15.101:0/2831253654 >> 172.21.15.101:0/2831253654 conn(0x55c0908f1e20 msgr2=0x55c0908f7660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:02.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.202+0000 7ffa738b6b80 1 -- 172.21.15.101:0/2831253654 shutdown_connections 2023-07-18T03:19:02.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.202+0000 7ffa738b6b80 1 -- 172.21.15.101:0/2831253654 wait complete. 2023-07-18T03:19:02.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.202+0000 7ffa738b6b80 1 -- start start 2023-07-18T03:19:02.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.203+0000 7ffa738b6b80 1 Processor -- start 2023-07-18T03:19:02.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.203+0000 7ffa738b6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0908fb350 0x55c09090ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:02.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.203+0000 7ffa738b6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c0908fbc60 0x55c09090c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:02.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.203+0000 7ffa738b6b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c09090f380 0x55c09090c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:02.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.203+0000 7ffa738b6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0907edb40 con 0x55c0908fbc60 2023-07-18T03:19:02.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.203+0000 7ffa738b6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0907ed840 con 0x55c0908fb350 2023-07-18T03:19:02.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.203+0000 7ffa738b6b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0906d19a0 con 0x55c09090f380 2023-07-18T03:19:02.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.203+0000 7ffa738b6b80 10 client.0 _ll_register_callbacks cb 0x55c0907eb7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:19:02.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.204+0000 7ffa5dd9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0908fb350 0x55c09090ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:02.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.204+0000 7ffa5e59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c09090f380 0x55c09090c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:02.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.204+0000 7ffa5e59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c09090f380 0x55c09090c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41672/0 (socket says 172.21.15.101:41672) 2023-07-18T03:19:02.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.204+0000 7ffa5e59c700 1 -- 172.21.15.101:0/3010111839 learned_addr learned my addr 172.21.15.101:0/3010111839 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:02.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.204+0000 7ffa5e59c700 1 -- 172.21.15.101:0/3010111839 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0908fb350 msgr2=0x55c09090ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:02.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.204+0000 7ffa5e59c700 1 --2- 172.21.15.101:0/3010111839 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0908fb350 0x55c09090ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.204+0000 7ffa5e59c700 1 -- 172.21.15.101:0/3010111839 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c0908fbc60 msgr2=0x55c09090c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:02.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.204+0000 7ffa5e59c700 1 --2- 172.21.15.101:0/3010111839 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c0908fbc60 0x55c09090c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.204+0000 7ffa5e59c700 1 -- 172.21.15.101:0/3010111839 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa5810f3d0 con 0x55c09090f380 2023-07-18T03:19:02.206 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.204+0000 7ffa5e59c700 1 --2- 172.21.15.101:0/3010111839 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c09090f380 0x55c09090c640 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ffa50007bc0 tx=0x7ffa50004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd859493a3a77e60 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:02.206 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.204+0000 7ffa417fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:19:02.206 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.205+0000 7ffa417fa700 1 -- 172.21.15.101:0/3010111839 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5001f070 con 0x55c09090f380 2023-07-18T03:19:02.206 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.205+0000 7ffa738b6b80 1 -- 172.21.15.101:0/3010111839 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c0907a7570 con 0x55c09090f380 2023-07-18T03:19:02.206 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.205+0000 7ffa417fa700 1 -- 172.21.15.101:0/3010111839 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7ffa5001a8e0 con 0x55c09090f380 2023-07-18T03:19:02.207 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.205+0000 7ffa417fa700 1 -- 172.21.15.101:0/3010111839 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa5000e040 con 0x55c09090f380 2023-07-18T03:19:02.207 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.205+0000 7ffa738b6b80 10 client.27952 fetch_fsmap learned FSMap version 33 2023-07-18T03:19:02.207 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.205+0000 7ffa738b6b80 1 -- 172.21.15.101:0/3010111839 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55c090664ce0 con 0x55c09090f380 2023-07-18T03:19:02.207 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.205+0000 7ffa417fa700 1 -- 172.21.15.101:0/3010111839 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c0907a7570 con 0x55c09090f380 2023-07-18T03:19:02.207 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.205+0000 7ffa417fa700 1 -- 172.21.15.101:0/3010111839 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ffa50018780 con 0x55c09090f380 2023-07-18T03:19:02.207 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.206+0000 7ffa738b6b80 10 client.27952 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:19:02.207 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.206+0000 7ffa738b6b80 10 client.27952 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:19:02.208 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.206+0000 7ffa738b6b80 20 client.27952 populate_metadata read hostname 'smithi101' 2023-07-18T03:19:02.208 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.206+0000 7ffa738b6b80 1 --2- 172.21.15.101:0/3010111839 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c09091ecc0 0x55c0909210d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:02.208 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.206+0000 7ffa738b6b80 4 client.27952 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:19:02.208 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.206+0000 7ffa738b6b80 1 -- 172.21.15.101:0/3010111839 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c090846f30 con 0x55c09091ecc0 2023-07-18T03:19:02.208 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.206+0000 7ffa5dd9b700 1 --2- 172.21.15.101:0/3010111839 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c09091ecc0 0x55c0909210d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:02.208 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.207+0000 7ffa417fa700 10 client.27952 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:19:02.208 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.207+0000 7ffa5dd9b700 1 --2- 172.21.15.101:0/3010111839 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c09091ecc0 0x55c0909210d0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc12c47d37fc7702 server_cookie=919c9877394b584d in_seq=0 out_seq=0 2023-07-18T03:19:02.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.207+0000 7ffa417fa700 1 -- 172.21.15.101:0/3010111839 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c090846f30 con 0x55c09091ecc0 2023-07-18T03:19:02.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.207+0000 7ffa417fa700 10 client.27952 handle_command_reply: tid=0 2023-07-18T03:19:02.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.207+0000 7ffa738b6b80 1 client.27952 shutdown 2023-07-18T03:19:02.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.208+0000 7ffa738b6b80 1 -- 172.21.15.101:0/3010111839 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c09090f380 msgr2=0x55c09090c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:02.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.208+0000 7ffa738b6b80 1 --2- 172.21.15.101:0/3010111839 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c09090f380 0x55c09090c640 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ffa50007bc0 tx=0x7ffa50004ad0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.208+0000 7ffa738b6b80 1 -- 172.21.15.101:0/3010111839 shutdown_connections 2023-07-18T03:19:02.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.208+0000 7ffa738b6b80 1 --2- 172.21.15.101:0/3010111839 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c09091ecc0 0x55c0909210d0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.208+0000 7ffa738b6b80 1 --2- 172.21.15.101:0/3010111839 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0908fb350 0x55c09090ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.208+0000 7ffa738b6b80 1 --2- 172.21.15.101:0/3010111839 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c09090f380 0x55c09090c640 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.208+0000 7ffa738b6b80 1 --2- 172.21.15.101:0/3010111839 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c0908fbc60 0x55c09090c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.208+0000 7ffa738b6b80 1 -- 172.21.15.101:0/3010111839 >> 172.21.15.101:0/3010111839 conn(0x55c0908f1e20 msgr2=0x55c0908f71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:02.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.208+0000 7ffa417fa700 0 client.27952 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:19:02.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.208+0000 7ffa738b6b80 1 -- 172.21.15.101:0/3010111839 shutdown_connections 2023-07-18T03:19:02.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.208+0000 7ffa738b6b80 1 -- 172.21.15.101:0/3010111839 wait complete. 2023-07-18T03:19:02.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.209+0000 7ffa738b6b80 20 client.27952 trim_cache size 0 max 16384 2023-07-18T03:19:02.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.231+0000 7ffa738b6b80 1 Processor -- start 2023-07-18T03:19:02.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.231+0000 7ffa738b6b80 1 -- start start 2023-07-18T03:19:02.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.231+0000 7ffa738b6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0908f1e20 0x55c09090e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:02.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.231+0000 7ffa738b6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c0908fb350 0x55c0908f71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:02.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.232+0000 7ffa738b6b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0908fbc60 0x55c0908f76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:02.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.232+0000 7ffa738b6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0908fc030 con 0x55c0908fb350 2023-07-18T03:19:02.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.232+0000 7ffa738b6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c09090a570 con 0x55c0908f1e20 2023-07-18T03:19:02.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.232+0000 7ffa738b6b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c09090cc40 con 0x55c0908fbc60 2023-07-18T03:19:02.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.232+0000 7ffa5d59a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c0908fb350 0x55c0908f71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:02.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.232+0000 7ffa5d59a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c0908fb350 0x55c0908f71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:59896/0 (socket says 172.21.15.101:59896) 2023-07-18T03:19:02.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.232+0000 7ffa5d59a700 1 -- 172.21.15.101:0/3471620257 learned_addr learned my addr 172.21.15.101:0/3471620257 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:02.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.232+0000 7ffa5dd9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0908f1e20 0x55c09090e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:02.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.232+0000 7ffa5e59c700 1 --2- 172.21.15.101:0/3471620257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0908fbc60 0x55c0908f76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:02.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.232+0000 7ffa5d59a700 1 -- 172.21.15.101:0/3471620257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0908fbc60 msgr2=0x55c0908f76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:02.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.232+0000 7ffa5d59a700 1 --2- 172.21.15.101:0/3471620257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0908fbc60 0x55c0908f76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.232+0000 7ffa5d59a700 1 -- 172.21.15.101:0/3471620257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0908f1e20 msgr2=0x55c09090e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:02.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.232+0000 7ffa5d59a700 1 --2- 172.21.15.101:0/3471620257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0908f1e20 0x55c09090e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.232+0000 7ffa5d59a700 1 -- 172.21.15.101:0/3471620257 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0906a22d0 con 0x55c0908fb350 2023-07-18T03:19:02.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.233+0000 7ffa5d59a700 1 --2- 172.21.15.101:0/3471620257 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c0908fb350 0x55c0908f71a0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7ffa48096510 tx=0x7ffa48005b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1f8ac9b1238efbb server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:02.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.233+0000 7ffa4cff9700 1 -- 172.21.15.101:0/3471620257 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa480cd170 con 0x55c0908fb350 2023-07-18T03:19:02.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.233+0000 7ffa4cff9700 1 -- 172.21.15.101:0/3471620257 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa48099b50 con 0x55c0908fb350 2023-07-18T03:19:02.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.233+0000 7ffa4cff9700 1 -- 172.21.15.101:0/3471620257 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4809a4d0 con 0x55c0908fb350 2023-07-18T03:19:02.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.234+0000 7ffa738b6b80 1 -- 172.21.15.101:0/3471620257 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c0908fb350 msgr2=0x55c0908f71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:02.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.234+0000 7ffa738b6b80 1 --2- 172.21.15.101:0/3471620257 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c0908fb350 0x55c0908f71a0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7ffa48096510 tx=0x7ffa48005b60 comp rx=0 tx=0).stop 2023-07-18T03:19:02.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.234+0000 7ffa738b6b80 1 -- 172.21.15.101:0/3471620257 shutdown_connections 2023-07-18T03:19:02.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.234+0000 7ffa738b6b80 1 --2- 172.21.15.101:0/3471620257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0908f1e20 0x55c09090e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.234+0000 7ffa738b6b80 1 --2- 172.21.15.101:0/3471620257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0908fbc60 0x55c0908f76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.234+0000 7ffa738b6b80 1 --2- 172.21.15.101:0/3471620257 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c0908fb350 0x55c0908f71a0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.234+0000 7ffa738b6b80 1 -- 172.21.15.101:0/3471620257 >> 172.21.15.101:0/3471620257 conn(0x55c0908f61f0 msgr2=0x55c090907380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:02.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.234+0000 7ffa738b6b80 1 -- 172.21.15.101:0/3471620257 shutdown_connections 2023-07-18T03:19:02.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.234+0000 7ffa738b6b80 1 -- 172.21.15.101:0/3471620257 wait complete. 2023-07-18T03:19:02.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.235+0000 7ffa738b6b80 1 -- start start 2023-07-18T03:19:02.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.235+0000 7ffa738b6b80 1 Processor -- start 2023-07-18T03:19:02.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.235+0000 7ffa738b6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c0908f1e20 0x55c09096bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:02.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.236+0000 7ffa738b6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0908fb350 0x55c09096c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:02.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.236+0000 7ffa738b6b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0908fbc60 0x55c090914c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:02.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.236+0000 7ffa738b6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c090915320 con 0x55c0908f1e20 2023-07-18T03:19:02.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.236+0000 7ffa738b6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c090915490 con 0x55c0908fb350 2023-07-18T03:19:02.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.236+0000 7ffa738b6b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c090915650 con 0x55c0908fbc60 2023-07-18T03:19:02.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.236+0000 7ffa738b6b80 10 client.0 _ll_register_callbacks cb 0x7ffa50007590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:19:02.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.236+0000 7ffa5dd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c0908f1e20 0x55c09096bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:02.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.236+0000 7ffa5dd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c0908f1e20 0x55c09096bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:59908/0 (socket says 172.21.15.101:59908) 2023-07-18T03:19:02.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.236+0000 7ffa5dd9b700 1 -- 172.21.15.101:0/3735770937 learned_addr learned my addr 172.21.15.101:0/3735770937 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:02.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.236+0000 7ffa5d59a700 1 --2- 172.21.15.101:0/3735770937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0908fb350 0x55c09096c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:02.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.236+0000 7ffa5e59c700 1 --2- 172.21.15.101:0/3735770937 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0908fbc60 0x55c090914c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:02.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.236+0000 7ffa5d59a700 1 -- 172.21.15.101:0/3735770937 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0908fbc60 msgr2=0x55c090914c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:02.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.236+0000 7ffa5d59a700 1 --2- 172.21.15.101:0/3735770937 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0908fbc60 0x55c090914c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.236+0000 7ffa5d59a700 1 -- 172.21.15.101:0/3735770937 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c0908f1e20 msgr2=0x55c09096bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:02.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.236+0000 7ffa5d59a700 1 --2- 172.21.15.101:0/3735770937 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c0908f1e20 0x55c09096bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.236+0000 7ffa5d59a700 1 -- 172.21.15.101:0/3735770937 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa581a4c90 con 0x55c0908fb350 2023-07-18T03:19:02.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.236+0000 7ffa5dd9b700 1 --2- 172.21.15.101:0/3735770937 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c0908f1e20 0x55c09096bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:19:02.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.237+0000 7ffa5d59a700 1 --2- 172.21.15.101:0/3735770937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0908fb350 0x55c09096c4a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ffa48096510 tx=0x7ffa480cd170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8850a052a9924ad server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:02.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.237+0000 7ffa417fa700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:19:02.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.237+0000 7ffa417fa700 1 -- 172.21.15.101:0/3735770937 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa480a2460 con 0x55c0908fb350 2023-07-18T03:19:02.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.237+0000 7ffa417fa700 1 -- 172.21.15.101:0/3735770937 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7ffa4809a010 con 0x55c0908fb350 2023-07-18T03:19:02.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.237+0000 7ffa738b6b80 1 -- 172.21.15.101:0/3735770937 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffa5401e040 con 0x55c0908fb350 2023-07-18T03:19:02.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.237+0000 7ffa417fa700 1 -- 172.21.15.101:0/3735770937 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa4809a270 con 0x55c0908fb350 2023-07-18T03:19:02.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.237+0000 7ffa738b6b80 10 client.27926 fetch_fsmap learned FSMap version 33 2023-07-18T03:19:02.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.237+0000 7ffa738b6b80 1 -- 172.21.15.101:0/3735770937 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55c09067a330 con 0x55c0908fb350 2023-07-18T03:19:02.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.238+0000 7ffa417fa700 1 -- 172.21.15.101:0/3735770937 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa480b8030 con 0x55c0908fb350 2023-07-18T03:19:02.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.238+0000 7ffa417fa700 1 -- 172.21.15.101:0/3735770937 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ffa48098f70 con 0x55c0908fb350 2023-07-18T03:19:02.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.238+0000 7ffa738b6b80 10 client.27926 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:19:02.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.238+0000 7ffa738b6b80 10 client.27926 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:19:02.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.238+0000 7ffa738b6b80 20 client.27926 populate_metadata read hostname 'smithi101' 2023-07-18T03:19:02.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.238+0000 7ffa738b6b80 1 --2- 172.21.15.101:0/3735770937 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c09090f380 0x55c090976a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:02.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.238+0000 7ffa738b6b80 4 client.27926 mds_command: new command op to 24562 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-18T03:19:02.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.238+0000 7ffa738b6b80 1 -- 172.21.15.101:0/3735770937 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55c0908090d0 con 0x55c09090f380 2023-07-18T03:19:02.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.238+0000 7ffa5dd9b700 1 --2- 172.21.15.101:0/3735770937 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c09090f380 0x55c090976a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:02.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.239+0000 7ffa417fa700 10 client.27926 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:19:02.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.239+0000 7ffa5dd9b700 1 --2- 172.21.15.101:0/3735770937 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c09090f380 0x55c090976a20 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41ef3249f5e16ff9 server_cookie=1c772b6135a8e16 in_seq=0 out_seq=0 2023-07-18T03:19:02.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.240+0000 7ffa417fa700 1 -- 172.21.15.101:0/3735770937 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c0908090d0 con 0x55c09090f380 2023-07-18T03:19:02.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.240+0000 7ffa417fa700 10 client.27926 handle_command_reply: tid=0 2023-07-18T03:19:02.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.240+0000 7ffa738b6b80 1 client.27926 shutdown 2023-07-18T03:19:02.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.240+0000 7ffa738b6b80 1 -- 172.21.15.101:0/3735770937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0908fb350 msgr2=0x55c09096c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:02.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.240+0000 7ffa738b6b80 1 --2- 172.21.15.101:0/3735770937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0908fb350 0x55c09096c4a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ffa48096510 tx=0x7ffa480cd170 comp rx=0 tx=0).stop 2023-07-18T03:19:02.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.240+0000 7ffa738b6b80 1 -- 172.21.15.101:0/3735770937 shutdown_connections 2023-07-18T03:19:02.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.240+0000 7ffa738b6b80 1 --2- 172.21.15.101:0/3735770937 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c09090f380 0x55c090976a20 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.240+0000 7ffa738b6b80 1 --2- 172.21.15.101:0/3735770937 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0908fbc60 0x55c090914c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.240+0000 7ffa738b6b80 1 --2- 172.21.15.101:0/3735770937 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c0908f1e20 0x55c09096bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.240+0000 7ffa417fa700 0 client.27926 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:19:02.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.240+0000 7ffa738b6b80 1 --2- 172.21.15.101:0/3735770937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0908fb350 0x55c09096c4a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.240+0000 7ffa738b6b80 1 -- 172.21.15.101:0/3735770937 >> 172.21.15.101:0/3735770937 conn(0x55c0908f61f0 msgr2=0x55c0908f71a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:02.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.240+0000 7ffa738b6b80 1 -- 172.21.15.101:0/3735770937 shutdown_connections 2023-07-18T03:19:02.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.240+0000 7ffa738b6b80 1 -- 172.21.15.101:0/3735770937 wait complete. 2023-07-18T03:19:02.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.241+0000 7ffa738b6b80 20 client.27926 trim_cache size 0 max 16384 2023-07-18T03:19:02.242 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:19:02.242 INFO:teuthology.orchestra.run.smithi101.stdout: "return_code": 0, 2023-07-18T03:19:02.242 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_tag": "ad511d47-76e6-4e42-aed5-607f1ac08f5c", 2023-07-18T03:19:02.242 INFO:teuthology.orchestra.run.smithi101.stdout: "mode": "asynchronous" 2023-07-18T03:19:02.243 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:19:02.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.241+0000 7ffa5ffff700 1 -- 172.21.15.101:0/1671134155 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ffa440a1340 msgr2=0x7ffa440a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:02.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.241+0000 7ffa5ffff700 1 --2- 172.21.15.101:0/1671134155 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ffa440a1340 0x7ffa440a37f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ffa50009fd0 tx=0x7ffa5000b040 comp rx=0 tx=0).stop 2023-07-18T03:19:02.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.241+0000 7ffa5ffff700 1 -- 172.21.15.101:0/1671134155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa58073180 msgr2=0x7ffa581983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:02.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.241+0000 7ffa5ffff700 1 --2- 172.21.15.101:0/1671134155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa58073180 0x7ffa581983a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ffa4800e9a0 tx=0x7ffa4800ecb0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.242+0000 7ffa5ffff700 1 -- 172.21.15.101:0/1671134155 shutdown_connections 2023-07-18T03:19:02.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.242+0000 7ffa5ffff700 1 --2- 172.21.15.101:0/1671134155 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa5810c380 0x7ffa5819c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.242+0000 7ffa5ffff700 1 --2- 172.21.15.101:0/1671134155 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffa58072870 0x7ffa58197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.242+0000 7ffa5ffff700 1 --2- 172.21.15.101:0/1671134155 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ffa440a1340 0x7ffa440a37f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.242+0000 7ffa5ffff700 1 --2- 172.21.15.101:0/1671134155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa58073180 0x7ffa581983a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.242+0000 7ffa5ffff700 1 -- 172.21.15.101:0/1671134155 >> 172.21.15.101:0/1671134155 conn(0x7ffa580fb600 msgr2=0x7ffa5810e460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:02.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.242+0000 7ffa5ffff700 1 -- 172.21.15.101:0/1671134155 shutdown_connections 2023-07-18T03:19:02.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.242+0000 7ffa5ffff700 1 -- 172.21.15.101:0/1671134155 wait complete. 2023-07-18T03:19:02.259 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:19:02.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.500+0000 7f0d181ff700 1 -- 172.21.15.101:0/3216090460 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0d10073160 msgr2=0x7f0d1010be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:02.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.500+0000 7f0d181ff700 1 --2- 172.21.15.101:0/3216090460 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0d10073160 0x7f0d1010be40 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f0d00009990 tx=0x7f0d00009ca0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.500+0000 7f0d181ff700 1 -- 172.21.15.101:0/3216090460 shutdown_connections 2023-07-18T03:19:02.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.500+0000 7f0d181ff700 1 --2- 172.21.15.101:0/3216090460 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d1010c380 0x7f0d1010e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.500+0000 7f0d181ff700 1 --2- 172.21.15.101:0/3216090460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0d10072850 0x7f0d10072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.500+0000 7f0d181ff700 1 --2- 172.21.15.101:0/3216090460 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0d10073160 0x7f0d1010be40 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.500+0000 7f0d181ff700 1 -- 172.21.15.101:0/3216090460 >> 172.21.15.101:0/3216090460 conn(0x7f0d100fb5e0 msgr2=0x7f0d100fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:02.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.500+0000 7f0d181ff700 1 -- 172.21.15.101:0/3216090460 shutdown_connections 2023-07-18T03:19:02.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.500+0000 7f0d181ff700 1 -- 172.21.15.101:0/3216090460 wait complete. 2023-07-18T03:19:02.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.501+0000 7f0d181ff700 1 Processor -- start 2023-07-18T03:19:02.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.501+0000 7f0d181ff700 1 -- start start 2023-07-18T03:19:02.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.501+0000 7f0d181ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0d10072850 0x7f0d10197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:02.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.501+0000 7f0d181ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d10073160 0x7f0d10198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:02.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.502+0000 7f0d181ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0d1010c380 0x7f0d1019c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:02.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.502+0000 7f0d181ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1019cec0 con 0x7f0d1010c380 2023-07-18T03:19:02.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.502+0000 7f0d181ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1019d030 con 0x7f0d10073160 2023-07-18T03:19:02.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.502+0000 7f0d181ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1019d1f0 con 0x7f0d10072850 2023-07-18T03:19:02.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.502+0000 7f0d15f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0d10072850 0x7f0d10197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:02.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.502+0000 7f0d15f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0d10072850 0x7f0d10197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41708/0 (socket says 172.21.15.101:41708) 2023-07-18T03:19:02.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.502+0000 7f0d15f9b700 1 -- 172.21.15.101:0/4230234964 learned_addr learned my addr 172.21.15.101:0/4230234964 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:02.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.502+0000 7f0d1679c700 1 --2- 172.21.15.101:0/4230234964 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0d1010c380 0x7f0d1019c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:02.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.502+0000 7f0d1579a700 1 --2- 172.21.15.101:0/4230234964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d10073160 0x7f0d10198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:02.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.502+0000 7f0d15f9b700 1 -- 172.21.15.101:0/4230234964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d10073160 msgr2=0x7f0d10198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:02.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.502+0000 7f0d15f9b700 1 --2- 172.21.15.101:0/4230234964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d10073160 0x7f0d10198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.502+0000 7f0d15f9b700 1 -- 172.21.15.101:0/4230234964 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0d1010c380 msgr2=0x7f0d1019c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:02.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.502+0000 7f0d15f9b700 1 --2- 172.21.15.101:0/4230234964 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0d1010c380 0x7f0d1019c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.502+0000 7f0d15f9b700 1 -- 172.21.15.101:0/4230234964 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d000095f0 con 0x7f0d10072850 2023-07-18T03:19:02.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.502+0000 7f0d15f9b700 1 --2- 172.21.15.101:0/4230234964 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0d10072850 0x7f0d10197ee0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c00e9a0 tx=0x7f0d0c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:02.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.503+0000 7f0d06ffd700 1 -- 172.21.15.101:0/4230234964 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c009790 con 0x7f0d10072850 2023-07-18T03:19:02.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.503+0000 7f0d181ff700 1 -- 172.21.15.101:0/4230234964 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d1019d4d0 con 0x7f0d10072850 2023-07-18T03:19:02.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.503+0000 7f0d06ffd700 1 -- 172.21.15.101:0/4230234964 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d0c00cbd0 con 0x7f0d10072850 2023-07-18T03:19:02.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.503+0000 7f0d06ffd700 1 -- 172.21.15.101:0/4230234964 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c018680 con 0x7f0d10072850 2023-07-18T03:19:02.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.503+0000 7f0d181ff700 1 -- 172.21.15.101:0/4230234964 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d101a4d10 con 0x7f0d10072850 2023-07-18T03:19:02.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.507+0000 7f0d06ffd700 1 -- 172.21.15.101:0/4230234964 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f0d0c00cd40 con 0x7f0d10072850 2023-07-18T03:19:02.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.509+0000 7f0d2ba31b80 1 Processor -- start 2023-07-18T03:19:02.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.509+0000 7f0d2ba31b80 1 -- start start 2023-07-18T03:19:02.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.509+0000 7f0d06ffd700 1 --2- 172.21.15.101:0/4230234964 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0cfc0a1340 0x7f0cfc0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:02.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.509+0000 7f0d2ba31b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d906c602d0 0x55d906c606a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:02.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.509+0000 7f0d2ba31b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d906c60be0 0x55d906c6b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:02.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.509+0000 7f0d2ba31b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d906c6bad0 0x55d906c6deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:02.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.509+0000 7f0d2ba31b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d906b66cb0 con 0x55d906c602d0 2023-07-18T03:19:02.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.509+0000 7f0d2ba31b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d906bb9a40 con 0x55d906c6bad0 2023-07-18T03:19:02.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.509+0000 7f0d2ba31b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d906bb2ed0 con 0x55d906c60be0 2023-07-18T03:19:02.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.509+0000 7f0d06ffd700 1 -- 172.21.15.101:0/4230234964 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f0d0c014070 con 0x7f0d10072850 2023-07-18T03:19:02.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.510+0000 7f0d15f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d906c602d0 0x55d906c606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:02.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.510+0000 7f0d1579a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d906c60be0 0x55d906c6b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:02.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.510+0000 7f0d1679c700 1 --2- 172.21.15.101:0/4230234964 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0cfc0a1340 0x7f0cfc0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:02.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.510+0000 7f0d15f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d906c602d0 0x55d906c606a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:59936/0 (socket says 172.21.15.101:59936) 2023-07-18T03:19:02.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.510+0000 7f0d1579a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d906c60be0 0x55d906c6b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41716/0 (socket says 172.21.15.101:41716) 2023-07-18T03:19:02.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.510+0000 7f0d15f9b700 1 -- 172.21.15.101:0/1775073317 learned_addr learned my addr 172.21.15.101:0/1775073317 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:02.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.510+0000 7f0d1679c700 1 --2- 172.21.15.101:0/1775073317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d906c6bad0 0x55d906c6deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:02.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.510+0000 7f0d15f9b700 1 -- 172.21.15.101:0/1775073317 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d906c60be0 msgr2=0x55d906c6b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:02.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.510+0000 7f0d15f9b700 1 --2- 172.21.15.101:0/1775073317 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d906c60be0 0x55d906c6b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.510+0000 7f0d15f9b700 1 -- 172.21.15.101:0/1775073317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d906c6bad0 msgr2=0x55d906c6deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:02.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.510+0000 7f0d15f9b700 1 --2- 172.21.15.101:0/1775073317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d906c6bad0 0x55d906c6deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.510+0000 7f0d15f9b700 1 -- 172.21.15.101:0/1775073317 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9069e0de0 con 0x55d906c602d0 2023-07-18T03:19:02.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.510+0000 7f0d15f9b700 1 --2- 172.21.15.101:0/1775073317 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d906c602d0 0x55d906c606a0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c0a1f30 tx=0x7f0d0c099e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f70ab9bfe643a0f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:02.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.510+0000 7f0cfbfff700 1 -- 172.21.15.101:0/1775073317 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c0cd160 con 0x55d906c602d0 2023-07-18T03:19:02.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.511+0000 7f0cfbfff700 1 -- 172.21.15.101:0/1775073317 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d0c049420 con 0x55d906c602d0 2023-07-18T03:19:02.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.511+0000 7f0cfbfff700 1 -- 172.21.15.101:0/1775073317 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c04aa10 con 0x55d906c602d0 2023-07-18T03:19:02.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.511+0000 7f0d1679c700 1 --2- 172.21.15.101:0/4230234964 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0cfc0a1340 0x7f0cfc0a37f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0d08009ec0 tx=0x7f0d08009920 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:02.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.511+0000 7f0d2ba31b80 1 -- 172.21.15.101:0/1775073317 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d906c602d0 msgr2=0x55d906c606a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:02.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.511+0000 7f0d2ba31b80 1 --2- 172.21.15.101:0/1775073317 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d906c602d0 0x55d906c606a0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c0a1f30 tx=0x7f0d0c099e60 comp rx=0 tx=0).stop 2023-07-18T03:19:02.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.511+0000 7f0d2ba31b80 1 -- 172.21.15.101:0/1775073317 shutdown_connections 2023-07-18T03:19:02.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.511+0000 7f0d2ba31b80 1 --2- 172.21.15.101:0/1775073317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d906c6bad0 0x55d906c6deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.511+0000 7f0d2ba31b80 1 --2- 172.21.15.101:0/1775073317 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d906c602d0 0x55d906c606a0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.511+0000 7f0d2ba31b80 1 --2- 172.21.15.101:0/1775073317 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d906c60be0 0x55d906c6b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.511+0000 7f0d2ba31b80 1 -- 172.21.15.101:0/1775073317 >> 172.21.15.101:0/1775073317 conn(0x55d906c56da0 msgr2=0x55d906c5c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:02.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.512+0000 7f0d2ba31b80 1 -- 172.21.15.101:0/1775073317 shutdown_connections 2023-07-18T03:19:02.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.512+0000 7f0d2ba31b80 1 -- 172.21.15.101:0/1775073317 wait complete. 2023-07-18T03:19:02.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.512+0000 7f0d2ba31b80 1 -- start start 2023-07-18T03:19:02.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.512+0000 7f0d2ba31b80 1 Processor -- start 2023-07-18T03:19:02.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.513+0000 7f0d2ba31b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d906c602d0 0x55d906c687e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:02.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.513+0000 7f0d2ba31b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d906c60be0 0x55d906c68d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:02.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.513+0000 7f0d2ba31b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d906c6bad0 0x55d906c69260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:02.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.513+0000 7f0d2ba31b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d906b52ac0 con 0x55d906c6bad0 2023-07-18T03:19:02.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.513+0000 7f0d2ba31b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d906b527c0 con 0x55d906c602d0 2023-07-18T03:19:02.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.513+0000 7f0d2ba31b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d906a36920 con 0x55d906c60be0 2023-07-18T03:19:02.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.513+0000 7f0d2ba31b80 10 client.0 _ll_register_callbacks cb 0x55d906b50760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:19:02.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.513+0000 7f0d15f9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d906c602d0 0x55d906c687e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:02.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.513+0000 7f0d1679c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d906c6bad0 0x55d906c69260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:02.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.513+0000 7f0d1579a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d906c60be0 0x55d906c68d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:02.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.513+0000 7f0d15f9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d906c602d0 0x55d906c687e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:45154/0 (socket says 172.21.15.101:45154) 2023-07-18T03:19:02.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.513+0000 7f0d15f9b700 1 -- 172.21.15.101:0/3958680788 learned_addr learned my addr 172.21.15.101:0/3958680788 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:02.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.513+0000 7f0d1579a700 1 -- 172.21.15.101:0/3958680788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d906c602d0 msgr2=0x55d906c687e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:02.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.513+0000 7f0d1579a700 1 --2- 172.21.15.101:0/3958680788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d906c602d0 0x55d906c687e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.513+0000 7f0d1579a700 1 -- 172.21.15.101:0/3958680788 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d906c6bad0 msgr2=0x55d906c69260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:02.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.513+0000 7f0d1579a700 1 --2- 172.21.15.101:0/3958680788 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d906c6bad0 0x55d906c69260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.513+0000 7f0d1579a700 1 -- 172.21.15.101:0/3958680788 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d101a4d10 con 0x55d906c60be0 2023-07-18T03:19:02.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.513+0000 7f0d15f9b700 1 --2- 172.21.15.101:0/3958680788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d906c602d0 0x55d906c687e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:19:02.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.514+0000 7f0d1579a700 1 --2- 172.21.15.101:0/3958680788 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d906c60be0 0x55d906c68d20 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0d101994f0 tx=0x7f0d0000bf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c325c24fbb5c3224 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:02.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.514+0000 7f0cf97fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:19:02.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.514+0000 7f0cf97fa700 1 -- 172.21.15.101:0/3958680788 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d00004500 con 0x55d906c60be0 2023-07-18T03:19:02.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.514+0000 7f0cf97fa700 1 -- 172.21.15.101:0/3958680788 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f0d00004660 con 0x55d906c60be0 2023-07-18T03:19:02.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.514+0000 7f0d2ba31b80 1 -- 172.21.15.101:0/3958680788 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d906b0c4f0 con 0x55d906c60be0 2023-07-18T03:19:02.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.514+0000 7f0cf97fa700 1 -- 172.21.15.101:0/3958680788 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d00004820 con 0x55d906c60be0 2023-07-18T03:19:02.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.514+0000 7f0d2ba31b80 10 client.27970 fetch_fsmap learned FSMap version 33 2023-07-18T03:19:02.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.514+0000 7f0d2ba31b80 1 -- 172.21.15.101:0/3958680788 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55d9069c9c60 con 0x55d906c60be0 2023-07-18T03:19:02.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.515+0000 7f0cf97fa700 1 -- 172.21.15.101:0/3958680788 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d906b0c4f0 con 0x55d906c60be0 2023-07-18T03:19:02.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.515+0000 7f0cf97fa700 1 -- 172.21.15.101:0/3958680788 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0d0000fa00 con 0x55d906c60be0 2023-07-18T03:19:02.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.515+0000 7f0d2ba31b80 10 client.27970 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:19:02.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.515+0000 7f0d2ba31b80 10 client.27970 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:19:02.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.515+0000 7f0d2ba31b80 20 client.27970 populate_metadata read hostname 'smithi101' 2023-07-18T03:19:02.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.515+0000 7f0d2ba31b80 1 --2- 172.21.15.101:0/3958680788 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55d906c7b450 0x55d906c7d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:02.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.515+0000 7f0d2ba31b80 4 client.27970 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:19:02.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.515+0000 7f0d2ba31b80 1 -- 172.21.15.101:0/3958680788 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d906babeb0 con 0x55d906c7b450 2023-07-18T03:19:02.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.515+0000 7f0d15f9b700 1 --2- 172.21.15.101:0/3958680788 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55d906c7b450 0x55d906c7d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:02.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.516+0000 7f0cf97fa700 10 client.27970 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:19:02.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.516+0000 7f0d15f9b700 1 --2- 172.21.15.101:0/3958680788 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55d906c7b450 0x55d906c7d860 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=821f1693858c5227 server_cookie=f4f07e40de238f2c in_seq=0 out_seq=0 2023-07-18T03:19:02.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.516+0000 7f0cf97fa700 1 -- 172.21.15.101:0/3958680788 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d906babeb0 con 0x55d906c7b450 2023-07-18T03:19:02.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.516+0000 7f0cf97fa700 10 client.27970 handle_command_reply: tid=0 2023-07-18T03:19:02.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.516+0000 7f0d2ba31b80 1 client.27970 shutdown 2023-07-18T03:19:02.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.516+0000 7f0d2ba31b80 1 -- 172.21.15.101:0/3958680788 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d906c60be0 msgr2=0x55d906c68d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:02.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.516+0000 7f0d2ba31b80 1 --2- 172.21.15.101:0/3958680788 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d906c60be0 0x55d906c68d20 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0d101994f0 tx=0x7f0d0000bf80 comp rx=0 tx=0).stop 2023-07-18T03:19:02.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.516+0000 7f0d2ba31b80 1 -- 172.21.15.101:0/3958680788 shutdown_connections 2023-07-18T03:19:02.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.516+0000 7f0d2ba31b80 1 --2- 172.21.15.101:0/3958680788 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55d906c7b450 0x55d906c7d860 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.516+0000 7f0d2ba31b80 1 --2- 172.21.15.101:0/3958680788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d906c602d0 0x55d906c687e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.516+0000 7f0d2ba31b80 1 --2- 172.21.15.101:0/3958680788 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d906c6bad0 0x55d906c69260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.516+0000 7f0d2ba31b80 1 --2- 172.21.15.101:0/3958680788 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d906c60be0 0x55d906c68d20 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.516+0000 7f0cf97fa700 0 client.27970 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:19:02.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.517+0000 7f0d2ba31b80 1 -- 172.21.15.101:0/3958680788 >> 172.21.15.101:0/3958680788 conn(0x55d906c56da0 msgr2=0x55d906c5c230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:02.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.517+0000 7f0d2ba31b80 1 -- 172.21.15.101:0/3958680788 shutdown_connections 2023-07-18T03:19:02.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.517+0000 7f0d2ba31b80 1 -- 172.21.15.101:0/3958680788 wait complete. 2023-07-18T03:19:02.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.517+0000 7f0d2ba31b80 20 client.27970 trim_cache size 0 max 16384 2023-07-18T03:19:02.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.536+0000 7f0d2ba31b80 1 Processor -- start 2023-07-18T03:19:02.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.536+0000 7f0d2ba31b80 1 -- start start 2023-07-18T03:19:02.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.536+0000 7f0d2ba31b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d906c602d0 0x55d906c63b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:02.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.536+0000 7f0d2ba31b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d906c60be0 0x55d906c64070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:02.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.536+0000 7f0d2ba31b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d906c6bad0 0x55d906c5c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:02.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.536+0000 7f0d2ba31b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d906c58ec0 con 0x55d906c6bad0 2023-07-18T03:19:02.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.536+0000 7f0d2ba31b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d906c61540 con 0x55d906c60be0 2023-07-18T03:19:02.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.536+0000 7f0d2ba31b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d906c5dd00 con 0x55d906c602d0 2023-07-18T03:19:02.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.536+0000 7f0d15f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d906c602d0 0x55d906c63b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:02.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.536+0000 7f0d1579a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d906c60be0 0x55d906c64070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:02.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.537+0000 7f0d15f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d906c602d0 0x55d906c63b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41724/0 (socket says 172.21.15.101:41724) 2023-07-18T03:19:02.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.537+0000 7f0d1579a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d906c60be0 0x55d906c64070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:45160/0 (socket says 172.21.15.101:45160) 2023-07-18T03:19:02.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.537+0000 7f0d15f9b700 1 -- 172.21.15.101:0/561796195 learned_addr learned my addr 172.21.15.101:0/561796195 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:02.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.537+0000 7f0d1679c700 1 --2- 172.21.15.101:0/561796195 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d906c6bad0 0x55d906c5c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:02.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.537+0000 7f0d15f9b700 1 -- 172.21.15.101:0/561796195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d906c60be0 msgr2=0x55d906c64070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:02.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.537+0000 7f0d15f9b700 1 --2- 172.21.15.101:0/561796195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d906c60be0 0x55d906c64070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.537+0000 7f0d15f9b700 1 -- 172.21.15.101:0/561796195 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d906c6bad0 msgr2=0x55d906c5c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:02.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.537+0000 7f0d15f9b700 1 --2- 172.21.15.101:0/561796195 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d906c6bad0 0x55d906c5c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.537+0000 7f0d15f9b700 1 -- 172.21.15.101:0/561796195 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d906c68350 con 0x55d906c602d0 2023-07-18T03:19:02.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.537+0000 7f0d1579a700 1 --2- 172.21.15.101:0/561796195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d906c60be0 0x55d906c64070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:19:02.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.537+0000 7f0d15f9b700 1 --2- 172.21.15.101:0/561796195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d906c602d0 0x55d906c63b30 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c099f50 tx=0x7f0d0c01f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98e00c5ed71d6b0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:02.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.537+0000 7f0d04ff9700 1 -- 172.21.15.101:0/561796195 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c0a2460 con 0x55d906c602d0 2023-07-18T03:19:02.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.537+0000 7f0d04ff9700 1 -- 172.21.15.101:0/561796195 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d0c0ca910 con 0x55d906c602d0 2023-07-18T03:19:02.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.537+0000 7f0d04ff9700 1 -- 172.21.15.101:0/561796195 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c099760 con 0x55d906c602d0 2023-07-18T03:19:02.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.538+0000 7f0d2ba31b80 1 -- 172.21.15.101:0/561796195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d906c602d0 msgr2=0x55d906c63b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:02.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.538+0000 7f0d2ba31b80 1 --2- 172.21.15.101:0/561796195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d906c602d0 0x55d906c63b30 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c099f50 tx=0x7f0d0c01f000 comp rx=0 tx=0).stop 2023-07-18T03:19:02.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.538+0000 7f0d2ba31b80 1 -- 172.21.15.101:0/561796195 shutdown_connections 2023-07-18T03:19:02.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.538+0000 7f0d2ba31b80 1 --2- 172.21.15.101:0/561796195 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d906c6bad0 0x55d906c5c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.538+0000 7f0d2ba31b80 1 --2- 172.21.15.101:0/561796195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d906c602d0 0x55d906c63b30 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.538+0000 7f0d2ba31b80 1 --2- 172.21.15.101:0/561796195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d906c60be0 0x55d906c64070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.538+0000 7f0d2ba31b80 1 -- 172.21.15.101:0/561796195 >> 172.21.15.101:0/561796195 conn(0x55d906c56da0 msgr2=0x55d906c76c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:02.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.538+0000 7f0d2ba31b80 1 -- 172.21.15.101:0/561796195 shutdown_connections 2023-07-18T03:19:02.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.538+0000 7f0d2ba31b80 1 -- 172.21.15.101:0/561796195 wait complete. 2023-07-18T03:19:02.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.539+0000 7f0d2ba31b80 1 -- start start 2023-07-18T03:19:02.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.539+0000 7f0d2ba31b80 1 Processor -- start 2023-07-18T03:19:02.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.540+0000 7f0d2ba31b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d906c602d0 0x55d906c71ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:02.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.540+0000 7f0d2ba31b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d906c60be0 0x55d906c72420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:02.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.540+0000 7f0d2ba31b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d906c6bad0 0x55d906c704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:02.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.540+0000 7f0d2ba31b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d906c70aa0 con 0x55d906c602d0 2023-07-18T03:19:02.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.540+0000 7f0d2ba31b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d906c70c10 con 0x55d906c60be0 2023-07-18T03:19:02.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.540+0000 7f0d2ba31b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d906c70dd0 con 0x55d906c6bad0 2023-07-18T03:19:02.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.540+0000 7f0d2ba31b80 10 client.0 _ll_register_callbacks cb 0x7f0d00009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:19:02.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.540+0000 7f0d15f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d906c602d0 0x55d906c71ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:02.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.540+0000 7f0d15f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d906c602d0 0x55d906c71ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:59964/0 (socket says 172.21.15.101:59964) 2023-07-18T03:19:02.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.540+0000 7f0d15f9b700 1 -- 172.21.15.101:0/3000650517 learned_addr learned my addr 172.21.15.101:0/3000650517 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:02.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.540+0000 7f0d1579a700 1 --2- 172.21.15.101:0/3000650517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d906c60be0 0x55d906c72420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:02.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.540+0000 7f0d1679c700 1 --2- 172.21.15.101:0/3000650517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d906c6bad0 0x55d906c704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:02.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.541+0000 7f0d15f9b700 1 -- 172.21.15.101:0/3000650517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d906c6bad0 msgr2=0x55d906c704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:02.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.541+0000 7f0d15f9b700 1 --2- 172.21.15.101:0/3000650517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d906c6bad0 0x55d906c704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.541+0000 7f0d15f9b700 1 -- 172.21.15.101:0/3000650517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d906c60be0 msgr2=0x55d906c72420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:02.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.541+0000 7f0d15f9b700 1 --2- 172.21.15.101:0/3000650517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d906c60be0 0x55d906c72420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.541+0000 7f0d15f9b700 1 -- 172.21.15.101:0/3000650517 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d9069c9c60 con 0x55d906c602d0 2023-07-18T03:19:02.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.541+0000 7f0d1579a700 1 --2- 172.21.15.101:0/3000650517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d906c60be0 0x55d906c72420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:19:02.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.541+0000 7f0d15f9b700 1 --2- 172.21.15.101:0/3000650517 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d906c602d0 0x55d906c71ee0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c09a1d0 tx=0x7f0d0c099b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=255198af0ab8592a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:02.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.541+0000 7f0cf97fa700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:19:02.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.541+0000 7f0cf97fa700 1 -- 172.21.15.101:0/3000650517 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c0b8280 con 0x55d906c602d0 2023-07-18T03:19:02.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.541+0000 7f0cf97fa700 1 -- 172.21.15.101:0/3000650517 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f0d0c0b83e0 con 0x55d906c602d0 2023-07-18T03:19:02.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.541+0000 7f0d2ba31b80 1 -- 172.21.15.101:0/3000650517 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0d0c049420 con 0x55d906c602d0 2023-07-18T03:19:02.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.541+0000 7f0cf97fa700 1 -- 172.21.15.101:0/3000650517 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d0c099500 con 0x55d906c602d0 2023-07-18T03:19:02.544 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.541+0000 7f0d2ba31b80 10 client.17898 fetch_fsmap learned FSMap version 33 2023-07-18T03:19:02.544 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.541+0000 7f0d2ba31b80 1 -- 172.21.15.101:0/3000650517 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55d9069e0de0 con 0x55d906c602d0 2023-07-18T03:19:02.544 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.542+0000 7f0cf97fa700 1 -- 172.21.15.101:0/3000650517 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d0c049420 con 0x55d906c602d0 2023-07-18T03:19:02.544 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.542+0000 7f0cf97fa700 1 -- 172.21.15.101:0/3000650517 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0d0c0a2060 con 0x55d906c602d0 2023-07-18T03:19:02.544 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.542+0000 7f0d2ba31b80 10 client.17898 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:19:02.544 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.542+0000 7f0d2ba31b80 10 client.17898 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:19:02.544 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.542+0000 7f0d2ba31b80 20 client.17898 populate_metadata read hostname 'smithi101' 2023-07-18T03:19:02.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.542+0000 7f0d2ba31b80 1 --2- 172.21.15.101:0/3000650517 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55d906ccffa0 0x55d906cd2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:02.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.542+0000 7f0d2ba31b80 4 client.17898 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:19:02.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.542+0000 7f0d2ba31b80 1 -- 172.21.15.101:0/3000650517 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d906b6e050 con 0x55d906ccffa0 2023-07-18T03:19:02.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.542+0000 7f0d1579a700 1 --2- 172.21.15.101:0/3000650517 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55d906ccffa0 0x55d906cd2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:02.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.543+0000 7f0d1579a700 1 --2- 172.21.15.101:0/3000650517 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55d906ccffa0 0x55d906cd2380 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93b17cd4843135f7 server_cookie=e22bc108c470e016 in_seq=0 out_seq=0 2023-07-18T03:19:02.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.543+0000 7f0cf97fa700 10 client.17898 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:19:02.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.543+0000 7f0cf97fa700 1 -- 172.21.15.101:0/3000650517 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55d906b6e050 con 0x55d906ccffa0 2023-07-18T03:19:02.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.543+0000 7f0cf97fa700 10 client.17898 handle_command_reply: tid=0 2023-07-18T03:19:02.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.543+0000 7f0d2ba31b80 1 client.17898 shutdown 2023-07-18T03:19:02.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.543+0000 7f0d2ba31b80 1 -- 172.21.15.101:0/3000650517 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d906c602d0 msgr2=0x55d906c71ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:02.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.543+0000 7f0d2ba31b80 1 --2- 172.21.15.101:0/3000650517 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d906c602d0 0x55d906c71ee0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c09a1d0 tx=0x7f0d0c099b50 comp rx=0 tx=0).stop 2023-07-18T03:19:02.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.543+0000 7f0d2ba31b80 1 -- 172.21.15.101:0/3000650517 shutdown_connections 2023-07-18T03:19:02.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.543+0000 7f0d2ba31b80 1 --2- 172.21.15.101:0/3000650517 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55d906ccffa0 0x55d906cd2380 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.543+0000 7f0d2ba31b80 1 --2- 172.21.15.101:0/3000650517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d906c6bad0 0x55d906c704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.547 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.543+0000 7f0d2ba31b80 1 --2- 172.21.15.101:0/3000650517 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d906c602d0 0x55d906c71ee0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.547 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.543+0000 7f0d2ba31b80 1 --2- 172.21.15.101:0/3000650517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d906c60be0 0x55d906c72420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.547 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.543+0000 7f0cf97fa700 0 client.17898 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:19:02.547 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.543+0000 7f0d2ba31b80 1 -- 172.21.15.101:0/3000650517 >> 172.21.15.101:0/3000650517 conn(0x55d906c56da0 msgr2=0x55d906c63b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:02.547 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.543+0000 7f0d2ba31b80 1 -- 172.21.15.101:0/3000650517 shutdown_connections 2023-07-18T03:19:02.547 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.543+0000 7f0d2ba31b80 1 -- 172.21.15.101:0/3000650517 wait complete. 2023-07-18T03:19:02.547 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.544+0000 7f0d2ba31b80 20 client.17898 trim_cache size 0 max 16384 2023-07-18T03:19:02.548 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:19:02.548 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:19:02.548 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:19:02.549 INFO:teuthology.orchestra.run.smithi101.stdout: "ad511d47-76e6-4e42-aed5-607f1ac08f5c": { 2023-07-18T03:19:02.549 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:19:02.549 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "ad511d47-76e6-4e42-aed5-607f1ac08f5c", 2023-07-18T03:19:02.549 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:19:02.549 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:19:02.549 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:19:02.549 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:19:02.550 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.544+0000 7f0d181ff700 1 -- 172.21.15.101:0/4230234964 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0cfc0a1340 msgr2=0x7f0cfc0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:02.550 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.544+0000 7f0d181ff700 1 --2- 172.21.15.101:0/4230234964 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0cfc0a1340 0x7f0cfc0a37f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0d08009ec0 tx=0x7f0d08009920 comp rx=0 tx=0).stop 2023-07-18T03:19:02.550 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.544+0000 7f0d181ff700 1 -- 172.21.15.101:0/4230234964 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0d10072850 msgr2=0x7f0d10197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:02.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.544+0000 7f0d181ff700 1 --2- 172.21.15.101:0/4230234964 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0d10072850 0x7f0d10197ee0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c00e9a0 tx=0x7f0d0c00ecb0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.544+0000 7f0d181ff700 1 -- 172.21.15.101:0/4230234964 shutdown_connections 2023-07-18T03:19:02.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.544+0000 7f0d181ff700 1 --2- 172.21.15.101:0/4230234964 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0d1010c380 0x7f0d1019c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.544+0000 7f0d181ff700 1 --2- 172.21.15.101:0/4230234964 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0d10072850 0x7f0d10197ee0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.544+0000 7f0d181ff700 1 --2- 172.21.15.101:0/4230234964 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0cfc0a1340 0x7f0cfc0a37f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.544+0000 7f0d181ff700 1 --2- 172.21.15.101:0/4230234964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d10073160 0x7f0d10198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:02.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.544+0000 7f0d181ff700 1 -- 172.21.15.101:0/4230234964 >> 172.21.15.101:0/4230234964 conn(0x7f0d100fb5e0 msgr2=0x7f0d10103de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:02.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.545+0000 7f0d181ff700 1 -- 172.21.15.101:0/4230234964 shutdown_connections 2023-07-18T03:19:02.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:02.545+0000 7f0d181ff700 1 -- 172.21.15.101:0/4230234964 wait complete. 2023-07-18T03:19:02.559 INFO:tasks.cephfs.filesystem:scrub status for tag:ad511d47-76e6-4e42-aed5-607f1ac08f5c - {'path': '/', 'tag': 'ad511d47-76e6-4e42-aed5-607f1ac08f5c', 'options': 'recursive,force'} 2023-07-18T03:19:02.623 INFO:tasks.workunit.client.0.smithi101.stdout: 1 697442 5.52 MB/sec execute 507 sec latency 82.820 ms 2023-07-18T03:19:03.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:03 smithi155 ceph-mon[128885]: pgmap v357: 129 pgs: 129 active+clean; 579 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 1.5 MiB/s wr, 100 op/s 2023-07-18T03:19:03.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:03 smithi155 ceph-mon[128885]: scrub queued for path: / 2023-07-18T03:19:03.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:03 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:19:03.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:03 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:19:03.298 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:03 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:19:03.298 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:03 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:19:03.298 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:03 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:19:03.298 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:03 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:19:03.298 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:03 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:19:03.298 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:03 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:19:03.298 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:03 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:19:03.299 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:03 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:19:03.299 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:03 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:19:03.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:03 smithi179 ceph-mon[129229]: pgmap v357: 129 pgs: 129 active+clean; 579 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 1.5 MiB/s wr, 100 op/s 2023-07-18T03:19:03.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:03 smithi179 ceph-mon[129229]: scrub queued for path: / 2023-07-18T03:19:03.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:03 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:19:03.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:03 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:19:03.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:03 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:19:03.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:03 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:19:03.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:03 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:19:03.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:03 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:19:03.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:03 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:19:03.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:03 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:19:03.454 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:03 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:19:03.454 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:03 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:19:03.454 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:03 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:19:03.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:03 smithi101 ceph-mon[126379]: pgmap v357: 129 pgs: 129 active+clean; 579 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 1.5 MiB/s wr, 100 op/s 2023-07-18T03:19:03.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:03 smithi101 ceph-mon[126379]: scrub queued for path: / 2023-07-18T03:19:03.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:03 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:19:03.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:03 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:19:03.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:03 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:19:03.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:03 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:19:03.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:03 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:19:03.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:03 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:19:03.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:03 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:19:03.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:03 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:19:03.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:03 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:19:03.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:03 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:19:03.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:03 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:19:03.623 INFO:tasks.workunit.client.0.smithi101.stdout: 1 697732 5.51 MB/sec execute 508 sec latency 546.215 ms 2023-07-18T03:19:04.624 INFO:tasks.workunit.client.0.smithi101.stdout: 1 698181 5.50 MB/sec execute 509 sec latency 1359.268 ms 2023-07-18T03:19:05.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:05 smithi155 ceph-mon[128885]: pgmap v358: 129 pgs: 129 active+clean; 579 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 1.5 MiB/s wr, 100 op/s 2023-07-18T03:19:05.451 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:05 smithi179 ceph-mon[129229]: pgmap v358: 129 pgs: 129 active+clean; 579 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 1.5 MiB/s wr, 100 op/s 2023-07-18T03:19:05.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:05 smithi101 ceph-mon[126379]: pgmap v358: 129 pgs: 129 active+clean; 579 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 1.5 MiB/s wr, 100 op/s 2023-07-18T03:19:05.623 INFO:tasks.workunit.client.0.smithi101.stdout: 1 699830 5.52 MB/sec execute 510 sec latency 16.135 ms 2023-07-18T03:19:06.623 INFO:tasks.workunit.client.0.smithi101.stdout: 1 700803 5.51 MB/sec execute 511 sec latency 33.429 ms 2023-07-18T03:19:07.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:07 smithi155 ceph-mon[128885]: pgmap v359: 129 pgs: 129 active+clean; 584 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 184 op/s 2023-07-18T03:19:07.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:07 smithi155 ceph-mon[128885]: osdmap e103: 12 total, 12 up, 12 in 2023-07-18T03:19:07.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:07 smithi179 ceph-mon[129229]: pgmap v359: 129 pgs: 129 active+clean; 584 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 184 op/s 2023-07-18T03:19:07.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:07 smithi179 ceph-mon[129229]: osdmap e103: 12 total, 12 up, 12 in 2023-07-18T03:19:07.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:07 smithi101 ceph-mon[126379]: pgmap v359: 129 pgs: 129 active+clean; 584 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 184 op/s 2023-07-18T03:19:07.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:07 smithi101 ceph-mon[126379]: osdmap e103: 12 total, 12 up, 12 in 2023-07-18T03:19:07.623 INFO:tasks.workunit.client.0.smithi101.stdout: 1 701275 5.51 MB/sec execute 512 sec latency 701.979 ms 2023-07-18T03:19:08.624 INFO:tasks.workunit.client.0.smithi101.stdout: 1 701275 5.50 MB/sec execute 513 sec latency 1702.111 ms 2023-07-18T03:19:09.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:09 smithi155 ceph-mon[128885]: pgmap v361: 129 pgs: 129 active+clean; 584 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 162 op/s 2023-07-18T03:19:09.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:09 smithi179 ceph-mon[129229]: pgmap v361: 129 pgs: 129 active+clean; 584 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 162 op/s 2023-07-18T03:19:09.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:09 smithi101 ceph-mon[126379]: pgmap v361: 129 pgs: 129 active+clean; 584 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 162 op/s 2023-07-18T03:19:09.624 INFO:tasks.workunit.client.0.smithi101.stdout: 1 701644 5.49 MB/sec execute 514 sec latency 2533.856 ms 2023-07-18T03:19:10.624 INFO:tasks.workunit.client.0.smithi101.stdout: 1 703363 5.50 MB/sec execute 515 sec latency 20.811 ms 2023-07-18T03:19:11.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:11 smithi155 ceph-mon[128885]: pgmap v362: 129 pgs: 129 active+clean; 566 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 162 op/s 2023-07-18T03:19:11.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:11 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:19:11.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:11 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:19:11.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:11 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:19:11.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:11 smithi179 ceph-mon[129229]: pgmap v362: 129 pgs: 129 active+clean; 566 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 162 op/s 2023-07-18T03:19:11.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:11 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:19:11.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:11 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:19:11.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:11 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:19:11.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:11 smithi101 ceph-mon[126379]: pgmap v362: 129 pgs: 129 active+clean; 566 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 162 op/s 2023-07-18T03:19:11.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:11 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:19:11.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:11 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:19:11.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:11 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:19:11.624 INFO:tasks.workunit.client.0.smithi101.stdout: 1 704339 5.50 MB/sec execute 516 sec latency 18.815 ms 2023-07-18T03:19:12.624 INFO:tasks.workunit.client.0.smithi101.stdout: 1 704822 5.49 MB/sec execute 517 sec latency 681.046 ms 2023-07-18T03:19:13.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:13 smithi155 ceph-mon[128885]: pgmap v363: 129 pgs: 129 active+clean; 566 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 162 op/s 2023-07-18T03:19:13.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:13 smithi179 ceph-mon[129229]: pgmap v363: 129 pgs: 129 active+clean; 566 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 162 op/s 2023-07-18T03:19:13.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:13 smithi101 ceph-mon[126379]: pgmap v363: 129 pgs: 129 active+clean; 566 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 162 op/s 2023-07-18T03:19:13.624 INFO:tasks.workunit.client.0.smithi101.stdout: 1 704822 5.48 MB/sec execute 518 sec latency 1681.233 ms 2023-07-18T03:19:14.624 INFO:tasks.workunit.client.0.smithi101.stdout: 1 705109 5.47 MB/sec execute 519 sec latency 2509.735 ms 2023-07-18T03:19:15.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:15 smithi155 ceph-mon[128885]: pgmap v364: 129 pgs: 129 active+clean; 566 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 162 op/s 2023-07-18T03:19:15.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:15 smithi155 ceph-mon[128885]: osdmap e104: 12 total, 12 up, 12 in 2023-07-18T03:19:15.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:15 smithi179 ceph-mon[129229]: pgmap v364: 129 pgs: 129 active+clean; 566 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 162 op/s 2023-07-18T03:19:15.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:15 smithi179 ceph-mon[129229]: osdmap e104: 12 total, 12 up, 12 in 2023-07-18T03:19:15.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:15 smithi101 ceph-mon[126379]: pgmap v364: 129 pgs: 129 active+clean; 566 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 162 op/s 2023-07-18T03:19:15.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:15 smithi101 ceph-mon[126379]: osdmap e104: 12 total, 12 up, 12 in 2023-07-18T03:19:15.624 INFO:tasks.workunit.client.0.smithi101.stdout: 1 706838 5.48 MB/sec execute 520 sec latency 24.074 ms 2023-07-18T03:19:16.624 INFO:tasks.workunit.client.0.smithi101.stdout: 1 707837 5.48 MB/sec execute 521 sec latency 13.989 ms 2023-07-18T03:19:17.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:17 smithi155 ceph-mon[128885]: pgmap v366: 129 pgs: 129 active+clean; 572 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 128 op/s 2023-07-18T03:19:17.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:17 smithi179 ceph-mon[129229]: pgmap v366: 129 pgs: 129 active+clean; 572 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 128 op/s 2023-07-18T03:19:17.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:17 smithi101 ceph-mon[126379]: pgmap v366: 129 pgs: 129 active+clean; 572 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 128 op/s 2023-07-18T03:19:17.624 INFO:tasks.workunit.client.0.smithi101.stdout: 1 708365 5.48 MB/sec execute 522 sec latency 405.305 ms 2023-07-18T03:19:18.624 INFO:tasks.workunit.client.0.smithi101.stdout: 1 708365 5.46 MB/sec execute 523 sec latency 1405.436 ms 2023-07-18T03:19:19.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:19 smithi155 ceph-mon[128885]: pgmap v367: 129 pgs: 129 active+clean; 572 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 123 op/s 2023-07-18T03:19:19.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:19 smithi179 ceph-mon[129229]: pgmap v367: 129 pgs: 129 active+clean; 572 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 123 op/s 2023-07-18T03:19:19.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:19 smithi101 ceph-mon[126379]: pgmap v367: 129 pgs: 129 active+clean; 572 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 123 op/s 2023-07-18T03:19:19.625 INFO:tasks.workunit.client.0.smithi101.stdout: 1 708814 5.46 MB/sec execute 524 sec latency 2212.776 ms 2023-07-18T03:19:20.625 INFO:tasks.workunit.client.0.smithi101.stdout: 1 710459 5.47 MB/sec execute 525 sec latency 20.609 ms 2023-07-18T03:19:21.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:21 smithi155 ceph-mon[128885]: pgmap v368: 129 pgs: 129 active+clean; 576 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.8 MiB/s wr, 138 op/s 2023-07-18T03:19:21.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:21 smithi179 ceph-mon[129229]: pgmap v368: 129 pgs: 129 active+clean; 576 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.8 MiB/s wr, 138 op/s 2023-07-18T03:19:21.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:21 smithi101 ceph-mon[126379]: pgmap v368: 129 pgs: 129 active+clean; 576 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.8 MiB/s wr, 138 op/s 2023-07-18T03:19:21.625 INFO:tasks.workunit.client.0.smithi101.stdout: 1 711400 5.46 MB/sec execute 526 sec latency 20.397 ms 2023-07-18T03:19:22.625 INFO:tasks.workunit.client.0.smithi101.stdout: 1 711916 5.46 MB/sec execute 527 sec latency 640.830 ms 2023-07-18T03:19:23.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:23 smithi155 ceph-mon[128885]: pgmap v369: 129 pgs: 129 active+clean; 576 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.8 MiB/s wr, 138 op/s 2023-07-18T03:19:23.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:23 smithi179 ceph-mon[129229]: pgmap v369: 129 pgs: 129 active+clean; 576 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.8 MiB/s wr, 138 op/s 2023-07-18T03:19:23.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:23 smithi101 ceph-mon[126379]: pgmap v369: 129 pgs: 129 active+clean; 576 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.8 MiB/s wr, 138 op/s 2023-07-18T03:19:23.625 INFO:tasks.workunit.client.0.smithi101.stdout: 1 711916 5.45 MB/sec execute 528 sec latency 1640.882 ms 2023-07-18T03:19:24.625 INFO:tasks.workunit.client.0.smithi101.stdout: 1 712365 5.44 MB/sec execute 529 sec latency 2460.641 ms 2023-07-18T03:19:25.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:25 smithi155 ceph-mon[128885]: pgmap v370: 129 pgs: 129 active+clean; 576 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.8 MiB/s wr, 138 op/s 2023-07-18T03:19:25.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:25 smithi179 ceph-mon[129229]: pgmap v370: 129 pgs: 129 active+clean; 576 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.8 MiB/s wr, 138 op/s 2023-07-18T03:19:25.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:25 smithi101 ceph-mon[126379]: pgmap v370: 129 pgs: 129 active+clean; 576 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.8 MiB/s wr, 138 op/s 2023-07-18T03:19:25.625 INFO:tasks.workunit.client.0.smithi101.stdout: 1 713911 5.45 MB/sec execute 530 sec latency 53.699 ms 2023-07-18T03:19:26.625 INFO:tasks.workunit.client.0.smithi101.stdout: 1 714744 5.45 MB/sec execute 531 sec latency 24.124 ms 2023-07-18T03:19:27.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:27 smithi155 ceph-mon[128885]: pgmap v371: 129 pgs: 129 active+clean; 581 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.3 MiB/s wr, 172 op/s 2023-07-18T03:19:27.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:27 smithi179 ceph-mon[129229]: pgmap v371: 129 pgs: 129 active+clean; 581 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.3 MiB/s wr, 172 op/s 2023-07-18T03:19:27.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:27 smithi101 ceph-mon[126379]: pgmap v371: 129 pgs: 129 active+clean; 581 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.3 MiB/s wr, 172 op/s 2023-07-18T03:19:27.625 INFO:tasks.workunit.client.0.smithi101.stdout: 1 715457 5.44 MB/sec execute 532 sec latency 525.225 ms 2023-07-18T03:19:28.625 INFO:tasks.workunit.client.0.smithi101.stdout: 1 715457 5.43 MB/sec execute 533 sec latency 1525.330 ms 2023-07-18T03:19:29.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:29 smithi155 ceph-mon[128885]: pgmap v372: 129 pgs: 129 active+clean; 581 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.4 MiB/s wr, 115 op/s 2023-07-18T03:19:29.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:29 smithi179 ceph-mon[129229]: pgmap v372: 129 pgs: 129 active+clean; 581 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.4 MiB/s wr, 115 op/s 2023-07-18T03:19:29.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:29 smithi101 ceph-mon[126379]: pgmap v372: 129 pgs: 129 active+clean; 581 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.4 MiB/s wr, 115 op/s 2023-07-18T03:19:29.625 INFO:tasks.workunit.client.0.smithi101.stdout: 1 715905 5.42 MB/sec execute 534 sec latency 2335.571 ms 2023-07-18T03:19:30.625 INFO:tasks.workunit.client.0.smithi101.stdout: 1 717522 5.44 MB/sec execute 535 sec latency 36.213 ms 2023-07-18T03:19:31.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:31 smithi155 ceph-mon[128885]: pgmap v373: 129 pgs: 129 active+clean; 586 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.1 MiB/s wr, 170 op/s 2023-07-18T03:19:31.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:31 smithi179 ceph-mon[129229]: pgmap v373: 129 pgs: 129 active+clean; 586 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.1 MiB/s wr, 170 op/s 2023-07-18T03:19:31.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:31 smithi101 ceph-mon[126379]: pgmap v373: 129 pgs: 129 active+clean; 586 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.1 MiB/s wr, 170 op/s 2023-07-18T03:19:31.625 INFO:tasks.workunit.client.0.smithi101.stdout: 1 718421 5.43 MB/sec execute 536 sec latency 27.419 ms 2023-07-18T03:19:32.560 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:19:32.625 INFO:tasks.workunit.client.0.smithi101.stdout: 1 719005 5.43 MB/sec execute 537 sec latency 572.636 ms 2023-07-18T03:19:32.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.767+0000 7f5da39ff700 1 -- 172.21.15.101:0/10541917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d9c0728b0 msgr2=0x7f5d9c072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:32.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.767+0000 7f5da39ff700 1 --2- 172.21.15.101:0/10541917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d9c0728b0 0x7f5d9c072c80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009990 tx=0x7f5d98009ca0 comp rx=0 tx=0).stop 2023-07-18T03:19:32.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.767+0000 7f5da39ff700 1 -- 172.21.15.101:0/10541917 shutdown_connections 2023-07-18T03:19:32.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.767+0000 7f5da39ff700 1 --2- 172.21.15.101:0/10541917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d9c0728b0 0x7f5d9c072c80 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:32.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.767+0000 7f5da39ff700 1 --2- 172.21.15.101:0/10541917 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5d9c110ae0 0x7f5d9c112ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:32.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.767+0000 7f5da39ff700 1 --2- 172.21.15.101:0/10541917 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d9c0731c0 0x7f5d9c1105a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:32.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.767+0000 7f5da39ff700 1 -- 172.21.15.101:0/10541917 >> 172.21.15.101:0/10541917 conn(0x7f5d9c0fb5f0 msgr2=0x7f5d9c0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:32.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.767+0000 7f5da39ff700 1 -- 172.21.15.101:0/10541917 shutdown_connections 2023-07-18T03:19:32.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.767+0000 7f5da39ff700 1 -- 172.21.15.101:0/10541917 wait complete. 2023-07-18T03:19:32.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.768+0000 7f5da39ff700 1 Processor -- start 2023-07-18T03:19:32.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.768+0000 7f5da39ff700 1 -- start start 2023-07-18T03:19:32.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.769+0000 7f5da39ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d9c0728b0 0x7f5d9c19c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:32.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.769+0000 7f5da39ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d9c0731c0 0x7f5d9c19c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:32.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.769+0000 7f5da39ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5d9c110ae0 0x7f5d9c1a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:32.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.769+0000 7f5da39ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c1a11a0 con 0x7f5d9c110ae0 2023-07-18T03:19:32.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.769+0000 7f5da39ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c1a1310 con 0x7f5d9c0731c0 2023-07-18T03:19:32.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.769+0000 7f5da39ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c1a14d0 con 0x7f5d9c0728b0 2023-07-18T03:19:32.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.769+0000 7f5da1f9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5d9c110ae0 0x7f5d9c1a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:32.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.769+0000 7f5da1f9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5d9c110ae0 0x7f5d9c1a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:37768/0 (socket says 172.21.15.101:37768) 2023-07-18T03:19:32.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.769+0000 7f5da1f9c700 1 -- 172.21.15.101:0/3144670141 learned_addr learned my addr 172.21.15.101:0/3144670141 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:32.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.769+0000 7f5da179b700 1 --2- 172.21.15.101:0/3144670141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d9c0728b0 0x7f5d9c19c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:32.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.769+0000 7f5da0f9a700 1 --2- 172.21.15.101:0/3144670141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d9c0731c0 0x7f5d9c19c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:32.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.769+0000 7f5da1f9c700 1 -- 172.21.15.101:0/3144670141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d9c0728b0 msgr2=0x7f5d9c19c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:32.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.769+0000 7f5da1f9c700 1 --2- 172.21.15.101:0/3144670141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d9c0728b0 0x7f5d9c19c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:32.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.769+0000 7f5da1f9c700 1 -- 172.21.15.101:0/3144670141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d9c0731c0 msgr2=0x7f5d9c19c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:32.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.769+0000 7f5da1f9c700 1 --2- 172.21.15.101:0/3144670141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d9c0731c0 0x7f5d9c19c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:32.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.770+0000 7f5da1f9c700 1 -- 172.21.15.101:0/3144670141 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d94009c40 con 0x7f5d9c110ae0 2023-07-18T03:19:32.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.770+0000 7f5da1f9c700 1 --2- 172.21.15.101:0/3144670141 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5d9c110ae0 0x7f5d9c1a0b60 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f5d9400c110 tx=0x7f5d9400c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:32.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.770+0000 7f5d927fc700 1 -- 172.21.15.101:0/3144670141 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d94017070 con 0x7f5d9c110ae0 2023-07-18T03:19:32.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.770+0000 7f5d927fc700 1 -- 172.21.15.101:0/3144670141 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d9400f040 con 0x7f5d9c110ae0 2023-07-18T03:19:32.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.770+0000 7f5da39ff700 1 -- 172.21.15.101:0/3144670141 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d980095f0 con 0x7f5d9c110ae0 2023-07-18T03:19:32.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.770+0000 7f5d927fc700 1 -- 172.21.15.101:0/3144670141 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9400e050 con 0x7f5d9c110ae0 2023-07-18T03:19:32.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.770+0000 7f5da39ff700 1 -- 172.21.15.101:0/3144670141 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d9c1a8e10 con 0x7f5d9c110ae0 2023-07-18T03:19:32.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.773+0000 7f5d927fc700 1 -- 172.21.15.101:0/3144670141 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f5d9400ca20 con 0x7f5d9c110ae0 2023-07-18T03:19:32.769 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.775+0000 7f5d927fc700 1 --2- 172.21.15.101:0/3144670141 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f5d880a12c0 0x7f5d880a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:32.769 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.775+0000 7f5db72d0b80 1 Processor -- start 2023-07-18T03:19:32.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.775+0000 7f5d927fc700 1 -- 172.21.15.101:0/3144670141 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f5d940cc940 con 0x7f5d9c110ae0 2023-07-18T03:19:32.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.775+0000 7f5da179b700 1 --2- 172.21.15.101:0/3144670141 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f5d880a12c0 0x7f5d880a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:32.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.775+0000 7f5db72d0b80 1 -- start start 2023-07-18T03:19:32.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.775+0000 7f5db72d0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5f711d2d0 0x55f5f711d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:32.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.775+0000 7f5db72d0b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5f711dbe0 0x55f5f7128590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:32.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.775+0000 7f5db72d0b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5f7128ad0 0x55f5f712aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:32.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.776+0000 7f5db72d0b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5f7023cb0 con 0x55f5f711dbe0 2023-07-18T03:19:32.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.776+0000 7f5db72d0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5f7076a40 con 0x55f5f711d2d0 2023-07-18T03:19:32.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.776+0000 7f5db72d0b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5f706fed0 con 0x55f5f7128ad0 2023-07-18T03:19:32.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.776+0000 7f5da179b700 1 --2- 172.21.15.101:0/3144670141 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f5d880a12c0 0x7f5d880a3770 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5d9800b2e0 tx=0x7f5d98005960 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:32.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.776+0000 7f5da0f9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5f711dbe0 0x55f5f7128590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:32.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.776+0000 7f5da179b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5f711d2d0 0x55f5f711d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:32.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.776+0000 7f5da0f9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5f711dbe0 0x55f5f7128590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:37782/0 (socket says 172.21.15.101:37782) 2023-07-18T03:19:32.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.776+0000 7f5da0f9a700 1 -- 172.21.15.101:0/1696295242 learned_addr learned my addr 172.21.15.101:0/1696295242 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:32.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.776+0000 7f5da1f9c700 1 --2- 172.21.15.101:0/1696295242 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5f7128ad0 0x55f5f712aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:32.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.776+0000 7f5da0f9a700 1 -- 172.21.15.101:0/1696295242 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5f7128ad0 msgr2=0x55f5f712aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:32.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.776+0000 7f5da0f9a700 1 --2- 172.21.15.101:0/1696295242 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5f7128ad0 0x55f5f712aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:32.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.776+0000 7f5da0f9a700 1 -- 172.21.15.101:0/1696295242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5f711d2d0 msgr2=0x55f5f711d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:32.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.776+0000 7f5da0f9a700 1 --2- 172.21.15.101:0/1696295242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5f711d2d0 0x55f5f711d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:32.774 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.776+0000 7f5da0f9a700 1 -- 172.21.15.101:0/1696295242 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5f6e9dde0 con 0x55f5f711dbe0 2023-07-18T03:19:32.774 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.776+0000 7f5da0f9a700 1 --2- 172.21.15.101:0/1696295242 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5f711dbe0 0x55f5f7128590 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c009d20 tx=0x7f5d8c011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ffdbe3877b13e7b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:32.774 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.777+0000 7f5d877fe700 1 -- 172.21.15.101:0/1696295242 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c019630 con 0x55f5f711dbe0 2023-07-18T03:19:32.774 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.777+0000 7f5d877fe700 1 -- 172.21.15.101:0/1696295242 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d8c019c50 con 0x55f5f711dbe0 2023-07-18T03:19:32.774 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.777+0000 7f5d877fe700 1 -- 172.21.15.101:0/1696295242 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c00ebe0 con 0x55f5f711dbe0 2023-07-18T03:19:32.776 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.777+0000 7f5db72d0b80 1 -- 172.21.15.101:0/1696295242 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5f711dbe0 msgr2=0x55f5f7128590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:32.777 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.777+0000 7f5db72d0b80 1 --2- 172.21.15.101:0/1696295242 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5f711dbe0 0x55f5f7128590 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c009d20 tx=0x7f5d8c011720 comp rx=0 tx=0).stop 2023-07-18T03:19:32.777 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.777+0000 7f5db72d0b80 1 -- 172.21.15.101:0/1696295242 shutdown_connections 2023-07-18T03:19:32.777 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.777+0000 7f5db72d0b80 1 --2- 172.21.15.101:0/1696295242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5f711d2d0 0x55f5f711d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:32.777 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.777+0000 7f5db72d0b80 1 --2- 172.21.15.101:0/1696295242 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5f7128ad0 0x55f5f712aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:32.777 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.777+0000 7f5db72d0b80 1 --2- 172.21.15.101:0/1696295242 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5f711dbe0 0x55f5f7128590 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:32.777 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.777+0000 7f5db72d0b80 1 -- 172.21.15.101:0/1696295242 >> 172.21.15.101:0/1696295242 conn(0x55f5f7113da0 msgr2=0x55f5f71196a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:32.777 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.778+0000 7f5db72d0b80 1 -- 172.21.15.101:0/1696295242 shutdown_connections 2023-07-18T03:19:32.778 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.778+0000 7f5db72d0b80 1 -- 172.21.15.101:0/1696295242 wait complete. 2023-07-18T03:19:32.778 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.778+0000 7f5db72d0b80 1 -- start start 2023-07-18T03:19:32.778 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.778+0000 7f5db72d0b80 1 Processor -- start 2023-07-18T03:19:32.778 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.779+0000 7f5db72d0b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5f711d2d0 0x55f5f71284e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:32.778 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.779+0000 7f5db72d0b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5f711dbe0 0x55f5f71257d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:32.778 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.779+0000 7f5db72d0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5f7128ad0 0x55f5f7125e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:32.778 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.779+0000 7f5db72d0b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5f700fac0 con 0x55f5f711d2d0 2023-07-18T03:19:32.779 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.779+0000 7f5db72d0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5f700f7c0 con 0x55f5f7128ad0 2023-07-18T03:19:32.779 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.779+0000 7f5db72d0b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5f6ef3920 con 0x55f5f711dbe0 2023-07-18T03:19:32.779 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.779+0000 7f5db72d0b80 10 client.0 _ll_register_callbacks cb 0x55f5f700d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:19:32.779 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.779+0000 7f5da179b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5f711d2d0 0x55f5f71284e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:32.779 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.779+0000 7f5da0f9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5f711dbe0 0x55f5f71257d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:32.779 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.779+0000 7f5da0f9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5f711dbe0 0x55f5f71257d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:44666/0 (socket says 172.21.15.101:44666) 2023-07-18T03:19:32.779 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.779+0000 7f5da0f9a700 1 -- 172.21.15.101:0/147884240 learned_addr learned my addr 172.21.15.101:0/147884240 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:32.779 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.779+0000 7f5da1f9c700 1 --2- 172.21.15.101:0/147884240 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5f7128ad0 0x55f5f7125e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:32.780 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.780+0000 7f5da179b700 1 -- 172.21.15.101:0/147884240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5f711dbe0 msgr2=0x55f5f71257d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:32.780 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.780+0000 7f5da179b700 1 --2- 172.21.15.101:0/147884240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5f711dbe0 0x55f5f71257d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:32.780 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.780+0000 7f5da179b700 1 -- 172.21.15.101:0/147884240 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5f7128ad0 msgr2=0x55f5f7125e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:32.780 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.780+0000 7f5da179b700 1 --2- 172.21.15.101:0/147884240 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5f7128ad0 0x55f5f7125e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:32.780 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.780+0000 7f5da179b700 1 -- 172.21.15.101:0/147884240 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d8c009980 con 0x55f5f711d2d0 2023-07-18T03:19:32.780 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.780+0000 7f5da0f9a700 1 --2- 172.21.15.101:0/147884240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5f711dbe0 0x55f5f71257d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:19:32.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.780+0000 7f5da1f9c700 1 --2- 172.21.15.101:0/147884240 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5f7128ad0 0x55f5f7125e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:19:32.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.780+0000 7f5da179b700 1 --2- 172.21.15.101:0/147884240 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5f711d2d0 0x55f5f71284e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f5d980059d0 tx=0x7f5d9801a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57d3fbb05d137335 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:32.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.780+0000 7f5d84ff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:19:32.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.780+0000 7f5d84ff9700 1 -- 172.21.15.101:0/147884240 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9800f870 con 0x55f5f711d2d0 2023-07-18T03:19:32.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.780+0000 7f5db72d0b80 1 -- 172.21.15.101:0/147884240 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f5f6fc94f0 con 0x55f5f711d2d0 2023-07-18T03:19:32.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.780+0000 7f5d84ff9700 1 -- 172.21.15.101:0/147884240 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f5d9800f9d0 con 0x55f5f711d2d0 2023-07-18T03:19:32.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.780+0000 7f5d84ff9700 1 -- 172.21.15.101:0/147884240 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d9800fbd0 con 0x55f5f711d2d0 2023-07-18T03:19:32.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.780+0000 7f5db72d0b80 10 client.17910 fetch_fsmap learned FSMap version 33 2023-07-18T03:19:32.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.780+0000 7f5db72d0b80 1 -- 172.21.15.101:0/147884240 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55f5f6e86c60 con 0x55f5f711d2d0 2023-07-18T03:19:32.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.781+0000 7f5d84ff9700 1 -- 172.21.15.101:0/147884240 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f5f6fc94f0 con 0x55f5f711d2d0 2023-07-18T03:19:32.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.781+0000 7f5d84ff9700 1 -- 172.21.15.101:0/147884240 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5d98018400 con 0x55f5f711d2d0 2023-07-18T03:19:32.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.781+0000 7f5db72d0b80 10 client.17910 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:19:32.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.781+0000 7f5db72d0b80 10 client.17910 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:19:32.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.781+0000 7f5db72d0b80 20 client.17910 populate_metadata read hostname 'smithi101' 2023-07-18T03:19:32.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.781+0000 7f5db72d0b80 1 --2- 172.21.15.101:0/147884240 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f5f7138410 0x55f5f713a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:32.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.781+0000 7f5db72d0b80 4 client.17910 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:19:32.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.781+0000 7f5db72d0b80 1 -- 172.21.15.101:0/147884240 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f5f7068eb0 con 0x55f5f7138410 2023-07-18T03:19:32.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.781+0000 7f5da0f9a700 1 --2- 172.21.15.101:0/147884240 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f5f7138410 0x55f5f713a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:32.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.782+0000 7f5d84ff9700 10 client.17910 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:19:32.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.782+0000 7f5da0f9a700 1 --2- 172.21.15.101:0/147884240 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f5f7138410 0x55f5f713a7f0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf1d38e93294c139 server_cookie=c7d230ed51cec8f6 in_seq=0 out_seq=0 2023-07-18T03:19:32.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.783+0000 7f5d84ff9700 1 -- 172.21.15.101:0/147884240 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f5f7068eb0 con 0x55f5f7138410 2023-07-18T03:19:32.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.783+0000 7f5d84ff9700 10 client.17910 handle_command_reply: tid=0 2023-07-18T03:19:32.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.783+0000 7f5db72d0b80 1 client.17910 shutdown 2023-07-18T03:19:32.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.783+0000 7f5db72d0b80 1 -- 172.21.15.101:0/147884240 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5f711d2d0 msgr2=0x55f5f71284e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:32.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.783+0000 7f5db72d0b80 1 --2- 172.21.15.101:0/147884240 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5f711d2d0 0x55f5f71284e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f5d980059d0 tx=0x7f5d9801a050 comp rx=0 tx=0).stop 2023-07-18T03:19:32.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.783+0000 7f5db72d0b80 1 -- 172.21.15.101:0/147884240 shutdown_connections 2023-07-18T03:19:32.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.783+0000 7f5db72d0b80 1 --2- 172.21.15.101:0/147884240 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f5f7138410 0x55f5f713a7f0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:32.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.783+0000 7f5db72d0b80 1 --2- 172.21.15.101:0/147884240 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5f7128ad0 0x55f5f7125e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:32.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.783+0000 7f5db72d0b80 1 --2- 172.21.15.101:0/147884240 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5f711d2d0 0x55f5f71284e0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:32.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.783+0000 7f5d84ff9700 0 client.17910 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:19:32.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.783+0000 7f5db72d0b80 1 --2- 172.21.15.101:0/147884240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5f711dbe0 0x55f5f71257d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:32.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.783+0000 7f5db72d0b80 1 -- 172.21.15.101:0/147884240 >> 172.21.15.101:0/147884240 conn(0x55f5f7113da0 msgr2=0x55f5f7119230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:32.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.783+0000 7f5db72d0b80 1 -- 172.21.15.101:0/147884240 shutdown_connections 2023-07-18T03:19:32.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.783+0000 7f5db72d0b80 1 -- 172.21.15.101:0/147884240 wait complete. 2023-07-18T03:19:32.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.784+0000 7f5db72d0b80 20 client.17910 trim_cache size 0 max 16384 2023-07-18T03:19:32.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.800+0000 7f5db72d0b80 1 Processor -- start 2023-07-18T03:19:32.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.801+0000 7f5db72d0b80 1 -- start start 2023-07-18T03:19:32.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.801+0000 7f5db72d0b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5f711d2d0 0x55f5f7128080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:32.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.801+0000 7f5db72d0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5f711dbe0 0x55f5f7124fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:32.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.801+0000 7f5db72d0b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5f7128ad0 0x55f5f71254f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:32.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.801+0000 7f5db72d0b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5f7126d90 con 0x55f5f711d2d0 2023-07-18T03:19:32.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.801+0000 7f5db72d0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5f7110fb0 con 0x55f5f711dbe0 2023-07-18T03:19:32.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.801+0000 7f5db72d0b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5f71139a0 con 0x55f5f7128ad0 2023-07-18T03:19:32.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.801+0000 7f5da179b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5f711d2d0 0x55f5f7128080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:32.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.801+0000 7f5da179b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5f711d2d0 0x55f5f7128080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:37798/0 (socket says 172.21.15.101:37798) 2023-07-18T03:19:32.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.801+0000 7f5da179b700 1 -- 172.21.15.101:0/2309885235 learned_addr learned my addr 172.21.15.101:0/2309885235 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:32.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.801+0000 7f5da1f9c700 1 --2- 172.21.15.101:0/2309885235 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5f7128ad0 0x55f5f71254f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:32.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.801+0000 7f5da0f9a700 1 --2- 172.21.15.101:0/2309885235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5f711dbe0 0x55f5f7124fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:32.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.801+0000 7f5da179b700 1 -- 172.21.15.101:0/2309885235 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5f7128ad0 msgr2=0x55f5f71254f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:32.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.802+0000 7f5da179b700 1 --2- 172.21.15.101:0/2309885235 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5f7128ad0 0x55f5f71254f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:32.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.802+0000 7f5da179b700 1 -- 172.21.15.101:0/2309885235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5f711dbe0 msgr2=0x55f5f7124fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:32.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.802+0000 7f5da179b700 1 --2- 172.21.15.101:0/2309885235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5f711dbe0 0x55f5f7124fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:32.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.802+0000 7f5da179b700 1 -- 172.21.15.101:0/2309885235 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5f6ec4250 con 0x55f5f711d2d0 2023-07-18T03:19:32.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.802+0000 7f5da179b700 1 --2- 172.21.15.101:0/2309885235 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5f711d2d0 0x55f5f7128080 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f5d98018760 tx=0x7f5d98009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e15cb78a16bc6531 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:32.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.802+0000 7f5d87fff700 1 -- 172.21.15.101:0/2309885235 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d98018400 con 0x55f5f711d2d0 2023-07-18T03:19:32.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.802+0000 7f5d87fff700 1 -- 172.21.15.101:0/2309885235 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d9801e040 con 0x55f5f711d2d0 2023-07-18T03:19:32.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.802+0000 7f5d87fff700 1 -- 172.21.15.101:0/2309885235 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d980386d0 con 0x55f5f711d2d0 2023-07-18T03:19:32.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.803+0000 7f5db72d0b80 1 -- 172.21.15.101:0/2309885235 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5f711d2d0 msgr2=0x55f5f7128080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:32.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.803+0000 7f5db72d0b80 1 --2- 172.21.15.101:0/2309885235 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5f711d2d0 0x55f5f7128080 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f5d98018760 tx=0x7f5d98009290 comp rx=0 tx=0).stop 2023-07-18T03:19:32.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.803+0000 7f5db72d0b80 1 -- 172.21.15.101:0/2309885235 shutdown_connections 2023-07-18T03:19:32.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.803+0000 7f5db72d0b80 1 --2- 172.21.15.101:0/2309885235 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5f7128ad0 0x55f5f71254f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:32.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.803+0000 7f5db72d0b80 1 --2- 172.21.15.101:0/2309885235 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5f711d2d0 0x55f5f7128080 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:32.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.803+0000 7f5db72d0b80 1 --2- 172.21.15.101:0/2309885235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5f711dbe0 0x55f5f7124fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:32.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.803+0000 7f5db72d0b80 1 -- 172.21.15.101:0/2309885235 >> 172.21.15.101:0/2309885235 conn(0x55f5f7113da0 msgr2=0x55f5f7120af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:32.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.803+0000 7f5db72d0b80 1 -- 172.21.15.101:0/2309885235 shutdown_connections 2023-07-18T03:19:32.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.803+0000 7f5db72d0b80 1 -- 172.21.15.101:0/2309885235 wait complete. 2023-07-18T03:19:32.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.803+0000 7f5db72d0b80 1 -- start start 2023-07-18T03:19:32.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.803+0000 7f5db72d0b80 1 Processor -- start 2023-07-18T03:19:32.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.804+0000 7f5db72d0b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5f711d2d0 0x55f5f712d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:32.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.804+0000 7f5db72d0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5f711dbe0 0x55f5f712dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:32.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.804+0000 7f5db72d0b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5f7128ad0 0x55f5f718b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:32.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.804+0000 7f5db72d0b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5f718b7e0 con 0x55f5f711d2d0 2023-07-18T03:19:32.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.804+0000 7f5db72d0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5f718b950 con 0x55f5f711dbe0 2023-07-18T03:19:32.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.804+0000 7f5db72d0b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5f718bb10 con 0x55f5f7128ad0 2023-07-18T03:19:32.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.804+0000 7f5db72d0b80 10 client.0 _ll_register_callbacks cb 0x7f5d9800bf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:19:32.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.804+0000 7f5da179b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5f711d2d0 0x55f5f712d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:32.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.804+0000 7f5da0f9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5f711dbe0 0x55f5f712dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:32.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.804+0000 7f5da0f9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5f711dbe0 0x55f5f712dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:55772/0 (socket says 172.21.15.101:55772) 2023-07-18T03:19:32.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.804+0000 7f5da0f9a700 1 -- 172.21.15.101:0/4062277159 learned_addr learned my addr 172.21.15.101:0/4062277159 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:32.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.804+0000 7f5da1f9c700 1 --2- 172.21.15.101:0/4062277159 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5f7128ad0 0x55f5f718b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:32.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.805+0000 7f5da179b700 1 -- 172.21.15.101:0/4062277159 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5f7128ad0 msgr2=0x55f5f718b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:32.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.805+0000 7f5da179b700 1 --2- 172.21.15.101:0/4062277159 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5f7128ad0 0x55f5f718b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:32.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.805+0000 7f5da179b700 1 -- 172.21.15.101:0/4062277159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5f711dbe0 msgr2=0x55f5f712dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:32.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.805+0000 7f5da179b700 1 --2- 172.21.15.101:0/4062277159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5f711dbe0 0x55f5f712dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:32.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.805+0000 7f5da179b700 1 -- 172.21.15.101:0/4062277159 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f5f6e86c60 con 0x55f5f711d2d0 2023-07-18T03:19:32.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.805+0000 7f5da0f9a700 1 --2- 172.21.15.101:0/4062277159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5f711dbe0 0x55f5f712dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:19:32.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.805+0000 7f5da179b700 1 --2- 172.21.15.101:0/4062277159 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5f711d2d0 0x55f5f712d6c0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009ce0 tx=0x7f5d98018630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f574f4c6cd146470 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:32.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.805+0000 7f5d84ff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:19:32.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.805+0000 7f5d84ff9700 1 -- 172.21.15.101:0/4062277159 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9800f870 con 0x55f5f711d2d0 2023-07-18T03:19:32.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.805+0000 7f5db72d0b80 1 -- 172.21.15.101:0/4062277159 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d8c019c50 con 0x55f5f711d2d0 2023-07-18T03:19:32.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.805+0000 7f5d84ff9700 1 -- 172.21.15.101:0/4062277159 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f5d9800f9d0 con 0x55f5f711d2d0 2023-07-18T03:19:32.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.805+0000 7f5d84ff9700 1 -- 172.21.15.101:0/4062277159 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d9800fbd0 con 0x55f5f711d2d0 2023-07-18T03:19:32.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.805+0000 7f5db72d0b80 10 client.17916 fetch_fsmap learned FSMap version 33 2023-07-18T03:19:32.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.805+0000 7f5db72d0b80 1 -- 172.21.15.101:0/4062277159 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55f5f6e9c2b0 con 0x55f5f711d2d0 2023-07-18T03:19:32.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.806+0000 7f5d84ff9700 1 -- 172.21.15.101:0/4062277159 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d8c019c50 con 0x55f5f711d2d0 2023-07-18T03:19:32.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.806+0000 7f5d84ff9700 1 -- 172.21.15.101:0/4062277159 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5d98041530 con 0x55f5f711d2d0 2023-07-18T03:19:32.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.806+0000 7f5db72d0b80 10 client.17916 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:19:32.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.806+0000 7f5db72d0b80 10 client.17916 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:19:32.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.806+0000 7f5db72d0b80 20 client.17916 populate_metadata read hostname 'smithi101' 2023-07-18T03:19:32.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.806+0000 7f5db72d0b80 1 --2- 172.21.15.101:0/4062277159 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f5f718cda0 0x55f5f718f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:32.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.806+0000 7f5db72d0b80 4 client.17916 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:19:32.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.806+0000 7f5db72d0b80 1 -- 172.21.15.101:0/4062277159 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f5f702b050 con 0x55f5f718cda0 2023-07-18T03:19:32.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.806+0000 7f5da0f9a700 1 --2- 172.21.15.101:0/4062277159 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f5f718cda0 0x55f5f718f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:32.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.806+0000 7f5d84ff9700 10 client.17916 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:19:32.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.806+0000 7f5da0f9a700 1 --2- 172.21.15.101:0/4062277159 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f5f718cda0 0x55f5f718f180 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b65cbaee2a78dcb1 server_cookie=ae08762aa2eca77d in_seq=0 out_seq=0 2023-07-18T03:19:32.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.807+0000 7f5d84ff9700 1 -- 172.21.15.101:0/4062277159 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f5f702b050 con 0x55f5f718cda0 2023-07-18T03:19:32.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.807+0000 7f5d84ff9700 10 client.17916 handle_command_reply: tid=0 2023-07-18T03:19:32.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.807+0000 7f5db72d0b80 1 client.17916 shutdown 2023-07-18T03:19:32.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.807+0000 7f5db72d0b80 1 -- 172.21.15.101:0/4062277159 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5f711d2d0 msgr2=0x55f5f712d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:32.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.807+0000 7f5db72d0b80 1 --2- 172.21.15.101:0/4062277159 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5f711d2d0 0x55f5f712d6c0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009ce0 tx=0x7f5d98018630 comp rx=0 tx=0).stop 2023-07-18T03:19:32.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.807+0000 7f5db72d0b80 1 -- 172.21.15.101:0/4062277159 shutdown_connections 2023-07-18T03:19:32.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.807+0000 7f5db72d0b80 1 --2- 172.21.15.101:0/4062277159 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f5f718cda0 0x55f5f718f180 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:32.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.807+0000 7f5db72d0b80 1 --2- 172.21.15.101:0/4062277159 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5f7128ad0 0x55f5f718b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:32.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.807+0000 7f5db72d0b80 1 --2- 172.21.15.101:0/4062277159 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5f711d2d0 0x55f5f712d6c0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:32.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.807+0000 7f5d84ff9700 0 client.17916 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:19:32.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.807+0000 7f5db72d0b80 1 --2- 172.21.15.101:0/4062277159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5f711dbe0 0x55f5f712dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:32.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.807+0000 7f5db72d0b80 1 -- 172.21.15.101:0/4062277159 >> 172.21.15.101:0/4062277159 conn(0x55f5f7113da0 msgr2=0x55f5f711f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:32.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.807+0000 7f5db72d0b80 1 -- 172.21.15.101:0/4062277159 shutdown_connections 2023-07-18T03:19:32.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.807+0000 7f5db72d0b80 1 -- 172.21.15.101:0/4062277159 wait complete. 2023-07-18T03:19:32.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.807+0000 7f5db72d0b80 20 client.17916 trim_cache size 0 max 16384 2023-07-18T03:19:32.810 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:19:32.811 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "no active scrubs running", 2023-07-18T03:19:32.811 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": {} 2023-07-18T03:19:32.811 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:19:32.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.808+0000 7f5da39ff700 1 -- 172.21.15.101:0/3144670141 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f5d880a12c0 msgr2=0x7f5d880a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:32.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.808+0000 7f5da39ff700 1 --2- 172.21.15.101:0/3144670141 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f5d880a12c0 0x7f5d880a3770 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5d9800b2e0 tx=0x7f5d98005960 comp rx=0 tx=0).stop 2023-07-18T03:19:32.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.808+0000 7f5da39ff700 1 -- 172.21.15.101:0/3144670141 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5d9c110ae0 msgr2=0x7f5d9c1a0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:32.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.808+0000 7f5da39ff700 1 --2- 172.21.15.101:0/3144670141 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5d9c110ae0 0x7f5d9c1a0b60 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f5d9400c110 tx=0x7f5d9400c420 comp rx=0 tx=0).stop 2023-07-18T03:19:32.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.808+0000 7f5da39ff700 1 -- 172.21.15.101:0/3144670141 shutdown_connections 2023-07-18T03:19:32.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.808+0000 7f5da39ff700 1 --2- 172.21.15.101:0/3144670141 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5d9c110ae0 0x7f5d9c1a0b60 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:32.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.808+0000 7f5da39ff700 1 --2- 172.21.15.101:0/3144670141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d9c0728b0 0x7f5d9c19c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:32.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.808+0000 7f5da39ff700 1 --2- 172.21.15.101:0/3144670141 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f5d880a12c0 0x7f5d880a3770 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:32.813 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.808+0000 7f5da39ff700 1 --2- 172.21.15.101:0/3144670141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d9c0731c0 0x7f5d9c19c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:32.813 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.808+0000 7f5da39ff700 1 -- 172.21.15.101:0/3144670141 >> 172.21.15.101:0/3144670141 conn(0x7f5d9c0fb5f0 msgr2=0x7f5d9c0fc250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:32.813 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.808+0000 7f5da39ff700 1 -- 172.21.15.101:0/3144670141 shutdown_connections 2023-07-18T03:19:32.813 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:32.808+0000 7f5da39ff700 1 -- 172.21.15.101:0/3144670141 wait complete. 2023-07-18T03:19:32.820 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-18T03:19:32.820 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:19:33.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.028+0000 7fc444bff700 1 -- 172.21.15.101:0/3950954875 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc440105b00 msgr2=0x7fc440105ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:33.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.028+0000 7fc444bff700 1 --2- 172.21.15.101:0/3950954875 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc440105b00 0x7fc440105ed0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fc430009990 tx=0x7fc430009ca0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.028+0000 7fc444bff700 1 -- 172.21.15.101:0/3950954875 shutdown_connections 2023-07-18T03:19:33.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.028+0000 7fc444bff700 1 --2- 172.21.15.101:0/3950954875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4401004a0 0x7fc44010c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.028+0000 7fc444bff700 1 --2- 172.21.15.101:0/3950954875 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc440105b00 0x7fc440105ed0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.028+0000 7fc444bff700 1 --2- 172.21.15.101:0/3950954875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4400ffaf0 0x7fc4400fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.028+0000 7fc444bff700 1 -- 172.21.15.101:0/3950954875 >> 172.21.15.101:0/3950954875 conn(0x7fc4400fb630 msgr2=0x7fc4400fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:33.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.028+0000 7fc444bff700 1 -- 172.21.15.101:0/3950954875 shutdown_connections 2023-07-18T03:19:33.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.028+0000 7fc444bff700 1 -- 172.21.15.101:0/3950954875 wait complete. 2023-07-18T03:19:33.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.029+0000 7fc444bff700 1 Processor -- start 2023-07-18T03:19:33.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.029+0000 7fc444bff700 1 -- start start 2023-07-18T03:19:33.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.029+0000 7fc444bff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4400ffaf0 0x7fc440072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:33.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.029+0000 7fc444bff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4401004a0 0x7fc440072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:33.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.029+0000 7fc444bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc440105b00 0x7fc4401a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:33.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.029+0000 7fc444bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc440073580 con 0x7fc440105b00 2023-07-18T03:19:33.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.029+0000 7fc444bff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4401a42e0 con 0x7fc4400ffaf0 2023-07-18T03:19:33.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.029+0000 7fc444bff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4401a4420 con 0x7fc4401004a0 2023-07-18T03:19:33.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.030+0000 7fc43ed9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc440105b00 0x7fc4401a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:33.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.030+0000 7fc43ed9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc440105b00 0x7fc4401a3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:37834/0 (socket says 172.21.15.101:37834) 2023-07-18T03:19:33.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.030+0000 7fc43ed9d700 1 -- 172.21.15.101:0/1937854468 learned_addr learned my addr 172.21.15.101:0/1937854468 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:33.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.030+0000 7fc437fff700 1 --2- 172.21.15.101:0/1937854468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4401004a0 0x7fc440072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:33.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.030+0000 7fc43e59c700 1 --2- 172.21.15.101:0/1937854468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4400ffaf0 0x7fc440072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:33.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.030+0000 7fc43ed9d700 1 -- 172.21.15.101:0/1937854468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4401004a0 msgr2=0x7fc440072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:33.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.030+0000 7fc43ed9d700 1 --2- 172.21.15.101:0/1937854468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4401004a0 0x7fc440072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.030+0000 7fc43ed9d700 1 -- 172.21.15.101:0/1937854468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4400ffaf0 msgr2=0x7fc440072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:33.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.030+0000 7fc43ed9d700 1 --2- 172.21.15.101:0/1937854468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4400ffaf0 0x7fc440072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.030+0000 7fc43ed9d700 1 -- 172.21.15.101:0/1937854468 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4300095f0 con 0x7fc440105b00 2023-07-18T03:19:33.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.030+0000 7fc43e59c700 1 --2- 172.21.15.101:0/1937854468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4400ffaf0 0x7fc440072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:19:33.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.030+0000 7fc43ed9d700 1 --2- 172.21.15.101:0/1937854468 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc440105b00 0x7fc4401a3da0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fc438007ee0 tx=0x7fc43800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:33.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.031+0000 7fc4377fe700 1 -- 172.21.15.101:0/1937854468 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc438017070 con 0x7fc440105b00 2023-07-18T03:19:33.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.031+0000 7fc4377fe700 1 -- 172.21.15.101:0/1937854468 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc43800f040 con 0x7fc440105b00 2023-07-18T03:19:33.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.031+0000 7fc4377fe700 1 -- 172.21.15.101:0/1937854468 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc438005740 con 0x7fc440105b00 2023-07-18T03:19:33.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.031+0000 7fc444bff700 1 -- 172.21.15.101:0/1937854468 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4401a4680 con 0x7fc440105b00 2023-07-18T03:19:33.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.031+0000 7fc444bff700 1 -- 172.21.15.101:0/1937854468 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4401a4b60 con 0x7fc440105b00 2023-07-18T03:19:33.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.032+0000 7fc444bff700 1 -- 172.21.15.101:0/1937854468 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc440067300 con 0x7fc440105b00 2023-07-18T03:19:33.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.034+0000 7fc4377fe700 1 -- 172.21.15.101:0/1937854468 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fc43800dc60 con 0x7fc440105b00 2023-07-18T03:19:33.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.035+0000 7fc4377fe700 1 --2- 172.21.15.101:0/1937854468 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc42c0a1380 0x7fc42c0a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:33.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.035+0000 7fc4377fe700 1 -- 172.21.15.101:0/1937854468 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fc4380cce90 con 0x7fc440105b00 2023-07-18T03:19:33.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.035+0000 7fc43e59c700 1 --2- 172.21.15.101:0/1937854468 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc42c0a1380 0x7fc42c0a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:33.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.036+0000 7fc43e59c700 1 --2- 172.21.15.101:0/1937854468 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc42c0a1380 0x7fc42c0a3830 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc43000b5c0 tx=0x7fc430005fd0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:33.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.037+0000 7fc4377fe700 1 -- 172.21.15.101:0/1937854468 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc438095e90 con 0x7fc440105b00 2023-07-18T03:19:33.239 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:33 smithi101 ceph-mon[126379]: pgmap v374: 129 pgs: 129 active+clean; 586 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 106 op/s 2023-07-18T03:19:33.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.280+0000 7fc444bff700 1 -- 172.21.15.101:0/1937854468 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc440073d40 con 0x7fc440105b00 2023-07-18T03:19:33.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.281+0000 7fc4377fe700 1 -- 172.21.15.101:0/1937854468 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+10858 (secure 0 0 0) 0x7fc4380994b0 con 0x7fc440105b00 2023-07-18T03:19:33.275 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:19:33.276 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/4192846769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":4192846769},{"type":"v1","addr":"172.21.15.101:6839","nonce":4192846769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6841/3824340175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6840","nonce":3824340175},{"type":"v1","addr":"172.21.15.101:6841","nonce":3824340175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:04:59.361185+0000","modified":"2023-07-18T03:16:36.002358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.155:6839/2055462483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2055462483},{"type":"v1","addr":"172.21.15.155:6839","nonce":2055462483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:19:33.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.283+0000 7fc444bff700 1 -- 172.21.15.101:0/1937854468 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc42c0a1380 msgr2=0x7fc42c0a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:33.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.283+0000 7fc444bff700 1 --2- 172.21.15.101:0/1937854468 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc42c0a1380 0x7fc42c0a3830 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc43000b5c0 tx=0x7fc430005fd0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.283+0000 7fc444bff700 1 -- 172.21.15.101:0/1937854468 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc440105b00 msgr2=0x7fc4401a3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:33.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.283+0000 7fc444bff700 1 --2- 172.21.15.101:0/1937854468 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc440105b00 0x7fc4401a3da0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fc438007ee0 tx=0x7fc43800d3b0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.283+0000 7fc444bff700 1 -- 172.21.15.101:0/1937854468 shutdown_connections 2023-07-18T03:19:33.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.283+0000 7fc444bff700 1 --2- 172.21.15.101:0/1937854468 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc42c0a1380 0x7fc42c0a3830 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.283+0000 7fc444bff700 1 --2- 172.21.15.101:0/1937854468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4400ffaf0 0x7fc440072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.283+0000 7fc444bff700 1 --2- 172.21.15.101:0/1937854468 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc440105b00 0x7fc4401a3da0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.283+0000 7fc444bff700 1 --2- 172.21.15.101:0/1937854468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4401004a0 0x7fc440072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.283+0000 7fc444bff700 1 -- 172.21.15.101:0/1937854468 >> 172.21.15.101:0/1937854468 conn(0x7fc4400fb630 msgr2=0x7fc440103600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:33.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.283+0000 7fc444bff700 1 -- 172.21.15.101:0/1937854468 shutdown_connections 2023-07-18T03:19:33.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.283+0000 7fc444bff700 1 -- 172.21.15.101:0/1937854468 wait complete. 2023-07-18T03:19:33.279 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 33 2023-07-18T03:19:33.296 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-18T03:19:33.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:33 smithi155 ceph-mon[128885]: pgmap v374: 129 pgs: 129 active+clean; 586 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 106 op/s 2023-07-18T03:19:33.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:33 smithi179 ceph-mon[129229]: pgmap v374: 129 pgs: 129 active+clean; 586 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 106 op/s 2023-07-18T03:19:33.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.506+0000 7fa7b9bff700 1 -- 172.21.15.101:0/1379448849 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa7b4105ae0 msgr2=0x7fa7b4105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:33.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.506+0000 7fa7b9bff700 1 --2- 172.21.15.101:0/1379448849 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa7b4105ae0 0x7fa7b4105eb0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4009990 tx=0x7fa7a4009ca0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.506+0000 7fa7b9bff700 1 -- 172.21.15.101:0/1379448849 shutdown_connections 2023-07-18T03:19:33.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.506+0000 7fa7b9bff700 1 --2- 172.21.15.101:0/1379448849 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7b4100480 0x7fa7b410c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.506+0000 7fa7b9bff700 1 --2- 172.21.15.101:0/1379448849 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa7b4105ae0 0x7fa7b4105eb0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.506+0000 7fa7b9bff700 1 --2- 172.21.15.101:0/1379448849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7b40ffad0 0x7fa7b40fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.506+0000 7fa7b9bff700 1 -- 172.21.15.101:0/1379448849 >> 172.21.15.101:0/1379448849 conn(0x7fa7b40fb5f0 msgr2=0x7fa7b40fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:33.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.506+0000 7fa7b9bff700 1 -- 172.21.15.101:0/1379448849 shutdown_connections 2023-07-18T03:19:33.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.506+0000 7fa7b9bff700 1 -- 172.21.15.101:0/1379448849 wait complete. 2023-07-18T03:19:33.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.507+0000 7fa7b9bff700 1 Processor -- start 2023-07-18T03:19:33.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.507+0000 7fa7b9bff700 1 -- start start 2023-07-18T03:19:33.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.507+0000 7fa7b9bff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7b40ffad0 0x7fa7b4197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:33.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.507+0000 7fa7b9bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa7b4100480 0x7fa7b4198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:33.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.507+0000 7fa7b9bff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7b4105ae0 0x7fa7b419c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:33.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.507+0000 7fa7b9bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b419cdd0 con 0x7fa7b4100480 2023-07-18T03:19:33.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.507+0000 7fa7b9bff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b419cf40 con 0x7fa7b40ffad0 2023-07-18T03:19:33.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.507+0000 7fa7b9bff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b419d100 con 0x7fa7b4105ae0 2023-07-18T03:19:33.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.507+0000 7fa7abfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa7b4100480 0x7fa7b4198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:33.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.507+0000 7fa7abfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa7b4100480 0x7fa7b4198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:37846/0 (socket says 172.21.15.101:37846) 2023-07-18T03:19:33.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.508+0000 7fa7abfff700 1 -- 172.21.15.101:0/4239960037 learned_addr learned my addr 172.21.15.101:0/4239960037 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:33.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.508+0000 7fa7b37fe700 1 --2- 172.21.15.101:0/4239960037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7b40ffad0 0x7fa7b4197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:33.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.508+0000 7fa7b3fff700 1 --2- 172.21.15.101:0/4239960037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7b4105ae0 0x7fa7b419c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:33.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.508+0000 7fa7abfff700 1 -- 172.21.15.101:0/4239960037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7b4105ae0 msgr2=0x7fa7b419c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:33.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.508+0000 7fa7abfff700 1 --2- 172.21.15.101:0/4239960037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7b4105ae0 0x7fa7b419c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.508+0000 7fa7abfff700 1 -- 172.21.15.101:0/4239960037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7b40ffad0 msgr2=0x7fa7b4197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:33.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.508+0000 7fa7abfff700 1 --2- 172.21.15.101:0/4239960037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7b40ffad0 0x7fa7b4197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.508+0000 7fa7abfff700 1 -- 172.21.15.101:0/4239960037 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a40095f0 con 0x7fa7b4100480 2023-07-18T03:19:33.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.508+0000 7fa7b3fff700 1 --2- 172.21.15.101:0/4239960037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7b4105ae0 0x7fa7b419c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:19:33.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.508+0000 7fa7abfff700 1 --2- 172.21.15.101:0/4239960037 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa7b4100480 0x7fa7b4198330 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fa79c00d950 tx=0x7fa79c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:33.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.508+0000 7fa7b17fa700 1 -- 172.21.15.101:0/4239960037 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79c009940 con 0x7fa7b4100480 2023-07-18T03:19:33.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.508+0000 7fa7b17fa700 1 -- 172.21.15.101:0/4239960037 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa79c010460 con 0x7fa7b4100480 2023-07-18T03:19:33.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.508+0000 7fa7b17fa700 1 -- 172.21.15.101:0/4239960037 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79c00f620 con 0x7fa7b4100480 2023-07-18T03:19:33.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.508+0000 7fa7b9bff700 1 -- 172.21.15.101:0/4239960037 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7b419d3e0 con 0x7fa7b4100480 2023-07-18T03:19:33.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.509+0000 7fa7b9bff700 1 -- 172.21.15.101:0/4239960037 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7b41a4c20 con 0x7fa7b4100480 2023-07-18T03:19:33.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.512+0000 7fa7b17fa700 1 -- 172.21.15.101:0/4239960037 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fa79c0108d0 con 0x7fa7b4100480 2023-07-18T03:19:33.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.513+0000 7fa7cd580b80 1 Processor -- start 2023-07-18T03:19:33.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.513+0000 7fa7cd580b80 1 -- start start 2023-07-18T03:19:33.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.513+0000 7fa7cd580b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ddd6db02d0 0x55ddd6db06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:33.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.513+0000 7fa7cd580b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ddd6db0be0 0x55ddd6dbb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:33.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.513+0000 7fa7cd580b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ddd6dbbad0 0x55ddd6dbdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:33.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.513+0000 7fa7cd580b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddd6cb6cb0 con 0x55ddd6db02d0 2023-07-18T03:19:33.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.513+0000 7fa7cd580b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddd6d09a40 con 0x55ddd6db0be0 2023-07-18T03:19:33.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.513+0000 7fa7cd580b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddd6d02ed0 con 0x55ddd6dbbad0 2023-07-18T03:19:33.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.513+0000 7fa7b17fa700 1 --2- 172.21.15.101:0/4239960037 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa7a00a1340 0x7fa7a00a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:33.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.513+0000 7fa7b17fa700 1 -- 172.21.15.101:0/4239960037 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fa79c0ccf30 con 0x7fa7b4100480 2023-07-18T03:19:33.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.514+0000 7fa7abfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ddd6dbbad0 0x55ddd6dbdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:33.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.514+0000 7fa7b37fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ddd6db0be0 0x55ddd6dbb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:33.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.514+0000 7fa7b3fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ddd6db02d0 0x55ddd6db06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:33.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.514+0000 7fa7abfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ddd6dbbad0 0x55ddd6dbdeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:44746/0 (socket says 172.21.15.101:44746) 2023-07-18T03:19:33.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.514+0000 7fa7abfff700 1 -- 172.21.15.101:0/2696672796 learned_addr learned my addr 172.21.15.101:0/2696672796 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:33.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.514+0000 7fa7b3fff700 1 --2- 172.21.15.101:0/4239960037 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa7a00a1340 0x7fa7a00a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:33.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.514+0000 7fa7abfff700 1 -- 172.21.15.101:0/2696672796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ddd6db0be0 msgr2=0x55ddd6dbb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:33.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.514+0000 7fa7abfff700 1 --2- 172.21.15.101:0/2696672796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ddd6db0be0 0x55ddd6dbb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.514+0000 7fa7abfff700 1 -- 172.21.15.101:0/2696672796 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ddd6db02d0 msgr2=0x55ddd6db06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:33.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.514+0000 7fa7abfff700 1 --2- 172.21.15.101:0/2696672796 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ddd6db02d0 0x55ddd6db06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.514+0000 7fa7abfff700 1 -- 172.21.15.101:0/2696672796 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddd6b30de0 con 0x55ddd6dbbad0 2023-07-18T03:19:33.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.514+0000 7fa7abfff700 1 --2- 172.21.15.101:0/2696672796 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ddd6dbbad0 0x55ddd6dbdeb0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa79c005950 tx=0x7fa79c096060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e5e335a7b838f0e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:33.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.514+0000 7fa7b3fff700 1 --2- 172.21.15.101:0/4239960037 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa7a00a1340 0x7fa7a00a37f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac009fd0 tx=0x7fa7ac009680 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:33.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.515+0000 7fa7a9ffb700 1 -- 172.21.15.101:0/2696672796 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79c099bf0 con 0x55ddd6dbbad0 2023-07-18T03:19:33.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.515+0000 7fa7a9ffb700 1 -- 172.21.15.101:0/2696672796 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa79c09dc20 con 0x55ddd6dbbad0 2023-07-18T03:19:33.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.515+0000 7fa7a9ffb700 1 -- 172.21.15.101:0/2696672796 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79c0990e0 con 0x55ddd6dbbad0 2023-07-18T03:19:33.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.515+0000 7fa7cd580b80 1 -- 172.21.15.101:0/2696672796 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ddd6dbbad0 msgr2=0x55ddd6dbdeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:33.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.515+0000 7fa7cd580b80 1 --2- 172.21.15.101:0/2696672796 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ddd6dbbad0 0x55ddd6dbdeb0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa79c005950 tx=0x7fa79c096060 comp rx=0 tx=0).stop 2023-07-18T03:19:33.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.515+0000 7fa7cd580b80 1 -- 172.21.15.101:0/2696672796 shutdown_connections 2023-07-18T03:19:33.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.515+0000 7fa7cd580b80 1 --2- 172.21.15.101:0/2696672796 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ddd6dbbad0 0x55ddd6dbdeb0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.515+0000 7fa7cd580b80 1 --2- 172.21.15.101:0/2696672796 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ddd6db02d0 0x55ddd6db06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.515+0000 7fa7cd580b80 1 --2- 172.21.15.101:0/2696672796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ddd6db0be0 0x55ddd6dbb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.515+0000 7fa7cd580b80 1 -- 172.21.15.101:0/2696672796 >> 172.21.15.101:0/2696672796 conn(0x55ddd6da6da0 msgr2=0x55ddd6dac6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:33.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.516+0000 7fa7cd580b80 1 -- 172.21.15.101:0/2696672796 shutdown_connections 2023-07-18T03:19:33.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.516+0000 7fa7cd580b80 1 -- 172.21.15.101:0/2696672796 wait complete. 2023-07-18T03:19:33.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.516+0000 7fa7cd580b80 1 -- start start 2023-07-18T03:19:33.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.516+0000 7fa7cd580b80 1 Processor -- start 2023-07-18T03:19:33.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.517+0000 7fa7cd580b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ddd6db02d0 0x55ddd6dbb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:33.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.517+0000 7fa7cd580b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ddd6db0be0 0x55ddd6db8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:33.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.517+0000 7fa7cd580b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ddd6dbbad0 0x55ddd6db8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:33.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.517+0000 7fa7cd580b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddd6ca2ac0 con 0x55ddd6dbbad0 2023-07-18T03:19:33.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.517+0000 7fa7cd580b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddd6ca27c0 con 0x55ddd6db0be0 2023-07-18T03:19:33.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.517+0000 7fa7cd580b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddd6b86920 con 0x55ddd6db02d0 2023-07-18T03:19:33.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.517+0000 7fa7cd580b80 10 client.0 _ll_register_callbacks cb 0x55ddd6ca0760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:19:33.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.517+0000 7fa7b37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ddd6db02d0 0x55ddd6dbb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:33.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.517+0000 7fa7b3fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ddd6dbbad0 0x55ddd6db8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:33.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.517+0000 7fa7b37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ddd6db02d0 0x55ddd6dbb510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:44756/0 (socket says 172.21.15.101:44756) 2023-07-18T03:19:33.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.517+0000 7fa7b37fe700 1 -- 172.21.15.101:0/2931208818 learned_addr learned my addr 172.21.15.101:0/2931208818 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:33.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.517+0000 7fa7abfff700 1 --2- 172.21.15.101:0/2931208818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ddd6db0be0 0x55ddd6db8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:33.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.517+0000 7fa7b3fff700 1 -- 172.21.15.101:0/2931208818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ddd6db02d0 msgr2=0x55ddd6dbb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:33.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.517+0000 7fa7b3fff700 1 --2- 172.21.15.101:0/2931208818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ddd6db02d0 0x55ddd6dbb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.517+0000 7fa7b3fff700 1 -- 172.21.15.101:0/2931208818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ddd6db0be0 msgr2=0x55ddd6db8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:33.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.517+0000 7fa7b3fff700 1 --2- 172.21.15.101:0/2931208818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ddd6db0be0 0x55ddd6db8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.517+0000 7fa7b3fff700 1 -- 172.21.15.101:0/2931208818 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7b41a4c20 con 0x55ddd6dbbad0 2023-07-18T03:19:33.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.517+0000 7fa7b37fe700 1 --2- 172.21.15.101:0/2931208818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ddd6db02d0 0x55ddd6dbb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:19:33.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.517+0000 7fa7abfff700 1 --2- 172.21.15.101:0/2931208818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ddd6db0be0 0x55ddd6db8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:19:33.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.518+0000 7fa7b3fff700 1 --2- 172.21.15.101:0/2931208818 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ddd6dbbad0 0x55ddd6db8ed0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac009550 tx=0x7fa7ac004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e6351401b7ef44 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:33.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.518+0000 7fa792ffd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:19:33.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.518+0000 7fa792ffd700 1 -- 172.21.15.101:0/2931208818 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac020070 con 0x55ddd6dbbad0 2023-07-18T03:19:33.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.518+0000 7fa7cd580b80 1 -- 172.21.15.101:0/2931208818 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ddd6c5c4f0 con 0x55ddd6dbbad0 2023-07-18T03:19:33.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.518+0000 7fa792ffd700 1 -- 172.21.15.101:0/2931208818 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fa7ac00f070 con 0x55ddd6dbbad0 2023-07-18T03:19:33.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.518+0000 7fa792ffd700 1 -- 172.21.15.101:0/2931208818 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7ac00e030 con 0x55ddd6dbbad0 2023-07-18T03:19:33.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.518+0000 7fa7cd580b80 10 client.17934 fetch_fsmap learned FSMap version 33 2023-07-18T03:19:33.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.518+0000 7fa7cd580b80 1 -- 172.21.15.101:0/2931208818 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55ddd6b19c60 con 0x55ddd6dbbad0 2023-07-18T03:19:33.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.519+0000 7fa792ffd700 1 -- 172.21.15.101:0/2931208818 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ddd6c5c4f0 con 0x55ddd6dbbad0 2023-07-18T03:19:33.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.519+0000 7fa792ffd700 1 -- 172.21.15.101:0/2931208818 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa7ac026a20 con 0x55ddd6dbbad0 2023-07-18T03:19:33.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.519+0000 7fa7cd580b80 10 client.17934 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:19:33.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.519+0000 7fa7cd580b80 10 client.17934 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:19:33.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.519+0000 7fa7cd580b80 20 client.17934 populate_metadata read hostname 'smithi101' 2023-07-18T03:19:33.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.519+0000 7fa7cd580b80 1 --2- 172.21.15.101:0/2931208818 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55ddd6dcb430 0x55ddd6dcd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:33.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.519+0000 7fa7cd580b80 4 client.17934 mds_command: new command op to 24541 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:19:33.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.519+0000 7fa7cd580b80 1 -- 172.21.15.101:0/2931208818 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ddd6cfbeb0 con 0x55ddd6dcb430 2023-07-18T03:19:33.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.519+0000 7fa7b37fe700 1 --2- 172.21.15.101:0/2931208818 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55ddd6dcb430 0x55ddd6dcd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:33.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.520+0000 7fa7b37fe700 1 --2- 172.21.15.101:0/2931208818 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55ddd6dcb430 0x55ddd6dcd840 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=591bdf2d0543ec02 server_cookie=689a47982f3d2d7a in_seq=0 out_seq=0 2023-07-18T03:19:33.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.520+0000 7fa792ffd700 10 client.17934 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:19:33.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.521+0000 7fa792ffd700 1 -- 172.21.15.101:0/2931208818 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ddd6cfbeb0 con 0x55ddd6dcb430 2023-07-18T03:19:33.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.521+0000 7fa792ffd700 10 client.17934 handle_command_reply: tid=0 2023-07-18T03:19:33.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.521+0000 7fa7cd580b80 1 client.17934 shutdown 2023-07-18T03:19:33.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.521+0000 7fa7cd580b80 1 -- 172.21.15.101:0/2931208818 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ddd6dbbad0 msgr2=0x55ddd6db8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:33.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.521+0000 7fa7cd580b80 1 --2- 172.21.15.101:0/2931208818 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ddd6dbbad0 0x55ddd6db8ed0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac009550 tx=0x7fa7ac004ad0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.521+0000 7fa7cd580b80 1 -- 172.21.15.101:0/2931208818 shutdown_connections 2023-07-18T03:19:33.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.521+0000 7fa7cd580b80 1 --2- 172.21.15.101:0/2931208818 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55ddd6dcb430 0x55ddd6dcd840 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.521+0000 7fa7cd580b80 1 --2- 172.21.15.101:0/2931208818 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ddd6dbbad0 0x55ddd6db8ed0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.521+0000 7fa7cd580b80 1 --2- 172.21.15.101:0/2931208818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ddd6db02d0 0x55ddd6dbb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.521+0000 7fa7cd580b80 1 --2- 172.21.15.101:0/2931208818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ddd6db0be0 0x55ddd6db8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.521+0000 7fa792ffd700 0 client.17934 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:19:33.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.521+0000 7fa7cd580b80 1 -- 172.21.15.101:0/2931208818 >> 172.21.15.101:0/2931208818 conn(0x55ddd6da6da0 msgr2=0x55ddd6dac290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:33.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.521+0000 7fa7cd580b80 1 -- 172.21.15.101:0/2931208818 shutdown_connections 2023-07-18T03:19:33.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.521+0000 7fa7cd580b80 1 -- 172.21.15.101:0/2931208818 wait complete. 2023-07-18T03:19:33.524 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.522+0000 7fa7cd580b80 20 client.17934 trim_cache size 0 max 16384 2023-07-18T03:19:33.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.535+0000 7fa7cd580b80 1 Processor -- start 2023-07-18T03:19:33.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.535+0000 7fa7cd580b80 1 -- start start 2023-07-18T03:19:33.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.535+0000 7fa7cd580b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ddd6db02d0 0x55ddd6db3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:33.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.535+0000 7fa7cd580b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ddd6db0be0 0x55ddd6db3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:33.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.535+0000 7fa7cd580b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ddd6dbbad0 0x55ddd6db7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:33.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.535+0000 7fa7cd580b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddd6c5c4f0 con 0x55ddd6dbbad0 2023-07-18T03:19:33.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.535+0000 7fa7cd580b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddd6da8ec0 con 0x55ddd6db02d0 2023-07-18T03:19:33.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.535+0000 7fa7cd580b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddd6da3ef0 con 0x55ddd6db0be0 2023-07-18T03:19:33.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.535+0000 7fa7b3fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ddd6dbbad0 0x55ddd6db7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:33.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.535+0000 7fa7b3fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ddd6dbbad0 0x55ddd6db7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:37878/0 (socket says 172.21.15.101:37878) 2023-07-18T03:19:33.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.535+0000 7fa7b3fff700 1 -- 172.21.15.101:0/1647845823 learned_addr learned my addr 172.21.15.101:0/1647845823 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:33.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.535+0000 7fa7b37fe700 1 --2- 172.21.15.101:0/1647845823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ddd6db02d0 0x55ddd6db3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:33.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.535+0000 7fa7abfff700 1 --2- 172.21.15.101:0/1647845823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ddd6db0be0 0x55ddd6db3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:33.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.536+0000 7fa7b3fff700 1 -- 172.21.15.101:0/1647845823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ddd6db0be0 msgr2=0x55ddd6db3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:33.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.536+0000 7fa7b3fff700 1 --2- 172.21.15.101:0/1647845823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ddd6db0be0 0x55ddd6db3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.536+0000 7fa7b3fff700 1 -- 172.21.15.101:0/1647845823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ddd6db02d0 msgr2=0x55ddd6db3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:33.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.536+0000 7fa7b3fff700 1 --2- 172.21.15.101:0/1647845823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ddd6db02d0 0x55ddd6db3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.536+0000 7fa7b3fff700 1 -- 172.21.15.101:0/1647845823 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddd6b57250 con 0x55ddd6dbbad0 2023-07-18T03:19:33.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.536+0000 7fa7abfff700 1 --2- 172.21.15.101:0/1647845823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ddd6db0be0 0x55ddd6db3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:19:33.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.536+0000 7fa7b3fff700 1 --2- 172.21.15.101:0/1647845823 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ddd6dbbad0 0x55ddd6db7f50 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac007560 tx=0x7fa7ac018f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afbba30fd0b3215 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:33.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.536+0000 7fa7aa7fc700 1 -- 172.21.15.101:0/1647845823 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac020070 con 0x55ddd6dbbad0 2023-07-18T03:19:33.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.536+0000 7fa7aa7fc700 1 -- 172.21.15.101:0/1647845823 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7ac009af0 con 0x55ddd6dbbad0 2023-07-18T03:19:33.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.536+0000 7fa7aa7fc700 1 -- 172.21.15.101:0/1647845823 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac026510 con 0x55ddd6dbbad0 2023-07-18T03:19:33.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.537+0000 7fa7cd580b80 1 -- 172.21.15.101:0/1647845823 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ddd6dbbad0 msgr2=0x55ddd6db7f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:33.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.537+0000 7fa7cd580b80 1 --2- 172.21.15.101:0/1647845823 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ddd6dbbad0 0x55ddd6db7f50 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac007560 tx=0x7fa7ac018f30 comp rx=0 tx=0).stop 2023-07-18T03:19:33.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.537+0000 7fa7cd580b80 1 -- 172.21.15.101:0/1647845823 shutdown_connections 2023-07-18T03:19:33.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.537+0000 7fa7cd580b80 1 --2- 172.21.15.101:0/1647845823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ddd6db02d0 0x55ddd6db3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.537+0000 7fa7cd580b80 1 --2- 172.21.15.101:0/1647845823 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ddd6dbbad0 0x55ddd6db7f50 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.537+0000 7fa7cd580b80 1 --2- 172.21.15.101:0/1647845823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ddd6db0be0 0x55ddd6db3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.537+0000 7fa7cd580b80 1 -- 172.21.15.101:0/1647845823 >> 172.21.15.101:0/1647845823 conn(0x55ddd6da6da0 msgr2=0x55ddd6dc77c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:33.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.537+0000 7fa7cd580b80 1 -- 172.21.15.101:0/1647845823 shutdown_connections 2023-07-18T03:19:33.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.537+0000 7fa7cd580b80 1 -- 172.21.15.101:0/1647845823 wait complete. 2023-07-18T03:19:33.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.537+0000 7fa7cd580b80 1 -- start start 2023-07-18T03:19:33.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.537+0000 7fa7cd580b80 1 Processor -- start 2023-07-18T03:19:33.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.538+0000 7fa7cd580b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ddd6db02d0 0x55ddd6dc2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:33.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.538+0000 7fa7cd580b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ddd6db0be0 0x55ddd6dc06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:33.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.538+0000 7fa7cd580b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ddd6dbbad0 0x55ddd6dc0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:33.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.538+0000 7fa7cd580b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddd6dc11d0 con 0x55ddd6db02d0 2023-07-18T03:19:33.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.538+0000 7fa7cd580b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddd6dc1340 con 0x55ddd6db0be0 2023-07-18T03:19:33.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.538+0000 7fa7cd580b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddd6dc1500 con 0x55ddd6dbbad0 2023-07-18T03:19:33.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.538+0000 7fa7cd580b80 10 client.0 _ll_register_callbacks cb 0x55ddd6ca0760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:19:33.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.538+0000 7fa7b37fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ddd6db02d0 0x55ddd6dc2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:33.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.538+0000 7fa7b37fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ddd6db02d0 0x55ddd6dc2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:37886/0 (socket says 172.21.15.101:37886) 2023-07-18T03:19:33.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.538+0000 7fa7b37fe700 1 -- 172.21.15.101:0/1463670884 learned_addr learned my addr 172.21.15.101:0/1463670884 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:33.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.538+0000 7fa7abfff700 1 --2- 172.21.15.101:0/1463670884 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ddd6db0be0 0x55ddd6dc06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:33.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.538+0000 7fa7b3fff700 1 --2- 172.21.15.101:0/1463670884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ddd6dbbad0 0x55ddd6dc0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:33.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.538+0000 7fa7b37fe700 1 -- 172.21.15.101:0/1463670884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ddd6dbbad0 msgr2=0x55ddd6dc0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:33.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.538+0000 7fa7b37fe700 1 --2- 172.21.15.101:0/1463670884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ddd6dbbad0 0x55ddd6dc0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.538+0000 7fa7b37fe700 1 -- 172.21.15.101:0/1463670884 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ddd6db0be0 msgr2=0x55ddd6dc06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:33.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.538+0000 7fa7b37fe700 1 --2- 172.21.15.101:0/1463670884 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ddd6db0be0 0x55ddd6dc06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.538+0000 7fa7b37fe700 1 -- 172.21.15.101:0/1463670884 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ddd6b19c60 con 0x55ddd6db02d0 2023-07-18T03:19:33.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.538+0000 7fa7b3fff700 1 --2- 172.21.15.101:0/1463670884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ddd6dbbad0 0x55ddd6dc0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:19:33.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.539+0000 7fa7b37fe700 1 --2- 172.21.15.101:0/1463670884 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ddd6db02d0 0x55ddd6dc2400 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4005f50 tx=0x7fa7a400bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=506a100f04e1aa6b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:33.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.539+0000 7fa792ffd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:19:33.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.539+0000 7fa792ffd700 1 -- 172.21.15.101:0/1463670884 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a401d040 con 0x55ddd6db02d0 2023-07-18T03:19:33.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.539+0000 7fa792ffd700 1 -- 172.21.15.101:0/1463670884 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fa7a4028350 con 0x55ddd6db02d0 2023-07-18T03:19:33.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.539+0000 7fa7cd580b80 1 -- 172.21.15.101:0/1463670884 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa79c09dc20 con 0x55ddd6db02d0 2023-07-18T03:19:33.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.539+0000 7fa792ffd700 1 -- 172.21.15.101:0/1463670884 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7a4028550 con 0x55ddd6db02d0 2023-07-18T03:19:33.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.539+0000 7fa7cd580b80 10 client.17940 fetch_fsmap learned FSMap version 33 2023-07-18T03:19:33.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.539+0000 7fa7cd580b80 1 -- 172.21.15.101:0/1463670884 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55ddd6b2f2b0 con 0x55ddd6db02d0 2023-07-18T03:19:33.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.539+0000 7fa792ffd700 1 -- 172.21.15.101:0/1463670884 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa79c09dc20 con 0x55ddd6db02d0 2023-07-18T03:19:33.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.540+0000 7fa792ffd700 1 -- 172.21.15.101:0/1463670884 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa7a400f460 con 0x55ddd6db02d0 2023-07-18T03:19:33.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.540+0000 7fa7cd580b80 10 client.17940 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:19:33.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.540+0000 7fa7cd580b80 10 client.17940 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:19:33.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.540+0000 7fa7cd580b80 20 client.17940 populate_metadata read hostname 'smithi101' 2023-07-18T03:19:33.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.540+0000 7fa7cd580b80 1 --2- 172.21.15.101:0/1463670884 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55ddd6e20810 0x55ddd6e22c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:33.544 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.540+0000 7fa7cd580b80 4 client.17940 mds_command: new command op to 24541 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:19:33.544 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.540+0000 7fa7cd580b80 1 -- 172.21.15.101:0/1463670884 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ddd6cbe050 con 0x55ddd6e20810 2023-07-18T03:19:33.544 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.540+0000 7fa7abfff700 1 --2- 172.21.15.101:0/1463670884 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55ddd6e20810 0x55ddd6e22c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:33.544 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.540+0000 7fa7abfff700 1 --2- 172.21.15.101:0/1463670884 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55ddd6e20810 0x55ddd6e22c20 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b9112340c4013ed0 server_cookie=1fd46378e24c0e47 in_seq=0 out_seq=0 2023-07-18T03:19:33.544 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.540+0000 7fa792ffd700 10 client.17940 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:19:33.544 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.541+0000 7fa792ffd700 1 -- 172.21.15.101:0/1463670884 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ddd6cbe050 con 0x55ddd6e20810 2023-07-18T03:19:33.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.541+0000 7fa792ffd700 10 client.17940 handle_command_reply: tid=0 2023-07-18T03:19:33.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.541+0000 7fa7cd580b80 1 client.17940 shutdown 2023-07-18T03:19:33.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.541+0000 7fa7cd580b80 1 -- 172.21.15.101:0/1463670884 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ddd6db02d0 msgr2=0x55ddd6dc2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:33.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.541+0000 7fa7cd580b80 1 --2- 172.21.15.101:0/1463670884 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ddd6db02d0 0x55ddd6dc2400 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4005f50 tx=0x7fa7a400bf70 comp rx=0 tx=0).stop 2023-07-18T03:19:33.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.541+0000 7fa7cd580b80 1 -- 172.21.15.101:0/1463670884 shutdown_connections 2023-07-18T03:19:33.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.541+0000 7fa7cd580b80 1 --2- 172.21.15.101:0/1463670884 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55ddd6e20810 0x55ddd6e22c20 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.541+0000 7fa7cd580b80 1 --2- 172.21.15.101:0/1463670884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ddd6dbbad0 0x55ddd6dc0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.541+0000 7fa7cd580b80 1 --2- 172.21.15.101:0/1463670884 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ddd6db02d0 0x55ddd6dc2400 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.541+0000 7fa792ffd700 0 client.17940 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:19:33.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.541+0000 7fa7cd580b80 1 --2- 172.21.15.101:0/1463670884 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ddd6db0be0 0x55ddd6dc06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.541+0000 7fa7cd580b80 1 -- 172.21.15.101:0/1463670884 >> 172.21.15.101:0/1463670884 conn(0x55ddd6da6da0 msgr2=0x55ddd6db3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:33.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.541+0000 7fa7cd580b80 1 -- 172.21.15.101:0/1463670884 shutdown_connections 2023-07-18T03:19:33.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.541+0000 7fa7cd580b80 1 -- 172.21.15.101:0/1463670884 wait complete. 2023-07-18T03:19:33.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.541+0000 7fa7cd580b80 20 client.17940 trim_cache size 0 max 16384 2023-07-18T03:19:33.547 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:19:33.547 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.542+0000 7fa7b9bff700 1 -- 172.21.15.101:0/4239960037 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa7a00a1340 msgr2=0x7fa7a00a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:33.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.542+0000 7fa7b9bff700 1 --2- 172.21.15.101:0/4239960037 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa7a00a1340 0x7fa7a00a37f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac009fd0 tx=0x7fa7ac009680 comp rx=0 tx=0).stop 2023-07-18T03:19:33.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.542+0000 7fa7b9bff700 1 -- 172.21.15.101:0/4239960037 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa7b4100480 msgr2=0x7fa7b4198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:33.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.542+0000 7fa7b9bff700 1 --2- 172.21.15.101:0/4239960037 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa7b4100480 0x7fa7b4198330 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fa79c00d950 tx=0x7fa79c00dc60 comp rx=0 tx=0).stop 2023-07-18T03:19:33.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.542+0000 7fa7b9bff700 1 -- 172.21.15.101:0/4239960037 shutdown_connections 2023-07-18T03:19:33.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.542+0000 7fa7b9bff700 1 --2- 172.21.15.101:0/4239960037 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa7a00a1340 0x7fa7a00a37f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.542+0000 7fa7b9bff700 1 --2- 172.21.15.101:0/4239960037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7b40ffad0 0x7fa7b4197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.542+0000 7fa7b9bff700 1 --2- 172.21.15.101:0/4239960037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7b4105ae0 0x7fa7b419c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.549 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.542+0000 7fa7b9bff700 1 --2- 172.21.15.101:0/4239960037 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa7b4100480 0x7fa7b4198330 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.549 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.542+0000 7fa7b9bff700 1 -- 172.21.15.101:0/4239960037 >> 172.21.15.101:0/4239960037 conn(0x7fa7b40fb5f0 msgr2=0x7fa7b410adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:33.549 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.542+0000 7fa7b9bff700 1 -- 172.21.15.101:0/4239960037 shutdown_connections 2023-07-18T03:19:33.549 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.542+0000 7fa7b9bff700 1 -- 172.21.15.101:0/4239960037 wait complete. 2023-07-18T03:19:33.554 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-18T03:19:33.626 INFO:tasks.workunit.client.0.smithi101.stdout: 1 719005 5.42 MB/sec execute 538 sec latency 1572.683 ms 2023-07-18T03:19:33.778 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.783+0000 7f12481ff700 1 -- 172.21.15.101:0/2787127272 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1240105af0 msgr2=0x7f1240105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:33.779 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.783+0000 7f12481ff700 1 --2- 172.21.15.101:0/2787127272 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1240105af0 0x7f1240105ec0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f123c009990 tx=0x7f123c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.779 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.784+0000 7f12481ff700 1 -- 172.21.15.101:0/2787127272 shutdown_connections 2023-07-18T03:19:33.779 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.784+0000 7f12481ff700 1 --2- 172.21.15.101:0/2787127272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1240100490 0x7f124010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.780 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.784+0000 7f12481ff700 1 --2- 172.21.15.101:0/2787127272 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1240105af0 0x7f1240105ec0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.780 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.784+0000 7f12481ff700 1 --2- 172.21.15.101:0/2787127272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12400ffae0 0x7f12400fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.780 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.784+0000 7f12481ff700 1 -- 172.21.15.101:0/2787127272 >> 172.21.15.101:0/2787127272 conn(0x7f12400fb5e0 msgr2=0x7f12400fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:33.780 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.784+0000 7f12481ff700 1 -- 172.21.15.101:0/2787127272 shutdown_connections 2023-07-18T03:19:33.780 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.784+0000 7f12481ff700 1 -- 172.21.15.101:0/2787127272 wait complete. 2023-07-18T03:19:33.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.785+0000 7f12481ff700 1 Processor -- start 2023-07-18T03:19:33.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.785+0000 7f12481ff700 1 -- start start 2023-07-18T03:19:33.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.785+0000 7f12481ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12400ffae0 0x7f1240197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:33.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.785+0000 7f12481ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1240100490 0x7f1240198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:33.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.785+0000 7f12481ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1240105af0 0x7f124019c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:33.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.785+0000 7f12481ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124019cdd0 con 0x7f1240100490 2023-07-18T03:19:33.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.785+0000 7f12481ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124019cf40 con 0x7f1240105af0 2023-07-18T03:19:33.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.785+0000 7f12481ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124019d100 con 0x7f12400ffae0 2023-07-18T03:19:33.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.785+0000 7f1245f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12400ffae0 0x7f1240197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:33.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.785+0000 7f1245f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12400ffae0 0x7f1240197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:44770/0 (socket says 172.21.15.101:44770) 2023-07-18T03:19:33.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.785+0000 7f1245f9b700 1 -- 172.21.15.101:0/3914759867 learned_addr learned my addr 172.21.15.101:0/3914759867 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:33.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.785+0000 7f124579a700 1 --2- 172.21.15.101:0/3914759867 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1240100490 0x7f1240198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:33.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.785+0000 7f124679c700 1 --2- 172.21.15.101:0/3914759867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1240105af0 0x7f124019c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:33.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.786+0000 7f1245f9b700 1 -- 172.21.15.101:0/3914759867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1240105af0 msgr2=0x7f124019c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:33.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.786+0000 7f1245f9b700 1 --2- 172.21.15.101:0/3914759867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1240105af0 0x7f124019c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.786+0000 7f1245f9b700 1 -- 172.21.15.101:0/3914759867 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1240100490 msgr2=0x7f1240198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:33.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.786+0000 7f1245f9b700 1 --2- 172.21.15.101:0/3914759867 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1240100490 0x7f1240198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.786+0000 7f1245f9b700 1 -- 172.21.15.101:0/3914759867 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f123c0095f0 con 0x7f12400ffae0 2023-07-18T03:19:33.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.786+0000 7f1245f9b700 1 --2- 172.21.15.101:0/3914759867 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12400ffae0 0x7f1240197df0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f123c005910 tx=0x7f123c005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:33.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.786+0000 7f1236ffd700 1 -- 172.21.15.101:0/3914759867 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123c01d070 con 0x7f12400ffae0 2023-07-18T03:19:33.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.786+0000 7f12481ff700 1 -- 172.21.15.101:0/3914759867 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f124019d380 con 0x7f12400ffae0 2023-07-18T03:19:33.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.786+0000 7f1236ffd700 1 -- 172.21.15.101:0/3914759867 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f123c022470 con 0x7f12400ffae0 2023-07-18T03:19:33.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.786+0000 7f1236ffd700 1 -- 172.21.15.101:0/3914759867 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123c017650 con 0x7f12400ffae0 2023-07-18T03:19:33.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.787+0000 7f12481ff700 1 -- 172.21.15.101:0/3914759867 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12401a4c20 con 0x7f12400ffae0 2023-07-18T03:19:33.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.791+0000 7f1236ffd700 1 -- 172.21.15.101:0/3914759867 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f123c00fad0 con 0x7f12400ffae0 2023-07-18T03:19:33.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.792+0000 7f125ba6eb80 1 Processor -- start 2023-07-18T03:19:33.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.792+0000 7f125ba6eb80 1 -- start start 2023-07-18T03:19:33.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.793+0000 7f125ba6eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa399f12d0 0x55aa399f16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:33.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.793+0000 7f125ba6eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aa399f1be0 0x55aa399fc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:33.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.793+0000 7f125ba6eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aa399fcad0 0x55aa399feeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:33.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.793+0000 7f125ba6eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa398f7cb0 con 0x55aa399fcad0 2023-07-18T03:19:33.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.793+0000 7f125ba6eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3994aa40 con 0x55aa399f12d0 2023-07-18T03:19:33.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.793+0000 7f125ba6eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa39943ed0 con 0x55aa399f1be0 2023-07-18T03:19:33.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.793+0000 7f124579a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aa399fcad0 0x55aa399feeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:33.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.793+0000 7f124579a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aa399fcad0 0x55aa399feeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:37912/0 (socket says 172.21.15.101:37912) 2023-07-18T03:19:33.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.793+0000 7f124579a700 1 -- 172.21.15.101:0/3422707648 learned_addr learned my addr 172.21.15.101:0/3422707648 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:33.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.793+0000 7f1245f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aa399f1be0 0x55aa399fc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:33.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.793+0000 7f124679c700 1 --2- 172.21.15.101:0/3422707648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa399f12d0 0x55aa399f16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:33.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.793+0000 7f124579a700 1 -- 172.21.15.101:0/3422707648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aa399f1be0 msgr2=0x55aa399fc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:33.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.793+0000 7f124579a700 1 --2- 172.21.15.101:0/3422707648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aa399f1be0 0x55aa399fc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.793+0000 7f124579a700 1 -- 172.21.15.101:0/3422707648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa399f12d0 msgr2=0x55aa399f16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:33.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.793+0000 7f124579a700 1 --2- 172.21.15.101:0/3422707648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa399f12d0 0x55aa399f16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.793+0000 7f124579a700 1 -- 172.21.15.101:0/3422707648 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa39771de0 con 0x55aa399fcad0 2023-07-18T03:19:33.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.794+0000 7f1236ffd700 1 --2- 172.21.15.101:0/3914759867 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f122c0a1340 0x7f122c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:33.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.794+0000 7f124579a700 1 --2- 172.21.15.101:0/3422707648 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aa399fcad0 0x55aa399feeb0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f123000bd30 tx=0x7f12300106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5d80b47fe3e210 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:33.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.794+0000 7f1236ffd700 1 -- 172.21.15.101:0/3914759867 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f123c0ce520 con 0x7f12400ffae0 2023-07-18T03:19:33.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.794+0000 7f124679c700 1 --2- 172.21.15.101:0/3914759867 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f122c0a1340 0x7f122c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:33.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.794+0000 7f122bfff700 1 -- 172.21.15.101:0/3422707648 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1230010db0 con 0x55aa399fcad0 2023-07-18T03:19:33.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.794+0000 7f122bfff700 1 -- 172.21.15.101:0/3422707648 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1230015070 con 0x55aa399fcad0 2023-07-18T03:19:33.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.794+0000 7f122bfff700 1 -- 172.21.15.101:0/3422707648 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1230003da0 con 0x55aa399fcad0 2023-07-18T03:19:33.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.795+0000 7f125ba6eb80 1 -- 172.21.15.101:0/3422707648 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aa399fcad0 msgr2=0x55aa399feeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:33.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.795+0000 7f124679c700 1 --2- 172.21.15.101:0/3914759867 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f122c0a1340 0x7f122c0a37f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f12380127f0 tx=0x7f123800b040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:33.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.795+0000 7f125ba6eb80 1 --2- 172.21.15.101:0/3422707648 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aa399fcad0 0x55aa399feeb0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f123000bd30 tx=0x7f12300106b0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.795+0000 7f125ba6eb80 1 -- 172.21.15.101:0/3422707648 shutdown_connections 2023-07-18T03:19:33.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.795+0000 7f125ba6eb80 1 --2- 172.21.15.101:0/3422707648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa399f12d0 0x55aa399f16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.795+0000 7f125ba6eb80 1 --2- 172.21.15.101:0/3422707648 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aa399fcad0 0x55aa399feeb0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.795+0000 7f125ba6eb80 1 --2- 172.21.15.101:0/3422707648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aa399f1be0 0x55aa399fc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.795+0000 7f125ba6eb80 1 -- 172.21.15.101:0/3422707648 >> 172.21.15.101:0/3422707648 conn(0x55aa399e7da0 msgr2=0x55aa399ed6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:33.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.795+0000 7f125ba6eb80 1 -- 172.21.15.101:0/3422707648 shutdown_connections 2023-07-18T03:19:33.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.795+0000 7f125ba6eb80 1 -- 172.21.15.101:0/3422707648 wait complete. 2023-07-18T03:19:33.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.795+0000 7f125ba6eb80 1 -- start start 2023-07-18T03:19:33.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.796+0000 7f125ba6eb80 1 Processor -- start 2023-07-18T03:19:33.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.796+0000 7f125ba6eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa399f12d0 0x55aa399fc530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:33.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.796+0000 7f125ba6eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aa399f1be0 0x55aa399f9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:33.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.796+0000 7f125ba6eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aa399fcad0 0x55aa399f9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:33.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.796+0000 7f125ba6eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa398e3ac0 con 0x55aa399fcad0 2023-07-18T03:19:33.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.796+0000 7f125ba6eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa398e37c0 con 0x55aa399f12d0 2023-07-18T03:19:33.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.796+0000 7f125ba6eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa397c7920 con 0x55aa399f1be0 2023-07-18T03:19:33.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.796+0000 7f125ba6eb80 10 client.0 _ll_register_callbacks cb 0x55aa398e1760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:19:33.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.796+0000 7f1245f9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa399f12d0 0x55aa399fc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:33.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.796+0000 7f1245f9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa399f12d0 0x55aa399fc530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:55886/0 (socket says 172.21.15.101:55886) 2023-07-18T03:19:33.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.796+0000 7f1245f9b700 1 -- 172.21.15.101:0/1888987636 learned_addr learned my addr 172.21.15.101:0/1888987636 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:33.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.796+0000 7f124679c700 1 --2- 172.21.15.101:0/1888987636 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aa399fcad0 0x55aa399f9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:33.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.797+0000 7f124579a700 1 --2- 172.21.15.101:0/1888987636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aa399f1be0 0x55aa399f9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:33.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.797+0000 7f1245f9b700 1 -- 172.21.15.101:0/1888987636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aa399f1be0 msgr2=0x55aa399f9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:33.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.797+0000 7f1245f9b700 1 --2- 172.21.15.101:0/1888987636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aa399f1be0 0x55aa399f9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.797+0000 7f1245f9b700 1 -- 172.21.15.101:0/1888987636 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aa399fcad0 msgr2=0x55aa399f9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:33.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.797+0000 7f1245f9b700 1 --2- 172.21.15.101:0/1888987636 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aa399fcad0 0x55aa399f9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.797+0000 7f1245f9b700 1 -- 172.21.15.101:0/1888987636 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f123000b960 con 0x55aa399f12d0 2023-07-18T03:19:33.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.797+0000 7f124579a700 1 --2- 172.21.15.101:0/1888987636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aa399f1be0 0x55aa399f9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:19:33.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.797+0000 7f124679c700 1 --2- 172.21.15.101:0/1888987636 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aa399fcad0 0x55aa399f9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:19:33.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.797+0000 7f1245f9b700 1 --2- 172.21.15.101:0/1888987636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa399f12d0 0x55aa399fc530 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f123c097ce0 tx=0x7f123c0cbdc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc9201965907db55 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:33.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.797+0000 7f12297fa700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:19:33.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.797+0000 7f12297fa700 1 -- 172.21.15.101:0/1888987636 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123c09b040 con 0x55aa399f12d0 2023-07-18T03:19:33.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.797+0000 7f125ba6eb80 1 -- 172.21.15.101:0/1888987636 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aa3989d4f0 con 0x55aa399f12d0 2023-07-18T03:19:33.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.797+0000 7f12297fa700 1 -- 172.21.15.101:0/1888987636 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f123c09b270 con 0x55aa399f12d0 2023-07-18T03:19:33.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.797+0000 7f12297fa700 1 -- 172.21.15.101:0/1888987636 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f123c09adb0 con 0x55aa399f12d0 2023-07-18T03:19:33.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.798+0000 7f125ba6eb80 10 client.27998 fetch_fsmap learned FSMap version 33 2023-07-18T03:19:33.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.798+0000 7f125ba6eb80 1 -- 172.21.15.101:0/1888987636 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55aa3975ac60 con 0x55aa399f12d0 2023-07-18T03:19:33.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.798+0000 7f12297fa700 1 -- 172.21.15.101:0/1888987636 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55aa3989d4f0 con 0x55aa399f12d0 2023-07-18T03:19:33.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.798+0000 7f12297fa700 1 -- 172.21.15.101:0/1888987636 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f123c0ce150 con 0x55aa399f12d0 2023-07-18T03:19:33.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.798+0000 7f125ba6eb80 10 client.27998 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:19:33.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.798+0000 7f125ba6eb80 10 client.27998 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:19:33.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.798+0000 7f125ba6eb80 20 client.27998 populate_metadata read hostname 'smithi101' 2023-07-18T03:19:33.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.798+0000 7f125ba6eb80 1 --2- 172.21.15.101:0/1888987636 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55aa39a0bfe0 0x55aa39a0e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:33.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.798+0000 7f125ba6eb80 4 client.27998 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:19:33.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.798+0000 7f125ba6eb80 1 -- 172.21.15.101:0/1888987636 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aa3993ceb0 con 0x55aa39a0bfe0 2023-07-18T03:19:33.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.798+0000 7f124579a700 1 --2- 172.21.15.101:0/1888987636 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55aa39a0bfe0 0x55aa39a0e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:33.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.799+0000 7f124579a700 1 --2- 172.21.15.101:0/1888987636 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55aa39a0bfe0 0x55aa39a0e3c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=abf973035bf8c71b server_cookie=8f2f86d135bc49d4 in_seq=0 out_seq=0 2023-07-18T03:19:33.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.799+0000 7f12297fa700 10 client.27998 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:19:33.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.799+0000 7f12297fa700 1 -- 172.21.15.101:0/1888987636 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55aa3993ceb0 con 0x55aa39a0bfe0 2023-07-18T03:19:33.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.799+0000 7f12297fa700 10 client.27998 handle_command_reply: tid=0 2023-07-18T03:19:33.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.800+0000 7f125ba6eb80 1 client.27998 shutdown 2023-07-18T03:19:33.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.800+0000 7f125ba6eb80 1 -- 172.21.15.101:0/1888987636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa399f12d0 msgr2=0x55aa399fc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:33.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.800+0000 7f125ba6eb80 1 --2- 172.21.15.101:0/1888987636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa399f12d0 0x55aa399fc530 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f123c097ce0 tx=0x7f123c0cbdc0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.800+0000 7f125ba6eb80 1 -- 172.21.15.101:0/1888987636 shutdown_connections 2023-07-18T03:19:33.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.800+0000 7f125ba6eb80 1 --2- 172.21.15.101:0/1888987636 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55aa39a0bfe0 0x55aa39a0e3c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.800+0000 7f125ba6eb80 1 --2- 172.21.15.101:0/1888987636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa399f12d0 0x55aa399fc530 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.800+0000 7f125ba6eb80 1 --2- 172.21.15.101:0/1888987636 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aa399fcad0 0x55aa399f9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.800+0000 7f125ba6eb80 1 --2- 172.21.15.101:0/1888987636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aa399f1be0 0x55aa399f9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.800+0000 7f12297fa700 0 client.27998 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:19:33.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.800+0000 7f125ba6eb80 1 -- 172.21.15.101:0/1888987636 >> 172.21.15.101:0/1888987636 conn(0x55aa399e7da0 msgr2=0x55aa399ed290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:33.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.800+0000 7f125ba6eb80 1 -- 172.21.15.101:0/1888987636 shutdown_connections 2023-07-18T03:19:33.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.800+0000 7f125ba6eb80 1 -- 172.21.15.101:0/1888987636 wait complete. 2023-07-18T03:19:33.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.800+0000 7f125ba6eb80 20 client.27998 trim_cache size 0 max 16384 2023-07-18T03:19:33.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.813+0000 7f125ba6eb80 1 Processor -- start 2023-07-18T03:19:33.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.814+0000 7f125ba6eb80 1 -- start start 2023-07-18T03:19:33.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.814+0000 7f125ba6eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aa399e7da0 0x55aa39a034f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:33.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.814+0000 7f125ba6eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa399f12d0 0x55aa399fc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:33.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.814+0000 7f125ba6eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aa399f1be0 0x55aa399f4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:33.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.814+0000 7f125ba6eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa399fad20 con 0x55aa399f1be0 2023-07-18T03:19:33.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.814+0000 7f125ba6eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa399f31a0 con 0x55aa399f12d0 2023-07-18T03:19:33.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.814+0000 7f125ba6eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa39a083a0 con 0x55aa399e7da0 2023-07-18T03:19:33.813 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.814+0000 7f124579a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa399f12d0 0x55aa399fc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:33.814 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.814+0000 7f124679c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aa399f1be0 0x55aa399f4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:33.814 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.814+0000 7f124579a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa399f12d0 0x55aa399fc060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:55898/0 (socket says 172.21.15.101:55898) 2023-07-18T03:19:33.814 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.814+0000 7f124579a700 1 -- 172.21.15.101:0/3573112520 learned_addr learned my addr 172.21.15.101:0/3573112520 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:33.814 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.814+0000 7f124679c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aa399f1be0 0x55aa399f4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:37932/0 (socket says 172.21.15.101:37932) 2023-07-18T03:19:33.814 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.814+0000 7f1245f9b700 1 --2- 172.21.15.101:0/3573112520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aa399e7da0 0x55aa39a034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:33.814 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.814+0000 7f124579a700 1 -- 172.21.15.101:0/3573112520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aa399e7da0 msgr2=0x55aa39a034f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:33.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.814+0000 7f124579a700 1 --2- 172.21.15.101:0/3573112520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aa399e7da0 0x55aa39a034f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.815+0000 7f124579a700 1 -- 172.21.15.101:0/3573112520 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aa399f1be0 msgr2=0x55aa399f4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:33.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.815+0000 7f124579a700 1 --2- 172.21.15.101:0/3573112520 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aa399f1be0 0x55aa399f4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.815+0000 7f124579a700 1 -- 172.21.15.101:0/3573112520 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa39798250 con 0x55aa399f12d0 2023-07-18T03:19:33.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.815+0000 7f124579a700 1 --2- 172.21.15.101:0/3573112520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa399f12d0 0x55aa399fc060 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f12300059d0 tx=0x7f123000b710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30ab07a7965ba223 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:33.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.815+0000 7f1234ff9700 1 -- 172.21.15.101:0/3573112520 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1230005ee0 con 0x55aa399f12d0 2023-07-18T03:19:33.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.815+0000 7f1234ff9700 1 -- 172.21.15.101:0/3573112520 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1230020050 con 0x55aa399f12d0 2023-07-18T03:19:33.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.815+0000 7f1234ff9700 1 -- 172.21.15.101:0/3573112520 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1230027070 con 0x55aa399f12d0 2023-07-18T03:19:33.816 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.816+0000 7f125ba6eb80 1 -- 172.21.15.101:0/3573112520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa399f12d0 msgr2=0x55aa399fc060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:33.816 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.816+0000 7f125ba6eb80 1 --2- 172.21.15.101:0/3573112520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa399f12d0 0x55aa399fc060 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f12300059d0 tx=0x7f123000b710 comp rx=0 tx=0).stop 2023-07-18T03:19:33.816 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.816+0000 7f125ba6eb80 1 -- 172.21.15.101:0/3573112520 shutdown_connections 2023-07-18T03:19:33.816 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.816+0000 7f125ba6eb80 1 --2- 172.21.15.101:0/3573112520 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aa399f1be0 0x55aa399f4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.816 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.816+0000 7f125ba6eb80 1 --2- 172.21.15.101:0/3573112520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aa399e7da0 0x55aa39a034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.816+0000 7f125ba6eb80 1 --2- 172.21.15.101:0/3573112520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa399f12d0 0x55aa399fc060 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.816+0000 7f125ba6eb80 1 -- 172.21.15.101:0/3573112520 >> 172.21.15.101:0/3573112520 conn(0x55aa399f9ff0 msgr2=0x55aa39a07c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:33.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.816+0000 7f125ba6eb80 1 -- 172.21.15.101:0/3573112520 shutdown_connections 2023-07-18T03:19:33.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.816+0000 7f125ba6eb80 1 -- 172.21.15.101:0/3573112520 wait complete. 2023-07-18T03:19:33.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.816+0000 7f125ba6eb80 1 -- start start 2023-07-18T03:19:33.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.816+0000 7f125ba6eb80 1 Processor -- start 2023-07-18T03:19:33.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.817+0000 7f125ba6eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aa399e7da0 0x55aa39a016e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:33.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.817+0000 7f125ba6eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aa399f12d0 0x55aa39a01c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:33.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.817+0000 7f125ba6eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa399f1be0 0x55aa39a58e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:33.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.817+0000 7f125ba6eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa39a593a0 con 0x55aa399e7da0 2023-07-18T03:19:33.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.817+0000 7f125ba6eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa39a59510 con 0x55aa399f1be0 2023-07-18T03:19:33.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.817+0000 7f125ba6eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa39a596d0 con 0x55aa399f12d0 2023-07-18T03:19:33.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.817+0000 7f125ba6eb80 10 client.0 _ll_register_callbacks cb 0x55aa398e1760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:19:33.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.817+0000 7f1245f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aa399e7da0 0x55aa39a016e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:33.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.817+0000 7f1245f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aa399e7da0 0x55aa39a016e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:37938/0 (socket says 172.21.15.101:37938) 2023-07-18T03:19:33.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.817+0000 7f1245f9b700 1 -- 172.21.15.101:0/1865872377 learned_addr learned my addr 172.21.15.101:0/1865872377 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:33.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.817+0000 7f124679c700 1 --2- 172.21.15.101:0/1865872377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa399f1be0 0x55aa39a58e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:33.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.817+0000 7f124579a700 1 --2- 172.21.15.101:0/1865872377 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aa399f12d0 0x55aa39a01c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:33.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.817+0000 7f1245f9b700 1 -- 172.21.15.101:0/1865872377 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aa399f12d0 msgr2=0x55aa39a01c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:33.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.817+0000 7f1245f9b700 1 --2- 172.21.15.101:0/1865872377 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aa399f12d0 0x55aa39a01c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.817+0000 7f1245f9b700 1 -- 172.21.15.101:0/1865872377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa399f1be0 msgr2=0x55aa39a58e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:33.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.817+0000 7f1245f9b700 1 --2- 172.21.15.101:0/1865872377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa399f1be0 0x55aa39a58e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.817+0000 7f1245f9b700 1 -- 172.21.15.101:0/1865872377 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aa39771de0 con 0x55aa399e7da0 2023-07-18T03:19:33.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.818+0000 7f1245f9b700 1 --2- 172.21.15.101:0/1865872377 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aa399e7da0 0x55aa39a016e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f123c0a12c0 tx=0x7f123c0d1d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=671339fca23caa76 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:33.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.818+0000 7f12297fa700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:19:33.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.818+0000 7f12297fa700 1 -- 172.21.15.101:0/1865872377 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123c0a33a0 con 0x55aa399e7da0 2023-07-18T03:19:33.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.818+0000 7f125ba6eb80 1 -- 172.21.15.101:0/1865872377 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1230015070 con 0x55aa399e7da0 2023-07-18T03:19:33.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.818+0000 7f12297fa700 1 -- 172.21.15.101:0/1865872377 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f123c09f2a0 con 0x55aa399e7da0 2023-07-18T03:19:33.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.818+0000 7f12297fa700 1 -- 172.21.15.101:0/1865872377 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f123c09b040 con 0x55aa399e7da0 2023-07-18T03:19:33.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.818+0000 7f125ba6eb80 10 client.17958 fetch_fsmap learned FSMap version 33 2023-07-18T03:19:33.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.818+0000 7f125ba6eb80 1 -- 172.21.15.101:0/1865872377 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55aa397702b0 con 0x55aa399e7da0 2023-07-18T03:19:33.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.818+0000 7f12297fa700 1 -- 172.21.15.101:0/1865872377 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1230015070 con 0x55aa399e7da0 2023-07-18T03:19:33.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.818+0000 7f12297fa700 1 -- 172.21.15.101:0/1865872377 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f123c09a880 con 0x55aa399e7da0 2023-07-18T03:19:33.822 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.818+0000 7f125ba6eb80 10 client.17958 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:19:33.822 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.818+0000 7f125ba6eb80 10 client.17958 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:19:33.822 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.818+0000 7f125ba6eb80 20 client.17958 populate_metadata read hostname 'smithi101' 2023-07-18T03:19:33.822 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.818+0000 7f125ba6eb80 1 --2- 172.21.15.101:0/1865872377 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55aa399fcad0 0x55aa39a62a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:33.822 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.818+0000 7f125ba6eb80 4 client.17958 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:19:33.822 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.818+0000 7f125ba6eb80 1 -- 172.21.15.101:0/1865872377 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55aa398ff050 con 0x55aa399fcad0 2023-07-18T03:19:33.822 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.819+0000 7f124579a700 1 --2- 172.21.15.101:0/1865872377 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55aa399fcad0 0x55aa39a62a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:33.823 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.819+0000 7f12297fa700 10 client.17958 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:19:33.824 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.819+0000 7f124579a700 1 --2- 172.21.15.101:0/1865872377 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55aa399fcad0 0x55aa39a62a70 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=38584fcefa5ebbee server_cookie=8bf8ac89b62e975d in_seq=0 out_seq=0 2023-07-18T03:19:33.824 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.820+0000 7f12297fa700 1 -- 172.21.15.101:0/1865872377 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55aa398ff050 con 0x55aa399fcad0 2023-07-18T03:19:33.824 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.820+0000 7f12297fa700 10 client.17958 handle_command_reply: tid=0 2023-07-18T03:19:33.824 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.820+0000 7f125ba6eb80 1 client.17958 shutdown 2023-07-18T03:19:33.824 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.820+0000 7f125ba6eb80 1 -- 172.21.15.101:0/1865872377 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aa399e7da0 msgr2=0x55aa39a016e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:33.824 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.820+0000 7f125ba6eb80 1 --2- 172.21.15.101:0/1865872377 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aa399e7da0 0x55aa39a016e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f123c0a12c0 tx=0x7f123c0d1d30 comp rx=0 tx=0).stop 2023-07-18T03:19:33.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.820+0000 7f125ba6eb80 1 -- 172.21.15.101:0/1865872377 shutdown_connections 2023-07-18T03:19:33.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.820+0000 7f125ba6eb80 1 --2- 172.21.15.101:0/1865872377 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55aa399fcad0 0x55aa39a62a70 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.820+0000 7f125ba6eb80 1 --2- 172.21.15.101:0/1865872377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa399f1be0 0x55aa39a58e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.820+0000 7f125ba6eb80 1 --2- 172.21.15.101:0/1865872377 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aa399e7da0 0x55aa39a016e0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.820+0000 7f12297fa700 0 client.17958 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:19:33.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.820+0000 7f125ba6eb80 1 --2- 172.21.15.101:0/1865872377 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aa399f12d0 0x55aa39a01c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.820+0000 7f125ba6eb80 1 -- 172.21.15.101:0/1865872377 >> 172.21.15.101:0/1865872377 conn(0x55aa399f9ff0 msgr2=0x55aa399f4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:33.826 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.820+0000 7f125ba6eb80 1 -- 172.21.15.101:0/1865872377 shutdown_connections 2023-07-18T03:19:33.826 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.820+0000 7f125ba6eb80 1 -- 172.21.15.101:0/1865872377 wait complete. 2023-07-18T03:19:33.826 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.820+0000 7f125ba6eb80 20 client.17958 trim_cache size 0 max 16384 2023-07-18T03:19:33.826 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:19:33.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.821+0000 7f12481ff700 1 -- 172.21.15.101:0/3914759867 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f122c0a1340 msgr2=0x7f122c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:33.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.821+0000 7f12481ff700 1 --2- 172.21.15.101:0/3914759867 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f122c0a1340 0x7f122c0a37f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f12380127f0 tx=0x7f123800b040 comp rx=0 tx=0).stop 2023-07-18T03:19:33.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.821+0000 7f12481ff700 1 -- 172.21.15.101:0/3914759867 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12400ffae0 msgr2=0x7f1240197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:33.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.821+0000 7f12481ff700 1 --2- 172.21.15.101:0/3914759867 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12400ffae0 0x7f1240197df0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f123c005910 tx=0x7f123c005740 comp rx=0 tx=0).stop 2023-07-18T03:19:33.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.821+0000 7f12481ff700 1 -- 172.21.15.101:0/3914759867 shutdown_connections 2023-07-18T03:19:33.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.821+0000 7f12481ff700 1 --2- 172.21.15.101:0/3914759867 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f122c0a1340 0x7f122c0a37f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.821+0000 7f12481ff700 1 --2- 172.21.15.101:0/3914759867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1240105af0 0x7f124019c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.821+0000 7f12481ff700 1 --2- 172.21.15.101:0/3914759867 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12400ffae0 0x7f1240197df0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.821+0000 7f12481ff700 1 --2- 172.21.15.101:0/3914759867 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1240100490 0x7f1240198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:33.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.821+0000 7f12481ff700 1 -- 172.21.15.101:0/3914759867 >> 172.21.15.101:0/3914759867 conn(0x7f12400fb5e0 msgr2=0x7f124010aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:33.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.821+0000 7f12481ff700 1 -- 172.21.15.101:0/3914759867 shutdown_connections 2023-07-18T03:19:33.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:33.821+0000 7f12481ff700 1 -- 172.21.15.101:0/3914759867 wait complete. 2023-07-18T03:19:33.833 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-18T03:19:34.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.040+0000 7f925cdff700 1 -- 172.21.15.101:0/1761603860 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9258100490 msgr2=0x7f925810c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:34.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.040+0000 7f925cdff700 1 --2- 172.21.15.101:0/1761603860 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9258100490 0x7f925810c920 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f925000b0a0 tx=0x7f925000b3b0 comp rx=0 tx=0).stop 2023-07-18T03:19:34.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.040+0000 7f925cdff700 1 -- 172.21.15.101:0/1761603860 shutdown_connections 2023-07-18T03:19:34.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.040+0000 7f925cdff700 1 --2- 172.21.15.101:0/1761603860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9258105af0 0x7f9258105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:34.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.040+0000 7f925cdff700 1 --2- 172.21.15.101:0/1761603860 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9258100490 0x7f925810c920 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:34.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.040+0000 7f925cdff700 1 --2- 172.21.15.101:0/1761603860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92580ffae0 0x7f92580fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:34.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.040+0000 7f925cdff700 1 -- 172.21.15.101:0/1761603860 >> 172.21.15.101:0/1761603860 conn(0x7f92580fb5e0 msgr2=0x7f92580fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:34.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.040+0000 7f925cdff700 1 -- 172.21.15.101:0/1761603860 shutdown_connections 2023-07-18T03:19:34.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.040+0000 7f925cdff700 1 -- 172.21.15.101:0/1761603860 wait complete. 2023-07-18T03:19:34.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.041+0000 7f925cdff700 1 Processor -- start 2023-07-18T03:19:34.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.041+0000 7f925cdff700 1 -- start start 2023-07-18T03:19:34.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.042+0000 7f925cdff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92580ffae0 0x7f9258197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:34.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.042+0000 7f925cdff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9258100490 0x7f9258198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:34.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.042+0000 7f925cdff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9258105af0 0x7f925819c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:34.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.042+0000 7f925cdff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925819cea0 con 0x7f9258100490 2023-07-18T03:19:34.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.042+0000 7f925cdff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925819d010 con 0x7f9258105af0 2023-07-18T03:19:34.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.042+0000 7f925cdff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925819d1d0 con 0x7f92580ffae0 2023-07-18T03:19:34.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.042+0000 7f9255d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9258100490 0x7f9258198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:34.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.042+0000 7f925659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92580ffae0 0x7f9258197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:34.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.042+0000 7f9255d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9258100490 0x7f9258198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:37956/0 (socket says 172.21.15.101:37956) 2023-07-18T03:19:34.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.042+0000 7f925659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92580ffae0 0x7f9258197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:44832/0 (socket says 172.21.15.101:44832) 2023-07-18T03:19:34.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.042+0000 7f9255d9b700 1 -- 172.21.15.101:0/1463901548 learned_addr learned my addr 172.21.15.101:0/1463901548 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:34.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.042+0000 7f9256d9d700 1 --2- 172.21.15.101:0/1463901548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9258105af0 0x7f925819c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:34.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.042+0000 7f9255d9b700 1 -- 172.21.15.101:0/1463901548 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92580ffae0 msgr2=0x7f9258197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:34.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.042+0000 7f9255d9b700 1 --2- 172.21.15.101:0/1463901548 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92580ffae0 0x7f9258197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:34.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.042+0000 7f9255d9b700 1 -- 172.21.15.101:0/1463901548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9258105af0 msgr2=0x7f925819c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:34.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.042+0000 7f9255d9b700 1 --2- 172.21.15.101:0/1463901548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9258105af0 0x7f925819c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:34.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.042+0000 7f9255d9b700 1 -- 172.21.15.101:0/1463901548 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9250009d00 con 0x7f9258100490 2023-07-18T03:19:34.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.042+0000 7f925659c700 1 --2- 172.21.15.101:0/1463901548 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92580ffae0 0x7f9258197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:19:34.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.043+0000 7f9255d9b700 1 --2- 172.21.15.101:0/1463901548 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9258100490 0x7f9258198400 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f924c00e9a0 tx=0x7f924c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:34.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.043+0000 7f92477fe700 1 -- 172.21.15.101:0/1463901548 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924c009790 con 0x7f9258100490 2023-07-18T03:19:34.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.043+0000 7f92477fe700 1 -- 172.21.15.101:0/1463901548 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f924c010430 con 0x7f9258100490 2023-07-18T03:19:34.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.043+0000 7f925cdff700 1 -- 172.21.15.101:0/1463901548 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f925819d4b0 con 0x7f9258100490 2023-07-18T03:19:34.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.043+0000 7f92477fe700 1 -- 172.21.15.101:0/1463901548 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924c018680 con 0x7f9258100490 2023-07-18T03:19:34.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.043+0000 7f925cdff700 1 -- 172.21.15.101:0/1463901548 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92581a4cf0 con 0x7f9258100490 2023-07-18T03:19:34.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.048+0000 7f92477fe700 1 -- 172.21.15.101:0/1463901548 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f924c0105a0 con 0x7f9258100490 2023-07-18T03:19:34.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.048+0000 7f92706d2b80 1 Processor -- start 2023-07-18T03:19:34.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.048+0000 7f92706d2b80 1 -- start start 2023-07-18T03:19:34.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.048+0000 7f92706d2b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55661fe092d0 0x55661fe096a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:34.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.048+0000 7f92706d2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55661fe09be0 0x55661fe14590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:34.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.048+0000 7f92706d2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55661fe14ad0 0x55661fe16eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:34.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.048+0000 7f92706d2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55661fd0fcb0 con 0x55661fe14ad0 2023-07-18T03:19:34.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.048+0000 7f92706d2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55661fd62a40 con 0x55661fe09be0 2023-07-18T03:19:34.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.048+0000 7f92706d2b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55661fd5bed0 con 0x55661fe092d0 2023-07-18T03:19:34.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.049+0000 7f9256d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55661fe092d0 0x55661fe096a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:34.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.049+0000 7f9255d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55661fe14ad0 0x55661fe16eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:34.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.049+0000 7f9256d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55661fe092d0 0x55661fe096a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:44848/0 (socket says 172.21.15.101:44848) 2023-07-18T03:19:34.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.049+0000 7f9256d9d700 1 -- 172.21.15.101:0/258835880 learned_addr learned my addr 172.21.15.101:0/258835880 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:34.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.049+0000 7f925659c700 1 --2- 172.21.15.101:0/258835880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55661fe09be0 0x55661fe14590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:34.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.049+0000 7f9256d9d700 1 -- 172.21.15.101:0/258835880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55661fe09be0 msgr2=0x55661fe14590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:34.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.049+0000 7f9256d9d700 1 --2- 172.21.15.101:0/258835880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55661fe09be0 0x55661fe14590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:34.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.049+0000 7f9256d9d700 1 -- 172.21.15.101:0/258835880 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55661fe14ad0 msgr2=0x55661fe16eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:34.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.049+0000 7f9256d9d700 1 --2- 172.21.15.101:0/258835880 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55661fe14ad0 0x55661fe16eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:34.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.049+0000 7f9256d9d700 1 -- 172.21.15.101:0/258835880 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55661fb89de0 con 0x55661fe092d0 2023-07-18T03:19:34.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.049+0000 7f9256d9d700 1 --2- 172.21.15.101:0/258835880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55661fe092d0 0x55661fe096a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9250009c70 tx=0x7f92500092e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15939e3a46a40432 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:34.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.050+0000 7f92477fe700 1 --2- 172.21.15.101:0/1463901548 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f92400a1340 0x7f92400a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:34.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.050+0000 7f9244ff9700 1 -- 172.21.15.101:0/258835880 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925000e040 con 0x55661fe092d0 2023-07-18T03:19:34.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.050+0000 7f92477fe700 1 -- 172.21.15.101:0/1463901548 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f924c014070 con 0x7f9258100490 2023-07-18T03:19:34.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.050+0000 7f9244ff9700 1 -- 172.21.15.101:0/258835880 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9250003a00 con 0x55661fe092d0 2023-07-18T03:19:34.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.050+0000 7f9244ff9700 1 -- 172.21.15.101:0/258835880 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925001c6a0 con 0x55661fe092d0 2023-07-18T03:19:34.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.050+0000 7f925659c700 1 --2- 172.21.15.101:0/1463901548 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f92400a1340 0x7f92400a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:34.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.050+0000 7f92706d2b80 1 -- 172.21.15.101:0/258835880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55661fe092d0 msgr2=0x55661fe096a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:34.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.050+0000 7f925659c700 1 --2- 172.21.15.101:0/1463901548 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f92400a1340 0x7f92400a37f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f92480125f0 tx=0x7f9248009260 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:34.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.050+0000 7f92706d2b80 1 --2- 172.21.15.101:0/258835880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55661fe092d0 0x55661fe096a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9250009c70 tx=0x7f92500092e0 comp rx=0 tx=0).stop 2023-07-18T03:19:34.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.051+0000 7f92706d2b80 1 -- 172.21.15.101:0/258835880 shutdown_connections 2023-07-18T03:19:34.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.051+0000 7f92706d2b80 1 --2- 172.21.15.101:0/258835880 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55661fe14ad0 0x55661fe16eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:34.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.051+0000 7f92706d2b80 1 --2- 172.21.15.101:0/258835880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55661fe092d0 0x55661fe096a0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:34.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.051+0000 7f92706d2b80 1 --2- 172.21.15.101:0/258835880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55661fe09be0 0x55661fe14590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:34.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.051+0000 7f92706d2b80 1 -- 172.21.15.101:0/258835880 >> 172.21.15.101:0/258835880 conn(0x55661fdffda0 msgr2=0x55661fe056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:34.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.051+0000 7f92706d2b80 1 -- 172.21.15.101:0/258835880 shutdown_connections 2023-07-18T03:19:34.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.051+0000 7f92706d2b80 1 -- 172.21.15.101:0/258835880 wait complete. 2023-07-18T03:19:34.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.051+0000 7f92706d2b80 1 -- start start 2023-07-18T03:19:34.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.051+0000 7f92706d2b80 1 Processor -- start 2023-07-18T03:19:34.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.052+0000 7f92706d2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55661fe092d0 0x55661fe14510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:34.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.052+0000 7f92706d2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55661fe09be0 0x55661fe11790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:34.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.052+0000 7f92706d2b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55661fe14ad0 0x55661fe11ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:34.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.052+0000 7f92706d2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55661fcfbac0 con 0x55661fe09be0 2023-07-18T03:19:34.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.052+0000 7f92706d2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55661fcfb7c0 con 0x55661fe092d0 2023-07-18T03:19:34.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.052+0000 7f92706d2b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55661fbdf920 con 0x55661fe14ad0 2023-07-18T03:19:34.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.052+0000 7f92706d2b80 10 client.0 _ll_register_callbacks cb 0x55661fcf9760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:19:34.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.052+0000 7f9255d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55661fe09be0 0x55661fe11790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:34.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.052+0000 7f9256d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55661fe14ad0 0x55661fe11ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:34.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.052+0000 7f9255d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55661fe09be0 0x55661fe11790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:37972/0 (socket says 172.21.15.101:37972) 2023-07-18T03:19:34.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.052+0000 7f9255d9b700 1 -- 172.21.15.101:0/2081777156 learned_addr learned my addr 172.21.15.101:0/2081777156 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:34.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.052+0000 7f925659c700 1 --2- 172.21.15.101:0/2081777156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55661fe092d0 0x55661fe14510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:34.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.052+0000 7f9255d9b700 1 -- 172.21.15.101:0/2081777156 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55661fe14ad0 msgr2=0x55661fe11ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:34.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.052+0000 7f9255d9b700 1 --2- 172.21.15.101:0/2081777156 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55661fe14ad0 0x55661fe11ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:34.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.052+0000 7f9255d9b700 1 -- 172.21.15.101:0/2081777156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55661fe092d0 msgr2=0x55661fe14510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:34.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.052+0000 7f9255d9b700 1 --2- 172.21.15.101:0/2081777156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55661fe092d0 0x55661fe14510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:34.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.052+0000 7f9255d9b700 1 -- 172.21.15.101:0/2081777156 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f925810d220 con 0x55661fe09be0 2023-07-18T03:19:34.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.053+0000 7f9255d9b700 1 --2- 172.21.15.101:0/2081777156 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55661fe09be0 0x55661fe11790 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f924c00e970 tx=0x7f924c005b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0d8f46383c84171 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:34.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.053+0000 7f923dffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:19:34.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.053+0000 7f923dffb700 1 -- 172.21.15.101:0/2081777156 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924c0cf310 con 0x55661fe09be0 2023-07-18T03:19:34.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.053+0000 7f923dffb700 1 -- 172.21.15.101:0/2081777156 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f924c099f40 con 0x55661fe09be0 2023-07-18T03:19:34.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.053+0000 7f92706d2b80 1 -- 172.21.15.101:0/2081777156 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55661fcb54f0 con 0x55661fe09be0 2023-07-18T03:19:34.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.053+0000 7f923dffb700 1 -- 172.21.15.101:0/2081777156 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f924c0cd120 con 0x55661fe09be0 2023-07-18T03:19:34.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.053+0000 7f92706d2b80 10 client.17970 fetch_fsmap learned FSMap version 33 2023-07-18T03:19:34.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.053+0000 7f92706d2b80 1 -- 172.21.15.101:0/2081777156 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55661fb72c60 con 0x55661fe09be0 2023-07-18T03:19:34.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.053+0000 7f923dffb700 1 -- 172.21.15.101:0/2081777156 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f924c0b1030 con 0x55661fe09be0 2023-07-18T03:19:34.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.053+0000 7f923dffb700 1 -- 172.21.15.101:0/2081777156 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f924c09a1a0 con 0x55661fe09be0 2023-07-18T03:19:34.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.053+0000 7f92706d2b80 10 client.17970 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:19:34.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.054+0000 7f92706d2b80 10 client.17970 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:19:34.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.054+0000 7f92706d2b80 20 client.17970 populate_metadata read hostname 'smithi101' 2023-07-18T03:19:34.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.054+0000 7f92706d2b80 1 --2- 172.21.15.101:0/2081777156 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55661fe24480 0x55661fe26890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:34.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.054+0000 7f92706d2b80 4 client.17970 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:19:34.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.054+0000 7f92706d2b80 1 -- 172.21.15.101:0/2081777156 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55661fd54eb0 con 0x55661fe24480 2023-07-18T03:19:34.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.054+0000 7f925659c700 1 --2- 172.21.15.101:0/2081777156 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55661fe24480 0x55661fe26890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:34.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.054+0000 7f925659c700 1 --2- 172.21.15.101:0/2081777156 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55661fe24480 0x55661fe26890 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=194d45dddb1727c7 server_cookie=2350ff28a7fc22bb in_seq=0 out_seq=0 2023-07-18T03:19:34.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.054+0000 7f923dffb700 10 client.17970 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:19:34.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.055+0000 7f923dffb700 1 -- 172.21.15.101:0/2081777156 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55661fd54eb0 con 0x55661fe24480 2023-07-18T03:19:34.059 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.055+0000 7f923dffb700 10 client.17970 handle_command_reply: tid=0 2023-07-18T03:19:34.059 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.055+0000 7f92706d2b80 1 client.17970 shutdown 2023-07-18T03:19:34.059 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.055+0000 7f92706d2b80 1 -- 172.21.15.101:0/2081777156 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55661fe09be0 msgr2=0x55661fe11790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:34.059 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.055+0000 7f92706d2b80 1 --2- 172.21.15.101:0/2081777156 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55661fe09be0 0x55661fe11790 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f924c00e970 tx=0x7f924c005b60 comp rx=0 tx=0).stop 2023-07-18T03:19:34.059 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.055+0000 7f92706d2b80 1 -- 172.21.15.101:0/2081777156 shutdown_connections 2023-07-18T03:19:34.059 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.055+0000 7f92706d2b80 1 --2- 172.21.15.101:0/2081777156 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55661fe24480 0x55661fe26890 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:34.059 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.055+0000 7f92706d2b80 1 --2- 172.21.15.101:0/2081777156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55661fe092d0 0x55661fe14510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:34.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.056+0000 7f92706d2b80 1 --2- 172.21.15.101:0/2081777156 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55661fe14ad0 0x55661fe11ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:34.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.056+0000 7f92706d2b80 1 --2- 172.21.15.101:0/2081777156 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55661fe09be0 0x55661fe11790 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:34.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.056+0000 7f92706d2b80 1 -- 172.21.15.101:0/2081777156 >> 172.21.15.101:0/2081777156 conn(0x55661fdffda0 msgr2=0x55661fe05230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:34.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.056+0000 7f923dffb700 0 client.17970 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:19:34.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.056+0000 7f92706d2b80 1 -- 172.21.15.101:0/2081777156 shutdown_connections 2023-07-18T03:19:34.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.056+0000 7f92706d2b80 1 -- 172.21.15.101:0/2081777156 wait complete. 2023-07-18T03:19:34.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.056+0000 7f92706d2b80 20 client.17970 trim_cache size 0 max 16384 2023-07-18T03:19:34.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.069+0000 7f92706d2b80 1 Processor -- start 2023-07-18T03:19:34.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.069+0000 7f92706d2b80 1 -- start start 2023-07-18T03:19:34.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.069+0000 7f92706d2b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55661fe092d0 0x55661fe140b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:34.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.069+0000 7f92706d2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55661fe09be0 0x55661fe0ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:34.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.069+0000 7f92706d2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55661fe14ad0 0x55661fe0cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:34.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.069+0000 7f92706d2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55661fe06ad0 con 0x55661fe09be0 2023-07-18T03:19:34.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.069+0000 7f92706d2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55661fe0a3d0 con 0x55661fe14ad0 2023-07-18T03:19:34.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.069+0000 7f92706d2b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55661fdff970 con 0x55661fe092d0 2023-07-18T03:19:34.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.070+0000 7f9255d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55661fe09be0 0x55661fe0ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:34.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.070+0000 7f9255d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55661fe09be0 0x55661fe0ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:37978/0 (socket says 172.21.15.101:37978) 2023-07-18T03:19:34.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.070+0000 7f9255d9b700 1 -- 172.21.15.101:0/2928223057 learned_addr learned my addr 172.21.15.101:0/2928223057 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:34.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.070+0000 7f925659c700 1 --2- 172.21.15.101:0/2928223057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55661fe092d0 0x55661fe140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:34.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.070+0000 7f9256d9d700 1 --2- 172.21.15.101:0/2928223057 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55661fe14ad0 0x55661fe0cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:34.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.070+0000 7f9255d9b700 1 -- 172.21.15.101:0/2928223057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55661fe092d0 msgr2=0x55661fe140b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:34.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.070+0000 7f9255d9b700 1 --2- 172.21.15.101:0/2928223057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55661fe092d0 0x55661fe140b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:34.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.070+0000 7f9255d9b700 1 -- 172.21.15.101:0/2928223057 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55661fe14ad0 msgr2=0x55661fe0cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:34.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.070+0000 7f9255d9b700 1 --2- 172.21.15.101:0/2928223057 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55661fe14ad0 0x55661fe0cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:34.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.070+0000 7f9255d9b700 1 -- 172.21.15.101:0/2928223057 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55661fbb0250 con 0x55661fe09be0 2023-07-18T03:19:34.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.070+0000 7f9255d9b700 1 --2- 172.21.15.101:0/2928223057 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55661fe09be0 0x55661fe0ca90 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f924c099ed0 tx=0x7f924c010a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=994426662d5bf5e3 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:34.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.070+0000 7f92457fa700 1 -- 172.21.15.101:0/2928223057 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924c09a3a0 con 0x55661fe09be0 2023-07-18T03:19:34.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.071+0000 7f92457fa700 1 -- 172.21.15.101:0/2928223057 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f924c0b1070 con 0x55661fe09be0 2023-07-18T03:19:34.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.071+0000 7f92457fa700 1 -- 172.21.15.101:0/2928223057 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924c0a1f00 con 0x55661fe09be0 2023-07-18T03:19:34.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.071+0000 7f92706d2b80 1 -- 172.21.15.101:0/2928223057 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55661fe09be0 msgr2=0x55661fe0ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:34.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.071+0000 7f92706d2b80 1 --2- 172.21.15.101:0/2928223057 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55661fe09be0 0x55661fe0ca90 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f924c099ed0 tx=0x7f924c010a90 comp rx=0 tx=0).stop 2023-07-18T03:19:34.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.071+0000 7f92706d2b80 1 -- 172.21.15.101:0/2928223057 shutdown_connections 2023-07-18T03:19:34.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.071+0000 7f92706d2b80 1 --2- 172.21.15.101:0/2928223057 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55661fe14ad0 0x55661fe0cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:34.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.071+0000 7f92706d2b80 1 --2- 172.21.15.101:0/2928223057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55661fe092d0 0x55661fe140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:34.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.071+0000 7f92706d2b80 1 --2- 172.21.15.101:0/2928223057 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55661fe09be0 0x55661fe0ca90 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:34.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.071+0000 7f92706d2b80 1 -- 172.21.15.101:0/2928223057 >> 172.21.15.101:0/2928223057 conn(0x55661fdffda0 msgr2=0x55661fe207c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:34.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.071+0000 7f92706d2b80 1 -- 172.21.15.101:0/2928223057 shutdown_connections 2023-07-18T03:19:34.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.071+0000 7f92706d2b80 1 -- 172.21.15.101:0/2928223057 wait complete. 2023-07-18T03:19:34.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.072+0000 7f92706d2b80 1 -- start start 2023-07-18T03:19:34.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.072+0000 7f92706d2b80 1 Processor -- start 2023-07-18T03:19:34.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.072+0000 7f92706d2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55661fe092d0 0x55661fe19530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:34.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.072+0000 7f92706d2b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55661fe09be0 0x55661fe19a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:34.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.072+0000 7f92706d2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55661fe14ad0 0x55661fe19fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:34.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.072+0000 7f92706d2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55661fe77520 con 0x55661fe14ad0 2023-07-18T03:19:34.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.072+0000 7f92706d2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55661fe77690 con 0x55661fe092d0 2023-07-18T03:19:34.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.072+0000 7f92706d2b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55661fe77850 con 0x55661fe09be0 2023-07-18T03:19:34.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.072+0000 7f92706d2b80 10 client.0 _ll_register_callbacks cb 0x55661fcf9760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:19:34.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.073+0000 7f925659c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55661fe092d0 0x55661fe19530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:34.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.073+0000 7f9256d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55661fe14ad0 0x55661fe19fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:34.075 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.073+0000 7f925659c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55661fe092d0 0x55661fe19530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:55962/0 (socket says 172.21.15.101:55962) 2023-07-18T03:19:34.075 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.073+0000 7f9256d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55661fe14ad0 0x55661fe19fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:37994/0 (socket says 172.21.15.101:37994) 2023-07-18T03:19:34.075 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.073+0000 7f925659c700 1 -- 172.21.15.101:0/4101217494 learned_addr learned my addr 172.21.15.101:0/4101217494 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:34.075 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.073+0000 7f9255d9b700 1 --2- 172.21.15.101:0/4101217494 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55661fe09be0 0x55661fe19a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:34.075 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.073+0000 7f925659c700 1 -- 172.21.15.101:0/4101217494 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55661fe09be0 msgr2=0x55661fe19a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:34.075 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.073+0000 7f925659c700 1 --2- 172.21.15.101:0/4101217494 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55661fe09be0 0x55661fe19a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:34.075 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.073+0000 7f925659c700 1 -- 172.21.15.101:0/4101217494 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55661fe14ad0 msgr2=0x55661fe19fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:34.075 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.073+0000 7f925659c700 1 --2- 172.21.15.101:0/4101217494 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55661fe14ad0 0x55661fe19fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:34.076 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.073+0000 7f925659c700 1 -- 172.21.15.101:0/4101217494 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55661fb72c60 con 0x55661fe092d0 2023-07-18T03:19:34.076 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.073+0000 7f9256d9d700 1 --2- 172.21.15.101:0/4101217494 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55661fe14ad0 0x55661fe19fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:19:34.076 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.073+0000 7f925659c700 1 --2- 172.21.15.101:0/4101217494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55661fe092d0 0x55661fe19530 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x55661fe271a0 tx=0x7f9248004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1709fb1ec4902a7 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:34.076 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.073+0000 7f923dffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:19:34.076 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.073+0000 7f923dffb700 1 -- 172.21.15.101:0/4101217494 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9248022040 con 0x55661fe092d0 2023-07-18T03:19:34.076 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.074+0000 7f92706d2b80 1 -- 172.21.15.101:0/4101217494 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9250003a00 con 0x55661fe092d0 2023-07-18T03:19:34.076 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.074+0000 7f923dffb700 1 -- 172.21.15.101:0/4101217494 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f924802bd90 con 0x55661fe092d0 2023-07-18T03:19:34.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.074+0000 7f923dffb700 1 -- 172.21.15.101:0/4101217494 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9248017070 con 0x55661fe092d0 2023-07-18T03:19:34.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.074+0000 7f92706d2b80 10 client.28022 fetch_fsmap learned FSMap version 33 2023-07-18T03:19:34.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.074+0000 7f92706d2b80 1 -- 172.21.15.101:0/4101217494 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55661fb882b0 con 0x55661fe092d0 2023-07-18T03:19:34.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.074+0000 7f923dffb700 1 -- 172.21.15.101:0/4101217494 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9250003a00 con 0x55661fe092d0 2023-07-18T03:19:34.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.074+0000 7f923dffb700 1 -- 172.21.15.101:0/4101217494 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9248005140 con 0x55661fe092d0 2023-07-18T03:19:34.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.074+0000 7f92706d2b80 10 client.28022 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:19:34.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.074+0000 7f92706d2b80 10 client.28022 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:19:34.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.074+0000 7f92706d2b80 20 client.28022 populate_metadata read hostname 'smithi101' 2023-07-18T03:19:34.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.074+0000 7f92706d2b80 1 --2- 172.21.15.101:0/4101217494 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55661fe78fb0 0x55661fe7b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:34.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.074+0000 7f92706d2b80 4 client.28022 mds_command: new command op to 24562 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:19:34.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.074+0000 7f92706d2b80 1 -- 172.21.15.101:0/4101217494 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55661fd17050 con 0x55661fe78fb0 2023-07-18T03:19:34.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.075+0000 7f9255d9b700 1 --2- 172.21.15.101:0/4101217494 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55661fe78fb0 0x55661fe7b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:34.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.075+0000 7f923dffb700 10 client.28022 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:19:34.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.075+0000 7f9255d9b700 1 --2- 172.21.15.101:0/4101217494 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55661fe78fb0 0x55661fe7b3c0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5ff645bac308c69 server_cookie=2c7219093bcb887d in_seq=0 out_seq=0 2023-07-18T03:19:34.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.075+0000 7f923dffb700 1 -- 172.21.15.101:0/4101217494 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55661fd17050 con 0x55661fe78fb0 2023-07-18T03:19:34.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.075+0000 7f923dffb700 10 client.28022 handle_command_reply: tid=0 2023-07-18T03:19:34.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.075+0000 7f92706d2b80 1 client.28022 shutdown 2023-07-18T03:19:34.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.075+0000 7f92706d2b80 1 -- 172.21.15.101:0/4101217494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55661fe092d0 msgr2=0x55661fe19530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:34.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.075+0000 7f92706d2b80 1 --2- 172.21.15.101:0/4101217494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55661fe092d0 0x55661fe19530 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x55661fe271a0 tx=0x7f9248004750 comp rx=0 tx=0).stop 2023-07-18T03:19:34.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.075+0000 7f92706d2b80 1 -- 172.21.15.101:0/4101217494 shutdown_connections 2023-07-18T03:19:34.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.075+0000 7f92706d2b80 1 --2- 172.21.15.101:0/4101217494 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55661fe78fb0 0x55661fe7b3c0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:34.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.075+0000 7f92706d2b80 1 --2- 172.21.15.101:0/4101217494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55661fe092d0 0x55661fe19530 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:34.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.075+0000 7f923dffb700 0 client.28022 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:19:34.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.075+0000 7f92706d2b80 1 --2- 172.21.15.101:0/4101217494 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55661fe14ad0 0x55661fe19fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:34.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.075+0000 7f92706d2b80 1 --2- 172.21.15.101:0/4101217494 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55661fe09be0 0x55661fe19a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:34.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.075+0000 7f92706d2b80 1 -- 172.21.15.101:0/4101217494 >> 172.21.15.101:0/4101217494 conn(0x55661fdffda0 msgr2=0x55661fe0ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:34.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.076+0000 7f92706d2b80 1 -- 172.21.15.101:0/4101217494 shutdown_connections 2023-07-18T03:19:34.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.076+0000 7f92706d2b80 1 -- 172.21.15.101:0/4101217494 wait complete. 2023-07-18T03:19:34.081 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.076+0000 7f92706d2b80 20 client.28022 trim_cache size 0 max 16384 2023-07-18T03:19:34.081 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:19:34.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.076+0000 7f925cdff700 1 -- 172.21.15.101:0/1463901548 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f92400a1340 msgr2=0x7f92400a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:34.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.076+0000 7f925cdff700 1 --2- 172.21.15.101:0/1463901548 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f92400a1340 0x7f92400a37f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f92480125f0 tx=0x7f9248009260 comp rx=0 tx=0).stop 2023-07-18T03:19:34.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.076+0000 7f925cdff700 1 -- 172.21.15.101:0/1463901548 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9258100490 msgr2=0x7f9258198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:34.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.076+0000 7f925cdff700 1 --2- 172.21.15.101:0/1463901548 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9258100490 0x7f9258198400 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f924c00e9a0 tx=0x7f924c00ecb0 comp rx=0 tx=0).stop 2023-07-18T03:19:34.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.076+0000 7f925cdff700 1 -- 172.21.15.101:0/1463901548 shutdown_connections 2023-07-18T03:19:34.083 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.076+0000 7f925cdff700 1 --2- 172.21.15.101:0/1463901548 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f92400a1340 0x7f92400a37f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:34.083 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.076+0000 7f925cdff700 1 --2- 172.21.15.101:0/1463901548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9258105af0 0x7f925819c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:34.083 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.076+0000 7f925cdff700 1 --2- 172.21.15.101:0/1463901548 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92580ffae0 0x7f9258197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:34.083 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.076+0000 7f925cdff700 1 --2- 172.21.15.101:0/1463901548 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9258100490 0x7f9258198400 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:34.083 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.076+0000 7f925cdff700 1 -- 172.21.15.101:0/1463901548 >> 172.21.15.101:0/1463901548 conn(0x7f92580fb5e0 msgr2=0x7f92580fbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:34.083 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.077+0000 7f925cdff700 1 -- 172.21.15.101:0/1463901548 shutdown_connections 2023-07-18T03:19:34.084 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:34.077+0000 7f925cdff700 1 -- 172.21.15.101:0/1463901548 wait complete. 2023-07-18T03:19:34.086 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:34 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1937854468' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:19:34.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:34 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1937854468' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:19:34.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:34 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1937854468' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:19:34.626 INFO:tasks.workunit.client.0.smithi101.stdout: 1 719454 5.41 MB/sec execute 539 sec latency 2391.230 ms 2023-07-18T03:19:35.089 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-18T03:19:35.089 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-18T03:19:35.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:35 smithi155 ceph-mon[128885]: pgmap v375: 129 pgs: 129 active+clean; 586 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 106 op/s 2023-07-18T03:19:35.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.322+0000 7f9659fff700 1 -- 172.21.15.101:0/1688810044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9654105af0 msgr2=0x7f9654105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:35.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.322+0000 7f9659fff700 1 --2- 172.21.15.101:0/1688810044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9654105af0 0x7f9654105ec0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9644009990 tx=0x7f9644009ca0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.322+0000 7f9659fff700 1 -- 172.21.15.101:0/1688810044 shutdown_connections 2023-07-18T03:19:35.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.322+0000 7f9659fff700 1 --2- 172.21.15.101:0/1688810044 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9654100490 0x7f965410c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.322+0000 7f9659fff700 1 --2- 172.21.15.101:0/1688810044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9654105af0 0x7f9654105ec0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.322+0000 7f9659fff700 1 --2- 172.21.15.101:0/1688810044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96540ffae0 0x7f96540fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.322+0000 7f9659fff700 1 -- 172.21.15.101:0/1688810044 >> 172.21.15.101:0/1688810044 conn(0x7f96540fb5e0 msgr2=0x7f96540fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:35.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.323+0000 7f9659fff700 1 -- 172.21.15.101:0/1688810044 shutdown_connections 2023-07-18T03:19:35.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.323+0000 7f9659fff700 1 -- 172.21.15.101:0/1688810044 wait complete. 2023-07-18T03:19:35.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.324+0000 7f9659fff700 1 Processor -- start 2023-07-18T03:19:35.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.324+0000 7f9659fff700 1 -- start start 2023-07-18T03:19:35.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.324+0000 7f9659fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f96540ffae0 0x7f9654197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:35.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.324+0000 7f9659fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9654100490 0x7f96541982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:35.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.324+0000 7f9659fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9654105af0 0x7f965419c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:35.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.324+0000 7f9659fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965419cd90 con 0x7f96540ffae0 2023-07-18T03:19:35.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.324+0000 7f9659fff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965419cf00 con 0x7f9654105af0 2023-07-18T03:19:35.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.324+0000 7f9659fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965419d0c0 con 0x7f9654100490 2023-07-18T03:19:35.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.325+0000 7f964bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9654100490 0x7f96541982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:35.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.325+0000 7f964bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9654100490 0x7f96541982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:44900/0 (socket says 172.21.15.101:44900) 2023-07-18T03:19:35.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.325+0000 7f964bfff700 1 -- 172.21.15.101:0/1957683887 learned_addr learned my addr 172.21.15.101:0/1957683887 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:35.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.325+0000 7f96537fe700 1 --2- 172.21.15.101:0/1957683887 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f96540ffae0 0x7f9654197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:35.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.325+0000 7f9653fff700 1 --2- 172.21.15.101:0/1957683887 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9654105af0 0x7f965419c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:35.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.325+0000 7f964bfff700 1 -- 172.21.15.101:0/1957683887 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9654105af0 msgr2=0x7f965419c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:35.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.325+0000 7f964bfff700 1 --2- 172.21.15.101:0/1957683887 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9654105af0 0x7f965419c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.325+0000 7f964bfff700 1 -- 172.21.15.101:0/1957683887 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f96540ffae0 msgr2=0x7f9654197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:35.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.325+0000 7f964bfff700 1 --2- 172.21.15.101:0/1957683887 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f96540ffae0 0x7f9654197db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.325+0000 7f964bfff700 1 -- 172.21.15.101:0/1957683887 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96440095f0 con 0x7f9654100490 2023-07-18T03:19:35.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.326+0000 7f964bfff700 1 --2- 172.21.15.101:0/1957683887 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9654100490 0x7f96541982f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f963c00e9a0 tx=0x7f963c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:35.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.326+0000 7f96517fa700 1 -- 172.21.15.101:0/1957683887 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963c009790 con 0x7f9654100490 2023-07-18T03:19:35.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.326+0000 7f96517fa700 1 -- 172.21.15.101:0/1957683887 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f963c010430 con 0x7f9654100490 2023-07-18T03:19:35.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.326+0000 7f96517fa700 1 -- 172.21.15.101:0/1957683887 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963c018680 con 0x7f9654100490 2023-07-18T03:19:35.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.326+0000 7f9659fff700 1 -- 172.21.15.101:0/1957683887 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f965419d3a0 con 0x7f9654100490 2023-07-18T03:19:35.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.326+0000 7f9659fff700 1 -- 172.21.15.101:0/1957683887 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96541a4be0 con 0x7f9654100490 2023-07-18T03:19:35.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.329+0000 7f96517fa700 1 -- 172.21.15.101:0/1957683887 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f963c0105a0 con 0x7f9654100490 2023-07-18T03:19:35.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.330+0000 7f966d9d3b80 1 Processor -- start 2023-07-18T03:19:35.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.331+0000 7f96517fa700 1 --2- 172.21.15.101:0/1957683887 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f96400a1340 0x7f96400a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:35.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.331+0000 7f966d9d3b80 1 -- start start 2023-07-18T03:19:35.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.331+0000 7f96517fa700 1 -- 172.21.15.101:0/1957683887 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f963c014070 con 0x7f9654100490 2023-07-18T03:19:35.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.331+0000 7f9653fff700 1 --2- 172.21.15.101:0/1957683887 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f96400a1340 0x7f96400a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:35.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.331+0000 7f966d9d3b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563abca97350 0x563abca97720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:35.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.331+0000 7f966d9d3b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563abca97c60 0x563abcaa2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:35.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.331+0000 7f966d9d3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abcaa2b50 0x563abcaa4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:35.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.331+0000 7f966d9d3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563abc99dd30 con 0x563abcaa2b50 2023-07-18T03:19:35.327 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.331+0000 7f966d9d3b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563abc9f0ac0 con 0x563abca97350 2023-07-18T03:19:35.327 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.331+0000 7f966d9d3b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563abc9e9f50 con 0x563abca97c60 2023-07-18T03:19:35.328 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.331+0000 7f96537fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563abca97350 0x563abca97720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:35.328 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.331+0000 7f96537fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563abca97350 0x563abca97720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:55984/0 (socket says 172.21.15.101:55984) 2023-07-18T03:19:35.328 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.332+0000 7f96537fe700 1 -- 172.21.15.101:0/1987699899 learned_addr learned my addr 172.21.15.101:0/1987699899 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:35.328 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.332+0000 7f9653fff700 1 --2- 172.21.15.101:0/1987699899 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abcaa2b50 0x563abcaa4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:35.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.332+0000 7f964bfff700 1 --2- 172.21.15.101:0/1987699899 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563abca97c60 0x563abcaa2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:35.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.332+0000 7f96537fe700 1 -- 172.21.15.101:0/1987699899 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563abca97c60 msgr2=0x563abcaa2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:35.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.332+0000 7f96537fe700 1 --2- 172.21.15.101:0/1987699899 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563abca97c60 0x563abcaa2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.332+0000 7f96537fe700 1 -- 172.21.15.101:0/1987699899 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abcaa2b50 msgr2=0x563abcaa4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:35.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.332+0000 7f96537fe700 1 --2- 172.21.15.101:0/1987699899 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abcaa2b50 0x563abcaa4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.332+0000 7f96537fe700 1 -- 172.21.15.101:0/1987699899 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563abc817e60 con 0x563abca97350 2023-07-18T03:19:35.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.332+0000 7f9653fff700 1 --2- 172.21.15.101:0/1957683887 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f96400a1340 0x7f96400a37f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f964c00f590 tx=0x7f964c009920 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:35.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.332+0000 7f96537fe700 1 --2- 172.21.15.101:0/1987699899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563abca97350 0x563abca97720 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9644005ea0 tx=0x7f9644005ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b838560e1454d433 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:35.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.333+0000 7f9649ffb700 1 -- 172.21.15.101:0/1987699899 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964401d070 con 0x563abca97350 2023-07-18T03:19:35.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.333+0000 7f9649ffb700 1 -- 172.21.15.101:0/1987699899 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f964401e040 con 0x563abca97350 2023-07-18T03:19:35.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.333+0000 7f9649ffb700 1 -- 172.21.15.101:0/1987699899 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9644004920 con 0x563abca97350 2023-07-18T03:19:35.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.333+0000 7f966d9d3b80 1 -- 172.21.15.101:0/1987699899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563abca97350 msgr2=0x563abca97720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:35.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.333+0000 7f966d9d3b80 1 --2- 172.21.15.101:0/1987699899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563abca97350 0x563abca97720 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9644005ea0 tx=0x7f9644005ed0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.334+0000 7f966d9d3b80 1 -- 172.21.15.101:0/1987699899 shutdown_connections 2023-07-18T03:19:35.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.334+0000 7f966d9d3b80 1 --2- 172.21.15.101:0/1987699899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563abca97350 0x563abca97720 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.334+0000 7f966d9d3b80 1 --2- 172.21.15.101:0/1987699899 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abcaa2b50 0x563abcaa4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.334+0000 7f966d9d3b80 1 --2- 172.21.15.101:0/1987699899 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563abca97c60 0x563abcaa2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.334+0000 7f966d9d3b80 1 -- 172.21.15.101:0/1987699899 >> 172.21.15.101:0/1987699899 conn(0x563abca8de20 msgr2=0x563abca93720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:35.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.334+0000 7f966d9d3b80 1 -- 172.21.15.101:0/1987699899 shutdown_connections 2023-07-18T03:19:35.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.334+0000 7f966d9d3b80 1 -- 172.21.15.101:0/1987699899 wait complete. 2023-07-18T03:19:35.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.334+0000 7f966d9d3b80 1 -- start start 2023-07-18T03:19:35.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.335+0000 7f966d9d3b80 1 Processor -- start 2023-07-18T03:19:35.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.335+0000 7f966d9d3b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563abca97350 0x563abcaa2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:35.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.335+0000 7f966d9d3b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563abca97c60 0x563abca9f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:35.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.335+0000 7f966d9d3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abcaa2b50 0x563abca9fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:35.338 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.335+0000 7f966d9d3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563abc989b40 con 0x563abcaa2b50 2023-07-18T03:19:35.338 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.335+0000 7f966d9d3b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563abc989840 con 0x563abca97350 2023-07-18T03:19:35.338 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.335+0000 7f966d9d3b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563abc86d9a0 con 0x563abca97c60 2023-07-18T03:19:35.338 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.335+0000 7f966d9d3b80 10 client.0 _ll_register_callbacks cb 0x563abc9877e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:19:35.338 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.335+0000 7f964bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563abca97c60 0x563abca9f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:35.339 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.335+0000 7f964bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563abca97c60 0x563abca9f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:44906/0 (socket says 172.21.15.101:44906) 2023-07-18T03:19:35.339 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.335+0000 7f964bfff700 1 -- 172.21.15.101:0/690829931 learned_addr learned my addr 172.21.15.101:0/690829931 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:35.339 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.336+0000 7f9653fff700 1 --2- 172.21.15.101:0/690829931 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abcaa2b50 0x563abca9fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:35.339 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.336+0000 7f96537fe700 1 --2- 172.21.15.101:0/690829931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563abca97350 0x563abcaa2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:35.339 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.336+0000 7f964bfff700 1 -- 172.21.15.101:0/690829931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563abca97350 msgr2=0x563abcaa2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:35.339 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.336+0000 7f964bfff700 1 --2- 172.21.15.101:0/690829931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563abca97350 0x563abcaa2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.339 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.336+0000 7f964bfff700 1 -- 172.21.15.101:0/690829931 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abcaa2b50 msgr2=0x563abca9fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:35.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.336+0000 7f964bfff700 1 --2- 172.21.15.101:0/690829931 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abcaa2b50 0x563abca9fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.336+0000 7f964bfff700 1 -- 172.21.15.101:0/690829931 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f965410d220 con 0x563abca97c60 2023-07-18T03:19:35.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.336+0000 7f9653fff700 1 --2- 172.21.15.101:0/690829931 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abcaa2b50 0x563abca9fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:19:35.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.336+0000 7f964bfff700 1 --2- 172.21.15.101:0/690829931 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563abca97c60 0x563abca9f790 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f963c096460 tx=0x7f963c09a050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77c29f34b5bbe769 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:35.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.336+0000 7f9632ffd700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:19:35.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.336+0000 7f9632ffd700 1 -- 172.21.15.101:0/690829931 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963c099650 con 0x563abca97c60 2023-07-18T03:19:35.340 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.336+0000 7f9632ffd700 1 -- 172.21.15.101:0/690829931 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f963c0a23b0 con 0x563abca97c60 2023-07-18T03:19:35.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.336+0000 7f966d9d3b80 1 -- 172.21.15.101:0/690829931 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563abc943570 con 0x563abca97c60 2023-07-18T03:19:35.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.336+0000 7f9632ffd700 1 -- 172.21.15.101:0/690829931 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f963c0b8120 con 0x563abca97c60 2023-07-18T03:19:35.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.336+0000 7f966d9d3b80 10 client.28066 fetch_fsmap learned FSMap version 33 2023-07-18T03:19:35.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.337+0000 7f966d9d3b80 1 -- 172.21.15.101:0/690829931 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x563abc800ce0 con 0x563abca97c60 2023-07-18T03:19:35.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.337+0000 7f9632ffd700 1 -- 172.21.15.101:0/690829931 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f963c0c8020 con 0x563abca97c60 2023-07-18T03:19:35.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.337+0000 7f9632ffd700 1 -- 172.21.15.101:0/690829931 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f963c094b70 con 0x563abca97c60 2023-07-18T03:19:35.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.337+0000 7f966d9d3b80 10 client.28066 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:19:35.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.337+0000 7f966d9d3b80 10 client.28066 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:19:35.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.337+0000 7f966d9d3b80 20 client.28066 populate_metadata read hostname 'smithi101' 2023-07-18T03:19:35.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.337+0000 7f966d9d3b80 1 --2- 172.21.15.101:0/690829931 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563abcab2360 0x563abcab4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:35.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.337+0000 7f966d9d3b80 4 client.28066 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:19:35.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.337+0000 7f966d9d3b80 1 -- 172.21.15.101:0/690829931 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563abc9e2f30 con 0x563abcab2360 2023-07-18T03:19:35.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.337+0000 7f96537fe700 1 --2- 172.21.15.101:0/690829931 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563abcab2360 0x563abcab4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:35.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.338+0000 7f96537fe700 1 --2- 172.21.15.101:0/690829931 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563abcab2360 0x563abcab4770 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a1584d9ed35479b server_cookie=1d454b656b9d1b84 in_seq=0 out_seq=0 2023-07-18T03:19:35.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.338+0000 7f9632ffd700 10 client.28066 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:19:35.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.338+0000 7f9632ffd700 1 -- 172.21.15.101:0/690829931 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563abc9e2f30 con 0x563abcab2360 2023-07-18T03:19:35.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.338+0000 7f9632ffd700 10 client.28066 handle_command_reply: tid=0 2023-07-18T03:19:35.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.338+0000 7f966d9d3b80 1 client.28066 shutdown 2023-07-18T03:19:35.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.339+0000 7f966d9d3b80 1 -- 172.21.15.101:0/690829931 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563abca97c60 msgr2=0x563abca9f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:35.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.339+0000 7f966d9d3b80 1 --2- 172.21.15.101:0/690829931 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563abca97c60 0x563abca9f790 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f963c096460 tx=0x7f963c09a050 comp rx=0 tx=0).stop 2023-07-18T03:19:35.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.339+0000 7f966d9d3b80 1 -- 172.21.15.101:0/690829931 shutdown_connections 2023-07-18T03:19:35.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.339+0000 7f966d9d3b80 1 --2- 172.21.15.101:0/690829931 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563abcab2360 0x563abcab4770 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.339+0000 7f966d9d3b80 1 --2- 172.21.15.101:0/690829931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563abca97350 0x563abcaa2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.339+0000 7f966d9d3b80 1 --2- 172.21.15.101:0/690829931 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abcaa2b50 0x563abca9fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.339+0000 7f966d9d3b80 1 --2- 172.21.15.101:0/690829931 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563abca97c60 0x563abca9f790 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.339+0000 7f9632ffd700 0 client.28066 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:19:35.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.339+0000 7f966d9d3b80 1 -- 172.21.15.101:0/690829931 >> 172.21.15.101:0/690829931 conn(0x563abca8de20 msgr2=0x563abca932b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:35.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.339+0000 7f966d9d3b80 1 -- 172.21.15.101:0/690829931 shutdown_connections 2023-07-18T03:19:35.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.339+0000 7f966d9d3b80 1 -- 172.21.15.101:0/690829931 wait complete. 2023-07-18T03:19:35.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.339+0000 7f966d9d3b80 20 client.28066 trim_cache size 0 max 16384 2023-07-18T03:19:35.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.356+0000 7f966d9d3b80 1 Processor -- start 2023-07-18T03:19:35.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.356+0000 7f966d9d3b80 1 -- start start 2023-07-18T03:19:35.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.357+0000 7f966d9d3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abca97350 0x563abcaa9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:35.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.357+0000 7f966d9d3b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563abca97c60 0x563abcaa17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:35.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.357+0000 7f966d9d3b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563abcaa2b50 0x563abca9ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:35.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.357+0000 7f966d9d3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563abca985c0 con 0x563abca97350 2023-07-18T03:19:35.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.357+0000 7f966d9d3b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563abca981e0 con 0x563abcaa2b50 2023-07-18T03:19:35.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.357+0000 7f966d9d3b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563abca9dcf0 con 0x563abca97c60 2023-07-18T03:19:35.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.357+0000 7f96537fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abca97350 0x563abcaa9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:35.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.357+0000 7f9653fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563abcaa2b50 0x563abca9ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:35.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.357+0000 7f96537fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abca97350 0x563abcaa9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:38030/0 (socket says 172.21.15.101:38030) 2023-07-18T03:19:35.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.357+0000 7f96537fe700 1 -- 172.21.15.101:0/35976933 learned_addr learned my addr 172.21.15.101:0/35976933 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:35.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.357+0000 7f964bfff700 1 --2- 172.21.15.101:0/35976933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563abca97c60 0x563abcaa17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:35.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.357+0000 7f96537fe700 1 -- 172.21.15.101:0/35976933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563abca97c60 msgr2=0x563abcaa17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:35.355 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.357+0000 7f96537fe700 1 --2- 172.21.15.101:0/35976933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563abca97c60 0x563abcaa17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.355 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.357+0000 7f96537fe700 1 -- 172.21.15.101:0/35976933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563abcaa2b50 msgr2=0x563abca9ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:35.355 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.357+0000 7f96537fe700 1 --2- 172.21.15.101:0/35976933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563abcaa2b50 0x563abca9ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.355 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.357+0000 7f96537fe700 1 -- 172.21.15.101:0/35976933 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563abc83e2d0 con 0x563abca97350 2023-07-18T03:19:35.355 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.357+0000 7f9653fff700 1 --2- 172.21.15.101:0/35976933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563abcaa2b50 0x563abca9ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:19:35.355 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.358+0000 7f96537fe700 1 --2- 172.21.15.101:0/35976933 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abca97350 0x563abcaa9530 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f9644005ea0 tx=0x7f9644009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44bb55be3b085005 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:35.355 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.358+0000 7f964a7fc700 1 -- 172.21.15.101:0/35976933 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9644030070 con 0x563abca97350 2023-07-18T03:19:35.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.358+0000 7f964a7fc700 1 -- 172.21.15.101:0/35976933 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9644005950 con 0x563abca97350 2023-07-18T03:19:35.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.358+0000 7f964a7fc700 1 -- 172.21.15.101:0/35976933 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964401d040 con 0x563abca97350 2023-07-18T03:19:35.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.359+0000 7f966d9d3b80 1 -- 172.21.15.101:0/35976933 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abca97350 msgr2=0x563abcaa9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:35.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.359+0000 7f966d9d3b80 1 --2- 172.21.15.101:0/35976933 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abca97350 0x563abcaa9530 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f9644005ea0 tx=0x7f9644009ca0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.359+0000 7f966d9d3b80 1 -- 172.21.15.101:0/35976933 shutdown_connections 2023-07-18T03:19:35.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.359+0000 7f966d9d3b80 1 --2- 172.21.15.101:0/35976933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563abcaa2b50 0x563abca9ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.359+0000 7f966d9d3b80 1 --2- 172.21.15.101:0/35976933 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abca97350 0x563abcaa9530 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.359+0000 7f966d9d3b80 1 --2- 172.21.15.101:0/35976933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563abca97c60 0x563abcaa17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.359+0000 7f966d9d3b80 1 -- 172.21.15.101:0/35976933 >> 172.21.15.101:0/35976933 conn(0x563abca8de20 msgr2=0x563abcaa10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:35.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.359+0000 7f966d9d3b80 1 -- 172.21.15.101:0/35976933 shutdown_connections 2023-07-18T03:19:35.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.359+0000 7f966d9d3b80 1 -- 172.21.15.101:0/35976933 wait complete. 2023-07-18T03:19:35.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.359+0000 7f966d9d3b80 1 -- start start 2023-07-18T03:19:35.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.359+0000 7f966d9d3b80 1 Processor -- start 2023-07-18T03:19:35.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.360+0000 7f966d9d3b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563abca97350 0x563abcaa81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:35.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.360+0000 7f966d9d3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abca97c60 0x563abcaa8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:35.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.360+0000 7f966d9d3b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563abcaa2b50 0x563abcb00200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:35.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.360+0000 7f966d9d3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563abcb00740 con 0x563abca97c60 2023-07-18T03:19:35.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.360+0000 7f966d9d3b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563abcb008b0 con 0x563abcaa2b50 2023-07-18T03:19:35.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.360+0000 7f966d9d3b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563abcb00a70 con 0x563abca97350 2023-07-18T03:19:35.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.360+0000 7f966d9d3b80 10 client.0 _ll_register_callbacks cb 0x7f963c099b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:19:35.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.360+0000 7f964bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abca97c60 0x563abcaa8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:35.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.360+0000 7f9653fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563abcaa2b50 0x563abcb00200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:35.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.360+0000 7f964bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abca97c60 0x563abcaa8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:38032/0 (socket says 172.21.15.101:38032) 2023-07-18T03:19:35.361 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.360+0000 7f964bfff700 1 -- 172.21.15.101:0/2165456543 learned_addr learned my addr 172.21.15.101:0/2165456543 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:35.361 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.360+0000 7f9653fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563abcaa2b50 0x563abcb00200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:56000/0 (socket says 172.21.15.101:56000) 2023-07-18T03:19:35.361 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.360+0000 7f96537fe700 1 --2- 172.21.15.101:0/2165456543 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563abca97350 0x563abcaa81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:35.361 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.360+0000 7f964bfff700 1 -- 172.21.15.101:0/2165456543 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563abca97350 msgr2=0x563abcaa81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:35.361 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.360+0000 7f964bfff700 1 --2- 172.21.15.101:0/2165456543 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563abca97350 0x563abcaa81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.361 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.360+0000 7f964bfff700 1 -- 172.21.15.101:0/2165456543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563abcaa2b50 msgr2=0x563abcb00200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:35.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.360+0000 7f964bfff700 1 --2- 172.21.15.101:0/2165456543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563abcaa2b50 0x563abcb00200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.360+0000 7f964bfff700 1 -- 172.21.15.101:0/2165456543 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f964c019460 con 0x563abca97c60 2023-07-18T03:19:35.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.360+0000 7f9653fff700 1 --2- 172.21.15.101:0/2165456543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563abcaa2b50 0x563abcb00200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:19:35.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.361+0000 7f964bfff700 1 --2- 172.21.15.101:0/2165456543 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abca97c60 0x563abcaa8720 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f963c099ef0 tx=0x7f963c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=264a221f00e46489 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:35.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.361+0000 7f9632ffd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:19:35.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.361+0000 7f9632ffd700 1 -- 172.21.15.101:0/2165456543 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963c0b8320 con 0x563abca97c60 2023-07-18T03:19:35.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.361+0000 7f9632ffd700 1 -- 172.21.15.101:0/2165456543 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f963c0c9080 con 0x563abca97c60 2023-07-18T03:19:35.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.361+0000 7f966d9d3b80 1 -- 172.21.15.101:0/2165456543 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f964401e040 con 0x563abca97c60 2023-07-18T03:19:35.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.361+0000 7f9632ffd700 1 -- 172.21.15.101:0/2165456543 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f963c099d30 con 0x563abca97c60 2023-07-18T03:19:35.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.361+0000 7f966d9d3b80 10 client.17994 fetch_fsmap learned FSMap version 33 2023-07-18T03:19:35.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.361+0000 7f966d9d3b80 1 -- 172.21.15.101:0/2165456543 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x563abc817e60 con 0x563abca97c60 2023-07-18T03:19:35.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.361+0000 7f9632ffd700 1 -- 172.21.15.101:0/2165456543 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f963c080080 con 0x563abca97c60 2023-07-18T03:19:35.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.361+0000 7f9632ffd700 1 -- 172.21.15.101:0/2165456543 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f963c099530 con 0x563abca97c60 2023-07-18T03:19:35.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.361+0000 7f966d9d3b80 10 client.17994 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:19:35.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.361+0000 7f966d9d3b80 10 client.17994 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:19:35.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.361+0000 7f966d9d3b80 20 client.17994 populate_metadata read hostname 'smithi101' 2023-07-18T03:19:35.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.361+0000 7f966d9d3b80 1 --2- 172.21.15.101:0/2165456543 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563abcb07d70 0x563abcb0a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:35.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.361+0000 7f966d9d3b80 4 client.17994 mds_command: new command op to 24562 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-18T03:19:35.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.361+0000 7f966d9d3b80 1 -- 172.21.15.101:0/2165456543 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x563abc9a50d0 con 0x563abcb07d70 2023-07-18T03:19:35.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.452+0000 7f96537fe700 1 --2- 172.21.15.101:0/2165456543 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563abcb07d70 0x563abcb0a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:35.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.453+0000 7f96537fe700 1 --2- 172.21.15.101:0/2165456543 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563abcb07d70 0x563abcb0a150 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8aed27000384a00c server_cookie=da6d6a94212e3cf8 in_seq=0 out_seq=0 2023-07-18T03:19:35.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.453+0000 7f9632ffd700 10 client.17994 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:19:35.450 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.455+0000 7f9632ffd700 1 -- 172.21.15.101:0/2165456543 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x563abc9a50d0 con 0x563abcb07d70 2023-07-18T03:19:35.450 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.455+0000 7f9632ffd700 10 client.17994 handle_command_reply: tid=0 2023-07-18T03:19:35.450 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.455+0000 7f966d9d3b80 1 client.17994 shutdown 2023-07-18T03:19:35.450 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.455+0000 7f966d9d3b80 1 -- 172.21.15.101:0/2165456543 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abca97c60 msgr2=0x563abcaa8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:35.450 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.455+0000 7f966d9d3b80 1 --2- 172.21.15.101:0/2165456543 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abca97c60 0x563abcaa8720 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f963c099ef0 tx=0x7f963c01f000 comp rx=0 tx=0).stop 2023-07-18T03:19:35.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.455+0000 7f966d9d3b80 1 -- 172.21.15.101:0/2165456543 shutdown_connections 2023-07-18T03:19:35.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.455+0000 7f966d9d3b80 1 --2- 172.21.15.101:0/2165456543 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563abcb07d70 0x563abcb0a150 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.452 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.456+0000 7f966d9d3b80 1 --2- 172.21.15.101:0/2165456543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563abcaa2b50 0x563abcb00200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.452 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.456+0000 7f966d9d3b80 1 --2- 172.21.15.101:0/2165456543 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563abca97350 0x563abcaa81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.452 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.456+0000 7f966d9d3b80 1 --2- 172.21.15.101:0/2165456543 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abca97c60 0x563abcaa8720 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.452 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.456+0000 7f9632ffd700 0 client.17994 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:19:35.452 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.456+0000 7f966d9d3b80 1 -- 172.21.15.101:0/2165456543 >> 172.21.15.101:0/2165456543 conn(0x563abca8de20 msgr2=0x563abcaa1930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:35.452 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.456+0000 7f966d9d3b80 1 -- 172.21.15.101:0/2165456543 shutdown_connections 2023-07-18T03:19:35.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.456+0000 7f966d9d3b80 1 -- 172.21.15.101:0/2165456543 wait complete. 2023-07-18T03:19:35.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.456+0000 7f966d9d3b80 20 client.17994 trim_cache size 0 max 16384 2023-07-18T03:19:35.453 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:19:35.453 INFO:teuthology.orchestra.run.smithi101.stdout: "return_code": 0, 2023-07-18T03:19:35.453 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_tag": "e113b767-2f2d-4f4f-914c-49d0f26fd987", 2023-07-18T03:19:35.454 INFO:teuthology.orchestra.run.smithi101.stdout: "mode": "asynchronous" 2023-07-18T03:19:35.454 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:19:35.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.457+0000 7f9659fff700 1 -- 172.21.15.101:0/1957683887 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f96400a1340 msgr2=0x7f96400a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:35.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.457+0000 7f9659fff700 1 --2- 172.21.15.101:0/1957683887 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f96400a1340 0x7f96400a37f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f964c00f590 tx=0x7f964c009920 comp rx=0 tx=0).stop 2023-07-18T03:19:35.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.457+0000 7f9659fff700 1 -- 172.21.15.101:0/1957683887 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9654100490 msgr2=0x7f96541982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:35.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.457+0000 7f9659fff700 1 --2- 172.21.15.101:0/1957683887 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9654100490 0x7f96541982f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f963c00e9a0 tx=0x7f963c00ecb0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.457+0000 7f9659fff700 1 -- 172.21.15.101:0/1957683887 shutdown_connections 2023-07-18T03:19:35.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.457+0000 7f9659fff700 1 --2- 172.21.15.101:0/1957683887 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f96400a1340 0x7f96400a37f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.457+0000 7f9659fff700 1 --2- 172.21.15.101:0/1957683887 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9654105af0 0x7f965419c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.457+0000 7f9659fff700 1 --2- 172.21.15.101:0/1957683887 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f96540ffae0 0x7f9654197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.457+0000 7f9659fff700 1 --2- 172.21.15.101:0/1957683887 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9654100490 0x7f96541982f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.457+0000 7f9659fff700 1 -- 172.21.15.101:0/1957683887 >> 172.21.15.101:0/1957683887 conn(0x7f96540fb5e0 msgr2=0x7f965410ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:35.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.458+0000 7f9659fff700 1 -- 172.21.15.101:0/1957683887 shutdown_connections 2023-07-18T03:19:35.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.458+0000 7f9659fff700 1 -- 172.21.15.101:0/1957683887 wait complete. 2023-07-18T03:19:35.457 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:35 smithi179 ceph-mon[129229]: pgmap v375: 129 pgs: 129 active+clean; 586 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 106 op/s 2023-07-18T03:19:35.476 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:35 smithi101 ceph-mon[126379]: pgmap v375: 129 pgs: 129 active+clean; 586 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 106 op/s 2023-07-18T03:19:35.479 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:19:35.626 INFO:tasks.workunit.client.0.smithi101.stdout: 1 720974 5.42 MB/sec execute 540 sec latency 114.821 ms 2023-07-18T03:19:35.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.700+0000 7f86d71ff700 1 -- 172.21.15.101:0/1470081519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86d00fe9d0 msgr2=0x7f86d00fee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:35.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.700+0000 7f86d71ff700 1 --2- 172.21.15.101:0/1470081519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86d00fe9d0 0x7f86d00fee40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f86bc009990 tx=0x7f86bc009ca0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.700+0000 7f86d71ff700 1 -- 172.21.15.101:0/1470081519 shutdown_connections 2023-07-18T03:19:35.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.700+0000 7f86d71ff700 1 --2- 172.21.15.101:0/1470081519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86d01049e0 0x7f86d0104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.700+0000 7f86d71ff700 1 --2- 172.21.15.101:0/1470081519 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f86d00ff380 0x7f86d010b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.700+0000 7f86d71ff700 1 --2- 172.21.15.101:0/1470081519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86d00fe9d0 0x7f86d00fee40 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.700+0000 7f86d71ff700 1 -- 172.21.15.101:0/1470081519 >> 172.21.15.101:0/1470081519 conn(0x7f86d00fa4b0 msgr2=0x7f86d00fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:35.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.700+0000 7f86d71ff700 1 -- 172.21.15.101:0/1470081519 shutdown_connections 2023-07-18T03:19:35.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.700+0000 7f86d71ff700 1 -- 172.21.15.101:0/1470081519 wait complete. 2023-07-18T03:19:35.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.701+0000 7f86d71ff700 1 Processor -- start 2023-07-18T03:19:35.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.701+0000 7f86d71ff700 1 -- start start 2023-07-18T03:19:35.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.702+0000 7f86d71ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f86d00fe9d0 0x7f86d01a0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:35.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.702+0000 7f86d71ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86d00ff380 0x7f86d01a13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:35.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.702+0000 7f86d71ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86d01049e0 0x7f86d019c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:35.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.702+0000 7f86d71ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d019ced0 con 0x7f86d00fe9d0 2023-07-18T03:19:35.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.702+0000 7f86d71ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d019d040 con 0x7f86d01049e0 2023-07-18T03:19:35.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.702+0000 7f86d71ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d019d200 con 0x7f86d00ff380 2023-07-18T03:19:35.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.702+0000 7f86d4f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f86d00fe9d0 0x7f86d01a0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:35.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.702+0000 7f86cffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86d00ff380 0x7f86d01a13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:35.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.702+0000 7f86d4f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f86d00fe9d0 0x7f86d01a0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:38042/0 (socket says 172.21.15.101:38042) 2023-07-18T03:19:35.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.702+0000 7f86d4f9b700 1 -- 172.21.15.101:0/656589297 learned_addr learned my addr 172.21.15.101:0/656589297 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:35.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.702+0000 7f86d579c700 1 --2- 172.21.15.101:0/656589297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86d01049e0 0x7f86d019c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:35.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.702+0000 7f86d4f9b700 1 -- 172.21.15.101:0/656589297 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86d00ff380 msgr2=0x7f86d01a13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:35.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.702+0000 7f86d4f9b700 1 --2- 172.21.15.101:0/656589297 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86d00ff380 0x7f86d01a13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.702+0000 7f86d4f9b700 1 -- 172.21.15.101:0/656589297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86d01049e0 msgr2=0x7f86d019c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:35.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.702+0000 7f86d4f9b700 1 --2- 172.21.15.101:0/656589297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86d01049e0 0x7f86d019c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.702+0000 7f86d4f9b700 1 -- 172.21.15.101:0/656589297 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86bc0095f0 con 0x7f86d00fe9d0 2023-07-18T03:19:35.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.703+0000 7f86d4f9b700 1 --2- 172.21.15.101:0/656589297 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f86d00fe9d0 0x7f86d01a0e60 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f86c400e970 tx=0x7f86c400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:35.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.703+0000 7f86cdffb700 1 -- 172.21.15.101:0/656589297 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c4009790 con 0x7f86d00fe9d0 2023-07-18T03:19:35.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.703+0000 7f86d71ff700 1 -- 172.21.15.101:0/656589297 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86d019d4e0 con 0x7f86d00fe9d0 2023-07-18T03:19:35.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.703+0000 7f86cdffb700 1 -- 172.21.15.101:0/656589297 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86c400c7e0 con 0x7f86d00fe9d0 2023-07-18T03:19:35.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.703+0000 7f86cdffb700 1 -- 172.21.15.101:0/656589297 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c4003c70 con 0x7f86d00fe9d0 2023-07-18T03:19:35.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.703+0000 7f86d71ff700 1 -- 172.21.15.101:0/656589297 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86d01a9010 con 0x7f86d00fe9d0 2023-07-18T03:19:35.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.707+0000 7f86cdffb700 1 -- 172.21.15.101:0/656589297 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f86c400cc50 con 0x7f86d00fe9d0 2023-07-18T03:19:35.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.708+0000 7f86eaa38b80 1 Processor -- start 2023-07-18T03:19:35.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.708+0000 7f86eaa38b80 1 -- start start 2023-07-18T03:19:35.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.709+0000 7f86eaa38b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c321e1c2d0 0x55c321e1c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:35.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.709+0000 7f86eaa38b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c321e1cbe0 0x55c321e27590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:35.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.709+0000 7f86eaa38b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c321e27ad0 0x55c321e29eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:35.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.709+0000 7f86eaa38b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c321d22cb0 con 0x55c321e27ad0 2023-07-18T03:19:35.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.709+0000 7f86eaa38b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c321d75a40 con 0x55c321e1c2d0 2023-07-18T03:19:35.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.709+0000 7f86eaa38b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c321d6eed0 con 0x55c321e1cbe0 2023-07-18T03:19:35.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.709+0000 7f86cdffb700 1 --2- 172.21.15.101:0/656589297 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f86c00a12c0 0x7f86c00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:35.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.709+0000 7f86d4f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c321e1cbe0 0x55c321e27590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:35.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.709+0000 7f86cffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c321e27ad0 0x55c321e29eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:35.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.709+0000 7f86d4f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c321e1cbe0 0x55c321e27590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:44952/0 (socket says 172.21.15.101:44952) 2023-07-18T03:19:35.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.709+0000 7f86d4f9b700 1 -- 172.21.15.101:0/1904313813 learned_addr learned my addr 172.21.15.101:0/1904313813 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:35.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.709+0000 7f86d579c700 1 --2- 172.21.15.101:0/1904313813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c321e1c2d0 0x55c321e1c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:35.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.709+0000 7f86cdffb700 1 -- 172.21.15.101:0/656589297 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f86c4014070 con 0x7f86d00fe9d0 2023-07-18T03:19:35.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.710+0000 7f86d579c700 1 --2- 172.21.15.101:0/656589297 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f86c00a12c0 0x7f86c00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:35.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.710+0000 7f86d4f9b700 1 -- 172.21.15.101:0/1904313813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c321e1c2d0 msgr2=0x55c321e1c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:35.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.710+0000 7f86d4f9b700 1 --2- 172.21.15.101:0/1904313813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c321e1c2d0 0x55c321e1c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.710+0000 7f86d4f9b700 1 -- 172.21.15.101:0/1904313813 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c321e27ad0 msgr2=0x55c321e29eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:35.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.710+0000 7f86d4f9b700 1 --2- 172.21.15.101:0/1904313813 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c321e27ad0 0x55c321e29eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.710+0000 7f86d4f9b700 1 -- 172.21.15.101:0/1904313813 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c321b9cde0 con 0x55c321e1cbe0 2023-07-18T03:19:35.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.710+0000 7f86d4f9b700 1 --2- 172.21.15.101:0/1904313813 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c321e1cbe0 0x55c321e27590 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f86c40ca500 tx=0x7f86c4003710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b708087804aaa39f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:35.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.710+0000 7f86baffd700 1 -- 172.21.15.101:0/1904313813 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c4099870 con 0x55c321e1cbe0 2023-07-18T03:19:35.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.710+0000 7f86d579c700 1 --2- 172.21.15.101:0/656589297 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f86c00a12c0 0x7f86c00a3770 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f86c8009ec0 tx=0x7f86c800b040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:35.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.710+0000 7f86baffd700 1 -- 172.21.15.101:0/1904313813 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86c4003dd0 con 0x55c321e1cbe0 2023-07-18T03:19:35.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.710+0000 7f86baffd700 1 -- 172.21.15.101:0/1904313813 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c4096620 con 0x55c321e1cbe0 2023-07-18T03:19:35.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.711+0000 7f86eaa38b80 1 -- 172.21.15.101:0/1904313813 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c321e1cbe0 msgr2=0x55c321e27590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:35.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.711+0000 7f86eaa38b80 1 --2- 172.21.15.101:0/1904313813 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c321e1cbe0 0x55c321e27590 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f86c40ca500 tx=0x7f86c4003710 comp rx=0 tx=0).stop 2023-07-18T03:19:35.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.711+0000 7f86eaa38b80 1 -- 172.21.15.101:0/1904313813 shutdown_connections 2023-07-18T03:19:35.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.711+0000 7f86eaa38b80 1 --2- 172.21.15.101:0/1904313813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c321e1c2d0 0x55c321e1c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.711+0000 7f86eaa38b80 1 --2- 172.21.15.101:0/1904313813 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c321e27ad0 0x55c321e29eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.711+0000 7f86eaa38b80 1 --2- 172.21.15.101:0/1904313813 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c321e1cbe0 0x55c321e27590 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.711+0000 7f86eaa38b80 1 -- 172.21.15.101:0/1904313813 >> 172.21.15.101:0/1904313813 conn(0x55c321e12da0 msgr2=0x55c321e186a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:35.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.711+0000 7f86eaa38b80 1 -- 172.21.15.101:0/1904313813 shutdown_connections 2023-07-18T03:19:35.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.711+0000 7f86eaa38b80 1 -- 172.21.15.101:0/1904313813 wait complete. 2023-07-18T03:19:35.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.712+0000 7f86eaa38b80 1 -- start start 2023-07-18T03:19:35.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.712+0000 7f86eaa38b80 1 Processor -- start 2023-07-18T03:19:35.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.712+0000 7f86eaa38b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c321e1c2d0 0x55c321e274e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:35.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.712+0000 7f86eaa38b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c321e1cbe0 0x55c321e246f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:35.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.712+0000 7f86eaa38b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c321e27ad0 0x55c321e24d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:35.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.712+0000 7f86eaa38b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c321d0eac0 con 0x55c321e1cbe0 2023-07-18T03:19:35.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.712+0000 7f86eaa38b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c321d0e7c0 con 0x55c321e1c2d0 2023-07-18T03:19:35.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.712+0000 7f86eaa38b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c321bf2920 con 0x55c321e27ad0 2023-07-18T03:19:35.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.712+0000 7f86eaa38b80 10 client.0 _ll_register_callbacks cb 0x55c321d0c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:19:35.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.713+0000 7f86cffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c321e1cbe0 0x55c321e246f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:35.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.713+0000 7f86cffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c321e1cbe0 0x55c321e246f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:38054/0 (socket says 172.21.15.101:38054) 2023-07-18T03:19:35.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.713+0000 7f86d4f9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c321e1c2d0 0x55c321e274e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:35.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.713+0000 7f86cffff700 1 -- 172.21.15.101:0/2025248878 learned_addr learned my addr 172.21.15.101:0/2025248878 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:35.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.713+0000 7f86d579c700 1 --2- 172.21.15.101:0/2025248878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c321e27ad0 0x55c321e24d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:35.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.713+0000 7f86d4f9b700 1 -- 172.21.15.101:0/2025248878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c321e27ad0 msgr2=0x55c321e24d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:35.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.713+0000 7f86d4f9b700 1 --2- 172.21.15.101:0/2025248878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c321e27ad0 0x55c321e24d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.713+0000 7f86d4f9b700 1 -- 172.21.15.101:0/2025248878 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c321e1cbe0 msgr2=0x55c321e246f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:35.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.713+0000 7f86d4f9b700 1 --2- 172.21.15.101:0/2025248878 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c321e1cbe0 0x55c321e246f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.713+0000 7f86d4f9b700 1 -- 172.21.15.101:0/2025248878 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86d01a9010 con 0x55c321e1c2d0 2023-07-18T03:19:35.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.713+0000 7f86d4f9b700 1 --2- 172.21.15.101:0/2025248878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c321e1c2d0 0x55c321e274e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f86c409a0d0 tx=0x7f86c40998e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=807bc7ef7b19676b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:35.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.713+0000 7f86affff700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:19:35.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.714+0000 7f86affff700 1 -- 172.21.15.101:0/2025248878 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c40a2360 con 0x55c321e1c2d0 2023-07-18T03:19:35.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.714+0000 7f86affff700 1 -- 172.21.15.101:0/2025248878 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f86c40cf0d0 con 0x55c321e1c2d0 2023-07-18T03:19:35.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.714+0000 7f86eaa38b80 1 -- 172.21.15.101:0/2025248878 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c321cc84f0 con 0x55c321e1c2d0 2023-07-18T03:19:35.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.714+0000 7f86affff700 1 -- 172.21.15.101:0/2025248878 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86c40cf290 con 0x55c321e1c2d0 2023-07-18T03:19:35.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.714+0000 7f86eaa38b80 10 client.28052 fetch_fsmap learned FSMap version 33 2023-07-18T03:19:35.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.714+0000 7f86eaa38b80 1 -- 172.21.15.101:0/2025248878 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55c321b85c60 con 0x55c321e1c2d0 2023-07-18T03:19:35.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.714+0000 7f86affff700 1 -- 172.21.15.101:0/2025248878 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c321cc84f0 con 0x55c321e1c2d0 2023-07-18T03:19:35.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.714+0000 7f86affff700 1 -- 172.21.15.101:0/2025248878 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f86c4099400 con 0x55c321e1c2d0 2023-07-18T03:19:35.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.714+0000 7f86eaa38b80 10 client.28052 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:19:35.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.714+0000 7f86eaa38b80 10 client.28052 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:19:35.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.714+0000 7f86eaa38b80 20 client.28052 populate_metadata read hostname 'smithi101' 2023-07-18T03:19:35.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.714+0000 7f86eaa38b80 1 --2- 172.21.15.101:0/2025248878 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c321e36fa0 0x55c321e39380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:35.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.714+0000 7f86eaa38b80 4 client.28052 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:19:35.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.714+0000 7f86eaa38b80 1 -- 172.21.15.101:0/2025248878 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c321d67eb0 con 0x55c321e36fa0 2023-07-18T03:19:35.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.715+0000 7f86cffff700 1 --2- 172.21.15.101:0/2025248878 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c321e36fa0 0x55c321e39380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:35.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.715+0000 7f86affff700 10 client.28052 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:19:35.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.715+0000 7f86cffff700 1 --2- 172.21.15.101:0/2025248878 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c321e36fa0 0x55c321e39380 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32059eb155d56a6b server_cookie=484541e054bf0f5 in_seq=0 out_seq=0 2023-07-18T03:19:35.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.716+0000 7f86affff700 1 -- 172.21.15.101:0/2025248878 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c321d67eb0 con 0x55c321e36fa0 2023-07-18T03:19:35.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.716+0000 7f86affff700 10 client.28052 handle_command_reply: tid=0 2023-07-18T03:19:35.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.716+0000 7f86eaa38b80 1 client.28052 shutdown 2023-07-18T03:19:35.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.716+0000 7f86eaa38b80 1 -- 172.21.15.101:0/2025248878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c321e1c2d0 msgr2=0x55c321e274e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:35.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.716+0000 7f86eaa38b80 1 --2- 172.21.15.101:0/2025248878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c321e1c2d0 0x55c321e274e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f86c409a0d0 tx=0x7f86c40998e0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.716+0000 7f86eaa38b80 1 -- 172.21.15.101:0/2025248878 shutdown_connections 2023-07-18T03:19:35.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.716+0000 7f86eaa38b80 1 --2- 172.21.15.101:0/2025248878 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c321e36fa0 0x55c321e39380 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.717+0000 7f86eaa38b80 1 --2- 172.21.15.101:0/2025248878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c321e1c2d0 0x55c321e274e0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.717+0000 7f86eaa38b80 1 --2- 172.21.15.101:0/2025248878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c321e27ad0 0x55c321e24d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.717+0000 7f86eaa38b80 1 --2- 172.21.15.101:0/2025248878 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c321e1cbe0 0x55c321e246f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.717+0000 7f86eaa38b80 1 -- 172.21.15.101:0/2025248878 >> 172.21.15.101:0/2025248878 conn(0x55c321e12da0 msgr2=0x55c321e18230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:35.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.717+0000 7f86affff700 0 client.28052 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:19:35.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.717+0000 7f86eaa38b80 1 -- 172.21.15.101:0/2025248878 shutdown_connections 2023-07-18T03:19:35.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.717+0000 7f86eaa38b80 1 -- 172.21.15.101:0/2025248878 wait complete. 2023-07-18T03:19:35.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.717+0000 7f86eaa38b80 20 client.28052 trim_cache size 0 max 16384 2023-07-18T03:19:35.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.741+0000 7f86eaa38b80 1 Processor -- start 2023-07-18T03:19:35.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.742+0000 7f86eaa38b80 1 -- start start 2023-07-18T03:19:35.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.742+0000 7f86eaa38b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c321e1c2d0 0x55c321e27010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:35.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.742+0000 7f86eaa38b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c321e1cbe0 0x55c321e23f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:35.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.742+0000 7f86eaa38b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c321e27ad0 0x55c321e244d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:35.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.742+0000 7f86eaa38b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c321e1d5b0 con 0x55c321e1c2d0 2023-07-18T03:19:35.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.742+0000 7f86eaa38b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c321e1d190 con 0x55c321e1cbe0 2023-07-18T03:19:35.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.742+0000 7f86eaa38b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c321e17510 con 0x55c321e27ad0 2023-07-18T03:19:35.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.742+0000 7f86d4f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c321e1c2d0 0x55c321e27010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:35.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.742+0000 7f86d4f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c321e1c2d0 0x55c321e27010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:38068/0 (socket says 172.21.15.101:38068) 2023-07-18T03:19:35.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.742+0000 7f86d579c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c321e27ad0 0x55c321e244d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:35.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.742+0000 7f86d4f9b700 1 -- 172.21.15.101:0/1020288595 learned_addr learned my addr 172.21.15.101:0/1020288595 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:35.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.742+0000 7f86cffff700 1 --2- 172.21.15.101:0/1020288595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c321e1cbe0 0x55c321e23f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:35.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.743+0000 7f86d4f9b700 1 -- 172.21.15.101:0/1020288595 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c321e27ad0 msgr2=0x55c321e244d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:35.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.743+0000 7f86d4f9b700 1 --2- 172.21.15.101:0/1020288595 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c321e27ad0 0x55c321e244d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.743+0000 7f86d4f9b700 1 -- 172.21.15.101:0/1020288595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c321e1cbe0 msgr2=0x55c321e23f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:35.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.743+0000 7f86d4f9b700 1 --2- 172.21.15.101:0/1020288595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c321e1cbe0 0x55c321e23f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.743+0000 7f86d4f9b700 1 -- 172.21.15.101:0/1020288595 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c321bc3250 con 0x55c321e1c2d0 2023-07-18T03:19:35.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.743+0000 7f86d4f9b700 1 --2- 172.21.15.101:0/1020288595 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c321e1c2d0 0x55c321e27010 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f86c400efd0 tx=0x7f86c40984c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dac9ab93711b840c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:35.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.743+0000 7f86bb7fe700 1 -- 172.21.15.101:0/1020288595 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c4099a50 con 0x55c321e1c2d0 2023-07-18T03:19:35.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.743+0000 7f86bb7fe700 1 -- 172.21.15.101:0/1020288595 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86c40a1f20 con 0x55c321e1c2d0 2023-07-18T03:19:35.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.743+0000 7f86bb7fe700 1 -- 172.21.15.101:0/1020288595 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c40ca2a0 con 0x55c321e1c2d0 2023-07-18T03:19:35.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.744+0000 7f86eaa38b80 1 -- 172.21.15.101:0/1020288595 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c321e1c2d0 msgr2=0x55c321e27010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:35.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.744+0000 7f86eaa38b80 1 --2- 172.21.15.101:0/1020288595 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c321e1c2d0 0x55c321e27010 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f86c400efd0 tx=0x7f86c40984c0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.744+0000 7f86eaa38b80 1 -- 172.21.15.101:0/1020288595 shutdown_connections 2023-07-18T03:19:35.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.744+0000 7f86eaa38b80 1 --2- 172.21.15.101:0/1020288595 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c321e27ad0 0x55c321e244d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.744+0000 7f86eaa38b80 1 --2- 172.21.15.101:0/1020288595 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c321e1c2d0 0x55c321e27010 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.744+0000 7f86eaa38b80 1 --2- 172.21.15.101:0/1020288595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c321e1cbe0 0x55c321e23f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.744+0000 7f86eaa38b80 1 -- 172.21.15.101:0/1020288595 >> 172.21.15.101:0/1020288595 conn(0x55c321e12da0 msgr2=0x55c321e1f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:35.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.744+0000 7f86eaa38b80 1 -- 172.21.15.101:0/1020288595 shutdown_connections 2023-07-18T03:19:35.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.744+0000 7f86eaa38b80 1 -- 172.21.15.101:0/1020288595 wait complete. 2023-07-18T03:19:35.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.745+0000 7f86eaa38b80 1 -- start start 2023-07-18T03:19:35.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.745+0000 7f86eaa38b80 1 Processor -- start 2023-07-18T03:19:35.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.745+0000 7f86eaa38b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c321e1c2d0 0x55c321e2e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:35.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.745+0000 7f86eaa38b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c321e1cbe0 0x55c321e2c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:35.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.745+0000 7f86eaa38b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c321e27ad0 0x55c321e2cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:35.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.745+0000 7f86eaa38b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c321e2d0a0 con 0x55c321e27ad0 2023-07-18T03:19:35.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.745+0000 7f86eaa38b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c321e2d1e0 con 0x55c321e1cbe0 2023-07-18T03:19:35.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.745+0000 7f86eaa38b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c321e2d3a0 con 0x55c321e1c2d0 2023-07-18T03:19:35.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.745+0000 7f86eaa38b80 10 client.0 _ll_register_callbacks cb 0x7f86c80097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:19:35.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.746+0000 7f86d4f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c321e1c2d0 0x55c321e2e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:35.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.746+0000 7f86d4f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c321e1c2d0 0x55c321e2e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:44992/0 (socket says 172.21.15.101:44992) 2023-07-18T03:19:35.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.746+0000 7f86d4f9b700 1 -- 172.21.15.101:0/2149286568 learned_addr learned my addr 172.21.15.101:0/2149286568 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:19:35.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.746+0000 7f86d579c700 1 --2- 172.21.15.101:0/2149286568 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c321e27ad0 0x55c321e2cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:35.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.746+0000 7f86cffff700 1 --2- 172.21.15.101:0/2149286568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c321e1cbe0 0x55c321e2c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:35.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.746+0000 7f86d4f9b700 1 -- 172.21.15.101:0/2149286568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c321e1cbe0 msgr2=0x55c321e2c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:35.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.746+0000 7f86d4f9b700 1 --2- 172.21.15.101:0/2149286568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c321e1cbe0 0x55c321e2c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.746+0000 7f86d4f9b700 1 -- 172.21.15.101:0/2149286568 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c321e27ad0 msgr2=0x55c321e2cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:19:35.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.746+0000 7f86d4f9b700 1 --2- 172.21.15.101:0/2149286568 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c321e27ad0 0x55c321e2cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.746+0000 7f86d4f9b700 1 -- 172.21.15.101:0/2149286568 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c321b85c60 con 0x55c321e1c2d0 2023-07-18T03:19:35.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.746+0000 7f86d579c700 1 --2- 172.21.15.101:0/2149286568 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c321e27ad0 0x55c321e2cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:19:35.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.746+0000 7f86d4f9b700 1 --2- 172.21.15.101:0/2149286568 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c321e1c2d0 0x55c321e2e2f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f86c4099c10 tx=0x7f86c4096250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa121af48d0abad2 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:19:35.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.746+0000 7f86affff700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:19:35.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.746+0000 7f86affff700 1 -- 172.21.15.101:0/2149286568 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c40bc030 con 0x55c321e1c2d0 2023-07-18T03:19:35.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.746+0000 7f86affff700 1 -- 172.21.15.101:0/2149286568 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f86c40bd070 con 0x55c321e1c2d0 2023-07-18T03:19:35.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.746+0000 7f86eaa38b80 1 -- 172.21.15.101:0/2149286568 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f86c4003dd0 con 0x55c321e1c2d0 2023-07-18T03:19:35.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.747+0000 7f86affff700 1 -- 172.21.15.101:0/2149286568 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86c4096620 con 0x55c321e1c2d0 2023-07-18T03:19:35.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.747+0000 7f86eaa38b80 10 client.28090 fetch_fsmap learned FSMap version 33 2023-07-18T03:19:35.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.747+0000 7f86eaa38b80 1 -- 172.21.15.101:0/2149286568 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55c321b9b2b0 con 0x55c321e1c2d0 2023-07-18T03:19:35.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.747+0000 7f86affff700 1 -- 172.21.15.101:0/2149286568 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86c407c020 con 0x55c321e1c2d0 2023-07-18T03:19:35.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.747+0000 7f86affff700 1 -- 172.21.15.101:0/2149286568 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f86c409a270 con 0x55c321e1c2d0 2023-07-18T03:19:35.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.747+0000 7f86eaa38b80 10 client.28090 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:19:35.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.747+0000 7f86eaa38b80 10 client.28090 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:19:35.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.747+0000 7f86eaa38b80 20 client.28090 populate_metadata read hostname 'smithi101' 2023-07-18T03:19:35.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.747+0000 7f86eaa38b80 1 --2- 172.21.15.101:0/2149286568 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c321e8bdd0 0x55c321e8e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:19:35.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.747+0000 7f86eaa38b80 4 client.28090 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:19:35.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.747+0000 7f86eaa38b80 1 -- 172.21.15.101:0/2149286568 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c321d2a050 con 0x55c321e8bdd0 2023-07-18T03:19:35.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.747+0000 7f86cffff700 1 --2- 172.21.15.101:0/2149286568 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c321e8bdd0 0x55c321e8e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:19:35.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.748+0000 7f86affff700 10 client.28090 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:19:35.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.748+0000 7f86cffff700 1 --2- 172.21.15.101:0/2149286568 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c321e8bdd0 0x55c321e8e1b0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9a4dd5786ef9400 server_cookie=599312cc27237718 in_seq=0 out_seq=0 2023-07-18T03:19:35.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.748+0000 7f86affff700 1 -- 172.21.15.101:0/2149286568 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c321d2a050 con 0x55c321e8bdd0 2023-07-18T03:19:35.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.748+0000 7f86affff700 10 client.28090 handle_command_reply: tid=0 2023-07-18T03:19:35.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.748+0000 7f86eaa38b80 1 client.28090 shutdown 2023-07-18T03:19:35.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.748+0000 7f86eaa38b80 1 -- 172.21.15.101:0/2149286568 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c321e1c2d0 msgr2=0x55c321e2e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:35.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.748+0000 7f86eaa38b80 1 --2- 172.21.15.101:0/2149286568 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c321e1c2d0 0x55c321e2e2f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f86c4099c10 tx=0x7f86c4096250 comp rx=0 tx=0).stop 2023-07-18T03:19:35.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.749+0000 7f86eaa38b80 1 -- 172.21.15.101:0/2149286568 shutdown_connections 2023-07-18T03:19:35.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.749+0000 7f86eaa38b80 1 --2- 172.21.15.101:0/2149286568 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c321e8bdd0 0x55c321e8e1b0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.749+0000 7f86eaa38b80 1 --2- 172.21.15.101:0/2149286568 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c321e27ad0 0x55c321e2cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.749+0000 7f86eaa38b80 1 --2- 172.21.15.101:0/2149286568 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c321e1c2d0 0x55c321e2e2f0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.749+0000 7f86affff700 0 client.28090 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:19:35.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.749+0000 7f86eaa38b80 1 --2- 172.21.15.101:0/2149286568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c321e1cbe0 0x55c321e2c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.749+0000 7f86eaa38b80 1 -- 172.21.15.101:0/2149286568 >> 172.21.15.101:0/2149286568 conn(0x55c321e12da0 msgr2=0x55c321e1f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:35.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.749+0000 7f86eaa38b80 1 -- 172.21.15.101:0/2149286568 shutdown_connections 2023-07-18T03:19:35.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.749+0000 7f86eaa38b80 1 -- 172.21.15.101:0/2149286568 wait complete. 2023-07-18T03:19:35.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.749+0000 7f86eaa38b80 20 client.28090 trim_cache size 0 max 16384 2023-07-18T03:19:35.764 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:19:35.765 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:19:35.765 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:19:35.765 INFO:teuthology.orchestra.run.smithi101.stdout: "e113b767-2f2d-4f4f-914c-49d0f26fd987": { 2023-07-18T03:19:35.765 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:19:35.765 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "e113b767-2f2d-4f4f-914c-49d0f26fd987", 2023-07-18T03:19:35.765 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:19:35.766 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:19:35.766 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:19:35.766 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:19:35.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.750+0000 7f86d71ff700 1 -- 172.21.15.101:0/656589297 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f86c00a12c0 msgr2=0x7f86c00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:35.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.750+0000 7f86d71ff700 1 --2- 172.21.15.101:0/656589297 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f86c00a12c0 0x7f86c00a3770 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f86c8009ec0 tx=0x7f86c800b040 comp rx=0 tx=0).stop 2023-07-18T03:19:35.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.750+0000 7f86d71ff700 1 -- 172.21.15.101:0/656589297 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f86d00fe9d0 msgr2=0x7f86d01a0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:19:35.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.750+0000 7f86d71ff700 1 --2- 172.21.15.101:0/656589297 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f86d00fe9d0 0x7f86d01a0e60 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f86c400e970 tx=0x7f86c400ec80 comp rx=0 tx=0).stop 2023-07-18T03:19:35.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.750+0000 7f86d71ff700 1 -- 172.21.15.101:0/656589297 shutdown_connections 2023-07-18T03:19:35.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.750+0000 7f86d71ff700 1 --2- 172.21.15.101:0/656589297 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f86c00a12c0 0x7f86c00a3770 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.750+0000 7f86d71ff700 1 --2- 172.21.15.101:0/656589297 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86d01049e0 0x7f86d019c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.750+0000 7f86d71ff700 1 --2- 172.21.15.101:0/656589297 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f86d00fe9d0 0x7f86d01a0e60 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.750+0000 7f86d71ff700 1 --2- 172.21.15.101:0/656589297 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86d00ff380 0x7f86d01a13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:19:35.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.750+0000 7f86d71ff700 1 -- 172.21.15.101:0/656589297 >> 172.21.15.101:0/656589297 conn(0x7f86d00fa4b0 msgr2=0x7f86d0067690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:19:35.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.750+0000 7f86d71ff700 1 -- 172.21.15.101:0/656589297 shutdown_connections 2023-07-18T03:19:35.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:19:35.750+0000 7f86d71ff700 1 -- 172.21.15.101:0/656589297 wait complete. 2023-07-18T03:19:35.778 INFO:tasks.cephfs.filesystem:scrub status for tag:e113b767-2f2d-4f4f-914c-49d0f26fd987 - {'path': '/', 'tag': 'e113b767-2f2d-4f4f-914c-49d0f26fd987', 'options': 'recursive,force'} 2023-07-18T03:19:36.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:36 smithi155 ceph-mon[128885]: scrub queued for path: / 2023-07-18T03:19:36.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:36 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:19:36.298 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:36 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:19:36.298 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:36 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:19:36.298 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:36 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:19:36.299 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:36 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:19:36.299 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:36 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:19:36.299 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:36 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:19:36.299 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:36 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:19:36.300 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:36 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:19:36.300 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:36 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:19:36.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:36 smithi179 ceph-mon[129229]: scrub queued for path: / 2023-07-18T03:19:36.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:36 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:19:36.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:36 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:19:36.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:36 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:19:36.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:36 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:19:36.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:36 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:19:36.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:36 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:19:36.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:36 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:19:36.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:36 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:19:36.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:36 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:19:36.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:36 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:19:36.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:36 smithi101 ceph-mon[126379]: scrub queued for path: / 2023-07-18T03:19:36.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:36 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:19:36.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:36 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:19:36.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:36 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:19:36.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:36 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:19:36.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:36 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:19:36.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:36 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:19:36.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:36 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:19:36.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:36 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:19:36.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:36 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:19:36.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:36 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:19:36.626 INFO:tasks.workunit.client.0.smithi101.stdout: 1 721835 5.42 MB/sec execute 541 sec latency 25.331 ms 2023-07-18T03:19:37.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:37 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:19:37.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:37 smithi155 ceph-mon[128885]: pgmap v376: 129 pgs: 129 active+clean; 592 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.3 MiB/s wr, 159 op/s 2023-07-18T03:19:37.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:37 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:19:37.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:37 smithi179 ceph-mon[129229]: pgmap v376: 129 pgs: 129 active+clean; 592 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.3 MiB/s wr, 159 op/s 2023-07-18T03:19:37.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:37 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:19:37.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:37 smithi101 ceph-mon[126379]: pgmap v376: 129 pgs: 129 active+clean; 592 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.3 MiB/s wr, 159 op/s 2023-07-18T03:19:37.626 INFO:tasks.workunit.client.0.smithi101.stdout: 1 722548 5.41 MB/sec execute 542 sec latency 521.316 ms 2023-07-18T03:19:38.626 INFO:tasks.workunit.client.0.smithi101.stdout: 1 722548 5.40 MB/sec execute 543 sec latency 1521.434 ms 2023-07-18T03:19:39.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:39 smithi179 ceph-mon[129229]: pgmap v377: 129 pgs: 129 active+clean; 592 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.6 MiB/s wr, 107 op/s 2023-07-18T03:19:39.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:39 smithi101 ceph-mon[126379]: pgmap v377: 129 pgs: 129 active+clean; 592 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.6 MiB/s wr, 107 op/s 2023-07-18T03:19:39.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:39 smithi155 ceph-mon[128885]: pgmap v377: 129 pgs: 129 active+clean; 592 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.6 MiB/s wr, 107 op/s 2023-07-18T03:19:39.626 INFO:tasks.workunit.client.0.smithi101.stdout: 1 722948 5.40 MB/sec execute 544 sec latency 2341.510 ms 2023-07-18T03:19:40.626 INFO:tasks.workunit.client.0.smithi101.stdout: 1 724564 5.41 MB/sec execute 545 sec latency 43.502 ms 2023-07-18T03:19:41.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:41 smithi179 ceph-mon[129229]: pgmap v378: 129 pgs: 129 active+clean; 597 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.2 MiB/s wr, 157 op/s 2023-07-18T03:19:41.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:41 smithi101 ceph-mon[126379]: pgmap v378: 129 pgs: 129 active+clean; 597 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.2 MiB/s wr, 157 op/s 2023-07-18T03:19:41.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:41 smithi155 ceph-mon[128885]: pgmap v378: 129 pgs: 129 active+clean; 597 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.2 MiB/s wr, 157 op/s 2023-07-18T03:19:41.626 INFO:tasks.workunit.client.0.smithi101.stdout: 1 725442 5.40 MB/sec execute 546 sec latency 17.623 ms 2023-07-18T03:19:42.626 INFO:tasks.workunit.client.0.smithi101.stdout: 1 726096 5.40 MB/sec execute 547 sec latency 574.812 ms 2023-07-18T03:19:43.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:43 smithi179 ceph-mon[129229]: pgmap v379: 129 pgs: 129 active+clean; 597 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 102 op/s 2023-07-18T03:19:43.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:43 smithi101 ceph-mon[126379]: pgmap v379: 129 pgs: 129 active+clean; 597 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 102 op/s 2023-07-18T03:19:43.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:43 smithi155 ceph-mon[128885]: pgmap v379: 129 pgs: 129 active+clean; 597 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 102 op/s 2023-07-18T03:19:43.626 INFO:tasks.workunit.client.0.smithi101.stdout: 1 726096 5.39 MB/sec execute 548 sec latency 1574.952 ms 2023-07-18T03:19:44.627 INFO:tasks.workunit.client.0.smithi101.stdout: 1 726495 5.38 MB/sec execute 549 sec latency 2391.684 ms 2023-07-18T03:19:45.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:45 smithi179 ceph-mon[129229]: pgmap v380: 129 pgs: 129 active+clean; 597 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 102 op/s 2023-07-18T03:19:45.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:45 smithi101 ceph-mon[126379]: pgmap v380: 129 pgs: 129 active+clean; 597 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 102 op/s 2023-07-18T03:19:45.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:45 smithi155 ceph-mon[128885]: pgmap v380: 129 pgs: 129 active+clean; 597 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 102 op/s 2023-07-18T03:19:45.627 INFO:tasks.workunit.client.0.smithi101.stdout: 1 728161 5.39 MB/sec execute 550 sec latency 30.790 ms 2023-07-18T03:19:46.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:46 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:19:46.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:46 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:19:46.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:46 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:19:46.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:46 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:19:46.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:46 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:19:46.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:46 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:19:46.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:46 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:19:46.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:46 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:19:46.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:46 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:19:46.627 INFO:tasks.workunit.client.0.smithi101.stdout: 1 729018 5.39 MB/sec execute 551 sec latency 21.823 ms 2023-07-18T03:19:47.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:47 smithi179 ceph-mon[129229]: pgmap v381: 129 pgs: 129 active+clean; 602 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 155 op/s 2023-07-18T03:19:47.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:47 smithi101 ceph-mon[126379]: pgmap v381: 129 pgs: 129 active+clean; 602 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 155 op/s 2023-07-18T03:19:47.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:47 smithi155 ceph-mon[128885]: pgmap v381: 129 pgs: 129 active+clean; 602 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 155 op/s 2023-07-18T03:19:47.627 INFO:tasks.workunit.client.0.smithi101.stdout: 1 729638 5.39 MB/sec execute 552 sec latency 583.083 ms 2023-07-18T03:19:48.627 INFO:tasks.workunit.client.0.smithi101.stdout: 1 729638 5.38 MB/sec execute 553 sec latency 1583.172 ms 2023-07-18T03:19:49.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:49 smithi179 ceph-mon[129229]: pgmap v382: 129 pgs: 129 active+clean; 602 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 102 op/s 2023-07-18T03:19:49.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:49 smithi101 ceph-mon[126379]: pgmap v382: 129 pgs: 129 active+clean; 602 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 102 op/s 2023-07-18T03:19:49.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:49 smithi155 ceph-mon[128885]: pgmap v382: 129 pgs: 129 active+clean; 602 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 102 op/s 2023-07-18T03:19:49.627 INFO:tasks.workunit.client.0.smithi101.stdout: 1 729789 5.37 MB/sec execute 554 sec latency 2404.965 ms 2023-07-18T03:19:50.627 INFO:tasks.workunit.client.0.smithi101.stdout: 1 731309 5.37 MB/sec execute 555 sec latency 42.439 ms 2023-07-18T03:19:51.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:51 smithi179 ceph-mon[129229]: pgmap v383: 129 pgs: 129 active+clean; 607 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 150 op/s 2023-07-18T03:19:51.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:51 smithi101 ceph-mon[126379]: pgmap v383: 129 pgs: 129 active+clean; 607 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 150 op/s 2023-07-18T03:19:51.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:51 smithi155 ceph-mon[128885]: pgmap v383: 129 pgs: 129 active+clean; 607 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 150 op/s 2023-07-18T03:19:51.627 INFO:tasks.workunit.client.0.smithi101.stdout: 1 732296 5.37 MB/sec execute 556 sec latency 23.583 ms 2023-07-18T03:19:52.627 INFO:tasks.workunit.client.0.smithi101.stdout: 1 733187 5.37 MB/sec execute 557 sec latency 394.821 ms 2023-07-18T03:19:53.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:53 smithi179 ceph-mon[129229]: pgmap v384: 129 pgs: 129 active+clean; 607 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 1.6 MiB/s wr, 100 op/s 2023-07-18T03:19:53.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:53 smithi101 ceph-mon[126379]: pgmap v384: 129 pgs: 129 active+clean; 607 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 1.6 MiB/s wr, 100 op/s 2023-07-18T03:19:53.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:53 smithi155 ceph-mon[128885]: pgmap v384: 129 pgs: 129 active+clean; 607 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 1.6 MiB/s wr, 100 op/s 2023-07-18T03:19:53.627 INFO:tasks.workunit.client.0.smithi101.stdout: 1 733187 5.36 MB/sec execute 558 sec latency 1394.957 ms 2023-07-18T03:19:54.627 INFO:tasks.workunit.client.0.smithi101.stdout: 1 733637 5.35 MB/sec execute 559 sec latency 2200.458 ms 2023-07-18T03:19:55.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:55 smithi179 ceph-mon[129229]: pgmap v385: 129 pgs: 129 active+clean; 607 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 1.6 MiB/s wr, 100 op/s 2023-07-18T03:19:55.494 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:55 smithi101 ceph-mon[126379]: pgmap v385: 129 pgs: 129 active+clean; 607 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 1.6 MiB/s wr, 100 op/s 2023-07-18T03:19:55.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:55 smithi155 ceph-mon[128885]: pgmap v385: 129 pgs: 129 active+clean; 607 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 1.6 MiB/s wr, 100 op/s 2023-07-18T03:19:55.627 INFO:tasks.workunit.client.0.smithi101.stdout: 1 735254 5.37 MB/sec execute 560 sec latency 32.036 ms 2023-07-18T03:19:56.627 INFO:tasks.workunit.client.0.smithi101.stdout: 1 736163 5.36 MB/sec execute 561 sec latency 29.933 ms 2023-07-18T03:19:57.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:57 smithi179 ceph-mon[129229]: pgmap v386: 129 pgs: 129 active+clean; 612 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 151 op/s 2023-07-18T03:19:57.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:57 smithi101 ceph-mon[126379]: pgmap v386: 129 pgs: 129 active+clean; 612 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 151 op/s 2023-07-18T03:19:57.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:57 smithi155 ceph-mon[128885]: pgmap v386: 129 pgs: 129 active+clean; 612 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 151 op/s 2023-07-18T03:19:57.628 INFO:tasks.workunit.client.0.smithi101.stdout: 1 736729 5.36 MB/sec execute 562 sec latency 629.862 ms 2023-07-18T03:19:58.628 INFO:tasks.workunit.client.0.smithi101.stdout: 1 736729 5.35 MB/sec execute 563 sec latency 1629.899 ms 2023-07-18T03:19:59.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:59 smithi179 ceph-mon[129229]: pgmap v387: 129 pgs: 129 active+clean; 612 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 99 op/s 2023-07-18T03:19:59.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:59 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:19:59.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:59 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:19:59.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:59 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:19:59.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:59 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:19:59.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:19:59 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:19:59.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:59 smithi101 ceph-mon[126379]: pgmap v387: 129 pgs: 129 active+clean; 612 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 99 op/s 2023-07-18T03:19:59.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:59 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:19:59.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:59 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:19:59.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:59 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:19:59.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:59 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:19:59.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:19:59 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:19:59.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:59 smithi155 ceph-mon[128885]: pgmap v387: 129 pgs: 129 active+clean; 612 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 99 op/s 2023-07-18T03:19:59.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:59 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:19:59.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:59 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:19:59.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:59 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:19:59.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:59 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:19:59.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:19:59 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:19:59.628 INFO:tasks.workunit.client.0.smithi101.stdout: 1 737214 5.34 MB/sec execute 564 sec latency 2435.383 ms 2023-07-18T03:20:00.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:00 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:20:00.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:00 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:20:00.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:00 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:20:00.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:00 smithi179 ceph-mon[129229]: overall HEALTH_OK 2023-07-18T03:20:00.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:00 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:20:00.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:00 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:20:00.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:00 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:20:00.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:00 smithi101 ceph-mon[126379]: overall HEALTH_OK 2023-07-18T03:20:00.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:00 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:20:00.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:00 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:20:00.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:00 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:20:00.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:00 smithi155 ceph-mon[128885]: overall HEALTH_OK 2023-07-18T03:20:00.628 INFO:tasks.workunit.client.0.smithi101.stdout: 1 738818 5.35 MB/sec execute 565 sec latency 41.883 ms 2023-07-18T03:20:01.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:01 smithi179 ceph-mon[129229]: pgmap v388: 129 pgs: 129 active+clean; 617 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 149 op/s 2023-07-18T03:20:01.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:01 smithi101 ceph-mon[126379]: pgmap v388: 129 pgs: 129 active+clean; 617 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 149 op/s 2023-07-18T03:20:01.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:01 smithi155 ceph-mon[128885]: pgmap v388: 129 pgs: 129 active+clean; 617 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 149 op/s 2023-07-18T03:20:01.628 INFO:tasks.workunit.client.0.smithi101.stdout: 1 738977 5.34 MB/sec execute 566 sec latency 709.426 ms 2023-07-18T03:20:02.628 INFO:tasks.workunit.client.0.smithi101.stdout: 1 738977 5.33 MB/sec execute 567 sec latency 1709.556 ms 2023-07-18T03:20:03.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:03 smithi179 ceph-mon[129229]: pgmap v389: 129 pgs: 129 active+clean; 617 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 102 op/s 2023-07-18T03:20:03.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:03 smithi101 ceph-mon[126379]: pgmap v389: 129 pgs: 129 active+clean; 617 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 102 op/s 2023-07-18T03:20:03.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:03 smithi155 ceph-mon[128885]: pgmap v389: 129 pgs: 129 active+clean; 617 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 102 op/s 2023-07-18T03:20:03.628 INFO:tasks.workunit.client.0.smithi101.stdout: 1 738977 5.33 MB/sec execute 568 sec latency 2709.681 ms 2023-07-18T03:20:04.628 INFO:tasks.workunit.client.0.smithi101.stdout: 1 738977 5.32 MB/sec execute 569 sec latency 3709.749 ms 2023-07-18T03:20:05.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:05 smithi179 ceph-mon[129229]: pgmap v390: 129 pgs: 129 active+clean; 617 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 102 op/s 2023-07-18T03:20:05.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:05 smithi101 ceph-mon[126379]: pgmap v390: 129 pgs: 129 active+clean; 617 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 102 op/s 2023-07-18T03:20:05.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:05 smithi155 ceph-mon[128885]: pgmap v390: 129 pgs: 129 active+clean; 617 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 102 op/s 2023-07-18T03:20:05.628 INFO:tasks.workunit.client.0.smithi101.stdout: 1 738977 5.31 MB/sec execute 570 sec latency 4709.879 ms 2023-07-18T03:20:05.780 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:20:06.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.019+0000 7fc55cdff700 1 -- 172.21.15.101:0/3890510001 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc55810c3d0 msgr2=0x7fc55810e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:06.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.019+0000 7fc55cdff700 1 --2- 172.21.15.101:0/3890510001 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc55810c3d0 0x7fc55810e7b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc55000b0a0 tx=0x7fc55000b3b0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.019+0000 7fc55cdff700 1 -- 172.21.15.101:0/3890510001 shutdown_connections 2023-07-18T03:20:06.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.019+0000 7fc55cdff700 1 --2- 172.21.15.101:0/3890510001 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc55810c3d0 0x7fc55810e7b0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.019+0000 7fc55cdff700 1 --2- 172.21.15.101:0/3890510001 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc5580fe820 0x7fc5580febf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.019+0000 7fc55cdff700 1 --2- 172.21.15.101:0/3890510001 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc5580ff1c0 0x7fc55810be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.019+0000 7fc55cdff700 1 -- 172.21.15.101:0/3890510001 >> 172.21.15.101:0/3890510001 conn(0x7fc5580fa510 msgr2=0x7fc5580fc920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:06.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.019+0000 7fc55cdff700 1 -- 172.21.15.101:0/3890510001 shutdown_connections 2023-07-18T03:20:06.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.019+0000 7fc55cdff700 1 -- 172.21.15.101:0/3890510001 wait complete. 2023-07-18T03:20:06.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.020+0000 7fc55cdff700 1 Processor -- start 2023-07-18T03:20:06.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.020+0000 7fc55cdff700 1 -- start start 2023-07-18T03:20:06.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.021+0000 7fc55cdff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc5580fe820 0x7fc558197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:06.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.021+0000 7fc55cdff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc5580ff1c0 0x7fc558198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:06.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.021+0000 7fc55cdff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc55810c3d0 0x7fc55819c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:06.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.021+0000 7fc55cdff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55819cfa0 con 0x7fc5580ff1c0 2023-07-18T03:20:06.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.021+0000 7fc55cdff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55819d110 con 0x7fc55810c3d0 2023-07-18T03:20:06.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.021+0000 7fc55cdff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55819d2d0 con 0x7fc5580fe820 2023-07-18T03:20:06.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.021+0000 7fc55659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc5580fe820 0x7fc558197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:06.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.021+0000 7fc55659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc5580fe820 0x7fc558197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:38428/0 (socket says 172.21.15.101:38428) 2023-07-18T03:20:06.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.021+0000 7fc55659c700 1 -- 172.21.15.101:0/1543866565 learned_addr learned my addr 172.21.15.101:0/1543866565 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:06.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.021+0000 7fc555d9b700 1 --2- 172.21.15.101:0/1543866565 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc5580ff1c0 0x7fc558198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:06.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.021+0000 7fc556d9d700 1 --2- 172.21.15.101:0/1543866565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc55810c3d0 0x7fc55819c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:06.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.021+0000 7fc55659c700 1 -- 172.21.15.101:0/1543866565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc55810c3d0 msgr2=0x7fc55819c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:06.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.021+0000 7fc55659c700 1 --2- 172.21.15.101:0/1543866565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc55810c3d0 0x7fc55819c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.021+0000 7fc55659c700 1 -- 172.21.15.101:0/1543866565 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc5580ff1c0 msgr2=0x7fc558198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:06.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.021+0000 7fc55659c700 1 --2- 172.21.15.101:0/1543866565 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc5580ff1c0 0x7fc558198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.021+0000 7fc55659c700 1 -- 172.21.15.101:0/1543866565 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc550009d00 con 0x7fc5580fe820 2023-07-18T03:20:06.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.022+0000 7fc55659c700 1 --2- 172.21.15.101:0/1543866565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc5580fe820 0x7fc558197fc0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc54800e9a0 tx=0x7fc54800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:06.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.022+0000 7fc5477fe700 1 -- 172.21.15.101:0/1543866565 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc548009790 con 0x7fc5580fe820 2023-07-18T03:20:06.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.022+0000 7fc5477fe700 1 -- 172.21.15.101:0/1543866565 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc548010430 con 0x7fc5580fe820 2023-07-18T03:20:06.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.022+0000 7fc5477fe700 1 -- 172.21.15.101:0/1543866565 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc548018680 con 0x7fc5580fe820 2023-07-18T03:20:06.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.022+0000 7fc55cdff700 1 -- 172.21.15.101:0/1543866565 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc55819d5b0 con 0x7fc5580fe820 2023-07-18T03:20:06.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.022+0000 7fc55cdff700 1 -- 172.21.15.101:0/1543866565 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5581a4df0 con 0x7fc5580fe820 2023-07-18T03:20:06.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.025+0000 7fc5477fe700 1 -- 172.21.15.101:0/1543866565 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fc5480105a0 con 0x7fc5580fe820 2023-07-18T03:20:06.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.026+0000 7fc57069bb80 1 Processor -- start 2023-07-18T03:20:06.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.027+0000 7fc57069bb80 1 -- start start 2023-07-18T03:20:06.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.027+0000 7fc5477fe700 1 --2- 172.21.15.101:0/1543866565 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc5400a1340 0x7fc5400a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:06.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.027+0000 7fc57069bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610b4a782d0 0x5610b4a786a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:06.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.027+0000 7fc57069bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5610b4a78be0 0x5610b4a83590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:06.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.027+0000 7fc57069bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610b4a83ad0 0x5610b4a85eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:06.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.027+0000 7fc5477fe700 1 -- 172.21.15.101:0/1543866565 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fc548014070 con 0x7fc5580fe820 2023-07-18T03:20:06.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.027+0000 7fc57069bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610b497ecb0 con 0x5610b4a78be0 2023-07-18T03:20:06.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.027+0000 7fc57069bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610b49d1a40 con 0x5610b4a83ad0 2023-07-18T03:20:06.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.027+0000 7fc57069bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610b49caed0 con 0x5610b4a782d0 2023-07-18T03:20:06.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.027+0000 7fc556d9d700 1 --2- 172.21.15.101:0/1543866565 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc5400a1340 0x7fc5400a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:06.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.027+0000 7fc55659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610b4a782d0 0x5610b4a786a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:06.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.027+0000 7fc55659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610b4a782d0 0x5610b4a786a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:38432/0 (socket says 172.21.15.101:38432) 2023-07-18T03:20:06.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.027+0000 7fc55659c700 1 -- 172.21.15.101:0/1430234086 learned_addr learned my addr 172.21.15.101:0/1430234086 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:06.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.027+0000 7fc555d9b700 1 --2- 172.21.15.101:0/1430234086 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5610b4a78be0 0x5610b4a83590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:06.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.027+0000 7fc556d9d700 1 --2- 172.21.15.101:0/1430234086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610b4a83ad0 0x5610b4a85eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:06.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.028+0000 7fc55659c700 1 -- 172.21.15.101:0/1430234086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610b4a83ad0 msgr2=0x5610b4a85eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:06.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.028+0000 7fc55659c700 1 --2- 172.21.15.101:0/1430234086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610b4a83ad0 0x5610b4a85eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.028+0000 7fc55659c700 1 -- 172.21.15.101:0/1430234086 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5610b4a78be0 msgr2=0x5610b4a83590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:06.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.028+0000 7fc55659c700 1 --2- 172.21.15.101:0/1430234086 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5610b4a78be0 0x5610b4a83590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.028+0000 7fc55659c700 1 -- 172.21.15.101:0/1430234086 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610b47f8de0 con 0x5610b4a782d0 2023-07-18T03:20:06.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.028+0000 7fc555d9b700 1 --2- 172.21.15.101:0/1430234086 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5610b4a78be0 0x5610b4a83590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:20:06.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.028+0000 7fc55659c700 1 --2- 172.21.15.101:0/1430234086 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610b4a782d0 0x5610b4a786a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc5480969d0 tx=0x7fc5480963c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95e381e53f3e6222 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:06.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.028+0000 7fc556d9d700 1 --2- 172.21.15.101:0/1543866565 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc5400a1340 0x7fc5400a37f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc5500062a0 tx=0x7fc55000b720 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:06.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.028+0000 7fc544ff9700 1 -- 172.21.15.101:0/1430234086 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc548099f60 con 0x5610b4a782d0 2023-07-18T03:20:06.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.028+0000 7fc544ff9700 1 -- 172.21.15.101:0/1430234086 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc54809df90 con 0x5610b4a782d0 2023-07-18T03:20:06.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.028+0000 7fc544ff9700 1 -- 172.21.15.101:0/1430234086 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5480556c0 con 0x5610b4a782d0 2023-07-18T03:20:06.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.029+0000 7fc57069bb80 1 -- 172.21.15.101:0/1430234086 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610b4a782d0 msgr2=0x5610b4a786a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:06.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.029+0000 7fc57069bb80 1 --2- 172.21.15.101:0/1430234086 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610b4a782d0 0x5610b4a786a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc5480969d0 tx=0x7fc5480963c0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.029+0000 7fc57069bb80 1 -- 172.21.15.101:0/1430234086 shutdown_connections 2023-07-18T03:20:06.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.029+0000 7fc57069bb80 1 --2- 172.21.15.101:0/1430234086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610b4a83ad0 0x5610b4a85eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.029+0000 7fc57069bb80 1 --2- 172.21.15.101:0/1430234086 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610b4a782d0 0x5610b4a786a0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.029+0000 7fc57069bb80 1 --2- 172.21.15.101:0/1430234086 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5610b4a78be0 0x5610b4a83590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.029+0000 7fc57069bb80 1 -- 172.21.15.101:0/1430234086 >> 172.21.15.101:0/1430234086 conn(0x5610b4a6eda0 msgr2=0x5610b4a746a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:06.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.029+0000 7fc57069bb80 1 -- 172.21.15.101:0/1430234086 shutdown_connections 2023-07-18T03:20:06.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.029+0000 7fc57069bb80 1 -- 172.21.15.101:0/1430234086 wait complete. 2023-07-18T03:20:06.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.029+0000 7fc57069bb80 1 -- start start 2023-07-18T03:20:06.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.030+0000 7fc57069bb80 1 Processor -- start 2023-07-18T03:20:06.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.030+0000 7fc57069bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610b4a782d0 0x5610b4a83510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:06.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.030+0000 7fc57069bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610b4a78be0 0x5610b4a80800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:06.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.030+0000 7fc57069bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5610b4a83ad0 0x5610b4a80f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:06.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.030+0000 7fc57069bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610b496aac0 con 0x5610b4a83ad0 2023-07-18T03:20:06.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.030+0000 7fc57069bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610b496a7c0 con 0x5610b4a78be0 2023-07-18T03:20:06.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.030+0000 7fc57069bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610b484e920 con 0x5610b4a782d0 2023-07-18T03:20:06.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.030+0000 7fc57069bb80 10 client.0 _ll_register_callbacks cb 0x5610b4968760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:20:06.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.031+0000 7fc555d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610b4a78be0 0x5610b4a80800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:06.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.031+0000 7fc555d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610b4a78be0 0x5610b4a80800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:43098/0 (socket says 172.21.15.101:43098) 2023-07-18T03:20:06.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.031+0000 7fc555d9b700 1 -- 172.21.15.101:0/1215263063 learned_addr learned my addr 172.21.15.101:0/1215263063 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:06.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.031+0000 7fc556d9d700 1 --2- 172.21.15.101:0/1215263063 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5610b4a83ad0 0x5610b4a80f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:06.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.031+0000 7fc55659c700 1 --2- 172.21.15.101:0/1215263063 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610b4a782d0 0x5610b4a83510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:06.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.031+0000 7fc555d9b700 1 -- 172.21.15.101:0/1215263063 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610b4a782d0 msgr2=0x5610b4a83510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:06.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.031+0000 7fc555d9b700 1 --2- 172.21.15.101:0/1215263063 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610b4a782d0 0x5610b4a83510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.031+0000 7fc555d9b700 1 -- 172.21.15.101:0/1215263063 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5610b4a83ad0 msgr2=0x5610b4a80f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:06.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.031+0000 7fc555d9b700 1 --2- 172.21.15.101:0/1215263063 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5610b4a83ad0 0x5610b4a80f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.031+0000 7fc555d9b700 1 -- 172.21.15.101:0/1215263063 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5581a4df0 con 0x5610b4a78be0 2023-07-18T03:20:06.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.031+0000 7fc555d9b700 1 --2- 172.21.15.101:0/1215263063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610b4a78be0 0x5610b4a80800 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc54c0116f0 tx=0x7fc54c011a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edf645e2cef722d8 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:06.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.031+0000 7fc53dffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:20:06.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.031+0000 7fc53dffb700 1 -- 172.21.15.101:0/1215263063 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c016070 con 0x5610b4a78be0 2023-07-18T03:20:06.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.032+0000 7fc57069bb80 1 -- 172.21.15.101:0/1215263063 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5610b49244f0 con 0x5610b4a78be0 2023-07-18T03:20:06.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.032+0000 7fc53dffb700 1 -- 172.21.15.101:0/1215263063 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fc54c00f690 con 0x5610b4a78be0 2023-07-18T03:20:06.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.032+0000 7fc53dffb700 1 -- 172.21.15.101:0/1215263063 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc54c00f850 con 0x5610b4a78be0 2023-07-18T03:20:06.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.032+0000 7fc57069bb80 10 client.28070 fetch_fsmap learned FSMap version 33 2023-07-18T03:20:06.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.032+0000 7fc57069bb80 1 -- 172.21.15.101:0/1215263063 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5610b47e1c60 con 0x5610b4a78be0 2023-07-18T03:20:06.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.032+0000 7fc53dffb700 1 -- 172.21.15.101:0/1215263063 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5610b49244f0 con 0x5610b4a78be0 2023-07-18T03:20:06.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.032+0000 7fc53dffb700 1 -- 172.21.15.101:0/1215263063 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc54c01a9f0 con 0x5610b4a78be0 2023-07-18T03:20:06.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.032+0000 7fc57069bb80 10 client.28070 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:20:06.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.032+0000 7fc57069bb80 10 client.28070 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:20:06.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.032+0000 7fc57069bb80 20 client.28070 populate_metadata read hostname 'smithi101' 2023-07-18T03:20:06.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.032+0000 7fc57069bb80 1 --2- 172.21.15.101:0/1215263063 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5610b4a93400 0x5610b4a95810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:06.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.032+0000 7fc57069bb80 4 client.28070 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:20:06.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.032+0000 7fc57069bb80 1 -- 172.21.15.101:0/1215263063 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5610b49c3eb0 con 0x5610b4a93400 2023-07-18T03:20:06.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.033+0000 7fc55659c700 1 --2- 172.21.15.101:0/1215263063 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5610b4a93400 0x5610b4a95810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:06.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.033+0000 7fc53dffb700 10 client.28070 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:20:06.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.033+0000 7fc55659c700 1 --2- 172.21.15.101:0/1215263063 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5610b4a93400 0x5610b4a95810 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ea5b34b66a7407a server_cookie=639a065d1bc91c4b in_seq=0 out_seq=0 2023-07-18T03:20:06.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.034+0000 7fc53dffb700 1 -- 172.21.15.101:0/1215263063 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5610b49c3eb0 con 0x5610b4a93400 2023-07-18T03:20:06.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.034+0000 7fc53dffb700 10 client.28070 handle_command_reply: tid=0 2023-07-18T03:20:06.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.034+0000 7fc57069bb80 1 client.28070 shutdown 2023-07-18T03:20:06.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.034+0000 7fc57069bb80 1 -- 172.21.15.101:0/1215263063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610b4a78be0 msgr2=0x5610b4a80800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:06.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.034+0000 7fc57069bb80 1 --2- 172.21.15.101:0/1215263063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610b4a78be0 0x5610b4a80800 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc54c0116f0 tx=0x7fc54c011a00 comp rx=0 tx=0).stop 2023-07-18T03:20:06.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.034+0000 7fc57069bb80 1 -- 172.21.15.101:0/1215263063 shutdown_connections 2023-07-18T03:20:06.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.034+0000 7fc57069bb80 1 --2- 172.21.15.101:0/1215263063 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5610b4a93400 0x5610b4a95810 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.034+0000 7fc57069bb80 1 --2- 172.21.15.101:0/1215263063 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5610b4a83ad0 0x5610b4a80f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.034+0000 7fc57069bb80 1 --2- 172.21.15.101:0/1215263063 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610b4a782d0 0x5610b4a83510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.034+0000 7fc57069bb80 1 --2- 172.21.15.101:0/1215263063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610b4a78be0 0x5610b4a80800 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.034+0000 7fc53dffb700 0 client.28070 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:20:06.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.034+0000 7fc57069bb80 1 -- 172.21.15.101:0/1215263063 >> 172.21.15.101:0/1215263063 conn(0x5610b4a6eda0 msgr2=0x5610b4a74230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:06.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.034+0000 7fc57069bb80 1 -- 172.21.15.101:0/1215263063 shutdown_connections 2023-07-18T03:20:06.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.034+0000 7fc57069bb80 1 -- 172.21.15.101:0/1215263063 wait complete. 2023-07-18T03:20:06.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.034+0000 7fc57069bb80 20 client.28070 trim_cache size 0 max 16384 2023-07-18T03:20:06.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.052+0000 7fc57069bb80 1 Processor -- start 2023-07-18T03:20:06.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.052+0000 7fc57069bb80 1 -- start start 2023-07-18T03:20:06.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.053+0000 7fc57069bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610b4a782d0 0x5610b4a8a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:06.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.053+0000 7fc57069bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5610b4a78be0 0x5610b4a8ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:06.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.053+0000 7fc57069bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610b4a83ad0 0x5610b4a7bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:06.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.053+0000 7fc57069bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610b4a79540 con 0x5610b4a78be0 2023-07-18T03:20:06.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.053+0000 7fc57069bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610b4a7a1a0 con 0x5610b4a782d0 2023-07-18T03:20:06.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.053+0000 7fc57069bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610b4a789d0 con 0x5610b4a83ad0 2023-07-18T03:20:06.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.053+0000 7fc555d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5610b4a78be0 0x5610b4a8ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:06.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.053+0000 7fc55659c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610b4a782d0 0x5610b4a8a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:06.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.053+0000 7fc555d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5610b4a78be0 0x5610b4a8ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:54454/0 (socket says 172.21.15.101:54454) 2023-07-18T03:20:06.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.053+0000 7fc555d9b700 1 -- 172.21.15.101:0/3776623420 learned_addr learned my addr 172.21.15.101:0/3776623420 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:06.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.053+0000 7fc556d9d700 1 --2- 172.21.15.101:0/3776623420 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610b4a83ad0 0x5610b4a7bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:06.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.053+0000 7fc55659c700 1 -- 172.21.15.101:0/3776623420 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610b4a83ad0 msgr2=0x5610b4a7bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:06.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.053+0000 7fc55659c700 1 --2- 172.21.15.101:0/3776623420 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610b4a83ad0 0x5610b4a7bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.053+0000 7fc55659c700 1 -- 172.21.15.101:0/3776623420 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5610b4a78be0 msgr2=0x5610b4a8ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:06.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.053+0000 7fc55659c700 1 --2- 172.21.15.101:0/3776623420 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5610b4a78be0 0x5610b4a8ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.053+0000 7fc55659c700 1 -- 172.21.15.101:0/3776623420 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610b481f250 con 0x5610b4a782d0 2023-07-18T03:20:06.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.053+0000 7fc555d9b700 1 --2- 172.21.15.101:0/3776623420 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5610b4a78be0 0x5610b4a8ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:20:06.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.053+0000 7fc55659c700 1 --2- 172.21.15.101:0/3776623420 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610b4a782d0 0x5610b4a8a4f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc548099c60 tx=0x7fc5480a2510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63b38862b0b1827c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:06.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.054+0000 7fc5457fa700 1 -- 172.21.15.101:0/3776623420 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc548099f60 con 0x5610b4a782d0 2023-07-18T03:20:06.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.054+0000 7fc5457fa700 1 -- 172.21.15.101:0/3776623420 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc548010880 con 0x5610b4a782d0 2023-07-18T03:20:06.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.054+0000 7fc5457fa700 1 -- 172.21.15.101:0/3776623420 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5480996b0 con 0x5610b4a782d0 2023-07-18T03:20:06.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.054+0000 7fc57069bb80 1 -- 172.21.15.101:0/3776623420 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610b4a782d0 msgr2=0x5610b4a8a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:06.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.054+0000 7fc57069bb80 1 --2- 172.21.15.101:0/3776623420 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610b4a782d0 0x5610b4a8a4f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc548099c60 tx=0x7fc5480a2510 comp rx=0 tx=0).stop 2023-07-18T03:20:06.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.054+0000 7fc57069bb80 1 -- 172.21.15.101:0/3776623420 shutdown_connections 2023-07-18T03:20:06.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.054+0000 7fc57069bb80 1 --2- 172.21.15.101:0/3776623420 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610b4a782d0 0x5610b4a8a4f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.054+0000 7fc57069bb80 1 --2- 172.21.15.101:0/3776623420 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610b4a83ad0 0x5610b4a7bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.054+0000 7fc57069bb80 1 --2- 172.21.15.101:0/3776623420 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5610b4a78be0 0x5610b4a8ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.054+0000 7fc57069bb80 1 -- 172.21.15.101:0/3776623420 >> 172.21.15.101:0/3776623420 conn(0x5610b4a6eda0 msgr2=0x5610b4a8f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:06.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.054+0000 7fc57069bb80 1 -- 172.21.15.101:0/3776623420 shutdown_connections 2023-07-18T03:20:06.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.054+0000 7fc57069bb80 1 -- 172.21.15.101:0/3776623420 wait complete. 2023-07-18T03:20:06.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.055+0000 7fc57069bb80 1 -- start start 2023-07-18T03:20:06.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.055+0000 7fc57069bb80 1 Processor -- start 2023-07-18T03:20:06.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.056+0000 7fc57069bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610b4a782d0 0x5610b4a8a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:06.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.056+0000 7fc57069bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610b4a78be0 0x5610b4a88590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:06.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.056+0000 7fc57069bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5610b4a83ad0 0x5610b4a88ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:06.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.056+0000 7fc57069bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610b4a89010 con 0x5610b4a83ad0 2023-07-18T03:20:06.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.056+0000 7fc57069bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610b4a89180 con 0x5610b4a782d0 2023-07-18T03:20:06.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.056+0000 7fc57069bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610b4a89340 con 0x5610b4a78be0 2023-07-18T03:20:06.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.056+0000 7fc57069bb80 10 client.0 _ll_register_callbacks cb 0x7fc54c009c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:20:06.059 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.056+0000 7fc55659c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610b4a782d0 0x5610b4a8a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:06.059 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.056+0000 7fc555d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610b4a78be0 0x5610b4a88590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:06.059 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.056+0000 7fc555d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610b4a78be0 0x5610b4a88590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:38452/0 (socket says 172.21.15.101:38452) 2023-07-18T03:20:06.059 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.056+0000 7fc55659c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610b4a782d0 0x5610b4a8a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:43122/0 (socket says 172.21.15.101:43122) 2023-07-18T03:20:06.059 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.056+0000 7fc555d9b700 1 -- 172.21.15.101:0/23932485 learned_addr learned my addr 172.21.15.101:0/23932485 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:06.059 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.056+0000 7fc556d9d700 1 --2- 172.21.15.101:0/23932485 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5610b4a83ad0 0x5610b4a88ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:06.059 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.056+0000 7fc555d9b700 1 -- 172.21.15.101:0/23932485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610b4a782d0 msgr2=0x5610b4a8a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:06.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.056+0000 7fc555d9b700 1 --2- 172.21.15.101:0/23932485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610b4a782d0 0x5610b4a8a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.056+0000 7fc555d9b700 1 -- 172.21.15.101:0/23932485 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5610b4a83ad0 msgr2=0x5610b4a88ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:06.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.056+0000 7fc555d9b700 1 --2- 172.21.15.101:0/23932485 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5610b4a83ad0 0x5610b4a88ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.056+0000 7fc555d9b700 1 -- 172.21.15.101:0/23932485 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5610b47e1c60 con 0x5610b4a78be0 2023-07-18T03:20:06.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.056+0000 7fc555d9b700 1 --2- 172.21.15.101:0/23932485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610b4a78be0 0x5610b4a88590 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc54c01a000 tx=0x7fc54c005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16a10bb7f38ce40d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:06.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.056+0000 7fc53dffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:20:06.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.057+0000 7fc53dffb700 1 -- 172.21.15.101:0/23932485 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c016070 con 0x5610b4a78be0 2023-07-18T03:20:06.061 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.057+0000 7fc57069bb80 1 -- 172.21.15.101:0/23932485 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc54809df90 con 0x5610b4a78be0 2023-07-18T03:20:06.061 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.057+0000 7fc53dffb700 1 -- 172.21.15.101:0/23932485 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fc54c000e40 con 0x5610b4a78be0 2023-07-18T03:20:06.061 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.057+0000 7fc53dffb700 1 -- 172.21.15.101:0/23932485 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc54c005720 con 0x5610b4a78be0 2023-07-18T03:20:06.061 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.057+0000 7fc57069bb80 10 client.28108 fetch_fsmap learned FSMap version 33 2023-07-18T03:20:06.061 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.057+0000 7fc57069bb80 1 -- 172.21.15.101:0/23932485 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5610b47f8de0 con 0x5610b4a78be0 2023-07-18T03:20:06.061 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.057+0000 7fc53dffb700 1 -- 172.21.15.101:0/23932485 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc54809df90 con 0x5610b4a78be0 2023-07-18T03:20:06.061 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.057+0000 7fc53dffb700 1 -- 172.21.15.101:0/23932485 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc54c00f890 con 0x5610b4a78be0 2023-07-18T03:20:06.061 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.057+0000 7fc57069bb80 10 client.28108 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:20:06.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.057+0000 7fc57069bb80 10 client.28108 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:20:06.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.057+0000 7fc57069bb80 20 client.28108 populate_metadata read hostname 'smithi101' 2023-07-18T03:20:06.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.057+0000 7fc57069bb80 1 --2- 172.21.15.101:0/23932485 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5610b4ae86c0 0x5610b4aeaad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:06.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.057+0000 7fc57069bb80 4 client.28108 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:20:06.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.057+0000 7fc57069bb80 1 -- 172.21.15.101:0/23932485 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5610b4986050 con 0x5610b4ae86c0 2023-07-18T03:20:06.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.058+0000 7fc55659c700 1 --2- 172.21.15.101:0/23932485 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5610b4ae86c0 0x5610b4aeaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:06.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.058+0000 7fc55659c700 1 --2- 172.21.15.101:0/23932485 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5610b4ae86c0 0x5610b4aeaad0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a9b2d8d192cfd89 server_cookie=135e826ff61159e5 in_seq=0 out_seq=0 2023-07-18T03:20:06.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.058+0000 7fc53dffb700 10 client.28108 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:20:06.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.059+0000 7fc53dffb700 1 -- 172.21.15.101:0/23932485 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5610b4986050 con 0x5610b4ae86c0 2023-07-18T03:20:06.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.059+0000 7fc53dffb700 10 client.28108 handle_command_reply: tid=0 2023-07-18T03:20:06.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.059+0000 7fc57069bb80 1 client.28108 shutdown 2023-07-18T03:20:06.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.059+0000 7fc57069bb80 1 -- 172.21.15.101:0/23932485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610b4a78be0 msgr2=0x5610b4a88590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:06.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.059+0000 7fc57069bb80 1 --2- 172.21.15.101:0/23932485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610b4a78be0 0x5610b4a88590 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc54c01a000 tx=0x7fc54c005270 comp rx=0 tx=0).stop 2023-07-18T03:20:06.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.059+0000 7fc57069bb80 1 -- 172.21.15.101:0/23932485 shutdown_connections 2023-07-18T03:20:06.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.059+0000 7fc57069bb80 1 --2- 172.21.15.101:0/23932485 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5610b4ae86c0 0x5610b4aeaad0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.059+0000 7fc57069bb80 1 --2- 172.21.15.101:0/23932485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5610b4a782d0 0x5610b4a8a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.059+0000 7fc57069bb80 1 --2- 172.21.15.101:0/23932485 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5610b4a83ad0 0x5610b4a88ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.059+0000 7fc57069bb80 1 --2- 172.21.15.101:0/23932485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610b4a78be0 0x5610b4a88590 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.059+0000 7fc53dffb700 0 client.28108 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:20:06.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.059+0000 7fc57069bb80 1 -- 172.21.15.101:0/23932485 >> 172.21.15.101:0/23932485 conn(0x5610b4a6eda0 msgr2=0x5610b4a8ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:06.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.059+0000 7fc57069bb80 1 -- 172.21.15.101:0/23932485 shutdown_connections 2023-07-18T03:20:06.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.059+0000 7fc57069bb80 1 -- 172.21.15.101:0/23932485 wait complete. 2023-07-18T03:20:06.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.059+0000 7fc57069bb80 20 client.28108 trim_cache size 0 max 16384 2023-07-18T03:20:06.065 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:20:06.065 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "no active scrubs running", 2023-07-18T03:20:06.066 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": {} 2023-07-18T03:20:06.066 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:20:06.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.060+0000 7fc55cdff700 1 -- 172.21.15.101:0/1543866565 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc5400a1340 msgr2=0x7fc5400a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:06.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.060+0000 7fc55cdff700 1 --2- 172.21.15.101:0/1543866565 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc5400a1340 0x7fc5400a37f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc5500062a0 tx=0x7fc55000b720 comp rx=0 tx=0).stop 2023-07-18T03:20:06.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.060+0000 7fc55cdff700 1 -- 172.21.15.101:0/1543866565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc5580fe820 msgr2=0x7fc558197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:06.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.060+0000 7fc55cdff700 1 --2- 172.21.15.101:0/1543866565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc5580fe820 0x7fc558197fc0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc54800e9a0 tx=0x7fc54800ecb0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.060+0000 7fc55cdff700 1 -- 172.21.15.101:0/1543866565 shutdown_connections 2023-07-18T03:20:06.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.060+0000 7fc55cdff700 1 --2- 172.21.15.101:0/1543866565 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc5400a1340 0x7fc5400a37f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.060+0000 7fc55cdff700 1 --2- 172.21.15.101:0/1543866565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc55810c3d0 0x7fc55819c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.060+0000 7fc55cdff700 1 --2- 172.21.15.101:0/1543866565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc5580fe820 0x7fc558197fc0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.060+0000 7fc55cdff700 1 --2- 172.21.15.101:0/1543866565 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc5580ff1c0 0x7fc558198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.060+0000 7fc55cdff700 1 -- 172.21.15.101:0/1543866565 >> 172.21.15.101:0/1543866565 conn(0x7fc5580fa510 msgr2=0x7fc55810d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:06.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.060+0000 7fc55cdff700 1 -- 172.21.15.101:0/1543866565 shutdown_connections 2023-07-18T03:20:06.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.060+0000 7fc55cdff700 1 -- 172.21.15.101:0/1543866565 wait complete. 2023-07-18T03:20:06.074 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-18T03:20:06.075 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:20:06.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.312+0000 7fd8087ff700 1 -- 172.21.15.101:0/1210970550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8000731c0 msgr2=0x7fd8001105c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:06.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.312+0000 7fd8087ff700 1 --2- 172.21.15.101:0/1210970550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8000731c0 0x7fd8001105c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0009990 tx=0x7fd7f0009ca0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.312+0000 7fd8087ff700 1 -- 172.21.15.101:0/1210970550 shutdown_connections 2023-07-18T03:20:06.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.312+0000 7fd8087ff700 1 --2- 172.21.15.101:0/1210970550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd800110b00 0x7fd800112ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.312+0000 7fd8087ff700 1 --2- 172.21.15.101:0/1210970550 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd8000728b0 0x7fd800072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.312+0000 7fd8087ff700 1 --2- 172.21.15.101:0/1210970550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8000731c0 0x7fd8001105c0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.312+0000 7fd8087ff700 1 -- 172.21.15.101:0/1210970550 >> 172.21.15.101:0/1210970550 conn(0x7fd8000fb630 msgr2=0x7fd8000fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:06.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.312+0000 7fd8087ff700 1 -- 172.21.15.101:0/1210970550 shutdown_connections 2023-07-18T03:20:06.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.312+0000 7fd8087ff700 1 -- 172.21.15.101:0/1210970550 wait complete. 2023-07-18T03:20:06.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.313+0000 7fd8087ff700 1 Processor -- start 2023-07-18T03:20:06.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.314+0000 7fd8087ff700 1 -- start start 2023-07-18T03:20:06.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.314+0000 7fd8087ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8000728b0 0x7fd80019c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:06.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.314+0000 7fd8087ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd8000731c0 0x7fd80019c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:06.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.314+0000 7fd8087ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd800110b00 0x7fd8001a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:06.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.314+0000 7fd8087ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8001a11c0 con 0x7fd8000731c0 2023-07-18T03:20:06.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.314+0000 7fd8087ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8001a1330 con 0x7fd8000728b0 2023-07-18T03:20:06.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.314+0000 7fd8087ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8001a14f0 con 0x7fd800110b00 2023-07-18T03:20:06.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.314+0000 7fd80659b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8000728b0 0x7fd80019c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:06.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.314+0000 7fd80659b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8000728b0 0x7fd80019c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:43144/0 (socket says 172.21.15.101:43144) 2023-07-18T03:20:06.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.314+0000 7fd80659b700 1 -- 172.21.15.101:0/112214217 learned_addr learned my addr 172.21.15.101:0/112214217 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:06.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.314+0000 7fd805d9a700 1 --2- 172.21.15.101:0/112214217 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd8000731c0 0x7fd80019c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:06.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.314+0000 7fd806d9c700 1 --2- 172.21.15.101:0/112214217 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd800110b00 0x7fd8001a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:06.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.315+0000 7fd80659b700 1 -- 172.21.15.101:0/112214217 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd800110b00 msgr2=0x7fd8001a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:06.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.315+0000 7fd80659b700 1 --2- 172.21.15.101:0/112214217 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd800110b00 0x7fd8001a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.315+0000 7fd80659b700 1 -- 172.21.15.101:0/112214217 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd8000731c0 msgr2=0x7fd80019c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:06.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.315+0000 7fd80659b700 1 --2- 172.21.15.101:0/112214217 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd8000731c0 0x7fd80019c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.315+0000 7fd80659b700 1 -- 172.21.15.101:0/112214217 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f00095f0 con 0x7fd8000728b0 2023-07-18T03:20:06.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.315+0000 7fd80659b700 1 --2- 172.21.15.101:0/112214217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8000728b0 0x7fd80019c250 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc00e9a0 tx=0x7fd7fc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:06.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.315+0000 7fd7f77fe700 1 -- 172.21.15.101:0/112214217 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc009790 con 0x7fd8000728b0 2023-07-18T03:20:06.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.315+0000 7fd7f77fe700 1 -- 172.21.15.101:0/112214217 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7fc010430 con 0x7fd8000728b0 2023-07-18T03:20:06.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.315+0000 7fd8087ff700 1 -- 172.21.15.101:0/112214217 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8001a17d0 con 0x7fd8000728b0 2023-07-18T03:20:06.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.315+0000 7fd7f77fe700 1 -- 172.21.15.101:0/112214217 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc018680 con 0x7fd8000728b0 2023-07-18T03:20:06.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.315+0000 7fd8087ff700 1 -- 172.21.15.101:0/112214217 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8001a9010 con 0x7fd8000728b0 2023-07-18T03:20:06.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.316+0000 7fd8087ff700 1 -- 172.21.15.101:0/112214217 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd800103020 con 0x7fd8000728b0 2023-07-18T03:20:06.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.320+0000 7fd7f77fe700 1 -- 172.21.15.101:0/112214217 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fd7fc0105a0 con 0x7fd8000728b0 2023-07-18T03:20:06.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.322+0000 7fd7f77fe700 1 --2- 172.21.15.101:0/112214217 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd7ec0a1380 0x7fd7ec0a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:06.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.322+0000 7fd7f77fe700 1 -- 172.21.15.101:0/112214217 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fd7fc014070 con 0x7fd8000728b0 2023-07-18T03:20:06.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.322+0000 7fd805d9a700 1 --2- 172.21.15.101:0/112214217 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd7ec0a1380 0x7fd7ec0a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:06.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.322+0000 7fd7f77fe700 1 -- 172.21.15.101:0/112214217 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd7fc096270 con 0x7fd8000728b0 2023-07-18T03:20:06.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.323+0000 7fd805d9a700 1 --2- 172.21.15.101:0/112214217 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd7ec0a1380 0x7fd7ec0a3830 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd80019d7f0 tx=0x7fd7f0005fb0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:06.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.621+0000 7fd8087ff700 1 -- 172.21.15.101:0/112214217 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd800067300 con 0x7fd8000728b0 2023-07-18T03:20:06.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.622+0000 7fd7f77fe700 1 -- 172.21.15.101:0/112214217 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+10858 (secure 0 0 0) 0x7fd7fc099890 con 0x7fd8000728b0 2023-07-18T03:20:06.618 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:20:06.618 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/4192846769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":4192846769},{"type":"v1","addr":"172.21.15.101:6839","nonce":4192846769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6841/3824340175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6840","nonce":3824340175},{"type":"v1","addr":"172.21.15.101:6841","nonce":3824340175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:04:59.361185+0000","modified":"2023-07-18T03:16:36.002358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.155:6839/2055462483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2055462483},{"type":"v1","addr":"172.21.15.155:6839","nonce":2055462483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:20:06.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.625+0000 7fd8087ff700 1 -- 172.21.15.101:0/112214217 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd7ec0a1380 msgr2=0x7fd7ec0a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:06.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.625+0000 7fd8087ff700 1 --2- 172.21.15.101:0/112214217 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd7ec0a1380 0x7fd7ec0a3830 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd80019d7f0 tx=0x7fd7f0005fb0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.625+0000 7fd8087ff700 1 -- 172.21.15.101:0/112214217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8000728b0 msgr2=0x7fd80019c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:06.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.625+0000 7fd8087ff700 1 --2- 172.21.15.101:0/112214217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8000728b0 0x7fd80019c250 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc00e9a0 tx=0x7fd7fc00ecb0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.625+0000 7fd8087ff700 1 -- 172.21.15.101:0/112214217 shutdown_connections 2023-07-18T03:20:06.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.625+0000 7fd8087ff700 1 --2- 172.21.15.101:0/112214217 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd7ec0a1380 0x7fd7ec0a3830 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.625+0000 7fd8087ff700 1 --2- 172.21.15.101:0/112214217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8000728b0 0x7fd80019c250 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.625+0000 7fd8087ff700 1 --2- 172.21.15.101:0/112214217 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd800110b00 0x7fd8001a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.625+0000 7fd8087ff700 1 --2- 172.21.15.101:0/112214217 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd8000731c0 0x7fd80019c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.625+0000 7fd8087ff700 1 -- 172.21.15.101:0/112214217 >> 172.21.15.101:0/112214217 conn(0x7fd8000fb630 msgr2=0x7fd8000fc290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:06.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.625+0000 7fd8087ff700 1 -- 172.21.15.101:0/112214217 shutdown_connections 2023-07-18T03:20:06.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.625+0000 7fd8087ff700 1 -- 172.21.15.101:0/112214217 wait complete. 2023-07-18T03:20:06.623 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 33 2023-07-18T03:20:06.628 INFO:tasks.workunit.client.0.smithi101.stdout: 1 739866 5.30 MB/sec execute 571 sec latency 4864.501 ms 2023-07-18T03:20:06.640 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-18T03:20:06.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.879+0000 7f63b0bff700 1 -- 172.21.15.101:0/1142071985 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63ac105af0 msgr2=0x7f63ac105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:06.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.879+0000 7f63b0bff700 1 --2- 172.21.15.101:0/1142071985 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63ac105af0 0x7f63ac105ec0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f639c009990 tx=0x7f639c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.879+0000 7f63b0bff700 1 -- 172.21.15.101:0/1142071985 shutdown_connections 2023-07-18T03:20:06.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.879+0000 7f63b0bff700 1 --2- 172.21.15.101:0/1142071985 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63ac100490 0x7f63ac10c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.879+0000 7f63b0bff700 1 --2- 172.21.15.101:0/1142071985 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63ac105af0 0x7f63ac105ec0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.879+0000 7f63b0bff700 1 --2- 172.21.15.101:0/1142071985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63ac0ffae0 0x7f63ac0fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.879+0000 7f63b0bff700 1 -- 172.21.15.101:0/1142071985 >> 172.21.15.101:0/1142071985 conn(0x7f63ac0fb5e0 msgr2=0x7f63ac0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:06.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.879+0000 7f63b0bff700 1 -- 172.21.15.101:0/1142071985 shutdown_connections 2023-07-18T03:20:06.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.879+0000 7f63b0bff700 1 -- 172.21.15.101:0/1142071985 wait complete. 2023-07-18T03:20:06.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.880+0000 7f63b0bff700 1 Processor -- start 2023-07-18T03:20:06.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.881+0000 7f63b0bff700 1 -- start start 2023-07-18T03:20:06.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.881+0000 7f63b0bff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63ac0ffae0 0x7f63ac197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:06.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.881+0000 7f63b0bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63ac100490 0x7f63ac198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:06.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.881+0000 7f63b0bff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63ac105af0 0x7f63ac19c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:06.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.881+0000 7f63b0bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac19cd80 con 0x7f63ac100490 2023-07-18T03:20:06.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.881+0000 7f63b0bff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac19cef0 con 0x7f63ac0ffae0 2023-07-18T03:20:06.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.881+0000 7f63b0bff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac19d0b0 con 0x7f63ac105af0 2023-07-18T03:20:06.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.881+0000 7f63aa59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63ac0ffae0 0x7f63ac197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:06.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.881+0000 7f63aa59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63ac0ffae0 0x7f63ac197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:43166/0 (socket says 172.21.15.101:43166) 2023-07-18T03:20:06.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.881+0000 7f63aa59c700 1 -- 172.21.15.101:0/4142560788 learned_addr learned my addr 172.21.15.101:0/4142560788 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:06.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.881+0000 7f63a9d9b700 1 --2- 172.21.15.101:0/4142560788 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63ac100490 0x7f63ac198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:06.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.881+0000 7f63aad9d700 1 --2- 172.21.15.101:0/4142560788 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63ac105af0 0x7f63ac19c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:06.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.881+0000 7f63aa59c700 1 -- 172.21.15.101:0/4142560788 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63ac105af0 msgr2=0x7f63ac19c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:06.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.881+0000 7f63aa59c700 1 --2- 172.21.15.101:0/4142560788 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63ac105af0 0x7f63ac19c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.882+0000 7f63aa59c700 1 -- 172.21.15.101:0/4142560788 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63ac100490 msgr2=0x7f63ac198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:06.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.882+0000 7f63aa59c700 1 --2- 172.21.15.101:0/4142560788 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63ac100490 0x7f63ac198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.882+0000 7f63aa59c700 1 -- 172.21.15.101:0/4142560788 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f639c0095f0 con 0x7f63ac0ffae0 2023-07-18T03:20:06.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.882+0000 7f63aa59c700 1 --2- 172.21.15.101:0/4142560788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63ac0ffae0 0x7f63ac197df0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f639c000c00 tx=0x7f639c005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:06.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.882+0000 7f639b7fe700 1 -- 172.21.15.101:0/4142560788 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639c01d070 con 0x7f63ac0ffae0 2023-07-18T03:20:06.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.882+0000 7f63b0bff700 1 -- 172.21.15.101:0/4142560788 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63ac19d330 con 0x7f63ac0ffae0 2023-07-18T03:20:06.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.882+0000 7f639b7fe700 1 -- 172.21.15.101:0/4142560788 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f639c022470 con 0x7f63ac0ffae0 2023-07-18T03:20:06.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.882+0000 7f639b7fe700 1 -- 172.21.15.101:0/4142560788 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639c017650 con 0x7f63ac0ffae0 2023-07-18T03:20:06.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.882+0000 7f63b0bff700 1 -- 172.21.15.101:0/4142560788 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63ac1a4bd0 con 0x7f63ac0ffae0 2023-07-18T03:20:06.882 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.885+0000 7f639b7fe700 1 -- 172.21.15.101:0/4142560788 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f639c0225e0 con 0x7f63ac0ffae0 2023-07-18T03:20:06.882 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.886+0000 7f63c4573b80 1 Processor -- start 2023-07-18T03:20:06.882 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.887+0000 7f63c4573b80 1 -- start start 2023-07-18T03:20:06.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.887+0000 7f63c4573b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b176d0f2d0 0x55b176d0f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:06.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.887+0000 7f63c4573b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b176d0fbe0 0x55b176d1a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:06.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.887+0000 7f63c4573b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b176d1aad0 0x55b176d1ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:06.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.887+0000 7f63c4573b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b176c15cb0 con 0x55b176d0f2d0 2023-07-18T03:20:06.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.887+0000 7f63c4573b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b176c68a40 con 0x55b176d1aad0 2023-07-18T03:20:06.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.887+0000 7f63c4573b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b176c61ed0 con 0x55b176d0fbe0 2023-07-18T03:20:06.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.887+0000 7f63a9d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b176d1aad0 0x55b176d1ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:06.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.887+0000 7f63aa59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b176d0fbe0 0x55b176d1a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:06.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.887+0000 7f63aad9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b176d0f2d0 0x55b176d0f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:06.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.887+0000 7f63a9d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b176d1aad0 0x55b176d1ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:43172/0 (socket says 172.21.15.101:43172) 2023-07-18T03:20:06.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.887+0000 7f63a9d9b700 1 -- 172.21.15.101:0/1035902680 learned_addr learned my addr 172.21.15.101:0/1035902680 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:06.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.888+0000 7f63a9d9b700 1 -- 172.21.15.101:0/1035902680 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b176d0fbe0 msgr2=0x55b176d1a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:06.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.888+0000 7f63a9d9b700 1 --2- 172.21.15.101:0/1035902680 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b176d0fbe0 0x55b176d1a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.888+0000 7f63a9d9b700 1 -- 172.21.15.101:0/1035902680 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b176d0f2d0 msgr2=0x55b176d0f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:06.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.888+0000 7f63a9d9b700 1 --2- 172.21.15.101:0/1035902680 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b176d0f2d0 0x55b176d0f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.888+0000 7f63a9d9b700 1 -- 172.21.15.101:0/1035902680 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b176a8fde0 con 0x55b176d1aad0 2023-07-18T03:20:06.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.888+0000 7f639b7fe700 1 --2- 172.21.15.101:0/4142560788 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f63940a12c0 0x7f63940a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:06.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.888+0000 7f63a9d9b700 1 --2- 172.21.15.101:0/1035902680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b176d1aad0 0x55b176d1ceb0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009d20 tx=0x7f63a0011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb1af1f8bf9fdea0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:06.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.888+0000 7f639b7fe700 1 -- 172.21.15.101:0/4142560788 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f639c0ce510 con 0x7f63ac0ffae0 2023-07-18T03:20:06.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.888+0000 7f63aad9d700 1 --2- 172.21.15.101:0/4142560788 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f63940a12c0 0x7f63940a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:06.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.888+0000 7f6398ff9700 1 -- 172.21.15.101:0/1035902680 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a0019630 con 0x55b176d1aad0 2023-07-18T03:20:06.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.888+0000 7f6398ff9700 1 -- 172.21.15.101:0/1035902680 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63a0019c50 con 0x55b176d1aad0 2023-07-18T03:20:06.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.888+0000 7f6398ff9700 1 -- 172.21.15.101:0/1035902680 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a000ec40 con 0x55b176d1aad0 2023-07-18T03:20:06.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.889+0000 7f63aad9d700 1 --2- 172.21.15.101:0/4142560788 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f63940a12c0 0x7f63940a3770 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f63a40127f0 tx=0x7f63a400b040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:06.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.889+0000 7f63c4573b80 1 -- 172.21.15.101:0/1035902680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b176d1aad0 msgr2=0x55b176d1ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:06.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.889+0000 7f63c4573b80 1 --2- 172.21.15.101:0/1035902680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b176d1aad0 0x55b176d1ceb0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009d20 tx=0x7f63a0011720 comp rx=0 tx=0).stop 2023-07-18T03:20:06.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.889+0000 7f63c4573b80 1 -- 172.21.15.101:0/1035902680 shutdown_connections 2023-07-18T03:20:06.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.889+0000 7f63c4573b80 1 --2- 172.21.15.101:0/1035902680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b176d1aad0 0x55b176d1ceb0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.889+0000 7f63c4573b80 1 --2- 172.21.15.101:0/1035902680 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b176d0f2d0 0x55b176d0f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.889+0000 7f63c4573b80 1 --2- 172.21.15.101:0/1035902680 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b176d0fbe0 0x55b176d1a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.889+0000 7f63c4573b80 1 -- 172.21.15.101:0/1035902680 >> 172.21.15.101:0/1035902680 conn(0x55b176d05da0 msgr2=0x55b176d0b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:06.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.889+0000 7f63c4573b80 1 -- 172.21.15.101:0/1035902680 shutdown_connections 2023-07-18T03:20:06.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.889+0000 7f63c4573b80 1 -- 172.21.15.101:0/1035902680 wait complete. 2023-07-18T03:20:06.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.890+0000 7f63c4573b80 1 -- start start 2023-07-18T03:20:06.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.890+0000 7f63c4573b80 1 Processor -- start 2023-07-18T03:20:06.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.890+0000 7f63c4573b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b176d0f2d0 0x55b176d1a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:06.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.891+0000 7f63c4573b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b176d0fbe0 0x55b176d177a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:06.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.891+0000 7f63c4573b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b176d1aad0 0x55b176d17e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:06.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.891+0000 7f63c4573b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b176c01ac0 con 0x55b176d1aad0 2023-07-18T03:20:06.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.891+0000 7f63c4573b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b176c017c0 con 0x55b176d0f2d0 2023-07-18T03:20:06.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.891+0000 7f63c4573b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b176ae5920 con 0x55b176d0fbe0 2023-07-18T03:20:06.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.891+0000 7f63c4573b80 10 client.0 _ll_register_callbacks cb 0x55b176bff760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:20:06.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.891+0000 7f63a9d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b176d0fbe0 0x55b176d177a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:06.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.891+0000 7f63a9d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b176d0fbe0 0x55b176d177a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:38488/0 (socket says 172.21.15.101:38488) 2023-07-18T03:20:06.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.891+0000 7f63a9d9b700 1 -- 172.21.15.101:0/1134930250 learned_addr learned my addr 172.21.15.101:0/1134930250 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:06.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.891+0000 7f63aa59c700 1 --2- 172.21.15.101:0/1134930250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b176d0f2d0 0x55b176d1a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:06.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.891+0000 7f63aad9d700 1 --2- 172.21.15.101:0/1134930250 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b176d1aad0 0x55b176d17e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:06.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.891+0000 7f63aa59c700 1 -- 172.21.15.101:0/1134930250 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b176d0fbe0 msgr2=0x55b176d177a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:06.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.891+0000 7f63aa59c700 1 --2- 172.21.15.101:0/1134930250 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b176d0fbe0 0x55b176d177a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.891+0000 7f63aa59c700 1 -- 172.21.15.101:0/1134930250 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b176d1aad0 msgr2=0x55b176d17e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:06.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.891+0000 7f63aa59c700 1 --2- 172.21.15.101:0/1134930250 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b176d1aad0 0x55b176d17e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.891+0000 7f63aa59c700 1 -- 172.21.15.101:0/1134930250 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63a0009980 con 0x55b176d0f2d0 2023-07-18T03:20:06.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.892+0000 7f63aa59c700 1 --2- 172.21.15.101:0/1134930250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b176d0f2d0 0x55b176d1a4b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f639c0cb850 tx=0x7f639c022bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3e0f7e8dde74e2d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:06.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.892+0000 7f6391ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:20:06.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.892+0000 7f6391ffb700 1 -- 172.21.15.101:0/1134930250 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639c01bde0 con 0x55b176d0f2d0 2023-07-18T03:20:06.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.892+0000 7f6391ffb700 1 -- 172.21.15.101:0/1134930250 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f639c09b260 con 0x55b176d0f2d0 2023-07-18T03:20:06.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.892+0000 7f63c4573b80 1 -- 172.21.15.101:0/1134930250 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b176bbb4f0 con 0x55b176d0f2d0 2023-07-18T03:20:06.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.892+0000 7f6391ffb700 1 -- 172.21.15.101:0/1134930250 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f639c09b030 con 0x55b176d0f2d0 2023-07-18T03:20:06.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.892+0000 7f63c4573b80 10 client.28094 fetch_fsmap learned FSMap version 33 2023-07-18T03:20:06.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.892+0000 7f63c4573b80 1 -- 172.21.15.101:0/1134930250 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55b176a78c60 con 0x55b176d0f2d0 2023-07-18T03:20:06.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.892+0000 7f6391ffb700 1 -- 172.21.15.101:0/1134930250 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b176bbb4f0 con 0x55b176d0f2d0 2023-07-18T03:20:06.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.893+0000 7f6391ffb700 1 -- 172.21.15.101:0/1134930250 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f639c09b4c0 con 0x55b176d0f2d0 2023-07-18T03:20:06.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.893+0000 7f63c4573b80 10 client.28094 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:20:06.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.893+0000 7f63c4573b80 10 client.28094 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:20:06.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.893+0000 7f63c4573b80 20 client.28094 populate_metadata read hostname 'smithi101' 2023-07-18T03:20:06.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.893+0000 7f63c4573b80 1 --2- 172.21.15.101:0/1134930250 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55b176d2a070 0x55b176d2c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:06.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.893+0000 7f63c4573b80 4 client.28094 mds_command: new command op to 24541 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:20:06.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.893+0000 7f63c4573b80 1 -- 172.21.15.101:0/1134930250 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b176c5aeb0 con 0x55b176d2a070 2023-07-18T03:20:06.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.894+0000 7f63a9d9b700 1 --2- 172.21.15.101:0/1134930250 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55b176d2a070 0x55b176d2c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:06.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.894+0000 7f63a9d9b700 1 --2- 172.21.15.101:0/1134930250 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55b176d2a070 0x55b176d2c480 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=230ed32a38582f66 server_cookie=71c56b36e8efdfd8 in_seq=0 out_seq=0 2023-07-18T03:20:06.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.894+0000 7f6391ffb700 10 client.28094 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:20:06.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.895+0000 7f6391ffb700 1 -- 172.21.15.101:0/1134930250 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b176c5aeb0 con 0x55b176d2a070 2023-07-18T03:20:06.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.895+0000 7f6391ffb700 10 client.28094 handle_command_reply: tid=0 2023-07-18T03:20:06.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.895+0000 7f63c4573b80 1 client.28094 shutdown 2023-07-18T03:20:06.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.895+0000 7f63c4573b80 1 -- 172.21.15.101:0/1134930250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b176d0f2d0 msgr2=0x55b176d1a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:06.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.895+0000 7f63c4573b80 1 --2- 172.21.15.101:0/1134930250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b176d0f2d0 0x55b176d1a4b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f639c0cb850 tx=0x7f639c022bc0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.895+0000 7f63c4573b80 1 -- 172.21.15.101:0/1134930250 shutdown_connections 2023-07-18T03:20:06.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.895+0000 7f63c4573b80 1 --2- 172.21.15.101:0/1134930250 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55b176d2a070 0x55b176d2c480 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.895+0000 7f63c4573b80 1 --2- 172.21.15.101:0/1134930250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b176d0f2d0 0x55b176d1a4b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.895+0000 7f63c4573b80 1 --2- 172.21.15.101:0/1134930250 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b176d1aad0 0x55b176d17e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.895+0000 7f6391ffb700 0 client.28094 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:20:06.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.895+0000 7f63c4573b80 1 --2- 172.21.15.101:0/1134930250 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b176d0fbe0 0x55b176d177a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.895+0000 7f63c4573b80 1 -- 172.21.15.101:0/1134930250 >> 172.21.15.101:0/1134930250 conn(0x55b176d05da0 msgr2=0x55b176d0b290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:06.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.895+0000 7f63c4573b80 1 -- 172.21.15.101:0/1134930250 shutdown_connections 2023-07-18T03:20:06.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.896+0000 7f63c4573b80 1 -- 172.21.15.101:0/1134930250 wait complete. 2023-07-18T03:20:06.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.896+0000 7f63c4573b80 20 client.28094 trim_cache size 0 max 16384 2023-07-18T03:20:06.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.910+0000 7f63c4573b80 1 Processor -- start 2023-07-18T03:20:06.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.910+0000 7f63c4573b80 1 -- start start 2023-07-18T03:20:06.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.910+0000 7f63c4573b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b176d0f2d0 0x55b176d214b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:06.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.910+0000 7f63c4573b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b176d0fbe0 0x55b176d1a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:06.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.910+0000 7f63c4573b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b176d1aad0 0x55b176d14a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:06.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.910+0000 7f63c4573b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b176d18d80 con 0x55b176d0fbe0 2023-07-18T03:20:06.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.910+0000 7f63c4573b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b176d02fd0 con 0x55b176d0f2d0 2023-07-18T03:20:06.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.910+0000 7f63c4573b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b176d059a0 con 0x55b176d1aad0 2023-07-18T03:20:06.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.911+0000 7f63aa59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b176d0f2d0 0x55b176d214b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:06.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.911+0000 7f63aa59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b176d0f2d0 0x55b176d214b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:43190/0 (socket says 172.21.15.101:43190) 2023-07-18T03:20:06.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.911+0000 7f63aa59c700 1 -- 172.21.15.101:0/1522413370 learned_addr learned my addr 172.21.15.101:0/1522413370 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:06.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.911+0000 7f63a9d9b700 1 --2- 172.21.15.101:0/1522413370 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b176d0fbe0 0x55b176d1a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:06.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.911+0000 7f63aad9d700 1 --2- 172.21.15.101:0/1522413370 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b176d1aad0 0x55b176d14a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:06.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.911+0000 7f63aa59c700 1 -- 172.21.15.101:0/1522413370 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b176d1aad0 msgr2=0x55b176d14a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:06.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.911+0000 7f63aa59c700 1 --2- 172.21.15.101:0/1522413370 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b176d1aad0 0x55b176d14a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.911+0000 7f63aa59c700 1 -- 172.21.15.101:0/1522413370 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b176d0fbe0 msgr2=0x55b176d1a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:06.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.911+0000 7f63aa59c700 1 --2- 172.21.15.101:0/1522413370 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b176d0fbe0 0x55b176d1a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.911+0000 7f63aa59c700 1 -- 172.21.15.101:0/1522413370 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b176ab6250 con 0x55b176d0f2d0 2023-07-18T03:20:06.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.911+0000 7f63aa59c700 1 --2- 172.21.15.101:0/1522413370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b176d0f2d0 0x55b176d214b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f639c022c30 tx=0x7f639c097660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20daecdd067e33e2 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:06.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.912+0000 7f63997fa700 1 -- 172.21.15.101:0/1522413370 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639c09abc0 con 0x55b176d0f2d0 2023-07-18T03:20:06.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.912+0000 7f63997fa700 1 -- 172.21.15.101:0/1522413370 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f639c09b030 con 0x55b176d0f2d0 2023-07-18T03:20:06.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.912+0000 7f63997fa700 1 -- 172.21.15.101:0/1522413370 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639c0a3620 con 0x55b176d0f2d0 2023-07-18T03:20:06.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.912+0000 7f63c4573b80 1 -- 172.21.15.101:0/1522413370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b176d0f2d0 msgr2=0x55b176d214b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:06.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.912+0000 7f63c4573b80 1 --2- 172.21.15.101:0/1522413370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b176d0f2d0 0x55b176d214b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f639c022c30 tx=0x7f639c097660 comp rx=0 tx=0).stop 2023-07-18T03:20:06.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.912+0000 7f63c4573b80 1 -- 172.21.15.101:0/1522413370 shutdown_connections 2023-07-18T03:20:06.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.912+0000 7f63c4573b80 1 --2- 172.21.15.101:0/1522413370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b176d0f2d0 0x55b176d214b0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.912+0000 7f63c4573b80 1 --2- 172.21.15.101:0/1522413370 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b176d1aad0 0x55b176d14a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.912+0000 7f63c4573b80 1 --2- 172.21.15.101:0/1522413370 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b176d0fbe0 0x55b176d1a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.912+0000 7f63c4573b80 1 -- 172.21.15.101:0/1522413370 >> 172.21.15.101:0/1522413370 conn(0x55b176d05da0 msgr2=0x55b176d12af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:06.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.912+0000 7f63c4573b80 1 -- 172.21.15.101:0/1522413370 shutdown_connections 2023-07-18T03:20:06.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.912+0000 7f63c4573b80 1 -- 172.21.15.101:0/1522413370 wait complete. 2023-07-18T03:20:06.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.913+0000 7f63c4573b80 1 -- start start 2023-07-18T03:20:06.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.913+0000 7f63c4573b80 1 Processor -- start 2023-07-18T03:20:06.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.913+0000 7f63c4573b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b176d0f2d0 0x55b176d720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:06.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.913+0000 7f63c4573b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b176d0fbe0 0x55b176d725f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:06.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.914+0000 7f63c4573b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b176d1aad0 0x55b176d76f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:06.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.914+0000 7f63c4573b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b176d77450 con 0x55b176d0f2d0 2023-07-18T03:20:06.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.914+0000 7f63c4573b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b176d77590 con 0x55b176d1aad0 2023-07-18T03:20:06.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.914+0000 7f63c4573b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b176d77750 con 0x55b176d0fbe0 2023-07-18T03:20:06.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.914+0000 7f63c4573b80 10 client.0 _ll_register_callbacks cb 0x7f639c0d1c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:20:06.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.914+0000 7f63a9d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b176d0fbe0 0x55b176d725f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:06.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.914+0000 7f63aa59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b176d0f2d0 0x55b176d720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:06.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.914+0000 7f63a9d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b176d0fbe0 0x55b176d725f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:38506/0 (socket says 172.21.15.101:38506) 2023-07-18T03:20:06.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.914+0000 7f63a9d9b700 1 -- 172.21.15.101:0/3524488369 learned_addr learned my addr 172.21.15.101:0/3524488369 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:06.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.914+0000 7f63aad9d700 1 --2- 172.21.15.101:0/3524488369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b176d1aad0 0x55b176d76f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:06.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.914+0000 7f63a9d9b700 1 -- 172.21.15.101:0/3524488369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b176d1aad0 msgr2=0x55b176d76f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:06.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.914+0000 7f63a9d9b700 1 --2- 172.21.15.101:0/3524488369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b176d1aad0 0x55b176d76f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.914+0000 7f63a9d9b700 1 -- 172.21.15.101:0/3524488369 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b176d0f2d0 msgr2=0x55b176d720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:06.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.914+0000 7f63a9d9b700 1 --2- 172.21.15.101:0/3524488369 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b176d0f2d0 0x55b176d720b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.916 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.914+0000 7f63a9d9b700 1 -- 172.21.15.101:0/3524488369 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b176a78c60 con 0x55b176d0fbe0 2023-07-18T03:20:06.916 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.914+0000 7f63aa59c700 1 --2- 172.21.15.101:0/3524488369 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b176d0f2d0 0x55b176d720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:20:06.916 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.914+0000 7f63a9d9b700 1 --2- 172.21.15.101:0/3524488369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b176d0fbe0 0x55b176d725f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f63a0005c20 tx=0x7f63a0005d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc52c6af6b9c9f4e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:06.916 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.914+0000 7f6391ffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:20:06.916 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.915+0000 7f6391ffb700 1 -- 172.21.15.101:0/3524488369 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a000e590 con 0x55b176d0fbe0 2023-07-18T03:20:06.916 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.915+0000 7f63c4573b80 1 -- 172.21.15.101:0/3524488369 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63a0019c50 con 0x55b176d0fbe0 2023-07-18T03:20:06.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.915+0000 7f6391ffb700 1 -- 172.21.15.101:0/3524488369 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f63a0029040 con 0x55b176d0fbe0 2023-07-18T03:20:06.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.915+0000 7f6391ffb700 1 -- 172.21.15.101:0/3524488369 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63a000e980 con 0x55b176d0fbe0 2023-07-18T03:20:06.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.915+0000 7f63c4573b80 10 client.28132 fetch_fsmap learned FSMap version 33 2023-07-18T03:20:06.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.915+0000 7f63c4573b80 1 -- 172.21.15.101:0/3524488369 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55b176a8e2b0 con 0x55b176d0fbe0 2023-07-18T03:20:06.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.915+0000 7f6391ffb700 1 -- 172.21.15.101:0/3524488369 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63a0019c50 con 0x55b176d0fbe0 2023-07-18T03:20:06.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.915+0000 7f6391ffb700 1 -- 172.21.15.101:0/3524488369 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f63a0030020 con 0x55b176d0fbe0 2023-07-18T03:20:06.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.916+0000 7f63c4573b80 10 client.28132 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:20:06.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.916+0000 7f63c4573b80 10 client.28132 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:20:06.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.916+0000 7f63c4573b80 20 client.28132 populate_metadata read hostname 'smithi101' 2023-07-18T03:20:06.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.916+0000 7f63c4573b80 1 --2- 172.21.15.101:0/3524488369 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55b176d7eec0 0x55b176d812d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:06.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.916+0000 7f63c4573b80 4 client.28132 mds_command: new command op to 24541 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:20:06.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.916+0000 7f63c4573b80 1 -- 172.21.15.101:0/3524488369 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b176c1d050 con 0x55b176d7eec0 2023-07-18T03:20:06.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.916+0000 7f63aa59c700 1 --2- 172.21.15.101:0/3524488369 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55b176d7eec0 0x55b176d812d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:06.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.916+0000 7f6391ffb700 10 client.28132 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:20:06.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.916+0000 7f63aa59c700 1 --2- 172.21.15.101:0/3524488369 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55b176d7eec0 0x55b176d812d0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dc3d472a8524c42f server_cookie=a0745805eaaef3dd in_seq=0 out_seq=0 2023-07-18T03:20:06.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.916+0000 7f6391ffb700 1 -- 172.21.15.101:0/3524488369 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b176c1d050 con 0x55b176d7eec0 2023-07-18T03:20:06.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.916+0000 7f6391ffb700 10 client.28132 handle_command_reply: tid=0 2023-07-18T03:20:06.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.916+0000 7f63c4573b80 1 client.28132 shutdown 2023-07-18T03:20:06.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.917+0000 7f63c4573b80 1 -- 172.21.15.101:0/3524488369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b176d0fbe0 msgr2=0x55b176d725f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:06.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.917+0000 7f63c4573b80 1 --2- 172.21.15.101:0/3524488369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b176d0fbe0 0x55b176d725f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f63a0005c20 tx=0x7f63a0005d00 comp rx=0 tx=0).stop 2023-07-18T03:20:06.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.917+0000 7f63c4573b80 1 -- 172.21.15.101:0/3524488369 shutdown_connections 2023-07-18T03:20:06.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.917+0000 7f63c4573b80 1 --2- 172.21.15.101:0/3524488369 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55b176d7eec0 0x55b176d812d0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.917+0000 7f63c4573b80 1 --2- 172.21.15.101:0/3524488369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b176d1aad0 0x55b176d76f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.917+0000 7f6391ffb700 0 client.28132 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:20:06.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.917+0000 7f63c4573b80 1 --2- 172.21.15.101:0/3524488369 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b176d0f2d0 0x55b176d720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.917+0000 7f63c4573b80 1 --2- 172.21.15.101:0/3524488369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b176d0fbe0 0x55b176d725f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.917+0000 7f63c4573b80 1 -- 172.21.15.101:0/3524488369 >> 172.21.15.101:0/3524488369 conn(0x55b176d05da0 msgr2=0x55b176d111a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:06.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.917+0000 7f63c4573b80 1 -- 172.21.15.101:0/3524488369 shutdown_connections 2023-07-18T03:20:06.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.917+0000 7f63c4573b80 1 -- 172.21.15.101:0/3524488369 wait complete. 2023-07-18T03:20:06.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.917+0000 7f63c4573b80 20 client.28132 trim_cache size 0 max 16384 2023-07-18T03:20:06.922 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:20:06.922 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.918+0000 7f63b0bff700 1 -- 172.21.15.101:0/4142560788 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f63940a12c0 msgr2=0x7f63940a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:06.923 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.918+0000 7f63b0bff700 1 --2- 172.21.15.101:0/4142560788 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f63940a12c0 0x7f63940a3770 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f63a40127f0 tx=0x7f63a400b040 comp rx=0 tx=0).stop 2023-07-18T03:20:06.923 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.918+0000 7f63b0bff700 1 -- 172.21.15.101:0/4142560788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63ac0ffae0 msgr2=0x7f63ac197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:06.923 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.918+0000 7f63b0bff700 1 --2- 172.21.15.101:0/4142560788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63ac0ffae0 0x7f63ac197df0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f639c000c00 tx=0x7f639c005740 comp rx=0 tx=0).stop 2023-07-18T03:20:06.923 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.918+0000 7f63b0bff700 1 -- 172.21.15.101:0/4142560788 shutdown_connections 2023-07-18T03:20:06.923 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.918+0000 7f63b0bff700 1 --2- 172.21.15.101:0/4142560788 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f63940a12c0 0x7f63940a3770 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.923 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.918+0000 7f63b0bff700 1 --2- 172.21.15.101:0/4142560788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63ac0ffae0 0x7f63ac197df0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.923 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.918+0000 7f63b0bff700 1 --2- 172.21.15.101:0/4142560788 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63ac105af0 0x7f63ac19c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.924 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.918+0000 7f63b0bff700 1 --2- 172.21.15.101:0/4142560788 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63ac100490 0x7f63ac198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:06.924 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.918+0000 7f63b0bff700 1 -- 172.21.15.101:0/4142560788 >> 172.21.15.101:0/4142560788 conn(0x7f63ac0fb5e0 msgr2=0x7f63ac0fbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:06.924 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.918+0000 7f63b0bff700 1 -- 172.21.15.101:0/4142560788 shutdown_connections 2023-07-18T03:20:06.924 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:06.918+0000 7f63b0bff700 1 -- 172.21.15.101:0/4142560788 wait complete. 2023-07-18T03:20:06.933 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-18T03:20:07.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.138+0000 7f0f7e9ff700 1 -- 172.21.15.101:0/817481993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f780ffad0 msgr2=0x7f0f780fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:07.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.138+0000 7f0f7e9ff700 1 --2- 172.21.15.101:0/817481993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f780ffad0 0x7f0f780fff40 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c009990 tx=0x7f0f6c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.138+0000 7f0f7e9ff700 1 -- 172.21.15.101:0/817481993 shutdown_connections 2023-07-18T03:20:07.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.138+0000 7f0f7e9ff700 1 --2- 172.21.15.101:0/817481993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f78100480 0x7f0f7810c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.138+0000 7f0f7e9ff700 1 --2- 172.21.15.101:0/817481993 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0f78105ae0 0x7f0f78105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.138+0000 7f0f7e9ff700 1 --2- 172.21.15.101:0/817481993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f780ffad0 0x7f0f780fff40 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.138+0000 7f0f7e9ff700 1 -- 172.21.15.101:0/817481993 >> 172.21.15.101:0/817481993 conn(0x7f0f780fb5f0 msgr2=0x7f0f780fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:07.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.138+0000 7f0f7e9ff700 1 -- 172.21.15.101:0/817481993 shutdown_connections 2023-07-18T03:20:07.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.138+0000 7f0f7e9ff700 1 -- 172.21.15.101:0/817481993 wait complete. 2023-07-18T03:20:07.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.139+0000 7f0f7e9ff700 1 Processor -- start 2023-07-18T03:20:07.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.139+0000 7f0f7e9ff700 1 -- start start 2023-07-18T03:20:07.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.140+0000 7f0f7e9ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f780ffad0 0x7f0f7819b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:07.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.140+0000 7f0f7e9ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f78100480 0x7f0f7819bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:07.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.140+0000 7f0f7e9ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0f78105ae0 0x7f0f78195a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:07.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.140+0000 7f0f7e9ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f781960d0 con 0x7f0f78105ae0 2023-07-18T03:20:07.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.140+0000 7f0f7e9ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f78196240 con 0x7f0f78100480 2023-07-18T03:20:07.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.140+0000 7f0f7e9ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f78196400 con 0x7f0f780ffad0 2023-07-18T03:20:07.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.140+0000 7f0f77fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f780ffad0 0x7f0f7819b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:07.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.140+0000 7f0f7cf9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0f78105ae0 0x7f0f78195a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:07.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.140+0000 7f0f77fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f780ffad0 0x7f0f7819b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:38526/0 (socket says 172.21.15.101:38526) 2023-07-18T03:20:07.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.140+0000 7f0f77fff700 1 -- 172.21.15.101:0/2048264706 learned_addr learned my addr 172.21.15.101:0/2048264706 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:07.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.140+0000 7f0f777fe700 1 --2- 172.21.15.101:0/2048264706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f78100480 0x7f0f7819bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:07.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.140+0000 7f0f77fff700 1 -- 172.21.15.101:0/2048264706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f78100480 msgr2=0x7f0f7819bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:07.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.140+0000 7f0f77fff700 1 --2- 172.21.15.101:0/2048264706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f78100480 0x7f0f7819bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.140+0000 7f0f77fff700 1 -- 172.21.15.101:0/2048264706 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0f78105ae0 msgr2=0x7f0f78195a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:07.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.140+0000 7f0f77fff700 1 --2- 172.21.15.101:0/2048264706 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0f78105ae0 0x7f0f78195a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.140+0000 7f0f77fff700 1 -- 172.21.15.101:0/2048264706 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f6c0095f0 con 0x7f0f780ffad0 2023-07-18T03:20:07.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.140+0000 7f0f7cf9c700 1 --2- 172.21.15.101:0/2048264706 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0f78105ae0 0x7f0f78195a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:20:07.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.141+0000 7f0f77fff700 1 --2- 172.21.15.101:0/2048264706 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f780ffad0 0x7f0f7819b9f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0f6800e9a0 tx=0x7f0f6800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:07.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.141+0000 7f0f757fa700 1 -- 172.21.15.101:0/2048264706 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f68009790 con 0x7f0f780ffad0 2023-07-18T03:20:07.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.141+0000 7f0f757fa700 1 -- 172.21.15.101:0/2048264706 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f68010430 con 0x7f0f780ffad0 2023-07-18T03:20:07.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.141+0000 7f0f7e9ff700 1 -- 172.21.15.101:0/2048264706 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f781966e0 con 0x7f0f780ffad0 2023-07-18T03:20:07.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.141+0000 7f0f757fa700 1 -- 172.21.15.101:0/2048264706 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f68018680 con 0x7f0f780ffad0 2023-07-18T03:20:07.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.141+0000 7f0f7e9ff700 1 -- 172.21.15.101:0/2048264706 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f781a29e0 con 0x7f0f780ffad0 2023-07-18T03:20:07.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.144+0000 7f0f757fa700 1 -- 172.21.15.101:0/2048264706 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f0f680105a0 con 0x7f0f780ffad0 2023-07-18T03:20:07.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.145+0000 7f0f757fa700 1 --2- 172.21.15.101:0/2048264706 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0f600a12c0 0x7f0f600a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:07.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.145+0000 7f0f922aab80 1 Processor -- start 2023-07-18T03:20:07.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.145+0000 7f0f757fa700 1 -- 172.21.15.101:0/2048264706 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f0f68014070 con 0x7f0f780ffad0 2023-07-18T03:20:07.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.146+0000 7f0f922aab80 1 -- start start 2023-07-18T03:20:07.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.146+0000 7f0f777fe700 1 --2- 172.21.15.101:0/2048264706 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0f600a12c0 0x7f0f600a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:07.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.146+0000 7f0f922aab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5570ff9112c0 0x5570ff90f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:07.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.146+0000 7f0f922aab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570ff90fc20 0x5570ff910090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:07.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.146+0000 7f0f922aab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570ff91f170 0x5570ff921550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:07.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.146+0000 7f0f922aab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570ff811cb0 con 0x5570ff9112c0 2023-07-18T03:20:07.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.146+0000 7f0f922aab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570ff864a40 con 0x5570ff91f170 2023-07-18T03:20:07.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.146+0000 7f0f922aab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570ff85ded0 con 0x5570ff90fc20 2023-07-18T03:20:07.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.146+0000 7f0f77fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5570ff9112c0 0x5570ff90f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:07.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.146+0000 7f0f77fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5570ff9112c0 0x5570ff90f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:54564/0 (socket says 172.21.15.101:54564) 2023-07-18T03:20:07.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.146+0000 7f0f7cf9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570ff91f170 0x5570ff921550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:07.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.146+0000 7f0f77fff700 1 -- 172.21.15.101:0/2020311660 learned_addr learned my addr 172.21.15.101:0/2020311660 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:07.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.146+0000 7f0f777fe700 1 --2- 172.21.15.101:0/2048264706 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0f600a12c0 0x7f0f600a3770 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0f78105210 tx=0x7f0f6c005fd0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:07.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.146+0000 7f0f777fe700 1 --2- 172.21.15.101:0/2020311660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570ff90fc20 0x5570ff910090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:07.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.147+0000 7f0f77fff700 1 -- 172.21.15.101:0/2020311660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570ff90fc20 msgr2=0x5570ff910090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:07.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.147+0000 7f0f77fff700 1 --2- 172.21.15.101:0/2020311660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570ff90fc20 0x5570ff910090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.147+0000 7f0f77fff700 1 -- 172.21.15.101:0/2020311660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570ff91f170 msgr2=0x5570ff921550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:07.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.147+0000 7f0f77fff700 1 --2- 172.21.15.101:0/2020311660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570ff91f170 0x5570ff921550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.147+0000 7f0f77fff700 1 -- 172.21.15.101:0/2020311660 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570ff68bde0 con 0x5570ff9112c0 2023-07-18T03:20:07.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.147+0000 7f0f7cf9c700 1 --2- 172.21.15.101:0/2020311660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570ff91f170 0x5570ff921550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:20:07.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.147+0000 7f0f77fff700 1 --2- 172.21.15.101:0/2020311660 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5570ff9112c0 0x5570ff90f6e0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f0f68096990 tx=0x7f0f68099970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a81c98f2ba8b073a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:07.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.147+0000 7f0f5e7fc700 1 -- 172.21.15.101:0/2020311660 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f680a2370 con 0x5570ff9112c0 2023-07-18T03:20:07.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.147+0000 7f0f5e7fc700 1 -- 172.21.15.101:0/2020311660 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f68010880 con 0x5570ff9112c0 2023-07-18T03:20:07.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.147+0000 7f0f5e7fc700 1 -- 172.21.15.101:0/2020311660 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f680a1ee0 con 0x5570ff9112c0 2023-07-18T03:20:07.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.148+0000 7f0f922aab80 1 -- 172.21.15.101:0/2020311660 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5570ff9112c0 msgr2=0x5570ff90f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:07.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.148+0000 7f0f922aab80 1 --2- 172.21.15.101:0/2020311660 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5570ff9112c0 0x5570ff90f6e0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f0f68096990 tx=0x7f0f68099970 comp rx=0 tx=0).stop 2023-07-18T03:20:07.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.148+0000 7f0f922aab80 1 -- 172.21.15.101:0/2020311660 shutdown_connections 2023-07-18T03:20:07.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.148+0000 7f0f922aab80 1 --2- 172.21.15.101:0/2020311660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570ff91f170 0x5570ff921550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.148+0000 7f0f922aab80 1 --2- 172.21.15.101:0/2020311660 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5570ff9112c0 0x5570ff90f6e0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.148+0000 7f0f922aab80 1 --2- 172.21.15.101:0/2020311660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570ff90fc20 0x5570ff910090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.148+0000 7f0f922aab80 1 -- 172.21.15.101:0/2020311660 >> 172.21.15.101:0/2020311660 conn(0x5570ff901da0 msgr2=0x5570ff9076a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:07.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.148+0000 7f0f922aab80 1 -- 172.21.15.101:0/2020311660 shutdown_connections 2023-07-18T03:20:07.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.148+0000 7f0f922aab80 1 -- 172.21.15.101:0/2020311660 wait complete. 2023-07-18T03:20:07.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.148+0000 7f0f922aab80 1 -- start start 2023-07-18T03:20:07.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.149+0000 7f0f922aab80 1 Processor -- start 2023-07-18T03:20:07.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.149+0000 7f0f922aab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5570ff90fc20 0x5570ff90a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:07.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.149+0000 7f0f922aab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570ff9112c0 0x5570ff90a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:07.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.149+0000 7f0f922aab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570ff91f170 0x5570ff90aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:07.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.149+0000 7f0f922aab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570ff7fdac0 con 0x5570ff90fc20 2023-07-18T03:20:07.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.149+0000 7f0f922aab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570ff7fd7c0 con 0x5570ff91f170 2023-07-18T03:20:07.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.149+0000 7f0f922aab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570ff6e1920 con 0x5570ff9112c0 2023-07-18T03:20:07.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.149+0000 7f0f922aab80 10 client.0 _ll_register_callbacks cb 0x5570ff7fb760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:20:07.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.149+0000 7f0f7cf9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570ff91f170 0x5570ff90aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:07.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.149+0000 7f0f77fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5570ff90fc20 0x5570ff90a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:07.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.149+0000 7f0f77fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5570ff90fc20 0x5570ff90a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:54572/0 (socket says 172.21.15.101:54572) 2023-07-18T03:20:07.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.149+0000 7f0f77fff700 1 -- 172.21.15.101:0/2743349768 learned_addr learned my addr 172.21.15.101:0/2743349768 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:07.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.149+0000 7f0f7cf9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570ff91f170 0x5570ff90aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:43250/0 (socket says 172.21.15.101:43250) 2023-07-18T03:20:07.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.149+0000 7f0f777fe700 1 --2- 172.21.15.101:0/2743349768 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570ff9112c0 0x5570ff90a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:07.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.150+0000 7f0f77fff700 1 -- 172.21.15.101:0/2743349768 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570ff9112c0 msgr2=0x5570ff90a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:07.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.150+0000 7f0f77fff700 1 --2- 172.21.15.101:0/2743349768 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570ff9112c0 0x5570ff90a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.150+0000 7f0f77fff700 1 -- 172.21.15.101:0/2743349768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570ff91f170 msgr2=0x5570ff90aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:07.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.150+0000 7f0f77fff700 1 --2- 172.21.15.101:0/2743349768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570ff91f170 0x5570ff90aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.150+0000 7f0f77fff700 1 -- 172.21.15.101:0/2743349768 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f70009db0 con 0x5570ff90fc20 2023-07-18T03:20:07.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.150+0000 7f0f7cf9c700 1 --2- 172.21.15.101:0/2743349768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570ff91f170 0x5570ff90aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:20:07.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.150+0000 7f0f77fff700 1 --2- 172.21.15.101:0/2743349768 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5570ff90fc20 0x5570ff90a030 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f0f68099c20 tx=0x7f0f68010a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e12fabb17ca72eee server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:07.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.150+0000 7f0f577fe700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:20:07.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.150+0000 7f0f577fe700 1 -- 172.21.15.101:0/2743349768 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f680a2370 con 0x5570ff90fc20 2023-07-18T03:20:07.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.150+0000 7f0f577fe700 1 -- 172.21.15.101:0/2743349768 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f0f6809a390 con 0x5570ff90fc20 2023-07-18T03:20:07.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.150+0000 7f0f922aab80 1 -- 172.21.15.101:0/2743349768 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5570ff7b74f0 con 0x5570ff90fc20 2023-07-18T03:20:07.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.150+0000 7f0f577fe700 1 -- 172.21.15.101:0/2743349768 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f680ca260 con 0x5570ff90fc20 2023-07-18T03:20:07.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.150+0000 7f0f922aab80 10 client.18066 fetch_fsmap learned FSMap version 33 2023-07-18T03:20:07.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.150+0000 7f0f922aab80 1 -- 172.21.15.101:0/2743349768 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f0f781a29e0 con 0x5570ff90fc20 2023-07-18T03:20:07.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.151+0000 7f0f577fe700 1 -- 172.21.15.101:0/2743349768 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5570ff7b74f0 con 0x5570ff90fc20 2023-07-18T03:20:07.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.151+0000 7f0f577fe700 1 -- 172.21.15.101:0/2743349768 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0f68098e80 con 0x5570ff90fc20 2023-07-18T03:20:07.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.151+0000 7f0f922aab80 10 client.18066 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:20:07.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.151+0000 7f0f922aab80 10 client.18066 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:20:07.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.151+0000 7f0f922aab80 20 client.18066 populate_metadata read hostname 'smithi101' 2023-07-18T03:20:07.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.151+0000 7f0f922aab80 1 --2- 172.21.15.101:0/2743349768 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5570ff92e830 0x5570ff930c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:07.156 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.151+0000 7f0f922aab80 4 client.18066 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:20:07.156 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.151+0000 7f0f922aab80 1 -- 172.21.15.101:0/2743349768 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5570ff856eb0 con 0x5570ff92e830 2023-07-18T03:20:07.156 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.151+0000 7f0f777fe700 1 --2- 172.21.15.101:0/2743349768 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5570ff92e830 0x5570ff930c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:07.156 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.152+0000 7f0f577fe700 10 client.18066 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:20:07.156 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.152+0000 7f0f777fe700 1 --2- 172.21.15.101:0/2743349768 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5570ff92e830 0x5570ff930c40 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e79a1f6450a0fe9b server_cookie=5d2fe4207195f914 in_seq=0 out_seq=0 2023-07-18T03:20:07.156 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.152+0000 7f0f577fe700 1 -- 172.21.15.101:0/2743349768 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5570ff856eb0 con 0x5570ff92e830 2023-07-18T03:20:07.156 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.153+0000 7f0f577fe700 10 client.18066 handle_command_reply: tid=0 2023-07-18T03:20:07.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.153+0000 7f0f922aab80 1 client.18066 shutdown 2023-07-18T03:20:07.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.153+0000 7f0f922aab80 1 -- 172.21.15.101:0/2743349768 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5570ff90fc20 msgr2=0x5570ff90a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:07.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.153+0000 7f0f922aab80 1 --2- 172.21.15.101:0/2743349768 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5570ff90fc20 0x5570ff90a030 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f0f68099c20 tx=0x7f0f68010a60 comp rx=0 tx=0).stop 2023-07-18T03:20:07.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.153+0000 7f0f922aab80 1 -- 172.21.15.101:0/2743349768 shutdown_connections 2023-07-18T03:20:07.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.153+0000 7f0f922aab80 1 --2- 172.21.15.101:0/2743349768 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5570ff92e830 0x5570ff930c40 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.153+0000 7f0f922aab80 1 --2- 172.21.15.101:0/2743349768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570ff91f170 0x5570ff90aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.153+0000 7f0f922aab80 1 --2- 172.21.15.101:0/2743349768 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5570ff90fc20 0x5570ff90a030 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.153+0000 7f0f922aab80 1 --2- 172.21.15.101:0/2743349768 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570ff9112c0 0x5570ff90a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.153+0000 7f0f577fe700 0 client.18066 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:20:07.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.153+0000 7f0f922aab80 1 -- 172.21.15.101:0/2743349768 >> 172.21.15.101:0/2743349768 conn(0x5570ff901da0 msgr2=0x5570ff907290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:07.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.153+0000 7f0f922aab80 1 -- 172.21.15.101:0/2743349768 shutdown_connections 2023-07-18T03:20:07.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.153+0000 7f0f922aab80 1 -- 172.21.15.101:0/2743349768 wait complete. 2023-07-18T03:20:07.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.153+0000 7f0f922aab80 20 client.18066 trim_cache size 0 max 16384 2023-07-18T03:20:07.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.166+0000 7f0f922aab80 1 Processor -- start 2023-07-18T03:20:07.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.166+0000 7f0f922aab80 1 -- start start 2023-07-18T03:20:07.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.167+0000 7f0f922aab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570ff90fc20 0x5570ff905e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:07.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.167+0000 7f0f922aab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570ff9112c0 0x5570ff90a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:07.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.167+0000 7f0f922aab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5570ff91f170 0x5570ff91e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:07.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.167+0000 7f0f922aab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570ff8fef60 con 0x5570ff91f170 2023-07-18T03:20:07.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.167+0000 7f0f922aab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570ff9078c0 con 0x5570ff9112c0 2023-07-18T03:20:07.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.167+0000 7f0f922aab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570ff906350 con 0x5570ff90fc20 2023-07-18T03:20:07.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.167+0000 7f0f7cf9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5570ff91f170 0x5570ff91e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:07.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.167+0000 7f0f77fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570ff90fc20 0x5570ff905e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:07.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.167+0000 7f0f7cf9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5570ff91f170 0x5570ff91e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:54584/0 (socket says 172.21.15.101:54584) 2023-07-18T03:20:07.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.167+0000 7f0f7cf9c700 1 -- 172.21.15.101:0/1642470975 learned_addr learned my addr 172.21.15.101:0/1642470975 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:07.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.167+0000 7f0f777fe700 1 --2- 172.21.15.101:0/1642470975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570ff9112c0 0x5570ff90a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:07.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.167+0000 7f0f7cf9c700 1 -- 172.21.15.101:0/1642470975 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570ff90fc20 msgr2=0x5570ff905e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:07.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.167+0000 7f0f7cf9c700 1 --2- 172.21.15.101:0/1642470975 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570ff90fc20 0x5570ff905e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.167+0000 7f0f7cf9c700 1 -- 172.21.15.101:0/1642470975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570ff9112c0 msgr2=0x5570ff90a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:07.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.167+0000 7f0f7cf9c700 1 --2- 172.21.15.101:0/1642470975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570ff9112c0 0x5570ff90a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.167+0000 7f0f7cf9c700 1 -- 172.21.15.101:0/1642470975 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570ff9094b0 con 0x5570ff91f170 2023-07-18T03:20:07.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.167+0000 7f0f77fff700 1 --2- 172.21.15.101:0/1642470975 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570ff90fc20 0x5570ff905e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:20:07.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.167+0000 7f0f777fe700 1 --2- 172.21.15.101:0/1642470975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570ff9112c0 0x5570ff90a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:20:07.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.167+0000 7f0f7cf9c700 1 --2- 172.21.15.101:0/1642470975 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5570ff91f170 0x5570ff91e230 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f0f70004200 tx=0x7f0f70004230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=334efe0840d47fe3 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:07.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.168+0000 7f0f5effd700 1 -- 172.21.15.101:0/1642470975 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f70007700 con 0x5570ff91f170 2023-07-18T03:20:07.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.168+0000 7f0f5effd700 1 -- 172.21.15.101:0/1642470975 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f7000f040 con 0x5570ff91f170 2023-07-18T03:20:07.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.168+0000 7f0f5effd700 1 -- 172.21.15.101:0/1642470975 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f70020c90 con 0x5570ff91f170 2023-07-18T03:20:07.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.168+0000 7f0f922aab80 1 -- 172.21.15.101:0/1642470975 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5570ff91f170 msgr2=0x5570ff91e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:07.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.168+0000 7f0f922aab80 1 --2- 172.21.15.101:0/1642470975 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5570ff91f170 0x5570ff91e230 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f0f70004200 tx=0x7f0f70004230 comp rx=0 tx=0).stop 2023-07-18T03:20:07.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.168+0000 7f0f922aab80 1 -- 172.21.15.101:0/1642470975 shutdown_connections 2023-07-18T03:20:07.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.168+0000 7f0f922aab80 1 --2- 172.21.15.101:0/1642470975 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5570ff91f170 0x5570ff91e230 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.168+0000 7f0f922aab80 1 --2- 172.21.15.101:0/1642470975 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570ff90fc20 0x5570ff905e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.168+0000 7f0f922aab80 1 --2- 172.21.15.101:0/1642470975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570ff9112c0 0x5570ff90a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.168+0000 7f0f922aab80 1 -- 172.21.15.101:0/1642470975 >> 172.21.15.101:0/1642470975 conn(0x5570ff901da0 msgr2=0x5570ff92b020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:07.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.168+0000 7f0f922aab80 1 -- 172.21.15.101:0/1642470975 shutdown_connections 2023-07-18T03:20:07.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.168+0000 7f0f922aab80 1 -- 172.21.15.101:0/1642470975 wait complete. 2023-07-18T03:20:07.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.169+0000 7f0f922aab80 1 -- start start 2023-07-18T03:20:07.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.169+0000 7f0f922aab80 1 Processor -- start 2023-07-18T03:20:07.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.169+0000 7f0f922aab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570ff90fc20 0x5570ff923ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:07.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.169+0000 7f0f922aab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570ff9112c0 0x5570ff924410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:07.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.169+0000 7f0f922aab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5570ff91f170 0x5570ff97b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:07.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.170+0000 7f0f922aab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570ff97bc00 con 0x5570ff91f170 2023-07-18T03:20:07.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.170+0000 7f0f922aab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570ff97bd40 con 0x5570ff9112c0 2023-07-18T03:20:07.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.170+0000 7f0f922aab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570ff97bf00 con 0x5570ff90fc20 2023-07-18T03:20:07.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.170+0000 7f0f922aab80 10 client.0 _ll_register_callbacks cb 0x7f0f68099b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:20:07.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.170+0000 7f0f7cf9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5570ff91f170 0x5570ff97b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:07.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.170+0000 7f0f777fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570ff9112c0 0x5570ff924410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:07.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.170+0000 7f0f77fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570ff90fc20 0x5570ff923ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:07.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.170+0000 7f0f777fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570ff9112c0 0x5570ff924410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:43272/0 (socket says 172.21.15.101:43272) 2023-07-18T03:20:07.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.170+0000 7f0f777fe700 1 -- 172.21.15.101:0/4185889787 learned_addr learned my addr 172.21.15.101:0/4185889787 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:07.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.170+0000 7f0f7cf9c700 1 --2- 172.21.15.101:0/4185889787 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5570ff91f170 0x5570ff97b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:54600/0 (socket says 172.21.15.101:54600) 2023-07-18T03:20:07.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.170+0000 7f0f7cf9c700 1 -- 172.21.15.101:0/4185889787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570ff90fc20 msgr2=0x5570ff923ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:07.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.170+0000 7f0f7cf9c700 1 --2- 172.21.15.101:0/4185889787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570ff90fc20 0x5570ff923ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.170+0000 7f0f7cf9c700 1 -- 172.21.15.101:0/4185889787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570ff9112c0 msgr2=0x5570ff924410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:07.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.170+0000 7f0f7cf9c700 1 --2- 172.21.15.101:0/4185889787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570ff9112c0 0x5570ff924410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.170+0000 7f0f7cf9c700 1 -- 172.21.15.101:0/4185889787 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f781a29e0 con 0x5570ff91f170 2023-07-18T03:20:07.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.170+0000 7f0f7cf9c700 1 --2- 172.21.15.101:0/4185889787 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5570ff91f170 0x5570ff97b6c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f0f70004a60 tx=0x7f0f70007730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=237f95638c53c0f9 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:07.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.170+0000 7f0f577fe700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:20:07.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.171+0000 7f0f577fe700 1 -- 172.21.15.101:0/4185889787 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f700205f0 con 0x5570ff91f170 2023-07-18T03:20:07.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.171+0000 7f0f577fe700 1 -- 172.21.15.101:0/4185889787 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f0f70020e40 con 0x5570ff91f170 2023-07-18T03:20:07.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.171+0000 7f0f922aab80 1 -- 172.21.15.101:0/4185889787 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0f68010880 con 0x5570ff91f170 2023-07-18T03:20:07.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.171+0000 7f0f577fe700 1 -- 172.21.15.101:0/4185889787 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f700077a0 con 0x5570ff91f170 2023-07-18T03:20:07.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.171+0000 7f0f922aab80 10 client.18072 fetch_fsmap learned FSMap version 33 2023-07-18T03:20:07.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.171+0000 7f0f922aab80 1 -- 172.21.15.101:0/4185889787 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f0f70003e70 con 0x5570ff91f170 2023-07-18T03:20:07.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.172+0000 7f0f577fe700 1 -- 172.21.15.101:0/4185889787 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f68010880 con 0x5570ff91f170 2023-07-18T03:20:07.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.172+0000 7f0f577fe700 1 -- 172.21.15.101:0/4185889787 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0f70005490 con 0x5570ff91f170 2023-07-18T03:20:07.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.172+0000 7f0f922aab80 10 client.18072 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:20:07.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.172+0000 7f0f922aab80 10 client.18072 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:20:07.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.172+0000 7f0f922aab80 20 client.18072 populate_metadata read hostname 'smithi101' 2023-07-18T03:20:07.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.172+0000 7f0f922aab80 1 --2- 172.21.15.101:0/4185889787 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5570ff9835b0 0x5570ff9859c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:07.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.172+0000 7f0f922aab80 4 client.18072 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:20:07.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.172+0000 7f0f922aab80 1 -- 172.21.15.101:0/4185889787 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5570ff819050 con 0x5570ff9835b0 2023-07-18T03:20:07.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.172+0000 7f0f77fff700 1 --2- 172.21.15.101:0/4185889787 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5570ff9835b0 0x5570ff9859c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:07.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.173+0000 7f0f577fe700 10 client.18072 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:20:07.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.173+0000 7f0f77fff700 1 --2- 172.21.15.101:0/4185889787 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5570ff9835b0 0x5570ff9859c0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1ad64ea36c32d03 server_cookie=54a6a6233f318e31 in_seq=0 out_seq=0 2023-07-18T03:20:07.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.173+0000 7f0f577fe700 1 -- 172.21.15.101:0/4185889787 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5570ff819050 con 0x5570ff9835b0 2023-07-18T03:20:07.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.173+0000 7f0f577fe700 10 client.18072 handle_command_reply: tid=0 2023-07-18T03:20:07.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.173+0000 7f0f922aab80 1 client.18072 shutdown 2023-07-18T03:20:07.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.173+0000 7f0f922aab80 1 -- 172.21.15.101:0/4185889787 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5570ff91f170 msgr2=0x5570ff97b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:07.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.173+0000 7f0f922aab80 1 --2- 172.21.15.101:0/4185889787 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5570ff91f170 0x5570ff97b6c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f0f70004a60 tx=0x7f0f70007730 comp rx=0 tx=0).stop 2023-07-18T03:20:07.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.173+0000 7f0f922aab80 1 -- 172.21.15.101:0/4185889787 shutdown_connections 2023-07-18T03:20:07.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.173+0000 7f0f922aab80 1 --2- 172.21.15.101:0/4185889787 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5570ff9835b0 0x5570ff9859c0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.173+0000 7f0f922aab80 1 --2- 172.21.15.101:0/4185889787 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5570ff91f170 0x5570ff97b6c0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.173+0000 7f0f922aab80 1 --2- 172.21.15.101:0/4185889787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570ff90fc20 0x5570ff923ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.173+0000 7f0f577fe700 0 client.18072 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:20:07.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.173+0000 7f0f922aab80 1 --2- 172.21.15.101:0/4185889787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5570ff9112c0 0x5570ff924410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.173+0000 7f0f922aab80 1 -- 172.21.15.101:0/4185889787 >> 172.21.15.101:0/4185889787 conn(0x5570ff901da0 msgr2=0x5570ff91e230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:07.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.173+0000 7f0f922aab80 1 -- 172.21.15.101:0/4185889787 shutdown_connections 2023-07-18T03:20:07.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.173+0000 7f0f922aab80 1 -- 172.21.15.101:0/4185889787 wait complete. 2023-07-18T03:20:07.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.173+0000 7f0f922aab80 20 client.18072 trim_cache size 0 max 16384 2023-07-18T03:20:07.180 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:20:07.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.174+0000 7f0f7e9ff700 1 -- 172.21.15.101:0/2048264706 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0f600a12c0 msgr2=0x7f0f600a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:07.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.174+0000 7f0f7e9ff700 1 --2- 172.21.15.101:0/2048264706 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0f600a12c0 0x7f0f600a3770 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0f78105210 tx=0x7f0f6c005fd0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.174+0000 7f0f7e9ff700 1 -- 172.21.15.101:0/2048264706 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f780ffad0 msgr2=0x7f0f7819b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:07.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.174+0000 7f0f7e9ff700 1 --2- 172.21.15.101:0/2048264706 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f780ffad0 0x7f0f7819b9f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0f6800e9a0 tx=0x7f0f6800ecb0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.174+0000 7f0f7e9ff700 1 -- 172.21.15.101:0/2048264706 shutdown_connections 2023-07-18T03:20:07.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.174+0000 7f0f7e9ff700 1 --2- 172.21.15.101:0/2048264706 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0f78105ae0 0x7f0f78195a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.174+0000 7f0f7e9ff700 1 --2- 172.21.15.101:0/2048264706 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f780ffad0 0x7f0f7819b9f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.174+0000 7f0f7e9ff700 1 --2- 172.21.15.101:0/2048264706 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0f600a12c0 0x7f0f600a3770 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.174+0000 7f0f7e9ff700 1 --2- 172.21.15.101:0/2048264706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f78100480 0x7f0f7819bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.174+0000 7f0f7e9ff700 1 -- 172.21.15.101:0/2048264706 >> 172.21.15.101:0/2048264706 conn(0x7f0f780fb5f0 msgr2=0x7f0f780fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:07.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.174+0000 7f0f7e9ff700 1 -- 172.21.15.101:0/2048264706 shutdown_connections 2023-07-18T03:20:07.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.174+0000 7f0f7e9ff700 1 -- 172.21.15.101:0/2048264706 wait complete. 2023-07-18T03:20:07.185 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:07 smithi101 ceph-mon[126379]: pgmap v391: 129 pgs: 129 active+clean; 620 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 137 op/s 2023-07-18T03:20:07.186 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:07 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/112214217' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:20:07.187 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-18T03:20:07.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.389+0000 7f58fdbff700 1 -- 172.21.15.101:0/2719853724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58f8100490 msgr2=0x7f58f810c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:07.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.389+0000 7f58fdbff700 1 --2- 172.21.15.101:0/2719853724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58f8100490 0x7f58f810c920 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f58f000b0a0 tx=0x7f58f000b3b0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.390+0000 7f58fdbff700 1 -- 172.21.15.101:0/2719853724 shutdown_connections 2023-07-18T03:20:07.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.390+0000 7f58fdbff700 1 --2- 172.21.15.101:0/2719853724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58f8100490 0x7f58f810c920 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.390+0000 7f58fdbff700 1 --2- 172.21.15.101:0/2719853724 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f58f8105af0 0x7f58f8105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.390+0000 7f58fdbff700 1 --2- 172.21.15.101:0/2719853724 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58f80ffae0 0x7f58f80fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.390+0000 7f58fdbff700 1 -- 172.21.15.101:0/2719853724 >> 172.21.15.101:0/2719853724 conn(0x7f58f80fb5e0 msgr2=0x7f58f80fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:07.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.390+0000 7f58fdbff700 1 -- 172.21.15.101:0/2719853724 shutdown_connections 2023-07-18T03:20:07.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.390+0000 7f58fdbff700 1 -- 172.21.15.101:0/2719853724 wait complete. 2023-07-18T03:20:07.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.391+0000 7f58fdbff700 1 Processor -- start 2023-07-18T03:20:07.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.391+0000 7f58fdbff700 1 -- start start 2023-07-18T03:20:07.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.391+0000 7f58fdbff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f58f80ffae0 0x7f58f8072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:07.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.391+0000 7f58fdbff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58f8100490 0x7f58f8072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:07.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.391+0000 7f58fdbff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58f8105af0 0x7f58f81a3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:07.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.391+0000 7f58fdbff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f8073580 con 0x7f58f80ffae0 2023-07-18T03:20:07.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.391+0000 7f58fdbff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f81a43d0 con 0x7f58f8100490 2023-07-18T03:20:07.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.391+0000 7f58fdbff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f81a4510 con 0x7f58f8105af0 2023-07-18T03:20:07.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.391+0000 7f58f77fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f58f80ffae0 0x7f58f8072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:07.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.391+0000 7f58f77fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f58f80ffae0 0x7f58f8072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:54630/0 (socket says 172.21.15.101:54630) 2023-07-18T03:20:07.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.391+0000 7f58f77fe700 1 -- 172.21.15.101:0/4155746651 learned_addr learned my addr 172.21.15.101:0/4155746651 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:07.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.392+0000 7f58f7fff700 1 --2- 172.21.15.101:0/4155746651 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58f8105af0 0x7f58f81a3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:07.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.392+0000 7f58f6ffd700 1 --2- 172.21.15.101:0/4155746651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58f8100490 0x7f58f8072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:07.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.392+0000 7f58f77fe700 1 -- 172.21.15.101:0/4155746651 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58f8105af0 msgr2=0x7f58f81a3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:07.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.392+0000 7f58f77fe700 1 --2- 172.21.15.101:0/4155746651 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58f8105af0 0x7f58f81a3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.392+0000 7f58f77fe700 1 -- 172.21.15.101:0/4155746651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58f8100490 msgr2=0x7f58f8072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:07.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.392+0000 7f58f77fe700 1 --2- 172.21.15.101:0/4155746651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58f8100490 0x7f58f8072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.392+0000 7f58f77fe700 1 -- 172.21.15.101:0/4155746651 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f0009d00 con 0x7f58f80ffae0 2023-07-18T03:20:07.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.392+0000 7f58f77fe700 1 --2- 172.21.15.101:0/4155746651 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f58f80ffae0 0x7f58f8072810 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f58e800e9a0 tx=0x7f58e800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:07.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.392+0000 7f58f4ff9700 1 -- 172.21.15.101:0/4155746651 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e8009790 con 0x7f58f80ffae0 2023-07-18T03:20:07.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.392+0000 7f58fdbff700 1 -- 172.21.15.101:0/4155746651 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58f81a4770 con 0x7f58f80ffae0 2023-07-18T03:20:07.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.392+0000 7f58f4ff9700 1 -- 172.21.15.101:0/4155746651 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58e8010430 con 0x7f58f80ffae0 2023-07-18T03:20:07.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.392+0000 7f58f4ff9700 1 -- 172.21.15.101:0/4155746651 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e8018680 con 0x7f58f80ffae0 2023-07-18T03:20:07.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.392+0000 7f58fdbff700 1 -- 172.21.15.101:0/4155746651 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f81a4ca0 con 0x7f58f80ffae0 2023-07-18T03:20:07.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.396+0000 7f58f4ff9700 1 -- 172.21.15.101:0/4155746651 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f58e80105a0 con 0x7f58f80ffae0 2023-07-18T03:20:07.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.397+0000 7f59114c8b80 1 Processor -- start 2023-07-18T03:20:07.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.397+0000 7f59114c8b80 1 -- start start 2023-07-18T03:20:07.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.397+0000 7f58f4ff9700 1 --2- 172.21.15.101:0/4155746651 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f58e00a1340 0x7f58e00a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:07.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.397+0000 7f58f4ff9700 1 -- 172.21.15.101:0/4155746651 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f58e8014070 con 0x7f58f80ffae0 2023-07-18T03:20:07.394 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.397+0000 7f59114c8b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555b1cee2d0 0x5555b1cee6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:07.394 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.397+0000 7f59114c8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5555b1ceebe0 0x5555b1d01da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:07.394 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.397+0000 7f59114c8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555b1d022e0 0x5555b1d046c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:07.394 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.397+0000 7f58f7fff700 1 --2- 172.21.15.101:0/4155746651 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f58e00a1340 0x7f58e00a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:07.394 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.397+0000 7f59114c8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b1bf4cb0 con 0x5555b1ceebe0 2023-07-18T03:20:07.394 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.397+0000 7f59114c8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b1c47a40 con 0x5555b1d022e0 2023-07-18T03:20:07.394 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.397+0000 7f59114c8b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b1c40ed0 con 0x5555b1cee2d0 2023-07-18T03:20:07.395 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.398+0000 7f58f77fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555b1cee2d0 0x5555b1cee6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:07.395 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.398+0000 7f58f77fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555b1cee2d0 0x5555b1cee6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:38592/0 (socket says 172.21.15.101:38592) 2023-07-18T03:20:07.395 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.398+0000 7f58f77fe700 1 -- 172.21.15.101:0/1371982394 learned_addr learned my addr 172.21.15.101:0/1371982394 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:07.395 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.398+0000 7f58f6ffd700 1 --2- 172.21.15.101:0/1371982394 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5555b1ceebe0 0x5555b1d01da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:07.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.398+0000 7f58f7fff700 1 --2- 172.21.15.101:0/1371982394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555b1d022e0 0x5555b1d046c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:07.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.398+0000 7f58f77fe700 1 -- 172.21.15.101:0/1371982394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555b1d022e0 msgr2=0x5555b1d046c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:07.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.398+0000 7f58f77fe700 1 --2- 172.21.15.101:0/1371982394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555b1d022e0 0x5555b1d046c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.398+0000 7f58f77fe700 1 -- 172.21.15.101:0/1371982394 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5555b1ceebe0 msgr2=0x5555b1d01da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:07.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.398+0000 7f58f77fe700 1 --2- 172.21.15.101:0/1371982394 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5555b1ceebe0 0x5555b1d01da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.398+0000 7f58f77fe700 1 -- 172.21.15.101:0/1371982394 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555b1a6ede0 con 0x5555b1cee2d0 2023-07-18T03:20:07.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.398+0000 7f58f7fff700 1 --2- 172.21.15.101:0/4155746651 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f58e00a1340 0x7f58e00a37f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f58f8074920 tx=0x7f58f00095a0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:07.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.398+0000 7f58f77fe700 1 --2- 172.21.15.101:0/1371982394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555b1cee2d0 0x5555b1cee6a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f58e8096980 tx=0x7f58e80998f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d99d33ac590ad42 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:07.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.398+0000 7f58ddffb700 1 -- 172.21.15.101:0/1371982394 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e809df40 con 0x5555b1cee2d0 2023-07-18T03:20:07.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.398+0000 7f58ddffb700 1 -- 172.21.15.101:0/1371982394 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58e80a2360 con 0x5555b1cee2d0 2023-07-18T03:20:07.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.399+0000 7f58ddffb700 1 -- 172.21.15.101:0/1371982394 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e8099660 con 0x5555b1cee2d0 2023-07-18T03:20:07.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.399+0000 7f59114c8b80 1 -- 172.21.15.101:0/1371982394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555b1cee2d0 msgr2=0x5555b1cee6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:07.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.399+0000 7f59114c8b80 1 --2- 172.21.15.101:0/1371982394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555b1cee2d0 0x5555b1cee6a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f58e8096980 tx=0x7f58e80998f0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.399+0000 7f59114c8b80 1 -- 172.21.15.101:0/1371982394 shutdown_connections 2023-07-18T03:20:07.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.399+0000 7f59114c8b80 1 --2- 172.21.15.101:0/1371982394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555b1d022e0 0x5555b1d046c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.399+0000 7f59114c8b80 1 --2- 172.21.15.101:0/1371982394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555b1cee2d0 0x5555b1cee6a0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.399+0000 7f59114c8b80 1 --2- 172.21.15.101:0/1371982394 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5555b1ceebe0 0x5555b1d01da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.399+0000 7f59114c8b80 1 -- 172.21.15.101:0/1371982394 >> 172.21.15.101:0/1371982394 conn(0x5555b1ce4da0 msgr2=0x5555b1cea6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:07.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.399+0000 7f59114c8b80 1 -- 172.21.15.101:0/1371982394 shutdown_connections 2023-07-18T03:20:07.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.399+0000 7f59114c8b80 1 -- 172.21.15.101:0/1371982394 wait complete. 2023-07-18T03:20:07.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.400+0000 7f59114c8b80 1 -- start start 2023-07-18T03:20:07.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.400+0000 7f59114c8b80 1 Processor -- start 2023-07-18T03:20:07.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.401+0000 7f59114c8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5555b1cee2d0 0x5555b1d01d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:07.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.401+0000 7f59114c8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555b1ceebe0 0x5555b1cfefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:07.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.401+0000 7f59114c8b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555b1d022e0 0x5555b1cff700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:07.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.401+0000 7f59114c8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b1be0ac0 con 0x5555b1cee2d0 2023-07-18T03:20:07.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.401+0000 7f59114c8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b1be07c0 con 0x5555b1ceebe0 2023-07-18T03:20:07.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.401+0000 7f59114c8b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b1ac4920 con 0x5555b1d022e0 2023-07-18T03:20:07.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.401+0000 7f59114c8b80 10 client.0 _ll_register_callbacks cb 0x5555b1bde760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:20:07.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.401+0000 7f58f7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555b1d022e0 0x5555b1cff700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:07.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.401+0000 7f58f7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555b1d022e0 0x5555b1cff700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:38604/0 (socket says 172.21.15.101:38604) 2023-07-18T03:20:07.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.401+0000 7f58f7fff700 1 -- 172.21.15.101:0/2809402117 learned_addr learned my addr 172.21.15.101:0/2809402117 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:07.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.401+0000 7f58f77fe700 1 --2- 172.21.15.101:0/2809402117 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5555b1cee2d0 0x5555b1d01d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:07.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.401+0000 7f58f6ffd700 1 --2- 172.21.15.101:0/2809402117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555b1ceebe0 0x5555b1cfefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:07.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.401+0000 7f58f7fff700 1 -- 172.21.15.101:0/2809402117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555b1ceebe0 msgr2=0x5555b1cfefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:07.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.401+0000 7f58f7fff700 1 --2- 172.21.15.101:0/2809402117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555b1ceebe0 0x5555b1cfefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.401+0000 7f58f7fff700 1 -- 172.21.15.101:0/2809402117 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5555b1cee2d0 msgr2=0x5555b1d01d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:07.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.401+0000 7f58f7fff700 1 --2- 172.21.15.101:0/2809402117 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5555b1cee2d0 0x5555b1d01d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.401+0000 7f58f7fff700 1 -- 172.21.15.101:0/2809402117 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f81a4ca0 con 0x5555b1d022e0 2023-07-18T03:20:07.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.402+0000 7f58f7fff700 1 --2- 172.21.15.101:0/2809402117 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555b1d022e0 0x5555b1cff700 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f58f0015000 tx=0x7f58f000b3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a2a64cf01dc2be server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:07.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.402+0000 7f58d6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:20:07.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.402+0000 7f58d6ffd700 1 -- 172.21.15.101:0/2809402117 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f000e040 con 0x5555b1d022e0 2023-07-18T03:20:07.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.402+0000 7f58d6ffd700 1 -- 172.21.15.101:0/2809402117 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f58f001e870 con 0x5555b1d022e0 2023-07-18T03:20:07.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.402+0000 7f59114c8b80 1 -- 172.21.15.101:0/2809402117 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5555b1b9a4f0 con 0x5555b1d022e0 2023-07-18T03:20:07.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.402+0000 7f58d6ffd700 1 -- 172.21.15.101:0/2809402117 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58f001ea30 con 0x5555b1d022e0 2023-07-18T03:20:07.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.402+0000 7f59114c8b80 10 client.28162 fetch_fsmap learned FSMap version 33 2023-07-18T03:20:07.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.402+0000 7f59114c8b80 1 -- 172.21.15.101:0/2809402117 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5555b1a57c60 con 0x5555b1d022e0 2023-07-18T03:20:07.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.402+0000 7f58d6ffd700 1 -- 172.21.15.101:0/2809402117 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5555b1b9a4f0 con 0x5555b1d022e0 2023-07-18T03:20:07.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.403+0000 7f58d6ffd700 1 -- 172.21.15.101:0/2809402117 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f58f0005040 con 0x5555b1d022e0 2023-07-18T03:20:07.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.403+0000 7f59114c8b80 10 client.28162 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:20:07.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.403+0000 7f59114c8b80 10 client.28162 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:20:07.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.403+0000 7f59114c8b80 20 client.28162 populate_metadata read hostname 'smithi101' 2023-07-18T03:20:07.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.403+0000 7f59114c8b80 1 --2- 172.21.15.101:0/2809402117 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5555b1d11cc0 0x5555b1d140d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:07.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.403+0000 7f59114c8b80 4 client.28162 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:20:07.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.403+0000 7f59114c8b80 1 -- 172.21.15.101:0/2809402117 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5555b1c39eb0 con 0x5555b1d11cc0 2023-07-18T03:20:07.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.403+0000 7f58f77fe700 1 --2- 172.21.15.101:0/2809402117 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5555b1d11cc0 0x5555b1d140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:07.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.403+0000 7f58f77fe700 1 --2- 172.21.15.101:0/2809402117 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5555b1d11cc0 0x5555b1d140d0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a09589baf0d8895 server_cookie=8916e3db57394677 in_seq=0 out_seq=0 2023-07-18T03:20:07.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.403+0000 7f58d6ffd700 10 client.28162 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:20:07.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.404+0000 7f58d6ffd700 1 -- 172.21.15.101:0/2809402117 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5555b1c39eb0 con 0x5555b1d11cc0 2023-07-18T03:20:07.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.404+0000 7f58d6ffd700 10 client.28162 handle_command_reply: tid=0 2023-07-18T03:20:07.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.404+0000 7f59114c8b80 1 client.28162 shutdown 2023-07-18T03:20:07.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.404+0000 7f59114c8b80 1 -- 172.21.15.101:0/2809402117 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555b1d022e0 msgr2=0x5555b1cff700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:07.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.404+0000 7f59114c8b80 1 --2- 172.21.15.101:0/2809402117 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555b1d022e0 0x5555b1cff700 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f58f0015000 tx=0x7f58f000b3b0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.404+0000 7f59114c8b80 1 -- 172.21.15.101:0/2809402117 shutdown_connections 2023-07-18T03:20:07.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.404+0000 7f59114c8b80 1 --2- 172.21.15.101:0/2809402117 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5555b1d11cc0 0x5555b1d140d0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.404+0000 7f59114c8b80 1 --2- 172.21.15.101:0/2809402117 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555b1d022e0 0x5555b1cff700 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.404+0000 7f59114c8b80 1 --2- 172.21.15.101:0/2809402117 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5555b1cee2d0 0x5555b1d01d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.404+0000 7f59114c8b80 1 --2- 172.21.15.101:0/2809402117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555b1ceebe0 0x5555b1cfefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.404+0000 7f58d6ffd700 0 client.28162 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:20:07.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.404+0000 7f59114c8b80 1 -- 172.21.15.101:0/2809402117 >> 172.21.15.101:0/2809402117 conn(0x5555b1ce4da0 msgr2=0x5555b1cea230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:07.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.404+0000 7f59114c8b80 1 -- 172.21.15.101:0/2809402117 shutdown_connections 2023-07-18T03:20:07.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.404+0000 7f59114c8b80 1 -- 172.21.15.101:0/2809402117 wait complete. 2023-07-18T03:20:07.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.405+0000 7f59114c8b80 20 client.28162 trim_cache size 0 max 16384 2023-07-18T03:20:07.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.418+0000 7f59114c8b80 1 Processor -- start 2023-07-18T03:20:07.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.418+0000 7f59114c8b80 1 -- start start 2023-07-18T03:20:07.414 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.418+0000 7f59114c8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5555b1cee2d0 0x5555b1cfef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:07.414 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.418+0000 7f59114c8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555b1ceebe0 0x5555b1cfe7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:07.414 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.418+0000 7f59114c8b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555b1d022e0 0x5555b1ce8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:07.414 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.418+0000 7f59114c8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b1cebd60 con 0x5555b1cee2d0 2023-07-18T03:20:07.414 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.418+0000 7f59114c8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b1cfed10 con 0x5555b1ceebe0 2023-07-18T03:20:07.415 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.418+0000 7f59114c8b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b1cfab70 con 0x5555b1d022e0 2023-07-18T03:20:07.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.419+0000 7f58f77fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5555b1cee2d0 0x5555b1cfef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:07.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.419+0000 7f58f77fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5555b1cee2d0 0x5555b1cfef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:54654/0 (socket says 172.21.15.101:54654) 2023-07-18T03:20:07.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.419+0000 7f58f77fe700 1 -- 172.21.15.101:0/981644001 learned_addr learned my addr 172.21.15.101:0/981644001 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:07.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.419+0000 7f58f6ffd700 1 --2- 172.21.15.101:0/981644001 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555b1ceebe0 0x5555b1cfe7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:07.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.419+0000 7f58f7fff700 1 --2- 172.21.15.101:0/981644001 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555b1d022e0 0x5555b1ce8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:07.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.419+0000 7f58f77fe700 1 -- 172.21.15.101:0/981644001 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555b1d022e0 msgr2=0x5555b1ce8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:07.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.419+0000 7f58f77fe700 1 --2- 172.21.15.101:0/981644001 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555b1d022e0 0x5555b1ce8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.419+0000 7f58f77fe700 1 -- 172.21.15.101:0/981644001 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555b1ceebe0 msgr2=0x5555b1cfe7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:07.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.419+0000 7f58f77fe700 1 --2- 172.21.15.101:0/981644001 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555b1ceebe0 0x5555b1cfe7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.419+0000 7f58f77fe700 1 -- 172.21.15.101:0/981644001 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555b1a95250 con 0x5555b1cee2d0 2023-07-18T03:20:07.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.419+0000 7f58f6ffd700 1 --2- 172.21.15.101:0/981644001 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555b1ceebe0 0x5555b1cfe7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:20:07.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.419+0000 7f58f77fe700 1 --2- 172.21.15.101:0/981644001 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5555b1cee2d0 0x5555b1cfef80 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f58e801f040 tx=0x7f58e8010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22b811f9ba688588 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:07.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.419+0000 7f58de7fc700 1 -- 172.21.15.101:0/981644001 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e809df40 con 0x5555b1cee2d0 2023-07-18T03:20:07.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.419+0000 7f58de7fc700 1 -- 172.21.15.101:0/981644001 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58e8099400 con 0x5555b1cee2d0 2023-07-18T03:20:07.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.419+0000 7f58de7fc700 1 -- 172.21.15.101:0/981644001 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e80a2130 con 0x5555b1cee2d0 2023-07-18T03:20:07.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.420+0000 7f59114c8b80 1 -- 172.21.15.101:0/981644001 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5555b1cee2d0 msgr2=0x5555b1cfef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:07.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.420+0000 7f59114c8b80 1 --2- 172.21.15.101:0/981644001 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5555b1cee2d0 0x5555b1cfef80 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f58e801f040 tx=0x7f58e8010c80 comp rx=0 tx=0).stop 2023-07-18T03:20:07.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.420+0000 7f59114c8b80 1 -- 172.21.15.101:0/981644001 shutdown_connections 2023-07-18T03:20:07.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.420+0000 7f59114c8b80 1 --2- 172.21.15.101:0/981644001 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555b1d022e0 0x5555b1ce8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.420+0000 7f59114c8b80 1 --2- 172.21.15.101:0/981644001 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5555b1cee2d0 0x5555b1cfef80 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.420+0000 7f59114c8b80 1 --2- 172.21.15.101:0/981644001 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555b1ceebe0 0x5555b1cfe7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.420+0000 7f59114c8b80 1 -- 172.21.15.101:0/981644001 >> 172.21.15.101:0/981644001 conn(0x5555b1ce4da0 msgr2=0x5555b1d189f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:07.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.420+0000 7f59114c8b80 1 -- 172.21.15.101:0/981644001 shutdown_connections 2023-07-18T03:20:07.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.420+0000 7f59114c8b80 1 -- 172.21.15.101:0/981644001 wait complete. 2023-07-18T03:20:07.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.421+0000 7f59114c8b80 1 -- start start 2023-07-18T03:20:07.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.421+0000 7f59114c8b80 1 Processor -- start 2023-07-18T03:20:07.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.421+0000 7f59114c8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5555b1cee2d0 0x5555b1d08a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:07.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.421+0000 7f59114c8b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555b1ceebe0 0x5555b1d06d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:07.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.421+0000 7f59114c8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555b1d022e0 0x5555b1d072c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:07.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.421+0000 7f59114c8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b1d09330 con 0x5555b1cee2d0 2023-07-18T03:20:07.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.421+0000 7f59114c8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b1d07800 con 0x5555b1d022e0 2023-07-18T03:20:07.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.421+0000 7f59114c8b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b1d07970 con 0x5555b1ceebe0 2023-07-18T03:20:07.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.421+0000 7f59114c8b80 10 client.0 _ll_register_callbacks cb 0x7f58f0003a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:20:07.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.421+0000 7f58f77fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5555b1cee2d0 0x5555b1d08a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:07.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.421+0000 7f58f6ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555b1ceebe0 0x5555b1d06d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:07.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.421+0000 7f58f6ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555b1ceebe0 0x5555b1d06d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:38616/0 (socket says 172.21.15.101:38616) 2023-07-18T03:20:07.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.421+0000 7f58f6ffd700 1 -- 172.21.15.101:0/2839717899 learned_addr learned my addr 172.21.15.101:0/2839717899 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:07.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.422+0000 7f58f7fff700 1 --2- 172.21.15.101:0/2839717899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555b1d022e0 0x5555b1d072c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:07.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.422+0000 7f58f6ffd700 1 -- 172.21.15.101:0/2839717899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555b1d022e0 msgr2=0x5555b1d072c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:07.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.422+0000 7f58f6ffd700 1 --2- 172.21.15.101:0/2839717899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555b1d022e0 0x5555b1d072c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.422+0000 7f58f6ffd700 1 -- 172.21.15.101:0/2839717899 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5555b1cee2d0 msgr2=0x5555b1d08a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:07.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.422+0000 7f58f6ffd700 1 --2- 172.21.15.101:0/2839717899 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5555b1cee2d0 0x5555b1d08a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.422+0000 7f58f6ffd700 1 -- 172.21.15.101:0/2839717899 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58ec009b20 con 0x5555b1ceebe0 2023-07-18T03:20:07.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.422+0000 7f58f7fff700 1 --2- 172.21.15.101:0/2839717899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555b1d022e0 0x5555b1d072c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:20:07.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.422+0000 7f58f77fe700 1 --2- 172.21.15.101:0/2839717899 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5555b1cee2d0 0x5555b1d08a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:20:07.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.422+0000 7f58f6ffd700 1 --2- 172.21.15.101:0/2839717899 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555b1ceebe0 0x5555b1d06d80 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f58ec012cb0 tx=0x7f58ec009d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83e349306e290ab3 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:07.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.422+0000 7f58d6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:20:07.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.422+0000 7f58d6ffd700 1 -- 172.21.15.101:0/2839717899 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ec017040 con 0x5555b1ceebe0 2023-07-18T03:20:07.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.422+0000 7f59114c8b80 1 -- 172.21.15.101:0/2839717899 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f58e80a2360 con 0x5555b1ceebe0 2023-07-18T03:20:07.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.422+0000 7f58d6ffd700 1 -- 172.21.15.101:0/2839717899 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f58ec00fa30 con 0x5555b1ceebe0 2023-07-18T03:20:07.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.423+0000 7f58d6ffd700 1 -- 172.21.15.101:0/2839717899 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58ec0122e0 con 0x5555b1ceebe0 2023-07-18T03:20:07.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.423+0000 7f59114c8b80 10 client.28168 fetch_fsmap learned FSMap version 33 2023-07-18T03:20:07.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.423+0000 7f59114c8b80 1 -- 172.21.15.101:0/2839717899 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5555b1a6ede0 con 0x5555b1ceebe0 2023-07-18T03:20:07.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.423+0000 7f58d6ffd700 1 -- 172.21.15.101:0/2839717899 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58e80a2360 con 0x5555b1ceebe0 2023-07-18T03:20:07.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.423+0000 7f58d6ffd700 1 -- 172.21.15.101:0/2839717899 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f58ec01ec70 con 0x5555b1ceebe0 2023-07-18T03:20:07.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.423+0000 7f59114c8b80 10 client.28168 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:20:07.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.423+0000 7f59114c8b80 10 client.28168 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:20:07.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.423+0000 7f59114c8b80 20 client.28168 populate_metadata read hostname 'smithi101' 2023-07-18T03:20:07.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.423+0000 7f59114c8b80 1 --2- 172.21.15.101:0/2839717899 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5555b1d66fc0 0x5555b1d693d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:07.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.423+0000 7f59114c8b80 4 client.28168 mds_command: new command op to 24562 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:20:07.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.423+0000 7f59114c8b80 1 -- 172.21.15.101:0/2839717899 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5555b1bfc050 con 0x5555b1d66fc0 2023-07-18T03:20:07.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.423+0000 7f58f77fe700 1 --2- 172.21.15.101:0/2839717899 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5555b1d66fc0 0x5555b1d693d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:07.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.424+0000 7f58d6ffd700 10 client.28168 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:20:07.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.424+0000 7f58f77fe700 1 --2- 172.21.15.101:0/2839717899 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5555b1d66fc0 0x5555b1d693d0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38967c99dd438fdc server_cookie=7b23acd559c2fcc5 in_seq=0 out_seq=0 2023-07-18T03:20:07.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.424+0000 7f58d6ffd700 1 -- 172.21.15.101:0/2839717899 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5555b1bfc050 con 0x5555b1d66fc0 2023-07-18T03:20:07.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.424+0000 7f58d6ffd700 10 client.28168 handle_command_reply: tid=0 2023-07-18T03:20:07.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.424+0000 7f59114c8b80 1 client.28168 shutdown 2023-07-18T03:20:07.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.424+0000 7f59114c8b80 1 -- 172.21.15.101:0/2839717899 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555b1ceebe0 msgr2=0x5555b1d06d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:07.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.424+0000 7f59114c8b80 1 --2- 172.21.15.101:0/2839717899 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555b1ceebe0 0x5555b1d06d80 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f58ec012cb0 tx=0x7f58ec009d10 comp rx=0 tx=0).stop 2023-07-18T03:20:07.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.424+0000 7f59114c8b80 1 -- 172.21.15.101:0/2839717899 shutdown_connections 2023-07-18T03:20:07.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.424+0000 7f59114c8b80 1 --2- 172.21.15.101:0/2839717899 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5555b1d66fc0 0x5555b1d693d0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.424+0000 7f59114c8b80 1 --2- 172.21.15.101:0/2839717899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555b1d022e0 0x5555b1d072c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.424+0000 7f59114c8b80 1 --2- 172.21.15.101:0/2839717899 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5555b1cee2d0 0x5555b1d08a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.424+0000 7f59114c8b80 1 --2- 172.21.15.101:0/2839717899 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5555b1ceebe0 0x5555b1d06d80 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.424+0000 7f58d6ffd700 0 client.28168 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:20:07.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.424+0000 7f59114c8b80 1 -- 172.21.15.101:0/2839717899 >> 172.21.15.101:0/2839717899 conn(0x5555b1ce4da0 msgr2=0x5555b1cfa2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:07.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.424+0000 7f59114c8b80 1 -- 172.21.15.101:0/2839717899 shutdown_connections 2023-07-18T03:20:07.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.424+0000 7f59114c8b80 1 -- 172.21.15.101:0/2839717899 wait complete. 2023-07-18T03:20:07.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.424+0000 7f59114c8b80 20 client.28168 trim_cache size 0 max 16384 2023-07-18T03:20:07.429 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:20:07.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.425+0000 7f58fdbff700 1 -- 172.21.15.101:0/4155746651 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f58e00a1340 msgr2=0x7f58e00a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:07.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.425+0000 7f58fdbff700 1 --2- 172.21.15.101:0/4155746651 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f58e00a1340 0x7f58e00a37f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f58f8074920 tx=0x7f58f00095a0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.425+0000 7f58fdbff700 1 -- 172.21.15.101:0/4155746651 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f58f80ffae0 msgr2=0x7f58f8072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:07.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.425+0000 7f58fdbff700 1 --2- 172.21.15.101:0/4155746651 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f58f80ffae0 0x7f58f8072810 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f58e800e9a0 tx=0x7f58e800ecb0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.425+0000 7f58fdbff700 1 -- 172.21.15.101:0/4155746651 shutdown_connections 2023-07-18T03:20:07.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.425+0000 7f58fdbff700 1 --2- 172.21.15.101:0/4155746651 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58f8105af0 0x7f58f81a3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.425+0000 7f58fdbff700 1 --2- 172.21.15.101:0/4155746651 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f58f80ffae0 0x7f58f8072810 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.425+0000 7f58fdbff700 1 --2- 172.21.15.101:0/4155746651 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f58e00a1340 0x7f58e00a37f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.425+0000 7f58fdbff700 1 --2- 172.21.15.101:0/4155746651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58f8100490 0x7f58f8072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:07.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.425+0000 7f58fdbff700 1 -- 172.21.15.101:0/4155746651 >> 172.21.15.101:0/4155746651 conn(0x7f58f80fb5e0 msgr2=0x7f58f80fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:07.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.425+0000 7f58fdbff700 1 -- 172.21.15.101:0/4155746651 shutdown_connections 2023-07-18T03:20:07.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:07.425+0000 7f58fdbff700 1 -- 172.21.15.101:0/4155746651 wait complete. 2023-07-18T03:20:07.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:07 smithi179 ceph-mon[129229]: pgmap v391: 129 pgs: 129 active+clean; 620 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 137 op/s 2023-07-18T03:20:07.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:07 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/112214217' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:20:07.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:07 smithi155 ceph-mon[128885]: pgmap v391: 129 pgs: 129 active+clean; 620 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 137 op/s 2023-07-18T03:20:07.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:07 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/112214217' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:20:07.628 INFO:tasks.workunit.client.0.smithi101.stdout: 1 740278 5.30 MB/sec execute 572 sec latency 684.667 ms 2023-07-18T03:20:08.439 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-18T03:20:08.439 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-18T03:20:08.628 INFO:tasks.workunit.client.0.smithi101.stdout: 1 740278 5.29 MB/sec execute 573 sec latency 1684.704 ms 2023-07-18T03:20:08.651 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.655+0000 7f9095dff700 1 -- 172.21.15.101:0/3703750839 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f90901015b0 msgr2=0x7f9090109aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:08.651 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.655+0000 7f9095dff700 1 --2- 172.21.15.101:0/3703750839 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f90901015b0 0x7f9090109aa0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f9084009990 tx=0x7f9084009ca0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.655+0000 7f9095dff700 1 -- 172.21.15.101:0/3703750839 shutdown_connections 2023-07-18T03:20:08.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.655+0000 7f9095dff700 1 --2- 172.21.15.101:0/3703750839 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9090100ca0 0x7f9090101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.655+0000 7f9095dff700 1 --2- 172.21.15.101:0/3703750839 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f909010a070 0x7f909010c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.655+0000 7f9095dff700 1 --2- 172.21.15.101:0/3703750839 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f90901015b0 0x7f9090109aa0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.655+0000 7f9095dff700 1 -- 172.21.15.101:0/3703750839 >> 172.21.15.101:0/3703750839 conn(0x7f90900fa510 msgr2=0x7f90900fc920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:08.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.655+0000 7f9095dff700 1 -- 172.21.15.101:0/3703750839 shutdown_connections 2023-07-18T03:20:08.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.655+0000 7f9095dff700 1 -- 172.21.15.101:0/3703750839 wait complete. 2023-07-18T03:20:08.654 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.656+0000 7f9095dff700 1 Processor -- start 2023-07-18T03:20:08.654 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.656+0000 7f9095dff700 1 -- start start 2023-07-18T03:20:08.654 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.657+0000 7f9095dff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9090100ca0 0x7f90900fe8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:08.654 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.657+0000 7f9095dff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90901015b0 0x7f90900fee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:08.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.657+0000 7f9095dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f909010a070 0x7f90901a3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:08.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.657+0000 7f9095dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90900ff660 con 0x7f909010a070 2023-07-18T03:20:08.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.657+0000 7f9095dff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90901a4340 con 0x7f90901015b0 2023-07-18T03:20:08.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.657+0000 7f9095dff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90901a4480 con 0x7f9090100ca0 2023-07-18T03:20:08.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.657+0000 7f908ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f909010a070 0x7f90901a3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:08.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.657+0000 7f908ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f909010a070 0x7f90901a3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:54678/0 (socket says 172.21.15.101:54678) 2023-07-18T03:20:08.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.657+0000 7f908ffff700 1 -- 172.21.15.101:0/3792249268 learned_addr learned my addr 172.21.15.101:0/3792249268 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:08.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.657+0000 7f908ffff700 1 -- 172.21.15.101:0/3792249268 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9090100ca0 msgr2=0x7f90900fe8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:08.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.657+0000 7f908ffff700 1 --2- 172.21.15.101:0/3792249268 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9090100ca0 0x7f90900fe8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.657+0000 7f908ffff700 1 -- 172.21.15.101:0/3792249268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90901015b0 msgr2=0x7f90900fee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:08.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.657+0000 7f908ffff700 1 --2- 172.21.15.101:0/3792249268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90901015b0 0x7f90900fee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.657+0000 7f908ffff700 1 -- 172.21.15.101:0/3792249268 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90840095f0 con 0x7f909010a070 2023-07-18T03:20:08.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.658+0000 7f908ffff700 1 --2- 172.21.15.101:0/3792249268 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f909010a070 0x7f90901a3e00 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f9088007ee0 tx=0x7f908800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:08.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.658+0000 7f908cff9700 1 -- 172.21.15.101:0/3792249268 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9088017070 con 0x7f909010a070 2023-07-18T03:20:08.657 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.658+0000 7f9095dff700 1 -- 172.21.15.101:0/3792249268 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90901a46e0 con 0x7f909010a070 2023-07-18T03:20:08.657 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.658+0000 7f908cff9700 1 -- 172.21.15.101:0/3792249268 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f908800f040 con 0x7f909010a070 2023-07-18T03:20:08.657 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.658+0000 7f908cff9700 1 -- 172.21.15.101:0/3792249268 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9088005740 con 0x7f909010a070 2023-07-18T03:20:08.657 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.658+0000 7f9095dff700 1 -- 172.21.15.101:0/3792249268 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90901a4bc0 con 0x7f909010a070 2023-07-18T03:20:08.657 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.661+0000 7f908cff9700 1 -- 172.21.15.101:0/3792249268 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f90880058a0 con 0x7f909010a070 2023-07-18T03:20:08.659 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.663+0000 7f90a9757b80 1 Processor -- start 2023-07-18T03:20:08.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.663+0000 7f90a9757b80 1 -- start start 2023-07-18T03:20:08.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.664+0000 7f908cff9700 1 --2- 172.21.15.101:0/3792249268 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f90780a12c0 0x7f90780a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:08.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.664+0000 7f908cff9700 1 -- 172.21.15.101:0/3792249268 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f90880cc900 con 0x7f909010a070 2023-07-18T03:20:08.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.664+0000 7f90a9757b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4ba162350 0x55c4ba162720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:08.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.664+0000 7f90a9757b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4ba162c60 0x55c4ba16d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:08.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.664+0000 7f908effd700 1 --2- 172.21.15.101:0/3792249268 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f90780a12c0 0x7f90780a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:08.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.664+0000 7f90a9757b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4ba16db50 0x55c4ba16ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:08.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.664+0000 7f90a9757b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4ba068d30 con 0x55c4ba16db50 2023-07-18T03:20:08.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.664+0000 7f90a9757b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4ba0bbac0 con 0x55c4ba162350 2023-07-18T03:20:08.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.664+0000 7f90a9757b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4ba0b4f50 con 0x55c4ba162c60 2023-07-18T03:20:08.662 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.664+0000 7f908f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4ba162350 0x55c4ba162720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:08.662 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.664+0000 7f908ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4ba16db50 0x55c4ba16ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:08.662 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.664+0000 7f908effd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4ba162c60 0x55c4ba16d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:08.662 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.664+0000 7f908f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4ba162350 0x55c4ba162720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:43342/0 (socket says 172.21.15.101:43342) 2023-07-18T03:20:08.662 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.664+0000 7f908ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4ba16db50 0x55c4ba16ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:54680/0 (socket says 172.21.15.101:54680) 2023-07-18T03:20:08.662 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.664+0000 7f908f7fe700 1 -- 172.21.15.101:0/1715736775 learned_addr learned my addr 172.21.15.101:0/1715736775 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:08.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.665+0000 7f908ffff700 1 -- 172.21.15.101:0/1715736775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4ba162c60 msgr2=0x55c4ba16d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:08.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.665+0000 7f908ffff700 1 --2- 172.21.15.101:0/1715736775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4ba162c60 0x55c4ba16d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.665+0000 7f908ffff700 1 -- 172.21.15.101:0/1715736775 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4ba162350 msgr2=0x55c4ba162720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:08.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.665+0000 7f908ffff700 1 --2- 172.21.15.101:0/1715736775 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4ba162350 0x55c4ba162720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.665+0000 7f908ffff700 1 -- 172.21.15.101:0/1715736775 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4b9ee2e60 con 0x55c4ba16db50 2023-07-18T03:20:08.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.665+0000 7f908f7fe700 1 --2- 172.21.15.101:0/1715736775 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4ba162350 0x55c4ba162720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:20:08.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.665+0000 7f908effd700 1 --2- 172.21.15.101:0/3792249268 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f90780a12c0 0x7f90780a3770 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f90840098e0 tx=0x7f9084005770 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:08.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.665+0000 7f908ffff700 1 --2- 172.21.15.101:0/1715736775 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4ba16db50 0x55c4ba16ff30 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f90880c9c40 tx=0x7f9088095a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ab0b45a635f3129 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:08.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.665+0000 7f9075ffb700 1 -- 172.21.15.101:0/1715736775 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90880995c0 con 0x55c4ba16db50 2023-07-18T03:20:08.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.665+0000 7f9075ffb700 1 -- 172.21.15.101:0/1715736775 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f908809d5f0 con 0x55c4ba16db50 2023-07-18T03:20:08.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.665+0000 7f9075ffb700 1 -- 172.21.15.101:0/1715736775 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908800dbd0 con 0x55c4ba16db50 2023-07-18T03:20:08.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.666+0000 7f90a9757b80 1 -- 172.21.15.101:0/1715736775 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4ba16db50 msgr2=0x55c4ba16ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:08.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.666+0000 7f90a9757b80 1 --2- 172.21.15.101:0/1715736775 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4ba16db50 0x55c4ba16ff30 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f90880c9c40 tx=0x7f9088095a30 comp rx=0 tx=0).stop 2023-07-18T03:20:08.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.666+0000 7f90a9757b80 1 -- 172.21.15.101:0/1715736775 shutdown_connections 2023-07-18T03:20:08.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.666+0000 7f90a9757b80 1 --2- 172.21.15.101:0/1715736775 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4ba162350 0x55c4ba162720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.668 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.666+0000 7f90a9757b80 1 --2- 172.21.15.101:0/1715736775 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4ba16db50 0x55c4ba16ff30 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.668 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.666+0000 7f90a9757b80 1 --2- 172.21.15.101:0/1715736775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4ba162c60 0x55c4ba16d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.668 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.666+0000 7f90a9757b80 1 -- 172.21.15.101:0/1715736775 >> 172.21.15.101:0/1715736775 conn(0x55c4ba158e20 msgr2=0x55c4ba15e720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:08.668 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.666+0000 7f90a9757b80 1 -- 172.21.15.101:0/1715736775 shutdown_connections 2023-07-18T03:20:08.668 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.666+0000 7f90a9757b80 1 -- 172.21.15.101:0/1715736775 wait complete. 2023-07-18T03:20:08.668 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.667+0000 7f90a9757b80 1 -- start start 2023-07-18T03:20:08.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.667+0000 7f90a9757b80 1 Processor -- start 2023-07-18T03:20:08.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.667+0000 7f90a9757b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4ba162350 0x55c4ba16d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:08.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.667+0000 7f90a9757b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4ba162c60 0x55c4ba16a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:08.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.667+0000 7f90a9757b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4ba16db50 0x55c4ba16afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:08.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.667+0000 7f90a9757b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4ba054b40 con 0x55c4ba16db50 2023-07-18T03:20:08.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.667+0000 7f90a9757b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4ba054840 con 0x55c4ba162350 2023-07-18T03:20:08.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.667+0000 7f90a9757b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4b9f389a0 con 0x55c4ba162c60 2023-07-18T03:20:08.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.667+0000 7f90a9757b80 10 client.0 _ll_register_callbacks cb 0x55c4ba0527e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:20:08.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.668+0000 7f908effd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4ba162c60 0x55c4ba16a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:08.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.668+0000 7f908effd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4ba162c60 0x55c4ba16a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:38652/0 (socket says 172.21.15.101:38652) 2023-07-18T03:20:08.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.668+0000 7f908effd700 1 -- 172.21.15.101:0/2356364487 learned_addr learned my addr 172.21.15.101:0/2356364487 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:08.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.668+0000 7f908ffff700 1 --2- 172.21.15.101:0/2356364487 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4ba16db50 0x55c4ba16afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:08.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.668+0000 7f908f7fe700 1 --2- 172.21.15.101:0/2356364487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4ba162350 0x55c4ba16d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:08.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.668+0000 7f908effd700 1 -- 172.21.15.101:0/2356364487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4ba162350 msgr2=0x55c4ba16d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:08.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.668+0000 7f908effd700 1 --2- 172.21.15.101:0/2356364487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4ba162350 0x55c4ba16d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.668+0000 7f908effd700 1 -- 172.21.15.101:0/2356364487 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4ba16db50 msgr2=0x55c4ba16afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:08.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.668+0000 7f908effd700 1 --2- 172.21.15.101:0/2356364487 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4ba16db50 0x55c4ba16afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.668+0000 7f908effd700 1 -- 172.21.15.101:0/2356364487 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9080009980 con 0x55c4ba162c60 2023-07-18T03:20:08.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.668+0000 7f908ffff700 1 --2- 172.21.15.101:0/2356364487 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4ba16db50 0x55c4ba16afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:20:08.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.668+0000 7f908effd700 1 --2- 172.21.15.101:0/2356364487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4ba162c60 0x55c4ba16a890 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9084015000 tx=0x7f9084000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bed57f4169d81e5f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:08.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.668+0000 7f906effd700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:20:08.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.668+0000 7f906effd700 1 -- 172.21.15.101:0/2356364487 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9084028660 con 0x55c4ba162c60 2023-07-18T03:20:08.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.669+0000 7f90a9757b80 1 -- 172.21.15.101:0/2356364487 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c4ba00e570 con 0x55c4ba162c60 2023-07-18T03:20:08.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.669+0000 7f906effd700 1 -- 172.21.15.101:0/2356364487 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f90840287c0 con 0x55c4ba162c60 2023-07-18T03:20:08.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.669+0000 7f906effd700 1 -- 172.21.15.101:0/2356364487 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9084028980 con 0x55c4ba162c60 2023-07-18T03:20:08.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.669+0000 7f90a9757b80 10 client.28177 fetch_fsmap learned FSMap version 33 2023-07-18T03:20:08.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.669+0000 7f90a9757b80 1 -- 172.21.15.101:0/2356364487 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f90901a4bc0 con 0x55c4ba162c60 2023-07-18T03:20:08.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.669+0000 7f906effd700 1 -- 172.21.15.101:0/2356364487 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c4ba00e570 con 0x55c4ba162c60 2023-07-18T03:20:08.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.669+0000 7f906effd700 1 -- 172.21.15.101:0/2356364487 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9084030cb0 con 0x55c4ba162c60 2023-07-18T03:20:08.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.669+0000 7f90a9757b80 10 client.28177 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:20:08.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.669+0000 7f90a9757b80 10 client.28177 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:20:08.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.669+0000 7f90a9757b80 20 client.28177 populate_metadata read hostname 'smithi101' 2023-07-18T03:20:08.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.669+0000 7f90a9757b80 1 --2- 172.21.15.101:0/2356364487 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c4ba17d560 0x55c4ba17f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:08.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.669+0000 7f90a9757b80 4 client.28177 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:20:08.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.669+0000 7f90a9757b80 1 -- 172.21.15.101:0/2356364487 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c4ba0adf30 con 0x55c4ba17d560 2023-07-18T03:20:08.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.670+0000 7f908f7fe700 1 --2- 172.21.15.101:0/2356364487 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c4ba17d560 0x55c4ba17f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:08.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.670+0000 7f906effd700 10 client.28177 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:20:08.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.670+0000 7f908f7fe700 1 --2- 172.21.15.101:0/2356364487 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c4ba17d560 0x55c4ba17f970 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc9b09d0dada9ea9 server_cookie=44d21802670543c4 in_seq=0 out_seq=0 2023-07-18T03:20:08.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.671+0000 7f906effd700 1 -- 172.21.15.101:0/2356364487 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c4ba0adf30 con 0x55c4ba17d560 2023-07-18T03:20:08.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.671+0000 7f906effd700 10 client.28177 handle_command_reply: tid=0 2023-07-18T03:20:08.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.671+0000 7f90a9757b80 1 client.28177 shutdown 2023-07-18T03:20:08.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.671+0000 7f90a9757b80 1 -- 172.21.15.101:0/2356364487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4ba162c60 msgr2=0x55c4ba16a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:08.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.671+0000 7f90a9757b80 1 --2- 172.21.15.101:0/2356364487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4ba162c60 0x55c4ba16a890 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9084015000 tx=0x7f9084000bc0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.671+0000 7f90a9757b80 1 -- 172.21.15.101:0/2356364487 shutdown_connections 2023-07-18T03:20:08.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.671+0000 7f90a9757b80 1 --2- 172.21.15.101:0/2356364487 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c4ba17d560 0x55c4ba17f970 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.671+0000 7f90a9757b80 1 --2- 172.21.15.101:0/2356364487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4ba162350 0x55c4ba16d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.671+0000 7f90a9757b80 1 --2- 172.21.15.101:0/2356364487 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4ba16db50 0x55c4ba16afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.671+0000 7f90a9757b80 1 --2- 172.21.15.101:0/2356364487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4ba162c60 0x55c4ba16a890 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.671+0000 7f906effd700 0 client.28177 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:20:08.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.671+0000 7f90a9757b80 1 -- 172.21.15.101:0/2356364487 >> 172.21.15.101:0/2356364487 conn(0x55c4ba158e20 msgr2=0x55c4ba15e310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:08.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.671+0000 7f90a9757b80 1 -- 172.21.15.101:0/2356364487 shutdown_connections 2023-07-18T03:20:08.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.671+0000 7f90a9757b80 1 -- 172.21.15.101:0/2356364487 wait complete. 2023-07-18T03:20:08.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.672+0000 7f90a9757b80 20 client.28177 trim_cache size 0 max 16384 2023-07-18T03:20:08.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.689+0000 7f90a9757b80 1 Processor -- start 2023-07-18T03:20:08.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.689+0000 7f90a9757b80 1 -- start start 2023-07-18T03:20:08.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.689+0000 7f90a9757b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4ba162350 0x55c4ba174570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:08.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.689+0000 7f90a9757b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4ba162c60 0x55c4ba15e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:08.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.689+0000 7f90a9757b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4ba16db50 0x55c4ba15e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:08.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.689+0000 7f90a9757b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4ba1589f0 con 0x55c4ba162350 2023-07-18T03:20:08.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.689+0000 7f90a9757b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4ba168df0 con 0x55c4ba16db50 2023-07-18T03:20:08.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.689+0000 7f90a9757b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4ba179e60 con 0x55c4ba162c60 2023-07-18T03:20:08.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.690+0000 7f908f7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4ba162350 0x55c4ba174570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:08.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.690+0000 7f908f7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4ba162350 0x55c4ba174570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:54694/0 (socket says 172.21.15.101:54694) 2023-07-18T03:20:08.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.690+0000 7f908f7fe700 1 -- 172.21.15.101:0/304412722 learned_addr learned my addr 172.21.15.101:0/304412722 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:08.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.690+0000 7f908ffff700 1 --2- 172.21.15.101:0/304412722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4ba16db50 0x55c4ba15e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:08.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.690+0000 7f908effd700 1 --2- 172.21.15.101:0/304412722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4ba162c60 0x55c4ba15e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:08.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.690+0000 7f908f7fe700 1 -- 172.21.15.101:0/304412722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4ba162c60 msgr2=0x55c4ba15e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:08.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.690+0000 7f908f7fe700 1 --2- 172.21.15.101:0/304412722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4ba162c60 0x55c4ba15e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.690+0000 7f908f7fe700 1 -- 172.21.15.101:0/304412722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4ba16db50 msgr2=0x55c4ba15e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:08.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.690+0000 7f908f7fe700 1 --2- 172.21.15.101:0/304412722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4ba16db50 0x55c4ba15e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.690+0000 7f908f7fe700 1 -- 172.21.15.101:0/304412722 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4b9ecbce0 con 0x55c4ba162350 2023-07-18T03:20:08.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.690+0000 7f908effd700 1 --2- 172.21.15.101:0/304412722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4ba162c60 0x55c4ba15e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:20:08.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.690+0000 7f908f7fe700 1 --2- 172.21.15.101:0/304412722 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4ba162350 0x55c4ba174570 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f9080000c00 tx=0x7f90800109e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c88f9035707319f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:08.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.690+0000 7f90767fc700 1 -- 172.21.15.101:0/304412722 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9080017040 con 0x55c4ba162350 2023-07-18T03:20:08.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.690+0000 7f90767fc700 1 -- 172.21.15.101:0/304412722 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f908001c5a0 con 0x55c4ba162350 2023-07-18T03:20:08.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.690+0000 7f90767fc700 1 -- 172.21.15.101:0/304412722 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90800038e0 con 0x55c4ba162350 2023-07-18T03:20:08.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.691+0000 7f90a9757b80 1 -- 172.21.15.101:0/304412722 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4ba162350 msgr2=0x55c4ba174570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:08.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.691+0000 7f90a9757b80 1 --2- 172.21.15.101:0/304412722 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4ba162350 0x55c4ba174570 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f9080000c00 tx=0x7f90800109e0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.691+0000 7f90a9757b80 1 -- 172.21.15.101:0/304412722 shutdown_connections 2023-07-18T03:20:08.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.691+0000 7f90a9757b80 1 --2- 172.21.15.101:0/304412722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4ba16db50 0x55c4ba15e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.691+0000 7f90a9757b80 1 --2- 172.21.15.101:0/304412722 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4ba162350 0x55c4ba174570 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.691+0000 7f90a9757b80 1 --2- 172.21.15.101:0/304412722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4ba162c60 0x55c4ba15e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.691+0000 7f90a9757b80 1 -- 172.21.15.101:0/304412722 >> 172.21.15.101:0/304412722 conn(0x55c4ba158e20 msgr2=0x55c4ba165b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:08.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.691+0000 7f90a9757b80 1 -- 172.21.15.101:0/304412722 shutdown_connections 2023-07-18T03:20:08.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.691+0000 7f90a9757b80 1 -- 172.21.15.101:0/304412722 wait complete. 2023-07-18T03:20:08.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.692+0000 7f90a9757b80 1 -- start start 2023-07-18T03:20:08.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.692+0000 7f90a9757b80 1 Processor -- start 2023-07-18T03:20:08.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.692+0000 7f90a9757b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4ba162350 0x55c4ba1cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:08.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.692+0000 7f90a9757b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4ba162c60 0x55c4ba1cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:08.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.692+0000 7f90a9757b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4ba16db50 0x55c4ba173500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:08.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.692+0000 7f90a9757b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4ba173c10 con 0x55c4ba162c60 2023-07-18T03:20:08.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.692+0000 7f90a9757b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4ba173d80 con 0x55c4ba16db50 2023-07-18T03:20:08.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.692+0000 7f90a9757b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4ba173f40 con 0x55c4ba162350 2023-07-18T03:20:08.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.692+0000 7f90a9757b80 10 client.0 _ll_register_callbacks cb 0x7f908400f460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:20:08.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.692+0000 7f908effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4ba162c60 0x55c4ba1cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:08.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.692+0000 7f908effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4ba162c60 0x55c4ba1cc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:54708/0 (socket says 172.21.15.101:54708) 2023-07-18T03:20:08.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.692+0000 7f908effd700 1 -- 172.21.15.101:0/2381667759 learned_addr learned my addr 172.21.15.101:0/2381667759 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:08.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.693+0000 7f908f7fe700 1 --2- 172.21.15.101:0/2381667759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4ba162350 0x55c4ba1cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:08.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.693+0000 7f908ffff700 1 --2- 172.21.15.101:0/2381667759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4ba16db50 0x55c4ba173500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:08.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.693+0000 7f908effd700 1 -- 172.21.15.101:0/2381667759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4ba162350 msgr2=0x55c4ba1cbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:08.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.693+0000 7f908effd700 1 --2- 172.21.15.101:0/2381667759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4ba162350 0x55c4ba1cbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.693+0000 7f908effd700 1 -- 172.21.15.101:0/2381667759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4ba16db50 msgr2=0x55c4ba173500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:08.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.693+0000 7f908effd700 1 --2- 172.21.15.101:0/2381667759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4ba16db50 0x55c4ba173500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.693+0000 7f908effd700 1 -- 172.21.15.101:0/2381667759 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9080010430 con 0x55c4ba162c60 2023-07-18T03:20:08.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.693+0000 7f908f7fe700 1 --2- 172.21.15.101:0/2381667759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4ba162350 0x55c4ba1cbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:20:08.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.693+0000 7f908effd700 1 --2- 172.21.15.101:0/2381667759 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4ba162c60 0x55c4ba1cc0c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f9084009350 tx=0x7f9084027660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d0622d167989aaa server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:08.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.693+0000 7f906effd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:20:08.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.693+0000 7f906effd700 1 -- 172.21.15.101:0/2381667759 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908400bb30 con 0x55c4ba162c60 2023-07-18T03:20:08.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.693+0000 7f90a9757b80 1 -- 172.21.15.101:0/2381667759 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f908809d5f0 con 0x55c4ba162c60 2023-07-18T03:20:08.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.693+0000 7f906effd700 1 -- 172.21.15.101:0/2381667759 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f9084017630 con 0x55c4ba162c60 2023-07-18T03:20:08.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.694+0000 7f906effd700 1 -- 172.21.15.101:0/2381667759 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9084017a80 con 0x55c4ba162c60 2023-07-18T03:20:08.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.694+0000 7f90a9757b80 10 client.18108 fetch_fsmap learned FSMap version 33 2023-07-18T03:20:08.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.694+0000 7f90a9757b80 1 -- 172.21.15.101:0/2381667759 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55c4b9f092d0 con 0x55c4ba162c60 2023-07-18T03:20:08.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.694+0000 7f906effd700 1 -- 172.21.15.101:0/2381667759 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f908809d5f0 con 0x55c4ba162c60 2023-07-18T03:20:08.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.694+0000 7f906effd700 1 -- 172.21.15.101:0/2381667759 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f908400f860 con 0x55c4ba162c60 2023-07-18T03:20:08.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.694+0000 7f90a9757b80 10 client.18108 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:20:08.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.694+0000 7f90a9757b80 10 client.18108 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:20:08.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.694+0000 7f90a9757b80 20 client.18108 populate_metadata read hostname 'smithi101' 2023-07-18T03:20:08.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.694+0000 7f90a9757b80 1 --2- 172.21.15.101:0/2381667759 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c4ba1d3560 0x55c4ba1d5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:08.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.694+0000 7f90a9757b80 4 client.18108 mds_command: new command op to 24562 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-18T03:20:08.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.694+0000 7f90a9757b80 1 -- 172.21.15.101:0/2381667759 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55c4ba0700d0 con 0x55c4ba1d3560 2023-07-18T03:20:08.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.694+0000 7f908f7fe700 1 --2- 172.21.15.101:0/2381667759 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c4ba1d3560 0x55c4ba1d5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:08.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.695+0000 7f908f7fe700 1 --2- 172.21.15.101:0/2381667759 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c4ba1d3560 0x55c4ba1d5970 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51b424e7ccf6bd78 server_cookie=af4965dcba5a083a in_seq=0 out_seq=0 2023-07-18T03:20:08.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.695+0000 7f906effd700 10 client.18108 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:20:08.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.696+0000 7f906effd700 1 -- 172.21.15.101:0/2381667759 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c4ba0700d0 con 0x55c4ba1d3560 2023-07-18T03:20:08.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.696+0000 7f906effd700 10 client.18108 handle_command_reply: tid=0 2023-07-18T03:20:08.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.696+0000 7f90a9757b80 1 client.18108 shutdown 2023-07-18T03:20:08.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.696+0000 7f90a9757b80 1 -- 172.21.15.101:0/2381667759 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4ba162c60 msgr2=0x55c4ba1cc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:08.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.696+0000 7f90a9757b80 1 --2- 172.21.15.101:0/2381667759 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4ba162c60 0x55c4ba1cc0c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f9084009350 tx=0x7f9084027660 comp rx=0 tx=0).stop 2023-07-18T03:20:08.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.697+0000 7f90a9757b80 1 -- 172.21.15.101:0/2381667759 shutdown_connections 2023-07-18T03:20:08.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.697+0000 7f90a9757b80 1 --2- 172.21.15.101:0/2381667759 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c4ba1d3560 0x55c4ba1d5970 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.697+0000 7f90a9757b80 1 --2- 172.21.15.101:0/2381667759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4ba16db50 0x55c4ba173500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.697+0000 7f90a9757b80 1 --2- 172.21.15.101:0/2381667759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4ba162350 0x55c4ba1cbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.697+0000 7f90a9757b80 1 --2- 172.21.15.101:0/2381667759 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4ba162c60 0x55c4ba1cc0c0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.697+0000 7f90a9757b80 1 -- 172.21.15.101:0/2381667759 >> 172.21.15.101:0/2381667759 conn(0x55c4ba158e20 msgr2=0x55c4ba15e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:08.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.697+0000 7f906effd700 0 client.18108 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:20:08.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.697+0000 7f90a9757b80 1 -- 172.21.15.101:0/2381667759 shutdown_connections 2023-07-18T03:20:08.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.697+0000 7f90a9757b80 1 -- 172.21.15.101:0/2381667759 wait complete. 2023-07-18T03:20:08.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.697+0000 7f90a9757b80 20 client.18108 trim_cache size 0 max 16384 2023-07-18T03:20:08.701 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:20:08.701 INFO:teuthology.orchestra.run.smithi101.stdout: "return_code": 0, 2023-07-18T03:20:08.701 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_tag": "ee8ef438-e060-467e-8910-55470c4b5830", 2023-07-18T03:20:08.701 INFO:teuthology.orchestra.run.smithi101.stdout: "mode": "asynchronous" 2023-07-18T03:20:08.701 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:20:08.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.697+0000 7f9095dff700 1 -- 172.21.15.101:0/3792249268 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f90780a12c0 msgr2=0x7f90780a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:08.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.697+0000 7f9095dff700 1 --2- 172.21.15.101:0/3792249268 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f90780a12c0 0x7f90780a3770 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f90840098e0 tx=0x7f9084005770 comp rx=0 tx=0).stop 2023-07-18T03:20:08.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.697+0000 7f9095dff700 1 -- 172.21.15.101:0/3792249268 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f909010a070 msgr2=0x7f90901a3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:08.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.697+0000 7f9095dff700 1 --2- 172.21.15.101:0/3792249268 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f909010a070 0x7f90901a3e00 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f9088007ee0 tx=0x7f908800d3b0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.698+0000 7f9095dff700 1 -- 172.21.15.101:0/3792249268 shutdown_connections 2023-07-18T03:20:08.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.698+0000 7f9095dff700 1 --2- 172.21.15.101:0/3792249268 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f909010a070 0x7f90901a3e00 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.698+0000 7f9095dff700 1 --2- 172.21.15.101:0/3792249268 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9090100ca0 0x7f90900fe8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.698+0000 7f9095dff700 1 --2- 172.21.15.101:0/3792249268 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f90780a12c0 0x7f90780a3770 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.698+0000 7f9095dff700 1 --2- 172.21.15.101:0/3792249268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90901015b0 0x7f90900fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.698+0000 7f9095dff700 1 -- 172.21.15.101:0/3792249268 >> 172.21.15.101:0/3792249268 conn(0x7f90900fa510 msgr2=0x7f909010c210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:08.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.698+0000 7f9095dff700 1 -- 172.21.15.101:0/3792249268 shutdown_connections 2023-07-18T03:20:08.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.698+0000 7f9095dff700 1 -- 172.21.15.101:0/3792249268 wait complete. 2023-07-18T03:20:08.713 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:20:08.935 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.939+0000 7f422e7ff700 1 -- 172.21.15.101:0/343956126 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4228105ab0 msgr2=0x7f4228105e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:08.935 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.939+0000 7f422e7ff700 1 --2- 172.21.15.101:0/343956126 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4228105ab0 0x7f4228105e80 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f4218009990 tx=0x7f4218009ca0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.939+0000 7f422e7ff700 1 -- 172.21.15.101:0/343956126 shutdown_connections 2023-07-18T03:20:08.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.939+0000 7f422e7ff700 1 --2- 172.21.15.101:0/343956126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4228100450 0x7f422810c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.939+0000 7f422e7ff700 1 --2- 172.21.15.101:0/343956126 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4228105ab0 0x7f4228105e80 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.939+0000 7f422e7ff700 1 --2- 172.21.15.101:0/343956126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42280ffaa0 0x7f42280fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.939+0000 7f422e7ff700 1 -- 172.21.15.101:0/343956126 >> 172.21.15.101:0/343956126 conn(0x7f42280fb5a0 msgr2=0x7f42280fd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:08.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.939+0000 7f422e7ff700 1 -- 172.21.15.101:0/343956126 shutdown_connections 2023-07-18T03:20:08.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.939+0000 7f422e7ff700 1 -- 172.21.15.101:0/343956126 wait complete. 2023-07-18T03:20:08.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.940+0000 7f422e7ff700 1 Processor -- start 2023-07-18T03:20:08.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.941+0000 7f422e7ff700 1 -- start start 2023-07-18T03:20:08.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.941+0000 7f422e7ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42280ffaa0 0x7f4228106940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:08.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.941+0000 7f422e7ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4228100450 0x7f4228106e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:08.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.941+0000 7f422e7ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4228105ab0 0x7f42281073c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:08.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.941+0000 7f422e7ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42281a8450 con 0x7f4228105ab0 2023-07-18T03:20:08.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.941+0000 7f422e7ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42281a85c0 con 0x7f4228100450 2023-07-18T03:20:08.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.941+0000 7f422e7ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42281a8780 con 0x7f42280ffaa0 2023-07-18T03:20:08.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.941+0000 7f42277fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4228100450 0x7f4228106e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:08.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.941+0000 7f4227fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42280ffaa0 0x7f4228106940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:08.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.941+0000 7f42277fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4228100450 0x7f4228106e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:43396/0 (socket says 172.21.15.101:43396) 2023-07-18T03:20:08.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.941+0000 7f4227fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42280ffaa0 0x7f4228106940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:38704/0 (socket says 172.21.15.101:38704) 2023-07-18T03:20:08.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.941+0000 7f42277fe700 1 -- 172.21.15.101:0/1875941889 learned_addr learned my addr 172.21.15.101:0/1875941889 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:08.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.941+0000 7f422cd9c700 1 --2- 172.21.15.101:0/1875941889 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4228105ab0 0x7f42281073c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:08.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.942+0000 7f42277fe700 1 -- 172.21.15.101:0/1875941889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42280ffaa0 msgr2=0x7f4228106940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:08.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.942+0000 7f42277fe700 1 --2- 172.21.15.101:0/1875941889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42280ffaa0 0x7f4228106940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.942+0000 7f42277fe700 1 -- 172.21.15.101:0/1875941889 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4228105ab0 msgr2=0x7f42281073c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:08.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.942+0000 7f42277fe700 1 --2- 172.21.15.101:0/1875941889 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4228105ab0 0x7f42281073c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.942+0000 7f42277fe700 1 -- 172.21.15.101:0/1875941889 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42180095f0 con 0x7f4228100450 2023-07-18T03:20:08.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.942+0000 7f42277fe700 1 --2- 172.21.15.101:0/1875941889 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4228100450 0x7f4228106e80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f421c00e9a0 tx=0x7f421c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:08.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.942+0000 7f42257fa700 1 -- 172.21.15.101:0/1875941889 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421c009790 con 0x7f4228100450 2023-07-18T03:20:08.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.942+0000 7f422e7ff700 1 -- 172.21.15.101:0/1875941889 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42281a8a60 con 0x7f4228100450 2023-07-18T03:20:08.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.942+0000 7f42257fa700 1 -- 172.21.15.101:0/1875941889 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f421c010430 con 0x7f4228100450 2023-07-18T03:20:08.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.942+0000 7f42257fa700 1 -- 172.21.15.101:0/1875941889 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421c018680 con 0x7f4228100450 2023-07-18T03:20:08.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.942+0000 7f422e7ff700 1 -- 172.21.15.101:0/1875941889 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42281a8fc0 con 0x7f4228100450 2023-07-18T03:20:08.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.946+0000 7f42257fa700 1 -- 172.21.15.101:0/1875941889 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f421c0105a0 con 0x7f4228100450 2023-07-18T03:20:08.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.947+0000 7f424214bb80 1 Processor -- start 2023-07-18T03:20:08.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.947+0000 7f424214bb80 1 -- start start 2023-07-18T03:20:08.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.947+0000 7f424214bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5633099ff2d0 0x5633099ff6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:08.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.947+0000 7f424214bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5633099ffbe0 0x563309a0a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:08.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.947+0000 7f424214bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563309a0aad0 0x563309a0ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:08.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.947+0000 7f424214bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563309905cb0 con 0x5633099ff2d0 2023-07-18T03:20:08.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.947+0000 7f424214bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563309958a40 con 0x5633099ffbe0 2023-07-18T03:20:08.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.947+0000 7f424214bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563309951ed0 con 0x563309a0aad0 2023-07-18T03:20:08.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.948+0000 7f422cd9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5633099ff2d0 0x5633099ff6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:08.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.948+0000 7f422cd9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5633099ff2d0 0x5633099ff6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:54718/0 (socket says 172.21.15.101:54718) 2023-07-18T03:20:08.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.948+0000 7f422cd9c700 1 -- 172.21.15.101:0/2325873359 learned_addr learned my addr 172.21.15.101:0/2325873359 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:08.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.948+0000 7f42277fe700 1 --2- 172.21.15.101:0/2325873359 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563309a0aad0 0x563309a0ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:08.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.948+0000 7f42257fa700 1 --2- 172.21.15.101:0/1875941889 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f42100a1340 0x7f42100a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:08.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.948+0000 7f4227fff700 1 --2- 172.21.15.101:0/2325873359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5633099ffbe0 0x563309a0a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:08.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.948+0000 7f422cd9c700 1 -- 172.21.15.101:0/2325873359 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563309a0aad0 msgr2=0x563309a0ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:08.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.948+0000 7f422cd9c700 1 --2- 172.21.15.101:0/2325873359 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563309a0aad0 0x563309a0ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.948+0000 7f422cd9c700 1 -- 172.21.15.101:0/2325873359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5633099ffbe0 msgr2=0x563309a0a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:08.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.948+0000 7f422cd9c700 1 --2- 172.21.15.101:0/2325873359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5633099ffbe0 0x563309a0a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.948+0000 7f422cd9c700 1 -- 172.21.15.101:0/2325873359 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56330977fde0 con 0x5633099ff2d0 2023-07-18T03:20:08.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.948+0000 7f42257fa700 1 -- 172.21.15.101:0/1875941889 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f421c014070 con 0x7f4228100450 2023-07-18T03:20:08.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.948+0000 7f4227fff700 1 --2- 172.21.15.101:0/2325873359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5633099ffbe0 0x563309a0a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:20:08.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.948+0000 7f422cd9c700 1 --2- 172.21.15.101:0/1875941889 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f42100a1340 0x7f42100a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:08.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.949+0000 7f422cd9c700 1 --2- 172.21.15.101:0/2325873359 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5633099ff2d0 0x5633099ff6a0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f4220011790 tx=0x7f4220011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c6956cc5dde7de1 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:08.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.949+0000 7f420e7fc700 1 -- 172.21.15.101:0/2325873359 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422000e040 con 0x5633099ff2d0 2023-07-18T03:20:08.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.949+0000 7f420e7fc700 1 -- 172.21.15.101:0/2325873359 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f422000f040 con 0x5633099ff2d0 2023-07-18T03:20:08.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.949+0000 7f420e7fc700 1 -- 172.21.15.101:0/2325873359 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4220023920 con 0x5633099ff2d0 2023-07-18T03:20:08.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.949+0000 7f422cd9c700 1 --2- 172.21.15.101:0/1875941889 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f42100a1340 0x7f42100a37f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4220022000 tx=0x7f422001f000 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:08.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.950+0000 7f424214bb80 1 -- 172.21.15.101:0/2325873359 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5633099ff2d0 msgr2=0x5633099ff6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:08.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.950+0000 7f424214bb80 1 --2- 172.21.15.101:0/2325873359 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5633099ff2d0 0x5633099ff6a0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f4220011790 tx=0x7f4220011aa0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.950+0000 7f424214bb80 1 -- 172.21.15.101:0/2325873359 shutdown_connections 2023-07-18T03:20:08.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.950+0000 7f424214bb80 1 --2- 172.21.15.101:0/2325873359 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563309a0aad0 0x563309a0ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.950+0000 7f424214bb80 1 --2- 172.21.15.101:0/2325873359 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5633099ff2d0 0x5633099ff6a0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.950+0000 7f424214bb80 1 --2- 172.21.15.101:0/2325873359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5633099ffbe0 0x563309a0a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.950+0000 7f424214bb80 1 -- 172.21.15.101:0/2325873359 >> 172.21.15.101:0/2325873359 conn(0x5633099f5da0 msgr2=0x5633099fb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:08.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.950+0000 7f424214bb80 1 -- 172.21.15.101:0/2325873359 shutdown_connections 2023-07-18T03:20:08.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.950+0000 7f424214bb80 1 -- 172.21.15.101:0/2325873359 wait complete. 2023-07-18T03:20:08.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.950+0000 7f424214bb80 1 -- start start 2023-07-18T03:20:08.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.950+0000 7f424214bb80 1 Processor -- start 2023-07-18T03:20:08.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.951+0000 7f424214bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5633099ff2d0 0x563309a077f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:08.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.951+0000 7f424214bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5633099ffbe0 0x563309a07d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:08.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.951+0000 7f424214bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563309a0aad0 0x563309a08270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:08.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.951+0000 7f424214bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633098f1ac0 con 0x5633099ff2d0 2023-07-18T03:20:08.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.951+0000 7f424214bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633098f17c0 con 0x563309a0aad0 2023-07-18T03:20:08.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.951+0000 7f424214bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633097d5920 con 0x5633099ffbe0 2023-07-18T03:20:08.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.951+0000 7f424214bb80 10 client.0 _ll_register_callbacks cb 0x5633098ef760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:20:08.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.951+0000 7f42277fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5633099ffbe0 0x563309a07d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:08.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.951+0000 7f422cd9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563309a0aad0 0x563309a08270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:08.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.951+0000 7f422cd9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563309a0aad0 0x563309a08270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:43420/0 (socket says 172.21.15.101:43420) 2023-07-18T03:20:08.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.951+0000 7f42277fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5633099ffbe0 0x563309a07d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:38714/0 (socket says 172.21.15.101:38714) 2023-07-18T03:20:08.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.951+0000 7f422cd9c700 1 -- 172.21.15.101:0/2649059981 learned_addr learned my addr 172.21.15.101:0/2649059981 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:08.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.952+0000 7f4227fff700 1 --2- 172.21.15.101:0/2649059981 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5633099ff2d0 0x563309a077f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:08.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.952+0000 7f422cd9c700 1 -- 172.21.15.101:0/2649059981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5633099ffbe0 msgr2=0x563309a07d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:08.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.952+0000 7f422cd9c700 1 --2- 172.21.15.101:0/2649059981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5633099ffbe0 0x563309a07d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.952+0000 7f422cd9c700 1 -- 172.21.15.101:0/2649059981 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5633099ff2d0 msgr2=0x563309a077f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:08.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.952+0000 7f422cd9c700 1 --2- 172.21.15.101:0/2649059981 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5633099ff2d0 0x563309a077f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.952+0000 7f422cd9c700 1 -- 172.21.15.101:0/2649059981 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4220011380 con 0x563309a0aad0 2023-07-18T03:20:08.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.952+0000 7f42277fe700 1 --2- 172.21.15.101:0/2649059981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5633099ffbe0 0x563309a07d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:20:08.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.952+0000 7f4227fff700 1 --2- 172.21.15.101:0/2649059981 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5633099ff2d0 0x563309a077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:20:08.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.952+0000 7f422cd9c700 1 --2- 172.21.15.101:0/2649059981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563309a0aad0 0x563309a08270 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f422001b040 tx=0x7f4220009090 comp rx=0 tx=0).ready entity=mon.2 client_cookie=764216b780bc3917 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:08.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.952+0000 7f42077fe700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:20:08.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.952+0000 7f42077fe700 1 -- 172.21.15.101:0/2649059981 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4220018480 con 0x563309a0aad0 2023-07-18T03:20:08.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.952+0000 7f424214bb80 1 -- 172.21.15.101:0/2649059981 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5633098ab4f0 con 0x563309a0aad0 2023-07-18T03:20:08.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.952+0000 7f42077fe700 1 -- 172.21.15.101:0/2649059981 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f42200185e0 con 0x563309a0aad0 2023-07-18T03:20:08.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.953+0000 7f42077fe700 1 -- 172.21.15.101:0/2649059981 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42200187a0 con 0x563309a0aad0 2023-07-18T03:20:08.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.953+0000 7f424214bb80 10 client.28160 fetch_fsmap learned FSMap version 33 2023-07-18T03:20:08.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.953+0000 7f424214bb80 1 -- 172.21.15.101:0/2649059981 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x563309768c60 con 0x563309a0aad0 2023-07-18T03:20:08.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.953+0000 7f42077fe700 1 -- 172.21.15.101:0/2649059981 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5633098ab4f0 con 0x563309a0aad0 2023-07-18T03:20:08.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.953+0000 7f42077fe700 1 -- 172.21.15.101:0/2649059981 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f422002baa0 con 0x563309a0aad0 2023-07-18T03:20:08.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.953+0000 7f424214bb80 10 client.28160 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:20:08.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.953+0000 7f424214bb80 10 client.28160 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:20:08.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.953+0000 7f424214bb80 20 client.28160 populate_metadata read hostname 'smithi101' 2023-07-18T03:20:08.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.953+0000 7f424214bb80 1 --2- 172.21.15.101:0/2649059981 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563309a1a2a0 0x563309a1c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:08.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.953+0000 7f424214bb80 4 client.28160 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:20:08.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.953+0000 7f424214bb80 1 -- 172.21.15.101:0/2649059981 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56330994aeb0 con 0x563309a1a2a0 2023-07-18T03:20:08.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.954+0000 7f4227fff700 1 --2- 172.21.15.101:0/2649059981 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563309a1a2a0 0x563309a1c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:08.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.954+0000 7f42077fe700 10 client.28160 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:20:08.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.954+0000 7f4227fff700 1 --2- 172.21.15.101:0/2649059981 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563309a1a2a0 0x563309a1c6b0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=550083fbd5b178a2 server_cookie=ed9845521980c58d in_seq=0 out_seq=0 2023-07-18T03:20:08.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.955+0000 7f42077fe700 1 -- 172.21.15.101:0/2649059981 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56330994aeb0 con 0x563309a1a2a0 2023-07-18T03:20:08.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.955+0000 7f42077fe700 10 client.28160 handle_command_reply: tid=0 2023-07-18T03:20:08.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.955+0000 7f424214bb80 1 client.28160 shutdown 2023-07-18T03:20:08.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.955+0000 7f424214bb80 1 -- 172.21.15.101:0/2649059981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563309a0aad0 msgr2=0x563309a08270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:08.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.955+0000 7f424214bb80 1 --2- 172.21.15.101:0/2649059981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563309a0aad0 0x563309a08270 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f422001b040 tx=0x7f4220009090 comp rx=0 tx=0).stop 2023-07-18T03:20:08.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.955+0000 7f424214bb80 1 -- 172.21.15.101:0/2649059981 shutdown_connections 2023-07-18T03:20:08.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.955+0000 7f424214bb80 1 --2- 172.21.15.101:0/2649059981 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563309a1a2a0 0x563309a1c6b0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.955+0000 7f424214bb80 1 --2- 172.21.15.101:0/2649059981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563309a0aad0 0x563309a08270 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.955+0000 7f424214bb80 1 --2- 172.21.15.101:0/2649059981 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5633099ff2d0 0x563309a077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.955+0000 7f42077fe700 0 client.28160 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:20:08.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.955+0000 7f424214bb80 1 --2- 172.21.15.101:0/2649059981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5633099ffbe0 0x563309a07d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.955+0000 7f424214bb80 1 -- 172.21.15.101:0/2649059981 >> 172.21.15.101:0/2649059981 conn(0x5633099f5da0 msgr2=0x5633099fb290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:08.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.955+0000 7f424214bb80 1 -- 172.21.15.101:0/2649059981 shutdown_connections 2023-07-18T03:20:08.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.955+0000 7f424214bb80 1 -- 172.21.15.101:0/2649059981 wait complete. 2023-07-18T03:20:08.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.955+0000 7f424214bb80 20 client.28160 trim_cache size 0 max 16384 2023-07-18T03:20:08.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.975+0000 7f424214bb80 1 Processor -- start 2023-07-18T03:20:08.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.976+0000 7f424214bb80 1 -- start start 2023-07-18T03:20:08.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.976+0000 7f424214bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5633099ff2d0 0x563309a06a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:08.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.976+0000 7f424214bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5633099ffbe0 0x563309a080a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:08.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.976+0000 7f424214bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563309a0aad0 0x5633099fb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:08.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.976+0000 7f424214bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563309a011a0 con 0x5633099ff2d0 2023-07-18T03:20:08.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.976+0000 7f424214bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563309a085e0 con 0x5633099ffbe0 2023-07-18T03:20:08.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.976+0000 7f424214bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563309a087a0 con 0x563309a0aad0 2023-07-18T03:20:08.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.976+0000 7f42277fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5633099ffbe0 0x563309a080a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:08.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.976+0000 7f42277fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5633099ffbe0 0x563309a080a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:43430/0 (socket says 172.21.15.101:43430) 2023-07-18T03:20:08.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.976+0000 7f42277fe700 1 -- 172.21.15.101:0/2412469753 learned_addr learned my addr 172.21.15.101:0/2412469753 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:08.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.976+0000 7f4227fff700 1 --2- 172.21.15.101:0/2412469753 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5633099ff2d0 0x563309a06a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:08.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.976+0000 7f422cd9c700 1 --2- 172.21.15.101:0/2412469753 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563309a0aad0 0x5633099fb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:08.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.976+0000 7f42277fe700 1 -- 172.21.15.101:0/2412469753 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563309a0aad0 msgr2=0x5633099fb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:08.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.976+0000 7f42277fe700 1 --2- 172.21.15.101:0/2412469753 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563309a0aad0 0x5633099fb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.976+0000 7f42277fe700 1 -- 172.21.15.101:0/2412469753 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5633099ff2d0 msgr2=0x563309a06a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:08.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.976+0000 7f42277fe700 1 --2- 172.21.15.101:0/2412469753 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5633099ff2d0 0x563309a06a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.976+0000 7f42277fe700 1 -- 172.21.15.101:0/2412469753 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633097a6250 con 0x5633099ffbe0 2023-07-18T03:20:08.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.976+0000 7f422cd9c700 1 --2- 172.21.15.101:0/2412469753 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563309a0aad0 0x5633099fb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:20:08.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.977+0000 7f42277fe700 1 --2- 172.21.15.101:0/2412469753 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5633099ffbe0 0x563309a080a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f421c010c30 tx=0x7f421c0cab20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=656d1e02a7e7a481 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:08.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.977+0000 7f420effd700 1 -- 172.21.15.101:0/2412469753 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421c0b1070 con 0x5633099ffbe0 2023-07-18T03:20:08.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.977+0000 7f420effd700 1 -- 172.21.15.101:0/2412469753 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f421c09df70 con 0x5633099ffbe0 2023-07-18T03:20:08.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.977+0000 7f420effd700 1 -- 172.21.15.101:0/2412469753 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421c0cf310 con 0x5633099ffbe0 2023-07-18T03:20:08.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.978+0000 7f424214bb80 1 -- 172.21.15.101:0/2412469753 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5633099ffbe0 msgr2=0x563309a080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:08.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.978+0000 7f424214bb80 1 --2- 172.21.15.101:0/2412469753 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5633099ffbe0 0x563309a080a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f421c010c30 tx=0x7f421c0cab20 comp rx=0 tx=0).stop 2023-07-18T03:20:08.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.978+0000 7f424214bb80 1 -- 172.21.15.101:0/2412469753 shutdown_connections 2023-07-18T03:20:08.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.978+0000 7f424214bb80 1 --2- 172.21.15.101:0/2412469753 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563309a0aad0 0x5633099fb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.978+0000 7f424214bb80 1 --2- 172.21.15.101:0/2412469753 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5633099ff2d0 0x563309a06a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.978+0000 7f424214bb80 1 --2- 172.21.15.101:0/2412469753 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5633099ffbe0 0x563309a080a0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.978+0000 7f424214bb80 1 -- 172.21.15.101:0/2412469753 >> 172.21.15.101:0/2412469753 conn(0x5633099f5da0 msgr2=0x563309a16820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:08.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.978+0000 7f424214bb80 1 -- 172.21.15.101:0/2412469753 shutdown_connections 2023-07-18T03:20:08.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.978+0000 7f424214bb80 1 -- 172.21.15.101:0/2412469753 wait complete. 2023-07-18T03:20:08.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.978+0000 7f424214bb80 1 -- start start 2023-07-18T03:20:08.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.979+0000 7f424214bb80 1 Processor -- start 2023-07-18T03:20:08.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.979+0000 7f424214bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5633099ff2d0 0x563309a11530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:08.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.979+0000 7f424214bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5633099ffbe0 0x563309a0f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:08.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.979+0000 7f424214bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563309a0aad0 0x563309a0fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:08.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.979+0000 7f424214bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563309a10580 con 0x5633099ffbe0 2023-07-18T03:20:08.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.979+0000 7f424214bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563309a106f0 con 0x5633099ff2d0 2023-07-18T03:20:08.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.979+0000 7f424214bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563309a6e170 con 0x563309a0aad0 2023-07-18T03:20:08.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.979+0000 7f424214bb80 10 client.0 _ll_register_callbacks cb 0x7f421c099d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:20:08.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.979+0000 7f4227fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5633099ff2d0 0x563309a11530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:08.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.979+0000 7f42277fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5633099ffbe0 0x563309a0f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:08.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.979+0000 7f4227fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5633099ff2d0 0x563309a11530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:43432/0 (socket says 172.21.15.101:43432) 2023-07-18T03:20:08.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.979+0000 7f4227fff700 1 -- 172.21.15.101:0/1481073697 learned_addr learned my addr 172.21.15.101:0/1481073697 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:08.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.980+0000 7f422cd9c700 1 --2- 172.21.15.101:0/1481073697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563309a0aad0 0x563309a0fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:08.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.980+0000 7f4227fff700 1 -- 172.21.15.101:0/1481073697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563309a0aad0 msgr2=0x563309a0fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:08.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.980+0000 7f4227fff700 1 --2- 172.21.15.101:0/1481073697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563309a0aad0 0x563309a0fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.980+0000 7f4227fff700 1 -- 172.21.15.101:0/1481073697 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5633099ffbe0 msgr2=0x563309a0f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:08.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.980+0000 7f4227fff700 1 --2- 172.21.15.101:0/1481073697 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5633099ffbe0 0x563309a0f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.980+0000 7f4227fff700 1 -- 172.21.15.101:0/1481073697 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42281a8fc0 con 0x5633099ff2d0 2023-07-18T03:20:08.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.980+0000 7f422cd9c700 1 --2- 172.21.15.101:0/1481073697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563309a0aad0 0x563309a0fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:20:08.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.980+0000 7f42277fe700 1 --2- 172.21.15.101:0/1481073697 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5633099ffbe0 0x563309a0f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:20:08.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.980+0000 7f4227fff700 1 --2- 172.21.15.101:0/1481073697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5633099ff2d0 0x563309a11530 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4218009990 tx=0x7f421800be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d1a62c2ea2ab4df server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:08.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.980+0000 7f42077fe700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:20:08.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.980+0000 7f42077fe700 1 -- 172.21.15.101:0/1481073697 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4218027a60 con 0x5633099ff2d0 2023-07-18T03:20:08.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.980+0000 7f42077fe700 1 -- 172.21.15.101:0/1481073697 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f4218027bc0 con 0x5633099ff2d0 2023-07-18T03:20:08.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.980+0000 7f424214bb80 1 -- 172.21.15.101:0/1481073697 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f422000f040 con 0x5633099ff2d0 2023-07-18T03:20:08.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.981+0000 7f42077fe700 1 -- 172.21.15.101:0/1481073697 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4218027d80 con 0x5633099ff2d0 2023-07-18T03:20:08.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.981+0000 7f424214bb80 10 client.28166 fetch_fsmap learned FSMap version 33 2023-07-18T03:20:08.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.981+0000 7f424214bb80 1 -- 172.21.15.101:0/1481073697 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x56330977e2b0 con 0x5633099ff2d0 2023-07-18T03:20:08.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.981+0000 7f42077fe700 1 -- 172.21.15.101:0/1481073697 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f422000f040 con 0x5633099ff2d0 2023-07-18T03:20:08.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.981+0000 7f42077fe700 1 -- 172.21.15.101:0/1481073697 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f42180099c0 con 0x5633099ff2d0 2023-07-18T03:20:08.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.981+0000 7f424214bb80 10 client.28166 fetch_fsmap finished waiting for FSMap version 33 2023-07-18T03:20:08.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.981+0000 7f424214bb80 10 client.28166 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:20:08.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.981+0000 7f424214bb80 20 client.28166 populate_metadata read hostname 'smithi101' 2023-07-18T03:20:08.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.981+0000 7f424214bb80 1 --2- 172.21.15.101:0/1481073697 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563309a6f890 0x563309a71ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:08.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.981+0000 7f424214bb80 4 client.28166 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:20:08.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.981+0000 7f424214bb80 1 -- 172.21.15.101:0/1481073697 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56330990d050 con 0x563309a6f890 2023-07-18T03:20:08.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.982+0000 7f42277fe700 1 --2- 172.21.15.101:0/1481073697 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563309a6f890 0x563309a71ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:08.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.982+0000 7f42077fe700 10 client.28166 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:20:08.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.982+0000 7f42277fe700 1 --2- 172.21.15.101:0/1481073697 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563309a6f890 0x563309a71ca0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f2302f12aa1325f server_cookie=833970add345accc in_seq=0 out_seq=0 2023-07-18T03:20:08.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.982+0000 7f42077fe700 1 -- 172.21.15.101:0/1481073697 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56330990d050 con 0x563309a6f890 2023-07-18T03:20:08.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.982+0000 7f42077fe700 10 client.28166 handle_command_reply: tid=0 2023-07-18T03:20:08.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.982+0000 7f424214bb80 1 client.28166 shutdown 2023-07-18T03:20:08.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.982+0000 7f424214bb80 1 -- 172.21.15.101:0/1481073697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5633099ff2d0 msgr2=0x563309a11530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:08.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.982+0000 7f424214bb80 1 --2- 172.21.15.101:0/1481073697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5633099ff2d0 0x563309a11530 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4218009990 tx=0x7f421800be10 comp rx=0 tx=0).stop 2023-07-18T03:20:08.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.983+0000 7f424214bb80 1 -- 172.21.15.101:0/1481073697 shutdown_connections 2023-07-18T03:20:08.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.983+0000 7f424214bb80 1 --2- 172.21.15.101:0/1481073697 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563309a6f890 0x563309a71ca0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.983+0000 7f424214bb80 1 --2- 172.21.15.101:0/1481073697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5633099ff2d0 0x563309a11530 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.983+0000 7f424214bb80 1 --2- 172.21.15.101:0/1481073697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563309a0aad0 0x563309a0fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.983+0000 7f424214bb80 1 --2- 172.21.15.101:0/1481073697 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5633099ffbe0 0x563309a0f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.983+0000 7f424214bb80 1 -- 172.21.15.101:0/1481073697 >> 172.21.15.101:0/1481073697 conn(0x5633099f5da0 msgr2=0x5633099fb160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:08.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.983+0000 7f42077fe700 0 client.28166 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:20:08.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.983+0000 7f424214bb80 1 -- 172.21.15.101:0/1481073697 shutdown_connections 2023-07-18T03:20:08.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.983+0000 7f424214bb80 1 -- 172.21.15.101:0/1481073697 wait complete. 2023-07-18T03:20:08.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.983+0000 7f424214bb80 20 client.28166 trim_cache size 0 max 16384 2023-07-18T03:20:08.995 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:20:08.995 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:20:08.995 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:20:08.996 INFO:teuthology.orchestra.run.smithi101.stdout: "ee8ef438-e060-467e-8910-55470c4b5830": { 2023-07-18T03:20:08.996 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:20:08.996 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "ee8ef438-e060-467e-8910-55470c4b5830", 2023-07-18T03:20:08.996 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:20:08.996 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:20:08.996 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:20:08.996 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:20:08.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.984+0000 7f422e7ff700 1 -- 172.21.15.101:0/1875941889 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f42100a1340 msgr2=0x7f42100a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:08.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.984+0000 7f422e7ff700 1 --2- 172.21.15.101:0/1875941889 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f42100a1340 0x7f42100a37f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4220022000 tx=0x7f422001f000 comp rx=0 tx=0).stop 2023-07-18T03:20:08.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.984+0000 7f422e7ff700 1 -- 172.21.15.101:0/1875941889 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4228100450 msgr2=0x7f4228106e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:08.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.984+0000 7f422e7ff700 1 --2- 172.21.15.101:0/1875941889 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4228100450 0x7f4228106e80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f421c00e9a0 tx=0x7f421c00ecb0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.984+0000 7f422e7ff700 1 -- 172.21.15.101:0/1875941889 shutdown_connections 2023-07-18T03:20:08.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.984+0000 7f422e7ff700 1 --2- 172.21.15.101:0/1875941889 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4228105ab0 0x7f42281073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.984+0000 7f422e7ff700 1 --2- 172.21.15.101:0/1875941889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42280ffaa0 0x7f4228106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.984+0000 7f422e7ff700 1 --2- 172.21.15.101:0/1875941889 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f42100a1340 0x7f42100a37f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.984+0000 7f422e7ff700 1 --2- 172.21.15.101:0/1875941889 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4228100450 0x7f4228106e80 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:08.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.984+0000 7f422e7ff700 1 -- 172.21.15.101:0/1875941889 >> 172.21.15.101:0/1875941889 conn(0x7f42280fb5a0 msgr2=0x7f4228072740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:08.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.984+0000 7f422e7ff700 1 -- 172.21.15.101:0/1875941889 shutdown_connections 2023-07-18T03:20:08.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:08.984+0000 7f422e7ff700 1 -- 172.21.15.101:0/1875941889 wait complete. 2023-07-18T03:20:08.999 INFO:tasks.cephfs.filesystem:scrub status for tag:ee8ef438-e060-467e-8910-55470c4b5830 - {'path': '/', 'tag': 'ee8ef438-e060-467e-8910-55470c4b5830', 'options': 'recursive,force'} 2023-07-18T03:20:09.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:09 smithi179 ceph-mon[129229]: pgmap v392: 129 pgs: 129 active+clean; 620 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 1.4 MiB/s wr, 86 op/s 2023-07-18T03:20:09.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:09 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:20:09.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:09 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:20:09.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:09 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:20:09.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:09 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:20:09.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:09 smithi101 ceph-mon[126379]: pgmap v392: 129 pgs: 129 active+clean; 620 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 1.4 MiB/s wr, 86 op/s 2023-07-18T03:20:09.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:09 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:20:09.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:09 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:20:09.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:09 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:20:09.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:09 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:20:09.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:09 smithi155 ceph-mon[128885]: pgmap v392: 129 pgs: 129 active+clean; 620 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 1.4 MiB/s wr, 86 op/s 2023-07-18T03:20:09.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:09 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:20:09.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:09 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:20:09.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:09 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:20:09.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:09 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:20:09.628 INFO:tasks.workunit.client.0.smithi101.stdout: 1 742111 5.30 MB/sec execute 574 sec latency 1774.594 ms 2023-07-18T03:20:10.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:10 smithi179 ceph-mon[129229]: scrub queued for path: / 2023-07-18T03:20:10.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:10 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:20:10.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:10 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:20:10.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:10 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:20:10.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:10 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:20:10.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:10 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:20:10.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:10 smithi101 ceph-mon[126379]: scrub queued for path: / 2023-07-18T03:20:10.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:10 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:20:10.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:10 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:20:10.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:10 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:20:10.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:10 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:20:10.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:10 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:20:10.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:10 smithi155 ceph-mon[128885]: scrub queued for path: / 2023-07-18T03:20:10.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:10 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:20:10.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:10 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:20:10.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:10 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:20:10.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:10 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:20:10.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:10 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:20:10.628 INFO:tasks.workunit.client.0.smithi101.stdout: 1 743076 5.30 MB/sec execute 575 sec latency 16.556 ms 2023-07-18T03:20:11.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:11 smithi179 ceph-mon[129229]: pgmap v393: 129 pgs: 129 active+clean; 625 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 145 op/s 2023-07-18T03:20:11.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:11 smithi101 ceph-mon[126379]: pgmap v393: 129 pgs: 129 active+clean; 625 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 145 op/s 2023-07-18T03:20:11.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:11 smithi155 ceph-mon[128885]: pgmap v393: 129 pgs: 129 active+clean; 625 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 145 op/s 2023-07-18T03:20:11.629 INFO:tasks.workunit.client.0.smithi101.stdout: 1 743820 5.29 MB/sec execute 576 sec latency 510.792 ms 2023-07-18T03:20:12.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:12 smithi179 ceph-mon[129229]: osdmap e105: 12 total, 12 up, 12 in 2023-07-18T03:20:12.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:12 smithi179 ceph-mon[129229]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:20:12.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:12 smithi101 ceph-mon[126379]: osdmap e105: 12 total, 12 up, 12 in 2023-07-18T03:20:12.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:12 smithi101 ceph-mon[126379]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:20:12.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:12 smithi155 ceph-mon[128885]: osdmap e105: 12 total, 12 up, 12 in 2023-07-18T03:20:12.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:12 smithi155 ceph-mon[128885]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:20:12.629 INFO:tasks.workunit.client.0.smithi101.stdout: 1 743820 5.28 MB/sec execute 577 sec latency 1510.950 ms 2023-07-18T03:20:13.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:13 smithi179 ceph-mon[129229]: pgmap v395: 129 pgs: 129 active+clean; 625 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 1.6 MiB/s wr, 113 op/s 2023-07-18T03:20:13.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:13 smithi101 ceph-mon[126379]: pgmap v395: 129 pgs: 129 active+clean; 625 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 1.6 MiB/s wr, 113 op/s 2023-07-18T03:20:13.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:13 smithi155 ceph-mon[128885]: pgmap v395: 129 pgs: 129 active+clean; 625 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 1.6 MiB/s wr, 113 op/s 2023-07-18T03:20:13.629 INFO:tasks.workunit.client.0.smithi101.stdout: 1 743820 5.28 MB/sec execute 578 sec latency 2511.035 ms 2023-07-18T03:20:14.629 INFO:tasks.workunit.client.0.smithi101.stdout: 1 744045 5.27 MB/sec execute 579 sec latency 3314.187 ms 2023-07-18T03:20:15.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:15 smithi179 ceph-mon[129229]: pgmap v396: 129 pgs: 129 active+clean; 625 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 1.6 MiB/s wr, 113 op/s 2023-07-18T03:20:15.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:15 smithi101 ceph-mon[126379]: pgmap v396: 129 pgs: 129 active+clean; 625 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 1.6 MiB/s wr, 113 op/s 2023-07-18T03:20:15.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:15 smithi155 ceph-mon[128885]: pgmap v396: 129 pgs: 129 active+clean; 625 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 1.6 MiB/s wr, 113 op/s 2023-07-18T03:20:15.629 INFO:tasks.workunit.client.0.smithi101.stdout: 1 745892 5.28 MB/sec execute 580 sec latency 94.559 ms 2023-07-18T03:20:16.629 INFO:tasks.workunit.client.0.smithi101.stdout: 1 746207 5.27 MB/sec execute 581 sec latency 492.743 ms 2023-07-18T03:20:17.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:17 smithi179 ceph-mon[129229]: pgmap v397: 129 pgs: 129 active+clean; 629 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.6 MiB/s wr, 119 op/s 2023-07-18T03:20:17.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:17 smithi101 ceph-mon[126379]: pgmap v397: 129 pgs: 129 active+clean; 629 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.6 MiB/s wr, 119 op/s 2023-07-18T03:20:17.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:17 smithi155 ceph-mon[128885]: pgmap v397: 129 pgs: 129 active+clean; 629 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.6 MiB/s wr, 119 op/s 2023-07-18T03:20:17.629 INFO:tasks.workunit.client.0.smithi101.stdout: 1 746207 5.26 MB/sec execute 582 sec latency 1492.894 ms 2023-07-18T03:20:18.629 INFO:tasks.workunit.client.0.smithi101.stdout: 1 746207 5.25 MB/sec execute 583 sec latency 2493.038 ms 2023-07-18T03:20:19.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:19 smithi179 ceph-mon[129229]: pgmap v398: 129 pgs: 129 active+clean; 629 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.6 MiB/s wr, 119 op/s 2023-07-18T03:20:19.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:19 smithi179 ceph-mon[129229]: osdmap e106: 12 total, 12 up, 12 in 2023-07-18T03:20:19.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:19 smithi101 ceph-mon[126379]: pgmap v398: 129 pgs: 129 active+clean; 629 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.6 MiB/s wr, 119 op/s 2023-07-18T03:20:19.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:19 smithi101 ceph-mon[126379]: osdmap e106: 12 total, 12 up, 12 in 2023-07-18T03:20:19.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:19 smithi155 ceph-mon[128885]: pgmap v398: 129 pgs: 129 active+clean; 629 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.6 MiB/s wr, 119 op/s 2023-07-18T03:20:19.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:19 smithi155 ceph-mon[128885]: osdmap e106: 12 total, 12 up, 12 in 2023-07-18T03:20:19.630 INFO:tasks.workunit.client.0.smithi101.stdout: 1 746207 5.25 MB/sec execute 584 sec latency 3493.113 ms 2023-07-18T03:20:20.630 INFO:tasks.workunit.client.0.smithi101.stdout: 1 746207 5.24 MB/sec execute 585 sec latency 4493.216 ms 2023-07-18T03:20:21.454 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:21 smithi179 ceph-mon[129229]: pgmap v400: 129 pgs: 129 active+clean; 632 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.7 MiB/s wr, 115 op/s 2023-07-18T03:20:21.455 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:21 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:20:21.455 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:21 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:20:21.455 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:21 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:20:21.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:21 smithi101 ceph-mon[126379]: pgmap v400: 129 pgs: 129 active+clean; 632 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.7 MiB/s wr, 115 op/s 2023-07-18T03:20:21.492 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:21 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:20:21.492 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:21 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:20:21.492 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:21 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:20:21.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:21 smithi155 ceph-mon[128885]: pgmap v400: 129 pgs: 129 active+clean; 632 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.7 MiB/s wr, 115 op/s 2023-07-18T03:20:21.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:21 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:20:21.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:21 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:20:21.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:21 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:20:21.630 INFO:tasks.workunit.client.0.smithi101.stdout: 1 746207 5.23 MB/sec execute 586 sec latency 5493.362 ms 2023-07-18T03:20:22.630 INFO:tasks.workunit.client.0.smithi101.stdout: 1 746207 5.22 MB/sec execute 587 sec latency 6493.442 ms 2023-07-18T03:20:23.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:23 smithi179 ceph-mon[129229]: pgmap v401: 129 pgs: 129 active+clean; 632 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.5 MiB/s wr, 99 op/s 2023-07-18T03:20:23.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:23 smithi101 ceph-mon[126379]: pgmap v401: 129 pgs: 129 active+clean; 632 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.5 MiB/s wr, 99 op/s 2023-07-18T03:20:23.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:23 smithi155 ceph-mon[128885]: pgmap v401: 129 pgs: 129 active+clean; 632 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.5 MiB/s wr, 99 op/s 2023-07-18T03:20:23.630 INFO:tasks.workunit.client.0.smithi101.stdout: 1 746207 5.21 MB/sec execute 588 sec latency 7493.525 ms 2023-07-18T03:20:24.630 INFO:tasks.workunit.client.0.smithi101.stdout: 1 746207 5.20 MB/sec execute 589 sec latency 8493.644 ms 2023-07-18T03:20:25.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:25 smithi179 ceph-mon[129229]: pgmap v402: 129 pgs: 129 active+clean; 632 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.5 MiB/s wr, 99 op/s 2023-07-18T03:20:25.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:25 smithi101 ceph-mon[126379]: pgmap v402: 129 pgs: 129 active+clean; 632 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.5 MiB/s wr, 99 op/s 2023-07-18T03:20:25.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:25 smithi155 ceph-mon[128885]: pgmap v402: 129 pgs: 129 active+clean; 632 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.5 MiB/s wr, 99 op/s 2023-07-18T03:20:25.630 INFO:tasks.workunit.client.0.smithi101.stdout: 1 746207 5.19 MB/sec execute 590 sec latency 9493.811 ms 2023-07-18T03:20:26.630 INFO:tasks.workunit.client.0.smithi101.stdout: 1 747335 5.19 MB/sec execute 591 sec latency 9636.552 ms 2023-07-18T03:20:27.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:27 smithi179 ceph-mon[129229]: pgmap v403: 129 pgs: 129 active+clean; 636 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.3 MiB/s wr, 70 op/s 2023-07-18T03:20:27.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:27 smithi101 ceph-mon[126379]: pgmap v403: 129 pgs: 129 active+clean; 636 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.3 MiB/s wr, 70 op/s 2023-07-18T03:20:27.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:27 smithi155 ceph-mon[128885]: pgmap v403: 129 pgs: 129 active+clean; 636 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.3 MiB/s wr, 70 op/s 2023-07-18T03:20:27.631 INFO:tasks.workunit.client.0.smithi101.stdout: 1 747368 5.18 MB/sec execute 592 sec latency 979.881 ms 2023-07-18T03:20:28.631 INFO:tasks.workunit.client.0.smithi101.stdout: 1 747368 5.17 MB/sec execute 593 sec latency 1979.971 ms 2023-07-18T03:20:29.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:29 smithi179 ceph-mon[129229]: pgmap v404: 129 pgs: 129 active+clean; 636 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.3 MiB/s wr, 70 op/s 2023-07-18T03:20:29.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:29 smithi101 ceph-mon[126379]: pgmap v404: 129 pgs: 129 active+clean; 636 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.3 MiB/s wr, 70 op/s 2023-07-18T03:20:29.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:29 smithi155 ceph-mon[128885]: pgmap v404: 129 pgs: 129 active+clean; 636 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.3 MiB/s wr, 70 op/s 2023-07-18T03:20:29.631 INFO:tasks.workunit.client.0.smithi101.stdout: 1 747817 5.17 MB/sec execute 594 sec latency 2804.612 ms 2023-07-18T03:20:30.631 INFO:tasks.workunit.client.0.smithi101.stdout: 1 749523 5.18 MB/sec execute 595 sec latency 18.389 ms 2023-07-18T03:20:31.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:31 smithi179 ceph-mon[129229]: pgmap v405: 129 pgs: 129 active+clean; 636 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.8 MiB/s wr, 105 op/s 2023-07-18T03:20:31.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:31 smithi101 ceph-mon[126379]: pgmap v405: 129 pgs: 129 active+clean; 636 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.8 MiB/s wr, 105 op/s 2023-07-18T03:20:31.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:31 smithi155 ceph-mon[128885]: pgmap v405: 129 pgs: 129 active+clean; 636 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.8 MiB/s wr, 105 op/s 2023-07-18T03:20:31.631 INFO:tasks.workunit.client.0.smithi101.stdout: 1 750655 5.18 MB/sec execute 596 sec latency 18.536 ms 2023-07-18T03:20:32.631 INFO:tasks.workunit.client.0.smithi101.stdout: 1 750911 5.17 MB/sec execute 597 sec latency 795.318 ms 2023-07-18T03:20:33.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:33 smithi179 ceph-mon[129229]: pgmap v406: 129 pgs: 129 active+clean; 636 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 997 KiB/s wr, 51 op/s 2023-07-18T03:20:33.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:33 smithi101 ceph-mon[126379]: pgmap v406: 129 pgs: 129 active+clean; 636 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 997 KiB/s wr, 51 op/s 2023-07-18T03:20:33.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:33 smithi155 ceph-mon[128885]: pgmap v406: 129 pgs: 129 active+clean; 636 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 997 KiB/s wr, 51 op/s 2023-07-18T03:20:33.631 INFO:tasks.workunit.client.0.smithi101.stdout: 1 750911 5.16 MB/sec execute 598 sec latency 1795.450 ms 2023-07-18T03:20:34.631 INFO:tasks.workunit.client.0.smithi101.stdout: 1 752393 5.17 MB/sec execute 599 sec latency 2068.058 ms 2023-07-18T03:20:35.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:35 smithi179 ceph-mon[129229]: pgmap v407: 129 pgs: 129 active+clean; 636 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 997 KiB/s wr, 51 op/s 2023-07-18T03:20:35.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:35 smithi101 ceph-mon[126379]: pgmap v407: 129 pgs: 129 active+clean; 636 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 997 KiB/s wr, 51 op/s 2023-07-18T03:20:35.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:35 smithi155 ceph-mon[128885]: pgmap v407: 129 pgs: 129 active+clean; 636 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 997 KiB/s wr, 51 op/s 2023-07-18T03:20:35.631 INFO:tasks.workunit.client.0.smithi101.stdout: 1 cleanup 600 sec 2023-07-18T03:20:35.844 INFO:tasks.workunit.client.0.smithi101.stdout: 0 cleanup 600 sec 2023-07-18T03:20:35.844 INFO:tasks.workunit.client.0.smithi101.stdout: 2023-07-18T03:20:35.844 INFO:tasks.workunit.client.0.smithi101.stdout: Operation Count AvgLat MaxLat 2023-07-18T03:20:35.845 INFO:tasks.workunit.client.0.smithi101.stdout: ---------------------------------------- 2023-07-18T03:20:35.845 INFO:tasks.workunit.client.0.smithi101.stdout: NTCreateX 99048 1.862 4925.638 2023-07-18T03:20:35.845 INFO:tasks.workunit.client.0.smithi101.stdout: Close 72710 0.004 0.188 2023-07-18T03:20:35.845 INFO:tasks.workunit.client.0.smithi101.stdout: Rename 4191 46.903 4871.122 2023-07-18T03:20:35.845 INFO:tasks.workunit.client.0.smithi101.stdout: Unlink 20037 2.429 9636.527 2023-07-18T03:20:35.845 INFO:tasks.workunit.client.0.smithi101.stdout: Deltree 2 93.320 109.095 2023-07-18T03:20:35.845 INFO:tasks.workunit.client.0.smithi101.stdout: Mkdir 1 0.016 0.016 2023-07-18T03:20:35.846 INFO:tasks.workunit.client.0.smithi101.stdout: Qpathinfo 89766 1.235 1382.925 2023-07-18T03:20:35.846 INFO:tasks.workunit.client.0.smithi101.stdout: Qfileinfo 15696 0.002 0.045 2023-07-18T03:20:35.846 INFO:tasks.workunit.client.0.smithi101.stdout: Qfsinfo 16457 0.291 34.141 2023-07-18T03:20:35.846 INFO:tasks.workunit.client.0.smithi101.stdout: Sfileinfo 8056 0.027 8.295 2023-07-18T03:20:35.846 INFO:tasks.workunit.client.0.smithi101.stdout: Find 34680 0.249 46.039 2023-07-18T03:20:35.846 INFO:tasks.workunit.client.0.smithi101.stdout: WriteX 49242 0.178 99.395 2023-07-18T03:20:35.846 INFO:tasks.workunit.client.0.smithi101.stdout: ReadX 155080 0.027 239.245 2023-07-18T03:20:35.847 INFO:tasks.workunit.client.0.smithi101.stdout: LockX 322 0.828 3.490 2023-07-18T03:20:35.847 INFO:tasks.workunit.client.0.smithi101.stdout: UnlockX 322 0.849 4.582 2023-07-18T03:20:35.847 INFO:tasks.workunit.client.0.smithi101.stdout: Flush 6932 4.293 157.033 2023-07-18T03:20:35.847 INFO:tasks.workunit.client.0.smithi101.stdout: 2023-07-18T03:20:35.847 INFO:tasks.workunit.client.0.smithi101.stdout:Throughput 5.1676 MB/sec 1 clients 1 procs max_latency=9636.552 ms 2023-07-18T03:20:35.847 INFO:tasks.workunit.client.0.smithi101.stderr:+ dbench 10 2023-07-18T03:20:35.848 INFO:tasks.workunit.client.0.smithi101.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2023-07-18T03:20:35.848 INFO:tasks.workunit.client.0.smithi101.stdout: 2023-07-18T03:20:35.848 INFO:tasks.workunit.client.0.smithi101.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2023-07-18T03:20:35.849 INFO:tasks.workunit.client.0.smithi101.stdout:9 of 10 processes prepared for launch 0 sec 2023-07-18T03:20:35.949 INFO:tasks.workunit.client.0.smithi101.stdout:10 of 10 processes prepared for launch 0 sec 2023-07-18T03:20:35.950 INFO:tasks.workunit.client.0.smithi101.stdout:releasing clients 2023-07-18T03:20:36.950 INFO:tasks.workunit.client.0.smithi101.stdout: 10 374 133.56 MB/sec warmup 1 sec latency 63.023 ms 2023-07-18T03:20:37.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:37 smithi179 ceph-mon[129229]: pgmap v408: 129 pgs: 129 active+clean; 643 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 105 op/s 2023-07-18T03:20:37.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:37 smithi101 ceph-mon[126379]: pgmap v408: 129 pgs: 129 active+clean; 643 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 105 op/s 2023-07-18T03:20:37.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:37 smithi155 ceph-mon[128885]: pgmap v408: 129 pgs: 129 active+clean; 643 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 105 op/s 2023-07-18T03:20:37.950 INFO:tasks.workunit.client.0.smithi101.stdout: 10 777 102.28 MB/sec warmup 2 sec latency 912.424 ms 2023-07-18T03:20:38.949 INFO:tasks.workunit.client.0.smithi101.stdout: 10 1338 81.66 MB/sec warmup 3 sec latency 1912.465 ms 2023-07-18T03:20:39.000 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:20:39.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.215+0000 7f9f5dbff700 1 -- 172.21.15.101:0/942498434 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f58100490 msgr2=0x7f9f5810c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:39.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.215+0000 7f9f5dbff700 1 --2- 172.21.15.101:0/942498434 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f58100490 0x7f9f5810c920 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000b0a0 tx=0x7f9f5000b3b0 comp rx=0 tx=0).stop 2023-07-18T03:20:39.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.215+0000 7f9f5dbff700 1 -- 172.21.15.101:0/942498434 shutdown_connections 2023-07-18T03:20:39.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.215+0000 7f9f5dbff700 1 --2- 172.21.15.101:0/942498434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f58105af0 0x7f9f58105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:39.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.215+0000 7f9f5dbff700 1 --2- 172.21.15.101:0/942498434 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f58100490 0x7f9f5810c920 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:39.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.215+0000 7f9f5dbff700 1 --2- 172.21.15.101:0/942498434 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9f580ffae0 0x7f9f580fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:39.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.215+0000 7f9f5dbff700 1 -- 172.21.15.101:0/942498434 >> 172.21.15.101:0/942498434 conn(0x7f9f580fb5e0 msgr2=0x7f9f580fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:39.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.215+0000 7f9f5dbff700 1 -- 172.21.15.101:0/942498434 shutdown_connections 2023-07-18T03:20:39.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.215+0000 7f9f5dbff700 1 -- 172.21.15.101:0/942498434 wait complete. 2023-07-18T03:20:39.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.216+0000 7f9f5dbff700 1 Processor -- start 2023-07-18T03:20:39.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.216+0000 7f9f5dbff700 1 -- start start 2023-07-18T03:20:39.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.216+0000 7f9f5dbff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9f580ffae0 0x7f9f58072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:39.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.216+0000 7f9f5dbff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f58100490 0x7f9f58072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:39.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.217+0000 7f9f5dbff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f58105af0 0x7f9f581a3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:39.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.217+0000 7f9f5dbff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f58073580 con 0x7f9f580ffae0 2023-07-18T03:20:39.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.217+0000 7f9f5dbff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f581a4440 con 0x7f9f58100490 2023-07-18T03:20:39.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.217+0000 7f9f5dbff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f581a4580 con 0x7f9f58105af0 2023-07-18T03:20:39.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.217+0000 7f9f57fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f58105af0 0x7f9f581a3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:39.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.217+0000 7f9f577fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9f580ffae0 0x7f9f58072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:39.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.217+0000 7f9f56ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f58100490 0x7f9f58072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:39.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.217+0000 7f9f57fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f58105af0 0x7f9f581a3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41498/0 (socket says 172.21.15.101:41498) 2023-07-18T03:20:39.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.217+0000 7f9f57fff700 1 -- 172.21.15.101:0/146061267 learned_addr learned my addr 172.21.15.101:0/146061267 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:39.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.217+0000 7f9f57fff700 1 -- 172.21.15.101:0/146061267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f58100490 msgr2=0x7f9f58072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:39.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.217+0000 7f9f57fff700 1 --2- 172.21.15.101:0/146061267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f58100490 0x7f9f58072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:39.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.217+0000 7f9f57fff700 1 -- 172.21.15.101:0/146061267 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9f580ffae0 msgr2=0x7f9f58072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:39.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.217+0000 7f9f57fff700 1 --2- 172.21.15.101:0/146061267 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9f580ffae0 0x7f9f58072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:39.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.217+0000 7f9f57fff700 1 -- 172.21.15.101:0/146061267 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f50009d00 con 0x7f9f58105af0 2023-07-18T03:20:39.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.217+0000 7f9f577fe700 1 --2- 172.21.15.101:0/146061267 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9f580ffae0 0x7f9f58072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:20:39.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.217+0000 7f9f56ffd700 1 --2- 172.21.15.101:0/146061267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f58100490 0x7f9f58072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:20:39.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.217+0000 7f9f57fff700 1 --2- 172.21.15.101:0/146061267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f58105af0 0x7f9f581a3f00 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9f50009fd0 tx=0x7f9f5000bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:39.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.218+0000 7f9f54ff9700 1 -- 172.21.15.101:0/146061267 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5000e040 con 0x7f9f58105af0 2023-07-18T03:20:39.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.218+0000 7f9f5dbff700 1 -- 172.21.15.101:0/146061267 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f581a4780 con 0x7f9f58105af0 2023-07-18T03:20:39.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.218+0000 7f9f54ff9700 1 -- 172.21.15.101:0/146061267 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f500094c0 con 0x7f9f58105af0 2023-07-18T03:20:39.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.218+0000 7f9f54ff9700 1 -- 172.21.15.101:0/146061267 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f500052a0 con 0x7f9f58105af0 2023-07-18T03:20:39.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.218+0000 7f9f5dbff700 1 -- 172.21.15.101:0/146061267 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f581a4bb0 con 0x7f9f58105af0 2023-07-18T03:20:39.218 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.223+0000 7f9f54ff9700 1 -- 172.21.15.101:0/146061267 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f9f5001d470 con 0x7f9f58105af0 2023-07-18T03:20:39.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.223+0000 7f9f7150ab80 1 Processor -- start 2023-07-18T03:20:39.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.223+0000 7f9f7150ab80 1 -- start start 2023-07-18T03:20:39.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.224+0000 7f9f7150ab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574f47bb2d0 0x5574f47bb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:39.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.224+0000 7f9f7150ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574f47bbbe0 0x5574f47c6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:39.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.224+0000 7f9f7150ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f47c6ad0 0x5574f47c8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:39.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.224+0000 7f9f7150ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f46c1cb0 con 0x5574f47bbbe0 2023-07-18T03:20:39.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.224+0000 7f9f7150ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f4714a40 con 0x5574f47c6ad0 2023-07-18T03:20:39.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.224+0000 7f9f7150ab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f470ded0 con 0x5574f47bb2d0 2023-07-18T03:20:39.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.224+0000 7f9f56ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574f47bb2d0 0x5574f47bb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:39.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.224+0000 7f9f577fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574f47bbbe0 0x5574f47c6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:39.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.224+0000 7f9f54ff9700 1 --2- 172.21.15.101:0/146061267 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9f400a12c0 0x7f9f400a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:39.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.224+0000 7f9f577fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574f47bbbe0 0x5574f47c6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:56956/0 (socket says 172.21.15.101:56956) 2023-07-18T03:20:39.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.224+0000 7f9f577fe700 1 -- 172.21.15.101:0/1660877121 learned_addr learned my addr 172.21.15.101:0/1660877121 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:39.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.224+0000 7f9f56ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574f47bb2d0 0x5574f47bb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41512/0 (socket says 172.21.15.101:41512) 2023-07-18T03:20:39.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.224+0000 7f9f56ffd700 1 --2- 172.21.15.101:0/1660877121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f47c6ad0 0x5574f47c8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:39.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.224+0000 7f9f54ff9700 1 -- 172.21.15.101:0/146061267 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f9f500ce6b0 con 0x7f9f58105af0 2023-07-18T03:20:39.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.224+0000 7f9f57fff700 1 --2- 172.21.15.101:0/146061267 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9f400a12c0 0x7f9f400a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:39.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.225+0000 7f9f577fe700 1 -- 172.21.15.101:0/1660877121 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574f47bb2d0 msgr2=0x5574f47bb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:39.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.225+0000 7f9f577fe700 1 --2- 172.21.15.101:0/1660877121 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574f47bb2d0 0x5574f47bb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:39.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.225+0000 7f9f577fe700 1 -- 172.21.15.101:0/1660877121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f47c6ad0 msgr2=0x5574f47c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:39.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.225+0000 7f9f577fe700 1 --2- 172.21.15.101:0/1660877121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f47c6ad0 0x5574f47c8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:39.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.225+0000 7f9f577fe700 1 -- 172.21.15.101:0/1660877121 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574f453bde0 con 0x5574f47bbbe0 2023-07-18T03:20:39.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.225+0000 7f9f56ffd700 1 --2- 172.21.15.101:0/1660877121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f47c6ad0 0x5574f47c8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:20:39.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.225+0000 7f9f577fe700 1 --2- 172.21.15.101:0/1660877121 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574f47bbbe0 0x5574f47c6590 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f9f480115f0 tx=0x7f9f48011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2441ae9c1756f595 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:39.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.225+0000 7f9f3dffb700 1 -- 172.21.15.101:0/1660877121 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f48019630 con 0x5574f47bbbe0 2023-07-18T03:20:39.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.225+0000 7f9f57fff700 1 --2- 172.21.15.101:0/146061267 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9f400a12c0 0x7f9f400a3770 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9f500cbab0 tx=0x7f9f5000b3b0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:39.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.225+0000 7f9f3dffb700 1 -- 172.21.15.101:0/1660877121 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f48019c50 con 0x5574f47bbbe0 2023-07-18T03:20:39.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.225+0000 7f9f3dffb700 1 -- 172.21.15.101:0/1660877121 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4800ec80 con 0x5574f47bbbe0 2023-07-18T03:20:39.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.226+0000 7f9f7150ab80 1 -- 172.21.15.101:0/1660877121 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574f47bbbe0 msgr2=0x5574f47c6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:39.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.226+0000 7f9f7150ab80 1 --2- 172.21.15.101:0/1660877121 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574f47bbbe0 0x5574f47c6590 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f9f480115f0 tx=0x7f9f48011900 comp rx=0 tx=0).stop 2023-07-18T03:20:39.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.226+0000 7f9f7150ab80 1 -- 172.21.15.101:0/1660877121 shutdown_connections 2023-07-18T03:20:39.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.226+0000 7f9f7150ab80 1 --2- 172.21.15.101:0/1660877121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f47c6ad0 0x5574f47c8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:39.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.226+0000 7f9f7150ab80 1 --2- 172.21.15.101:0/1660877121 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574f47bb2d0 0x5574f47bb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:39.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.226+0000 7f9f7150ab80 1 --2- 172.21.15.101:0/1660877121 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574f47bbbe0 0x5574f47c6590 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:39.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.226+0000 7f9f7150ab80 1 -- 172.21.15.101:0/1660877121 >> 172.21.15.101:0/1660877121 conn(0x5574f47b1da0 msgr2=0x5574f47b76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:39.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.226+0000 7f9f7150ab80 1 -- 172.21.15.101:0/1660877121 shutdown_connections 2023-07-18T03:20:39.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.226+0000 7f9f7150ab80 1 -- 172.21.15.101:0/1660877121 wait complete. 2023-07-18T03:20:39.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.227+0000 7f9f7150ab80 1 -- start start 2023-07-18T03:20:39.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.227+0000 7f9f7150ab80 1 Processor -- start 2023-07-18T03:20:39.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.228+0000 7f9f7150ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574f47bb2d0 0x5574f47c6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:39.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.228+0000 7f9f7150ab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574f47bbbe0 0x5574f47c3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:39.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.228+0000 7f9f7150ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f47c6ad0 0x5574f47c3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:39.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.228+0000 7f9f7150ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f46adac0 con 0x5574f47bb2d0 2023-07-18T03:20:39.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.228+0000 7f9f7150ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f46ad7c0 con 0x5574f47c6ad0 2023-07-18T03:20:39.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.228+0000 7f9f7150ab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f4591920 con 0x5574f47bbbe0 2023-07-18T03:20:39.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.228+0000 7f9f7150ab80 10 client.0 _ll_register_callbacks cb 0x5574f46ab760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:20:39.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.228+0000 7f9f577fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574f47bbbe0 0x5574f47c3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:39.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.228+0000 7f9f56ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574f47bb2d0 0x5574f47c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:39.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.228+0000 7f9f577fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574f47bbbe0 0x5574f47c3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41528/0 (socket says 172.21.15.101:41528) 2023-07-18T03:20:39.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.228+0000 7f9f577fe700 1 -- 172.21.15.101:0/3496862607 learned_addr learned my addr 172.21.15.101:0/3496862607 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:39.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.228+0000 7f9f56ffd700 1 --2- 172.21.15.101:0/3496862607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f47c6ad0 0x5574f47c3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:39.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.228+0000 7f9f577fe700 1 -- 172.21.15.101:0/3496862607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f47c6ad0 msgr2=0x5574f47c3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:39.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.228+0000 7f9f577fe700 1 --2- 172.21.15.101:0/3496862607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f47c6ad0 0x5574f47c3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:39.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.228+0000 7f9f577fe700 1 -- 172.21.15.101:0/3496862607 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574f47bb2d0 msgr2=0x5574f47c6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:39.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.228+0000 7f9f577fe700 1 --2- 172.21.15.101:0/3496862607 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574f47bb2d0 0x5574f47c6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:39.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.228+0000 7f9f577fe700 1 -- 172.21.15.101:0/3496862607 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f48009d50 con 0x5574f47bbbe0 2023-07-18T03:20:39.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.229+0000 7f9f577fe700 1 --2- 172.21.15.101:0/3496862607 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574f47bbbe0 0x5574f47c3750 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9f48011c50 tx=0x7f9f4800ea50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4da06be4879673c6 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:39.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.229+0000 7f9f36ffd700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:20:39.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.229+0000 7f9f36ffd700 1 -- 172.21.15.101:0/3496862607 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f48019630 con 0x5574f47bbbe0 2023-07-18T03:20:39.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.229+0000 7f9f7150ab80 1 -- 172.21.15.101:0/3496862607 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5574f46674f0 con 0x5574f47bbbe0 2023-07-18T03:20:39.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.229+0000 7f9f36ffd700 1 -- 172.21.15.101:0/3496862607 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f9f48005ce0 con 0x5574f47bbbe0 2023-07-18T03:20:39.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.229+0000 7f9f36ffd700 1 -- 172.21.15.101:0/3496862607 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f4800ee50 con 0x5574f47bbbe0 2023-07-18T03:20:39.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.229+0000 7f9f7150ab80 10 client.28213 fetch_fsmap learned FSMap version 34 2023-07-18T03:20:39.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.229+0000 7f9f7150ab80 1 -- 172.21.15.101:0/3496862607 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5574f4524c60 con 0x5574f47bbbe0 2023-07-18T03:20:39.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.230+0000 7f9f36ffd700 1 -- 172.21.15.101:0/3496862607 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5574f46674f0 con 0x5574f47bbbe0 2023-07-18T03:20:39.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.230+0000 7f9f36ffd700 1 -- 172.21.15.101:0/3496862607 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 34) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f9f48029020 con 0x5574f47bbbe0 2023-07-18T03:20:39.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.230+0000 7f9f7150ab80 10 client.28213 fetch_fsmap finished waiting for FSMap version 34 2023-07-18T03:20:39.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.230+0000 7f9f7150ab80 10 client.28213 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:20:39.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.230+0000 7f9f7150ab80 20 client.28213 populate_metadata read hostname 'smithi101' 2023-07-18T03:20:39.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.230+0000 7f9f7150ab80 1 --2- 172.21.15.101:0/3496862607 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5574f47d61d0 0x5574f47d85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:39.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.230+0000 7f9f7150ab80 4 client.28213 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:20:39.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.230+0000 7f9f7150ab80 1 -- 172.21.15.101:0/3496862607 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5574f4706eb0 con 0x5574f47d61d0 2023-07-18T03:20:39.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.230+0000 7f9f56ffd700 1 --2- 172.21.15.101:0/3496862607 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5574f47d61d0 0x5574f47d85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:39.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.231+0000 7f9f36ffd700 10 client.28213 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:20:39.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.231+0000 7f9f56ffd700 1 --2- 172.21.15.101:0/3496862607 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5574f47d61d0 0x5574f47d85e0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1055410658eb71c9 server_cookie=a515d3dc63bb881e in_seq=0 out_seq=0 2023-07-18T03:20:39.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.235+0000 7f9f36ffd700 1 -- 172.21.15.101:0/3496862607 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5574f4706eb0 con 0x5574f47d61d0 2023-07-18T03:20:39.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.235+0000 7f9f36ffd700 10 client.28213 handle_command_reply: tid=0 2023-07-18T03:20:39.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.235+0000 7f9f7150ab80 1 client.28213 shutdown 2023-07-18T03:20:39.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.235+0000 7f9f7150ab80 1 -- 172.21.15.101:0/3496862607 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574f47bbbe0 msgr2=0x5574f47c3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:39.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.235+0000 7f9f7150ab80 1 --2- 172.21.15.101:0/3496862607 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574f47bbbe0 0x5574f47c3750 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9f48011c50 tx=0x7f9f4800ea50 comp rx=0 tx=0).stop 2023-07-18T03:20:39.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.235+0000 7f9f7150ab80 1 -- 172.21.15.101:0/3496862607 shutdown_connections 2023-07-18T03:20:39.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.235+0000 7f9f7150ab80 1 --2- 172.21.15.101:0/3496862607 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5574f47d61d0 0x5574f47d85e0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:39.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.235+0000 7f9f7150ab80 1 --2- 172.21.15.101:0/3496862607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f47c6ad0 0x5574f47c3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:39.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.235+0000 7f9f7150ab80 1 --2- 172.21.15.101:0/3496862607 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574f47bb2d0 0x5574f47c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:39.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.235+0000 7f9f36ffd700 0 client.28213 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:20:39.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.235+0000 7f9f7150ab80 1 --2- 172.21.15.101:0/3496862607 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574f47bbbe0 0x5574f47c3750 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:39.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.235+0000 7f9f7150ab80 1 -- 172.21.15.101:0/3496862607 >> 172.21.15.101:0/3496862607 conn(0x5574f47b1da0 msgr2=0x5574f47b7230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:39.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.235+0000 7f9f7150ab80 1 -- 172.21.15.101:0/3496862607 shutdown_connections 2023-07-18T03:20:39.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.236+0000 7f9f7150ab80 1 -- 172.21.15.101:0/3496862607 wait complete. 2023-07-18T03:20:39.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.236+0000 7f9f7150ab80 20 client.28213 trim_cache size 0 max 16384 2023-07-18T03:20:39.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.253+0000 7f9f7150ab80 1 Processor -- start 2023-07-18T03:20:39.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.253+0000 7f9f7150ab80 1 -- start start 2023-07-18T03:20:39.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.253+0000 7f9f7150ab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574f47bb2d0 0x5574f47c6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:39.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.253+0000 7f9f7150ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f47bbbe0 0x5574f47c2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:39.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.253+0000 7f9f7150ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574f47c6ad0 0x5574f47c3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:39.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.253+0000 7f9f7150ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f47d8c40 con 0x5574f47c6ad0 2023-07-18T03:20:39.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.253+0000 7f9f7150ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f47c3a70 con 0x5574f47bbbe0 2023-07-18T03:20:39.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.253+0000 7f9f7150ab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f47b8c70 con 0x5574f47bb2d0 2023-07-18T03:20:39.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.254+0000 7f9f577fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f47bbbe0 0x5574f47c2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:39.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.254+0000 7f9f56ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574f47bb2d0 0x5574f47c6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:39.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.254+0000 7f9f56ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574f47bb2d0 0x5574f47c6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41540/0 (socket says 172.21.15.101:41540) 2023-07-18T03:20:39.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.254+0000 7f9f577fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f47bbbe0 0x5574f47c2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60732/0 (socket says 172.21.15.101:60732) 2023-07-18T03:20:39.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.254+0000 7f9f56ffd700 1 -- 172.21.15.101:0/3365499632 learned_addr learned my addr 172.21.15.101:0/3365499632 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:39.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.254+0000 7f9f56ffd700 1 --2- 172.21.15.101:0/3365499632 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574f47c6ad0 0x5574f47c3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:39.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.254+0000 7f9f577fe700 1 -- 172.21.15.101:0/3365499632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574f47bb2d0 msgr2=0x5574f47c6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:39.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.254+0000 7f9f577fe700 1 --2- 172.21.15.101:0/3365499632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574f47bb2d0 0x5574f47c6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:39.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.254+0000 7f9f577fe700 1 -- 172.21.15.101:0/3365499632 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574f47c6ad0 msgr2=0x5574f47c3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:39.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.254+0000 7f9f577fe700 1 --2- 172.21.15.101:0/3365499632 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574f47c6ad0 0x5574f47c3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:39.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.254+0000 7f9f577fe700 1 -- 172.21.15.101:0/3365499632 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574f4562250 con 0x5574f47bbbe0 2023-07-18T03:20:39.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.254+0000 7f9f577fe700 1 --2- 172.21.15.101:0/3365499632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f47bbbe0 0x5574f47c2ff0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9f48004860 tx=0x7f9f48005a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f9cd4dd44453f62 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:39.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.254+0000 7f9f3e7fc700 1 -- 172.21.15.101:0/3365499632 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f48005ce0 con 0x5574f47bbbe0 2023-07-18T03:20:39.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.254+0000 7f9f3e7fc700 1 -- 172.21.15.101:0/3365499632 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f48004ad0 con 0x5574f47bbbe0 2023-07-18T03:20:39.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.254+0000 7f9f3e7fc700 1 -- 172.21.15.101:0/3365499632 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f48019430 con 0x5574f47bbbe0 2023-07-18T03:20:39.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.255+0000 7f9f7150ab80 1 -- 172.21.15.101:0/3365499632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f47bbbe0 msgr2=0x5574f47c2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:39.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.255+0000 7f9f7150ab80 1 --2- 172.21.15.101:0/3365499632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f47bbbe0 0x5574f47c2ff0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9f48004860 tx=0x7f9f48005a00 comp rx=0 tx=0).stop 2023-07-18T03:20:39.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.255+0000 7f9f7150ab80 1 -- 172.21.15.101:0/3365499632 shutdown_connections 2023-07-18T03:20:39.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.255+0000 7f9f7150ab80 1 --2- 172.21.15.101:0/3365499632 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574f47c6ad0 0x5574f47c3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:39.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.255+0000 7f9f7150ab80 1 --2- 172.21.15.101:0/3365499632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574f47bb2d0 0x5574f47c6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:39.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.255+0000 7f9f7150ab80 1 --2- 172.21.15.101:0/3365499632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f47bbbe0 0x5574f47c2ff0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:39.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.255+0000 7f9f7150ab80 1 -- 172.21.15.101:0/3365499632 >> 172.21.15.101:0/3365499632 conn(0x5574f47b1da0 msgr2=0x5574f47bea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:39.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.255+0000 7f9f7150ab80 1 -- 172.21.15.101:0/3365499632 shutdown_connections 2023-07-18T03:20:39.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.255+0000 7f9f7150ab80 1 -- 172.21.15.101:0/3365499632 wait complete. 2023-07-18T03:20:39.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.256+0000 7f9f7150ab80 1 -- start start 2023-07-18T03:20:39.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.256+0000 7f9f7150ab80 1 Processor -- start 2023-07-18T03:20:39.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.256+0000 7f9f7150ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574f47bb2d0 0x5574f47cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:39.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.256+0000 7f9f7150ab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574f47bbbe0 0x5574f47cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:39.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.256+0000 7f9f7150ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f47c6ad0 0x5574f47cb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:39.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.256+0000 7f9f7150ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f47cbe00 con 0x5574f47bb2d0 2023-07-18T03:20:39.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.256+0000 7f9f7150ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f47cbf70 con 0x5574f47c6ad0 2023-07-18T03:20:39.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.256+0000 7f9f7150ab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f47cc130 con 0x5574f47bbbe0 2023-07-18T03:20:39.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.257+0000 7f9f7150ab80 10 client.0 _ll_register_callbacks cb 0x7f9f4c009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:20:39.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.257+0000 7f9f56ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574f47bb2d0 0x5574f47cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:39.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.257+0000 7f9f56ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574f47bb2d0 0x5574f47cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:56996/0 (socket says 172.21.15.101:56996) 2023-07-18T03:20:39.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.257+0000 7f9f56ffd700 1 -- 172.21.15.101:0/1015689365 learned_addr learned my addr 172.21.15.101:0/1015689365 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:39.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.257+0000 7f9f577fe700 1 --2- 172.21.15.101:0/1015689365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574f47bbbe0 0x5574f47cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:39.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.257+0000 7f9f56ffd700 1 --2- 172.21.15.101:0/1015689365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f47c6ad0 0x5574f47cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:39.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.257+0000 7f9f56ffd700 1 -- 172.21.15.101:0/1015689365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574f47bbbe0 msgr2=0x5574f47cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:39.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.257+0000 7f9f56ffd700 1 --2- 172.21.15.101:0/1015689365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574f47bbbe0 0x5574f47cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:39.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.257+0000 7f9f56ffd700 1 -- 172.21.15.101:0/1015689365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f47c6ad0 msgr2=0x5574f47cb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:39.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.257+0000 7f9f56ffd700 1 --2- 172.21.15.101:0/1015689365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f47c6ad0 0x5574f47cb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:39.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.257+0000 7f9f56ffd700 1 -- 172.21.15.101:0/1015689365 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5574f4524c60 con 0x5574f47bb2d0 2023-07-18T03:20:39.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.257+0000 7f9f56ffd700 1 --2- 172.21.15.101:0/1015689365 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574f47bb2d0 0x5574f47cd240 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x5574f47d8eb0 tx=0x7f9f4c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=767dd148a2785a3e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:39.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.257+0000 7f9f36ffd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:20:39.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.257+0000 7f9f36ffd700 1 -- 172.21.15.101:0/1015689365 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c004120 con 0x5574f47bb2d0 2023-07-18T03:20:39.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.258+0000 7f9f7150ab80 1 -- 172.21.15.101:0/1015689365 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9f48019c50 con 0x5574f47bb2d0 2023-07-18T03:20:39.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.258+0000 7f9f36ffd700 1 -- 172.21.15.101:0/1015689365 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f9f4c004280 con 0x5574f47bb2d0 2023-07-18T03:20:39.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.258+0000 7f9f36ffd700 1 -- 172.21.15.101:0/1015689365 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f4c004440 con 0x5574f47bb2d0 2023-07-18T03:20:39.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.258+0000 7f9f7150ab80 10 client.18144 fetch_fsmap learned FSMap version 34 2023-07-18T03:20:39.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.258+0000 7f9f7150ab80 1 -- 172.21.15.101:0/1015689365 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5574f453a2b0 con 0x5574f47bb2d0 2023-07-18T03:20:39.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.258+0000 7f9f36ffd700 1 -- 172.21.15.101:0/1015689365 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f48019c50 con 0x5574f47bb2d0 2023-07-18T03:20:39.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.258+0000 7f9f36ffd700 1 -- 172.21.15.101:0/1015689365 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 34) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f9f4c00f600 con 0x5574f47bb2d0 2023-07-18T03:20:39.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.258+0000 7f9f7150ab80 10 client.18144 fetch_fsmap finished waiting for FSMap version 34 2023-07-18T03:20:39.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.258+0000 7f9f7150ab80 10 client.18144 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:20:39.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.258+0000 7f9f7150ab80 20 client.18144 populate_metadata read hostname 'smithi101' 2023-07-18T03:20:39.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.258+0000 7f9f7150ab80 1 --2- 172.21.15.101:0/1015689365 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5574f482b440 0x5574f482d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:39.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.258+0000 7f9f7150ab80 4 client.18144 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:20:39.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.258+0000 7f9f7150ab80 1 -- 172.21.15.101:0/1015689365 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5574f46c9050 con 0x5574f482b440 2023-07-18T03:20:39.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.259+0000 7f9f577fe700 1 --2- 172.21.15.101:0/1015689365 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5574f482b440 0x5574f482d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:39.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.259+0000 7f9f577fe700 1 --2- 172.21.15.101:0/1015689365 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5574f482b440 0x5574f482d850 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7885e9275d065fca server_cookie=4bb85be8ba53def9 in_seq=0 out_seq=0 2023-07-18T03:20:39.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.259+0000 7f9f36ffd700 10 client.18144 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:20:39.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.263+0000 7f9f36ffd700 1 -- 172.21.15.101:0/1015689365 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5574f46c9050 con 0x5574f482b440 2023-07-18T03:20:39.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.263+0000 7f9f36ffd700 10 client.18144 handle_command_reply: tid=0 2023-07-18T03:20:39.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.263+0000 7f9f7150ab80 1 client.18144 shutdown 2023-07-18T03:20:39.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.263+0000 7f9f7150ab80 1 -- 172.21.15.101:0/1015689365 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574f47bb2d0 msgr2=0x5574f47cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:39.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.263+0000 7f9f7150ab80 1 --2- 172.21.15.101:0/1015689365 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574f47bb2d0 0x5574f47cd240 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x5574f47d8eb0 tx=0x7f9f4c009260 comp rx=0 tx=0).stop 2023-07-18T03:20:39.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.264+0000 7f9f7150ab80 1 -- 172.21.15.101:0/1015689365 shutdown_connections 2023-07-18T03:20:39.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.264+0000 7f9f7150ab80 1 --2- 172.21.15.101:0/1015689365 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5574f482b440 0x5574f482d850 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:39.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.264+0000 7f9f7150ab80 1 --2- 172.21.15.101:0/1015689365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f47c6ad0 0x5574f47cb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:39.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.264+0000 7f9f7150ab80 1 --2- 172.21.15.101:0/1015689365 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574f47bb2d0 0x5574f47cd240 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:39.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.264+0000 7f9f7150ab80 1 --2- 172.21.15.101:0/1015689365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574f47bbbe0 0x5574f47cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:39.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.264+0000 7f9f7150ab80 1 -- 172.21.15.101:0/1015689365 >> 172.21.15.101:0/1015689365 conn(0x5574f47b1da0 msgr2=0x5574f47c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:39.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.264+0000 7f9f36ffd700 0 client.18144 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:20:39.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.264+0000 7f9f7150ab80 1 -- 172.21.15.101:0/1015689365 shutdown_connections 2023-07-18T03:20:39.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.264+0000 7f9f7150ab80 1 -- 172.21.15.101:0/1015689365 wait complete. 2023-07-18T03:20:39.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.264+0000 7f9f7150ab80 20 client.18144 trim_cache size 0 max 16384 2023-07-18T03:20:39.265 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:20:39.265 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "no active scrubs running", 2023-07-18T03:20:39.265 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": {} 2023-07-18T03:20:39.265 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:20:39.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.264+0000 7f9f5dbff700 1 -- 172.21.15.101:0/146061267 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9f400a12c0 msgr2=0x7f9f400a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:39.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.264+0000 7f9f5dbff700 1 --2- 172.21.15.101:0/146061267 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9f400a12c0 0x7f9f400a3770 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9f500cbab0 tx=0x7f9f5000b3b0 comp rx=0 tx=0).stop 2023-07-18T03:20:39.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.265+0000 7f9f5dbff700 1 -- 172.21.15.101:0/146061267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f58105af0 msgr2=0x7f9f581a3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:39.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.265+0000 7f9f5dbff700 1 --2- 172.21.15.101:0/146061267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f58105af0 0x7f9f581a3f00 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9f50009fd0 tx=0x7f9f5000bce0 comp rx=0 tx=0).stop 2023-07-18T03:20:39.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.265+0000 7f9f5dbff700 1 -- 172.21.15.101:0/146061267 shutdown_connections 2023-07-18T03:20:39.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.265+0000 7f9f5dbff700 1 --2- 172.21.15.101:0/146061267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f58105af0 0x7f9f581a3f00 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:39.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.265+0000 7f9f5dbff700 1 --2- 172.21.15.101:0/146061267 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9f580ffae0 0x7f9f58072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:39.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.265+0000 7f9f5dbff700 1 --2- 172.21.15.101:0/146061267 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9f400a12c0 0x7f9f400a3770 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:39.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.265+0000 7f9f5dbff700 1 --2- 172.21.15.101:0/146061267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f58100490 0x7f9f58072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:39.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.265+0000 7f9f5dbff700 1 -- 172.21.15.101:0/146061267 >> 172.21.15.101:0/146061267 conn(0x7f9f580fb5e0 msgr2=0x7f9f580fba90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:39.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.265+0000 7f9f5dbff700 1 -- 172.21.15.101:0/146061267 shutdown_connections 2023-07-18T03:20:39.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.265+0000 7f9f5dbff700 1 -- 172.21.15.101:0/146061267 wait complete. 2023-07-18T03:20:39.280 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-18T03:20:39.280 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:20:39.347 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:39 smithi101 ceph-mon[126379]: pgmap v409: 129 pgs: 129 active+clean; 643 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 1.2 MiB/s wr, 88 op/s 2023-07-18T03:20:39.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:39 smithi179 ceph-mon[129229]: pgmap v409: 129 pgs: 129 active+clean; 643 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 1.2 MiB/s wr, 88 op/s 2023-07-18T03:20:39.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.514+0000 7f5c2ddff700 1 -- 172.21.15.101:0/168258110 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c2810c100 msgr2=0x7f5c2810c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:39.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.514+0000 7f5c2ddff700 1 --2- 172.21.15.101:0/168258110 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c2810c100 0x7f5c2810c4d0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5c180099c0 tx=0x7f5c18009cd0 comp rx=0 tx=0).stop 2023-07-18T03:20:39.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.514+0000 7f5c2ddff700 1 -- 172.21.15.101:0/168258110 shutdown_connections 2023-07-18T03:20:39.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.514+0000 7f5c2ddff700 1 --2- 172.21.15.101:0/168258110 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5c28106cd0 0x7f5c28112f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:39.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.514+0000 7f5c2ddff700 1 --2- 172.21.15.101:0/168258110 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c2810c100 0x7f5c2810c4d0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:39.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.514+0000 7f5c2ddff700 1 --2- 172.21.15.101:0/168258110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c28106340 0x7f5c28106790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:39.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.514+0000 7f5c2ddff700 1 -- 172.21.15.101:0/168258110 >> 172.21.15.101:0/168258110 conn(0x7f5c280759f0 msgr2=0x7f5c28077e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:39.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.515+0000 7f5c2ddff700 1 -- 172.21.15.101:0/168258110 shutdown_connections 2023-07-18T03:20:39.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.515+0000 7f5c2ddff700 1 -- 172.21.15.101:0/168258110 wait complete. 2023-07-18T03:20:39.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.516+0000 7f5c2ddff700 1 Processor -- start 2023-07-18T03:20:39.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.516+0000 7f5c2ddff700 1 -- start start 2023-07-18T03:20:39.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.516+0000 7f5c2ddff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5c28106340 0x7f5c2819c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:39.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.516+0000 7f5c2ddff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c28106cd0 0x7f5c2819c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:39.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.516+0000 7f5c2ddff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c2810c100 0x7f5c281a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:39.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.516+0000 7f5c2ddff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c281a1160 con 0x7f5c28106340 2023-07-18T03:20:39.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.516+0000 7f5c2ddff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c281a12d0 con 0x7f5c28106cd0 2023-07-18T03:20:39.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.516+0000 7f5c2ddff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c281a1490 con 0x7f5c2810c100 2023-07-18T03:20:39.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.517+0000 7f5c277fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5c28106340 0x7f5c2819c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:39.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.517+0000 7f5c27fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c2810c100 0x7f5c281a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:39.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.517+0000 7f5c277fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5c28106340 0x7f5c2819c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:57016/0 (socket says 172.21.15.101:57016) 2023-07-18T03:20:39.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.517+0000 7f5c277fe700 1 -- 172.21.15.101:0/3999525353 learned_addr learned my addr 172.21.15.101:0/3999525353 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:39.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.517+0000 7f5c26ffd700 1 --2- 172.21.15.101:0/3999525353 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c28106cd0 0x7f5c2819c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:39.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.517+0000 7f5c277fe700 1 -- 172.21.15.101:0/3999525353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c2810c100 msgr2=0x7f5c281a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:39.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.517+0000 7f5c277fe700 1 --2- 172.21.15.101:0/3999525353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c2810c100 0x7f5c281a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:39.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.517+0000 7f5c277fe700 1 -- 172.21.15.101:0/3999525353 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c28106cd0 msgr2=0x7f5c2819c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:39.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.517+0000 7f5c277fe700 1 --2- 172.21.15.101:0/3999525353 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c28106cd0 0x7f5c2819c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:39.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.517+0000 7f5c277fe700 1 -- 172.21.15.101:0/3999525353 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c180095f0 con 0x7f5c28106340 2023-07-18T03:20:39.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.517+0000 7f5c277fe700 1 --2- 172.21.15.101:0/3999525353 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5c28106340 0x7f5c2819c180 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f5c1800b580 tx=0x7f5c1800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:39.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.518+0000 7f5c24ff9700 1 -- 172.21.15.101:0/3999525353 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1801d070 con 0x7f5c28106340 2023-07-18T03:20:39.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.518+0000 7f5c2ddff700 1 -- 172.21.15.101:0/3999525353 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c281a1710 con 0x7f5c28106340 2023-07-18T03:20:39.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.518+0000 7f5c24ff9700 1 -- 172.21.15.101:0/3999525353 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c18005050 con 0x7f5c28106340 2023-07-18T03:20:39.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.518+0000 7f5c24ff9700 1 -- 172.21.15.101:0/3999525353 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c18004a40 con 0x7f5c28106340 2023-07-18T03:20:39.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.518+0000 7f5c2ddff700 1 -- 172.21.15.101:0/3999525353 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c281a8fb0 con 0x7f5c28106340 2023-07-18T03:20:39.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.519+0000 7f5c2ddff700 1 -- 172.21.15.101:0/3999525353 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c28067300 con 0x7f5c28106340 2023-07-18T03:20:39.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.521+0000 7f5c24ff9700 1 -- 172.21.15.101:0/3999525353 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f5c180051c0 con 0x7f5c28106340 2023-07-18T03:20:39.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.522+0000 7f5c24ff9700 1 --2- 172.21.15.101:0/3999525353 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f5c100a1300 0x7f5c100a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:39.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.523+0000 7f5c24ff9700 1 -- 172.21.15.101:0/3999525353 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f5c180cdf30 con 0x7f5c28106340 2023-07-18T03:20:39.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.523+0000 7f5c26ffd700 1 --2- 172.21.15.101:0/3999525353 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f5c100a1300 0x7f5c100a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:39.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.523+0000 7f5c26ffd700 1 --2- 172.21.15.101:0/3999525353 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f5c100a1300 0x7f5c100a37b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5c2819d790 tx=0x7f5c1c009260 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:39.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.528+0000 7f5c24ff9700 1 -- 172.21.15.101:0/3999525353 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5c18096fc0 con 0x7f5c28106340 2023-07-18T03:20:39.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:39 smithi155 ceph-mon[128885]: pgmap v409: 129 pgs: 129 active+clean; 643 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 1.2 MiB/s wr, 88 op/s 2023-07-18T03:20:39.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.882+0000 7f5c2ddff700 1 -- 172.21.15.101:0/3999525353 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c281a9140 con 0x7f5c28106340 2023-07-18T03:20:39.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.883+0000 7f5c24ff9700 1 -- 172.21.15.101:0/3999525353 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+10859 (secure 0 0 0) 0x7f5c1809a5e0 con 0x7f5c28106340 2023-07-18T03:20:39.879 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:20:39.879 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/4192846769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":4192846769},{"type":"v1","addr":"172.21.15.101:6839","nonce":4192846769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6841/3824340175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6840","nonce":3824340175},{"type":"v1","addr":"172.21.15.101:6841","nonce":3824340175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:04:59.361185+0000","modified":"2023-07-18T03:20:11.118241+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.155:6839/2055462483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2055462483},{"type":"v1","addr":"172.21.15.155:6839","nonce":2055462483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:20:39.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.885+0000 7f5c2ddff700 1 -- 172.21.15.101:0/3999525353 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f5c100a1300 msgr2=0x7f5c100a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:39.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.885+0000 7f5c2ddff700 1 --2- 172.21.15.101:0/3999525353 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f5c100a1300 0x7f5c100a37b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5c2819d790 tx=0x7f5c1c009260 comp rx=0 tx=0).stop 2023-07-18T03:20:39.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.885+0000 7f5c2ddff700 1 -- 172.21.15.101:0/3999525353 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5c28106340 msgr2=0x7f5c2819c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:39.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.885+0000 7f5c2ddff700 1 --2- 172.21.15.101:0/3999525353 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5c28106340 0x7f5c2819c180 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f5c1800b580 tx=0x7f5c1800bfd0 comp rx=0 tx=0).stop 2023-07-18T03:20:39.882 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.885+0000 7f5c2ddff700 1 -- 172.21.15.101:0/3999525353 shutdown_connections 2023-07-18T03:20:39.882 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.885+0000 7f5c2ddff700 1 --2- 172.21.15.101:0/3999525353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c2810c100 0x7f5c281a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:39.882 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.885+0000 7f5c2ddff700 1 --2- 172.21.15.101:0/3999525353 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5c28106340 0x7f5c2819c180 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:39.882 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.885+0000 7f5c2ddff700 1 --2- 172.21.15.101:0/3999525353 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f5c100a1300 0x7f5c100a37b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:39.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.885+0000 7f5c2ddff700 1 --2- 172.21.15.101:0/3999525353 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c28106cd0 0x7f5c2819c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:39.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.885+0000 7f5c2ddff700 1 -- 172.21.15.101:0/3999525353 >> 172.21.15.101:0/3999525353 conn(0x7f5c280759f0 msgr2=0x7f5c28077060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:39.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.886+0000 7f5c2ddff700 1 -- 172.21.15.101:0/3999525353 shutdown_connections 2023-07-18T03:20:39.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:39.886+0000 7f5c2ddff700 1 -- 172.21.15.101:0/3999525353 wait complete. 2023-07-18T03:20:39.883 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 34 2023-07-18T03:20:39.904 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-18T03:20:39.950 INFO:tasks.workunit.client.0.smithi101.stdout: 10 2046 64.59 MB/sec warmup 4 sec latency 2912.502 ms 2023-07-18T03:20:40.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.141+0000 7f8831fff700 1 -- 172.21.15.101:0/365715424 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f882c1004b0 msgr2=0x7f882c10c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:40.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.141+0000 7f8831fff700 1 --2- 172.21.15.101:0/365715424 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f882c1004b0 0x7f882c10c940 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f882400b0a0 tx=0x7f882400b3b0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.141+0000 7f8831fff700 1 -- 172.21.15.101:0/365715424 shutdown_connections 2023-07-18T03:20:40.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.141+0000 7f8831fff700 1 --2- 172.21.15.101:0/365715424 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f882c1004b0 0x7f882c10c940 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.141+0000 7f8831fff700 1 --2- 172.21.15.101:0/365715424 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f882c105b10 0x7f882c105ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.141+0000 7f8831fff700 1 --2- 172.21.15.101:0/365715424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f882c0ffb00 0x7f882c0fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.141+0000 7f8831fff700 1 -- 172.21.15.101:0/365715424 >> 172.21.15.101:0/365715424 conn(0x7f882c0fb600 msgr2=0x7f882c0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:40.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.141+0000 7f8831fff700 1 -- 172.21.15.101:0/365715424 shutdown_connections 2023-07-18T03:20:40.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.141+0000 7f8831fff700 1 -- 172.21.15.101:0/365715424 wait complete. 2023-07-18T03:20:40.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.142+0000 7f8831fff700 1 Processor -- start 2023-07-18T03:20:40.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.142+0000 7f8831fff700 1 -- start start 2023-07-18T03:20:40.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.143+0000 7f8831fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f882c0ffb00 0x7f882c197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.143+0000 7f8831fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f882c1004b0 0x7f882c198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.143+0000 7f8831fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f882c105b10 0x7f882c19c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.143+0000 7f8831fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882c19cea0 con 0x7f882c105b10 2023-07-18T03:20:40.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.143+0000 7f8831fff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882c19d010 con 0x7f882c0ffb00 2023-07-18T03:20:40.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.143+0000 7f8831fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882c19d1d0 con 0x7f882c1004b0 2023-07-18T03:20:40.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.143+0000 7f882bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f882c105b10 0x7f882c19c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.143+0000 7f882affd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f882c1004b0 0x7f882c198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.143+0000 7f882bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f882c105b10 0x7f882c19c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:57032/0 (socket says 172.21.15.101:57032) 2023-07-18T03:20:40.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.143+0000 7f882bfff700 1 -- 172.21.15.101:0/4209806683 learned_addr learned my addr 172.21.15.101:0/4209806683 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:40.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.143+0000 7f882affd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f882c1004b0 0x7f882c198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41590/0 (socket says 172.21.15.101:41590) 2023-07-18T03:20:40.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.143+0000 7f882b7fe700 1 --2- 172.21.15.101:0/4209806683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f882c0ffb00 0x7f882c197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.143+0000 7f882bfff700 1 -- 172.21.15.101:0/4209806683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f882c1004b0 msgr2=0x7f882c198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:40.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.143+0000 7f882bfff700 1 --2- 172.21.15.101:0/4209806683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f882c1004b0 0x7f882c198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.143+0000 7f882bfff700 1 -- 172.21.15.101:0/4209806683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f882c0ffb00 msgr2=0x7f882c197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:40.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.143+0000 7f882bfff700 1 --2- 172.21.15.101:0/4209806683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f882c0ffb00 0x7f882c197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.143+0000 7f882bfff700 1 -- 172.21.15.101:0/4209806683 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8824009d00 con 0x7f882c105b10 2023-07-18T03:20:40.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.143+0000 7f882affd700 1 --2- 172.21.15.101:0/4209806683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f882c1004b0 0x7f882c198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:20:40.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.144+0000 7f882bfff700 1 --2- 172.21.15.101:0/4209806683 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f882c105b10 0x7f882c19c860 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f882400b780 tx=0x7f8824007990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:40.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.144+0000 7f8828ff9700 1 -- 172.21.15.101:0/4209806683 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882400e040 con 0x7f882c105b10 2023-07-18T03:20:40.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.144+0000 7f8831fff700 1 -- 172.21.15.101:0/4209806683 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f882c19d450 con 0x7f882c105b10 2023-07-18T03:20:40.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.144+0000 7f8828ff9700 1 -- 172.21.15.101:0/4209806683 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f882401d470 con 0x7f882c105b10 2023-07-18T03:20:40.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.144+0000 7f8828ff9700 1 -- 172.21.15.101:0/4209806683 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88240052a0 con 0x7f882c105b10 2023-07-18T03:20:40.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.144+0000 7f8831fff700 1 -- 172.21.15.101:0/4209806683 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f882c1a4cf0 con 0x7f882c105b10 2023-07-18T03:20:40.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.148+0000 7f8828ff9700 1 -- 172.21.15.101:0/4209806683 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f882401d5e0 con 0x7f882c105b10 2023-07-18T03:20:40.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.149+0000 7f8845966b80 1 Processor -- start 2023-07-18T03:20:40.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.149+0000 7f8845966b80 1 -- start start 2023-07-18T03:20:40.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.149+0000 7f8828ff9700 1 --2- 172.21.15.101:0/4209806683 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f88140a12c0 0x7f88140a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.149+0000 7f8828ff9700 1 -- 172.21.15.101:0/4209806683 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f88240ce650 con 0x7f882c105b10 2023-07-18T03:20:40.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.150+0000 7f882affd700 1 --2- 172.21.15.101:0/4209806683 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f88140a12c0 0x7f88140a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.150+0000 7f8845966b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dfcc9c82d0 0x55dfcc9c86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.150+0000 7f8845966b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dfcc9c8be0 0x55dfcc9d3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.150+0000 7f8845966b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dfcc9d3ad0 0x55dfcc9d5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.150+0000 7f8845966b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfcc8cecb0 con 0x55dfcc9d3ad0 2023-07-18T03:20:40.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.150+0000 7f8845966b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfcc921a40 con 0x55dfcc9c8be0 2023-07-18T03:20:40.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.150+0000 7f8845966b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfcc91aed0 con 0x55dfcc9c82d0 2023-07-18T03:20:40.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.150+0000 7f882b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dfcc9c82d0 0x55dfcc9c86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.150+0000 7f882b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dfcc9c82d0 0x55dfcc9c86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41596/0 (socket says 172.21.15.101:41596) 2023-07-18T03:20:40.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.150+0000 7f882b7fe700 1 -- 172.21.15.101:0/1225853408 learned_addr learned my addr 172.21.15.101:0/1225853408 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:40.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.150+0000 7f882affd700 1 --2- 172.21.15.101:0/1225853408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dfcc9c8be0 0x55dfcc9d3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.150+0000 7f882bfff700 1 --2- 172.21.15.101:0/1225853408 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dfcc9d3ad0 0x55dfcc9d5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.150+0000 7f882b7fe700 1 -- 172.21.15.101:0/1225853408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dfcc9c8be0 msgr2=0x55dfcc9d3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:40.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.150+0000 7f882b7fe700 1 --2- 172.21.15.101:0/1225853408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dfcc9c8be0 0x55dfcc9d3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.150+0000 7f882b7fe700 1 -- 172.21.15.101:0/1225853408 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dfcc9d3ad0 msgr2=0x55dfcc9d5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:40.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.150+0000 7f882b7fe700 1 --2- 172.21.15.101:0/1225853408 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dfcc9d3ad0 0x55dfcc9d5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.150+0000 7f882b7fe700 1 -- 172.21.15.101:0/1225853408 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dfcc748de0 con 0x55dfcc9c82d0 2023-07-18T03:20:40.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.151+0000 7f882affd700 1 --2- 172.21.15.101:0/4209806683 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f88140a12c0 0x7f88140a3770 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f882c1994d0 tx=0x7f8820011cd0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:40.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.151+0000 7f882b7fe700 1 --2- 172.21.15.101:0/1225853408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dfcc9c82d0 0x55dfcc9c86a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f881c009d00 tx=0x7f881c011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85ac3c04c681fb75 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:40.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.151+0000 7f8811ffb700 1 -- 172.21.15.101:0/1225853408 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881c019630 con 0x55dfcc9c82d0 2023-07-18T03:20:40.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.151+0000 7f8811ffb700 1 -- 172.21.15.101:0/1225853408 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f881c019c50 con 0x55dfcc9c82d0 2023-07-18T03:20:40.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.151+0000 7f8811ffb700 1 -- 172.21.15.101:0/1225853408 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881c00eb30 con 0x55dfcc9c82d0 2023-07-18T03:20:40.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.152+0000 7f8845966b80 1 -- 172.21.15.101:0/1225853408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dfcc9c82d0 msgr2=0x55dfcc9c86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:40.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.152+0000 7f8845966b80 1 --2- 172.21.15.101:0/1225853408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dfcc9c82d0 0x55dfcc9c86a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f881c009d00 tx=0x7f881c011720 comp rx=0 tx=0).stop 2023-07-18T03:20:40.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.152+0000 7f8845966b80 1 -- 172.21.15.101:0/1225853408 shutdown_connections 2023-07-18T03:20:40.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.152+0000 7f8845966b80 1 --2- 172.21.15.101:0/1225853408 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dfcc9d3ad0 0x55dfcc9d5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.152+0000 7f8845966b80 1 --2- 172.21.15.101:0/1225853408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dfcc9c82d0 0x55dfcc9c86a0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.152+0000 7f8845966b80 1 --2- 172.21.15.101:0/1225853408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dfcc9c8be0 0x55dfcc9d3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.152+0000 7f8845966b80 1 -- 172.21.15.101:0/1225853408 >> 172.21.15.101:0/1225853408 conn(0x55dfcc9beda0 msgr2=0x55dfcc9c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:40.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.152+0000 7f8845966b80 1 -- 172.21.15.101:0/1225853408 shutdown_connections 2023-07-18T03:20:40.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.152+0000 7f8845966b80 1 -- 172.21.15.101:0/1225853408 wait complete. 2023-07-18T03:20:40.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.153+0000 7f8845966b80 1 -- start start 2023-07-18T03:20:40.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.153+0000 7f8845966b80 1 Processor -- start 2023-07-18T03:20:40.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.154+0000 7f8845966b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dfcc9c82d0 0x55dfcc9d34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.154+0000 7f8845966b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dfcc9c8be0 0x55dfcc9d07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.154+0000 7f8845966b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dfcc9d3ad0 0x55dfcc9d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.154+0000 7f8845966b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfcc8baac0 con 0x55dfcc9c8be0 2023-07-18T03:20:40.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.154+0000 7f8845966b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfcc8ba7c0 con 0x55dfcc9d3ad0 2023-07-18T03:20:40.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.154+0000 7f8845966b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfcc79e920 con 0x55dfcc9c82d0 2023-07-18T03:20:40.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.154+0000 7f8845966b80 10 client.0 _ll_register_callbacks cb 0x55dfcc8b8760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:20:40.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.154+0000 7f882b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dfcc9c82d0 0x55dfcc9d34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.154+0000 7f882b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dfcc9c82d0 0x55dfcc9d34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41608/0 (socket says 172.21.15.101:41608) 2023-07-18T03:20:40.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.154+0000 7f882b7fe700 1 -- 172.21.15.101:0/3350319746 learned_addr learned my addr 172.21.15.101:0/3350319746 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:40.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.154+0000 7f882affd700 1 --2- 172.21.15.101:0/3350319746 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dfcc9c8be0 0x55dfcc9d07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.154+0000 7f882bfff700 1 --2- 172.21.15.101:0/3350319746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dfcc9d3ad0 0x55dfcc9d0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.155+0000 7f882b7fe700 1 -- 172.21.15.101:0/3350319746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dfcc9d3ad0 msgr2=0x55dfcc9d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:40.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.155+0000 7f882b7fe700 1 --2- 172.21.15.101:0/3350319746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dfcc9d3ad0 0x55dfcc9d0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.155+0000 7f882b7fe700 1 -- 172.21.15.101:0/3350319746 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dfcc9c8be0 msgr2=0x55dfcc9d07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:40.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.155+0000 7f882b7fe700 1 --2- 172.21.15.101:0/3350319746 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dfcc9c8be0 0x55dfcc9d07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.155+0000 7f882b7fe700 1 -- 172.21.15.101:0/3350319746 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f881c009960 con 0x55dfcc9c82d0 2023-07-18T03:20:40.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.155+0000 7f882bfff700 1 --2- 172.21.15.101:0/3350319746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dfcc9d3ad0 0x55dfcc9d0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:20:40.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.155+0000 7f882affd700 1 --2- 172.21.15.101:0/3350319746 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dfcc9c8be0 0x55dfcc9d07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:20:40.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.155+0000 7f882b7fe700 1 --2- 172.21.15.101:0/3350319746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dfcc9c82d0 0x55dfcc9d34e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f881c000c00 tx=0x7f881c00e3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf228be96d97bf3e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:40.156 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.155+0000 7f880affd700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:20:40.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.155+0000 7f880affd700 1 -- 172.21.15.101:0/3350319746 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881c005440 con 0x55dfcc9c82d0 2023-07-18T03:20:40.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.155+0000 7f880affd700 1 -- 172.21.15.101:0/3350319746 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f881c0055a0 con 0x55dfcc9c82d0 2023-07-18T03:20:40.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.155+0000 7f8845966b80 1 -- 172.21.15.101:0/3350319746 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dfcc8744f0 con 0x55dfcc9c82d0 2023-07-18T03:20:40.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.156+0000 7f880affd700 1 -- 172.21.15.101:0/3350319746 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f881c019430 con 0x55dfcc9c82d0 2023-07-18T03:20:40.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.156+0000 7f8845966b80 10 client.28237 fetch_fsmap learned FSMap version 34 2023-07-18T03:20:40.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.156+0000 7f8845966b80 1 -- 172.21.15.101:0/3350319746 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55dfcc731c60 con 0x55dfcc9c82d0 2023-07-18T03:20:40.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.156+0000 7f880affd700 1 -- 172.21.15.101:0/3350319746 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dfcc8744f0 con 0x55dfcc9c82d0 2023-07-18T03:20:40.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.156+0000 7f880affd700 1 -- 172.21.15.101:0/3350319746 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 34) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f881c00ecf0 con 0x55dfcc9c82d0 2023-07-18T03:20:40.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.156+0000 7f8845966b80 10 client.28237 fetch_fsmap finished waiting for FSMap version 34 2023-07-18T03:20:40.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.156+0000 7f8845966b80 10 client.28237 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:20:40.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.156+0000 7f8845966b80 20 client.28237 populate_metadata read hostname 'smithi101' 2023-07-18T03:20:40.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.156+0000 7f8845966b80 1 --2- 172.21.15.101:0/3350319746 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55dfcc9e33c0 0x55dfcc9e57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.156+0000 7f8845966b80 4 client.28237 mds_command: new command op to 24541 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:20:40.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.156+0000 7f8845966b80 1 -- 172.21.15.101:0/3350319746 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dfcc913eb0 con 0x55dfcc9e33c0 2023-07-18T03:20:40.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.157+0000 7f882affd700 1 --2- 172.21.15.101:0/3350319746 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55dfcc9e33c0 0x55dfcc9e57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.157+0000 7f882affd700 1 --2- 172.21.15.101:0/3350319746 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55dfcc9e33c0 0x55dfcc9e57d0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=53ff7fba06387686 server_cookie=e5cd0310d5d8121f in_seq=0 out_seq=0 2023-07-18T03:20:40.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.157+0000 7f880affd700 10 client.28237 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:20:40.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.158+0000 7f880affd700 1 -- 172.21.15.101:0/3350319746 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55dfcc913eb0 con 0x55dfcc9e33c0 2023-07-18T03:20:40.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.158+0000 7f880affd700 10 client.28237 handle_command_reply: tid=0 2023-07-18T03:20:40.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.158+0000 7f8845966b80 1 client.28237 shutdown 2023-07-18T03:20:40.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.159+0000 7f8845966b80 1 -- 172.21.15.101:0/3350319746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dfcc9c82d0 msgr2=0x55dfcc9d34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:40.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.159+0000 7f8845966b80 1 --2- 172.21.15.101:0/3350319746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dfcc9c82d0 0x55dfcc9d34e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f881c000c00 tx=0x7f881c00e3b0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.159+0000 7f8845966b80 1 -- 172.21.15.101:0/3350319746 shutdown_connections 2023-07-18T03:20:40.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.159+0000 7f8845966b80 1 --2- 172.21.15.101:0/3350319746 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55dfcc9e33c0 0x55dfcc9e57d0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.159+0000 7f8845966b80 1 --2- 172.21.15.101:0/3350319746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dfcc9d3ad0 0x55dfcc9d0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.159+0000 7f880affd700 0 client.28237 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:20:40.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.159+0000 7f8845966b80 1 --2- 172.21.15.101:0/3350319746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dfcc9c82d0 0x55dfcc9d34e0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.159+0000 7f8845966b80 1 --2- 172.21.15.101:0/3350319746 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dfcc9c8be0 0x55dfcc9d07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.159+0000 7f8845966b80 1 -- 172.21.15.101:0/3350319746 >> 172.21.15.101:0/3350319746 conn(0x55dfcc9beda0 msgr2=0x55dfcc9c4230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:40.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.159+0000 7f8845966b80 1 -- 172.21.15.101:0/3350319746 shutdown_connections 2023-07-18T03:20:40.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.159+0000 7f8845966b80 1 -- 172.21.15.101:0/3350319746 wait complete. 2023-07-18T03:20:40.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.159+0000 7f8845966b80 20 client.28237 trim_cache size 0 max 16384 2023-07-18T03:20:40.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.176+0000 7f8845966b80 1 Processor -- start 2023-07-18T03:20:40.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.177+0000 7f8845966b80 1 -- start start 2023-07-18T03:20:40.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.177+0000 7f8845966b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dfcc9c82d0 0x55dfcc9cb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.177+0000 7f8845966b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dfcc9c8be0 0x55dfcc9da4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.177+0000 7f8845966b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dfcc9d3ad0 0x55dfcc9cbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.177+0000 7f8845966b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfcc9c9d60 con 0x55dfcc9c8be0 2023-07-18T03:20:40.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.177+0000 7f8845966b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfcc9bbf80 con 0x55dfcc9c82d0 2023-07-18T03:20:40.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.177+0000 7f8845966b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfcc9be970 con 0x55dfcc9d3ad0 2023-07-18T03:20:40.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.177+0000 7f882b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dfcc9c82d0 0x55dfcc9cb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.177+0000 7f882b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dfcc9c82d0 0x55dfcc9cb2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60818/0 (socket says 172.21.15.101:60818) 2023-07-18T03:20:40.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.177+0000 7f882b7fe700 1 -- 172.21.15.101:0/1190215706 learned_addr learned my addr 172.21.15.101:0/1190215706 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:40.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.177+0000 7f882bfff700 1 --2- 172.21.15.101:0/1190215706 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dfcc9d3ad0 0x55dfcc9cbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.177+0000 7f882affd700 1 --2- 172.21.15.101:0/1190215706 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dfcc9c8be0 0x55dfcc9da4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.178+0000 7f882b7fe700 1 -- 172.21.15.101:0/1190215706 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dfcc9d3ad0 msgr2=0x55dfcc9cbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:40.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.178+0000 7f882b7fe700 1 --2- 172.21.15.101:0/1190215706 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dfcc9d3ad0 0x55dfcc9cbb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.178+0000 7f882b7fe700 1 -- 172.21.15.101:0/1190215706 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dfcc9c8be0 msgr2=0x55dfcc9da4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:40.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.178+0000 7f882b7fe700 1 --2- 172.21.15.101:0/1190215706 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dfcc9c8be0 0x55dfcc9da4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.178+0000 7f882b7fe700 1 -- 172.21.15.101:0/1190215706 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dfcc9c05e0 con 0x55dfcc9c82d0 2023-07-18T03:20:40.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.178+0000 7f882b7fe700 1 --2- 172.21.15.101:0/1190215706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dfcc9c82d0 0x55dfcc9cb2e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f881c009f50 tx=0x7f881c001e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29905b76e2890355 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:40.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.178+0000 7f88127fc700 1 -- 172.21.15.101:0/1190215706 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881c004a40 con 0x55dfcc9c82d0 2023-07-18T03:20:40.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.178+0000 7f88127fc700 1 -- 172.21.15.101:0/1190215706 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f881c004c80 con 0x55dfcc9c82d0 2023-07-18T03:20:40.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.178+0000 7f88127fc700 1 -- 172.21.15.101:0/1190215706 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881c0197b0 con 0x55dfcc9c82d0 2023-07-18T03:20:40.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.179+0000 7f8845966b80 1 -- 172.21.15.101:0/1190215706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dfcc9c82d0 msgr2=0x55dfcc9cb2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:40.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.179+0000 7f8845966b80 1 --2- 172.21.15.101:0/1190215706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dfcc9c82d0 0x55dfcc9cb2e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f881c009f50 tx=0x7f881c001e50 comp rx=0 tx=0).stop 2023-07-18T03:20:40.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.179+0000 7f8845966b80 1 -- 172.21.15.101:0/1190215706 shutdown_connections 2023-07-18T03:20:40.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.179+0000 7f8845966b80 1 --2- 172.21.15.101:0/1190215706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dfcc9c82d0 0x55dfcc9cb2e0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.179+0000 7f8845966b80 1 --2- 172.21.15.101:0/1190215706 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dfcc9d3ad0 0x55dfcc9cbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.179+0000 7f8845966b80 1 --2- 172.21.15.101:0/1190215706 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dfcc9c8be0 0x55dfcc9da4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.179+0000 7f8845966b80 1 -- 172.21.15.101:0/1190215706 >> 172.21.15.101:0/1190215706 conn(0x55dfcc9beda0 msgr2=0x55dfcc9dec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:40.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.179+0000 7f8845966b80 1 -- 172.21.15.101:0/1190215706 shutdown_connections 2023-07-18T03:20:40.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.179+0000 7f8845966b80 1 -- 172.21.15.101:0/1190215706 wait complete. 2023-07-18T03:20:40.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.179+0000 7f8845966b80 1 -- start start 2023-07-18T03:20:40.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.180+0000 7f8845966b80 1 Processor -- start 2023-07-18T03:20:40.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.180+0000 7f8845966b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dfcc9c82d0 0x55dfcc9da650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.180+0000 7f8845966b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dfcc9c8be0 0x55dfcc9d7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.180+0000 7f8845966b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dfcc9d3ad0 0x55dfcc9d7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.180+0000 7f8845966b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfcc9d8710 con 0x55dfcc9c8be0 2023-07-18T03:20:40.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.180+0000 7f8845966b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfcca36240 con 0x55dfcc9d3ad0 2023-07-18T03:20:40.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.180+0000 7f8845966b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfcca36380 con 0x55dfcc9c82d0 2023-07-18T03:20:40.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.180+0000 7f8845966b80 10 client.0 _ll_register_callbacks cb 0x7f882401db90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:20:40.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.181+0000 7f882b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dfcc9c82d0 0x55dfcc9da650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.181+0000 7f882b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dfcc9c82d0 0x55dfcc9da650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41614/0 (socket says 172.21.15.101:41614) 2023-07-18T03:20:40.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.181+0000 7f882b7fe700 1 -- 172.21.15.101:0/982829656 learned_addr learned my addr 172.21.15.101:0/982829656 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:40.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.181+0000 7f882affd700 1 --2- 172.21.15.101:0/982829656 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dfcc9c8be0 0x55dfcc9d7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.181+0000 7f882bfff700 1 --2- 172.21.15.101:0/982829656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dfcc9d3ad0 0x55dfcc9d7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.181+0000 7f882b7fe700 1 -- 172.21.15.101:0/982829656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dfcc9d3ad0 msgr2=0x55dfcc9d7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:40.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.181+0000 7f882b7fe700 1 --2- 172.21.15.101:0/982829656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dfcc9d3ad0 0x55dfcc9d7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.181+0000 7f882b7fe700 1 -- 172.21.15.101:0/982829656 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dfcc9c8be0 msgr2=0x55dfcc9d7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:40.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.181+0000 7f882b7fe700 1 --2- 172.21.15.101:0/982829656 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dfcc9c8be0 0x55dfcc9d7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.181+0000 7f882b7fe700 1 -- 172.21.15.101:0/982829656 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dfcc731c60 con 0x55dfcc9c82d0 2023-07-18T03:20:40.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.181+0000 7f882b7fe700 1 --2- 172.21.15.101:0/982829656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dfcc9c82d0 0x55dfcc9da650 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f881c011a70 tx=0x7f881c005e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=349573902d56d0dc server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:40.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.181+0000 7f880affd700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:20:40.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.182+0000 7f880affd700 1 -- 172.21.15.101:0/982829656 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881c019630 con 0x55dfcc9c82d0 2023-07-18T03:20:40.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.182+0000 7f8845966b80 1 -- 172.21.15.101:0/982829656 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f881c019c50 con 0x55dfcc9c82d0 2023-07-18T03:20:40.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.182+0000 7f880affd700 1 -- 172.21.15.101:0/982829656 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f881c019790 con 0x55dfcc9c82d0 2023-07-18T03:20:40.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.182+0000 7f880affd700 1 -- 172.21.15.101:0/982829656 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f881c019970 con 0x55dfcc9c82d0 2023-07-18T03:20:40.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.182+0000 7f8845966b80 10 client.28243 fetch_fsmap learned FSMap version 35 2023-07-18T03:20:40.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.182+0000 7f8845966b80 1 -- 172.21.15.101:0/982829656 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55dfcc76f250 con 0x55dfcc9c82d0 2023-07-18T03:20:40.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.182+0000 7f880affd700 1 -- 172.21.15.101:0/982829656 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f881c019c50 con 0x55dfcc9c82d0 2023-07-18T03:20:40.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.182+0000 7f880affd700 1 -- 172.21.15.101:0/982829656 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 35) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f881c022410 con 0x55dfcc9c82d0 2023-07-18T03:20:40.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.182+0000 7f8845966b80 10 client.28243 fetch_fsmap finished waiting for FSMap version 35 2023-07-18T03:20:40.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.182+0000 7f8845966b80 10 client.28243 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:20:40.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.182+0000 7f8845966b80 20 client.28243 populate_metadata read hostname 'smithi101' 2023-07-18T03:20:40.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.182+0000 7f8845966b80 1 --2- 172.21.15.101:0/982829656 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55dfcca374f0 0x55dfcca398d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.182+0000 7f8845966b80 4 client.28243 mds_command: new command op to 24541 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:20:40.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.182+0000 7f8845966b80 1 -- 172.21.15.101:0/982829656 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dfcc8d6050 con 0x55dfcca374f0 2023-07-18T03:20:40.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.183+0000 7f882affd700 1 --2- 172.21.15.101:0/982829656 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55dfcca374f0 0x55dfcca398d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.183+0000 7f882affd700 1 --2- 172.21.15.101:0/982829656 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55dfcca374f0 0x55dfcca398d0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=932e4658b962433e server_cookie=ab37e95e1a974bfd in_seq=0 out_seq=0 2023-07-18T03:20:40.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.183+0000 7f880affd700 10 client.28243 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:20:40.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.183+0000 7f880affd700 1 -- 172.21.15.101:0/982829656 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dfcc8d6050 con 0x55dfcca374f0 2023-07-18T03:20:40.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.183+0000 7f880affd700 10 client.28243 handle_command_reply: tid=0 2023-07-18T03:20:40.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.183+0000 7f8845966b80 1 client.28243 shutdown 2023-07-18T03:20:40.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.183+0000 7f8845966b80 1 -- 172.21.15.101:0/982829656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dfcc9c82d0 msgr2=0x55dfcc9da650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:40.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.183+0000 7f8845966b80 1 --2- 172.21.15.101:0/982829656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dfcc9c82d0 0x55dfcc9da650 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f881c011a70 tx=0x7f881c005e10 comp rx=0 tx=0).stop 2023-07-18T03:20:40.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.183+0000 7f8845966b80 1 -- 172.21.15.101:0/982829656 shutdown_connections 2023-07-18T03:20:40.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.183+0000 7f8845966b80 1 --2- 172.21.15.101:0/982829656 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55dfcca374f0 0x55dfcca398d0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.184+0000 7f8845966b80 1 --2- 172.21.15.101:0/982829656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dfcc9d3ad0 0x55dfcc9d7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.184+0000 7f880affd700 0 client.28243 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:20:40.192 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.184+0000 7f8845966b80 1 --2- 172.21.15.101:0/982829656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dfcc9c82d0 0x55dfcc9da650 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.192 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.184+0000 7f8845966b80 1 --2- 172.21.15.101:0/982829656 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dfcc9c8be0 0x55dfcc9d7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.192 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.184+0000 7f8845966b80 1 -- 172.21.15.101:0/982829656 >> 172.21.15.101:0/982829656 conn(0x55dfcc9beda0 msgr2=0x55dfcc9cbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:40.192 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.184+0000 7f8845966b80 1 -- 172.21.15.101:0/982829656 shutdown_connections 2023-07-18T03:20:40.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.184+0000 7f8845966b80 1 -- 172.21.15.101:0/982829656 wait complete. 2023-07-18T03:20:40.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.184+0000 7f8845966b80 20 client.28243 trim_cache size 0 max 16384 2023-07-18T03:20:40.193 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:20:40.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.184+0000 7f8831fff700 1 -- 172.21.15.101:0/4209806683 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f88140a12c0 msgr2=0x7f88140a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:40.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.184+0000 7f8831fff700 1 --2- 172.21.15.101:0/4209806683 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f88140a12c0 0x7f88140a3770 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f882c1994d0 tx=0x7f8820011cd0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.184+0000 7f8831fff700 1 -- 172.21.15.101:0/4209806683 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f882c105b10 msgr2=0x7f882c19c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:40.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.184+0000 7f8831fff700 1 --2- 172.21.15.101:0/4209806683 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f882c105b10 0x7f882c19c860 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f882400b780 tx=0x7f8824007990 comp rx=0 tx=0).stop 2023-07-18T03:20:40.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.185+0000 7f8831fff700 1 -- 172.21.15.101:0/4209806683 shutdown_connections 2023-07-18T03:20:40.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.185+0000 7f8831fff700 1 --2- 172.21.15.101:0/4209806683 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f88140a12c0 0x7f88140a3770 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.185+0000 7f8831fff700 1 --2- 172.21.15.101:0/4209806683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f882c0ffb00 0x7f882c197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.185+0000 7f8831fff700 1 --2- 172.21.15.101:0/4209806683 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f882c105b10 0x7f882c19c860 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.185+0000 7f8831fff700 1 --2- 172.21.15.101:0/4209806683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f882c1004b0 0x7f882c198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.185+0000 7f8831fff700 1 -- 172.21.15.101:0/4209806683 >> 172.21.15.101:0/4209806683 conn(0x7f882c0fb600 msgr2=0x7f882c10afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:40.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.185+0000 7f8831fff700 1 -- 172.21.15.101:0/4209806683 shutdown_connections 2023-07-18T03:20:40.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.185+0000 7f8831fff700 1 -- 172.21.15.101:0/4209806683 wait complete. 2023-07-18T03:20:40.200 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-18T03:20:40.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:40 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/3999525353' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:20:40.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.468+0000 7f5cab3ff700 1 -- 172.21.15.101:0/3679415612 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5ca40fe9d0 msgr2=0x7f5ca40fee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:40.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.468+0000 7f5cab3ff700 1 --2- 172.21.15.101:0/3679415612 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5ca40fe9d0 0x7f5ca40fee40 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f5c940098f0 tx=0x7f5c94009c00 comp rx=0 tx=0).stop 2023-07-18T03:20:40.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.468+0000 7f5cab3ff700 1 -- 172.21.15.101:0/3679415612 shutdown_connections 2023-07-18T03:20:40.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.468+0000 7f5cab3ff700 1 --2- 172.21.15.101:0/3679415612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ca41049e0 0x7f5ca4104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.468+0000 7f5cab3ff700 1 --2- 172.21.15.101:0/3679415612 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ca40ff380 0x7f5ca410b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.468+0000 7f5cab3ff700 1 --2- 172.21.15.101:0/3679415612 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5ca40fe9d0 0x7f5ca40fee40 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.468+0000 7f5cab3ff700 1 -- 172.21.15.101:0/3679415612 >> 172.21.15.101:0/3679415612 conn(0x7f5ca40fa4b0 msgr2=0x7f5ca40fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:40.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.469+0000 7f5cab3ff700 1 -- 172.21.15.101:0/3679415612 shutdown_connections 2023-07-18T03:20:40.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.469+0000 7f5cab3ff700 1 -- 172.21.15.101:0/3679415612 wait complete. 2023-07-18T03:20:40.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.469+0000 7f5cab3ff700 1 Processor -- start 2023-07-18T03:20:40.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.470+0000 7f5cab3ff700 1 -- start start 2023-07-18T03:20:40.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.470+0000 7f5cab3ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5ca40fe9d0 0x7f5ca4197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.470+0000 7f5cab3ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ca40ff380 0x7f5ca4198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.470+0000 7f5cab3ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ca41049e0 0x7f5ca419c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.470+0000 7f5cab3ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca419ce00 con 0x7f5ca40fe9d0 2023-07-18T03:20:40.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.470+0000 7f5cab3ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca419cf70 con 0x7f5ca40ff380 2023-07-18T03:20:40.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.470+0000 7f5cab3ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca419d130 con 0x7f5ca41049e0 2023-07-18T03:20:40.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.470+0000 7f5ca919b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5ca40fe9d0 0x7f5ca4197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.470+0000 7f5ca899a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ca40ff380 0x7f5ca4198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.470+0000 7f5ca919b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5ca40fe9d0 0x7f5ca4197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:57076/0 (socket says 172.21.15.101:57076) 2023-07-18T03:20:40.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.470+0000 7f5ca919b700 1 -- 172.21.15.101:0/3710592754 learned_addr learned my addr 172.21.15.101:0/3710592754 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:40.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.470+0000 7f5ca999c700 1 --2- 172.21.15.101:0/3710592754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ca41049e0 0x7f5ca419c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.471+0000 7f5ca899a700 1 -- 172.21.15.101:0/3710592754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ca41049e0 msgr2=0x7f5ca419c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:40.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.471+0000 7f5ca899a700 1 --2- 172.21.15.101:0/3710592754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ca41049e0 0x7f5ca419c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.471+0000 7f5ca899a700 1 -- 172.21.15.101:0/3710592754 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5ca40fe9d0 msgr2=0x7f5ca4197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:40.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.471+0000 7f5ca899a700 1 --2- 172.21.15.101:0/3710592754 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5ca40fe9d0 0x7f5ca4197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.471+0000 7f5ca899a700 1 -- 172.21.15.101:0/3710592754 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca00095f0 con 0x7f5ca40ff380 2023-07-18T03:20:40.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.471+0000 7f5ca899a700 1 --2- 172.21.15.101:0/3710592754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ca40ff380 0x7f5ca4198360 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5c94009810 tx=0x7f5c9400fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:40.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.471+0000 7f5c9a7fc700 1 -- 172.21.15.101:0/3710592754 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9401c070 con 0x7f5ca40ff380 2023-07-18T03:20:40.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.471+0000 7f5cab3ff700 1 -- 172.21.15.101:0/3710592754 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c94009520 con 0x7f5ca40ff380 2023-07-18T03:20:40.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.471+0000 7f5c9a7fc700 1 -- 172.21.15.101:0/3710592754 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c9400bba0 con 0x7f5ca40ff380 2023-07-18T03:20:40.471 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.471+0000 7f5c9a7fc700 1 -- 172.21.15.101:0/3710592754 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c94003970 con 0x7f5ca40ff380 2023-07-18T03:20:40.471 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.471+0000 7f5cab3ff700 1 -- 172.21.15.101:0/3710592754 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ca41a4a70 con 0x7f5ca40ff380 2023-07-18T03:20:40.472 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.476+0000 7f5c9a7fc700 1 -- 172.21.15.101:0/3710592754 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f5c94020030 con 0x7f5ca40ff380 2023-07-18T03:20:40.472 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.477+0000 7f5cbed03b80 1 Processor -- start 2023-07-18T03:20:40.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.477+0000 7f5cbed03b80 1 -- start start 2023-07-18T03:20:40.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.478+0000 7f5cbed03b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56317f5ce2d0 0x56317f5ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.478+0000 7f5cbed03b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56317f5cebe0 0x56317f5d9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.478+0000 7f5cbed03b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56317f5d9ad0 0x56317f5dbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.478+0000 7f5cbed03b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317f4d4cb0 con 0x56317f5d9ad0 2023-07-18T03:20:40.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.478+0000 7f5cbed03b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317f527a40 con 0x56317f5ce2d0 2023-07-18T03:20:40.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.478+0000 7f5cbed03b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317f520ed0 con 0x56317f5cebe0 2023-07-18T03:20:40.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.478+0000 7f5ca899a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56317f5d9ad0 0x56317f5dbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.479+0000 7f5ca899a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56317f5d9ad0 0x56317f5dbeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:57092/0 (socket says 172.21.15.101:57092) 2023-07-18T03:20:40.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.479+0000 7f5ca899a700 1 -- 172.21.15.101:0/846304731 learned_addr learned my addr 172.21.15.101:0/846304731 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:40.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.479+0000 7f5ca919b700 1 --2- 172.21.15.101:0/846304731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56317f5cebe0 0x56317f5d9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.479+0000 7f5ca999c700 1 --2- 172.21.15.101:0/846304731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56317f5ce2d0 0x56317f5ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.479+0000 7f5ca899a700 1 -- 172.21.15.101:0/846304731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56317f5cebe0 msgr2=0x56317f5d9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:40.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.479+0000 7f5ca899a700 1 --2- 172.21.15.101:0/846304731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56317f5cebe0 0x56317f5d9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.479+0000 7f5ca899a700 1 -- 172.21.15.101:0/846304731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56317f5ce2d0 msgr2=0x56317f5ce6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:40.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.479+0000 7f5ca899a700 1 --2- 172.21.15.101:0/846304731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56317f5ce2d0 0x56317f5ce6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.479+0000 7f5ca899a700 1 -- 172.21.15.101:0/846304731 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56317f34ede0 con 0x56317f5d9ad0 2023-07-18T03:20:40.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.480+0000 7f5ca899a700 1 --2- 172.21.15.101:0/846304731 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56317f5d9ad0 0x56317f5dbeb0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f5c940cb170 tx=0x7f5c940d1d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d8acf2df254b2 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:40.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.480+0000 7f5c9a7fc700 1 --2- 172.21.15.101:0/3710592754 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f5c900a1340 0x7f5c900a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.480+0000 7f5ca999c700 1 --2- 172.21.15.101:0/3710592754 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f5c900a1340 0x7f5c900a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.480+0000 7f5c9a7fc700 1 -- 172.21.15.101:0/3710592754 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f5c940cdef0 con 0x7f5ca40ff380 2023-07-18T03:20:40.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.480+0000 7f5c8f7fe700 1 -- 172.21.15.101:0/846304731 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c94096f80 con 0x56317f5d9ad0 2023-07-18T03:20:40.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.480+0000 7f5c8f7fe700 1 -- 172.21.15.101:0/846304731 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c9409aa10 con 0x56317f5d9ad0 2023-07-18T03:20:40.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.480+0000 7f5c8f7fe700 1 -- 172.21.15.101:0/846304731 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9409ac40 con 0x56317f5d9ad0 2023-07-18T03:20:40.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.481+0000 7f5cbed03b80 1 -- 172.21.15.101:0/846304731 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56317f5d9ad0 msgr2=0x56317f5dbeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:40.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.481+0000 7f5ca999c700 1 --2- 172.21.15.101:0/3710592754 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f5c900a1340 0x7f5c900a37f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c0127f0 tx=0x7f5c9c00b040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:40.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.481+0000 7f5cbed03b80 1 --2- 172.21.15.101:0/846304731 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56317f5d9ad0 0x56317f5dbeb0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f5c940cb170 tx=0x7f5c940d1d80 comp rx=0 tx=0).stop 2023-07-18T03:20:40.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.481+0000 7f5cbed03b80 1 -- 172.21.15.101:0/846304731 shutdown_connections 2023-07-18T03:20:40.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.481+0000 7f5cbed03b80 1 --2- 172.21.15.101:0/846304731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56317f5ce2d0 0x56317f5ce6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.480 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.481+0000 7f5cbed03b80 1 --2- 172.21.15.101:0/846304731 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56317f5d9ad0 0x56317f5dbeb0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.480 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.481+0000 7f5cbed03b80 1 --2- 172.21.15.101:0/846304731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56317f5cebe0 0x56317f5d9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.480 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.481+0000 7f5cbed03b80 1 -- 172.21.15.101:0/846304731 >> 172.21.15.101:0/846304731 conn(0x56317f5c4da0 msgr2=0x56317f5ca6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:40.480 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.482+0000 7f5cbed03b80 1 -- 172.21.15.101:0/846304731 shutdown_connections 2023-07-18T03:20:40.480 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.482+0000 7f5cbed03b80 1 -- 172.21.15.101:0/846304731 wait complete. 2023-07-18T03:20:40.480 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.482+0000 7f5cbed03b80 1 -- start start 2023-07-18T03:20:40.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.482+0000 7f5cbed03b80 1 Processor -- start 2023-07-18T03:20:40.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.483+0000 7f5cbed03b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56317f5ce2d0 0x56317f5d9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.483+0000 7f5cbed03b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56317f5cebe0 0x56317f5da3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.483+0000 7f5cbed03b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56317f5d9ad0 0x56317f5da920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.483+0000 7f5cbed03b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317f4c0ac0 con 0x56317f5d9ad0 2023-07-18T03:20:40.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.483+0000 7f5cbed03b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317f4c07c0 con 0x56317f5ce2d0 2023-07-18T03:20:40.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.483+0000 7f5cbed03b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317f3a4920 con 0x56317f5cebe0 2023-07-18T03:20:40.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.483+0000 7f5cbed03b80 10 client.0 _ll_register_callbacks cb 0x56317f4be760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:20:40.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.483+0000 7f5ca919b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56317f5ce2d0 0x56317f5d9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.483+0000 7f5ca919b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56317f5ce2d0 0x56317f5d9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60866/0 (socket says 172.21.15.101:60866) 2023-07-18T03:20:40.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.483+0000 7f5ca919b700 1 -- 172.21.15.101:0/877899892 learned_addr learned my addr 172.21.15.101:0/877899892 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:40.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.483+0000 7f5ca899a700 1 --2- 172.21.15.101:0/877899892 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56317f5cebe0 0x56317f5da3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.484+0000 7f5ca999c700 1 --2- 172.21.15.101:0/877899892 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56317f5d9ad0 0x56317f5da920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.484+0000 7f5ca919b700 1 -- 172.21.15.101:0/877899892 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56317f5cebe0 msgr2=0x56317f5da3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:40.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.484+0000 7f5ca919b700 1 --2- 172.21.15.101:0/877899892 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56317f5cebe0 0x56317f5da3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.484+0000 7f5ca919b700 1 -- 172.21.15.101:0/877899892 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56317f5d9ad0 msgr2=0x56317f5da920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:40.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.484+0000 7f5ca919b700 1 --2- 172.21.15.101:0/877899892 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56317f5d9ad0 0x56317f5da920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.484+0000 7f5ca919b700 1 -- 172.21.15.101:0/877899892 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ca41a4a70 con 0x56317f5ce2d0 2023-07-18T03:20:40.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.484+0000 7f5ca899a700 1 --2- 172.21.15.101:0/877899892 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56317f5cebe0 0x56317f5da3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:20:40.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.484+0000 7f5ca919b700 1 --2- 172.21.15.101:0/877899892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56317f5ce2d0 0x56317f5d9ea0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f5ca00129e0 tx=0x7f5ca0012cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63bfeddc0c340428 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:40.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.484+0000 7f5c8cff9700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:20:40.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.484+0000 7f5c8cff9700 1 -- 172.21.15.101:0/877899892 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca0017070 con 0x56317f5ce2d0 2023-07-18T03:20:40.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.484+0000 7f5cbed03b80 1 -- 172.21.15.101:0/877899892 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56317f47a4f0 con 0x56317f5ce2d0 2023-07-18T03:20:40.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.484+0000 7f5c8cff9700 1 -- 172.21.15.101:0/877899892 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f5ca000f7b0 con 0x56317f5ce2d0 2023-07-18T03:20:40.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.485+0000 7f5c8cff9700 1 -- 172.21.15.101:0/877899892 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ca00122e0 con 0x56317f5ce2d0 2023-07-18T03:20:40.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.485+0000 7f5cbed03b80 10 client.28217 fetch_fsmap learned FSMap version 35 2023-07-18T03:20:40.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.485+0000 7f5cbed03b80 1 -- 172.21.15.101:0/877899892 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x56317f337c60 con 0x56317f5ce2d0 2023-07-18T03:20:40.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.485+0000 7f5c8cff9700 1 -- 172.21.15.101:0/877899892 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56317f47a4f0 con 0x56317f5ce2d0 2023-07-18T03:20:40.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.485+0000 7f5c8cff9700 1 -- 172.21.15.101:0/877899892 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 35) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f5ca0010be0 con 0x56317f5ce2d0 2023-07-18T03:20:40.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.485+0000 7f5cbed03b80 10 client.28217 fetch_fsmap finished waiting for FSMap version 35 2023-07-18T03:20:40.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.485+0000 7f5cbed03b80 10 client.28217 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:20:40.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.485+0000 7f5cbed03b80 20 client.28217 populate_metadata read hostname 'smithi101' 2023-07-18T03:20:40.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.485+0000 7f5cbed03b80 1 --2- 172.21.15.101:0/877899892 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x56317f5e91b0 0x56317f5eb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.485+0000 7f5cbed03b80 4 client.28217 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:20:40.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.485+0000 7f5cbed03b80 1 -- 172.21.15.101:0/877899892 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56317f519eb0 con 0x56317f5e91b0 2023-07-18T03:20:40.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.485+0000 7f5ca899a700 1 --2- 172.21.15.101:0/877899892 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x56317f5e91b0 0x56317f5eb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.486+0000 7f5ca899a700 1 --2- 172.21.15.101:0/877899892 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x56317f5e91b0 0x56317f5eb5c0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e00a73509c0b3eb7 server_cookie=4dac16f4837f1672 in_seq=0 out_seq=0 2023-07-18T03:20:40.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.486+0000 7f5c8cff9700 10 client.28217 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:20:40.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.486+0000 7f5c8cff9700 1 -- 172.21.15.101:0/877899892 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56317f519eb0 con 0x56317f5e91b0 2023-07-18T03:20:40.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.486+0000 7f5c8cff9700 10 client.28217 handle_command_reply: tid=0 2023-07-18T03:20:40.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.486+0000 7f5cbed03b80 1 client.28217 shutdown 2023-07-18T03:20:40.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.487+0000 7f5cbed03b80 1 -- 172.21.15.101:0/877899892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56317f5ce2d0 msgr2=0x56317f5d9ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:40.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.487+0000 7f5cbed03b80 1 --2- 172.21.15.101:0/877899892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56317f5ce2d0 0x56317f5d9ea0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f5ca00129e0 tx=0x7f5ca0012cf0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.487+0000 7f5cbed03b80 1 -- 172.21.15.101:0/877899892 shutdown_connections 2023-07-18T03:20:40.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.487+0000 7f5cbed03b80 1 --2- 172.21.15.101:0/877899892 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x56317f5e91b0 0x56317f5eb5c0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.487+0000 7f5cbed03b80 1 --2- 172.21.15.101:0/877899892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56317f5ce2d0 0x56317f5d9ea0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.487+0000 7f5cbed03b80 1 --2- 172.21.15.101:0/877899892 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56317f5d9ad0 0x56317f5da920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.487+0000 7f5cbed03b80 1 --2- 172.21.15.101:0/877899892 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56317f5cebe0 0x56317f5da3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.487+0000 7f5cbed03b80 1 -- 172.21.15.101:0/877899892 >> 172.21.15.101:0/877899892 conn(0x56317f5c4da0 msgr2=0x56317f5ca230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:40.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.487+0000 7f5c8cff9700 0 client.28217 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:20:40.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.487+0000 7f5cbed03b80 1 -- 172.21.15.101:0/877899892 shutdown_connections 2023-07-18T03:20:40.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.487+0000 7f5cbed03b80 1 -- 172.21.15.101:0/877899892 wait complete. 2023-07-18T03:20:40.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.487+0000 7f5cbed03b80 20 client.28217 trim_cache size 0 max 16384 2023-07-18T03:20:40.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:40 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3999525353' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:20:40.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.501+0000 7f5cbed03b80 1 Processor -- start 2023-07-18T03:20:40.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.501+0000 7f5cbed03b80 1 -- start start 2023-07-18T03:20:40.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.501+0000 7f5cbed03b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56317f5ce2d0 0x56317f5d3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.501+0000 7f5cbed03b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56317f5cebe0 0x56317f5d82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.501+0000 7f5cbed03b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56317f5d9ad0 0x56317f5d2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.501+0000 7f5cbed03b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317f5e6830 con 0x56317f5d9ad0 2023-07-18T03:20:40.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.501+0000 7f5cbed03b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317f5e4fc0 con 0x56317f5cebe0 2023-07-18T03:20:40.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.501+0000 7f5cbed03b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317f5d01c0 con 0x56317f5ce2d0 2023-07-18T03:20:40.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.501+0000 7f5ca919b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56317f5ce2d0 0x56317f5d3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.502+0000 7f5ca919b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56317f5ce2d0 0x56317f5d3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41648/0 (socket says 172.21.15.101:41648) 2023-07-18T03:20:40.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.502+0000 7f5ca919b700 1 -- 172.21.15.101:0/4234017984 learned_addr learned my addr 172.21.15.101:0/4234017984 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:40.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.502+0000 7f5ca899a700 1 --2- 172.21.15.101:0/4234017984 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56317f5cebe0 0x56317f5d82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.502+0000 7f5ca999c700 1 --2- 172.21.15.101:0/4234017984 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56317f5d9ad0 0x56317f5d2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.502+0000 7f5ca899a700 1 -- 172.21.15.101:0/4234017984 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56317f5ce2d0 msgr2=0x56317f5d3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:40.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.502+0000 7f5ca899a700 1 --2- 172.21.15.101:0/4234017984 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56317f5ce2d0 0x56317f5d3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.502+0000 7f5ca899a700 1 -- 172.21.15.101:0/4234017984 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56317f5d9ad0 msgr2=0x56317f5d2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:40.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.502+0000 7f5ca899a700 1 --2- 172.21.15.101:0/4234017984 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56317f5d9ad0 0x56317f5d2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.502+0000 7f5ca899a700 1 -- 172.21.15.101:0/4234017984 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56317f375250 con 0x56317f5cebe0 2023-07-18T03:20:40.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.502+0000 7f5ca919b700 1 --2- 172.21.15.101:0/4234017984 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56317f5ce2d0 0x56317f5d3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:20:40.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.502+0000 7f5ca999c700 1 --2- 172.21.15.101:0/4234017984 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56317f5d9ad0 0x56317f5d2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:20:40.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.502+0000 7f5ca899a700 1 --2- 172.21.15.101:0/4234017984 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56317f5cebe0 0x56317f5d82e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5c940976b0 tx=0x7f5c9400fa70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb739fd2da37f976 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:40.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.502+0000 7f5c8ffff700 1 -- 172.21.15.101:0/4234017984 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c940d1410 con 0x56317f5cebe0 2023-07-18T03:20:40.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.502+0000 7f5c8ffff700 1 -- 172.21.15.101:0/4234017984 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c94096f80 con 0x56317f5cebe0 2023-07-18T03:20:40.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.502+0000 7f5c8ffff700 1 -- 172.21.15.101:0/4234017984 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9409ac40 con 0x56317f5cebe0 2023-07-18T03:20:40.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.503+0000 7f5cbed03b80 1 -- 172.21.15.101:0/4234017984 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56317f5cebe0 msgr2=0x56317f5d82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:40.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.503+0000 7f5cbed03b80 1 --2- 172.21.15.101:0/4234017984 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56317f5cebe0 0x56317f5d82e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5c940976b0 tx=0x7f5c9400fa70 comp rx=0 tx=0).stop 2023-07-18T03:20:40.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.503+0000 7f5cbed03b80 1 -- 172.21.15.101:0/4234017984 shutdown_connections 2023-07-18T03:20:40.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.503+0000 7f5cbed03b80 1 --2- 172.21.15.101:0/4234017984 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56317f5d9ad0 0x56317f5d2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.503+0000 7f5cbed03b80 1 --2- 172.21.15.101:0/4234017984 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56317f5ce2d0 0x56317f5d3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.503+0000 7f5cbed03b80 1 --2- 172.21.15.101:0/4234017984 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56317f5cebe0 0x56317f5d82e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.503+0000 7f5cbed03b80 1 -- 172.21.15.101:0/4234017984 >> 172.21.15.101:0/4234017984 conn(0x56317f5c4da0 msgr2=0x56317f5d3340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:40.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.503+0000 7f5cbed03b80 1 -- 172.21.15.101:0/4234017984 shutdown_connections 2023-07-18T03:20:40.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.503+0000 7f5cbed03b80 1 -- 172.21.15.101:0/4234017984 wait complete. 2023-07-18T03:20:40.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.504+0000 7f5cbed03b80 1 -- start start 2023-07-18T03:20:40.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.504+0000 7f5cbed03b80 1 Processor -- start 2023-07-18T03:20:40.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.504+0000 7f5cbed03b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56317f5ce2d0 0x56317f5dd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.505+0000 7f5cbed03b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56317f5cebe0 0x56317f5dd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.505+0000 7f5cbed03b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56317f5d9ad0 0x56317f63c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.505+0000 7f5cbed03b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317f5ddf40 con 0x56317f5d9ad0 2023-07-18T03:20:40.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.505+0000 7f5cbed03b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317f63c5b0 con 0x56317f5ce2d0 2023-07-18T03:20:40.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.505+0000 7f5cbed03b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317f63c6f0 con 0x56317f5cebe0 2023-07-18T03:20:40.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.505+0000 7f5cbed03b80 10 client.0 _ll_register_callbacks cb 0x7f5ca00124e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:20:40.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.505+0000 7f5ca999c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56317f5d9ad0 0x56317f63c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.505+0000 7f5ca999c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56317f5d9ad0 0x56317f63c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:57122/0 (socket says 172.21.15.101:57122) 2023-07-18T03:20:40.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.505+0000 7f5ca999c700 1 -- 172.21.15.101:0/2007988402 learned_addr learned my addr 172.21.15.101:0/2007988402 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:40.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.505+0000 7f5ca899a700 1 --2- 172.21.15.101:0/2007988402 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56317f5cebe0 0x56317f5dd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.505+0000 7f5ca919b700 1 --2- 172.21.15.101:0/2007988402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56317f5ce2d0 0x56317f5dd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.505+0000 7f5ca999c700 1 -- 172.21.15.101:0/2007988402 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56317f5cebe0 msgr2=0x56317f5dd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:40.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.505+0000 7f5ca999c700 1 --2- 172.21.15.101:0/2007988402 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56317f5cebe0 0x56317f5dd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.505+0000 7f5ca999c700 1 -- 172.21.15.101:0/2007988402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56317f5ce2d0 msgr2=0x56317f5dd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:40.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.505+0000 7f5ca999c700 1 --2- 172.21.15.101:0/2007988402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56317f5ce2d0 0x56317f5dd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.505+0000 7f5ca999c700 1 -- 172.21.15.101:0/2007988402 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56317f337c60 con 0x56317f5d9ad0 2023-07-18T03:20:40.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.505+0000 7f5ca899a700 1 --2- 172.21.15.101:0/2007988402 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56317f5cebe0 0x56317f5dd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:20:40.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.505+0000 7f5ca919b700 1 --2- 172.21.15.101:0/2007988402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56317f5ce2d0 0x56317f5dd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:20:40.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.506+0000 7f5ca999c700 1 --2- 172.21.15.101:0/2007988402 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56317f5d9ad0 0x56317f63c070 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c0043b0 tx=0x7f5c9c004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11a9c23a7976682e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:40.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.506+0000 7f5c8cff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:20:40.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.506+0000 7f5c8cff9700 1 -- 172.21.15.101:0/2007988402 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c00e040 con 0x56317f5d9ad0 2023-07-18T03:20:40.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.506+0000 7f5cbed03b80 1 -- 172.21.15.101:0/2007988402 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c9409aa10 con 0x56317f5d9ad0 2023-07-18T03:20:40.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.506+0000 7f5c8cff9700 1 -- 172.21.15.101:0/2007988402 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f5c9c004db0 con 0x56317f5d9ad0 2023-07-18T03:20:40.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.506+0000 7f5c8cff9700 1 -- 172.21.15.101:0/2007988402 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c9c009400 con 0x56317f5d9ad0 2023-07-18T03:20:40.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.506+0000 7f5cbed03b80 10 client.18186 fetch_fsmap learned FSMap version 35 2023-07-18T03:20:40.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.506+0000 7f5cbed03b80 1 -- 172.21.15.101:0/2007988402 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x56317f34ede0 con 0x56317f5d9ad0 2023-07-18T03:20:40.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.506+0000 7f5c8cff9700 1 -- 172.21.15.101:0/2007988402 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c9409aa10 con 0x56317f5d9ad0 2023-07-18T03:20:40.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.507+0000 7f5c8cff9700 1 -- 172.21.15.101:0/2007988402 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 35) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f5c9c01e070 con 0x56317f5d9ad0 2023-07-18T03:20:40.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.507+0000 7f5cbed03b80 10 client.18186 fetch_fsmap finished waiting for FSMap version 35 2023-07-18T03:20:40.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.507+0000 7f5cbed03b80 10 client.18186 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:20:40.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.507+0000 7f5cbed03b80 20 client.18186 populate_metadata read hostname 'smithi101' 2023-07-18T03:20:40.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.507+0000 7f5cbed03b80 1 --2- 172.21.15.101:0/2007988402 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x56317f646670 0x56317f648a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.507+0000 7f5cbed03b80 4 client.18186 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:20:40.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.507+0000 7f5cbed03b80 1 -- 172.21.15.101:0/2007988402 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56317f4dc050 con 0x56317f646670 2023-07-18T03:20:40.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.507+0000 7f5ca919b700 1 --2- 172.21.15.101:0/2007988402 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x56317f646670 0x56317f648a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.507+0000 7f5c8cff9700 10 client.18186 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:20:40.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.507+0000 7f5ca919b700 1 --2- 172.21.15.101:0/2007988402 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x56317f646670 0x56317f648a80 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dd8fa2538b995c0a server_cookie=b331b16e3a2e6ca in_seq=0 out_seq=0 2023-07-18T03:20:40.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.508+0000 7f5c8cff9700 1 -- 172.21.15.101:0/2007988402 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56317f4dc050 con 0x56317f646670 2023-07-18T03:20:40.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.508+0000 7f5c8cff9700 10 client.18186 handle_command_reply: tid=0 2023-07-18T03:20:40.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.508+0000 7f5cbed03b80 1 client.18186 shutdown 2023-07-18T03:20:40.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.508+0000 7f5cbed03b80 1 -- 172.21.15.101:0/2007988402 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56317f5d9ad0 msgr2=0x56317f63c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:40.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.508+0000 7f5cbed03b80 1 --2- 172.21.15.101:0/2007988402 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56317f5d9ad0 0x56317f63c070 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c0043b0 tx=0x7f5c9c004b40 comp rx=0 tx=0).stop 2023-07-18T03:20:40.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.508+0000 7f5cbed03b80 1 -- 172.21.15.101:0/2007988402 shutdown_connections 2023-07-18T03:20:40.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.508+0000 7f5cbed03b80 1 --2- 172.21.15.101:0/2007988402 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x56317f646670 0x56317f648a80 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.508+0000 7f5cbed03b80 1 --2- 172.21.15.101:0/2007988402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56317f5ce2d0 0x56317f5dd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.508+0000 7f5cbed03b80 1 --2- 172.21.15.101:0/2007988402 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56317f5d9ad0 0x56317f63c070 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.508+0000 7f5c8cff9700 0 client.18186 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:20:40.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.508+0000 7f5cbed03b80 1 --2- 172.21.15.101:0/2007988402 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56317f5cebe0 0x56317f5dd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.508+0000 7f5cbed03b80 1 -- 172.21.15.101:0/2007988402 >> 172.21.15.101:0/2007988402 conn(0x56317f5c4da0 msgr2=0x56317f5d3e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:40.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.508+0000 7f5cbed03b80 1 -- 172.21.15.101:0/2007988402 shutdown_connections 2023-07-18T03:20:40.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.508+0000 7f5cbed03b80 1 -- 172.21.15.101:0/2007988402 wait complete. 2023-07-18T03:20:40.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.509+0000 7f5cbed03b80 20 client.18186 trim_cache size 0 max 16384 2023-07-18T03:20:40.517 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:20:40.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.509+0000 7f5cab3ff700 1 -- 172.21.15.101:0/3710592754 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f5c900a1340 msgr2=0x7f5c900a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:40.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.509+0000 7f5cab3ff700 1 --2- 172.21.15.101:0/3710592754 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f5c900a1340 0x7f5c900a37f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c0127f0 tx=0x7f5c9c00b040 comp rx=0 tx=0).stop 2023-07-18T03:20:40.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.509+0000 7f5cab3ff700 1 -- 172.21.15.101:0/3710592754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ca40ff380 msgr2=0x7f5ca4198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:40.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.509+0000 7f5cab3ff700 1 --2- 172.21.15.101:0/3710592754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ca40ff380 0x7f5ca4198360 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5c94009810 tx=0x7f5c9400fa00 comp rx=0 tx=0).stop 2023-07-18T03:20:40.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.509+0000 7f5cab3ff700 1 -- 172.21.15.101:0/3710592754 shutdown_connections 2023-07-18T03:20:40.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.509+0000 7f5cab3ff700 1 --2- 172.21.15.101:0/3710592754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ca41049e0 0x7f5ca419c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.509+0000 7f5cab3ff700 1 --2- 172.21.15.101:0/3710592754 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5ca40fe9d0 0x7f5ca4197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.509+0000 7f5cab3ff700 1 --2- 172.21.15.101:0/3710592754 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f5c900a1340 0x7f5c900a37f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.509+0000 7f5cab3ff700 1 --2- 172.21.15.101:0/3710592754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ca40ff380 0x7f5ca4198360 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.509+0000 7f5cab3ff700 1 -- 172.21.15.101:0/3710592754 >> 172.21.15.101:0/3710592754 conn(0x7f5ca40fa4b0 msgr2=0x7f5ca40fbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:40.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.510+0000 7f5cab3ff700 1 -- 172.21.15.101:0/3710592754 shutdown_connections 2023-07-18T03:20:40.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.510+0000 7f5cab3ff700 1 -- 172.21.15.101:0/3710592754 wait complete. 2023-07-18T03:20:40.524 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-18T03:20:40.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:40 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/3999525353' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:20:40.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.751+0000 7f2687dff700 1 -- 172.21.15.101:0/3992097358 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2680100490 msgr2=0x7f268010c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:40.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.751+0000 7f2687dff700 1 --2- 172.21.15.101:0/3992097358 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2680100490 0x7f268010c920 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2678009990 tx=0x7f2678009ca0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.751+0000 7f2687dff700 1 -- 172.21.15.101:0/3992097358 shutdown_connections 2023-07-18T03:20:40.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.751+0000 7f2687dff700 1 --2- 172.21.15.101:0/3992097358 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2680100490 0x7f268010c920 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.751+0000 7f2687dff700 1 --2- 172.21.15.101:0/3992097358 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2680105af0 0x7f2680105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.751+0000 7f2687dff700 1 --2- 172.21.15.101:0/3992097358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26800ffae0 0x7f26800fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.751+0000 7f2687dff700 1 -- 172.21.15.101:0/3992097358 >> 172.21.15.101:0/3992097358 conn(0x7f26800fb5e0 msgr2=0x7f26800fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:40.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.751+0000 7f2687dff700 1 -- 172.21.15.101:0/3992097358 shutdown_connections 2023-07-18T03:20:40.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.752+0000 7f2687dff700 1 -- 172.21.15.101:0/3992097358 wait complete. 2023-07-18T03:20:40.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.753+0000 7f2687dff700 1 Processor -- start 2023-07-18T03:20:40.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.753+0000 7f2687dff700 1 -- start start 2023-07-18T03:20:40.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.754+0000 7f2687dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f26800ffae0 0x7f2680197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.754+0000 7f2687dff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2680100490 0x7f2680198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.754+0000 7f2687dff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2680105af0 0x7f268019c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.754+0000 7f2687dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268019cd60 con 0x7f26800ffae0 2023-07-18T03:20:40.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.754+0000 7f2687dff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268019ced0 con 0x7f2680100490 2023-07-18T03:20:40.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.754+0000 7f2687dff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268019d090 con 0x7f2680105af0 2023-07-18T03:20:40.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.754+0000 7f2685b9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f26800ffae0 0x7f2680197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.754+0000 7f268539a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2680100490 0x7f2680198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.754+0000 7f268539a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2680100490 0x7f2680198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60906/0 (socket says 172.21.15.101:60906) 2023-07-18T03:20:40.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.754+0000 7f2685b9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f26800ffae0 0x7f2680197dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:57144/0 (socket says 172.21.15.101:57144) 2023-07-18T03:20:40.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.754+0000 7f268539a700 1 -- 172.21.15.101:0/769372625 learned_addr learned my addr 172.21.15.101:0/769372625 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:40.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.754+0000 7f268639c700 1 --2- 172.21.15.101:0/769372625 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2680105af0 0x7f268019c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.755+0000 7f268539a700 1 -- 172.21.15.101:0/769372625 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2680105af0 msgr2=0x7f268019c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:40.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.755+0000 7f268539a700 1 --2- 172.21.15.101:0/769372625 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2680105af0 0x7f268019c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.755+0000 7f268539a700 1 -- 172.21.15.101:0/769372625 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f26800ffae0 msgr2=0x7f2680197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:40.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.755+0000 7f268539a700 1 --2- 172.21.15.101:0/769372625 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f26800ffae0 0x7f2680197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.755+0000 7f268539a700 1 -- 172.21.15.101:0/769372625 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26780095f0 con 0x7f2680100490 2023-07-18T03:20:40.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.755+0000 7f268539a700 1 --2- 172.21.15.101:0/769372625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2680100490 0x7f2680198310 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f267000d950 tx=0x7f267000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:40.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.755+0000 7f2676ffd700 1 -- 172.21.15.101:0/769372625 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267000f890 con 0x7f2680100490 2023-07-18T03:20:40.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.755+0000 7f2687dff700 1 -- 172.21.15.101:0/769372625 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f268019d370 con 0x7f2680100490 2023-07-18T03:20:40.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.755+0000 7f2676ffd700 1 -- 172.21.15.101:0/769372625 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2670010460 con 0x7f2680100490 2023-07-18T03:20:40.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.755+0000 7f2676ffd700 1 -- 172.21.15.101:0/769372625 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267000e610 con 0x7f2680100490 2023-07-18T03:20:40.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.755+0000 7f2687dff700 1 -- 172.21.15.101:0/769372625 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26801a4bb0 con 0x7f2680100490 2023-07-18T03:20:40.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.759+0000 7f2676ffd700 1 -- 172.21.15.101:0/769372625 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f26700108d0 con 0x7f2680100490 2023-07-18T03:20:40.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.760+0000 7f269b633b80 1 Processor -- start 2023-07-18T03:20:40.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.760+0000 7f269b633b80 1 -- start start 2023-07-18T03:20:40.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.760+0000 7f2676ffd700 1 --2- 172.21.15.101:0/769372625 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f266c0a1300 0x7f266c0a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.761+0000 7f2676ffd700 1 -- 172.21.15.101:0/769372625 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f26700ccdf0 con 0x7f2680100490 2023-07-18T03:20:40.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.761+0000 7f269b633b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e55ab32d0 0x555e55ab36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.761+0000 7f269b633b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e55ab3be0 0x555e55abe590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.761+0000 7f269b633b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e55abead0 0x555e55ac0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.761+0000 7f269b633b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e559b9cb0 con 0x555e55ab3be0 2023-07-18T03:20:40.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.761+0000 7f269b633b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e55a0ca40 con 0x555e55ab32d0 2023-07-18T03:20:40.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.761+0000 7f269b633b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e55a05ed0 con 0x555e55abead0 2023-07-18T03:20:40.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.761+0000 7f268639c700 1 --2- 172.21.15.101:0/769372625 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f266c0a1300 0x7f266c0a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.761+0000 7f268539a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e55ab3be0 0x555e55abe590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.761+0000 7f2685b9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e55ab32d0 0x555e55ab36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.761+0000 7f2685b9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e55ab32d0 0x555e55ab36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60922/0 (socket says 172.21.15.101:60922) 2023-07-18T03:20:40.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.761+0000 7f268539a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e55ab3be0 0x555e55abe590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:57146/0 (socket says 172.21.15.101:57146) 2023-07-18T03:20:40.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.761+0000 7f2685b9b700 1 -- 172.21.15.101:0/1335671115 learned_addr learned my addr 172.21.15.101:0/1335671115 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:40.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.761+0000 7f268639c700 1 --2- 172.21.15.101:0/1335671115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e55abead0 0x555e55ac0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.762+0000 7f268539a700 1 -- 172.21.15.101:0/1335671115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e55abead0 msgr2=0x555e55ac0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:40.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.762+0000 7f268539a700 1 --2- 172.21.15.101:0/1335671115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e55abead0 0x555e55ac0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.762+0000 7f268539a700 1 -- 172.21.15.101:0/1335671115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e55ab32d0 msgr2=0x555e55ab36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:40.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.762+0000 7f268539a700 1 --2- 172.21.15.101:0/1335671115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e55ab32d0 0x555e55ab36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.762+0000 7f268539a700 1 -- 172.21.15.101:0/1335671115 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e55833de0 con 0x555e55ab3be0 2023-07-18T03:20:40.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.762+0000 7f2685b9b700 1 --2- 172.21.15.101:0/1335671115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e55ab32d0 0x555e55ab36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:20:40.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.762+0000 7f268539a700 1 --2- 172.21.15.101:0/1335671115 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e55ab3be0 0x555e55abe590 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f26700ca190 tx=0x7f2670099780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6961bb79793111f6 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:40.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.762+0000 7f268639c700 1 --2- 172.21.15.101:0/769372625 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f266c0a1300 0x7f266c0a37b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f26780095c0 tx=0x7f267800b7d0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:40.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.762+0000 7f266bfff700 1 -- 172.21.15.101:0/1335671115 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2670096220 con 0x555e55ab3be0 2023-07-18T03:20:40.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.762+0000 7f266bfff700 1 -- 172.21.15.101:0/1335671115 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26700b9050 con 0x555e55ab3be0 2023-07-18T03:20:40.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.762+0000 7f266bfff700 1 -- 172.21.15.101:0/1335671115 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2670099240 con 0x555e55ab3be0 2023-07-18T03:20:40.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.763+0000 7f269b633b80 1 -- 172.21.15.101:0/1335671115 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e55ab3be0 msgr2=0x555e55abe590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:40.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.763+0000 7f269b633b80 1 --2- 172.21.15.101:0/1335671115 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e55ab3be0 0x555e55abe590 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f26700ca190 tx=0x7f2670099780 comp rx=0 tx=0).stop 2023-07-18T03:20:40.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.763+0000 7f269b633b80 1 -- 172.21.15.101:0/1335671115 shutdown_connections 2023-07-18T03:20:40.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.763+0000 7f269b633b80 1 --2- 172.21.15.101:0/1335671115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e55ab32d0 0x555e55ab36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.763+0000 7f269b633b80 1 --2- 172.21.15.101:0/1335671115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e55abead0 0x555e55ac0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.763+0000 7f269b633b80 1 --2- 172.21.15.101:0/1335671115 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e55ab3be0 0x555e55abe590 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.763+0000 7f269b633b80 1 -- 172.21.15.101:0/1335671115 >> 172.21.15.101:0/1335671115 conn(0x555e55aa9da0 msgr2=0x555e55aaf6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:40.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.763+0000 7f269b633b80 1 -- 172.21.15.101:0/1335671115 shutdown_connections 2023-07-18T03:20:40.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.763+0000 7f269b633b80 1 -- 172.21.15.101:0/1335671115 wait complete. 2023-07-18T03:20:40.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.764+0000 7f269b633b80 1 -- start start 2023-07-18T03:20:40.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.764+0000 7f269b633b80 1 Processor -- start 2023-07-18T03:20:40.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.765+0000 7f269b633b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e55ab32d0 0x555e55abeea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.765+0000 7f269b633b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e55ab3be0 0x555e55abf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.765+0000 7f269b633b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e55abead0 0x555e55ac9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.765+0000 7f269b633b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e559a5ac0 con 0x555e55ab32d0 2023-07-18T03:20:40.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.765+0000 7f269b633b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e559a57c0 con 0x555e55ab3be0 2023-07-18T03:20:40.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.765+0000 7f269b633b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e55889920 con 0x555e55abead0 2023-07-18T03:20:40.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.765+0000 7f269b633b80 10 client.0 _ll_register_callbacks cb 0x555e559a3760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:20:40.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.765+0000 7f2685b9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e55ab32d0 0x555e55abeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.765+0000 7f268639c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e55abead0 0x555e55ac9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.765+0000 7f2685b9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e55ab32d0 0x555e55abeea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:57150/0 (socket says 172.21.15.101:57150) 2023-07-18T03:20:40.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.765+0000 7f2685b9b700 1 -- 172.21.15.101:0/3618431856 learned_addr learned my addr 172.21.15.101:0/3618431856 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:40.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.765+0000 7f268539a700 1 --2- 172.21.15.101:0/3618431856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e55ab3be0 0x555e55abf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.765+0000 7f2685b9b700 1 -- 172.21.15.101:0/3618431856 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e55abead0 msgr2=0x555e55ac9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:40.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.765+0000 7f2685b9b700 1 --2- 172.21.15.101:0/3618431856 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e55abead0 0x555e55ac9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.765+0000 7f2685b9b700 1 -- 172.21.15.101:0/3618431856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e55ab3be0 msgr2=0x555e55abf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:40.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.765+0000 7f2685b9b700 1 --2- 172.21.15.101:0/3618431856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e55ab3be0 0x555e55abf3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.765+0000 7f2685b9b700 1 -- 172.21.15.101:0/3618431856 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f267c003c70 con 0x555e55ab32d0 2023-07-18T03:20:40.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.765+0000 7f268539a700 1 --2- 172.21.15.101:0/3618431856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e55ab3be0 0x555e55abf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:20:40.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.766+0000 7f2685b9b700 1 --2- 172.21.15.101:0/3618431856 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e55ab32d0 0x555e55abeea0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f267c0042f0 tx=0x7f267c004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48c76e71972b528f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:40.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.766+0000 7f26697fa700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:20:40.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.766+0000 7f26697fa700 1 -- 172.21.15.101:0/3618431856 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c004db0 con 0x555e55ab32d0 2023-07-18T03:20:40.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.766+0000 7f26697fa700 1 -- 172.21.15.101:0/3618431856 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f267c007710 con 0x555e55ab32d0 2023-07-18T03:20:40.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.766+0000 7f269b633b80 1 -- 172.21.15.101:0/3618431856 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555e5595f4f0 con 0x555e55ab32d0 2023-07-18T03:20:40.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.766+0000 7f26697fa700 1 -- 172.21.15.101:0/3618431856 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f267c0078d0 con 0x555e55ab32d0 2023-07-18T03:20:40.769 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.766+0000 7f269b633b80 10 client.18198 fetch_fsmap learned FSMap version 35 2023-07-18T03:20:40.769 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.766+0000 7f269b633b80 1 -- 172.21.15.101:0/3618431856 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x7f26801a4bb0 con 0x555e55ab32d0 2023-07-18T03:20:40.769 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.766+0000 7f26697fa700 1 -- 172.21.15.101:0/3618431856 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555e5595f4f0 con 0x555e55ab32d0 2023-07-18T03:20:40.769 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.766+0000 7f26697fa700 1 -- 172.21.15.101:0/3618431856 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 35) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f267c016b30 con 0x555e55ab32d0 2023-07-18T03:20:40.769 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.767+0000 7f269b633b80 10 client.18198 fetch_fsmap finished waiting for FSMap version 35 2023-07-18T03:20:40.769 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.767+0000 7f269b633b80 10 client.18198 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:20:40.769 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.767+0000 7f269b633b80 20 client.18198 populate_metadata read hostname 'smithi101' 2023-07-18T03:20:40.769 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.767+0000 7f269b633b80 1 --2- 172.21.15.101:0/3618431856 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x555e55ace2c0 0x555e55ad06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.767+0000 7f269b633b80 4 client.18198 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:20:40.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.767+0000 7f269b633b80 1 -- 172.21.15.101:0/3618431856 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e559feeb0 con 0x555e55ace2c0 2023-07-18T03:20:40.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.767+0000 7f268539a700 1 --2- 172.21.15.101:0/3618431856 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x555e55ace2c0 0x555e55ad06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.767+0000 7f268539a700 1 --2- 172.21.15.101:0/3618431856 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x555e55ace2c0 0x555e55ad06d0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9313c0bd1f58d00a server_cookie=6adfe486f6aad9e5 in_seq=0 out_seq=0 2023-07-18T03:20:40.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.767+0000 7f26697fa700 10 client.18198 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:20:40.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.770+0000 7f26697fa700 1 -- 172.21.15.101:0/3618431856 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555e559feeb0 con 0x555e55ace2c0 2023-07-18T03:20:40.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.770+0000 7f26697fa700 10 client.18198 handle_command_reply: tid=0 2023-07-18T03:20:40.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.770+0000 7f269b633b80 1 client.18198 shutdown 2023-07-18T03:20:40.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.770+0000 7f269b633b80 1 -- 172.21.15.101:0/3618431856 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e55ab32d0 msgr2=0x555e55abeea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:40.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.770+0000 7f269b633b80 1 --2- 172.21.15.101:0/3618431856 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e55ab32d0 0x555e55abeea0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f267c0042f0 tx=0x7f267c004320 comp rx=0 tx=0).stop 2023-07-18T03:20:40.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.770+0000 7f269b633b80 1 -- 172.21.15.101:0/3618431856 shutdown_connections 2023-07-18T03:20:40.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.770+0000 7f269b633b80 1 --2- 172.21.15.101:0/3618431856 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x555e55ace2c0 0x555e55ad06d0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.770+0000 7f269b633b80 1 --2- 172.21.15.101:0/3618431856 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e55abead0 0x555e55ac9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.770+0000 7f269b633b80 1 --2- 172.21.15.101:0/3618431856 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e55ab32d0 0x555e55abeea0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.770+0000 7f269b633b80 1 --2- 172.21.15.101:0/3618431856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e55ab3be0 0x555e55abf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.770+0000 7f26697fa700 0 client.18198 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:20:40.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.770+0000 7f269b633b80 1 -- 172.21.15.101:0/3618431856 >> 172.21.15.101:0/3618431856 conn(0x555e55aa9da0 msgr2=0x555e55aaf230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:40.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.770+0000 7f269b633b80 1 -- 172.21.15.101:0/3618431856 shutdown_connections 2023-07-18T03:20:40.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.770+0000 7f269b633b80 1 -- 172.21.15.101:0/3618431856 wait complete. 2023-07-18T03:20:40.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.771+0000 7f269b633b80 20 client.18198 trim_cache size 0 max 16384 2023-07-18T03:20:40.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.785+0000 7f269b633b80 1 Processor -- start 2023-07-18T03:20:40.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.785+0000 7f269b633b80 1 -- start start 2023-07-18T03:20:40.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.786+0000 7f269b633b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e55ab32d0 0x555e55abd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.786+0000 7f269b633b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e55ab3be0 0x555e55ab8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.786+0000 7f269b633b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e55abead0 0x555e55ab9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.786+0000 7f269b633b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e55ab4d60 con 0x555e55ab32d0 2023-07-18T03:20:40.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.786+0000 7f269b633b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e55ab4200 con 0x555e55ab3be0 2023-07-18T03:20:40.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.786+0000 7f269b633b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e55aa9970 con 0x555e55abead0 2023-07-18T03:20:40.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.786+0000 7f2685b9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e55ab32d0 0x555e55abd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.786+0000 7f2685b9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e55ab32d0 0x555e55abd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:57158/0 (socket says 172.21.15.101:57158) 2023-07-18T03:20:40.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.786+0000 7f2685b9b700 1 -- 172.21.15.101:0/2371712770 learned_addr learned my addr 172.21.15.101:0/2371712770 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:40.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.786+0000 7f268539a700 1 --2- 172.21.15.101:0/2371712770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e55ab3be0 0x555e55ab8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.786+0000 7f268639c700 1 --2- 172.21.15.101:0/2371712770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e55abead0 0x555e55ab9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.786+0000 7f268539a700 1 -- 172.21.15.101:0/2371712770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e55abead0 msgr2=0x555e55ab9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:40.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.786+0000 7f268539a700 1 --2- 172.21.15.101:0/2371712770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e55abead0 0x555e55ab9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.786+0000 7f268539a700 1 -- 172.21.15.101:0/2371712770 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e55ab32d0 msgr2=0x555e55abd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:40.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.786+0000 7f268539a700 1 --2- 172.21.15.101:0/2371712770 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e55ab32d0 0x555e55abd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.786+0000 7f268539a700 1 -- 172.21.15.101:0/2371712770 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e5581cc60 con 0x555e55ab3be0 2023-07-18T03:20:40.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.787+0000 7f268539a700 1 --2- 172.21.15.101:0/2371712770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e55ab3be0 0x555e55ab8ff0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2670097e60 tx=0x7f2670000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac49b5d9855f9b49 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:40.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.787+0000 7f2674bff700 1 -- 172.21.15.101:0/2371712770 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2670096220 con 0x555e55ab3be0 2023-07-18T03:20:40.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.787+0000 7f2674bff700 1 -- 172.21.15.101:0/2371712770 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2670096380 con 0x555e55ab3be0 2023-07-18T03:20:40.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.787+0000 7f2674bff700 1 -- 172.21.15.101:0/2371712770 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2670099fd0 con 0x555e55ab3be0 2023-07-18T03:20:40.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.788+0000 7f269b633b80 1 -- 172.21.15.101:0/2371712770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e55ab3be0 msgr2=0x555e55ab8ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:40.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.788+0000 7f269b633b80 1 --2- 172.21.15.101:0/2371712770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e55ab3be0 0x555e55ab8ff0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2670097e60 tx=0x7f2670000bc0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.788+0000 7f269b633b80 1 -- 172.21.15.101:0/2371712770 shutdown_connections 2023-07-18T03:20:40.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.788+0000 7f269b633b80 1 --2- 172.21.15.101:0/2371712770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e55abead0 0x555e55ab9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.788+0000 7f269b633b80 1 --2- 172.21.15.101:0/2371712770 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e55ab32d0 0x555e55abd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.788+0000 7f269b633b80 1 --2- 172.21.15.101:0/2371712770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e55ab3be0 0x555e55ab8ff0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.788+0000 7f269b633b80 1 -- 172.21.15.101:0/2371712770 >> 172.21.15.101:0/2371712770 conn(0x555e55aa9da0 msgr2=0x555e55ab84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:40.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.788+0000 7f269b633b80 1 -- 172.21.15.101:0/2371712770 shutdown_connections 2023-07-18T03:20:40.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.788+0000 7f269b633b80 1 -- 172.21.15.101:0/2371712770 wait complete. 2023-07-18T03:20:40.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.788+0000 7f269b633b80 1 -- start start 2023-07-18T03:20:40.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.788+0000 7f269b633b80 1 Processor -- start 2023-07-18T03:20:40.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.789+0000 7f269b633b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e55ab32d0 0x555e55ac1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.789+0000 7f269b633b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e55ab3be0 0x555e55ac2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.789+0000 7f269b633b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e55abead0 0x555e55b21180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.789+0000 7f269b633b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e55b216c0 con 0x555e55ab32d0 2023-07-18T03:20:40.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.789+0000 7f269b633b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e55b21800 con 0x555e55ab3be0 2023-07-18T03:20:40.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.789+0000 7f269b633b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e55b21990 con 0x555e55abead0 2023-07-18T03:20:40.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.789+0000 7f269b633b80 10 client.0 _ll_register_callbacks cb 0x7f267c009b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:20:40.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.789+0000 7f268639c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e55abead0 0x555e55b21180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.789+0000 7f2685b9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e55ab32d0 0x555e55ac1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.789+0000 7f268639c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e55abead0 0x555e55b21180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41710/0 (socket says 172.21.15.101:41710) 2023-07-18T03:20:40.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.789+0000 7f268639c700 1 -- 172.21.15.101:0/946369163 learned_addr learned my addr 172.21.15.101:0/946369163 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:40.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.789+0000 7f268539a700 1 --2- 172.21.15.101:0/946369163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e55ab3be0 0x555e55ac2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.790+0000 7f268639c700 1 -- 172.21.15.101:0/946369163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e55ab3be0 msgr2=0x555e55ac2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:40.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.790+0000 7f268639c700 1 --2- 172.21.15.101:0/946369163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e55ab3be0 0x555e55ac2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.790+0000 7f268639c700 1 -- 172.21.15.101:0/946369163 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e55ab32d0 msgr2=0x555e55ac1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:40.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.790+0000 7f268639c700 1 --2- 172.21.15.101:0/946369163 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e55ab32d0 0x555e55ac1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.790+0000 7f268639c700 1 -- 172.21.15.101:0/946369163 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e55833de0 con 0x555e55abead0 2023-07-18T03:20:40.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.790+0000 7f2685b9b700 1 --2- 172.21.15.101:0/946369163 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e55ab32d0 0x555e55ac1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:20:40.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.790+0000 7f268639c700 1 --2- 172.21.15.101:0/946369163 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e55abead0 0x555e55b21180 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2678003b50 tx=0x7f2678027000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a10b9cd77ea29e1 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:40.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.790+0000 7f26697fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:20:40.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.790+0000 7f26697fa700 1 -- 172.21.15.101:0/946369163 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267802d070 con 0x555e55abead0 2023-07-18T03:20:40.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.790+0000 7f269b633b80 1 -- 172.21.15.101:0/946369163 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f26700b9050 con 0x555e55abead0 2023-07-18T03:20:40.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.790+0000 7f26697fa700 1 -- 172.21.15.101:0/946369163 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f2678005690 con 0x555e55abead0 2023-07-18T03:20:40.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.790+0000 7f26697fa700 1 -- 172.21.15.101:0/946369163 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f267801d070 con 0x555e55abead0 2023-07-18T03:20:40.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.790+0000 7f269b633b80 10 client.28279 fetch_fsmap learned FSMap version 35 2023-07-18T03:20:40.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.790+0000 7f269b633b80 1 -- 172.21.15.101:0/946369163 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x555e5585a250 con 0x555e55abead0 2023-07-18T03:20:40.793 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.791+0000 7f26697fa700 1 -- 172.21.15.101:0/946369163 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26700b9050 con 0x555e55abead0 2023-07-18T03:20:40.793 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.791+0000 7f26697fa700 1 -- 172.21.15.101:0/946369163 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 35) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f267800f460 con 0x555e55abead0 2023-07-18T03:20:40.793 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.791+0000 7f269b633b80 10 client.28279 fetch_fsmap finished waiting for FSMap version 35 2023-07-18T03:20:40.793 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.791+0000 7f269b633b80 10 client.28279 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:20:40.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.791+0000 7f269b633b80 20 client.28279 populate_metadata read hostname 'smithi101' 2023-07-18T03:20:40.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.791+0000 7f269b633b80 1 --2- 172.21.15.101:0/946369163 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x555e55b23140 0x555e55b25550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:40.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.791+0000 7f269b633b80 4 client.28279 mds_command: new command op to 24562 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:20:40.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.791+0000 7f269b633b80 1 -- 172.21.15.101:0/946369163 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555e559c1050 con 0x555e55b23140 2023-07-18T03:20:40.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.791+0000 7f2685b9b700 1 --2- 172.21.15.101:0/946369163 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x555e55b23140 0x555e55b25550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:40.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.792+0000 7f2685b9b700 1 --2- 172.21.15.101:0/946369163 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x555e55b23140 0x555e55b25550 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4244191180847e21 server_cookie=460e25fd7d8943f in_seq=0 out_seq=0 2023-07-18T03:20:40.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.792+0000 7f26697fa700 10 client.28279 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:20:40.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.794+0000 7f26697fa700 1 -- 172.21.15.101:0/946369163 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555e559c1050 con 0x555e55b23140 2023-07-18T03:20:40.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.794+0000 7f26697fa700 10 client.28279 handle_command_reply: tid=0 2023-07-18T03:20:40.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.794+0000 7f269b633b80 1 client.28279 shutdown 2023-07-18T03:20:40.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.794+0000 7f269b633b80 1 -- 172.21.15.101:0/946369163 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e55abead0 msgr2=0x555e55b21180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:40.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.794+0000 7f269b633b80 1 --2- 172.21.15.101:0/946369163 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e55abead0 0x555e55b21180 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2678003b50 tx=0x7f2678027000 comp rx=0 tx=0).stop 2023-07-18T03:20:40.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.794+0000 7f269b633b80 1 -- 172.21.15.101:0/946369163 shutdown_connections 2023-07-18T03:20:40.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.794+0000 7f269b633b80 1 --2- 172.21.15.101:0/946369163 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x555e55b23140 0x555e55b25550 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.794+0000 7f269b633b80 1 --2- 172.21.15.101:0/946369163 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555e55abead0 0x555e55b21180 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.794+0000 7f269b633b80 1 --2- 172.21.15.101:0/946369163 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e55ab32d0 0x555e55ac1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.794+0000 7f269b633b80 1 --2- 172.21.15.101:0/946369163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e55ab3be0 0x555e55ac2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.794+0000 7f26697fa700 0 client.28279 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:20:40.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.794+0000 7f269b633b80 1 -- 172.21.15.101:0/946369163 >> 172.21.15.101:0/946369163 conn(0x555e55aa9da0 msgr2=0x555e55aca620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:40.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.794+0000 7f269b633b80 1 -- 172.21.15.101:0/946369163 shutdown_connections 2023-07-18T03:20:40.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.794+0000 7f269b633b80 1 -- 172.21.15.101:0/946369163 wait complete. 2023-07-18T03:20:40.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.794+0000 7f269b633b80 20 client.28279 trim_cache size 0 max 16384 2023-07-18T03:20:40.797 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:20:40.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.795+0000 7f2687dff700 1 -- 172.21.15.101:0/769372625 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f266c0a1300 msgr2=0x7f266c0a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:40.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.795+0000 7f2687dff700 1 --2- 172.21.15.101:0/769372625 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f266c0a1300 0x7f266c0a37b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f26780095c0 tx=0x7f267800b7d0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.795+0000 7f2687dff700 1 -- 172.21.15.101:0/769372625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2680100490 msgr2=0x7f2680198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:40.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.795+0000 7f2687dff700 1 --2- 172.21.15.101:0/769372625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2680100490 0x7f2680198310 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f267000d950 tx=0x7f267000dc60 comp rx=0 tx=0).stop 2023-07-18T03:20:40.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.795+0000 7f2687dff700 1 -- 172.21.15.101:0/769372625 shutdown_connections 2023-07-18T03:20:40.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.795+0000 7f2687dff700 1 --2- 172.21.15.101:0/769372625 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2680105af0 0x7f268019c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.795+0000 7f2687dff700 1 --2- 172.21.15.101:0/769372625 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f26800ffae0 0x7f2680197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.795+0000 7f2687dff700 1 --2- 172.21.15.101:0/769372625 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f266c0a1300 0x7f266c0a37b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.795+0000 7f2687dff700 1 --2- 172.21.15.101:0/769372625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2680100490 0x7f2680198310 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:40.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.795+0000 7f2687dff700 1 -- 172.21.15.101:0/769372625 >> 172.21.15.101:0/769372625 conn(0x7f26800fb5e0 msgr2=0x7f26801035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:40.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.795+0000 7f2687dff700 1 -- 172.21.15.101:0/769372625 shutdown_connections 2023-07-18T03:20:40.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:40.795+0000 7f2687dff700 1 -- 172.21.15.101:0/769372625 wait complete. 2023-07-18T03:20:40.950 INFO:tasks.workunit.client.0.smithi101.stdout: 10 2616 61.03 MB/sec warmup 5 sec latency 3748.239 ms 2023-07-18T03:20:41.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:41 smithi179 ceph-mon[129229]: pgmap v410: 129 pgs: 129 active+clean; 657 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 199 op/s 2023-07-18T03:20:41.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:41 smithi179 ceph-mon[129229]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:20:41.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:41 smithi101 ceph-mon[126379]: pgmap v410: 129 pgs: 129 active+clean; 657 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 199 op/s 2023-07-18T03:20:41.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:41 smithi101 ceph-mon[126379]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:20:41.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:41 smithi155 ceph-mon[128885]: pgmap v410: 129 pgs: 129 active+clean; 657 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 199 op/s 2023-07-18T03:20:41.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:41 smithi155 ceph-mon[128885]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:20:41.810 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-18T03:20:41.811 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-18T03:20:41.950 INFO:tasks.workunit.client.0.smithi101.stdout: 10 3009 55.00 MB/sec warmup 6 sec latency 440.539 ms 2023-07-18T03:20:42.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.068+0000 7f889b3ff700 1 -- 172.21.15.101:0/905926938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8894101f70 msgr2=0x7f8894102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:42.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.068+0000 7f889b3ff700 1 --2- 172.21.15.101:0/905926938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8894101f70 0x7f8894102340 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f88900099c0 tx=0x7f8890009cd0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.068+0000 7f889b3ff700 1 -- 172.21.15.101:0/905926938 shutdown_connections 2023-07-18T03:20:42.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.068+0000 7f889b3ff700 1 --2- 172.21.15.101:0/905926938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8894106fb0 0x7f889410eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.068+0000 7f889b3ff700 1 --2- 172.21.15.101:0/905926938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8894101f70 0x7f8894102340 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.068+0000 7f889b3ff700 1 --2- 172.21.15.101:0/905926938 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8894102910 0x7f88941068d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.068+0000 7f889b3ff700 1 -- 172.21.15.101:0/905926938 >> 172.21.15.101:0/905926938 conn(0x7f88940fd7e0 msgr2=0x7f88940ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:42.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.068+0000 7f889b3ff700 1 -- 172.21.15.101:0/905926938 shutdown_connections 2023-07-18T03:20:42.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.068+0000 7f889b3ff700 1 -- 172.21.15.101:0/905926938 wait complete. 2023-07-18T03:20:42.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.069+0000 7f889b3ff700 1 Processor -- start 2023-07-18T03:20:42.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.070+0000 7f889b3ff700 1 -- start start 2023-07-18T03:20:42.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.070+0000 7f889b3ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8894101f70 0x7f8894197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:42.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.070+0000 7f889b3ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8894102910 0x7f88941983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:42.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.070+0000 7f889b3ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8894106fb0 0x7f889419c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:42.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.070+0000 7f889b3ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889419ce50 con 0x7f8894106fb0 2023-07-18T03:20:42.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.070+0000 7f889b3ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889419cfc0 con 0x7f8894102910 2023-07-18T03:20:42.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.070+0000 7f889b3ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889419d180 con 0x7f8894101f70 2023-07-18T03:20:42.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.070+0000 7f889919b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8894101f70 0x7f8894197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:42.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.070+0000 7f889919b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8894101f70 0x7f8894197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41734/0 (socket says 172.21.15.101:41734) 2023-07-18T03:20:42.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.070+0000 7f889919b700 1 -- 172.21.15.101:0/2875519176 learned_addr learned my addr 172.21.15.101:0/2875519176 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:42.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.070+0000 7f889899a700 1 --2- 172.21.15.101:0/2875519176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8894102910 0x7f88941983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:42.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.070+0000 7f889999c700 1 --2- 172.21.15.101:0/2875519176 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8894106fb0 0x7f889419c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:42.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.071+0000 7f889919b700 1 -- 172.21.15.101:0/2875519176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8894102910 msgr2=0x7f88941983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:42.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.071+0000 7f889919b700 1 --2- 172.21.15.101:0/2875519176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8894102910 0x7f88941983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.071+0000 7f889919b700 1 -- 172.21.15.101:0/2875519176 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8894106fb0 msgr2=0x7f889419c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:42.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.071+0000 7f889919b700 1 --2- 172.21.15.101:0/2875519176 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8894106fb0 0x7f889419c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.071+0000 7f889919b700 1 -- 172.21.15.101:0/2875519176 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88900095f0 con 0x7f8894101f70 2023-07-18T03:20:42.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.071+0000 7f889999c700 1 --2- 172.21.15.101:0/2875519176 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8894106fb0 0x7f889419c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:20:42.069 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.071+0000 7f889919b700 1 --2- 172.21.15.101:0/2875519176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8894101f70 0x7f8894197e70 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f88900098e0 tx=0x7f889000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:42.070 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.071+0000 7f888a7fc700 1 -- 172.21.15.101:0/2875519176 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889001d070 con 0x7f8894101f70 2023-07-18T03:20:42.070 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.071+0000 7f888a7fc700 1 -- 172.21.15.101:0/2875519176 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f889000fb80 con 0x7f8894101f70 2023-07-18T03:20:42.070 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.071+0000 7f888a7fc700 1 -- 172.21.15.101:0/2875519176 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88900173f0 con 0x7f8894101f70 2023-07-18T03:20:42.070 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.071+0000 7f889b3ff700 1 -- 172.21.15.101:0/2875519176 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f889419d400 con 0x7f8894101f70 2023-07-18T03:20:42.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.071+0000 7f889b3ff700 1 -- 172.21.15.101:0/2875519176 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f889419d680 con 0x7f8894101f70 2023-07-18T03:20:42.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.075+0000 7f888a7fc700 1 -- 172.21.15.101:0/2875519176 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f889000fcf0 con 0x7f8894101f70 2023-07-18T03:20:42.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.076+0000 7f88aed07b80 1 Processor -- start 2023-07-18T03:20:42.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.076+0000 7f88aed07b80 1 -- start start 2023-07-18T03:20:42.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.076+0000 7f88aed07b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556e7bf2f350 0x556e7bf2f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:42.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.076+0000 7f88aed07b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556e7bf2fc60 0x556e7bf3a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:42.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.076+0000 7f88aed07b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e7bf3ab50 0x556e7bf3cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:42.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.076+0000 7f88aed07b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e7be35d30 con 0x556e7bf2f350 2023-07-18T03:20:42.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.076+0000 7f88aed07b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e7be88ac0 con 0x556e7bf3ab50 2023-07-18T03:20:42.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.076+0000 7f88aed07b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e7be81f50 con 0x556e7bf2fc60 2023-07-18T03:20:42.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.077+0000 7f889899a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e7bf3ab50 0x556e7bf3cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:42.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.077+0000 7f889899a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e7bf3ab50 0x556e7bf3cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60966/0 (socket says 172.21.15.101:60966) 2023-07-18T03:20:42.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.077+0000 7f889899a700 1 -- 172.21.15.101:0/1726750765 learned_addr learned my addr 172.21.15.101:0/1726750765 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:42.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.077+0000 7f889919b700 1 --2- 172.21.15.101:0/1726750765 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556e7bf2fc60 0x556e7bf3a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:42.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.077+0000 7f888a7fc700 1 --2- 172.21.15.101:0/2875519176 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f88800a1340 0x7f88800a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:42.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.077+0000 7f889999c700 1 --2- 172.21.15.101:0/1726750765 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556e7bf2f350 0x556e7bf2f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:42.075 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.077+0000 7f889899a700 1 -- 172.21.15.101:0/1726750765 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556e7bf2fc60 msgr2=0x556e7bf3a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:42.075 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.077+0000 7f889899a700 1 --2- 172.21.15.101:0/1726750765 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556e7bf2fc60 0x556e7bf3a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.075 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.077+0000 7f889899a700 1 -- 172.21.15.101:0/1726750765 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556e7bf2f350 msgr2=0x556e7bf2f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:42.075 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.077+0000 7f889899a700 1 --2- 172.21.15.101:0/1726750765 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556e7bf2f350 0x556e7bf2f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.075 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.077+0000 7f889899a700 1 -- 172.21.15.101:0/1726750765 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e7bcafe60 con 0x556e7bf3ab50 2023-07-18T03:20:42.075 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.077+0000 7f889999c700 1 --2- 172.21.15.101:0/1726750765 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556e7bf2f350 0x556e7bf2f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:20:42.075 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.077+0000 7f888a7fc700 1 -- 172.21.15.101:0/2875519176 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f88900cdb50 con 0x7f8894101f70 2023-07-18T03:20:42.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.077+0000 7f889899a700 1 --2- 172.21.15.101:0/1726750765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e7bf3ab50 0x556e7bf3cf30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8884009cb0 tx=0x7f8884011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d512edf4c74092df server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:42.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.077+0000 7f889999c700 1 --2- 172.21.15.101:0/2875519176 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f88800a1340 0x7f88800a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:42.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.078+0000 7f887f7fe700 1 -- 172.21.15.101:0/1726750765 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8884011d80 con 0x556e7bf3ab50 2023-07-18T03:20:42.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.078+0000 7f887f7fe700 1 -- 172.21.15.101:0/1726750765 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8884004510 con 0x556e7bf3ab50 2023-07-18T03:20:42.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.078+0000 7f887f7fe700 1 -- 172.21.15.101:0/1726750765 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8884005280 con 0x556e7bf3ab50 2023-07-18T03:20:42.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.079+0000 7f88aed07b80 1 -- 172.21.15.101:0/1726750765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e7bf3ab50 msgr2=0x556e7bf3cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:42.080 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.079+0000 7f889999c700 1 --2- 172.21.15.101:0/2875519176 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f88800a1340 0x7f88800a37f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f888c009480 tx=0x7f888c004ad0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:42.081 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.079+0000 7f88aed07b80 1 --2- 172.21.15.101:0/1726750765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e7bf3ab50 0x556e7bf3cf30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8884009cb0 tx=0x7f8884011440 comp rx=0 tx=0).stop 2023-07-18T03:20:42.081 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.079+0000 7f88aed07b80 1 -- 172.21.15.101:0/1726750765 shutdown_connections 2023-07-18T03:20:42.081 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.079+0000 7f88aed07b80 1 --2- 172.21.15.101:0/1726750765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e7bf3ab50 0x556e7bf3cf30 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.081 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.079+0000 7f88aed07b80 1 --2- 172.21.15.101:0/1726750765 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556e7bf2f350 0x556e7bf2f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.081 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.079+0000 7f88aed07b80 1 --2- 172.21.15.101:0/1726750765 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556e7bf2fc60 0x556e7bf3a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.079+0000 7f88aed07b80 1 -- 172.21.15.101:0/1726750765 >> 172.21.15.101:0/1726750765 conn(0x556e7bf25e20 msgr2=0x556e7bf2b720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:42.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.079+0000 7f88aed07b80 1 -- 172.21.15.101:0/1726750765 shutdown_connections 2023-07-18T03:20:42.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.079+0000 7f88aed07b80 1 -- 172.21.15.101:0/1726750765 wait complete. 2023-07-18T03:20:42.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.080+0000 7f88aed07b80 1 -- start start 2023-07-18T03:20:42.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.080+0000 7f88aed07b80 1 Processor -- start 2023-07-18T03:20:42.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.080+0000 7f88aed07b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e7bf2f350 0x556e7bf3a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:42.083 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.080+0000 7f88aed07b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556e7bf2fc60 0x556e7bf376f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:42.083 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.080+0000 7f88aed07b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556e7bf3ab50 0x556e7bf37c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:42.083 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.080+0000 7f88aed07b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e7be21b40 con 0x556e7bf2fc60 2023-07-18T03:20:42.083 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.080+0000 7f88aed07b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e7be21840 con 0x556e7bf2f350 2023-07-18T03:20:42.083 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.080+0000 7f88aed07b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e7bd059a0 con 0x556e7bf3ab50 2023-07-18T03:20:42.083 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.080+0000 7f88aed07b80 10 client.0 _ll_register_callbacks cb 0x556e7be1f7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:20:42.083 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.081+0000 7f889919b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e7bf2f350 0x556e7bf3a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:42.083 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.081+0000 7f889919b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e7bf2f350 0x556e7bf3a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60970/0 (socket says 172.21.15.101:60970) 2023-07-18T03:20:42.084 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.081+0000 7f889919b700 1 -- 172.21.15.101:0/3397945957 learned_addr learned my addr 172.21.15.101:0/3397945957 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:42.084 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.081+0000 7f889899a700 1 --2- 172.21.15.101:0/3397945957 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556e7bf2fc60 0x556e7bf376f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:42.084 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.081+0000 7f889999c700 1 --2- 172.21.15.101:0/3397945957 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556e7bf3ab50 0x556e7bf37c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:42.084 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.081+0000 7f889999c700 1 -- 172.21.15.101:0/3397945957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e7bf2f350 msgr2=0x556e7bf3a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:42.084 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.081+0000 7f889999c700 1 --2- 172.21.15.101:0/3397945957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e7bf2f350 0x556e7bf3a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.084 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.081+0000 7f889999c700 1 -- 172.21.15.101:0/3397945957 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556e7bf2fc60 msgr2=0x556e7bf376f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:42.085 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.081+0000 7f889999c700 1 --2- 172.21.15.101:0/3397945957 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556e7bf2fc60 0x556e7bf376f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.085 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.081+0000 7f889999c700 1 -- 172.21.15.101:0/3397945957 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88840098e0 con 0x556e7bf3ab50 2023-07-18T03:20:42.085 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.081+0000 7f889999c700 1 --2- 172.21.15.101:0/3397945957 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556e7bf3ab50 0x556e7bf37c90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f888c019e50 tx=0x7f888c00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e6cf60dad69c1ea server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:42.085 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.081+0000 7f887cff9700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:20:42.085 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.081+0000 7f887cff9700 1 -- 172.21.15.101:0/3397945957 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888c01e070 con 0x556e7bf3ab50 2023-07-18T03:20:42.085 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.082+0000 7f88aed07b80 1 -- 172.21.15.101:0/3397945957 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556e7bddb570 con 0x556e7bf3ab50 2023-07-18T03:20:42.085 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.082+0000 7f887cff9700 1 -- 172.21.15.101:0/3397945957 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f888c0190d0 con 0x556e7bf3ab50 2023-07-18T03:20:42.085 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.082+0000 7f887cff9700 1 -- 172.21.15.101:0/3397945957 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f888c00e050 con 0x556e7bf3ab50 2023-07-18T03:20:42.086 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.082+0000 7f88aed07b80 10 client.28291 fetch_fsmap learned FSMap version 35 2023-07-18T03:20:42.086 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.082+0000 7f88aed07b80 1 -- 172.21.15.101:0/3397945957 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x556e7bc98ce0 con 0x556e7bf3ab50 2023-07-18T03:20:42.086 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.082+0000 7f887cff9700 1 -- 172.21.15.101:0/3397945957 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556e7bddb570 con 0x556e7bf3ab50 2023-07-18T03:20:42.086 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.082+0000 7f887cff9700 1 -- 172.21.15.101:0/3397945957 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 35) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f888c0179b0 con 0x556e7bf3ab50 2023-07-18T03:20:42.086 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.082+0000 7f88aed07b80 10 client.28291 fetch_fsmap finished waiting for FSMap version 35 2023-07-18T03:20:42.086 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.082+0000 7f88aed07b80 10 client.28291 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:20:42.086 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.082+0000 7f88aed07b80 20 client.28291 populate_metadata read hostname 'smithi101' 2023-07-18T03:20:42.087 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.082+0000 7f88aed07b80 1 --2- 172.21.15.101:0/3397945957 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x556e7bf4a350 0x556e7bf4c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:42.087 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.082+0000 7f88aed07b80 4 client.28291 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:20:42.087 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.082+0000 7f88aed07b80 1 -- 172.21.15.101:0/3397945957 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556e7be7af30 con 0x556e7bf4a350 2023-07-18T03:20:42.087 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.083+0000 7f889919b700 1 --2- 172.21.15.101:0/3397945957 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x556e7bf4a350 0x556e7bf4c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:42.087 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.083+0000 7f889919b700 1 --2- 172.21.15.101:0/3397945957 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x556e7bf4a350 0x556e7bf4c760 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f481099bb53b6808 server_cookie=64275592142a212e in_seq=0 out_seq=0 2023-07-18T03:20:42.087 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.083+0000 7f887cff9700 10 client.28291 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:20:42.088 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.088+0000 7f887cff9700 1 -- 172.21.15.101:0/3397945957 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556e7be7af30 con 0x556e7bf4a350 2023-07-18T03:20:42.088 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.088+0000 7f887cff9700 10 client.28291 handle_command_reply: tid=0 2023-07-18T03:20:42.089 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.088+0000 7f88aed07b80 1 client.28291 shutdown 2023-07-18T03:20:42.089 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.088+0000 7f88aed07b80 1 -- 172.21.15.101:0/3397945957 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556e7bf3ab50 msgr2=0x556e7bf37c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:42.089 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.088+0000 7f88aed07b80 1 --2- 172.21.15.101:0/3397945957 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556e7bf3ab50 0x556e7bf37c90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f888c019e50 tx=0x7f888c00b040 comp rx=0 tx=0).stop 2023-07-18T03:20:42.089 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.088+0000 7f88aed07b80 1 -- 172.21.15.101:0/3397945957 shutdown_connections 2023-07-18T03:20:42.089 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.088+0000 7f88aed07b80 1 --2- 172.21.15.101:0/3397945957 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x556e7bf4a350 0x556e7bf4c760 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.089 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.088+0000 7f88aed07b80 1 --2- 172.21.15.101:0/3397945957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e7bf2f350 0x556e7bf3a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.089 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.088+0000 7f88aed07b80 1 --2- 172.21.15.101:0/3397945957 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556e7bf3ab50 0x556e7bf37c90 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.090 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.088+0000 7f88aed07b80 1 --2- 172.21.15.101:0/3397945957 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556e7bf2fc60 0x556e7bf376f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.090 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.088+0000 7f887cff9700 0 client.28291 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:20:42.090 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.088+0000 7f88aed07b80 1 -- 172.21.15.101:0/3397945957 >> 172.21.15.101:0/3397945957 conn(0x556e7bf25e20 msgr2=0x556e7bf2b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:42.090 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.088+0000 7f88aed07b80 1 -- 172.21.15.101:0/3397945957 shutdown_connections 2023-07-18T03:20:42.090 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.088+0000 7f88aed07b80 1 -- 172.21.15.101:0/3397945957 wait complete. 2023-07-18T03:20:42.090 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.089+0000 7f88aed07b80 20 client.28291 trim_cache size 0 max 16384 2023-07-18T03:20:42.102 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.107+0000 7f88aed07b80 1 Processor -- start 2023-07-18T03:20:42.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.107+0000 7f88aed07b80 1 -- start start 2023-07-18T03:20:42.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.107+0000 7f88aed07b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e7bf2f350 0x556e7bf32a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:42.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.107+0000 7f88aed07b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556e7bf2fc60 0x556e7bf32fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:42.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.107+0000 7f88aed07b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556e7bf3ab50 0x556e7bf3bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:42.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.107+0000 7f88aed07b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e7bf2cda0 con 0x556e7bf2fc60 2023-07-18T03:20:42.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.107+0000 7f88aed07b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e7bf305c0 con 0x556e7bf2f350 2023-07-18T03:20:42.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.107+0000 7f88aed07b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e7bf46420 con 0x556e7bf3ab50 2023-07-18T03:20:42.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.107+0000 7f889919b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e7bf2f350 0x556e7bf32a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:42.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.107+0000 7f889919b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e7bf2f350 0x556e7bf32a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60972/0 (socket says 172.21.15.101:60972) 2023-07-18T03:20:42.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.107+0000 7f889919b700 1 -- 172.21.15.101:0/2805076834 learned_addr learned my addr 172.21.15.101:0/2805076834 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:42.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.107+0000 7f889999c700 1 --2- 172.21.15.101:0/2805076834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556e7bf3ab50 0x556e7bf3bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:42.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.108+0000 7f889899a700 1 --2- 172.21.15.101:0/2805076834 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556e7bf2fc60 0x556e7bf32fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:42.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.108+0000 7f889919b700 1 -- 172.21.15.101:0/2805076834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556e7bf3ab50 msgr2=0x556e7bf3bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:42.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.108+0000 7f889919b700 1 --2- 172.21.15.101:0/2805076834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556e7bf3ab50 0x556e7bf3bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.108+0000 7f889919b700 1 -- 172.21.15.101:0/2805076834 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556e7bf2fc60 msgr2=0x556e7bf32fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:42.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.108+0000 7f889919b700 1 --2- 172.21.15.101:0/2805076834 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556e7bf2fc60 0x556e7bf32fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.108+0000 7f889919b700 1 -- 172.21.15.101:0/2805076834 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e7bf2a160 con 0x556e7bf2f350 2023-07-18T03:20:42.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.108+0000 7f889919b700 1 --2- 172.21.15.101:0/2805076834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e7bf2f350 0x556e7bf32a70 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8890009d70 tx=0x7f88900cff60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0555f37c86a4251 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:42.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.108+0000 7f887ffff700 1 -- 172.21.15.101:0/2805076834 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889009a3e0 con 0x556e7bf2f350 2023-07-18T03:20:42.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.108+0000 7f887ffff700 1 -- 172.21.15.101:0/2805076834 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f889009a200 con 0x556e7bf2f350 2023-07-18T03:20:42.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.108+0000 7f887ffff700 1 -- 172.21.15.101:0/2805076834 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8890098d60 con 0x556e7bf2f350 2023-07-18T03:20:42.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.109+0000 7f88aed07b80 1 -- 172.21.15.101:0/2805076834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e7bf2f350 msgr2=0x556e7bf32a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:42.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.109+0000 7f88aed07b80 1 --2- 172.21.15.101:0/2805076834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e7bf2f350 0x556e7bf32a70 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8890009d70 tx=0x7f88900cff60 comp rx=0 tx=0).stop 2023-07-18T03:20:42.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.109+0000 7f88aed07b80 1 -- 172.21.15.101:0/2805076834 shutdown_connections 2023-07-18T03:20:42.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.109+0000 7f88aed07b80 1 --2- 172.21.15.101:0/2805076834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e7bf2f350 0x556e7bf32a70 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.109+0000 7f88aed07b80 1 --2- 172.21.15.101:0/2805076834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556e7bf3ab50 0x556e7bf3bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.109+0000 7f88aed07b80 1 --2- 172.21.15.101:0/2805076834 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556e7bf2fc60 0x556e7bf32fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.109+0000 7f88aed07b80 1 -- 172.21.15.101:0/2805076834 >> 172.21.15.101:0/2805076834 conn(0x556e7bf25e20 msgr2=0x556e7bf45d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:42.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.109+0000 7f88aed07b80 1 -- 172.21.15.101:0/2805076834 shutdown_connections 2023-07-18T03:20:42.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.109+0000 7f88aed07b80 1 -- 172.21.15.101:0/2805076834 wait complete. 2023-07-18T03:20:42.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.110+0000 7f88aed07b80 1 -- start start 2023-07-18T03:20:42.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.110+0000 7f88aed07b80 1 Processor -- start 2023-07-18T03:20:42.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.111+0000 7f88aed07b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556e7bf2f350 0x556e7bf3fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:42.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.111+0000 7f88aed07b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e7bf2fc60 0x556e7bf40260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:42.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.111+0000 7f88aed07b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556e7bf3ab50 0x556e7bf407a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:42.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.111+0000 7f88aed07b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e7bf9e2c0 con 0x556e7bf3ab50 2023-07-18T03:20:42.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.111+0000 7f88aed07b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e7bf9e400 con 0x556e7bf2fc60 2023-07-18T03:20:42.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.111+0000 7f88aed07b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e7bf9e5c0 con 0x556e7bf2f350 2023-07-18T03:20:42.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.111+0000 7f88aed07b80 10 client.0 _ll_register_callbacks cb 0x7f888c019820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:20:42.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.111+0000 7f889919b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556e7bf2f350 0x556e7bf3fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:42.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.111+0000 7f889919b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556e7bf2f350 0x556e7bf3fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41760/0 (socket says 172.21.15.101:41760) 2023-07-18T03:20:42.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.111+0000 7f889919b700 1 -- 172.21.15.101:0/3537241616 learned_addr learned my addr 172.21.15.101:0/3537241616 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:42.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.111+0000 7f889899a700 1 --2- 172.21.15.101:0/3537241616 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e7bf2fc60 0x556e7bf40260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:42.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.111+0000 7f889999c700 1 --2- 172.21.15.101:0/3537241616 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556e7bf3ab50 0x556e7bf407a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:42.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.111+0000 7f889919b700 1 -- 172.21.15.101:0/3537241616 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e7bf2fc60 msgr2=0x556e7bf40260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:42.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.111+0000 7f889919b700 1 --2- 172.21.15.101:0/3537241616 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e7bf2fc60 0x556e7bf40260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.111+0000 7f889919b700 1 -- 172.21.15.101:0/3537241616 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556e7bf3ab50 msgr2=0x556e7bf407a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:42.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.111+0000 7f889919b700 1 --2- 172.21.15.101:0/3537241616 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556e7bf3ab50 0x556e7bf407a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.111+0000 7f889919b700 1 -- 172.21.15.101:0/3537241616 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f889419d400 con 0x556e7bf2f350 2023-07-18T03:20:42.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.112+0000 7f889919b700 1 --2- 172.21.15.101:0/3537241616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556e7bf2f350 0x556e7bf3fd20 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f889009a830 tx=0x7f8890009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b953e5d280ddf1a3 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:42.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.112+0000 7f887cff9700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:20:42.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.112+0000 7f887cff9700 1 -- 172.21.15.101:0/3537241616 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88900ba2e0 con 0x556e7bf2f350 2023-07-18T03:20:42.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.112+0000 7f887cff9700 1 -- 172.21.15.101:0/3537241616 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f88900b8e40 con 0x556e7bf2f350 2023-07-18T03:20:42.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.112+0000 7f88aed07b80 1 -- 172.21.15.101:0/3537241616 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8884004510 con 0x556e7bf2f350 2023-07-18T03:20:42.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.112+0000 7f887cff9700 1 -- 172.21.15.101:0/3537241616 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f889009e8d0 con 0x556e7bf2f350 2023-07-18T03:20:42.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.112+0000 7f88aed07b80 10 client.28297 fetch_fsmap learned FSMap version 35 2023-07-18T03:20:42.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.112+0000 7f88aed07b80 1 -- 172.21.15.101:0/3537241616 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x556e7bf27660 con 0x556e7bf2f350 2023-07-18T03:20:42.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.112+0000 7f887cff9700 1 -- 172.21.15.101:0/3537241616 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8884004510 con 0x556e7bf2f350 2023-07-18T03:20:42.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.113+0000 7f887cff9700 1 -- 172.21.15.101:0/3537241616 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 35) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f8890099eb0 con 0x556e7bf2f350 2023-07-18T03:20:42.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.113+0000 7f88aed07b80 10 client.28297 fetch_fsmap finished waiting for FSMap version 35 2023-07-18T03:20:42.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.113+0000 7f88aed07b80 10 client.28297 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:20:42.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.113+0000 7f88aed07b80 20 client.28297 populate_metadata read hostname 'smithi101' 2023-07-18T03:20:42.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.113+0000 7f88aed07b80 1 --2- 172.21.15.101:0/3537241616 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x556e7bf9f850 0x556e7bfa1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:42.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.113+0000 7f88aed07b80 4 client.28297 mds_command: new command op to 24562 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-18T03:20:42.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.113+0000 7f88aed07b80 1 -- 172.21.15.101:0/3537241616 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x556e7be3d0d0 con 0x556e7bf9f850 2023-07-18T03:20:42.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.113+0000 7f889899a700 1 --2- 172.21.15.101:0/3537241616 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x556e7bf9f850 0x556e7bfa1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:42.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.113+0000 7f889899a700 1 --2- 172.21.15.101:0/3537241616 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x556e7bf9f850 0x556e7bfa1c30 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2eb0b6edecb15dfd server_cookie=7bac0887c819d66a in_seq=0 out_seq=0 2023-07-18T03:20:42.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.113+0000 7f887cff9700 10 client.28297 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:20:42.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.116+0000 7f887cff9700 1 -- 172.21.15.101:0/3537241616 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x556e7be3d0d0 con 0x556e7bf9f850 2023-07-18T03:20:42.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.116+0000 7f887cff9700 10 client.28297 handle_command_reply: tid=0 2023-07-18T03:20:42.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.116+0000 7f88aed07b80 1 client.28297 shutdown 2023-07-18T03:20:42.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.116+0000 7f88aed07b80 1 -- 172.21.15.101:0/3537241616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556e7bf2f350 msgr2=0x556e7bf3fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:42.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.116+0000 7f88aed07b80 1 --2- 172.21.15.101:0/3537241616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556e7bf2f350 0x556e7bf3fd20 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f889009a830 tx=0x7f8890009310 comp rx=0 tx=0).stop 2023-07-18T03:20:42.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.116+0000 7f88aed07b80 1 -- 172.21.15.101:0/3537241616 shutdown_connections 2023-07-18T03:20:42.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.116+0000 7f88aed07b80 1 --2- 172.21.15.101:0/3537241616 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x556e7bf9f850 0x556e7bfa1c30 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.116+0000 7f88aed07b80 1 --2- 172.21.15.101:0/3537241616 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556e7bf3ab50 0x556e7bf407a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.116+0000 7f887cff9700 0 client.28297 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:20:42.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.116+0000 7f88aed07b80 1 --2- 172.21.15.101:0/3537241616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556e7bf2f350 0x556e7bf3fd20 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.116+0000 7f88aed07b80 1 --2- 172.21.15.101:0/3537241616 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e7bf2fc60 0x556e7bf40260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.116+0000 7f88aed07b80 1 -- 172.21.15.101:0/3537241616 >> 172.21.15.101:0/3537241616 conn(0x556e7bf25e20 msgr2=0x556e7bf31360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:42.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.117+0000 7f88aed07b80 1 -- 172.21.15.101:0/3537241616 shutdown_connections 2023-07-18T03:20:42.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.117+0000 7f88aed07b80 1 -- 172.21.15.101:0/3537241616 wait complete. 2023-07-18T03:20:42.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.117+0000 7f88aed07b80 20 client.28297 trim_cache size 0 max 16384 2023-07-18T03:20:42.119 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:20:42.119 INFO:teuthology.orchestra.run.smithi101.stdout: "return_code": 0, 2023-07-18T03:20:42.119 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_tag": "4aad1c81-c135-4232-a1f3-51d35baa9c83", 2023-07-18T03:20:42.119 INFO:teuthology.orchestra.run.smithi101.stdout: "mode": "asynchronous" 2023-07-18T03:20:42.120 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:20:42.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.117+0000 7f889b3ff700 1 -- 172.21.15.101:0/2875519176 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f88800a1340 msgr2=0x7f88800a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:42.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.117+0000 7f889b3ff700 1 --2- 172.21.15.101:0/2875519176 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f88800a1340 0x7f88800a37f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f888c009480 tx=0x7f888c004ad0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.117+0000 7f889b3ff700 1 -- 172.21.15.101:0/2875519176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8894101f70 msgr2=0x7f8894197e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:42.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.117+0000 7f889b3ff700 1 --2- 172.21.15.101:0/2875519176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8894101f70 0x7f8894197e70 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f88900098e0 tx=0x7f889000f690 comp rx=0 tx=0).stop 2023-07-18T03:20:42.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.117+0000 7f889b3ff700 1 -- 172.21.15.101:0/2875519176 shutdown_connections 2023-07-18T03:20:42.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.117+0000 7f889b3ff700 1 --2- 172.21.15.101:0/2875519176 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8894106fb0 0x7f889419c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.117+0000 7f889b3ff700 1 --2- 172.21.15.101:0/2875519176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8894101f70 0x7f8894197e70 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.117+0000 7f889b3ff700 1 --2- 172.21.15.101:0/2875519176 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f88800a1340 0x7f88800a37f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.117+0000 7f889b3ff700 1 --2- 172.21.15.101:0/2875519176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8894102910 0x7f88941983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.117+0000 7f889b3ff700 1 -- 172.21.15.101:0/2875519176 >> 172.21.15.101:0/2875519176 conn(0x7f88940fd7e0 msgr2=0x7f88941058b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:42.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.118+0000 7f889b3ff700 1 -- 172.21.15.101:0/2875519176 shutdown_connections 2023-07-18T03:20:42.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.118+0000 7f889b3ff700 1 -- 172.21.15.101:0/2875519176 wait complete. 2023-07-18T03:20:42.133 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:20:42.349 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.353+0000 7fd97c5ff700 1 -- 172.21.15.101:0/3475514468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9741015b0 msgr2=0x7fd974109aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:42.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.353+0000 7fd97c5ff700 1 --2- 172.21.15.101:0/3475514468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9741015b0 0x7fd974109aa0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd964009990 tx=0x7fd964009ca0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.353+0000 7fd97c5ff700 1 -- 172.21.15.101:0/3475514468 shutdown_connections 2023-07-18T03:20:42.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.353+0000 7fd97c5ff700 1 --2- 172.21.15.101:0/3475514468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd97410a070 0x7fd97410c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.353+0000 7fd97c5ff700 1 --2- 172.21.15.101:0/3475514468 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd974100ca0 0x7fd974101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.353+0000 7fd97c5ff700 1 --2- 172.21.15.101:0/3475514468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9741015b0 0x7fd974109aa0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.353+0000 7fd97c5ff700 1 -- 172.21.15.101:0/3475514468 >> 172.21.15.101:0/3475514468 conn(0x7fd9740fa510 msgr2=0x7fd9740fc920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:42.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.353+0000 7fd97c5ff700 1 -- 172.21.15.101:0/3475514468 shutdown_connections 2023-07-18T03:20:42.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.353+0000 7fd97c5ff700 1 -- 172.21.15.101:0/3475514468 wait complete. 2023-07-18T03:20:42.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.355+0000 7fd97c5ff700 1 Processor -- start 2023-07-18T03:20:42.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.355+0000 7fd97c5ff700 1 -- start start 2023-07-18T03:20:42.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.356+0000 7fd97c5ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd974100ca0 0x7fd97419b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:42.355 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.356+0000 7fd97c5ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd9741015b0 0x7fd97419bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:42.355 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.356+0000 7fd97c5ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd97410a070 0x7fd974195a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:42.355 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.356+0000 7fd97c5ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9741960c0 con 0x7fd9741015b0 2023-07-18T03:20:42.355 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.356+0000 7fd97c5ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd974196230 con 0x7fd97410a070 2023-07-18T03:20:42.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.356+0000 7fd97c5ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9741963f0 con 0x7fd974100ca0 2023-07-18T03:20:42.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.356+0000 7fd97a39b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd974100ca0 0x7fd97419b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:42.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.356+0000 7fd97a39b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd974100ca0 0x7fd97419b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41780/0 (socket says 172.21.15.101:41780) 2023-07-18T03:20:42.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.356+0000 7fd97a39b700 1 -- 172.21.15.101:0/3883004745 learned_addr learned my addr 172.21.15.101:0/3883004745 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:42.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.356+0000 7fd979b9a700 1 --2- 172.21.15.101:0/3883004745 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd9741015b0 0x7fd97419bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:42.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.356+0000 7fd97ab9c700 1 --2- 172.21.15.101:0/3883004745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd97410a070 0x7fd974195a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:42.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.356+0000 7fd97a39b700 1 -- 172.21.15.101:0/3883004745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd97410a070 msgr2=0x7fd974195a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:42.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.356+0000 7fd97a39b700 1 --2- 172.21.15.101:0/3883004745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd97410a070 0x7fd974195a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.356+0000 7fd97a39b700 1 -- 172.21.15.101:0/3883004745 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd9741015b0 msgr2=0x7fd97419bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:42.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.356+0000 7fd97a39b700 1 --2- 172.21.15.101:0/3883004745 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd9741015b0 0x7fd97419bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.356+0000 7fd97a39b700 1 -- 172.21.15.101:0/3883004745 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9640095f0 con 0x7fd974100ca0 2023-07-18T03:20:42.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.357+0000 7fd97a39b700 1 --2- 172.21.15.101:0/3883004745 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd974100ca0 0x7fd97419b9e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd97000e9a0 tx=0x7fd97000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:42.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.357+0000 7fd96b7fe700 1 -- 172.21.15.101:0/3883004745 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd970009790 con 0x7fd974100ca0 2023-07-18T03:20:42.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.357+0000 7fd97c5ff700 1 -- 172.21.15.101:0/3883004745 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9741966d0 con 0x7fd974100ca0 2023-07-18T03:20:42.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.357+0000 7fd96b7fe700 1 -- 172.21.15.101:0/3883004745 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd970010430 con 0x7fd974100ca0 2023-07-18T03:20:42.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.357+0000 7fd96b7fe700 1 -- 172.21.15.101:0/3883004745 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd970018680 con 0x7fd974100ca0 2023-07-18T03:20:42.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.357+0000 7fd97c5ff700 1 -- 172.21.15.101:0/3883004745 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9741a29d0 con 0x7fd974100ca0 2023-07-18T03:20:42.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.360+0000 7fd96b7fe700 1 -- 172.21.15.101:0/3883004745 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fd9700105a0 con 0x7fd974100ca0 2023-07-18T03:20:42.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.362+0000 7fd98ffc4b80 1 Processor -- start 2023-07-18T03:20:42.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.362+0000 7fd98ffc4b80 1 -- start start 2023-07-18T03:20:42.365 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.363+0000 7fd98ffc4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a3318e2d0 0x558a3318e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:42.365 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.363+0000 7fd98ffc4b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a3318ebe0 0x558a3319d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:42.365 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.363+0000 7fd98ffc4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a3319dee0 0x558a331a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:42.365 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.363+0000 7fd98ffc4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a33094cb0 con 0x558a3318e2d0 2023-07-18T03:20:42.365 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.363+0000 7fd98ffc4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a330e7a40 con 0x558a3319dee0 2023-07-18T03:20:42.365 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.363+0000 7fd98ffc4b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a330e0ed0 con 0x558a3318ebe0 2023-07-18T03:20:42.365 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.363+0000 7fd96b7fe700 1 --2- 172.21.15.101:0/3883004745 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd9600a12c0 0x7fd9600a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:42.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.363+0000 7fd97a39b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a3318ebe0 0x558a3319d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:42.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.363+0000 7fd979b9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a3319dee0 0x558a331a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:42.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.363+0000 7fd979b9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a3319dee0 0x558a331a02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:32778/0 (socket says 172.21.15.101:32778) 2023-07-18T03:20:42.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.363+0000 7fd979b9a700 1 -- 172.21.15.101:0/4053967710 learned_addr learned my addr 172.21.15.101:0/4053967710 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:42.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.363+0000 7fd97ab9c700 1 --2- 172.21.15.101:0/4053967710 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a3318e2d0 0x558a3318e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:42.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.363+0000 7fd96b7fe700 1 -- 172.21.15.101:0/3883004745 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fd970014070 con 0x7fd974100ca0 2023-07-18T03:20:42.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.363+0000 7fd97ab9c700 1 --2- 172.21.15.101:0/3883004745 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd9600a12c0 0x7fd9600a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:42.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.363+0000 7fd979b9a700 1 -- 172.21.15.101:0/4053967710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a3318ebe0 msgr2=0x558a3319d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:42.367 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.363+0000 7fd979b9a700 1 --2- 172.21.15.101:0/4053967710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a3318ebe0 0x558a3319d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.367 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.363+0000 7fd979b9a700 1 -- 172.21.15.101:0/4053967710 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a3318e2d0 msgr2=0x558a3318e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:42.367 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.363+0000 7fd979b9a700 1 --2- 172.21.15.101:0/4053967710 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a3318e2d0 0x558a3318e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.367 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.363+0000 7fd979b9a700 1 -- 172.21.15.101:0/4053967710 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a32f0ede0 con 0x558a3319dee0 2023-07-18T03:20:42.367 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.364+0000 7fd979b9a700 1 --2- 172.21.15.101:0/4053967710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a3319dee0 0x558a331a02c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd964015000 tx=0x7fd964005ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ff001232bf6f82e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:42.367 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.364+0000 7fd97ab9c700 1 --2- 172.21.15.101:0/3883004745 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd9600a12c0 0x7fd9600a3770 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd96c009ec0 tx=0x7fd96c00b040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:42.367 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.364+0000 7fd968bfe700 1 -- 172.21.15.101:0/4053967710 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96401d070 con 0x558a3319dee0 2023-07-18T03:20:42.368 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.364+0000 7fd968bfe700 1 -- 172.21.15.101:0/4053967710 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd96401e040 con 0x558a3319dee0 2023-07-18T03:20:42.368 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.364+0000 7fd968bfe700 1 -- 172.21.15.101:0/4053967710 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9640048f0 con 0x558a3319dee0 2023-07-18T03:20:42.368 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.365+0000 7fd98ffc4b80 1 -- 172.21.15.101:0/4053967710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a3319dee0 msgr2=0x558a331a02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:42.368 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.365+0000 7fd98ffc4b80 1 --2- 172.21.15.101:0/4053967710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a3319dee0 0x558a331a02c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd964015000 tx=0x7fd964005ea0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.368 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.365+0000 7fd98ffc4b80 1 -- 172.21.15.101:0/4053967710 shutdown_connections 2023-07-18T03:20:42.368 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.365+0000 7fd98ffc4b80 1 --2- 172.21.15.101:0/4053967710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a3319dee0 0x558a331a02c0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.368 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.365+0000 7fd98ffc4b80 1 --2- 172.21.15.101:0/4053967710 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a3318e2d0 0x558a3318e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.369 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.365+0000 7fd98ffc4b80 1 --2- 172.21.15.101:0/4053967710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a3318ebe0 0x558a3319d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.369 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.365+0000 7fd98ffc4b80 1 -- 172.21.15.101:0/4053967710 >> 172.21.15.101:0/4053967710 conn(0x558a33184da0 msgr2=0x558a3318a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:42.369 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.365+0000 7fd98ffc4b80 1 -- 172.21.15.101:0/4053967710 shutdown_connections 2023-07-18T03:20:42.369 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.365+0000 7fd98ffc4b80 1 -- 172.21.15.101:0/4053967710 wait complete. 2023-07-18T03:20:42.369 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.366+0000 7fd98ffc4b80 1 -- start start 2023-07-18T03:20:42.369 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.366+0000 7fd98ffc4b80 1 Processor -- start 2023-07-18T03:20:42.369 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.366+0000 7fd98ffc4b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a3318e2d0 0x558a3319d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:42.370 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.366+0000 7fd98ffc4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a3318ebe0 0x558a3319ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:42.370 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.366+0000 7fd98ffc4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a3319dee0 0x558a3319b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:42.370 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.366+0000 7fd98ffc4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a33080ac0 con 0x558a3319dee0 2023-07-18T03:20:42.370 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.366+0000 7fd98ffc4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a330807c0 con 0x558a3318ebe0 2023-07-18T03:20:42.370 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.366+0000 7fd98ffc4b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a32f64920 con 0x558a3318e2d0 2023-07-18T03:20:42.370 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.366+0000 7fd98ffc4b80 10 client.0 _ll_register_callbacks cb 0x558a3307e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:20:42.370 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.367+0000 7fd97a39b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a3318e2d0 0x558a3319d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:42.370 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.367+0000 7fd97a39b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a3318e2d0 0x558a3319d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41802/0 (socket says 172.21.15.101:41802) 2023-07-18T03:20:42.371 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.367+0000 7fd97a39b700 1 -- 172.21.15.101:0/748894544 learned_addr learned my addr 172.21.15.101:0/748894544 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:42.371 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.367+0000 7fd97ab9c700 1 --2- 172.21.15.101:0/748894544 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a3319dee0 0x558a3319b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:42.371 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.367+0000 7fd979b9a700 1 --2- 172.21.15.101:0/748894544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a3318ebe0 0x558a3319ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:42.371 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.367+0000 7fd97a39b700 1 -- 172.21.15.101:0/748894544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a3318ebe0 msgr2=0x558a3319ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:42.371 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.367+0000 7fd97a39b700 1 --2- 172.21.15.101:0/748894544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a3318ebe0 0x558a3319ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.371 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.367+0000 7fd97a39b700 1 -- 172.21.15.101:0/748894544 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a3319dee0 msgr2=0x558a3319b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:42.371 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.367+0000 7fd97a39b700 1 --2- 172.21.15.101:0/748894544 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a3319dee0 0x558a3319b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.372 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.367+0000 7fd97a39b700 1 -- 172.21.15.101:0/748894544 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd97410d170 con 0x558a3318e2d0 2023-07-18T03:20:42.372 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.367+0000 7fd97ab9c700 1 --2- 172.21.15.101:0/748894544 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a3319dee0 0x558a3319b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:20:42.372 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.367+0000 7fd97a39b700 1 --2- 172.21.15.101:0/748894544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a3318e2d0 0x558a3319d860 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd970096a20 tx=0x7fd9700cab90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a837f066c032aade server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:42.372 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.367+0000 7fd95dffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:20:42.372 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.367+0000 7fd95dffb700 1 -- 172.21.15.101:0/748894544 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97009a410 con 0x558a3318e2d0 2023-07-18T03:20:42.372 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.367+0000 7fd98ffc4b80 1 -- 172.21.15.101:0/748894544 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558a3303a4f0 con 0x558a3318e2d0 2023-07-18T03:20:42.372 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.367+0000 7fd95dffb700 1 -- 172.21.15.101:0/748894544 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fd97009dfe0 con 0x558a3318e2d0 2023-07-18T03:20:42.373 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.368+0000 7fd95dffb700 1 -- 172.21.15.101:0/748894544 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9700a2400 con 0x558a3318e2d0 2023-07-18T03:20:42.373 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.368+0000 7fd98ffc4b80 10 client.28309 fetch_fsmap learned FSMap version 35 2023-07-18T03:20:42.373 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.368+0000 7fd98ffc4b80 1 -- 172.21.15.101:0/748894544 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x558a32ef7c60 con 0x558a3318e2d0 2023-07-18T03:20:42.373 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.368+0000 7fd95dffb700 1 -- 172.21.15.101:0/748894544 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558a3303a4f0 con 0x558a3318e2d0 2023-07-18T03:20:42.373 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.368+0000 7fd95dffb700 1 -- 172.21.15.101:0/748894544 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 35) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fd9700995c0 con 0x558a3318e2d0 2023-07-18T03:20:42.373 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.368+0000 7fd98ffc4b80 10 client.28309 fetch_fsmap finished waiting for FSMap version 35 2023-07-18T03:20:42.373 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.368+0000 7fd98ffc4b80 10 client.28309 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:20:42.373 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.368+0000 7fd98ffc4b80 20 client.28309 populate_metadata read hostname 'smithi101' 2023-07-18T03:20:42.374 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.368+0000 7fd98ffc4b80 1 --2- 172.21.15.101:0/748894544 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x558a331ad6d0 0x558a331afab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:42.374 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.368+0000 7fd98ffc4b80 4 client.28309 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:20:42.374 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.368+0000 7fd98ffc4b80 1 -- 172.21.15.101:0/748894544 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558a330d9eb0 con 0x558a331ad6d0 2023-07-18T03:20:42.375 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.369+0000 7fd979b9a700 1 --2- 172.21.15.101:0/748894544 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x558a331ad6d0 0x558a331afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:42.375 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.369+0000 7fd979b9a700 1 --2- 172.21.15.101:0/748894544 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x558a331ad6d0 0x558a331afab0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dde531dc6cd757f server_cookie=130448c3b079682b in_seq=0 out_seq=0 2023-07-18T03:20:42.375 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.369+0000 7fd95dffb700 10 client.28309 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:20:42.375 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.371+0000 7fd95dffb700 1 -- 172.21.15.101:0/748894544 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558a330d9eb0 con 0x558a331ad6d0 2023-07-18T03:20:42.375 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.371+0000 7fd95dffb700 10 client.28309 handle_command_reply: tid=0 2023-07-18T03:20:42.376 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.371+0000 7fd98ffc4b80 1 client.28309 shutdown 2023-07-18T03:20:42.376 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.371+0000 7fd98ffc4b80 1 -- 172.21.15.101:0/748894544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a3318e2d0 msgr2=0x558a3319d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:42.376 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.371+0000 7fd98ffc4b80 1 --2- 172.21.15.101:0/748894544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a3318e2d0 0x558a3319d860 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd970096a20 tx=0x7fd9700cab90 comp rx=0 tx=0).stop 2023-07-18T03:20:42.376 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.371+0000 7fd98ffc4b80 1 -- 172.21.15.101:0/748894544 shutdown_connections 2023-07-18T03:20:42.376 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.371+0000 7fd98ffc4b80 1 --2- 172.21.15.101:0/748894544 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x558a331ad6d0 0x558a331afab0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.376 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.371+0000 7fd98ffc4b80 1 --2- 172.21.15.101:0/748894544 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a3319dee0 0x558a3319b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.376 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.371+0000 7fd98ffc4b80 1 --2- 172.21.15.101:0/748894544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a3318e2d0 0x558a3319d860 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.377 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.371+0000 7fd98ffc4b80 1 --2- 172.21.15.101:0/748894544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a3318ebe0 0x558a3319ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.377 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.371+0000 7fd95dffb700 0 client.28309 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:20:42.377 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.371+0000 7fd98ffc4b80 1 -- 172.21.15.101:0/748894544 >> 172.21.15.101:0/748894544 conn(0x558a33184da0 msgr2=0x558a3318a290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:42.377 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.372+0000 7fd98ffc4b80 1 -- 172.21.15.101:0/748894544 shutdown_connections 2023-07-18T03:20:42.377 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.372+0000 7fd98ffc4b80 1 -- 172.21.15.101:0/748894544 wait complete. 2023-07-18T03:20:42.377 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.372+0000 7fd98ffc4b80 20 client.28309 trim_cache size 0 max 16384 2023-07-18T03:20:42.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.389+0000 7fd98ffc4b80 1 Processor -- start 2023-07-18T03:20:42.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.389+0000 7fd98ffc4b80 1 -- start start 2023-07-18T03:20:42.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.390+0000 7fd98ffc4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a33184da0 0x558a331a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:42.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.390+0000 7fd98ffc4b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a3318e2d0 0x558a33195e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:42.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.390+0000 7fd98ffc4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a3318ebe0 0x558a331963a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:42.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.390+0000 7fd98ffc4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a331a9770 con 0x558a33184da0 2023-07-18T03:20:42.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.390+0000 7fd98ffc4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a331990b0 con 0x558a3318ebe0 2023-07-18T03:20:42.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.390+0000 7fd98ffc4b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a33189490 con 0x558a3318e2d0 2023-07-18T03:20:42.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.390+0000 7fd97a39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a33184da0 0x558a331a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:42.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.390+0000 7fd97ab9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a3318ebe0 0x558a331963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:42.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.390+0000 7fd979b9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a3318e2d0 0x558a33195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:42.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.390+0000 7fd97ab9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a3318ebe0 0x558a331963a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:32806/0 (socket says 172.21.15.101:32806) 2023-07-18T03:20:42.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.390+0000 7fd979b9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a3318e2d0 0x558a33195e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41810/0 (socket says 172.21.15.101:41810) 2023-07-18T03:20:42.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.390+0000 7fd97ab9c700 1 -- 172.21.15.101:0/1927503901 learned_addr learned my addr 172.21.15.101:0/1927503901 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:42.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.391+0000 7fd97ab9c700 1 -- 172.21.15.101:0/1927503901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a3318e2d0 msgr2=0x558a33195e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:42.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.391+0000 7fd97ab9c700 1 --2- 172.21.15.101:0/1927503901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a3318e2d0 0x558a33195e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.391+0000 7fd97ab9c700 1 -- 172.21.15.101:0/1927503901 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a33184da0 msgr2=0x558a331a48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:42.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.391+0000 7fd97ab9c700 1 --2- 172.21.15.101:0/1927503901 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a33184da0 0x558a331a48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.391+0000 7fd97ab9c700 1 -- 172.21.15.101:0/1927503901 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a32f35250 con 0x558a3318ebe0 2023-07-18T03:20:42.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.391+0000 7fd97a39b700 1 --2- 172.21.15.101:0/1927503901 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a33184da0 0x558a331a48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:20:42.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.391+0000 7fd979b9a700 1 --2- 172.21.15.101:0/1927503901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a3318e2d0 0x558a33195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:20:42.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.391+0000 7fd97ab9c700 1 --2- 172.21.15.101:0/1927503901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a3318ebe0 0x558a331963a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd96c0185a0 tx=0x7fd96c011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cec4d6097e701fd4 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:42.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.391+0000 7fd9693ff700 1 -- 172.21.15.101:0/1927503901 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c00e070 con 0x558a3318ebe0 2023-07-18T03:20:42.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.391+0000 7fd9693ff700 1 -- 172.21.15.101:0/1927503901 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd96c00f040 con 0x558a3318ebe0 2023-07-18T03:20:42.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.391+0000 7fd9693ff700 1 -- 172.21.15.101:0/1927503901 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c023560 con 0x558a3318ebe0 2023-07-18T03:20:42.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.392+0000 7fd98ffc4b80 1 -- 172.21.15.101:0/1927503901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a3318ebe0 msgr2=0x558a331963a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:42.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.392+0000 7fd98ffc4b80 1 --2- 172.21.15.101:0/1927503901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a3318ebe0 0x558a331963a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd96c0185a0 tx=0x7fd96c011000 comp rx=0 tx=0).stop 2023-07-18T03:20:42.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.392+0000 7fd98ffc4b80 1 -- 172.21.15.101:0/1927503901 shutdown_connections 2023-07-18T03:20:42.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.392+0000 7fd98ffc4b80 1 --2- 172.21.15.101:0/1927503901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a3318ebe0 0x558a331963a0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.392+0000 7fd98ffc4b80 1 --2- 172.21.15.101:0/1927503901 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a33184da0 0x558a331a48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.392+0000 7fd98ffc4b80 1 --2- 172.21.15.101:0/1927503901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a3318e2d0 0x558a33195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.392+0000 7fd98ffc4b80 1 -- 172.21.15.101:0/1927503901 >> 172.21.15.101:0/1927503901 conn(0x558a331a0430 msgr2=0x558a331a9060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:42.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.392+0000 7fd98ffc4b80 1 -- 172.21.15.101:0/1927503901 shutdown_connections 2023-07-18T03:20:42.394 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.392+0000 7fd98ffc4b80 1 -- 172.21.15.101:0/1927503901 wait complete. 2023-07-18T03:20:42.394 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.392+0000 7fd98ffc4b80 1 -- start start 2023-07-18T03:20:42.394 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.393+0000 7fd98ffc4b80 1 Processor -- start 2023-07-18T03:20:42.394 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.393+0000 7fd98ffc4b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a33184da0 0x558a331a34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:42.395 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.393+0000 7fd98ffc4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a3318e2d0 0x558a331a3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:42.395 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.393+0000 7fd98ffc4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a3318ebe0 0x558a33201570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:42.395 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.393+0000 7fd98ffc4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a33201ab0 con 0x558a3318e2d0 2023-07-18T03:20:42.395 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.393+0000 7fd98ffc4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a33201bf0 con 0x558a3318ebe0 2023-07-18T03:20:42.395 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.393+0000 7fd98ffc4b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a33201d60 con 0x558a33184da0 2023-07-18T03:20:42.395 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.393+0000 7fd98ffc4b80 10 client.0 _ll_register_callbacks cb 0x7fd970099af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:20:42.395 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.393+0000 7fd979b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a3318e2d0 0x558a331a3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:42.395 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.393+0000 7fd979b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a3318e2d0 0x558a331a3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:57286/0 (socket says 172.21.15.101:57286) 2023-07-18T03:20:42.396 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.393+0000 7fd979b9a700 1 -- 172.21.15.101:0/2382154111 learned_addr learned my addr 172.21.15.101:0/2382154111 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:20:42.396 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.393+0000 7fd97a39b700 1 --2- 172.21.15.101:0/2382154111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a33184da0 0x558a331a34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:42.396 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.393+0000 7fd97ab9c700 1 --2- 172.21.15.101:0/2382154111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a3318ebe0 0x558a33201570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:42.396 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.394+0000 7fd979b9a700 1 -- 172.21.15.101:0/2382154111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a33184da0 msgr2=0x558a331a34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:42.396 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.394+0000 7fd979b9a700 1 --2- 172.21.15.101:0/2382154111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a33184da0 0x558a331a34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.396 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.394+0000 7fd979b9a700 1 -- 172.21.15.101:0/2382154111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a3318ebe0 msgr2=0x558a33201570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:20:42.396 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.394+0000 7fd979b9a700 1 --2- 172.21.15.101:0/2382154111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a3318ebe0 0x558a33201570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.394+0000 7fd979b9a700 1 -- 172.21.15.101:0/2382154111 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558a32ef7c60 con 0x558a3318e2d0 2023-07-18T03:20:42.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.394+0000 7fd97ab9c700 1 --2- 172.21.15.101:0/2382154111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a3318ebe0 0x558a33201570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:20:42.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.394+0000 7fd979b9a700 1 --2- 172.21.15.101:0/2382154111 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a3318e2d0 0x558a331a3a00 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd964000c00 tx=0x7fd964005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=476de98e644abe8f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:20:42.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.394+0000 7fd95dffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:20:42.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.394+0000 7fd95dffb700 1 -- 172.21.15.101:0/2382154111 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd964031040 con 0x558a3318e2d0 2023-07-18T03:20:42.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.394+0000 7fd95dffb700 1 -- 172.21.15.101:0/2382154111 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fd964033860 con 0x558a3318e2d0 2023-07-18T03:20:42.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.394+0000 7fd98ffc4b80 1 -- 172.21.15.101:0/2382154111 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd96401e040 con 0x558a3318e2d0 2023-07-18T03:20:42.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.394+0000 7fd95dffb700 1 -- 172.21.15.101:0/2382154111 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd96401d040 con 0x558a3318e2d0 2023-07-18T03:20:42.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.394+0000 7fd98ffc4b80 10 client.18237 fetch_fsmap learned FSMap version 35 2023-07-18T03:20:42.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.395+0000 7fd98ffc4b80 1 -- 172.21.15.101:0/2382154111 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x558a32f0ede0 con 0x558a3318e2d0 2023-07-18T03:20:42.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.395+0000 7fd95dffb700 1 -- 172.21.15.101:0/2382154111 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd96401e040 con 0x558a3318e2d0 2023-07-18T03:20:42.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.395+0000 7fd95dffb700 1 -- 172.21.15.101:0/2382154111 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 35) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fd964004500 con 0x558a3318e2d0 2023-07-18T03:20:42.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.395+0000 7fd98ffc4b80 10 client.18237 fetch_fsmap finished waiting for FSMap version 35 2023-07-18T03:20:42.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.395+0000 7fd98ffc4b80 10 client.18237 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:20:42.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.395+0000 7fd98ffc4b80 20 client.18237 populate_metadata read hostname 'smithi101' 2023-07-18T03:20:42.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.395+0000 7fd98ffc4b80 1 --2- 172.21.15.101:0/2382154111 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x558a3319dee0 0x558a33205100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:20:42.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.395+0000 7fd98ffc4b80 4 client.18237 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:20:42.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.395+0000 7fd98ffc4b80 1 -- 172.21.15.101:0/2382154111 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558a3309c050 con 0x558a3319dee0 2023-07-18T03:20:42.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.395+0000 7fd97a39b700 1 --2- 172.21.15.101:0/2382154111 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x558a3319dee0 0x558a33205100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:20:42.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.396+0000 7fd97a39b700 1 --2- 172.21.15.101:0/2382154111 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x558a3319dee0 0x558a33205100 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa24272f4e8474f3 server_cookie=f0b324253f4b6c60 in_seq=0 out_seq=0 2023-07-18T03:20:42.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.396+0000 7fd95dffb700 10 client.18237 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:20:42.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.402+0000 7fd95dffb700 1 -- 172.21.15.101:0/2382154111 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x558a3309c050 con 0x558a3319dee0 2023-07-18T03:20:42.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.402+0000 7fd95dffb700 10 client.18237 handle_command_reply: tid=0 2023-07-18T03:20:42.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.402+0000 7fd98ffc4b80 1 client.18237 shutdown 2023-07-18T03:20:42.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.402+0000 7fd98ffc4b80 1 -- 172.21.15.101:0/2382154111 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a3318e2d0 msgr2=0x558a331a3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:42.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.402+0000 7fd98ffc4b80 1 --2- 172.21.15.101:0/2382154111 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a3318e2d0 0x558a331a3a00 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd964000c00 tx=0x7fd964005f10 comp rx=0 tx=0).stop 2023-07-18T03:20:42.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.402+0000 7fd98ffc4b80 1 -- 172.21.15.101:0/2382154111 shutdown_connections 2023-07-18T03:20:42.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.402+0000 7fd98ffc4b80 1 --2- 172.21.15.101:0/2382154111 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x558a3319dee0 0x558a33205100 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.402+0000 7fd98ffc4b80 1 --2- 172.21.15.101:0/2382154111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a3318ebe0 0x558a33201570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.402+0000 7fd98ffc4b80 1 --2- 172.21.15.101:0/2382154111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a33184da0 0x558a331a34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.402+0000 7fd98ffc4b80 1 --2- 172.21.15.101:0/2382154111 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a3318e2d0 0x558a331a3a00 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.402+0000 7fd95dffb700 0 client.18237 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:20:42.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.402+0000 7fd98ffc4b80 1 -- 172.21.15.101:0/2382154111 >> 172.21.15.101:0/2382154111 conn(0x558a331a0430 msgr2=0x558a331901a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:42.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.403+0000 7fd98ffc4b80 1 -- 172.21.15.101:0/2382154111 shutdown_connections 2023-07-18T03:20:42.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.403+0000 7fd98ffc4b80 1 -- 172.21.15.101:0/2382154111 wait complete. 2023-07-18T03:20:42.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.403+0000 7fd98ffc4b80 20 client.18237 trim_cache size 0 max 16384 2023-07-18T03:20:42.408 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:20:42.408 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "scrub active (350 inodes in the stack)", 2023-07-18T03:20:42.408 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:20:42.409 INFO:teuthology.orchestra.run.smithi101.stdout: "4aad1c81-c135-4232-a1f3-51d35baa9c83": { 2023-07-18T03:20:42.409 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:20:42.409 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "4aad1c81-c135-4232-a1f3-51d35baa9c83", 2023-07-18T03:20:42.409 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:20:42.409 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:20:42.409 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:20:42.409 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:20:42.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.403+0000 7fd97c5ff700 1 -- 172.21.15.101:0/3883004745 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd9600a12c0 msgr2=0x7fd9600a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:42.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.403+0000 7fd97c5ff700 1 --2- 172.21.15.101:0/3883004745 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd9600a12c0 0x7fd9600a3770 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd96c009ec0 tx=0x7fd96c00b040 comp rx=0 tx=0).stop 2023-07-18T03:20:42.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.403+0000 7fd97c5ff700 1 -- 172.21.15.101:0/3883004745 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd974100ca0 msgr2=0x7fd97419b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:20:42.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.403+0000 7fd97c5ff700 1 --2- 172.21.15.101:0/3883004745 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd974100ca0 0x7fd97419b9e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd97000e9a0 tx=0x7fd97000ecb0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.403+0000 7fd97c5ff700 1 -- 172.21.15.101:0/3883004745 shutdown_connections 2023-07-18T03:20:42.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.403+0000 7fd97c5ff700 1 --2- 172.21.15.101:0/3883004745 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd9600a12c0 0x7fd9600a3770 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.403+0000 7fd97c5ff700 1 --2- 172.21.15.101:0/3883004745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd97410a070 0x7fd974195a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.403+0000 7fd97c5ff700 1 --2- 172.21.15.101:0/3883004745 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd974100ca0 0x7fd97419b9e0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.403+0000 7fd97c5ff700 1 --2- 172.21.15.101:0/3883004745 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd9741015b0 0x7fd97419bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:20:42.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.403+0000 7fd97c5ff700 1 -- 172.21.15.101:0/3883004745 >> 172.21.15.101:0/3883004745 conn(0x7fd9740fa510 msgr2=0x7fd97410c170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:20:42.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.404+0000 7fd97c5ff700 1 -- 172.21.15.101:0/3883004745 shutdown_connections 2023-07-18T03:20:42.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:20:42.404+0000 7fd97c5ff700 1 -- 172.21.15.101:0/3883004745 wait complete. 2023-07-18T03:20:42.418 INFO:tasks.cephfs.filesystem:scrub status for tag:4aad1c81-c135-4232-a1f3-51d35baa9c83 - {'path': '/', 'tag': '4aad1c81-c135-4232-a1f3-51d35baa9c83', 'options': 'recursive,force'} 2023-07-18T03:20:42.950 INFO:tasks.workunit.client.0.smithi101.stdout: 10 3396 52.58 MB/sec warmup 7 sec latency 136.988 ms 2023-07-18T03:20:43.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:43 smithi179 ceph-mon[129229]: pgmap v411: 129 pgs: 129 active+clean; 657 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 164 op/s 2023-07-18T03:20:43.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:43 smithi179 ceph-mon[129229]: scrub queued for path: / 2023-07-18T03:20:43.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:43 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:20:43.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:43 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:20:43.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:43 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:20:43.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:43 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:20:43.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:43 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:20:43.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:43 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:20:43.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:43 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:20:43.454 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:43 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:20:43.454 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:43 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:20:43.454 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:43 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:20:43.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:43 smithi101 ceph-mon[126379]: pgmap v411: 129 pgs: 129 active+clean; 657 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 164 op/s 2023-07-18T03:20:43.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:43 smithi101 ceph-mon[126379]: scrub queued for path: / 2023-07-18T03:20:43.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:43 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:20:43.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:43 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:20:43.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:43 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:20:43.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:43 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:20:43.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:43 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:20:43.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:43 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:20:43.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:43 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:20:43.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:43 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:20:43.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:43 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:20:43.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:43 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:20:43.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:43 smithi155 ceph-mon[128885]: pgmap v411: 129 pgs: 129 active+clean; 657 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 164 op/s 2023-07-18T03:20:43.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:43 smithi155 ceph-mon[128885]: scrub queued for path: / 2023-07-18T03:20:43.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:43 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:20:43.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:43 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:20:43.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:43 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:20:43.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:43 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:20:43.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:43 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:20:43.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:43 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:20:43.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:43 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:20:43.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:43 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:20:43.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:43 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:20:43.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:43 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:20:43.950 INFO:tasks.workunit.client.0.smithi101.stdout: 10 3574 47.55 MB/sec warmup 8 sec latency 223.656 ms 2023-07-18T03:20:44.950 INFO:tasks.workunit.client.0.smithi101.stdout: 10 3774 42.63 MB/sec warmup 9 sec latency 77.848 ms 2023-07-18T03:20:45.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:45 smithi179 ceph-mon[129229]: pgmap v412: 129 pgs: 129 active+clean; 657 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 164 op/s 2023-07-18T03:20:45.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:45 smithi101 ceph-mon[126379]: pgmap v412: 129 pgs: 129 active+clean; 657 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 164 op/s 2023-07-18T03:20:45.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:45 smithi155 ceph-mon[128885]: pgmap v412: 129 pgs: 129 active+clean; 657 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 164 op/s 2023-07-18T03:20:45.950 INFO:tasks.workunit.client.0.smithi101.stdout: 10 4049 39.05 MB/sec warmup 10 sec latency 109.061 ms 2023-07-18T03:20:46.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:46 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:20:46.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:46 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:20:46.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:46 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:20:46.950 INFO:tasks.workunit.client.0.smithi101.stdout: 10 4445 36.87 MB/sec warmup 11 sec latency 116.509 ms 2023-07-18T03:20:47.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:47 smithi179 ceph-mon[129229]: pgmap v413: 129 pgs: 129 active+clean; 743 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 10 MiB/s wr, 303 op/s 2023-07-18T03:20:47.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:47 smithi101 ceph-mon[126379]: pgmap v413: 129 pgs: 129 active+clean; 743 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 10 MiB/s wr, 303 op/s 2023-07-18T03:20:47.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:47 smithi155 ceph-mon[128885]: pgmap v413: 129 pgs: 129 active+clean; 743 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 10 MiB/s wr, 303 op/s 2023-07-18T03:20:47.950 INFO:tasks.workunit.client.0.smithi101.stdout: 10 4688 36.05 MB/sec warmup 12 sec latency 624.645 ms 2023-07-18T03:20:48.950 INFO:tasks.workunit.client.0.smithi101.stdout: 10 5172 33.62 MB/sec warmup 13 sec latency 382.787 ms 2023-07-18T03:20:49.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:49 smithi179 ceph-mon[129229]: pgmap v414: 129 pgs: 129 active+clean; 743 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.8 MiB/s wr, 249 op/s 2023-07-18T03:20:49.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:49 smithi101 ceph-mon[126379]: pgmap v414: 129 pgs: 129 active+clean; 743 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.8 MiB/s wr, 249 op/s 2023-07-18T03:20:49.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:49 smithi155 ceph-mon[128885]: pgmap v414: 129 pgs: 129 active+clean; 743 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.8 MiB/s wr, 249 op/s 2023-07-18T03:20:49.951 INFO:tasks.workunit.client.0.smithi101.stdout: 10 5775 33.28 MB/sec warmup 14 sec latency 119.243 ms 2023-07-18T03:20:50.951 INFO:tasks.workunit.client.0.smithi101.stdout: 10 6419 34.35 MB/sec warmup 15 sec latency 66.682 ms 2023-07-18T03:20:51.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:51 smithi179 ceph-mon[129229]: pgmap v415: 129 pgs: 129 active+clean; 875 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 23 MiB/s wr, 455 op/s 2023-07-18T03:20:51.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:51 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:20:51.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:51 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:20:51.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:51 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:20:51.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:51 smithi101 ceph-mon[126379]: pgmap v415: 129 pgs: 129 active+clean; 875 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 23 MiB/s wr, 455 op/s 2023-07-18T03:20:51.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:51 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:20:51.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:51 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:20:51.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:51 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:20:51.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:51 smithi155 ceph-mon[128885]: pgmap v415: 129 pgs: 129 active+clean; 875 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 23 MiB/s wr, 455 op/s 2023-07-18T03:20:51.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:51 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:20:51.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:51 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:20:51.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:51 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:20:51.951 INFO:tasks.workunit.client.0.smithi101.stdout: 10 6907 35.20 MB/sec warmup 16 sec latency 68.862 ms 2023-07-18T03:20:52.951 INFO:tasks.workunit.client.0.smithi101.stdout: 10 7108 33.59 MB/sec warmup 17 sec latency 62.828 ms 2023-07-18T03:20:53.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:53 smithi179 ceph-mon[129229]: pgmap v416: 129 pgs: 129 active+clean; 875 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 22 MiB/s wr, 344 op/s 2023-07-18T03:20:53.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:53 smithi101 ceph-mon[126379]: pgmap v416: 129 pgs: 129 active+clean; 875 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 22 MiB/s wr, 344 op/s 2023-07-18T03:20:53.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:53 smithi155 ceph-mon[128885]: pgmap v416: 129 pgs: 129 active+clean; 875 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 22 MiB/s wr, 344 op/s 2023-07-18T03:20:53.951 INFO:tasks.workunit.client.0.smithi101.stdout: 10 7276 31.81 MB/sec warmup 18 sec latency 67.242 ms 2023-07-18T03:20:54.951 INFO:tasks.workunit.client.0.smithi101.stdout: 10 7538 30.40 MB/sec warmup 19 sec latency 89.804 ms 2023-07-18T03:20:55.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:55 smithi179 ceph-mon[129229]: pgmap v417: 129 pgs: 129 active+clean; 875 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 22 MiB/s wr, 344 op/s 2023-07-18T03:20:55.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:55 smithi101 ceph-mon[126379]: pgmap v417: 129 pgs: 129 active+clean; 875 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 22 MiB/s wr, 344 op/s 2023-07-18T03:20:55.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:55 smithi155 ceph-mon[128885]: pgmap v417: 129 pgs: 129 active+clean; 875 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 22 MiB/s wr, 344 op/s 2023-07-18T03:20:55.951 INFO:tasks.workunit.client.0.smithi101.stdout: 10 7960 30.03 MB/sec warmup 20 sec latency 77.364 ms 2023-07-18T03:20:56.951 INFO:tasks.workunit.client.0.smithi101.stdout: 10 8537 29.69 MB/sec warmup 21 sec latency 217.771 ms 2023-07-18T03:20:57.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:57 smithi179 ceph-mon[129229]: pgmap v418: 129 pgs: 129 active+clean; 892 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 23 MiB/s wr, 446 op/s 2023-07-18T03:20:57.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:57 smithi101 ceph-mon[126379]: pgmap v418: 129 pgs: 129 active+clean; 892 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 23 MiB/s wr, 446 op/s 2023-07-18T03:20:57.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:57 smithi155 ceph-mon[128885]: pgmap v418: 129 pgs: 129 active+clean; 892 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 23 MiB/s wr, 446 op/s 2023-07-18T03:20:57.951 INFO:tasks.workunit.client.0.smithi101.stdout: 10 9206 29.76 MB/sec warmup 22 sec latency 50.111 ms 2023-07-18T03:20:58.951 INFO:tasks.workunit.client.0.smithi101.stdout: 10 9868 30.96 MB/sec warmup 23 sec latency 119.408 ms 2023-07-18T03:20:59.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:20:59 smithi179 ceph-mon[129229]: pgmap v419: 129 pgs: 129 active+clean; 892 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 15 MiB/s wr, 306 op/s 2023-07-18T03:20:59.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:20:59 smithi101 ceph-mon[126379]: pgmap v419: 129 pgs: 129 active+clean; 892 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 15 MiB/s wr, 306 op/s 2023-07-18T03:20:59.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:20:59 smithi155 ceph-mon[128885]: pgmap v419: 129 pgs: 129 active+clean; 892 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 15 MiB/s wr, 306 op/s 2023-07-18T03:20:59.951 INFO:tasks.workunit.client.0.smithi101.stdout: 10 10266 30.99 MB/sec warmup 24 sec latency 141.510 ms 2023-07-18T03:21:00.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:00 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:21:00.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:00 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:21:00.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:00 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:21:00.951 INFO:tasks.workunit.client.0.smithi101.stdout: 10 10429 29.95 MB/sec warmup 25 sec latency 152.667 ms 2023-07-18T03:21:01.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:01 smithi101 ceph-mon[126379]: pgmap v420: 129 pgs: 129 active+clean; 909 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 16 MiB/s wr, 435 op/s 2023-07-18T03:21:01.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:01 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:21:01.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:01 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:21:01.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:01 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:21:01.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:01 smithi155 ceph-mon[128885]: pgmap v420: 129 pgs: 129 active+clean; 909 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 16 MiB/s wr, 435 op/s 2023-07-18T03:21:01.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:01 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:21:01.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:01 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:21:01.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:01 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:21:01.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:01 smithi179 ceph-mon[129229]: pgmap v420: 129 pgs: 129 active+clean; 909 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 16 MiB/s wr, 435 op/s 2023-07-18T03:21:01.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:01 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:21:01.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:01 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:21:01.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:01 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:21:01.951 INFO:tasks.workunit.client.0.smithi101.stdout: 10 10714 28.92 MB/sec warmup 26 sec latency 207.117 ms 2023-07-18T03:21:02.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:02 smithi101 ceph-mon[126379]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:21:02.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:02 smithi155 ceph-mon[128885]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:21:02.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:02 smithi179 ceph-mon[129229]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:21:02.951 INFO:tasks.workunit.client.0.smithi101.stdout: 10 11003 28.13 MB/sec warmup 27 sec latency 161.489 ms 2023-07-18T03:21:03.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:03 smithi101 ceph-mon[126379]: pgmap v421: 129 pgs: 129 active+clean; 909 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 230 op/s 2023-07-18T03:21:03.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:03 smithi155 ceph-mon[128885]: pgmap v421: 129 pgs: 129 active+clean; 909 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 230 op/s 2023-07-18T03:21:03.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:03 smithi179 ceph-mon[129229]: pgmap v421: 129 pgs: 129 active+clean; 909 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 230 op/s 2023-07-18T03:21:03.951 INFO:tasks.workunit.client.0.smithi101.stdout: 10 11118 27.36 MB/sec warmup 28 sec latency 192.160 ms 2023-07-18T03:21:04.952 INFO:tasks.workunit.client.0.smithi101.stdout: 10 11620 27.37 MB/sec warmup 29 sec latency 171.109 ms 2023-07-18T03:21:05.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:05 smithi101 ceph-mon[126379]: pgmap v422: 129 pgs: 129 active+clean; 909 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 230 op/s 2023-07-18T03:21:05.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:05 smithi155 ceph-mon[128885]: pgmap v422: 129 pgs: 129 active+clean; 909 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 230 op/s 2023-07-18T03:21:05.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:05 smithi179 ceph-mon[129229]: pgmap v422: 129 pgs: 129 active+clean; 909 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 230 op/s 2023-07-18T03:21:05.952 INFO:tasks.workunit.client.0.smithi101.stdout: 10 12453 28.22 MB/sec warmup 30 sec latency 142.859 ms 2023-07-18T03:21:06.952 INFO:tasks.workunit.client.0.smithi101.stdout: 10 13165 28.33 MB/sec warmup 31 sec latency 727.379 ms 2023-07-18T03:21:07.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:07 smithi101 ceph-mon[126379]: pgmap v423: 129 pgs: 129 active+clean; 927 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.4 MiB/s wr, 501 op/s 2023-07-18T03:21:07.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:07 smithi101 ceph-mon[126379]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:21:07.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:07 smithi101 ceph-mon[126379]: osdmap e107: 12 total, 12 up, 12 in 2023-07-18T03:21:07.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:07 smithi155 ceph-mon[128885]: pgmap v423: 129 pgs: 129 active+clean; 927 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.4 MiB/s wr, 501 op/s 2023-07-18T03:21:07.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:07 smithi155 ceph-mon[128885]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:21:07.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:07 smithi155 ceph-mon[128885]: osdmap e107: 12 total, 12 up, 12 in 2023-07-18T03:21:07.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:07 smithi179 ceph-mon[129229]: pgmap v423: 129 pgs: 129 active+clean; 927 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.4 MiB/s wr, 501 op/s 2023-07-18T03:21:07.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:07 smithi179 ceph-mon[129229]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:21:07.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:07 smithi179 ceph-mon[129229]: osdmap e107: 12 total, 12 up, 12 in 2023-07-18T03:21:07.952 INFO:tasks.workunit.client.0.smithi101.stdout: 10 13930 28.94 MB/sec warmup 32 sec latency 1727.419 ms 2023-07-18T03:21:08.952 INFO:tasks.workunit.client.0.smithi101.stdout: 10 14590 29.08 MB/sec warmup 33 sec latency 2727.459 ms 2023-07-18T03:21:09.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:09 smithi101 ceph-mon[126379]: pgmap v425: 129 pgs: 129 active+clean; 927 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.7 MiB/s wr, 479 op/s 2023-07-18T03:21:09.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:09 smithi155 ceph-mon[128885]: pgmap v425: 129 pgs: 129 active+clean; 927 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.7 MiB/s wr, 479 op/s 2023-07-18T03:21:09.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:09 smithi179 ceph-mon[129229]: pgmap v425: 129 pgs: 129 active+clean; 927 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.7 MiB/s wr, 479 op/s 2023-07-18T03:21:09.952 INFO:tasks.workunit.client.0.smithi101.stdout: 10 15144 28.94 MB/sec warmup 34 sec latency 3727.506 ms 2023-07-18T03:21:10.952 INFO:tasks.workunit.client.0.smithi101.stdout: 10 15982 29.26 MB/sec warmup 35 sec latency 4727.558 ms 2023-07-18T03:21:11.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:11 smithi101 ceph-mon[126379]: pgmap v426: 129 pgs: 129 active+clean; 950 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.8 MiB/s wr, 581 op/s 2023-07-18T03:21:11.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:11 smithi155 ceph-mon[128885]: pgmap v426: 129 pgs: 129 active+clean; 950 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.8 MiB/s wr, 581 op/s 2023-07-18T03:21:11.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:11 smithi179 ceph-mon[129229]: pgmap v426: 129 pgs: 129 active+clean; 950 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.8 MiB/s wr, 581 op/s 2023-07-18T03:21:11.952 INFO:tasks.workunit.client.0.smithi101.stdout: 10 16989 30.22 MB/sec warmup 36 sec latency 5727.609 ms 2023-07-18T03:21:12.419 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:21:12.658 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.662+0000 7f9b901ff700 1 -- 172.21.15.101:0/2573310403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b880ffad0 msgr2=0x7f9b880fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:12.658 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.662+0000 7f9b901ff700 1 --2- 172.21.15.101:0/2573310403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b880ffad0 0x7f9b880fff40 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9b780098f0 tx=0x7f9b78009c00 comp rx=0 tx=0).stop 2023-07-18T03:21:12.658 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.662+0000 7f9b901ff700 1 -- 172.21.15.101:0/2573310403 shutdown_connections 2023-07-18T03:21:12.658 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.662+0000 7f9b901ff700 1 --2- 172.21.15.101:0/2573310403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b88105ae0 0x7f9b88105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:12.659 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.662+0000 7f9b901ff700 1 --2- 172.21.15.101:0/2573310403 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9b88100480 0x7f9b8810c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:12.659 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.662+0000 7f9b901ff700 1 --2- 172.21.15.101:0/2573310403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b880ffad0 0x7f9b880fff40 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:12.659 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.662+0000 7f9b901ff700 1 -- 172.21.15.101:0/2573310403 >> 172.21.15.101:0/2573310403 conn(0x7f9b880fb5f0 msgr2=0x7f9b880fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:12.659 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.662+0000 7f9b901ff700 1 -- 172.21.15.101:0/2573310403 shutdown_connections 2023-07-18T03:21:12.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.663+0000 7f9b901ff700 1 -- 172.21.15.101:0/2573310403 wait complete. 2023-07-18T03:21:12.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.664+0000 7f9b901ff700 1 Processor -- start 2023-07-18T03:21:12.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.664+0000 7f9b901ff700 1 -- start start 2023-07-18T03:21:12.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.665+0000 7f9b901ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9b880ffad0 0x7f9b88199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:12.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.665+0000 7f9b901ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b88100480 0x7f9b88199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:12.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.665+0000 7f9b901ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b88105ae0 0x7f9b881938a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:12.662 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.665+0000 7f9b901ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b88193f10 con 0x7f9b880ffad0 2023-07-18T03:21:12.662 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.665+0000 7f9b901ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b88194080 con 0x7f9b88100480 2023-07-18T03:21:12.662 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.665+0000 7f9b901ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b88194240 con 0x7f9b88105ae0 2023-07-18T03:21:12.662 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.665+0000 7f9b8df9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9b880ffad0 0x7f9b88199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:12.662 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.665+0000 7f9b8df9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9b880ffad0 0x7f9b88199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:35890/0 (socket says 172.21.15.101:35890) 2023-07-18T03:21:12.662 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.665+0000 7f9b8df9b700 1 -- 172.21.15.101:0/3983554810 learned_addr learned my addr 172.21.15.101:0/3983554810 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:12.663 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.665+0000 7f9b8d79a700 1 --2- 172.21.15.101:0/3983554810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b88100480 0x7f9b88199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:12.663 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.666+0000 7f9b8df9b700 1 -- 172.21.15.101:0/3983554810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b88105ae0 msgr2=0x7f9b881938a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:12.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.665+0000 7f9b8e79c700 1 --2- 172.21.15.101:0/3983554810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b88105ae0 0x7f9b881938a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:12.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.666+0000 7f9b8df9b700 1 --2- 172.21.15.101:0/3983554810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b88105ae0 0x7f9b881938a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:12.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.666+0000 7f9b8df9b700 1 -- 172.21.15.101:0/3983554810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b88100480 msgr2=0x7f9b88199d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:12.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.666+0000 7f9b8df9b700 1 --2- 172.21.15.101:0/3983554810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b88100480 0x7f9b88199d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:12.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.666+0000 7f9b8df9b700 1 -- 172.21.15.101:0/3983554810 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b78009520 con 0x7f9b880ffad0 2023-07-18T03:21:12.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.666+0000 7f9b8d79a700 1 --2- 172.21.15.101:0/3983554810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b88100480 0x7f9b88199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:21:12.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.666+0000 7f9b8e79c700 1 --2- 172.21.15.101:0/3983554810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b88105ae0 0x7f9b881938a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:21:12.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.666+0000 7f9b8df9b700 1 --2- 172.21.15.101:0/3983554810 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9b880ffad0 0x7f9b88199830 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f9b8400e9c0 tx=0x7f9b8400ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:12.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.666+0000 7f9b7effd700 1 -- 172.21.15.101:0/3983554810 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8400cbd0 con 0x7f9b880ffad0 2023-07-18T03:21:12.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.666+0000 7f9b901ff700 1 -- 172.21.15.101:0/3983554810 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b88194520 con 0x7f9b880ffad0 2023-07-18T03:21:12.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.666+0000 7f9b7effd700 1 -- 172.21.15.101:0/3983554810 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b8400cd30 con 0x7f9b880ffad0 2023-07-18T03:21:12.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.666+0000 7f9b7effd700 1 -- 172.21.15.101:0/3983554810 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b84005590 con 0x7f9b880ffad0 2023-07-18T03:21:12.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.667+0000 7f9b901ff700 1 -- 172.21.15.101:0/3983554810 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b881a0820 con 0x7f9b880ffad0 2023-07-18T03:21:12.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.675+0000 7f9ba3a62b80 1 Processor -- start 2023-07-18T03:21:12.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.675+0000 7f9ba3a62b80 1 -- start start 2023-07-18T03:21:12.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.675+0000 7f9ba3a62b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ade5c92d0 0x556ade5c96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:12.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.675+0000 7f9ba3a62b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ade5c9be0 0x556ade5dcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:12.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.675+0000 7f9b7effd700 1 -- 172.21.15.101:0/3983554810 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f9b84004750 con 0x7f9b880ffad0 2023-07-18T03:21:12.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.675+0000 7f9ba3a62b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ade5dd2e0 0x556ade5df6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:12.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.675+0000 7f9ba3a62b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ade4cfcb0 con 0x556ade5c9be0 2023-07-18T03:21:12.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.675+0000 7f9ba3a62b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ade522a40 con 0x556ade5c92d0 2023-07-18T03:21:12.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.676+0000 7f9ba3a62b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ade51bed0 con 0x556ade5dd2e0 2023-07-18T03:21:12.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.676+0000 7f9b8e79c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ade5c92d0 0x556ade5c96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:12.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.676+0000 7f9b8e79c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ade5c92d0 0x556ade5c96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:59930/0 (socket says 172.21.15.101:59930) 2023-07-18T03:21:12.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.676+0000 7f9b8e79c700 1 -- 172.21.15.101:0/2964180060 learned_addr learned my addr 172.21.15.101:0/2964180060 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:12.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.676+0000 7f9b8d79a700 1 --2- 172.21.15.101:0/2964180060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ade5dd2e0 0x556ade5df6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:12.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.676+0000 7f9b8df9b700 1 --2- 172.21.15.101:0/2964180060 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ade5c9be0 0x556ade5dcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:12.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.676+0000 7f9b8e79c700 1 -- 172.21.15.101:0/2964180060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ade5dd2e0 msgr2=0x556ade5df6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:12.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.676+0000 7f9b8e79c700 1 --2- 172.21.15.101:0/2964180060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ade5dd2e0 0x556ade5df6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:12.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.676+0000 7f9b8e79c700 1 -- 172.21.15.101:0/2964180060 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ade5c9be0 msgr2=0x556ade5dcda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:12.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.676+0000 7f9b8e79c700 1 --2- 172.21.15.101:0/2964180060 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ade5c9be0 0x556ade5dcda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:12.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.676+0000 7f9b8e79c700 1 -- 172.21.15.101:0/2964180060 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ade349de0 con 0x556ade5c92d0 2023-07-18T03:21:12.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.677+0000 7f9b8e79c700 1 --2- 172.21.15.101:0/2964180060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ade5c92d0 0x556ade5c96a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9b80009d20 tx=0x7f9b800077e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=480450ca86984a81 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:12.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.677+0000 7f9b73fff700 1 -- 172.21.15.101:0/2964180060 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b80003c00 con 0x556ade5c92d0 2023-07-18T03:21:12.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.677+0000 7f9b73fff700 1 -- 172.21.15.101:0/2964180060 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b80009e70 con 0x556ade5c92d0 2023-07-18T03:21:12.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.677+0000 7f9b73fff700 1 -- 172.21.15.101:0/2964180060 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b800146f0 con 0x556ade5c92d0 2023-07-18T03:21:12.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.677+0000 7f9b7effd700 1 --2- 172.21.15.101:0/3983554810 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9b740a1340 0x7f9b740a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:12.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.678+0000 7f9ba3a62b80 1 -- 172.21.15.101:0/2964180060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ade5c92d0 msgr2=0x556ade5c96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:12.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.678+0000 7f9ba3a62b80 1 --2- 172.21.15.101:0/2964180060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ade5c92d0 0x556ade5c96a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9b80009d20 tx=0x7f9b800077e0 comp rx=0 tx=0).stop 2023-07-18T03:21:12.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.678+0000 7f9ba3a62b80 1 -- 172.21.15.101:0/2964180060 shutdown_connections 2023-07-18T03:21:12.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.678+0000 7f9ba3a62b80 1 --2- 172.21.15.101:0/2964180060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ade5c92d0 0x556ade5c96a0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:12.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.678+0000 7f9ba3a62b80 1 --2- 172.21.15.101:0/2964180060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ade5dd2e0 0x556ade5df6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:12.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.678+0000 7f9ba3a62b80 1 --2- 172.21.15.101:0/2964180060 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ade5c9be0 0x556ade5dcda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:12.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.678+0000 7f9ba3a62b80 1 -- 172.21.15.101:0/2964180060 >> 172.21.15.101:0/2964180060 conn(0x556ade5bfda0 msgr2=0x556ade5c56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:12.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.678+0000 7f9b8df9b700 1 --2- 172.21.15.101:0/3983554810 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9b740a1340 0x7f9b740a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:12.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.678+0000 7f9ba3a62b80 1 -- 172.21.15.101:0/2964180060 shutdown_connections 2023-07-18T03:21:12.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.678+0000 7f9b7effd700 1 -- 172.21.15.101:0/3983554810 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f9b84014070 con 0x7f9b880ffad0 2023-07-18T03:21:12.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.678+0000 7f9ba3a62b80 1 -- 172.21.15.101:0/2964180060 wait complete. 2023-07-18T03:21:12.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.679+0000 7f9b8df9b700 1 --2- 172.21.15.101:0/3983554810 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9b740a1340 0x7f9b740a37f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9b840ca6c0 tx=0x7f9b840d0940 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:12.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.679+0000 7f9ba3a62b80 1 -- start start 2023-07-18T03:21:12.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.679+0000 7f9ba3a62b80 1 Processor -- start 2023-07-18T03:21:12.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.680+0000 7f9ba3a62b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ade5c92d0 0x556ade5dcd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:12.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.680+0000 7f9ba3a62b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ade5c9be0 0x556ade5d9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:12.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.680+0000 7f9ba3a62b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ade5dd2e0 0x556ade5da6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:12.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.680+0000 7f9ba3a62b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ade4bbac0 con 0x556ade5dd2e0 2023-07-18T03:21:12.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.680+0000 7f9ba3a62b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ade4bb7c0 con 0x556ade5c9be0 2023-07-18T03:21:12.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.680+0000 7f9ba3a62b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ade39f920 con 0x556ade5c92d0 2023-07-18T03:21:12.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.680+0000 7f9ba3a62b80 10 client.0 _ll_register_callbacks cb 0x556ade4b9760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:21:12.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.681+0000 7f9b8e79c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ade5c92d0 0x556ade5dcd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:12.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.681+0000 7f9b8e79c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ade5dd2e0 0x556ade5da6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:12.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.681+0000 7f9b8e79c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ade5dd2e0 0x556ade5da6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:35904/0 (socket says 172.21.15.101:35904) 2023-07-18T03:21:12.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.681+0000 7f9b8e79c700 1 -- 172.21.15.101:0/4227824895 learned_addr learned my addr 172.21.15.101:0/4227824895 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:12.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.682+0000 7f9b8e79c700 1 -- 172.21.15.101:0/4227824895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ade5c92d0 msgr2=0x556ade5dcd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:12.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.682+0000 7f9b8e79c700 1 --2- 172.21.15.101:0/4227824895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ade5c92d0 0x556ade5dcd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:12.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.682+0000 7f9b8e79c700 1 -- 172.21.15.101:0/4227824895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ade5c9be0 msgr2=0x556ade5d9fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-18T03:21:12.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.682+0000 7f9b8e79c700 1 --2- 172.21.15.101:0/4227824895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ade5c9be0 0x556ade5d9fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:12.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.682+0000 7f9b8e79c700 1 -- 172.21.15.101:0/4227824895 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b80009980 con 0x556ade5dd2e0 2023-07-18T03:21:12.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.682+0000 7f9b8e79c700 1 --2- 172.21.15.101:0/4227824895 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ade5dd2e0 0x556ade5da6c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f9b80007b30 tx=0x7f9b80007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39ffea1638c3eccc server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:12.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.683+0000 7f9b717fa700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:21:12.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.683+0000 7f9b717fa700 1 -- 172.21.15.101:0/4227824895 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b80004cd0 con 0x556ade5dd2e0 2023-07-18T03:21:12.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.683+0000 7f9b717fa700 1 -- 172.21.15.101:0/4227824895 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f9b80004e30 con 0x556ade5dd2e0 2023-07-18T03:21:12.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.683+0000 7f9ba3a62b80 1 -- 172.21.15.101:0/4227824895 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556ade4754f0 con 0x556ade5dd2e0 2023-07-18T03:21:12.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.683+0000 7f9b717fa700 1 -- 172.21.15.101:0/4227824895 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b80005000 con 0x556ade5dd2e0 2023-07-18T03:21:12.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.683+0000 7f9ba3a62b80 10 client.18249 fetch_fsmap learned FSMap version 37 2023-07-18T03:21:12.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.683+0000 7f9ba3a62b80 1 -- 172.21.15.101:0/4227824895 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x556ade332c60 con 0x556ade5dd2e0 2023-07-18T03:21:12.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.684+0000 7f9b717fa700 1 -- 172.21.15.101:0/4227824895 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556ade4754f0 con 0x556ade5dd2e0 2023-07-18T03:21:12.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.684+0000 7f9b717fa700 1 -- 172.21.15.101:0/4227824895 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 37) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9b80014600 con 0x556ade5dd2e0 2023-07-18T03:21:12.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.684+0000 7f9ba3a62b80 10 client.18249 fetch_fsmap finished waiting for FSMap version 37 2023-07-18T03:21:12.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.684+0000 7f9ba3a62b80 10 client.18249 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:21:12.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.684+0000 7f9ba3a62b80 20 client.18249 populate_metadata read hostname 'smithi101' 2023-07-18T03:21:12.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.684+0000 7f9ba3a62b80 1 --2- 172.21.15.101:0/4227824895 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x556ade5fdcb0 0x556ade6000c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:12.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.684+0000 7f9ba3a62b80 4 client.18249 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:21:12.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.684+0000 7f9ba3a62b80 1 -- 172.21.15.101:0/4227824895 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556ade514eb0 con 0x556ade5fdcb0 2023-07-18T03:21:12.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.684+0000 7f9b8d79a700 1 --2- 172.21.15.101:0/4227824895 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x556ade5fdcb0 0x556ade6000c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:12.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.685+0000 7f9b717fa700 10 client.18249 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:21:12.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.685+0000 7f9b8d79a700 1 --2- 172.21.15.101:0/4227824895 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x556ade5fdcb0 0x556ade6000c0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d08b331983663846 server_cookie=a24ac3b12bc2d7d3 in_seq=0 out_seq=0 2023-07-18T03:21:12.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.689+0000 7f9b717fa700 1 -- 172.21.15.101:0/4227824895 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556ade514eb0 con 0x556ade5fdcb0 2023-07-18T03:21:12.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.689+0000 7f9b717fa700 10 client.18249 handle_command_reply: tid=0 2023-07-18T03:21:12.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.689+0000 7f9ba3a62b80 1 client.18249 shutdown 2023-07-18T03:21:12.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.689+0000 7f9ba3a62b80 1 -- 172.21.15.101:0/4227824895 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ade5dd2e0 msgr2=0x556ade5da6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:12.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.689+0000 7f9ba3a62b80 1 --2- 172.21.15.101:0/4227824895 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ade5dd2e0 0x556ade5da6c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f9b80007b30 tx=0x7f9b80007f50 comp rx=0 tx=0).stop 2023-07-18T03:21:12.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.689+0000 7f9ba3a62b80 1 -- 172.21.15.101:0/4227824895 shutdown_connections 2023-07-18T03:21:12.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.689+0000 7f9ba3a62b80 1 --2- 172.21.15.101:0/4227824895 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x556ade5fdcb0 0x556ade6000c0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:12.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.689+0000 7f9ba3a62b80 1 --2- 172.21.15.101:0/4227824895 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ade5dd2e0 0x556ade5da6c0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:12.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.689+0000 7f9ba3a62b80 1 --2- 172.21.15.101:0/4227824895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ade5c92d0 0x556ade5dcd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:12.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.689+0000 7f9ba3a62b80 1 --2- 172.21.15.101:0/4227824895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ade5c9be0 0x556ade5d9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:12.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.689+0000 7f9ba3a62b80 1 -- 172.21.15.101:0/4227824895 >> 172.21.15.101:0/4227824895 conn(0x556ade5bfda0 msgr2=0x556ade5c5230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:12.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.689+0000 7f9b717fa700 0 client.18249 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:21:12.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.690+0000 7f9ba3a62b80 1 -- 172.21.15.101:0/4227824895 shutdown_connections 2023-07-18T03:21:12.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.690+0000 7f9ba3a62b80 1 -- 172.21.15.101:0/4227824895 wait complete. 2023-07-18T03:21:12.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.690+0000 7f9ba3a62b80 20 client.18249 trim_cache size 0 max 16384 2023-07-18T03:21:12.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.709+0000 7f9ba3a62b80 1 Processor -- start 2023-07-18T03:21:12.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.709+0000 7f9ba3a62b80 1 -- start start 2023-07-18T03:21:12.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.709+0000 7f9ba3a62b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ade5c92d0 0x556ade5d5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:12.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.709+0000 7f9ba3a62b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ade5c9be0 0x556ade5d58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:12.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.709+0000 7f9ba3a62b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ade5dd2e0 0x556ade5d9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:12.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.709+0000 7f9ba3a62b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ade5c1ec0 con 0x556ade5c9be0 2023-07-18T03:21:12.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.709+0000 7f9ba3a62b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ade5fb390 con 0x556ade5c92d0 2023-07-18T03:21:12.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.709+0000 7f9ba3a62b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ade5ca540 con 0x556ade5dd2e0 2023-07-18T03:21:12.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.709+0000 7f9b8e79c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ade5c92d0 0x556ade5d5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:12.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.710+0000 7f9b8e79c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ade5c92d0 0x556ade5d5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:59946/0 (socket says 172.21.15.101:59946) 2023-07-18T03:21:12.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.710+0000 7f9b8e79c700 1 -- 172.21.15.101:0/1415189537 learned_addr learned my addr 172.21.15.101:0/1415189537 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:12.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.710+0000 7f9b8d79a700 1 --2- 172.21.15.101:0/1415189537 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ade5c9be0 0x556ade5d58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:12.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.710+0000 7f9b8e79c700 1 --2- 172.21.15.101:0/1415189537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ade5dd2e0 0x556ade5d9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:12.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.710+0000 7f9b8e79c700 1 -- 172.21.15.101:0/1415189537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ade5dd2e0 msgr2=0x556ade5d9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:12.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.710+0000 7f9b8e79c700 1 --2- 172.21.15.101:0/1415189537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ade5dd2e0 0x556ade5d9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:12.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.710+0000 7f9b8e79c700 1 -- 172.21.15.101:0/1415189537 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ade5c9be0 msgr2=0x556ade5d58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:12.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.710+0000 7f9b8e79c700 1 --2- 172.21.15.101:0/1415189537 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ade5c9be0 0x556ade5d58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:12.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.710+0000 7f9b8e79c700 1 -- 172.21.15.101:0/1415189537 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ade370250 con 0x556ade5c92d0 2023-07-18T03:21:12.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.710+0000 7f9b8d79a700 1 --2- 172.21.15.101:0/1415189537 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ade5c9be0 0x556ade5d58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:21:12.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.710+0000 7f9b8e79c700 1 --2- 172.21.15.101:0/1415189537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ade5c92d0 0x556ade5d5380 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f9b80009900 tx=0x7f9b8000b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16b426b13fe05cc0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:12.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.710+0000 7f9b7cff9700 1 -- 172.21.15.101:0/1415189537 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b80014be0 con 0x556ade5c92d0 2023-07-18T03:21:12.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.710+0000 7f9b7cff9700 1 -- 172.21.15.101:0/1415189537 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b8000d030 con 0x556ade5c92d0 2023-07-18T03:21:12.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.710+0000 7f9b7cff9700 1 -- 172.21.15.101:0/1415189537 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b80005270 con 0x556ade5c92d0 2023-07-18T03:21:12.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.711+0000 7f9ba3a62b80 1 -- 172.21.15.101:0/1415189537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ade5c92d0 msgr2=0x556ade5d5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:12.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.711+0000 7f9ba3a62b80 1 --2- 172.21.15.101:0/1415189537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ade5c92d0 0x556ade5d5380 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f9b80009900 tx=0x7f9b8000b040 comp rx=0 tx=0).stop 2023-07-18T03:21:12.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.711+0000 7f9ba3a62b80 1 -- 172.21.15.101:0/1415189537 shutdown_connections 2023-07-18T03:21:12.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.711+0000 7f9ba3a62b80 1 --2- 172.21.15.101:0/1415189537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ade5c92d0 0x556ade5d5380 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:12.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.711+0000 7f9ba3a62b80 1 --2- 172.21.15.101:0/1415189537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ade5dd2e0 0x556ade5d9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:12.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.711+0000 7f9ba3a62b80 1 --2- 172.21.15.101:0/1415189537 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ade5c9be0 0x556ade5d58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:12.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.711+0000 7f9ba3a62b80 1 -- 172.21.15.101:0/1415189537 >> 172.21.15.101:0/1415189537 conn(0x556ade5bfda0 msgr2=0x556ade5dc980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:12.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.711+0000 7f9ba3a62b80 1 -- 172.21.15.101:0/1415189537 shutdown_connections 2023-07-18T03:21:12.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.711+0000 7f9ba3a62b80 1 -- 172.21.15.101:0/1415189537 wait complete. 2023-07-18T03:21:12.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.712+0000 7f9ba3a62b80 1 -- start start 2023-07-18T03:21:12.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.712+0000 7f9ba3a62b80 1 Processor -- start 2023-07-18T03:21:12.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.712+0000 7f9ba3a62b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ade5c92d0 0x556ade5e1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:12.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.712+0000 7f9ba3a62b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ade5c9be0 0x556ade5e1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:12.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.712+0000 7f9ba3a62b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ade5dd2e0 0x556ade650b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:12.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.712+0000 7f9ba3a62b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ade5e26a0 con 0x556ade5dd2e0 2023-07-18T03:21:12.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.713+0000 7f9ba3a62b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ade651070 con 0x556ade5c9be0 2023-07-18T03:21:12.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.713+0000 7f9ba3a62b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ade651200 con 0x556ade5c92d0 2023-07-18T03:21:12.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.713+0000 7f9ba3a62b80 10 client.0 _ll_register_callbacks cb 0x7f9b80003e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:21:12.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.713+0000 7f9b8e79c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ade5c92d0 0x556ade5e1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:12.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.713+0000 7f9b8d79a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ade5c9be0 0x556ade5e1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:12.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.713+0000 7f9b8d79a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ade5c9be0 0x556ade5e1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:59956/0 (socket says 172.21.15.101:59956) 2023-07-18T03:21:12.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.713+0000 7f9b8e79c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ade5dd2e0 0x556ade650b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:12.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.713+0000 7f9b8d79a700 1 -- 172.21.15.101:0/1155975349 learned_addr learned my addr 172.21.15.101:0/1155975349 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:12.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.713+0000 7f9b8e79c700 1 -- 172.21.15.101:0/1155975349 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ade5c9be0 msgr2=0x556ade5e1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:12.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.713+0000 7f9b8e79c700 1 --2- 172.21.15.101:0/1155975349 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ade5c9be0 0x556ade5e1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:12.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.713+0000 7f9b8e79c700 1 -- 172.21.15.101:0/1155975349 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ade5dd2e0 msgr2=0x556ade650b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:12.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.713+0000 7f9b8e79c700 1 --2- 172.21.15.101:0/1155975349 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ade5dd2e0 0x556ade650b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:12.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.713+0000 7f9b8e79c700 1 -- 172.21.15.101:0/1155975349 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556ade332c60 con 0x556ade5c92d0 2023-07-18T03:21:12.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.713+0000 7f9b8d79a700 1 --2- 172.21.15.101:0/1155975349 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ade5c9be0 0x556ade5e1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:21:12.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.713+0000 7f9b8e79c700 1 --2- 172.21.15.101:0/1155975349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ade5c92d0 0x556ade5e1a30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9b80009c70 tx=0x7f9b800091c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5aa2faf6198a1253 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:12.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.713+0000 7f9b717fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:21:12.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.714+0000 7f9b717fa700 1 -- 172.21.15.101:0/1155975349 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b80004ae0 con 0x556ade5c92d0 2023-07-18T03:21:12.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.714+0000 7f9b717fa700 1 -- 172.21.15.101:0/1155975349 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f9b80016e40 con 0x556ade5c92d0 2023-07-18T03:21:12.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.714+0000 7f9ba3a62b80 1 -- 172.21.15.101:0/1155975349 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9b80009e70 con 0x556ade5c92d0 2023-07-18T03:21:12.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.714+0000 7f9b717fa700 1 -- 172.21.15.101:0/1155975349 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b80004c40 con 0x556ade5c92d0 2023-07-18T03:21:12.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.714+0000 7f9ba3a62b80 10 client.28330 fetch_fsmap learned FSMap version 37 2023-07-18T03:21:12.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.714+0000 7f9ba3a62b80 1 -- 172.21.15.101:0/1155975349 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x556ade3482b0 con 0x556ade5c92d0 2023-07-18T03:21:12.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.714+0000 7f9b717fa700 1 -- 172.21.15.101:0/1155975349 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b80009e70 con 0x556ade5c92d0 2023-07-18T03:21:12.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.714+0000 7f9b717fa700 1 -- 172.21.15.101:0/1155975349 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 37) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9b800444f0 con 0x556ade5c92d0 2023-07-18T03:21:12.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.714+0000 7f9ba3a62b80 10 client.28330 fetch_fsmap finished waiting for FSMap version 37 2023-07-18T03:21:12.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.714+0000 7f9ba3a62b80 10 client.28330 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:21:12.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.714+0000 7f9ba3a62b80 20 client.28330 populate_metadata read hostname 'smithi101' 2023-07-18T03:21:12.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.714+0000 7f9ba3a62b80 1 --2- 172.21.15.101:0/1155975349 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x556ade652550 0x556ade654930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:12.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.714+0000 7f9ba3a62b80 4 client.28330 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:21:12.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.714+0000 7f9ba3a62b80 1 -- 172.21.15.101:0/1155975349 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556ade4d7050 con 0x556ade652550 2023-07-18T03:21:12.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.715+0000 7f9b8d79a700 1 --2- 172.21.15.101:0/1155975349 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x556ade652550 0x556ade654930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:12.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.715+0000 7f9b8d79a700 1 --2- 172.21.15.101:0/1155975349 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x556ade652550 0x556ade654930 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5c3fa6713cc8288 server_cookie=7bf9862a05ca0d8d in_seq=0 out_seq=0 2023-07-18T03:21:12.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.715+0000 7f9b717fa700 10 client.28330 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:21:12.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.718+0000 7f9b717fa700 1 -- 172.21.15.101:0/1155975349 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x556ade4d7050 con 0x556ade652550 2023-07-18T03:21:12.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.718+0000 7f9b717fa700 10 client.28330 handle_command_reply: tid=0 2023-07-18T03:21:12.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.718+0000 7f9ba3a62b80 1 client.28330 shutdown 2023-07-18T03:21:12.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.718+0000 7f9ba3a62b80 1 -- 172.21.15.101:0/1155975349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ade5c92d0 msgr2=0x556ade5e1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:12.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.718+0000 7f9ba3a62b80 1 --2- 172.21.15.101:0/1155975349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ade5c92d0 0x556ade5e1a30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9b80009c70 tx=0x7f9b800091c0 comp rx=0 tx=0).stop 2023-07-18T03:21:12.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.718+0000 7f9ba3a62b80 1 -- 172.21.15.101:0/1155975349 shutdown_connections 2023-07-18T03:21:12.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.718+0000 7f9ba3a62b80 1 --2- 172.21.15.101:0/1155975349 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x556ade652550 0x556ade654930 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:12.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.718+0000 7f9ba3a62b80 1 --2- 172.21.15.101:0/1155975349 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ade5dd2e0 0x556ade650b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:12.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.718+0000 7f9ba3a62b80 1 --2- 172.21.15.101:0/1155975349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ade5c92d0 0x556ade5e1a30 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:12.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.718+0000 7f9ba3a62b80 1 --2- 172.21.15.101:0/1155975349 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ade5c9be0 0x556ade5e1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:12.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.718+0000 7f9b717fa700 0 client.28330 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:21:12.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.718+0000 7f9ba3a62b80 1 -- 172.21.15.101:0/1155975349 >> 172.21.15.101:0/1155975349 conn(0x556ade5bfda0 msgr2=0x556ade5d5380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:12.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.718+0000 7f9ba3a62b80 1 -- 172.21.15.101:0/1155975349 shutdown_connections 2023-07-18T03:21:12.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.718+0000 7f9ba3a62b80 1 -- 172.21.15.101:0/1155975349 wait complete. 2023-07-18T03:21:12.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.719+0000 7f9ba3a62b80 20 client.28330 trim_cache size 0 max 16384 2023-07-18T03:21:12.727 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:21:12.727 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "no active scrubs running", 2023-07-18T03:21:12.727 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": {} 2023-07-18T03:21:12.727 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:21:12.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.719+0000 7f9b901ff700 1 -- 172.21.15.101:0/3983554810 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9b740a1340 msgr2=0x7f9b740a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:12.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.719+0000 7f9b901ff700 1 --2- 172.21.15.101:0/3983554810 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9b740a1340 0x7f9b740a37f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9b840ca6c0 tx=0x7f9b840d0940 comp rx=0 tx=0).stop 2023-07-18T03:21:12.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.719+0000 7f9b901ff700 1 -- 172.21.15.101:0/3983554810 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9b880ffad0 msgr2=0x7f9b88199830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:12.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.719+0000 7f9b901ff700 1 --2- 172.21.15.101:0/3983554810 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9b880ffad0 0x7f9b88199830 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f9b8400e9c0 tx=0x7f9b8400ed80 comp rx=0 tx=0).stop 2023-07-18T03:21:12.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.719+0000 7f9b901ff700 1 -- 172.21.15.101:0/3983554810 shutdown_connections 2023-07-18T03:21:12.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.719+0000 7f9b901ff700 1 --2- 172.21.15.101:0/3983554810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b88105ae0 0x7f9b881938a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:12.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.719+0000 7f9b901ff700 1 --2- 172.21.15.101:0/3983554810 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9b880ffad0 0x7f9b88199830 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:12.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.719+0000 7f9b901ff700 1 --2- 172.21.15.101:0/3983554810 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9b740a1340 0x7f9b740a37f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:12.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.719+0000 7f9b901ff700 1 --2- 172.21.15.101:0/3983554810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b88100480 0x7f9b88199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:12.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.719+0000 7f9b901ff700 1 -- 172.21.15.101:0/3983554810 >> 172.21.15.101:0/3983554810 conn(0x7f9b880fb5f0 msgr2=0x7f9b880fbc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:12.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.719+0000 7f9b901ff700 1 -- 172.21.15.101:0/3983554810 shutdown_connections 2023-07-18T03:21:12.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:12.719+0000 7f9b901ff700 1 -- 172.21.15.101:0/3983554810 wait complete. 2023-07-18T03:21:12.735 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-18T03:21:12.735 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:21:12.952 INFO:tasks.workunit.client.0.smithi101.stdout: 10 17380 29.95 MB/sec warmup 37 sec latency 6727.658 ms 2023-07-18T03:21:13.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.047+0000 7fae737fe700 1 --2- 172.21.15.101:0/4247031725 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae74104990 0x7fae74104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:21:13.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.048+0000 7fae79dff700 1 -- 172.21.15.101:0/4247031725 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fae740fe980 msgr2=0x7fae740fedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:13.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.048+0000 7fae79dff700 1 --2- 172.21.15.101:0/4247031725 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fae740fe980 0x7fae740fedf0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fae6c0098f0 tx=0x7fae6c009c00 comp rx=0 tx=0).stop 2023-07-18T03:21:13.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.048+0000 7fae79dff700 1 -- 172.21.15.101:0/4247031725 shutdown_connections 2023-07-18T03:21:13.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.049+0000 7fae79dff700 1 --2- 172.21.15.101:0/4247031725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae740ff330 0x7fae7410b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:13.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.049+0000 7fae79dff700 1 --2- 172.21.15.101:0/4247031725 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae74104990 0x7fae74104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:13.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.049+0000 7fae79dff700 1 --2- 172.21.15.101:0/4247031725 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fae740fe980 0x7fae740fedf0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:13.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.049+0000 7fae79dff700 1 -- 172.21.15.101:0/4247031725 >> 172.21.15.101:0/4247031725 conn(0x7fae740fa4e0 msgr2=0x7fae740fc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:13.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.049+0000 7fae79dff700 1 -- 172.21.15.101:0/4247031725 shutdown_connections 2023-07-18T03:21:13.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.050+0000 7fae79dff700 1 -- 172.21.15.101:0/4247031725 wait complete. 2023-07-18T03:21:13.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.051+0000 7fae79dff700 1 Processor -- start 2023-07-18T03:21:13.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.051+0000 7fae79dff700 1 -- start start 2023-07-18T03:21:13.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.051+0000 7fae79dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fae740fe980 0x7fae7410af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:13.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.051+0000 7fae79dff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae740ff330 0x7fae7410b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:13.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.051+0000 7fae79dff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae74104990 0x7fae74100590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:13.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.051+0000 7fae79dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae74100ad0 con 0x7fae740fe980 2023-07-18T03:21:13.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.051+0000 7fae79dff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae74100c40 con 0x7fae74104990 2023-07-18T03:21:13.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.051+0000 7fae79dff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae74100e00 con 0x7fae740ff330 2023-07-18T03:21:13.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.052+0000 7fae737fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fae740fe980 0x7fae7410af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:13.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.052+0000 7fae73fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae74104990 0x7fae74100590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:13.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.052+0000 7fae73fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae74104990 0x7fae74100590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:59978/0 (socket says 172.21.15.101:59978) 2023-07-18T03:21:13.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.052+0000 7fae737fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fae740fe980 0x7fae7410af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:35932/0 (socket says 172.21.15.101:35932) 2023-07-18T03:21:13.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.052+0000 7fae73fff700 1 -- 172.21.15.101:0/2011940353 learned_addr learned my addr 172.21.15.101:0/2011940353 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:13.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.052+0000 7fae72ffd700 1 --2- 172.21.15.101:0/2011940353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae740ff330 0x7fae7410b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:13.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.052+0000 7fae73fff700 1 -- 172.21.15.101:0/2011940353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae740ff330 msgr2=0x7fae7410b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:13.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.052+0000 7fae73fff700 1 --2- 172.21.15.101:0/2011940353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae740ff330 0x7fae7410b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:13.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.052+0000 7fae73fff700 1 -- 172.21.15.101:0/2011940353 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fae740fe980 msgr2=0x7fae7410af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:13.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.052+0000 7fae73fff700 1 --2- 172.21.15.101:0/2011940353 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fae740fe980 0x7fae7410af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:13.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.052+0000 7fae73fff700 1 -- 172.21.15.101:0/2011940353 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae6c009520 con 0x7fae74104990 2023-07-18T03:21:13.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.053+0000 7fae73fff700 1 --2- 172.21.15.101:0/2011940353 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae74104990 0x7fae74100590 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fae6400b780 tx=0x7fae6400ba90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:13.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.053+0000 7fae70ff9700 1 -- 172.21.15.101:0/2011940353 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae64010860 con 0x7fae74104990 2023-07-18T03:21:13.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.053+0000 7fae79dff700 1 -- 172.21.15.101:0/2011940353 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae741010e0 con 0x7fae74104990 2023-07-18T03:21:13.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.053+0000 7fae70ff9700 1 -- 172.21.15.101:0/2011940353 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae64010e80 con 0x7fae74104990 2023-07-18T03:21:13.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.053+0000 7fae70ff9700 1 -- 172.21.15.101:0/2011940353 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae64007370 con 0x7fae74104990 2023-07-18T03:21:13.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.053+0000 7fae79dff700 1 -- 172.21.15.101:0/2011940353 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae741941c0 con 0x7fae74104990 2023-07-18T03:21:13.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.054+0000 7fae79dff700 1 -- 172.21.15.101:0/2011940353 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae740680b0 con 0x7fae74104990 2023-07-18T03:21:13.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.056+0000 7fae70ff9700 1 -- 172.21.15.101:0/2011940353 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fae6400f5d0 con 0x7fae74104990 2023-07-18T03:21:13.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.058+0000 7fae70ff9700 1 --2- 172.21.15.101:0/2011940353 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fae5c0a1370 0x7fae5c0a3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:13.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.058+0000 7fae70ff9700 1 -- 172.21.15.101:0/2011940353 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9549+0+0 (secure 0 0 0) 0x7fae640ccef0 con 0x7fae74104990 2023-07-18T03:21:13.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.058+0000 7fae737fe700 1 --2- 172.21.15.101:0/2011940353 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fae5c0a1370 0x7fae5c0a3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:13.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.059+0000 7fae737fe700 1 --2- 172.21.15.101:0/2011940353 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fae5c0a1370 0x7fae5c0a3820 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fae68005e50 tx=0x7fae6801a440 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:13.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.061+0000 7fae70ff9700 1 -- 172.21.15.101:0/2011940353 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fae64095ea0 con 0x7fae74104990 2023-07-18T03:21:13.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.440+0000 7fae79dff700 1 -- 172.21.15.101:0/2011940353 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae740676e0 con 0x7fae74104990 2023-07-18T03:21:13.446 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.450+0000 7fae70ff9700 1 -- 172.21.15.101:0/2011940353 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+10863 (secure 0 0 0) 0x7fae640994c0 con 0x7fae74104990 2023-07-18T03:21:13.449 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:21:13.449 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/4192846769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":4192846769},{"type":"v1","addr":"172.21.15.101:6839","nonce":4192846769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6841/3824340175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6840","nonce":3824340175},{"type":"v1","addr":"172.21.15.101:6841","nonce":3824340175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:04:59.361185+0000","modified":"2023-07-18T03:21:06.230659+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.155:6839/2055462483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2055462483},{"type":"v1","addr":"172.21.15.155:6839","nonce":2055462483}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:21:13.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.460+0000 7fae79dff700 1 -- 172.21.15.101:0/2011940353 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fae5c0a1370 msgr2=0x7fae5c0a3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:13.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.460+0000 7fae79dff700 1 --2- 172.21.15.101:0/2011940353 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fae5c0a1370 0x7fae5c0a3820 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fae68005e50 tx=0x7fae6801a440 comp rx=0 tx=0).stop 2023-07-18T03:21:13.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.460+0000 7fae79dff700 1 -- 172.21.15.101:0/2011940353 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae74104990 msgr2=0x7fae74100590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:13.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.460+0000 7fae79dff700 1 --2- 172.21.15.101:0/2011940353 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae74104990 0x7fae74100590 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fae6400b780 tx=0x7fae6400ba90 comp rx=0 tx=0).stop 2023-07-18T03:21:13.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.465+0000 7fae79dff700 1 -- 172.21.15.101:0/2011940353 shutdown_connections 2023-07-18T03:21:13.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.465+0000 7fae79dff700 1 --2- 172.21.15.101:0/2011940353 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fae5c0a1370 0x7fae5c0a3820 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:13.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.465+0000 7fae79dff700 1 --2- 172.21.15.101:0/2011940353 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae74104990 0x7fae74100590 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:13.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.465+0000 7fae79dff700 1 --2- 172.21.15.101:0/2011940353 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fae740fe980 0x7fae7410af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:13.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.465+0000 7fae79dff700 1 --2- 172.21.15.101:0/2011940353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae740ff330 0x7fae7410b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:13.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.465+0000 7fae79dff700 1 -- 172.21.15.101:0/2011940353 >> 172.21.15.101:0/2011940353 conn(0x7fae740fa4e0 msgr2=0x7fae74068900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:13.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.465+0000 7fae79dff700 1 -- 172.21.15.101:0/2011940353 shutdown_connections 2023-07-18T03:21:13.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.466+0000 7fae79dff700 1 -- 172.21.15.101:0/2011940353 wait complete. 2023-07-18T03:21:13.484 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 37 2023-07-18T03:21:13.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:13 smithi101 ceph-mon[126379]: pgmap v427: 129 pgs: 129 active+clean; 950 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.8 MiB/s wr, 581 op/s 2023-07-18T03:21:13.518 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-18T03:21:13.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:13 smithi155 ceph-mon[128885]: pgmap v427: 129 pgs: 129 active+clean; 950 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.8 MiB/s wr, 581 op/s 2023-07-18T03:21:13.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:13 smithi179 ceph-mon[129229]: pgmap v427: 129 pgs: 129 active+clean; 950 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.8 MiB/s wr, 581 op/s 2023-07-18T03:21:13.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.907+0000 7fdada7ff700 1 -- 172.21.15.101:0/695476785 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdad40ffae0 msgr2=0x7fdad40fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:13.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.907+0000 7fdada7ff700 1 --2- 172.21.15.101:0/695476785 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdad40ffae0 0x7fdad40fff50 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fdac8009990 tx=0x7fdac8009ca0 comp rx=0 tx=0).stop 2023-07-18T03:21:13.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.907+0000 7fdada7ff700 1 -- 172.21.15.101:0/695476785 shutdown_connections 2023-07-18T03:21:13.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.907+0000 7fdada7ff700 1 --2- 172.21.15.101:0/695476785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdad4100490 0x7fdad410c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:13.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.907+0000 7fdada7ff700 1 --2- 172.21.15.101:0/695476785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdad4105af0 0x7fdad4105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:13.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.907+0000 7fdada7ff700 1 --2- 172.21.15.101:0/695476785 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdad40ffae0 0x7fdad40fff50 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:13.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.907+0000 7fdada7ff700 1 -- 172.21.15.101:0/695476785 >> 172.21.15.101:0/695476785 conn(0x7fdad40fb5e0 msgr2=0x7fdad40fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:13.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.909+0000 7fdada7ff700 1 -- 172.21.15.101:0/695476785 shutdown_connections 2023-07-18T03:21:13.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.909+0000 7fdada7ff700 1 -- 172.21.15.101:0/695476785 wait complete. 2023-07-18T03:21:13.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.910+0000 7fdada7ff700 1 Processor -- start 2023-07-18T03:21:13.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.910+0000 7fdada7ff700 1 -- start start 2023-07-18T03:21:13.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.910+0000 7fdada7ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdad40ffae0 0x7fdad406a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:13.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.911+0000 7fdada7ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdad4100490 0x7fdad406aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:13.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.911+0000 7fdada7ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdad4105af0 0x7fdad41b0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:13.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.911+0000 7fdada7ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad41b0cc0 con 0x7fdad40ffae0 2023-07-18T03:21:13.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.911+0000 7fdada7ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad41a3f80 con 0x7fdad4100490 2023-07-18T03:21:13.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.911+0000 7fdada7ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad41a4140 con 0x7fdad4105af0 2023-07-18T03:21:13.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.911+0000 7fdad8d9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdad4105af0 0x7fdad41b0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:13.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.911+0000 7fdad8d9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdad4105af0 0x7fdad41b0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:53406/0 (socket says 172.21.15.101:53406) 2023-07-18T03:21:13.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.911+0000 7fdad8d9c700 1 -- 172.21.15.101:0/2083071702 learned_addr learned my addr 172.21.15.101:0/2083071702 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:13.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.912+0000 7fdad3fff700 1 --2- 172.21.15.101:0/2083071702 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdad40ffae0 0x7fdad406a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:13.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.912+0000 7fdad8d9c700 1 -- 172.21.15.101:0/2083071702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdad4100490 msgr2=0x7fdad406aef0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-18T03:21:13.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.912+0000 7fdad8d9c700 1 --2- 172.21.15.101:0/2083071702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdad4100490 0x7fdad406aef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:13.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.912+0000 7fdad8d9c700 1 -- 172.21.15.101:0/2083071702 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdad40ffae0 msgr2=0x7fdad406a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:13.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.912+0000 7fdad8d9c700 1 --2- 172.21.15.101:0/2083071702 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdad40ffae0 0x7fdad406a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:13.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.912+0000 7fdad8d9c700 1 -- 172.21.15.101:0/2083071702 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac80095f0 con 0x7fdad4105af0 2023-07-18T03:21:13.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.913+0000 7fdad8d9c700 1 --2- 172.21.15.101:0/2083071702 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdad4105af0 0x7fdad41b0780 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fdacc00d3f0 tx=0x7fdacc00d7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:13.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.913+0000 7fdad17fa700 1 -- 172.21.15.101:0/2083071702 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdacc004db0 con 0x7fdad4105af0 2023-07-18T03:21:13.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.913+0000 7fdad17fa700 1 -- 172.21.15.101:0/2083071702 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdacc008bb0 con 0x7fdad4105af0 2023-07-18T03:21:13.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.913+0000 7fdad17fa700 1 -- 172.21.15.101:0/2083071702 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdacc0046a0 con 0x7fdad4105af0 2023-07-18T03:21:13.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.914+0000 7fdada7ff700 1 -- 172.21.15.101:0/2083071702 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdad41a4420 con 0x7fdad4105af0 2023-07-18T03:21:13.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.914+0000 7fdada7ff700 1 -- 172.21.15.101:0/2083071702 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdad41a48e0 con 0x7fdad4105af0 2023-07-18T03:21:13.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.920+0000 7fdad17fa700 1 -- 172.21.15.101:0/2083071702 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fdacc008d20 con 0x7fdad4105af0 2023-07-18T03:21:13.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.921+0000 7fdaee191b80 1 Processor -- start 2023-07-18T03:21:13.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.922+0000 7fdad17fa700 1 --2- 172.21.15.101:0/2083071702 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fdabc0a1400 0x7fdabc0a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:13.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.922+0000 7fdad17fa700 1 -- 172.21.15.101:0/2083071702 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9549+0+0 (secure 0 0 0) 0x7fdacc0cc9f0 con 0x7fdad4105af0 2023-07-18T03:21:13.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.923+0000 7fdaee191b80 1 -- start start 2023-07-18T03:21:13.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.923+0000 7fdaee191b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5594b542c2d0 0x5594b542c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:13.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.923+0000 7fdaee191b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5594b542cbe0 0x5594b5437590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:13.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.923+0000 7fdaee191b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594b5437ad0 0x5594b5439eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:13.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.923+0000 7fdaee191b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594b5332cb0 con 0x5594b542c2d0 2023-07-18T03:21:13.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.923+0000 7fdaee191b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594b5385a40 con 0x5594b5437ad0 2023-07-18T03:21:13.922 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.923+0000 7fdaee191b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594b537eed0 con 0x5594b542cbe0 2023-07-18T03:21:13.924 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.924+0000 7fdad8d9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594b5437ad0 0x5594b5439eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:13.924 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.925+0000 7fdad8d9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594b5437ad0 0x5594b5439eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:59994/0 (socket says 172.21.15.101:59994) 2023-07-18T03:21:13.924 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.925+0000 7fdad8d9c700 1 -- 172.21.15.101:0/1855327689 learned_addr learned my addr 172.21.15.101:0/1855327689 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:13.924 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.925+0000 7fdad37fe700 1 --2- 172.21.15.101:0/2083071702 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fdabc0a1400 0x7fdabc0a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:13.924 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.925+0000 7fdad37fe700 1 --2- 172.21.15.101:0/1855327689 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5594b542cbe0 0x5594b5437590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:13.924 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.925+0000 7fdad3fff700 1 --2- 172.21.15.101:0/1855327689 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5594b542c2d0 0x5594b542c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:13.925 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.926+0000 7fdad8d9c700 1 -- 172.21.15.101:0/1855327689 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5594b542cbe0 msgr2=0x5594b5437590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:13.925 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.926+0000 7fdad8d9c700 1 --2- 172.21.15.101:0/1855327689 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5594b542cbe0 0x5594b5437590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:13.925 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.926+0000 7fdad8d9c700 1 -- 172.21.15.101:0/1855327689 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5594b542c2d0 msgr2=0x5594b542c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:13.925 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.926+0000 7fdad8d9c700 1 --2- 172.21.15.101:0/1855327689 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5594b542c2d0 0x5594b542c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:13.925 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.926+0000 7fdad8d9c700 1 -- 172.21.15.101:0/1855327689 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594b51acde0 con 0x5594b5437ad0 2023-07-18T03:21:13.925 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.926+0000 7fdad8d9c700 1 --2- 172.21.15.101:0/1855327689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594b5437ad0 0x5594b5439eb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fdacc0a1580 tx=0x7fdacc00bb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=992bb8ef26486c64 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:13.925 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.927+0000 7fdaba7fc700 1 -- 172.21.15.101:0/1855327689 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdacc099660 con 0x5594b5437ad0 2023-07-18T03:21:13.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.927+0000 7fdaba7fc700 1 -- 172.21.15.101:0/1855327689 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdacc09d690 con 0x5594b5437ad0 2023-07-18T03:21:13.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.927+0000 7fdaba7fc700 1 -- 172.21.15.101:0/1855327689 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdacc0998c0 con 0x5594b5437ad0 2023-07-18T03:21:13.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.928+0000 7fdaee191b80 1 -- 172.21.15.101:0/1855327689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594b5437ad0 msgr2=0x5594b5439eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:13.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.928+0000 7fdaee191b80 1 --2- 172.21.15.101:0/1855327689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594b5437ad0 0x5594b5439eb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fdacc0a1580 tx=0x7fdacc00bb70 comp rx=0 tx=0).stop 2023-07-18T03:21:13.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.929+0000 7fdaee191b80 1 -- 172.21.15.101:0/1855327689 shutdown_connections 2023-07-18T03:21:13.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.929+0000 7fdaee191b80 1 --2- 172.21.15.101:0/1855327689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594b5437ad0 0x5594b5439eb0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:13.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.929+0000 7fdaee191b80 1 --2- 172.21.15.101:0/1855327689 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5594b542c2d0 0x5594b542c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:13.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.929+0000 7fdaee191b80 1 --2- 172.21.15.101:0/1855327689 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5594b542cbe0 0x5594b5437590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:13.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.929+0000 7fdaee191b80 1 -- 172.21.15.101:0/1855327689 >> 172.21.15.101:0/1855327689 conn(0x5594b5422da0 msgr2=0x5594b54286a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:13.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.929+0000 7fdaee191b80 1 -- 172.21.15.101:0/1855327689 shutdown_connections 2023-07-18T03:21:13.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.929+0000 7fdad37fe700 1 --2- 172.21.15.101:0/2083071702 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fdabc0a1400 0x7fdabc0a38b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdac80098e0 tx=0x7fdac8000bc0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:13.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.929+0000 7fdaee191b80 1 -- 172.21.15.101:0/1855327689 wait complete. 2023-07-18T03:21:13.928 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.930+0000 7fdaee191b80 1 -- start start 2023-07-18T03:21:13.928 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.930+0000 7fdaee191b80 1 Processor -- start 2023-07-18T03:21:13.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.935+0000 7fdaee191b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5594b542c2d0 0x5594b54374c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:13.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.935+0000 7fdaee191b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594b542cbe0 0x5594b5434740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:13.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.935+0000 7fdaee191b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5594b5437ad0 0x5594b5434dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:13.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.935+0000 7fdaee191b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594b531eac0 con 0x5594b5437ad0 2023-07-18T03:21:13.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.935+0000 7fdaee191b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594b531e7c0 con 0x5594b542cbe0 2023-07-18T03:21:13.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.935+0000 7fdaee191b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594b5202920 con 0x5594b542c2d0 2023-07-18T03:21:13.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.935+0000 7fdaee191b80 10 client.0 _ll_register_callbacks cb 0x5594b531c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:21:13.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.935+0000 7fdad8d9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5594b5437ad0 0x5594b5434dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:13.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.935+0000 7fdad8d9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5594b5437ad0 0x5594b5434dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:35974/0 (socket says 172.21.15.101:35974) 2023-07-18T03:21:13.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.935+0000 7fdad8d9c700 1 -- 172.21.15.101:0/4155325355 learned_addr learned my addr 172.21.15.101:0/4155325355 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:13.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.935+0000 7fdad37fe700 1 --2- 172.21.15.101:0/4155325355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594b542cbe0 0x5594b5434740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:13.934 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.936+0000 7fdad8d9c700 1 -- 172.21.15.101:0/4155325355 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5594b542c2d0 msgr2=0x5594b54374c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:13.934 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.936+0000 7fdad8d9c700 1 --2- 172.21.15.101:0/4155325355 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5594b542c2d0 0x5594b54374c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:13.934 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.936+0000 7fdad8d9c700 1 -- 172.21.15.101:0/4155325355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594b542cbe0 msgr2=0x5594b5434740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:13.934 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.936+0000 7fdad8d9c700 1 --2- 172.21.15.101:0/4155325355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594b542cbe0 0x5594b5434740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:13.934 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.936+0000 7fdad8d9c700 1 -- 172.21.15.101:0/4155325355 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdad41a48e0 con 0x5594b5437ad0 2023-07-18T03:21:13.934 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.936+0000 7fdad8d9c700 1 --2- 172.21.15.101:0/4155325355 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5594b5437ad0 0x5594b5434dc0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fdacc095b60 tx=0x7fdacc0c9cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e3c80887344cab0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:13.934 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.937+0000 7fdab37fe700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:21:13.935 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.937+0000 7fdab37fe700 1 -- 172.21.15.101:0/4155325355 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdacc098b50 con 0x5594b5437ad0 2023-07-18T03:21:13.935 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.937+0000 7fdab37fe700 1 -- 172.21.15.101:0/4155325355 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9549+0+0 (secure 0 0 0) 0x7fdacc098cb0 con 0x5594b5437ad0 2023-07-18T03:21:13.935 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.937+0000 7fdaee191b80 1 -- 172.21.15.101:0/4155325355 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5594b52d84f0 con 0x5594b5437ad0 2023-07-18T03:21:13.935 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.937+0000 7fdab37fe700 1 -- 172.21.15.101:0/4155325355 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdacc0998c0 con 0x5594b5437ad0 2023-07-18T03:21:13.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.937+0000 7fdaee191b80 10 client.18273 fetch_fsmap learned FSMap version 37 2023-07-18T03:21:13.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.937+0000 7fdaee191b80 1 -- 172.21.15.101:0/4155325355 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5594b5195c60 con 0x5594b5437ad0 2023-07-18T03:21:13.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.938+0000 7fdab37fe700 1 -- 172.21.15.101:0/4155325355 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdacc07c080 con 0x5594b5437ad0 2023-07-18T03:21:13.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.938+0000 7fdab37fe700 1 -- 172.21.15.101:0/4155325355 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 37) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fdacc095450 con 0x5594b5437ad0 2023-07-18T03:21:13.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.939+0000 7fdaee191b80 10 client.18273 fetch_fsmap finished waiting for FSMap version 37 2023-07-18T03:21:13.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.939+0000 7fdaee191b80 10 client.18273 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:21:13.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.939+0000 7fdaee191b80 20 client.18273 populate_metadata read hostname 'smithi101' 2023-07-18T03:21:13.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.939+0000 7fdaee191b80 1 --2- 172.21.15.101:0/4155325355 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5594b5446e80 0x5594b5449260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:13.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.939+0000 7fdaee191b80 4 client.18273 mds_command: new command op to 24541 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:21:13.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.939+0000 7fdaee191b80 1 -- 172.21.15.101:0/4155325355 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5594b5377eb0 con 0x5594b5446e80 2023-07-18T03:21:13.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.939+0000 7fdad3fff700 1 --2- 172.21.15.101:0/4155325355 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5594b5446e80 0x5594b5449260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:13.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.941+0000 7fdab37fe700 10 client.18273 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:21:13.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.941+0000 7fdad3fff700 1 --2- 172.21.15.101:0/4155325355 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5594b5446e80 0x5594b5449260 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1713ae40bddbfb1d server_cookie=c9a04c3b4f5b995c in_seq=0 out_seq=0 2023-07-18T03:21:13.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.941+0000 7fdab37fe700 1 -- 172.21.15.101:0/4155325355 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5594b5377eb0 con 0x5594b5446e80 2023-07-18T03:21:13.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.941+0000 7fdab37fe700 10 client.18273 handle_command_reply: tid=0 2023-07-18T03:21:13.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.941+0000 7fdaee191b80 1 client.18273 shutdown 2023-07-18T03:21:13.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.942+0000 7fdaee191b80 1 -- 172.21.15.101:0/4155325355 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5594b5437ad0 msgr2=0x5594b5434dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:13.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.942+0000 7fdaee191b80 1 --2- 172.21.15.101:0/4155325355 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5594b5437ad0 0x5594b5434dc0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fdacc095b60 tx=0x7fdacc0c9cd0 comp rx=0 tx=0).stop 2023-07-18T03:21:13.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.942+0000 7fdaee191b80 1 -- 172.21.15.101:0/4155325355 shutdown_connections 2023-07-18T03:21:13.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.942+0000 7fdaee191b80 1 --2- 172.21.15.101:0/4155325355 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5594b5446e80 0x5594b5449260 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:13.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.942+0000 7fdaee191b80 1 --2- 172.21.15.101:0/4155325355 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5594b5437ad0 0x5594b5434dc0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:13.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.942+0000 7fdaee191b80 1 --2- 172.21.15.101:0/4155325355 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5594b542c2d0 0x5594b54374c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:13.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.942+0000 7fdaee191b80 1 --2- 172.21.15.101:0/4155325355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594b542cbe0 0x5594b5434740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:13.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.942+0000 7fdaee191b80 1 -- 172.21.15.101:0/4155325355 >> 172.21.15.101:0/4155325355 conn(0x5594b5422da0 msgr2=0x5594b54281d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:13.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.942+0000 7fdab37fe700 0 client.18273 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:21:13.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.942+0000 7fdaee191b80 1 -- 172.21.15.101:0/4155325355 shutdown_connections 2023-07-18T03:21:13.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.942+0000 7fdaee191b80 1 -- 172.21.15.101:0/4155325355 wait complete. 2023-07-18T03:21:13.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.942+0000 7fdaee191b80 20 client.18273 trim_cache size 0 max 16384 2023-07-18T03:21:13.953 INFO:tasks.workunit.client.0.smithi101.stdout: 10 17575 29.42 MB/sec warmup 38 sec latency 7728.721 ms 2023-07-18T03:21:13.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.972+0000 7fdaee191b80 1 Processor -- start 2023-07-18T03:21:13.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.972+0000 7fdaee191b80 1 -- start start 2023-07-18T03:21:13.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.972+0000 7fdaee191b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5594b542c2d0 0x5594b543e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:13.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.972+0000 7fdaee191b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594b542cbe0 0x5594b54370f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:13.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.972+0000 7fdaee191b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5594b5437ad0 0x5594b542fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:13.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.972+0000 7fdaee191b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594b54356d0 con 0x5594b5437ad0 2023-07-18T03:21:13.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.972+0000 7fdaee191b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594b5435d20 con 0x5594b542cbe0 2023-07-18T03:21:13.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.972+0000 7fdaee191b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594b543e9f0 con 0x5594b542c2d0 2023-07-18T03:21:13.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.973+0000 7fdad3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5594b542c2d0 0x5594b543e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:13.970 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.973+0000 7fdad3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5594b542c2d0 0x5594b543e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:53430/0 (socket says 172.21.15.101:53430) 2023-07-18T03:21:13.970 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.973+0000 7fdad3fff700 1 -- 172.21.15.101:0/1337242274 learned_addr learned my addr 172.21.15.101:0/1337242274 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:13.970 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.974+0000 7fdad3fff700 1 -- 172.21.15.101:0/1337242274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594b542cbe0 msgr2=0x5594b54370f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-18T03:21:13.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.974+0000 7fdad3fff700 1 --2- 172.21.15.101:0/1337242274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594b542cbe0 0x5594b54370f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:13.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.974+0000 7fdad3fff700 1 -- 172.21.15.101:0/1337242274 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5594b5437ad0 msgr2=0x5594b542fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:13.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.974+0000 7fdad3fff700 1 --2- 172.21.15.101:0/1337242274 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5594b5437ad0 0x5594b542fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:13.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.974+0000 7fdad3fff700 1 -- 172.21.15.101:0/1337242274 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594b51d3250 con 0x5594b542c2d0 2023-07-18T03:21:13.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.974+0000 7fdad3fff700 1 --2- 172.21.15.101:0/1337242274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5594b542c2d0 0x5594b543e4b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fdac4008d20 tx=0x7fdac4005df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47e93691a0578aeb server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:13.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.975+0000 7fdabaffd700 1 -- 172.21.15.101:0/1337242274 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac4016040 con 0x5594b542c2d0 2023-07-18T03:21:13.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.975+0000 7fdabaffd700 1 -- 172.21.15.101:0/1337242274 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdac40184b0 con 0x5594b542c2d0 2023-07-18T03:21:13.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.975+0000 7fdabaffd700 1 -- 172.21.15.101:0/1337242274 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac40056f0 con 0x5594b542c2d0 2023-07-18T03:21:13.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.976+0000 7fdaee191b80 1 -- 172.21.15.101:0/1337242274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5594b542c2d0 msgr2=0x5594b543e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:13.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.976+0000 7fdaee191b80 1 --2- 172.21.15.101:0/1337242274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5594b542c2d0 0x5594b543e4b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fdac4008d20 tx=0x7fdac4005df0 comp rx=0 tx=0).stop 2023-07-18T03:21:13.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.976+0000 7fdaee191b80 1 -- 172.21.15.101:0/1337242274 shutdown_connections 2023-07-18T03:21:13.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.976+0000 7fdaee191b80 1 --2- 172.21.15.101:0/1337242274 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5594b5437ad0 0x5594b542fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:13.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.976+0000 7fdaee191b80 1 --2- 172.21.15.101:0/1337242274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5594b542c2d0 0x5594b543e4b0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:13.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.976+0000 7fdaee191b80 1 --2- 172.21.15.101:0/1337242274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594b542cbe0 0x5594b54370f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:13.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.976+0000 7fdaee191b80 1 -- 172.21.15.101:0/1337242274 >> 172.21.15.101:0/1337242274 conn(0x5594b5422da0 msgr2=0x5594b54436a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:13.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.976+0000 7fdaee191b80 1 -- 172.21.15.101:0/1337242274 shutdown_connections 2023-07-18T03:21:13.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.976+0000 7fdaee191b80 1 -- 172.21.15.101:0/1337242274 wait complete. 2023-07-18T03:21:13.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.977+0000 7fdaee191b80 1 -- start start 2023-07-18T03:21:13.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.977+0000 7fdaee191b80 1 Processor -- start 2023-07-18T03:21:13.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.979+0000 7fdaee191b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5594b542c2d0 0x5594b54930a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:13.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.979+0000 7fdaee191b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594b5437ad0 0x5594b54935e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:13.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.979+0000 7fdaee191b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5594b548ec40 0x5594b548f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:13.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.979+0000 7fdaee191b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594b548f5f0 con 0x5594b542c2d0 2023-07-18T03:21:13.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.979+0000 7fdaee191b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594b548f730 con 0x5594b5437ad0 2023-07-18T03:21:13.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.979+0000 7fdaee191b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594b548f8f0 con 0x5594b548ec40 2023-07-18T03:21:13.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.979+0000 7fdaee191b80 10 client.0 _ll_register_callbacks cb 0x7fdac4008310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:21:13.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.980+0000 7fdad37fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594b5437ad0 0x5594b54935e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:13.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.980+0000 7fdad37fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594b5437ad0 0x5594b54935e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60006/0 (socket says 172.21.15.101:60006) 2023-07-18T03:21:13.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.980+0000 7fdad37fe700 1 -- 172.21.15.101:0/1012421149 learned_addr learned my addr 172.21.15.101:0/1012421149 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:13.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.980+0000 7fdad3fff700 1 --2- 172.21.15.101:0/1012421149 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5594b542c2d0 0x5594b54930a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:13.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.980+0000 7fdad37fe700 1 -- 172.21.15.101:0/1012421149 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5594b548ec40 msgr2=0x5594b548f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:13.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.980+0000 7fdad37fe700 1 --2- 172.21.15.101:0/1012421149 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5594b548ec40 0x5594b548f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:13.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.980+0000 7fdad37fe700 1 -- 172.21.15.101:0/1012421149 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5594b542c2d0 msgr2=0x5594b54930a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:13.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.980+0000 7fdad37fe700 1 --2- 172.21.15.101:0/1012421149 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5594b542c2d0 0x5594b54930a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:13.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.981+0000 7fdad37fe700 1 -- 172.21.15.101:0/1012421149 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdac40088e0 con 0x5594b5437ad0 2023-07-18T03:21:13.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.982+0000 7fdad37fe700 1 --2- 172.21.15.101:0/1012421149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594b5437ad0 0x5594b54935e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fdac8009990 tx=0x7fdac8009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d87545bd94998fcc server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:13.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.982+0000 7fdab37fe700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:21:13.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.985+0000 7fdab37fe700 1 -- 172.21.15.101:0/1012421149 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac8003e50 con 0x5594b5437ad0 2023-07-18T03:21:13.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.985+0000 7fdaee191b80 1 -- 172.21.15.101:0/1012421149 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdacc09d690 con 0x5594b5437ad0 2023-07-18T03:21:13.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.985+0000 7fdab37fe700 1 -- 172.21.15.101:0/1012421149 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9549+0+0 (secure 0 0 0) 0x7fdac800f460 con 0x5594b5437ad0 2023-07-18T03:21:13.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.985+0000 7fdab37fe700 1 -- 172.21.15.101:0/1012421149 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdac800f620 con 0x5594b5437ad0 2023-07-18T03:21:13.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.985+0000 7fdaee191b80 10 client.28307 fetch_fsmap learned FSMap version 37 2023-07-18T03:21:13.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.985+0000 7fdaee191b80 1 -- 172.21.15.101:0/1012421149 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5594b51ab2b0 con 0x5594b5437ad0 2023-07-18T03:21:13.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.986+0000 7fdab37fe700 1 -- 172.21.15.101:0/1012421149 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdacc09d690 con 0x5594b5437ad0 2023-07-18T03:21:13.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.986+0000 7fdab37fe700 1 -- 172.21.15.101:0/1012421149 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 37) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fdac80317c0 con 0x5594b5437ad0 2023-07-18T03:21:13.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.986+0000 7fdaee191b80 10 client.28307 fetch_fsmap finished waiting for FSMap version 37 2023-07-18T03:21:13.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.986+0000 7fdaee191b80 10 client.28307 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:21:13.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.986+0000 7fdaee191b80 20 client.28307 populate_metadata read hostname 'smithi101' 2023-07-18T03:21:13.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.986+0000 7fdaee191b80 1 --2- 172.21.15.101:0/1012421149 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5594b549c410 0x5594b549e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:13.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.986+0000 7fdaee191b80 4 client.28307 mds_command: new command op to 24541 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:21:13.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.986+0000 7fdaee191b80 1 -- 172.21.15.101:0/1012421149 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5594b533a050 con 0x5594b549c410 2023-07-18T03:21:13.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.986+0000 7fdad3fff700 1 --2- 172.21.15.101:0/1012421149 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5594b549c410 0x5594b549e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:13.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.987+0000 7fdab37fe700 10 client.28307 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:21:13.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.987+0000 7fdad3fff700 1 --2- 172.21.15.101:0/1012421149 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5594b549c410 0x5594b549e820 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3fe01c02a51e66b0 server_cookie=bd93f16a815017ef in_seq=0 out_seq=0 2023-07-18T03:21:13.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.987+0000 7fdab37fe700 1 -- 172.21.15.101:0/1012421149 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5594b533a050 con 0x5594b549c410 2023-07-18T03:21:13.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.987+0000 7fdab37fe700 10 client.28307 handle_command_reply: tid=0 2023-07-18T03:21:13.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.987+0000 7fdaee191b80 1 client.28307 shutdown 2023-07-18T03:21:13.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.988+0000 7fdaee191b80 1 -- 172.21.15.101:0/1012421149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594b5437ad0 msgr2=0x5594b54935e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:13.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.988+0000 7fdaee191b80 1 --2- 172.21.15.101:0/1012421149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594b5437ad0 0x5594b54935e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fdac8009990 tx=0x7fdac8009ca0 comp rx=0 tx=0).stop 2023-07-18T03:21:13.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.991+0000 7fdaee191b80 1 -- 172.21.15.101:0/1012421149 shutdown_connections 2023-07-18T03:21:13.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.991+0000 7fdaee191b80 1 --2- 172.21.15.101:0/1012421149 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5594b549c410 0x5594b549e820 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:13.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.991+0000 7fdaee191b80 1 --2- 172.21.15.101:0/1012421149 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5594b548ec40 0x5594b548f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:13.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.991+0000 7fdaee191b80 1 --2- 172.21.15.101:0/1012421149 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5594b542c2d0 0x5594b54930a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:13.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.991+0000 7fdaee191b80 1 --2- 172.21.15.101:0/1012421149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594b5437ad0 0x5594b54935e0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:13.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.991+0000 7fdaee191b80 1 -- 172.21.15.101:0/1012421149 >> 172.21.15.101:0/1012421149 conn(0x5594b5422da0 msgr2=0x5594b54370b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:13.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.991+0000 7fdab37fe700 0 client.28307 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:21:13.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.991+0000 7fdaee191b80 1 -- 172.21.15.101:0/1012421149 shutdown_connections 2023-07-18T03:21:13.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.991+0000 7fdaee191b80 1 -- 172.21.15.101:0/1012421149 wait complete. 2023-07-18T03:21:13.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.992+0000 7fdaee191b80 20 client.28307 trim_cache size 0 max 16384 2023-07-18T03:21:13.990 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:21:13.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.994+0000 7fdada7ff700 1 -- 172.21.15.101:0/2083071702 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fdabc0a1400 msgr2=0x7fdabc0a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:13.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.994+0000 7fdada7ff700 1 --2- 172.21.15.101:0/2083071702 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fdabc0a1400 0x7fdabc0a38b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdac80098e0 tx=0x7fdac8000bc0 comp rx=0 tx=0).stop 2023-07-18T03:21:13.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.994+0000 7fdada7ff700 1 -- 172.21.15.101:0/2083071702 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdad4105af0 msgr2=0x7fdad41b0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:13.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.994+0000 7fdada7ff700 1 --2- 172.21.15.101:0/2083071702 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdad4105af0 0x7fdad41b0780 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fdacc00d3f0 tx=0x7fdacc00d7b0 comp rx=0 tx=0).stop 2023-07-18T03:21:13.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.994+0000 7fdada7ff700 1 -- 172.21.15.101:0/2083071702 shutdown_connections 2023-07-18T03:21:13.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.994+0000 7fdada7ff700 1 --2- 172.21.15.101:0/2083071702 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdad4105af0 0x7fdad41b0780 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:13.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.994+0000 7fdada7ff700 1 --2- 172.21.15.101:0/2083071702 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdad40ffae0 0x7fdad406a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:13.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.994+0000 7fdada7ff700 1 --2- 172.21.15.101:0/2083071702 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fdabc0a1400 0x7fdabc0a38b0 secure :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdac80098e0 tx=0x7fdac8000bc0 comp rx=0 tx=0).stop 2023-07-18T03:21:13.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.994+0000 7fdada7ff700 1 --2- 172.21.15.101:0/2083071702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdad4100490 0x7fdad406aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:13.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.994+0000 7fdada7ff700 1 -- 172.21.15.101:0/2083071702 >> 172.21.15.101:0/2083071702 conn(0x7fdad40fb5e0 msgr2=0x7fdad4106940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:13.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.996+0000 7fdada7ff700 1 -- 172.21.15.101:0/2083071702 shutdown_connections 2023-07-18T03:21:13.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:13.996+0000 7fdada7ff700 1 -- 172.21.15.101:0/2083071702 wait complete. 2023-07-18T03:21:14.014 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-18T03:21:14.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.278+0000 7fe3d83ff700 1 -- 172.21.15.101:0/873499525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe3d010c3b0 msgr2=0x7fe3d010e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:14.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.278+0000 7fe3d83ff700 1 --2- 172.21.15.101:0/873499525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe3d010c3b0 0x7fe3d010e790 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe3c800b0a0 tx=0x7fe3c800b3b0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.278+0000 7fe3d83ff700 1 -- 172.21.15.101:0/873499525 shutdown_connections 2023-07-18T03:21:14.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.278+0000 7fe3d83ff700 1 --2- 172.21.15.101:0/873499525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3d0072830 0x7fe3d0072c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.278+0000 7fe3d83ff700 1 --2- 172.21.15.101:0/873499525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe3d010c3b0 0x7fe3d010e790 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.278+0000 7fe3d83ff700 1 --2- 172.21.15.101:0/873499525 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe3d0073140 0x7fe3d010be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.278+0000 7fe3d83ff700 1 -- 172.21.15.101:0/873499525 >> 172.21.15.101:0/873499525 conn(0x7fe3d00fb650 msgr2=0x7fe3d00fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:14.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.278+0000 7fe3d83ff700 1 -- 172.21.15.101:0/873499525 shutdown_connections 2023-07-18T03:21:14.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.278+0000 7fe3d83ff700 1 -- 172.21.15.101:0/873499525 wait complete. 2023-07-18T03:21:14.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.279+0000 7fe3d83ff700 1 Processor -- start 2023-07-18T03:21:14.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.280+0000 7fe3d83ff700 1 -- start start 2023-07-18T03:21:14.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.280+0000 7fe3d83ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3d0072830 0x7fe3d0197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:14.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.280+0000 7fe3d83ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe3d0073140 0x7fe3d0198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:14.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.281+0000 7fe3d619b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3d0072830 0x7fe3d0197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:14.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.281+0000 7fe3d619b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3d0072830 0x7fe3d0197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60034/0 (socket says 172.21.15.101:60034) 2023-07-18T03:21:14.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.281+0000 7fe3d83ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe3d010c3b0 0x7fe3d019c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:14.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.281+0000 7fe3d83ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d019cec0 con 0x7fe3d0073140 2023-07-18T03:21:14.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.281+0000 7fe3d83ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d019d030 con 0x7fe3d0072830 2023-07-18T03:21:14.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.281+0000 7fe3d83ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d019d1f0 con 0x7fe3d010c3b0 2023-07-18T03:21:14.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.281+0000 7fe3d619b700 1 -- 172.21.15.101:0/3106032251 learned_addr learned my addr 172.21.15.101:0/3106032251 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:14.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.281+0000 7fe3d599a700 1 --2- 172.21.15.101:0/3106032251 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe3d0073140 0x7fe3d0198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:14.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.281+0000 7fe3d699c700 1 --2- 172.21.15.101:0/3106032251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe3d010c3b0 0x7fe3d019c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:14.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.281+0000 7fe3d619b700 1 -- 172.21.15.101:0/3106032251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe3d010c3b0 msgr2=0x7fe3d019c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:14.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.281+0000 7fe3d619b700 1 --2- 172.21.15.101:0/3106032251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe3d010c3b0 0x7fe3d019c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.281+0000 7fe3d619b700 1 -- 172.21.15.101:0/3106032251 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe3d0073140 msgr2=0x7fe3d0198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:14.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.281+0000 7fe3d619b700 1 --2- 172.21.15.101:0/3106032251 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe3d0073140 0x7fe3d0198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.281+0000 7fe3d619b700 1 -- 172.21.15.101:0/3106032251 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c8009d00 con 0x7fe3d0072830 2023-07-18T03:21:14.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.281+0000 7fe3d699c700 1 --2- 172.21.15.101:0/3106032251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe3d010c3b0 0x7fe3d019c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:21:14.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.281+0000 7fe3d599a700 1 --2- 172.21.15.101:0/3106032251 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe3d0073140 0x7fe3d0198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:21:14.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.282+0000 7fe3d619b700 1 --2- 172.21.15.101:0/3106032251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3d0072830 0x7fe3d0197ee0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc00e9a0 tx=0x7fe3cc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:14.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.283+0000 7fe3c77fe700 1 -- 172.21.15.101:0/3106032251 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc009790 con 0x7fe3d0072830 2023-07-18T03:21:14.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.283+0000 7fe3c77fe700 1 -- 172.21.15.101:0/3106032251 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3cc010430 con 0x7fe3d0072830 2023-07-18T03:21:14.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.283+0000 7fe3c77fe700 1 -- 172.21.15.101:0/3106032251 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc018680 con 0x7fe3d0072830 2023-07-18T03:21:14.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.283+0000 7fe3d83ff700 1 -- 172.21.15.101:0/3106032251 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3d019d4d0 con 0x7fe3d0072830 2023-07-18T03:21:14.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.283+0000 7fe3d83ff700 1 -- 172.21.15.101:0/3106032251 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3d01a4d10 con 0x7fe3d0072830 2023-07-18T03:21:14.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.287+0000 7fe3ebca3b80 1 Processor -- start 2023-07-18T03:21:14.284 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.288+0000 7fe3ebca3b80 1 -- start start 2023-07-18T03:21:14.284 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.288+0000 7fe3c77fe700 1 -- 172.21.15.101:0/3106032251 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fe3cc0105a0 con 0x7fe3d0072830 2023-07-18T03:21:14.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.289+0000 7fe3ebca3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e9e60ff2d0 0x55e9e60ff6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:14.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.289+0000 7fe3ebca3b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9e60ffbe0 0x55e9e610a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:14.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.289+0000 7fe3ebca3b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e9e610aad0 0x55e9e610ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:14.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.289+0000 7fe3ebca3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9e6005cb0 con 0x55e9e60ff2d0 2023-07-18T03:21:14.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.289+0000 7fe3ebca3b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9e6058a40 con 0x55e9e60ffbe0 2023-07-18T03:21:14.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.289+0000 7fe3ebca3b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9e6051ed0 con 0x55e9e610aad0 2023-07-18T03:21:14.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.289+0000 7fe3d619b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9e60ffbe0 0x55e9e610a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:14.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.289+0000 7fe3d599a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e9e610aad0 0x55e9e610ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:14.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.289+0000 7fe3d599a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e9e610aad0 0x55e9e610ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:53456/0 (socket says 172.21.15.101:53456) 2023-07-18T03:21:14.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.289+0000 7fe3d599a700 1 -- 172.21.15.101:0/1774618553 learned_addr learned my addr 172.21.15.101:0/1774618553 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:14.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.290+0000 7fe3d699c700 1 --2- 172.21.15.101:0/1774618553 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e9e60ff2d0 0x55e9e60ff6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:14.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.290+0000 7fe3d599a700 1 -- 172.21.15.101:0/1774618553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9e60ffbe0 msgr2=0x55e9e610a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:14.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.290+0000 7fe3d599a700 1 --2- 172.21.15.101:0/1774618553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9e60ffbe0 0x55e9e610a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.290+0000 7fe3d599a700 1 -- 172.21.15.101:0/1774618553 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e9e60ff2d0 msgr2=0x55e9e60ff6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:14.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.290+0000 7fe3d599a700 1 --2- 172.21.15.101:0/1774618553 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e9e60ff2d0 0x55e9e60ff6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.290+0000 7fe3d599a700 1 -- 172.21.15.101:0/1774618553 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9e5e7fde0 con 0x55e9e610aad0 2023-07-18T03:21:14.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.290+0000 7fe3d699c700 1 --2- 172.21.15.101:0/1774618553 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e9e60ff2d0 0x55e9e60ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:21:14.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.290+0000 7fe3d599a700 1 --2- 172.21.15.101:0/1774618553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e9e610aad0 0x55e9e610ceb0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe3c00114f0 tx=0x7fe3c0011800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1265b36f5c38415 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:14.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.290+0000 7fe3c77fe700 1 --2- 172.21.15.101:0/3106032251 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fe3bc0a1340 0x7fe3bc0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:14.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.291+0000 7fe3c77fe700 1 -- 172.21.15.101:0/3106032251 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fe3cc014070 con 0x7fe3d0072830 2023-07-18T03:21:14.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.291+0000 7fe3c4bfe700 1 -- 172.21.15.101:0/1774618553 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0004230 con 0x55e9e610aad0 2023-07-18T03:21:14.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.291+0000 7fe3c4bfe700 1 -- 172.21.15.101:0/1774618553 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3c0004390 con 0x55e9e610aad0 2023-07-18T03:21:14.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.291+0000 7fe3c4bfe700 1 -- 172.21.15.101:0/1774618553 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c000e920 con 0x55e9e610aad0 2023-07-18T03:21:14.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.291+0000 7fe3d699c700 1 --2- 172.21.15.101:0/3106032251 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fe3bc0a1340 0x7fe3bc0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:14.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.292+0000 7fe3ebca3b80 1 -- 172.21.15.101:0/1774618553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e9e610aad0 msgr2=0x55e9e610ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:14.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.292+0000 7fe3ebca3b80 1 --2- 172.21.15.101:0/1774618553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e9e610aad0 0x55e9e610ceb0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe3c00114f0 tx=0x7fe3c0011800 comp rx=0 tx=0).stop 2023-07-18T03:21:14.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.292+0000 7fe3ebca3b80 1 -- 172.21.15.101:0/1774618553 shutdown_connections 2023-07-18T03:21:14.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.292+0000 7fe3ebca3b80 1 --2- 172.21.15.101:0/1774618553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e9e610aad0 0x55e9e610ceb0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.292+0000 7fe3ebca3b80 1 --2- 172.21.15.101:0/1774618553 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e9e60ff2d0 0x55e9e60ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.292+0000 7fe3ebca3b80 1 --2- 172.21.15.101:0/1774618553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9e60ffbe0 0x55e9e610a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.292+0000 7fe3ebca3b80 1 -- 172.21.15.101:0/1774618553 >> 172.21.15.101:0/1774618553 conn(0x55e9e60f5da0 msgr2=0x55e9e60fb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:14.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.292+0000 7fe3d699c700 1 --2- 172.21.15.101:0/3106032251 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fe3bc0a1340 0x7fe3bc0a37f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe3c800b780 tx=0x7fe3c8006230 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:14.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.292+0000 7fe3ebca3b80 1 -- 172.21.15.101:0/1774618553 shutdown_connections 2023-07-18T03:21:14.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.293+0000 7fe3ebca3b80 1 -- 172.21.15.101:0/1774618553 wait complete. 2023-07-18T03:21:14.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.293+0000 7fe3ebca3b80 1 -- start start 2023-07-18T03:21:14.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.294+0000 7fe3ebca3b80 1 Processor -- start 2023-07-18T03:21:14.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.295+0000 7fe3ebca3b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e9e60ff2d0 0x55e9e610a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:14.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.295+0000 7fe3ebca3b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9e60ffbe0 0x55e9e6107770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:14.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.295+0000 7fe3ebca3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e9e610aad0 0x55e9e6107e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:14.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.295+0000 7fe3ebca3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9e5ff1ac0 con 0x55e9e610aad0 2023-07-18T03:21:14.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.295+0000 7fe3ebca3b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9e5ff17c0 con 0x55e9e60ffbe0 2023-07-18T03:21:14.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.295+0000 7fe3ebca3b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9e5ed5920 con 0x55e9e60ff2d0 2023-07-18T03:21:14.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.295+0000 7fe3ebca3b80 10 client.0 _ll_register_callbacks cb 0x55e9e5fef760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:21:14.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.295+0000 7fe3d599a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9e60ffbe0 0x55e9e6107770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:14.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.295+0000 7fe3d599a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9e60ffbe0 0x55e9e6107770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60052/0 (socket says 172.21.15.101:60052) 2023-07-18T03:21:14.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.295+0000 7fe3d599a700 1 -- 172.21.15.101:0/2102851895 learned_addr learned my addr 172.21.15.101:0/2102851895 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:14.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.296+0000 7fe3d599a700 1 -- 172.21.15.101:0/2102851895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e9e60ff2d0 msgr2=0x55e9e610a4f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-18T03:21:14.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.296+0000 7fe3d599a700 1 --2- 172.21.15.101:0/2102851895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e9e60ff2d0 0x55e9e610a4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.296+0000 7fe3d699c700 1 --2- 172.21.15.101:0/2102851895 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e9e610aad0 0x55e9e6107e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:14.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.296+0000 7fe3d599a700 1 -- 172.21.15.101:0/2102851895 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e9e610aad0 msgr2=0x55e9e6107e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:14.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.296+0000 7fe3d599a700 1 --2- 172.21.15.101:0/2102851895 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e9e610aad0 0x55e9e6107e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.296+0000 7fe3d599a700 1 -- 172.21.15.101:0/2102851895 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3c0009cd0 con 0x55e9e60ffbe0 2023-07-18T03:21:14.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.296+0000 7fe3d699c700 1 --2- 172.21.15.101:0/2102851895 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e9e610aad0 0x55e9e6107e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:21:14.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.296+0000 7fe3d599a700 1 --2- 172.21.15.101:0/2102851895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9e60ffbe0 0x55e9e6107770 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe3c00114c0 tx=0x7fe3c0004500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=503e438e8e92cb99 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:14.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.297+0000 7fe3b9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:21:14.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.297+0000 7fe3b9ffb700 1 -- 172.21.15.101:0/2102851895 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0004be0 con 0x55e9e60ffbe0 2023-07-18T03:21:14.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.297+0000 7fe3b9ffb700 1 -- 172.21.15.101:0/2102851895 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fe3c0004d40 con 0x55e9e60ffbe0 2023-07-18T03:21:14.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.297+0000 7fe3ebca3b80 1 -- 172.21.15.101:0/2102851895 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e9e5fab4f0 con 0x55e9e60ffbe0 2023-07-18T03:21:14.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.297+0000 7fe3b9ffb700 1 -- 172.21.15.101:0/2102851895 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3c000e580 con 0x55e9e60ffbe0 2023-07-18T03:21:14.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.297+0000 7fe3ebca3b80 10 client.28319 fetch_fsmap learned FSMap version 37 2023-07-18T03:21:14.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.297+0000 7fe3ebca3b80 1 -- 172.21.15.101:0/2102851895 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55e9e5e68c60 con 0x55e9e60ffbe0 2023-07-18T03:21:14.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.298+0000 7fe3b9ffb700 1 -- 172.21.15.101:0/2102851895 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3c0031030 con 0x55e9e60ffbe0 2023-07-18T03:21:14.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.298+0000 7fe3b9ffb700 1 -- 172.21.15.101:0/2102851895 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 37) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fe3c0028020 con 0x55e9e60ffbe0 2023-07-18T03:21:14.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.298+0000 7fe3ebca3b80 10 client.28319 fetch_fsmap finished waiting for FSMap version 37 2023-07-18T03:21:14.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.298+0000 7fe3ebca3b80 10 client.28319 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:21:14.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.298+0000 7fe3ebca3b80 20 client.28319 populate_metadata read hostname 'smithi101' 2023-07-18T03:21:14.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.298+0000 7fe3ebca3b80 1 --2- 172.21.15.101:0/2102851895 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55e9e6122a20 0x55e9e6124e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:14.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.298+0000 7fe3ebca3b80 4 client.28319 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:21:14.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.298+0000 7fe3ebca3b80 1 -- 172.21.15.101:0/2102851895 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e9e604aeb0 con 0x55e9e6122a20 2023-07-18T03:21:14.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.299+0000 7fe3d619b700 1 --2- 172.21.15.101:0/2102851895 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55e9e6122a20 0x55e9e6124e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:14.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.299+0000 7fe3d619b700 1 --2- 172.21.15.101:0/2102851895 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55e9e6122a20 0x55e9e6124e30 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=384a5c10d29c0cf4 server_cookie=744384ff2894ae36 in_seq=0 out_seq=0 2023-07-18T03:21:14.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.299+0000 7fe3b9ffb700 10 client.28319 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:21:14.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.302+0000 7fe3b9ffb700 1 -- 172.21.15.101:0/2102851895 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e9e604aeb0 con 0x55e9e6122a20 2023-07-18T03:21:14.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.302+0000 7fe3b9ffb700 10 client.28319 handle_command_reply: tid=0 2023-07-18T03:21:14.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.302+0000 7fe3ebca3b80 1 client.28319 shutdown 2023-07-18T03:21:14.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.302+0000 7fe3ebca3b80 1 -- 172.21.15.101:0/2102851895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9e60ffbe0 msgr2=0x55e9e6107770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:14.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.302+0000 7fe3ebca3b80 1 --2- 172.21.15.101:0/2102851895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9e60ffbe0 0x55e9e6107770 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe3c00114c0 tx=0x7fe3c0004500 comp rx=0 tx=0).stop 2023-07-18T03:21:14.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.302+0000 7fe3ebca3b80 1 -- 172.21.15.101:0/2102851895 shutdown_connections 2023-07-18T03:21:14.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.302+0000 7fe3ebca3b80 1 --2- 172.21.15.101:0/2102851895 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55e9e6122a20 0x55e9e6124e30 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.302+0000 7fe3ebca3b80 1 --2- 172.21.15.101:0/2102851895 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e9e610aad0 0x55e9e6107e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.302+0000 7fe3ebca3b80 1 --2- 172.21.15.101:0/2102851895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e9e60ff2d0 0x55e9e610a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.302+0000 7fe3ebca3b80 1 --2- 172.21.15.101:0/2102851895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9e60ffbe0 0x55e9e6107770 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.302+0000 7fe3ebca3b80 1 -- 172.21.15.101:0/2102851895 >> 172.21.15.101:0/2102851895 conn(0x55e9e60f5da0 msgr2=0x55e9e60fb230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:14.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.302+0000 7fe3b9ffb700 0 client.28319 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:21:14.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.302+0000 7fe3ebca3b80 1 -- 172.21.15.101:0/2102851895 shutdown_connections 2023-07-18T03:21:14.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.302+0000 7fe3ebca3b80 1 -- 172.21.15.101:0/2102851895 wait complete. 2023-07-18T03:21:14.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.303+0000 7fe3ebca3b80 20 client.28319 trim_cache size 0 max 16384 2023-07-18T03:21:14.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.320+0000 7fe3ebca3b80 1 Processor -- start 2023-07-18T03:21:14.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.321+0000 7fe3ebca3b80 1 -- start start 2023-07-18T03:21:14.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.321+0000 7fe3ebca3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e9e60f5da0 0x55e9e6102a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:14.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.321+0000 7fe3ebca3b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9e60ff2d0 0x55e9e6102fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:14.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.321+0000 7fe3ebca3b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e9e60ffbe0 0x55e9e6106f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:14.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.321+0000 7fe3ebca3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9e60f7ec0 con 0x55e9e60f5da0 2023-07-18T03:21:14.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.321+0000 7fe3ebca3b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9e6108720 con 0x55e9e60ff2d0 2023-07-18T03:21:14.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.321+0000 7fe3ebca3b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9e6100540 con 0x55e9e60ffbe0 2023-07-18T03:21:14.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.321+0000 7fe3d599a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9e60ff2d0 0x55e9e6102fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:14.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.321+0000 7fe3d619b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e9e60f5da0 0x55e9e6102a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:14.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.321+0000 7fe3d599a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9e60ff2d0 0x55e9e6102fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60068/0 (socket says 172.21.15.101:60068) 2023-07-18T03:21:14.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.321+0000 7fe3d699c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e9e60ffbe0 0x55e9e6106f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:14.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.321+0000 7fe3d599a700 1 -- 172.21.15.101:0/3131938622 learned_addr learned my addr 172.21.15.101:0/3131938622 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:14.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.322+0000 7fe3d599a700 1 -- 172.21.15.101:0/3131938622 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e9e60ffbe0 msgr2=0x55e9e6106f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:14.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.322+0000 7fe3d599a700 1 --2- 172.21.15.101:0/3131938622 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e9e60ffbe0 0x55e9e6106f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.322+0000 7fe3d599a700 1 -- 172.21.15.101:0/3131938622 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e9e60f5da0 msgr2=0x55e9e6102a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:14.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.322+0000 7fe3d599a700 1 --2- 172.21.15.101:0/3131938622 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e9e60f5da0 0x55e9e6102a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.322+0000 7fe3d599a700 1 -- 172.21.15.101:0/3131938622 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9e60f75e0 con 0x55e9e60ff2d0 2023-07-18T03:21:14.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.322+0000 7fe3d619b700 1 --2- 172.21.15.101:0/3131938622 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e9e60f5da0 0x55e9e6102a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:21:14.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.322+0000 7fe3d599a700 1 --2- 172.21.15.101:0/3131938622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9e60ff2d0 0x55e9e6102fd0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0021b60 tx=0x7fe3c0011800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5adb49bd7a20ad12 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:14.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.322+0000 7fe3c53ff700 1 -- 172.21.15.101:0/3131938622 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0004620 con 0x55e9e60ff2d0 2023-07-18T03:21:14.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.322+0000 7fe3c53ff700 1 -- 172.21.15.101:0/3131938622 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3c0031030 con 0x55e9e60ff2d0 2023-07-18T03:21:14.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.322+0000 7fe3c53ff700 1 -- 172.21.15.101:0/3131938622 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0022430 con 0x55e9e60ff2d0 2023-07-18T03:21:14.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.323+0000 7fe3ebca3b80 1 -- 172.21.15.101:0/3131938622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9e60ff2d0 msgr2=0x55e9e6102fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:14.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.323+0000 7fe3ebca3b80 1 --2- 172.21.15.101:0/3131938622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9e60ff2d0 0x55e9e6102fd0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0021b60 tx=0x7fe3c0011800 comp rx=0 tx=0).stop 2023-07-18T03:21:14.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.323+0000 7fe3ebca3b80 1 -- 172.21.15.101:0/3131938622 shutdown_connections 2023-07-18T03:21:14.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.323+0000 7fe3ebca3b80 1 --2- 172.21.15.101:0/3131938622 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e9e60ffbe0 0x55e9e6106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.323+0000 7fe3ebca3b80 1 --2- 172.21.15.101:0/3131938622 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e9e60f5da0 0x55e9e6102a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.323+0000 7fe3ebca3b80 1 --2- 172.21.15.101:0/3131938622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9e60ff2d0 0x55e9e6102fd0 secure :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0021b60 tx=0x7fe3c0011800 comp rx=0 tx=0).stop 2023-07-18T03:21:14.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.323+0000 7fe3ebca3b80 1 -- 172.21.15.101:0/3131938622 >> 172.21.15.101:0/3131938622 conn(0x55e9e60fa170 msgr2=0x55e9e6124a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:14.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.324+0000 7fe3ebca3b80 1 -- 172.21.15.101:0/3131938622 shutdown_connections 2023-07-18T03:21:14.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.324+0000 7fe3ebca3b80 1 -- 172.21.15.101:0/3131938622 wait complete. 2023-07-18T03:21:14.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.325+0000 7fe3ebca3b80 1 -- start start 2023-07-18T03:21:14.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.325+0000 7fe3ebca3b80 1 Processor -- start 2023-07-18T03:21:14.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.326+0000 7fe3ebca3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e9e60f5da0 0x55e9e6119670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:14.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.326+0000 7fe3ebca3b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9e60ffbe0 0x55e9e6119bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:14.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.326+0000 7fe3ebca3b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e9e610aad0 0x55e9e6117d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:14.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.326+0000 7fe3ebca3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9e6118380 con 0x55e9e60f5da0 2023-07-18T03:21:14.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.326+0000 7fe3ebca3b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9e61184f0 con 0x55e9e60ffbe0 2023-07-18T03:21:14.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.326+0000 7fe3ebca3b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9e61186b0 con 0x55e9e610aad0 2023-07-18T03:21:14.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.326+0000 7fe3ebca3b80 10 client.0 _ll_register_callbacks cb 0x7fe3cc0d0a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:21:14.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.326+0000 7fe3d699c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e9e610aad0 0x55e9e6117d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:14.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.326+0000 7fe3d699c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e9e610aad0 0x55e9e6117d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:53472/0 (socket says 172.21.15.101:53472) 2023-07-18T03:21:14.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.326+0000 7fe3d699c700 1 -- 172.21.15.101:0/1130727033 learned_addr learned my addr 172.21.15.101:0/1130727033 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:14.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.327+0000 7fe3d699c700 1 -- 172.21.15.101:0/1130727033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9e60ffbe0 msgr2=0x55e9e6119bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:14.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.327+0000 7fe3d699c700 1 --2- 172.21.15.101:0/1130727033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9e60ffbe0 0x55e9e6119bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.327+0000 7fe3d699c700 1 -- 172.21.15.101:0/1130727033 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e9e60f5da0 msgr2=0x55e9e6119670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:14.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.327+0000 7fe3d699c700 1 --2- 172.21.15.101:0/1130727033 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e9e60f5da0 0x55e9e6119670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.327+0000 7fe3d699c700 1 -- 172.21.15.101:0/1130727033 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3d01a4d10 con 0x55e9e610aad0 2023-07-18T03:21:14.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.327+0000 7fe3d699c700 1 --2- 172.21.15.101:0/1130727033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e9e610aad0 0x55e9e6117d60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe3c800b0a0 tx=0x7fe3c800b3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de2af8c635bcf142 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:14.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.327+0000 7fe3b9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:21:14.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.328+0000 7fe3b9ffb700 1 -- 172.21.15.101:0/1130727033 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c800e040 con 0x55e9e610aad0 2023-07-18T03:21:14.327 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.328+0000 7fe3b9ffb700 1 -- 172.21.15.101:0/1130727033 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fe3c80054d0 con 0x55e9e610aad0 2023-07-18T03:21:14.327 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.329+0000 7fe3ebca3b80 1 -- 172.21.15.101:0/1130727033 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe3c0004390 con 0x55e9e610aad0 2023-07-18T03:21:14.327 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.329+0000 7fe3b9ffb700 1 -- 172.21.15.101:0/1130727033 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3c8007980 con 0x55e9e610aad0 2023-07-18T03:21:14.327 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.330+0000 7fe3ebca3b80 10 client.28357 fetch_fsmap learned FSMap version 37 2023-07-18T03:21:14.327 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.330+0000 7fe3ebca3b80 1 -- 172.21.15.101:0/1130727033 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55e9e5e68c60 con 0x55e9e610aad0 2023-07-18T03:21:14.328 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.331+0000 7fe3b9ffb700 1 -- 172.21.15.101:0/1130727033 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3c8039030 con 0x55e9e610aad0 2023-07-18T03:21:14.328 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.332+0000 7fe3b9ffb700 1 -- 172.21.15.101:0/1130727033 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 37) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fe3c8030020 con 0x55e9e610aad0 2023-07-18T03:21:14.328 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.332+0000 7fe3ebca3b80 10 client.28357 fetch_fsmap finished waiting for FSMap version 37 2023-07-18T03:21:14.328 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.332+0000 7fe3ebca3b80 10 client.28357 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:21:14.328 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.332+0000 7fe3ebca3b80 20 client.28357 populate_metadata read hostname 'smithi101' 2023-07-18T03:21:14.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.332+0000 7fe3ebca3b80 1 --2- 172.21.15.101:0/1130727033 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55e9e6177980 0x55e9e6179dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:14.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.332+0000 7fe3ebca3b80 4 client.28357 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:21:14.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.332+0000 7fe3ebca3b80 1 -- 172.21.15.101:0/1130727033 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e9e600d050 con 0x55e9e6177980 2023-07-18T03:21:14.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.333+0000 7fe3d619b700 1 --2- 172.21.15.101:0/1130727033 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55e9e6177980 0x55e9e6179dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:14.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.334+0000 7fe3d619b700 1 --2- 172.21.15.101:0/1130727033 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55e9e6177980 0x55e9e6179dd0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3839c4035001307c server_cookie=6684b98c0d51f7df in_seq=0 out_seq=0 2023-07-18T03:21:14.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.334+0000 7fe3b9ffb700 10 client.28357 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:21:14.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.337+0000 7fe3b9ffb700 1 -- 172.21.15.101:0/1130727033 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e9e600d050 con 0x55e9e6177980 2023-07-18T03:21:14.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.337+0000 7fe3b9ffb700 10 client.28357 handle_command_reply: tid=0 2023-07-18T03:21:14.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.337+0000 7fe3ebca3b80 1 client.28357 shutdown 2023-07-18T03:21:14.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.337+0000 7fe3ebca3b80 1 -- 172.21.15.101:0/1130727033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e9e610aad0 msgr2=0x55e9e6117d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:14.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.337+0000 7fe3ebca3b80 1 --2- 172.21.15.101:0/1130727033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e9e610aad0 0x55e9e6117d60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe3c800b0a0 tx=0x7fe3c800b3b0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.338+0000 7fe3ebca3b80 1 -- 172.21.15.101:0/1130727033 shutdown_connections 2023-07-18T03:21:14.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.338+0000 7fe3ebca3b80 1 --2- 172.21.15.101:0/1130727033 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55e9e6177980 0x55e9e6179dd0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.338+0000 7fe3ebca3b80 1 --2- 172.21.15.101:0/1130727033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e9e610aad0 0x55e9e6117d60 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.338+0000 7fe3ebca3b80 1 --2- 172.21.15.101:0/1130727033 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e9e60f5da0 0x55e9e6119670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.338+0000 7fe3ebca3b80 1 --2- 172.21.15.101:0/1130727033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e9e60ffbe0 0x55e9e6119bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.338+0000 7fe3ebca3b80 1 -- 172.21.15.101:0/1130727033 >> 172.21.15.101:0/1130727033 conn(0x55e9e60fa170 msgr2=0x55e9e6102a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:14.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.338+0000 7fe3b9ffb700 0 client.28357 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:21:14.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.338+0000 7fe3ebca3b80 1 -- 172.21.15.101:0/1130727033 shutdown_connections 2023-07-18T03:21:14.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.338+0000 7fe3ebca3b80 1 -- 172.21.15.101:0/1130727033 wait complete. 2023-07-18T03:21:14.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.338+0000 7fe3ebca3b80 20 client.28357 trim_cache size 0 max 16384 2023-07-18T03:21:14.336 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:21:14.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.339+0000 7fe3d83ff700 1 -- 172.21.15.101:0/3106032251 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fe3bc0a1340 msgr2=0x7fe3bc0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:14.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.339+0000 7fe3d83ff700 1 --2- 172.21.15.101:0/3106032251 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fe3bc0a1340 0x7fe3bc0a37f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe3c800b780 tx=0x7fe3c8006230 comp rx=0 tx=0).stop 2023-07-18T03:21:14.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.339+0000 7fe3d83ff700 1 -- 172.21.15.101:0/3106032251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3d0072830 msgr2=0x7fe3d0197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:14.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.339+0000 7fe3d83ff700 1 --2- 172.21.15.101:0/3106032251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3d0072830 0x7fe3d0197ee0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc00e9a0 tx=0x7fe3cc00ecb0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.340+0000 7fe3d83ff700 1 -- 172.21.15.101:0/3106032251 shutdown_connections 2023-07-18T03:21:14.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.340+0000 7fe3d83ff700 1 --2- 172.21.15.101:0/3106032251 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fe3bc0a1340 0x7fe3bc0a37f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.338 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.340+0000 7fe3d83ff700 1 --2- 172.21.15.101:0/3106032251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3d0072830 0x7fe3d0197ee0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.338 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.340+0000 7fe3d83ff700 1 --2- 172.21.15.101:0/3106032251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe3d010c3b0 0x7fe3d019c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.338 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.340+0000 7fe3d83ff700 1 --2- 172.21.15.101:0/3106032251 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe3d0073140 0x7fe3d0198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.338 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.340+0000 7fe3d83ff700 1 -- 172.21.15.101:0/3106032251 >> 172.21.15.101:0/3106032251 conn(0x7fe3d00fb650 msgr2=0x7fe3d0103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:14.338 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.340+0000 7fe3d83ff700 1 -- 172.21.15.101:0/3106032251 shutdown_connections 2023-07-18T03:21:14.338 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.340+0000 7fe3d83ff700 1 -- 172.21.15.101:0/3106032251 wait complete. 2023-07-18T03:21:14.357 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:14 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2011940353' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:21:14.357 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:14 smithi101 ceph-mon[126379]: osdmap e108: 12 total, 12 up, 12 in 2023-07-18T03:21:14.359 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-18T03:21:14.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:14 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/2011940353' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:21:14.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:14 smithi155 ceph-mon[128885]: osdmap e108: 12 total, 12 up, 12 in 2023-07-18T03:21:14.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.628+0000 7f7947bff700 1 -- 172.21.15.101:0/3557601268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7940105af0 msgr2=0x7f7940105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:14.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.628+0000 7f7947bff700 1 --2- 172.21.15.101:0/3557601268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7940105af0 0x7f7940105ec0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f793c009990 tx=0x7f793c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.628+0000 7f7947bff700 1 -- 172.21.15.101:0/3557601268 shutdown_connections 2023-07-18T03:21:14.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.628+0000 7f7947bff700 1 --2- 172.21.15.101:0/3557601268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7940105af0 0x7f7940105ec0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.628+0000 7f7947bff700 1 --2- 172.21.15.101:0/3557601268 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7940100490 0x7f794010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.628+0000 7f7947bff700 1 --2- 172.21.15.101:0/3557601268 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79400ffae0 0x7f79400fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.628+0000 7f7947bff700 1 -- 172.21.15.101:0/3557601268 >> 172.21.15.101:0/3557601268 conn(0x7f79400fb5e0 msgr2=0x7f79400fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:14.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.628+0000 7f7947bff700 1 -- 172.21.15.101:0/3557601268 shutdown_connections 2023-07-18T03:21:14.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.628+0000 7f7947bff700 1 -- 172.21.15.101:0/3557601268 wait complete. 2023-07-18T03:21:14.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.630+0000 7f7947bff700 1 Processor -- start 2023-07-18T03:21:14.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.630+0000 7f7947bff700 1 -- start start 2023-07-18T03:21:14.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.630+0000 7f7947bff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79400ffae0 0x7f7940197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:14.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.630+0000 7f7947bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7940100490 0x7f7940198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:14.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.630+0000 7f7947bff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7940105af0 0x7f794019c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:14.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.630+0000 7f7947bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794019cde0 con 0x7f7940100490 2023-07-18T03:21:14.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.630+0000 7f7947bff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794019cf50 con 0x7f79400ffae0 2023-07-18T03:21:14.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.630+0000 7f7947bff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794019d110 con 0x7f7940105af0 2023-07-18T03:21:14.628 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.630+0000 7f794599b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79400ffae0 0x7f7940197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:14.628 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.631+0000 7f794599b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79400ffae0 0x7f7940197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60098/0 (socket says 172.21.15.101:60098) 2023-07-18T03:21:14.628 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.631+0000 7f794599b700 1 -- 172.21.15.101:0/79328507 learned_addr learned my addr 172.21.15.101:0/79328507 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:14.628 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.631+0000 7f794619c700 1 --2- 172.21.15.101:0/79328507 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7940105af0 0x7f794019c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:14.628 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.631+0000 7f794519a700 1 --2- 172.21.15.101:0/79328507 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7940100490 0x7f7940198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:14.629 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.631+0000 7f794599b700 1 -- 172.21.15.101:0/79328507 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7940105af0 msgr2=0x7f794019c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:14.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.631+0000 7f794599b700 1 --2- 172.21.15.101:0/79328507 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7940105af0 0x7f794019c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.631+0000 7f794599b700 1 -- 172.21.15.101:0/79328507 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7940100490 msgr2=0x7f7940198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:14.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.631+0000 7f794599b700 1 --2- 172.21.15.101:0/79328507 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7940100490 0x7f7940198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.631+0000 7f794599b700 1 -- 172.21.15.101:0/79328507 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f793c0095f0 con 0x7f79400ffae0 2023-07-18T03:21:14.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.631+0000 7f794599b700 1 --2- 172.21.15.101:0/79328507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79400ffae0 0x7f7940197e00 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f793c005910 tx=0x7f793c005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:14.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.631+0000 7f7936ffd700 1 -- 172.21.15.101:0/79328507 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793c01d070 con 0x7f79400ffae0 2023-07-18T03:21:14.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.632+0000 7f7936ffd700 1 -- 172.21.15.101:0/79328507 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f793c022470 con 0x7f79400ffae0 2023-07-18T03:21:14.631 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.632+0000 7f7936ffd700 1 -- 172.21.15.101:0/79328507 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793c017650 con 0x7f79400ffae0 2023-07-18T03:21:14.631 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.632+0000 7f7947bff700 1 -- 172.21.15.101:0/79328507 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f794019d390 con 0x7f79400ffae0 2023-07-18T03:21:14.631 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.632+0000 7f7947bff700 1 -- 172.21.15.101:0/79328507 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f794019d610 con 0x7f79400ffae0 2023-07-18T03:21:14.631 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.635+0000 7f7936ffd700 1 -- 172.21.15.101:0/79328507 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f793c0225e0 con 0x7f79400ffae0 2023-07-18T03:21:14.632 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.636+0000 7f795b4b4b80 1 Processor -- start 2023-07-18T03:21:14.632 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.636+0000 7f795b4b4b80 1 -- start start 2023-07-18T03:21:14.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.636+0000 7f795b4b4b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e5791262d0 0x55e5791266a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:14.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.636+0000 7f795b4b4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e579126be0 0x55e579131590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:14.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.637+0000 7f795b4b4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e579131ad0 0x55e579133eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:14.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.637+0000 7f795b4b4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e57902ccb0 con 0x55e579131ad0 2023-07-18T03:21:14.634 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.637+0000 7f795b4b4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e57907fa40 con 0x55e579126be0 2023-07-18T03:21:14.634 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.637+0000 7f795b4b4b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e579078ed0 con 0x55e5791262d0 2023-07-18T03:21:14.634 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.637+0000 7f794599b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e579126be0 0x55e579131590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:14.634 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.637+0000 7f794619c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e5791262d0 0x55e5791266a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:14.634 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.637+0000 7f794619c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e5791262d0 0x55e5791266a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:53508/0 (socket says 172.21.15.101:53508) 2023-07-18T03:21:14.634 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.637+0000 7f794619c700 1 -- 172.21.15.101:0/406691710 learned_addr learned my addr 172.21.15.101:0/406691710 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:14.634 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.637+0000 7f794519a700 1 --2- 172.21.15.101:0/406691710 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e579131ad0 0x55e579133eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:14.636 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.637+0000 7f794619c700 1 -- 172.21.15.101:0/406691710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e579126be0 msgr2=0x55e579131590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:14.636 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.637+0000 7f794619c700 1 --2- 172.21.15.101:0/406691710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e579126be0 0x55e579131590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.636 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.637+0000 7f794619c700 1 -- 172.21.15.101:0/406691710 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e579131ad0 msgr2=0x55e579133eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:14.636 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.637+0000 7f794619c700 1 --2- 172.21.15.101:0/406691710 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e579131ad0 0x55e579133eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.636 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.637+0000 7f794619c700 1 -- 172.21.15.101:0/406691710 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e578ea6de0 con 0x55e5791262d0 2023-07-18T03:21:14.636 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.637+0000 7f794519a700 1 --2- 172.21.15.101:0/406691710 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e579131ad0 0x55e579133eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:21:14.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.638+0000 7f7936ffd700 1 --2- 172.21.15.101:0/79328507 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f792c0a1340 0x7f792c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:14.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.638+0000 7f794619c700 1 --2- 172.21.15.101:0/406691710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e5791262d0 0x55e5791266a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7938011680 tx=0x7f7938009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67fc238915131382 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:14.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.638+0000 7f794599b700 1 --2- 172.21.15.101:0/79328507 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f792c0a1340 0x7f792c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:14.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.638+0000 7f792bfff700 1 -- 172.21.15.101:0/406691710 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793800e040 con 0x55e5791262d0 2023-07-18T03:21:14.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.638+0000 7f792bfff700 1 -- 172.21.15.101:0/406691710 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f793800f040 con 0x55e5791262d0 2023-07-18T03:21:14.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.638+0000 7f792bfff700 1 -- 172.21.15.101:0/406691710 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7938004850 con 0x55e5791262d0 2023-07-18T03:21:14.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.638+0000 7f7936ffd700 1 -- 172.21.15.101:0/79328507 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f793c0ce200 con 0x7f79400ffae0 2023-07-18T03:21:14.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.639+0000 7f794599b700 1 --2- 172.21.15.101:0/79328507 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f792c0a1340 0x7f792c0a37f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f793c006010 tx=0x7f793c0d1c70 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:14.638 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.639+0000 7f795b4b4b80 1 -- 172.21.15.101:0/406691710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e5791262d0 msgr2=0x55e5791266a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:14.638 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.639+0000 7f795b4b4b80 1 --2- 172.21.15.101:0/406691710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e5791262d0 0x55e5791266a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7938011680 tx=0x7f7938009920 comp rx=0 tx=0).stop 2023-07-18T03:21:14.638 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.639+0000 7f795b4b4b80 1 -- 172.21.15.101:0/406691710 shutdown_connections 2023-07-18T03:21:14.638 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.639+0000 7f795b4b4b80 1 --2- 172.21.15.101:0/406691710 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e579131ad0 0x55e579133eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.638 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.639+0000 7f795b4b4b80 1 --2- 172.21.15.101:0/406691710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e5791262d0 0x55e5791266a0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.639 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.639+0000 7f795b4b4b80 1 --2- 172.21.15.101:0/406691710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e579126be0 0x55e579131590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.639 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.639+0000 7f795b4b4b80 1 -- 172.21.15.101:0/406691710 >> 172.21.15.101:0/406691710 conn(0x55e57911cda0 msgr2=0x55e5791226a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:14.639 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.639+0000 7f795b4b4b80 1 -- 172.21.15.101:0/406691710 shutdown_connections 2023-07-18T03:21:14.639 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.640+0000 7f795b4b4b80 1 -- 172.21.15.101:0/406691710 wait complete. 2023-07-18T03:21:14.641 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.640+0000 7f795b4b4b80 1 -- start start 2023-07-18T03:21:14.641 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.641+0000 7f795b4b4b80 1 Processor -- start 2023-07-18T03:21:14.641 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.641+0000 7f795b4b4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5791262d0 0x55e579131510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:14.642 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.641+0000 7f795b4b4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e579126be0 0x55e57912e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:14.642 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.641+0000 7f795b4b4b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e579131ad0 0x55e57912ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:14.642 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.641+0000 7f795b4b4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e579018ac0 con 0x55e579126be0 2023-07-18T03:21:14.642 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.641+0000 7f795b4b4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5790187c0 con 0x55e5791262d0 2023-07-18T03:21:14.642 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.641+0000 7f795b4b4b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e578efc920 con 0x55e579131ad0 2023-07-18T03:21:14.642 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.641+0000 7f795b4b4b80 10 client.0 _ll_register_callbacks cb 0x55e579016760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:21:14.642 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.641+0000 7f794619c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5791262d0 0x55e579131510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:14.642 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.641+0000 7f794619c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5791262d0 0x55e579131510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60126/0 (socket says 172.21.15.101:60126) 2023-07-18T03:21:14.643 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.641+0000 7f794619c700 1 -- 172.21.15.101:0/599924820 learned_addr learned my addr 172.21.15.101:0/599924820 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:14.643 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.642+0000 7f794519a700 1 --2- 172.21.15.101:0/599924820 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e579126be0 0x55e57912e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:14.643 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.642+0000 7f794619c700 1 --2- 172.21.15.101:0/599924820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e579131ad0 0x55e57912ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:14.643 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.642+0000 7f794619c700 1 -- 172.21.15.101:0/599924820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e579131ad0 msgr2=0x55e57912ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:14.643 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.642+0000 7f794619c700 1 --2- 172.21.15.101:0/599924820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e579131ad0 0x55e57912ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.643 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.642+0000 7f794619c700 1 -- 172.21.15.101:0/599924820 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e579126be0 msgr2=0x55e57912e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:14.643 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.642+0000 7f794619c700 1 --2- 172.21.15.101:0/599924820 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e579126be0 0x55e57912e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.644 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.642+0000 7f794619c700 1 -- 172.21.15.101:0/599924820 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7938011330 con 0x55e5791262d0 2023-07-18T03:21:14.644 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.642+0000 7f794619c700 1 --2- 172.21.15.101:0/599924820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5791262d0 0x55e579131510 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f7938006080 tx=0x7f79380112a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a14a494b22fd4350 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:14.644 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.642+0000 7f79297fa700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:21:14.644 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.642+0000 7f79297fa700 1 -- 172.21.15.101:0/599924820 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793800e070 con 0x55e5791262d0 2023-07-18T03:21:14.644 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.642+0000 7f795b4b4b80 1 -- 172.21.15.101:0/599924820 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e578fd24f0 con 0x55e5791262d0 2023-07-18T03:21:14.644 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.642+0000 7f79297fa700 1 -- 172.21.15.101:0/599924820 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f7938003d10 con 0x55e5791262d0 2023-07-18T03:21:14.644 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.643+0000 7f79297fa700 1 -- 172.21.15.101:0/599924820 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7938003ed0 con 0x55e5791262d0 2023-07-18T03:21:14.645 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.643+0000 7f795b4b4b80 10 client.28334 fetch_fsmap learned FSMap version 37 2023-07-18T03:21:14.645 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.643+0000 7f795b4b4b80 1 -- 172.21.15.101:0/599924820 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55e578e8fc60 con 0x55e5791262d0 2023-07-18T03:21:14.645 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.643+0000 7f79297fa700 1 -- 172.21.15.101:0/599924820 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e578fd24f0 con 0x55e5791262d0 2023-07-18T03:21:14.645 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.643+0000 7f79297fa700 1 -- 172.21.15.101:0/599924820 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 37) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f79380155e0 con 0x55e5791262d0 2023-07-18T03:21:14.645 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.643+0000 7f795b4b4b80 10 client.28334 fetch_fsmap finished waiting for FSMap version 37 2023-07-18T03:21:14.645 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.643+0000 7f795b4b4b80 10 client.28334 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:21:14.645 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.643+0000 7f795b4b4b80 20 client.28334 populate_metadata read hostname 'smithi101' 2023-07-18T03:21:14.645 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.643+0000 7f795b4b4b80 1 --2- 172.21.15.101:0/599924820 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55e579149ab0 0x55e57914bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:14.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.643+0000 7f795b4b4b80 4 client.28334 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:21:14.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.643+0000 7f795b4b4b80 1 -- 172.21.15.101:0/599924820 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e579071eb0 con 0x55e579149ab0 2023-07-18T03:21:14.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.644+0000 7f794519a700 1 --2- 172.21.15.101:0/599924820 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55e579149ab0 0x55e57914bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:14.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.644+0000 7f794519a700 1 --2- 172.21.15.101:0/599924820 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55e579149ab0 0x55e57914bec0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=292bb61cc4d3e744 server_cookie=6564f8c3fa7de4e8 in_seq=0 out_seq=0 2023-07-18T03:21:14.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.644+0000 7f79297fa700 10 client.28334 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:21:14.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.654+0000 7f79297fa700 1 -- 172.21.15.101:0/599924820 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e579071eb0 con 0x55e579149ab0 2023-07-18T03:21:14.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.654+0000 7f79297fa700 10 client.28334 handle_command_reply: tid=0 2023-07-18T03:21:14.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.654+0000 7f795b4b4b80 1 client.28334 shutdown 2023-07-18T03:21:14.651 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.654+0000 7f795b4b4b80 1 -- 172.21.15.101:0/599924820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5791262d0 msgr2=0x55e579131510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:14.651 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.654+0000 7f795b4b4b80 1 --2- 172.21.15.101:0/599924820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5791262d0 0x55e579131510 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f7938006080 tx=0x7f79380112a0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.651 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.654+0000 7f795b4b4b80 1 -- 172.21.15.101:0/599924820 shutdown_connections 2023-07-18T03:21:14.651 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.655+0000 7f795b4b4b80 1 --2- 172.21.15.101:0/599924820 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55e579149ab0 0x55e57914bec0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.651 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.655+0000 7f795b4b4b80 1 --2- 172.21.15.101:0/599924820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5791262d0 0x55e579131510 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.655+0000 7f795b4b4b80 1 --2- 172.21.15.101:0/599924820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e579131ad0 0x55e57912ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.655+0000 7f795b4b4b80 1 --2- 172.21.15.101:0/599924820 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e579126be0 0x55e57912e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.655+0000 7f795b4b4b80 1 -- 172.21.15.101:0/599924820 >> 172.21.15.101:0/599924820 conn(0x55e57911cda0 msgr2=0x55e579122230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:14.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.664+0000 7f79297fa700 0 client.28334 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:21:14.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.664+0000 7f795b4b4b80 1 -- 172.21.15.101:0/599924820 shutdown_connections 2023-07-18T03:21:14.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.664+0000 7f795b4b4b80 1 -- 172.21.15.101:0/599924820 wait complete. 2023-07-18T03:21:14.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.666+0000 7f795b4b4b80 20 client.28334 trim_cache size 0 max 16384 2023-07-18T03:21:14.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.695+0000 7f795b4b4b80 1 Processor -- start 2023-07-18T03:21:14.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.697+0000 7f795b4b4b80 1 -- start start 2023-07-18T03:21:14.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.698+0000 7f795b4b4b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e5791262d0 0x55e579131040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:14.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.698+0000 7f795b4b4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e579126be0 0x55e579129c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:14.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.698+0000 7f795b4b4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e579131ad0 0x55e57912e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:14.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.698+0000 7f795b4b4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5791297f0 con 0x55e579126be0 2023-07-18T03:21:14.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.698+0000 7f795b4b4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e57912f7e0 con 0x55e579131ad0 2023-07-18T03:21:14.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.698+0000 7f795b4b4b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e579119fb0 con 0x55e5791262d0 2023-07-18T03:21:14.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.698+0000 7f794619c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e5791262d0 0x55e579131040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:14.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.698+0000 7f794619c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e5791262d0 0x55e579131040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:53518/0 (socket says 172.21.15.101:53518) 2023-07-18T03:21:14.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.698+0000 7f794619c700 1 -- 172.21.15.101:0/805876275 learned_addr learned my addr 172.21.15.101:0/805876275 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:14.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.698+0000 7f794619c700 1 --2- 172.21.15.101:0/805876275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e579131ad0 0x55e57912e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:14.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.698+0000 7f794519a700 1 --2- 172.21.15.101:0/805876275 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e579126be0 0x55e579129c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:14.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.699+0000 7f794619c700 1 -- 172.21.15.101:0/805876275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e579131ad0 msgr2=0x55e57912e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:14.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.699+0000 7f794619c700 1 --2- 172.21.15.101:0/805876275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e579131ad0 0x55e57912e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.699+0000 7f794619c700 1 -- 172.21.15.101:0/805876275 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e579126be0 msgr2=0x55e579129c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:14.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.699+0000 7f794619c700 1 --2- 172.21.15.101:0/805876275 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e579126be0 0x55e579129c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.699+0000 7f794619c700 1 -- 172.21.15.101:0/805876275 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e578ecd250 con 0x55e5791262d0 2023-07-18T03:21:14.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.699+0000 7f794619c700 1 --2- 172.21.15.101:0/805876275 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e5791262d0 0x55e579131040 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x55e57911ca20 tx=0x7f7938000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb81713c105fd094 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:14.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.699+0000 7f7934bff700 1 -- 172.21.15.101:0/805876275 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793800e070 con 0x55e5791262d0 2023-07-18T03:21:14.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.699+0000 7f7934bff700 1 -- 172.21.15.101:0/805876275 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7938015c10 con 0x55e5791262d0 2023-07-18T03:21:14.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.699+0000 7f7934bff700 1 -- 172.21.15.101:0/805876275 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7938015740 con 0x55e5791262d0 2023-07-18T03:21:14.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.700+0000 7f795b4b4b80 1 -- 172.21.15.101:0/805876275 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e5791262d0 msgr2=0x55e579131040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:14.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.700+0000 7f795b4b4b80 1 --2- 172.21.15.101:0/805876275 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e5791262d0 0x55e579131040 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x55e57911ca20 tx=0x7f7938000f40 comp rx=0 tx=0).stop 2023-07-18T03:21:14.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.701+0000 7f795b4b4b80 1 -- 172.21.15.101:0/805876275 shutdown_connections 2023-07-18T03:21:14.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.701+0000 7f795b4b4b80 1 --2- 172.21.15.101:0/805876275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e579131ad0 0x55e57912e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.701+0000 7f795b4b4b80 1 --2- 172.21.15.101:0/805876275 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e5791262d0 0x55e579131040 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.701+0000 7f795b4b4b80 1 --2- 172.21.15.101:0/805876275 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e579126be0 0x55e579129c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.701+0000 7f795b4b4b80 1 -- 172.21.15.101:0/805876275 >> 172.21.15.101:0/805876275 conn(0x55e57911cda0 msgr2=0x55e57914bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:14.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.701+0000 7f795b4b4b80 1 -- 172.21.15.101:0/805876275 shutdown_connections 2023-07-18T03:21:14.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.701+0000 7f795b4b4b80 1 -- 172.21.15.101:0/805876275 wait complete. 2023-07-18T03:21:14.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.701+0000 7f795b4b4b80 1 -- start start 2023-07-18T03:21:14.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.702+0000 7f795b4b4b80 1 Processor -- start 2023-07-18T03:21:14.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.703+0000 7f795b4b4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5791262d0 0x55e57913eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:14.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.703+0000 7f795b4b4b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e579126be0 0x55e57913f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:14.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.703+0000 7f795b4b4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e579131ad0 0x55e57913f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:14.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.703+0000 7f795b4b4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e57919ca90 con 0x55e579131ad0 2023-07-18T03:21:14.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.703+0000 7f795b4b4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e57919cc00 con 0x55e5791262d0 2023-07-18T03:21:14.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.703+0000 7f795b4b4b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e57919cdc0 con 0x55e579126be0 2023-07-18T03:21:14.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.703+0000 7f795b4b4b80 10 client.0 _ll_register_callbacks cb 0x7f793c0d1b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:21:14.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.704+0000 7f794619c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5791262d0 0x55e57913eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:14.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.704+0000 7f794519a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e579126be0 0x55e57913f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:14.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.704+0000 7f794519a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e579126be0 0x55e57913f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:53528/0 (socket says 172.21.15.101:53528) 2023-07-18T03:21:14.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.704+0000 7f794619c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5791262d0 0x55e57913eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60140/0 (socket says 172.21.15.101:60140) 2023-07-18T03:21:14.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.704+0000 7f794519a700 1 -- 172.21.15.101:0/2429815358 learned_addr learned my addr 172.21.15.101:0/2429815358 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:14.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.704+0000 7f794619c700 1 --2- 172.21.15.101:0/2429815358 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e579131ad0 0x55e57913f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:14.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.705+0000 7f794519a700 1 -- 172.21.15.101:0/2429815358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5791262d0 msgr2=0x55e57913eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:14.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.705+0000 7f794519a700 1 --2- 172.21.15.101:0/2429815358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5791262d0 0x55e57913eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.705+0000 7f794519a700 1 -- 172.21.15.101:0/2429815358 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e579131ad0 msgr2=0x55e57913f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:14.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.705+0000 7f794519a700 1 --2- 172.21.15.101:0/2429815358 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e579131ad0 0x55e57913f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.705+0000 7f794519a700 1 -- 172.21.15.101:0/2429815358 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e578e8fc60 con 0x55e579126be0 2023-07-18T03:21:14.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.705+0000 7f794519a700 1 --2- 172.21.15.101:0/2429815358 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e579126be0 0x55e57913f1e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x55e57912fde0 tx=0x7f793001ed20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eaf02a46cf120f00 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:14.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.705+0000 7f79297fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:21:14.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:14 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/2011940353' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:21:14.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:14 smithi179 ceph-mon[129229]: osdmap e108: 12 total, 12 up, 12 in 2023-07-18T03:21:14.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.705+0000 7f79297fa700 1 -- 172.21.15.101:0/2429815358 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793001e240 con 0x55e579126be0 2023-07-18T03:21:14.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.705+0000 7f79297fa700 1 -- 172.21.15.101:0/2429815358 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f793001e3a0 con 0x55e579126be0 2023-07-18T03:21:14.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.705+0000 7f795b4b4b80 1 -- 172.21.15.101:0/2429815358 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f793800f040 con 0x55e579126be0 2023-07-18T03:21:14.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.707+0000 7f79297fa700 1 -- 172.21.15.101:0/2429815358 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f793001e560 con 0x55e579126be0 2023-07-18T03:21:14.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.707+0000 7f795b4b4b80 10 client.28375 fetch_fsmap learned FSMap version 37 2023-07-18T03:21:14.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.707+0000 7f795b4b4b80 1 -- 172.21.15.101:0/2429815358 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55e578ea52b0 con 0x55e579126be0 2023-07-18T03:21:14.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.707+0000 7f79297fa700 1 -- 172.21.15.101:0/2429815358 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f793800f040 con 0x55e579126be0 2023-07-18T03:21:14.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.709+0000 7f79297fa700 1 -- 172.21.15.101:0/2429815358 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 37) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f7930005400 con 0x55e579126be0 2023-07-18T03:21:14.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.709+0000 7f795b4b4b80 10 client.28375 fetch_fsmap finished waiting for FSMap version 37 2023-07-18T03:21:14.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.709+0000 7f795b4b4b80 10 client.28375 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:21:14.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.709+0000 7f795b4b4b80 20 client.28375 populate_metadata read hostname 'smithi101' 2023-07-18T03:21:14.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.709+0000 7f795b4b4b80 1 --2- 172.21.15.101:0/2429815358 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55e57919e550 0x55e5791a0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:14.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.709+0000 7f795b4b4b80 4 client.28375 mds_command: new command op to 24562 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:21:14.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.709+0000 7f795b4b4b80 1 -- 172.21.15.101:0/2429815358 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e579034050 con 0x55e57919e550 2023-07-18T03:21:14.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.711+0000 7f794619c700 1 --2- 172.21.15.101:0/2429815358 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55e57919e550 0x55e5791a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:14.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.711+0000 7f794619c700 1 --2- 172.21.15.101:0/2429815358 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55e57919e550 0x55e5791a0960 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26b5c50d346147c3 server_cookie=f91fc073c2accf35 in_seq=0 out_seq=0 2023-07-18T03:21:14.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.711+0000 7f79297fa700 10 client.28375 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:21:14.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.714+0000 7f79297fa700 1 -- 172.21.15.101:0/2429815358 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e579034050 con 0x55e57919e550 2023-07-18T03:21:14.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.714+0000 7f79297fa700 10 client.28375 handle_command_reply: tid=0 2023-07-18T03:21:14.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.714+0000 7f795b4b4b80 1 client.28375 shutdown 2023-07-18T03:21:14.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.714+0000 7f795b4b4b80 1 -- 172.21.15.101:0/2429815358 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e579126be0 msgr2=0x55e57913f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:14.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.714+0000 7f795b4b4b80 1 --2- 172.21.15.101:0/2429815358 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e579126be0 0x55e57913f1e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x55e57912fde0 tx=0x7f793001ed20 comp rx=0 tx=0).stop 2023-07-18T03:21:14.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.714+0000 7f795b4b4b80 1 -- 172.21.15.101:0/2429815358 shutdown_connections 2023-07-18T03:21:14.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.714+0000 7f795b4b4b80 1 --2- 172.21.15.101:0/2429815358 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55e57919e550 0x55e5791a0960 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.714+0000 7f795b4b4b80 1 --2- 172.21.15.101:0/2429815358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e5791262d0 0x55e57913eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.714+0000 7f795b4b4b80 1 --2- 172.21.15.101:0/2429815358 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e579131ad0 0x55e57913f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.714+0000 7f795b4b4b80 1 --2- 172.21.15.101:0/2429815358 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e579126be0 0x55e57913f1e0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.715+0000 7f795b4b4b80 1 -- 172.21.15.101:0/2429815358 >> 172.21.15.101:0/2429815358 conn(0x55e57911cda0 msgr2=0x55e579129c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:14.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.715+0000 7f79297fa700 0 client.28375 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:21:14.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.715+0000 7f795b4b4b80 1 -- 172.21.15.101:0/2429815358 shutdown_connections 2023-07-18T03:21:14.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.715+0000 7f795b4b4b80 1 -- 172.21.15.101:0/2429815358 wait complete. 2023-07-18T03:21:14.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.715+0000 7f795b4b4b80 20 client.28375 trim_cache size 0 max 16384 2023-07-18T03:21:14.713 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:21:14.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.716+0000 7f7947bff700 1 -- 172.21.15.101:0/79328507 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f792c0a1340 msgr2=0x7f792c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:14.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.716+0000 7f7947bff700 1 --2- 172.21.15.101:0/79328507 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f792c0a1340 0x7f792c0a37f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f793c006010 tx=0x7f793c0d1c70 comp rx=0 tx=0).stop 2023-07-18T03:21:14.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.716+0000 7f7947bff700 1 -- 172.21.15.101:0/79328507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79400ffae0 msgr2=0x7f7940197e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:14.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.716+0000 7f7947bff700 1 --2- 172.21.15.101:0/79328507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79400ffae0 0x7f7940197e00 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f793c005910 tx=0x7f793c005720 comp rx=0 tx=0).stop 2023-07-18T03:21:14.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.719+0000 7f7947bff700 1 -- 172.21.15.101:0/79328507 shutdown_connections 2023-07-18T03:21:14.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.719+0000 7f7947bff700 1 --2- 172.21.15.101:0/79328507 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f792c0a1340 0x7f792c0a37f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.719+0000 7f7947bff700 1 --2- 172.21.15.101:0/79328507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79400ffae0 0x7f7940197e00 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.719+0000 7f7947bff700 1 --2- 172.21.15.101:0/79328507 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7940105af0 0x7f794019c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.719+0000 7f7947bff700 1 --2- 172.21.15.101:0/79328507 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7940100490 0x7f7940198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:14.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.719+0000 7f7947bff700 1 -- 172.21.15.101:0/79328507 >> 172.21.15.101:0/79328507 conn(0x7f79400fb5e0 msgr2=0x7f794010ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:14.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.720+0000 7f7947bff700 1 -- 172.21.15.101:0/79328507 shutdown_connections 2023-07-18T03:21:14.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:14.721+0000 7f7947bff700 1 -- 172.21.15.101:0/79328507 wait complete. 2023-07-18T03:21:14.953 INFO:tasks.workunit.client.0.smithi101.stdout: 10 18008 29.09 MB/sec warmup 39 sec latency 8728.773 ms 2023-07-18T03:21:15.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:15 smithi155 ceph-mon[128885]: pgmap v428: 129 pgs: 129 active+clean; 950 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.8 MiB/s wr, 581 op/s 2023-07-18T03:21:15.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:15 smithi179 ceph-mon[129229]: pgmap v428: 129 pgs: 129 active+clean; 950 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.8 MiB/s wr, 581 op/s 2023-07-18T03:21:15.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:15 smithi101 ceph-mon[126379]: pgmap v428: 129 pgs: 129 active+clean; 950 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.8 MiB/s wr, 581 op/s 2023-07-18T03:21:15.747 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-18T03:21:15.748 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-18T03:21:15.955 INFO:tasks.workunit.client.0.smithi101.stdout: 10 18555 29.18 MB/sec warmup 40 sec latency 9587.169 ms 2023-07-18T03:21:16.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.068+0000 7fc4593ff700 1 -- 172.21.15.101:0/480127008 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc454110900 msgr2=0x7fc454112ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:16.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.068+0000 7fc4593ff700 1 --2- 172.21.15.101:0/480127008 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc454110900 0x7fc454112ce0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fc44c009f90 tx=0x7fc44c00b330 comp rx=0 tx=0).stop 2023-07-18T03:21:16.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.068+0000 7fc4593ff700 1 -- 172.21.15.101:0/480127008 shutdown_connections 2023-07-18T03:21:16.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.068+0000 7fc4593ff700 1 --2- 172.21.15.101:0/480127008 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc454110900 0x7fc454112ce0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.068+0000 7fc4593ff700 1 --2- 172.21.15.101:0/480127008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc454107350 0x7fc454107720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.068+0000 7fc4593ff700 1 --2- 172.21.15.101:0/480127008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc454107c60 0x7fc4541103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.068+0000 7fc4593ff700 1 -- 172.21.15.101:0/480127008 >> 172.21.15.101:0/480127008 conn(0x7fc454075a00 msgr2=0x7fc454077e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:16.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.070+0000 7fc4593ff700 1 -- 172.21.15.101:0/480127008 shutdown_connections 2023-07-18T03:21:16.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.071+0000 7fc4593ff700 1 -- 172.21.15.101:0/480127008 wait complete. 2023-07-18T03:21:16.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.072+0000 7fc4593ff700 1 Processor -- start 2023-07-18T03:21:16.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.072+0000 7fc4593ff700 1 -- start start 2023-07-18T03:21:16.070 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.074+0000 7fc4593ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc454107350 0x7fc45419c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:16.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.074+0000 7fc4593ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc454107c60 0x7fc45419c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:16.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.074+0000 7fc4593ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc454110900 0x7fc4541a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:16.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.074+0000 7fc4593ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4541a1240 con 0x7fc454110900 2023-07-18T03:21:16.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.074+0000 7fc4593ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4541a13b0 con 0x7fc454107350 2023-07-18T03:21:16.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.074+0000 7fc4593ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4541a1570 con 0x7fc454107c60 2023-07-18T03:21:16.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.075+0000 7fc4537fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc454110900 0x7fc4541a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:16.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.075+0000 7fc4537fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc454110900 0x7fc4541a0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:50702/0 (socket says 172.21.15.101:50702) 2023-07-18T03:21:16.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.075+0000 7fc4537fe700 1 -- 172.21.15.101:0/257483872 learned_addr learned my addr 172.21.15.101:0/257483872 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:16.072 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.076+0000 7fc4527fc700 1 --2- 172.21.15.101:0/257483872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc454107c60 0x7fc45419c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:16.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.076+0000 7fc4537fe700 1 -- 172.21.15.101:0/257483872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc454107c60 msgr2=0x7fc45419c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:16.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.076+0000 7fc4537fe700 1 --2- 172.21.15.101:0/257483872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc454107c60 0x7fc45419c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.076+0000 7fc452ffd700 1 --2- 172.21.15.101:0/257483872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc454107350 0x7fc45419c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:16.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.076+0000 7fc4537fe700 1 -- 172.21.15.101:0/257483872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc454107350 msgr2=0x7fc45419c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:16.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.076+0000 7fc4537fe700 1 --2- 172.21.15.101:0/257483872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc454107350 0x7fc45419c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.076+0000 7fc4537fe700 1 -- 172.21.15.101:0/257483872 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc44c009c40 con 0x7fc454110900 2023-07-18T03:21:16.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.076+0000 7fc452ffd700 1 --2- 172.21.15.101:0/257483872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc454107350 0x7fc45419c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:21:16.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.077+0000 7fc4537fe700 1 --2- 172.21.15.101:0/257483872 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc454110900 0x7fc4541a0c00 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fc44c00b6c0 tx=0x7fc44c004b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:16.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.077+0000 7fc43bfff700 1 -- 172.21.15.101:0/257483872 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44c00e040 con 0x7fc454110900 2023-07-18T03:21:16.075 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.078+0000 7fc4593ff700 1 -- 172.21.15.101:0/257483872 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4541a17f0 con 0x7fc454110900 2023-07-18T03:21:16.075 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.078+0000 7fc4593ff700 1 -- 172.21.15.101:0/257483872 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4541a9090 con 0x7fc454110900 2023-07-18T03:21:16.079 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.083+0000 7fc46cca5b80 1 Processor -- start 2023-07-18T03:21:16.084 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.086+0000 7fc43bfff700 1 -- 172.21.15.101:0/257483872 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc44c009ac0 con 0x7fc454110900 2023-07-18T03:21:16.084 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.086+0000 7fc43bfff700 1 -- 172.21.15.101:0/257483872 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44c005040 con 0x7fc454110900 2023-07-18T03:21:16.084 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.087+0000 7fc46cca5b80 1 -- start start 2023-07-18T03:21:16.084 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.088+0000 7fc46cca5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf11f05350 0x55bf11f05720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:16.084 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.088+0000 7fc46cca5b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bf11f05c60 0x55bf11f10610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:16.084 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.088+0000 7fc46cca5b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf11f10b50 0x55bf11f12f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:16.085 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.088+0000 7fc46cca5b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf11e0bd30 con 0x55bf11f05c60 2023-07-18T03:21:16.085 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.088+0000 7fc46cca5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf11e5eac0 con 0x55bf11f05350 2023-07-18T03:21:16.085 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.088+0000 7fc46cca5b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf11e57f50 con 0x55bf11f10b50 2023-07-18T03:21:16.086 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.088+0000 7fc4527fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf11f05350 0x55bf11f05720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:16.086 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.088+0000 7fc4527fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf11f05350 0x55bf11f05720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:42386/0 (socket says 172.21.15.101:42386) 2023-07-18T03:21:16.086 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.088+0000 7fc4527fc700 1 -- 172.21.15.101:0/298214709 learned_addr learned my addr 172.21.15.101:0/298214709 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:16.086 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.088+0000 7fc4527fc700 1 --2- 172.21.15.101:0/298214709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf11f10b50 0x55bf11f12f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:16.086 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.089+0000 7fc4527fc700 1 -- 172.21.15.101:0/298214709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf11f10b50 msgr2=0x55bf11f12f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:16.086 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.089+0000 7fc4527fc700 1 --2- 172.21.15.101:0/298214709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf11f10b50 0x55bf11f12f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.086 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.089+0000 7fc4527fc700 1 -- 172.21.15.101:0/298214709 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bf11f05c60 msgr2=0x55bf11f10610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-18T03:21:16.087 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.089+0000 7fc4527fc700 1 --2- 172.21.15.101:0/298214709 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bf11f05c60 0x55bf11f10610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.087 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.089+0000 7fc4527fc700 1 -- 172.21.15.101:0/298214709 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf11c85e60 con 0x55bf11f05350 2023-07-18T03:21:16.087 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.089+0000 7fc4527fc700 1 --2- 172.21.15.101:0/298214709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf11f05350 0x55bf11f05720 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc444014900 tx=0x7fc4440096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aefb96b3d2da6cb6 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:16.088 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.090+0000 7fc4397fa700 1 -- 172.21.15.101:0/298214709 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4440142d0 con 0x55bf11f05350 2023-07-18T03:21:16.088 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.090+0000 7fc4397fa700 1 -- 172.21.15.101:0/298214709 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc44401a070 con 0x55bf11f05350 2023-07-18T03:21:16.088 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.090+0000 7fc4397fa700 1 -- 172.21.15.101:0/298214709 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc444003dc0 con 0x55bf11f05350 2023-07-18T03:21:16.088 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.091+0000 7fc46cca5b80 1 -- 172.21.15.101:0/298214709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf11f05350 msgr2=0x55bf11f05720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:16.088 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.091+0000 7fc46cca5b80 1 --2- 172.21.15.101:0/298214709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf11f05350 0x55bf11f05720 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc444014900 tx=0x7fc4440096c0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.090 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.093+0000 7fc46cca5b80 1 -- 172.21.15.101:0/298214709 shutdown_connections 2023-07-18T03:21:16.090 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.093+0000 7fc46cca5b80 1 --2- 172.21.15.101:0/298214709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf11f05350 0x55bf11f05720 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.090 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.093+0000 7fc46cca5b80 1 --2- 172.21.15.101:0/298214709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf11f10b50 0x55bf11f12f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.090 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.093+0000 7fc46cca5b80 1 --2- 172.21.15.101:0/298214709 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bf11f05c60 0x55bf11f10610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.091 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.093+0000 7fc46cca5b80 1 -- 172.21.15.101:0/298214709 >> 172.21.15.101:0/298214709 conn(0x55bf11efbe20 msgr2=0x55bf11f01720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:16.091 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.093+0000 7fc43bfff700 1 -- 172.21.15.101:0/257483872 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fc44c01d470 con 0x7fc454110900 2023-07-18T03:21:16.091 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.094+0000 7fc43bfff700 1 --2- 172.21.15.101:0/257483872 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc43c0a13c0 0x7fc43c0a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:16.091 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.095+0000 7fc43bfff700 1 -- 172.21.15.101:0/257483872 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fc44c0ce560 con 0x7fc454110900 2023-07-18T03:21:16.093 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.096+0000 7fc452ffd700 1 --2- 172.21.15.101:0/257483872 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc43c0a13c0 0x7fc43c0a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:16.093 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.097+0000 7fc46cca5b80 1 -- 172.21.15.101:0/298214709 shutdown_connections 2023-07-18T03:21:16.096 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.098+0000 7fc452ffd700 1 --2- 172.21.15.101:0/257483872 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc43c0a13c0 0x7fc43c0a3870 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc448009ab0 tx=0x7fc448009510 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:16.096 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.099+0000 7fc46cca5b80 1 -- 172.21.15.101:0/298214709 wait complete. 2023-07-18T03:21:16.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.099+0000 7fc46cca5b80 1 -- start start 2023-07-18T03:21:16.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.100+0000 7fc46cca5b80 1 Processor -- start 2023-07-18T03:21:16.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.100+0000 7fc46cca5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf11f05350 0x55bf11f10470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:16.100 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.100+0000 7fc46cca5b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf11f05c60 0x55bf11f0d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:16.100 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.100+0000 7fc46cca5b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bf11f10b50 0x55bf11f0dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:16.100 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.100+0000 7fc46cca5b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf11df7b40 con 0x55bf11f10b50 2023-07-18T03:21:16.100 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.100+0000 7fc46cca5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf11df7840 con 0x55bf11f05350 2023-07-18T03:21:16.100 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.101+0000 7fc46cca5b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf11cdb9a0 con 0x55bf11f05c60 2023-07-18T03:21:16.100 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.101+0000 7fc46cca5b80 10 client.0 _ll_register_callbacks cb 0x55bf11df57e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:21:16.101 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.101+0000 7fc4527fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf11f05c60 0x55bf11f0d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:16.101 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.101+0000 7fc4527fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf11f05c60 0x55bf11f0d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:55878/0 (socket says 172.21.15.101:55878) 2023-07-18T03:21:16.101 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.101+0000 7fc4527fc700 1 -- 172.21.15.101:0/3792525773 learned_addr learned my addr 172.21.15.101:0/3792525773 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:16.101 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.101+0000 7fc4537fe700 1 --2- 172.21.15.101:0/3792525773 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bf11f10b50 0x55bf11f0dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:16.101 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.102+0000 7fc452ffd700 1 --2- 172.21.15.101:0/3792525773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf11f05350 0x55bf11f10470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:16.101 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.102+0000 7fc4537fe700 1 -- 172.21.15.101:0/3792525773 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf11f05c60 msgr2=0x55bf11f0d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:16.101 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.102+0000 7fc4537fe700 1 --2- 172.21.15.101:0/3792525773 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf11f05c60 0x55bf11f0d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.102 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.102+0000 7fc4537fe700 1 -- 172.21.15.101:0/3792525773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf11f05350 msgr2=0x55bf11f10470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:16.102 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.102+0000 7fc4537fe700 1 --2- 172.21.15.101:0/3792525773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf11f05350 0x55bf11f10470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.102 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.102+0000 7fc4537fe700 1 -- 172.21.15.101:0/3792525773 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4440145b0 con 0x55bf11f10b50 2023-07-18T03:21:16.102 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.103+0000 7fc4537fe700 1 --2- 172.21.15.101:0/3792525773 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bf11f10b50 0x55bf11f0dd00 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc44c0cb8a0 tx=0x7fc44c09b1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d21ee3510b43906f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:16.102 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.103+0000 7fc436ff5700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:21:16.102 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.103+0000 7fc436ff5700 1 -- 172.21.15.101:0/3792525773 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44c0cbd20 con 0x55bf11f10b50 2023-07-18T03:21:16.102 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.103+0000 7fc436ff5700 1 -- 172.21.15.101:0/3792525773 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fc44c09b2b0 con 0x55bf11f10b50 2023-07-18T03:21:16.102 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.103+0000 7fc46cca5b80 1 -- 172.21.15.101:0/3792525773 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bf11db1570 con 0x55bf11f10b50 2023-07-18T03:21:16.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.104+0000 7fc436ff5700 1 -- 172.21.15.101:0/3792525773 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc44c09f2e0 con 0x55bf11f10b50 2023-07-18T03:21:16.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.104+0000 7fc46cca5b80 10 client.18312 fetch_fsmap learned FSMap version 37 2023-07-18T03:21:16.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.104+0000 7fc46cca5b80 1 -- 172.21.15.101:0/3792525773 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55bf11c6ece0 con 0x55bf11f10b50 2023-07-18T03:21:16.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.106+0000 7fc436ff5700 1 -- 172.21.15.101:0/3792525773 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc44c0b2030 con 0x55bf11f10b50 2023-07-18T03:21:16.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.106+0000 7fc436ff5700 1 -- 172.21.15.101:0/3792525773 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 37) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fc44c09b620 con 0x55bf11f10b50 2023-07-18T03:21:16.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.106+0000 7fc46cca5b80 10 client.18312 fetch_fsmap finished waiting for FSMap version 37 2023-07-18T03:21:16.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.106+0000 7fc46cca5b80 10 client.18312 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:21:16.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.106+0000 7fc46cca5b80 20 client.18312 populate_metadata read hostname 'smithi101' 2023-07-18T03:21:16.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.106+0000 7fc46cca5b80 1 --2- 172.21.15.101:0/3792525773 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55bf11f311a0 0x55bf11f335b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:16.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.106+0000 7fc46cca5b80 4 client.18312 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:21:16.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.106+0000 7fc46cca5b80 1 -- 172.21.15.101:0/3792525773 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bf11e50f30 con 0x55bf11f311a0 2023-07-18T03:21:16.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.107+0000 7fc452ffd700 1 --2- 172.21.15.101:0/3792525773 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55bf11f311a0 0x55bf11f335b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:16.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.107+0000 7fc452ffd700 1 --2- 172.21.15.101:0/3792525773 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55bf11f311a0 0x55bf11f335b0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d151477f1931b3c server_cookie=64539525457cbad9 in_seq=0 out_seq=0 2023-07-18T03:21:16.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.108+0000 7fc436ff5700 10 client.18312 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:21:16.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.108+0000 7fc436ff5700 1 -- 172.21.15.101:0/3792525773 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bf11e50f30 con 0x55bf11f311a0 2023-07-18T03:21:16.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.108+0000 7fc436ff5700 10 client.18312 handle_command_reply: tid=0 2023-07-18T03:21:16.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.108+0000 7fc46cca5b80 1 client.18312 shutdown 2023-07-18T03:21:16.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.110+0000 7fc46cca5b80 1 -- 172.21.15.101:0/3792525773 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bf11f10b50 msgr2=0x55bf11f0dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:16.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.110+0000 7fc46cca5b80 1 --2- 172.21.15.101:0/3792525773 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bf11f10b50 0x55bf11f0dd00 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc44c0cb8a0 tx=0x7fc44c09b1b0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.110+0000 7fc46cca5b80 1 -- 172.21.15.101:0/3792525773 shutdown_connections 2023-07-18T03:21:16.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.110+0000 7fc46cca5b80 1 --2- 172.21.15.101:0/3792525773 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55bf11f311a0 0x55bf11f335b0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.110+0000 7fc46cca5b80 1 --2- 172.21.15.101:0/3792525773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf11f05350 0x55bf11f10470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.110+0000 7fc46cca5b80 1 --2- 172.21.15.101:0/3792525773 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bf11f10b50 0x55bf11f0dd00 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.110+0000 7fc46cca5b80 1 --2- 172.21.15.101:0/3792525773 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf11f05c60 0x55bf11f0d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.108 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.110+0000 7fc46cca5b80 1 -- 172.21.15.101:0/3792525773 >> 172.21.15.101:0/3792525773 conn(0x55bf11efbe20 msgr2=0x55bf11f012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:16.108 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.110+0000 7fc436ff5700 0 client.18312 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:21:16.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.114+0000 7fc46cca5b80 1 -- 172.21.15.101:0/3792525773 shutdown_connections 2023-07-18T03:21:16.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.114+0000 7fc46cca5b80 1 -- 172.21.15.101:0/3792525773 wait complete. 2023-07-18T03:21:16.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.115+0000 7fc46cca5b80 20 client.18312 trim_cache size 0 max 16384 2023-07-18T03:21:16.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.151+0000 7fc46cca5b80 1 Processor -- start 2023-07-18T03:21:16.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.151+0000 7fc46cca5b80 1 -- start start 2023-07-18T03:21:16.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.151+0000 7fc46cca5b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bf11f05350 0x55bf11f08300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:16.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.151+0000 7fc46cca5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf11f05c60 0x55bf11f018d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:16.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.151+0000 7fc46cca5b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf11f10b50 0x55bf11f17570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:16.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.151+0000 7fc46cca5b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf11f02d80 con 0x55bf11f05350 2023-07-18T03:21:16.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.151+0000 7fc46cca5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf11ef9020 con 0x55bf11f05c60 2023-07-18T03:21:16.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.151+0000 7fc46cca5b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf11efb9f0 con 0x55bf11f10b50 2023-07-18T03:21:16.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.151+0000 7fc4527fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf11f05c60 0x55bf11f018d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:16.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.151+0000 7fc4537fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf11f10b50 0x55bf11f17570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:16.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.151+0000 7fc4527fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf11f05c60 0x55bf11f018d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:42406/0 (socket says 172.21.15.101:42406) 2023-07-18T03:21:16.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.151+0000 7fc4527fc700 1 -- 172.21.15.101:0/1279449004 learned_addr learned my addr 172.21.15.101:0/1279449004 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:16.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.152+0000 7fc4537fe700 1 -- 172.21.15.101:0/1279449004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf11f05c60 msgr2=0x55bf11f018d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:16.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.152+0000 7fc4537fe700 1 --2- 172.21.15.101:0/1279449004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf11f05c60 0x55bf11f018d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.152+0000 7fc4537fe700 1 -- 172.21.15.101:0/1279449004 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bf11f05350 msgr2=0x55bf11f08300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:16.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.152+0000 7fc4537fe700 1 --2- 172.21.15.101:0/1279449004 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bf11f05350 0x55bf11f08300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.152+0000 7fc4537fe700 1 -- 172.21.15.101:0/1279449004 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf11cac2d0 con 0x55bf11f10b50 2023-07-18T03:21:16.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.152+0000 7fc4537fe700 1 --2- 172.21.15.101:0/1279449004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf11f10b50 0x55bf11f17570 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc44c0b9f30 tx=0x7fc44c09af20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fbed97fd678c7cf server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:16.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.153+0000 7fc439ffb700 1 -- 172.21.15.101:0/1279449004 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44c0cbd20 con 0x55bf11f10b50 2023-07-18T03:21:16.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.153+0000 7fc439ffb700 1 -- 172.21.15.101:0/1279449004 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc44c0b2070 con 0x55bf11f10b50 2023-07-18T03:21:16.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.153+0000 7fc439ffb700 1 -- 172.21.15.101:0/1279449004 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44c0ce340 con 0x55bf11f10b50 2023-07-18T03:21:16.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.153+0000 7fc46cca5b80 1 -- 172.21.15.101:0/1279449004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf11f10b50 msgr2=0x55bf11f17570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:16.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.153+0000 7fc46cca5b80 1 --2- 172.21.15.101:0/1279449004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf11f10b50 0x55bf11f17570 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc44c0b9f30 tx=0x7fc44c09af20 comp rx=0 tx=0).stop 2023-07-18T03:21:16.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.154+0000 7fc46cca5b80 1 -- 172.21.15.101:0/1279449004 shutdown_connections 2023-07-18T03:21:16.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.154+0000 7fc46cca5b80 1 --2- 172.21.15.101:0/1279449004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf11f10b50 0x55bf11f17570 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.154+0000 7fc46cca5b80 1 --2- 172.21.15.101:0/1279449004 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bf11f05350 0x55bf11f08300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.154+0000 7fc46cca5b80 1 --2- 172.21.15.101:0/1279449004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf11f05c60 0x55bf11f018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.154+0000 7fc46cca5b80 1 -- 172.21.15.101:0/1279449004 >> 172.21.15.101:0/1279449004 conn(0x55bf11efbe20 msgr2=0x55bf11f011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:16.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.154+0000 7fc46cca5b80 1 -- 172.21.15.101:0/1279449004 shutdown_connections 2023-07-18T03:21:16.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.154+0000 7fc46cca5b80 1 -- 172.21.15.101:0/1279449004 wait complete. 2023-07-18T03:21:16.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.154+0000 7fc46cca5b80 1 -- start start 2023-07-18T03:21:16.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.155+0000 7fc46cca5b80 1 Processor -- start 2023-07-18T03:21:16.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.155+0000 7fc46cca5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf11f05350 0x55bf11f7edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:16.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.155+0000 7fc46cca5b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf11f05c60 0x55bf11f7f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:16.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.155+0000 7fc46cca5b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bf11f10b50 0x55bf11f79170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:16.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.155+0000 7fc46cca5b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf11f79750 con 0x55bf11f10b50 2023-07-18T03:21:16.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.155+0000 7fc46cca5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf11f798c0 con 0x55bf11f05350 2023-07-18T03:21:16.156 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.155+0000 7fc46cca5b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf11f79a80 con 0x55bf11f05c60 2023-07-18T03:21:16.156 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.155+0000 7fc46cca5b80 10 client.0 _ll_register_callbacks cb 0x7fc44c0975f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:21:16.156 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.156+0000 7fc4527fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf11f05c60 0x55bf11f7f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:16.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.156+0000 7fc4527fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf11f05c60 0x55bf11f7f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:55894/0 (socket says 172.21.15.101:55894) 2023-07-18T03:21:16.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.156+0000 7fc4527fc700 1 -- 172.21.15.101:0/3506236046 learned_addr learned my addr 172.21.15.101:0/3506236046 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:16.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.156+0000 7fc4537fe700 1 --2- 172.21.15.101:0/3506236046 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bf11f10b50 0x55bf11f79170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:16.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.156+0000 7fc452ffd700 1 --2- 172.21.15.101:0/3506236046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf11f05350 0x55bf11f7edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:16.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.156+0000 7fc4527fc700 1 -- 172.21.15.101:0/3506236046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf11f05350 msgr2=0x55bf11f7edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:16.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.156+0000 7fc4527fc700 1 --2- 172.21.15.101:0/3506236046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf11f05350 0x55bf11f7edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.156+0000 7fc4527fc700 1 -- 172.21.15.101:0/3506236046 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bf11f10b50 msgr2=0x55bf11f79170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:16.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.156+0000 7fc4527fc700 1 --2- 172.21.15.101:0/3506236046 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bf11f10b50 0x55bf11f79170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.156+0000 7fc4527fc700 1 -- 172.21.15.101:0/3506236046 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bf11c6ece0 con 0x55bf11f05c60 2023-07-18T03:21:16.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.158+0000 7fc4527fc700 1 --2- 172.21.15.101:0/3506236046 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf11f05c60 0x55bf11f7f310 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc444014ef0 tx=0x7fc444005d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5409b5a5c2c3d0c9 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:16.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.158+0000 7fc436ff5700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:21:16.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.158+0000 7fc436ff5700 1 -- 172.21.15.101:0/3506236046 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc444025070 con 0x55bf11f05c60 2023-07-18T03:21:16.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.158+0000 7fc436ff5700 1 -- 172.21.15.101:0/3506236046 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fc44402a460 con 0x55bf11f05c60 2023-07-18T03:21:16.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.158+0000 7fc46cca5b80 1 -- 172.21.15.101:0/3506236046 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc44401a070 con 0x55bf11f05c60 2023-07-18T03:21:16.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.159+0000 7fc436ff5700 1 -- 172.21.15.101:0/3506236046 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4440142c0 con 0x55bf11f05c60 2023-07-18T03:21:16.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.160+0000 7fc46cca5b80 10 client.28390 fetch_fsmap learned FSMap version 37 2023-07-18T03:21:16.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.160+0000 7fc46cca5b80 1 -- 172.21.15.101:0/3506236046 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55bf11c84330 con 0x55bf11f05c60 2023-07-18T03:21:16.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.163+0000 7fc436ff5700 1 -- 172.21.15.101:0/3506236046 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc44401a070 con 0x55bf11f05c60 2023-07-18T03:21:16.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.163+0000 7fc436ff5700 1 -- 172.21.15.101:0/3506236046 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 37) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fc44401f700 con 0x55bf11f05c60 2023-07-18T03:21:16.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.164+0000 7fc46cca5b80 10 client.28390 fetch_fsmap finished waiting for FSMap version 37 2023-07-18T03:21:16.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.164+0000 7fc46cca5b80 10 client.28390 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:21:16.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.164+0000 7fc46cca5b80 20 client.28390 populate_metadata read hostname 'smithi101' 2023-07-18T03:21:16.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.164+0000 7fc46cca5b80 1 --2- 172.21.15.101:0/3506236046 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55bf11f86ba0 0x55bf11f88fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:16.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.164+0000 7fc46cca5b80 4 client.28390 mds_command: new command op to 24562 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-18T03:21:16.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.164+0000 7fc46cca5b80 1 -- 172.21.15.101:0/3506236046 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55bf11e130d0 con 0x55bf11f86ba0 2023-07-18T03:21:16.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.165+0000 7fc452ffd700 1 --2- 172.21.15.101:0/3506236046 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55bf11f86ba0 0x55bf11f88fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:16.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.166+0000 7fc452ffd700 1 --2- 172.21.15.101:0/3506236046 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55bf11f86ba0 0x55bf11f88fb0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2b57639e3f64d35 server_cookie=6e1b1f5034deb64f in_seq=0 out_seq=0 2023-07-18T03:21:16.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.166+0000 7fc436ff5700 10 client.28390 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:21:16.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.168+0000 7fc436ff5700 1 -- 172.21.15.101:0/3506236046 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55bf11e130d0 con 0x55bf11f86ba0 2023-07-18T03:21:16.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.168+0000 7fc436ff5700 10 client.28390 handle_command_reply: tid=0 2023-07-18T03:21:16.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.168+0000 7fc46cca5b80 1 client.28390 shutdown 2023-07-18T03:21:16.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.168+0000 7fc46cca5b80 1 -- 172.21.15.101:0/3506236046 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf11f05c60 msgr2=0x55bf11f7f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:16.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.168+0000 7fc46cca5b80 1 --2- 172.21.15.101:0/3506236046 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf11f05c60 0x55bf11f7f310 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc444014ef0 tx=0x7fc444005d10 comp rx=0 tx=0).stop 2023-07-18T03:21:16.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.168+0000 7fc46cca5b80 1 -- 172.21.15.101:0/3506236046 shutdown_connections 2023-07-18T03:21:16.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.168+0000 7fc46cca5b80 1 --2- 172.21.15.101:0/3506236046 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55bf11f86ba0 0x55bf11f88fb0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.168+0000 7fc46cca5b80 1 --2- 172.21.15.101:0/3506236046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bf11f05350 0x55bf11f7edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.168+0000 7fc46cca5b80 1 --2- 172.21.15.101:0/3506236046 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bf11f10b50 0x55bf11f79170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.168+0000 7fc436ff5700 0 client.28390 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:21:16.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.168+0000 7fc46cca5b80 1 --2- 172.21.15.101:0/3506236046 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bf11f05c60 0x55bf11f7f310 secure :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc444014ef0 tx=0x7fc444005d10 comp rx=0 tx=0).stop 2023-07-18T03:21:16.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.168+0000 7fc46cca5b80 1 -- 172.21.15.101:0/3506236046 >> 172.21.15.101:0/3506236046 conn(0x55bf11efbe20 msgr2=0x55bf11f072f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:16.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.168+0000 7fc46cca5b80 1 -- 172.21.15.101:0/3506236046 shutdown_connections 2023-07-18T03:21:16.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.169+0000 7fc46cca5b80 1 -- 172.21.15.101:0/3506236046 wait complete. 2023-07-18T03:21:16.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.175+0000 7fc46cca5b80 20 client.28390 trim_cache size 0 max 16384 2023-07-18T03:21:16.172 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:21:16.172 INFO:teuthology.orchestra.run.smithi101.stdout: "return_code": 0, 2023-07-18T03:21:16.172 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_tag": "33787411-14d9-4062-bc85-e96f22f4cbfa", 2023-07-18T03:21:16.172 INFO:teuthology.orchestra.run.smithi101.stdout: "mode": "asynchronous" 2023-07-18T03:21:16.172 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:21:16.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.176+0000 7fc4593ff700 1 -- 172.21.15.101:0/257483872 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc43c0a13c0 msgr2=0x7fc43c0a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:16.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.176+0000 7fc4593ff700 1 --2- 172.21.15.101:0/257483872 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc43c0a13c0 0x7fc43c0a3870 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc448009ab0 tx=0x7fc448009510 comp rx=0 tx=0).stop 2023-07-18T03:21:16.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.176+0000 7fc4593ff700 1 -- 172.21.15.101:0/257483872 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc454110900 msgr2=0x7fc4541a0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:16.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.176+0000 7fc4593ff700 1 --2- 172.21.15.101:0/257483872 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc454110900 0x7fc4541a0c00 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fc44c00b6c0 tx=0x7fc44c004b80 comp rx=0 tx=0).stop 2023-07-18T03:21:16.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.176+0000 7fc4593ff700 1 -- 172.21.15.101:0/257483872 shutdown_connections 2023-07-18T03:21:16.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.176+0000 7fc4593ff700 1 --2- 172.21.15.101:0/257483872 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc43c0a13c0 0x7fc43c0a3870 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.176+0000 7fc4593ff700 1 --2- 172.21.15.101:0/257483872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc454107350 0x7fc45419c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.176+0000 7fc4593ff700 1 --2- 172.21.15.101:0/257483872 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc454110900 0x7fc4541a0c00 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.176+0000 7fc4593ff700 1 --2- 172.21.15.101:0/257483872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc454107c60 0x7fc45419c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.176+0000 7fc4593ff700 1 -- 172.21.15.101:0/257483872 >> 172.21.15.101:0/257483872 conn(0x7fc454075a00 msgr2=0x7fc454075f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:16.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.177+0000 7fc4593ff700 1 -- 172.21.15.101:0/257483872 shutdown_connections 2023-07-18T03:21:16.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.178+0000 7fc4593ff700 1 -- 172.21.15.101:0/257483872 wait complete. 2023-07-18T03:21:16.199 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:21:16.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.468+0000 7fdc8d7ff700 1 -- 172.21.15.101:0/3053359177 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc880ffad0 msgr2=0x7fdc880fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:16.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.468+0000 7fdc8d7ff700 1 --2- 172.21.15.101:0/3053359177 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc880ffad0 0x7fdc880fff40 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fdc70009990 tx=0x7fdc70009ca0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.468+0000 7fdc8d7ff700 1 -- 172.21.15.101:0/3053359177 shutdown_connections 2023-07-18T03:21:16.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.468+0000 7fdc8d7ff700 1 --2- 172.21.15.101:0/3053359177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc88105ae0 0x7fdc88105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.468+0000 7fdc8d7ff700 1 --2- 172.21.15.101:0/3053359177 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdc88100480 0x7fdc8810c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.468+0000 7fdc8d7ff700 1 --2- 172.21.15.101:0/3053359177 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc880ffad0 0x7fdc880fff40 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.468+0000 7fdc8d7ff700 1 -- 172.21.15.101:0/3053359177 >> 172.21.15.101:0/3053359177 conn(0x7fdc880fb5f0 msgr2=0x7fdc880fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:16.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.468+0000 7fdc8d7ff700 1 -- 172.21.15.101:0/3053359177 shutdown_connections 2023-07-18T03:21:16.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.468+0000 7fdc8d7ff700 1 -- 172.21.15.101:0/3053359177 wait complete. 2023-07-18T03:21:16.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.470+0000 7fdc8d7ff700 1 Processor -- start 2023-07-18T03:21:16.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.470+0000 7fdc8d7ff700 1 -- start start 2023-07-18T03:21:16.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.471+0000 7fdc8d7ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc880ffad0 0x7fdc88106990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:16.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.471+0000 7fdc8d7ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdc88100480 0x7fdc88106ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:16.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.471+0000 7fdc8d7ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc88105ae0 0x7fdc88107410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:16.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.471+0000 7fdc8d7ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc881a8560 con 0x7fdc88100480 2023-07-18T03:21:16.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.471+0000 7fdc8d7ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc88197c30 con 0x7fdc880ffad0 2023-07-18T03:21:16.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.471+0000 7fdc8d7ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc88197df0 con 0x7fdc88105ae0 2023-07-18T03:21:16.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.471+0000 7fdc86ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc880ffad0 0x7fdc88106990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:16.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.471+0000 7fdc86ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc880ffad0 0x7fdc88106990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:42438/0 (socket says 172.21.15.101:42438) 2023-07-18T03:21:16.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.471+0000 7fdc86ffd700 1 -- 172.21.15.101:0/1019818639 learned_addr learned my addr 172.21.15.101:0/1019818639 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:16.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.471+0000 7fdc877fe700 1 --2- 172.21.15.101:0/1019818639 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc88105ae0 0x7fdc88107410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:16.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.472+0000 7fdc7ffff700 1 --2- 172.21.15.101:0/1019818639 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdc88100480 0x7fdc88106ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:16.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.472+0000 7fdc86ffd700 1 -- 172.21.15.101:0/1019818639 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc88105ae0 msgr2=0x7fdc88107410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:16.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.472+0000 7fdc86ffd700 1 --2- 172.21.15.101:0/1019818639 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc88105ae0 0x7fdc88107410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.472+0000 7fdc86ffd700 1 -- 172.21.15.101:0/1019818639 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdc88100480 msgr2=0x7fdc88106ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:16.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.472+0000 7fdc86ffd700 1 --2- 172.21.15.101:0/1019818639 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdc88100480 0x7fdc88106ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.472+0000 7fdc86ffd700 1 -- 172.21.15.101:0/1019818639 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc700095f0 con 0x7fdc880ffad0 2023-07-18T03:21:16.472 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.472+0000 7fdc86ffd700 1 --2- 172.21.15.101:0/1019818639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc880ffad0 0x7fdc88106990 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fdc7800e9a0 tx=0x7fdc7800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:16.472 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.472+0000 7fdc84ff9700 1 -- 172.21.15.101:0/1019818639 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7800cbd0 con 0x7fdc880ffad0 2023-07-18T03:21:16.472 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.472+0000 7fdc84ff9700 1 -- 172.21.15.101:0/1019818639 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc7800cd30 con 0x7fdc880ffad0 2023-07-18T03:21:16.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.472+0000 7fdc8d7ff700 1 -- 172.21.15.101:0/1019818639 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc881980d0 con 0x7fdc880ffad0 2023-07-18T03:21:16.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.472+0000 7fdc84ff9700 1 -- 172.21.15.101:0/1019818639 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc78018680 con 0x7fdc880ffad0 2023-07-18T03:21:16.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.473+0000 7fdc8d7ff700 1 -- 172.21.15.101:0/1019818639 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc88198630 con 0x7fdc880ffad0 2023-07-18T03:21:16.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.476+0000 7fdc84ff9700 1 -- 172.21.15.101:0/1019818639 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fdc78010430 con 0x7fdc880ffad0 2023-07-18T03:21:16.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.478+0000 7fdc84ff9700 1 --2- 172.21.15.101:0/1019818639 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fdc740a1340 0x7fdc740a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:16.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.478+0000 7fdc84ff9700 1 -- 172.21.15.101:0/1019818639 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fdc78014070 con 0x7fdc880ffad0 2023-07-18T03:21:16.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.478+0000 7fdc7ffff700 1 --2- 172.21.15.101:0/1019818639 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fdc740a1340 0x7fdc740a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:16.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.479+0000 7fdca10fab80 1 Processor -- start 2023-07-18T03:21:16.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.479+0000 7fdc7ffff700 1 --2- 172.21.15.101:0/1019818639 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fdc740a1340 0x7fdc740a37f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdc7000b2e0 tx=0x7fdc70005fb0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:16.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.479+0000 7fdca10fab80 1 -- start start 2023-07-18T03:21:16.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.479+0000 7fdca10fab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560b17522d0 0x5560b17526a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:16.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.479+0000 7fdca10fab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5560b1752be0 0x5560b175d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:16.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.480+0000 7fdca10fab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5560b175dad0 0x5560b175feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:16.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.480+0000 7fdca10fab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560b1658cb0 con 0x5560b175dad0 2023-07-18T03:21:16.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.480+0000 7fdca10fab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560b16aba40 con 0x5560b17522d0 2023-07-18T03:21:16.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.480+0000 7fdca10fab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560b16a4ed0 con 0x5560b1752be0 2023-07-18T03:21:16.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.480+0000 7fdc86ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560b17522d0 0x5560b17526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:16.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.480+0000 7fdc86ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560b17522d0 0x5560b17526a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:42444/0 (socket says 172.21.15.101:42444) 2023-07-18T03:21:16.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.480+0000 7fdc86ffd700 1 -- 172.21.15.101:0/3562681562 learned_addr learned my addr 172.21.15.101:0/3562681562 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:16.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.480+0000 7fdc7ffff700 1 --2- 172.21.15.101:0/3562681562 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5560b1752be0 0x5560b175d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:16.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.480+0000 7fdc86ffd700 1 -- 172.21.15.101:0/3562681562 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5560b1752be0 msgr2=0x5560b175d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:16.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.480+0000 7fdc86ffd700 1 --2- 172.21.15.101:0/3562681562 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5560b1752be0 0x5560b175d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.480+0000 7fdc877fe700 1 --2- 172.21.15.101:0/3562681562 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5560b175dad0 0x5560b175feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:16.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.480+0000 7fdc86ffd700 1 -- 172.21.15.101:0/3562681562 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5560b175dad0 msgr2=0x5560b175feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:16.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.480+0000 7fdc86ffd700 1 --2- 172.21.15.101:0/3562681562 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5560b175dad0 0x5560b175feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.480+0000 7fdc86ffd700 1 -- 172.21.15.101:0/3562681562 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560b14d2de0 con 0x5560b17522d0 2023-07-18T03:21:16.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.480+0000 7fdc877fe700 1 --2- 172.21.15.101:0/3562681562 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5560b175dad0 0x5560b175feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:21:16.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.481+0000 7fdc86ffd700 1 --2- 172.21.15.101:0/3562681562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560b17522d0 0x5560b17526a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fdc78096550 tx=0x7fdc78099c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9febc691e104591 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:16.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.481+0000 7fdc7d7fa700 1 -- 172.21.15.101:0/3562681562 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc780cd1b0 con 0x5560b17522d0 2023-07-18T03:21:16.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.481+0000 7fdc7d7fa700 1 -- 172.21.15.101:0/3562681562 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc7809e080 con 0x5560b17522d0 2023-07-18T03:21:16.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.481+0000 7fdc7d7fa700 1 -- 172.21.15.101:0/3562681562 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc780a2010 con 0x5560b17522d0 2023-07-18T03:21:16.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.482+0000 7fdca10fab80 1 -- 172.21.15.101:0/3562681562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560b17522d0 msgr2=0x5560b17526a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:16.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.482+0000 7fdca10fab80 1 --2- 172.21.15.101:0/3562681562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560b17522d0 0x5560b17526a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fdc78096550 tx=0x7fdc78099c60 comp rx=0 tx=0).stop 2023-07-18T03:21:16.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.482+0000 7fdca10fab80 1 -- 172.21.15.101:0/3562681562 shutdown_connections 2023-07-18T03:21:16.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.482+0000 7fdca10fab80 1 --2- 172.21.15.101:0/3562681562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560b17522d0 0x5560b17526a0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.482+0000 7fdca10fab80 1 --2- 172.21.15.101:0/3562681562 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5560b175dad0 0x5560b175feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.482+0000 7fdca10fab80 1 --2- 172.21.15.101:0/3562681562 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5560b1752be0 0x5560b175d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.482+0000 7fdca10fab80 1 -- 172.21.15.101:0/3562681562 >> 172.21.15.101:0/3562681562 conn(0x5560b1748da0 msgr2=0x5560b174e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:16.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.482+0000 7fdca10fab80 1 -- 172.21.15.101:0/3562681562 shutdown_connections 2023-07-18T03:21:16.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.482+0000 7fdca10fab80 1 -- 172.21.15.101:0/3562681562 wait complete. 2023-07-18T03:21:16.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.483+0000 7fdca10fab80 1 -- start start 2023-07-18T03:21:16.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.483+0000 7fdca10fab80 1 Processor -- start 2023-07-18T03:21:16.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.484+0000 7fdca10fab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5560b17522d0 0x5560b175a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:16.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.484+0000 7fdca10fab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5560b1752be0 0x5560b175ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:16.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.484+0000 7fdca10fab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560b175dad0 0x5560b175b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:16.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.484+0000 7fdca10fab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560b1644ac0 con 0x5560b1752be0 2023-07-18T03:21:16.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.484+0000 7fdca10fab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560b16447c0 con 0x5560b175dad0 2023-07-18T03:21:16.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.484+0000 7fdca10fab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560b1528920 con 0x5560b17522d0 2023-07-18T03:21:16.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.484+0000 7fdca10fab80 10 client.0 _ll_register_callbacks cb 0x5560b1642760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:21:16.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.484+0000 7fdc86ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5560b17522d0 0x5560b175a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:16.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.484+0000 7fdc7ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5560b1752be0 0x5560b175ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:16.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.484+0000 7fdc86ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5560b17522d0 0x5560b175a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:55926/0 (socket says 172.21.15.101:55926) 2023-07-18T03:21:16.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.484+0000 7fdc7ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5560b1752be0 0x5560b175ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:50778/0 (socket says 172.21.15.101:50778) 2023-07-18T03:21:16.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.484+0000 7fdc86ffd700 1 -- 172.21.15.101:0/476002005 learned_addr learned my addr 172.21.15.101:0/476002005 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:16.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.484+0000 7fdc877fe700 1 --2- 172.21.15.101:0/476002005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560b175dad0 0x5560b175b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:16.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.485+0000 7fdc86ffd700 1 -- 172.21.15.101:0/476002005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560b175dad0 msgr2=0x5560b175b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:16.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.485+0000 7fdc86ffd700 1 --2- 172.21.15.101:0/476002005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560b175dad0 0x5560b175b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.485+0000 7fdc86ffd700 1 -- 172.21.15.101:0/476002005 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5560b1752be0 msgr2=0x5560b175ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:16.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.485+0000 7fdc86ffd700 1 --2- 172.21.15.101:0/476002005 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5560b1752be0 0x5560b175ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.485+0000 7fdc86ffd700 1 -- 172.21.15.101:0/476002005 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc88198630 con 0x5560b17522d0 2023-07-18T03:21:16.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.485+0000 7fdc877fe700 1 --2- 172.21.15.101:0/476002005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560b175dad0 0x5560b175b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:21:16.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.485+0000 7fdc86ffd700 1 --2- 172.21.15.101:0/476002005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5560b17522d0 0x5560b175a8f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fdc7800e970 tx=0x7fdc780964c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80d0d9299b81522c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:16.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.485+0000 7fdc6e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:21:16.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.485+0000 7fdc6e7fc700 1 -- 172.21.15.101:0/476002005 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc780cd1b0 con 0x5560b17522d0 2023-07-18T03:21:16.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.485+0000 7fdca10fab80 1 -- 172.21.15.101:0/476002005 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5560b15fe4f0 con 0x5560b17522d0 2023-07-18T03:21:16.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.485+0000 7fdc6e7fc700 1 -- 172.21.15.101:0/476002005 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fdc780a24a0 con 0x5560b17522d0 2023-07-18T03:21:16.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.485+0000 7fdc6e7fc700 1 -- 172.21.15.101:0/476002005 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc780a2220 con 0x5560b17522d0 2023-07-18T03:21:16.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.485+0000 7fdca10fab80 10 client.28402 fetch_fsmap learned FSMap version 37 2023-07-18T03:21:16.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.485+0000 7fdca10fab80 1 -- 172.21.15.101:0/476002005 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5560b14bbc60 con 0x5560b17522d0 2023-07-18T03:21:16.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.486+0000 7fdc6e7fc700 1 -- 172.21.15.101:0/476002005 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5560b15fe4f0 con 0x5560b17522d0 2023-07-18T03:21:16.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.486+0000 7fdc6e7fc700 1 -- 172.21.15.101:0/476002005 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 37) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fdc780106e0 con 0x5560b17522d0 2023-07-18T03:21:16.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.486+0000 7fdca10fab80 10 client.28402 fetch_fsmap finished waiting for FSMap version 37 2023-07-18T03:21:16.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.486+0000 7fdca10fab80 10 client.28402 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:21:16.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.486+0000 7fdca10fab80 20 client.28402 populate_metadata read hostname 'smithi101' 2023-07-18T03:21:16.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.486+0000 7fdca10fab80 1 --2- 172.21.15.101:0/476002005 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5560b176ce90 0x5560b176f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:16.492 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.486+0000 7fdca10fab80 4 client.28402 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:21:16.492 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.486+0000 7fdca10fab80 1 -- 172.21.15.101:0/476002005 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5560b169deb0 con 0x5560b176ce90 2023-07-18T03:21:16.492 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.487+0000 7fdc7ffff700 1 --2- 172.21.15.101:0/476002005 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5560b176ce90 0x5560b176f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:16.492 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.488+0000 7fdc7ffff700 1 --2- 172.21.15.101:0/476002005 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5560b176ce90 0x5560b176f270 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69456beb744c68cd server_cookie=f534d1db68abc99b in_seq=0 out_seq=0 2023-07-18T03:21:16.492 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.488+0000 7fdc6e7fc700 10 client.28402 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:21:16.492 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.492+0000 7fdc6e7fc700 1 -- 172.21.15.101:0/476002005 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5560b169deb0 con 0x5560b176ce90 2023-07-18T03:21:16.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.492+0000 7fdc6e7fc700 10 client.28402 handle_command_reply: tid=0 2023-07-18T03:21:16.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.492+0000 7fdca10fab80 1 client.28402 shutdown 2023-07-18T03:21:16.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.492+0000 7fdca10fab80 1 -- 172.21.15.101:0/476002005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5560b17522d0 msgr2=0x5560b175a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:16.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.492+0000 7fdca10fab80 1 --2- 172.21.15.101:0/476002005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5560b17522d0 0x5560b175a8f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fdc7800e970 tx=0x7fdc780964c0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.492+0000 7fdca10fab80 1 -- 172.21.15.101:0/476002005 shutdown_connections 2023-07-18T03:21:16.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.492+0000 7fdca10fab80 1 --2- 172.21.15.101:0/476002005 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5560b176ce90 0x5560b176f270 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.492+0000 7fdca10fab80 1 --2- 172.21.15.101:0/476002005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560b175dad0 0x5560b175b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.492+0000 7fdca10fab80 1 --2- 172.21.15.101:0/476002005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5560b17522d0 0x5560b175a8f0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.492+0000 7fdca10fab80 1 --2- 172.21.15.101:0/476002005 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5560b1752be0 0x5560b175ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.492+0000 7fdc6e7fc700 0 client.28402 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:21:16.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.492+0000 7fdca10fab80 1 -- 172.21.15.101:0/476002005 >> 172.21.15.101:0/476002005 conn(0x5560b1748da0 msgr2=0x5560b174e290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:16.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.493+0000 7fdca10fab80 1 -- 172.21.15.101:0/476002005 shutdown_connections 2023-07-18T03:21:16.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.493+0000 7fdca10fab80 1 -- 172.21.15.101:0/476002005 wait complete. 2023-07-18T03:21:16.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.493+0000 7fdca10fab80 20 client.28402 trim_cache size 0 max 16384 2023-07-18T03:21:16.495 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:16 smithi101 ceph-mon[126379]: pgmap v430: 129 pgs: 129 active+clean; 1006 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 9.6 MiB/s wr, 542 op/s 2023-07-18T03:21:16.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.513+0000 7fdca10fab80 1 Processor -- start 2023-07-18T03:21:16.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.514+0000 7fdca10fab80 1 -- start start 2023-07-18T03:21:16.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.514+0000 7fdca10fab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560b17522d0 0x5560b1761ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:16.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.514+0000 7fdca10fab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5560b1752be0 0x5560b1768680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:16.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.514+0000 7fdca10fab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5560b175dad0 0x5560b1768bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:16.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.514+0000 7fdca10fab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560b175be60 con 0x5560b175dad0 2023-07-18T03:21:16.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.514+0000 7fdca10fab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560b1748970 con 0x5560b17522d0 2023-07-18T03:21:16.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.514+0000 7fdca10fab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560b1755250 con 0x5560b1752be0 2023-07-18T03:21:16.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.514+0000 7fdc86ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560b17522d0 0x5560b1761ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:16.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.514+0000 7fdc86ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560b17522d0 0x5560b1761ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:42454/0 (socket says 172.21.15.101:42454) 2023-07-18T03:21:16.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.514+0000 7fdc86ffd700 1 -- 172.21.15.101:0/1187684987 learned_addr learned my addr 172.21.15.101:0/1187684987 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:16.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.514+0000 7fdc7ffff700 1 --2- 172.21.15.101:0/1187684987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5560b1752be0 0x5560b1768680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:16.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.514+0000 7fdc877fe700 1 --2- 172.21.15.101:0/1187684987 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5560b175dad0 0x5560b1768bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:16.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.515+0000 7fdc86ffd700 1 -- 172.21.15.101:0/1187684987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5560b1752be0 msgr2=0x5560b1768680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:16.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.515+0000 7fdc86ffd700 1 --2- 172.21.15.101:0/1187684987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5560b1752be0 0x5560b1768680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.515+0000 7fdc86ffd700 1 -- 172.21.15.101:0/1187684987 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5560b175dad0 msgr2=0x5560b1768bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:16.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.515+0000 7fdc86ffd700 1 --2- 172.21.15.101:0/1187684987 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5560b175dad0 0x5560b1768bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.515+0000 7fdc86ffd700 1 -- 172.21.15.101:0/1187684987 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560b174a5e0 con 0x5560b17522d0 2023-07-18T03:21:16.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.515+0000 7fdc877fe700 1 --2- 172.21.15.101:0/1187684987 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5560b175dad0 0x5560b1768bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:21:16.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.515+0000 7fdc86ffd700 1 --2- 172.21.15.101:0/1187684987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560b17522d0 0x5560b1761ed0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fdc780983a0 tx=0x7fdc780cac30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a18de13ecd48ca78 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:16.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.515+0000 7fdc7dffb700 1 -- 172.21.15.101:0/1187684987 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc780cd1b0 con 0x5560b17522d0 2023-07-18T03:21:16.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.515+0000 7fdc7dffb700 1 -- 172.21.15.101:0/1187684987 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc780ca5a0 con 0x5560b17522d0 2023-07-18T03:21:16.515 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.515+0000 7fdc7dffb700 1 -- 172.21.15.101:0/1187684987 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc780990d0 con 0x5560b17522d0 2023-07-18T03:21:16.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.516+0000 7fdca10fab80 1 -- 172.21.15.101:0/1187684987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560b17522d0 msgr2=0x5560b1761ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:16.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.516+0000 7fdca10fab80 1 --2- 172.21.15.101:0/1187684987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560b17522d0 0x5560b1761ed0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fdc780983a0 tx=0x7fdc780cac30 comp rx=0 tx=0).stop 2023-07-18T03:21:16.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.516+0000 7fdca10fab80 1 -- 172.21.15.101:0/1187684987 shutdown_connections 2023-07-18T03:21:16.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.516+0000 7fdca10fab80 1 --2- 172.21.15.101:0/1187684987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560b17522d0 0x5560b1761ed0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.516+0000 7fdca10fab80 1 --2- 172.21.15.101:0/1187684987 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5560b175dad0 0x5560b1768bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.516+0000 7fdca10fab80 1 --2- 172.21.15.101:0/1187684987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5560b1752be0 0x5560b1768680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.516+0000 7fdca10fab80 1 -- 172.21.15.101:0/1187684987 >> 172.21.15.101:0/1187684987 conn(0x5560b1748da0 msgr2=0x5560b17697a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:16.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.517+0000 7fdca10fab80 1 -- 172.21.15.101:0/1187684987 shutdown_connections 2023-07-18T03:21:16.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.517+0000 7fdca10fab80 1 -- 172.21.15.101:0/1187684987 wait complete. 2023-07-18T03:21:16.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.517+0000 7fdca10fab80 1 -- start start 2023-07-18T03:21:16.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.517+0000 7fdca10fab80 1 Processor -- start 2023-07-18T03:21:16.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.518+0000 7fdca10fab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5560b17522d0 0x5560b17b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:16.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.518+0000 7fdca10fab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5560b1752be0 0x5560b17b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:16.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.518+0000 7fdca10fab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560b175dad0 0x5560b17bfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:16.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.518+0000 7fdca10fab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560b17b59f0 con 0x5560b17522d0 2023-07-18T03:21:16.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.518+0000 7fdca10fab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560b17c0160 con 0x5560b175dad0 2023-07-18T03:21:16.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.518+0000 7fdca10fab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560b17c02a0 con 0x5560b1752be0 2023-07-18T03:21:16.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.518+0000 7fdca10fab80 10 client.0 _ll_register_callbacks cb 0x7fdc70005c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:21:16.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.518+0000 7fdc86ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5560b17522d0 0x5560b17b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:16.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.518+0000 7fdc7ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5560b1752be0 0x5560b17b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:16.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.519+0000 7fdc86ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5560b17522d0 0x5560b17b4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:50788/0 (socket says 172.21.15.101:50788) 2023-07-18T03:21:16.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.519+0000 7fdc86ffd700 1 -- 172.21.15.101:0/3725254747 learned_addr learned my addr 172.21.15.101:0/3725254747 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:16.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.519+0000 7fdc7ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5560b1752be0 0x5560b17b50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:55944/0 (socket says 172.21.15.101:55944) 2023-07-18T03:21:16.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.519+0000 7fdc877fe700 1 --2- 172.21.15.101:0/3725254747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560b175dad0 0x5560b17bfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:16.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.519+0000 7fdc7ffff700 1 -- 172.21.15.101:0/3725254747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560b175dad0 msgr2=0x5560b17bfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:16.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.519+0000 7fdc7ffff700 1 --2- 172.21.15.101:0/3725254747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560b175dad0 0x5560b17bfc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.519+0000 7fdc7ffff700 1 -- 172.21.15.101:0/3725254747 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5560b17522d0 msgr2=0x5560b17b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:16.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.519+0000 7fdc7ffff700 1 --2- 172.21.15.101:0/3725254747 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5560b17522d0 0x5560b17b4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.519+0000 7fdc7ffff700 1 -- 172.21.15.101:0/3725254747 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5560b14bbc60 con 0x5560b1752be0 2023-07-18T03:21:16.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.519+0000 7fdc7ffff700 1 --2- 172.21.15.101:0/3725254747 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5560b1752be0 0x5560b17b50e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fdc881082c0 tx=0x7fdc70009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6208e07ce1af4ea server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:16.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.519+0000 7fdc6e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:21:16.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.520+0000 7fdc6e7fc700 1 -- 172.21.15.101:0/3725254747 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7002b040 con 0x5560b1752be0 2023-07-18T03:21:16.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.520+0000 7fdc6e7fc700 1 -- 172.21.15.101:0/3725254747 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fdc700350d0 con 0x5560b1752be0 2023-07-18T03:21:16.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.520+0000 7fdca10fab80 1 -- 172.21.15.101:0/3725254747 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc7809e080 con 0x5560b1752be0 2023-07-18T03:21:16.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.520+0000 7fdc6e7fc700 1 -- 172.21.15.101:0/3725254747 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc70035290 con 0x5560b1752be0 2023-07-18T03:21:16.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.520+0000 7fdca10fab80 10 client.28408 fetch_fsmap learned FSMap version 37 2023-07-18T03:21:16.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.520+0000 7fdca10fab80 1 -- 172.21.15.101:0/3725254747 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5560b14f9250 con 0x5560b1752be0 2023-07-18T03:21:16.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.520+0000 7fdc6e7fc700 1 -- 172.21.15.101:0/3725254747 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc7809e080 con 0x5560b1752be0 2023-07-18T03:21:16.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.520+0000 7fdc6e7fc700 1 -- 172.21.15.101:0/3725254747 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 37) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fdc70004cc0 con 0x5560b1752be0 2023-07-18T03:21:16.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.521+0000 7fdca10fab80 10 client.28408 fetch_fsmap finished waiting for FSMap version 37 2023-07-18T03:21:16.524 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.521+0000 7fdca10fab80 10 client.28408 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:21:16.524 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.521+0000 7fdca10fab80 20 client.28408 populate_metadata read hostname 'smithi101' 2023-07-18T03:21:16.524 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.521+0000 7fdca10fab80 1 --2- 172.21.15.101:0/3725254747 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5560b17c19d0 0x5560b17c3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:16.524 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.521+0000 7fdca10fab80 4 client.28408 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:21:16.524 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.521+0000 7fdca10fab80 1 -- 172.21.15.101:0/3725254747 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5560b1660050 con 0x5560b17c19d0 2023-07-18T03:21:16.524 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.521+0000 7fdc86ffd700 1 --2- 172.21.15.101:0/3725254747 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5560b17c19d0 0x5560b17c3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:16.524 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.521+0000 7fdc86ffd700 1 --2- 172.21.15.101:0/3725254747 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5560b17c19d0 0x5560b17c3de0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79994d0018b906f9 server_cookie=f7e78dbe267b44fe in_seq=0 out_seq=0 2023-07-18T03:21:16.525 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.521+0000 7fdc6e7fc700 10 client.28408 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:21:16.525 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.522+0000 7fdc6e7fc700 1 -- 172.21.15.101:0/3725254747 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5560b1660050 con 0x5560b17c19d0 2023-07-18T03:21:16.526 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.522+0000 7fdc6e7fc700 10 client.28408 handle_command_reply: tid=0 2023-07-18T03:21:16.526 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.522+0000 7fdca10fab80 1 client.28408 shutdown 2023-07-18T03:21:16.526 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.522+0000 7fdca10fab80 1 -- 172.21.15.101:0/3725254747 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5560b1752be0 msgr2=0x5560b17b50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:16.526 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.522+0000 7fdca10fab80 1 --2- 172.21.15.101:0/3725254747 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5560b1752be0 0x5560b17b50e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fdc881082c0 tx=0x7fdc70009ca0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.526 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.522+0000 7fdca10fab80 1 -- 172.21.15.101:0/3725254747 shutdown_connections 2023-07-18T03:21:16.526 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.522+0000 7fdca10fab80 1 --2- 172.21.15.101:0/3725254747 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5560b17c19d0 0x5560b17c3de0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.522+0000 7fdca10fab80 1 --2- 172.21.15.101:0/3725254747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560b175dad0 0x5560b17bfc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.522+0000 7fdca10fab80 1 --2- 172.21.15.101:0/3725254747 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5560b17522d0 0x5560b17b4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.522+0000 7fdc6e7fc700 0 client.28408 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:21:16.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.522+0000 7fdca10fab80 1 --2- 172.21.15.101:0/3725254747 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5560b1752be0 0x5560b17b50e0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.522+0000 7fdca10fab80 1 -- 172.21.15.101:0/3725254747 >> 172.21.15.101:0/3725254747 conn(0x5560b1748da0 msgr2=0x5560b1768680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:16.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.522+0000 7fdca10fab80 1 -- 172.21.15.101:0/3725254747 shutdown_connections 2023-07-18T03:21:16.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.522+0000 7fdca10fab80 1 -- 172.21.15.101:0/3725254747 wait complete. 2023-07-18T03:21:16.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.523+0000 7fdca10fab80 20 client.28408 trim_cache size 0 max 16384 2023-07-18T03:21:16.528 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:21:16.528 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "scrub active (3 inodes in the stack)", 2023-07-18T03:21:16.528 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:21:16.528 INFO:teuthology.orchestra.run.smithi101.stdout: "33787411-14d9-4062-bc85-e96f22f4cbfa": { 2023-07-18T03:21:16.529 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:21:16.529 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "33787411-14d9-4062-bc85-e96f22f4cbfa", 2023-07-18T03:21:16.529 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:21:16.529 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:21:16.529 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:21:16.529 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:21:16.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.523+0000 7fdc8d7ff700 1 -- 172.21.15.101:0/1019818639 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fdc740a1340 msgr2=0x7fdc740a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:16.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.523+0000 7fdc8d7ff700 1 --2- 172.21.15.101:0/1019818639 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fdc740a1340 0x7fdc740a37f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdc7000b2e0 tx=0x7fdc70005fb0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.523+0000 7fdc8d7ff700 1 -- 172.21.15.101:0/1019818639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc880ffad0 msgr2=0x7fdc88106990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:16.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.523+0000 7fdc8d7ff700 1 --2- 172.21.15.101:0/1019818639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc880ffad0 0x7fdc88106990 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fdc7800e9a0 tx=0x7fdc7800ecb0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.523+0000 7fdc8d7ff700 1 -- 172.21.15.101:0/1019818639 shutdown_connections 2023-07-18T03:21:16.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.523+0000 7fdc8d7ff700 1 --2- 172.21.15.101:0/1019818639 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fdc740a1340 0x7fdc740a37f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.523+0000 7fdc8d7ff700 1 --2- 172.21.15.101:0/1019818639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc880ffad0 0x7fdc88106990 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.523+0000 7fdc8d7ff700 1 --2- 172.21.15.101:0/1019818639 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc88105ae0 0x7fdc88107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.523+0000 7fdc8d7ff700 1 --2- 172.21.15.101:0/1019818639 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdc88100480 0x7fdc88106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:16.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.523+0000 7fdc8d7ff700 1 -- 172.21.15.101:0/1019818639 >> 172.21.15.101:0/1019818639 conn(0x7fdc880fb5f0 msgr2=0x7fdc8810af10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:16.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.524+0000 7fdc8d7ff700 1 -- 172.21.15.101:0/1019818639 shutdown_connections 2023-07-18T03:21:16.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:16.524+0000 7fdc8d7ff700 1 -- 172.21.15.101:0/1019818639 wait complete. 2023-07-18T03:21:16.540 INFO:tasks.cephfs.filesystem:scrub status for tag:33787411-14d9-4062-bc85-e96f22f4cbfa - {'path': '/', 'tag': '33787411-14d9-4062-bc85-e96f22f4cbfa', 'options': 'recursive,force'} 2023-07-18T03:21:16.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:16 smithi155 ceph-mon[128885]: pgmap v430: 129 pgs: 129 active+clean; 1006 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 9.6 MiB/s wr, 542 op/s 2023-07-18T03:21:16.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:16 smithi179 ceph-mon[129229]: pgmap v430: 129 pgs: 129 active+clean; 1006 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 9.6 MiB/s wr, 542 op/s 2023-07-18T03:21:16.955 INFO:tasks.workunit.client.0.smithi101.stdout: 10 19373 29.67 MB/sec warmup 41 sec latency 1640.512 ms 2023-07-18T03:21:17.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:17 smithi155 ceph-mon[128885]: scrub queued for path: / 2023-07-18T03:21:17.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:17 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:21:17.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:17 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:21:17.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:17 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:21:17.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:17 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:21:17.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:17 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:21:17.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:17 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:21:17.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:17 smithi179 ceph-mon[129229]: scrub queued for path: / 2023-07-18T03:21:17.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:17 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:21:17.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:17 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:21:17.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:17 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:21:17.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:17 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:21:17.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:17 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:21:17.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:17 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:21:17.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:17 smithi101 ceph-mon[126379]: scrub queued for path: / 2023-07-18T03:21:17.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:17 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:21:17.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:17 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:21:17.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:17 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:21:17.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:17 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:21:17.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:17 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:21:17.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:17 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:21:17.955 INFO:tasks.workunit.client.0.smithi101.stdout: 10 19951 29.79 MB/sec warmup 42 sec latency 2640.570 ms 2023-07-18T03:21:18.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:18 smithi155 ceph-mon[128885]: pgmap v431: 129 pgs: 129 active+clean; 1006 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 9.1 MiB/s wr, 515 op/s 2023-07-18T03:21:18.705 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:18 smithi179 ceph-mon[129229]: pgmap v431: 129 pgs: 129 active+clean; 1006 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 9.1 MiB/s wr, 515 op/s 2023-07-18T03:21:18.745 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:18 smithi101 ceph-mon[126379]: pgmap v431: 129 pgs: 129 active+clean; 1006 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 9.1 MiB/s wr, 515 op/s 2023-07-18T03:21:18.956 INFO:tasks.workunit.client.0.smithi101.stdout: 10 20244 29.44 MB/sec warmup 43 sec latency 3641.218 ms 2023-07-18T03:21:19.956 INFO:tasks.workunit.client.0.smithi101.stdout: 10 20572 29.04 MB/sec warmup 44 sec latency 4641.281 ms 2023-07-18T03:21:20.956 INFO:tasks.workunit.client.0.smithi101.stdout: 10 20900 28.75 MB/sec warmup 45 sec latency 5641.335 ms 2023-07-18T03:21:21.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:21 smithi179 ceph-mon[129229]: pgmap v432: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 19 MiB/s wr, 682 op/s 2023-07-18T03:21:21.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:21 smithi179 ceph-mon[129229]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:21:21.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:21 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:21:21.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:21 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:21:21.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:21 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:21:21.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:21 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:21:21.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:21 smithi101 ceph-mon[126379]: pgmap v432: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 19 MiB/s wr, 682 op/s 2023-07-18T03:21:21.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:21 smithi101 ceph-mon[126379]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:21:21.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:21 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:21:21.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:21 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:21:21.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:21 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:21:21.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:21 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:21:21.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:21 smithi155 ceph-mon[128885]: pgmap v432: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 19 MiB/s wr, 682 op/s 2023-07-18T03:21:21.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:21 smithi155 ceph-mon[128885]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:21:21.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:21 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:21:21.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:21 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:21:21.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:21 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:21:21.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:21 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:21:21.956 INFO:tasks.workunit.client.0.smithi101.stdout: 10 21558 28.85 MB/sec warmup 46 sec latency 6641.384 ms 2023-07-18T03:21:22.956 INFO:tasks.workunit.client.0.smithi101.stdout: 10 22302 29.11 MB/sec warmup 47 sec latency 7641.449 ms 2023-07-18T03:21:23.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:23 smithi179 ceph-mon[129229]: pgmap v433: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 19 MiB/s wr, 682 op/s 2023-07-18T03:21:23.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:23 smithi101 ceph-mon[126379]: pgmap v433: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 19 MiB/s wr, 682 op/s 2023-07-18T03:21:23.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:23 smithi155 ceph-mon[128885]: pgmap v433: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 19 MiB/s wr, 682 op/s 2023-07-18T03:21:23.957 INFO:tasks.workunit.client.0.smithi101.stdout: 10 22881 29.16 MB/sec warmup 48 sec latency 8642.327 ms 2023-07-18T03:21:24.957 INFO:tasks.workunit.client.0.smithi101.stdout: 10 23529 29.14 MB/sec warmup 49 sec latency 9642.390 ms 2023-07-18T03:21:25.454 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:25 smithi179 ceph-mon[129229]: pgmap v434: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 19 MiB/s wr, 682 op/s 2023-07-18T03:21:25.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:25 smithi101 ceph-mon[126379]: pgmap v434: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 19 MiB/s wr, 682 op/s 2023-07-18T03:21:25.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:25 smithi155 ceph-mon[128885]: pgmap v434: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 19 MiB/s wr, 682 op/s 2023-07-18T03:21:25.957 INFO:tasks.workunit.client.0.smithi101.stdout: 10 24479 29.79 MB/sec warmup 50 sec latency 10202.787 ms 2023-07-18T03:21:26.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:26 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:21:26.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:26 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:21:26.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:26 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:21:26.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:26 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:21:26.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:26 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:21:26.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:26 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:21:26.957 INFO:tasks.workunit.client.0.smithi101.stdout: 10 25348 30.33 MB/sec warmup 51 sec latency 1291.009 ms 2023-07-18T03:21:27.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:27 smithi179 ceph-mon[129229]: pgmap v435: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 19 MiB/s wr, 887 op/s 2023-07-18T03:21:27.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:27 smithi101 ceph-mon[126379]: pgmap v435: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 19 MiB/s wr, 887 op/s 2023-07-18T03:21:27.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:27 smithi155 ceph-mon[128885]: pgmap v435: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 19 MiB/s wr, 887 op/s 2023-07-18T03:21:27.957 INFO:tasks.workunit.client.0.smithi101.stdout: 10 25967 30.37 MB/sec warmup 52 sec latency 2291.071 ms 2023-07-18T03:21:28.958 INFO:tasks.workunit.client.0.smithi101.stdout: 10 26752 30.54 MB/sec warmup 53 sec latency 3291.535 ms 2023-07-18T03:21:29.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:29 smithi179 ceph-mon[129229]: pgmap v436: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 13 MiB/s wr, 649 op/s 2023-07-18T03:21:29.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:29 smithi101 ceph-mon[126379]: pgmap v436: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 13 MiB/s wr, 649 op/s 2023-07-18T03:21:29.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:29 smithi155 ceph-mon[128885]: pgmap v436: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 13 MiB/s wr, 649 op/s 2023-07-18T03:21:29.961 INFO:tasks.workunit.client.0.smithi101.stdout: 10 27391 30.61 MB/sec warmup 54 sec latency 3862.718 ms 2023-07-18T03:21:30.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:30 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:21:30.454 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:30 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:21:30.454 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:30 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:21:30.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:30 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:21:30.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:30 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:21:30.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:30 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:21:30.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:30 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:21:30.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:30 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:21:30.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:30 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:21:30.960 INFO:tasks.workunit.client.0.smithi101.stdout: 10 28219 30.82 MB/sec warmup 55 sec latency 147.160 ms 2023-07-18T03:21:31.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:31 smithi179 ceph-mon[129229]: pgmap v437: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 16 MiB/s wr, 877 op/s 2023-07-18T03:21:31.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:31 smithi101 ceph-mon[126379]: pgmap v437: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 16 MiB/s wr, 877 op/s 2023-07-18T03:21:31.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:31 smithi155 ceph-mon[128885]: pgmap v437: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 16 MiB/s wr, 877 op/s 2023-07-18T03:21:31.960 INFO:tasks.workunit.client.0.smithi101.stdout: 10 29227 31.24 MB/sec warmup 56 sec latency 49.298 ms 2023-07-18T03:21:32.960 INFO:tasks.workunit.client.0.smithi101.stdout: 10 30048 31.56 MB/sec warmup 57 sec latency 49.627 ms 2023-07-18T03:21:33.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:33 smithi179 ceph-mon[129229]: pgmap v438: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 5.7 MiB/s wr, 524 op/s 2023-07-18T03:21:33.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:33 smithi101 ceph-mon[126379]: pgmap v438: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 5.7 MiB/s wr, 524 op/s 2023-07-18T03:21:33.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:33 smithi155 ceph-mon[128885]: pgmap v438: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 5.7 MiB/s wr, 524 op/s 2023-07-18T03:21:33.960 INFO:tasks.workunit.client.0.smithi101.stdout: 10 30681 31.55 MB/sec warmup 58 sec latency 60.408 ms 2023-07-18T03:21:34.960 INFO:tasks.workunit.client.0.smithi101.stdout: 10 31370 31.50 MB/sec warmup 59 sec latency 90.382 ms 2023-07-18T03:21:35.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:35 smithi179 ceph-mon[129229]: pgmap v439: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 5.7 MiB/s wr, 524 op/s 2023-07-18T03:21:35.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:35 smithi101 ceph-mon[126379]: pgmap v439: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 5.7 MiB/s wr, 524 op/s 2023-07-18T03:21:35.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:35 smithi155 ceph-mon[128885]: pgmap v439: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 5.7 MiB/s wr, 524 op/s 2023-07-18T03:21:35.960 INFO:tasks.workunit.client.0.smithi101.stdout: 10 32393 31.87 MB/sec warmup 60 sec latency 55.867 ms 2023-07-18T03:21:36.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:36 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:21:36.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:36 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:21:36.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:36 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:21:36.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:36 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:21:36.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:36 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:21:36.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:36 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:21:36.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:36 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:21:36.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:36 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:21:36.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:36 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:21:36.960 INFO:tasks.workunit.client.0.smithi101.stdout: 10 33102 32.07 MB/sec warmup 61 sec latency 290.679 ms 2023-07-18T03:21:37.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:37 smithi179 ceph-mon[129229]: pgmap v440: 129 pgs: 129 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.5 MiB/s wr, 753 op/s 2023-07-18T03:21:37.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:37 smithi101 ceph-mon[126379]: pgmap v440: 129 pgs: 129 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.5 MiB/s wr, 753 op/s 2023-07-18T03:21:37.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:37 smithi155 ceph-mon[128885]: pgmap v440: 129 pgs: 129 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.5 MiB/s wr, 753 op/s 2023-07-18T03:21:37.960 INFO:tasks.workunit.client.0.smithi101.stdout: 10 34045 32.44 MB/sec warmup 62 sec latency 79.974 ms 2023-07-18T03:21:38.960 INFO:tasks.workunit.client.0.smithi101.stdout: 10 34687 32.46 MB/sec warmup 63 sec latency 131.384 ms 2023-07-18T03:21:39.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:39 smithi179 ceph-mon[129229]: pgmap v441: 129 pgs: 129 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 5.9 MiB/s wr, 456 op/s 2023-07-18T03:21:39.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:39 smithi101 ceph-mon[126379]: pgmap v441: 129 pgs: 129 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 5.9 MiB/s wr, 456 op/s 2023-07-18T03:21:39.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:39 smithi155 ceph-mon[128885]: pgmap v441: 129 pgs: 129 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 5.9 MiB/s wr, 456 op/s 2023-07-18T03:21:39.960 INFO:tasks.workunit.client.0.smithi101.stdout: 10 35360 32.59 MB/sec warmup 64 sec latency 71.075 ms 2023-07-18T03:21:40.960 INFO:tasks.workunit.client.0.smithi101.stdout: 10 36164 32.56 MB/sec warmup 65 sec latency 134.271 ms 2023-07-18T03:21:41.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:41 smithi179 ceph-mon[129229]: pgmap v442: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.7 MiB/s wr, 679 op/s 2023-07-18T03:21:41.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:41 smithi101 ceph-mon[126379]: pgmap v442: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.7 MiB/s wr, 679 op/s 2023-07-18T03:21:41.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:41 smithi155 ceph-mon[128885]: pgmap v442: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.7 MiB/s wr, 679 op/s 2023-07-18T03:21:41.960 INFO:tasks.workunit.client.0.smithi101.stdout: 10 36943 32.76 MB/sec warmup 66 sec latency 64.935 ms 2023-07-18T03:21:42.961 INFO:tasks.workunit.client.0.smithi101.stdout: 10 37825 33.10 MB/sec warmup 67 sec latency 56.411 ms 2023-07-18T03:21:43.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:43 smithi179 ceph-mon[129229]: pgmap v443: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 5.7 MiB/s wr, 452 op/s 2023-07-18T03:21:43.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:43 smithi101 ceph-mon[126379]: pgmap v443: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 5.7 MiB/s wr, 452 op/s 2023-07-18T03:21:43.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:43 smithi155 ceph-mon[128885]: pgmap v443: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 5.7 MiB/s wr, 452 op/s 2023-07-18T03:21:43.961 INFO:tasks.workunit.client.0.smithi101.stdout: 10 38525 33.08 MB/sec warmup 68 sec latency 61.120 ms 2023-07-18T03:21:44.967 INFO:tasks.workunit.client.0.smithi101.stdout: 10 38963 32.88 MB/sec warmup 69 sec latency 448.026 ms 2023-07-18T03:21:45.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:45 smithi179 ceph-mon[129229]: pgmap v444: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 5.7 MiB/s wr, 452 op/s 2023-07-18T03:21:45.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:45 smithi101 ceph-mon[126379]: pgmap v444: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 5.7 MiB/s wr, 452 op/s 2023-07-18T03:21:45.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:45 smithi155 ceph-mon[128885]: pgmap v444: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 5.7 MiB/s wr, 452 op/s 2023-07-18T03:21:45.963 INFO:tasks.workunit.client.0.smithi101.stdout: 10 39023 32.48 MB/sec warmup 70 sec latency 1239.367 ms 2023-07-18T03:21:46.541 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:21:46.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.894+0000 7f25b8bff700 1 -- 172.21.15.101:0/2954480358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25b40fea90 msgr2=0x7f25b40fee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:46.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.894+0000 7f25b8bff700 1 --2- 172.21.15.101:0/2954480358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25b40fea90 0x7f25b40fee60 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f25a4009990 tx=0x7f25a4009ca0 comp rx=0 tx=0).stop 2023-07-18T03:21:46.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.894+0000 7f25b8bff700 1 -- 172.21.15.101:0/2954480358 shutdown_connections 2023-07-18T03:21:46.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.894+0000 7f25b8bff700 1 --2- 172.21.15.101:0/2954480358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25b40fea90 0x7f25b40fee60 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:46.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.894+0000 7f25b8bff700 1 --2- 172.21.15.101:0/2954480358 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f25b4103e40 0x7f25b41042e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:46.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.894+0000 7f25b8bff700 1 --2- 172.21.15.101:0/2954480358 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25b40ff3a0 0x7f25b4103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:46.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.894+0000 7f25b8bff700 1 -- 172.21.15.101:0/2954480358 >> 172.21.15.101:0/2954480358 conn(0x7f25b40fa4b0 msgr2=0x7f25b40fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:46.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.895+0000 7f25b8bff700 1 -- 172.21.15.101:0/2954480358 shutdown_connections 2023-07-18T03:21:46.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.895+0000 7f25b8bff700 1 -- 172.21.15.101:0/2954480358 wait complete. 2023-07-18T03:21:46.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.896+0000 7f25b8bff700 1 Processor -- start 2023-07-18T03:21:46.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.896+0000 7f25b8bff700 1 -- start start 2023-07-18T03:21:46.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.896+0000 7f25b8bff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25b40fea90 0x7f25b41106e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:46.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.896+0000 7f25b8bff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25b40ff3a0 0x7f25b4110c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:46.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.896+0000 7f25b8bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f25b4103e40 0x7f25b410b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:46.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.896+0000 7f25b8bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b410bef0 con 0x7f25b4103e40 2023-07-18T03:21:46.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.896+0000 7f25b8bff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b410c060 con 0x7f25b40fea90 2023-07-18T03:21:46.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.896+0000 7f25b8bff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b410c220 con 0x7f25b40ff3a0 2023-07-18T03:21:46.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.897+0000 7f25b259c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25b40fea90 0x7f25b41106e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:46.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.897+0000 7f25b259c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25b40fea90 0x7f25b41106e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47304/0 (socket says 172.21.15.101:47304) 2023-07-18T03:21:46.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.897+0000 7f25b259c700 1 -- 172.21.15.101:0/1055289104 learned_addr learned my addr 172.21.15.101:0/1055289104 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:46.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.897+0000 7f25b1d9b700 1 --2- 172.21.15.101:0/1055289104 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25b40ff3a0 0x7f25b4110c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:46.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.897+0000 7f25b1d9b700 1 -- 172.21.15.101:0/1055289104 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25b40fea90 msgr2=0x7f25b41106e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:46.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.897+0000 7f25b1d9b700 1 --2- 172.21.15.101:0/1055289104 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25b40fea90 0x7f25b41106e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:46.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.897+0000 7f25b1d9b700 1 -- 172.21.15.101:0/1055289104 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f25b4103e40 msgr2=0x7f25b410b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:46.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.897+0000 7f25b1d9b700 1 --2- 172.21.15.101:0/1055289104 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f25b4103e40 0x7f25b410b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:46.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.897+0000 7f25b1d9b700 1 -- 172.21.15.101:0/1055289104 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a40095f0 con 0x7f25b40ff3a0 2023-07-18T03:21:46.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.898+0000 7f25b1d9b700 1 --2- 172.21.15.101:0/1055289104 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25b40ff3a0 0x7f25b4110c20 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f25a8006fd0 tx=0x7f25a800b7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:46.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.899+0000 7f25a37fe700 1 -- 172.21.15.101:0/1055289104 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a8011830 con 0x7f25b40ff3a0 2023-07-18T03:21:46.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.899+0000 7f25a37fe700 1 -- 172.21.15.101:0/1055289104 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25a8011e50 con 0x7f25b40ff3a0 2023-07-18T03:21:46.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.899+0000 7f25a37fe700 1 -- 172.21.15.101:0/1055289104 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a8010e80 con 0x7f25b40ff3a0 2023-07-18T03:21:46.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.899+0000 7f25b8bff700 1 -- 172.21.15.101:0/1055289104 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25b410c500 con 0x7f25b40ff3a0 2023-07-18T03:21:46.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.899+0000 7f25b8bff700 1 -- 172.21.15.101:0/1055289104 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25b410f620 con 0x7f25b40ff3a0 2023-07-18T03:21:46.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.906+0000 7f25cc455b80 1 Processor -- start 2023-07-18T03:21:46.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.906+0000 7f25a37fe700 1 -- 172.21.15.101:0/1055289104 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f25a8010380 con 0x7f25b40ff3a0 2023-07-18T03:21:46.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.906+0000 7f25cc455b80 1 -- start start 2023-07-18T03:21:46.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.907+0000 7f25cc455b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55780a3842d0 0x55780a3846a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:46.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.907+0000 7f25cc455b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55780a384be0 0x55780a38f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:46.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.907+0000 7f25cc455b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55780a38fad0 0x55780a391eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:46.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.907+0000 7f25cc455b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55780a28acb0 con 0x55780a38fad0 2023-07-18T03:21:46.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.907+0000 7f25cc455b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55780a2dda40 con 0x55780a384be0 2023-07-18T03:21:46.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.907+0000 7f25cc455b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55780a2d6ed0 con 0x55780a3842d0 2023-07-18T03:21:46.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.907+0000 7f25b2d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55780a3842d0 0x55780a3846a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:46.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.907+0000 7f25b2d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55780a3842d0 0x55780a3846a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:56926/0 (socket says 172.21.15.101:56926) 2023-07-18T03:21:46.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.907+0000 7f25b2d9d700 1 -- 172.21.15.101:0/3320524650 learned_addr learned my addr 172.21.15.101:0/3320524650 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:46.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.908+0000 7f25b259c700 1 --2- 172.21.15.101:0/3320524650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55780a384be0 0x55780a38f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:46.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.908+0000 7f25b1d9b700 1 --2- 172.21.15.101:0/3320524650 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55780a38fad0 0x55780a391eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:46.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.908+0000 7f25b259c700 1 -- 172.21.15.101:0/3320524650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55780a3842d0 msgr2=0x55780a3846a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:46.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.908+0000 7f25b259c700 1 --2- 172.21.15.101:0/3320524650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55780a3842d0 0x55780a3846a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:46.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.908+0000 7f25b259c700 1 -- 172.21.15.101:0/3320524650 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55780a38fad0 msgr2=0x55780a391eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:46.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.908+0000 7f25b259c700 1 --2- 172.21.15.101:0/3320524650 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55780a38fad0 0x55780a391eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:46.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.908+0000 7f25b259c700 1 -- 172.21.15.101:0/3320524650 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55780a104de0 con 0x55780a384be0 2023-07-18T03:21:46.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.908+0000 7f25b259c700 1 --2- 172.21.15.101:0/3320524650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55780a384be0 0x55780a38f590 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f25a40098e0 tx=0x7f25a4005da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4f75c18e3a537a3 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:46.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.909+0000 7f25a37fe700 1 --2- 172.21.15.101:0/1055289104 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f259c0a1340 0x7f259c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:46.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.909+0000 7f25a37fe700 1 -- 172.21.15.101:0/1055289104 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f25a80ccac0 con 0x7f25b40ff3a0 2023-07-18T03:21:46.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.909+0000 7f25a0ff9700 1 -- 172.21.15.101:0/3320524650 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a4005340 con 0x55780a384be0 2023-07-18T03:21:46.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.909+0000 7f25a0ff9700 1 -- 172.21.15.101:0/3320524650 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25a401e040 con 0x55780a384be0 2023-07-18T03:21:46.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.909+0000 7f25a0ff9700 1 -- 172.21.15.101:0/3320524650 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a4003c50 con 0x55780a384be0 2023-07-18T03:21:46.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.909+0000 7f25b2d9d700 1 --2- 172.21.15.101:0/1055289104 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f259c0a1340 0x7f259c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:46.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.910+0000 7f25cc455b80 1 -- 172.21.15.101:0/3320524650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55780a384be0 msgr2=0x55780a38f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:46.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.910+0000 7f25cc455b80 1 --2- 172.21.15.101:0/3320524650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55780a384be0 0x55780a38f590 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f25a40098e0 tx=0x7f25a4005da0 comp rx=0 tx=0).stop 2023-07-18T03:21:46.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.910+0000 7f25cc455b80 1 -- 172.21.15.101:0/3320524650 shutdown_connections 2023-07-18T03:21:46.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.910+0000 7f25cc455b80 1 --2- 172.21.15.101:0/3320524650 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55780a38fad0 0x55780a391eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:46.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.910+0000 7f25cc455b80 1 --2- 172.21.15.101:0/3320524650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55780a3842d0 0x55780a3846a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:46.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.910+0000 7f25cc455b80 1 --2- 172.21.15.101:0/3320524650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55780a384be0 0x55780a38f590 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:46.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.910+0000 7f25cc455b80 1 -- 172.21.15.101:0/3320524650 >> 172.21.15.101:0/3320524650 conn(0x55780a37ada0 msgr2=0x55780a3806a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:46.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.910+0000 7f25b2d9d700 1 --2- 172.21.15.101:0/1055289104 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f259c0a1340 0x7f259c0a37f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f25ac009e70 tx=0x7f25ac0094f0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:46.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.911+0000 7f25cc455b80 1 -- 172.21.15.101:0/3320524650 shutdown_connections 2023-07-18T03:21:46.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.911+0000 7f25cc455b80 1 -- 172.21.15.101:0/3320524650 wait complete. 2023-07-18T03:21:46.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.912+0000 7f25cc455b80 1 -- start start 2023-07-18T03:21:46.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.912+0000 7f25cc455b80 1 Processor -- start 2023-07-18T03:21:46.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.912+0000 7f25cc455b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55780a3842d0 0x55780a38f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:46.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.912+0000 7f25cc455b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55780a384be0 0x55780a38c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:46.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.912+0000 7f25cc455b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55780a38fad0 0x55780a38ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:46.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.912+0000 7f25cc455b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55780a276ac0 con 0x55780a384be0 2023-07-18T03:21:46.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.912+0000 7f25cc455b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55780a2767c0 con 0x55780a38fad0 2023-07-18T03:21:46.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.912+0000 7f25cc455b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55780a15a920 con 0x55780a3842d0 2023-07-18T03:21:46.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.912+0000 7f25cc455b80 10 client.0 _ll_register_callbacks cb 0x55780a274760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:21:46.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.913+0000 7f25b2d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55780a38fad0 0x55780a38ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:46.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.913+0000 7f25b2d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55780a38fad0 0x55780a38ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47332/0 (socket says 172.21.15.101:47332) 2023-07-18T03:21:46.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.913+0000 7f25b2d9d700 1 -- 172.21.15.101:0/4144889936 learned_addr learned my addr 172.21.15.101:0/4144889936 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:46.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.913+0000 7f25b2d9d700 1 -- 172.21.15.101:0/4144889936 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55780a3842d0 msgr2=0x55780a38f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:46.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.913+0000 7f25b2d9d700 1 --2- 172.21.15.101:0/4144889936 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55780a3842d0 0x55780a38f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:46.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.913+0000 7f25b2d9d700 1 -- 172.21.15.101:0/4144889936 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55780a384be0 msgr2=0x55780a38c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:46.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.913+0000 7f25b2d9d700 1 --2- 172.21.15.101:0/4144889936 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55780a384be0 0x55780a38c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:46.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.913+0000 7f25b2d9d700 1 -- 172.21.15.101:0/4144889936 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25b41137c0 con 0x55780a38fad0 2023-07-18T03:21:46.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.914+0000 7f25b2d9d700 1 --2- 172.21.15.101:0/4144889936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55780a38fad0 0x55780a38ce50 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f25ac017a40 tx=0x7f25ac004bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f497bdda5a0c9eb server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:46.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.914+0000 7f2599ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:21:46.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.916+0000 7f2599ffb700 1 -- 172.21.15.101:0/4144889936 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac01c070 con 0x55780a38fad0 2023-07-18T03:21:46.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.916+0000 7f2599ffb700 1 -- 172.21.15.101:0/4144889936 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f25ac0157c0 con 0x55780a38fad0 2023-07-18T03:21:46.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.916+0000 7f25cc455b80 1 -- 172.21.15.101:0/4144889936 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55780a2304f0 con 0x55780a38fad0 2023-07-18T03:21:46.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.916+0000 7f2599ffb700 1 -- 172.21.15.101:0/4144889936 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25ac015980 con 0x55780a38fad0 2023-07-18T03:21:46.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.917+0000 7f25cc455b80 10 client.28385 fetch_fsmap learned FSMap version 38 2023-07-18T03:21:46.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.917+0000 7f25cc455b80 1 -- 172.21.15.101:0/4144889936 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55780a0edc60 con 0x55780a38fad0 2023-07-18T03:21:46.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.917+0000 7f2599ffb700 1 -- 172.21.15.101:0/4144889936 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55780a2304f0 con 0x55780a38fad0 2023-07-18T03:21:46.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.917+0000 7f2599ffb700 1 -- 172.21.15.101:0/4144889936 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 38) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f25ac02bc80 con 0x55780a38fad0 2023-07-18T03:21:46.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.917+0000 7f25cc455b80 10 client.28385 fetch_fsmap finished waiting for FSMap version 38 2023-07-18T03:21:46.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.917+0000 7f25cc455b80 10 client.28385 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:21:46.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.917+0000 7f25cc455b80 20 client.28385 populate_metadata read hostname 'smithi101' 2023-07-18T03:21:46.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.917+0000 7f25cc455b80 1 --2- 172.21.15.101:0/4144889936 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55780a39f380 0x55780a3a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:46.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.917+0000 7f25cc455b80 4 client.28385 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:21:46.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.917+0000 7f25cc455b80 1 -- 172.21.15.101:0/4144889936 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55780a2cfeb0 con 0x55780a39f380 2023-07-18T03:21:46.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.924+0000 7f25b259c700 1 --2- 172.21.15.101:0/4144889936 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55780a39f380 0x55780a3a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:46.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.937+0000 7f25b259c700 1 --2- 172.21.15.101:0/4144889936 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55780a39f380 0x55780a3a17b0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da781d5d1dc2c61d server_cookie=a08466e126ed91e1 in_seq=0 out_seq=0 2023-07-18T03:21:46.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.937+0000 7f2599ffb700 10 client.28385 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:21:46.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.942+0000 7f2599ffb700 1 -- 172.21.15.101:0/4144889936 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55780a2cfeb0 con 0x55780a39f380 2023-07-18T03:21:46.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.943+0000 7f2599ffb700 10 client.28385 handle_command_reply: tid=0 2023-07-18T03:21:46.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.943+0000 7f25cc455b80 1 client.28385 shutdown 2023-07-18T03:21:46.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.943+0000 7f25cc455b80 1 -- 172.21.15.101:0/4144889936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55780a38fad0 msgr2=0x55780a38ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:46.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.943+0000 7f25cc455b80 1 --2- 172.21.15.101:0/4144889936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55780a38fad0 0x55780a38ce50 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f25ac017a40 tx=0x7f25ac004bf0 comp rx=0 tx=0).stop 2023-07-18T03:21:46.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.945+0000 7f25cc455b80 1 -- 172.21.15.101:0/4144889936 shutdown_connections 2023-07-18T03:21:46.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.945+0000 7f25cc455b80 1 --2- 172.21.15.101:0/4144889936 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55780a39f380 0x55780a3a17b0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:46.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.945+0000 7f25cc455b80 1 --2- 172.21.15.101:0/4144889936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55780a38fad0 0x55780a38ce50 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:46.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.945+0000 7f25cc455b80 1 --2- 172.21.15.101:0/4144889936 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55780a3842d0 0x55780a38f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:46.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.945+0000 7f25cc455b80 1 --2- 172.21.15.101:0/4144889936 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55780a384be0 0x55780a38c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:46.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.945+0000 7f25cc455b80 1 -- 172.21.15.101:0/4144889936 >> 172.21.15.101:0/4144889936 conn(0x55780a37ada0 msgr2=0x55780a380290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:46.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.945+0000 7f2599ffb700 0 client.28385 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:21:46.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.945+0000 7f25cc455b80 1 -- 172.21.15.101:0/4144889936 shutdown_connections 2023-07-18T03:21:46.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.945+0000 7f25cc455b80 1 -- 172.21.15.101:0/4144889936 wait complete. 2023-07-18T03:21:46.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.946+0000 7f25cc455b80 20 client.28385 trim_cache size 0 max 16384 2023-07-18T03:21:46.963 INFO:tasks.workunit.client.0.smithi101.stdout: 10 39314 32.18 MB/sec warmup 71 sec latency 361.210 ms 2023-07-18T03:21:46.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.969+0000 7f25cc455b80 1 Processor -- start 2023-07-18T03:21:46.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.969+0000 7f25cc455b80 1 -- start start 2023-07-18T03:21:46.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.970+0000 7f25cc455b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55780a3842d0 0x55780a3846a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:46.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.970+0000 7f25cc455b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55780a384be0 0x55780a3871b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:46.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.970+0000 7f25cc455b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55780a38fad0 0x55780a38f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:46.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.970+0000 7f25cc455b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55780a39ca20 con 0x55780a384be0 2023-07-18T03:21:46.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.970+0000 7f25cc455b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55780a38cdb0 con 0x55780a3842d0 2023-07-18T03:21:46.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.970+0000 7f25cc455b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55780a38cf70 con 0x55780a38fad0 2023-07-18T03:21:46.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.971+0000 7f25b259c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55780a3842d0 0x55780a3846a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:46.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.971+0000 7f25b259c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55780a3842d0 0x55780a3846a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47342/0 (socket says 172.21.15.101:47342) 2023-07-18T03:21:46.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.971+0000 7f25b259c700 1 -- 172.21.15.101:0/92384770 learned_addr learned my addr 172.21.15.101:0/92384770 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:46.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.971+0000 7f25b2d9d700 1 --2- 172.21.15.101:0/92384770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55780a38fad0 0x55780a38f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:46.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.971+0000 7f25b259c700 1 -- 172.21.15.101:0/92384770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55780a38fad0 msgr2=0x55780a38f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:46.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.971+0000 7f25b259c700 1 --2- 172.21.15.101:0/92384770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55780a38fad0 0x55780a38f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:46.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.971+0000 7f25b259c700 1 -- 172.21.15.101:0/92384770 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55780a384be0 msgr2=0x55780a3871b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:46.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.971+0000 7f25b259c700 1 --2- 172.21.15.101:0/92384770 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55780a384be0 0x55780a3871b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:46.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.971+0000 7f25b259c700 1 -- 172.21.15.101:0/92384770 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55780a12b250 con 0x55780a3842d0 2023-07-18T03:21:46.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.972+0000 7f25b259c700 1 --2- 172.21.15.101:0/92384770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55780a3842d0 0x55780a3846a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f25a4025040 tx=0x7f25a4003740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f63fd2efd7700e8 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:46.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.972+0000 7f25a17fa700 1 -- 172.21.15.101:0/92384770 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a400bbb0 con 0x55780a3842d0 2023-07-18T03:21:46.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.972+0000 7f25a17fa700 1 -- 172.21.15.101:0/92384770 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25a400b770 con 0x55780a3842d0 2023-07-18T03:21:46.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.972+0000 7f25a17fa700 1 -- 172.21.15.101:0/92384770 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a4005320 con 0x55780a3842d0 2023-07-18T03:21:46.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.973+0000 7f25cc455b80 1 -- 172.21.15.101:0/92384770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55780a3842d0 msgr2=0x55780a3846a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:46.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.973+0000 7f25cc455b80 1 --2- 172.21.15.101:0/92384770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55780a3842d0 0x55780a3846a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f25a4025040 tx=0x7f25a4003740 comp rx=0 tx=0).stop 2023-07-18T03:21:46.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.973+0000 7f25cc455b80 1 -- 172.21.15.101:0/92384770 shutdown_connections 2023-07-18T03:21:46.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.973+0000 7f25cc455b80 1 --2- 172.21.15.101:0/92384770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55780a3842d0 0x55780a3846a0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:46.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.973+0000 7f25cc455b80 1 --2- 172.21.15.101:0/92384770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55780a38fad0 0x55780a38f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:46.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.973+0000 7f25cc455b80 1 --2- 172.21.15.101:0/92384770 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55780a384be0 0x55780a3871b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:46.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.973+0000 7f25cc455b80 1 -- 172.21.15.101:0/92384770 >> 172.21.15.101:0/92384770 conn(0x55780a37ada0 msgr2=0x55780a37ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:46.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.975+0000 7f25cc455b80 1 -- 172.21.15.101:0/92384770 shutdown_connections 2023-07-18T03:21:46.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.990+0000 7f25cc455b80 1 -- 172.21.15.101:0/92384770 wait complete. 2023-07-18T03:21:46.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.990+0000 7f25cc455b80 1 -- start start 2023-07-18T03:21:46.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.991+0000 7f25cc455b80 1 Processor -- start 2023-07-18T03:21:46.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.991+0000 7f25cc455b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55780a3842d0 0x55780a395960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:46.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.991+0000 7f25cc455b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55780a38fad0 0x55780a395ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:46.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.991+0000 7f25cc455b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55780a3f0290 0x55780a3f2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:46.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.991+0000 7f25cc455b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55780a3965e0 con 0x55780a38fad0 2023-07-18T03:21:46.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.991+0000 7f25cc455b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55780a3f2bb0 con 0x55780a3842d0 2023-07-18T03:21:46.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.991+0000 7f25cc455b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55780a3f2cf0 con 0x55780a3f0290 2023-07-18T03:21:46.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.991+0000 7f25cc455b80 10 client.0 _ll_register_callbacks cb 0x7f25ac017430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:21:46.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.992+0000 7f25b259c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55780a3842d0 0x55780a395960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:46.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.992+0000 7f25b259c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55780a3842d0 0x55780a395960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47356/0 (socket says 172.21.15.101:47356) 2023-07-18T03:21:46.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.992+0000 7f25b259c700 1 -- 172.21.15.101:0/2938041342 learned_addr learned my addr 172.21.15.101:0/2938041342 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:46.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.992+0000 7f25b1d9b700 1 --2- 172.21.15.101:0/2938041342 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55780a38fad0 0x55780a395ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:46.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.992+0000 7f25b2d9d700 1 --2- 172.21.15.101:0/2938041342 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55780a3f0290 0x55780a3f2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:46.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.993+0000 7f25b259c700 1 -- 172.21.15.101:0/2938041342 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55780a3f0290 msgr2=0x55780a3f2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:46.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.993+0000 7f25b259c700 1 --2- 172.21.15.101:0/2938041342 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55780a3f0290 0x55780a3f2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:46.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.993+0000 7f25b259c700 1 -- 172.21.15.101:0/2938041342 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55780a38fad0 msgr2=0x55780a395ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:46.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.993+0000 7f25b259c700 1 --2- 172.21.15.101:0/2938041342 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55780a38fad0 0x55780a395ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:46.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.993+0000 7f25b259c700 1 -- 172.21.15.101:0/2938041342 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55780a104de0 con 0x55780a3842d0 2023-07-18T03:21:46.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.994+0000 7f25b259c700 1 --2- 172.21.15.101:0/2938041342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55780a3842d0 0x55780a395960 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f25a400b900 tx=0x7f25a4005c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28b9f1ea6c8b9a5c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:46.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:46.994+0000 7f2599ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:21:46.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.001+0000 7f2599ffb700 1 -- 172.21.15.101:0/2938041342 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a4031240 con 0x55780a3842d0 2023-07-18T03:21:46.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.001+0000 7f2599ffb700 1 -- 172.21.15.101:0/2938041342 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f25a4031ad0 con 0x55780a3842d0 2023-07-18T03:21:46.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.001+0000 7f25cc455b80 1 -- 172.21.15.101:0/2938041342 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f25a401e040 con 0x55780a3842d0 2023-07-18T03:21:47.000 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.005+0000 7f2599ffb700 1 -- 172.21.15.101:0/2938041342 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25a400bbc0 con 0x55780a3842d0 2023-07-18T03:21:47.000 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.005+0000 7f25cc455b80 10 client.28391 fetch_fsmap learned FSMap version 38 2023-07-18T03:21:47.000 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.005+0000 7f25cc455b80 1 -- 172.21.15.101:0/2938041342 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55780a1032b0 con 0x55780a3842d0 2023-07-18T03:21:47.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.010+0000 7f2599ffb700 1 -- 172.21.15.101:0/2938041342 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25a401e040 con 0x55780a3842d0 2023-07-18T03:21:47.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.010+0000 7f2599ffb700 1 -- 172.21.15.101:0/2938041342 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 38) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f25a400b260 con 0x55780a3842d0 2023-07-18T03:21:47.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.011+0000 7f25cc455b80 10 client.28391 fetch_fsmap finished waiting for FSMap version 38 2023-07-18T03:21:47.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.011+0000 7f25cc455b80 10 client.28391 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:21:47.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.011+0000 7f25cc455b80 20 client.28391 populate_metadata read hostname 'smithi101' 2023-07-18T03:21:47.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.011+0000 7f25cc455b80 1 --2- 172.21.15.101:0/2938041342 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55780a3f3f50 0x55780a3f6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:47.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.011+0000 7f25cc455b80 4 client.28391 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:21:47.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.011+0000 7f25cc455b80 1 -- 172.21.15.101:0/2938041342 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55780a292050 con 0x55780a3f3f50 2023-07-18T03:21:47.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.011+0000 7f25b1d9b700 1 --2- 172.21.15.101:0/2938041342 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55780a3f3f50 0x55780a3f6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:47.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.015+0000 7f25b1d9b700 1 --2- 172.21.15.101:0/2938041342 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55780a3f3f50 0x55780a3f6330 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abb88684102f2b7d server_cookie=2c59b755bac07001 in_seq=0 out_seq=0 2023-07-18T03:21:47.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.016+0000 7f2599ffb700 10 client.28391 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:21:47.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.016+0000 7f2599ffb700 1 -- 172.21.15.101:0/2938041342 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55780a292050 con 0x55780a3f3f50 2023-07-18T03:21:47.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.016+0000 7f2599ffb700 10 client.28391 handle_command_reply: tid=0 2023-07-18T03:21:47.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.016+0000 7f25cc455b80 1 client.28391 shutdown 2023-07-18T03:21:47.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.022+0000 7f25cc455b80 1 -- 172.21.15.101:0/2938041342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55780a3842d0 msgr2=0x55780a395960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:47.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.022+0000 7f25cc455b80 1 --2- 172.21.15.101:0/2938041342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55780a3842d0 0x55780a395960 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f25a400b900 tx=0x7f25a4005c50 comp rx=0 tx=0).stop 2023-07-18T03:21:47.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.023+0000 7f25cc455b80 1 -- 172.21.15.101:0/2938041342 shutdown_connections 2023-07-18T03:21:47.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.023+0000 7f25cc455b80 1 --2- 172.21.15.101:0/2938041342 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55780a3f3f50 0x55780a3f6330 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:47.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.023+0000 7f25cc455b80 1 --2- 172.21.15.101:0/2938041342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55780a3842d0 0x55780a395960 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:47.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.023+0000 7f25cc455b80 1 --2- 172.21.15.101:0/2938041342 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55780a3f0290 0x55780a3f2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:47.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.023+0000 7f25cc455b80 1 --2- 172.21.15.101:0/2938041342 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55780a38fad0 0x55780a395ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:47.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.023+0000 7f25cc455b80 1 -- 172.21.15.101:0/2938041342 >> 172.21.15.101:0/2938041342 conn(0x55780a37ada0 msgr2=0x55780a387a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:47.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.023+0000 7f2599ffb700 0 client.28391 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:21:47.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.027+0000 7f25cc455b80 1 -- 172.21.15.101:0/2938041342 shutdown_connections 2023-07-18T03:21:47.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.028+0000 7f25cc455b80 1 -- 172.21.15.101:0/2938041342 wait complete. 2023-07-18T03:21:47.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.029+0000 7f25cc455b80 20 client.28391 trim_cache size 0 max 16384 2023-07-18T03:21:47.024 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:21:47.025 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "no active scrubs running", 2023-07-18T03:21:47.025 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": {} 2023-07-18T03:21:47.025 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:21:47.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.031+0000 7f25b8bff700 1 -- 172.21.15.101:0/1055289104 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f259c0a1340 msgr2=0x7f259c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:47.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.031+0000 7f25b8bff700 1 --2- 172.21.15.101:0/1055289104 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f259c0a1340 0x7f259c0a37f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f25ac009e70 tx=0x7f25ac0094f0 comp rx=0 tx=0).stop 2023-07-18T03:21:47.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.031+0000 7f25b8bff700 1 -- 172.21.15.101:0/1055289104 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25b40ff3a0 msgr2=0x7f25b4110c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:47.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.031+0000 7f25b8bff700 1 --2- 172.21.15.101:0/1055289104 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25b40ff3a0 0x7f25b4110c20 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f25a8006fd0 tx=0x7f25a800b7e0 comp rx=0 tx=0).stop 2023-07-18T03:21:47.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.032+0000 7f25b8bff700 1 -- 172.21.15.101:0/1055289104 shutdown_connections 2023-07-18T03:21:47.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.032+0000 7f25b8bff700 1 --2- 172.21.15.101:0/1055289104 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f259c0a1340 0x7f259c0a37f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:47.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.032+0000 7f25b8bff700 1 --2- 172.21.15.101:0/1055289104 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25b40fea90 0x7f25b41106e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:47.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.032+0000 7f25b8bff700 1 --2- 172.21.15.101:0/1055289104 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f25b4103e40 0x7f25b410b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:47.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.032+0000 7f25b8bff700 1 --2- 172.21.15.101:0/1055289104 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25b40ff3a0 0x7f25b4110c20 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:47.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.032+0000 7f25b8bff700 1 -- 172.21.15.101:0/1055289104 >> 172.21.15.101:0/1055289104 conn(0x7f25b40fa4b0 msgr2=0x7f25b4101840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:47.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.032+0000 7f25b8bff700 1 -- 172.21.15.101:0/1055289104 shutdown_connections 2023-07-18T03:21:47.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.032+0000 7f25b8bff700 1 -- 172.21.15.101:0/1055289104 wait complete. 2023-07-18T03:21:47.072 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-18T03:21:47.073 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:21:47.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:47 smithi179 ceph-mon[129229]: pgmap v445: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 698 op/s 2023-07-18T03:21:47.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:47 smithi101 ceph-mon[126379]: pgmap v445: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 698 op/s 2023-07-18T03:21:47.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.512+0000 7fa00c1ff700 1 -- 172.21.15.101:0/2898450723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa004107510 msgr2=0x7fa0041078e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:47.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.512+0000 7fa00c1ff700 1 --2- 172.21.15.101:0/2898450723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa004107510 0x7fa0041078e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc008580 tx=0x7f9ffc008890 comp rx=0 tx=0).stop 2023-07-18T03:21:47.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.514+0000 7fa00c1ff700 1 -- 172.21.15.101:0/2898450723 shutdown_connections 2023-07-18T03:21:47.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.514+0000 7fa00c1ff700 1 --2- 172.21.15.101:0/2898450723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa004107510 0x7fa0041078e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:47.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.514+0000 7fa00c1ff700 1 --2- 172.21.15.101:0/2898450723 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa004076ca0 0x7fa004079150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:47.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.514+0000 7fa00c1ff700 1 --2- 172.21.15.101:0/2898450723 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa004107e20 0x7fa004076760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:47.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.514+0000 7fa00c1ff700 1 -- 172.21.15.101:0/2898450723 >> 172.21.15.101:0/2898450723 conn(0x7fa00406b090 msgr2=0x7fa004070f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:47.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.521+0000 7fa00c1ff700 1 -- 172.21.15.101:0/2898450723 shutdown_connections 2023-07-18T03:21:47.525 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.526+0000 7fa00c1ff700 1 -- 172.21.15.101:0/2898450723 wait complete. 2023-07-18T03:21:47.526 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.527+0000 7fa00c1ff700 1 Processor -- start 2023-07-18T03:21:47.526 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.529+0000 7fa00c1ff700 1 -- start start 2023-07-18T03:21:47.526 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.529+0000 7fa00c1ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa004076ca0 0x7fa00407d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:47.526 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.529+0000 7fa00c1ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa004107510 0x7fa00407db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:47.526 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.529+0000 7fa00c1ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa00407e0c0 0x7fa0041345a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:47.526 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.529+0000 7fa00c1ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa004134cb0 con 0x7fa004107510 2023-07-18T03:21:47.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.529+0000 7fa00c1ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa004134e20 con 0x7fa00407e0c0 2023-07-18T03:21:47.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.529+0000 7fa00c1ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa004134fe0 con 0x7fa004076ca0 2023-07-18T03:21:47.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.530+0000 7fa00a79c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa00407e0c0 0x7fa0041345a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:47.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.530+0000 7fa00a79c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa00407e0c0 0x7fa0041345a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47372/0 (socket says 172.21.15.101:47372) 2023-07-18T03:21:47.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.530+0000 7fa00a79c700 1 -- 172.21.15.101:0/2421118119 learned_addr learned my addr 172.21.15.101:0/2421118119 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:47.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.531+0000 7fa009f9b700 1 --2- 172.21.15.101:0/2421118119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa004076ca0 0x7fa00407d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:47.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.532+0000 7fa00a79c700 1 -- 172.21.15.101:0/2421118119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa004076ca0 msgr2=0x7fa00407d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:47.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.532+0000 7fa00a79c700 1 --2- 172.21.15.101:0/2421118119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa004076ca0 0x7fa00407d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:47.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.532+0000 7fa00a79c700 1 -- 172.21.15.101:0/2421118119 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa004107510 msgr2=0x7fa00407db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:47.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.532+0000 7fa00a79c700 1 --2- 172.21.15.101:0/2421118119 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa004107510 0x7fa00407db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:47.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.532+0000 7fa00a79c700 1 -- 172.21.15.101:0/2421118119 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ffc008230 con 0x7fa00407e0c0 2023-07-18T03:21:47.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.532+0000 7fa00a79c700 1 --2- 172.21.15.101:0/2421118119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa00407e0c0 0x7fa0041345a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9ff400d360 tx=0x7f9ff400d720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:47.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.536+0000 7fa002ffd700 1 -- 172.21.15.101:0/2421118119 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff4008db0 con 0x7fa00407e0c0 2023-07-18T03:21:47.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.536+0000 7fa002ffd700 1 -- 172.21.15.101:0/2421118119 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ff4004020 con 0x7fa00407e0c0 2023-07-18T03:21:47.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.536+0000 7fa002ffd700 1 -- 172.21.15.101:0/2421118119 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff40165e0 con 0x7fa00407e0c0 2023-07-18T03:21:47.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.536+0000 7fa00c1ff700 1 -- 172.21.15.101:0/2421118119 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0041352c0 con 0x7fa00407e0c0 2023-07-18T03:21:47.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.536+0000 7fa00c1ff700 1 -- 172.21.15.101:0/2421118119 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa004135780 con 0x7fa00407e0c0 2023-07-18T03:21:47.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.538+0000 7fa00c1ff700 1 -- 172.21.15.101:0/2421118119 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa004067300 con 0x7fa00407e0c0 2023-07-18T03:21:47.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.547+0000 7fa002ffd700 1 -- 172.21.15.101:0/2421118119 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f9ff4004190 con 0x7fa00407e0c0 2023-07-18T03:21:47.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.550+0000 7fa002ffd700 1 --2- 172.21.15.101:0/2421118119 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9fec0a14a0 0x7f9fec0a3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:47.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.550+0000 7fa002ffd700 1 -- 172.21.15.101:0/2421118119 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f9ff40ccb70 con 0x7fa00407e0c0 2023-07-18T03:21:47.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.551+0000 7fa002ffd700 1 -- 172.21.15.101:0/2421118119 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9ff40f8810 con 0x7fa00407e0c0 2023-07-18T03:21:47.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:47 smithi155 ceph-mon[128885]: pgmap v445: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 698 op/s 2023-07-18T03:21:47.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.553+0000 7fa009f9b700 1 --2- 172.21.15.101:0/2421118119 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9fec0a14a0 0x7f9fec0a3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:47.549 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:47.554+0000 7fa009f9b700 1 --2- 172.21.15.101:0/2421118119 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9fec0a14a0 0x7f9fec0a3950 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc000c00 tx=0x7f9ffc00a130 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:47.966 INFO:tasks.workunit.client.0.smithi101.stdout: 10 39672 32.03 MB/sec warmup 72 sec latency 129.623 ms 2023-07-18T03:21:48.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.061+0000 7fa00c1ff700 1 -- 172.21.15.101:0/2421118119 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa00407eb20 con 0x7fa00407e0c0 2023-07-18T03:21:48.059 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.064+0000 7fa002ffd700 1 -- 172.21.15.101:0/2421118119 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+10865 (secure 0 0 0) 0x7f9ff4095c50 con 0x7fa00407e0c0 2023-07-18T03:21:48.059 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:21:48.059 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/4192846769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":4192846769},{"type":"v1","addr":"172.21.15.101:6839","nonce":4192846769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6841/3824340175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6840","nonce":3824340175},{"type":"v1","addr":"172.21.15.101:6841","nonce":3824340175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":38,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:04:59.361185+0000","modified":"2023-07-18T03:21:20.092974+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.155:6839/2055462483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2055462483},{"type":"v1","addr":"172.21.15.155:6839","nonce":2055462483}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:21:48.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.067+0000 7fa00c1ff700 1 -- 172.21.15.101:0/2421118119 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9fec0a14a0 msgr2=0x7f9fec0a3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:48.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.067+0000 7fa00c1ff700 1 --2- 172.21.15.101:0/2421118119 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9fec0a14a0 0x7f9fec0a3950 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc000c00 tx=0x7f9ffc00a130 comp rx=0 tx=0).stop 2023-07-18T03:21:48.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.068+0000 7fa00c1ff700 1 -- 172.21.15.101:0/2421118119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa00407e0c0 msgr2=0x7fa0041345a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:48.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.068+0000 7fa00c1ff700 1 --2- 172.21.15.101:0/2421118119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa00407e0c0 0x7fa0041345a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9ff400d360 tx=0x7f9ff400d720 comp rx=0 tx=0).stop 2023-07-18T03:21:48.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.068+0000 7fa00c1ff700 1 -- 172.21.15.101:0/2421118119 shutdown_connections 2023-07-18T03:21:48.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.068+0000 7fa00c1ff700 1 --2- 172.21.15.101:0/2421118119 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9fec0a14a0 0x7f9fec0a3950 secure :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc000c00 tx=0x7f9ffc00a130 comp rx=0 tx=0).stop 2023-07-18T03:21:48.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.068+0000 7fa00c1ff700 1 --2- 172.21.15.101:0/2421118119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa00407e0c0 0x7fa0041345a0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.068+0000 7fa00c1ff700 1 --2- 172.21.15.101:0/2421118119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa004076ca0 0x7fa00407d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.068+0000 7fa00c1ff700 1 --2- 172.21.15.101:0/2421118119 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa004107510 0x7fa00407db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.068+0000 7fa00c1ff700 1 -- 172.21.15.101:0/2421118119 >> 172.21.15.101:0/2421118119 conn(0x7fa00406b090 msgr2=0x7fa00406f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:48.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.070+0000 7fa00c1ff700 1 -- 172.21.15.101:0/2421118119 shutdown_connections 2023-07-18T03:21:48.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.070+0000 7fa00c1ff700 1 -- 172.21.15.101:0/2421118119 wait complete. 2023-07-18T03:21:48.081 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 38 2023-07-18T03:21:48.110 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-18T03:21:48.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:48 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2421118119' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:21:48.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.511+0000 7fa0cdfff700 1 -- 172.21.15.101:0/3378761065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0c80fe9d0 msgr2=0x7fa0c80fee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:48.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.511+0000 7fa0cdfff700 1 --2- 172.21.15.101:0/3378761065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0c80fe9d0 0x7fa0c80fee40 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8009990 tx=0x7fa0b8009ca0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.514+0000 7fa0cdfff700 1 -- 172.21.15.101:0/3378761065 shutdown_connections 2023-07-18T03:21:48.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.514+0000 7fa0cdfff700 1 --2- 172.21.15.101:0/3378761065 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0c80ff380 0x7fa0c810b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.514+0000 7fa0cdfff700 1 --2- 172.21.15.101:0/3378761065 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa0c81049e0 0x7fa0c8104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.514+0000 7fa0cdfff700 1 --2- 172.21.15.101:0/3378761065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0c80fe9d0 0x7fa0c80fee40 secure :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8009990 tx=0x7fa0b8009ca0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.514+0000 7fa0cdfff700 1 -- 172.21.15.101:0/3378761065 >> 172.21.15.101:0/3378761065 conn(0x7fa0c80fa4b0 msgr2=0x7fa0c80fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:48.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.517+0000 7fa0cdfff700 1 -- 172.21.15.101:0/3378761065 shutdown_connections 2023-07-18T03:21:48.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.519+0000 7fa0cdfff700 1 -- 172.21.15.101:0/3378761065 wait complete. 2023-07-18T03:21:48.516 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.521+0000 7fa0cdfff700 1 Processor -- start 2023-07-18T03:21:48.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.521+0000 7fa0cdfff700 1 -- start start 2023-07-18T03:21:48.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.522+0000 7fa0cdfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0c80ff380 0x7fa0c819bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:48.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.522+0000 7fa0cdfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0c81049e0 0x7fa0c819c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:48.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.522+0000 7fa0cdfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa0c81940a0 0x7fa0c8194510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:48.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.522+0000 7fa0cdfff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c8194ce0 con 0x7fa0c81940a0 2023-07-18T03:21:48.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.522+0000 7fa0cdfff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c8194e50 con 0x7fa0c80ff380 2023-07-18T03:21:48.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.522+0000 7fa0cdfff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c81b18e0 con 0x7fa0c81049e0 2023-07-18T03:21:48.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.522+0000 7fa0c6ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0c81049e0 0x7fa0c819c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:48.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.523+0000 7fa0c6ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0c81049e0 0x7fa0c819c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:56986/0 (socket says 172.21.15.101:56986) 2023-07-18T03:21:48.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.523+0000 7fa0c6ffd700 1 -- 172.21.15.101:0/3876806643 learned_addr learned my addr 172.21.15.101:0/3876806643 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:48.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.523+0000 7fa0c6ffd700 1 -- 172.21.15.101:0/3876806643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0c80ff380 msgr2=0x7fa0c819bca0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-18T03:21:48.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.523+0000 7fa0c6ffd700 1 --2- 172.21.15.101:0/3876806643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0c80ff380 0x7fa0c819bca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.523+0000 7fa0c6ffd700 1 -- 172.21.15.101:0/3876806643 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa0c81940a0 msgr2=0x7fa0c8194510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:48.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.523+0000 7fa0c6ffd700 1 --2- 172.21.15.101:0/3876806643 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa0c81940a0 0x7fa0c8194510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.523+0000 7fa0c6ffd700 1 -- 172.21.15.101:0/3876806643 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b80095f0 con 0x7fa0c81049e0 2023-07-18T03:21:48.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.524+0000 7fa0c6ffd700 1 --2- 172.21.15.101:0/3876806643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0c81049e0 0x7fa0c819c1e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa0b800b680 tx=0x7fa0b8005b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:48.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.532+0000 7fa0c4ff9700 1 -- 172.21.15.101:0/3876806643 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b801d070 con 0x7fa0c81049e0 2023-07-18T03:21:48.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.532+0000 7fa0c4ff9700 1 -- 172.21.15.101:0/3876806643 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0b8003dd0 con 0x7fa0c81049e0 2023-07-18T03:21:48.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.532+0000 7fa0c4ff9700 1 -- 172.21.15.101:0/3876806643 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8021410 con 0x7fa0c81049e0 2023-07-18T03:21:48.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.533+0000 7fa0cdfff700 1 -- 172.21.15.101:0/3876806643 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c81b1a80 con 0x7fa0c81049e0 2023-07-18T03:21:48.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.533+0000 7fa0cdfff700 1 -- 172.21.15.101:0/3876806643 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c81b1e50 con 0x7fa0c81049e0 2023-07-18T03:21:48.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.539+0000 7fa0e1831b80 1 Processor -- start 2023-07-18T03:21:48.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.540+0000 7fa0e1831b80 1 -- start start 2023-07-18T03:21:48.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.542+0000 7fa0e1831b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5626744002d0 0x5626744006a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:48.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.542+0000 7fa0e1831b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562674400be0 0x56267440b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:48.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.542+0000 7fa0e1831b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56267440bad0 0x56267440deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:48.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.542+0000 7fa0e1831b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562674306cb0 con 0x562674400be0 2023-07-18T03:21:48.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.542+0000 7fa0e1831b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562674359a40 con 0x56267440bad0 2023-07-18T03:21:48.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.542+0000 7fa0e1831b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562674352ed0 con 0x5626744002d0 2023-07-18T03:21:48.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.544+0000 7fa0c7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5626744002d0 0x5626744006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:48.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.544+0000 7fa0c7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5626744002d0 0x5626744006a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:56996/0 (socket says 172.21.15.101:56996) 2023-07-18T03:21:48.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.544+0000 7fa0c7fff700 1 -- 172.21.15.101:0/4165027304 learned_addr learned my addr 172.21.15.101:0/4165027304 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:48.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.544+0000 7fa0c77fe700 1 --2- 172.21.15.101:0/4165027304 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562674400be0 0x56267440b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:48.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.544+0000 7fa0c4ff9700 1 -- 172.21.15.101:0/3876806643 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fa0b800f640 con 0x7fa0c81049e0 2023-07-18T03:21:48.544 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.545+0000 7fa0c77fe700 1 -- 172.21.15.101:0/4165027304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5626744002d0 msgr2=0x5626744006a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:48.544 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.545+0000 7fa0c77fe700 1 --2- 172.21.15.101:0/4165027304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5626744002d0 0x5626744006a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.544 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.545+0000 7fa0c77fe700 1 -- 172.21.15.101:0/4165027304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56267440bad0 msgr2=0x56267440deb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-18T03:21:48.544 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.545+0000 7fa0c77fe700 1 --2- 172.21.15.101:0/4165027304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56267440bad0 0x56267440deb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.544 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.545+0000 7fa0c77fe700 1 -- 172.21.15.101:0/4165027304 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562674180de0 con 0x562674400be0 2023-07-18T03:21:48.544 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.546+0000 7fa0c77fe700 1 --2- 172.21.15.101:0/4165027304 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562674400be0 0x56267440b590 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc009740 tx=0x7fa0bc009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=550815c6b77c7e58 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:48.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.546+0000 7fa0adffb700 1 -- 172.21.15.101:0/4165027304 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc011830 con 0x562674400be0 2023-07-18T03:21:48.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.546+0000 7fa0adffb700 1 -- 172.21.15.101:0/4165027304 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0bc011e50 con 0x562674400be0 2023-07-18T03:21:48.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.546+0000 7fa0adffb700 1 -- 172.21.15.101:0/4165027304 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc00f2e0 con 0x562674400be0 2023-07-18T03:21:48.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.547+0000 7fa0c4ff9700 1 --2- 172.21.15.101:0/3876806643 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa0b00a1450 0x7fa0b00a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:48.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.547+0000 7fa0c4ff9700 1 -- 172.21.15.101:0/3876806643 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fa0b80cde00 con 0x7fa0c81049e0 2023-07-18T03:21:48.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.547+0000 7fa0e1831b80 1 -- 172.21.15.101:0/4165027304 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562674400be0 msgr2=0x56267440b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:48.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.547+0000 7fa0e1831b80 1 --2- 172.21.15.101:0/4165027304 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562674400be0 0x56267440b590 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc009740 tx=0x7fa0bc009a50 comp rx=0 tx=0).stop 2023-07-18T03:21:48.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.548+0000 7fa0c7fff700 1 --2- 172.21.15.101:0/4165027304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5626744002d0 0x5626744006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:21:48.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.549+0000 7fa0e1831b80 1 -- 172.21.15.101:0/4165027304 shutdown_connections 2023-07-18T03:21:48.547 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.549+0000 7fa0e1831b80 1 --2- 172.21.15.101:0/4165027304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56267440bad0 0x56267440deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.547 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.549+0000 7fa0e1831b80 1 --2- 172.21.15.101:0/4165027304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5626744002d0 0x5626744006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.547 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.549+0000 7fa0e1831b80 1 --2- 172.21.15.101:0/4165027304 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562674400be0 0x56267440b590 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.547 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.549+0000 7fa0e1831b80 1 -- 172.21.15.101:0/4165027304 >> 172.21.15.101:0/4165027304 conn(0x5626743f6da0 msgr2=0x5626743fc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:48.547 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.551+0000 7fa0c7fff700 1 --2- 172.21.15.101:0/3876806643 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa0b00a1450 0x7fa0b00a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:48.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.552+0000 7fa0e1831b80 1 -- 172.21.15.101:0/4165027304 shutdown_connections 2023-07-18T03:21:48.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.552+0000 7fa0e1831b80 1 -- 172.21.15.101:0/4165027304 wait complete. 2023-07-18T03:21:48.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:48 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/2421118119' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:21:48.549 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.553+0000 7fa0e1831b80 1 -- start start 2023-07-18T03:21:48.549 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.554+0000 7fa0c7fff700 1 --2- 172.21.15.101:0/3876806643 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa0b00a1450 0x7fa0b00a3900 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0009c10 tx=0x7fa0c000b040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:48.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.558+0000 7fa0e1831b80 1 Processor -- start 2023-07-18T03:21:48.564 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.566+0000 7fa0e1831b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5626744002d0 0x56267440b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:48.564 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.566+0000 7fa0e1831b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562674400be0 0x562674408790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:48.564 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.566+0000 7fa0e1831b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56267440bad0 0x562674408ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:48.564 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.566+0000 7fa0e1831b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626742f2ac0 con 0x56267440bad0 2023-07-18T03:21:48.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.566+0000 7fa0e1831b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626742f27c0 con 0x562674400be0 2023-07-18T03:21:48.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.567+0000 7fa0e1831b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626741d6920 con 0x5626744002d0 2023-07-18T03:21:48.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.567+0000 7fa0e1831b80 10 client.0 _ll_register_callbacks cb 0x5626742f0760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:21:48.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.568+0000 7fa0c7fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56267440bad0 0x562674408ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:48.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.568+0000 7fa0c7fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56267440bad0 0x562674408ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:41320/0 (socket says 172.21.15.101:41320) 2023-07-18T03:21:48.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.568+0000 7fa0c7fff700 1 -- 172.21.15.101:0/1933477140 learned_addr learned my addr 172.21.15.101:0/1933477140 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:48.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.568+0000 7fa0c6ffd700 1 --2- 172.21.15.101:0/1933477140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562674400be0 0x562674408790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:48.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.569+0000 7fa0c7fff700 1 -- 172.21.15.101:0/1933477140 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5626744002d0 msgr2=0x56267440b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:48.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.569+0000 7fa0c7fff700 1 --2- 172.21.15.101:0/1933477140 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5626744002d0 0x56267440b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.569+0000 7fa0c7fff700 1 -- 172.21.15.101:0/1933477140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562674400be0 msgr2=0x562674408790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:48.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.569+0000 7fa0c7fff700 1 --2- 172.21.15.101:0/1933477140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562674400be0 0x562674408790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.569+0000 7fa0c7fff700 1 -- 172.21.15.101:0/1933477140 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0bc0093a0 con 0x56267440bad0 2023-07-18T03:21:48.567 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.569+0000 7fa0c7fff700 1 --2- 172.21.15.101:0/1933477140 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56267440bad0 0x562674408ed0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0009550 tx=0x7fa0c0009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cbc673877498806 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:48.567 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.569+0000 7fa0a6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:21:48.567 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.570+0000 7fa0a6ffd700 1 -- 172.21.15.101:0/1933477140 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0007ee0 con 0x56267440bad0 2023-07-18T03:21:48.567 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.570+0000 7fa0a6ffd700 1 -- 172.21.15.101:0/1933477140 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fa0c0038890 con 0x56267440bad0 2023-07-18T03:21:48.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.570+0000 7fa0e1831b80 1 -- 172.21.15.101:0/1933477140 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5626742ac4f0 con 0x56267440bad0 2023-07-18T03:21:48.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.570+0000 7fa0a6ffd700 1 -- 172.21.15.101:0/1933477140 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0c0038a50 con 0x56267440bad0 2023-07-18T03:21:48.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.571+0000 7fa0e1831b80 10 client.18342 fetch_fsmap learned FSMap version 38 2023-07-18T03:21:48.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.571+0000 7fa0e1831b80 1 -- 172.21.15.101:0/1933477140 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x562674169c60 con 0x56267440bad0 2023-07-18T03:21:48.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.572+0000 7fa0a6ffd700 1 -- 172.21.15.101:0/1933477140 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5626742ac4f0 con 0x56267440bad0 2023-07-18T03:21:48.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.572+0000 7fa0a6ffd700 1 -- 172.21.15.101:0/1933477140 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 38) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fa0c004c3b0 con 0x56267440bad0 2023-07-18T03:21:48.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.572+0000 7fa0e1831b80 10 client.18342 fetch_fsmap finished waiting for FSMap version 38 2023-07-18T03:21:48.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.572+0000 7fa0e1831b80 10 client.18342 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:21:48.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.572+0000 7fa0e1831b80 20 client.18342 populate_metadata read hostname 'smithi101' 2023-07-18T03:21:48.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.572+0000 7fa0e1831b80 1 --2- 172.21.15.101:0/1933477140 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x56267442c540 0x56267442e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:48.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.572+0000 7fa0e1831b80 4 client.18342 mds_command: new command op to 24541 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:21:48.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.572+0000 7fa0e1831b80 1 -- 172.21.15.101:0/1933477140 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56267434beb0 con 0x56267442c540 2023-07-18T03:21:48.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.572+0000 7fa0c77fe700 1 --2- 172.21.15.101:0/1933477140 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x56267442c540 0x56267442e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:48.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.574+0000 7fa0c77fe700 1 --2- 172.21.15.101:0/1933477140 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x56267442c540 0x56267442e950 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f56c981303397c5f server_cookie=6dda633e81eb647e in_seq=0 out_seq=0 2023-07-18T03:21:48.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.574+0000 7fa0a6ffd700 10 client.18342 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:21:48.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.579+0000 7fa0a6ffd700 1 -- 172.21.15.101:0/1933477140 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56267434beb0 con 0x56267442c540 2023-07-18T03:21:48.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.579+0000 7fa0a6ffd700 10 client.18342 handle_command_reply: tid=0 2023-07-18T03:21:48.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.579+0000 7fa0e1831b80 1 client.18342 shutdown 2023-07-18T03:21:48.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.580+0000 7fa0e1831b80 1 -- 172.21.15.101:0/1933477140 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56267440bad0 msgr2=0x562674408ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:48.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.580+0000 7fa0e1831b80 1 --2- 172.21.15.101:0/1933477140 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56267440bad0 0x562674408ed0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0009550 tx=0x7fa0c0009920 comp rx=0 tx=0).stop 2023-07-18T03:21:48.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.581+0000 7fa0e1831b80 1 -- 172.21.15.101:0/1933477140 shutdown_connections 2023-07-18T03:21:48.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.581+0000 7fa0e1831b80 1 --2- 172.21.15.101:0/1933477140 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x56267442c540 0x56267442e950 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.581+0000 7fa0e1831b80 1 --2- 172.21.15.101:0/1933477140 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56267440bad0 0x562674408ed0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.581+0000 7fa0e1831b80 1 --2- 172.21.15.101:0/1933477140 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5626744002d0 0x56267440b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.581+0000 7fa0e1831b80 1 --2- 172.21.15.101:0/1933477140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562674400be0 0x562674408790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.581+0000 7fa0e1831b80 1 -- 172.21.15.101:0/1933477140 >> 172.21.15.101:0/1933477140 conn(0x5626743f6da0 msgr2=0x5626743fc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:48.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.581+0000 7fa0a6ffd700 0 client.18342 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:21:48.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.581+0000 7fa0e1831b80 1 -- 172.21.15.101:0/1933477140 shutdown_connections 2023-07-18T03:21:48.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.582+0000 7fa0e1831b80 1 -- 172.21.15.101:0/1933477140 wait complete. 2023-07-18T03:21:48.582 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.587+0000 7fa0e1831b80 20 client.18342 trim_cache size 0 max 16384 2023-07-18T03:21:48.604 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.609+0000 7fa0e1831b80 1 Processor -- start 2023-07-18T03:21:48.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.609+0000 7fa0e1831b80 1 -- start start 2023-07-18T03:21:48.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.610+0000 7fa0e1831b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5626744002d0 0x5626743fae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:48.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.610+0000 7fa0e1831b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562674400be0 0x56267440b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:48.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.610+0000 7fa0e1831b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56267440bad0 0x562674407d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:48.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.610+0000 7fa0e1831b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562674429b70 con 0x562674400be0 2023-07-18T03:21:48.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.610+0000 7fa0e1831b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626743fb350 con 0x56267440bad0 2023-07-18T03:21:48.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.610+0000 7fa0e1831b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626744089a0 con 0x5626744002d0 2023-07-18T03:21:48.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.610+0000 7fa0c77fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5626744002d0 0x5626743fae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:48.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.610+0000 7fa0c77fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5626744002d0 0x5626743fae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:57020/0 (socket says 172.21.15.101:57020) 2023-07-18T03:21:48.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.610+0000 7fa0c77fe700 1 -- 172.21.15.101:0/31724404 learned_addr learned my addr 172.21.15.101:0/31724404 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:48.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.610+0000 7fa0c7fff700 1 --2- 172.21.15.101:0/31724404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56267440bad0 0x562674407d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:48.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.610+0000 7fa0c6ffd700 1 --2- 172.21.15.101:0/31724404 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562674400be0 0x56267440b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:48.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.610+0000 7fa0c77fe700 1 -- 172.21.15.101:0/31724404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56267440bad0 msgr2=0x562674407d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:48.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.610+0000 7fa0c77fe700 1 --2- 172.21.15.101:0/31724404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56267440bad0 0x562674407d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.610+0000 7fa0c77fe700 1 -- 172.21.15.101:0/31724404 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562674400be0 msgr2=0x56267440b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:48.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.610+0000 7fa0c77fe700 1 --2- 172.21.15.101:0/31724404 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562674400be0 0x56267440b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.611+0000 7fa0c77fe700 1 -- 172.21.15.101:0/31724404 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626741a7250 con 0x5626744002d0 2023-07-18T03:21:48.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.611+0000 7fa0c77fe700 1 --2- 172.21.15.101:0/31724404 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5626744002d0 0x5626743fae10 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc009740 tx=0x7fa0bc010bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aaab95f82359fdf1 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:48.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.611+0000 7fa0ae7fc700 1 -- 172.21.15.101:0/31724404 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc024380 con 0x5626744002d0 2023-07-18T03:21:48.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.611+0000 7fa0ae7fc700 1 -- 172.21.15.101:0/31724404 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0bc021070 con 0x5626744002d0 2023-07-18T03:21:48.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.611+0000 7fa0ae7fc700 1 -- 172.21.15.101:0/31724404 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc0106d0 con 0x5626744002d0 2023-07-18T03:21:48.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.612+0000 7fa0e1831b80 1 -- 172.21.15.101:0/31724404 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5626744002d0 msgr2=0x5626743fae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:48.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.612+0000 7fa0e1831b80 1 --2- 172.21.15.101:0/31724404 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5626744002d0 0x5626743fae10 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc009740 tx=0x7fa0bc010bd0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.612+0000 7fa0e1831b80 1 -- 172.21.15.101:0/31724404 shutdown_connections 2023-07-18T03:21:48.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.612+0000 7fa0e1831b80 1 --2- 172.21.15.101:0/31724404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56267440bad0 0x562674407d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.612+0000 7fa0e1831b80 1 --2- 172.21.15.101:0/31724404 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5626744002d0 0x5626743fae10 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.612+0000 7fa0e1831b80 1 --2- 172.21.15.101:0/31724404 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562674400be0 0x56267440b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.612+0000 7fa0e1831b80 1 -- 172.21.15.101:0/31724404 >> 172.21.15.101:0/31724404 conn(0x5626743f6da0 msgr2=0x5626744332b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:48.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.613+0000 7fa0e1831b80 1 -- 172.21.15.101:0/31724404 shutdown_connections 2023-07-18T03:21:48.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.613+0000 7fa0e1831b80 1 -- 172.21.15.101:0/31724404 wait complete. 2023-07-18T03:21:48.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.613+0000 7fa0e1831b80 1 -- start start 2023-07-18T03:21:48.616 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.613+0000 7fa0e1831b80 1 Processor -- start 2023-07-18T03:21:48.616 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.614+0000 7fa0e1831b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626744002d0 0x5626744121b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:48.616 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.614+0000 7fa0e1831b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562674400be0 0x5626744126f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:48.616 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.614+0000 7fa0e1831b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56267440bad0 0x5626744106d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:48.616 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.614+0000 7fa0e1831b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562674410da0 con 0x562674400be0 2023-07-18T03:21:48.616 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.614+0000 7fa0e1831b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562674410f10 con 0x5626744002d0 2023-07-18T03:21:48.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.614+0000 7fa0e1831b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626744110d0 con 0x56267440bad0 2023-07-18T03:21:48.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.614+0000 7fa0e1831b80 10 client.0 _ll_register_callbacks cb 0x7fa0c00377f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:21:48.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.614+0000 7fa0c7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56267440bad0 0x5626744106d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:48.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.614+0000 7fa0c6ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562674400be0 0x5626744126f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:48.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.614+0000 7fa0c7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56267440bad0 0x5626744106d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:57030/0 (socket says 172.21.15.101:57030) 2023-07-18T03:21:48.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.614+0000 7fa0c7fff700 1 -- 172.21.15.101:0/1610516524 learned_addr learned my addr 172.21.15.101:0/1610516524 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:48.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.614+0000 7fa0c77fe700 1 --2- 172.21.15.101:0/1610516524 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626744002d0 0x5626744121b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:48.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.615+0000 7fa0c7fff700 1 -- 172.21.15.101:0/1610516524 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626744002d0 msgr2=0x5626744121b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:48.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.615+0000 7fa0c7fff700 1 --2- 172.21.15.101:0/1610516524 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626744002d0 0x5626744121b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.615+0000 7fa0c7fff700 1 -- 172.21.15.101:0/1610516524 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562674400be0 msgr2=0x5626744126f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:48.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.615+0000 7fa0c7fff700 1 --2- 172.21.15.101:0/1610516524 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562674400be0 0x5626744126f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.615+0000 7fa0c7fff700 1 -- 172.21.15.101:0/1610516524 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562674180de0 con 0x56267440bad0 2023-07-18T03:21:48.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.615+0000 7fa0c6ffd700 1 --2- 172.21.15.101:0/1610516524 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562674400be0 0x5626744126f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:21:48.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.615+0000 7fa0c7fff700 1 --2- 172.21.15.101:0/1610516524 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56267440bad0 0x5626744106d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa0c000b010 tx=0x7fa0c0007fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6052b6de2bdc3f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:48.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.615+0000 7fa0a6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:21:48.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.616+0000 7fa0a6ffd700 1 -- 172.21.15.101:0/1610516524 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0004bb0 con 0x56267440bad0 2023-07-18T03:21:48.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.616+0000 7fa0a6ffd700 1 -- 172.21.15.101:0/1610516524 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fa0c0004d10 con 0x56267440bad0 2023-07-18T03:21:48.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.616+0000 7fa0e1831b80 1 -- 172.21.15.101:0/1610516524 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa0bc011e50 con 0x56267440bad0 2023-07-18T03:21:48.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.616+0000 7fa0a6ffd700 1 -- 172.21.15.101:0/1610516524 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0c0038790 con 0x56267440bad0 2023-07-18T03:21:48.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.616+0000 7fa0e1831b80 10 client.28435 fetch_fsmap learned FSMap version 38 2023-07-18T03:21:48.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.616+0000 7fa0e1831b80 1 -- 172.21.15.101:0/1610516524 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x56267417f2b0 con 0x56267440bad0 2023-07-18T03:21:48.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.616+0000 7fa0a6ffd700 1 -- 172.21.15.101:0/1610516524 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0bc011e50 con 0x56267440bad0 2023-07-18T03:21:48.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.617+0000 7fa0a6ffd700 1 -- 172.21.15.101:0/1610516524 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 38) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fa0c0043880 con 0x56267440bad0 2023-07-18T03:21:48.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.617+0000 7fa0e1831b80 10 client.28435 fetch_fsmap finished waiting for FSMap version 38 2023-07-18T03:21:48.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.617+0000 7fa0e1831b80 10 client.28435 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:21:48.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.617+0000 7fa0e1831b80 20 client.28435 populate_metadata read hostname 'smithi101' 2023-07-18T03:21:48.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.617+0000 7fa0e1831b80 1 --2- 172.21.15.101:0/1610516524 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5626744816b0 0x562674483ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:48.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.617+0000 7fa0e1831b80 4 client.28435 mds_command: new command op to 24541 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:21:48.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.617+0000 7fa0e1831b80 1 -- 172.21.15.101:0/1610516524 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56267430e050 con 0x5626744816b0 2023-07-18T03:21:48.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.617+0000 7fa0c77fe700 1 --2- 172.21.15.101:0/1610516524 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5626744816b0 0x562674483ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:48.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.618+0000 7fa0a6ffd700 10 client.28435 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:21:48.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.618+0000 7fa0c77fe700 1 --2- 172.21.15.101:0/1610516524 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5626744816b0 0x562674483ac0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6d0c716de3a5cbe6 server_cookie=a3d18dcf9626e1b5 in_seq=0 out_seq=0 2023-07-18T03:21:48.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.623+0000 7fa0a6ffd700 1 -- 172.21.15.101:0/1610516524 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56267430e050 con 0x5626744816b0 2023-07-18T03:21:48.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.623+0000 7fa0a6ffd700 10 client.28435 handle_command_reply: tid=0 2023-07-18T03:21:48.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.623+0000 7fa0e1831b80 1 client.28435 shutdown 2023-07-18T03:21:48.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.624+0000 7fa0e1831b80 1 -- 172.21.15.101:0/1610516524 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56267440bad0 msgr2=0x5626744106d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:48.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.624+0000 7fa0e1831b80 1 --2- 172.21.15.101:0/1610516524 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56267440bad0 0x5626744106d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa0c000b010 tx=0x7fa0c0007fb0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.624+0000 7fa0e1831b80 1 -- 172.21.15.101:0/1610516524 shutdown_connections 2023-07-18T03:21:48.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.624+0000 7fa0e1831b80 1 --2- 172.21.15.101:0/1610516524 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5626744816b0 0x562674483ac0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.624+0000 7fa0e1831b80 1 --2- 172.21.15.101:0/1610516524 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626744002d0 0x5626744121b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.624+0000 7fa0e1831b80 1 --2- 172.21.15.101:0/1610516524 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56267440bad0 0x5626744106d0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.624+0000 7fa0e1831b80 1 --2- 172.21.15.101:0/1610516524 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562674400be0 0x5626744126f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.624+0000 7fa0e1831b80 1 -- 172.21.15.101:0/1610516524 >> 172.21.15.101:0/1610516524 conn(0x5626743f6da0 msgr2=0x562674403a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:48.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.624+0000 7fa0a6ffd700 0 client.28435 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:21:48.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.624+0000 7fa0e1831b80 1 -- 172.21.15.101:0/1610516524 shutdown_connections 2023-07-18T03:21:48.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.624+0000 7fa0e1831b80 1 -- 172.21.15.101:0/1610516524 wait complete. 2023-07-18T03:21:48.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.624+0000 7fa0e1831b80 20 client.28435 trim_cache size 0 max 16384 2023-07-18T03:21:48.625 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:21:48.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.625+0000 7fa0cdfff700 1 -- 172.21.15.101:0/3876806643 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa0b00a1450 msgr2=0x7fa0b00a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:48.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.625+0000 7fa0cdfff700 1 --2- 172.21.15.101:0/3876806643 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa0b00a1450 0x7fa0b00a3900 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0009c10 tx=0x7fa0c000b040 comp rx=0 tx=0).stop 2023-07-18T03:21:48.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.625+0000 7fa0cdfff700 1 -- 172.21.15.101:0/3876806643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0c81049e0 msgr2=0x7fa0c819c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:48.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.625+0000 7fa0cdfff700 1 --2- 172.21.15.101:0/3876806643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0c81049e0 0x7fa0c819c1e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa0b800b680 tx=0x7fa0b8005b00 comp rx=0 tx=0).stop 2023-07-18T03:21:48.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.625+0000 7fa0cdfff700 1 -- 172.21.15.101:0/3876806643 shutdown_connections 2023-07-18T03:21:48.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.625+0000 7fa0cdfff700 1 --2- 172.21.15.101:0/3876806643 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa0b00a1450 0x7fa0b00a3900 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.625+0000 7fa0cdfff700 1 --2- 172.21.15.101:0/3876806643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0c80ff380 0x7fa0c819bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.625+0000 7fa0cdfff700 1 --2- 172.21.15.101:0/3876806643 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa0c81940a0 0x7fa0c8194510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.625+0000 7fa0cdfff700 1 --2- 172.21.15.101:0/3876806643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0c81049e0 0x7fa0c819c1e0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.625+0000 7fa0cdfff700 1 -- 172.21.15.101:0/3876806643 >> 172.21.15.101:0/3876806643 conn(0x7fa0c80fa4b0 msgr2=0x7fa0c80fbf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:48.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.626+0000 7fa0cdfff700 1 -- 172.21.15.101:0/3876806643 shutdown_connections 2023-07-18T03:21:48.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.626+0000 7fa0cdfff700 1 -- 172.21.15.101:0/3876806643 wait complete. 2023-07-18T03:21:48.643 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-18T03:21:48.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:48 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/2421118119' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:21:48.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.889+0000 7f0f3e5ff700 1 -- 172.21.15.101:0/1738366298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f380fe9d0 msgr2=0x7f0f380fee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:48.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.889+0000 7f0f3e5ff700 1 --2- 172.21.15.101:0/1738366298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f380fe9d0 0x7f0f380fee40 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c009990 tx=0x7f0f2c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.890+0000 7f0f3e5ff700 1 -- 172.21.15.101:0/1738366298 shutdown_connections 2023-07-18T03:21:48.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.890+0000 7f0f3e5ff700 1 --2- 172.21.15.101:0/1738366298 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0f380ff380 0x7f0f3810b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.890+0000 7f0f3e5ff700 1 --2- 172.21.15.101:0/1738366298 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f381049e0 0x7f0f38104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.890+0000 7f0f3e5ff700 1 --2- 172.21.15.101:0/1738366298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f380fe9d0 0x7f0f380fee40 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.890+0000 7f0f3e5ff700 1 -- 172.21.15.101:0/1738366298 >> 172.21.15.101:0/1738366298 conn(0x7f0f380fa4b0 msgr2=0x7f0f380fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:48.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.890+0000 7f0f3e5ff700 1 -- 172.21.15.101:0/1738366298 shutdown_connections 2023-07-18T03:21:48.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.890+0000 7f0f3e5ff700 1 -- 172.21.15.101:0/1738366298 wait complete. 2023-07-18T03:21:48.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.891+0000 7f0f3e5ff700 1 Processor -- start 2023-07-18T03:21:48.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.891+0000 7f0f3e5ff700 1 -- start start 2023-07-18T03:21:48.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.892+0000 7f0f3e5ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f380fe9d0 0x7f0f381948a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:48.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.892+0000 7f0f3e5ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f380ff380 0x7f0f38194de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:48.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.892+0000 7f0f3e5ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0f381049e0 0x7f0f38195320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:48.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.892+0000 7f0f3e5ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f381a4130 con 0x7f0f381049e0 2023-07-18T03:21:48.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.892+0000 7f0f3e5ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f381a42a0 con 0x7f0f380ff380 2023-07-18T03:21:48.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.892+0000 7f0f3e5ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f381a4460 con 0x7f0f380fe9d0 2023-07-18T03:21:48.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.892+0000 7f0f377fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f380ff380 0x7f0f38194de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:48.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.892+0000 7f0f37fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f380fe9d0 0x7f0f381948a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:48.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.892+0000 7f0f377fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f380ff380 0x7f0f38194de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47416/0 (socket says 172.21.15.101:47416) 2023-07-18T03:21:48.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.892+0000 7f0f37fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f380fe9d0 0x7f0f381948a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:57052/0 (socket says 172.21.15.101:57052) 2023-07-18T03:21:48.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.892+0000 7f0f377fe700 1 -- 172.21.15.101:0/867669069 learned_addr learned my addr 172.21.15.101:0/867669069 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:48.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.892+0000 7f0f3cb9c700 1 --2- 172.21.15.101:0/867669069 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0f381049e0 0x7f0f38195320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:48.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.893+0000 7f0f377fe700 1 -- 172.21.15.101:0/867669069 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f380fe9d0 msgr2=0x7f0f381948a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:48.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.893+0000 7f0f377fe700 1 --2- 172.21.15.101:0/867669069 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f380fe9d0 0x7f0f381948a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.893+0000 7f0f377fe700 1 -- 172.21.15.101:0/867669069 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0f381049e0 msgr2=0x7f0f38195320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:48.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.893+0000 7f0f377fe700 1 --2- 172.21.15.101:0/867669069 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0f381049e0 0x7f0f38195320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.893+0000 7f0f377fe700 1 -- 172.21.15.101:0/867669069 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f2c0095f0 con 0x7f0f380ff380 2023-07-18T03:21:48.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.893+0000 7f0f377fe700 1 --2- 172.21.15.101:0/867669069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f380ff380 0x7f0f38194de0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c005910 tx=0x7f0f2c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:48.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.893+0000 7f0f357fa700 1 -- 172.21.15.101:0/867669069 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c01d070 con 0x7f0f380ff380 2023-07-18T03:21:48.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.893+0000 7f0f357fa700 1 -- 172.21.15.101:0/867669069 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f2c022470 con 0x7f0f380ff380 2023-07-18T03:21:48.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.893+0000 7f0f357fa700 1 -- 172.21.15.101:0/867669069 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c017650 con 0x7f0f380ff380 2023-07-18T03:21:48.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.893+0000 7f0f3e5ff700 1 -- 172.21.15.101:0/867669069 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f381a46e0 con 0x7f0f380ff380 2023-07-18T03:21:48.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.893+0000 7f0f3e5ff700 1 -- 172.21.15.101:0/867669069 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f381a4ab0 con 0x7f0f380ff380 2023-07-18T03:21:48.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.897+0000 7f0f357fa700 1 -- 172.21.15.101:0/867669069 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f0f2c00fad0 con 0x7f0f380ff380 2023-07-18T03:21:48.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.898+0000 7f0f51f4cb80 1 Processor -- start 2023-07-18T03:21:48.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.898+0000 7f0f51f4cb80 1 -- start start 2023-07-18T03:21:48.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.898+0000 7f0f51f4cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563c8fca42d0 0x563c8fca46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:48.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.898+0000 7f0f51f4cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c8fca4be0 0x563c8fcaf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:48.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.898+0000 7f0f51f4cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c8fcafad0 0x563c8fcb1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:48.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.898+0000 7f0f51f4cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c8fbaacb0 con 0x563c8fca42d0 2023-07-18T03:21:48.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.898+0000 7f0f51f4cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c8fbfda40 con 0x563c8fcafad0 2023-07-18T03:21:48.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.898+0000 7f0f51f4cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c8fbf6ed0 con 0x563c8fca4be0 2023-07-18T03:21:48.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.898+0000 7f0f357fa700 1 --2- 172.21.15.101:0/867669069 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0f200a1340 0x7f0f200a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:48.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.898+0000 7f0f37fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c8fca4be0 0x563c8fcaf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:48.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.899+0000 7f0f37fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c8fca4be0 0x563c8fcaf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:57058/0 (socket says 172.21.15.101:57058) 2023-07-18T03:21:48.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.899+0000 7f0f37fff700 1 -- 172.21.15.101:0/1583945761 learned_addr learned my addr 172.21.15.101:0/1583945761 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:48.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.899+0000 7f0f357fa700 1 -- 172.21.15.101:0/867669069 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f0f2c0ce510 con 0x7f0f380ff380 2023-07-18T03:21:48.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.899+0000 7f0f3cb9c700 1 --2- 172.21.15.101:0/867669069 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0f200a1340 0x7f0f200a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:48.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.899+0000 7f0f3cb9c700 1 --2- 172.21.15.101:0/1583945761 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563c8fca42d0 0x563c8fca46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:48.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.899+0000 7f0f37fff700 1 -- 172.21.15.101:0/1583945761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c8fcafad0 msgr2=0x563c8fcb1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:48.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.899+0000 7f0f37fff700 1 --2- 172.21.15.101:0/1583945761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c8fcafad0 0x563c8fcb1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.899+0000 7f0f37fff700 1 -- 172.21.15.101:0/1583945761 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563c8fca42d0 msgr2=0x563c8fca46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:48.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.899+0000 7f0f37fff700 1 --2- 172.21.15.101:0/1583945761 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563c8fca42d0 0x563c8fca46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.899+0000 7f0f37fff700 1 -- 172.21.15.101:0/1583945761 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c8fa24de0 con 0x563c8fca4be0 2023-07-18T03:21:48.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.899+0000 7f0f3cb9c700 1 --2- 172.21.15.101:0/1583945761 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563c8fca42d0 0x563c8fca46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:21:48.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.899+0000 7f0f37fff700 1 --2- 172.21.15.101:0/1583945761 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c8fca4be0 0x563c8fcaf590 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0f28009ac0 tx=0x7f0f28009dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab52f7cefacd4bc3 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:48.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.899+0000 7f0f3cb9c700 1 --2- 172.21.15.101:0/867669069 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0f200a1340 0x7f0f200a37f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0f30009550 tx=0x7f0f30009920 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:48.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.899+0000 7f0f1e7fc700 1 -- 172.21.15.101:0/1583945761 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f28009590 con 0x563c8fca4be0 2023-07-18T03:21:48.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.900+0000 7f0f1e7fc700 1 -- 172.21.15.101:0/1583945761 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f2800edf0 con 0x563c8fca4be0 2023-07-18T03:21:48.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.900+0000 7f0f1e7fc700 1 -- 172.21.15.101:0/1583945761 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f280123d0 con 0x563c8fca4be0 2023-07-18T03:21:48.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.900+0000 7f0f51f4cb80 1 -- 172.21.15.101:0/1583945761 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c8fca4be0 msgr2=0x563c8fcaf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:48.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.900+0000 7f0f51f4cb80 1 --2- 172.21.15.101:0/1583945761 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c8fca4be0 0x563c8fcaf590 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0f28009ac0 tx=0x7f0f28009dd0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.900+0000 7f0f51f4cb80 1 -- 172.21.15.101:0/1583945761 shutdown_connections 2023-07-18T03:21:48.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.900+0000 7f0f51f4cb80 1 --2- 172.21.15.101:0/1583945761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c8fcafad0 0x563c8fcb1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.900+0000 7f0f51f4cb80 1 --2- 172.21.15.101:0/1583945761 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563c8fca42d0 0x563c8fca46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.900+0000 7f0f51f4cb80 1 --2- 172.21.15.101:0/1583945761 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c8fca4be0 0x563c8fcaf590 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.900+0000 7f0f51f4cb80 1 -- 172.21.15.101:0/1583945761 >> 172.21.15.101:0/1583945761 conn(0x563c8fc9ada0 msgr2=0x563c8fca06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:48.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.901+0000 7f0f51f4cb80 1 -- 172.21.15.101:0/1583945761 shutdown_connections 2023-07-18T03:21:48.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.901+0000 7f0f51f4cb80 1 -- 172.21.15.101:0/1583945761 wait complete. 2023-07-18T03:21:48.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.901+0000 7f0f51f4cb80 1 -- start start 2023-07-18T03:21:48.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.901+0000 7f0f51f4cb80 1 Processor -- start 2023-07-18T03:21:48.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.902+0000 7f0f51f4cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563c8fca42d0 0x563c8fcaf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:48.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.902+0000 7f0f51f4cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c8fca4be0 0x563c8fcac750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:48.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.902+0000 7f0f51f4cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c8fcafad0 0x563c8fcacdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:48.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.902+0000 7f0f51f4cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c8fb96ac0 con 0x563c8fca42d0 2023-07-18T03:21:48.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.902+0000 7f0f51f4cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c8fb967c0 con 0x563c8fcafad0 2023-07-18T03:21:48.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.902+0000 7f0f51f4cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c8fa7a920 con 0x563c8fca4be0 2023-07-18T03:21:48.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.902+0000 7f0f51f4cb80 10 client.0 _ll_register_callbacks cb 0x563c8fb94760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:21:48.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.902+0000 7f0f377fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c8fca4be0 0x563c8fcac750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:48.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.902+0000 7f0f377fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c8fca4be0 0x563c8fcac750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:57062/0 (socket says 172.21.15.101:57062) 2023-07-18T03:21:48.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.902+0000 7f0f377fe700 1 -- 172.21.15.101:0/1124871000 learned_addr learned my addr 172.21.15.101:0/1124871000 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:48.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.902+0000 7f0f3cb9c700 1 --2- 172.21.15.101:0/1124871000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c8fcafad0 0x563c8fcacdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:48.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.902+0000 7f0f37fff700 1 --2- 172.21.15.101:0/1124871000 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563c8fca42d0 0x563c8fcaf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:48.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.903+0000 7f0f377fe700 1 -- 172.21.15.101:0/1124871000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c8fcafad0 msgr2=0x563c8fcacdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:48.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.903+0000 7f0f377fe700 1 --2- 172.21.15.101:0/1124871000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c8fcafad0 0x563c8fcacdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.903+0000 7f0f377fe700 1 -- 172.21.15.101:0/1124871000 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563c8fca42d0 msgr2=0x563c8fcaf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:48.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.903+0000 7f0f377fe700 1 --2- 172.21.15.101:0/1124871000 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563c8fca42d0 0x563c8fcaf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.903+0000 7f0f377fe700 1 -- 172.21.15.101:0/1124871000 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f280096f0 con 0x563c8fca4be0 2023-07-18T03:21:48.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.903+0000 7f0f3cb9c700 1 --2- 172.21.15.101:0/1124871000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c8fcafad0 0x563c8fcacdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:21:48.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.903+0000 7f0f37fff700 1 --2- 172.21.15.101:0/1124871000 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563c8fca42d0 0x563c8fcaf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:21:48.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.903+0000 7f0f377fe700 1 --2- 172.21.15.101:0/1124871000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c8fca4be0 0x563c8fcac750 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c005f50 tx=0x7f0f2c0cbdb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=751262913fc2b2e4 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:48.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.903+0000 7f0f177fe700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:21:48.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.903+0000 7f0f177fe700 1 -- 172.21.15.101:0/1124871000 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c09abc0 con 0x563c8fca4be0 2023-07-18T03:21:48.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.904+0000 7f0f177fe700 1 -- 172.21.15.101:0/1124871000 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f0f2c09b260 con 0x563c8fca4be0 2023-07-18T03:21:48.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.904+0000 7f0f51f4cb80 1 -- 172.21.15.101:0/1124871000 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563c8fb504f0 con 0x563c8fca4be0 2023-07-18T03:21:48.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.904+0000 7f0f177fe700 1 -- 172.21.15.101:0/1124871000 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f2c09f290 con 0x563c8fca4be0 2023-07-18T03:21:48.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.904+0000 7f0f51f4cb80 10 client.28444 fetch_fsmap learned FSMap version 38 2023-07-18T03:21:48.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.904+0000 7f0f51f4cb80 1 -- 172.21.15.101:0/1124871000 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x563c8fa0dc60 con 0x563c8fca4be0 2023-07-18T03:21:48.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.904+0000 7f0f177fe700 1 -- 172.21.15.101:0/1124871000 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563c8fb504f0 con 0x563c8fca4be0 2023-07-18T03:21:48.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.904+0000 7f0f177fe700 1 -- 172.21.15.101:0/1124871000 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 38) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f0f2c09a750 con 0x563c8fca4be0 2023-07-18T03:21:48.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.904+0000 7f0f51f4cb80 10 client.28444 fetch_fsmap finished waiting for FSMap version 38 2023-07-18T03:21:48.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.904+0000 7f0f51f4cb80 10 client.28444 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:21:48.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.904+0000 7f0f51f4cb80 20 client.28444 populate_metadata read hostname 'smithi101' 2023-07-18T03:21:48.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.904+0000 7f0f51f4cb80 1 --2- 172.21.15.101:0/1124871000 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x563c8fcbeff0 0x563c8fcc1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:48.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.904+0000 7f0f51f4cb80 4 client.28444 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:21:48.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.904+0000 7f0f51f4cb80 1 -- 172.21.15.101:0/1124871000 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563c8fbefeb0 con 0x563c8fcbeff0 2023-07-18T03:21:48.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.905+0000 7f0f37fff700 1 --2- 172.21.15.101:0/1124871000 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x563c8fcbeff0 0x563c8fcc1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:48.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.905+0000 7f0f177fe700 10 client.28444 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:21:48.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.905+0000 7f0f37fff700 1 --2- 172.21.15.101:0/1124871000 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x563c8fcbeff0 0x563c8fcc1400 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=88509dc2a80a7917 server_cookie=f5812f201278cd36 in_seq=0 out_seq=0 2023-07-18T03:21:48.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.907+0000 7f0f177fe700 1 -- 172.21.15.101:0/1124871000 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563c8fbefeb0 con 0x563c8fcbeff0 2023-07-18T03:21:48.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.907+0000 7f0f177fe700 10 client.28444 handle_command_reply: tid=0 2023-07-18T03:21:48.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.907+0000 7f0f51f4cb80 1 client.28444 shutdown 2023-07-18T03:21:48.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.908+0000 7f0f51f4cb80 1 -- 172.21.15.101:0/1124871000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c8fca4be0 msgr2=0x563c8fcac750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:48.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.908+0000 7f0f51f4cb80 1 --2- 172.21.15.101:0/1124871000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c8fca4be0 0x563c8fcac750 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c005f50 tx=0x7f0f2c0cbdb0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.908+0000 7f0f51f4cb80 1 -- 172.21.15.101:0/1124871000 shutdown_connections 2023-07-18T03:21:48.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.908+0000 7f0f51f4cb80 1 --2- 172.21.15.101:0/1124871000 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x563c8fcbeff0 0x563c8fcc1400 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.908+0000 7f0f51f4cb80 1 --2- 172.21.15.101:0/1124871000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c8fcafad0 0x563c8fcacdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.908+0000 7f0f51f4cb80 1 --2- 172.21.15.101:0/1124871000 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563c8fca42d0 0x563c8fcaf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.908+0000 7f0f177fe700 0 client.28444 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:21:48.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.908+0000 7f0f51f4cb80 1 --2- 172.21.15.101:0/1124871000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c8fca4be0 0x563c8fcac750 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.908+0000 7f0f51f4cb80 1 -- 172.21.15.101:0/1124871000 >> 172.21.15.101:0/1124871000 conn(0x563c8fc9ada0 msgr2=0x563c8fca0230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:48.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.908+0000 7f0f51f4cb80 1 -- 172.21.15.101:0/1124871000 shutdown_connections 2023-07-18T03:21:48.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.908+0000 7f0f51f4cb80 1 -- 172.21.15.101:0/1124871000 wait complete. 2023-07-18T03:21:48.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.908+0000 7f0f51f4cb80 20 client.28444 trim_cache size 0 max 16384 2023-07-18T03:21:48.923 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.925+0000 7f0f51f4cb80 1 Processor -- start 2023-07-18T03:21:48.924 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.925+0000 7f0f51f4cb80 1 -- start start 2023-07-18T03:21:48.924 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.925+0000 7f0f51f4cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563c8fca42d0 0x563c8fca7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:48.925 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.925+0000 7f0f51f4cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c8fca4be0 0x563c8fca7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:48.925 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.925+0000 7f0f51f4cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c8fcafad0 0x563c8fcabf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:48.925 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.925+0000 7f0f51f4cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c8fcbbe30 con 0x563c8fca42d0 2023-07-18T03:21:48.925 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.925+0000 7f0f51f4cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c8fca1d20 con 0x563c8fcafad0 2023-07-18T03:21:48.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.925+0000 7f0f51f4cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c8fca5540 con 0x563c8fca4be0 2023-07-18T03:21:48.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.926+0000 7f0f37fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563c8fca42d0 0x563c8fca7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:48.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.926+0000 7f0f377fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c8fca4be0 0x563c8fca7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:48.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.926+0000 7f0f377fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c8fca4be0 0x563c8fca7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:57074/0 (socket says 172.21.15.101:57074) 2023-07-18T03:21:48.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.926+0000 7f0f377fe700 1 -- 172.21.15.101:0/3923014547 learned_addr learned my addr 172.21.15.101:0/3923014547 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:48.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.926+0000 7f0f3cb9c700 1 --2- 172.21.15.101:0/3923014547 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c8fcafad0 0x563c8fcabf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:48.928 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.926+0000 7f0f37fff700 1 -- 172.21.15.101:0/3923014547 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c8fca4be0 msgr2=0x563c8fca7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:48.928 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.926+0000 7f0f37fff700 1 --2- 172.21.15.101:0/3923014547 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c8fca4be0 0x563c8fca7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.928 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.926+0000 7f0f37fff700 1 -- 172.21.15.101:0/3923014547 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c8fcafad0 msgr2=0x563c8fcabf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:48.928 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.926+0000 7f0f37fff700 1 --2- 172.21.15.101:0/3923014547 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c8fcafad0 0x563c8fcabf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.928 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.926+0000 7f0f37fff700 1 -- 172.21.15.101:0/3923014547 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c8fa4b250 con 0x563c8fca42d0 2023-07-18T03:21:48.929 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.926+0000 7f0f377fe700 1 --2- 172.21.15.101:0/3923014547 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c8fca4be0 0x563c8fca7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:21:48.929 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.926+0000 7f0f37fff700 1 --2- 172.21.15.101:0/3923014547 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563c8fca42d0 0x563c8fca7a30 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f0f28009a90 tx=0x7f0f28008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd2cdf27e56139f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:48.929 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.927+0000 7f0f1effd700 1 -- 172.21.15.101:0/3923014547 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2802c070 con 0x563c8fca42d0 2023-07-18T03:21:48.929 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.927+0000 7f0f1effd700 1 -- 172.21.15.101:0/3923014547 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f28023040 con 0x563c8fca42d0 2023-07-18T03:21:48.929 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.927+0000 7f0f1effd700 1 -- 172.21.15.101:0/3923014547 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f28021040 con 0x563c8fca42d0 2023-07-18T03:21:48.929 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.927+0000 7f0f51f4cb80 1 -- 172.21.15.101:0/3923014547 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563c8fca42d0 msgr2=0x563c8fca7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:48.930 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.927+0000 7f0f51f4cb80 1 --2- 172.21.15.101:0/3923014547 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563c8fca42d0 0x563c8fca7a30 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f0f28009a90 tx=0x7f0f28008040 comp rx=0 tx=0).stop 2023-07-18T03:21:48.930 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.927+0000 7f0f51f4cb80 1 -- 172.21.15.101:0/3923014547 shutdown_connections 2023-07-18T03:21:48.930 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.927+0000 7f0f51f4cb80 1 --2- 172.21.15.101:0/3923014547 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c8fcafad0 0x563c8fcabf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.930 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.927+0000 7f0f51f4cb80 1 --2- 172.21.15.101:0/3923014547 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563c8fca42d0 0x563c8fca7a30 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.930 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.927+0000 7f0f51f4cb80 1 --2- 172.21.15.101:0/3923014547 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c8fca4be0 0x563c8fca7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.930 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.927+0000 7f0f51f4cb80 1 -- 172.21.15.101:0/3923014547 >> 172.21.15.101:0/3923014547 conn(0x563c8fc9ada0 msgr2=0x563c8fcbb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:48.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.928+0000 7f0f51f4cb80 1 -- 172.21.15.101:0/3923014547 shutdown_connections 2023-07-18T03:21:48.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.928+0000 7f0f51f4cb80 1 -- 172.21.15.101:0/3923014547 wait complete. 2023-07-18T03:21:48.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.928+0000 7f0f51f4cb80 1 -- start start 2023-07-18T03:21:48.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.928+0000 7f0f51f4cb80 1 Processor -- start 2023-07-18T03:21:48.934 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.929+0000 7f0f51f4cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563c8fca42d0 0x563c8fcb5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:48.934 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.929+0000 7f0f51f4cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c8fca4be0 0x563c8fcb6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:48.934 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.929+0000 7f0f51f4cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c8fcafad0 0x563c8fcb4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:48.934 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.929+0000 7f0f51f4cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c8fcb4820 con 0x563c8fca42d0 2023-07-18T03:21:48.934 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.929+0000 7f0f51f4cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c8fcb4990 con 0x563c8fca4be0 2023-07-18T03:21:48.934 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.929+0000 7f0f51f4cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c8fcb4b50 con 0x563c8fcafad0 2023-07-18T03:21:48.935 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.929+0000 7f0f51f4cb80 10 client.0 _ll_register_callbacks cb 0x563c8fb94760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:21:48.935 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.929+0000 7f0f37fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563c8fca42d0 0x563c8fcb5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:48.935 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.929+0000 7f0f3cb9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c8fcafad0 0x563c8fcb4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:48.935 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.929+0000 7f0f377fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c8fca4be0 0x563c8fcb6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:48.935 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.929+0000 7f0f3cb9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c8fcafad0 0x563c8fcb4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:57080/0 (socket says 172.21.15.101:57080) 2023-07-18T03:21:48.935 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.929+0000 7f0f3cb9c700 1 -- 172.21.15.101:0/1491370394 learned_addr learned my addr 172.21.15.101:0/1491370394 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:48.935 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.929+0000 7f0f3cb9c700 1 -- 172.21.15.101:0/1491370394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c8fca4be0 msgr2=0x563c8fcb6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:48.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.929+0000 7f0f3cb9c700 1 --2- 172.21.15.101:0/1491370394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c8fca4be0 0x563c8fcb6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.930+0000 7f0f3cb9c700 1 -- 172.21.15.101:0/1491370394 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563c8fca42d0 msgr2=0x563c8fcb5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:48.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.930+0000 7f0f3cb9c700 1 --2- 172.21.15.101:0/1491370394 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563c8fca42d0 0x563c8fcb5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.930+0000 7f0f3cb9c700 1 -- 172.21.15.101:0/1491370394 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563c8fa0dc60 con 0x563c8fcafad0 2023-07-18T03:21:48.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.930+0000 7f0f377fe700 1 --2- 172.21.15.101:0/1491370394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c8fca4be0 0x563c8fcb6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:21:48.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.930+0000 7f0f37fff700 1 --2- 172.21.15.101:0/1491370394 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563c8fca42d0 0x563c8fcb5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:21:48.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.930+0000 7f0f3cb9c700 1 --2- 172.21.15.101:0/1491370394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c8fcafad0 0x563c8fcb4200 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f0f38104460 tx=0x7f0f30004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90ef8f947bbadeb server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:48.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.930+0000 7f0f177fe700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:21:48.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.930+0000 7f0f177fe700 1 -- 172.21.15.101:0/1491370394 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3000e070 con 0x563c8fcafad0 2023-07-18T03:21:48.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.930+0000 7f0f51f4cb80 1 -- 172.21.15.101:0/1491370394 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0f2800edf0 con 0x563c8fcafad0 2023-07-18T03:21:48.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.930+0000 7f0f177fe700 1 -- 172.21.15.101:0/1491370394 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f0f30004bb0 con 0x563c8fcafad0 2023-07-18T03:21:48.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.930+0000 7f0f177fe700 1 -- 172.21.15.101:0/1491370394 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f30004d70 con 0x563c8fcafad0 2023-07-18T03:21:48.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.930+0000 7f0f51f4cb80 10 client.28450 fetch_fsmap learned FSMap version 38 2023-07-18T03:21:48.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.930+0000 7f0f51f4cb80 1 -- 172.21.15.101:0/1491370394 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x563c8fa24de0 con 0x563c8fcafad0 2023-07-18T03:21:48.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.931+0000 7f0f177fe700 1 -- 172.21.15.101:0/1491370394 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f2800edf0 con 0x563c8fcafad0 2023-07-18T03:21:48.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.931+0000 7f0f177fe700 1 -- 172.21.15.101:0/1491370394 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 38) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f0f30024050 con 0x563c8fcafad0 2023-07-18T03:21:48.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.931+0000 7f0f51f4cb80 10 client.28450 fetch_fsmap finished waiting for FSMap version 38 2023-07-18T03:21:48.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.931+0000 7f0f51f4cb80 10 client.28450 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:21:48.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.931+0000 7f0f51f4cb80 20 client.28450 populate_metadata read hostname 'smithi101' 2023-07-18T03:21:48.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.931+0000 7f0f51f4cb80 1 --2- 172.21.15.101:0/1491370394 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x563c8fd14030 0x563c8fd16440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:48.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.931+0000 7f0f51f4cb80 4 client.28450 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:21:48.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.931+0000 7f0f51f4cb80 1 -- 172.21.15.101:0/1491370394 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563c8fbb2050 con 0x563c8fd14030 2023-07-18T03:21:48.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.931+0000 7f0f37fff700 1 --2- 172.21.15.101:0/1491370394 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x563c8fd14030 0x563c8fd16440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:48.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.932+0000 7f0f37fff700 1 --2- 172.21.15.101:0/1491370394 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x563c8fd14030 0x563c8fd16440 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=155e6426fa893fd8 server_cookie=237dd65d27123596 in_seq=0 out_seq=0 2023-07-18T03:21:48.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.932+0000 7f0f177fe700 10 client.28450 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:21:48.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.933+0000 7f0f177fe700 1 -- 172.21.15.101:0/1491370394 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563c8fbb2050 con 0x563c8fd14030 2023-07-18T03:21:48.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.933+0000 7f0f177fe700 10 client.28450 handle_command_reply: tid=0 2023-07-18T03:21:48.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.933+0000 7f0f51f4cb80 1 client.28450 shutdown 2023-07-18T03:21:48.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.933+0000 7f0f51f4cb80 1 -- 172.21.15.101:0/1491370394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c8fcafad0 msgr2=0x563c8fcb4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:48.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.933+0000 7f0f51f4cb80 1 --2- 172.21.15.101:0/1491370394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c8fcafad0 0x563c8fcb4200 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f0f38104460 tx=0x7f0f30004ad0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.933+0000 7f0f51f4cb80 1 -- 172.21.15.101:0/1491370394 shutdown_connections 2023-07-18T03:21:48.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.933+0000 7f0f51f4cb80 1 --2- 172.21.15.101:0/1491370394 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x563c8fd14030 0x563c8fd16440 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.933+0000 7f0f51f4cb80 1 --2- 172.21.15.101:0/1491370394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c8fcafad0 0x563c8fcb4200 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.933+0000 7f0f51f4cb80 1 --2- 172.21.15.101:0/1491370394 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563c8fca42d0 0x563c8fcb5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.933+0000 7f0f51f4cb80 1 --2- 172.21.15.101:0/1491370394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c8fca4be0 0x563c8fcb6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.933+0000 7f0f51f4cb80 1 -- 172.21.15.101:0/1491370394 >> 172.21.15.101:0/1491370394 conn(0x563c8fc9ada0 msgr2=0x563c8fca63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:48.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.933+0000 7f0f177fe700 0 client.28450 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:21:48.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.934+0000 7f0f51f4cb80 1 -- 172.21.15.101:0/1491370394 shutdown_connections 2023-07-18T03:21:48.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.934+0000 7f0f51f4cb80 1 -- 172.21.15.101:0/1491370394 wait complete. 2023-07-18T03:21:48.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.934+0000 7f0f51f4cb80 20 client.28450 trim_cache size 0 max 16384 2023-07-18T03:21:48.942 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:21:48.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.935+0000 7f0f3e5ff700 1 -- 172.21.15.101:0/867669069 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0f200a1340 msgr2=0x7f0f200a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:48.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.935+0000 7f0f3e5ff700 1 --2- 172.21.15.101:0/867669069 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0f200a1340 0x7f0f200a37f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0f30009550 tx=0x7f0f30009920 comp rx=0 tx=0).stop 2023-07-18T03:21:48.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.935+0000 7f0f3e5ff700 1 -- 172.21.15.101:0/867669069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f380ff380 msgr2=0x7f0f38194de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:48.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.935+0000 7f0f3e5ff700 1 --2- 172.21.15.101:0/867669069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f380ff380 0x7f0f38194de0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c005910 tx=0x7f0f2c004730 comp rx=0 tx=0).stop 2023-07-18T03:21:48.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.935+0000 7f0f3e5ff700 1 -- 172.21.15.101:0/867669069 shutdown_connections 2023-07-18T03:21:48.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.935+0000 7f0f3e5ff700 1 --2- 172.21.15.101:0/867669069 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0f381049e0 0x7f0f38195320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.935+0000 7f0f3e5ff700 1 --2- 172.21.15.101:0/867669069 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f380fe9d0 0x7f0f381948a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.935+0000 7f0f3e5ff700 1 --2- 172.21.15.101:0/867669069 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0f200a1340 0x7f0f200a37f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.935+0000 7f0f3e5ff700 1 --2- 172.21.15.101:0/867669069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f380ff380 0x7f0f38194de0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:48.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.935+0000 7f0f3e5ff700 1 -- 172.21.15.101:0/867669069 >> 172.21.15.101:0/867669069 conn(0x7f0f380fa4b0 msgr2=0x7f0f380fba80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:48.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.935+0000 7f0f3e5ff700 1 -- 172.21.15.101:0/867669069 shutdown_connections 2023-07-18T03:21:48.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:48.935+0000 7f0f3e5ff700 1 -- 172.21.15.101:0/867669069 wait complete. 2023-07-18T03:21:48.950 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-18T03:21:48.964 INFO:tasks.workunit.client.0.smithi101.stdout: 10 40371 32.11 MB/sec warmup 73 sec latency 89.937 ms 2023-07-18T03:21:49.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.230+0000 7fee9f3ff700 1 -- 172.21.15.101:0/2111255342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee98105ae0 msgr2=0x7fee98105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:49.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.230+0000 7fee9f3ff700 1 --2- 172.21.15.101:0/2111255342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee98105ae0 0x7fee98105eb0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fee94009990 tx=0x7fee94009ca0 comp rx=0 tx=0).stop 2023-07-18T03:21:49.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.230+0000 7fee9f3ff700 1 -- 172.21.15.101:0/2111255342 shutdown_connections 2023-07-18T03:21:49.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.230+0000 7fee9f3ff700 1 --2- 172.21.15.101:0/2111255342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee98105ae0 0x7fee98105eb0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:49.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.230+0000 7fee9f3ff700 1 --2- 172.21.15.101:0/2111255342 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fee98100480 0x7fee9810c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:49.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.230+0000 7fee9f3ff700 1 --2- 172.21.15.101:0/2111255342 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee980ffad0 0x7fee980fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:49.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.230+0000 7fee9f3ff700 1 -- 172.21.15.101:0/2111255342 >> 172.21.15.101:0/2111255342 conn(0x7fee980fb5f0 msgr2=0x7fee980fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:49.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.231+0000 7fee9f3ff700 1 -- 172.21.15.101:0/2111255342 shutdown_connections 2023-07-18T03:21:49.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.231+0000 7fee9f3ff700 1 -- 172.21.15.101:0/2111255342 wait complete. 2023-07-18T03:21:49.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.232+0000 7fee9f3ff700 1 Processor -- start 2023-07-18T03:21:49.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.232+0000 7fee9f3ff700 1 -- start start 2023-07-18T03:21:49.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.232+0000 7fee9f3ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee980ffad0 0x7fee98197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:49.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.232+0000 7fee9f3ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee98100480 0x7fee98198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:49.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.232+0000 7fee9f3ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fee98105ae0 0x7fee9819c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:49.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.232+0000 7fee9f3ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9819cdf0 con 0x7fee98105ae0 2023-07-18T03:21:49.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.232+0000 7fee9f3ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9819cf60 con 0x7fee980ffad0 2023-07-18T03:21:49.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.232+0000 7fee9f3ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9819d120 con 0x7fee98100480 2023-07-18T03:21:49.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.233+0000 7fee9d19b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee980ffad0 0x7fee98197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:49.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.233+0000 7fee9c99a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee98100480 0x7fee98198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:49.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.233+0000 7fee9d19b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee980ffad0 0x7fee98197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47476/0 (socket says 172.21.15.101:47476) 2023-07-18T03:21:49.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.233+0000 7fee9d19b700 1 -- 172.21.15.101:0/894822993 learned_addr learned my addr 172.21.15.101:0/894822993 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:49.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.233+0000 7fee9d99c700 1 --2- 172.21.15.101:0/894822993 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fee98105ae0 0x7fee9819c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:49.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.233+0000 7fee9c99a700 1 -- 172.21.15.101:0/894822993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee980ffad0 msgr2=0x7fee98197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:49.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.233+0000 7fee9c99a700 1 --2- 172.21.15.101:0/894822993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee980ffad0 0x7fee98197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:49.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.233+0000 7fee9c99a700 1 -- 172.21.15.101:0/894822993 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fee98105ae0 msgr2=0x7fee9819c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:49.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.233+0000 7fee9c99a700 1 --2- 172.21.15.101:0/894822993 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fee98105ae0 0x7fee9819c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:49.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.233+0000 7fee9c99a700 1 -- 172.21.15.101:0/894822993 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee940095f0 con 0x7fee98100480 2023-07-18T03:21:49.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.234+0000 7fee9c99a700 1 --2- 172.21.15.101:0/894822993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee98100480 0x7fee98198350 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fee8800e9f0 tx=0x7fee8800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:49.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.234+0000 7fee8e7fc700 1 -- 172.21.15.101:0/894822993 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8800c9c0 con 0x7fee98100480 2023-07-18T03:21:49.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.234+0000 7fee8e7fc700 1 -- 172.21.15.101:0/894822993 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee88010430 con 0x7fee98100480 2023-07-18T03:21:49.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.234+0000 7fee9f3ff700 1 -- 172.21.15.101:0/894822993 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee9819d400 con 0x7fee98100480 2023-07-18T03:21:49.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.234+0000 7fee8e7fc700 1 -- 172.21.15.101:0/894822993 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee88018680 con 0x7fee98100480 2023-07-18T03:21:49.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.234+0000 7fee9f3ff700 1 -- 172.21.15.101:0/894822993 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee981a4c40 con 0x7fee98100480 2023-07-18T03:21:49.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.237+0000 7fee8e7fc700 1 -- 172.21.15.101:0/894822993 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fee88010860 con 0x7fee98100480 2023-07-18T03:21:49.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.238+0000 7fee8e7fc700 1 --2- 172.21.15.101:0/894822993 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fee840a1340 0x7fee840a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:49.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.238+0000 7fee8e7fc700 1 -- 172.21.15.101:0/894822993 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fee88014070 con 0x7fee98100480 2023-07-18T03:21:49.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.239+0000 7fee9d19b700 1 --2- 172.21.15.101:0/894822993 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fee840a1340 0x7fee840a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:49.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.239+0000 7feeb2da4b80 1 Processor -- start 2023-07-18T03:21:49.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.239+0000 7feeb2da4b80 1 -- start start 2023-07-18T03:21:49.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.239+0000 7feeb2da4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56138c8fb2d0 0x56138c8fb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:49.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.239+0000 7feeb2da4b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56138c8fbbe0 0x56138c906590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:49.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.239+0000 7feeb2da4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56138c906ad0 0x56138c908eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:49.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.239+0000 7feeb2da4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56138c801cb0 con 0x56138c8fb2d0 2023-07-18T03:21:49.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.239+0000 7feeb2da4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56138c854a40 con 0x56138c906ad0 2023-07-18T03:21:49.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.239+0000 7feeb2da4b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56138c84ded0 con 0x56138c8fbbe0 2023-07-18T03:21:49.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.240+0000 7fee9c99a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56138c8fbbe0 0x56138c906590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:49.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.240+0000 7fee9c99a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56138c8fbbe0 0x56138c906590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:57098/0 (socket says 172.21.15.101:57098) 2023-07-18T03:21:49.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.240+0000 7fee9c99a700 1 -- 172.21.15.101:0/3378017926 learned_addr learned my addr 172.21.15.101:0/3378017926 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:49.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.240+0000 7fee9d19b700 1 --2- 172.21.15.101:0/894822993 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fee840a1340 0x7fee840a37f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fee94006010 tx=0x7fee9400b260 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:49.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.240+0000 7fee9d99c700 1 --2- 172.21.15.101:0/3378017926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56138c906ad0 0x56138c908eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:49.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.240+0000 7fee9d19b700 1 --2- 172.21.15.101:0/3378017926 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56138c8fb2d0 0x56138c8fb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:49.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.240+0000 7fee9c99a700 1 -- 172.21.15.101:0/3378017926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56138c906ad0 msgr2=0x56138c908eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:49.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.240+0000 7fee9c99a700 1 --2- 172.21.15.101:0/3378017926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56138c906ad0 0x56138c908eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:49.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.240+0000 7fee9c99a700 1 -- 172.21.15.101:0/3378017926 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56138c8fb2d0 msgr2=0x56138c8fb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:49.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.240+0000 7fee9c99a700 1 --2- 172.21.15.101:0/3378017926 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56138c8fb2d0 0x56138c8fb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:49.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.240+0000 7fee9c99a700 1 -- 172.21.15.101:0/3378017926 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56138c67bde0 con 0x56138c8fbbe0 2023-07-18T03:21:49.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.240+0000 7fee9c99a700 1 --2- 172.21.15.101:0/3378017926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56138c8fbbe0 0x56138c906590 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fee88096910 tx=0x7fee88099900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4768a6be3194bcc4 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:49.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.240+0000 7fee837fe700 1 -- 172.21.15.101:0/3378017926 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee880105a0 con 0x56138c8fbbe0 2023-07-18T03:21:49.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.241+0000 7fee837fe700 1 -- 172.21.15.101:0/3378017926 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee8809a100 con 0x56138c8fbbe0 2023-07-18T03:21:49.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.241+0000 7fee837fe700 1 -- 172.21.15.101:0/3378017926 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee880a1e60 con 0x56138c8fbbe0 2023-07-18T03:21:49.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.241+0000 7feeb2da4b80 1 -- 172.21.15.101:0/3378017926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56138c8fbbe0 msgr2=0x56138c906590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:49.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.241+0000 7feeb2da4b80 1 --2- 172.21.15.101:0/3378017926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56138c8fbbe0 0x56138c906590 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fee88096910 tx=0x7fee88099900 comp rx=0 tx=0).stop 2023-07-18T03:21:49.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.241+0000 7feeb2da4b80 1 -- 172.21.15.101:0/3378017926 shutdown_connections 2023-07-18T03:21:49.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.241+0000 7feeb2da4b80 1 --2- 172.21.15.101:0/3378017926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56138c906ad0 0x56138c908eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:49.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.241+0000 7feeb2da4b80 1 --2- 172.21.15.101:0/3378017926 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56138c8fb2d0 0x56138c8fb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:49.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.241+0000 7feeb2da4b80 1 --2- 172.21.15.101:0/3378017926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56138c8fbbe0 0x56138c906590 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:49.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.241+0000 7feeb2da4b80 1 -- 172.21.15.101:0/3378017926 >> 172.21.15.101:0/3378017926 conn(0x56138c8f1da0 msgr2=0x56138c8f76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:49.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.242+0000 7feeb2da4b80 1 -- 172.21.15.101:0/3378017926 shutdown_connections 2023-07-18T03:21:49.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.242+0000 7feeb2da4b80 1 -- 172.21.15.101:0/3378017926 wait complete. 2023-07-18T03:21:49.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.242+0000 7feeb2da4b80 1 -- start start 2023-07-18T03:21:49.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.242+0000 7feeb2da4b80 1 Processor -- start 2023-07-18T03:21:49.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.243+0000 7feeb2da4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56138c8fb2d0 0x56138c903820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:49.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.243+0000 7feeb2da4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56138c8fbbe0 0x56138c903d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:49.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.243+0000 7feeb2da4b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56138c906ad0 0x56138c9042a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:49.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.243+0000 7feeb2da4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56138c7edac0 con 0x56138c8fbbe0 2023-07-18T03:21:49.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.243+0000 7feeb2da4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56138c7ed7c0 con 0x56138c8fb2d0 2023-07-18T03:21:49.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.243+0000 7feeb2da4b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56138c6d1920 con 0x56138c906ad0 2023-07-18T03:21:49.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.243+0000 7feeb2da4b80 10 client.0 _ll_register_callbacks cb 0x56138c7eb760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:21:49.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.243+0000 7fee9d19b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56138c8fb2d0 0x56138c903820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:49.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.243+0000 7fee9c99a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56138c8fbbe0 0x56138c903d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:49.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.243+0000 7fee9d19b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56138c8fb2d0 0x56138c903820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47492/0 (socket says 172.21.15.101:47492) 2023-07-18T03:21:49.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.243+0000 7fee9d19b700 1 -- 172.21.15.101:0/1717503077 learned_addr learned my addr 172.21.15.101:0/1717503077 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:49.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.244+0000 7fee9d99c700 1 --2- 172.21.15.101:0/1717503077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56138c906ad0 0x56138c9042a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:49.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.244+0000 7fee9c99a700 1 -- 172.21.15.101:0/1717503077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56138c906ad0 msgr2=0x56138c9042a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:49.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.244+0000 7fee9c99a700 1 --2- 172.21.15.101:0/1717503077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56138c906ad0 0x56138c9042a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:49.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.244+0000 7fee9c99a700 1 -- 172.21.15.101:0/1717503077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56138c8fb2d0 msgr2=0x56138c903820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:49.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.244+0000 7fee9c99a700 1 --2- 172.21.15.101:0/1717503077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56138c8fb2d0 0x56138c903820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:49.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.244+0000 7fee9c99a700 1 -- 172.21.15.101:0/1717503077 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee981a4c40 con 0x56138c8fbbe0 2023-07-18T03:21:49.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.244+0000 7fee9c99a700 1 --2- 172.21.15.101:0/1717503077 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56138c8fbbe0 0x56138c903d60 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fee88096910 tx=0x7fee88099da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fe47d6aa43c717c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:49.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.244+0000 7fee80ff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:21:49.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.244+0000 7fee80ff9700 1 -- 172.21.15.101:0/1717503077 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee880b8220 con 0x56138c8fbbe0 2023-07-18T03:21:49.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.244+0000 7fee80ff9700 1 -- 172.21.15.101:0/1717503077 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fee880a1e60 con 0x56138c8fbbe0 2023-07-18T03:21:49.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.244+0000 7feeb2da4b80 1 -- 172.21.15.101:0/1717503077 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56138c7a74f0 con 0x56138c8fbbe0 2023-07-18T03:21:49.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.245+0000 7fee80ff9700 1 -- 172.21.15.101:0/1717503077 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee880ca2c0 con 0x56138c8fbbe0 2023-07-18T03:21:49.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.245+0000 7feeb2da4b80 10 client.18378 fetch_fsmap learned FSMap version 38 2023-07-18T03:21:49.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.245+0000 7feeb2da4b80 1 -- 172.21.15.101:0/1717503077 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x56138c664c60 con 0x56138c8fbbe0 2023-07-18T03:21:49.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.245+0000 7fee80ff9700 1 -- 172.21.15.101:0/1717503077 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56138c7a74f0 con 0x56138c8fbbe0 2023-07-18T03:21:49.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.245+0000 7fee80ff9700 1 -- 172.21.15.101:0/1717503077 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 38) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fee88095d20 con 0x56138c8fbbe0 2023-07-18T03:21:49.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.245+0000 7feeb2da4b80 10 client.18378 fetch_fsmap finished waiting for FSMap version 38 2023-07-18T03:21:49.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.245+0000 7feeb2da4b80 10 client.18378 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:21:49.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.245+0000 7feeb2da4b80 20 client.18378 populate_metadata read hostname 'smithi101' 2023-07-18T03:21:49.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.245+0000 7feeb2da4b80 1 --2- 172.21.15.101:0/1717503077 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56138c927080 0x56138c929490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:49.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.245+0000 7feeb2da4b80 4 client.18378 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:21:49.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.245+0000 7feeb2da4b80 1 -- 172.21.15.101:0/1717503077 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56138c846eb0 con 0x56138c927080 2023-07-18T03:21:49.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.246+0000 7fee9d19b700 1 --2- 172.21.15.101:0/1717503077 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56138c927080 0x56138c929490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:49.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.247+0000 7fee80ff9700 10 client.18378 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:21:49.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.247+0000 7fee9d19b700 1 --2- 172.21.15.101:0/1717503077 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56138c927080 0x56138c929490 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc5cf6a8851a549c server_cookie=3f92f3f9935438c7 in_seq=0 out_seq=0 2023-07-18T03:21:49.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.249+0000 7fee80ff9700 1 -- 172.21.15.101:0/1717503077 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56138c846eb0 con 0x56138c927080 2023-07-18T03:21:49.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.249+0000 7fee80ff9700 10 client.18378 handle_command_reply: tid=0 2023-07-18T03:21:49.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.249+0000 7feeb2da4b80 1 client.18378 shutdown 2023-07-18T03:21:49.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.249+0000 7feeb2da4b80 1 -- 172.21.15.101:0/1717503077 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56138c8fbbe0 msgr2=0x56138c903d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:49.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.249+0000 7feeb2da4b80 1 --2- 172.21.15.101:0/1717503077 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56138c8fbbe0 0x56138c903d60 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fee88096910 tx=0x7fee88099da0 comp rx=0 tx=0).stop 2023-07-18T03:21:49.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.249+0000 7feeb2da4b80 1 -- 172.21.15.101:0/1717503077 shutdown_connections 2023-07-18T03:21:49.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.249+0000 7feeb2da4b80 1 --2- 172.21.15.101:0/1717503077 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56138c927080 0x56138c929490 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:49.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.249+0000 7feeb2da4b80 1 --2- 172.21.15.101:0/1717503077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56138c8fb2d0 0x56138c903820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:49.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.249+0000 7feeb2da4b80 1 --2- 172.21.15.101:0/1717503077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56138c906ad0 0x56138c9042a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:49.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.249+0000 7feeb2da4b80 1 --2- 172.21.15.101:0/1717503077 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56138c8fbbe0 0x56138c903d60 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:49.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.249+0000 7feeb2da4b80 1 -- 172.21.15.101:0/1717503077 >> 172.21.15.101:0/1717503077 conn(0x56138c8f1da0 msgr2=0x56138c8f7230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:49.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.249+0000 7fee80ff9700 0 client.18378 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:21:49.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.249+0000 7feeb2da4b80 1 -- 172.21.15.101:0/1717503077 shutdown_connections 2023-07-18T03:21:49.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.249+0000 7feeb2da4b80 1 -- 172.21.15.101:0/1717503077 wait complete. 2023-07-18T03:21:49.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.250+0000 7feeb2da4b80 20 client.18378 trim_cache size 0 max 16384 2023-07-18T03:21:49.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.267+0000 7feeb2da4b80 1 Processor -- start 2023-07-18T03:21:49.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.267+0000 7feeb2da4b80 1 -- start start 2023-07-18T03:21:49.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.267+0000 7feeb2da4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56138c8fb2d0 0x56138c8fe1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:49.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.267+0000 7feeb2da4b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56138c906ad0 0x56138c90d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:49.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.268+0000 7feeb2da4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56138c8ff240 0x56138c903d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:49.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.268+0000 7feeb2da4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56138c8eefa0 con 0x56138c8ff240 2023-07-18T03:21:49.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.268+0000 7feeb2da4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56138c8f84d0 con 0x56138c8fb2d0 2023-07-18T03:21:49.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.268+0000 7feeb2da4b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56138c8fcdd0 con 0x56138c906ad0 2023-07-18T03:21:49.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.268+0000 7fee9d19b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56138c8fb2d0 0x56138c8fe1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:49.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.268+0000 7fee9c99a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56138c906ad0 0x56138c90d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:49.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.268+0000 7fee9d19b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56138c8fb2d0 0x56138c8fe1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47496/0 (socket says 172.21.15.101:47496) 2023-07-18T03:21:49.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.268+0000 7fee9d19b700 1 -- 172.21.15.101:0/2768259864 learned_addr learned my addr 172.21.15.101:0/2768259864 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:49.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.268+0000 7fee9d99c700 1 --2- 172.21.15.101:0/2768259864 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56138c8ff240 0x56138c903d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:49.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.268+0000 7fee9c99a700 1 -- 172.21.15.101:0/2768259864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56138c8fb2d0 msgr2=0x56138c8fe1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:49.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.268+0000 7fee9c99a700 1 --2- 172.21.15.101:0/2768259864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56138c8fb2d0 0x56138c8fe1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:49.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.268+0000 7fee9c99a700 1 -- 172.21.15.101:0/2768259864 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56138c8ff240 msgr2=0x56138c903d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:49.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.268+0000 7fee9c99a700 1 --2- 172.21.15.101:0/2768259864 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56138c8ff240 0x56138c903d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:49.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.268+0000 7fee9c99a700 1 -- 172.21.15.101:0/2768259864 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56138c8f35e0 con 0x56138c906ad0 2023-07-18T03:21:49.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.268+0000 7fee9d19b700 1 --2- 172.21.15.101:0/2768259864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56138c8fb2d0 0x56138c8fe1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:21:49.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.269+0000 7fee9c99a700 1 --2- 172.21.15.101:0/2768259864 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56138c906ad0 0x56138c90d4f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x56138c9048b0 tx=0x7fee88099900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f76cc2f2ad36832d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:49.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.269+0000 7fee83fff700 1 -- 172.21.15.101:0/2768259864 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee88094c50 con 0x56138c906ad0 2023-07-18T03:21:49.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.269+0000 7fee83fff700 1 -- 172.21.15.101:0/2768259864 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee880b8220 con 0x56138c906ad0 2023-07-18T03:21:49.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.269+0000 7fee83fff700 1 -- 172.21.15.101:0/2768259864 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee88095e80 con 0x56138c906ad0 2023-07-18T03:21:49.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.270+0000 7feeb2da4b80 1 -- 172.21.15.101:0/2768259864 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56138c906ad0 msgr2=0x56138c90d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:49.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.270+0000 7feeb2da4b80 1 --2- 172.21.15.101:0/2768259864 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56138c906ad0 0x56138c90d4f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x56138c9048b0 tx=0x7fee88099900 comp rx=0 tx=0).stop 2023-07-18T03:21:49.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.270+0000 7feeb2da4b80 1 -- 172.21.15.101:0/2768259864 shutdown_connections 2023-07-18T03:21:49.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.270+0000 7feeb2da4b80 1 --2- 172.21.15.101:0/2768259864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56138c8fb2d0 0x56138c8fe1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:49.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.270+0000 7feeb2da4b80 1 --2- 172.21.15.101:0/2768259864 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56138c8ff240 0x56138c903d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:49.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.270+0000 7feeb2da4b80 1 --2- 172.21.15.101:0/2768259864 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56138c906ad0 0x56138c90d4f0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:49.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.270+0000 7feeb2da4b80 1 -- 172.21.15.101:0/2768259864 >> 172.21.15.101:0/2768259864 conn(0x56138c8f1da0 msgr2=0x56138c8feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:49.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.270+0000 7feeb2da4b80 1 -- 172.21.15.101:0/2768259864 shutdown_connections 2023-07-18T03:21:49.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.270+0000 7feeb2da4b80 1 -- 172.21.15.101:0/2768259864 wait complete. 2023-07-18T03:21:49.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.270+0000 7feeb2da4b80 1 -- start start 2023-07-18T03:21:49.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.270+0000 7feeb2da4b80 1 Processor -- start 2023-07-18T03:21:49.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.271+0000 7feeb2da4b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56138c8fb2d0 0x56138c96f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:49.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.271+0000 7feeb2da4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56138c8ff240 0x56138c96f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:49.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.271+0000 7feeb2da4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56138c906ad0 0x56138c973f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:49.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.271+0000 7feeb2da4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56138c974460 con 0x56138c8ff240 2023-07-18T03:21:49.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.271+0000 7feeb2da4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56138c9745d0 con 0x56138c906ad0 2023-07-18T03:21:49.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.271+0000 7feeb2da4b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56138c974790 con 0x56138c8fb2d0 2023-07-18T03:21:49.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.271+0000 7feeb2da4b80 10 client.0 _ll_register_callbacks cb 0x7fee88099a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:21:49.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.271+0000 7fee9d19b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56138c8fb2d0 0x56138c96f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:49.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.271+0000 7fee9c99a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56138c8ff240 0x56138c96f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:49.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.271+0000 7fee9d19b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56138c8fb2d0 0x56138c96f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:57124/0 (socket says 172.21.15.101:57124) 2023-07-18T03:21:49.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.271+0000 7fee9d19b700 1 -- 172.21.15.101:0/409058914 learned_addr learned my addr 172.21.15.101:0/409058914 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:49.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.272+0000 7fee9d19b700 1 -- 172.21.15.101:0/409058914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56138c906ad0 msgr2=0x56138c973f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:49.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.272+0000 7fee9d19b700 1 --2- 172.21.15.101:0/409058914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56138c906ad0 0x56138c973f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:49.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.272+0000 7fee9d19b700 1 -- 172.21.15.101:0/409058914 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56138c8ff240 msgr2=0x56138c96f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:49.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.272+0000 7fee9d19b700 1 --2- 172.21.15.101:0/409058914 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56138c8ff240 0x56138c96f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:49.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.272+0000 7fee9d19b700 1 -- 172.21.15.101:0/409058914 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee9402c040 con 0x56138c8fb2d0 2023-07-18T03:21:49.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.272+0000 7fee9d19b700 1 --2- 172.21.15.101:0/409058914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56138c8fb2d0 0x56138c96f0c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x56138c929da0 tx=0x7fee94009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=938b962b1434915 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:49.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.272+0000 7fee80ff9700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:21:49.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.272+0000 7fee80ff9700 1 -- 172.21.15.101:0/409058914 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee940054d0 con 0x56138c8fb2d0 2023-07-18T03:21:49.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.272+0000 7feeb2da4b80 1 -- 172.21.15.101:0/409058914 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fee8809a100 con 0x56138c8fb2d0 2023-07-18T03:21:49.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.272+0000 7fee80ff9700 1 -- 172.21.15.101:0/409058914 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fee94005630 con 0x56138c8fb2d0 2023-07-18T03:21:49.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.272+0000 7fee80ff9700 1 -- 172.21.15.101:0/409058914 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee94033a60 con 0x56138c8fb2d0 2023-07-18T03:21:49.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.273+0000 7feeb2da4b80 10 client.28468 fetch_fsmap learned FSMap version 38 2023-07-18T03:21:49.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.273+0000 7feeb2da4b80 1 -- 172.21.15.101:0/409058914 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x56138c664c60 con 0x56138c8fb2d0 2023-07-18T03:21:49.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.273+0000 7fee80ff9700 1 -- 172.21.15.101:0/409058914 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee8809a100 con 0x56138c8fb2d0 2023-07-18T03:21:49.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.273+0000 7fee80ff9700 1 -- 172.21.15.101:0/409058914 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 38) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fee9400fc20 con 0x56138c8fb2d0 2023-07-18T03:21:49.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.273+0000 7feeb2da4b80 10 client.28468 fetch_fsmap finished waiting for FSMap version 38 2023-07-18T03:21:49.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.273+0000 7feeb2da4b80 10 client.28468 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:21:49.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.273+0000 7feeb2da4b80 20 client.28468 populate_metadata read hostname 'smithi101' 2023-07-18T03:21:49.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.273+0000 7feeb2da4b80 1 --2- 172.21.15.101:0/409058914 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56138c97bfa0 0x56138c97e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:49.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.273+0000 7feeb2da4b80 4 client.28468 mds_command: new command op to 24562 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:21:49.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.273+0000 7feeb2da4b80 1 -- 172.21.15.101:0/409058914 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56138c809050 con 0x56138c97bfa0 2023-07-18T03:21:49.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.273+0000 7fee9c99a700 1 --2- 172.21.15.101:0/409058914 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56138c97bfa0 0x56138c97e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:49.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.274+0000 7fee80ff9700 10 client.28468 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:21:49.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.274+0000 7fee9c99a700 1 --2- 172.21.15.101:0/409058914 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56138c97bfa0 0x56138c97e3b0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c067df3ec54a2ee server_cookie=57f3e4aeba870867 in_seq=0 out_seq=0 2023-07-18T03:21:49.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.276+0000 7fee80ff9700 1 -- 172.21.15.101:0/409058914 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56138c809050 con 0x56138c97bfa0 2023-07-18T03:21:49.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.276+0000 7fee80ff9700 10 client.28468 handle_command_reply: tid=0 2023-07-18T03:21:49.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.276+0000 7feeb2da4b80 1 client.28468 shutdown 2023-07-18T03:21:49.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.276+0000 7feeb2da4b80 1 -- 172.21.15.101:0/409058914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56138c8fb2d0 msgr2=0x56138c96f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:49.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.276+0000 7feeb2da4b80 1 --2- 172.21.15.101:0/409058914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56138c8fb2d0 0x56138c96f0c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x56138c929da0 tx=0x7fee94009ca0 comp rx=0 tx=0).stop 2023-07-18T03:21:49.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.276+0000 7feeb2da4b80 1 -- 172.21.15.101:0/409058914 shutdown_connections 2023-07-18T03:21:49.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.276+0000 7feeb2da4b80 1 --2- 172.21.15.101:0/409058914 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56138c97bfa0 0x56138c97e3b0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:49.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.276+0000 7feeb2da4b80 1 --2- 172.21.15.101:0/409058914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56138c906ad0 0x56138c973f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:49.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.276+0000 7fee80ff9700 0 client.28468 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:21:49.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.276+0000 7feeb2da4b80 1 --2- 172.21.15.101:0/409058914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56138c8fb2d0 0x56138c96f0c0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:49.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.276+0000 7feeb2da4b80 1 --2- 172.21.15.101:0/409058914 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56138c8ff240 0x56138c96f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:49.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.276+0000 7feeb2da4b80 1 -- 172.21.15.101:0/409058914 >> 172.21.15.101:0/409058914 conn(0x56138c8f1da0 msgr2=0x56138c903d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:49.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.277+0000 7feeb2da4b80 1 -- 172.21.15.101:0/409058914 shutdown_connections 2023-07-18T03:21:49.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.277+0000 7feeb2da4b80 1 -- 172.21.15.101:0/409058914 wait complete. 2023-07-18T03:21:49.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.277+0000 7feeb2da4b80 20 client.28468 trim_cache size 0 max 16384 2023-07-18T03:21:49.278 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:21:49.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.277+0000 7fee9f3ff700 1 -- 172.21.15.101:0/894822993 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fee840a1340 msgr2=0x7fee840a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:49.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.277+0000 7fee9f3ff700 1 --2- 172.21.15.101:0/894822993 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fee840a1340 0x7fee840a37f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fee94006010 tx=0x7fee9400b260 comp rx=0 tx=0).stop 2023-07-18T03:21:49.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.277+0000 7fee9f3ff700 1 -- 172.21.15.101:0/894822993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee98100480 msgr2=0x7fee98198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:49.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.277+0000 7fee9f3ff700 1 --2- 172.21.15.101:0/894822993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee98100480 0x7fee98198350 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fee8800e9f0 tx=0x7fee8800ed00 comp rx=0 tx=0).stop 2023-07-18T03:21:49.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.278+0000 7fee9f3ff700 1 -- 172.21.15.101:0/894822993 shutdown_connections 2023-07-18T03:21:49.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.278+0000 7fee9f3ff700 1 --2- 172.21.15.101:0/894822993 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fee840a1340 0x7fee840a37f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:49.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.278+0000 7fee9f3ff700 1 --2- 172.21.15.101:0/894822993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee980ffad0 0x7fee98197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:49.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.278+0000 7fee9f3ff700 1 --2- 172.21.15.101:0/894822993 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fee98105ae0 0x7fee9819c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:49.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.278+0000 7fee9f3ff700 1 --2- 172.21.15.101:0/894822993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee98100480 0x7fee98198350 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:49.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.278+0000 7fee9f3ff700 1 -- 172.21.15.101:0/894822993 >> 172.21.15.101:0/894822993 conn(0x7fee980fb5f0 msgr2=0x7fee9810ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:49.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.278+0000 7fee9f3ff700 1 -- 172.21.15.101:0/894822993 shutdown_connections 2023-07-18T03:21:49.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:49.278+0000 7fee9f3ff700 1 -- 172.21.15.101:0/894822993 wait complete. 2023-07-18T03:21:49.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:49 smithi101 ceph-mon[126379]: pgmap v446: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 7.5 MiB/s wr, 469 op/s 2023-07-18T03:21:49.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:49 smithi155 ceph-mon[128885]: pgmap v446: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 7.5 MiB/s wr, 469 op/s 2023-07-18T03:21:49.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:49 smithi179 ceph-mon[129229]: pgmap v446: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 7.5 MiB/s wr, 469 op/s 2023-07-18T03:21:49.964 INFO:tasks.workunit.client.0.smithi101.stdout: 10 41224 32.41 MB/sec warmup 74 sec latency 62.775 ms 2023-07-18T03:21:50.297 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-18T03:21:50.297 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-18T03:21:50.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.768+0000 7f18ce3ff700 1 -- 172.21.15.101:0/3595986790 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18c80728b0 msgr2=0x7f18c8072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:50.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.768+0000 7f18ce3ff700 1 --2- 172.21.15.101:0/3595986790 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18c80728b0 0x7f18c8072c80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f18c001c0d0 tx=0x7f18c001c3e0 comp rx=0 tx=0).stop 2023-07-18T03:21:50.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.768+0000 7f18ce3ff700 1 -- 172.21.15.101:0/3595986790 shutdown_connections 2023-07-18T03:21:50.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.768+0000 7f18ce3ff700 1 --2- 172.21.15.101:0/3595986790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18c8110ae0 0x7f18c8112ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:50.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.768+0000 7f18ce3ff700 1 --2- 172.21.15.101:0/3595986790 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18c80728b0 0x7f18c8072c80 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:50.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.768+0000 7f18ce3ff700 1 --2- 172.21.15.101:0/3595986790 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18c80731c0 0x7f18c81105a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:50.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.768+0000 7f18ce3ff700 1 -- 172.21.15.101:0/3595986790 >> 172.21.15.101:0/3595986790 conn(0x7f18c80fb5f0 msgr2=0x7f18c80fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:50.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.768+0000 7f18ce3ff700 1 -- 172.21.15.101:0/3595986790 shutdown_connections 2023-07-18T03:21:50.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.769+0000 7f18ce3ff700 1 -- 172.21.15.101:0/3595986790 wait complete. 2023-07-18T03:21:50.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.770+0000 7f18ce3ff700 1 Processor -- start 2023-07-18T03:21:50.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.770+0000 7f18ce3ff700 1 -- start start 2023-07-18T03:21:50.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.774+0000 7f18ce3ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18c80728b0 0x7f18c810bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:50.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.774+0000 7f18ce3ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18c80731c0 0x7f18c810c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:50.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.774+0000 7f18ce3ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18c8110ae0 0x7f18c81c1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:50.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.774+0000 7f18ce3ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c81c1c70 con 0x7f18c80728b0 2023-07-18T03:21:50.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.774+0000 7f18ce3ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c81c1de0 con 0x7f18c80731c0 2023-07-18T03:21:50.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.774+0000 7f18ce3ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c81c1fa0 con 0x7f18c8110ae0 2023-07-18T03:21:50.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.774+0000 7f18c7fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18c80728b0 0x7f18c810bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:50.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.774+0000 7f18c7fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18c80728b0 0x7f18c810bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:41458/0 (socket says 172.21.15.101:41458) 2023-07-18T03:21:50.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.774+0000 7f18c7fff700 1 -- 172.21.15.101:0/2195137846 learned_addr learned my addr 172.21.15.101:0/2195137846 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:50.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.775+0000 7f18cc99c700 1 --2- 172.21.15.101:0/2195137846 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18c8110ae0 0x7f18c81c1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:50.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.775+0000 7f18c77fe700 1 --2- 172.21.15.101:0/2195137846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18c80731c0 0x7f18c810c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:50.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.775+0000 7f18c7fff700 1 -- 172.21.15.101:0/2195137846 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18c8110ae0 msgr2=0x7f18c81c1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:50.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.775+0000 7f18c7fff700 1 --2- 172.21.15.101:0/2195137846 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18c8110ae0 0x7f18c81c1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:50.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.775+0000 7f18c7fff700 1 -- 172.21.15.101:0/2195137846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18c80731c0 msgr2=0x7f18c810c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:50.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.775+0000 7f18c7fff700 1 --2- 172.21.15.101:0/2195137846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18c80731c0 0x7f18c810c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:50.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.775+0000 7f18c7fff700 1 -- 172.21.15.101:0/2195137846 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c001ad10 con 0x7f18c80728b0 2023-07-18T03:21:50.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.776+0000 7f18c7fff700 1 --2- 172.21.15.101:0/2195137846 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18c80728b0 0x7f18c810bde0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f18c0000f80 tx=0x7f18c0004e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:50.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.776+0000 7f18c57fa700 1 -- 172.21.15.101:0/2195137846 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c001f040 con 0x7f18c80728b0 2023-07-18T03:21:50.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.776+0000 7f18c57fa700 1 -- 172.21.15.101:0/2195137846 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18c0018ae0 con 0x7f18c80728b0 2023-07-18T03:21:50.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.776+0000 7f18c57fa700 1 -- 172.21.15.101:0/2195137846 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c0005490 con 0x7f18c80728b0 2023-07-18T03:21:50.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.776+0000 7f18ce3ff700 1 -- 172.21.15.101:0/2195137846 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18c81c2220 con 0x7f18c80728b0 2023-07-18T03:21:50.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.776+0000 7f18ce3ff700 1 -- 172.21.15.101:0/2195137846 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18c81c26e0 con 0x7f18c80728b0 2023-07-18T03:21:50.779 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.784+0000 7f18e1d83b80 1 Processor -- start 2023-07-18T03:21:50.779 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.784+0000 7f18c57fa700 1 -- 172.21.15.101:0/2195137846 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f18c00182a0 con 0x7f18c80728b0 2023-07-18T03:21:50.780 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.784+0000 7f18e1d83b80 1 -- start start 2023-07-18T03:21:50.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.786+0000 7f18e1d83b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559468850350 0x559468850720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:50.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.786+0000 7f18e1d83b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559468850c60 0x55946885b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:50.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.786+0000 7f18e1d83b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55946885bb50 0x55946885df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:50.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.786+0000 7f18e1d83b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559468756d30 con 0x559468850c60 2023-07-18T03:21:50.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.786+0000 7f18e1d83b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594687a9ac0 con 0x55946885bb50 2023-07-18T03:21:50.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.786+0000 7f18e1d83b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594687a2f50 con 0x559468850350 2023-07-18T03:21:50.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.787+0000 7f18c77fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55946885bb50 0x55946885df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:50.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.787+0000 7f18c77fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55946885bb50 0x55946885df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47532/0 (socket says 172.21.15.101:47532) 2023-07-18T03:21:50.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.787+0000 7f18c77fe700 1 -- 172.21.15.101:0/1634003322 learned_addr learned my addr 172.21.15.101:0/1634003322 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:50.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.789+0000 7f18c57fa700 1 --2- 172.21.15.101:0/2195137846 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f18b00a1450 0x7f18b00a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:50.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.789+0000 7f18c77fe700 1 -- 172.21.15.101:0/1634003322 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559468850350 msgr2=0x559468850720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:50.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.789+0000 7f18c77fe700 1 --2- 172.21.15.101:0/1634003322 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559468850350 0x559468850720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:50.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.789+0000 7f18c77fe700 1 -- 172.21.15.101:0/1634003322 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559468850c60 msgr2=0x55946885b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:50.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.790+0000 7f18c7fff700 1 --2- 172.21.15.101:0/1634003322 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559468850c60 0x55946885b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:50.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.790+0000 7f18c57fa700 1 -- 172.21.15.101:0/2195137846 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f18c000d030 con 0x7f18c80728b0 2023-07-18T03:21:50.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.791+0000 7f18c77fe700 1 --2- 172.21.15.101:0/1634003322 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559468850c60 0x55946885b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:50.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.791+0000 7f18c77fe700 1 -- 172.21.15.101:0/1634003322 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594685d0e60 con 0x55946885bb50 2023-07-18T03:21:50.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.791+0000 7f18c7fff700 1 --2- 172.21.15.101:0/1634003322 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559468850c60 0x55946885b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:21:50.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.791+0000 7f18c77fe700 1 --2- 172.21.15.101:0/1634003322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55946885bb50 0x55946885df30 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f18b8009af0 tx=0x7f18b8009e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67ecb6822bd2fa6c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:50.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.792+0000 7f18cc99c700 1 --2- 172.21.15.101:0/2195137846 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f18b00a1450 0x7f18b00a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:50.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.793+0000 7f18ae7fc700 1 -- 172.21.15.101:0/1634003322 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b800eaa0 con 0x55946885bb50 2023-07-18T03:21:50.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.793+0000 7f18ae7fc700 1 -- 172.21.15.101:0/1634003322 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18b8004cf0 con 0x55946885bb50 2023-07-18T03:21:50.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.793+0000 7f18ae7fc700 1 -- 172.21.15.101:0/1634003322 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b8005550 con 0x55946885bb50 2023-07-18T03:21:50.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.794+0000 7f18e1d83b80 1 -- 172.21.15.101:0/1634003322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55946885bb50 msgr2=0x55946885df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:50.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.794+0000 7f18e1d83b80 1 --2- 172.21.15.101:0/1634003322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55946885bb50 0x55946885df30 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f18b8009af0 tx=0x7f18b8009e00 comp rx=0 tx=0).stop 2023-07-18T03:21:50.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.794+0000 7f18cc99c700 1 --2- 172.21.15.101:0/2195137846 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f18b00a1450 0x7f18b00a3900 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f18bc005950 tx=0x7f18bc005f20 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:50.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.796+0000 7f18e1d83b80 1 -- 172.21.15.101:0/1634003322 shutdown_connections 2023-07-18T03:21:50.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.796+0000 7f18e1d83b80 1 --2- 172.21.15.101:0/1634003322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55946885bb50 0x55946885df30 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:50.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.796+0000 7f18e1d83b80 1 --2- 172.21.15.101:0/1634003322 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559468850350 0x559468850720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:50.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.796+0000 7f18e1d83b80 1 --2- 172.21.15.101:0/1634003322 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559468850c60 0x55946885b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:50.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.796+0000 7f18e1d83b80 1 -- 172.21.15.101:0/1634003322 >> 172.21.15.101:0/1634003322 conn(0x559468846e20 msgr2=0x55946884c720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:50.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.801+0000 7f18e1d83b80 1 -- 172.21.15.101:0/1634003322 shutdown_connections 2023-07-18T03:21:50.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.801+0000 7f18e1d83b80 1 -- 172.21.15.101:0/1634003322 wait complete. 2023-07-18T03:21:50.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.802+0000 7f18e1d83b80 1 -- start start 2023-07-18T03:21:50.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.802+0000 7f18e1d83b80 1 Processor -- start 2023-07-18T03:21:50.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.803+0000 7f18e1d83b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559468850350 0x55946885b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:50.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.803+0000 7f18e1d83b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559468850c60 0x559468858880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:50.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.803+0000 7f18e1d83b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55946885bb50 0x559468858fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:50.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.803+0000 7f18e1d83b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559468742b40 con 0x55946885bb50 2023-07-18T03:21:50.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.803+0000 7f18e1d83b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559468742840 con 0x559468850c60 2023-07-18T03:21:50.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.803+0000 7f18e1d83b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594686269a0 con 0x559468850350 2023-07-18T03:21:50.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.803+0000 7f18e1d83b80 10 client.0 _ll_register_callbacks cb 0x5594687407e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:21:50.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.803+0000 7f18c7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559468850350 0x55946885b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:50.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.803+0000 7f18c7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559468850350 0x55946885b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:57136/0 (socket says 172.21.15.101:57136) 2023-07-18T03:21:50.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.803+0000 7f18c7fff700 1 -- 172.21.15.101:0/2788136806 learned_addr learned my addr 172.21.15.101:0/2788136806 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:50.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.804+0000 7f18c77fe700 1 --2- 172.21.15.101:0/2788136806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559468850c60 0x559468858880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:50.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.804+0000 7f18c7fff700 1 -- 172.21.15.101:0/2788136806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559468850c60 msgr2=0x559468858880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:50.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.804+0000 7f18c7fff700 1 --2- 172.21.15.101:0/2788136806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559468850c60 0x559468858880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:50.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.804+0000 7f18c7fff700 1 -- 172.21.15.101:0/2788136806 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55946885bb50 msgr2=0x559468858fc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-18T03:21:50.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.804+0000 7f18c7fff700 1 --2- 172.21.15.101:0/2788136806 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55946885bb50 0x559468858fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:50.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.804+0000 7f18c7fff700 1 -- 172.21.15.101:0/2788136806 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18b8009720 con 0x559468850350 2023-07-18T03:21:50.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.805+0000 7f18c7fff700 1 --2- 172.21.15.101:0/2788136806 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559468850350 0x55946885b600 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f18c0004f50 tx=0x7f18c009a610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a52a610e6df1091 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:50.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.805+0000 7f18a77fe700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:21:50.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.805+0000 7f18a77fe700 1 -- 172.21.15.101:0/2788136806 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c009afe0 con 0x559468850350 2023-07-18T03:21:50.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.805+0000 7f18a77fe700 1 -- 172.21.15.101:0/2788136806 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f18c009ebb0 con 0x559468850350 2023-07-18T03:21:50.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.806+0000 7f18e1d83b80 1 -- 172.21.15.101:0/2788136806 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5594686fc570 con 0x559468850350 2023-07-18T03:21:50.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.806+0000 7f18a77fe700 1 -- 172.21.15.101:0/2788136806 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18c00cb470 con 0x559468850350 2023-07-18T03:21:50.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.806+0000 7f18e1d83b80 10 client.28474 fetch_fsmap learned FSMap version 38 2023-07-18T03:21:50.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.806+0000 7f18e1d83b80 1 -- 172.21.15.101:0/2788136806 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5594685b9ce0 con 0x559468850350 2023-07-18T03:21:50.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.808+0000 7f18a77fe700 1 -- 172.21.15.101:0/2788136806 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5594686fc570 con 0x559468850350 2023-07-18T03:21:50.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.808+0000 7f18a77fe700 1 -- 172.21.15.101:0/2788136806 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 38) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f18c00a2ab0 con 0x559468850350 2023-07-18T03:21:50.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.809+0000 7f18e1d83b80 10 client.28474 fetch_fsmap finished waiting for FSMap version 38 2023-07-18T03:21:50.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.809+0000 7f18e1d83b80 10 client.28474 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:21:50.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.809+0000 7f18e1d83b80 20 client.28474 populate_metadata read hostname 'smithi101' 2023-07-18T03:21:50.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.809+0000 7f18e1d83b80 1 --2- 172.21.15.101:0/2788136806 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55946886b2b0 0x55946886d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:50.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.809+0000 7f18e1d83b80 4 client.28474 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:21:50.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.809+0000 7f18e1d83b80 1 -- 172.21.15.101:0/2788136806 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55946879bf30 con 0x55946886b2b0 2023-07-18T03:21:50.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.809+0000 7f18c77fe700 1 --2- 172.21.15.101:0/2788136806 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55946886b2b0 0x55946886d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:50.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.811+0000 7f18a77fe700 10 client.28474 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:21:50.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.811+0000 7f18c77fe700 1 --2- 172.21.15.101:0/2788136806 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55946886b2b0 0x55946886d6c0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9985e5f2e5623a43 server_cookie=485fe7017eefdbc5 in_seq=0 out_seq=0 2023-07-18T03:21:50.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.813+0000 7f18a77fe700 1 -- 172.21.15.101:0/2788136806 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55946879bf30 con 0x55946886b2b0 2023-07-18T03:21:50.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.813+0000 7f18a77fe700 10 client.28474 handle_command_reply: tid=0 2023-07-18T03:21:50.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.813+0000 7f18e1d83b80 1 client.28474 shutdown 2023-07-18T03:21:50.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.813+0000 7f18e1d83b80 1 -- 172.21.15.101:0/2788136806 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559468850350 msgr2=0x55946885b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:50.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.813+0000 7f18e1d83b80 1 --2- 172.21.15.101:0/2788136806 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559468850350 0x55946885b600 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f18c0004f50 tx=0x7f18c009a610 comp rx=0 tx=0).stop 2023-07-18T03:21:50.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.813+0000 7f18e1d83b80 1 -- 172.21.15.101:0/2788136806 shutdown_connections 2023-07-18T03:21:50.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.813+0000 7f18e1d83b80 1 --2- 172.21.15.101:0/2788136806 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55946886b2b0 0x55946886d6c0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:50.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.813+0000 7f18e1d83b80 1 --2- 172.21.15.101:0/2788136806 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55946885bb50 0x559468858fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:50.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.813+0000 7f18e1d83b80 1 --2- 172.21.15.101:0/2788136806 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559468850350 0x55946885b600 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:50.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.813+0000 7f18e1d83b80 1 --2- 172.21.15.101:0/2788136806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559468850c60 0x559468858880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:50.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.813+0000 7f18e1d83b80 1 -- 172.21.15.101:0/2788136806 >> 172.21.15.101:0/2788136806 conn(0x559468846e20 msgr2=0x55946884c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:50.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.813+0000 7f18a77fe700 0 client.28474 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:21:50.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.814+0000 7f18e1d83b80 1 -- 172.21.15.101:0/2788136806 shutdown_connections 2023-07-18T03:21:50.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.814+0000 7f18e1d83b80 1 -- 172.21.15.101:0/2788136806 wait complete. 2023-07-18T03:21:50.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.814+0000 7f18e1d83b80 20 client.28474 trim_cache size 0 max 16384 2023-07-18T03:21:50.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.842+0000 7f18e1d83b80 1 Processor -- start 2023-07-18T03:21:50.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.842+0000 7f18e1d83b80 1 -- start start 2023-07-18T03:21:50.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.842+0000 7f18e1d83b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559468850350 0x5594688625a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:50.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.842+0000 7f18e1d83b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559468850c60 0x559468852b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:50.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.842+0000 7f18e1d83b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55946885bb50 0x559468858090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:50.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.842+0000 7f18e1d83b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559468859e60 con 0x559468850350 2023-07-18T03:21:50.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.842+0000 7f18e1d83b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559468867f50 con 0x55946885bb50 2023-07-18T03:21:50.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.842+0000 7f18e1d83b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594688469f0 con 0x559468850c60 2023-07-18T03:21:50.839 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.843+0000 7f18cc99c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55946885bb50 0x559468858090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:50.839 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.843+0000 7f18cc99c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55946885bb50 0x559468858090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47548/0 (socket says 172.21.15.101:47548) 2023-07-18T03:21:50.840 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.843+0000 7f18cc99c700 1 -- 172.21.15.101:0/665295397 learned_addr learned my addr 172.21.15.101:0/665295397 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:50.840 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.843+0000 7f18c77fe700 1 --2- 172.21.15.101:0/665295397 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559468850c60 0x559468852b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:50.840 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.843+0000 7f18c7fff700 1 --2- 172.21.15.101:0/665295397 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559468850350 0x5594688625a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:50.840 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.843+0000 7f18c7fff700 1 -- 172.21.15.101:0/665295397 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559468850c60 msgr2=0x559468852b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:50.840 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.843+0000 7f18c7fff700 1 --2- 172.21.15.101:0/665295397 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559468850c60 0x559468852b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:50.840 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.843+0000 7f18c7fff700 1 -- 172.21.15.101:0/665295397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55946885bb50 msgr2=0x559468858090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:50.840 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.843+0000 7f18c7fff700 1 --2- 172.21.15.101:0/665295397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55946885bb50 0x559468858090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:50.841 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.843+0000 7f18c7fff700 1 -- 172.21.15.101:0/665295397 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594685f72d0 con 0x559468850350 2023-07-18T03:21:50.841 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.844+0000 7f18c7fff700 1 --2- 172.21.15.101:0/665295397 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559468850350 0x5594688625a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f18c0099ae0 tx=0x7f18c009acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c62921d7543828e6 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:50.841 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.844+0000 7f18aeffd700 1 -- 172.21.15.101:0/665295397 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c00cb470 con 0x559468850350 2023-07-18T03:21:50.841 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.844+0000 7f18aeffd700 1 -- 172.21.15.101:0/665295397 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18c00cde90 con 0x559468850350 2023-07-18T03:21:50.841 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.844+0000 7f18aeffd700 1 -- 172.21.15.101:0/665295397 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c009a070 con 0x559468850350 2023-07-18T03:21:50.842 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.845+0000 7f18e1d83b80 1 -- 172.21.15.101:0/665295397 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559468850350 msgr2=0x5594688625a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:50.842 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.845+0000 7f18e1d83b80 1 --2- 172.21.15.101:0/665295397 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559468850350 0x5594688625a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f18c0099ae0 tx=0x7f18c009acc0 comp rx=0 tx=0).stop 2023-07-18T03:21:50.843 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.845+0000 7f18e1d83b80 1 -- 172.21.15.101:0/665295397 shutdown_connections 2023-07-18T03:21:50.843 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.845+0000 7f18e1d83b80 1 --2- 172.21.15.101:0/665295397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55946885bb50 0x559468858090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:50.843 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.845+0000 7f18e1d83b80 1 --2- 172.21.15.101:0/665295397 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559468850350 0x5594688625a0 secure :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f18c0099ae0 tx=0x7f18c009acc0 comp rx=0 tx=0).stop 2023-07-18T03:21:50.843 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.845+0000 7f18e1d83b80 1 --2- 172.21.15.101:0/665295397 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559468850c60 0x559468852b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:50.843 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.845+0000 7f18e1d83b80 1 -- 172.21.15.101:0/665295397 >> 172.21.15.101:0/665295397 conn(0x559468846e20 msgr2=0x559468852400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:50.843 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.845+0000 7f18e1d83b80 1 -- 172.21.15.101:0/665295397 shutdown_connections 2023-07-18T03:21:50.843 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.845+0000 7f18e1d83b80 1 -- 172.21.15.101:0/665295397 wait complete. 2023-07-18T03:21:50.843 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.846+0000 7f18e1d83b80 1 -- start start 2023-07-18T03:21:50.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.846+0000 7f18e1d83b80 1 Processor -- start 2023-07-18T03:21:50.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.847+0000 7f18e1d83b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559468850c60 0x559468861190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:50.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.847+0000 7f18e1d83b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55946885bb50 0x5594688616d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:50.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.847+0000 7f18e1d83b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5594688bd130 0x5594688bf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:50.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.847+0000 7f18e1d83b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559468861e50 con 0x5594688bd130 2023-07-18T03:21:50.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.847+0000 7f18e1d83b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594688bfa80 con 0x559468850c60 2023-07-18T03:21:50.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.847+0000 7f18e1d83b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594688bfc40 con 0x55946885bb50 2023-07-18T03:21:50.845 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.847+0000 7f18e1d83b80 10 client.0 _ll_register_callbacks cb 0x7f18c009aa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:21:50.845 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.847+0000 7f18c7fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559468850c60 0x559468861190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:50.845 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.847+0000 7f18c7fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559468850c60 0x559468861190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47560/0 (socket says 172.21.15.101:47560) 2023-07-18T03:21:50.845 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.847+0000 7f18c7fff700 1 -- 172.21.15.101:0/9899142 learned_addr learned my addr 172.21.15.101:0/9899142 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:50.846 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.847+0000 7f18cc99c700 1 --2- 172.21.15.101:0/9899142 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5594688bd130 0x5594688bf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:50.846 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.847+0000 7f18c7fff700 1 -- 172.21.15.101:0/9899142 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55946885bb50 msgr2=0x5594688616d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-18T03:21:50.847 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.847+0000 7f18c7fff700 1 --2- 172.21.15.101:0/9899142 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55946885bb50 0x5594688616d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:50.847 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.847+0000 7f18c7fff700 1 -- 172.21.15.101:0/9899142 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5594688bd130 msgr2=0x5594688bf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:50.847 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.848+0000 7f18c7fff700 1 --2- 172.21.15.101:0/9899142 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5594688bd130 0x5594688bf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:50.847 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.848+0000 7f18c7fff700 1 -- 172.21.15.101:0/9899142 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5594685b9ce0 con 0x559468850c60 2023-07-18T03:21:50.847 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.848+0000 7f18cc99c700 1 --2- 172.21.15.101:0/9899142 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5594688bd130 0x5594688bf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:21:50.847 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.848+0000 7f18c7fff700 1 --2- 172.21.15.101:0/9899142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559468850c60 0x559468861190 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f18c001c860 tx=0x7f18c00cde00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d8367e4c628a44f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:50.847 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.848+0000 7f18a77fe700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:21:50.848 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.848+0000 7f18a77fe700 1 -- 172.21.15.101:0/9899142 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c0099bb0 con 0x559468850c60 2023-07-18T03:21:50.848 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.848+0000 7f18a77fe700 1 -- 172.21.15.101:0/9899142 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f18c00d1070 con 0x559468850c60 2023-07-18T03:21:50.848 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.848+0000 7f18e1d83b80 1 -- 172.21.15.101:0/9899142 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f18b8004cf0 con 0x559468850c60 2023-07-18T03:21:50.848 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.848+0000 7f18a77fe700 1 -- 172.21.15.101:0/9899142 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18c00a2f40 con 0x559468850c60 2023-07-18T03:21:50.848 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.848+0000 7f18e1d83b80 10 client.28454 fetch_fsmap learned FSMap version 38 2023-07-18T03:21:50.848 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.848+0000 7f18e1d83b80 1 -- 172.21.15.101:0/9899142 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5594685cf330 con 0x559468850c60 2023-07-18T03:21:50.848 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.849+0000 7f18a77fe700 1 -- 172.21.15.101:0/9899142 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18b8004cf0 con 0x559468850c60 2023-07-18T03:21:50.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.849+0000 7f18a77fe700 1 -- 172.21.15.101:0/9899142 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 38) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f18c009ba40 con 0x559468850c60 2023-07-18T03:21:50.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.849+0000 7f18e1d83b80 10 client.28454 fetch_fsmap finished waiting for FSMap version 38 2023-07-18T03:21:50.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.849+0000 7f18e1d83b80 10 client.28454 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:21:50.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.849+0000 7f18e1d83b80 20 client.28454 populate_metadata read hostname 'smithi101' 2023-07-18T03:21:50.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.849+0000 7f18e1d83b80 1 --2- 172.21.15.101:0/9899142 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5594688c0eb0 0x5594688c3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:50.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.849+0000 7f18e1d83b80 4 client.28454 mds_command: new command op to 24562 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-18T03:21:50.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.849+0000 7f18e1d83b80 1 -- 172.21.15.101:0/9899142 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55946875e0d0 con 0x5594688c0eb0 2023-07-18T03:21:50.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.849+0000 7f18c77fe700 1 --2- 172.21.15.101:0/9899142 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5594688c0eb0 0x5594688c3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:50.850 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.849+0000 7f18c77fe700 1 --2- 172.21.15.101:0/9899142 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5594688c0eb0 0x5594688c3290 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4a90119346d2460 server_cookie=9b115f7536e12312 in_seq=0 out_seq=0 2023-07-18T03:21:50.850 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.849+0000 7f18a77fe700 10 client.28454 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:21:50.864 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.869+0000 7f18a77fe700 1 -- 172.21.15.101:0/9899142 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55946875e0d0 con 0x5594688c0eb0 2023-07-18T03:21:50.864 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.869+0000 7f18a77fe700 10 client.28454 handle_command_reply: tid=0 2023-07-18T03:21:50.864 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.869+0000 7f18e1d83b80 1 client.28454 shutdown 2023-07-18T03:21:50.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.869+0000 7f18e1d83b80 1 -- 172.21.15.101:0/9899142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559468850c60 msgr2=0x559468861190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:50.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.869+0000 7f18e1d83b80 1 --2- 172.21.15.101:0/9899142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559468850c60 0x559468861190 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f18c001c860 tx=0x7f18c00cde00 comp rx=0 tx=0).stop 2023-07-18T03:21:50.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.869+0000 7f18e1d83b80 1 -- 172.21.15.101:0/9899142 shutdown_connections 2023-07-18T03:21:50.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.869+0000 7f18e1d83b80 1 --2- 172.21.15.101:0/9899142 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5594688c0eb0 0x5594688c3290 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:50.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.869+0000 7f18e1d83b80 1 --2- 172.21.15.101:0/9899142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559468850c60 0x559468861190 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:50.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.869+0000 7f18e1d83b80 1 --2- 172.21.15.101:0/9899142 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5594688bd130 0x5594688bf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:50.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.869+0000 7f18e1d83b80 1 --2- 172.21.15.101:0/9899142 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55946885bb50 0x5594688616d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:50.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.869+0000 7f18e1d83b80 1 -- 172.21.15.101:0/9899142 >> 172.21.15.101:0/9899142 conn(0x559468846e20 msgr2=0x559468852c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:50.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.869+0000 7f18a77fe700 0 client.28454 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:21:50.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.869+0000 7f18e1d83b80 1 -- 172.21.15.101:0/9899142 shutdown_connections 2023-07-18T03:21:50.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.869+0000 7f18e1d83b80 1 -- 172.21.15.101:0/9899142 wait complete. 2023-07-18T03:21:50.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.870+0000 7f18e1d83b80 20 client.28454 trim_cache size 0 max 16384 2023-07-18T03:21:50.867 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:21:50.867 INFO:teuthology.orchestra.run.smithi101.stdout: "return_code": 0, 2023-07-18T03:21:50.867 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_tag": "e5c63d65-eeef-495a-87fe-01ceeeeb36d1", 2023-07-18T03:21:50.867 INFO:teuthology.orchestra.run.smithi101.stdout: "mode": "asynchronous" 2023-07-18T03:21:50.867 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:21:50.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.870+0000 7f18ce3ff700 1 -- 172.21.15.101:0/2195137846 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f18b00a1450 msgr2=0x7f18b00a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:50.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.870+0000 7f18ce3ff700 1 --2- 172.21.15.101:0/2195137846 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f18b00a1450 0x7f18b00a3900 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f18bc005950 tx=0x7f18bc005f20 comp rx=0 tx=0).stop 2023-07-18T03:21:50.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.870+0000 7f18ce3ff700 1 -- 172.21.15.101:0/2195137846 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18c80728b0 msgr2=0x7f18c810bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:50.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.870+0000 7f18ce3ff700 1 --2- 172.21.15.101:0/2195137846 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18c80728b0 0x7f18c810bde0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f18c0000f80 tx=0x7f18c0004e30 comp rx=0 tx=0).stop 2023-07-18T03:21:50.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.871+0000 7f18ce3ff700 1 -- 172.21.15.101:0/2195137846 shutdown_connections 2023-07-18T03:21:50.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.871+0000 7f18ce3ff700 1 --2- 172.21.15.101:0/2195137846 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18c8110ae0 0x7f18c81c1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:50.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.871+0000 7f18ce3ff700 1 --2- 172.21.15.101:0/2195137846 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18c80728b0 0x7f18c810bde0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:50.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.871+0000 7f18ce3ff700 1 --2- 172.21.15.101:0/2195137846 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f18b00a1450 0x7f18b00a3900 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:50.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.871+0000 7f18ce3ff700 1 --2- 172.21.15.101:0/2195137846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18c80731c0 0x7f18c810c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:50.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.871+0000 7f18ce3ff700 1 -- 172.21.15.101:0/2195137846 >> 172.21.15.101:0/2195137846 conn(0x7f18c80fb5f0 msgr2=0x7f18c8112ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:50.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.871+0000 7f18ce3ff700 1 -- 172.21.15.101:0/2195137846 shutdown_connections 2023-07-18T03:21:50.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:50.871+0000 7f18ce3ff700 1 -- 172.21.15.101:0/2195137846 wait complete. 2023-07-18T03:21:50.886 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:21:50.964 INFO:tasks.workunit.client.0.smithi101.stdout: 10 41442 32.16 MB/sec warmup 75 sec latency 762.916 ms 2023-07-18T03:21:51.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.182+0000 7f0e5d1ff700 1 -- 172.21.15.101:0/2371930356 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0e58103e40 msgr2=0x7f0e581042e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:51.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.182+0000 7f0e5d1ff700 1 --2- 172.21.15.101:0/2371930356 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0e58103e40 0x7f0e581042e0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f0e5000b0a0 tx=0x7f0e5000b3b0 comp rx=0 tx=0).stop 2023-07-18T03:21:51.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.182+0000 7f0e5d1ff700 1 -- 172.21.15.101:0/2371930356 shutdown_connections 2023-07-18T03:21:51.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.182+0000 7f0e5d1ff700 1 --2- 172.21.15.101:0/2371930356 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0e58103e40 0x7f0e581042e0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:51.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.182+0000 7f0e5d1ff700 1 --2- 172.21.15.101:0/2371930356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e580fea90 0x7f0e580fee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:51.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.182+0000 7f0e5d1ff700 1 --2- 172.21.15.101:0/2371930356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e580ff3a0 0x7f0e58103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:51.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.182+0000 7f0e5d1ff700 1 -- 172.21.15.101:0/2371930356 >> 172.21.15.101:0/2371930356 conn(0x7f0e580fa4b0 msgr2=0x7f0e580fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:51.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.182+0000 7f0e5d1ff700 1 -- 172.21.15.101:0/2371930356 shutdown_connections 2023-07-18T03:21:51.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.182+0000 7f0e5d1ff700 1 -- 172.21.15.101:0/2371930356 wait complete. 2023-07-18T03:21:51.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.184+0000 7f0e5d1ff700 1 Processor -- start 2023-07-18T03:21:51.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.184+0000 7f0e5d1ff700 1 -- start start 2023-07-18T03:21:51.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.185+0000 7f0e5d1ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0e580fea90 0x7f0e58110670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:51.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.185+0000 7f0e5d1ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e580ff3a0 0x7f0e58110bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:51.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.185+0000 7f0e5d1ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e58103e40 0x7f0e5810b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:51.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.185+0000 7f0e5d1ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5810bea0 con 0x7f0e580fea90 2023-07-18T03:21:51.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.185+0000 7f0e5d1ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5810c010 con 0x7f0e580ff3a0 2023-07-18T03:21:51.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.185+0000 7f0e5d1ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5810c1d0 con 0x7f0e58103e40 2023-07-18T03:21:51.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.186+0000 7f0e56d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0e580fea90 0x7f0e58110670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:51.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.186+0000 7f0e56d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0e580fea90 0x7f0e58110670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:41506/0 (socket says 172.21.15.101:41506) 2023-07-18T03:21:51.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.186+0000 7f0e56d9d700 1 -- 172.21.15.101:0/1294382677 learned_addr learned my addr 172.21.15.101:0/1294382677 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:51.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.186+0000 7f0e5759e700 1 --2- 172.21.15.101:0/1294382677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e58103e40 0x7f0e5810b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:51.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.187+0000 7f0e56d9d700 1 -- 172.21.15.101:0/1294382677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e58103e40 msgr2=0x7f0e5810b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:51.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.187+0000 7f0e56d9d700 1 --2- 172.21.15.101:0/1294382677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e58103e40 0x7f0e5810b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:51.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.187+0000 7f0e56d9d700 1 -- 172.21.15.101:0/1294382677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e580ff3a0 msgr2=0x7f0e58110bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:51.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.187+0000 7f0e56d9d700 1 --2- 172.21.15.101:0/1294382677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e580ff3a0 0x7f0e58110bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:51.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.187+0000 7f0e56d9d700 1 -- 172.21.15.101:0/1294382677 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e50009d00 con 0x7f0e580fea90 2023-07-18T03:21:51.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.187+0000 7f0e56d9d700 1 --2- 172.21.15.101:0/1294382677 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0e580fea90 0x7f0e58110670 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f0e4800c9b0 tx=0x7f0e4800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:51.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.190+0000 7f0e54d99700 1 -- 172.21.15.101:0/1294382677 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e48007b00 con 0x7f0e580fea90 2023-07-18T03:21:51.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.190+0000 7f0e54d99700 1 -- 172.21.15.101:0/1294382677 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e48007c60 con 0x7f0e580fea90 2023-07-18T03:21:51.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.190+0000 7f0e54d99700 1 -- 172.21.15.101:0/1294382677 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e48018660 con 0x7f0e580fea90 2023-07-18T03:21:51.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.190+0000 7f0e5d1ff700 1 -- 172.21.15.101:0/1294382677 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e5810c4b0 con 0x7f0e580fea90 2023-07-18T03:21:51.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.191+0000 7f0e5d1ff700 1 -- 172.21.15.101:0/1294382677 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e5810f5b0 con 0x7f0e580fea90 2023-07-18T03:21:51.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.196+0000 7f0e70b8bb80 1 Processor -- start 2023-07-18T03:21:51.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.196+0000 7f0e70b8bb80 1 -- start start 2023-07-18T03:21:51.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.196+0000 7f0e70b8bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56181e3c62d0 0x56181e3c66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:51.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.197+0000 7f0e70b8bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56181e3c6be0 0x56181e3d1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:51.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.197+0000 7f0e70b8bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56181e3d1ad0 0x56181e3d3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:51.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.197+0000 7f0e70b8bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56181e2cccb0 con 0x56181e3c6be0 2023-07-18T03:21:51.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.197+0000 7f0e70b8bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56181e31fa40 con 0x56181e3d1ad0 2023-07-18T03:21:51.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.197+0000 7f0e70b8bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56181e318ed0 con 0x56181e3c62d0 2023-07-18T03:21:51.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.197+0000 7f0e4ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56181e3d1ad0 0x56181e3d3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:51.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.197+0000 7f0e4ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56181e3d1ad0 0x56181e3d3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47582/0 (socket says 172.21.15.101:47582) 2023-07-18T03:21:51.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.197+0000 7f0e4ffff700 1 -- 172.21.15.101:0/2034747496 learned_addr learned my addr 172.21.15.101:0/2034747496 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:51.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.197+0000 7f0e4ffff700 1 -- 172.21.15.101:0/2034747496 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56181e3c62d0 msgr2=0x56181e3c66a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-18T03:21:51.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.197+0000 7f0e4ffff700 1 --2- 172.21.15.101:0/2034747496 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56181e3c62d0 0x56181e3c66a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:51.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.197+0000 7f0e4ffff700 1 -- 172.21.15.101:0/2034747496 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56181e3c6be0 msgr2=0x56181e3d1590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-18T03:21:51.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.197+0000 7f0e4ffff700 1 --2- 172.21.15.101:0/2034747496 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56181e3c6be0 0x56181e3d1590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:51.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.197+0000 7f0e4ffff700 1 -- 172.21.15.101:0/2034747496 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56181e146de0 con 0x56181e3d1ad0 2023-07-18T03:21:51.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.198+0000 7f0e4ffff700 1 --2- 172.21.15.101:0/2034747496 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56181e3d1ad0 0x56181e3d3eb0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f0e40009bc0 tx=0x7f0e40009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae4f2d0166ed86ec server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:51.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.198+0000 7f0e4d7fa700 1 -- 172.21.15.101:0/2034747496 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e4000eb60 con 0x56181e3d1ad0 2023-07-18T03:21:51.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.199+0000 7f0e4d7fa700 1 -- 172.21.15.101:0/2034747496 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e40004cf0 con 0x56181e3d1ad0 2023-07-18T03:21:51.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.199+0000 7f0e4d7fa700 1 -- 172.21.15.101:0/2034747496 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e40010430 con 0x56181e3d1ad0 2023-07-18T03:21:51.198 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.199+0000 7f0e54d99700 1 -- 172.21.15.101:0/1294382677 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f0e4800f450 con 0x7f0e580fea90 2023-07-18T03:21:51.198 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.200+0000 7f0e70b8bb80 1 -- 172.21.15.101:0/2034747496 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56181e3d1ad0 msgr2=0x56181e3d3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:51.198 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.200+0000 7f0e70b8bb80 1 --2- 172.21.15.101:0/2034747496 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56181e3d1ad0 0x56181e3d3eb0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f0e40009bc0 tx=0x7f0e40009ed0 comp rx=0 tx=0).stop 2023-07-18T03:21:51.198 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.200+0000 7f0e70b8bb80 1 -- 172.21.15.101:0/2034747496 shutdown_connections 2023-07-18T03:21:51.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.200+0000 7f0e70b8bb80 1 --2- 172.21.15.101:0/2034747496 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56181e3d1ad0 0x56181e3d3eb0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:51.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.200+0000 7f0e70b8bb80 1 --2- 172.21.15.101:0/2034747496 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56181e3c62d0 0x56181e3c66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:51.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.200+0000 7f0e70b8bb80 1 --2- 172.21.15.101:0/2034747496 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56181e3c6be0 0x56181e3d1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:51.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.200+0000 7f0e70b8bb80 1 -- 172.21.15.101:0/2034747496 >> 172.21.15.101:0/2034747496 conn(0x56181e3bcda0 msgr2=0x56181e3c26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:51.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.200+0000 7f0e70b8bb80 1 -- 172.21.15.101:0/2034747496 shutdown_connections 2023-07-18T03:21:51.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.200+0000 7f0e70b8bb80 1 -- 172.21.15.101:0/2034747496 wait complete. 2023-07-18T03:21:51.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.201+0000 7f0e70b8bb80 1 -- start start 2023-07-18T03:21:51.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.201+0000 7f0e70b8bb80 1 Processor -- start 2023-07-18T03:21:51.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.202+0000 7f0e70b8bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56181e3c62d0 0x56181e3d14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:51.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.202+0000 7f0e70b8bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56181e3c6be0 0x56181e3ce730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:51.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.202+0000 7f0e70b8bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56181e3d1ad0 0x56181e3ced90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:51.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.202+0000 7f0e70b8bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56181e2b8ac0 con 0x56181e3c6be0 2023-07-18T03:21:51.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.202+0000 7f0e70b8bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56181e2b87c0 con 0x56181e3d1ad0 2023-07-18T03:21:51.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.202+0000 7f0e70b8bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56181e19c920 con 0x56181e3c62d0 2023-07-18T03:21:51.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.202+0000 7f0e70b8bb80 10 client.0 _ll_register_callbacks cb 0x56181e2b6760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:21:51.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.202+0000 7f0e56d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56181e3c6be0 0x56181e3ce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:51.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.203+0000 7f0e56d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56181e3c6be0 0x56181e3ce730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:41512/0 (socket says 172.21.15.101:41512) 2023-07-18T03:21:51.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.203+0000 7f0e56d9d700 1 -- 172.21.15.101:0/1753436556 learned_addr learned my addr 172.21.15.101:0/1753436556 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:51.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.203+0000 7f0e4ffff700 1 --2- 172.21.15.101:0/1753436556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56181e3d1ad0 0x56181e3ced90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:51.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.204+0000 7f0e56d9d700 1 -- 172.21.15.101:0/1753436556 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56181e3c62d0 msgr2=0x56181e3d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:51.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.204+0000 7f0e56d9d700 1 --2- 172.21.15.101:0/1753436556 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56181e3c62d0 0x56181e3d14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:51.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.204+0000 7f0e56d9d700 1 -- 172.21.15.101:0/1753436556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56181e3d1ad0 msgr2=0x56181e3ced90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:51.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.204+0000 7f0e56d9d700 1 --2- 172.21.15.101:0/1753436556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56181e3d1ad0 0x56181e3ced90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:51.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.204+0000 7f0e56d9d700 1 -- 172.21.15.101:0/1753436556 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e400098a0 con 0x56181e3c6be0 2023-07-18T03:21:51.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.204+0000 7f0e56d9d700 1 --2- 172.21.15.101:0/1753436556 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56181e3c6be0 0x56181e3ce730 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f0e480ca150 tx=0x7f0e480d3a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c2a6fabdb6bb773 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:51.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.204+0000 7f0e54d99700 1 --2- 172.21.15.101:0/1294382677 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0e440a13e0 0x7f0e440a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:51.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.204+0000 7f0e3e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:21:51.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.205+0000 7f0e3e7fc700 1 -- 172.21.15.101:0/1753436556 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e480d8070 con 0x56181e3c6be0 2023-07-18T03:21:51.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.205+0000 7f0e3e7fc700 1 -- 172.21.15.101:0/1753436556 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f0e480d1420 con 0x56181e3c6be0 2023-07-18T03:21:51.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.205+0000 7f0e54d99700 1 -- 172.21.15.101:0/1294382677 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f0e480cced0 con 0x7f0e580fea90 2023-07-18T03:21:51.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.205+0000 7f0e70b8bb80 1 -- 172.21.15.101:0/1753436556 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56181e2724f0 con 0x56181e3c6be0 2023-07-18T03:21:51.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.206+0000 7f0e5759e700 1 --2- 172.21.15.101:0/1294382677 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0e440a13e0 0x7f0e440a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:51.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.207+0000 7f0e3e7fc700 1 -- 172.21.15.101:0/1753436556 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e480d32e0 con 0x56181e3c6be0 2023-07-18T03:21:51.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.207+0000 7f0e70b8bb80 10 client.18402 fetch_fsmap learned FSMap version 38 2023-07-18T03:21:51.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.207+0000 7f0e70b8bb80 1 -- 172.21.15.101:0/1753436556 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x56181e12fc60 con 0x56181e3c6be0 2023-07-18T03:21:51.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.207+0000 7f0e3e7fc700 1 -- 172.21.15.101:0/1753436556 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56181e2724f0 con 0x56181e3c6be0 2023-07-18T03:21:51.205 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.208+0000 7f0e3e7fc700 1 -- 172.21.15.101:0/1753436556 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 38) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f0e480d0070 con 0x56181e3c6be0 2023-07-18T03:21:51.205 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.208+0000 7f0e70b8bb80 10 client.18402 fetch_fsmap finished waiting for FSMap version 38 2023-07-18T03:21:51.205 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.208+0000 7f0e70b8bb80 10 client.18402 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:21:51.206 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.208+0000 7f0e70b8bb80 20 client.18402 populate_metadata read hostname 'smithi101' 2023-07-18T03:21:51.206 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.208+0000 7f0e70b8bb80 1 --2- 172.21.15.101:0/1753436556 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56181e3e1270 0x56181e3e3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:51.206 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.208+0000 7f0e70b8bb80 4 client.18402 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:21:51.206 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.208+0000 7f0e70b8bb80 1 -- 172.21.15.101:0/1753436556 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56181e311eb0 con 0x56181e3e1270 2023-07-18T03:21:51.206 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.209+0000 7f0e5759e700 1 --2- 172.21.15.101:0/1294382677 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0e440a13e0 0x7f0e440a3890 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009fd0 tx=0x7f0e50009360 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:51.206 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.209+0000 7f0e4ffff700 1 --2- 172.21.15.101:0/1753436556 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56181e3e1270 0x56181e3e3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:51.207 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.211+0000 7f0e4ffff700 1 --2- 172.21.15.101:0/1753436556 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56181e3e1270 0x56181e3e3680 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=687d82c76e359798 server_cookie=b289c5641bd7ff3b in_seq=0 out_seq=0 2023-07-18T03:21:51.207 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.211+0000 7f0e3e7fc700 10 client.18402 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:21:51.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.216+0000 7f0e3e7fc700 1 -- 172.21.15.101:0/1753436556 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56181e311eb0 con 0x56181e3e1270 2023-07-18T03:21:51.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.216+0000 7f0e3e7fc700 10 client.18402 handle_command_reply: tid=0 2023-07-18T03:21:51.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.216+0000 7f0e70b8bb80 1 client.18402 shutdown 2023-07-18T03:21:51.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.217+0000 7f0e70b8bb80 1 -- 172.21.15.101:0/1753436556 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56181e3c6be0 msgr2=0x56181e3ce730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:51.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.217+0000 7f0e70b8bb80 1 --2- 172.21.15.101:0/1753436556 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56181e3c6be0 0x56181e3ce730 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f0e480ca150 tx=0x7f0e480d3a30 comp rx=0 tx=0).stop 2023-07-18T03:21:51.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.219+0000 7f0e70b8bb80 1 -- 172.21.15.101:0/1753436556 shutdown_connections 2023-07-18T03:21:51.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.219+0000 7f0e70b8bb80 1 --2- 172.21.15.101:0/1753436556 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56181e3e1270 0x56181e3e3680 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:51.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.219+0000 7f0e70b8bb80 1 --2- 172.21.15.101:0/1753436556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56181e3d1ad0 0x56181e3ced90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:51.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.219+0000 7f0e70b8bb80 1 --2- 172.21.15.101:0/1753436556 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56181e3c62d0 0x56181e3d14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:51.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.219+0000 7f0e70b8bb80 1 --2- 172.21.15.101:0/1753436556 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56181e3c6be0 0x56181e3ce730 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:51.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.219+0000 7f0e70b8bb80 1 -- 172.21.15.101:0/1753436556 >> 172.21.15.101:0/1753436556 conn(0x56181e3bcda0 msgr2=0x56181e3c2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:51.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.219+0000 7f0e3e7fc700 0 client.18402 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:21:51.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.219+0000 7f0e70b8bb80 1 -- 172.21.15.101:0/1753436556 shutdown_connections 2023-07-18T03:21:51.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.220+0000 7f0e70b8bb80 1 -- 172.21.15.101:0/1753436556 wait complete. 2023-07-18T03:21:51.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.220+0000 7f0e70b8bb80 20 client.18402 trim_cache size 0 max 16384 2023-07-18T03:21:51.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.247+0000 7f0e70b8bb80 1 Processor -- start 2023-07-18T03:21:51.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.247+0000 7f0e70b8bb80 1 -- start start 2023-07-18T03:21:51.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.247+0000 7f0e70b8bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56181e3c62d0 0x56181e3d1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:51.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.247+0000 7f0e70b8bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56181e3c6be0 0x56181e3cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:51.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.247+0000 7f0e70b8bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56181e3d1ad0 0x56181e3ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:51.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.247+0000 7f0e70b8bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56181e3beec0 con 0x56181e3c62d0 2023-07-18T03:21:51.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.247+0000 7f0e70b8bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56181e3c7540 con 0x56181e3d1ad0 2023-07-18T03:21:51.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.247+0000 7f0e70b8bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56181e3c71c0 con 0x56181e3c6be0 2023-07-18T03:21:51.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.248+0000 7f0e5759e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56181e3d1ad0 0x56181e3ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:51.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.248+0000 7f0e5759e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56181e3d1ad0 0x56181e3ce430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47600/0 (socket says 172.21.15.101:47600) 2023-07-18T03:21:51.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.248+0000 7f0e5759e700 1 -- 172.21.15.101:0/2719691227 learned_addr learned my addr 172.21.15.101:0/2719691227 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:51.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.249+0000 7f0e5759e700 1 -- 172.21.15.101:0/2719691227 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56181e3c6be0 msgr2=0x56181e3cdef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-18T03:21:51.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.249+0000 7f0e5759e700 1 --2- 172.21.15.101:0/2719691227 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56181e3c6be0 0x56181e3cdef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:51.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.249+0000 7f0e5759e700 1 -- 172.21.15.101:0/2719691227 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56181e3c62d0 msgr2=0x56181e3d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:51.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.249+0000 7f0e5759e700 1 --2- 172.21.15.101:0/2719691227 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56181e3c62d0 0x56181e3d1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:51.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.249+0000 7f0e5759e700 1 -- 172.21.15.101:0/2719691227 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56181e16d250 con 0x56181e3d1ad0 2023-07-18T03:21:51.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.249+0000 7f0e5759e700 1 --2- 172.21.15.101:0/2719691227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56181e3d1ad0 0x56181e3ce430 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f0e50007f50 tx=0x7f0e5001c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23a4e81fb073df77 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:51.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.252+0000 7f0e4dffb700 1 -- 172.21.15.101:0/2719691227 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e50023040 con 0x56181e3d1ad0 2023-07-18T03:21:51.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.252+0000 7f0e4dffb700 1 -- 172.21.15.101:0/2719691227 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e500078f0 con 0x56181e3d1ad0 2023-07-18T03:21:51.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.252+0000 7f0e4dffb700 1 -- 172.21.15.101:0/2719691227 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5000e040 con 0x56181e3d1ad0 2023-07-18T03:21:51.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.253+0000 7f0e70b8bb80 1 -- 172.21.15.101:0/2719691227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56181e3d1ad0 msgr2=0x56181e3ce430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:51.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.253+0000 7f0e70b8bb80 1 --2- 172.21.15.101:0/2719691227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56181e3d1ad0 0x56181e3ce430 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f0e50007f50 tx=0x7f0e5001c000 comp rx=0 tx=0).stop 2023-07-18T03:21:51.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.253+0000 7f0e70b8bb80 1 -- 172.21.15.101:0/2719691227 shutdown_connections 2023-07-18T03:21:51.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.253+0000 7f0e70b8bb80 1 --2- 172.21.15.101:0/2719691227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56181e3d1ad0 0x56181e3ce430 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:51.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.253+0000 7f0e70b8bb80 1 --2- 172.21.15.101:0/2719691227 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56181e3c62d0 0x56181e3d1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:51.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.253+0000 7f0e70b8bb80 1 --2- 172.21.15.101:0/2719691227 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56181e3c6be0 0x56181e3cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:51.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.253+0000 7f0e70b8bb80 1 -- 172.21.15.101:0/2719691227 >> 172.21.15.101:0/2719691227 conn(0x56181e3bcda0 msgr2=0x56181e3dd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:51.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.256+0000 7f0e70b8bb80 1 -- 172.21.15.101:0/2719691227 shutdown_connections 2023-07-18T03:21:51.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.261+0000 7f0e70b8bb80 1 -- 172.21.15.101:0/2719691227 wait complete. 2023-07-18T03:21:51.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.261+0000 7f0e70b8bb80 1 -- start start 2023-07-18T03:21:51.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.262+0000 7f0e70b8bb80 1 Processor -- start 2023-07-18T03:21:51.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.262+0000 7f0e70b8bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56181e3c62d0 0x56181e3d6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:51.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.262+0000 7f0e70b8bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56181e3c6be0 0x56181e3d7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:51.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.262+0000 7f0e70b8bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56181e3d1ad0 0x56181e3d77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:51.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.263+0000 7f0e70b8bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56181e435280 con 0x56181e3d1ad0 2023-07-18T03:21:51.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.263+0000 7f0e70b8bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56181e4353f0 con 0x56181e3c6be0 2023-07-18T03:21:51.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.263+0000 7f0e70b8bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56181e4355b0 con 0x56181e3c62d0 2023-07-18T03:21:51.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.263+0000 7f0e70b8bb80 10 client.0 _ll_register_callbacks cb 0x7f0e480d0ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:21:51.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.263+0000 7f0e4ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56181e3c6be0 0x56181e3d7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:51.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.263+0000 7f0e4ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56181e3c6be0 0x56181e3d7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47612/0 (socket says 172.21.15.101:47612) 2023-07-18T03:21:51.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.263+0000 7f0e4ffff700 1 -- 172.21.15.101:0/1104365840 learned_addr learned my addr 172.21.15.101:0/1104365840 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:21:51.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.265+0000 7f0e4ffff700 1 -- 172.21.15.101:0/1104365840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56181e3c62d0 msgr2=0x56181e3d6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:51.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.265+0000 7f0e4ffff700 1 --2- 172.21.15.101:0/1104365840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56181e3c62d0 0x56181e3d6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:51.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.265+0000 7f0e4ffff700 1 -- 172.21.15.101:0/1104365840 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56181e3d1ad0 msgr2=0x56181e3d77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:21:51.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.265+0000 7f0e4ffff700 1 --2- 172.21.15.101:0/1104365840 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56181e3d1ad0 0x56181e3d77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:51.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.265+0000 7f0e4ffff700 1 -- 172.21.15.101:0/1104365840 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e581137c0 con 0x56181e3c6be0 2023-07-18T03:21:51.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.266+0000 7f0e4ffff700 1 --2- 172.21.15.101:0/1104365840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56181e3c6be0 0x56181e3d7260 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f0e40005310 tx=0x7f0e40004f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c30b3ee12270a898 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:21:51.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.269+0000 7f0e3e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:21:51.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.269+0000 7f0e3e7fc700 1 -- 172.21.15.101:0/1104365840 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e400055e0 con 0x56181e3c6be0 2023-07-18T03:21:51.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.269+0000 7f0e3e7fc700 1 -- 172.21.15.101:0/1104365840 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f0e40014070 con 0x56181e3c6be0 2023-07-18T03:21:51.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.275+0000 7f0e70b8bb80 1 -- 172.21.15.101:0/1104365840 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0e40004cf0 con 0x56181e3c6be0 2023-07-18T03:21:51.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.277+0000 7f0e3e7fc700 1 -- 172.21.15.101:0/1104365840 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e40004830 con 0x56181e3c6be0 2023-07-18T03:21:51.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.278+0000 7f0e70b8bb80 10 client.28469 fetch_fsmap learned FSMap version 38 2023-07-18T03:21:51.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.278+0000 7f0e70b8bb80 1 -- 172.21.15.101:0/1104365840 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x56181e1452b0 con 0x56181e3c6be0 2023-07-18T03:21:51.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.279+0000 7f0e3e7fc700 1 -- 172.21.15.101:0/1104365840 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e40004cf0 con 0x56181e3c6be0 2023-07-18T03:21:51.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.279+0000 7f0e3e7fc700 1 -- 172.21.15.101:0/1104365840 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 38) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f0e40004320 con 0x56181e3c6be0 2023-07-18T03:21:51.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.279+0000 7f0e70b8bb80 10 client.28469 fetch_fsmap finished waiting for FSMap version 38 2023-07-18T03:21:51.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.279+0000 7f0e70b8bb80 10 client.28469 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:21:51.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.279+0000 7f0e70b8bb80 20 client.28469 populate_metadata read hostname 'smithi101' 2023-07-18T03:21:51.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.279+0000 7f0e70b8bb80 1 --2- 172.21.15.101:0/1104365840 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56181e458d00 0x56181e45b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:21:51.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.279+0000 7f0e70b8bb80 4 client.28469 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:21:51.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.279+0000 7f0e70b8bb80 1 -- 172.21.15.101:0/1104365840 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56181e2d4050 con 0x56181e458d00 2023-07-18T03:21:51.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.280+0000 7f0e56d9d700 1 --2- 172.21.15.101:0/1104365840 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56181e458d00 0x56181e45b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:21:51.284 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.289+0000 7f0e56d9d700 1 --2- 172.21.15.101:0/1104365840 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56181e458d00 0x56181e45b110 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f770113bdf4502e server_cookie=cba9bb40b9e5838a in_seq=0 out_seq=0 2023-07-18T03:21:51.284 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.289+0000 7f0e3e7fc700 10 client.28469 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:21:51.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.291+0000 7f0e3e7fc700 1 -- 172.21.15.101:0/1104365840 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x56181e2d4050 con 0x56181e458d00 2023-07-18T03:21:51.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.291+0000 7f0e3e7fc700 10 client.28469 handle_command_reply: tid=0 2023-07-18T03:21:51.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.291+0000 7f0e70b8bb80 1 client.28469 shutdown 2023-07-18T03:21:51.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.293+0000 7f0e70b8bb80 1 -- 172.21.15.101:0/1104365840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56181e3c6be0 msgr2=0x56181e3d7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:51.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.293+0000 7f0e70b8bb80 1 --2- 172.21.15.101:0/1104365840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56181e3c6be0 0x56181e3d7260 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f0e40005310 tx=0x7f0e40004f10 comp rx=0 tx=0).stop 2023-07-18T03:21:51.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.293+0000 7f0e70b8bb80 1 -- 172.21.15.101:0/1104365840 shutdown_connections 2023-07-18T03:21:51.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.293+0000 7f0e70b8bb80 1 --2- 172.21.15.101:0/1104365840 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56181e458d00 0x56181e45b110 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:51.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.293+0000 7f0e70b8bb80 1 --2- 172.21.15.101:0/1104365840 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56181e3d1ad0 0x56181e3d77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:51.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.293+0000 7f0e70b8bb80 1 --2- 172.21.15.101:0/1104365840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56181e3c62d0 0x56181e3d6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:51.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.293+0000 7f0e70b8bb80 1 --2- 172.21.15.101:0/1104365840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56181e3c6be0 0x56181e3d7260 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:51.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.293+0000 7f0e70b8bb80 1 -- 172.21.15.101:0/1104365840 >> 172.21.15.101:0/1104365840 conn(0x56181e3bcda0 msgr2=0x56181e3c81a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:51.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.294+0000 7f0e3e7fc700 0 client.28469 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:21:51.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.294+0000 7f0e70b8bb80 1 -- 172.21.15.101:0/1104365840 shutdown_connections 2023-07-18T03:21:51.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.294+0000 7f0e70b8bb80 1 -- 172.21.15.101:0/1104365840 wait complete. 2023-07-18T03:21:51.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.296+0000 7f0e70b8bb80 20 client.28469 trim_cache size 0 max 16384 2023-07-18T03:21:51.292 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:21:51.292 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "scrub active (2 inodes in the stack)", 2023-07-18T03:21:51.292 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:21:51.292 INFO:teuthology.orchestra.run.smithi101.stdout: "e5c63d65-eeef-495a-87fe-01ceeeeb36d1": { 2023-07-18T03:21:51.292 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:21:51.292 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "e5c63d65-eeef-495a-87fe-01ceeeeb36d1", 2023-07-18T03:21:51.293 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:21:51.293 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:21:51.293 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:21:51.293 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:21:51.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.297+0000 7f0e5d1ff700 1 -- 172.21.15.101:0/1294382677 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0e440a13e0 msgr2=0x7f0e440a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:51.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.297+0000 7f0e5d1ff700 1 --2- 172.21.15.101:0/1294382677 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0e440a13e0 0x7f0e440a3890 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009fd0 tx=0x7f0e50009360 comp rx=0 tx=0).stop 2023-07-18T03:21:51.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.297+0000 7f0e5d1ff700 1 -- 172.21.15.101:0/1294382677 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0e580fea90 msgr2=0x7f0e58110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:21:51.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.297+0000 7f0e5d1ff700 1 --2- 172.21.15.101:0/1294382677 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0e580fea90 0x7f0e58110670 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f0e4800c9b0 tx=0x7f0e4800ccc0 comp rx=0 tx=0).stop 2023-07-18T03:21:51.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.297+0000 7f0e5d1ff700 1 -- 172.21.15.101:0/1294382677 shutdown_connections 2023-07-18T03:21:51.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.297+0000 7f0e5d1ff700 1 --2- 172.21.15.101:0/1294382677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e58103e40 0x7f0e5810b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:51.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.297+0000 7f0e5d1ff700 1 --2- 172.21.15.101:0/1294382677 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0e580fea90 0x7f0e58110670 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:51.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.297+0000 7f0e5d1ff700 1 --2- 172.21.15.101:0/1294382677 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0e440a13e0 0x7f0e440a3890 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:51.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.297+0000 7f0e5d1ff700 1 --2- 172.21.15.101:0/1294382677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e580ff3a0 0x7f0e58110bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:21:51.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.297+0000 7f0e5d1ff700 1 -- 172.21.15.101:0/1294382677 >> 172.21.15.101:0/1294382677 conn(0x7f0e580fa4b0 msgr2=0x7f0e58101920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:21:51.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.300+0000 7f0e5d1ff700 1 -- 172.21.15.101:0/1294382677 shutdown_connections 2023-07-18T03:21:51.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:21:51.302+0000 7f0e5d1ff700 1 -- 172.21.15.101:0/1294382677 wait complete. 2023-07-18T03:21:51.341 INFO:tasks.cephfs.filesystem:scrub status for tag:e5c63d65-eeef-495a-87fe-01ceeeeb36d1 - {'path': '/', 'tag': 'e5c63d65-eeef-495a-87fe-01ceeeeb36d1', 'options': 'recursive,force'} 2023-07-18T03:21:51.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:51 smithi155 ceph-mon[128885]: pgmap v447: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 772 op/s 2023-07-18T03:21:51.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:51 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:21:51.705 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:51 smithi179 ceph-mon[129229]: pgmap v447: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 772 op/s 2023-07-18T03:21:51.705 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:51 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:21:51.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:51 smithi101 ceph-mon[126379]: pgmap v447: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 772 op/s 2023-07-18T03:21:51.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:51 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:21:51.965 INFO:tasks.workunit.client.0.smithi101.stdout: 10 41808 31.99 MB/sec warmup 76 sec latency 264.105 ms 2023-07-18T03:21:52.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:52 smithi155 ceph-mon[128885]: scrub queued for path: / 2023-07-18T03:21:52.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:52 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:21:52.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:52 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:21:52.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:52 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:21:52.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:52 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:21:52.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:52 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:21:52.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:52 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:21:52.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:52 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:21:52.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:52 smithi179 ceph-mon[129229]: scrub queued for path: / 2023-07-18T03:21:52.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:52 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:21:52.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:52 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:21:52.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:52 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:21:52.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:52 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:21:52.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:52 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:21:52.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:52 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:21:52.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:52 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:21:52.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:52 smithi101 ceph-mon[126379]: scrub queued for path: / 2023-07-18T03:21:52.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:52 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:21:52.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:52 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:21:52.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:52 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:21:52.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:52 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:21:52.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:52 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:21:52.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:52 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:21:52.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:52 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:21:52.964 INFO:tasks.workunit.client.0.smithi101.stdout: 10 42392 31.96 MB/sec warmup 77 sec latency 128.767 ms 2023-07-18T03:21:53.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:53 smithi155 ceph-mon[128885]: pgmap v448: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 16 MiB/s wr, 707 op/s 2023-07-18T03:21:53.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:53 smithi179 ceph-mon[129229]: pgmap v448: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 16 MiB/s wr, 707 op/s 2023-07-18T03:21:53.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:53 smithi101 ceph-mon[126379]: pgmap v448: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 16 MiB/s wr, 707 op/s 2023-07-18T03:21:53.965 INFO:tasks.workunit.client.0.smithi101.stdout: 10 43194 32.01 MB/sec warmup 78 sec latency 45.795 ms 2023-07-18T03:21:54.965 INFO:tasks.workunit.client.0.smithi101.stdout: 10 43787 32.06 MB/sec warmup 79 sec latency 72.284 ms 2023-07-18T03:21:55.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:55 smithi179 ceph-mon[129229]: pgmap v449: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 16 MiB/s wr, 707 op/s 2023-07-18T03:21:55.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:55 smithi101 ceph-mon[126379]: pgmap v449: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 16 MiB/s wr, 707 op/s 2023-07-18T03:21:55.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:55 smithi155 ceph-mon[128885]: pgmap v449: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 16 MiB/s wr, 707 op/s 2023-07-18T03:21:55.965 INFO:tasks.workunit.client.0.smithi101.stdout: 10 44667 32.19 MB/sec warmup 80 sec latency 66.943 ms 2023-07-18T03:21:56.965 INFO:tasks.workunit.client.0.smithi101.stdout: 10 45539 32.46 MB/sec warmup 81 sec latency 61.262 ms 2023-07-18T03:21:57.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:57 smithi179 ceph-mon[129229]: pgmap v450: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 23 MiB/s wr, 1.03k op/s 2023-07-18T03:21:57.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:57 smithi101 ceph-mon[126379]: pgmap v450: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 23 MiB/s wr, 1.03k op/s 2023-07-18T03:21:57.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:57 smithi155 ceph-mon[128885]: pgmap v450: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 23 MiB/s wr, 1.03k op/s 2023-07-18T03:21:57.965 INFO:tasks.workunit.client.0.smithi101.stdout: 10 46361 32.62 MB/sec warmup 82 sec latency 63.402 ms 2023-07-18T03:21:58.965 INFO:tasks.workunit.client.0.smithi101.stdout: 10 46914 32.55 MB/sec warmup 83 sec latency 88.441 ms 2023-07-18T03:21:59.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:21:59 smithi179 ceph-mon[129229]: pgmap v451: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 18 MiB/s wr, 786 op/s 2023-07-18T03:21:59.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:21:59 smithi101 ceph-mon[126379]: pgmap v451: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 18 MiB/s wr, 786 op/s 2023-07-18T03:21:59.550 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:21:59 smithi155 ceph-mon[128885]: pgmap v451: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 18 MiB/s wr, 786 op/s 2023-07-18T03:21:59.965 INFO:tasks.workunit.client.0.smithi101.stdout: 10 47649 32.58 MB/sec warmup 84 sec latency 76.047 ms 2023-07-18T03:22:00.965 INFO:tasks.workunit.client.0.smithi101.stdout: 10 48175 32.50 MB/sec warmup 85 sec latency 267.948 ms 2023-07-18T03:22:01.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:01 smithi179 ceph-mon[129229]: pgmap v452: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 20 MiB/s wr, 970 op/s 2023-07-18T03:22:01.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:01 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:22:01.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:01 smithi101 ceph-mon[126379]: pgmap v452: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 20 MiB/s wr, 970 op/s 2023-07-18T03:22:01.492 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:01 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:22:01.550 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:01 smithi155 ceph-mon[128885]: pgmap v452: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 20 MiB/s wr, 970 op/s 2023-07-18T03:22:01.550 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:01 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:22:01.969 INFO:tasks.workunit.client.0.smithi101.stdout: 10 48377 32.18 MB/sec warmup 86 sec latency 264.879 ms 2023-07-18T03:22:02.969 INFO:tasks.workunit.client.0.smithi101.stdout: 10 48391 31.85 MB/sec warmup 87 sec latency 341.705 ms 2023-07-18T03:22:03.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:03 smithi179 ceph-mon[129229]: pgmap v453: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 13 MiB/s wr, 694 op/s 2023-07-18T03:22:03.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:03 smithi101 ceph-mon[126379]: pgmap v453: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 13 MiB/s wr, 694 op/s 2023-07-18T03:22:03.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:03 smithi155 ceph-mon[128885]: pgmap v453: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 13 MiB/s wr, 694 op/s 2023-07-18T03:22:03.969 INFO:tasks.workunit.client.0.smithi101.stdout: 10 48537 31.57 MB/sec warmup 88 sec latency 297.962 ms 2023-07-18T03:22:04.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:04 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:22:04.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:04 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:22:04.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:04 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:22:04.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:04 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:22:04.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:04 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:22:04.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:04 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:22:04.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:04 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:22:04.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:04 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:22:04.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:04 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:22:04.969 INFO:tasks.workunit.client.0.smithi101.stdout: 10 49233 31.79 MB/sec warmup 89 sec latency 265.234 ms 2023-07-18T03:22:05.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:05 smithi179 ceph-mon[129229]: pgmap v454: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 9.3 MiB/s wr, 536 op/s 2023-07-18T03:22:05.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:05 smithi101 ceph-mon[126379]: pgmap v454: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 9.3 MiB/s wr, 536 op/s 2023-07-18T03:22:05.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:05 smithi155 ceph-mon[128885]: pgmap v454: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 9.3 MiB/s wr, 536 op/s 2023-07-18T03:22:05.969 INFO:tasks.workunit.client.0.smithi101.stdout: 10 49899 31.86 MB/sec warmup 90 sec latency 458.558 ms 2023-07-18T03:22:06.969 INFO:tasks.workunit.client.0.smithi101.stdout: 10 50562 31.81 MB/sec warmup 91 sec latency 1458.607 ms 2023-07-18T03:22:07.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:07 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:22:07.454 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:07 smithi179 ceph-mon[129229]: pgmap v455: 129 pgs: 129 active+clean; 1.2 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 12 MiB/s wr, 853 op/s 2023-07-18T03:22:07.454 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:07 smithi179 ceph-mon[129229]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:22:07.454 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:07 smithi179 ceph-mon[129229]: osdmap e109: 12 total, 12 up, 12 in 2023-07-18T03:22:07.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:07 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:22:07.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:07 smithi101 ceph-mon[126379]: pgmap v455: 129 pgs: 129 active+clean; 1.2 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 12 MiB/s wr, 853 op/s 2023-07-18T03:22:07.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:07 smithi101 ceph-mon[126379]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:22:07.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:07 smithi101 ceph-mon[126379]: osdmap e109: 12 total, 12 up, 12 in 2023-07-18T03:22:07.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:07 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:22:07.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:07 smithi155 ceph-mon[128885]: pgmap v455: 129 pgs: 129 active+clean; 1.2 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 12 MiB/s wr, 853 op/s 2023-07-18T03:22:07.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:07 smithi155 ceph-mon[128885]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:22:07.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:07 smithi155 ceph-mon[128885]: osdmap e109: 12 total, 12 up, 12 in 2023-07-18T03:22:07.969 INFO:tasks.workunit.client.0.smithi101.stdout: 10 51277 31.92 MB/sec warmup 92 sec latency 2458.655 ms 2023-07-18T03:22:08.969 INFO:tasks.workunit.client.0.smithi101.stdout: 10 51904 31.87 MB/sec warmup 93 sec latency 3458.709 ms 2023-07-18T03:22:09.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:09 smithi179 ceph-mon[129229]: pgmap v457: 129 pgs: 129 active+clean; 1.2 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.5 MiB/s wr, 634 op/s 2023-07-18T03:22:09.492 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:09 smithi101 ceph-mon[126379]: pgmap v457: 129 pgs: 129 active+clean; 1.2 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.5 MiB/s wr, 634 op/s 2023-07-18T03:22:09.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:09 smithi155 ceph-mon[128885]: pgmap v457: 129 pgs: 129 active+clean; 1.2 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.5 MiB/s wr, 634 op/s 2023-07-18T03:22:09.969 INFO:tasks.workunit.client.0.smithi101.stdout: 10 52712 32.02 MB/sec warmup 94 sec latency 4458.757 ms 2023-07-18T03:22:10.969 INFO:tasks.workunit.client.0.smithi101.stdout: 10 53512 32.18 MB/sec warmup 95 sec latency 5458.806 ms 2023-07-18T03:22:11.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:11 smithi179 ceph-mon[129229]: pgmap v458: 129 pgs: 129 active+clean; 1.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 5.8 MiB/s wr, 563 op/s 2023-07-18T03:22:11.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:11 smithi101 ceph-mon[126379]: pgmap v458: 129 pgs: 129 active+clean; 1.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 5.8 MiB/s wr, 563 op/s 2023-07-18T03:22:11.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:11 smithi155 ceph-mon[128885]: pgmap v458: 129 pgs: 129 active+clean; 1.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 5.8 MiB/s wr, 563 op/s 2023-07-18T03:22:11.970 INFO:tasks.workunit.client.0.smithi101.stdout: 10 54211 32.30 MB/sec warmup 96 sec latency 6458.938 ms 2023-07-18T03:22:12.970 INFO:tasks.workunit.client.0.smithi101.stdout: 10 54841 32.27 MB/sec warmup 97 sec latency 7458.987 ms 2023-07-18T03:22:13.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:13 smithi179 ceph-mon[129229]: pgmap v459: 129 pgs: 129 active+clean; 1.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.8 MiB/s wr, 627 op/s 2023-07-18T03:22:13.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:13 smithi101 ceph-mon[126379]: pgmap v459: 129 pgs: 129 active+clean; 1.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.8 MiB/s wr, 627 op/s 2023-07-18T03:22:13.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:13 smithi155 ceph-mon[128885]: pgmap v459: 129 pgs: 129 active+clean; 1.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.8 MiB/s wr, 627 op/s 2023-07-18T03:22:13.970 INFO:tasks.workunit.client.0.smithi101.stdout: 10 55564 32.21 MB/sec warmup 98 sec latency 8459.028 ms 2023-07-18T03:22:14.970 INFO:tasks.workunit.client.0.smithi101.stdout: 10 56389 32.43 MB/sec warmup 99 sec latency 9459.142 ms 2023-07-18T03:22:15.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:15 smithi179 ceph-mon[129229]: pgmap v460: 129 pgs: 129 active+clean; 1.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.8 MiB/s wr, 627 op/s 2023-07-18T03:22:15.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:15 smithi179 ceph-mon[129229]: osdmap e110: 12 total, 12 up, 12 in 2023-07-18T03:22:15.492 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:15 smithi101 ceph-mon[126379]: pgmap v460: 129 pgs: 129 active+clean; 1.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.8 MiB/s wr, 627 op/s 2023-07-18T03:22:15.492 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:15 smithi101 ceph-mon[126379]: osdmap e110: 12 total, 12 up, 12 in 2023-07-18T03:22:15.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:15 smithi155 ceph-mon[128885]: pgmap v460: 129 pgs: 129 active+clean; 1.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.8 MiB/s wr, 627 op/s 2023-07-18T03:22:15.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:15 smithi155 ceph-mon[128885]: osdmap e110: 12 total, 12 up, 12 in 2023-07-18T03:22:15.970 INFO:tasks.workunit.client.0.smithi101.stdout: 10 57128 32.50 MB/sec warmup 100 sec latency 10331.454 ms 2023-07-18T03:22:16.454 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:16 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:22:16.454 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:16 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:22:16.454 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:16 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:22:16.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:16 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:22:16.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:16 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:22:16.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:16 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:22:16.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:16 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:22:16.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:16 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:22:16.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:16 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:22:16.972 INFO:tasks.workunit.client.0.smithi101.stdout: 10 57270 32.24 MB/sec warmup 101 sec latency 1603.336 ms 2023-07-18T03:22:17.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:17 smithi179 ceph-mon[129229]: pgmap v462: 129 pgs: 129 active+clean; 1.2 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 7.2 MiB/s wr, 528 op/s 2023-07-18T03:22:17.494 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:17 smithi101 ceph-mon[126379]: pgmap v462: 129 pgs: 129 active+clean; 1.2 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 7.2 MiB/s wr, 528 op/s 2023-07-18T03:22:17.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:17 smithi155 ceph-mon[128885]: pgmap v462: 129 pgs: 129 active+clean; 1.2 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 7.2 MiB/s wr, 528 op/s 2023-07-18T03:22:17.970 INFO:tasks.workunit.client.0.smithi101.stdout: 10 57443 32.02 MB/sec warmup 102 sec latency 2603.397 ms 2023-07-18T03:22:18.970 INFO:tasks.workunit.client.0.smithi101.stdout: 10 57770 31.91 MB/sec warmup 103 sec latency 3603.446 ms 2023-07-18T03:22:19.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:19 smithi179 ceph-mon[129229]: pgmap v463: 129 pgs: 129 active+clean; 1.2 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 7.0 MiB/s wr, 509 op/s 2023-07-18T03:22:19.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:19 smithi101 ceph-mon[126379]: pgmap v463: 129 pgs: 129 active+clean; 1.2 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 7.0 MiB/s wr, 509 op/s 2023-07-18T03:22:19.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:19 smithi155 ceph-mon[128885]: pgmap v463: 129 pgs: 129 active+clean; 1.2 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 7.0 MiB/s wr, 509 op/s 2023-07-18T03:22:19.971 INFO:tasks.workunit.client.0.smithi101.stdout: 10 58643 32.06 MB/sec warmup 104 sec latency 4603.493 ms 2023-07-18T03:22:20.972 INFO:tasks.workunit.client.0.smithi101.stdout: 10 59389 32.11 MB/sec warmup 105 sec latency 5604.622 ms 2023-07-18T03:22:21.342 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:22:21.420 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:21 smithi101 ceph-mon[126379]: pgmap v464: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 12 MiB/s wr, 655 op/s 2023-07-18T03:22:21.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:21 smithi179 ceph-mon[129229]: pgmap v464: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 12 MiB/s wr, 655 op/s 2023-07-18T03:22:21.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:21 smithi155 ceph-mon[128885]: pgmap v464: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 12 MiB/s wr, 655 op/s 2023-07-18T03:22:21.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.608+0000 7f32c9bff700 1 -- 172.21.15.101:0/2627300470 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32c4100480 msgr2=0x7f32c410c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:21.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.608+0000 7f32c9bff700 1 --2- 172.21.15.101:0/2627300470 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32c4100480 0x7f32c410c910 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f32bc00b0a0 tx=0x7f32bc00b3b0 comp rx=0 tx=0).stop 2023-07-18T03:22:21.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.608+0000 7f32c9bff700 1 -- 172.21.15.101:0/2627300470 shutdown_connections 2023-07-18T03:22:21.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.608+0000 7f32c9bff700 1 --2- 172.21.15.101:0/2627300470 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32c4100480 0x7f32c410c910 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:21.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.608+0000 7f32c9bff700 1 --2- 172.21.15.101:0/2627300470 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f32c4105ae0 0x7f32c4105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:21.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.608+0000 7f32c9bff700 1 --2- 172.21.15.101:0/2627300470 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32c40ffad0 0x7f32c40fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:21.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.608+0000 7f32c9bff700 1 -- 172.21.15.101:0/2627300470 >> 172.21.15.101:0/2627300470 conn(0x7f32c40fb5f0 msgr2=0x7f32c40fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:21.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.608+0000 7f32c9bff700 1 -- 172.21.15.101:0/2627300470 shutdown_connections 2023-07-18T03:22:21.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.609+0000 7f32c9bff700 1 -- 172.21.15.101:0/2627300470 wait complete. 2023-07-18T03:22:21.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.610+0000 7f32c9bff700 1 Processor -- start 2023-07-18T03:22:21.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.610+0000 7f32c9bff700 1 -- start start 2023-07-18T03:22:21.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.610+0000 7f32c9bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f32c40ffad0 0x7f32c4072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:21.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.610+0000 7f32c9bff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32c4100480 0x7f32c4072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:21.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.610+0000 7f32c9bff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32c4105ae0 0x7f32c41a3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:21.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.610+0000 7f32c9bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c4073580 con 0x7f32c40ffad0 2023-07-18T03:22:21.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.610+0000 7f32c9bff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c41a4440 con 0x7f32c4100480 2023-07-18T03:22:21.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.610+0000 7f32c9bff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c41a4580 con 0x7f32c4105ae0 2023-07-18T03:22:21.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.610+0000 7f32c37fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f32c40ffad0 0x7f32c4072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:21.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.610+0000 7f32c2ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32c4100480 0x7f32c4072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:21.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.611+0000 7f32c37fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f32c40ffad0 0x7f32c4072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:37244/0 (socket says 172.21.15.101:37244) 2023-07-18T03:22:21.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.611+0000 7f32c37fe700 1 -- 172.21.15.101:0/2298340872 learned_addr learned my addr 172.21.15.101:0/2298340872 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:21.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.611+0000 7f32c3fff700 1 --2- 172.21.15.101:0/2298340872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32c4105ae0 0x7f32c41a3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:21.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.611+0000 7f32c37fe700 1 -- 172.21.15.101:0/2298340872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32c4105ae0 msgr2=0x7f32c41a3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:21.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.611+0000 7f32c37fe700 1 --2- 172.21.15.101:0/2298340872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32c4105ae0 0x7f32c41a3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:21.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.611+0000 7f32c37fe700 1 -- 172.21.15.101:0/2298340872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32c4100480 msgr2=0x7f32c4072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:21.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.611+0000 7f32c37fe700 1 --2- 172.21.15.101:0/2298340872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32c4100480 0x7f32c4072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:21.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.611+0000 7f32c37fe700 1 -- 172.21.15.101:0/2298340872 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32bc009d00 con 0x7f32c40ffad0 2023-07-18T03:22:21.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.611+0000 7f32c37fe700 1 --2- 172.21.15.101:0/2298340872 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f32c40ffad0 0x7f32c4072810 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f32b400e9a0 tx=0x7f32b400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:21.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.612+0000 7f32c0ff9700 1 -- 172.21.15.101:0/2298340872 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b4009790 con 0x7f32c40ffad0 2023-07-18T03:22:21.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.612+0000 7f32c9bff700 1 -- 172.21.15.101:0/2298340872 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32c41a47e0 con 0x7f32c40ffad0 2023-07-18T03:22:21.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.612+0000 7f32c0ff9700 1 -- 172.21.15.101:0/2298340872 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32b4010430 con 0x7f32c40ffad0 2023-07-18T03:22:21.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.612+0000 7f32c0ff9700 1 -- 172.21.15.101:0/2298340872 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b4018680 con 0x7f32c40ffad0 2023-07-18T03:22:21.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.612+0000 7f32c9bff700 1 -- 172.21.15.101:0/2298340872 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32c41a4d10 con 0x7f32c40ffad0 2023-07-18T03:22:21.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.616+0000 7f32c0ff9700 1 -- 172.21.15.101:0/2298340872 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f32b40105a0 con 0x7f32c40ffad0 2023-07-18T03:22:21.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.617+0000 7f32dd478b80 1 Processor -- start 2023-07-18T03:22:21.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.618+0000 7f32dd478b80 1 -- start start 2023-07-18T03:22:21.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.618+0000 7f32dd478b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574fd17d2d0 0x5574fd17d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:21.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.618+0000 7f32dd478b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574fd17dbe0 0x5574fd188590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:21.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.618+0000 7f32c0ff9700 1 --2- 172.21.15.101:0/2298340872 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f32ac0a12c0 0x7f32ac0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:21.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.618+0000 7f32dd478b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574fd188ad0 0x5574fd18aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:21.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.618+0000 7f32dd478b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574fd083cb0 con 0x5574fd17dbe0 2023-07-18T03:22:21.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.618+0000 7f32dd478b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574fd0d6a40 con 0x5574fd17d2d0 2023-07-18T03:22:21.616 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.618+0000 7f32dd478b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574fd0cfed0 con 0x5574fd188ad0 2023-07-18T03:22:21.616 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.618+0000 7f32c0ff9700 1 -- 172.21.15.101:0/2298340872 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f32b4014070 con 0x7f32c40ffad0 2023-07-18T03:22:21.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.619+0000 7f32c3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574fd188ad0 0x5574fd18aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:21.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.619+0000 7f32c3fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574fd17d2d0 0x5574fd17d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:21.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.619+0000 7f32c3fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574fd17d2d0 0x5574fd17d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47376/0 (socket says 172.21.15.101:47376) 2023-07-18T03:22:21.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.619+0000 7f32c3fff700 1 -- 172.21.15.101:0/3005969512 learned_addr learned my addr 172.21.15.101:0/3005969512 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:21.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.619+0000 7f32c2ffd700 1 --2- 172.21.15.101:0/2298340872 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f32ac0a12c0 0x7f32ac0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:21.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.619+0000 7f32c37fe700 1 --2- 172.21.15.101:0/3005969512 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574fd17dbe0 0x5574fd188590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:21.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.620+0000 7f32c3fff700 1 -- 172.21.15.101:0/3005969512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574fd188ad0 msgr2=0x5574fd18aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:21.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.620+0000 7f32c3fff700 1 --2- 172.21.15.101:0/3005969512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574fd188ad0 0x5574fd18aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:21.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.620+0000 7f32c3fff700 1 -- 172.21.15.101:0/3005969512 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574fd17dbe0 msgr2=0x5574fd188590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:21.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.620+0000 7f32c3fff700 1 --2- 172.21.15.101:0/3005969512 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574fd17dbe0 0x5574fd188590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:21.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.620+0000 7f32c3fff700 1 -- 172.21.15.101:0/3005969512 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574fcefdde0 con 0x5574fd17d2d0 2023-07-18T03:22:21.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.620+0000 7f32c37fe700 1 --2- 172.21.15.101:0/3005969512 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574fd17dbe0 0x5574fd188590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:22:21.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.620+0000 7f32c3fff700 1 --2- 172.21.15.101:0/3005969512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574fd17d2d0 0x5574fd17d6a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f32bc00b070 tx=0x7f32bc006230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=969aa0d4a901a10 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:21.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.620+0000 7f32c2ffd700 1 --2- 172.21.15.101:0/2298340872 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f32ac0a12c0 0x7f32ac0a3770 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f32c4074000 tx=0x7f32b8009190 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:21.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.620+0000 7f32a9ffb700 1 -- 172.21.15.101:0/3005969512 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32bc00e040 con 0x5574fd17d2d0 2023-07-18T03:22:21.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.620+0000 7f32a9ffb700 1 -- 172.21.15.101:0/3005969512 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32bc00bd60 con 0x5574fd17d2d0 2023-07-18T03:22:21.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.620+0000 7f32a9ffb700 1 -- 172.21.15.101:0/3005969512 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32bc01e460 con 0x5574fd17d2d0 2023-07-18T03:22:21.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.621+0000 7f32dd478b80 1 -- 172.21.15.101:0/3005969512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574fd17d2d0 msgr2=0x5574fd17d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:21.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.621+0000 7f32dd478b80 1 --2- 172.21.15.101:0/3005969512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574fd17d2d0 0x5574fd17d6a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f32bc00b070 tx=0x7f32bc006230 comp rx=0 tx=0).stop 2023-07-18T03:22:21.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.622+0000 7f32dd478b80 1 -- 172.21.15.101:0/3005969512 shutdown_connections 2023-07-18T03:22:21.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.622+0000 7f32dd478b80 1 --2- 172.21.15.101:0/3005969512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574fd17d2d0 0x5574fd17d6a0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:21.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.622+0000 7f32dd478b80 1 --2- 172.21.15.101:0/3005969512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574fd188ad0 0x5574fd18aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:21.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.622+0000 7f32dd478b80 1 --2- 172.21.15.101:0/3005969512 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574fd17dbe0 0x5574fd188590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:21.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.622+0000 7f32dd478b80 1 -- 172.21.15.101:0/3005969512 >> 172.21.15.101:0/3005969512 conn(0x5574fd173da0 msgr2=0x5574fd1796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:21.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.622+0000 7f32dd478b80 1 -- 172.21.15.101:0/3005969512 shutdown_connections 2023-07-18T03:22:21.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.622+0000 7f32dd478b80 1 -- 172.21.15.101:0/3005969512 wait complete. 2023-07-18T03:22:21.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.622+0000 7f32dd478b80 1 -- start start 2023-07-18T03:22:21.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.623+0000 7f32dd478b80 1 Processor -- start 2023-07-18T03:22:21.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.623+0000 7f32dd478b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574fd17d2d0 0x5574fd188560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:21.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.623+0000 7f32dd478b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574fd17dbe0 0x5574fd1857e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:21.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.623+0000 7f32dd478b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574fd188ad0 0x5574fd185f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:21.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.623+0000 7f32dd478b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574fd06fac0 con 0x5574fd17dbe0 2023-07-18T03:22:21.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.623+0000 7f32dd478b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574fd06f7c0 con 0x5574fd188ad0 2023-07-18T03:22:21.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.623+0000 7f32dd478b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574fcf53920 con 0x5574fd17d2d0 2023-07-18T03:22:21.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.623+0000 7f32dd478b80 10 client.0 _ll_register_callbacks cb 0x5574fd06d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:22:21.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.623+0000 7f32c37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574fd17d2d0 0x5574fd188560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:21.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.624+0000 7f32c3fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574fd188ad0 0x5574fd185f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:21.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.624+0000 7f32c37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574fd17d2d0 0x5574fd188560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:33018/0 (socket says 172.21.15.101:33018) 2023-07-18T03:22:21.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.624+0000 7f32c37fe700 1 -- 172.21.15.101:0/727603762 learned_addr learned my addr 172.21.15.101:0/727603762 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:21.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.624+0000 7f32c3fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574fd188ad0 0x5574fd185f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47380/0 (socket says 172.21.15.101:47380) 2023-07-18T03:22:21.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.624+0000 7f32c3fff700 1 -- 172.21.15.101:0/727603762 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574fd17d2d0 msgr2=0x5574fd188560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:21.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.624+0000 7f32c2ffd700 1 --2- 172.21.15.101:0/727603762 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574fd17dbe0 0x5574fd1857e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:21.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.624+0000 7f32c3fff700 1 --2- 172.21.15.101:0/727603762 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574fd17d2d0 0x5574fd188560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:21.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.624+0000 7f32c3fff700 1 -- 172.21.15.101:0/727603762 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574fd17dbe0 msgr2=0x5574fd1857e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:21.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.624+0000 7f32c3fff700 1 --2- 172.21.15.101:0/727603762 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574fd17dbe0 0x5574fd1857e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:21.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.624+0000 7f32c3fff700 1 -- 172.21.15.101:0/727603762 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32c410d210 con 0x5574fd188ad0 2023-07-18T03:22:21.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.624+0000 7f32c37fe700 1 --2- 172.21.15.101:0/727603762 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574fd17d2d0 0x5574fd188560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:22:21.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.624+0000 7f32c2ffd700 1 --2- 172.21.15.101:0/727603762 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574fd17dbe0 0x5574fd1857e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:22:21.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.624+0000 7f32c3fff700 1 --2- 172.21.15.101:0/727603762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574fd188ad0 0x5574fd185f20 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f32bc009610 tx=0x7f32bc01e710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb4888e4c1089516 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:21.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.624+0000 7f32a2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:22:21.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.625+0000 7f32a2ffd700 1 -- 172.21.15.101:0/727603762 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32bc00e040 con 0x5574fd188ad0 2023-07-18T03:22:21.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.625+0000 7f32a2ffd700 1 -- 172.21.15.101:0/727603762 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f32bc003c10 con 0x5574fd188ad0 2023-07-18T03:22:21.626 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.625+0000 7f32dd478b80 1 -- 172.21.15.101:0/727603762 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5574fd0294f0 con 0x5574fd188ad0 2023-07-18T03:22:21.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.625+0000 7f32a2ffd700 1 -- 172.21.15.101:0/727603762 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32bc003dd0 con 0x5574fd188ad0 2023-07-18T03:22:21.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.625+0000 7f32dd478b80 10 client.28478 fetch_fsmap learned FSMap version 39 2023-07-18T03:22:21.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.625+0000 7f32dd478b80 1 -- 172.21.15.101:0/727603762 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5574fcee6c60 con 0x5574fd188ad0 2023-07-18T03:22:21.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.625+0000 7f32a2ffd700 1 -- 172.21.15.101:0/727603762 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5574fd0294f0 con 0x5574fd188ad0 2023-07-18T03:22:21.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.626+0000 7f32a2ffd700 1 -- 172.21.15.101:0/727603762 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 39) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f32bc032070 con 0x5574fd188ad0 2023-07-18T03:22:21.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.627+0000 7f32dd478b80 10 client.28478 fetch_fsmap finished waiting for FSMap version 39 2023-07-18T03:22:21.627 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.627+0000 7f32dd478b80 10 client.28478 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:22:21.628 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.627+0000 7f32dd478b80 20 client.28478 populate_metadata read hostname 'smithi101' 2023-07-18T03:22:21.628 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.627+0000 7f32dd478b80 1 --2- 172.21.15.101:0/727603762 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5574fd198000 0x5574fd19a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:21.628 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.627+0000 7f32dd478b80 4 client.28478 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:22:21.628 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.627+0000 7f32dd478b80 1 -- 172.21.15.101:0/727603762 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5574fd0c8eb0 con 0x5574fd198000 2023-07-18T03:22:21.628 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.627+0000 7f32c37fe700 1 --2- 172.21.15.101:0/727603762 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5574fd198000 0x5574fd19a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:21.628 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.627+0000 7f32c37fe700 1 --2- 172.21.15.101:0/727603762 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5574fd198000 0x5574fd19a410 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=548e00d01b780974 server_cookie=f5447c067146fb19 in_seq=0 out_seq=0 2023-07-18T03:22:21.628 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.627+0000 7f32a2ffd700 10 client.28478 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:22:21.629 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.631+0000 7f32a2ffd700 1 -- 172.21.15.101:0/727603762 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5574fd0c8eb0 con 0x5574fd198000 2023-07-18T03:22:21.629 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.631+0000 7f32a2ffd700 10 client.28478 handle_command_reply: tid=0 2023-07-18T03:22:21.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.631+0000 7f32dd478b80 1 client.28478 shutdown 2023-07-18T03:22:21.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.631+0000 7f32dd478b80 1 -- 172.21.15.101:0/727603762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574fd188ad0 msgr2=0x5574fd185f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:21.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.631+0000 7f32dd478b80 1 --2- 172.21.15.101:0/727603762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574fd188ad0 0x5574fd185f20 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f32bc009610 tx=0x7f32bc01e710 comp rx=0 tx=0).stop 2023-07-18T03:22:21.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.631+0000 7f32dd478b80 1 -- 172.21.15.101:0/727603762 shutdown_connections 2023-07-18T03:22:21.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.631+0000 7f32dd478b80 1 --2- 172.21.15.101:0/727603762 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5574fd198000 0x5574fd19a410 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:21.631 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.631+0000 7f32dd478b80 1 --2- 172.21.15.101:0/727603762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574fd188ad0 0x5574fd185f20 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:21.631 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.631+0000 7f32dd478b80 1 --2- 172.21.15.101:0/727603762 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574fd17d2d0 0x5574fd188560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:21.631 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.631+0000 7f32a2ffd700 0 client.28478 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:22:21.631 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.631+0000 7f32dd478b80 1 --2- 172.21.15.101:0/727603762 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574fd17dbe0 0x5574fd1857e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:21.631 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.631+0000 7f32dd478b80 1 -- 172.21.15.101:0/727603762 >> 172.21.15.101:0/727603762 conn(0x5574fd173da0 msgr2=0x5574fd179230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:21.631 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.631+0000 7f32dd478b80 1 -- 172.21.15.101:0/727603762 shutdown_connections 2023-07-18T03:22:21.632 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.631+0000 7f32dd478b80 1 -- 172.21.15.101:0/727603762 wait complete. 2023-07-18T03:22:21.632 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.632+0000 7f32dd478b80 20 client.28478 trim_cache size 0 max 16384 2023-07-18T03:22:21.647 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.651+0000 7f32dd478b80 1 Processor -- start 2023-07-18T03:22:21.648 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.651+0000 7f32dd478b80 1 -- start start 2023-07-18T03:22:21.648 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.652+0000 7f32dd478b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574fd17d2d0 0x5574fd180280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:21.648 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.652+0000 7f32dd478b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574fd17dbe0 0x5574fd188190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:21.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.652+0000 7f32dd478b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574fd188ad0 0x5574fd180b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:21.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.652+0000 7f32dd478b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574fd17e540 con 0x5574fd17dbe0 2023-07-18T03:22:21.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.652+0000 7f32dd478b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574fd17ed60 con 0x5574fd188ad0 2023-07-18T03:22:21.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.652+0000 7f32dd478b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574fd173970 con 0x5574fd17d2d0 2023-07-18T03:22:21.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.652+0000 7f32c2ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574fd17dbe0 0x5574fd188190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:21.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.652+0000 7f32c3fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574fd188ad0 0x5574fd180b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:21.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.652+0000 7f32c3fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574fd188ad0 0x5574fd180b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47392/0 (socket says 172.21.15.101:47392) 2023-07-18T03:22:21.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.652+0000 7f32c3fff700 1 -- 172.21.15.101:0/559898284 learned_addr learned my addr 172.21.15.101:0/559898284 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:21.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.652+0000 7f32c37fe700 1 --2- 172.21.15.101:0/559898284 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574fd17d2d0 0x5574fd180280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:21.651 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.653+0000 7f32c37fe700 1 -- 172.21.15.101:0/559898284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574fd188ad0 msgr2=0x5574fd180b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:21.651 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.653+0000 7f32c37fe700 1 --2- 172.21.15.101:0/559898284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574fd188ad0 0x5574fd180b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:21.651 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.653+0000 7f32c37fe700 1 -- 172.21.15.101:0/559898284 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574fd17dbe0 msgr2=0x5574fd188190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:21.651 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.653+0000 7f32c37fe700 1 --2- 172.21.15.101:0/559898284 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574fd17dbe0 0x5574fd188190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:21.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.653+0000 7f32c37fe700 1 -- 172.21.15.101:0/559898284 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574fcf24250 con 0x5574fd17d2d0 2023-07-18T03:22:21.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.653+0000 7f32c37fe700 1 --2- 172.21.15.101:0/559898284 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574fd17d2d0 0x5574fd180280 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f32b4099cb0 tx=0x7f32b4096420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b130f697466c49a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:21.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.653+0000 7f32aa7fc700 1 -- 172.21.15.101:0/559898284 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b40cd0b0 con 0x5574fd17d2d0 2023-07-18T03:22:21.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.653+0000 7f32aa7fc700 1 -- 172.21.15.101:0/559898284 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32b409b410 con 0x5574fd17d2d0 2023-07-18T03:22:21.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.653+0000 7f32aa7fc700 1 -- 172.21.15.101:0/559898284 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b4010880 con 0x5574fd17d2d0 2023-07-18T03:22:21.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.654+0000 7f32dd478b80 1 -- 172.21.15.101:0/559898284 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574fd17d2d0 msgr2=0x5574fd180280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:21.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.654+0000 7f32dd478b80 1 --2- 172.21.15.101:0/559898284 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574fd17d2d0 0x5574fd180280 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f32b4099cb0 tx=0x7f32b4096420 comp rx=0 tx=0).stop 2023-07-18T03:22:21.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.654+0000 7f32dd478b80 1 -- 172.21.15.101:0/559898284 shutdown_connections 2023-07-18T03:22:21.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.654+0000 7f32dd478b80 1 --2- 172.21.15.101:0/559898284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574fd188ad0 0x5574fd180b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:21.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.654+0000 7f32dd478b80 1 --2- 172.21.15.101:0/559898284 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574fd17d2d0 0x5574fd180280 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:21.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.654+0000 7f32dd478b80 1 --2- 172.21.15.101:0/559898284 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574fd17dbe0 0x5574fd188190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:21.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.654+0000 7f32dd478b80 1 -- 172.21.15.101:0/559898284 >> 172.21.15.101:0/559898284 conn(0x5574fd173da0 msgr2=0x5574fd177e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:21.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.654+0000 7f32dd478b80 1 -- 172.21.15.101:0/559898284 shutdown_connections 2023-07-18T03:22:21.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.654+0000 7f32dd478b80 1 -- 172.21.15.101:0/559898284 wait complete. 2023-07-18T03:22:21.654 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.655+0000 7f32dd478b80 1 -- start start 2023-07-18T03:22:21.654 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.655+0000 7f32dd478b80 1 Processor -- start 2023-07-18T03:22:21.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.655+0000 7f32dd478b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574fd17d2d0 0x5574fd18ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:21.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.655+0000 7f32dd478b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574fd17dbe0 0x5574fd18f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:21.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.655+0000 7f32dd478b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574fd188ad0 0x5574fd18d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:21.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.655+0000 7f32dd478b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574fd18da90 con 0x5574fd17dbe0 2023-07-18T03:22:21.657 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.655+0000 7f32dd478b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574fd18dc00 con 0x5574fd188ad0 2023-07-18T03:22:21.657 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.655+0000 7f32dd478b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574fd18ddc0 con 0x5574fd17d2d0 2023-07-18T03:22:21.657 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.655+0000 7f32dd478b80 10 client.0 _ll_register_callbacks cb 0x7f32bc008dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:22:21.657 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.656+0000 7f32c37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574fd17d2d0 0x5574fd18ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:21.658 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.656+0000 7f32c37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574fd17d2d0 0x5574fd18ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:33024/0 (socket says 172.21.15.101:33024) 2023-07-18T03:22:21.658 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.656+0000 7f32c37fe700 1 -- 172.21.15.101:0/3173590682 learned_addr learned my addr 172.21.15.101:0/3173590682 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:21.658 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.656+0000 7f32c3fff700 1 --2- 172.21.15.101:0/3173590682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574fd188ad0 0x5574fd18d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:21.658 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.656+0000 7f32c2ffd700 1 --2- 172.21.15.101:0/3173590682 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574fd17dbe0 0x5574fd18f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:21.658 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.656+0000 7f32c37fe700 1 -- 172.21.15.101:0/3173590682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574fd188ad0 msgr2=0x5574fd18d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:21.658 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.656+0000 7f32c37fe700 1 --2- 172.21.15.101:0/3173590682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574fd188ad0 0x5574fd18d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:21.658 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.656+0000 7f32c37fe700 1 -- 172.21.15.101:0/3173590682 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574fd17dbe0 msgr2=0x5574fd18f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:21.659 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.656+0000 7f32c37fe700 1 --2- 172.21.15.101:0/3173590682 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574fd17dbe0 0x5574fd18f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:21.659 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.656+0000 7f32c37fe700 1 -- 172.21.15.101:0/3173590682 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32c41a4d10 con 0x5574fd17d2d0 2023-07-18T03:22:21.659 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.656+0000 7f32c3fff700 1 --2- 172.21.15.101:0/3173590682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574fd188ad0 0x5574fd18d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:22:21.659 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.656+0000 7f32c2ffd700 1 --2- 172.21.15.101:0/3173590682 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574fd17dbe0 0x5574fd18f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:22:21.659 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.656+0000 7f32c37fe700 1 --2- 172.21.15.101:0/3173590682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574fd17d2d0 0x5574fd18ed90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f32b40964b0 tx=0x7f32b40a2400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0fac78b3c9d77ad server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:21.659 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.656+0000 7f32a2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:22:21.659 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.657+0000 7f32a2ffd700 1 -- 172.21.15.101:0/3173590682 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b409dfe0 con 0x5574fd17d2d0 2023-07-18T03:22:21.659 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.657+0000 7f32a2ffd700 1 -- 172.21.15.101:0/3173590682 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f32b40a2650 con 0x5574fd17d2d0 2023-07-18T03:22:21.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.657+0000 7f32dd478b80 1 -- 172.21.15.101:0/3173590682 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f32bc00bd60 con 0x5574fd17d2d0 2023-07-18T03:22:21.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.657+0000 7f32a2ffd700 1 -- 172.21.15.101:0/3173590682 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32b40b82b0 con 0x5574fd17d2d0 2023-07-18T03:22:21.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.657+0000 7f32dd478b80 10 client.28498 fetch_fsmap learned FSMap version 39 2023-07-18T03:22:21.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.657+0000 7f32dd478b80 1 -- 172.21.15.101:0/3173590682 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5574fcefc2b0 con 0x5574fd17d2d0 2023-07-18T03:22:21.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.657+0000 7f32a2ffd700 1 -- 172.21.15.101:0/3173590682 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32b40ca080 con 0x5574fd17d2d0 2023-07-18T03:22:21.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.657+0000 7f32a2ffd700 1 -- 172.21.15.101:0/3173590682 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 39) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f32b409a330 con 0x5574fd17d2d0 2023-07-18T03:22:21.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.657+0000 7f32dd478b80 10 client.28498 fetch_fsmap finished waiting for FSMap version 39 2023-07-18T03:22:21.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.657+0000 7f32dd478b80 10 client.28498 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:22:21.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.657+0000 7f32dd478b80 20 client.28498 populate_metadata read hostname 'smithi101' 2023-07-18T03:22:21.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.657+0000 7f32dd478b80 1 --2- 172.21.15.101:0/3173590682 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5574fd1ecdb0 0x5574fd1ef1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:21.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.657+0000 7f32dd478b80 4 client.28498 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:22:21.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.657+0000 7f32dd478b80 1 -- 172.21.15.101:0/3173590682 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5574fd08b050 con 0x5574fd1ecdb0 2023-07-18T03:22:21.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.658+0000 7f32c2ffd700 1 --2- 172.21.15.101:0/3173590682 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5574fd1ecdb0 0x5574fd1ef1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:21.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.658+0000 7f32c2ffd700 1 --2- 172.21.15.101:0/3173590682 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5574fd1ecdb0 0x5574fd1ef1c0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c6210f9a3099781 server_cookie=726b37d470919e89 in_seq=0 out_seq=0 2023-07-18T03:22:21.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.658+0000 7f32a2ffd700 10 client.28498 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:22:21.662 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.659+0000 7f32a2ffd700 1 -- 172.21.15.101:0/3173590682 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5574fd08b050 con 0x5574fd1ecdb0 2023-07-18T03:22:21.662 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.659+0000 7f32a2ffd700 10 client.28498 handle_command_reply: tid=0 2023-07-18T03:22:21.662 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.659+0000 7f32dd478b80 1 client.28498 shutdown 2023-07-18T03:22:21.662 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.659+0000 7f32dd478b80 1 -- 172.21.15.101:0/3173590682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574fd17d2d0 msgr2=0x5574fd18ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:21.662 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.659+0000 7f32dd478b80 1 --2- 172.21.15.101:0/3173590682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574fd17d2d0 0x5574fd18ed90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f32b40964b0 tx=0x7f32b40a2400 comp rx=0 tx=0).stop 2023-07-18T03:22:21.662 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.660+0000 7f32dd478b80 1 -- 172.21.15.101:0/3173590682 shutdown_connections 2023-07-18T03:22:21.662 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.660+0000 7f32dd478b80 1 --2- 172.21.15.101:0/3173590682 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5574fd1ecdb0 0x5574fd1ef1c0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:21.663 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.660+0000 7f32dd478b80 1 --2- 172.21.15.101:0/3173590682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574fd188ad0 0x5574fd18d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:21.663 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.660+0000 7f32dd478b80 1 --2- 172.21.15.101:0/3173590682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574fd17d2d0 0x5574fd18ed90 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:21.663 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.660+0000 7f32a2ffd700 0 client.28498 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:22:21.663 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.660+0000 7f32dd478b80 1 --2- 172.21.15.101:0/3173590682 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574fd17dbe0 0x5574fd18f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:21.663 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.660+0000 7f32dd478b80 1 -- 172.21.15.101:0/3173590682 >> 172.21.15.101:0/3173590682 conn(0x5574fd173da0 msgr2=0x5574fd180b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:21.663 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.660+0000 7f32dd478b80 1 -- 172.21.15.101:0/3173590682 shutdown_connections 2023-07-18T03:22:21.663 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.660+0000 7f32dd478b80 1 -- 172.21.15.101:0/3173590682 wait complete. 2023-07-18T03:22:21.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.660+0000 7f32dd478b80 20 client.28498 trim_cache size 0 max 16384 2023-07-18T03:22:21.664 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:22:21.664 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "no active scrubs running", 2023-07-18T03:22:21.665 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": {} 2023-07-18T03:22:21.665 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:22:21.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.661+0000 7f32c9bff700 1 -- 172.21.15.101:0/2298340872 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f32ac0a12c0 msgr2=0x7f32ac0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:21.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.661+0000 7f32c9bff700 1 --2- 172.21.15.101:0/2298340872 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f32ac0a12c0 0x7f32ac0a3770 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f32c4074000 tx=0x7f32b8009190 comp rx=0 tx=0).stop 2023-07-18T03:22:21.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.661+0000 7f32c9bff700 1 -- 172.21.15.101:0/2298340872 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f32c40ffad0 msgr2=0x7f32c4072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:21.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.661+0000 7f32c9bff700 1 --2- 172.21.15.101:0/2298340872 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f32c40ffad0 0x7f32c4072810 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f32b400e9a0 tx=0x7f32b400ecb0 comp rx=0 tx=0).stop 2023-07-18T03:22:21.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.661+0000 7f32c9bff700 1 -- 172.21.15.101:0/2298340872 shutdown_connections 2023-07-18T03:22:21.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.661+0000 7f32c9bff700 1 --2- 172.21.15.101:0/2298340872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32c4105ae0 0x7f32c41a3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:21.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.661+0000 7f32c9bff700 1 --2- 172.21.15.101:0/2298340872 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f32c40ffad0 0x7f32c4072810 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:21.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.661+0000 7f32c9bff700 1 --2- 172.21.15.101:0/2298340872 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f32ac0a12c0 0x7f32ac0a3770 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:21.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.661+0000 7f32c9bff700 1 --2- 172.21.15.101:0/2298340872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32c4100480 0x7f32c4072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:21.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.661+0000 7f32c9bff700 1 -- 172.21.15.101:0/2298340872 >> 172.21.15.101:0/2298340872 conn(0x7f32c40fb5f0 msgr2=0x7f32c40fd100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:21.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.661+0000 7f32c9bff700 1 -- 172.21.15.101:0/2298340872 shutdown_connections 2023-07-18T03:22:21.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:21.661+0000 7f32c9bff700 1 -- 172.21.15.101:0/2298340872 wait complete. 2023-07-18T03:22:21.677 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-18T03:22:21.678 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:22:21.972 INFO:tasks.workunit.client.0.smithi101.stdout: 10 60012 32.18 MB/sec warmup 106 sec latency 6604.846 ms 2023-07-18T03:22:22.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.020+0000 7fbc95fff700 1 -- 172.21.15.101:0/1961797424 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc9010e200 msgr2=0x7fbc9010e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:22.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.020+0000 7fbc95fff700 1 --2- 172.21.15.101:0/1961797424 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc9010e200 0x7fbc9010e5d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fbc88009f90 tx=0x7fbc8800b330 comp rx=0 tx=0).stop 2023-07-18T03:22:22.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.020+0000 7fbc95fff700 1 -- 172.21.15.101:0/1961797424 shutdown_connections 2023-07-18T03:22:22.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.020+0000 7fbc95fff700 1 --2- 172.21.15.101:0/1961797424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc9006c500 0x7fbc9011b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:22.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.020+0000 7fbc95fff700 1 --2- 172.21.15.101:0/1961797424 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc9010e200 0x7fbc9010e5d0 secure :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fbc88009f90 tx=0x7fbc8800b330 comp rx=0 tx=0).stop 2023-07-18T03:22:22.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.020+0000 7fbc95fff700 1 --2- 172.21.15.101:0/1961797424 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbc9006ba90 0x7fbc9006bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:22.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.020+0000 7fbc95fff700 1 -- 172.21.15.101:0/1961797424 >> 172.21.15.101:0/1961797424 conn(0x7fbc9006b090 msgr2=0x7fbc90070f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:22.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.030+0000 7fbc95fff700 1 -- 172.21.15.101:0/1961797424 shutdown_connections 2023-07-18T03:22:22.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.036+0000 7fbc95fff700 1 -- 172.21.15.101:0/1961797424 wait complete. 2023-07-18T03:22:22.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.038+0000 7fbc95fff700 1 Processor -- start 2023-07-18T03:22:22.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.038+0000 7fbc95fff700 1 -- start start 2023-07-18T03:22:22.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.038+0000 7fbc95fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc9006ba90 0x7fbc9011b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:22.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.038+0000 7fbc95fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbc9006c500 0x7fbc90115070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:22.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.038+0000 7fbc95fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc901155b0 0x7fbc90115a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:22.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.038+0000 7fbc95fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc901b9e60 con 0x7fbc9006c500 2023-07-18T03:22:22.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.038+0000 7fbc95fff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc901b9fa0 con 0x7fbc901155b0 2023-07-18T03:22:22.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.038+0000 7fbc95fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc901ba0e0 con 0x7fbc9006ba90 2023-07-18T03:22:22.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.040+0000 7fbc8ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc901155b0 0x7fbc90115a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:22.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.041+0000 7fbc8ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc901155b0 0x7fbc90115a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47426/0 (socket says 172.21.15.101:47426) 2023-07-18T03:22:22.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.041+0000 7fbc8ffff700 1 -- 172.21.15.101:0/1857314812 learned_addr learned my addr 172.21.15.101:0/1857314812 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:22.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.043+0000 7fbc8f7fe700 1 --2- 172.21.15.101:0/1857314812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc9006ba90 0x7fbc9011b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:22.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.055+0000 7fbc8f7fe700 1 -- 172.21.15.101:0/1857314812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc901155b0 msgr2=0x7fbc90115a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:22.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.055+0000 7fbc8f7fe700 1 --2- 172.21.15.101:0/1857314812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc901155b0 0x7fbc90115a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:22.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.055+0000 7fbc8f7fe700 1 -- 172.21.15.101:0/1857314812 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbc9006c500 msgr2=0x7fbc90115070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:22.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.055+0000 7fbc8f7fe700 1 --2- 172.21.15.101:0/1857314812 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbc9006c500 0x7fbc90115070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:22.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.055+0000 7fbc8f7fe700 1 -- 172.21.15.101:0/1857314812 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc88009c40 con 0x7fbc9006ba90 2023-07-18T03:22:22.061 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.062+0000 7fbc8f7fe700 1 --2- 172.21.15.101:0/1857314812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc9006ba90 0x7fbc9011b0e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fbc88008000 tx=0x7fbc88004c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:22.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.070+0000 7fbc8cff9700 1 -- 172.21.15.101:0/1857314812 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8800e040 con 0x7fbc9006ba90 2023-07-18T03:22:22.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.070+0000 7fbc8cff9700 1 -- 172.21.15.101:0/1857314812 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc88004e70 con 0x7fbc9006ba90 2023-07-18T03:22:22.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.070+0000 7fbc8cff9700 1 -- 172.21.15.101:0/1857314812 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8801b6c0 con 0x7fbc9006ba90 2023-07-18T03:22:22.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.070+0000 7fbc95fff700 1 -- 172.21.15.101:0/1857314812 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc901ba310 con 0x7fbc9006ba90 2023-07-18T03:22:22.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.070+0000 7fbc95fff700 1 -- 172.21.15.101:0/1857314812 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc901ba6e0 con 0x7fbc9006ba90 2023-07-18T03:22:22.071 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.072+0000 7fbc95fff700 1 -- 172.21.15.101:0/1857314812 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc7c005320 con 0x7fbc9006ba90 2023-07-18T03:22:22.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.117+0000 7fbc8cff9700 1 -- 172.21.15.101:0/1857314812 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fbc8801b820 con 0x7fbc9006ba90 2023-07-18T03:22:22.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.119+0000 7fbc8cff9700 1 --2- 172.21.15.101:0/1857314812 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fbc780a1520 0x7fbc780a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:22.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.119+0000 7fbc8cff9700 1 -- 172.21.15.101:0/1857314812 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fbc880cdbc0 con 0x7fbc9006ba90 2023-07-18T03:22:22.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.119+0000 7fbc8cff9700 1 -- 172.21.15.101:0/1857314812 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fbc880f9830 con 0x7fbc9006ba90 2023-07-18T03:22:22.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.206+0000 7fbc8effd700 1 --2- 172.21.15.101:0/1857314812 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fbc780a1520 0x7fbc780a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:22.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.277+0000 7fbc8effd700 1 --2- 172.21.15.101:0/1857314812 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fbc780a1520 0x7fbc780a39d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbc90116910 tx=0x7fbc84006d20 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:22.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.622+0000 7fbc95fff700 1 -- 172.21.15.101:0/1857314812 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc7c005190 con 0x7fbc9006ba90 2023-07-18T03:22:22.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.627+0000 7fbc8cff9700 1 -- 172.21.15.101:0/1857314812 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+10867 (secure 0 0 0) 0x7fbc88096d50 con 0x7fbc9006ba90 2023-07-18T03:22:22.627 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:22:22.627 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/4192846769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":4192846769},{"type":"v1","addr":"172.21.15.101:6839","nonce":4192846769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6841/3824340175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6840","nonce":3824340175},{"type":"v1","addr":"172.21.15.101:6841","nonce":3824340175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:04:59.361185+0000","modified":"2023-07-18T03:22:06.121618+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.155:6839/2055462483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2055462483},{"type":"v1","addr":"172.21.15.155:6839","nonce":2055462483}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:22:22.634 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.637+0000 7fbc95fff700 1 -- 172.21.15.101:0/1857314812 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fbc780a1520 msgr2=0x7fbc780a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:22.634 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.637+0000 7fbc95fff700 1 --2- 172.21.15.101:0/1857314812 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fbc780a1520 0x7fbc780a39d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbc90116910 tx=0x7fbc84006d20 comp rx=0 tx=0).stop 2023-07-18T03:22:22.634 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.637+0000 7fbc95fff700 1 -- 172.21.15.101:0/1857314812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc9006ba90 msgr2=0x7fbc9011b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:22.634 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.637+0000 7fbc95fff700 1 --2- 172.21.15.101:0/1857314812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc9006ba90 0x7fbc9011b0e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fbc88008000 tx=0x7fbc88004c80 comp rx=0 tx=0).stop 2023-07-18T03:22:22.638 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.642+0000 7fbc95fff700 1 -- 172.21.15.101:0/1857314812 shutdown_connections 2023-07-18T03:22:22.638 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.642+0000 7fbc95fff700 1 --2- 172.21.15.101:0/1857314812 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fbc780a1520 0x7fbc780a39d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:22.639 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.642+0000 7fbc95fff700 1 --2- 172.21.15.101:0/1857314812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc901155b0 0x7fbc90115a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:22.639 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.642+0000 7fbc95fff700 1 --2- 172.21.15.101:0/1857314812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc9006ba90 0x7fbc9011b0e0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:22.639 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.642+0000 7fbc95fff700 1 --2- 172.21.15.101:0/1857314812 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbc9006c500 0x7fbc90115070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:22.639 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.642+0000 7fbc95fff700 1 -- 172.21.15.101:0/1857314812 >> 172.21.15.101:0/1857314812 conn(0x7fbc9006b090 msgr2=0x7fbc90070980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:22.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.659+0000 7fbc95fff700 1 -- 172.21.15.101:0/1857314812 shutdown_connections 2023-07-18T03:22:22.662 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:22.664+0000 7fbc95fff700 1 -- 172.21.15.101:0/1857314812 wait complete. 2023-07-18T03:22:22.688 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 39 2023-07-18T03:22:22.709 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-18T03:22:22.972 INFO:tasks.workunit.client.0.smithi101.stdout: 10 60079 31.92 MB/sec warmup 107 sec latency 7263.717 ms 2023-07-18T03:22:23.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.150+0000 7f985cfff700 1 -- 172.21.15.101:0/2425380824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9858101f70 msgr2=0x7f9858102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:23.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.150+0000 7f985cfff700 1 --2- 172.21.15.101:0/2425380824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9858101f70 0x7f9858102340 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9848009990 tx=0x7f9848009ca0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.150+0000 7f985cfff700 1 -- 172.21.15.101:0/2425380824 shutdown_connections 2023-07-18T03:22:23.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.150+0000 7f985cfff700 1 --2- 172.21.15.101:0/2425380824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9858101f70 0x7f9858102340 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.150+0000 7f985cfff700 1 --2- 172.21.15.101:0/2425380824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9858106fb0 0x7f985810eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.150+0000 7f985cfff700 1 --2- 172.21.15.101:0/2425380824 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9858102910 0x7f98581068d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.150+0000 7f985cfff700 1 -- 172.21.15.101:0/2425380824 >> 172.21.15.101:0/2425380824 conn(0x7f98580fd7e0 msgr2=0x7f98580ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:23.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.150+0000 7f985cfff700 1 -- 172.21.15.101:0/2425380824 shutdown_connections 2023-07-18T03:22:23.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.150+0000 7f985cfff700 1 -- 172.21.15.101:0/2425380824 wait complete. 2023-07-18T03:22:23.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.151+0000 7f985cfff700 1 Processor -- start 2023-07-18T03:22:23.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.152+0000 7f985cfff700 1 -- start start 2023-07-18T03:22:23.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.152+0000 7f985cfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9858101f70 0x7f985810db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:23.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.152+0000 7f985cfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9858102910 0x7f985810e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:23.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.152+0000 7f985cfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9858106fb0 0x7f98581a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:23.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.152+0000 7f985cfff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98581a12e0 con 0x7f9858106fb0 2023-07-18T03:22:23.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.152+0000 7f985cfff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98581a1450 con 0x7f9858102910 2023-07-18T03:22:23.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.152+0000 7f985cfff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98581a1610 con 0x7f9858101f70 2023-07-18T03:22:23.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.152+0000 7f9855d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9858102910 0x7f985810e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:23.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.152+0000 7f9855d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9858102910 0x7f985810e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47446/0 (socket says 172.21.15.101:47446) 2023-07-18T03:22:23.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.152+0000 7f9855d9b700 1 -- 172.21.15.101:0/3596901800 learned_addr learned my addr 172.21.15.101:0/3596901800 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:23.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.152+0000 7f9856d9d700 1 --2- 172.21.15.101:0/3596901800 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9858106fb0 0x7f98581a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:23.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.153+0000 7f9855d9b700 1 -- 172.21.15.101:0/3596901800 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9858101f70 msgr2=0x7f985810db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:23.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.153+0000 7f9855d9b700 1 --2- 172.21.15.101:0/3596901800 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9858101f70 0x7f985810db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.153+0000 7f9855d9b700 1 -- 172.21.15.101:0/3596901800 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9858106fb0 msgr2=0x7f98581a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:23.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.153+0000 7f9855d9b700 1 --2- 172.21.15.101:0/3596901800 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9858106fb0 0x7f98581a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.153+0000 7f9855d9b700 1 -- 172.21.15.101:0/3596901800 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98480095f0 con 0x7f9858102910 2023-07-18T03:22:23.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.153+0000 7f9855d9b700 1 --2- 172.21.15.101:0/3596901800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9858102910 0x7f985810e0b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f984c00d950 tx=0x7f984c00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:23.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.154+0000 7f98477fe700 1 -- 172.21.15.101:0/3596901800 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c009930 con 0x7f9858102910 2023-07-18T03:22:23.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.154+0000 7f98477fe700 1 -- 172.21.15.101:0/3596901800 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f984c010460 con 0x7f9858102910 2023-07-18T03:22:23.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.154+0000 7f98477fe700 1 -- 172.21.15.101:0/3596901800 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c00f620 con 0x7f9858102910 2023-07-18T03:22:23.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.154+0000 7f985cfff700 1 -- 172.21.15.101:0/3596901800 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98581a18f0 con 0x7f9858102910 2023-07-18T03:22:23.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.154+0000 7f985cfff700 1 -- 172.21.15.101:0/3596901800 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98581104c0 con 0x7f9858102910 2023-07-18T03:22:23.156 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.160+0000 7f9870988b80 1 Processor -- start 2023-07-18T03:22:23.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.160+0000 7f9870988b80 1 -- start start 2023-07-18T03:22:23.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.161+0000 7f9870988b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ab5257c2d0 0x55ab5257c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:23.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.161+0000 7f9870988b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab5257cbe0 0x55ab52587590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:23.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.161+0000 7f9870988b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab52587ad0 0x55ab52589eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:23.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.161+0000 7f9870988b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab52482cb0 con 0x55ab5257c2d0 2023-07-18T03:22:23.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.161+0000 7f9870988b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab524d5a40 con 0x55ab52587ad0 2023-07-18T03:22:23.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.161+0000 7f9870988b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab524ceed0 con 0x55ab5257cbe0 2023-07-18T03:22:23.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.162+0000 7f985659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab5257cbe0 0x55ab52587590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:23.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.162+0000 7f985659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab5257cbe0 0x55ab52587590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:33054/0 (socket says 172.21.15.101:33054) 2023-07-18T03:22:23.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.162+0000 7f985659c700 1 -- 172.21.15.101:0/1141427770 learned_addr learned my addr 172.21.15.101:0/1141427770 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:23.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.162+0000 7f9856d9d700 1 --2- 172.21.15.101:0/1141427770 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ab5257c2d0 0x55ab5257c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:23.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.162+0000 7f98477fe700 1 -- 172.21.15.101:0/3596901800 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f984c010860 con 0x7f9858102910 2023-07-18T03:22:23.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.162+0000 7f9855d9b700 1 --2- 172.21.15.101:0/1141427770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab52587ad0 0x55ab52589eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:23.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.162+0000 7f985659c700 1 -- 172.21.15.101:0/1141427770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab52587ad0 msgr2=0x55ab52589eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:23.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.162+0000 7f985659c700 1 --2- 172.21.15.101:0/1141427770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab52587ad0 0x55ab52589eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.162+0000 7f985659c700 1 -- 172.21.15.101:0/1141427770 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ab5257c2d0 msgr2=0x55ab5257c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:23.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.162+0000 7f985659c700 1 --2- 172.21.15.101:0/1141427770 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ab5257c2d0 0x55ab5257c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.162+0000 7f985659c700 1 -- 172.21.15.101:0/1141427770 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab522fcde0 con 0x55ab5257cbe0 2023-07-18T03:22:23.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.162+0000 7f9855d9b700 1 --2- 172.21.15.101:0/1141427770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab52587ad0 0x55ab52589eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:22:23.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.162+0000 7f9856d9d700 1 --2- 172.21.15.101:0/1141427770 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ab5257c2d0 0x55ab5257c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:22:23.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.163+0000 7f985659c700 1 --2- 172.21.15.101:0/1141427770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab5257cbe0 0x55ab52587590 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f984800b5c0 tx=0x7f9848005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1088962bac53fdaf server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:23.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.163+0000 7f9844ff9700 1 -- 172.21.15.101:0/1141427770 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984801d070 con 0x55ab5257cbe0 2023-07-18T03:22:23.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.163+0000 7f9844ff9700 1 -- 172.21.15.101:0/1141427770 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f984800be00 con 0x55ab5257cbe0 2023-07-18T03:22:23.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.163+0000 7f9844ff9700 1 -- 172.21.15.101:0/1141427770 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98480052c0 con 0x55ab5257cbe0 2023-07-18T03:22:23.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.164+0000 7f98477fe700 1 --2- 172.21.15.101:0/3596901800 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f98400a1410 0x7f98400a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:23.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.164+0000 7f98477fe700 1 -- 172.21.15.101:0/3596901800 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f984c0cc8c0 con 0x7f9858102910 2023-07-18T03:22:23.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.164+0000 7f9856d9d700 1 --2- 172.21.15.101:0/3596901800 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f98400a1410 0x7f98400a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:23.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.164+0000 7f9870988b80 1 -- 172.21.15.101:0/1141427770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab5257cbe0 msgr2=0x55ab52587590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:23.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.164+0000 7f9870988b80 1 --2- 172.21.15.101:0/1141427770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab5257cbe0 0x55ab52587590 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f984800b5c0 tx=0x7f9848005f00 comp rx=0 tx=0).stop 2023-07-18T03:22:23.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.164+0000 7f9870988b80 1 -- 172.21.15.101:0/1141427770 shutdown_connections 2023-07-18T03:22:23.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.164+0000 7f9870988b80 1 --2- 172.21.15.101:0/1141427770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab52587ad0 0x55ab52589eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.164+0000 7f9870988b80 1 --2- 172.21.15.101:0/1141427770 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ab5257c2d0 0x55ab5257c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.164+0000 7f9870988b80 1 --2- 172.21.15.101:0/1141427770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab5257cbe0 0x55ab52587590 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.164+0000 7f9870988b80 1 -- 172.21.15.101:0/1141427770 >> 172.21.15.101:0/1141427770 conn(0x55ab52572da0 msgr2=0x55ab525786a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:23.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.165+0000 7f9870988b80 1 -- 172.21.15.101:0/1141427770 shutdown_connections 2023-07-18T03:22:23.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.165+0000 7f9870988b80 1 -- 172.21.15.101:0/1141427770 wait complete. 2023-07-18T03:22:23.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.165+0000 7f9856d9d700 1 --2- 172.21.15.101:0/3596901800 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f98400a1410 0x7f98400a38c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f985000a5d0 tx=0x7f985000d040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:23.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.165+0000 7f9870988b80 1 -- start start 2023-07-18T03:22:23.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.166+0000 7f9870988b80 1 Processor -- start 2023-07-18T03:22:23.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.166+0000 7f9870988b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab5257c2d0 0x55ab52587410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:23.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.166+0000 7f9870988b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab5257cbe0 0x55ab52584690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:23.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.168+0000 7f9870988b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ab52587ad0 0x55ab52584c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:23.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.168+0000 7f9870988b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab5246eac0 con 0x55ab52587ad0 2023-07-18T03:22:23.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.168+0000 7f9870988b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab5246e7c0 con 0x55ab5257cbe0 2023-07-18T03:22:23.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.168+0000 7f9870988b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab52352920 con 0x55ab5257c2d0 2023-07-18T03:22:23.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.168+0000 7f9870988b80 10 client.0 _ll_register_callbacks cb 0x55ab5246c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:22:23.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.168+0000 7f9856d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ab52587ad0 0x55ab52584c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:23.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.168+0000 7f985659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab5257c2d0 0x55ab52587410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:23.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.168+0000 7f9856d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ab52587ad0 0x55ab52584c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:37344/0 (socket says 172.21.15.101:37344) 2023-07-18T03:22:23.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.169+0000 7f9856d9d700 1 -- 172.21.15.101:0/2914468311 learned_addr learned my addr 172.21.15.101:0/2914468311 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:23.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.169+0000 7f9855d9b700 1 --2- 172.21.15.101:0/2914468311 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab5257cbe0 0x55ab52584690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:23.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.170+0000 7f9855d9b700 1 -- 172.21.15.101:0/2914468311 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab5257c2d0 msgr2=0x55ab52587410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:23.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.170+0000 7f9855d9b700 1 --2- 172.21.15.101:0/2914468311 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab5257c2d0 0x55ab52587410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.170+0000 7f9855d9b700 1 -- 172.21.15.101:0/2914468311 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ab52587ad0 msgr2=0x55ab52584c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:23.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.170+0000 7f9855d9b700 1 --2- 172.21.15.101:0/2914468311 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ab52587ad0 0x55ab52584c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.170+0000 7f9855d9b700 1 -- 172.21.15.101:0/2914468311 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9850009ba0 con 0x55ab5257cbe0 2023-07-18T03:22:23.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.170+0000 7f9855d9b700 1 --2- 172.21.15.101:0/2914468311 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab5257cbe0 0x55ab52584690 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f984c0c9c60 tx=0x7f984c0994b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2652c95d712698a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:23.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.170+0000 7f983dffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:22:23.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.170+0000 7f983dffb700 1 -- 172.21.15.101:0/2914468311 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c009c90 con 0x55ab5257cbe0 2023-07-18T03:22:23.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.170+0000 7f9870988b80 1 -- 172.21.15.101:0/2914468311 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ab524284f0 con 0x55ab5257cbe0 2023-07-18T03:22:23.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.170+0000 7f983dffb700 1 -- 172.21.15.101:0/2914468311 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f984c018260 con 0x55ab5257cbe0 2023-07-18T03:22:23.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.170+0000 7f983dffb700 1 -- 172.21.15.101:0/2914468311 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f984c099660 con 0x55ab5257cbe0 2023-07-18T03:22:23.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.171+0000 7f9870988b80 10 client.28499 fetch_fsmap learned FSMap version 39 2023-07-18T03:22:23.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.171+0000 7f9870988b80 1 -- 172.21.15.101:0/2914468311 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7f985810f3c0 con 0x55ab5257cbe0 2023-07-18T03:22:23.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.173+0000 7f983dffb700 1 -- 172.21.15.101:0/2914468311 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ab524284f0 con 0x55ab5257cbe0 2023-07-18T03:22:23.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.173+0000 7f983dffb700 1 -- 172.21.15.101:0/2914468311 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 39) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f984c098b50 con 0x55ab5257cbe0 2023-07-18T03:22:23.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.174+0000 7f9870988b80 10 client.28499 fetch_fsmap finished waiting for FSMap version 39 2023-07-18T03:22:23.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.174+0000 7f9870988b80 10 client.28499 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:22:23.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.174+0000 7f9870988b80 20 client.28499 populate_metadata read hostname 'smithi101' 2023-07-18T03:22:23.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.174+0000 7f9870988b80 1 --2- 172.21.15.101:0/2914468311 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55ab525971d0 0x55ab525995e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:23.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.174+0000 7f9870988b80 4 client.28499 mds_command: new command op to 24541 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:22:23.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.174+0000 7f9870988b80 1 -- 172.21.15.101:0/2914468311 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ab524c7eb0 con 0x55ab525971d0 2023-07-18T03:22:23.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.175+0000 7f985659c700 1 --2- 172.21.15.101:0/2914468311 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55ab525971d0 0x55ab525995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:23.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.177+0000 7f985659c700 1 --2- 172.21.15.101:0/2914468311 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55ab525971d0 0x55ab525995e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dd61e62d971986df server_cookie=55e40f7487d673e0 in_seq=0 out_seq=0 2023-07-18T03:22:23.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.177+0000 7f983dffb700 10 client.28499 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:22:23.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.184+0000 7f983dffb700 1 -- 172.21.15.101:0/2914468311 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ab524c7eb0 con 0x55ab525971d0 2023-07-18T03:22:23.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.184+0000 7f983dffb700 10 client.28499 handle_command_reply: tid=0 2023-07-18T03:22:23.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.184+0000 7f9870988b80 1 client.28499 shutdown 2023-07-18T03:22:23.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.185+0000 7f9870988b80 1 -- 172.21.15.101:0/2914468311 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab5257cbe0 msgr2=0x55ab52584690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:23.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.185+0000 7f9870988b80 1 --2- 172.21.15.101:0/2914468311 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab5257cbe0 0x55ab52584690 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f984c0c9c60 tx=0x7f984c0994b0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.186+0000 7f9870988b80 1 -- 172.21.15.101:0/2914468311 shutdown_connections 2023-07-18T03:22:23.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.186+0000 7f9870988b80 1 --2- 172.21.15.101:0/2914468311 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55ab525971d0 0x55ab525995e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.186+0000 7f9870988b80 1 --2- 172.21.15.101:0/2914468311 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ab52587ad0 0x55ab52584c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.186+0000 7f9870988b80 1 --2- 172.21.15.101:0/2914468311 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab5257c2d0 0x55ab52587410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.186+0000 7f9870988b80 1 --2- 172.21.15.101:0/2914468311 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab5257cbe0 0x55ab52584690 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.186+0000 7f9870988b80 1 -- 172.21.15.101:0/2914468311 >> 172.21.15.101:0/2914468311 conn(0x55ab52572da0 msgr2=0x55ab52578290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:23.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.186+0000 7f983dffb700 0 client.28499 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:22:23.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.187+0000 7f9870988b80 1 -- 172.21.15.101:0/2914468311 shutdown_connections 2023-07-18T03:22:23.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.187+0000 7f9870988b80 1 -- 172.21.15.101:0/2914468311 wait complete. 2023-07-18T03:22:23.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.190+0000 7f9870988b80 20 client.28499 trim_cache size 0 max 16384 2023-07-18T03:22:23.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.214+0000 7f9870988b80 1 Processor -- start 2023-07-18T03:22:23.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.214+0000 7f9870988b80 1 -- start start 2023-07-18T03:22:23.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.215+0000 7f9870988b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab5257c2d0 0x55ab5258e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:23.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.215+0000 7f9870988b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ab5257cbe0 0x55ab52586fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:23.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.215+0000 7f9870988b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab52587ad0 0x55ab52583e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:23.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.215+0000 7f9870988b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab5258a2b0 con 0x55ab5257cbe0 2023-07-18T03:22:23.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.215+0000 7f9870988b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab52585c70 con 0x55ab52587ad0 2023-07-18T03:22:23.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.215+0000 7f9870988b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab525729a0 con 0x55ab5257c2d0 2023-07-18T03:22:23.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.215+0000 7f9856d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab52587ad0 0x55ab52583e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:23.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.216+0000 7f9856d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab52587ad0 0x55ab52583e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47470/0 (socket says 172.21.15.101:47470) 2023-07-18T03:22:23.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.216+0000 7f9856d9d700 1 -- 172.21.15.101:0/3397093207 learned_addr learned my addr 172.21.15.101:0/3397093207 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:23.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.216+0000 7f9855d9b700 1 --2- 172.21.15.101:0/3397093207 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ab5257cbe0 0x55ab52586fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:23.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.217+0000 7f985659c700 1 --2- 172.21.15.101:0/3397093207 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab5257c2d0 0x55ab5258e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:23.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.218+0000 7f985659c700 1 -- 172.21.15.101:0/3397093207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab52587ad0 msgr2=0x55ab52583e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:23.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.218+0000 7f985659c700 1 --2- 172.21.15.101:0/3397093207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab52587ad0 0x55ab52583e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.218+0000 7f985659c700 1 -- 172.21.15.101:0/3397093207 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ab5257cbe0 msgr2=0x55ab52586fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:23.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.218+0000 7f985659c700 1 --2- 172.21.15.101:0/3397093207 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ab5257cbe0 0x55ab52586fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.218+0000 7f985659c700 1 -- 172.21.15.101:0/3397093207 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab522e5c60 con 0x55ab5257c2d0 2023-07-18T03:22:23.218 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.222+0000 7f985659c700 1 --2- 172.21.15.101:0/3397093207 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab5257c2d0 0x55ab5258e4a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9848005b40 tx=0x7f9848000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c231aafd8cf8e74 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:23.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.223+0000 7f98457fa700 1 -- 172.21.15.101:0/3397093207 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984801d040 con 0x55ab5257c2d0 2023-07-18T03:22:23.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.223+0000 7f98457fa700 1 -- 172.21.15.101:0/3397093207 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9848003c70 con 0x55ab5257c2d0 2023-07-18T03:22:23.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.223+0000 7f98457fa700 1 -- 172.21.15.101:0/3397093207 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984802c270 con 0x55ab5257c2d0 2023-07-18T03:22:23.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.224+0000 7f9870988b80 1 -- 172.21.15.101:0/3397093207 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab5257c2d0 msgr2=0x55ab5258e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:23.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.224+0000 7f9870988b80 1 --2- 172.21.15.101:0/3397093207 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab5257c2d0 0x55ab5258e4a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9848005b40 tx=0x7f9848000bc0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.224+0000 7f9870988b80 1 -- 172.21.15.101:0/3397093207 shutdown_connections 2023-07-18T03:22:23.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.224+0000 7f9870988b80 1 --2- 172.21.15.101:0/3397093207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab52587ad0 0x55ab52583e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.224+0000 7f9870988b80 1 --2- 172.21.15.101:0/3397093207 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab5257c2d0 0x55ab5258e4a0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.224+0000 7f9870988b80 1 --2- 172.21.15.101:0/3397093207 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ab5257cbe0 0x55ab52586fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.224+0000 7f9870988b80 1 -- 172.21.15.101:0/3397093207 >> 172.21.15.101:0/3397093207 conn(0x55ab52572da0 msgr2=0x55ab52593620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:23.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.225+0000 7f9870988b80 1 -- 172.21.15.101:0/3397093207 shutdown_connections 2023-07-18T03:22:23.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.225+0000 7f9870988b80 1 -- 172.21.15.101:0/3397093207 wait complete. 2023-07-18T03:22:23.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.225+0000 7f9870988b80 1 -- start start 2023-07-18T03:22:23.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.227+0000 7f9870988b80 1 Processor -- start 2023-07-18T03:22:23.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.227+0000 7f9870988b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab5257cbe0 0x55ab5258e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:23.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.227+0000 7f9870988b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab52587ad0 0x55ab525e3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:23.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.227+0000 7f9870988b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ab525e3580 0x55ab525e39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:23.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.227+0000 7f9870988b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab525e6cd0 con 0x55ab525e3580 2023-07-18T03:22:23.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.227+0000 7f9870988b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab525e6e40 con 0x55ab5257cbe0 2023-07-18T03:22:23.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.227+0000 7f9870988b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab525fc090 con 0x55ab52587ad0 2023-07-18T03:22:23.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.227+0000 7f9870988b80 10 client.0 _ll_register_callbacks cb 0x7f984c0952e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:22:23.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.228+0000 7f985659c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab5257cbe0 0x55ab5258e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:23.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.228+0000 7f985659c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab5257cbe0 0x55ab5258e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47484/0 (socket says 172.21.15.101:47484) 2023-07-18T03:22:23.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.228+0000 7f985659c700 1 -- 172.21.15.101:0/1911550709 learned_addr learned my addr 172.21.15.101:0/1911550709 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:23.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.228+0000 7f9855d9b700 1 --2- 172.21.15.101:0/1911550709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab52587ad0 0x55ab525e3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:23.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.228+0000 7f9856d9d700 1 --2- 172.21.15.101:0/1911550709 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ab525e3580 0x55ab525e39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:23.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.229+0000 7f985659c700 1 -- 172.21.15.101:0/1911550709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab52587ad0 msgr2=0x55ab525e3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:23.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.229+0000 7f985659c700 1 --2- 172.21.15.101:0/1911550709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab52587ad0 0x55ab525e3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.229+0000 7f985659c700 1 -- 172.21.15.101:0/1911550709 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ab525e3580 msgr2=0x55ab525e39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:23.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.229+0000 7f985659c700 1 --2- 172.21.15.101:0/1911550709 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ab525e3580 0x55ab525e39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.229+0000 7f985659c700 1 -- 172.21.15.101:0/1911550709 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9848026040 con 0x55ab5257cbe0 2023-07-18T03:22:23.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.229+0000 7f985659c700 1 --2- 172.21.15.101:0/1911550709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab5257cbe0 0x55ab5258e4b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f984800b5c0 tx=0x7f9848005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e18630175b69651d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:23.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.229+0000 7f983dffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:22:23.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.231+0000 7f983dffb700 1 -- 172.21.15.101:0/1911550709 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984801d070 con 0x55ab5257cbe0 2023-07-18T03:22:23.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.231+0000 7f983dffb700 1 -- 172.21.15.101:0/1911550709 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f984800b7d0 con 0x55ab5257cbe0 2023-07-18T03:22:23.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.231+0000 7f9870988b80 1 -- 172.21.15.101:0/1911550709 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f984800be00 con 0x55ab5257cbe0 2023-07-18T03:22:23.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.235+0000 7f983dffb700 1 -- 172.21.15.101:0/1911550709 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f984800b990 con 0x55ab5257cbe0 2023-07-18T03:22:23.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.235+0000 7f9870988b80 10 client.28505 fetch_fsmap learned FSMap version 39 2023-07-18T03:22:23.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.235+0000 7f9870988b80 1 -- 172.21.15.101:0/1911550709 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55ab52323250 con 0x55ab5257cbe0 2023-07-18T03:22:23.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.236+0000 7f983dffb700 1 -- 172.21.15.101:0/1911550709 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f984800be00 con 0x55ab5257cbe0 2023-07-18T03:22:23.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.236+0000 7f983dffb700 1 -- 172.21.15.101:0/1911550709 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 39) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f9848033070 con 0x55ab5257cbe0 2023-07-18T03:22:23.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.236+0000 7f9870988b80 10 client.28505 fetch_fsmap finished waiting for FSMap version 39 2023-07-18T03:22:23.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.236+0000 7f9870988b80 10 client.28505 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:22:23.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.236+0000 7f9870988b80 20 client.28505 populate_metadata read hostname 'smithi101' 2023-07-18T03:22:23.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.236+0000 7f9870988b80 1 --2- 172.21.15.101:0/1911550709 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55ab525fd250 0x55ab525ff630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:23.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.236+0000 7f9870988b80 4 client.28505 mds_command: new command op to 24541 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:22:23.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.236+0000 7f9870988b80 1 -- 172.21.15.101:0/1911550709 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ab5248a050 con 0x55ab525fd250 2023-07-18T03:22:23.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.236+0000 7f9855d9b700 1 --2- 172.21.15.101:0/1911550709 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55ab525fd250 0x55ab525ff630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:23.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.237+0000 7f9855d9b700 1 --2- 172.21.15.101:0/1911550709 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55ab525fd250 0x55ab525ff630 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e9d0dbabf6cc45b1 server_cookie=c750850fd160c1ea in_seq=0 out_seq=0 2023-07-18T03:22:23.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.237+0000 7f983dffb700 10 client.28505 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:22:23.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.248+0000 7f983dffb700 1 -- 172.21.15.101:0/1911550709 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ab5248a050 con 0x55ab525fd250 2023-07-18T03:22:23.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.248+0000 7f983dffb700 10 client.28505 handle_command_reply: tid=0 2023-07-18T03:22:23.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.248+0000 7f9870988b80 1 client.28505 shutdown 2023-07-18T03:22:23.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.248+0000 7f9870988b80 1 -- 172.21.15.101:0/1911550709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab5257cbe0 msgr2=0x55ab5258e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:23.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.248+0000 7f9870988b80 1 --2- 172.21.15.101:0/1911550709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab5257cbe0 0x55ab5258e4b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f984800b5c0 tx=0x7f9848005e80 comp rx=0 tx=0).stop 2023-07-18T03:22:23.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.251+0000 7f9870988b80 1 -- 172.21.15.101:0/1911550709 shutdown_connections 2023-07-18T03:22:23.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.251+0000 7f9870988b80 1 --2- 172.21.15.101:0/1911550709 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55ab525fd250 0x55ab525ff630 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.251+0000 7f9870988b80 1 --2- 172.21.15.101:0/1911550709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab5257cbe0 0x55ab5258e4b0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.251+0000 7f9870988b80 1 --2- 172.21.15.101:0/1911550709 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ab525e3580 0x55ab525e39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.251+0000 7f9870988b80 1 --2- 172.21.15.101:0/1911550709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab52587ad0 0x55ab525e3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.251+0000 7f9870988b80 1 -- 172.21.15.101:0/1911550709 >> 172.21.15.101:0/1911550709 conn(0x55ab52572da0 msgr2=0x55ab5257e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:23.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.251+0000 7f983dffb700 0 client.28505 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:22:23.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.255+0000 7f9870988b80 1 -- 172.21.15.101:0/1911550709 shutdown_connections 2023-07-18T03:22:23.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.255+0000 7f9870988b80 1 -- 172.21.15.101:0/1911550709 wait complete. 2023-07-18T03:22:23.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.255+0000 7f9870988b80 20 client.28505 trim_cache size 0 max 16384 2023-07-18T03:22:23.252 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:22:23.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.256+0000 7f985cfff700 1 -- 172.21.15.101:0/3596901800 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f98400a1410 msgr2=0x7f98400a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:23.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.256+0000 7f985cfff700 1 --2- 172.21.15.101:0/3596901800 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f98400a1410 0x7f98400a38c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f985000a5d0 tx=0x7f985000d040 comp rx=0 tx=0).stop 2023-07-18T03:22:23.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.256+0000 7f985cfff700 1 -- 172.21.15.101:0/3596901800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9858102910 msgr2=0x7f985810e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:23.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.256+0000 7f985cfff700 1 --2- 172.21.15.101:0/3596901800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9858102910 0x7f985810e0b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f984c00d950 tx=0x7f984c00dc60 comp rx=0 tx=0).stop 2023-07-18T03:22:23.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.257+0000 7f985cfff700 1 -- 172.21.15.101:0/3596901800 shutdown_connections 2023-07-18T03:22:23.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.257+0000 7f985cfff700 1 --2- 172.21.15.101:0/3596901800 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9858106fb0 0x7f98581a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.257+0000 7f985cfff700 1 --2- 172.21.15.101:0/3596901800 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9858101f70 0x7f985810db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.257+0000 7f985cfff700 1 --2- 172.21.15.101:0/3596901800 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f98400a1410 0x7f98400a38c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.257+0000 7f985cfff700 1 --2- 172.21.15.101:0/3596901800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9858102910 0x7f985810e0b0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.257+0000 7f985cfff700 1 -- 172.21.15.101:0/3596901800 >> 172.21.15.101:0/3596901800 conn(0x7f98580fd7e0 msgr2=0x7f98580ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:23.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.257+0000 7f985cfff700 1 -- 172.21.15.101:0/3596901800 shutdown_connections 2023-07-18T03:22:23.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.257+0000 7f985cfff700 1 -- 172.21.15.101:0/3596901800 wait complete. 2023-07-18T03:22:23.280 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-18T03:22:23.454 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:23 smithi179 ceph-mon[129229]: pgmap v465: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 14 MiB/s wr, 710 op/s 2023-07-18T03:22:23.454 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:23 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1857314812' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:22:23.492 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:23 smithi101 ceph-mon[126379]: pgmap v465: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 14 MiB/s wr, 710 op/s 2023-07-18T03:22:23.492 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:23 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1857314812' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:22:23.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:23 smithi155 ceph-mon[128885]: pgmap v465: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 14 MiB/s wr, 710 op/s 2023-07-18T03:22:23.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:23 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1857314812' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:22:23.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.673+0000 7f16003ff700 1 -- 172.21.15.101:0/3912450585 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f15f806c9a0 msgr2=0x7f15f806ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:23.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.673+0000 7f16003ff700 1 --2- 172.21.15.101:0/3912450585 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f15f806c9a0 0x7f15f806ce10 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f15f4009990 tx=0x7f15f4009ca0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.678+0000 7f16003ff700 1 -- 172.21.15.101:0/3912450585 shutdown_connections 2023-07-18T03:22:23.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.678+0000 7f16003ff700 1 --2- 172.21.15.101:0/3912450585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15f806d3a0 0x7f15f811b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.678+0000 7f16003ff700 1 --2- 172.21.15.101:0/3912450585 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15f810e280 0x7f15f810e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.678+0000 7f16003ff700 1 --2- 172.21.15.101:0/3912450585 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f15f806c9a0 0x7f15f806ce10 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.678+0000 7f16003ff700 1 -- 172.21.15.101:0/3912450585 >> 172.21.15.101:0/3912450585 conn(0x7f15f806bf70 msgr2=0x7f15f8070ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:23.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.681+0000 7f16003ff700 1 -- 172.21.15.101:0/3912450585 shutdown_connections 2023-07-18T03:22:23.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.684+0000 7f16003ff700 1 -- 172.21.15.101:0/3912450585 wait complete. 2023-07-18T03:22:23.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.686+0000 7f16003ff700 1 Processor -- start 2023-07-18T03:22:23.690 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.690+0000 7f16003ff700 1 -- start start 2023-07-18T03:22:23.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.691+0000 7f16003ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15f806c9a0 0x7f15f8113d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:23.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.691+0000 7f16003ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15f806d3a0 0x7f15f81142b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:23.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.691+0000 7f16003ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f15f810e280 0x7f15f81176c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:23.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.691+0000 7f16003ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f8117d20 con 0x7f15f810e280 2023-07-18T03:22:23.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.691+0000 7f16003ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f8117e90 con 0x7f15f806d3a0 2023-07-18T03:22:23.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.691+0000 7f16003ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f8118050 con 0x7f15f806c9a0 2023-07-18T03:22:23.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.693+0000 7f15fe99c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f15f810e280 0x7f15f81176c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:23.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.693+0000 7f15fe99c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f15f810e280 0x7f15f81176c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:37380/0 (socket says 172.21.15.101:37380) 2023-07-18T03:22:23.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.693+0000 7f15fe99c700 1 -- 172.21.15.101:0/1731362861 learned_addr learned my addr 172.21.15.101:0/1731362861 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:23.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.700+0000 7f15fd99a700 1 --2- 172.21.15.101:0/1731362861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15f806d3a0 0x7f15f81142b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:23.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.700+0000 7f15fe99c700 1 -- 172.21.15.101:0/1731362861 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15f806c9a0 msgr2=0x7f15f8113d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:23.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.700+0000 7f15fe99c700 1 --2- 172.21.15.101:0/1731362861 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15f806c9a0 0x7f15f8113d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.700+0000 7f15fe99c700 1 -- 172.21.15.101:0/1731362861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15f806d3a0 msgr2=0x7f15f81142b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:23.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.700+0000 7f15fe99c700 1 --2- 172.21.15.101:0/1731362861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15f806d3a0 0x7f15f81142b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.700+0000 7f15fe99c700 1 -- 172.21.15.101:0/1731362861 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f40095f0 con 0x7f15f810e280 2023-07-18T03:22:23.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.700+0000 7f15fd99a700 1 --2- 172.21.15.101:0/1731362861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15f806d3a0 0x7f15f81142b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:22:23.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.705+0000 7f15fe99c700 1 --2- 172.21.15.101:0/1731362861 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f15f810e280 0x7f15f81176c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f15f0007ee0 tx=0x7f15f000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:23.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.708+0000 7f15ef7fe700 1 -- 172.21.15.101:0/1731362861 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f0017070 con 0x7f15f810e280 2023-07-18T03:22:23.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.708+0000 7f15ef7fe700 1 -- 172.21.15.101:0/1731362861 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15f000f040 con 0x7f15f810e280 2023-07-18T03:22:23.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.708+0000 7f15ef7fe700 1 -- 172.21.15.101:0/1731362861 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f0005740 con 0x7f15f810e280 2023-07-18T03:22:23.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.708+0000 7f16003ff700 1 -- 172.21.15.101:0/1731362861 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15f8118330 con 0x7f15f810e280 2023-07-18T03:22:23.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.709+0000 7f16003ff700 1 -- 172.21.15.101:0/1731362861 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15f81188d0 con 0x7f15f810e280 2023-07-18T03:22:23.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.719+0000 7f1613c5cb80 1 Processor -- start 2023-07-18T03:22:23.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.724+0000 7f15ef7fe700 1 -- 172.21.15.101:0/1731362861 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f15f00058a0 con 0x7f15f810e280 2023-07-18T03:22:23.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.726+0000 7f15ef7fe700 1 --2- 172.21.15.101:0/1731362861 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f15e40a1490 0x7f15e40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:23.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.726+0000 7f15ef7fe700 1 -- 172.21.15.101:0/1731362861 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f15f00ccd30 con 0x7f15f810e280 2023-07-18T03:22:23.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.727+0000 7f1613c5cb80 1 -- start start 2023-07-18T03:22:23.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.727+0000 7f1613c5cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d476ad32d0 0x55d476ad36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:23.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.727+0000 7f1613c5cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d476ad3be0 0x55d476ade590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:23.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.727+0000 7f1613c5cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d476adead0 0x55d476ae0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:23.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.728+0000 7f1613c5cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4769d9cb0 con 0x55d476ad3be0 2023-07-18T03:22:23.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.728+0000 7f1613c5cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d476a2ca40 con 0x55d476adead0 2023-07-18T03:22:23.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.728+0000 7f1613c5cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d476a25ed0 con 0x55d476ad32d0 2023-07-18T03:22:23.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.728+0000 7f15fd99a700 1 --2- 172.21.15.101:0/1731362861 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f15e40a1490 0x7f15e40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:23.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.728+0000 7f15fe19b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d476ad32d0 0x55d476ad36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:23.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.728+0000 7f15fe19b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d476ad32d0 0x55d476ad36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:33114/0 (socket says 172.21.15.101:33114) 2023-07-18T03:22:23.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.728+0000 7f15fe19b700 1 -- 172.21.15.101:0/2384807155 learned_addr learned my addr 172.21.15.101:0/2384807155 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:23.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.728+0000 7f15fd99a700 1 --2- 172.21.15.101:0/2384807155 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d476ad3be0 0x55d476ade590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:23.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.730+0000 7f15fe99c700 1 --2- 172.21.15.101:0/2384807155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d476adead0 0x55d476ae0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:23.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.730+0000 7f15fd99a700 1 -- 172.21.15.101:0/2384807155 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d476ad32d0 msgr2=0x55d476ad36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:23.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.730+0000 7f15fd99a700 1 --2- 172.21.15.101:0/2384807155 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d476ad32d0 0x55d476ad36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.730+0000 7f15fd99a700 1 -- 172.21.15.101:0/2384807155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d476adead0 msgr2=0x55d476ae0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:23.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.730+0000 7f15fd99a700 1 --2- 172.21.15.101:0/2384807155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d476adead0 0x55d476ae0eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.730+0000 7f15fd99a700 1 -- 172.21.15.101:0/2384807155 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d476853de0 con 0x55d476ad3be0 2023-07-18T03:22:23.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.731+0000 7f15fd99a700 1 --2- 172.21.15.101:0/1731362861 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f15e40a1490 0x7f15e40a3940 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f15f40095c0 tx=0x7f15f400b8b0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:23.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.731+0000 7f15fd99a700 1 --2- 172.21.15.101:0/2384807155 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d476ad3be0 0x55d476ade590 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f15f400b5c0 tx=0x7f15f4028000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83cbabb3d1becf1e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:23.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.732+0000 7f15ecbfe700 1 -- 172.21.15.101:0/2384807155 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f4005250 con 0x55d476ad3be0 2023-07-18T03:22:23.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.732+0000 7f15ecbfe700 1 -- 172.21.15.101:0/2384807155 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15f4003e50 con 0x55d476ad3be0 2023-07-18T03:22:23.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.732+0000 7f15ecbfe700 1 -- 172.21.15.101:0/2384807155 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f400fe60 con 0x55d476ad3be0 2023-07-18T03:22:23.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.734+0000 7f1613c5cb80 1 -- 172.21.15.101:0/2384807155 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d476ad3be0 msgr2=0x55d476ade590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:23.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.734+0000 7f1613c5cb80 1 --2- 172.21.15.101:0/2384807155 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d476ad3be0 0x55d476ade590 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f15f400b5c0 tx=0x7f15f4028000 comp rx=0 tx=0).stop 2023-07-18T03:22:23.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.735+0000 7f1613c5cb80 1 -- 172.21.15.101:0/2384807155 shutdown_connections 2023-07-18T03:22:23.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.735+0000 7f1613c5cb80 1 --2- 172.21.15.101:0/2384807155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d476adead0 0x55d476ae0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.735+0000 7f1613c5cb80 1 --2- 172.21.15.101:0/2384807155 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d476ad32d0 0x55d476ad36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.735+0000 7f1613c5cb80 1 --2- 172.21.15.101:0/2384807155 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d476ad3be0 0x55d476ade590 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.735+0000 7f1613c5cb80 1 -- 172.21.15.101:0/2384807155 >> 172.21.15.101:0/2384807155 conn(0x55d476ac9da0 msgr2=0x55d476acf6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:23.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.736+0000 7f1613c5cb80 1 -- 172.21.15.101:0/2384807155 shutdown_connections 2023-07-18T03:22:23.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.736+0000 7f1613c5cb80 1 -- 172.21.15.101:0/2384807155 wait complete. 2023-07-18T03:22:23.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.736+0000 7f1613c5cb80 1 -- start start 2023-07-18T03:22:23.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.736+0000 7f1613c5cb80 1 Processor -- start 2023-07-18T03:22:23.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.737+0000 7f1613c5cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d476ad32d0 0x55d476ade580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:23.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.737+0000 7f1613c5cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d476ad3be0 0x55d476adb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:23.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.737+0000 7f1613c5cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d476adead0 0x55d476adbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:23.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.737+0000 7f1613c5cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4769c5ac0 con 0x55d476adead0 2023-07-18T03:22:23.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.737+0000 7f1613c5cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4769c57c0 con 0x55d476ad3be0 2023-07-18T03:22:23.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.737+0000 7f1613c5cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4768a9920 con 0x55d476ad32d0 2023-07-18T03:22:23.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.737+0000 7f1613c5cb80 10 client.0 _ll_register_callbacks cb 0x55d4769c3760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:22:23.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.738+0000 7f15fd99a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d476ad3be0 0x55d476adb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:23.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.738+0000 7f15fd99a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d476ad3be0 0x55d476adb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47528/0 (socket says 172.21.15.101:47528) 2023-07-18T03:22:23.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.738+0000 7f15fd99a700 1 -- 172.21.15.101:0/843350573 learned_addr learned my addr 172.21.15.101:0/843350573 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:23.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.738+0000 7f15fd99a700 1 -- 172.21.15.101:0/843350573 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d476ad32d0 msgr2=0x55d476ade580 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-18T03:22:23.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.738+0000 7f15fd99a700 1 --2- 172.21.15.101:0/843350573 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d476ad32d0 0x55d476ade580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.738+0000 7f15fd99a700 1 -- 172.21.15.101:0/843350573 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d476adead0 msgr2=0x55d476adbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:23.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.738+0000 7f15fd99a700 1 --2- 172.21.15.101:0/843350573 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d476adead0 0x55d476adbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.738+0000 7f15fd99a700 1 -- 172.21.15.101:0/843350573 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15f811bfa0 con 0x55d476ad3be0 2023-07-18T03:22:23.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.739+0000 7f15fd99a700 1 --2- 172.21.15.101:0/843350573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d476ad3be0 0x55d476adb790 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f15f400fcf0 tx=0x7f15f4009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7d995875c6ed99 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:23.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.739+0000 7f15e1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:22:23.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.741+0000 7f15e1ffb700 1 -- 172.21.15.101:0/843350573 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f4034040 con 0x55d476ad3be0 2023-07-18T03:22:23.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.741+0000 7f15e1ffb700 1 -- 172.21.15.101:0/843350573 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f15f4005460 con 0x55d476ad3be0 2023-07-18T03:22:23.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.741+0000 7f1613c5cb80 1 -- 172.21.15.101:0/843350573 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d47697f4f0 con 0x55d476ad3be0 2023-07-18T03:22:23.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.745+0000 7f15e1ffb700 1 -- 172.21.15.101:0/843350573 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15f4005620 con 0x55d476ad3be0 2023-07-18T03:22:23.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.745+0000 7f1613c5cb80 10 client.28511 fetch_fsmap learned FSMap version 39 2023-07-18T03:22:23.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.745+0000 7f1613c5cb80 1 -- 172.21.15.101:0/843350573 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55d47683cc60 con 0x55d476ad3be0 2023-07-18T03:22:23.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.748+0000 7f15e1ffb700 1 -- 172.21.15.101:0/843350573 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d47697f4f0 con 0x55d476ad3be0 2023-07-18T03:22:23.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.748+0000 7f15e1ffb700 1 -- 172.21.15.101:0/843350573 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 39) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f15f4037050 con 0x55d476ad3be0 2023-07-18T03:22:23.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.748+0000 7f1613c5cb80 10 client.28511 fetch_fsmap finished waiting for FSMap version 39 2023-07-18T03:22:23.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.748+0000 7f1613c5cb80 10 client.28511 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:22:23.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.748+0000 7f1613c5cb80 20 client.28511 populate_metadata read hostname 'smithi101' 2023-07-18T03:22:23.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.748+0000 7f1613c5cb80 1 --2- 172.21.15.101:0/843350573 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55d476aee130 0x55d476af0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:23.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.748+0000 7f1613c5cb80 4 client.28511 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:22:23.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.748+0000 7f1613c5cb80 1 -- 172.21.15.101:0/843350573 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d476a1eeb0 con 0x55d476aee130 2023-07-18T03:22:23.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.750+0000 7f15fe19b700 1 --2- 172.21.15.101:0/843350573 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55d476aee130 0x55d476af0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:23.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.753+0000 7f15fe19b700 1 --2- 172.21.15.101:0/843350573 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55d476aee130 0x55d476af0540 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d9315d7b8b1ea779 server_cookie=39b52c48e408e698 in_seq=0 out_seq=0 2023-07-18T03:22:23.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.753+0000 7f15e1ffb700 10 client.28511 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:22:23.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.755+0000 7f15e1ffb700 1 -- 172.21.15.101:0/843350573 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d476a1eeb0 con 0x55d476aee130 2023-07-18T03:22:23.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.755+0000 7f15e1ffb700 10 client.28511 handle_command_reply: tid=0 2023-07-18T03:22:23.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.757+0000 7f1613c5cb80 1 client.28511 shutdown 2023-07-18T03:22:23.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.757+0000 7f1613c5cb80 1 -- 172.21.15.101:0/843350573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d476ad3be0 msgr2=0x55d476adb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:23.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.757+0000 7f1613c5cb80 1 --2- 172.21.15.101:0/843350573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d476ad3be0 0x55d476adb790 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f15f400fcf0 tx=0x7f15f4009310 comp rx=0 tx=0).stop 2023-07-18T03:22:23.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.757+0000 7f1613c5cb80 1 -- 172.21.15.101:0/843350573 shutdown_connections 2023-07-18T03:22:23.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.757+0000 7f1613c5cb80 1 --2- 172.21.15.101:0/843350573 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55d476aee130 0x55d476af0540 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.757+0000 7f1613c5cb80 1 --2- 172.21.15.101:0/843350573 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d476adead0 0x55d476adbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.757+0000 7f1613c5cb80 1 --2- 172.21.15.101:0/843350573 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d476ad32d0 0x55d476ade580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.757+0000 7f1613c5cb80 1 --2- 172.21.15.101:0/843350573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d476ad3be0 0x55d476adb790 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.758+0000 7f1613c5cb80 1 -- 172.21.15.101:0/843350573 >> 172.21.15.101:0/843350573 conn(0x55d476ac9da0 msgr2=0x55d476acf290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:23.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.758+0000 7f15e1ffb700 0 client.28511 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:22:23.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.758+0000 7f1613c5cb80 1 -- 172.21.15.101:0/843350573 shutdown_connections 2023-07-18T03:22:23.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.758+0000 7f1613c5cb80 1 -- 172.21.15.101:0/843350573 wait complete. 2023-07-18T03:22:23.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.758+0000 7f1613c5cb80 20 client.28511 trim_cache size 0 max 16384 2023-07-18T03:22:23.776 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.778+0000 7f1613c5cb80 1 Processor -- start 2023-07-18T03:22:23.776 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.778+0000 7f1613c5cb80 1 -- start start 2023-07-18T03:22:23.776 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.778+0000 7f1613c5cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d476ad32d0 0x55d476adaa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:23.777 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.778+0000 7f1613c5cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d476ad3be0 0x55d476ade0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:23.777 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.778+0000 7f1613c5cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d476adead0 0x55d476ad6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:23.777 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.778+0000 7f1613c5cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d476ac9970 con 0x55d476ad32d0 2023-07-18T03:22:23.777 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.778+0000 7f1613c5cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d476ad4160 con 0x55d476adead0 2023-07-18T03:22:23.777 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.778+0000 7f1613c5cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d476adc4d0 con 0x55d476ad3be0 2023-07-18T03:22:23.777 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.779+0000 7f15fe19b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d476ad32d0 0x55d476adaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:23.777 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.779+0000 7f15fe19b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d476ad32d0 0x55d476adaa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:37398/0 (socket says 172.21.15.101:37398) 2023-07-18T03:22:23.778 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.779+0000 7f15fe99c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d476adead0 0x55d476ad6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:23.778 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.779+0000 7f15fe19b700 1 -- 172.21.15.101:0/4040802893 learned_addr learned my addr 172.21.15.101:0/4040802893 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:23.778 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.780+0000 7f15fd99a700 1 --2- 172.21.15.101:0/4040802893 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d476ad3be0 0x55d476ade0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:23.778 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.780+0000 7f15fe99c700 1 -- 172.21.15.101:0/4040802893 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d476ad3be0 msgr2=0x55d476ade0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:23.778 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.780+0000 7f15fe99c700 1 --2- 172.21.15.101:0/4040802893 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d476ad3be0 0x55d476ade0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.778 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.780+0000 7f15fe99c700 1 -- 172.21.15.101:0/4040802893 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d476ad32d0 msgr2=0x55d476adaa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:23.779 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.780+0000 7f15fe99c700 1 --2- 172.21.15.101:0/4040802893 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d476ad32d0 0x55d476adaa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.779 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.780+0000 7f15fe99c700 1 -- 172.21.15.101:0/4040802893 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d47687a250 con 0x55d476adead0 2023-07-18T03:22:23.780 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.780+0000 7f15fe99c700 1 --2- 172.21.15.101:0/4040802893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d476adead0 0x55d476ad6a90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f15f00ca070 tx=0x7f15f0099350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9ec415f03c2448b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:23.780 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.783+0000 7f15ed3ff700 1 -- 172.21.15.101:0/4040802893 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f00b2210 con 0x55d476adead0 2023-07-18T03:22:23.780 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.783+0000 7f15ed3ff700 1 -- 172.21.15.101:0/4040802893 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15f009da20 con 0x55d476adead0 2023-07-18T03:22:23.780 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.783+0000 7f15ed3ff700 1 -- 172.21.15.101:0/4040802893 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f0098ee0 con 0x55d476adead0 2023-07-18T03:22:23.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.784+0000 7f1613c5cb80 1 -- 172.21.15.101:0/4040802893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d476adead0 msgr2=0x55d476ad6a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:23.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.784+0000 7f1613c5cb80 1 --2- 172.21.15.101:0/4040802893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d476adead0 0x55d476ad6a90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f15f00ca070 tx=0x7f15f0099350 comp rx=0 tx=0).stop 2023-07-18T03:22:23.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.784+0000 7f1613c5cb80 1 -- 172.21.15.101:0/4040802893 shutdown_connections 2023-07-18T03:22:23.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.784+0000 7f1613c5cb80 1 --2- 172.21.15.101:0/4040802893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d476adead0 0x55d476ad6a90 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.784+0000 7f1613c5cb80 1 --2- 172.21.15.101:0/4040802893 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d476ad32d0 0x55d476adaa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.784+0000 7f1613c5cb80 1 --2- 172.21.15.101:0/4040802893 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d476ad3be0 0x55d476ade0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.784+0000 7f1613c5cb80 1 -- 172.21.15.101:0/4040802893 >> 172.21.15.101:0/4040802893 conn(0x55d476ac9da0 msgr2=0x55d476aea7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:23.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.785+0000 7f1613c5cb80 1 -- 172.21.15.101:0/4040802893 shutdown_connections 2023-07-18T03:22:23.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.785+0000 7f1613c5cb80 1 -- 172.21.15.101:0/4040802893 wait complete. 2023-07-18T03:22:23.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.785+0000 7f1613c5cb80 1 -- start start 2023-07-18T03:22:23.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.786+0000 7f1613c5cb80 1 Processor -- start 2023-07-18T03:22:23.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.786+0000 7f1613c5cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d476ad32d0 0x55d476ae36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:23.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.786+0000 7f1613c5cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d476ad3be0 0x55d476ae3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:23.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.786+0000 7f1613c5cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d476adead0 0x55d476b3af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:23.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.786+0000 7f1613c5cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d476b3b4f0 con 0x55d476ad32d0 2023-07-18T03:22:23.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.786+0000 7f1613c5cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d476b3b660 con 0x55d476adead0 2023-07-18T03:22:23.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.786+0000 7f1613c5cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d476b3b820 con 0x55d476ad3be0 2023-07-18T03:22:23.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.786+0000 7f1613c5cb80 10 client.0 _ll_register_callbacks cb 0x55d4769c3760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:22:23.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.787+0000 7f15fd99a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d476ad3be0 0x55d476ae3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:23.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.787+0000 7f15fd99a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d476ad3be0 0x55d476ae3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:33140/0 (socket says 172.21.15.101:33140) 2023-07-18T03:22:23.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.787+0000 7f15fd99a700 1 -- 172.21.15.101:0/1076620651 learned_addr learned my addr 172.21.15.101:0/1076620651 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:23.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.787+0000 7f15fe99c700 1 --2- 172.21.15.101:0/1076620651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d476adead0 0x55d476b3af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:23.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.788+0000 7f15fe99c700 1 -- 172.21.15.101:0/1076620651 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d476ad3be0 msgr2=0x55d476ae3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:23.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.788+0000 7f15fe99c700 1 --2- 172.21.15.101:0/1076620651 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d476ad3be0 0x55d476ae3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.788+0000 7f15fe99c700 1 -- 172.21.15.101:0/1076620651 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d476ad32d0 msgr2=0x55d476ae36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:23.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.788+0000 7f15fe99c700 1 --2- 172.21.15.101:0/1076620651 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d476ad32d0 0x55d476ae36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.788+0000 7f15fe99c700 1 -- 172.21.15.101:0/1076620651 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15f81188d0 con 0x55d476adead0 2023-07-18T03:22:23.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.789+0000 7f15fe99c700 1 --2- 172.21.15.101:0/1076620651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d476adead0 0x55d476b3af80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f15f00ca0a0 tx=0x7f15f00b2280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9d5e0b409d1cb6e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:23.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.789+0000 7f15e1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:22:23.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.791+0000 7f15e1ffb700 1 -- 172.21.15.101:0/1076620651 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f000ddd0 con 0x55d476adead0 2023-07-18T03:22:23.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.791+0000 7f15e1ffb700 1 -- 172.21.15.101:0/1076620651 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f15f0098ee0 con 0x55d476adead0 2023-07-18T03:22:23.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.792+0000 7f1613c5cb80 1 -- 172.21.15.101:0/1076620651 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f15f4003e50 con 0x55d476adead0 2023-07-18T03:22:23.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.795+0000 7f15e1ffb700 1 -- 172.21.15.101:0/1076620651 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15f00990a0 con 0x55d476adead0 2023-07-18T03:22:23.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.795+0000 7f1613c5cb80 10 client.28517 fetch_fsmap learned FSMap version 39 2023-07-18T03:22:23.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.795+0000 7f1613c5cb80 1 -- 172.21.15.101:0/1076620651 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55d4768522b0 con 0x55d476adead0 2023-07-18T03:22:23.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.797+0000 7f15e1ffb700 1 -- 172.21.15.101:0/1076620651 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15f4003e50 con 0x55d476adead0 2023-07-18T03:22:23.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.799+0000 7f15e1ffb700 1 -- 172.21.15.101:0/1076620651 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 39) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f15f00957e0 con 0x55d476adead0 2023-07-18T03:22:23.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.799+0000 7f1613c5cb80 10 client.28517 fetch_fsmap finished waiting for FSMap version 39 2023-07-18T03:22:23.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.799+0000 7f1613c5cb80 10 client.28517 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:22:23.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.799+0000 7f1613c5cb80 20 client.28517 populate_metadata read hostname 'smithi101' 2023-07-18T03:22:23.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.799+0000 7f1613c5cb80 1 --2- 172.21.15.101:0/1076620651 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55d476b42ba0 0x55d476b44fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:23.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.799+0000 7f1613c5cb80 4 client.28517 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:22:23.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.799+0000 7f1613c5cb80 1 -- 172.21.15.101:0/1076620651 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d4769e1050 con 0x55d476b42ba0 2023-07-18T03:22:23.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.802+0000 7f15fe19b700 1 --2- 172.21.15.101:0/1076620651 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55d476b42ba0 0x55d476b44fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:23.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.803+0000 7f15fe19b700 1 --2- 172.21.15.101:0/1076620651 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55d476b42ba0 0x55d476b44fb0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=50376cb28ca61bfa server_cookie=78d4f2e0e5d91c3a in_seq=0 out_seq=0 2023-07-18T03:22:23.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.804+0000 7f15e1ffb700 10 client.28517 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:22:23.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.804+0000 7f15e1ffb700 1 -- 172.21.15.101:0/1076620651 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d4769e1050 con 0x55d476b42ba0 2023-07-18T03:22:23.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.804+0000 7f15e1ffb700 10 client.28517 handle_command_reply: tid=0 2023-07-18T03:22:23.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.804+0000 7f1613c5cb80 1 client.28517 shutdown 2023-07-18T03:22:23.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.809+0000 7f1613c5cb80 1 -- 172.21.15.101:0/1076620651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d476adead0 msgr2=0x55d476b3af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:23.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.809+0000 7f1613c5cb80 1 --2- 172.21.15.101:0/1076620651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d476adead0 0x55d476b3af80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f15f00ca0a0 tx=0x7f15f00b2280 comp rx=0 tx=0).stop 2023-07-18T03:22:23.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.809+0000 7f1613c5cb80 1 -- 172.21.15.101:0/1076620651 shutdown_connections 2023-07-18T03:22:23.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.809+0000 7f1613c5cb80 1 --2- 172.21.15.101:0/1076620651 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55d476b42ba0 0x55d476b44fb0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.809+0000 7f1613c5cb80 1 --2- 172.21.15.101:0/1076620651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d476adead0 0x55d476b3af80 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.809+0000 7f1613c5cb80 1 --2- 172.21.15.101:0/1076620651 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d476ad32d0 0x55d476ae36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.809+0000 7f1613c5cb80 1 --2- 172.21.15.101:0/1076620651 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d476ad3be0 0x55d476ae3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.809+0000 7f1613c5cb80 1 -- 172.21.15.101:0/1076620651 >> 172.21.15.101:0/1076620651 conn(0x55d476ac9da0 msgr2=0x55d476ad6a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:23.813 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.810+0000 7f15e1ffb700 0 client.28517 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:22:23.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.819+0000 7f1613c5cb80 1 -- 172.21.15.101:0/1076620651 shutdown_connections 2023-07-18T03:22:23.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.819+0000 7f1613c5cb80 1 -- 172.21.15.101:0/1076620651 wait complete. 2023-07-18T03:22:23.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.825+0000 7f1613c5cb80 20 client.28517 trim_cache size 0 max 16384 2023-07-18T03:22:23.821 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:22:23.824 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.828+0000 7f16003ff700 1 -- 172.21.15.101:0/1731362861 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f15e40a1490 msgr2=0x7f15e40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:23.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.828+0000 7f16003ff700 1 --2- 172.21.15.101:0/1731362861 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f15e40a1490 0x7f15e40a3940 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f15f40095c0 tx=0x7f15f400b8b0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.828+0000 7f16003ff700 1 -- 172.21.15.101:0/1731362861 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f15f810e280 msgr2=0x7f15f81176c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:23.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.828+0000 7f16003ff700 1 --2- 172.21.15.101:0/1731362861 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f15f810e280 0x7f15f81176c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f15f0007ee0 tx=0x7f15f000d3b0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.832+0000 7f16003ff700 1 -- 172.21.15.101:0/1731362861 shutdown_connections 2023-07-18T03:22:23.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.832+0000 7f16003ff700 1 --2- 172.21.15.101:0/1731362861 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f15f810e280 0x7f15f81176c0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.832+0000 7f16003ff700 1 --2- 172.21.15.101:0/1731362861 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15f806c9a0 0x7f15f8113d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.832 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.832+0000 7f16003ff700 1 --2- 172.21.15.101:0/1731362861 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f15e40a1490 0x7f15e40a3940 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.832 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.832+0000 7f16003ff700 1 --2- 172.21.15.101:0/1731362861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15f806d3a0 0x7f15f81142b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:23.832 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.832+0000 7f16003ff700 1 -- 172.21.15.101:0/1731362861 >> 172.21.15.101:0/1731362861 conn(0x7f15f806bf70 msgr2=0x7f15f806f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:23.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.837+0000 7f16003ff700 1 -- 172.21.15.101:0/1731362861 shutdown_connections 2023-07-18T03:22:23.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:23.837+0000 7f16003ff700 1 -- 172.21.15.101:0/1731362861 wait complete. 2023-07-18T03:22:23.869 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-18T03:22:23.973 INFO:tasks.workunit.client.0.smithi101.stdout: 10 60377 31.76 MB/sec warmup 108 sec latency 333.795 ms 2023-07-18T03:22:24.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.274+0000 7fdc0ddff700 1 -- 172.21.15.101:0/3382962061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc0806d8e0 msgr2=0x7fdc08107510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:24.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.274+0000 7fdc0ddff700 1 --2- 172.21.15.101:0/3382962061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc0806d8e0 0x7fdc08107510 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc009990 tx=0x7fdbfc009ca0 comp rx=0 tx=0).stop 2023-07-18T03:22:24.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.280+0000 7fdc0ddff700 1 -- 172.21.15.101:0/3382962061 shutdown_connections 2023-07-18T03:22:24.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.280+0000 7fdc0ddff700 1 --2- 172.21.15.101:0/3382962061 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdc08112920 0x7fdc08114d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:24.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.280+0000 7fdc0ddff700 1 --2- 172.21.15.101:0/3382962061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc0806d8e0 0x7fdc08107510 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:24.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.280+0000 7fdc0ddff700 1 --2- 172.21.15.101:0/3382962061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc08107a50 0x7fdc08107ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:24.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.280+0000 7fdc0ddff700 1 -- 172.21.15.101:0/3382962061 >> 172.21.15.101:0/3382962061 conn(0x7fdc0806bf70 msgr2=0x7fdc08070ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:24.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.285+0000 7fdc0ddff700 1 -- 172.21.15.101:0/3382962061 shutdown_connections 2023-07-18T03:22:24.284 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.286+0000 7fdc0ddff700 1 -- 172.21.15.101:0/3382962061 wait complete. 2023-07-18T03:22:24.284 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.287+0000 7fdc0ddff700 1 Processor -- start 2023-07-18T03:22:24.284 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.287+0000 7fdc0ddff700 1 -- start start 2023-07-18T03:22:24.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.288+0000 7fdc0ddff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdc0806d8e0 0x7fdc08108b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:24.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.288+0000 7fdc0ddff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc08107a50 0x7fdc081090a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:24.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.288+0000 7fdc0ddff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc08112920 0x7fdc081b9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:24.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.288+0000 7fdc0ddff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc081b9760 con 0x7fdc0806d8e0 2023-07-18T03:22:24.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.288+0000 7fdc0ddff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc081b98a0 con 0x7fdc08107a50 2023-07-18T03:22:24.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.288+0000 7fdc0ddff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc081b99e0 con 0x7fdc08112920 2023-07-18T03:22:24.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.290+0000 7fdc06ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc08107a50 0x7fdc081090a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:24.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.290+0000 7fdc06ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc08107a50 0x7fdc081090a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47548/0 (socket says 172.21.15.101:47548) 2023-07-18T03:22:24.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.290+0000 7fdc06ffd700 1 -- 172.21.15.101:0/2165124776 learned_addr learned my addr 172.21.15.101:0/2165124776 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:24.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.290+0000 7fdc06ffd700 1 -- 172.21.15.101:0/2165124776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc08112920 msgr2=0x7fdc081b9220 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-18T03:22:24.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.290+0000 7fdc06ffd700 1 --2- 172.21.15.101:0/2165124776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc08112920 0x7fdc081b9220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:24.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.290+0000 7fdc06ffd700 1 -- 172.21.15.101:0/2165124776 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdc0806d8e0 msgr2=0x7fdc08108b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:24.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.290+0000 7fdc06ffd700 1 --2- 172.21.15.101:0/2165124776 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdc0806d8e0 0x7fdc08108b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:24.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.290+0000 7fdc06ffd700 1 -- 172.21.15.101:0/2165124776 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbfc0095f0 con 0x7fdc08107a50 2023-07-18T03:22:24.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.291+0000 7fdc06ffd700 1 --2- 172.21.15.101:0/2165124776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc08107a50 0x7fdc081090a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fdc0000ef10 tx=0x7fdc0000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:24.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.292+0000 7fdc04ff9700 1 -- 172.21.15.101:0/2165124776 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0000cad0 con 0x7fdc08107a50 2023-07-18T03:22:24.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.292+0000 7fdc04ff9700 1 -- 172.21.15.101:0/2165124776 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc0000f040 con 0x7fdc08107a50 2023-07-18T03:22:24.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.292+0000 7fdc04ff9700 1 -- 172.21.15.101:0/2165124776 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc000146e0 con 0x7fdc08107a50 2023-07-18T03:22:24.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.292+0000 7fdc0ddff700 1 -- 172.21.15.101:0/2165124776 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc081b9be0 con 0x7fdc08107a50 2023-07-18T03:22:24.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.292+0000 7fdc0ddff700 1 -- 172.21.15.101:0/2165124776 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc081ba0a0 con 0x7fdc08107a50 2023-07-18T03:22:24.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.298+0000 7fdc21739b80 1 Processor -- start 2023-07-18T03:22:24.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.313+0000 7fdc21739b80 1 -- start start 2023-07-18T03:22:24.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.313+0000 7fdc21739b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecaa5ee2d0 0x55ecaa5ee6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:24.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.313+0000 7fdc21739b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ecaa5eebe0 0x55ecaa5f9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:24.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.313+0000 7fdc21739b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ecaa5f9ad0 0x55ecaa5fbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:24.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.313+0000 7fdc21739b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecaa4f4cb0 con 0x55ecaa5eebe0 2023-07-18T03:22:24.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.313+0000 7fdc21739b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecaa547a40 con 0x55ecaa5ee2d0 2023-07-18T03:22:24.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.313+0000 7fdc21739b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecaa540ed0 con 0x55ecaa5f9ad0 2023-07-18T03:22:24.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.321+0000 7fdc04ff9700 1 -- 172.21.15.101:0/2165124776 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fdc0000cc30 con 0x7fdc08107a50 2023-07-18T03:22:24.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.322+0000 7fdc06ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ecaa5f9ad0 0x55ecaa5fbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:24.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.322+0000 7fdc06ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ecaa5f9ad0 0x55ecaa5fbeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:33174/0 (socket says 172.21.15.101:33174) 2023-07-18T03:22:24.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.322+0000 7fdc06ffd700 1 -- 172.21.15.101:0/3495646227 learned_addr learned my addr 172.21.15.101:0/3495646227 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:24.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.323+0000 7fdc04ff9700 1 --2- 172.21.15.101:0/2165124776 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fdbf00a14d0 0x7fdbf00a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:24.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.323+0000 7fdc04ff9700 1 -- 172.21.15.101:0/2165124776 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fdc000cc950 con 0x7fdc08107a50 2023-07-18T03:22:24.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.324+0000 7fdc06ffd700 1 -- 172.21.15.101:0/3495646227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecaa5ee2d0 msgr2=0x55ecaa5ee6a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-18T03:22:24.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.324+0000 7fdc06ffd700 1 --2- 172.21.15.101:0/3495646227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecaa5ee2d0 0x55ecaa5ee6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:24.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.324+0000 7fdc06ffd700 1 -- 172.21.15.101:0/3495646227 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ecaa5eebe0 msgr2=0x55ecaa5f9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:24.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.324+0000 7fdc06ffd700 1 --2- 172.21.15.101:0/3495646227 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ecaa5eebe0 0x55ecaa5f9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:24.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.324+0000 7fdc06ffd700 1 -- 172.21.15.101:0/3495646227 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecaa36ede0 con 0x55ecaa5f9ad0 2023-07-18T03:22:24.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.327+0000 7fdc06ffd700 1 --2- 172.21.15.101:0/3495646227 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ecaa5f9ad0 0x55ecaa5fbeb0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fdc000c9c00 tx=0x7fdc00000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a02cc819375953b7 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:24.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.328+0000 7fdbedffb700 1 -- 172.21.15.101:0/3495646227 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc000991e0 con 0x55ecaa5f9ad0 2023-07-18T03:22:24.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.328+0000 7fdbedffb700 1 -- 172.21.15.101:0/3495646227 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc00099000 con 0x55ecaa5f9ad0 2023-07-18T03:22:24.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.328+0000 7fdbedffb700 1 -- 172.21.15.101:0/3495646227 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc00097b60 con 0x55ecaa5f9ad0 2023-07-18T03:22:24.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.329+0000 7fdc21739b80 1 -- 172.21.15.101:0/3495646227 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ecaa5f9ad0 msgr2=0x55ecaa5fbeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:24.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.329+0000 7fdc21739b80 1 --2- 172.21.15.101:0/3495646227 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ecaa5f9ad0 0x55ecaa5fbeb0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fdc000c9c00 tx=0x7fdc00000f40 comp rx=0 tx=0).stop 2023-07-18T03:22:24.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.333+0000 7fdc07fff700 1 --2- 172.21.15.101:0/2165124776 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fdbf00a14d0 0x7fdbf00a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:24.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.334+0000 7fdc07fff700 1 --2- 172.21.15.101:0/2165124776 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fdbf00a14d0 0x7fdbf00a3980 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdc0810a010 tx=0x7fdbf8006c50 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:24.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.334+0000 7fdc21739b80 1 -- 172.21.15.101:0/3495646227 shutdown_connections 2023-07-18T03:22:24.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.334+0000 7fdc21739b80 1 --2- 172.21.15.101:0/3495646227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecaa5ee2d0 0x55ecaa5ee6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:24.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.334+0000 7fdc21739b80 1 --2- 172.21.15.101:0/3495646227 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ecaa5f9ad0 0x55ecaa5fbeb0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:24.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.334+0000 7fdc21739b80 1 --2- 172.21.15.101:0/3495646227 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ecaa5eebe0 0x55ecaa5f9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:24.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.334+0000 7fdc21739b80 1 -- 172.21.15.101:0/3495646227 >> 172.21.15.101:0/3495646227 conn(0x55ecaa5e4da0 msgr2=0x55ecaa5ea6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:24.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.335+0000 7fdc21739b80 1 -- 172.21.15.101:0/3495646227 shutdown_connections 2023-07-18T03:22:24.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.336+0000 7fdc21739b80 1 -- 172.21.15.101:0/3495646227 wait complete. 2023-07-18T03:22:24.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.336+0000 7fdc21739b80 1 -- start start 2023-07-18T03:22:24.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.337+0000 7fdc21739b80 1 Processor -- start 2023-07-18T03:22:24.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.337+0000 7fdc21739b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecaa5ee2d0 0x55ecaa5f9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:24.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.337+0000 7fdc21739b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ecaa5eebe0 0x55ecaa5f66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:24.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.337+0000 7fdc21739b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ecaa5f9ad0 0x55ecaa5f6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:24.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.337+0000 7fdc21739b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecaa4e0ac0 con 0x55ecaa5eebe0 2023-07-18T03:22:24.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.337+0000 7fdc21739b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecaa4e07c0 con 0x55ecaa5ee2d0 2023-07-18T03:22:24.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.337+0000 7fdc21739b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecaa3c4920 con 0x55ecaa5f9ad0 2023-07-18T03:22:24.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.337+0000 7fdc21739b80 10 client.0 _ll_register_callbacks cb 0x55ecaa4de760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:22:24.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.338+0000 7fdc06ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ecaa5eebe0 0x55ecaa5f66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:24.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.338+0000 7fdc077fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecaa5ee2d0 0x55ecaa5f9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:24.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.338+0000 7fdc06ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ecaa5eebe0 0x55ecaa5f66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:37436/0 (socket says 172.21.15.101:37436) 2023-07-18T03:22:24.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.338+0000 7fdc06ffd700 1 -- 172.21.15.101:0/2262830910 learned_addr learned my addr 172.21.15.101:0/2262830910 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:24.336 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.339+0000 7fdc077fe700 1 -- 172.21.15.101:0/2262830910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ecaa5f9ad0 msgr2=0x55ecaa5f6c90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-18T03:22:24.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.339+0000 7fdc077fe700 1 --2- 172.21.15.101:0/2262830910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ecaa5f9ad0 0x55ecaa5f6c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:24.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.339+0000 7fdc077fe700 1 -- 172.21.15.101:0/2262830910 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ecaa5eebe0 msgr2=0x55ecaa5f66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:24.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.339+0000 7fdc077fe700 1 --2- 172.21.15.101:0/2262830910 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ecaa5eebe0 0x55ecaa5f66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:24.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.339+0000 7fdc077fe700 1 -- 172.21.15.101:0/2262830910 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc081ba0a0 con 0x55ecaa5ee2d0 2023-07-18T03:22:24.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.339+0000 7fdc077fe700 1 --2- 172.21.15.101:0/2262830910 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecaa5ee2d0 0x55ecaa5f9430 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc009990 tx=0x7fdbfc00b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be5631fb2080f5a1 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:24.337 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.339+0000 7fdbe6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:22:24.338 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.341+0000 7fdbe6ffd700 1 -- 172.21.15.101:0/2262830910 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc01d070 con 0x55ecaa5ee2d0 2023-07-18T03:22:24.338 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.341+0000 7fdbe6ffd700 1 -- 172.21.15.101:0/2262830910 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fdbfc00f670 con 0x55ecaa5ee2d0 2023-07-18T03:22:24.338 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.342+0000 7fdc21739b80 1 -- 172.21.15.101:0/2262830910 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ecaa49a4f0 con 0x55ecaa5ee2d0 2023-07-18T03:22:24.338 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.342+0000 7fdbe6ffd700 1 -- 172.21.15.101:0/2262830910 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbfc00f830 con 0x55ecaa5ee2d0 2023-07-18T03:22:24.339 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.343+0000 7fdc21739b80 10 client.28526 fetch_fsmap learned FSMap version 39 2023-07-18T03:22:24.339 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.343+0000 7fdc21739b80 1 -- 172.21.15.101:0/2262830910 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55ecaa357c60 con 0x55ecaa5ee2d0 2023-07-18T03:22:24.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.344+0000 7fdbe6ffd700 1 -- 172.21.15.101:0/2262830910 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ecaa49a4f0 con 0x55ecaa5ee2d0 2023-07-18T03:22:24.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.344+0000 7fdbe6ffd700 1 -- 172.21.15.101:0/2262830910 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 39) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fdbfc021a90 con 0x55ecaa5ee2d0 2023-07-18T03:22:24.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.346+0000 7fdc21739b80 10 client.28526 fetch_fsmap finished waiting for FSMap version 39 2023-07-18T03:22:24.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.346+0000 7fdc21739b80 10 client.28526 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:22:24.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.346+0000 7fdc21739b80 20 client.28526 populate_metadata read hostname 'smithi101' 2023-07-18T03:22:24.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.346+0000 7fdc21739b80 1 --2- 172.21.15.101:0/2262830910 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ecaa6093f0 0x55ecaa60b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:24.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.346+0000 7fdc21739b80 4 client.28526 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:22:24.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.346+0000 7fdc21739b80 1 -- 172.21.15.101:0/2262830910 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ecaa539eb0 con 0x55ecaa6093f0 2023-07-18T03:22:24.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.348+0000 7fdc06ffd700 1 --2- 172.21.15.101:0/2262830910 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ecaa6093f0 0x55ecaa60b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:24.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.364+0000 7fdc06ffd700 1 --2- 172.21.15.101:0/2262830910 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ecaa6093f0 0x55ecaa60b800 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c525684821305786 server_cookie=e78c62e5c533c67c in_seq=0 out_seq=0 2023-07-18T03:22:24.361 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.364+0000 7fdbe6ffd700 10 client.28526 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:22:24.365 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.368+0000 7fdbe6ffd700 1 -- 172.21.15.101:0/2262830910 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ecaa539eb0 con 0x55ecaa6093f0 2023-07-18T03:22:24.365 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.368+0000 7fdbe6ffd700 10 client.28526 handle_command_reply: tid=0 2023-07-18T03:22:24.367 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.369+0000 7fdc21739b80 1 client.28526 shutdown 2023-07-18T03:22:24.375 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.379+0000 7fdc21739b80 1 -- 172.21.15.101:0/2262830910 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecaa5ee2d0 msgr2=0x55ecaa5f9430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:24.375 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.379+0000 7fdc21739b80 1 --2- 172.21.15.101:0/2262830910 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecaa5ee2d0 0x55ecaa5f9430 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc009990 tx=0x7fdbfc00b9c0 comp rx=0 tx=0).stop 2023-07-18T03:22:24.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.382+0000 7fdc21739b80 1 -- 172.21.15.101:0/2262830910 shutdown_connections 2023-07-18T03:22:24.381 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.382+0000 7fdc21739b80 1 --2- 172.21.15.101:0/2262830910 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ecaa6093f0 0x55ecaa60b800 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:24.381 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.382+0000 7fdc21739b80 1 --2- 172.21.15.101:0/2262830910 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecaa5ee2d0 0x55ecaa5f9430 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:24.381 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.382+0000 7fdc21739b80 1 --2- 172.21.15.101:0/2262830910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ecaa5f9ad0 0x55ecaa5f6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:24.381 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.382+0000 7fdc21739b80 1 --2- 172.21.15.101:0/2262830910 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ecaa5eebe0 0x55ecaa5f66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:24.381 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.382+0000 7fdc21739b80 1 -- 172.21.15.101:0/2262830910 >> 172.21.15.101:0/2262830910 conn(0x55ecaa5e4da0 msgr2=0x55ecaa5ea230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:24.381 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.382+0000 7fdbe6ffd700 0 client.28526 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:22:24.381 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.384+0000 7fdc21739b80 1 -- 172.21.15.101:0/2262830910 shutdown_connections 2023-07-18T03:22:24.382 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.384+0000 7fdc21739b80 1 -- 172.21.15.101:0/2262830910 wait complete. 2023-07-18T03:22:24.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.389+0000 7fdc21739b80 20 client.28526 trim_cache size 0 max 16384 2023-07-18T03:22:24.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.414+0000 7fdc21739b80 1 Processor -- start 2023-07-18T03:22:24.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.415+0000 7fdc21739b80 1 -- start start 2023-07-18T03:22:24.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.415+0000 7fdc21739b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ecaa5ee2d0 0x55ecaa60b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:24.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.415+0000 7fdc21739b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecaa5eebe0 0x55ecaa5f1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:24.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.415+0000 7fdc21739b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ecaa5f9ad0 0x55ecaa5f1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:24.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.415+0000 7fdc21739b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecaa5e49b0 con 0x55ecaa5ee2d0 2023-07-18T03:22:24.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.415+0000 7fdc21739b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecaa5f4c60 con 0x55ecaa5eebe0 2023-07-18T03:22:24.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.415+0000 7fdc21739b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecaa5f7a00 con 0x55ecaa5f9ad0 2023-07-18T03:22:24.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.417+0000 7fdc07fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ecaa5f9ad0 0x55ecaa5f1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:24.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.417+0000 7fdc07fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ecaa5f9ad0 0x55ecaa5f1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:33202/0 (socket says 172.21.15.101:33202) 2023-07-18T03:22:24.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.417+0000 7fdc07fff700 1 -- 172.21.15.101:0/3564943900 learned_addr learned my addr 172.21.15.101:0/3564943900 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:24.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.419+0000 7fdc06ffd700 1 --2- 172.21.15.101:0/3564943900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecaa5eebe0 0x55ecaa5f1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:24.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.420+0000 7fdc07fff700 1 -- 172.21.15.101:0/3564943900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecaa5eebe0 msgr2=0x55ecaa5f1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:24.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.420+0000 7fdc07fff700 1 --2- 172.21.15.101:0/3564943900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecaa5eebe0 0x55ecaa5f1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:24.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.420+0000 7fdc07fff700 1 -- 172.21.15.101:0/3564943900 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ecaa5ee2d0 msgr2=0x55ecaa60b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:24.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.420+0000 7fdc07fff700 1 --2- 172.21.15.101:0/3564943900 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ecaa5ee2d0 0x55ecaa60b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:24.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.420+0000 7fdc07fff700 1 -- 172.21.15.101:0/3564943900 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecaa395250 con 0x55ecaa5f9ad0 2023-07-18T03:22:24.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.422+0000 7fdc07fff700 1 --2- 172.21.15.101:0/3564943900 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ecaa5f9ad0 0x55ecaa5f1f70 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8013a20 tx=0x7fdbf8009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63439b70ca172e15 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:24.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.423+0000 7fdbee7fc700 1 -- 172.21.15.101:0/3564943900 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf8012920 con 0x55ecaa5f9ad0 2023-07-18T03:22:24.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.423+0000 7fdbee7fc700 1 -- 172.21.15.101:0/3564943900 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbf8019040 con 0x55ecaa5f9ad0 2023-07-18T03:22:24.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.423+0000 7fdbee7fc700 1 -- 172.21.15.101:0/3564943900 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf8010360 con 0x55ecaa5f9ad0 2023-07-18T03:22:24.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.424+0000 7fdc21739b80 1 -- 172.21.15.101:0/3564943900 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ecaa5f9ad0 msgr2=0x55ecaa5f1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:24.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.424+0000 7fdc21739b80 1 --2- 172.21.15.101:0/3564943900 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ecaa5f9ad0 0x55ecaa5f1f70 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8013a20 tx=0x7fdbf8009f90 comp rx=0 tx=0).stop 2023-07-18T03:22:24.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.424+0000 7fdc21739b80 1 -- 172.21.15.101:0/3564943900 shutdown_connections 2023-07-18T03:22:24.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.424+0000 7fdc21739b80 1 --2- 172.21.15.101:0/3564943900 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ecaa5f9ad0 0x55ecaa5f1f70 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:24.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.424+0000 7fdc21739b80 1 --2- 172.21.15.101:0/3564943900 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ecaa5ee2d0 0x55ecaa60b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:24.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.424+0000 7fdc21739b80 1 --2- 172.21.15.101:0/3564943900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecaa5eebe0 0x55ecaa5f1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:24.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.424+0000 7fdc21739b80 1 -- 172.21.15.101:0/3564943900 >> 172.21.15.101:0/3564943900 conn(0x55ecaa5e4da0 msgr2=0x55ecaa6056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:24.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.425+0000 7fdc21739b80 1 -- 172.21.15.101:0/3564943900 shutdown_connections 2023-07-18T03:22:24.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.425+0000 7fdc21739b80 1 -- 172.21.15.101:0/3564943900 wait complete. 2023-07-18T03:22:24.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.425+0000 7fdc21739b80 1 -- start start 2023-07-18T03:22:24.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.426+0000 7fdc21739b80 1 Processor -- start 2023-07-18T03:22:24.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.426+0000 7fdc21739b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ecaa5ee2d0 0x55ecaa600530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:24.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.426+0000 7fdc21739b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ecaa5eebe0 0x55ecaa5fe950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:24.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.426+0000 7fdc21739b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecaa5f9ad0 0x55ecaa5fee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:24.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.426+0000 7fdc21739b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecaa5ff530 con 0x55ecaa5ee2d0 2023-07-18T03:22:24.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.426+0000 7fdc21739b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecaa5ff6a0 con 0x55ecaa5f9ad0 2023-07-18T03:22:24.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.426+0000 7fdc21739b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecaa65d2c0 con 0x55ecaa5eebe0 2023-07-18T03:22:24.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.426+0000 7fdc21739b80 10 client.0 _ll_register_callbacks cb 0x7fdc000097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:22:24.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.427+0000 7fdc07fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecaa5f9ad0 0x55ecaa5fee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:24.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.427+0000 7fdc07fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecaa5f9ad0 0x55ecaa5fee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47586/0 (socket says 172.21.15.101:47586) 2023-07-18T03:22:24.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.427+0000 7fdc07fff700 1 -- 172.21.15.101:0/2731437798 learned_addr learned my addr 172.21.15.101:0/2731437798 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:24.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.427+0000 7fdc06ffd700 1 --2- 172.21.15.101:0/2731437798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ecaa5eebe0 0x55ecaa5fe950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:24.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.429+0000 7fdc077fe700 1 --2- 172.21.15.101:0/2731437798 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ecaa5ee2d0 0x55ecaa600530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:24.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.430+0000 7fdc07fff700 1 -- 172.21.15.101:0/2731437798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ecaa5eebe0 msgr2=0x55ecaa5fe950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:24.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.430+0000 7fdc07fff700 1 --2- 172.21.15.101:0/2731437798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ecaa5eebe0 0x55ecaa5fe950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:24.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.430+0000 7fdc07fff700 1 -- 172.21.15.101:0/2731437798 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ecaa5ee2d0 msgr2=0x55ecaa600530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:24.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.430+0000 7fdc07fff700 1 --2- 172.21.15.101:0/2731437798 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ecaa5ee2d0 0x55ecaa600530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:24.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.430+0000 7fdc07fff700 1 -- 172.21.15.101:0/2731437798 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbf80136d0 con 0x55ecaa5f9ad0 2023-07-18T03:22:24.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.430+0000 7fdc077fe700 1 --2- 172.21.15.101:0/2731437798 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ecaa5ee2d0 0x55ecaa600530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:22:24.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.430+0000 7fdc07fff700 1 --2- 172.21.15.101:0/2731437798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecaa5f9ad0 0x55ecaa5fee90 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fdbf80139f0 tx=0x7fdbf8009490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10cd523eec705619 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:24.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.430+0000 7fdbe6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:22:24.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.431+0000 7fdbe6ffd700 1 -- 172.21.15.101:0/2731437798 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf801d8e0 con 0x55ecaa5f9ad0 2023-07-18T03:22:24.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.431+0000 7fdbe6ffd700 1 -- 172.21.15.101:0/2731437798 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fdbf802f460 con 0x55ecaa5f9ad0 2023-07-18T03:22:24.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.431+0000 7fdc21739b80 1 -- 172.21.15.101:0/2731437798 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc00099000 con 0x55ecaa5f9ad0 2023-07-18T03:22:24.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.431+0000 7fdbe6ffd700 1 -- 172.21.15.101:0/2731437798 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbf802f620 con 0x55ecaa5f9ad0 2023-07-18T03:22:24.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.431+0000 7fdc21739b80 10 client.28532 fetch_fsmap learned FSMap version 39 2023-07-18T03:22:24.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.431+0000 7fdc21739b80 1 -- 172.21.15.101:0/2731437798 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55ecaa36d2b0 con 0x55ecaa5f9ad0 2023-07-18T03:22:24.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.432+0000 7fdbe6ffd700 1 -- 172.21.15.101:0/2731437798 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc00099000 con 0x55ecaa5f9ad0 2023-07-18T03:22:24.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.432+0000 7fdbe6ffd700 1 -- 172.21.15.101:0/2731437798 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 39) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fdbf801c070 con 0x55ecaa5f9ad0 2023-07-18T03:22:24.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.432+0000 7fdc21739b80 10 client.28532 fetch_fsmap finished waiting for FSMap version 39 2023-07-18T03:22:24.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.432+0000 7fdc21739b80 10 client.28532 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:22:24.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.432+0000 7fdc21739b80 20 client.28532 populate_metadata read hostname 'smithi101' 2023-07-18T03:22:24.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.432+0000 7fdc21739b80 1 --2- 172.21.15.101:0/2731437798 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ecaa65e810 0x55ecaa660c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:24.432 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.432+0000 7fdc21739b80 4 client.28532 mds_command: new command op to 24562 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:22:24.432 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.432+0000 7fdc21739b80 1 -- 172.21.15.101:0/2731437798 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ecaa4fc050 con 0x55ecaa65e810 2023-07-18T03:22:24.432 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.432+0000 7fdc077fe700 1 --2- 172.21.15.101:0/2731437798 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ecaa65e810 0x55ecaa660c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:24.432 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.433+0000 7fdbe6ffd700 10 client.28532 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:22:24.432 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.433+0000 7fdc077fe700 1 --2- 172.21.15.101:0/2731437798 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ecaa65e810 0x55ecaa660c20 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a194aee5ca7cb1c5 server_cookie=1d684ac65e6990b2 in_seq=0 out_seq=0 2023-07-18T03:22:24.433 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.435+0000 7fdbe6ffd700 1 -- 172.21.15.101:0/2731437798 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ecaa4fc050 con 0x55ecaa65e810 2023-07-18T03:22:24.433 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.435+0000 7fdbe6ffd700 10 client.28532 handle_command_reply: tid=0 2023-07-18T03:22:24.433 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.435+0000 7fdc21739b80 1 client.28532 shutdown 2023-07-18T03:22:24.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.435+0000 7fdc21739b80 1 -- 172.21.15.101:0/2731437798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecaa5f9ad0 msgr2=0x55ecaa5fee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:24.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.435+0000 7fdc21739b80 1 --2- 172.21.15.101:0/2731437798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecaa5f9ad0 0x55ecaa5fee90 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fdbf80139f0 tx=0x7fdbf8009490 comp rx=0 tx=0).stop 2023-07-18T03:22:24.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.435+0000 7fdc21739b80 1 -- 172.21.15.101:0/2731437798 shutdown_connections 2023-07-18T03:22:24.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.435+0000 7fdc21739b80 1 --2- 172.21.15.101:0/2731437798 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ecaa65e810 0x55ecaa660c20 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:24.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.435+0000 7fdc21739b80 1 --2- 172.21.15.101:0/2731437798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ecaa5f9ad0 0x55ecaa5fee90 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:24.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.435+0000 7fdc21739b80 1 --2- 172.21.15.101:0/2731437798 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ecaa5ee2d0 0x55ecaa600530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:24.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.435+0000 7fdc21739b80 1 --2- 172.21.15.101:0/2731437798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ecaa5eebe0 0x55ecaa5fe950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:24.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.435+0000 7fdc21739b80 1 -- 172.21.15.101:0/2731437798 >> 172.21.15.101:0/2731437798 conn(0x55ecaa5e4da0 msgr2=0x55ecaa5f1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:24.435 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.435+0000 7fdbe6ffd700 0 client.28532 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:22:24.435 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.436+0000 7fdc21739b80 1 -- 172.21.15.101:0/2731437798 shutdown_connections 2023-07-18T03:22:24.435 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.436+0000 7fdc21739b80 1 -- 172.21.15.101:0/2731437798 wait complete. 2023-07-18T03:22:24.435 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.436+0000 7fdc21739b80 20 client.28532 trim_cache size 0 max 16384 2023-07-18T03:22:24.435 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:22:24.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.439+0000 7fdc0ddff700 1 -- 172.21.15.101:0/2165124776 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fdbf00a14d0 msgr2=0x7fdbf00a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:24.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.439+0000 7fdc0ddff700 1 --2- 172.21.15.101:0/2165124776 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fdbf00a14d0 0x7fdbf00a3980 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdc0810a010 tx=0x7fdbf8006c50 comp rx=0 tx=0).stop 2023-07-18T03:22:24.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.439+0000 7fdc0ddff700 1 -- 172.21.15.101:0/2165124776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc08107a50 msgr2=0x7fdc081090a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:24.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.439+0000 7fdc0ddff700 1 --2- 172.21.15.101:0/2165124776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc08107a50 0x7fdc081090a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fdc0000ef10 tx=0x7fdc0000c3a0 comp rx=0 tx=0).stop 2023-07-18T03:22:24.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.440+0000 7fdc0ddff700 1 -- 172.21.15.101:0/2165124776 shutdown_connections 2023-07-18T03:22:24.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.440+0000 7fdc0ddff700 1 --2- 172.21.15.101:0/2165124776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc08112920 0x7fdc081b9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:24.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.440+0000 7fdc0ddff700 1 --2- 172.21.15.101:0/2165124776 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdc0806d8e0 0x7fdc08108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:24.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.440+0000 7fdc0ddff700 1 --2- 172.21.15.101:0/2165124776 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fdbf00a14d0 0x7fdbf00a3980 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:24.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.440+0000 7fdc0ddff700 1 --2- 172.21.15.101:0/2165124776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc08107a50 0x7fdc081090a0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:24.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.440+0000 7fdc0ddff700 1 -- 172.21.15.101:0/2165124776 >> 172.21.15.101:0/2165124776 conn(0x7fdc0806bf70 msgr2=0x7fdc0819d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:24.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.441+0000 7fdc0ddff700 1 -- 172.21.15.101:0/2165124776 shutdown_connections 2023-07-18T03:22:24.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:24.441+0000 7fdc0ddff700 1 -- 172.21.15.101:0/2165124776 wait complete. 2023-07-18T03:22:24.973 INFO:tasks.workunit.client.0.smithi101.stdout: 10 60761 31.60 MB/sec warmup 109 sec latency 123.825 ms 2023-07-18T03:22:25.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:25 smithi179 ceph-mon[129229]: pgmap v466: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 14 MiB/s wr, 710 op/s 2023-07-18T03:22:25.476 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-18T03:22:25.476 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-18T03:22:25.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:25 smithi101 ceph-mon[126379]: pgmap v466: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 14 MiB/s wr, 710 op/s 2023-07-18T03:22:25.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:25 smithi155 ceph-mon[128885]: pgmap v466: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 14 MiB/s wr, 710 op/s 2023-07-18T03:22:25.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.752+0000 7f183cb9d700 1 -- 172.21.15.101:0/1210116617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18381003a0 msgr2=0x7f183810c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:25.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.752+0000 7f183cb9d700 1 --2- 172.21.15.101:0/1210116617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18381003a0 0x7f183810c830 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1830014a30 tx=0x7f1830014d40 comp rx=0 tx=0).stop 2023-07-18T03:22:25.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.753+0000 7f183cb9d700 1 -- 172.21.15.101:0/1210116617 shutdown_connections 2023-07-18T03:22:25.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.753+0000 7f183cb9d700 1 --2- 172.21.15.101:0/1210116617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18381003a0 0x7f183810c830 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:25.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.753+0000 7f183cb9d700 1 --2- 172.21.15.101:0/1210116617 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1838105a00 0x7f1838105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:25.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.753+0000 7f183cb9d700 1 --2- 172.21.15.101:0/1210116617 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18380ff9f0 0x7f18380ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:25.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.753+0000 7f183cb9d700 1 -- 172.21.15.101:0/1210116617 >> 172.21.15.101:0/1210116617 conn(0x7f18380fb510 msgr2=0x7f18380fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:25.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.754+0000 7f183cb9d700 1 -- 172.21.15.101:0/1210116617 shutdown_connections 2023-07-18T03:22:25.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.754+0000 7f183cb9d700 1 -- 172.21.15.101:0/1210116617 wait complete. 2023-07-18T03:22:25.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.755+0000 7f183cb9d700 1 Processor -- start 2023-07-18T03:22:25.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.755+0000 7f183cb9d700 1 -- start start 2023-07-18T03:22:25.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.755+0000 7f183cb9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18380ff9f0 0x7f1838199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:25.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.755+0000 7f183cb9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18381003a0 0x7f1838199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:25.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.755+0000 7f183cb9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1838105a00 0x7f1838193580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:25.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.755+0000 7f183cb9d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1838193bf0 con 0x7f18380ff9f0 2023-07-18T03:22:25.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.756+0000 7f183cb9d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1838193d60 con 0x7f18381003a0 2023-07-18T03:22:25.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.756+0000 7f183cb9d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1838193f20 con 0x7f1838105a00 2023-07-18T03:22:25.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.756+0000 7f1837fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1838105a00 0x7f1838193580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:25.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.756+0000 7f1837fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1838105a00 0x7f1838193580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:33238/0 (socket says 172.21.15.101:33238) 2023-07-18T03:22:25.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.756+0000 7f1837fff700 1 -- 172.21.15.101:0/985650843 learned_addr learned my addr 172.21.15.101:0/985650843 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:25.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.756+0000 7f18377fe700 1 --2- 172.21.15.101:0/985650843 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18380ff9f0 0x7f1838199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:25.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.757+0000 7f1837fff700 1 -- 172.21.15.101:0/985650843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18381003a0 msgr2=0x7f1838199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:25.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.757+0000 7f1837fff700 1 --2- 172.21.15.101:0/985650843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18381003a0 0x7f1838199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:25.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.757+0000 7f1837fff700 1 -- 172.21.15.101:0/985650843 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18380ff9f0 msgr2=0x7f1838199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:25.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.757+0000 7f1837fff700 1 --2- 172.21.15.101:0/985650843 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18380ff9f0 0x7f1838199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:25.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.757+0000 7f1837fff700 1 -- 172.21.15.101:0/985650843 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1830014660 con 0x7f1838105a00 2023-07-18T03:22:25.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.757+0000 7f1837fff700 1 --2- 172.21.15.101:0/985650843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1838105a00 0x7f1838193580 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f18300136d0 tx=0x7f1830014330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:25.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.757+0000 7f1834ff9700 1 -- 172.21.15.101:0/985650843 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1830003c10 con 0x7f1838105a00 2023-07-18T03:22:25.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.757+0000 7f1834ff9700 1 -- 172.21.15.101:0/985650843 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1830003d70 con 0x7f1838105a00 2023-07-18T03:22:25.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.757+0000 7f183cb9d700 1 -- 172.21.15.101:0/985650843 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18381941a0 con 0x7f1838105a00 2023-07-18T03:22:25.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.757+0000 7f1834ff9700 1 -- 172.21.15.101:0/985650843 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183000f210 con 0x7f1838105a00 2023-07-18T03:22:25.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.757+0000 7f183cb9d700 1 -- 172.21.15.101:0/985650843 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18381b1510 con 0x7f1838105a00 2023-07-18T03:22:25.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.763+0000 7f1834ff9700 1 -- 172.21.15.101:0/985650843 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f1830005020 con 0x7f1838105a00 2023-07-18T03:22:25.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.764+0000 7f18516fdb80 1 Processor -- start 2023-07-18T03:22:25.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.764+0000 7f18516fdb80 1 -- start start 2023-07-18T03:22:25.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.764+0000 7f18516fdb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6fb55d570 0x55c6fb55d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:25.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.764+0000 7f18516fdb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c6fb55de80 0x55c6fb567870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:25.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.764+0000 7f18516fdb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6fb567db0 0x55c6fb56a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:25.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.764+0000 7f18516fdb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6fb462d30 con 0x55c6fb55de80 2023-07-18T03:22:25.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.764+0000 7f18516fdb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6fb4b5ac0 con 0x55c6fb55d570 2023-07-18T03:22:25.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.764+0000 7f18516fdb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6fb4aef50 con 0x55c6fb567db0 2023-07-18T03:22:25.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.767+0000 7f18377fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c6fb55de80 0x55c6fb567870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:25.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.767+0000 7f18377fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c6fb55de80 0x55c6fb567870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:37476/0 (socket says 172.21.15.101:37476) 2023-07-18T03:22:25.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.767+0000 7f18377fe700 1 -- 172.21.15.101:0/1721976305 learned_addr learned my addr 172.21.15.101:0/1721976305 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:25.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.767+0000 7f1836ffd700 1 --2- 172.21.15.101:0/1721976305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6fb55d570 0x55c6fb55d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:25.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.767+0000 7f1836ffd700 1 --2- 172.21.15.101:0/1721976305 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6fb567db0 0x55c6fb56a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:25.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.767+0000 7f1834ff9700 1 --2- 172.21.15.101:0/985650843 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f18200a1540 0x7f18200a39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:25.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.767+0000 7f1834ff9700 1 -- 172.21.15.101:0/985650843 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f18300cd270 con 0x7f1838105a00 2023-07-18T03:22:25.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.767+0000 7f1836ffd700 1 -- 172.21.15.101:0/1721976305 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6fb567db0 msgr2=0x55c6fb56a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:25.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.768+0000 7f1836ffd700 1 --2- 172.21.15.101:0/1721976305 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6fb567db0 0x55c6fb56a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:25.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.768+0000 7f1836ffd700 1 -- 172.21.15.101:0/1721976305 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c6fb55de80 msgr2=0x55c6fb567870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:25.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.768+0000 7f1836ffd700 1 --2- 172.21.15.101:0/1721976305 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c6fb55de80 0x55c6fb567870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:25.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.768+0000 7f1836ffd700 1 -- 172.21.15.101:0/1721976305 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6fb2dce60 con 0x55c6fb55d570 2023-07-18T03:22:25.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.768+0000 7f1836ffd700 1 --2- 172.21.15.101:0/1721976305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6fb55d570 0x55c6fb55d940 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f182c005950 tx=0x7f182c0096b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cf27a10b28ebdbb server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:25.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.768+0000 7f1837fff700 1 --2- 172.21.15.101:0/985650843 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f18200a1540 0x7f18200a39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:25.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.768+0000 7f181dffb700 1 -- 172.21.15.101:0/1721976305 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c0130d0 con 0x55c6fb55d570 2023-07-18T03:22:25.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.769+0000 7f1837fff700 1 --2- 172.21.15.101:0/985650843 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f18200a1540 0x7f18200a39f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f18300ca610 tx=0x7f1830005410 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:25.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.770+0000 7f181dffb700 1 -- 172.21.15.101:0/1721976305 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f182c019020 con 0x55c6fb55d570 2023-07-18T03:22:25.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.771+0000 7f181dffb700 1 -- 172.21.15.101:0/1721976305 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c003bb0 con 0x55c6fb55d570 2023-07-18T03:22:25.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.772+0000 7f18516fdb80 1 -- 172.21.15.101:0/1721976305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6fb55d570 msgr2=0x55c6fb55d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:25.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.772+0000 7f18516fdb80 1 --2- 172.21.15.101:0/1721976305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6fb55d570 0x55c6fb55d940 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f182c005950 tx=0x7f182c0096b0 comp rx=0 tx=0).stop 2023-07-18T03:22:25.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.772+0000 7f18516fdb80 1 -- 172.21.15.101:0/1721976305 shutdown_connections 2023-07-18T03:22:25.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.772+0000 7f18516fdb80 1 --2- 172.21.15.101:0/1721976305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6fb55d570 0x55c6fb55d940 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:25.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.772+0000 7f18516fdb80 1 --2- 172.21.15.101:0/1721976305 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6fb567db0 0x55c6fb56a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:25.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.772+0000 7f18516fdb80 1 --2- 172.21.15.101:0/1721976305 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c6fb55de80 0x55c6fb567870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:25.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.772+0000 7f18516fdb80 1 -- 172.21.15.101:0/1721976305 >> 172.21.15.101:0/1721976305 conn(0x55c6fb553070 msgr2=0x55c6fb5589b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:25.769 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.774+0000 7f18516fdb80 1 -- 172.21.15.101:0/1721976305 shutdown_connections 2023-07-18T03:22:25.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.774+0000 7f18516fdb80 1 -- 172.21.15.101:0/1721976305 wait complete. 2023-07-18T03:22:25.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.775+0000 7f18516fdb80 1 -- start start 2023-07-18T03:22:25.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.775+0000 7f18516fdb80 1 Processor -- start 2023-07-18T03:22:25.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.776+0000 7f18516fdb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6fb55d570 0x55c6fb567780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:25.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.776+0000 7f18516fdb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6fb55de80 0x55c6fb564a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:25.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.776+0000 7f18516fdb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c6fb567db0 0x55c6fb5650f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:25.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.776+0000 7f18516fdb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6fb44eb40 con 0x55c6fb567db0 2023-07-18T03:22:25.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.776+0000 7f18516fdb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6fb44e840 con 0x55c6fb55d570 2023-07-18T03:22:25.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.776+0000 7f18516fdb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6fb3329a0 con 0x55c6fb55de80 2023-07-18T03:22:25.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.776+0000 7f18516fdb80 10 client.0 _ll_register_callbacks cb 0x55c6fb44c7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:22:25.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.777+0000 7f18377fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6fb55de80 0x55c6fb564a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:25.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.777+0000 7f18377fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6fb55de80 0x55c6fb564a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:33266/0 (socket says 172.21.15.101:33266) 2023-07-18T03:22:25.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.777+0000 7f18377fe700 1 -- 172.21.15.101:0/3140696022 learned_addr learned my addr 172.21.15.101:0/3140696022 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:25.774 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.777+0000 7f18377fe700 1 -- 172.21.15.101:0/3140696022 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6fb55d570 msgr2=0x55c6fb567780 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-18T03:22:25.774 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.777+0000 7f18377fe700 1 --2- 172.21.15.101:0/3140696022 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6fb55d570 0x55c6fb567780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:25.774 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.777+0000 7f18377fe700 1 -- 172.21.15.101:0/3140696022 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c6fb567db0 msgr2=0x55c6fb5650f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-18T03:22:25.774 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.778+0000 7f18377fe700 1 --2- 172.21.15.101:0/3140696022 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c6fb567db0 0x55c6fb5650f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:25.775 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.778+0000 7f18377fe700 1 -- 172.21.15.101:0/3140696022 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f182c013520 con 0x55c6fb55de80 2023-07-18T03:22:25.775 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.778+0000 7f18377fe700 1 --2- 172.21.15.101:0/3140696022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6fb55de80 0x55c6fb564a70 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1828005e10 tx=0x7f1828012ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab5fb7e184c648a9 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:25.775 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.778+0000 7f1816ffd700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:22:25.775 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.778+0000 7f1816ffd700 1 -- 172.21.15.101:0/3140696022 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1828017070 con 0x55c6fb55de80 2023-07-18T03:22:25.776 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.778+0000 7f1816ffd700 1 -- 172.21.15.101:0/3140696022 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f182800f540 con 0x55c6fb55de80 2023-07-18T03:22:25.776 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.779+0000 7f18516fdb80 1 -- 172.21.15.101:0/3140696022 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c6fb408570 con 0x55c6fb55de80 2023-07-18T03:22:25.776 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.780+0000 7f1816ffd700 1 -- 172.21.15.101:0/3140696022 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18280122e0 con 0x55c6fb55de80 2023-07-18T03:22:25.776 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.780+0000 7f18516fdb80 10 client.28549 fetch_fsmap learned FSMap version 39 2023-07-18T03:22:25.776 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.780+0000 7f18516fdb80 1 -- 172.21.15.101:0/3140696022 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55c6fb2c5ce0 con 0x55c6fb55de80 2023-07-18T03:22:25.777 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.780+0000 7f1816ffd700 1 -- 172.21.15.101:0/3140696022 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c6fb408570 con 0x55c6fb55de80 2023-07-18T03:22:25.777 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.781+0000 7f1816ffd700 1 -- 172.21.15.101:0/3140696022 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 39) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f1828026930 con 0x55c6fb55de80 2023-07-18T03:22:25.777 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.781+0000 7f18516fdb80 10 client.28549 fetch_fsmap finished waiting for FSMap version 39 2023-07-18T03:22:25.778 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.781+0000 7f18516fdb80 10 client.28549 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:22:25.778 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.781+0000 7f18516fdb80 20 client.28549 populate_metadata read hostname 'smithi101' 2023-07-18T03:22:25.778 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.781+0000 7f18516fdb80 1 --2- 172.21.15.101:0/3140696022 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c6fb577850 0x55c6fb579c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:25.778 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.781+0000 7f18516fdb80 4 client.28549 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:22:25.778 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.781+0000 7f18516fdb80 1 -- 172.21.15.101:0/3140696022 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6fb4a7f30 con 0x55c6fb577850 2023-07-18T03:22:25.778 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.781+0000 7f1836ffd700 1 --2- 172.21.15.101:0/3140696022 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c6fb577850 0x55c6fb579c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:25.778 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.782+0000 7f1836ffd700 1 --2- 172.21.15.101:0/3140696022 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c6fb577850 0x55c6fb579c60 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=478e1c7267a5ec3 server_cookie=2a1e81944920df48 in_seq=0 out_seq=0 2023-07-18T03:22:25.779 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.782+0000 7f1816ffd700 10 client.28549 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:22:25.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.786+0000 7f1816ffd700 1 -- 172.21.15.101:0/3140696022 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c6fb4a7f30 con 0x55c6fb577850 2023-07-18T03:22:25.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.786+0000 7f1816ffd700 10 client.28549 handle_command_reply: tid=0 2023-07-18T03:22:25.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.786+0000 7f18516fdb80 1 client.28549 shutdown 2023-07-18T03:22:25.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.786+0000 7f18516fdb80 1 -- 172.21.15.101:0/3140696022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6fb55de80 msgr2=0x55c6fb564a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:25.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.786+0000 7f18516fdb80 1 --2- 172.21.15.101:0/3140696022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6fb55de80 0x55c6fb564a70 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1828005e10 tx=0x7f1828012ad0 comp rx=0 tx=0).stop 2023-07-18T03:22:25.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.786+0000 7f18516fdb80 1 -- 172.21.15.101:0/3140696022 shutdown_connections 2023-07-18T03:22:25.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.786+0000 7f18516fdb80 1 --2- 172.21.15.101:0/3140696022 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c6fb577850 0x55c6fb579c60 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:25.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.787+0000 7f18516fdb80 1 --2- 172.21.15.101:0/3140696022 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6fb55d570 0x55c6fb567780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:25.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.787+0000 7f18516fdb80 1 --2- 172.21.15.101:0/3140696022 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c6fb567db0 0x55c6fb5650f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:25.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.787+0000 7f18516fdb80 1 --2- 172.21.15.101:0/3140696022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6fb55de80 0x55c6fb564a70 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:25.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.787+0000 7f18516fdb80 1 -- 172.21.15.101:0/3140696022 >> 172.21.15.101:0/3140696022 conn(0x55c6fb553070 msgr2=0x55c6fb5585a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:25.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.787+0000 7f1816ffd700 0 client.28549 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:22:25.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.787+0000 7f18516fdb80 1 -- 172.21.15.101:0/3140696022 shutdown_connections 2023-07-18T03:22:25.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.787+0000 7f18516fdb80 1 -- 172.21.15.101:0/3140696022 wait complete. 2023-07-18T03:22:25.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.787+0000 7f18516fdb80 20 client.28549 trim_cache size 0 max 16384 2023-07-18T03:22:25.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.815+0000 7f18516fdb80 1 Processor -- start 2023-07-18T03:22:25.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.815+0000 7f18516fdb80 1 -- start start 2023-07-18T03:22:25.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.816+0000 7f18516fdb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6fb55d570 0x55c6fb567320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:25.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.816+0000 7f18516fdb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6fb55de80 0x55c6fb564230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:25.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.816+0000 7f18516fdb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c6fb567db0 0x55c6fb564770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:25.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.816+0000 7f18516fdb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6fb55f220 con 0x55c6fb567db0 2023-07-18T03:22:25.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.816+0000 7f18516fdb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6fb55e250 con 0x55c6fb55d570 2023-07-18T03:22:25.813 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.816+0000 7f18516fdb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6fb562fd0 con 0x55c6fb55de80 2023-07-18T03:22:25.813 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.817+0000 7f1836ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6fb55d570 0x55c6fb567320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:25.813 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.817+0000 7f1836ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6fb55d570 0x55c6fb567320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47630/0 (socket says 172.21.15.101:47630) 2023-07-18T03:22:25.813 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.817+0000 7f1836ffd700 1 -- 172.21.15.101:0/2138305349 learned_addr learned my addr 172.21.15.101:0/2138305349 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:25.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.817+0000 7f18377fe700 1 --2- 172.21.15.101:0/2138305349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6fb55de80 0x55c6fb564230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:25.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.817+0000 7f1836ffd700 1 --2- 172.21.15.101:0/2138305349 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c6fb567db0 0x55c6fb564770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:25.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.817+0000 7f1836ffd700 1 -- 172.21.15.101:0/2138305349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6fb55de80 msgr2=0x55c6fb564230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:25.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.817+0000 7f1836ffd700 1 --2- 172.21.15.101:0/2138305349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6fb55de80 0x55c6fb564230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:25.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.817+0000 7f1836ffd700 1 -- 172.21.15.101:0/2138305349 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c6fb567db0 msgr2=0x55c6fb564770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:25.816 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.817+0000 7f1836ffd700 1 --2- 172.21.15.101:0/2138305349 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c6fb567db0 0x55c6fb564770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:25.816 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.817+0000 7f1836ffd700 1 -- 172.21.15.101:0/2138305349 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6fb3032d0 con 0x55c6fb55d570 2023-07-18T03:22:25.816 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.818+0000 7f1836ffd700 1 --2- 172.21.15.101:0/2138305349 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6fb55d570 0x55c6fb567320 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f182c01a000 tx=0x7f182c009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c5bf58aa13d350d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:25.816 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.818+0000 7f181e7fc700 1 -- 172.21.15.101:0/2138305349 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c013250 con 0x55c6fb55d570 2023-07-18T03:22:25.816 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.818+0000 7f181e7fc700 1 -- 172.21.15.101:0/2138305349 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f182c0133b0 con 0x55c6fb55d570 2023-07-18T03:22:25.816 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.818+0000 7f181e7fc700 1 -- 172.21.15.101:0/2138305349 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c025dc0 con 0x55c6fb55d570 2023-07-18T03:22:25.816 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.819+0000 7f18516fdb80 1 -- 172.21.15.101:0/2138305349 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6fb55d570 msgr2=0x55c6fb567320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:25.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.819+0000 7f18516fdb80 1 --2- 172.21.15.101:0/2138305349 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6fb55d570 0x55c6fb567320 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f182c01a000 tx=0x7f182c009310 comp rx=0 tx=0).stop 2023-07-18T03:22:25.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.819+0000 7f18516fdb80 1 -- 172.21.15.101:0/2138305349 shutdown_connections 2023-07-18T03:22:25.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.819+0000 7f18516fdb80 1 --2- 172.21.15.101:0/2138305349 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6fb55d570 0x55c6fb567320 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:25.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.819+0000 7f18516fdb80 1 --2- 172.21.15.101:0/2138305349 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c6fb567db0 0x55c6fb564770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:25.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.819+0000 7f18516fdb80 1 --2- 172.21.15.101:0/2138305349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6fb55de80 0x55c6fb564230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:25.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.819+0000 7f18516fdb80 1 -- 172.21.15.101:0/2138305349 >> 172.21.15.101:0/2138305349 conn(0x55c6fb553070 msgr2=0x55c6fb55fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:25.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.820+0000 7f18516fdb80 1 -- 172.21.15.101:0/2138305349 shutdown_connections 2023-07-18T03:22:25.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.820+0000 7f18516fdb80 1 -- 172.21.15.101:0/2138305349 wait complete. 2023-07-18T03:22:25.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.821+0000 7f18516fdb80 1 -- start start 2023-07-18T03:22:25.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.821+0000 7f18516fdb80 1 Processor -- start 2023-07-18T03:22:25.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.822+0000 7f18516fdb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6fb55d570 0x55c6fb56d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:25.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.822+0000 7f18516fdb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c6fb55de80 0x55c6fb56d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:25.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.822+0000 7f18516fdb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6fb567db0 0x55c6fb5cb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:25.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.822+0000 7f18516fdb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6fb56ded0 con 0x55c6fb55de80 2023-07-18T03:22:25.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.822+0000 7f18516fdb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6fb56e040 con 0x55c6fb567db0 2023-07-18T03:22:25.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.822+0000 7f18516fdb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6fb5cbc60 con 0x55c6fb55d570 2023-07-18T03:22:25.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.822+0000 7f18516fdb80 10 client.0 _ll_register_callbacks cb 0x7f1828009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:22:25.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.822+0000 7f1836ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6fb567db0 0x55c6fb5cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:25.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.822+0000 7f1836ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6fb567db0 0x55c6fb5cb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47638/0 (socket says 172.21.15.101:47638) 2023-07-18T03:22:25.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.822+0000 7f1836ffd700 1 -- 172.21.15.101:0/3589083692 learned_addr learned my addr 172.21.15.101:0/3589083692 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:25.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.823+0000 7f18377fe700 1 --2- 172.21.15.101:0/3589083692 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c6fb55de80 0x55c6fb56d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:25.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.823+0000 7f1836ffd700 1 --2- 172.21.15.101:0/3589083692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6fb55d570 0x55c6fb56d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:25.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.823+0000 7f1836ffd700 1 -- 172.21.15.101:0/3589083692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6fb55d570 msgr2=0x55c6fb56d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:25.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.823+0000 7f1836ffd700 1 --2- 172.21.15.101:0/3589083692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6fb55d570 0x55c6fb56d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:25.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.823+0000 7f1836ffd700 1 -- 172.21.15.101:0/3589083692 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c6fb55de80 msgr2=0x55c6fb56d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:25.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.823+0000 7f1836ffd700 1 --2- 172.21.15.101:0/3589083692 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c6fb55de80 0x55c6fb56d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:25.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.823+0000 7f1836ffd700 1 -- 172.21.15.101:0/3589083692 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c6fb2dce60 con 0x55c6fb567db0 2023-07-18T03:22:25.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.823+0000 7f1836ffd700 1 --2- 172.21.15.101:0/3589083692 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6fb567db0 0x55c6fb5cb720 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f182c013810 tx=0x7f182c0259a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f0264ddb04f23a2 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:25.822 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.823+0000 7f1816ffd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:22:25.822 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.824+0000 7f1816ffd700 1 -- 172.21.15.101:0/3589083692 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c0130d0 con 0x55c6fb567db0 2023-07-18T03:22:25.823 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.824+0000 7f1816ffd700 1 -- 172.21.15.101:0/3589083692 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f182c023070 con 0x55c6fb567db0 2023-07-18T03:22:25.823 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.824+0000 7f18516fdb80 1 -- 172.21.15.101:0/3589083692 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f182c019020 con 0x55c6fb567db0 2023-07-18T03:22:25.823 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.826+0000 7f1816ffd700 1 -- 172.21.15.101:0/3589083692 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f182c013230 con 0x55c6fb567db0 2023-07-18T03:22:25.823 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.826+0000 7f18516fdb80 10 client.28544 fetch_fsmap learned FSMap version 39 2023-07-18T03:22:25.823 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.826+0000 7f18516fdb80 1 -- 172.21.15.101:0/3589083692 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55c6fb2db330 con 0x55c6fb567db0 2023-07-18T03:22:25.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.828+0000 7f1816ffd700 1 -- 172.21.15.101:0/3589083692 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f182c019020 con 0x55c6fb567db0 2023-07-18T03:22:25.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.829+0000 7f1816ffd700 1 -- 172.21.15.101:0/3589083692 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 39) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f182c004830 con 0x55c6fb567db0 2023-07-18T03:22:25.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.829+0000 7f18516fdb80 10 client.28544 fetch_fsmap finished waiting for FSMap version 39 2023-07-18T03:22:25.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.829+0000 7f18516fdb80 10 client.28544 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:22:25.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.829+0000 7f18516fdb80 20 client.28544 populate_metadata read hostname 'smithi101' 2023-07-18T03:22:25.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.829+0000 7f18516fdb80 1 --2- 172.21.15.101:0/3589083692 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c6fb5ccf70 0x55c6fb5cf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:25.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.829+0000 7f18516fdb80 4 client.28544 mds_command: new command op to 24562 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-18T03:22:25.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.829+0000 7f18516fdb80 1 -- 172.21.15.101:0/3589083692 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55c6fb46a0d0 con 0x55c6fb5ccf70 2023-07-18T03:22:25.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.830+0000 7f18377fe700 1 --2- 172.21.15.101:0/3589083692 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c6fb5ccf70 0x55c6fb5cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:25.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.830+0000 7f18377fe700 1 --2- 172.21.15.101:0/3589083692 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c6fb5ccf70 0x55c6fb5cf380 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2f759648e41ad70 server_cookie=2a14e18fd9769217 in_seq=0 out_seq=0 2023-07-18T03:22:25.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.830+0000 7f1816ffd700 10 client.28544 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:22:25.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.831+0000 7f1816ffd700 1 -- 172.21.15.101:0/3589083692 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c6fb46a0d0 con 0x55c6fb5ccf70 2023-07-18T03:22:25.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.831+0000 7f1816ffd700 10 client.28544 handle_command_reply: tid=0 2023-07-18T03:22:25.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.832+0000 7f18516fdb80 1 client.28544 shutdown 2023-07-18T03:22:25.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.836+0000 7f18516fdb80 1 -- 172.21.15.101:0/3589083692 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6fb567db0 msgr2=0x55c6fb5cb720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:25.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.836+0000 7f18516fdb80 1 --2- 172.21.15.101:0/3589083692 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6fb567db0 0x55c6fb5cb720 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f182c013810 tx=0x7f182c0259a0 comp rx=0 tx=0).stop 2023-07-18T03:22:25.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.836+0000 7f18516fdb80 1 -- 172.21.15.101:0/3589083692 shutdown_connections 2023-07-18T03:22:25.832 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.836+0000 7f18516fdb80 1 --2- 172.21.15.101:0/3589083692 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c6fb5ccf70 0x55c6fb5cf380 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:25.832 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.836+0000 7f18516fdb80 1 --2- 172.21.15.101:0/3589083692 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6fb567db0 0x55c6fb5cb720 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:25.832 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.836+0000 7f18516fdb80 1 --2- 172.21.15.101:0/3589083692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6fb55d570 0x55c6fb56d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:25.832 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.836+0000 7f18516fdb80 1 --2- 172.21.15.101:0/3589083692 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c6fb55de80 0x55c6fb56d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:25.832 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.836+0000 7f18516fdb80 1 -- 172.21.15.101:0/3589083692 >> 172.21.15.101:0/3589083692 conn(0x55c6fb553070 msgr2=0x55c6fb55c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:25.832 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.836+0000 7f1816ffd700 0 client.28544 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:22:25.832 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.836+0000 7f18516fdb80 1 -- 172.21.15.101:0/3589083692 shutdown_connections 2023-07-18T03:22:25.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.836+0000 7f18516fdb80 1 -- 172.21.15.101:0/3589083692 wait complete. 2023-07-18T03:22:25.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.836+0000 7f18516fdb80 20 client.28544 trim_cache size 0 max 16384 2023-07-18T03:22:25.833 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:22:25.834 INFO:teuthology.orchestra.run.smithi101.stdout: "return_code": 0, 2023-07-18T03:22:25.834 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_tag": "a451cabe-c0bc-4cb3-a51c-c222f55c387b", 2023-07-18T03:22:25.834 INFO:teuthology.orchestra.run.smithi101.stdout: "mode": "asynchronous" 2023-07-18T03:22:25.834 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:22:25.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.838+0000 7f183cb9d700 1 -- 172.21.15.101:0/985650843 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f18200a1540 msgr2=0x7f18200a39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:25.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.838+0000 7f183cb9d700 1 --2- 172.21.15.101:0/985650843 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f18200a1540 0x7f18200a39f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f18300ca610 tx=0x7f1830005410 comp rx=0 tx=0).stop 2023-07-18T03:22:25.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.838+0000 7f183cb9d700 1 -- 172.21.15.101:0/985650843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1838105a00 msgr2=0x7f1838193580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:25.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.838+0000 7f183cb9d700 1 --2- 172.21.15.101:0/985650843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1838105a00 0x7f1838193580 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f18300136d0 tx=0x7f1830014330 comp rx=0 tx=0).stop 2023-07-18T03:22:25.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.838+0000 7f183cb9d700 1 -- 172.21.15.101:0/985650843 shutdown_connections 2023-07-18T03:22:25.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.838+0000 7f183cb9d700 1 --2- 172.21.15.101:0/985650843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1838105a00 0x7f1838193580 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:25.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.838+0000 7f183cb9d700 1 --2- 172.21.15.101:0/985650843 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18380ff9f0 0x7f1838199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:25.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.838+0000 7f183cb9d700 1 --2- 172.21.15.101:0/985650843 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f18200a1540 0x7f18200a39f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:25.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.838+0000 7f183cb9d700 1 --2- 172.21.15.101:0/985650843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18381003a0 0x7f1838199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:25.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.838+0000 7f183cb9d700 1 -- 172.21.15.101:0/985650843 >> 172.21.15.101:0/985650843 conn(0x7f18380fb510 msgr2=0x7f1838106850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:25.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.839+0000 7f183cb9d700 1 -- 172.21.15.101:0/985650843 shutdown_connections 2023-07-18T03:22:25.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:25.839+0000 7f183cb9d700 1 -- 172.21.15.101:0/985650843 wait complete. 2023-07-18T03:22:25.861 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:22:25.973 INFO:tasks.workunit.client.0.smithi101.stdout: 10 61659 31.79 MB/sec warmup 110 sec latency 63.754 ms 2023-07-18T03:22:26.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.186+0000 7fd1ebfff700 1 -- 172.21.15.101:0/3323356418 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1e40fe9d0 msgr2=0x7fd1e40fee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:26.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.186+0000 7fd1ebfff700 1 --2- 172.21.15.101:0/3323356418 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1e40fe9d0 0x7fd1e40fee40 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4009990 tx=0x7fd1d4009ca0 comp rx=0 tx=0).stop 2023-07-18T03:22:26.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.188+0000 7fd1ebfff700 1 -- 172.21.15.101:0/3323356418 shutdown_connections 2023-07-18T03:22:26.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.188+0000 7fd1ebfff700 1 --2- 172.21.15.101:0/3323356418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1e41049e0 0x7fd1e4104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:26.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.188+0000 7fd1ebfff700 1 --2- 172.21.15.101:0/3323356418 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd1e40ff380 0x7fd1e410b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:26.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.188+0000 7fd1ebfff700 1 --2- 172.21.15.101:0/3323356418 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1e40fe9d0 0x7fd1e40fee40 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:26.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.188+0000 7fd1ebfff700 1 -- 172.21.15.101:0/3323356418 >> 172.21.15.101:0/3323356418 conn(0x7fd1e40fa4b0 msgr2=0x7fd1e40fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:26.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.188+0000 7fd1ebfff700 1 -- 172.21.15.101:0/3323356418 shutdown_connections 2023-07-18T03:22:26.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.188+0000 7fd1ebfff700 1 -- 172.21.15.101:0/3323356418 wait complete. 2023-07-18T03:22:26.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.190+0000 7fd1ebfff700 1 Processor -- start 2023-07-18T03:22:26.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.190+0000 7fd1ebfff700 1 -- start start 2023-07-18T03:22:26.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.190+0000 7fd1ebfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1e40fe9d0 0x7fd1e41005e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:26.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.191+0000 7fd1e9d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1e40fe9d0 0x7fd1e41005e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:26.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.191+0000 7fd1e9d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1e40fe9d0 0x7fd1e41005e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:33168/0 (socket says 172.21.15.101:33168) 2023-07-18T03:22:26.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.191+0000 7fd1ebfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd1e40ff380 0x7fd1e4100b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:26.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.191+0000 7fd1ebfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1e41049e0 0x7fd1e41a3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:26.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.191+0000 7fd1ebfff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e41013c0 con 0x7fd1e40ff380 2023-07-18T03:22:26.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.191+0000 7fd1ebfff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e41a4440 con 0x7fd1e40fe9d0 2023-07-18T03:22:26.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.191+0000 7fd1ebfff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e41a4580 con 0x7fd1e41049e0 2023-07-18T03:22:26.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.192+0000 7fd1e9d9b700 1 -- 172.21.15.101:0/1426410047 learned_addr learned my addr 172.21.15.101:0/1426410047 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:26.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.192+0000 7fd1ea59c700 1 --2- 172.21.15.101:0/1426410047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1e41049e0 0x7fd1e41a3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:26.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.193+0000 7fd1ea59c700 1 -- 172.21.15.101:0/1426410047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1e40fe9d0 msgr2=0x7fd1e41005e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:26.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.193+0000 7fd1ea59c700 1 --2- 172.21.15.101:0/1426410047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1e40fe9d0 0x7fd1e41005e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:26.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.193+0000 7fd1ea59c700 1 -- 172.21.15.101:0/1426410047 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd1e40ff380 msgr2=0x7fd1e4100b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:26.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.193+0000 7fd1ea59c700 1 --2- 172.21.15.101:0/1426410047 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd1e40ff380 0x7fd1e4100b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:26.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.193+0000 7fd1ea59c700 1 -- 172.21.15.101:0/1426410047 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d40095f0 con 0x7fd1e41049e0 2023-07-18T03:22:26.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.193+0000 7fd1ea59c700 1 --2- 172.21.15.101:0/1426410047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1e41049e0 0x7fd1e41a3f00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc00be20 tx=0x7fd1dc00bf00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:26.192 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.193+0000 7fd1daffd700 1 -- 172.21.15.101:0/1426410047 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc00cd10 con 0x7fd1e41049e0 2023-07-18T03:22:26.192 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.193+0000 7fd1daffd700 1 -- 172.21.15.101:0/1426410047 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1dc00ce70 con 0x7fd1e41049e0 2023-07-18T03:22:26.192 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.193+0000 7fd1daffd700 1 -- 172.21.15.101:0/1426410047 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc012840 con 0x7fd1e41049e0 2023-07-18T03:22:26.192 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.196+0000 7fd1ebfff700 1 -- 172.21.15.101:0/1426410047 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1e41a4830 con 0x7fd1e41049e0 2023-07-18T03:22:26.192 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.196+0000 7fd1ebfff700 1 -- 172.21.15.101:0/1426410047 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1e41a4cf0 con 0x7fd1e41049e0 2023-07-18T03:22:26.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.203+0000 7fd1ffa0db80 1 Processor -- start 2023-07-18T03:22:26.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.204+0000 7fd1daffd700 1 -- 172.21.15.101:0/1426410047 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fd1dc0129a0 con 0x7fd1e41049e0 2023-07-18T03:22:26.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.204+0000 7fd1ffa0db80 1 -- start start 2023-07-18T03:22:26.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.204+0000 7fd1ffa0db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563abaf2bae0 0x563abaf2beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:26.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.204+0000 7fd1ffa0db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abaf2c3f0 0x563abaf36da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:26.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.204+0000 7fd1ffa0db80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563abaf372e0 0x563abaf396c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:26.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.204+0000 7fd1ffa0db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563abae29cb0 con 0x563abaf2c3f0 2023-07-18T03:22:26.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.204+0000 7fd1ffa0db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563abae7ca40 con 0x563abaf2bae0 2023-07-18T03:22:26.201 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.204+0000 7fd1ffa0db80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563abae75ed0 con 0x563abaf372e0 2023-07-18T03:22:26.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.205+0000 7fd1e959a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563abaf2bae0 0x563abaf2beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:26.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.205+0000 7fd1e959a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563abaf2bae0 0x563abaf2beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:33178/0 (socket says 172.21.15.101:33178) 2023-07-18T03:22:26.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.205+0000 7fd1e959a700 1 -- 172.21.15.101:0/2034353004 learned_addr learned my addr 172.21.15.101:0/2034353004 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:26.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.207+0000 7fd1e9d9b700 1 --2- 172.21.15.101:0/2034353004 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abaf2c3f0 0x563abaf36da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:26.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.207+0000 7fd1e959a700 1 -- 172.21.15.101:0/2034353004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563abaf372e0 msgr2=0x563abaf396c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:26.205 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.207+0000 7fd1e959a700 1 --2- 172.21.15.101:0/2034353004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563abaf372e0 0x563abaf396c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:26.205 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.207+0000 7fd1e959a700 1 -- 172.21.15.101:0/2034353004 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abaf2c3f0 msgr2=0x563abaf36da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:26.205 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.207+0000 7fd1e959a700 1 --2- 172.21.15.101:0/2034353004 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abaf2c3f0 0x563abaf36da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:26.205 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.207+0000 7fd1e959a700 1 -- 172.21.15.101:0/2034353004 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563abaca3de0 con 0x563abaf2bae0 2023-07-18T03:22:26.205 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.208+0000 7fd1daffd700 1 --2- 172.21.15.101:0/1426410047 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd1d00a1340 0x7fd1d00a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:26.205 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.209+0000 7fd1daffd700 1 -- 172.21.15.101:0/1426410047 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fd1dc006e40 con 0x7fd1e41049e0 2023-07-18T03:22:26.205 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.209+0000 7fd1e959a700 1 --2- 172.21.15.101:0/2034353004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563abaf2bae0 0x563abaf2beb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd1d400b5c0 tx=0x7fd1d4003680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efb3251bf5c9d61a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:26.206 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.209+0000 7fd1cffff700 1 -- 172.21.15.101:0/2034353004 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d400b7d0 con 0x563abaf2bae0 2023-07-18T03:22:26.206 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.210+0000 7fd1cffff700 1 -- 172.21.15.101:0/2034353004 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1d400b930 con 0x563abaf2bae0 2023-07-18T03:22:26.206 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.210+0000 7fd1cffff700 1 -- 172.21.15.101:0/2034353004 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d4017650 con 0x563abaf2bae0 2023-07-18T03:22:26.207 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.210+0000 7fd1ffa0db80 1 -- 172.21.15.101:0/2034353004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563abaf2bae0 msgr2=0x563abaf2beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:26.208 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.210+0000 7fd1ffa0db80 1 --2- 172.21.15.101:0/2034353004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563abaf2bae0 0x563abaf2beb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd1d400b5c0 tx=0x7fd1d4003680 comp rx=0 tx=0).stop 2023-07-18T03:22:26.208 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.211+0000 7fd1ffa0db80 1 -- 172.21.15.101:0/2034353004 shutdown_connections 2023-07-18T03:22:26.208 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.211+0000 7fd1ffa0db80 1 --2- 172.21.15.101:0/2034353004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563abaf2bae0 0x563abaf2beb0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:26.208 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.211+0000 7fd1ffa0db80 1 --2- 172.21.15.101:0/2034353004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563abaf372e0 0x563abaf396c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:26.208 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.211+0000 7fd1ffa0db80 1 --2- 172.21.15.101:0/2034353004 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abaf2c3f0 0x563abaf36da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:26.208 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.211+0000 7fd1ffa0db80 1 -- 172.21.15.101:0/2034353004 >> 172.21.15.101:0/2034353004 conn(0x563abaf19da0 msgr2=0x563abaf1f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:26.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.213+0000 7fd1e9d9b700 1 --2- 172.21.15.101:0/1426410047 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd1d00a1340 0x7fd1d00a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:26.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.213+0000 7fd1ffa0db80 1 -- 172.21.15.101:0/2034353004 shutdown_connections 2023-07-18T03:22:26.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.213+0000 7fd1ffa0db80 1 -- 172.21.15.101:0/2034353004 wait complete. 2023-07-18T03:22:26.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.214+0000 7fd1ffa0db80 1 -- start start 2023-07-18T03:22:26.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.214+0000 7fd1ffa0db80 1 Processor -- start 2023-07-18T03:22:26.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.215+0000 7fd1ffa0db80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563abaf2bae0 0x563abaf34040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:26.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.215+0000 7fd1ffa0db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abaf2c3f0 0x563abaf34580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:26.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.215+0000 7fd1ffa0db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563abaf372e0 0x563abaf34ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:26.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.215+0000 7fd1ffa0db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563abae15ac0 con 0x563abaf2c3f0 2023-07-18T03:22:26.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.215+0000 7fd1ffa0db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563abae157c0 con 0x563abaf372e0 2023-07-18T03:22:26.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.215+0000 7fd1ffa0db80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563abacf9920 con 0x563abaf2bae0 2023-07-18T03:22:26.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.215+0000 7fd1ffa0db80 10 client.0 _ll_register_callbacks cb 0x563abae13760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:22:26.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.216+0000 7fd1e9d9b700 1 --2- 172.21.15.101:0/1426410047 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd1d00a1340 0x7fd1d00a37f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd1e000ba10 tx=0x7fd1e0008040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:26.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.216+0000 7fd1e959a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abaf2c3f0 0x563abaf34580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:26.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.216+0000 7fd1e959a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abaf2c3f0 0x563abaf34580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:53230/0 (socket says 172.21.15.101:53230) 2023-07-18T03:22:26.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.216+0000 7fd1e959a700 1 -- 172.21.15.101:0/3199322216 learned_addr learned my addr 172.21.15.101:0/3199322216 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:26.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.216+0000 7fd1ea59c700 1 --2- 172.21.15.101:0/3199322216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563abaf372e0 0x563abaf34ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:26.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.217+0000 7fd1ea59c700 1 -- 172.21.15.101:0/3199322216 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563abaf2bae0 msgr2=0x563abaf34040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:26.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.217+0000 7fd1ea59c700 1 --2- 172.21.15.101:0/3199322216 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563abaf2bae0 0x563abaf34040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:26.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.217+0000 7fd1ea59c700 1 -- 172.21.15.101:0/3199322216 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abaf2c3f0 msgr2=0x563abaf34580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:26.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.217+0000 7fd1ea59c700 1 --2- 172.21.15.101:0/3199322216 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abaf2c3f0 0x563abaf34580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:26.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.217+0000 7fd1ea59c700 1 -- 172.21.15.101:0/3199322216 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1e410c110 con 0x563abaf372e0 2023-07-18T03:22:26.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.218+0000 7fd1ea59c700 1 --2- 172.21.15.101:0/3199322216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563abaf372e0 0x563abaf34ac0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc0c9b00 tx=0x7fd1dc098ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcfff2999b24c42b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:26.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.218+0000 7fd1cd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:22:26.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.218+0000 7fd1cd7fa700 1 -- 172.21.15.101:0/3199322216 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc0a18d0 con 0x563abaf372e0 2023-07-18T03:22:26.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.218+0000 7fd1cd7fa700 1 -- 172.21.15.101:0/3199322216 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fd1dc014b50 con 0x563abaf372e0 2023-07-18T03:22:26.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.218+0000 7fd1ffa0db80 1 -- 172.21.15.101:0/3199322216 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563abadcf4f0 con 0x563abaf372e0 2023-07-18T03:22:26.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.220+0000 7fd1cd7fa700 1 -- 172.21.15.101:0/3199322216 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1dc09d120 con 0x563abaf372e0 2023-07-18T03:22:26.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.222+0000 7fd1ffa0db80 10 client.28553 fetch_fsmap learned FSMap version 39 2023-07-18T03:22:26.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.222+0000 7fd1ffa0db80 1 -- 172.21.15.101:0/3199322216 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x563abac8cc60 con 0x563abaf372e0 2023-07-18T03:22:26.218 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.222+0000 7fd1cd7fa700 1 -- 172.21.15.101:0/3199322216 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563abadcf4f0 con 0x563abaf372e0 2023-07-18T03:22:26.218 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.223+0000 7fd1cd7fa700 1 -- 172.21.15.101:0/3199322216 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 39) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fd1dc0b0050 con 0x563abaf372e0 2023-07-18T03:22:26.218 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.223+0000 7fd1ffa0db80 10 client.28553 fetch_fsmap finished waiting for FSMap version 39 2023-07-18T03:22:26.218 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.223+0000 7fd1ffa0db80 10 client.28553 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:22:26.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.223+0000 7fd1ffa0db80 20 client.28553 populate_metadata read hostname 'smithi101' 2023-07-18T03:22:26.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.223+0000 7fd1ffa0db80 1 --2- 172.21.15.101:0/3199322216 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563abaf25200 0x563abaf27610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:26.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.223+0000 7fd1ffa0db80 4 client.28553 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:22:26.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.223+0000 7fd1ffa0db80 1 -- 172.21.15.101:0/3199322216 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563abae6eeb0 con 0x563abaf25200 2023-07-18T03:22:26.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.223+0000 7fd1e9d9b700 1 --2- 172.21.15.101:0/3199322216 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563abaf25200 0x563abaf27610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:26.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.224+0000 7fd1e9d9b700 1 --2- 172.21.15.101:0/3199322216 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563abaf25200 0x563abaf27610 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d3ad49d0dd07ada server_cookie=1305a2c961ee7fe9 in_seq=0 out_seq=0 2023-07-18T03:22:26.220 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.224+0000 7fd1cd7fa700 10 client.28553 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:22:26.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.225+0000 7fd1cd7fa700 1 -- 172.21.15.101:0/3199322216 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563abae6eeb0 con 0x563abaf25200 2023-07-18T03:22:26.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.225+0000 7fd1cd7fa700 10 client.28553 handle_command_reply: tid=0 2023-07-18T03:22:26.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.228+0000 7fd1ffa0db80 1 client.28553 shutdown 2023-07-18T03:22:26.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.229+0000 7fd1ffa0db80 1 -- 172.21.15.101:0/3199322216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563abaf372e0 msgr2=0x563abaf34ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:26.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.229+0000 7fd1ffa0db80 1 --2- 172.21.15.101:0/3199322216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563abaf372e0 0x563abaf34ac0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc0c9b00 tx=0x7fd1dc098ed0 comp rx=0 tx=0).stop 2023-07-18T03:22:26.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.230+0000 7fd1ffa0db80 1 -- 172.21.15.101:0/3199322216 shutdown_connections 2023-07-18T03:22:26.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.230+0000 7fd1ffa0db80 1 --2- 172.21.15.101:0/3199322216 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563abaf25200 0x563abaf27610 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:26.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.230+0000 7fd1ffa0db80 1 --2- 172.21.15.101:0/3199322216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563abaf372e0 0x563abaf34ac0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:26.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.230+0000 7fd1ffa0db80 1 --2- 172.21.15.101:0/3199322216 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563abaf2bae0 0x563abaf34040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:26.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.230+0000 7fd1ffa0db80 1 --2- 172.21.15.101:0/3199322216 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abaf2c3f0 0x563abaf34580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:26.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.230+0000 7fd1ffa0db80 1 -- 172.21.15.101:0/3199322216 >> 172.21.15.101:0/3199322216 conn(0x563abaf19da0 msgr2=0x563abaf1f230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:26.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.230+0000 7fd1cd7fa700 0 client.28553 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:22:26.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.231+0000 7fd1ffa0db80 1 -- 172.21.15.101:0/3199322216 shutdown_connections 2023-07-18T03:22:26.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.231+0000 7fd1ffa0db80 1 -- 172.21.15.101:0/3199322216 wait complete. 2023-07-18T03:22:26.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.233+0000 7fd1ffa0db80 20 client.28553 trim_cache size 0 max 16384 2023-07-18T03:22:26.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.267+0000 7fd1ffa0db80 1 Processor -- start 2023-07-18T03:22:26.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.268+0000 7fd1ffa0db80 1 -- start start 2023-07-18T03:22:26.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.268+0000 7fd1ffa0db80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563abaf1e170 0x563abaf36850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:26.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.268+0000 7fd1ffa0db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563abaf19da0 0x563abaf338a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:26.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.268+0000 7fd1ffa0db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abaf2bae0 0x563abaf3dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:26.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.268+0000 7fd1ffa0db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563abaf22810 con 0x563abaf2bae0 2023-07-18T03:22:26.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.268+0000 7fd1ffa0db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563abaf35620 con 0x563abaf19da0 2023-07-18T03:22:26.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.268+0000 7fd1ffa0db80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563abaf16fb0 con 0x563abaf1e170 2023-07-18T03:22:26.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.271+0000 7fd1e9d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563abaf1e170 0x563abaf36850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:26.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.271+0000 7fd1e9d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563abaf1e170 0x563abaf36850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:34274/0 (socket says 172.21.15.101:34274) 2023-07-18T03:22:26.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.271+0000 7fd1e9d9b700 1 -- 172.21.15.101:0/580581811 learned_addr learned my addr 172.21.15.101:0/580581811 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:26.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.271+0000 7fd1e9d9b700 1 -- 172.21.15.101:0/580581811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563abaf19da0 msgr2=0x563abaf338a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-18T03:22:26.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.271+0000 7fd1e9d9b700 1 --2- 172.21.15.101:0/580581811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563abaf19da0 0x563abaf338a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:26.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.271+0000 7fd1e9d9b700 1 -- 172.21.15.101:0/580581811 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abaf2bae0 msgr2=0x563abaf3dcb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-18T03:22:26.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.271+0000 7fd1e9d9b700 1 --2- 172.21.15.101:0/580581811 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abaf2bae0 0x563abaf3dcb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:26.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.271+0000 7fd1e9d9b700 1 -- 172.21.15.101:0/580581811 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563abacca250 con 0x563abaf1e170 2023-07-18T03:22:26.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.272+0000 7fd1e9d9b700 1 --2- 172.21.15.101:0/580581811 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563abaf1e170 0x563abaf36850 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd1e000b690 tx=0x7fd1e0004860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69a7f20f7b6eaf7 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:26.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.272+0000 7fd1d8ff9700 1 -- 172.21.15.101:0/580581811 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e0024e80 con 0x563abaf1e170 2023-07-18T03:22:26.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.272+0000 7fd1d8ff9700 1 -- 172.21.15.101:0/580581811 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1e0024500 con 0x563abaf1e170 2023-07-18T03:22:26.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.272+0000 7fd1d8ff9700 1 -- 172.21.15.101:0/580581811 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e002dd80 con 0x563abaf1e170 2023-07-18T03:22:26.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.273+0000 7fd1ffa0db80 1 -- 172.21.15.101:0/580581811 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563abaf1e170 msgr2=0x563abaf36850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:26.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.273+0000 7fd1ffa0db80 1 --2- 172.21.15.101:0/580581811 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563abaf1e170 0x563abaf36850 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd1e000b690 tx=0x7fd1e0004860 comp rx=0 tx=0).stop 2023-07-18T03:22:26.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.276+0000 7fd1ffa0db80 1 -- 172.21.15.101:0/580581811 shutdown_connections 2023-07-18T03:22:26.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.276+0000 7fd1ffa0db80 1 --2- 172.21.15.101:0/580581811 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abaf2bae0 0x563abaf3dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:26.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.276+0000 7fd1ffa0db80 1 --2- 172.21.15.101:0/580581811 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563abaf1e170 0x563abaf36850 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:26.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.276+0000 7fd1ffa0db80 1 --2- 172.21.15.101:0/580581811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563abaf19da0 0x563abaf338a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:26.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.276+0000 7fd1ffa0db80 1 -- 172.21.15.101:0/580581811 >> 172.21.15.101:0/580581811 conn(0x563abaf427c0 msgr2=0x563abaf34000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:26.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.277+0000 7fd1ffa0db80 1 -- 172.21.15.101:0/580581811 shutdown_connections 2023-07-18T03:22:26.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.277+0000 7fd1ffa0db80 1 -- 172.21.15.101:0/580581811 wait complete. 2023-07-18T03:22:26.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.277+0000 7fd1ffa0db80 1 -- start start 2023-07-18T03:22:26.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.278+0000 7fd1ffa0db80 1 Processor -- start 2023-07-18T03:22:26.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.280+0000 7fd1ffa0db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563abaf19da0 0x563abaf9d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:26.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.280+0000 7fd1ffa0db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abaf1e170 0x563abaf9d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:26.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.280+0000 7fd1ffa0db80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563abaf2bae0 0x563abaf30ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:26.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.280+0000 7fd1ffa0db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563abaf31570 con 0x563abaf1e170 2023-07-18T03:22:26.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.280+0000 7fd1ffa0db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563abaf316e0 con 0x563abaf19da0 2023-07-18T03:22:26.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.280+0000 7fd1ffa0db80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563abaf318a0 con 0x563abaf2bae0 2023-07-18T03:22:26.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.280+0000 7fd1ffa0db80 10 client.0 _ll_register_callbacks cb 0x7fd1dc0992d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:22:26.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.280+0000 7fd1e959a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abaf1e170 0x563abaf9d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:26.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.280+0000 7fd1e959a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abaf1e170 0x563abaf9d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:53244/0 (socket says 172.21.15.101:53244) 2023-07-18T03:22:26.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.280+0000 7fd1e959a700 1 -- 172.21.15.101:0/58306444 learned_addr learned my addr 172.21.15.101:0/58306444 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:26.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.281+0000 7fd1e959a700 1 -- 172.21.15.101:0/58306444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563abaf2bae0 msgr2=0x563abaf30ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:26.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.281+0000 7fd1e959a700 1 --2- 172.21.15.101:0/58306444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563abaf2bae0 0x563abaf30ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:26.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.281+0000 7fd1e959a700 1 -- 172.21.15.101:0/58306444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563abaf19da0 msgr2=0x563abaf9d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:26.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.281+0000 7fd1e959a700 1 --2- 172.21.15.101:0/58306444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563abaf19da0 0x563abaf9d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:26.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.281+0000 7fd1e959a700 1 -- 172.21.15.101:0/58306444 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1e000e040 con 0x563abaf1e170 2023-07-18T03:22:26.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.281+0000 7fd1e959a700 1 --2- 172.21.15.101:0/58306444 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abaf1e170 0x563abaf9d6c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fd1d400fda0 tx=0x7fd1d400b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1eecef0e2f08c7e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:26.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.281+0000 7fd1cd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:22:26.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.282+0000 7fd1cd7fa700 1 -- 172.21.15.101:0/58306444 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d4004710 con 0x563abaf1e170 2023-07-18T03:22:26.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.282+0000 7fd1cd7fa700 1 -- 172.21.15.101:0/58306444 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fd1d4004870 con 0x563abaf1e170 2023-07-18T03:22:26.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.282+0000 7fd1ffa0db80 1 -- 172.21.15.101:0/58306444 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd1d400b930 con 0x563abaf1e170 2023-07-18T03:22:26.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.282+0000 7fd1cd7fa700 1 -- 172.21.15.101:0/58306444 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1d4004a30 con 0x563abaf1e170 2023-07-18T03:22:26.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.282+0000 7fd1ffa0db80 10 client.18477 fetch_fsmap learned FSMap version 39 2023-07-18T03:22:26.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.282+0000 7fd1ffa0db80 1 -- 172.21.15.101:0/58306444 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x563abaca22b0 con 0x563abaf1e170 2023-07-18T03:22:26.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.283+0000 7fd1cd7fa700 1 -- 172.21.15.101:0/58306444 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1d400b930 con 0x563abaf1e170 2023-07-18T03:22:26.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.284+0000 7fd1cd7fa700 1 -- 172.21.15.101:0/58306444 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 39) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fd1d4024440 con 0x563abaf1e170 2023-07-18T03:22:26.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.284+0000 7fd1ffa0db80 10 client.18477 fetch_fsmap finished waiting for FSMap version 39 2023-07-18T03:22:26.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.284+0000 7fd1ffa0db80 10 client.18477 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:22:26.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.284+0000 7fd1ffa0db80 20 client.18477 populate_metadata read hostname 'smithi101' 2023-07-18T03:22:26.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.284+0000 7fd1ffa0db80 1 --2- 172.21.15.101:0/58306444 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563abaf2c3f0 0x563abafb7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:26.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.284+0000 7fd1ffa0db80 4 client.18477 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:22:26.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.284+0000 7fd1ffa0db80 1 -- 172.21.15.101:0/58306444 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563abae31050 con 0x563abaf2c3f0 2023-07-18T03:22:26.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.284+0000 7fd1e9d9b700 1 --2- 172.21.15.101:0/58306444 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563abaf2c3f0 0x563abafb7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:26.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.285+0000 7fd1e9d9b700 1 --2- 172.21.15.101:0/58306444 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563abaf2c3f0 0x563abafb7ab0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c501e882eca123f server_cookie=d5ec3b093882592b in_seq=0 out_seq=0 2023-07-18T03:22:26.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.285+0000 7fd1cd7fa700 10 client.18477 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:22:26.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.286+0000 7fd1cd7fa700 1 -- 172.21.15.101:0/58306444 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x563abae31050 con 0x563abaf2c3f0 2023-07-18T03:22:26.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.286+0000 7fd1cd7fa700 10 client.18477 handle_command_reply: tid=0 2023-07-18T03:22:26.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.286+0000 7fd1ffa0db80 1 client.18477 shutdown 2023-07-18T03:22:26.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.290+0000 7fd1ffa0db80 1 -- 172.21.15.101:0/58306444 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abaf1e170 msgr2=0x563abaf9d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:26.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.290+0000 7fd1ffa0db80 1 --2- 172.21.15.101:0/58306444 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abaf1e170 0x563abaf9d6c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fd1d400fda0 tx=0x7fd1d400b830 comp rx=0 tx=0).stop 2023-07-18T03:22:26.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.290+0000 7fd1ffa0db80 1 -- 172.21.15.101:0/58306444 shutdown_connections 2023-07-18T03:22:26.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.290+0000 7fd1ffa0db80 1 --2- 172.21.15.101:0/58306444 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563abaf2c3f0 0x563abafb7ab0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:26.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.291+0000 7fd1ffa0db80 1 --2- 172.21.15.101:0/58306444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563abaf19da0 0x563abaf9d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:26.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.291+0000 7fd1ffa0db80 1 --2- 172.21.15.101:0/58306444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563abaf2bae0 0x563abaf30ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:26.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.291+0000 7fd1ffa0db80 1 --2- 172.21.15.101:0/58306444 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563abaf1e170 0x563abaf9d6c0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:26.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.291+0000 7fd1ffa0db80 1 -- 172.21.15.101:0/58306444 >> 172.21.15.101:0/58306444 conn(0x563abaf427c0 msgr2=0x563abaf2e240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:26.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.291+0000 7fd1cd7fa700 0 client.18477 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:22:26.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.292+0000 7fd1ffa0db80 1 -- 172.21.15.101:0/58306444 shutdown_connections 2023-07-18T03:22:26.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.292+0000 7fd1ffa0db80 1 -- 172.21.15.101:0/58306444 wait complete. 2023-07-18T03:22:26.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.293+0000 7fd1ffa0db80 20 client.18477 trim_cache size 0 max 16384 2023-07-18T03:22:26.289 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:22:26.289 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-18T03:22:26.289 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:22:26.289 INFO:teuthology.orchestra.run.smithi101.stdout: "a451cabe-c0bc-4cb3-a51c-c222f55c387b": { 2023-07-18T03:22:26.289 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:22:26.289 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "a451cabe-c0bc-4cb3-a51c-c222f55c387b", 2023-07-18T03:22:26.290 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:22:26.290 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:22:26.290 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:22:26.290 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:22:26.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.293+0000 7fd1ebfff700 1 -- 172.21.15.101:0/1426410047 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd1d00a1340 msgr2=0x7fd1d00a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:26.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.293+0000 7fd1ebfff700 1 --2- 172.21.15.101:0/1426410047 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd1d00a1340 0x7fd1d00a37f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd1e000ba10 tx=0x7fd1e0008040 comp rx=0 tx=0).stop 2023-07-18T03:22:26.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.293+0000 7fd1ebfff700 1 -- 172.21.15.101:0/1426410047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1e41049e0 msgr2=0x7fd1e41a3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:26.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.293+0000 7fd1ebfff700 1 --2- 172.21.15.101:0/1426410047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1e41049e0 0x7fd1e41a3f00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc00be20 tx=0x7fd1dc00bf00 comp rx=0 tx=0).stop 2023-07-18T03:22:26.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.296+0000 7fd1ebfff700 1 -- 172.21.15.101:0/1426410047 shutdown_connections 2023-07-18T03:22:26.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.296+0000 7fd1ebfff700 1 --2- 172.21.15.101:0/1426410047 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd1d00a1340 0x7fd1d00a37f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:26.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.296+0000 7fd1ebfff700 1 --2- 172.21.15.101:0/1426410047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1e40fe9d0 0x7fd1e41005e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:26.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.296+0000 7fd1ebfff700 1 --2- 172.21.15.101:0/1426410047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1e41049e0 0x7fd1e41a3f00 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:26.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.296+0000 7fd1ebfff700 1 --2- 172.21.15.101:0/1426410047 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd1e40ff380 0x7fd1e4100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:26.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.296+0000 7fd1ebfff700 1 -- 172.21.15.101:0/1426410047 >> 172.21.15.101:0/1426410047 conn(0x7fd1e40fa4b0 msgr2=0x7fd1e40fbec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:26.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.297+0000 7fd1ebfff700 1 -- 172.21.15.101:0/1426410047 shutdown_connections 2023-07-18T03:22:26.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:26.297+0000 7fd1ebfff700 1 -- 172.21.15.101:0/1426410047 wait complete. 2023-07-18T03:22:26.324 INFO:tasks.cephfs.filesystem:scrub status for tag:a451cabe-c0bc-4cb3-a51c-c222f55c387b - {'path': '/', 'tag': 'a451cabe-c0bc-4cb3-a51c-c222f55c387b', 'options': 'recursive,force'} 2023-07-18T03:22:26.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:26 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:22:26.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:26 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:22:26.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:26 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:22:26.973 INFO:tasks.workunit.client.0.smithi101.stdout: 10 62526 31.95 MB/sec warmup 111 sec latency 75.241 ms 2023-07-18T03:22:27.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:27 smithi101 ceph-mon[126379]: pgmap v467: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 21 MiB/s wr, 974 op/s 2023-07-18T03:22:27.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:27 smithi101 ceph-mon[126379]: scrub queued for path: / 2023-07-18T03:22:27.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:27 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:22:27.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:27 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:22:27.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:27 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:22:27.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:27 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:22:27.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:27 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:22:27.492 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:27 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:22:27.492 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:27 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:22:27.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:27 smithi155 ceph-mon[128885]: pgmap v467: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 21 MiB/s wr, 974 op/s 2023-07-18T03:22:27.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:27 smithi155 ceph-mon[128885]: scrub queued for path: / 2023-07-18T03:22:27.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:27 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:22:27.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:27 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:22:27.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:27 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:22:27.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:27 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:22:27.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:27 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:22:27.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:27 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:22:27.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:27 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:22:27.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:27 smithi179 ceph-mon[129229]: pgmap v467: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 21 MiB/s wr, 974 op/s 2023-07-18T03:22:27.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:27 smithi179 ceph-mon[129229]: scrub queued for path: / 2023-07-18T03:22:27.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:27 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:22:27.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:27 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:22:27.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:27 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:22:27.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:27 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:22:27.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:27 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:22:27.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:27 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:22:27.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:27 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:22:27.973 INFO:tasks.workunit.client.0.smithi101.stdout: 10 63172 32.02 MB/sec warmup 112 sec latency 585.763 ms 2023-07-18T03:22:28.973 INFO:tasks.workunit.client.0.smithi101.stdout: 10 63788 31.91 MB/sec warmup 113 sec latency 1585.812 ms 2023-07-18T03:22:29.496 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:29 smithi101 ceph-mon[126379]: pgmap v468: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 18 MiB/s wr, 730 op/s 2023-07-18T03:22:29.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:29 smithi155 ceph-mon[128885]: pgmap v468: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 18 MiB/s wr, 730 op/s 2023-07-18T03:22:29.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:29 smithi179 ceph-mon[129229]: pgmap v468: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 18 MiB/s wr, 730 op/s 2023-07-18T03:22:29.974 INFO:tasks.workunit.client.0.smithi101.stdout: 10 64388 31.92 MB/sec warmup 114 sec latency 2190.365 ms 2023-07-18T03:22:30.974 INFO:tasks.workunit.client.0.smithi101.stdout: 10 65219 32.07 MB/sec warmup 115 sec latency 124.802 ms 2023-07-18T03:22:31.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:31 smithi101 ceph-mon[126379]: pgmap v469: 129 pgs: 129 active+clean; 1.2 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 21 MiB/s wr, 917 op/s 2023-07-18T03:22:31.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:31 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:22:31.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:31 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:22:31.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:31 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:22:31.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:31 smithi155 ceph-mon[128885]: pgmap v469: 129 pgs: 129 active+clean; 1.2 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 21 MiB/s wr, 917 op/s 2023-07-18T03:22:31.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:31 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:22:31.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:31 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:22:31.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:31 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:22:31.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:31 smithi179 ceph-mon[129229]: pgmap v469: 129 pgs: 129 active+clean; 1.2 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 21 MiB/s wr, 917 op/s 2023-07-18T03:22:31.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:31 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:22:31.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:31 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:22:31.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:31 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:22:31.974 INFO:tasks.workunit.client.0.smithi101.stdout: 10 66005 32.19 MB/sec warmup 116 sec latency 56.258 ms 2023-07-18T03:22:32.974 INFO:tasks.workunit.client.0.smithi101.stdout: 10 66641 32.19 MB/sec warmup 117 sec latency 89.127 ms 2023-07-18T03:22:33.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:33 smithi101 ceph-mon[126379]: pgmap v470: 129 pgs: 129 active+clean; 1.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 15 MiB/s wr, 704 op/s 2023-07-18T03:22:33.553 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:33 smithi155 ceph-mon[128885]: pgmap v470: 129 pgs: 129 active+clean; 1.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 15 MiB/s wr, 704 op/s 2023-07-18T03:22:33.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:33 smithi179 ceph-mon[129229]: pgmap v470: 129 pgs: 129 active+clean; 1.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 15 MiB/s wr, 704 op/s 2023-07-18T03:22:33.974 INFO:tasks.workunit.client.0.smithi101.stdout: 10 67412 32.16 MB/sec warmup 118 sec latency 65.340 ms 2023-07-18T03:22:34.974 INFO:tasks.workunit.client.0.smithi101.stdout: 10 68229 32.38 MB/sec warmup 119 sec latency 93.105 ms 2023-07-18T03:22:35.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:35 smithi101 ceph-mon[126379]: pgmap v471: 129 pgs: 129 active+clean; 1.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 12 MiB/s wr, 578 op/s 2023-07-18T03:22:35.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:35 smithi155 ceph-mon[128885]: pgmap v471: 129 pgs: 129 active+clean; 1.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 12 MiB/s wr, 578 op/s 2023-07-18T03:22:35.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:35 smithi179 ceph-mon[129229]: pgmap v471: 129 pgs: 129 active+clean; 1.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 12 MiB/s wr, 578 op/s 2023-07-18T03:22:36.974 INFO:tasks.workunit.client.0.smithi101.stdout: 10 69594 32.48 MB/sec execute 1 sec latency 72.749 ms 2023-07-18T03:22:37.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:37 smithi101 ceph-mon[126379]: pgmap v472: 129 pgs: 129 active+clean; 1.3 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 824 op/s 2023-07-18T03:22:37.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:37 smithi155 ceph-mon[128885]: pgmap v472: 129 pgs: 129 active+clean; 1.3 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 824 op/s 2023-07-18T03:22:37.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:37 smithi179 ceph-mon[129229]: pgmap v472: 129 pgs: 129 active+clean; 1.3 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 824 op/s 2023-07-18T03:22:37.974 INFO:tasks.workunit.client.0.smithi101.stdout: 10 70419 37.46 MB/sec execute 2 sec latency 70.225 ms 2023-07-18T03:22:38.974 INFO:tasks.workunit.client.0.smithi101.stdout: 10 71281 41.06 MB/sec execute 3 sec latency 203.899 ms 2023-07-18T03:22:39.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:39 smithi155 ceph-mon[128885]: pgmap v473: 129 pgs: 129 active+clean; 1.3 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 6.8 MiB/s wr, 467 op/s 2023-07-18T03:22:39.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:39 smithi179 ceph-mon[129229]: pgmap v473: 129 pgs: 129 active+clean; 1.3 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 6.8 MiB/s wr, 467 op/s 2023-07-18T03:22:39.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:39 smithi101 ceph-mon[126379]: pgmap v473: 129 pgs: 129 active+clean; 1.3 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 6.8 MiB/s wr, 467 op/s 2023-07-18T03:22:39.974 INFO:tasks.workunit.client.0.smithi101.stdout: 10 71840 39.86 MB/sec execute 4 sec latency 100.623 ms 2023-07-18T03:22:40.974 INFO:tasks.workunit.client.0.smithi101.stdout: 10 72601 39.81 MB/sec execute 5 sec latency 48.868 ms 2023-07-18T03:22:41.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:41 smithi155 ceph-mon[128885]: pgmap v474: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.0 MiB/s wr, 555 op/s 2023-07-18T03:22:41.550 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:41 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:22:41.550 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:41 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:22:41.550 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:41 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:22:41.706 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:41 smithi179 ceph-mon[129229]: pgmap v474: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.0 MiB/s wr, 555 op/s 2023-07-18T03:22:41.706 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:41 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:22:41.706 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:41 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:22:41.706 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:41 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:22:41.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:41 smithi101 ceph-mon[126379]: pgmap v474: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.0 MiB/s wr, 555 op/s 2023-07-18T03:22:41.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:41 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:22:41.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:41 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:22:41.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:41 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:22:41.974 INFO:tasks.workunit.client.0.smithi101.stdout: 10 73356 38.42 MB/sec execute 6 sec latency 68.679 ms 2023-07-18T03:22:42.975 INFO:tasks.workunit.client.0.smithi101.stdout: 10 74200 39.25 MB/sec execute 7 sec latency 58.744 ms 2023-07-18T03:22:43.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:43 smithi155 ceph-mon[128885]: pgmap v475: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.1 MiB/s wr, 499 op/s 2023-07-18T03:22:43.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:43 smithi179 ceph-mon[129229]: pgmap v475: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.1 MiB/s wr, 499 op/s 2023-07-18T03:22:43.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:43 smithi101 ceph-mon[126379]: pgmap v475: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.1 MiB/s wr, 499 op/s 2023-07-18T03:22:43.975 INFO:tasks.workunit.client.0.smithi101.stdout: 10 74891 39.58 MB/sec execute 8 sec latency 74.041 ms 2023-07-18T03:22:44.975 INFO:tasks.workunit.client.0.smithi101.stdout: 10 75659 40.29 MB/sec execute 9 sec latency 101.049 ms 2023-07-18T03:22:45.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:45 smithi179 ceph-mon[129229]: pgmap v476: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.3 MiB/s wr, 464 op/s 2023-07-18T03:22:45.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:45 smithi101 ceph-mon[126379]: pgmap v476: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.3 MiB/s wr, 464 op/s 2023-07-18T03:22:45.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:45 smithi155 ceph-mon[128885]: pgmap v476: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.3 MiB/s wr, 464 op/s 2023-07-18T03:22:45.975 INFO:tasks.workunit.client.0.smithi101.stdout: 10 76454 40.62 MB/sec execute 10 sec latency 81.147 ms 2023-07-18T03:22:46.975 INFO:tasks.workunit.client.0.smithi101.stdout: 10 77009 39.61 MB/sec execute 11 sec latency 137.556 ms 2023-07-18T03:22:47.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:47 smithi179 ceph-mon[129229]: pgmap v477: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 9.1 MiB/s wr, 681 op/s 2023-07-18T03:22:47.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:47 smithi101 ceph-mon[126379]: pgmap v477: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 9.1 MiB/s wr, 681 op/s 2023-07-18T03:22:47.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:47 smithi155 ceph-mon[128885]: pgmap v477: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 9.1 MiB/s wr, 681 op/s 2023-07-18T03:22:47.975 INFO:tasks.workunit.client.0.smithi101.stdout: 10 77745 39.19 MB/sec execute 12 sec latency 51.942 ms 2023-07-18T03:22:48.975 INFO:tasks.workunit.client.0.smithi101.stdout: 10 77828 36.31 MB/sec execute 13 sec latency 565.302 ms 2023-07-18T03:22:49.454 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:49 smithi179 ceph-mon[129229]: pgmap v478: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.6 MiB/s wr, 435 op/s 2023-07-18T03:22:49.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:49 smithi101 ceph-mon[126379]: pgmap v478: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.6 MiB/s wr, 435 op/s 2023-07-18T03:22:49.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:49 smithi155 ceph-mon[128885]: pgmap v478: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.6 MiB/s wr, 435 op/s 2023-07-18T03:22:49.975 INFO:tasks.workunit.client.0.smithi101.stdout: 10 78113 35.01 MB/sec execute 14 sec latency 320.289 ms 2023-07-18T03:22:50.975 INFO:tasks.workunit.client.0.smithi101.stdout: 10 78730 34.83 MB/sec execute 15 sec latency 103.874 ms 2023-07-18T03:22:51.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:51 smithi179 ceph-mon[129229]: pgmap v479: 129 pgs: 129 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 640 op/s 2023-07-18T03:22:51.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:51 smithi101 ceph-mon[126379]: pgmap v479: 129 pgs: 129 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 640 op/s 2023-07-18T03:22:51.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:51 smithi155 ceph-mon[128885]: pgmap v479: 129 pgs: 129 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 640 op/s 2023-07-18T03:22:51.975 INFO:tasks.workunit.client.0.smithi101.stdout: 10 79545 35.44 MB/sec execute 16 sec latency 67.180 ms 2023-07-18T03:22:52.975 INFO:tasks.workunit.client.0.smithi101.stdout: 10 80439 36.72 MB/sec execute 17 sec latency 44.499 ms 2023-07-18T03:22:53.454 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:53 smithi179 ceph-mon[129229]: pgmap v480: 129 pgs: 129 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 615 op/s 2023-07-18T03:22:53.493 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:53 smithi101 ceph-mon[126379]: pgmap v480: 129 pgs: 129 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 615 op/s 2023-07-18T03:22:53.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:53 smithi155 ceph-mon[128885]: pgmap v480: 129 pgs: 129 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 615 op/s 2023-07-18T03:22:53.977 INFO:tasks.workunit.client.0.smithi101.stdout: 10 80884 35.91 MB/sec execute 18 sec latency 322.792 ms 2023-07-18T03:22:54.976 INFO:tasks.workunit.client.0.smithi101.stdout: 10 80895 34.03 MB/sec execute 19 sec latency 1313.358 ms 2023-07-18T03:22:55.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:55 smithi101 ceph-mon[126379]: pgmap v481: 129 pgs: 129 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 11 MiB/s wr, 485 op/s 2023-07-18T03:22:55.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:55 smithi155 ceph-mon[128885]: pgmap v481: 129 pgs: 129 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 11 MiB/s wr, 485 op/s 2023-07-18T03:22:55.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:55 smithi179 ceph-mon[129229]: pgmap v481: 129 pgs: 129 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 11 MiB/s wr, 485 op/s 2023-07-18T03:22:55.977 INFO:tasks.workunit.client.0.smithi101.stdout: 10 81073 32.76 MB/sec execute 20 sec latency 1317.617 ms 2023-07-18T03:22:56.326 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:22:56.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.703+0000 7f5424dff700 1 -- 172.21.15.101:0/695277688 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f542006d3a0 msgr2=0x7f542011b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:56.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.703+0000 7f5424dff700 1 --2- 172.21.15.101:0/695277688 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f542006d3a0 0x7f542011b470 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f54080098f0 tx=0x7f5408009c00 comp rx=0 tx=0).stop 2023-07-18T03:22:56.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.711+0000 7f5424dff700 1 -- 172.21.15.101:0/695277688 shutdown_connections 2023-07-18T03:22:56.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.711+0000 7f5424dff700 1 --2- 172.21.15.101:0/695277688 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f542006d3a0 0x7f542011b470 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:56.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.711+0000 7f5424dff700 1 --2- 172.21.15.101:0/695277688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f542010e280 0x7f542010e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:56.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.711+0000 7f5424dff700 1 --2- 172.21.15.101:0/695277688 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f542006c9a0 0x7f542006ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:56.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.711+0000 7f5424dff700 1 -- 172.21.15.101:0/695277688 >> 172.21.15.101:0/695277688 conn(0x7f542006bf70 msgr2=0x7f5420070ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:56.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.713+0000 7f5424dff700 1 -- 172.21.15.101:0/695277688 shutdown_connections 2023-07-18T03:22:56.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.718+0000 7f5424dff700 1 -- 172.21.15.101:0/695277688 wait complete. 2023-07-18T03:22:56.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.719+0000 7f5424dff700 1 Processor -- start 2023-07-18T03:22:56.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.723+0000 7f5424dff700 1 -- start start 2023-07-18T03:22:56.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.723+0000 7f5424dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f542006c9a0 0x7f5420116db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:56.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.724+0000 7f5424dff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f542006d3a0 0x7f54201172f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:56.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.724+0000 7f5424dff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f542010e280 0x7f54201a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:56.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.724+0000 7f5424dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54201a2510 con 0x7f542006c9a0 2023-07-18T03:22:56.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.724+0000 7f5424dff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54201a2680 con 0x7f542006d3a0 2023-07-18T03:22:56.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.724+0000 7f5424dff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54201a2840 con 0x7f542010e280 2023-07-18T03:22:56.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.725+0000 7f541e59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f542006c9a0 0x7f5420116db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:56.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.725+0000 7f541e59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f542006c9a0 0x7f5420116db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:40478/0 (socket says 172.21.15.101:40478) 2023-07-18T03:22:56.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.725+0000 7f541e59c700 1 -- 172.21.15.101:0/310524512 learned_addr learned my addr 172.21.15.101:0/310524512 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:56.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.725+0000 7f541ed9d700 1 --2- 172.21.15.101:0/310524512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f542010e280 0x7f54201a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:56.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.725+0000 7f541dd9b700 1 --2- 172.21.15.101:0/310524512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f542006d3a0 0x7f54201172f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:56.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.726+0000 7f541e59c700 1 -- 172.21.15.101:0/310524512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f542010e280 msgr2=0x7f54201a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:56.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.726+0000 7f541e59c700 1 --2- 172.21.15.101:0/310524512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f542010e280 0x7f54201a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:56.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.726+0000 7f541e59c700 1 -- 172.21.15.101:0/310524512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f542006d3a0 msgr2=0x7f54201172f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:56.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.726+0000 7f541e59c700 1 --2- 172.21.15.101:0/310524512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f542006d3a0 0x7f54201172f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:56.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.726+0000 7f541e59c700 1 -- 172.21.15.101:0/310524512 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5408009520 con 0x7f542006c9a0 2023-07-18T03:22:56.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.726+0000 7f541e59c700 1 --2- 172.21.15.101:0/310524512 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f542006c9a0 0x7f5420116db0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f541801faf0 tx=0x7f541801feb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:56.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.728+0000 7f54177fe700 1 -- 172.21.15.101:0/310524512 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541801dad0 con 0x7f542006c9a0 2023-07-18T03:22:56.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.728+0000 7f54177fe700 1 -- 172.21.15.101:0/310524512 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5418004cf0 con 0x7f542006c9a0 2023-07-18T03:22:56.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.728+0000 7f54177fe700 1 -- 172.21.15.101:0/310524512 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5418005680 con 0x7f542006c9a0 2023-07-18T03:22:56.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.728+0000 7f5424dff700 1 -- 172.21.15.101:0/310524512 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54201a2b20 con 0x7f542006c9a0 2023-07-18T03:22:56.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.728+0000 7f5424dff700 1 -- 172.21.15.101:0/310524512 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54201a91a0 con 0x7f542006c9a0 2023-07-18T03:22:56.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.735+0000 7f5438660b80 1 Processor -- start 2023-07-18T03:22:56.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.742+0000 7f5438660b80 1 -- start start 2023-07-18T03:22:56.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.743+0000 7f5438660b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56028650c2d0 0x56028650c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:56.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.743+0000 7f5438660b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56028650cbe0 0x560286517590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:56.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.743+0000 7f5438660b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560286517ad0 0x560286519eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:56.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.743+0000 7f5438660b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560286412cb0 con 0x56028650cbe0 2023-07-18T03:22:56.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.743+0000 7f5438660b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560286465a40 con 0x56028650c2d0 2023-07-18T03:22:56.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.743+0000 7f5438660b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56028645eed0 con 0x560286517ad0 2023-07-18T03:22:56.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.744+0000 7f541dd9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560286517ad0 0x560286519eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:56.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.744+0000 7f541dd9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560286517ad0 0x560286519eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41068/0 (socket says 172.21.15.101:41068) 2023-07-18T03:22:56.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.744+0000 7f541dd9b700 1 -- 172.21.15.101:0/1759418126 learned_addr learned my addr 172.21.15.101:0/1759418126 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:56.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.744+0000 7f541dd9b700 1 -- 172.21.15.101:0/1759418126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56028650c2d0 msgr2=0x56028650c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:56.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.744+0000 7f541dd9b700 1 --2- 172.21.15.101:0/1759418126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56028650c2d0 0x56028650c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:56.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.744+0000 7f541dd9b700 1 -- 172.21.15.101:0/1759418126 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56028650cbe0 msgr2=0x560286517590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:56.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.744+0000 7f541dd9b700 1 --2- 172.21.15.101:0/1759418126 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56028650cbe0 0x560286517590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:56.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.744+0000 7f541dd9b700 1 -- 172.21.15.101:0/1759418126 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56028628cde0 con 0x560286517ad0 2023-07-18T03:22:56.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.745+0000 7f541dd9b700 1 --2- 172.21.15.101:0/1759418126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560286517ad0 0x560286519eb0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f5410007db0 tx=0x7f541000c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=444e138c32939721 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:56.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.745+0000 7f5414ff9700 1 -- 172.21.15.101:0/1759418126 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541000ce70 con 0x560286517ad0 2023-07-18T03:22:56.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.745+0000 7f5414ff9700 1 -- 172.21.15.101:0/1759418126 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54100118f0 con 0x560286517ad0 2023-07-18T03:22:56.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.745+0000 7f5414ff9700 1 -- 172.21.15.101:0/1759418126 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5410010890 con 0x560286517ad0 2023-07-18T03:22:56.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.746+0000 7f54177fe700 1 -- 172.21.15.101:0/310524512 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f541801dc30 con 0x7f542006c9a0 2023-07-18T03:22:56.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.746+0000 7f5438660b80 1 -- 172.21.15.101:0/1759418126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560286517ad0 msgr2=0x560286519eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:56.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.746+0000 7f5438660b80 1 --2- 172.21.15.101:0/1759418126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560286517ad0 0x560286519eb0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f5410007db0 tx=0x7f541000c470 comp rx=0 tx=0).stop 2023-07-18T03:22:56.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.747+0000 7f5438660b80 1 -- 172.21.15.101:0/1759418126 shutdown_connections 2023-07-18T03:22:56.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.747+0000 7f5438660b80 1 --2- 172.21.15.101:0/1759418126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56028650c2d0 0x56028650c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:56.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.747+0000 7f5438660b80 1 --2- 172.21.15.101:0/1759418126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560286517ad0 0x560286519eb0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:56.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.747+0000 7f5438660b80 1 --2- 172.21.15.101:0/1759418126 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56028650cbe0 0x560286517590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:56.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.747+0000 7f5438660b80 1 -- 172.21.15.101:0/1759418126 >> 172.21.15.101:0/1759418126 conn(0x560286502da0 msgr2=0x5602865086a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:56.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.747+0000 7f5438660b80 1 -- 172.21.15.101:0/1759418126 shutdown_connections 2023-07-18T03:22:56.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.747+0000 7f5438660b80 1 -- 172.21.15.101:0/1759418126 wait complete. 2023-07-18T03:22:56.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.748+0000 7f5438660b80 1 -- start start 2023-07-18T03:22:56.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.748+0000 7f5438660b80 1 Processor -- start 2023-07-18T03:22:56.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.749+0000 7f5438660b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56028650c2d0 0x560286517580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:56.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.749+0000 7f5438660b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56028650cbe0 0x560286514800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:56.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.749+0000 7f5438660b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560286517ad0 0x560286514f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:56.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.749+0000 7f5438660b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602863feac0 con 0x56028650cbe0 2023-07-18T03:22:56.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.749+0000 7f5438660b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602863fe7c0 con 0x560286517ad0 2023-07-18T03:22:56.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.749+0000 7f5438660b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602862e2920 con 0x56028650c2d0 2023-07-18T03:22:56.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.750+0000 7f5438660b80 10 client.0 _ll_register_callbacks cb 0x5602863fc760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:22:56.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.750+0000 7f541ed9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56028650c2d0 0x560286517580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:56.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.751+0000 7f54177fe700 1 --2- 172.21.15.101:0/310524512 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f540c0a13e0 0x7f540c0a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:56.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.751+0000 7f54177fe700 1 -- 172.21.15.101:0/310524512 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f5418025070 con 0x7f542006c9a0 2023-07-18T03:22:56.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.751+0000 7f541e59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56028650cbe0 0x560286514800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:56.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.751+0000 7f541e59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56028650cbe0 0x560286514800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:40500/0 (socket says 172.21.15.101:40500) 2023-07-18T03:22:56.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.751+0000 7f541e59c700 1 -- 172.21.15.101:0/941413097 learned_addr learned my addr 172.21.15.101:0/941413097 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:56.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.751+0000 7f541dd9b700 1 --2- 172.21.15.101:0/941413097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560286517ad0 0x560286514f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:56.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.753+0000 7f541e59c700 1 -- 172.21.15.101:0/941413097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56028650c2d0 msgr2=0x560286517580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:56.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.753+0000 7f541e59c700 1 --2- 172.21.15.101:0/941413097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56028650c2d0 0x560286517580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:56.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.753+0000 7f541e59c700 1 -- 172.21.15.101:0/941413097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560286517ad0 msgr2=0x560286514f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:56.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.753+0000 7f541e59c700 1 --2- 172.21.15.101:0/941413097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560286517ad0 0x560286514f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:56.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.753+0000 7f541e59c700 1 -- 172.21.15.101:0/941413097 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54100079e0 con 0x56028650cbe0 2023-07-18T03:22:56.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.754+0000 7f541e59c700 1 --2- 172.21.15.101:0/941413097 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56028650cbe0 0x560286514800 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f5418000c00 tx=0x7f54180aac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7f3c4974a12d21d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:56.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.757+0000 7f541ed9d700 1 --2- 172.21.15.101:0/310524512 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f540c0a13e0 0x7f540c0a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:56.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.764+0000 7f5405ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:22:56.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.764+0000 7f5405ffb700 1 -- 172.21.15.101:0/941413097 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54180ab450 con 0x56028650cbe0 2023-07-18T03:22:56.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.764+0000 7f5405ffb700 1 -- 172.21.15.101:0/941413097 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f54180af020 con 0x56028650cbe0 2023-07-18T03:22:56.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.764+0000 7f5438660b80 1 -- 172.21.15.101:0/941413097 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5602863b84f0 con 0x56028650cbe0 2023-07-18T03:22:56.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.765+0000 7f5405ffb700 1 -- 172.21.15.101:0/941413097 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54180b3440 con 0x56028650cbe0 2023-07-18T03:22:56.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.765+0000 7f5438660b80 10 client.18486 fetch_fsmap learned FSMap version 39 2023-07-18T03:22:56.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.765+0000 7f5438660b80 1 -- 172.21.15.101:0/941413097 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x560286275c60 con 0x56028650cbe0 2023-07-18T03:22:56.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.766+0000 7f5405ffb700 1 -- 172.21.15.101:0/941413097 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5602863b84f0 con 0x56028650cbe0 2023-07-18T03:22:56.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.766+0000 7f5405ffb700 1 -- 172.21.15.101:0/941413097 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 39) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f54180b2fb0 con 0x56028650cbe0 2023-07-18T03:22:56.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.766+0000 7f5438660b80 10 client.18486 fetch_fsmap finished waiting for FSMap version 39 2023-07-18T03:22:56.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.766+0000 7f5438660b80 10 client.18486 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:22:56.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.766+0000 7f5438660b80 20 client.18486 populate_metadata read hostname 'smithi101' 2023-07-18T03:22:56.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.766+0000 7f5438660b80 1 --2- 172.21.15.101:0/941413097 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5602865494f0 0x56028654b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:56.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.766+0000 7f5438660b80 4 client.18486 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:22:56.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.766+0000 7f5438660b80 1 -- 172.21.15.101:0/941413097 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560286457eb0 con 0x5602865494f0 2023-07-18T03:22:56.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.767+0000 7f541dd9b700 1 --2- 172.21.15.101:0/941413097 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5602865494f0 0x56028654b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:56.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.768+0000 7f541dd9b700 1 --2- 172.21.15.101:0/941413097 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5602865494f0 0x56028654b900 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc9a36328e52e475 server_cookie=15ed72325813323e in_seq=0 out_seq=0 2023-07-18T03:22:56.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.774+0000 7f5405ffb700 10 client.18486 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:22:56.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.777+0000 7f5405ffb700 1 -- 172.21.15.101:0/941413097 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560286457eb0 con 0x5602865494f0 2023-07-18T03:22:56.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.777+0000 7f5405ffb700 10 client.18486 handle_command_reply: tid=0 2023-07-18T03:22:56.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.777+0000 7f5438660b80 1 client.18486 shutdown 2023-07-18T03:22:56.774 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.778+0000 7f5438660b80 1 -- 172.21.15.101:0/941413097 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56028650cbe0 msgr2=0x560286514800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:56.774 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.778+0000 7f5438660b80 1 --2- 172.21.15.101:0/941413097 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56028650cbe0 0x560286514800 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f5418000c00 tx=0x7f54180aac20 comp rx=0 tx=0).stop 2023-07-18T03:22:56.774 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.778+0000 7f5438660b80 1 -- 172.21.15.101:0/941413097 shutdown_connections 2023-07-18T03:22:56.774 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.778+0000 7f5438660b80 1 --2- 172.21.15.101:0/941413097 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5602865494f0 0x56028654b900 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:56.775 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.778+0000 7f5438660b80 1 --2- 172.21.15.101:0/941413097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560286517ad0 0x560286514f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:56.775 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.778+0000 7f5438660b80 1 --2- 172.21.15.101:0/941413097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56028650c2d0 0x560286517580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:56.775 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.778+0000 7f5438660b80 1 --2- 172.21.15.101:0/941413097 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56028650cbe0 0x560286514800 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:56.775 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.778+0000 7f5405ffb700 0 client.18486 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:22:56.775 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.778+0000 7f5438660b80 1 -- 172.21.15.101:0/941413097 >> 172.21.15.101:0/941413097 conn(0x560286502da0 msgr2=0x560286508230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:56.776 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.780+0000 7f541ed9d700 1 --2- 172.21.15.101:0/310524512 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f540c0a13e0 0x7f540c0a3890 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f540800a010 tx=0x7f5408019040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:56.779 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.781+0000 7f5438660b80 1 -- 172.21.15.101:0/941413097 shutdown_connections 2023-07-18T03:22:56.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.788+0000 7f5438660b80 1 -- 172.21.15.101:0/941413097 wait complete. 2023-07-18T03:22:56.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.788+0000 7f5438660b80 20 client.18486 trim_cache size 0 max 16384 2023-07-18T03:22:56.826 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.829+0000 7f5438660b80 1 Processor -- start 2023-07-18T03:22:56.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.831+0000 7f5438660b80 1 -- start start 2023-07-18T03:22:56.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.832+0000 7f5438660b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56028650c2d0 0x560286517120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:56.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.832+0000 7f5438660b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56028650cbe0 0x560286513fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:56.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.832+0000 7f5438660b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560286517ad0 0x560286514500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:56.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.832+0000 7f5438660b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560286509d20 con 0x56028650c2d0 2023-07-18T03:22:56.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.832+0000 7f5438660b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560286546c40 con 0x56028650cbe0 2023-07-18T03:22:56.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.832+0000 7f5438660b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x560286515de0 con 0x560286517ad0 2023-07-18T03:22:56.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.832+0000 7f541dd9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56028650cbe0 0x560286513fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:56.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.832+0000 7f541dd9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56028650cbe0 0x560286513fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:53760/0 (socket says 172.21.15.101:53760) 2023-07-18T03:22:56.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.832+0000 7f541dd9b700 1 -- 172.21.15.101:0/4051850714 learned_addr learned my addr 172.21.15.101:0/4051850714 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:56.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.832+0000 7f541e59c700 1 --2- 172.21.15.101:0/4051850714 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56028650c2d0 0x560286517120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:56.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.833+0000 7f541ed9d700 1 --2- 172.21.15.101:0/4051850714 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560286517ad0 0x560286514500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:56.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.833+0000 7f541e59c700 1 -- 172.21.15.101:0/4051850714 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560286517ad0 msgr2=0x560286514500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:56.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.833+0000 7f541e59c700 1 --2- 172.21.15.101:0/4051850714 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560286517ad0 0x560286514500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:56.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.833+0000 7f541e59c700 1 -- 172.21.15.101:0/4051850714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56028650cbe0 msgr2=0x560286513fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:56.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.833+0000 7f541e59c700 1 --2- 172.21.15.101:0/4051850714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56028650cbe0 0x560286513fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:56.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.833+0000 7f541e59c700 1 -- 172.21.15.101:0/4051850714 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602862b3250 con 0x56028650c2d0 2023-07-18T03:22:56.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.834+0000 7f541e59c700 1 --2- 172.21.15.101:0/4051850714 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56028650c2d0 0x560286517120 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f54180aaf80 tx=0x7f54180aaa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d871d41650724083 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:56.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.834+0000 7f54157fa700 1 -- 172.21.15.101:0/4051850714 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54180de100 con 0x56028650c2d0 2023-07-18T03:22:56.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.834+0000 7f54157fa700 1 -- 172.21.15.101:0/4051850714 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54180af020 con 0x56028650c2d0 2023-07-18T03:22:56.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.834+0000 7f54157fa700 1 -- 172.21.15.101:0/4051850714 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54180b2fb0 con 0x56028650c2d0 2023-07-18T03:22:56.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.835+0000 7f5438660b80 1 -- 172.21.15.101:0/4051850714 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56028650c2d0 msgr2=0x560286517120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:56.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.835+0000 7f5438660b80 1 --2- 172.21.15.101:0/4051850714 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56028650c2d0 0x560286517120 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f54180aaf80 tx=0x7f54180aaa80 comp rx=0 tx=0).stop 2023-07-18T03:22:56.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.839+0000 7f5438660b80 1 -- 172.21.15.101:0/4051850714 shutdown_connections 2023-07-18T03:22:56.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.839+0000 7f5438660b80 1 --2- 172.21.15.101:0/4051850714 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560286517ad0 0x560286514500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:56.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.839+0000 7f5438660b80 1 --2- 172.21.15.101:0/4051850714 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56028650c2d0 0x560286517120 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:56.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.839+0000 7f5438660b80 1 --2- 172.21.15.101:0/4051850714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56028650cbe0 0x560286513fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:56.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.839+0000 7f5438660b80 1 -- 172.21.15.101:0/4051850714 >> 172.21.15.101:0/4051850714 conn(0x560286502da0 msgr2=0x5602865458e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:56.842 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.846+0000 7f5438660b80 1 -- 172.21.15.101:0/4051850714 shutdown_connections 2023-07-18T03:22:56.843 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.846+0000 7f5438660b80 1 -- 172.21.15.101:0/4051850714 wait complete. 2023-07-18T03:22:56.843 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.846+0000 7f5438660b80 1 -- start start 2023-07-18T03:22:56.843 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.846+0000 7f5438660b80 1 Processor -- start 2023-07-18T03:22:56.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.847+0000 7f5438660b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56028650c2d0 0x56028651c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:56.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.847+0000 7f5438660b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56028650cbe0 0x56028651ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:56.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.847+0000 7f5438660b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560286517ad0 0x560286596380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:56.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.847+0000 7f5438660b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602865968c0 con 0x560286517ad0 2023-07-18T03:22:56.845 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.847+0000 7f5438660b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560286596a00 con 0x56028650cbe0 2023-07-18T03:22:56.845 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.847+0000 7f5438660b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x560286596bc0 con 0x56028650c2d0 2023-07-18T03:22:56.845 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.847+0000 7f5438660b80 10 client.0 _ll_register_callbacks cb 0x5602863fc760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:22:56.845 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.848+0000 7f541dd9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56028650cbe0 0x56028651ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:56.846 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.848+0000 7f541dd9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56028650cbe0 0x56028651ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:53762/0 (socket says 172.21.15.101:53762) 2023-07-18T03:22:56.846 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.848+0000 7f541dd9b700 1 -- 172.21.15.101:0/415114722 learned_addr learned my addr 172.21.15.101:0/415114722 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:56.846 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.848+0000 7f541e59c700 1 --2- 172.21.15.101:0/415114722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56028650c2d0 0x56028651c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:56.846 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.850+0000 7f541ed9d700 1 --2- 172.21.15.101:0/415114722 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560286517ad0 0x560286596380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:56.847 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.851+0000 7f541dd9b700 1 -- 172.21.15.101:0/415114722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56028650c2d0 msgr2=0x56028651c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:56.847 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.851+0000 7f541dd9b700 1 --2- 172.21.15.101:0/415114722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56028650c2d0 0x56028651c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:56.847 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.851+0000 7f541dd9b700 1 -- 172.21.15.101:0/415114722 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560286517ad0 msgr2=0x560286596380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:56.847 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.851+0000 7f541dd9b700 1 --2- 172.21.15.101:0/415114722 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560286517ad0 0x560286596380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:56.848 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.851+0000 7f541dd9b700 1 -- 172.21.15.101:0/415114722 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560286275c60 con 0x56028650cbe0 2023-07-18T03:22:56.848 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.851+0000 7f541dd9b700 1 --2- 172.21.15.101:0/415114722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56028650cbe0 0x56028651ccb0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5410007d80 tx=0x7f541001e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82fb35bf6920981b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:56.848 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.852+0000 7f541e59c700 1 --2- 172.21.15.101:0/415114722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56028650c2d0 0x56028651c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:22:56.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.852+0000 7f5405ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:22:56.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.852+0000 7f5405ffb700 1 -- 172.21.15.101:0/415114722 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5410021040 con 0x56028650cbe0 2023-07-18T03:22:56.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.852+0000 7f5405ffb700 1 -- 172.21.15.101:0/415114722 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f541000d040 con 0x56028650cbe0 2023-07-18T03:22:56.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.852+0000 7f5438660b80 1 -- 172.21.15.101:0/415114722 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f54100118f0 con 0x56028650cbe0 2023-07-18T03:22:56.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.853+0000 7f5405ffb700 1 -- 172.21.15.101:0/415114722 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5410026510 con 0x56028650cbe0 2023-07-18T03:22:56.850 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.853+0000 7f5438660b80 10 client.28562 fetch_fsmap learned FSMap version 39 2023-07-18T03:22:56.850 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.853+0000 7f5438660b80 1 -- 172.21.15.101:0/415114722 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x56028628b2b0 con 0x56028650cbe0 2023-07-18T03:22:56.850 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.854+0000 7f5405ffb700 1 -- 172.21.15.101:0/415114722 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5410036030 con 0x56028650cbe0 2023-07-18T03:22:56.850 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.854+0000 7f5405ffb700 1 -- 172.21.15.101:0/415114722 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 39) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f5410026a20 con 0x56028650cbe0 2023-07-18T03:22:56.851 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.854+0000 7f5438660b80 10 client.28562 fetch_fsmap finished waiting for FSMap version 39 2023-07-18T03:22:56.851 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.854+0000 7f5438660b80 10 client.28562 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:22:56.851 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.854+0000 7f5438660b80 20 client.28562 populate_metadata read hostname 'smithi101' 2023-07-18T03:22:56.851 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.854+0000 7f5438660b80 1 --2- 172.21.15.101:0/415114722 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56028659e3e0 0x5602865a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:56.851 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.854+0000 7f5438660b80 4 client.28562 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:22:56.851 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.854+0000 7f5438660b80 1 -- 172.21.15.101:0/415114722 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56028641a050 con 0x56028659e3e0 2023-07-18T03:22:56.852 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.855+0000 7f541e59c700 1 --2- 172.21.15.101:0/415114722 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56028659e3e0 0x5602865a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:56.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.858+0000 7f541e59c700 1 --2- 172.21.15.101:0/415114722 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56028659e3e0 0x5602865a07f0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1969e26e87a6fa0 server_cookie=b2069df158bf001c in_seq=0 out_seq=0 2023-07-18T03:22:56.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.858+0000 7f5405ffb700 10 client.28562 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:22:56.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.858+0000 7f5405ffb700 1 -- 172.21.15.101:0/415114722 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56028641a050 con 0x56028659e3e0 2023-07-18T03:22:56.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.858+0000 7f5405ffb700 10 client.28562 handle_command_reply: tid=0 2023-07-18T03:22:56.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.858+0000 7f5438660b80 1 client.28562 shutdown 2023-07-18T03:22:56.856 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.859+0000 7f5438660b80 1 -- 172.21.15.101:0/415114722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56028650cbe0 msgr2=0x56028651ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:56.856 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.859+0000 7f5438660b80 1 --2- 172.21.15.101:0/415114722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56028650cbe0 0x56028651ccb0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5410007d80 tx=0x7f541001e040 comp rx=0 tx=0).stop 2023-07-18T03:22:56.856 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.859+0000 7f5438660b80 1 -- 172.21.15.101:0/415114722 shutdown_connections 2023-07-18T03:22:56.856 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.859+0000 7f5438660b80 1 --2- 172.21.15.101:0/415114722 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56028659e3e0 0x5602865a07f0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:56.856 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.859+0000 7f5438660b80 1 --2- 172.21.15.101:0/415114722 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560286517ad0 0x560286596380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:56.857 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.859+0000 7f5438660b80 1 --2- 172.21.15.101:0/415114722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56028650c2d0 0x56028651c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:56.857 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.859+0000 7f5438660b80 1 --2- 172.21.15.101:0/415114722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56028650cbe0 0x56028651ccb0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:56.857 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.859+0000 7f5438660b80 1 -- 172.21.15.101:0/415114722 >> 172.21.15.101:0/415114722 conn(0x560286502da0 msgr2=0x56028650e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:56.857 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.860+0000 7f5405ffb700 0 client.28562 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:22:56.857 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.860+0000 7f5438660b80 1 -- 172.21.15.101:0/415114722 shutdown_connections 2023-07-18T03:22:56.858 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.860+0000 7f5438660b80 1 -- 172.21.15.101:0/415114722 wait complete. 2023-07-18T03:22:56.858 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.860+0000 7f5438660b80 20 client.28562 trim_cache size 0 max 16384 2023-07-18T03:22:56.864 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:22:56.864 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "no active scrubs running", 2023-07-18T03:22:56.865 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": {} 2023-07-18T03:22:56.865 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:22:56.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.868+0000 7f5424dff700 1 -- 172.21.15.101:0/310524512 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f540c0a13e0 msgr2=0x7f540c0a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:56.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.868+0000 7f5424dff700 1 --2- 172.21.15.101:0/310524512 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f540c0a13e0 0x7f540c0a3890 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f540800a010 tx=0x7f5408019040 comp rx=0 tx=0).stop 2023-07-18T03:22:56.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.868+0000 7f5424dff700 1 -- 172.21.15.101:0/310524512 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f542006c9a0 msgr2=0x7f5420116db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:56.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.868+0000 7f5424dff700 1 --2- 172.21.15.101:0/310524512 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f542006c9a0 0x7f5420116db0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f541801faf0 tx=0x7f541801feb0 comp rx=0 tx=0).stop 2023-07-18T03:22:56.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.870+0000 7f5424dff700 1 -- 172.21.15.101:0/310524512 shutdown_connections 2023-07-18T03:22:56.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.870+0000 7f5424dff700 1 --2- 172.21.15.101:0/310524512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f542010e280 0x7f54201a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:56.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.870+0000 7f5424dff700 1 --2- 172.21.15.101:0/310524512 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f542006c9a0 0x7f5420116db0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:56.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.870+0000 7f5424dff700 1 --2- 172.21.15.101:0/310524512 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f540c0a13e0 0x7f540c0a3890 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:56.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.870+0000 7f5424dff700 1 --2- 172.21.15.101:0/310524512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f542006d3a0 0x7f54201172f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:56.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.870+0000 7f5424dff700 1 -- 172.21.15.101:0/310524512 >> 172.21.15.101:0/310524512 conn(0x7f542006bf70 msgr2=0x7f54200707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:56.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.871+0000 7f5424dff700 1 -- 172.21.15.101:0/310524512 shutdown_connections 2023-07-18T03:22:56.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:56.871+0000 7f5424dff700 1 -- 172.21.15.101:0/310524512 wait complete. 2023-07-18T03:22:56.900 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-18T03:22:56.901 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:22:56.976 INFO:tasks.workunit.client.0.smithi101.stdout: 10 81305 31.71 MB/sec execute 21 sec latency 146.009 ms 2023-07-18T03:22:57.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.307+0000 7f9460dff700 1 -- 172.21.15.101:0/3630473488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f945c0731c0 msgr2=0x7f945c1105a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:57.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.307+0000 7f9460dff700 1 --2- 172.21.15.101:0/3630473488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f945c0731c0 0x7f945c1105a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9450009990 tx=0x7f9450009ca0 comp rx=0 tx=0).stop 2023-07-18T03:22:57.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.307+0000 7f9460dff700 1 -- 172.21.15.101:0/3630473488 shutdown_connections 2023-07-18T03:22:57.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.307+0000 7f9460dff700 1 --2- 172.21.15.101:0/3630473488 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f945c110ae0 0x7f945c112ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:57.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.307+0000 7f9460dff700 1 --2- 172.21.15.101:0/3630473488 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f945c0728b0 0x7f945c072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:57.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.307+0000 7f9460dff700 1 --2- 172.21.15.101:0/3630473488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f945c0731c0 0x7f945c1105a0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:57.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.307+0000 7f9460dff700 1 -- 172.21.15.101:0/3630473488 >> 172.21.15.101:0/3630473488 conn(0x7f945c0fb590 msgr2=0x7f945c0fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:57.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.308+0000 7f9460dff700 1 -- 172.21.15.101:0/3630473488 shutdown_connections 2023-07-18T03:22:57.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.308+0000 7f9460dff700 1 -- 172.21.15.101:0/3630473488 wait complete. 2023-07-18T03:22:57.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.309+0000 7f9460dff700 1 Processor -- start 2023-07-18T03:22:57.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.310+0000 7f9460dff700 1 -- start start 2023-07-18T03:22:57.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.310+0000 7f9460dff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f945c0728b0 0x7f945c19c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:57.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.310+0000 7f9460dff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f945c0731c0 0x7f945c19c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:57.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.310+0000 7f9460dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f945c110ae0 0x7f945c1a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:57.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.310+0000 7f9460dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945c1a1290 con 0x7f945c110ae0 2023-07-18T03:22:57.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.310+0000 7f9460dff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945c1a1400 con 0x7f945c0728b0 2023-07-18T03:22:57.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.310+0000 7f9460dff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945c1a15c0 con 0x7f945c0731c0 2023-07-18T03:22:57.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.312+0000 7f9459d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f945c0731c0 0x7f945c19c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:57.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.313+0000 7f945a59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f945c0728b0 0x7f945c19c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:57.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.313+0000 7f945a59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f945c0728b0 0x7f945c19c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:53768/0 (socket says 172.21.15.101:53768) 2023-07-18T03:22:57.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.313+0000 7f945a59c700 1 -- 172.21.15.101:0/2811287076 learned_addr learned my addr 172.21.15.101:0/2811287076 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:57.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.313+0000 7f945ad9d700 1 --2- 172.21.15.101:0/2811287076 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f945c110ae0 0x7f945c1a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:57.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.314+0000 7f945a59c700 1 -- 172.21.15.101:0/2811287076 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f945c0731c0 msgr2=0x7f945c19c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:57.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.314+0000 7f945a59c700 1 --2- 172.21.15.101:0/2811287076 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f945c0731c0 0x7f945c19c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:57.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.314+0000 7f945a59c700 1 -- 172.21.15.101:0/2811287076 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f945c110ae0 msgr2=0x7f945c1a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:57.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.314+0000 7f945a59c700 1 --2- 172.21.15.101:0/2811287076 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f945c110ae0 0x7f945c1a0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:57.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.314+0000 7f945a59c700 1 -- 172.21.15.101:0/2811287076 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94500095f0 con 0x7f945c0728b0 2023-07-18T03:22:57.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.314+0000 7f945a59c700 1 --2- 172.21.15.101:0/2811287076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f945c0728b0 0x7f945c19c2b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f944c00b530 tx=0x7f944c00b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:57.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.315+0000 7f944b7fe700 1 -- 172.21.15.101:0/2811287076 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944c011830 con 0x7f945c0728b0 2023-07-18T03:22:57.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.315+0000 7f944b7fe700 1 -- 172.21.15.101:0/2811287076 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f944c011e50 con 0x7f945c0728b0 2023-07-18T03:22:57.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.315+0000 7f944b7fe700 1 -- 172.21.15.101:0/2811287076 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944c00f2e0 con 0x7f945c0728b0 2023-07-18T03:22:57.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.315+0000 7f9460dff700 1 -- 172.21.15.101:0/2811287076 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f945c1a18a0 con 0x7f945c0728b0 2023-07-18T03:22:57.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.315+0000 7f9460dff700 1 -- 172.21.15.101:0/2811287076 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f945c1a90e0 con 0x7f945c0728b0 2023-07-18T03:22:57.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.317+0000 7f9460dff700 1 -- 172.21.15.101:0/2811287076 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f945c102fc0 con 0x7f945c0728b0 2023-07-18T03:22:57.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.321+0000 7f944b7fe700 1 -- 172.21.15.101:0/2811287076 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f944c010580 con 0x7f945c0728b0 2023-07-18T03:22:57.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.323+0000 7f944b7fe700 1 --2- 172.21.15.101:0/2811287076 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f94440a1440 0x7f94440a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:57.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.323+0000 7f944b7fe700 1 -- 172.21.15.101:0/2811287076 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f944c0cd040 con 0x7f945c0728b0 2023-07-18T03:22:57.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.324+0000 7f9459d9b700 1 --2- 172.21.15.101:0/2811287076 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f94440a1440 0x7f94440a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:57.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.328+0000 7f944b7fe700 1 -- 172.21.15.101:0/2811287076 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f944c096150 con 0x7f945c0728b0 2023-07-18T03:22:57.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.334+0000 7f9459d9b700 1 --2- 172.21.15.101:0/2811287076 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f94440a1440 0x7f94440a38f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f945000b5c0 tx=0x7f9450005fb0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:57.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:57 smithi155 ceph-mon[128885]: pgmap v482: 129 pgs: 129 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 22 MiB/s wr, 858 op/s 2023-07-18T03:22:57.706 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:57 smithi179 ceph-mon[129229]: pgmap v482: 129 pgs: 129 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 22 MiB/s wr, 858 op/s 2023-07-18T03:22:57.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:57 smithi101 ceph-mon[126379]: pgmap v482: 129 pgs: 129 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 22 MiB/s wr, 858 op/s 2023-07-18T03:22:57.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.762+0000 7f9460dff700 1 -- 172.21.15.101:0/2811287076 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f945c067300 con 0x7f945c0728b0 2023-07-18T03:22:57.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.766+0000 7f944b7fe700 1 -- 172.21.15.101:0/2811287076 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+10867 (secure 0 0 0) 0x7f944c015090 con 0x7f945c0728b0 2023-07-18T03:22:57.766 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:22:57.767 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/4192846769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":4192846769},{"type":"v1","addr":"172.21.15.101:6839","nonce":4192846769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6841/3824340175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6840","nonce":3824340175},{"type":"v1","addr":"172.21.15.101:6841","nonce":3824340175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:04:59.361185+0000","modified":"2023-07-18T03:22:06.121618+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.155:6839/2055462483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2055462483},{"type":"v1","addr":"172.21.15.155:6839","nonce":2055462483}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:22:57.779 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.777+0000 7f9460dff700 1 -- 172.21.15.101:0/2811287076 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f94440a1440 msgr2=0x7f94440a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:57.779 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.777+0000 7f9460dff700 1 --2- 172.21.15.101:0/2811287076 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f94440a1440 0x7f94440a38f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f945000b5c0 tx=0x7f9450005fb0 comp rx=0 tx=0).stop 2023-07-18T03:22:57.779 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.777+0000 7f9460dff700 1 -- 172.21.15.101:0/2811287076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f945c0728b0 msgr2=0x7f945c19c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:57.779 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.777+0000 7f9460dff700 1 --2- 172.21.15.101:0/2811287076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f945c0728b0 0x7f945c19c2b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f944c00b530 tx=0x7f944c00b8f0 comp rx=0 tx=0).stop 2023-07-18T03:22:57.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.785+0000 7f9460dff700 1 -- 172.21.15.101:0/2811287076 shutdown_connections 2023-07-18T03:22:57.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.785+0000 7f9460dff700 1 --2- 172.21.15.101:0/2811287076 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f94440a1440 0x7f94440a38f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:57.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.785+0000 7f9460dff700 1 --2- 172.21.15.101:0/2811287076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f945c0728b0 0x7f945c19c2b0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:57.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.785+0000 7f9460dff700 1 --2- 172.21.15.101:0/2811287076 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f945c110ae0 0x7f945c1a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:57.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.785+0000 7f9460dff700 1 --2- 172.21.15.101:0/2811287076 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f945c0731c0 0x7f945c19c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:57.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.785+0000 7f9460dff700 1 -- 172.21.15.101:0/2811287076 >> 172.21.15.101:0/2811287076 conn(0x7f945c0fb590 msgr2=0x7f945c0fc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:57.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.788+0000 7f9460dff700 1 -- 172.21.15.101:0/2811287076 shutdown_connections 2023-07-18T03:22:57.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:57.789+0000 7f9460dff700 1 -- 172.21.15.101:0/2811287076 wait complete. 2023-07-18T03:22:57.793 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 39 2023-07-18T03:22:57.825 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-18T03:22:57.977 INFO:tasks.workunit.client.0.smithi101.stdout: 10 81585 30.68 MB/sec execute 22 sec latency 175.961 ms 2023-07-18T03:22:58.192 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.195+0000 7f44825ff700 1 -- 172.21.15.101:0/4140694658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f447c10e280 msgr2=0x7f447c10e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:58.192 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.195+0000 7f44825ff700 1 --2- 172.21.15.101:0/4140694658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f447c10e280 0x7f447c10e650 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f446c009990 tx=0x7f446c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.192 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.195+0000 7f44825ff700 1 -- 172.21.15.101:0/4140694658 shutdown_connections 2023-07-18T03:22:58.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.195+0000 7f44825ff700 1 --2- 172.21.15.101:0/4140694658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f447c10e280 0x7f447c10e650 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.195+0000 7f44825ff700 1 --2- 172.21.15.101:0/4140694658 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f447c06d3a0 0x7f447c11b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.195+0000 7f44825ff700 1 --2- 172.21.15.101:0/4140694658 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f447c06c9a0 0x7f447c06ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.195+0000 7f44825ff700 1 -- 172.21.15.101:0/4140694658 >> 172.21.15.101:0/4140694658 conn(0x7f447c06bf70 msgr2=0x7f447c070ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:58.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.195+0000 7f44825ff700 1 -- 172.21.15.101:0/4140694658 shutdown_connections 2023-07-18T03:22:58.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.195+0000 7f44825ff700 1 -- 172.21.15.101:0/4140694658 wait complete. 2023-07-18T03:22:58.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.197+0000 7f44825ff700 1 Processor -- start 2023-07-18T03:22:58.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.197+0000 7f44825ff700 1 -- start start 2023-07-18T03:22:58.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.197+0000 7f44825ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f447c06c9a0 0x7f447c1aa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.197+0000 7f44825ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f447c06d3a0 0x7f447c1aae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.197+0000 7f44825ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f447c10e280 0x7f447c1a4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.197+0000 7f44825ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447c1a4fb0 con 0x7f447c06d3a0 2023-07-18T03:22:58.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.197+0000 7f44825ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447c1a5120 con 0x7f447c10e280 2023-07-18T03:22:58.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.197+0000 7f44825ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447c1a52e0 con 0x7f447c06c9a0 2023-07-18T03:22:58.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.198+0000 7f447b7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f447c06d3a0 0x7f447c1aae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:58.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.198+0000 7f447b7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f447c06d3a0 0x7f447c1aae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:40556/0 (socket says 172.21.15.101:40556) 2023-07-18T03:22:58.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.198+0000 7f447b7fe700 1 -- 172.21.15.101:0/315511802 learned_addr learned my addr 172.21.15.101:0/315511802 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:58.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.198+0000 7f4480b9c700 1 --2- 172.21.15.101:0/315511802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f447c10e280 0x7f447c1a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:58.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.198+0000 7f447b7fe700 1 -- 172.21.15.101:0/315511802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f447c06c9a0 msgr2=0x7f447c1aa8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:58.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.198+0000 7f447b7fe700 1 --2- 172.21.15.101:0/315511802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f447c06c9a0 0x7f447c1aa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.198+0000 7f447b7fe700 1 -- 172.21.15.101:0/315511802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f447c10e280 msgr2=0x7f447c1a4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:58.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.198+0000 7f447b7fe700 1 --2- 172.21.15.101:0/315511802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f447c10e280 0x7f447c1a4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.198+0000 7f447b7fe700 1 -- 172.21.15.101:0/315511802 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f446c0095f0 con 0x7f447c06d3a0 2023-07-18T03:22:58.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.199+0000 7f447b7fe700 1 --2- 172.21.15.101:0/315511802 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f447c06d3a0 0x7f447c1aae20 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f447000c9b0 tx=0x7f447000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:58.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.199+0000 7f44797fa700 1 -- 172.21.15.101:0/315511802 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470007b00 con 0x7f447c06d3a0 2023-07-18T03:22:58.198 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.199+0000 7f44797fa700 1 -- 172.21.15.101:0/315511802 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4470007c60 con 0x7f447c06d3a0 2023-07-18T03:22:58.198 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.199+0000 7f44825ff700 1 -- 172.21.15.101:0/315511802 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f447c1a55c0 con 0x7f447c06d3a0 2023-07-18T03:22:58.198 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.200+0000 7f44797fa700 1 -- 172.21.15.101:0/315511802 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447000e6a0 con 0x7f447c06d3a0 2023-07-18T03:22:58.198 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.200+0000 7f44825ff700 1 -- 172.21.15.101:0/315511802 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f447c1b18d0 con 0x7f447c06d3a0 2023-07-18T03:22:58.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.208+0000 7f4495e64b80 1 Processor -- start 2023-07-18T03:22:58.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.208+0000 7f44797fa700 1 -- 172.21.15.101:0/315511802 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f447000f9a0 con 0x7f447c06d3a0 2023-07-18T03:22:58.207 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.208+0000 7f4495e64b80 1 -- start start 2023-07-18T03:22:58.207 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.208+0000 7f4495e64b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595c917f2d0 0x5595c917f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.208 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.208+0000 7f4495e64b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595c917fbe0 0x5595c918a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.208 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.208+0000 7f4495e64b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595c918aad0 0x5595c918ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.208 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.208+0000 7f4495e64b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595c9085cb0 con 0x5595c917f2d0 2023-07-18T03:22:58.208 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.208+0000 7f4495e64b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595c90d8a40 con 0x5595c918aad0 2023-07-18T03:22:58.208 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.208+0000 7f4495e64b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595c90d1ed0 con 0x5595c917fbe0 2023-07-18T03:22:58.208 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.209+0000 7f447b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595c918aad0 0x5595c918ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:58.208 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.210+0000 7f447b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595c918aad0 0x5595c918ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:53794/0 (socket says 172.21.15.101:53794) 2023-07-18T03:22:58.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.210+0000 7f447b7fe700 1 -- 172.21.15.101:0/3803359684 learned_addr learned my addr 172.21.15.101:0/3803359684 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:58.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.210+0000 7f447bfff700 1 --2- 172.21.15.101:0/3803359684 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595c917fbe0 0x5595c918a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:58.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.210+0000 7f447b7fe700 1 -- 172.21.15.101:0/3803359684 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595c917fbe0 msgr2=0x5595c918a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:58.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.210+0000 7f447b7fe700 1 --2- 172.21.15.101:0/3803359684 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595c917fbe0 0x5595c918a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.210+0000 7f447b7fe700 1 -- 172.21.15.101:0/3803359684 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595c917f2d0 msgr2=0x5595c917f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:58.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.210+0000 7f447b7fe700 1 --2- 172.21.15.101:0/3803359684 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595c917f2d0 0x5595c917f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.210+0000 7f44797fa700 1 --2- 172.21.15.101:0/315511802 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f44640a14d0 0x7f44640a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.210+0000 7f447b7fe700 1 -- 172.21.15.101:0/3803359684 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595c8effde0 con 0x5595c918aad0 2023-07-18T03:22:58.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.211+0000 7f44797fa700 1 -- 172.21.15.101:0/315511802 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f44700cd120 con 0x7f447c06d3a0 2023-07-18T03:22:58.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.212+0000 7f447b7fe700 1 --2- 172.21.15.101:0/3803359684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595c918aad0 0x5595c918ceb0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f44700ca3a0 tx=0x7f4470096120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c39df731612ae04e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:58.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.212+0000 7f4480b9c700 1 --2- 172.21.15.101:0/315511802 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f44640a14d0 0x7f44640a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:58.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.213+0000 7f44627fc700 1 -- 172.21.15.101:0/3803359684 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470099bb0 con 0x5595c918aad0 2023-07-18T03:22:58.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.214+0000 7f44627fc700 1 -- 172.21.15.101:0/3803359684 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4470099920 con 0x5595c918aad0 2023-07-18T03:22:58.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.214+0000 7f44627fc700 1 -- 172.21.15.101:0/3803359684 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470064e90 con 0x5595c918aad0 2023-07-18T03:22:58.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.216+0000 7f4495e64b80 1 -- 172.21.15.101:0/3803359684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595c918aad0 msgr2=0x5595c918ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:58.213 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.216+0000 7f4495e64b80 1 --2- 172.21.15.101:0/3803359684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595c918aad0 0x5595c918ceb0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f44700ca3a0 tx=0x7f4470096120 comp rx=0 tx=0).stop 2023-07-18T03:22:58.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.218+0000 7f4495e64b80 1 -- 172.21.15.101:0/3803359684 shutdown_connections 2023-07-18T03:22:58.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.218+0000 7f4495e64b80 1 --2- 172.21.15.101:0/3803359684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595c918aad0 0x5595c918ceb0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.218+0000 7f4495e64b80 1 --2- 172.21.15.101:0/3803359684 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595c917f2d0 0x5595c917f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.218+0000 7f4495e64b80 1 --2- 172.21.15.101:0/3803359684 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595c917fbe0 0x5595c918a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.218+0000 7f4495e64b80 1 -- 172.21.15.101:0/3803359684 >> 172.21.15.101:0/3803359684 conn(0x5595c9175da0 msgr2=0x5595c917b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:58.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.220+0000 7f4495e64b80 1 -- 172.21.15.101:0/3803359684 shutdown_connections 2023-07-18T03:22:58.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.220+0000 7f4480b9c700 1 --2- 172.21.15.101:0/315511802 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f44640a14d0 0x7f44640a3980 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4474009e20 tx=0x7f4474009480 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:58.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.221+0000 7f4495e64b80 1 -- 172.21.15.101:0/3803359684 wait complete. 2023-07-18T03:22:58.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.222+0000 7f4495e64b80 1 -- start start 2023-07-18T03:22:58.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.222+0000 7f4495e64b80 1 Processor -- start 2023-07-18T03:22:58.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.228+0000 7f4495e64b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595c917f2d0 0x5595c9187870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.228+0000 7f4495e64b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595c917fbe0 0x5595c9187db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.228+0000 7f4495e64b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595c918aad0 0x5595c91882f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.228+0000 7f4495e64b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595c9071ac0 con 0x5595c917f2d0 2023-07-18T03:22:58.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.228+0000 7f4495e64b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595c90717c0 con 0x5595c918aad0 2023-07-18T03:22:58.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.228+0000 7f4495e64b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595c8f55920 con 0x5595c917fbe0 2023-07-18T03:22:58.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.228+0000 7f4495e64b80 10 client.0 _ll_register_callbacks cb 0x5595c906f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:22:58.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.230+0000 7f4480b9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595c918aad0 0x5595c91882f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:58.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.230+0000 7f4480b9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595c918aad0 0x5595c91882f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:53804/0 (socket says 172.21.15.101:53804) 2023-07-18T03:22:58.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.230+0000 7f4480b9c700 1 -- 172.21.15.101:0/3022748987 learned_addr learned my addr 172.21.15.101:0/3022748987 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:58.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.230+0000 7f447bfff700 1 --2- 172.21.15.101:0/3022748987 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595c917f2d0 0x5595c9187870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:58.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.231+0000 7f447b7fe700 1 --2- 172.21.15.101:0/3022748987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595c917fbe0 0x5595c9187db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:58.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.231+0000 7f4480b9c700 1 -- 172.21.15.101:0/3022748987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595c917fbe0 msgr2=0x5595c9187db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:58.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.231+0000 7f4480b9c700 1 --2- 172.21.15.101:0/3022748987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595c917fbe0 0x5595c9187db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.231+0000 7f4480b9c700 1 -- 172.21.15.101:0/3022748987 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595c917f2d0 msgr2=0x5595c9187870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:58.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.231+0000 7f4480b9c700 1 --2- 172.21.15.101:0/3022748987 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595c917f2d0 0x5595c9187870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.231+0000 7f4480b9c700 1 -- 172.21.15.101:0/3022748987 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f447c1b18d0 con 0x5595c918aad0 2023-07-18T03:22:58.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.232+0000 7f4480b9c700 1 --2- 172.21.15.101:0/3022748987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595c918aad0 0x5595c91882f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f44740186d0 tx=0x7f4474004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92bc63d7518e84da server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:58.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.232+0000 7f445b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:22:58.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.235+0000 7f445b7fe700 1 -- 172.21.15.101:0/3022748987 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447401d070 con 0x5595c918aad0 2023-07-18T03:22:58.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.235+0000 7f445b7fe700 1 -- 172.21.15.101:0/3022748987 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f4474006e40 con 0x5595c918aad0 2023-07-18T03:22:58.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.235+0000 7f4495e64b80 1 -- 172.21.15.101:0/3022748987 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5595c902b4f0 con 0x5595c918aad0 2023-07-18T03:22:58.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.236+0000 7f445b7fe700 1 -- 172.21.15.101:0/3022748987 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4474011070 con 0x5595c918aad0 2023-07-18T03:22:58.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.236+0000 7f4495e64b80 10 client.28580 fetch_fsmap learned FSMap version 39 2023-07-18T03:22:58.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.236+0000 7f4495e64b80 1 -- 172.21.15.101:0/3022748987 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5595c8ee8c60 con 0x5595c918aad0 2023-07-18T03:22:58.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.236+0000 7f445b7fe700 1 -- 172.21.15.101:0/3022748987 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5595c902b4f0 con 0x5595c918aad0 2023-07-18T03:22:58.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.236+0000 7f445b7fe700 1 -- 172.21.15.101:0/3022748987 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 39) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f447402b070 con 0x5595c918aad0 2023-07-18T03:22:58.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.236+0000 7f4495e64b80 10 client.28580 fetch_fsmap finished waiting for FSMap version 39 2023-07-18T03:22:58.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.236+0000 7f4495e64b80 10 client.28580 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:22:58.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.236+0000 7f4495e64b80 20 client.28580 populate_metadata read hostname 'smithi101' 2023-07-18T03:22:58.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.236+0000 7f4495e64b80 1 --2- 172.21.15.101:0/3022748987 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5595c91ab450 0x5595c91ad860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.237+0000 7f4495e64b80 4 client.28580 mds_command: new command op to 24541 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:22:58.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.237+0000 7f4495e64b80 1 -- 172.21.15.101:0/3022748987 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5595c90caeb0 con 0x5595c91ab450 2023-07-18T03:22:58.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.237+0000 7f447bfff700 1 --2- 172.21.15.101:0/3022748987 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5595c91ab450 0x5595c91ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:58.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.248+0000 7f447bfff700 1 --2- 172.21.15.101:0/3022748987 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5595c91ab450 0x5595c91ad860 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1cc429c1b212b73d server_cookie=34ec7a73cf057b1e in_seq=0 out_seq=0 2023-07-18T03:22:58.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.248+0000 7f445b7fe700 10 client.28580 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:22:58.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.250+0000 7f445b7fe700 1 -- 172.21.15.101:0/3022748987 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5595c90caeb0 con 0x5595c91ab450 2023-07-18T03:22:58.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.250+0000 7f445b7fe700 10 client.28580 handle_command_reply: tid=0 2023-07-18T03:22:58.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.250+0000 7f4495e64b80 1 client.28580 shutdown 2023-07-18T03:22:58.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.254+0000 7f4495e64b80 1 -- 172.21.15.101:0/3022748987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595c918aad0 msgr2=0x5595c91882f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:58.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.254+0000 7f4495e64b80 1 --2- 172.21.15.101:0/3022748987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595c918aad0 0x5595c91882f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f44740186d0 tx=0x7f4474004ad0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.255+0000 7f4495e64b80 1 -- 172.21.15.101:0/3022748987 shutdown_connections 2023-07-18T03:22:58.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.255+0000 7f4495e64b80 1 --2- 172.21.15.101:0/3022748987 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5595c91ab450 0x5595c91ad860 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.255+0000 7f4495e64b80 1 --2- 172.21.15.101:0/3022748987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595c918aad0 0x5595c91882f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.255+0000 7f4495e64b80 1 --2- 172.21.15.101:0/3022748987 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595c917f2d0 0x5595c9187870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.255+0000 7f4495e64b80 1 --2- 172.21.15.101:0/3022748987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595c917fbe0 0x5595c9187db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.255+0000 7f445b7fe700 0 client.28580 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:22:58.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.255+0000 7f4495e64b80 1 -- 172.21.15.101:0/3022748987 >> 172.21.15.101:0/3022748987 conn(0x5595c9175da0 msgr2=0x5595c917b290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:58.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.255+0000 7f4495e64b80 1 -- 172.21.15.101:0/3022748987 shutdown_connections 2023-07-18T03:22:58.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.255+0000 7f4495e64b80 1 -- 172.21.15.101:0/3022748987 wait complete. 2023-07-18T03:22:58.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.256+0000 7f4495e64b80 20 client.28580 trim_cache size 0 max 16384 2023-07-18T03:22:58.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.278+0000 7f4495e64b80 1 Processor -- start 2023-07-18T03:22:58.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.279+0000 7f4495e64b80 1 -- start start 2023-07-18T03:22:58.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.279+0000 7f4495e64b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595c917f2d0 0x5595c91a24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.279+0000 7f4495e64b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595c917fbe0 0x5595c9188120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.279+0000 7f4495e64b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595c918aad0 0x5595c9184b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.279+0000 7f4495e64b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595c9182770 con 0x5595c917f2d0 2023-07-18T03:22:58.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.279+0000 7f4495e64b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595c9180570 con 0x5595c917fbe0 2023-07-18T03:22:58.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.279+0000 7f4495e64b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595c9185e50 con 0x5595c918aad0 2023-07-18T03:22:58.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.279+0000 7f4480b9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595c918aad0 0x5595c9184b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:58.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.280+0000 7f4480b9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595c918aad0 0x5595c9184b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41154/0 (socket says 172.21.15.101:41154) 2023-07-18T03:22:58.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.280+0000 7f4480b9c700 1 -- 172.21.15.101:0/1280160747 learned_addr learned my addr 172.21.15.101:0/1280160747 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:58.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.280+0000 7f447bfff700 1 --2- 172.21.15.101:0/1280160747 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595c917f2d0 0x5595c91a24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:58.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.281+0000 7f447bfff700 1 -- 172.21.15.101:0/1280160747 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595c918aad0 msgr2=0x5595c9184b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:58.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.281+0000 7f447bfff700 1 --2- 172.21.15.101:0/1280160747 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595c918aad0 0x5595c9184b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.281+0000 7f447bfff700 1 -- 172.21.15.101:0/1280160747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595c917fbe0 msgr2=0x5595c9188120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:58.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.281+0000 7f447bfff700 1 --2- 172.21.15.101:0/1280160747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595c917fbe0 0x5595c9188120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.281+0000 7f447bfff700 1 -- 172.21.15.101:0/1280160747 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595c8f26250 con 0x5595c917f2d0 2023-07-18T03:22:58.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.283+0000 7f447bfff700 1 --2- 172.21.15.101:0/1280160747 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595c917f2d0 0x5595c91a24b0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x5595c91ae170 tx=0x7f446c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4babb4713cf94fe1 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:58.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.284+0000 7f4462ffd700 1 -- 172.21.15.101:0/1280160747 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c01f070 con 0x5595c917f2d0 2023-07-18T03:22:58.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.284+0000 7f4462ffd700 1 -- 172.21.15.101:0/1280160747 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f446c020040 con 0x5595c917f2d0 2023-07-18T03:22:58.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.284+0000 7f4462ffd700 1 -- 172.21.15.101:0/1280160747 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c003b00 con 0x5595c917f2d0 2023-07-18T03:22:58.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.285+0000 7f4495e64b80 1 -- 172.21.15.101:0/1280160747 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595c917f2d0 msgr2=0x5595c91a24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:58.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.285+0000 7f4495e64b80 1 --2- 172.21.15.101:0/1280160747 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595c917f2d0 0x5595c91a24b0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x5595c91ae170 tx=0x7f446c009310 comp rx=0 tx=0).stop 2023-07-18T03:22:58.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.286+0000 7f4495e64b80 1 -- 172.21.15.101:0/1280160747 shutdown_connections 2023-07-18T03:22:58.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.286+0000 7f4495e64b80 1 --2- 172.21.15.101:0/1280160747 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595c918aad0 0x5595c9184b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.286+0000 7f4495e64b80 1 --2- 172.21.15.101:0/1280160747 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595c917f2d0 0x5595c91a24b0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.286+0000 7f4495e64b80 1 --2- 172.21.15.101:0/1280160747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595c917fbe0 0x5595c9188120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.286+0000 7f4495e64b80 1 -- 172.21.15.101:0/1280160747 >> 172.21.15.101:0/1280160747 conn(0x5595c9175da0 msgr2=0x5595c91a7770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:58.284 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.287+0000 7f4495e64b80 1 -- 172.21.15.101:0/1280160747 shutdown_connections 2023-07-18T03:22:58.284 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.287+0000 7f4495e64b80 1 -- 172.21.15.101:0/1280160747 wait complete. 2023-07-18T03:22:58.284 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.288+0000 7f4495e64b80 1 -- start start 2023-07-18T03:22:58.284 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.288+0000 7f4495e64b80 1 Processor -- start 2023-07-18T03:22:58.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.289+0000 7f4495e64b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595c917f2d0 0x5595c91a0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.289+0000 7f4495e64b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595c917fbe0 0x5595c91a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.289+0000 7f4495e64b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595c918aad0 0x5595c91a1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.289+0000 7f4495e64b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595c91a2ad0 con 0x5595c917f2d0 2023-07-18T03:22:58.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.289+0000 7f4495e64b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595c91ff320 con 0x5595c918aad0 2023-07-18T03:22:58.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.289+0000 7f4495e64b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595c91ff4e0 con 0x5595c917fbe0 2023-07-18T03:22:58.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.289+0000 7f4495e64b80 10 client.0 _ll_register_callbacks cb 0x7f447400cdd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:22:58.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.290+0000 7f4480b9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595c918aad0 0x5595c91a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:58.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.290+0000 7f4480b9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595c918aad0 0x5595c91a1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:53824/0 (socket says 172.21.15.101:53824) 2023-07-18T03:22:58.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.290+0000 7f4480b9c700 1 -- 172.21.15.101:0/401920414 learned_addr learned my addr 172.21.15.101:0/401920414 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:58.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.290+0000 7f447b7fe700 1 --2- 172.21.15.101:0/401920414 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595c917fbe0 0x5595c91a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:58.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.293+0000 7f447b7fe700 1 -- 172.21.15.101:0/401920414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595c918aad0 msgr2=0x5595c91a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:58.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.293+0000 7f447b7fe700 1 --2- 172.21.15.101:0/401920414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595c918aad0 0x5595c91a1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.293+0000 7f447b7fe700 1 -- 172.21.15.101:0/401920414 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595c917f2d0 msgr2=0x5595c91a0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:58.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.293+0000 7f447b7fe700 1 --2- 172.21.15.101:0/401920414 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595c917f2d0 0x5595c91a0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.293+0000 7f447b7fe700 1 -- 172.21.15.101:0/401920414 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f447c11bfa0 con 0x5595c917fbe0 2023-07-18T03:22:58.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.299+0000 7f447b7fe700 1 --2- 172.21.15.101:0/401920414 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595c917fbe0 0x5595c91a0fc0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f44700ca3d0 tx=0x7f4470098390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76062602c33c472d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:58.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.299+0000 7f445b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:22:58.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.299+0000 7f445b7fe700 1 -- 172.21.15.101:0/401920414 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470099bb0 con 0x5595c917fbe0 2023-07-18T03:22:58.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.300+0000 7f445b7fe700 1 -- 172.21.15.101:0/401920414 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f4470017360 con 0x5595c917fbe0 2023-07-18T03:22:58.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.300+0000 7f4495e64b80 1 -- 172.21.15.101:0/401920414 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4470099920 con 0x5595c917fbe0 2023-07-18T03:22:58.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.303+0000 7f445b7fe700 1 -- 172.21.15.101:0/401920414 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4470099740 con 0x5595c917fbe0 2023-07-18T03:22:58.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.303+0000 7f4495e64b80 10 client.28594 fetch_fsmap learned FSMap version 39 2023-07-18T03:22:58.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.303+0000 7f4495e64b80 1 -- 172.21.15.101:0/401920414 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5595c8efe2b0 con 0x5595c917fbe0 2023-07-18T03:22:58.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.304+0000 7f445b7fe700 1 -- 172.21.15.101:0/401920414 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4470099920 con 0x5595c917fbe0 2023-07-18T03:22:58.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.304+0000 7f445b7fe700 1 -- 172.21.15.101:0/401920414 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 39) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f44700ca020 con 0x5595c917fbe0 2023-07-18T03:22:58.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.304+0000 7f4495e64b80 10 client.28594 fetch_fsmap finished waiting for FSMap version 39 2023-07-18T03:22:58.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.304+0000 7f4495e64b80 10 client.28594 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:22:58.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.304+0000 7f4495e64b80 20 client.28594 populate_metadata read hostname 'smithi101' 2023-07-18T03:22:58.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.304+0000 7f4495e64b80 1 --2- 172.21.15.101:0/401920414 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5595c92007d0 0x5595c9202bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.304+0000 7f4495e64b80 4 client.28594 mds_command: new command op to 24541 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:22:58.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.304+0000 7f4495e64b80 1 -- 172.21.15.101:0/401920414 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5595c908d050 con 0x5595c92007d0 2023-07-18T03:22:58.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.305+0000 7f447bfff700 1 --2- 172.21.15.101:0/401920414 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5595c92007d0 0x5595c9202bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:58.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.306+0000 7f445b7fe700 10 client.28594 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:22:58.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.306+0000 7f447bfff700 1 --2- 172.21.15.101:0/401920414 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5595c92007d0 0x5595c9202bb0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b2a1783a433574d5 server_cookie=905ab68f25a651f3 in_seq=0 out_seq=0 2023-07-18T03:22:58.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.313+0000 7f445b7fe700 1 -- 172.21.15.101:0/401920414 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5595c908d050 con 0x5595c92007d0 2023-07-18T03:22:58.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.313+0000 7f445b7fe700 10 client.28594 handle_command_reply: tid=0 2023-07-18T03:22:58.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.313+0000 7f4495e64b80 1 client.28594 shutdown 2023-07-18T03:22:58.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.313+0000 7f4495e64b80 1 -- 172.21.15.101:0/401920414 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595c917fbe0 msgr2=0x5595c91a0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:58.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.313+0000 7f4495e64b80 1 --2- 172.21.15.101:0/401920414 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595c917fbe0 0x5595c91a0fc0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f44700ca3d0 tx=0x7f4470098390 comp rx=0 tx=0).stop 2023-07-18T03:22:58.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.313+0000 7f4495e64b80 1 -- 172.21.15.101:0/401920414 shutdown_connections 2023-07-18T03:22:58.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.313+0000 7f4495e64b80 1 --2- 172.21.15.101:0/401920414 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5595c92007d0 0x5595c9202bb0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.313+0000 7f4495e64b80 1 --2- 172.21.15.101:0/401920414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595c918aad0 0x5595c91a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.313+0000 7f4495e64b80 1 --2- 172.21.15.101:0/401920414 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595c917f2d0 0x5595c91a0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.313+0000 7f4495e64b80 1 --2- 172.21.15.101:0/401920414 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595c917fbe0 0x5595c91a0fc0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.314+0000 7f4495e64b80 1 -- 172.21.15.101:0/401920414 >> 172.21.15.101:0/401920414 conn(0x5595c9175da0 msgr2=0x5595c9188030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:58.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.314+0000 7f445b7fe700 0 client.28594 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:22:58.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.314+0000 7f4495e64b80 1 -- 172.21.15.101:0/401920414 shutdown_connections 2023-07-18T03:22:58.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.314+0000 7f4495e64b80 1 -- 172.21.15.101:0/401920414 wait complete. 2023-07-18T03:22:58.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.314+0000 7f4495e64b80 20 client.28594 trim_cache size 0 max 16384 2023-07-18T03:22:58.313 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:22:58.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.315+0000 7f44825ff700 1 -- 172.21.15.101:0/315511802 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f44640a14d0 msgr2=0x7f44640a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:58.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.315+0000 7f44825ff700 1 --2- 172.21.15.101:0/315511802 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f44640a14d0 0x7f44640a3980 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4474009e20 tx=0x7f4474009480 comp rx=0 tx=0).stop 2023-07-18T03:22:58.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.315+0000 7f44825ff700 1 -- 172.21.15.101:0/315511802 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f447c06d3a0 msgr2=0x7f447c1aae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:58.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.315+0000 7f44825ff700 1 --2- 172.21.15.101:0/315511802 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f447c06d3a0 0x7f447c1aae20 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f447000c9b0 tx=0x7f447000cd70 comp rx=0 tx=0).stop 2023-07-18T03:22:58.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.315+0000 7f44825ff700 1 -- 172.21.15.101:0/315511802 shutdown_connections 2023-07-18T03:22:58.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.315+0000 7f44825ff700 1 --2- 172.21.15.101:0/315511802 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f44640a14d0 0x7f44640a3980 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.315+0000 7f44825ff700 1 --2- 172.21.15.101:0/315511802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f447c10e280 0x7f447c1a4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.315+0000 7f44825ff700 1 --2- 172.21.15.101:0/315511802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f447c06c9a0 0x7f447c1aa8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.315+0000 7f44825ff700 1 --2- 172.21.15.101:0/315511802 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f447c06d3a0 0x7f447c1aae20 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.315+0000 7f44825ff700 1 -- 172.21.15.101:0/315511802 >> 172.21.15.101:0/315511802 conn(0x7f447c06bf70 msgr2=0x7f447c109d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:58.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.316+0000 7f44825ff700 1 -- 172.21.15.101:0/315511802 shutdown_connections 2023-07-18T03:22:58.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.316+0000 7f44825ff700 1 -- 172.21.15.101:0/315511802 wait complete. 2023-07-18T03:22:58.334 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-18T03:22:58.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:58 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/2811287076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:22:58.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.620+0000 7f636e3ff700 1 -- 172.21.15.101:0/2430172148 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63680ffad0 msgr2=0x7f63680fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:58.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.620+0000 7f636e3ff700 1 --2- 172.21.15.101:0/2430172148 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63680ffad0 0x7f63680fff40 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f635c009990 tx=0x7f635c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.620+0000 7f636e3ff700 1 -- 172.21.15.101:0/2430172148 shutdown_connections 2023-07-18T03:22:58.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.620+0000 7f636e3ff700 1 --2- 172.21.15.101:0/2430172148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6368100480 0x7f636810c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.620+0000 7f636e3ff700 1 --2- 172.21.15.101:0/2430172148 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6368105ae0 0x7f6368105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.620+0000 7f636e3ff700 1 --2- 172.21.15.101:0/2430172148 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63680ffad0 0x7f63680fff40 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.620+0000 7f636e3ff700 1 -- 172.21.15.101:0/2430172148 >> 172.21.15.101:0/2430172148 conn(0x7f63680fb5f0 msgr2=0x7f63680fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:58.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.621+0000 7f636e3ff700 1 -- 172.21.15.101:0/2430172148 shutdown_connections 2023-07-18T03:22:58.618 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.621+0000 7f636e3ff700 1 -- 172.21.15.101:0/2430172148 wait complete. 2023-07-18T03:22:58.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.622+0000 7f636e3ff700 1 Processor -- start 2023-07-18T03:22:58.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.623+0000 7f636e3ff700 1 -- start start 2023-07-18T03:22:58.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.623+0000 7f636e3ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63680ffad0 0x7f6368198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.623+0000 7f636e3ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6368100480 0x7f6368198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.623+0000 7f636e3ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6368105ae0 0x7f6368109400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.623+0000 7f636e3ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6368109bd0 con 0x7f6368105ae0 2023-07-18T03:22:58.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.623+0000 7f636e3ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6368109d40 con 0x7f63680ffad0 2023-07-18T03:22:58.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.623+0000 7f636e3ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6368109f00 con 0x7f6368100480 2023-07-18T03:22:58.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.624+0000 7f636c99c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6368105ae0 0x7f6368109400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:58.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.624+0000 7f636c99c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6368105ae0 0x7f6368109400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:40598/0 (socket says 172.21.15.101:40598) 2023-07-18T03:22:58.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.624+0000 7f636c99c700 1 -- 172.21.15.101:0/1218219015 learned_addr learned my addr 172.21.15.101:0/1218219015 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:58.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.624+0000 7f6367fff700 1 --2- 172.21.15.101:0/1218219015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63680ffad0 0x7f6368198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:58.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.624+0000 7f636c99c700 1 -- 172.21.15.101:0/1218219015 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6368100480 msgr2=0x7f6368198cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:58.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.624+0000 7f636c99c700 1 --2- 172.21.15.101:0/1218219015 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6368100480 0x7f6368198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.624+0000 7f636c99c700 1 -- 172.21.15.101:0/1218219015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63680ffad0 msgr2=0x7f6368198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:58.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.624+0000 7f636c99c700 1 --2- 172.21.15.101:0/1218219015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63680ffad0 0x7f6368198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.624+0000 7f636c99c700 1 -- 172.21.15.101:0/1218219015 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f635c0095f0 con 0x7f6368105ae0 2023-07-18T03:22:58.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.624+0000 7f6367fff700 1 --2- 172.21.15.101:0/1218219015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63680ffad0 0x7f6368198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:22:58.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.625+0000 7f636c99c700 1 --2- 172.21.15.101:0/1218219015 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6368105ae0 0x7f6368109400 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f636000ef30 tx=0x7f636000c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:58.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.625+0000 7f63657fa700 1 -- 172.21.15.101:0/1218219015 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6360018070 con 0x7f6368105ae0 2023-07-18T03:22:58.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.625+0000 7f63657fa700 1 -- 172.21.15.101:0/1218219015 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f636000f040 con 0x7f6368105ae0 2023-07-18T03:22:58.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.625+0000 7f63657fa700 1 -- 172.21.15.101:0/1218219015 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6360004020 con 0x7f6368105ae0 2023-07-18T03:22:58.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.625+0000 7f636e3ff700 1 -- 172.21.15.101:0/1218219015 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f636810a180 con 0x7f6368105ae0 2023-07-18T03:22:58.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.625+0000 7f636e3ff700 1 -- 172.21.15.101:0/1218219015 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f636819c730 con 0x7f6368105ae0 2023-07-18T03:22:58.628 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.631+0000 7f6381d85b80 1 Processor -- start 2023-07-18T03:22:58.628 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.632+0000 7f63657fa700 1 -- 172.21.15.101:0/1218219015 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f6360004ad0 con 0x7f6368105ae0 2023-07-18T03:22:58.629 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.632+0000 7f6381d85b80 1 -- start start 2023-07-18T03:22:58.629 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.633+0000 7f6381d85b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a34d4102d0 0x55a34d4106a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.633+0000 7f6381d85b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a34d410be0 0x55a34d41b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.633+0000 7f6381d85b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a34d41bad0 0x55a34d41deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.633+0000 7f6381d85b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a34d316cb0 con 0x55a34d410be0 2023-07-18T03:22:58.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.633+0000 7f6381d85b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a34d369a40 con 0x55a34d41bad0 2023-07-18T03:22:58.630 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.633+0000 7f6381d85b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a34d362ed0 con 0x55a34d4102d0 2023-07-18T03:22:58.631 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.633+0000 7f63677fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a34d4102d0 0x55a34d4106a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:58.631 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.634+0000 7f63677fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a34d4102d0 0x55a34d4106a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41176/0 (socket says 172.21.15.101:41176) 2023-07-18T03:22:58.632 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.634+0000 7f63677fe700 1 -- 172.21.15.101:0/1799229951 learned_addr learned my addr 172.21.15.101:0/1799229951 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:58.632 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.634+0000 7f63677fe700 1 --2- 172.21.15.101:0/1799229951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a34d41bad0 0x55a34d41deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:58.632 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.634+0000 7f6367fff700 1 --2- 172.21.15.101:0/1799229951 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a34d410be0 0x55a34d41b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:58.632 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.635+0000 7f63677fe700 1 -- 172.21.15.101:0/1799229951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a34d41bad0 msgr2=0x55a34d41deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:58.632 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.635+0000 7f63677fe700 1 --2- 172.21.15.101:0/1799229951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a34d41bad0 0x55a34d41deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.632 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.635+0000 7f63677fe700 1 -- 172.21.15.101:0/1799229951 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a34d410be0 msgr2=0x55a34d41b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:58.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.635+0000 7f63677fe700 1 --2- 172.21.15.101:0/1799229951 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a34d410be0 0x55a34d41b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.635+0000 7f63677fe700 1 -- 172.21.15.101:0/1799229951 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a34d190de0 con 0x55a34d4102d0 2023-07-18T03:22:58.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.635+0000 7f63677fe700 1 --2- 172.21.15.101:0/1799229951 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a34d4102d0 0x55a34d4106a0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f635c00b2e0 tx=0x7f635c009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9c1f94a74a7015c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:58.634 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.635+0000 7f63657fa700 1 --2- 172.21.15.101:0/1218219015 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f63500a1340 0x7f63500a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.634 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.636+0000 7f634e7fc700 1 -- 172.21.15.101:0/1799229951 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c00bd30 con 0x55a34d4102d0 2023-07-18T03:22:58.634 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.636+0000 7f634e7fc700 1 -- 172.21.15.101:0/1799229951 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f635c00be90 con 0x55a34d4102d0 2023-07-18T03:22:58.634 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.636+0000 7f634e7fc700 1 -- 172.21.15.101:0/1799229951 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c004dd0 con 0x55a34d4102d0 2023-07-18T03:22:58.634 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.636+0000 7f63657fa700 1 -- 172.21.15.101:0/1218219015 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f63600ccca0 con 0x7f6368105ae0 2023-07-18T03:22:58.635 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.637+0000 7f6367fff700 1 --2- 172.21.15.101:0/1218219015 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f63500a1340 0x7f63500a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:58.635 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.637+0000 7f6381d85b80 1 -- 172.21.15.101:0/1799229951 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a34d4102d0 msgr2=0x55a34d4106a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:58.635 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.637+0000 7f6381d85b80 1 --2- 172.21.15.101:0/1799229951 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a34d4102d0 0x55a34d4106a0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f635c00b2e0 tx=0x7f635c009510 comp rx=0 tx=0).stop 2023-07-18T03:22:58.635 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.637+0000 7f6381d85b80 1 -- 172.21.15.101:0/1799229951 shutdown_connections 2023-07-18T03:22:58.635 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.637+0000 7f6381d85b80 1 --2- 172.21.15.101:0/1799229951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a34d41bad0 0x55a34d41deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.635 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.637+0000 7f6381d85b80 1 --2- 172.21.15.101:0/1799229951 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a34d4102d0 0x55a34d4106a0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.635 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.637+0000 7f6381d85b80 1 --2- 172.21.15.101:0/1799229951 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a34d410be0 0x55a34d41b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.636 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.637+0000 7f6381d85b80 1 -- 172.21.15.101:0/1799229951 >> 172.21.15.101:0/1799229951 conn(0x55a34d406da0 msgr2=0x55a34d40c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:58.636 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.637+0000 7f6367fff700 1 --2- 172.21.15.101:0/1218219015 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f63500a1340 0x7f63500a37f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6358004430 tx=0x7f6358009260 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:58.636 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.638+0000 7f6381d85b80 1 -- 172.21.15.101:0/1799229951 shutdown_connections 2023-07-18T03:22:58.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.638+0000 7f6381d85b80 1 -- 172.21.15.101:0/1799229951 wait complete. 2023-07-18T03:22:58.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.639+0000 7f6381d85b80 1 -- start start 2023-07-18T03:22:58.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.639+0000 7f6381d85b80 1 Processor -- start 2023-07-18T03:22:58.637 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.640+0000 7f6381d85b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a34d4102d0 0x55a34d418820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.638 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.640+0000 7f6381d85b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a34d410be0 0x55a34d418d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.638 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.640+0000 7f6381d85b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a34d41bad0 0x55a34d4192a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.638 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.640+0000 7f6381d85b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a34d302ac0 con 0x55a34d41bad0 2023-07-18T03:22:58.638 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.640+0000 7f6381d85b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a34d3027c0 con 0x55a34d4102d0 2023-07-18T03:22:58.638 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.640+0000 7f6381d85b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a34d1e6920 con 0x55a34d410be0 2023-07-18T03:22:58.638 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.640+0000 7f6381d85b80 10 client.0 _ll_register_callbacks cb 0x55a34d300760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:22:58.638 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.640+0000 7f6367fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a34d4102d0 0x55a34d418820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:58.639 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.640+0000 7f6367fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a34d4102d0 0x55a34d418820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:53846/0 (socket says 172.21.15.101:53846) 2023-07-18T03:22:58.639 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.640+0000 7f6367fff700 1 -- 172.21.15.101:0/584664503 learned_addr learned my addr 172.21.15.101:0/584664503 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:58.639 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.640+0000 7f63677fe700 1 --2- 172.21.15.101:0/584664503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a34d410be0 0x55a34d418d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:58.639 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.640+0000 7f6367fff700 1 -- 172.21.15.101:0/584664503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a34d410be0 msgr2=0x55a34d418d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:58.639 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.640+0000 7f6367fff700 1 --2- 172.21.15.101:0/584664503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a34d410be0 0x55a34d418d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.639 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.640+0000 7f6367fff700 1 -- 172.21.15.101:0/584664503 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a34d41bad0 msgr2=0x55a34d4192a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-18T03:22:58.639 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.640+0000 7f6367fff700 1 --2- 172.21.15.101:0/584664503 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a34d41bad0 0x55a34d4192a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.639 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.640+0000 7f6367fff700 1 -- 172.21.15.101:0/584664503 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f636810d210 con 0x55a34d4102d0 2023-07-18T03:22:58.640 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.641+0000 7f6367fff700 1 --2- 172.21.15.101:0/584664503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a34d4102d0 0x55a34d418820 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f63580059d0 tx=0x7f6358004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f18346affd2248be server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:58.640 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.641+0000 7f63477fe700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:22:58.641 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.641+0000 7f63477fe700 1 -- 172.21.15.101:0/584664503 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6358012da0 con 0x55a34d4102d0 2023-07-18T03:22:58.641 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.641+0000 7f63477fe700 1 -- 172.21.15.101:0/584664503 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f635801b460 con 0x55a34d4102d0 2023-07-18T03:22:58.641 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.641+0000 7f6381d85b80 1 -- 172.21.15.101:0/584664503 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a34d2bc4f0 con 0x55a34d4102d0 2023-07-18T03:22:58.641 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.642+0000 7f63477fe700 1 -- 172.21.15.101:0/584664503 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f635801b620 con 0x55a34d4102d0 2023-07-18T03:22:58.641 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.642+0000 7f6381d85b80 10 client.28592 fetch_fsmap learned FSMap version 39 2023-07-18T03:22:58.642 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.642+0000 7f6381d85b80 1 -- 172.21.15.101:0/584664503 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55a34d179c60 con 0x55a34d4102d0 2023-07-18T03:22:58.642 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.642+0000 7f63477fe700 1 -- 172.21.15.101:0/584664503 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a34d2bc4f0 con 0x55a34d4102d0 2023-07-18T03:22:58.642 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.642+0000 7f63477fe700 1 -- 172.21.15.101:0/584664503 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 39) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f6358026980 con 0x55a34d4102d0 2023-07-18T03:22:58.642 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.642+0000 7f6381d85b80 10 client.28592 fetch_fsmap finished waiting for FSMap version 39 2023-07-18T03:22:58.642 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.642+0000 7f6381d85b80 10 client.28592 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:22:58.642 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.642+0000 7f6381d85b80 20 client.28592 populate_metadata read hostname 'smithi101' 2023-07-18T03:22:58.642 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.642+0000 7f6381d85b80 1 --2- 172.21.15.101:0/584664503 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55a34d42b4f0 0x55a34d42d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.642 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.642+0000 7f6381d85b80 4 client.28592 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:22:58.643 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.642+0000 7f6381d85b80 1 -- 172.21.15.101:0/584664503 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a34d35beb0 con 0x55a34d42b4f0 2023-07-18T03:22:58.643 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.643+0000 7f63677fe700 1 --2- 172.21.15.101:0/584664503 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55a34d42b4f0 0x55a34d42d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:58.644 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.648+0000 7f63677fe700 1 --2- 172.21.15.101:0/584664503 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55a34d42b4f0 0x55a34d42d900 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bdd5f45e5be7cebb server_cookie=4ba0569e68ad797d in_seq=0 out_seq=0 2023-07-18T03:22:58.645 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.648+0000 7f63477fe700 10 client.28592 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:22:58.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.650+0000 7f63477fe700 1 -- 172.21.15.101:0/584664503 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a34d35beb0 con 0x55a34d42b4f0 2023-07-18T03:22:58.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.650+0000 7f63477fe700 10 client.28592 handle_command_reply: tid=0 2023-07-18T03:22:58.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.650+0000 7f6381d85b80 1 client.28592 shutdown 2023-07-18T03:22:58.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.650+0000 7f6381d85b80 1 -- 172.21.15.101:0/584664503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a34d4102d0 msgr2=0x55a34d418820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:58.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.650+0000 7f6381d85b80 1 --2- 172.21.15.101:0/584664503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a34d4102d0 0x55a34d418820 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f63580059d0 tx=0x7f6358004750 comp rx=0 tx=0).stop 2023-07-18T03:22:58.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.650+0000 7f6381d85b80 1 -- 172.21.15.101:0/584664503 shutdown_connections 2023-07-18T03:22:58.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.650+0000 7f6381d85b80 1 --2- 172.21.15.101:0/584664503 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55a34d42b4f0 0x55a34d42d900 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.650+0000 7f6381d85b80 1 --2- 172.21.15.101:0/584664503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a34d4102d0 0x55a34d418820 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.650+0000 7f6381d85b80 1 --2- 172.21.15.101:0/584664503 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a34d41bad0 0x55a34d4192a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.650+0000 7f6381d85b80 1 --2- 172.21.15.101:0/584664503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a34d410be0 0x55a34d418d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.650+0000 7f6381d85b80 1 -- 172.21.15.101:0/584664503 >> 172.21.15.101:0/584664503 conn(0x55a34d406da0 msgr2=0x55a34d40c230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:58.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.652+0000 7f63477fe700 0 client.28592 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:22:58.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.652+0000 7f6381d85b80 1 -- 172.21.15.101:0/584664503 shutdown_connections 2023-07-18T03:22:58.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.652+0000 7f6381d85b80 1 -- 172.21.15.101:0/584664503 wait complete. 2023-07-18T03:22:58.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.652+0000 7f6381d85b80 20 client.28592 trim_cache size 0 max 16384 2023-07-18T03:22:58.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.670+0000 7f6381d85b80 1 Processor -- start 2023-07-18T03:22:58.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.670+0000 7f6381d85b80 1 -- start start 2023-07-18T03:22:58.668 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.670+0000 7f6381d85b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a34d4102d0 0x55a34d4224b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.668 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.670+0000 7f6381d85b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a34d410be0 0x55a34d413b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.668 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.670+0000 7f6381d85b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a34d41bad0 0x55a34d414070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.668 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.671+0000 7f6381d85b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a34d403fa0 con 0x55a34d41bad0 2023-07-18T03:22:58.668 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.671+0000 7f6381d85b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a34d406970 con 0x55a34d4102d0 2023-07-18T03:22:58.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.671+0000 7f6381d85b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a34d4108f0 con 0x55a34d410be0 2023-07-18T03:22:58.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.671+0000 7f6367fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a34d4102d0 0x55a34d4224b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:58.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.671+0000 7f63677fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a34d410be0 0x55a34d413b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:58.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.671+0000 7f636c99c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a34d41bad0 0x55a34d414070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:58.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.671+0000 7f6367fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a34d4102d0 0x55a34d4224b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:53850/0 (socket says 172.21.15.101:53850) 2023-07-18T03:22:58.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.671+0000 7f63677fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a34d410be0 0x55a34d413b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41202/0 (socket says 172.21.15.101:41202) 2023-07-18T03:22:58.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.671+0000 7f6367fff700 1 -- 172.21.15.101:0/3173910432 learned_addr learned my addr 172.21.15.101:0/3173910432 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:58.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.671+0000 7f63677fe700 1 -- 172.21.15.101:0/3173910432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a34d4102d0 msgr2=0x55a34d4224b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:58.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.671+0000 7f63677fe700 1 --2- 172.21.15.101:0/3173910432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a34d4102d0 0x55a34d4224b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.671+0000 7f63677fe700 1 -- 172.21.15.101:0/3173910432 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a34d41bad0 msgr2=0x55a34d414070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:58.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.671+0000 7f63677fe700 1 --2- 172.21.15.101:0/3173910432 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a34d41bad0 0x55a34d414070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.671+0000 7f63677fe700 1 -- 172.21.15.101:0/3173910432 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a34d1b7250 con 0x55a34d410be0 2023-07-18T03:22:58.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.671+0000 7f6367fff700 1 --2- 172.21.15.101:0/3173910432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a34d4102d0 0x55a34d4224b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:22:58.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.671+0000 7f636c99c700 1 --2- 172.21.15.101:0/3173910432 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a34d41bad0 0x55a34d414070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:22:58.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.672+0000 7f63677fe700 1 --2- 172.21.15.101:0/3173910432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a34d410be0 0x55a34d413b30 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f635c009350 tx=0x7f635c005a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b363555daf72ce5 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:58.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.672+0000 7f634effd700 1 -- 172.21.15.101:0/3173910432 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c034620 con 0x55a34d410be0 2023-07-18T03:22:58.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.672+0000 7f634effd700 1 -- 172.21.15.101:0/3173910432 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f635c00b310 con 0x55a34d410be0 2023-07-18T03:22:58.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.672+0000 7f634effd700 1 -- 172.21.15.101:0/3173910432 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c004500 con 0x55a34d410be0 2023-07-18T03:22:58.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.673+0000 7f6381d85b80 1 -- 172.21.15.101:0/3173910432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a34d410be0 msgr2=0x55a34d413b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:58.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.673+0000 7f6381d85b80 1 --2- 172.21.15.101:0/3173910432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a34d410be0 0x55a34d413b30 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f635c009350 tx=0x7f635c005a90 comp rx=0 tx=0).stop 2023-07-18T03:22:58.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.673+0000 7f6381d85b80 1 -- 172.21.15.101:0/3173910432 shutdown_connections 2023-07-18T03:22:58.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.673+0000 7f6381d85b80 1 --2- 172.21.15.101:0/3173910432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a34d4102d0 0x55a34d4224b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.673+0000 7f6381d85b80 1 --2- 172.21.15.101:0/3173910432 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a34d41bad0 0x55a34d414070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.673+0000 7f6381d85b80 1 --2- 172.21.15.101:0/3173910432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a34d410be0 0x55a34d413b30 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.673+0000 7f6381d85b80 1 -- 172.21.15.101:0/3173910432 >> 172.21.15.101:0/3173910432 conn(0x55a34d406da0 msgr2=0x55a34d418fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:58.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.673+0000 7f6381d85b80 1 -- 172.21.15.101:0/3173910432 shutdown_connections 2023-07-18T03:22:58.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.673+0000 7f6381d85b80 1 -- 172.21.15.101:0/3173910432 wait complete. 2023-07-18T03:22:58.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.674+0000 7f6381d85b80 1 -- start start 2023-07-18T03:22:58.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.674+0000 7f6381d85b80 1 Processor -- start 2023-07-18T03:22:58.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.675+0000 7f6381d85b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a34d4102d0 0x55a34d477f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.675+0000 7f6381d85b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a34d410be0 0x55a34d478450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.675+0000 7f6381d85b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a34d41bad0 0x55a34d421ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.675+0000 7f6381d85b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a34d422180 con 0x55a34d410be0 2023-07-18T03:22:58.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.675+0000 7f6381d85b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a34d4222f0 con 0x55a34d41bad0 2023-07-18T03:22:58.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.675+0000 7f6381d85b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a34d4224b0 con 0x55a34d4102d0 2023-07-18T03:22:58.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.675+0000 7f6381d85b80 10 client.0 _ll_register_callbacks cb 0x7f6358003850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:22:58.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.675+0000 7f6367fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a34d4102d0 0x55a34d477f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:58.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.675+0000 7f6367fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a34d4102d0 0x55a34d477f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41216/0 (socket says 172.21.15.101:41216) 2023-07-18T03:22:58.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.675+0000 7f6367fff700 1 -- 172.21.15.101:0/548590314 learned_addr learned my addr 172.21.15.101:0/548590314 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:58.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.675+0000 7f63677fe700 1 --2- 172.21.15.101:0/548590314 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a34d410be0 0x55a34d478450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:58.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.675+0000 7f636c99c700 1 --2- 172.21.15.101:0/548590314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a34d41bad0 0x55a34d421ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:58.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.676+0000 7f6367fff700 1 -- 172.21.15.101:0/548590314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a34d41bad0 msgr2=0x55a34d421ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:58.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.676+0000 7f6367fff700 1 --2- 172.21.15.101:0/548590314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a34d41bad0 0x55a34d421ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.676+0000 7f6367fff700 1 -- 172.21.15.101:0/548590314 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a34d410be0 msgr2=0x55a34d478450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:58.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.676+0000 7f6367fff700 1 --2- 172.21.15.101:0/548590314 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a34d410be0 0x55a34d478450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.676+0000 7f6367fff700 1 -- 172.21.15.101:0/548590314 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a34d179c60 con 0x55a34d4102d0 2023-07-18T03:22:58.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.677+0000 7f6367fff700 1 --2- 172.21.15.101:0/548590314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a34d4102d0 0x55a34d477f10 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f635802f9b0 tx=0x7f63580127e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=679d1a6957ef0fe0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:58.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.677+0000 7f63477fe700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:22:58.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.678+0000 7f63477fe700 1 -- 172.21.15.101:0/548590314 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635802fe20 con 0x55a34d4102d0 2023-07-18T03:22:58.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.678+0000 7f63477fe700 1 -- 172.21.15.101:0/548590314 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f6358012d20 con 0x55a34d4102d0 2023-07-18T03:22:58.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.678+0000 7f6381d85b80 1 -- 172.21.15.101:0/548590314 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a34d31e050 con 0x55a34d4102d0 2023-07-18T03:22:58.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.678+0000 7f63477fe700 1 -- 172.21.15.101:0/548590314 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6358005040 con 0x55a34d4102d0 2023-07-18T03:22:58.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.678+0000 7f6381d85b80 10 client.28609 fetch_fsmap learned FSMap version 39 2023-07-18T03:22:58.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.678+0000 7f6381d85b80 1 -- 172.21.15.101:0/548590314 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7f636819c730 con 0x55a34d4102d0 2023-07-18T03:22:58.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.678+0000 7f63477fe700 1 -- 172.21.15.101:0/548590314 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a34d31e050 con 0x55a34d4102d0 2023-07-18T03:22:58.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.679+0000 7f63477fe700 1 -- 172.21.15.101:0/548590314 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 39) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f6358026440 con 0x55a34d4102d0 2023-07-18T03:22:58.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.679+0000 7f6381d85b80 10 client.28609 fetch_fsmap finished waiting for FSMap version 39 2023-07-18T03:22:58.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.679+0000 7f6381d85b80 10 client.28609 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:22:58.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.679+0000 7f6381d85b80 20 client.28609 populate_metadata read hostname 'smithi101' 2023-07-18T03:22:58.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.679+0000 7f6381d85b80 1 --2- 172.21.15.101:0/548590314 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55a34d490da0 0x55a34d4931b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.679+0000 7f6381d85b80 4 client.28609 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:22:58.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.679+0000 7f6381d85b80 1 -- 172.21.15.101:0/548590314 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a34d1b1f70 con 0x55a34d490da0 2023-07-18T03:22:58.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.679+0000 7f63677fe700 1 --2- 172.21.15.101:0/548590314 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55a34d490da0 0x55a34d4931b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:58.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.680+0000 7f63677fe700 1 --2- 172.21.15.101:0/548590314 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55a34d490da0 0x55a34d4931b0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3cc6c960ca839aae server_cookie=2afd8a5fab198d80 in_seq=0 out_seq=0 2023-07-18T03:22:58.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.680+0000 7f63477fe700 10 client.28609 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:22:58.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.680+0000 7f63477fe700 1 -- 172.21.15.101:0/548590314 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a34d1b1f70 con 0x55a34d490da0 2023-07-18T03:22:58.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.680+0000 7f63477fe700 10 client.28609 handle_command_reply: tid=0 2023-07-18T03:22:58.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.680+0000 7f6381d85b80 1 client.28609 shutdown 2023-07-18T03:22:58.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.680+0000 7f6381d85b80 1 -- 172.21.15.101:0/548590314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a34d4102d0 msgr2=0x55a34d477f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:58.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.680+0000 7f6381d85b80 1 --2- 172.21.15.101:0/548590314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a34d4102d0 0x55a34d477f10 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f635802f9b0 tx=0x7f63580127e0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.680+0000 7f6381d85b80 1 -- 172.21.15.101:0/548590314 shutdown_connections 2023-07-18T03:22:58.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.680+0000 7f6381d85b80 1 --2- 172.21.15.101:0/548590314 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x55a34d490da0 0x55a34d4931b0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.680+0000 7f6381d85b80 1 --2- 172.21.15.101:0/548590314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a34d41bad0 0x55a34d421ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.680+0000 7f6381d85b80 1 --2- 172.21.15.101:0/548590314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a34d4102d0 0x55a34d477f10 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.680+0000 7f6381d85b80 1 --2- 172.21.15.101:0/548590314 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a34d410be0 0x55a34d478450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.680+0000 7f6381d85b80 1 -- 172.21.15.101:0/548590314 >> 172.21.15.101:0/548590314 conn(0x55a34d406da0 msgr2=0x55a34d413b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:58.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.680+0000 7f63477fe700 0 client.28609 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:22:58.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.681+0000 7f6381d85b80 1 -- 172.21.15.101:0/548590314 shutdown_connections 2023-07-18T03:22:58.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.681+0000 7f6381d85b80 1 -- 172.21.15.101:0/548590314 wait complete. 2023-07-18T03:22:58.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.681+0000 7f6381d85b80 20 client.28609 trim_cache size 0 max 16384 2023-07-18T03:22:58.690 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:22:58.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.681+0000 7f636e3ff700 1 -- 172.21.15.101:0/1218219015 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f63500a1340 msgr2=0x7f63500a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:58.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.681+0000 7f636e3ff700 1 --2- 172.21.15.101:0/1218219015 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f63500a1340 0x7f63500a37f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6358004430 tx=0x7f6358009260 comp rx=0 tx=0).stop 2023-07-18T03:22:58.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.681+0000 7f636e3ff700 1 -- 172.21.15.101:0/1218219015 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6368105ae0 msgr2=0x7f6368109400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:58.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.681+0000 7f636e3ff700 1 --2- 172.21.15.101:0/1218219015 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6368105ae0 0x7f6368109400 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f636000ef30 tx=0x7f636000c450 comp rx=0 tx=0).stop 2023-07-18T03:22:58.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.681+0000 7f636e3ff700 1 -- 172.21.15.101:0/1218219015 shutdown_connections 2023-07-18T03:22:58.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.681+0000 7f636e3ff700 1 --2- 172.21.15.101:0/1218219015 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f63500a1340 0x7f63500a37f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.681+0000 7f636e3ff700 1 --2- 172.21.15.101:0/1218219015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63680ffad0 0x7f6368198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.681+0000 7f636e3ff700 1 --2- 172.21.15.101:0/1218219015 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6368105ae0 0x7f6368109400 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.681+0000 7f636e3ff700 1 --2- 172.21.15.101:0/1218219015 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6368100480 0x7f6368198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.681+0000 7f636e3ff700 1 -- 172.21.15.101:0/1218219015 >> 172.21.15.101:0/1218219015 conn(0x7f63680fb5f0 msgr2=0x7f636810ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:58.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.682+0000 7f636e3ff700 1 -- 172.21.15.101:0/1218219015 shutdown_connections 2023-07-18T03:22:58.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.682+0000 7f636e3ff700 1 -- 172.21.15.101:0/1218219015 wait complete. 2023-07-18T03:22:58.698 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:58 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2811287076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:22:58.700 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-18T03:22:58.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:58 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/2811287076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:22:58.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.957+0000 7fda4579d700 1 -- 172.21.15.101:0/4149722599 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda400ff9f0 msgr2=0x7fda400ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:58.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.957+0000 7fda4579d700 1 --2- 172.21.15.101:0/4149722599 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda400ff9f0 0x7fda400ffe60 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fda34009990 tx=0x7fda34009ca0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.957+0000 7fda4579d700 1 -- 172.21.15.101:0/4149722599 shutdown_connections 2023-07-18T03:22:58.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.957+0000 7fda4579d700 1 --2- 172.21.15.101:0/4149722599 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda401003a0 0x7fda4010c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.957+0000 7fda4579d700 1 --2- 172.21.15.101:0/4149722599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda40105a00 0x7fda40105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.957+0000 7fda4579d700 1 --2- 172.21.15.101:0/4149722599 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda400ff9f0 0x7fda400ffe60 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.957+0000 7fda4579d700 1 -- 172.21.15.101:0/4149722599 >> 172.21.15.101:0/4149722599 conn(0x7fda400fb510 msgr2=0x7fda400fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:58.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.957+0000 7fda4579d700 1 -- 172.21.15.101:0/4149722599 shutdown_connections 2023-07-18T03:22:58.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.957+0000 7fda4579d700 1 -- 172.21.15.101:0/4149722599 wait complete. 2023-07-18T03:22:58.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.958+0000 7fda4579d700 1 Processor -- start 2023-07-18T03:22:58.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.959+0000 7fda4579d700 1 -- start start 2023-07-18T03:22:58.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.959+0000 7fda4579d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda400ff9f0 0x7fda4010a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.959+0000 7fda4579d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda401003a0 0x7fda4010a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.959+0000 7fda4579d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda40105a00 0x7fda401a7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.959+0000 7fda4579d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda401a81f0 con 0x7fda401003a0 2023-07-18T03:22:58.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.959+0000 7fda4579d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda401a8330 con 0x7fda400ff9f0 2023-07-18T03:22:58.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.959+0000 7fda4579d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda401a84c0 con 0x7fda40105a00 2023-07-18T03:22:58.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.959+0000 7fda3ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda400ff9f0 0x7fda4010a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:58.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.959+0000 7fda3ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda400ff9f0 0x7fda4010a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:53890/0 (socket says 172.21.15.101:53890) 2023-07-18T03:22:58.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.959+0000 7fda3ffff700 1 -- 172.21.15.101:0/843022827 learned_addr learned my addr 172.21.15.101:0/843022827 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:58.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.960+0000 7fda3f7fe700 1 --2- 172.21.15.101:0/843022827 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda401003a0 0x7fda4010a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:58.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.960+0000 7fda3ffff700 1 -- 172.21.15.101:0/843022827 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda40105a00 msgr2=0x7fda401a7cb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-18T03:22:58.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.960+0000 7fda3ffff700 1 --2- 172.21.15.101:0/843022827 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda40105a00 0x7fda401a7cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.960+0000 7fda3ffff700 1 -- 172.21.15.101:0/843022827 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda401003a0 msgr2=0x7fda4010a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:58.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.960+0000 7fda3ffff700 1 --2- 172.21.15.101:0/843022827 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda401003a0 0x7fda4010a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.960+0000 7fda3ffff700 1 -- 172.21.15.101:0/843022827 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda340095f0 con 0x7fda400ff9f0 2023-07-18T03:22:58.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.960+0000 7fda3ffff700 1 --2- 172.21.15.101:0/843022827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda400ff9f0 0x7fda4010a040 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fda3000e970 tx=0x7fda3000ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:58.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.961+0000 7fda3d7fa700 1 -- 172.21.15.101:0/843022827 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda30009790 con 0x7fda400ff9f0 2023-07-18T03:22:58.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.961+0000 7fda3d7fa700 1 -- 172.21.15.101:0/843022827 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda300044e0 con 0x7fda400ff9f0 2023-07-18T03:22:58.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.961+0000 7fda4579d700 1 -- 172.21.15.101:0/843022827 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda401a87a0 con 0x7fda400ff9f0 2023-07-18T03:22:58.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.961+0000 7fda3d7fa700 1 -- 172.21.15.101:0/843022827 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda30003d10 con 0x7fda400ff9f0 2023-07-18T03:22:58.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.961+0000 7fda4579d700 1 -- 172.21.15.101:0/843022827 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda401a8d00 con 0x7fda400ff9f0 2023-07-18T03:22:58.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.965+0000 7fda3d7fa700 1 -- 172.21.15.101:0/843022827 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fda30003eb0 con 0x7fda400ff9f0 2023-07-18T03:22:58.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.966+0000 7fda5a34eb80 1 Processor -- start 2023-07-18T03:22:58.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.967+0000 7fda5a34eb80 1 -- start start 2023-07-18T03:22:58.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.967+0000 7fda5a34eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564cadaca4f0 0x564cadaca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.967+0000 7fda5a34eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564cadacae00 0x564cadad47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.967+0000 7fda5a34eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564cadad4d30 0x564cadad7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.967+0000 7fda5a34eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cad9cfcb0 con 0x564cadaca4f0 2023-07-18T03:22:58.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.967+0000 7fda5a34eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cada22a40 con 0x564cadacae00 2023-07-18T03:22:58.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.967+0000 7fda5a34eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cada1bed0 con 0x564cadad4d30 2023-07-18T03:22:58.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.967+0000 7fda44f9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564cadaca4f0 0x564cadaca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:58.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.968+0000 7fda3f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564cadad4d30 0x564cadad7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:58.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.968+0000 7fda3f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564cadad4d30 0x564cadad7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41248/0 (socket says 172.21.15.101:41248) 2023-07-18T03:22:58.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.968+0000 7fda3f7fe700 1 -- 172.21.15.101:0/3084310315 learned_addr learned my addr 172.21.15.101:0/3084310315 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:58.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.968+0000 7fda3d7fa700 1 --2- 172.21.15.101:0/843022827 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fda280a13e0 0x7fda280a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.968+0000 7fda3ffff700 1 --2- 172.21.15.101:0/3084310315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564cadacae00 0x564cadad47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:58.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.968+0000 7fda3d7fa700 1 -- 172.21.15.101:0/843022827 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fda30014070 con 0x7fda400ff9f0 2023-07-18T03:22:58.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.968+0000 7fda3f7fe700 1 -- 172.21.15.101:0/3084310315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564cadacae00 msgr2=0x564cadad47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:58.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.968+0000 7fda3f7fe700 1 --2- 172.21.15.101:0/3084310315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564cadacae00 0x564cadad47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.968+0000 7fda3f7fe700 1 -- 172.21.15.101:0/3084310315 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564cadaca4f0 msgr2=0x564cadaca8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:58.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.968+0000 7fda3f7fe700 1 --2- 172.21.15.101:0/3084310315 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564cadaca4f0 0x564cadaca8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.968+0000 7fda3f7fe700 1 -- 172.21.15.101:0/3084310315 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564cad849de0 con 0x564cadad4d30 2023-07-18T03:22:58.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.968+0000 7fda44f9c700 1 --2- 172.21.15.101:0/843022827 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fda280a13e0 0x7fda280a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:58.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.968+0000 7fda3f7fe700 1 --2- 172.21.15.101:0/3084310315 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564cadad4d30 0x564cadad7110 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fda34009990 tx=0x7fda34005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb17912fc41ea70a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:58.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.969+0000 7fda267fc700 1 -- 172.21.15.101:0/3084310315 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3400beb0 con 0x564cadad4d30 2023-07-18T03:22:58.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.969+0000 7fda267fc700 1 -- 172.21.15.101:0/3084310315 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda3401e040 con 0x564cadad4d30 2023-07-18T03:22:58.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.969+0000 7fda267fc700 1 -- 172.21.15.101:0/3084310315 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda34004a60 con 0x564cadad4d30 2023-07-18T03:22:58.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.969+0000 7fda44f9c700 1 --2- 172.21.15.101:0/843022827 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fda280a13e0 0x7fda280a3890 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fda380093f0 tx=0x7fda38009040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:58.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.969+0000 7fda5a34eb80 1 -- 172.21.15.101:0/3084310315 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564cadad4d30 msgr2=0x564cadad7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:58.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.969+0000 7fda5a34eb80 1 --2- 172.21.15.101:0/3084310315 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564cadad4d30 0x564cadad7110 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fda34009990 tx=0x7fda34005e80 comp rx=0 tx=0).stop 2023-07-18T03:22:58.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.970+0000 7fda5a34eb80 1 -- 172.21.15.101:0/3084310315 shutdown_connections 2023-07-18T03:22:58.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.970+0000 7fda5a34eb80 1 --2- 172.21.15.101:0/3084310315 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564cadad4d30 0x564cadad7110 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.970+0000 7fda5a34eb80 1 --2- 172.21.15.101:0/3084310315 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564cadaca4f0 0x564cadaca8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.970+0000 7fda5a34eb80 1 --2- 172.21.15.101:0/3084310315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564cadacae00 0x564cadad47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.970+0000 7fda5a34eb80 1 -- 172.21.15.101:0/3084310315 >> 172.21.15.101:0/3084310315 conn(0x564cadabfff0 msgr2=0x564cadac5930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:58.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.970+0000 7fda5a34eb80 1 -- 172.21.15.101:0/3084310315 shutdown_connections 2023-07-18T03:22:58.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.970+0000 7fda5a34eb80 1 -- 172.21.15.101:0/3084310315 wait complete. 2023-07-18T03:22:58.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.970+0000 7fda5a34eb80 1 -- start start 2023-07-18T03:22:58.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.971+0000 7fda5a34eb80 1 Processor -- start 2023-07-18T03:22:58.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.972+0000 7fda5a34eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564cadaca4f0 0x564cadad4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.972+0000 7fda5a34eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564cadacae00 0x564cadad1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.972+0000 7fda5a34eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564cadad4d30 0x564cadad1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.972+0000 7fda5a34eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cad9bbac0 con 0x564cadacae00 2023-07-18T03:22:58.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.972+0000 7fda5a34eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cad9bb7c0 con 0x564cadad4d30 2023-07-18T03:22:58.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.972+0000 7fda5a34eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cad89f920 con 0x564cadaca4f0 2023-07-18T03:22:58.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.972+0000 7fda5a34eb80 10 client.0 _ll_register_callbacks cb 0x564cad9b9760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:22:58.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.972+0000 7fda44f9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564cadad4d30 0x564cadad1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:58.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.972+0000 7fda3f7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564cadacae00 0x564cadad1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:58.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.972+0000 7fda44f9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564cadad4d30 0x564cadad1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:53916/0 (socket says 172.21.15.101:53916) 2023-07-18T03:22:58.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.972+0000 7fda44f9c700 1 -- 172.21.15.101:0/7260399 learned_addr learned my addr 172.21.15.101:0/7260399 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:58.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.973+0000 7fda44f9c700 1 -- 172.21.15.101:0/7260399 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564cadaca4f0 msgr2=0x564cadad4610 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-18T03:22:58.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.973+0000 7fda44f9c700 1 --2- 172.21.15.101:0/7260399 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564cadaca4f0 0x564cadad4610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.973+0000 7fda44f9c700 1 -- 172.21.15.101:0/7260399 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564cadacae00 msgr2=0x564cadad1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:58.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.973+0000 7fda44f9c700 1 --2- 172.21.15.101:0/7260399 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564cadacae00 0x564cadad1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.973+0000 7fda44f9c700 1 -- 172.21.15.101:0/7260399 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda4010d130 con 0x564cadad4d30 2023-07-18T03:22:58.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.973+0000 7fda3f7fe700 1 --2- 172.21.15.101:0/7260399 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564cadacae00 0x564cadad1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:22:58.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.973+0000 7fda44f9c700 1 --2- 172.21.15.101:0/7260399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564cadad4d30 0x564cadad1e60 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fda38016bd0 tx=0x7fda38004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5f3e1ef451f5380 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:58.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.973+0000 7fda1f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:22:58.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.973+0000 7fda1f7fe700 1 -- 172.21.15.101:0/7260399 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3800e030 con 0x564cadad4d30 2023-07-18T03:22:58.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.973+0000 7fda5a34eb80 1 -- 172.21.15.101:0/7260399 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564cad9754f0 con 0x564cadad4d30 2023-07-18T03:22:58.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.973+0000 7fda1f7fe700 1 -- 172.21.15.101:0/7260399 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fda38016300 con 0x564cadad4d30 2023-07-18T03:22:58.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.973+0000 7fda1f7fe700 1 -- 172.21.15.101:0/7260399 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda38014810 con 0x564cadad4d30 2023-07-18T03:22:58.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.973+0000 7fda5a34eb80 10 client.28610 fetch_fsmap learned FSMap version 39 2023-07-18T03:22:58.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.973+0000 7fda5a34eb80 1 -- 172.21.15.101:0/7260399 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x564cad832c60 con 0x564cadad4d30 2023-07-18T03:22:58.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.974+0000 7fda1f7fe700 1 -- 172.21.15.101:0/7260399 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564cad9754f0 con 0x564cadad4d30 2023-07-18T03:22:58.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.974+0000 7fda1f7fe700 1 -- 172.21.15.101:0/7260399 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 39) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fda38015b20 con 0x564cadad4d30 2023-07-18T03:22:58.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.974+0000 7fda5a34eb80 10 client.28610 fetch_fsmap finished waiting for FSMap version 39 2023-07-18T03:22:58.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.974+0000 7fda5a34eb80 10 client.28610 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:22:58.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.974+0000 7fda5a34eb80 20 client.28610 populate_metadata read hostname 'smithi101' 2023-07-18T03:22:58.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.974+0000 7fda5a34eb80 1 --2- 172.21.15.101:0/7260399 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x564cadae46d0 0x564cadae6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.974+0000 7fda5a34eb80 4 client.28610 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:22:58.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.974+0000 7fda5a34eb80 1 -- 172.21.15.101:0/7260399 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564cada14eb0 con 0x564cadae46d0 2023-07-18T03:22:58.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.977+0000 7fda3ffff700 1 --2- 172.21.15.101:0/7260399 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x564cadae46d0 0x564cadae6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:58.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.978+0000 7fda3ffff700 1 --2- 172.21.15.101:0/7260399 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x564cadae46d0 0x564cadae6ae0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6999f6d44fd189d8 server_cookie=5abfc36c5c70668c in_seq=0 out_seq=0 2023-07-18T03:22:58.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.978+0000 7fda1f7fe700 10 client.28610 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:22:58.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.980+0000 7fda1f7fe700 1 -- 172.21.15.101:0/7260399 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564cada14eb0 con 0x564cadae46d0 2023-07-18T03:22:58.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.980+0000 7fda1f7fe700 10 client.28610 handle_command_reply: tid=0 2023-07-18T03:22:58.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.980+0000 7fda5a34eb80 1 client.28610 shutdown 2023-07-18T03:22:58.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.981+0000 7fda5a34eb80 1 -- 172.21.15.101:0/7260399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564cadad4d30 msgr2=0x564cadad1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:58.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.981+0000 7fda5a34eb80 1 --2- 172.21.15.101:0/7260399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564cadad4d30 0x564cadad1e60 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fda38016bd0 tx=0x7fda38004b40 comp rx=0 tx=0).stop 2023-07-18T03:22:58.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.982+0000 7fda5a34eb80 1 -- 172.21.15.101:0/7260399 shutdown_connections 2023-07-18T03:22:58.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.982+0000 7fda5a34eb80 1 --2- 172.21.15.101:0/7260399 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x564cadae46d0 0x564cadae6ae0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.982+0000 7fda5a34eb80 1 --2- 172.21.15.101:0/7260399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564cadad4d30 0x564cadad1e60 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.982+0000 7fda5a34eb80 1 --2- 172.21.15.101:0/7260399 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564cadaca4f0 0x564cadad4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.982+0000 7fda1f7fe700 0 client.28610 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:22:58.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.982+0000 7fda5a34eb80 1 --2- 172.21.15.101:0/7260399 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564cadacae00 0x564cadad1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.982+0000 7fda5a34eb80 1 -- 172.21.15.101:0/7260399 >> 172.21.15.101:0/7260399 conn(0x564cadabfff0 msgr2=0x564cadac54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:58.981 INFO:tasks.workunit.client.0.smithi101.stdout: 10 82265 30.92 MB/sec execute 23 sec latency 72.242 ms 2023-07-18T03:22:58.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.984+0000 7fda5a34eb80 1 -- 172.21.15.101:0/7260399 shutdown_connections 2023-07-18T03:22:58.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.985+0000 7fda5a34eb80 1 -- 172.21.15.101:0/7260399 wait complete. 2023-07-18T03:22:58.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:58.985+0000 7fda5a34eb80 20 client.28610 trim_cache size 0 max 16384 2023-07-18T03:22:58.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.003+0000 7fda5a34eb80 1 Processor -- start 2023-07-18T03:22:59.000 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.003+0000 7fda5a34eb80 1 -- start start 2023-07-18T03:22:59.000 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.003+0000 7fda5a34eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564cadaca4f0 0x564cadadb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:59.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.003+0000 7fda5a34eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564cadacae00 0x564cadad41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:59.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.003+0000 7fda5a34eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564cadad4d30 0x564cadaccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:59.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.003+0000 7fda5a34eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cadac5e50 con 0x564cadacae00 2023-07-18T03:22:59.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.003+0000 7fda5a34eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cadacc1a0 con 0x564cadaca4f0 2023-07-18T03:22:59.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.003+0000 7fda5a34eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cadacfeb0 con 0x564cadad4d30 2023-07-18T03:22:59.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.004+0000 7fda44f9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564cadad4d30 0x564cadaccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:59.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.004+0000 7fda44f9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564cadad4d30 0x564cadaccc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41266/0 (socket says 172.21.15.101:41266) 2023-07-18T03:22:59.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.004+0000 7fda44f9c700 1 -- 172.21.15.101:0/1175070816 learned_addr learned my addr 172.21.15.101:0/1175070816 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:59.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.004+0000 7fda3f7fe700 1 --2- 172.21.15.101:0/1175070816 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564cadacae00 0x564cadad41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:59.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.004+0000 7fda44f9c700 1 -- 172.21.15.101:0/1175070816 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564cadaca4f0 msgr2=0x564cadadb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:59.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.004+0000 7fda44f9c700 1 --2- 172.21.15.101:0/1175070816 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564cadaca4f0 0x564cadadb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:59.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.004+0000 7fda44f9c700 1 -- 172.21.15.101:0/1175070816 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564cadacae00 msgr2=0x564cadad41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:59.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.004+0000 7fda44f9c700 1 --2- 172.21.15.101:0/1175070816 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564cadacae00 0x564cadad41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:59.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.004+0000 7fda44f9c700 1 -- 172.21.15.101:0/1175070816 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564cad870250 con 0x564cadad4d30 2023-07-18T03:22:59.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.004+0000 7fda44f9c700 1 --2- 172.21.15.101:0/1175070816 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564cadad4d30 0x564cadaccc90 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fda38009010 tx=0x7fda380099a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18f9b059fd6b6349 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:59.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.005+0000 7fda26ffd700 1 -- 172.21.15.101:0/1175070816 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda38021620 con 0x564cadad4d30 2023-07-18T03:22:59.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.005+0000 7fda26ffd700 1 -- 172.21.15.101:0/1175070816 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda38016300 con 0x564cadad4d30 2023-07-18T03:22:59.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.005+0000 7fda26ffd700 1 -- 172.21.15.101:0/1175070816 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda38014b30 con 0x564cadad4d30 2023-07-18T03:22:59.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.006+0000 7fda5a34eb80 1 -- 172.21.15.101:0/1175070816 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564cadad4d30 msgr2=0x564cadaccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:59.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.006+0000 7fda5a34eb80 1 --2- 172.21.15.101:0/1175070816 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564cadad4d30 0x564cadaccc90 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fda38009010 tx=0x7fda380099a0 comp rx=0 tx=0).stop 2023-07-18T03:22:59.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.006+0000 7fda5a34eb80 1 -- 172.21.15.101:0/1175070816 shutdown_connections 2023-07-18T03:22:59.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.006+0000 7fda5a34eb80 1 --2- 172.21.15.101:0/1175070816 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564cadaca4f0 0x564cadadb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:59.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.006+0000 7fda5a34eb80 1 --2- 172.21.15.101:0/1175070816 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564cadad4d30 0x564cadaccc90 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:59.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.006+0000 7fda5a34eb80 1 --2- 172.21.15.101:0/1175070816 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564cadacae00 0x564cadad41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:59.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.006+0000 7fda5a34eb80 1 -- 172.21.15.101:0/1175070816 >> 172.21.15.101:0/1175070816 conn(0x564cadabfff0 msgr2=0x564cadae09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:59.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.006+0000 7fda5a34eb80 1 -- 172.21.15.101:0/1175070816 shutdown_connections 2023-07-18T03:22:59.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.006+0000 7fda5a34eb80 1 -- 172.21.15.101:0/1175070816 wait complete. 2023-07-18T03:22:59.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.006+0000 7fda5a34eb80 1 -- start start 2023-07-18T03:22:59.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.007+0000 7fda5a34eb80 1 Processor -- start 2023-07-18T03:22:59.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.008+0000 7fda5a34eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564cadaca4f0 0x564cadb2c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:59.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.008+0000 7fda5a34eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564cadacae00 0x564cadb2cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:59.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.008+0000 7fda5a34eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564cadad4d30 0x564cadb316c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:59.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.008+0000 7fda5a34eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cadb31c00 con 0x564cadaca4f0 2023-07-18T03:22:59.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.008+0000 7fda5a34eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cadb31d70 con 0x564cadad4d30 2023-07-18T03:22:59.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.008+0000 7fda5a34eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cadb31f30 con 0x564cadacae00 2023-07-18T03:22:59.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.008+0000 7fda5a34eb80 10 client.0 _ll_register_callbacks cb 0x564cad9b9760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:22:59.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.009+0000 7fda3f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564cadacae00 0x564cadb2cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:59.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.009+0000 7fda3f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564cadacae00 0x564cadb2cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41274/0 (socket says 172.21.15.101:41274) 2023-07-18T03:22:59.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.009+0000 7fda3f7fe700 1 -- 172.21.15.101:0/2130365361 learned_addr learned my addr 172.21.15.101:0/2130365361 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:22:59.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.009+0000 7fda44f9c700 1 --2- 172.21.15.101:0/2130365361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564cadad4d30 0x564cadb316c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:59.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.009+0000 7fda3ffff700 1 --2- 172.21.15.101:0/2130365361 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564cadaca4f0 0x564cadb2c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:59.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.009+0000 7fda3f7fe700 1 -- 172.21.15.101:0/2130365361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564cadad4d30 msgr2=0x564cadb316c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:59.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.009+0000 7fda3f7fe700 1 --2- 172.21.15.101:0/2130365361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564cadad4d30 0x564cadb316c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:59.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.010+0000 7fda3f7fe700 1 -- 172.21.15.101:0/2130365361 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564cadaca4f0 msgr2=0x564cadb2c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:22:59.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.010+0000 7fda3f7fe700 1 --2- 172.21.15.101:0/2130365361 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564cadaca4f0 0x564cadb2c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:59.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.010+0000 7fda3f7fe700 1 -- 172.21.15.101:0/2130365361 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564cad832c60 con 0x564cadacae00 2023-07-18T03:22:59.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.010+0000 7fda3f7fe700 1 --2- 172.21.15.101:0/2130365361 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564cadacae00 0x564cadb2cc50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fda3400b2e0 tx=0x7fda340106a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc9ed382cc27fb89 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:59.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.010+0000 7fda1f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:22:59.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.011+0000 7fda1f7fe700 1 -- 172.21.15.101:0/2130365361 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda34010c50 con 0x564cadacae00 2023-07-18T03:22:59.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.011+0000 7fda1f7fe700 1 -- 172.21.15.101:0/2130365361 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fda34010db0 con 0x564cadacae00 2023-07-18T03:22:59.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.011+0000 7fda5a34eb80 1 -- 172.21.15.101:0/2130365361 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda3401e040 con 0x564cadacae00 2023-07-18T03:22:59.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.011+0000 7fda1f7fe700 1 -- 172.21.15.101:0/2130365361 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda34036460 con 0x564cadacae00 2023-07-18T03:22:59.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.013+0000 7fda5a34eb80 10 client.28621 fetch_fsmap learned FSMap version 39 2023-07-18T03:22:59.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.013+0000 7fda5a34eb80 1 -- 172.21.15.101:0/2130365361 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x564cad8482b0 con 0x564cadacae00 2023-07-18T03:22:59.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.013+0000 7fda1f7fe700 1 -- 172.21.15.101:0/2130365361 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda3401e040 con 0x564cadacae00 2023-07-18T03:22:59.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.013+0000 7fda1f7fe700 1 -- 172.21.15.101:0/2130365361 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 39) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fda34040680 con 0x564cadacae00 2023-07-18T03:22:59.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.013+0000 7fda5a34eb80 10 client.28621 fetch_fsmap finished waiting for FSMap version 39 2023-07-18T03:22:59.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.013+0000 7fda5a34eb80 10 client.28621 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:22:59.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.013+0000 7fda5a34eb80 20 client.28621 populate_metadata read hostname 'smithi101' 2023-07-18T03:22:59.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.013+0000 7fda5a34eb80 1 --2- 172.21.15.101:0/2130365361 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x564cadb391d0 0x564cadb3b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:59.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.013+0000 7fda5a34eb80 4 client.28621 mds_command: new command op to 24562 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:22:59.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.013+0000 7fda5a34eb80 1 -- 172.21.15.101:0/2130365361 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564cad9d7050 con 0x564cadb391d0 2023-07-18T03:22:59.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.014+0000 7fda3ffff700 1 --2- 172.21.15.101:0/2130365361 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x564cadb391d0 0x564cadb3b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:59.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.014+0000 7fda3ffff700 1 --2- 172.21.15.101:0/2130365361 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x564cadb391d0 0x564cadb3b5e0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=687adba991eceee9 server_cookie=ec6a71eb4f143537 in_seq=0 out_seq=0 2023-07-18T03:22:59.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.014+0000 7fda1f7fe700 10 client.28621 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:22:59.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.018+0000 7fda1f7fe700 1 -- 172.21.15.101:0/2130365361 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564cad9d7050 con 0x564cadb391d0 2023-07-18T03:22:59.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.018+0000 7fda1f7fe700 10 client.28621 handle_command_reply: tid=0 2023-07-18T03:22:59.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.018+0000 7fda5a34eb80 1 client.28621 shutdown 2023-07-18T03:22:59.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.018+0000 7fda5a34eb80 1 -- 172.21.15.101:0/2130365361 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564cadacae00 msgr2=0x564cadb2cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:59.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.018+0000 7fda5a34eb80 1 --2- 172.21.15.101:0/2130365361 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564cadacae00 0x564cadb2cc50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fda3400b2e0 tx=0x7fda340106a0 comp rx=0 tx=0).stop 2023-07-18T03:22:59.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.018+0000 7fda5a34eb80 1 -- 172.21.15.101:0/2130365361 shutdown_connections 2023-07-18T03:22:59.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.018+0000 7fda5a34eb80 1 --2- 172.21.15.101:0/2130365361 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x564cadb391d0 0x564cadb3b5e0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:59.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.018+0000 7fda5a34eb80 1 --2- 172.21.15.101:0/2130365361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564cadad4d30 0x564cadb316c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:59.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.018+0000 7fda5a34eb80 1 --2- 172.21.15.101:0/2130365361 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564cadaca4f0 0x564cadb2c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:59.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.018+0000 7fda5a34eb80 1 --2- 172.21.15.101:0/2130365361 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564cadacae00 0x564cadb2cc50 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:59.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.018+0000 7fda1f7fe700 0 client.28621 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:22:59.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.018+0000 7fda5a34eb80 1 -- 172.21.15.101:0/2130365361 >> 172.21.15.101:0/2130365361 conn(0x564cadabfff0 msgr2=0x564cadaccc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:59.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.018+0000 7fda5a34eb80 1 -- 172.21.15.101:0/2130365361 shutdown_connections 2023-07-18T03:22:59.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.018+0000 7fda5a34eb80 1 -- 172.21.15.101:0/2130365361 wait complete. 2023-07-18T03:22:59.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.019+0000 7fda5a34eb80 20 client.28621 trim_cache size 0 max 16384 2023-07-18T03:22:59.022 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:22:59.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.019+0000 7fda4579d700 1 -- 172.21.15.101:0/843022827 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fda280a13e0 msgr2=0x7fda280a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:59.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.019+0000 7fda4579d700 1 --2- 172.21.15.101:0/843022827 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fda280a13e0 0x7fda280a3890 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fda380093f0 tx=0x7fda38009040 comp rx=0 tx=0).stop 2023-07-18T03:22:59.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.019+0000 7fda4579d700 1 -- 172.21.15.101:0/843022827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda400ff9f0 msgr2=0x7fda4010a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:59.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.019+0000 7fda4579d700 1 --2- 172.21.15.101:0/843022827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda400ff9f0 0x7fda4010a040 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fda3000e970 tx=0x7fda3000ed30 comp rx=0 tx=0).stop 2023-07-18T03:22:59.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.019+0000 7fda4579d700 1 -- 172.21.15.101:0/843022827 shutdown_connections 2023-07-18T03:22:59.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.019+0000 7fda4579d700 1 --2- 172.21.15.101:0/843022827 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fda280a13e0 0x7fda280a3890 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:59.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.019+0000 7fda4579d700 1 --2- 172.21.15.101:0/843022827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda400ff9f0 0x7fda4010a040 secure :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fda3000e970 tx=0x7fda3000ed30 comp rx=0 tx=0).stop 2023-07-18T03:22:59.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.019+0000 7fda4579d700 1 --2- 172.21.15.101:0/843022827 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda40105a00 0x7fda401a7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:59.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.019+0000 7fda4579d700 1 --2- 172.21.15.101:0/843022827 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda401003a0 0x7fda4010a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:59.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.019+0000 7fda4579d700 1 -- 172.21.15.101:0/843022827 >> 172.21.15.101:0/843022827 conn(0x7fda400fb510 msgr2=0x7fda400fd220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:59.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.019+0000 7fda4579d700 1 -- 172.21.15.101:0/843022827 shutdown_connections 2023-07-18T03:22:59.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:22:59.020+0000 7fda4579d700 1 -- 172.21.15.101:0/843022827 wait complete. 2023-07-18T03:22:59.492 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:22:59 smithi101 ceph-mon[126379]: pgmap v483: 129 pgs: 129 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 20 MiB/s wr, 641 op/s 2023-07-18T03:22:59.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:22:59 smithi155 ceph-mon[128885]: pgmap v483: 129 pgs: 129 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 20 MiB/s wr, 641 op/s 2023-07-18T03:22:59.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:22:59 smithi179 ceph-mon[129229]: pgmap v483: 129 pgs: 129 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 20 MiB/s wr, 641 op/s 2023-07-18T03:22:59.978 INFO:tasks.workunit.client.0.smithi101.stdout: 10 83265 31.98 MB/sec execute 24 sec latency 71.436 ms 2023-07-18T03:23:00.036 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-18T03:23:00.037 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-18T03:23:00.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.281+0000 7fe9399ff700 1 -- 172.21.15.101:0/2754900501 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe934101610 msgr2=0x7fe934109b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:00.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.281+0000 7fe9399ff700 1 --2- 172.21.15.101:0/2754900501 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe934101610 0x7fe934109b00 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe928009990 tx=0x7fe928009ca0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.281+0000 7fe9399ff700 1 -- 172.21.15.101:0/2754900501 shutdown_connections 2023-07-18T03:23:00.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.281+0000 7fe9399ff700 1 --2- 172.21.15.101:0/2754900501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe93410a040 0x7fe93410c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.281+0000 7fe9399ff700 1 --2- 172.21.15.101:0/2754900501 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe934100d00 0x7fe9341010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.281+0000 7fe9399ff700 1 --2- 172.21.15.101:0/2754900501 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe934101610 0x7fe934109b00 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.281+0000 7fe9399ff700 1 -- 172.21.15.101:0/2754900501 >> 172.21.15.101:0/2754900501 conn(0x7fe9340fa4b0 msgr2=0x7fe9340fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:00.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.281+0000 7fe9399ff700 1 -- 172.21.15.101:0/2754900501 shutdown_connections 2023-07-18T03:23:00.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.281+0000 7fe9399ff700 1 -- 172.21.15.101:0/2754900501 wait complete. 2023-07-18T03:23:00.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.282+0000 7fe9399ff700 1 Processor -- start 2023-07-18T03:23:00.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.282+0000 7fe9399ff700 1 -- start start 2023-07-18T03:23:00.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.282+0000 7fe9399ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe934100d00 0x7fe934197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:00.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.282+0000 7fe9399ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe934101610 0x7fe934198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:00.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.282+0000 7fe9399ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe93410a040 0x7fe93419c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:00.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.282+0000 7fe9399ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe93419ce20 con 0x7fe934100d00 2023-07-18T03:23:00.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.282+0000 7fe9399ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe93419cf90 con 0x7fe934101610 2023-07-18T03:23:00.284 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.282+0000 7fe9399ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe93419d150 con 0x7fe93410a040 2023-07-18T03:23:00.284 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.283+0000 7fe9327fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe934101610 0x7fe934198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:00.284 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.283+0000 7fe932ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe934100d00 0x7fe934197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:00.284 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.283+0000 7fe9327fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe934101610 0x7fe934198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:53956/0 (socket says 172.21.15.101:53956) 2023-07-18T03:23:00.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.283+0000 7fe9327fc700 1 -- 172.21.15.101:0/2218417 learned_addr learned my addr 172.21.15.101:0/2218417 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:23:00.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.283+0000 7fe9337fe700 1 --2- 172.21.15.101:0/2218417 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe93410a040 0x7fe93419c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:00.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.283+0000 7fe9327fc700 1 -- 172.21.15.101:0/2218417 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe93410a040 msgr2=0x7fe93419c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:00.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.283+0000 7fe9327fc700 1 --2- 172.21.15.101:0/2218417 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe93410a040 0x7fe93419c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.283+0000 7fe9327fc700 1 -- 172.21.15.101:0/2218417 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe934100d00 msgr2=0x7fe934197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:00.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.283+0000 7fe9327fc700 1 --2- 172.21.15.101:0/2218417 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe934100d00 0x7fe934197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.283+0000 7fe9327fc700 1 -- 172.21.15.101:0/2218417 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe924009520 con 0x7fe934101610 2023-07-18T03:23:00.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.283+0000 7fe932ffd700 1 --2- 172.21.15.101:0/2218417 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe934100d00 0x7fe934197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:23:00.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.283+0000 7fe9327fc700 1 --2- 172.21.15.101:0/2218417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe934101610 0x7fe934198380 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe928005910 tx=0x7fe928005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:00.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.284+0000 7fe9389fd700 1 -- 172.21.15.101:0/2218417 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92801d070 con 0x7fe934101610 2023-07-18T03:23:00.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.284+0000 7fe9399ff700 1 -- 172.21.15.101:0/2218417 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe92c009c40 con 0x7fe934101610 2023-07-18T03:23:00.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.284+0000 7fe9389fd700 1 -- 172.21.15.101:0/2218417 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe928022470 con 0x7fe934101610 2023-07-18T03:23:00.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.284+0000 7fe9389fd700 1 -- 172.21.15.101:0/2218417 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe928017650 con 0x7fe934101610 2023-07-18T03:23:00.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.284+0000 7fe9399ff700 1 -- 172.21.15.101:0/2218417 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9280095f0 con 0x7fe934101610 2023-07-18T03:23:00.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.288+0000 7fe9389fd700 1 -- 172.21.15.101:0/2218417 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fe9280225e0 con 0x7fe934101610 2023-07-18T03:23:00.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.288+0000 7fe94d3bfb80 1 Processor -- start 2023-07-18T03:23:00.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.289+0000 7fe94d3bfb80 1 -- start start 2023-07-18T03:23:00.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.289+0000 7fe94d3bfb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ec87f6a350 0x55ec87f6a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:00.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.289+0000 7fe94d3bfb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec87f6ac60 0x55ec87f75610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:00.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.289+0000 7fe94d3bfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec87f75b50 0x55ec87f77f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:00.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.289+0000 7fe94d3bfb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec87e70d30 con 0x55ec87f6a350 2023-07-18T03:23:00.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.289+0000 7fe94d3bfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec87ec3ac0 con 0x55ec87f75b50 2023-07-18T03:23:00.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.289+0000 7fe94d3bfb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec87ebcf50 con 0x55ec87f6ac60 2023-07-18T03:23:00.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.289+0000 7fe932ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec87f6ac60 0x55ec87f75610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:00.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.289+0000 7fe932ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec87f6ac60 0x55ec87f75610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41288/0 (socket says 172.21.15.101:41288) 2023-07-18T03:23:00.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.289+0000 7fe932ffd700 1 -- 172.21.15.101:0/2084224774 learned_addr learned my addr 172.21.15.101:0/2084224774 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:23:00.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.289+0000 7fe9337fe700 1 --2- 172.21.15.101:0/2084224774 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ec87f6a350 0x55ec87f6a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:00.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.289+0000 7fe9327fc700 1 --2- 172.21.15.101:0/2084224774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec87f75b50 0x55ec87f77f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:00.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.290+0000 7fe9389fd700 1 --2- 172.21.15.101:0/2218417 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fe91c0a12c0 0x7fe91c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:00.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.290+0000 7fe932ffd700 1 -- 172.21.15.101:0/2084224774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec87f75b50 msgr2=0x55ec87f77f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:00.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.290+0000 7fe932ffd700 1 --2- 172.21.15.101:0/2084224774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec87f75b50 0x55ec87f77f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.290+0000 7fe932ffd700 1 -- 172.21.15.101:0/2084224774 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ec87f6a350 msgr2=0x55ec87f6a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:00.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.290+0000 7fe932ffd700 1 --2- 172.21.15.101:0/2084224774 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ec87f6a350 0x55ec87f6a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.290+0000 7fe932ffd700 1 -- 172.21.15.101:0/2084224774 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec87ceae60 con 0x55ec87f6ac60 2023-07-18T03:23:00.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.290+0000 7fe9389fd700 1 -- 172.21.15.101:0/2218417 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fe9280ce430 con 0x7fe934101610 2023-07-18T03:23:00.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.290+0000 7fe932ffd700 1 --2- 172.21.15.101:0/2084224774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec87f6ac60 0x55ec87f75610 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe924011760 tx=0x7fe924011a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42c4841ddc6ad5d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:00.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.291+0000 7fe919ffb700 1 -- 172.21.15.101:0/2084224774 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92400f480 con 0x55ec87f6ac60 2023-07-18T03:23:00.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.291+0000 7fe919ffb700 1 -- 172.21.15.101:0/2084224774 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe92400faa0 con 0x55ec87f6ac60 2023-07-18T03:23:00.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.291+0000 7fe919ffb700 1 -- 172.21.15.101:0/2084224774 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe924019b40 con 0x55ec87f6ac60 2023-07-18T03:23:00.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.292+0000 7fe94d3bfb80 1 -- 172.21.15.101:0/2084224774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec87f6ac60 msgr2=0x55ec87f75610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:00.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.292+0000 7fe94d3bfb80 1 --2- 172.21.15.101:0/2084224774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec87f6ac60 0x55ec87f75610 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe924011760 tx=0x7fe924011a70 comp rx=0 tx=0).stop 2023-07-18T03:23:00.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.292+0000 7fe9337fe700 1 --2- 172.21.15.101:0/2218417 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fe91c0a12c0 0x7fe91c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:00.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.292+0000 7fe94d3bfb80 1 -- 172.21.15.101:0/2084224774 shutdown_connections 2023-07-18T03:23:00.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.292+0000 7fe94d3bfb80 1 --2- 172.21.15.101:0/2084224774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec87f75b50 0x55ec87f77f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.292+0000 7fe94d3bfb80 1 --2- 172.21.15.101:0/2084224774 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ec87f6a350 0x55ec87f6a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.292+0000 7fe94d3bfb80 1 --2- 172.21.15.101:0/2084224774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec87f6ac60 0x55ec87f75610 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.292+0000 7fe94d3bfb80 1 -- 172.21.15.101:0/2084224774 >> 172.21.15.101:0/2084224774 conn(0x55ec87f60e20 msgr2=0x55ec87f66720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:00.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.292+0000 7fe94d3bfb80 1 -- 172.21.15.101:0/2084224774 shutdown_connections 2023-07-18T03:23:00.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.292+0000 7fe94d3bfb80 1 -- 172.21.15.101:0/2084224774 wait complete. 2023-07-18T03:23:00.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.292+0000 7fe9337fe700 1 --2- 172.21.15.101:0/2218417 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fe91c0a12c0 0x7fe91c0a3770 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe92c004a60 tx=0x7fe92c009040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:00.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.293+0000 7fe94d3bfb80 1 -- start start 2023-07-18T03:23:00.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.293+0000 7fe94d3bfb80 1 Processor -- start 2023-07-18T03:23:00.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.294+0000 7fe94d3bfb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ec87f6a350 0x55ec87f755b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:00.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.294+0000 7fe94d3bfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec87f6ac60 0x55ec87f72830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:00.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.294+0000 7fe94d3bfb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec87f75b50 0x55ec87f72f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:00.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.294+0000 7fe94d3bfb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec87e5cb40 con 0x55ec87f6a350 2023-07-18T03:23:00.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.294+0000 7fe94d3bfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec87e5c840 con 0x55ec87f6ac60 2023-07-18T03:23:00.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.294+0000 7fe94d3bfb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec87d409a0 con 0x55ec87f75b50 2023-07-18T03:23:00.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.294+0000 7fe94d3bfb80 10 client.0 _ll_register_callbacks cb 0x55ec87e5a7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:23:00.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.294+0000 7fe9337fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec87f75b50 0x55ec87f72f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:00.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.294+0000 7fe9337fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec87f75b50 0x55ec87f72f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41302/0 (socket says 172.21.15.101:41302) 2023-07-18T03:23:00.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.294+0000 7fe9337fe700 1 -- 172.21.15.101:0/3569784327 learned_addr learned my addr 172.21.15.101:0/3569784327 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:23:00.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.295+0000 7fe9327fc700 1 --2- 172.21.15.101:0/3569784327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec87f6ac60 0x55ec87f72830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:00.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.295+0000 7fe932ffd700 1 --2- 172.21.15.101:0/3569784327 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ec87f6a350 0x55ec87f755b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:00.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.295+0000 7fe9337fe700 1 -- 172.21.15.101:0/3569784327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec87f6ac60 msgr2=0x55ec87f72830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:00.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.295+0000 7fe9337fe700 1 --2- 172.21.15.101:0/3569784327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec87f6ac60 0x55ec87f72830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.295+0000 7fe9337fe700 1 -- 172.21.15.101:0/3569784327 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ec87f6a350 msgr2=0x55ec87f755b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:00.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.295+0000 7fe9337fe700 1 --2- 172.21.15.101:0/3569784327 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ec87f6a350 0x55ec87f755b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.295+0000 7fe9337fe700 1 -- 172.21.15.101:0/3569784327 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe924011440 con 0x55ec87f75b50 2023-07-18T03:23:00.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.295+0000 7fe932ffd700 1 --2- 172.21.15.101:0/3569784327 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ec87f6a350 0x55ec87f755b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:23:00.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.295+0000 7fe9337fe700 1 --2- 172.21.15.101:0/3569784327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec87f75b50 0x55ec87f72f70 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe92c0163d0 tx=0x7fe92c004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b79ffe26f35035d1 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:00.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.295+0000 7fe912ffd700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:23:00.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.296+0000 7fe912ffd700 1 -- 172.21.15.101:0/3569784327 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92c00e040 con 0x55ec87f75b50 2023-07-18T03:23:00.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.296+0000 7fe94d3bfb80 1 -- 172.21.15.101:0/3569784327 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ec87e16570 con 0x55ec87f75b50 2023-07-18T03:23:00.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.296+0000 7fe912ffd700 1 -- 172.21.15.101:0/3569784327 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fe92c022440 con 0x55ec87f75b50 2023-07-18T03:23:00.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.296+0000 7fe912ffd700 1 -- 172.21.15.101:0/3569784327 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe92c02b810 con 0x55ec87f75b50 2023-07-18T03:23:00.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.296+0000 7fe94d3bfb80 10 client.28633 fetch_fsmap learned FSMap version 39 2023-07-18T03:23:00.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.296+0000 7fe94d3bfb80 1 -- 172.21.15.101:0/3569784327 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55ec87cd3ce0 con 0x55ec87f75b50 2023-07-18T03:23:00.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.296+0000 7fe912ffd700 1 -- 172.21.15.101:0/3569784327 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ec87e16570 con 0x55ec87f75b50 2023-07-18T03:23:00.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.296+0000 7fe912ffd700 1 -- 172.21.15.101:0/3569784327 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 39) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fe92c0227a0 con 0x55ec87f75b50 2023-07-18T03:23:00.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.297+0000 7fe94d3bfb80 10 client.28633 fetch_fsmap finished waiting for FSMap version 39 2023-07-18T03:23:00.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.297+0000 7fe94d3bfb80 10 client.28633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:23:00.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.297+0000 7fe94d3bfb80 20 client.28633 populate_metadata read hostname 'smithi101' 2023-07-18T03:23:00.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.297+0000 7fe94d3bfb80 1 --2- 172.21.15.101:0/3569784327 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ec87f85520 0x55ec87f87930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:00.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.297+0000 7fe94d3bfb80 4 client.28633 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:23:00.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.297+0000 7fe94d3bfb80 1 -- 172.21.15.101:0/3569784327 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ec87eb5f30 con 0x55ec87f85520 2023-07-18T03:23:00.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.297+0000 7fe932ffd700 1 --2- 172.21.15.101:0/3569784327 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ec87f85520 0x55ec87f87930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:00.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.298+0000 7fe912ffd700 10 client.28633 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:23:00.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.298+0000 7fe932ffd700 1 --2- 172.21.15.101:0/3569784327 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ec87f85520 0x55ec87f87930 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64ff04d331310dd5 server_cookie=b0b296287678e3a5 in_seq=0 out_seq=0 2023-07-18T03:23:00.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.305+0000 7fe912ffd700 1 -- 172.21.15.101:0/3569784327 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ec87eb5f30 con 0x55ec87f85520 2023-07-18T03:23:00.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.305+0000 7fe912ffd700 10 client.28633 handle_command_reply: tid=0 2023-07-18T03:23:00.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.305+0000 7fe94d3bfb80 1 client.28633 shutdown 2023-07-18T03:23:00.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.306+0000 7fe94d3bfb80 1 -- 172.21.15.101:0/3569784327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec87f75b50 msgr2=0x55ec87f72f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:00.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.306+0000 7fe94d3bfb80 1 --2- 172.21.15.101:0/3569784327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec87f75b50 0x55ec87f72f70 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe92c0163d0 tx=0x7fe92c004ad0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.306+0000 7fe94d3bfb80 1 -- 172.21.15.101:0/3569784327 shutdown_connections 2023-07-18T03:23:00.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.306+0000 7fe94d3bfb80 1 --2- 172.21.15.101:0/3569784327 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ec87f85520 0x55ec87f87930 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.306+0000 7fe94d3bfb80 1 --2- 172.21.15.101:0/3569784327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec87f75b50 0x55ec87f72f70 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.306+0000 7fe94d3bfb80 1 --2- 172.21.15.101:0/3569784327 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ec87f6a350 0x55ec87f755b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.306+0000 7fe94d3bfb80 1 --2- 172.21.15.101:0/3569784327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec87f6ac60 0x55ec87f72830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.306+0000 7fe912ffd700 0 client.28633 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:23:00.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.306+0000 7fe94d3bfb80 1 -- 172.21.15.101:0/3569784327 >> 172.21.15.101:0/3569784327 conn(0x55ec87f60e20 msgr2=0x55ec87f662b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:00.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.307+0000 7fe94d3bfb80 1 -- 172.21.15.101:0/3569784327 shutdown_connections 2023-07-18T03:23:00.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.307+0000 7fe94d3bfb80 1 -- 172.21.15.101:0/3569784327 wait complete. 2023-07-18T03:23:00.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.307+0000 7fe94d3bfb80 20 client.28633 trim_cache size 0 max 16384 2023-07-18T03:23:00.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.334+0000 7fe94d3bfb80 1 Processor -- start 2023-07-18T03:23:00.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.334+0000 7fe94d3bfb80 1 -- start start 2023-07-18T03:23:00.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.335+0000 7fe94d3bfb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec87f6a350 0x55ec87f750e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:00.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.335+0000 7fe94d3bfb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ec87f6ac60 0x55ec87f6faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:00.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.335+0000 7fe94d3bfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec87f75b50 0x55ec87f6ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:00.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.335+0000 7fe94d3bfb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec87f6c220 con 0x55ec87f6ac60 2023-07-18T03:23:00.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.335+0000 7fe94d3bfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec87f67cf0 con 0x55ec87f75b50 2023-07-18T03:23:00.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.335+0000 7fe94d3bfb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec87f720d0 con 0x55ec87f6a350 2023-07-18T03:23:00.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.335+0000 7fe9337fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec87f75b50 0x55ec87f6ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:00.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.335+0000 7fe9337fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec87f75b50 0x55ec87f6ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:53990/0 (socket says 172.21.15.101:53990) 2023-07-18T03:23:00.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.335+0000 7fe9337fe700 1 -- 172.21.15.101:0/3124047566 learned_addr learned my addr 172.21.15.101:0/3124047566 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:23:00.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.337+0000 7fe9327fc700 1 --2- 172.21.15.101:0/3124047566 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ec87f6ac60 0x55ec87f6faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:00.334 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.337+0000 7fe932ffd700 1 --2- 172.21.15.101:0/3124047566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec87f6a350 0x55ec87f750e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:00.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.337+0000 7fe9327fc700 1 -- 172.21.15.101:0/3124047566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec87f6a350 msgr2=0x55ec87f750e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:00.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.337+0000 7fe9327fc700 1 --2- 172.21.15.101:0/3124047566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec87f6a350 0x55ec87f750e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.337+0000 7fe9327fc700 1 -- 172.21.15.101:0/3124047566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec87f75b50 msgr2=0x55ec87f6ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:00.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.337+0000 7fe9327fc700 1 --2- 172.21.15.101:0/3124047566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec87f75b50 0x55ec87f6ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.337+0000 7fe9327fc700 1 -- 172.21.15.101:0/3124047566 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec87f723a0 con 0x55ec87f6ac60 2023-07-18T03:23:00.335 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.338+0000 7fe9327fc700 1 --2- 172.21.15.101:0/3124047566 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ec87f6ac60 0x55ec87f6faa0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fe9280cb770 tx=0x7fe9280d1ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5ae28dfa345fd3f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:00.342 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.346+0000 7fe91a7fc700 1 -- 172.21.15.101:0/3124047566 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9280d19d0 con 0x55ec87f6ac60 2023-07-18T03:23:00.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.346+0000 7fe91a7fc700 1 -- 172.21.15.101:0/3124047566 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9280974c0 con 0x55ec87f6ac60 2023-07-18T03:23:00.343 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.346+0000 7fe91a7fc700 1 -- 172.21.15.101:0/3124047566 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92809b180 con 0x55ec87f6ac60 2023-07-18T03:23:00.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.347+0000 7fe94d3bfb80 1 -- 172.21.15.101:0/3124047566 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ec87f6ac60 msgr2=0x55ec87f6faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:00.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.347+0000 7fe94d3bfb80 1 --2- 172.21.15.101:0/3124047566 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ec87f6ac60 0x55ec87f6faa0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fe9280cb770 tx=0x7fe9280d1ea0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.348+0000 7fe94d3bfb80 1 -- 172.21.15.101:0/3124047566 shutdown_connections 2023-07-18T03:23:00.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.348+0000 7fe94d3bfb80 1 --2- 172.21.15.101:0/3124047566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec87f75b50 0x55ec87f6ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.348+0000 7fe94d3bfb80 1 --2- 172.21.15.101:0/3124047566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec87f6a350 0x55ec87f750e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.348+0000 7fe94d3bfb80 1 --2- 172.21.15.101:0/3124047566 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ec87f6ac60 0x55ec87f6faa0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.348+0000 7fe94d3bfb80 1 -- 172.21.15.101:0/3124047566 >> 172.21.15.101:0/3124047566 conn(0x55ec87f60e20 msgr2=0x55ec87f6db10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:00.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.350+0000 7fe94d3bfb80 1 -- 172.21.15.101:0/3124047566 shutdown_connections 2023-07-18T03:23:00.348 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.351+0000 7fe94d3bfb80 1 -- 172.21.15.101:0/3124047566 wait complete. 2023-07-18T03:23:00.348 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.351+0000 7fe94d3bfb80 1 -- start start 2023-07-18T03:23:00.348 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.352+0000 7fe94d3bfb80 1 Processor -- start 2023-07-18T03:23:00.349 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.353+0000 7fe94d3bfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec87f6a350 0x55ec87f7b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:00.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.353+0000 7fe94d3bfb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ec87f6ac60 0x55ec87f7b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:00.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.353+0000 7fe94d3bfb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec87f75b50 0x55ec87ff2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:00.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.353+0000 7fe94d3bfb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec87ff3150 con 0x55ec87f6ac60 2023-07-18T03:23:00.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.353+0000 7fe94d3bfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec87ff3290 con 0x55ec87f6a350 2023-07-18T03:23:00.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.353+0000 7fe94d3bfb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec87ff3400 con 0x55ec87f75b50 2023-07-18T03:23:00.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.353+0000 7fe94d3bfb80 10 client.0 _ll_register_callbacks cb 0x7fe92c017980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:23:00.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.353+0000 7fe9337fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec87f75b50 0x55ec87ff2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:00.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.353+0000 7fe9337fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec87f75b50 0x55ec87ff2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41316/0 (socket says 172.21.15.101:41316) 2023-07-18T03:23:00.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.353+0000 7fe9337fe700 1 -- 172.21.15.101:0/844440754 learned_addr learned my addr 172.21.15.101:0/844440754 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:23:00.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.354+0000 7fe9327fc700 1 --2- 172.21.15.101:0/844440754 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ec87f6ac60 0x55ec87f7b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:00.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.354+0000 7fe9337fe700 1 -- 172.21.15.101:0/844440754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec87f6a350 msgr2=0x55ec87f7b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:00.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.354+0000 7fe9337fe700 1 --2- 172.21.15.101:0/844440754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec87f6a350 0x55ec87f7b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.354+0000 7fe9337fe700 1 -- 172.21.15.101:0/844440754 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ec87f6ac60 msgr2=0x55ec87f7b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:00.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.354+0000 7fe9337fe700 1 --2- 172.21.15.101:0/844440754 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ec87f6ac60 0x55ec87f7b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.354+0000 7fe9337fe700 1 -- 172.21.15.101:0/844440754 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9280095f0 con 0x55ec87f75b50 2023-07-18T03:23:00.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.354+0000 7fe9327fc700 1 --2- 172.21.15.101:0/844440754 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ec87f6ac60 0x55ec87f7b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:23:00.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.355+0000 7fe9337fe700 1 --2- 172.21.15.101:0/844440754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec87f75b50 0x55ec87ff2c10 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe92c022b00 tx=0x7fe92c0093d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a016c68b6e313694 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:00.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.355+0000 7fe912ffd700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:23:00.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.356+0000 7fe912ffd700 1 -- 172.21.15.101:0/844440754 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92c004e80 con 0x55ec87f75b50 2023-07-18T03:23:00.354 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.356+0000 7fe912ffd700 1 -- 172.21.15.101:0/844440754 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fe92c016b20 con 0x55ec87f75b50 2023-07-18T03:23:00.355 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.356+0000 7fe94d3bfb80 1 -- 172.21.15.101:0/844440754 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe92400faa0 con 0x55ec87f75b50 2023-07-18T03:23:00.355 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.356+0000 7fe912ffd700 1 -- 172.21.15.101:0/844440754 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe92c016ce0 con 0x55ec87f75b50 2023-07-18T03:23:00.355 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.356+0000 7fe94d3bfb80 10 client.28636 fetch_fsmap learned FSMap version 39 2023-07-18T03:23:00.355 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.356+0000 7fe94d3bfb80 1 -- 172.21.15.101:0/844440754 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55ec87d112d0 con 0x55ec87f75b50 2023-07-18T03:23:00.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.357+0000 7fe912ffd700 1 -- 172.21.15.101:0/844440754 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe92400faa0 con 0x55ec87f75b50 2023-07-18T03:23:00.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.358+0000 7fe912ffd700 1 -- 172.21.15.101:0/844440754 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 39) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fe92c048800 con 0x55ec87f75b50 2023-07-18T03:23:00.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.358+0000 7fe94d3bfb80 10 client.28636 fetch_fsmap finished waiting for FSMap version 39 2023-07-18T03:23:00.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.358+0000 7fe94d3bfb80 10 client.28636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:23:00.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.358+0000 7fe94d3bfb80 20 client.28636 populate_metadata read hostname 'smithi101' 2023-07-18T03:23:00.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.358+0000 7fe94d3bfb80 1 --2- 172.21.15.101:0/844440754 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ec87ff4b40 0x55ec87ff6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:00.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.358+0000 7fe94d3bfb80 4 client.28636 mds_command: new command op to 24562 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-18T03:23:00.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.358+0000 7fe94d3bfb80 1 -- 172.21.15.101:0/844440754 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55ec87e780d0 con 0x55ec87ff4b40 2023-07-18T03:23:00.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.360+0000 7fe932ffd700 1 --2- 172.21.15.101:0/844440754 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ec87ff4b40 0x55ec87ff6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:00.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.363+0000 7fe932ffd700 1 --2- 172.21.15.101:0/844440754 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ec87ff4b40 0x55ec87ff6f50 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4593b94eb487c89 server_cookie=d2b2138b077d2fca in_seq=0 out_seq=0 2023-07-18T03:23:00.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.363+0000 7fe912ffd700 10 client.28636 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:23:00.376 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.380+0000 7fe912ffd700 1 -- 172.21.15.101:0/844440754 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ec87e780d0 con 0x55ec87ff4b40 2023-07-18T03:23:00.377 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.380+0000 7fe912ffd700 10 client.28636 handle_command_reply: tid=0 2023-07-18T03:23:00.377 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.380+0000 7fe94d3bfb80 1 client.28636 shutdown 2023-07-18T03:23:00.377 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.381+0000 7fe94d3bfb80 1 -- 172.21.15.101:0/844440754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec87f75b50 msgr2=0x55ec87ff2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:00.377 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.381+0000 7fe94d3bfb80 1 --2- 172.21.15.101:0/844440754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec87f75b50 0x55ec87ff2c10 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe92c022b00 tx=0x7fe92c0093d0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.378 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.381+0000 7fe94d3bfb80 1 -- 172.21.15.101:0/844440754 shutdown_connections 2023-07-18T03:23:00.378 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.381+0000 7fe94d3bfb80 1 --2- 172.21.15.101:0/844440754 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ec87ff4b40 0x55ec87ff6f50 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.379 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.381+0000 7fe94d3bfb80 1 --2- 172.21.15.101:0/844440754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec87f6a350 0x55ec87f7b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.379 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.381+0000 7fe94d3bfb80 1 --2- 172.21.15.101:0/844440754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec87f75b50 0x55ec87ff2c10 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.379 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.381+0000 7fe94d3bfb80 1 --2- 172.21.15.101:0/844440754 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ec87f6ac60 0x55ec87f7b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.379 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.381+0000 7fe94d3bfb80 1 -- 172.21.15.101:0/844440754 >> 172.21.15.101:0/844440754 conn(0x55ec87f60e20 msgr2=0x55ec87f70d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:00.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.381+0000 7fe912ffd700 0 client.28636 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:23:00.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.384+0000 7fe94d3bfb80 1 -- 172.21.15.101:0/844440754 shutdown_connections 2023-07-18T03:23:00.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.384+0000 7fe94d3bfb80 1 -- 172.21.15.101:0/844440754 wait complete. 2023-07-18T03:23:00.381 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.385+0000 7fe94d3bfb80 20 client.28636 trim_cache size 0 max 16384 2023-07-18T03:23:00.381 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:23:00.381 INFO:teuthology.orchestra.run.smithi101.stdout: "return_code": 0, 2023-07-18T03:23:00.382 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_tag": "d03dabb3-16bd-47ae-8bd1-6aaaee0edd5f", 2023-07-18T03:23:00.382 INFO:teuthology.orchestra.run.smithi101.stdout: "mode": "asynchronous" 2023-07-18T03:23:00.382 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:23:00.382 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.385+0000 7fe9399ff700 1 -- 172.21.15.101:0/2218417 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fe91c0a12c0 msgr2=0x7fe91c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:00.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.385+0000 7fe9399ff700 1 --2- 172.21.15.101:0/2218417 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fe91c0a12c0 0x7fe91c0a3770 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe92c004a60 tx=0x7fe92c009040 comp rx=0 tx=0).stop 2023-07-18T03:23:00.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.385+0000 7fe9399ff700 1 -- 172.21.15.101:0/2218417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe934101610 msgr2=0x7fe934198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:00.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.385+0000 7fe9399ff700 1 --2- 172.21.15.101:0/2218417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe934101610 0x7fe934198380 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe928005910 tx=0x7fe928005720 comp rx=0 tx=0).stop 2023-07-18T03:23:00.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.386+0000 7fe9399ff700 1 -- 172.21.15.101:0/2218417 shutdown_connections 2023-07-18T03:23:00.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.386+0000 7fe9399ff700 1 --2- 172.21.15.101:0/2218417 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe93410a040 0x7fe93419c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.386+0000 7fe9399ff700 1 --2- 172.21.15.101:0/2218417 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe934100d00 0x7fe934197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.386+0000 7fe9399ff700 1 --2- 172.21.15.101:0/2218417 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fe91c0a12c0 0x7fe91c0a3770 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.386+0000 7fe9399ff700 1 --2- 172.21.15.101:0/2218417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe934101610 0x7fe934198380 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.386+0000 7fe9399ff700 1 -- 172.21.15.101:0/2218417 >> 172.21.15.101:0/2218417 conn(0x7fe9340fa4b0 msgr2=0x7fe93410c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:00.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.388+0000 7fe9399ff700 1 -- 172.21.15.101:0/2218417 shutdown_connections 2023-07-18T03:23:00.384 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.388+0000 7fe9399ff700 1 -- 172.21.15.101:0/2218417 wait complete. 2023-07-18T03:23:00.415 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:23:00.690 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.694+0000 7fa3573ff700 1 -- 172.21.15.101:0/707327209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3501015b0 msgr2=0x7fa350109aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:00.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.694+0000 7fa3573ff700 1 --2- 172.21.15.101:0/707327209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3501015b0 0x7fa350109aa0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa3400098f0 tx=0x7fa340009c00 comp rx=0 tx=0).stop 2023-07-18T03:23:00.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.694+0000 7fa3573ff700 1 -- 172.21.15.101:0/707327209 shutdown_connections 2023-07-18T03:23:00.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.694+0000 7fa3573ff700 1 --2- 172.21.15.101:0/707327209 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa35010a070 0x7fa35010c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.694+0000 7fa3573ff700 1 --2- 172.21.15.101:0/707327209 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa350100ca0 0x7fa350101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.694+0000 7fa3573ff700 1 --2- 172.21.15.101:0/707327209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3501015b0 0x7fa350109aa0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.694+0000 7fa3573ff700 1 -- 172.21.15.101:0/707327209 >> 172.21.15.101:0/707327209 conn(0x7fa3500fa4b0 msgr2=0x7fa3500fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:00.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.694+0000 7fa3573ff700 1 -- 172.21.15.101:0/707327209 shutdown_connections 2023-07-18T03:23:00.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.694+0000 7fa3573ff700 1 -- 172.21.15.101:0/707327209 wait complete. 2023-07-18T03:23:00.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.695+0000 7fa3573ff700 1 Processor -- start 2023-07-18T03:23:00.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.695+0000 7fa3573ff700 1 -- start start 2023-07-18T03:23:00.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.696+0000 7fa3573ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa350100ca0 0x7fa350195cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:00.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.696+0000 7fa3573ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3501015b0 0x7fa3501961f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:00.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.696+0000 7fa3573ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa35010a070 0x7fa3501995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:00.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.696+0000 7fa3573ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa350199ba0 con 0x7fa350100ca0 2023-07-18T03:23:00.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.696+0000 7fa3573ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa350199d10 con 0x7fa3501015b0 2023-07-18T03:23:00.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.696+0000 7fa3573ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa350199ed0 con 0x7fa35010a070 2023-07-18T03:23:00.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.696+0000 7fa35519b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa350100ca0 0x7fa350195cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:00.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.696+0000 7fa35519b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa350100ca0 0x7fa350195cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:40746/0 (socket says 172.21.15.101:40746) 2023-07-18T03:23:00.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.696+0000 7fa35519b700 1 -- 172.21.15.101:0/2453603074 learned_addr learned my addr 172.21.15.101:0/2453603074 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:23:00.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.696+0000 7fa35499a700 1 --2- 172.21.15.101:0/2453603074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3501015b0 0x7fa3501961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:00.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.696+0000 7fa35519b700 1 -- 172.21.15.101:0/2453603074 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa35010a070 msgr2=0x7fa3501995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:00.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.696+0000 7fa35519b700 1 --2- 172.21.15.101:0/2453603074 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa35010a070 0x7fa3501995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.696+0000 7fa35519b700 1 -- 172.21.15.101:0/2453603074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3501015b0 msgr2=0x7fa3501961f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:00.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.696+0000 7fa35519b700 1 --2- 172.21.15.101:0/2453603074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3501015b0 0x7fa3501961f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.696+0000 7fa35519b700 1 -- 172.21.15.101:0/2453603074 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa34c0095f0 con 0x7fa350100ca0 2023-07-18T03:23:00.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.697+0000 7fa35519b700 1 --2- 172.21.15.101:0/2453603074 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa350100ca0 0x7fa350195cb0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fa34c00ec10 tx=0x7fa34c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:00.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.697+0000 7fa3467fc700 1 -- 172.21.15.101:0/2453603074 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c00ccc0 con 0x7fa350100ca0 2023-07-18T03:23:00.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.697+0000 7fa3467fc700 1 -- 172.21.15.101:0/2453603074 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa34c00ce20 con 0x7fa350100ca0 2023-07-18T03:23:00.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.697+0000 7fa3467fc700 1 -- 172.21.15.101:0/2453603074 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c010430 con 0x7fa350100ca0 2023-07-18T03:23:00.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.698+0000 7fa3573ff700 1 -- 172.21.15.101:0/2453603074 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa340009520 con 0x7fa350100ca0 2023-07-18T03:23:00.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.698+0000 7fa3573ff700 1 -- 172.21.15.101:0/2453603074 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3501a2930 con 0x7fa350100ca0 2023-07-18T03:23:00.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.704+0000 7fa3467fc700 1 -- 172.21.15.101:0/2453603074 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fa34c003680 con 0x7fa350100ca0 2023-07-18T03:23:00.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.704+0000 7fa36adbfb80 1 Processor -- start 2023-07-18T03:23:00.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.705+0000 7fa36adbfb80 1 -- start start 2023-07-18T03:23:00.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.705+0000 7fa36adbfb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f4244722d0 0x55f4244726a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:00.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.705+0000 7fa36adbfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f424472be0 0x55f4244819a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:00.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.705+0000 7fa36adbfb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f424481ee0 0x55f4244842c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:00.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.705+0000 7fa36adbfb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f424378cb0 con 0x55f424481ee0 2023-07-18T03:23:00.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.705+0000 7fa36adbfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4243cba40 con 0x55f424472be0 2023-07-18T03:23:00.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.705+0000 7fa36adbfb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4243c4ed0 con 0x55f4244722d0 2023-07-18T03:23:00.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.705+0000 7fa35519b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f424472be0 0x55f4244819a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:00.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.705+0000 7fa35599c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f4244722d0 0x55f4244726a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:00.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.705+0000 7fa35599c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f4244722d0 0x55f4244726a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41342/0 (socket says 172.21.15.101:41342) 2023-07-18T03:23:00.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.706+0000 7fa35599c700 1 -- 172.21.15.101:0/3529452831 learned_addr learned my addr 172.21.15.101:0/3529452831 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:23:00.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.706+0000 7fa35499a700 1 --2- 172.21.15.101:0/3529452831 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f424481ee0 0x55f4244842c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:00.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.706+0000 7fa35519b700 1 -- 172.21.15.101:0/3529452831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f4244722d0 msgr2=0x55f4244726a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:00.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.706+0000 7fa35519b700 1 --2- 172.21.15.101:0/3529452831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f4244722d0 0x55f4244726a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.706+0000 7fa35519b700 1 -- 172.21.15.101:0/3529452831 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f424481ee0 msgr2=0x55f4244842c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:00.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.706+0000 7fa35519b700 1 --2- 172.21.15.101:0/3529452831 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f424481ee0 0x55f4244842c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.706+0000 7fa35519b700 1 -- 172.21.15.101:0/3529452831 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4241f2de0 con 0x55f424472be0 2023-07-18T03:23:00.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.706+0000 7fa35499a700 1 --2- 172.21.15.101:0/3529452831 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f424481ee0 0x55f4244842c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:23:00.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.707+0000 7fa3467fc700 1 --2- 172.21.15.101:0/2453603074 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa33c0a12c0 0x7fa33c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:00.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.707+0000 7fa35519b700 1 --2- 172.21.15.101:0/3529452831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f424472be0 0x55f4244819a0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa34c0ca910 tx=0x7fa34c0d53e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bba2f9152294ce60 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:00.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.707+0000 7fa35599c700 1 --2- 172.21.15.101:0/2453603074 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa33c0a12c0 0x7fa33c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:00.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.707+0000 7fa3467fc700 1 -- 172.21.15.101:0/2453603074 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fa34c014070 con 0x7fa350100ca0 2023-07-18T03:23:00.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.708+0000 7fa35599c700 1 --2- 172.21.15.101:0/2453603074 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa33c0a12c0 0x7fa33c0a3770 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa348009c10 tx=0x7fa34800d040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:00.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.708+0000 7fa33b7fe700 1 -- 172.21.15.101:0/3529452831 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c0965d0 con 0x55f424472be0 2023-07-18T03:23:00.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.708+0000 7fa33b7fe700 1 -- 172.21.15.101:0/3529452831 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa34c09a060 con 0x55f424472be0 2023-07-18T03:23:00.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.708+0000 7fa33b7fe700 1 -- 172.21.15.101:0/3529452831 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c099bf0 con 0x55f424472be0 2023-07-18T03:23:00.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.709+0000 7fa36adbfb80 1 -- 172.21.15.101:0/3529452831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f424472be0 msgr2=0x55f4244819a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:00.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.709+0000 7fa36adbfb80 1 --2- 172.21.15.101:0/3529452831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f424472be0 0x55f4244819a0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa34c0ca910 tx=0x7fa34c0d53e0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.709+0000 7fa36adbfb80 1 -- 172.21.15.101:0/3529452831 shutdown_connections 2023-07-18T03:23:00.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.709+0000 7fa36adbfb80 1 --2- 172.21.15.101:0/3529452831 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f424481ee0 0x55f4244842c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.710+0000 7fa36adbfb80 1 --2- 172.21.15.101:0/3529452831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f4244722d0 0x55f4244726a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.710+0000 7fa36adbfb80 1 --2- 172.21.15.101:0/3529452831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f424472be0 0x55f4244819a0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.710+0000 7fa36adbfb80 1 -- 172.21.15.101:0/3529452831 >> 172.21.15.101:0/3529452831 conn(0x55f424468da0 msgr2=0x55f42446e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:00.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.710+0000 7fa36adbfb80 1 -- 172.21.15.101:0/3529452831 shutdown_connections 2023-07-18T03:23:00.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.710+0000 7fa36adbfb80 1 -- 172.21.15.101:0/3529452831 wait complete. 2023-07-18T03:23:00.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.711+0000 7fa36adbfb80 1 -- start start 2023-07-18T03:23:00.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.711+0000 7fa36adbfb80 1 Processor -- start 2023-07-18T03:23:00.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.711+0000 7fa36adbfb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4244722d0 0x55f424481990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:00.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.711+0000 7fa36adbfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f424472be0 0x55f42447ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:00.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.711+0000 7fa36adbfb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f424481ee0 0x55f42447f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:00.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.711+0000 7fa36adbfb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f424364ac0 con 0x55f4244722d0 2023-07-18T03:23:00.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.711+0000 7fa36adbfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4243647c0 con 0x55f424472be0 2023-07-18T03:23:00.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.711+0000 7fa36adbfb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f424248920 con 0x55f424481ee0 2023-07-18T03:23:00.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.711+0000 7fa36adbfb80 10 client.0 _ll_register_callbacks cb 0x55f424362760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:23:00.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.712+0000 7fa35599c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f424481ee0 0x55f42447f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:00.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.712+0000 7fa35599c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f424481ee0 0x55f42447f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41352/0 (socket says 172.21.15.101:41352) 2023-07-18T03:23:00.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.712+0000 7fa35599c700 1 -- 172.21.15.101:0/1769857185 learned_addr learned my addr 172.21.15.101:0/1769857185 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:23:00.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.712+0000 7fa35499a700 1 --2- 172.21.15.101:0/1769857185 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f424472be0 0x55f42447ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:00.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.712+0000 7fa35519b700 1 --2- 172.21.15.101:0/1769857185 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4244722d0 0x55f424481990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:00.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.712+0000 7fa35599c700 1 -- 172.21.15.101:0/1769857185 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f424472be0 msgr2=0x55f42447ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:00.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.712+0000 7fa35599c700 1 --2- 172.21.15.101:0/1769857185 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f424472be0 0x55f42447ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.712+0000 7fa35599c700 1 -- 172.21.15.101:0/1769857185 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4244722d0 msgr2=0x55f424481990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:00.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.712+0000 7fa35599c700 1 --2- 172.21.15.101:0/1769857185 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4244722d0 0x55f424481990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.712+0000 7fa35599c700 1 -- 172.21.15.101:0/1769857185 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3501a2930 con 0x55f424481ee0 2023-07-18T03:23:00.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.712+0000 7fa35499a700 1 --2- 172.21.15.101:0/1769857185 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f424472be0 0x55f42447ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:23:00.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.712+0000 7fa35519b700 1 --2- 172.21.15.101:0/1769857185 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4244722d0 0x55f424481990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:23:00.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.712+0000 7fa35599c700 1 --2- 172.21.15.101:0/1769857185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f424481ee0 0x55f42447f3c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa348017990 tx=0x7fa348009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a12e65295531755 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:00.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.712+0000 7fa338ff9700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:23:00.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.713+0000 7fa338ff9700 1 -- 172.21.15.101:0/1769857185 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa348015a60 con 0x55f424481ee0 2023-07-18T03:23:00.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.713+0000 7fa36adbfb80 1 -- 172.21.15.101:0/1769857185 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f42431e4f0 con 0x55f424481ee0 2023-07-18T03:23:00.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.713+0000 7fa338ff9700 1 -- 172.21.15.101:0/1769857185 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fa348015bc0 con 0x55f424481ee0 2023-07-18T03:23:00.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.713+0000 7fa338ff9700 1 -- 172.21.15.101:0/1769857185 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa348015d80 con 0x55f424481ee0 2023-07-18T03:23:00.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.713+0000 7fa36adbfb80 10 client.28645 fetch_fsmap learned FSMap version 39 2023-07-18T03:23:00.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.713+0000 7fa36adbfb80 1 -- 172.21.15.101:0/1769857185 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55f4241dbc60 con 0x55f424481ee0 2023-07-18T03:23:00.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.713+0000 7fa338ff9700 1 -- 172.21.15.101:0/1769857185 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f42431e4f0 con 0x55f424481ee0 2023-07-18T03:23:00.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.713+0000 7fa338ff9700 1 -- 172.21.15.101:0/1769857185 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 39) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fa348021470 con 0x55f424481ee0 2023-07-18T03:23:00.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.714+0000 7fa36adbfb80 10 client.28645 fetch_fsmap finished waiting for FSMap version 39 2023-07-18T03:23:00.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.714+0000 7fa36adbfb80 10 client.28645 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:23:00.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.714+0000 7fa36adbfb80 20 client.28645 populate_metadata read hostname 'smithi101' 2023-07-18T03:23:00.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.714+0000 7fa36adbfb80 1 --2- 172.21.15.101:0/1769857185 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f4244917e0 0x55f424493bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:00.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.714+0000 7fa36adbfb80 4 client.28645 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:23:00.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.714+0000 7fa36adbfb80 1 -- 172.21.15.101:0/1769857185 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f4243bdeb0 con 0x55f4244917e0 2023-07-18T03:23:00.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.714+0000 7fa35519b700 1 --2- 172.21.15.101:0/1769857185 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f4244917e0 0x55f424493bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:00.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.714+0000 7fa35519b700 1 --2- 172.21.15.101:0/1769857185 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f4244917e0 0x55f424493bf0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0acaf274f8f4ede server_cookie=d93b3ad31a3c694e in_seq=0 out_seq=0 2023-07-18T03:23:00.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.714+0000 7fa338ff9700 10 client.28645 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:23:00.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.718+0000 7fa338ff9700 1 -- 172.21.15.101:0/1769857185 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f4243bdeb0 con 0x55f4244917e0 2023-07-18T03:23:00.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.718+0000 7fa338ff9700 10 client.28645 handle_command_reply: tid=0 2023-07-18T03:23:00.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.718+0000 7fa36adbfb80 1 client.28645 shutdown 2023-07-18T03:23:00.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.719+0000 7fa36adbfb80 1 -- 172.21.15.101:0/1769857185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f424481ee0 msgr2=0x55f42447f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:00.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.719+0000 7fa36adbfb80 1 --2- 172.21.15.101:0/1769857185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f424481ee0 0x55f42447f3c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa348017990 tx=0x7fa348009920 comp rx=0 tx=0).stop 2023-07-18T03:23:00.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.720+0000 7fa36adbfb80 1 -- 172.21.15.101:0/1769857185 shutdown_connections 2023-07-18T03:23:00.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.720+0000 7fa36adbfb80 1 --2- 172.21.15.101:0/1769857185 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f4244917e0 0x55f424493bf0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.720+0000 7fa36adbfb80 1 --2- 172.21.15.101:0/1769857185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f424481ee0 0x55f42447f3c0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.720+0000 7fa36adbfb80 1 --2- 172.21.15.101:0/1769857185 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4244722d0 0x55f424481990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.720+0000 7fa36adbfb80 1 --2- 172.21.15.101:0/1769857185 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f424472be0 0x55f42447ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.720+0000 7fa338ff9700 0 client.28645 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:23:00.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.720+0000 7fa36adbfb80 1 -- 172.21.15.101:0/1769857185 >> 172.21.15.101:0/1769857185 conn(0x55f424468da0 msgr2=0x55f42446e290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:00.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.720+0000 7fa36adbfb80 1 -- 172.21.15.101:0/1769857185 shutdown_connections 2023-07-18T03:23:00.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.721+0000 7fa36adbfb80 1 -- 172.21.15.101:0/1769857185 wait complete. 2023-07-18T03:23:00.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.721+0000 7fa36adbfb80 20 client.28645 trim_cache size 0 max 16384 2023-07-18T03:23:00.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.744+0000 7fa36adbfb80 1 Processor -- start 2023-07-18T03:23:00.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.745+0000 7fa36adbfb80 1 -- start start 2023-07-18T03:23:00.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.745+0000 7fa36adbfb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f4244722d0 0x55f42447e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:00.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.745+0000 7fa36adbfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f424472be0 0x55f424481530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:00.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.745+0000 7fa36adbfb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f424481ee0 0x55f42447bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:00.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.745+0000 7fa36adbfb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f42446fad0 con 0x55f424481ee0 2023-07-18T03:23:00.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.745+0000 7fa36adbfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f424473e30 con 0x55f424472be0 2023-07-18T03:23:00.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.745+0000 7fa36adbfb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f424465fa0 con 0x55f4244722d0 2023-07-18T03:23:00.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.745+0000 7fa35499a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f424472be0 0x55f424481530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:00.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.745+0000 7fa35519b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f4244722d0 0x55f42447e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:00.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.745+0000 7fa35499a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f424472be0 0x55f424481530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:54048/0 (socket says 172.21.15.101:54048) 2023-07-18T03:23:00.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.745+0000 7fa35499a700 1 -- 172.21.15.101:0/4198472925 learned_addr learned my addr 172.21.15.101:0/4198472925 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:23:00.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.745+0000 7fa35599c700 1 --2- 172.21.15.101:0/4198472925 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f424481ee0 0x55f42447bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:00.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.745+0000 7fa35519b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f4244722d0 0x55f42447e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41358/0 (socket says 172.21.15.101:41358) 2023-07-18T03:23:00.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.746+0000 7fa35499a700 1 -- 172.21.15.101:0/4198472925 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f4244722d0 msgr2=0x55f42447e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:00.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.746+0000 7fa35499a700 1 --2- 172.21.15.101:0/4198472925 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f4244722d0 0x55f42447e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.746+0000 7fa35499a700 1 -- 172.21.15.101:0/4198472925 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f424481ee0 msgr2=0x55f42447bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:00.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.746+0000 7fa35499a700 1 --2- 172.21.15.101:0/4198472925 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f424481ee0 0x55f42447bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.746+0000 7fa35499a700 1 -- 172.21.15.101:0/4198472925 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f424219250 con 0x55f424472be0 2023-07-18T03:23:00.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.746+0000 7fa35499a700 1 --2- 172.21.15.101:0/4198472925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f424472be0 0x55f424481530 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa3400098f0 tx=0x7fa34000bef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=daa7499a4f08c6ef server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:00.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.747+0000 7fa33bfff700 1 -- 172.21.15.101:0/4198472925 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34000f6e0 con 0x55f424472be0 2023-07-18T03:23:00.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.747+0000 7fa33bfff700 1 -- 172.21.15.101:0/4198472925 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa34001d040 con 0x55f424472be0 2023-07-18T03:23:00.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.747+0000 7fa33bfff700 1 -- 172.21.15.101:0/4198472925 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa340005680 con 0x55f424472be0 2023-07-18T03:23:00.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.748+0000 7fa36adbfb80 1 -- 172.21.15.101:0/4198472925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f424472be0 msgr2=0x55f424481530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:00.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.748+0000 7fa36adbfb80 1 --2- 172.21.15.101:0/4198472925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f424472be0 0x55f424481530 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa3400098f0 tx=0x7fa34000bef0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.748+0000 7fa36adbfb80 1 -- 172.21.15.101:0/4198472925 shutdown_connections 2023-07-18T03:23:00.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.748+0000 7fa36adbfb80 1 --2- 172.21.15.101:0/4198472925 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f424481ee0 0x55f42447bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.748+0000 7fa36adbfb80 1 --2- 172.21.15.101:0/4198472925 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f4244722d0 0x55f42447e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.748+0000 7fa36adbfb80 1 --2- 172.21.15.101:0/4198472925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f424472be0 0x55f424481530 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.748+0000 7fa36adbfb80 1 -- 172.21.15.101:0/4198472925 >> 172.21.15.101:0/4198472925 conn(0x55f424468da0 msgr2=0x55f42448dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:00.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.748+0000 7fa36adbfb80 1 -- 172.21.15.101:0/4198472925 shutdown_connections 2023-07-18T03:23:00.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.748+0000 7fa36adbfb80 1 -- 172.21.15.101:0/4198472925 wait complete. 2023-07-18T03:23:00.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.749+0000 7fa36adbfb80 1 -- start start 2023-07-18T03:23:00.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.749+0000 7fa36adbfb80 1 Processor -- start 2023-07-18T03:23:00.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.750+0000 7fa36adbfb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4244722d0 0x55f4244887f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:00.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.750+0000 7fa36adbfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f424472be0 0x55f424486d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:00.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.750+0000 7fa36adbfb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f424481ee0 0x55f424487250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:00.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.750+0000 7fa36adbfb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4244877d0 con 0x55f4244722d0 2023-07-18T03:23:00.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.750+0000 7fa36adbfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f424487940 con 0x55f424472be0 2023-07-18T03:23:00.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.750+0000 7fa36adbfb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f424487b00 con 0x55f424481ee0 2023-07-18T03:23:00.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.750+0000 7fa36adbfb80 10 client.0 _ll_register_callbacks cb 0x7fa348009f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:23:00.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.750+0000 7fa35499a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f424472be0 0x55f424486d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:00.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.750+0000 7fa35499a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f424472be0 0x55f424486d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:54060/0 (socket says 172.21.15.101:54060) 2023-07-18T03:23:00.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.750+0000 7fa35499a700 1 -- 172.21.15.101:0/4198185817 learned_addr learned my addr 172.21.15.101:0/4198185817 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:23:00.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.750+0000 7fa35599c700 1 --2- 172.21.15.101:0/4198185817 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f424481ee0 0x55f424487250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:00.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.751+0000 7fa35499a700 1 -- 172.21.15.101:0/4198185817 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f424481ee0 msgr2=0x55f424487250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:00.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.751+0000 7fa35499a700 1 --2- 172.21.15.101:0/4198185817 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f424481ee0 0x55f424487250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.751+0000 7fa35499a700 1 -- 172.21.15.101:0/4198185817 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4244722d0 msgr2=0x55f4244887f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:00.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.751+0000 7fa35519b700 1 --2- 172.21.15.101:0/4198185817 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4244722d0 0x55f4244887f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:00.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.751+0000 7fa35499a700 1 --2- 172.21.15.101:0/4198185817 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4244722d0 0x55f4244887f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.751+0000 7fa35499a700 1 -- 172.21.15.101:0/4198185817 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa35010d170 con 0x55f424472be0 2023-07-18T03:23:00.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.751+0000 7fa35519b700 1 --2- 172.21.15.101:0/4198185817 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4244722d0 0x55f4244887f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:23:00.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.754+0000 7fa35499a700 1 --2- 172.21.15.101:0/4198185817 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f424472be0 0x55f424486d10 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa340009810 tx=0x7fa340009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2300a54872c44603 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:00.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.754+0000 7fa338ff9700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:23:00.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.754+0000 7fa338ff9700 1 -- 172.21.15.101:0/4198185817 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34000f940 con 0x55f424472be0 2023-07-18T03:23:00.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.754+0000 7fa36adbfb80 1 -- 172.21.15.101:0/4198185817 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa34c09a060 con 0x55f424472be0 2023-07-18T03:23:00.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.754+0000 7fa338ff9700 1 -- 172.21.15.101:0/4198185817 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fa340005690 con 0x55f424472be0 2023-07-18T03:23:00.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.754+0000 7fa338ff9700 1 -- 172.21.15.101:0/4198185817 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa34000b770 con 0x55f424472be0 2023-07-18T03:23:00.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.754+0000 7fa36adbfb80 10 client.28643 fetch_fsmap learned FSMap version 39 2023-07-18T03:23:00.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.754+0000 7fa36adbfb80 1 -- 172.21.15.101:0/4198185817 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55f4241f12b0 con 0x55f424472be0 2023-07-18T03:23:00.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.755+0000 7fa338ff9700 1 -- 172.21.15.101:0/4198185817 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa34c09a060 con 0x55f424472be0 2023-07-18T03:23:00.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.756+0000 7fa338ff9700 1 -- 172.21.15.101:0/4198185817 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 39) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fa340015020 con 0x55f424472be0 2023-07-18T03:23:00.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.756+0000 7fa36adbfb80 10 client.28643 fetch_fsmap finished waiting for FSMap version 39 2023-07-18T03:23:00.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.756+0000 7fa36adbfb80 10 client.28643 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:23:00.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.756+0000 7fa36adbfb80 20 client.28643 populate_metadata read hostname 'smithi101' 2023-07-18T03:23:00.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.756+0000 7fa36adbfb80 1 --2- 172.21.15.101:0/4198185817 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f4244e0770 0x55f4244e0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:00.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.756+0000 7fa36adbfb80 4 client.28643 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:23:00.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.756+0000 7fa36adbfb80 1 -- 172.21.15.101:0/4198185817 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f424380050 con 0x55f4244e0770 2023-07-18T03:23:00.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.757+0000 7fa35519b700 1 --2- 172.21.15.101:0/4198185817 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f4244e0770 0x55f4244e0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:00.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.757+0000 7fa35519b700 1 --2- 172.21.15.101:0/4198185817 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f4244e0770 0x55f4244e0b70 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbd2eacfb9231dfa server_cookie=7e38210bb5247b2f in_seq=0 out_seq=0 2023-07-18T03:23:00.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.757+0000 7fa338ff9700 10 client.28643 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:23:00.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.761+0000 7fa338ff9700 1 -- 172.21.15.101:0/4198185817 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55f424380050 con 0x55f4244e0770 2023-07-18T03:23:00.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.761+0000 7fa338ff9700 10 client.28643 handle_command_reply: tid=0 2023-07-18T03:23:00.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.761+0000 7fa36adbfb80 1 client.28643 shutdown 2023-07-18T03:23:00.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.761+0000 7fa36adbfb80 1 -- 172.21.15.101:0/4198185817 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f424472be0 msgr2=0x55f424486d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:00.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.761+0000 7fa36adbfb80 1 --2- 172.21.15.101:0/4198185817 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f424472be0 0x55f424486d10 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa340009810 tx=0x7fa340009c00 comp rx=0 tx=0).stop 2023-07-18T03:23:00.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.761+0000 7fa36adbfb80 1 -- 172.21.15.101:0/4198185817 shutdown_connections 2023-07-18T03:23:00.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.761+0000 7fa36adbfb80 1 --2- 172.21.15.101:0/4198185817 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f4244e0770 0x55f4244e0b70 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.761+0000 7fa36adbfb80 1 --2- 172.21.15.101:0/4198185817 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f424481ee0 0x55f424487250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.761+0000 7fa36adbfb80 1 --2- 172.21.15.101:0/4198185817 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4244722d0 0x55f4244887f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.761+0000 7fa338ff9700 0 client.28643 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:23:00.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.761+0000 7fa36adbfb80 1 --2- 172.21.15.101:0/4198185817 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f424472be0 0x55f424486d10 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.761+0000 7fa36adbfb80 1 -- 172.21.15.101:0/4198185817 >> 172.21.15.101:0/4198185817 conn(0x55f424468da0 msgr2=0x55f424474240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:00.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.761+0000 7fa36adbfb80 1 -- 172.21.15.101:0/4198185817 shutdown_connections 2023-07-18T03:23:00.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.761+0000 7fa36adbfb80 1 -- 172.21.15.101:0/4198185817 wait complete. 2023-07-18T03:23:00.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.762+0000 7fa36adbfb80 20 client.28643 trim_cache size 0 max 16384 2023-07-18T03:23:00.760 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:23:00.760 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:23:00.760 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:23:00.761 INFO:teuthology.orchestra.run.smithi101.stdout: "d03dabb3-16bd-47ae-8bd1-6aaaee0edd5f": { 2023-07-18T03:23:00.761 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:23:00.761 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "d03dabb3-16bd-47ae-8bd1-6aaaee0edd5f", 2023-07-18T03:23:00.761 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:23:00.761 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:23:00.761 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:23:00.761 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:23:00.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.763+0000 7fa3573ff700 1 -- 172.21.15.101:0/2453603074 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa33c0a12c0 msgr2=0x7fa33c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:00.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.763+0000 7fa3573ff700 1 --2- 172.21.15.101:0/2453603074 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa33c0a12c0 0x7fa33c0a3770 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa348009c10 tx=0x7fa34800d040 comp rx=0 tx=0).stop 2023-07-18T03:23:00.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.763+0000 7fa3573ff700 1 -- 172.21.15.101:0/2453603074 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa350100ca0 msgr2=0x7fa350195cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:00.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.763+0000 7fa3573ff700 1 --2- 172.21.15.101:0/2453603074 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa350100ca0 0x7fa350195cb0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fa34c00ec10 tx=0x7fa34c00ef20 comp rx=0 tx=0).stop 2023-07-18T03:23:00.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.763+0000 7fa3573ff700 1 -- 172.21.15.101:0/2453603074 shutdown_connections 2023-07-18T03:23:00.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.763+0000 7fa3573ff700 1 --2- 172.21.15.101:0/2453603074 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa35010a070 0x7fa3501995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.763+0000 7fa3573ff700 1 --2- 172.21.15.101:0/2453603074 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa350100ca0 0x7fa350195cb0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.763+0000 7fa3573ff700 1 --2- 172.21.15.101:0/2453603074 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa33c0a12c0 0x7fa33c0a3770 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.763+0000 7fa3573ff700 1 --2- 172.21.15.101:0/2453603074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3501015b0 0x7fa3501961f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:00.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.763+0000 7fa3573ff700 1 -- 172.21.15.101:0/2453603074 >> 172.21.15.101:0/2453603074 conn(0x7fa3500fa4b0 msgr2=0x7fa35010c230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:00.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.763+0000 7fa3573ff700 1 -- 172.21.15.101:0/2453603074 shutdown_connections 2023-07-18T03:23:00.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:00.763+0000 7fa3573ff700 1 -- 172.21.15.101:0/2453603074 wait complete. 2023-07-18T03:23:00.786 INFO:tasks.cephfs.filesystem:scrub status for tag:d03dabb3-16bd-47ae-8bd1-6aaaee0edd5f - {'path': '/', 'tag': 'd03dabb3-16bd-47ae-8bd1-6aaaee0edd5f', 'options': 'recursive,force'} 2023-07-18T03:23:00.979 INFO:tasks.workunit.client.0.smithi101.stdout: 10 83837 32.24 MB/sec execute 25 sec latency 148.694 ms 2023-07-18T03:23:01.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:01 smithi155 ceph-mon[128885]: pgmap v484: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 21 MiB/s wr, 808 op/s 2023-07-18T03:23:01.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:01 smithi155 ceph-mon[128885]: scrub queued for path: / 2023-07-18T03:23:01.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:01 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:23:01.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:01 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:23:01.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:01 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:23:01.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:01 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:23:01.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:01 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:23:01.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:01 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:23:01.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:01 smithi179 ceph-mon[129229]: pgmap v484: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 21 MiB/s wr, 808 op/s 2023-07-18T03:23:01.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:01 smithi179 ceph-mon[129229]: scrub queued for path: / 2023-07-18T03:23:01.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:01 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:23:01.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:01 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:23:01.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:01 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:23:01.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:01 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:23:01.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:01 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:23:01.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:01 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:23:01.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:01 smithi101 ceph-mon[126379]: pgmap v484: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 21 MiB/s wr, 808 op/s 2023-07-18T03:23:01.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:01 smithi101 ceph-mon[126379]: scrub queued for path: / 2023-07-18T03:23:01.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:01 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:23:01.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:01 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:23:01.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:01 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:23:01.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:01 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:23:01.743 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:01 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:23:01.743 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:01 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:23:01.979 INFO:tasks.workunit.client.0.smithi101.stdout: 10 84010 31.21 MB/sec execute 26 sec latency 270.315 ms 2023-07-18T03:23:02.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:02 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:23:02.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:02 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:23:02.745 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:02 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:23:02.979 INFO:tasks.workunit.client.0.smithi101.stdout: 10 84080 30.20 MB/sec execute 27 sec latency 273.394 ms 2023-07-18T03:23:03.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:03 smithi179 ceph-mon[129229]: pgmap v485: 129 pgs: 129 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 16 MiB/s wr, 749 op/s 2023-07-18T03:23:03.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:03 smithi101 ceph-mon[126379]: pgmap v485: 129 pgs: 129 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 16 MiB/s wr, 749 op/s 2023-07-18T03:23:03.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:03 smithi155 ceph-mon[128885]: pgmap v485: 129 pgs: 129 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 16 MiB/s wr, 749 op/s 2023-07-18T03:23:03.979 INFO:tasks.workunit.client.0.smithi101.stdout: 10 84100 29.21 MB/sec execute 28 sec latency 410.821 ms 2023-07-18T03:23:04.979 INFO:tasks.workunit.client.0.smithi101.stdout: 10 84460 28.93 MB/sec execute 29 sec latency 269.250 ms 2023-07-18T03:23:05.454 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:05 smithi179 ceph-mon[129229]: pgmap v486: 129 pgs: 129 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 14 MiB/s wr, 685 op/s 2023-07-18T03:23:05.455 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:05 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:23:05.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:05 smithi101 ceph-mon[126379]: pgmap v486: 129 pgs: 129 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 14 MiB/s wr, 685 op/s 2023-07-18T03:23:05.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:05 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:23:05.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:05 smithi155 ceph-mon[128885]: pgmap v486: 129 pgs: 129 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 14 MiB/s wr, 685 op/s 2023-07-18T03:23:05.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:05 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:23:05.979 INFO:tasks.workunit.client.0.smithi101.stdout: 10 84933 28.77 MB/sec execute 30 sec latency 200.389 ms 2023-07-18T03:23:06.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:06 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:23:06.454 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:06 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:23:06.454 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:06 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:23:06.492 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:06 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:23:06.493 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:06 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:23:06.493 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:06 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:23:06.552 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:06 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:23:06.553 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:06 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:23:06.553 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:06 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:23:06.979 INFO:tasks.workunit.client.0.smithi101.stdout: 10 85644 28.70 MB/sec execute 31 sec latency 92.951 ms 2023-07-18T03:23:07.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:07 smithi179 ceph-mon[129229]: pgmap v487: 129 pgs: 129 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 17 MiB/s wr, 980 op/s 2023-07-18T03:23:07.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:07 smithi179 ceph-mon[129229]: osdmap e111: 12 total, 12 up, 12 in 2023-07-18T03:23:07.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:07 smithi101 ceph-mon[126379]: pgmap v487: 129 pgs: 129 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 17 MiB/s wr, 980 op/s 2023-07-18T03:23:07.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:07 smithi101 ceph-mon[126379]: osdmap e111: 12 total, 12 up, 12 in 2023-07-18T03:23:07.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:07 smithi155 ceph-mon[128885]: pgmap v487: 129 pgs: 129 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 17 MiB/s wr, 980 op/s 2023-07-18T03:23:07.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:07 smithi155 ceph-mon[128885]: osdmap e111: 12 total, 12 up, 12 in 2023-07-18T03:23:07.979 INFO:tasks.workunit.client.0.smithi101.stdout: 10 86515 29.46 MB/sec execute 32 sec latency 49.999 ms 2023-07-18T03:23:08.980 INFO:tasks.workunit.client.0.smithi101.stdout: 10 87232 29.95 MB/sec execute 33 sec latency 78.811 ms 2023-07-18T03:23:09.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:09 smithi179 ceph-mon[129229]: pgmap v489: 129 pgs: 129 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 5.9 MiB/s wr, 729 op/s 2023-07-18T03:23:09.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:09 smithi101 ceph-mon[126379]: pgmap v489: 129 pgs: 129 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 5.9 MiB/s wr, 729 op/s 2023-07-18T03:23:09.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:09 smithi155 ceph-mon[128885]: pgmap v489: 129 pgs: 129 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 5.9 MiB/s wr, 729 op/s 2023-07-18T03:23:09.980 INFO:tasks.workunit.client.0.smithi101.stdout: 10 87881 29.80 MB/sec execute 34 sec latency 47.813 ms 2023-07-18T03:23:10.980 INFO:tasks.workunit.client.0.smithi101.stdout: 10 88643 30.15 MB/sec execute 35 sec latency 84.435 ms 2023-07-18T03:23:11.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:11 smithi179 ceph-mon[129229]: pgmap v490: 129 pgs: 129 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.4 MiB/s wr, 664 op/s 2023-07-18T03:23:11.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:11 smithi101 ceph-mon[126379]: pgmap v490: 129 pgs: 129 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.4 MiB/s wr, 664 op/s 2023-07-18T03:23:11.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:11 smithi155 ceph-mon[128885]: pgmap v490: 129 pgs: 129 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.4 MiB/s wr, 664 op/s 2023-07-18T03:23:11.980 INFO:tasks.workunit.client.0.smithi101.stdout: 10 89492 30.88 MB/sec execute 36 sec latency 77.051 ms 2023-07-18T03:23:12.980 INFO:tasks.workunit.client.0.smithi101.stdout: 10 90087 30.87 MB/sec execute 37 sec latency 58.430 ms 2023-07-18T03:23:13.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:13 smithi179 ceph-mon[129229]: pgmap v491: 129 pgs: 129 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.3 MiB/s wr, 601 op/s 2023-07-18T03:23:13.492 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:13 smithi101 ceph-mon[126379]: pgmap v491: 129 pgs: 129 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.3 MiB/s wr, 601 op/s 2023-07-18T03:23:13.551 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:13 smithi155 ceph-mon[128885]: pgmap v491: 129 pgs: 129 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.3 MiB/s wr, 601 op/s 2023-07-18T03:23:13.980 INFO:tasks.workunit.client.0.smithi101.stdout: 10 90470 30.50 MB/sec execute 38 sec latency 80.787 ms 2023-07-18T03:23:14.980 INFO:tasks.workunit.client.0.smithi101.stdout: 10 91282 30.58 MB/sec execute 39 sec latency 57.864 ms 2023-07-18T03:23:15.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:15 smithi179 ceph-mon[129229]: pgmap v492: 129 pgs: 129 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.3 MiB/s wr, 601 op/s 2023-07-18T03:23:15.454 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:15 smithi179 ceph-mon[129229]: osdmap e112: 12 total, 12 up, 12 in 2023-07-18T03:23:15.492 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:15 smithi101 ceph-mon[126379]: pgmap v492: 129 pgs: 129 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.3 MiB/s wr, 601 op/s 2023-07-18T03:23:15.492 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:15 smithi101 ceph-mon[126379]: osdmap e112: 12 total, 12 up, 12 in 2023-07-18T03:23:15.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:15 smithi155 ceph-mon[128885]: pgmap v492: 129 pgs: 129 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.3 MiB/s wr, 601 op/s 2023-07-18T03:23:15.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:15 smithi155 ceph-mon[128885]: osdmap e112: 12 total, 12 up, 12 in 2023-07-18T03:23:15.980 INFO:tasks.workunit.client.0.smithi101.stdout: 10 92130 30.97 MB/sec execute 40 sec latency 144.628 ms 2023-07-18T03:23:16.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:16 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:23:16.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:16 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:23:16.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:16 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:23:16.493 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:16 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:23:16.493 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:16 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:23:16.493 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:16 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:23:16.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:16 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:23:16.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:16 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:23:16.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:16 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:23:16.980 INFO:tasks.workunit.client.0.smithi101.stdout: 10 92867 31.49 MB/sec execute 41 sec latency 73.708 ms 2023-07-18T03:23:17.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:17 smithi179 ceph-mon[129229]: pgmap v494: 129 pgs: 129 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.6 MiB/s wr, 517 op/s 2023-07-18T03:23:17.493 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:17 smithi101 ceph-mon[126379]: pgmap v494: 129 pgs: 129 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.6 MiB/s wr, 517 op/s 2023-07-18T03:23:17.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:17 smithi155 ceph-mon[128885]: pgmap v494: 129 pgs: 129 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.6 MiB/s wr, 517 op/s 2023-07-18T03:23:17.980 INFO:tasks.workunit.client.0.smithi101.stdout: 10 93355 31.41 MB/sec execute 42 sec latency 108.283 ms 2023-07-18T03:23:18.980 INFO:tasks.workunit.client.0.smithi101.stdout: 10 93888 31.21 MB/sec execute 43 sec latency 77.407 ms 2023-07-18T03:23:19.454 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:19 smithi179 ceph-mon[129229]: pgmap v495: 129 pgs: 129 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 7.3 MiB/s wr, 500 op/s 2023-07-18T03:23:19.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:19 smithi101 ceph-mon[126379]: pgmap v495: 129 pgs: 129 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 7.3 MiB/s wr, 500 op/s 2023-07-18T03:23:19.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:19 smithi155 ceph-mon[128885]: pgmap v495: 129 pgs: 129 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 7.3 MiB/s wr, 500 op/s 2023-07-18T03:23:19.980 INFO:tasks.workunit.client.0.smithi101.stdout: 10 94567 31.21 MB/sec execute 44 sec latency 178.516 ms 2023-07-18T03:23:20.981 INFO:tasks.workunit.client.0.smithi101.stdout: 10 95240 31.31 MB/sec execute 45 sec latency 189.636 ms 2023-07-18T03:23:21.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:21 smithi179 ceph-mon[129229]: pgmap v496: 129 pgs: 129 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.5 MiB/s wr, 535 op/s 2023-07-18T03:23:21.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:21 smithi101 ceph-mon[126379]: pgmap v496: 129 pgs: 129 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.5 MiB/s wr, 535 op/s 2023-07-18T03:23:21.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:21 smithi155 ceph-mon[128885]: pgmap v496: 129 pgs: 129 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.5 MiB/s wr, 535 op/s 2023-07-18T03:23:21.982 INFO:tasks.workunit.client.0.smithi101.stdout: 10 96092 31.88 MB/sec execute 46 sec latency 63.357 ms 2023-07-18T03:23:22.982 INFO:tasks.workunit.client.0.smithi101.stdout: 10 96699 31.85 MB/sec execute 47 sec latency 61.972 ms 2023-07-18T03:23:23.452 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:23 smithi179 ceph-mon[129229]: pgmap v497: 129 pgs: 129 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.8 MiB/s wr, 501 op/s 2023-07-18T03:23:23.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:23 smithi101 ceph-mon[126379]: pgmap v497: 129 pgs: 129 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.8 MiB/s wr, 501 op/s 2023-07-18T03:23:23.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:23 smithi155 ceph-mon[128885]: pgmap v497: 129 pgs: 129 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.8 MiB/s wr, 501 op/s 2023-07-18T03:23:23.982 INFO:tasks.workunit.client.0.smithi101.stdout: 10 97248 31.78 MB/sec execute 48 sec latency 92.185 ms 2023-07-18T03:23:24.982 INFO:tasks.workunit.client.0.smithi101.stdout: 10 97948 31.85 MB/sec execute 49 sec latency 120.824 ms 2023-07-18T03:23:25.454 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:25 smithi179 ceph-mon[129229]: pgmap v498: 129 pgs: 129 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.8 MiB/s wr, 501 op/s 2023-07-18T03:23:25.492 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:25 smithi101 ceph-mon[126379]: pgmap v498: 129 pgs: 129 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.8 MiB/s wr, 501 op/s 2023-07-18T03:23:25.559 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:25 smithi155 ceph-mon[128885]: pgmap v498: 129 pgs: 129 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.8 MiB/s wr, 501 op/s 2023-07-18T03:23:25.986 INFO:tasks.workunit.client.0.smithi101.stdout: 10 97979 31.23 MB/sec execute 50 sec latency 903.811 ms 2023-07-18T03:23:26.984 INFO:tasks.workunit.client.0.smithi101.stdout: 10 98245 30.77 MB/sec execute 51 sec latency 442.639 ms 2023-07-18T03:23:27.454 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:27 smithi179 ceph-mon[129229]: pgmap v499: 129 pgs: 129 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 633 op/s 2023-07-18T03:23:27.492 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:27 smithi101 ceph-mon[126379]: pgmap v499: 129 pgs: 129 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 633 op/s 2023-07-18T03:23:27.553 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:27 smithi155 ceph-mon[128885]: pgmap v499: 129 pgs: 129 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 633 op/s 2023-07-18T03:23:27.984 INFO:tasks.workunit.client.0.smithi101.stdout: 10 98525 30.51 MB/sec execute 52 sec latency 153.722 ms 2023-07-18T03:23:28.984 INFO:tasks.workunit.client.0.smithi101.stdout: 10 99198 30.75 MB/sec execute 53 sec latency 99.438 ms 2023-07-18T03:23:29.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:29 smithi101 ceph-mon[126379]: pgmap v500: 129 pgs: 129 active+clean; 1.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.5 MiB/s wr, 441 op/s 2023-07-18T03:23:29.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:29 smithi155 ceph-mon[128885]: pgmap v500: 129 pgs: 129 active+clean; 1.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.5 MiB/s wr, 441 op/s 2023-07-18T03:23:29.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:29 smithi179 ceph-mon[129229]: pgmap v500: 129 pgs: 129 active+clean; 1.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.5 MiB/s wr, 441 op/s 2023-07-18T03:23:29.985 INFO:tasks.workunit.client.0.smithi101.stdout: 10 99992 30.99 MB/sec execute 54 sec latency 60.723 ms 2023-07-18T03:23:30.787 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:23:30.989 INFO:tasks.workunit.client.0.smithi101.stdout: 10 100359 30.84 MB/sec execute 55 sec latency 441.354 ms 2023-07-18T03:23:31.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.303+0000 7f0b5d1ff700 1 -- 172.21.15.101:0/3773307243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b5806d3a0 msgr2=0x7f0b5811b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:31.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.303+0000 7f0b5d1ff700 1 --2- 172.21.15.101:0/3773307243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b5806d3a0 0x7f0b5811b470 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c01a9a0 tx=0x7f0b4c01acb0 comp rx=0 tx=0).stop 2023-07-18T03:23:31.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.303+0000 7f0b5d1ff700 1 -- 172.21.15.101:0/3773307243 shutdown_connections 2023-07-18T03:23:31.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.303+0000 7f0b5d1ff700 1 --2- 172.21.15.101:0/3773307243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b5806d3a0 0x7f0b5811b470 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:31.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.303+0000 7f0b5d1ff700 1 --2- 172.21.15.101:0/3773307243 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0b5810e280 0x7f0b5810e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:31.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.303+0000 7f0b5d1ff700 1 --2- 172.21.15.101:0/3773307243 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b5806c9a0 0x7f0b5806ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:31.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.303+0000 7f0b5d1ff700 1 -- 172.21.15.101:0/3773307243 >> 172.21.15.101:0/3773307243 conn(0x7f0b5806bf70 msgr2=0x7f0b58070ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:31.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.304+0000 7f0b5d1ff700 1 -- 172.21.15.101:0/3773307243 shutdown_connections 2023-07-18T03:23:31.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.304+0000 7f0b5d1ff700 1 -- 172.21.15.101:0/3773307243 wait complete. 2023-07-18T03:23:31.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.305+0000 7f0b5d1ff700 1 Processor -- start 2023-07-18T03:23:31.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.305+0000 7f0b5d1ff700 1 -- start start 2023-07-18T03:23:31.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.306+0000 7f0b5d1ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b5806c9a0 0x7f0b58116df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:31.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.306+0000 7f0b5d1ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b5806d3a0 0x7f0b58117330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:31.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.306+0000 7f0b5d1ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0b5810e280 0x7f0b581a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:31.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.306+0000 7f0b5d1ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b581a24e0 con 0x7f0b5810e280 2023-07-18T03:23:31.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.306+0000 7f0b5d1ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b581a2650 con 0x7f0b5806d3a0 2023-07-18T03:23:31.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.306+0000 7f0b5d1ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b581a2810 con 0x7f0b5806c9a0 2023-07-18T03:23:31.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.307+0000 7f0b5659c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b5806d3a0 0x7f0b58117330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:31.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.307+0000 7f0b5659c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b5806d3a0 0x7f0b58117330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:54208/0 (socket says 172.21.15.101:54208) 2023-07-18T03:23:31.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.307+0000 7f0b5659c700 1 -- 172.21.15.101:0/3766230417 learned_addr learned my addr 172.21.15.101:0/3766230417 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:23:31.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.310+0000 7f0b5759e700 1 --2- 172.21.15.101:0/3766230417 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0b5810e280 0x7f0b581a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:31.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.310+0000 7f0b56d9d700 1 --2- 172.21.15.101:0/3766230417 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b5806c9a0 0x7f0b58116df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:31.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.317+0000 7f0b5659c700 1 -- 172.21.15.101:0/3766230417 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b5806c9a0 msgr2=0x7f0b58116df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:31.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.317+0000 7f0b5659c700 1 --2- 172.21.15.101:0/3766230417 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b5806c9a0 0x7f0b58116df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:31.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.317+0000 7f0b5659c700 1 -- 172.21.15.101:0/3766230417 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0b5810e280 msgr2=0x7f0b581a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:31.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.317+0000 7f0b5659c700 1 --2- 172.21.15.101:0/3766230417 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0b5810e280 0x7f0b581a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:31.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.317+0000 7f0b5659c700 1 -- 172.21.15.101:0/3766230417 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b4c01a600 con 0x7f0b5806d3a0 2023-07-18T03:23:31.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.317+0000 7f0b5759e700 1 --2- 172.21.15.101:0/3766230417 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0b5810e280 0x7f0b581a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:23:31.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.318+0000 7f0b56d9d700 1 --2- 172.21.15.101:0/3766230417 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b5806c9a0 0x7f0b58116df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:23:31.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.318+0000 7f0b5659c700 1 --2- 172.21.15.101:0/3766230417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b5806d3a0 0x7f0b58117330 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0b4800b550 tx=0x7f0b4800b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:31.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.318+0000 7f0b3ffff700 1 -- 172.21.15.101:0/3766230417 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b48011830 con 0x7f0b5806d3a0 2023-07-18T03:23:31.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.318+0000 7f0b3ffff700 1 -- 172.21.15.101:0/3766230417 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b48011e50 con 0x7f0b5806d3a0 2023-07-18T03:23:31.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.318+0000 7f0b3ffff700 1 -- 172.21.15.101:0/3766230417 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4800f2e0 con 0x7f0b5806d3a0 2023-07-18T03:23:31.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.324+0000 7f0b5d1ff700 1 -- 172.21.15.101:0/3766230417 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b581a2af0 con 0x7f0b5806d3a0 2023-07-18T03:23:31.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.324+0000 7f0b5d1ff700 1 -- 172.21.15.101:0/3766230417 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b581a91a0 con 0x7f0b5806d3a0 2023-07-18T03:23:31.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.339+0000 7f0b70bb6b80 1 Processor -- start 2023-07-18T03:23:31.341 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.341+0000 7f0b70bb6b80 1 -- start start 2023-07-18T03:23:31.349 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.351+0000 7f0b3ffff700 1 -- 172.21.15.101:0/3766230417 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f0b48011990 con 0x7f0b5806d3a0 2023-07-18T03:23:31.355 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.353+0000 7f0b70bb6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f255ff2d0 0x557f255ff6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:31.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.353+0000 7f0b70bb6b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f255ffbe0 0x557f2560a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:31.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.353+0000 7f0b70bb6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f2560aad0 0x557f2560ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:31.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.354+0000 7f0b70bb6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f25505cb0 con 0x557f255ff2d0 2023-07-18T03:23:31.356 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.354+0000 7f0b70bb6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f25558a40 con 0x557f2560aad0 2023-07-18T03:23:31.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.354+0000 7f0b70bb6b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f25551ed0 con 0x557f255ffbe0 2023-07-18T03:23:31.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.357+0000 7f0b3ffff700 1 --2- 172.21.15.101:0/3766230417 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0b400a1400 0x7f0b400a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:31.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.358+0000 7f0b3ffff700 1 -- 172.21.15.101:0/3766230417 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f0b480cca70 con 0x7f0b5806d3a0 2023-07-18T03:23:31.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.359+0000 7f0b5659c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f2560aad0 0x557f2560ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:31.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.359+0000 7f0b5659c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f2560aad0 0x557f2560ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:54216/0 (socket says 172.21.15.101:54216) 2023-07-18T03:23:31.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.359+0000 7f0b5659c700 1 -- 172.21.15.101:0/4083699956 learned_addr learned my addr 172.21.15.101:0/4083699956 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:23:31.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.360+0000 7f0b5659c700 1 -- 172.21.15.101:0/4083699956 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f255ffbe0 msgr2=0x557f2560a590 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-18T03:23:31.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.360+0000 7f0b5659c700 1 --2- 172.21.15.101:0/4083699956 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f255ffbe0 0x557f2560a590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:31.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.360+0000 7f0b5659c700 1 -- 172.21.15.101:0/4083699956 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f255ff2d0 msgr2=0x557f255ff6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:31.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.360+0000 7f0b5659c700 1 --2- 172.21.15.101:0/4083699956 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f255ff2d0 0x557f255ff6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:31.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.360+0000 7f0b5659c700 1 -- 172.21.15.101:0/4083699956 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f2537fde0 con 0x557f2560aad0 2023-07-18T03:23:31.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.361+0000 7f0b5659c700 1 --2- 172.21.15.101:0/4083699956 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f2560aad0 0x557f2560ceb0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0b48095d40 tx=0x7f0b48099380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6191f693e5606d76 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:31.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.368+0000 7f0b3d7fa700 1 -- 172.21.15.101:0/4083699956 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b48099840 con 0x557f2560aad0 2023-07-18T03:23:31.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.368+0000 7f0b3d7fa700 1 -- 172.21.15.101:0/4083699956 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b4809d870 con 0x557f2560aad0 2023-07-18T03:23:31.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.368+0000 7f0b3d7fa700 1 -- 172.21.15.101:0/4083699956 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b480b9310 con 0x557f2560aad0 2023-07-18T03:23:31.367 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.369+0000 7f0b70bb6b80 1 -- 172.21.15.101:0/4083699956 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f2560aad0 msgr2=0x557f2560ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:31.367 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.369+0000 7f0b70bb6b80 1 --2- 172.21.15.101:0/4083699956 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f2560aad0 0x557f2560ceb0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0b48095d40 tx=0x7f0b48099380 comp rx=0 tx=0).stop 2023-07-18T03:23:31.367 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.370+0000 7f0b5759e700 1 --2- 172.21.15.101:0/3766230417 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0b400a1400 0x7f0b400a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:31.367 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.370+0000 7f0b70bb6b80 1 -- 172.21.15.101:0/4083699956 shutdown_connections 2023-07-18T03:23:31.367 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.370+0000 7f0b70bb6b80 1 --2- 172.21.15.101:0/4083699956 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f2560aad0 0x557f2560ceb0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:31.368 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.370+0000 7f0b70bb6b80 1 --2- 172.21.15.101:0/4083699956 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f255ff2d0 0x557f255ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:31.368 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.370+0000 7f0b70bb6b80 1 --2- 172.21.15.101:0/4083699956 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f255ffbe0 0x557f2560a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:31.368 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.370+0000 7f0b70bb6b80 1 -- 172.21.15.101:0/4083699956 >> 172.21.15.101:0/4083699956 conn(0x557f255f5da0 msgr2=0x557f255fb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:31.368 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.370+0000 7f0b70bb6b80 1 -- 172.21.15.101:0/4083699956 shutdown_connections 2023-07-18T03:23:31.368 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.370+0000 7f0b70bb6b80 1 -- 172.21.15.101:0/4083699956 wait complete. 2023-07-18T03:23:31.369 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.371+0000 7f0b70bb6b80 1 -- start start 2023-07-18T03:23:31.369 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.372+0000 7f0b70bb6b80 1 Processor -- start 2023-07-18T03:23:31.369 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.372+0000 7f0b70bb6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f255ff2d0 0x557f256078b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:31.370 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.373+0000 7f0b70bb6b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f255ffbe0 0x557f25607df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:31.370 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.373+0000 7f0b70bb6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f2560aad0 0x557f25608330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:31.370 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.373+0000 7f0b70bb6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f254f1ac0 con 0x557f2560aad0 2023-07-18T03:23:31.370 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.373+0000 7f0b70bb6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f254f17c0 con 0x557f255ff2d0 2023-07-18T03:23:31.370 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.373+0000 7f0b70bb6b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f253d5920 con 0x557f255ffbe0 2023-07-18T03:23:31.370 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.373+0000 7f0b70bb6b80 10 client.0 _ll_register_callbacks cb 0x557f254ef760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:23:31.371 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.374+0000 7f0b5659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f255ffbe0 0x557f25607df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:31.371 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.374+0000 7f0b5659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f255ffbe0 0x557f25607df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41726/0 (socket says 172.21.15.101:41726) 2023-07-18T03:23:31.371 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.374+0000 7f0b5659c700 1 -- 172.21.15.101:0/1480828435 learned_addr learned my addr 172.21.15.101:0/1480828435 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:23:31.371 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.374+0000 7f0b5759e700 1 --2- 172.21.15.101:0/1480828435 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f2560aad0 0x557f25608330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:31.372 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.374+0000 7f0b5659c700 1 -- 172.21.15.101:0/1480828435 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f255ff2d0 msgr2=0x557f256078b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:31.372 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.374+0000 7f0b5659c700 1 --2- 172.21.15.101:0/1480828435 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f255ff2d0 0x557f256078b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:31.372 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.375+0000 7f0b5659c700 1 -- 172.21.15.101:0/1480828435 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f2560aad0 msgr2=0x557f25608330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:31.372 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.375+0000 7f0b5659c700 1 --2- 172.21.15.101:0/1480828435 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f2560aad0 0x557f25608330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:31.372 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.375+0000 7f0b5659c700 1 -- 172.21.15.101:0/1480828435 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b480b7e70 con 0x557f255ffbe0 2023-07-18T03:23:31.373 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.375+0000 7f0b5659c700 1 --2- 172.21.15.101:0/1480828435 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f255ffbe0 0x557f25607df0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0b480ccd60 tx=0x7f0b480a1d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aae8973ceca2ab22 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:31.373 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.375+0000 7f0b3aff5700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:23:31.373 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.376+0000 7f0b3aff5700 1 -- 172.21.15.101:0/1480828435 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b48099aa0 con 0x557f255ffbe0 2023-07-18T03:23:31.374 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.376+0000 7f0b3aff5700 1 -- 172.21.15.101:0/1480828435 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f0b48099c00 con 0x557f255ffbe0 2023-07-18T03:23:31.374 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.377+0000 7f0b70bb6b80 1 -- 172.21.15.101:0/1480828435 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557f254ab4f0 con 0x557f255ffbe0 2023-07-18T03:23:31.379 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.383+0000 7f0b3aff5700 1 -- 172.21.15.101:0/1480828435 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b48098d30 con 0x557f255ffbe0 2023-07-18T03:23:31.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.384+0000 7f0b70bb6b80 10 client.28657 fetch_fsmap learned FSMap version 39 2023-07-18T03:23:31.380 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.384+0000 7f0b70bb6b80 1 -- 172.21.15.101:0/1480828435 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x557f25368c60 con 0x557f255ffbe0 2023-07-18T03:23:31.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.386+0000 7f0b5759e700 1 --2- 172.21.15.101:0/3766230417 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0b400a1400 0x7f0b400a38b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c003800 tx=0x7f0b4c01a320 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:31.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.387+0000 7f0b3aff5700 1 -- 172.21.15.101:0/1480828435 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557f254ab4f0 con 0x557f255ffbe0 2023-07-18T03:23:31.383 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.387+0000 7f0b3aff5700 1 -- 172.21.15.101:0/1480828435 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 39) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f0b480b9040 con 0x557f255ffbe0 2023-07-18T03:23:31.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.396+0000 7f0b70bb6b80 10 client.28657 fetch_fsmap finished waiting for FSMap version 39 2023-07-18T03:23:31.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.396+0000 7f0b70bb6b80 10 client.28657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:23:31.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.396+0000 7f0b70bb6b80 20 client.28657 populate_metadata read hostname 'smithi101' 2023-07-18T03:23:31.394 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.396+0000 7f0b70bb6b80 1 --2- 172.21.15.101:0/1480828435 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557f2561a000 0x557f2561c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:31.394 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.396+0000 7f0b70bb6b80 4 client.28657 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:23:31.394 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.396+0000 7f0b70bb6b80 1 -- 172.21.15.101:0/1480828435 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f2554aeb0 con 0x557f2561a000 2023-07-18T03:23:31.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.399+0000 7f0b56d9d700 1 --2- 172.21.15.101:0/1480828435 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557f2561a000 0x557f2561c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:31.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.405+0000 7f0b56d9d700 1 --2- 172.21.15.101:0/1480828435 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557f2561a000 0x557f2561c3e0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96ab44b031aa929d server_cookie=a4867ac1dfb33662 in_seq=0 out_seq=0 2023-07-18T03:23:31.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.405+0000 7f0b3aff5700 10 client.28657 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:23:31.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.409+0000 7f0b3aff5700 1 -- 172.21.15.101:0/1480828435 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557f2554aeb0 con 0x557f2561a000 2023-07-18T03:23:31.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.409+0000 7f0b3aff5700 10 client.28657 handle_command_reply: tid=0 2023-07-18T03:23:31.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.413+0000 7f0b70bb6b80 1 client.28657 shutdown 2023-07-18T03:23:31.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.413+0000 7f0b70bb6b80 1 -- 172.21.15.101:0/1480828435 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f255ffbe0 msgr2=0x557f25607df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:31.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.413+0000 7f0b70bb6b80 1 --2- 172.21.15.101:0/1480828435 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f255ffbe0 0x557f25607df0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0b480ccd60 tx=0x7f0b480a1d00 comp rx=0 tx=0).stop 2023-07-18T03:23:31.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.422+0000 7f0b70bb6b80 1 -- 172.21.15.101:0/1480828435 shutdown_connections 2023-07-18T03:23:31.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.422+0000 7f0b70bb6b80 1 --2- 172.21.15.101:0/1480828435 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557f2561a000 0x557f2561c3e0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:31.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.422+0000 7f0b70bb6b80 1 --2- 172.21.15.101:0/1480828435 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f255ff2d0 0x557f256078b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:31.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.422+0000 7f0b70bb6b80 1 --2- 172.21.15.101:0/1480828435 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f2560aad0 0x557f25608330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:31.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.422+0000 7f0b70bb6b80 1 --2- 172.21.15.101:0/1480828435 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f255ffbe0 0x557f25607df0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:31.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.422+0000 7f0b70bb6b80 1 -- 172.21.15.101:0/1480828435 >> 172.21.15.101:0/1480828435 conn(0x557f255f5da0 msgr2=0x557f255fb230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:31.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.429+0000 7f0b3aff5700 0 client.28657 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:23:31.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.437+0000 7f0b70bb6b80 1 -- 172.21.15.101:0/1480828435 shutdown_connections 2023-07-18T03:23:31.442 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.443+0000 7f0b70bb6b80 1 -- 172.21.15.101:0/1480828435 wait complete. 2023-07-18T03:23:31.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.447+0000 7f0b70bb6b80 20 client.28657 trim_cache size 0 max 16384 2023-07-18T03:23:31.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.480+0000 7f0b70bb6b80 1 Processor -- start 2023-07-18T03:23:31.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.480+0000 7f0b70bb6b80 1 -- start start 2023-07-18T03:23:31.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.480+0000 7f0b70bb6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f255ff2d0 0x557f256022f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:31.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.480+0000 7f0b70bb6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f255ffbe0 0x557f256114b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:31.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.480+0000 7f0b70bb6b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f2560aad0 0x557f25607e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:31.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.480+0000 7f0b70bb6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f25600e70 con 0x557f255ffbe0 2023-07-18T03:23:31.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.480+0000 7f0b70bb6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f2560a4e0 con 0x557f255ff2d0 2023-07-18T03:23:31.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.480+0000 7f0b70bb6b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f25600540 con 0x557f2560aad0 2023-07-18T03:23:31.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.482+0000 7f0b5759e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f2560aad0 0x557f25607e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:31.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.482+0000 7f0b5759e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f2560aad0 0x557f25607e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41738/0 (socket says 172.21.15.101:41738) 2023-07-18T03:23:31.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.482+0000 7f0b5759e700 1 -- 172.21.15.101:0/354908847 learned_addr learned my addr 172.21.15.101:0/354908847 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:23:31.480 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.483+0000 7f0b5759e700 1 -- 172.21.15.101:0/354908847 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f255ff2d0 msgr2=0x557f256022f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-18T03:23:31.480 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.483+0000 7f0b5759e700 1 --2- 172.21.15.101:0/354908847 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f255ff2d0 0x557f256022f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:31.480 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.483+0000 7f0b5759e700 1 -- 172.21.15.101:0/354908847 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f255ffbe0 msgr2=0x557f256114b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:31.480 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.483+0000 7f0b5759e700 1 --2- 172.21.15.101:0/354908847 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f255ffbe0 0x557f256114b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:31.480 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.483+0000 7f0b5759e700 1 -- 172.21.15.101:0/354908847 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f253a6250 con 0x557f2560aad0 2023-07-18T03:23:31.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.484+0000 7f0b5759e700 1 --2- 172.21.15.101:0/354908847 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f2560aad0 0x557f25607e40 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c026000 tx=0x7f0b4c02a050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4165e06e0983d79 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:31.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.484+0000 7f0b3dffb700 1 -- 172.21.15.101:0/354908847 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c03a8e0 con 0x557f2560aad0 2023-07-18T03:23:31.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.484+0000 7f0b3dffb700 1 -- 172.21.15.101:0/354908847 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b4c003940 con 0x557f2560aad0 2023-07-18T03:23:31.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.484+0000 7f0b3dffb700 1 -- 172.21.15.101:0/354908847 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c037410 con 0x557f2560aad0 2023-07-18T03:23:31.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.485+0000 7f0b70bb6b80 1 -- 172.21.15.101:0/354908847 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f2560aad0 msgr2=0x557f25607e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:31.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.485+0000 7f0b70bb6b80 1 --2- 172.21.15.101:0/354908847 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f2560aad0 0x557f25607e40 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c026000 tx=0x7f0b4c02a050 comp rx=0 tx=0).stop 2023-07-18T03:23:31.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.487+0000 7f0b70bb6b80 1 -- 172.21.15.101:0/354908847 shutdown_connections 2023-07-18T03:23:31.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.487+0000 7f0b70bb6b80 1 --2- 172.21.15.101:0/354908847 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f255ff2d0 0x557f256022f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:31.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.487+0000 7f0b70bb6b80 1 --2- 172.21.15.101:0/354908847 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f2560aad0 0x557f25607e40 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:31.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.487+0000 7f0b70bb6b80 1 --2- 172.21.15.101:0/354908847 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f255ffbe0 0x557f256114b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:31.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.487+0000 7f0b70bb6b80 1 -- 172.21.15.101:0/354908847 >> 172.21.15.101:0/354908847 conn(0x557f255f5da0 msgr2=0x557f25602bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:31.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.498+0000 7f0b70bb6b80 1 -- 172.21.15.101:0/354908847 shutdown_connections 2023-07-18T03:23:31.496 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:31 smithi101 ceph-mon[126379]: pgmap v501: 129 pgs: 129 active+clean; 1.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 645 op/s 2023-07-18T03:23:31.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.498+0000 7f0b70bb6b80 1 -- 172.21.15.101:0/354908847 wait complete. 2023-07-18T03:23:31.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.499+0000 7f0b70bb6b80 1 -- start start 2023-07-18T03:23:31.497 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.499+0000 7f0b70bb6b80 1 Processor -- start 2023-07-18T03:23:31.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.500+0000 7f0b70bb6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f255ff2d0 0x557f25667640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:31.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.500+0000 7f0b70bb6b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f255ffbe0 0x557f25667b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:31.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.500+0000 7f0b70bb6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f2560aad0 0x557f25661e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:31.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.500+0000 7f0b70bb6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f256624c0 con 0x557f2560aad0 2023-07-18T03:23:31.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.500+0000 7f0b70bb6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f25662630 con 0x557f255ff2d0 2023-07-18T03:23:31.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.500+0000 7f0b70bb6b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f256627f0 con 0x557f255ffbe0 2023-07-18T03:23:31.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.500+0000 7f0b70bb6b80 10 client.0 _ll_register_callbacks cb 0x7f0b48099610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:23:31.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.504+0000 7f0b5659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f255ffbe0 0x557f25667b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:31.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.504+0000 7f0b5659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f255ffbe0 0x557f25667b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41744/0 (socket says 172.21.15.101:41744) 2023-07-18T03:23:31.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.504+0000 7f0b5659c700 1 -- 172.21.15.101:0/2477489231 learned_addr learned my addr 172.21.15.101:0/2477489231 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:23:31.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.504+0000 7f0b56d9d700 1 --2- 172.21.15.101:0/2477489231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f255ff2d0 0x557f25667640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:31.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.505+0000 7f0b5759e700 1 --2- 172.21.15.101:0/2477489231 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f2560aad0 0x557f25661e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:31.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.506+0000 7f0b56d9d700 1 -- 172.21.15.101:0/2477489231 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f255ffbe0 msgr2=0x557f25667b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:31.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.506+0000 7f0b56d9d700 1 --2- 172.21.15.101:0/2477489231 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f255ffbe0 0x557f25667b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:31.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.506+0000 7f0b56d9d700 1 -- 172.21.15.101:0/2477489231 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f2560aad0 msgr2=0x557f25661e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:31.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.506+0000 7f0b56d9d700 1 --2- 172.21.15.101:0/2477489231 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f2560aad0 0x557f25661e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:31.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.507+0000 7f0b56d9d700 1 -- 172.21.15.101:0/2477489231 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b4c02d030 con 0x557f255ff2d0 2023-07-18T03:23:31.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.507+0000 7f0b56d9d700 1 --2- 172.21.15.101:0/2477489231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f255ff2d0 0x557f25667640 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0b50000f80 tx=0x7f0b50018f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d81e205be347dcc4 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:31.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.507+0000 7f0b3aff5700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:23:31.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.508+0000 7f0b3aff5700 1 -- 172.21.15.101:0/2477489231 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b50010040 con 0x557f255ff2d0 2023-07-18T03:23:31.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.508+0000 7f0b3aff5700 1 -- 172.21.15.101:0/2477489231 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f0b50008070 con 0x557f255ff2d0 2023-07-18T03:23:31.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.509+0000 7f0b70bb6b80 1 -- 172.21.15.101:0/2477489231 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0b4809d870 con 0x557f255ff2d0 2023-07-18T03:23:31.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.513+0000 7f0b3aff5700 1 -- 172.21.15.101:0/2477489231 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b5000ada0 con 0x557f255ff2d0 2023-07-18T03:23:31.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.522+0000 7f0b70bb6b80 10 client.28655 fetch_fsmap learned FSMap version 39 2023-07-18T03:23:31.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.522+0000 7f0b70bb6b80 1 -- 172.21.15.101:0/2477489231 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x557f2537e2b0 con 0x557f255ff2d0 2023-07-18T03:23:31.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.523+0000 7f0b3aff5700 1 -- 172.21.15.101:0/2477489231 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b4809d870 con 0x557f255ff2d0 2023-07-18T03:23:31.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.523+0000 7f0b3aff5700 1 -- 172.21.15.101:0/2477489231 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 39) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f0b50021510 con 0x557f255ff2d0 2023-07-18T03:23:31.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.523+0000 7f0b70bb6b80 10 client.28655 fetch_fsmap finished waiting for FSMap version 39 2023-07-18T03:23:31.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.523+0000 7f0b70bb6b80 10 client.28655 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:23:31.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.523+0000 7f0b70bb6b80 20 client.28655 populate_metadata read hostname 'smithi101' 2023-07-18T03:23:31.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.523+0000 7f0b70bb6b80 1 --2- 172.21.15.101:0/2477489231 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557f2566f4b0 0x557f256718c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:31.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.523+0000 7f0b70bb6b80 4 client.28655 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:23:31.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.523+0000 7f0b70bb6b80 1 -- 172.21.15.101:0/2477489231 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557f2550d050 con 0x557f2566f4b0 2023-07-18T03:23:31.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.526+0000 7f0b5659c700 1 --2- 172.21.15.101:0/2477489231 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557f2566f4b0 0x557f256718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:31.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.526+0000 7f0b5659c700 1 --2- 172.21.15.101:0/2477489231 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557f2566f4b0 0x557f256718c0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3758cdfe713cbf6 server_cookie=dffa35e6c7ec4002 in_seq=0 out_seq=0 2023-07-18T03:23:31.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.527+0000 7f0b3aff5700 10 client.28655 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:23:31.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.534+0000 7f0b3aff5700 1 -- 172.21.15.101:0/2477489231 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x557f2550d050 con 0x557f2566f4b0 2023-07-18T03:23:31.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.535+0000 7f0b3aff5700 10 client.28655 handle_command_reply: tid=0 2023-07-18T03:23:31.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.535+0000 7f0b70bb6b80 1 client.28655 shutdown 2023-07-18T03:23:31.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.535+0000 7f0b70bb6b80 1 -- 172.21.15.101:0/2477489231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f255ff2d0 msgr2=0x557f25667640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:31.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.535+0000 7f0b70bb6b80 1 --2- 172.21.15.101:0/2477489231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f255ff2d0 0x557f25667640 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0b50000f80 tx=0x7f0b50018f30 comp rx=0 tx=0).stop 2023-07-18T03:23:31.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.538+0000 7f0b70bb6b80 1 -- 172.21.15.101:0/2477489231 shutdown_connections 2023-07-18T03:23:31.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.538+0000 7f0b70bb6b80 1 --2- 172.21.15.101:0/2477489231 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557f2566f4b0 0x557f256718c0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:31.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.538+0000 7f0b70bb6b80 1 --2- 172.21.15.101:0/2477489231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557f255ff2d0 0x557f25667640 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:31.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.538+0000 7f0b70bb6b80 1 --2- 172.21.15.101:0/2477489231 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f2560aad0 0x557f25661e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:31.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.538+0000 7f0b70bb6b80 1 --2- 172.21.15.101:0/2477489231 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f255ffbe0 0x557f25667b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:31.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.538+0000 7f0b70bb6b80 1 -- 172.21.15.101:0/2477489231 >> 172.21.15.101:0/2477489231 conn(0x557f255f5da0 msgr2=0x557f25607e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:31.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.538+0000 7f0b3aff5700 0 client.28655 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:23:31.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.539+0000 7f0b70bb6b80 1 -- 172.21.15.101:0/2477489231 shutdown_connections 2023-07-18T03:23:31.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.540+0000 7f0b70bb6b80 1 -- 172.21.15.101:0/2477489231 wait complete. 2023-07-18T03:23:31.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.543+0000 7f0b70bb6b80 20 client.28655 trim_cache size 0 max 16384 2023-07-18T03:23:31.540 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:23:31.540 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "no active scrubs running", 2023-07-18T03:23:31.541 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": {} 2023-07-18T03:23:31.541 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:23:31.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.550+0000 7f0b5d1ff700 1 -- 172.21.15.101:0/3766230417 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0b400a1400 msgr2=0x7f0b400a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:31.547 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.550+0000 7f0b5d1ff700 1 --2- 172.21.15.101:0/3766230417 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0b400a1400 0x7f0b400a38b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c003800 tx=0x7f0b4c01a320 comp rx=0 tx=0).stop 2023-07-18T03:23:31.547 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.550+0000 7f0b5d1ff700 1 -- 172.21.15.101:0/3766230417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b5806d3a0 msgr2=0x7f0b58117330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:31.547 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.550+0000 7f0b5d1ff700 1 --2- 172.21.15.101:0/3766230417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b5806d3a0 0x7f0b58117330 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0b4800b550 tx=0x7f0b4800b860 comp rx=0 tx=0).stop 2023-07-18T03:23:31.547 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.550+0000 7f0b5d1ff700 1 -- 172.21.15.101:0/3766230417 shutdown_connections 2023-07-18T03:23:31.547 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.550+0000 7f0b5d1ff700 1 --2- 172.21.15.101:0/3766230417 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0b5810e280 0x7f0b581a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:31.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.550+0000 7f0b5d1ff700 1 --2- 172.21.15.101:0/3766230417 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b5806c9a0 0x7f0b58116df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:31.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.550+0000 7f0b5d1ff700 1 --2- 172.21.15.101:0/3766230417 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0b400a1400 0x7f0b400a38b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:31.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.550+0000 7f0b5d1ff700 1 --2- 172.21.15.101:0/3766230417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b5806d3a0 0x7f0b58117330 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:31.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.550+0000 7f0b5d1ff700 1 -- 172.21.15.101:0/3766230417 >> 172.21.15.101:0/3766230417 conn(0x7f0b5806bf70 msgr2=0x7f0b5806f160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:31.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:31 smithi155 ceph-mon[128885]: pgmap v501: 129 pgs: 129 active+clean; 1.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 645 op/s 2023-07-18T03:23:31.550 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.554+0000 7f0b5d1ff700 1 -- 172.21.15.101:0/3766230417 shutdown_connections 2023-07-18T03:23:31.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.555+0000 7f0b5d1ff700 1 -- 172.21.15.101:0/3766230417 wait complete. 2023-07-18T03:23:31.587 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-18T03:23:31.588 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:23:31.705 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:31 smithi179 ceph-mon[129229]: pgmap v501: 129 pgs: 129 active+clean; 1.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 645 op/s 2023-07-18T03:23:31.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.978+0000 7f61ac9ff700 1 -- 172.21.15.101:0/435851796 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f61a8103e40 msgr2=0x7f61a81042e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:31.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.978+0000 7f61ac9ff700 1 --2- 172.21.15.101:0/435851796 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f61a8103e40 0x7f61a81042e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f61a001c0d0 tx=0x7f61a001c3e0 comp rx=0 tx=0).stop 2023-07-18T03:23:31.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.982+0000 7f61ac9ff700 1 -- 172.21.15.101:0/435851796 shutdown_connections 2023-07-18T03:23:31.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.982+0000 7f61ac9ff700 1 --2- 172.21.15.101:0/435851796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61a80fea90 0x7f61a80fee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:31.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.982+0000 7f61ac9ff700 1 --2- 172.21.15.101:0/435851796 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f61a8103e40 0x7f61a81042e0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:31.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.982+0000 7f61ac9ff700 1 --2- 172.21.15.101:0/435851796 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61a80ff3a0 0x7f61a8103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:31.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.982+0000 7f61ac9ff700 1 -- 172.21.15.101:0/435851796 >> 172.21.15.101:0/435851796 conn(0x7f61a80fa490 msgr2=0x7f61a80fc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:31.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.984+0000 7f61ac9ff700 1 -- 172.21.15.101:0/435851796 shutdown_connections 2023-07-18T03:23:31.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.984+0000 7f61ac9ff700 1 -- 172.21.15.101:0/435851796 wait complete. 2023-07-18T03:23:31.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.986+0000 7f61ac9ff700 1 Processor -- start 2023-07-18T03:23:31.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.987+0000 7f61ac9ff700 1 -- start start 2023-07-18T03:23:31.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.987+0000 7f61ac9ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61a80fea90 0x7f61a80707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:31.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.987+0000 7f61ac9ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61a80ff3a0 0x7f61a8070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:31.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.987+0000 7f61ac9ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f61a8103e40 0x7f61a8111ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:31.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.987+0000 7f61ac9ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a8112530 con 0x7f61a8103e40 2023-07-18T03:23:31.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.987+0000 7f61ac9ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a81126a0 con 0x7f61a80ff3a0 2023-07-18T03:23:31.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.987+0000 7f61ac9ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a8112860 con 0x7f61a80fea90 2023-07-18T03:23:31.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.988+0000 7f61a659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61a80fea90 0x7f61a80707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:31.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.988+0000 7f61a5d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61a80ff3a0 0x7f61a8070ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:31.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.988+0000 7f61a5d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61a80ff3a0 0x7f61a8070ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:54254/0 (socket says 172.21.15.101:54254) 2023-07-18T03:23:31.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.988+0000 7f61a5d9b700 1 -- 172.21.15.101:0/407615324 learned_addr learned my addr 172.21.15.101:0/407615324 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:23:31.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.989+0000 7f61a5d9b700 1 -- 172.21.15.101:0/407615324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61a80fea90 msgr2=0x7f61a80707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:31.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.989+0000 7f61a5d9b700 1 --2- 172.21.15.101:0/407615324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61a80fea90 0x7f61a80707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:31.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.989+0000 7f61a5d9b700 1 -- 172.21.15.101:0/407615324 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f61a8103e40 msgr2=0x7f61a8111ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:31.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.989+0000 7f61a5d9b700 1 --2- 172.21.15.101:0/407615324 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f61a8103e40 0x7f61a8111ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:31.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.989+0000 7f61a5d9b700 1 -- 172.21.15.101:0/407615324 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a001ad10 con 0x7f61a80ff3a0 2023-07-18T03:23:31.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.990+0000 7f61a5d9b700 1 --2- 172.21.15.101:0/407615324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61a80ff3a0 0x7f61a8070ce0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f619800b550 tx=0x7f619800b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:31.992 INFO:tasks.workunit.client.0.smithi101.stdout: 10 100482 30.40 MB/sec execute 56 sec latency 755.663 ms 2023-07-18T03:23:31.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.993+0000 7f61977fe700 1 -- 172.21.15.101:0/407615324 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6198011830 con 0x7f61a80ff3a0 2023-07-18T03:23:31.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.993+0000 7f61977fe700 1 -- 172.21.15.101:0/407615324 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6198011e50 con 0x7f61a80ff3a0 2023-07-18T03:23:31.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.993+0000 7f61ac9ff700 1 -- 172.21.15.101:0/407615324 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61a8112b40 con 0x7f61a80ff3a0 2023-07-18T03:23:31.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.993+0000 7f61977fe700 1 -- 172.21.15.101:0/407615324 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6198010e80 con 0x7f61a80ff3a0 2023-07-18T03:23:31.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.993+0000 7f61ac9ff700 1 -- 172.21.15.101:0/407615324 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61a8113050 con 0x7f61a80ff3a0 2023-07-18T03:23:31.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:31.998+0000 7f61ac9ff700 1 -- 172.21.15.101:0/407615324 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61a8067300 con 0x7f61a80ff3a0 2023-07-18T03:23:31.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.001+0000 7f61977fe700 1 -- 172.21.15.101:0/407615324 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f6198011990 con 0x7f61a80ff3a0 2023-07-18T03:23:31.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.002+0000 7f61977fe700 1 --2- 172.21.15.101:0/407615324 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f61900a1450 0x7f61900a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:32.000 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.002+0000 7f61977fe700 1 -- 172.21.15.101:0/407615324 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f61980cccb0 con 0x7f61a80ff3a0 2023-07-18T03:23:32.001 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.004+0000 7f61a659c700 1 --2- 172.21.15.101:0/407615324 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f61900a1450 0x7f61900a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:32.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.007+0000 7f61977fe700 1 -- 172.21.15.101:0/407615324 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6198095dc0 con 0x7f61a80ff3a0 2023-07-18T03:23:32.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.010+0000 7f61a659c700 1 --2- 172.21.15.101:0/407615324 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f61900a1450 0x7f61900a3900 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f619c006fd0 tx=0x7f619c009190 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:32.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.443+0000 7f61ac9ff700 1 -- 172.21.15.101:0/407615324 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61a8113330 con 0x7f61a80ff3a0 2023-07-18T03:23:32.446 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.448+0000 7f61977fe700 1 -- 172.21.15.101:0/407615324 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+10867 (secure 0 0 0) 0x7f6198015020 con 0x7f61a80ff3a0 2023-07-18T03:23:32.447 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:23:32.448 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/4192846769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":4192846769},{"type":"v1","addr":"172.21.15.101:6839","nonce":4192846769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6841/3824340175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6840","nonce":3824340175},{"type":"v1","addr":"172.21.15.101:6841","nonce":3824340175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:04:59.361185+0000","modified":"2023-07-18T03:22:06.121618+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.155:6839/2055462483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2055462483},{"type":"v1","addr":"172.21.15.155:6839","nonce":2055462483}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:23:32.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.458+0000 7f61ac9ff700 1 -- 172.21.15.101:0/407615324 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f61900a1450 msgr2=0x7f61900a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:32.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.458+0000 7f61ac9ff700 1 --2- 172.21.15.101:0/407615324 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f61900a1450 0x7f61900a3900 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f619c006fd0 tx=0x7f619c009190 comp rx=0 tx=0).stop 2023-07-18T03:23:32.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.458+0000 7f61ac9ff700 1 -- 172.21.15.101:0/407615324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61a80ff3a0 msgr2=0x7f61a8070ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:32.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.458+0000 7f61ac9ff700 1 --2- 172.21.15.101:0/407615324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61a80ff3a0 0x7f61a8070ce0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f619800b550 tx=0x7f619800b860 comp rx=0 tx=0).stop 2023-07-18T03:23:32.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.458+0000 7f61ac9ff700 1 -- 172.21.15.101:0/407615324 shutdown_connections 2023-07-18T03:23:32.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.458+0000 7f61ac9ff700 1 --2- 172.21.15.101:0/407615324 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f61a8103e40 0x7f61a8111ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:32.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.458+0000 7f61ac9ff700 1 --2- 172.21.15.101:0/407615324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61a80fea90 0x7f61a80707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:32.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.458+0000 7f61ac9ff700 1 --2- 172.21.15.101:0/407615324 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f61900a1450 0x7f61900a3900 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:32.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.458+0000 7f61ac9ff700 1 --2- 172.21.15.101:0/407615324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61a80ff3a0 0x7f61a8070ce0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:32.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.458+0000 7f61ac9ff700 1 -- 172.21.15.101:0/407615324 >> 172.21.15.101:0/407615324 conn(0x7f61a80fa490 msgr2=0x7f61a8101840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:32.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.462+0000 7f61ac9ff700 1 -- 172.21.15.101:0/407615324 shutdown_connections 2023-07-18T03:23:32.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.463+0000 7f61ac9ff700 1 -- 172.21.15.101:0/407615324 wait complete. 2023-07-18T03:23:32.470 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 39 2023-07-18T03:23:32.496 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-18T03:23:32.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.876+0000 7f37685ff700 1 -- 172.21.15.101:0/3260637230 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f376010e280 msgr2=0x7f376010e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:32.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.876+0000 7f37685ff700 1 --2- 172.21.15.101:0/3260637230 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f376010e280 0x7f376010e650 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f37500098f0 tx=0x7f3750009c00 comp rx=0 tx=0).stop 2023-07-18T03:23:32.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.877+0000 7f37685ff700 1 -- 172.21.15.101:0/3260637230 shutdown_connections 2023-07-18T03:23:32.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.877+0000 7f37685ff700 1 --2- 172.21.15.101:0/3260637230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f376006d3a0 0x7f376011b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:32.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.877+0000 7f37685ff700 1 --2- 172.21.15.101:0/3260637230 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f376010e280 0x7f376010e650 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:32.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.877+0000 7f37685ff700 1 --2- 172.21.15.101:0/3260637230 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f376006c9a0 0x7f376006ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:32.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.877+0000 7f37685ff700 1 -- 172.21.15.101:0/3260637230 >> 172.21.15.101:0/3260637230 conn(0x7f376006bf70 msgr2=0x7f3760070ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:32.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.877+0000 7f37685ff700 1 -- 172.21.15.101:0/3260637230 shutdown_connections 2023-07-18T03:23:32.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.877+0000 7f37685ff700 1 -- 172.21.15.101:0/3260637230 wait complete. 2023-07-18T03:23:32.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.879+0000 7f37685ff700 1 Processor -- start 2023-07-18T03:23:32.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.879+0000 7f37685ff700 1 -- start start 2023-07-18T03:23:32.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.879+0000 7f37685ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f376006c9a0 0x7f3760116df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:32.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.879+0000 7f37685ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f376006d3a0 0x7f3760117330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:32.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.879+0000 7f37685ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f376010e280 0x7f37601a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:32.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.879+0000 7f37685ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37601a24e0 con 0x7f376006c9a0 2023-07-18T03:23:32.877 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.879+0000 7f37685ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37601a2650 con 0x7f376010e280 2023-07-18T03:23:32.877 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.879+0000 7f37685ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37601a2810 con 0x7f376006d3a0 2023-07-18T03:23:32.877 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.880+0000 7f3765b9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f376006d3a0 0x7f3760117330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:32.877 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.881+0000 7f3765b9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f376006d3a0 0x7f3760117330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41768/0 (socket says 172.21.15.101:41768) 2023-07-18T03:23:32.877 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.881+0000 7f3765b9a700 1 -- 172.21.15.101:0/213242701 learned_addr learned my addr 172.21.15.101:0/213242701 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:23:32.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.881+0000 7f376639b700 1 --2- 172.21.15.101:0/213242701 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f376006c9a0 0x7f3760116df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:32.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.882+0000 7f3765b9a700 1 -- 172.21.15.101:0/213242701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f376010e280 msgr2=0x7f37601a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:32.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.882+0000 7f3765b9a700 1 --2- 172.21.15.101:0/213242701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f376010e280 0x7f37601a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:32.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.882+0000 7f3765b9a700 1 -- 172.21.15.101:0/213242701 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f376006c9a0 msgr2=0x7f3760116df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:32.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.882+0000 7f3765b9a700 1 --2- 172.21.15.101:0/213242701 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f376006c9a0 0x7f3760116df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:32.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.882+0000 7f3765b9a700 1 -- 172.21.15.101:0/213242701 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3750009520 con 0x7f376006d3a0 2023-07-18T03:23:32.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.883+0000 7f3765b9a700 1 --2- 172.21.15.101:0/213242701 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f376006d3a0 0x7f3760117330 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f375800ba50 tx=0x7f375800bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:32.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.883+0000 7f37577fe700 1 -- 172.21.15.101:0/213242701 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375800c9b0 con 0x7f376006d3a0 2023-07-18T03:23:32.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.884+0000 7f37577fe700 1 -- 172.21.15.101:0/213242701 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3758014440 con 0x7f376006d3a0 2023-07-18T03:23:32.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.884+0000 7f37577fe700 1 -- 172.21.15.101:0/213242701 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37580075d0 con 0x7f376006d3a0 2023-07-18T03:23:32.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.884+0000 7f37685ff700 1 -- 172.21.15.101:0/213242701 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37601a2af0 con 0x7f376006d3a0 2023-07-18T03:23:32.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.885+0000 7f37685ff700 1 -- 172.21.15.101:0/213242701 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37601a91a0 con 0x7f376006d3a0 2023-07-18T03:23:32.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.892+0000 7f377beebb80 1 Processor -- start 2023-07-18T03:23:32.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.896+0000 7f377beebb80 1 -- start start 2023-07-18T03:23:32.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.896+0000 7f377beebb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c7d165a2d0 0x55c7d165a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:32.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.897+0000 7f377beebb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7d165abe0 0x55c7d1665590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:32.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.897+0000 7f377beebb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c7d1665ad0 0x55c7d1667eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:32.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.897+0000 7f377beebb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7d1560cb0 con 0x55c7d1665ad0 2023-07-18T03:23:32.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.897+0000 7f377beebb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7d15b3a40 con 0x55c7d165abe0 2023-07-18T03:23:32.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.897+0000 7f377beebb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7d15aced0 con 0x55c7d165a2d0 2023-07-18T03:23:32.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.897+0000 7f3766b9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c7d165a2d0 0x55c7d165a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:32.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.897+0000 7f3766b9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c7d165a2d0 0x55c7d165a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41774/0 (socket says 172.21.15.101:41774) 2023-07-18T03:23:32.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.897+0000 7f3766b9c700 1 -- 172.21.15.101:0/2382875504 learned_addr learned my addr 172.21.15.101:0/2382875504 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:23:32.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.899+0000 7f37577fe700 1 -- 172.21.15.101:0/213242701 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f37580145b0 con 0x7f376006d3a0 2023-07-18T03:23:32.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.899+0000 7f376639b700 1 --2- 172.21.15.101:0/2382875504 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7d165abe0 0x55c7d1665590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:32.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.899+0000 7f376639b700 1 -- 172.21.15.101:0/2382875504 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c7d165a2d0 msgr2=0x55c7d165a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:32.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.899+0000 7f376639b700 1 --2- 172.21.15.101:0/2382875504 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c7d165a2d0 0x55c7d165a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:32.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.899+0000 7f376639b700 1 -- 172.21.15.101:0/2382875504 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c7d1665ad0 msgr2=0x55c7d1667eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:32.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.899+0000 7f376639b700 1 --2- 172.21.15.101:0/2382875504 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c7d1665ad0 0x55c7d1667eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:32.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.899+0000 7f376639b700 1 -- 172.21.15.101:0/2382875504 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7d13dade0 con 0x55c7d165abe0 2023-07-18T03:23:32.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.900+0000 7f376639b700 1 --2- 172.21.15.101:0/2382875504 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7d165abe0 0x55c7d1665590 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3750009230 tx=0x7f3750009f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f249014164dfe9 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:32.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.905+0000 7f3754bfe700 1 -- 172.21.15.101:0/2382875504 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375000f8d0 con 0x55c7d165abe0 2023-07-18T03:23:32.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.905+0000 7f3754bfe700 1 -- 172.21.15.101:0/2382875504 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f375001d040 con 0x55c7d165abe0 2023-07-18T03:23:32.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.905+0000 7f3754bfe700 1 -- 172.21.15.101:0/2382875504 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3750005630 con 0x55c7d165abe0 2023-07-18T03:23:32.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.906+0000 7f377beebb80 1 -- 172.21.15.101:0/2382875504 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7d165abe0 msgr2=0x55c7d1665590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:32.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.906+0000 7f377beebb80 1 --2- 172.21.15.101:0/2382875504 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7d165abe0 0x55c7d1665590 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3750009230 tx=0x7f3750009f50 comp rx=0 tx=0).stop 2023-07-18T03:23:32.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.906+0000 7f37577fe700 1 --2- 172.21.15.101:0/213242701 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f374c0a1380 0x7f374c0a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:32.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.906+0000 7f37577fe700 1 -- 172.21.15.101:0/213242701 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f3758006e40 con 0x7f376006d3a0 2023-07-18T03:23:32.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.909+0000 7f377beebb80 1 -- 172.21.15.101:0/2382875504 shutdown_connections 2023-07-18T03:23:32.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.909+0000 7f377beebb80 1 --2- 172.21.15.101:0/2382875504 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c7d1665ad0 0x55c7d1667eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:32.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.909+0000 7f377beebb80 1 --2- 172.21.15.101:0/2382875504 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c7d165a2d0 0x55c7d165a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:32.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.909+0000 7f377beebb80 1 --2- 172.21.15.101:0/2382875504 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7d165abe0 0x55c7d1665590 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:32.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.909+0000 7f377beebb80 1 -- 172.21.15.101:0/2382875504 >> 172.21.15.101:0/2382875504 conn(0x55c7d1650da0 msgr2=0x55c7d16566a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:32.916 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.911+0000 7f377beebb80 1 -- 172.21.15.101:0/2382875504 shutdown_connections 2023-07-18T03:23:32.916 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.917+0000 7f3766b9c700 1 --2- 172.21.15.101:0/213242701 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f374c0a1380 0x7f374c0a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:32.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.917+0000 7f377beebb80 1 -- 172.21.15.101:0/2382875504 wait complete. 2023-07-18T03:23:32.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.918+0000 7f377beebb80 1 -- start start 2023-07-18T03:23:32.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.918+0000 7f377beebb80 1 Processor -- start 2023-07-18T03:23:32.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.920+0000 7f377beebb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c7d165a2d0 0x55c7d1665580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:32.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.920+0000 7f377beebb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c7d165abe0 0x55c7d1662800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:32.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.920+0000 7f377beebb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7d1665ad0 0x55c7d1662f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:32.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.920+0000 7f377beebb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7d154cac0 con 0x55c7d165a2d0 2023-07-18T03:23:32.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.921+0000 7f377beebb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7d154c7c0 con 0x55c7d1665ad0 2023-07-18T03:23:32.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.921+0000 7f377beebb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7d1430920 con 0x55c7d165abe0 2023-07-18T03:23:32.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.921+0000 7f377beebb80 10 client.0 _ll_register_callbacks cb 0x55c7d154a760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:23:32.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.921+0000 7f3765b9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c7d165abe0 0x55c7d1662800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:32.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.921+0000 7f3765b9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c7d165abe0 0x55c7d1662800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41788/0 (socket says 172.21.15.101:41788) 2023-07-18T03:23:32.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.921+0000 7f3765b9a700 1 -- 172.21.15.101:0/1420205959 learned_addr learned my addr 172.21.15.101:0/1420205959 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:23:32.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.921+0000 7f376639b700 1 --2- 172.21.15.101:0/1420205959 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c7d165a2d0 0x55c7d1665580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:32.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.922+0000 7f3766b9c700 1 --2- 172.21.15.101:0/213242701 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f374c0a1380 0x7f374c0a3830 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f375c0046b0 tx=0x7f375c01a860 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:32.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.922+0000 7f3766b9c700 1 --2- 172.21.15.101:0/1420205959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7d1665ad0 0x55c7d1662f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:32.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.922+0000 7f3765b9a700 1 -- 172.21.15.101:0/1420205959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7d1665ad0 msgr2=0x55c7d1662f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:32.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.922+0000 7f3765b9a700 1 --2- 172.21.15.101:0/1420205959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7d1665ad0 0x55c7d1662f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:32.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.922+0000 7f3765b9a700 1 -- 172.21.15.101:0/1420205959 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c7d165a2d0 msgr2=0x55c7d1665580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:32.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.922+0000 7f3765b9a700 1 --2- 172.21.15.101:0/1420205959 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c7d165a2d0 0x55c7d1665580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:32.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.922+0000 7f3765b9a700 1 -- 172.21.15.101:0/1420205959 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f376011bfa0 con 0x55c7d165abe0 2023-07-18T03:23:32.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.923+0000 7f3765b9a700 1 --2- 172.21.15.101:0/1420205959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c7d165abe0 0x55c7d1662800 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3758095a80 tx=0x7f3758099010 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2747c2dd13771d2 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:32.922 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.924+0000 7f3749ffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:23:32.922 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.924+0000 7f3749ffb700 1 -- 172.21.15.101:0/1420205959 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375809d350 con 0x55c7d165abe0 2023-07-18T03:23:32.923 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.924+0000 7f3749ffb700 1 -- 172.21.15.101:0/1420205959 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f3758095040 con 0x55c7d165abe0 2023-07-18T03:23:32.923 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.925+0000 7f377beebb80 1 -- 172.21.15.101:0/1420205959 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c7d15064f0 con 0x55c7d165abe0 2023-07-18T03:23:32.923 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.925+0000 7f3749ffb700 1 -- 172.21.15.101:0/1420205959 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37580b9070 con 0x55c7d165abe0 2023-07-18T03:23:32.923 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.925+0000 7f377beebb80 10 client.28678 fetch_fsmap learned FSMap version 39 2023-07-18T03:23:32.923 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.925+0000 7f377beebb80 1 -- 172.21.15.101:0/1420205959 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55c7d13c3c60 con 0x55c7d165abe0 2023-07-18T03:23:32.923 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.926+0000 7f3749ffb700 1 -- 172.21.15.101:0/1420205959 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c7d15064f0 con 0x55c7d165abe0 2023-07-18T03:23:32.923 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.926+0000 7f3749ffb700 1 -- 172.21.15.101:0/1420205959 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 39) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f3758098b00 con 0x55c7d165abe0 2023-07-18T03:23:32.924 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.926+0000 7f377beebb80 10 client.28678 fetch_fsmap finished waiting for FSMap version 39 2023-07-18T03:23:32.924 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.926+0000 7f377beebb80 10 client.28678 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:23:32.924 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.926+0000 7f377beebb80 20 client.28678 populate_metadata read hostname 'smithi101' 2023-07-18T03:23:32.924 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.926+0000 7f377beebb80 1 --2- 172.21.15.101:0/1420205959 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55c7d1675400 0x55c7d1677810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:32.924 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.926+0000 7f377beebb80 4 client.28678 mds_command: new command op to 24541 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:23:32.924 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.926+0000 7f377beebb80 1 -- 172.21.15.101:0/1420205959 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c7d15a5eb0 con 0x55c7d1675400 2023-07-18T03:23:32.925 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.926+0000 7f376639b700 1 --2- 172.21.15.101:0/1420205959 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55c7d1675400 0x55c7d1677810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:32.925 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.929+0000 7f376639b700 1 --2- 172.21.15.101:0/1420205959 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55c7d1675400 0x55c7d1677810 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d73676ab947b7d51 server_cookie=2f5bddbbb305be93 in_seq=0 out_seq=0 2023-07-18T03:23:32.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.929+0000 7f3749ffb700 10 client.28678 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:23:32.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.935+0000 7f3749ffb700 1 -- 172.21.15.101:0/1420205959 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c7d15a5eb0 con 0x55c7d1675400 2023-07-18T03:23:32.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.935+0000 7f3749ffb700 10 client.28678 handle_command_reply: tid=0 2023-07-18T03:23:32.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.936+0000 7f377beebb80 1 client.28678 shutdown 2023-07-18T03:23:32.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.945+0000 7f377beebb80 1 -- 172.21.15.101:0/1420205959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c7d165abe0 msgr2=0x55c7d1662800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:32.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.945+0000 7f377beebb80 1 --2- 172.21.15.101:0/1420205959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c7d165abe0 0x55c7d1662800 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3758095a80 tx=0x7f3758099010 comp rx=0 tx=0).stop 2023-07-18T03:23:32.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.946+0000 7f377beebb80 1 -- 172.21.15.101:0/1420205959 shutdown_connections 2023-07-18T03:23:32.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.946+0000 7f377beebb80 1 --2- 172.21.15.101:0/1420205959 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55c7d1675400 0x55c7d1677810 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:32.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.946+0000 7f377beebb80 1 --2- 172.21.15.101:0/1420205959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7d1665ad0 0x55c7d1662f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:32.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.946+0000 7f377beebb80 1 --2- 172.21.15.101:0/1420205959 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c7d165a2d0 0x55c7d1665580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:32.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.946+0000 7f377beebb80 1 --2- 172.21.15.101:0/1420205959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c7d165abe0 0x55c7d1662800 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:32.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.946+0000 7f377beebb80 1 -- 172.21.15.101:0/1420205959 >> 172.21.15.101:0/1420205959 conn(0x55c7d1650da0 msgr2=0x55c7d1656230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:32.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.946+0000 7f3749ffb700 0 client.28678 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:23:32.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.946+0000 7f377beebb80 1 -- 172.21.15.101:0/1420205959 shutdown_connections 2023-07-18T03:23:32.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.947+0000 7f377beebb80 1 -- 172.21.15.101:0/1420205959 wait complete. 2023-07-18T03:23:32.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.948+0000 7f377beebb80 20 client.28678 trim_cache size 0 max 16384 2023-07-18T03:23:32.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.975+0000 7f377beebb80 1 Processor -- start 2023-07-18T03:23:32.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.975+0000 7f377beebb80 1 -- start start 2023-07-18T03:23:32.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.976+0000 7f377beebb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7d165a2d0 0x55c7d165d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:32.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.976+0000 7f377beebb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c7d165abe0 0x55c7d1677410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:32.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.976+0000 7f377beebb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c7d1665ad0 0x55c7d165db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:32.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.976+0000 7f377beebb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7d164dfa0 con 0x55c7d165abe0 2023-07-18T03:23:32.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.976+0000 7f377beebb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7d1657310 con 0x55c7d165a2d0 2023-07-18T03:23:32.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.976+0000 7f377beebb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7d165a8f0 con 0x55c7d1665ad0 2023-07-18T03:23:32.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.976+0000 7f376639b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7d165a2d0 0x55c7d165d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:32.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.976+0000 7f376639b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7d165a2d0 0x55c7d165d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:54290/0 (socket says 172.21.15.101:54290) 2023-07-18T03:23:32.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.976+0000 7f376639b700 1 -- 172.21.15.101:0/2027791277 learned_addr learned my addr 172.21.15.101:0/2027791277 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:23:32.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.976+0000 7f3765b9a700 1 --2- 172.21.15.101:0/2027791277 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c7d165abe0 0x55c7d1677410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:32.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.976+0000 7f376639b700 1 -- 172.21.15.101:0/2027791277 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c7d1665ad0 msgr2=0x55c7d165db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:32.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.976+0000 7f376639b700 1 --2- 172.21.15.101:0/2027791277 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c7d1665ad0 0x55c7d165db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:32.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.976+0000 7f376639b700 1 -- 172.21.15.101:0/2027791277 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c7d165abe0 msgr2=0x55c7d1677410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:32.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.976+0000 7f376639b700 1 --2- 172.21.15.101:0/2027791277 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c7d165abe0 0x55c7d1677410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:32.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.976+0000 7f376639b700 1 -- 172.21.15.101:0/2027791277 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7d1401250 con 0x55c7d165a2d0 2023-07-18T03:23:32.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.977+0000 7f376639b700 1 --2- 172.21.15.101:0/2027791277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7d165a2d0 0x55c7d165d1b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3750009230 tx=0x7f375000f990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e17740f133068d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:32.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.977+0000 7f37553ff700 1 -- 172.21.15.101:0/2027791277 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375002ed10 con 0x55c7d165a2d0 2023-07-18T03:23:32.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.977+0000 7f37553ff700 1 -- 172.21.15.101:0/2027791277 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3750005b00 con 0x55c7d165a2d0 2023-07-18T03:23:32.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.977+0000 7f37553ff700 1 -- 172.21.15.101:0/2027791277 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37500056c0 con 0x55c7d165a2d0 2023-07-18T03:23:32.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.978+0000 7f377beebb80 1 -- 172.21.15.101:0/2027791277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7d165a2d0 msgr2=0x55c7d165d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:32.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.978+0000 7f377beebb80 1 --2- 172.21.15.101:0/2027791277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7d165a2d0 0x55c7d165d1b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3750009230 tx=0x7f375000f990 comp rx=0 tx=0).stop 2023-07-18T03:23:32.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.978+0000 7f377beebb80 1 -- 172.21.15.101:0/2027791277 shutdown_connections 2023-07-18T03:23:32.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.978+0000 7f377beebb80 1 --2- 172.21.15.101:0/2027791277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7d165a2d0 0x55c7d165d1b0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:32.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.978+0000 7f377beebb80 1 --2- 172.21.15.101:0/2027791277 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c7d1665ad0 0x55c7d165db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:32.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.978+0000 7f377beebb80 1 --2- 172.21.15.101:0/2027791277 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c7d165abe0 0x55c7d1677410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:32.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.978+0000 7f377beebb80 1 -- 172.21.15.101:0/2027791277 >> 172.21.15.101:0/2027791277 conn(0x55c7d1650da0 msgr2=0x55c7d16651b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:32.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.979+0000 7f377beebb80 1 -- 172.21.15.101:0/2027791277 shutdown_connections 2023-07-18T03:23:32.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.979+0000 7f377beebb80 1 -- 172.21.15.101:0/2027791277 wait complete. 2023-07-18T03:23:32.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.979+0000 7f377beebb80 1 -- start start 2023-07-18T03:23:32.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.980+0000 7f377beebb80 1 Processor -- start 2023-07-18T03:23:32.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.981+0000 7f377beebb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c7d165a2d0 0x55c7d166b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:32.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.981+0000 7f377beebb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7d165abe0 0x55c7d166bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:32.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.981+0000 7f377beebb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c7d1665ad0 0x55c7d16c8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:32.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.981+0000 7f377beebb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7d166c310 con 0x55c7d165a2d0 2023-07-18T03:23:32.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.981+0000 7f377beebb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7d166c480 con 0x55c7d165abe0 2023-07-18T03:23:32.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.981+0000 7f377beebb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7d16c87c0 con 0x55c7d1665ad0 2023-07-18T03:23:32.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.981+0000 7f377beebb80 10 client.0 _ll_register_callbacks cb 0x55c7d154a760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:23:32.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.981+0000 7f3765b9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7d165abe0 0x55c7d166bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:32.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.981+0000 7f3765b9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7d165abe0 0x55c7d166bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:54306/0 (socket says 172.21.15.101:54306) 2023-07-18T03:23:32.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.981+0000 7f3765b9a700 1 -- 172.21.15.101:0/3582498674 learned_addr learned my addr 172.21.15.101:0/3582498674 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:23:32.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.981+0000 7f376639b700 1 --2- 172.21.15.101:0/3582498674 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c7d165a2d0 0x55c7d166b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:32.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.982+0000 7f3766b9c700 1 --2- 172.21.15.101:0/3582498674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c7d1665ad0 0x55c7d16c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:32.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.982+0000 7f3765b9a700 1 -- 172.21.15.101:0/3582498674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c7d1665ad0 msgr2=0x55c7d16c8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:32.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.982+0000 7f3765b9a700 1 --2- 172.21.15.101:0/3582498674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c7d1665ad0 0x55c7d16c8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:32.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.982+0000 7f3765b9a700 1 -- 172.21.15.101:0/3582498674 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c7d165a2d0 msgr2=0x55c7d166b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:32.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.982+0000 7f3765b9a700 1 --2- 172.21.15.101:0/3582498674 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c7d165a2d0 0x55c7d166b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:32.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.982+0000 7f3765b9a700 1 -- 172.21.15.101:0/3582498674 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c7d13dade0 con 0x55c7d165abe0 2023-07-18T03:23:32.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.982+0000 7f376639b700 1 --2- 172.21.15.101:0/3582498674 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c7d165a2d0 0x55c7d166b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:23:32.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.983+0000 7f3765b9a700 1 --2- 172.21.15.101:0/3582498674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7d165abe0 0x55c7d166bba0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3758099510 tx=0x7f37580a1650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5dccc1a6e484d2 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:32.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.983+0000 7f3749ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:23:32.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.983+0000 7f3749ffb700 1 -- 172.21.15.101:0/3582498674 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3758098c70 con 0x55c7d165abe0 2023-07-18T03:23:32.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.983+0000 7f3749ffb700 1 -- 172.21.15.101:0/3582498674 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f37580a19d0 con 0x55c7d165abe0 2023-07-18T03:23:32.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.983+0000 7f377beebb80 1 -- 172.21.15.101:0/3582498674 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c7d1568050 con 0x55c7d165abe0 2023-07-18T03:23:32.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.984+0000 7f3749ffb700 1 -- 172.21.15.101:0/3582498674 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37580b9070 con 0x55c7d165abe0 2023-07-18T03:23:32.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.984+0000 7f377beebb80 10 client.28670 fetch_fsmap learned FSMap version 39 2023-07-18T03:23:32.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.984+0000 7f377beebb80 1 -- 172.21.15.101:0/3582498674 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55c7d13d92b0 con 0x55c7d165abe0 2023-07-18T03:23:32.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.984+0000 7f3749ffb700 1 -- 172.21.15.101:0/3582498674 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37580c7030 con 0x55c7d165abe0 2023-07-18T03:23:32.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.984+0000 7f3749ffb700 1 -- 172.21.15.101:0/3582498674 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 39) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f37580cc330 con 0x55c7d165abe0 2023-07-18T03:23:32.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.985+0000 7f377beebb80 10 client.28670 fetch_fsmap finished waiting for FSMap version 39 2023-07-18T03:23:32.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.985+0000 7f377beebb80 10 client.28670 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:23:32.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.985+0000 7f377beebb80 20 client.28670 populate_metadata read hostname 'smithi101' 2023-07-18T03:23:32.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.985+0000 7f377beebb80 1 --2- 172.21.15.101:0/3582498674 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55c7d16c9b50 0x55c7d16cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:32.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.985+0000 7f377beebb80 4 client.28670 mds_command: new command op to 24541 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:23:32.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.985+0000 7f377beebb80 1 -- 172.21.15.101:0/3582498674 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c7d13fbf70 con 0x55c7d16c9b50 2023-07-18T03:23:32.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.985+0000 7f376639b700 1 --2- 172.21.15.101:0/3582498674 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55c7d16c9b50 0x55c7d16cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:32.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.986+0000 7f376639b700 1 --2- 172.21.15.101:0/3582498674 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55c7d16c9b50 0x55c7d16cbf60 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f18480a69e262d80 server_cookie=7361f65d7b64ce20 in_seq=0 out_seq=0 2023-07-18T03:23:32.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.986+0000 7f3749ffb700 10 client.28670 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:23:32.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.986+0000 7f3749ffb700 1 -- 172.21.15.101:0/3582498674 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c7d13fbf70 con 0x55c7d16c9b50 2023-07-18T03:23:32.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.986+0000 7f3749ffb700 10 client.28670 handle_command_reply: tid=0 2023-07-18T03:23:32.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.986+0000 7f377beebb80 1 client.28670 shutdown 2023-07-18T03:23:32.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.986+0000 7f377beebb80 1 -- 172.21.15.101:0/3582498674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7d165abe0 msgr2=0x55c7d166bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:32.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.986+0000 7f377beebb80 1 --2- 172.21.15.101:0/3582498674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7d165abe0 0x55c7d166bba0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3758099510 tx=0x7f37580a1650 comp rx=0 tx=0).stop 2023-07-18T03:23:32.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.987+0000 7f377beebb80 1 -- 172.21.15.101:0/3582498674 shutdown_connections 2023-07-18T03:23:32.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.987+0000 7f377beebb80 1 --2- 172.21.15.101:0/3582498674 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55c7d16c9b50 0x55c7d16cbf60 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:32.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.987+0000 7f377beebb80 1 --2- 172.21.15.101:0/3582498674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c7d1665ad0 0x55c7d16c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:32.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.987+0000 7f377beebb80 1 --2- 172.21.15.101:0/3582498674 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c7d165a2d0 0x55c7d166b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:32.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.987+0000 7f377beebb80 1 --2- 172.21.15.101:0/3582498674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7d165abe0 0x55c7d166bba0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:32.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.987+0000 7f377beebb80 1 -- 172.21.15.101:0/3582498674 >> 172.21.15.101:0/3582498674 conn(0x55c7d1650da0 msgr2=0x55c7d165db30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:32.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.987+0000 7f3749ffb700 0 client.28670 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:23:32.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.987+0000 7f377beebb80 1 -- 172.21.15.101:0/3582498674 shutdown_connections 2023-07-18T03:23:32.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.989+0000 7f377beebb80 1 -- 172.21.15.101:0/3582498674 wait complete. 2023-07-18T03:23:32.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.990+0000 7f377beebb80 20 client.28670 trim_cache size 0 max 16384 2023-07-18T03:23:32.988 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:23:32.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.993+0000 7f37685ff700 1 -- 172.21.15.101:0/213242701 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f374c0a1380 msgr2=0x7f374c0a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:32.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.993+0000 7f37685ff700 1 --2- 172.21.15.101:0/213242701 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f374c0a1380 0x7f374c0a3830 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f375c0046b0 tx=0x7f375c01a860 comp rx=0 tx=0).stop 2023-07-18T03:23:32.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.993+0000 7f37685ff700 1 -- 172.21.15.101:0/213242701 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f376006d3a0 msgr2=0x7f3760117330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:32.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.993+0000 7f37685ff700 1 --2- 172.21.15.101:0/213242701 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f376006d3a0 0x7f3760117330 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f375800ba50 tx=0x7f375800bd60 comp rx=0 tx=0).stop 2023-07-18T03:23:32.990 INFO:tasks.workunit.client.0.smithi101.stdout: 10 100744 30.10 MB/sec execute 57 sec latency 163.766 ms 2023-07-18T03:23:32.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.995+0000 7f37685ff700 1 -- 172.21.15.101:0/213242701 shutdown_connections 2023-07-18T03:23:32.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.995+0000 7f37685ff700 1 --2- 172.21.15.101:0/213242701 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f374c0a1380 0x7f374c0a3830 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:32.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.995+0000 7f37685ff700 1 --2- 172.21.15.101:0/213242701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f376010e280 0x7f37601a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:32.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.995+0000 7f37685ff700 1 --2- 172.21.15.101:0/213242701 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f376006c9a0 0x7f3760116df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:32.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.995+0000 7f37685ff700 1 --2- 172.21.15.101:0/213242701 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f376006d3a0 0x7f3760117330 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:32.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.995+0000 7f37685ff700 1 -- 172.21.15.101:0/213242701 >> 172.21.15.101:0/213242701 conn(0x7f376006bf70 msgr2=0x7f3760109dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:32.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.999+0000 7f37685ff700 1 -- 172.21.15.101:0/213242701 shutdown_connections 2023-07-18T03:23:32.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:32.999+0000 7f37685ff700 1 -- 172.21.15.101:0/213242701 wait complete. 2023-07-18T03:23:33.037 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-18T03:23:33.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.438+0000 7f1733bff700 1 -- 172.21.15.101:0/2352462063 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f172c10e280 msgr2=0x7f172c10e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:33.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.438+0000 7f1733bff700 1 --2- 172.21.15.101:0/2352462063 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f172c10e280 0x7f172c10e650 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f172801a9a0 tx=0x7f172801acb0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.446+0000 7f1733bff700 1 -- 172.21.15.101:0/2352462063 shutdown_connections 2023-07-18T03:23:33.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.446+0000 7f1733bff700 1 --2- 172.21.15.101:0/2352462063 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f172c06d3a0 0x7f172c11b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.446+0000 7f1733bff700 1 --2- 172.21.15.101:0/2352462063 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f172c10e280 0x7f172c10e650 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.446+0000 7f1733bff700 1 --2- 172.21.15.101:0/2352462063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f172c06c9a0 0x7f172c06ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.446+0000 7f1733bff700 1 -- 172.21.15.101:0/2352462063 >> 172.21.15.101:0/2352462063 conn(0x7f172c06bf70 msgr2=0x7f172c070ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:33.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.457+0000 7f1733bff700 1 -- 172.21.15.101:0/2352462063 shutdown_connections 2023-07-18T03:23:33.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.458+0000 7f1733bff700 1 -- 172.21.15.101:0/2352462063 wait complete. 2023-07-18T03:23:33.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.459+0000 7f1733bff700 1 Processor -- start 2023-07-18T03:23:33.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.460+0000 7f1733bff700 1 -- start start 2023-07-18T03:23:33.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.460+0000 7f1733bff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f172c06c9a0 0x7f172c116de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:33.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.460+0000 7f1733bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f172c06d3a0 0x7f172c117320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:33.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.460+0000 7f1733bff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f172c10e280 0x7f172c1a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:33.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.460+0000 7f1733bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172c1a24e0 con 0x7f172c06d3a0 2023-07-18T03:23:33.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.460+0000 7f1733bff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172c1a2650 con 0x7f172c06c9a0 2023-07-18T03:23:33.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.460+0000 7f1733bff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172c1a2810 con 0x7f172c10e280 2023-07-18T03:23:33.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.460+0000 7f173199b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f172c06c9a0 0x7f172c116de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:33.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.461+0000 7f173199b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f172c06c9a0 0x7f172c116de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:54314/0 (socket says 172.21.15.101:54314) 2023-07-18T03:23:33.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.461+0000 7f173199b700 1 -- 172.21.15.101:0/3191661172 learned_addr learned my addr 172.21.15.101:0/3191661172 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:23:33.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.461+0000 7f173219c700 1 --2- 172.21.15.101:0/3191661172 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f172c10e280 0x7f172c1a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:33.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.461+0000 7f173119a700 1 --2- 172.21.15.101:0/3191661172 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f172c06d3a0 0x7f172c117320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:33.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.461+0000 7f173199b700 1 -- 172.21.15.101:0/3191661172 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f172c10e280 msgr2=0x7f172c1a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:33.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.461+0000 7f173199b700 1 --2- 172.21.15.101:0/3191661172 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f172c10e280 0x7f172c1a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.461+0000 7f173199b700 1 -- 172.21.15.101:0/3191661172 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f172c06d3a0 msgr2=0x7f172c117320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:33.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.462+0000 7f173199b700 1 --2- 172.21.15.101:0/3191661172 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f172c06d3a0 0x7f172c117320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.462+0000 7f173199b700 1 -- 172.21.15.101:0/3191661172 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f172801a600 con 0x7f172c06c9a0 2023-07-18T03:23:33.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.462+0000 7f173119a700 1 --2- 172.21.15.101:0/3191661172 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f172c06d3a0 0x7f172c117320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:23:33.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.462+0000 7f173199b700 1 --2- 172.21.15.101:0/3191661172 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f172c06c9a0 0x7f172c116de0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1728003ba0 tx=0x7f1728003bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:33.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.463+0000 7f1722ffd700 1 -- 172.21.15.101:0/3191661172 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172802d070 con 0x7f172c06c9a0 2023-07-18T03:23:33.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.463+0000 7f1722ffd700 1 -- 172.21.15.101:0/3191661172 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1728003680 con 0x7f172c06c9a0 2023-07-18T03:23:33.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.463+0000 7f1722ffd700 1 -- 172.21.15.101:0/3191661172 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1728031be0 con 0x7f172c06c9a0 2023-07-18T03:23:33.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.463+0000 7f1733bff700 1 -- 172.21.15.101:0/3191661172 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f172c1a2a90 con 0x7f172c06c9a0 2023-07-18T03:23:33.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.463+0000 7f1733bff700 1 -- 172.21.15.101:0/3191661172 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f172c1a8fc0 con 0x7f172c06c9a0 2023-07-18T03:23:33.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.471+0000 7f1747486b80 1 Processor -- start 2023-07-18T03:23:33.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.472+0000 7f1747486b80 1 -- start start 2023-07-18T03:23:33.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.472+0000 7f1747486b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622fd1292d0 0x5622fd1296a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:33.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.472+0000 7f1747486b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622fd129be0 0x5622fd134590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:33.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.472+0000 7f1747486b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5622fd134ad0 0x5622fd136eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:33.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.472+0000 7f1747486b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622fd02fcb0 con 0x5622fd134ad0 2023-07-18T03:23:33.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.472+0000 7f1747486b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622fd082a40 con 0x5622fd1292d0 2023-07-18T03:23:33.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.472+0000 7f1747486b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622fd07bed0 con 0x5622fd129be0 2023-07-18T03:23:33.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.473+0000 7f173219c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622fd1292d0 0x5622fd1296a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:33.471 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.473+0000 7f173219c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622fd1292d0 0x5622fd1296a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:54320/0 (socket says 172.21.15.101:54320) 2023-07-18T03:23:33.471 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.473+0000 7f173219c700 1 -- 172.21.15.101:0/3943987886 learned_addr learned my addr 172.21.15.101:0/3943987886 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:23:33.471 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.473+0000 7f173119a700 1 --2- 172.21.15.101:0/3943987886 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5622fd134ad0 0x5622fd136eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:33.471 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.474+0000 7f173219c700 1 -- 172.21.15.101:0/3943987886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622fd129be0 msgr2=0x5622fd134590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-18T03:23:33.471 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.474+0000 7f173219c700 1 --2- 172.21.15.101:0/3943987886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622fd129be0 0x5622fd134590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.471 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.474+0000 7f173219c700 1 -- 172.21.15.101:0/3943987886 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5622fd134ad0 msgr2=0x5622fd136eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:33.472 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.474+0000 7f173219c700 1 --2- 172.21.15.101:0/3943987886 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5622fd134ad0 0x5622fd136eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.472 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.474+0000 7f173219c700 1 -- 172.21.15.101:0/3943987886 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622fcea9de0 con 0x5622fd1292d0 2023-07-18T03:23:33.472 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.474+0000 7f173219c700 1 --2- 172.21.15.101:0/3943987886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622fd1292d0 0x5622fd1296a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f17240098f0 tx=0x7f1724009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a6874bd15f0df8f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:33.480 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.479+0000 7f1722ffd700 1 -- 172.21.15.101:0/3191661172 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f172801f3f0 con 0x7f172c06c9a0 2023-07-18T03:23:33.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.481+0000 7f1717fff700 1 -- 172.21.15.101:0/3943987886 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1724019070 con 0x5622fd1292d0 2023-07-18T03:23:33.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.481+0000 7f1717fff700 1 -- 172.21.15.101:0/3943987886 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f172400f040 con 0x5622fd1292d0 2023-07-18T03:23:33.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.481+0000 7f1717fff700 1 -- 172.21.15.101:0/3943987886 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1724007c70 con 0x5622fd1292d0 2023-07-18T03:23:33.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.481+0000 7f1722ffd700 1 --2- 172.21.15.101:0/3191661172 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f17180a14d0 0x7f17180a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:33.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.481+0000 7f1722ffd700 1 -- 172.21.15.101:0/3191661172 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f17280de8d0 con 0x7f172c06c9a0 2023-07-18T03:23:33.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.482+0000 7f1747486b80 1 -- 172.21.15.101:0/3943987886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622fd1292d0 msgr2=0x5622fd1296a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:33.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.482+0000 7f1747486b80 1 --2- 172.21.15.101:0/3943987886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622fd1292d0 0x5622fd1296a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f17240098f0 tx=0x7f1724009c00 comp rx=0 tx=0).stop 2023-07-18T03:23:33.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.483+0000 7f173199b700 1 --2- 172.21.15.101:0/3191661172 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f17180a14d0 0x7f17180a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:33.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.483+0000 7f1747486b80 1 -- 172.21.15.101:0/3943987886 shutdown_connections 2023-07-18T03:23:33.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.483+0000 7f1747486b80 1 --2- 172.21.15.101:0/3943987886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622fd1292d0 0x5622fd1296a0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.483+0000 7f1747486b80 1 --2- 172.21.15.101:0/3943987886 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5622fd134ad0 0x5622fd136eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.483+0000 7f1747486b80 1 --2- 172.21.15.101:0/3943987886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622fd129be0 0x5622fd134590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.483+0000 7f1747486b80 1 -- 172.21.15.101:0/3943987886 >> 172.21.15.101:0/3943987886 conn(0x5622fd11fda0 msgr2=0x5622fd1256a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:33.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.491+0000 7f1747486b80 1 -- 172.21.15.101:0/3943987886 shutdown_connections 2023-07-18T03:23:33.493 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:33 smithi101 ceph-mon[126379]: pgmap v502: 129 pgs: 129 active+clean; 1.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 15 MiB/s wr, 672 op/s 2023-07-18T03:23:33.494 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:33 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/407615324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:23:33.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.497+0000 7f1747486b80 1 -- 172.21.15.101:0/3943987886 wait complete. 2023-07-18T03:23:33.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.498+0000 7f1747486b80 1 -- start start 2023-07-18T03:23:33.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.498+0000 7f1747486b80 1 Processor -- start 2023-07-18T03:23:33.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.499+0000 7f1747486b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5622fd1292d0 0x5622fd134500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:33.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.499+0000 7f1747486b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622fd129be0 0x5622fd131780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:33.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.499+0000 7f1747486b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622fd134ad0 0x5622fd131ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:33.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.499+0000 7f1747486b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622fd01bac0 con 0x5622fd1292d0 2023-07-18T03:23:33.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.499+0000 7f1747486b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622fd01b7c0 con 0x5622fd134ad0 2023-07-18T03:23:33.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.499+0000 7f1747486b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622fceff920 con 0x5622fd129be0 2023-07-18T03:23:33.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.499+0000 7f1747486b80 10 client.0 _ll_register_callbacks cb 0x5622fd019760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:23:33.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.500+0000 7f173219c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622fd134ad0 0x5622fd131ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:33.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.500+0000 7f173219c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622fd134ad0 0x5622fd131ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:54324/0 (socket says 172.21.15.101:54324) 2023-07-18T03:23:33.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.500+0000 7f173219c700 1 -- 172.21.15.101:0/2672871586 learned_addr learned my addr 172.21.15.101:0/2672871586 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:23:33.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.500+0000 7f173119a700 1 --2- 172.21.15.101:0/2672871586 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622fd129be0 0x5622fd131780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:33.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.501+0000 7f173219c700 1 -- 172.21.15.101:0/2672871586 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622fd129be0 msgr2=0x5622fd131780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:33.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.501+0000 7f173219c700 1 --2- 172.21.15.101:0/2672871586 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622fd129be0 0x5622fd131780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.501+0000 7f173219c700 1 -- 172.21.15.101:0/2672871586 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5622fd1292d0 msgr2=0x5622fd134500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:33.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.501+0000 7f173219c700 1 --2- 172.21.15.101:0/2672871586 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5622fd1292d0 0x5622fd134500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.509 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.501+0000 7f173219c700 1 -- 172.21.15.101:0/2672871586 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1724003c50 con 0x5622fd134ad0 2023-07-18T03:23:33.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.501+0000 7f173219c700 1 --2- 172.21.15.101:0/2672871586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622fd134ad0 0x5622fd131ec0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f1724009530 tx=0x7f1724007610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23c394deebb78bcc server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:33.510 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.501+0000 7f17157fa700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:23:33.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.511+0000 7f17157fa700 1 -- 172.21.15.101:0/2672871586 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1724019070 con 0x5622fd134ad0 2023-07-18T03:23:33.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.511+0000 7f17157fa700 1 -- 172.21.15.101:0/2672871586 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f1724015740 con 0x5622fd134ad0 2023-07-18T03:23:33.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.516+0000 7f1747486b80 1 -- 172.21.15.101:0/2672871586 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5622fcfd54f0 con 0x5622fd134ad0 2023-07-18T03:23:33.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.519+0000 7f173199b700 1 --2- 172.21.15.101:0/3191661172 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f17180a14d0 0x7f17180a3980 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f17280ab5b0 tx=0x7f1728005c00 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:33.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.522+0000 7f17157fa700 1 -- 172.21.15.101:0/2672871586 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1724015900 con 0x5622fd134ad0 2023-07-18T03:23:33.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.522+0000 7f1747486b80 10 client.28679 fetch_fsmap learned FSMap version 39 2023-07-18T03:23:33.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.522+0000 7f1747486b80 1 -- 172.21.15.101:0/2672871586 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5622fce92c60 con 0x5622fd134ad0 2023-07-18T03:23:33.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.523+0000 7f17157fa700 1 -- 172.21.15.101:0/2672871586 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5622fcfd54f0 con 0x5622fd134ad0 2023-07-18T03:23:33.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.523+0000 7f17157fa700 1 -- 172.21.15.101:0/2672871586 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 39) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f172401eb80 con 0x5622fd134ad0 2023-07-18T03:23:33.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.524+0000 7f1747486b80 10 client.28679 fetch_fsmap finished waiting for FSMap version 39 2023-07-18T03:23:33.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.524+0000 7f1747486b80 10 client.28679 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:23:33.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.524+0000 7f1747486b80 20 client.28679 populate_metadata read hostname 'smithi101' 2023-07-18T03:23:33.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.524+0000 7f1747486b80 1 --2- 172.21.15.101:0/2672871586 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5622fd1443a0 0x5622fd1467b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:33.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.524+0000 7f1747486b80 4 client.28679 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:23:33.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.524+0000 7f1747486b80 1 -- 172.21.15.101:0/2672871586 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5622fd074eb0 con 0x5622fd1443a0 2023-07-18T03:23:33.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.532+0000 7f173119a700 1 --2- 172.21.15.101:0/2672871586 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5622fd1443a0 0x5622fd1467b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:33.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.534+0000 7f173119a700 1 --2- 172.21.15.101:0/2672871586 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5622fd1443a0 0x5622fd1467b0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d803b3a4d602cac3 server_cookie=855a7a1a58a4edd8 in_seq=0 out_seq=0 2023-07-18T03:23:33.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.534+0000 7f17157fa700 10 client.28679 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:23:33.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.535+0000 7f17157fa700 1 -- 172.21.15.101:0/2672871586 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5622fd074eb0 con 0x5622fd1443a0 2023-07-18T03:23:33.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.535+0000 7f17157fa700 10 client.28679 handle_command_reply: tid=0 2023-07-18T03:23:33.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.537+0000 7f1747486b80 1 client.28679 shutdown 2023-07-18T03:23:33.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.540+0000 7f1747486b80 1 -- 172.21.15.101:0/2672871586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622fd134ad0 msgr2=0x5622fd131ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:33.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.540+0000 7f1747486b80 1 --2- 172.21.15.101:0/2672871586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622fd134ad0 0x5622fd131ec0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f1724009530 tx=0x7f1724007610 comp rx=0 tx=0).stop 2023-07-18T03:23:33.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.540+0000 7f1747486b80 1 -- 172.21.15.101:0/2672871586 shutdown_connections 2023-07-18T03:23:33.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.540+0000 7f1747486b80 1 --2- 172.21.15.101:0/2672871586 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5622fd1443a0 0x5622fd1467b0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.540+0000 7f1747486b80 1 --2- 172.21.15.101:0/2672871586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622fd134ad0 0x5622fd131ec0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.540+0000 7f1747486b80 1 --2- 172.21.15.101:0/2672871586 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5622fd1292d0 0x5622fd134500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.540+0000 7f1747486b80 1 --2- 172.21.15.101:0/2672871586 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622fd129be0 0x5622fd131780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.540+0000 7f1747486b80 1 -- 172.21.15.101:0/2672871586 >> 172.21.15.101:0/2672871586 conn(0x5622fd11fda0 msgr2=0x5622fd125230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:33.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.540+0000 7f17157fa700 0 client.28679 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:23:33.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.542+0000 7f1747486b80 1 -- 172.21.15.101:0/2672871586 shutdown_connections 2023-07-18T03:23:33.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.542+0000 7f1747486b80 1 -- 172.21.15.101:0/2672871586 wait complete. 2023-07-18T03:23:33.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.542+0000 7f1747486b80 20 client.28679 trim_cache size 0 max 16384 2023-07-18T03:23:33.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:33 smithi155 ceph-mon[128885]: pgmap v502: 129 pgs: 129 active+clean; 1.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 15 MiB/s wr, 672 op/s 2023-07-18T03:23:33.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:33 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/407615324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:23:33.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.566+0000 7f1747486b80 1 Processor -- start 2023-07-18T03:23:33.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.566+0000 7f1747486b80 1 -- start start 2023-07-18T03:23:33.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.567+0000 7f1747486b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622fd1292d0 0x5622fd1463b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:33.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.567+0000 7f1747486b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5622fd129be0 0x5622fd12ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:33.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.567+0000 7f1747486b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622fd134ad0 0x5622fd12cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:33.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.567+0000 7f1747486b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622fd12a590 con 0x5622fd129be0 2023-07-18T03:23:33.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.567+0000 7f1747486b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622fd11f970 con 0x5622fd134ad0 2023-07-18T03:23:33.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.567+0000 7f1747486b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622fd1264c0 con 0x5622fd1292d0 2023-07-18T03:23:33.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.568+0000 7f173219c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622fd1292d0 0x5622fd1463b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:33.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.568+0000 7f173219c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622fd1292d0 0x5622fd1463b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41828/0 (socket says 172.21.15.101:41828) 2023-07-18T03:23:33.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.568+0000 7f173219c700 1 -- 172.21.15.101:0/1939302664 learned_addr learned my addr 172.21.15.101:0/1939302664 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:23:33.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.568+0000 7f173219c700 1 --2- 172.21.15.101:0/1939302664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622fd134ad0 0x5622fd12cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:33.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.568+0000 7f173219c700 1 -- 172.21.15.101:0/1939302664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622fd134ad0 msgr2=0x5622fd12cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:33.567 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.568+0000 7f173219c700 1 --2- 172.21.15.101:0/1939302664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622fd134ad0 0x5622fd12cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.567 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.568+0000 7f173219c700 1 -- 172.21.15.101:0/1939302664 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5622fd129be0 msgr2=0x5622fd12ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:33.567 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.568+0000 7f173219c700 1 --2- 172.21.15.101:0/1939302664 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5622fd129be0 0x5622fd12ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.567 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.568+0000 7f173219c700 1 -- 172.21.15.101:0/1939302664 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622fced0250 con 0x5622fd1292d0 2023-07-18T03:23:33.567 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.569+0000 7f173219c700 1 --2- 172.21.15.101:0/1939302664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622fd1292d0 0x5622fd1463b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f17240287b0 tx=0x7f1724000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5699cbabbbe6a8 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:33.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.577+0000 7f1720bff700 1 -- 172.21.15.101:0/1939302664 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1724019040 con 0x5622fd1292d0 2023-07-18T03:23:33.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.577+0000 7f1720bff700 1 -- 172.21.15.101:0/1939302664 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f172401a040 con 0x5622fd1292d0 2023-07-18T03:23:33.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.577+0000 7f1720bff700 1 -- 172.21.15.101:0/1939302664 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17240296e0 con 0x5622fd1292d0 2023-07-18T03:23:33.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.579+0000 7f1747486b80 1 -- 172.21.15.101:0/1939302664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622fd1292d0 msgr2=0x5622fd1463b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:33.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.579+0000 7f1747486b80 1 --2- 172.21.15.101:0/1939302664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622fd1292d0 0x5622fd1463b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f17240287b0 tx=0x7f1724000f40 comp rx=0 tx=0).stop 2023-07-18T03:23:33.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.579+0000 7f1747486b80 1 -- 172.21.15.101:0/1939302664 shutdown_connections 2023-07-18T03:23:33.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.579+0000 7f1747486b80 1 --2- 172.21.15.101:0/1939302664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622fd134ad0 0x5622fd12cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.579+0000 7f1747486b80 1 --2- 172.21.15.101:0/1939302664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622fd1292d0 0x5622fd1463b0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.579+0000 7f1747486b80 1 --2- 172.21.15.101:0/1939302664 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5622fd129be0 0x5622fd12ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.579+0000 7f1747486b80 1 -- 172.21.15.101:0/1939302664 >> 172.21.15.101:0/1939302664 conn(0x5622fd11fda0 msgr2=0x5622fd140720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:33.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.582+0000 7f1747486b80 1 -- 172.21.15.101:0/1939302664 shutdown_connections 2023-07-18T03:23:33.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.584+0000 7f1747486b80 1 -- 172.21.15.101:0/1939302664 wait complete. 2023-07-18T03:23:33.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.584+0000 7f1747486b80 1 -- start start 2023-07-18T03:23:33.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.585+0000 7f1747486b80 1 Processor -- start 2023-07-18T03:23:33.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.586+0000 7f1747486b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622fd1292d0 0x5622fd13b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:33.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.586+0000 7f1747486b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5622fd129be0 0x5622fd138ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:33.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.586+0000 7f1747486b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622fd134ad0 0x5622fd1391f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:33.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.586+0000 7f1747486b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622fd139930 con 0x5622fd129be0 2023-07-18T03:23:33.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.586+0000 7f1747486b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622fd197260 con 0x5622fd134ad0 2023-07-18T03:23:33.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.586+0000 7f1747486b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622fd197420 con 0x5622fd1292d0 2023-07-18T03:23:33.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.586+0000 7f1747486b80 10 client.0 _ll_register_callbacks cb 0x7f1724003e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:23:33.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.587+0000 7f173219c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622fd1292d0 0x5622fd13b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:33.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.587+0000 7f173219c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622fd1292d0 0x5622fd13b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41836/0 (socket says 172.21.15.101:41836) 2023-07-18T03:23:33.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.587+0000 7f173219c700 1 -- 172.21.15.101:0/3767106416 learned_addr learned my addr 172.21.15.101:0/3767106416 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:23:33.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.587+0000 7f173219c700 1 --2- 172.21.15.101:0/3767106416 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622fd134ad0 0x5622fd1391f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:33.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.587+0000 7f173219c700 1 -- 172.21.15.101:0/3767106416 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622fd134ad0 msgr2=0x5622fd1391f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:33.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.587+0000 7f173219c700 1 --2- 172.21.15.101:0/3767106416 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622fd134ad0 0x5622fd1391f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.587+0000 7f173219c700 1 -- 172.21.15.101:0/3767106416 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5622fd129be0 msgr2=0x5622fd138ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:33.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.587+0000 7f173219c700 1 --2- 172.21.15.101:0/3767106416 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5622fd129be0 0x5622fd138ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.588+0000 7f173219c700 1 -- 172.21.15.101:0/3767106416 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5622fce92c60 con 0x5622fd1292d0 2023-07-18T03:23:33.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.589+0000 7f173219c700 1 --2- 172.21.15.101:0/3767106416 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622fd1292d0 0x5622fd13b720 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f1724020040 tx=0x7f1724006160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c49b0cb92c57b2a2 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:33.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.589+0000 7f17157fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:23:33.589 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.589+0000 7f17157fa700 1 -- 172.21.15.101:0/3767106416 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1724019070 con 0x5622fd1292d0 2023-07-18T03:23:33.589 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.589+0000 7f17157fa700 1 -- 172.21.15.101:0/3767106416 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f1724030040 con 0x5622fd1292d0 2023-07-18T03:23:33.589 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.589+0000 7f1747486b80 1 -- 172.21.15.101:0/3767106416 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f172400f040 con 0x5622fd1292d0 2023-07-18T03:23:33.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.590+0000 7f17157fa700 1 -- 172.21.15.101:0/3767106416 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1724029690 con 0x5622fd1292d0 2023-07-18T03:23:33.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.590+0000 7f1747486b80 10 client.28693 fetch_fsmap learned FSMap version 39 2023-07-18T03:23:33.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.590+0000 7f1747486b80 1 -- 172.21.15.101:0/3767106416 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5622fcea82b0 con 0x5622fd1292d0 2023-07-18T03:23:33.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.593+0000 7f17157fa700 1 -- 172.21.15.101:0/3767106416 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f172400f040 con 0x5622fd1292d0 2023-07-18T03:23:33.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.593+0000 7f17157fa700 1 -- 172.21.15.101:0/3767106416 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 39) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f1724015540 con 0x5622fd1292d0 2023-07-18T03:23:33.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.593+0000 7f1747486b80 10 client.28693 fetch_fsmap finished waiting for FSMap version 39 2023-07-18T03:23:33.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.593+0000 7f1747486b80 10 client.28693 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:23:33.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.593+0000 7f1747486b80 20 client.28693 populate_metadata read hostname 'smithi101' 2023-07-18T03:23:33.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.593+0000 7f1747486b80 1 --2- 172.21.15.101:0/3767106416 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5622fd198630 0x5622fd19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:33.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.593+0000 7f1747486b80 4 client.28693 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:23:33.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.593+0000 7f1747486b80 1 -- 172.21.15.101:0/3767106416 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5622fd037050 con 0x5622fd198630 2023-07-18T03:23:33.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.595+0000 7f173119a700 1 --2- 172.21.15.101:0/3767106416 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5622fd198630 0x5622fd19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:33.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.595+0000 7f173119a700 1 --2- 172.21.15.101:0/3767106416 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5622fd198630 0x5622fd19aa10 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a99842d3703846ca server_cookie=13970c05f4d0e17b in_seq=0 out_seq=0 2023-07-18T03:23:33.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.596+0000 7f17157fa700 10 client.28693 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:23:33.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.597+0000 7f17157fa700 1 -- 172.21.15.101:0/3767106416 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5622fd037050 con 0x5622fd198630 2023-07-18T03:23:33.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.597+0000 7f17157fa700 10 client.28693 handle_command_reply: tid=0 2023-07-18T03:23:33.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.597+0000 7f1747486b80 1 client.28693 shutdown 2023-07-18T03:23:33.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.598+0000 7f1747486b80 1 -- 172.21.15.101:0/3767106416 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622fd1292d0 msgr2=0x5622fd13b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:33.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.598+0000 7f1747486b80 1 --2- 172.21.15.101:0/3767106416 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622fd1292d0 0x5622fd13b720 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f1724020040 tx=0x7f1724006160 comp rx=0 tx=0).stop 2023-07-18T03:23:33.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.598+0000 7f1747486b80 1 -- 172.21.15.101:0/3767106416 shutdown_connections 2023-07-18T03:23:33.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.598+0000 7f1747486b80 1 --2- 172.21.15.101:0/3767106416 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5622fd198630 0x5622fd19aa10 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.598+0000 7f1747486b80 1 --2- 172.21.15.101:0/3767106416 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5622fd134ad0 0x5622fd1391f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.598+0000 7f1747486b80 1 --2- 172.21.15.101:0/3767106416 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5622fd1292d0 0x5622fd13b720 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.598+0000 7f1747486b80 1 --2- 172.21.15.101:0/3767106416 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5622fd129be0 0x5622fd138ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.598+0000 7f1747486b80 1 -- 172.21.15.101:0/3767106416 >> 172.21.15.101:0/3767106416 conn(0x5622fd11fda0 msgr2=0x5622fd134110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:33.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.598+0000 7f17157fa700 0 client.28693 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:23:33.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.599+0000 7f1747486b80 1 -- 172.21.15.101:0/3767106416 shutdown_connections 2023-07-18T03:23:33.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.599+0000 7f1747486b80 1 -- 172.21.15.101:0/3767106416 wait complete. 2023-07-18T03:23:33.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.602+0000 7f1747486b80 20 client.28693 trim_cache size 0 max 16384 2023-07-18T03:23:33.599 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:23:33.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.606+0000 7f1733bff700 1 -- 172.21.15.101:0/3191661172 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f17180a14d0 msgr2=0x7f17180a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:33.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.606+0000 7f1733bff700 1 --2- 172.21.15.101:0/3191661172 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f17180a14d0 0x7f17180a3980 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f17280ab5b0 tx=0x7f1728005c00 comp rx=0 tx=0).stop 2023-07-18T03:23:33.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.606+0000 7f1733bff700 1 -- 172.21.15.101:0/3191661172 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f172c06c9a0 msgr2=0x7f172c116de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:33.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.606+0000 7f1733bff700 1 --2- 172.21.15.101:0/3191661172 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f172c06c9a0 0x7f172c116de0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1728003ba0 tx=0x7f1728003bd0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.606+0000 7f1733bff700 1 -- 172.21.15.101:0/3191661172 shutdown_connections 2023-07-18T03:23:33.604 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.606+0000 7f1733bff700 1 --2- 172.21.15.101:0/3191661172 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f17180a14d0 0x7f17180a3980 secure :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f17280ab5b0 tx=0x7f1728005c00 comp rx=0 tx=0).stop 2023-07-18T03:23:33.604 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.606+0000 7f1733bff700 1 --2- 172.21.15.101:0/3191661172 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f172c06c9a0 0x7f172c116de0 secure :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1728003ba0 tx=0x7f1728003bd0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.604 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.606+0000 7f1733bff700 1 --2- 172.21.15.101:0/3191661172 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f172c10e280 0x7f172c1a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.604 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.606+0000 7f1733bff700 1 --2- 172.21.15.101:0/3191661172 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f172c06d3a0 0x7f172c117320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.604 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.606+0000 7f1733bff700 1 -- 172.21.15.101:0/3191661172 >> 172.21.15.101:0/3191661172 conn(0x7f172c06bf70 msgr2=0x7f172c06f180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:33.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.610+0000 7f1733bff700 1 -- 172.21.15.101:0/3191661172 shutdown_connections 2023-07-18T03:23:33.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.610+0000 7f1733bff700 1 -- 172.21.15.101:0/3191661172 wait complete. 2023-07-18T03:23:33.633 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-18T03:23:33.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:33 smithi179 ceph-mon[129229]: pgmap v502: 129 pgs: 129 active+clean; 1.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 15 MiB/s wr, 672 op/s 2023-07-18T03:23:33.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:33 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/407615324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:23:33.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.899+0000 7f6f327ff700 1 -- 172.21.15.101:0/1073652133 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6f2c106fb0 msgr2=0x7f6f2c10eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:33.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.899+0000 7f6f327ff700 1 --2- 172.21.15.101:0/1073652133 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6f2c106fb0 0x7f6f2c10eb80 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f6f24009fc0 tx=0x7f6f2400b330 comp rx=0 tx=0).stop 2023-07-18T03:23:33.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.899+0000 7f6f327ff700 1 -- 172.21.15.101:0/1073652133 shutdown_connections 2023-07-18T03:23:33.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.899+0000 7f6f327ff700 1 --2- 172.21.15.101:0/1073652133 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6f2c106fb0 0x7f6f2c10eb80 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.899+0000 7f6f327ff700 1 --2- 172.21.15.101:0/1073652133 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f2c101f70 0x7f6f2c102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.899+0000 7f6f327ff700 1 --2- 172.21.15.101:0/1073652133 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f2c102910 0x7f6f2c1068d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.899+0000 7f6f327ff700 1 -- 172.21.15.101:0/1073652133 >> 172.21.15.101:0/1073652133 conn(0x7f6f2c0fd7e0 msgr2=0x7f6f2c0ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:33.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.900+0000 7f6f327ff700 1 -- 172.21.15.101:0/1073652133 shutdown_connections 2023-07-18T03:23:33.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.900+0000 7f6f327ff700 1 -- 172.21.15.101:0/1073652133 wait complete. 2023-07-18T03:23:33.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.901+0000 7f6f327ff700 1 Processor -- start 2023-07-18T03:23:33.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.901+0000 7f6f327ff700 1 -- start start 2023-07-18T03:23:33.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.902+0000 7f6f327ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f2c101f70 0x7f6f2c197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:33.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.902+0000 7f6f327ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f2c102910 0x7f6f2c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:33.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.902+0000 7f6f327ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6f2c106fb0 0x7f6f2c19c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:33.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.902+0000 7f6f327ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c19cf10 con 0x7f6f2c106fb0 2023-07-18T03:23:33.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.902+0000 7f6f327ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c19d080 con 0x7f6f2c102910 2023-07-18T03:23:33.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.902+0000 7f6f327ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c19d240 con 0x7f6f2c101f70 2023-07-18T03:23:33.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.902+0000 7f6f2bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f2c101f70 0x7f6f2c197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:33.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.902+0000 7f6f2bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f2c101f70 0x7f6f2c197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41858/0 (socket says 172.21.15.101:41858) 2023-07-18T03:23:33.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.902+0000 7f6f2bfff700 1 -- 172.21.15.101:0/745644418 learned_addr learned my addr 172.21.15.101:0/745644418 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:23:33.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.902+0000 7f6f2b7fe700 1 --2- 172.21.15.101:0/745644418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f2c102910 0x7f6f2c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:33.900 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.902+0000 7f6f30d9c700 1 --2- 172.21.15.101:0/745644418 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6f2c106fb0 0x7f6f2c19c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:33.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.903+0000 7f6f2bfff700 1 -- 172.21.15.101:0/745644418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f2c102910 msgr2=0x7f6f2c198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:33.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.903+0000 7f6f2bfff700 1 --2- 172.21.15.101:0/745644418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f2c102910 0x7f6f2c198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.903+0000 7f6f2bfff700 1 -- 172.21.15.101:0/745644418 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6f2c106fb0 msgr2=0x7f6f2c19c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:33.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.903+0000 7f6f2bfff700 1 --2- 172.21.15.101:0/745644418 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6f2c106fb0 0x7f6f2c19c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.903+0000 7f6f2bfff700 1 -- 172.21.15.101:0/745644418 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f200095f0 con 0x7f6f2c101f70 2023-07-18T03:23:33.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.903+0000 7f6f2bfff700 1 --2- 172.21.15.101:0/745644418 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f2c101f70 0x7f6f2c197f30 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c00e9a0 tx=0x7f6f1c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:33.901 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.903+0000 7f6f2b7fe700 1 --2- 172.21.15.101:0/745644418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f2c102910 0x7f6f2c198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:23:33.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.903+0000 7f6f297fa700 1 -- 172.21.15.101:0/745644418 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c00ca10 con 0x7f6f2c101f70 2023-07-18T03:23:33.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.903+0000 7f6f327ff700 1 -- 172.21.15.101:0/745644418 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f24009c40 con 0x7f6f2c101f70 2023-07-18T03:23:33.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.903+0000 7f6f327ff700 1 -- 172.21.15.101:0/745644418 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f2c19d710 con 0x7f6f2c101f70 2023-07-18T03:23:33.902 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.905+0000 7f6f297fa700 1 -- 172.21.15.101:0/745644418 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f1c00cb70 con 0x7f6f2c101f70 2023-07-18T03:23:33.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.905+0000 7f6f297fa700 1 -- 172.21.15.101:0/745644418 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c018680 con 0x7f6f2c101f70 2023-07-18T03:23:33.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.909+0000 7f6f297fa700 1 -- 172.21.15.101:0/745644418 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f6f1c018820 con 0x7f6f2c101f70 2023-07-18T03:23:33.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.909+0000 7f6f46100b80 1 Processor -- start 2023-07-18T03:23:33.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.911+0000 7f6f46100b80 1 -- start start 2023-07-18T03:23:33.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.912+0000 7f6f46100b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fbb2232d0 0x557fbb2236a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:33.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.912+0000 7f6f46100b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557fbb223be0 0x557fbb22e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:33.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.912+0000 7f6f46100b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557fbb22ead0 0x557fbb230eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:33.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.912+0000 7f6f46100b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fbb129cb0 con 0x557fbb223be0 2023-07-18T03:23:33.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.912+0000 7f6f46100b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fbb17ca40 con 0x557fbb2232d0 2023-07-18T03:23:33.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.912+0000 7f6f46100b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fbb175ed0 con 0x557fbb22ead0 2023-07-18T03:23:33.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.912+0000 7f6f30d9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fbb2232d0 0x557fbb2236a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:33.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.912+0000 7f6f30d9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fbb2232d0 0x557fbb2236a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:54372/0 (socket says 172.21.15.101:54372) 2023-07-18T03:23:33.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.912+0000 7f6f30d9c700 1 -- 172.21.15.101:0/166349324 learned_addr learned my addr 172.21.15.101:0/166349324 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:23:33.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.912+0000 7f6f2b7fe700 1 --2- 172.21.15.101:0/166349324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557fbb22ead0 0x557fbb230eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:33.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.912+0000 7f6f30d9c700 1 -- 172.21.15.101:0/166349324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557fbb22ead0 msgr2=0x557fbb230eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:33.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.912+0000 7f6f2bfff700 1 --2- 172.21.15.101:0/166349324 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557fbb223be0 0x557fbb22e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:33.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.912+0000 7f6f30d9c700 1 --2- 172.21.15.101:0/166349324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557fbb22ead0 0x557fbb230eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.912+0000 7f6f30d9c700 1 -- 172.21.15.101:0/166349324 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557fbb223be0 msgr2=0x557fbb22e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:33.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.912+0000 7f6f30d9c700 1 --2- 172.21.15.101:0/166349324 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557fbb223be0 0x557fbb22e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.913+0000 7f6f30d9c700 1 -- 172.21.15.101:0/166349324 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fbafa3de0 con 0x557fbb2232d0 2023-07-18T03:23:33.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.913+0000 7f6f297fa700 1 --2- 172.21.15.101:0/745644418 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6f140a1340 0x7f6f140a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:33.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.913+0000 7f6f297fa700 1 -- 172.21.15.101:0/745644418 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f6f1c014070 con 0x7f6f2c101f70 2023-07-18T03:23:33.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.913+0000 7f6f2b7fe700 1 --2- 172.21.15.101:0/166349324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557fbb22ead0 0x557fbb230eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:23:33.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.913+0000 7f6f30d9c700 1 --2- 172.21.15.101:0/166349324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fbb2232d0 0x557fbb2236a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f6f24009bf0 tx=0x7f6f24009340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=425290e1ec02904f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:33.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.913+0000 7f6f2bfff700 1 --2- 172.21.15.101:0/166349324 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557fbb223be0 0x557fbb22e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:23:33.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.913+0000 7f6f2bfff700 1 --2- 172.21.15.101:0/745644418 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6f140a1340 0x7f6f140a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:33.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.914+0000 7f6f127fc700 1 -- 172.21.15.101:0/166349324 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2400e040 con 0x557fbb2232d0 2023-07-18T03:23:33.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.914+0000 7f6f127fc700 1 -- 172.21.15.101:0/166349324 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f24007bd0 con 0x557fbb2232d0 2023-07-18T03:23:33.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.914+0000 7f6f127fc700 1 -- 172.21.15.101:0/166349324 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2400bc30 con 0x557fbb2232d0 2023-07-18T03:23:33.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.915+0000 7f6f46100b80 1 -- 172.21.15.101:0/166349324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fbb2232d0 msgr2=0x557fbb2236a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:33.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.915+0000 7f6f46100b80 1 --2- 172.21.15.101:0/166349324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fbb2232d0 0x557fbb2236a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f6f24009bf0 tx=0x7f6f24009340 comp rx=0 tx=0).stop 2023-07-18T03:23:33.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.915+0000 7f6f2bfff700 1 --2- 172.21.15.101:0/745644418 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6f140a1340 0x7f6f140a37f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c0ca710 tx=0x7f6f1c000bc0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:33.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.915+0000 7f6f46100b80 1 -- 172.21.15.101:0/166349324 shutdown_connections 2023-07-18T03:23:33.916 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.915+0000 7f6f46100b80 1 --2- 172.21.15.101:0/166349324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fbb2232d0 0x557fbb2236a0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.916 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.915+0000 7f6f46100b80 1 --2- 172.21.15.101:0/166349324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557fbb22ead0 0x557fbb230eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.916 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.915+0000 7f6f46100b80 1 --2- 172.21.15.101:0/166349324 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557fbb223be0 0x557fbb22e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.916 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.915+0000 7f6f46100b80 1 -- 172.21.15.101:0/166349324 >> 172.21.15.101:0/166349324 conn(0x557fbb219da0 msgr2=0x557fbb21f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:33.916 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.916+0000 7f6f46100b80 1 -- 172.21.15.101:0/166349324 shutdown_connections 2023-07-18T03:23:33.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.916+0000 7f6f46100b80 1 -- 172.21.15.101:0/166349324 wait complete. 2023-07-18T03:23:33.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.916+0000 7f6f46100b80 1 -- start start 2023-07-18T03:23:33.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.916+0000 7f6f46100b80 1 Processor -- start 2023-07-18T03:23:33.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.917+0000 7f6f46100b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557fbb2232d0 0x557fbb22e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:33.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.917+0000 7f6f46100b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fbb223be0 0x557fbb22b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:33.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.917+0000 7f6f46100b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557fbb22ead0 0x557fbb22bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:33.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.917+0000 7f6f46100b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fbb115ac0 con 0x557fbb2232d0 2023-07-18T03:23:33.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.917+0000 7f6f46100b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fbb1157c0 con 0x557fbb223be0 2023-07-18T03:23:33.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.917+0000 7f6f46100b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fbaff9920 con 0x557fbb22ead0 2023-07-18T03:23:33.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.917+0000 7f6f46100b80 10 client.0 _ll_register_callbacks cb 0x557fbb113760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:23:33.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.918+0000 7f6f2b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fbb223be0 0x557fbb22b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:33.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.918+0000 7f6f2b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fbb223be0 0x557fbb22b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:54380/0 (socket says 172.21.15.101:54380) 2023-07-18T03:23:33.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.918+0000 7f6f2b7fe700 1 -- 172.21.15.101:0/4015571356 learned_addr learned my addr 172.21.15.101:0/4015571356 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:23:33.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.918+0000 7f6f2b7fe700 1 -- 172.21.15.101:0/4015571356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557fbb22ead0 msgr2=0x557fbb22bbe0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-18T03:23:33.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.918+0000 7f6f2b7fe700 1 --2- 172.21.15.101:0/4015571356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557fbb22ead0 0x557fbb22bbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.918+0000 7f6f2b7fe700 1 -- 172.21.15.101:0/4015571356 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557fbb2232d0 msgr2=0x557fbb22e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:33.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.918+0000 7f6f2b7fe700 1 --2- 172.21.15.101:0/4015571356 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557fbb2232d0 0x557fbb22e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.918+0000 7f6f2b7fe700 1 -- 172.21.15.101:0/4015571356 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f2c10f3c0 con 0x557fbb223be0 2023-07-18T03:23:33.922 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.919+0000 7f6f2b7fe700 1 --2- 172.21.15.101:0/4015571356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fbb223be0 0x557fbb22b680 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f6f20013ce0 tx=0x7f6f200116c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b5a65f1ca4dc2ab server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:33.922 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.919+0000 7f6f0b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:23:33.922 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.919+0000 7f6f0b7fe700 1 -- 172.21.15.101:0/4015571356 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f20004a30 con 0x557fbb223be0 2023-07-18T03:23:33.922 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.919+0000 7f6f0b7fe700 1 -- 172.21.15.101:0/4015571356 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f6f20004b90 con 0x557fbb223be0 2023-07-18T03:23:33.922 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.919+0000 7f6f46100b80 1 -- 172.21.15.101:0/4015571356 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557fbb0cf4f0 con 0x557fbb223be0 2023-07-18T03:23:33.923 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.920+0000 7f6f0b7fe700 1 -- 172.21.15.101:0/4015571356 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f20004d50 con 0x557fbb223be0 2023-07-18T03:23:33.923 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.920+0000 7f6f46100b80 10 client.28697 fetch_fsmap learned FSMap version 39 2023-07-18T03:23:33.923 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.920+0000 7f6f46100b80 1 -- 172.21.15.101:0/4015571356 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x557fbaf8cc60 con 0x557fbb223be0 2023-07-18T03:23:33.923 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.920+0000 7f6f0b7fe700 1 -- 172.21.15.101:0/4015571356 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557fbb0cf4f0 con 0x557fbb223be0 2023-07-18T03:23:33.923 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.920+0000 7f6f0b7fe700 1 -- 172.21.15.101:0/4015571356 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 39) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f6f2001b790 con 0x557fbb223be0 2023-07-18T03:23:33.923 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.920+0000 7f6f46100b80 10 client.28697 fetch_fsmap finished waiting for FSMap version 39 2023-07-18T03:23:33.923 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.920+0000 7f6f46100b80 10 client.28697 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:23:33.924 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.920+0000 7f6f46100b80 20 client.28697 populate_metadata read hostname 'smithi101' 2023-07-18T03:23:33.924 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.920+0000 7f6f46100b80 1 --2- 172.21.15.101:0/4015571356 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557fbb23e390 0x557fbb2407a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:33.925 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.920+0000 7f6f46100b80 4 client.28697 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:23:33.925 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.920+0000 7f6f46100b80 1 -- 172.21.15.101:0/4015571356 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557fbb16eeb0 con 0x557fbb23e390 2023-07-18T03:23:33.925 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.921+0000 7f6f30d9c700 1 --2- 172.21.15.101:0/4015571356 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557fbb23e390 0x557fbb2407a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:33.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.921+0000 7f6f0b7fe700 10 client.28697 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:23:33.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.921+0000 7f6f30d9c700 1 --2- 172.21.15.101:0/4015571356 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557fbb23e390 0x557fbb2407a0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=613431c9ce9439f server_cookie=652bdb5dafeff18c in_seq=0 out_seq=0 2023-07-18T03:23:33.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.922+0000 7f6f0b7fe700 1 -- 172.21.15.101:0/4015571356 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557fbb16eeb0 con 0x557fbb23e390 2023-07-18T03:23:33.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.922+0000 7f6f0b7fe700 10 client.28697 handle_command_reply: tid=0 2023-07-18T03:23:33.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.922+0000 7f6f46100b80 1 client.28697 shutdown 2023-07-18T03:23:33.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.922+0000 7f6f46100b80 1 -- 172.21.15.101:0/4015571356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fbb223be0 msgr2=0x557fbb22b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:33.926 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.922+0000 7f6f46100b80 1 --2- 172.21.15.101:0/4015571356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fbb223be0 0x557fbb22b680 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f6f20013ce0 tx=0x7f6f200116c0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.922+0000 7f6f46100b80 1 -- 172.21.15.101:0/4015571356 shutdown_connections 2023-07-18T03:23:33.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.922+0000 7f6f46100b80 1 --2- 172.21.15.101:0/4015571356 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557fbb23e390 0x557fbb2407a0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.922+0000 7f6f46100b80 1 --2- 172.21.15.101:0/4015571356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557fbb22ead0 0x557fbb22bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.922+0000 7f6f46100b80 1 --2- 172.21.15.101:0/4015571356 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557fbb2232d0 0x557fbb22e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.922+0000 7f6f46100b80 1 --2- 172.21.15.101:0/4015571356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fbb223be0 0x557fbb22b680 secure :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f6f20013ce0 tx=0x7f6f200116c0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.922+0000 7f6f46100b80 1 -- 172.21.15.101:0/4015571356 >> 172.21.15.101:0/4015571356 conn(0x557fbb219da0 msgr2=0x557fbb21f230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:33.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.922+0000 7f6f0b7fe700 0 client.28697 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:23:33.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.922+0000 7f6f46100b80 1 -- 172.21.15.101:0/4015571356 shutdown_connections 2023-07-18T03:23:33.928 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.922+0000 7f6f46100b80 1 -- 172.21.15.101:0/4015571356 wait complete. 2023-07-18T03:23:33.928 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.923+0000 7f6f46100b80 20 client.28697 trim_cache size 0 max 16384 2023-07-18T03:23:33.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.940+0000 7f6f46100b80 1 Processor -- start 2023-07-18T03:23:33.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.941+0000 7f6f46100b80 1 -- start start 2023-07-18T03:23:33.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.941+0000 7f6f46100b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557fbb2232d0 0x557fbb2354f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:33.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.941+0000 7f6f46100b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557fbb22ead0 0x557fbb23a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:33.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.941+0000 7f6f46100b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fbb22df30 0x557fbb22e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:33.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.941+0000 7f6f46100b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fbb224160 con 0x557fbb22ead0 2023-07-18T03:23:33.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.941+0000 7f6f46100b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fbb224320 con 0x557fbb22df30 2023-07-18T03:23:33.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.941+0000 7f6f46100b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fbb230eb0 con 0x557fbb2232d0 2023-07-18T03:23:33.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.941+0000 7f6f30d9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557fbb2232d0 0x557fbb2354f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:33.940 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.941+0000 7f6f2b7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557fbb22ead0 0x557fbb23a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:33.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.941+0000 7f6f30d9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557fbb2232d0 0x557fbb2354f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41884/0 (socket says 172.21.15.101:41884) 2023-07-18T03:23:33.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.941+0000 7f6f30d9c700 1 -- 172.21.15.101:0/1533076852 learned_addr learned my addr 172.21.15.101:0/1533076852 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:23:33.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.941+0000 7f6f30d9c700 1 --2- 172.21.15.101:0/1533076852 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fbb22df30 0x557fbb22e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:33.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.942+0000 7f6f30d9c700 1 -- 172.21.15.101:0/1533076852 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fbb22df30 msgr2=0x557fbb22e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:33.941 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.942+0000 7f6f30d9c700 1 --2- 172.21.15.101:0/1533076852 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fbb22df30 0x557fbb22e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.942+0000 7f6f30d9c700 1 -- 172.21.15.101:0/1533076852 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557fbb22ead0 msgr2=0x557fbb23a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:33.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.942+0000 7f6f30d9c700 1 --2- 172.21.15.101:0/1533076852 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557fbb22ead0 0x557fbb23a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.942+0000 7f6f30d9c700 1 -- 172.21.15.101:0/1533076852 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fbb21b5e0 con 0x557fbb2232d0 2023-07-18T03:23:33.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.942+0000 7f6f2b7fe700 1 --2- 172.21.15.101:0/1533076852 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557fbb22ead0 0x557fbb23a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:23:33.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.942+0000 7f6f30d9c700 1 --2- 172.21.15.101:0/1533076852 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557fbb2232d0 0x557fbb2354f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6f24003c90 tx=0x7f6f24003a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f1b83719239c3b4 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:33.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.942+0000 7f6f12ffd700 1 -- 172.21.15.101:0/1533076852 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2400e040 con 0x557fbb2232d0 2023-07-18T03:23:33.942 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.942+0000 7f6f12ffd700 1 -- 172.21.15.101:0/1533076852 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f2402b070 con 0x557fbb2232d0 2023-07-18T03:23:33.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.942+0000 7f6f12ffd700 1 -- 172.21.15.101:0/1533076852 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f24025840 con 0x557fbb2232d0 2023-07-18T03:23:33.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.943+0000 7f6f46100b80 1 -- 172.21.15.101:0/1533076852 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557fbb2232d0 msgr2=0x557fbb2354f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:33.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.943+0000 7f6f46100b80 1 --2- 172.21.15.101:0/1533076852 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557fbb2232d0 0x557fbb2354f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6f24003c90 tx=0x7f6f24003a00 comp rx=0 tx=0).stop 2023-07-18T03:23:33.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.943+0000 7f6f46100b80 1 -- 172.21.15.101:0/1533076852 shutdown_connections 2023-07-18T03:23:33.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.943+0000 7f6f46100b80 1 --2- 172.21.15.101:0/1533076852 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fbb22df30 0x557fbb22e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.943+0000 7f6f46100b80 1 --2- 172.21.15.101:0/1533076852 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557fbb2232d0 0x557fbb2354f0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.943 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.943+0000 7f6f46100b80 1 --2- 172.21.15.101:0/1533076852 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557fbb22ead0 0x557fbb23a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.943+0000 7f6f46100b80 1 -- 172.21.15.101:0/1533076852 >> 172.21.15.101:0/1533076852 conn(0x557fbb219da0 msgr2=0x557fbb21de10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:33.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.944+0000 7f6f46100b80 1 -- 172.21.15.101:0/1533076852 shutdown_connections 2023-07-18T03:23:33.944 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.944+0000 7f6f46100b80 1 -- 172.21.15.101:0/1533076852 wait complete. 2023-07-18T03:23:33.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.944+0000 7f6f46100b80 1 -- start start 2023-07-18T03:23:33.945 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.944+0000 7f6f46100b80 1 Processor -- start 2023-07-18T03:23:33.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.945+0000 7f6f46100b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557fbb2232d0 0x557fbb233340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:33.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.945+0000 7f6f46100b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fbb22df30 0x557fbb233880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:33.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.945+0000 7f6f46100b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557fbb22ead0 0x557fbb299a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:33.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.945+0000 7f6f46100b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fbb299fc0 con 0x557fbb22ead0 2023-07-18T03:23:33.946 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.945+0000 7f6f46100b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fbb29a100 con 0x557fbb22df30 2023-07-18T03:23:33.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.945+0000 7f6f46100b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fbb288290 con 0x557fbb2232d0 2023-07-18T03:23:33.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.945+0000 7f6f46100b80 10 client.0 _ll_register_callbacks cb 0x7f6f20013800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:23:33.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.946+0000 7f6f30d9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557fbb2232d0 0x557fbb233340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:33.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.946+0000 7f6f2b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fbb22df30 0x557fbb233880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:33.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.946+0000 7f6f2b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fbb22df30 0x557fbb233880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:54402/0 (socket says 172.21.15.101:54402) 2023-07-18T03:23:33.947 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.946+0000 7f6f2b7fe700 1 -- 172.21.15.101:0/3088921816 learned_addr learned my addr 172.21.15.101:0/3088921816 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:23:33.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.946+0000 7f6f30d9c700 1 --2- 172.21.15.101:0/3088921816 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557fbb22ead0 0x557fbb299a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:33.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.946+0000 7f6f2b7fe700 1 -- 172.21.15.101:0/3088921816 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557fbb2232d0 msgr2=0x557fbb233340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:33.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.946+0000 7f6f2b7fe700 1 --2- 172.21.15.101:0/3088921816 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557fbb2232d0 0x557fbb233340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.946+0000 7f6f2b7fe700 1 -- 172.21.15.101:0/3088921816 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557fbb22ead0 msgr2=0x557fbb299a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:33.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.946+0000 7f6f2b7fe700 1 --2- 172.21.15.101:0/3088921816 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557fbb22ead0 0x557fbb299a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.947+0000 7f6f2b7fe700 1 -- 172.21.15.101:0/3088921816 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557fbafa3de0 con 0x557fbb22df30 2023-07-18T03:23:33.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.947+0000 7f6f2b7fe700 1 --2- 172.21.15.101:0/3088921816 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fbb22df30 0x557fbb233880 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6f2001bc40 tx=0x7f6f20024680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4aaf4a1d49cb567 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:33.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.947+0000 7f6f0b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:23:33.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.947+0000 7f6f0b7fe700 1 -- 172.21.15.101:0/3088921816 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f20009380 con 0x557fbb22df30 2023-07-18T03:23:33.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.947+0000 7f6f0b7fe700 1 -- 172.21.15.101:0/3088921816 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f6f20000e40 con 0x557fbb22df30 2023-07-18T03:23:33.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.947+0000 7f6f46100b80 1 -- 172.21.15.101:0/3088921816 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6f24007bd0 con 0x557fbb22df30 2023-07-18T03:23:33.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.948+0000 7f6f0b7fe700 1 -- 172.21.15.101:0/3088921816 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f20011ac0 con 0x557fbb22df30 2023-07-18T03:23:33.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.948+0000 7f6f46100b80 10 client.28703 fetch_fsmap learned FSMap version 39 2023-07-18T03:23:33.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.948+0000 7f6f46100b80 1 -- 172.21.15.101:0/3088921816 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x557fbafca250 con 0x557fbb22df30 2023-07-18T03:23:33.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.948+0000 7f6f0b7fe700 1 -- 172.21.15.101:0/3088921816 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f20037080 con 0x557fbb22df30 2023-07-18T03:23:33.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.948+0000 7f6f0b7fe700 1 -- 172.21.15.101:0/3088921816 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 39) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f6f2002c070 con 0x557fbb22df30 2023-07-18T03:23:33.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.948+0000 7f6f46100b80 10 client.28703 fetch_fsmap finished waiting for FSMap version 39 2023-07-18T03:23:33.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.948+0000 7f6f46100b80 10 client.28703 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:23:33.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.948+0000 7f6f46100b80 20 client.28703 populate_metadata read hostname 'smithi101' 2023-07-18T03:23:33.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.949+0000 7f6f46100b80 1 --2- 172.21.15.101:0/3088921816 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557fbb289a30 0x557fbb28be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:33.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.949+0000 7f6f46100b80 4 client.28703 mds_command: new command op to 24562 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:23:33.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.949+0000 7f6f46100b80 1 -- 172.21.15.101:0/3088921816 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557fbb131050 con 0x557fbb289a30 2023-07-18T03:23:33.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.949+0000 7f6f30d9c700 1 --2- 172.21.15.101:0/3088921816 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557fbb289a30 0x557fbb28be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:33.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.949+0000 7f6f30d9c700 1 --2- 172.21.15.101:0/3088921816 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557fbb289a30 0x557fbb28be10 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd084cc503b262bf server_cookie=97a3863602f4d4b2 in_seq=0 out_seq=0 2023-07-18T03:23:33.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.950+0000 7f6f0b7fe700 10 client.28703 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:23:33.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.953+0000 7f6f0b7fe700 1 -- 172.21.15.101:0/3088921816 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557fbb131050 con 0x557fbb289a30 2023-07-18T03:23:33.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.953+0000 7f6f0b7fe700 10 client.28703 handle_command_reply: tid=0 2023-07-18T03:23:33.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.953+0000 7f6f46100b80 1 client.28703 shutdown 2023-07-18T03:23:33.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.954+0000 7f6f46100b80 1 -- 172.21.15.101:0/3088921816 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fbb22df30 msgr2=0x557fbb233880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:33.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.954+0000 7f6f46100b80 1 --2- 172.21.15.101:0/3088921816 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fbb22df30 0x557fbb233880 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6f2001bc40 tx=0x7f6f20024680 comp rx=0 tx=0).stop 2023-07-18T03:23:33.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.954+0000 7f6f46100b80 1 -- 172.21.15.101:0/3088921816 shutdown_connections 2023-07-18T03:23:33.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.954+0000 7f6f46100b80 1 --2- 172.21.15.101:0/3088921816 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557fbb289a30 0x557fbb28be10 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.954+0000 7f6f46100b80 1 --2- 172.21.15.101:0/3088921816 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557fbb22ead0 0x557fbb299a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.954+0000 7f6f46100b80 1 --2- 172.21.15.101:0/3088921816 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557fbb2232d0 0x557fbb233340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.954+0000 7f6f0b7fe700 0 client.28703 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:23:33.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.954+0000 7f6f46100b80 1 --2- 172.21.15.101:0/3088921816 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fbb22df30 0x557fbb233880 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.954+0000 7f6f46100b80 1 -- 172.21.15.101:0/3088921816 >> 172.21.15.101:0/3088921816 conn(0x557fbb219da0 msgr2=0x557fbb226990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:33.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.954+0000 7f6f46100b80 1 -- 172.21.15.101:0/3088921816 shutdown_connections 2023-07-18T03:23:33.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.954+0000 7f6f46100b80 1 -- 172.21.15.101:0/3088921816 wait complete. 2023-07-18T03:23:33.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.954+0000 7f6f46100b80 20 client.28703 trim_cache size 0 max 16384 2023-07-18T03:23:33.955 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:23:33.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.955+0000 7f6f327ff700 1 -- 172.21.15.101:0/745644418 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6f140a1340 msgr2=0x7f6f140a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:33.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.955+0000 7f6f327ff700 1 --2- 172.21.15.101:0/745644418 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6f140a1340 0x7f6f140a37f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c0ca710 tx=0x7f6f1c000bc0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.955+0000 7f6f327ff700 1 -- 172.21.15.101:0/745644418 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f2c101f70 msgr2=0x7f6f2c197f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:33.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.955+0000 7f6f327ff700 1 --2- 172.21.15.101:0/745644418 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f2c101f70 0x7f6f2c197f30 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c00e9a0 tx=0x7f6f1c00ecb0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.955+0000 7f6f327ff700 1 -- 172.21.15.101:0/745644418 shutdown_connections 2023-07-18T03:23:33.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.955+0000 7f6f327ff700 1 --2- 172.21.15.101:0/745644418 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6f2c106fb0 0x7f6f2c19c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.955+0000 7f6f327ff700 1 --2- 172.21.15.101:0/745644418 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f2c101f70 0x7f6f2c197f30 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.956+0000 7f6f327ff700 1 --2- 172.21.15.101:0/745644418 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6f140a1340 0x7f6f140a37f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.956+0000 7f6f327ff700 1 --2- 172.21.15.101:0/745644418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f2c102910 0x7f6f2c198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.956+0000 7f6f327ff700 1 -- 172.21.15.101:0/745644418 >> 172.21.15.101:0/745644418 conn(0x7f6f2c0fd7e0 msgr2=0x7f6f2c0fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:33.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.956+0000 7f6f327ff700 1 -- 172.21.15.101:0/745644418 shutdown_connections 2023-07-18T03:23:33.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:33.956+0000 7f6f327ff700 1 -- 172.21.15.101:0/745644418 wait complete. 2023-07-18T03:23:33.990 INFO:tasks.workunit.client.0.smithi101.stdout: 10 101198 29.93 MB/sec execute 58 sec latency 103.705 ms 2023-07-18T03:23:34.975 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-18T03:23:34.976 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-18T03:23:34.989 INFO:tasks.workunit.client.0.smithi101.stdout: 10 101870 30.05 MB/sec execute 59 sec latency 58.881 ms 2023-07-18T03:23:35.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.227+0000 7f2a8cdff700 1 -- 172.21.15.101:0/232380834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a88100480 msgr2=0x7f2a8810c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:35.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.227+0000 7f2a8cdff700 1 --2- 172.21.15.101:0/232380834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a88100480 0x7f2a8810c910 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2a8000b0a0 tx=0x7f2a8000b3b0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.227+0000 7f2a8cdff700 1 -- 172.21.15.101:0/232380834 shutdown_connections 2023-07-18T03:23:35.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.227+0000 7f2a8cdff700 1 --2- 172.21.15.101:0/232380834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a88105ae0 0x7f2a88105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.227+0000 7f2a8cdff700 1 --2- 172.21.15.101:0/232380834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a88100480 0x7f2a8810c910 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.227+0000 7f2a8cdff700 1 --2- 172.21.15.101:0/232380834 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2a880ffad0 0x7f2a880fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.227+0000 7f2a8cdff700 1 -- 172.21.15.101:0/232380834 >> 172.21.15.101:0/232380834 conn(0x7f2a880fb5f0 msgr2=0x7f2a880fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:35.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.227+0000 7f2a8cdff700 1 -- 172.21.15.101:0/232380834 shutdown_connections 2023-07-18T03:23:35.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.227+0000 7f2a8cdff700 1 -- 172.21.15.101:0/232380834 wait complete. 2023-07-18T03:23:35.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.228+0000 7f2a8cdff700 1 Processor -- start 2023-07-18T03:23:35.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.228+0000 7f2a8cdff700 1 -- start start 2023-07-18T03:23:35.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.229+0000 7f2a8cdff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a880ffad0 0x7f2a88072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:35.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.229+0000 7f2a8cdff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2a88100480 0x7f2a88072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:35.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.229+0000 7f2a8cdff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a88105ae0 0x7f2a881a3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:35.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.229+0000 7f2a8cdff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a88073580 con 0x7f2a88100480 2023-07-18T03:23:35.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.229+0000 7f2a8cdff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a881a4490 con 0x7f2a88105ae0 2023-07-18T03:23:35.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.229+0000 7f2a8cdff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a881a45d0 con 0x7f2a880ffad0 2023-07-18T03:23:35.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.229+0000 7f2a8659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a880ffad0 0x7f2a88072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:35.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.229+0000 7f2a8659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a880ffad0 0x7f2a88072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41922/0 (socket says 172.21.15.101:41922) 2023-07-18T03:23:35.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.229+0000 7f2a8659c700 1 -- 172.21.15.101:0/2509321594 learned_addr learned my addr 172.21.15.101:0/2509321594 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:23:35.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.229+0000 7f2a85d9b700 1 --2- 172.21.15.101:0/2509321594 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2a88100480 0x7f2a88072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:35.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.229+0000 7f2a86d9d700 1 --2- 172.21.15.101:0/2509321594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a88105ae0 0x7f2a881a3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:35.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.229+0000 7f2a85d9b700 1 -- 172.21.15.101:0/2509321594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a880ffad0 msgr2=0x7f2a88072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:35.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.229+0000 7f2a85d9b700 1 --2- 172.21.15.101:0/2509321594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a880ffad0 0x7f2a88072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.229+0000 7f2a85d9b700 1 -- 172.21.15.101:0/2509321594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a88105ae0 msgr2=0x7f2a881a3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:35.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.230+0000 7f2a85d9b700 1 --2- 172.21.15.101:0/2509321594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a88105ae0 0x7f2a881a3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.230+0000 7f2a85d9b700 1 -- 172.21.15.101:0/2509321594 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a80009d00 con 0x7f2a88100480 2023-07-18T03:23:35.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.230+0000 7f2a8659c700 1 --2- 172.21.15.101:0/2509321594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a880ffad0 0x7f2a88072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:23:35.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.230+0000 7f2a85d9b700 1 --2- 172.21.15.101:0/2509321594 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2a88100480 0x7f2a88072d50 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00e9f0 tx=0x7f2a7c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:35.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.230+0000 7f2a777fe700 1 -- 172.21.15.101:0/2509321594 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c00c9c0 con 0x7f2a88100480 2023-07-18T03:23:35.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.230+0000 7f2a777fe700 1 -- 172.21.15.101:0/2509321594 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a7c010430 con 0x7f2a88100480 2023-07-18T03:23:35.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.230+0000 7f2a777fe700 1 -- 172.21.15.101:0/2509321594 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c018680 con 0x7f2a88100480 2023-07-18T03:23:35.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.230+0000 7f2a8cdff700 1 -- 172.21.15.101:0/2509321594 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a881a4830 con 0x7f2a88100480 2023-07-18T03:23:35.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.230+0000 7f2a8cdff700 1 -- 172.21.15.101:0/2509321594 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a881a4cc0 con 0x7f2a88100480 2023-07-18T03:23:35.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.235+0000 7f2a777fe700 1 -- 172.21.15.101:0/2509321594 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f2a7c010860 con 0x7f2a88100480 2023-07-18T03:23:35.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.236+0000 7f2aa06e3b80 1 Processor -- start 2023-07-18T03:23:35.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.236+0000 7f2aa06e3b80 1 -- start start 2023-07-18T03:23:35.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.236+0000 7f2a777fe700 1 --2- 172.21.15.101:0/2509321594 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2a700a1340 0x7f2a700a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:35.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.237+0000 7f2a777fe700 1 -- 172.21.15.101:0/2509321594 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f2a7c014070 con 0x7f2a88100480 2023-07-18T03:23:35.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.237+0000 7f2aa06e3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5557cd603350 0x5557cd603720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:35.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.237+0000 7f2aa06e3b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557cd603c60 0x5557cd60e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:35.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.237+0000 7f2aa06e3b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557cd60eb50 0x5557cd610f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:35.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.237+0000 7f2aa06e3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557cd509d30 con 0x5557cd603350 2023-07-18T03:23:35.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.237+0000 7f2aa06e3b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557cd55cac0 con 0x5557cd60eb50 2023-07-18T03:23:35.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.237+0000 7f2aa06e3b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557cd555f50 con 0x5557cd603c60 2023-07-18T03:23:35.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.237+0000 7f2a8659c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5557cd603350 0x5557cd603720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:35.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.237+0000 7f2a86d9d700 1 --2- 172.21.15.101:0/2509321594 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2a700a1340 0x7f2a700a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:35.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.237+0000 7f2a85d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557cd603c60 0x5557cd60e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:35.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.237+0000 7f2a85d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557cd603c60 0x5557cd60e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41932/0 (socket says 172.21.15.101:41932) 2023-07-18T03:23:35.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.237+0000 7f2a85d9b700 1 -- 172.21.15.101:0/3030673846 learned_addr learned my addr 172.21.15.101:0/3030673846 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:23:35.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.238+0000 7f2a85d9b700 1 -- 172.21.15.101:0/3030673846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557cd60eb50 msgr2=0x5557cd610f30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-18T03:23:35.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.238+0000 7f2a85d9b700 1 --2- 172.21.15.101:0/3030673846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557cd60eb50 0x5557cd610f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.238+0000 7f2a85d9b700 1 -- 172.21.15.101:0/3030673846 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5557cd603350 msgr2=0x5557cd603720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:35.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.238+0000 7f2a85d9b700 1 --2- 172.21.15.101:0/3030673846 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5557cd603350 0x5557cd603720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.238+0000 7f2a85d9b700 1 -- 172.21.15.101:0/3030673846 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557cd383e60 con 0x5557cd603c60 2023-07-18T03:23:35.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.238+0000 7f2a85d9b700 1 --2- 172.21.15.101:0/3030673846 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557cd603c60 0x5557cd60e610 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c0963b0 tx=0x7f2a7c099a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe988e90a9114034 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:35.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.238+0000 7f2a74ff9700 1 -- 172.21.15.101:0/3030673846 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c09dee0 con 0x5557cd603c60 2023-07-18T03:23:35.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.238+0000 7f2a74ff9700 1 -- 172.21.15.101:0/3030673846 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a7c0a2300 con 0x5557cd603c60 2023-07-18T03:23:35.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.238+0000 7f2a74ff9700 1 -- 172.21.15.101:0/3030673846 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c099600 con 0x5557cd603c60 2023-07-18T03:23:35.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.239+0000 7f2aa06e3b80 1 -- 172.21.15.101:0/3030673846 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557cd603c60 msgr2=0x5557cd60e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:35.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.239+0000 7f2aa06e3b80 1 --2- 172.21.15.101:0/3030673846 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557cd603c60 0x5557cd60e610 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c0963b0 tx=0x7f2a7c099a70 comp rx=0 tx=0).stop 2023-07-18T03:23:35.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.240+0000 7f2a86d9d700 1 --2- 172.21.15.101:0/2509321594 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2a700a1340 0x7f2a700a37f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2a88074920 tx=0x7f2a80006230 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:35.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.240+0000 7f2aa06e3b80 1 -- 172.21.15.101:0/3030673846 shutdown_connections 2023-07-18T03:23:35.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.240+0000 7f2aa06e3b80 1 --2- 172.21.15.101:0/3030673846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557cd60eb50 0x5557cd610f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.240+0000 7f2aa06e3b80 1 --2- 172.21.15.101:0/3030673846 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5557cd603350 0x5557cd603720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.240+0000 7f2aa06e3b80 1 --2- 172.21.15.101:0/3030673846 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557cd603c60 0x5557cd60e610 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.240+0000 7f2aa06e3b80 1 -- 172.21.15.101:0/3030673846 >> 172.21.15.101:0/3030673846 conn(0x5557cd5f9e20 msgr2=0x5557cd5ff720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:35.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.240+0000 7f2aa06e3b80 1 -- 172.21.15.101:0/3030673846 shutdown_connections 2023-07-18T03:23:35.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.240+0000 7f2aa06e3b80 1 -- 172.21.15.101:0/3030673846 wait complete. 2023-07-18T03:23:35.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.240+0000 7f2aa06e3b80 1 -- start start 2023-07-18T03:23:35.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.241+0000 7f2aa06e3b80 1 Processor -- start 2023-07-18T03:23:35.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.241+0000 7f2aa06e3b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557cd603350 0x5557cd60b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:35.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.241+0000 7f2aa06e3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5557cd603c60 0x5557cd60bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:35.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.241+0000 7f2aa06e3b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557cd60eb50 0x5557cd60c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:35.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.241+0000 7f2aa06e3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557cd4f5b40 con 0x5557cd603c60 2023-07-18T03:23:35.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.241+0000 7f2aa06e3b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557cd4f5840 con 0x5557cd60eb50 2023-07-18T03:23:35.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.241+0000 7f2aa06e3b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557cd3d99a0 con 0x5557cd603350 2023-07-18T03:23:35.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.241+0000 7f2aa06e3b80 10 client.0 _ll_register_callbacks cb 0x5557cd4f37e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:23:35.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.241+0000 7f2a8659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557cd603350 0x5557cd60b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:35.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.241+0000 7f2a85d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5557cd603c60 0x5557cd60bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:35.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.241+0000 7f2a8659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557cd603350 0x5557cd60b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41946/0 (socket says 172.21.15.101:41946) 2023-07-18T03:23:35.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.241+0000 7f2a8659c700 1 -- 172.21.15.101:0/1209531448 learned_addr learned my addr 172.21.15.101:0/1209531448 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:23:35.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.241+0000 7f2a86d9d700 1 --2- 172.21.15.101:0/1209531448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557cd60eb50 0x5557cd60c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:35.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.242+0000 7f2a85d9b700 1 -- 172.21.15.101:0/1209531448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557cd603350 msgr2=0x5557cd60b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:35.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.242+0000 7f2a85d9b700 1 --2- 172.21.15.101:0/1209531448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557cd603350 0x5557cd60b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.242+0000 7f2a85d9b700 1 -- 172.21.15.101:0/1209531448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557cd60eb50 msgr2=0x5557cd60c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:35.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.242+0000 7f2a85d9b700 1 --2- 172.21.15.101:0/1209531448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557cd60eb50 0x5557cd60c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.242+0000 7f2a85d9b700 1 -- 172.21.15.101:0/1209531448 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a881a4cc0 con 0x5557cd603c60 2023-07-18T03:23:35.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.242+0000 7f2a8659c700 1 --2- 172.21.15.101:0/1209531448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557cd603350 0x5557cd60b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:23:35.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.242+0000 7f2a86d9d700 1 --2- 172.21.15.101:0/1209531448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557cd60eb50 0x5557cd60c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:23:35.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.242+0000 7f2a85d9b700 1 --2- 172.21.15.101:0/1209531448 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5557cd603c60 0x5557cd60bde0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x5557cd604150 tx=0x7f2a7c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d754706e73fae3 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:35.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.242+0000 7f2a6dffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:23:35.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.242+0000 7f2a6dffb700 1 -- 172.21.15.101:0/1209531448 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c098250 con 0x5557cd603c60 2023-07-18T03:23:35.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.242+0000 7f2a6dffb700 1 -- 172.21.15.101:0/1209531448 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f2a7c09a320 con 0x5557cd603c60 2023-07-18T03:23:35.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.242+0000 7f2aa06e3b80 1 -- 172.21.15.101:0/1209531448 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5557cd4af570 con 0x5557cd603c60 2023-07-18T03:23:35.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.243+0000 7f2a6dffb700 1 -- 172.21.15.101:0/1209531448 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a7c0b81c0 con 0x5557cd603c60 2023-07-18T03:23:35.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.243+0000 7f2aa06e3b80 10 client.18621 fetch_fsmap learned FSMap version 39 2023-07-18T03:23:35.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.243+0000 7f2aa06e3b80 1 -- 172.21.15.101:0/1209531448 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5557cd36cce0 con 0x5557cd603c60 2023-07-18T03:23:35.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.243+0000 7f2a6dffb700 1 -- 172.21.15.101:0/1209531448 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5557cd4af570 con 0x5557cd603c60 2023-07-18T03:23:35.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.243+0000 7f2a6dffb700 1 -- 172.21.15.101:0/1209531448 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 39) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f2a7c0adbb0 con 0x5557cd603c60 2023-07-18T03:23:35.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.243+0000 7f2aa06e3b80 10 client.18621 fetch_fsmap finished waiting for FSMap version 39 2023-07-18T03:23:35.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.243+0000 7f2aa06e3b80 10 client.18621 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:23:35.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.243+0000 7f2aa06e3b80 20 client.18621 populate_metadata read hostname 'smithi101' 2023-07-18T03:23:35.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.243+0000 7f2aa06e3b80 1 --2- 172.21.15.101:0/1209531448 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5557cd61e100 0x5557cd620510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:35.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.243+0000 7f2aa06e3b80 4 client.18621 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:23:35.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.243+0000 7f2aa06e3b80 1 -- 172.21.15.101:0/1209531448 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5557cd54ef30 con 0x5557cd61e100 2023-07-18T03:23:35.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.244+0000 7f2a8659c700 1 --2- 172.21.15.101:0/1209531448 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5557cd61e100 0x5557cd620510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:35.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.245+0000 7f2a8659c700 1 --2- 172.21.15.101:0/1209531448 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5557cd61e100 0x5557cd620510 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6d329110710ee59 server_cookie=dd7a25f186bcfffa in_seq=0 out_seq=0 2023-07-18T03:23:35.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.245+0000 7f2a6dffb700 10 client.18621 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:23:35.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.246+0000 7f2a6dffb700 1 -- 172.21.15.101:0/1209531448 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5557cd54ef30 con 0x5557cd61e100 2023-07-18T03:23:35.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.246+0000 7f2a6dffb700 10 client.18621 handle_command_reply: tid=0 2023-07-18T03:23:35.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.246+0000 7f2aa06e3b80 1 client.18621 shutdown 2023-07-18T03:23:35.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.247+0000 7f2aa06e3b80 1 -- 172.21.15.101:0/1209531448 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5557cd603c60 msgr2=0x5557cd60bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:35.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.247+0000 7f2aa06e3b80 1 --2- 172.21.15.101:0/1209531448 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5557cd603c60 0x5557cd60bde0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x5557cd604150 tx=0x7f2a7c01f000 comp rx=0 tx=0).stop 2023-07-18T03:23:35.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.247+0000 7f2aa06e3b80 1 -- 172.21.15.101:0/1209531448 shutdown_connections 2023-07-18T03:23:35.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.247+0000 7f2aa06e3b80 1 --2- 172.21.15.101:0/1209531448 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5557cd61e100 0x5557cd620510 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.247+0000 7f2aa06e3b80 1 --2- 172.21.15.101:0/1209531448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557cd60eb50 0x5557cd60c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.247+0000 7f2aa06e3b80 1 --2- 172.21.15.101:0/1209531448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557cd603350 0x5557cd60b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.247+0000 7f2aa06e3b80 1 --2- 172.21.15.101:0/1209531448 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5557cd603c60 0x5557cd60bde0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.247+0000 7f2aa06e3b80 1 -- 172.21.15.101:0/1209531448 >> 172.21.15.101:0/1209531448 conn(0x5557cd5f9e20 msgr2=0x5557cd5ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:35.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.247+0000 7f2a6dffb700 0 client.18621 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:23:35.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.247+0000 7f2aa06e3b80 1 -- 172.21.15.101:0/1209531448 shutdown_connections 2023-07-18T03:23:35.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.247+0000 7f2aa06e3b80 1 -- 172.21.15.101:0/1209531448 wait complete. 2023-07-18T03:23:35.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.248+0000 7f2aa06e3b80 20 client.18621 trim_cache size 0 max 16384 2023-07-18T03:23:35.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.269+0000 7f2aa06e3b80 1 Processor -- start 2023-07-18T03:23:35.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.269+0000 7f2aa06e3b80 1 -- start start 2023-07-18T03:23:35.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.269+0000 7f2aa06e3b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557cd603350 0x5557cd615530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:35.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.269+0000 7f2aa06e3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5557cd603c60 0x5557cd608b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:35.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.269+0000 7f2aa06e3b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557cd60eb50 0x5557cd609050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:35.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.269+0000 7f2aa06e3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557cd61af20 con 0x5557cd603c60 2023-07-18T03:23:35.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.269+0000 7f2aa06e3b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557cd5f7020 con 0x5557cd603350 2023-07-18T03:23:35.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.269+0000 7f2aa06e3b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557cd604180 con 0x5557cd60eb50 2023-07-18T03:23:35.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.270+0000 7f2a8659c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557cd603350 0x5557cd615530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:35.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.270+0000 7f2a8659c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557cd603350 0x5557cd615530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:54444/0 (socket says 172.21.15.101:54444) 2023-07-18T03:23:35.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.270+0000 7f2a8659c700 1 -- 172.21.15.101:0/1970310645 learned_addr learned my addr 172.21.15.101:0/1970310645 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:23:35.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.270+0000 7f2a85d9b700 1 --2- 172.21.15.101:0/1970310645 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5557cd603c60 0x5557cd608b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:35.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.270+0000 7f2a86d9d700 1 --2- 172.21.15.101:0/1970310645 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557cd60eb50 0x5557cd609050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:35.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.270+0000 7f2a8659c700 1 -- 172.21.15.101:0/1970310645 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557cd60eb50 msgr2=0x5557cd609050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:35.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.270+0000 7f2a8659c700 1 --2- 172.21.15.101:0/1970310645 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557cd60eb50 0x5557cd609050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.270+0000 7f2a8659c700 1 -- 172.21.15.101:0/1970310645 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5557cd603c60 msgr2=0x5557cd608b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:35.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.270+0000 7f2a8659c700 1 --2- 172.21.15.101:0/1970310645 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5557cd603c60 0x5557cd608b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.270+0000 7f2a8659c700 1 -- 172.21.15.101:0/1970310645 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557cd3aa2d0 con 0x5557cd603350 2023-07-18T03:23:35.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.270+0000 7f2a8659c700 1 --2- 172.21.15.101:0/1970310645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557cd603350 0x5557cd615530 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f2a78011780 tx=0x7f2a78008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3001abe83b741d9a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:35.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.270+0000 7f2a757fa700 1 -- 172.21.15.101:0/1970310645 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7801e460 con 0x5557cd603350 2023-07-18T03:23:35.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.271+0000 7f2a757fa700 1 -- 172.21.15.101:0/1970310645 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a7801e5c0 con 0x5557cd603350 2023-07-18T03:23:35.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.271+0000 7f2a757fa700 1 -- 172.21.15.101:0/1970310645 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a78016e60 con 0x5557cd603350 2023-07-18T03:23:35.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.271+0000 7f2aa06e3b80 1 -- 172.21.15.101:0/1970310645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557cd603350 msgr2=0x5557cd615530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:35.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.271+0000 7f2aa06e3b80 1 --2- 172.21.15.101:0/1970310645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557cd603350 0x5557cd615530 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f2a78011780 tx=0x7f2a78008040 comp rx=0 tx=0).stop 2023-07-18T03:23:35.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.271+0000 7f2aa06e3b80 1 -- 172.21.15.101:0/1970310645 shutdown_connections 2023-07-18T03:23:35.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.271+0000 7f2aa06e3b80 1 --2- 172.21.15.101:0/1970310645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557cd603350 0x5557cd615530 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.271+0000 7f2aa06e3b80 1 --2- 172.21.15.101:0/1970310645 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557cd60eb50 0x5557cd609050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.271+0000 7f2aa06e3b80 1 --2- 172.21.15.101:0/1970310645 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5557cd603c60 0x5557cd608b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.271+0000 7f2aa06e3b80 1 -- 172.21.15.101:0/1970310645 >> 172.21.15.101:0/1970310645 conn(0x5557cd5f9e20 msgr2=0x5557cd60be10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:35.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.271+0000 7f2aa06e3b80 1 -- 172.21.15.101:0/1970310645 shutdown_connections 2023-07-18T03:23:35.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.271+0000 7f2aa06e3b80 1 -- 172.21.15.101:0/1970310645 wait complete. 2023-07-18T03:23:35.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.272+0000 7f2aa06e3b80 1 -- start start 2023-07-18T03:23:35.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.272+0000 7f2aa06e3b80 1 Processor -- start 2023-07-18T03:23:35.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.273+0000 7f2aa06e3b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557cd603350 0x5557cd666140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:35.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.273+0000 7f2aa06e3b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557cd603c60 0x5557cd6666e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:35.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.273+0000 7f2aa06e3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5557cd60eb50 0x5557cd66a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:35.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.273+0000 7f2aa06e3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557cd66a840 con 0x5557cd60eb50 2023-07-18T03:23:35.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.273+0000 7f2aa06e3b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557cd66a9b0 con 0x5557cd603350 2023-07-18T03:23:35.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.273+0000 7f2aa06e3b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557cd66ab70 con 0x5557cd603c60 2023-07-18T03:23:35.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.273+0000 7f2aa06e3b80 10 client.0 _ll_register_callbacks cb 0x7f2a780098d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:23:35.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.273+0000 7f2a8659c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557cd603350 0x5557cd666140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:35.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.273+0000 7f2a8659c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557cd603350 0x5557cd666140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:54454/0 (socket says 172.21.15.101:54454) 2023-07-18T03:23:35.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.273+0000 7f2a8659c700 1 -- 172.21.15.101:0/4115995037 learned_addr learned my addr 172.21.15.101:0/4115995037 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:23:35.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.273+0000 7f2a85d9b700 1 --2- 172.21.15.101:0/4115995037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557cd603c60 0x5557cd6666e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:35.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.273+0000 7f2a86d9d700 1 --2- 172.21.15.101:0/4115995037 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5557cd60eb50 0x5557cd66a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:35.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.273+0000 7f2a8659c700 1 -- 172.21.15.101:0/4115995037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557cd603c60 msgr2=0x5557cd6666e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:35.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.273+0000 7f2a8659c700 1 --2- 172.21.15.101:0/4115995037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557cd603c60 0x5557cd6666e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.273+0000 7f2a8659c700 1 -- 172.21.15.101:0/4115995037 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5557cd60eb50 msgr2=0x5557cd66a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:35.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.273+0000 7f2a8659c700 1 --2- 172.21.15.101:0/4115995037 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5557cd60eb50 0x5557cd66a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.273+0000 7f2a8659c700 1 -- 172.21.15.101:0/4115995037 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a78009bd0 con 0x5557cd603350 2023-07-18T03:23:35.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.273+0000 7f2a85d9b700 1 --2- 172.21.15.101:0/4115995037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557cd603c60 0x5557cd6666e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:23:35.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.274+0000 7f2a8659c700 1 --2- 172.21.15.101:0/4115995037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557cd603350 0x5557cd666140 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f2a7801e750 tx=0x7f2a7801e460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1eb7c86aecaf1117 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:35.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.274+0000 7f2a6dffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:23:35.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.274+0000 7f2a6dffb700 1 -- 172.21.15.101:0/4115995037 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a78011c90 con 0x5557cd603350 2023-07-18T03:23:35.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.274+0000 7f2aa06e3b80 1 -- 172.21.15.101:0/4115995037 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a7c0a2300 con 0x5557cd603350 2023-07-18T03:23:35.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.274+0000 7f2a6dffb700 1 -- 172.21.15.101:0/4115995037 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f2a7801b040 con 0x5557cd603350 2023-07-18T03:23:35.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.278+0000 7f2a6dffb700 1 -- 172.21.15.101:0/4115995037 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a78011df0 con 0x5557cd603350 2023-07-18T03:23:35.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.278+0000 7f2aa06e3b80 10 client.28715 fetch_fsmap learned FSMap version 39 2023-07-18T03:23:35.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.278+0000 7f2aa06e3b80 1 -- 172.21.15.101:0/4115995037 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5557cd382330 con 0x5557cd603350 2023-07-18T03:23:35.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.279+0000 7f2a6dffb700 1 -- 172.21.15.101:0/4115995037 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a7c0a2300 con 0x5557cd603350 2023-07-18T03:23:35.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.279+0000 7f2a6dffb700 1 -- 172.21.15.101:0/4115995037 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 39) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f2a78006160 con 0x5557cd603350 2023-07-18T03:23:35.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.279+0000 7f2aa06e3b80 10 client.28715 fetch_fsmap finished waiting for FSMap version 39 2023-07-18T03:23:35.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.279+0000 7f2aa06e3b80 10 client.28715 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:23:35.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.279+0000 7f2aa06e3b80 20 client.28715 populate_metadata read hostname 'smithi101' 2023-07-18T03:23:35.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.279+0000 7f2aa06e3b80 1 --2- 172.21.15.101:0/4115995037 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5557cd6734f0 0x5557cd675900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:35.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.279+0000 7f2aa06e3b80 4 client.28715 mds_command: new command op to 24562 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-18T03:23:35.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.279+0000 7f2aa06e3b80 1 -- 172.21.15.101:0/4115995037 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5557cd5110d0 con 0x5557cd6734f0 2023-07-18T03:23:35.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.279+0000 7f2a85d9b700 1 --2- 172.21.15.101:0/4115995037 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5557cd6734f0 0x5557cd675900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:35.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.280+0000 7f2a85d9b700 1 --2- 172.21.15.101:0/4115995037 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5557cd6734f0 0x5557cd675900 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=247a83213c847341 server_cookie=c89ab81c28f2ea5a in_seq=0 out_seq=0 2023-07-18T03:23:35.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.280+0000 7f2a6dffb700 10 client.28715 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:23:35.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.289+0000 7f2a6dffb700 1 -- 172.21.15.101:0/4115995037 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5557cd5110d0 con 0x5557cd6734f0 2023-07-18T03:23:35.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.289+0000 7f2a6dffb700 10 client.28715 handle_command_reply: tid=0 2023-07-18T03:23:35.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.289+0000 7f2aa06e3b80 1 client.28715 shutdown 2023-07-18T03:23:35.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.289+0000 7f2aa06e3b80 1 -- 172.21.15.101:0/4115995037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557cd603350 msgr2=0x5557cd666140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:35.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.289+0000 7f2aa06e3b80 1 --2- 172.21.15.101:0/4115995037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557cd603350 0x5557cd666140 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f2a7801e750 tx=0x7f2a7801e460 comp rx=0 tx=0).stop 2023-07-18T03:23:35.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.289+0000 7f2aa06e3b80 1 -- 172.21.15.101:0/4115995037 shutdown_connections 2023-07-18T03:23:35.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.289+0000 7f2aa06e3b80 1 --2- 172.21.15.101:0/4115995037 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5557cd6734f0 0x5557cd675900 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.289+0000 7f2aa06e3b80 1 --2- 172.21.15.101:0/4115995037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5557cd603350 0x5557cd666140 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.289+0000 7f2aa06e3b80 1 --2- 172.21.15.101:0/4115995037 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5557cd60eb50 0x5557cd66a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.289+0000 7f2a6dffb700 0 client.28715 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:23:35.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.289+0000 7f2aa06e3b80 1 --2- 172.21.15.101:0/4115995037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557cd603c60 0x5557cd6666e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.289+0000 7f2aa06e3b80 1 -- 172.21.15.101:0/4115995037 >> 172.21.15.101:0/4115995037 conn(0x5557cd5f9e20 msgr2=0x5557cd60d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:35.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.289+0000 7f2aa06e3b80 1 -- 172.21.15.101:0/4115995037 shutdown_connections 2023-07-18T03:23:35.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.289+0000 7f2aa06e3b80 1 -- 172.21.15.101:0/4115995037 wait complete. 2023-07-18T03:23:35.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.289+0000 7f2aa06e3b80 20 client.28715 trim_cache size 0 max 16384 2023-07-18T03:23:35.288 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:23:35.288 INFO:teuthology.orchestra.run.smithi101.stdout: "return_code": 0, 2023-07-18T03:23:35.289 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_tag": "4a948d03-674f-4ebf-a4b7-84a3c2f27a18", 2023-07-18T03:23:35.289 INFO:teuthology.orchestra.run.smithi101.stdout: "mode": "asynchronous" 2023-07-18T03:23:35.289 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:23:35.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.290+0000 7f2a8cdff700 1 -- 172.21.15.101:0/2509321594 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2a700a1340 msgr2=0x7f2a700a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:35.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.290+0000 7f2a8cdff700 1 --2- 172.21.15.101:0/2509321594 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2a700a1340 0x7f2a700a37f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2a88074920 tx=0x7f2a80006230 comp rx=0 tx=0).stop 2023-07-18T03:23:35.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.290+0000 7f2a8cdff700 1 -- 172.21.15.101:0/2509321594 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2a88100480 msgr2=0x7f2a88072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:35.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.290+0000 7f2a8cdff700 1 --2- 172.21.15.101:0/2509321594 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2a88100480 0x7f2a88072d50 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00e9f0 tx=0x7f2a7c00ed00 comp rx=0 tx=0).stop 2023-07-18T03:23:35.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.291+0000 7f2a8cdff700 1 -- 172.21.15.101:0/2509321594 shutdown_connections 2023-07-18T03:23:35.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.291+0000 7f2a8cdff700 1 --2- 172.21.15.101:0/2509321594 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2a700a1340 0x7f2a700a37f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.291+0000 7f2a8cdff700 1 --2- 172.21.15.101:0/2509321594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a88105ae0 0x7f2a881a3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.291+0000 7f2a8cdff700 1 --2- 172.21.15.101:0/2509321594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a880ffad0 0x7f2a88072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.291+0000 7f2a8cdff700 1 --2- 172.21.15.101:0/2509321594 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2a88100480 0x7f2a88072d50 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.291+0000 7f2a8cdff700 1 -- 172.21.15.101:0/2509321594 >> 172.21.15.101:0/2509321594 conn(0x7f2a880fb5f0 msgr2=0x7f2a881035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:35.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.291+0000 7f2a8cdff700 1 -- 172.21.15.101:0/2509321594 shutdown_connections 2023-07-18T03:23:35.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.291+0000 7f2a8cdff700 1 -- 172.21.15.101:0/2509321594 wait complete. 2023-07-18T03:23:35.305 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:35 smithi101 ceph-mon[126379]: pgmap v503: 129 pgs: 129 active+clean; 1.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 14 MiB/s wr, 608 op/s 2023-07-18T03:23:35.307 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:23:35.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:35 smithi155 ceph-mon[128885]: pgmap v503: 129 pgs: 129 active+clean; 1.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 14 MiB/s wr, 608 op/s 2023-07-18T03:23:35.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.636+0000 7f64865ff700 1 -- 172.21.15.101:0/3641470001 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f648010e280 msgr2=0x7f648010e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:35.633 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.636+0000 7f64865ff700 1 --2- 172.21.15.101:0/3641470001 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f648010e280 0x7f648010e650 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f647001a9a0 tx=0x7f647001acb0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.643 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.642+0000 7f64865ff700 1 -- 172.21.15.101:0/3641470001 shutdown_connections 2023-07-18T03:23:35.644 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.642+0000 7f64865ff700 1 --2- 172.21.15.101:0/3641470001 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f648010e280 0x7f648010e650 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.644 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.642+0000 7f64865ff700 1 --2- 172.21.15.101:0/3641470001 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f648006d3a0 0x7f648011b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.644 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.642+0000 7f64865ff700 1 --2- 172.21.15.101:0/3641470001 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f648006c9a0 0x7f648006ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.644 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.642+0000 7f64865ff700 1 -- 172.21.15.101:0/3641470001 >> 172.21.15.101:0/3641470001 conn(0x7f648006bf70 msgr2=0x7f6480070ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:35.644 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.644+0000 7f64865ff700 1 -- 172.21.15.101:0/3641470001 shutdown_connections 2023-07-18T03:23:35.644 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.644+0000 7f64865ff700 1 -- 172.21.15.101:0/3641470001 wait complete. 2023-07-18T03:23:35.645 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.645+0000 7f64865ff700 1 Processor -- start 2023-07-18T03:23:35.645 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.646+0000 7f64865ff700 1 -- start start 2023-07-18T03:23:35.645 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.647+0000 7f64865ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f648006c9a0 0x7f6480116da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:35.645 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.647+0000 7f64865ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f648006d3a0 0x7f64801172e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:35.645 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.647+0000 7f64865ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f648010e280 0x7f6480073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:35.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.647+0000 7f64865ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6480073d20 con 0x7f648006c9a0 2023-07-18T03:23:35.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.647+0000 7f64865ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6480073e90 con 0x7f648006d3a0 2023-07-18T03:23:35.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.647+0000 7f64865ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6480074050 con 0x7f648010e280 2023-07-18T03:23:35.647 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.647+0000 7f647ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f648006c9a0 0x7f6480116da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:35.647 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.648+0000 7f647ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f648006c9a0 0x7f6480116da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:39930/0 (socket says 172.21.15.101:39930) 2023-07-18T03:23:35.647 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.648+0000 7f647ffff700 1 -- 172.21.15.101:0/2843934910 learned_addr learned my addr 172.21.15.101:0/2843934910 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:23:35.647 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.648+0000 7f6484b9c700 1 --2- 172.21.15.101:0/2843934910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f648010e280 0x7f6480073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:35.648 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.648+0000 7f647ffff700 1 -- 172.21.15.101:0/2843934910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f648010e280 msgr2=0x7f6480073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:35.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.648+0000 7f647ffff700 1 --2- 172.21.15.101:0/2843934910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f648010e280 0x7f6480073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.648+0000 7f647ffff700 1 -- 172.21.15.101:0/2843934910 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f648006d3a0 msgr2=0x7f64801172e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:35.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.648+0000 7f647ffff700 1 --2- 172.21.15.101:0/2843934910 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f648006d3a0 0x7f64801172e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.648+0000 7f647ffff700 1 -- 172.21.15.101:0/2843934910 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f647001a600 con 0x7f648006c9a0 2023-07-18T03:23:35.649 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.649+0000 7f647ffff700 1 --2- 172.21.15.101:0/2843934910 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f648006c9a0 0x7f6480116da0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f6470005910 tx=0x7f647001f6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:35.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.649+0000 7f647d7fa700 1 -- 172.21.15.101:0/2843934910 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647001fdc0 con 0x7f648006c9a0 2023-07-18T03:23:35.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.649+0000 7f647d7fa700 1 -- 172.21.15.101:0/2843934910 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6470003680 con 0x7f648006c9a0 2023-07-18T03:23:35.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.649+0000 7f647d7fa700 1 -- 172.21.15.101:0/2843934910 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647000e760 con 0x7f648006c9a0 2023-07-18T03:23:35.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.649+0000 7f64865ff700 1 -- 172.21.15.101:0/2843934910 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64800742d0 con 0x7f648006c9a0 2023-07-18T03:23:35.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.649+0000 7f64865ff700 1 -- 172.21.15.101:0/2843934910 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f648007a820 con 0x7f648006c9a0 2023-07-18T03:23:35.654 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.657+0000 7f6499f77b80 1 Processor -- start 2023-07-18T03:23:35.654 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.657+0000 7f6499f77b80 1 -- start start 2023-07-18T03:23:35.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.658+0000 7f6499f77b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56499a7ff2d0 0x56499a7ff6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:35.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.658+0000 7f6499f77b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56499a7ffbe0 0x56499a80a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:35.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.658+0000 7f6499f77b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56499a80aad0 0x56499a80ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:35.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.658+0000 7f6499f77b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56499a705cb0 con 0x56499a7ff2d0 2023-07-18T03:23:35.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.658+0000 7f6499f77b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56499a758a40 con 0x56499a7ffbe0 2023-07-18T03:23:35.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.658+0000 7f6499f77b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56499a751ed0 con 0x56499a80aad0 2023-07-18T03:23:35.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.659+0000 7f6484b9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56499a7ff2d0 0x56499a7ff6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:35.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.659+0000 7f6484b9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56499a7ff2d0 0x56499a7ff6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:39942/0 (socket says 172.21.15.101:39942) 2023-07-18T03:23:35.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.659+0000 7f6484b9c700 1 -- 172.21.15.101:0/330240921 learned_addr learned my addr 172.21.15.101:0/330240921 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:23:35.657 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.659+0000 7f647d7fa700 1 -- 172.21.15.101:0/2843934910 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f647000e8c0 con 0x7f648006c9a0 2023-07-18T03:23:35.657 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.659+0000 7f647f7fe700 1 --2- 172.21.15.101:0/330240921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56499a80aad0 0x56499a80ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:35.657 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.659+0000 7f6484b9c700 1 -- 172.21.15.101:0/330240921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56499a80aad0 msgr2=0x56499a80ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:35.658 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.659+0000 7f6484b9c700 1 --2- 172.21.15.101:0/330240921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56499a80aad0 0x56499a80ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.658 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.659+0000 7f6484b9c700 1 -- 172.21.15.101:0/330240921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56499a7ffbe0 msgr2=0x56499a80a590 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-18T03:23:35.658 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.659+0000 7f6484b9c700 1 --2- 172.21.15.101:0/330240921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56499a7ffbe0 0x56499a80a590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.658 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.659+0000 7f6484b9c700 1 -- 172.21.15.101:0/330240921 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56499a57fde0 con 0x56499a7ff2d0 2023-07-18T03:23:35.658 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.660+0000 7f647f7fe700 1 --2- 172.21.15.101:0/330240921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56499a80aad0 0x56499a80ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:23:35.658 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.660+0000 7f6484b9c700 1 --2- 172.21.15.101:0/330240921 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56499a7ff2d0 0x56499a7ff6a0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f6478009440 tx=0x7f6478009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2e4ba79218c8641 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:35.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.661+0000 7f647d7fa700 1 --2- 172.21.15.101:0/2843934910 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f64680a13c0 0x7f64680a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:35.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.662+0000 7f647d7fa700 1 -- 172.21.15.101:0/2843934910 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f64700ce390 con 0x7f648006c9a0 2023-07-18T03:23:35.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.663+0000 7f647ffff700 1 --2- 172.21.15.101:0/2843934910 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f64680a13c0 0x7f64680a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:35.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.663+0000 7f64667fc700 1 -- 172.21.15.101:0/330240921 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647800e070 con 0x56499a7ff2d0 2023-07-18T03:23:35.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.663+0000 7f64667fc700 1 -- 172.21.15.101:0/330240921 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f647800f040 con 0x56499a7ff2d0 2023-07-18T03:23:35.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.663+0000 7f64667fc700 1 -- 172.21.15.101:0/330240921 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6478015b90 con 0x56499a7ff2d0 2023-07-18T03:23:35.662 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.664+0000 7f6499f77b80 1 -- 172.21.15.101:0/330240921 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56499a7ff2d0 msgr2=0x56499a7ff6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:35.662 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.664+0000 7f6499f77b80 1 --2- 172.21.15.101:0/330240921 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56499a7ff2d0 0x56499a7ff6a0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f6478009440 tx=0x7f6478009920 comp rx=0 tx=0).stop 2023-07-18T03:23:35.662 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.665+0000 7f647ffff700 1 --2- 172.21.15.101:0/2843934910 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f64680a13c0 0x7f64680a3870 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f647009b070 tx=0x7f6470018000 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:35.663 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.666+0000 7f6499f77b80 1 -- 172.21.15.101:0/330240921 shutdown_connections 2023-07-18T03:23:35.663 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.666+0000 7f6499f77b80 1 --2- 172.21.15.101:0/330240921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56499a80aad0 0x56499a80ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.663 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.666+0000 7f6499f77b80 1 --2- 172.21.15.101:0/330240921 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56499a7ff2d0 0x56499a7ff6a0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.663 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.666+0000 7f6499f77b80 1 --2- 172.21.15.101:0/330240921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56499a7ffbe0 0x56499a80a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.667+0000 7f6499f77b80 1 -- 172.21.15.101:0/330240921 >> 172.21.15.101:0/330240921 conn(0x56499a7f5da0 msgr2=0x56499a7fb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:35.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.668+0000 7f6499f77b80 1 -- 172.21.15.101:0/330240921 shutdown_connections 2023-07-18T03:23:35.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.668+0000 7f6499f77b80 1 -- 172.21.15.101:0/330240921 wait complete. 2023-07-18T03:23:35.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.669+0000 7f6499f77b80 1 -- start start 2023-07-18T03:23:35.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.669+0000 7f6499f77b80 1 Processor -- start 2023-07-18T03:23:35.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.670+0000 7f6499f77b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56499a7ff2d0 0x56499a80a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:35.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.670+0000 7f6499f77b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56499a7ffbe0 0x56499a8077d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:35.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.670+0000 7f6499f77b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56499a80aad0 0x56499a807e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:35.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.670+0000 7f6499f77b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56499a6f1ac0 con 0x56499a80aad0 2023-07-18T03:23:35.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.670+0000 7f6499f77b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56499a6f17c0 con 0x56499a7ff2d0 2023-07-18T03:23:35.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.670+0000 7f6499f77b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56499a5d5920 con 0x56499a7ffbe0 2023-07-18T03:23:35.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.670+0000 7f6499f77b80 10 client.0 _ll_register_callbacks cb 0x56499a6ef760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:23:35.668 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.671+0000 7f647f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56499a7ffbe0 0x56499a8077d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:35.668 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.671+0000 7f647f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56499a7ffbe0 0x56499a8077d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:42014/0 (socket says 172.21.15.101:42014) 2023-07-18T03:23:35.668 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.671+0000 7f647f7fe700 1 -- 172.21.15.101:0/531230662 learned_addr learned my addr 172.21.15.101:0/531230662 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:23:35.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.672+0000 7f6484b9c700 1 --2- 172.21.15.101:0/531230662 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56499a80aad0 0x56499a807e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:35.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.673+0000 7f6484b9c700 1 --2- 172.21.15.101:0/531230662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56499a7ff2d0 0x56499a80a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:35.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.673+0000 7f647f7fe700 1 -- 172.21.15.101:0/531230662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56499a7ff2d0 msgr2=0x56499a80a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:35.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.673+0000 7f647f7fe700 1 --2- 172.21.15.101:0/531230662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56499a7ff2d0 0x56499a80a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.673+0000 7f647f7fe700 1 -- 172.21.15.101:0/531230662 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56499a80aad0 msgr2=0x56499a807e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:35.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.673+0000 7f647f7fe700 1 --2- 172.21.15.101:0/531230662 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56499a80aad0 0x56499a807e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.673+0000 7f647f7fe700 1 -- 172.21.15.101:0/531230662 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64780090f0 con 0x56499a7ffbe0 2023-07-18T03:23:35.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.674+0000 7f647f7fe700 1 --2- 172.21.15.101:0/531230662 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56499a7ffbe0 0x56499a8077d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f647400c250 tx=0x7f647400c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f01aa60f8ab1b767 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:35.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.674+0000 7f645f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:23:35.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.674+0000 7f645f7fe700 1 -- 172.21.15.101:0/531230662 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647400cea0 con 0x56499a7ffbe0 2023-07-18T03:23:35.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.674+0000 7f645f7fe700 1 -- 172.21.15.101:0/531230662 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f64740115f0 con 0x56499a7ffbe0 2023-07-18T03:23:35.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.674+0000 7f6499f77b80 1 -- 172.21.15.101:0/531230662 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56499a6ab4f0 con 0x56499a7ffbe0 2023-07-18T03:23:35.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.676+0000 7f645f7fe700 1 -- 172.21.15.101:0/531230662 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64740117b0 con 0x56499a7ffbe0 2023-07-18T03:23:35.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.676+0000 7f6499f77b80 10 client.28732 fetch_fsmap learned FSMap version 39 2023-07-18T03:23:35.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.676+0000 7f6499f77b80 1 -- 172.21.15.101:0/531230662 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x56499a568c60 con 0x56499a7ffbe0 2023-07-18T03:23:35.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.677+0000 7f645f7fe700 1 -- 172.21.15.101:0/531230662 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56499a6ab4f0 con 0x56499a7ffbe0 2023-07-18T03:23:35.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.677+0000 7f645f7fe700 1 -- 172.21.15.101:0/531230662 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 39) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f647401aca0 con 0x56499a7ffbe0 2023-07-18T03:23:35.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.677+0000 7f6499f77b80 10 client.28732 fetch_fsmap finished waiting for FSMap version 39 2023-07-18T03:23:35.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.677+0000 7f6499f77b80 10 client.28732 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:23:35.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.677+0000 7f6499f77b80 20 client.28732 populate_metadata read hostname 'smithi101' 2023-07-18T03:23:35.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.677+0000 7f6499f77b80 1 --2- 172.21.15.101:0/531230662 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56499a81a520 0x56499a81c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:35.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.677+0000 7f6499f77b80 4 client.28732 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:23:35.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.677+0000 7f6499f77b80 1 -- 172.21.15.101:0/531230662 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56499a74aeb0 con 0x56499a81a520 2023-07-18T03:23:35.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.677+0000 7f6484b9c700 1 --2- 172.21.15.101:0/531230662 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56499a81a520 0x56499a81c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:35.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.678+0000 7f645f7fe700 10 client.28732 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:23:35.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.678+0000 7f6484b9c700 1 --2- 172.21.15.101:0/531230662 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56499a81a520 0x56499a81c930 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7e59438833981d6 server_cookie=e56234608f3442d2 in_seq=0 out_seq=0 2023-07-18T03:23:35.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.682+0000 7f645f7fe700 1 -- 172.21.15.101:0/531230662 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56499a74aeb0 con 0x56499a81a520 2023-07-18T03:23:35.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.682+0000 7f645f7fe700 10 client.28732 handle_command_reply: tid=0 2023-07-18T03:23:35.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.682+0000 7f6499f77b80 1 client.28732 shutdown 2023-07-18T03:23:35.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.682+0000 7f6499f77b80 1 -- 172.21.15.101:0/531230662 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56499a7ffbe0 msgr2=0x56499a8077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:35.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.682+0000 7f6499f77b80 1 --2- 172.21.15.101:0/531230662 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56499a7ffbe0 0x56499a8077d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f647400c250 tx=0x7f647400c560 comp rx=0 tx=0).stop 2023-07-18T03:23:35.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.682+0000 7f6499f77b80 1 -- 172.21.15.101:0/531230662 shutdown_connections 2023-07-18T03:23:35.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.682+0000 7f6499f77b80 1 --2- 172.21.15.101:0/531230662 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56499a81a520 0x56499a81c930 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.682+0000 7f6499f77b80 1 --2- 172.21.15.101:0/531230662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56499a7ff2d0 0x56499a80a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.682+0000 7f6499f77b80 1 --2- 172.21.15.101:0/531230662 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56499a80aad0 0x56499a807e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.682+0000 7f6499f77b80 1 --2- 172.21.15.101:0/531230662 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56499a7ffbe0 0x56499a8077d0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.682+0000 7f6499f77b80 1 -- 172.21.15.101:0/531230662 >> 172.21.15.101:0/531230662 conn(0x56499a7f5da0 msgr2=0x56499a7fb290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:35.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.682+0000 7f645f7fe700 0 client.28732 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:23:35.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.683+0000 7f6499f77b80 1 -- 172.21.15.101:0/531230662 shutdown_connections 2023-07-18T03:23:35.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.683+0000 7f6499f77b80 1 -- 172.21.15.101:0/531230662 wait complete. 2023-07-18T03:23:35.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.684+0000 7f6499f77b80 20 client.28732 trim_cache size 0 max 16384 2023-07-18T03:23:35.705 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:35 smithi179 ceph-mon[129229]: pgmap v503: 129 pgs: 129 active+clean; 1.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 14 MiB/s wr, 608 op/s 2023-07-18T03:23:35.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.715+0000 7f6499f77b80 1 Processor -- start 2023-07-18T03:23:35.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.717+0000 7f6499f77b80 1 -- start start 2023-07-18T03:23:35.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.717+0000 7f6499f77b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56499a7ff2d0 0x56499a802ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:35.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.718+0000 7f6499f77b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56499a7ffbe0 0x56499a803010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:35.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.718+0000 7f6499f77b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56499a80aad0 0x56499a806f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:35.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.718+0000 7f6499f77b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56499a7f2fa0 con 0x56499a7ff2d0 2023-07-18T03:23:35.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.718+0000 7f6499f77b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56499a7f5970 con 0x56499a7ffbe0 2023-07-18T03:23:35.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.718+0000 7f6499f77b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56499a7fc4d0 con 0x56499a80aad0 2023-07-18T03:23:35.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.720+0000 7f647f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56499a7ffbe0 0x56499a803010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:35.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.720+0000 7f647f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56499a7ffbe0 0x56499a803010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:54508/0 (socket says 172.21.15.101:54508) 2023-07-18T03:23:35.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.720+0000 7f647f7fe700 1 -- 172.21.15.101:0/3645399905 learned_addr learned my addr 172.21.15.101:0/3645399905 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:23:35.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.721+0000 7f6484b9c700 1 --2- 172.21.15.101:0/3645399905 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56499a7ff2d0 0x56499a802ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:35.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.721+0000 7f6484b9c700 1 --2- 172.21.15.101:0/3645399905 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56499a80aad0 0x56499a806f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:35.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.722+0000 7f647f7fe700 1 -- 172.21.15.101:0/3645399905 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56499a80aad0 msgr2=0x56499a806f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:35.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.722+0000 7f647f7fe700 1 --2- 172.21.15.101:0/3645399905 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56499a80aad0 0x56499a806f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.722+0000 7f647f7fe700 1 -- 172.21.15.101:0/3645399905 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56499a7ff2d0 msgr2=0x56499a802ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:35.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.722+0000 7f647f7fe700 1 --2- 172.21.15.101:0/3645399905 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56499a7ff2d0 0x56499a802ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.722+0000 7f647f7fe700 1 -- 172.21.15.101:0/3645399905 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56499a5a6250 con 0x56499a7ffbe0 2023-07-18T03:23:35.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.723+0000 7f647f7fe700 1 --2- 172.21.15.101:0/3645399905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56499a7ffbe0 0x56499a803010 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f647400c220 tx=0x7f647400ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b50525e1f5f9f7e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:35.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.725+0000 7f6466ffd700 1 -- 172.21.15.101:0/3645399905 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647400cea0 con 0x56499a7ffbe0 2023-07-18T03:23:35.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.725+0000 7f6466ffd700 1 -- 172.21.15.101:0/3645399905 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64740196a0 con 0x56499a7ffbe0 2023-07-18T03:23:35.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.725+0000 7f6466ffd700 1 -- 172.21.15.101:0/3645399905 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647401aba0 con 0x56499a7ffbe0 2023-07-18T03:23:35.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.728+0000 7f6499f77b80 1 -- 172.21.15.101:0/3645399905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56499a7ffbe0 msgr2=0x56499a803010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:35.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.728+0000 7f6499f77b80 1 --2- 172.21.15.101:0/3645399905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56499a7ffbe0 0x56499a803010 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f647400c220 tx=0x7f647400ca10 comp rx=0 tx=0).stop 2023-07-18T03:23:35.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.729+0000 7f6499f77b80 1 -- 172.21.15.101:0/3645399905 shutdown_connections 2023-07-18T03:23:35.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.729+0000 7f6499f77b80 1 --2- 172.21.15.101:0/3645399905 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56499a80aad0 0x56499a806f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.729+0000 7f6499f77b80 1 --2- 172.21.15.101:0/3645399905 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56499a7ff2d0 0x56499a802ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.729+0000 7f6499f77b80 1 --2- 172.21.15.101:0/3645399905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56499a7ffbe0 0x56499a803010 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.729+0000 7f6499f77b80 1 -- 172.21.15.101:0/3645399905 >> 172.21.15.101:0/3645399905 conn(0x56499a7f5da0 msgr2=0x56499a816810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:35.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.731+0000 7f6499f77b80 1 -- 172.21.15.101:0/3645399905 shutdown_connections 2023-07-18T03:23:35.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.733+0000 7f6499f77b80 1 -- 172.21.15.101:0/3645399905 wait complete. 2023-07-18T03:23:35.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.734+0000 7f6499f77b80 1 -- start start 2023-07-18T03:23:35.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.734+0000 7f6499f77b80 1 Processor -- start 2023-07-18T03:23:35.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.735+0000 7f6499f77b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56499a7ff2d0 0x56499a8117c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:35.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.735+0000 7f6499f77b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56499a7ffbe0 0x56499a80fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:35.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.735+0000 7f6499f77b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56499a80aad0 0x56499a8102e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:35.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.735+0000 7f6499f77b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56499a8109b0 con 0x56499a7ffbe0 2023-07-18T03:23:35.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.735+0000 7f6499f77b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56499a86e3f0 con 0x56499a7ff2d0 2023-07-18T03:23:35.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.735+0000 7f6499f77b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56499a86e530 con 0x56499a80aad0 2023-07-18T03:23:35.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.735+0000 7f6499f77b80 10 client.0 _ll_register_callbacks cb 0x7f647400a900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:23:35.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.736+0000 7f647f7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56499a7ffbe0 0x56499a80fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:35.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.736+0000 7f647f7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56499a7ffbe0 0x56499a80fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:39974/0 (socket says 172.21.15.101:39974) 2023-07-18T03:23:35.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.736+0000 7f647f7fe700 1 -- 172.21.15.101:0/1412149532 learned_addr learned my addr 172.21.15.101:0/1412149532 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:23:35.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.736+0000 7f6484b9c700 1 --2- 172.21.15.101:0/1412149532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56499a80aad0 0x56499a8102e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:35.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.737+0000 7f6484b9c700 1 -- 172.21.15.101:0/1412149532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56499a7ff2d0 msgr2=0x56499a8117c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:35.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.737+0000 7f6484b9c700 1 --2- 172.21.15.101:0/1412149532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56499a7ff2d0 0x56499a8117c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.737+0000 7f6484b9c700 1 -- 172.21.15.101:0/1412149532 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56499a7ffbe0 msgr2=0x56499a80fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:23:35.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.737+0000 7f6484b9c700 1 --2- 172.21.15.101:0/1412149532 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56499a7ffbe0 0x56499a80fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.737+0000 7f6484b9c700 1 -- 172.21.15.101:0/1412149532 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56499a568c60 con 0x56499a80aad0 2023-07-18T03:23:35.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.738+0000 7f6484b9c700 1 --2- 172.21.15.101:0/1412149532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56499a80aad0 0x56499a8102e0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x56499a81d240 tx=0x7f6478015650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e01649fc2e5049f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:35.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.738+0000 7f645f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:23:35.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.738+0000 7f645f7fe700 1 -- 172.21.15.101:0/1412149532 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647802d070 con 0x56499a80aad0 2023-07-18T03:23:35.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.738+0000 7f645f7fe700 1 -- 172.21.15.101:0/1412149532 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f6478015da0 con 0x56499a80aad0 2023-07-18T03:23:35.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.739+0000 7f6499f77b80 1 -- 172.21.15.101:0/1412149532 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f647800f040 con 0x56499a80aad0 2023-07-18T03:23:35.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.741+0000 7f645f7fe700 1 -- 172.21.15.101:0/1412149532 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f647800e040 con 0x56499a80aad0 2023-07-18T03:23:35.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.741+0000 7f6499f77b80 10 client.28738 fetch_fsmap learned FSMap version 39 2023-07-18T03:23:35.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.741+0000 7f6499f77b80 1 -- 172.21.15.101:0/1412149532 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x56499a57e2b0 con 0x56499a80aad0 2023-07-18T03:23:35.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.742+0000 7f645f7fe700 1 -- 172.21.15.101:0/1412149532 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f647800f040 con 0x56499a80aad0 2023-07-18T03:23:35.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.742+0000 7f645f7fe700 1 -- 172.21.15.101:0/1412149532 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 39) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f6478030050 con 0x56499a80aad0 2023-07-18T03:23:35.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.743+0000 7f6499f77b80 10 client.28738 fetch_fsmap finished waiting for FSMap version 39 2023-07-18T03:23:35.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.743+0000 7f6499f77b80 10 client.28738 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:23:35.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.743+0000 7f6499f77b80 20 client.28738 populate_metadata read hostname 'smithi101' 2023-07-18T03:23:35.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.743+0000 7f6499f77b80 1 --2- 172.21.15.101:0/1412149532 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56499a86fbb0 0x56499a871fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:35.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.743+0000 7f6499f77b80 4 client.28738 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:23:35.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.743+0000 7f6499f77b80 1 -- 172.21.15.101:0/1412149532 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56499a70d050 con 0x56499a86fbb0 2023-07-18T03:23:35.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.744+0000 7f647f7fe700 1 --2- 172.21.15.101:0/1412149532 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56499a86fbb0 0x56499a871fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:35.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.746+0000 7f647f7fe700 1 --2- 172.21.15.101:0/1412149532 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56499a86fbb0 0x56499a871fc0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10a8d999382883f7 server_cookie=490eca142faff751 in_seq=0 out_seq=0 2023-07-18T03:23:35.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.746+0000 7f645f7fe700 10 client.28738 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:23:35.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.746+0000 7f645f7fe700 1 -- 172.21.15.101:0/1412149532 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56499a70d050 con 0x56499a86fbb0 2023-07-18T03:23:35.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.746+0000 7f645f7fe700 10 client.28738 handle_command_reply: tid=0 2023-07-18T03:23:35.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.747+0000 7f6499f77b80 1 client.28738 shutdown 2023-07-18T03:23:35.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.747+0000 7f6499f77b80 1 -- 172.21.15.101:0/1412149532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56499a80aad0 msgr2=0x56499a8102e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:35.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.747+0000 7f6499f77b80 1 --2- 172.21.15.101:0/1412149532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56499a80aad0 0x56499a8102e0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x56499a81d240 tx=0x7f6478015650 comp rx=0 tx=0).stop 2023-07-18T03:23:35.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.747+0000 7f6499f77b80 1 -- 172.21.15.101:0/1412149532 shutdown_connections 2023-07-18T03:23:35.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.747+0000 7f6499f77b80 1 --2- 172.21.15.101:0/1412149532 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56499a86fbb0 0x56499a871fc0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.747+0000 7f6499f77b80 1 --2- 172.21.15.101:0/1412149532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56499a7ff2d0 0x56499a8117c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.747+0000 7f6499f77b80 1 --2- 172.21.15.101:0/1412149532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56499a80aad0 0x56499a8102e0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.747+0000 7f6499f77b80 1 --2- 172.21.15.101:0/1412149532 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56499a7ffbe0 0x56499a80fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.747+0000 7f6499f77b80 1 -- 172.21.15.101:0/1412149532 >> 172.21.15.101:0/1412149532 conn(0x56499a7f5da0 msgr2=0x56499a802ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:35.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.747+0000 7f645f7fe700 0 client.28738 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:23:35.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.748+0000 7f6499f77b80 1 -- 172.21.15.101:0/1412149532 shutdown_connections 2023-07-18T03:23:35.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.748+0000 7f6499f77b80 1 -- 172.21.15.101:0/1412149532 wait complete. 2023-07-18T03:23:35.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.748+0000 7f6499f77b80 20 client.28738 trim_cache size 0 max 16384 2023-07-18T03:23:35.749 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:23:35.750 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:23:35.750 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:23:35.750 INFO:teuthology.orchestra.run.smithi101.stdout: "4a948d03-674f-4ebf-a4b7-84a3c2f27a18": { 2023-07-18T03:23:35.751 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:23:35.751 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "4a948d03-674f-4ebf-a4b7-84a3c2f27a18", 2023-07-18T03:23:35.751 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:23:35.751 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:23:35.751 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:23:35.752 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:23:35.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.751+0000 7f64865ff700 1 -- 172.21.15.101:0/2843934910 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f64680a13c0 msgr2=0x7f64680a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:35.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.751+0000 7f64865ff700 1 --2- 172.21.15.101:0/2843934910 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f64680a13c0 0x7f64680a3870 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f647009b070 tx=0x7f6470018000 comp rx=0 tx=0).stop 2023-07-18T03:23:35.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.751+0000 7f64865ff700 1 -- 172.21.15.101:0/2843934910 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f648006c9a0 msgr2=0x7f6480116da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:35.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.751+0000 7f64865ff700 1 --2- 172.21.15.101:0/2843934910 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f648006c9a0 0x7f6480116da0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f6470005910 tx=0x7f647001f6d0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.752+0000 7f64865ff700 1 -- 172.21.15.101:0/2843934910 shutdown_connections 2023-07-18T03:23:35.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.752+0000 7f64865ff700 1 --2- 172.21.15.101:0/2843934910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f648010e280 0x7f6480073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.752+0000 7f64865ff700 1 --2- 172.21.15.101:0/2843934910 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f648006c9a0 0x7f6480116da0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.752+0000 7f64865ff700 1 --2- 172.21.15.101:0/2843934910 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f64680a13c0 0x7f64680a3870 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.752+0000 7f64865ff700 1 --2- 172.21.15.101:0/2843934910 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f648006d3a0 0x7f64801172e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:35.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.752+0000 7f64865ff700 1 -- 172.21.15.101:0/2843934910 >> 172.21.15.101:0/2843934910 conn(0x7f648006bf70 msgr2=0x7f648006f240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:35.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.753+0000 7f64865ff700 1 -- 172.21.15.101:0/2843934910 shutdown_connections 2023-07-18T03:23:35.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:23:35.753+0000 7f64865ff700 1 -- 172.21.15.101:0/2843934910 wait complete. 2023-07-18T03:23:35.776 INFO:tasks.cephfs.filesystem:scrub status for tag:4a948d03-674f-4ebf-a4b7-84a3c2f27a18 - {'path': '/', 'tag': '4a948d03-674f-4ebf-a4b7-84a3c2f27a18', 'options': 'recursive,force'} 2023-07-18T03:23:35.990 INFO:tasks.workunit.client.0.smithi101.stdout: 10 102395 30.06 MB/sec execute 60 sec latency 111.209 ms 2023-07-18T03:23:36.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:36 smithi155 ceph-mon[128885]: scrub queued for path: / 2023-07-18T03:23:36.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:36 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:23:36.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:36 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:23:36.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:36 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:23:36.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:36 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:23:36.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:36 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:23:36.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:36 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:23:36.550 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:36 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:23:36.550 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:36 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:23:36.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:36 smithi179 ceph-mon[129229]: scrub queued for path: / 2023-07-18T03:23:36.705 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:36 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:23:36.705 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:36 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:23:36.705 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:36 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:23:36.705 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:36 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:23:36.705 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:36 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:23:36.705 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:36 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:23:36.705 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:36 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:23:36.706 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:36 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:23:36.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:36 smithi101 ceph-mon[126379]: scrub queued for path: / 2023-07-18T03:23:36.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:36 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:23:36.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:36 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:23:36.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:36 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:23:36.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:36 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:23:36.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:36 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:23:36.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:36 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:23:36.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:36 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:23:36.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:36 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:23:36.990 INFO:tasks.workunit.client.0.smithi101.stdout: 10 103143 30.20 MB/sec execute 61 sec latency 74.260 ms 2023-07-18T03:23:37.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:37 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:23:37.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:37 smithi155 ceph-mon[128885]: pgmap v504: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 21 MiB/s wr, 898 op/s 2023-07-18T03:23:37.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:37 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:23:37.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:37 smithi179 ceph-mon[129229]: pgmap v504: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 21 MiB/s wr, 898 op/s 2023-07-18T03:23:37.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:37 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:23:37.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:37 smithi101 ceph-mon[126379]: pgmap v504: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 21 MiB/s wr, 898 op/s 2023-07-18T03:23:37.990 INFO:tasks.workunit.client.0.smithi101.stdout: 10 103898 30.32 MB/sec execute 62 sec latency 52.842 ms 2023-07-18T03:23:38.990 INFO:tasks.workunit.client.0.smithi101.stdout: 10 104732 30.53 MB/sec execute 63 sec latency 76.335 ms 2023-07-18T03:23:39.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:39 smithi155 ceph-mon[128885]: pgmap v505: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 18 MiB/s wr, 722 op/s 2023-07-18T03:23:39.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:39 smithi179 ceph-mon[129229]: pgmap v505: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 18 MiB/s wr, 722 op/s 2023-07-18T03:23:39.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:39 smithi101 ceph-mon[126379]: pgmap v505: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 18 MiB/s wr, 722 op/s 2023-07-18T03:23:39.990 INFO:tasks.workunit.client.0.smithi101.stdout: 10 105337 30.61 MB/sec execute 64 sec latency 164.981 ms 2023-07-18T03:23:40.990 INFO:tasks.workunit.client.0.smithi101.stdout: 10 105873 30.67 MB/sec execute 65 sec latency 74.796 ms 2023-07-18T03:23:41.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:41 smithi155 ceph-mon[128885]: pgmap v506: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 19 MiB/s wr, 799 op/s 2023-07-18T03:23:41.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:41 smithi179 ceph-mon[129229]: pgmap v506: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 19 MiB/s wr, 799 op/s 2023-07-18T03:23:41.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:41 smithi101 ceph-mon[126379]: pgmap v506: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 19 MiB/s wr, 799 op/s 2023-07-18T03:23:41.990 INFO:tasks.workunit.client.0.smithi101.stdout: 10 106505 30.70 MB/sec execute 66 sec latency 94.094 ms 2023-07-18T03:23:42.990 INFO:tasks.workunit.client.0.smithi101.stdout: 10 107319 30.81 MB/sec execute 67 sec latency 73.591 ms 2023-07-18T03:23:43.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:43 smithi155 ceph-mon[128885]: pgmap v507: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 16 MiB/s wr, 718 op/s 2023-07-18T03:23:43.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:43 smithi179 ceph-mon[129229]: pgmap v507: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 16 MiB/s wr, 718 op/s 2023-07-18T03:23:43.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:43 smithi101 ceph-mon[126379]: pgmap v507: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 16 MiB/s wr, 718 op/s 2023-07-18T03:23:43.990 INFO:tasks.workunit.client.0.smithi101.stdout: 10 108205 31.11 MB/sec execute 68 sec latency 41.959 ms 2023-07-18T03:23:44.990 INFO:tasks.workunit.client.0.smithi101.stdout: 10 108825 31.20 MB/sec execute 69 sec latency 60.797 ms 2023-07-18T03:23:45.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:45 smithi155 ceph-mon[128885]: pgmap v508: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 11 MiB/s wr, 548 op/s 2023-07-18T03:23:45.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:45 smithi179 ceph-mon[129229]: pgmap v508: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 11 MiB/s wr, 548 op/s 2023-07-18T03:23:45.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:45 smithi101 ceph-mon[126379]: pgmap v508: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 11 MiB/s wr, 548 op/s 2023-07-18T03:23:45.990 INFO:tasks.workunit.client.0.smithi101.stdout: 10 109282 30.98 MB/sec execute 70 sec latency 255.782 ms 2023-07-18T03:23:46.990 INFO:tasks.workunit.client.0.smithi101.stdout: 10 110060 31.09 MB/sec execute 71 sec latency 118.410 ms 2023-07-18T03:23:47.707 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:47 smithi179 ceph-mon[129229]: pgmap v509: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 717 op/s 2023-07-18T03:23:47.707 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:47 smithi179 ceph-mon[129229]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:23:47.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:47 smithi101 ceph-mon[126379]: pgmap v509: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 717 op/s 2023-07-18T03:23:47.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:47 smithi101 ceph-mon[126379]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:23:47.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:47 smithi155 ceph-mon[128885]: pgmap v509: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 717 op/s 2023-07-18T03:23:47.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:47 smithi155 ceph-mon[128885]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:23:47.990 INFO:tasks.workunit.client.0.smithi101.stdout: 10 110685 31.21 MB/sec execute 72 sec latency 240.772 ms 2023-07-18T03:23:48.990 INFO:tasks.workunit.client.0.smithi101.stdout: 10 111455 31.41 MB/sec execute 73 sec latency 79.734 ms 2023-07-18T03:23:49.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:49 smithi179 ceph-mon[129229]: pgmap v510: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 6.9 MiB/s wr, 461 op/s 2023-07-18T03:23:49.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:49 smithi101 ceph-mon[126379]: pgmap v510: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 6.9 MiB/s wr, 461 op/s 2023-07-18T03:23:49.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:49 smithi155 ceph-mon[128885]: pgmap v510: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 6.9 MiB/s wr, 461 op/s 2023-07-18T03:23:49.991 INFO:tasks.workunit.client.0.smithi101.stdout: 10 111968 31.24 MB/sec execute 74 sec latency 70.531 ms 2023-07-18T03:23:50.990 INFO:tasks.workunit.client.0.smithi101.stdout: 10 112737 31.54 MB/sec execute 75 sec latency 78.290 ms 2023-07-18T03:23:51.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:51 smithi179 ceph-mon[129229]: pgmap v511: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.2 MiB/s wr, 516 op/s 2023-07-18T03:23:51.705 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:51 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:23:51.705 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:51 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:23:51.705 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:51 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:23:51.745 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:51 smithi101 ceph-mon[126379]: pgmap v511: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.2 MiB/s wr, 516 op/s 2023-07-18T03:23:51.745 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:51 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:23:51.745 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:51 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:23:51.746 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:51 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:23:51.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:51 smithi155 ceph-mon[128885]: pgmap v511: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.2 MiB/s wr, 516 op/s 2023-07-18T03:23:51.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:51 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:23:51.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:51 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:23:51.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:51 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:23:51.994 INFO:tasks.workunit.client.0.smithi101.stdout: 10 113241 31.36 MB/sec execute 76 sec latency 459.496 ms 2023-07-18T03:23:52.994 INFO:tasks.workunit.client.0.smithi101.stdout: 10 113446 31.05 MB/sec execute 77 sec latency 710.001 ms 2023-07-18T03:23:53.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:53 smithi179 ceph-mon[129229]: pgmap v512: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.2 MiB/s wr, 518 op/s 2023-07-18T03:23:53.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:53 smithi101 ceph-mon[126379]: pgmap v512: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.2 MiB/s wr, 518 op/s 2023-07-18T03:23:53.805 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:53 smithi155 ceph-mon[128885]: pgmap v512: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.2 MiB/s wr, 518 op/s 2023-07-18T03:23:53.994 INFO:tasks.workunit.client.0.smithi101.stdout: 10 113728 30.95 MB/sec execute 78 sec latency 96.909 ms 2023-07-18T03:23:54.994 INFO:tasks.workunit.client.0.smithi101.stdout: 10 114385 31.09 MB/sec execute 79 sec latency 86.291 ms 2023-07-18T03:23:55.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:55 smithi179 ceph-mon[129229]: pgmap v513: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 6.2 MiB/s wr, 395 op/s 2023-07-18T03:23:55.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:55 smithi101 ceph-mon[126379]: pgmap v513: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 6.2 MiB/s wr, 395 op/s 2023-07-18T03:23:55.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:55 smithi155 ceph-mon[128885]: pgmap v513: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 6.2 MiB/s wr, 395 op/s 2023-07-18T03:23:55.994 INFO:tasks.workunit.client.0.smithi101.stdout: 10 114872 30.95 MB/sec execute 80 sec latency 191.418 ms 2023-07-18T03:23:56.994 INFO:tasks.workunit.client.0.smithi101.stdout: 10 115351 30.77 MB/sec execute 81 sec latency 113.743 ms 2023-07-18T03:23:57.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:57 smithi179 ceph-mon[129229]: pgmap v514: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 651 op/s 2023-07-18T03:23:57.494 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:57 smithi101 ceph-mon[126379]: pgmap v514: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 651 op/s 2023-07-18T03:23:57.554 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:57 smithi155 ceph-mon[128885]: pgmap v514: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 651 op/s 2023-07-18T03:23:57.995 INFO:tasks.workunit.client.0.smithi101.stdout: 10 115474 30.47 MB/sec execute 82 sec latency 715.948 ms 2023-07-18T03:23:58.995 INFO:tasks.workunit.client.0.smithi101.stdout: 10 116087 30.59 MB/sec execute 83 sec latency 896.427 ms 2023-07-18T03:23:59.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:23:59 smithi179 ceph-mon[129229]: pgmap v515: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 12 MiB/s wr, 526 op/s 2023-07-18T03:23:59.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:23:59 smithi101 ceph-mon[126379]: pgmap v515: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 12 MiB/s wr, 526 op/s 2023-07-18T03:23:59.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:23:59 smithi155 ceph-mon[128885]: pgmap v515: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 12 MiB/s wr, 526 op/s 2023-07-18T03:23:59.995 INFO:tasks.workunit.client.0.smithi101.stdout: 10 116566 30.47 MB/sec execute 84 sec latency 121.114 ms 2023-07-18T03:24:00.995 INFO:tasks.workunit.client.0.smithi101.stdout: 10 116989 30.48 MB/sec execute 85 sec latency 247.668 ms 2023-07-18T03:24:01.455 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:01 smithi179 ceph-mon[129229]: pgmap v516: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 16 MiB/s wr, 649 op/s 2023-07-18T03:24:01.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:01 smithi101 ceph-mon[126379]: pgmap v516: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 16 MiB/s wr, 649 op/s 2023-07-18T03:24:01.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:01 smithi155 ceph-mon[128885]: pgmap v516: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 16 MiB/s wr, 649 op/s 2023-07-18T03:24:01.995 INFO:tasks.workunit.client.0.smithi101.stdout: 10 117267 30.26 MB/sec execute 86 sec latency 366.506 ms 2023-07-18T03:24:02.995 INFO:tasks.workunit.client.0.smithi101.stdout: 10 117365 29.96 MB/sec execute 87 sec latency 375.394 ms 2023-07-18T03:24:03.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:03 smithi179 ceph-mon[129229]: pgmap v517: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 18 MiB/s wr, 734 op/s 2023-07-18T03:24:03.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:03 smithi101 ceph-mon[126379]: pgmap v517: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 18 MiB/s wr, 734 op/s 2023-07-18T03:24:03.550 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:03 smithi155 ceph-mon[128885]: pgmap v517: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 18 MiB/s wr, 734 op/s 2023-07-18T03:24:03.996 INFO:tasks.workunit.client.0.smithi101.stdout: 10 117370 29.65 MB/sec execute 88 sec latency 363.916 ms 2023-07-18T03:24:04.996 INFO:tasks.workunit.client.0.smithi101.stdout: 10 117506 29.45 MB/sec execute 89 sec latency 292.558 ms 2023-07-18T03:24:05.456 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:05 smithi179 ceph-mon[129229]: pgmap v518: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 16 MiB/s wr, 620 op/s 2023-07-18T03:24:05.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:05 smithi101 ceph-mon[126379]: pgmap v518: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 16 MiB/s wr, 620 op/s 2023-07-18T03:24:05.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:05 smithi155 ceph-mon[128885]: pgmap v518: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 16 MiB/s wr, 620 op/s 2023-07-18T03:24:05.777 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:24:05.996 INFO:tasks.workunit.client.0.smithi101.stdout: 10 117897 29.34 MB/sec execute 90 sec latency 253.859 ms 2023-07-18T03:24:06.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.041+0000 7f85551ff700 1 -- 172.21.15.101:0/3161660693 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8550101f60 msgr2=0x7f8550102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:06.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.041+0000 7f85551ff700 1 --2- 172.21.15.101:0/3161660693 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8550101f60 0x7f8550102330 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f85400099c0 tx=0x7f8540009cd0 comp rx=0 tx=0).stop 2023-07-18T03:24:06.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.041+0000 7f85551ff700 1 -- 172.21.15.101:0/3161660693 shutdown_connections 2023-07-18T03:24:06.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.041+0000 7f85551ff700 1 --2- 172.21.15.101:0/3161660693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8550106fa0 0x7f855010eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:06.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.042+0000 7f85551ff700 1 --2- 172.21.15.101:0/3161660693 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8550101f60 0x7f8550102330 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:06.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.042+0000 7f85551ff700 1 --2- 172.21.15.101:0/3161660693 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8550102900 0x7f85501068c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:06.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.042+0000 7f85551ff700 1 -- 172.21.15.101:0/3161660693 >> 172.21.15.101:0/3161660693 conn(0x7f85500fd7d0 msgr2=0x7f85500ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:06.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.042+0000 7f85551ff700 1 -- 172.21.15.101:0/3161660693 shutdown_connections 2023-07-18T03:24:06.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.042+0000 7f85551ff700 1 -- 172.21.15.101:0/3161660693 wait complete. 2023-07-18T03:24:06.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.043+0000 7f85551ff700 1 Processor -- start 2023-07-18T03:24:06.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.043+0000 7f85551ff700 1 -- start start 2023-07-18T03:24:06.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.044+0000 7f85551ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8550101f60 0x7f8550074a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:06.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.044+0000 7f85551ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8550102900 0x7f8550074f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:06.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.044+0000 7f85551ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8550106fa0 0x7f85500754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:06.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.044+0000 7f85551ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8550197c60 con 0x7f8550106fa0 2023-07-18T03:24:06.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.044+0000 7f85551ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8550197dd0 con 0x7f8550102900 2023-07-18T03:24:06.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.044+0000 7f85551ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8550197f90 con 0x7f8550101f60 2023-07-18T03:24:06.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.045+0000 7f854e59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8550102900 0x7f8550074f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:06.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.045+0000 7f854e59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8550102900 0x7f8550074f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:40952/0 (socket says 172.21.15.101:40952) 2023-07-18T03:24:06.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.045+0000 7f854e59c700 1 -- 172.21.15.101:0/3399929926 learned_addr learned my addr 172.21.15.101:0/3399929926 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:06.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.045+0000 7f854f59e700 1 --2- 172.21.15.101:0/3399929926 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8550106fa0 0x7f85500754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:06.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.045+0000 7f854ed9d700 1 --2- 172.21.15.101:0/3399929926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8550101f60 0x7f8550074a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:06.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.046+0000 7f854e59c700 1 -- 172.21.15.101:0/3399929926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8550101f60 msgr2=0x7f8550074a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:06.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.046+0000 7f854e59c700 1 --2- 172.21.15.101:0/3399929926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8550101f60 0x7f8550074a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:06.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.046+0000 7f854e59c700 1 -- 172.21.15.101:0/3399929926 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8550106fa0 msgr2=0x7f85500754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:06.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.046+0000 7f854e59c700 1 --2- 172.21.15.101:0/3399929926 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8550106fa0 0x7f85500754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:06.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.046+0000 7f854e59c700 1 -- 172.21.15.101:0/3399929926 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85400095f0 con 0x7f8550102900 2023-07-18T03:24:06.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.047+0000 7f854e59c700 1 --2- 172.21.15.101:0/3399929926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8550102900 0x7f8550074f60 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f854400b4d0 tx=0x7f854400b7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:06.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.048+0000 7f8537fff700 1 -- 172.21.15.101:0/3399929926 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8544011830 con 0x7f8550102900 2023-07-18T03:24:06.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.048+0000 7f8537fff700 1 -- 172.21.15.101:0/3399929926 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8544011e50 con 0x7f8550102900 2023-07-18T03:24:06.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.048+0000 7f8537fff700 1 -- 172.21.15.101:0/3399929926 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8544010e50 con 0x7f8550102900 2023-07-18T03:24:06.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.048+0000 7f85551ff700 1 -- 172.21.15.101:0/3399929926 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8550198270 con 0x7f8550102900 2023-07-18T03:24:06.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.048+0000 7f85551ff700 1 -- 172.21.15.101:0/3399929926 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8550198730 con 0x7f8550102900 2023-07-18T03:24:06.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.052+0000 7f8537fff700 1 -- 172.21.15.101:0/3399929926 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f85440104f0 con 0x7f8550102900 2023-07-18T03:24:06.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.054+0000 7f8568ad7b80 1 Processor -- start 2023-07-18T03:24:06.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.055+0000 7f8568ad7b80 1 -- start start 2023-07-18T03:24:06.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.055+0000 7f8537fff700 1 --2- 172.21.15.101:0/3399929926 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f85380a1410 0x7f85380a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:06.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.055+0000 7f8537fff700 1 -- 172.21.15.101:0/3399929926 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f85440cc940 con 0x7f8550102900 2023-07-18T03:24:06.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.055+0000 7f8568ad7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c0289fa2d0 0x55c0289fa6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:06.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.055+0000 7f8568ad7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0289fabe0 0x55c028a05590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:06.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.055+0000 7f8568ad7b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c028a05ad0 0x55c028a07eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:06.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.055+0000 7f8568ad7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c028900cb0 con 0x55c0289fa2d0 2023-07-18T03:24:06.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.055+0000 7f8568ad7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c028953a40 con 0x55c0289fabe0 2023-07-18T03:24:06.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.055+0000 7f8568ad7b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c02894ced0 con 0x55c028a05ad0 2023-07-18T03:24:06.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.056+0000 7f854e59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0289fabe0 0x55c028a05590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:06.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.056+0000 7f854e59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0289fabe0 0x55c028a05590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:40954/0 (socket says 172.21.15.101:40954) 2023-07-18T03:24:06.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.056+0000 7f854e59c700 1 -- 172.21.15.101:0/3961650024 learned_addr learned my addr 172.21.15.101:0/3961650024 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:06.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.056+0000 7f854f59e700 1 --2- 172.21.15.101:0/3399929926 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f85380a1410 0x7f85380a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:06.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.056+0000 7f854f59e700 1 --2- 172.21.15.101:0/3961650024 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c028a05ad0 0x55c028a07eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:06.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.056+0000 7f854ed9d700 1 --2- 172.21.15.101:0/3961650024 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c0289fa2d0 0x55c0289fa6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:06.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.056+0000 7f854e59c700 1 -- 172.21.15.101:0/3961650024 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c028a05ad0 msgr2=0x55c028a07eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:06.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.056+0000 7f854e59c700 1 --2- 172.21.15.101:0/3961650024 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c028a05ad0 0x55c028a07eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:06.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.056+0000 7f854e59c700 1 -- 172.21.15.101:0/3961650024 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c0289fa2d0 msgr2=0x55c0289fa6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:06.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.056+0000 7f854e59c700 1 --2- 172.21.15.101:0/3961650024 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c0289fa2d0 0x55c0289fa6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:06.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.056+0000 7f854e59c700 1 -- 172.21.15.101:0/3961650024 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c02877ade0 con 0x55c0289fabe0 2023-07-18T03:24:06.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.057+0000 7f854e59c700 1 --2- 172.21.15.101:0/3961650024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0289fabe0 0x55c028a05590 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f85440996a0 tx=0x7f8544099560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9973e4892974d532 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:06.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.057+0000 7f85357fa700 1 -- 172.21.15.101:0/3961650024 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8544099710 con 0x55c0289fabe0 2023-07-18T03:24:06.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.057+0000 7f854f59e700 1 --2- 172.21.15.101:0/3399929926 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f85380a1410 0x7f85380a38c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8548009ec0 tx=0x7f8548009920 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:06.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.057+0000 7f85357fa700 1 -- 172.21.15.101:0/3961650024 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f854409d740 con 0x55c0289fabe0 2023-07-18T03:24:06.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.057+0000 7f85357fa700 1 -- 172.21.15.101:0/3961650024 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8544098e00 con 0x55c0289fabe0 2023-07-18T03:24:06.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.058+0000 7f8568ad7b80 1 -- 172.21.15.101:0/3961650024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0289fabe0 msgr2=0x55c028a05590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:06.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.058+0000 7f8568ad7b80 1 --2- 172.21.15.101:0/3961650024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0289fabe0 0x55c028a05590 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f85440996a0 tx=0x7f8544099560 comp rx=0 tx=0).stop 2023-07-18T03:24:06.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.058+0000 7f8568ad7b80 1 -- 172.21.15.101:0/3961650024 shutdown_connections 2023-07-18T03:24:06.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.058+0000 7f8568ad7b80 1 --2- 172.21.15.101:0/3961650024 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c028a05ad0 0x55c028a07eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:06.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.058+0000 7f8568ad7b80 1 --2- 172.21.15.101:0/3961650024 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c0289fa2d0 0x55c0289fa6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:06.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.058+0000 7f8568ad7b80 1 --2- 172.21.15.101:0/3961650024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0289fabe0 0x55c028a05590 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:06.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.058+0000 7f8568ad7b80 1 -- 172.21.15.101:0/3961650024 >> 172.21.15.101:0/3961650024 conn(0x55c0289f0da0 msgr2=0x55c0289f66a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:06.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.058+0000 7f8568ad7b80 1 -- 172.21.15.101:0/3961650024 shutdown_connections 2023-07-18T03:24:06.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.058+0000 7f8568ad7b80 1 -- 172.21.15.101:0/3961650024 wait complete. 2023-07-18T03:24:06.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.059+0000 7f8568ad7b80 1 -- start start 2023-07-18T03:24:06.061 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.059+0000 7f8568ad7b80 1 Processor -- start 2023-07-18T03:24:06.061 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.060+0000 7f8568ad7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c0289fa2d0 0x55c028a05540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:06.061 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.060+0000 7f8568ad7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0289fabe0 0x55c028a027c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:06.061 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.060+0000 7f8568ad7b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c028a05ad0 0x55c028a02eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:06.061 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.060+0000 7f8568ad7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0288ecac0 con 0x55c0289fa2d0 2023-07-18T03:24:06.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.060+0000 7f8568ad7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0288ec7c0 con 0x55c0289fabe0 2023-07-18T03:24:06.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.060+0000 7f8568ad7b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0287d0920 con 0x55c028a05ad0 2023-07-18T03:24:06.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.060+0000 7f8568ad7b80 10 client.0 _ll_register_callbacks cb 0x55c0288ea760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:24:06.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.060+0000 7f854e59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0289fabe0 0x55c028a027c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:06.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.060+0000 7f854e59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0289fabe0 0x55c028a027c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:40966/0 (socket says 172.21.15.101:40966) 2023-07-18T03:24:06.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.060+0000 7f854e59c700 1 -- 172.21.15.101:0/1115067715 learned_addr learned my addr 172.21.15.101:0/1115067715 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:06.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.060+0000 7f854f59e700 1 --2- 172.21.15.101:0/1115067715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c028a05ad0 0x55c028a02eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:06.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.061+0000 7f854e59c700 1 -- 172.21.15.101:0/1115067715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c028a05ad0 msgr2=0x55c028a02eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:06.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.061+0000 7f854e59c700 1 --2- 172.21.15.101:0/1115067715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c028a05ad0 0x55c028a02eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:06.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.061+0000 7f854e59c700 1 -- 172.21.15.101:0/1115067715 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c0289fa2d0 msgr2=0x55c028a05540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:06.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.061+0000 7f854e59c700 1 --2- 172.21.15.101:0/1115067715 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c0289fa2d0 0x55c028a05540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:06.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.061+0000 7f854e59c700 1 -- 172.21.15.101:0/1115067715 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8550198730 con 0x55c0289fabe0 2023-07-18T03:24:06.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.061+0000 7f854e59c700 1 --2- 172.21.15.101:0/1115067715 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0289fabe0 0x55c028a027c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f85440a1630 tx=0x7f85440b9270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c1231dd2cc78372 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:06.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.061+0000 7f8532ff5700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:24:06.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.061+0000 7f8532ff5700 1 -- 172.21.15.101:0/1115067715 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8544099710 con 0x55c0289fabe0 2023-07-18T03:24:06.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.061+0000 7f8568ad7b80 1 -- 172.21.15.101:0/1115067715 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c0288a64f0 con 0x55c0289fabe0 2023-07-18T03:24:06.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.061+0000 7f8532ff5700 1 -- 172.21.15.101:0/1115067715 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f85440b4070 con 0x55c0289fabe0 2023-07-18T03:24:06.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.061+0000 7f8532ff5700 1 -- 172.21.15.101:0/1115067715 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8544098e10 con 0x55c0289fabe0 2023-07-18T03:24:06.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.062+0000 7f8568ad7b80 10 client.28736 fetch_fsmap learned FSMap version 41 2023-07-18T03:24:06.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.062+0000 7f8568ad7b80 1 -- 172.21.15.101:0/1115067715 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55c028763c60 con 0x55c0289fabe0 2023-07-18T03:24:06.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.062+0000 7f8532ff5700 1 -- 172.21.15.101:0/1115067715 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c0288a64f0 con 0x55c0289fabe0 2023-07-18T03:24:06.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.062+0000 7f8532ff5700 1 -- 172.21.15.101:0/1115067715 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 41) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f854409ab70 con 0x55c0289fabe0 2023-07-18T03:24:06.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.062+0000 7f8568ad7b80 10 client.28736 fetch_fsmap finished waiting for FSMap version 41 2023-07-18T03:24:06.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.062+0000 7f8568ad7b80 10 client.28736 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:24:06.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.062+0000 7f8568ad7b80 20 client.28736 populate_metadata read hostname 'smithi101' 2023-07-18T03:24:06.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.062+0000 7f8568ad7b80 1 --2- 172.21.15.101:0/1115067715 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c028a15030 0x55c028a17440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:06.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.062+0000 7f8568ad7b80 4 client.28736 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:24:06.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.062+0000 7f8568ad7b80 1 -- 172.21.15.101:0/1115067715 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c028945eb0 con 0x55c028a15030 2023-07-18T03:24:06.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.062+0000 7f854ed9d700 1 --2- 172.21.15.101:0/1115067715 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c028a15030 0x55c028a17440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:06.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.063+0000 7f8532ff5700 10 client.28736 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:24:06.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.063+0000 7f854ed9d700 1 --2- 172.21.15.101:0/1115067715 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c028a15030 0x55c028a17440 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1e790d2ec0fa043 server_cookie=d91a94e7dbe5323f in_seq=0 out_seq=0 2023-07-18T03:24:06.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.064+0000 7f8532ff5700 1 -- 172.21.15.101:0/1115067715 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c028945eb0 con 0x55c028a15030 2023-07-18T03:24:06.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.064+0000 7f8532ff5700 10 client.28736 handle_command_reply: tid=0 2023-07-18T03:24:06.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.064+0000 7f8568ad7b80 1 client.28736 shutdown 2023-07-18T03:24:06.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.064+0000 7f8568ad7b80 1 -- 172.21.15.101:0/1115067715 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0289fabe0 msgr2=0x55c028a027c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:06.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.064+0000 7f8568ad7b80 1 --2- 172.21.15.101:0/1115067715 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0289fabe0 0x55c028a027c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f85440a1630 tx=0x7f85440b9270 comp rx=0 tx=0).stop 2023-07-18T03:24:06.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.064+0000 7f8568ad7b80 1 -- 172.21.15.101:0/1115067715 shutdown_connections 2023-07-18T03:24:06.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.064+0000 7f8568ad7b80 1 --2- 172.21.15.101:0/1115067715 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c028a15030 0x55c028a17440 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:06.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.064+0000 7f8568ad7b80 1 --2- 172.21.15.101:0/1115067715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c028a05ad0 0x55c028a02eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:06.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.064+0000 7f8568ad7b80 1 --2- 172.21.15.101:0/1115067715 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c0289fa2d0 0x55c028a05540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:06.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.064+0000 7f8568ad7b80 1 --2- 172.21.15.101:0/1115067715 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0289fabe0 0x55c028a027c0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:06.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.064+0000 7f8568ad7b80 1 -- 172.21.15.101:0/1115067715 >> 172.21.15.101:0/1115067715 conn(0x55c0289f0da0 msgr2=0x55c0289f6230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:06.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.064+0000 7f8532ff5700 0 client.28736 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:24:06.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.064+0000 7f8568ad7b80 1 -- 172.21.15.101:0/1115067715 shutdown_connections 2023-07-18T03:24:06.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.064+0000 7f8568ad7b80 1 -- 172.21.15.101:0/1115067715 wait complete. 2023-07-18T03:24:06.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.065+0000 7f8568ad7b80 20 client.28736 trim_cache size 0 max 16384 2023-07-18T03:24:06.093 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.096+0000 7f8568ad7b80 1 Processor -- start 2023-07-18T03:24:06.093 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.096+0000 7f8568ad7b80 1 -- start start 2023-07-18T03:24:06.094 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.097+0000 7f8568ad7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c0289f5170 0x55c028a0c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:06.094 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.097+0000 7f8568ad7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0289f0da0 0x55c0289fc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:06.094 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.097+0000 7f8568ad7b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0289fa2d0 0x55c0289fcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:06.094 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.097+0000 7f8568ad7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0289fb160 con 0x55c0289f5170 2023-07-18T03:24:06.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.097+0000 7f8568ad7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0289f2ec0 con 0x55c0289f0da0 2023-07-18T03:24:06.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.097+0000 7f8568ad7b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0289edef0 con 0x55c0289fa2d0 2023-07-18T03:24:06.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.097+0000 7f854e59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0289f0da0 0x55c0289fc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:06.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.097+0000 7f854e59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0289f0da0 0x55c0289fc8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:40972/0 (socket says 172.21.15.101:40972) 2023-07-18T03:24:06.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.097+0000 7f854e59c700 1 -- 172.21.15.101:0/2660700670 learned_addr learned my addr 172.21.15.101:0/2660700670 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:06.095 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.097+0000 7f854ed9d700 1 --2- 172.21.15.101:0/2660700670 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c0289f5170 0x55c028a0c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:06.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.098+0000 7f854e59c700 1 -- 172.21.15.101:0/2660700670 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0289fa2d0 msgr2=0x55c0289fcdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:06.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.098+0000 7f854e59c700 1 --2- 172.21.15.101:0/2660700670 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0289fa2d0 0x55c0289fcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:06.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.098+0000 7f854e59c700 1 -- 172.21.15.101:0/2660700670 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c0289f5170 msgr2=0x55c028a0c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:06.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.098+0000 7f854e59c700 1 --2- 172.21.15.101:0/2660700670 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c0289f5170 0x55c028a0c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:06.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.098+0000 7f854e59c700 1 -- 172.21.15.101:0/2660700670 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0289f25e0 con 0x55c0289f0da0 2023-07-18T03:24:06.097 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.098+0000 7f854ed9d700 1 --2- 172.21.15.101:0/2660700670 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c0289f5170 0x55c028a0c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:24:06.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.098+0000 7f854e59c700 1 --2- 172.21.15.101:0/2660700670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0289f0da0 0x55c0289fc8b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f85440996a0 tx=0x7f8544000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b0ca142882c6497 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:06.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.098+0000 7f8535ffb700 1 -- 172.21.15.101:0/2660700670 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8544095620 con 0x55c0289f0da0 2023-07-18T03:24:06.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.098+0000 7f8535ffb700 1 -- 172.21.15.101:0/2660700670 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8544099710 con 0x55c0289f0da0 2023-07-18T03:24:06.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.098+0000 7f8535ffb700 1 -- 172.21.15.101:0/2660700670 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85440a16d0 con 0x55c0289f0da0 2023-07-18T03:24:06.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.099+0000 7f8568ad7b80 1 -- 172.21.15.101:0/2660700670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0289f0da0 msgr2=0x55c0289fc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:06.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.099+0000 7f8568ad7b80 1 --2- 172.21.15.101:0/2660700670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0289f0da0 0x55c0289fc8b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f85440996a0 tx=0x7f8544000bc0 comp rx=0 tx=0).stop 2023-07-18T03:24:06.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.099+0000 7f8568ad7b80 1 -- 172.21.15.101:0/2660700670 shutdown_connections 2023-07-18T03:24:06.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.099+0000 7f8568ad7b80 1 --2- 172.21.15.101:0/2660700670 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0289fa2d0 0x55c0289fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:06.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.099+0000 7f8568ad7b80 1 --2- 172.21.15.101:0/2660700670 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c0289f5170 0x55c028a0c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:06.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.099+0000 7f8568ad7b80 1 --2- 172.21.15.101:0/2660700670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0289f0da0 0x55c0289fc8b0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:06.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.099+0000 7f8568ad7b80 1 -- 172.21.15.101:0/2660700670 >> 172.21.15.101:0/2660700670 conn(0x55c028a11a20 msgr2=0x55c0289fc1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:06.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.100+0000 7f8568ad7b80 1 -- 172.21.15.101:0/2660700670 shutdown_connections 2023-07-18T03:24:06.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.100+0000 7f8568ad7b80 1 -- 172.21.15.101:0/2660700670 wait complete. 2023-07-18T03:24:06.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.100+0000 7f8568ad7b80 1 -- start start 2023-07-18T03:24:06.101 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.101+0000 7f8568ad7b80 1 Processor -- start 2023-07-18T03:24:06.101 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.101+0000 7f8568ad7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c0289f0da0 0x55c028a0aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:06.101 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.101+0000 7f8568ad7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0289f5170 0x55c028a0b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:06.101 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.101+0000 7f8568ad7b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0289fa2d0 0x55c028a0b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:06.101 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.101+0000 7f8568ad7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c028a68f40 con 0x55c0289f0da0 2023-07-18T03:24:06.101 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.101+0000 7f8568ad7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c028a69080 con 0x55c0289f5170 2023-07-18T03:24:06.101 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.101+0000 7f8568ad7b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c028a691c0 con 0x55c0289fa2d0 2023-07-18T03:24:06.102 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.101+0000 7f8568ad7b80 10 client.0 _ll_register_callbacks cb 0x7f85440b0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:24:06.102 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.101+0000 7f854e59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0289f5170 0x55c028a0b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:06.102 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.102+0000 7f854e59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0289f5170 0x55c028a0b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:40986/0 (socket says 172.21.15.101:40986) 2023-07-18T03:24:06.102 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.102+0000 7f854e59c700 1 -- 172.21.15.101:0/1569066499 learned_addr learned my addr 172.21.15.101:0/1569066499 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:06.102 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.102+0000 7f854f59e700 1 --2- 172.21.15.101:0/1569066499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0289fa2d0 0x55c028a0b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:06.102 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.102+0000 7f854e59c700 1 -- 172.21.15.101:0/1569066499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0289fa2d0 msgr2=0x55c028a0b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:06.102 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.102+0000 7f854e59c700 1 --2- 172.21.15.101:0/1569066499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0289fa2d0 0x55c028a0b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:06.102 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.102+0000 7f854e59c700 1 -- 172.21.15.101:0/1569066499 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c0289f0da0 msgr2=0x55c028a0aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:06.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.102+0000 7f854ed9d700 1 --2- 172.21.15.101:0/1569066499 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c0289f0da0 0x55c028a0aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:06.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.102+0000 7f854e59c700 1 --2- 172.21.15.101:0/1569066499 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c0289f0da0 0x55c028a0aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:06.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.102+0000 7f854e59c700 1 -- 172.21.15.101:0/1569066499 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f855010f3b0 con 0x55c0289f5170 2023-07-18T03:24:06.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.102+0000 7f854e59c700 1 --2- 172.21.15.101:0/1569066499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0289f5170 0x55c028a0b010 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8544097a60 tx=0x7f8544009380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9b464813d96c1bc server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:06.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.102+0000 7f8532ff5700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:24:06.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.102+0000 7f8532ff5700 1 -- 172.21.15.101:0/1569066499 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85440ad0d0 con 0x55c0289f5170 2023-07-18T03:24:06.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.103+0000 7f8532ff5700 1 -- 172.21.15.101:0/1569066499 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f85440b4040 con 0x55c0289f5170 2023-07-18T03:24:06.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.103+0000 7f8568ad7b80 1 -- 172.21.15.101:0/1569066499 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c028a11df0 con 0x55c0289f5170 2023-07-18T03:24:06.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.106+0000 7f8532ff5700 1 -- 172.21.15.101:0/1569066499 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8544095620 con 0x55c0289f5170 2023-07-18T03:24:06.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.106+0000 7f8568ad7b80 10 client.28742 fetch_fsmap learned FSMap version 41 2023-07-18T03:24:06.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.106+0000 7f8568ad7b80 1 -- 172.21.15.101:0/1569066499 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55c028763c60 con 0x55c0289f5170 2023-07-18T03:24:06.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.107+0000 7f8532ff5700 1 -- 172.21.15.101:0/1569066499 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c028a11df0 con 0x55c0289f5170 2023-07-18T03:24:06.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.107+0000 7f8532ff5700 1 -- 172.21.15.101:0/1569066499 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 41) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f8544099970 con 0x55c0289f5170 2023-07-18T03:24:06.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.107+0000 7f8568ad7b80 10 client.28742 fetch_fsmap finished waiting for FSMap version 41 2023-07-18T03:24:06.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.107+0000 7f8568ad7b80 10 client.28742 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:24:06.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.107+0000 7f8568ad7b80 20 client.28742 populate_metadata read hostname 'smithi101' 2023-07-18T03:24:06.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.107+0000 7f8568ad7b80 1 --2- 172.21.15.101:0/1569066499 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c0289fabe0 0x55c028a6c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:06.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.107+0000 7f8568ad7b80 4 client.28742 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:24:06.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.107+0000 7f8568ad7b80 1 -- 172.21.15.101:0/1569066499 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f854409d740 con 0x55c0289fabe0 2023-07-18T03:24:06.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.109+0000 7f854ed9d700 1 --2- 172.21.15.101:0/1569066499 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c0289fabe0 0x55c028a6c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:06.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.110+0000 7f854ed9d700 1 --2- 172.21.15.101:0/1569066499 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c0289fabe0 0x55c028a6c350 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b64c2221277bca5e server_cookie=5d0df1c208bf8e20 in_seq=0 out_seq=0 2023-07-18T03:24:06.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.110+0000 7f8532ff5700 10 client.28742 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:24:06.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.113+0000 7f8532ff5700 1 -- 172.21.15.101:0/1569066499 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7f854409d740 con 0x55c0289fabe0 2023-07-18T03:24:06.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.113+0000 7f8532ff5700 10 client.28742 handle_command_reply: tid=0 2023-07-18T03:24:06.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.113+0000 7f8568ad7b80 1 client.28742 shutdown 2023-07-18T03:24:06.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.114+0000 7f8568ad7b80 1 -- 172.21.15.101:0/1569066499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0289f5170 msgr2=0x55c028a0b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:06.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.114+0000 7f8568ad7b80 1 --2- 172.21.15.101:0/1569066499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0289f5170 0x55c028a0b010 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8544097a60 tx=0x7f8544009380 comp rx=0 tx=0).stop 2023-07-18T03:24:06.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.114+0000 7f8568ad7b80 1 -- 172.21.15.101:0/1569066499 shutdown_connections 2023-07-18T03:24:06.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.114+0000 7f8568ad7b80 1 --2- 172.21.15.101:0/1569066499 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c0289fabe0 0x55c028a6c350 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:06.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.114+0000 7f8568ad7b80 1 --2- 172.21.15.101:0/1569066499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0289fa2d0 0x55c028a0b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:06.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.114+0000 7f8568ad7b80 1 --2- 172.21.15.101:0/1569066499 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c0289f0da0 0x55c028a0aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:06.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.114+0000 7f8568ad7b80 1 --2- 172.21.15.101:0/1569066499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0289f5170 0x55c028a0b010 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:06.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.114+0000 7f8568ad7b80 1 -- 172.21.15.101:0/1569066499 >> 172.21.15.101:0/1569066499 conn(0x55c028a11a20 msgr2=0x55c0289fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:06.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.114+0000 7f8532ff5700 0 client.28742 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:24:06.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.114+0000 7f8568ad7b80 1 -- 172.21.15.101:0/1569066499 shutdown_connections 2023-07-18T03:24:06.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.114+0000 7f8568ad7b80 1 -- 172.21.15.101:0/1569066499 wait complete. 2023-07-18T03:24:06.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.114+0000 7f8568ad7b80 20 client.28742 trim_cache size 0 max 16384 2023-07-18T03:24:06.113 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:24:06.113 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "no active scrubs running", 2023-07-18T03:24:06.113 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": {} 2023-07-18T03:24:06.114 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:24:06.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.115+0000 7f85551ff700 1 -- 172.21.15.101:0/3399929926 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f85380a1410 msgr2=0x7f85380a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:06.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.115+0000 7f85551ff700 1 --2- 172.21.15.101:0/3399929926 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f85380a1410 0x7f85380a38c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8548009ec0 tx=0x7f8548009920 comp rx=0 tx=0).stop 2023-07-18T03:24:06.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.115+0000 7f85551ff700 1 -- 172.21.15.101:0/3399929926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8550102900 msgr2=0x7f8550074f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:06.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.115+0000 7f85551ff700 1 --2- 172.21.15.101:0/3399929926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8550102900 0x7f8550074f60 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f854400b4d0 tx=0x7f854400b7e0 comp rx=0 tx=0).stop 2023-07-18T03:24:06.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.115+0000 7f85551ff700 1 -- 172.21.15.101:0/3399929926 shutdown_connections 2023-07-18T03:24:06.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.115+0000 7f85551ff700 1 --2- 172.21.15.101:0/3399929926 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8550106fa0 0x7f85500754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:06.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.115+0000 7f85551ff700 1 --2- 172.21.15.101:0/3399929926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8550101f60 0x7f8550074a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:06.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.115+0000 7f85551ff700 1 --2- 172.21.15.101:0/3399929926 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f85380a1410 0x7f85380a38c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:06.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.115+0000 7f85551ff700 1 --2- 172.21.15.101:0/3399929926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8550102900 0x7f8550074f60 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:06.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.115+0000 7f85551ff700 1 -- 172.21.15.101:0/3399929926 >> 172.21.15.101:0/3399929926 conn(0x7f85500fd7d0 msgr2=0x7f8550105900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:06.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.115+0000 7f85551ff700 1 -- 172.21.15.101:0/3399929926 shutdown_connections 2023-07-18T03:24:06.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.115+0000 7f85551ff700 1 -- 172.21.15.101:0/3399929926 wait complete. 2023-07-18T03:24:06.134 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-18T03:24:06.134 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:24:06.472 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.474+0000 7f45175ff700 1 -- 172.21.15.101:0/2581273808 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45101015b0 msgr2=0x7f4510109aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:06.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.474+0000 7f45175ff700 1 --2- 172.21.15.101:0/2581273808 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45101015b0 0x7f4510109aa0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f450c0098f0 tx=0x7f450c009c00 comp rx=0 tx=0).stop 2023-07-18T03:24:06.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.475+0000 7f45175ff700 1 -- 172.21.15.101:0/2581273808 shutdown_connections 2023-07-18T03:24:06.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.475+0000 7f45175ff700 1 --2- 172.21.15.101:0/2581273808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f451010a070 0x7f451010c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:06.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.475+0000 7f45175ff700 1 --2- 172.21.15.101:0/2581273808 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4510100ca0 0x7f4510101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:06.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.475+0000 7f45175ff700 1 --2- 172.21.15.101:0/2581273808 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45101015b0 0x7f4510109aa0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:06.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.475+0000 7f45175ff700 1 -- 172.21.15.101:0/2581273808 >> 172.21.15.101:0/2581273808 conn(0x7f45100fa490 msgr2=0x7f45100fc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:06.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.475+0000 7f45175ff700 1 -- 172.21.15.101:0/2581273808 shutdown_connections 2023-07-18T03:24:06.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.475+0000 7f45175ff700 1 -- 172.21.15.101:0/2581273808 wait complete. 2023-07-18T03:24:06.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.476+0000 7f45175ff700 1 Processor -- start 2023-07-18T03:24:06.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.476+0000 7f45175ff700 1 -- start start 2023-07-18T03:24:06.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.477+0000 7f45175ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4510100ca0 0x7f45101a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:06.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.477+0000 7f45175ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f45101015b0 0x7f45101a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:06.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.477+0000 7f45175ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f451010a070 0x7f4510108710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:06.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.477+0000 7f45175ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4510108e50 con 0x7f45101015b0 2023-07-18T03:24:06.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.477+0000 7f45175ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4510108fc0 con 0x7f451010a070 2023-07-18T03:24:06.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.477+0000 7f45175ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4510109180 con 0x7f4510100ca0 2023-07-18T03:24:06.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.477+0000 7f451539b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4510100ca0 0x7f45101a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:06.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.477+0000 7f4515b9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f451010a070 0x7f4510108710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:06.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.477+0000 7f451539b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4510100ca0 0x7f45101a0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:36978/0 (socket says 172.21.15.101:36978) 2023-07-18T03:24:06.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.477+0000 7f4515b9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f451010a070 0x7f4510108710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:40994/0 (socket says 172.21.15.101:40994) 2023-07-18T03:24:06.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.477+0000 7f451539b700 1 -- 172.21.15.101:0/2315423489 learned_addr learned my addr 172.21.15.101:0/2315423489 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:06.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.477+0000 7f4514b9a700 1 --2- 172.21.15.101:0/2315423489 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f45101015b0 0x7f45101a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:06.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.478+0000 7f451539b700 1 -- 172.21.15.101:0/2315423489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f451010a070 msgr2=0x7f4510108710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:06.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.478+0000 7f451539b700 1 --2- 172.21.15.101:0/2315423489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f451010a070 0x7f4510108710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:06.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.478+0000 7f451539b700 1 -- 172.21.15.101:0/2315423489 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f45101015b0 msgr2=0x7f45101a14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:06.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.478+0000 7f451539b700 1 --2- 172.21.15.101:0/2315423489 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f45101015b0 0x7f45101a14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:06.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.478+0000 7f451539b700 1 -- 172.21.15.101:0/2315423489 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f450c009520 con 0x7f4510100ca0 2023-07-18T03:24:06.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.478+0000 7f4514b9a700 1 --2- 172.21.15.101:0/2315423489 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f45101015b0 0x7f45101a14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:24:06.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.478+0000 7f4515b9c700 1 --2- 172.21.15.101:0/2315423489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f451010a070 0x7f4510108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:24:06.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.478+0000 7f451539b700 1 --2- 172.21.15.101:0/2315423489 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4510100ca0 0x7f45101a0f80 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f450000b530 tx=0x7f450000b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:06.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.478+0000 7f45067fc700 1 -- 172.21.15.101:0/2315423489 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4500011830 con 0x7f4510100ca0 2023-07-18T03:24:06.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.479+0000 7f45175ff700 1 -- 172.21.15.101:0/2315423489 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4510109460 con 0x7f4510100ca0 2023-07-18T03:24:06.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.479+0000 7f45067fc700 1 -- 172.21.15.101:0/2315423489 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4500011e50 con 0x7f4510100ca0 2023-07-18T03:24:06.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.479+0000 7f45067fc700 1 -- 172.21.15.101:0/2315423489 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450000f2e0 con 0x7f4510100ca0 2023-07-18T03:24:06.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.479+0000 7f45175ff700 1 -- 172.21.15.101:0/2315423489 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45101a9130 con 0x7f4510100ca0 2023-07-18T03:24:06.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.480+0000 7f45175ff700 1 -- 172.21.15.101:0/2315423489 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4510067300 con 0x7f4510100ca0 2023-07-18T03:24:06.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.485+0000 7f45067fc700 1 -- 172.21.15.101:0/2315423489 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f4500010380 con 0x7f4510100ca0 2023-07-18T03:24:06.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.487+0000 7f45067fc700 1 --2- 172.21.15.101:0/2315423489 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f44fc0a1380 0x7f44fc0a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:06.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.487+0000 7f45067fc700 1 -- 172.21.15.101:0/2315423489 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f45000ccd30 con 0x7f4510100ca0 2023-07-18T03:24:06.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.488+0000 7f4514b9a700 1 --2- 172.21.15.101:0/2315423489 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f44fc0a1380 0x7f44fc0a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:06.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.489+0000 7f45067fc700 1 -- 172.21.15.101:0/2315423489 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4500095e50 con 0x7f4510100ca0 2023-07-18T03:24:06.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.490+0000 7f4514b9a700 1 --2- 172.21.15.101:0/2315423489 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f44fc0a1380 0x7f44fc0a3830 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f451019da10 tx=0x7f450c019040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:06.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:06 smithi101 ceph-mon[126379]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:24:06.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:06 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:24:06.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:06 smithi155 ceph-mon[128885]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:24:06.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:06 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:24:06.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.876+0000 7f45175ff700 1 -- 172.21.15.101:0/2315423489 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4510109c40 con 0x7f4510100ca0 2023-07-18T03:24:06.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.879+0000 7f45067fc700 1 -- 172.21.15.101:0/2315423489 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+10863 (secure 0 0 0) 0x7f4500015020 con 0x7f4510100ca0 2023-07-18T03:24:06.876 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:24:06.877 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/4192846769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":4192846769},{"type":"v1","addr":"172.21.15.101:6839","nonce":4192846769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6841/3824340175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6840","nonce":3824340175},{"type":"v1","addr":"172.21.15.101:6841","nonce":3824340175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:04:59.361185+0000","modified":"2023-07-18T03:24:05.188519+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.155:6839/2055462483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2055462483},{"type":"v1","addr":"172.21.15.155:6839","nonce":2055462483}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:24:06.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.883+0000 7f45175ff700 1 -- 172.21.15.101:0/2315423489 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f44fc0a1380 msgr2=0x7f44fc0a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:06.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.883+0000 7f45175ff700 1 --2- 172.21.15.101:0/2315423489 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f44fc0a1380 0x7f44fc0a3830 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f451019da10 tx=0x7f450c019040 comp rx=0 tx=0).stop 2023-07-18T03:24:06.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.883+0000 7f45175ff700 1 -- 172.21.15.101:0/2315423489 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4510100ca0 msgr2=0x7f45101a0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:06.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.883+0000 7f45175ff700 1 --2- 172.21.15.101:0/2315423489 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4510100ca0 0x7f45101a0f80 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f450000b530 tx=0x7f450000b840 comp rx=0 tx=0).stop 2023-07-18T03:24:06.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.884+0000 7f45175ff700 1 -- 172.21.15.101:0/2315423489 shutdown_connections 2023-07-18T03:24:06.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.884+0000 7f45175ff700 1 --2- 172.21.15.101:0/2315423489 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f44fc0a1380 0x7f44fc0a3830 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:06.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.884+0000 7f45175ff700 1 --2- 172.21.15.101:0/2315423489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f451010a070 0x7f4510108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:06.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.884+0000 7f45175ff700 1 --2- 172.21.15.101:0/2315423489 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4510100ca0 0x7f45101a0f80 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:06.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.884+0000 7f45175ff700 1 --2- 172.21.15.101:0/2315423489 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f45101015b0 0x7f45101a14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:06.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.884+0000 7f45175ff700 1 -- 172.21.15.101:0/2315423489 >> 172.21.15.101:0/2315423489 conn(0x7f45100fa490 msgr2=0x7f45100fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:06.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.884+0000 7f45175ff700 1 -- 172.21.15.101:0/2315423489 shutdown_connections 2023-07-18T03:24:06.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:06.884+0000 7f45175ff700 1 -- 172.21.15.101:0/2315423489 wait complete. 2023-07-18T03:24:06.886 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 41 2023-07-18T03:24:06.915 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-18T03:24:06.996 INFO:tasks.workunit.client.0.smithi101.stdout: 10 118358 29.25 MB/sec execute 91 sec latency 678.143 ms 2023-07-18T03:24:07.023 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:06 smithi179 ceph-mon[129229]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:24:07.023 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:06 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:24:07.207 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.210+0000 7ffa407ff700 1 -- 172.21.15.101:0/1713682336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa38101490 msgr2=0x7ffa38109b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:07.207 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.210+0000 7ffa407ff700 1 --2- 172.21.15.101:0/1713682336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa38101490 0x7ffa38109b60 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ffa28009990 tx=0x7ffa28009ca0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.207 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.210+0000 7ffa407ff700 1 -- 172.21.15.101:0/1713682336 shutdown_connections 2023-07-18T03:24:07.207 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.210+0000 7ffa407ff700 1 --2- 172.21.15.101:0/1713682336 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa3810a0a0 0x7ffa3810c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.208 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.210+0000 7ffa407ff700 1 --2- 172.21.15.101:0/1713682336 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffa38100b80 0x7ffa38100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.208 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.210+0000 7ffa407ff700 1 --2- 172.21.15.101:0/1713682336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa38101490 0x7ffa38109b60 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.208 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.210+0000 7ffa407ff700 1 -- 172.21.15.101:0/1713682336 >> 172.21.15.101:0/1713682336 conn(0x7ffa380fa4b0 msgr2=0x7ffa380fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:07.208 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.210+0000 7ffa407ff700 1 -- 172.21.15.101:0/1713682336 shutdown_connections 2023-07-18T03:24:07.208 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.210+0000 7ffa407ff700 1 -- 172.21.15.101:0/1713682336 wait complete. 2023-07-18T03:24:07.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.212+0000 7ffa407ff700 1 Processor -- start 2023-07-18T03:24:07.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.212+0000 7ffa407ff700 1 -- start start 2023-07-18T03:24:07.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.212+0000 7ffa407ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffa38100b80 0x7ffa38197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.213+0000 7ffa407ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa38101490 0x7ffa38198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.213+0000 7ffa407ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa3810a0a0 0x7ffa3819c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.213+0000 7ffa407ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3819ceb0 con 0x7ffa38100b80 2023-07-18T03:24:07.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.213+0000 7ffa407ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3819d020 con 0x7ffa3810a0a0 2023-07-18T03:24:07.210 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.213+0000 7ffa407ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3819d1e0 con 0x7ffa38101490 2023-07-18T03:24:07.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.213+0000 7ffa3dd9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa38101490 0x7ffa38198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.213+0000 7ffa3dd9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa38101490 0x7ffa38198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:36998/0 (socket says 172.21.15.101:36998) 2023-07-18T03:24:07.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.213+0000 7ffa3dd9a700 1 -- 172.21.15.101:0/2886174936 learned_addr learned my addr 172.21.15.101:0/2886174936 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:07.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.213+0000 7ffa3ed9c700 1 --2- 172.21.15.101:0/2886174936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa3810a0a0 0x7ffa3819c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.213+0000 7ffa3e59b700 1 --2- 172.21.15.101:0/2886174936 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffa38100b80 0x7ffa38197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.211 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.214+0000 7ffa3dd9a700 1 -- 172.21.15.101:0/2886174936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa3810a0a0 msgr2=0x7ffa3819c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:07.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.214+0000 7ffa3dd9a700 1 --2- 172.21.15.101:0/2886174936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa3810a0a0 0x7ffa3819c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.214+0000 7ffa3dd9a700 1 -- 172.21.15.101:0/2886174936 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffa38100b80 msgr2=0x7ffa38197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:07.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.214+0000 7ffa3dd9a700 1 --2- 172.21.15.101:0/2886174936 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffa38100b80 0x7ffa38197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.214+0000 7ffa3dd9a700 1 -- 172.21.15.101:0/2886174936 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa280095f0 con 0x7ffa38101490 2023-07-18T03:24:07.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.214+0000 7ffa3dd9a700 1 --2- 172.21.15.101:0/2886174936 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa38101490 0x7ffa38198410 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ffa28005910 tx=0x7ffa28005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:07.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.217+0000 7ffa2f7fe700 1 -- 172.21.15.101:0/2886174936 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2801d070 con 0x7ffa38101490 2023-07-18T03:24:07.214 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.218+0000 7ffa407ff700 1 -- 172.21.15.101:0/2886174936 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa3819d460 con 0x7ffa38101490 2023-07-18T03:24:07.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.218+0000 7ffa2f7fe700 1 -- 172.21.15.101:0/2886174936 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa28022470 con 0x7ffa38101490 2023-07-18T03:24:07.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.218+0000 7ffa2f7fe700 1 -- 172.21.15.101:0/2886174936 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa28017650 con 0x7ffa38101490 2023-07-18T03:24:07.215 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.218+0000 7ffa407ff700 1 -- 172.21.15.101:0/2886174936 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa381a4d00 con 0x7ffa38101490 2023-07-18T03:24:07.219 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.223+0000 7ffa2f7fe700 1 -- 172.21.15.101:0/2886174936 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7ffa2800fad0 con 0x7ffa38101490 2023-07-18T03:24:07.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.224+0000 7ffa2f7fe700 1 --2- 172.21.15.101:0/2886174936 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ffa240a1340 0x7ffa240a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.224+0000 7ffa2f7fe700 1 -- 172.21.15.101:0/2886174936 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9549+0+0 (secure 0 0 0) 0x7ffa280ce4d0 con 0x7ffa38101490 2023-07-18T03:24:07.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.224+0000 7ffa3e59b700 1 --2- 172.21.15.101:0/2886174936 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ffa240a1340 0x7ffa240a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.225+0000 7ffa3e59b700 1 --2- 172.21.15.101:0/2886174936 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ffa240a1340 0x7ffa240a37f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ffa34005950 tx=0x7ffa3400b410 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:07.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.225+0000 7ffa5404eb80 1 Processor -- start 2023-07-18T03:24:07.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.226+0000 7ffa5404eb80 1 -- start start 2023-07-18T03:24:07.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.226+0000 7ffa5404eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e49684b2d0 0x55e49684b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.226+0000 7ffa5404eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e49684bbe0 0x55e496856590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.226+0000 7ffa5404eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e496856ad0 0x55e496858eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.226+0000 7ffa5404eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e496751cb0 con 0x55e49684bbe0 2023-07-18T03:24:07.223 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.226+0000 7ffa5404eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4967a4a40 con 0x55e496856ad0 2023-07-18T03:24:07.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.226+0000 7ffa5404eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e49679ded0 con 0x55e49684b2d0 2023-07-18T03:24:07.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.226+0000 7ffa3dd9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e49684bbe0 0x55e496856590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.226+0000 7ffa3ed9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e496856ad0 0x55e496858eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.226+0000 7ffa3ed9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e496856ad0 0x55e496858eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:41028/0 (socket says 172.21.15.101:41028) 2023-07-18T03:24:07.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.226+0000 7ffa3ed9c700 1 -- 172.21.15.101:0/3170717785 learned_addr learned my addr 172.21.15.101:0/3170717785 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:07.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.227+0000 7ffa3ed9c700 1 -- 172.21.15.101:0/3170717785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e49684b2d0 msgr2=0x55e49684b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:07.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.227+0000 7ffa3ed9c700 1 --2- 172.21.15.101:0/3170717785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e49684b2d0 0x55e49684b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.227+0000 7ffa3ed9c700 1 -- 172.21.15.101:0/3170717785 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e49684bbe0 msgr2=0x55e496856590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:07.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.227+0000 7ffa3ed9c700 1 --2- 172.21.15.101:0/3170717785 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e49684bbe0 0x55e496856590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.227+0000 7ffa3ed9c700 1 -- 172.21.15.101:0/3170717785 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4965cbde0 con 0x55e496856ad0 2023-07-18T03:24:07.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.227+0000 7ffa3ed9c700 1 --2- 172.21.15.101:0/3170717785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e496856ad0 0x55e496858eb0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ffa30011600 tx=0x7ffa30009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aafcefb8dbcd3d0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:07.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.228+0000 7ffa2cff9700 1 -- 172.21.15.101:0/3170717785 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3000e040 con 0x55e496856ad0 2023-07-18T03:24:07.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.228+0000 7ffa2cff9700 1 -- 172.21.15.101:0/3170717785 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa3000f040 con 0x55e496856ad0 2023-07-18T03:24:07.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.228+0000 7ffa2cff9700 1 -- 172.21.15.101:0/3170717785 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa30004800 con 0x55e496856ad0 2023-07-18T03:24:07.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.229+0000 7ffa5404eb80 1 -- 172.21.15.101:0/3170717785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e496856ad0 msgr2=0x55e496858eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:07.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.229+0000 7ffa5404eb80 1 --2- 172.21.15.101:0/3170717785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e496856ad0 0x55e496858eb0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ffa30011600 tx=0x7ffa30009920 comp rx=0 tx=0).stop 2023-07-18T03:24:07.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.229+0000 7ffa5404eb80 1 -- 172.21.15.101:0/3170717785 shutdown_connections 2023-07-18T03:24:07.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.229+0000 7ffa5404eb80 1 --2- 172.21.15.101:0/3170717785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e496856ad0 0x55e496858eb0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.229+0000 7ffa5404eb80 1 --2- 172.21.15.101:0/3170717785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e49684b2d0 0x55e49684b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.229+0000 7ffa5404eb80 1 --2- 172.21.15.101:0/3170717785 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e49684bbe0 0x55e496856590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.229+0000 7ffa5404eb80 1 -- 172.21.15.101:0/3170717785 >> 172.21.15.101:0/3170717785 conn(0x55e496841da0 msgr2=0x55e4968476a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:07.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.229+0000 7ffa5404eb80 1 -- 172.21.15.101:0/3170717785 shutdown_connections 2023-07-18T03:24:07.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.230+0000 7ffa5404eb80 1 -- 172.21.15.101:0/3170717785 wait complete. 2023-07-18T03:24:07.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.230+0000 7ffa5404eb80 1 -- start start 2023-07-18T03:24:07.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.231+0000 7ffa5404eb80 1 Processor -- start 2023-07-18T03:24:07.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.231+0000 7ffa5404eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e49684b2d0 0x55e496856540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.231+0000 7ffa5404eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e49684bbe0 0x55e4968537c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.231+0000 7ffa5404eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e496856ad0 0x55e496853eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.231+0000 7ffa5404eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e49673dac0 con 0x55e49684bbe0 2023-07-18T03:24:07.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.231+0000 7ffa5404eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e49673d7c0 con 0x55e49684b2d0 2023-07-18T03:24:07.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.231+0000 7ffa5404eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e496621920 con 0x55e496856ad0 2023-07-18T03:24:07.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.231+0000 7ffa5404eb80 10 client.0 _ll_register_callbacks cb 0x55e49673b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:24:07.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.232+0000 7ffa3ed9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e496856ad0 0x55e496853eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.232+0000 7ffa3ed9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e496856ad0 0x55e496853eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:37004/0 (socket says 172.21.15.101:37004) 2023-07-18T03:24:07.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.232+0000 7ffa3ed9c700 1 -- 172.21.15.101:0/1289370550 learned_addr learned my addr 172.21.15.101:0/1289370550 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:07.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.232+0000 7ffa3e59b700 1 --2- 172.21.15.101:0/1289370550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e49684b2d0 0x55e496856540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.232+0000 7ffa3ed9c700 1 -- 172.21.15.101:0/1289370550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e49684b2d0 msgr2=0x55e496856540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:07.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.232+0000 7ffa3ed9c700 1 --2- 172.21.15.101:0/1289370550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e49684b2d0 0x55e496856540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.232+0000 7ffa3ed9c700 1 -- 172.21.15.101:0/1289370550 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e49684bbe0 msgr2=0x55e4968537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:07.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.232+0000 7ffa3ed9c700 1 --2- 172.21.15.101:0/1289370550 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e49684bbe0 0x55e4968537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.232+0000 7ffa3ed9c700 1 -- 172.21.15.101:0/1289370550 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa300112b0 con 0x55e496856ad0 2023-07-18T03:24:07.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.233+0000 7ffa3ed9c700 1 --2- 172.21.15.101:0/1289370550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e496856ad0 0x55e496853eb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ffa30011f60 tx=0x7ffa30005ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbdb54a189b8efa9 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:07.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.233+0000 7ffa21ffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:24:07.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.233+0000 7ffa21ffb700 1 -- 172.21.15.101:0/1289370550 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3000e070 con 0x55e496856ad0 2023-07-18T03:24:07.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.233+0000 7ffa5404eb80 1 -- 172.21.15.101:0/1289370550 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e4966f74f0 con 0x55e496856ad0 2023-07-18T03:24:07.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.233+0000 7ffa21ffb700 1 -- 172.21.15.101:0/1289370550 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9549+0+0 (secure 0 0 0) 0x7ffa30003c10 con 0x55e496856ad0 2023-07-18T03:24:07.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.234+0000 7ffa21ffb700 1 -- 172.21.15.101:0/1289370550 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa30003dd0 con 0x55e496856ad0 2023-07-18T03:24:07.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.234+0000 7ffa5404eb80 10 client.28765 fetch_fsmap learned FSMap version 41 2023-07-18T03:24:07.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.234+0000 7ffa5404eb80 1 -- 172.21.15.101:0/1289370550 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55e4965b4c60 con 0x55e496856ad0 2023-07-18T03:24:07.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.234+0000 7ffa21ffb700 1 -- 172.21.15.101:0/1289370550 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e4966f74f0 con 0x55e496856ad0 2023-07-18T03:24:07.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.234+0000 7ffa21ffb700 1 -- 172.21.15.101:0/1289370550 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 41) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ffa30027460 con 0x55e496856ad0 2023-07-18T03:24:07.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.235+0000 7ffa5404eb80 10 client.28765 fetch_fsmap finished waiting for FSMap version 41 2023-07-18T03:24:07.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.235+0000 7ffa5404eb80 10 client.28765 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:24:07.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.235+0000 7ffa5404eb80 20 client.28765 populate_metadata read hostname 'smithi101' 2023-07-18T03:24:07.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.235+0000 7ffa5404eb80 1 --2- 172.21.15.101:0/1289370550 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55e496866300 0x55e496868710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.235+0000 7ffa5404eb80 4 client.28765 mds_command: new command op to 24541 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:24:07.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.235+0000 7ffa5404eb80 1 -- 172.21.15.101:0/1289370550 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e496796eb0 con 0x55e496866300 2023-07-18T03:24:07.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.235+0000 7ffa3e59b700 1 --2- 172.21.15.101:0/1289370550 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55e496866300 0x55e496868710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.236+0000 7ffa21ffb700 10 client.28765 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:24:07.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.236+0000 7ffa3e59b700 1 --2- 172.21.15.101:0/1289370550 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55e496866300 0x55e496868710 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=950b7f18f75eeca7 server_cookie=55616203834f7db0 in_seq=0 out_seq=0 2023-07-18T03:24:07.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.236+0000 7ffa21ffb700 1 -- 172.21.15.101:0/1289370550 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e496796eb0 con 0x55e496866300 2023-07-18T03:24:07.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.236+0000 7ffa21ffb700 10 client.28765 handle_command_reply: tid=0 2023-07-18T03:24:07.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.236+0000 7ffa5404eb80 1 client.28765 shutdown 2023-07-18T03:24:07.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.237+0000 7ffa5404eb80 1 -- 172.21.15.101:0/1289370550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e496856ad0 msgr2=0x55e496853eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:07.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.237+0000 7ffa5404eb80 1 --2- 172.21.15.101:0/1289370550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e496856ad0 0x55e496853eb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ffa30011f60 tx=0x7ffa30005ae0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.237+0000 7ffa5404eb80 1 -- 172.21.15.101:0/1289370550 shutdown_connections 2023-07-18T03:24:07.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.237+0000 7ffa5404eb80 1 --2- 172.21.15.101:0/1289370550 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55e496866300 0x55e496868710 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.237+0000 7ffa5404eb80 1 --2- 172.21.15.101:0/1289370550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e49684b2d0 0x55e496856540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.237+0000 7ffa5404eb80 1 --2- 172.21.15.101:0/1289370550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e496856ad0 0x55e496853eb0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.237+0000 7ffa5404eb80 1 --2- 172.21.15.101:0/1289370550 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e49684bbe0 0x55e4968537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.237+0000 7ffa5404eb80 1 -- 172.21.15.101:0/1289370550 >> 172.21.15.101:0/1289370550 conn(0x55e496841da0 msgr2=0x55e496847230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:07.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.237+0000 7ffa21ffb700 0 client.28765 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:24:07.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.237+0000 7ffa5404eb80 1 -- 172.21.15.101:0/1289370550 shutdown_connections 2023-07-18T03:24:07.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.237+0000 7ffa5404eb80 1 -- 172.21.15.101:0/1289370550 wait complete. 2023-07-18T03:24:07.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.237+0000 7ffa5404eb80 20 client.28765 trim_cache size 0 max 16384 2023-07-18T03:24:07.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.252+0000 7ffa5404eb80 1 Processor -- start 2023-07-18T03:24:07.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.252+0000 7ffa5404eb80 1 -- start start 2023-07-18T03:24:07.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.252+0000 7ffa5404eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e49684b2d0 0x55e496856070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.252+0000 7ffa5404eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e49684bbe0 0x55e49684ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.252+0000 7ffa5404eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e496856ad0 0x55e49684f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.253+0000 7ffa5404eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e49684c540 con 0x55e49684b2d0 2023-07-18T03:24:07.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.253+0000 7ffa5404eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e496848330 con 0x55e49684bbe0 2023-07-18T03:24:07.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.253+0000 7ffa5404eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e49684e2f0 con 0x55e496856ad0 2023-07-18T03:24:07.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.253+0000 7ffa3dd9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e49684bbe0 0x55e49684ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.253+0000 7ffa3e59b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e49684b2d0 0x55e496856070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.253+0000 7ffa3dd9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e49684bbe0 0x55e49684ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:41050/0 (socket says 172.21.15.101:41050) 2023-07-18T03:24:07.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.253+0000 7ffa3e59b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e49684b2d0 0x55e496856070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:54630/0 (socket says 172.21.15.101:54630) 2023-07-18T03:24:07.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.253+0000 7ffa3dd9a700 1 -- 172.21.15.101:0/3424637776 learned_addr learned my addr 172.21.15.101:0/3424637776 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:07.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.253+0000 7ffa3ed9c700 1 --2- 172.21.15.101:0/3424637776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e496856ad0 0x55e49684f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.253+0000 7ffa3e59b700 1 -- 172.21.15.101:0/3424637776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e496856ad0 msgr2=0x55e49684f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:07.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.253+0000 7ffa3e59b700 1 --2- 172.21.15.101:0/3424637776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e496856ad0 0x55e49684f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.253+0000 7ffa3e59b700 1 -- 172.21.15.101:0/3424637776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e49684bbe0 msgr2=0x55e49684ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:07.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.253+0000 7ffa3e59b700 1 --2- 172.21.15.101:0/3424637776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e49684bbe0 0x55e49684ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.253+0000 7ffa3e59b700 1 -- 172.21.15.101:0/3424637776 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4965f2250 con 0x55e49684b2d0 2023-07-18T03:24:07.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.253+0000 7ffa3dd9a700 1 --2- 172.21.15.101:0/3424637776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e49684bbe0 0x55e49684ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:24:07.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.254+0000 7ffa3e59b700 1 --2- 172.21.15.101:0/3424637776 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e49684b2d0 0x55e496856070 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x55e496868fb0 tx=0x7ffa34013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b48712574d70da29 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:07.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.254+0000 7ffa2d7fa700 1 -- 172.21.15.101:0/3424637776 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3402a070 con 0x55e49684b2d0 2023-07-18T03:24:07.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.254+0000 7ffa2d7fa700 1 -- 172.21.15.101:0/3424637776 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa34015070 con 0x55e49684b2d0 2023-07-18T03:24:07.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.254+0000 7ffa2d7fa700 1 -- 172.21.15.101:0/3424637776 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa34025070 con 0x55e49684b2d0 2023-07-18T03:24:07.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.255+0000 7ffa5404eb80 1 -- 172.21.15.101:0/3424637776 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e49684b2d0 msgr2=0x55e496856070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:07.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.255+0000 7ffa5404eb80 1 --2- 172.21.15.101:0/3424637776 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e49684b2d0 0x55e496856070 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x55e496868fb0 tx=0x7ffa34013000 comp rx=0 tx=0).stop 2023-07-18T03:24:07.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.255+0000 7ffa5404eb80 1 -- 172.21.15.101:0/3424637776 shutdown_connections 2023-07-18T03:24:07.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.255+0000 7ffa5404eb80 1 --2- 172.21.15.101:0/3424637776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e496856ad0 0x55e49684f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.255+0000 7ffa5404eb80 1 --2- 172.21.15.101:0/3424637776 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e49684b2d0 0x55e496856070 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.255+0000 7ffa5404eb80 1 --2- 172.21.15.101:0/3424637776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e49684bbe0 0x55e49684ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.255+0000 7ffa5404eb80 1 -- 172.21.15.101:0/3424637776 >> 172.21.15.101:0/3424637776 conn(0x55e496841da0 msgr2=0x55e49686d030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:07.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.255+0000 7ffa5404eb80 1 -- 172.21.15.101:0/3424637776 shutdown_connections 2023-07-18T03:24:07.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.255+0000 7ffa5404eb80 1 -- 172.21.15.101:0/3424637776 wait complete. 2023-07-18T03:24:07.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.256+0000 7ffa5404eb80 1 -- start start 2023-07-18T03:24:07.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.256+0000 7ffa5404eb80 1 Processor -- start 2023-07-18T03:24:07.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.256+0000 7ffa5404eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e49684b2d0 0x55e49685cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.256+0000 7ffa5404eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e49684bbe0 0x55e49685d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.256+0000 7ffa5404eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e496856ad0 0x55e49685b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.256+0000 7ffa5404eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e49685b870 con 0x55e49684bbe0 2023-07-18T03:24:07.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.256+0000 7ffa5404eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e49685b9e0 con 0x55e496856ad0 2023-07-18T03:24:07.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.256+0000 7ffa5404eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e49685bba0 con 0x55e49684b2d0 2023-07-18T03:24:07.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.256+0000 7ffa5404eb80 10 client.0 _ll_register_callbacks cb 0x7ffa280057b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:24:07.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.257+0000 7ffa3e59b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e49684b2d0 0x55e49685cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.257+0000 7ffa3e59b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e49684b2d0 0x55e49685cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:37012/0 (socket says 172.21.15.101:37012) 2023-07-18T03:24:07.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.257+0000 7ffa3e59b700 1 -- 172.21.15.101:0/3965381760 learned_addr learned my addr 172.21.15.101:0/3965381760 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:07.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.257+0000 7ffa3dd9a700 1 --2- 172.21.15.101:0/3965381760 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e49684bbe0 0x55e49685d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.257+0000 7ffa3ed9c700 1 --2- 172.21.15.101:0/3965381760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e496856ad0 0x55e49685b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.257+0000 7ffa3e59b700 1 -- 172.21.15.101:0/3965381760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e496856ad0 msgr2=0x55e49685b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:07.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.257+0000 7ffa3e59b700 1 --2- 172.21.15.101:0/3965381760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e496856ad0 0x55e49685b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.257+0000 7ffa3e59b700 1 -- 172.21.15.101:0/3965381760 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e49684bbe0 msgr2=0x55e49685d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:07.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.257+0000 7ffa3e59b700 1 --2- 172.21.15.101:0/3965381760 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e49684bbe0 0x55e49685d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.257+0000 7ffa3e59b700 1 -- 172.21.15.101:0/3965381760 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa381a4d00 con 0x55e49684b2d0 2023-07-18T03:24:07.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.257+0000 7ffa3dd9a700 1 --2- 172.21.15.101:0/3965381760 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e49684bbe0 0x55e49685d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:24:07.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.258+0000 7ffa3e59b700 1 --2- 172.21.15.101:0/3965381760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e49684b2d0 0x55e49685cdd0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ffa3400c010 tx=0x7ffa34027bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcc4d9b18c5034c0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:07.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.258+0000 7ffa21ffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:24:07.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.258+0000 7ffa21ffb700 1 -- 172.21.15.101:0/3965381760 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3402a040 con 0x55e49684b2d0 2023-07-18T03:24:07.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.258+0000 7ffa21ffb700 1 -- 172.21.15.101:0/3965381760 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9549+0+0 (secure 0 0 0) 0x7ffa34017070 con 0x55e49684b2d0 2023-07-18T03:24:07.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.258+0000 7ffa5404eb80 1 -- 172.21.15.101:0/3965381760 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffa3000f040 con 0x55e49684b2d0 2023-07-18T03:24:07.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.259+0000 7ffa21ffb700 1 -- 172.21.15.101:0/3965381760 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa34025040 con 0x55e49684b2d0 2023-07-18T03:24:07.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.259+0000 7ffa5404eb80 10 client.28771 fetch_fsmap learned FSMap version 41 2023-07-18T03:24:07.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.259+0000 7ffa5404eb80 1 -- 172.21.15.101:0/3965381760 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7ffa34010e10 con 0x55e49684b2d0 2023-07-18T03:24:07.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.259+0000 7ffa21ffb700 1 -- 172.21.15.101:0/3965381760 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa3000f040 con 0x55e49684b2d0 2023-07-18T03:24:07.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.260+0000 7ffa21ffb700 1 -- 172.21.15.101:0/3965381760 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 41) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ffa34004bc0 con 0x55e49684b2d0 2023-07-18T03:24:07.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.260+0000 7ffa5404eb80 10 client.28771 fetch_fsmap finished waiting for FSMap version 41 2023-07-18T03:24:07.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.260+0000 7ffa5404eb80 10 client.28771 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:24:07.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.260+0000 7ffa5404eb80 20 client.28771 populate_metadata read hostname 'smithi101' 2023-07-18T03:24:07.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.260+0000 7ffa5404eb80 1 --2- 172.21.15.101:0/3965381760 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55e4968bb2c0 0x55e4968bd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.260+0000 7ffa5404eb80 4 client.28771 mds_command: new command op to 24541 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:24:07.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.260+0000 7ffa5404eb80 1 -- 172.21.15.101:0/3965381760 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e496759050 con 0x55e4968bb2c0 2023-07-18T03:24:07.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.260+0000 7ffa3dd9a700 1 --2- 172.21.15.101:0/3965381760 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55e4968bb2c0 0x55e4968bd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.261+0000 7ffa3dd9a700 1 --2- 172.21.15.101:0/3965381760 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55e4968bb2c0 0x55e4968bd6d0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a6268b79c086353d server_cookie=2fd9d6b24dd61b7b in_seq=0 out_seq=0 2023-07-18T03:24:07.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.261+0000 7ffa21ffb700 10 client.28771 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:24:07.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.265+0000 7ffa21ffb700 1 -- 172.21.15.101:0/3965381760 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e496759050 con 0x55e4968bb2c0 2023-07-18T03:24:07.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.265+0000 7ffa21ffb700 10 client.28771 handle_command_reply: tid=0 2023-07-18T03:24:07.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.265+0000 7ffa5404eb80 1 client.28771 shutdown 2023-07-18T03:24:07.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.265+0000 7ffa5404eb80 1 -- 172.21.15.101:0/3965381760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e49684b2d0 msgr2=0x55e49685cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:07.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.265+0000 7ffa5404eb80 1 --2- 172.21.15.101:0/3965381760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e49684b2d0 0x55e49685cdd0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ffa3400c010 tx=0x7ffa34027bc0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.266+0000 7ffa5404eb80 1 -- 172.21.15.101:0/3965381760 shutdown_connections 2023-07-18T03:24:07.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.266+0000 7ffa5404eb80 1 --2- 172.21.15.101:0/3965381760 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x55e4968bb2c0 0x55e4968bd6d0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.266+0000 7ffa5404eb80 1 --2- 172.21.15.101:0/3965381760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e496856ad0 0x55e49685b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.266+0000 7ffa5404eb80 1 --2- 172.21.15.101:0/3965381760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e49684b2d0 0x55e49685cdd0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.266+0000 7ffa5404eb80 1 --2- 172.21.15.101:0/3965381760 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e49684bbe0 0x55e49685d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.266+0000 7ffa5404eb80 1 -- 172.21.15.101:0/3965381760 >> 172.21.15.101:0/3965381760 conn(0x55e496841da0 msgr2=0x55e49684ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:07.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.266+0000 7ffa21ffb700 0 client.28771 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:24:07.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.266+0000 7ffa5404eb80 1 -- 172.21.15.101:0/3965381760 shutdown_connections 2023-07-18T03:24:07.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.266+0000 7ffa5404eb80 1 -- 172.21.15.101:0/3965381760 wait complete. 2023-07-18T03:24:07.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.267+0000 7ffa5404eb80 20 client.28771 trim_cache size 0 max 16384 2023-07-18T03:24:07.266 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:24:07.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.269+0000 7ffa407ff700 1 -- 172.21.15.101:0/2886174936 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ffa240a1340 msgr2=0x7ffa240a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:07.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.269+0000 7ffa407ff700 1 --2- 172.21.15.101:0/2886174936 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ffa240a1340 0x7ffa240a37f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ffa34005950 tx=0x7ffa3400b410 comp rx=0 tx=0).stop 2023-07-18T03:24:07.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.269+0000 7ffa407ff700 1 -- 172.21.15.101:0/2886174936 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa38101490 msgr2=0x7ffa38198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:07.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.269+0000 7ffa407ff700 1 --2- 172.21.15.101:0/2886174936 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa38101490 0x7ffa38198410 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ffa28005910 tx=0x7ffa28005740 comp rx=0 tx=0).stop 2023-07-18T03:24:07.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.269+0000 7ffa407ff700 1 -- 172.21.15.101:0/2886174936 shutdown_connections 2023-07-18T03:24:07.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.269+0000 7ffa407ff700 1 --2- 172.21.15.101:0/2886174936 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ffa240a1340 0x7ffa240a37f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.269+0000 7ffa407ff700 1 --2- 172.21.15.101:0/2886174936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa3810a0a0 0x7ffa3819c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.269+0000 7ffa407ff700 1 --2- 172.21.15.101:0/2886174936 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffa38100b80 0x7ffa38197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.269+0000 7ffa407ff700 1 --2- 172.21.15.101:0/2886174936 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa38101490 0x7ffa38198410 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.269+0000 7ffa407ff700 1 -- 172.21.15.101:0/2886174936 >> 172.21.15.101:0/2886174936 conn(0x7ffa380fa4b0 msgr2=0x7ffa3810c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:07.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.271+0000 7ffa407ff700 1 -- 172.21.15.101:0/2886174936 shutdown_connections 2023-07-18T03:24:07.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.271+0000 7ffa407ff700 1 -- 172.21.15.101:0/2886174936 wait complete. 2023-07-18T03:24:07.292 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-18T03:24:07.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:07 smithi101 ceph-mon[126379]: pgmap v519: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 18 MiB/s wr, 838 op/s 2023-07-18T03:24:07.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:07 smithi101 ceph-mon[126379]: osdmap e113: 12 total, 12 up, 12 in 2023-07-18T03:24:07.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:07 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2315423489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:24:07.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:07 smithi155 ceph-mon[128885]: pgmap v519: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 18 MiB/s wr, 838 op/s 2023-07-18T03:24:07.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:07 smithi155 ceph-mon[128885]: osdmap e113: 12 total, 12 up, 12 in 2023-07-18T03:24:07.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:07 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/2315423489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:24:07.549 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.552+0000 7f20471ff700 1 -- 172.21.15.101:0/2428875764 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f204010c3a0 msgr2=0x7f204010e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:07.549 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.552+0000 7f20471ff700 1 --2- 172.21.15.101:0/2428875764 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f204010c3a0 0x7f204010e780 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f203800b0a0 tx=0x7f203800b3b0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.550 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.553+0000 7f20471ff700 1 -- 172.21.15.101:0/2428875764 shutdown_connections 2023-07-18T03:24:07.550 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.553+0000 7f20471ff700 1 --2- 172.21.15.101:0/2428875764 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f204010c3a0 0x7f204010e780 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.550 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.553+0000 7f20471ff700 1 --2- 172.21.15.101:0/2428875764 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2040072830 0x7f2040072c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.550 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.553+0000 7f20471ff700 1 --2- 172.21.15.101:0/2428875764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2040073140 0x7f204010be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.550 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.553+0000 7f20471ff700 1 -- 172.21.15.101:0/2428875764 >> 172.21.15.101:0/2428875764 conn(0x7f20400fb5e0 msgr2=0x7f20400fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:07.550 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.553+0000 7f20471ff700 1 -- 172.21.15.101:0/2428875764 shutdown_connections 2023-07-18T03:24:07.550 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.553+0000 7f20471ff700 1 -- 172.21.15.101:0/2428875764 wait complete. 2023-07-18T03:24:07.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.554+0000 7f20471ff700 1 Processor -- start 2023-07-18T03:24:07.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.554+0000 7f20471ff700 1 -- start start 2023-07-18T03:24:07.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.554+0000 7f20471ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2040072830 0x7f2040197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.554+0000 7f20471ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2040073140 0x7f2040198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.553 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.554+0000 7f20471ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f204010c3a0 0x7f204019c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.553 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.554+0000 7f20471ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204019cee0 con 0x7f2040073140 2023-07-18T03:24:07.553 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.554+0000 7f20471ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204019d050 con 0x7f204010c3a0 2023-07-18T03:24:07.553 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.554+0000 7f20471ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204019d210 con 0x7f2040072830 2023-07-18T03:24:07.553 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.554+0000 7f203ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2040073140 0x7f2040198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.553 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.554+0000 7f2044f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2040072830 0x7f2040197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.553 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.554+0000 7f204579c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f204010c3a0 0x7f204019c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.553 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.554+0000 7f204579c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f204010c3a0 0x7f204019c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:41070/0 (socket says 172.21.15.101:41070) 2023-07-18T03:24:07.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.554+0000 7f204579c700 1 -- 172.21.15.101:0/1541285195 learned_addr learned my addr 172.21.15.101:0/1541285195 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:07.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.554+0000 7f2044f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2040072830 0x7f2040197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:37036/0 (socket says 172.21.15.101:37036) 2023-07-18T03:24:07.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.555+0000 7f203ffff700 1 -- 172.21.15.101:0/1541285195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2040072830 msgr2=0x7f2040197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:07.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.555+0000 7f203ffff700 1 --2- 172.21.15.101:0/1541285195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2040072830 0x7f2040197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.555+0000 7f203ffff700 1 -- 172.21.15.101:0/1541285195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f204010c3a0 msgr2=0x7f204019c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:07.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.555+0000 7f203ffff700 1 --2- 172.21.15.101:0/1541285195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f204010c3a0 0x7f204019c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.555+0000 7f203ffff700 1 -- 172.21.15.101:0/1541285195 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f202c009520 con 0x7f2040073140 2023-07-18T03:24:07.555 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.555+0000 7f204579c700 1 --2- 172.21.15.101:0/1541285195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f204010c3a0 0x7f204019c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:24:07.555 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.555+0000 7f203ffff700 1 --2- 172.21.15.101:0/1541285195 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2040073140 0x7f2040198440 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f202c00e9f0 tx=0x7f202c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:07.555 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.555+0000 7f203dffb700 1 -- 172.21.15.101:0/1541285195 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c00cb70 con 0x7f2040073140 2023-07-18T03:24:07.555 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.555+0000 7f20471ff700 1 -- 172.21.15.101:0/1541285195 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2038009d00 con 0x7f2040073140 2023-07-18T03:24:07.555 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.555+0000 7f203dffb700 1 -- 172.21.15.101:0/1541285195 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f202c00ee90 con 0x7f2040073140 2023-07-18T03:24:07.555 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.555+0000 7f203dffb700 1 -- 172.21.15.101:0/1541285195 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c003cf0 con 0x7f2040073140 2023-07-18T03:24:07.555 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.555+0000 7f20471ff700 1 -- 172.21.15.101:0/1541285195 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f204019d6e0 con 0x7f2040073140 2023-07-18T03:24:07.556 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.559+0000 7f203dffb700 1 -- 172.21.15.101:0/1541285195 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f202c00ccd0 con 0x7f2040073140 2023-07-18T03:24:07.557 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.561+0000 7f205abd1b80 1 Processor -- start 2023-07-18T03:24:07.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.561+0000 7f203dffb700 1 --2- 172.21.15.101:0/1541285195 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f20300a12c0 0x7f20300a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.561+0000 7f205abd1b80 1 -- start start 2023-07-18T03:24:07.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.562+0000 7f203dffb700 1 -- 172.21.15.101:0/1541285195 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f202c014070 con 0x7f2040073140 2023-07-18T03:24:07.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.562+0000 7f204579c700 1 --2- 172.21.15.101:0/1541285195 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f20300a12c0 0x7f20300a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.562+0000 7f205abd1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a387d42d0 0x559a387d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.562+0000 7f205abd1b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a387d4be0 0x559a387df590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.562+0000 7f205abd1b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559a387dfad0 0x559a387e1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.562+0000 7f205abd1b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a386dacb0 con 0x559a387dfad0 2023-07-18T03:24:07.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.562+0000 7f205abd1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a3872da40 con 0x559a387d42d0 2023-07-18T03:24:07.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.562+0000 7f205abd1b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a38726ed0 con 0x559a387d4be0 2023-07-18T03:24:07.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.563+0000 7f2044f9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a387d42d0 0x559a387d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.563+0000 7f2044f9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a387d42d0 0x559a387d46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:41084/0 (socket says 172.21.15.101:41084) 2023-07-18T03:24:07.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.563+0000 7f2044f9b700 1 -- 172.21.15.101:0/2882911651 learned_addr learned my addr 172.21.15.101:0/2882911651 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:07.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.563+0000 7f203ffff700 1 --2- 172.21.15.101:0/2882911651 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a387d4be0 0x559a387df590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.563+0000 7f204579c700 1 --2- 172.21.15.101:0/2882911651 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559a387dfad0 0x559a387e1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.562 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.563+0000 7f2044f9b700 1 -- 172.21.15.101:0/2882911651 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a387d4be0 msgr2=0x559a387df590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:07.562 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.563+0000 7f2044f9b700 1 --2- 172.21.15.101:0/2882911651 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a387d4be0 0x559a387df590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.562 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.563+0000 7f2044f9b700 1 -- 172.21.15.101:0/2882911651 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559a387dfad0 msgr2=0x559a387e1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:07.562 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.563+0000 7f2044f9b700 1 --2- 172.21.15.101:0/2882911651 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559a387dfad0 0x559a387e1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.562 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.563+0000 7f2044f9b700 1 -- 172.21.15.101:0/2882911651 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a38554de0 con 0x559a387d42d0 2023-07-18T03:24:07.562 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.563+0000 7f204579c700 1 --2- 172.21.15.101:0/1541285195 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f20300a12c0 0x7f20300a3770 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f203800b780 tx=0x7f2038009780 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:07.562 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.563+0000 7f203ffff700 1 --2- 172.21.15.101:0/2882911651 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a387d4be0 0x559a387df590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:24:07.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.564+0000 7f2044f9b700 1 --2- 172.21.15.101:0/2882911651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a387d42d0 0x559a387d46a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f2034009d80 tx=0x7f2034011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f969ba0032c1d79e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:07.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.564+0000 7f202affd700 1 -- 172.21.15.101:0/2882911651 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2034004230 con 0x559a387d42d0 2023-07-18T03:24:07.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.564+0000 7f202affd700 1 -- 172.21.15.101:0/2882911651 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2034004390 con 0x559a387d42d0 2023-07-18T03:24:07.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.564+0000 7f202affd700 1 -- 172.21.15.101:0/2882911651 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2034019880 con 0x559a387d42d0 2023-07-18T03:24:07.564 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.565+0000 7f205abd1b80 1 -- 172.21.15.101:0/2882911651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a387d42d0 msgr2=0x559a387d46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:07.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.565+0000 7f205abd1b80 1 --2- 172.21.15.101:0/2882911651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a387d42d0 0x559a387d46a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f2034009d80 tx=0x7f2034011720 comp rx=0 tx=0).stop 2023-07-18T03:24:07.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.565+0000 7f205abd1b80 1 -- 172.21.15.101:0/2882911651 shutdown_connections 2023-07-18T03:24:07.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.565+0000 7f205abd1b80 1 --2- 172.21.15.101:0/2882911651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a387d42d0 0x559a387d46a0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.565+0000 7f205abd1b80 1 --2- 172.21.15.101:0/2882911651 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559a387dfad0 0x559a387e1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.565+0000 7f205abd1b80 1 --2- 172.21.15.101:0/2882911651 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a387d4be0 0x559a387df590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.565+0000 7f205abd1b80 1 -- 172.21.15.101:0/2882911651 >> 172.21.15.101:0/2882911651 conn(0x559a387cada0 msgr2=0x559a387d06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:07.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.566+0000 7f205abd1b80 1 -- 172.21.15.101:0/2882911651 shutdown_connections 2023-07-18T03:24:07.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.566+0000 7f205abd1b80 1 -- 172.21.15.101:0/2882911651 wait complete. 2023-07-18T03:24:07.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.566+0000 7f205abd1b80 1 -- start start 2023-07-18T03:24:07.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.567+0000 7f205abd1b80 1 Processor -- start 2023-07-18T03:24:07.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.567+0000 7f205abd1b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559a387d42d0 0x559a387df4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.567+0000 7f205abd1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a387d4be0 0x559a387dc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.567+0000 7f205abd1b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a387dfad0 0x559a387dce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.567+0000 7f205abd1b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a386c6ac0 con 0x559a387d42d0 2023-07-18T03:24:07.567 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.567+0000 7f205abd1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a386c67c0 con 0x559a387d4be0 2023-07-18T03:24:07.567 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.567+0000 7f205abd1b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a385aa920 con 0x559a387dfad0 2023-07-18T03:24:07.567 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.567+0000 7f205abd1b80 10 client.0 _ll_register_callbacks cb 0x559a386c4760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:24:07.567 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.567+0000 7f204579c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a387dfad0 0x559a387dce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.567 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.567+0000 7f204579c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a387dfad0 0x559a387dce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:37048/0 (socket says 172.21.15.101:37048) 2023-07-18T03:24:07.567 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.567+0000 7f204579c700 1 -- 172.21.15.101:0/3392144465 learned_addr learned my addr 172.21.15.101:0/3392144465 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:07.567 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.567+0000 7f2044f9b700 1 --2- 172.21.15.101:0/3392144465 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559a387d42d0 0x559a387df4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.568+0000 7f203ffff700 1 --2- 172.21.15.101:0/3392144465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a387d4be0 0x559a387dc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.568+0000 7f204579c700 1 -- 172.21.15.101:0/3392144465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a387d4be0 msgr2=0x559a387dc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:07.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.568+0000 7f204579c700 1 --2- 172.21.15.101:0/3392144465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a387d4be0 0x559a387dc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.568+0000 7f204579c700 1 -- 172.21.15.101:0/3392144465 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559a387d42d0 msgr2=0x559a387df4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:07.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.568+0000 7f204579c700 1 --2- 172.21.15.101:0/3392144465 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559a387d42d0 0x559a387df4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.568+0000 7f204579c700 1 -- 172.21.15.101:0/3392144465 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20340099e0 con 0x559a387dfad0 2023-07-18T03:24:07.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.568+0000 7f204579c700 1 --2- 172.21.15.101:0/3392144465 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a387dfad0 0x559a387dce70 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2038003d80 tx=0x7f20380099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79cd63b8cf6843a9 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:07.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.568+0000 7f201ffff700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:24:07.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.569+0000 7f201ffff700 1 -- 172.21.15.101:0/3392144465 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203800e040 con 0x559a387dfad0 2023-07-18T03:24:07.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.569+0000 7f205abd1b80 1 -- 172.21.15.101:0/3392144465 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559a386804f0 con 0x559a387dfad0 2023-07-18T03:24:07.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.569+0000 7f201ffff700 1 -- 172.21.15.101:0/3392144465 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f2038005050 con 0x559a387dfad0 2023-07-18T03:24:07.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.569+0000 7f201ffff700 1 -- 172.21.15.101:0/3392144465 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2038005210 con 0x559a387dfad0 2023-07-18T03:24:07.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.569+0000 7f205abd1b80 10 client.28783 fetch_fsmap learned FSMap version 41 2023-07-18T03:24:07.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.569+0000 7f205abd1b80 1 -- 172.21.15.101:0/3392144465 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x559a3853dc60 con 0x559a387dfad0 2023-07-18T03:24:07.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.569+0000 7f201ffff700 1 -- 172.21.15.101:0/3392144465 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559a386804f0 con 0x559a387dfad0 2023-07-18T03:24:07.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.570+0000 7f201ffff700 1 -- 172.21.15.101:0/3392144465 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 41) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f20380296e0 con 0x559a387dfad0 2023-07-18T03:24:07.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.570+0000 7f205abd1b80 10 client.28783 fetch_fsmap finished waiting for FSMap version 41 2023-07-18T03:24:07.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.570+0000 7f205abd1b80 10 client.28783 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:24:07.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.570+0000 7f205abd1b80 20 client.28783 populate_metadata read hostname 'smithi101' 2023-07-18T03:24:07.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.570+0000 7f205abd1b80 1 --2- 172.21.15.101:0/3392144465 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x559a387ef2a0 0x559a387f16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.570+0000 7f205abd1b80 4 client.28783 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:24:07.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.570+0000 7f205abd1b80 1 -- 172.21.15.101:0/3392144465 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559a3871feb0 con 0x559a387ef2a0 2023-07-18T03:24:07.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.572+0000 7f2044f9b700 1 --2- 172.21.15.101:0/3392144465 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x559a387ef2a0 0x559a387f16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.573+0000 7f2044f9b700 1 --2- 172.21.15.101:0/3392144465 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x559a387ef2a0 0x559a387f16b0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c3201b8a5562ec87 server_cookie=ae9074e96198083f in_seq=0 out_seq=0 2023-07-18T03:24:07.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.573+0000 7f201ffff700 10 client.28783 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:24:07.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.576+0000 7f201ffff700 1 -- 172.21.15.101:0/3392144465 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559a3871feb0 con 0x559a387ef2a0 2023-07-18T03:24:07.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.576+0000 7f201ffff700 10 client.28783 handle_command_reply: tid=0 2023-07-18T03:24:07.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.576+0000 7f205abd1b80 1 client.28783 shutdown 2023-07-18T03:24:07.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.576+0000 7f205abd1b80 1 -- 172.21.15.101:0/3392144465 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a387dfad0 msgr2=0x559a387dce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:07.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.576+0000 7f205abd1b80 1 --2- 172.21.15.101:0/3392144465 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a387dfad0 0x559a387dce70 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2038003d80 tx=0x7f20380099e0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.576+0000 7f205abd1b80 1 -- 172.21.15.101:0/3392144465 shutdown_connections 2023-07-18T03:24:07.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.576+0000 7f205abd1b80 1 --2- 172.21.15.101:0/3392144465 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x559a387ef2a0 0x559a387f16b0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.576+0000 7f205abd1b80 1 --2- 172.21.15.101:0/3392144465 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a387dfad0 0x559a387dce70 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.576+0000 7f205abd1b80 1 --2- 172.21.15.101:0/3392144465 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559a387d42d0 0x559a387df4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.576+0000 7f201ffff700 0 client.28783 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:24:07.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.576+0000 7f205abd1b80 1 --2- 172.21.15.101:0/3392144465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a387d4be0 0x559a387dc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.576+0000 7f205abd1b80 1 -- 172.21.15.101:0/3392144465 >> 172.21.15.101:0/3392144465 conn(0x559a387cada0 msgr2=0x559a387d0230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:07.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.577+0000 7f205abd1b80 1 -- 172.21.15.101:0/3392144465 shutdown_connections 2023-07-18T03:24:07.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.577+0000 7f205abd1b80 1 -- 172.21.15.101:0/3392144465 wait complete. 2023-07-18T03:24:07.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.577+0000 7f205abd1b80 20 client.28783 trim_cache size 0 max 16384 2023-07-18T03:24:07.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.595+0000 7f205abd1b80 1 Processor -- start 2023-07-18T03:24:07.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.595+0000 7f205abd1b80 1 -- start start 2023-07-18T03:24:07.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.595+0000 7f205abd1b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a387d42d0 0x559a387d7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.595+0000 7f205abd1b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559a387d4be0 0x559a387d7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.595+0000 7f205abd1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a387dfad0 0x559a387dbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.595+0000 7f205abd1b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a387d5d90 con 0x559a387d4be0 2023-07-18T03:24:07.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.595+0000 7f205abd1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a387c7fa0 con 0x559a387dfad0 2023-07-18T03:24:07.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.595+0000 7f205abd1b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a387ca970 con 0x559a387d42d0 2023-07-18T03:24:07.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.596+0000 7f2044f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a387d42d0 0x559a387d7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.596+0000 7f2044f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a387d42d0 0x559a387d7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:37056/0 (socket says 172.21.15.101:37056) 2023-07-18T03:24:07.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.596+0000 7f2044f9b700 1 -- 172.21.15.101:0/2307135699 learned_addr learned my addr 172.21.15.101:0/2307135699 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:07.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.596+0000 7f203ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559a387d4be0 0x559a387d7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.596+0000 7f2044f9b700 1 -- 172.21.15.101:0/2307135699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a387dfad0 msgr2=0x559a387dbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:07.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.596+0000 7f2044f9b700 1 --2- 172.21.15.101:0/2307135699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a387dfad0 0x559a387dbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.596+0000 7f2044f9b700 1 -- 172.21.15.101:0/2307135699 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559a387d4be0 msgr2=0x559a387d7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:07.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.596+0000 7f2044f9b700 1 --2- 172.21.15.101:0/2307135699 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559a387d4be0 0x559a387d7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.596+0000 7f2044f9b700 1 -- 172.21.15.101:0/2307135699 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a3857b250 con 0x559a387d42d0 2023-07-18T03:24:07.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.596+0000 7f203ffff700 1 --2- 172.21.15.101:0/2307135699 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559a387d4be0 0x559a387d7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:24:07.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.596+0000 7f2044f9b700 1 --2- 172.21.15.101:0/2307135699 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a387d42d0 0x559a387d7310 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f2034009d80 tx=0x7f2034009710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b035180e9aab21a0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:07.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.597+0000 7f202b7fe700 1 -- 172.21.15.101:0/2307135699 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2034029070 con 0x559a387d42d0 2023-07-18T03:24:07.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.597+0000 7f202b7fe700 1 -- 172.21.15.101:0/2307135699 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f203402e020 con 0x559a387d42d0 2023-07-18T03:24:07.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.597+0000 7f202b7fe700 1 -- 172.21.15.101:0/2307135699 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2034005280 con 0x559a387d42d0 2023-07-18T03:24:07.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.597+0000 7f205abd1b80 1 -- 172.21.15.101:0/2307135699 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a387d42d0 msgr2=0x559a387d7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:07.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.597+0000 7f205abd1b80 1 --2- 172.21.15.101:0/2307135699 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a387d42d0 0x559a387d7310 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f2034009d80 tx=0x7f2034009710 comp rx=0 tx=0).stop 2023-07-18T03:24:07.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.597+0000 7f205abd1b80 1 -- 172.21.15.101:0/2307135699 shutdown_connections 2023-07-18T03:24:07.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.597+0000 7f205abd1b80 1 --2- 172.21.15.101:0/2307135699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a387dfad0 0x559a387dbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.597+0000 7f205abd1b80 1 --2- 172.21.15.101:0/2307135699 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a387d42d0 0x559a387d7310 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.597+0000 7f205abd1b80 1 --2- 172.21.15.101:0/2307135699 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559a387d4be0 0x559a387d7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.598+0000 7f205abd1b80 1 -- 172.21.15.101:0/2307135699 >> 172.21.15.101:0/2307135699 conn(0x559a387cada0 msgr2=0x559a387eb660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:07.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.598+0000 7f205abd1b80 1 -- 172.21.15.101:0/2307135699 shutdown_connections 2023-07-18T03:24:07.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.598+0000 7f205abd1b80 1 -- 172.21.15.101:0/2307135699 wait complete. 2023-07-18T03:24:07.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.598+0000 7f205abd1b80 1 -- start start 2023-07-18T03:24:07.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.598+0000 7f205abd1b80 1 Processor -- start 2023-07-18T03:24:07.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.599+0000 7f205abd1b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559a387d42d0 0x559a387e61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.599+0000 7f205abd1b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a387d4be0 0x559a387e66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.599+0000 7f205abd1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a387dfad0 0x559a387e4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.599+0000 7f205abd1b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a387e4d80 con 0x559a387d42d0 2023-07-18T03:24:07.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.599+0000 7f205abd1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a387e4ef0 con 0x559a387dfad0 2023-07-18T03:24:07.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.599+0000 7f205abd1b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a387e50b0 con 0x559a387d4be0 2023-07-18T03:24:07.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.599+0000 7f205abd1b80 10 client.0 _ll_register_callbacks cb 0x7f203800bf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:24:07.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.599+0000 7f2044f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559a387d42d0 0x559a387e61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.599+0000 7f203ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a387d4be0 0x559a387e66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.599+0000 7f203ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a387d4be0 0x559a387e66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:37064/0 (socket says 172.21.15.101:37064) 2023-07-18T03:24:07.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.599+0000 7f203ffff700 1 -- 172.21.15.101:0/1458887767 learned_addr learned my addr 172.21.15.101:0/1458887767 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:07.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.600+0000 7f204579c700 1 --2- 172.21.15.101:0/1458887767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a387dfad0 0x559a387e4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.600+0000 7f203ffff700 1 -- 172.21.15.101:0/1458887767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a387dfad0 msgr2=0x559a387e4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:07.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.600+0000 7f203ffff700 1 --2- 172.21.15.101:0/1458887767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a387dfad0 0x559a387e4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.600+0000 7f203ffff700 1 -- 172.21.15.101:0/1458887767 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559a387d42d0 msgr2=0x559a387e61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:07.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.600+0000 7f203ffff700 1 --2- 172.21.15.101:0/1458887767 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559a387d42d0 0x559a387e61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.600+0000 7f203ffff700 1 -- 172.21.15.101:0/1458887767 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f204019d6e0 con 0x559a387d4be0 2023-07-18T03:24:07.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.600+0000 7f203ffff700 1 --2- 172.21.15.101:0/1458887767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a387d4be0 0x559a387e66f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f202c00c5b0 tx=0x7f202c099a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a22b6a1717d9232f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:07.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.600+0000 7f201ffff700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:24:07.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.600+0000 7f201ffff700 1 -- 172.21.15.101:0/1458887767 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c09a470 con 0x559a387d4be0 2023-07-18T03:24:07.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.600+0000 7f201ffff700 1 -- 172.21.15.101:0/1458887767 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f202c0a2460 con 0x559a387d4be0 2023-07-18T03:24:07.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.600+0000 7f205abd1b80 1 -- 172.21.15.101:0/1458887767 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2034004390 con 0x559a387d4be0 2023-07-18T03:24:07.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.601+0000 7f201ffff700 1 -- 172.21.15.101:0/1458887767 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f202c099500 con 0x559a387d4be0 2023-07-18T03:24:07.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.601+0000 7f205abd1b80 10 client.28789 fetch_fsmap learned FSMap version 41 2023-07-18T03:24:07.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.601+0000 7f205abd1b80 1 -- 172.21.15.101:0/1458887767 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x559a38554de0 con 0x559a387d4be0 2023-07-18T03:24:07.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.601+0000 7f201ffff700 1 -- 172.21.15.101:0/1458887767 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2034004390 con 0x559a387d4be0 2023-07-18T03:24:07.604 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.601+0000 7f201ffff700 1 -- 172.21.15.101:0/1458887767 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 41) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f202c0a20f0 con 0x559a387d4be0 2023-07-18T03:24:07.604 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.601+0000 7f205abd1b80 10 client.28789 fetch_fsmap finished waiting for FSMap version 41 2023-07-18T03:24:07.604 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.601+0000 7f205abd1b80 10 client.28789 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:24:07.604 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.601+0000 7f205abd1b80 20 client.28789 populate_metadata read hostname 'smithi101' 2023-07-18T03:24:07.604 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.601+0000 7f205abd1b80 1 --2- 172.21.15.101:0/1458887767 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x559a38844500 0x559a38846910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.604 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.601+0000 7f205abd1b80 4 client.28789 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:24:07.604 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.601+0000 7f205abd1b80 1 -- 172.21.15.101:0/1458887767 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559a386e2050 con 0x559a38844500 2023-07-18T03:24:07.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.602+0000 7f2044f9b700 1 --2- 172.21.15.101:0/1458887767 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x559a38844500 0x559a38846910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.602+0000 7f2044f9b700 1 --2- 172.21.15.101:0/1458887767 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x559a38844500 0x559a38846910 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5cfe9c602b9ffb62 server_cookie=c983f520c699246 in_seq=0 out_seq=0 2023-07-18T03:24:07.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.602+0000 7f201ffff700 10 client.28789 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:24:07.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.604+0000 7f201ffff700 1 -- 172.21.15.101:0/1458887767 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559a386e2050 con 0x559a38844500 2023-07-18T03:24:07.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.604+0000 7f201ffff700 10 client.28789 handle_command_reply: tid=0 2023-07-18T03:24:07.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.604+0000 7f205abd1b80 1 client.28789 shutdown 2023-07-18T03:24:07.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.605+0000 7f205abd1b80 1 -- 172.21.15.101:0/1458887767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a387d4be0 msgr2=0x559a387e66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:07.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.605+0000 7f205abd1b80 1 --2- 172.21.15.101:0/1458887767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a387d4be0 0x559a387e66f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f202c00c5b0 tx=0x7f202c099a80 comp rx=0 tx=0).stop 2023-07-18T03:24:07.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.605+0000 7f205abd1b80 1 -- 172.21.15.101:0/1458887767 shutdown_connections 2023-07-18T03:24:07.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.605+0000 7f205abd1b80 1 --2- 172.21.15.101:0/1458887767 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x559a38844500 0x559a38846910 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.605+0000 7f205abd1b80 1 --2- 172.21.15.101:0/1458887767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a387dfad0 0x559a387e4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.605+0000 7f205abd1b80 1 --2- 172.21.15.101:0/1458887767 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559a387d42d0 0x559a387e61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.605+0000 7f201ffff700 0 client.28789 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:24:07.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.605+0000 7f205abd1b80 1 --2- 172.21.15.101:0/1458887767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a387d4be0 0x559a387e66f0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.605+0000 7f205abd1b80 1 -- 172.21.15.101:0/1458887767 >> 172.21.15.101:0/1458887767 conn(0x559a387cada0 msgr2=0x559a387d7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:07.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.605+0000 7f205abd1b80 1 -- 172.21.15.101:0/1458887767 shutdown_connections 2023-07-18T03:24:07.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.605+0000 7f205abd1b80 1 -- 172.21.15.101:0/1458887767 wait complete. 2023-07-18T03:24:07.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.606+0000 7f205abd1b80 20 client.28789 trim_cache size 0 max 16384 2023-07-18T03:24:07.607 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:24:07.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.606+0000 7f20471ff700 1 -- 172.21.15.101:0/1541285195 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f20300a12c0 msgr2=0x7f20300a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:07.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.606+0000 7f20471ff700 1 --2- 172.21.15.101:0/1541285195 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f20300a12c0 0x7f20300a3770 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f203800b780 tx=0x7f2038009780 comp rx=0 tx=0).stop 2023-07-18T03:24:07.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.606+0000 7f20471ff700 1 -- 172.21.15.101:0/1541285195 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2040073140 msgr2=0x7f2040198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:07.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.606+0000 7f20471ff700 1 --2- 172.21.15.101:0/1541285195 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2040073140 0x7f2040198440 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f202c00e9f0 tx=0x7f202c00ed00 comp rx=0 tx=0).stop 2023-07-18T03:24:07.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.607+0000 7f20471ff700 1 -- 172.21.15.101:0/1541285195 shutdown_connections 2023-07-18T03:24:07.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.607+0000 7f20471ff700 1 --2- 172.21.15.101:0/1541285195 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f20300a12c0 0x7f20300a3770 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.607+0000 7f20471ff700 1 --2- 172.21.15.101:0/1541285195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f204010c3a0 0x7f204019c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.607+0000 7f20471ff700 1 --2- 172.21.15.101:0/1541285195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2040072830 0x7f2040197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.607+0000 7f20471ff700 1 --2- 172.21.15.101:0/1541285195 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2040073140 0x7f2040198440 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.607+0000 7f20471ff700 1 -- 172.21.15.101:0/1541285195 >> 172.21.15.101:0/1541285195 conn(0x7f20400fb5e0 msgr2=0x7f2040103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:07.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.609+0000 7f20471ff700 1 -- 172.21.15.101:0/1541285195 shutdown_connections 2023-07-18T03:24:07.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.609+0000 7f20471ff700 1 -- 172.21.15.101:0/1541285195 wait complete. 2023-07-18T03:24:07.632 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-18T03:24:07.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:07 smithi179 ceph-mon[129229]: pgmap v519: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 18 MiB/s wr, 838 op/s 2023-07-18T03:24:07.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:07 smithi179 ceph-mon[129229]: osdmap e113: 12 total, 12 up, 12 in 2023-07-18T03:24:07.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:07 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/2315423489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:24:07.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.906+0000 7f4a0dfff700 1 -- 172.21.15.101:0/2908447663 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4a08101f70 msgr2=0x7f4a08102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:07.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.906+0000 7f4a0dfff700 1 --2- 172.21.15.101:0/2908447663 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4a08101f70 0x7f4a08102340 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f49f80099c0 tx=0x7f49f8009cd0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.906+0000 7f4a0dfff700 1 -- 172.21.15.101:0/2908447663 shutdown_connections 2023-07-18T03:24:07.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.906+0000 7f4a0dfff700 1 --2- 172.21.15.101:0/2908447663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a08106fb0 0x7f4a0810eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.906+0000 7f4a0dfff700 1 --2- 172.21.15.101:0/2908447663 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4a08101f70 0x7f4a08102340 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.906+0000 7f4a0dfff700 1 --2- 172.21.15.101:0/2908447663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a08102910 0x7f4a081068d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.906+0000 7f4a0dfff700 1 -- 172.21.15.101:0/2908447663 >> 172.21.15.101:0/2908447663 conn(0x7f4a080fd7e0 msgr2=0x7f4a080ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:07.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.906+0000 7f4a0dfff700 1 -- 172.21.15.101:0/2908447663 shutdown_connections 2023-07-18T03:24:07.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.906+0000 7f4a0dfff700 1 -- 172.21.15.101:0/2908447663 wait complete. 2023-07-18T03:24:07.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.908+0000 7f4a0dfff700 1 Processor -- start 2023-07-18T03:24:07.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.908+0000 7f4a0dfff700 1 -- start start 2023-07-18T03:24:07.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.908+0000 7f4a0dfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a08101f70 0x7f4a0810e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.908+0000 7f4a0dfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a08102910 0x7f4a0810ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.908+0000 7f4a0dfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4a08106fb0 0x7f4a0810b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.908+0000 7f4a0dfff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0810be00 con 0x7f4a08106fb0 2023-07-18T03:24:07.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.908+0000 7f4a0dfff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0810bf70 con 0x7f4a08101f70 2023-07-18T03:24:07.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.908+0000 7f4a0dfff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0810c130 con 0x7f4a08102910 2023-07-18T03:24:07.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.908+0000 7f4a07fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4a08106fb0 0x7f4a0810b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.908+0000 7f4a07fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4a08106fb0 0x7f4a0810b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:54726/0 (socket says 172.21.15.101:54726) 2023-07-18T03:24:07.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.908+0000 7f4a07fff700 1 -- 172.21.15.101:0/4198060999 learned_addr learned my addr 172.21.15.101:0/4198060999 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:07.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.908+0000 7f4a077fe700 1 --2- 172.21.15.101:0/4198060999 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a08101f70 0x7f4a0810e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.909+0000 7f4a06ffd700 1 --2- 172.21.15.101:0/4198060999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a08102910 0x7f4a0810ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.909+0000 7f4a07fff700 1 -- 172.21.15.101:0/4198060999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a08102910 msgr2=0x7f4a0810ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:07.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.909+0000 7f4a07fff700 1 --2- 172.21.15.101:0/4198060999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a08102910 0x7f4a0810ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.909+0000 7f4a07fff700 1 -- 172.21.15.101:0/4198060999 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a08101f70 msgr2=0x7f4a0810e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:07.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.909+0000 7f4a07fff700 1 --2- 172.21.15.101:0/4198060999 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a08101f70 0x7f4a0810e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.909 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.909+0000 7f4a07fff700 1 -- 172.21.15.101:0/4198060999 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f80095f0 con 0x7f4a08106fb0 2023-07-18T03:24:07.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.909+0000 7f4a07fff700 1 --2- 172.21.15.101:0/4198060999 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4a08106fb0 0x7f4a0810b630 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f4a00007ee0 tx=0x7f4a0000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:07.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.910+0000 7f4a04ff9700 1 -- 172.21.15.101:0/4198060999 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0000ddd0 con 0x7f4a08106fb0 2023-07-18T03:24:07.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.910+0000 7f4a04ff9700 1 -- 172.21.15.101:0/4198060999 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a0000f040 con 0x7f4a08106fb0 2023-07-18T03:24:07.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.910+0000 7f4a04ff9700 1 -- 172.21.15.101:0/4198060999 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a00015400 con 0x7f4a08106fb0 2023-07-18T03:24:07.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.911+0000 7f4a0dfff700 1 -- 172.21.15.101:0/4198060999 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a081a8bd0 con 0x7f4a08106fb0 2023-07-18T03:24:07.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.911+0000 7f4a0dfff700 1 -- 172.21.15.101:0/4198060999 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a081a8f40 con 0x7f4a08106fb0 2023-07-18T03:24:07.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.917+0000 7f4a04ff9700 1 -- 172.21.15.101:0/4198060999 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f4a00003a00 con 0x7f4a08106fb0 2023-07-18T03:24:07.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.917+0000 7f4a21992b80 1 Processor -- start 2023-07-18T03:24:07.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.917+0000 7f4a21992b80 1 -- start start 2023-07-18T03:24:07.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.917+0000 7f4a21992b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5585e88052d0 0x5585e88056a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.918+0000 7f4a21992b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5585e8805be0 0x5585e8810590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.918+0000 7f4a21992b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585e8810ad0 0x5585e8812eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.918+0000 7f4a21992b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585e870bcb0 con 0x5585e88052d0 2023-07-18T03:24:07.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.918+0000 7f4a21992b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585e875ea40 con 0x5585e8810ad0 2023-07-18T03:24:07.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.918+0000 7f4a21992b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585e8757ed0 con 0x5585e8805be0 2023-07-18T03:24:07.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.918+0000 7f4a06ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5585e88052d0 0x5585e88056a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.916 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.918+0000 7f4a06ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5585e88052d0 0x5585e88056a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:54734/0 (socket says 172.21.15.101:54734) 2023-07-18T03:24:07.916 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.918+0000 7f4a06ffd700 1 -- 172.21.15.101:0/2510535979 learned_addr learned my addr 172.21.15.101:0/2510535979 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:07.916 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.918+0000 7f4a06ffd700 1 --2- 172.21.15.101:0/2510535979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585e8810ad0 0x5585e8812eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.916 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.918+0000 7f4a077fe700 1 --2- 172.21.15.101:0/2510535979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5585e8805be0 0x5585e8810590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.918+0000 7f4a06ffd700 1 -- 172.21.15.101:0/2510535979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5585e8805be0 msgr2=0x5585e8810590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:07.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.918+0000 7f4a06ffd700 1 --2- 172.21.15.101:0/2510535979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5585e8805be0 0x5585e8810590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.917 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.918+0000 7f4a06ffd700 1 -- 172.21.15.101:0/2510535979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585e8810ad0 msgr2=0x5585e8812eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:07.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.918+0000 7f4a06ffd700 1 --2- 172.21.15.101:0/2510535979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585e8810ad0 0x5585e8812eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.918+0000 7f4a06ffd700 1 -- 172.21.15.101:0/2510535979 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585e8585de0 con 0x5585e88052d0 2023-07-18T03:24:07.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.919+0000 7f4a04ff9700 1 --2- 172.21.15.101:0/4198060999 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f49f00a12c0 0x7f49f00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.918 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.919+0000 7f4a04ff9700 1 -- 172.21.15.101:0/4198060999 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f4a000cc970 con 0x7f4a08106fb0 2023-07-18T03:24:07.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.919+0000 7f4a06ffd700 1 --2- 172.21.15.101:0/2510535979 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5585e88052d0 0x5585e88056a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f49fc0124c0 tx=0x7f49fc008380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28bf225995d46777 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:07.919 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.919+0000 7f4a077fe700 1 --2- 172.21.15.101:0/4198060999 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f49f00a12c0 0x7f49f00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.919+0000 7f49edffb700 1 -- 172.21.15.101:0/2510535979 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49fc004830 con 0x5585e88052d0 2023-07-18T03:24:07.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.919+0000 7f49edffb700 1 -- 172.21.15.101:0/2510535979 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49fc004e50 con 0x5585e88052d0 2023-07-18T03:24:07.920 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.919+0000 7f49edffb700 1 -- 172.21.15.101:0/2510535979 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49fc010860 con 0x5585e88052d0 2023-07-18T03:24:07.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.920+0000 7f4a21992b80 1 -- 172.21.15.101:0/2510535979 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5585e88052d0 msgr2=0x5585e88056a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:07.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.920+0000 7f4a077fe700 1 --2- 172.21.15.101:0/4198060999 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f49f00a12c0 0x7f49f00a3770 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f49f800b580 tx=0x7f49f8009530 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:07.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.920+0000 7f4a21992b80 1 --2- 172.21.15.101:0/2510535979 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5585e88052d0 0x5585e88056a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f49fc0124c0 tx=0x7f49fc008380 comp rx=0 tx=0).stop 2023-07-18T03:24:07.921 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.920+0000 7f4a21992b80 1 -- 172.21.15.101:0/2510535979 shutdown_connections 2023-07-18T03:24:07.922 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.920+0000 7f4a21992b80 1 --2- 172.21.15.101:0/2510535979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585e8810ad0 0x5585e8812eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.922 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.920+0000 7f4a21992b80 1 --2- 172.21.15.101:0/2510535979 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5585e88052d0 0x5585e88056a0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.922 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.920+0000 7f4a21992b80 1 --2- 172.21.15.101:0/2510535979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5585e8805be0 0x5585e8810590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.923 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.920+0000 7f4a21992b80 1 -- 172.21.15.101:0/2510535979 >> 172.21.15.101:0/2510535979 conn(0x5585e87fbda0 msgr2=0x5585e88016a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:07.923 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.921+0000 7f4a21992b80 1 -- 172.21.15.101:0/2510535979 shutdown_connections 2023-07-18T03:24:07.923 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.921+0000 7f4a21992b80 1 -- 172.21.15.101:0/2510535979 wait complete. 2023-07-18T03:24:07.927 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.921+0000 7f4a21992b80 1 -- start start 2023-07-18T03:24:07.928 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.921+0000 7f4a21992b80 1 Processor -- start 2023-07-18T03:24:07.928 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.923+0000 7f4a21992b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5585e88052d0 0x5585e8810550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.928 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.923+0000 7f4a21992b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585e8805be0 0x5585e880d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.929 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.923+0000 7f4a21992b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5585e8810ad0 0x5585e880de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.929 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.923+0000 7f4a21992b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585e86f7ac0 con 0x5585e88052d0 2023-07-18T03:24:07.929 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.923+0000 7f4a21992b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585e86f77c0 con 0x5585e8805be0 2023-07-18T03:24:07.929 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.923+0000 7f4a21992b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585e85db920 con 0x5585e8810ad0 2023-07-18T03:24:07.930 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.923+0000 7f4a21992b80 10 client.0 _ll_register_callbacks cb 0x5585e86f5760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:24:07.930 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.923+0000 7f4a077fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5585e88052d0 0x5585e8810550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.930 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.923+0000 7f4a06ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585e8805be0 0x5585e880d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.930 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.923+0000 7f4a06ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585e8805be0 0x5585e880d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:41152/0 (socket says 172.21.15.101:41152) 2023-07-18T03:24:07.930 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.923+0000 7f4a06ffd700 1 -- 172.21.15.101:0/3733325054 learned_addr learned my addr 172.21.15.101:0/3733325054 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:07.930 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.924+0000 7f4a06ffd700 1 -- 172.21.15.101:0/3733325054 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5585e8810ad0 msgr2=0x5585e880de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:07.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.924+0000 7f4a06ffd700 1 --2- 172.21.15.101:0/3733325054 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5585e8810ad0 0x5585e880de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.924+0000 7f4a06ffd700 1 -- 172.21.15.101:0/3733325054 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5585e88052d0 msgr2=0x5585e8810550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:07.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.924+0000 7f4a06ffd700 1 --2- 172.21.15.101:0/3733325054 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5585e88052d0 0x5585e8810550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.924+0000 7f4a06ffd700 1 -- 172.21.15.101:0/3733325054 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49fc008d60 con 0x5585e8805be0 2023-07-18T03:24:07.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.924+0000 7f4a077fe700 1 --2- 172.21.15.101:0/3733325054 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5585e88052d0 0x5585e8810550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:24:07.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.924+0000 7f4a06ffd700 1 --2- 172.21.15.101:0/3733325054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585e8805be0 0x5585e880d760 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f49fc005950 tx=0x7f49fc004980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=159ee5d20f893c88 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:07.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.924+0000 7f49e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:24:07.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.926+0000 7f49e6ffd700 1 -- 172.21.15.101:0/3733325054 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49fc0056d0 con 0x5585e8805be0 2023-07-18T03:24:07.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.926+0000 7f49e6ffd700 1 -- 172.21.15.101:0/3733325054 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f49fc004c50 con 0x5585e8805be0 2023-07-18T03:24:07.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.926+0000 7f4a21992b80 1 -- 172.21.15.101:0/3733325054 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5585e86b14f0 con 0x5585e8805be0 2023-07-18T03:24:07.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.927+0000 7f49e6ffd700 1 -- 172.21.15.101:0/3733325054 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49fc010810 con 0x5585e8805be0 2023-07-18T03:24:07.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.927+0000 7f4a21992b80 10 client.28793 fetch_fsmap learned FSMap version 41 2023-07-18T03:24:07.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.927+0000 7f4a21992b80 1 -- 172.21.15.101:0/3733325054 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5585e856ec60 con 0x5585e8805be0 2023-07-18T03:24:07.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.927+0000 7f49e6ffd700 1 -- 172.21.15.101:0/3733325054 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5585e86b14f0 con 0x5585e8805be0 2023-07-18T03:24:07.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.927+0000 7f49e6ffd700 1 -- 172.21.15.101:0/3733325054 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 41) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f49fc0279a0 con 0x5585e8805be0 2023-07-18T03:24:07.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.927+0000 7f4a21992b80 10 client.28793 fetch_fsmap finished waiting for FSMap version 41 2023-07-18T03:24:07.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.927+0000 7f4a21992b80 10 client.28793 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:24:07.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.927+0000 7f4a21992b80 20 client.28793 populate_metadata read hostname 'smithi101' 2023-07-18T03:24:07.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.927+0000 7f4a21992b80 1 --2- 172.21.15.101:0/3733325054 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5585e88203e0 0x5585e88227f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.927+0000 7f4a21992b80 4 client.28793 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:24:07.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.927+0000 7f4a21992b80 1 -- 172.21.15.101:0/3733325054 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5585e8750eb0 con 0x5585e88203e0 2023-07-18T03:24:07.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.928+0000 7f4a077fe700 1 --2- 172.21.15.101:0/3733325054 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5585e88203e0 0x5585e88227f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.934 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.928+0000 7f4a077fe700 1 --2- 172.21.15.101:0/3733325054 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5585e88203e0 0x5585e88227f0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8bfef0afed0f7d server_cookie=d8ad9c08f90b2459 in_seq=0 out_seq=0 2023-07-18T03:24:07.934 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.928+0000 7f49e6ffd700 10 client.28793 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:24:07.935 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.932+0000 7f49e6ffd700 1 -- 172.21.15.101:0/3733325054 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5585e8750eb0 con 0x5585e88203e0 2023-07-18T03:24:07.935 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.932+0000 7f49e6ffd700 10 client.28793 handle_command_reply: tid=0 2023-07-18T03:24:07.935 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.932+0000 7f4a21992b80 1 client.28793 shutdown 2023-07-18T03:24:07.935 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.933+0000 7f4a21992b80 1 -- 172.21.15.101:0/3733325054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585e8805be0 msgr2=0x5585e880d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:07.935 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.933+0000 7f4a21992b80 1 --2- 172.21.15.101:0/3733325054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585e8805be0 0x5585e880d760 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f49fc005950 tx=0x7f49fc004980 comp rx=0 tx=0).stop 2023-07-18T03:24:07.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.933+0000 7f4a21992b80 1 -- 172.21.15.101:0/3733325054 shutdown_connections 2023-07-18T03:24:07.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.933+0000 7f4a21992b80 1 --2- 172.21.15.101:0/3733325054 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5585e88203e0 0x5585e88227f0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.933+0000 7f4a21992b80 1 --2- 172.21.15.101:0/3733325054 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5585e8810ad0 0x5585e880de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.933+0000 7f4a21992b80 1 --2- 172.21.15.101:0/3733325054 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5585e88052d0 0x5585e8810550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.933+0000 7f4a21992b80 1 --2- 172.21.15.101:0/3733325054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585e8805be0 0x5585e880d760 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.933+0000 7f4a21992b80 1 -- 172.21.15.101:0/3733325054 >> 172.21.15.101:0/3733325054 conn(0x5585e87fbda0 msgr2=0x5585e8801290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:07.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.933+0000 7f49e6ffd700 0 client.28793 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:24:07.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.934+0000 7f4a21992b80 1 -- 172.21.15.101:0/3733325054 shutdown_connections 2023-07-18T03:24:07.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.934+0000 7f4a21992b80 1 -- 172.21.15.101:0/3733325054 wait complete. 2023-07-18T03:24:07.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.934+0000 7f4a21992b80 20 client.28793 trim_cache size 0 max 16384 2023-07-18T03:24:07.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.951+0000 7f4a21992b80 1 Processor -- start 2023-07-18T03:24:07.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.952+0000 7f4a21992b80 1 -- start start 2023-07-18T03:24:07.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.952+0000 7f4a21992b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5585e88052d0 0x5585e880de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.952+0000 7f4a21992b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5585e8805be0 0x5585e8827110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.952+0000 7f4a21992b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585e8810ad0 0x5585e8817520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.952+0000 7f4a21992b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585e8827650 con 0x5585e88052d0 2023-07-18T03:24:07.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.952+0000 7f4a21992b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585e8817a60 con 0x5585e8810ad0 2023-07-18T03:24:07.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.952+0000 7f4a21992b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585e880d970 con 0x5585e8805be0 2023-07-18T03:24:07.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.952+0000 7f4a077fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5585e88052d0 0x5585e880de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.952+0000 7f4a07fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585e8810ad0 0x5585e8817520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.952+0000 7f4a06ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5585e8805be0 0x5585e8827110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.952+0000 7f4a07fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585e8810ad0 0x5585e8817520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:41158/0 (socket says 172.21.15.101:41158) 2023-07-18T03:24:07.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.952+0000 7f4a07fff700 1 -- 172.21.15.101:0/2189668518 learned_addr learned my addr 172.21.15.101:0/2189668518 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:07.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.953+0000 7f4a077fe700 1 -- 172.21.15.101:0/2189668518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5585e8805be0 msgr2=0x5585e8827110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:07.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.953+0000 7f4a077fe700 1 --2- 172.21.15.101:0/2189668518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5585e8805be0 0x5585e8827110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.953+0000 7f4a077fe700 1 -- 172.21.15.101:0/2189668518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585e8810ad0 msgr2=0x5585e8817520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:07.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.953+0000 7f4a077fe700 1 --2- 172.21.15.101:0/2189668518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585e8810ad0 0x5585e8817520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.953+0000 7f4a077fe700 1 -- 172.21.15.101:0/2189668518 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585e85ac250 con 0x5585e88052d0 2023-07-18T03:24:07.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.953+0000 7f4a077fe700 1 --2- 172.21.15.101:0/2189668518 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5585e88052d0 0x5585e880de30 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f49f8009f50 tx=0x7f49f8023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d96a32bc34284 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:07.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.953+0000 7f49ee7fc700 1 -- 172.21.15.101:0/2189668518 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f8037360 con 0x5585e88052d0 2023-07-18T03:24:07.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.953+0000 7f49ee7fc700 1 -- 172.21.15.101:0/2189668518 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49f801e040 con 0x5585e88052d0 2023-07-18T03:24:07.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.953+0000 7f49ee7fc700 1 -- 172.21.15.101:0/2189668518 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f8004170 con 0x5585e88052d0 2023-07-18T03:24:07.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.954+0000 7f4a21992b80 1 -- 172.21.15.101:0/2189668518 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5585e88052d0 msgr2=0x5585e880de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:07.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.954+0000 7f4a21992b80 1 --2- 172.21.15.101:0/2189668518 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5585e88052d0 0x5585e880de30 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f49f8009f50 tx=0x7f49f8023000 comp rx=0 tx=0).stop 2023-07-18T03:24:07.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.954+0000 7f4a21992b80 1 -- 172.21.15.101:0/2189668518 shutdown_connections 2023-07-18T03:24:07.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.954+0000 7f4a21992b80 1 --2- 172.21.15.101:0/2189668518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585e8810ad0 0x5585e8817520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.954+0000 7f4a21992b80 1 --2- 172.21.15.101:0/2189668518 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5585e88052d0 0x5585e880de30 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.954+0000 7f4a21992b80 1 --2- 172.21.15.101:0/2189668518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5585e8805be0 0x5585e8827110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.954+0000 7f4a21992b80 1 -- 172.21.15.101:0/2189668518 >> 172.21.15.101:0/2189668518 conn(0x5585e87fbda0 msgr2=0x5585e87ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:07.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.954+0000 7f4a21992b80 1 -- 172.21.15.101:0/2189668518 shutdown_connections 2023-07-18T03:24:07.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.954+0000 7f4a21992b80 1 -- 172.21.15.101:0/2189668518 wait complete. 2023-07-18T03:24:07.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.955+0000 7f4a21992b80 1 -- start start 2023-07-18T03:24:07.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.955+0000 7f4a21992b80 1 Processor -- start 2023-07-18T03:24:07.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.955+0000 7f4a21992b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5585e88052d0 0x5585e880b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.955+0000 7f4a21992b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5585e8805be0 0x5585e8815500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.955+0000 7f4a21992b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585e8810ad0 0x5585e8815c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.955+0000 7f4a21992b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585e8816320 con 0x5585e88052d0 2023-07-18T03:24:07.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.955+0000 7f4a21992b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585e8873260 con 0x5585e8810ad0 2023-07-18T03:24:07.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.955+0000 7f4a21992b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585e88733a0 con 0x5585e8805be0 2023-07-18T03:24:07.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.955+0000 7f4a21992b80 10 client.0 _ll_register_callbacks cb 0x7f49f8009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:24:07.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.956+0000 7f4a07fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585e8810ad0 0x5585e8815c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.956+0000 7f4a07fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585e8810ad0 0x5585e8815c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:41166/0 (socket says 172.21.15.101:41166) 2023-07-18T03:24:07.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.956+0000 7f4a07fff700 1 -- 172.21.15.101:0/999087577 learned_addr learned my addr 172.21.15.101:0/999087577 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:07.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.956+0000 7f4a077fe700 1 --2- 172.21.15.101:0/999087577 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5585e88052d0 0x5585e880b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.956+0000 7f4a06ffd700 1 --2- 172.21.15.101:0/999087577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5585e8805be0 0x5585e8815500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.956+0000 7f4a07fff700 1 -- 172.21.15.101:0/999087577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5585e8805be0 msgr2=0x5585e8815500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:07.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.956+0000 7f4a07fff700 1 --2- 172.21.15.101:0/999087577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5585e8805be0 0x5585e8815500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.956+0000 7f4a07fff700 1 -- 172.21.15.101:0/999087577 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5585e88052d0 msgr2=0x5585e880b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:07.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.956+0000 7f4a07fff700 1 --2- 172.21.15.101:0/999087577 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5585e88052d0 0x5585e880b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.956+0000 7f4a07fff700 1 -- 172.21.15.101:0/999087577 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a0810f3c0 con 0x5585e8810ad0 2023-07-18T03:24:07.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.956+0000 7f4a06ffd700 1 --2- 172.21.15.101:0/999087577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5585e8805be0 0x5585e8815500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:24:07.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.956+0000 7f4a07fff700 1 --2- 172.21.15.101:0/999087577 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585e8810ad0 0x5585e8815c40 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4a00004430 tx=0x7f4a00099490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36851d55607df670 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:07.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.956+0000 7f49e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:24:07.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.956+0000 7f49e6ffd700 1 -- 172.21.15.101:0/999087577 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a00099640 con 0x5585e8810ad0 2023-07-18T03:24:07.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.957+0000 7f4a21992b80 1 -- 172.21.15.101:0/999087577 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f49fc004e50 con 0x5585e8810ad0 2023-07-18T03:24:07.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.957+0000 7f49e6ffd700 1 -- 172.21.15.101:0/999087577 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f4a00082e40 con 0x5585e8810ad0 2023-07-18T03:24:07.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.957+0000 7f49e6ffd700 1 -- 172.21.15.101:0/999087577 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a0009d670 con 0x5585e8810ad0 2023-07-18T03:24:07.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.957+0000 7f4a21992b80 10 client.28799 fetch_fsmap learned FSMap version 41 2023-07-18T03:24:07.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.957+0000 7f4a21992b80 1 -- 172.21.15.101:0/999087577 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5585e85842b0 con 0x5585e8810ad0 2023-07-18T03:24:07.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.957+0000 7f49e6ffd700 1 -- 172.21.15.101:0/999087577 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49fc004e50 con 0x5585e8810ad0 2023-07-18T03:24:07.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.957+0000 7f49e6ffd700 1 -- 172.21.15.101:0/999087577 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 41) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f4a00084140 con 0x5585e8810ad0 2023-07-18T03:24:07.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.957+0000 7f4a21992b80 10 client.28799 fetch_fsmap finished waiting for FSMap version 41 2023-07-18T03:24:07.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.957+0000 7f4a21992b80 10 client.28799 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:24:07.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.957+0000 7f4a21992b80 20 client.28799 populate_metadata read hostname 'smithi101' 2023-07-18T03:24:07.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.957+0000 7f4a21992b80 1 --2- 172.21.15.101:0/999087577 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5585e8874a30 0x5585e8876e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.957+0000 7f4a21992b80 4 client.28799 mds_command: new command op to 24562 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:24:07.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.957+0000 7f4a21992b80 1 -- 172.21.15.101:0/999087577 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5585e8713050 con 0x5585e8874a30 2023-07-18T03:24:07.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.957+0000 7f4a077fe700 1 --2- 172.21.15.101:0/999087577 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5585e8874a30 0x5585e8876e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.958+0000 7f4a077fe700 1 --2- 172.21.15.101:0/999087577 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5585e8874a30 0x5585e8876e40 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4728e0947cfb820 server_cookie=1bdd1b7232f8756 in_seq=0 out_seq=0 2023-07-18T03:24:07.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.958+0000 7f49e6ffd700 10 client.28799 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:24:07.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.961+0000 7f49e6ffd700 1 -- 172.21.15.101:0/999087577 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5585e8713050 con 0x5585e8874a30 2023-07-18T03:24:07.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.961+0000 7f49e6ffd700 10 client.28799 handle_command_reply: tid=0 2023-07-18T03:24:07.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.961+0000 7f4a21992b80 1 client.28799 shutdown 2023-07-18T03:24:07.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.961+0000 7f4a21992b80 1 -- 172.21.15.101:0/999087577 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585e8810ad0 msgr2=0x5585e8815c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:07.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.961+0000 7f4a21992b80 1 --2- 172.21.15.101:0/999087577 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585e8810ad0 0x5585e8815c40 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4a00004430 tx=0x7f4a00099490 comp rx=0 tx=0).stop 2023-07-18T03:24:07.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.961+0000 7f4a21992b80 1 -- 172.21.15.101:0/999087577 shutdown_connections 2023-07-18T03:24:07.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.961+0000 7f4a21992b80 1 --2- 172.21.15.101:0/999087577 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5585e8874a30 0x5585e8876e40 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.961+0000 7f4a21992b80 1 --2- 172.21.15.101:0/999087577 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585e8810ad0 0x5585e8815c40 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.961+0000 7f4a21992b80 1 --2- 172.21.15.101:0/999087577 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5585e88052d0 0x5585e880b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.961+0000 7f49e6ffd700 0 client.28799 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:24:07.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.961+0000 7f4a21992b80 1 --2- 172.21.15.101:0/999087577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5585e8805be0 0x5585e8815500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.961+0000 7f4a21992b80 1 -- 172.21.15.101:0/999087577 >> 172.21.15.101:0/999087577 conn(0x5585e87fbda0 msgr2=0x5585e88223f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:07.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.961+0000 7f4a21992b80 1 -- 172.21.15.101:0/999087577 shutdown_connections 2023-07-18T03:24:07.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.962+0000 7f4a21992b80 1 -- 172.21.15.101:0/999087577 wait complete. 2023-07-18T03:24:07.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.962+0000 7f4a21992b80 20 client.28799 trim_cache size 0 max 16384 2023-07-18T03:24:07.966 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:24:07.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.962+0000 7f4a0dfff700 1 -- 172.21.15.101:0/4198060999 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f49f00a12c0 msgr2=0x7f49f00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:07.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.962+0000 7f4a0dfff700 1 --2- 172.21.15.101:0/4198060999 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f49f00a12c0 0x7f49f00a3770 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f49f800b580 tx=0x7f49f8009530 comp rx=0 tx=0).stop 2023-07-18T03:24:07.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.962+0000 7f4a0dfff700 1 -- 172.21.15.101:0/4198060999 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4a08106fb0 msgr2=0x7f4a0810b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:07.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.962+0000 7f4a0dfff700 1 --2- 172.21.15.101:0/4198060999 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4a08106fb0 0x7f4a0810b630 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f4a00007ee0 tx=0x7f4a0000d3b0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.962+0000 7f4a0dfff700 1 -- 172.21.15.101:0/4198060999 shutdown_connections 2023-07-18T03:24:07.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.962+0000 7f4a0dfff700 1 --2- 172.21.15.101:0/4198060999 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f49f00a12c0 0x7f49f00a3770 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.962+0000 7f4a0dfff700 1 --2- 172.21.15.101:0/4198060999 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a08101f70 0x7f4a0810e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.962+0000 7f4a0dfff700 1 --2- 172.21.15.101:0/4198060999 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4a08106fb0 0x7f4a0810b630 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.963+0000 7f4a0dfff700 1 --2- 172.21.15.101:0/4198060999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a08102910 0x7f4a0810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.963+0000 7f4a0dfff700 1 -- 172.21.15.101:0/4198060999 >> 172.21.15.101:0/4198060999 conn(0x7f4a080fd7e0 msgr2=0x7f4a080fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:07.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.963+0000 7f4a0dfff700 1 -- 172.21.15.101:0/4198060999 shutdown_connections 2023-07-18T03:24:07.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:07.963+0000 7f4a0dfff700 1 -- 172.21.15.101:0/4198060999 wait complete. 2023-07-18T03:24:07.996 INFO:tasks.workunit.client.0.smithi101.stdout: 10 119111 29.35 MB/sec execute 92 sec latency 1678.212 ms 2023-07-18T03:24:08.240 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:08 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:08.241 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:08 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:24:08.241 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:08 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:24:08.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:08 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:08.550 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:08 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:24:08.550 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:08 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:24:08.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:08 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:08.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:08 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:24:08.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:08 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:24:08.981 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-18T03:24:08.982 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-18T03:24:08.996 INFO:tasks.workunit.client.0.smithi101.stdout: 10 119781 29.44 MB/sec execute 93 sec latency 2678.252 ms 2023-07-18T03:24:09.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.245+0000 7fee62bff700 1 -- 172.21.15.101:0/4002381790 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee5c100480 msgr2=0x7fee5c10c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:09.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.245+0000 7fee62bff700 1 --2- 172.21.15.101:0/4002381790 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee5c100480 0x7fee5c10c910 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fee5400b0a0 tx=0x7fee5400b3b0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.245+0000 7fee62bff700 1 -- 172.21.15.101:0/4002381790 shutdown_connections 2023-07-18T03:24:09.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.245+0000 7fee62bff700 1 --2- 172.21.15.101:0/4002381790 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee5c100480 0x7fee5c10c910 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.245+0000 7fee62bff700 1 --2- 172.21.15.101:0/4002381790 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fee5c105ae0 0x7fee5c105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.245+0000 7fee62bff700 1 --2- 172.21.15.101:0/4002381790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee5c0ffad0 0x7fee5c0fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.245+0000 7fee62bff700 1 -- 172.21.15.101:0/4002381790 >> 172.21.15.101:0/4002381790 conn(0x7fee5c0fb5f0 msgr2=0x7fee5c0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:09.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.246+0000 7fee62bff700 1 -- 172.21.15.101:0/4002381790 shutdown_connections 2023-07-18T03:24:09.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.246+0000 7fee62bff700 1 -- 172.21.15.101:0/4002381790 wait complete. 2023-07-18T03:24:09.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.247+0000 7fee62bff700 1 Processor -- start 2023-07-18T03:24:09.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.247+0000 7fee62bff700 1 -- start start 2023-07-18T03:24:09.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.248+0000 7fee62bff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee5c0ffad0 0x7fee5c197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:09.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.248+0000 7fee62bff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee5c100480 0x7fee5c1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:09.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.248+0000 7fee62bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fee5c105ae0 0x7fee5c19c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:09.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.248+0000 7fee62bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c19cf50 con 0x7fee5c105ae0 2023-07-18T03:24:09.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.248+0000 7fee62bff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c19d0c0 con 0x7fee5c100480 2023-07-18T03:24:09.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.248+0000 7fee62bff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c19d280 con 0x7fee5c0ffad0 2023-07-18T03:24:09.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.249+0000 7fee5bfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee5c100480 0x7fee5c1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:09.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.249+0000 7fee6119c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fee5c105ae0 0x7fee5c19c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:09.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.249+0000 7fee6099b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee5c0ffad0 0x7fee5c197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:09.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.249+0000 7fee5bfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee5c100480 0x7fee5c1984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:41190/0 (socket says 172.21.15.101:41190) 2023-07-18T03:24:09.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.249+0000 7fee5bfff700 1 -- 172.21.15.101:0/4060459221 learned_addr learned my addr 172.21.15.101:0/4060459221 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:09.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.249+0000 7fee6119c700 1 -- 172.21.15.101:0/4060459221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee5c0ffad0 msgr2=0x7fee5c197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:09.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.249+0000 7fee6119c700 1 --2- 172.21.15.101:0/4060459221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee5c0ffad0 0x7fee5c197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.249+0000 7fee6119c700 1 -- 172.21.15.101:0/4060459221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee5c100480 msgr2=0x7fee5c1984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:09.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.249+0000 7fee6119c700 1 --2- 172.21.15.101:0/4060459221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee5c100480 0x7fee5c1984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.249+0000 7fee6119c700 1 -- 172.21.15.101:0/4060459221 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee54009d00 con 0x7fee5c105ae0 2023-07-18T03:24:09.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.249+0000 7fee6099b700 1 --2- 172.21.15.101:0/4060459221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee5c0ffad0 0x7fee5c197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:24:09.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.250+0000 7fee6119c700 1 --2- 172.21.15.101:0/4060459221 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fee5c105ae0 0x7fee5c19c910 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fee54009fd0 tx=0x7fee5400bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:09.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.250+0000 7fee59ffb700 1 -- 172.21.15.101:0/4060459221 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5400e040 con 0x7fee5c105ae0 2023-07-18T03:24:09.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.250+0000 7fee59ffb700 1 -- 172.21.15.101:0/4060459221 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee540094c0 con 0x7fee5c105ae0 2023-07-18T03:24:09.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.250+0000 7fee59ffb700 1 -- 172.21.15.101:0/4060459221 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee54005040 con 0x7fee5c105ae0 2023-07-18T03:24:09.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.250+0000 7fee62bff700 1 -- 172.21.15.101:0/4060459221 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee5c19d500 con 0x7fee5c105ae0 2023-07-18T03:24:09.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.250+0000 7fee62bff700 1 -- 172.21.15.101:0/4060459221 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee5c106aa0 con 0x7fee5c105ae0 2023-07-18T03:24:09.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.255+0000 7fee59ffb700 1 -- 172.21.15.101:0/4060459221 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fee5401d470 con 0x7fee5c105ae0 2023-07-18T03:24:09.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.256+0000 7fee7646cb80 1 Processor -- start 2023-07-18T03:24:09.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.256+0000 7fee7646cb80 1 -- start start 2023-07-18T03:24:09.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.256+0000 7fee7646cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b025aed350 0x55b025aed720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:09.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.256+0000 7fee7646cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b025aedc60 0x55b025af8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:09.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.256+0000 7fee7646cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b025af8b50 0x55b025afaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:09.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.256+0000 7fee59ffb700 1 --2- 172.21.15.101:0/4060459221 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fee4c0a13c0 0x7fee4c0a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:09.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.256+0000 7fee7646cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0259f3d30 con 0x55b025aed350 2023-07-18T03:24:09.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.256+0000 7fee7646cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b025a46ac0 con 0x55b025af8b50 2023-07-18T03:24:09.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.256+0000 7fee7646cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b025a3ff50 con 0x55b025aedc60 2023-07-18T03:24:09.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.257+0000 7fee59ffb700 1 -- 172.21.15.101:0/4060459221 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9549+0+0 (secure 0 0 0) 0x7fee540ce5c0 con 0x7fee5c105ae0 2023-07-18T03:24:09.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.257+0000 7fee6099b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b025aedc60 0x55b025af8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:09.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.257+0000 7fee5bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b025aed350 0x55b025aed720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:09.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.257+0000 7fee6099b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b025aedc60 0x55b025af8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:37150/0 (socket says 172.21.15.101:37150) 2023-07-18T03:24:09.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.257+0000 7fee6099b700 1 -- 172.21.15.101:0/3495523274 learned_addr learned my addr 172.21.15.101:0/3495523274 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:09.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.257+0000 7fee6119c700 1 --2- 172.21.15.101:0/4060459221 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fee4c0a13c0 0x7fee4c0a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:09.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.257+0000 7fee5bfff700 1 --2- 172.21.15.101:0/3495523274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b025af8b50 0x55b025afaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:09.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.257+0000 7fee6099b700 1 -- 172.21.15.101:0/3495523274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b025af8b50 msgr2=0x55b025afaf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:09.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.257+0000 7fee6099b700 1 --2- 172.21.15.101:0/3495523274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b025af8b50 0x55b025afaf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.257+0000 7fee6099b700 1 -- 172.21.15.101:0/3495523274 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b025aed350 msgr2=0x55b025aed720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:09.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.257+0000 7fee6099b700 1 --2- 172.21.15.101:0/3495523274 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b025aed350 0x55b025aed720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.257+0000 7fee6099b700 1 -- 172.21.15.101:0/3495523274 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b02586de60 con 0x55b025aedc60 2023-07-18T03:24:09.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.258+0000 7fee6119c700 1 --2- 172.21.15.101:0/4060459221 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fee4c0a13c0 0x7fee4c0a3870 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fee5409b250 tx=0x7fee5400b040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:09.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.258+0000 7fee6099b700 1 --2- 172.21.15.101:0/3495523274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b025aedc60 0x55b025af8610 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fee500115f0 tx=0x7fee50011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37874f774ac7db4d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:09.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.258+0000 7fee46ffd700 1 -- 172.21.15.101:0/3495523274 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee50019630 con 0x55b025aedc60 2023-07-18T03:24:09.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.258+0000 7fee46ffd700 1 -- 172.21.15.101:0/3495523274 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee50019c50 con 0x55b025aedc60 2023-07-18T03:24:09.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.258+0000 7fee46ffd700 1 -- 172.21.15.101:0/3495523274 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5000ebd0 con 0x55b025aedc60 2023-07-18T03:24:09.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.259+0000 7fee7646cb80 1 -- 172.21.15.101:0/3495523274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b025aedc60 msgr2=0x55b025af8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:09.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.259+0000 7fee7646cb80 1 --2- 172.21.15.101:0/3495523274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b025aedc60 0x55b025af8610 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fee500115f0 tx=0x7fee50011900 comp rx=0 tx=0).stop 2023-07-18T03:24:09.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.259+0000 7fee7646cb80 1 -- 172.21.15.101:0/3495523274 shutdown_connections 2023-07-18T03:24:09.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.259+0000 7fee7646cb80 1 --2- 172.21.15.101:0/3495523274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b025af8b50 0x55b025afaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.259+0000 7fee7646cb80 1 --2- 172.21.15.101:0/3495523274 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b025aed350 0x55b025aed720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.259+0000 7fee7646cb80 1 --2- 172.21.15.101:0/3495523274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b025aedc60 0x55b025af8610 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.259+0000 7fee7646cb80 1 -- 172.21.15.101:0/3495523274 >> 172.21.15.101:0/3495523274 conn(0x55b025ae3e20 msgr2=0x55b025ae9720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:09.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.259+0000 7fee7646cb80 1 -- 172.21.15.101:0/3495523274 shutdown_connections 2023-07-18T03:24:09.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.259+0000 7fee7646cb80 1 -- 172.21.15.101:0/3495523274 wait complete. 2023-07-18T03:24:09.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.260+0000 7fee7646cb80 1 -- start start 2023-07-18T03:24:09.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.260+0000 7fee7646cb80 1 Processor -- start 2023-07-18T03:24:09.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.261+0000 7fee7646cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b025aed350 0x55b025af8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:09.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.261+0000 7fee7646cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b025aedc60 0x55b025af57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:09.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.261+0000 7fee7646cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b025af8b50 0x55b025af5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:09.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.261+0000 7fee7646cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0259dfb40 con 0x55b025aedc60 2023-07-18T03:24:09.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.261+0000 7fee7646cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0259df840 con 0x55b025aed350 2023-07-18T03:24:09.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.261+0000 7fee7646cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0258c39a0 con 0x55b025af8b50 2023-07-18T03:24:09.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.261+0000 7fee7646cb80 10 client.0 _ll_register_callbacks cb 0x55b0259dd7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:24:09.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.262+0000 7fee5bfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b025aed350 0x55b025af8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:09.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.262+0000 7fee5bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b025af8b50 0x55b025af5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:09.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.262+0000 7fee5bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b025af8b50 0x55b025af5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:37156/0 (socket says 172.21.15.101:37156) 2023-07-18T03:24:09.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.262+0000 7fee5bfff700 1 -- 172.21.15.101:0/766496155 learned_addr learned my addr 172.21.15.101:0/766496155 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:09.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.262+0000 7fee6099b700 1 --2- 172.21.15.101:0/766496155 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b025aedc60 0x55b025af57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:09.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.262+0000 7fee5bfff700 1 -- 172.21.15.101:0/766496155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b025aed350 msgr2=0x55b025af8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:09.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.262+0000 7fee5bfff700 1 --2- 172.21.15.101:0/766496155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b025aed350 0x55b025af8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.262+0000 7fee5bfff700 1 -- 172.21.15.101:0/766496155 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b025aedc60 msgr2=0x55b025af57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:09.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.262+0000 7fee5bfff700 1 --2- 172.21.15.101:0/766496155 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b025aedc60 0x55b025af57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.262+0000 7fee5bfff700 1 -- 172.21.15.101:0/766496155 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee50009dd0 con 0x55b025af8b50 2023-07-18T03:24:09.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.263+0000 7fee5bfff700 1 --2- 172.21.15.101:0/766496155 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b025af8b50 0x55b025af5ee0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fee48006b30 tx=0x7fee480095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=177279795b40a353 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:09.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.263+0000 7fee3bfff700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:24:09.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.263+0000 7fee3bfff700 1 -- 172.21.15.101:0/766496155 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee48015c80 con 0x55b025af8b50 2023-07-18T03:24:09.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.263+0000 7fee7646cb80 1 -- 172.21.15.101:0/766496155 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b025999570 con 0x55b025af8b50 2023-07-18T03:24:09.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.263+0000 7fee3bfff700 1 -- 172.21.15.101:0/766496155 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9549+0+0 (secure 0 0 0) 0x7fee48015de0 con 0x55b025af8b50 2023-07-18T03:24:09.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.263+0000 7fee3bfff700 1 -- 172.21.15.101:0/766496155 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee480279d0 con 0x55b025af8b50 2023-07-18T03:24:09.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.263+0000 7fee7646cb80 10 client.28810 fetch_fsmap learned FSMap version 41 2023-07-18T03:24:09.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.263+0000 7fee7646cb80 1 -- 172.21.15.101:0/766496155 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55b025856ce0 con 0x55b025af8b50 2023-07-18T03:24:09.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.264+0000 7fee3bfff700 1 -- 172.21.15.101:0/766496155 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b025999570 con 0x55b025af8b50 2023-07-18T03:24:09.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.264+0000 7fee3bfff700 1 -- 172.21.15.101:0/766496155 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 41) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fee48018450 con 0x55b025af8b50 2023-07-18T03:24:09.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.264+0000 7fee7646cb80 10 client.28810 fetch_fsmap finished waiting for FSMap version 41 2023-07-18T03:24:09.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.264+0000 7fee7646cb80 10 client.28810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:24:09.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.264+0000 7fee7646cb80 20 client.28810 populate_metadata read hostname 'smithi101' 2023-07-18T03:24:09.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.264+0000 7fee7646cb80 1 --2- 172.21.15.101:0/766496155 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b025b08400 0x55b025b0a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:09.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.264+0000 7fee7646cb80 4 client.28810 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:24:09.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.264+0000 7fee7646cb80 1 -- 172.21.15.101:0/766496155 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b025a38f30 con 0x55b025b08400 2023-07-18T03:24:09.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.264+0000 7fee6099b700 1 --2- 172.21.15.101:0/766496155 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b025b08400 0x55b025b0a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:09.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.265+0000 7fee6099b700 1 --2- 172.21.15.101:0/766496155 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b025b08400 0x55b025b0a810 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=325e45c5ccf53433 server_cookie=fe4b6b8919d8493a in_seq=0 out_seq=0 2023-07-18T03:24:09.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.265+0000 7fee3bfff700 10 client.28810 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:24:09.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.269+0000 7fee3bfff700 1 -- 172.21.15.101:0/766496155 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b025a38f30 con 0x55b025b08400 2023-07-18T03:24:09.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.269+0000 7fee3bfff700 10 client.28810 handle_command_reply: tid=0 2023-07-18T03:24:09.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.269+0000 7fee7646cb80 1 client.28810 shutdown 2023-07-18T03:24:09.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.269+0000 7fee7646cb80 1 -- 172.21.15.101:0/766496155 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b025af8b50 msgr2=0x55b025af5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:09.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.269+0000 7fee7646cb80 1 --2- 172.21.15.101:0/766496155 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b025af8b50 0x55b025af5ee0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fee48006b30 tx=0x7fee480095c0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.270+0000 7fee7646cb80 1 -- 172.21.15.101:0/766496155 shutdown_connections 2023-07-18T03:24:09.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.270+0000 7fee7646cb80 1 --2- 172.21.15.101:0/766496155 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b025b08400 0x55b025b0a810 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.270+0000 7fee7646cb80 1 --2- 172.21.15.101:0/766496155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b025aed350 0x55b025af8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.270+0000 7fee7646cb80 1 --2- 172.21.15.101:0/766496155 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b025af8b50 0x55b025af5ee0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.270+0000 7fee7646cb80 1 --2- 172.21.15.101:0/766496155 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b025aedc60 0x55b025af57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.270+0000 7fee7646cb80 1 -- 172.21.15.101:0/766496155 >> 172.21.15.101:0/766496155 conn(0x55b025ae3e20 msgr2=0x55b025ae9310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:09.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.270+0000 7fee3bfff700 0 client.28810 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:24:09.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.270+0000 7fee7646cb80 1 -- 172.21.15.101:0/766496155 shutdown_connections 2023-07-18T03:24:09.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.270+0000 7fee7646cb80 1 -- 172.21.15.101:0/766496155 wait complete. 2023-07-18T03:24:09.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.270+0000 7fee7646cb80 20 client.28810 trim_cache size 0 max 16384 2023-07-18T03:24:09.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.293+0000 7fee7646cb80 1 Processor -- start 2023-07-18T03:24:09.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.293+0000 7fee7646cb80 1 -- start start 2023-07-18T03:24:09.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.294+0000 7fee7646cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b025aed350 0x55b025af5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:09.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.294+0000 7fee7646cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b025aedc60 0x55b025af80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:09.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.294+0000 7fee7646cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b025af8b50 0x55b025af2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:09.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.294+0000 7fee7646cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b025b05b60 con 0x55b025aed350 2023-07-18T03:24:09.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.294+0000 7fee7646cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b025aee5c0 con 0x55b025af8b50 2023-07-18T03:24:09.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.294+0000 7fee7646cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b025aef220 con 0x55b025aedc60 2023-07-18T03:24:09.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.294+0000 7fee6099b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b025aedc60 0x55b025af80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:09.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.294+0000 7fee5bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b025aed350 0x55b025af5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:09.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.294+0000 7fee6099b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b025aedc60 0x55b025af80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:37158/0 (socket says 172.21.15.101:37158) 2023-07-18T03:24:09.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.294+0000 7fee6099b700 1 -- 172.21.15.101:0/1133138079 learned_addr learned my addr 172.21.15.101:0/1133138079 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:09.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.294+0000 7fee5bfff700 1 --2- 172.21.15.101:0/1133138079 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b025af8b50 0x55b025af2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:09.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.295+0000 7fee6099b700 1 -- 172.21.15.101:0/1133138079 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b025af8b50 msgr2=0x55b025af2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:09.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.295+0000 7fee6099b700 1 --2- 172.21.15.101:0/1133138079 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b025af8b50 0x55b025af2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.295+0000 7fee6099b700 1 -- 172.21.15.101:0/1133138079 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b025aed350 msgr2=0x55b025af5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:09.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.295+0000 7fee6099b700 1 --2- 172.21.15.101:0/1133138079 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b025aed350 0x55b025af5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.295+0000 7fee6099b700 1 -- 172.21.15.101:0/1133138079 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b025ae5660 con 0x55b025aedc60 2023-07-18T03:24:09.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.295+0000 7fee6099b700 1 --2- 172.21.15.101:0/1133138079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b025aedc60 0x55b025af80c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fee50011510 tx=0x7fee5000e6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12c34b3d5bf48d2d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:09.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.295+0000 7fee477fe700 1 -- 172.21.15.101:0/1133138079 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5002b340 con 0x55b025aedc60 2023-07-18T03:24:09.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.295+0000 7fee477fe700 1 -- 172.21.15.101:0/1133138079 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee5002b4a0 con 0x55b025aedc60 2023-07-18T03:24:09.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.295+0000 7fee477fe700 1 -- 172.21.15.101:0/1133138079 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee50005560 con 0x55b025aedc60 2023-07-18T03:24:09.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.296+0000 7fee7646cb80 1 -- 172.21.15.101:0/1133138079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b025aedc60 msgr2=0x55b025af80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:09.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.296+0000 7fee7646cb80 1 --2- 172.21.15.101:0/1133138079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b025aedc60 0x55b025af80c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fee50011510 tx=0x7fee5000e6b0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.296+0000 7fee7646cb80 1 -- 172.21.15.101:0/1133138079 shutdown_connections 2023-07-18T03:24:09.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.296+0000 7fee7646cb80 1 --2- 172.21.15.101:0/1133138079 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b025af8b50 0x55b025af2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.296+0000 7fee7646cb80 1 --2- 172.21.15.101:0/1133138079 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b025aed350 0x55b025af5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.296+0000 7fee7646cb80 1 --2- 172.21.15.101:0/1133138079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b025aedc60 0x55b025af80c0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.296+0000 7fee7646cb80 1 -- 172.21.15.101:0/1133138079 >> 172.21.15.101:0/1133138079 conn(0x55b025ae3e20 msgr2=0x55b025af5040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:09.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.296+0000 7fee7646cb80 1 -- 172.21.15.101:0/1133138079 shutdown_connections 2023-07-18T03:24:09.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.296+0000 7fee7646cb80 1 -- 172.21.15.101:0/1133138079 wait complete. 2023-07-18T03:24:09.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.297+0000 7fee7646cb80 1 -- start start 2023-07-18T03:24:09.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.297+0000 7fee7646cb80 1 Processor -- start 2023-07-18T03:24:09.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.297+0000 7fee7646cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b025aed350 0x55b025afe260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:09.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.297+0000 7fee7646cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b025aedc60 0x55b025afe7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:09.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.297+0000 7fee7646cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b025af8b50 0x55b025b5c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:09.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.297+0000 7fee7646cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b025afee90 con 0x55b025aedc60 2023-07-18T03:24:09.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.297+0000 7fee7646cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b025b5c7d0 con 0x55b025aed350 2023-07-18T03:24:09.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.297+0000 7fee7646cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b025b5c910 con 0x55b025af8b50 2023-07-18T03:24:09.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.297+0000 7fee7646cb80 10 client.0 _ll_register_callbacks cb 0x7fee48017850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:24:09.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.298+0000 7fee6099b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b025aedc60 0x55b025afe7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:09.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.298+0000 7fee5bfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b025aed350 0x55b025afe260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:09.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.298+0000 7fee6099b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b025aedc60 0x55b025afe7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:54806/0 (socket says 172.21.15.101:54806) 2023-07-18T03:24:09.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.298+0000 7fee6099b700 1 -- 172.21.15.101:0/3776906061 learned_addr learned my addr 172.21.15.101:0/3776906061 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:09.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.298+0000 7fee5bfff700 1 --2- 172.21.15.101:0/3776906061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b025af8b50 0x55b025b5c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:09.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.298+0000 7fee6099b700 1 -- 172.21.15.101:0/3776906061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b025af8b50 msgr2=0x55b025b5c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:09.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.298+0000 7fee6099b700 1 --2- 172.21.15.101:0/3776906061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b025af8b50 0x55b025b5c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.298+0000 7fee6099b700 1 -- 172.21.15.101:0/3776906061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b025aed350 msgr2=0x55b025afe260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:09.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.298+0000 7fee6099b700 1 --2- 172.21.15.101:0/3776906061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b025aed350 0x55b025afe260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.298+0000 7fee6099b700 1 -- 172.21.15.101:0/3776906061 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b02586de60 con 0x55b025aedc60 2023-07-18T03:24:09.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.298+0000 7fee5bfff700 1 --2- 172.21.15.101:0/3776906061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b025af8b50 0x55b025b5c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:24:09.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.298+0000 7fee6099b700 1 --2- 172.21.15.101:0/3776906061 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b025aedc60 0x55b025afe7a0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fee5001c040 tx=0x7fee50005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6354e1e92627856 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:09.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.298+0000 7fee3bfff700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:24:09.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.299+0000 7fee3bfff700 1 -- 172.21.15.101:0/3776906061 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee50005570 con 0x55b025aedc60 2023-07-18T03:24:09.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.299+0000 7fee7646cb80 1 -- 172.21.15.101:0/3776906061 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fee50019c50 con 0x55b025aedc60 2023-07-18T03:24:09.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.299+0000 7fee3bfff700 1 -- 172.21.15.101:0/3776906061 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9549+0+0 (secure 0 0 0) 0x7fee50019430 con 0x55b025aedc60 2023-07-18T03:24:09.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.299+0000 7fee3bfff700 1 -- 172.21.15.101:0/3776906061 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee500056d0 con 0x55b025aedc60 2023-07-18T03:24:09.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.299+0000 7fee7646cb80 10 client.18723 fetch_fsmap learned FSMap version 41 2023-07-18T03:24:09.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.299+0000 7fee7646cb80 1 -- 172.21.15.101:0/3776906061 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55b0258942d0 con 0x55b025aedc60 2023-07-18T03:24:09.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.299+0000 7fee3bfff700 1 -- 172.21.15.101:0/3776906061 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee50019c50 con 0x55b025aedc60 2023-07-18T03:24:09.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.300+0000 7fee3bfff700 1 -- 172.21.15.101:0/3776906061 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 41) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fee5000ec40 con 0x55b025aedc60 2023-07-18T03:24:09.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.300+0000 7fee7646cb80 10 client.18723 fetch_fsmap finished waiting for FSMap version 41 2023-07-18T03:24:09.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.300+0000 7fee7646cb80 10 client.18723 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:24:09.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.300+0000 7fee7646cb80 20 client.18723 populate_metadata read hostname 'smithi101' 2023-07-18T03:24:09.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.300+0000 7fee7646cb80 1 --2- 172.21.15.101:0/3776906061 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b025b5de80 0x55b025b60290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:09.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.300+0000 7fee7646cb80 4 client.18723 mds_command: new command op to 24562 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-18T03:24:09.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.300+0000 7fee7646cb80 1 -- 172.21.15.101:0/3776906061 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55b0259fb0d0 con 0x55b025b5de80 2023-07-18T03:24:09.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.300+0000 7fee5bfff700 1 --2- 172.21.15.101:0/3776906061 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b025b5de80 0x55b025b60290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:09.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.300+0000 7fee3bfff700 10 client.18723 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:24:09.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.300+0000 7fee5bfff700 1 --2- 172.21.15.101:0/3776906061 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b025b5de80 0x55b025b60290 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=145adf3d628156cf server_cookie=e609ae2cf2f11e1b in_seq=0 out_seq=0 2023-07-18T03:24:09.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.304+0000 7fee3bfff700 1 -- 172.21.15.101:0/3776906061 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b0259fb0d0 con 0x55b025b5de80 2023-07-18T03:24:09.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.304+0000 7fee3bfff700 10 client.18723 handle_command_reply: tid=0 2023-07-18T03:24:09.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.304+0000 7fee7646cb80 1 client.18723 shutdown 2023-07-18T03:24:09.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.304+0000 7fee7646cb80 1 -- 172.21.15.101:0/3776906061 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b025aedc60 msgr2=0x55b025afe7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:09.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.304+0000 7fee7646cb80 1 --2- 172.21.15.101:0/3776906061 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b025aedc60 0x55b025afe7a0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fee5001c040 tx=0x7fee50005050 comp rx=0 tx=0).stop 2023-07-18T03:24:09.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.305+0000 7fee7646cb80 1 -- 172.21.15.101:0/3776906061 shutdown_connections 2023-07-18T03:24:09.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.305+0000 7fee7646cb80 1 --2- 172.21.15.101:0/3776906061 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b025b5de80 0x55b025b60290 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.305+0000 7fee7646cb80 1 --2- 172.21.15.101:0/3776906061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b025aed350 0x55b025afe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.305+0000 7fee7646cb80 1 --2- 172.21.15.101:0/3776906061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b025af8b50 0x55b025b5c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.305+0000 7fee7646cb80 1 --2- 172.21.15.101:0/3776906061 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b025aedc60 0x55b025afe7a0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.305+0000 7fee7646cb80 1 -- 172.21.15.101:0/3776906061 >> 172.21.15.101:0/3776906061 conn(0x55b025ae3e20 msgr2=0x55b025aefcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:09.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.305+0000 7fee3bfff700 0 client.18723 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:24:09.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.305+0000 7fee7646cb80 1 -- 172.21.15.101:0/3776906061 shutdown_connections 2023-07-18T03:24:09.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.305+0000 7fee7646cb80 1 -- 172.21.15.101:0/3776906061 wait complete. 2023-07-18T03:24:09.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.305+0000 7fee7646cb80 20 client.18723 trim_cache size 0 max 16384 2023-07-18T03:24:09.307 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:24:09.308 INFO:teuthology.orchestra.run.smithi101.stdout: "return_code": 0, 2023-07-18T03:24:09.308 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_tag": "666b0d9a-2af3-478a-bad3-ce9f13c9ba69", 2023-07-18T03:24:09.308 INFO:teuthology.orchestra.run.smithi101.stdout: "mode": "asynchronous" 2023-07-18T03:24:09.308 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:24:09.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.306+0000 7fee62bff700 1 -- 172.21.15.101:0/4060459221 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fee4c0a13c0 msgr2=0x7fee4c0a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:09.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.306+0000 7fee62bff700 1 --2- 172.21.15.101:0/4060459221 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fee4c0a13c0 0x7fee4c0a3870 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fee5409b250 tx=0x7fee5400b040 comp rx=0 tx=0).stop 2023-07-18T03:24:09.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.306+0000 7fee62bff700 1 -- 172.21.15.101:0/4060459221 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fee5c105ae0 msgr2=0x7fee5c19c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:09.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.306+0000 7fee62bff700 1 --2- 172.21.15.101:0/4060459221 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fee5c105ae0 0x7fee5c19c910 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fee54009fd0 tx=0x7fee5400bce0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.306+0000 7fee62bff700 1 -- 172.21.15.101:0/4060459221 shutdown_connections 2023-07-18T03:24:09.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.306+0000 7fee62bff700 1 --2- 172.21.15.101:0/4060459221 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fee5c105ae0 0x7fee5c19c910 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.306+0000 7fee62bff700 1 --2- 172.21.15.101:0/4060459221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee5c0ffad0 0x7fee5c197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.306+0000 7fee62bff700 1 --2- 172.21.15.101:0/4060459221 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fee4c0a13c0 0x7fee4c0a3870 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.306+0000 7fee62bff700 1 --2- 172.21.15.101:0/4060459221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee5c100480 0x7fee5c1984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.306+0000 7fee62bff700 1 -- 172.21.15.101:0/4060459221 >> 172.21.15.101:0/4060459221 conn(0x7fee5c0fb5f0 msgr2=0x7fee5c10ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:09.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.306+0000 7fee62bff700 1 -- 172.21.15.101:0/4060459221 shutdown_connections 2023-07-18T03:24:09.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.306+0000 7fee62bff700 1 -- 172.21.15.101:0/4060459221 wait complete. 2023-07-18T03:24:09.324 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:09 smithi101 ceph-mon[126379]: pgmap v521: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 12 MiB/s wr, 702 op/s 2023-07-18T03:24:09.326 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:24:09.550 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:09 smithi155 ceph-mon[128885]: pgmap v521: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 12 MiB/s wr, 702 op/s 2023-07-18T03:24:09.639 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.642+0000 7ff6495ff700 1 -- 172.21.15.101:0/3667170288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff644074a40 msgr2=0x7ff644074e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:09.640 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.642+0000 7ff6495ff700 1 --2- 172.21.15.101:0/3667170288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff644074a40 0x7ff644074e10 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ff634009990 tx=0x7ff634009ca0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.640 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.642+0000 7ff6495ff700 1 -- 172.21.15.101:0/3667170288 shutdown_connections 2023-07-18T03:24:09.640 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.642+0000 7ff6495ff700 1 --2- 172.21.15.101:0/3667170288 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6441109a0 0x7ff644112d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.641 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.642+0000 7ff6495ff700 1 --2- 172.21.15.101:0/3667170288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff644074a40 0x7ff644074e10 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.641 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.642+0000 7ff6495ff700 1 --2- 172.21.15.101:0/3667170288 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff644075350 0x7ff644110460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.641 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.642+0000 7ff6495ff700 1 -- 172.21.15.101:0/3667170288 >> 172.21.15.101:0/3667170288 conn(0x7ff6440fd7e0 msgr2=0x7ff6440ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:09.641 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.642+0000 7ff6495ff700 1 -- 172.21.15.101:0/3667170288 shutdown_connections 2023-07-18T03:24:09.642 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.643+0000 7ff6495ff700 1 -- 172.21.15.101:0/3667170288 wait complete. 2023-07-18T03:24:09.644 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.644+0000 7ff6495ff700 1 Processor -- start 2023-07-18T03:24:09.644 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.644+0000 7ff6495ff700 1 -- start start 2023-07-18T03:24:09.644 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.644+0000 7ff6495ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff644074a40 0x7ff64406a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:09.644 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.645+0000 7ff6495ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff644075350 0x7ff64406af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:09.645 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.645+0000 7ff6495ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6441109a0 0x7ff6441b07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:09.645 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.645+0000 7ff6495ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6441b0d40 con 0x7ff644074a40 2023-07-18T03:24:09.645 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.645+0000 7ff6495ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff644101c40 con 0x7ff6441109a0 2023-07-18T03:24:09.645 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.645+0000 7ff6495ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff644101e00 con 0x7ff644075350 2023-07-18T03:24:09.645 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.645+0000 7ff6427fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff644075350 0x7ff64406af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:09.645 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.645+0000 7ff6427fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff644075350 0x7ff64406af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:37182/0 (socket says 172.21.15.101:37182) 2023-07-18T03:24:09.645 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.645+0000 7ff6427fc700 1 -- 172.21.15.101:0/1592309371 learned_addr learned my addr 172.21.15.101:0/1592309371 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:09.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.646+0000 7ff642ffd700 1 --2- 172.21.15.101:0/1592309371 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff644074a40 0x7ff64406a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:09.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.646+0000 7ff6427fc700 1 -- 172.21.15.101:0/1592309371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6441109a0 msgr2=0x7ff6441b07d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-18T03:24:09.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.646+0000 7ff6427fc700 1 --2- 172.21.15.101:0/1592309371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6441109a0 0x7ff6441b07d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.646+0000 7ff6427fc700 1 -- 172.21.15.101:0/1592309371 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff644074a40 msgr2=0x7ff64406a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:09.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.646+0000 7ff6427fc700 1 --2- 172.21.15.101:0/1592309371 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff644074a40 0x7ff64406a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.646+0000 7ff6427fc700 1 -- 172.21.15.101:0/1592309371 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6340095f0 con 0x7ff644075350 2023-07-18T03:24:09.646 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.646+0000 7ff642ffd700 1 --2- 172.21.15.101:0/1592309371 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff644074a40 0x7ff64406a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:24:09.647 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.646+0000 7ff6427fc700 1 --2- 172.21.15.101:0/1592309371 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff644075350 0x7ff64406af20 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ff63800e9a0 tx=0x7ff63800ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:09.647 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.646+0000 7ff62bfff700 1 -- 172.21.15.101:0/1592309371 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff638009790 con 0x7ff644075350 2023-07-18T03:24:09.647 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.646+0000 7ff62bfff700 1 -- 172.21.15.101:0/1592309371 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff63800cbd0 con 0x7ff644075350 2023-07-18T03:24:09.647 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.646+0000 7ff62bfff700 1 -- 172.21.15.101:0/1592309371 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff638018680 con 0x7ff644075350 2023-07-18T03:24:09.647 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.647+0000 7ff6495ff700 1 -- 172.21.15.101:0/1592309371 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6441020e0 con 0x7ff644075350 2023-07-18T03:24:09.647 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.647+0000 7ff6495ff700 1 -- 172.21.15.101:0/1592309371 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6441025a0 con 0x7ff644075350 2023-07-18T03:24:09.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.653+0000 7ff65cfe4b80 1 Processor -- start 2023-07-18T03:24:09.650 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.654+0000 7ff65cfe4b80 1 -- start start 2023-07-18T03:24:09.651 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.654+0000 7ff65cfe4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557e654f62d0 0x557e654f66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:09.651 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.654+0000 7ff65cfe4b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557e654f6be0 0x557e65501590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:09.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.654+0000 7ff65cfe4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e65501ad0 0x557e65503eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:09.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.654+0000 7ff65cfe4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e653fccb0 con 0x557e654f62d0 2023-07-18T03:24:09.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.654+0000 7ff65cfe4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6544fa40 con 0x557e65501ad0 2023-07-18T03:24:09.652 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.654+0000 7ff65cfe4b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e65448ed0 con 0x557e654f6be0 2023-07-18T03:24:09.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.655+0000 7ff62bfff700 1 -- 172.21.15.101:0/1592309371 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7ff63800cd40 con 0x7ff644075350 2023-07-18T03:24:09.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.656+0000 7ff6427fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e65501ad0 0x557e65503eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:09.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.656+0000 7ff6427fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e65501ad0 0x557e65503eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:41254/0 (socket says 172.21.15.101:41254) 2023-07-18T03:24:09.653 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.656+0000 7ff6427fc700 1 -- 172.21.15.101:0/4008811588 learned_addr learned my addr 172.21.15.101:0/4008811588 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:09.654 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.656+0000 7ff6437fe700 1 --2- 172.21.15.101:0/4008811588 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557e654f62d0 0x557e654f66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:09.654 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.657+0000 7ff6427fc700 1 -- 172.21.15.101:0/4008811588 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557e654f6be0 msgr2=0x557e65501590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:09.654 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.657+0000 7ff6427fc700 1 --2- 172.21.15.101:0/4008811588 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557e654f6be0 0x557e65501590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.657+0000 7ff6427fc700 1 -- 172.21.15.101:0/4008811588 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557e654f62d0 msgr2=0x557e654f66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:09.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.657+0000 7ff6427fc700 1 --2- 172.21.15.101:0/4008811588 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557e654f62d0 0x557e654f66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.657+0000 7ff6427fc700 1 -- 172.21.15.101:0/4008811588 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e65276de0 con 0x557e65501ad0 2023-07-18T03:24:09.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.657+0000 7ff6437fe700 1 --2- 172.21.15.101:0/4008811588 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557e654f62d0 0x557e654f66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:24:09.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.657+0000 7ff62bfff700 1 --2- 172.21.15.101:0/1592309371 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff62c0a1340 0x7ff62c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:09.655 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.658+0000 7ff62bfff700 1 -- 172.21.15.101:0/1592309371 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9549+0+0 (secure 0 0 0) 0x7ff638014070 con 0x7ff644075350 2023-07-18T03:24:09.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.658+0000 7ff6427fc700 1 --2- 172.21.15.101:0/4008811588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e65501ad0 0x557e65503eb0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff6380ca710 tx=0x7ff6380d5610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71f8aa0bb0e6c6f9 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:09.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.658+0000 7ff6437fe700 1 --2- 172.21.15.101:0/1592309371 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff62c0a1340 0x7ff62c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:09.656 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.659+0000 7ff6437fe700 1 --2- 172.21.15.101:0/1592309371 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff62c0a1340 0x7ff62c0a37f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff6440697f0 tx=0x7ff63c00d040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:09.658 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.661+0000 7ff6297fa700 1 -- 172.21.15.101:0/4008811588 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6380d08b0 con 0x557e65501ad0 2023-07-18T03:24:09.658 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.661+0000 7ff6297fa700 1 -- 172.21.15.101:0/4008811588 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6380963d0 con 0x557e65501ad0 2023-07-18T03:24:09.658 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.661+0000 7ff6297fa700 1 -- 172.21.15.101:0/4008811588 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff638098300 con 0x557e65501ad0 2023-07-18T03:24:09.659 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.662+0000 7ff65cfe4b80 1 -- 172.21.15.101:0/4008811588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e65501ad0 msgr2=0x557e65503eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:09.659 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.662+0000 7ff65cfe4b80 1 --2- 172.21.15.101:0/4008811588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e65501ad0 0x557e65503eb0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff6380ca710 tx=0x7ff6380d5610 comp rx=0 tx=0).stop 2023-07-18T03:24:09.659 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.663+0000 7ff65cfe4b80 1 -- 172.21.15.101:0/4008811588 shutdown_connections 2023-07-18T03:24:09.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.663+0000 7ff65cfe4b80 1 --2- 172.21.15.101:0/4008811588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e65501ad0 0x557e65503eb0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.663+0000 7ff65cfe4b80 1 --2- 172.21.15.101:0/4008811588 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557e654f62d0 0x557e654f66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.663+0000 7ff65cfe4b80 1 --2- 172.21.15.101:0/4008811588 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557e654f6be0 0x557e65501590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.663+0000 7ff65cfe4b80 1 -- 172.21.15.101:0/4008811588 >> 172.21.15.101:0/4008811588 conn(0x557e654ecda0 msgr2=0x557e654f26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:09.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.663+0000 7ff65cfe4b80 1 -- 172.21.15.101:0/4008811588 shutdown_connections 2023-07-18T03:24:09.660 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.663+0000 7ff65cfe4b80 1 -- 172.21.15.101:0/4008811588 wait complete. 2023-07-18T03:24:09.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.663+0000 7ff65cfe4b80 1 -- start start 2023-07-18T03:24:09.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.664+0000 7ff65cfe4b80 1 Processor -- start 2023-07-18T03:24:09.663 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.666+0000 7ff65cfe4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e654f62d0 0x557e654fe8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:09.663 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.666+0000 7ff65cfe4b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557e654f6be0 0x557e654fede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:09.663 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.666+0000 7ff65cfe4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557e65501ad0 0x557e654ff320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:09.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.666+0000 7ff65cfe4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e653e8ac0 con 0x557e65501ad0 2023-07-18T03:24:09.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.666+0000 7ff65cfe4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e653e87c0 con 0x557e654f62d0 2023-07-18T03:24:09.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.666+0000 7ff65cfe4b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e652cc920 con 0x557e654f6be0 2023-07-18T03:24:09.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.666+0000 7ff65cfe4b80 10 client.0 _ll_register_callbacks cb 0x557e653e6760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:24:09.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.666+0000 7ff6427fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557e654f6be0 0x557e654fede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:09.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.666+0000 7ff642ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e654f62d0 0x557e654fe8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:09.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.666+0000 7ff6427fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557e654f6be0 0x557e654fede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:37192/0 (socket says 172.21.15.101:37192) 2023-07-18T03:24:09.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.666+0000 7ff6427fc700 1 -- 172.21.15.101:0/958528021 learned_addr learned my addr 172.21.15.101:0/958528021 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:09.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.668+0000 7ff6437fe700 1 --2- 172.21.15.101:0/958528021 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557e65501ad0 0x557e654ff320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:09.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.668+0000 7ff6437fe700 1 -- 172.21.15.101:0/958528021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557e654f6be0 msgr2=0x557e654fede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:09.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.668+0000 7ff6437fe700 1 --2- 172.21.15.101:0/958528021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557e654f6be0 0x557e654fede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.668+0000 7ff6437fe700 1 -- 172.21.15.101:0/958528021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e654f62d0 msgr2=0x557e654fe8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:09.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.668+0000 7ff6437fe700 1 --2- 172.21.15.101:0/958528021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e654f62d0 0x557e654fe8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.668+0000 7ff6437fe700 1 -- 172.21.15.101:0/958528021 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff63c0098f0 con 0x557e65501ad0 2023-07-18T03:24:09.667 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.669+0000 7ff6437fe700 1 --2- 172.21.15.101:0/958528021 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557e65501ad0 0x557e654ff320 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7ff63c00ac30 tx=0x7ff63c004be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e7e642cfdce6a20 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:09.668 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.669+0000 7ff626ff5700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:24:09.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.669+0000 7ff626ff5700 1 -- 172.21.15.101:0/958528021 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63c010040 con 0x557e65501ad0 2023-07-18T03:24:09.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.669+0000 7ff626ff5700 1 -- 172.21.15.101:0/958528021 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9549+0+0 (secure 0 0 0) 0x7ff63c006e40 con 0x557e65501ad0 2023-07-18T03:24:09.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.669+0000 7ff65cfe4b80 1 -- 172.21.15.101:0/958528021 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557e653a24f0 con 0x557e65501ad0 2023-07-18T03:24:09.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.670+0000 7ff626ff5700 1 -- 172.21.15.101:0/958528021 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff63c015d10 con 0x557e65501ad0 2023-07-18T03:24:09.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.670+0000 7ff65cfe4b80 10 client.18732 fetch_fsmap learned FSMap version 41 2023-07-18T03:24:09.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.670+0000 7ff65cfe4b80 1 -- 172.21.15.101:0/958528021 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7ff6441025a0 con 0x557e65501ad0 2023-07-18T03:24:09.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.671+0000 7ff626ff5700 1 -- 172.21.15.101:0/958528021 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557e653a24f0 con 0x557e65501ad0 2023-07-18T03:24:09.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.671+0000 7ff626ff5700 1 -- 172.21.15.101:0/958528021 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 41) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff63c01f070 con 0x557e65501ad0 2023-07-18T03:24:09.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.672+0000 7ff65cfe4b80 10 client.18732 fetch_fsmap finished waiting for FSMap version 41 2023-07-18T03:24:09.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.672+0000 7ff65cfe4b80 10 client.18732 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:24:09.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.672+0000 7ff65cfe4b80 20 client.18732 populate_metadata read hostname 'smithi101' 2023-07-18T03:24:09.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.672+0000 7ff65cfe4b80 1 --2- 172.21.15.101:0/958528021 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557e655114c0 0x557e655138d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:09.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.672+0000 7ff65cfe4b80 4 client.18732 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:24:09.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.672+0000 7ff65cfe4b80 1 -- 172.21.15.101:0/958528021 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557e65441eb0 con 0x557e655114c0 2023-07-18T03:24:09.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.672+0000 7ff642ffd700 1 --2- 172.21.15.101:0/958528021 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557e655114c0 0x557e655138d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:09.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.673+0000 7ff626ff5700 10 client.18732 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:24:09.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.673+0000 7ff642ffd700 1 --2- 172.21.15.101:0/958528021 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557e655114c0 0x557e655138d0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65d0f9dade44b957 server_cookie=a32971eadf48a1d6 in_seq=0 out_seq=0 2023-07-18T03:24:09.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.674+0000 7ff626ff5700 1 -- 172.21.15.101:0/958528021 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557e65441eb0 con 0x557e655114c0 2023-07-18T03:24:09.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.674+0000 7ff626ff5700 10 client.18732 handle_command_reply: tid=0 2023-07-18T03:24:09.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.674+0000 7ff65cfe4b80 1 client.18732 shutdown 2023-07-18T03:24:09.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.675+0000 7ff65cfe4b80 1 -- 172.21.15.101:0/958528021 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557e65501ad0 msgr2=0x557e654ff320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:09.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.675+0000 7ff65cfe4b80 1 --2- 172.21.15.101:0/958528021 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557e65501ad0 0x557e654ff320 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7ff63c00ac30 tx=0x7ff63c004be0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.675+0000 7ff65cfe4b80 1 -- 172.21.15.101:0/958528021 shutdown_connections 2023-07-18T03:24:09.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.675+0000 7ff65cfe4b80 1 --2- 172.21.15.101:0/958528021 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557e655114c0 0x557e655138d0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.675+0000 7ff65cfe4b80 1 --2- 172.21.15.101:0/958528021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e654f62d0 0x557e654fe8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.675+0000 7ff65cfe4b80 1 --2- 172.21.15.101:0/958528021 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557e65501ad0 0x557e654ff320 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.675+0000 7ff65cfe4b80 1 --2- 172.21.15.101:0/958528021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557e654f6be0 0x557e654fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.675+0000 7ff65cfe4b80 1 -- 172.21.15.101:0/958528021 >> 172.21.15.101:0/958528021 conn(0x557e654ecda0 msgr2=0x557e654f2290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:09.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.675+0000 7ff626ff5700 0 client.18732 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:24:09.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.675+0000 7ff65cfe4b80 1 -- 172.21.15.101:0/958528021 shutdown_connections 2023-07-18T03:24:09.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.675+0000 7ff65cfe4b80 1 -- 172.21.15.101:0/958528021 wait complete. 2023-07-18T03:24:09.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.675+0000 7ff65cfe4b80 20 client.18732 trim_cache size 0 max 16384 2023-07-18T03:24:09.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.702+0000 7ff65cfe4b80 1 Processor -- start 2023-07-18T03:24:09.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.705+0000 7ff65cfe4b80 1 -- start start 2023-07-18T03:24:09.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.706+0000 7ff65cfe4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557e654f62d0 0x557e655084b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:09.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.706+0000 7ff65cfe4b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557e654f6be0 0x557e654fee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:09.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.706+0000 7ff65cfe4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e65501ad0 0x557e654ff350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:09.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.706+0000 7ff65cfe4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e65501460 con 0x557e654f62d0 2023-07-18T03:24:09.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.706+0000 7ff65cfe4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e654f3cc0 con 0x557e65501ad0 2023-07-18T03:24:09.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.706+0000 7ff65cfe4b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e654f3b20 con 0x557e654f6be0 2023-07-18T03:24:09.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.706+0000 7ff6427fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557e654f6be0 0x557e654fee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:09.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.706+0000 7ff6427fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557e654f6be0 0x557e654fee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:37204/0 (socket says 172.21.15.101:37204) 2023-07-18T03:24:09.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.706+0000 7ff6427fc700 1 -- 172.21.15.101:0/1351759218 learned_addr learned my addr 172.21.15.101:0/1351759218 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:09.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.706+0000 7ff642ffd700 1 --2- 172.21.15.101:0/1351759218 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557e654f62d0 0x557e655084b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:09.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.706+0000 7ff6437fe700 1 --2- 172.21.15.101:0/1351759218 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e65501ad0 0x557e654ff350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:09.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.706+0000 7ff6427fc700 1 -- 172.21.15.101:0/1351759218 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e65501ad0 msgr2=0x557e654ff350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:09.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.706+0000 7ff6427fc700 1 --2- 172.21.15.101:0/1351759218 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e65501ad0 0x557e654ff350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.706+0000 7ff6427fc700 1 -- 172.21.15.101:0/1351759218 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557e654f62d0 msgr2=0x557e655084b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:09.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.706+0000 7ff6427fc700 1 --2- 172.21.15.101:0/1351759218 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557e654f62d0 0x557e655084b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.706+0000 7ff6427fc700 1 -- 172.21.15.101:0/1351759218 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e654ee5e0 con 0x557e654f6be0 2023-07-18T03:24:09.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.707+0000 7ff642ffd700 1 --2- 172.21.15.101:0/1351759218 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557e654f62d0 0x557e655084b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:24:09.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.707+0000 7ff6427fc700 1 --2- 172.21.15.101:0/1351759218 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557e654f6be0 0x557e654fee10 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ff638098240 tx=0x7ff6380981d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41bcc7a91abe7be1 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:09.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.707+0000 7ff629ffb700 1 -- 172.21.15.101:0/1351759218 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6380d08b0 con 0x557e654f6be0 2023-07-18T03:24:09.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.707+0000 7ff629ffb700 1 -- 172.21.15.101:0/1351759218 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff638098300 con 0x557e654f6be0 2023-07-18T03:24:09.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.707+0000 7ff629ffb700 1 -- 172.21.15.101:0/1351759218 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6380a2570 con 0x557e654f6be0 2023-07-18T03:24:09.711 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:09 smithi179 ceph-mon[129229]: pgmap v521: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 12 MiB/s wr, 702 op/s 2023-07-18T03:24:09.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.708+0000 7ff65cfe4b80 1 -- 172.21.15.101:0/1351759218 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557e654f6be0 msgr2=0x557e654fee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:09.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.708+0000 7ff65cfe4b80 1 --2- 172.21.15.101:0/1351759218 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557e654f6be0 0x557e654fee10 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ff638098240 tx=0x7ff6380981d0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.709+0000 7ff65cfe4b80 1 -- 172.21.15.101:0/1351759218 shutdown_connections 2023-07-18T03:24:09.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.709+0000 7ff65cfe4b80 1 --2- 172.21.15.101:0/1351759218 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e65501ad0 0x557e654ff350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.709+0000 7ff65cfe4b80 1 --2- 172.21.15.101:0/1351759218 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557e654f62d0 0x557e655084b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.709+0000 7ff65cfe4b80 1 --2- 172.21.15.101:0/1351759218 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557e654f6be0 0x557e654fee10 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.709+0000 7ff65cfe4b80 1 -- 172.21.15.101:0/1351759218 >> 172.21.15.101:0/1351759218 conn(0x557e654ecda0 msgr2=0x557e654f9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:09.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.710+0000 7ff65cfe4b80 1 -- 172.21.15.101:0/1351759218 shutdown_connections 2023-07-18T03:24:09.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.710+0000 7ff65cfe4b80 1 -- 172.21.15.101:0/1351759218 wait complete. 2023-07-18T03:24:09.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.710+0000 7ff65cfe4b80 1 -- start start 2023-07-18T03:24:09.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.711+0000 7ff65cfe4b80 1 Processor -- start 2023-07-18T03:24:09.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.712+0000 7ff65cfe4b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557e654f62d0 0x557e6555d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:09.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.712+0000 7ff65cfe4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557e654f6be0 0x557e6555dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:09.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.712+0000 7ff65cfe4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e65501ad0 0x557e65559310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:09.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.712+0000 7ff65cfe4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e65559a20 con 0x557e654f6be0 2023-07-18T03:24:09.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.712+0000 7ff65cfe4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e65559b90 con 0x557e65501ad0 2023-07-18T03:24:09.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.712+0000 7ff65cfe4b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e65559d50 con 0x557e654f62d0 2023-07-18T03:24:09.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.712+0000 7ff65cfe4b80 10 client.0 _ll_register_callbacks cb 0x7ff63c0167b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:24:09.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.713+0000 7ff6437fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e65501ad0 0x557e65559310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:09.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.713+0000 7ff6437fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e65501ad0 0x557e65559310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:41274/0 (socket says 172.21.15.101:41274) 2023-07-18T03:24:09.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.713+0000 7ff6437fe700 1 -- 172.21.15.101:0/2389969617 learned_addr learned my addr 172.21.15.101:0/2389969617 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:09.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.713+0000 7ff642ffd700 1 --2- 172.21.15.101:0/2389969617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557e654f62d0 0x557e6555d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:09.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.713+0000 7ff6427fc700 1 --2- 172.21.15.101:0/2389969617 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557e654f6be0 0x557e6555dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:09.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.714+0000 7ff642ffd700 1 -- 172.21.15.101:0/2389969617 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e65501ad0 msgr2=0x557e65559310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:09.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.714+0000 7ff642ffd700 1 --2- 172.21.15.101:0/2389969617 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e65501ad0 0x557e65559310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.714+0000 7ff642ffd700 1 -- 172.21.15.101:0/2389969617 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557e654f6be0 msgr2=0x557e6555dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:09.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.714+0000 7ff642ffd700 1 --2- 172.21.15.101:0/2389969617 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557e654f6be0 0x557e6555dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.714+0000 7ff642ffd700 1 -- 172.21.15.101:0/2389969617 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557e65276de0 con 0x557e654f62d0 2023-07-18T03:24:09.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.715+0000 7ff642ffd700 1 --2- 172.21.15.101:0/2389969617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557e654f62d0 0x557e6555d7b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff6340098e0 tx=0x7ff63400b8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d8291205802fb69 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:09.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.715+0000 7ff626ff5700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:24:09.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.715+0000 7ff626ff5700 1 -- 172.21.15.101:0/2389969617 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff634026610 con 0x557e654f62d0 2023-07-18T03:24:09.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.715+0000 7ff65cfe4b80 1 -- 172.21.15.101:0/2389969617 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff6380963d0 con 0x557e654f62d0 2023-07-18T03:24:09.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.715+0000 7ff626ff5700 1 -- 172.21.15.101:0/2389969617 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9549+0+0 (secure 0 0 0) 0x7ff63400be80 con 0x557e654f62d0 2023-07-18T03:24:09.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.715+0000 7ff626ff5700 1 -- 172.21.15.101:0/2389969617 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff634026a40 con 0x557e654f62d0 2023-07-18T03:24:09.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.716+0000 7ff65cfe4b80 10 client.28831 fetch_fsmap learned FSMap version 41 2023-07-18T03:24:09.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.716+0000 7ff65cfe4b80 1 -- 172.21.15.101:0/2389969617 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x557e6525fc60 con 0x557e654f62d0 2023-07-18T03:24:09.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.717+0000 7ff626ff5700 1 -- 172.21.15.101:0/2389969617 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6380963d0 con 0x557e654f62d0 2023-07-18T03:24:09.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.717+0000 7ff626ff5700 1 -- 172.21.15.101:0/2389969617 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 41) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff6340046b0 con 0x557e654f62d0 2023-07-18T03:24:09.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.717+0000 7ff65cfe4b80 10 client.28831 fetch_fsmap finished waiting for FSMap version 41 2023-07-18T03:24:09.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.717+0000 7ff65cfe4b80 10 client.28831 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:24:09.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.717+0000 7ff65cfe4b80 20 client.28831 populate_metadata read hostname 'smithi101' 2023-07-18T03:24:09.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.717+0000 7ff65cfe4b80 1 --2- 172.21.15.101:0/2389969617 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557e655667c0 0x557e65568bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:09.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.717+0000 7ff65cfe4b80 4 client.28831 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:24:09.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.717+0000 7ff65cfe4b80 1 -- 172.21.15.101:0/2389969617 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557e65404050 con 0x557e655667c0 2023-07-18T03:24:09.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.717+0000 7ff6427fc700 1 --2- 172.21.15.101:0/2389969617 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557e655667c0 0x557e65568bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:09.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.718+0000 7ff626ff5700 10 client.28831 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:24:09.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.718+0000 7ff6427fc700 1 --2- 172.21.15.101:0/2389969617 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557e655667c0 0x557e65568bd0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c150e5d64705dfc server_cookie=7b113ce96528de7a in_seq=0 out_seq=0 2023-07-18T03:24:09.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.718+0000 7ff626ff5700 1 -- 172.21.15.101:0/2389969617 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x557e65404050 con 0x557e655667c0 2023-07-18T03:24:09.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.718+0000 7ff626ff5700 10 client.28831 handle_command_reply: tid=0 2023-07-18T03:24:09.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.718+0000 7ff65cfe4b80 1 client.28831 shutdown 2023-07-18T03:24:09.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.720+0000 7ff65cfe4b80 1 -- 172.21.15.101:0/2389969617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557e654f62d0 msgr2=0x557e6555d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:09.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.720+0000 7ff65cfe4b80 1 --2- 172.21.15.101:0/2389969617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557e654f62d0 0x557e6555d7b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff6340098e0 tx=0x7ff63400b8c0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.721+0000 7ff65cfe4b80 1 -- 172.21.15.101:0/2389969617 shutdown_connections 2023-07-18T03:24:09.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.721+0000 7ff65cfe4b80 1 --2- 172.21.15.101:0/2389969617 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557e655667c0 0x557e65568bd0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.721+0000 7ff65cfe4b80 1 --2- 172.21.15.101:0/2389969617 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e65501ad0 0x557e65559310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.721+0000 7ff65cfe4b80 1 --2- 172.21.15.101:0/2389969617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557e654f62d0 0x557e6555d7b0 secure :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff6340098e0 tx=0x7ff63400b8c0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.721+0000 7ff65cfe4b80 1 --2- 172.21.15.101:0/2389969617 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557e654f6be0 0x557e6555dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.721+0000 7ff65cfe4b80 1 -- 172.21.15.101:0/2389969617 >> 172.21.15.101:0/2389969617 conn(0x557e654ecda0 msgr2=0x557e654f9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:09.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.721+0000 7ff626ff5700 0 client.28831 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:24:09.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.727+0000 7ff65cfe4b80 1 -- 172.21.15.101:0/2389969617 shutdown_connections 2023-07-18T03:24:09.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.729+0000 7ff65cfe4b80 1 -- 172.21.15.101:0/2389969617 wait complete. 2023-07-18T03:24:09.733 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:24:09.733 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "scrub active (87 inodes in the stack)", 2023-07-18T03:24:09.733 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:24:09.733 INFO:teuthology.orchestra.run.smithi101.stdout: "666b0d9a-2af3-478a-bad3-ce9f13c9ba69": { 2023-07-18T03:24:09.733 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:24:09.734 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "666b0d9a-2af3-478a-bad3-ce9f13c9ba69", 2023-07-18T03:24:09.734 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:24:09.734 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:24:09.734 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:24:09.734 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:24:09.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.735+0000 7ff65cfe4b80 20 client.28831 trim_cache size 0 max 16384 2023-07-18T03:24:09.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.736+0000 7ff6495ff700 1 -- 172.21.15.101:0/1592309371 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff62c0a1340 msgr2=0x7ff62c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:09.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.736+0000 7ff6495ff700 1 --2- 172.21.15.101:0/1592309371 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff62c0a1340 0x7ff62c0a37f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff6440697f0 tx=0x7ff63c00d040 comp rx=0 tx=0).stop 2023-07-18T03:24:09.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.736+0000 7ff6495ff700 1 -- 172.21.15.101:0/1592309371 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff644075350 msgr2=0x7ff64406af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:09.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.736+0000 7ff6495ff700 1 --2- 172.21.15.101:0/1592309371 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff644075350 0x7ff64406af20 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ff63800e9a0 tx=0x7ff63800ed60 comp rx=0 tx=0).stop 2023-07-18T03:24:09.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.737+0000 7ff6495ff700 1 -- 172.21.15.101:0/1592309371 shutdown_connections 2023-07-18T03:24:09.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.737+0000 7ff6495ff700 1 --2- 172.21.15.101:0/1592309371 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff62c0a1340 0x7ff62c0a37f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.737+0000 7ff6495ff700 1 --2- 172.21.15.101:0/1592309371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6441109a0 0x7ff6441b07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.737+0000 7ff6495ff700 1 --2- 172.21.15.101:0/1592309371 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff644074a40 0x7ff64406a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:09.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.737+0000 7ff6495ff700 1 --2- 172.21.15.101:0/1592309371 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff644075350 0x7ff64406af20 secure :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ff63800e9a0 tx=0x7ff63800ed60 comp rx=0 tx=0).stop 2023-07-18T03:24:09.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.737+0000 7ff6495ff700 1 -- 172.21.15.101:0/1592309371 >> 172.21.15.101:0/1592309371 conn(0x7ff6440fd7e0 msgr2=0x7ff6440ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:09.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.739+0000 7ff6495ff700 1 -- 172.21.15.101:0/1592309371 shutdown_connections 2023-07-18T03:24:09.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:09.739+0000 7ff6495ff700 1 -- 172.21.15.101:0/1592309371 wait complete. 2023-07-18T03:24:09.776 INFO:tasks.cephfs.filesystem:scrub status for tag:666b0d9a-2af3-478a-bad3-ce9f13c9ba69 - {'path': '/', 'tag': '666b0d9a-2af3-478a-bad3-ce9f13c9ba69', 'options': 'recursive,force'} 2023-07-18T03:24:09.996 INFO:tasks.workunit.client.0.smithi101.stdout: 10 120357 29.53 MB/sec execute 94 sec latency 3678.297 ms 2023-07-18T03:24:10.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:10 smithi101 ceph-mon[126379]: scrub queued for path: / 2023-07-18T03:24:10.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:10 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:24:10.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:10 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:24:10.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:10 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:24:10.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:10 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:24:10.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:10 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:24:10.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:10 smithi155 ceph-mon[128885]: scrub queued for path: / 2023-07-18T03:24:10.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:10 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:24:10.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:10 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:24:10.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:10 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:24:10.550 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:10 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:24:10.550 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:10 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:24:10.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:10 smithi179 ceph-mon[129229]: scrub queued for path: / 2023-07-18T03:24:10.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:10 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:24:10.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:10 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:24:10.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:10 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:24:10.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:10 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:24:10.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:10 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:24:10.996 INFO:tasks.workunit.client.0.smithi101.stdout: 10 120875 29.50 MB/sec execute 95 sec latency 4678.365 ms 2023-07-18T03:24:11.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:11 smithi101 ceph-mon[126379]: pgmap v522: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 8.9 MiB/s wr, 664 op/s 2023-07-18T03:24:11.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:11 smithi155 ceph-mon[128885]: pgmap v522: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 8.9 MiB/s wr, 664 op/s 2023-07-18T03:24:11.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:11 smithi179 ceph-mon[129229]: pgmap v522: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 8.9 MiB/s wr, 664 op/s 2023-07-18T03:24:11.996 INFO:tasks.workunit.client.0.smithi101.stdout: 10 121478 29.42 MB/sec execute 96 sec latency 5678.438 ms 2023-07-18T03:24:12.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:12 smithi155 ceph-mon[128885]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:24:12.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:12 smithi179 ceph-mon[129229]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:24:12.744 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:12 smithi101 ceph-mon[126379]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:24:12.996 INFO:tasks.workunit.client.0.smithi101.stdout: 10 122053 29.47 MB/sec execute 97 sec latency 6678.591 ms 2023-07-18T03:24:13.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:13 smithi155 ceph-mon[128885]: pgmap v523: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.4 MiB/s wr, 548 op/s 2023-07-18T03:24:13.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:13 smithi179 ceph-mon[129229]: pgmap v523: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.4 MiB/s wr, 548 op/s 2023-07-18T03:24:13.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:13 smithi101 ceph-mon[126379]: pgmap v523: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.4 MiB/s wr, 548 op/s 2023-07-18T03:24:13.996 INFO:tasks.workunit.client.0.smithi101.stdout: 10 122660 29.58 MB/sec execute 98 sec latency 7678.648 ms 2023-07-18T03:24:14.996 INFO:tasks.workunit.client.0.smithi101.stdout: 10 122907 29.43 MB/sec execute 99 sec latency 8662.175 ms 2023-07-18T03:24:15.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:15 smithi179 ceph-mon[129229]: pgmap v524: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.4 MiB/s wr, 548 op/s 2023-07-18T03:24:15.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:15 smithi179 ceph-mon[129229]: osdmap e114: 12 total, 12 up, 12 in 2023-07-18T03:24:15.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:15 smithi101 ceph-mon[126379]: pgmap v524: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.4 MiB/s wr, 548 op/s 2023-07-18T03:24:15.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:15 smithi101 ceph-mon[126379]: osdmap e114: 12 total, 12 up, 12 in 2023-07-18T03:24:15.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:15 smithi155 ceph-mon[128885]: pgmap v524: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.4 MiB/s wr, 548 op/s 2023-07-18T03:24:15.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:15 smithi155 ceph-mon[128885]: osdmap e114: 12 total, 12 up, 12 in 2023-07-18T03:24:15.996 INFO:tasks.workunit.client.0.smithi101.stdout: 10 123304 29.35 MB/sec execute 100 sec latency 4699.881 ms 2023-07-18T03:24:16.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:16 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:24:16.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:16 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:24:16.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:16 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:24:16.996 INFO:tasks.workunit.client.0.smithi101.stdout: 10 123672 29.21 MB/sec execute 101 sec latency 5699.936 ms 2023-07-18T03:24:17.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:17 smithi179 ceph-mon[129229]: pgmap v526: 129 pgs: 129 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.3 MiB/s wr, 484 op/s 2023-07-18T03:24:17.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:17 smithi101 ceph-mon[126379]: pgmap v526: 129 pgs: 129 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.3 MiB/s wr, 484 op/s 2023-07-18T03:24:17.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:17 smithi155 ceph-mon[128885]: pgmap v526: 129 pgs: 129 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.3 MiB/s wr, 484 op/s 2023-07-18T03:24:17.997 INFO:tasks.workunit.client.0.smithi101.stdout: 10 124197 29.19 MB/sec execute 102 sec latency 6700.016 ms 2023-07-18T03:24:18.997 INFO:tasks.workunit.client.0.smithi101.stdout: 10 124617 29.16 MB/sec execute 103 sec latency 7700.524 ms 2023-07-18T03:24:19.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:19 smithi179 ceph-mon[129229]: pgmap v527: 129 pgs: 129 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 7.4 MiB/s wr, 433 op/s 2023-07-18T03:24:19.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:19 smithi101 ceph-mon[126379]: pgmap v527: 129 pgs: 129 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 7.4 MiB/s wr, 433 op/s 2023-07-18T03:24:19.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:19 smithi155 ceph-mon[128885]: pgmap v527: 129 pgs: 129 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 7.4 MiB/s wr, 433 op/s 2023-07-18T03:24:19.997 INFO:tasks.workunit.client.0.smithi101.stdout: 10 124977 29.02 MB/sec execute 104 sec latency 8700.624 ms 2023-07-18T03:24:20.997 INFO:tasks.workunit.client.0.smithi101.stdout: 10 125409 29.00 MB/sec execute 105 sec latency 9230.031 ms 2023-07-18T03:24:21.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:21 smithi179 ceph-mon[129229]: pgmap v528: 129 pgs: 129 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 7.3 MiB/s wr, 427 op/s 2023-07-18T03:24:21.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:21 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:24:21.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:21 smithi101 ceph-mon[126379]: pgmap v528: 129 pgs: 129 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 7.3 MiB/s wr, 427 op/s 2023-07-18T03:24:21.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:21 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:24:21.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:21 smithi155 ceph-mon[128885]: pgmap v528: 129 pgs: 129 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 7.3 MiB/s wr, 427 op/s 2023-07-18T03:24:21.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:21 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:24:21.997 INFO:tasks.workunit.client.0.smithi101.stdout: 10 126004 29.00 MB/sec execute 106 sec latency 5629.652 ms 2023-07-18T03:24:22.997 INFO:tasks.workunit.client.0.smithi101.stdout: 10 126522 29.02 MB/sec execute 107 sec latency 220.045 ms 2023-07-18T03:24:23.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:23 smithi179 ceph-mon[129229]: pgmap v529: 129 pgs: 129 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.9 MiB/s wr, 391 op/s 2023-07-18T03:24:23.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:23 smithi101 ceph-mon[126379]: pgmap v529: 129 pgs: 129 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.9 MiB/s wr, 391 op/s 2023-07-18T03:24:23.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:23 smithi155 ceph-mon[128885]: pgmap v529: 129 pgs: 129 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.9 MiB/s wr, 391 op/s 2023-07-18T03:24:23.997 INFO:tasks.workunit.client.0.smithi101.stdout: 10 126963 28.89 MB/sec execute 108 sec latency 156.159 ms 2023-07-18T03:24:24.998 INFO:tasks.workunit.client.0.smithi101.stdout: 10 127645 29.04 MB/sec execute 109 sec latency 115.838 ms 2023-07-18T03:24:25.454 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:25 smithi179 ceph-mon[129229]: pgmap v530: 129 pgs: 129 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.9 MiB/s wr, 391 op/s 2023-07-18T03:24:25.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:25 smithi101 ceph-mon[126379]: pgmap v530: 129 pgs: 129 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.9 MiB/s wr, 391 op/s 2023-07-18T03:24:25.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:25 smithi155 ceph-mon[128885]: pgmap v530: 129 pgs: 129 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.9 MiB/s wr, 391 op/s 2023-07-18T03:24:25.997 INFO:tasks.workunit.client.0.smithi101.stdout: 10 128168 28.99 MB/sec execute 110 sec latency 89.110 ms 2023-07-18T03:24:26.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:26 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:24:26.454 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:26 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:24:26.454 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:26 smithi179 ceph-mon[129229]: scrub summary: idle 2023-07-18T03:24:26.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:26 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:24:26.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:26 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:24:26.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:26 smithi101 ceph-mon[126379]: scrub summary: idle 2023-07-18T03:24:26.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:26 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:24:26.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:26 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:24:26.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:26 smithi155 ceph-mon[128885]: scrub summary: idle 2023-07-18T03:24:26.998 INFO:tasks.workunit.client.0.smithi101.stdout: 10 128826 29.05 MB/sec execute 111 sec latency 75.938 ms 2023-07-18T03:24:27.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:27 smithi179 ceph-mon[129229]: pgmap v531: 129 pgs: 129 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.0 MiB/s wr, 516 op/s 2023-07-18T03:24:27.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:27 smithi101 ceph-mon[126379]: pgmap v531: 129 pgs: 129 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.0 MiB/s wr, 516 op/s 2023-07-18T03:24:27.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:27 smithi155 ceph-mon[128885]: pgmap v531: 129 pgs: 129 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.0 MiB/s wr, 516 op/s 2023-07-18T03:24:27.998 INFO:tasks.workunit.client.0.smithi101.stdout: 10 129444 29.11 MB/sec execute 112 sec latency 103.207 ms 2023-07-18T03:24:28.998 INFO:tasks.workunit.client.0.smithi101.stdout: 10 129652 28.98 MB/sec execute 113 sec latency 517.482 ms 2023-07-18T03:24:29.456 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:29 smithi179 ceph-mon[129229]: pgmap v532: 129 pgs: 129 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 6.4 MiB/s wr, 377 op/s 2023-07-18T03:24:29.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:29 smithi101 ceph-mon[126379]: pgmap v532: 129 pgs: 129 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 6.4 MiB/s wr, 377 op/s 2023-07-18T03:24:29.550 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:29 smithi155 ceph-mon[128885]: pgmap v532: 129 pgs: 129 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 6.4 MiB/s wr, 377 op/s 2023-07-18T03:24:29.998 INFO:tasks.workunit.client.0.smithi101.stdout: 10 129836 28.81 MB/sec execute 114 sec latency 692.449 ms 2023-07-18T03:24:30.998 INFO:tasks.workunit.client.0.smithi101.stdout: 10 130270 28.74 MB/sec execute 115 sec latency 362.991 ms 2023-07-18T03:24:31.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:31 smithi101 ceph-mon[126379]: pgmap v533: 129 pgs: 129 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 8.7 MiB/s wr, 450 op/s 2023-07-18T03:24:31.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:31 smithi155 ceph-mon[128885]: pgmap v533: 129 pgs: 129 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 8.7 MiB/s wr, 450 op/s 2023-07-18T03:24:31.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:31 smithi179 ceph-mon[129229]: pgmap v533: 129 pgs: 129 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 8.7 MiB/s wr, 450 op/s 2023-07-18T03:24:31.998 INFO:tasks.workunit.client.0.smithi101.stdout: 10 130926 28.80 MB/sec execute 116 sec latency 1363.045 ms 2023-07-18T03:24:32.998 INFO:tasks.workunit.client.0.smithi101.stdout: 10 131543 28.85 MB/sec execute 117 sec latency 2363.086 ms 2023-07-18T03:24:33.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:33 smithi101 ceph-mon[126379]: pgmap v534: 129 pgs: 129 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 11 MiB/s wr, 485 op/s 2023-07-18T03:24:33.550 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:33 smithi155 ceph-mon[128885]: pgmap v534: 129 pgs: 129 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 11 MiB/s wr, 485 op/s 2023-07-18T03:24:33.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:33 smithi179 ceph-mon[129229]: pgmap v534: 129 pgs: 129 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 11 MiB/s wr, 485 op/s 2023-07-18T03:24:33.998 INFO:tasks.workunit.client.0.smithi101.stdout: 10 132066 28.79 MB/sec execute 118 sec latency 3363.136 ms 2023-07-18T03:24:34.999 INFO:tasks.workunit.client.0.smithi101.stdout: 10 132104 28.58 MB/sec execute 119 sec latency 4363.615 ms 2023-07-18T03:24:35.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:35 smithi155 ceph-mon[128885]: pgmap v535: 129 pgs: 129 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 10 MiB/s wr, 448 op/s 2023-07-18T03:24:35.705 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:35 smithi179 ceph-mon[129229]: pgmap v535: 129 pgs: 129 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 10 MiB/s wr, 448 op/s 2023-07-18T03:24:35.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:35 smithi101 ceph-mon[126379]: pgmap v535: 129 pgs: 129 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 10 MiB/s wr, 448 op/s 2023-07-18T03:24:36.001 INFO:tasks.workunit.client.0.smithi101.stdout: 10 132295 28.43 MB/sec execute 120 sec latency 5076.714 ms 2023-07-18T03:24:37.000 INFO:tasks.workunit.client.0.smithi101.stdout: 10 132525 28.27 MB/sec execute 121 sec latency 181.073 ms 2023-07-18T03:24:37.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:37 smithi155 ceph-mon[128885]: pgmap v536: 129 pgs: 129 active+clean; 1.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 17 MiB/s wr, 679 op/s 2023-07-18T03:24:37.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:37 smithi179 ceph-mon[129229]: pgmap v536: 129 pgs: 129 active+clean; 1.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 17 MiB/s wr, 679 op/s 2023-07-18T03:24:37.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:37 smithi101 ceph-mon[126379]: pgmap v536: 129 pgs: 129 active+clean; 1.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 17 MiB/s wr, 679 op/s 2023-07-18T03:24:37.999 INFO:tasks.workunit.client.0.smithi101.stdout: 10 133079 28.33 MB/sec execute 122 sec latency 141.205 ms 2023-07-18T03:24:39.000 INFO:tasks.workunit.client.0.smithi101.stdout: 10 133710 28.37 MB/sec execute 123 sec latency 99.892 ms 2023-07-18T03:24:39.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:39 smithi155 ceph-mon[128885]: pgmap v537: 129 pgs: 129 active+clean; 1.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 16 MiB/s wr, 547 op/s 2023-07-18T03:24:39.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:39 smithi179 ceph-mon[129229]: pgmap v537: 129 pgs: 129 active+clean; 1.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 16 MiB/s wr, 547 op/s 2023-07-18T03:24:39.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:39 smithi101 ceph-mon[126379]: pgmap v537: 129 pgs: 129 active+clean; 1.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 16 MiB/s wr, 547 op/s 2023-07-18T03:24:39.777 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:24:40.001 INFO:tasks.workunit.client.0.smithi101.stdout: 10 134244 28.33 MB/sec execute 124 sec latency 106.109 ms 2023-07-18T03:24:40.096 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.099+0000 7fcd787ff700 1 -- 172.21.15.101:0/3520419132 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd7010e280 msgr2=0x7fcd7010e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:40.096 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.099+0000 7fcd787ff700 1 --2- 172.21.15.101:0/3520419132 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd7010e280 0x7fcd7010e650 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c0098f0 tx=0x7fcd5c009c00 comp rx=0 tx=0).stop 2023-07-18T03:24:40.102 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.103+0000 7fcd787ff700 1 -- 172.21.15.101:0/3520419132 shutdown_connections 2023-07-18T03:24:40.102 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.103+0000 7fcd787ff700 1 --2- 172.21.15.101:0/3520419132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd7006d3a0 0x7fcd7011b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:40.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.103+0000 7fcd787ff700 1 --2- 172.21.15.101:0/3520419132 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd7010e280 0x7fcd7010e650 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:40.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.103+0000 7fcd787ff700 1 --2- 172.21.15.101:0/3520419132 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcd7006c9a0 0x7fcd7006ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:40.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.103+0000 7fcd787ff700 1 -- 172.21.15.101:0/3520419132 >> 172.21.15.101:0/3520419132 conn(0x7fcd7006bf70 msgr2=0x7fcd70070ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:40.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.107+0000 7fcd787ff700 1 -- 172.21.15.101:0/3520419132 shutdown_connections 2023-07-18T03:24:40.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.107+0000 7fcd787ff700 1 -- 172.21.15.101:0/3520419132 wait complete. 2023-07-18T03:24:40.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.108+0000 7fcd787ff700 1 Processor -- start 2023-07-18T03:24:40.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.109+0000 7fcd787ff700 1 -- start start 2023-07-18T03:24:40.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.109+0000 7fcd787ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcd7006c9a0 0x7fcd70118990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:40.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.109+0000 7fcd787ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd7006d3a0 0x7fcd70118ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:40.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.109+0000 7fcd787ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd7010e280 0x7fcd70113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:40.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.109+0000 7fcd787ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd70114150 con 0x7fcd7006c9a0 2023-07-18T03:24:40.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.109+0000 7fcd787ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd701142c0 con 0x7fcd7006d3a0 2023-07-18T03:24:40.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.109+0000 7fcd787ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd70114480 con 0x7fcd7010e280 2023-07-18T03:24:40.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.110+0000 7fcd76d9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd7010e280 0x7fcd70113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:40.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.110+0000 7fcd76d9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd7010e280 0x7fcd70113b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:40570/0 (socket says 172.21.15.101:40570) 2023-07-18T03:24:40.108 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.110+0000 7fcd76d9c700 1 -- 172.21.15.101:0/3020938387 learned_addr learned my addr 172.21.15.101:0/3020938387 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:40.108 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.110+0000 7fcd7659b700 1 --2- 172.21.15.101:0/3020938387 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcd7006c9a0 0x7fcd70118990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:40.108 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.111+0000 7fcd7659b700 1 -- 172.21.15.101:0/3020938387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd7010e280 msgr2=0x7fcd70113b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:40.108 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.111+0000 7fcd7659b700 1 --2- 172.21.15.101:0/3020938387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd7010e280 0x7fcd70113b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:40.108 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.111+0000 7fcd7659b700 1 -- 172.21.15.101:0/3020938387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd7006d3a0 msgr2=0x7fcd70118ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:40.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.111+0000 7fcd7659b700 1 --2- 172.21.15.101:0/3020938387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd7006d3a0 0x7fcd70118ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:40.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.111+0000 7fcd7659b700 1 -- 172.21.15.101:0/3020938387 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd5c009520 con 0x7fcd7006c9a0 2023-07-18T03:24:40.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.112+0000 7fcd7659b700 1 --2- 172.21.15.101:0/3020938387 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcd7006c9a0 0x7fcd70118990 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c009230 tx=0x7fcd5c00fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:40.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.115+0000 7fcd6f7fe700 1 -- 172.21.15.101:0/3020938387 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c01c070 con 0x7fcd7006c9a0 2023-07-18T03:24:40.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.115+0000 7fcd6f7fe700 1 -- 172.21.15.101:0/3020938387 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd5c00bbd0 con 0x7fcd7006c9a0 2023-07-18T03:24:40.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.115+0000 7fcd6f7fe700 1 -- 172.21.15.101:0/3020938387 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c003970 con 0x7fcd7006c9a0 2023-07-18T03:24:40.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.116+0000 7fcd787ff700 1 -- 172.21.15.101:0/3020938387 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd70114700 con 0x7fcd7006c9a0 2023-07-18T03:24:40.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.116+0000 7fcd787ff700 1 -- 172.21.15.101:0/3020938387 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd701176f0 con 0x7fcd7006c9a0 2023-07-18T03:24:40.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.123+0000 7fcd8c110b80 1 Processor -- start 2023-07-18T03:24:40.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.124+0000 7fcd8c110b80 1 -- start start 2023-07-18T03:24:40.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.124+0000 7fcd8c110b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558b5684b2d0 0x558b5684b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:40.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.124+0000 7fcd8c110b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b5684bbe0 0x558b56856590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:40.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.124+0000 7fcd8c110b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558b56856ad0 0x558b56858eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:40.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.124+0000 7fcd8c110b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b56751cb0 con 0x558b5684b2d0 2023-07-18T03:24:40.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.124+0000 7fcd8c110b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b567a4a40 con 0x558b56856ad0 2023-07-18T03:24:40.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.124+0000 7fcd8c110b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b5679ded0 con 0x558b5684bbe0 2023-07-18T03:24:40.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.128+0000 7fcd76d9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558b5684b2d0 0x558b5684b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:40.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.128+0000 7fcd76d9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558b5684b2d0 0x558b5684b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:51732/0 (socket says 172.21.15.101:51732) 2023-07-18T03:24:40.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.128+0000 7fcd76d9c700 1 -- 172.21.15.101:0/2241890868 learned_addr learned my addr 172.21.15.101:0/2241890868 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:40.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.129+0000 7fcd75d9a700 1 --2- 172.21.15.101:0/2241890868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558b56856ad0 0x558b56858eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:40.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.130+0000 7fcd6f7fe700 1 -- 172.21.15.101:0/3020938387 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fcd5c020030 con 0x7fcd7006c9a0 2023-07-18T03:24:40.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.131+0000 7fcd76d9c700 1 -- 172.21.15.101:0/2241890868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b5684bbe0 msgr2=0x558b56856590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:40.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.131+0000 7fcd76d9c700 1 --2- 172.21.15.101:0/2241890868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b5684bbe0 0x558b56856590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:40.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.131+0000 7fcd76d9c700 1 -- 172.21.15.101:0/2241890868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558b56856ad0 msgr2=0x558b56858eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:40.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.131+0000 7fcd76d9c700 1 --2- 172.21.15.101:0/2241890868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558b56856ad0 0x558b56858eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:40.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.131+0000 7fcd76d9c700 1 -- 172.21.15.101:0/2241890868 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b565cbde0 con 0x558b5684b2d0 2023-07-18T03:24:40.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.131+0000 7fcd76d9c700 1 --2- 172.21.15.101:0/2241890868 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558b5684b2d0 0x558b5684b6a0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009b30 tx=0x7fcd68009e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7011fd9d99fca33a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:40.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.132+0000 7fcd6cff9700 1 -- 172.21.15.101:0/2241890868 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6800e930 con 0x558b5684b2d0 2023-07-18T03:24:40.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.132+0000 7fcd6cff9700 1 -- 172.21.15.101:0/2241890868 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd68013470 con 0x558b5684b2d0 2023-07-18T03:24:40.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.132+0000 7fcd6cff9700 1 -- 172.21.15.101:0/2241890868 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd68012630 con 0x558b5684b2d0 2023-07-18T03:24:40.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.132+0000 7fcd6f7fe700 1 --2- 172.21.15.101:0/3020938387 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fcd600a1400 0x7fcd600a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:40.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.132+0000 7fcd6f7fe700 1 -- 172.21.15.101:0/3020938387 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fcd5c0ce0c0 con 0x7fcd7006c9a0 2023-07-18T03:24:40.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.133+0000 7fcd8c110b80 1 -- 172.21.15.101:0/2241890868 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558b5684b2d0 msgr2=0x558b5684b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:40.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.133+0000 7fcd8c110b80 1 --2- 172.21.15.101:0/2241890868 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558b5684b2d0 0x558b5684b6a0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009b30 tx=0x7fcd68009e40 comp rx=0 tx=0).stop 2023-07-18T03:24:40.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.133+0000 7fcd8c110b80 1 -- 172.21.15.101:0/2241890868 shutdown_connections 2023-07-18T03:24:40.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.133+0000 7fcd8c110b80 1 --2- 172.21.15.101:0/2241890868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558b56856ad0 0x558b56858eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:40.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.133+0000 7fcd8c110b80 1 --2- 172.21.15.101:0/2241890868 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558b5684b2d0 0x558b5684b6a0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:40.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.133+0000 7fcd8c110b80 1 --2- 172.21.15.101:0/2241890868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b5684bbe0 0x558b56856590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:40.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.133+0000 7fcd8c110b80 1 -- 172.21.15.101:0/2241890868 >> 172.21.15.101:0/2241890868 conn(0x558b56841da0 msgr2=0x558b568476a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:40.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.133+0000 7fcd7659b700 1 --2- 172.21.15.101:0/3020938387 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fcd600a1400 0x7fcd600a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:40.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.133+0000 7fcd8c110b80 1 -- 172.21.15.101:0/2241890868 shutdown_connections 2023-07-18T03:24:40.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.134+0000 7fcd8c110b80 1 -- 172.21.15.101:0/2241890868 wait complete. 2023-07-18T03:24:40.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.134+0000 7fcd7659b700 1 --2- 172.21.15.101:0/3020938387 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fcd600a1400 0x7fcd600a38b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c09ab10 tx=0x7fcd5c000bc0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:40.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.134+0000 7fcd8c110b80 1 -- start start 2023-07-18T03:24:40.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.135+0000 7fcd8c110b80 1 Processor -- start 2023-07-18T03:24:40.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.136+0000 7fcd8c110b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b5684b2d0 0x558b568564d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:40.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.136+0000 7fcd8c110b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558b5684bbe0 0x558b56853750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:40.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.136+0000 7fcd8c110b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558b56856ad0 0x558b56853dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:40.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.136+0000 7fcd8c110b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b5673dac0 con 0x558b5684bbe0 2023-07-18T03:24:40.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.136+0000 7fcd8c110b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b5673d7c0 con 0x558b56856ad0 2023-07-18T03:24:40.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.136+0000 7fcd8c110b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b56621920 con 0x558b5684b2d0 2023-07-18T03:24:40.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.136+0000 7fcd8c110b80 10 client.0 _ll_register_callbacks cb 0x558b5673b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:24:40.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.136+0000 7fcd76d9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558b56856ad0 0x558b56853dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:40.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.137+0000 7fcd76d9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558b56856ad0 0x558b56853dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:59942/0 (socket says 172.21.15.101:59942) 2023-07-18T03:24:40.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.137+0000 7fcd76d9c700 1 -- 172.21.15.101:0/134793681 learned_addr learned my addr 172.21.15.101:0/134793681 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:40.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.137+0000 7fcd76d9c700 1 -- 172.21.15.101:0/134793681 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b5684b2d0 msgr2=0x558b568564d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:40.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.137+0000 7fcd76d9c700 1 --2- 172.21.15.101:0/134793681 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b5684b2d0 0x558b568564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:40.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.137+0000 7fcd76d9c700 1 -- 172.21.15.101:0/134793681 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558b5684bbe0 msgr2=0x558b56853750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:40.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.137+0000 7fcd76d9c700 1 --2- 172.21.15.101:0/134793681 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558b5684bbe0 0x558b56853750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:40.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.137+0000 7fcd76d9c700 1 -- 172.21.15.101:0/134793681 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd68009790 con 0x558b56856ad0 2023-07-18T03:24:40.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.138+0000 7fcd76d9c700 1 --2- 172.21.15.101:0/134793681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558b56856ad0 0x558b56853dd0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fcd68013b40 tx=0x7fcd68013d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a5254df5d3b95d1 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:40.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.139+0000 7fcd59ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:24:40.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.140+0000 7fcd59ffb700 1 -- 172.21.15.101:0/134793681 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd68021070 con 0x558b56856ad0 2023-07-18T03:24:40.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.140+0000 7fcd59ffb700 1 -- 172.21.15.101:0/134793681 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fcd6800ee30 con 0x558b56856ad0 2023-07-18T03:24:40.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.140+0000 7fcd8c110b80 1 -- 172.21.15.101:0/134793681 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558b566f74f0 con 0x558b56856ad0 2023-07-18T03:24:40.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.140+0000 7fcd59ffb700 1 -- 172.21.15.101:0/134793681 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd6801c450 con 0x558b56856ad0 2023-07-18T03:24:40.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.140+0000 7fcd8c110b80 10 client.28832 fetch_fsmap learned FSMap version 42 2023-07-18T03:24:40.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.140+0000 7fcd8c110b80 1 -- 172.21.15.101:0/134793681 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x558b565b4c60 con 0x558b56856ad0 2023-07-18T03:24:40.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.140+0000 7fcd59ffb700 1 -- 172.21.15.101:0/134793681 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558b566f74f0 con 0x558b56856ad0 2023-07-18T03:24:40.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.141+0000 7fcd59ffb700 1 -- 172.21.15.101:0/134793681 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 42) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fcd68011070 con 0x558b56856ad0 2023-07-18T03:24:40.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.141+0000 7fcd8c110b80 10 client.28832 fetch_fsmap finished waiting for FSMap version 42 2023-07-18T03:24:40.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.141+0000 7fcd8c110b80 10 client.28832 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:24:40.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.141+0000 7fcd8c110b80 20 client.28832 populate_metadata read hostname 'smithi101' 2023-07-18T03:24:40.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.141+0000 7fcd8c110b80 1 --2- 172.21.15.101:0/134793681 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x558b568662d0 0x558b568686e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:40.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.141+0000 7fcd8c110b80 4 client.28832 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:24:40.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.141+0000 7fcd8c110b80 1 -- 172.21.15.101:0/134793681 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558b56796eb0 con 0x558b568662d0 2023-07-18T03:24:40.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.142+0000 7fcd75d9a700 1 --2- 172.21.15.101:0/134793681 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x558b568662d0 0x558b568686e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:40.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.143+0000 7fcd75d9a700 1 --2- 172.21.15.101:0/134793681 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x558b568662d0 0x558b568686e0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=203e146516623891 server_cookie=103cba91c506f683 in_seq=0 out_seq=0 2023-07-18T03:24:40.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.143+0000 7fcd59ffb700 10 client.28832 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:24:40.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.148+0000 7fcd59ffb700 1 -- 172.21.15.101:0/134793681 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558b56796eb0 con 0x558b568662d0 2023-07-18T03:24:40.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.148+0000 7fcd59ffb700 10 client.28832 handle_command_reply: tid=0 2023-07-18T03:24:40.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.148+0000 7fcd8c110b80 1 client.28832 shutdown 2023-07-18T03:24:40.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.149+0000 7fcd8c110b80 1 -- 172.21.15.101:0/134793681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558b56856ad0 msgr2=0x558b56853dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:40.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.149+0000 7fcd8c110b80 1 --2- 172.21.15.101:0/134793681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558b56856ad0 0x558b56853dd0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fcd68013b40 tx=0x7fcd68013d60 comp rx=0 tx=0).stop 2023-07-18T03:24:40.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.149+0000 7fcd8c110b80 1 -- 172.21.15.101:0/134793681 shutdown_connections 2023-07-18T03:24:40.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.149+0000 7fcd8c110b80 1 --2- 172.21.15.101:0/134793681 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x558b568662d0 0x558b568686e0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:40.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.149+0000 7fcd8c110b80 1 --2- 172.21.15.101:0/134793681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558b56856ad0 0x558b56853dd0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:40.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.149+0000 7fcd8c110b80 1 --2- 172.21.15.101:0/134793681 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b5684b2d0 0x558b568564d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:40.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.149+0000 7fcd8c110b80 1 --2- 172.21.15.101:0/134793681 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558b5684bbe0 0x558b56853750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:40.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.149+0000 7fcd8c110b80 1 -- 172.21.15.101:0/134793681 >> 172.21.15.101:0/134793681 conn(0x558b56841da0 msgr2=0x558b56847230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:40.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.150+0000 7fcd59ffb700 0 client.28832 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:24:40.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.150+0000 7fcd8c110b80 1 -- 172.21.15.101:0/134793681 shutdown_connections 2023-07-18T03:24:40.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.150+0000 7fcd8c110b80 1 -- 172.21.15.101:0/134793681 wait complete. 2023-07-18T03:24:40.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.150+0000 7fcd8c110b80 20 client.28832 trim_cache size 0 max 16384 2023-07-18T03:24:40.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.176+0000 7fcd8c110b80 1 Processor -- start 2023-07-18T03:24:40.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.177+0000 7fcd8c110b80 1 -- start start 2023-07-18T03:24:40.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.178+0000 7fcd8c110b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558b5684b2d0 0x558b5684e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:40.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.178+0000 7fcd8c110b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b5684bbe0 0x558b56856100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:40.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.178+0000 7fcd8c110b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558b56856ad0 0x558b568682e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:40.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.178+0000 7fcd8c110b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b56848330 con 0x558b56856ad0 2023-07-18T03:24:40.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.178+0000 7fcd8c110b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b56846490 con 0x558b5684b2d0 2023-07-18T03:24:40.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.178+0000 7fcd8c110b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b5684c540 con 0x558b5684bbe0 2023-07-18T03:24:40.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.179+0000 7fcd75d9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b5684bbe0 0x558b56856100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:40.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.179+0000 7fcd76d9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558b5684b2d0 0x558b5684e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:40.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.179+0000 7fcd75d9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b5684bbe0 0x558b56856100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:40602/0 (socket says 172.21.15.101:40602) 2023-07-18T03:24:40.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.179+0000 7fcd75d9a700 1 -- 172.21.15.101:0/3973668913 learned_addr learned my addr 172.21.15.101:0/3973668913 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:40.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.180+0000 7fcd76d9c700 1 --2- 172.21.15.101:0/3973668913 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558b56856ad0 0x558b568682e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:40.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.180+0000 7fcd75d9a700 1 -- 172.21.15.101:0/3973668913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558b5684b2d0 msgr2=0x558b5684e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:40.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.180+0000 7fcd75d9a700 1 --2- 172.21.15.101:0/3973668913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558b5684b2d0 0x558b5684e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:40.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.180+0000 7fcd75d9a700 1 -- 172.21.15.101:0/3973668913 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558b56856ad0 msgr2=0x558b568682e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:40.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.180+0000 7fcd75d9a700 1 --2- 172.21.15.101:0/3973668913 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558b56856ad0 0x558b568682e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:40.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.180+0000 7fcd75d9a700 1 -- 172.21.15.101:0/3973668913 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b565f2250 con 0x558b5684bbe0 2023-07-18T03:24:40.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.181+0000 7fcd75d9a700 1 --2- 172.21.15.101:0/3973668913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b5684bbe0 0x558b56856100 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fcd64009cd0 tx=0x7fcd64019e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77521a28011fca33 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:40.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.182+0000 7fcd6d7fa700 1 -- 172.21.15.101:0/3973668913 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6400e040 con 0x558b5684bbe0 2023-07-18T03:24:40.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.182+0000 7fcd6d7fa700 1 -- 172.21.15.101:0/3973668913 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd6401d070 con 0x558b5684bbe0 2023-07-18T03:24:40.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.182+0000 7fcd6d7fa700 1 -- 172.21.15.101:0/3973668913 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd64005020 con 0x558b5684bbe0 2023-07-18T03:24:40.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.185+0000 7fcd8c110b80 1 -- 172.21.15.101:0/3973668913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b5684bbe0 msgr2=0x558b56856100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:40.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.185+0000 7fcd8c110b80 1 --2- 172.21.15.101:0/3973668913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b5684bbe0 0x558b56856100 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fcd64009cd0 tx=0x7fcd64019e10 comp rx=0 tx=0).stop 2023-07-18T03:24:40.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.185+0000 7fcd8c110b80 1 -- 172.21.15.101:0/3973668913 shutdown_connections 2023-07-18T03:24:40.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.185+0000 7fcd8c110b80 1 --2- 172.21.15.101:0/3973668913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558b5684b2d0 0x558b5684e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:40.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.185+0000 7fcd8c110b80 1 --2- 172.21.15.101:0/3973668913 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558b56856ad0 0x558b568682e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:40.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.185+0000 7fcd8c110b80 1 --2- 172.21.15.101:0/3973668913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b5684bbe0 0x558b56856100 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:40.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.185+0000 7fcd8c110b80 1 -- 172.21.15.101:0/3973668913 >> 172.21.15.101:0/3973668913 conn(0x558b56841da0 msgr2=0x558b56862790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:40.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.185+0000 7fcd8c110b80 1 -- 172.21.15.101:0/3973668913 shutdown_connections 2023-07-18T03:24:40.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.185+0000 7fcd8c110b80 1 -- 172.21.15.101:0/3973668913 wait complete. 2023-07-18T03:24:40.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.186+0000 7fcd8c110b80 1 -- start start 2023-07-18T03:24:40.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.186+0000 7fcd8c110b80 1 Processor -- start 2023-07-18T03:24:40.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.187+0000 7fcd8c110b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558b5684b2d0 0x558b5685d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:40.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.187+0000 7fcd8c110b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558b5684bbe0 0x558b5685b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:40.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.187+0000 7fcd8c110b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b56856ad0 0x558b5685beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:40.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.187+0000 7fcd8c110b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b5685c4b0 con 0x558b5684b2d0 2023-07-18T03:24:40.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.187+0000 7fcd8c110b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b5685c620 con 0x558b5684bbe0 2023-07-18T03:24:40.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.187+0000 7fcd8c110b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b5685c7e0 con 0x558b56856ad0 2023-07-18T03:24:40.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.187+0000 7fcd8c110b80 10 client.0 _ll_register_callbacks cb 0x7fcd680099b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:24:40.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.189+0000 7fcd76d9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b56856ad0 0x558b5685beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:40.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.189+0000 7fcd76d9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558b5684b2d0 0x558b5685d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:40.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.189+0000 7fcd76d9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558b5684b2d0 0x558b5685d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:51758/0 (socket says 172.21.15.101:51758) 2023-07-18T03:24:40.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.189+0000 7fcd76d9c700 1 -- 172.21.15.101:0/3092927794 learned_addr learned my addr 172.21.15.101:0/3092927794 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:40.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.191+0000 7fcd76d9c700 1 -- 172.21.15.101:0/3092927794 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558b5684bbe0 msgr2=0x558b5685b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:40.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.191+0000 7fcd76d9c700 1 --2- 172.21.15.101:0/3092927794 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558b5684bbe0 0x558b5685b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:40.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.191+0000 7fcd76d9c700 1 -- 172.21.15.101:0/3092927794 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558b5684b2d0 msgr2=0x558b5685d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:40.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.191+0000 7fcd76d9c700 1 --2- 172.21.15.101:0/3092927794 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558b5684b2d0 0x558b5685d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:40.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.191+0000 7fcd76d9c700 1 -- 172.21.15.101:0/3092927794 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd64009430 con 0x558b56856ad0 2023-07-18T03:24:40.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.193+0000 7fcd76d9c700 1 --2- 172.21.15.101:0/3092927794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b56856ad0 0x558b5685beb0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fcd68013e40 tx=0x7fcd68015040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d778c7e3175c522 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:40.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.193+0000 7fcd59ffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:24:40.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.193+0000 7fcd59ffb700 1 -- 172.21.15.101:0/3092927794 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd68021040 con 0x558b56856ad0 2023-07-18T03:24:40.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.193+0000 7fcd59ffb700 1 -- 172.21.15.101:0/3092927794 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fcd68000e40 con 0x558b56856ad0 2023-07-18T03:24:40.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.193+0000 7fcd8c110b80 1 -- 172.21.15.101:0/3092927794 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcd68013470 con 0x558b56856ad0 2023-07-18T03:24:40.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.194+0000 7fcd59ffb700 1 -- 172.21.15.101:0/3092927794 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd68025910 con 0x558b56856ad0 2023-07-18T03:24:40.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.194+0000 7fcd8c110b80 10 client.28843 fetch_fsmap learned FSMap version 42 2023-07-18T03:24:40.190 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.194+0000 7fcd8c110b80 1 -- 172.21.15.101:0/3092927794 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x558b565ca2b0 con 0x558b56856ad0 2023-07-18T03:24:40.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.194+0000 7fcd59ffb700 1 -- 172.21.15.101:0/3092927794 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd68013470 con 0x558b56856ad0 2023-07-18T03:24:40.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.195+0000 7fcd59ffb700 1 -- 172.21.15.101:0/3092927794 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 42) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fcd68030070 con 0x558b56856ad0 2023-07-18T03:24:40.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.197+0000 7fcd8c110b80 10 client.28843 fetch_fsmap finished waiting for FSMap version 42 2023-07-18T03:24:40.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.197+0000 7fcd8c110b80 10 client.28843 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:24:40.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.197+0000 7fcd8c110b80 20 client.28843 populate_metadata read hostname 'smithi101' 2023-07-18T03:24:40.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.197+0000 7fcd8c110b80 1 --2- 172.21.15.101:0/3092927794 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x558b568bb260 0x558b568bd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:40.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.197+0000 7fcd8c110b80 4 client.28843 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:24:40.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.197+0000 7fcd8c110b80 1 -- 172.21.15.101:0/3092927794 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558b56759050 con 0x558b568bb260 2023-07-18T03:24:40.198 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.202+0000 7fcd75d9a700 1 --2- 172.21.15.101:0/3092927794 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x558b568bb260 0x558b568bd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:40.199 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.203+0000 7fcd75d9a700 1 --2- 172.21.15.101:0/3092927794 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x558b568bb260 0x558b568bd640 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb70828561563e7 server_cookie=ebd2ecd51a2d389a in_seq=0 out_seq=0 2023-07-18T03:24:40.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.203+0000 7fcd59ffb700 10 client.28843 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:24:40.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.204+0000 7fcd59ffb700 1 -- 172.21.15.101:0/3092927794 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558b56759050 con 0x558b568bb260 2023-07-18T03:24:40.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.204+0000 7fcd59ffb700 10 client.28843 handle_command_reply: tid=0 2023-07-18T03:24:40.200 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.204+0000 7fcd8c110b80 1 client.28843 shutdown 2023-07-18T03:24:40.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.204+0000 7fcd8c110b80 1 -- 172.21.15.101:0/3092927794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b56856ad0 msgr2=0x558b5685beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:40.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.204+0000 7fcd8c110b80 1 --2- 172.21.15.101:0/3092927794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b56856ad0 0x558b5685beb0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fcd68013e40 tx=0x7fcd68015040 comp rx=0 tx=0).stop 2023-07-18T03:24:40.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.205+0000 7fcd8c110b80 1 -- 172.21.15.101:0/3092927794 shutdown_connections 2023-07-18T03:24:40.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.205+0000 7fcd8c110b80 1 --2- 172.21.15.101:0/3092927794 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x558b568bb260 0x558b568bd640 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:40.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.205+0000 7fcd8c110b80 1 --2- 172.21.15.101:0/3092927794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b56856ad0 0x558b5685beb0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:40.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.205+0000 7fcd8c110b80 1 --2- 172.21.15.101:0/3092927794 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558b5684b2d0 0x558b5685d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:40.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.205+0000 7fcd8c110b80 1 --2- 172.21.15.101:0/3092927794 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558b5684bbe0 0x558b5685b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:40.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.205+0000 7fcd8c110b80 1 -- 172.21.15.101:0/3092927794 >> 172.21.15.101:0/3092927794 conn(0x558b56841da0 msgr2=0x558b56862790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:40.203 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.205+0000 7fcd59ffb700 0 client.28843 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:24:40.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.205+0000 7fcd8c110b80 1 -- 172.21.15.101:0/3092927794 shutdown_connections 2023-07-18T03:24:40.204 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.205+0000 7fcd8c110b80 1 -- 172.21.15.101:0/3092927794 wait complete. 2023-07-18T03:24:40.205 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.206+0000 7fcd8c110b80 20 client.28843 trim_cache size 0 max 16384 2023-07-18T03:24:40.205 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:24:40.205 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "no active scrubs running", 2023-07-18T03:24:40.206 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": {} 2023-07-18T03:24:40.206 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:24:40.207 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.207+0000 7fcd787ff700 1 -- 172.21.15.101:0/3020938387 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fcd600a1400 msgr2=0x7fcd600a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:40.207 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.207+0000 7fcd787ff700 1 --2- 172.21.15.101:0/3020938387 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fcd600a1400 0x7fcd600a38b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c09ab10 tx=0x7fcd5c000bc0 comp rx=0 tx=0).stop 2023-07-18T03:24:40.207 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.207+0000 7fcd787ff700 1 -- 172.21.15.101:0/3020938387 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcd7006c9a0 msgr2=0x7fcd70118990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:40.207 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.207+0000 7fcd787ff700 1 --2- 172.21.15.101:0/3020938387 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcd7006c9a0 0x7fcd70118990 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c009230 tx=0x7fcd5c00fa70 comp rx=0 tx=0).stop 2023-07-18T03:24:40.208 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.208+0000 7fcd787ff700 1 -- 172.21.15.101:0/3020938387 shutdown_connections 2023-07-18T03:24:40.208 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.208+0000 7fcd787ff700 1 --2- 172.21.15.101:0/3020938387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd7010e280 0x7fcd70113b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:40.208 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.208+0000 7fcd787ff700 1 --2- 172.21.15.101:0/3020938387 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcd7006c9a0 0x7fcd70118990 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:40.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.208+0000 7fcd787ff700 1 --2- 172.21.15.101:0/3020938387 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fcd600a1400 0x7fcd600a38b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:40.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.208+0000 7fcd787ff700 1 --2- 172.21.15.101:0/3020938387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd7006d3a0 0x7fcd70118ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:40.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.208+0000 7fcd787ff700 1 -- 172.21.15.101:0/3020938387 >> 172.21.15.101:0/3020938387 conn(0x7fcd7006bf70 msgr2=0x7fcd70070810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:40.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.209+0000 7fcd787ff700 1 -- 172.21.15.101:0/3020938387 shutdown_connections 2023-07-18T03:24:40.209 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.210+0000 7fcd787ff700 1 -- 172.21.15.101:0/3020938387 wait complete. 2023-07-18T03:24:40.241 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-18T03:24:40.242 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:24:40.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.609+0000 7f1c633ff700 1 -- 172.21.15.101:0/3361135866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c5c0ff3a0 msgr2=0x7f1c5c103870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:40.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.609+0000 7f1c633ff700 1 --2- 172.21.15.101:0/3361135866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c5c0ff3a0 0x7f1c5c103870 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f1c50009990 tx=0x7f1c50009ca0 comp rx=0 tx=0).stop 2023-07-18T03:24:40.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.609+0000 7f1c633ff700 1 -- 172.21.15.101:0/3361135866 shutdown_connections 2023-07-18T03:24:40.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.609+0000 7f1c633ff700 1 --2- 172.21.15.101:0/3361135866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c5c103e40 0x7f1c5c1042e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:40.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.609+0000 7f1c633ff700 1 --2- 172.21.15.101:0/3361135866 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1c5c0fea90 0x7f1c5c0fee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:40.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.609+0000 7f1c633ff700 1 --2- 172.21.15.101:0/3361135866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c5c0ff3a0 0x7f1c5c103870 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:40.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.609+0000 7f1c633ff700 1 -- 172.21.15.101:0/3361135866 >> 172.21.15.101:0/3361135866 conn(0x7f1c5c0fa490 msgr2=0x7f1c5c0fc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:40.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.610+0000 7f1c633ff700 1 -- 172.21.15.101:0/3361135866 shutdown_connections 2023-07-18T03:24:40.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.610+0000 7f1c633ff700 1 -- 172.21.15.101:0/3361135866 wait complete. 2023-07-18T03:24:40.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.611+0000 7f1c633ff700 1 Processor -- start 2023-07-18T03:24:40.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.611+0000 7f1c633ff700 1 -- start start 2023-07-18T03:24:40.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.612+0000 7f1c633ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1c5c0fea90 0x7f1c5c19c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:40.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.612+0000 7f1c633ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c5c0ff3a0 0x7f1c5c19c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:40.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.612+0000 7f1c633ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c5c103e40 0x7f1c5c1a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:40.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.612+0000 7f1c633ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c1a11c0 con 0x7f1c5c0fea90 2023-07-18T03:24:40.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.612+0000 7f1c633ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c1a1330 con 0x7f1c5c103e40 2023-07-18T03:24:40.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.612+0000 7f1c633ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c1a14f0 con 0x7f1c5c0ff3a0 2023-07-18T03:24:40.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.612+0000 7f1c6099a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c5c0ff3a0 0x7f1c5c19c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:40.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.613+0000 7f1c6199c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c5c103e40 0x7f1c5c1a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:40.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.613+0000 7f1c6099a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c5c0ff3a0 0x7f1c5c19c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:40636/0 (socket says 172.21.15.101:40636) 2023-07-18T03:24:40.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.613+0000 7f1c6099a700 1 -- 172.21.15.101:0/1890917068 learned_addr learned my addr 172.21.15.101:0/1890917068 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:40.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.613+0000 7f1c6119b700 1 --2- 172.21.15.101:0/1890917068 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1c5c0fea90 0x7f1c5c19c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:40.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.613+0000 7f1c6099a700 1 -- 172.21.15.101:0/1890917068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c5c103e40 msgr2=0x7f1c5c1a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:40.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.613+0000 7f1c6099a700 1 --2- 172.21.15.101:0/1890917068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c5c103e40 0x7f1c5c1a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:40.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.613+0000 7f1c6099a700 1 -- 172.21.15.101:0/1890917068 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1c5c0fea90 msgr2=0x7f1c5c19c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:40.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.613+0000 7f1c6099a700 1 --2- 172.21.15.101:0/1890917068 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1c5c0fea90 0x7f1c5c19c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:40.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.613+0000 7f1c6099a700 1 -- 172.21.15.101:0/1890917068 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c500095f0 con 0x7f1c5c0ff3a0 2023-07-18T03:24:40.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.614+0000 7f1c6099a700 1 --2- 172.21.15.101:0/1890917068 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c5c0ff3a0 0x7f1c5c19c720 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f1c50005910 tx=0x7f1c5000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:40.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.616+0000 7f1c4e7fc700 1 -- 172.21.15.101:0/1890917068 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5001d070 con 0x7f1c5c0ff3a0 2023-07-18T03:24:40.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.616+0000 7f1c4e7fc700 1 -- 172.21.15.101:0/1890917068 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c5000fbb0 con 0x7f1c5c0ff3a0 2023-07-18T03:24:40.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.616+0000 7f1c4e7fc700 1 -- 172.21.15.101:0/1890917068 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c50017650 con 0x7f1c5c0ff3a0 2023-07-18T03:24:40.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.616+0000 7f1c633ff700 1 -- 172.21.15.101:0/1890917068 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c5c1a1770 con 0x7f1c5c0ff3a0 2023-07-18T03:24:40.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.616+0000 7f1c633ff700 1 -- 172.21.15.101:0/1890917068 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c5c1a9010 con 0x7f1c5c0ff3a0 2023-07-18T03:24:40.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.618+0000 7f1c633ff700 1 -- 172.21.15.101:0/1890917068 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c5c067300 con 0x7f1c5c0ff3a0 2023-07-18T03:24:40.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.621+0000 7f1c4e7fc700 1 -- 172.21.15.101:0/1890917068 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f1c50022470 con 0x7f1c5c0ff3a0 2023-07-18T03:24:40.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.623+0000 7f1c4e7fc700 1 --2- 172.21.15.101:0/1890917068 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f1c480a14d0 0x7f1c480a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:40.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.623+0000 7f1c4e7fc700 1 -- 172.21.15.101:0/1890917068 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f1c500cdf70 con 0x7f1c5c0ff3a0 2023-07-18T03:24:40.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.623+0000 7f1c6119b700 1 --2- 172.21.15.101:0/1890917068 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f1c480a14d0 0x7f1c480a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:40.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.625+0000 7f1c6119b700 1 --2- 172.21.15.101:0/1890917068 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f1c480a14d0 0x7f1c480a3980 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1c58005ea0 tx=0x7f1c58005e30 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:40.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:40.629+0000 7f1c4e7fc700 1 -- 172.21.15.101:0/1890917068 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1c50097000 con 0x7f1c5c0ff3a0 2023-07-18T03:24:41.001 INFO:tasks.workunit.client.0.smithi101.stdout: 10 134713 28.34 MB/sec execute 125 sec latency 135.955 ms 2023-07-18T03:24:41.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.045+0000 7f1c633ff700 1 -- 172.21.15.101:0/1890917068 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1c5c1a91a0 con 0x7f1c5c0ff3a0 2023-07-18T03:24:41.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.046+0000 7f1c4e7fc700 1 -- 172.21.15.101:0/1890917068 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+10862 (secure 0 0 0) 0x7f1c5009a620 con 0x7f1c5c0ff3a0 2023-07-18T03:24:41.042 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:24:41.042 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/4192846769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":4192846769},{"type":"v1","addr":"172.21.15.101:6839","nonce":4192846769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6841/3824340175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6840","nonce":3824340175},{"type":"v1","addr":"172.21.15.101:6841","nonce":3824340175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:04:59.361185+0000","modified":"2023-07-18T03:24:11.237191+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.155:6839/2055462483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2055462483},{"type":"v1","addr":"172.21.15.155:6839","nonce":2055462483}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:24:41.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.049+0000 7f1c633ff700 1 -- 172.21.15.101:0/1890917068 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f1c480a14d0 msgr2=0x7f1c480a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:41.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.049+0000 7f1c633ff700 1 --2- 172.21.15.101:0/1890917068 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f1c480a14d0 0x7f1c480a3980 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1c58005ea0 tx=0x7f1c58005e30 comp rx=0 tx=0).stop 2023-07-18T03:24:41.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.049+0000 7f1c633ff700 1 -- 172.21.15.101:0/1890917068 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c5c0ff3a0 msgr2=0x7f1c5c19c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:41.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.049+0000 7f1c633ff700 1 --2- 172.21.15.101:0/1890917068 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c5c0ff3a0 0x7f1c5c19c720 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f1c50005910 tx=0x7f1c5000f690 comp rx=0 tx=0).stop 2023-07-18T03:24:41.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.049+0000 7f1c633ff700 1 -- 172.21.15.101:0/1890917068 shutdown_connections 2023-07-18T03:24:41.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.049+0000 7f1c633ff700 1 --2- 172.21.15.101:0/1890917068 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f1c480a14d0 0x7f1c480a3980 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.049+0000 7f1c633ff700 1 --2- 172.21.15.101:0/1890917068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c5c103e40 0x7f1c5c1a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.049+0000 7f1c633ff700 1 --2- 172.21.15.101:0/1890917068 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1c5c0fea90 0x7f1c5c19c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.049+0000 7f1c633ff700 1 --2- 172.21.15.101:0/1890917068 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c5c0ff3a0 0x7f1c5c19c720 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.049+0000 7f1c633ff700 1 -- 172.21.15.101:0/1890917068 >> 172.21.15.101:0/1890917068 conn(0x7f1c5c0fa490 msgr2=0x7f1c5c101840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:41.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.049+0000 7f1c633ff700 1 -- 172.21.15.101:0/1890917068 shutdown_connections 2023-07-18T03:24:41.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.049+0000 7f1c633ff700 1 -- 172.21.15.101:0/1890917068 wait complete. 2023-07-18T03:24:41.047 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 42 2023-07-18T03:24:41.067 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-18T03:24:41.344 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.348+0000 7fa1de5ff700 1 -- 172.21.15.101:0/531746905 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1d806d8e0 msgr2=0x7fa1d8107510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:41.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.348+0000 7fa1de5ff700 1 --2- 172.21.15.101:0/531746905 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1d806d8e0 0x7fa1d8107510 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc0098f0 tx=0x7fa1cc009c00 comp rx=0 tx=0).stop 2023-07-18T03:24:41.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.349+0000 7fa1de5ff700 1 -- 172.21.15.101:0/531746905 shutdown_connections 2023-07-18T03:24:41.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.349+0000 7fa1de5ff700 1 --2- 172.21.15.101:0/531746905 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa1d8112920 0x7fa1d8114d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.345 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.349+0000 7fa1de5ff700 1 --2- 172.21.15.101:0/531746905 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1d806d8e0 0x7fa1d8107510 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.349+0000 7fa1de5ff700 1 --2- 172.21.15.101:0/531746905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1d8107a50 0x7fa1d8107ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.349+0000 7fa1de5ff700 1 -- 172.21.15.101:0/531746905 >> 172.21.15.101:0/531746905 conn(0x7fa1d806bf70 msgr2=0x7fa1d8070ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:41.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.349+0000 7fa1de5ff700 1 -- 172.21.15.101:0/531746905 shutdown_connections 2023-07-18T03:24:41.346 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.349+0000 7fa1de5ff700 1 -- 172.21.15.101:0/531746905 wait complete. 2023-07-18T03:24:41.347 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.351+0000 7fa1de5ff700 1 Processor -- start 2023-07-18T03:24:41.349 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.352+0000 7fa1de5ff700 1 -- start start 2023-07-18T03:24:41.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.352+0000 7fa1de5ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1d806d8e0 0x7fa1d819c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:41.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.352+0000 7fa1de5ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1d8107a50 0x7fa1d819c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:41.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.352+0000 7fa1de5ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa1d8112920 0x7fa1d81a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:41.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.352+0000 7fa1de5ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d81a12b0 con 0x7fa1d8112920 2023-07-18T03:24:41.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.352+0000 7fa1de5ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d81a1420 con 0x7fa1d8107a50 2023-07-18T03:24:41.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.352+0000 7fa1de5ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d81a15e0 con 0x7fa1d806d8e0 2023-07-18T03:24:41.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.353+0000 7fa1d7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1d806d8e0 0x7fa1d819c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:41.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.353+0000 7fa1d7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1d806d8e0 0x7fa1d819c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:40658/0 (socket says 172.21.15.101:40658) 2023-07-18T03:24:41.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.353+0000 7fa1d7fff700 1 -- 172.21.15.101:0/1456614175 learned_addr learned my addr 172.21.15.101:0/1456614175 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:41.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.353+0000 7fa1dcb9c700 1 --2- 172.21.15.101:0/1456614175 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa1d8112920 0x7fa1d81a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:41.351 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.353+0000 7fa1d77fe700 1 --2- 172.21.15.101:0/1456614175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1d8107a50 0x7fa1d819c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:41.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.353+0000 7fa1d7fff700 1 -- 172.21.15.101:0/1456614175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1d8107a50 msgr2=0x7fa1d819c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:41.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.353+0000 7fa1d7fff700 1 --2- 172.21.15.101:0/1456614175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1d8107a50 0x7fa1d819c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.353+0000 7fa1d7fff700 1 -- 172.21.15.101:0/1456614175 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa1d8112920 msgr2=0x7fa1d81a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:41.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.353+0000 7fa1d7fff700 1 --2- 172.21.15.101:0/1456614175 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa1d8112920 0x7fa1d81a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.353+0000 7fa1d7fff700 1 -- 172.21.15.101:0/1456614175 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1cc009520 con 0x7fa1d806d8e0 2023-07-18T03:24:41.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.354+0000 7fa1d7fff700 1 --2- 172.21.15.101:0/1456614175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1d806d8e0 0x7fa1d819c2d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc009810 tx=0x7fa1cc00fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:41.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.354+0000 7fa1d57fa700 1 -- 172.21.15.101:0/1456614175 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc01c070 con 0x7fa1d806d8e0 2023-07-18T03:24:41.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.354+0000 7fa1d57fa700 1 -- 172.21.15.101:0/1456614175 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1cc005040 con 0x7fa1d806d8e0 2023-07-18T03:24:41.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.354+0000 7fa1d57fa700 1 -- 172.21.15.101:0/1456614175 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc004950 con 0x7fa1d806d8e0 2023-07-18T03:24:41.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.354+0000 7fa1de5ff700 1 -- 172.21.15.101:0/1456614175 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1d81a1860 con 0x7fa1d806d8e0 2023-07-18T03:24:41.353 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.355+0000 7fa1de5ff700 1 -- 172.21.15.101:0/1456614175 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1d81a1ae0 con 0x7fa1d806d8e0 2023-07-18T03:24:41.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.361+0000 7fa1f1f88b80 1 Processor -- start 2023-07-18T03:24:41.357 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.361+0000 7fa1f1f88b80 1 -- start start 2023-07-18T03:24:41.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.362+0000 7fa1f1f88b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572573de2d0 0x5572573de6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:41.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.362+0000 7fa1f1f88b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5572573debe0 0x5572573e9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:41.358 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.362+0000 7fa1f1f88b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572573e9ad0 0x5572573ebeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:41.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.362+0000 7fa1f1f88b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572572e4cb0 con 0x5572573debe0 2023-07-18T03:24:41.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.362+0000 7fa1f1f88b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557257337a40 con 0x5572573de2d0 2023-07-18T03:24:41.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.362+0000 7fa1f1f88b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x557257330ed0 con 0x5572573e9ad0 2023-07-18T03:24:41.359 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.362+0000 7fa1d57fa700 1 -- 172.21.15.101:0/1456614175 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fa1cc020070 con 0x7fa1d806d8e0 2023-07-18T03:24:41.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.362+0000 7fa1dcb9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572573de2d0 0x5572573de6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:41.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.362+0000 7fa1dcb9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572573de2d0 0x5572573de6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60018/0 (socket says 172.21.15.101:60018) 2023-07-18T03:24:41.360 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.362+0000 7fa1dcb9c700 1 -- 172.21.15.101:0/2886041781 learned_addr learned my addr 172.21.15.101:0/2886041781 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:41.361 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.363+0000 7fa1d7fff700 1 --2- 172.21.15.101:0/2886041781 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5572573debe0 0x5572573e9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:41.361 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.363+0000 7fa1dcb9c700 1 -- 172.21.15.101:0/2886041781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572573e9ad0 msgr2=0x5572573ebeb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-18T03:24:41.361 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.363+0000 7fa1dcb9c700 1 --2- 172.21.15.101:0/2886041781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572573e9ad0 0x5572573ebeb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.363+0000 7fa1dcb9c700 1 -- 172.21.15.101:0/2886041781 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5572573debe0 msgr2=0x5572573e9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:41.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.363+0000 7fa1dcb9c700 1 --2- 172.21.15.101:0/2886041781 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5572573debe0 0x5572573e9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.363+0000 7fa1dcb9c700 1 -- 172.21.15.101:0/2886041781 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55725715ede0 con 0x5572573de2d0 2023-07-18T03:24:41.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.364+0000 7fa1dcb9c700 1 --2- 172.21.15.101:0/2886041781 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572573de2d0 0x5572573de6a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8009c20 tx=0x7fa1c8009f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5a0e309d4b28663 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:41.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.364+0000 7fa1be7fc700 1 -- 172.21.15.101:0/2886041781 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8009770 con 0x5572573de2d0 2023-07-18T03:24:41.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.364+0000 7fa1be7fc700 1 -- 172.21.15.101:0/2886041781 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1c8015070 con 0x5572573de2d0 2023-07-18T03:24:41.362 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.364+0000 7fa1be7fc700 1 -- 172.21.15.101:0/2886041781 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8011680 con 0x5572573de2d0 2023-07-18T03:24:41.363 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.365+0000 7fa1d57fa700 1 --2- 172.21.15.101:0/1456614175 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa1c00a5810 0x7fa1c00a7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:41.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.365+0000 7fa1d57fa700 1 -- 172.21.15.101:0/1456614175 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fa1cc0cdc60 con 0x7fa1d806d8e0 2023-07-18T03:24:41.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.365+0000 7fa1d7fff700 1 --2- 172.21.15.101:0/1456614175 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa1c00a5810 0x7fa1c00a7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:41.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.365+0000 7fa1f1f88b80 1 -- 172.21.15.101:0/2886041781 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572573de2d0 msgr2=0x5572573de6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:41.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.365+0000 7fa1f1f88b80 1 --2- 172.21.15.101:0/2886041781 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572573de2d0 0x5572573de6a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8009c20 tx=0x7fa1c8009f30 comp rx=0 tx=0).stop 2023-07-18T03:24:41.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.366+0000 7fa1f1f88b80 1 -- 172.21.15.101:0/2886041781 shutdown_connections 2023-07-18T03:24:41.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.366+0000 7fa1f1f88b80 1 --2- 172.21.15.101:0/2886041781 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572573de2d0 0x5572573de6a0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.365 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.366+0000 7fa1f1f88b80 1 --2- 172.21.15.101:0/2886041781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572573e9ad0 0x5572573ebeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.365 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.366+0000 7fa1f1f88b80 1 --2- 172.21.15.101:0/2886041781 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5572573debe0 0x5572573e9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.365 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.366+0000 7fa1f1f88b80 1 -- 172.21.15.101:0/2886041781 >> 172.21.15.101:0/2886041781 conn(0x5572573d4da0 msgr2=0x5572573da6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:41.365 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.366+0000 7fa1f1f88b80 1 -- 172.21.15.101:0/2886041781 shutdown_connections 2023-07-18T03:24:41.365 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.366+0000 7fa1f1f88b80 1 -- 172.21.15.101:0/2886041781 wait complete. 2023-07-18T03:24:41.365 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.366+0000 7fa1d7fff700 1 --2- 172.21.15.101:0/1456614175 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa1c00a5810 0x7fa1c00a7cc0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc09a940 tx=0x7fa1cc009c00 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:41.365 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.367+0000 7fa1f1f88b80 1 -- start start 2023-07-18T03:24:41.365 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.367+0000 7fa1f1f88b80 1 Processor -- start 2023-07-18T03:24:41.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.368+0000 7fa1f1f88b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5572573de2d0 0x5572573e94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:41.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.368+0000 7fa1f1f88b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572573debe0 0x5572573e6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:41.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.368+0000 7fa1f1f88b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572573e9ad0 0x5572573e6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:41.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.368+0000 7fa1f1f88b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572572d0ac0 con 0x5572573de2d0 2023-07-18T03:24:41.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.368+0000 7fa1f1f88b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572572d07c0 con 0x5572573debe0 2023-07-18T03:24:41.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.368+0000 7fa1f1f88b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572571b4920 con 0x5572573e9ad0 2023-07-18T03:24:41.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.368+0000 7fa1f1f88b80 10 client.0 _ll_register_callbacks cb 0x5572572ce760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:24:41.368 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.368+0000 7fa1dcb9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5572573de2d0 0x5572573e94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:41.368 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.368+0000 7fa1dcb9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5572573de2d0 0x5572573e94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:51800/0 (socket says 172.21.15.101:51800) 2023-07-18T03:24:41.368 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.368+0000 7fa1dcb9c700 1 -- 172.21.15.101:0/1002343774 learned_addr learned my addr 172.21.15.101:0/1002343774 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:41.368 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.368+0000 7fa1dcb9c700 1 --2- 172.21.15.101:0/1002343774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572573e9ad0 0x5572573e6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:41.368 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.369+0000 7fa1d77fe700 1 --2- 172.21.15.101:0/1002343774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572573debe0 0x5572573e6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:41.368 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.369+0000 7fa1dcb9c700 1 -- 172.21.15.101:0/1002343774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572573debe0 msgr2=0x5572573e6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:41.369 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.369+0000 7fa1dcb9c700 1 --2- 172.21.15.101:0/1002343774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572573debe0 0x5572573e6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.369 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.369+0000 7fa1dcb9c700 1 -- 172.21.15.101:0/1002343774 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5572573de2d0 msgr2=0x5572573e94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:41.369 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.369+0000 7fa1dcb9c700 1 --2- 172.21.15.101:0/1002343774 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5572573de2d0 0x5572573e94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.369 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.370+0000 7fa1dcb9c700 1 -- 172.21.15.101:0/1002343774 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1c80098d0 con 0x5572573e9ad0 2023-07-18T03:24:41.369 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.370+0000 7fa1d77fe700 1 --2- 172.21.15.101:0/1002343774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572573debe0 0x5572573e6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:24:41.369 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.370+0000 7fa1dcb9c700 1 --2- 172.21.15.101:0/1002343774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572573e9ad0 0x5572573e6da0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8009bf0 tx=0x7fa1c801ccd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1532c7691cb8b6bf server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:41.370 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.370+0000 7fa1b77fe700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:24:41.370 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.370+0000 7fa1b77fe700 1 -- 172.21.15.101:0/1002343774 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8005040 con 0x5572573e9ad0 2023-07-18T03:24:41.370 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.370+0000 7fa1b77fe700 1 -- 172.21.15.101:0/1002343774 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fa1c8017070 con 0x5572573e9ad0 2023-07-18T03:24:41.370 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.371+0000 7fa1f1f88b80 1 -- 172.21.15.101:0/1002343774 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55725728a4f0 con 0x5572573e9ad0 2023-07-18T03:24:41.370 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.371+0000 7fa1b77fe700 1 -- 172.21.15.101:0/1002343774 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1c80051a0 con 0x5572573e9ad0 2023-07-18T03:24:41.370 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.371+0000 7fa1f1f88b80 10 client.28858 fetch_fsmap learned FSMap version 42 2023-07-18T03:24:41.371 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.371+0000 7fa1f1f88b80 1 -- 172.21.15.101:0/1002343774 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x557257147c60 con 0x5572573e9ad0 2023-07-18T03:24:41.371 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.371+0000 7fa1b77fe700 1 -- 172.21.15.101:0/1002343774 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55725728a4f0 con 0x5572573e9ad0 2023-07-18T03:24:41.371 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.371+0000 7fa1b77fe700 1 -- 172.21.15.101:0/1002343774 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 42) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fa1c8029070 con 0x5572573e9ad0 2023-07-18T03:24:41.371 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.372+0000 7fa1f1f88b80 10 client.28858 fetch_fsmap finished waiting for FSMap version 42 2023-07-18T03:24:41.371 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.372+0000 7fa1f1f88b80 10 client.28858 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:24:41.371 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.372+0000 7fa1f1f88b80 20 client.28858 populate_metadata read hostname 'smithi101' 2023-07-18T03:24:41.371 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.372+0000 7fa1f1f88b80 1 --2- 172.21.15.101:0/1002343774 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x557257401ab0 0x557257403ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:41.372 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.372+0000 7fa1f1f88b80 4 client.28858 mds_command: new command op to 24541 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:24:41.373 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.372+0000 7fa1f1f88b80 1 -- 172.21.15.101:0/1002343774 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557257329eb0 con 0x557257401ab0 2023-07-18T03:24:41.373 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.373+0000 7fa1d77fe700 1 --2- 172.21.15.101:0/1002343774 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x557257401ab0 0x557257403ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:41.374 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.373+0000 7fa1b77fe700 10 client.28858 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:24:41.374 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.373+0000 7fa1d77fe700 1 --2- 172.21.15.101:0/1002343774 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x557257401ab0 0x557257403ec0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f2d3efb624f137d3 server_cookie=9a60233851d281ba in_seq=0 out_seq=0 2023-07-18T03:24:41.374 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.377+0000 7fa1b77fe700 1 -- 172.21.15.101:0/1002343774 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557257329eb0 con 0x557257401ab0 2023-07-18T03:24:41.374 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.377+0000 7fa1b77fe700 10 client.28858 handle_command_reply: tid=0 2023-07-18T03:24:41.374 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.377+0000 7fa1f1f88b80 1 client.28858 shutdown 2023-07-18T03:24:41.375 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.378+0000 7fa1f1f88b80 1 -- 172.21.15.101:0/1002343774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572573e9ad0 msgr2=0x5572573e6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:41.375 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.378+0000 7fa1f1f88b80 1 --2- 172.21.15.101:0/1002343774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572573e9ad0 0x5572573e6da0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8009bf0 tx=0x7fa1c801ccd0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.375 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.378+0000 7fa1f1f88b80 1 -- 172.21.15.101:0/1002343774 shutdown_connections 2023-07-18T03:24:41.375 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.378+0000 7fa1f1f88b80 1 --2- 172.21.15.101:0/1002343774 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x557257401ab0 0x557257403ec0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.375 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.378+0000 7fa1f1f88b80 1 --2- 172.21.15.101:0/1002343774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572573e9ad0 0x5572573e6da0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.375 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.378+0000 7fa1f1f88b80 1 --2- 172.21.15.101:0/1002343774 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5572573de2d0 0x5572573e94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.376 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.378+0000 7fa1f1f88b80 1 --2- 172.21.15.101:0/1002343774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572573debe0 0x5572573e6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.376 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.378+0000 7fa1f1f88b80 1 -- 172.21.15.101:0/1002343774 >> 172.21.15.101:0/1002343774 conn(0x5572573d4da0 msgr2=0x5572573da230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:41.376 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.378+0000 7fa1b77fe700 0 client.28858 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:24:41.376 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.378+0000 7fa1f1f88b80 1 -- 172.21.15.101:0/1002343774 shutdown_connections 2023-07-18T03:24:41.376 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.379+0000 7fa1f1f88b80 1 -- 172.21.15.101:0/1002343774 wait complete. 2023-07-18T03:24:41.377 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.380+0000 7fa1f1f88b80 20 client.28858 trim_cache size 0 max 16384 2023-07-18T03:24:41.394 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.398+0000 7fa1f1f88b80 1 Processor -- start 2023-07-18T03:24:41.395 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.399+0000 7fa1f1f88b80 1 -- start start 2023-07-18T03:24:41.395 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.399+0000 7fa1f1f88b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572573de2d0 0x5572573f04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:41.395 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.399+0000 7fa1f1f88b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5572573debe0 0x557257403ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:41.395 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.399+0000 7fa1f1f88b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572573e9ad0 0x5572573e9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:41.396 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.399+0000 7fa1f1f88b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572573d6ec0 con 0x5572573debe0 2023-07-18T03:24:41.396 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.399+0000 7fa1f1f88b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572573dfe30 con 0x5572573e9ad0 2023-07-18T03:24:41.396 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.399+0000 7fa1f1f88b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572573df540 con 0x5572573de2d0 2023-07-18T03:24:41.396 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.399+0000 7fa1dcb9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572573de2d0 0x5572573f04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:41.396 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.399+0000 7fa1dcb9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572573e9ad0 0x5572573e9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:41.396 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.399+0000 7fa1dcb9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572573e9ad0 0x5572573e9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60038/0 (socket says 172.21.15.101:60038) 2023-07-18T03:24:41.396 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.399+0000 7fa1dcb9c700 1 -- 172.21.15.101:0/1701333034 learned_addr learned my addr 172.21.15.101:0/1701333034 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:41.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.400+0000 7fa1dcb9c700 1 -- 172.21.15.101:0/1701333034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572573de2d0 msgr2=0x5572573f04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:41.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.400+0000 7fa1dcb9c700 1 --2- 172.21.15.101:0/1701333034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572573de2d0 0x5572573f04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.400+0000 7fa1dcb9c700 1 -- 172.21.15.101:0/1701333034 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5572573debe0 msgr2=0x557257403ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:41.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.400+0000 7fa1dcb9c700 1 --2- 172.21.15.101:0/1701333034 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5572573debe0 0x557257403ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.400+0000 7fa1dcb9c700 1 -- 172.21.15.101:0/1701333034 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557257185250 con 0x5572573e9ad0 2023-07-18T03:24:41.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.400+0000 7fa1dcb9c700 1 --2- 172.21.15.101:0/1701333034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572573e9ad0 0x5572573e9020 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8009bc0 tx=0x7fa1c8005d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e771734a74ce3293 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:41.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.400+0000 7fa1beffd700 1 -- 172.21.15.101:0/1701333034 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8010760 con 0x5572573e9ad0 2023-07-18T03:24:41.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.400+0000 7fa1beffd700 1 -- 172.21.15.101:0/1701333034 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1c8035540 con 0x5572573e9ad0 2023-07-18T03:24:41.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.400+0000 7fa1beffd700 1 -- 172.21.15.101:0/1701333034 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8005580 con 0x5572573e9ad0 2023-07-18T03:24:41.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.401+0000 7fa1f1f88b80 1 -- 172.21.15.101:0/1701333034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572573e9ad0 msgr2=0x5572573e9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:41.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.401+0000 7fa1f1f88b80 1 --2- 172.21.15.101:0/1701333034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572573e9ad0 0x5572573e9020 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8009bc0 tx=0x7fa1c8005d60 comp rx=0 tx=0).stop 2023-07-18T03:24:41.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.401+0000 7fa1f1f88b80 1 -- 172.21.15.101:0/1701333034 shutdown_connections 2023-07-18T03:24:41.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.401+0000 7fa1f1f88b80 1 --2- 172.21.15.101:0/1701333034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572573e9ad0 0x5572573e9020 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.401+0000 7fa1f1f88b80 1 --2- 172.21.15.101:0/1701333034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572573de2d0 0x5572573f04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.401+0000 7fa1f1f88b80 1 --2- 172.21.15.101:0/1701333034 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5572573debe0 0x557257403ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.401+0000 7fa1f1f88b80 1 -- 172.21.15.101:0/1701333034 >> 172.21.15.101:0/1701333034 conn(0x5572573d4da0 msgr2=0x5572573f4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:41.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.401+0000 7fa1f1f88b80 1 -- 172.21.15.101:0/1701333034 shutdown_connections 2023-07-18T03:24:41.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.401+0000 7fa1f1f88b80 1 -- 172.21.15.101:0/1701333034 wait complete. 2023-07-18T03:24:41.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.402+0000 7fa1f1f88b80 1 -- start start 2023-07-18T03:24:41.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.402+0000 7fa1f1f88b80 1 Processor -- start 2023-07-18T03:24:41.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.403+0000 7fa1f1f88b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572573de2d0 0x557257450910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:41.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.403+0000 7fa1f1f88b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572573debe0 0x557257450e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:41.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.403+0000 7fa1f1f88b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5572573e9ad0 0x557257451390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:41.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.403+0000 7fa1f1f88b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572573edc90 con 0x5572573e9ad0 2023-07-18T03:24:41.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.403+0000 7fa1f1f88b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572573eddd0 con 0x5572573de2d0 2023-07-18T03:24:41.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.403+0000 7fa1f1f88b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572573edf90 con 0x5572573debe0 2023-07-18T03:24:41.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.403+0000 7fa1f1f88b80 10 client.0 _ll_register_callbacks cb 0x7fa1c801c790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:24:41.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.403+0000 7fa1dcb9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572573de2d0 0x557257450910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:41.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.403+0000 7fa1d77fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572573debe0 0x557257450e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:41.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.403+0000 7fa1dcb9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572573de2d0 0x557257450910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60042/0 (socket says 172.21.15.101:60042) 2023-07-18T03:24:41.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.403+0000 7fa1dcb9c700 1 -- 172.21.15.101:0/1585162727 learned_addr learned my addr 172.21.15.101:0/1585162727 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:41.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.403+0000 7fa1dcb9c700 1 -- 172.21.15.101:0/1585162727 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572573debe0 msgr2=0x557257450e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:41.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.403+0000 7fa1dcb9c700 1 --2- 172.21.15.101:0/1585162727 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572573debe0 0x557257450e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.403+0000 7fa1dcb9c700 1 -- 172.21.15.101:0/1585162727 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5572573e9ad0 msgr2=0x557257451390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:41.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.403+0000 7fa1dcb9c700 1 --2- 172.21.15.101:0/1585162727 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5572573e9ad0 0x557257451390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.403+0000 7fa1dcb9c700 1 -- 172.21.15.101:0/1585162727 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557257147c60 con 0x5572573de2d0 2023-07-18T03:24:41.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.404+0000 7fa1dcb9c700 1 --2- 172.21.15.101:0/1585162727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572573de2d0 0x557257450910 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8009460 tx=0x7fa1c8000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3616c9b74c060c3 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:41.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.404+0000 7fa1b77fe700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:24:41.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.404+0000 7fa1b77fe700 1 -- 172.21.15.101:0/1585162727 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8004ad0 con 0x5572573de2d0 2023-07-18T03:24:41.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.404+0000 7fa1f1f88b80 1 -- 172.21.15.101:0/1585162727 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa1c8015070 con 0x5572573de2d0 2023-07-18T03:24:41.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.404+0000 7fa1b77fe700 1 -- 172.21.15.101:0/1585162727 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fa1c8017040 con 0x5572573de2d0 2023-07-18T03:24:41.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.404+0000 7fa1b77fe700 1 -- 172.21.15.101:0/1585162727 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1c8004c30 con 0x5572573de2d0 2023-07-18T03:24:41.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.404+0000 7fa1f1f88b80 10 client.28853 fetch_fsmap learned FSMap version 42 2023-07-18T03:24:41.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.404+0000 7fa1f1f88b80 1 -- 172.21.15.101:0/1585162727 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55725715d2b0 con 0x5572573de2d0 2023-07-18T03:24:41.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.405+0000 7fa1b77fe700 1 -- 172.21.15.101:0/1585162727 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1c8015070 con 0x5572573de2d0 2023-07-18T03:24:41.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.405+0000 7fa1b77fe700 1 -- 172.21.15.101:0/1585162727 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 42) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fa1c8029070 con 0x5572573de2d0 2023-07-18T03:24:41.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.405+0000 7fa1f1f88b80 10 client.28853 fetch_fsmap finished waiting for FSMap version 42 2023-07-18T03:24:41.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.405+0000 7fa1f1f88b80 10 client.28853 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-18T03:24:41.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.405+0000 7fa1f1f88b80 20 client.28853 populate_metadata read hostname 'smithi101' 2023-07-18T03:24:41.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.405+0000 7fa1f1f88b80 1 --2- 172.21.15.101:0/1585162727 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5572573f9c20 0x5572573fc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:41.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.405+0000 7fa1f1f88b80 4 client.28853 mds_command: new command op to 24541 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:24:41.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.405+0000 7fa1f1f88b80 1 -- 172.21.15.101:0/1585162727 --> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5572572ec050 con 0x5572573f9c20 2023-07-18T03:24:41.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.405+0000 7fa1d77fe700 1 --2- 172.21.15.101:0/1585162727 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5572573f9c20 0x5572573fc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:41.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.406+0000 7fa1d77fe700 1 --2- 172.21.15.101:0/1585162727 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5572573f9c20 0x5572573fc030 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=54952e88c9f37c8 server_cookie=97b3ce2e5c7dcc18 in_seq=0 out_seq=0 2023-07-18T03:24:41.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.406+0000 7fa1b77fe700 10 client.28853 ms_handle_connect on v2:172.21.15.179:6836/3458593442 2023-07-18T03:24:41.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.406+0000 7fa1b77fe700 1 -- 172.21.15.101:0/1585162727 <== mds.2 v2:172.21.15.179:6836/3458593442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5572572ec050 con 0x5572573f9c20 2023-07-18T03:24:41.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.406+0000 7fa1b77fe700 10 client.28853 handle_command_reply: tid=0 2023-07-18T03:24:41.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.406+0000 7fa1f1f88b80 1 client.28853 shutdown 2023-07-18T03:24:41.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.406+0000 7fa1f1f88b80 1 -- 172.21.15.101:0/1585162727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572573de2d0 msgr2=0x557257450910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:41.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.406+0000 7fa1f1f88b80 1 --2- 172.21.15.101:0/1585162727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572573de2d0 0x557257450910 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8009460 tx=0x7fa1c8000bc0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.407+0000 7fa1f1f88b80 1 -- 172.21.15.101:0/1585162727 shutdown_connections 2023-07-18T03:24:41.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.407+0000 7fa1f1f88b80 1 --2- 172.21.15.101:0/1585162727 >> [v2:172.21.15.179:6836/3458593442,v1:172.21.15.179:6837/3458593442] conn(0x5572573f9c20 0x5572573fc030 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.407+0000 7fa1f1f88b80 1 --2- 172.21.15.101:0/1585162727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572573de2d0 0x557257450910 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.407+0000 7fa1f1f88b80 1 --2- 172.21.15.101:0/1585162727 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5572573e9ad0 0x557257451390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.407+0000 7fa1f1f88b80 1 --2- 172.21.15.101:0/1585162727 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572573debe0 0x557257450e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.407+0000 7fa1f1f88b80 1 -- 172.21.15.101:0/1585162727 >> 172.21.15.101:0/1585162727 conn(0x5572573d4da0 msgr2=0x557257403ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:41.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.407+0000 7fa1b77fe700 0 client.28853 ms_handle_reset on v2:172.21.15.179:6836/3458593442 2023-07-18T03:24:41.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.407+0000 7fa1f1f88b80 1 -- 172.21.15.101:0/1585162727 shutdown_connections 2023-07-18T03:24:41.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.407+0000 7fa1f1f88b80 1 -- 172.21.15.101:0/1585162727 wait complete. 2023-07-18T03:24:41.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.407+0000 7fa1f1f88b80 20 client.28853 trim_cache size 0 max 16384 2023-07-18T03:24:41.411 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:24:41.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.408+0000 7fa1de5ff700 1 -- 172.21.15.101:0/1456614175 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa1c00a5810 msgr2=0x7fa1c00a7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:41.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.408+0000 7fa1de5ff700 1 --2- 172.21.15.101:0/1456614175 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa1c00a5810 0x7fa1c00a7cc0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc09a940 tx=0x7fa1cc009c00 comp rx=0 tx=0).stop 2023-07-18T03:24:41.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.408+0000 7fa1de5ff700 1 -- 172.21.15.101:0/1456614175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1d806d8e0 msgr2=0x7fa1d819c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:41.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.408+0000 7fa1de5ff700 1 --2- 172.21.15.101:0/1456614175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1d806d8e0 0x7fa1d819c2d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc009810 tx=0x7fa1cc00fa00 comp rx=0 tx=0).stop 2023-07-18T03:24:41.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.408+0000 7fa1de5ff700 1 -- 172.21.15.101:0/1456614175 shutdown_connections 2023-07-18T03:24:41.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.408+0000 7fa1de5ff700 1 --2- 172.21.15.101:0/1456614175 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa1d8112920 0x7fa1d81a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.408+0000 7fa1de5ff700 1 --2- 172.21.15.101:0/1456614175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1d806d8e0 0x7fa1d819c2d0 secure :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc009810 tx=0x7fa1cc00fa00 comp rx=0 tx=0).stop 2023-07-18T03:24:41.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.408+0000 7fa1de5ff700 1 --2- 172.21.15.101:0/1456614175 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa1c00a5810 0x7fa1c00a7cc0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.408+0000 7fa1de5ff700 1 --2- 172.21.15.101:0/1456614175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1d8107a50 0x7fa1d819c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.408+0000 7fa1de5ff700 1 -- 172.21.15.101:0/1456614175 >> 172.21.15.101:0/1456614175 conn(0x7fa1d806bf70 msgr2=0x7fa1d810a830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:41.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.408+0000 7fa1de5ff700 1 -- 172.21.15.101:0/1456614175 shutdown_connections 2023-07-18T03:24:41.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.408+0000 7fa1de5ff700 1 -- 172.21.15.101:0/1456614175 wait complete. 2023-07-18T03:24:41.426 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:41 smithi101 ceph-mon[126379]: pgmap v538: 129 pgs: 129 active+clean; 1.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 17 MiB/s wr, 623 op/s 2023-07-18T03:24:41.427 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:41 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1890917068' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:24:41.429 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-18T03:24:41.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:41 smithi179 ceph-mon[129229]: pgmap v538: 129 pgs: 129 active+clean; 1.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 17 MiB/s wr, 623 op/s 2023-07-18T03:24:41.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:41 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1890917068' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:24:41.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.719+0000 7f9bec5ff700 1 -- 172.21.15.101:0/1309179451 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9be40731c0 msgr2=0x7f9be41105a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:41.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.719+0000 7f9bec5ff700 1 --2- 172.21.15.101:0/1309179451 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9be40731c0 0x7f9be41105a0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f9bd400b0a0 tx=0x7f9bd400b3b0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.720+0000 7f9bec5ff700 1 -- 172.21.15.101:0/1309179451 shutdown_connections 2023-07-18T03:24:41.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.720+0000 7f9bec5ff700 1 --2- 172.21.15.101:0/1309179451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9be4110ae0 0x7f9be4112ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.720+0000 7f9bec5ff700 1 --2- 172.21.15.101:0/1309179451 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9be40728b0 0x7f9be4072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.720+0000 7f9bec5ff700 1 --2- 172.21.15.101:0/1309179451 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9be40731c0 0x7f9be41105a0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.720+0000 7f9bec5ff700 1 -- 172.21.15.101:0/1309179451 >> 172.21.15.101:0/1309179451 conn(0x7f9be40fb5f0 msgr2=0x7f9be40fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:41.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.720+0000 7f9bec5ff700 1 -- 172.21.15.101:0/1309179451 shutdown_connections 2023-07-18T03:24:41.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.720+0000 7f9bec5ff700 1 -- 172.21.15.101:0/1309179451 wait complete. 2023-07-18T03:24:41.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.721+0000 7f9bec5ff700 1 Processor -- start 2023-07-18T03:24:41.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.721+0000 7f9bec5ff700 1 -- start start 2023-07-18T03:24:41.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.722+0000 7f9bec5ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9be40728b0 0x7f9be419c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:41.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.722+0000 7f9bec5ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9be40731c0 0x7f9be419c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:41.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.722+0000 7f9bec5ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9be4110ae0 0x7f9be41a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:41.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.722+0000 7f9bec5ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be41a1270 con 0x7f9be4110ae0 2023-07-18T03:24:41.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.722+0000 7f9bec5ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be41a13e0 con 0x7f9be40728b0 2023-07-18T03:24:41.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.722+0000 7f9bec5ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be41a15a0 con 0x7f9be40731c0 2023-07-18T03:24:41.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.722+0000 7f9beab9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9be4110ae0 0x7f9be41a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:41.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.722+0000 7f9beab9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9be4110ae0 0x7f9be41a0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:51844/0 (socket says 172.21.15.101:51844) 2023-07-18T03:24:41.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.722+0000 7f9beab9c700 1 -- 172.21.15.101:0/1229033189 learned_addr learned my addr 172.21.15.101:0/1229033189 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:41.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.722+0000 7f9be9b9a700 1 --2- 172.21.15.101:0/1229033189 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9be40731c0 0x7f9be419c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:41.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.723+0000 7f9beab9c700 1 -- 172.21.15.101:0/1229033189 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9be40731c0 msgr2=0x7f9be419c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:41.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.723+0000 7f9beab9c700 1 --2- 172.21.15.101:0/1229033189 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9be40731c0 0x7f9be419c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.723+0000 7f9beab9c700 1 -- 172.21.15.101:0/1229033189 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9be40728b0 msgr2=0x7f9be419c290 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-18T03:24:41.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.723+0000 7f9beab9c700 1 --2- 172.21.15.101:0/1229033189 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9be40728b0 0x7f9be419c290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.723+0000 7f9beab9c700 1 -- 172.21.15.101:0/1229033189 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd4009d00 con 0x7f9be4110ae0 2023-07-18T03:24:41.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.723+0000 7f9beab9c700 1 --2- 172.21.15.101:0/1229033189 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9be4110ae0 0x7f9be41a0c30 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc00d950 tx=0x7f9bdc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:41.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.724+0000 7f9be37fe700 1 -- 172.21.15.101:0/1229033189 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc004a30 con 0x7f9be4110ae0 2023-07-18T03:24:41.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.724+0000 7f9be37fe700 1 -- 172.21.15.101:0/1229033189 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9bdc00b5c0 con 0x7f9be4110ae0 2023-07-18T03:24:41.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.724+0000 7f9bec5ff700 1 -- 172.21.15.101:0/1229033189 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9be41a1880 con 0x7f9be4110ae0 2023-07-18T03:24:41.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.724+0000 7f9be37fe700 1 -- 172.21.15.101:0/1229033189 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc009af0 con 0x7f9be4110ae0 2023-07-18T03:24:41.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.724+0000 7f9bec5ff700 1 -- 172.21.15.101:0/1229033189 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9be4103f50 con 0x7f9be4110ae0 2023-07-18T03:24:41.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.733+0000 7f9bffe4eb80 1 Processor -- start 2023-07-18T03:24:41.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.738+0000 7f9be37fe700 1 -- 172.21.15.101:0/1229033189 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f9bdc004b90 con 0x7f9be4110ae0 2023-07-18T03:24:41.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.739+0000 7f9bffe4eb80 1 -- start start 2023-07-18T03:24:41.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.739+0000 7f9bffe4eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614258bf2d0 0x5614258bf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:41.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.739+0000 7f9bffe4eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614258bfbe0 0x5614258ca590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:41.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.739+0000 7f9bffe4eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614258caad0 0x5614258cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:41.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.739+0000 7f9bffe4eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614257c5cb0 con 0x5614258bf2d0 2023-07-18T03:24:41.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.739+0000 7f9bffe4eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561425818a40 con 0x5614258caad0 2023-07-18T03:24:41.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.739+0000 7f9bffe4eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x561425811ed0 con 0x5614258bfbe0 2023-07-18T03:24:41.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.740+0000 7f9be9b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614258bf2d0 0x5614258bf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:41.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.740+0000 7f9be9b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614258bf2d0 0x5614258bf6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:51856/0 (socket says 172.21.15.101:51856) 2023-07-18T03:24:41.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.740+0000 7f9be9b9a700 1 -- 172.21.15.101:0/1116028531 learned_addr learned my addr 172.21.15.101:0/1116028531 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:41.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.740+0000 7f9be9b9a700 1 --2- 172.21.15.101:0/1116028531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614258caad0 0x5614258cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:41.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.741+0000 7f9be9b9a700 1 -- 172.21.15.101:0/1116028531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614258bfbe0 msgr2=0x5614258ca590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-18T03:24:41.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.741+0000 7f9be9b9a700 1 --2- 172.21.15.101:0/1116028531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614258bfbe0 0x5614258ca590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.741+0000 7f9be9b9a700 1 -- 172.21.15.101:0/1116028531 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614258bf2d0 msgr2=0x5614258bf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:41.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.741+0000 7f9be9b9a700 1 --2- 172.21.15.101:0/1116028531 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614258bf2d0 0x5614258bf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.741+0000 7f9be9b9a700 1 -- 172.21.15.101:0/1116028531 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56142563fde0 con 0x5614258caad0 2023-07-18T03:24:41.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.741+0000 7f9be37fe700 1 --2- 172.21.15.101:0/1229033189 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9bcc0a1400 0x7f9bcc0a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:41.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.742+0000 7f9be37fe700 1 -- 172.21.15.101:0/1229033189 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f9bdc0ccb00 con 0x7f9be4110ae0 2023-07-18T03:24:41.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.742+0000 7f9be9b9a700 1 --2- 172.21.15.101:0/1116028531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614258caad0 0x5614258cceb0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f9bd400b0a0 tx=0x7f9bd40099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71cd863d111c035 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:41.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.743+0000 7f9be0bfe700 1 -- 172.21.15.101:0/1116028531 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd400e070 con 0x5614258caad0 2023-07-18T03:24:41.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.743+0000 7f9be0bfe700 1 -- 172.21.15.101:0/1116028531 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9bd40079a0 con 0x5614258caad0 2023-07-18T03:24:41.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.743+0000 7f9be0bfe700 1 -- 172.21.15.101:0/1116028531 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd4005750 con 0x5614258caad0 2023-07-18T03:24:41.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.744+0000 7f9bffe4eb80 1 -- 172.21.15.101:0/1116028531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614258caad0 msgr2=0x5614258cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:41.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.744+0000 7f9bffe4eb80 1 --2- 172.21.15.101:0/1116028531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614258caad0 0x5614258cceb0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f9bd400b0a0 tx=0x7f9bd40099e0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.744+0000 7f9bffe4eb80 1 -- 172.21.15.101:0/1116028531 shutdown_connections 2023-07-18T03:24:41.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.744+0000 7f9bffe4eb80 1 --2- 172.21.15.101:0/1116028531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614258caad0 0x5614258cceb0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.744+0000 7f9bffe4eb80 1 --2- 172.21.15.101:0/1116028531 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614258bf2d0 0x5614258bf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.744+0000 7f9bffe4eb80 1 --2- 172.21.15.101:0/1116028531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614258bfbe0 0x5614258ca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.744+0000 7f9bffe4eb80 1 -- 172.21.15.101:0/1116028531 >> 172.21.15.101:0/1116028531 conn(0x5614258b5da0 msgr2=0x5614258bb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:41.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.748+0000 7f9bea39b700 1 --2- 172.21.15.101:0/1229033189 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9bcc0a1400 0x7f9bcc0a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:41.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.748+0000 7f9bffe4eb80 1 -- 172.21.15.101:0/1116028531 shutdown_connections 2023-07-18T03:24:41.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.749+0000 7f9bffe4eb80 1 -- 172.21.15.101:0/1116028531 wait complete. 2023-07-18T03:24:41.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.749+0000 7f9bea39b700 1 --2- 172.21.15.101:0/1229033189 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9bcc0a1400 0x7f9bcc0a38b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x5614258c0370 tx=0x7f9bd8006c00 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:41.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.750+0000 7f9bffe4eb80 1 -- start start 2023-07-18T03:24:41.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.750+0000 7f9bffe4eb80 1 Processor -- start 2023-07-18T03:24:41.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.751+0000 7f9bffe4eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614258bf2d0 0x5614258c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:41.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.751+0000 7f9bffe4eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614258caad0 0x5614258c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:41.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.751+0000 7f9bffe4eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614258c8410 0x5614258d64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:41.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.751+0000 7f9bffe4eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614257b1ac0 con 0x5614258caad0 2023-07-18T03:24:41.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.751+0000 7f9bffe4eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614257b17c0 con 0x5614258bf2d0 2023-07-18T03:24:41.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.751+0000 7f9bffe4eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x561425695920 con 0x5614258c8410 2023-07-18T03:24:41.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.752+0000 7f9bffe4eb80 10 client.0 _ll_register_callbacks cb 0x5614257af760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:24:41.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.752+0000 7f9be9b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614258caad0 0x5614258c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:41.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.753+0000 7f9be9b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614258caad0 0x5614258c7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:51870/0 (socket says 172.21.15.101:51870) 2023-07-18T03:24:41.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.753+0000 7f9be9b9a700 1 -- 172.21.15.101:0/3727451248 learned_addr learned my addr 172.21.15.101:0/3727451248 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:41.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.753+0000 7f9beab9c700 1 --2- 172.21.15.101:0/3727451248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614258c8410 0x5614258d64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:41.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.753+0000 7f9beab9c700 1 -- 172.21.15.101:0/3727451248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614258bf2d0 msgr2=0x5614258c7990 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-18T03:24:41.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.753+0000 7f9beab9c700 1 --2- 172.21.15.101:0/3727451248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614258bf2d0 0x5614258c7990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.753+0000 7f9beab9c700 1 -- 172.21.15.101:0/3727451248 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614258caad0 msgr2=0x5614258c7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:41.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.753+0000 7f9beab9c700 1 --2- 172.21.15.101:0/3727451248 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614258caad0 0x5614258c7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.753+0000 7f9beab9c700 1 -- 172.21.15.101:0/3727451248 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9be4113780 con 0x5614258c8410 2023-07-18T03:24:41.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.754+0000 7f9beab9c700 1 --2- 172.21.15.101:0/3727451248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614258c8410 0x5614258d64e0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc0c9e40 tx=0x7f9bdc099840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac38cfc8dac2a8f3 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:41.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.754+0000 7f9bc9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:24:41.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.754+0000 7f9bc9ffb700 1 -- 172.21.15.101:0/3727451248 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc009e50 con 0x5614258c8410 2023-07-18T03:24:41.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.754+0000 7f9bc9ffb700 1 -- 172.21.15.101:0/3727451248 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f9bdc0ca0c0 con 0x5614258c8410 2023-07-18T03:24:41.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.755+0000 7f9bffe4eb80 1 -- 172.21.15.101:0/3727451248 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56142576b4f0 con 0x5614258c8410 2023-07-18T03:24:41.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.755+0000 7f9bc9ffb700 1 -- 172.21.15.101:0/3727451248 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9bdc0ca280 con 0x5614258c8410 2023-07-18T03:24:41.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.755+0000 7f9bffe4eb80 10 client.28870 fetch_fsmap learned FSMap version 42 2023-07-18T03:24:41.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.755+0000 7f9bffe4eb80 1 -- 172.21.15.101:0/3727451248 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x561425628c60 con 0x5614258c8410 2023-07-18T03:24:41.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.757+0000 7f9bc9ffb700 1 -- 172.21.15.101:0/3727451248 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56142576b4f0 con 0x5614258c8410 2023-07-18T03:24:41.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.757+0000 7f9bc9ffb700 1 -- 172.21.15.101:0/3727451248 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 42) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f9bdc0a1780 con 0x5614258c8410 2023-07-18T03:24:41.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.757+0000 7f9bffe4eb80 10 client.28870 fetch_fsmap finished waiting for FSMap version 42 2023-07-18T03:24:41.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.757+0000 7f9bffe4eb80 10 client.28870 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:24:41.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.757+0000 7f9bffe4eb80 20 client.28870 populate_metadata read hostname 'smithi101' 2023-07-18T03:24:41.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.757+0000 7f9bffe4eb80 1 --2- 172.21.15.101:0/3727451248 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5614258da850 0x5614258dcc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:41.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.757+0000 7f9bffe4eb80 4 client.28870 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:24:41.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.757+0000 7f9bffe4eb80 1 -- 172.21.15.101:0/3727451248 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56142580aeb0 con 0x5614258da850 2023-07-18T03:24:41.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.759+0000 7f9bea39b700 1 --2- 172.21.15.101:0/3727451248 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5614258da850 0x5614258dcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:41.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.760+0000 7f9bc9ffb700 10 client.28870 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:24:41.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.760+0000 7f9bea39b700 1 --2- 172.21.15.101:0/3727451248 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5614258da850 0x5614258dcc60 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=df8f84aa9063503d server_cookie=f516dd118bbfa918 in_seq=0 out_seq=0 2023-07-18T03:24:41.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.761+0000 7f9bc9ffb700 1 -- 172.21.15.101:0/3727451248 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56142580aeb0 con 0x5614258da850 2023-07-18T03:24:41.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.761+0000 7f9bc9ffb700 10 client.28870 handle_command_reply: tid=0 2023-07-18T03:24:41.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.762+0000 7f9bffe4eb80 1 client.28870 shutdown 2023-07-18T03:24:41.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.762+0000 7f9bffe4eb80 1 -- 172.21.15.101:0/3727451248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614258c8410 msgr2=0x5614258d64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:41.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.762+0000 7f9bffe4eb80 1 --2- 172.21.15.101:0/3727451248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614258c8410 0x5614258d64e0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc0c9e40 tx=0x7f9bdc099840 comp rx=0 tx=0).stop 2023-07-18T03:24:41.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.762+0000 7f9bffe4eb80 1 -- 172.21.15.101:0/3727451248 shutdown_connections 2023-07-18T03:24:41.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.762+0000 7f9bffe4eb80 1 --2- 172.21.15.101:0/3727451248 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5614258da850 0x5614258dcc60 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.762+0000 7f9bffe4eb80 1 --2- 172.21.15.101:0/3727451248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614258bf2d0 0x5614258c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.762+0000 7f9bffe4eb80 1 --2- 172.21.15.101:0/3727451248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614258c8410 0x5614258d64e0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.762+0000 7f9bffe4eb80 1 --2- 172.21.15.101:0/3727451248 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614258caad0 0x5614258c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.762+0000 7f9bffe4eb80 1 -- 172.21.15.101:0/3727451248 >> 172.21.15.101:0/3727451248 conn(0x5614258b5da0 msgr2=0x5614258bb290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:41.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.762+0000 7f9bc9ffb700 0 client.28870 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:24:41.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.763+0000 7f9bffe4eb80 1 -- 172.21.15.101:0/3727451248 shutdown_connections 2023-07-18T03:24:41.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.763+0000 7f9bffe4eb80 1 -- 172.21.15.101:0/3727451248 wait complete. 2023-07-18T03:24:41.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.764+0000 7f9bffe4eb80 20 client.28870 trim_cache size 0 max 16384 2023-07-18T03:24:41.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.785+0000 7f9bffe4eb80 1 Processor -- start 2023-07-18T03:24:41.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.785+0000 7f9bffe4eb80 1 -- start start 2023-07-18T03:24:41.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.785+0000 7f9bffe4eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614258b5da0 0x5614258d6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:41.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.785+0000 7f9bffe4eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614258bf2d0 0x5614258c21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:41.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.785+0000 7f9bffe4eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614258c8410 0x5614258d14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:41.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.785+0000 7f9bffe4eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614258c8ce0 con 0x5614258bf2d0 2023-07-18T03:24:41.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.785+0000 7f9bffe4eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614258bcce0 con 0x5614258c8410 2023-07-18T03:24:41.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.785+0000 7f9bffe4eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614258d68a0 con 0x5614258b5da0 2023-07-18T03:24:41.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.786+0000 7f9be9b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614258bf2d0 0x5614258c21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:41.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.786+0000 7f9be9b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614258bf2d0 0x5614258c21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:51878/0 (socket says 172.21.15.101:51878) 2023-07-18T03:24:41.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.786+0000 7f9be9b9a700 1 -- 172.21.15.101:0/1455321260 learned_addr learned my addr 172.21.15.101:0/1455321260 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:41.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.787+0000 7f9be9b9a700 1 -- 172.21.15.101:0/1455321260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614258b5da0 msgr2=0x5614258d6360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-18T03:24:41.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.787+0000 7f9be9b9a700 1 --2- 172.21.15.101:0/1455321260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614258b5da0 0x5614258d6360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.787+0000 7f9be9b9a700 1 -- 172.21.15.101:0/1455321260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614258c8410 msgr2=0x5614258d14b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-18T03:24:41.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.787+0000 7f9be9b9a700 1 --2- 172.21.15.101:0/1455321260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614258c8410 0x5614258d14b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.787+0000 7f9be9b9a700 1 -- 172.21.15.101:0/1455321260 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614258b75e0 con 0x5614258bf2d0 2023-07-18T03:24:41.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.787+0000 7f9be9b9a700 1 --2- 172.21.15.101:0/1455321260 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614258bf2d0 0x5614258c21d0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4021040 tx=0x7f9bd4009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70bb765283be273a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:41.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.788+0000 7f9be13ff700 1 -- 172.21.15.101:0/1455321260 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd400e040 con 0x5614258bf2d0 2023-07-18T03:24:41.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.788+0000 7f9be13ff700 1 -- 172.21.15.101:0/1455321260 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9bd40090d0 con 0x5614258bf2d0 2023-07-18T03:24:41.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.788+0000 7f9be13ff700 1 -- 172.21.15.101:0/1455321260 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd4005990 con 0x5614258bf2d0 2023-07-18T03:24:41.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.789+0000 7f9bffe4eb80 1 -- 172.21.15.101:0/1455321260 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614258bf2d0 msgr2=0x5614258c21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:41.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.789+0000 7f9bffe4eb80 1 --2- 172.21.15.101:0/1455321260 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614258bf2d0 0x5614258c21d0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4021040 tx=0x7f9bd4009520 comp rx=0 tx=0).stop 2023-07-18T03:24:41.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.789+0000 7f9bffe4eb80 1 -- 172.21.15.101:0/1455321260 shutdown_connections 2023-07-18T03:24:41.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.789+0000 7f9bffe4eb80 1 --2- 172.21.15.101:0/1455321260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614258c8410 0x5614258d14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.789+0000 7f9bffe4eb80 1 --2- 172.21.15.101:0/1455321260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614258b5da0 0x5614258d6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.789+0000 7f9bffe4eb80 1 --2- 172.21.15.101:0/1455321260 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614258bf2d0 0x5614258c21d0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.789+0000 7f9bffe4eb80 1 -- 172.21.15.101:0/1455321260 >> 172.21.15.101:0/1455321260 conn(0x5614258d6f60 msgr2=0x5614258d5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:41.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.794+0000 7f9bffe4eb80 1 -- 172.21.15.101:0/1455321260 shutdown_connections 2023-07-18T03:24:41.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.795+0000 7f9bffe4eb80 1 -- 172.21.15.101:0/1455321260 wait complete. 2023-07-18T03:24:41.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.796+0000 7f9bffe4eb80 1 -- start start 2023-07-18T03:24:41.792 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.796+0000 7f9bffe4eb80 1 Processor -- start 2023-07-18T03:24:41.793 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.797+0000 7f9bffe4eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614258b5da0 0x561425922890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:41.793 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.797+0000 7f9bffe4eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614258bf2d0 0x561425922dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:41.793 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.797+0000 7f9bffe4eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614258c8410 0x56142592d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:41.793 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.797+0000 7f9bffe4eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56142592dc60 con 0x5614258bf2d0 2023-07-18T03:24:41.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.797+0000 7f9bffe4eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56142592dda0 con 0x5614258c8410 2023-07-18T03:24:41.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.797+0000 7f9bffe4eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56142593ef20 con 0x5614258b5da0 2023-07-18T03:24:41.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.797+0000 7f9bffe4eb80 10 client.0 _ll_register_callbacks cb 0x7f9bdc0993d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:24:41.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.798+0000 7f9be9b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614258bf2d0 0x561425922dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:41.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.798+0000 7f9be9b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614258bf2d0 0x561425922dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:51884/0 (socket says 172.21.15.101:51884) 2023-07-18T03:24:41.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.798+0000 7f9be9b9a700 1 -- 172.21.15.101:0/55726749 learned_addr learned my addr 172.21.15.101:0/55726749 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:41.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.798+0000 7f9beab9c700 1 --2- 172.21.15.101:0/55726749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614258c8410 0x56142592d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:41.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.799+0000 7f9be9b9a700 1 -- 172.21.15.101:0/55726749 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614258b5da0 msgr2=0x561425922890 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-18T03:24:41.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.799+0000 7f9be9b9a700 1 --2- 172.21.15.101:0/55726749 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614258b5da0 0x561425922890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.799+0000 7f9be9b9a700 1 -- 172.21.15.101:0/55726749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614258c8410 msgr2=0x56142592d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:41.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.799+0000 7f9be9b9a700 1 --2- 172.21.15.101:0/55726749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614258c8410 0x56142592d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.799+0000 7f9be9b9a700 1 -- 172.21.15.101:0/55726749 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bd4025040 con 0x5614258bf2d0 2023-07-18T03:24:41.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.799+0000 7f9be9b9a700 1 --2- 172.21.15.101:0/55726749 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614258bf2d0 0x561425922dd0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4000f80 tx=0x7f9bd4003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5b910ff1b35247c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:41.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.799+0000 7f9bc9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:24:41.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.800+0000 7f9bc9ffb700 1 -- 172.21.15.101:0/55726749 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd400e040 con 0x5614258bf2d0 2023-07-18T03:24:41.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.800+0000 7f9bc9ffb700 1 -- 172.21.15.101:0/55726749 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f9bd40054d0 con 0x5614258bf2d0 2023-07-18T03:24:41.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.800+0000 7f9bffe4eb80 1 -- 172.21.15.101:0/55726749 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9bd40079a0 con 0x5614258bf2d0 2023-07-18T03:24:41.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.801+0000 7f9bc9ffb700 1 -- 172.21.15.101:0/55726749 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9bd40077f0 con 0x5614258bf2d0 2023-07-18T03:24:41.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.801+0000 7f9bffe4eb80 10 client.18786 fetch_fsmap learned FSMap version 42 2023-07-18T03:24:41.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.801+0000 7f9bffe4eb80 1 -- 172.21.15.101:0/55726749 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x561425666250 con 0x5614258bf2d0 2023-07-18T03:24:41.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.801+0000 7f9bc9ffb700 1 -- 172.21.15.101:0/55726749 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9bd40079a0 con 0x5614258bf2d0 2023-07-18T03:24:41.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.801+0000 7f9bc9ffb700 1 -- 172.21.15.101:0/55726749 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 42) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f9bd4019070 con 0x5614258bf2d0 2023-07-18T03:24:41.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.802+0000 7f9bffe4eb80 10 client.18786 fetch_fsmap finished waiting for FSMap version 42 2023-07-18T03:24:41.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.802+0000 7f9bffe4eb80 10 client.18786 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-18T03:24:41.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.802+0000 7f9bffe4eb80 20 client.18786 populate_metadata read hostname 'smithi101' 2023-07-18T03:24:41.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.802+0000 7f9bffe4eb80 1 --2- 172.21.15.101:0/55726749 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5614258caad0 0x561425942180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:41.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.802+0000 7f9bffe4eb80 4 client.18786 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:24:41.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.802+0000 7f9bffe4eb80 1 -- 172.21.15.101:0/55726749 --> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5614257cd050 con 0x5614258caad0 2023-07-18T03:24:41.800 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:41 smithi155 ceph-mon[128885]: pgmap v538: 129 pgs: 129 active+clean; 1.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 17 MiB/s wr, 623 op/s 2023-07-18T03:24:41.800 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:41 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1890917068' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:24:41.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.804+0000 7f9bea39b700 1 --2- 172.21.15.101:0/55726749 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5614258caad0 0x561425942180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:41.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.812+0000 7f9bc9ffb700 10 client.18786 ms_handle_connect on v2:172.21.15.155:6838/2055462483 2023-07-18T03:24:41.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.812+0000 7f9bea39b700 1 --2- 172.21.15.101:0/55726749 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5614258caad0 0x561425942180 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=308a29c7bb51df41 server_cookie=18eb3f7f957ef4a in_seq=0 out_seq=0 2023-07-18T03:24:41.818 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.823+0000 7f9bc9ffb700 1 -- 172.21.15.101:0/55726749 <== mds.1 v2:172.21.15.155:6838/2055462483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5614257cd050 con 0x5614258caad0 2023-07-18T03:24:41.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.823+0000 7f9bc9ffb700 10 client.18786 handle_command_reply: tid=0 2023-07-18T03:24:41.822 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.826+0000 7f9bffe4eb80 1 client.18786 shutdown 2023-07-18T03:24:41.822 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.827+0000 7f9bffe4eb80 1 -- 172.21.15.101:0/55726749 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614258bf2d0 msgr2=0x561425922dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:41.822 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.827+0000 7f9bffe4eb80 1 --2- 172.21.15.101:0/55726749 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614258bf2d0 0x561425922dd0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4000f80 tx=0x7f9bd4003ef0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.823 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.828+0000 7f9bffe4eb80 1 -- 172.21.15.101:0/55726749 shutdown_connections 2023-07-18T03:24:41.823 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.828+0000 7f9bffe4eb80 1 --2- 172.21.15.101:0/55726749 >> [v2:172.21.15.155:6838/2055462483,v1:172.21.15.155:6839/2055462483] conn(0x5614258caad0 0x561425942180 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.823 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.828+0000 7f9bffe4eb80 1 --2- 172.21.15.101:0/55726749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614258c8410 0x56142592d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.824 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.828+0000 7f9bffe4eb80 1 --2- 172.21.15.101:0/55726749 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614258b5da0 0x561425922890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.824 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.828+0000 7f9bffe4eb80 1 --2- 172.21.15.101:0/55726749 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614258bf2d0 0x561425922dd0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.824 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.828+0000 7f9bffe4eb80 1 -- 172.21.15.101:0/55726749 >> 172.21.15.101:0/55726749 conn(0x5614258d6f60 msgr2=0x5614258c9230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:41.826 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.830+0000 7f9bc9ffb700 0 client.18786 ms_handle_reset on v2:172.21.15.155:6838/2055462483 2023-07-18T03:24:41.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.831+0000 7f9bffe4eb80 1 -- 172.21.15.101:0/55726749 shutdown_connections 2023-07-18T03:24:41.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.832+0000 7f9bffe4eb80 1 -- 172.21.15.101:0/55726749 wait complete. 2023-07-18T03:24:41.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.832+0000 7f9bffe4eb80 20 client.18786 trim_cache size 0 max 16384 2023-07-18T03:24:41.829 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:24:41.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.836+0000 7f9bec5ff700 1 -- 172.21.15.101:0/1229033189 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9bcc0a1400 msgr2=0x7f9bcc0a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:41.832 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.836+0000 7f9bec5ff700 1 --2- 172.21.15.101:0/1229033189 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9bcc0a1400 0x7f9bcc0a38b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x5614258c0370 tx=0x7f9bd8006c00 comp rx=0 tx=0).stop 2023-07-18T03:24:41.832 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.836+0000 7f9bec5ff700 1 -- 172.21.15.101:0/1229033189 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9be4110ae0 msgr2=0x7f9be41a0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:41.832 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.836+0000 7f9bec5ff700 1 --2- 172.21.15.101:0/1229033189 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9be4110ae0 0x7f9be41a0c30 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc00d950 tx=0x7f9bdc00dc60 comp rx=0 tx=0).stop 2023-07-18T03:24:41.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.837+0000 7f9bec5ff700 1 -- 172.21.15.101:0/1229033189 shutdown_connections 2023-07-18T03:24:41.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.837+0000 7f9bec5ff700 1 --2- 172.21.15.101:0/1229033189 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f9bcc0a1400 0x7f9bcc0a38b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.837+0000 7f9bec5ff700 1 --2- 172.21.15.101:0/1229033189 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9be40728b0 0x7f9be419c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.837+0000 7f9bec5ff700 1 --2- 172.21.15.101:0/1229033189 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9be4110ae0 0x7f9be41a0c30 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.837+0000 7f9bec5ff700 1 --2- 172.21.15.101:0/1229033189 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9be40731c0 0x7f9be419c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:41.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.837+0000 7f9bec5ff700 1 -- 172.21.15.101:0/1229033189 >> 172.21.15.101:0/1229033189 conn(0x7f9be40fb5f0 msgr2=0x7f9be40fc250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:41.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.838+0000 7f9bec5ff700 1 -- 172.21.15.101:0/1229033189 shutdown_connections 2023-07-18T03:24:41.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:41.839+0000 7f9bec5ff700 1 -- 172.21.15.101:0/1229033189 wait complete. 2023-07-18T03:24:41.864 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-18T03:24:42.001 INFO:tasks.workunit.client.0.smithi101.stdout: 10 135318 28.38 MB/sec execute 126 sec latency 159.174 ms 2023-07-18T03:24:42.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.148+0000 7ff74e7ff700 1 -- 172.21.15.101:0/950366068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff74806d3a0 msgr2=0x7ff74811b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:42.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.148+0000 7ff74e7ff700 1 --2- 172.21.15.101:0/950366068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff74806d3a0 0x7ff74811b470 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff74000b0a0 tx=0x7ff74000b3b0 comp rx=0 tx=0).stop 2023-07-18T03:24:42.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.148+0000 7ff74e7ff700 1 -- 172.21.15.101:0/950366068 shutdown_connections 2023-07-18T03:24:42.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.148+0000 7ff74e7ff700 1 --2- 172.21.15.101:0/950366068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff74806d3a0 0x7ff74811b470 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:42.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.148+0000 7ff74e7ff700 1 --2- 172.21.15.101:0/950366068 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff74810e280 0x7ff74810e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:42.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.148+0000 7ff74e7ff700 1 --2- 172.21.15.101:0/950366068 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff74806c9a0 0x7ff74806ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:42.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.148+0000 7ff74e7ff700 1 -- 172.21.15.101:0/950366068 >> 172.21.15.101:0/950366068 conn(0x7ff74806bf70 msgr2=0x7ff748070ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:42.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.149+0000 7ff74e7ff700 1 -- 172.21.15.101:0/950366068 shutdown_connections 2023-07-18T03:24:42.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.149+0000 7ff74e7ff700 1 -- 172.21.15.101:0/950366068 wait complete. 2023-07-18T03:24:42.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.150+0000 7ff74e7ff700 1 Processor -- start 2023-07-18T03:24:42.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.151+0000 7ff74e7ff700 1 -- start start 2023-07-18T03:24:42.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.151+0000 7ff74e7ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff74806c9a0 0x7ff74811ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:42.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.151+0000 7ff74e7ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff74806d3a0 0x7ff74811b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:42.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.151+0000 7ff74e7ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff74810e280 0x7ff748114de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:42.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.151+0000 7ff74e7ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7481155b0 con 0x7ff74806d3a0 2023-07-18T03:24:42.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.151+0000 7ff74e7ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff748115720 con 0x7ff74810e280 2023-07-18T03:24:42.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.151+0000 7ff74e7ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7481158e0 con 0x7ff74806c9a0 2023-07-18T03:24:42.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.152+0000 7ff74cd9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff74810e280 0x7ff748114de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:42.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.152+0000 7ff74cd9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff74810e280 0x7ff748114de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60132/0 (socket says 172.21.15.101:60132) 2023-07-18T03:24:42.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.152+0000 7ff74cd9c700 1 -- 172.21.15.101:0/4111979305 learned_addr learned my addr 172.21.15.101:0/4111979305 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:42.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.153+0000 7ff74cd9c700 1 -- 172.21.15.101:0/4111979305 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff74806c9a0 msgr2=0x7ff74811ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:42.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.153+0000 7ff74cd9c700 1 --2- 172.21.15.101:0/4111979305 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff74806c9a0 0x7ff74811ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:42.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.153+0000 7ff74cd9c700 1 -- 172.21.15.101:0/4111979305 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff74806d3a0 msgr2=0x7ff74811b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:42.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.153+0000 7ff74cd9c700 1 --2- 172.21.15.101:0/4111979305 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff74806d3a0 0x7ff74811b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:42.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.153+0000 7ff74cd9c700 1 -- 172.21.15.101:0/4111979305 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff740009d00 con 0x7ff74810e280 2023-07-18T03:24:42.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.153+0000 7ff74cd9c700 1 --2- 172.21.15.101:0/4111979305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff74810e280 0x7ff748114de0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff7400062a0 tx=0x7ff74000bd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:42.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.156+0000 7ff745ffb700 1 -- 172.21.15.101:0/4111979305 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74000e040 con 0x7ff74810e280 2023-07-18T03:24:42.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.156+0000 7ff745ffb700 1 -- 172.21.15.101:0/4111979305 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff740004e60 con 0x7ff74810e280 2023-07-18T03:24:42.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.156+0000 7ff745ffb700 1 -- 172.21.15.101:0/4111979305 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74001b790 con 0x7ff74810e280 2023-07-18T03:24:42.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.156+0000 7ff74e7ff700 1 -- 172.21.15.101:0/4111979305 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff748115b60 con 0x7ff74810e280 2023-07-18T03:24:42.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.156+0000 7ff74e7ff700 1 -- 172.21.15.101:0/4111979305 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7481b9ff0 con 0x7ff74810e280 2023-07-18T03:24:42.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.164+0000 7ff76219eb80 1 Processor -- start 2023-07-18T03:24:42.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.164+0000 7ff745ffb700 1 -- 172.21.15.101:0/4111979305 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7ff74001d9c0 con 0x7ff74810e280 2023-07-18T03:24:42.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.164+0000 7ff76219eb80 1 -- start start 2023-07-18T03:24:42.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.165+0000 7ff76219eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3b6c612d0 0x55d3b6c616a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:42.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.165+0000 7ff76219eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b6c61be0 0x55d3b6c6c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:42.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.165+0000 7ff76219eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b6c6cad0 0x55d3b6c6eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:42.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.165+0000 7ff76219eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b6b67cb0 con 0x55d3b6c612d0 2023-07-18T03:24:42.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.165+0000 7ff76219eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b6bbaa40 con 0x55d3b6c61be0 2023-07-18T03:24:42.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.165+0000 7ff76219eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b6bb3ed0 con 0x55d3b6c6cad0 2023-07-18T03:24:42.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.166+0000 7ff73f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b6c6cad0 0x55d3b6c6eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:42.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.166+0000 7ff747fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b6c61be0 0x55d3b6c6c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:42.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.166+0000 7ff747fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b6c61be0 0x55d3b6c6c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60138/0 (socket says 172.21.15.101:60138) 2023-07-18T03:24:42.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.166+0000 7ff73f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b6c6cad0 0x55d3b6c6eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:40738/0 (socket says 172.21.15.101:40738) 2023-07-18T03:24:42.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.166+0000 7ff747fff700 1 -- 172.21.15.101:0/3667885016 learned_addr learned my addr 172.21.15.101:0/3667885016 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:42.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.166+0000 7ff73f7fe700 1 --2- 172.21.15.101:0/3667885016 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3b6c612d0 0x55d3b6c616a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:42.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.166+0000 7ff747fff700 1 -- 172.21.15.101:0/3667885016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b6c6cad0 msgr2=0x55d3b6c6eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:42.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.166+0000 7ff747fff700 1 --2- 172.21.15.101:0/3667885016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b6c6cad0 0x55d3b6c6eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:42.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.166+0000 7ff747fff700 1 -- 172.21.15.101:0/3667885016 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3b6c612d0 msgr2=0x55d3b6c616a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:42.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.166+0000 7ff747fff700 1 --2- 172.21.15.101:0/3667885016 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3b6c612d0 0x55d3b6c616a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:42.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.166+0000 7ff747fff700 1 -- 172.21.15.101:0/3667885016 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3b69e1de0 con 0x55d3b6c61be0 2023-07-18T03:24:42.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.167+0000 7ff747fff700 1 --2- 172.21.15.101:0/3667885016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b6c61be0 0x55d3b6c6c590 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff738009c50 tx=0x7ff738009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c56c934d20c65641 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:42.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.167+0000 7ff73e7fc700 1 -- 172.21.15.101:0/3667885016 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7380097a0 con 0x55d3b6c61be0 2023-07-18T03:24:42.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.167+0000 7ff73e7fc700 1 -- 172.21.15.101:0/3667885016 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff738004830 con 0x55d3b6c61be0 2023-07-18T03:24:42.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.167+0000 7ff73e7fc700 1 -- 172.21.15.101:0/3667885016 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7380055d0 con 0x55d3b6c61be0 2023-07-18T03:24:42.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.168+0000 7ff745ffb700 1 --2- 172.21.15.101:0/4111979305 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff7340a1400 0x7ff7340a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:42.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.168+0000 7ff745ffb700 1 -- 172.21.15.101:0/4111979305 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9461+0+0 (secure 0 0 0) 0x7ff7400ce180 con 0x7ff74810e280 2023-07-18T03:24:42.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.168+0000 7ff76219eb80 1 -- 172.21.15.101:0/3667885016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b6c61be0 msgr2=0x55d3b6c6c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:42.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.168+0000 7ff76219eb80 1 --2- 172.21.15.101:0/3667885016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b6c61be0 0x55d3b6c6c590 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff738009c50 tx=0x7ff738009f60 comp rx=0 tx=0).stop 2023-07-18T03:24:42.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.168+0000 7ff73f7fe700 1 --2- 172.21.15.101:0/4111979305 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff7340a1400 0x7ff7340a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:42.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.169+0000 7ff73f7fe700 1 --2- 172.21.15.101:0/4111979305 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff7340a1400 0x7ff7340a38b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff730005950 tx=0x7ff7300058e0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:42.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.170+0000 7ff76219eb80 1 -- 172.21.15.101:0/3667885016 shutdown_connections 2023-07-18T03:24:42.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.170+0000 7ff76219eb80 1 --2- 172.21.15.101:0/3667885016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b6c6cad0 0x55d3b6c6eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:42.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.170+0000 7ff76219eb80 1 --2- 172.21.15.101:0/3667885016 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3b6c612d0 0x55d3b6c616a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:42.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.170+0000 7ff76219eb80 1 --2- 172.21.15.101:0/3667885016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b6c61be0 0x55d3b6c6c590 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:42.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.170+0000 7ff76219eb80 1 -- 172.21.15.101:0/3667885016 >> 172.21.15.101:0/3667885016 conn(0x55d3b6c57da0 msgr2=0x55d3b6c5d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:42.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.172+0000 7ff76219eb80 1 -- 172.21.15.101:0/3667885016 shutdown_connections 2023-07-18T03:24:42.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.173+0000 7ff76219eb80 1 -- 172.21.15.101:0/3667885016 wait complete. 2023-07-18T03:24:42.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.173+0000 7ff76219eb80 1 -- start start 2023-07-18T03:24:42.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.174+0000 7ff76219eb80 1 Processor -- start 2023-07-18T03:24:42.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.174+0000 7ff76219eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b6c612d0 0x55d3b6c6c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:42.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.174+0000 7ff76219eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3b6c61be0 0x55d3b6c69760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:42.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.174+0000 7ff76219eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b6c6cad0 0x55d3b6c69d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:42.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.174+0000 7ff76219eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b6b53ac0 con 0x55d3b6c61be0 2023-07-18T03:24:42.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.174+0000 7ff76219eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b6b537c0 con 0x55d3b6c6cad0 2023-07-18T03:24:42.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.174+0000 7ff76219eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b6a37920 con 0x55d3b6c612d0 2023-07-18T03:24:42.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.175+0000 7ff76219eb80 10 client.0 _ll_register_callbacks cb 0x55d3b6b51760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:24:42.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.175+0000 7ff747fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b6c612d0 0x55d3b6c6c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:42.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.175+0000 7ff74cd9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b6c6cad0 0x55d3b6c69d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:42.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.175+0000 7ff747fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b6c612d0 0x55d3b6c6c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:40744/0 (socket says 172.21.15.101:40744) 2023-07-18T03:24:42.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.175+0000 7ff747fff700 1 -- 172.21.15.101:0/2423009110 learned_addr learned my addr 172.21.15.101:0/2423009110 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:42.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.176+0000 7ff74cd9c700 1 -- 172.21.15.101:0/2423009110 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b6c612d0 msgr2=0x55d3b6c6c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:42.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.176+0000 7ff74cd9c700 1 --2- 172.21.15.101:0/2423009110 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b6c612d0 0x55d3b6c6c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:42.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.176+0000 7ff74cd9c700 1 -- 172.21.15.101:0/2423009110 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3b6c61be0 msgr2=0x55d3b6c69760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:42.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.176+0000 7ff74cd9c700 1 --2- 172.21.15.101:0/2423009110 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3b6c61be0 0x55d3b6c69760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:42.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.176+0000 7ff74cd9c700 1 -- 172.21.15.101:0/2423009110 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff738009900 con 0x55d3b6c6cad0 2023-07-18T03:24:42.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.176+0000 7ff74cd9c700 1 --2- 172.21.15.101:0/2423009110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b6c6cad0 0x55d3b6c69d90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff7400cb4c0 tx=0x7ff74009ab00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe89e608a496f0d5 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:42.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.177+0000 7ff7277fe700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:24:42.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.177+0000 7ff7277fe700 1 -- 172.21.15.101:0/2423009110 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74009ef00 con 0x55d3b6c6cad0 2023-07-18T03:24:42.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.177+0000 7ff7277fe700 1 -- 172.21.15.101:0/2423009110 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9461+0+0 (secure 0 0 0) 0x7ff7400a3290 con 0x55d3b6c6cad0 2023-07-18T03:24:42.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.177+0000 7ff76219eb80 1 -- 172.21.15.101:0/2423009110 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d3b6b0d4f0 con 0x55d3b6c6cad0 2023-07-18T03:24:42.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.177+0000 7ff7277fe700 1 -- 172.21.15.101:0/2423009110 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff74009a3c0 con 0x55d3b6c6cad0 2023-07-18T03:24:42.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.177+0000 7ff76219eb80 10 client.28874 fetch_fsmap learned FSMap version 42 2023-07-18T03:24:42.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.177+0000 7ff76219eb80 1 -- 172.21.15.101:0/2423009110 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55d3b69cac60 con 0x55d3b6c6cad0 2023-07-18T03:24:42.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.178+0000 7ff7277fe700 1 -- 172.21.15.101:0/2423009110 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d3b6b0d4f0 con 0x55d3b6c6cad0 2023-07-18T03:24:42.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.178+0000 7ff7277fe700 1 -- 172.21.15.101:0/2423009110 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 42) v1 ==== 7482+0+0 (secure 0 0 0) 0x7ff740095ae0 con 0x55d3b6c6cad0 2023-07-18T03:24:42.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.178+0000 7ff76219eb80 10 client.28874 fetch_fsmap finished waiting for FSMap version 42 2023-07-18T03:24:42.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.178+0000 7ff76219eb80 10 client.28874 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:24:42.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.178+0000 7ff76219eb80 20 client.28874 populate_metadata read hostname 'smithi101' 2023-07-18T03:24:42.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.178+0000 7ff76219eb80 1 --2- 172.21.15.101:0/2423009110 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55d3b6c7bfc0 0x55d3b6c7e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:42.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.178+0000 7ff76219eb80 4 client.28874 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:24:42.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.178+0000 7ff76219eb80 1 -- 172.21.15.101:0/2423009110 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d3b6baceb0 con 0x55d3b6c7bfc0 2023-07-18T03:24:42.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.183+0000 7ff747fff700 1 --2- 172.21.15.101:0/2423009110 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55d3b6c7bfc0 0x55d3b6c7e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:42.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.187+0000 7ff747fff700 1 --2- 172.21.15.101:0/2423009110 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55d3b6c7bfc0 0x55d3b6c7e3d0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96d3ed220b9ea06 server_cookie=4f4faadb3176cc41 in_seq=0 out_seq=0 2023-07-18T03:24:42.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.187+0000 7ff7277fe700 10 client.28874 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:24:42.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.194+0000 7ff7277fe700 1 -- 172.21.15.101:0/2423009110 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d3b6baceb0 con 0x55d3b6c7bfc0 2023-07-18T03:24:42.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.194+0000 7ff7277fe700 10 client.28874 handle_command_reply: tid=0 2023-07-18T03:24:42.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.195+0000 7ff76219eb80 1 client.28874 shutdown 2023-07-18T03:24:42.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.197+0000 7ff76219eb80 1 -- 172.21.15.101:0/2423009110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b6c6cad0 msgr2=0x55d3b6c69d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:42.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.197+0000 7ff76219eb80 1 --2- 172.21.15.101:0/2423009110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b6c6cad0 0x55d3b6c69d90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff7400cb4c0 tx=0x7ff74009ab00 comp rx=0 tx=0).stop 2023-07-18T03:24:42.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.199+0000 7ff76219eb80 1 -- 172.21.15.101:0/2423009110 shutdown_connections 2023-07-18T03:24:42.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.199+0000 7ff76219eb80 1 --2- 172.21.15.101:0/2423009110 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55d3b6c7bfc0 0x55d3b6c7e3d0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:42.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.199+0000 7ff76219eb80 1 --2- 172.21.15.101:0/2423009110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b6c6cad0 0x55d3b6c69d90 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:42.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.199+0000 7ff76219eb80 1 --2- 172.21.15.101:0/2423009110 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b6c612d0 0x55d3b6c6c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:42.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.199+0000 7ff76219eb80 1 --2- 172.21.15.101:0/2423009110 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3b6c61be0 0x55d3b6c69760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:42.195 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.199+0000 7ff76219eb80 1 -- 172.21.15.101:0/2423009110 >> 172.21.15.101:0/2423009110 conn(0x55d3b6c57da0 msgr2=0x55d3b6c5d230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:42.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.199+0000 7ff7277fe700 0 client.28874 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:24:42.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.200+0000 7ff76219eb80 1 -- 172.21.15.101:0/2423009110 shutdown_connections 2023-07-18T03:24:42.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.200+0000 7ff76219eb80 1 -- 172.21.15.101:0/2423009110 wait complete. 2023-07-18T03:24:42.197 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.201+0000 7ff76219eb80 20 client.28874 trim_cache size 0 max 16384 2023-07-18T03:24:42.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.225+0000 7ff76219eb80 1 Processor -- start 2023-07-18T03:24:42.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.225+0000 7ff76219eb80 1 -- start start 2023-07-18T03:24:42.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.225+0000 7ff76219eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b6c612d0 0x55d3b6c6c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:42.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.225+0000 7ff76219eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3b6c61be0 0x55d3b6c68f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:42.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.225+0000 7ff76219eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b6c6cad0 0x55d3b6c69460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:42.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.225+0000 7ff76219eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b6c625b0 con 0x55d3b6c61be0 2023-07-18T03:24:42.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.225+0000 7ff76219eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b6c54fd0 con 0x55d3b6c6cad0 2023-07-18T03:24:42.222 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.225+0000 7ff76219eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b6c579a0 con 0x55d3b6c612d0 2023-07-18T03:24:42.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.226+0000 7ff73f7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3b6c61be0 0x55d3b6c68f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:42.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.226+0000 7ff73f7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3b6c61be0 0x55d3b6c68f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:51936/0 (socket says 172.21.15.101:51936) 2023-07-18T03:24:42.224 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.226+0000 7ff73f7fe700 1 -- 172.21.15.101:0/1523465216 learned_addr learned my addr 172.21.15.101:0/1523465216 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:42.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.226+0000 7ff74cd9c700 1 --2- 172.21.15.101:0/1523465216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b6c6cad0 0x55d3b6c69460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:42.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.226+0000 7ff747fff700 1 --2- 172.21.15.101:0/1523465216 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b6c612d0 0x55d3b6c6c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:42.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.226+0000 7ff747fff700 1 -- 172.21.15.101:0/1523465216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b6c6cad0 msgr2=0x55d3b6c69460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:42.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.226+0000 7ff747fff700 1 --2- 172.21.15.101:0/1523465216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b6c6cad0 0x55d3b6c69460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:42.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.226+0000 7ff747fff700 1 -- 172.21.15.101:0/1523465216 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3b6c61be0 msgr2=0x55d3b6c68f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:42.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.226+0000 7ff747fff700 1 --2- 172.21.15.101:0/1523465216 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3b6c61be0 0x55d3b6c68f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:42.225 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.226+0000 7ff747fff700 1 -- 172.21.15.101:0/1523465216 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3b6c595e0 con 0x55d3b6c612d0 2023-07-18T03:24:42.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.227+0000 7ff74cd9c700 1 --2- 172.21.15.101:0/1523465216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b6c6cad0 0x55d3b6c69460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:24:42.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.227+0000 7ff747fff700 1 --2- 172.21.15.101:0/1523465216 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b6c612d0 0x55d3b6c6c010 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x55d3b6c7ecd0 tx=0x7ff738009230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65d7bc6ad5c8a6fe server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:42.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.227+0000 7ff73effd700 1 -- 172.21.15.101:0/1523465216 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff738009730 con 0x55d3b6c612d0 2023-07-18T03:24:42.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.227+0000 7ff73effd700 1 -- 172.21.15.101:0/1523465216 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff73800e930 con 0x55d3b6c612d0 2023-07-18T03:24:42.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.227+0000 7ff73effd700 1 -- 172.21.15.101:0/1523465216 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff738004550 con 0x55d3b6c612d0 2023-07-18T03:24:42.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.228+0000 7ff76219eb80 1 -- 172.21.15.101:0/1523465216 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b6c612d0 msgr2=0x55d3b6c6c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:42.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.228+0000 7ff76219eb80 1 --2- 172.21.15.101:0/1523465216 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b6c612d0 0x55d3b6c6c010 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x55d3b6c7ecd0 tx=0x7ff738009230 comp rx=0 tx=0).stop 2023-07-18T03:24:42.226 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.228+0000 7ff76219eb80 1 -- 172.21.15.101:0/1523465216 shutdown_connections 2023-07-18T03:24:42.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.228+0000 7ff76219eb80 1 --2- 172.21.15.101:0/1523465216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b6c6cad0 0x55d3b6c69460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:42.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.228+0000 7ff76219eb80 1 --2- 172.21.15.101:0/1523465216 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b6c612d0 0x55d3b6c6c010 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:42.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.228+0000 7ff76219eb80 1 --2- 172.21.15.101:0/1523465216 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3b6c61be0 0x55d3b6c68f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:42.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.228+0000 7ff76219eb80 1 -- 172.21.15.101:0/1523465216 >> 172.21.15.101:0/1523465216 conn(0x55d3b6c57da0 msgr2=0x55d3b6c64a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:42.227 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.228+0000 7ff76219eb80 1 -- 172.21.15.101:0/1523465216 shutdown_connections 2023-07-18T03:24:42.228 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.228+0000 7ff76219eb80 1 -- 172.21.15.101:0/1523465216 wait complete. 2023-07-18T03:24:42.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.229+0000 7ff76219eb80 1 -- start start 2023-07-18T03:24:42.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.229+0000 7ff76219eb80 1 Processor -- start 2023-07-18T03:24:42.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.229+0000 7ff76219eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3b6c612d0 0x55d3b6c71680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:42.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.230+0000 7ff76219eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b6c61be0 0x55d3b6c71bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:42.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.230+0000 7ff76219eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b6c6cad0 0x55d3b6cceea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:42.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.230+0000 7ff76219eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b6c72350 con 0x55d3b6c612d0 2023-07-18T03:24:42.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.230+0000 7ff76219eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b6ccf410 con 0x55d3b6c6cad0 2023-07-18T03:24:42.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.230+0000 7ff76219eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b6ccf5d0 con 0x55d3b6c61be0 2023-07-18T03:24:42.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.230+0000 7ff76219eb80 10 client.0 _ll_register_callbacks cb 0x7ff7400972c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:24:42.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.230+0000 7ff73f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b6c61be0 0x55d3b6c71bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:42.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.230+0000 7ff73f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b6c61be0 0x55d3b6c71bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:40768/0 (socket says 172.21.15.101:40768) 2023-07-18T03:24:42.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.230+0000 7ff73f7fe700 1 -- 172.21.15.101:0/1884191952 learned_addr learned my addr 172.21.15.101:0/1884191952 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:42.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.230+0000 7ff747fff700 1 --2- 172.21.15.101:0/1884191952 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3b6c612d0 0x55d3b6c71680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:42.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.230+0000 7ff74cd9c700 1 --2- 172.21.15.101:0/1884191952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b6c6cad0 0x55d3b6cceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:42.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.230+0000 7ff747fff700 1 -- 172.21.15.101:0/1884191952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b6c61be0 msgr2=0x55d3b6c71bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:42.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.230+0000 7ff747fff700 1 --2- 172.21.15.101:0/1884191952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b6c61be0 0x55d3b6c71bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:42.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.231+0000 7ff747fff700 1 -- 172.21.15.101:0/1884191952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b6c6cad0 msgr2=0x55d3b6cceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:42.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.231+0000 7ff747fff700 1 --2- 172.21.15.101:0/1884191952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b6c6cad0 0x55d3b6cceea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:42.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.231+0000 7ff747fff700 1 -- 172.21.15.101:0/1884191952 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff738009900 con 0x55d3b6c612d0 2023-07-18T03:24:42.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.231+0000 7ff73f7fe700 1 --2- 172.21.15.101:0/1884191952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b6c61be0 0x55d3b6c71bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:24:42.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.231+0000 7ff74cd9c700 1 --2- 172.21.15.101:0/1884191952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b6c6cad0 0x55d3b6cceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:24:42.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.231+0000 7ff747fff700 1 --2- 172.21.15.101:0/1884191952 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3b6c612d0 0x55d3b6c71680 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x55d3b6c7ecd0 tx=0x7ff73800eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3daac87e3834395 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:42.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.231+0000 7ff7277fe700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:24:42.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.231+0000 7ff7277fe700 1 -- 172.21.15.101:0/1884191952 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73800ed70 con 0x55d3b6c612d0 2023-07-18T03:24:42.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.231+0000 7ff7277fe700 1 -- 172.21.15.101:0/1884191952 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9461+0+0 (secure 0 0 0) 0x7ff738014070 con 0x55d3b6c612d0 2023-07-18T03:24:42.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.231+0000 7ff76219eb80 1 -- 172.21.15.101:0/1884191952 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff738004830 con 0x55d3b6c612d0 2023-07-18T03:24:42.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.232+0000 7ff7277fe700 1 -- 172.21.15.101:0/1884191952 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff738004a20 con 0x55d3b6c612d0 2023-07-18T03:24:42.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.232+0000 7ff76219eb80 10 client.18798 fetch_fsmap learned FSMap version 42 2023-07-18T03:24:42.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.232+0000 7ff76219eb80 1 -- 172.21.15.101:0/1884191952 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7ff73801c040 con 0x55d3b6c612d0 2023-07-18T03:24:42.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.232+0000 7ff7277fe700 1 -- 172.21.15.101:0/1884191952 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff738004830 con 0x55d3b6c612d0 2023-07-18T03:24:42.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.232+0000 7ff7277fe700 1 -- 172.21.15.101:0/1884191952 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 42) v1 ==== 7482+0+0 (secure 0 0 0) 0x7ff73803b070 con 0x55d3b6c612d0 2023-07-18T03:24:42.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.232+0000 7ff76219eb80 10 client.18798 fetch_fsmap finished waiting for FSMap version 42 2023-07-18T03:24:42.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.232+0000 7ff76219eb80 10 client.18798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:24:42.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.232+0000 7ff76219eb80 20 client.18798 populate_metadata read hostname 'smithi101' 2023-07-18T03:24:42.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.232+0000 7ff76219eb80 1 --2- 172.21.15.101:0/1884191952 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55d3b6cd0ad0 0x55d3b6cd2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:42.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.232+0000 7ff76219eb80 4 client.18798 mds_command: new command op to 24562 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:24:42.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.232+0000 7ff76219eb80 1 -- 172.21.15.101:0/1884191952 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d3b6b6f050 con 0x55d3b6cd0ad0 2023-07-18T03:24:42.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.233+0000 7ff73f7fe700 1 --2- 172.21.15.101:0/1884191952 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55d3b6cd0ad0 0x55d3b6cd2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:42.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.233+0000 7ff73f7fe700 1 --2- 172.21.15.101:0/1884191952 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55d3b6cd0ad0 0x55d3b6cd2ee0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=babe6fd1ee479572 server_cookie=813ce0768277d5dd in_seq=0 out_seq=0 2023-07-18T03:24:42.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.233+0000 7ff7277fe700 10 client.18798 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:24:42.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.234+0000 7ff7277fe700 1 -- 172.21.15.101:0/1884191952 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d3b6b6f050 con 0x55d3b6cd0ad0 2023-07-18T03:24:42.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.234+0000 7ff7277fe700 10 client.18798 handle_command_reply: tid=0 2023-07-18T03:24:42.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.234+0000 7ff76219eb80 1 client.18798 shutdown 2023-07-18T03:24:42.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.234+0000 7ff76219eb80 1 -- 172.21.15.101:0/1884191952 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3b6c612d0 msgr2=0x55d3b6c71680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:42.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.234+0000 7ff76219eb80 1 --2- 172.21.15.101:0/1884191952 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3b6c612d0 0x55d3b6c71680 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x55d3b6c7ecd0 tx=0x7ff73800eb80 comp rx=0 tx=0).stop 2023-07-18T03:24:42.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.234+0000 7ff76219eb80 1 -- 172.21.15.101:0/1884191952 shutdown_connections 2023-07-18T03:24:42.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.235+0000 7ff76219eb80 1 --2- 172.21.15.101:0/1884191952 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55d3b6cd0ad0 0x55d3b6cd2ee0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:42.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.235+0000 7ff76219eb80 1 --2- 172.21.15.101:0/1884191952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3b6c6cad0 0x55d3b6cceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:42.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.235+0000 7ff76219eb80 1 --2- 172.21.15.101:0/1884191952 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3b6c612d0 0x55d3b6c71680 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:42.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.235+0000 7ff7277fe700 0 client.18798 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:24:42.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.235+0000 7ff76219eb80 1 --2- 172.21.15.101:0/1884191952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b6c61be0 0x55d3b6c71bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:42.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.235+0000 7ff76219eb80 1 -- 172.21.15.101:0/1884191952 >> 172.21.15.101:0/1884191952 conn(0x55d3b6c57da0 msgr2=0x55d3b6c5c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:42.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.235+0000 7ff76219eb80 1 -- 172.21.15.101:0/1884191952 shutdown_connections 2023-07-18T03:24:42.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.235+0000 7ff76219eb80 1 -- 172.21.15.101:0/1884191952 wait complete. 2023-07-18T03:24:42.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.235+0000 7ff76219eb80 20 client.18798 trim_cache size 0 max 16384 2023-07-18T03:24:42.242 INFO:teuthology.orchestra.run.smithi101.stdout:[] 2023-07-18T03:24:42.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.235+0000 7ff74e7ff700 1 -- 172.21.15.101:0/4111979305 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff7340a1400 msgr2=0x7ff7340a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:42.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.235+0000 7ff74e7ff700 1 --2- 172.21.15.101:0/4111979305 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff7340a1400 0x7ff7340a38b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff730005950 tx=0x7ff7300058e0 comp rx=0 tx=0).stop 2023-07-18T03:24:42.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.236+0000 7ff74e7ff700 1 -- 172.21.15.101:0/4111979305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff74810e280 msgr2=0x7ff748114de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:42.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.236+0000 7ff74e7ff700 1 --2- 172.21.15.101:0/4111979305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff74810e280 0x7ff748114de0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff7400062a0 tx=0x7ff74000bd90 comp rx=0 tx=0).stop 2023-07-18T03:24:42.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.236+0000 7ff74e7ff700 1 -- 172.21.15.101:0/4111979305 shutdown_connections 2023-07-18T03:24:42.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.236+0000 7ff74e7ff700 1 --2- 172.21.15.101:0/4111979305 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff7340a1400 0x7ff7340a38b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:42.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.236+0000 7ff74e7ff700 1 --2- 172.21.15.101:0/4111979305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff74810e280 0x7ff748114de0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:42.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.236+0000 7ff74e7ff700 1 --2- 172.21.15.101:0/4111979305 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff74806c9a0 0x7ff74811ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:42.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.236+0000 7ff74e7ff700 1 --2- 172.21.15.101:0/4111979305 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff74806d3a0 0x7ff74811b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:42.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.236+0000 7ff74e7ff700 1 -- 172.21.15.101:0/4111979305 >> 172.21.15.101:0/4111979305 conn(0x7ff74806bf70 msgr2=0x7ff74806f160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:42.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.236+0000 7ff74e7ff700 1 -- 172.21.15.101:0/4111979305 shutdown_connections 2023-07-18T03:24:42.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:42.236+0000 7ff74e7ff700 1 -- 172.21.15.101:0/4111979305 wait complete. 2023-07-18T03:24:43.004 INFO:tasks.workunit.client.0.smithi101.stdout: 10 135933 28.42 MB/sec execute 127 sec latency 95.418 ms 2023-07-18T03:24:43.255 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-18T03:24:43.255 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-18T03:24:43.533 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.537+0000 7f2c929ff700 1 -- 172.21.15.101:0/139251632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c8c105af0 msgr2=0x7f2c8c105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:43.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.537+0000 7f2c929ff700 1 --2- 172.21.15.101:0/139251632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c8c105af0 0x7f2c8c105ec0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c009990 tx=0x7f2c7c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.538+0000 7f2c929ff700 1 -- 172.21.15.101:0/139251632 shutdown_connections 2023-07-18T03:24:43.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.538+0000 7f2c929ff700 1 --2- 172.21.15.101:0/139251632 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2c8c100490 0x7f2c8c10c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.538+0000 7f2c929ff700 1 --2- 172.21.15.101:0/139251632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c8c105af0 0x7f2c8c105ec0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.538+0000 7f2c929ff700 1 --2- 172.21.15.101:0/139251632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c8c0ffae0 0x7f2c8c0fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.534 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.538+0000 7f2c929ff700 1 -- 172.21.15.101:0/139251632 >> 172.21.15.101:0/139251632 conn(0x7f2c8c0fb5e0 msgr2=0x7f2c8c0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:43.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.538+0000 7f2c929ff700 1 -- 172.21.15.101:0/139251632 shutdown_connections 2023-07-18T03:24:43.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.538+0000 7f2c929ff700 1 -- 172.21.15.101:0/139251632 wait complete. 2023-07-18T03:24:43.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.539+0000 7f2c929ff700 1 Processor -- start 2023-07-18T03:24:43.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.539+0000 7f2c929ff700 1 -- start start 2023-07-18T03:24:43.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.540+0000 7f2c929ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2c8c0ffae0 0x7f2c8c072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:43.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.540+0000 7f2c929ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c8c100490 0x7f2c8c072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:43.536 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.540+0000 7f2c929ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c8c073290 0x7f2c8c1a40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:43.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.540+0000 7f2c929ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8c1a4730 con 0x7f2c8c0ffae0 2023-07-18T03:24:43.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.540+0000 7f2c929ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8c1a48a0 con 0x7f2c8c100490 2023-07-18T03:24:43.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.540+0000 7f2c929ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8c1a4a60 con 0x7f2c8c073290 2023-07-18T03:24:43.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.540+0000 7f2c8b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c8c100490 0x7f2c8c072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:43.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.540+0000 7f2c8b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c8c100490 0x7f2c8c072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60194/0 (socket says 172.21.15.101:60194) 2023-07-18T03:24:43.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.540+0000 7f2c8b7fe700 1 -- 172.21.15.101:0/3141975380 learned_addr learned my addr 172.21.15.101:0/3141975380 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:43.537 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.541+0000 7f2c90f9c700 1 --2- 172.21.15.101:0/3141975380 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c8c073290 0x7f2c8c1a40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:43.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.541+0000 7f2c8b7fe700 1 -- 172.21.15.101:0/3141975380 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c8c073290 msgr2=0x7f2c8c1a40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:43.538 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.541+0000 7f2c8b7fe700 1 --2- 172.21.15.101:0/3141975380 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c8c073290 0x7f2c8c1a40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.541+0000 7f2c8bfff700 1 --2- 172.21.15.101:0/3141975380 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2c8c0ffae0 0x7f2c8c072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:43.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.541+0000 7f2c8b7fe700 1 -- 172.21.15.101:0/3141975380 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2c8c0ffae0 msgr2=0x7f2c8c072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:43.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.541+0000 7f2c8b7fe700 1 --2- 172.21.15.101:0/3141975380 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2c8c0ffae0 0x7f2c8c072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.541+0000 7f2c8b7fe700 1 -- 172.21.15.101:0/3141975380 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c80009520 con 0x7f2c8c100490 2023-07-18T03:24:43.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.541+0000 7f2c8bfff700 1 --2- 172.21.15.101:0/3141975380 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2c8c0ffae0 0x7f2c8c072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:24:43.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.541+0000 7f2c8b7fe700 1 --2- 172.21.15.101:0/3141975380 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c8c100490 0x7f2c8c072d50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000eb30 tx=0x7f2c8000ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:43.539 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.542+0000 7f2c897fa700 1 -- 172.21.15.101:0/3141975380 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c80009880 con 0x7f2c8c100490 2023-07-18T03:24:43.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.542+0000 7f2c897fa700 1 -- 172.21.15.101:0/3141975380 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c800044e0 con 0x7f2c8c100490 2023-07-18T03:24:43.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.542+0000 7f2c929ff700 1 -- 172.21.15.101:0/3141975380 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c7c0095f0 con 0x7f2c8c100490 2023-07-18T03:24:43.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.542+0000 7f2c897fa700 1 -- 172.21.15.101:0/3141975380 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c80003e90 con 0x7f2c8c100490 2023-07-18T03:24:43.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.542+0000 7f2c929ff700 1 -- 172.21.15.101:0/3141975380 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c8c1a50c0 con 0x7f2c8c100490 2023-07-18T03:24:43.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.545+0000 7f2c897fa700 1 -- 172.21.15.101:0/3141975380 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f2c80004000 con 0x7f2c8c100490 2023-07-18T03:24:43.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.547+0000 7f2ca63a8b80 1 Processor -- start 2023-07-18T03:24:43.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.547+0000 7f2ca63a8b80 1 -- start start 2023-07-18T03:24:43.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.547+0000 7f2ca63a8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f35ef98b60 0x55f35ef98f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:43.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.547+0000 7f2ca63a8b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f35ef99470 0x55f35efa3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:43.543 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.547+0000 7f2ca63a8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f35efa4360 0x55f35efa6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:43.544 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.547+0000 7f2ca63a8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f35ee96d30 con 0x55f35ef98b60 2023-07-18T03:24:43.544 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.547+0000 7f2ca63a8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f35eee9ac0 con 0x55f35efa4360 2023-07-18T03:24:43.544 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.547+0000 7f2ca63a8b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f35eee2f50 con 0x55f35ef99470 2023-07-18T03:24:43.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.548+0000 7f2c897fa700 1 --2- 172.21.15.101:0/3141975380 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2c740a12c0 0x7f2c740a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:43.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.548+0000 7f2c90f9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f35ef98b60 0x55f35ef98f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:43.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.548+0000 7f2c897fa700 1 -- 172.21.15.101:0/3141975380 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f2c80014070 con 0x7f2c8c100490 2023-07-18T03:24:43.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.548+0000 7f2c90f9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f35ef98b60 0x55f35ef98f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:51958/0 (socket says 172.21.15.101:51958) 2023-07-18T03:24:43.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.548+0000 7f2c90f9c700 1 -- 172.21.15.101:0/2904698582 learned_addr learned my addr 172.21.15.101:0/2904698582 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:43.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.548+0000 7f2c90f9c700 1 --2- 172.21.15.101:0/3141975380 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2c740a12c0 0x7f2c740a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:43.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.549+0000 7f2c8b7fe700 1 --2- 172.21.15.101:0/2904698582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f35efa4360 0x55f35efa6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:43.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.549+0000 7f2c8bfff700 1 --2- 172.21.15.101:0/2904698582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f35ef99470 0x55f35efa3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:43.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.549+0000 7f2c90f9c700 1 -- 172.21.15.101:0/2904698582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f35ef99470 msgr2=0x55f35efa3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:43.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.549+0000 7f2c90f9c700 1 --2- 172.21.15.101:0/2904698582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f35ef99470 0x55f35efa3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.549+0000 7f2c90f9c700 1 -- 172.21.15.101:0/2904698582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f35efa4360 msgr2=0x55f35efa6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:43.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.549+0000 7f2c90f9c700 1 --2- 172.21.15.101:0/2904698582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f35efa4360 0x55f35efa6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.549+0000 7f2c90f9c700 1 -- 172.21.15.101:0/2904698582 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f35ed10e60 con 0x55f35ef98b60 2023-07-18T03:24:43.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.549+0000 7f2c8b7fe700 1 --2- 172.21.15.101:0/2904698582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f35efa4360 0x55f35efa6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:24:43.547 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.549+0000 7f2c90f9c700 1 --2- 172.21.15.101:0/2904698582 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f35ef98b60 0x55f35ef98f30 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f2c84008fa0 tx=0x7f2c8401a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf8e3b024752bb9d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:43.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.550+0000 7f2c90f9c700 1 --2- 172.21.15.101:0/3141975380 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2c740a12c0 0x7f2c740a3770 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2c84010000 tx=0x7f2c8401f000 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:43.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.550+0000 7f2c727fc700 1 -- 172.21.15.101:0/2904698582 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8400e030 con 0x55f35ef98b60 2023-07-18T03:24:43.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.550+0000 7f2c727fc700 1 -- 172.21.15.101:0/2904698582 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c8400f040 con 0x55f35ef98b60 2023-07-18T03:24:43.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.550+0000 7f2c727fc700 1 -- 172.21.15.101:0/2904698582 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c840239c0 con 0x55f35ef98b60 2023-07-18T03:24:43.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.551+0000 7f2ca63a8b80 1 -- 172.21.15.101:0/2904698582 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f35ef98b60 msgr2=0x55f35ef98f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:43.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.551+0000 7f2ca63a8b80 1 --2- 172.21.15.101:0/2904698582 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f35ef98b60 0x55f35ef98f30 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f2c84008fa0 tx=0x7f2c8401a6c0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.549 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.551+0000 7f2ca63a8b80 1 -- 172.21.15.101:0/2904698582 shutdown_connections 2023-07-18T03:24:43.549 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.551+0000 7f2ca63a8b80 1 --2- 172.21.15.101:0/2904698582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f35efa4360 0x55f35efa6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.549 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.551+0000 7f2ca63a8b80 1 --2- 172.21.15.101:0/2904698582 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f35ef98b60 0x55f35ef98f30 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.549 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.551+0000 7f2ca63a8b80 1 --2- 172.21.15.101:0/2904698582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f35ef99470 0x55f35efa3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.549 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.551+0000 7f2ca63a8b80 1 -- 172.21.15.101:0/2904698582 >> 172.21.15.101:0/2904698582 conn(0x55f35ef86e20 msgr2=0x55f35ef8c720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:43.549 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.551+0000 7f2ca63a8b80 1 -- 172.21.15.101:0/2904698582 shutdown_connections 2023-07-18T03:24:43.550 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.551+0000 7f2ca63a8b80 1 -- 172.21.15.101:0/2904698582 wait complete. 2023-07-18T03:24:43.550 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.552+0000 7f2ca63a8b80 1 -- start start 2023-07-18T03:24:43.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.552+0000 7f2ca63a8b80 1 Processor -- start 2023-07-18T03:24:43.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.553+0000 7f2ca63a8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f35ef98b60 0x55f35efa3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:43.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.553+0000 7f2ca63a8b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f35ef99470 0x55f35efa10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:43.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.553+0000 7f2ca63a8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f35efa4360 0x55f35efa17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:43.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.553+0000 7f2ca63a8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f35ee82b40 con 0x55f35ef98b60 2023-07-18T03:24:43.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.553+0000 7f2ca63a8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f35ee82840 con 0x55f35efa4360 2023-07-18T03:24:43.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.553+0000 7f2ca63a8b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f35ed669a0 con 0x55f35ef99470 2023-07-18T03:24:43.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.553+0000 7f2ca63a8b80 10 client.0 _ll_register_callbacks cb 0x55f35ee807e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:24:43.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.553+0000 7f2c8bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f35ef98b60 0x55f35efa3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:43.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.553+0000 7f2c8bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f35ef98b60 0x55f35efa3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:51974/0 (socket says 172.21.15.101:51974) 2023-07-18T03:24:43.553 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.553+0000 7f2c8bfff700 1 -- 172.21.15.101:0/2548516937 learned_addr learned my addr 172.21.15.101:0/2548516937 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:43.553 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.553+0000 7f2c90f9c700 1 --2- 172.21.15.101:0/2548516937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f35efa4360 0x55f35efa17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:43.553 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.554+0000 7f2c8bfff700 1 -- 172.21.15.101:0/2548516937 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f35ef99470 msgr2=0x55f35efa10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:43.553 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.554+0000 7f2c8bfff700 1 --2- 172.21.15.101:0/2548516937 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f35ef99470 0x55f35efa10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.553 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.554+0000 7f2c8bfff700 1 -- 172.21.15.101:0/2548516937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f35efa4360 msgr2=0x55f35efa17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:43.553 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.554+0000 7f2c8bfff700 1 --2- 172.21.15.101:0/2548516937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f35efa4360 0x55f35efa17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.553 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.554+0000 7f2c8bfff700 1 -- 172.21.15.101:0/2548516937 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c84008b50 con 0x55f35ef98b60 2023-07-18T03:24:43.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.554+0000 7f2c8bfff700 1 --2- 172.21.15.101:0/2548516937 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f35ef98b60 0x55f35efa3e20 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c0ffa10 tx=0x7f2c7c00ba80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a617e7823e6a575 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:43.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.554+0000 7f2c6b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:24:43.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.555+0000 7f2c6b7fe700 1 -- 172.21.15.101:0/2548516937 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c00f870 con 0x55f35ef98b60 2023-07-18T03:24:43.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.555+0000 7f2c6b7fe700 1 -- 172.21.15.101:0/2548516937 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f2c7c00f9d0 con 0x55f35ef98b60 2023-07-18T03:24:43.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.555+0000 7f2ca63a8b80 1 -- 172.21.15.101:0/2548516937 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f35ee3c570 con 0x55f35ef98b60 2023-07-18T03:24:43.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.555+0000 7f2c6b7fe700 1 -- 172.21.15.101:0/2548516937 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c7c00fb90 con 0x55f35ef98b60 2023-07-18T03:24:43.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.555+0000 7f2ca63a8b80 10 client.18807 fetch_fsmap learned FSMap version 42 2023-07-18T03:24:43.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.555+0000 7f2ca63a8b80 1 -- 172.21.15.101:0/2548516937 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55f35ecf9ce0 con 0x55f35ef98b60 2023-07-18T03:24:43.555 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.556+0000 7f2c6b7fe700 1 -- 172.21.15.101:0/2548516937 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f35ee3c570 con 0x55f35ef98b60 2023-07-18T03:24:43.556 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.556+0000 7f2c6b7fe700 1 -- 172.21.15.101:0/2548516937 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 42) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f2c7c02b070 con 0x55f35ef98b60 2023-07-18T03:24:43.556 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.556+0000 7f2ca63a8b80 10 client.18807 fetch_fsmap finished waiting for FSMap version 42 2023-07-18T03:24:43.556 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.556+0000 7f2ca63a8b80 10 client.18807 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:24:43.556 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.556+0000 7f2ca63a8b80 20 client.18807 populate_metadata read hostname 'smithi101' 2023-07-18T03:24:43.556 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.556+0000 7f2ca63a8b80 1 --2- 172.21.15.101:0/2548516937 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f35efb3c10 0x55f35efb6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:43.557 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.556+0000 7f2ca63a8b80 4 client.18807 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:24:43.557 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.556+0000 7f2ca63a8b80 1 -- 172.21.15.101:0/2548516937 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f35eedbf30 con 0x55f35efb3c10 2023-07-18T03:24:43.557 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.556+0000 7f2c8b7fe700 1 --2- 172.21.15.101:0/2548516937 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f35efb3c10 0x55f35efb6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:43.557 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.557+0000 7f2c8b7fe700 1 --2- 172.21.15.101:0/2548516937 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f35efb3c10 0x55f35efb6020 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2a30d3672e82e37 server_cookie=c495b5396f06ee4 in_seq=0 out_seq=0 2023-07-18T03:24:43.557 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.557+0000 7f2c6b7fe700 10 client.18807 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:24:43.557 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.559+0000 7f2c6b7fe700 1 -- 172.21.15.101:0/2548516937 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f35eedbf30 con 0x55f35efb3c10 2023-07-18T03:24:43.557 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.559+0000 7f2c6b7fe700 10 client.18807 handle_command_reply: tid=0 2023-07-18T03:24:43.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.560+0000 7f2ca63a8b80 1 client.18807 shutdown 2023-07-18T03:24:43.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.560+0000 7f2ca63a8b80 1 -- 172.21.15.101:0/2548516937 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f35ef98b60 msgr2=0x55f35efa3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:43.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.560+0000 7f2ca63a8b80 1 --2- 172.21.15.101:0/2548516937 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f35ef98b60 0x55f35efa3e20 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c0ffa10 tx=0x7f2c7c00ba80 comp rx=0 tx=0).stop 2023-07-18T03:24:43.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.560+0000 7f2ca63a8b80 1 -- 172.21.15.101:0/2548516937 shutdown_connections 2023-07-18T03:24:43.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.560+0000 7f2ca63a8b80 1 --2- 172.21.15.101:0/2548516937 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f35efb3c10 0x55f35efb6020 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.560+0000 7f2ca63a8b80 1 --2- 172.21.15.101:0/2548516937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f35efa4360 0x55f35efa17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.560+0000 7f2ca63a8b80 1 --2- 172.21.15.101:0/2548516937 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f35ef98b60 0x55f35efa3e20 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.560+0000 7f2ca63a8b80 1 --2- 172.21.15.101:0/2548516937 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f35ef99470 0x55f35efa10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.560+0000 7f2ca63a8b80 1 -- 172.21.15.101:0/2548516937 >> 172.21.15.101:0/2548516937 conn(0x55f35ef86e20 msgr2=0x55f35ef8c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:43.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.560+0000 7f2c6b7fe700 0 client.18807 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:24:43.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.560+0000 7f2ca63a8b80 1 -- 172.21.15.101:0/2548516937 shutdown_connections 2023-07-18T03:24:43.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.560+0000 7f2ca63a8b80 1 -- 172.21.15.101:0/2548516937 wait complete. 2023-07-18T03:24:43.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.560+0000 7f2ca63a8b80 20 client.18807 trim_cache size 0 max 16384 2023-07-18T03:24:43.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.583+0000 7f2ca63a8b80 1 Processor -- start 2023-07-18T03:24:43.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.583+0000 7f2ca63a8b80 1 -- start start 2023-07-18T03:24:43.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.583+0000 7f2ca63a8b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f35ef86e20 0x55f35efaad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:43.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.583+0000 7f2ca63a8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f35ef98b60 0x55f35ef9b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:43.581 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.583+0000 7f2ca63a8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f35ef99470 0x55f35efa0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:43.581 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.583+0000 7f2ca63a8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f35ef8dda0 con 0x55f35ef99470 2023-07-18T03:24:43.581 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.583+0000 7f2ca63a8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f35ef9bfc0 con 0x55f35ef98b60 2023-07-18T03:24:43.581 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.583+0000 7f2ca63a8b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f35efa26b0 con 0x55f35ef86e20 2023-07-18T03:24:43.581 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.584+0000 7f2c8bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f35ef86e20 0x55f35efaad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:43.581 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.584+0000 7f2c8bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f35ef86e20 0x55f35efaad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:40802/0 (socket says 172.21.15.101:40802) 2023-07-18T03:24:43.581 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.584+0000 7f2c8bfff700 1 -- 172.21.15.101:0/2125419660 learned_addr learned my addr 172.21.15.101:0/2125419660 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:43.582 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.584+0000 7f2c8b7fe700 1 --2- 172.21.15.101:0/2125419660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f35ef98b60 0x55f35ef9b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:43.582 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.584+0000 7f2c90f9c700 1 --2- 172.21.15.101:0/2125419660 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f35ef99470 0x55f35efa0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:43.582 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.584+0000 7f2c8bfff700 1 -- 172.21.15.101:0/2125419660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f35ef98b60 msgr2=0x55f35ef9b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:43.582 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.584+0000 7f2c8bfff700 1 --2- 172.21.15.101:0/2125419660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f35ef98b60 0x55f35ef9b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.582 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.584+0000 7f2c8bfff700 1 -- 172.21.15.101:0/2125419660 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f35ef99470 msgr2=0x55f35efa0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:43.582 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.584+0000 7f2c8bfff700 1 --2- 172.21.15.101:0/2125419660 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f35ef99470 0x55f35efa0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.582 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.584+0000 7f2c8bfff700 1 -- 172.21.15.101:0/2125419660 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f35ed372d0 con 0x55f35ef86e20 2023-07-18T03:24:43.582 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.584+0000 7f2c8b7fe700 1 --2- 172.21.15.101:0/2125419660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f35ef98b60 0x55f35ef9b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:24:43.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.585+0000 7f2c8bfff700 1 --2- 172.21.15.101:0/2125419660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f35ef86e20 0x55f35efaad40 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c005f50 tx=0x7f2c7c017d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddfb712dd59cb57c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:43.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.585+0000 7f2c72ffd700 1 -- 172.21.15.101:0/2125419660 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c017ea0 con 0x55f35ef86e20 2023-07-18T03:24:43.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.585+0000 7f2c72ffd700 1 -- 172.21.15.101:0/2125419660 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c7c01f040 con 0x55f35ef86e20 2023-07-18T03:24:43.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.585+0000 7f2c72ffd700 1 -- 172.21.15.101:0/2125419660 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c00fa90 con 0x55f35ef86e20 2023-07-18T03:24:43.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.586+0000 7f2ca63a8b80 1 -- 172.21.15.101:0/2125419660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f35ef86e20 msgr2=0x55f35efaad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:43.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.586+0000 7f2ca63a8b80 1 --2- 172.21.15.101:0/2125419660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f35ef86e20 0x55f35efaad40 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c005f50 tx=0x7f2c7c017d50 comp rx=0 tx=0).stop 2023-07-18T03:24:43.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.586+0000 7f2ca63a8b80 1 -- 172.21.15.101:0/2125419660 shutdown_connections 2023-07-18T03:24:43.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.586+0000 7f2ca63a8b80 1 --2- 172.21.15.101:0/2125419660 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f35ef99470 0x55f35efa0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.586+0000 7f2ca63a8b80 1 --2- 172.21.15.101:0/2125419660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f35ef86e20 0x55f35efaad40 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.586+0000 7f2ca63a8b80 1 --2- 172.21.15.101:0/2125419660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f35ef98b60 0x55f35ef9b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.586+0000 7f2ca63a8b80 1 -- 172.21.15.101:0/2125419660 >> 172.21.15.101:0/2125419660 conn(0x55f35efaf840 msgr2=0x55f35ef9ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:43.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.586+0000 7f2ca63a8b80 1 -- 172.21.15.101:0/2125419660 shutdown_connections 2023-07-18T03:24:43.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.586+0000 7f2ca63a8b80 1 -- 172.21.15.101:0/2125419660 wait complete. 2023-07-18T03:24:43.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.587+0000 7f2ca63a8b80 1 -- start start 2023-07-18T03:24:43.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.587+0000 7f2ca63a8b80 1 Processor -- start 2023-07-18T03:24:43.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.587+0000 7f2ca63a8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f35ef86e20 0x55f35efa9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:43.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.588+0000 7f2ca63a8b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f35ef98b60 0x55f35efa9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:43.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.588+0000 7f2ca63a8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f35ef99470 0x55f35efa9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:43.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.588+0000 7f2ca63a8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f35f007bd0 con 0x55f35ef86e20 2023-07-18T03:24:43.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.588+0000 7f2ca63a8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f35f007d10 con 0x55f35ef99470 2023-07-18T03:24:43.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.588+0000 7f2ca63a8b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f35f007ed0 con 0x55f35ef98b60 2023-07-18T03:24:43.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.588+0000 7f2ca63a8b80 10 client.0 _ll_register_callbacks cb 0x7f2c840097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:24:43.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.588+0000 7f2c8bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f35ef86e20 0x55f35efa9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:43.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.588+0000 7f2c90f9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f35ef99470 0x55f35efa9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:43.589 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.588+0000 7f2c8bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f35ef86e20 0x55f35efa9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:51988/0 (socket says 172.21.15.101:51988) 2023-07-18T03:24:43.589 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.588+0000 7f2c90f9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f35ef99470 0x55f35efa9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60234/0 (socket says 172.21.15.101:60234) 2023-07-18T03:24:43.589 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.588+0000 7f2c8bfff700 1 -- 172.21.15.101:0/3136026303 learned_addr learned my addr 172.21.15.101:0/3136026303 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:43.589 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.588+0000 7f2c8b7fe700 1 --2- 172.21.15.101:0/3136026303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f35ef98b60 0x55f35efa9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:43.589 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.588+0000 7f2c90f9c700 1 -- 172.21.15.101:0/3136026303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f35ef98b60 msgr2=0x55f35efa9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:43.589 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.588+0000 7f2c90f9c700 1 --2- 172.21.15.101:0/3136026303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f35ef98b60 0x55f35efa9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.589 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.588+0000 7f2c90f9c700 1 -- 172.21.15.101:0/3136026303 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f35ef86e20 msgr2=0x55f35efa9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:43.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.588+0000 7f2c90f9c700 1 --2- 172.21.15.101:0/3136026303 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f35ef86e20 0x55f35efa9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.588+0000 7f2c90f9c700 1 -- 172.21.15.101:0/3136026303 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c8c1a50c0 con 0x55f35ef99470 2023-07-18T03:24:43.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.589+0000 7f2c90f9c700 1 --2- 172.21.15.101:0/3136026303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f35ef99470 0x55f35efa9f90 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f2c84009f70 tx=0x7f2c8401bcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=492c37c547533f99 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:43.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.589+0000 7f2c6b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:24:43.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.589+0000 7f2c6b7fe700 1 -- 172.21.15.101:0/3136026303 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8400e040 con 0x55f35ef99470 2023-07-18T03:24:43.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.589+0000 7f2ca63a8b80 1 -- 172.21.15.101:0/3136026303 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c8400f040 con 0x55f35ef99470 2023-07-18T03:24:43.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.589+0000 7f2c6b7fe700 1 -- 172.21.15.101:0/3136026303 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f2c84003c00 con 0x55f35ef99470 2023-07-18T03:24:43.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.589+0000 7f2c6b7fe700 1 -- 172.21.15.101:0/3136026303 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c84003de0 con 0x55f35ef99470 2023-07-18T03:24:43.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.589+0000 7f2ca63a8b80 10 client.28898 fetch_fsmap learned FSMap version 42 2023-07-18T03:24:43.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.589+0000 7f2ca63a8b80 1 -- 172.21.15.101:0/3136026303 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55f35ecf9ce0 con 0x55f35ef99470 2023-07-18T03:24:43.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.589+0000 7f2c6b7fe700 1 -- 172.21.15.101:0/3136026303 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c8400f040 con 0x55f35ef99470 2023-07-18T03:24:43.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.589+0000 7f2c6b7fe700 1 -- 172.21.15.101:0/3136026303 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 42) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f2c8401a3e0 con 0x55f35ef99470 2023-07-18T03:24:43.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.589+0000 7f2ca63a8b80 10 client.28898 fetch_fsmap finished waiting for FSMap version 42 2023-07-18T03:24:43.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.589+0000 7f2ca63a8b80 10 client.28898 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:24:43.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.590+0000 7f2ca63a8b80 20 client.28898 populate_metadata read hostname 'smithi101' 2023-07-18T03:24:43.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.590+0000 7f2ca63a8b80 1 --2- 172.21.15.101:0/3136026303 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f35efa4360 0x55f35f00b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:43.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.590+0000 7f2ca63a8b80 4 client.28898 mds_command: new command op to 24562 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-18T03:24:43.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.590+0000 7f2ca63a8b80 1 -- 172.21.15.101:0/3136026303 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55f35ee9e0d0 con 0x55f35efa4360 2023-07-18T03:24:43.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.590+0000 7f2c8bfff700 1 --2- 172.21.15.101:0/3136026303 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f35efa4360 0x55f35f00b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:43.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.590+0000 7f2c8bfff700 1 --2- 172.21.15.101:0/3136026303 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f35efa4360 0x55f35f00b6a0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6a2c1ca94396c90 server_cookie=5df802a91311376 in_seq=0 out_seq=0 2023-07-18T03:24:43.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.590+0000 7f2c6b7fe700 10 client.28898 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:24:43.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.593+0000 7f2c6b7fe700 1 -- 172.21.15.101:0/3136026303 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f35ee9e0d0 con 0x55f35efa4360 2023-07-18T03:24:43.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.593+0000 7f2c6b7fe700 10 client.28898 handle_command_reply: tid=0 2023-07-18T03:24:43.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.593+0000 7f2ca63a8b80 1 client.28898 shutdown 2023-07-18T03:24:43.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.594+0000 7f2ca63a8b80 1 -- 172.21.15.101:0/3136026303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f35ef99470 msgr2=0x55f35efa9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:43.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.594+0000 7f2ca63a8b80 1 --2- 172.21.15.101:0/3136026303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f35ef99470 0x55f35efa9f90 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f2c84009f70 tx=0x7f2c8401bcf0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.594+0000 7f2ca63a8b80 1 -- 172.21.15.101:0/3136026303 shutdown_connections 2023-07-18T03:24:43.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.594+0000 7f2ca63a8b80 1 --2- 172.21.15.101:0/3136026303 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f35efa4360 0x55f35f00b6a0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.594+0000 7f2ca63a8b80 1 --2- 172.21.15.101:0/3136026303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f35ef99470 0x55f35efa9f90 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.594+0000 7f2ca63a8b80 1 --2- 172.21.15.101:0/3136026303 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f35ef86e20 0x55f35efa9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.594+0000 7f2ca63a8b80 1 --2- 172.21.15.101:0/3136026303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f35ef98b60 0x55f35efa9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.594+0000 7f2ca63a8b80 1 -- 172.21.15.101:0/3136026303 >> 172.21.15.101:0/3136026303 conn(0x55f35efaf840 msgr2=0x55f35ef9b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:43.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.594+0000 7f2c6b7fe700 0 client.28898 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:24:43.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.594+0000 7f2ca63a8b80 1 -- 172.21.15.101:0/3136026303 shutdown_connections 2023-07-18T03:24:43.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.594+0000 7f2ca63a8b80 1 -- 172.21.15.101:0/3136026303 wait complete. 2023-07-18T03:24:43.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.595+0000 7f2ca63a8b80 20 client.28898 trim_cache size 0 max 16384 2023-07-18T03:24:43.596 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:24:43.597 INFO:teuthology.orchestra.run.smithi101.stdout: "return_code": 0, 2023-07-18T03:24:43.597 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_tag": "0f2f19e0-0429-465c-935e-223eb09490c3", 2023-07-18T03:24:43.597 INFO:teuthology.orchestra.run.smithi101.stdout: "mode": "asynchronous" 2023-07-18T03:24:43.597 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:24:43.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.595+0000 7f2c929ff700 1 -- 172.21.15.101:0/3141975380 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2c740a12c0 msgr2=0x7f2c740a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:43.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.595+0000 7f2c929ff700 1 --2- 172.21.15.101:0/3141975380 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2c740a12c0 0x7f2c740a3770 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2c84010000 tx=0x7f2c8401f000 comp rx=0 tx=0).stop 2023-07-18T03:24:43.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.595+0000 7f2c929ff700 1 -- 172.21.15.101:0/3141975380 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c8c100490 msgr2=0x7f2c8c072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:43.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.595+0000 7f2c929ff700 1 --2- 172.21.15.101:0/3141975380 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c8c100490 0x7f2c8c072d50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000eb30 tx=0x7f2c8000ee40 comp rx=0 tx=0).stop 2023-07-18T03:24:43.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.596+0000 7f2c929ff700 1 -- 172.21.15.101:0/3141975380 shutdown_connections 2023-07-18T03:24:43.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.596+0000 7f2c929ff700 1 --2- 172.21.15.101:0/3141975380 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c8c073290 0x7f2c8c1a40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.596+0000 7f2c929ff700 1 --2- 172.21.15.101:0/3141975380 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2c8c0ffae0 0x7f2c8c072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.596+0000 7f2c929ff700 1 --2- 172.21.15.101:0/3141975380 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2c740a12c0 0x7f2c740a3770 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.596+0000 7f2c929ff700 1 --2- 172.21.15.101:0/3141975380 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c8c100490 0x7f2c8c072d50 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.596+0000 7f2c929ff700 1 -- 172.21.15.101:0/3141975380 >> 172.21.15.101:0/3141975380 conn(0x7f2c8c0fb5e0 msgr2=0x7f2c8c0fbde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:43.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.596+0000 7f2c929ff700 1 -- 172.21.15.101:0/3141975380 shutdown_connections 2023-07-18T03:24:43.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.596+0000 7f2c929ff700 1 -- 172.21.15.101:0/3141975380 wait complete. 2023-07-18T03:24:43.609 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:43 smithi101 ceph-mon[126379]: pgmap v539: 129 pgs: 129 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 16 MiB/s wr, 695 op/s 2023-07-18T03:24:43.611 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:24:43.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:43 smithi179 ceph-mon[129229]: pgmap v539: 129 pgs: 129 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 16 MiB/s wr, 695 op/s 2023-07-18T03:24:43.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:43 smithi155 ceph-mon[128885]: pgmap v539: 129 pgs: 129 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 16 MiB/s wr, 695 op/s 2023-07-18T03:24:43.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.911+0000 7f8c507ff700 1 -- 172.21.15.101:0/2992004754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c4806d3a0 msgr2=0x7f8c4811b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:43.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.911+0000 7f8c507ff700 1 --2- 172.21.15.101:0/2992004754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c4806d3a0 0x7f8c4811b470 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8c4000b0a0 tx=0x7f8c4000b3b0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.911+0000 7f8c507ff700 1 -- 172.21.15.101:0/2992004754 shutdown_connections 2023-07-18T03:24:43.907 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.911+0000 7f8c507ff700 1 --2- 172.21.15.101:0/2992004754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c4806d3a0 0x7f8c4811b470 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.911+0000 7f8c507ff700 1 --2- 172.21.15.101:0/2992004754 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c4810e280 0x7f8c4810e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.911+0000 7f8c507ff700 1 --2- 172.21.15.101:0/2992004754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c4806c9a0 0x7f8c4806ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.911+0000 7f8c507ff700 1 -- 172.21.15.101:0/2992004754 >> 172.21.15.101:0/2992004754 conn(0x7f8c4806bf70 msgr2=0x7f8c48070ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:43.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.913+0000 7f8c507ff700 1 -- 172.21.15.101:0/2992004754 shutdown_connections 2023-07-18T03:24:43.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.913+0000 7f8c507ff700 1 -- 172.21.15.101:0/2992004754 wait complete. 2023-07-18T03:24:43.910 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.914+0000 7f8c507ff700 1 Processor -- start 2023-07-18T03:24:43.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.915+0000 7f8c507ff700 1 -- start start 2023-07-18T03:24:43.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.915+0000 7f8c507ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c4806c9a0 0x7f8c481aaac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:43.911 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.915+0000 7f8c507ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c4806d3a0 0x7f8c481ab000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:43.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.915+0000 7f8c507ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c4810e280 0x7f8c481a4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:43.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.915+0000 7f8c507ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c481a5270 con 0x7f8c4806d3a0 2023-07-18T03:24:43.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.915+0000 7f8c507ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c481a53b0 con 0x7f8c4806c9a0 2023-07-18T03:24:43.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.915+0000 7f8c507ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c481a5570 con 0x7f8c4810e280 2023-07-18T03:24:43.912 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.916+0000 7f8c4e59b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c4806c9a0 0x7f8c481aaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:43.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.916+0000 7f8c4e59b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c4806c9a0 0x7f8c481aaac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60256/0 (socket says 172.21.15.101:60256) 2023-07-18T03:24:43.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.916+0000 7f8c4e59b700 1 -- 172.21.15.101:0/4055362174 learned_addr learned my addr 172.21.15.101:0/4055362174 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:43.913 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.918+0000 7f8c4ed9c700 1 --2- 172.21.15.101:0/4055362174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c4810e280 0x7f8c481a4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:43.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.918+0000 7f8c4e59b700 1 -- 172.21.15.101:0/4055362174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c4810e280 msgr2=0x7f8c481a4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:43.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.918+0000 7f8c4e59b700 1 --2- 172.21.15.101:0/4055362174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c4810e280 0x7f8c481a4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.918+0000 7f8c4e59b700 1 -- 172.21.15.101:0/4055362174 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c4806d3a0 msgr2=0x7f8c481ab000 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-18T03:24:43.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.918+0000 7f8c4e59b700 1 --2- 172.21.15.101:0/4055362174 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c4806d3a0 0x7f8c481ab000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.914 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.918+0000 7f8c4e59b700 1 -- 172.21.15.101:0/4055362174 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c40009d00 con 0x7f8c4806c9a0 2023-07-18T03:24:43.915 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.919+0000 7f8c4e59b700 1 --2- 172.21.15.101:0/4055362174 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c4806c9a0 0x7f8c481aaac0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8c3800b530 tx=0x7f8c3800b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:43.922 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.924+0000 7f8c3f7fe700 1 -- 172.21.15.101:0/4055362174 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c38011830 con 0x7f8c4806c9a0 2023-07-18T03:24:43.922 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.924+0000 7f8c3f7fe700 1 -- 172.21.15.101:0/4055362174 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c38011e50 con 0x7f8c4806c9a0 2023-07-18T03:24:43.923 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.924+0000 7f8c3f7fe700 1 -- 172.21.15.101:0/4055362174 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3800f2e0 con 0x7f8c4806c9a0 2023-07-18T03:24:43.923 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.927+0000 7f8c507ff700 1 -- 172.21.15.101:0/4055362174 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c481a5850 con 0x7f8c4806c9a0 2023-07-18T03:24:43.923 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.927+0000 7f8c507ff700 1 -- 172.21.15.101:0/4055362174 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c4810ee20 con 0x7f8c4806c9a0 2023-07-18T03:24:43.929 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.933+0000 7f8c640a9b80 1 Processor -- start 2023-07-18T03:24:43.930 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.934+0000 7f8c640a9b80 1 -- start start 2023-07-18T03:24:43.930 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.934+0000 7f8c640a9b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ed9f1e2d0 0x560ed9f1e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:43.930 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.934+0000 7f8c640a9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ed9f1ebe0 0x560ed9f29590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:43.930 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.934+0000 7f8c640a9b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ed9f29ad0 0x560ed9f2beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:43.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.934+0000 7f8c640a9b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ed9e24cb0 con 0x560ed9f29ad0 2023-07-18T03:24:43.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.934+0000 7f8c640a9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ed9e77a40 con 0x560ed9f1ebe0 2023-07-18T03:24:43.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.934+0000 7f8c640a9b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ed9e70ed0 con 0x560ed9f1e2d0 2023-07-18T03:24:43.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.935+0000 7f8c4e59b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ed9f1ebe0 0x560ed9f29590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:43.931 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.935+0000 7f8c4ed9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ed9f1e2d0 0x560ed9f1e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:43.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.935+0000 7f8c4e59b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ed9f1ebe0 0x560ed9f29590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60260/0 (socket says 172.21.15.101:60260) 2023-07-18T03:24:43.932 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.935+0000 7f8c4e59b700 1 -- 172.21.15.101:0/2708690164 learned_addr learned my addr 172.21.15.101:0/2708690164 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:43.935 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.936+0000 7f8c4dd9a700 1 --2- 172.21.15.101:0/2708690164 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ed9f29ad0 0x560ed9f2beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:43.935 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.936+0000 7f8c4ed9c700 1 -- 172.21.15.101:0/2708690164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ed9f1ebe0 msgr2=0x560ed9f29590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:43.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.936+0000 7f8c4ed9c700 1 --2- 172.21.15.101:0/2708690164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ed9f1ebe0 0x560ed9f29590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.936+0000 7f8c4ed9c700 1 -- 172.21.15.101:0/2708690164 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ed9f29ad0 msgr2=0x560ed9f2beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:43.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.936+0000 7f8c4ed9c700 1 --2- 172.21.15.101:0/2708690164 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ed9f29ad0 0x560ed9f2beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.936+0000 7f8c4ed9c700 1 -- 172.21.15.101:0/2708690164 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ed9c9ede0 con 0x560ed9f1e2d0 2023-07-18T03:24:43.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.938+0000 7f8c4ed9c700 1 --2- 172.21.15.101:0/2708690164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ed9f1e2d0 0x560ed9f1e6a0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8c481ab750 tx=0x7f8c40008d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=495be7e2bc3b40d1 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:43.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.939+0000 7f8c3cff9700 1 -- 172.21.15.101:0/2708690164 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4000e050 con 0x560ed9f1e2d0 2023-07-18T03:24:43.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.939+0000 7f8c3cff9700 1 -- 172.21.15.101:0/2708690164 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c400092d0 con 0x560ed9f1e2d0 2023-07-18T03:24:43.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.939+0000 7f8c3cff9700 1 -- 172.21.15.101:0/2708690164 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c400052a0 con 0x560ed9f1e2d0 2023-07-18T03:24:43.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.940+0000 7f8c3f7fe700 1 -- 172.21.15.101:0/4055362174 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f8c380105a0 con 0x7f8c4806c9a0 2023-07-18T03:24:43.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.940+0000 7f8c640a9b80 1 -- 172.21.15.101:0/2708690164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ed9f1e2d0 msgr2=0x560ed9f1e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:43.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.940+0000 7f8c640a9b80 1 --2- 172.21.15.101:0/2708690164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ed9f1e2d0 0x560ed9f1e6a0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8c481ab750 tx=0x7f8c40008d60 comp rx=0 tx=0).stop 2023-07-18T03:24:43.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.941+0000 7f8c640a9b80 1 -- 172.21.15.101:0/2708690164 shutdown_connections 2023-07-18T03:24:43.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.941+0000 7f8c640a9b80 1 --2- 172.21.15.101:0/2708690164 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ed9f29ad0 0x560ed9f2beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.941+0000 7f8c640a9b80 1 --2- 172.21.15.101:0/2708690164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ed9f1e2d0 0x560ed9f1e6a0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.941+0000 7f8c640a9b80 1 --2- 172.21.15.101:0/2708690164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ed9f1ebe0 0x560ed9f29590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.941+0000 7f8c640a9b80 1 -- 172.21.15.101:0/2708690164 >> 172.21.15.101:0/2708690164 conn(0x560ed9f14da0 msgr2=0x560ed9f1a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:43.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.941+0000 7f8c640a9b80 1 -- 172.21.15.101:0/2708690164 shutdown_connections 2023-07-18T03:24:43.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.941+0000 7f8c640a9b80 1 -- 172.21.15.101:0/2708690164 wait complete. 2023-07-18T03:24:43.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.942+0000 7f8c640a9b80 1 -- start start 2023-07-18T03:24:43.939 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.942+0000 7f8c640a9b80 1 Processor -- start 2023-07-18T03:24:43.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.952+0000 7f8c640a9b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ed9f1e2d0 0x560ed9f29510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:43.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.952+0000 7f8c640a9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ed9f1ebe0 0x560ed9f26720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:43.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.952+0000 7f8c640a9b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ed9f29ad0 0x560ed9f26e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:43.948 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.952+0000 7f8c640a9b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ed9e10ac0 con 0x560ed9f29ad0 2023-07-18T03:24:43.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.952+0000 7f8c640a9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ed9e107c0 con 0x560ed9f1ebe0 2023-07-18T03:24:43.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.952+0000 7f8c640a9b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ed9cf4920 con 0x560ed9f1e2d0 2023-07-18T03:24:43.949 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.952+0000 7f8c640a9b80 10 client.0 _ll_register_callbacks cb 0x560ed9e0e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:24:43.950 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.953+0000 7f8c4ed9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ed9f1e2d0 0x560ed9f29510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:43.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.953+0000 7f8c4ed9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ed9f1e2d0 0x560ed9f29510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:40854/0 (socket says 172.21.15.101:40854) 2023-07-18T03:24:43.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.953+0000 7f8c4ed9c700 1 -- 172.21.15.101:0/3831280165 learned_addr learned my addr 172.21.15.101:0/3831280165 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:43.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.953+0000 7f8c4e59b700 1 --2- 172.21.15.101:0/3831280165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ed9f1ebe0 0x560ed9f26720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:43.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.953+0000 7f8c4dd9a700 1 --2- 172.21.15.101:0/3831280165 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ed9f29ad0 0x560ed9f26e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:43.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.954+0000 7f8c3f7fe700 1 --2- 172.21.15.101:0/4055362174 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f8c340a1360 0x7f8c340a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:43.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.954+0000 7f8c4e59b700 1 -- 172.21.15.101:0/3831280165 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ed9f1e2d0 msgr2=0x560ed9f29510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:43.951 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.954+0000 7f8c4e59b700 1 --2- 172.21.15.101:0/3831280165 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ed9f1e2d0 0x560ed9f29510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.954+0000 7f8c4e59b700 1 -- 172.21.15.101:0/3831280165 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ed9f29ad0 msgr2=0x560ed9f26e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:43.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.954+0000 7f8c4e59b700 1 --2- 172.21.15.101:0/3831280165 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ed9f29ad0 0x560ed9f26e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.954+0000 7f8c4e59b700 1 -- 172.21.15.101:0/3831280165 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c4811bfa0 con 0x560ed9f1ebe0 2023-07-18T03:24:43.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.954+0000 7f8c4ed9c700 1 --2- 172.21.15.101:0/3831280165 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ed9f1e2d0 0x560ed9f29510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:24:43.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.954+0000 7f8c3f7fe700 1 -- 172.21.15.101:0/4055362174 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f8c380d21d0 con 0x7f8c4806c9a0 2023-07-18T03:24:43.952 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.955+0000 7f8c4ed9c700 1 --2- 172.21.15.101:0/4055362174 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f8c340a1360 0x7f8c340a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:43.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.956+0000 7f8c4ed9c700 1 --2- 172.21.15.101:0/4055362174 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f8c340a1360 0x7f8c340a3810 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8c40008fc0 tx=0x7f8c40027040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:43.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.957+0000 7f8c4e59b700 1 --2- 172.21.15.101:0/3831280165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ed9f1ebe0 0x560ed9f26720 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f8c380cf680 tx=0x7f8c3806a3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca9c538e0ff5315d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:43.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.957+0000 7f8c31ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:24:43.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.960+0000 7f8c31ffb700 1 -- 172.21.15.101:0/3831280165 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c38009380 con 0x560ed9f1ebe0 2023-07-18T03:24:43.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.960+0000 7f8c31ffb700 1 -- 172.21.15.101:0/3831280165 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f8c38017550 con 0x560ed9f1ebe0 2023-07-18T03:24:43.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.961+0000 7f8c640a9b80 1 -- 172.21.15.101:0/3831280165 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560ed9dca4f0 con 0x560ed9f1ebe0 2023-07-18T03:24:43.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.962+0000 7f8c31ffb700 1 -- 172.21.15.101:0/3831280165 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c3809edd0 con 0x560ed9f1ebe0 2023-07-18T03:24:43.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.962+0000 7f8c640a9b80 10 client.28910 fetch_fsmap learned FSMap version 42 2023-07-18T03:24:43.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.962+0000 7f8c640a9b80 1 -- 172.21.15.101:0/3831280165 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x560ed9c87c60 con 0x560ed9f1ebe0 2023-07-18T03:24:43.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.963+0000 7f8c31ffb700 1 -- 172.21.15.101:0/3831280165 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560ed9dca4f0 con 0x560ed9f1ebe0 2023-07-18T03:24:43.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.963+0000 7f8c31ffb700 1 -- 172.21.15.101:0/3831280165 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 42) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f8c3809ade0 con 0x560ed9f1ebe0 2023-07-18T03:24:43.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.966+0000 7f8c640a9b80 10 client.28910 fetch_fsmap finished waiting for FSMap version 42 2023-07-18T03:24:43.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.966+0000 7f8c640a9b80 10 client.28910 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:24:43.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.966+0000 7f8c640a9b80 20 client.28910 populate_metadata read hostname 'smithi101' 2023-07-18T03:24:43.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.966+0000 7f8c640a9b80 1 --2- 172.21.15.101:0/3831280165 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x560ed9f392b0 0x560ed9f3b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:43.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.966+0000 7f8c640a9b80 4 client.28910 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:24:43.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.966+0000 7f8c640a9b80 1 -- 172.21.15.101:0/3831280165 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560ed9e69eb0 con 0x560ed9f392b0 2023-07-18T03:24:43.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.966+0000 7f8c4dd9a700 1 --2- 172.21.15.101:0/3831280165 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x560ed9f392b0 0x560ed9f3b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:43.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.969+0000 7f8c4dd9a700 1 --2- 172.21.15.101:0/3831280165 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x560ed9f392b0 0x560ed9f3b6c0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d07dfb773c1bfbbf server_cookie=7779d6e3e2308cb4 in_seq=0 out_seq=0 2023-07-18T03:24:43.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.969+0000 7f8c31ffb700 10 client.28910 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:24:43.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.972+0000 7f8c31ffb700 1 -- 172.21.15.101:0/3831280165 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560ed9e69eb0 con 0x560ed9f392b0 2023-07-18T03:24:43.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.972+0000 7f8c31ffb700 10 client.28910 handle_command_reply: tid=0 2023-07-18T03:24:43.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.973+0000 7f8c640a9b80 1 client.28910 shutdown 2023-07-18T03:24:43.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.976+0000 7f8c640a9b80 1 -- 172.21.15.101:0/3831280165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ed9f1ebe0 msgr2=0x560ed9f26720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:43.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.976+0000 7f8c640a9b80 1 --2- 172.21.15.101:0/3831280165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ed9f1ebe0 0x560ed9f26720 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f8c380cf680 tx=0x7f8c3806a3c0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.976+0000 7f8c640a9b80 1 -- 172.21.15.101:0/3831280165 shutdown_connections 2023-07-18T03:24:43.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.976+0000 7f8c640a9b80 1 --2- 172.21.15.101:0/3831280165 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x560ed9f392b0 0x560ed9f3b6c0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.977+0000 7f8c640a9b80 1 --2- 172.21.15.101:0/3831280165 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ed9f29ad0 0x560ed9f26e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.977+0000 7f8c640a9b80 1 --2- 172.21.15.101:0/3831280165 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ed9f1e2d0 0x560ed9f29510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.977+0000 7f8c640a9b80 1 --2- 172.21.15.101:0/3831280165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ed9f1ebe0 0x560ed9f26720 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:43.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.977+0000 7f8c640a9b80 1 -- 172.21.15.101:0/3831280165 >> 172.21.15.101:0/3831280165 conn(0x560ed9f14da0 msgr2=0x560ed9f1a230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:43.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.977+0000 7f8c31ffb700 0 client.28910 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:24:43.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.980+0000 7f8c640a9b80 1 -- 172.21.15.101:0/3831280165 shutdown_connections 2023-07-18T03:24:43.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.981+0000 7f8c640a9b80 1 -- 172.21.15.101:0/3831280165 wait complete. 2023-07-18T03:24:43.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:43.985+0000 7f8c640a9b80 20 client.28910 trim_cache size 0 max 16384 2023-07-18T03:24:44.002 INFO:tasks.workunit.client.0.smithi101.stdout: 10 136434 28.42 MB/sec execute 128 sec latency 119.974 ms 2023-07-18T03:24:44.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.014+0000 7f8c640a9b80 1 Processor -- start 2023-07-18T03:24:44.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.016+0000 7f8c640a9b80 1 -- start start 2023-07-18T03:24:44.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.017+0000 7f8c640a9b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ed9f1e2d0 0x560ed9f35700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:44.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.017+0000 7f8c640a9b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ed9f1ebe0 0x560ed9f29040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:44.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.017+0000 7f8c640a9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ed9f29ad0 0x560ed9f21a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:44.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.017+0000 7f8c640a9b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ed9f1bd00 con 0x560ed9f1e2d0 2023-07-18T03:24:44.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.017+0000 7f8c640a9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ed9f201a0 con 0x560ed9f29ad0 2023-07-18T03:24:44.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.017+0000 7f8c640a9b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ed9f2beb0 con 0x560ed9f1ebe0 2023-07-18T03:24:44.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.017+0000 7f8c4ed9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ed9f29ad0 0x560ed9f21a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:44.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.017+0000 7f8c4ed9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ed9f29ad0 0x560ed9f21a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60278/0 (socket says 172.21.15.101:60278) 2023-07-18T03:24:44.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.017+0000 7f8c4ed9c700 1 -- 172.21.15.101:0/1565840007 learned_addr learned my addr 172.21.15.101:0/1565840007 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:44.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.018+0000 7f8c4dd9a700 1 --2- 172.21.15.101:0/1565840007 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ed9f1ebe0 0x560ed9f29040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:44.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.018+0000 7f8c4ed9c700 1 -- 172.21.15.101:0/1565840007 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ed9f1ebe0 msgr2=0x560ed9f29040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:44.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.018+0000 7f8c4ed9c700 1 --2- 172.21.15.101:0/1565840007 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ed9f1ebe0 0x560ed9f29040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:44.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.018+0000 7f8c4ed9c700 1 -- 172.21.15.101:0/1565840007 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ed9f1e2d0 msgr2=0x560ed9f35700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:44.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.018+0000 7f8c4ed9c700 1 --2- 172.21.15.101:0/1565840007 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ed9f1e2d0 0x560ed9f35700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:44.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.018+0000 7f8c4ed9c700 1 -- 172.21.15.101:0/1565840007 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ed9cc5250 con 0x560ed9f29ad0 2023-07-18T03:24:44.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.018+0000 7f8c4dd9a700 1 --2- 172.21.15.101:0/1565840007 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ed9f1ebe0 0x560ed9f29040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:24:44.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.019+0000 7f8c4ed9c700 1 --2- 172.21.15.101:0/1565840007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ed9f29ad0 0x560ed9f21a30 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f8c4000d010 tx=0x7f8c4002a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0b0c7e94969ac8d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:44.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.019+0000 7f8c3d7fa700 1 -- 172.21.15.101:0/1565840007 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c40032070 con 0x560ed9f29ad0 2023-07-18T03:24:44.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.019+0000 7f8c3d7fa700 1 -- 172.21.15.101:0/1565840007 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c400078b0 con 0x560ed9f29ad0 2023-07-18T03:24:44.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.019+0000 7f8c3d7fa700 1 -- 172.21.15.101:0/1565840007 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4001b8b0 con 0x560ed9f29ad0 2023-07-18T03:24:44.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.020+0000 7f8c640a9b80 1 -- 172.21.15.101:0/1565840007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ed9f29ad0 msgr2=0x560ed9f21a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:44.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.020+0000 7f8c640a9b80 1 --2- 172.21.15.101:0/1565840007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ed9f29ad0 0x560ed9f21a30 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f8c4000d010 tx=0x7f8c4002a000 comp rx=0 tx=0).stop 2023-07-18T03:24:44.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.020+0000 7f8c640a9b80 1 -- 172.21.15.101:0/1565840007 shutdown_connections 2023-07-18T03:24:44.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.020+0000 7f8c640a9b80 1 --2- 172.21.15.101:0/1565840007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ed9f29ad0 0x560ed9f21a30 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:44.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.020+0000 7f8c640a9b80 1 --2- 172.21.15.101:0/1565840007 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ed9f1e2d0 0x560ed9f35700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:44.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.020+0000 7f8c640a9b80 1 --2- 172.21.15.101:0/1565840007 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ed9f1ebe0 0x560ed9f29040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:44.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.021+0000 7f8c640a9b80 1 -- 172.21.15.101:0/1565840007 >> 172.21.15.101:0/1565840007 conn(0x560ed9f14da0 msgr2=0x560ed9f40020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:44.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.021+0000 7f8c640a9b80 1 -- 172.21.15.101:0/1565840007 shutdown_connections 2023-07-18T03:24:44.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.021+0000 7f8c640a9b80 1 -- 172.21.15.101:0/1565840007 wait complete. 2023-07-18T03:24:44.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.021+0000 7f8c640a9b80 1 -- start start 2023-07-18T03:24:44.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.022+0000 7f8c640a9b80 1 Processor -- start 2023-07-18T03:24:44.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.022+0000 7f8c640a9b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ed9f1e2d0 0x560ed9f2ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:44.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.022+0000 7f8c640a9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ed9f1ebe0 0x560ed9f30510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:44.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.023+0000 7f8c640a9b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ed9f29ad0 0x560ed9f2e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:44.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.023+0000 7f8c640a9b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ed9f2eb00 con 0x560ed9f29ad0 2023-07-18T03:24:44.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.023+0000 7f8c640a9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ed9f2ec70 con 0x560ed9f1ebe0 2023-07-18T03:24:44.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.023+0000 7f8c640a9b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ed9f2ee30 con 0x560ed9f1e2d0 2023-07-18T03:24:44.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.023+0000 7f8c640a9b80 10 client.0 _ll_register_callbacks cb 0x7f8c3809b340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:24:44.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.023+0000 7f8c4dd9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ed9f1ebe0 0x560ed9f30510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:44.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.023+0000 7f8c4dd9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ed9f1ebe0 0x560ed9f30510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60288/0 (socket says 172.21.15.101:60288) 2023-07-18T03:24:44.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.023+0000 7f8c4dd9a700 1 -- 172.21.15.101:0/2210772265 learned_addr learned my addr 172.21.15.101:0/2210772265 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:24:44.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.023+0000 7f8c4e59b700 1 --2- 172.21.15.101:0/2210772265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ed9f1e2d0 0x560ed9f2ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:44.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.023+0000 7f8c4dd9a700 1 -- 172.21.15.101:0/2210772265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ed9f1e2d0 msgr2=0x560ed9f2ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:44.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.024+0000 7f8c4dd9a700 1 --2- 172.21.15.101:0/2210772265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ed9f1e2d0 0x560ed9f2ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:44.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.024+0000 7f8c4dd9a700 1 -- 172.21.15.101:0/2210772265 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ed9f29ad0 msgr2=0x560ed9f2e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:24:44.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.024+0000 7f8c4dd9a700 1 --2- 172.21.15.101:0/2210772265 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ed9f29ad0 0x560ed9f2e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:44.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.024+0000 7f8c4dd9a700 1 -- 172.21.15.101:0/2210772265 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560ed9c9ede0 con 0x560ed9f1ebe0 2023-07-18T03:24:44.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.024+0000 7f8c4e59b700 1 --2- 172.21.15.101:0/2210772265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ed9f1e2d0 0x560ed9f2ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:24:44.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.024+0000 7f8c4dd9a700 1 --2- 172.21.15.101:0/2210772265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ed9f1ebe0 0x560ed9f30510 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8c440047b0 tx=0x7f8c4400bd00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd22ca211980b372 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:44.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.024+0000 7f8c31ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:24:44.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.025+0000 7f8c31ffb700 1 -- 172.21.15.101:0/2210772265 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c44018480 con 0x560ed9f1ebe0 2023-07-18T03:24:44.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.025+0000 7f8c31ffb700 1 -- 172.21.15.101:0/2210772265 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f8c44016430 con 0x560ed9f1ebe0 2023-07-18T03:24:44.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.025+0000 7f8c640a9b80 1 -- 172.21.15.101:0/2210772265 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c400092d0 con 0x560ed9f1ebe0 2023-07-18T03:24:44.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.025+0000 7f8c31ffb700 1 -- 172.21.15.101:0/2210772265 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c440185e0 con 0x560ed9f1ebe0 2023-07-18T03:24:44.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.025+0000 7f8c640a9b80 10 client.28916 fetch_fsmap learned FSMap version 42 2023-07-18T03:24:44.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.025+0000 7f8c640a9b80 1 -- 172.21.15.101:0/2210772265 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x560ed9c9d2b0 con 0x560ed9f1ebe0 2023-07-18T03:24:44.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.026+0000 7f8c31ffb700 1 -- 172.21.15.101:0/2210772265 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c400092d0 con 0x560ed9f1ebe0 2023-07-18T03:24:44.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.026+0000 7f8c31ffb700 1 -- 172.21.15.101:0/2210772265 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 42) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f8c4402b6d0 con 0x560ed9f1ebe0 2023-07-18T03:24:44.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.026+0000 7f8c640a9b80 10 client.28916 fetch_fsmap finished waiting for FSMap version 42 2023-07-18T03:24:44.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.026+0000 7f8c640a9b80 10 client.28916 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:24:44.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.026+0000 7f8c640a9b80 20 client.28916 populate_metadata read hostname 'smithi101' 2023-07-18T03:24:44.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.026+0000 7f8c640a9b80 1 --2- 172.21.15.101:0/2210772265 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x560ed9f8e390 0x560ed9f907a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:44.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.026+0000 7f8c640a9b80 4 client.28916 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:24:44.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.026+0000 7f8c640a9b80 1 -- 172.21.15.101:0/2210772265 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560ed9e2c050 con 0x560ed9f8e390 2023-07-18T03:24:44.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.028+0000 7f8c4e59b700 1 --2- 172.21.15.101:0/2210772265 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x560ed9f8e390 0x560ed9f907a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:44.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.029+0000 7f8c31ffb700 10 client.28916 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:24:44.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.029+0000 7f8c4e59b700 1 --2- 172.21.15.101:0/2210772265 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x560ed9f8e390 0x560ed9f907a0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dd6975faf7fab13 server_cookie=cbb65a1132d01d5c in_seq=0 out_seq=0 2023-07-18T03:24:44.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.030+0000 7f8c31ffb700 1 -- 172.21.15.101:0/2210772265 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560ed9e2c050 con 0x560ed9f8e390 2023-07-18T03:24:44.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.030+0000 7f8c31ffb700 10 client.28916 handle_command_reply: tid=0 2023-07-18T03:24:44.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.030+0000 7f8c640a9b80 1 client.28916 shutdown 2023-07-18T03:24:44.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.030+0000 7f8c640a9b80 1 -- 172.21.15.101:0/2210772265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ed9f1ebe0 msgr2=0x560ed9f30510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:44.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.030+0000 7f8c640a9b80 1 --2- 172.21.15.101:0/2210772265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ed9f1ebe0 0x560ed9f30510 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8c440047b0 tx=0x7f8c4400bd00 comp rx=0 tx=0).stop 2023-07-18T03:24:44.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.030+0000 7f8c640a9b80 1 -- 172.21.15.101:0/2210772265 shutdown_connections 2023-07-18T03:24:44.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.030+0000 7f8c640a9b80 1 --2- 172.21.15.101:0/2210772265 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x560ed9f8e390 0x560ed9f907a0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:44.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.030+0000 7f8c640a9b80 1 --2- 172.21.15.101:0/2210772265 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ed9f29ad0 0x560ed9f2e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:44.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.030+0000 7f8c640a9b80 1 --2- 172.21.15.101:0/2210772265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ed9f1e2d0 0x560ed9f2ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:44.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.030+0000 7f8c640a9b80 1 --2- 172.21.15.101:0/2210772265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560ed9f1ebe0 0x560ed9f30510 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:44.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.030+0000 7f8c31ffb700 0 client.28916 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:24:44.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.030+0000 7f8c640a9b80 1 -- 172.21.15.101:0/2210772265 >> 172.21.15.101:0/2210772265 conn(0x560ed9f14da0 msgr2=0x560ed9f35700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:44.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.031+0000 7f8c640a9b80 1 -- 172.21.15.101:0/2210772265 shutdown_connections 2023-07-18T03:24:44.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.031+0000 7f8c640a9b80 1 -- 172.21.15.101:0/2210772265 wait complete. 2023-07-18T03:24:44.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.031+0000 7f8c640a9b80 20 client.28916 trim_cache size 0 max 16384 2023-07-18T03:24:44.030 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:24:44.031 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:24:44.031 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:24:44.031 INFO:teuthology.orchestra.run.smithi101.stdout: "0f2f19e0-0429-465c-935e-223eb09490c3": { 2023-07-18T03:24:44.031 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:24:44.031 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "0f2f19e0-0429-465c-935e-223eb09490c3", 2023-07-18T03:24:44.031 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:24:44.032 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:24:44.032 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:24:44.032 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:24:44.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.031+0000 7f8c507ff700 1 -- 172.21.15.101:0/4055362174 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f8c340a1360 msgr2=0x7f8c340a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:44.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.031+0000 7f8c507ff700 1 --2- 172.21.15.101:0/4055362174 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f8c340a1360 0x7f8c340a3810 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8c40008fc0 tx=0x7f8c40027040 comp rx=0 tx=0).stop 2023-07-18T03:24:44.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.031+0000 7f8c507ff700 1 -- 172.21.15.101:0/4055362174 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c4806c9a0 msgr2=0x7f8c481aaac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:44.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.031+0000 7f8c507ff700 1 --2- 172.21.15.101:0/4055362174 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c4806c9a0 0x7f8c481aaac0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8c3800b530 tx=0x7f8c3800b8f0 comp rx=0 tx=0).stop 2023-07-18T03:24:44.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.031+0000 7f8c507ff700 1 -- 172.21.15.101:0/4055362174 shutdown_connections 2023-07-18T03:24:44.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.031+0000 7f8c507ff700 1 --2- 172.21.15.101:0/4055362174 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f8c340a1360 0x7f8c340a3810 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:44.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.031+0000 7f8c507ff700 1 --2- 172.21.15.101:0/4055362174 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c4806c9a0 0x7f8c481aaac0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:44.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.031+0000 7f8c507ff700 1 --2- 172.21.15.101:0/4055362174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c4810e280 0x7f8c481a4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:44.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.031+0000 7f8c507ff700 1 --2- 172.21.15.101:0/4055362174 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c4806d3a0 0x7f8c481ab000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:44.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.031+0000 7f8c507ff700 1 -- 172.21.15.101:0/4055362174 >> 172.21.15.101:0/4055362174 conn(0x7f8c4806bf70 msgr2=0x7f8c48070880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:44.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.032+0000 7f8c507ff700 1 -- 172.21.15.101:0/4055362174 shutdown_connections 2023-07-18T03:24:44.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:24:44.032+0000 7f8c507ff700 1 -- 172.21.15.101:0/4055362174 wait complete. 2023-07-18T03:24:44.047 INFO:tasks.cephfs.filesystem:scrub status for tag:0f2f19e0-0429-465c-935e-223eb09490c3 - {'path': '/', 'tag': '0f2f19e0-0429-465c-935e-223eb09490c3', 'options': 'recursive,force'} 2023-07-18T03:24:44.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:44 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:24:44.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:44 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:24:44.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:44 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:24:45.002 INFO:tasks.workunit.client.0.smithi101.stdout: 10 137002 28.46 MB/sec execute 129 sec latency 134.755 ms 2023-07-18T03:24:45.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:45 smithi179 ceph-mon[129229]: scrub queued for path: / 2023-07-18T03:24:45.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:45 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:24:45.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:45 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:24:45.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:45 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:24:45.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:45 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:24:45.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:45 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:24:45.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:45 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:24:45.705 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:45 smithi179 ceph-mon[129229]: scrub summary: active paths [/] 2023-07-18T03:24:45.705 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:45 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:24:45.705 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:45 smithi179 ceph-mon[129229]: pgmap v540: 129 pgs: 129 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 12 MiB/s wr, 540 op/s 2023-07-18T03:24:45.705 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:45 smithi179 ceph-mon[129229]: scrub summary: idle+waiting paths [/] 2023-07-18T03:24:45.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:45 smithi101 ceph-mon[126379]: scrub queued for path: / 2023-07-18T03:24:45.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:45 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:24:45.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:45 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:24:45.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:45 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:24:45.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:45 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:24:45.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:45 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:24:45.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:45 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:24:45.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:45 smithi101 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-18T03:24:45.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:45 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:24:45.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:45 smithi101 ceph-mon[126379]: pgmap v540: 129 pgs: 129 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 12 MiB/s wr, 540 op/s 2023-07-18T03:24:45.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:45 smithi101 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-18T03:24:45.800 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:45 smithi155 ceph-mon[128885]: scrub queued for path: / 2023-07-18T03:24:45.801 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:45 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:24:45.801 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:45 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:24:45.801 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:45 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:24:45.801 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:45 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:24:45.801 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:45 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:24:45.801 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:45 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:24:45.801 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:45 smithi155 ceph-mon[128885]: scrub summary: active paths [/] 2023-07-18T03:24:45.802 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:45 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:24:45.802 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:45 smithi155 ceph-mon[128885]: pgmap v540: 129 pgs: 129 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 12 MiB/s wr, 540 op/s 2023-07-18T03:24:45.802 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:45 smithi155 ceph-mon[128885]: scrub summary: idle+waiting paths [/] 2023-07-18T03:24:46.001 INFO:tasks.workunit.client.0.smithi101.stdout: 10 137536 28.42 MB/sec execute 130 sec latency 112.640 ms 2023-07-18T03:24:47.002 INFO:tasks.workunit.client.0.smithi101.stdout: 10 138117 28.40 MB/sec execute 131 sec latency 91.282 ms 2023-07-18T03:24:47.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:47 smithi179 ceph-mon[129229]: pgmap v541: 129 pgs: 129 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 686 op/s 2023-07-18T03:24:47.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:47 smithi101 ceph-mon[126379]: pgmap v541: 129 pgs: 129 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 686 op/s 2023-07-18T03:24:47.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:47 smithi155 ceph-mon[128885]: pgmap v541: 129 pgs: 129 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 686 op/s 2023-07-18T03:24:48.002 INFO:tasks.workunit.client.0.smithi101.stdout: 10 138753 28.48 MB/sec execute 132 sec latency 83.931 ms 2023-07-18T03:24:49.002 INFO:tasks.workunit.client.0.smithi101.stdout: 10 139205 28.45 MB/sec execute 133 sec latency 124.423 ms 2023-07-18T03:24:49.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:49 smithi179 ceph-mon[129229]: pgmap v542: 129 pgs: 129 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 7.7 MiB/s wr, 477 op/s 2023-07-18T03:24:49.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:49 smithi101 ceph-mon[126379]: pgmap v542: 129 pgs: 129 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 7.7 MiB/s wr, 477 op/s 2023-07-18T03:24:49.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:49 smithi155 ceph-mon[128885]: pgmap v542: 129 pgs: 129 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 7.7 MiB/s wr, 477 op/s 2023-07-18T03:24:50.002 INFO:tasks.workunit.client.0.smithi101.stdout: 10 139815 28.48 MB/sec execute 134 sec latency 88.484 ms 2023-07-18T03:24:51.002 INFO:tasks.workunit.client.0.smithi101.stdout: 10 140356 28.49 MB/sec execute 135 sec latency 136.621 ms 2023-07-18T03:24:51.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:51 smithi179 ceph-mon[129229]: pgmap v543: 129 pgs: 129 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 7.0 MiB/s wr, 519 op/s 2023-07-18T03:24:51.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:51 smithi101 ceph-mon[126379]: pgmap v543: 129 pgs: 129 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 7.0 MiB/s wr, 519 op/s 2023-07-18T03:24:51.800 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:51 smithi155 ceph-mon[128885]: pgmap v543: 129 pgs: 129 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 7.0 MiB/s wr, 519 op/s 2023-07-18T03:24:52.002 INFO:tasks.workunit.client.0.smithi101.stdout: 10 141029 28.59 MB/sec execute 136 sec latency 108.418 ms 2023-07-18T03:24:53.004 INFO:tasks.workunit.client.0.smithi101.stdout: 10 141514 28.55 MB/sec execute 137 sec latency 100.841 ms 2023-07-18T03:24:53.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:53 smithi179 ceph-mon[129229]: pgmap v544: 129 pgs: 129 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.5 MiB/s wr, 507 op/s 2023-07-18T03:24:53.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:53 smithi101 ceph-mon[126379]: pgmap v544: 129 pgs: 129 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.5 MiB/s wr, 507 op/s 2023-07-18T03:24:53.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:53 smithi155 ceph-mon[128885]: pgmap v544: 129 pgs: 129 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.5 MiB/s wr, 507 op/s 2023-07-18T03:24:54.004 INFO:tasks.workunit.client.0.smithi101.stdout: 10 142047 28.54 MB/sec execute 138 sec latency 121.491 ms 2023-07-18T03:24:55.010 INFO:tasks.workunit.client.0.smithi101.stdout: 10 142708 28.56 MB/sec execute 139 sec latency 75.645 ms 2023-07-18T03:24:55.497 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:55 smithi101 ceph-mon[126379]: pgmap v545: 129 pgs: 129 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 5.2 MiB/s wr, 343 op/s 2023-07-18T03:24:55.550 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:55 smithi155 ceph-mon[128885]: pgmap v545: 129 pgs: 129 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 5.2 MiB/s wr, 343 op/s 2023-07-18T03:24:55.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:55 smithi179 ceph-mon[129229]: pgmap v545: 129 pgs: 129 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 5.2 MiB/s wr, 343 op/s 2023-07-18T03:24:56.010 INFO:tasks.workunit.client.0.smithi101.stdout: 10 142745 28.36 MB/sec execute 140 sec latency 894.458 ms 2023-07-18T03:24:57.013 INFO:tasks.workunit.client.0.smithi101.stdout: 10 143043 28.24 MB/sec execute 141 sec latency 729.759 ms 2023-07-18T03:24:57.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:57 smithi101 ceph-mon[126379]: pgmap v546: 129 pgs: 129 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.5 MiB/s wr, 513 op/s 2023-07-18T03:24:57.550 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:57 smithi155 ceph-mon[128885]: pgmap v546: 129 pgs: 129 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.5 MiB/s wr, 513 op/s 2023-07-18T03:24:57.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:57 smithi179 ceph-mon[129229]: pgmap v546: 129 pgs: 129 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.5 MiB/s wr, 513 op/s 2023-07-18T03:24:58.012 INFO:tasks.workunit.client.0.smithi101.stdout: 10 143424 28.20 MB/sec execute 142 sec latency 204.102 ms 2023-07-18T03:24:59.012 INFO:tasks.workunit.client.0.smithi101.stdout: 10 143901 28.24 MB/sec execute 143 sec latency 128.369 ms 2023-07-18T03:24:59.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:24:59 smithi101 ceph-mon[126379]: pgmap v547: 129 pgs: 129 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 7.7 MiB/s wr, 396 op/s 2023-07-18T03:24:59.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:24:59 smithi155 ceph-mon[128885]: pgmap v547: 129 pgs: 129 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 7.7 MiB/s wr, 396 op/s 2023-07-18T03:24:59.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:24:59 smithi179 ceph-mon[129229]: pgmap v547: 129 pgs: 129 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 7.7 MiB/s wr, 396 op/s 2023-07-18T03:25:00.013 INFO:tasks.workunit.client.0.smithi101.stdout: 10 144486 28.28 MB/sec execute 144 sec latency 105.431 ms 2023-07-18T03:25:01.013 INFO:tasks.workunit.client.0.smithi101.stdout: 10 144870 28.23 MB/sec execute 145 sec latency 340.022 ms 2023-07-18T03:25:01.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:25:01 smithi155 ceph-mon[128885]: pgmap v548: 129 pgs: 129 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 10 MiB/s wr, 508 op/s 2023-07-18T03:25:01.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:25:01 smithi179 ceph-mon[129229]: pgmap v548: 129 pgs: 129 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 10 MiB/s wr, 508 op/s 2023-07-18T03:25:01.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:25:01 smithi101 ceph-mon[126379]: pgmap v548: 129 pgs: 129 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 10 MiB/s wr, 508 op/s 2023-07-18T03:25:01.886 INFO:tasks.workunit.client.0.smithi101.stdout:[133382] write failed on handle 12784 (Resource temporarily unavailable) 2023-07-18T03:25:01.888 INFO:tasks.workunit.client.0.smithi101.stdout:Child failed with status 1 2023-07-18T03:25:01.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-18T03:25:02.301 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2023-07-18T03:25:02.301 DEBUG:teuthology.orchestra.run.smithi101:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-07-18T03:25:03.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:25:03 smithi155 ceph-mon[128885]: pgmap v549: 129 pgs: 129 active+clean; 1.7 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 13 MiB/s wr, 564 op/s 2023-07-18T03:25:03.583 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:25:03 smithi101 ceph-mon[126379]: pgmap v549: 129 pgs: 129 active+clean; 1.7 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 13 MiB/s wr, 564 op/s 2023-07-18T03:25:03.585 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/run_tasks.py", line 84, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi101 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=81cf785aa7781c4e08381c540f890cb9bc987ef9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2023-07-18T03:25:03.719 ERROR:teuthology.run_tasks: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=d487ea3f80bd4a6fbe3371da4256d74e Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/run_tasks.py", line 84, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi101 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=81cf785aa7781c4e08381c540f890cb9bc987ef9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2023-07-18T03:25:03.724 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2023-07-18T03:25:03.739 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2023-07-18T03:25:03.740 INFO:teuthology.task.exec:Executing custom commands... 2023-07-18T03:25:03.740 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi101.front.sepia.ceph.com 2023-07-18T03:25:03.740 DEBUG:teuthology.orchestra.run.smithi101:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2023-07-18T03:25:03.745 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:25:03 smithi179 ceph-mon[129229]: pgmap v549: 129 pgs: 129 active+clean; 1.7 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 13 MiB/s wr, 564 op/s 2023-07-18T03:25:03.822 DEBUG:teuthology.orchestra.run.smithi101:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2023-07-18T03:25:03.907 DEBUG:teuthology.orchestra.run.smithi101:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-07-18T03:25:04.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.107+0000 7fe10fdff700 1 -- 172.21.15.101:0/4016277832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1080728b0 msgr2=0x7fe108072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:25:04.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.107+0000 7fe10fdff700 1 --2- 172.21.15.101:0/4016277832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1080728b0 0x7fe108072c80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fe104009990 tx=0x7fe104009ca0 comp rx=0 tx=0).stop 2023-07-18T03:25:04.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.107+0000 7fe10fdff700 1 -- 172.21.15.101:0/4016277832 shutdown_connections 2023-07-18T03:25:04.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.107+0000 7fe10fdff700 1 --2- 172.21.15.101:0/4016277832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe10810c350 0x7fe10810e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:04.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.107+0000 7fe10fdff700 1 --2- 172.21.15.101:0/4016277832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1080728b0 0x7fe108072c80 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:04.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.107+0000 7fe10fdff700 1 --2- 172.21.15.101:0/4016277832 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe1080731c0 0x7fe10810be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:04.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.107+0000 7fe10fdff700 1 -- 172.21.15.101:0/4016277832 >> 172.21.15.101:0/4016277832 conn(0x7fe1080fb5f0 msgr2=0x7fe1080fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:25:04.108 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.107+0000 7fe10fdff700 1 -- 172.21.15.101:0/4016277832 shutdown_connections 2023-07-18T03:25:04.108 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.107+0000 7fe10fdff700 1 -- 172.21.15.101:0/4016277832 wait complete. 2023-07-18T03:25:04.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.108+0000 7fe10fdff700 1 Processor -- start 2023-07-18T03:25:04.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.108+0000 7fe10fdff700 1 -- start start 2023-07-18T03:25:04.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.108+0000 7fe10fdff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1080728b0 0x7fe108197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:04.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.108+0000 7fe10fdff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1080731c0 0x7fe108198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:04.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.108+0000 7fe10fdff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe10810c350 0x7fe10819c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:04.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.108+0000 7fe10fdff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10819d030 con 0x7fe10810c350 2023-07-18T03:25:04.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.108+0000 7fe10fdff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10819d1a0 con 0x7fe1080731c0 2023-07-18T03:25:04.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.108+0000 7fe10fdff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10819d360 con 0x7fe1080728b0 2023-07-18T03:25:04.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.109+0000 7fe10d39a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1080731c0 0x7fe108198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:04.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.109+0000 7fe10db9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1080728b0 0x7fe108197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:04.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.109+0000 7fe10d39a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1080731c0 0x7fe108198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:38448/0 (socket says 172.21.15.101:38448) 2023-07-18T03:25:04.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.109+0000 7fe10db9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1080728b0 0x7fe108197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:43226/0 (socket says 172.21.15.101:43226) 2023-07-18T03:25:04.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.109+0000 7fe10d39a700 1 -- 172.21.15.101:0/538391859 learned_addr learned my addr 172.21.15.101:0/538391859 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:25:04.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.109+0000 7fe10e39c700 1 --2- 172.21.15.101:0/538391859 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe10810c350 0x7fe10819c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:04.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.109+0000 7fe10d39a700 1 -- 172.21.15.101:0/538391859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1080728b0 msgr2=0x7fe108197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:25:04.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.109+0000 7fe10d39a700 1 --2- 172.21.15.101:0/538391859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1080728b0 0x7fe108197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:04.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.109+0000 7fe10d39a700 1 -- 172.21.15.101:0/538391859 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe10810c350 msgr2=0x7fe10819c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:25:04.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.109+0000 7fe10d39a700 1 --2- 172.21.15.101:0/538391859 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe10810c350 0x7fe10819c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:04.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.109+0000 7fe10d39a700 1 -- 172.21.15.101:0/538391859 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1040095f0 con 0x7fe1080731c0 2023-07-18T03:25:04.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.109+0000 7fe10d39a700 1 --2- 172.21.15.101:0/538391859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1080731c0 0x7fe108198380 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fe0f800e9a0 tx=0x7fe0f800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:25:04.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.109+0000 7fe0feffd700 1 -- 172.21.15.101:0/538391859 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f8009790 con 0x7fe1080731c0 2023-07-18T03:25:04.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.110+0000 7fe10fdff700 1 -- 172.21.15.101:0/538391859 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe10819d640 con 0x7fe1080731c0 2023-07-18T03:25:04.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.110+0000 7fe0feffd700 1 -- 172.21.15.101:0/538391859 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0f8010430 con 0x7fe1080731c0 2023-07-18T03:25:04.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.110+0000 7fe0feffd700 1 -- 172.21.15.101:0/538391859 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f8018680 con 0x7fe1080731c0 2023-07-18T03:25:04.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.110+0000 7fe10fdff700 1 -- 172.21.15.101:0/538391859 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1081a4e80 con 0x7fe1080731c0 2023-07-18T03:25:04.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.111+0000 7fe10fdff700 1 -- 172.21.15.101:0/538391859 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1080ff250 con 0x7fe1080731c0 2023-07-18T03:25:04.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.113+0000 7fe0feffd700 1 -- 172.21.15.101:0/538391859 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fe0f80105a0 con 0x7fe1080731c0 2023-07-18T03:25:04.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.114+0000 7fe0feffd700 1 --2- 172.21.15.101:0/538391859 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fe0f40a1340 0x7fe0f40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:04.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.114+0000 7fe0feffd700 1 -- 172.21.15.101:0/538391859 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fe0f8014070 con 0x7fe1080731c0 2023-07-18T03:25:04.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.114+0000 7fe10db9b700 1 --2- 172.21.15.101:0/538391859 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fe0f40a1340 0x7fe0f40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:04.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.115+0000 7fe10db9b700 1 --2- 172.21.15.101:0/538391859 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fe0f40a1340 0x7fe0f40a37f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe10400b2e0 tx=0x7fe104005fd0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:25:04.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.116+0000 7fe0feffd700 1 -- 172.21.15.101:0/538391859 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe0f8096270 con 0x7fe1080731c0 2023-07-18T03:25:04.350 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.353+0000 7fe10fdff700 1 -- 172.21.15.101:0/538391859 --> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7fe1080fd260 con 0x7fe0f40a1340 2023-07-18T03:25:04.361 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.364+0000 7fe0feffd700 1 -- 172.21.15.101:0/538391859 <== mgr.15486 v2:172.21.15.101:6800/486424816 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7fe1080fd260 con 0x7fe0f40a1340 2023-07-18T03:25:04.362 INFO:teuthology.orchestra.run.smithi101.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1M", "retention": {"h": 3, "M": 6}, "start": "2023-07-18T00:00:00", "created": "2023-07-18T03:07:34", "first": "2023-07-18T03:08:00", "last": "2023-07-18T03:25:01", "last_pruned": "2023-07-18T03:25:01", "created_count": 18, "pruned_count": 12, "active": true} 2023-07-18T03:25:04.364 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.367+0000 7fe10fdff700 1 -- 172.21.15.101:0/538391859 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fe0f40a1340 msgr2=0x7fe0f40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:25:04.365 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.367+0000 7fe10fdff700 1 --2- 172.21.15.101:0/538391859 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fe0f40a1340 0x7fe0f40a37f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe10400b2e0 tx=0x7fe104005fd0 comp rx=0 tx=0).stop 2023-07-18T03:25:04.365 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.367+0000 7fe10fdff700 1 -- 172.21.15.101:0/538391859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1080731c0 msgr2=0x7fe108198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:25:04.365 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.367+0000 7fe10fdff700 1 --2- 172.21.15.101:0/538391859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1080731c0 0x7fe108198380 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fe0f800e9a0 tx=0x7fe0f800ecb0 comp rx=0 tx=0).stop 2023-07-18T03:25:04.365 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.367+0000 7fe10fdff700 1 -- 172.21.15.101:0/538391859 shutdown_connections 2023-07-18T03:25:04.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.367+0000 7fe10fdff700 1 --2- 172.21.15.101:0/538391859 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe10810c350 0x7fe10819c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:04.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.367+0000 7fe10fdff700 1 --2- 172.21.15.101:0/538391859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1080728b0 0x7fe108197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:04.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.367+0000 7fe10fdff700 1 --2- 172.21.15.101:0/538391859 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fe0f40a1340 0x7fe0f40a37f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:04.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.367+0000 7fe10fdff700 1 --2- 172.21.15.101:0/538391859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1080731c0 0x7fe108198380 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:04.366 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.367+0000 7fe10fdff700 1 -- 172.21.15.101:0/538391859 >> 172.21.15.101:0/538391859 conn(0x7fe1080fb5f0 msgr2=0x7fe10810e450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:25:04.367 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.367+0000 7fe10fdff700 1 -- 172.21.15.101:0/538391859 shutdown_connections 2023-07-18T03:25:04.367 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.367+0000 7fe10fdff700 1 -- 172.21.15.101:0/538391859 wait complete. 2023-07-18T03:25:04.383 DEBUG:teuthology.orchestra.run.smithi101:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-07-18T03:25:04.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.586+0000 7f45923ff700 1 -- 172.21.15.101:0/1571483150 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f458c105b60 msgr2=0x7f458c105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:25:04.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.586+0000 7f45923ff700 1 --2- 172.21.15.101:0/1571483150 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f458c105b60 0x7f458c105f30 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f457c009990 tx=0x7f457c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:25:04.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.586+0000 7f45923ff700 1 -- 172.21.15.101:0/1571483150 shutdown_connections 2023-07-18T03:25:04.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.586+0000 7f45923ff700 1 --2- 172.21.15.101:0/1571483150 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f458c105b60 0x7f458c105f30 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:04.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.586+0000 7f45923ff700 1 --2- 172.21.15.101:0/1571483150 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f458c1004e0 0x7f458c10c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:04.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.586+0000 7f45923ff700 1 --2- 172.21.15.101:0/1571483150 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f458c0ffb30 0x7f458c0fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:04.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.586+0000 7f45923ff700 1 -- 172.21.15.101:0/1571483150 >> 172.21.15.101:0/1571483150 conn(0x7f458c0fb5f0 msgr2=0x7f458c0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:25:04.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.586+0000 7f45923ff700 1 -- 172.21.15.101:0/1571483150 shutdown_connections 2023-07-18T03:25:04.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.586+0000 7f45923ff700 1 -- 172.21.15.101:0/1571483150 wait complete. 2023-07-18T03:25:04.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.587+0000 7f45923ff700 1 Processor -- start 2023-07-18T03:25:04.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.587+0000 7f45923ff700 1 -- start start 2023-07-18T03:25:04.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.587+0000 7f45923ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f458c0ffb30 0x7f458c197d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:04.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.587+0000 7f45923ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f458c1004e0 0x7f458c1982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:04.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.587+0000 7f45923ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f458c105b60 0x7f458c19c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:04.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.587+0000 7f45923ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458c19cf80 con 0x7f458c1004e0 2023-07-18T03:25:04.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.587+0000 7f45923ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458c19d0f0 con 0x7f458c105b60 2023-07-18T03:25:04.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.587+0000 7f45923ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458c19d2b0 con 0x7f458c0ffb30 2023-07-18T03:25:04.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.588+0000 7f4583fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f458c1004e0 0x7f458c1982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:04.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.588+0000 7f4583fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f458c1004e0 0x7f458c1982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:56116/0 (socket says 172.21.15.101:56116) 2023-07-18T03:25:04.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.588+0000 7f459099c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f458c105b60 0x7f458c19c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:04.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.588+0000 7f458bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f458c0ffb30 0x7f458c197d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:04.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.588+0000 7f4583fff700 1 -- 172.21.15.101:0/1220108273 learned_addr learned my addr 172.21.15.101:0/1220108273 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:25:04.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.588+0000 7f4583fff700 1 -- 172.21.15.101:0/1220108273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f458c0ffb30 msgr2=0x7f458c197d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:25:04.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.588+0000 7f4583fff700 1 --2- 172.21.15.101:0/1220108273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f458c0ffb30 0x7f458c197d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:04.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.588+0000 7f4583fff700 1 -- 172.21.15.101:0/1220108273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f458c105b60 msgr2=0x7f458c19c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:25:04.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.588+0000 7f4583fff700 1 --2- 172.21.15.101:0/1220108273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f458c105b60 0x7f458c19c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:04.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.588+0000 7f4583fff700 1 -- 172.21.15.101:0/1220108273 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f457c0095f0 con 0x7f458c1004e0 2023-07-18T03:25:04.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.588+0000 7f459099c700 1 --2- 172.21.15.101:0/1220108273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f458c105b60 0x7f458c19c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:25:04.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.588+0000 7f4583fff700 1 --2- 172.21.15.101:0/1220108273 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f458c1004e0 0x7f458c1982d0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f457400e9a0 tx=0x7f457400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:25:04.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.588+0000 7f4589ffb700 1 -- 172.21.15.101:0/1220108273 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4574009790 con 0x7f458c1004e0 2023-07-18T03:25:04.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.588+0000 7f45923ff700 1 -- 172.21.15.101:0/1220108273 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f458c19d590 con 0x7f458c1004e0 2023-07-18T03:25:04.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.589+0000 7f4589ffb700 1 -- 172.21.15.101:0/1220108273 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4574010430 con 0x7f458c1004e0 2023-07-18T03:25:04.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.589+0000 7f4589ffb700 1 -- 172.21.15.101:0/1220108273 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4574018680 con 0x7f458c1004e0 2023-07-18T03:25:04.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.589+0000 7f45923ff700 1 -- 172.21.15.101:0/1220108273 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f458c1a4dd0 con 0x7f458c1004e0 2023-07-18T03:25:04.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.590+0000 7f45923ff700 1 -- 172.21.15.101:0/1220108273 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f458c0ff250 con 0x7f458c1004e0 2023-07-18T03:25:04.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.591+0000 7f4589ffb700 1 -- 172.21.15.101:0/1220108273 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f45740105a0 con 0x7f458c1004e0 2023-07-18T03:25:04.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.593+0000 7f4589ffb700 1 --2- 172.21.15.101:0/1220108273 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f45780a12c0 0x7f45780a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:04.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.593+0000 7f4589ffb700 1 -- 172.21.15.101:0/1220108273 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f4574014070 con 0x7f458c1004e0 2023-07-18T03:25:04.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.593+0000 7f458bfff700 1 --2- 172.21.15.101:0/1220108273 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f45780a12c0 0x7f45780a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:04.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.594+0000 7f458bfff700 1 --2- 172.21.15.101:0/1220108273 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f45780a12c0 0x7f45780a3770 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f457c00b2e0 tx=0x7f457c005fb0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:25:04.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.597+0000 7f4589ffb700 1 -- 172.21.15.101:0/1220108273 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4574051790 con 0x7f458c1004e0 2023-07-18T03:25:04.822 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.825+0000 7f45923ff700 1 -- 172.21.15.101:0/1220108273 --> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f458c073e60 con 0x7f45780a12c0 2023-07-18T03:25:04.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.836+0000 7f4589ffb700 1 -- 172.21.15.101:0/1220108273 <== mgr.15486 v2:172.21.15.101:6800/486424816 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f458c073e60 con 0x7f45780a12c0 2023-07-18T03:25:04.833 INFO:teuthology.orchestra.run.smithi101.stdout:/ 1M 6M3h 2023-07-18T03:25:04.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.839+0000 7f45923ff700 1 -- 172.21.15.101:0/1220108273 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f45780a12c0 msgr2=0x7f45780a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:25:04.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.839+0000 7f45923ff700 1 --2- 172.21.15.101:0/1220108273 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f45780a12c0 0x7f45780a3770 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f457c00b2e0 tx=0x7f457c005fb0 comp rx=0 tx=0).stop 2023-07-18T03:25:04.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.839+0000 7f45923ff700 1 -- 172.21.15.101:0/1220108273 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f458c1004e0 msgr2=0x7f458c1982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:25:04.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.839+0000 7f45923ff700 1 --2- 172.21.15.101:0/1220108273 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f458c1004e0 0x7f458c1982d0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f457400e9a0 tx=0x7f457400ecb0 comp rx=0 tx=0).stop 2023-07-18T03:25:04.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.839+0000 7f45923ff700 1 -- 172.21.15.101:0/1220108273 shutdown_connections 2023-07-18T03:25:04.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.839+0000 7f45923ff700 1 --2- 172.21.15.101:0/1220108273 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f45780a12c0 0x7f45780a3770 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:04.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.839+0000 7f45923ff700 1 --2- 172.21.15.101:0/1220108273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f458c105b60 0x7f458c19c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:04.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.839+0000 7f45923ff700 1 --2- 172.21.15.101:0/1220108273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f458c0ffb30 0x7f458c197d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:04.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.839+0000 7f45923ff700 1 --2- 172.21.15.101:0/1220108273 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f458c1004e0 0x7f458c1982d0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:04.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.839+0000 7f45923ff700 1 -- 172.21.15.101:0/1220108273 >> 172.21.15.101:0/1220108273 conn(0x7f458c0fb5f0 msgr2=0x7f458c072740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:25:04.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.839+0000 7f45923ff700 1 -- 172.21.15.101:0/1220108273 shutdown_connections 2023-07-18T03:25:04.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:04.839+0000 7f45923ff700 1 -- 172.21.15.101:0/1220108273 wait complete. 2023-07-18T03:25:04.855 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2023-07-18T03:25:04.866 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2023-07-18T03:25:05.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:25:05 smithi155 ceph-mon[128885]: pgmap v550: 129 pgs: 129 active+clean; 1.7 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 11 MiB/s wr, 476 op/s 2023-07-18T03:25:05.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:25:05 smithi179 ceph-mon[129229]: pgmap v550: 129 pgs: 129 active+clean; 1.7 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 11 MiB/s wr, 476 op/s 2023-07-18T03:25:05.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:25:05 smithi101 ceph-mon[126379]: pgmap v550: 129 pgs: 129 active+clean; 1.7 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 11 MiB/s wr, 476 op/s 2023-07-18T03:25:06.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:25:06 smithi179 ceph-mon[129229]: from='client.28922 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:25:06.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:25:06 smithi179 ceph-mon[129229]: from='client.18831 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:25:06.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:25:06 smithi179 ceph-mon[129229]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:25:06.743 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:25:06 smithi101 ceph-mon[126379]: from='client.28922 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:25:06.744 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:25:06 smithi101 ceph-mon[126379]: from='client.18831 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:25:06.744 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:25:06 smithi101 ceph-mon[126379]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:25:06.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:25:06 smithi155 ceph-mon[128885]: from='client.28922 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:25:06.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:25:06 smithi155 ceph-mon[128885]: from='client.18831 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:25:06.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:25:06 smithi155 ceph-mon[128885]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:25:07.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:25:07 smithi179 ceph-mon[129229]: pgmap v551: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 16 MiB/s wr, 660 op/s 2023-07-18T03:25:07.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:25:07 smithi179 ceph-mon[129229]: osdmap e115: 12 total, 12 up, 12 in 2023-07-18T03:25:07.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:25:07 smithi101 ceph-mon[126379]: pgmap v551: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 16 MiB/s wr, 660 op/s 2023-07-18T03:25:07.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:25:07 smithi101 ceph-mon[126379]: osdmap e115: 12 total, 12 up, 12 in 2023-07-18T03:25:07.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:25:07 smithi155 ceph-mon[128885]: pgmap v551: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 16 MiB/s wr, 660 op/s 2023-07-18T03:25:07.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:25:07 smithi155 ceph-mon[128885]: osdmap e115: 12 total, 12 up, 12 in 2023-07-18T03:25:09.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:25:09 smithi179 ceph-mon[129229]: pgmap v553: 129 pgs: 1 active+clean+snaptrim, 121 active+clean, 7 active+clean+snaptrim_wait; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 13 MiB/s wr, 570 op/s 2023-07-18T03:25:09.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:25:09 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:25:09.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:25:09 smithi101 ceph-mon[126379]: pgmap v553: 129 pgs: 1 active+clean+snaptrim, 121 active+clean, 7 active+clean+snaptrim_wait; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 13 MiB/s wr, 570 op/s 2023-07-18T03:25:09.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:25:09 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:25:09.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:25:09 smithi155 ceph-mon[128885]: pgmap v553: 129 pgs: 1 active+clean+snaptrim, 121 active+clean, 7 active+clean+snaptrim_wait; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 13 MiB/s wr, 570 op/s 2023-07-18T03:25:09.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:25:09 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:25:11.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:25:11 smithi179 ceph-mon[129229]: pgmap v554: 129 pgs: 1 active+clean+snaptrim, 121 active+clean, 7 active+clean+snaptrim_wait; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 9.8 MiB/s wr, 413 op/s 2023-07-18T03:25:11.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:25:11 smithi101 ceph-mon[126379]: pgmap v554: 129 pgs: 1 active+clean+snaptrim, 121 active+clean, 7 active+clean+snaptrim_wait; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 9.8 MiB/s wr, 413 op/s 2023-07-18T03:25:11.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:25:11 smithi155 ceph-mon[128885]: pgmap v554: 129 pgs: 1 active+clean+snaptrim, 121 active+clean, 7 active+clean+snaptrim_wait; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 9.8 MiB/s wr, 413 op/s 2023-07-18T03:25:13.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:25:13 smithi179 ceph-mon[129229]: pgmap v555: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.4 MiB/s wr, 244 op/s 2023-07-18T03:25:13.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:25:13 smithi101 ceph-mon[126379]: pgmap v555: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.4 MiB/s wr, 244 op/s 2023-07-18T03:25:13.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:25:13 smithi155 ceph-mon[128885]: pgmap v555: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.4 MiB/s wr, 244 op/s 2023-07-18T03:25:14.048 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:25:14.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.263+0000 7fcdc39ff700 1 -- 172.21.15.101:0/2934949104 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdbc0ffad0 msgr2=0x7fcdbc0fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:25:14.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.263+0000 7fcdc39ff700 1 --2- 172.21.15.101:0/2934949104 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdbc0ffad0 0x7fcdbc0fff40 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fcdac009990 tx=0x7fcdac009ca0 comp rx=0 tx=0).stop 2023-07-18T03:25:14.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.264+0000 7fcdc39ff700 1 -- 172.21.15.101:0/2934949104 shutdown_connections 2023-07-18T03:25:14.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.264+0000 7fcdc39ff700 1 --2- 172.21.15.101:0/2934949104 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcdbc100480 0x7fcdbc10c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:14.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.264+0000 7fcdc39ff700 1 --2- 172.21.15.101:0/2934949104 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcdbc105ae0 0x7fcdbc105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:14.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.264+0000 7fcdc39ff700 1 --2- 172.21.15.101:0/2934949104 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdbc0ffad0 0x7fcdbc0fff40 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:14.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.264+0000 7fcdc39ff700 1 -- 172.21.15.101:0/2934949104 >> 172.21.15.101:0/2934949104 conn(0x7fcdbc0fb5f0 msgr2=0x7fcdbc0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:25:14.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.264+0000 7fcdc39ff700 1 -- 172.21.15.101:0/2934949104 shutdown_connections 2023-07-18T03:25:14.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.264+0000 7fcdc39ff700 1 -- 172.21.15.101:0/2934949104 wait complete. 2023-07-18T03:25:14.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.265+0000 7fcdc39ff700 1 Processor -- start 2023-07-18T03:25:14.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.265+0000 7fcdc39ff700 1 -- start start 2023-07-18T03:25:14.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.265+0000 7fcdc39ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcdbc0ffad0 0x7fcdbc197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:14.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.265+0000 7fcdc39ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcdbc100480 0x7fcdbc198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:14.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.265+0000 7fcdc39ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdbc105ae0 0x7fcdbc19c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:14.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.265+0000 7fcdc39ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdbc19ce20 con 0x7fcdbc100480 2023-07-18T03:25:14.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.265+0000 7fcdc39ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdbc19cf90 con 0x7fcdbc0ffad0 2023-07-18T03:25:14.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.265+0000 7fcdc39ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdbc19d150 con 0x7fcdbc105ae0 2023-07-18T03:25:14.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.265+0000 7fcdc179b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcdbc0ffad0 0x7fcdbc197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:14.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.265+0000 7fcdc179b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcdbc0ffad0 0x7fcdbc197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:43300/0 (socket says 172.21.15.101:43300) 2023-07-18T03:25:14.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.265+0000 7fcdc179b700 1 -- 172.21.15.101:0/138956868 learned_addr learned my addr 172.21.15.101:0/138956868 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:25:14.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.266+0000 7fcdc1f9c700 1 --2- 172.21.15.101:0/138956868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdbc105ae0 0x7fcdbc19c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:14.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.266+0000 7fcdc0f9a700 1 --2- 172.21.15.101:0/138956868 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcdbc100480 0x7fcdbc198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:14.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.266+0000 7fcdc179b700 1 -- 172.21.15.101:0/138956868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdbc105ae0 msgr2=0x7fcdbc19c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:25:14.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.266+0000 7fcdc179b700 1 --2- 172.21.15.101:0/138956868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdbc105ae0 0x7fcdbc19c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:14.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.266+0000 7fcdc179b700 1 -- 172.21.15.101:0/138956868 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcdbc100480 msgr2=0x7fcdbc198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:25:14.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.266+0000 7fcdc179b700 1 --2- 172.21.15.101:0/138956868 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcdbc100480 0x7fcdbc198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:14.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.266+0000 7fcdc179b700 1 -- 172.21.15.101:0/138956868 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb8009520 con 0x7fcdbc0ffad0 2023-07-18T03:25:14.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.266+0000 7fcdc0f9a700 1 --2- 172.21.15.101:0/138956868 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcdbc100480 0x7fcdbc198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:25:14.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.266+0000 7fcdc179b700 1 --2- 172.21.15.101:0/138956868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcdbc0ffad0 0x7fcdbc197e40 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fcdb800eb30 tx=0x7fcdb800ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:25:14.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.266+0000 7fcdb27fc700 1 -- 172.21.15.101:0/138956868 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb8009880 con 0x7fcdbc0ffad0 2023-07-18T03:25:14.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.266+0000 7fcdc39ff700 1 -- 172.21.15.101:0/138956868 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdac0095f0 con 0x7fcdbc0ffad0 2023-07-18T03:25:14.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.266+0000 7fcdb27fc700 1 -- 172.21.15.101:0/138956868 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcdb80044e0 con 0x7fcdbc0ffad0 2023-07-18T03:25:14.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.266+0000 7fcdb27fc700 1 -- 172.21.15.101:0/138956868 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb8003e90 con 0x7fcdbc0ffad0 2023-07-18T03:25:14.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.266+0000 7fcdc39ff700 1 -- 172.21.15.101:0/138956868 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdbc1a4a90 con 0x7fcdbc0ffad0 2023-07-18T03:25:14.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.270+0000 7fcdb27fc700 1 -- 172.21.15.101:0/138956868 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fcdb8004000 con 0x7fcdbc0ffad0 2023-07-18T03:25:14.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.271+0000 7fcdd7381b80 1 Processor -- start 2023-07-18T03:25:14.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.271+0000 7fcdd7381b80 1 -- start start 2023-07-18T03:25:14.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.271+0000 7fcdd7381b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15de142d0 0x55c15de146a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:14.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.271+0000 7fcdd7381b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15de14be0 0x55c15de1f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:14.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.271+0000 7fcdb27fc700 1 --2- 172.21.15.101:0/138956868 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fcda80a12c0 0x7fcda80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:14.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.271+0000 7fcdd7381b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c15de1fad0 0x55c15de21eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:14.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.271+0000 7fcdd7381b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15dd1acb0 con 0x55c15de14be0 2023-07-18T03:25:14.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.271+0000 7fcdd7381b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15dd6da40 con 0x55c15de142d0 2023-07-18T03:25:14.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.271+0000 7fcdd7381b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15dd66ed0 con 0x55c15de1fad0 2023-07-18T03:25:14.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.271+0000 7fcdb27fc700 1 -- 172.21.15.101:0/138956868 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fcdb8014070 con 0x7fcdbc0ffad0 2023-07-18T03:25:14.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.272+0000 7fcdc179b700 1 --2- 172.21.15.101:0/138956868 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fcda80a12c0 0x7fcda80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:14.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.271+0000 7fcdc1f9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15de142d0 0x55c15de146a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:14.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.272+0000 7fcdc0f9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15de14be0 0x55c15de1f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:14.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.272+0000 7fcdc1f9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15de142d0 0x55c15de146a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:43306/0 (socket says 172.21.15.101:43306) 2023-07-18T03:25:14.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.272+0000 7fcdc1f9c700 1 -- 172.21.15.101:0/3780390696 learned_addr learned my addr 172.21.15.101:0/3780390696 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:25:14.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.272+0000 7fcdc1f9c700 1 --2- 172.21.15.101:0/3780390696 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c15de1fad0 0x55c15de21eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:14.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.272+0000 7fcdc0f9a700 1 -- 172.21.15.101:0/3780390696 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c15de1fad0 msgr2=0x55c15de21eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:25:14.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.272+0000 7fcdc0f9a700 1 --2- 172.21.15.101:0/3780390696 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c15de1fad0 0x55c15de21eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:14.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.272+0000 7fcdc0f9a700 1 -- 172.21.15.101:0/3780390696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15de142d0 msgr2=0x55c15de146a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:25:14.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.272+0000 7fcdc0f9a700 1 --2- 172.21.15.101:0/3780390696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15de142d0 0x55c15de146a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:14.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.272+0000 7fcdc0f9a700 1 -- 172.21.15.101:0/3780390696 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c15db94de0 con 0x55c15de14be0 2023-07-18T03:25:14.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.272+0000 7fcdc1f9c700 1 --2- 172.21.15.101:0/3780390696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15de142d0 0x55c15de146a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:25:14.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.272+0000 7fcdc179b700 1 --2- 172.21.15.101:0/138956868 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fcda80a12c0 0x7fcda80a3770 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcdb809a150 tx=0x7fcdb80047c0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:25:14.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.272+0000 7fcdc0f9a700 1 --2- 172.21.15.101:0/3780390696 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15de14be0 0x55c15de1f590 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc199450 tx=0x7fcdac005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=171b9503cb3a7883 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:25:14.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.273+0000 7fcda77fe700 1 -- 172.21.15.101:0/3780390696 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdac00ba50 con 0x55c15de14be0 2023-07-18T03:25:14.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.273+0000 7fcda77fe700 1 -- 172.21.15.101:0/3780390696 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcdac01e040 con 0x55c15de14be0 2023-07-18T03:25:14.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.273+0000 7fcda77fe700 1 -- 172.21.15.101:0/3780390696 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdac005290 con 0x55c15de14be0 2023-07-18T03:25:14.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.273+0000 7fcdd7381b80 1 -- 172.21.15.101:0/3780390696 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15de14be0 msgr2=0x55c15de1f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:25:14.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.273+0000 7fcdd7381b80 1 --2- 172.21.15.101:0/3780390696 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15de14be0 0x55c15de1f590 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc199450 tx=0x7fcdac005c00 comp rx=0 tx=0).stop 2023-07-18T03:25:14.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.273+0000 7fcdd7381b80 1 -- 172.21.15.101:0/3780390696 shutdown_connections 2023-07-18T03:25:14.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.273+0000 7fcdd7381b80 1 --2- 172.21.15.101:0/3780390696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15de142d0 0x55c15de146a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:14.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.273+0000 7fcdd7381b80 1 --2- 172.21.15.101:0/3780390696 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c15de1fad0 0x55c15de21eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:14.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.273+0000 7fcdd7381b80 1 --2- 172.21.15.101:0/3780390696 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15de14be0 0x55c15de1f590 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:14.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.273+0000 7fcdd7381b80 1 -- 172.21.15.101:0/3780390696 >> 172.21.15.101:0/3780390696 conn(0x55c15de0ada0 msgr2=0x55c15de106a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:25:14.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.273+0000 7fcdd7381b80 1 -- 172.21.15.101:0/3780390696 shutdown_connections 2023-07-18T03:25:14.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.273+0000 7fcdd7381b80 1 -- 172.21.15.101:0/3780390696 wait complete. 2023-07-18T03:25:14.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.274+0000 7fcdd7381b80 1 -- start start 2023-07-18T03:25:14.284 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.274+0000 7fcdd7381b80 1 Processor -- start 2023-07-18T03:25:14.284 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.275+0000 7fcdd7381b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c15de142d0 0x55c15de1f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:14.284 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.275+0000 7fcdd7381b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15de14be0 0x55c15de1c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:14.284 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.275+0000 7fcdd7381b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15de1fad0 0x55c15de1ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:14.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.275+0000 7fcdd7381b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15dd06ac0 con 0x55c15de1fad0 2023-07-18T03:25:14.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.275+0000 7fcdd7381b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15dd067c0 con 0x55c15de14be0 2023-07-18T03:25:14.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.275+0000 7fcdd7381b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15dbea920 con 0x55c15de142d0 2023-07-18T03:25:14.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.275+0000 7fcdd7381b80 10 client.0 _ll_register_callbacks cb 0x55c15dd04760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:25:14.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.275+0000 7fcdc0f9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15de14be0 0x55c15de1c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:14.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.275+0000 7fcdc0f9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15de14be0 0x55c15de1c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:43318/0 (socket says 172.21.15.101:43318) 2023-07-18T03:25:14.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.275+0000 7fcdc0f9a700 1 -- 172.21.15.101:0/1159868970 learned_addr learned my addr 172.21.15.101:0/1159868970 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:25:14.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.275+0000 7fcdc1f9c700 1 --2- 172.21.15.101:0/1159868970 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c15de142d0 0x55c15de1f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:14.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.275+0000 7fcdc1f9c700 1 --2- 172.21.15.101:0/1159868970 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15de1fad0 0x55c15de1ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:14.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.275+0000 7fcdc0f9a700 1 -- 172.21.15.101:0/1159868970 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c15de142d0 msgr2=0x55c15de1f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:25:14.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.275+0000 7fcdc0f9a700 1 --2- 172.21.15.101:0/1159868970 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c15de142d0 0x55c15de1f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:14.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.275+0000 7fcdc0f9a700 1 -- 172.21.15.101:0/1159868970 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15de1fad0 msgr2=0x55c15de1ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:25:14.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.275+0000 7fcdc0f9a700 1 --2- 172.21.15.101:0/1159868970 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15de1fad0 0x55c15de1ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:14.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.275+0000 7fcdc0f9a700 1 -- 172.21.15.101:0/1159868970 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdb40089b0 con 0x55c15de14be0 2023-07-18T03:25:14.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.276+0000 7fcdc0f9a700 1 --2- 172.21.15.101:0/1159868970 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15de14be0 0x55c15de1c750 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fcdac009990 tx=0x7fcdac004b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4551643a2fd1f477 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:25:14.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.276+0000 7fcda4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:25:14.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.276+0000 7fcda4ff9700 1 -- 172.21.15.101:0/1159868970 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdac005030 con 0x55c15de14be0 2023-07-18T03:25:14.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.276+0000 7fcdd7381b80 1 -- 172.21.15.101:0/1159868970 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c15dcc04f0 con 0x55c15de14be0 2023-07-18T03:25:14.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.276+0000 7fcda4ff9700 1 -- 172.21.15.101:0/1159868970 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fcdac005190 con 0x55c15de14be0 2023-07-18T03:25:14.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.276+0000 7fcda4ff9700 1 -- 172.21.15.101:0/1159868970 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcdac005350 con 0x55c15de14be0 2023-07-18T03:25:14.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.276+0000 7fcdd7381b80 10 client.28940 fetch_fsmap learned FSMap version 43 2023-07-18T03:25:14.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.276+0000 7fcdd7381b80 1 -- 172.21.15.101:0/1159868970 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7fcdbc10d210 con 0x55c15de14be0 2023-07-18T03:25:14.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.276+0000 7fcda4ff9700 1 -- 172.21.15.101:0/1159868970 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c15dcc04f0 con 0x55c15de14be0 2023-07-18T03:25:14.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.276+0000 7fcda4ff9700 1 -- 172.21.15.101:0/1159868970 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 43) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fcdac00fc40 con 0x55c15de14be0 2023-07-18T03:25:14.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.276+0000 7fcdd7381b80 10 client.28940 fetch_fsmap finished waiting for FSMap version 43 2023-07-18T03:25:14.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.276+0000 7fcdd7381b80 10 client.28940 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:25:14.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.277+0000 7fcdd7381b80 20 client.28940 populate_metadata read hostname 'smithi101' 2023-07-18T03:25:14.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.277+0000 7fcdd7381b80 1 --2- 172.21.15.101:0/1159868970 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c15de2eec0 0x55c15de312a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:14.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.277+0000 7fcdd7381b80 4 client.28940 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:25:14.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.277+0000 7fcdd7381b80 1 -- 172.21.15.101:0/1159868970 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c15dd5feb0 con 0x55c15de2eec0 2023-07-18T03:25:14.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.277+0000 7fcdc1f9c700 1 --2- 172.21.15.101:0/1159868970 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c15de2eec0 0x55c15de312a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:14.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.277+0000 7fcda4ff9700 10 client.28940 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:25:14.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.277+0000 7fcdc1f9c700 1 --2- 172.21.15.101:0/1159868970 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c15de2eec0 0x55c15de312a0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=654cc0c08ed3e94c server_cookie=b1c039fd3f403f4c in_seq=0 out_seq=0 2023-07-18T03:25:14.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.278+0000 7fcda4ff9700 1 -- 172.21.15.101:0/1159868970 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c15dd5feb0 con 0x55c15de2eec0 2023-07-18T03:25:14.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.278+0000 7fcda4ff9700 10 client.28940 handle_command_reply: tid=0 2023-07-18T03:25:14.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.278+0000 7fcdd7381b80 1 client.28940 shutdown 2023-07-18T03:25:14.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.278+0000 7fcdd7381b80 1 -- 172.21.15.101:0/1159868970 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15de14be0 msgr2=0x55c15de1c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:25:14.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.278+0000 7fcdd7381b80 1 --2- 172.21.15.101:0/1159868970 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15de14be0 0x55c15de1c750 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fcdac009990 tx=0x7fcdac004b10 comp rx=0 tx=0).stop 2023-07-18T03:25:14.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.278+0000 7fcdd7381b80 1 -- 172.21.15.101:0/1159868970 shutdown_connections 2023-07-18T03:25:14.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.278+0000 7fcdd7381b80 1 --2- 172.21.15.101:0/1159868970 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c15de2eec0 0x55c15de312a0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:14.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.278+0000 7fcdd7381b80 1 --2- 172.21.15.101:0/1159868970 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15de1fad0 0x55c15de1ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:14.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.278+0000 7fcdd7381b80 1 --2- 172.21.15.101:0/1159868970 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c15de142d0 0x55c15de1f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:14.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.278+0000 7fcdd7381b80 1 --2- 172.21.15.101:0/1159868970 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15de14be0 0x55c15de1c750 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:14.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.278+0000 7fcda4ff9700 0 client.28940 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:25:14.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.278+0000 7fcdd7381b80 1 -- 172.21.15.101:0/1159868970 >> 172.21.15.101:0/1159868970 conn(0x55c15de0ada0 msgr2=0x55c15de10230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:25:14.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.278+0000 7fcdd7381b80 1 -- 172.21.15.101:0/1159868970 shutdown_connections 2023-07-18T03:25:14.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.278+0000 7fcdd7381b80 1 -- 172.21.15.101:0/1159868970 wait complete. 2023-07-18T03:25:14.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.279+0000 7fcdd7381b80 20 client.28940 trim_cache size 0 max 16384 2023-07-18T03:25:14.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.295+0000 7fcdd7381b80 1 Processor -- start 2023-07-18T03:25:14.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.296+0000 7fcdd7381b80 1 -- start start 2023-07-18T03:25:14.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.296+0000 7fcdd7381b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15de142d0 0x55c15de0ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:14.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.296+0000 7fcdd7381b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15de14be0 0x55c15de1f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:14.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.296+0000 7fcdd7381b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c15de1fad0 0x55c15de17a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:14.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.296+0000 7fcdd7381b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15de0f350 con 0x55c15de142d0 2023-07-18T03:25:14.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.296+0000 7fcdd7381b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15de1c710 con 0x55c15de14be0 2023-07-18T03:25:14.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.296+0000 7fcdd7381b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15de1bff0 con 0x55c15de1fad0 2023-07-18T03:25:14.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.296+0000 7fcdc0f9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15de14be0 0x55c15de1f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:14.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.296+0000 7fcdc0f9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15de14be0 0x55c15de1f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:43332/0 (socket says 172.21.15.101:43332) 2023-07-18T03:25:14.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.296+0000 7fcdc0f9a700 1 -- 172.21.15.101:0/662920616 learned_addr learned my addr 172.21.15.101:0/662920616 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:25:14.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.296+0000 7fcdc1f9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15de142d0 0x55c15de0ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:14.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.296+0000 7fcdc1f9c700 1 --2- 172.21.15.101:0/662920616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c15de1fad0 0x55c15de17a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:14.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.296+0000 7fcdc0f9a700 1 -- 172.21.15.101:0/662920616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c15de1fad0 msgr2=0x55c15de17a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:25:14.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.296+0000 7fcdc0f9a700 1 --2- 172.21.15.101:0/662920616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c15de1fad0 0x55c15de17a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:14.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.296+0000 7fcdc0f9a700 1 -- 172.21.15.101:0/662920616 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15de142d0 msgr2=0x55c15de0ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:25:14.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.296+0000 7fcdc0f9a700 1 --2- 172.21.15.101:0/662920616 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15de142d0 0x55c15de0ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:14.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.296+0000 7fcdc0f9a700 1 -- 172.21.15.101:0/662920616 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c15db7dc60 con 0x55c15de14be0 2023-07-18T03:25:14.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.296+0000 7fcdc1f9c700 1 --2- 172.21.15.101:0/662920616 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15de142d0 0x55c15de0ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:25:14.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.297+0000 7fcdc0f9a700 1 --2- 172.21.15.101:0/662920616 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15de14be0 0x55c15de1f070 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fcdac000c00 tx=0x7fcdac02b600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af86eb13474daf60 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:25:14.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.297+0000 7fcda7fff700 1 -- 172.21.15.101:0/662920616 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdac041070 con 0x55c15de14be0 2023-07-18T03:25:14.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.297+0000 7fcda7fff700 1 -- 172.21.15.101:0/662920616 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcdac004b80 con 0x55c15de14be0 2023-07-18T03:25:14.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.297+0000 7fcda7fff700 1 -- 172.21.15.101:0/662920616 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdac0347e0 con 0x55c15de14be0 2023-07-18T03:25:14.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.297+0000 7fcdd7381b80 1 -- 172.21.15.101:0/662920616 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15de14be0 msgr2=0x55c15de1f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:25:14.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.297+0000 7fcdd7381b80 1 --2- 172.21.15.101:0/662920616 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15de14be0 0x55c15de1f070 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fcdac000c00 tx=0x7fcdac02b600 comp rx=0 tx=0).stop 2023-07-18T03:25:14.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.297+0000 7fcdd7381b80 1 -- 172.21.15.101:0/662920616 shutdown_connections 2023-07-18T03:25:14.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.297+0000 7fcdd7381b80 1 --2- 172.21.15.101:0/662920616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c15de1fad0 0x55c15de17a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:14.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.297+0000 7fcdd7381b80 1 --2- 172.21.15.101:0/662920616 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15de142d0 0x55c15de0ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:14.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.297+0000 7fcdd7381b80 1 --2- 172.21.15.101:0/662920616 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15de14be0 0x55c15de1f070 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:14.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.297+0000 7fcdd7381b80 1 -- 172.21.15.101:0/662920616 >> 172.21.15.101:0/662920616 conn(0x55c15de0ada0 msgr2=0x55c15de35b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:25:14.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.298+0000 7fcdd7381b80 1 -- 172.21.15.101:0/662920616 shutdown_connections 2023-07-18T03:25:14.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.298+0000 7fcdd7381b80 1 -- 172.21.15.101:0/662920616 wait complete. 2023-07-18T03:25:14.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.298+0000 7fcdd7381b80 1 -- start start 2023-07-18T03:25:14.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.298+0000 7fcdd7381b80 1 Processor -- start 2023-07-18T03:25:14.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.298+0000 7fcdd7381b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c15de142d0 0x55c15de26570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:14.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.298+0000 7fcdd7381b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15de14be0 0x55c15de248a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:14.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.298+0000 7fcdd7381b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15de1fad0 0x55c15de24de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:14.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.299+0000 7fcdd7381b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15de254f0 con 0x55c15de14be0 2023-07-18T03:25:14.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.299+0000 7fcdd7381b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15de25660 con 0x55c15de1fad0 2023-07-18T03:25:14.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.299+0000 7fcdd7381b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15de82cc0 con 0x55c15de142d0 2023-07-18T03:25:14.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.299+0000 7fcdd7381b80 10 client.0 _ll_register_callbacks cb 0x7fcdb4009320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:25:14.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.299+0000 7fcdc1f9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c15de142d0 0x55c15de26570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:14.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.299+0000 7fcdc1f9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c15de142d0 0x55c15de26570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:36230/0 (socket says 172.21.15.101:36230) 2023-07-18T03:25:14.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.299+0000 7fcdc1f9c700 1 -- 172.21.15.101:0/4051881982 learned_addr learned my addr 172.21.15.101:0/4051881982 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:25:14.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.299+0000 7fcdc1f9c700 1 --2- 172.21.15.101:0/4051881982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15de1fad0 0x55c15de24de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:14.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.299+0000 7fcdc0f9a700 1 --2- 172.21.15.101:0/4051881982 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15de14be0 0x55c15de248a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:14.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.299+0000 7fcdc1f9c700 1 -- 172.21.15.101:0/4051881982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15de1fad0 msgr2=0x55c15de24de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:25:14.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.299+0000 7fcdc1f9c700 1 --2- 172.21.15.101:0/4051881982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15de1fad0 0x55c15de24de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:14.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.299+0000 7fcdc1f9c700 1 -- 172.21.15.101:0/4051881982 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15de14be0 msgr2=0x55c15de248a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:25:14.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.299+0000 7fcdc1f9c700 1 --2- 172.21.15.101:0/4051881982 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15de14be0 0x55c15de248a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:14.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.299+0000 7fcdc1f9c700 1 -- 172.21.15.101:0/4051881982 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdbc10d210 con 0x55c15de142d0 2023-07-18T03:25:14.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.299+0000 7fcdc0f9a700 1 --2- 172.21.15.101:0/4051881982 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15de14be0 0x55c15de248a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:25:14.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.299+0000 7fcdc1f9c700 1 --2- 172.21.15.101:0/4051881982 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c15de142d0 0x55c15de26570 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4009550 tx=0x7fcdb4025040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2a17f8166f1366b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:25:14.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.299+0000 7fcda4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:25:14.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.300+0000 7fcda4ff9700 1 -- 172.21.15.101:0/4051881982 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb400e040 con 0x55c15de142d0 2023-07-18T03:25:14.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.300+0000 7fcdd7381b80 1 -- 172.21.15.101:0/4051881982 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcdac01e040 con 0x55c15de142d0 2023-07-18T03:25:14.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.300+0000 7fcda4ff9700 1 -- 172.21.15.101:0/4051881982 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fcdb4020040 con 0x55c15de142d0 2023-07-18T03:25:14.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.300+0000 7fcda4ff9700 1 -- 172.21.15.101:0/4051881982 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcdb4023c70 con 0x55c15de142d0 2023-07-18T03:25:14.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.300+0000 7fcdd7381b80 10 client.28936 fetch_fsmap learned FSMap version 43 2023-07-18T03:25:14.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.300+0000 7fcdd7381b80 1 -- 172.21.15.101:0/4051881982 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55c15dbbb250 con 0x55c15de142d0 2023-07-18T03:25:14.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.300+0000 7fcda4ff9700 1 -- 172.21.15.101:0/4051881982 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcdac01e040 con 0x55c15de142d0 2023-07-18T03:25:14.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.300+0000 7fcda4ff9700 1 -- 172.21.15.101:0/4051881982 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 43) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fcdb402e070 con 0x55c15de142d0 2023-07-18T03:25:14.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.300+0000 7fcdd7381b80 10 client.28936 fetch_fsmap finished waiting for FSMap version 43 2023-07-18T03:25:14.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.300+0000 7fcdd7381b80 10 client.28936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:25:14.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.300+0000 7fcdd7381b80 20 client.28936 populate_metadata read hostname 'smithi101' 2023-07-18T03:25:14.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.300+0000 7fcdd7381b80 1 --2- 172.21.15.101:0/4051881982 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c15de843b0 0x55c15de867c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:14.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.300+0000 7fcdd7381b80 4 client.28936 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:25:14.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.300+0000 7fcdd7381b80 1 -- 172.21.15.101:0/4051881982 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c15dd22050 con 0x55c15de843b0 2023-07-18T03:25:14.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.300+0000 7fcdc0f9a700 1 --2- 172.21.15.101:0/4051881982 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c15de843b0 0x55c15de867c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:14.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.301+0000 7fcda4ff9700 10 client.28936 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:25:14.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.301+0000 7fcdc0f9a700 1 --2- 172.21.15.101:0/4051881982 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c15de843b0 0x55c15de867c0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91bea124ad92a317 server_cookie=604be99f2e7f8ffb in_seq=0 out_seq=0 2023-07-18T03:25:14.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.301+0000 7fcda4ff9700 1 -- 172.21.15.101:0/4051881982 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c15dd22050 con 0x55c15de843b0 2023-07-18T03:25:14.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.301+0000 7fcda4ff9700 10 client.28936 handle_command_reply: tid=0 2023-07-18T03:25:14.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.301+0000 7fcdd7381b80 1 client.28936 shutdown 2023-07-18T03:25:14.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.301+0000 7fcdd7381b80 1 -- 172.21.15.101:0/4051881982 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c15de142d0 msgr2=0x55c15de26570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:25:14.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.301+0000 7fcdd7381b80 1 --2- 172.21.15.101:0/4051881982 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c15de142d0 0x55c15de26570 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4009550 tx=0x7fcdb4025040 comp rx=0 tx=0).stop 2023-07-18T03:25:14.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.301+0000 7fcdd7381b80 1 -- 172.21.15.101:0/4051881982 shutdown_connections 2023-07-18T03:25:14.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.301+0000 7fcdd7381b80 1 --2- 172.21.15.101:0/4051881982 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c15de843b0 0x55c15de867c0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:14.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.301+0000 7fcdd7381b80 1 --2- 172.21.15.101:0/4051881982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15de1fad0 0x55c15de24de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:14.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.301+0000 7fcda4ff9700 0 client.28936 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:25:14.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.301+0000 7fcdd7381b80 1 --2- 172.21.15.101:0/4051881982 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c15de142d0 0x55c15de26570 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:14.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.301+0000 7fcdd7381b80 1 --2- 172.21.15.101:0/4051881982 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15de14be0 0x55c15de248a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:14.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.301+0000 7fcdd7381b80 1 -- 172.21.15.101:0/4051881982 >> 172.21.15.101:0/4051881982 conn(0x55c15de0ada0 msgr2=0x55c15de17a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:25:14.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.302+0000 7fcdd7381b80 1 -- 172.21.15.101:0/4051881982 shutdown_connections 2023-07-18T03:25:14.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.302+0000 7fcdd7381b80 1 -- 172.21.15.101:0/4051881982 wait complete. 2023-07-18T03:25:14.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.302+0000 7fcdd7381b80 20 client.28936 trim_cache size 0 max 16384 2023-07-18T03:25:14.325 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:25:14.326 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:25:14.326 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:25:14.326 INFO:teuthology.orchestra.run.smithi101.stdout: "0f2f19e0-0429-465c-935e-223eb09490c3": { 2023-07-18T03:25:14.326 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:25:14.326 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "0f2f19e0-0429-465c-935e-223eb09490c3", 2023-07-18T03:25:14.327 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:25:14.327 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:25:14.327 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:25:14.327 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:25:14.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.302+0000 7fcdc39ff700 1 -- 172.21.15.101:0/138956868 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fcda80a12c0 msgr2=0x7fcda80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:25:14.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.302+0000 7fcdc39ff700 1 --2- 172.21.15.101:0/138956868 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fcda80a12c0 0x7fcda80a3770 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcdb809a150 tx=0x7fcdb80047c0 comp rx=0 tx=0).stop 2023-07-18T03:25:14.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.302+0000 7fcdc39ff700 1 -- 172.21.15.101:0/138956868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcdbc0ffad0 msgr2=0x7fcdbc197e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:25:14.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.302+0000 7fcdc39ff700 1 --2- 172.21.15.101:0/138956868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcdbc0ffad0 0x7fcdbc197e40 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fcdb800eb30 tx=0x7fcdb800ee40 comp rx=0 tx=0).stop 2023-07-18T03:25:14.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.302+0000 7fcdc39ff700 1 -- 172.21.15.101:0/138956868 shutdown_connections 2023-07-18T03:25:14.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.302+0000 7fcdc39ff700 1 --2- 172.21.15.101:0/138956868 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fcda80a12c0 0x7fcda80a3770 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:14.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.302+0000 7fcdc39ff700 1 --2- 172.21.15.101:0/138956868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcdbc0ffad0 0x7fcdbc197e40 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:14.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.302+0000 7fcdc39ff700 1 --2- 172.21.15.101:0/138956868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdbc105ae0 0x7fcdbc19c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:14.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.302+0000 7fcdc39ff700 1 --2- 172.21.15.101:0/138956868 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcdbc100480 0x7fcdbc198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:14.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.302+0000 7fcdc39ff700 1 -- 172.21.15.101:0/138956868 >> 172.21.15.101:0/138956868 conn(0x7fcdbc0fb5f0 msgr2=0x7fcdbc10ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:25:14.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.302+0000 7fcdc39ff700 1 -- 172.21.15.101:0/138956868 shutdown_connections 2023-07-18T03:25:14.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:14.302+0000 7fcdc39ff700 1 -- 172.21.15.101:0/138956868 wait complete. 2023-07-18T03:25:14.332 INFO:tasks.cephfs.filesystem:scrub status for tag:0f2f19e0-0429-465c-935e-223eb09490c3 - {'path': '/', 'tag': '0f2f19e0-0429-465c-935e-223eb09490c3', 'options': 'recursive,force'} 2023-07-18T03:25:14.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:25:14 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:25:14.492 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:25:14 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:25:14.492 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:25:14 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:25:14.492 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:25:14 smithi101 ceph-mon[126379]: osdmap e116: 12 total, 12 up, 12 in 2023-07-18T03:25:14.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:25:14 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:25:14.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:25:14 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:25:14.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:25:14 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:25:14.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:25:14 smithi179 ceph-mon[129229]: osdmap e116: 12 total, 12 up, 12 in 2023-07-18T03:25:14.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:25:14 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:25:14.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:25:14 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:25:14.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:25:14 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:25:14.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:25:14 smithi155 ceph-mon[128885]: osdmap e116: 12 total, 12 up, 12 in 2023-07-18T03:25:15.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:25:15 smithi179 ceph-mon[129229]: pgmap v556: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.4 MiB/s wr, 244 op/s 2023-07-18T03:25:15.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:25:15 smithi101 ceph-mon[126379]: pgmap v556: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.4 MiB/s wr, 244 op/s 2023-07-18T03:25:15.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:25:15 smithi155 ceph-mon[128885]: pgmap v556: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.4 MiB/s wr, 244 op/s 2023-07-18T03:25:17.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:25:17 smithi179 ceph-mon[129229]: pgmap v558: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 238 KiB/s wr, 31 op/s 2023-07-18T03:25:17.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:25:17 smithi101 ceph-mon[126379]: pgmap v558: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 238 KiB/s wr, 31 op/s 2023-07-18T03:25:17.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:25:17 smithi155 ceph-mon[128885]: pgmap v558: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 238 KiB/s wr, 31 op/s 2023-07-18T03:25:19.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:25:19 smithi179 ceph-mon[129229]: pgmap v559: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 154 KiB/s wr, 14 op/s 2023-07-18T03:25:19.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:25:19 smithi101 ceph-mon[126379]: pgmap v559: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 154 KiB/s wr, 14 op/s 2023-07-18T03:25:19.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:25:19 smithi155 ceph-mon[128885]: pgmap v559: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 154 KiB/s wr, 14 op/s 2023-07-18T03:25:21.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:25:21 smithi179 ceph-mon[129229]: pgmap v560: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 215 KiB/s wr, 14 op/s 2023-07-18T03:25:21.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:25:21 smithi101 ceph-mon[126379]: pgmap v560: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 215 KiB/s wr, 14 op/s 2023-07-18T03:25:21.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:25:21 smithi155 ceph-mon[128885]: pgmap v560: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 215 KiB/s wr, 14 op/s 2023-07-18T03:25:23.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:25:23 smithi179 ceph-mon[129229]: pgmap v561: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 311 KiB/s wr, 12 op/s 2023-07-18T03:25:23.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:25:23 smithi101 ceph-mon[126379]: pgmap v561: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 311 KiB/s wr, 12 op/s 2023-07-18T03:25:23.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:25:23 smithi155 ceph-mon[128885]: pgmap v561: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 311 KiB/s wr, 12 op/s 2023-07-18T03:25:25.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:25:25 smithi179 ceph-mon[129229]: pgmap v562: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 311 KiB/s wr, 12 op/s 2023-07-18T03:25:25.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:25:25 smithi101 ceph-mon[126379]: pgmap v562: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 311 KiB/s wr, 12 op/s 2023-07-18T03:25:25.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:25:25 smithi155 ceph-mon[128885]: pgmap v562: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 311 KiB/s wr, 12 op/s 2023-07-18T03:25:27.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:25:27 smithi179 ceph-mon[129229]: pgmap v563: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 313 KiB/s wr, 17 op/s 2023-07-18T03:25:27.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:25:27 smithi101 ceph-mon[126379]: pgmap v563: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 313 KiB/s wr, 17 op/s 2023-07-18T03:25:27.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:25:27 smithi155 ceph-mon[128885]: pgmap v563: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 313 KiB/s wr, 17 op/s 2023-07-18T03:25:29.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:25:29 smithi179 ceph-mon[129229]: pgmap v564: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 284 KiB/s wr, 12 op/s 2023-07-18T03:25:29.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:25:29 smithi101 ceph-mon[126379]: pgmap v564: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 284 KiB/s wr, 12 op/s 2023-07-18T03:25:29.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:25:29 smithi155 ceph-mon[128885]: pgmap v564: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 284 KiB/s wr, 12 op/s 2023-07-18T03:25:31.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:25:31 smithi179 ceph-mon[129229]: pgmap v565: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 240 KiB/s wr, 16 op/s 2023-07-18T03:25:31.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:25:31 smithi101 ceph-mon[126379]: pgmap v565: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 240 KiB/s wr, 16 op/s 2023-07-18T03:25:31.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:25:31 smithi155 ceph-mon[128885]: pgmap v565: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 240 KiB/s wr, 16 op/s 2023-07-18T03:25:32.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:25:32 smithi179 ceph-mon[129229]: pgmap v566: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 509 KiB/s wr, 23 op/s 2023-07-18T03:25:32.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:25:32 smithi101 ceph-mon[126379]: pgmap v566: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 509 KiB/s wr, 23 op/s 2023-07-18T03:25:32.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:25:32 smithi155 ceph-mon[128885]: pgmap v566: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 509 KiB/s wr, 23 op/s 2023-07-18T03:25:35.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:25:35 smithi179 ceph-mon[129229]: pgmap v567: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 376 KiB/s wr, 19 op/s 2023-07-18T03:25:35.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:25:35 smithi101 ceph-mon[126379]: pgmap v567: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 376 KiB/s wr, 19 op/s 2023-07-18T03:25:35.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:25:35 smithi155 ceph-mon[128885]: pgmap v567: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 376 KiB/s wr, 19 op/s 2023-07-18T03:25:37.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:25:37 smithi101 ceph-mon[126379]: pgmap v568: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 2.5 MiB/s wr, 58 op/s 2023-07-18T03:25:37.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:25:37 smithi155 ceph-mon[128885]: pgmap v568: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 2.5 MiB/s wr, 58 op/s 2023-07-18T03:25:37.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:25:37 smithi179 ceph-mon[129229]: pgmap v568: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 2.5 MiB/s wr, 58 op/s 2023-07-18T03:25:39.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:25:39 smithi101 ceph-mon[126379]: pgmap v569: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 2.8 MiB/s wr, 56 op/s 2023-07-18T03:25:39.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:25:39 smithi155 ceph-mon[128885]: pgmap v569: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 2.8 MiB/s wr, 56 op/s 2023-07-18T03:25:39.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:25:39 smithi179 ceph-mon[129229]: pgmap v569: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 2.8 MiB/s wr, 56 op/s 2023-07-18T03:25:41.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:25:41 smithi101 ceph-mon[126379]: pgmap v570: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 2.8 MiB/s wr, 64 op/s 2023-07-18T03:25:41.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:25:41 smithi155 ceph-mon[128885]: pgmap v570: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 2.8 MiB/s wr, 64 op/s 2023-07-18T03:25:41.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:25:41 smithi179 ceph-mon[129229]: pgmap v570: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 2.8 MiB/s wr, 64 op/s 2023-07-18T03:25:43.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:25:43 smithi101 ceph-mon[126379]: pgmap v571: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 2.9 MiB/s wr, 66 op/s 2023-07-18T03:25:43.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:25:43 smithi155 ceph-mon[128885]: pgmap v571: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 2.9 MiB/s wr, 66 op/s 2023-07-18T03:25:43.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:25:43 smithi179 ceph-mon[129229]: pgmap v571: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 2.9 MiB/s wr, 66 op/s 2023-07-18T03:25:44.333 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:25:44.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.549+0000 7fb7b45ff700 1 -- 172.21.15.101:0/731820621 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7ac072850 msgr2=0x7fb7ac072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:25:44.547 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.549+0000 7fb7b45ff700 1 --2- 172.21.15.101:0/731820621 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7ac072850 0x7fb7ac072c20 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8009990 tx=0x7fb7a8009ca0 comp rx=0 tx=0).stop 2023-07-18T03:25:44.547 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.549+0000 7fb7b45ff700 1 -- 172.21.15.101:0/731820621 shutdown_connections 2023-07-18T03:25:44.547 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.549+0000 7fb7b45ff700 1 --2- 172.21.15.101:0/731820621 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7ac10c3a0 0x7fb7ac10e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:44.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.549+0000 7fb7b45ff700 1 --2- 172.21.15.101:0/731820621 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7ac072850 0x7fb7ac072c20 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:44.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.549+0000 7fb7b45ff700 1 --2- 172.21.15.101:0/731820621 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb7ac073160 0x7fb7ac10be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:44.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.549+0000 7fb7b45ff700 1 -- 172.21.15.101:0/731820621 >> 172.21.15.101:0/731820621 conn(0x7fb7ac0fb600 msgr2=0x7fb7ac0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:25:44.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.549+0000 7fb7b45ff700 1 -- 172.21.15.101:0/731820621 shutdown_connections 2023-07-18T03:25:44.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.549+0000 7fb7b45ff700 1 -- 172.21.15.101:0/731820621 wait complete. 2023-07-18T03:25:44.550 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.550+0000 7fb7b45ff700 1 Processor -- start 2023-07-18T03:25:44.550 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.550+0000 7fb7b45ff700 1 -- start start 2023-07-18T03:25:44.550 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.551+0000 7fb7b45ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb7ac072850 0x7fb7ac197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:44.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.551+0000 7fb7b45ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7ac073160 0x7fb7ac1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:44.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.551+0000 7fb7b45ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7ac10c3a0 0x7fb7ac19c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:44.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.551+0000 7fb7b45ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ac19ce90 con 0x7fb7ac072850 2023-07-18T03:25:44.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.551+0000 7fb7b45ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ac19d000 con 0x7fb7ac10c3a0 2023-07-18T03:25:44.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.551+0000 7fb7b45ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ac19d1c0 con 0x7fb7ac073160 2023-07-18T03:25:44.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.551+0000 7fb7b239b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb7ac072850 0x7fb7ac197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:44.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.551+0000 7fb7b1b9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7ac073160 0x7fb7ac1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:44.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.551+0000 7fb7b1b9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7ac073160 0x7fb7ac1983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:48338/0 (socket says 172.21.15.101:48338) 2023-07-18T03:25:44.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.551+0000 7fb7b1b9a700 1 -- 172.21.15.101:0/356343824 learned_addr learned my addr 172.21.15.101:0/356343824 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:25:44.553 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.551+0000 7fb7b2b9c700 1 --2- 172.21.15.101:0/356343824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7ac10c3a0 0x7fb7ac19c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:44.553 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.551+0000 7fb7b1b9a700 1 -- 172.21.15.101:0/356343824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7ac10c3a0 msgr2=0x7fb7ac19c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:25:44.553 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.551+0000 7fb7b1b9a700 1 --2- 172.21.15.101:0/356343824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7ac10c3a0 0x7fb7ac19c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:44.553 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.551+0000 7fb7b1b9a700 1 -- 172.21.15.101:0/356343824 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb7ac072850 msgr2=0x7fb7ac197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:25:44.553 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.551+0000 7fb7b1b9a700 1 --2- 172.21.15.101:0/356343824 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb7ac072850 0x7fb7ac197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:44.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.551+0000 7fb7b1b9a700 1 -- 172.21.15.101:0/356343824 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a80095f0 con 0x7fb7ac073160 2023-07-18T03:25:44.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.551+0000 7fb7b239b700 1 --2- 172.21.15.101:0/356343824 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb7ac072850 0x7fb7ac197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:25:44.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.551+0000 7fb7b1b9a700 1 --2- 172.21.15.101:0/356343824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7ac073160 0x7fb7ac1983f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb79c00e9a0 tx=0x7fb79c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:25:44.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.552+0000 7fb7a37fe700 1 -- 172.21.15.101:0/356343824 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c009790 con 0x7fb7ac073160 2023-07-18T03:25:44.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.552+0000 7fb7b45ff700 1 -- 172.21.15.101:0/356343824 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7ac19d4a0 con 0x7fb7ac073160 2023-07-18T03:25:44.555 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.552+0000 7fb7a37fe700 1 -- 172.21.15.101:0/356343824 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb79c010430 con 0x7fb7ac073160 2023-07-18T03:25:44.555 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.552+0000 7fb7a37fe700 1 -- 172.21.15.101:0/356343824 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c018680 con 0x7fb7ac073160 2023-07-18T03:25:44.555 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.552+0000 7fb7b45ff700 1 -- 172.21.15.101:0/356343824 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7ac1a4ce0 con 0x7fb7ac073160 2023-07-18T03:25:44.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.555+0000 7fb7a37fe700 1 -- 172.21.15.101:0/356343824 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fb79c0105a0 con 0x7fb7ac073160 2023-07-18T03:25:44.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.556+0000 7fb7c7f0fb80 1 Processor -- start 2023-07-18T03:25:44.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.557+0000 7fb7c7f0fb80 1 -- start start 2023-07-18T03:25:44.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.557+0000 7fb7c7f0fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56130001d2d0 0x56130001d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:44.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.557+0000 7fb7c7f0fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56130001dbe0 0x561300028590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:44.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.557+0000 7fb7c7f0fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561300028ad0 0x56130002aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:44.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.557+0000 7fb7c7f0fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612fff23cb0 con 0x56130001dbe0 2023-07-18T03:25:44.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.557+0000 7fb7c7f0fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612fff76a40 con 0x561300028ad0 2023-07-18T03:25:44.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.557+0000 7fb7c7f0fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612fff6fed0 con 0x56130001d2d0 2023-07-18T03:25:44.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.557+0000 7fb7a37fe700 1 --2- 172.21.15.101:0/356343824 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fb7980a12c0 0x7fb7980a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:44.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.558+0000 7fb7b1b9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561300028ad0 0x56130002aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:44.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.558+0000 7fb7b239b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56130001dbe0 0x561300028590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:44.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.558+0000 7fb7b2b9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56130001d2d0 0x56130001d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:44.562 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.558+0000 7fb7b2b9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56130001d2d0 0x56130001d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:48352/0 (socket says 172.21.15.101:48352) 2023-07-18T03:25:44.562 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.558+0000 7fb7b2b9c700 1 -- 172.21.15.101:0/569545292 learned_addr learned my addr 172.21.15.101:0/569545292 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:25:44.562 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.558+0000 7fb7b1b9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561300028ad0 0x56130002aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:43780/0 (socket says 172.21.15.101:43780) 2023-07-18T03:25:44.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.558+0000 7fb7a37fe700 1 -- 172.21.15.101:0/356343824 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fb79c014070 con 0x7fb7ac073160 2023-07-18T03:25:44.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.558+0000 7fb7b2b9c700 1 --2- 172.21.15.101:0/356343824 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fb7980a12c0 0x7fb7980a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:44.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.558+0000 7fb7b1b9a700 1 -- 172.21.15.101:0/569545292 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56130001d2d0 msgr2=0x56130001d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:25:44.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.558+0000 7fb7b1b9a700 1 --2- 172.21.15.101:0/569545292 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56130001d2d0 0x56130001d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:44.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.558+0000 7fb7b1b9a700 1 -- 172.21.15.101:0/569545292 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56130001dbe0 msgr2=0x561300028590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:25:44.564 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.558+0000 7fb7b1b9a700 1 --2- 172.21.15.101:0/569545292 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56130001dbe0 0x561300028590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:44.564 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.558+0000 7fb7b1b9a700 1 -- 172.21.15.101:0/569545292 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612ffd9dde0 con 0x561300028ad0 2023-07-18T03:25:44.564 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.558+0000 7fb7b1b9a700 1 --2- 172.21.15.101:0/569545292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561300028ad0 0x56130002aeb0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb79c0ca620 tx=0x7fb79c010c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=138e3a5a6ef9835f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:25:44.564 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.558+0000 7fb7b2b9c700 1 --2- 172.21.15.101:0/356343824 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fb7980a12c0 0x7fb7980a3770 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4009ec0 tx=0x7fb7a400b040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:25:44.564 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.558+0000 7fb7a0bfe700 1 -- 172.21.15.101:0/569545292 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c099fa0 con 0x561300028ad0 2023-07-18T03:25:44.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.558+0000 7fb7a0bfe700 1 -- 172.21.15.101:0/569545292 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb79c09dfd0 con 0x561300028ad0 2023-07-18T03:25:44.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.558+0000 7fb7a0bfe700 1 -- 172.21.15.101:0/569545292 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c09a200 con 0x561300028ad0 2023-07-18T03:25:44.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.559+0000 7fb7c7f0fb80 1 -- 172.21.15.101:0/569545292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561300028ad0 msgr2=0x56130002aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:25:44.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.559+0000 7fb7c7f0fb80 1 --2- 172.21.15.101:0/569545292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561300028ad0 0x56130002aeb0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb79c0ca620 tx=0x7fb79c010c80 comp rx=0 tx=0).stop 2023-07-18T03:25:44.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.559+0000 7fb7c7f0fb80 1 -- 172.21.15.101:0/569545292 shutdown_connections 2023-07-18T03:25:44.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.559+0000 7fb7c7f0fb80 1 --2- 172.21.15.101:0/569545292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561300028ad0 0x56130002aeb0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:44.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.559+0000 7fb7c7f0fb80 1 --2- 172.21.15.101:0/569545292 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56130001d2d0 0x56130001d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:44.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.559+0000 7fb7c7f0fb80 1 --2- 172.21.15.101:0/569545292 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56130001dbe0 0x561300028590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:44.567 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.559+0000 7fb7c7f0fb80 1 -- 172.21.15.101:0/569545292 >> 172.21.15.101:0/569545292 conn(0x561300013da0 msgr2=0x5613000196a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:25:44.567 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.559+0000 7fb7c7f0fb80 1 -- 172.21.15.101:0/569545292 shutdown_connections 2023-07-18T03:25:44.567 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.559+0000 7fb7c7f0fb80 1 -- 172.21.15.101:0/569545292 wait complete. 2023-07-18T03:25:44.567 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.560+0000 7fb7c7f0fb80 1 -- start start 2023-07-18T03:25:44.567 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.560+0000 7fb7c7f0fb80 1 Processor -- start 2023-07-18T03:25:44.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.560+0000 7fb7c7f0fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56130001d2d0 0x5613000284f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:44.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.560+0000 7fb7c7f0fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56130001dbe0 0x561300025770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:44.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.560+0000 7fb7c7f0fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561300028ad0 0x561300025e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:44.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.560+0000 7fb7c7f0fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612fff0fac0 con 0x56130001d2d0 2023-07-18T03:25:44.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.560+0000 7fb7c7f0fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612fff0f7c0 con 0x561300028ad0 2023-07-18T03:25:44.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.560+0000 7fb7c7f0fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612ffdf3920 con 0x56130001dbe0 2023-07-18T03:25:44.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.561+0000 7fb7c7f0fb80 10 client.0 _ll_register_callbacks cb 0x5612fff0d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:25:44.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.561+0000 7fb7b1b9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56130001dbe0 0x561300025770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:44.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.561+0000 7fb7b239b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56130001d2d0 0x5613000284f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:44.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.561+0000 7fb7b2b9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561300028ad0 0x561300025e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:44.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.561+0000 7fb7b1b9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56130001dbe0 0x561300025770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:48364/0 (socket says 172.21.15.101:48364) 2023-07-18T03:25:44.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.561+0000 7fb7b1b9a700 1 -- 172.21.15.101:0/618152686 learned_addr learned my addr 172.21.15.101:0/618152686 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:25:44.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.561+0000 7fb7b1b9a700 1 -- 172.21.15.101:0/618152686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561300028ad0 msgr2=0x561300025e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:25:44.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.561+0000 7fb7b1b9a700 1 --2- 172.21.15.101:0/618152686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561300028ad0 0x561300025e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:44.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.561+0000 7fb7b1b9a700 1 -- 172.21.15.101:0/618152686 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56130001d2d0 msgr2=0x5613000284f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:25:44.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.561+0000 7fb7b1b9a700 1 --2- 172.21.15.101:0/618152686 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56130001d2d0 0x5613000284f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:44.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.561+0000 7fb7b1b9a700 1 -- 172.21.15.101:0/618152686 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7ac1a4ce0 con 0x56130001dbe0 2023-07-18T03:25:44.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.561+0000 7fb7b2b9c700 1 --2- 172.21.15.101:0/618152686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561300028ad0 0x561300025e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:25:44.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.562+0000 7fb7b1b9a700 1 --2- 172.21.15.101:0/618152686 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56130001dbe0 0x561300025770 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb79c0a1ec0 tx=0x7fb79c0cd160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9d8a67315d0c8a8 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:25:44.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.562+0000 7fb795ffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:25:44.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.562+0000 7fb795ffb700 1 -- 172.21.15.101:0/618152686 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c099fa0 con 0x56130001dbe0 2023-07-18T03:25:44.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.562+0000 7fb795ffb700 1 -- 172.21.15.101:0/618152686 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fb79c0996a0 con 0x56130001dbe0 2023-07-18T03:25:44.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.562+0000 7fb7c7f0fb80 1 -- 172.21.15.101:0/618152686 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5612ffec94f0 con 0x56130001dbe0 2023-07-18T03:25:44.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.562+0000 7fb795ffb700 1 -- 172.21.15.101:0/618152686 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb79c09a200 con 0x56130001dbe0 2023-07-18T03:25:44.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.562+0000 7fb7c7f0fb80 10 client.28948 fetch_fsmap learned FSMap version 43 2023-07-18T03:25:44.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.562+0000 7fb7c7f0fb80 1 -- 172.21.15.101:0/618152686 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x5612ffd86c60 con 0x56130001dbe0 2023-07-18T03:25:44.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.562+0000 7fb795ffb700 1 -- 172.21.15.101:0/618152686 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5612ffec94f0 con 0x56130001dbe0 2023-07-18T03:25:44.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.562+0000 7fb795ffb700 1 -- 172.21.15.101:0/618152686 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 43) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fb79c0b81b0 con 0x56130001dbe0 2023-07-18T03:25:44.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.563+0000 7fb7c7f0fb80 10 client.28948 fetch_fsmap finished waiting for FSMap version 43 2023-07-18T03:25:44.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.563+0000 7fb7c7f0fb80 10 client.28948 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:25:44.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.563+0000 7fb7c7f0fb80 20 client.28948 populate_metadata read hostname 'smithi101' 2023-07-18T03:25:44.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.563+0000 7fb7c7f0fb80 1 --2- 172.21.15.101:0/618152686 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x561300037f30 0x56130003a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:44.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.563+0000 7fb7c7f0fb80 4 client.28948 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:25:44.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.563+0000 7fb7c7f0fb80 1 -- 172.21.15.101:0/618152686 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5612fff68eb0 con 0x561300037f30 2023-07-18T03:25:44.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.563+0000 7fb7b239b700 1 --2- 172.21.15.101:0/618152686 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x561300037f30 0x56130003a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:44.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.563+0000 7fb795ffb700 10 client.28948 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:25:44.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.563+0000 7fb7b239b700 1 --2- 172.21.15.101:0/618152686 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x561300037f30 0x56130003a310 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4eccb0bf85cddf server_cookie=3c8729aea9675af9 in_seq=0 out_seq=0 2023-07-18T03:25:44.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.564+0000 7fb795ffb700 1 -- 172.21.15.101:0/618152686 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5612fff68eb0 con 0x561300037f30 2023-07-18T03:25:44.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.564+0000 7fb795ffb700 10 client.28948 handle_command_reply: tid=0 2023-07-18T03:25:44.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.564+0000 7fb7c7f0fb80 1 client.28948 shutdown 2023-07-18T03:25:44.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.565+0000 7fb7c7f0fb80 1 -- 172.21.15.101:0/618152686 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56130001dbe0 msgr2=0x561300025770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:25:44.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.565+0000 7fb7c7f0fb80 1 --2- 172.21.15.101:0/618152686 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56130001dbe0 0x561300025770 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb79c0a1ec0 tx=0x7fb79c0cd160 comp rx=0 tx=0).stop 2023-07-18T03:25:44.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.565+0000 7fb7c7f0fb80 1 -- 172.21.15.101:0/618152686 shutdown_connections 2023-07-18T03:25:44.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.565+0000 7fb7c7f0fb80 1 --2- 172.21.15.101:0/618152686 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x561300037f30 0x56130003a310 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:44.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.565+0000 7fb7c7f0fb80 1 --2- 172.21.15.101:0/618152686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561300028ad0 0x561300025e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:44.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.565+0000 7fb795ffb700 0 client.28948 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:25:44.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.565+0000 7fb7c7f0fb80 1 --2- 172.21.15.101:0/618152686 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56130001d2d0 0x5613000284f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:44.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.565+0000 7fb7c7f0fb80 1 --2- 172.21.15.101:0/618152686 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56130001dbe0 0x561300025770 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:44.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.565+0000 7fb7c7f0fb80 1 -- 172.21.15.101:0/618152686 >> 172.21.15.101:0/618152686 conn(0x561300013da0 msgr2=0x561300019230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:25:44.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.565+0000 7fb7c7f0fb80 1 -- 172.21.15.101:0/618152686 shutdown_connections 2023-07-18T03:25:44.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.565+0000 7fb7c7f0fb80 1 -- 172.21.15.101:0/618152686 wait complete. 2023-07-18T03:25:44.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.565+0000 7fb7c7f0fb80 20 client.28948 trim_cache size 0 max 16384 2023-07-18T03:25:44.582 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.582+0000 7fb7c7f0fb80 1 Processor -- start 2023-07-18T03:25:44.582 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.582+0000 7fb7c7f0fb80 1 -- start start 2023-07-18T03:25:44.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.583+0000 7fb7c7f0fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56130001d2d0 0x561300020a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:44.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.583+0000 7fb7c7f0fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56130001dbe0 0x561300020f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:44.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.583+0000 7fb7c7f0fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561300028ad0 0x5613000229e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:44.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.583+0000 7fb7c7f0fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56130001dfb0 con 0x56130001d2d0 2023-07-18T03:25:44.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.583+0000 7fb7c7f0fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56130001a2e0 con 0x56130001dbe0 2023-07-18T03:25:44.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.583+0000 7fb7c7f0fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x561300026000 con 0x561300028ad0 2023-07-18T03:25:44.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.583+0000 7fb7b239b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56130001d2d0 0x561300020a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:44.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.583+0000 7fb7b2b9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561300028ad0 0x5613000229e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:44.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.583+0000 7fb7b239b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56130001d2d0 0x561300020a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:35742/0 (socket says 172.21.15.101:35742) 2023-07-18T03:25:44.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.583+0000 7fb7b1b9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56130001dbe0 0x561300020f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:44.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.583+0000 7fb7b239b700 1 -- 172.21.15.101:0/1391324641 learned_addr learned my addr 172.21.15.101:0/1391324641 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:25:44.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.583+0000 7fb7b239b700 1 -- 172.21.15.101:0/1391324641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561300028ad0 msgr2=0x5613000229e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:25:44.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.583+0000 7fb7b239b700 1 --2- 172.21.15.101:0/1391324641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561300028ad0 0x5613000229e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:44.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.583+0000 7fb7b239b700 1 -- 172.21.15.101:0/1391324641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56130001dbe0 msgr2=0x561300020f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:25:44.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.583+0000 7fb7b239b700 1 --2- 172.21.15.101:0/1391324641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56130001dbe0 0x561300020f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:44.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.583+0000 7fb7b239b700 1 -- 172.21.15.101:0/1391324641 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612ffdc4250 con 0x56130001d2d0 2023-07-18T03:25:44.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.584+0000 7fb7b239b700 1 --2- 172.21.15.101:0/1391324641 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56130001d2d0 0x561300020a50 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8000c00 tx=0x7fb7a800bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b5eb2e69eedb291 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:25:44.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.584+0000 7fb7a13ff700 1 -- 172.21.15.101:0/1391324641 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a801d040 con 0x56130001d2d0 2023-07-18T03:25:44.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.584+0000 7fb7a13ff700 1 -- 172.21.15.101:0/1391324641 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7a801e040 con 0x56130001d2d0 2023-07-18T03:25:44.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.584+0000 7fb7a13ff700 1 -- 172.21.15.101:0/1391324641 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a8004500 con 0x56130001d2d0 2023-07-18T03:25:44.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.584+0000 7fb7c7f0fb80 1 -- 172.21.15.101:0/1391324641 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56130001d2d0 msgr2=0x561300020a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:25:44.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.584+0000 7fb7c7f0fb80 1 --2- 172.21.15.101:0/1391324641 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56130001d2d0 0x561300020a50 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8000c00 tx=0x7fb7a800bcf0 comp rx=0 tx=0).stop 2023-07-18T03:25:44.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.584+0000 7fb7c7f0fb80 1 -- 172.21.15.101:0/1391324641 shutdown_connections 2023-07-18T03:25:44.589 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.584+0000 7fb7c7f0fb80 1 --2- 172.21.15.101:0/1391324641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561300028ad0 0x5613000229e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:44.589 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.584+0000 7fb7c7f0fb80 1 --2- 172.21.15.101:0/1391324641 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56130001d2d0 0x561300020a50 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:44.589 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.584+0000 7fb7c7f0fb80 1 --2- 172.21.15.101:0/1391324641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56130001dbe0 0x561300020f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:44.589 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.585+0000 7fb7c7f0fb80 1 -- 172.21.15.101:0/1391324641 >> 172.21.15.101:0/1391324641 conn(0x561300013da0 msgr2=0x561300028120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:25:44.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.585+0000 7fb7c7f0fb80 1 -- 172.21.15.101:0/1391324641 shutdown_connections 2023-07-18T03:25:44.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.585+0000 7fb7c7f0fb80 1 -- 172.21.15.101:0/1391324641 wait complete. 2023-07-18T03:25:44.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.585+0000 7fb7c7f0fb80 1 -- start start 2023-07-18T03:25:44.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.585+0000 7fb7c7f0fb80 1 Processor -- start 2023-07-18T03:25:44.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.586+0000 7fb7c7f0fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56130001d2d0 0x56130002f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:44.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.586+0000 7fb7c7f0fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56130001dbe0 0x56130002d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:44.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.586+0000 7fb7c7f0fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561300028ad0 0x56130002dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:44.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.586+0000 7fb7c7f0fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56130002e150 con 0x561300028ad0 2023-07-18T03:25:44.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.586+0000 7fb7c7f0fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56130002e2c0 con 0x56130001d2d0 2023-07-18T03:25:44.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.586+0000 7fb7c7f0fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56130002e480 con 0x56130001dbe0 2023-07-18T03:25:44.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.586+0000 7fb7c7f0fb80 10 client.0 _ll_register_callbacks cb 0x7fb79c005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:25:44.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.586+0000 7fb7b239b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56130001d2d0 0x56130002f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:44.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.586+0000 7fb7b239b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56130001d2d0 0x56130002f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:43814/0 (socket says 172.21.15.101:43814) 2023-07-18T03:25:44.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.586+0000 7fb7b239b700 1 -- 172.21.15.101:0/658448787 learned_addr learned my addr 172.21.15.101:0/658448787 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:25:44.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.586+0000 7fb7b1b9a700 1 --2- 172.21.15.101:0/658448787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56130001dbe0 0x56130002d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:44.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.586+0000 7fb7b2b9c700 1 --2- 172.21.15.101:0/658448787 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561300028ad0 0x56130002dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:44.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.587+0000 7fb7b239b700 1 -- 172.21.15.101:0/658448787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56130001dbe0 msgr2=0x56130002d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:25:44.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.587+0000 7fb7b239b700 1 --2- 172.21.15.101:0/658448787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56130001dbe0 0x56130002d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:44.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.587+0000 7fb7b239b700 1 -- 172.21.15.101:0/658448787 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561300028ad0 msgr2=0x56130002dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:25:44.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.587+0000 7fb7b239b700 1 --2- 172.21.15.101:0/658448787 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561300028ad0 0x56130002dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:44.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.587+0000 7fb7b239b700 1 -- 172.21.15.101:0/658448787 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7ac10f3c0 con 0x56130001d2d0 2023-07-18T03:25:44.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.587+0000 7fb7b1b9a700 1 --2- 172.21.15.101:0/658448787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56130001dbe0 0x56130002d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:25:44.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.587+0000 7fb7b239b700 1 --2- 172.21.15.101:0/658448787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56130001d2d0 0x56130002f360 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb7a800b2e0 tx=0x7fb7a8005190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cd2efd8c5b7cc22 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:25:44.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.587+0000 7fb795ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:25:44.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.587+0000 7fb795ffb700 1 -- 172.21.15.101:0/658448787 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a801d070 con 0x56130001d2d0 2023-07-18T03:25:44.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.587+0000 7fb795ffb700 1 -- 172.21.15.101:0/658448787 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fb7a8005560 con 0x56130001d2d0 2023-07-18T03:25:44.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.587+0000 7fb7c7f0fb80 1 -- 172.21.15.101:0/658448787 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb79c09dfd0 con 0x56130001d2d0 2023-07-18T03:25:44.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.588+0000 7fb795ffb700 1 -- 172.21.15.101:0/658448787 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7a8029150 con 0x56130001d2d0 2023-07-18T03:25:44.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.588+0000 7fb7c7f0fb80 10 client.28964 fetch_fsmap learned FSMap version 43 2023-07-18T03:25:44.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.588+0000 7fb7c7f0fb80 1 -- 172.21.15.101:0/658448787 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x5612ffd9c2b0 con 0x56130001d2d0 2023-07-18T03:25:44.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.588+0000 7fb795ffb700 1 -- 172.21.15.101:0/658448787 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb79c09dfd0 con 0x56130001d2d0 2023-07-18T03:25:44.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.588+0000 7fb795ffb700 1 -- 172.21.15.101:0/658448787 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 43) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fb7a8030020 con 0x56130001d2d0 2023-07-18T03:25:44.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.588+0000 7fb7c7f0fb80 10 client.28964 fetch_fsmap finished waiting for FSMap version 43 2023-07-18T03:25:44.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.588+0000 7fb7c7f0fb80 10 client.28964 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:25:44.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.588+0000 7fb7c7f0fb80 20 client.28964 populate_metadata read hostname 'smithi101' 2023-07-18T03:25:44.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.588+0000 7fb7c7f0fb80 1 --2- 172.21.15.101:0/658448787 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56130008cef0 0x56130008f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:44.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.588+0000 7fb7c7f0fb80 4 client.28964 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:25:44.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.588+0000 7fb7c7f0fb80 1 -- 172.21.15.101:0/658448787 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5612fff2b050 con 0x56130008cef0 2023-07-18T03:25:44.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.588+0000 7fb7b1b9a700 1 --2- 172.21.15.101:0/658448787 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56130008cef0 0x56130008f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:44.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.589+0000 7fb795ffb700 10 client.28964 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:25:44.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.589+0000 7fb7b1b9a700 1 --2- 172.21.15.101:0/658448787 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56130008cef0 0x56130008f2d0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=535d6e6d68855043 server_cookie=fe73c3404a644949 in_seq=0 out_seq=0 2023-07-18T03:25:44.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.589+0000 7fb795ffb700 1 -- 172.21.15.101:0/658448787 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5612fff2b050 con 0x56130008cef0 2023-07-18T03:25:44.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.589+0000 7fb795ffb700 10 client.28964 handle_command_reply: tid=0 2023-07-18T03:25:44.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.589+0000 7fb7c7f0fb80 1 client.28964 shutdown 2023-07-18T03:25:44.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.589+0000 7fb7c7f0fb80 1 -- 172.21.15.101:0/658448787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56130001d2d0 msgr2=0x56130002f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:25:44.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.589+0000 7fb7c7f0fb80 1 --2- 172.21.15.101:0/658448787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56130001d2d0 0x56130002f360 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb7a800b2e0 tx=0x7fb7a8005190 comp rx=0 tx=0).stop 2023-07-18T03:25:44.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.589+0000 7fb7c7f0fb80 1 -- 172.21.15.101:0/658448787 shutdown_connections 2023-07-18T03:25:44.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.589+0000 7fb7c7f0fb80 1 --2- 172.21.15.101:0/658448787 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56130008cef0 0x56130008f2d0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:44.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.589+0000 7fb7c7f0fb80 1 --2- 172.21.15.101:0/658448787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56130001d2d0 0x56130002f360 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:44.604 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.589+0000 7fb795ffb700 0 client.28964 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:25:44.604 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.589+0000 7fb7c7f0fb80 1 --2- 172.21.15.101:0/658448787 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561300028ad0 0x56130002dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:44.604 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.589+0000 7fb7c7f0fb80 1 --2- 172.21.15.101:0/658448787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56130001dbe0 0x56130002d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:44.604 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.589+0000 7fb7c7f0fb80 1 -- 172.21.15.101:0/658448787 >> 172.21.15.101:0/658448787 conn(0x561300013da0 msgr2=0x561300020a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:25:44.604 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.589+0000 7fb7c7f0fb80 1 -- 172.21.15.101:0/658448787 shutdown_connections 2023-07-18T03:25:44.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.589+0000 7fb7c7f0fb80 1 -- 172.21.15.101:0/658448787 wait complete. 2023-07-18T03:25:44.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.590+0000 7fb7c7f0fb80 20 client.28964 trim_cache size 0 max 16384 2023-07-18T03:25:44.605 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:25:44.606 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:25:44.606 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:25:44.606 INFO:teuthology.orchestra.run.smithi101.stdout: "0f2f19e0-0429-465c-935e-223eb09490c3": { 2023-07-18T03:25:44.606 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:25:44.606 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "0f2f19e0-0429-465c-935e-223eb09490c3", 2023-07-18T03:25:44.607 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:25:44.607 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:25:44.607 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:25:44.607 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:25:44.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.590+0000 7fb7b45ff700 1 -- 172.21.15.101:0/356343824 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fb7980a12c0 msgr2=0x7fb7980a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:25:44.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.590+0000 7fb7b45ff700 1 --2- 172.21.15.101:0/356343824 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fb7980a12c0 0x7fb7980a3770 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4009ec0 tx=0x7fb7a400b040 comp rx=0 tx=0).stop 2023-07-18T03:25:44.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.590+0000 7fb7b45ff700 1 -- 172.21.15.101:0/356343824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7ac073160 msgr2=0x7fb7ac1983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:25:44.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.590+0000 7fb7b45ff700 1 --2- 172.21.15.101:0/356343824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7ac073160 0x7fb7ac1983f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb79c00e9a0 tx=0x7fb79c00ecb0 comp rx=0 tx=0).stop 2023-07-18T03:25:44.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.590+0000 7fb7b45ff700 1 -- 172.21.15.101:0/356343824 shutdown_connections 2023-07-18T03:25:44.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.590+0000 7fb7b45ff700 1 --2- 172.21.15.101:0/356343824 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fb7980a12c0 0x7fb7980a3770 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:44.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.590+0000 7fb7b45ff700 1 --2- 172.21.15.101:0/356343824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7ac10c3a0 0x7fb7ac19c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:44.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.590+0000 7fb7b45ff700 1 --2- 172.21.15.101:0/356343824 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb7ac072850 0x7fb7ac197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:44.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.590+0000 7fb7b45ff700 1 --2- 172.21.15.101:0/356343824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7ac073160 0x7fb7ac1983f0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:44.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.590+0000 7fb7b45ff700 1 -- 172.21.15.101:0/356343824 >> 172.21.15.101:0/356343824 conn(0x7fb7ac0fb600 msgr2=0x7fb7ac10e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:25:44.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.590+0000 7fb7b45ff700 1 -- 172.21.15.101:0/356343824 shutdown_connections 2023-07-18T03:25:44.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:25:44.590+0000 7fb7b45ff700 1 -- 172.21.15.101:0/356343824 wait complete. 2023-07-18T03:25:44.611 INFO:tasks.cephfs.filesystem:scrub status for tag:0f2f19e0-0429-465c-935e-223eb09490c3 - {'path': '/', 'tag': '0f2f19e0-0429-465c-935e-223eb09490c3', 'options': 'recursive,force'} 2023-07-18T03:25:45.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:25:45 smithi101 ceph-mon[126379]: pgmap v572: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 2.6 MiB/s wr, 57 op/s 2023-07-18T03:25:45.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:25:45 smithi155 ceph-mon[128885]: pgmap v572: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 2.6 MiB/s wr, 57 op/s 2023-07-18T03:25:45.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:25:45 smithi179 ceph-mon[129229]: pgmap v572: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 2.6 MiB/s wr, 57 op/s 2023-07-18T03:25:47.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:25:47 smithi101 ceph-mon[126379]: pgmap v573: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 2.6 MiB/s wr, 63 op/s 2023-07-18T03:25:47.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:25:47 smithi155 ceph-mon[128885]: pgmap v573: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 2.6 MiB/s wr, 63 op/s 2023-07-18T03:25:47.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:25:47 smithi179 ceph-mon[129229]: pgmap v573: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 2.6 MiB/s wr, 63 op/s 2023-07-18T03:25:49.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:25:49 smithi101 ceph-mon[126379]: pgmap v574: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 476 KiB/s wr, 24 op/s 2023-07-18T03:25:49.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:25:49 smithi155 ceph-mon[128885]: pgmap v574: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 476 KiB/s wr, 24 op/s 2023-07-18T03:25:49.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:25:49 smithi179 ceph-mon[129229]: pgmap v574: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 476 KiB/s wr, 24 op/s 2023-07-18T03:25:51.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:25:51 smithi101 ceph-mon[126379]: pgmap v575: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 156 KiB/s wr, 22 op/s 2023-07-18T03:25:51.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:25:51 smithi155 ceph-mon[128885]: pgmap v575: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 156 KiB/s wr, 22 op/s 2023-07-18T03:25:51.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:25:51 smithi179 ceph-mon[129229]: pgmap v575: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 156 KiB/s wr, 22 op/s 2023-07-18T03:25:53.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:25:53 smithi101 ceph-mon[126379]: pgmap v576: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 155 KiB/s wr, 16 op/s 2023-07-18T03:25:53.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:25:53 smithi155 ceph-mon[128885]: pgmap v576: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 155 KiB/s wr, 16 op/s 2023-07-18T03:25:53.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:25:53 smithi179 ceph-mon[129229]: pgmap v576: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 155 KiB/s wr, 16 op/s 2023-07-18T03:25:55.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:25:55 smithi101 ceph-mon[126379]: pgmap v577: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 52 KiB/s wr, 10 op/s 2023-07-18T03:25:55.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:25:55 smithi155 ceph-mon[128885]: pgmap v577: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 52 KiB/s wr, 10 op/s 2023-07-18T03:25:55.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:25:55 smithi179 ceph-mon[129229]: pgmap v577: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 52 KiB/s wr, 10 op/s 2023-07-18T03:25:57.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:25:57 smithi155 ceph-mon[128885]: pgmap v578: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 88 KiB/s wr, 15 op/s 2023-07-18T03:25:57.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:25:57 smithi155 ceph-mon[128885]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:25:57.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:25:57 smithi179 ceph-mon[129229]: pgmap v578: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 88 KiB/s wr, 15 op/s 2023-07-18T03:25:57.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:25:57 smithi179 ceph-mon[129229]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:25:57.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:25:57 smithi101 ceph-mon[126379]: pgmap v578: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 88 KiB/s wr, 15 op/s 2023-07-18T03:25:57.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:25:57 smithi101 ceph-mon[126379]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:25:59.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:25:59 smithi155 ceph-mon[128885]: pgmap v579: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 53 KiB/s wr, 9 op/s 2023-07-18T03:25:59.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:25:59 smithi179 ceph-mon[129229]: pgmap v579: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 53 KiB/s wr, 9 op/s 2023-07-18T03:25:59.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:25:59 smithi101 ceph-mon[126379]: pgmap v579: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 53 KiB/s wr, 9 op/s 2023-07-18T03:26:01.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:26:01 smithi155 ceph-mon[128885]: pgmap v580: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 169 KiB/s wr, 14 op/s 2023-07-18T03:26:01.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:26:01 smithi179 ceph-mon[129229]: pgmap v580: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 169 KiB/s wr, 14 op/s 2023-07-18T03:26:01.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:26:01 smithi101 ceph-mon[126379]: pgmap v580: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 169 KiB/s wr, 14 op/s 2023-07-18T03:26:02.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:26:02 smithi155 ceph-mon[128885]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:26:02.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:26:02 smithi179 ceph-mon[129229]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:26:02.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:26:02 smithi101 ceph-mon[126379]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:26:03.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:26:03 smithi155 ceph-mon[128885]: pgmap v581: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 182 KiB/s wr, 15 op/s 2023-07-18T03:26:03.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:26:03 smithi179 ceph-mon[129229]: pgmap v581: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 182 KiB/s wr, 15 op/s 2023-07-18T03:26:03.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:26:03 smithi101 ceph-mon[126379]: pgmap v581: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 182 KiB/s wr, 15 op/s 2023-07-18T03:26:05.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:26:05 smithi155 ceph-mon[128885]: pgmap v582: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 175 KiB/s wr, 13 op/s 2023-07-18T03:26:05.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:26:05 smithi179 ceph-mon[129229]: pgmap v582: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 175 KiB/s wr, 13 op/s 2023-07-18T03:26:05.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:26:05 smithi101 ceph-mon[126379]: pgmap v582: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 175 KiB/s wr, 13 op/s 2023-07-18T03:26:07.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:26:07 smithi155 ceph-mon[128885]: pgmap v583: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 408 KiB/s wr, 36 op/s 2023-07-18T03:26:07.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:26:07 smithi155 ceph-mon[128885]: osdmap e117: 12 total, 12 up, 12 in 2023-07-18T03:26:07.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:26:07 smithi179 ceph-mon[129229]: pgmap v583: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 408 KiB/s wr, 36 op/s 2023-07-18T03:26:07.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:26:07 smithi179 ceph-mon[129229]: osdmap e117: 12 total, 12 up, 12 in 2023-07-18T03:26:07.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:26:07 smithi101 ceph-mon[126379]: pgmap v583: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 408 KiB/s wr, 36 op/s 2023-07-18T03:26:07.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:26:07 smithi101 ceph-mon[126379]: osdmap e117: 12 total, 12 up, 12 in 2023-07-18T03:26:09.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:26:09 smithi155 ceph-mon[128885]: pgmap v585: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 446 KiB/s wr, 40 op/s 2023-07-18T03:26:09.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:26:09 smithi179 ceph-mon[129229]: pgmap v585: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 446 KiB/s wr, 40 op/s 2023-07-18T03:26:09.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:26:09 smithi101 ceph-mon[126379]: pgmap v585: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 446 KiB/s wr, 40 op/s 2023-07-18T03:26:11.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:26:11 smithi155 ceph-mon[128885]: pgmap v586: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 488 KiB/s wr, 52 op/s 2023-07-18T03:26:11.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:26:11 smithi179 ceph-mon[129229]: pgmap v586: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 488 KiB/s wr, 52 op/s 2023-07-18T03:26:11.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:26:11 smithi101 ceph-mon[126379]: pgmap v586: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 488 KiB/s wr, 52 op/s 2023-07-18T03:26:13.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:26:13 smithi155 ceph-mon[128885]: pgmap v587: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 470 KiB/s wr, 54 op/s 2023-07-18T03:26:13.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:26:13 smithi179 ceph-mon[129229]: pgmap v587: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 470 KiB/s wr, 54 op/s 2023-07-18T03:26:13.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:26:13 smithi101 ceph-mon[126379]: pgmap v587: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 470 KiB/s wr, 54 op/s 2023-07-18T03:26:14.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:26:14 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:26:14.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:26:14 smithi155 ceph-mon[128885]: osdmap e118: 12 total, 12 up, 12 in 2023-07-18T03:26:14.613 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:26:14.677 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:26:14 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:26:14.677 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:26:14 smithi101 ceph-mon[126379]: osdmap e118: 12 total, 12 up, 12 in 2023-07-18T03:26:14.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:26:14 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:26:14.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:26:14 smithi179 ceph-mon[129229]: osdmap e118: 12 total, 12 up, 12 in 2023-07-18T03:26:14.819 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.823+0000 7fc5701ff700 1 -- 172.21.15.101:0/4167651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5680ffae0 msgr2=0x7fc5680fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:26:14.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.823+0000 7fc5701ff700 1 --2- 172.21.15.101:0/4167651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5680ffae0 0x7fc5680fff50 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fc558009990 tx=0x7fc558009ca0 comp rx=0 tx=0).stop 2023-07-18T03:26:14.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.823+0000 7fc5701ff700 1 -- 172.21.15.101:0/4167651 shutdown_connections 2023-07-18T03:26:14.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.823+0000 7fc5701ff700 1 --2- 172.21.15.101:0/4167651 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc568100490 0x7fc56810c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:14.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.823+0000 7fc5701ff700 1 --2- 172.21.15.101:0/4167651 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc568105af0 0x7fc568105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:14.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.823+0000 7fc5701ff700 1 --2- 172.21.15.101:0/4167651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5680ffae0 0x7fc5680fff50 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:14.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.823+0000 7fc5701ff700 1 -- 172.21.15.101:0/4167651 >> 172.21.15.101:0/4167651 conn(0x7fc5680fb5e0 msgr2=0x7fc5680fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:26:14.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.823+0000 7fc5701ff700 1 -- 172.21.15.101:0/4167651 shutdown_connections 2023-07-18T03:26:14.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.823+0000 7fc5701ff700 1 -- 172.21.15.101:0/4167651 wait complete. 2023-07-18T03:26:14.823 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.824+0000 7fc5701ff700 1 Processor -- start 2023-07-18T03:26:14.823 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.824+0000 7fc5701ff700 1 -- start start 2023-07-18T03:26:14.824 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.824+0000 7fc5701ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc5680ffae0 0x7fc568072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:26:14.824 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.824+0000 7fc5701ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc568100490 0x7fc568072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:26:14.824 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.824+0000 7fc5701ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc568105af0 0x7fc5681a3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:26:14.824 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.824+0000 7fc5701ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc568073580 con 0x7fc5680ffae0 2023-07-18T03:26:14.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.824+0000 7fc5701ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5681a42b0 con 0x7fc568100490 2023-07-18T03:26:14.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.824+0000 7fc5701ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5681a43f0 con 0x7fc568105af0 2023-07-18T03:26:14.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.825+0000 7fc56df9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc5680ffae0 0x7fc568072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:26:14.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.825+0000 7fc56d79a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc568100490 0x7fc568072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:26:14.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.825+0000 7fc56e79c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc568105af0 0x7fc5681a3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:26:14.826 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.825+0000 7fc56d79a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc568100490 0x7fc568072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:40004/0 (socket says 172.21.15.101:40004) 2023-07-18T03:26:14.826 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.825+0000 7fc56d79a700 1 -- 172.21.15.101:0/1250040990 learned_addr learned my addr 172.21.15.101:0/1250040990 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:26:14.826 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.825+0000 7fc56d79a700 1 -- 172.21.15.101:0/1250040990 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc568105af0 msgr2=0x7fc5681a3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:26:14.826 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.825+0000 7fc56d79a700 1 --2- 172.21.15.101:0/1250040990 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc568105af0 0x7fc5681a3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:14.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.825+0000 7fc56d79a700 1 -- 172.21.15.101:0/1250040990 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc5680ffae0 msgr2=0x7fc568072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:26:14.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.825+0000 7fc56d79a700 1 --2- 172.21.15.101:0/1250040990 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc5680ffae0 0x7fc568072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:14.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.825+0000 7fc56d79a700 1 -- 172.21.15.101:0/1250040990 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc564009520 con 0x7fc568100490 2023-07-18T03:26:14.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.825+0000 7fc56df9b700 1 --2- 172.21.15.101:0/1250040990 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc5680ffae0 0x7fc568072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:26:14.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.825+0000 7fc56d79a700 1 --2- 172.21.15.101:0/1250040990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc568100490 0x7fc568072d50 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc558000c00 tx=0x7fc558004790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:26:14.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.825+0000 7fc55effd700 1 -- 172.21.15.101:0/1250040990 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55801d070 con 0x7fc568100490 2023-07-18T03:26:14.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.825+0000 7fc55effd700 1 -- 172.21.15.101:0/1250040990 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc558021410 con 0x7fc568100490 2023-07-18T03:26:14.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.825+0000 7fc5701ff700 1 -- 172.21.15.101:0/1250040990 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5580095f0 con 0x7fc568100490 2023-07-18T03:26:14.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.825+0000 7fc55effd700 1 -- 172.21.15.101:0/1250040990 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc558017650 con 0x7fc568100490 2023-07-18T03:26:14.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.826+0000 7fc5701ff700 1 -- 172.21.15.101:0/1250040990 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5681a48e0 con 0x7fc568100490 2023-07-18T03:26:14.832 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.829+0000 7fc55effd700 1 -- 172.21.15.101:0/1250040990 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fc558021580 con 0x7fc568100490 2023-07-18T03:26:14.832 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.830+0000 7fc583acdb80 1 Processor -- start 2023-07-18T03:26:14.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.830+0000 7fc583acdb80 1 -- start start 2023-07-18T03:26:14.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.831+0000 7fc583acdb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ab1a3972d0 0x55ab1a3976a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:26:14.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.831+0000 7fc583acdb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab1a397be0 0x55ab1a3a2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:26:14.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.831+0000 7fc583acdb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab1a3a2ad0 0x55ab1a3a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:26:14.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.831+0000 7fc583acdb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab1a29dcb0 con 0x55ab1a3972d0 2023-07-18T03:26:14.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.831+0000 7fc583acdb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab1a2f0a40 con 0x55ab1a397be0 2023-07-18T03:26:14.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.831+0000 7fc583acdb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab1a2e9ed0 con 0x55ab1a3a2ad0 2023-07-18T03:26:14.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.831+0000 7fc56df9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab1a397be0 0x55ab1a3a2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:26:14.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.831+0000 7fc56d79a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab1a3a2ad0 0x55ab1a3a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:26:14.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.831+0000 7fc56e79c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ab1a3972d0 0x55ab1a3976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:26:14.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.831+0000 7fc55effd700 1 --2- 172.21.15.101:0/1250040990 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc5540a12c0 0x7fc5540a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:26:14.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.831+0000 7fc56df9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab1a397be0 0x55ab1a3a2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:40020/0 (socket says 172.21.15.101:40020) 2023-07-18T03:26:14.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.831+0000 7fc56df9b700 1 -- 172.21.15.101:0/3923024743 learned_addr learned my addr 172.21.15.101:0/3923024743 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:26:14.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.831+0000 7fc56df9b700 1 -- 172.21.15.101:0/3923024743 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab1a3a2ad0 msgr2=0x55ab1a3a4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:26:14.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.831+0000 7fc56df9b700 1 --2- 172.21.15.101:0/3923024743 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab1a3a2ad0 0x55ab1a3a4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:14.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.831+0000 7fc55effd700 1 -- 172.21.15.101:0/1250040990 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fc5580ce3b0 con 0x7fc568100490 2023-07-18T03:26:14.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.831+0000 7fc56df9b700 1 -- 172.21.15.101:0/3923024743 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ab1a3972d0 msgr2=0x55ab1a3976a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:26:14.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.831+0000 7fc56df9b700 1 --2- 172.21.15.101:0/3923024743 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ab1a3972d0 0x55ab1a3976a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:14.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.831+0000 7fc56df9b700 1 -- 172.21.15.101:0/3923024743 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab1a117de0 con 0x55ab1a397be0 2023-07-18T03:26:14.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.832+0000 7fc56e79c700 1 --2- 172.21.15.101:0/1250040990 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc5540a12c0 0x7fc5540a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:26:14.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.832+0000 7fc56df9b700 1 --2- 172.21.15.101:0/3923024743 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab1a397be0 0x55ab1a3a2590 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc5640117e0 tx=0x7fc564011af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a7d16ccaad4e1ad server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:26:14.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.832+0000 7fc553fff700 1 -- 172.21.15.101:0/3923024743 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc56400f480 con 0x55ab1a397be0 2023-07-18T03:26:14.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.832+0000 7fc553fff700 1 -- 172.21.15.101:0/3923024743 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc56400faa0 con 0x55ab1a397be0 2023-07-18T03:26:14.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.832+0000 7fc553fff700 1 -- 172.21.15.101:0/3923024743 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc564019b20 con 0x55ab1a397be0 2023-07-18T03:26:14.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.833+0000 7fc583acdb80 1 -- 172.21.15.101:0/3923024743 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab1a397be0 msgr2=0x55ab1a3a2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:26:14.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.833+0000 7fc56e79c700 1 --2- 172.21.15.101:0/1250040990 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc5540a12c0 0x7fc5540a3770 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc560009ec0 tx=0x7fc560009040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:26:14.839 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.833+0000 7fc583acdb80 1 --2- 172.21.15.101:0/3923024743 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab1a397be0 0x55ab1a3a2590 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc5640117e0 tx=0x7fc564011af0 comp rx=0 tx=0).stop 2023-07-18T03:26:14.839 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.833+0000 7fc583acdb80 1 -- 172.21.15.101:0/3923024743 shutdown_connections 2023-07-18T03:26:14.839 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.833+0000 7fc583acdb80 1 --2- 172.21.15.101:0/3923024743 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab1a3a2ad0 0x55ab1a3a4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:14.839 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.833+0000 7fc583acdb80 1 --2- 172.21.15.101:0/3923024743 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ab1a3972d0 0x55ab1a3976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:14.839 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.833+0000 7fc583acdb80 1 --2- 172.21.15.101:0/3923024743 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab1a397be0 0x55ab1a3a2590 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:14.840 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.833+0000 7fc583acdb80 1 -- 172.21.15.101:0/3923024743 >> 172.21.15.101:0/3923024743 conn(0x55ab1a38dda0 msgr2=0x55ab1a3936a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:26:14.840 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.833+0000 7fc583acdb80 1 -- 172.21.15.101:0/3923024743 shutdown_connections 2023-07-18T03:26:14.840 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.833+0000 7fc583acdb80 1 -- 172.21.15.101:0/3923024743 wait complete. 2023-07-18T03:26:14.840 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.833+0000 7fc583acdb80 1 -- start start 2023-07-18T03:26:14.841 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.833+0000 7fc583acdb80 1 Processor -- start 2023-07-18T03:26:14.841 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.834+0000 7fc583acdb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab1a3972d0 0x55ab1a3a24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:26:14.841 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.834+0000 7fc583acdb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab1a397be0 0x55ab1a39f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:26:14.841 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.834+0000 7fc583acdb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ab1a3a2ad0 0x55ab1a39fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:26:14.842 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.834+0000 7fc583acdb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab1a289ac0 con 0x55ab1a3a2ad0 2023-07-18T03:26:14.842 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.834+0000 7fc583acdb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab1a2897c0 con 0x55ab1a3972d0 2023-07-18T03:26:14.842 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.834+0000 7fc583acdb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab1a16d920 con 0x55ab1a397be0 2023-07-18T03:26:14.842 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.834+0000 7fc583acdb80 10 client.0 _ll_register_callbacks cb 0x55ab1a287760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:26:14.842 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.834+0000 7fc56df9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab1a3972d0 0x55ab1a3a24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:26:14.843 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.834+0000 7fc56d79a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab1a397be0 0x55ab1a39f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:26:14.843 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.834+0000 7fc56df9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab1a3972d0 0x55ab1a3a24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:40030/0 (socket says 172.21.15.101:40030) 2023-07-18T03:26:14.843 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.834+0000 7fc56df9b700 1 -- 172.21.15.101:0/4013131221 learned_addr learned my addr 172.21.15.101:0/4013131221 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:26:14.843 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.834+0000 7fc56d79a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab1a397be0 0x55ab1a39f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:51240/0 (socket says 172.21.15.101:51240) 2023-07-18T03:26:14.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.834+0000 7fc56e79c700 1 --2- 172.21.15.101:0/4013131221 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ab1a3a2ad0 0x55ab1a39fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:26:14.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.835+0000 7fc56df9b700 1 -- 172.21.15.101:0/4013131221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab1a397be0 msgr2=0x55ab1a39f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:26:14.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.835+0000 7fc56df9b700 1 --2- 172.21.15.101:0/4013131221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab1a397be0 0x55ab1a39f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:14.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.835+0000 7fc56df9b700 1 -- 172.21.15.101:0/4013131221 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ab1a3a2ad0 msgr2=0x55ab1a39fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:26:14.845 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.835+0000 7fc56df9b700 1 --2- 172.21.15.101:0/4013131221 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ab1a3a2ad0 0x55ab1a39fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:14.845 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.835+0000 7fc56df9b700 1 -- 172.21.15.101:0/4013131221 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc564011440 con 0x55ab1a3972d0 2023-07-18T03:26:14.845 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.835+0000 7fc56e79c700 1 --2- 172.21.15.101:0/4013131221 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ab1a3a2ad0 0x55ab1a39fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:26:14.845 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.835+0000 7fc56d79a700 1 --2- 172.21.15.101:0/4013131221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab1a397be0 0x55ab1a39f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:26:14.845 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.835+0000 7fc56df9b700 1 --2- 172.21.15.101:0/4013131221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab1a3972d0 0x55ab1a3a24b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc564009230 tx=0x7fc56400f7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7d04d3539bc824c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:26:14.846 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.835+0000 7fc5517fa700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:26:14.846 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.835+0000 7fc5517fa700 1 -- 172.21.15.101:0/4013131221 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc564004250 con 0x55ab1a3972d0 2023-07-18T03:26:14.846 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.835+0000 7fc583acdb80 1 -- 172.21.15.101:0/4013131221 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ab1a2434f0 con 0x55ab1a3972d0 2023-07-18T03:26:14.846 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.835+0000 7fc5517fa700 1 -- 172.21.15.101:0/4013131221 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fc5640043b0 con 0x55ab1a3972d0 2023-07-18T03:26:14.847 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.835+0000 7fc5517fa700 1 -- 172.21.15.101:0/4013131221 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc564004570 con 0x55ab1a3972d0 2023-07-18T03:26:14.847 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.835+0000 7fc583acdb80 10 client.28976 fetch_fsmap learned FSMap version 45 2023-07-18T03:26:14.847 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.835+0000 7fc583acdb80 1 -- 172.21.15.101:0/4013131221 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55ab1a100c60 con 0x55ab1a3972d0 2023-07-18T03:26:14.847 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.836+0000 7fc5517fa700 1 -- 172.21.15.101:0/4013131221 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ab1a2434f0 con 0x55ab1a3972d0 2023-07-18T03:26:14.847 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.836+0000 7fc5517fa700 1 -- 172.21.15.101:0/4013131221 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 45) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fc564020070 con 0x55ab1a3972d0 2023-07-18T03:26:14.848 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.836+0000 7fc583acdb80 10 client.28976 fetch_fsmap finished waiting for FSMap version 45 2023-07-18T03:26:14.848 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.836+0000 7fc583acdb80 10 client.28976 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:26:14.848 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.836+0000 7fc583acdb80 20 client.28976 populate_metadata read hostname 'smithi101' 2023-07-18T03:26:14.848 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.836+0000 7fc583acdb80 1 --2- 172.21.15.101:0/4013131221 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ab1a3b2350 0x55ab1a3b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:26:14.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.836+0000 7fc583acdb80 4 client.28976 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:26:14.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.836+0000 7fc583acdb80 1 -- 172.21.15.101:0/4013131221 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ab1a2e2eb0 con 0x55ab1a3b2350 2023-07-18T03:26:14.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.836+0000 7fc56d79a700 1 --2- 172.21.15.101:0/4013131221 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ab1a3b2350 0x55ab1a3b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:26:14.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.836+0000 7fc56d79a700 1 --2- 172.21.15.101:0/4013131221 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ab1a3b2350 0x55ab1a3b4760 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a27ec6ff587cb694 server_cookie=ea181e760295414 in_seq=0 out_seq=0 2023-07-18T03:26:14.850 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.836+0000 7fc5517fa700 10 client.28976 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:26:14.852 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.837+0000 7fc5517fa700 1 -- 172.21.15.101:0/4013131221 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ab1a2e2eb0 con 0x55ab1a3b2350 2023-07-18T03:26:14.852 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.837+0000 7fc5517fa700 10 client.28976 handle_command_reply: tid=0 2023-07-18T03:26:14.852 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.837+0000 7fc583acdb80 1 client.28976 shutdown 2023-07-18T03:26:14.852 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.837+0000 7fc583acdb80 1 -- 172.21.15.101:0/4013131221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab1a3972d0 msgr2=0x55ab1a3a24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:26:14.853 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.837+0000 7fc583acdb80 1 --2- 172.21.15.101:0/4013131221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab1a3972d0 0x55ab1a3a24b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc564009230 tx=0x7fc56400f7e0 comp rx=0 tx=0).stop 2023-07-18T03:26:14.853 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.837+0000 7fc583acdb80 1 -- 172.21.15.101:0/4013131221 shutdown_connections 2023-07-18T03:26:14.853 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.837+0000 7fc583acdb80 1 --2- 172.21.15.101:0/4013131221 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ab1a3b2350 0x55ab1a3b4760 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:14.853 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.837+0000 7fc583acdb80 1 --2- 172.21.15.101:0/4013131221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab1a3972d0 0x55ab1a3a24b0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:14.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.837+0000 7fc583acdb80 1 --2- 172.21.15.101:0/4013131221 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ab1a3a2ad0 0x55ab1a39fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:14.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.837+0000 7fc583acdb80 1 --2- 172.21.15.101:0/4013131221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab1a397be0 0x55ab1a39f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:14.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.837+0000 7fc5517fa700 0 client.28976 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:26:14.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.837+0000 7fc583acdb80 1 -- 172.21.15.101:0/4013131221 >> 172.21.15.101:0/4013131221 conn(0x55ab1a38dda0 msgr2=0x55ab1a393290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:26:14.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.838+0000 7fc583acdb80 1 -- 172.21.15.101:0/4013131221 shutdown_connections 2023-07-18T03:26:14.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.838+0000 7fc583acdb80 1 -- 172.21.15.101:0/4013131221 wait complete. 2023-07-18T03:26:14.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.838+0000 7fc583acdb80 20 client.28976 trim_cache size 0 max 16384 2023-07-18T03:26:14.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.855+0000 7fc583acdb80 1 Processor -- start 2023-07-18T03:26:14.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.855+0000 7fc583acdb80 1 -- start start 2023-07-18T03:26:14.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.855+0000 7fc583acdb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ab1a3972d0 0x55ab1a3ae740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:26:14.856 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.855+0000 7fc583acdb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab1a397be0 0x55ab1a3a2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:26:14.856 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.855+0000 7fc583acdb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab1a3a2ad0 0x55ab1a39efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:26:14.856 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.855+0000 7fc583acdb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab1a3a0d40 con 0x55ab1a3972d0 2023-07-18T03:26:14.856 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.855+0000 7fc583acdb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab1a38afa0 con 0x55ab1a3a2ad0 2023-07-18T03:26:14.857 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.855+0000 7fc583acdb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab1a38d970 con 0x55ab1a397be0 2023-07-18T03:26:14.857 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.856+0000 7fc56d79a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab1a397be0 0x55ab1a3a2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:26:14.857 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.856+0000 7fc56df9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ab1a3972d0 0x55ab1a3ae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:26:14.857 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.856+0000 7fc56e79c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab1a3a2ad0 0x55ab1a39efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:26:14.858 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.856+0000 7fc56d79a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab1a397be0 0x55ab1a3a2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:51244/0 (socket says 172.21.15.101:51244) 2023-07-18T03:26:14.858 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.856+0000 7fc56d79a700 1 -- 172.21.15.101:0/4124397534 learned_addr learned my addr 172.21.15.101:0/4124397534 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:26:14.858 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.856+0000 7fc56e79c700 1 --2- 172.21.15.101:0/4124397534 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab1a3a2ad0 0x55ab1a39efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:40034/0 (socket says 172.21.15.101:40034) 2023-07-18T03:26:14.858 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.856+0000 7fc56d79a700 1 -- 172.21.15.101:0/4124397534 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab1a3a2ad0 msgr2=0x55ab1a39efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:26:14.858 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.856+0000 7fc56d79a700 1 --2- 172.21.15.101:0/4124397534 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab1a3a2ad0 0x55ab1a39efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:14.859 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.856+0000 7fc56d79a700 1 -- 172.21.15.101:0/4124397534 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ab1a3972d0 msgr2=0x55ab1a3ae740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:26:14.859 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.856+0000 7fc56d79a700 1 --2- 172.21.15.101:0/4124397534 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ab1a3972d0 0x55ab1a3ae740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:14.859 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.856+0000 7fc56d79a700 1 -- 172.21.15.101:0/4124397534 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab1a13e250 con 0x55ab1a397be0 2023-07-18T03:26:14.859 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.856+0000 7fc56df9b700 1 --2- 172.21.15.101:0/4124397534 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ab1a3972d0 0x55ab1a3ae740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:26:14.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.856+0000 7fc56d79a700 1 --2- 172.21.15.101:0/4124397534 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab1a397be0 0x55ab1a3a2050 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fc558097b20 tx=0x7fc55809b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a2a778bab754a17 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:26:14.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.856+0000 7fc55cff9700 1 -- 172.21.15.101:0/4124397534 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55809f130 con 0x55ab1a397be0 2023-07-18T03:26:14.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.856+0000 7fc55cff9700 1 -- 172.21.15.101:0/4124397534 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5580a34c0 con 0x55ab1a397be0 2023-07-18T03:26:14.864 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.856+0000 7fc55cff9700 1 -- 172.21.15.101:0/4124397534 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55809a850 con 0x55ab1a397be0 2023-07-18T03:26:14.864 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.857+0000 7fc583acdb80 1 -- 172.21.15.101:0/4124397534 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab1a397be0 msgr2=0x55ab1a3a2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:26:14.864 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.857+0000 7fc583acdb80 1 --2- 172.21.15.101:0/4124397534 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab1a397be0 0x55ab1a3a2050 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fc558097b20 tx=0x7fc55809b000 comp rx=0 tx=0).stop 2023-07-18T03:26:14.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.857+0000 7fc583acdb80 1 -- 172.21.15.101:0/4124397534 shutdown_connections 2023-07-18T03:26:14.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.857+0000 7fc583acdb80 1 --2- 172.21.15.101:0/4124397534 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab1a3a2ad0 0x55ab1a39efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:14.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.857+0000 7fc583acdb80 1 --2- 172.21.15.101:0/4124397534 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ab1a3972d0 0x55ab1a3ae740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:14.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.857+0000 7fc583acdb80 1 --2- 172.21.15.101:0/4124397534 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab1a397be0 0x55ab1a3a2050 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:14.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.857+0000 7fc583acdb80 1 -- 172.21.15.101:0/4124397534 >> 172.21.15.101:0/4124397534 conn(0x55ab1a38dda0 msgr2=0x55ab1a3b9080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:26:14.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.857+0000 7fc583acdb80 1 -- 172.21.15.101:0/4124397534 shutdown_connections 2023-07-18T03:26:14.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.857+0000 7fc583acdb80 1 -- 172.21.15.101:0/4124397534 wait complete. 2023-07-18T03:26:14.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.857+0000 7fc583acdb80 1 -- start start 2023-07-18T03:26:14.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.858+0000 7fc583acdb80 1 Processor -- start 2023-07-18T03:26:14.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.858+0000 7fc583acdb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ab1a3972d0 0x55ab1a3ff400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:26:14.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.858+0000 7fc583acdb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab1a397be0 0x55ab1a3ff940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:26:14.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.858+0000 7fc583acdb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab1a3a2ad0 0x55ab1a3a7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:26:14.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.858+0000 7fc583acdb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab1a3a79d0 con 0x55ab1a3972d0 2023-07-18T03:26:14.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.858+0000 7fc583acdb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab1a3a7b10 con 0x55ab1a397be0 2023-07-18T03:26:14.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.858+0000 7fc583acdb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab1a3a7cd0 con 0x55ab1a3a2ad0 2023-07-18T03:26:14.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.858+0000 7fc583acdb80 10 client.0 _ll_register_callbacks cb 0x7fc55809aed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:26:14.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.858+0000 7fc56d79a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab1a397be0 0x55ab1a3ff940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:26:14.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.858+0000 7fc56d79a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab1a397be0 0x55ab1a3ff940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:40046/0 (socket says 172.21.15.101:40046) 2023-07-18T03:26:14.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.858+0000 7fc56d79a700 1 -- 172.21.15.101:0/1060057917 learned_addr learned my addr 172.21.15.101:0/1060057917 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:26:14.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.858+0000 7fc56e79c700 1 --2- 172.21.15.101:0/1060057917 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab1a3a2ad0 0x55ab1a3a7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:26:14.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.858+0000 7fc56df9b700 1 --2- 172.21.15.101:0/1060057917 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ab1a3972d0 0x55ab1a3ff400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:26:14.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.859+0000 7fc56d79a700 1 -- 172.21.15.101:0/1060057917 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab1a3a2ad0 msgr2=0x55ab1a3a7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:26:14.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.859+0000 7fc56d79a700 1 --2- 172.21.15.101:0/1060057917 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab1a3a2ad0 0x55ab1a3a7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:14.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.859+0000 7fc56d79a700 1 -- 172.21.15.101:0/1060057917 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ab1a3972d0 msgr2=0x55ab1a3ff400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:26:14.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.859+0000 7fc56d79a700 1 --2- 172.21.15.101:0/1060057917 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ab1a3972d0 0x55ab1a3ff400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:14.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.859+0000 7fc56d79a700 1 -- 172.21.15.101:0/1060057917 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ab1a100c60 con 0x55ab1a397be0 2023-07-18T03:26:14.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.859+0000 7fc56df9b700 1 --2- 172.21.15.101:0/1060057917 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ab1a3972d0 0x55ab1a3ff400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:26:14.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.859+0000 7fc56d79a700 1 --2- 172.21.15.101:0/1060057917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab1a397be0 0x55ab1a3ff940 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc5580a2f90 tx=0x7fc558009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea97ddfe0682e831 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:26:14.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.859+0000 7fc5517fa700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:26:14.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.859+0000 7fc5517fa700 1 -- 172.21.15.101:0/1060057917 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55809b560 con 0x55ab1a397be0 2023-07-18T03:26:14.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.859+0000 7fc5517fa700 1 -- 172.21.15.101:0/1060057917 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fc5580cbe40 con 0x55ab1a397be0 2023-07-18T03:26:14.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.859+0000 7fc583acdb80 1 -- 172.21.15.101:0/1060057917 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc56400faa0 con 0x55ab1a397be0 2023-07-18T03:26:14.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.859+0000 7fc5517fa700 1 -- 172.21.15.101:0/1060057917 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc55809f130 con 0x55ab1a397be0 2023-07-18T03:26:14.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.859+0000 7fc583acdb80 10 client.28982 fetch_fsmap learned FSMap version 45 2023-07-18T03:26:14.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.859+0000 7fc583acdb80 1 -- 172.21.15.101:0/1060057917 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x7fc5681a48e0 con 0x55ab1a397be0 2023-07-18T03:26:14.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.860+0000 7fc5517fa700 1 -- 172.21.15.101:0/1060057917 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc56400faa0 con 0x55ab1a397be0 2023-07-18T03:26:14.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.860+0000 7fc5517fa700 1 -- 172.21.15.101:0/1060057917 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 45) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fc558096ef0 con 0x55ab1a397be0 2023-07-18T03:26:14.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.860+0000 7fc583acdb80 10 client.28982 fetch_fsmap finished waiting for FSMap version 45 2023-07-18T03:26:14.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.860+0000 7fc583acdb80 10 client.28982 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:26:14.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.860+0000 7fc583acdb80 20 client.28982 populate_metadata read hostname 'smithi101' 2023-07-18T03:26:14.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.860+0000 7fc583acdb80 1 --2- 172.21.15.101:0/1060057917 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ab1a406d40 0x55ab1a409120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:26:14.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.860+0000 7fc583acdb80 4 client.28982 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:26:14.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.860+0000 7fc583acdb80 1 -- 172.21.15.101:0/1060057917 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ab1a2a5050 con 0x55ab1a406d40 2023-07-18T03:26:14.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.860+0000 7fc56df9b700 1 --2- 172.21.15.101:0/1060057917 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ab1a406d40 0x55ab1a409120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:26:14.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.860+0000 7fc56df9b700 1 --2- 172.21.15.101:0/1060057917 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ab1a406d40 0x55ab1a409120 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ea45fd1788d4ebc server_cookie=e7144b99ae912609 in_seq=0 out_seq=0 2023-07-18T03:26:14.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.860+0000 7fc5517fa700 10 client.28982 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:26:14.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.861+0000 7fc5517fa700 1 -- 172.21.15.101:0/1060057917 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ab1a2a5050 con 0x55ab1a406d40 2023-07-18T03:26:14.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.861+0000 7fc5517fa700 10 client.28982 handle_command_reply: tid=0 2023-07-18T03:26:14.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.861+0000 7fc583acdb80 1 client.28982 shutdown 2023-07-18T03:26:14.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.861+0000 7fc583acdb80 1 -- 172.21.15.101:0/1060057917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab1a397be0 msgr2=0x55ab1a3ff940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:26:14.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.861+0000 7fc583acdb80 1 --2- 172.21.15.101:0/1060057917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab1a397be0 0x55ab1a3ff940 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc5580a2f90 tx=0x7fc558009310 comp rx=0 tx=0).stop 2023-07-18T03:26:14.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.861+0000 7fc583acdb80 1 -- 172.21.15.101:0/1060057917 shutdown_connections 2023-07-18T03:26:14.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.861+0000 7fc583acdb80 1 --2- 172.21.15.101:0/1060057917 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55ab1a406d40 0x55ab1a409120 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:14.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.861+0000 7fc583acdb80 1 --2- 172.21.15.101:0/1060057917 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab1a3a2ad0 0x55ab1a3a7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:14.877 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.861+0000 7fc583acdb80 1 --2- 172.21.15.101:0/1060057917 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ab1a3972d0 0x55ab1a3ff400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:14.877 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.861+0000 7fc5517fa700 0 client.28982 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:26:14.877 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.861+0000 7fc583acdb80 1 --2- 172.21.15.101:0/1060057917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab1a397be0 0x55ab1a3ff940 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:14.877 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.861+0000 7fc583acdb80 1 -- 172.21.15.101:0/1060057917 >> 172.21.15.101:0/1060057917 conn(0x55ab1a38dda0 msgr2=0x55ab1a39aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:26:14.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.861+0000 7fc583acdb80 1 -- 172.21.15.101:0/1060057917 shutdown_connections 2023-07-18T03:26:14.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.861+0000 7fc583acdb80 1 -- 172.21.15.101:0/1060057917 wait complete. 2023-07-18T03:26:14.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.861+0000 7fc583acdb80 20 client.28982 trim_cache size 0 max 16384 2023-07-18T03:26:14.879 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:26:14.879 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:26:14.879 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:26:14.880 INFO:teuthology.orchestra.run.smithi101.stdout: "0f2f19e0-0429-465c-935e-223eb09490c3": { 2023-07-18T03:26:14.880 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:26:14.880 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "0f2f19e0-0429-465c-935e-223eb09490c3", 2023-07-18T03:26:14.880 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:26:14.880 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:26:14.881 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:26:14.881 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:26:14.882 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.862+0000 7fc5701ff700 1 -- 172.21.15.101:0/1250040990 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc5540a12c0 msgr2=0x7fc5540a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:26:14.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.862+0000 7fc5701ff700 1 --2- 172.21.15.101:0/1250040990 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc5540a12c0 0x7fc5540a3770 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc560009ec0 tx=0x7fc560009040 comp rx=0 tx=0).stop 2023-07-18T03:26:14.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.862+0000 7fc5701ff700 1 -- 172.21.15.101:0/1250040990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc568100490 msgr2=0x7fc568072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:26:14.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.862+0000 7fc5701ff700 1 --2- 172.21.15.101:0/1250040990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc568100490 0x7fc568072d50 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc558000c00 tx=0x7fc558004790 comp rx=0 tx=0).stop 2023-07-18T03:26:14.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.862+0000 7fc5701ff700 1 -- 172.21.15.101:0/1250040990 shutdown_connections 2023-07-18T03:26:14.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.862+0000 7fc5701ff700 1 --2- 172.21.15.101:0/1250040990 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc568105af0 0x7fc5681a3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:14.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.862+0000 7fc5701ff700 1 --2- 172.21.15.101:0/1250040990 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc5680ffae0 0x7fc568072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:14.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.862+0000 7fc5701ff700 1 --2- 172.21.15.101:0/1250040990 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc5540a12c0 0x7fc5540a3770 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:14.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.862+0000 7fc5701ff700 1 --2- 172.21.15.101:0/1250040990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc568100490 0x7fc568072d50 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:14.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.862+0000 7fc5701ff700 1 -- 172.21.15.101:0/1250040990 >> 172.21.15.101:0/1250040990 conn(0x7fc5680fb5e0 msgr2=0x7fc5681035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:26:14.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.862+0000 7fc5701ff700 1 -- 172.21.15.101:0/1250040990 shutdown_connections 2023-07-18T03:26:14.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:14.862+0000 7fc5701ff700 1 -- 172.21.15.101:0/1250040990 wait complete. 2023-07-18T03:26:14.885 INFO:tasks.cephfs.filesystem:scrub status for tag:0f2f19e0-0429-465c-935e-223eb09490c3 - {'path': '/', 'tag': '0f2f19e0-0429-465c-935e-223eb09490c3', 'options': 'recursive,force'} 2023-07-18T03:26:15.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:26:15 smithi155 ceph-mon[128885]: pgmap v588: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 470 KiB/s wr, 54 op/s 2023-07-18T03:26:15.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:26:15 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:26:15.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:26:15 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:26:15.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:26:15 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:26:15.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:26:15 smithi179 ceph-mon[129229]: pgmap v588: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 470 KiB/s wr, 54 op/s 2023-07-18T03:26:15.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:26:15 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:26:15.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:26:15 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:26:15.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:26:15 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:26:15.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:26:15 smithi101 ceph-mon[126379]: pgmap v588: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 470 KiB/s wr, 54 op/s 2023-07-18T03:26:15.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:26:15 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:26:15.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:26:15 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:26:15.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:26:15 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:26:17.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:26:17 smithi155 ceph-mon[128885]: pgmap v590: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 216 KiB/s wr, 32 op/s 2023-07-18T03:26:17.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:26:17 smithi179 ceph-mon[129229]: pgmap v590: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 216 KiB/s wr, 32 op/s 2023-07-18T03:26:17.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:26:17 smithi101 ceph-mon[126379]: pgmap v590: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 216 KiB/s wr, 32 op/s 2023-07-18T03:26:19.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:26:19 smithi155 ceph-mon[128885]: pgmap v591: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 208 KiB/s wr, 28 op/s 2023-07-18T03:26:19.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:26:19 smithi179 ceph-mon[129229]: pgmap v591: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 208 KiB/s wr, 28 op/s 2023-07-18T03:26:19.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:26:19 smithi101 ceph-mon[126379]: pgmap v591: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 208 KiB/s wr, 28 op/s 2023-07-18T03:26:21.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:26:21 smithi155 ceph-mon[128885]: pgmap v592: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 46 KiB/s wr, 14 op/s 2023-07-18T03:26:21.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:26:21 smithi179 ceph-mon[129229]: pgmap v592: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 46 KiB/s wr, 14 op/s 2023-07-18T03:26:21.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:26:21 smithi101 ceph-mon[126379]: pgmap v592: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 46 KiB/s wr, 14 op/s 2023-07-18T03:26:23.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:26:23 smithi155 ceph-mon[128885]: pgmap v593: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 9 op/s 2023-07-18T03:26:23.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:26:23 smithi179 ceph-mon[129229]: pgmap v593: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 9 op/s 2023-07-18T03:26:23.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:26:23 smithi101 ceph-mon[126379]: pgmap v593: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 9 op/s 2023-07-18T03:26:25.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:26:25 smithi179 ceph-mon[129229]: pgmap v594: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 9 op/s 2023-07-18T03:26:25.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:26:25 smithi101 ceph-mon[126379]: pgmap v594: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 9 op/s 2023-07-18T03:26:25.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:26:25 smithi155 ceph-mon[128885]: pgmap v594: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 9 op/s 2023-07-18T03:26:27.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:26:27 smithi179 ceph-mon[129229]: pgmap v595: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-07-18T03:26:27.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:26:27 smithi101 ceph-mon[126379]: pgmap v595: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-07-18T03:26:27.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:26:27 smithi155 ceph-mon[128885]: pgmap v595: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-07-18T03:26:29.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:26:29 smithi179 ceph-mon[129229]: pgmap v596: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 7 op/s 2023-07-18T03:26:29.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:26:29 smithi101 ceph-mon[126379]: pgmap v596: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 7 op/s 2023-07-18T03:26:29.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:26:29 smithi155 ceph-mon[128885]: pgmap v596: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 7 op/s 2023-07-18T03:26:31.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:26:31 smithi179 ceph-mon[129229]: pgmap v597: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 KiB/s wr, 9 op/s 2023-07-18T03:26:31.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:26:31 smithi101 ceph-mon[126379]: pgmap v597: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 KiB/s wr, 9 op/s 2023-07-18T03:26:31.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:26:31 smithi155 ceph-mon[128885]: pgmap v597: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 KiB/s wr, 9 op/s 2023-07-18T03:26:33.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:26:33 smithi179 ceph-mon[129229]: pgmap v598: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 7 op/s 2023-07-18T03:26:33.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:26:33 smithi101 ceph-mon[126379]: pgmap v598: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 7 op/s 2023-07-18T03:26:33.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:26:33 smithi155 ceph-mon[128885]: pgmap v598: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 7 op/s 2023-07-18T03:26:35.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:26:35 smithi179 ceph-mon[129229]: pgmap v599: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 6 op/s 2023-07-18T03:26:35.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:26:35 smithi101 ceph-mon[126379]: pgmap v599: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 6 op/s 2023-07-18T03:26:35.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:26:35 smithi155 ceph-mon[128885]: pgmap v599: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 6 op/s 2023-07-18T03:26:37.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:26:37 smithi179 ceph-mon[129229]: pgmap v600: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 9 op/s 2023-07-18T03:26:37.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:26:37 smithi179 ceph-mon[129229]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:26:37.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:26:37 smithi101 ceph-mon[126379]: pgmap v600: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 9 op/s 2023-07-18T03:26:37.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:26:37 smithi101 ceph-mon[126379]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:26:37.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:26:37 smithi155 ceph-mon[128885]: pgmap v600: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 9 op/s 2023-07-18T03:26:37.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:26:37 smithi155 ceph-mon[128885]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:26:39.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:26:39 smithi179 ceph-mon[129229]: pgmap v601: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.8 KiB/s wr, 5 op/s 2023-07-18T03:26:39.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:26:39 smithi101 ceph-mon[126379]: pgmap v601: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.8 KiB/s wr, 5 op/s 2023-07-18T03:26:39.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:26:39 smithi155 ceph-mon[128885]: pgmap v601: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.8 KiB/s wr, 5 op/s 2023-07-18T03:26:41.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:26:41 smithi179 ceph-mon[129229]: pgmap v602: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.9 KiB/s wr, 7 op/s 2023-07-18T03:26:41.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:26:41 smithi101 ceph-mon[126379]: pgmap v602: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.9 KiB/s wr, 7 op/s 2023-07-18T03:26:41.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:26:41 smithi155 ceph-mon[128885]: pgmap v602: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.9 KiB/s wr, 7 op/s 2023-07-18T03:26:43.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:26:43 smithi179 ceph-mon[129229]: pgmap v603: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.2 KiB/s wr, 6 op/s 2023-07-18T03:26:43.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:26:43 smithi101 ceph-mon[126379]: pgmap v603: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.2 KiB/s wr, 6 op/s 2023-07-18T03:26:43.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:26:43 smithi155 ceph-mon[128885]: pgmap v603: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.2 KiB/s wr, 6 op/s 2023-07-18T03:26:44.886 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:26:45.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.100+0000 7f7c2ebff700 1 -- 172.21.15.101:0/278263532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c280ffad0 msgr2=0x7f7c280fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:26:45.098 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.100+0000 7f7c2ebff700 1 --2- 172.21.15.101:0/278263532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c280ffad0 0x7f7c280fff40 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7c14009990 tx=0x7f7c14009ca0 comp rx=0 tx=0).stop 2023-07-18T03:26:45.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.100+0000 7f7c2ebff700 1 -- 172.21.15.101:0/278263532 shutdown_connections 2023-07-18T03:26:45.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.100+0000 7f7c2ebff700 1 --2- 172.21.15.101:0/278263532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c28105ae0 0x7f7c28105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:45.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.100+0000 7f7c2ebff700 1 --2- 172.21.15.101:0/278263532 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7c28100480 0x7f7c2810c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:45.099 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.100+0000 7f7c2ebff700 1 --2- 172.21.15.101:0/278263532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c280ffad0 0x7f7c280fff40 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:45.100 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.100+0000 7f7c2ebff700 1 -- 172.21.15.101:0/278263532 >> 172.21.15.101:0/278263532 conn(0x7f7c280fb5f0 msgr2=0x7f7c280fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:26:45.100 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.100+0000 7f7c2ebff700 1 -- 172.21.15.101:0/278263532 shutdown_connections 2023-07-18T03:26:45.100 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.100+0000 7f7c2ebff700 1 -- 172.21.15.101:0/278263532 wait complete. 2023-07-18T03:26:45.102 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.101+0000 7f7c2ebff700 1 Processor -- start 2023-07-18T03:26:45.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.102+0000 7f7c2ebff700 1 -- start start 2023-07-18T03:26:45.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.102+0000 7f7c2ebff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7c280ffad0 0x7f7c2819b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:26:45.103 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.102+0000 7f7c2ebff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c28100480 0x7f7c2819bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:26:45.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.102+0000 7f7c2ebff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c28105ae0 0x7f7c28195a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:26:45.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.102+0000 7f7c2ebff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c281960b0 con 0x7f7c280ffad0 2023-07-18T03:26:45.104 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.102+0000 7f7c2ebff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c28196220 con 0x7f7c28100480 2023-07-18T03:26:45.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.102+0000 7f7c2ebff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c281963e0 con 0x7f7c28105ae0 2023-07-18T03:26:45.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.102+0000 7f7c27fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c28100480 0x7f7c2819bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:26:45.105 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.102+0000 7f7c27fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c28100480 0x7f7c2819bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47384/0 (socket says 172.21.15.101:47384) 2023-07-18T03:26:45.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.102+0000 7f7c2c99b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7c280ffad0 0x7f7c2819b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:26:45.106 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.102+0000 7f7c27fff700 1 -- 172.21.15.101:0/1326910895 learned_addr learned my addr 172.21.15.101:0/1326910895 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:26:45.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.102+0000 7f7c2d19c700 1 --2- 172.21.15.101:0/1326910895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c28105ae0 0x7f7c28195a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:26:45.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.102+0000 7f7c27fff700 1 -- 172.21.15.101:0/1326910895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c28105ae0 msgr2=0x7f7c28195a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:26:45.107 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.102+0000 7f7c27fff700 1 --2- 172.21.15.101:0/1326910895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c28105ae0 0x7f7c28195a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:45.108 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.102+0000 7f7c27fff700 1 -- 172.21.15.101:0/1326910895 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7c280ffad0 msgr2=0x7f7c2819b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:26:45.108 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.102+0000 7f7c27fff700 1 --2- 172.21.15.101:0/1326910895 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7c280ffad0 0x7f7c2819b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:45.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.102+0000 7f7c27fff700 1 -- 172.21.15.101:0/1326910895 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c140095f0 con 0x7f7c28100480 2023-07-18T03:26:45.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.103+0000 7f7c27fff700 1 --2- 172.21.15.101:0/1326910895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c28100480 0x7f7c2819bf10 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f7c14000c00 tx=0x7f7c14005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:26:45.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.103+0000 7f7c25ffb700 1 -- 172.21.15.101:0/1326910895 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1401d070 con 0x7f7c28100480 2023-07-18T03:26:45.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.103+0000 7f7c2ebff700 1 -- 172.21.15.101:0/1326910895 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c28196660 con 0x7f7c28100480 2023-07-18T03:26:45.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.103+0000 7f7c25ffb700 1 -- 172.21.15.101:0/1326910895 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c14022470 con 0x7f7c28100480 2023-07-18T03:26:45.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.103+0000 7f7c25ffb700 1 -- 172.21.15.101:0/1326910895 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c14017630 con 0x7f7c28100480 2023-07-18T03:26:45.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.103+0000 7f7c2ebff700 1 -- 172.21.15.101:0/1326910895 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c281a29c0 con 0x7f7c28100480 2023-07-18T03:26:45.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.106+0000 7f7c25ffb700 1 -- 172.21.15.101:0/1326910895 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f7c1400fad0 con 0x7f7c28100480 2023-07-18T03:26:45.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.107+0000 7f7c4260cb80 1 Processor -- start 2023-07-18T03:26:45.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.107+0000 7f7c25ffb700 1 --2- 172.21.15.101:0/1326910895 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f7c180a12c0 0x7f7c180a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:26:45.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.107+0000 7f7c4260cb80 1 -- start start 2023-07-18T03:26:45.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.108+0000 7f7c25ffb700 1 -- 172.21.15.101:0/1326910895 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f7c140ce510 con 0x7f7c28100480 2023-07-18T03:26:45.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.108+0000 7f7c4260cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4b148e2d0 0x55d4b148e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:26:45.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.108+0000 7f7c2d19c700 1 --2- 172.21.15.101:0/1326910895 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f7c180a12c0 0x7f7c180a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:26:45.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.108+0000 7f7c4260cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4b148ebe0 0x55d4b149d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:26:45.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.108+0000 7f7c4260cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4b149dee0 0x55d4b14a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:26:45.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.108+0000 7f7c4260cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4b1394cb0 con 0x55d4b149dee0 2023-07-18T03:26:45.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.108+0000 7f7c4260cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4b13e7a40 con 0x55d4b148e2d0 2023-07-18T03:26:45.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.108+0000 7f7c4260cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4b13e0ed0 con 0x55d4b148ebe0 2023-07-18T03:26:45.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.108+0000 7f7c2c99b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4b148e2d0 0x55d4b148e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:26:45.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.108+0000 7f7c2c99b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4b148e2d0 0x55d4b148e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47392/0 (socket says 172.21.15.101:47392) 2023-07-18T03:26:45.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.108+0000 7f7c2c99b700 1 -- 172.21.15.101:0/853863361 learned_addr learned my addr 172.21.15.101:0/853863361 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:26:45.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.108+0000 7f7c27fff700 1 --2- 172.21.15.101:0/853863361 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4b148ebe0 0x55d4b149d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:26:45.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.108+0000 7f7c2d19c700 1 --2- 172.21.15.101:0/853863361 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4b149dee0 0x55d4b14a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:26:45.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.108+0000 7f7c2c99b700 1 -- 172.21.15.101:0/853863361 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4b148ebe0 msgr2=0x55d4b149d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:26:45.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.108+0000 7f7c2c99b700 1 --2- 172.21.15.101:0/853863361 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4b148ebe0 0x55d4b149d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:45.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.108+0000 7f7c2c99b700 1 -- 172.21.15.101:0/853863361 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4b149dee0 msgr2=0x55d4b14a02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:26:45.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.108+0000 7f7c2c99b700 1 --2- 172.21.15.101:0/853863361 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4b149dee0 0x55d4b14a02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:45.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.108+0000 7f7c2c99b700 1 -- 172.21.15.101:0/853863361 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4b120ede0 con 0x55d4b148e2d0 2023-07-18T03:26:45.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.109+0000 7f7c2d19c700 1 --2- 172.21.15.101:0/1326910895 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f7c180a12c0 0x7f7c180a3770 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7c2819c6d0 tx=0x7f7c2000fc00 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:26:45.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.109+0000 7f7c2c99b700 1 --2- 172.21.15.101:0/853863361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4b148e2d0 0x55d4b148e6a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c009d00 tx=0x7f7c1c011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63696ae8ae2f8eec server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:26:45.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.109+0000 7f7c12ffd700 1 -- 172.21.15.101:0/853863361 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c004230 con 0x55d4b148e2d0 2023-07-18T03:26:45.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.109+0000 7f7c12ffd700 1 -- 172.21.15.101:0/853863361 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c1c004390 con 0x55d4b148e2d0 2023-07-18T03:26:45.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.109+0000 7f7c12ffd700 1 -- 172.21.15.101:0/853863361 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c0197b0 con 0x55d4b148e2d0 2023-07-18T03:26:45.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.110+0000 7f7c4260cb80 1 -- 172.21.15.101:0/853863361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4b148e2d0 msgr2=0x55d4b148e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:26:45.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.110+0000 7f7c4260cb80 1 --2- 172.21.15.101:0/853863361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4b148e2d0 0x55d4b148e6a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c009d00 tx=0x7f7c1c011720 comp rx=0 tx=0).stop 2023-07-18T03:26:45.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.110+0000 7f7c4260cb80 1 -- 172.21.15.101:0/853863361 shutdown_connections 2023-07-18T03:26:45.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.110+0000 7f7c4260cb80 1 --2- 172.21.15.101:0/853863361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4b148e2d0 0x55d4b148e6a0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:45.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.110+0000 7f7c4260cb80 1 --2- 172.21.15.101:0/853863361 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4b149dee0 0x55d4b14a02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:45.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.110+0000 7f7c4260cb80 1 --2- 172.21.15.101:0/853863361 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4b148ebe0 0x55d4b149d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:45.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.110+0000 7f7c4260cb80 1 -- 172.21.15.101:0/853863361 >> 172.21.15.101:0/853863361 conn(0x55d4b1484da0 msgr2=0x55d4b148a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:26:45.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.110+0000 7f7c4260cb80 1 -- 172.21.15.101:0/853863361 shutdown_connections 2023-07-18T03:26:45.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.110+0000 7f7c4260cb80 1 -- 172.21.15.101:0/853863361 wait complete. 2023-07-18T03:26:45.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.110+0000 7f7c4260cb80 1 -- start start 2023-07-18T03:26:45.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.110+0000 7f7c4260cb80 1 Processor -- start 2023-07-18T03:26:45.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.111+0000 7f7c4260cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4b148e2d0 0x55d4b149d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:26:45.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.111+0000 7f7c4260cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4b148ebe0 0x55d4b149ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:26:45.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.111+0000 7f7c4260cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4b149dee0 0x55d4b149b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:26:45.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.111+0000 7f7c4260cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4b1380ac0 con 0x55d4b149dee0 2023-07-18T03:26:45.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.111+0000 7f7c4260cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4b13807c0 con 0x55d4b148ebe0 2023-07-18T03:26:45.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.111+0000 7f7c4260cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4b1264920 con 0x55d4b148e2d0 2023-07-18T03:26:45.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.111+0000 7f7c4260cb80 10 client.0 _ll_register_callbacks cb 0x55d4b137e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:26:45.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.111+0000 7f7c2c99b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4b148e2d0 0x55d4b149d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:26:45.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.111+0000 7f7c2c99b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4b148e2d0 0x55d4b149d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:55494/0 (socket says 172.21.15.101:55494) 2023-07-18T03:26:45.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.111+0000 7f7c2c99b700 1 -- 172.21.15.101:0/1047388616 learned_addr learned my addr 172.21.15.101:0/1047388616 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:26:45.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.111+0000 7f7c2d19c700 1 --2- 172.21.15.101:0/1047388616 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4b149dee0 0x55d4b149b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:26:45.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.111+0000 7f7c27fff700 1 --2- 172.21.15.101:0/1047388616 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4b148ebe0 0x55d4b149ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:26:45.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.111+0000 7f7c2c99b700 1 -- 172.21.15.101:0/1047388616 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4b148ebe0 msgr2=0x55d4b149ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:26:45.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.111+0000 7f7c2c99b700 1 --2- 172.21.15.101:0/1047388616 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4b148ebe0 0x55d4b149ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:45.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.111+0000 7f7c2c99b700 1 -- 172.21.15.101:0/1047388616 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4b149dee0 msgr2=0x55d4b149b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:26:45.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.111+0000 7f7c2c99b700 1 --2- 172.21.15.101:0/1047388616 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4b149dee0 0x55d4b149b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:45.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.111+0000 7f7c2c99b700 1 -- 172.21.15.101:0/1047388616 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c1c009960 con 0x55d4b148e2d0 2023-07-18T03:26:45.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.112+0000 7f7c2c99b700 1 --2- 172.21.15.101:0/1047388616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4b148e2d0 0x55d4b149d900 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c003e80 tx=0x7f7c1c0038b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=faae07d004518565 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:26:45.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.112+0000 7f7c07fff700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:26:45.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.112+0000 7f7c07fff700 1 -- 172.21.15.101:0/1047388616 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c003aa0 con 0x55d4b148e2d0 2023-07-18T03:26:45.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.112+0000 7f7c4260cb80 1 -- 172.21.15.101:0/1047388616 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d4b133a4f0 con 0x55d4b148e2d0 2023-07-18T03:26:45.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.112+0000 7f7c07fff700 1 -- 172.21.15.101:0/1047388616 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f7c1c005250 con 0x55d4b148e2d0 2023-07-18T03:26:45.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.112+0000 7f7c07fff700 1 -- 172.21.15.101:0/1047388616 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c1c005410 con 0x55d4b148e2d0 2023-07-18T03:26:45.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.112+0000 7f7c4260cb80 10 client.28984 fetch_fsmap learned FSMap version 46 2023-07-18T03:26:45.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.112+0000 7f7c4260cb80 1 -- 172.21.15.101:0/1047388616 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55d4b11f7c60 con 0x55d4b148e2d0 2023-07-18T03:26:45.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.113+0000 7f7c07fff700 1 -- 172.21.15.101:0/1047388616 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d4b133a4f0 con 0x55d4b148e2d0 2023-07-18T03:26:45.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.113+0000 7f7c07fff700 1 -- 172.21.15.101:0/1047388616 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 46) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f7c1c028020 con 0x55d4b148e2d0 2023-07-18T03:26:45.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.113+0000 7f7c4260cb80 10 client.28984 fetch_fsmap finished waiting for FSMap version 46 2023-07-18T03:26:45.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.113+0000 7f7c4260cb80 10 client.28984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:26:45.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.113+0000 7f7c4260cb80 20 client.28984 populate_metadata read hostname 'smithi101' 2023-07-18T03:26:45.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.113+0000 7f7c4260cb80 1 --2- 172.21.15.101:0/1047388616 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55d4b14ad740 0x55d4b14afb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:26:45.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.113+0000 7f7c4260cb80 4 client.28984 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:26:45.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.113+0000 7f7c4260cb80 1 -- 172.21.15.101:0/1047388616 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d4b13d9eb0 con 0x55d4b14ad740 2023-07-18T03:26:45.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.113+0000 7f7c27fff700 1 --2- 172.21.15.101:0/1047388616 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55d4b14ad740 0x55d4b14afb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:26:45.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.114+0000 7f7c07fff700 10 client.28984 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:26:45.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.114+0000 7f7c27fff700 1 --2- 172.21.15.101:0/1047388616 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55d4b14ad740 0x55d4b14afb50 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b02c5ec9a1588cf server_cookie=e046bd878772d7db in_seq=0 out_seq=0 2023-07-18T03:26:45.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.114+0000 7f7c07fff700 1 -- 172.21.15.101:0/1047388616 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d4b13d9eb0 con 0x55d4b14ad740 2023-07-18T03:26:45.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.114+0000 7f7c07fff700 10 client.28984 handle_command_reply: tid=0 2023-07-18T03:26:45.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.115+0000 7f7c4260cb80 1 client.28984 shutdown 2023-07-18T03:26:45.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.115+0000 7f7c4260cb80 1 -- 172.21.15.101:0/1047388616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4b148e2d0 msgr2=0x55d4b149d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:26:45.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.115+0000 7f7c4260cb80 1 --2- 172.21.15.101:0/1047388616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4b148e2d0 0x55d4b149d900 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c003e80 tx=0x7f7c1c0038b0 comp rx=0 tx=0).stop 2023-07-18T03:26:45.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.115+0000 7f7c4260cb80 1 -- 172.21.15.101:0/1047388616 shutdown_connections 2023-07-18T03:26:45.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.115+0000 7f7c4260cb80 1 --2- 172.21.15.101:0/1047388616 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55d4b14ad740 0x55d4b14afb50 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:45.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.115+0000 7f7c4260cb80 1 --2- 172.21.15.101:0/1047388616 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4b149dee0 0x55d4b149b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:45.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.115+0000 7f7c4260cb80 1 --2- 172.21.15.101:0/1047388616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4b148e2d0 0x55d4b149d900 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:45.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.115+0000 7f7c4260cb80 1 --2- 172.21.15.101:0/1047388616 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4b148ebe0 0x55d4b149ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:45.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.115+0000 7f7c07fff700 0 client.28984 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:26:45.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.115+0000 7f7c4260cb80 1 -- 172.21.15.101:0/1047388616 >> 172.21.15.101:0/1047388616 conn(0x55d4b1484da0 msgr2=0x55d4b148a230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:26:45.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.115+0000 7f7c4260cb80 1 -- 172.21.15.101:0/1047388616 shutdown_connections 2023-07-18T03:26:45.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.115+0000 7f7c4260cb80 1 -- 172.21.15.101:0/1047388616 wait complete. 2023-07-18T03:26:45.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.115+0000 7f7c4260cb80 20 client.28984 trim_cache size 0 max 16384 2023-07-18T03:26:45.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.132+0000 7f7c4260cb80 1 Processor -- start 2023-07-18T03:26:45.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.132+0000 7f7c4260cb80 1 -- start start 2023-07-18T03:26:45.156 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.133+0000 7f7c4260cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4b148e2d0 0x55d4b149d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:26:45.156 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.133+0000 7f7c4260cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4b148ebe0 0x55d4b149a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:26:45.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.133+0000 7f7c4260cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4b149dee0 0x55d4b149a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:26:45.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.133+0000 7f7c4260cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4b148bd20 con 0x55d4b148ebe0 2023-07-18T03:26:45.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.133+0000 7f7c4260cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4b14965f0 con 0x55d4b148e2d0 2023-07-18T03:26:45.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.133+0000 7f7c4260cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4b148f540 con 0x55d4b149dee0 2023-07-18T03:26:45.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.133+0000 7f7c2d19c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4b149dee0 0x55d4b149a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:26:45.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.133+0000 7f7c27fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4b148ebe0 0x55d4b149a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:26:45.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.133+0000 7f7c2c99b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4b148e2d0 0x55d4b149d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:26:45.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.133+0000 7f7c2d19c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4b149dee0 0x55d4b149a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:55502/0 (socket says 172.21.15.101:55502) 2023-07-18T03:26:45.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.133+0000 7f7c2d19c700 1 -- 172.21.15.101:0/372668872 learned_addr learned my addr 172.21.15.101:0/372668872 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:26:45.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.133+0000 7f7c2d19c700 1 -- 172.21.15.101:0/372668872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4b148e2d0 msgr2=0x55d4b149d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:26:45.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.133+0000 7f7c2d19c700 1 --2- 172.21.15.101:0/372668872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4b148e2d0 0x55d4b149d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:45.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.133+0000 7f7c2d19c700 1 -- 172.21.15.101:0/372668872 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4b148ebe0 msgr2=0x55d4b149a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:26:45.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.133+0000 7f7c2d19c700 1 --2- 172.21.15.101:0/372668872 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4b148ebe0 0x55d4b149a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:45.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.133+0000 7f7c2d19c700 1 -- 172.21.15.101:0/372668872 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4b1235250 con 0x55d4b149dee0 2023-07-18T03:26:45.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.133+0000 7f7c27fff700 1 --2- 172.21.15.101:0/372668872 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4b148ebe0 0x55d4b149a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:26:45.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.134+0000 7f7c2d19c700 1 --2- 172.21.15.101:0/372668872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4b149dee0 0x55d4b149a880 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f7c200099e0 tx=0x7f7c2001f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46c6de41b93912a1 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:26:45.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.134+0000 7f7c137fe700 1 -- 172.21.15.101:0/372668872 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2000e040 con 0x55d4b149dee0 2023-07-18T03:26:45.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.134+0000 7f7c137fe700 1 -- 172.21.15.101:0/372668872 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c2000f040 con 0x55d4b149dee0 2023-07-18T03:26:45.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.134+0000 7f7c137fe700 1 -- 172.21.15.101:0/372668872 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2001a7c0 con 0x55d4b149dee0 2023-07-18T03:26:45.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.134+0000 7f7c4260cb80 1 -- 172.21.15.101:0/372668872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4b149dee0 msgr2=0x55d4b149a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:26:45.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.134+0000 7f7c4260cb80 1 --2- 172.21.15.101:0/372668872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4b149dee0 0x55d4b149a880 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f7c200099e0 tx=0x7f7c2001f000 comp rx=0 tx=0).stop 2023-07-18T03:26:45.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.134+0000 7f7c4260cb80 1 -- 172.21.15.101:0/372668872 shutdown_connections 2023-07-18T03:26:45.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.134+0000 7f7c4260cb80 1 --2- 172.21.15.101:0/372668872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4b148e2d0 0x55d4b149d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:45.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.134+0000 7f7c4260cb80 1 --2- 172.21.15.101:0/372668872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4b149dee0 0x55d4b149a880 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:45.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.134+0000 7f7c4260cb80 1 --2- 172.21.15.101:0/372668872 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4b148ebe0 0x55d4b149a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:45.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.134+0000 7f7c4260cb80 1 -- 172.21.15.101:0/372668872 >> 172.21.15.101:0/372668872 conn(0x55d4b1484da0 msgr2=0x55d4b1495ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:26:45.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.134+0000 7f7c4260cb80 1 -- 172.21.15.101:0/372668872 shutdown_connections 2023-07-18T03:26:45.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.134+0000 7f7c4260cb80 1 -- 172.21.15.101:0/372668872 wait complete. 2023-07-18T03:26:45.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.135+0000 7f7c4260cb80 1 -- start start 2023-07-18T03:26:45.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.135+0000 7f7c4260cb80 1 Processor -- start 2023-07-18T03:26:45.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.136+0000 7f7c4260cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4b148e2d0 0x55d4b14a4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:26:45.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.136+0000 7f7c4260cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4b148ebe0 0x55d4b14a2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:26:45.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.136+0000 7f7c4260cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4b149dee0 0x55d4b14a3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:26:45.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.136+0000 7f7c4260cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4b14a3830 con 0x55d4b148e2d0 2023-07-18T03:26:45.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.136+0000 7f7c4260cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4b14a39a0 con 0x55d4b149dee0 2023-07-18T03:26:45.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.136+0000 7f7c4260cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4b14a3b60 con 0x55d4b148ebe0 2023-07-18T03:26:45.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.136+0000 7f7c4260cb80 10 client.0 _ll_register_callbacks cb 0x7f7c14005c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:26:45.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.136+0000 7f7c2d19c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4b149dee0 0x55d4b14a3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:26:45.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.136+0000 7f7c27fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4b148ebe0 0x55d4b14a2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:26:45.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.136+0000 7f7c2c99b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4b148e2d0 0x55d4b14a4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:26:45.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.136+0000 7f7c2d19c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4b149dee0 0x55d4b14a3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47410/0 (socket says 172.21.15.101:47410) 2023-07-18T03:26:45.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.136+0000 7f7c2d19c700 1 -- 172.21.15.101:0/2741800634 learned_addr learned my addr 172.21.15.101:0/2741800634 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:26:45.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.136+0000 7f7c2c99b700 1 -- 172.21.15.101:0/2741800634 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4b148ebe0 msgr2=0x55d4b14a2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:26:45.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.136+0000 7f7c2c99b700 1 --2- 172.21.15.101:0/2741800634 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4b148ebe0 0x55d4b14a2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:45.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.136+0000 7f7c2c99b700 1 -- 172.21.15.101:0/2741800634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4b149dee0 msgr2=0x55d4b14a3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:26:45.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.136+0000 7f7c2c99b700 1 --2- 172.21.15.101:0/2741800634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4b149dee0 0x55d4b14a3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:45.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.137+0000 7f7c2c99b700 1 -- 172.21.15.101:0/2741800634 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c14095e70 con 0x55d4b148e2d0 2023-07-18T03:26:45.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.137+0000 7f7c27fff700 1 --2- 172.21.15.101:0/2741800634 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4b148ebe0 0x55d4b14a2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:26:45.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.137+0000 7f7c2d19c700 1 --2- 172.21.15.101:0/2741800634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4b149dee0 0x55d4b14a3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:26:45.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.137+0000 7f7c2c99b700 1 --2- 172.21.15.101:0/2741800634 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4b148e2d0 0x55d4b14a4860 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c0225c0 tx=0x7f7c1c022ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15f74a85649e3346 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:26:45.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.137+0000 7f7c07fff700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:26:45.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.137+0000 7f7c07fff700 1 -- 172.21.15.101:0/2741800634 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c003b30 con 0x55d4b148e2d0 2023-07-18T03:26:45.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.137+0000 7f7c4260cb80 1 -- 172.21.15.101:0/2741800634 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7c1c004390 con 0x55d4b148e2d0 2023-07-18T03:26:45.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.137+0000 7f7c07fff700 1 -- 172.21.15.101:0/2741800634 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f7c1c022830 con 0x55d4b148e2d0 2023-07-18T03:26:45.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.137+0000 7f7c07fff700 1 -- 172.21.15.101:0/2741800634 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c1c005250 con 0x55d4b148e2d0 2023-07-18T03:26:45.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.137+0000 7f7c4260cb80 10 client.18903 fetch_fsmap learned FSMap version 46 2023-07-18T03:26:45.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.137+0000 7f7c4260cb80 1 -- 172.21.15.101:0/2741800634 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7f7c20009140 con 0x55d4b148e2d0 2023-07-18T03:26:45.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.137+0000 7f7c07fff700 1 -- 172.21.15.101:0/2741800634 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c1c004390 con 0x55d4b148e2d0 2023-07-18T03:26:45.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.138+0000 7f7c07fff700 1 -- 172.21.15.101:0/2741800634 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 46) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f7c1c011790 con 0x55d4b148e2d0 2023-07-18T03:26:45.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.138+0000 7f7c4260cb80 10 client.18903 fetch_fsmap finished waiting for FSMap version 46 2023-07-18T03:26:45.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.138+0000 7f7c4260cb80 10 client.18903 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:26:45.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.138+0000 7f7c4260cb80 20 client.18903 populate_metadata read hostname 'smithi101' 2023-07-18T03:26:45.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.138+0000 7f7c4260cb80 1 --2- 172.21.15.101:0/2741800634 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55d4b15027c0 0x55d4b1504ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:26:45.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.138+0000 7f7c4260cb80 4 client.18903 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:26:45.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.138+0000 7f7c4260cb80 1 -- 172.21.15.101:0/2741800634 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d4b139c050 con 0x55d4b15027c0 2023-07-18T03:26:45.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.138+0000 7f7c27fff700 1 --2- 172.21.15.101:0/2741800634 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55d4b15027c0 0x55d4b1504ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:26:45.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.138+0000 7f7c27fff700 1 --2- 172.21.15.101:0/2741800634 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55d4b15027c0 0x55d4b1504ba0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92c13e124ea1caed server_cookie=2c934aad41cace07 in_seq=0 out_seq=0 2023-07-18T03:26:45.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.138+0000 7f7c07fff700 10 client.18903 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:26:45.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.139+0000 7f7c07fff700 1 -- 172.21.15.101:0/2741800634 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55d4b139c050 con 0x55d4b15027c0 2023-07-18T03:26:45.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.139+0000 7f7c07fff700 10 client.18903 handle_command_reply: tid=0 2023-07-18T03:26:45.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.139+0000 7f7c4260cb80 1 client.18903 shutdown 2023-07-18T03:26:45.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.139+0000 7f7c4260cb80 1 -- 172.21.15.101:0/2741800634 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4b148e2d0 msgr2=0x55d4b14a4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:26:45.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.139+0000 7f7c4260cb80 1 --2- 172.21.15.101:0/2741800634 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4b148e2d0 0x55d4b14a4860 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c0225c0 tx=0x7f7c1c022ca0 comp rx=0 tx=0).stop 2023-07-18T03:26:45.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.139+0000 7f7c4260cb80 1 -- 172.21.15.101:0/2741800634 shutdown_connections 2023-07-18T03:26:45.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.139+0000 7f7c4260cb80 1 --2- 172.21.15.101:0/2741800634 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55d4b15027c0 0x55d4b1504ba0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:45.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.139+0000 7f7c4260cb80 1 --2- 172.21.15.101:0/2741800634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4b149dee0 0x55d4b14a3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:45.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.139+0000 7f7c4260cb80 1 --2- 172.21.15.101:0/2741800634 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4b148e2d0 0x55d4b14a4860 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:45.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.139+0000 7f7c07fff700 0 client.18903 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:26:45.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.139+0000 7f7c4260cb80 1 --2- 172.21.15.101:0/2741800634 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4b148ebe0 0x55d4b14a2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:45.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.139+0000 7f7c4260cb80 1 -- 172.21.15.101:0/2741800634 >> 172.21.15.101:0/2741800634 conn(0x55d4b1484da0 msgr2=0x55d4b14901a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:26:45.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.139+0000 7f7c4260cb80 1 -- 172.21.15.101:0/2741800634 shutdown_connections 2023-07-18T03:26:45.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.139+0000 7f7c4260cb80 1 -- 172.21.15.101:0/2741800634 wait complete. 2023-07-18T03:26:45.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.139+0000 7f7c4260cb80 20 client.18903 trim_cache size 0 max 16384 2023-07-18T03:26:45.182 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:26:45.183 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:26:45.183 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:26:45.183 INFO:teuthology.orchestra.run.smithi101.stdout: "0f2f19e0-0429-465c-935e-223eb09490c3": { 2023-07-18T03:26:45.183 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:26:45.184 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "0f2f19e0-0429-465c-935e-223eb09490c3", 2023-07-18T03:26:45.184 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:26:45.184 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:26:45.184 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:26:45.184 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:26:45.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.139+0000 7f7c2ebff700 1 -- 172.21.15.101:0/1326910895 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f7c180a12c0 msgr2=0x7f7c180a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:26:45.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.139+0000 7f7c2ebff700 1 --2- 172.21.15.101:0/1326910895 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f7c180a12c0 0x7f7c180a3770 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7c2819c6d0 tx=0x7f7c2000fc00 comp rx=0 tx=0).stop 2023-07-18T03:26:45.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.139+0000 7f7c2ebff700 1 -- 172.21.15.101:0/1326910895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c28100480 msgr2=0x7f7c2819bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:26:45.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.139+0000 7f7c2ebff700 1 --2- 172.21.15.101:0/1326910895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c28100480 0x7f7c2819bf10 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f7c14000c00 tx=0x7f7c14005720 comp rx=0 tx=0).stop 2023-07-18T03:26:45.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.140+0000 7f7c2ebff700 1 -- 172.21.15.101:0/1326910895 shutdown_connections 2023-07-18T03:26:45.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.140+0000 7f7c2ebff700 1 --2- 172.21.15.101:0/1326910895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c28105ae0 0x7f7c28195a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:45.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.140+0000 7f7c2ebff700 1 --2- 172.21.15.101:0/1326910895 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7c280ffad0 0x7f7c2819b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:45.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.140+0000 7f7c2ebff700 1 --2- 172.21.15.101:0/1326910895 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f7c180a12c0 0x7f7c180a3770 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:45.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.140+0000 7f7c2ebff700 1 --2- 172.21.15.101:0/1326910895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c28100480 0x7f7c2819bf10 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:45.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.140+0000 7f7c2ebff700 1 -- 172.21.15.101:0/1326910895 >> 172.21.15.101:0/1326910895 conn(0x7f7c280fb5f0 msgr2=0x7f7c280fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:26:45.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.140+0000 7f7c2ebff700 1 -- 172.21.15.101:0/1326910895 shutdown_connections 2023-07-18T03:26:45.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:26:45.140+0000 7f7c2ebff700 1 -- 172.21.15.101:0/1326910895 wait complete. 2023-07-18T03:26:45.188 INFO:tasks.cephfs.filesystem:scrub status for tag:0f2f19e0-0429-465c-935e-223eb09490c3 - {'path': '/', 'tag': '0f2f19e0-0429-465c-935e-223eb09490c3', 'options': 'recursive,force'} 2023-07-18T03:26:45.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:26:45 smithi179 ceph-mon[129229]: pgmap v604: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.1 KiB/s wr, 5 op/s 2023-07-18T03:26:45.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:26:45 smithi101 ceph-mon[126379]: pgmap v604: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.1 KiB/s wr, 5 op/s 2023-07-18T03:26:45.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:26:45 smithi155 ceph-mon[128885]: pgmap v604: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.1 KiB/s wr, 5 op/s 2023-07-18T03:26:47.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:26:47 smithi179 ceph-mon[129229]: pgmap v605: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2023-07-18T03:26:47.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:26:47 smithi101 ceph-mon[126379]: pgmap v605: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2023-07-18T03:26:47.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:26:47 smithi155 ceph-mon[128885]: pgmap v605: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2023-07-18T03:26:49.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:26:49 smithi179 ceph-mon[129229]: pgmap v606: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2023-07-18T03:26:49.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:26:49 smithi101 ceph-mon[126379]: pgmap v606: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2023-07-18T03:26:49.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:26:49 smithi155 ceph-mon[128885]: pgmap v606: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2023-07-18T03:26:51.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:26:51 smithi179 ceph-mon[129229]: pgmap v607: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2023-07-18T03:26:51.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:26:51 smithi101 ceph-mon[126379]: pgmap v607: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2023-07-18T03:26:51.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:26:51 smithi155 ceph-mon[128885]: pgmap v607: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2023-07-18T03:26:53.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:26:53 smithi179 ceph-mon[129229]: pgmap v608: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 6 op/s 2023-07-18T03:26:53.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:26:53 smithi101 ceph-mon[126379]: pgmap v608: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 6 op/s 2023-07-18T03:26:53.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:26:53 smithi155 ceph-mon[128885]: pgmap v608: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 6 op/s 2023-07-18T03:26:55.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:26:55 smithi179 ceph-mon[129229]: pgmap v609: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2023-07-18T03:26:55.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:26:55 smithi101 ceph-mon[126379]: pgmap v609: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2023-07-18T03:26:55.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:26:55 smithi155 ceph-mon[128885]: pgmap v609: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2023-07-18T03:26:57.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:26:57 smithi179 ceph-mon[129229]: pgmap v610: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 7 op/s 2023-07-18T03:26:57.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:26:57 smithi101 ceph-mon[126379]: pgmap v610: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 7 op/s 2023-07-18T03:26:57.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:26:57 smithi155 ceph-mon[128885]: pgmap v610: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 7 op/s 2023-07-18T03:26:59.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:26:59 smithi179 ceph-mon[129229]: pgmap v611: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2023-07-18T03:26:59.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:26:59 smithi101 ceph-mon[126379]: pgmap v611: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2023-07-18T03:26:59.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:26:59 smithi155 ceph-mon[128885]: pgmap v611: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2023-07-18T03:27:00.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:27:00 smithi179 ceph-mon[129229]: pgmap v612: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 6 op/s 2023-07-18T03:27:00.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:27:00 smithi101 ceph-mon[126379]: pgmap v612: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 6 op/s 2023-07-18T03:27:00.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:27:00 smithi155 ceph-mon[128885]: pgmap v612: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 6 op/s 2023-07-18T03:27:03.299 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:27:03 smithi155 ceph-mon[128885]: pgmap v613: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 22 KiB/s wr, 8 op/s 2023-07-18T03:27:03.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:27:03 smithi179 ceph-mon[129229]: pgmap v613: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 22 KiB/s wr, 8 op/s 2023-07-18T03:27:03.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:27:03 smithi101 ceph-mon[126379]: pgmap v613: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 22 KiB/s wr, 8 op/s 2023-07-18T03:27:05.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:27:05 smithi101 ceph-mon[126379]: pgmap v614: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-07-18T03:27:05.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:27:05 smithi155 ceph-mon[128885]: pgmap v614: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-07-18T03:27:05.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:27:05 smithi179 ceph-mon[129229]: pgmap v614: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-07-18T03:27:07.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:27:07 smithi101 ceph-mon[126379]: pgmap v615: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 58 KiB/s wr, 22 op/s 2023-07-18T03:27:07.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:27:07 smithi101 ceph-mon[126379]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:27:07.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:27:07 smithi155 ceph-mon[128885]: pgmap v615: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 58 KiB/s wr, 22 op/s 2023-07-18T03:27:07.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:27:07 smithi155 ceph-mon[128885]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:27:07.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:27:07 smithi179 ceph-mon[129229]: pgmap v615: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 58 KiB/s wr, 22 op/s 2023-07-18T03:27:07.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:27:07 smithi179 ceph-mon[129229]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:27:09.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:27:09 smithi101 ceph-mon[126379]: pgmap v616: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 58 KiB/s wr, 20 op/s 2023-07-18T03:27:09.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:27:09 smithi155 ceph-mon[128885]: pgmap v616: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 58 KiB/s wr, 20 op/s 2023-07-18T03:27:09.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:27:09 smithi179 ceph-mon[129229]: pgmap v616: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 58 KiB/s wr, 20 op/s 2023-07-18T03:27:11.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:27:11 smithi101 ceph-mon[126379]: pgmap v617: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 66 KiB/s wr, 22 op/s 2023-07-18T03:27:11.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:27:11 smithi155 ceph-mon[128885]: pgmap v617: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 66 KiB/s wr, 22 op/s 2023-07-18T03:27:11.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:27:11 smithi179 ceph-mon[129229]: pgmap v617: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 66 KiB/s wr, 22 op/s 2023-07-18T03:27:12.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:27:12 smithi155 ceph-mon[128885]: osdmap e119: 12 total, 12 up, 12 in 2023-07-18T03:27:12.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:27:12 smithi179 ceph-mon[129229]: osdmap e119: 12 total, 12 up, 12 in 2023-07-18T03:27:12.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:27:12 smithi101 ceph-mon[126379]: osdmap e119: 12 total, 12 up, 12 in 2023-07-18T03:27:13.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:27:13 smithi155 ceph-mon[128885]: pgmap v619: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 52 KiB/s wr, 22 op/s 2023-07-18T03:27:13.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:27:13 smithi179 ceph-mon[129229]: pgmap v619: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 52 KiB/s wr, 22 op/s 2023-07-18T03:27:13.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:27:13 smithi101 ceph-mon[126379]: pgmap v619: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 52 KiB/s wr, 22 op/s 2023-07-18T03:27:15.189 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:27:15.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.400+0000 7f2cfe9ff700 1 -- 172.21.15.101:0/435532805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cf80728b0 msgr2=0x7f2cf8072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:27:15.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.400+0000 7f2cfe9ff700 1 --2- 172.21.15.101:0/435532805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cf80728b0 0x7f2cf8072c80 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8009990 tx=0x7f2ce8009ca0 comp rx=0 tx=0).stop 2023-07-18T03:27:15.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.400+0000 7f2cfe9ff700 1 -- 172.21.15.101:0/435532805 shutdown_connections 2023-07-18T03:27:15.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.400+0000 7f2cfe9ff700 1 --2- 172.21.15.101:0/435532805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cf80728b0 0x7f2cf8072c80 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:15.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.400+0000 7f2cfe9ff700 1 --2- 172.21.15.101:0/435532805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cf8110ae0 0x7f2cf8112ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:15.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.400+0000 7f2cfe9ff700 1 --2- 172.21.15.101:0/435532805 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2cf80731c0 0x7f2cf81105a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:15.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.400+0000 7f2cfe9ff700 1 -- 172.21.15.101:0/435532805 >> 172.21.15.101:0/435532805 conn(0x7f2cf80fb5f0 msgr2=0x7f2cf80fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:27:15.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.401+0000 7f2cfe9ff700 1 -- 172.21.15.101:0/435532805 shutdown_connections 2023-07-18T03:27:15.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.401+0000 7f2cfe9ff700 1 -- 172.21.15.101:0/435532805 wait complete. 2023-07-18T03:27:15.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.402+0000 7f2cfe9ff700 1 Processor -- start 2023-07-18T03:27:15.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.402+0000 7f2cfe9ff700 1 -- start start 2023-07-18T03:27:15.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.402+0000 7f2cfe9ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cf80728b0 0x7f2cf819c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:15.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.402+0000 7f2cfe9ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2cf80731c0 0x7f2cf819c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:15.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.402+0000 7f2cfe9ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cf8110ae0 0x7f2cf81a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:15.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.402+0000 7f2cfe9ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf81a1290 con 0x7f2cf80731c0 2023-07-18T03:27:15.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.402+0000 7f2cfe9ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf81a1400 con 0x7f2cf80728b0 2023-07-18T03:27:15.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.402+0000 7f2cfe9ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf81a15c0 con 0x7f2cf8110ae0 2023-07-18T03:27:15.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.402+0000 7f2cf7fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cf80728b0 0x7f2cf819c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:15.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.402+0000 7f2cf7fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cf80728b0 0x7f2cf819c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:44066/0 (socket says 172.21.15.101:44066) 2023-07-18T03:27:15.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.402+0000 7f2cf7fff700 1 -- 172.21.15.101:0/1964732033 learned_addr learned my addr 172.21.15.101:0/1964732033 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:27:15.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.402+0000 7f2cfcf9c700 1 --2- 172.21.15.101:0/1964732033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cf8110ae0 0x7f2cf81a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:15.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.402+0000 7f2cf77fe700 1 --2- 172.21.15.101:0/1964732033 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2cf80731c0 0x7f2cf819c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:15.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.403+0000 7f2cf7fff700 1 -- 172.21.15.101:0/1964732033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cf8110ae0 msgr2=0x7f2cf81a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:27:15.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.403+0000 7f2cf7fff700 1 --2- 172.21.15.101:0/1964732033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cf8110ae0 0x7f2cf81a0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:15.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.403+0000 7f2cf7fff700 1 -- 172.21.15.101:0/1964732033 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2cf80731c0 msgr2=0x7f2cf819c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:27:15.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.403+0000 7f2cf7fff700 1 --2- 172.21.15.101:0/1964732033 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2cf80731c0 0x7f2cf819c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:15.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.403+0000 7f2cf7fff700 1 -- 172.21.15.101:0/1964732033 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce80095f0 con 0x7f2cf80728b0 2023-07-18T03:27:15.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.403+0000 7f2cfcf9c700 1 --2- 172.21.15.101:0/1964732033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cf8110ae0 0x7f2cf81a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:27:15.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.403+0000 7f2cf7fff700 1 --2- 172.21.15.101:0/1964732033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cf80728b0 0x7f2cf819c2b0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8000c00 tx=0x7f2ce8005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:27:15.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.403+0000 7f2cf57fa700 1 -- 172.21.15.101:0/1964732033 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce801d070 con 0x7f2cf80728b0 2023-07-18T03:27:15.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.403+0000 7f2cf57fa700 1 -- 172.21.15.101:0/1964732033 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ce8022470 con 0x7f2cf80728b0 2023-07-18T03:27:15.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.403+0000 7f2cf57fa700 1 -- 172.21.15.101:0/1964732033 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce8017630 con 0x7f2cf80728b0 2023-07-18T03:27:15.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.403+0000 7f2cfe9ff700 1 -- 172.21.15.101:0/1964732033 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cf81a1840 con 0x7f2cf80728b0 2023-07-18T03:27:15.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.404+0000 7f2cfe9ff700 1 -- 172.21.15.101:0/1964732033 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cf81a8f00 con 0x7f2cf80728b0 2023-07-18T03:27:15.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.408+0000 7f2cf57fa700 1 -- 172.21.15.101:0/1964732033 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f2ce800fad0 con 0x7f2cf80728b0 2023-07-18T03:27:15.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.409+0000 7f2d123ddb80 1 Processor -- start 2023-07-18T03:27:15.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.409+0000 7f2d123ddb80 1 -- start start 2023-07-18T03:27:15.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.410+0000 7f2d123ddb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbeb7bf2d0 0x55dbeb7bf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:15.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.410+0000 7f2d123ddb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbeb7bfbe0 0x55dbeb7ca590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:15.409 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.410+0000 7f2d123ddb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dbeb7caad0 0x55dbeb7cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:15.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.410+0000 7f2d123ddb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbeb6c5cb0 con 0x55dbeb7caad0 2023-07-18T03:27:15.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.410+0000 7f2d123ddb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbeb718a40 con 0x55dbeb7bf2d0 2023-07-18T03:27:15.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.410+0000 7f2cf57fa700 1 --2- 172.21.15.101:0/1964732033 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2ce00a1340 0x7f2ce00a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:15.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.410+0000 7f2d123ddb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbeb711ed0 con 0x55dbeb7bfbe0 2023-07-18T03:27:15.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.410+0000 7f2cf57fa700 1 -- 172.21.15.101:0/1964732033 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f2ce80ce4a0 con 0x7f2cf80728b0 2023-07-18T03:27:15.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.410+0000 7f2cf7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbeb7bfbe0 0x55dbeb7ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:15.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.410+0000 7f2cf7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbeb7bfbe0 0x55dbeb7ca590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:49812/0 (socket says 172.21.15.101:49812) 2023-07-18T03:27:15.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.410+0000 7f2cf7fff700 1 -- 172.21.15.101:0/3817289116 learned_addr learned my addr 172.21.15.101:0/3817289116 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:27:15.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.410+0000 7f2cfcf9c700 1 --2- 172.21.15.101:0/3817289116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbeb7bf2d0 0x55dbeb7bf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:15.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.410+0000 7f2cf77fe700 1 --2- 172.21.15.101:0/3817289116 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dbeb7caad0 0x55dbeb7cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:15.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.410+0000 7f2cfcf9c700 1 --2- 172.21.15.101:0/1964732033 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2ce00a1340 0x7f2ce00a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:15.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.411+0000 7f2cf7fff700 1 -- 172.21.15.101:0/3817289116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbeb7bf2d0 msgr2=0x55dbeb7bf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:27:15.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.411+0000 7f2cf7fff700 1 --2- 172.21.15.101:0/3817289116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbeb7bf2d0 0x55dbeb7bf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:15.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.411+0000 7f2cf7fff700 1 -- 172.21.15.101:0/3817289116 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dbeb7caad0 msgr2=0x55dbeb7cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:27:15.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.411+0000 7f2cf7fff700 1 --2- 172.21.15.101:0/3817289116 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dbeb7caad0 0x55dbeb7cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:15.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.411+0000 7f2cf7fff700 1 -- 172.21.15.101:0/3817289116 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dbeb53fde0 con 0x55dbeb7bfbe0 2023-07-18T03:27:15.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.411+0000 7f2cfcf9c700 1 --2- 172.21.15.101:0/3817289116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbeb7bf2d0 0x55dbeb7bf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:27:15.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.411+0000 7f2cf77fe700 1 --2- 172.21.15.101:0/3817289116 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dbeb7caad0 0x55dbeb7cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:27:15.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.411+0000 7f2cf7fff700 1 --2- 172.21.15.101:0/3817289116 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbeb7bfbe0 0x55dbeb7ca590 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2ce80cb7f0 tx=0x7f2ce8022af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24f9ecb89461800f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:27:15.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.411+0000 7f2cde7fc700 1 -- 172.21.15.101:0/3817289116 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce809af70 con 0x55dbeb7bfbe0 2023-07-18T03:27:15.414 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.411+0000 7f2cfcf9c700 1 --2- 172.21.15.101:0/1964732033 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2ce00a1340 0x7f2ce00a37f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0009550 tx=0x7f2cf000b040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:27:15.414 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.411+0000 7f2cde7fc700 1 -- 172.21.15.101:0/3817289116 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ce809ace0 con 0x55dbeb7bfbe0 2023-07-18T03:27:15.414 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.411+0000 7f2cde7fc700 1 -- 172.21.15.101:0/3817289116 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce800fd80 con 0x55dbeb7bfbe0 2023-07-18T03:27:15.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.412+0000 7f2d123ddb80 1 -- 172.21.15.101:0/3817289116 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbeb7bfbe0 msgr2=0x55dbeb7ca590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:27:15.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.412+0000 7f2d123ddb80 1 --2- 172.21.15.101:0/3817289116 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbeb7bfbe0 0x55dbeb7ca590 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2ce80cb7f0 tx=0x7f2ce8022af0 comp rx=0 tx=0).stop 2023-07-18T03:27:15.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.412+0000 7f2d123ddb80 1 -- 172.21.15.101:0/3817289116 shutdown_connections 2023-07-18T03:27:15.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.412+0000 7f2d123ddb80 1 --2- 172.21.15.101:0/3817289116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbeb7bf2d0 0x55dbeb7bf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:15.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.412+0000 7f2d123ddb80 1 --2- 172.21.15.101:0/3817289116 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dbeb7caad0 0x55dbeb7cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:15.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.412+0000 7f2d123ddb80 1 --2- 172.21.15.101:0/3817289116 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbeb7bfbe0 0x55dbeb7ca590 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:15.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.412+0000 7f2d123ddb80 1 -- 172.21.15.101:0/3817289116 >> 172.21.15.101:0/3817289116 conn(0x55dbeb7b5da0 msgr2=0x55dbeb7bb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:27:15.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.412+0000 7f2d123ddb80 1 -- 172.21.15.101:0/3817289116 shutdown_connections 2023-07-18T03:27:15.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.412+0000 7f2d123ddb80 1 -- 172.21.15.101:0/3817289116 wait complete. 2023-07-18T03:27:15.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.413+0000 7f2d123ddb80 1 -- start start 2023-07-18T03:27:15.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.413+0000 7f2d123ddb80 1 Processor -- start 2023-07-18T03:27:15.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.413+0000 7f2d123ddb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbeb7bf2d0 0x55dbeb7caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:15.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.413+0000 7f2d123ddb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbeb7bfbe0 0x55dbeb7cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:15.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.413+0000 7f2d123ddb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dbeb7caad0 0x55dbeb7cb920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:15.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.413+0000 7f2d123ddb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbeb6b1ac0 con 0x55dbeb7caad0 2023-07-18T03:27:15.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.413+0000 7f2d123ddb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbeb6b17c0 con 0x55dbeb7bfbe0 2023-07-18T03:27:15.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.413+0000 7f2d123ddb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbeb595920 con 0x55dbeb7bf2d0 2023-07-18T03:27:15.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.413+0000 7f2d123ddb80 10 client.0 _ll_register_callbacks cb 0x55dbeb6af760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:27:15.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.414+0000 7f2cf77fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbeb7bfbe0 0x55dbeb7cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:15.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.414+0000 7f2cf7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbeb7bf2d0 0x55dbeb7caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:15.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.414+0000 7f2cfcf9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dbeb7caad0 0x55dbeb7cb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:15.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.414+0000 7f2cf77fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbeb7bfbe0 0x55dbeb7cb3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:44088/0 (socket says 172.21.15.101:44088) 2023-07-18T03:27:15.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.414+0000 7f2cf77fe700 1 -- 172.21.15.101:0/777240895 learned_addr learned my addr 172.21.15.101:0/777240895 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:27:15.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.414+0000 7f2cf7fff700 1 -- 172.21.15.101:0/777240895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbeb7bfbe0 msgr2=0x55dbeb7cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:27:15.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.414+0000 7f2cf7fff700 1 --2- 172.21.15.101:0/777240895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbeb7bfbe0 0x55dbeb7cb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:15.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.414+0000 7f2cf7fff700 1 -- 172.21.15.101:0/777240895 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dbeb7caad0 msgr2=0x55dbeb7cb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:27:15.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.414+0000 7f2cf7fff700 1 --2- 172.21.15.101:0/777240895 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dbeb7caad0 0x55dbeb7cb920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:15.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.414+0000 7f2cf7fff700 1 -- 172.21.15.101:0/777240895 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cf0008a40 con 0x55dbeb7bf2d0 2023-07-18T03:27:15.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.414+0000 7f2cf77fe700 1 --2- 172.21.15.101:0/777240895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbeb7bfbe0 0x55dbeb7cb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:27:15.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.414+0000 7f2cf7fff700 1 --2- 172.21.15.101:0/777240895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbeb7bf2d0 0x55dbeb7caea0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x55dbeb7c00d0 tx=0x7f2ce80a1170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55488e9c47f4acd1 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:27:15.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.414+0000 7f2cd77fe700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:27:15.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.414+0000 7f2cd77fe700 1 -- 172.21.15.101:0/777240895 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce809ab00 con 0x55dbeb7bf2d0 2023-07-18T03:27:15.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.414+0000 7f2cd77fe700 1 -- 172.21.15.101:0/777240895 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f2ce809b1a0 con 0x55dbeb7bf2d0 2023-07-18T03:27:15.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.414+0000 7f2d123ddb80 1 -- 172.21.15.101:0/777240895 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dbeb66b4f0 con 0x55dbeb7bf2d0 2023-07-18T03:27:15.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.415+0000 7f2cd77fe700 1 -- 172.21.15.101:0/777240895 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ce809f1d0 con 0x55dbeb7bf2d0 2023-07-18T03:27:15.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.415+0000 7f2d123ddb80 10 client.29002 fetch_fsmap learned FSMap version 47 2023-07-18T03:27:15.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.415+0000 7f2d123ddb80 1 -- 172.21.15.101:0/777240895 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x7f2cec0098e0 con 0x55dbeb7bf2d0 2023-07-18T03:27:15.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.415+0000 7f2cd77fe700 1 -- 172.21.15.101:0/777240895 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dbeb66b4f0 con 0x55dbeb7bf2d0 2023-07-18T03:27:15.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.415+0000 7f2cd77fe700 1 -- 172.21.15.101:0/777240895 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 47) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f2ce809b400 con 0x55dbeb7bf2d0 2023-07-18T03:27:15.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.415+0000 7f2d123ddb80 10 client.29002 fetch_fsmap finished waiting for FSMap version 47 2023-07-18T03:27:15.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.415+0000 7f2d123ddb80 10 client.29002 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:27:15.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.415+0000 7f2d123ddb80 20 client.29002 populate_metadata read hostname 'smithi101' 2023-07-18T03:27:15.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.415+0000 7f2d123ddb80 1 --2- 172.21.15.101:0/777240895 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55dbeb7d9d30 0x55dbeb7dc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:15.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.415+0000 7f2d123ddb80 4 client.29002 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:27:15.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.415+0000 7f2d123ddb80 1 -- 172.21.15.101:0/777240895 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dbeb70aeb0 con 0x55dbeb7d9d30 2023-07-18T03:27:15.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.415+0000 7f2cf77fe700 1 --2- 172.21.15.101:0/777240895 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55dbeb7d9d30 0x55dbeb7dc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:15.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.416+0000 7f2cd77fe700 10 client.29002 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:27:15.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.416+0000 7f2cf77fe700 1 --2- 172.21.15.101:0/777240895 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55dbeb7d9d30 0x55dbeb7dc140 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1d661387bbbbd9f server_cookie=c317a4407de5d4c0 in_seq=0 out_seq=0 2023-07-18T03:27:15.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.417+0000 7f2cd77fe700 1 -- 172.21.15.101:0/777240895 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55dbeb70aeb0 con 0x55dbeb7d9d30 2023-07-18T03:27:15.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.417+0000 7f2cd77fe700 10 client.29002 handle_command_reply: tid=0 2023-07-18T03:27:15.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.417+0000 7f2d123ddb80 1 client.29002 shutdown 2023-07-18T03:27:15.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.417+0000 7f2d123ddb80 1 -- 172.21.15.101:0/777240895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbeb7bf2d0 msgr2=0x55dbeb7caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:27:15.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.417+0000 7f2d123ddb80 1 --2- 172.21.15.101:0/777240895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbeb7bf2d0 0x55dbeb7caea0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x55dbeb7c00d0 tx=0x7f2ce80a1170 comp rx=0 tx=0).stop 2023-07-18T03:27:15.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.417+0000 7f2d123ddb80 1 -- 172.21.15.101:0/777240895 shutdown_connections 2023-07-18T03:27:15.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.417+0000 7f2d123ddb80 1 --2- 172.21.15.101:0/777240895 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55dbeb7d9d30 0x55dbeb7dc140 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:15.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.417+0000 7f2d123ddb80 1 --2- 172.21.15.101:0/777240895 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dbeb7caad0 0x55dbeb7cb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:15.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.417+0000 7f2d123ddb80 1 --2- 172.21.15.101:0/777240895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbeb7bf2d0 0x55dbeb7caea0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:15.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.417+0000 7f2cd77fe700 0 client.29002 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:27:15.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.417+0000 7f2d123ddb80 1 --2- 172.21.15.101:0/777240895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbeb7bfbe0 0x55dbeb7cb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:15.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.417+0000 7f2d123ddb80 1 -- 172.21.15.101:0/777240895 >> 172.21.15.101:0/777240895 conn(0x55dbeb7b5da0 msgr2=0x55dbeb7bb290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:27:15.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.417+0000 7f2d123ddb80 1 -- 172.21.15.101:0/777240895 shutdown_connections 2023-07-18T03:27:15.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.417+0000 7f2d123ddb80 1 -- 172.21.15.101:0/777240895 wait complete. 2023-07-18T03:27:15.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.417+0000 7f2d123ddb80 20 client.29002 trim_cache size 0 max 16384 2023-07-18T03:27:15.432 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.435+0000 7f2d123ddb80 1 Processor -- start 2023-07-18T03:27:15.433 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.435+0000 7f2d123ddb80 1 -- start start 2023-07-18T03:27:15.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.436+0000 7f2d123ddb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbeb7bf2d0 0x55dbeb7ceed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:15.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.436+0000 7f2d123ddb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbeb7bfbe0 0x55dbeb7d5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:15.435 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.436+0000 7f2d123ddb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dbeb7caad0 0x55dbeb7c4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:15.435 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.436+0000 7f2d123ddb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbeb7c0540 con 0x55dbeb7caad0 2023-07-18T03:27:15.435 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.436+0000 7f2d123ddb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbeb7b5970 con 0x55dbeb7bf2d0 2023-07-18T03:27:15.435 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.436+0000 7f2d123ddb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbeb7c11a0 con 0x55dbeb7bfbe0 2023-07-18T03:27:15.435 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.436+0000 7f2cf7fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbeb7bf2d0 0x55dbeb7ceed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:15.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.436+0000 7f2cf7fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbeb7bf2d0 0x55dbeb7ceed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:44098/0 (socket says 172.21.15.101:44098) 2023-07-18T03:27:15.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.436+0000 7f2cf7fff700 1 -- 172.21.15.101:0/2290853535 learned_addr learned my addr 172.21.15.101:0/2290853535 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:27:15.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.436+0000 7f2cf77fe700 1 --2- 172.21.15.101:0/2290853535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbeb7bfbe0 0x55dbeb7d5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:15.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.436+0000 7f2cfcf9c700 1 --2- 172.21.15.101:0/2290853535 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dbeb7caad0 0x55dbeb7c4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:15.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.436+0000 7f2cf7fff700 1 -- 172.21.15.101:0/2290853535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbeb7bfbe0 msgr2=0x55dbeb7d5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:27:15.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.436+0000 7f2cf7fff700 1 --2- 172.21.15.101:0/2290853535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbeb7bfbe0 0x55dbeb7d5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:15.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.436+0000 7f2cf7fff700 1 -- 172.21.15.101:0/2290853535 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dbeb7caad0 msgr2=0x55dbeb7c4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:27:15.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.436+0000 7f2cf7fff700 1 --2- 172.21.15.101:0/2290853535 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dbeb7caad0 0x55dbeb7c4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:15.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.436+0000 7f2cf7fff700 1 -- 172.21.15.101:0/2290853535 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf81a8f00 con 0x55dbeb7bf2d0 2023-07-18T03:27:15.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.436+0000 7f2cfcf9c700 1 --2- 172.21.15.101:0/2290853535 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dbeb7caad0 0x55dbeb7c4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:27:15.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.436+0000 7f2cf77fe700 1 --2- 172.21.15.101:0/2290853535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbeb7bfbe0 0x55dbeb7d5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:27:15.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.437+0000 7f2cf7fff700 1 --2- 172.21.15.101:0/2290853535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbeb7bf2d0 0x55dbeb7ceed0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8097c10 tx=0x7f2ce8099750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a85d53146c86214 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:27:15.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.437+0000 7f2cdeffd700 1 -- 172.21.15.101:0/2290853535 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce809ab00 con 0x55dbeb7bf2d0 2023-07-18T03:27:15.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.437+0000 7f2cdeffd700 1 -- 172.21.15.101:0/2290853535 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ce809af70 con 0x55dbeb7bf2d0 2023-07-18T03:27:15.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.437+0000 7f2cdeffd700 1 -- 172.21.15.101:0/2290853535 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce809a690 con 0x55dbeb7bf2d0 2023-07-18T03:27:15.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.437+0000 7f2d123ddb80 1 -- 172.21.15.101:0/2290853535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbeb7bf2d0 msgr2=0x55dbeb7ceed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:27:15.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.437+0000 7f2d123ddb80 1 --2- 172.21.15.101:0/2290853535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbeb7bf2d0 0x55dbeb7ceed0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8097c10 tx=0x7f2ce8099750 comp rx=0 tx=0).stop 2023-07-18T03:27:15.442 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.438+0000 7f2d123ddb80 1 -- 172.21.15.101:0/2290853535 shutdown_connections 2023-07-18T03:27:15.442 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.438+0000 7f2d123ddb80 1 --2- 172.21.15.101:0/2290853535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbeb7bf2d0 0x55dbeb7ceed0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:15.442 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.438+0000 7f2d123ddb80 1 --2- 172.21.15.101:0/2290853535 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dbeb7caad0 0x55dbeb7c4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:15.442 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.438+0000 7f2d123ddb80 1 --2- 172.21.15.101:0/2290853535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbeb7bfbe0 0x55dbeb7d5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:15.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.438+0000 7f2d123ddb80 1 -- 172.21.15.101:0/2290853535 >> 172.21.15.101:0/2290853535 conn(0x55dbeb7b5da0 msgr2=0x55dbeb7b9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:27:15.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.438+0000 7f2d123ddb80 1 -- 172.21.15.101:0/2290853535 shutdown_connections 2023-07-18T03:27:15.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.438+0000 7f2d123ddb80 1 -- 172.21.15.101:0/2290853535 wait complete. 2023-07-18T03:27:15.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.438+0000 7f2d123ddb80 1 -- start start 2023-07-18T03:27:15.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.438+0000 7f2d123ddb80 1 Processor -- start 2023-07-18T03:27:15.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.439+0000 7f2d123ddb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbeb7bf2d0 0x55dbeb821d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:15.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.439+0000 7f2d123ddb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dbeb7bfbe0 0x55dbeb8222b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:15.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.439+0000 7f2d123ddb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbeb7caad0 0x55dbeb826c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:15.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.439+0000 7f2d123ddb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbeb827180 con 0x55dbeb7bfbe0 2023-07-18T03:27:15.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.439+0000 7f2d123ddb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbeb8272f0 con 0x55dbeb7bf2d0 2023-07-18T03:27:15.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.439+0000 7f2d123ddb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbeb8274b0 con 0x55dbeb7caad0 2023-07-18T03:27:15.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.439+0000 7f2d123ddb80 10 client.0 _ll_register_callbacks cb 0x55dbeb6af760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:27:15.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.439+0000 7f2cf7fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbeb7bf2d0 0x55dbeb821d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:15.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.439+0000 7f2cf7fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbeb7bf2d0 0x55dbeb821d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:44102/0 (socket says 172.21.15.101:44102) 2023-07-18T03:27:15.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.439+0000 7f2cf7fff700 1 -- 172.21.15.101:0/222580854 learned_addr learned my addr 172.21.15.101:0/222580854 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:27:15.446 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.439+0000 7f2cf77fe700 1 --2- 172.21.15.101:0/222580854 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dbeb7bfbe0 0x55dbeb8222b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:15.446 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.439+0000 7f2cfcf9c700 1 --2- 172.21.15.101:0/222580854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbeb7caad0 0x55dbeb826c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:15.446 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.439+0000 7f2cf7fff700 1 -- 172.21.15.101:0/222580854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbeb7caad0 msgr2=0x55dbeb826c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:27:15.446 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.439+0000 7f2cf7fff700 1 --2- 172.21.15.101:0/222580854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbeb7caad0 0x55dbeb826c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:15.446 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.439+0000 7f2cf7fff700 1 -- 172.21.15.101:0/222580854 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dbeb7bfbe0 msgr2=0x55dbeb8222b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:27:15.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.439+0000 7f2cf7fff700 1 --2- 172.21.15.101:0/222580854 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dbeb7bfbe0 0x55dbeb8222b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:15.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.439+0000 7f2cf7fff700 1 -- 172.21.15.101:0/222580854 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cec0044a0 con 0x55dbeb7bf2d0 2023-07-18T03:27:15.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.440+0000 7f2cf7fff700 1 --2- 172.21.15.101:0/222580854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbeb7bf2d0 0x55dbeb821d70 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8022fd0 tx=0x7f2ce80cbd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fab5c931fe8491c1 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:27:15.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.440+0000 7f2cd77fe700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:27:15.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.440+0000 7f2cd77fe700 1 -- 172.21.15.101:0/222580854 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce809b1a0 con 0x55dbeb7bf2d0 2023-07-18T03:27:15.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.440+0000 7f2d123ddb80 1 -- 172.21.15.101:0/222580854 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2ce809ace0 con 0x55dbeb7bf2d0 2023-07-18T03:27:15.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.440+0000 7f2cd77fe700 1 -- 172.21.15.101:0/222580854 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f2ce809f1d0 con 0x55dbeb7bf2d0 2023-07-18T03:27:15.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.440+0000 7f2cd77fe700 1 -- 172.21.15.101:0/222580854 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ce80c7ea0 con 0x55dbeb7bf2d0 2023-07-18T03:27:15.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.440+0000 7f2d123ddb80 10 client.29018 fetch_fsmap learned FSMap version 47 2023-07-18T03:27:15.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.440+0000 7f2d123ddb80 1 -- 172.21.15.101:0/222580854 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x7f2cec0098e0 con 0x55dbeb7bf2d0 2023-07-18T03:27:15.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.440+0000 7f2cd77fe700 1 -- 172.21.15.101:0/222580854 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ce809ace0 con 0x55dbeb7bf2d0 2023-07-18T03:27:15.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.440+0000 7f2cd77fe700 1 -- 172.21.15.101:0/222580854 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 47) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f2ce80aa110 con 0x55dbeb7bf2d0 2023-07-18T03:27:15.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.441+0000 7f2d123ddb80 10 client.29018 fetch_fsmap finished waiting for FSMap version 47 2023-07-18T03:27:15.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.441+0000 7f2d123ddb80 10 client.29018 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:27:15.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.441+0000 7f2d123ddb80 20 client.29018 populate_metadata read hostname 'smithi101' 2023-07-18T03:27:15.450 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.441+0000 7f2d123ddb80 1 --2- 172.21.15.101:0/222580854 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55dbeb82e7f0 0x55dbeb830c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:15.450 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.441+0000 7f2d123ddb80 4 client.29018 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:27:15.450 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.441+0000 7f2d123ddb80 1 -- 172.21.15.101:0/222580854 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55dbeb6cd050 con 0x55dbeb82e7f0 2023-07-18T03:27:15.450 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.441+0000 7f2cf77fe700 1 --2- 172.21.15.101:0/222580854 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55dbeb82e7f0 0x55dbeb830c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:15.450 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.441+0000 7f2cf77fe700 1 --2- 172.21.15.101:0/222580854 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55dbeb82e7f0 0x55dbeb830c00 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4348ebd7c42987 server_cookie=12811f5d6c603e29 in_seq=0 out_seq=0 2023-07-18T03:27:15.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.441+0000 7f2cd77fe700 10 client.29018 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:27:15.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.441+0000 7f2cd77fe700 1 -- 172.21.15.101:0/222580854 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55dbeb6cd050 con 0x55dbeb82e7f0 2023-07-18T03:27:15.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.441+0000 7f2cd77fe700 10 client.29018 handle_command_reply: tid=0 2023-07-18T03:27:15.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.441+0000 7f2d123ddb80 1 client.29018 shutdown 2023-07-18T03:27:15.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.442+0000 7f2d123ddb80 1 -- 172.21.15.101:0/222580854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbeb7bf2d0 msgr2=0x55dbeb821d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:27:15.452 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.442+0000 7f2d123ddb80 1 --2- 172.21.15.101:0/222580854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbeb7bf2d0 0x55dbeb821d70 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8022fd0 tx=0x7f2ce80cbd30 comp rx=0 tx=0).stop 2023-07-18T03:27:15.452 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.442+0000 7f2d123ddb80 1 -- 172.21.15.101:0/222580854 shutdown_connections 2023-07-18T03:27:15.452 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.442+0000 7f2d123ddb80 1 --2- 172.21.15.101:0/222580854 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55dbeb82e7f0 0x55dbeb830c00 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:15.452 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.442+0000 7f2d123ddb80 1 --2- 172.21.15.101:0/222580854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbeb7bf2d0 0x55dbeb821d70 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:15.452 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.442+0000 7f2d123ddb80 1 --2- 172.21.15.101:0/222580854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dbeb7caad0 0x55dbeb826c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:15.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.442+0000 7f2cd77fe700 0 client.29018 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:27:15.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.442+0000 7f2d123ddb80 1 --2- 172.21.15.101:0/222580854 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dbeb7bfbe0 0x55dbeb8222b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:15.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.442+0000 7f2d123ddb80 1 -- 172.21.15.101:0/222580854 >> 172.21.15.101:0/222580854 conn(0x55dbeb7b5da0 msgr2=0x55dbeb7c9330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:27:15.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.442+0000 7f2d123ddb80 1 -- 172.21.15.101:0/222580854 shutdown_connections 2023-07-18T03:27:15.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.442+0000 7f2d123ddb80 1 -- 172.21.15.101:0/222580854 wait complete. 2023-07-18T03:27:15.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.442+0000 7f2d123ddb80 20 client.29018 trim_cache size 0 max 16384 2023-07-18T03:27:15.454 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:27:15.454 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:27:15.455 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:27:15.455 INFO:teuthology.orchestra.run.smithi101.stdout: "0f2f19e0-0429-465c-935e-223eb09490c3": { 2023-07-18T03:27:15.455 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:27:15.455 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "0f2f19e0-0429-465c-935e-223eb09490c3", 2023-07-18T03:27:15.455 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:27:15.456 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:27:15.456 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:27:15.456 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:27:15.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.442+0000 7f2cfe9ff700 1 -- 172.21.15.101:0/1964732033 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2ce00a1340 msgr2=0x7f2ce00a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:27:15.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.442+0000 7f2cfe9ff700 1 --2- 172.21.15.101:0/1964732033 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2ce00a1340 0x7f2ce00a37f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0009550 tx=0x7f2cf000b040 comp rx=0 tx=0).stop 2023-07-18T03:27:15.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.443+0000 7f2cfe9ff700 1 -- 172.21.15.101:0/1964732033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cf80728b0 msgr2=0x7f2cf819c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:27:15.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.443+0000 7f2cfe9ff700 1 --2- 172.21.15.101:0/1964732033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cf80728b0 0x7f2cf819c2b0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8000c00 tx=0x7f2ce8005720 comp rx=0 tx=0).stop 2023-07-18T03:27:15.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.443+0000 7f2cfe9ff700 1 -- 172.21.15.101:0/1964732033 shutdown_connections 2023-07-18T03:27:15.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.443+0000 7f2cfe9ff700 1 --2- 172.21.15.101:0/1964732033 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f2ce00a1340 0x7f2ce00a37f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:15.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.443+0000 7f2cfe9ff700 1 --2- 172.21.15.101:0/1964732033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cf80728b0 0x7f2cf819c2b0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:15.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.443+0000 7f2cfe9ff700 1 --2- 172.21.15.101:0/1964732033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cf8110ae0 0x7f2cf81a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:15.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.443+0000 7f2cfe9ff700 1 --2- 172.21.15.101:0/1964732033 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2cf80731c0 0x7f2cf819c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:15.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.443+0000 7f2cfe9ff700 1 -- 172.21.15.101:0/1964732033 >> 172.21.15.101:0/1964732033 conn(0x7f2cf80fb5f0 msgr2=0x7f2cf80fc250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:27:15.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.443+0000 7f2cfe9ff700 1 -- 172.21.15.101:0/1964732033 shutdown_connections 2023-07-18T03:27:15.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:15.443+0000 7f2cfe9ff700 1 -- 172.21.15.101:0/1964732033 wait complete. 2023-07-18T03:27:15.460 INFO:tasks.cephfs.filesystem:scrub status for tag:0f2f19e0-0429-465c-935e-223eb09490c3 - {'path': '/', 'tag': '0f2f19e0-0429-465c-935e-223eb09490c3', 'options': 'recursive,force'} 2023-07-18T03:27:15.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:27:15 smithi155 ceph-mon[128885]: pgmap v620: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 52 KiB/s wr, 22 op/s 2023-07-18T03:27:15.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:27:15 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:27:15.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:27:15 smithi179 ceph-mon[129229]: pgmap v620: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 52 KiB/s wr, 22 op/s 2023-07-18T03:27:15.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:27:15 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:27:15.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:27:15 smithi101 ceph-mon[126379]: pgmap v620: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 52 KiB/s wr, 22 op/s 2023-07-18T03:27:15.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:27:15 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:27:16.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:27:16 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:27:16.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:27:16 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:27:16.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:27:16 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:27:16.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:27:16 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:27:16.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:27:16 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:27:16.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:27:16 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:27:16.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:27:16 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:27:16.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:27:16 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:27:16.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:27:16 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:27:17.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:27:17 smithi155 ceph-mon[128885]: pgmap v621: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 7 op/s 2023-07-18T03:27:17.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:27:17 smithi179 ceph-mon[129229]: pgmap v621: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 7 op/s 2023-07-18T03:27:17.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:27:17 smithi101 ceph-mon[126379]: pgmap v621: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 7 op/s 2023-07-18T03:27:19.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:27:19 smithi155 ceph-mon[128885]: pgmap v622: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 7 op/s 2023-07-18T03:27:19.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:27:19 smithi155 ceph-mon[128885]: osdmap e120: 12 total, 12 up, 12 in 2023-07-18T03:27:19.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:27:19 smithi179 ceph-mon[129229]: pgmap v622: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 7 op/s 2023-07-18T03:27:19.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:27:19 smithi179 ceph-mon[129229]: osdmap e120: 12 total, 12 up, 12 in 2023-07-18T03:27:19.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:27:19 smithi101 ceph-mon[126379]: pgmap v622: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 7 op/s 2023-07-18T03:27:19.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:27:19 smithi101 ceph-mon[126379]: osdmap e120: 12 total, 12 up, 12 in 2023-07-18T03:27:21.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:27:21 smithi155 ceph-mon[128885]: pgmap v624: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.3 KiB/s wr, 6 op/s 2023-07-18T03:27:21.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:27:21 smithi179 ceph-mon[129229]: pgmap v624: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.3 KiB/s wr, 6 op/s 2023-07-18T03:27:21.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:27:21 smithi101 ceph-mon[126379]: pgmap v624: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.3 KiB/s wr, 6 op/s 2023-07-18T03:27:23.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:27:23 smithi155 ceph-mon[128885]: pgmap v625: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.1 KiB/s wr, 6 op/s 2023-07-18T03:27:23.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:27:23 smithi179 ceph-mon[129229]: pgmap v625: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.1 KiB/s wr, 6 op/s 2023-07-18T03:27:23.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:27:23 smithi101 ceph-mon[126379]: pgmap v625: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.1 KiB/s wr, 6 op/s 2023-07-18T03:27:25.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:27:25 smithi155 ceph-mon[128885]: pgmap v626: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.1 KiB/s wr, 6 op/s 2023-07-18T03:27:25.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:27:25 smithi179 ceph-mon[129229]: pgmap v626: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.1 KiB/s wr, 6 op/s 2023-07-18T03:27:25.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:27:25 smithi101 ceph-mon[126379]: pgmap v626: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.1 KiB/s wr, 6 op/s 2023-07-18T03:27:27.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:27:27 smithi155 ceph-mon[128885]: pgmap v627: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.5 KiB/s wr, 6 op/s 2023-07-18T03:27:27.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:27:27 smithi179 ceph-mon[129229]: pgmap v627: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.5 KiB/s wr, 6 op/s 2023-07-18T03:27:27.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:27:27 smithi101 ceph-mon[126379]: pgmap v627: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.5 KiB/s wr, 6 op/s 2023-07-18T03:27:29.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:27:29 smithi155 ceph-mon[128885]: pgmap v628: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.5 KiB/s wr, 6 op/s 2023-07-18T03:27:29.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:27:29 smithi179 ceph-mon[129229]: pgmap v628: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.5 KiB/s wr, 6 op/s 2023-07-18T03:27:29.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:27:29 smithi101 ceph-mon[126379]: pgmap v628: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.5 KiB/s wr, 6 op/s 2023-07-18T03:27:31.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:27:31 smithi155 ceph-mon[128885]: pgmap v629: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 8 op/s 2023-07-18T03:27:31.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:27:31 smithi179 ceph-mon[129229]: pgmap v629: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 8 op/s 2023-07-18T03:27:31.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:27:31 smithi101 ceph-mon[126379]: pgmap v629: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 8 op/s 2023-07-18T03:27:33.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:27:33 smithi155 ceph-mon[128885]: pgmap v630: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.7 KiB/s wr, 6 op/s 2023-07-18T03:27:33.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:27:33 smithi179 ceph-mon[129229]: pgmap v630: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.7 KiB/s wr, 6 op/s 2023-07-18T03:27:33.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:27:33 smithi101 ceph-mon[126379]: pgmap v630: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.7 KiB/s wr, 6 op/s 2023-07-18T03:27:35.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:27:35 smithi155 ceph-mon[128885]: pgmap v631: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.7 KiB/s wr, 5 op/s 2023-07-18T03:27:35.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:27:35 smithi179 ceph-mon[129229]: pgmap v631: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.7 KiB/s wr, 5 op/s 2023-07-18T03:27:35.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:27:35 smithi101 ceph-mon[126379]: pgmap v631: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.7 KiB/s wr, 5 op/s 2023-07-18T03:27:37.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:27:37 smithi155 ceph-mon[128885]: pgmap v632: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.7 KiB/s wr, 7 op/s 2023-07-18T03:27:37.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:27:37 smithi179 ceph-mon[129229]: pgmap v632: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.7 KiB/s wr, 7 op/s 2023-07-18T03:27:37.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:27:37 smithi101 ceph-mon[126379]: pgmap v632: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.7 KiB/s wr, 7 op/s 2023-07-18T03:27:39.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:27:39 smithi155 ceph-mon[128885]: pgmap v633: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.6 KiB/s wr, 5 op/s 2023-07-18T03:27:39.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:27:39 smithi179 ceph-mon[129229]: pgmap v633: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.6 KiB/s wr, 5 op/s 2023-07-18T03:27:39.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:27:39 smithi101 ceph-mon[126379]: pgmap v633: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.6 KiB/s wr, 5 op/s 2023-07-18T03:27:41.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:27:41 smithi155 ceph-mon[128885]: pgmap v634: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.6 KiB/s wr, 6 op/s 2023-07-18T03:27:41.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:27:41 smithi179 ceph-mon[129229]: pgmap v634: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.6 KiB/s wr, 6 op/s 2023-07-18T03:27:41.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:27:41 smithi101 ceph-mon[126379]: pgmap v634: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.6 KiB/s wr, 6 op/s 2023-07-18T03:27:43.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:27:43 smithi155 ceph-mon[128885]: pgmap v635: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:27:43.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:27:43 smithi179 ceph-mon[129229]: pgmap v635: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:27:43.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:27:43 smithi101 ceph-mon[126379]: pgmap v635: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:27:45.461 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:27:45.549 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:27:45 smithi101 ceph-mon[126379]: pgmap v636: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:27:45.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:27:45 smithi155 ceph-mon[128885]: pgmap v636: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:27:45.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.692+0000 7fc6879ff700 1 -- 172.21.15.101:0/2592248004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6800ffae0 msgr2=0x7fc6800fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:27:45.690 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.692+0000 7fc6879ff700 1 --2- 172.21.15.101:0/2592248004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6800ffae0 0x7fc6800fff50 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc670009990 tx=0x7fc670009ca0 comp rx=0 tx=0).stop 2023-07-18T03:27:45.690 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.692+0000 7fc6879ff700 1 -- 172.21.15.101:0/2592248004 shutdown_connections 2023-07-18T03:27:45.690 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.692+0000 7fc6879ff700 1 --2- 172.21.15.101:0/2592248004 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc680100490 0x7fc68010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:45.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.692+0000 7fc6879ff700 1 --2- 172.21.15.101:0/2592248004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc680105af0 0x7fc680105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:45.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.692+0000 7fc6879ff700 1 --2- 172.21.15.101:0/2592248004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6800ffae0 0x7fc6800fff50 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:45.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.692+0000 7fc6879ff700 1 -- 172.21.15.101:0/2592248004 >> 172.21.15.101:0/2592248004 conn(0x7fc6800fb5e0 msgr2=0x7fc6800fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:27:45.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.692+0000 7fc6879ff700 1 -- 172.21.15.101:0/2592248004 shutdown_connections 2023-07-18T03:27:45.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.692+0000 7fc6879ff700 1 -- 172.21.15.101:0/2592248004 wait complete. 2023-07-18T03:27:45.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.693+0000 7fc6879ff700 1 Processor -- start 2023-07-18T03:27:45.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.693+0000 7fc6879ff700 1 -- start start 2023-07-18T03:27:45.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.693+0000 7fc6879ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6800ffae0 0x7fc680197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:45.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.693+0000 7fc6879ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc680100490 0x7fc680198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:45.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.693+0000 7fc6879ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc680105af0 0x7fc68019c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:45.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.693+0000 7fc6879ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68019ce10 con 0x7fc680100490 2023-07-18T03:27:45.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.693+0000 7fc6879ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68019cf80 con 0x7fc6800ffae0 2023-07-18T03:27:45.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.693+0000 7fc6879ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68019d140 con 0x7fc680105af0 2023-07-18T03:27:45.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.694+0000 7fc68579b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6800ffae0 0x7fc680197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:45.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.694+0000 7fc684f9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc680100490 0x7fc680198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:45.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.694+0000 7fc68579b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6800ffae0 0x7fc680197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:45596/0 (socket says 172.21.15.101:45596) 2023-07-18T03:27:45.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.694+0000 7fc68579b700 1 -- 172.21.15.101:0/2833708761 learned_addr learned my addr 172.21.15.101:0/2833708761 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:27:45.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.694+0000 7fc685f9c700 1 --2- 172.21.15.101:0/2833708761 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc680105af0 0x7fc68019c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:45.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.694+0000 7fc68579b700 1 -- 172.21.15.101:0/2833708761 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc680105af0 msgr2=0x7fc68019c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:27:45.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.694+0000 7fc68579b700 1 --2- 172.21.15.101:0/2833708761 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc680105af0 0x7fc68019c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:45.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.694+0000 7fc68579b700 1 -- 172.21.15.101:0/2833708761 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc680100490 msgr2=0x7fc680198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:27:45.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.694+0000 7fc68579b700 1 --2- 172.21.15.101:0/2833708761 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc680100490 0x7fc680198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:45.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.694+0000 7fc68579b700 1 -- 172.21.15.101:0/2833708761 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6700095f0 con 0x7fc6800ffae0 2023-07-18T03:27:45.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.694+0000 7fc684f9a700 1 --2- 172.21.15.101:0/2833708761 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc680100490 0x7fc680198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:27:45.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.694+0000 7fc68579b700 1 --2- 172.21.15.101:0/2833708761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6800ffae0 0x7fc680197e30 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc67c00e9a0 tx=0x7fc67c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:27:45.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.694+0000 7fc6767fc700 1 -- 172.21.15.101:0/2833708761 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67c009790 con 0x7fc6800ffae0 2023-07-18T03:27:45.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.695+0000 7fc6879ff700 1 -- 172.21.15.101:0/2833708761 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc68019d420 con 0x7fc6800ffae0 2023-07-18T03:27:45.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.695+0000 7fc6767fc700 1 -- 172.21.15.101:0/2833708761 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc67c010430 con 0x7fc6800ffae0 2023-07-18T03:27:45.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.695+0000 7fc6767fc700 1 -- 172.21.15.101:0/2833708761 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67c018680 con 0x7fc6800ffae0 2023-07-18T03:27:45.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.695+0000 7fc6879ff700 1 -- 172.21.15.101:0/2833708761 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6801a4c60 con 0x7fc6800ffae0 2023-07-18T03:27:45.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.698+0000 7fc6767fc700 1 -- 172.21.15.101:0/2833708761 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fc67c0105a0 con 0x7fc6800ffae0 2023-07-18T03:27:45.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.699+0000 7fc69b276b80 1 Processor -- start 2023-07-18T03:27:45.704 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.699+0000 7fc69b276b80 1 -- start start 2023-07-18T03:27:45.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.699+0000 7fc6767fc700 1 --2- 172.21.15.101:0/2833708761 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc66c0a12c0 0x7fc66c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:45.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.699+0000 7fc69b276b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574d27c32d0 0x5574d27c36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:45.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.699+0000 7fc69b276b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574d27c3be0 0x5574d27ce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:45.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.699+0000 7fc69b276b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574d27cead0 0x5574d27d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:45.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.699+0000 7fc69b276b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574d26c9cb0 con 0x5574d27cead0 2023-07-18T03:27:45.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.699+0000 7fc69b276b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574d271ca40 con 0x5574d27c32d0 2023-07-18T03:27:45.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.699+0000 7fc69b276b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574d2715ed0 con 0x5574d27c3be0 2023-07-18T03:27:45.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.699+0000 7fc6767fc700 1 -- 172.21.15.101:0/2833708761 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fc67c014070 con 0x7fc6800ffae0 2023-07-18T03:27:45.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.699+0000 7fc68579b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574d27c32d0 0x5574d27c36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:45.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.699+0000 7fc684f9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574d27c3be0 0x5574d27ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:45.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.700+0000 7fc68579b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574d27c32d0 0x5574d27c36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:45612/0 (socket says 172.21.15.101:45612) 2023-07-18T03:27:45.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.700+0000 7fc68579b700 1 -- 172.21.15.101:0/1468521253 learned_addr learned my addr 172.21.15.101:0/1468521253 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:27:45.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.700+0000 7fc685f9c700 1 --2- 172.21.15.101:0/2833708761 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc66c0a12c0 0x7fc66c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:45.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.700+0000 7fc685f9c700 1 --2- 172.21.15.101:0/1468521253 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574d27cead0 0x5574d27d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:45.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.700+0000 7fc68579b700 1 -- 172.21.15.101:0/1468521253 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574d27c3be0 msgr2=0x5574d27ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:27:45.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.700+0000 7fc68579b700 1 --2- 172.21.15.101:0/1468521253 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574d27c3be0 0x5574d27ce590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:45.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.700+0000 7fc68579b700 1 -- 172.21.15.101:0/1468521253 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574d27cead0 msgr2=0x5574d27d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:27:45.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.700+0000 7fc68579b700 1 --2- 172.21.15.101:0/1468521253 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574d27cead0 0x5574d27d0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:45.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.700+0000 7fc68579b700 1 -- 172.21.15.101:0/1468521253 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574d2543de0 con 0x5574d27c32d0 2023-07-18T03:27:45.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.700+0000 7fc685f9c700 1 --2- 172.21.15.101:0/1468521253 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574d27cead0 0x5574d27d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:27:45.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.700+0000 7fc68579b700 1 --2- 172.21.15.101:0/1468521253 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574d27c32d0 0x5574d27c36a0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fc67c096990 tx=0x7fc67c099880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc50e93f96c29efb server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:27:45.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.700+0000 7fc66b7fe700 1 -- 172.21.15.101:0/1468521253 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67c0b2900 con 0x5574d27c32d0 2023-07-18T03:27:45.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.700+0000 7fc685f9c700 1 --2- 172.21.15.101:0/2833708761 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc66c0a12c0 0x7fc66c0a3770 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc678009ec0 tx=0x7fc678009920 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:27:45.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.700+0000 7fc66b7fe700 1 -- 172.21.15.101:0/1468521253 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc67c099f20 con 0x5574d27c32d0 2023-07-18T03:27:45.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.700+0000 7fc66b7fe700 1 -- 172.21.15.101:0/1468521253 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67c0addb0 con 0x5574d27c32d0 2023-07-18T03:27:45.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.701+0000 7fc69b276b80 1 -- 172.21.15.101:0/1468521253 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574d27c32d0 msgr2=0x5574d27c36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:27:45.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.701+0000 7fc69b276b80 1 --2- 172.21.15.101:0/1468521253 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574d27c32d0 0x5574d27c36a0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fc67c096990 tx=0x7fc67c099880 comp rx=0 tx=0).stop 2023-07-18T03:27:45.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.701+0000 7fc69b276b80 1 -- 172.21.15.101:0/1468521253 shutdown_connections 2023-07-18T03:27:45.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.701+0000 7fc69b276b80 1 --2- 172.21.15.101:0/1468521253 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574d27c32d0 0x5574d27c36a0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:45.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.701+0000 7fc69b276b80 1 --2- 172.21.15.101:0/1468521253 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574d27cead0 0x5574d27d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:45.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.701+0000 7fc69b276b80 1 --2- 172.21.15.101:0/1468521253 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574d27c3be0 0x5574d27ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:45.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.701+0000 7fc69b276b80 1 -- 172.21.15.101:0/1468521253 >> 172.21.15.101:0/1468521253 conn(0x5574d27b9da0 msgr2=0x5574d27bf6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:27:45.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.701+0000 7fc69b276b80 1 -- 172.21.15.101:0/1468521253 shutdown_connections 2023-07-18T03:27:45.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.701+0000 7fc69b276b80 1 -- 172.21.15.101:0/1468521253 wait complete. 2023-07-18T03:27:45.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.702+0000 7fc69b276b80 1 -- start start 2023-07-18T03:27:45.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.702+0000 7fc69b276b80 1 Processor -- start 2023-07-18T03:27:45.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.702+0000 7fc69b276b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574d27c32d0 0x5574d27ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:45.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.702+0000 7fc69b276b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574d27c3be0 0x5574d27cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:45.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.702+0000 7fc69b276b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574d27cead0 0x5574d27cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:45.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.702+0000 7fc69b276b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574d26b5ac0 con 0x5574d27c3be0 2023-07-18T03:27:45.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.702+0000 7fc69b276b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574d26b57c0 con 0x5574d27cead0 2023-07-18T03:27:45.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.702+0000 7fc69b276b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574d2599920 con 0x5574d27c32d0 2023-07-18T03:27:45.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.703+0000 7fc69b276b80 10 client.0 _ll_register_callbacks cb 0x5574d26b3760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:27:45.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.703+0000 7fc68579b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574d27c32d0 0x5574d27ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:45.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.703+0000 7fc685f9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574d27cead0 0x5574d27cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:45.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.703+0000 7fc68579b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574d27c32d0 0x5574d27ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:44210/0 (socket says 172.21.15.101:44210) 2023-07-18T03:27:45.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.703+0000 7fc68579b700 1 -- 172.21.15.101:0/3399736988 learned_addr learned my addr 172.21.15.101:0/3399736988 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:27:45.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.703+0000 7fc684f9a700 1 --2- 172.21.15.101:0/3399736988 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574d27c3be0 0x5574d27cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:45.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.703+0000 7fc68579b700 1 -- 172.21.15.101:0/3399736988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574d27cead0 msgr2=0x5574d27cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:27:45.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.703+0000 7fc68579b700 1 --2- 172.21.15.101:0/3399736988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574d27cead0 0x5574d27cbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:45.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.703+0000 7fc68579b700 1 -- 172.21.15.101:0/3399736988 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574d27c3be0 msgr2=0x5574d27cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:27:45.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.703+0000 7fc68579b700 1 --2- 172.21.15.101:0/3399736988 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574d27c3be0 0x5574d27cb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:45.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.703+0000 7fc68579b700 1 -- 172.21.15.101:0/3399736988 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6801a4c60 con 0x5574d27c32d0 2023-07-18T03:27:45.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.703+0000 7fc684f9a700 1 --2- 172.21.15.101:0/3399736988 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574d27c3be0 0x5574d27cb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:27:45.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.703+0000 7fc68579b700 1 --2- 172.21.15.101:0/3399736988 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574d27c32d0 0x5574d27ce510 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fc67c00c2d0 tx=0x7fc67c09e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9b67f4511e4eb39 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:27:45.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.703+0000 7fc668ff9700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:27:45.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.703+0000 7fc668ff9700 1 -- 172.21.15.101:0/3399736988 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67c0b2980 con 0x5574d27c32d0 2023-07-18T03:27:45.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.703+0000 7fc668ff9700 1 -- 172.21.15.101:0/3399736988 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fc67c09a180 con 0x5574d27c32d0 2023-07-18T03:27:45.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.703+0000 7fc69b276b80 1 -- 172.21.15.101:0/3399736988 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5574d266f4f0 con 0x5574d27c32d0 2023-07-18T03:27:45.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.704+0000 7fc668ff9700 1 -- 172.21.15.101:0/3399736988 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc67c0caa30 con 0x5574d27c32d0 2023-07-18T03:27:45.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.704+0000 7fc69b276b80 10 client.29020 fetch_fsmap learned FSMap version 47 2023-07-18T03:27:45.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.704+0000 7fc69b276b80 1 -- 172.21.15.101:0/3399736988 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x5574d252cc60 con 0x5574d27c32d0 2023-07-18T03:27:45.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.704+0000 7fc668ff9700 1 -- 172.21.15.101:0/3399736988 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc67c0ca020 con 0x5574d27c32d0 2023-07-18T03:27:45.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.704+0000 7fc668ff9700 1 -- 172.21.15.101:0/3399736988 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 47) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fc67c0b3e80 con 0x5574d27c32d0 2023-07-18T03:27:45.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.704+0000 7fc69b276b80 10 client.29020 fetch_fsmap finished waiting for FSMap version 47 2023-07-18T03:27:45.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.704+0000 7fc69b276b80 10 client.29020 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:27:45.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.704+0000 7fc69b276b80 20 client.29020 populate_metadata read hostname 'smithi101' 2023-07-18T03:27:45.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.704+0000 7fc69b276b80 1 --2- 172.21.15.101:0/3399736988 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5574d27dde40 0x5574d27e0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:45.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.704+0000 7fc69b276b80 4 client.29020 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:27:45.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.704+0000 7fc69b276b80 1 -- 172.21.15.101:0/3399736988 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5574d270eeb0 con 0x5574d27dde40 2023-07-18T03:27:45.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.704+0000 7fc684f9a700 1 --2- 172.21.15.101:0/3399736988 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5574d27dde40 0x5574d27e0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:45.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.705+0000 7fc668ff9700 10 client.29020 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:27:45.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.705+0000 7fc684f9a700 1 --2- 172.21.15.101:0/3399736988 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5574d27dde40 0x5574d27e0220 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7e37bdcb02700c3 server_cookie=2ab423cb0663e8c8 in_seq=0 out_seq=0 2023-07-18T03:27:45.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.706+0000 7fc668ff9700 1 -- 172.21.15.101:0/3399736988 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5574d270eeb0 con 0x5574d27dde40 2023-07-18T03:27:45.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.706+0000 7fc668ff9700 10 client.29020 handle_command_reply: tid=0 2023-07-18T03:27:45.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.706+0000 7fc69b276b80 1 client.29020 shutdown 2023-07-18T03:27:45.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.706+0000 7fc69b276b80 1 -- 172.21.15.101:0/3399736988 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574d27c32d0 msgr2=0x5574d27ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:27:45.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.706+0000 7fc69b276b80 1 --2- 172.21.15.101:0/3399736988 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574d27c32d0 0x5574d27ce510 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fc67c00c2d0 tx=0x7fc67c09e040 comp rx=0 tx=0).stop 2023-07-18T03:27:45.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.706+0000 7fc69b276b80 1 -- 172.21.15.101:0/3399736988 shutdown_connections 2023-07-18T03:27:45.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.706+0000 7fc69b276b80 1 --2- 172.21.15.101:0/3399736988 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5574d27dde40 0x5574d27e0220 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:45.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.706+0000 7fc69b276b80 1 --2- 172.21.15.101:0/3399736988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574d27cead0 0x5574d27cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:45.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.706+0000 7fc69b276b80 1 --2- 172.21.15.101:0/3399736988 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574d27c32d0 0x5574d27ce510 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:45.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.706+0000 7fc69b276b80 1 --2- 172.21.15.101:0/3399736988 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574d27c3be0 0x5574d27cb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:45.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.706+0000 7fc668ff9700 0 client.29020 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:27:45.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.706+0000 7fc69b276b80 1 -- 172.21.15.101:0/3399736988 >> 172.21.15.101:0/3399736988 conn(0x5574d27b9da0 msgr2=0x5574d27bf230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:27:45.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.706+0000 7fc69b276b80 1 -- 172.21.15.101:0/3399736988 shutdown_connections 2023-07-18T03:27:45.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.706+0000 7fc69b276b80 1 -- 172.21.15.101:0/3399736988 wait complete. 2023-07-18T03:27:45.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.706+0000 7fc69b276b80 20 client.29020 trim_cache size 0 max 16384 2023-07-18T03:27:45.728 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:27:45 smithi179 ceph-mon[129229]: pgmap v636: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:27:45.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.723+0000 7fc69b276b80 1 Processor -- start 2023-07-18T03:27:45.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.723+0000 7fc69b276b80 1 -- start start 2023-07-18T03:27:45.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.723+0000 7fc69b276b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574d27c32d0 0x5574d27d54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:45.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.723+0000 7fc69b276b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574d27c3be0 0x5574d27c51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:45.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.723+0000 7fc69b276b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574d27cead0 0x5574d27c6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:45.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.723+0000 7fc69b276b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574d27c6680 con 0x5574d27cead0 2023-07-18T03:27:45.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.723+0000 7fc69b276b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574d27b6fa0 con 0x5574d27c3be0 2023-07-18T03:27:45.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.723+0000 7fc69b276b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574d27b9970 con 0x5574d27c32d0 2023-07-18T03:27:45.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.723+0000 7fc684f9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574d27c3be0 0x5574d27c51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:45.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.723+0000 7fc68579b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574d27c32d0 0x5574d27d54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:45.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.724+0000 7fc684f9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574d27c3be0 0x5574d27c51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:45632/0 (socket says 172.21.15.101:45632) 2023-07-18T03:27:45.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.724+0000 7fc68579b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574d27c32d0 0x5574d27d54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:44212/0 (socket says 172.21.15.101:44212) 2023-07-18T03:27:45.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.724+0000 7fc684f9a700 1 -- 172.21.15.101:0/276698866 learned_addr learned my addr 172.21.15.101:0/276698866 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:27:45.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.724+0000 7fc685f9c700 1 --2- 172.21.15.101:0/276698866 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574d27cead0 0x5574d27c6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:45.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.724+0000 7fc684f9a700 1 -- 172.21.15.101:0/276698866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574d27c32d0 msgr2=0x5574d27d54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:27:45.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.724+0000 7fc684f9a700 1 --2- 172.21.15.101:0/276698866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574d27c32d0 0x5574d27d54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:45.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.724+0000 7fc684f9a700 1 -- 172.21.15.101:0/276698866 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574d27cead0 msgr2=0x5574d27c6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:27:45.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.724+0000 7fc684f9a700 1 --2- 172.21.15.101:0/276698866 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574d27cead0 0x5574d27c6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:45.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.724+0000 7fc684f9a700 1 -- 172.21.15.101:0/276698866 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574d256a250 con 0x5574d27c3be0 2023-07-18T03:27:45.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.724+0000 7fc68579b700 1 --2- 172.21.15.101:0/276698866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574d27c32d0 0x5574d27d54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:27:45.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.724+0000 7fc684f9a700 1 --2- 172.21.15.101:0/276698866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574d27c3be0 0x5574d27c51a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fc670009990 tx=0x7fc670009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb9e4a523bcc80fd server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:27:45.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.724+0000 7fc66bfff700 1 -- 172.21.15.101:0/276698866 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67001d070 con 0x5574d27c3be0 2023-07-18T03:27:45.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.724+0000 7fc66bfff700 1 -- 172.21.15.101:0/276698866 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc67001e040 con 0x5574d27c3be0 2023-07-18T03:27:45.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.724+0000 7fc66bfff700 1 -- 172.21.15.101:0/276698866 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc670004500 con 0x5574d27c3be0 2023-07-18T03:27:45.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.725+0000 7fc69b276b80 1 -- 172.21.15.101:0/276698866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574d27c3be0 msgr2=0x5574d27c51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:27:45.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.725+0000 7fc69b276b80 1 --2- 172.21.15.101:0/276698866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574d27c3be0 0x5574d27c51a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fc670009990 tx=0x7fc670009310 comp rx=0 tx=0).stop 2023-07-18T03:27:45.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.725+0000 7fc69b276b80 1 -- 172.21.15.101:0/276698866 shutdown_connections 2023-07-18T03:27:45.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.725+0000 7fc69b276b80 1 --2- 172.21.15.101:0/276698866 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574d27cead0 0x5574d27c6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:45.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.725+0000 7fc69b276b80 1 --2- 172.21.15.101:0/276698866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574d27c32d0 0x5574d27d54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:45.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.725+0000 7fc69b276b80 1 --2- 172.21.15.101:0/276698866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574d27c3be0 0x5574d27c51a0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:45.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.725+0000 7fc69b276b80 1 -- 172.21.15.101:0/276698866 >> 172.21.15.101:0/276698866 conn(0x5574d27b9da0 msgr2=0x5574d27da700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:27:45.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.725+0000 7fc69b276b80 1 -- 172.21.15.101:0/276698866 shutdown_connections 2023-07-18T03:27:45.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.725+0000 7fc69b276b80 1 -- 172.21.15.101:0/276698866 wait complete. 2023-07-18T03:27:45.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.725+0000 7fc69b276b80 1 -- start start 2023-07-18T03:27:45.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.726+0000 7fc69b276b80 1 Processor -- start 2023-07-18T03:27:45.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.726+0000 7fc69b276b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574d27c32d0 0x5574d27d5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:45.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.726+0000 7fc69b276b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574d27c3be0 0x5574d27d3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:45.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.726+0000 7fc69b276b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574d27cead0 0x5574d27d3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:45.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.726+0000 7fc69b276b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574d27d4490 con 0x5574d27cead0 2023-07-18T03:27:45.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.726+0000 7fc69b276b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574d27d4600 con 0x5574d27c32d0 2023-07-18T03:27:45.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.726+0000 7fc69b276b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574d27d47c0 con 0x5574d27c3be0 2023-07-18T03:27:45.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.726+0000 7fc69b276b80 10 client.0 _ll_register_callbacks cb 0x7fc6780097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:27:45.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.726+0000 7fc684f9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574d27c3be0 0x5574d27d3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:45.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.726+0000 7fc68579b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574d27c32d0 0x5574d27d5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:45.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.726+0000 7fc684f9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574d27c3be0 0x5574d27d3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:44214/0 (socket says 172.21.15.101:44214) 2023-07-18T03:27:45.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.726+0000 7fc685f9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574d27cead0 0x5574d27d3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:45.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.726+0000 7fc68579b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574d27c32d0 0x5574d27d5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:45642/0 (socket says 172.21.15.101:45642) 2023-07-18T03:27:45.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.726+0000 7fc684f9a700 1 -- 172.21.15.101:0/3519714665 learned_addr learned my addr 172.21.15.101:0/3519714665 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:27:45.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.727+0000 7fc684f9a700 1 -- 172.21.15.101:0/3519714665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574d27c32d0 msgr2=0x5574d27d5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:27:45.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.727+0000 7fc684f9a700 1 --2- 172.21.15.101:0/3519714665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574d27c32d0 0x5574d27d5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:45.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.727+0000 7fc684f9a700 1 -- 172.21.15.101:0/3519714665 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574d27cead0 msgr2=0x5574d27d3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:27:45.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.727+0000 7fc684f9a700 1 --2- 172.21.15.101:0/3519714665 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574d27cead0 0x5574d27d3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:45.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.727+0000 7fc684f9a700 1 -- 172.21.15.101:0/3519714665 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5574d252cc60 con 0x5574d27c3be0 2023-07-18T03:27:45.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.727+0000 7fc68579b700 1 --2- 172.21.15.101:0/3519714665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574d27c32d0 0x5574d27d5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:27:45.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.727+0000 7fc684f9a700 1 --2- 172.21.15.101:0/3519714665 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574d27c3be0 0x5574d27d3900 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc67000b2e0 tx=0x7fc6700098c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c1368efcc4fe64c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:27:45.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.727+0000 7fc668ff9700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:27:45.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.727+0000 7fc668ff9700 1 -- 172.21.15.101:0/3519714665 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67001d040 con 0x5574d27c3be0 2023-07-18T03:27:45.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.727+0000 7fc69b276b80 1 -- 172.21.15.101:0/3519714665 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc67c099f20 con 0x5574d27c3be0 2023-07-18T03:27:45.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.727+0000 7fc668ff9700 1 -- 172.21.15.101:0/3519714665 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fc670028160 con 0x5574d27c3be0 2023-07-18T03:27:45.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.727+0000 7fc668ff9700 1 -- 172.21.15.101:0/3519714665 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc670028b30 con 0x5574d27c3be0 2023-07-18T03:27:45.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.727+0000 7fc69b276b80 10 client.29026 fetch_fsmap learned FSMap version 47 2023-07-18T03:27:45.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.727+0000 7fc69b276b80 1 -- 172.21.15.101:0/3519714665 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x7fc68010d220 con 0x5574d27c3be0 2023-07-18T03:27:45.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.728+0000 7fc668ff9700 1 -- 172.21.15.101:0/3519714665 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc67c099f20 con 0x5574d27c3be0 2023-07-18T03:27:45.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.728+0000 7fc668ff9700 1 -- 172.21.15.101:0/3519714665 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 47) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fc670004bf0 con 0x5574d27c3be0 2023-07-18T03:27:45.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.728+0000 7fc69b276b80 10 client.29026 fetch_fsmap finished waiting for FSMap version 47 2023-07-18T03:27:45.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.728+0000 7fc69b276b80 10 client.29026 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:27:45.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.728+0000 7fc69b276b80 20 client.29026 populate_metadata read hostname 'smithi101' 2023-07-18T03:27:45.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.728+0000 7fc69b276b80 1 --2- 172.21.15.101:0/3519714665 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5574d2832fe0 0x5574d28353f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:45.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.728+0000 7fc69b276b80 4 client.29026 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:27:45.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.728+0000 7fc69b276b80 1 -- 172.21.15.101:0/3519714665 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5574d26d1050 con 0x5574d2832fe0 2023-07-18T03:27:45.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.728+0000 7fc68579b700 1 --2- 172.21.15.101:0/3519714665 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5574d2832fe0 0x5574d28353f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:45.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.729+0000 7fc668ff9700 10 client.29026 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:27:45.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.729+0000 7fc68579b700 1 --2- 172.21.15.101:0/3519714665 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5574d2832fe0 0x5574d28353f0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=329ac204d2055ba7 server_cookie=55dd4ac88f3d9621 in_seq=0 out_seq=0 2023-07-18T03:27:45.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.729+0000 7fc668ff9700 1 -- 172.21.15.101:0/3519714665 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5574d26d1050 con 0x5574d2832fe0 2023-07-18T03:27:45.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.729+0000 7fc668ff9700 10 client.29026 handle_command_reply: tid=0 2023-07-18T03:27:45.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.729+0000 7fc69b276b80 1 client.29026 shutdown 2023-07-18T03:27:45.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.729+0000 7fc69b276b80 1 -- 172.21.15.101:0/3519714665 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574d27c3be0 msgr2=0x5574d27d3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:27:45.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.729+0000 7fc69b276b80 1 --2- 172.21.15.101:0/3519714665 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574d27c3be0 0x5574d27d3900 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc67000b2e0 tx=0x7fc6700098c0 comp rx=0 tx=0).stop 2023-07-18T03:27:45.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.729+0000 7fc69b276b80 1 -- 172.21.15.101:0/3519714665 shutdown_connections 2023-07-18T03:27:45.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.729+0000 7fc69b276b80 1 --2- 172.21.15.101:0/3519714665 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5574d2832fe0 0x5574d28353f0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:45.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.729+0000 7fc69b276b80 1 --2- 172.21.15.101:0/3519714665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574d27c32d0 0x5574d27d5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:45.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.729+0000 7fc69b276b80 1 --2- 172.21.15.101:0/3519714665 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5574d27cead0 0x5574d27d3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:45.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.729+0000 7fc69b276b80 1 --2- 172.21.15.101:0/3519714665 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574d27c3be0 0x5574d27d3900 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:45.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.729+0000 7fc668ff9700 0 client.29026 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:27:45.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.729+0000 7fc69b276b80 1 -- 172.21.15.101:0/3519714665 >> 172.21.15.101:0/3519714665 conn(0x5574d27b9da0 msgr2=0x5574d27c51a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:27:45.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.729+0000 7fc69b276b80 1 -- 172.21.15.101:0/3519714665 shutdown_connections 2023-07-18T03:27:45.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.729+0000 7fc69b276b80 1 -- 172.21.15.101:0/3519714665 wait complete. 2023-07-18T03:27:45.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.729+0000 7fc69b276b80 20 client.29026 trim_cache size 0 max 16384 2023-07-18T03:27:45.751 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:27:45.751 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:27:45.752 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:27:45.752 INFO:teuthology.orchestra.run.smithi101.stdout: "0f2f19e0-0429-465c-935e-223eb09490c3": { 2023-07-18T03:27:45.752 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:27:45.752 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "0f2f19e0-0429-465c-935e-223eb09490c3", 2023-07-18T03:27:45.752 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:27:45.753 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:27:45.753 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:27:45.753 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:27:45.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.730+0000 7fc6879ff700 1 -- 172.21.15.101:0/2833708761 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc66c0a12c0 msgr2=0x7fc66c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:27:45.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.730+0000 7fc6879ff700 1 --2- 172.21.15.101:0/2833708761 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc66c0a12c0 0x7fc66c0a3770 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc678009ec0 tx=0x7fc678009920 comp rx=0 tx=0).stop 2023-07-18T03:27:45.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.730+0000 7fc6879ff700 1 -- 172.21.15.101:0/2833708761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6800ffae0 msgr2=0x7fc680197e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:27:45.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.730+0000 7fc6879ff700 1 --2- 172.21.15.101:0/2833708761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6800ffae0 0x7fc680197e30 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc67c00e9a0 tx=0x7fc67c00ecb0 comp rx=0 tx=0).stop 2023-07-18T03:27:45.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.730+0000 7fc6879ff700 1 -- 172.21.15.101:0/2833708761 shutdown_connections 2023-07-18T03:27:45.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.730+0000 7fc6879ff700 1 --2- 172.21.15.101:0/2833708761 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc66c0a12c0 0x7fc66c0a3770 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:45.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.730+0000 7fc6879ff700 1 --2- 172.21.15.101:0/2833708761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6800ffae0 0x7fc680197e30 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:45.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.730+0000 7fc6879ff700 1 --2- 172.21.15.101:0/2833708761 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc680105af0 0x7fc68019c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:45.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.730+0000 7fc6879ff700 1 --2- 172.21.15.101:0/2833708761 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc680100490 0x7fc680198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:45.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.730+0000 7fc6879ff700 1 -- 172.21.15.101:0/2833708761 >> 172.21.15.101:0/2833708761 conn(0x7fc6800fb5e0 msgr2=0x7fc68010aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:27:45.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.730+0000 7fc6879ff700 1 -- 172.21.15.101:0/2833708761 shutdown_connections 2023-07-18T03:27:45.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:27:45.730+0000 7fc6879ff700 1 -- 172.21.15.101:0/2833708761 wait complete. 2023-07-18T03:27:45.757 INFO:tasks.cephfs.filesystem:scrub status for tag:0f2f19e0-0429-465c-935e-223eb09490c3 - {'path': '/', 'tag': '0f2f19e0-0429-465c-935e-223eb09490c3', 'options': 'recursive,force'} 2023-07-18T03:27:47.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:27:47 smithi179 ceph-mon[129229]: pgmap v637: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:27:47.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:27:47 smithi101 ceph-mon[126379]: pgmap v637: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:27:47.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:27:47 smithi155 ceph-mon[128885]: pgmap v637: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:27:49.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:27:49 smithi179 ceph-mon[129229]: pgmap v638: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:27:49.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:27:49 smithi101 ceph-mon[126379]: pgmap v638: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:27:49.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:27:49 smithi155 ceph-mon[128885]: pgmap v638: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:27:51.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:27:51 smithi179 ceph-mon[129229]: pgmap v639: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-18T03:27:51.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:27:51 smithi101 ceph-mon[126379]: pgmap v639: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-18T03:27:51.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:27:51 smithi155 ceph-mon[128885]: pgmap v639: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-18T03:27:52.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:27:52 smithi179 ceph-mon[129229]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:27:52.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:27:52 smithi101 ceph-mon[126379]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:27:52.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:27:52 smithi155 ceph-mon[128885]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:27:53.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:27:53 smithi179 ceph-mon[129229]: pgmap v640: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:27:53.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:27:53 smithi101 ceph-mon[126379]: pgmap v640: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:27:53.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:27:53 smithi155 ceph-mon[128885]: pgmap v640: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:27:55.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:27:55 smithi179 ceph-mon[129229]: pgmap v641: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:27:55.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:27:55 smithi101 ceph-mon[126379]: pgmap v641: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:27:55.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:27:55 smithi155 ceph-mon[128885]: pgmap v641: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:27:57.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:27:57 smithi179 ceph-mon[129229]: pgmap v642: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:27:57.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:27:57 smithi101 ceph-mon[126379]: pgmap v642: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:27:57.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:27:57 smithi155 ceph-mon[128885]: pgmap v642: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:27:59.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:27:59 smithi179 ceph-mon[129229]: pgmap v643: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:27:59.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:27:59 smithi101 ceph-mon[126379]: pgmap v643: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:27:59.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:27:59 smithi155 ceph-mon[128885]: pgmap v643: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:28:01.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:01 smithi179 ceph-mon[129229]: pgmap v644: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-18T03:28:01.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:01 smithi101 ceph-mon[126379]: pgmap v644: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-18T03:28:01.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:01 smithi155 ceph-mon[128885]: pgmap v644: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-18T03:28:03.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:03 smithi179 ceph-mon[129229]: pgmap v645: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-07-18T03:28:03.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:03 smithi101 ceph-mon[126379]: pgmap v645: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-07-18T03:28:03.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:03 smithi155 ceph-mon[128885]: pgmap v645: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-07-18T03:28:05.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:05 smithi179 ceph-mon[129229]: pgmap v646: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-07-18T03:28:05.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:05 smithi101 ceph-mon[126379]: pgmap v646: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-07-18T03:28:05.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:05 smithi155 ceph-mon[128885]: pgmap v646: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-07-18T03:28:07.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:07 smithi179 ceph-mon[129229]: pgmap v647: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-07-18T03:28:07.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:07 smithi101 ceph-mon[126379]: pgmap v647: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-07-18T03:28:07.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:07 smithi155 ceph-mon[128885]: pgmap v647: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-07-18T03:28:09.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:09 smithi179 ceph-mon[129229]: pgmap v648: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-07-18T03:28:09.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:09 smithi101 ceph-mon[126379]: pgmap v648: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-07-18T03:28:09.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:09 smithi155 ceph-mon[128885]: pgmap v648: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-07-18T03:28:11.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:11 smithi179 ceph-mon[129229]: pgmap v649: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 21 op/s 2023-07-18T03:28:11.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:11 smithi179 ceph-mon[129229]: osdmap e121: 12 total, 12 up, 12 in 2023-07-18T03:28:11.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:11 smithi101 ceph-mon[126379]: pgmap v649: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 21 op/s 2023-07-18T03:28:11.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:11 smithi101 ceph-mon[126379]: osdmap e121: 12 total, 12 up, 12 in 2023-07-18T03:28:11.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:11 smithi155 ceph-mon[128885]: pgmap v649: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 21 op/s 2023-07-18T03:28:11.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:11 smithi155 ceph-mon[128885]: osdmap e121: 12 total, 12 up, 12 in 2023-07-18T03:28:13.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:13 smithi179 ceph-mon[129229]: pgmap v651: 129 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 114 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 27 KiB/s wr, 21 op/s 2023-07-18T03:28:13.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:13 smithi101 ceph-mon[126379]: pgmap v651: 129 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 114 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 27 KiB/s wr, 21 op/s 2023-07-18T03:28:13.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:13 smithi155 ceph-mon[128885]: pgmap v651: 129 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 114 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 27 KiB/s wr, 21 op/s 2023-07-18T03:28:15.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:15 smithi179 ceph-mon[129229]: pgmap v652: 129 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 114 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 27 KiB/s wr, 21 op/s 2023-07-18T03:28:15.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:15 smithi101 ceph-mon[126379]: pgmap v652: 129 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 114 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 27 KiB/s wr, 21 op/s 2023-07-18T03:28:15.759 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:28:15.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:15 smithi155 ceph-mon[128885]: pgmap v652: 129 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 114 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 27 KiB/s wr, 21 op/s 2023-07-18T03:28:15.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.980+0000 7f1bb97ff700 1 -- 172.21.15.101:0/3633362801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bb4105ae0 msgr2=0x7f1bb4105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:15.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.980+0000 7f1bb97ff700 1 --2- 172.21.15.101:0/3633362801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bb4105ae0 0x7f1bb4105eb0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4009990 tx=0x7f1ba4009ca0 comp rx=0 tx=0).stop 2023-07-18T03:28:15.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.980+0000 7f1bb97ff700 1 -- 172.21.15.101:0/3633362801 shutdown_connections 2023-07-18T03:28:15.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.980+0000 7f1bb97ff700 1 --2- 172.21.15.101:0/3633362801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bb4105ae0 0x7f1bb4105eb0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:15.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.980+0000 7f1bb97ff700 1 --2- 172.21.15.101:0/3633362801 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1bb4100480 0x7f1bb410c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:15.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.980+0000 7f1bb97ff700 1 --2- 172.21.15.101:0/3633362801 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1bb40ffad0 0x7f1bb40fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:15.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.980+0000 7f1bb97ff700 1 -- 172.21.15.101:0/3633362801 >> 172.21.15.101:0/3633362801 conn(0x7f1bb40fb5f0 msgr2=0x7f1bb40fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:28:15.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.980+0000 7f1bb97ff700 1 -- 172.21.15.101:0/3633362801 shutdown_connections 2023-07-18T03:28:15.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.981+0000 7f1bb97ff700 1 -- 172.21.15.101:0/3633362801 wait complete. 2023-07-18T03:28:15.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.981+0000 7f1bb97ff700 1 Processor -- start 2023-07-18T03:28:15.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.982+0000 7f1bb97ff700 1 -- start start 2023-07-18T03:28:15.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.982+0000 7f1bb97ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1bb40ffad0 0x7f1bb4072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:15.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.982+0000 7f1bb97ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1bb4100480 0x7f1bb4072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:15.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.982+0000 7f1bb97ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bb4105ae0 0x7f1bb4073290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:15.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.982+0000 7f1bb97ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb41a4060 con 0x7f1bb40ffad0 2023-07-18T03:28:15.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.982+0000 7f1bb97ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb41a41d0 con 0x7f1bb4105ae0 2023-07-18T03:28:15.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.982+0000 7f1bb97ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb41a4390 con 0x7f1bb4100480 2023-07-18T03:28:15.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.982+0000 7f1bb2ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1bb40ffad0 0x7f1bb4072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:15.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.982+0000 7f1bb2ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1bb40ffad0 0x7f1bb4072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:39816/0 (socket says 172.21.15.101:39816) 2023-07-18T03:28:15.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.982+0000 7f1bb2ffd700 1 -- 172.21.15.101:0/1217566467 learned_addr learned my addr 172.21.15.101:0/1217566467 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:28:15.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.982+0000 7f1bb27fc700 1 --2- 172.21.15.101:0/1217566467 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1bb4100480 0x7f1bb4072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:15.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.982+0000 7f1bb37fe700 1 --2- 172.21.15.101:0/1217566467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bb4105ae0 0x7f1bb4073290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:15.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.983+0000 7f1bb2ffd700 1 -- 172.21.15.101:0/1217566467 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1bb4100480 msgr2=0x7f1bb4072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:15.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.983+0000 7f1bb2ffd700 1 --2- 172.21.15.101:0/1217566467 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1bb4100480 0x7f1bb4072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:15.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.983+0000 7f1bb2ffd700 1 -- 172.21.15.101:0/1217566467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bb4105ae0 msgr2=0x7f1bb4073290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:15.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.983+0000 7f1bb2ffd700 1 --2- 172.21.15.101:0/1217566467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bb4105ae0 0x7f1bb4073290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:15.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.983+0000 7f1bb2ffd700 1 -- 172.21.15.101:0/1217566467 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba8009520 con 0x7f1bb40ffad0 2023-07-18T03:28:15.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.983+0000 7f1bb2ffd700 1 --2- 172.21.15.101:0/1217566467 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1bb40ffad0 0x7f1bb4072810 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4005910 tx=0x7f1ba4004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:28:15.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.983+0000 7f1b9bfff700 1 -- 172.21.15.101:0/1217566467 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba401d070 con 0x7f1bb40ffad0 2023-07-18T03:28:15.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.983+0000 7f1b9bfff700 1 -- 172.21.15.101:0/1217566467 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ba4022470 con 0x7f1bb40ffad0 2023-07-18T03:28:15.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.983+0000 7f1bb97ff700 1 -- 172.21.15.101:0/1217566467 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ba40095f0 con 0x7f1bb40ffad0 2023-07-18T03:28:15.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.983+0000 7f1b9bfff700 1 -- 172.21.15.101:0/1217566467 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba4017650 con 0x7f1bb40ffad0 2023-07-18T03:28:15.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.983+0000 7f1bb97ff700 1 -- 172.21.15.101:0/1217566467 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bb41a48a0 con 0x7f1bb40ffad0 2023-07-18T03:28:15.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.987+0000 7f1b9bfff700 1 -- 172.21.15.101:0/1217566467 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f1ba40225e0 con 0x7f1bb40ffad0 2023-07-18T03:28:15.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.988+0000 7f1bcd15fb80 1 Processor -- start 2023-07-18T03:28:15.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.988+0000 7f1bcd15fb80 1 -- start start 2023-07-18T03:28:15.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.988+0000 7f1b9bfff700 1 --2- 172.21.15.101:0/1217566467 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f1b9c0a12c0 0x7f1b9c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:15.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.988+0000 7f1b9bfff700 1 -- 172.21.15.101:0/1217566467 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f1ba40ce390 con 0x7f1bb40ffad0 2023-07-18T03:28:15.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.988+0000 7f1bb37fe700 1 --2- 172.21.15.101:0/1217566467 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f1b9c0a12c0 0x7f1b9c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:15.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.989+0000 7f1bcd15fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56517ef1d2d0 0x56517ef1d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:15.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.989+0000 7f1bcd15fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56517ef1dbe0 0x56517ef28590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:15.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.989+0000 7f1bcd15fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56517ef28ad0 0x56517ef2aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:15.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.989+0000 7f1bcd15fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517ee23cb0 con 0x56517ef1dbe0 2023-07-18T03:28:15.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.989+0000 7f1bcd15fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517ee76a40 con 0x56517ef28ad0 2023-07-18T03:28:15.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.989+0000 7f1bcd15fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517ee6fed0 con 0x56517ef1d2d0 2023-07-18T03:28:15.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.989+0000 7f1bb2ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56517ef1d2d0 0x56517ef1d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:15.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.989+0000 7f1bb2ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56517ef1d2d0 0x56517ef1d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:36278/0 (socket says 172.21.15.101:36278) 2023-07-18T03:28:15.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.989+0000 7f1bb2ffd700 1 -- 172.21.15.101:0/3579073548 learned_addr learned my addr 172.21.15.101:0/3579073548 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:28:15.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.989+0000 7f1bb37fe700 1 --2- 172.21.15.101:0/3579073548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56517ef28ad0 0x56517ef2aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:15.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.989+0000 7f1bb27fc700 1 --2- 172.21.15.101:0/3579073548 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56517ef1dbe0 0x56517ef28590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:15.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.989+0000 7f1bb2ffd700 1 -- 172.21.15.101:0/3579073548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56517ef28ad0 msgr2=0x56517ef2aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:28:15.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.989+0000 7f1bb2ffd700 1 --2- 172.21.15.101:0/3579073548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56517ef28ad0 0x56517ef2aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:15.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.989+0000 7f1bb2ffd700 1 -- 172.21.15.101:0/3579073548 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56517ef1dbe0 msgr2=0x56517ef28590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:28:15.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.989+0000 7f1bb2ffd700 1 --2- 172.21.15.101:0/3579073548 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56517ef1dbe0 0x56517ef28590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:15.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.989+0000 7f1bb2ffd700 1 -- 172.21.15.101:0/3579073548 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56517ec9dde0 con 0x56517ef1d2d0 2023-07-18T03:28:15.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.989+0000 7f1bb37fe700 1 --2- 172.21.15.101:0/1217566467 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f1b9c0a12c0 0x7f1b9c0a3770 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1bac00f870 tx=0x7f1bac00fc30 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:28:15.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.989+0000 7f1bb2ffd700 1 --2- 172.21.15.101:0/3579073548 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56517ef1d2d0 0x56517ef1d6a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4006010 tx=0x7f1ba409ac50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcb5643d1ebde529 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:28:15.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.990+0000 7f1b997fa700 1 -- 172.21.15.101:0/3579073548 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba40ce170 con 0x56517ef1d2d0 2023-07-18T03:28:15.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.990+0000 7f1b997fa700 1 -- 172.21.15.101:0/3579073548 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ba409f0c0 con 0x56517ef1d2d0 2023-07-18T03:28:15.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.990+0000 7f1b997fa700 1 -- 172.21.15.101:0/3579073548 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba409a580 con 0x56517ef1d2d0 2023-07-18T03:28:15.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.990+0000 7f1bcd15fb80 1 -- 172.21.15.101:0/3579073548 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56517ef1d2d0 msgr2=0x56517ef1d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:15.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.990+0000 7f1bcd15fb80 1 --2- 172.21.15.101:0/3579073548 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56517ef1d2d0 0x56517ef1d6a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4006010 tx=0x7f1ba409ac50 comp rx=0 tx=0).stop 2023-07-18T03:28:15.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.991+0000 7f1bcd15fb80 1 -- 172.21.15.101:0/3579073548 shutdown_connections 2023-07-18T03:28:15.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.991+0000 7f1bcd15fb80 1 --2- 172.21.15.101:0/3579073548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56517ef28ad0 0x56517ef2aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:15.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.991+0000 7f1bcd15fb80 1 --2- 172.21.15.101:0/3579073548 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56517ef1d2d0 0x56517ef1d6a0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:15.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.991+0000 7f1bcd15fb80 1 --2- 172.21.15.101:0/3579073548 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56517ef1dbe0 0x56517ef28590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:15.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.991+0000 7f1bcd15fb80 1 -- 172.21.15.101:0/3579073548 >> 172.21.15.101:0/3579073548 conn(0x56517ef13da0 msgr2=0x56517ef196a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:28:15.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.991+0000 7f1bcd15fb80 1 -- 172.21.15.101:0/3579073548 shutdown_connections 2023-07-18T03:28:15.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.991+0000 7f1bcd15fb80 1 -- 172.21.15.101:0/3579073548 wait complete. 2023-07-18T03:28:15.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.991+0000 7f1bcd15fb80 1 -- start start 2023-07-18T03:28:15.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.992+0000 7f1bcd15fb80 1 Processor -- start 2023-07-18T03:28:15.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.992+0000 7f1bcd15fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56517ef1d2d0 0x56517ef2db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:15.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.992+0000 7f1bcd15fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56517ef1dbe0 0x56517ef2e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:15.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.992+0000 7f1bcd15fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56517ef28ad0 0x56517ef27aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:15.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.992+0000 7f1bcd15fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517ee0fac0 con 0x56517ef1dbe0 2023-07-18T03:28:15.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.992+0000 7f1bcd15fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517ee0f7c0 con 0x56517ef28ad0 2023-07-18T03:28:15.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.992+0000 7f1bcd15fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517ecf3920 con 0x56517ef1d2d0 2023-07-18T03:28:15.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.992+0000 7f1bcd15fb80 10 client.0 _ll_register_callbacks cb 0x56517ee0d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:28:15.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.993+0000 7f1bb2ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56517ef1d2d0 0x56517ef2db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:15.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.993+0000 7f1bb37fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56517ef28ad0 0x56517ef27aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:16.000 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.993+0000 7f1bb2ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56517ef1d2d0 0x56517ef2db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:36288/0 (socket says 172.21.15.101:36288) 2023-07-18T03:28:16.000 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.993+0000 7f1bb2ffd700 1 -- 172.21.15.101:0/3999964864 learned_addr learned my addr 172.21.15.101:0/3999964864 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:28:16.000 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.993+0000 7f1bb27fc700 1 --2- 172.21.15.101:0/3999964864 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56517ef1dbe0 0x56517ef2e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:16.000 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.993+0000 7f1bb37fe700 1 -- 172.21.15.101:0/3999964864 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56517ef1d2d0 msgr2=0x56517ef2db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:28:16.000 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.993+0000 7f1bb37fe700 1 --2- 172.21.15.101:0/3999964864 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56517ef1d2d0 0x56517ef2db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:16.001 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.993+0000 7f1bb37fe700 1 -- 172.21.15.101:0/3999964864 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56517ef1dbe0 msgr2=0x56517ef2e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:28:16.001 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.993+0000 7f1bb37fe700 1 --2- 172.21.15.101:0/3999964864 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56517ef1dbe0 0x56517ef2e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:16.001 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.993+0000 7f1bb37fe700 1 -- 172.21.15.101:0/3999964864 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bb41a48a0 con 0x56517ef28ad0 2023-07-18T03:28:16.001 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.993+0000 7f1bb2ffd700 1 --2- 172.21.15.101:0/3999964864 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56517ef1d2d0 0x56517ef2db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:28:16.001 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.993+0000 7f1bb37fe700 1 --2- 172.21.15.101:0/3999964864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56517ef28ad0 0x56517ef27aa0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1bac018ad0 tx=0x7f1bac004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be28201e000b72ea server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:28:16.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.993+0000 7f1b96ff5700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:28:16.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.994+0000 7f1b96ff5700 1 -- 172.21.15.101:0/3999964864 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bac028070 con 0x56517ef28ad0 2023-07-18T03:28:16.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.994+0000 7f1b96ff5700 1 -- 172.21.15.101:0/3999964864 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f1bac0093f0 con 0x56517ef28ad0 2023-07-18T03:28:16.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.994+0000 7f1bcd15fb80 1 -- 172.21.15.101:0/3999964864 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56517edc94f0 con 0x56517ef28ad0 2023-07-18T03:28:16.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.994+0000 7f1b96ff5700 1 -- 172.21.15.101:0/3999964864 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bac00e040 con 0x56517ef28ad0 2023-07-18T03:28:16.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.994+0000 7f1bcd15fb80 10 client.29048 fetch_fsmap learned FSMap version 48 2023-07-18T03:28:16.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.994+0000 7f1bcd15fb80 1 -- 172.21.15.101:0/3999964864 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x56517ec86c60 con 0x56517ef28ad0 2023-07-18T03:28:16.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.994+0000 7f1b96ff5700 1 -- 172.21.15.101:0/3999964864 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56517edc94f0 con 0x56517ef28ad0 2023-07-18T03:28:16.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.994+0000 7f1b96ff5700 1 -- 172.21.15.101:0/3999964864 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 48) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f1bac02b020 con 0x56517ef28ad0 2023-07-18T03:28:16.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.995+0000 7f1bcd15fb80 10 client.29048 fetch_fsmap finished waiting for FSMap version 48 2023-07-18T03:28:16.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.995+0000 7f1bcd15fb80 10 client.29048 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:28:16.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.995+0000 7f1bcd15fb80 20 client.29048 populate_metadata read hostname 'smithi101' 2023-07-18T03:28:16.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.995+0000 7f1bcd15fb80 1 --2- 172.21.15.101:0/3999964864 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56517ef38160 0x56517ef3a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:16.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.995+0000 7f1bcd15fb80 4 client.29048 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:28:16.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.995+0000 7f1bcd15fb80 1 -- 172.21.15.101:0/3999964864 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56517ee68eb0 con 0x56517ef38160 2023-07-18T03:28:16.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.995+0000 7f1bb2ffd700 1 --2- 172.21.15.101:0/3999964864 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56517ef38160 0x56517ef3a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:16.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.995+0000 7f1bb2ffd700 1 --2- 172.21.15.101:0/3999964864 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56517ef38160 0x56517ef3a570 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8d11d936e97d1e7 server_cookie=de77220bc4df7027 in_seq=0 out_seq=0 2023-07-18T03:28:16.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.995+0000 7f1b96ff5700 10 client.29048 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:28:16.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.996+0000 7f1b96ff5700 1 -- 172.21.15.101:0/3999964864 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56517ee68eb0 con 0x56517ef38160 2023-07-18T03:28:16.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.996+0000 7f1b96ff5700 10 client.29048 handle_command_reply: tid=0 2023-07-18T03:28:16.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.996+0000 7f1bcd15fb80 1 client.29048 shutdown 2023-07-18T03:28:16.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.996+0000 7f1bcd15fb80 1 -- 172.21.15.101:0/3999964864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56517ef28ad0 msgr2=0x56517ef27aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:16.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.996+0000 7f1bcd15fb80 1 --2- 172.21.15.101:0/3999964864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56517ef28ad0 0x56517ef27aa0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1bac018ad0 tx=0x7f1bac004b40 comp rx=0 tx=0).stop 2023-07-18T03:28:16.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.996+0000 7f1bcd15fb80 1 -- 172.21.15.101:0/3999964864 shutdown_connections 2023-07-18T03:28:16.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.996+0000 7f1bcd15fb80 1 --2- 172.21.15.101:0/3999964864 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56517ef38160 0x56517ef3a570 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:16.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.996+0000 7f1bcd15fb80 1 --2- 172.21.15.101:0/3999964864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56517ef28ad0 0x56517ef27aa0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:16.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.996+0000 7f1bcd15fb80 1 --2- 172.21.15.101:0/3999964864 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56517ef1d2d0 0x56517ef2db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:16.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.996+0000 7f1bcd15fb80 1 --2- 172.21.15.101:0/3999964864 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56517ef1dbe0 0x56517ef2e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:16.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.996+0000 7f1b96ff5700 0 client.29048 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:28:16.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.996+0000 7f1bcd15fb80 1 -- 172.21.15.101:0/3999964864 >> 172.21.15.101:0/3999964864 conn(0x56517ef13da0 msgr2=0x56517ef19290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:28:16.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.996+0000 7f1bcd15fb80 1 -- 172.21.15.101:0/3999964864 shutdown_connections 2023-07-18T03:28:16.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.996+0000 7f1bcd15fb80 1 -- 172.21.15.101:0/3999964864 wait complete. 2023-07-18T03:28:16.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:15.997+0000 7f1bcd15fb80 20 client.29048 trim_cache size 0 max 16384 2023-07-18T03:28:16.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.014+0000 7f1bcd15fb80 1 Processor -- start 2023-07-18T03:28:16.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.015+0000 7f1bcd15fb80 1 -- start start 2023-07-18T03:28:16.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.015+0000 7f1bcd15fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56517ef13da0 0x56517ef27bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:16.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.015+0000 7f1bcd15fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56517ef1d2d0 0x56517ef2d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:16.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.015+0000 7f1bcd15fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56517ef1dbe0 0x56517ef2dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:16.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.015+0000 7f1bcd15fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517ef13990 con 0x56517ef1dbe0 2023-07-18T03:28:16.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.015+0000 7f1bcd15fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517ef1ed60 con 0x56517ef13da0 2023-07-18T03:28:16.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.015+0000 7f1bcd15fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517ef28f80 con 0x56517ef1d2d0 2023-07-18T03:28:16.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.015+0000 7f1bb2ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56517ef13da0 0x56517ef27bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:16.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.015+0000 7f1bb2ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56517ef13da0 0x56517ef27bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:55140/0 (socket says 172.21.15.101:55140) 2023-07-18T03:28:16.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.015+0000 7f1bb2ffd700 1 -- 172.21.15.101:0/3360202664 learned_addr learned my addr 172.21.15.101:0/3360202664 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:28:16.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.015+0000 7f1bb27fc700 1 --2- 172.21.15.101:0/3360202664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56517ef1d2d0 0x56517ef2d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:16.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.016+0000 7f1bb2ffd700 1 -- 172.21.15.101:0/3360202664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56517ef1d2d0 msgr2=0x56517ef2d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:28:16.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.016+0000 7f1bb2ffd700 1 --2- 172.21.15.101:0/3360202664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56517ef1d2d0 0x56517ef2d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:16.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.016+0000 7f1bb37fe700 1 --2- 172.21.15.101:0/3360202664 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56517ef1dbe0 0x56517ef2dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:16.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.016+0000 7f1bb2ffd700 1 -- 172.21.15.101:0/3360202664 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56517ef1dbe0 msgr2=0x56517ef2dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:28:16.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.016+0000 7f1bb2ffd700 1 --2- 172.21.15.101:0/3360202664 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56517ef1dbe0 0x56517ef2dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:16.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.016+0000 7f1bb2ffd700 1 -- 172.21.15.101:0/3360202664 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56517ecc4250 con 0x56517ef13da0 2023-07-18T03:28:16.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.016+0000 7f1bb37fe700 1 --2- 172.21.15.101:0/3360202664 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56517ef1dbe0 0x56517ef2dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:28:16.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.016+0000 7f1bb2ffd700 1 --2- 172.21.15.101:0/3360202664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56517ef13da0 0x56517ef27bd0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4024000 tx=0x7f1ba401a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a296c582a6a99330 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:28:16.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.016+0000 7f1b99ffb700 1 -- 172.21.15.101:0/3360202664 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba4022900 con 0x56517ef13da0 2023-07-18T03:28:16.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.016+0000 7f1b99ffb700 1 -- 172.21.15.101:0/3360202664 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ba409b2f0 con 0x56517ef13da0 2023-07-18T03:28:16.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.016+0000 7f1b99ffb700 1 -- 172.21.15.101:0/3360202664 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba40a3200 con 0x56517ef13da0 2023-07-18T03:28:16.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.017+0000 7f1bcd15fb80 1 -- 172.21.15.101:0/3360202664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56517ef13da0 msgr2=0x56517ef27bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:16.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.017+0000 7f1bcd15fb80 1 --2- 172.21.15.101:0/3360202664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56517ef13da0 0x56517ef27bd0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4024000 tx=0x7f1ba401a040 comp rx=0 tx=0).stop 2023-07-18T03:28:16.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.017+0000 7f1bcd15fb80 1 -- 172.21.15.101:0/3360202664 shutdown_connections 2023-07-18T03:28:16.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.017+0000 7f1bcd15fb80 1 --2- 172.21.15.101:0/3360202664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56517ef13da0 0x56517ef27bd0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:16.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.017+0000 7f1bcd15fb80 1 --2- 172.21.15.101:0/3360202664 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56517ef1dbe0 0x56517ef2dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:16.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.017+0000 7f1bcd15fb80 1 --2- 172.21.15.101:0/3360202664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56517ef1d2d0 0x56517ef2d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:16.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.017+0000 7f1bcd15fb80 1 -- 172.21.15.101:0/3360202664 >> 172.21.15.101:0/3360202664 conn(0x56517ef28700 msgr2=0x56517ef3ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:28:16.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.017+0000 7f1bcd15fb80 1 -- 172.21.15.101:0/3360202664 shutdown_connections 2023-07-18T03:28:16.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.017+0000 7f1bcd15fb80 1 -- 172.21.15.101:0/3360202664 wait complete. 2023-07-18T03:28:16.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.017+0000 7f1bcd15fb80 1 -- start start 2023-07-18T03:28:16.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.018+0000 7f1bcd15fb80 1 Processor -- start 2023-07-18T03:28:16.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.018+0000 7f1bcd15fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56517ef13da0 0x56517ef80160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:16.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.018+0000 7f1bcd15fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56517ef1d2d0 0x56517ef806a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:16.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.018+0000 7f1bcd15fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56517ef1dbe0 0x56517ef85030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:16.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.018+0000 7f1bcd15fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517ef85570 con 0x56517ef1d2d0 2023-07-18T03:28:16.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.018+0000 7f1bcd15fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517ef856b0 con 0x56517ef13da0 2023-07-18T03:28:16.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.018+0000 7f1bcd15fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517ef85870 con 0x56517ef1dbe0 2023-07-18T03:28:16.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.018+0000 7f1bcd15fb80 10 client.0 _ll_register_callbacks cb 0x7f1bac009340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:28:16.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.018+0000 7f1bb2ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56517ef13da0 0x56517ef80160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:16.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.018+0000 7f1bb2ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56517ef13da0 0x56517ef80160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:55142/0 (socket says 172.21.15.101:55142) 2023-07-18T03:28:16.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.018+0000 7f1bb2ffd700 1 -- 172.21.15.101:0/2803567853 learned_addr learned my addr 172.21.15.101:0/2803567853 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:28:16.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.019+0000 7f1bb37fe700 1 --2- 172.21.15.101:0/2803567853 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56517ef1dbe0 0x56517ef85030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:16.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.019+0000 7f1bb27fc700 1 --2- 172.21.15.101:0/2803567853 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56517ef1d2d0 0x56517ef806a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:16.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.019+0000 7f1bb2ffd700 1 -- 172.21.15.101:0/2803567853 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56517ef1dbe0 msgr2=0x56517ef85030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:28:16.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.019+0000 7f1bb2ffd700 1 --2- 172.21.15.101:0/2803567853 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56517ef1dbe0 0x56517ef85030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:16.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.019+0000 7f1bb2ffd700 1 -- 172.21.15.101:0/2803567853 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56517ef1d2d0 msgr2=0x56517ef806a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:28:16.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.019+0000 7f1bb2ffd700 1 --2- 172.21.15.101:0/2803567853 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56517ef1d2d0 0x56517ef806a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:16.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.019+0000 7f1bb2ffd700 1 -- 172.21.15.101:0/2803567853 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56517ec9dde0 con 0x56517ef13da0 2023-07-18T03:28:16.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.019+0000 7f1bb2ffd700 1 --2- 172.21.15.101:0/2803567853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56517ef13da0 0x56517ef80160 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4097540 tx=0x7f1ba4022cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad3a71137ac46904 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:28:16.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.019+0000 7f1b96ff5700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:28:16.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.019+0000 7f1b96ff5700 1 -- 172.21.15.101:0/2803567853 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba409b460 con 0x56517ef13da0 2023-07-18T03:28:16.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.019+0000 7f1bcd15fb80 1 -- 172.21.15.101:0/2803567853 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ba409f0c0 con 0x56517ef13da0 2023-07-18T03:28:16.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.019+0000 7f1b96ff5700 1 -- 172.21.15.101:0/2803567853 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f1ba40cb2b0 con 0x56517ef13da0 2023-07-18T03:28:16.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.019+0000 7f1b96ff5700 1 -- 172.21.15.101:0/2803567853 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ba409b5c0 con 0x56517ef13da0 2023-07-18T03:28:16.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.020+0000 7f1bcd15fb80 10 client.29054 fetch_fsmap learned FSMap version 48 2023-07-18T03:28:16.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.020+0000 7f1bcd15fb80 1 -- 172.21.15.101:0/2803567853 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x56517ec9c2b0 con 0x56517ef13da0 2023-07-18T03:28:16.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.020+0000 7f1b96ff5700 1 -- 172.21.15.101:0/2803567853 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ba409f0c0 con 0x56517ef13da0 2023-07-18T03:28:16.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.020+0000 7f1b96ff5700 1 -- 172.21.15.101:0/2803567853 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 48) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f1ba4084380 con 0x56517ef13da0 2023-07-18T03:28:16.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.020+0000 7f1bcd15fb80 10 client.29054 fetch_fsmap finished waiting for FSMap version 48 2023-07-18T03:28:16.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.020+0000 7f1bcd15fb80 10 client.29054 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:28:16.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.020+0000 7f1bcd15fb80 20 client.29054 populate_metadata read hostname 'smithi101' 2023-07-18T03:28:16.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.020+0000 7f1bcd15fb80 1 --2- 172.21.15.101:0/2803567853 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56517ef28ad0 0x56517ef8ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:16.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.020+0000 7f1bcd15fb80 4 client.29054 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:28:16.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.020+0000 7f1bcd15fb80 1 -- 172.21.15.101:0/2803567853 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56517ee2b050 con 0x56517ef28ad0 2023-07-18T03:28:16.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.020+0000 7f1bb27fc700 1 --2- 172.21.15.101:0/2803567853 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56517ef28ad0 0x56517ef8ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:16.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.021+0000 7f1bb27fc700 1 --2- 172.21.15.101:0/2803567853 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56517ef28ad0 0x56517ef8ec00 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28b40c04e632b57f server_cookie=401ee0b0d5e88ce3 in_seq=0 out_seq=0 2023-07-18T03:28:16.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.021+0000 7f1b96ff5700 10 client.29054 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:28:16.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.021+0000 7f1b96ff5700 1 -- 172.21.15.101:0/2803567853 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56517ee2b050 con 0x56517ef28ad0 2023-07-18T03:28:16.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.021+0000 7f1b96ff5700 10 client.29054 handle_command_reply: tid=0 2023-07-18T03:28:16.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.021+0000 7f1bcd15fb80 1 client.29054 shutdown 2023-07-18T03:28:16.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.021+0000 7f1bcd15fb80 1 -- 172.21.15.101:0/2803567853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56517ef13da0 msgr2=0x56517ef80160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:16.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.021+0000 7f1bcd15fb80 1 --2- 172.21.15.101:0/2803567853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56517ef13da0 0x56517ef80160 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4097540 tx=0x7f1ba4022cd0 comp rx=0 tx=0).stop 2023-07-18T03:28:16.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.021+0000 7f1bcd15fb80 1 -- 172.21.15.101:0/2803567853 shutdown_connections 2023-07-18T03:28:16.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.021+0000 7f1bcd15fb80 1 --2- 172.21.15.101:0/2803567853 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x56517ef28ad0 0x56517ef8ec00 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:16.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.021+0000 7f1bcd15fb80 1 --2- 172.21.15.101:0/2803567853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56517ef13da0 0x56517ef80160 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:16.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.021+0000 7f1bcd15fb80 1 --2- 172.21.15.101:0/2803567853 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56517ef1dbe0 0x56517ef85030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:16.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.021+0000 7f1b96ff5700 0 client.29054 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:28:16.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.021+0000 7f1bcd15fb80 1 --2- 172.21.15.101:0/2803567853 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56517ef1d2d0 0x56517ef806a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:16.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.021+0000 7f1bcd15fb80 1 -- 172.21.15.101:0/2803567853 >> 172.21.15.101:0/2803567853 conn(0x56517ef28700 msgr2=0x56517ef1f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:28:16.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.021+0000 7f1bcd15fb80 1 -- 172.21.15.101:0/2803567853 shutdown_connections 2023-07-18T03:28:16.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.021+0000 7f1bcd15fb80 1 -- 172.21.15.101:0/2803567853 wait complete. 2023-07-18T03:28:16.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.022+0000 7f1bcd15fb80 20 client.29054 trim_cache size 0 max 16384 2023-07-18T03:28:16.032 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:28:16.032 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:28:16.032 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:28:16.032 INFO:teuthology.orchestra.run.smithi101.stdout: "0f2f19e0-0429-465c-935e-223eb09490c3": { 2023-07-18T03:28:16.033 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:28:16.033 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "0f2f19e0-0429-465c-935e-223eb09490c3", 2023-07-18T03:28:16.033 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:28:16.033 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:28:16.033 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:28:16.034 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:28:16.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.022+0000 7f1bb97ff700 1 -- 172.21.15.101:0/1217566467 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f1b9c0a12c0 msgr2=0x7f1b9c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:16.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.022+0000 7f1bb97ff700 1 --2- 172.21.15.101:0/1217566467 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f1b9c0a12c0 0x7f1b9c0a3770 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1bac00f870 tx=0x7f1bac00fc30 comp rx=0 tx=0).stop 2023-07-18T03:28:16.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.022+0000 7f1bb97ff700 1 -- 172.21.15.101:0/1217566467 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1bb40ffad0 msgr2=0x7f1bb4072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:16.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.022+0000 7f1bb97ff700 1 --2- 172.21.15.101:0/1217566467 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1bb40ffad0 0x7f1bb4072810 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4005910 tx=0x7f1ba4004710 comp rx=0 tx=0).stop 2023-07-18T03:28:16.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.022+0000 7f1bb97ff700 1 -- 172.21.15.101:0/1217566467 shutdown_connections 2023-07-18T03:28:16.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.022+0000 7f1bb97ff700 1 --2- 172.21.15.101:0/1217566467 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f1b9c0a12c0 0x7f1b9c0a3770 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:16.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.022+0000 7f1bb97ff700 1 --2- 172.21.15.101:0/1217566467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bb4105ae0 0x7f1bb4073290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:16.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.022+0000 7f1bb97ff700 1 --2- 172.21.15.101:0/1217566467 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1bb40ffad0 0x7f1bb4072810 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:16.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.022+0000 7f1bb97ff700 1 --2- 172.21.15.101:0/1217566467 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1bb4100480 0x7f1bb4072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:16.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.022+0000 7f1bb97ff700 1 -- 172.21.15.101:0/1217566467 >> 172.21.15.101:0/1217566467 conn(0x7f1bb40fb5f0 msgr2=0x7f1bb40fbc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:28:16.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.022+0000 7f1bb97ff700 1 -- 172.21.15.101:0/1217566467 shutdown_connections 2023-07-18T03:28:16.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:16.022+0000 7f1bb97ff700 1 -- 172.21.15.101:0/1217566467 wait complete. 2023-07-18T03:28:16.038 INFO:tasks.cephfs.filesystem:scrub status for tag:0f2f19e0-0429-465c-935e-223eb09490c3 - {'path': '/', 'tag': '0f2f19e0-0429-465c-935e-223eb09490c3', 'options': 'recursive,force'} 2023-07-18T03:28:16.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:16 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:28:16.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:16 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:28:16.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:16 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:28:17.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:17 smithi179 ceph-mon[129229]: pgmap v653: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:28:17.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:17 smithi101 ceph-mon[126379]: pgmap v653: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:28:17.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:17 smithi155 ceph-mon[128885]: pgmap v653: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:28:18.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:18 smithi179 ceph-mon[129229]: pgmap v654: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:28:18.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:18 smithi101 ceph-mon[126379]: pgmap v654: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:28:18.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:18 smithi155 ceph-mon[128885]: pgmap v654: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:28:20.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:20 smithi155 ceph-mon[128885]: osdmap e122: 12 total, 12 up, 12 in 2023-07-18T03:28:20.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:20 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:28:20.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:20 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:28:20.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:20 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:28:20.550 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:20 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:28:20.550 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:20 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:20.550 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:20 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:20.550 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:20 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:20.550 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:20 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:20.551 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:20 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:20.551 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:20 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:20.551 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:20 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:20.551 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:20 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:20.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:20 smithi179 ceph-mon[129229]: osdmap e122: 12 total, 12 up, 12 in 2023-07-18T03:28:20.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:20 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:28:20.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:20 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:28:20.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:20 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:28:20.705 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:20 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:28:20.705 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:20 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:20.705 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:20 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:20.705 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:20 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:20.705 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:20 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:20.706 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:20 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:20.706 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:20 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:20.706 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:20 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:20.706 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:20 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:20.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:20 smithi101 ceph-mon[126379]: osdmap e122: 12 total, 12 up, 12 in 2023-07-18T03:28:20.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:20 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:28:20.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:20 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:28:20.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:20 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:28:20.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:20 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:28:20.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:20 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:20.743 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:20 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:20.743 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:20 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:20.743 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:20 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:20.743 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:20 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:20.743 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:20 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:20.744 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:20 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:20.744 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:20 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:21.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:21 smithi155 ceph-mon[128885]: pgmap v656: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 212 B/s wr, 6 op/s 2023-07-18T03:28:21.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:21 smithi155 ceph-mon[128885]: Adjusting osd_memory_target on smithi101 to 220.9M 2023-07-18T03:28:21.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:21 smithi155 ceph-mon[128885]: Unable to set osd_memory_target on smithi101 to 231653990: error parsing value: Value '231653990' is below minimum 939524096 2023-07-18T03:28:21.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:21 smithi155 ceph-mon[128885]: Adjusting osd_memory_target on smithi155 to 196.2M 2023-07-18T03:28:21.550 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:21 smithi155 ceph-mon[128885]: Unable to set osd_memory_target on smithi155 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-07-18T03:28:21.550 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:21 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:28:21.550 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:21 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:28:21.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:21 smithi179 ceph-mon[129229]: pgmap v656: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 212 B/s wr, 6 op/s 2023-07-18T03:28:21.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:21 smithi179 ceph-mon[129229]: Adjusting osd_memory_target on smithi101 to 220.9M 2023-07-18T03:28:21.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:21 smithi179 ceph-mon[129229]: Unable to set osd_memory_target on smithi101 to 231653990: error parsing value: Value '231653990' is below minimum 939524096 2023-07-18T03:28:21.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:21 smithi179 ceph-mon[129229]: Adjusting osd_memory_target on smithi155 to 196.2M 2023-07-18T03:28:21.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:21 smithi179 ceph-mon[129229]: Unable to set osd_memory_target on smithi155 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-07-18T03:28:21.705 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:21 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:28:21.705 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:21 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:28:21.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:21 smithi101 ceph-mon[126379]: pgmap v656: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 212 B/s wr, 6 op/s 2023-07-18T03:28:21.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:21 smithi101 ceph-mon[126379]: Adjusting osd_memory_target on smithi101 to 220.9M 2023-07-18T03:28:21.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:21 smithi101 ceph-mon[126379]: Unable to set osd_memory_target on smithi101 to 231653990: error parsing value: Value '231653990' is below minimum 939524096 2023-07-18T03:28:21.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:21 smithi101 ceph-mon[126379]: Adjusting osd_memory_target on smithi155 to 196.2M 2023-07-18T03:28:21.743 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:21 smithi101 ceph-mon[126379]: Unable to set osd_memory_target on smithi155 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-07-18T03:28:21.743 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:21 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:28:21.743 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:21 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:28:22.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:22 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:22.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:22 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:22.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:22 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:22.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:22 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:22.550 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:22 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:28:22.550 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:22 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:28:22.550 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:22 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:28:22.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:22 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:22.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:22 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:22.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:22 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:22.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:22 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:22.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:22 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:28:22.705 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:22 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:28:22.705 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:22 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:28:22.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:22 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:22.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:22 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:22.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:22 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:22.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:22 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:22.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:22 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:28:22.743 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:22 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:28:22.743 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:22 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:28:23.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:23 smithi155 ceph-mon[128885]: Adjusting osd_memory_target on smithi179 to 196.2M 2023-07-18T03:28:23.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:23 smithi155 ceph-mon[128885]: Unable to set osd_memory_target on smithi179 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-07-18T03:28:23.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:23 smithi155 ceph-mon[128885]: pgmap v657: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:28:23.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:23 smithi179 ceph-mon[129229]: Adjusting osd_memory_target on smithi179 to 196.2M 2023-07-18T03:28:23.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:23 smithi179 ceph-mon[129229]: Unable to set osd_memory_target on smithi179 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-07-18T03:28:23.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:23 smithi179 ceph-mon[129229]: pgmap v657: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:28:23.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:23 smithi101 ceph-mon[126379]: Adjusting osd_memory_target on smithi179 to 196.2M 2023-07-18T03:28:23.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:23 smithi101 ceph-mon[126379]: Unable to set osd_memory_target on smithi179 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-07-18T03:28:23.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:23 smithi101 ceph-mon[126379]: pgmap v657: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:28:25.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:25 smithi155 ceph-mon[128885]: pgmap v658: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:28:25.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:25 smithi179 ceph-mon[129229]: pgmap v658: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:28:25.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:25 smithi101 ceph-mon[126379]: pgmap v658: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:28:27.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:27 smithi155 ceph-mon[128885]: pgmap v659: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:28:27.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:27 smithi179 ceph-mon[129229]: pgmap v659: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:28:27.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:27 smithi101 ceph-mon[126379]: pgmap v659: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:28:29.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:29 smithi155 ceph-mon[128885]: pgmap v660: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:28:29.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:29 smithi179 ceph-mon[129229]: pgmap v660: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:28:29.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:29 smithi101 ceph-mon[126379]: pgmap v660: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:28:31.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:31 smithi155 ceph-mon[128885]: pgmap v661: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 95 B/s wr, 7 op/s 2023-07-18T03:28:31.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:31 smithi179 ceph-mon[129229]: pgmap v661: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 95 B/s wr, 7 op/s 2023-07-18T03:28:31.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:31 smithi101 ceph-mon[126379]: pgmap v661: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 95 B/s wr, 7 op/s 2023-07-18T03:28:33.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:33 smithi155 ceph-mon[128885]: pgmap v662: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:28:33.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:33 smithi179 ceph-mon[129229]: pgmap v662: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:28:33.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:33 smithi101 ceph-mon[126379]: pgmap v662: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:28:35.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:35 smithi155 ceph-mon[128885]: pgmap v663: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:28:35.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:35 smithi179 ceph-mon[129229]: pgmap v663: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:28:35.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:35 smithi101 ceph-mon[126379]: pgmap v663: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:28:37.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:37 smithi155 ceph-mon[128885]: pgmap v664: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:28:37.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:37 smithi179 ceph-mon[129229]: pgmap v664: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:28:37.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:37 smithi101 ceph-mon[126379]: pgmap v664: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:28:39.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:39 smithi155 ceph-mon[128885]: pgmap v665: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:28:39.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:39 smithi179 ceph-mon[129229]: pgmap v665: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:28:39.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:39 smithi101 ceph-mon[126379]: pgmap v665: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:28:41.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:41 smithi155 ceph-mon[128885]: pgmap v666: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-18T03:28:41.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:41 smithi179 ceph-mon[129229]: pgmap v666: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-18T03:28:41.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:41 smithi101 ceph-mon[126379]: pgmap v666: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-18T03:28:43.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:43 smithi155 ceph-mon[128885]: pgmap v667: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:28:43.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:43 smithi179 ceph-mon[129229]: pgmap v667: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:28:43.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:43 smithi101 ceph-mon[126379]: pgmap v667: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:28:45.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:45 smithi155 ceph-mon[128885]: pgmap v668: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:28:45.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:45 smithi179 ceph-mon[129229]: pgmap v668: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:28:45.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:45 smithi101 ceph-mon[126379]: pgmap v668: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:28:46.039 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:28:46.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.255+0000 7f32831ff700 1 -- 172.21.15.101:0/1662067413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f327c105af0 msgr2=0x7f327c105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:46.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.255+0000 7f32831ff700 1 --2- 172.21.15.101:0/1662067413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f327c105af0 0x7f327c105ec0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f3270009990 tx=0x7f3270009ca0 comp rx=0 tx=0).stop 2023-07-18T03:28:46.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.255+0000 7f32831ff700 1 -- 172.21.15.101:0/1662067413 shutdown_connections 2023-07-18T03:28:46.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.255+0000 7f32831ff700 1 --2- 172.21.15.101:0/1662067413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f327c105af0 0x7f327c105ec0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:46.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.255+0000 7f32831ff700 1 --2- 172.21.15.101:0/1662067413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f327c100490 0x7f327c10c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:46.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.255+0000 7f32831ff700 1 --2- 172.21.15.101:0/1662067413 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f327c0ffae0 0x7f327c0fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:46.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.255+0000 7f32831ff700 1 -- 172.21.15.101:0/1662067413 >> 172.21.15.101:0/1662067413 conn(0x7f327c0fb5e0 msgr2=0x7f327c0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:28:46.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.255+0000 7f32831ff700 1 -- 172.21.15.101:0/1662067413 shutdown_connections 2023-07-18T03:28:46.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.255+0000 7f32831ff700 1 -- 172.21.15.101:0/1662067413 wait complete. 2023-07-18T03:28:46.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.256+0000 7f32831ff700 1 Processor -- start 2023-07-18T03:28:46.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.256+0000 7f32831ff700 1 -- start start 2023-07-18T03:28:46.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.257+0000 7f32831ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f327c0ffae0 0x7f327c19b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:46.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.257+0000 7f32831ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f327c100490 0x7f327c19bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:46.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.257+0000 7f32831ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f327c105af0 0x7f327c195a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:46.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.257+0000 7f32831ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327c1960b0 con 0x7f327c100490 2023-07-18T03:28:46.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.257+0000 7f32831ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327c196220 con 0x7f327c0ffae0 2023-07-18T03:28:46.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.257+0000 7f32831ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327c1963e0 con 0x7f327c105af0 2023-07-18T03:28:46.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.257+0000 7f3280f9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f327c0ffae0 0x7f327c19b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:46.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.257+0000 7f328179c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f327c105af0 0x7f327c195a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:46.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.257+0000 7f3280f9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f327c0ffae0 0x7f327c19b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:60994/0 (socket says 172.21.15.101:60994) 2023-07-18T03:28:46.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.257+0000 7f3280f9b700 1 -- 172.21.15.101:0/2201798562 learned_addr learned my addr 172.21.15.101:0/2201798562 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:28:46.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.257+0000 7f327bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f327c100490 0x7f327c19bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:46.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.257+0000 7f3280f9b700 1 -- 172.21.15.101:0/2201798562 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f327c105af0 msgr2=0x7f327c195a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:46.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.257+0000 7f3280f9b700 1 --2- 172.21.15.101:0/2201798562 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f327c105af0 0x7f327c195a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:46.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.257+0000 7f3280f9b700 1 -- 172.21.15.101:0/2201798562 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f327c100490 msgr2=0x7f327c19bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:46.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.257+0000 7f3280f9b700 1 --2- 172.21.15.101:0/2201798562 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f327c100490 0x7f327c19bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:46.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.257+0000 7f3280f9b700 1 -- 172.21.15.101:0/2201798562 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32700095f0 con 0x7f327c0ffae0 2023-07-18T03:28:46.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.258+0000 7f3280f9b700 1 --2- 172.21.15.101:0/2201798562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f327c0ffae0 0x7f327c19b9d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f3270005910 tx=0x7f3270005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:28:46.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.258+0000 7f3279ffb700 1 -- 172.21.15.101:0/2201798562 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327001d070 con 0x7f327c0ffae0 2023-07-18T03:28:46.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.258+0000 7f3279ffb700 1 -- 172.21.15.101:0/2201798562 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3270022470 con 0x7f327c0ffae0 2023-07-18T03:28:46.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.258+0000 7f32831ff700 1 -- 172.21.15.101:0/2201798562 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f327c196660 con 0x7f327c0ffae0 2023-07-18T03:28:46.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.258+0000 7f3279ffb700 1 -- 172.21.15.101:0/2201798562 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3270017650 con 0x7f327c0ffae0 2023-07-18T03:28:46.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.258+0000 7f32831ff700 1 -- 172.21.15.101:0/2201798562 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f327c1a29c0 con 0x7f327c0ffae0 2023-07-18T03:28:46.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.261+0000 7f3279ffb700 1 -- 172.21.15.101:0/2201798562 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f32700225e0 con 0x7f327c0ffae0 2023-07-18T03:28:46.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.262+0000 7f3296be6b80 1 Processor -- start 2023-07-18T03:28:46.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.262+0000 7f3279ffb700 1 --2- 172.21.15.101:0/2201798562 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f326c0a12c0 0x7f326c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:46.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.262+0000 7f3296be6b80 1 -- start start 2023-07-18T03:28:46.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.262+0000 7f3279ffb700 1 -- 172.21.15.101:0/2201798562 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f32700ce430 con 0x7f327c0ffae0 2023-07-18T03:28:46.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.263+0000 7f328179c700 1 --2- 172.21.15.101:0/2201798562 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f326c0a12c0 0x7f326c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:46.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.263+0000 7f3296be6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b17d6076e0 0x55b17d607ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:46.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.263+0000 7f3296be6b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b17d607ff0 0x55b17d6129a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:46.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.263+0000 7f3296be6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b17d612ee0 0x55b17d6152c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:46.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.263+0000 7f3296be6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b17d509cb0 con 0x55b17d612ee0 2023-07-18T03:28:46.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.263+0000 7f3296be6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b17d55ca40 con 0x55b17d6076e0 2023-07-18T03:28:46.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.263+0000 7f3296be6b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b17d555ed0 con 0x55b17d607ff0 2023-07-18T03:28:46.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.263+0000 7f3280f9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b17d6076e0 0x55b17d607ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:46.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.263+0000 7f327bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b17d607ff0 0x55b17d6129a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:46.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.263+0000 7f3280f9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b17d6076e0 0x55b17d607ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:32774/0 (socket says 172.21.15.101:32774) 2023-07-18T03:28:46.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.263+0000 7f327bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b17d607ff0 0x55b17d6129a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:47350/0 (socket says 172.21.15.101:47350) 2023-07-18T03:28:46.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.263+0000 7f3280f9b700 1 -- 172.21.15.101:0/3498096729 learned_addr learned my addr 172.21.15.101:0/3498096729 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:28:46.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.263+0000 7f328179c700 1 --2- 172.21.15.101:0/3498096729 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b17d612ee0 0x55b17d6152c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:46.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.263+0000 7f3280f9b700 1 -- 172.21.15.101:0/3498096729 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b17d607ff0 msgr2=0x55b17d6129a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:28:46.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.263+0000 7f3280f9b700 1 --2- 172.21.15.101:0/3498096729 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b17d607ff0 0x55b17d6129a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:46.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.263+0000 7f3280f9b700 1 -- 172.21.15.101:0/3498096729 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b17d612ee0 msgr2=0x55b17d6152c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:28:46.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.263+0000 7f3280f9b700 1 --2- 172.21.15.101:0/3498096729 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b17d612ee0 0x55b17d6152c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:46.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.263+0000 7f3280f9b700 1 -- 172.21.15.101:0/3498096729 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b17d383de0 con 0x55b17d6076e0 2023-07-18T03:28:46.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.263+0000 7f328179c700 1 --2- 172.21.15.101:0/3498096729 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b17d612ee0 0x55b17d6152c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:28:46.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.264+0000 7f328179c700 1 --2- 172.21.15.101:0/2201798562 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f326c0a12c0 0x7f326c0a3770 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f327c19c6d0 tx=0x7f3274009920 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:28:46.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.264+0000 7f3280f9b700 1 --2- 172.21.15.101:0/3498096729 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b17d6076e0 0x55b17d607ab0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3270097680 tx=0x7f3270005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2467bbea1b25b848 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:28:46.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.264+0000 7f3266ffd700 1 -- 172.21.15.101:0/3498096729 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32700a1070 con 0x55b17d6076e0 2023-07-18T03:28:46.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.264+0000 7f3266ffd700 1 -- 172.21.15.101:0/3498096729 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f327009acc0 con 0x55b17d6076e0 2023-07-18T03:28:46.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.264+0000 7f3266ffd700 1 -- 172.21.15.101:0/3498096729 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3270099640 con 0x55b17d6076e0 2023-07-18T03:28:46.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.264+0000 7f3296be6b80 1 -- 172.21.15.101:0/3498096729 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b17d6076e0 msgr2=0x55b17d607ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:46.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.264+0000 7f3296be6b80 1 --2- 172.21.15.101:0/3498096729 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b17d6076e0 0x55b17d607ab0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3270097680 tx=0x7f3270005f10 comp rx=0 tx=0).stop 2023-07-18T03:28:46.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.264+0000 7f3296be6b80 1 -- 172.21.15.101:0/3498096729 shutdown_connections 2023-07-18T03:28:46.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.265+0000 7f3296be6b80 1 --2- 172.21.15.101:0/3498096729 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b17d6076e0 0x55b17d607ab0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:46.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.265+0000 7f3296be6b80 1 --2- 172.21.15.101:0/3498096729 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b17d612ee0 0x55b17d6152c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:46.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.265+0000 7f3296be6b80 1 --2- 172.21.15.101:0/3498096729 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b17d607ff0 0x55b17d6129a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:46.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.265+0000 7f3296be6b80 1 -- 172.21.15.101:0/3498096729 >> 172.21.15.101:0/3498096729 conn(0x55b17d5f9da0 msgr2=0x55b17d5ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:28:46.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.265+0000 7f3296be6b80 1 -- 172.21.15.101:0/3498096729 shutdown_connections 2023-07-18T03:28:46.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.265+0000 7f3296be6b80 1 -- 172.21.15.101:0/3498096729 wait complete. 2023-07-18T03:28:46.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.265+0000 7f3296be6b80 1 -- start start 2023-07-18T03:28:46.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.265+0000 7f3296be6b80 1 Processor -- start 2023-07-18T03:28:46.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.266+0000 7f3296be6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b17d6076e0 0x55b17d617f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:46.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.266+0000 7f3296be6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b17d607ff0 0x55b17d6184b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:46.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.266+0000 7f3296be6b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b17d612ee0 0x55b17d611df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:46.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.266+0000 7f3296be6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b17d4f5ac0 con 0x55b17d607ff0 2023-07-18T03:28:46.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.266+0000 7f3296be6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b17d4f57c0 con 0x55b17d6076e0 2023-07-18T03:28:46.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.266+0000 7f3296be6b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b17d3d9920 con 0x55b17d612ee0 2023-07-18T03:28:46.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.266+0000 7f3296be6b80 10 client.0 _ll_register_callbacks cb 0x55b17d4f3760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:28:46.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.266+0000 7f328179c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b17d612ee0 0x55b17d611df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:46.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.266+0000 7f3280f9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b17d6076e0 0x55b17d617f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:46.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.266+0000 7f328179c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b17d612ee0 0x55b17d611df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:47358/0 (socket says 172.21.15.101:47358) 2023-07-18T03:28:46.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.266+0000 7f328179c700 1 -- 172.21.15.101:0/3642560859 learned_addr learned my addr 172.21.15.101:0/3642560859 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:28:46.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.266+0000 7f327bfff700 1 --2- 172.21.15.101:0/3642560859 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b17d607ff0 0x55b17d6184b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:46.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.266+0000 7f328179c700 1 -- 172.21.15.101:0/3642560859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b17d6076e0 msgr2=0x55b17d617f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:28:46.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.266+0000 7f328179c700 1 --2- 172.21.15.101:0/3642560859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b17d6076e0 0x55b17d617f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:46.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.266+0000 7f328179c700 1 -- 172.21.15.101:0/3642560859 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b17d607ff0 msgr2=0x55b17d6184b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:28:46.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.266+0000 7f328179c700 1 --2- 172.21.15.101:0/3642560859 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b17d607ff0 0x55b17d6184b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:46.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.266+0000 7f328179c700 1 -- 172.21.15.101:0/3642560859 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f327c1a29c0 con 0x55b17d612ee0 2023-07-18T03:28:46.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.266+0000 7f3280f9b700 1 --2- 172.21.15.101:0/3642560859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b17d6076e0 0x55b17d617f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:28:46.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.267+0000 7f328179c700 1 --2- 172.21.15.101:0/3642560859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b17d612ee0 0x55b17d611df0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f3274019dd0 tx=0x7f3274004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b7a8409a2fcad53 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:28:46.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.267+0000 7f325bfff700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:28:46.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.267+0000 7f325bfff700 1 -- 172.21.15.101:0/3642560859 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3274027070 con 0x55b17d612ee0 2023-07-18T03:28:46.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.267+0000 7f3296be6b80 1 -- 172.21.15.101:0/3642560859 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b17d4af4f0 con 0x55b17d612ee0 2023-07-18T03:28:46.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.267+0000 7f325bfff700 1 -- 172.21.15.101:0/3642560859 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f3274019e00 con 0x55b17d612ee0 2023-07-18T03:28:46.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.267+0000 7f325bfff700 1 -- 172.21.15.101:0/3642560859 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f327400e040 con 0x55b17d612ee0 2023-07-18T03:28:46.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.267+0000 7f3296be6b80 10 client.29056 fetch_fsmap learned FSMap version 48 2023-07-18T03:28:46.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.267+0000 7f3296be6b80 1 -- 172.21.15.101:0/3642560859 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55b17d36cc60 con 0x55b17d612ee0 2023-07-18T03:28:46.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.267+0000 7f325bfff700 1 -- 172.21.15.101:0/3642560859 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b17d4af4f0 con 0x55b17d612ee0 2023-07-18T03:28:46.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.268+0000 7f325bfff700 1 -- 172.21.15.101:0/3642560859 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 48) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f327402a020 con 0x55b17d612ee0 2023-07-18T03:28:46.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.268+0000 7f3296be6b80 10 client.29056 fetch_fsmap finished waiting for FSMap version 48 2023-07-18T03:28:46.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.268+0000 7f3296be6b80 10 client.29056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:28:46.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.268+0000 7f3296be6b80 20 client.29056 populate_metadata read hostname 'smithi101' 2023-07-18T03:28:46.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.268+0000 7f3296be6b80 1 --2- 172.21.15.101:0/3642560859 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b17d603b10 0x55b17d603f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:46.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.268+0000 7f3296be6b80 4 client.29056 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:28:46.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.268+0000 7f3296be6b80 1 -- 172.21.15.101:0/3642560859 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b17d54eeb0 con 0x55b17d603b10 2023-07-18T03:28:46.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.268+0000 7f3280f9b700 1 --2- 172.21.15.101:0/3642560859 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b17d603b10 0x55b17d603f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:46.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.268+0000 7f325bfff700 10 client.29056 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:28:46.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.268+0000 7f3280f9b700 1 --2- 172.21.15.101:0/3642560859 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b17d603b10 0x55b17d603f10 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa8b91c4b7f26c6d server_cookie=9931108b63a89381 in_seq=0 out_seq=0 2023-07-18T03:28:46.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.269+0000 7f325bfff700 1 -- 172.21.15.101:0/3642560859 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b17d54eeb0 con 0x55b17d603b10 2023-07-18T03:28:46.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.269+0000 7f325bfff700 10 client.29056 handle_command_reply: tid=0 2023-07-18T03:28:46.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.269+0000 7f3296be6b80 1 client.29056 shutdown 2023-07-18T03:28:46.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.269+0000 7f3296be6b80 1 -- 172.21.15.101:0/3642560859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b17d612ee0 msgr2=0x55b17d611df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:46.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.269+0000 7f3296be6b80 1 --2- 172.21.15.101:0/3642560859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b17d612ee0 0x55b17d611df0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f3274019dd0 tx=0x7f3274004b40 comp rx=0 tx=0).stop 2023-07-18T03:28:46.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.269+0000 7f3296be6b80 1 -- 172.21.15.101:0/3642560859 shutdown_connections 2023-07-18T03:28:46.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.269+0000 7f3296be6b80 1 --2- 172.21.15.101:0/3642560859 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b17d603b10 0x55b17d603f10 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:46.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.269+0000 7f3296be6b80 1 --2- 172.21.15.101:0/3642560859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b17d6076e0 0x55b17d617f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:46.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.269+0000 7f325bfff700 0 client.29056 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:28:46.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.269+0000 7f3296be6b80 1 --2- 172.21.15.101:0/3642560859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b17d612ee0 0x55b17d611df0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:46.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.269+0000 7f3296be6b80 1 --2- 172.21.15.101:0/3642560859 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b17d607ff0 0x55b17d6184b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:46.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.269+0000 7f3296be6b80 1 -- 172.21.15.101:0/3642560859 >> 172.21.15.101:0/3642560859 conn(0x55b17d5f9da0 msgr2=0x55b17d5ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:28:46.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.269+0000 7f3296be6b80 1 -- 172.21.15.101:0/3642560859 shutdown_connections 2023-07-18T03:28:46.284 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.270+0000 7f3296be6b80 1 -- 172.21.15.101:0/3642560859 wait complete. 2023-07-18T03:28:46.284 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.270+0000 7f3296be6b80 20 client.29056 trim_cache size 0 max 16384 2023-07-18T03:28:46.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.287+0000 7f3296be6b80 1 Processor -- start 2023-07-18T03:28:46.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.287+0000 7f3296be6b80 1 -- start start 2023-07-18T03:28:46.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.287+0000 7f3296be6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b17d6076e0 0x55b17d611f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:46.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.287+0000 7f3296be6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b17d607ff0 0x55b17d606f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:46.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.287+0000 7f3296be6b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b17d612ee0 0x55b17d60c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:46.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.287+0000 7f3296be6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b17d6137a0 con 0x55b17d607ff0 2023-07-18T03:28:46.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.287+0000 7f3296be6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b17d6132d0 con 0x55b17d6076e0 2023-07-18T03:28:46.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.287+0000 7f3296be6b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b17d5f9970 con 0x55b17d612ee0 2023-07-18T03:28:46.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.288+0000 7f3280f9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b17d6076e0 0x55b17d611f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:46.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.288+0000 7f3280f9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b17d6076e0 0x55b17d611f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:32790/0 (socket says 172.21.15.101:32790) 2023-07-18T03:28:46.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.288+0000 7f328179c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b17d612ee0 0x55b17d60c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:46.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.288+0000 7f3280f9b700 1 -- 172.21.15.101:0/2031691478 learned_addr learned my addr 172.21.15.101:0/2031691478 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:28:46.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.288+0000 7f327bfff700 1 --2- 172.21.15.101:0/2031691478 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b17d607ff0 0x55b17d606f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:46.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.288+0000 7f328179c700 1 -- 172.21.15.101:0/2031691478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b17d6076e0 msgr2=0x55b17d611f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:28:46.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.288+0000 7f328179c700 1 --2- 172.21.15.101:0/2031691478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b17d6076e0 0x55b17d611f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:46.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.288+0000 7f328179c700 1 -- 172.21.15.101:0/2031691478 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b17d607ff0 msgr2=0x55b17d606f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:28:46.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.288+0000 7f328179c700 1 --2- 172.21.15.101:0/2031691478 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b17d607ff0 0x55b17d606f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:46.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.288+0000 7f328179c700 1 -- 172.21.15.101:0/2031691478 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b17d3aa250 con 0x55b17d612ee0 2023-07-18T03:28:46.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.288+0000 7f3280f9b700 1 --2- 172.21.15.101:0/2031691478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b17d6076e0 0x55b17d611f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:28:46.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.288+0000 7f327bfff700 1 --2- 172.21.15.101:0/2031691478 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b17d607ff0 0x55b17d606f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:28:46.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.288+0000 7f328179c700 1 --2- 172.21.15.101:0/2031691478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b17d612ee0 0x55b17d60c6c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f3274019da0 tx=0x7f327400fcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8791455e86b40fc server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:28:46.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.289+0000 7f32677fe700 1 -- 172.21.15.101:0/2031691478 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327400e030 con 0x55b17d612ee0 2023-07-18T03:28:46.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.289+0000 7f32677fe700 1 -- 172.21.15.101:0/2031691478 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f327400f040 con 0x55b17d612ee0 2023-07-18T03:28:46.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.289+0000 7f32677fe700 1 -- 172.21.15.101:0/2031691478 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3274027070 con 0x55b17d612ee0 2023-07-18T03:28:46.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.290+0000 7f3296be6b80 1 -- 172.21.15.101:0/2031691478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b17d612ee0 msgr2=0x55b17d60c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:46.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.290+0000 7f3296be6b80 1 --2- 172.21.15.101:0/2031691478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b17d612ee0 0x55b17d60c6c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f3274019da0 tx=0x7f327400fcb0 comp rx=0 tx=0).stop 2023-07-18T03:28:46.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.290+0000 7f3296be6b80 1 -- 172.21.15.101:0/2031691478 shutdown_connections 2023-07-18T03:28:46.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.290+0000 7f3296be6b80 1 --2- 172.21.15.101:0/2031691478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b17d6076e0 0x55b17d611f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:46.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.290+0000 7f3296be6b80 1 --2- 172.21.15.101:0/2031691478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b17d612ee0 0x55b17d60c6c0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:46.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.290+0000 7f3296be6b80 1 --2- 172.21.15.101:0/2031691478 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b17d607ff0 0x55b17d606f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:46.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.290+0000 7f3296be6b80 1 -- 172.21.15.101:0/2031691478 >> 172.21.15.101:0/2031691478 conn(0x55b17d5f9da0 msgr2=0x55b17d606820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:28:46.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.290+0000 7f3296be6b80 1 -- 172.21.15.101:0/2031691478 shutdown_connections 2023-07-18T03:28:46.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.290+0000 7f3296be6b80 1 -- 172.21.15.101:0/2031691478 wait complete. 2023-07-18T03:28:46.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.290+0000 7f3296be6b80 1 -- start start 2023-07-18T03:28:46.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.291+0000 7f3296be6b80 1 Processor -- start 2023-07-18T03:28:46.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.291+0000 7f3296be6b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b17d6076e0 0x55b17d66a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:46.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.291+0000 7f3296be6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b17d607ff0 0x55b17d66a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:46.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.291+0000 7f3296be6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b17d612ee0 0x55b17d617600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:46.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.291+0000 7f3296be6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b17d617d40 con 0x55b17d612ee0 2023-07-18T03:28:46.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.291+0000 7f3296be6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b17d617eb0 con 0x55b17d607ff0 2023-07-18T03:28:46.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.291+0000 7f3296be6b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b17d618070 con 0x55b17d6076e0 2023-07-18T03:28:46.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.292+0000 7f3296be6b80 10 client.0 _ll_register_callbacks cb 0x7f3274019770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:28:46.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.292+0000 7f3280f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b17d6076e0 0x55b17d66a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:46.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.292+0000 7f327bfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b17d607ff0 0x55b17d66a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:46.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.292+0000 7f3280f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b17d6076e0 0x55b17d66a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:47378/0 (socket says 172.21.15.101:47378) 2023-07-18T03:28:46.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.292+0000 7f3280f9b700 1 -- 172.21.15.101:0/3271551150 learned_addr learned my addr 172.21.15.101:0/3271551150 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:28:46.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.292+0000 7f328179c700 1 --2- 172.21.15.101:0/3271551150 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b17d612ee0 0x55b17d617600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:46.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.292+0000 7f3280f9b700 1 -- 172.21.15.101:0/3271551150 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b17d607ff0 msgr2=0x55b17d66a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:28:46.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.292+0000 7f3280f9b700 1 --2- 172.21.15.101:0/3271551150 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b17d607ff0 0x55b17d66a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:46.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.292+0000 7f3280f9b700 1 -- 172.21.15.101:0/3271551150 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b17d612ee0 msgr2=0x55b17d617600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:28:46.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.292+0000 7f3280f9b700 1 --2- 172.21.15.101:0/3271551150 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b17d612ee0 0x55b17d617600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:46.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.292+0000 7f3280f9b700 1 -- 172.21.15.101:0/3271551150 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b17d383de0 con 0x55b17d6076e0 2023-07-18T03:28:46.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.292+0000 7f327bfff700 1 --2- 172.21.15.101:0/3271551150 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b17d607ff0 0x55b17d66a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:28:46.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.292+0000 7f3280f9b700 1 --2- 172.21.15.101:0/3271551150 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b17d6076e0 0x55b17d66a3e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f32700994d0 tx=0x7f3270022c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=727640d9195f8761 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:28:46.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.292+0000 7f325bfff700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:28:46.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.293+0000 7f325bfff700 1 -- 172.21.15.101:0/3271551150 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32700b2040 con 0x55b17d6076e0 2023-07-18T03:28:46.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.293+0000 7f325bfff700 1 -- 172.21.15.101:0/3271551150 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f327009f1b0 con 0x55b17d6076e0 2023-07-18T03:28:46.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.293+0000 7f3296be6b80 1 -- 172.21.15.101:0/3271551150 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f327009acc0 con 0x55b17d6076e0 2023-07-18T03:28:46.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.293+0000 7f325bfff700 1 -- 172.21.15.101:0/3271551150 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32700a3540 con 0x55b17d6076e0 2023-07-18T03:28:46.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.293+0000 7f3296be6b80 10 client.29062 fetch_fsmap learned FSMap version 49 2023-07-18T03:28:46.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.293+0000 7f3296be6b80 1 -- 172.21.15.101:0/3271551150 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55b17d36cc60 con 0x55b17d6076e0 2023-07-18T03:28:46.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.293+0000 7f325bfff700 1 -- 172.21.15.101:0/3271551150 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f327009acc0 con 0x55b17d6076e0 2023-07-18T03:28:46.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.293+0000 7f325bfff700 1 -- 172.21.15.101:0/3271551150 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 49) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f32700cb2b0 con 0x55b17d6076e0 2023-07-18T03:28:46.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.293+0000 7f3296be6b80 10 client.29062 fetch_fsmap finished waiting for FSMap version 49 2023-07-18T03:28:46.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.293+0000 7f3296be6b80 10 client.29062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:28:46.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.293+0000 7f3296be6b80 20 client.29062 populate_metadata read hostname 'smithi101' 2023-07-18T03:28:46.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.293+0000 7f3296be6b80 1 --2- 172.21.15.101:0/3271551150 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b17d673040 0x55b17d675450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:46.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.293+0000 7f3296be6b80 4 client.29062 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:28:46.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.293+0000 7f3296be6b80 1 -- 172.21.15.101:0/3271551150 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b17d511050 con 0x55b17d673040 2023-07-18T03:28:46.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.293+0000 7f327bfff700 1 --2- 172.21.15.101:0/3271551150 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b17d673040 0x55b17d675450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:46.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.294+0000 7f325bfff700 10 client.29062 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:28:46.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.294+0000 7f327bfff700 1 --2- 172.21.15.101:0/3271551150 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b17d673040 0x55b17d675450 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f92ca8f3e186b35 server_cookie=fbb0bc9eb19fb745 in_seq=0 out_seq=0 2023-07-18T03:28:46.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.294+0000 7f325bfff700 1 -- 172.21.15.101:0/3271551150 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55b17d511050 con 0x55b17d673040 2023-07-18T03:28:46.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.294+0000 7f325bfff700 10 client.29062 handle_command_reply: tid=0 2023-07-18T03:28:46.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.294+0000 7f3296be6b80 1 client.29062 shutdown 2023-07-18T03:28:46.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.294+0000 7f3296be6b80 1 -- 172.21.15.101:0/3271551150 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b17d6076e0 msgr2=0x55b17d66a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:46.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.294+0000 7f3296be6b80 1 --2- 172.21.15.101:0/3271551150 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b17d6076e0 0x55b17d66a3e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f32700994d0 tx=0x7f3270022c90 comp rx=0 tx=0).stop 2023-07-18T03:28:46.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.294+0000 7f3296be6b80 1 -- 172.21.15.101:0/3271551150 shutdown_connections 2023-07-18T03:28:46.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.294+0000 7f3296be6b80 1 --2- 172.21.15.101:0/3271551150 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b17d673040 0x55b17d675450 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:46.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.294+0000 7f3296be6b80 1 --2- 172.21.15.101:0/3271551150 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b17d612ee0 0x55b17d617600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:46.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.294+0000 7f3296be6b80 1 --2- 172.21.15.101:0/3271551150 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b17d6076e0 0x55b17d66a3e0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:46.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.294+0000 7f3296be6b80 1 --2- 172.21.15.101:0/3271551150 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b17d607ff0 0x55b17d66a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:46.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.294+0000 7f325bfff700 0 client.29062 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:28:46.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.294+0000 7f3296be6b80 1 -- 172.21.15.101:0/3271551150 >> 172.21.15.101:0/3271551150 conn(0x55b17d5f9da0 msgr2=0x55b17d611d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:28:46.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.295+0000 7f3296be6b80 1 -- 172.21.15.101:0/3271551150 shutdown_connections 2023-07-18T03:28:46.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.295+0000 7f3296be6b80 1 -- 172.21.15.101:0/3271551150 wait complete. 2023-07-18T03:28:46.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.295+0000 7f3296be6b80 20 client.29062 trim_cache size 0 max 16384 2023-07-18T03:28:46.306 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:28:46.306 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:28:46.307 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:28:46.307 INFO:teuthology.orchestra.run.smithi101.stdout: "0f2f19e0-0429-465c-935e-223eb09490c3": { 2023-07-18T03:28:46.307 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:28:46.307 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "0f2f19e0-0429-465c-935e-223eb09490c3", 2023-07-18T03:28:46.308 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:28:46.308 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:28:46.308 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:28:46.308 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:28:46.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.295+0000 7f32831ff700 1 -- 172.21.15.101:0/2201798562 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f326c0a12c0 msgr2=0x7f326c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:46.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.295+0000 7f32831ff700 1 --2- 172.21.15.101:0/2201798562 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f326c0a12c0 0x7f326c0a3770 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f327c19c6d0 tx=0x7f3274009920 comp rx=0 tx=0).stop 2023-07-18T03:28:46.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.295+0000 7f32831ff700 1 -- 172.21.15.101:0/2201798562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f327c0ffae0 msgr2=0x7f327c19b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:46.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.295+0000 7f32831ff700 1 --2- 172.21.15.101:0/2201798562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f327c0ffae0 0x7f327c19b9d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f3270005910 tx=0x7f3270005720 comp rx=0 tx=0).stop 2023-07-18T03:28:46.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.295+0000 7f32831ff700 1 -- 172.21.15.101:0/2201798562 shutdown_connections 2023-07-18T03:28:46.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.295+0000 7f32831ff700 1 --2- 172.21.15.101:0/2201798562 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f326c0a12c0 0x7f326c0a3770 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:46.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.295+0000 7f32831ff700 1 --2- 172.21.15.101:0/2201798562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f327c0ffae0 0x7f327c19b9d0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:46.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.295+0000 7f32831ff700 1 --2- 172.21.15.101:0/2201798562 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f327c105af0 0x7f327c195a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:46.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.295+0000 7f32831ff700 1 --2- 172.21.15.101:0/2201798562 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f327c100490 0x7f327c19bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:46.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.295+0000 7f32831ff700 1 -- 172.21.15.101:0/2201798562 >> 172.21.15.101:0/2201798562 conn(0x7f327c0fb5e0 msgr2=0x7f327c0fbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:28:46.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.296+0000 7f32831ff700 1 -- 172.21.15.101:0/2201798562 shutdown_connections 2023-07-18T03:28:46.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:28:46.296+0000 7f32831ff700 1 -- 172.21.15.101:0/2201798562 wait complete. 2023-07-18T03:28:46.312 INFO:tasks.cephfs.filesystem:scrub status for tag:0f2f19e0-0429-465c-935e-223eb09490c3 - {'path': '/', 'tag': '0f2f19e0-0429-465c-935e-223eb09490c3', 'options': 'recursive,force'} 2023-07-18T03:28:47.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:47 smithi155 ceph-mon[128885]: pgmap v669: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:28:47.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:47 smithi155 ceph-mon[128885]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:28:47.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:47 smithi179 ceph-mon[129229]: pgmap v669: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:28:47.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:47 smithi179 ceph-mon[129229]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:28:47.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:47 smithi101 ceph-mon[126379]: pgmap v669: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:28:47.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:47 smithi101 ceph-mon[126379]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-07-18T03:28:49.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:49 smithi155 ceph-mon[128885]: pgmap v670: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:28:49.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:49 smithi179 ceph-mon[129229]: pgmap v670: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:28:49.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:49 smithi101 ceph-mon[126379]: pgmap v670: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:28:51.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:51 smithi155 ceph-mon[128885]: pgmap v671: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2023-07-18T03:28:51.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:51 smithi179 ceph-mon[129229]: pgmap v671: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2023-07-18T03:28:51.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:51 smithi101 ceph-mon[126379]: pgmap v671: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2023-07-18T03:28:53.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:53 smithi179 ceph-mon[129229]: pgmap v672: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2023-07-18T03:28:53.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:53 smithi101 ceph-mon[126379]: pgmap v672: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2023-07-18T03:28:53.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:53 smithi155 ceph-mon[128885]: pgmap v672: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2023-07-18T03:28:55.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:55 smithi179 ceph-mon[129229]: pgmap v673: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-07-18T03:28:55.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:55 smithi101 ceph-mon[126379]: pgmap v673: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-07-18T03:28:55.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:55 smithi155 ceph-mon[128885]: pgmap v673: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-07-18T03:28:57.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:57 smithi179 ceph-mon[129229]: pgmap v674: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:28:57.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:57 smithi101 ceph-mon[126379]: pgmap v674: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:28:57.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:57 smithi155 ceph-mon[128885]: pgmap v674: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:28:59.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:28:59 smithi179 ceph-mon[129229]: pgmap v675: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-07-18T03:28:59.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:28:59 smithi101 ceph-mon[126379]: pgmap v675: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-07-18T03:28:59.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:28:59 smithi155 ceph-mon[128885]: pgmap v675: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-07-18T03:29:01.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:29:01 smithi179 ceph-mon[129229]: pgmap v676: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2023-07-18T03:29:01.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:29:01 smithi101 ceph-mon[126379]: pgmap v676: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2023-07-18T03:29:01.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:29:01 smithi155 ceph-mon[128885]: pgmap v676: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2023-07-18T03:29:03.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:29:03 smithi179 ceph-mon[129229]: pgmap v677: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 33 KiB/s wr, 19 op/s 2023-07-18T03:29:03.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:29:03 smithi101 ceph-mon[126379]: pgmap v677: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 33 KiB/s wr, 19 op/s 2023-07-18T03:29:03.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:29:03 smithi155 ceph-mon[128885]: pgmap v677: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 33 KiB/s wr, 19 op/s 2023-07-18T03:29:05.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:29:05 smithi179 ceph-mon[129229]: pgmap v678: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 33 KiB/s wr, 18 op/s 2023-07-18T03:29:05.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:29:05 smithi101 ceph-mon[126379]: pgmap v678: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 33 KiB/s wr, 18 op/s 2023-07-18T03:29:05.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:29:05 smithi155 ceph-mon[128885]: pgmap v678: 129 pgs: 129 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 33 KiB/s wr, 18 op/s 2023-07-18T03:29:07.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:29:07 smithi179 ceph-mon[129229]: pgmap v679: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-07-18T03:29:07.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:29:07 smithi179 ceph-mon[129229]: osdmap e123: 12 total, 12 up, 12 in 2023-07-18T03:29:07.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:29:07 smithi101 ceph-mon[126379]: pgmap v679: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-07-18T03:29:07.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:29:07 smithi101 ceph-mon[126379]: osdmap e123: 12 total, 12 up, 12 in 2023-07-18T03:29:07.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:29:07 smithi155 ceph-mon[128885]: pgmap v679: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-07-18T03:29:07.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:29:07 smithi155 ceph-mon[128885]: osdmap e123: 12 total, 12 up, 12 in 2023-07-18T03:29:09.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:29:09 smithi179 ceph-mon[129229]: pgmap v681: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-07-18T03:29:09.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:29:09 smithi101 ceph-mon[126379]: pgmap v681: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-07-18T03:29:09.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:29:09 smithi155 ceph-mon[128885]: pgmap v681: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-07-18T03:29:11.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:29:11 smithi179 ceph-mon[129229]: pgmap v682: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-07-18T03:29:11.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:29:11 smithi101 ceph-mon[126379]: pgmap v682: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-07-18T03:29:11.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:29:11 smithi155 ceph-mon[128885]: pgmap v682: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-07-18T03:29:13.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:29:13 smithi179 ceph-mon[129229]: pgmap v683: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 14 KiB/s wr, 7 op/s 2023-07-18T03:29:13.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:29:13 smithi101 ceph-mon[126379]: pgmap v683: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 14 KiB/s wr, 7 op/s 2023-07-18T03:29:13.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:29:13 smithi155 ceph-mon[128885]: pgmap v683: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 14 KiB/s wr, 7 op/s 2023-07-18T03:29:15.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:29:15 smithi155 ceph-mon[128885]: pgmap v684: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 14 KiB/s wr, 7 op/s 2023-07-18T03:29:15.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:29:15 smithi155 ceph-mon[128885]: osdmap e124: 12 total, 12 up, 12 in 2023-07-18T03:29:15.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:29:15 smithi179 ceph-mon[129229]: pgmap v684: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 14 KiB/s wr, 7 op/s 2023-07-18T03:29:15.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:29:15 smithi179 ceph-mon[129229]: osdmap e124: 12 total, 12 up, 12 in 2023-07-18T03:29:15.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:29:15 smithi101 ceph-mon[126379]: pgmap v684: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 14 KiB/s wr, 7 op/s 2023-07-18T03:29:15.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:29:15 smithi101 ceph-mon[126379]: osdmap e124: 12 total, 12 up, 12 in 2023-07-18T03:29:16.314 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:29:16.524 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.527+0000 7f7766bff700 1 -- 172.21.15.101:0/2985466275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7760100490 msgr2=0x7f776010c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:16.564 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.527+0000 7f7766bff700 1 --2- 172.21.15.101:0/2985466275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7760100490 0x7f776010c920 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7758009f90 tx=0x7f775800b330 comp rx=0 tx=0).stop 2023-07-18T03:29:16.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.527+0000 7f7766bff700 1 -- 172.21.15.101:0/2985466275 shutdown_connections 2023-07-18T03:29:16.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.527+0000 7f7766bff700 1 --2- 172.21.15.101:0/2985466275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7760100490 0x7f776010c920 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:16.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.527+0000 7f7766bff700 1 --2- 172.21.15.101:0/2985466275 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7760105af0 0x7f7760105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:16.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.527+0000 7f7766bff700 1 --2- 172.21.15.101:0/2985466275 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f77600ffae0 0x7f77600fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:16.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.527+0000 7f7766bff700 1 -- 172.21.15.101:0/2985466275 >> 172.21.15.101:0/2985466275 conn(0x7f77600fb5e0 msgr2=0x7f77600fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:16.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.527+0000 7f7766bff700 1 -- 172.21.15.101:0/2985466275 shutdown_connections 2023-07-18T03:29:16.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.527+0000 7f7766bff700 1 -- 172.21.15.101:0/2985466275 wait complete. 2023-07-18T03:29:16.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.528+0000 7f7766bff700 1 Processor -- start 2023-07-18T03:29:16.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.528+0000 7f7766bff700 1 -- start start 2023-07-18T03:29:16.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.528+0000 7f7766bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f77600ffae0 0x7f776019bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:16.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.528+0000 7f7766bff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7760100490 0x7f776019c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:16.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.528+0000 7f7766bff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7760105af0 0x7f7760195b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:16.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.528+0000 7f7766bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77601961b0 con 0x7f77600ffae0 2023-07-18T03:29:16.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.528+0000 7f7766bff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7760196320 con 0x7f7760105af0 2023-07-18T03:29:16.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.528+0000 7f7766bff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77601964e0 con 0x7f7760100490 2023-07-18T03:29:16.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.529+0000 7f775ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7760100490 0x7f776019c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:16.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.529+0000 7f776499b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f77600ffae0 0x7f776019bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:16.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.529+0000 7f776519c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7760105af0 0x7f7760195b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:16.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.529+0000 7f775ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7760100490 0x7f776019c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:35130/0 (socket says 172.21.15.101:35130) 2023-07-18T03:29:16.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.529+0000 7f775ffff700 1 -- 172.21.15.101:0/2817760820 learned_addr learned my addr 172.21.15.101:0/2817760820 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:29:16.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.529+0000 7f776519c700 1 -- 172.21.15.101:0/2817760820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7760100490 msgr2=0x7f776019c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:16.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.529+0000 7f776519c700 1 --2- 172.21.15.101:0/2817760820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7760100490 0x7f776019c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:16.581 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.529+0000 7f776519c700 1 -- 172.21.15.101:0/2817760820 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f77600ffae0 msgr2=0x7f776019bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:16.581 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.529+0000 7f776519c700 1 --2- 172.21.15.101:0/2817760820 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f77600ffae0 0x7f776019bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:16.581 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.529+0000 7f776519c700 1 -- 172.21.15.101:0/2817760820 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7758009c40 con 0x7f7760105af0 2023-07-18T03:29:16.581 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.529+0000 7f775ffff700 1 --2- 172.21.15.101:0/2817760820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7760100490 0x7f776019c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:29:16.581 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.529+0000 7f776499b700 1 --2- 172.21.15.101:0/2817760820 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f77600ffae0 0x7f776019bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:29:16.581 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.529+0000 7f776519c700 1 --2- 172.21.15.101:0/2817760820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7760105af0 0x7f7760195b40 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f775800b6c0 tx=0x7f77580094b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:16.582 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.529+0000 7f775dffb700 1 -- 172.21.15.101:0/2817760820 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775800e040 con 0x7f7760105af0 2023-07-18T03:29:16.582 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.529+0000 7f7766bff700 1 -- 172.21.15.101:0/2817760820 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7760196760 con 0x7f7760105af0 2023-07-18T03:29:16.582 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.529+0000 7f775dffb700 1 -- 172.21.15.101:0/2817760820 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f775801d470 con 0x7f7760105af0 2023-07-18T03:29:16.582 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.529+0000 7f775dffb700 1 -- 172.21.15.101:0/2817760820 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77580052a0 con 0x7f7760105af0 2023-07-18T03:29:16.582 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.530+0000 7f7766bff700 1 -- 172.21.15.101:0/2817760820 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77601a2ac0 con 0x7f7760105af0 2023-07-18T03:29:16.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.533+0000 7f775dffb700 1 -- 172.21.15.101:0/2817760820 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f775801d5e0 con 0x7f7760105af0 2023-07-18T03:29:16.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.534+0000 7f777a510b80 1 Processor -- start 2023-07-18T03:29:16.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.534+0000 7f777a510b80 1 -- start start 2023-07-18T03:29:16.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.534+0000 7f777a510b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbf72802d0 0x55cbf72806a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:16.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.534+0000 7f777a510b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbf7280be0 0x55cbf728b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:16.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.534+0000 7f775dffb700 1 --2- 172.21.15.101:0/2817760820 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f77500a12c0 0x7f77500a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:16.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.534+0000 7f777a510b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cbf728bad0 0x55cbf728deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:16.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.534+0000 7f777a510b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbf7186cb0 con 0x55cbf72802d0 2023-07-18T03:29:16.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.534+0000 7f777a510b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbf71d9a40 con 0x55cbf7280be0 2023-07-18T03:29:16.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.534+0000 7f777a510b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbf71d2ed0 con 0x55cbf728bad0 2023-07-18T03:29:16.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.534+0000 7f775dffb700 1 -- 172.21.15.101:0/2817760820 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f77580ce530 con 0x7f7760105af0 2023-07-18T03:29:16.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.534+0000 7f776499b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbf7280be0 0x55cbf728b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:16.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.534+0000 7f776519c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cbf728bad0 0x55cbf728deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:16.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.534+0000 7f776499b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbf7280be0 0x55cbf728b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:46876/0 (socket says 172.21.15.101:46876) 2023-07-18T03:29:16.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.535+0000 7f776499b700 1 -- 172.21.15.101:0/2325767857 learned_addr learned my addr 172.21.15.101:0/2325767857 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:29:16.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.535+0000 7f775ffff700 1 --2- 172.21.15.101:0/2325767857 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbf72802d0 0x55cbf72806a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:16.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.535+0000 7f775ffff700 1 --2- 172.21.15.101:0/2817760820 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f77500a12c0 0x7f77500a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:16.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.535+0000 7f776519c700 1 -- 172.21.15.101:0/2325767857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbf7280be0 msgr2=0x55cbf728b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:29:16.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.535+0000 7f776519c700 1 --2- 172.21.15.101:0/2325767857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbf7280be0 0x55cbf728b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:16.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.535+0000 7f776519c700 1 -- 172.21.15.101:0/2325767857 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbf72802d0 msgr2=0x55cbf72806a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:29:16.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.535+0000 7f776519c700 1 --2- 172.21.15.101:0/2325767857 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbf72802d0 0x55cbf72806a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:16.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.535+0000 7f776519c700 1 -- 172.21.15.101:0/2325767857 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbf7000de0 con 0x55cbf728bad0 2023-07-18T03:29:16.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.535+0000 7f776519c700 1 --2- 172.21.15.101:0/2325767857 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cbf728bad0 0x55cbf728deb0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f77580cb8d0 tx=0x7f7758009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20155674316a87b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:16.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.535+0000 7f774affd700 1 -- 172.21.15.101:0/2325767857 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77580a80b0 con 0x55cbf728bad0 2023-07-18T03:29:16.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.535+0000 7f775ffff700 1 --2- 172.21.15.101:0/2817760820 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f77500a12c0 0x7f77500a3770 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f774c005ae0 tx=0x7f774c009310 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:16.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.535+0000 7f774affd700 1 -- 172.21.15.101:0/2325767857 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77580ad930 con 0x55cbf728bad0 2023-07-18T03:29:16.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.535+0000 7f774affd700 1 -- 172.21.15.101:0/2325767857 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775801d8c0 con 0x55cbf728bad0 2023-07-18T03:29:16.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.536+0000 7f777a510b80 1 -- 172.21.15.101:0/2325767857 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cbf728bad0 msgr2=0x55cbf728deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:16.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.536+0000 7f777a510b80 1 --2- 172.21.15.101:0/2325767857 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cbf728bad0 0x55cbf728deb0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f77580cb8d0 tx=0x7f7758009920 comp rx=0 tx=0).stop 2023-07-18T03:29:16.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.536+0000 7f777a510b80 1 -- 172.21.15.101:0/2325767857 shutdown_connections 2023-07-18T03:29:16.589 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.536+0000 7f777a510b80 1 --2- 172.21.15.101:0/2325767857 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cbf728bad0 0x55cbf728deb0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:16.589 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.536+0000 7f777a510b80 1 --2- 172.21.15.101:0/2325767857 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbf72802d0 0x55cbf72806a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:16.589 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.536+0000 7f777a510b80 1 --2- 172.21.15.101:0/2325767857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbf7280be0 0x55cbf728b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:16.589 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.536+0000 7f777a510b80 1 -- 172.21.15.101:0/2325767857 >> 172.21.15.101:0/2325767857 conn(0x55cbf7276da0 msgr2=0x55cbf727c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:16.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.536+0000 7f777a510b80 1 -- 172.21.15.101:0/2325767857 shutdown_connections 2023-07-18T03:29:16.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.536+0000 7f777a510b80 1 -- 172.21.15.101:0/2325767857 wait complete. 2023-07-18T03:29:16.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.537+0000 7f777a510b80 1 -- start start 2023-07-18T03:29:16.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.537+0000 7f777a510b80 1 Processor -- start 2023-07-18T03:29:16.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.537+0000 7f777a510b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbf72802d0 0x55cbf728bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:16.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.537+0000 7f777a510b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cbf7280be0 0x55cbf728c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:16.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.537+0000 7f777a510b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbf728bad0 0x55cbf728c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:16.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.537+0000 7f777a510b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbf7172ac0 con 0x55cbf728bad0 2023-07-18T03:29:16.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.537+0000 7f777a510b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbf71727c0 con 0x55cbf72802d0 2023-07-18T03:29:16.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.537+0000 7f777a510b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbf7056920 con 0x55cbf7280be0 2023-07-18T03:29:16.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.537+0000 7f777a510b80 10 client.0 _ll_register_callbacks cb 0x55cbf7170760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:29:16.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.538+0000 7f776499b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbf72802d0 0x55cbf728bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:16.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.538+0000 7f776499b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbf72802d0 0x55cbf728bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:46888/0 (socket says 172.21.15.101:46888) 2023-07-18T03:29:16.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.538+0000 7f776499b700 1 -- 172.21.15.101:0/1444227853 learned_addr learned my addr 172.21.15.101:0/1444227853 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:29:16.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.538+0000 7f775ffff700 1 --2- 172.21.15.101:0/1444227853 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cbf7280be0 0x55cbf728c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:16.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.538+0000 7f776519c700 1 --2- 172.21.15.101:0/1444227853 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbf728bad0 0x55cbf728c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:16.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.538+0000 7f776499b700 1 -- 172.21.15.101:0/1444227853 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cbf7280be0 msgr2=0x55cbf728c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:29:16.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.538+0000 7f776499b700 1 --2- 172.21.15.101:0/1444227853 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cbf7280be0 0x55cbf728c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:16.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.538+0000 7f776499b700 1 -- 172.21.15.101:0/1444227853 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbf728bad0 msgr2=0x55cbf728c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:29:16.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.538+0000 7f776499b700 1 --2- 172.21.15.101:0/1444227853 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbf728bad0 0x55cbf728c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:16.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.538+0000 7f776499b700 1 -- 172.21.15.101:0/1444227853 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77601a2ac0 con 0x55cbf72802d0 2023-07-18T03:29:16.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.538+0000 7f776499b700 1 --2- 172.21.15.101:0/1444227853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbf72802d0 0x55cbf728bea0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f775400e630 tx=0x7f775400e940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89131561e377ac7a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:16.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.538+0000 7f773ffff700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:29:16.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.538+0000 7f773ffff700 1 -- 172.21.15.101:0/1444227853 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775401d3f0 con 0x55cbf72802d0 2023-07-18T03:29:16.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.538+0000 7f777a510b80 1 -- 172.21.15.101:0/1444227853 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cbf712c4f0 con 0x55cbf72802d0 2023-07-18T03:29:16.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.538+0000 7f773ffff700 1 -- 172.21.15.101:0/1444227853 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f775401d550 con 0x55cbf72802d0 2023-07-18T03:29:16.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.539+0000 7f773ffff700 1 -- 172.21.15.101:0/1444227853 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f775401d750 con 0x55cbf72802d0 2023-07-18T03:29:16.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.539+0000 7f777a510b80 10 client.29084 fetch_fsmap learned FSMap version 49 2023-07-18T03:29:16.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.539+0000 7f777a510b80 1 -- 172.21.15.101:0/1444227853 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55cbf6fe9c60 con 0x55cbf72802d0 2023-07-18T03:29:16.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.539+0000 7f773ffff700 1 -- 172.21.15.101:0/1444227853 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cbf712c4f0 con 0x55cbf72802d0 2023-07-18T03:29:16.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.539+0000 7f773ffff700 1 -- 172.21.15.101:0/1444227853 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 49) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f775401c070 con 0x55cbf72802d0 2023-07-18T03:29:16.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.539+0000 7f777a510b80 10 client.29084 fetch_fsmap finished waiting for FSMap version 49 2023-07-18T03:29:16.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.539+0000 7f777a510b80 10 client.29084 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:29:16.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.539+0000 7f777a510b80 20 client.29084 populate_metadata read hostname 'smithi101' 2023-07-18T03:29:16.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.539+0000 7f777a510b80 1 --2- 172.21.15.101:0/1444227853 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55cbf729b2a0 0x55cbf729d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:16.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.539+0000 7f777a510b80 4 client.29084 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:29:16.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.539+0000 7f777a510b80 1 -- 172.21.15.101:0/1444227853 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cbf71cbeb0 con 0x55cbf729b2a0 2023-07-18T03:29:16.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.539+0000 7f775ffff700 1 --2- 172.21.15.101:0/1444227853 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55cbf729b2a0 0x55cbf729d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:16.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.540+0000 7f775ffff700 1 --2- 172.21.15.101:0/1444227853 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55cbf729b2a0 0x55cbf729d6b0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6a7e2ee75ab111d server_cookie=5f53a5785bdbe6b6 in_seq=0 out_seq=0 2023-07-18T03:29:16.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.540+0000 7f773ffff700 10 client.29084 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:29:16.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.540+0000 7f773ffff700 1 -- 172.21.15.101:0/1444227853 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cbf71cbeb0 con 0x55cbf729b2a0 2023-07-18T03:29:16.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.541+0000 7f773ffff700 10 client.29084 handle_command_reply: tid=0 2023-07-18T03:29:16.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.541+0000 7f777a510b80 1 client.29084 shutdown 2023-07-18T03:29:16.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.541+0000 7f777a510b80 1 -- 172.21.15.101:0/1444227853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbf72802d0 msgr2=0x55cbf728bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:16.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.541+0000 7f777a510b80 1 --2- 172.21.15.101:0/1444227853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbf72802d0 0x55cbf728bea0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f775400e630 tx=0x7f775400e940 comp rx=0 tx=0).stop 2023-07-18T03:29:16.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.541+0000 7f777a510b80 1 -- 172.21.15.101:0/1444227853 shutdown_connections 2023-07-18T03:29:16.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.541+0000 7f777a510b80 1 --2- 172.21.15.101:0/1444227853 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55cbf729b2a0 0x55cbf729d6b0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:16.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.541+0000 7f777a510b80 1 --2- 172.21.15.101:0/1444227853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbf72802d0 0x55cbf728bea0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:16.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.541+0000 7f773ffff700 0 client.29084 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:29:16.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.541+0000 7f777a510b80 1 --2- 172.21.15.101:0/1444227853 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbf728bad0 0x55cbf728c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:16.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.541+0000 7f777a510b80 1 --2- 172.21.15.101:0/1444227853 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cbf7280be0 0x55cbf728c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:16.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.541+0000 7f777a510b80 1 -- 172.21.15.101:0/1444227853 >> 172.21.15.101:0/1444227853 conn(0x55cbf7276da0 msgr2=0x55cbf727c230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:16.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.541+0000 7f777a510b80 1 -- 172.21.15.101:0/1444227853 shutdown_connections 2023-07-18T03:29:16.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.541+0000 7f777a510b80 1 -- 172.21.15.101:0/1444227853 wait complete. 2023-07-18T03:29:16.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.541+0000 7f777a510b80 20 client.29084 trim_cache size 0 max 16384 2023-07-18T03:29:16.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.558+0000 7f777a510b80 1 Processor -- start 2023-07-18T03:29:16.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.558+0000 7f777a510b80 1 -- start start 2023-07-18T03:29:16.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.558+0000 7f777a510b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbf72802d0 0x55cbf7290ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:16.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.558+0000 7f777a510b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbf7280be0 0x55cbf728c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:16.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.558+0000 7f777a510b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cbf728bad0 0x55cbf7296660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:16.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.558+0000 7f777a510b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbf7276970 con 0x55cbf72802d0 2023-07-18T03:29:16.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.558+0000 7f777a510b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbf728aff0 con 0x55cbf7280be0 2023-07-18T03:29:16.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.558+0000 7f777a510b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbf728b1f0 con 0x55cbf728bad0 2023-07-18T03:29:16.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.559+0000 7f775ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbf7280be0 0x55cbf728c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:16.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.559+0000 7f776499b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbf72802d0 0x55cbf7290ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:16.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.559+0000 7f775ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbf7280be0 0x55cbf728c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:46900/0 (socket says 172.21.15.101:46900) 2023-07-18T03:29:16.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.559+0000 7f775ffff700 1 -- 172.21.15.101:0/4079937247 learned_addr learned my addr 172.21.15.101:0/4079937247 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:29:16.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.559+0000 7f776519c700 1 --2- 172.21.15.101:0/4079937247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cbf728bad0 0x55cbf7296660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:16.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.559+0000 7f775ffff700 1 -- 172.21.15.101:0/4079937247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cbf728bad0 msgr2=0x55cbf7296660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:29:16.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.559+0000 7f775ffff700 1 --2- 172.21.15.101:0/4079937247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cbf728bad0 0x55cbf7296660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:16.604 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.559+0000 7f775ffff700 1 -- 172.21.15.101:0/4079937247 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbf72802d0 msgr2=0x55cbf7290ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:29:16.604 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.559+0000 7f775ffff700 1 --2- 172.21.15.101:0/4079937247 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbf72802d0 0x55cbf7290ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:16.604 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.559+0000 7f775ffff700 1 -- 172.21.15.101:0/4079937247 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbf72785e0 con 0x55cbf7280be0 2023-07-18T03:29:16.604 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.559+0000 7f776499b700 1 --2- 172.21.15.101:0/4079937247 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbf72802d0 0x55cbf7290ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:29:16.604 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.559+0000 7f775ffff700 1 --2- 172.21.15.101:0/4079937247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbf7280be0 0x55cbf728c5b0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f77601050d0 tx=0x7f774c019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d21e1a32242f7d5 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:16.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.559+0000 7f774b7fe700 1 -- 172.21.15.101:0/4079937247 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774c025040 con 0x55cbf7280be0 2023-07-18T03:29:16.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.559+0000 7f774b7fe700 1 -- 172.21.15.101:0/4079937247 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f774c02a5b0 con 0x55cbf7280be0 2023-07-18T03:29:16.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.559+0000 7f774b7fe700 1 -- 172.21.15.101:0/4079937247 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774c0038e0 con 0x55cbf7280be0 2023-07-18T03:29:16.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.560+0000 7f777a510b80 1 -- 172.21.15.101:0/4079937247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbf7280be0 msgr2=0x55cbf728c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:16.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.560+0000 7f777a510b80 1 --2- 172.21.15.101:0/4079937247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbf7280be0 0x55cbf728c5b0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f77601050d0 tx=0x7f774c019000 comp rx=0 tx=0).stop 2023-07-18T03:29:16.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.560+0000 7f777a510b80 1 -- 172.21.15.101:0/4079937247 shutdown_connections 2023-07-18T03:29:16.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.560+0000 7f777a510b80 1 --2- 172.21.15.101:0/4079937247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cbf728bad0 0x55cbf7296660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:16.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.560+0000 7f777a510b80 1 --2- 172.21.15.101:0/4079937247 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbf72802d0 0x55cbf7290ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:16.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.560+0000 7f777a510b80 1 --2- 172.21.15.101:0/4079937247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbf7280be0 0x55cbf728c5b0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:16.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.560+0000 7f777a510b80 1 -- 172.21.15.101:0/4079937247 >> 172.21.15.101:0/4079937247 conn(0x55cbf7276da0 msgr2=0x55cbf728bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:16.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.560+0000 7f777a510b80 1 -- 172.21.15.101:0/4079937247 shutdown_connections 2023-07-18T03:29:16.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.560+0000 7f777a510b80 1 -- 172.21.15.101:0/4079937247 wait complete. 2023-07-18T03:29:16.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.560+0000 7f777a510b80 1 -- start start 2023-07-18T03:29:16.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.561+0000 7f777a510b80 1 Processor -- start 2023-07-18T03:29:16.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.561+0000 7f777a510b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbf72802d0 0x55cbf7290cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:16.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.561+0000 7f777a510b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbf7280be0 0x55cbf728cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:16.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.561+0000 7f777a510b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cbf728bad0 0x55cbf728d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:16.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.561+0000 7f777a510b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbf728da40 con 0x55cbf72802d0 2023-07-18T03:29:16.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.561+0000 7f777a510b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbf728dbb0 con 0x55cbf7280be0 2023-07-18T03:29:16.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.561+0000 7f777a510b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbf728dd70 con 0x55cbf728bad0 2023-07-18T03:29:16.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.561+0000 7f777a510b80 10 client.0 _ll_register_callbacks cb 0x55cbf7170760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:29:16.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.561+0000 7f776499b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbf72802d0 0x55cbf7290cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:16.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.561+0000 7f776499b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbf72802d0 0x55cbf7290cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:33506/0 (socket says 172.21.15.101:33506) 2023-07-18T03:29:16.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.561+0000 7f776499b700 1 -- 172.21.15.101:0/3797034880 learned_addr learned my addr 172.21.15.101:0/3797034880 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:29:16.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.561+0000 7f775ffff700 1 --2- 172.21.15.101:0/3797034880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbf7280be0 0x55cbf728cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:16.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.561+0000 7f776519c700 1 --2- 172.21.15.101:0/3797034880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cbf728bad0 0x55cbf728d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:16.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.562+0000 7f776499b700 1 -- 172.21.15.101:0/3797034880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cbf728bad0 msgr2=0x55cbf728d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:29:16.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.562+0000 7f776499b700 1 --2- 172.21.15.101:0/3797034880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cbf728bad0 0x55cbf728d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:16.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.562+0000 7f776499b700 1 -- 172.21.15.101:0/3797034880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbf7280be0 msgr2=0x55cbf728cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:29:16.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.562+0000 7f776499b700 1 --2- 172.21.15.101:0/3797034880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbf7280be0 0x55cbf728cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:16.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.562+0000 7f776499b700 1 -- 172.21.15.101:0/3797034880 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cbf6fe9c60 con 0x55cbf72802d0 2023-07-18T03:29:16.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.562+0000 7f776499b700 1 --2- 172.21.15.101:0/3797034880 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbf72802d0 0x55cbf7290cb0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f775400e630 tx=0x7f7754026660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2661fb47bbe8e3d6 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:16.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.562+0000 7f773ffff700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:29:16.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.562+0000 7f773ffff700 1 -- 172.21.15.101:0/3797034880 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7754026cf0 con 0x55cbf72802d0 2023-07-18T03:29:16.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.562+0000 7f777a510b80 1 -- 172.21.15.101:0/3797034880 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f77580ad930 con 0x55cbf72802d0 2023-07-18T03:29:16.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.562+0000 7f773ffff700 1 -- 172.21.15.101:0/3797034880 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f77540050b0 con 0x55cbf72802d0 2023-07-18T03:29:16.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.562+0000 7f773ffff700 1 -- 172.21.15.101:0/3797034880 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77540052b0 con 0x55cbf72802d0 2023-07-18T03:29:16.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.562+0000 7f777a510b80 10 client.18984 fetch_fsmap learned FSMap version 49 2023-07-18T03:29:16.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.562+0000 7f777a510b80 1 -- 172.21.15.101:0/3797034880 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f774c011400 con 0x55cbf72802d0 2023-07-18T03:29:16.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.563+0000 7f773ffff700 1 -- 172.21.15.101:0/3797034880 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77580ad930 con 0x55cbf72802d0 2023-07-18T03:29:16.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.563+0000 7f773ffff700 1 -- 172.21.15.101:0/3797034880 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 49) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f775401c070 con 0x55cbf72802d0 2023-07-18T03:29:16.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.563+0000 7f777a510b80 10 client.18984 fetch_fsmap finished waiting for FSMap version 49 2023-07-18T03:29:16.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.563+0000 7f777a510b80 10 client.18984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:29:16.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.563+0000 7f777a510b80 20 client.18984 populate_metadata read hostname 'smithi101' 2023-07-18T03:29:16.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.563+0000 7f777a510b80 1 --2- 172.21.15.101:0/3797034880 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55cbf72f04a0 0x55cbf72f28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:16.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.563+0000 7f777a510b80 4 client.18984 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:29:16.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.563+0000 7f777a510b80 1 -- 172.21.15.101:0/3797034880 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cbf718e050 con 0x55cbf72f04a0 2023-07-18T03:29:16.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.563+0000 7f775ffff700 1 --2- 172.21.15.101:0/3797034880 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55cbf72f04a0 0x55cbf72f28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:16.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.564+0000 7f775ffff700 1 --2- 172.21.15.101:0/3797034880 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55cbf72f04a0 0x55cbf72f28b0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=436055aec5b25ba6 server_cookie=733e21c09b3d0199 in_seq=0 out_seq=0 2023-07-18T03:29:16.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.564+0000 7f773ffff700 10 client.18984 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:29:16.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.564+0000 7f773ffff700 1 -- 172.21.15.101:0/3797034880 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55cbf718e050 con 0x55cbf72f04a0 2023-07-18T03:29:16.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.564+0000 7f773ffff700 10 client.18984 handle_command_reply: tid=0 2023-07-18T03:29:16.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.564+0000 7f777a510b80 1 client.18984 shutdown 2023-07-18T03:29:16.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.564+0000 7f777a510b80 1 -- 172.21.15.101:0/3797034880 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbf72802d0 msgr2=0x55cbf7290cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:16.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.564+0000 7f777a510b80 1 --2- 172.21.15.101:0/3797034880 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbf72802d0 0x55cbf7290cb0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f775400e630 tx=0x7f7754026660 comp rx=0 tx=0).stop 2023-07-18T03:29:16.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.564+0000 7f777a510b80 1 -- 172.21.15.101:0/3797034880 shutdown_connections 2023-07-18T03:29:16.616 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.564+0000 7f777a510b80 1 --2- 172.21.15.101:0/3797034880 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55cbf72f04a0 0x55cbf72f28b0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:16.616 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.564+0000 7f777a510b80 1 --2- 172.21.15.101:0/3797034880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cbf728bad0 0x55cbf728d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:16.616 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.564+0000 7f777a510b80 1 --2- 172.21.15.101:0/3797034880 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbf72802d0 0x55cbf7290cb0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:16.616 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.564+0000 7f777a510b80 1 --2- 172.21.15.101:0/3797034880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbf7280be0 0x55cbf728cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:16.616 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.564+0000 7f777a510b80 1 -- 172.21.15.101:0/3797034880 >> 172.21.15.101:0/3797034880 conn(0x55cbf7276da0 msgr2=0x55cbf7282c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:16.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.564+0000 7f773ffff700 0 client.18984 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:29:16.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.564+0000 7f777a510b80 1 -- 172.21.15.101:0/3797034880 shutdown_connections 2023-07-18T03:29:16.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.564+0000 7f777a510b80 1 -- 172.21.15.101:0/3797034880 wait complete. 2023-07-18T03:29:16.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.564+0000 7f777a510b80 20 client.18984 trim_cache size 0 max 16384 2023-07-18T03:29:16.618 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:29:16.618 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:29:16.618 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:29:16.618 INFO:teuthology.orchestra.run.smithi101.stdout: "0f2f19e0-0429-465c-935e-223eb09490c3": { 2023-07-18T03:29:16.619 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:29:16.619 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "0f2f19e0-0429-465c-935e-223eb09490c3", 2023-07-18T03:29:16.619 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:29:16.619 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:29:16.619 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:29:16.620 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:29:16.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.565+0000 7f7766bff700 1 -- 172.21.15.101:0/2817760820 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f77500a12c0 msgr2=0x7f77500a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:16.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.565+0000 7f7766bff700 1 --2- 172.21.15.101:0/2817760820 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f77500a12c0 0x7f77500a3770 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f774c005ae0 tx=0x7f774c009310 comp rx=0 tx=0).stop 2023-07-18T03:29:16.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.565+0000 7f7766bff700 1 -- 172.21.15.101:0/2817760820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7760105af0 msgr2=0x7f7760195b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:16.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.565+0000 7f7766bff700 1 --2- 172.21.15.101:0/2817760820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7760105af0 0x7f7760195b40 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f775800b6c0 tx=0x7f77580094b0 comp rx=0 tx=0).stop 2023-07-18T03:29:16.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.565+0000 7f7766bff700 1 -- 172.21.15.101:0/2817760820 shutdown_connections 2023-07-18T03:29:16.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.565+0000 7f7766bff700 1 --2- 172.21.15.101:0/2817760820 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f77500a12c0 0x7f77500a3770 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:16.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.565+0000 7f7766bff700 1 --2- 172.21.15.101:0/2817760820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7760105af0 0x7f7760195b40 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:16.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.565+0000 7f7766bff700 1 --2- 172.21.15.101:0/2817760820 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f77600ffae0 0x7f776019bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:16.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.565+0000 7f7766bff700 1 --2- 172.21.15.101:0/2817760820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7760100490 0x7f776019c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:16.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.565+0000 7f7766bff700 1 -- 172.21.15.101:0/2817760820 >> 172.21.15.101:0/2817760820 conn(0x7f77600fb5e0 msgr2=0x7f77601035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:16.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.565+0000 7f7766bff700 1 -- 172.21.15.101:0/2817760820 shutdown_connections 2023-07-18T03:29:16.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:16.565+0000 7f7766bff700 1 -- 172.21.15.101:0/2817760820 wait complete. 2023-07-18T03:29:16.623 INFO:tasks.cephfs.filesystem:scrub status for tag:0f2f19e0-0429-465c-935e-223eb09490c3 - {'path': '/', 'tag': '0f2f19e0-0429-465c-935e-223eb09490c3', 'options': 'recursive,force'} 2023-07-18T03:29:17.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:29:17 smithi155 ceph-mon[128885]: pgmap v686: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 211 B/s wr, 6 op/s 2023-07-18T03:29:17.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:29:17 smithi179 ceph-mon[129229]: pgmap v686: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 211 B/s wr, 6 op/s 2023-07-18T03:29:17.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:29:17 smithi101 ceph-mon[126379]: pgmap v686: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 211 B/s wr, 6 op/s 2023-07-18T03:29:19.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:29:19 smithi155 ceph-mon[128885]: pgmap v687: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:29:19.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:29:19 smithi179 ceph-mon[129229]: pgmap v687: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:29:19.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:29:19 smithi101 ceph-mon[126379]: pgmap v687: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:29:21.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:29:21 smithi155 ceph-mon[128885]: pgmap v688: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:29:21.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:29:21 smithi179 ceph-mon[129229]: pgmap v688: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:29:21.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:29:21 smithi101 ceph-mon[126379]: pgmap v688: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:29:22.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:29:22 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:29:22.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:29:22 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:29:22.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:29:22 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:29:23.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:29:23 smithi155 ceph-mon[128885]: pgmap v689: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:29:23.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:29:23 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:29:23.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:29:23 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:29:23.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:29:23 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:29:23.550 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:29:23 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:29:23.550 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:29:23 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:29:23.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:29:23 smithi179 ceph-mon[129229]: pgmap v689: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:29:23.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:29:23 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:29:23.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:29:23 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:29:23.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:29:23 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:29:23.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:29:23 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:29:23.705 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:29:23 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:29:23.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:29:23 smithi101 ceph-mon[126379]: pgmap v689: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:29:23.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:29:23 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:29:23.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:29:23 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:29:23.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:29:23 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:29:23.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:29:23 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:29:23.743 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:29:23 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:29:25.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:29:25 smithi155 ceph-mon[128885]: pgmap v690: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:29:25.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:29:25 smithi179 ceph-mon[129229]: pgmap v690: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:29:25.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:29:25 smithi101 ceph-mon[126379]: pgmap v690: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:29:27.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:29:27 smithi155 ceph-mon[128885]: pgmap v691: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 87 B/s wr, 7 op/s 2023-07-18T03:29:27.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:29:27 smithi179 ceph-mon[129229]: pgmap v691: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 87 B/s wr, 7 op/s 2023-07-18T03:29:27.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:29:27 smithi101 ceph-mon[126379]: pgmap v691: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 87 B/s wr, 7 op/s 2023-07-18T03:29:29.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:29:29 smithi155 ceph-mon[128885]: pgmap v692: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:29:29.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:29:29 smithi179 ceph-mon[129229]: pgmap v692: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:29:29.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:29:29 smithi101 ceph-mon[126379]: pgmap v692: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:29:31.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:29:31 smithi155 ceph-mon[128885]: pgmap v693: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-18T03:29:31.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:29:31 smithi179 ceph-mon[129229]: pgmap v693: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-18T03:29:31.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:29:31 smithi101 ceph-mon[126379]: pgmap v693: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-18T03:29:33.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:29:33 smithi155 ceph-mon[128885]: pgmap v694: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:29:33.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:29:33 smithi179 ceph-mon[129229]: pgmap v694: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:29:33.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:29:33 smithi101 ceph-mon[126379]: pgmap v694: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:29:35.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:29:35 smithi179 ceph-mon[129229]: pgmap v695: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:29:35.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:29:35 smithi101 ceph-mon[126379]: pgmap v695: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:29:35.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:29:35 smithi155 ceph-mon[128885]: pgmap v695: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:29:37.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:29:37 smithi179 ceph-mon[129229]: pgmap v696: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:29:37.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:29:37 smithi101 ceph-mon[126379]: pgmap v696: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:29:37.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:29:37 smithi155 ceph-mon[128885]: pgmap v696: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:29:39.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:29:39 smithi179 ceph-mon[129229]: pgmap v697: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:29:39.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:29:39 smithi101 ceph-mon[126379]: pgmap v697: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:29:39.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:29:39 smithi155 ceph-mon[128885]: pgmap v697: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:29:41.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:29:41 smithi179 ceph-mon[129229]: pgmap v698: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-18T03:29:41.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:29:41 smithi101 ceph-mon[126379]: pgmap v698: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-18T03:29:41.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:29:41 smithi155 ceph-mon[128885]: pgmap v698: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-18T03:29:43.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:29:43 smithi179 ceph-mon[129229]: pgmap v699: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:29:43.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:29:43 smithi101 ceph-mon[126379]: pgmap v699: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:29:43.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:29:43 smithi155 ceph-mon[128885]: pgmap v699: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:29:45.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:29:45 smithi179 ceph-mon[129229]: pgmap v700: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:29:45.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:29:45 smithi101 ceph-mon[126379]: pgmap v700: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:29:45.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:29:45 smithi155 ceph-mon[128885]: pgmap v700: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:29:46.624 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:29:46.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.839+0000 7f91249ff700 1 -- 172.21.15.101:0/3075387978 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f912010c3a0 msgr2=0x7f912010e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:46.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.839+0000 7f91249ff700 1 --2- 172.21.15.101:0/3075387978 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f912010c3a0 0x7f912010e780 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f911800b0a0 tx=0x7f911800b3b0 comp rx=0 tx=0).stop 2023-07-18T03:29:46.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.839+0000 7f91249ff700 1 -- 172.21.15.101:0/3075387978 shutdown_connections 2023-07-18T03:29:46.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.839+0000 7f91249ff700 1 --2- 172.21.15.101:0/3075387978 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f912010c3a0 0x7f912010e780 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:46.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.839+0000 7f91249ff700 1 --2- 172.21.15.101:0/3075387978 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9120072850 0x7f9120072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:46.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.839+0000 7f91249ff700 1 --2- 172.21.15.101:0/3075387978 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9120073160 0x7f912010be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:46.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.839+0000 7f91249ff700 1 -- 172.21.15.101:0/3075387978 >> 172.21.15.101:0/3075387978 conn(0x7f91200fb600 msgr2=0x7f91200fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:46.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.839+0000 7f91249ff700 1 -- 172.21.15.101:0/3075387978 shutdown_connections 2023-07-18T03:29:46.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.839+0000 7f91249ff700 1 -- 172.21.15.101:0/3075387978 wait complete. 2023-07-18T03:29:46.839 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.840+0000 7f91249ff700 1 Processor -- start 2023-07-18T03:29:46.839 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.840+0000 7f91249ff700 1 -- start start 2023-07-18T03:29:46.839 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.840+0000 7f91249ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9120072850 0x7f9120197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:46.839 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.840+0000 7f91249ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9120073160 0x7f9120198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:46.840 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.840+0000 7f91249ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f912010c3a0 0x7f912019c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:46.840 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.840+0000 7f91249ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912019cf20 con 0x7f9120073160 2023-07-18T03:29:46.840 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.840+0000 7f91249ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912019d090 con 0x7f912010c3a0 2023-07-18T03:29:46.840 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.840+0000 7f91249ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912019d250 con 0x7f9120072850 2023-07-18T03:29:46.841 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.841+0000 7f911e59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9120072850 0x7f9120197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:46.841 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.841+0000 7f911ed9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f912010c3a0 0x7f912019c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:46.841 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.841+0000 7f911e59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9120072850 0x7f9120197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:58316/0 (socket says 172.21.15.101:58316) 2023-07-18T03:29:46.841 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.841+0000 7f911ed9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f912010c3a0 0x7f912019c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:44410/0 (socket says 172.21.15.101:44410) 2023-07-18T03:29:46.842 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.841+0000 7f911e59c700 1 -- 172.21.15.101:0/886797476 learned_addr learned my addr 172.21.15.101:0/886797476 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:29:46.842 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.841+0000 7f9117fff700 1 --2- 172.21.15.101:0/886797476 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9120073160 0x7f9120198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:46.842 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.841+0000 7f911ed9d700 1 -- 172.21.15.101:0/886797476 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9120072850 msgr2=0x7f9120197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:46.842 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.841+0000 7f911ed9d700 1 --2- 172.21.15.101:0/886797476 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9120072850 0x7f9120197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:46.842 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.841+0000 7f911ed9d700 1 -- 172.21.15.101:0/886797476 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9120073160 msgr2=0x7f9120198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:46.843 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.841+0000 7f911ed9d700 1 --2- 172.21.15.101:0/886797476 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9120073160 0x7f9120198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:46.843 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.841+0000 7f911ed9d700 1 -- 172.21.15.101:0/886797476 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9118009d00 con 0x7f912010c3a0 2023-07-18T03:29:46.843 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.841+0000 7f911e59c700 1 --2- 172.21.15.101:0/886797476 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9120072850 0x7f9120197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:29:46.843 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.841+0000 7f911ed9d700 1 --2- 172.21.15.101:0/886797476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f912010c3a0 0x7f912019c8e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f911800b070 tx=0x7f9118007930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:46.843 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.842+0000 7f91177fe700 1 -- 172.21.15.101:0/886797476 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911800e040 con 0x7f912010c3a0 2023-07-18T03:29:46.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.842+0000 7f91177fe700 1 -- 172.21.15.101:0/886797476 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f911801d470 con 0x7f912010c3a0 2023-07-18T03:29:46.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.842+0000 7f91249ff700 1 -- 172.21.15.101:0/886797476 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f912019d4d0 con 0x7f912010c3a0 2023-07-18T03:29:46.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.842+0000 7f91177fe700 1 -- 172.21.15.101:0/886797476 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91180052a0 con 0x7f912010c3a0 2023-07-18T03:29:46.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.842+0000 7f91249ff700 1 -- 172.21.15.101:0/886797476 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91201a4d70 con 0x7f912010c3a0 2023-07-18T03:29:46.847 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.845+0000 7f91177fe700 1 -- 172.21.15.101:0/886797476 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f911801d5e0 con 0x7f912010c3a0 2023-07-18T03:29:46.847 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.846+0000 7f9138331b80 1 Processor -- start 2023-07-18T03:29:46.847 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.846+0000 7f91177fe700 1 --2- 172.21.15.101:0/886797476 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f910c0a12c0 0x7f910c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:46.848 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.846+0000 7f9138331b80 1 -- start start 2023-07-18T03:29:46.848 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.847+0000 7f91177fe700 1 -- 172.21.15.101:0/886797476 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f91180ce520 con 0x7f912010c3a0 2023-07-18T03:29:46.848 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.847+0000 7f9117fff700 1 --2- 172.21.15.101:0/886797476 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f910c0a12c0 0x7f910c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:46.848 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.847+0000 7f9138331b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a7497222d0 0x55a7497226a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:46.848 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.847+0000 7f9138331b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a749722be0 0x55a74972d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:46.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.847+0000 7f9138331b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a74972dad0 0x55a74972feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:46.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.847+0000 7f9138331b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a749628cb0 con 0x55a74972dad0 2023-07-18T03:29:46.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.847+0000 7f9138331b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a74967ba40 con 0x55a7497222d0 2023-07-18T03:29:46.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.847+0000 7f9138331b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a749674ed0 con 0x55a749722be0 2023-07-18T03:29:46.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.847+0000 7f911e59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a7497222d0 0x55a7497226a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:46.850 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.847+0000 7f9117fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a749722be0 0x55a74972d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:46.850 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.847+0000 7f911e59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a7497222d0 0x55a7497226a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:44412/0 (socket says 172.21.15.101:44412) 2023-07-18T03:29:46.850 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.847+0000 7f9117fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a749722be0 0x55a74972d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:58330/0 (socket says 172.21.15.101:58330) 2023-07-18T03:29:46.850 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.847+0000 7f911e59c700 1 -- 172.21.15.101:0/789182840 learned_addr learned my addr 172.21.15.101:0/789182840 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:29:46.850 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.847+0000 7f911ed9d700 1 --2- 172.21.15.101:0/789182840 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a74972dad0 0x55a74972feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:46.851 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.848+0000 7f911e59c700 1 -- 172.21.15.101:0/789182840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a749722be0 msgr2=0x55a74972d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:29:46.851 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.848+0000 7f911e59c700 1 --2- 172.21.15.101:0/789182840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a749722be0 0x55a74972d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:46.851 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.848+0000 7f911e59c700 1 -- 172.21.15.101:0/789182840 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a74972dad0 msgr2=0x55a74972feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:29:46.852 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.848+0000 7f911e59c700 1 --2- 172.21.15.101:0/789182840 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a74972dad0 0x55a74972feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:46.852 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.848+0000 7f911e59c700 1 -- 172.21.15.101:0/789182840 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7494a2de0 con 0x55a7497222d0 2023-07-18T03:29:46.852 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.848+0000 7f9117fff700 1 --2- 172.21.15.101:0/886797476 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f910c0a12c0 0x7f910c0a3770 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9120199550 tx=0x7f91080118e0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:46.852 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.848+0000 7f911e59c700 1 --2- 172.21.15.101:0/789182840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a7497222d0 0x55a7497226a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f91100117e0 tx=0x7f9110011af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=158aa5532caddc63 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:46.852 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.848+0000 7f9114ff9700 1 -- 172.21.15.101:0/789182840 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9110019410 con 0x55a7497222d0 2023-07-18T03:29:46.853 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.848+0000 7f9114ff9700 1 -- 172.21.15.101:0/789182840 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9110019a30 con 0x55a7497222d0 2023-07-18T03:29:46.853 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.848+0000 7f9114ff9700 1 -- 172.21.15.101:0/789182840 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9110018a90 con 0x55a7497222d0 2023-07-18T03:29:46.853 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.849+0000 7f9138331b80 1 -- 172.21.15.101:0/789182840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a7497222d0 msgr2=0x55a7497226a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:46.853 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.849+0000 7f9138331b80 1 --2- 172.21.15.101:0/789182840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a7497222d0 0x55a7497226a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f91100117e0 tx=0x7f9110011af0 comp rx=0 tx=0).stop 2023-07-18T03:29:46.853 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.849+0000 7f9138331b80 1 -- 172.21.15.101:0/789182840 shutdown_connections 2023-07-18T03:29:46.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.849+0000 7f9138331b80 1 --2- 172.21.15.101:0/789182840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a7497222d0 0x55a7497226a0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:46.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.849+0000 7f9138331b80 1 --2- 172.21.15.101:0/789182840 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a74972dad0 0x55a74972feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:46.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.849+0000 7f9138331b80 1 --2- 172.21.15.101:0/789182840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a749722be0 0x55a74972d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:46.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.849+0000 7f9138331b80 1 -- 172.21.15.101:0/789182840 >> 172.21.15.101:0/789182840 conn(0x55a749718da0 msgr2=0x55a74971e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:46.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.849+0000 7f9138331b80 1 -- 172.21.15.101:0/789182840 shutdown_connections 2023-07-18T03:29:46.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.849+0000 7f9138331b80 1 -- 172.21.15.101:0/789182840 wait complete. 2023-07-18T03:29:46.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.849+0000 7f9138331b80 1 -- start start 2023-07-18T03:29:46.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.850+0000 7f9138331b80 1 Processor -- start 2023-07-18T03:29:46.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.850+0000 7f9138331b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a7497222d0 0x55a74972d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:46.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.850+0000 7f9138331b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a749722be0 0x55a74972a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:46.856 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.850+0000 7f9138331b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a74972dad0 0x55a74972aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:46.856 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.850+0000 7f9138331b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a749614ac0 con 0x55a7497222d0 2023-07-18T03:29:46.856 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.850+0000 7f9138331b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7496147c0 con 0x55a74972dad0 2023-07-18T03:29:46.856 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.850+0000 7f9138331b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7494f8920 con 0x55a749722be0 2023-07-18T03:29:46.856 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.850+0000 7f9138331b80 10 client.0 _ll_register_callbacks cb 0x55a749612760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:29:46.857 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.850+0000 7f911e59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a7497222d0 0x55a74972d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:46.857 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.850+0000 7f9117fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a749722be0 0x55a74972a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:46.857 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.850+0000 7f911e59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a7497222d0 0x55a74972d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:51050/0 (socket says 172.21.15.101:51050) 2023-07-18T03:29:46.857 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.850+0000 7f911ed9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a74972dad0 0x55a74972aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:46.857 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.850+0000 7f9117fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a749722be0 0x55a74972a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:58332/0 (socket says 172.21.15.101:58332) 2023-07-18T03:29:46.858 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.850+0000 7f911e59c700 1 -- 172.21.15.101:0/3388258287 learned_addr learned my addr 172.21.15.101:0/3388258287 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:29:46.858 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.851+0000 7f911e59c700 1 -- 172.21.15.101:0/3388258287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a749722be0 msgr2=0x55a74972a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:29:46.858 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.851+0000 7f911e59c700 1 --2- 172.21.15.101:0/3388258287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a749722be0 0x55a74972a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:46.858 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.851+0000 7f911e59c700 1 -- 172.21.15.101:0/3388258287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a74972dad0 msgr2=0x55a74972aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:29:46.858 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.851+0000 7f911e59c700 1 --2- 172.21.15.101:0/3388258287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a74972dad0 0x55a74972aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:46.859 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.851+0000 7f911e59c700 1 -- 172.21.15.101:0/3388258287 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9110011440 con 0x55a7497222d0 2023-07-18T03:29:46.859 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.851+0000 7f9117fff700 1 --2- 172.21.15.101:0/3388258287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a749722be0 0x55a74972a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:29:46.859 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.851+0000 7f911ed9d700 1 --2- 172.21.15.101:0/3388258287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a74972dad0 0x55a74972aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:29:46.861 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.851+0000 7f911e59c700 1 --2- 172.21.15.101:0/3388258287 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a7497222d0 0x55a74972d530 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f9110009e00 tx=0x7f911001af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ca2d18298657c1 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:46.861 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.851+0000 7f9105ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:29:46.861 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.851+0000 7f9105ffb700 1 -- 172.21.15.101:0/3388258287 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9110005ce0 con 0x55a7497222d0 2023-07-18T03:29:46.861 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.851+0000 7f9138331b80 1 -- 172.21.15.101:0/3388258287 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a7495ce4f0 con 0x55a7497222d0 2023-07-18T03:29:46.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.851+0000 7f9105ffb700 1 -- 172.21.15.101:0/3388258287 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f9110005040 con 0x55a7497222d0 2023-07-18T03:29:46.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.852+0000 7f9105ffb700 1 -- 172.21.15.101:0/3388258287 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9110005200 con 0x55a7497222d0 2023-07-18T03:29:46.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.852+0000 7f9138331b80 10 client.18996 fetch_fsmap learned FSMap version 49 2023-07-18T03:29:46.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.852+0000 7f9138331b80 1 -- 172.21.15.101:0/3388258287 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55a74948bc60 con 0x55a7497222d0 2023-07-18T03:29:46.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.852+0000 7f9105ffb700 1 -- 172.21.15.101:0/3388258287 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a7495ce4f0 con 0x55a7497222d0 2023-07-18T03:29:46.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.852+0000 7f9105ffb700 1 -- 172.21.15.101:0/3388258287 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 49) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f91100185f0 con 0x55a7497222d0 2023-07-18T03:29:46.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.852+0000 7f9138331b80 10 client.18996 fetch_fsmap finished waiting for FSMap version 49 2023-07-18T03:29:46.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.852+0000 7f9138331b80 10 client.18996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:29:46.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.852+0000 7f9138331b80 20 client.18996 populate_metadata read hostname 'smithi101' 2023-07-18T03:29:46.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.852+0000 7f9138331b80 1 --2- 172.21.15.101:0/3388258287 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a74973d390 0x55a74973f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:46.864 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.852+0000 7f9138331b80 4 client.18996 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:29:46.864 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.852+0000 7f9138331b80 1 -- 172.21.15.101:0/3388258287 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a74966deb0 con 0x55a74973d390 2023-07-18T03:29:46.864 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.853+0000 7f9117fff700 1 --2- 172.21.15.101:0/3388258287 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a74973d390 0x55a74973f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:46.864 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.853+0000 7f9105ffb700 10 client.18996 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:29:46.864 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.853+0000 7f9117fff700 1 --2- 172.21.15.101:0/3388258287 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a74973d390 0x55a74973f7a0 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fe89f5e82cd79e9 server_cookie=7d29a1087b743f2 in_seq=0 out_seq=0 2023-07-18T03:29:46.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.854+0000 7f9105ffb700 1 -- 172.21.15.101:0/3388258287 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a74966deb0 con 0x55a74973d390 2023-07-18T03:29:46.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.854+0000 7f9105ffb700 10 client.18996 handle_command_reply: tid=0 2023-07-18T03:29:46.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.854+0000 7f9138331b80 1 client.18996 shutdown 2023-07-18T03:29:46.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.854+0000 7f9138331b80 1 -- 172.21.15.101:0/3388258287 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a7497222d0 msgr2=0x55a74972d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:46.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.854+0000 7f9138331b80 1 --2- 172.21.15.101:0/3388258287 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a7497222d0 0x55a74972d530 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f9110009e00 tx=0x7f911001af90 comp rx=0 tx=0).stop 2023-07-18T03:29:46.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.854+0000 7f9138331b80 1 -- 172.21.15.101:0/3388258287 shutdown_connections 2023-07-18T03:29:46.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.854+0000 7f9138331b80 1 --2- 172.21.15.101:0/3388258287 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a74973d390 0x55a74973f7a0 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:46.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.854+0000 7f9138331b80 1 --2- 172.21.15.101:0/3388258287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a74972dad0 0x55a74972aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:46.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.854+0000 7f9138331b80 1 --2- 172.21.15.101:0/3388258287 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a7497222d0 0x55a74972d530 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:46.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.854+0000 7f9105ffb700 0 client.18996 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:29:46.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.854+0000 7f9138331b80 1 --2- 172.21.15.101:0/3388258287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a749722be0 0x55a74972a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:46.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.854+0000 7f9138331b80 1 -- 172.21.15.101:0/3388258287 >> 172.21.15.101:0/3388258287 conn(0x55a749718da0 msgr2=0x55a74971e290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:46.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.854+0000 7f9138331b80 1 -- 172.21.15.101:0/3388258287 shutdown_connections 2023-07-18T03:29:46.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.854+0000 7f9138331b80 1 -- 172.21.15.101:0/3388258287 wait complete. 2023-07-18T03:29:46.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.854+0000 7f9138331b80 20 client.18996 trim_cache size 0 max 16384 2023-07-18T03:29:46.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.871+0000 7f9138331b80 1 Processor -- start 2023-07-18T03:29:46.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.871+0000 7f9138331b80 1 -- start start 2023-07-18T03:29:46.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.872+0000 7f9138331b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a7497222d0 0x55a74972d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:46.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.872+0000 7f9138331b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a749722be0 0x55a749727a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:46.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.872+0000 7f9138331b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a74972dad0 0x55a749727f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:46.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.872+0000 7f9138331b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a74971fd20 con 0x55a749722be0 2023-07-18T03:29:46.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.872+0000 7f9138331b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a749723540 con 0x55a74972dad0 2023-07-18T03:29:46.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.872+0000 7f9138331b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a749722fb0 con 0x55a7497222d0 2023-07-18T03:29:46.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.872+0000 7f9117fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a749722be0 0x55a749727a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:46.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.872+0000 7f911ed9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a74972dad0 0x55a749727f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:46.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.872+0000 7f911e59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a7497222d0 0x55a74972d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:46.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.872+0000 7f911ed9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a74972dad0 0x55a749727f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:44432/0 (socket says 172.21.15.101:44432) 2023-07-18T03:29:46.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.872+0000 7f9117fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a749722be0 0x55a749727a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:51060/0 (socket says 172.21.15.101:51060) 2023-07-18T03:29:46.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.872+0000 7f911ed9d700 1 -- 172.21.15.101:0/155849634 learned_addr learned my addr 172.21.15.101:0/155849634 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:29:46.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.872+0000 7f911e59c700 1 -- 172.21.15.101:0/155849634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a74972dad0 msgr2=0x55a749727f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:29:46.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.872+0000 7f911e59c700 1 --2- 172.21.15.101:0/155849634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a74972dad0 0x55a749727f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:46.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.872+0000 7f911e59c700 1 -- 172.21.15.101:0/155849634 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a749722be0 msgr2=0x55a749727a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:29:46.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.872+0000 7f911e59c700 1 --2- 172.21.15.101:0/155849634 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a749722be0 0x55a749727a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:46.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.872+0000 7f911e59c700 1 -- 172.21.15.101:0/155849634 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7494c9250 con 0x55a7497222d0 2023-07-18T03:29:46.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.873+0000 7f911e59c700 1 --2- 172.21.15.101:0/155849634 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a7497222d0 0x55a74972d0d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f9110011730 tx=0x7f9110011af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eda8fa93ab15af0d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:46.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.873+0000 7f91157fa700 1 -- 172.21.15.101:0/155849634 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911002ae60 con 0x55a7497222d0 2023-07-18T03:29:46.875 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.873+0000 7f91157fa700 1 -- 172.21.15.101:0/155849634 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9110019620 con 0x55a7497222d0 2023-07-18T03:29:46.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.873+0000 7f91157fa700 1 -- 172.21.15.101:0/155849634 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911002a3a0 con 0x55a7497222d0 2023-07-18T03:29:46.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.873+0000 7f9138331b80 1 -- 172.21.15.101:0/155849634 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a7497222d0 msgr2=0x55a74972d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:46.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.873+0000 7f9138331b80 1 --2- 172.21.15.101:0/155849634 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a7497222d0 0x55a74972d0d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f9110011730 tx=0x7f9110011af0 comp rx=0 tx=0).stop 2023-07-18T03:29:46.876 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.873+0000 7f9138331b80 1 -- 172.21.15.101:0/155849634 shutdown_connections 2023-07-18T03:29:46.877 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.873+0000 7f9138331b80 1 --2- 172.21.15.101:0/155849634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a74972dad0 0x55a749727f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:46.877 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.873+0000 7f9138331b80 1 --2- 172.21.15.101:0/155849634 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a7497222d0 0x55a74972d0d0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:46.877 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.873+0000 7f9138331b80 1 --2- 172.21.15.101:0/155849634 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a749722be0 0x55a749727a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:46.877 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.873+0000 7f9138331b80 1 -- 172.21.15.101:0/155849634 >> 172.21.15.101:0/155849634 conn(0x55a749718da0 msgr2=0x55a74971ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:46.877 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.873+0000 7f9138331b80 1 -- 172.21.15.101:0/155849634 shutdown_connections 2023-07-18T03:29:46.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.873+0000 7f9138331b80 1 -- 172.21.15.101:0/155849634 wait complete. 2023-07-18T03:29:46.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.874+0000 7f9138331b80 1 -- start start 2023-07-18T03:29:46.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.874+0000 7f9138331b80 1 Processor -- start 2023-07-18T03:29:46.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.875+0000 7f9138331b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a7497222d0 0x55a749733e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:46.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.875+0000 7f9138331b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a749722be0 0x55a749734360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:46.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.875+0000 7f9138331b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a74972dad0 0x55a749732270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:46.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.875+0000 7f9138331b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a749732980 con 0x55a74972dad0 2023-07-18T03:29:46.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.875+0000 7f9138331b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a749732af0 con 0x55a7497222d0 2023-07-18T03:29:46.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.875+0000 7f9138331b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a749732cb0 con 0x55a749722be0 2023-07-18T03:29:46.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.875+0000 7f9138331b80 10 client.0 _ll_register_callbacks cb 0x7f91180074e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:29:46.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.875+0000 7f911e59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a7497222d0 0x55a749733e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:46.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.875+0000 7f911e59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a7497222d0 0x55a749733e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:44440/0 (socket says 172.21.15.101:44440) 2023-07-18T03:29:46.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.875+0000 7f9117fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a749722be0 0x55a749734360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:46.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.875+0000 7f911e59c700 1 -- 172.21.15.101:0/861865422 learned_addr learned my addr 172.21.15.101:0/861865422 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:29:46.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.875+0000 7f911ed9d700 1 --2- 172.21.15.101:0/861865422 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a74972dad0 0x55a749732270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:46.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.876+0000 7f9117fff700 1 -- 172.21.15.101:0/861865422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a7497222d0 msgr2=0x55a749733e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:29:46.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.876+0000 7f9117fff700 1 --2- 172.21.15.101:0/861865422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a7497222d0 0x55a749733e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:46.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.876+0000 7f9117fff700 1 -- 172.21.15.101:0/861865422 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a74972dad0 msgr2=0x55a749732270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:29:46.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.876+0000 7f9117fff700 1 --2- 172.21.15.101:0/861865422 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a74972dad0 0x55a749732270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:46.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.876+0000 7f9117fff700 1 -- 172.21.15.101:0/861865422 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a74948bc60 con 0x55a749722be0 2023-07-18T03:29:46.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.876+0000 7f911ed9d700 1 --2- 172.21.15.101:0/861865422 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a74972dad0 0x55a749732270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:29:46.882 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.876+0000 7f911e59c700 1 --2- 172.21.15.101:0/861865422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a7497222d0 0x55a749733e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:29:46.882 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.876+0000 7f9117fff700 1 --2- 172.21.15.101:0/861865422 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a749722be0 0x55a749734360 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x55a749740010 tx=0x7f91080047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10946af33da195e6 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:46.882 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.876+0000 7f9105ffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:29:46.882 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.876+0000 7f9105ffb700 1 -- 172.21.15.101:0/861865422 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9108026050 con 0x55a749722be0 2023-07-18T03:29:46.882 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.876+0000 7f9138331b80 1 -- 172.21.15.101:0/861865422 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9110019a30 con 0x55a749722be0 2023-07-18T03:29:46.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.876+0000 7f9105ffb700 1 -- 172.21.15.101:0/861865422 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f9108003cf0 con 0x55a749722be0 2023-07-18T03:29:46.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.876+0000 7f9105ffb700 1 -- 172.21.15.101:0/861865422 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9108016040 con 0x55a749722be0 2023-07-18T03:29:46.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.876+0000 7f9138331b80 10 client.29098 fetch_fsmap learned FSMap version 49 2023-07-18T03:29:46.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.876+0000 7f9138331b80 1 -- 172.21.15.101:0/861865422 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55a7494a12b0 con 0x55a749722be0 2023-07-18T03:29:46.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.877+0000 7f9105ffb700 1 -- 172.21.15.101:0/861865422 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9110019a30 con 0x55a749722be0 2023-07-18T03:29:46.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.877+0000 7f9105ffb700 1 -- 172.21.15.101:0/861865422 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 49) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f910802e030 con 0x55a749722be0 2023-07-18T03:29:46.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.877+0000 7f9138331b80 10 client.29098 fetch_fsmap finished waiting for FSMap version 49 2023-07-18T03:29:46.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.877+0000 7f9138331b80 10 client.29098 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:29:46.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.877+0000 7f9138331b80 20 client.29098 populate_metadata read hostname 'smithi101' 2023-07-18T03:29:46.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.877+0000 7f9138331b80 1 --2- 172.21.15.101:0/861865422 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a749792350 0x55a749794760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:46.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.877+0000 7f9138331b80 4 client.29098 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:29:46.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.877+0000 7f9138331b80 1 -- 172.21.15.101:0/861865422 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a749630050 con 0x55a749792350 2023-07-18T03:29:46.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.877+0000 7f911e59c700 1 --2- 172.21.15.101:0/861865422 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a749792350 0x55a749794760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:46.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.877+0000 7f911e59c700 1 --2- 172.21.15.101:0/861865422 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a749792350 0x55a749794760 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6109c2798b580f67 server_cookie=e918319742625be1 in_seq=0 out_seq=0 2023-07-18T03:29:46.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.877+0000 7f9105ffb700 10 client.29098 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:29:46.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.878+0000 7f9105ffb700 1 -- 172.21.15.101:0/861865422 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55a749630050 con 0x55a749792350 2023-07-18T03:29:46.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.878+0000 7f9105ffb700 10 client.29098 handle_command_reply: tid=0 2023-07-18T03:29:46.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.878+0000 7f9138331b80 1 client.29098 shutdown 2023-07-18T03:29:46.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.878+0000 7f9138331b80 1 -- 172.21.15.101:0/861865422 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a749722be0 msgr2=0x55a749734360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:46.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.878+0000 7f9138331b80 1 --2- 172.21.15.101:0/861865422 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a749722be0 0x55a749734360 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x55a749740010 tx=0x7f91080047c0 comp rx=0 tx=0).stop 2023-07-18T03:29:46.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.878+0000 7f9138331b80 1 -- 172.21.15.101:0/861865422 shutdown_connections 2023-07-18T03:29:46.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.878+0000 7f9138331b80 1 --2- 172.21.15.101:0/861865422 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55a749792350 0x55a749794760 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:46.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.878+0000 7f9138331b80 1 --2- 172.21.15.101:0/861865422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a7497222d0 0x55a749733e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:46.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.878+0000 7f9105ffb700 0 client.29098 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:29:46.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.878+0000 7f9138331b80 1 --2- 172.21.15.101:0/861865422 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a74972dad0 0x55a749732270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:46.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.878+0000 7f9138331b80 1 --2- 172.21.15.101:0/861865422 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a749722be0 0x55a749734360 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:46.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.878+0000 7f9138331b80 1 -- 172.21.15.101:0/861865422 >> 172.21.15.101:0/861865422 conn(0x55a749718da0 msgr2=0x55a74971d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:46.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.878+0000 7f9138331b80 1 -- 172.21.15.101:0/861865422 shutdown_connections 2023-07-18T03:29:46.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.878+0000 7f9138331b80 1 -- 172.21.15.101:0/861865422 wait complete. 2023-07-18T03:29:46.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.878+0000 7f9138331b80 20 client.29098 trim_cache size 0 max 16384 2023-07-18T03:29:46.889 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:29:46.889 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:29:46.889 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:29:46.890 INFO:teuthology.orchestra.run.smithi101.stdout: "0f2f19e0-0429-465c-935e-223eb09490c3": { 2023-07-18T03:29:46.890 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:29:46.890 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "0f2f19e0-0429-465c-935e-223eb09490c3", 2023-07-18T03:29:46.890 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:29:46.890 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:29:46.891 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:29:46.891 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:29:46.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.879+0000 7f91249ff700 1 -- 172.21.15.101:0/886797476 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f910c0a12c0 msgr2=0x7f910c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:46.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.879+0000 7f91249ff700 1 --2- 172.21.15.101:0/886797476 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f910c0a12c0 0x7f910c0a3770 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9120199550 tx=0x7f91080118e0 comp rx=0 tx=0).stop 2023-07-18T03:29:46.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.879+0000 7f91249ff700 1 -- 172.21.15.101:0/886797476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f912010c3a0 msgr2=0x7f912019c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:46.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.879+0000 7f91249ff700 1 --2- 172.21.15.101:0/886797476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f912010c3a0 0x7f912019c8e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f911800b070 tx=0x7f9118007930 comp rx=0 tx=0).stop 2023-07-18T03:29:46.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.879+0000 7f91249ff700 1 -- 172.21.15.101:0/886797476 shutdown_connections 2023-07-18T03:29:46.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.879+0000 7f91249ff700 1 --2- 172.21.15.101:0/886797476 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f910c0a12c0 0x7f910c0a3770 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:46.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.879+0000 7f91249ff700 1 --2- 172.21.15.101:0/886797476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f912010c3a0 0x7f912019c8e0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:46.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.879+0000 7f91249ff700 1 --2- 172.21.15.101:0/886797476 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9120072850 0x7f9120197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:46.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.879+0000 7f91249ff700 1 --2- 172.21.15.101:0/886797476 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9120073160 0x7f9120198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:46.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.879+0000 7f91249ff700 1 -- 172.21.15.101:0/886797476 >> 172.21.15.101:0/886797476 conn(0x7f91200fb600 msgr2=0x7f9120103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:46.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.879+0000 7f91249ff700 1 -- 172.21.15.101:0/886797476 shutdown_connections 2023-07-18T03:29:46.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:29:46.879+0000 7f91249ff700 1 -- 172.21.15.101:0/886797476 wait complete. 2023-07-18T03:29:46.895 INFO:tasks.cephfs.filesystem:scrub status for tag:0f2f19e0-0429-465c-935e-223eb09490c3 - {'path': '/', 'tag': '0f2f19e0-0429-465c-935e-223eb09490c3', 'options': 'recursive,force'} 2023-07-18T03:29:47.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:29:47 smithi179 ceph-mon[129229]: pgmap v701: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:29:47.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:29:47 smithi101 ceph-mon[126379]: pgmap v701: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:29:47.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:29:47 smithi155 ceph-mon[128885]: pgmap v701: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:29:49.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:29:49 smithi179 ceph-mon[129229]: pgmap v702: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:29:49.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:29:49 smithi101 ceph-mon[126379]: pgmap v702: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:29:49.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:29:49 smithi155 ceph-mon[128885]: pgmap v702: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:29:51.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:29:51 smithi179 ceph-mon[129229]: pgmap v703: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-18T03:29:51.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:29:51 smithi101 ceph-mon[126379]: pgmap v703: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-18T03:29:51.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:29:51 smithi155 ceph-mon[128885]: pgmap v703: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-18T03:29:53.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:29:53 smithi179 ceph-mon[129229]: pgmap v704: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:29:53.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:29:53 smithi101 ceph-mon[126379]: pgmap v704: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:29:53.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:29:53 smithi155 ceph-mon[128885]: pgmap v704: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:29:55.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:29:55 smithi179 ceph-mon[129229]: pgmap v705: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:29:55.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:29:55 smithi101 ceph-mon[126379]: pgmap v705: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:29:55.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:29:55 smithi155 ceph-mon[128885]: pgmap v705: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:29:57.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:29:57 smithi179 ceph-mon[129229]: pgmap v706: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:29:57.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:29:57 smithi101 ceph-mon[126379]: pgmap v706: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:29:57.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:29:57 smithi155 ceph-mon[128885]: pgmap v706: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:29:59.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:29:59 smithi179 ceph-mon[129229]: pgmap v707: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:29:59.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:29:59 smithi101 ceph-mon[126379]: pgmap v707: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:29:59.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:29:59 smithi155 ceph-mon[128885]: pgmap v707: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:30:00.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:30:00 smithi179 ceph-mon[129229]: overall HEALTH_OK 2023-07-18T03:30:00.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:30:00 smithi101 ceph-mon[126379]: overall HEALTH_OK 2023-07-18T03:30:00.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:30:00 smithi155 ceph-mon[128885]: overall HEALTH_OK 2023-07-18T03:30:01.600 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:30:01 smithi179 ceph-mon[129229]: pgmap v708: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-18T03:30:01.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:30:01 smithi101 ceph-mon[126379]: pgmap v708: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-18T03:30:01.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:30:01 smithi155 ceph-mon[128885]: pgmap v708: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-18T03:30:03.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:30:03 smithi179 ceph-mon[129229]: pgmap v709: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 22 KiB/s wr, 9 op/s 2023-07-18T03:30:03.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:30:03 smithi101 ceph-mon[126379]: pgmap v709: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 22 KiB/s wr, 9 op/s 2023-07-18T03:30:03.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:30:03 smithi155 ceph-mon[128885]: pgmap v709: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 22 KiB/s wr, 9 op/s 2023-07-18T03:30:05.684 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:30:05 smithi155 ceph-mon[128885]: pgmap v710: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 22 KiB/s wr, 8 op/s 2023-07-18T03:30:05.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:30:05 smithi179 ceph-mon[129229]: pgmap v710: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 22 KiB/s wr, 8 op/s 2023-07-18T03:30:05.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:30:05 smithi101 ceph-mon[126379]: pgmap v710: 129 pgs: 129 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 22 KiB/s wr, 8 op/s 2023-07-18T03:30:06.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:30:06 smithi179 ceph-mon[129229]: osdmap e125: 12 total, 12 up, 12 in 2023-07-18T03:30:06.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:30:06 smithi101 ceph-mon[126379]: osdmap e125: 12 total, 12 up, 12 in 2023-07-18T03:30:06.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:30:06 smithi155 ceph-mon[128885]: osdmap e125: 12 total, 12 up, 12 in 2023-07-18T03:30:07.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:30:07 smithi179 ceph-mon[129229]: pgmap v712: 129 pgs: 4 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 111 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-07-18T03:30:07.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:30:07 smithi101 ceph-mon[126379]: pgmap v712: 129 pgs: 4 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 111 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-07-18T03:30:07.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:30:07 smithi155 ceph-mon[128885]: pgmap v712: 129 pgs: 4 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 111 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-07-18T03:30:09.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:30:09 smithi179 ceph-mon[129229]: pgmap v713: 129 pgs: 4 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 111 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-07-18T03:30:09.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:30:09 smithi101 ceph-mon[126379]: pgmap v713: 129 pgs: 4 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 111 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-07-18T03:30:09.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:30:09 smithi155 ceph-mon[128885]: pgmap v713: 129 pgs: 4 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 111 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-07-18T03:30:11.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:30:11 smithi179 ceph-mon[129229]: pgmap v714: 129 pgs: 4 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 111 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-07-18T03:30:11.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:30:11 smithi101 ceph-mon[126379]: pgmap v714: 129 pgs: 4 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 111 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-07-18T03:30:11.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:30:11 smithi155 ceph-mon[128885]: pgmap v714: 129 pgs: 4 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 111 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-07-18T03:30:13.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:30:13 smithi179 ceph-mon[129229]: pgmap v715: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 KiB/s wr, 19 op/s 2023-07-18T03:30:13.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:30:13 smithi101 ceph-mon[126379]: pgmap v715: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 KiB/s wr, 19 op/s 2023-07-18T03:30:13.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:30:13 smithi155 ceph-mon[128885]: pgmap v715: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 KiB/s wr, 19 op/s 2023-07-18T03:30:15.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:30:15 smithi155 ceph-mon[128885]: pgmap v716: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 KiB/s wr, 19 op/s 2023-07-18T03:30:15.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:30:15 smithi155 ceph-mon[128885]: osdmap e126: 12 total, 12 up, 12 in 2023-07-18T03:30:15.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:30:15 smithi179 ceph-mon[129229]: pgmap v716: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 KiB/s wr, 19 op/s 2023-07-18T03:30:15.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:30:15 smithi179 ceph-mon[129229]: osdmap e126: 12 total, 12 up, 12 in 2023-07-18T03:30:15.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:30:15 smithi101 ceph-mon[126379]: pgmap v716: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 KiB/s wr, 19 op/s 2023-07-18T03:30:15.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:30:15 smithi101 ceph-mon[126379]: osdmap e126: 12 total, 12 up, 12 in 2023-07-18T03:30:16.896 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:30:17.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.112+0000 7f793f5ff700 1 -- 172.21.15.101:0/926243390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7938105ae0 msgr2=0x7f7938105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:17.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.112+0000 7f793f5ff700 1 --2- 172.21.15.101:0/926243390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7938105ae0 0x7f7938105eb0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f7934009990 tx=0x7f7934009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.113+0000 7f793f5ff700 1 -- 172.21.15.101:0/926243390 shutdown_connections 2023-07-18T03:30:17.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.113+0000 7f793f5ff700 1 --2- 172.21.15.101:0/926243390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7938100480 0x7f793810c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.113+0000 7f793f5ff700 1 --2- 172.21.15.101:0/926243390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7938105ae0 0x7f7938105eb0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.113+0000 7f793f5ff700 1 --2- 172.21.15.101:0/926243390 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f79380ffad0 0x7f79380fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.113+0000 7f793f5ff700 1 -- 172.21.15.101:0/926243390 >> 172.21.15.101:0/926243390 conn(0x7f79380fb5f0 msgr2=0x7f79380fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:17.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.113+0000 7f793f5ff700 1 -- 172.21.15.101:0/926243390 shutdown_connections 2023-07-18T03:30:17.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.113+0000 7f793f5ff700 1 -- 172.21.15.101:0/926243390 wait complete. 2023-07-18T03:30:17.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.114+0000 7f793f5ff700 1 Processor -- start 2023-07-18T03:30:17.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.114+0000 7f793f5ff700 1 -- start start 2023-07-18T03:30:17.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.114+0000 7f793f5ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79380ffad0 0x7f7938072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:17.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.114+0000 7f793f5ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7938100480 0x7f7938072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:17.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.114+0000 7f793f5ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7938105ae0 0x7f79381a3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:17.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.114+0000 7f793f5ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7938073580 con 0x7f7938105ae0 2023-07-18T03:30:17.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.114+0000 7f793f5ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79381a4330 con 0x7f7938100480 2023-07-18T03:30:17.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.114+0000 7f793f5ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79381a4470 con 0x7f79380ffad0 2023-07-18T03:30:17.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.114+0000 7f793d39b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79380ffad0 0x7f7938072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:17.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.115+0000 7f793cb9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7938100480 0x7f7938072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:17.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.115+0000 7f793db9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7938105ae0 0x7f79381a3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:17.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.115+0000 7f793d39b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79380ffad0 0x7f7938072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:45022/0 (socket says 172.21.15.101:45022) 2023-07-18T03:30:17.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.115+0000 7f793d39b700 1 -- 172.21.15.101:0/1284920828 learned_addr learned my addr 172.21.15.101:0/1284920828 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:30:17.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.115+0000 7f793d39b700 1 -- 172.21.15.101:0/1284920828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7938100480 msgr2=0x7f7938072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:17.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.115+0000 7f793d39b700 1 --2- 172.21.15.101:0/1284920828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7938100480 0x7f7938072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.115+0000 7f793d39b700 1 -- 172.21.15.101:0/1284920828 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7938105ae0 msgr2=0x7f79381a3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:17.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.115+0000 7f793d39b700 1 --2- 172.21.15.101:0/1284920828 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7938105ae0 0x7f79381a3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.115+0000 7f793d39b700 1 -- 172.21.15.101:0/1284920828 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79340095f0 con 0x7f79380ffad0 2023-07-18T03:30:17.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.115+0000 7f793cb9a700 1 --2- 172.21.15.101:0/1284920828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7938100480 0x7f7938072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:30:17.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.115+0000 7f793db9c700 1 --2- 172.21.15.101:0/1284920828 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7938105ae0 0x7f79381a3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:30:17.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.115+0000 7f793d39b700 1 --2- 172.21.15.101:0/1284920828 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79380ffad0 0x7f7938072810 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f7934005910 tx=0x7f7934005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:17.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.115+0000 7f792e7fc700 1 -- 172.21.15.101:0/1284920828 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793401d070 con 0x7f79380ffad0 2023-07-18T03:30:17.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.115+0000 7f792e7fc700 1 -- 172.21.15.101:0/1284920828 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7934022470 con 0x7f79380ffad0 2023-07-18T03:30:17.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.115+0000 7f793f5ff700 1 -- 172.21.15.101:0/1284920828 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79381a4670 con 0x7f79380ffad0 2023-07-18T03:30:17.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.115+0000 7f792e7fc700 1 -- 172.21.15.101:0/1284920828 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7934017650 con 0x7f79380ffad0 2023-07-18T03:30:17.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.116+0000 7f793f5ff700 1 -- 172.21.15.101:0/1284920828 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79381a4af0 con 0x7f79380ffad0 2023-07-18T03:30:17.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.119+0000 7f792e7fc700 1 -- 172.21.15.101:0/1284920828 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f79340225e0 con 0x7f79380ffad0 2023-07-18T03:30:17.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.120+0000 7f7953014b80 1 Processor -- start 2023-07-18T03:30:17.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.120+0000 7f7953014b80 1 -- start start 2023-07-18T03:30:17.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.120+0000 7f7953014b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614d92d02d0 0x5614d92d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:17.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.120+0000 7f7953014b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614d92d0be0 0x5614d92db590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:17.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.120+0000 7f7953014b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614d92dbad0 0x5614d92ddeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:17.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.120+0000 7f7953014b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d91d6cb0 con 0x5614d92d0be0 2023-07-18T03:30:17.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.120+0000 7f7953014b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d9229a40 con 0x5614d92dbad0 2023-07-18T03:30:17.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.120+0000 7f7953014b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d9222ed0 con 0x5614d92d02d0 2023-07-18T03:30:17.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.120+0000 7f792e7fc700 1 --2- 172.21.15.101:0/1284920828 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f79240a1340 0x7f79240a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:17.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.120+0000 7f793cb9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614d92dbad0 0x5614d92ddeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:17.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.121+0000 7f793cb9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614d92dbad0 0x5614d92ddeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:55588/0 (socket says 172.21.15.101:55588) 2023-07-18T03:30:17.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.121+0000 7f793cb9a700 1 -- 172.21.15.101:0/1025486008 learned_addr learned my addr 172.21.15.101:0/1025486008 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:30:17.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.121+0000 7f793db9c700 1 --2- 172.21.15.101:0/1025486008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614d92d02d0 0x5614d92d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:17.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.121+0000 7f792e7fc700 1 -- 172.21.15.101:0/1284920828 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f79340ce390 con 0x7f79380ffad0 2023-07-18T03:30:17.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.121+0000 7f793d39b700 1 --2- 172.21.15.101:0/1025486008 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614d92d0be0 0x5614d92db590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:17.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.121+0000 7f793db9c700 1 --2- 172.21.15.101:0/1284920828 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f79240a1340 0x7f79240a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:17.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.121+0000 7f793cb9a700 1 -- 172.21.15.101:0/1025486008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614d92d02d0 msgr2=0x5614d92d06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:17.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.121+0000 7f793cb9a700 1 --2- 172.21.15.101:0/1025486008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614d92d02d0 0x5614d92d06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.121+0000 7f793cb9a700 1 -- 172.21.15.101:0/1025486008 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614d92d0be0 msgr2=0x5614d92db590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:17.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.121+0000 7f793cb9a700 1 --2- 172.21.15.101:0/1025486008 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614d92d0be0 0x5614d92db590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.121+0000 7f793cb9a700 1 -- 172.21.15.101:0/1025486008 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614d9050de0 con 0x5614d92dbad0 2023-07-18T03:30:17.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.121+0000 7f793cb9a700 1 --2- 172.21.15.101:0/1025486008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614d92dbad0 0x5614d92ddeb0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f79280115f0 tx=0x7f7928011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=993a5108a2557052 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:17.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.121+0000 7f793db9c700 1 --2- 172.21.15.101:0/1284920828 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f79240a1340 0x7f79240a37f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7930009ec0 tx=0x7f793000b040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:17.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.121+0000 7f79237fe700 1 -- 172.21.15.101:0/1025486008 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7928004230 con 0x5614d92dbad0 2023-07-18T03:30:17.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.121+0000 7f79237fe700 1 -- 172.21.15.101:0/1025486008 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7928004390 con 0x5614d92dbad0 2023-07-18T03:30:17.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.121+0000 7f79237fe700 1 -- 172.21.15.101:0/1025486008 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79280198c0 con 0x5614d92dbad0 2023-07-18T03:30:17.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.122+0000 7f7953014b80 1 -- 172.21.15.101:0/1025486008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614d92dbad0 msgr2=0x5614d92ddeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:17.128 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.122+0000 7f7953014b80 1 --2- 172.21.15.101:0/1025486008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614d92dbad0 0x5614d92ddeb0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f79280115f0 tx=0x7f7928011900 comp rx=0 tx=0).stop 2023-07-18T03:30:17.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.122+0000 7f7953014b80 1 -- 172.21.15.101:0/1025486008 shutdown_connections 2023-07-18T03:30:17.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.122+0000 7f7953014b80 1 --2- 172.21.15.101:0/1025486008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614d92dbad0 0x5614d92ddeb0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.122+0000 7f7953014b80 1 --2- 172.21.15.101:0/1025486008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614d92d02d0 0x5614d92d06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.122+0000 7f7953014b80 1 --2- 172.21.15.101:0/1025486008 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614d92d0be0 0x5614d92db590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.129 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.122+0000 7f7953014b80 1 -- 172.21.15.101:0/1025486008 >> 172.21.15.101:0/1025486008 conn(0x5614d92c6da0 msgr2=0x5614d92cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:17.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.122+0000 7f7953014b80 1 -- 172.21.15.101:0/1025486008 shutdown_connections 2023-07-18T03:30:17.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.122+0000 7f7953014b80 1 -- 172.21.15.101:0/1025486008 wait complete. 2023-07-18T03:30:17.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.123+0000 7f7953014b80 1 -- start start 2023-07-18T03:30:17.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.123+0000 7f7953014b80 1 Processor -- start 2023-07-18T03:30:17.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.123+0000 7f7953014b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614d92d02d0 0x5614d92db3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:17.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.123+0000 7f7953014b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614d92d0be0 0x5614d92d86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:17.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.123+0000 7f7953014b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614d92dbad0 0x5614d92d8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:17.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.123+0000 7f7953014b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d91c2ac0 con 0x5614d92d0be0 2023-07-18T03:30:17.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.123+0000 7f7953014b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d91c27c0 con 0x5614d92dbad0 2023-07-18T03:30:17.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.124+0000 7f7953014b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d90a6920 con 0x5614d92d02d0 2023-07-18T03:30:17.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.124+0000 7f7953014b80 10 client.0 _ll_register_callbacks cb 0x5614d91c0760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:30:17.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.124+0000 7f793cb9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614d92d0be0 0x5614d92d86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:17.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.124+0000 7f793db9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614d92dbad0 0x5614d92d8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:17.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.124+0000 7f793d39b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614d92d02d0 0x5614d92db3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:17.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.124+0000 7f793d39b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614d92d02d0 0x5614d92db3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:45048/0 (socket says 172.21.15.101:45048) 2023-07-18T03:30:17.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.124+0000 7f793d39b700 1 -- 172.21.15.101:0/2083333544 learned_addr learned my addr 172.21.15.101:0/2083333544 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:30:17.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.124+0000 7f793d39b700 1 -- 172.21.15.101:0/2083333544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614d92dbad0 msgr2=0x5614d92d8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:17.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.124+0000 7f793d39b700 1 --2- 172.21.15.101:0/2083333544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614d92dbad0 0x5614d92d8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.124+0000 7f793d39b700 1 -- 172.21.15.101:0/2083333544 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614d92d0be0 msgr2=0x5614d92d86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:17.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.124+0000 7f793d39b700 1 --2- 172.21.15.101:0/2083333544 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614d92d0be0 0x5614d92d86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.124+0000 7f793d39b700 1 -- 172.21.15.101:0/2083333544 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7928009d50 con 0x5614d92d02d0 2023-07-18T03:30:17.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.124+0000 7f793d39b700 1 --2- 172.21.15.101:0/2083333544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614d92d02d0 0x5614d92db3d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f793409b070 tx=0x7f793409acf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df490e4b6d7239d9 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:17.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.124+0000 7f7920ff9700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:30:17.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.124+0000 7f7920ff9700 1 -- 172.21.15.101:0/2083333544 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793409f110 con 0x5614d92d02d0 2023-07-18T03:30:17.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.124+0000 7f7920ff9700 1 -- 172.21.15.101:0/2083333544 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f79340a34a0 con 0x5614d92d02d0 2023-07-18T03:30:17.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.125+0000 7f7953014b80 1 -- 172.21.15.101:0/2083333544 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5614d917c4f0 con 0x5614d92d02d0 2023-07-18T03:30:17.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.125+0000 7f7920ff9700 1 -- 172.21.15.101:0/2083333544 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f793409b340 con 0x5614d92d02d0 2023-07-18T03:30:17.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.125+0000 7f7953014b80 10 client.29110 fetch_fsmap learned FSMap version 49 2023-07-18T03:30:17.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.125+0000 7f7953014b80 1 -- 172.21.15.101:0/2083333544 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5614d9039c60 con 0x5614d92d02d0 2023-07-18T03:30:17.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.125+0000 7f7920ff9700 1 -- 172.21.15.101:0/2083333544 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5614d917c4f0 con 0x5614d92d02d0 2023-07-18T03:30:17.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.125+0000 7f7920ff9700 1 -- 172.21.15.101:0/2083333544 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 49) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f79340a3010 con 0x5614d92d02d0 2023-07-18T03:30:17.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.125+0000 7f7953014b80 10 client.29110 fetch_fsmap finished waiting for FSMap version 49 2023-07-18T03:30:17.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.125+0000 7f7953014b80 10 client.29110 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:30:17.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.125+0000 7f7953014b80 20 client.29110 populate_metadata read hostname 'smithi101' 2023-07-18T03:30:17.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.125+0000 7f7953014b80 1 --2- 172.21.15.101:0/2083333544 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5614d92eaf00 0x5614d92ed2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:17.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.125+0000 7f7953014b80 4 client.29110 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:30:17.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.125+0000 7f7953014b80 1 -- 172.21.15.101:0/2083333544 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5614d921beb0 con 0x5614d92eaf00 2023-07-18T03:30:17.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.125+0000 7f793cb9a700 1 --2- 172.21.15.101:0/2083333544 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5614d92eaf00 0x5614d92ed2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:17.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.126+0000 7f793cb9a700 1 --2- 172.21.15.101:0/2083333544 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5614d92eaf00 0x5614d92ed2e0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c46bed1b018ab4c6 server_cookie=2dd754387057f427 in_seq=0 out_seq=0 2023-07-18T03:30:17.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.126+0000 7f7920ff9700 10 client.29110 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:30:17.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.127+0000 7f7920ff9700 1 -- 172.21.15.101:0/2083333544 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5614d921beb0 con 0x5614d92eaf00 2023-07-18T03:30:17.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.127+0000 7f7920ff9700 10 client.29110 handle_command_reply: tid=0 2023-07-18T03:30:17.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.127+0000 7f7953014b80 1 client.29110 shutdown 2023-07-18T03:30:17.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.127+0000 7f7953014b80 1 -- 172.21.15.101:0/2083333544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614d92d02d0 msgr2=0x5614d92db3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:17.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.127+0000 7f7953014b80 1 --2- 172.21.15.101:0/2083333544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614d92d02d0 0x5614d92db3d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f793409b070 tx=0x7f793409acf0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.127+0000 7f7953014b80 1 -- 172.21.15.101:0/2083333544 shutdown_connections 2023-07-18T03:30:17.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.127+0000 7f7953014b80 1 --2- 172.21.15.101:0/2083333544 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5614d92eaf00 0x5614d92ed2e0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.127+0000 7f7953014b80 1 --2- 172.21.15.101:0/2083333544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614d92dbad0 0x5614d92d8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.127+0000 7f7953014b80 1 --2- 172.21.15.101:0/2083333544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614d92d02d0 0x5614d92db3d0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.127+0000 7f7953014b80 1 --2- 172.21.15.101:0/2083333544 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614d92d0be0 0x5614d92d86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.127+0000 7f7920ff9700 0 client.29110 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:30:17.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.127+0000 7f7953014b80 1 -- 172.21.15.101:0/2083333544 >> 172.21.15.101:0/2083333544 conn(0x5614d92c6da0 msgr2=0x5614d92cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:17.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.127+0000 7f7953014b80 1 -- 172.21.15.101:0/2083333544 shutdown_connections 2023-07-18T03:30:17.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.127+0000 7f7953014b80 1 -- 172.21.15.101:0/2083333544 wait complete. 2023-07-18T03:30:17.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.127+0000 7f7953014b80 20 client.29110 trim_cache size 0 max 16384 2023-07-18T03:30:17.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.144+0000 7f7953014b80 1 Processor -- start 2023-07-18T03:30:17.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.144+0000 7f7953014b80 1 -- start start 2023-07-18T03:30:17.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.145+0000 7f7953014b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614d92d02d0 0x5614d92daf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:17.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.145+0000 7f7953014b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614d92d0be0 0x5614d92d7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:17.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.145+0000 7f7953014b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614d92dbad0 0x5614d92d83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:17.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.145+0000 7f7953014b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d92d1e70 con 0x5614d92dbad0 2023-07-18T03:30:17.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.145+0000 7f7953014b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d92d9c30 con 0x5614d92d0be0 2023-07-18T03:30:17.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.145+0000 7f7953014b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d92e7ca0 con 0x5614d92d02d0 2023-07-18T03:30:17.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.145+0000 7f793cb9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614d92d0be0 0x5614d92d7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:17.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.145+0000 7f793d39b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614d92d02d0 0x5614d92daf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:17.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.145+0000 7f793cb9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614d92d0be0 0x5614d92d7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:55620/0 (socket says 172.21.15.101:55620) 2023-07-18T03:30:17.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.145+0000 7f793cb9a700 1 -- 172.21.15.101:0/2395761169 learned_addr learned my addr 172.21.15.101:0/2395761169 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:30:17.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.145+0000 7f793d39b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614d92d02d0 0x5614d92daf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:45056/0 (socket says 172.21.15.101:45056) 2023-07-18T03:30:17.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.145+0000 7f793db9c700 1 --2- 172.21.15.101:0/2395761169 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614d92dbad0 0x5614d92d83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:17.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.145+0000 7f793cb9a700 1 -- 172.21.15.101:0/2395761169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614d92d02d0 msgr2=0x5614d92daf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:17.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.145+0000 7f793cb9a700 1 --2- 172.21.15.101:0/2395761169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614d92d02d0 0x5614d92daf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.145+0000 7f793cb9a700 1 -- 172.21.15.101:0/2395761169 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614d92dbad0 msgr2=0x5614d92d83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:17.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.145+0000 7f793cb9a700 1 --2- 172.21.15.101:0/2395761169 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614d92dbad0 0x5614d92d83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.146+0000 7f793cb9a700 1 -- 172.21.15.101:0/2395761169 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614d9077250 con 0x5614d92d0be0 2023-07-18T03:30:17.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.146+0000 7f793db9c700 1 --2- 172.21.15.101:0/2395761169 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614d92dbad0 0x5614d92d83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:30:17.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.146+0000 7f793cb9a700 1 --2- 172.21.15.101:0/2395761169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614d92d0be0 0x5614d92d7e80 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f7928004360 tx=0x7f7928011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d600a1a22ee84c7a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:17.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.146+0000 7f7923fff700 1 -- 172.21.15.101:0/2395761169 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7928029040 con 0x5614d92d0be0 2023-07-18T03:30:17.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.146+0000 7f7923fff700 1 -- 172.21.15.101:0/2395761169 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7928004500 con 0x5614d92d0be0 2023-07-18T03:30:17.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.146+0000 7f7923fff700 1 -- 172.21.15.101:0/2395761169 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792802aa50 con 0x5614d92d0be0 2023-07-18T03:30:17.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.147+0000 7f7953014b80 1 -- 172.21.15.101:0/2395761169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614d92d0be0 msgr2=0x5614d92d7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:17.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.147+0000 7f7953014b80 1 --2- 172.21.15.101:0/2395761169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614d92d0be0 0x5614d92d7e80 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f7928004360 tx=0x7f7928011900 comp rx=0 tx=0).stop 2023-07-18T03:30:17.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.147+0000 7f7953014b80 1 -- 172.21.15.101:0/2395761169 shutdown_connections 2023-07-18T03:30:17.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.147+0000 7f7953014b80 1 --2- 172.21.15.101:0/2395761169 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614d92dbad0 0x5614d92d83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.147+0000 7f7953014b80 1 --2- 172.21.15.101:0/2395761169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614d92d02d0 0x5614d92daf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.147+0000 7f7953014b80 1 --2- 172.21.15.101:0/2395761169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614d92d0be0 0x5614d92d7e80 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.147+0000 7f7953014b80 1 -- 172.21.15.101:0/2395761169 >> 172.21.15.101:0/2395761169 conn(0x5614d92c6da0 msgr2=0x5614d92d3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:17.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.147+0000 7f7953014b80 1 -- 172.21.15.101:0/2395761169 shutdown_connections 2023-07-18T03:30:17.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.147+0000 7f7953014b80 1 -- 172.21.15.101:0/2395761169 wait complete. 2023-07-18T03:30:17.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.147+0000 7f7953014b80 1 -- start start 2023-07-18T03:30:17.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.147+0000 7f7953014b80 1 Processor -- start 2023-07-18T03:30:17.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.148+0000 7f7953014b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614d92d02d0 0x5614d92e23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:17.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.148+0000 7f7953014b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614d92d0be0 0x5614d92e06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:17.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.148+0000 7f7953014b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614d92dbad0 0x5614d92e0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:17.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.148+0000 7f7953014b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d92e11c0 con 0x5614d92dbad0 2023-07-18T03:30:17.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.148+0000 7f7953014b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d92e1300 con 0x5614d92d02d0 2023-07-18T03:30:17.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.148+0000 7f7953014b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d92e14c0 con 0x5614d92d0be0 2023-07-18T03:30:17.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.148+0000 7f7953014b80 10 client.0 _ll_register_callbacks cb 0x5614d91c0760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:30:17.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.148+0000 7f793cb9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614d92d0be0 0x5614d92e06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:17.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.148+0000 7f793d39b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614d92d02d0 0x5614d92e23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:17.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.148+0000 7f793cb9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614d92d0be0 0x5614d92e06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:45066/0 (socket says 172.21.15.101:45066) 2023-07-18T03:30:17.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.148+0000 7f793cb9a700 1 -- 172.21.15.101:0/1952443512 learned_addr learned my addr 172.21.15.101:0/1952443512 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:30:17.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.148+0000 7f793d39b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614d92d02d0 0x5614d92e23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:55626/0 (socket says 172.21.15.101:55626) 2023-07-18T03:30:17.156 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.149+0000 7f793db9c700 1 --2- 172.21.15.101:0/1952443512 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614d92dbad0 0x5614d92e0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:17.156 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.149+0000 7f793cb9a700 1 -- 172.21.15.101:0/1952443512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614d92d02d0 msgr2=0x5614d92e23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:17.156 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.149+0000 7f793cb9a700 1 --2- 172.21.15.101:0/1952443512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614d92d02d0 0x5614d92e23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.156 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.149+0000 7f793cb9a700 1 -- 172.21.15.101:0/1952443512 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614d92dbad0 msgr2=0x5614d92e0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:17.156 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.149+0000 7f793cb9a700 1 --2- 172.21.15.101:0/1952443512 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614d92dbad0 0x5614d92e0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.149+0000 7f793cb9a700 1 -- 172.21.15.101:0/1952443512 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5614d9050de0 con 0x5614d92d0be0 2023-07-18T03:30:17.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.149+0000 7f793d39b700 1 --2- 172.21.15.101:0/1952443512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614d92d02d0 0x5614d92e23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:30:17.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.149+0000 7f793db9c700 1 --2- 172.21.15.101:0/1952443512 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614d92dbad0 0x5614d92e0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:30:17.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.149+0000 7f793cb9a700 1 --2- 172.21.15.101:0/1952443512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614d92d0be0 0x5614d92e06f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f7928005740 tx=0x7f7928005770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ef87077f562562 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:17.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.149+0000 7f7920ff9700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:30:17.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.149+0000 7f7920ff9700 1 -- 172.21.15.101:0/1952443512 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7928029040 con 0x5614d92d0be0 2023-07-18T03:30:17.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.149+0000 7f7953014b80 1 -- 172.21.15.101:0/1952443512 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5614d91de050 con 0x5614d92d0be0 2023-07-18T03:30:17.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.149+0000 7f7920ff9700 1 -- 172.21.15.101:0/1952443512 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f792802a780 con 0x5614d92d0be0 2023-07-18T03:30:17.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.149+0000 7f7920ff9700 1 -- 172.21.15.101:0/1952443512 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f792802a560 con 0x5614d92d0be0 2023-07-18T03:30:17.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.149+0000 7f7953014b80 10 client.29116 fetch_fsmap learned FSMap version 49 2023-07-18T03:30:17.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.150+0000 7f7953014b80 1 -- 172.21.15.101:0/1952443512 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5614d904f2b0 con 0x5614d92d0be0 2023-07-18T03:30:17.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.150+0000 7f7920ff9700 1 -- 172.21.15.101:0/1952443512 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5614d91de050 con 0x5614d92d0be0 2023-07-18T03:30:17.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.150+0000 7f7920ff9700 1 -- 172.21.15.101:0/1952443512 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 49) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7928030020 con 0x5614d92d0be0 2023-07-18T03:30:17.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.150+0000 7f7953014b80 10 client.29116 fetch_fsmap finished waiting for FSMap version 49 2023-07-18T03:30:17.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.150+0000 7f7953014b80 10 client.29116 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:30:17.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.150+0000 7f7953014b80 20 client.29116 populate_metadata read hostname 'smithi101' 2023-07-18T03:30:17.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.150+0000 7f7953014b80 1 --2- 172.21.15.101:0/1952443512 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5614d933fdb0 0x5614d93421c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:17.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.150+0000 7f7953014b80 4 client.29116 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:30:17.160 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.150+0000 7f7953014b80 1 -- 172.21.15.101:0/1952443512 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5614d9071f70 con 0x5614d933fdb0 2023-07-18T03:30:17.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.150+0000 7f793d39b700 1 --2- 172.21.15.101:0/1952443512 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5614d933fdb0 0x5614d93421c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:17.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.151+0000 7f7920ff9700 10 client.29116 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:30:17.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.151+0000 7f793d39b700 1 --2- 172.21.15.101:0/1952443512 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5614d933fdb0 0x5614d93421c0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12d653531e97e3f7 server_cookie=31fb92972ebc5b37 in_seq=0 out_seq=0 2023-07-18T03:30:17.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.151+0000 7f7920ff9700 1 -- 172.21.15.101:0/1952443512 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5614d9071f70 con 0x5614d933fdb0 2023-07-18T03:30:17.161 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.151+0000 7f7920ff9700 10 client.29116 handle_command_reply: tid=0 2023-07-18T03:30:17.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.151+0000 7f7953014b80 1 client.29116 shutdown 2023-07-18T03:30:17.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.151+0000 7f7953014b80 1 -- 172.21.15.101:0/1952443512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614d92d0be0 msgr2=0x5614d92e06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:17.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.151+0000 7f7953014b80 1 --2- 172.21.15.101:0/1952443512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614d92d0be0 0x5614d92e06f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f7928005740 tx=0x7f7928005770 comp rx=0 tx=0).stop 2023-07-18T03:30:17.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.151+0000 7f7953014b80 1 -- 172.21.15.101:0/1952443512 shutdown_connections 2023-07-18T03:30:17.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.151+0000 7f7953014b80 1 --2- 172.21.15.101:0/1952443512 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5614d933fdb0 0x5614d93421c0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.151+0000 7f7953014b80 1 --2- 172.21.15.101:0/1952443512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5614d92d02d0 0x5614d92e23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.151+0000 7f7920ff9700 0 client.29116 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:30:17.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.151+0000 7f7953014b80 1 --2- 172.21.15.101:0/1952443512 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614d92dbad0 0x5614d92e0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.151+0000 7f7953014b80 1 --2- 172.21.15.101:0/1952443512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614d92d0be0 0x5614d92e06f0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.151+0000 7f7953014b80 1 -- 172.21.15.101:0/1952443512 >> 172.21.15.101:0/1952443512 conn(0x5614d92c6da0 msgr2=0x5614d92d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:17.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.151+0000 7f7953014b80 1 -- 172.21.15.101:0/1952443512 shutdown_connections 2023-07-18T03:30:17.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.151+0000 7f7953014b80 1 -- 172.21.15.101:0/1952443512 wait complete. 2023-07-18T03:30:17.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.151+0000 7f7953014b80 20 client.29116 trim_cache size 0 max 16384 2023-07-18T03:30:17.164 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:30:17.165 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:30:17.165 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:30:17.165 INFO:teuthology.orchestra.run.smithi101.stdout: "0f2f19e0-0429-465c-935e-223eb09490c3": { 2023-07-18T03:30:17.165 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:30:17.166 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "0f2f19e0-0429-465c-935e-223eb09490c3", 2023-07-18T03:30:17.166 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:30:17.166 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:30:17.166 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:30:17.167 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:30:17.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.152+0000 7f793f5ff700 1 -- 172.21.15.101:0/1284920828 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f79240a1340 msgr2=0x7f79240a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:17.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.152+0000 7f793f5ff700 1 --2- 172.21.15.101:0/1284920828 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f79240a1340 0x7f79240a37f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7930009ec0 tx=0x7f793000b040 comp rx=0 tx=0).stop 2023-07-18T03:30:17.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.152+0000 7f793f5ff700 1 -- 172.21.15.101:0/1284920828 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79380ffad0 msgr2=0x7f7938072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:17.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.152+0000 7f793f5ff700 1 --2- 172.21.15.101:0/1284920828 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79380ffad0 0x7f7938072810 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f7934005910 tx=0x7f7934005720 comp rx=0 tx=0).stop 2023-07-18T03:30:17.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.152+0000 7f793f5ff700 1 -- 172.21.15.101:0/1284920828 shutdown_connections 2023-07-18T03:30:17.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.152+0000 7f793f5ff700 1 --2- 172.21.15.101:0/1284920828 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7938105ae0 0x7f79381a3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.152+0000 7f793f5ff700 1 --2- 172.21.15.101:0/1284920828 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79380ffad0 0x7f7938072810 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.152+0000 7f793f5ff700 1 --2- 172.21.15.101:0/1284920828 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f79240a1340 0x7f79240a37f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.152+0000 7f793f5ff700 1 --2- 172.21.15.101:0/1284920828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7938100480 0x7f7938072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.152+0000 7f793f5ff700 1 -- 172.21.15.101:0/1284920828 >> 172.21.15.101:0/1284920828 conn(0x7f79380fb5f0 msgr2=0x7f79380fbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:17.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.152+0000 7f793f5ff700 1 -- 172.21.15.101:0/1284920828 shutdown_connections 2023-07-18T03:30:17.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:17.152+0000 7f793f5ff700 1 -- 172.21.15.101:0/1284920828 wait complete. 2023-07-18T03:30:17.170 INFO:tasks.cephfs.filesystem:scrub status for tag:0f2f19e0-0429-465c-935e-223eb09490c3 - {'path': '/', 'tag': '0f2f19e0-0429-465c-935e-223eb09490c3', 'options': 'recursive,force'} 2023-07-18T03:30:17.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:30:17 smithi155 ceph-mon[128885]: pgmap v718: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:30:17.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:30:17 smithi179 ceph-mon[129229]: pgmap v718: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:30:17.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:30:17 smithi101 ceph-mon[126379]: pgmap v718: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:30:19.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:30:19 smithi155 ceph-mon[128885]: pgmap v719: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:30:19.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:30:19 smithi179 ceph-mon[129229]: pgmap v719: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:30:19.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:30:19 smithi101 ceph-mon[126379]: pgmap v719: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:30:21.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:30:21 smithi155 ceph-mon[128885]: pgmap v720: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:30:21.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:30:21 smithi179 ceph-mon[129229]: pgmap v720: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:30:21.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:30:21 smithi101 ceph-mon[126379]: pgmap v720: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:30:23.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:30:23 smithi155 ceph-mon[128885]: pgmap v721: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:30:23.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:30:23 smithi179 ceph-mon[129229]: pgmap v721: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:30:23.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:30:23 smithi101 ceph-mon[126379]: pgmap v721: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:30:24.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:30:24 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:30:24.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:30:24 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:30:24.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:30:24 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:30:24.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:30:24 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:30:24.550 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:30:24 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:30:24.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:30:24 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:30:24.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:30:24 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:30:24.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:30:24 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:30:24.705 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:30:24 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:30:24.705 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:30:24 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:30:24.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:30:24 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:30:24.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:30:24 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:30:24.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:30:24 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:30:24.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:30:24 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:30:24.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:30:24 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:30:25.548 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:30:25 smithi155 ceph-mon[128885]: pgmap v722: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:30:25.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:30:25 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:30:25.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:30:25 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:30:25.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:30:25 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:30:25.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:30:25 smithi179 ceph-mon[129229]: pgmap v722: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:30:25.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:30:25 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:30:25.705 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:30:25 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:30:25.705 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:30:25 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:30:25.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:30:25 smithi101 ceph-mon[126379]: pgmap v722: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:30:25.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:30:25 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:30:25.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:30:25 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:30:25.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:30:25 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:30:27.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:30:27 smithi179 ceph-mon[129229]: pgmap v723: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 87 B/s wr, 7 op/s 2023-07-18T03:30:27.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:30:27 smithi101 ceph-mon[126379]: pgmap v723: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 87 B/s wr, 7 op/s 2023-07-18T03:30:27.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:30:27 smithi155 ceph-mon[128885]: pgmap v723: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 87 B/s wr, 7 op/s 2023-07-18T03:30:29.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:30:29 smithi179 ceph-mon[129229]: pgmap v724: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:30:29.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:30:29 smithi101 ceph-mon[126379]: pgmap v724: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:30:29.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:30:29 smithi155 ceph-mon[128885]: pgmap v724: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:30:31.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:30:31 smithi179 ceph-mon[129229]: pgmap v725: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-18T03:30:31.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:30:31 smithi101 ceph-mon[126379]: pgmap v725: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-18T03:30:31.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:30:31 smithi155 ceph-mon[128885]: pgmap v725: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-18T03:30:33.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:30:33 smithi179 ceph-mon[129229]: pgmap v726: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:30:33.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:30:33 smithi101 ceph-mon[126379]: pgmap v726: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:30:33.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:30:33 smithi155 ceph-mon[128885]: pgmap v726: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:30:35.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:30:35 smithi179 ceph-mon[129229]: pgmap v727: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:30:35.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:30:35 smithi101 ceph-mon[126379]: pgmap v727: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:30:35.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:30:35 smithi155 ceph-mon[128885]: pgmap v727: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:30:37.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:30:37 smithi179 ceph-mon[129229]: pgmap v728: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:30:37.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:30:37 smithi101 ceph-mon[126379]: pgmap v728: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:30:37.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:30:37 smithi155 ceph-mon[128885]: pgmap v728: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:30:39.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:30:39 smithi179 ceph-mon[129229]: pgmap v729: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:30:39.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:30:39 smithi101 ceph-mon[126379]: pgmap v729: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:30:39.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:30:39 smithi155 ceph-mon[128885]: pgmap v729: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:30:41.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:30:41 smithi179 ceph-mon[129229]: pgmap v730: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-18T03:30:41.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:30:41 smithi101 ceph-mon[126379]: pgmap v730: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-18T03:30:41.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:30:41 smithi155 ceph-mon[128885]: pgmap v730: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-18T03:30:43.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:30:43 smithi179 ceph-mon[129229]: pgmap v731: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:30:43.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:30:43 smithi101 ceph-mon[126379]: pgmap v731: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:30:43.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:30:43 smithi155 ceph-mon[128885]: pgmap v731: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:30:45.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:30:45 smithi179 ceph-mon[129229]: pgmap v732: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:30:45.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:30:45 smithi101 ceph-mon[126379]: pgmap v732: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:30:45.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:30:45 smithi155 ceph-mon[128885]: pgmap v732: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:30:47.171 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:30:47.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.387+0000 7f617bdff700 1 -- 172.21.15.101:0/3030142624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6174105af0 msgr2=0x7f6174105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:47.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.387+0000 7f617bdff700 1 --2- 172.21.15.101:0/3030142624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6174105af0 0x7f6174105ec0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f6170009990 tx=0x7f6170009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.385 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.387+0000 7f617bdff700 1 -- 172.21.15.101:0/3030142624 shutdown_connections 2023-07-18T03:30:47.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.387+0000 7f617bdff700 1 --2- 172.21.15.101:0/3030142624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6174105af0 0x7f6174105ec0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.387+0000 7f617bdff700 1 --2- 172.21.15.101:0/3030142624 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6174100490 0x7f617410c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.387+0000 7f617bdff700 1 --2- 172.21.15.101:0/3030142624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61740ffae0 0x7f61740fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.387+0000 7f617bdff700 1 -- 172.21.15.101:0/3030142624 >> 172.21.15.101:0/3030142624 conn(0x7f61740fb5e0 msgr2=0x7f61740fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:47.386 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.387+0000 7f617bdff700 1 -- 172.21.15.101:0/3030142624 shutdown_connections 2023-07-18T03:30:47.387 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.387+0000 7f617bdff700 1 -- 172.21.15.101:0/3030142624 wait complete. 2023-07-18T03:30:47.388 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.388+0000 7f617bdff700 1 Processor -- start 2023-07-18T03:30:47.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.388+0000 7f617bdff700 1 -- start start 2023-07-18T03:30:47.389 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.389+0000 7f617bdff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61740ffae0 0x7f6174072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:47.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.389+0000 7f617bdff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6174100490 0x7f6174072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:47.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.389+0000 7f617bdff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6174105af0 0x7f61741a3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:47.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.389+0000 7f617bdff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61740735f0 con 0x7f6174105af0 2023-07-18T03:30:47.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.389+0000 7f617bdff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61741a42f0 con 0x7f61740ffae0 2023-07-18T03:30:47.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.389+0000 7f617bdff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61741a4430 con 0x7f6174100490 2023-07-18T03:30:47.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.389+0000 7f6179b9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61740ffae0 0x7f6174072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:47.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.389+0000 7f6179b9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61740ffae0 0x7f6174072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:46486/0 (socket says 172.21.15.101:46486) 2023-07-18T03:30:47.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.389+0000 7f617939a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6174100490 0x7f6174072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:47.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.389+0000 7f6179b9b700 1 -- 172.21.15.101:0/939754863 learned_addr learned my addr 172.21.15.101:0/939754863 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:30:47.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.389+0000 7f617a39c700 1 --2- 172.21.15.101:0/939754863 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6174105af0 0x7f61741a3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:47.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.389+0000 7f6179b9b700 1 -- 172.21.15.101:0/939754863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6174100490 msgr2=0x7f6174072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:47.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.389+0000 7f6179b9b700 1 --2- 172.21.15.101:0/939754863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6174100490 0x7f6174072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.389+0000 7f6179b9b700 1 -- 172.21.15.101:0/939754863 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6174105af0 msgr2=0x7f61741a3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:47.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.389+0000 7f6179b9b700 1 --2- 172.21.15.101:0/939754863 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6174105af0 0x7f61741a3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.389+0000 7f6179b9b700 1 -- 172.21.15.101:0/939754863 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61700095f0 con 0x7f61740ffae0 2023-07-18T03:30:47.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.389+0000 7f617939a700 1 --2- 172.21.15.101:0/939754863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6174100490 0x7f6174072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:30:47.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.390+0000 7f6179b9b700 1 --2- 172.21.15.101:0/939754863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61740ffae0 0x7f6174072810 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f6170005720 tx=0x7f6170005750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:47.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.390+0000 7f616affd700 1 -- 172.21.15.101:0/939754863 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617001d070 con 0x7f61740ffae0 2023-07-18T03:30:47.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.390+0000 7f616affd700 1 -- 172.21.15.101:0/939754863 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6170022470 con 0x7f61740ffae0 2023-07-18T03:30:47.393 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.390+0000 7f617bdff700 1 -- 172.21.15.101:0/939754863 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61741a4680 con 0x7f61740ffae0 2023-07-18T03:30:47.394 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.390+0000 7f616affd700 1 -- 172.21.15.101:0/939754863 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6170017650 con 0x7f61740ffae0 2023-07-18T03:30:47.394 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.390+0000 7f617bdff700 1 -- 172.21.15.101:0/939754863 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61741a4aa0 con 0x7f61740ffae0 2023-07-18T03:30:47.396 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.393+0000 7f616affd700 1 -- 172.21.15.101:0/939754863 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f61700225e0 con 0x7f61740ffae0 2023-07-18T03:30:47.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.395+0000 7f618f687b80 1 Processor -- start 2023-07-18T03:30:47.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.395+0000 7f618f687b80 1 -- start start 2023-07-18T03:30:47.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.395+0000 7f616affd700 1 --2- 172.21.15.101:0/939754863 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f61600a12c0 0x7f61600a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:47.397 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.395+0000 7f618f687b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5649ab97f2d0 0x5649ab97f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:47.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.395+0000 7f618f687b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649ab97fbe0 0x5649ab98a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:47.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.395+0000 7f618f687b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649ab98aad0 0x5649ab98ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:47.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.395+0000 7f618f687b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649ab885cb0 con 0x5649ab97f2d0 2023-07-18T03:30:47.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.395+0000 7f618f687b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649ab8d8a40 con 0x5649ab98aad0 2023-07-18T03:30:47.398 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.395+0000 7f618f687b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649ab8d1ed0 con 0x5649ab97fbe0 2023-07-18T03:30:47.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.396+0000 7f616affd700 1 -- 172.21.15.101:0/939754863 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f61700ce430 con 0x7f61740ffae0 2023-07-18T03:30:47.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.396+0000 7f6179b9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5649ab97f2d0 0x5649ab97f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:47.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.396+0000 7f617939a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649ab97fbe0 0x5649ab98a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:47.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.396+0000 7f617939a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649ab97fbe0 0x5649ab98a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:59240/0 (socket says 172.21.15.101:59240) 2023-07-18T03:30:47.399 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.396+0000 7f617939a700 1 -- 172.21.15.101:0/3570681420 learned_addr learned my addr 172.21.15.101:0/3570681420 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:30:47.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.396+0000 7f617a39c700 1 --2- 172.21.15.101:0/939754863 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f61600a12c0 0x7f61600a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:47.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.396+0000 7f617a39c700 1 --2- 172.21.15.101:0/3570681420 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649ab98aad0 0x5649ab98ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:47.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.396+0000 7f6179b9b700 1 -- 172.21.15.101:0/3570681420 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649ab97fbe0 msgr2=0x5649ab98a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:47.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.396+0000 7f6179b9b700 1 --2- 172.21.15.101:0/3570681420 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649ab97fbe0 0x5649ab98a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.400 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.396+0000 7f6179b9b700 1 -- 172.21.15.101:0/3570681420 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649ab98aad0 msgr2=0x5649ab98ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:47.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.396+0000 7f6179b9b700 1 --2- 172.21.15.101:0/3570681420 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649ab98aad0 0x5649ab98ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.396+0000 7f6179b9b700 1 -- 172.21.15.101:0/3570681420 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649ab6ffde0 con 0x5649ab97f2d0 2023-07-18T03:30:47.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.396+0000 7f617a39c700 1 --2- 172.21.15.101:0/3570681420 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649ab98aad0 0x5649ab98ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:30:47.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.396+0000 7f617939a700 1 --2- 172.21.15.101:0/3570681420 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649ab97fbe0 0x5649ab98a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:30:47.401 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.396+0000 7f6179b9b700 1 --2- 172.21.15.101:0/3570681420 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5649ab97f2d0 0x5649ab97f6a0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f61700cb770 tx=0x7f6170009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a52689eff5bc65f8 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:47.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.396+0000 7f617a39c700 1 --2- 172.21.15.101:0/939754863 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f61600a12c0 0x7f61600a3770 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f616c009ec0 tx=0x7f616c009920 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:47.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.397+0000 7f615ffff700 1 -- 172.21.15.101:0/3570681420 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617009aae0 con 0x5649ab97f2d0 2023-07-18T03:30:47.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.397+0000 7f615ffff700 1 -- 172.21.15.101:0/3570681420 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f61700ad5f0 con 0x5649ab97f2d0 2023-07-18T03:30:47.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.397+0000 7f615ffff700 1 -- 172.21.15.101:0/3570681420 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61700a3540 con 0x5649ab97f2d0 2023-07-18T03:30:47.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.397+0000 7f618f687b80 1 -- 172.21.15.101:0/3570681420 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5649ab97f2d0 msgr2=0x5649ab97f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:47.402 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.397+0000 7f618f687b80 1 --2- 172.21.15.101:0/3570681420 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5649ab97f2d0 0x5649ab97f6a0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f61700cb770 tx=0x7f6170009310 comp rx=0 tx=0).stop 2023-07-18T03:30:47.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.397+0000 7f618f687b80 1 -- 172.21.15.101:0/3570681420 shutdown_connections 2023-07-18T03:30:47.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.397+0000 7f618f687b80 1 --2- 172.21.15.101:0/3570681420 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649ab98aad0 0x5649ab98ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.397+0000 7f618f687b80 1 --2- 172.21.15.101:0/3570681420 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5649ab97f2d0 0x5649ab97f6a0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.397+0000 7f618f687b80 1 --2- 172.21.15.101:0/3570681420 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649ab97fbe0 0x5649ab98a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.403 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.397+0000 7f618f687b80 1 -- 172.21.15.101:0/3570681420 >> 172.21.15.101:0/3570681420 conn(0x5649ab975da0 msgr2=0x5649ab97b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:47.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.398+0000 7f618f687b80 1 -- 172.21.15.101:0/3570681420 shutdown_connections 2023-07-18T03:30:47.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.398+0000 7f618f687b80 1 -- 172.21.15.101:0/3570681420 wait complete. 2023-07-18T03:30:47.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.398+0000 7f618f687b80 1 -- start start 2023-07-18T03:30:47.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.398+0000 7f618f687b80 1 Processor -- start 2023-07-18T03:30:47.404 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.399+0000 7f618f687b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649ab97f2d0 0x5649ab98aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:47.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.399+0000 7f618f687b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649ab97fbe0 0x5649ab98b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:47.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.399+0000 7f618f687b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5649ab98aad0 0x5649ab98b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:47.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.399+0000 7f618f687b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649ab871ac0 con 0x5649ab98aad0 2023-07-18T03:30:47.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.399+0000 7f618f687b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649ab8717c0 con 0x5649ab97fbe0 2023-07-18T03:30:47.405 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.399+0000 7f618f687b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649ab755920 con 0x5649ab97f2d0 2023-07-18T03:30:47.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.399+0000 7f618f687b80 10 client.0 _ll_register_callbacks cb 0x5649ab86f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:30:47.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.399+0000 7f6179b9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649ab97f2d0 0x5649ab98aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:47.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.399+0000 7f617939a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649ab97fbe0 0x5649ab98b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:47.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.399+0000 7f617a39c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5649ab98aad0 0x5649ab98b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:47.406 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.399+0000 7f6179b9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649ab97f2d0 0x5649ab98aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:59244/0 (socket says 172.21.15.101:59244) 2023-07-18T03:30:47.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.399+0000 7f6179b9b700 1 -- 172.21.15.101:0/3180077391 learned_addr learned my addr 172.21.15.101:0/3180077391 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:30:47.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.399+0000 7f6179b9b700 1 -- 172.21.15.101:0/3180077391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649ab97fbe0 msgr2=0x5649ab98b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:47.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.399+0000 7f6179b9b700 1 --2- 172.21.15.101:0/3180077391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649ab97fbe0 0x5649ab98b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.399+0000 7f6179b9b700 1 -- 172.21.15.101:0/3180077391 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5649ab98aad0 msgr2=0x5649ab98b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:47.407 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.399+0000 7f6179b9b700 1 --2- 172.21.15.101:0/3180077391 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5649ab98aad0 0x5649ab98b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.399+0000 7f6179b9b700 1 -- 172.21.15.101:0/3180077391 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6164009d10 con 0x5649ab97f2d0 2023-07-18T03:30:47.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.399+0000 7f617939a700 1 --2- 172.21.15.101:0/3180077391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649ab97fbe0 0x5649ab98b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:30:47.408 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.399+0000 7f617a39c700 1 --2- 172.21.15.101:0/3180077391 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5649ab98aad0 0x5649ab98b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:30:47.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.400+0000 7f6179b9b700 1 --2- 172.21.15.101:0/3180077391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649ab97f2d0 0x5649ab98aea0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f61700cb7d0 tx=0x7f61700cbec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c677dc99b48bacf0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:47.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.400+0000 7f615d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:30:47.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.400+0000 7f615d7fa700 1 -- 172.21.15.101:0/3180077391 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61700ad410 con 0x5649ab97f2d0 2023-07-18T03:30:47.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.400+0000 7f615d7fa700 1 -- 172.21.15.101:0/3180077391 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f61700a3540 con 0x5649ab97f2d0 2023-07-18T03:30:47.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.400+0000 7f618f687b80 1 -- 172.21.15.101:0/3180077391 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5649ab82b4f0 con 0x5649ab97f2d0 2023-07-18T03:30:47.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.400+0000 7f615d7fa700 1 -- 172.21.15.101:0/3180077391 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f617009b180 con 0x5649ab97f2d0 2023-07-18T03:30:47.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.400+0000 7f618f687b80 10 client.29128 fetch_fsmap learned FSMap version 49 2023-07-18T03:30:47.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.400+0000 7f618f687b80 1 -- 172.21.15.101:0/3180077391 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f61741a4aa0 con 0x5649ab97f2d0 2023-07-18T03:30:47.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.400+0000 7f615d7fa700 1 -- 172.21.15.101:0/3180077391 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5649ab82b4f0 con 0x5649ab97f2d0 2023-07-18T03:30:47.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.400+0000 7f615d7fa700 1 -- 172.21.15.101:0/3180077391 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 49) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f617009b3e0 con 0x5649ab97f2d0 2023-07-18T03:30:47.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.400+0000 7f618f687b80 10 client.29128 fetch_fsmap finished waiting for FSMap version 49 2023-07-18T03:30:47.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.400+0000 7f618f687b80 10 client.29128 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:30:47.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.400+0000 7f618f687b80 20 client.29128 populate_metadata read hostname 'smithi101' 2023-07-18T03:30:47.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.400+0000 7f618f687b80 1 --2- 172.21.15.101:0/3180077391 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5649ab999cb0 0x5649ab99c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:47.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.400+0000 7f618f687b80 4 client.29128 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:30:47.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.400+0000 7f618f687b80 1 -- 172.21.15.101:0/3180077391 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5649ab8caeb0 con 0x5649ab999cb0 2023-07-18T03:30:47.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.401+0000 7f617939a700 1 --2- 172.21.15.101:0/3180077391 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5649ab999cb0 0x5649ab99c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:47.413 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.401+0000 7f615d7fa700 10 client.29128 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:30:47.414 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.401+0000 7f617939a700 1 --2- 172.21.15.101:0/3180077391 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5649ab999cb0 0x5649ab99c0c0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=444cf5619fd11f57 server_cookie=9529c9f0006c93ec in_seq=0 out_seq=0 2023-07-18T03:30:47.414 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.402+0000 7f615d7fa700 1 -- 172.21.15.101:0/3180077391 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5649ab8caeb0 con 0x5649ab999cb0 2023-07-18T03:30:47.414 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.402+0000 7f615d7fa700 10 client.29128 handle_command_reply: tid=0 2023-07-18T03:30:47.414 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.402+0000 7f618f687b80 1 client.29128 shutdown 2023-07-18T03:30:47.414 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.402+0000 7f618f687b80 1 -- 172.21.15.101:0/3180077391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649ab97f2d0 msgr2=0x5649ab98aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:47.415 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.402+0000 7f618f687b80 1 --2- 172.21.15.101:0/3180077391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649ab97f2d0 0x5649ab98aea0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f61700cb7d0 tx=0x7f61700cbec0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.415 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.402+0000 7f618f687b80 1 -- 172.21.15.101:0/3180077391 shutdown_connections 2023-07-18T03:30:47.415 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.402+0000 7f618f687b80 1 --2- 172.21.15.101:0/3180077391 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5649ab999cb0 0x5649ab99c0c0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.415 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.402+0000 7f618f687b80 1 --2- 172.21.15.101:0/3180077391 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5649ab98aad0 0x5649ab98b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.415 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.402+0000 7f618f687b80 1 --2- 172.21.15.101:0/3180077391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649ab97f2d0 0x5649ab98aea0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.402+0000 7f618f687b80 1 --2- 172.21.15.101:0/3180077391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649ab97fbe0 0x5649ab98b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.402+0000 7f615d7fa700 0 client.29128 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:30:47.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.402+0000 7f618f687b80 1 -- 172.21.15.101:0/3180077391 >> 172.21.15.101:0/3180077391 conn(0x5649ab975da0 msgr2=0x5649ab97b230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:47.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.403+0000 7f618f687b80 1 -- 172.21.15.101:0/3180077391 shutdown_connections 2023-07-18T03:30:47.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.403+0000 7f618f687b80 1 -- 172.21.15.101:0/3180077391 wait complete. 2023-07-18T03:30:47.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.403+0000 7f618f687b80 20 client.29128 trim_cache size 0 max 16384 2023-07-18T03:30:47.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.420+0000 7f618f687b80 1 Processor -- start 2023-07-18T03:30:47.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.420+0000 7f618f687b80 1 -- start start 2023-07-18T03:30:47.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.420+0000 7f618f687b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649ab97f2d0 0x5649ab98eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:47.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.420+0000 7f618f687b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5649ab97fbe0 0x5649ab995680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:47.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.420+0000 7f618f687b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649ab98aad0 0x5649ab984de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:47.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.420+0000 7f618f687b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649ab980540 con 0x5649ab97fbe0 2023-07-18T03:30:47.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.420+0000 7f618f687b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649ab980160 con 0x5649ab98aad0 2023-07-18T03:30:47.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.420+0000 7f618f687b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649ab975970 con 0x5649ab97f2d0 2023-07-18T03:30:47.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.420+0000 7f6179b9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649ab97f2d0 0x5649ab98eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:47.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.421+0000 7f6179b9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649ab97f2d0 0x5649ab98eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:59260/0 (socket says 172.21.15.101:59260) 2023-07-18T03:30:47.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.421+0000 7f6179b9b700 1 -- 172.21.15.101:0/2309924544 learned_addr learned my addr 172.21.15.101:0/2309924544 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:30:47.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.421+0000 7f617a39c700 1 --2- 172.21.15.101:0/2309924544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649ab98aad0 0x5649ab984de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:47.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.421+0000 7f617939a700 1 --2- 172.21.15.101:0/2309924544 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5649ab97fbe0 0x5649ab995680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:47.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.421+0000 7f6179b9b700 1 -- 172.21.15.101:0/2309924544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649ab98aad0 msgr2=0x5649ab984de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:47.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.421+0000 7f6179b9b700 1 --2- 172.21.15.101:0/2309924544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649ab98aad0 0x5649ab984de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.421+0000 7f6179b9b700 1 -- 172.21.15.101:0/2309924544 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5649ab97fbe0 msgr2=0x5649ab995680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:47.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.421+0000 7f6179b9b700 1 --2- 172.21.15.101:0/2309924544 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5649ab97fbe0 0x5649ab995680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.421 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.421+0000 7f6179b9b700 1 -- 172.21.15.101:0/2309924544 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649ab6e8c60 con 0x5649ab97f2d0 2023-07-18T03:30:47.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.421+0000 7f6179b9b700 1 --2- 172.21.15.101:0/2309924544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649ab97f2d0 0x5649ab98eed0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f617009b110 tx=0x7f6170009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=132a1ed417f5863 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:47.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.421+0000 7f6168bff700 1 -- 172.21.15.101:0/2309924544 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617009aae0 con 0x5649ab97f2d0 2023-07-18T03:30:47.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.421+0000 7f6168bff700 1 -- 172.21.15.101:0/2309924544 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f61700a3540 con 0x5649ab97f2d0 2023-07-18T03:30:47.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.421+0000 7f6168bff700 1 -- 172.21.15.101:0/2309924544 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617009f1b0 con 0x5649ab97f2d0 2023-07-18T03:30:47.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.422+0000 7f618f687b80 1 -- 172.21.15.101:0/2309924544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649ab97f2d0 msgr2=0x5649ab98eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:47.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.422+0000 7f618f687b80 1 --2- 172.21.15.101:0/2309924544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649ab97f2d0 0x5649ab98eed0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f617009b110 tx=0x7f6170009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.422+0000 7f618f687b80 1 -- 172.21.15.101:0/2309924544 shutdown_connections 2023-07-18T03:30:47.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.422+0000 7f618f687b80 1 --2- 172.21.15.101:0/2309924544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649ab98aad0 0x5649ab984de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.422+0000 7f618f687b80 1 --2- 172.21.15.101:0/2309924544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649ab97f2d0 0x5649ab98eed0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.422+0000 7f618f687b80 1 --2- 172.21.15.101:0/2309924544 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5649ab97fbe0 0x5649ab995680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.422+0000 7f618f687b80 1 -- 172.21.15.101:0/2309924544 >> 172.21.15.101:0/2309924544 conn(0x5649ab975da0 msgr2=0x5649ab9893a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:47.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.422+0000 7f618f687b80 1 -- 172.21.15.101:0/2309924544 shutdown_connections 2023-07-18T03:30:47.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.422+0000 7f618f687b80 1 -- 172.21.15.101:0/2309924544 wait complete. 2023-07-18T03:30:47.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.422+0000 7f618f687b80 1 -- start start 2023-07-18T03:30:47.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.423+0000 7f618f687b80 1 Processor -- start 2023-07-18T03:30:47.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.423+0000 7f618f687b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649ab97f2d0 0x5649ab9e1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:47.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.423+0000 7f618f687b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649ab97fbe0 0x5649ab9e2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:47.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.423+0000 7f618f687b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5649ab98aad0 0x5649ab9ecb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:47.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.423+0000 7f618f687b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649ab9e27c0 con 0x5649ab98aad0 2023-07-18T03:30:47.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.423+0000 7f618f687b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649ab9e2930 con 0x5649ab97fbe0 2023-07-18T03:30:47.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.423+0000 7f618f687b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649ab9ed0b0 con 0x5649ab97f2d0 2023-07-18T03:30:47.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.423+0000 7f618f687b80 10 client.0 _ll_register_callbacks cb 0x5649ab86f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:30:47.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.423+0000 7f617939a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649ab97fbe0 0x5649ab9e2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:47.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.423+0000 7f6179b9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649ab97f2d0 0x5649ab9e1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:47.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.423+0000 7f6179b9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649ab97f2d0 0x5649ab9e1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:59272/0 (socket says 172.21.15.101:59272) 2023-07-18T03:30:47.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.423+0000 7f6179b9b700 1 -- 172.21.15.101:0/1726844357 learned_addr learned my addr 172.21.15.101:0/1726844357 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:30:47.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.423+0000 7f617939a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649ab97fbe0 0x5649ab9e2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:46514/0 (socket says 172.21.15.101:46514) 2023-07-18T03:30:47.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.423+0000 7f617a39c700 1 --2- 172.21.15.101:0/1726844357 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5649ab98aad0 0x5649ab9ecb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:47.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.424+0000 7f6179b9b700 1 -- 172.21.15.101:0/1726844357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649ab97fbe0 msgr2=0x5649ab9e2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:47.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.424+0000 7f6179b9b700 1 --2- 172.21.15.101:0/1726844357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649ab97fbe0 0x5649ab9e2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.424+0000 7f6179b9b700 1 -- 172.21.15.101:0/1726844357 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5649ab98aad0 msgr2=0x5649ab9ecb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:47.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.424+0000 7f6179b9b700 1 --2- 172.21.15.101:0/1726844357 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5649ab98aad0 0x5649ab9ecb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.424+0000 7f6179b9b700 1 -- 172.21.15.101:0/1726844357 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61741a4aa0 con 0x5649ab97f2d0 2023-07-18T03:30:47.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.424+0000 7f617939a700 1 --2- 172.21.15.101:0/1726844357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649ab97fbe0 0x5649ab9e2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:30:47.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.424+0000 7f617a39c700 1 --2- 172.21.15.101:0/1726844357 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5649ab98aad0 0x5649ab9ecb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:30:47.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.424+0000 7f6179b9b700 1 --2- 172.21.15.101:0/1726844357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649ab97f2d0 0x5649ab9e1ac0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f6170096f00 tx=0x7f617009b180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9346e45ed95d0313 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:47.432 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.424+0000 7f615d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:30:47.432 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.424+0000 7f615d7fa700 1 -- 172.21.15.101:0/1726844357 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61700b7070 con 0x5649ab97f2d0 2023-07-18T03:30:47.432 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.424+0000 7f615d7fa700 1 -- 172.21.15.101:0/1726844357 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f617009a8b0 con 0x5649ab97f2d0 2023-07-18T03:30:47.432 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.424+0000 7f618f687b80 1 -- 172.21.15.101:0/1726844357 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f61700ad5f0 con 0x5649ab97f2d0 2023-07-18T03:30:47.432 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.424+0000 7f615d7fa700 1 -- 172.21.15.101:0/1726844357 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f617009f1b0 con 0x5649ab97f2d0 2023-07-18T03:30:47.433 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.424+0000 7f618f687b80 10 client.29134 fetch_fsmap learned FSMap version 49 2023-07-18T03:30:47.433 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.424+0000 7f618f687b80 1 -- 172.21.15.101:0/1726844357 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5649ab726250 con 0x5649ab97f2d0 2023-07-18T03:30:47.433 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.424+0000 7f615d7fa700 1 -- 172.21.15.101:0/1726844357 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f61700ad5f0 con 0x5649ab97f2d0 2023-07-18T03:30:47.433 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.424+0000 7f615d7fa700 1 -- 172.21.15.101:0/1726844357 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 49) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f61700a30b0 con 0x5649ab97f2d0 2023-07-18T03:30:47.433 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.425+0000 7f618f687b80 10 client.29134 fetch_fsmap finished waiting for FSMap version 49 2023-07-18T03:30:47.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.425+0000 7f618f687b80 10 client.29134 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:30:47.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.425+0000 7f618f687b80 20 client.29134 populate_metadata read hostname 'smithi101' 2023-07-18T03:30:47.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.425+0000 7f618f687b80 1 --2- 172.21.15.101:0/1726844357 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5649ab9ee310 0x5649ab9f06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:47.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.425+0000 7f618f687b80 4 client.29134 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:30:47.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.425+0000 7f618f687b80 1 -- 172.21.15.101:0/1726844357 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5649ab88d050 con 0x5649ab9ee310 2023-07-18T03:30:47.435 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.425+0000 7f617939a700 1 --2- 172.21.15.101:0/1726844357 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5649ab9ee310 0x5649ab9f06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:47.435 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.425+0000 7f615d7fa700 10 client.29134 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:30:47.435 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.425+0000 7f617939a700 1 --2- 172.21.15.101:0/1726844357 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5649ab9ee310 0x5649ab9f06f0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=943e166c1f6e22d2 server_cookie=21e1ac09c6453d4c in_seq=0 out_seq=0 2023-07-18T03:30:47.435 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.426+0000 7f615d7fa700 1 -- 172.21.15.101:0/1726844357 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5649ab88d050 con 0x5649ab9ee310 2023-07-18T03:30:47.435 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.426+0000 7f615d7fa700 10 client.29134 handle_command_reply: tid=0 2023-07-18T03:30:47.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.426+0000 7f618f687b80 1 client.29134 shutdown 2023-07-18T03:30:47.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.426+0000 7f618f687b80 1 -- 172.21.15.101:0/1726844357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649ab97f2d0 msgr2=0x5649ab9e1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:47.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.426+0000 7f618f687b80 1 --2- 172.21.15.101:0/1726844357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649ab97f2d0 0x5649ab9e1ac0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f6170096f00 tx=0x7f617009b180 comp rx=0 tx=0).stop 2023-07-18T03:30:47.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.426+0000 7f618f687b80 1 -- 172.21.15.101:0/1726844357 shutdown_connections 2023-07-18T03:30:47.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.426+0000 7f618f687b80 1 --2- 172.21.15.101:0/1726844357 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5649ab9ee310 0x5649ab9f06f0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.426+0000 7f618f687b80 1 --2- 172.21.15.101:0/1726844357 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5649ab98aad0 0x5649ab9ecb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.426+0000 7f618f687b80 1 --2- 172.21.15.101:0/1726844357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649ab97f2d0 0x5649ab9e1ac0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.426+0000 7f618f687b80 1 --2- 172.21.15.101:0/1726844357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649ab97fbe0 0x5649ab9e2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.426+0000 7f615d7fa700 0 client.29134 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:30:47.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.426+0000 7f618f687b80 1 -- 172.21.15.101:0/1726844357 >> 172.21.15.101:0/1726844357 conn(0x5649ab975da0 msgr2=0x5649ab995680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:47.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.426+0000 7f618f687b80 1 -- 172.21.15.101:0/1726844357 shutdown_connections 2023-07-18T03:30:47.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.426+0000 7f618f687b80 1 -- 172.21.15.101:0/1726844357 wait complete. 2023-07-18T03:30:47.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.426+0000 7f618f687b80 20 client.29134 trim_cache size 0 max 16384 2023-07-18T03:30:47.438 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:30:47.439 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:30:47.439 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:30:47.439 INFO:teuthology.orchestra.run.smithi101.stdout: "0f2f19e0-0429-465c-935e-223eb09490c3": { 2023-07-18T03:30:47.439 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:30:47.440 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "0f2f19e0-0429-465c-935e-223eb09490c3", 2023-07-18T03:30:47.440 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:30:47.440 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:30:47.444 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:30:47.444 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:30:47.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.426+0000 7f617bdff700 1 -- 172.21.15.101:0/939754863 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f61600a12c0 msgr2=0x7f61600a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:47.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.427+0000 7f617bdff700 1 --2- 172.21.15.101:0/939754863 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f61600a12c0 0x7f61600a3770 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f616c009ec0 tx=0x7f616c009920 comp rx=0 tx=0).stop 2023-07-18T03:30:47.446 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.427+0000 7f617bdff700 1 -- 172.21.15.101:0/939754863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61740ffae0 msgr2=0x7f6174072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:47.446 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.427+0000 7f617bdff700 1 --2- 172.21.15.101:0/939754863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61740ffae0 0x7f6174072810 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f6170005720 tx=0x7f6170005750 comp rx=0 tx=0).stop 2023-07-18T03:30:47.446 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.427+0000 7f617bdff700 1 -- 172.21.15.101:0/939754863 shutdown_connections 2023-07-18T03:30:47.446 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.427+0000 7f617bdff700 1 --2- 172.21.15.101:0/939754863 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f61600a12c0 0x7f61600a3770 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.446 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.427+0000 7f617bdff700 1 --2- 172.21.15.101:0/939754863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61740ffae0 0x7f6174072810 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.427+0000 7f617bdff700 1 --2- 172.21.15.101:0/939754863 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6174105af0 0x7f61741a3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.427+0000 7f617bdff700 1 --2- 172.21.15.101:0/939754863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6174100490 0x7f6174072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.427+0000 7f617bdff700 1 -- 172.21.15.101:0/939754863 >> 172.21.15.101:0/939754863 conn(0x7f61740fb5e0 msgr2=0x7f61740fbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:47.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.427+0000 7f617bdff700 1 -- 172.21.15.101:0/939754863 shutdown_connections 2023-07-18T03:30:47.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:30:47.427+0000 7f617bdff700 1 -- 172.21.15.101:0/939754863 wait complete. 2023-07-18T03:30:47.448 INFO:tasks.cephfs.filesystem:scrub status for tag:0f2f19e0-0429-465c-935e-223eb09490c3 - {'path': '/', 'tag': '0f2f19e0-0429-465c-935e-223eb09490c3', 'options': 'recursive,force'} 2023-07-18T03:30:47.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:30:47 smithi101 ceph-mon[126379]: pgmap v733: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:30:47.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:30:47 smithi179 ceph-mon[129229]: pgmap v733: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:30:47.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:30:47 smithi155 ceph-mon[128885]: pgmap v733: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:30:49.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:30:49 smithi179 ceph-mon[129229]: pgmap v734: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:30:49.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:30:49 smithi101 ceph-mon[126379]: pgmap v734: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:30:49.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:30:49 smithi155 ceph-mon[128885]: pgmap v734: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:30:51.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:30:51 smithi179 ceph-mon[129229]: pgmap v735: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-18T03:30:51.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:30:51 smithi101 ceph-mon[126379]: pgmap v735: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-18T03:30:51.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:30:51 smithi155 ceph-mon[128885]: pgmap v735: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-18T03:30:53.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:30:53 smithi179 ceph-mon[129229]: pgmap v736: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:30:53.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:30:53 smithi101 ceph-mon[126379]: pgmap v736: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:30:53.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:30:53 smithi155 ceph-mon[128885]: pgmap v736: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:30:55.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:30:55 smithi179 ceph-mon[129229]: pgmap v737: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:30:55.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:30:55 smithi101 ceph-mon[126379]: pgmap v737: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:30:55.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:30:55 smithi155 ceph-mon[128885]: pgmap v737: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:30:57.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:30:57 smithi179 ceph-mon[129229]: pgmap v738: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:30:57.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:30:57 smithi101 ceph-mon[126379]: pgmap v738: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:30:57.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:30:57 smithi155 ceph-mon[128885]: pgmap v738: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:30:59.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:30:59 smithi179 ceph-mon[129229]: pgmap v739: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-07-18T03:30:59.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:30:59 smithi101 ceph-mon[126379]: pgmap v739: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-07-18T03:30:59.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:30:59 smithi155 ceph-mon[128885]: pgmap v739: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-07-18T03:31:01.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:31:01 smithi179 ceph-mon[129229]: pgmap v740: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2023-07-18T03:31:01.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:31:01 smithi101 ceph-mon[126379]: pgmap v740: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2023-07-18T03:31:01.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:31:01 smithi155 ceph-mon[128885]: pgmap v740: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2023-07-18T03:31:03.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:31:03 smithi179 ceph-mon[129229]: pgmap v741: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-07-18T03:31:03.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:31:03 smithi101 ceph-mon[126379]: pgmap v741: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-07-18T03:31:03.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:31:03 smithi155 ceph-mon[128885]: pgmap v741: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-07-18T03:31:05.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:31:05 smithi179 ceph-mon[129229]: pgmap v742: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-07-18T03:31:05.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:31:05 smithi101 ceph-mon[126379]: pgmap v742: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-07-18T03:31:05.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:31:05 smithi155 ceph-mon[128885]: pgmap v742: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-07-18T03:31:07.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:31:07 smithi179 ceph-mon[129229]: pgmap v743: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-07-18T03:31:07.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:31:07 smithi101 ceph-mon[126379]: pgmap v743: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-07-18T03:31:07.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:31:07 smithi155 ceph-mon[128885]: pgmap v743: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-07-18T03:31:09.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:31:09 smithi179 ceph-mon[129229]: pgmap v744: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-07-18T03:31:09.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:31:09 smithi101 ceph-mon[126379]: pgmap v744: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-07-18T03:31:09.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:31:09 smithi155 ceph-mon[128885]: pgmap v744: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-07-18T03:31:11.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:31:11 smithi179 ceph-mon[129229]: pgmap v745: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 21 op/s 2023-07-18T03:31:11.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:31:11 smithi179 ceph-mon[129229]: osdmap e127: 12 total, 12 up, 12 in 2023-07-18T03:31:11.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:31:11 smithi101 ceph-mon[126379]: pgmap v745: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 21 op/s 2023-07-18T03:31:11.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:31:11 smithi101 ceph-mon[126379]: osdmap e127: 12 total, 12 up, 12 in 2023-07-18T03:31:11.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:31:11 smithi155 ceph-mon[128885]: pgmap v745: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 21 op/s 2023-07-18T03:31:11.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:31:11 smithi155 ceph-mon[128885]: osdmap e127: 12 total, 12 up, 12 in 2023-07-18T03:31:13.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:31:13 smithi179 ceph-mon[129229]: pgmap v747: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 27 KiB/s wr, 21 op/s 2023-07-18T03:31:13.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:31:13 smithi101 ceph-mon[126379]: pgmap v747: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 27 KiB/s wr, 21 op/s 2023-07-18T03:31:13.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:31:13 smithi155 ceph-mon[128885]: pgmap v747: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 27 KiB/s wr, 21 op/s 2023-07-18T03:31:15.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:31:15 smithi179 ceph-mon[129229]: pgmap v748: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 27 KiB/s wr, 21 op/s 2023-07-18T03:31:15.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:31:15 smithi101 ceph-mon[126379]: pgmap v748: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 27 KiB/s wr, 21 op/s 2023-07-18T03:31:15.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:31:15 smithi155 ceph-mon[128885]: pgmap v748: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 27 KiB/s wr, 21 op/s 2023-07-18T03:31:17.448 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:31:17.657 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.660+0000 7f24c8fff700 1 -- 172.21.15.101:0/1493233566 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f24c4105ae0 msgr2=0x7f24c4105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:17.657 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.660+0000 7f24c8fff700 1 --2- 172.21.15.101:0/1493233566 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f24c4105ae0 0x7f24c4105eb0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f24b4009990 tx=0x7f24b4009ca0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.657 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.660+0000 7f24c8fff700 1 -- 172.21.15.101:0/1493233566 shutdown_connections 2023-07-18T03:31:17.658 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.660+0000 7f24c8fff700 1 --2- 172.21.15.101:0/1493233566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24c4100480 0x7f24c410c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.658 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.660+0000 7f24c8fff700 1 --2- 172.21.15.101:0/1493233566 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f24c4105ae0 0x7f24c4105eb0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.658 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.660+0000 7f24c8fff700 1 --2- 172.21.15.101:0/1493233566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24c40ffad0 0x7f24c40fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.658 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.660+0000 7f24c8fff700 1 -- 172.21.15.101:0/1493233566 >> 172.21.15.101:0/1493233566 conn(0x7f24c40fb5f0 msgr2=0x7f24c40fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:17.659 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.660+0000 7f24c8fff700 1 -- 172.21.15.101:0/1493233566 shutdown_connections 2023-07-18T03:31:17.659 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.660+0000 7f24c8fff700 1 -- 172.21.15.101:0/1493233566 wait complete. 2023-07-18T03:31:17.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.661+0000 7f24c8fff700 1 Processor -- start 2023-07-18T03:31:17.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.662+0000 7f24c8fff700 1 -- start start 2023-07-18T03:31:17.661 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.662+0000 7f24c8fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24c40ffad0 0x7f24c4197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:17.662 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.662+0000 7f24c8fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f24c4100480 0x7f24c41983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:17.662 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.662+0000 7f24c8fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24c4105ae0 0x7f24c419c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:17.662 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.662+0000 7f24c8fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c419ce40 con 0x7f24c4100480 2023-07-18T03:31:17.662 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.662+0000 7f24c8fff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c419cfb0 con 0x7f24c40ffad0 2023-07-18T03:31:17.662 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.662+0000 7f24c8fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c419d170 con 0x7f24c4105ae0 2023-07-18T03:31:17.663 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.663+0000 7f24c259c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24c40ffad0 0x7f24c4197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:17.663 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.663+0000 7f24c259c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24c40ffad0 0x7f24c4197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47348/0 (socket says 172.21.15.101:47348) 2023-07-18T03:31:17.663 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.663+0000 7f24c259c700 1 -- 172.21.15.101:0/1990715678 learned_addr learned my addr 172.21.15.101:0/1990715678 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:31:17.663 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.663+0000 7f24c1d9b700 1 --2- 172.21.15.101:0/1990715678 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f24c4100480 0x7f24c41983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:17.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.663+0000 7f24c2d9d700 1 --2- 172.21.15.101:0/1990715678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24c4105ae0 0x7f24c419c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:17.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.663+0000 7f24c259c700 1 -- 172.21.15.101:0/1990715678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24c4105ae0 msgr2=0x7f24c419c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:17.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.663+0000 7f24c259c700 1 --2- 172.21.15.101:0/1990715678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24c4105ae0 0x7f24c419c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.664 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.663+0000 7f24c259c700 1 -- 172.21.15.101:0/1990715678 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f24c4100480 msgr2=0x7f24c41983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:17.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.663+0000 7f24c259c700 1 --2- 172.21.15.101:0/1990715678 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f24c4100480 0x7f24c41983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.663+0000 7f24c259c700 1 -- 172.21.15.101:0/1990715678 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b40095f0 con 0x7f24c40ffad0 2023-07-18T03:31:17.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.663+0000 7f24c259c700 1 --2- 172.21.15.101:0/1990715678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24c40ffad0 0x7f24c4197e60 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f24b4005910 tx=0x7f24b4005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:17.665 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.663+0000 7f24b37fe700 1 -- 172.21.15.101:0/1990715678 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b401d070 con 0x7f24c40ffad0 2023-07-18T03:31:17.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.663+0000 7f24c8fff700 1 -- 172.21.15.101:0/1990715678 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24c419d3f0 con 0x7f24c40ffad0 2023-07-18T03:31:17.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.664+0000 7f24b37fe700 1 -- 172.21.15.101:0/1990715678 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24b4022470 con 0x7f24c40ffad0 2023-07-18T03:31:17.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.664+0000 7f24b37fe700 1 -- 172.21.15.101:0/1990715678 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b4017650 con 0x7f24c40ffad0 2023-07-18T03:31:17.666 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.664+0000 7f24c8fff700 1 -- 172.21.15.101:0/1990715678 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24c41a4c90 con 0x7f24c40ffad0 2023-07-18T03:31:17.668 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.668+0000 7f24b37fe700 1 -- 172.21.15.101:0/1990715678 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f24b40225e0 con 0x7f24c40ffad0 2023-07-18T03:31:17.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.669+0000 7f24dc9eab80 1 Processor -- start 2023-07-18T03:31:17.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.669+0000 7f24dc9eab80 1 -- start start 2023-07-18T03:31:17.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.670+0000 7f24dc9eab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634411362d0 0x5634411366a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:17.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.670+0000 7f24dc9eab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563441136be0 0x563441141590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:17.669 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.670+0000 7f24dc9eab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563441141ad0 0x563441143eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:17.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.670+0000 7f24dc9eab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56344103ccb0 con 0x563441141ad0 2023-07-18T03:31:17.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.670+0000 7f24dc9eab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56344108fa40 con 0x5634411362d0 2023-07-18T03:31:17.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.670+0000 7f24dc9eab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563441088ed0 con 0x563441136be0 2023-07-18T03:31:17.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.670+0000 7f24b37fe700 1 --2- 172.21.15.101:0/1990715678 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f24ac0a12c0 0x7f24ac0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:17.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.670+0000 7f24c2d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634411362d0 0x5634411366a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:17.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.670+0000 7f24c259c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563441136be0 0x563441141590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:17.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.670+0000 7f24c2d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634411362d0 0x5634411366a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47360/0 (socket says 172.21.15.101:47360) 2023-07-18T03:31:17.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.670+0000 7f24c2d9d700 1 -- 172.21.15.101:0/1925163476 learned_addr learned my addr 172.21.15.101:0/1925163476 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:31:17.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.670+0000 7f24c1d9b700 1 --2- 172.21.15.101:0/1925163476 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563441141ad0 0x563441143eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:17.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.670+0000 7f24b37fe700 1 -- 172.21.15.101:0/1990715678 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f24b40ce420 con 0x7f24c40ffad0 2023-07-18T03:31:17.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.670+0000 7f24c2d9d700 1 --2- 172.21.15.101:0/1990715678 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f24ac0a12c0 0x7f24ac0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:17.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.670+0000 7f24c259c700 1 -- 172.21.15.101:0/1925163476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634411362d0 msgr2=0x5634411366a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:17.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.670+0000 7f24c259c700 1 --2- 172.21.15.101:0/1925163476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634411362d0 0x5634411366a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.670+0000 7f24c259c700 1 -- 172.21.15.101:0/1925163476 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563441141ad0 msgr2=0x563441143eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:17.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.670+0000 7f24c259c700 1 --2- 172.21.15.101:0/1925163476 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563441141ad0 0x563441143eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.670+0000 7f24c259c700 1 -- 172.21.15.101:0/1925163476 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563440eb6de0 con 0x563441136be0 2023-07-18T03:31:17.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.671+0000 7f24c259c700 1 --2- 172.21.15.101:0/1925163476 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563441136be0 0x563441141590 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f24b40cb770 tx=0x7f24b409b080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ca95ef2748be8a6 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:17.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.671+0000 7f24b0ff9700 1 -- 172.21.15.101:0/1925163476 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b409aae0 con 0x563441136be0 2023-07-18T03:31:17.673 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.671+0000 7f24b0ff9700 1 -- 172.21.15.101:0/1925163476 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24b409f1b0 con 0x563441136be0 2023-07-18T03:31:17.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.671+0000 7f24b0ff9700 1 -- 172.21.15.101:0/1925163476 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b409b3e0 con 0x563441136be0 2023-07-18T03:31:17.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.671+0000 7f24dc9eab80 1 -- 172.21.15.101:0/1925163476 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563441136be0 msgr2=0x563441141590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:17.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.671+0000 7f24c2d9d700 1 --2- 172.21.15.101:0/1990715678 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f24ac0a12c0 0x7f24ac0a3770 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f24bc00fb80 tx=0x7f24bc004b40 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:17.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.671+0000 7f24dc9eab80 1 --2- 172.21.15.101:0/1925163476 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563441136be0 0x563441141590 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f24b40cb770 tx=0x7f24b409b080 comp rx=0 tx=0).stop 2023-07-18T03:31:17.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.671+0000 7f24dc9eab80 1 -- 172.21.15.101:0/1925163476 shutdown_connections 2023-07-18T03:31:17.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.671+0000 7f24dc9eab80 1 --2- 172.21.15.101:0/1925163476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634411362d0 0x5634411366a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.671+0000 7f24dc9eab80 1 --2- 172.21.15.101:0/1925163476 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563441141ad0 0x563441143eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.671+0000 7f24dc9eab80 1 --2- 172.21.15.101:0/1925163476 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563441136be0 0x563441141590 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.671+0000 7f24dc9eab80 1 -- 172.21.15.101:0/1925163476 >> 172.21.15.101:0/1925163476 conn(0x56344112cda0 msgr2=0x5634411326a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:17.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.672+0000 7f24dc9eab80 1 -- 172.21.15.101:0/1925163476 shutdown_connections 2023-07-18T03:31:17.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.672+0000 7f24dc9eab80 1 -- 172.21.15.101:0/1925163476 wait complete. 2023-07-18T03:31:17.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.672+0000 7f24dc9eab80 1 -- start start 2023-07-18T03:31:17.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.672+0000 7f24dc9eab80 1 Processor -- start 2023-07-18T03:31:17.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.673+0000 7f24dc9eab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634411362d0 0x563441141ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:17.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.673+0000 7f24dc9eab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563441136be0 0x5634411423e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:17.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.673+0000 7f24dc9eab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563441141ad0 0x563441142920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:17.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.673+0000 7f24dc9eab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563441028ac0 con 0x563441136be0 2023-07-18T03:31:17.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.673+0000 7f24dc9eab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634410287c0 con 0x5634411362d0 2023-07-18T03:31:17.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.673+0000 7f24dc9eab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563440f0c920 con 0x563441141ad0 2023-07-18T03:31:17.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.673+0000 7f24dc9eab80 10 client.0 _ll_register_callbacks cb 0x563441026760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:31:17.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.673+0000 7f24c259c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634411362d0 0x563441141ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:17.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.673+0000 7f24c259c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634411362d0 0x563441141ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47376/0 (socket says 172.21.15.101:47376) 2023-07-18T03:31:17.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.673+0000 7f24c259c700 1 -- 172.21.15.101:0/2683133882 learned_addr learned my addr 172.21.15.101:0/2683133882 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:31:17.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.673+0000 7f24c2d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563441141ad0 0x563441142920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:17.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.673+0000 7f24c1d9b700 1 --2- 172.21.15.101:0/2683133882 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563441136be0 0x5634411423e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:17.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.673+0000 7f24c259c700 1 -- 172.21.15.101:0/2683133882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563441141ad0 msgr2=0x563441142920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:17.684 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.673+0000 7f24c259c700 1 --2- 172.21.15.101:0/2683133882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563441141ad0 0x563441142920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.673+0000 7f24c259c700 1 -- 172.21.15.101:0/2683133882 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563441136be0 msgr2=0x5634411423e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:17.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.673+0000 7f24c259c700 1 --2- 172.21.15.101:0/2683133882 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563441136be0 0x5634411423e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.673+0000 7f24c259c700 1 -- 172.21.15.101:0/2683133882 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24c41a4c90 con 0x5634411362d0 2023-07-18T03:31:17.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.673+0000 7f24c259c700 1 --2- 172.21.15.101:0/2683133882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634411362d0 0x563441141ea0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f24b409ae80 tx=0x7f24b4009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f325b18392bf660e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:17.686 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.674+0000 7f24a9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:31:17.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.674+0000 7f24a9ffb700 1 -- 172.21.15.101:0/2683133882 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b40a3540 con 0x5634411362d0 2023-07-18T03:31:17.687 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.674+0000 7f24a9ffb700 1 -- 172.21.15.101:0/2683133882 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f24b409b180 con 0x5634411362d0 2023-07-18T03:31:17.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.674+0000 7f24dc9eab80 1 -- 172.21.15.101:0/2683133882 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563440fe24f0 con 0x5634411362d0 2023-07-18T03:31:17.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.674+0000 7f24a9ffb700 1 -- 172.21.15.101:0/2683133882 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24b409aae0 con 0x5634411362d0 2023-07-18T03:31:17.688 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.674+0000 7f24dc9eab80 10 client.29156 fetch_fsmap learned FSMap version 49 2023-07-18T03:31:17.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.674+0000 7f24dc9eab80 1 -- 172.21.15.101:0/2683133882 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x563440e9fc60 con 0x5634411362d0 2023-07-18T03:31:17.689 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.674+0000 7f24a9ffb700 1 -- 172.21.15.101:0/2683133882 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563440fe24f0 con 0x5634411362d0 2023-07-18T03:31:17.690 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.674+0000 7f24a9ffb700 1 -- 172.21.15.101:0/2683133882 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 49) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f24b40a3140 con 0x5634411362d0 2023-07-18T03:31:17.690 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.674+0000 7f24dc9eab80 10 client.29156 fetch_fsmap finished waiting for FSMap version 49 2023-07-18T03:31:17.690 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.674+0000 7f24dc9eab80 10 client.29156 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:31:17.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.674+0000 7f24dc9eab80 20 client.29156 populate_metadata read hostname 'smithi101' 2023-07-18T03:31:17.691 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.674+0000 7f24dc9eab80 1 --2- 172.21.15.101:0/2683133882 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563441150b60 0x563441152f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:17.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.674+0000 7f24dc9eab80 4 client.29156 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:31:17.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.674+0000 7f24dc9eab80 1 -- 172.21.15.101:0/2683133882 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563441081eb0 con 0x563441150b60 2023-07-18T03:31:17.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.675+0000 7f24c1d9b700 1 --2- 172.21.15.101:0/2683133882 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563441150b60 0x563441152f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:17.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.675+0000 7f24c1d9b700 1 --2- 172.21.15.101:0/2683133882 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563441150b60 0x563441152f40 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d991136f20e51c64 server_cookie=1cfd42527a25c2bc in_seq=0 out_seq=0 2023-07-18T03:31:17.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.675+0000 7f24a9ffb700 10 client.29156 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:31:17.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.676+0000 7f24a9ffb700 1 -- 172.21.15.101:0/2683133882 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563441081eb0 con 0x563441150b60 2023-07-18T03:31:17.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.676+0000 7f24a9ffb700 10 client.29156 handle_command_reply: tid=0 2023-07-18T03:31:17.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.676+0000 7f24dc9eab80 1 client.29156 shutdown 2023-07-18T03:31:17.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.676+0000 7f24dc9eab80 1 -- 172.21.15.101:0/2683133882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634411362d0 msgr2=0x563441141ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:17.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.676+0000 7f24dc9eab80 1 --2- 172.21.15.101:0/2683133882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634411362d0 0x563441141ea0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f24b409ae80 tx=0x7f24b4009310 comp rx=0 tx=0).stop 2023-07-18T03:31:17.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.676+0000 7f24dc9eab80 1 -- 172.21.15.101:0/2683133882 shutdown_connections 2023-07-18T03:31:17.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.676+0000 7f24dc9eab80 1 --2- 172.21.15.101:0/2683133882 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563441150b60 0x563441152f40 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.676+0000 7f24dc9eab80 1 --2- 172.21.15.101:0/2683133882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634411362d0 0x563441141ea0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.676+0000 7f24dc9eab80 1 --2- 172.21.15.101:0/2683133882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563441141ad0 0x563441142920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.676+0000 7f24a9ffb700 0 client.29156 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:31:17.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.676+0000 7f24dc9eab80 1 --2- 172.21.15.101:0/2683133882 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563441136be0 0x5634411423e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.676+0000 7f24dc9eab80 1 -- 172.21.15.101:0/2683133882 >> 172.21.15.101:0/2683133882 conn(0x56344112cda0 msgr2=0x563441132290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:17.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.676+0000 7f24dc9eab80 1 -- 172.21.15.101:0/2683133882 shutdown_connections 2023-07-18T03:31:17.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.676+0000 7f24dc9eab80 1 -- 172.21.15.101:0/2683133882 wait complete. 2023-07-18T03:31:17.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.676+0000 7f24dc9eab80 20 client.29156 trim_cache size 0 max 16384 2023-07-18T03:31:17.709 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:31:17 smithi179 ceph-mon[129229]: pgmap v749: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:31:17.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.693+0000 7f24dc9eab80 1 Processor -- start 2023-07-18T03:31:17.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.693+0000 7f24dc9eab80 1 -- start start 2023-07-18T03:31:17.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.693+0000 7f24dc9eab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5634411362d0 0x56344113b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:17.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.693+0000 7f24dc9eab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563441136be0 0x56344114c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:17.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.693+0000 7f24dc9eab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563441141ad0 0x56344113dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:17.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.693+0000 7f24dc9eab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634411410f0 con 0x563441141ad0 2023-07-18T03:31:17.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.693+0000 7f24dc9eab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634411534d0 con 0x563441136be0 2023-07-18T03:31:17.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.693+0000 7f24dc9eab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56344113c590 con 0x5634411362d0 2023-07-18T03:31:17.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.694+0000 7f24c259c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5634411362d0 0x56344113b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:17.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.694+0000 7f24c1d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563441136be0 0x56344114c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:17.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.694+0000 7f24c259c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5634411362d0 0x56344113b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:60824/0 (socket says 172.21.15.101:60824) 2023-07-18T03:31:17.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.694+0000 7f24c1d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563441136be0 0x56344114c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47392/0 (socket says 172.21.15.101:47392) 2023-07-18T03:31:17.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.694+0000 7f24c259c700 1 -- 172.21.15.101:0/1814552916 learned_addr learned my addr 172.21.15.101:0/1814552916 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:31:17.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.694+0000 7f24c2d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563441141ad0 0x56344113dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:17.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.694+0000 7f24c259c700 1 -- 172.21.15.101:0/1814552916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563441136be0 msgr2=0x56344114c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:17.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.694+0000 7f24c259c700 1 --2- 172.21.15.101:0/1814552916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563441136be0 0x56344114c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.694+0000 7f24c259c700 1 -- 172.21.15.101:0/1814552916 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563441141ad0 msgr2=0x56344113dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:17.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.694+0000 7f24c259c700 1 --2- 172.21.15.101:0/1814552916 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563441141ad0 0x56344113dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.694+0000 7f24c259c700 1 -- 172.21.15.101:0/1814552916 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563440edd250 con 0x5634411362d0 2023-07-18T03:31:17.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.694+0000 7f24c1d9b700 1 --2- 172.21.15.101:0/1814552916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563441136be0 0x56344114c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:31:17.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.694+0000 7f24c259c700 1 --2- 172.21.15.101:0/1814552916 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5634411362d0 0x56344113b520 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f24b4097680 tx=0x7f24b4022f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2914a1396e77b48f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:17.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.694+0000 7f24b17fa700 1 -- 172.21.15.101:0/1814552916 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b40a3540 con 0x5634411362d0 2023-07-18T03:31:17.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.694+0000 7f24b17fa700 1 -- 172.21.15.101:0/1814552916 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24b409aae0 con 0x5634411362d0 2023-07-18T03:31:17.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.694+0000 7f24b17fa700 1 -- 172.21.15.101:0/1814552916 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b4097110 con 0x5634411362d0 2023-07-18T03:31:17.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.695+0000 7f24dc9eab80 1 -- 172.21.15.101:0/1814552916 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5634411362d0 msgr2=0x56344113b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:17.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.695+0000 7f24dc9eab80 1 --2- 172.21.15.101:0/1814552916 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5634411362d0 0x56344113b520 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f24b4097680 tx=0x7f24b4022f90 comp rx=0 tx=0).stop 2023-07-18T03:31:17.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.695+0000 7f24dc9eab80 1 -- 172.21.15.101:0/1814552916 shutdown_connections 2023-07-18T03:31:17.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.695+0000 7f24dc9eab80 1 --2- 172.21.15.101:0/1814552916 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563441141ad0 0x56344113dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.695+0000 7f24dc9eab80 1 --2- 172.21.15.101:0/1814552916 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5634411362d0 0x56344113b520 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.695+0000 7f24dc9eab80 1 --2- 172.21.15.101:0/1814552916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563441136be0 0x56344114c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.695+0000 7f24dc9eab80 1 -- 172.21.15.101:0/1814552916 >> 172.21.15.101:0/1814552916 conn(0x56344112cda0 msgr2=0x56344113be80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:17.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.695+0000 7f24dc9eab80 1 -- 172.21.15.101:0/1814552916 shutdown_connections 2023-07-18T03:31:17.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.695+0000 7f24dc9eab80 1 -- 172.21.15.101:0/1814552916 wait complete. 2023-07-18T03:31:17.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.695+0000 7f24dc9eab80 1 -- start start 2023-07-18T03:31:17.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.696+0000 7f24dc9eab80 1 Processor -- start 2023-07-18T03:31:17.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.696+0000 7f24dc9eab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5634411362d0 0x563441142da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:17.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.696+0000 7f24dc9eab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563441136be0 0x5634411432e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:17.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.696+0000 7f24dc9eab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563441141ad0 0x56344119d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:17.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.696+0000 7f24dc9eab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56344119de30 con 0x563441141ad0 2023-07-18T03:31:17.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.696+0000 7f24dc9eab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56344119dfa0 con 0x563441136be0 2023-07-18T03:31:17.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.696+0000 7f24dc9eab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56344119e160 con 0x5634411362d0 2023-07-18T03:31:17.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.696+0000 7f24dc9eab80 10 client.0 _ll_register_callbacks cb 0x7f24bc00f1b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:31:17.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.696+0000 7f24c1d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563441136be0 0x5634411432e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:17.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.696+0000 7f24c259c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5634411362d0 0x563441142da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:17.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.696+0000 7f24c259c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5634411362d0 0x563441142da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:60838/0 (socket says 172.21.15.101:60838) 2023-07-18T03:31:17.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.696+0000 7f24c1d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563441136be0 0x5634411432e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47400/0 (socket says 172.21.15.101:47400) 2023-07-18T03:31:17.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.696+0000 7f24c259c700 1 -- 172.21.15.101:0/907222776 learned_addr learned my addr 172.21.15.101:0/907222776 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:31:17.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.696+0000 7f24c2d9d700 1 --2- 172.21.15.101:0/907222776 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563441141ad0 0x56344119d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:17.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.697+0000 7f24c1d9b700 1 -- 172.21.15.101:0/907222776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5634411362d0 msgr2=0x563441142da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:17.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.697+0000 7f24c1d9b700 1 --2- 172.21.15.101:0/907222776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5634411362d0 0x563441142da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.697+0000 7f24c1d9b700 1 -- 172.21.15.101:0/907222776 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563441141ad0 msgr2=0x56344119d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:17.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.697+0000 7f24c1d9b700 1 --2- 172.21.15.101:0/907222776 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563441141ad0 0x56344119d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.697+0000 7f24c1d9b700 1 -- 172.21.15.101:0/907222776 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24b8009ab0 con 0x563441136be0 2023-07-18T03:31:17.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.697+0000 7f24c259c700 1 --2- 172.21.15.101:0/907222776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5634411362d0 0x563441142da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:31:17.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.697+0000 7f24c2d9d700 1 --2- 172.21.15.101:0/907222776 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563441141ad0 0x56344119d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:31:17.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.697+0000 7f24c1d9b700 1 --2- 172.21.15.101:0/907222776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563441136be0 0x5634411432e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x563441153730 tx=0x7f24b8000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41eacdceb5317367 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:17.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.697+0000 7f24a9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:31:17.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.697+0000 7f24a9ffb700 1 -- 172.21.15.101:0/907222776 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b8017050 con 0x563441136be0 2023-07-18T03:31:17.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.697+0000 7f24dc9eab80 1 -- 172.21.15.101:0/907222776 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f24b409f1b0 con 0x563441136be0 2023-07-18T03:31:17.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.697+0000 7f24a9ffb700 1 -- 172.21.15.101:0/907222776 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f24b8004230 con 0x563441136be0 2023-07-18T03:31:17.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.697+0000 7f24a9ffb700 1 -- 172.21.15.101:0/907222776 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24b80043f0 con 0x563441136be0 2023-07-18T03:31:17.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.697+0000 7f24dc9eab80 10 client.29162 fetch_fsmap learned FSMap version 49 2023-07-18T03:31:17.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.697+0000 7f24dc9eab80 1 -- 172.21.15.101:0/907222776 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x563440e9fc60 con 0x563441136be0 2023-07-18T03:31:17.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.698+0000 7f24a9ffb700 1 -- 172.21.15.101:0/907222776 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24b409f1b0 con 0x563441136be0 2023-07-18T03:31:17.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.698+0000 7f24a9ffb700 1 -- 172.21.15.101:0/907222776 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 49) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f24b8027740 con 0x563441136be0 2023-07-18T03:31:17.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.698+0000 7f24dc9eab80 10 client.29162 fetch_fsmap finished waiting for FSMap version 49 2023-07-18T03:31:17.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.698+0000 7f24dc9eab80 10 client.29162 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:31:17.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.698+0000 7f24dc9eab80 20 client.29162 populate_metadata read hostname 'smithi101' 2023-07-18T03:31:17.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.698+0000 7f24dc9eab80 1 --2- 172.21.15.101:0/907222776 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5634411a5980 0x5634411a7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:17.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.698+0000 7f24dc9eab80 4 client.29162 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:31:17.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.698+0000 7f24dc9eab80 1 -- 172.21.15.101:0/907222776 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563441044050 con 0x5634411a5980 2023-07-18T03:31:17.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.698+0000 7f24c259c700 1 --2- 172.21.15.101:0/907222776 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5634411a5980 0x5634411a7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:17.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.698+0000 7f24a9ffb700 10 client.29162 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:31:17.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.698+0000 7f24c259c700 1 --2- 172.21.15.101:0/907222776 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5634411a5980 0x5634411a7d90 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fc77d24c4b5848a server_cookie=5a798750d760f3d1 in_seq=0 out_seq=0 2023-07-18T03:31:17.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.699+0000 7f24a9ffb700 1 -- 172.21.15.101:0/907222776 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x563441044050 con 0x5634411a5980 2023-07-18T03:31:17.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.699+0000 7f24a9ffb700 10 client.29162 handle_command_reply: tid=0 2023-07-18T03:31:17.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.699+0000 7f24dc9eab80 1 client.29162 shutdown 2023-07-18T03:31:17.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.699+0000 7f24dc9eab80 1 -- 172.21.15.101:0/907222776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563441136be0 msgr2=0x5634411432e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:17.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.699+0000 7f24dc9eab80 1 --2- 172.21.15.101:0/907222776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563441136be0 0x5634411432e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x563441153730 tx=0x7f24b8000bc0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.699+0000 7f24dc9eab80 1 -- 172.21.15.101:0/907222776 shutdown_connections 2023-07-18T03:31:17.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.699+0000 7f24dc9eab80 1 --2- 172.21.15.101:0/907222776 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5634411a5980 0x5634411a7d90 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.699+0000 7f24dc9eab80 1 --2- 172.21.15.101:0/907222776 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563441141ad0 0x56344119d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.699+0000 7f24dc9eab80 1 --2- 172.21.15.101:0/907222776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5634411362d0 0x563441142da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.699+0000 7f24dc9eab80 1 --2- 172.21.15.101:0/907222776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563441136be0 0x5634411432e0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.699+0000 7f24a9ffb700 0 client.29162 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:31:17.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.699+0000 7f24dc9eab80 1 -- 172.21.15.101:0/907222776 >> 172.21.15.101:0/907222776 conn(0x56344112cda0 msgr2=0x56344113f290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:17.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.699+0000 7f24dc9eab80 1 -- 172.21.15.101:0/907222776 shutdown_connections 2023-07-18T03:31:17.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.699+0000 7f24dc9eab80 1 -- 172.21.15.101:0/907222776 wait complete. 2023-07-18T03:31:17.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.699+0000 7f24dc9eab80 20 client.29162 trim_cache size 0 max 16384 2023-07-18T03:31:17.730 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:31:17.730 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:31:17.730 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:31:17.730 INFO:teuthology.orchestra.run.smithi101.stdout: "0f2f19e0-0429-465c-935e-223eb09490c3": { 2023-07-18T03:31:17.731 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:31:17.731 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "0f2f19e0-0429-465c-935e-223eb09490c3", 2023-07-18T03:31:17.731 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:31:17.731 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:31:17.731 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:31:17.732 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:31:17.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.700+0000 7f24c8fff700 1 -- 172.21.15.101:0/1990715678 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f24ac0a12c0 msgr2=0x7f24ac0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:17.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.700+0000 7f24c8fff700 1 --2- 172.21.15.101:0/1990715678 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f24ac0a12c0 0x7f24ac0a3770 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f24bc00fb80 tx=0x7f24bc004b40 comp rx=0 tx=0).stop 2023-07-18T03:31:17.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.700+0000 7f24c8fff700 1 -- 172.21.15.101:0/1990715678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24c40ffad0 msgr2=0x7f24c4197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:17.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.700+0000 7f24c8fff700 1 --2- 172.21.15.101:0/1990715678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24c40ffad0 0x7f24c4197e60 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f24b4005910 tx=0x7f24b4005720 comp rx=0 tx=0).stop 2023-07-18T03:31:17.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.700+0000 7f24c8fff700 1 -- 172.21.15.101:0/1990715678 shutdown_connections 2023-07-18T03:31:17.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.700+0000 7f24c8fff700 1 --2- 172.21.15.101:0/1990715678 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f24ac0a12c0 0x7f24ac0a3770 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.700+0000 7f24c8fff700 1 --2- 172.21.15.101:0/1990715678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24c40ffad0 0x7f24c4197e60 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.700+0000 7f24c8fff700 1 --2- 172.21.15.101:0/1990715678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24c4105ae0 0x7f24c419c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.700+0000 7f24c8fff700 1 --2- 172.21.15.101:0/1990715678 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f24c4100480 0x7f24c41983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.700+0000 7f24c8fff700 1 -- 172.21.15.101:0/1990715678 >> 172.21.15.101:0/1990715678 conn(0x7f24c40fb5f0 msgr2=0x7f24c410ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:17.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.700+0000 7f24c8fff700 1 -- 172.21.15.101:0/1990715678 shutdown_connections 2023-07-18T03:31:17.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:17.700+0000 7f24c8fff700 1 -- 172.21.15.101:0/1990715678 wait complete. 2023-07-18T03:31:17.735 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:31:17 smithi101 ceph-mon[126379]: pgmap v749: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:31:17.736 INFO:tasks.cephfs.filesystem:scrub status for tag:0f2f19e0-0429-465c-935e-223eb09490c3 - {'path': '/', 'tag': '0f2f19e0-0429-465c-935e-223eb09490c3', 'options': 'recursive,force'} 2023-07-18T03:31:17.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:31:17 smithi155 ceph-mon[128885]: pgmap v749: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:31:19.705 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:31:19 smithi179 ceph-mon[129229]: pgmap v750: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:31:19.705 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:31:19 smithi179 ceph-mon[129229]: osdmap e128: 12 total, 12 up, 12 in 2023-07-18T03:31:19.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:31:19 smithi101 ceph-mon[126379]: pgmap v750: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:31:19.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:31:19 smithi101 ceph-mon[126379]: osdmap e128: 12 total, 12 up, 12 in 2023-07-18T03:31:19.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:31:19 smithi155 ceph-mon[128885]: pgmap v750: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:31:19.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:31:19 smithi155 ceph-mon[128885]: osdmap e128: 12 total, 12 up, 12 in 2023-07-18T03:31:21.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:31:21 smithi179 ceph-mon[129229]: pgmap v752: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 317 B/s wr, 6 op/s 2023-07-18T03:31:21.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:31:21 smithi101 ceph-mon[126379]: pgmap v752: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 317 B/s wr, 6 op/s 2023-07-18T03:31:21.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:31:21 smithi155 ceph-mon[128885]: pgmap v752: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 317 B/s wr, 6 op/s 2023-07-18T03:31:23.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:31:23 smithi179 ceph-mon[129229]: pgmap v753: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2023-07-18T03:31:23.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:31:23 smithi101 ceph-mon[126379]: pgmap v753: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2023-07-18T03:31:23.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:31:23 smithi155 ceph-mon[128885]: pgmap v753: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2023-07-18T03:31:25.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:31:25 smithi179 ceph-mon[129229]: pgmap v754: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2023-07-18T03:31:25.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:31:25 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:31:25.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:31:25 smithi101 ceph-mon[126379]: pgmap v754: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2023-07-18T03:31:25.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:31:25 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:31:25.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:31:25 smithi155 ceph-mon[128885]: pgmap v754: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2023-07-18T03:31:25.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:31:25 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:31:26.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:31:26 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:31:26.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:31:26 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:31:26.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:31:26 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:31:26.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:31:26 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:31:26.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:31:26 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:31:26.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:31:26 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:31:26.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:31:26 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:31:26.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:31:26 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:31:26.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:31:26 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:31:27.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:31:27 smithi179 ceph-mon[129229]: pgmap v755: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:31:27.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:31:27 smithi101 ceph-mon[126379]: pgmap v755: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:31:27.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:31:27 smithi155 ceph-mon[128885]: pgmap v755: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:31:29.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:31:29 smithi179 ceph-mon[129229]: pgmap v756: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:31:29.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:31:29 smithi101 ceph-mon[126379]: pgmap v756: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:31:29.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:31:29 smithi155 ceph-mon[128885]: pgmap v756: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:31:31.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:31:31 smithi179 ceph-mon[129229]: pgmap v757: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 190 B/s wr, 7 op/s 2023-07-18T03:31:31.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:31:31 smithi101 ceph-mon[126379]: pgmap v757: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 190 B/s wr, 7 op/s 2023-07-18T03:31:31.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:31:31 smithi155 ceph-mon[128885]: pgmap v757: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 190 B/s wr, 7 op/s 2023-07-18T03:31:33.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:31:33 smithi179 ceph-mon[129229]: pgmap v758: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:31:33.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:31:33 smithi101 ceph-mon[126379]: pgmap v758: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:31:33.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:31:33 smithi155 ceph-mon[128885]: pgmap v758: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:31:35.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:31:35 smithi179 ceph-mon[129229]: pgmap v759: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:31:35.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:31:35 smithi101 ceph-mon[126379]: pgmap v759: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:31:35.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:31:35 smithi155 ceph-mon[128885]: pgmap v759: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:31:37.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:31:37 smithi179 ceph-mon[129229]: pgmap v760: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:31:37.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:31:37 smithi101 ceph-mon[126379]: pgmap v760: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:31:37.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:31:37 smithi155 ceph-mon[128885]: pgmap v760: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:31:38.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:31:38 smithi101 ceph-mon[126379]: pgmap v761: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:31:38.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:31:38 smithi155 ceph-mon[128885]: pgmap v761: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:31:38.953 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:31:38 smithi179 ceph-mon[129229]: pgmap v761: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:31:41.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:31:41 smithi179 ceph-mon[129229]: pgmap v762: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-18T03:31:41.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:31:41 smithi101 ceph-mon[126379]: pgmap v762: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-18T03:31:41.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:31:41 smithi155 ceph-mon[128885]: pgmap v762: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-18T03:31:43.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:31:43 smithi179 ceph-mon[129229]: pgmap v763: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:31:43.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:31:43 smithi101 ceph-mon[126379]: pgmap v763: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:31:43.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:31:43 smithi155 ceph-mon[128885]: pgmap v763: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:31:45.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:31:45 smithi179 ceph-mon[129229]: pgmap v764: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:31:45.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:31:45 smithi101 ceph-mon[126379]: pgmap v764: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:31:45.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:31:45 smithi155 ceph-mon[128885]: pgmap v764: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:31:47.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:31:47 smithi179 ceph-mon[129229]: pgmap v765: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:31:47.737 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:31:47.743 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:31:47 smithi101 ceph-mon[126379]: pgmap v765: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:31:47.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:31:47 smithi155 ceph-mon[128885]: pgmap v765: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:31:47.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.955+0000 7f11893ff700 1 -- 172.21.15.101:0/2150141578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1184100490 msgr2=0x7f118410c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:47.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.955+0000 7f11893ff700 1 --2- 172.21.15.101:0/2150141578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1184100490 0x7f118410c920 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f117c00b0a0 tx=0x7f117c00b3b0 comp rx=0 tx=0).stop 2023-07-18T03:31:47.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.955+0000 7f11893ff700 1 -- 172.21.15.101:0/2150141578 shutdown_connections 2023-07-18T03:31:47.953 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.955+0000 7f11893ff700 1 --2- 172.21.15.101:0/2150141578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1184100490 0x7f118410c920 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:47.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.955+0000 7f11893ff700 1 --2- 172.21.15.101:0/2150141578 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1184105af0 0x7f1184105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:47.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.955+0000 7f11893ff700 1 --2- 172.21.15.101:0/2150141578 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f11840ffae0 0x7f11840fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:47.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.955+0000 7f11893ff700 1 -- 172.21.15.101:0/2150141578 >> 172.21.15.101:0/2150141578 conn(0x7f11840fb5e0 msgr2=0x7f11840fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:47.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.955+0000 7f11893ff700 1 -- 172.21.15.101:0/2150141578 shutdown_connections 2023-07-18T03:31:47.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.955+0000 7f11893ff700 1 -- 172.21.15.101:0/2150141578 wait complete. 2023-07-18T03:31:47.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.956+0000 7f11893ff700 1 Processor -- start 2023-07-18T03:31:47.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.957+0000 7f11893ff700 1 -- start start 2023-07-18T03:31:47.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.957+0000 7f11893ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11840ffae0 0x7f1184072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:47.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.957+0000 7f11893ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1184100490 0x7f1184072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:47.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.957+0000 7f11893ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1184105af0 0x7f1184073290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:47.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.957+0000 7f11893ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11841a4140 con 0x7f1184105af0 2023-07-18T03:31:47.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.957+0000 7f11893ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11841a42b0 con 0x7f11840ffae0 2023-07-18T03:31:47.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.957+0000 7f11893ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11841a4470 con 0x7f1184100490 2023-07-18T03:31:47.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.957+0000 7f1182ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11840ffae0 0x7f1184072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:47.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.957+0000 7f1182ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11840ffae0 0x7f1184072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:36418/0 (socket says 172.21.15.101:36418) 2023-07-18T03:31:47.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.957+0000 7f1182ffd700 1 -- 172.21.15.101:0/3313161835 learned_addr learned my addr 172.21.15.101:0/3313161835 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:31:47.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.957+0000 7f11827fc700 1 --2- 172.21.15.101:0/3313161835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1184100490 0x7f1184072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:47.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.957+0000 7f11837fe700 1 --2- 172.21.15.101:0/3313161835 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1184105af0 0x7f1184073290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:47.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.958+0000 7f1182ffd700 1 -- 172.21.15.101:0/3313161835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1184100490 msgr2=0x7f1184072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:47.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.958+0000 7f1182ffd700 1 --2- 172.21.15.101:0/3313161835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1184100490 0x7f1184072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:47.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.958+0000 7f1182ffd700 1 -- 172.21.15.101:0/3313161835 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1184105af0 msgr2=0x7f1184073290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:47.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.958+0000 7f1182ffd700 1 --2- 172.21.15.101:0/3313161835 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1184105af0 0x7f1184073290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:47.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.958+0000 7f1182ffd700 1 -- 172.21.15.101:0/3313161835 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117c009d00 con 0x7f11840ffae0 2023-07-18T03:31:47.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.958+0000 7f1182ffd700 1 --2- 172.21.15.101:0/3313161835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11840ffae0 0x7f1184072810 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f117400e9a0 tx=0x7f117400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:47.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.958+0000 7f116bfff700 1 -- 172.21.15.101:0/3313161835 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1174009790 con 0x7f11840ffae0 2023-07-18T03:31:47.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.958+0000 7f116bfff700 1 -- 172.21.15.101:0/3313161835 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1174010430 con 0x7f11840ffae0 2023-07-18T03:31:47.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.958+0000 7f11893ff700 1 -- 172.21.15.101:0/3313161835 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11841a4750 con 0x7f11840ffae0 2023-07-18T03:31:47.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.958+0000 7f116bfff700 1 -- 172.21.15.101:0/3313161835 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1174018680 con 0x7f11840ffae0 2023-07-18T03:31:47.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.958+0000 7f11893ff700 1 -- 172.21.15.101:0/3313161835 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11841a4c60 con 0x7f11840ffae0 2023-07-18T03:31:47.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.961+0000 7f116bfff700 1 -- 172.21.15.101:0/3313161835 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f11740105a0 con 0x7f11840ffae0 2023-07-18T03:31:47.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.962+0000 7f119cd55b80 1 Processor -- start 2023-07-18T03:31:47.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.963+0000 7f119cd55b80 1 -- start start 2023-07-18T03:31:47.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.963+0000 7f119cd55b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560cc6fff2d0 0x560cc6fff6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:47.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.963+0000 7f119cd55b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560cc6fffbe0 0x560cc700a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:47.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.963+0000 7f119cd55b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560cc700aad0 0x560cc700ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:47.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.963+0000 7f119cd55b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc6f05cb0 con 0x560cc6fffbe0 2023-07-18T03:31:47.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.963+0000 7f119cd55b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc6f58a40 con 0x560cc700aad0 2023-07-18T03:31:47.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.963+0000 7f119cd55b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc6f51ed0 con 0x560cc6fff2d0 2023-07-18T03:31:47.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.963+0000 7f116bfff700 1 --2- 172.21.15.101:0/3313161835 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f116c0a12c0 0x7f116c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:47.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.963+0000 7f1182ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560cc6fffbe0 0x560cc700a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:47.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.963+0000 7f11837fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560cc6fff2d0 0x560cc6fff6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:47.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.963+0000 7f11827fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560cc700aad0 0x560cc700ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:47.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.963+0000 7f116bfff700 1 -- 172.21.15.101:0/3313161835 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f1174014070 con 0x7f11840ffae0 2023-07-18T03:31:47.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.963+0000 7f11827fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560cc700aad0 0x560cc700ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:36420/0 (socket says 172.21.15.101:36420) 2023-07-18T03:31:47.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.963+0000 7f11827fc700 1 -- 172.21.15.101:0/2216120961 learned_addr learned my addr 172.21.15.101:0/2216120961 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:31:47.970 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.963+0000 7f11837fe700 1 --2- 172.21.15.101:0/3313161835 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f116c0a12c0 0x7f116c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:47.970 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.964+0000 7f11827fc700 1 -- 172.21.15.101:0/2216120961 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560cc6fff2d0 msgr2=0x560cc6fff6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:47.970 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.964+0000 7f11827fc700 1 --2- 172.21.15.101:0/2216120961 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560cc6fff2d0 0x560cc6fff6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:47.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.964+0000 7f11827fc700 1 -- 172.21.15.101:0/2216120961 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560cc6fffbe0 msgr2=0x560cc700a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:47.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.964+0000 7f11827fc700 1 --2- 172.21.15.101:0/2216120961 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560cc6fffbe0 0x560cc700a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:47.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.964+0000 7f11827fc700 1 -- 172.21.15.101:0/2216120961 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560cc6d7fde0 con 0x560cc700aad0 2023-07-18T03:31:47.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.964+0000 7f1182ffd700 1 --2- 172.21.15.101:0/2216120961 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560cc6fffbe0 0x560cc700a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:31:47.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.964+0000 7f11827fc700 1 --2- 172.21.15.101:0/2216120961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560cc700aad0 0x560cc700ceb0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f1178009d20 tx=0x7f1178011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a10789cc67b3eb55 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:47.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.964+0000 7f11837fe700 1 --2- 172.21.15.101:0/3313161835 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f116c0a12c0 0x7f116c0a3770 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f117c00b780 tx=0x7f117c006230 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:47.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.964+0000 7f11697fa700 1 -- 172.21.15.101:0/2216120961 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1178019630 con 0x560cc700aad0 2023-07-18T03:31:47.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.964+0000 7f11697fa700 1 -- 172.21.15.101:0/2216120961 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1178019c50 con 0x560cc700aad0 2023-07-18T03:31:47.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.964+0000 7f11697fa700 1 -- 172.21.15.101:0/2216120961 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117800ec40 con 0x560cc700aad0 2023-07-18T03:31:47.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.965+0000 7f119cd55b80 1 -- 172.21.15.101:0/2216120961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560cc700aad0 msgr2=0x560cc700ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:47.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.965+0000 7f119cd55b80 1 --2- 172.21.15.101:0/2216120961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560cc700aad0 0x560cc700ceb0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f1178009d20 tx=0x7f1178011720 comp rx=0 tx=0).stop 2023-07-18T03:31:47.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.965+0000 7f119cd55b80 1 -- 172.21.15.101:0/2216120961 shutdown_connections 2023-07-18T03:31:47.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.965+0000 7f119cd55b80 1 --2- 172.21.15.101:0/2216120961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560cc700aad0 0x560cc700ceb0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:47.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.965+0000 7f119cd55b80 1 --2- 172.21.15.101:0/2216120961 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560cc6fff2d0 0x560cc6fff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:47.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.965+0000 7f119cd55b80 1 --2- 172.21.15.101:0/2216120961 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560cc6fffbe0 0x560cc700a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:47.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.965+0000 7f119cd55b80 1 -- 172.21.15.101:0/2216120961 >> 172.21.15.101:0/2216120961 conn(0x560cc6ff5da0 msgr2=0x560cc6ffb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:47.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.965+0000 7f119cd55b80 1 -- 172.21.15.101:0/2216120961 shutdown_connections 2023-07-18T03:31:47.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.965+0000 7f119cd55b80 1 -- 172.21.15.101:0/2216120961 wait complete. 2023-07-18T03:31:47.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.965+0000 7f119cd55b80 1 -- start start 2023-07-18T03:31:47.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.966+0000 7f119cd55b80 1 Processor -- start 2023-07-18T03:31:47.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.966+0000 7f119cd55b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560cc6fff2d0 0x560cc700a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:47.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.966+0000 7f119cd55b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560cc6fffbe0 0x560cc7007810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:47.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.966+0000 7f119cd55b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560cc700aad0 0x560cc7007f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:47.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.966+0000 7f119cd55b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc6ef1ac0 con 0x560cc6fff2d0 2023-07-18T03:31:47.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.966+0000 7f119cd55b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc6ef17c0 con 0x560cc700aad0 2023-07-18T03:31:47.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.966+0000 7f119cd55b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc6dd5920 con 0x560cc6fffbe0 2023-07-18T03:31:47.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.966+0000 7f119cd55b80 10 client.0 _ll_register_callbacks cb 0x560cc6eef760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:31:47.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.966+0000 7f1182ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560cc6fff2d0 0x560cc700a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:47.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.966+0000 7f11827fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560cc6fffbe0 0x560cc7007810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:47.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.966+0000 7f1182ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560cc6fff2d0 0x560cc700a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:57340/0 (socket says 172.21.15.101:57340) 2023-07-18T03:31:47.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.966+0000 7f11827fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560cc6fffbe0 0x560cc7007810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:55500/0 (socket says 172.21.15.101:55500) 2023-07-18T03:31:47.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.966+0000 7f1182ffd700 1 -- 172.21.15.101:0/3109369751 learned_addr learned my addr 172.21.15.101:0/3109369751 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:31:47.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.966+0000 7f11837fe700 1 --2- 172.21.15.101:0/3109369751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560cc700aad0 0x560cc7007f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:47.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.967+0000 7f11827fc700 1 -- 172.21.15.101:0/3109369751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560cc700aad0 msgr2=0x560cc7007f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:47.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.967+0000 7f11827fc700 1 --2- 172.21.15.101:0/3109369751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560cc700aad0 0x560cc7007f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:47.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.967+0000 7f11827fc700 1 -- 172.21.15.101:0/3109369751 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560cc6fff2d0 msgr2=0x560cc700a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:47.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.967+0000 7f11827fc700 1 --2- 172.21.15.101:0/3109369751 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560cc6fff2d0 0x560cc700a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:47.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.967+0000 7f11827fc700 1 -- 172.21.15.101:0/3109369751 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11841a4c60 con 0x560cc6fffbe0 2023-07-18T03:31:47.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.967+0000 7f1182ffd700 1 --2- 172.21.15.101:0/3109369751 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560cc6fff2d0 0x560cc700a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:31:47.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.967+0000 7f11827fc700 1 --2- 172.21.15.101:0/3109369751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560cc6fffbe0 0x560cc7007810 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f1178009d20 tx=0x7f117800ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97611037ed5dbf02 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:47.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.967+0000 7f1166ff5700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:31:47.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.967+0000 7f1166ff5700 1 -- 172.21.15.101:0/3109369751 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1178019840 con 0x560cc6fffbe0 2023-07-18T03:31:47.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.967+0000 7f119cd55b80 1 -- 172.21.15.101:0/3109369751 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560cc6eab4f0 con 0x560cc6fffbe0 2023-07-18T03:31:47.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.967+0000 7f1166ff5700 1 -- 172.21.15.101:0/3109369751 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(128..128 src has 1..128) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f117800e3b0 con 0x560cc6fffbe0 2023-07-18T03:31:47.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.968+0000 7f1166ff5700 1 -- 172.21.15.101:0/3109369751 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f117800e5a0 con 0x560cc6fffbe0 2023-07-18T03:31:47.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.968+0000 7f119cd55b80 10 client.29164 fetch_fsmap learned FSMap version 49 2023-07-18T03:31:47.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.968+0000 7f119cd55b80 1 -- 172.21.15.101:0/3109369751 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f1178009980 con 0x560cc6fffbe0 2023-07-18T03:31:47.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.968+0000 7f1166ff5700 1 -- 172.21.15.101:0/3109369751 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560cc6eab4f0 con 0x560cc6fffbe0 2023-07-18T03:31:47.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.968+0000 7f1166ff5700 1 -- 172.21.15.101:0/3109369751 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 49) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1178005170 con 0x560cc6fffbe0 2023-07-18T03:31:47.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.968+0000 7f119cd55b80 10 client.29164 fetch_fsmap finished waiting for FSMap version 49 2023-07-18T03:31:47.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.968+0000 7f119cd55b80 10 client.29164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:31:47.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.968+0000 7f119cd55b80 20 client.29164 populate_metadata read hostname 'smithi101' 2023-07-18T03:31:47.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.968+0000 7f119cd55b80 1 --2- 172.21.15.101:0/3109369751 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x560cc701a420 0x560cc701c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:47.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.968+0000 7f119cd55b80 4 client.29164 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:31:47.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.968+0000 7f119cd55b80 1 -- 172.21.15.101:0/3109369751 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560cc6f4aeb0 con 0x560cc701a420 2023-07-18T03:31:47.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.968+0000 7f1182ffd700 1 --2- 172.21.15.101:0/3109369751 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x560cc701a420 0x560cc701c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:47.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.969+0000 7f1182ffd700 1 --2- 172.21.15.101:0/3109369751 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x560cc701a420 0x560cc701c830 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=953facf1b9ea3897 server_cookie=78a172c27c6677b4 in_seq=0 out_seq=0 2023-07-18T03:31:47.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.969+0000 7f1166ff5700 10 client.29164 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:31:47.990 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.969+0000 7f1166ff5700 1 -- 172.21.15.101:0/3109369751 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560cc6f4aeb0 con 0x560cc701a420 2023-07-18T03:31:47.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.969+0000 7f1166ff5700 10 client.29164 handle_command_reply: tid=0 2023-07-18T03:31:47.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.969+0000 7f119cd55b80 1 client.29164 shutdown 2023-07-18T03:31:47.991 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.970+0000 7f119cd55b80 1 -- 172.21.15.101:0/3109369751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560cc6fffbe0 msgr2=0x560cc7007810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:47.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.970+0000 7f119cd55b80 1 --2- 172.21.15.101:0/3109369751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560cc6fffbe0 0x560cc7007810 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f1178009d20 tx=0x7f117800ee00 comp rx=0 tx=0).stop 2023-07-18T03:31:47.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.970+0000 7f119cd55b80 1 -- 172.21.15.101:0/3109369751 shutdown_connections 2023-07-18T03:31:47.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.970+0000 7f119cd55b80 1 --2- 172.21.15.101:0/3109369751 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x560cc701a420 0x560cc701c830 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:47.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.970+0000 7f119cd55b80 1 --2- 172.21.15.101:0/3109369751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560cc700aad0 0x560cc7007f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:47.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.970+0000 7f119cd55b80 1 --2- 172.21.15.101:0/3109369751 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560cc6fff2d0 0x560cc700a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:47.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.970+0000 7f119cd55b80 1 --2- 172.21.15.101:0/3109369751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560cc6fffbe0 0x560cc7007810 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:47.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.970+0000 7f1166ff5700 0 client.29164 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:31:47.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.970+0000 7f119cd55b80 1 -- 172.21.15.101:0/3109369751 >> 172.21.15.101:0/3109369751 conn(0x560cc6ff5da0 msgr2=0x560cc6ffb230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:47.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.970+0000 7f119cd55b80 1 -- 172.21.15.101:0/3109369751 shutdown_connections 2023-07-18T03:31:47.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.970+0000 7f119cd55b80 1 -- 172.21.15.101:0/3109369751 wait complete. 2023-07-18T03:31:47.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.970+0000 7f119cd55b80 20 client.29164 trim_cache size 0 max 16384 2023-07-18T03:31:47.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.987+0000 7f119cd55b80 1 Processor -- start 2023-07-18T03:31:47.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.987+0000 7f119cd55b80 1 -- start start 2023-07-18T03:31:47.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.987+0000 7f119cd55b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560cc6fff2d0 0x560cc70022c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:47.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.987+0000 7f119cd55b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560cc6fffbe0 0x560cc7002b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:47.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.987+0000 7f119cd55b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560cc700aad0 0x560cc7003070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:47.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.987+0000 7f119cd55b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc7000540 con 0x560cc700aad0 2023-07-18T03:31:47.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.988+0000 7f119cd55b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc7000d60 con 0x560cc6fff2d0 2023-07-18T03:31:47.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.988+0000 7f119cd55b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc6ff2fa0 con 0x560cc6fffbe0 2023-07-18T03:31:47.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.988+0000 7f1182ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560cc6fff2d0 0x560cc70022c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:47.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.988+0000 7f11837fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560cc700aad0 0x560cc7003070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:47.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.988+0000 7f11827fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560cc6fffbe0 0x560cc7002b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:47.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.988+0000 7f1182ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560cc6fff2d0 0x560cc70022c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:36436/0 (socket says 172.21.15.101:36436) 2023-07-18T03:31:47.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.988+0000 7f1182ffd700 1 -- 172.21.15.101:0/2183206182 learned_addr learned my addr 172.21.15.101:0/2183206182 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:31:47.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.988+0000 7f1182ffd700 1 -- 172.21.15.101:0/2183206182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560cc6fffbe0 msgr2=0x560cc7002b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:47.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.988+0000 7f1182ffd700 1 --2- 172.21.15.101:0/2183206182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560cc6fffbe0 0x560cc7002b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:47.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.988+0000 7f1182ffd700 1 -- 172.21.15.101:0/2183206182 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560cc700aad0 msgr2=0x560cc7003070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:47.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.988+0000 7f1182ffd700 1 --2- 172.21.15.101:0/2183206182 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560cc700aad0 0x560cc7003070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:47.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.988+0000 7f1182ffd700 1 -- 172.21.15.101:0/2183206182 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560cc6d68c60 con 0x560cc6fff2d0 2023-07-18T03:31:47.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.988+0000 7f1182ffd700 1 --2- 172.21.15.101:0/2183206182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560cc6fff2d0 0x560cc70022c0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1174096990 tx=0x7f11740693a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db771ab0e319bbc5 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:47.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.988+0000 7f1169ffb700 1 -- 172.21.15.101:0/2183206182 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117409df50 con 0x560cc6fff2d0 2023-07-18T03:31:47.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.989+0000 7f1169ffb700 1 -- 172.21.15.101:0/2183206182 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11740c8020 con 0x560cc6fff2d0 2023-07-18T03:31:48.000 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.989+0000 7f1169ffb700 1 -- 172.21.15.101:0/2183206182 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117409a3e0 con 0x560cc6fff2d0 2023-07-18T03:31:48.000 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.989+0000 7f119cd55b80 1 -- 172.21.15.101:0/2183206182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560cc6fff2d0 msgr2=0x560cc70022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:48.000 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.989+0000 7f119cd55b80 1 --2- 172.21.15.101:0/2183206182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560cc6fff2d0 0x560cc70022c0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1174096990 tx=0x7f11740693a0 comp rx=0 tx=0).stop 2023-07-18T03:31:48.000 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.989+0000 7f119cd55b80 1 -- 172.21.15.101:0/2183206182 shutdown_connections 2023-07-18T03:31:48.000 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.989+0000 7f119cd55b80 1 --2- 172.21.15.101:0/2183206182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560cc6fff2d0 0x560cc70022c0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:48.001 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.989+0000 7f119cd55b80 1 --2- 172.21.15.101:0/2183206182 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560cc700aad0 0x560cc7003070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:48.001 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.989+0000 7f119cd55b80 1 --2- 172.21.15.101:0/2183206182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560cc6fffbe0 0x560cc7002b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:48.001 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.989+0000 7f119cd55b80 1 -- 172.21.15.101:0/2183206182 >> 172.21.15.101:0/2183206182 conn(0x560cc6ff5da0 msgr2=0x560cc7016780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:48.001 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.989+0000 7f119cd55b80 1 -- 172.21.15.101:0/2183206182 shutdown_connections 2023-07-18T03:31:48.001 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.989+0000 7f119cd55b80 1 -- 172.21.15.101:0/2183206182 wait complete. 2023-07-18T03:31:48.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.990+0000 7f119cd55b80 1 -- start start 2023-07-18T03:31:48.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.990+0000 7f119cd55b80 1 Processor -- start 2023-07-18T03:31:48.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.990+0000 7f119cd55b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560cc6fff2d0 0x560cc70115d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:48.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.990+0000 7f119cd55b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560cc6fffbe0 0x560cc700f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:48.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.990+0000 7f119cd55b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560cc700aad0 0x560cc700fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:48.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.990+0000 7f119cd55b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc7010550 con 0x560cc6fff2d0 2023-07-18T03:31:48.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.990+0000 7f119cd55b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc70106c0 con 0x560cc6fffbe0 2023-07-18T03:31:48.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.990+0000 7f119cd55b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc706e2b0 con 0x560cc700aad0 2023-07-18T03:31:48.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.990+0000 7f119cd55b80 10 client.0 _ll_register_callbacks cb 0x7f1174010b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:31:48.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.991+0000 7f1182ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560cc6fff2d0 0x560cc70115d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:48.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.991+0000 7f11837fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560cc700aad0 0x560cc700fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:48.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.991+0000 7f11837fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560cc700aad0 0x560cc700fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:55520/0 (socket says 172.21.15.101:55520) 2023-07-18T03:31:48.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.991+0000 7f1182ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560cc6fff2d0 0x560cc70115d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:57362/0 (socket says 172.21.15.101:57362) 2023-07-18T03:31:48.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.991+0000 7f11837fe700 1 -- 172.21.15.101:0/2650464092 learned_addr learned my addr 172.21.15.101:0/2650464092 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:31:48.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.991+0000 7f11827fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560cc6fffbe0 0x560cc700f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:48.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.991+0000 7f1182ffd700 1 -- 172.21.15.101:0/2650464092 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560cc700aad0 msgr2=0x560cc700fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:48.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.991+0000 7f1182ffd700 1 --2- 172.21.15.101:0/2650464092 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560cc700aad0 0x560cc700fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:48.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.991+0000 7f1182ffd700 1 -- 172.21.15.101:0/2650464092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560cc6fffbe0 msgr2=0x560cc700f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:48.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.991+0000 7f1182ffd700 1 --2- 172.21.15.101:0/2650464092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560cc6fffbe0 0x560cc700f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:48.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.991+0000 7f1182ffd700 1 -- 172.21.15.101:0/2650464092 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11841a4750 con 0x560cc6fff2d0 2023-07-18T03:31:48.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.991+0000 7f11827fc700 1 --2- 172.21.15.101:0/2650464092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560cc6fffbe0 0x560cc700f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:31:48.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.991+0000 7f1182ffd700 1 --2- 172.21.15.101:0/2650464092 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560cc6fff2d0 0x560cc70115d0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f1174096990 tx=0x7f1174098060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a1da8ba39ea91da server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:48.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.991+0000 7f1166ff5700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:31:48.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.991+0000 7f1166ff5700 1 -- 172.21.15.101:0/2650464092 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11740a2370 con 0x560cc6fff2d0 2023-07-18T03:31:48.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.991+0000 7f1166ff5700 1 -- 172.21.15.101:0/2650464092 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(128..128 src has 1..128) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f11740c9070 con 0x560cc6fff2d0 2023-07-18T03:31:48.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.991+0000 7f119cd55b80 1 -- 172.21.15.101:0/2650464092 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1178019c50 con 0x560cc6fff2d0 2023-07-18T03:31:48.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.992+0000 7f1166ff5700 1 -- 172.21.15.101:0/2650464092 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11740ca2a0 con 0x560cc6fff2d0 2023-07-18T03:31:48.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.992+0000 7f119cd55b80 10 client.19074 fetch_fsmap learned FSMap version 49 2023-07-18T03:31:48.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.992+0000 7f119cd55b80 1 -- 172.21.15.101:0/2650464092 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x560cc6da6250 con 0x560cc6fff2d0 2023-07-18T03:31:48.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.992+0000 7f1166ff5700 1 -- 172.21.15.101:0/2650464092 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1178019c50 con 0x560cc6fff2d0 2023-07-18T03:31:48.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.992+0000 7f1166ff5700 1 -- 172.21.15.101:0/2650464092 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 49) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f117409a180 con 0x560cc6fff2d0 2023-07-18T03:31:48.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.992+0000 7f119cd55b80 10 client.19074 fetch_fsmap finished waiting for FSMap version 49 2023-07-18T03:31:48.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.992+0000 7f119cd55b80 10 client.19074 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:31:48.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.992+0000 7f119cd55b80 20 client.19074 populate_metadata read hostname 'smithi101' 2023-07-18T03:31:48.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.992+0000 7f119cd55b80 1 --2- 172.21.15.101:0/2650464092 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x560cc706f740 0x560cc7071b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:48.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.992+0000 7f119cd55b80 4 client.19074 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:31:48.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.992+0000 7f119cd55b80 1 -- 172.21.15.101:0/2650464092 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560cc6f0d050 con 0x560cc706f740 2023-07-18T03:31:48.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.992+0000 7f11827fc700 1 --2- 172.21.15.101:0/2650464092 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x560cc706f740 0x560cc7071b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:48.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.993+0000 7f1166ff5700 10 client.19074 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:31:48.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.993+0000 7f11827fc700 1 --2- 172.21.15.101:0/2650464092 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x560cc706f740 0x560cc7071b20 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5efd8aa73373c4f server_cookie=b700babcc5c77a8a in_seq=0 out_seq=0 2023-07-18T03:31:48.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.993+0000 7f1166ff5700 1 -- 172.21.15.101:0/2650464092 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560cc6f0d050 con 0x560cc706f740 2023-07-18T03:31:48.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.993+0000 7f1166ff5700 10 client.19074 handle_command_reply: tid=0 2023-07-18T03:31:48.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.993+0000 7f119cd55b80 1 client.19074 shutdown 2023-07-18T03:31:48.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.993+0000 7f119cd55b80 1 -- 172.21.15.101:0/2650464092 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560cc6fff2d0 msgr2=0x560cc70115d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:48.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.993+0000 7f119cd55b80 1 --2- 172.21.15.101:0/2650464092 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560cc6fff2d0 0x560cc70115d0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f1174096990 tx=0x7f1174098060 comp rx=0 tx=0).stop 2023-07-18T03:31:48.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.993+0000 7f119cd55b80 1 -- 172.21.15.101:0/2650464092 shutdown_connections 2023-07-18T03:31:48.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.993+0000 7f119cd55b80 1 --2- 172.21.15.101:0/2650464092 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x560cc706f740 0x560cc7071b20 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:48.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.993+0000 7f119cd55b80 1 --2- 172.21.15.101:0/2650464092 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560cc700aad0 0x560cc700fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:48.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.993+0000 7f119cd55b80 1 --2- 172.21.15.101:0/2650464092 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560cc6fff2d0 0x560cc70115d0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:48.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.993+0000 7f1166ff5700 0 client.19074 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:31:48.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.993+0000 7f119cd55b80 1 --2- 172.21.15.101:0/2650464092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560cc6fffbe0 0x560cc700f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:48.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.993+0000 7f119cd55b80 1 -- 172.21.15.101:0/2650464092 >> 172.21.15.101:0/2650464092 conn(0x560cc6ff5da0 msgr2=0x560cc6ffa8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:48.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.993+0000 7f119cd55b80 1 -- 172.21.15.101:0/2650464092 shutdown_connections 2023-07-18T03:31:48.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.993+0000 7f119cd55b80 1 -- 172.21.15.101:0/2650464092 wait complete. 2023-07-18T03:31:48.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.994+0000 7f119cd55b80 20 client.19074 trim_cache size 0 max 16384 2023-07-18T03:31:48.015 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:31:48.015 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:31:48.015 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:31:48.016 INFO:teuthology.orchestra.run.smithi101.stdout: "0f2f19e0-0429-465c-935e-223eb09490c3": { 2023-07-18T03:31:48.016 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:31:48.016 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "0f2f19e0-0429-465c-935e-223eb09490c3", 2023-07-18T03:31:48.016 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:31:48.016 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:31:48.017 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:31:48.017 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:31:48.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.994+0000 7f11893ff700 1 -- 172.21.15.101:0/3313161835 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f116c0a12c0 msgr2=0x7f116c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:48.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.994+0000 7f11893ff700 1 --2- 172.21.15.101:0/3313161835 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f116c0a12c0 0x7f116c0a3770 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f117c00b780 tx=0x7f117c006230 comp rx=0 tx=0).stop 2023-07-18T03:31:48.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.994+0000 7f11893ff700 1 -- 172.21.15.101:0/3313161835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11840ffae0 msgr2=0x7f1184072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:48.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.994+0000 7f11893ff700 1 --2- 172.21.15.101:0/3313161835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11840ffae0 0x7f1184072810 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f117400e9a0 tx=0x7f117400ecb0 comp rx=0 tx=0).stop 2023-07-18T03:31:48.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.994+0000 7f11893ff700 1 -- 172.21.15.101:0/3313161835 shutdown_connections 2023-07-18T03:31:48.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.994+0000 7f11893ff700 1 --2- 172.21.15.101:0/3313161835 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f116c0a12c0 0x7f116c0a3770 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:48.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.994+0000 7f11893ff700 1 --2- 172.21.15.101:0/3313161835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11840ffae0 0x7f1184072810 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:48.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.994+0000 7f11893ff700 1 --2- 172.21.15.101:0/3313161835 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1184105af0 0x7f1184073290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:48.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.994+0000 7f11893ff700 1 --2- 172.21.15.101:0/3313161835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1184100490 0x7f1184072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:48.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.994+0000 7f11893ff700 1 -- 172.21.15.101:0/3313161835 >> 172.21.15.101:0/3313161835 conn(0x7f11840fb5e0 msgr2=0x7f11841035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:48.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.994+0000 7f11893ff700 1 -- 172.21.15.101:0/3313161835 shutdown_connections 2023-07-18T03:31:48.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:31:47.994+0000 7f11893ff700 1 -- 172.21.15.101:0/3313161835 wait complete. 2023-07-18T03:31:48.021 INFO:tasks.cephfs.filesystem:scrub status for tag:0f2f19e0-0429-465c-935e-223eb09490c3 - {'path': '/', 'tag': '0f2f19e0-0429-465c-935e-223eb09490c3', 'options': 'recursive,force'} 2023-07-18T03:31:49.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:31:49 smithi179 ceph-mon[129229]: pgmap v766: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:31:49.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:31:49 smithi101 ceph-mon[126379]: pgmap v766: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:31:49.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:31:49 smithi155 ceph-mon[128885]: pgmap v766: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:31:51.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:31:51 smithi179 ceph-mon[129229]: pgmap v767: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-18T03:31:51.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:31:51 smithi101 ceph-mon[126379]: pgmap v767: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-18T03:31:51.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:31:51 smithi155 ceph-mon[128885]: pgmap v767: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-18T03:31:53.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:31:53 smithi179 ceph-mon[129229]: pgmap v768: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:31:53.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:31:53 smithi101 ceph-mon[126379]: pgmap v768: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:31:53.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:31:53 smithi155 ceph-mon[128885]: pgmap v768: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:31:55.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:31:55 smithi179 ceph-mon[129229]: pgmap v769: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:31:55.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:31:55 smithi101 ceph-mon[126379]: pgmap v769: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:31:55.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:31:55 smithi155 ceph-mon[128885]: pgmap v769: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:31:57.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:31:57 smithi179 ceph-mon[129229]: pgmap v770: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:31:57.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:31:57 smithi101 ceph-mon[126379]: pgmap v770: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:31:57.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:31:57 smithi155 ceph-mon[128885]: pgmap v770: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:31:59.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:31:59 smithi179 ceph-mon[129229]: pgmap v771: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:31:59.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:31:59 smithi101 ceph-mon[126379]: pgmap v771: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:31:59.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:31:59 smithi155 ceph-mon[128885]: pgmap v771: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:32:01.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:32:01 smithi179 ceph-mon[129229]: pgmap v772: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-18T03:32:01.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:32:01 smithi101 ceph-mon[126379]: pgmap v772: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-18T03:32:01.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:32:01 smithi155 ceph-mon[128885]: pgmap v772: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-18T03:32:03.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:32:03 smithi179 ceph-mon[129229]: pgmap v773: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-07-18T03:32:03.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:32:03 smithi101 ceph-mon[126379]: pgmap v773: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-07-18T03:32:03.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:32:03 smithi155 ceph-mon[128885]: pgmap v773: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-07-18T03:32:05.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:32:05 smithi179 ceph-mon[129229]: pgmap v774: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-07-18T03:32:05.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:32:05 smithi101 ceph-mon[126379]: pgmap v774: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-07-18T03:32:05.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:32:05 smithi155 ceph-mon[128885]: pgmap v774: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-07-18T03:32:07.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:32:07 smithi179 ceph-mon[129229]: pgmap v775: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-07-18T03:32:07.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:32:07 smithi101 ceph-mon[126379]: pgmap v775: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-07-18T03:32:07.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:32:07 smithi155 ceph-mon[128885]: pgmap v775: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-07-18T03:32:09.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:32:09 smithi179 ceph-mon[129229]: pgmap v776: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-07-18T03:32:09.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:32:09 smithi101 ceph-mon[126379]: pgmap v776: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-07-18T03:32:09.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:32:09 smithi155 ceph-mon[128885]: pgmap v776: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-07-18T03:32:11.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:32:11 smithi179 ceph-mon[129229]: pgmap v777: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-07-18T03:32:11.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:32:11 smithi179 ceph-mon[129229]: osdmap e129: 12 total, 12 up, 12 in 2023-07-18T03:32:11.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:32:11 smithi101 ceph-mon[126379]: pgmap v777: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-07-18T03:32:11.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:32:11 smithi101 ceph-mon[126379]: osdmap e129: 12 total, 12 up, 12 in 2023-07-18T03:32:11.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:32:11 smithi155 ceph-mon[128885]: pgmap v777: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-07-18T03:32:11.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:32:11 smithi155 ceph-mon[128885]: osdmap e129: 12 total, 12 up, 12 in 2023-07-18T03:32:13.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:32:13 smithi179 ceph-mon[129229]: pgmap v779: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 27 KiB/s wr, 21 op/s 2023-07-18T03:32:13.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:32:13 smithi101 ceph-mon[126379]: pgmap v779: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 27 KiB/s wr, 21 op/s 2023-07-18T03:32:13.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:32:13 smithi155 ceph-mon[128885]: pgmap v779: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 27 KiB/s wr, 21 op/s 2023-07-18T03:32:15.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:32:15 smithi179 ceph-mon[129229]: pgmap v780: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 27 KiB/s wr, 21 op/s 2023-07-18T03:32:15.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:32:15 smithi101 ceph-mon[126379]: pgmap v780: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 27 KiB/s wr, 21 op/s 2023-07-18T03:32:15.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:32:15 smithi155 ceph-mon[128885]: pgmap v780: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 27 KiB/s wr, 21 op/s 2023-07-18T03:32:17.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:32:17 smithi179 ceph-mon[129229]: pgmap v781: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:32:17.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:32:17 smithi101 ceph-mon[126379]: pgmap v781: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:32:17.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:32:17 smithi155 ceph-mon[128885]: pgmap v781: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:32:18.022 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:32:18.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.236+0000 7f005dfff700 1 -- 172.21.15.101:0/1333099096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00580ffae0 msgr2=0x7f00580fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:18.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.236+0000 7f005dfff700 1 --2- 172.21.15.101:0/1333099096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00580ffae0 0x7f00580fff50 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f00400098f0 tx=0x7f0040009c00 comp rx=0 tx=0).stop 2023-07-18T03:32:18.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.236+0000 7f005dfff700 1 -- 172.21.15.101:0/1333099096 shutdown_connections 2023-07-18T03:32:18.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.236+0000 7f005dfff700 1 --2- 172.21.15.101:0/1333099096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0058100490 0x7f005810c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:18.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.236+0000 7f005dfff700 1 --2- 172.21.15.101:0/1333099096 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0058105af0 0x7f0058105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:18.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.236+0000 7f005dfff700 1 --2- 172.21.15.101:0/1333099096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00580ffae0 0x7f00580fff50 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:18.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.236+0000 7f005dfff700 1 -- 172.21.15.101:0/1333099096 >> 172.21.15.101:0/1333099096 conn(0x7f00580fb5e0 msgr2=0x7f00580fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:18.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.236+0000 7f005dfff700 1 -- 172.21.15.101:0/1333099096 shutdown_connections 2023-07-18T03:32:18.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.236+0000 7f005dfff700 1 -- 172.21.15.101:0/1333099096 wait complete. 2023-07-18T03:32:18.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.237+0000 7f005dfff700 1 Processor -- start 2023-07-18T03:32:18.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.237+0000 7f005dfff700 1 -- start start 2023-07-18T03:32:18.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.238+0000 7f005dfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00580ffae0 0x7f005819ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:18.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.238+0000 7f005dfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0058100490 0x7f005819bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:18.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.238+0000 7f005dfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0058105af0 0x7f0058195ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:18.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.238+0000 7f005dfff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0058196120 con 0x7f0058105af0 2023-07-18T03:32:18.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.238+0000 7f005dfff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0058196290 con 0x7f00580ffae0 2023-07-18T03:32:18.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.238+0000 7f005dfff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0058196450 con 0x7f0058100490 2023-07-18T03:32:18.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.238+0000 7f004ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0058100490 0x7f005819bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:18.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.238+0000 7f00577fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00580ffae0 0x7f005819ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:18.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.238+0000 7f0057fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0058105af0 0x7f0058195ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:18.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.238+0000 7f004ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0058100490 0x7f005819bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:36656/0 (socket says 172.21.15.101:36656) 2023-07-18T03:32:18.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.238+0000 7f00577fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00580ffae0 0x7f005819ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:50376/0 (socket says 172.21.15.101:50376) 2023-07-18T03:32:18.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.238+0000 7f004ffff700 1 -- 172.21.15.101:0/3236355986 learned_addr learned my addr 172.21.15.101:0/3236355986 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:32:18.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.238+0000 7f004ffff700 1 -- 172.21.15.101:0/3236355986 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00580ffae0 msgr2=0x7f005819ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:18.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.238+0000 7f004ffff700 1 --2- 172.21.15.101:0/3236355986 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00580ffae0 0x7f005819ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:18.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.238+0000 7f004ffff700 1 -- 172.21.15.101:0/3236355986 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0058105af0 msgr2=0x7f0058195ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:18.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.238+0000 7f004ffff700 1 --2- 172.21.15.101:0/3236355986 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0058105af0 0x7f0058195ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:18.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.238+0000 7f004ffff700 1 -- 172.21.15.101:0/3236355986 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0050009d00 con 0x7f0058100490 2023-07-18T03:32:18.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.238+0000 7f00577fe700 1 --2- 172.21.15.101:0/3236355986 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00580ffae0 0x7f005819ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:32:18.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.238+0000 7f0057fff700 1 --2- 172.21.15.101:0/3236355986 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0058105af0 0x7f0058195ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:32:18.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.239+0000 7f004ffff700 1 --2- 172.21.15.101:0/3236355986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0058100490 0x7f005819bf80 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f0040009810 tx=0x7f004000fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:18.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.239+0000 7f00557fa700 1 -- 172.21.15.101:0/3236355986 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004001c070 con 0x7f0058100490 2023-07-18T03:32:18.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.239+0000 7f005dfff700 1 -- 172.21.15.101:0/3236355986 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0048009520 con 0x7f0058100490 2023-07-18T03:32:18.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.239+0000 7f00557fa700 1 -- 172.21.15.101:0/3236355986 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f004000bb20 con 0x7f0058100490 2023-07-18T03:32:18.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.239+0000 7f00557fa700 1 -- 172.21.15.101:0/3236355986 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0040004930 con 0x7f0058100490 2023-07-18T03:32:18.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.239+0000 7f005dfff700 1 -- 172.21.15.101:0/3236355986 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0040009520 con 0x7f0058100490 2023-07-18T03:32:18.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.243+0000 7f00557fa700 1 -- 172.21.15.101:0/3236355986 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f0040020070 con 0x7f0058100490 2023-07-18T03:32:18.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.244+0000 7f007193bb80 1 Processor -- start 2023-07-18T03:32:18.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.245+0000 7f007193bb80 1 -- start start 2023-07-18T03:32:18.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.245+0000 7f007193bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a6ee202d0 0x563a6ee206a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:18.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.245+0000 7f007193bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6ee20be0 0x563a6ee2f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:18.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.245+0000 7f007193bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a6ee2fee0 0x563a6ee322c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:18.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.245+0000 7f007193bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a6ed26cb0 con 0x563a6ee2fee0 2023-07-18T03:32:18.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.245+0000 7f007193bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a6ed79a40 con 0x563a6ee20be0 2023-07-18T03:32:18.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.245+0000 7f007193bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a6ed72ed0 con 0x563a6ee202d0 2023-07-18T03:32:18.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.245+0000 7f00557fa700 1 --2- 172.21.15.101:0/3236355986 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f00440a12c0 0x7f00440a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:18.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.246+0000 7f00577fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6ee20be0 0x563a6ee2f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:18.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.246+0000 7f00577fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6ee20be0 0x563a6ee2f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:50386/0 (socket says 172.21.15.101:50386) 2023-07-18T03:32:18.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.246+0000 7f00577fe700 1 -- 172.21.15.101:0/3325556809 learned_addr learned my addr 172.21.15.101:0/3325556809 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:32:18.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.246+0000 7f004ffff700 1 --2- 172.21.15.101:0/3325556809 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a6ee2fee0 0x563a6ee322c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:18.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.246+0000 7f0057fff700 1 --2- 172.21.15.101:0/3325556809 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a6ee202d0 0x563a6ee206a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:18.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.246+0000 7f00557fa700 1 -- 172.21.15.101:0/3236355986 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f00400cddf0 con 0x7f0058100490 2023-07-18T03:32:18.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.246+0000 7f0057fff700 1 --2- 172.21.15.101:0/3236355986 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f00440a12c0 0x7f00440a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:18.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.246+0000 7f00577fe700 1 -- 172.21.15.101:0/3325556809 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a6ee202d0 msgr2=0x563a6ee206a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:18.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.246+0000 7f00577fe700 1 --2- 172.21.15.101:0/3325556809 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a6ee202d0 0x563a6ee206a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:18.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.246+0000 7f00577fe700 1 -- 172.21.15.101:0/3325556809 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a6ee2fee0 msgr2=0x563a6ee322c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:18.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.246+0000 7f00577fe700 1 --2- 172.21.15.101:0/3325556809 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a6ee2fee0 0x563a6ee322c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:18.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.246+0000 7f00577fe700 1 -- 172.21.15.101:0/3325556809 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a6eba0de0 con 0x563a6ee20be0 2023-07-18T03:32:18.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.246+0000 7f004ffff700 1 --2- 172.21.15.101:0/3325556809 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a6ee2fee0 0x563a6ee322c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:32:18.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.246+0000 7f00577fe700 1 --2- 172.21.15.101:0/3325556809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6ee20be0 0x563a6ee2f9a0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f00480117e0 tx=0x7f0048011af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a1743d0126db899 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:18.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.246+0000 7f0057fff700 1 --2- 172.21.15.101:0/3236355986 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f00440a12c0 0x7f00440a3770 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f005819c740 tx=0x7f005000b040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:18.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.246+0000 7f004dffb700 1 -- 172.21.15.101:0/3325556809 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004800f4e0 con 0x563a6ee20be0 2023-07-18T03:32:18.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.247+0000 7f004dffb700 1 -- 172.21.15.101:0/3325556809 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f004800fb00 con 0x563a6ee20be0 2023-07-18T03:32:18.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.247+0000 7f004dffb700 1 -- 172.21.15.101:0/3325556809 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0048019bd0 con 0x563a6ee20be0 2023-07-18T03:32:18.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.247+0000 7f007193bb80 1 -- 172.21.15.101:0/3325556809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6ee20be0 msgr2=0x563a6ee2f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:18.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.247+0000 7f007193bb80 1 --2- 172.21.15.101:0/3325556809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6ee20be0 0x563a6ee2f9a0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f00480117e0 tx=0x7f0048011af0 comp rx=0 tx=0).stop 2023-07-18T03:32:18.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.247+0000 7f007193bb80 1 -- 172.21.15.101:0/3325556809 shutdown_connections 2023-07-18T03:32:18.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.247+0000 7f007193bb80 1 --2- 172.21.15.101:0/3325556809 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a6ee2fee0 0x563a6ee322c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:18.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.247+0000 7f007193bb80 1 --2- 172.21.15.101:0/3325556809 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a6ee202d0 0x563a6ee206a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:18.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.247+0000 7f007193bb80 1 --2- 172.21.15.101:0/3325556809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6ee20be0 0x563a6ee2f9a0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:18.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.247+0000 7f007193bb80 1 -- 172.21.15.101:0/3325556809 >> 172.21.15.101:0/3325556809 conn(0x563a6ee16da0 msgr2=0x563a6ee1c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:18.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.247+0000 7f007193bb80 1 -- 172.21.15.101:0/3325556809 shutdown_connections 2023-07-18T03:32:18.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.247+0000 7f007193bb80 1 -- 172.21.15.101:0/3325556809 wait complete. 2023-07-18T03:32:18.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.248+0000 7f007193bb80 1 -- start start 2023-07-18T03:32:18.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.248+0000 7f007193bb80 1 Processor -- start 2023-07-18T03:32:18.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.248+0000 7f007193bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a6ee202d0 0x563a6ee2f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:18.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.248+0000 7f007193bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a6ee20be0 0x563a6ee2cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:18.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.248+0000 7f007193bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6ee2fee0 0x563a6ee2d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:18.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.248+0000 7f007193bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a6ed12ac0 con 0x563a6ee20be0 2023-07-18T03:32:18.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.248+0000 7f007193bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a6ed127c0 con 0x563a6ee2fee0 2023-07-18T03:32:18.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.248+0000 7f007193bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a6ebf6920 con 0x563a6ee202d0 2023-07-18T03:32:18.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.248+0000 7f007193bb80 10 client.0 _ll_register_callbacks cb 0x563a6ed10760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:32:18.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.249+0000 7f00577fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a6ee202d0 0x563a6ee2f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:18.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.249+0000 7f0057fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6ee2fee0 0x563a6ee2d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:18.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.249+0000 7f00577fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a6ee202d0 0x563a6ee2f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:36682/0 (socket says 172.21.15.101:36682) 2023-07-18T03:32:18.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.249+0000 7f00577fe700 1 -- 172.21.15.101:0/113057338 learned_addr learned my addr 172.21.15.101:0/113057338 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:32:18.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.249+0000 7f004ffff700 1 --2- 172.21.15.101:0/113057338 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a6ee20be0 0x563a6ee2cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:18.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.249+0000 7f00577fe700 1 -- 172.21.15.101:0/113057338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6ee2fee0 msgr2=0x563a6ee2d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:18.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.249+0000 7f00577fe700 1 --2- 172.21.15.101:0/113057338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6ee2fee0 0x563a6ee2d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:18.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.249+0000 7f00577fe700 1 -- 172.21.15.101:0/113057338 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a6ee20be0 msgr2=0x563a6ee2cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:18.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.249+0000 7f00577fe700 1 --2- 172.21.15.101:0/113057338 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a6ee20be0 0x563a6ee2cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:18.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.249+0000 7f00577fe700 1 -- 172.21.15.101:0/113057338 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0048011440 con 0x563a6ee202d0 2023-07-18T03:32:18.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.249+0000 7f004ffff700 1 --2- 172.21.15.101:0/113057338 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a6ee20be0 0x563a6ee2cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:32:18.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.249+0000 7f00577fe700 1 --2- 172.21.15.101:0/113057338 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a6ee202d0 0x563a6ee2f850 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f0048000c00 tx=0x7f004800f7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e0a4d231474ee3c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:18.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.249+0000 7f0036ffd700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:32:18.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.249+0000 7f0036ffd700 1 -- 172.21.15.101:0/113057338 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0048004250 con 0x563a6ee202d0 2023-07-18T03:32:18.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.249+0000 7f0036ffd700 1 -- 172.21.15.101:0/113057338 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f00480043b0 con 0x563a6ee202d0 2023-07-18T03:32:18.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.250+0000 7f007193bb80 1 -- 172.21.15.101:0/113057338 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563a6eccc4f0 con 0x563a6ee202d0 2023-07-18T03:32:18.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.250+0000 7f0036ffd700 1 -- 172.21.15.101:0/113057338 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0048004570 con 0x563a6ee202d0 2023-07-18T03:32:18.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.250+0000 7f007193bb80 10 client.29182 fetch_fsmap learned FSMap version 49 2023-07-18T03:32:18.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.250+0000 7f007193bb80 1 -- 172.21.15.101:0/113057338 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x563a6eb89c60 con 0x563a6ee202d0 2023-07-18T03:32:18.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.250+0000 7f0036ffd700 1 -- 172.21.15.101:0/113057338 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563a6eccc4f0 con 0x563a6ee202d0 2023-07-18T03:32:18.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.250+0000 7f0036ffd700 1 -- 172.21.15.101:0/113057338 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 49) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f00480052b0 con 0x563a6ee202d0 2023-07-18T03:32:18.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.250+0000 7f007193bb80 10 client.29182 fetch_fsmap finished waiting for FSMap version 49 2023-07-18T03:32:18.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.250+0000 7f007193bb80 10 client.29182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:32:18.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.250+0000 7f007193bb80 20 client.29182 populate_metadata read hostname 'smithi101' 2023-07-18T03:32:18.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.250+0000 7f007193bb80 1 --2- 172.21.15.101:0/113057338 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563a6ee3f5e0 0x563a6ee419f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:18.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.250+0000 7f007193bb80 4 client.29182 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:32:18.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.250+0000 7f007193bb80 1 -- 172.21.15.101:0/113057338 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563a6ed6beb0 con 0x563a6ee3f5e0 2023-07-18T03:32:18.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.251+0000 7f004ffff700 1 --2- 172.21.15.101:0/113057338 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563a6ee3f5e0 0x563a6ee419f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:18.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.251+0000 7f004ffff700 1 --2- 172.21.15.101:0/113057338 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563a6ee3f5e0 0x563a6ee419f0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=688a014a568ef0dd server_cookie=d84f44d717275984 in_seq=0 out_seq=0 2023-07-18T03:32:18.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.251+0000 7f0036ffd700 10 client.29182 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:32:18.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.252+0000 7f0036ffd700 1 -- 172.21.15.101:0/113057338 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563a6ed6beb0 con 0x563a6ee3f5e0 2023-07-18T03:32:18.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.252+0000 7f0036ffd700 10 client.29182 handle_command_reply: tid=0 2023-07-18T03:32:18.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.252+0000 7f007193bb80 1 client.29182 shutdown 2023-07-18T03:32:18.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.252+0000 7f007193bb80 1 -- 172.21.15.101:0/113057338 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a6ee202d0 msgr2=0x563a6ee2f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:18.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.252+0000 7f007193bb80 1 --2- 172.21.15.101:0/113057338 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a6ee202d0 0x563a6ee2f850 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f0048000c00 tx=0x7f004800f7c0 comp rx=0 tx=0).stop 2023-07-18T03:32:18.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.252+0000 7f007193bb80 1 -- 172.21.15.101:0/113057338 shutdown_connections 2023-07-18T03:32:18.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.252+0000 7f007193bb80 1 --2- 172.21.15.101:0/113057338 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563a6ee3f5e0 0x563a6ee419f0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:18.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.252+0000 7f007193bb80 1 --2- 172.21.15.101:0/113057338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6ee2fee0 0x563a6ee2d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:18.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.252+0000 7f0036ffd700 0 client.29182 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:32:18.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.252+0000 7f007193bb80 1 --2- 172.21.15.101:0/113057338 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a6ee202d0 0x563a6ee2f850 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:18.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.252+0000 7f007193bb80 1 --2- 172.21.15.101:0/113057338 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a6ee20be0 0x563a6ee2cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:18.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.252+0000 7f007193bb80 1 -- 172.21.15.101:0/113057338 >> 172.21.15.101:0/113057338 conn(0x563a6ee16da0 msgr2=0x563a6ee1c290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:18.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.252+0000 7f007193bb80 1 -- 172.21.15.101:0/113057338 shutdown_connections 2023-07-18T03:32:18.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.252+0000 7f007193bb80 1 -- 172.21.15.101:0/113057338 wait complete. 2023-07-18T03:32:18.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.252+0000 7f007193bb80 20 client.29182 trim_cache size 0 max 16384 2023-07-18T03:32:18.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.269+0000 7f007193bb80 1 Processor -- start 2023-07-18T03:32:18.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.269+0000 7f007193bb80 1 -- start start 2023-07-18T03:32:18.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.270+0000 7f007193bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6ee202d0 0x563a6ee2f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:18.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.270+0000 7f007193bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a6ee20be0 0x563a6ee29e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:18.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.270+0000 7f007193bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a6ee2fee0 0x563a6ee2a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:18.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.270+0000 7f007193bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a6eccc4f0 con 0x563a6ee2fee0 2023-07-18T03:32:18.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.270+0000 7f007193bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a6ee2db00 con 0x563a6ee202d0 2023-07-18T03:32:18.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.270+0000 7f007193bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a6ee21540 con 0x563a6ee20be0 2023-07-18T03:32:18.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.270+0000 7f00577fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6ee202d0 0x563a6ee2f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:18.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.270+0000 7f004ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a6ee20be0 0x563a6ee29e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:18.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.270+0000 7f00577fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6ee202d0 0x563a6ee2f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:50402/0 (socket says 172.21.15.101:50402) 2023-07-18T03:32:18.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.270+0000 7f004ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a6ee20be0 0x563a6ee29e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:36692/0 (socket says 172.21.15.101:36692) 2023-07-18T03:32:18.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.270+0000 7f00577fe700 1 -- 172.21.15.101:0/1037521483 learned_addr learned my addr 172.21.15.101:0/1037521483 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:32:18.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.270+0000 7f0057fff700 1 --2- 172.21.15.101:0/1037521483 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a6ee2fee0 0x563a6ee2a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:18.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.270+0000 7f00577fe700 1 -- 172.21.15.101:0/1037521483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a6ee20be0 msgr2=0x563a6ee29e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:18.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.270+0000 7f00577fe700 1 --2- 172.21.15.101:0/1037521483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a6ee20be0 0x563a6ee29e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:18.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.270+0000 7f00577fe700 1 -- 172.21.15.101:0/1037521483 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a6ee2fee0 msgr2=0x563a6ee2a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:18.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.270+0000 7f00577fe700 1 --2- 172.21.15.101:0/1037521483 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a6ee2fee0 0x563a6ee2a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:18.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.270+0000 7f00577fe700 1 -- 172.21.15.101:0/1037521483 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a6ebc7250 con 0x563a6ee202d0 2023-07-18T03:32:18.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.270+0000 7f004ffff700 1 --2- 172.21.15.101:0/1037521483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a6ee20be0 0x563a6ee29e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:32:18.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.270+0000 7f00577fe700 1 --2- 172.21.15.101:0/1037521483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6ee202d0 0x563a6ee2f3f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f0048009f80 tx=0x7f0048009fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=feb759b5f3ebe805 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:18.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.271+0000 7f004e7fc700 1 -- 172.21.15.101:0/1037521483 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0048005790 con 0x563a6ee202d0 2023-07-18T03:32:18.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.271+0000 7f004e7fc700 1 -- 172.21.15.101:0/1037521483 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0048019410 con 0x563a6ee202d0 2023-07-18T03:32:18.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.271+0000 7f004e7fc700 1 -- 172.21.15.101:0/1037521483 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004802b3b0 con 0x563a6ee202d0 2023-07-18T03:32:18.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.271+0000 7f007193bb80 1 -- 172.21.15.101:0/1037521483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6ee202d0 msgr2=0x563a6ee2f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:18.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.271+0000 7f007193bb80 1 --2- 172.21.15.101:0/1037521483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6ee202d0 0x563a6ee2f3f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f0048009f80 tx=0x7f0048009fb0 comp rx=0 tx=0).stop 2023-07-18T03:32:18.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.271+0000 7f007193bb80 1 -- 172.21.15.101:0/1037521483 shutdown_connections 2023-07-18T03:32:18.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.271+0000 7f007193bb80 1 --2- 172.21.15.101:0/1037521483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6ee202d0 0x563a6ee2f3f0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:18.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.271+0000 7f007193bb80 1 --2- 172.21.15.101:0/1037521483 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a6ee2fee0 0x563a6ee2a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:18.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.271+0000 7f007193bb80 1 --2- 172.21.15.101:0/1037521483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a6ee20be0 0x563a6ee29e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:18.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.271+0000 7f007193bb80 1 -- 172.21.15.101:0/1037521483 >> 172.21.15.101:0/1037521483 conn(0x563a6ee16da0 msgr2=0x563a6ee3bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:18.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.271+0000 7f007193bb80 1 -- 172.21.15.101:0/1037521483 shutdown_connections 2023-07-18T03:32:18.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.271+0000 7f007193bb80 1 -- 172.21.15.101:0/1037521483 wait complete. 2023-07-18T03:32:18.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.272+0000 7f007193bb80 1 -- start start 2023-07-18T03:32:18.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.272+0000 7f007193bb80 1 Processor -- start 2023-07-18T03:32:18.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.272+0000 7f007193bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a6ee202d0 0x563a6ee365e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:18.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.272+0000 7f007193bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6ee20be0 0x563a6ee36b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:18.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.272+0000 7f007193bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a6ee2fee0 0x563a6ee34ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:18.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.272+0000 7f007193bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a6ee35190 con 0x563a6ee2fee0 2023-07-18T03:32:18.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.272+0000 7f007193bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a6ee35300 con 0x563a6ee20be0 2023-07-18T03:32:18.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.272+0000 7f007193bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a6ee354c0 con 0x563a6ee202d0 2023-07-18T03:32:18.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.272+0000 7f007193bb80 10 client.0 _ll_register_callbacks cb 0x7f0048011660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:32:18.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.273+0000 7f00577fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a6ee202d0 0x563a6ee365e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:18.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.273+0000 7f00577fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a6ee202d0 0x563a6ee365e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:36698/0 (socket says 172.21.15.101:36698) 2023-07-18T03:32:18.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.273+0000 7f00577fe700 1 -- 172.21.15.101:0/4059668521 learned_addr learned my addr 172.21.15.101:0/4059668521 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:32:18.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.273+0000 7f004ffff700 1 --2- 172.21.15.101:0/4059668521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6ee20be0 0x563a6ee36b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:18.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.273+0000 7f0057fff700 1 --2- 172.21.15.101:0/4059668521 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a6ee2fee0 0x563a6ee34ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:18.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.273+0000 7f00577fe700 1 -- 172.21.15.101:0/4059668521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6ee20be0 msgr2=0x563a6ee36b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:18.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.273+0000 7f00577fe700 1 --2- 172.21.15.101:0/4059668521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6ee20be0 0x563a6ee36b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:18.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.273+0000 7f00577fe700 1 -- 172.21.15.101:0/4059668521 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a6ee2fee0 msgr2=0x563a6ee34ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:18.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.273+0000 7f00577fe700 1 --2- 172.21.15.101:0/4059668521 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a6ee2fee0 0x563a6ee34ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:18.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.273+0000 7f00577fe700 1 -- 172.21.15.101:0/4059668521 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0040009520 con 0x563a6ee202d0 2023-07-18T03:32:18.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.273+0000 7f00577fe700 1 --2- 172.21.15.101:0/4059668521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a6ee202d0 0x563a6ee365e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0048010000 tx=0x7f0048005ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5d2850c48f3a967 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:18.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.273+0000 7f0036ffd700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:32:18.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.273+0000 7f0036ffd700 1 -- 172.21.15.101:0/4059668521 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0048004250 con 0x563a6ee202d0 2023-07-18T03:32:18.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.273+0000 7f0036ffd700 1 -- 172.21.15.101:0/4059668521 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f0048000e40 con 0x563a6ee202d0 2023-07-18T03:32:18.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.273+0000 7f007193bb80 1 -- 172.21.15.101:0/4059668521 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f004800fb00 con 0x563a6ee202d0 2023-07-18T03:32:18.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.274+0000 7f0036ffd700 1 -- 172.21.15.101:0/4059668521 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00480043b0 con 0x563a6ee202d0 2023-07-18T03:32:18.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.274+0000 7f007193bb80 10 client.29188 fetch_fsmap learned FSMap version 49 2023-07-18T03:32:18.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.274+0000 7f007193bb80 1 -- 172.21.15.101:0/4059668521 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x563a6eb89c60 con 0x563a6ee202d0 2023-07-18T03:32:18.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.274+0000 7f0036ffd700 1 -- 172.21.15.101:0/4059668521 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f004800fb00 con 0x563a6ee202d0 2023-07-18T03:32:18.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.274+0000 7f0036ffd700 1 -- 172.21.15.101:0/4059668521 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 49) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f004802b860 con 0x563a6ee202d0 2023-07-18T03:32:18.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.274+0000 7f007193bb80 10 client.29188 fetch_fsmap finished waiting for FSMap version 49 2023-07-18T03:32:18.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.274+0000 7f007193bb80 10 client.29188 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:32:18.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.274+0000 7f007193bb80 20 client.29188 populate_metadata read hostname 'smithi101' 2023-07-18T03:32:18.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.274+0000 7f007193bb80 1 --2- 172.21.15.101:0/4059668521 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563a6ee94300 0x563a6ee966e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:18.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.274+0000 7f007193bb80 4 client.29188 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:32:18.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.274+0000 7f007193bb80 1 -- 172.21.15.101:0/4059668521 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563a6ed2e050 con 0x563a6ee94300 2023-07-18T03:32:18.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.274+0000 7f004ffff700 1 --2- 172.21.15.101:0/4059668521 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563a6ee94300 0x563a6ee966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:18.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.275+0000 7f004ffff700 1 --2- 172.21.15.101:0/4059668521 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563a6ee94300 0x563a6ee966e0 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abe07a3e8d740f0f server_cookie=3a2b078e554ed9a8 in_seq=0 out_seq=0 2023-07-18T03:32:18.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.275+0000 7f0036ffd700 10 client.29188 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:32:18.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.275+0000 7f0036ffd700 1 -- 172.21.15.101:0/4059668521 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x563a6ed2e050 con 0x563a6ee94300 2023-07-18T03:32:18.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.275+0000 7f0036ffd700 10 client.29188 handle_command_reply: tid=0 2023-07-18T03:32:18.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.275+0000 7f007193bb80 1 client.29188 shutdown 2023-07-18T03:32:18.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.275+0000 7f007193bb80 1 -- 172.21.15.101:0/4059668521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a6ee202d0 msgr2=0x563a6ee365e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:18.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.275+0000 7f007193bb80 1 --2- 172.21.15.101:0/4059668521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a6ee202d0 0x563a6ee365e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0048010000 tx=0x7f0048005ce0 comp rx=0 tx=0).stop 2023-07-18T03:32:18.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.275+0000 7f007193bb80 1 -- 172.21.15.101:0/4059668521 shutdown_connections 2023-07-18T03:32:18.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.275+0000 7f007193bb80 1 --2- 172.21.15.101:0/4059668521 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x563a6ee94300 0x563a6ee966e0 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:18.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.275+0000 7f007193bb80 1 --2- 172.21.15.101:0/4059668521 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a6ee2fee0 0x563a6ee34ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:18.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.275+0000 7f007193bb80 1 --2- 172.21.15.101:0/4059668521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a6ee202d0 0x563a6ee365e0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:18.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.275+0000 7f007193bb80 1 --2- 172.21.15.101:0/4059668521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563a6ee20be0 0x563a6ee36b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:18.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.275+0000 7f0036ffd700 0 client.29188 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:32:18.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.275+0000 7f007193bb80 1 -- 172.21.15.101:0/4059668521 >> 172.21.15.101:0/4059668521 conn(0x563a6ee16da0 msgr2=0x563a6ee221a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:18.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.275+0000 7f007193bb80 1 -- 172.21.15.101:0/4059668521 shutdown_connections 2023-07-18T03:32:18.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.275+0000 7f007193bb80 1 -- 172.21.15.101:0/4059668521 wait complete. 2023-07-18T03:32:18.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.276+0000 7f007193bb80 20 client.29188 trim_cache size 0 max 16384 2023-07-18T03:32:18.313 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:32:18.314 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:32:18.314 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:32:18.314 INFO:teuthology.orchestra.run.smithi101.stdout: "0f2f19e0-0429-465c-935e-223eb09490c3": { 2023-07-18T03:32:18.314 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:32:18.314 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "0f2f19e0-0429-465c-935e-223eb09490c3", 2023-07-18T03:32:18.315 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:32:18.315 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:32:18.315 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:32:18.315 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:32:18.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.276+0000 7f005dfff700 1 -- 172.21.15.101:0/3236355986 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f00440a12c0 msgr2=0x7f00440a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:18.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.276+0000 7f005dfff700 1 --2- 172.21.15.101:0/3236355986 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f00440a12c0 0x7f00440a3770 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f005819c740 tx=0x7f005000b040 comp rx=0 tx=0).stop 2023-07-18T03:32:18.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.276+0000 7f005dfff700 1 -- 172.21.15.101:0/3236355986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0058100490 msgr2=0x7f005819bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:18.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.276+0000 7f005dfff700 1 --2- 172.21.15.101:0/3236355986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0058100490 0x7f005819bf80 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f0040009810 tx=0x7f004000fa00 comp rx=0 tx=0).stop 2023-07-18T03:32:18.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.276+0000 7f005dfff700 1 -- 172.21.15.101:0/3236355986 shutdown_connections 2023-07-18T03:32:18.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.276+0000 7f005dfff700 1 --2- 172.21.15.101:0/3236355986 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f00440a12c0 0x7f00440a3770 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:18.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.276+0000 7f005dfff700 1 --2- 172.21.15.101:0/3236355986 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00580ffae0 0x7f005819ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:18.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.276+0000 7f005dfff700 1 --2- 172.21.15.101:0/3236355986 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0058105af0 0x7f0058195ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:18.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.276+0000 7f005dfff700 1 --2- 172.21.15.101:0/3236355986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0058100490 0x7f005819bf80 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:18.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.276+0000 7f005dfff700 1 -- 172.21.15.101:0/3236355986 >> 172.21.15.101:0/3236355986 conn(0x7f00580fb5e0 msgr2=0x7f00581035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:18.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.276+0000 7f005dfff700 1 -- 172.21.15.101:0/3236355986 shutdown_connections 2023-07-18T03:32:18.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:18.276+0000 7f005dfff700 1 -- 172.21.15.101:0/3236355986 wait complete. 2023-07-18T03:32:18.319 INFO:tasks.cephfs.filesystem:scrub status for tag:0f2f19e0-0429-465c-935e-223eb09490c3 - {'path': '/', 'tag': '0f2f19e0-0429-465c-935e-223eb09490c3', 'options': 'recursive,force'} 2023-07-18T03:32:19.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:32:19 smithi179 ceph-mon[129229]: pgmap v782: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:32:19.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:32:19 smithi179 ceph-mon[129229]: osdmap e130: 12 total, 12 up, 12 in 2023-07-18T03:32:19.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:32:19 smithi101 ceph-mon[126379]: pgmap v782: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:32:19.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:32:19 smithi101 ceph-mon[126379]: osdmap e130: 12 total, 12 up, 12 in 2023-07-18T03:32:19.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:32:19 smithi155 ceph-mon[128885]: pgmap v782: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:32:19.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:32:19 smithi155 ceph-mon[128885]: osdmap e130: 12 total, 12 up, 12 in 2023-07-18T03:32:21.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:32:21 smithi179 ceph-mon[129229]: pgmap v784: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 211 B/s wr, 6 op/s 2023-07-18T03:32:21.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:32:21 smithi101 ceph-mon[126379]: pgmap v784: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 211 B/s wr, 6 op/s 2023-07-18T03:32:21.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:32:21 smithi155 ceph-mon[128885]: pgmap v784: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 211 B/s wr, 6 op/s 2023-07-18T03:32:23.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:32:23 smithi179 ceph-mon[129229]: pgmap v785: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:32:23.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:32:23 smithi101 ceph-mon[126379]: pgmap v785: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:32:23.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:32:23 smithi155 ceph-mon[128885]: pgmap v785: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:32:25.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:32:25 smithi179 ceph-mon[129229]: pgmap v786: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:32:25.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:32:25 smithi101 ceph-mon[126379]: pgmap v786: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:32:25.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:32:25 smithi155 ceph-mon[128885]: pgmap v786: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:32:26.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:32:26 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:32:26.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:32:26 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:32:26.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:32:26 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:32:27.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:32:27 smithi179 ceph-mon[129229]: pgmap v787: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:32:27.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:32:27 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:32:27.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:32:27 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:32:27.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:32:27 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:32:27.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:32:27 smithi101 ceph-mon[126379]: pgmap v787: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:32:27.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:32:27 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:32:27.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:32:27 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:32:27.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:32:27 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:32:27.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:32:27 smithi155 ceph-mon[128885]: pgmap v787: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:32:27.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:32:27 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:32:27.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:32:27 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:32:27.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:32:27 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:32:29.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:32:29 smithi179 ceph-mon[129229]: pgmap v788: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:32:29.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:32:29 smithi101 ceph-mon[126379]: pgmap v788: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:32:29.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:32:29 smithi155 ceph-mon[128885]: pgmap v788: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:32:31.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:32:31 smithi179 ceph-mon[129229]: pgmap v789: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 95 B/s wr, 7 op/s 2023-07-18T03:32:31.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:32:31 smithi101 ceph-mon[126379]: pgmap v789: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 95 B/s wr, 7 op/s 2023-07-18T03:32:31.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:32:31 smithi155 ceph-mon[128885]: pgmap v789: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 95 B/s wr, 7 op/s 2023-07-18T03:32:33.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:32:33 smithi179 ceph-mon[129229]: pgmap v790: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:32:33.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:32:33 smithi101 ceph-mon[126379]: pgmap v790: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:32:33.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:32:33 smithi155 ceph-mon[128885]: pgmap v790: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:32:35.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:32:35 smithi179 ceph-mon[129229]: pgmap v791: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:32:35.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:32:35 smithi101 ceph-mon[126379]: pgmap v791: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:32:35.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:32:35 smithi155 ceph-mon[128885]: pgmap v791: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:32:37.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:32:37 smithi179 ceph-mon[129229]: pgmap v792: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:32:37.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:32:37 smithi101 ceph-mon[126379]: pgmap v792: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:32:37.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:32:37 smithi155 ceph-mon[128885]: pgmap v792: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:32:39.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:32:39 smithi179 ceph-mon[129229]: pgmap v793: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:32:39.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:32:39 smithi101 ceph-mon[126379]: pgmap v793: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:32:39.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:32:39 smithi155 ceph-mon[128885]: pgmap v793: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:32:41.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:32:41 smithi179 ceph-mon[129229]: pgmap v794: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-18T03:32:41.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:32:41 smithi101 ceph-mon[126379]: pgmap v794: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-18T03:32:41.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:32:41 smithi155 ceph-mon[128885]: pgmap v794: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-18T03:32:43.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:32:43 smithi179 ceph-mon[129229]: pgmap v795: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:32:43.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:32:43 smithi101 ceph-mon[126379]: pgmap v795: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:32:43.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:32:43 smithi155 ceph-mon[128885]: pgmap v795: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:32:45.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:32:45 smithi179 ceph-mon[129229]: pgmap v796: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:32:45.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:32:45 smithi101 ceph-mon[126379]: pgmap v796: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:32:45.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:32:45 smithi155 ceph-mon[128885]: pgmap v796: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:32:47.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:32:47 smithi179 ceph-mon[129229]: pgmap v797: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:32:47.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:32:47 smithi101 ceph-mon[126379]: pgmap v797: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:32:47.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:32:47 smithi155 ceph-mon[128885]: pgmap v797: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:32:48.320 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:32:48.549 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.553+0000 7ff045dff700 1 -- 172.21.15.101:0/2570497442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff040073180 msgr2=0x7ff04010be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:48.550 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.553+0000 7ff045dff700 1 --2- 172.21.15.101:0/2570497442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff040073180 0x7ff04010be20 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff028009990 tx=0x7ff028009ca0 comp rx=0 tx=0).stop 2023-07-18T03:32:48.550 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.553+0000 7ff045dff700 1 -- 172.21.15.101:0/2570497442 shutdown_connections 2023-07-18T03:32:48.550 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.553+0000 7ff045dff700 1 --2- 172.21.15.101:0/2570497442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff040072870 0x7ff040072c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:48.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.553+0000 7ff045dff700 1 --2- 172.21.15.101:0/2570497442 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff04010c360 0x7ff04010e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:48.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.553+0000 7ff045dff700 1 --2- 172.21.15.101:0/2570497442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff040073180 0x7ff04010be20 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:48.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.553+0000 7ff045dff700 1 -- 172.21.15.101:0/2570497442 >> 172.21.15.101:0/2570497442 conn(0x7ff0400fb5e0 msgr2=0x7ff0400fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:48.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.553+0000 7ff045dff700 1 -- 172.21.15.101:0/2570497442 shutdown_connections 2023-07-18T03:32:48.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.553+0000 7ff045dff700 1 -- 172.21.15.101:0/2570497442 wait complete. 2023-07-18T03:32:48.553 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.554+0000 7ff045dff700 1 Processor -- start 2023-07-18T03:32:48.553 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.554+0000 7ff045dff700 1 -- start start 2023-07-18T03:32:48.553 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.554+0000 7ff045dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff040072870 0x7ff040197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:48.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.554+0000 7ff045dff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff040073180 0x7ff040198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:48.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.554+0000 7ff045dff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff04010c360 0x7ff04019c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:48.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.554+0000 7ff045dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04019ce10 con 0x7ff040072870 2023-07-18T03:32:48.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.554+0000 7ff045dff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04019cf80 con 0x7ff040073180 2023-07-18T03:32:48.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.554+0000 7ff045dff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04019d140 con 0x7ff04010c360 2023-07-18T03:32:48.555 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.555+0000 7ff03f7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff040072870 0x7ff040197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:48.555 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.555+0000 7ff037fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff040073180 0x7ff040198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:48.555 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.555+0000 7ff03ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff04010c360 0x7ff04019c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:48.555 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.555+0000 7ff037fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff040073180 0x7ff040198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:33886/0 (socket says 172.21.15.101:33886) 2023-07-18T03:32:48.555 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.555+0000 7ff03f7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff040072870 0x7ff040197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:51278/0 (socket says 172.21.15.101:51278) 2023-07-18T03:32:48.556 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.555+0000 7ff037fff700 1 -- 172.21.15.101:0/3333453383 learned_addr learned my addr 172.21.15.101:0/3333453383 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:32:48.556 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.555+0000 7ff037fff700 1 -- 172.21.15.101:0/3333453383 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff04010c360 msgr2=0x7ff04019c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:48.556 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.555+0000 7ff037fff700 1 --2- 172.21.15.101:0/3333453383 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff04010c360 0x7ff04019c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:48.556 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.555+0000 7ff037fff700 1 -- 172.21.15.101:0/3333453383 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff040072870 msgr2=0x7ff040197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:48.556 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.555+0000 7ff037fff700 1 --2- 172.21.15.101:0/3333453383 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff040072870 0x7ff040197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:48.557 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.555+0000 7ff037fff700 1 -- 172.21.15.101:0/3333453383 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0280095f0 con 0x7ff040073180 2023-07-18T03:32:48.557 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.555+0000 7ff03f7fe700 1 --2- 172.21.15.101:0/3333453383 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff040072870 0x7ff040197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:32:48.557 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.555+0000 7ff03ffff700 1 --2- 172.21.15.101:0/3333453383 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff04010c360 0x7ff04019c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:32:48.557 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.555+0000 7ff037fff700 1 --2- 172.21.15.101:0/3333453383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff040073180 0x7ff040198370 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ff028005910 tx=0x7ff028005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:48.557 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.555+0000 7ff03d7fa700 1 -- 172.21.15.101:0/3333453383 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02801d070 con 0x7ff040073180 2023-07-18T03:32:48.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.556+0000 7ff045dff700 1 -- 172.21.15.101:0/3333453383 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff04019d3c0 con 0x7ff040073180 2023-07-18T03:32:48.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.556+0000 7ff03d7fa700 1 -- 172.21.15.101:0/3333453383 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff028022470 con 0x7ff040073180 2023-07-18T03:32:48.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.556+0000 7ff03d7fa700 1 -- 172.21.15.101:0/3333453383 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff028017650 con 0x7ff040073180 2023-07-18T03:32:48.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.556+0000 7ff045dff700 1 -- 172.21.15.101:0/3333453383 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0401a4c60 con 0x7ff040073180 2023-07-18T03:32:48.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.561+0000 7ff03d7fa700 1 -- 172.21.15.101:0/3333453383 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7ff02800fad0 con 0x7ff040073180 2023-07-18T03:32:48.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.561+0000 7ff05980cb80 1 Processor -- start 2023-07-18T03:32:48.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.561+0000 7ff05980cb80 1 -- start start 2023-07-18T03:32:48.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.562+0000 7ff05980cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5615ea8862d0 0x5615ea8866a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:48.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.562+0000 7ff05980cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615ea886be0 0x5615ea891590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:48.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.562+0000 7ff05980cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ea891ad0 0x5615ea893eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:48.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.562+0000 7ff05980cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ea78ccb0 con 0x5615ea8862d0 2023-07-18T03:32:48.562 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.562+0000 7ff05980cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ea7dfa40 con 0x5615ea891ad0 2023-07-18T03:32:48.562 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.562+0000 7ff05980cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ea7d8ed0 con 0x5615ea886be0 2023-07-18T03:32:48.562 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.562+0000 7ff03f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615ea886be0 0x5615ea891590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:48.562 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.562+0000 7ff037fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ea891ad0 0x5615ea893eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:48.562 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.562+0000 7ff03f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615ea886be0 0x5615ea891590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:35370/0 (socket says 172.21.15.101:35370) 2023-07-18T03:32:48.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.562+0000 7ff03f7fe700 1 -- 172.21.15.101:0/4028595483 learned_addr learned my addr 172.21.15.101:0/4028595483 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:32:48.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.562+0000 7ff03ffff700 1 --2- 172.21.15.101:0/4028595483 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5615ea8862d0 0x5615ea8866a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:48.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.562+0000 7ff03f7fe700 1 -- 172.21.15.101:0/4028595483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ea891ad0 msgr2=0x5615ea893eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:48.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.562+0000 7ff03f7fe700 1 --2- 172.21.15.101:0/4028595483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ea891ad0 0x5615ea893eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:48.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.562+0000 7ff03f7fe700 1 -- 172.21.15.101:0/4028595483 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5615ea8862d0 msgr2=0x5615ea8866a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:48.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.562+0000 7ff03f7fe700 1 --2- 172.21.15.101:0/4028595483 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5615ea8862d0 0x5615ea8866a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:48.564 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.562+0000 7ff03f7fe700 1 -- 172.21.15.101:0/4028595483 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615ea606de0 con 0x5615ea886be0 2023-07-18T03:32:48.564 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.562+0000 7ff03d7fa700 1 --2- 172.21.15.101:0/3333453383 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff02c0a1340 0x7ff02c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:48.564 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.563+0000 7ff03d7fa700 1 -- 172.21.15.101:0/3333453383 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 9461+0+0 (secure 0 0 0) 0x7ff0280ce520 con 0x7ff040073180 2023-07-18T03:32:48.564 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.563+0000 7ff03f7fe700 1 --2- 172.21.15.101:0/4028595483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615ea886be0 0x5615ea891590 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff0300115f0 tx=0x7ff030011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7798853f1e478421 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:48.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.563+0000 7ff037fff700 1 --2- 172.21.15.101:0/3333453383 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff02c0a1340 0x7ff02c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:48.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.563+0000 7ff035ffb700 1 -- 172.21.15.101:0/4028595483 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff030019630 con 0x5615ea886be0 2023-07-18T03:32:48.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.563+0000 7ff035ffb700 1 -- 172.21.15.101:0/4028595483 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff030019c50 con 0x5615ea886be0 2023-07-18T03:32:48.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.563+0000 7ff035ffb700 1 -- 172.21.15.101:0/4028595483 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03000ec90 con 0x5615ea886be0 2023-07-18T03:32:48.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.564+0000 7ff05980cb80 1 -- 172.21.15.101:0/4028595483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615ea886be0 msgr2=0x5615ea891590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:48.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.564+0000 7ff037fff700 1 --2- 172.21.15.101:0/3333453383 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff02c0a1340 0x7ff02c0a37f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff02809b200 tx=0x7ff0280d1f90 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:48.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.564+0000 7ff05980cb80 1 --2- 172.21.15.101:0/4028595483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615ea886be0 0x5615ea891590 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff0300115f0 tx=0x7ff030011900 comp rx=0 tx=0).stop 2023-07-18T03:32:48.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.564+0000 7ff05980cb80 1 -- 172.21.15.101:0/4028595483 shutdown_connections 2023-07-18T03:32:48.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.564+0000 7ff05980cb80 1 --2- 172.21.15.101:0/4028595483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ea891ad0 0x5615ea893eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:48.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.564+0000 7ff05980cb80 1 --2- 172.21.15.101:0/4028595483 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5615ea8862d0 0x5615ea8866a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:48.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.564+0000 7ff05980cb80 1 --2- 172.21.15.101:0/4028595483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615ea886be0 0x5615ea891590 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:48.567 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.564+0000 7ff05980cb80 1 -- 172.21.15.101:0/4028595483 >> 172.21.15.101:0/4028595483 conn(0x5615ea87cda0 msgr2=0x5615ea8826a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:48.567 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.564+0000 7ff05980cb80 1 -- 172.21.15.101:0/4028595483 shutdown_connections 2023-07-18T03:32:48.567 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.564+0000 7ff05980cb80 1 -- 172.21.15.101:0/4028595483 wait complete. 2023-07-18T03:32:48.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.564+0000 7ff05980cb80 1 -- start start 2023-07-18T03:32:48.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.565+0000 7ff05980cb80 1 Processor -- start 2023-07-18T03:32:48.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.565+0000 7ff05980cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615ea8862d0 0x5615ea891490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:48.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.565+0000 7ff05980cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ea886be0 0x5615ea88e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:48.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.565+0000 7ff05980cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5615ea891ad0 0x5615ea88ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:48.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.565+0000 7ff05980cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ea778ac0 con 0x5615ea891ad0 2023-07-18T03:32:48.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.565+0000 7ff05980cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ea7787c0 con 0x5615ea886be0 2023-07-18T03:32:48.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.565+0000 7ff05980cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ea65c920 con 0x5615ea8862d0 2023-07-18T03:32:48.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.565+0000 7ff05980cb80 10 client.0 _ll_register_callbacks cb 0x5615ea776760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:32:48.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.565+0000 7ff03f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ea886be0 0x5615ea88e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:48.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.565+0000 7ff03ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615ea8862d0 0x5615ea891490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:48.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.565+0000 7ff03f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ea886be0 0x5615ea88e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:33898/0 (socket says 172.21.15.101:33898) 2023-07-18T03:32:48.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.565+0000 7ff03f7fe700 1 -- 172.21.15.101:0/1270431299 learned_addr learned my addr 172.21.15.101:0/1270431299 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:32:48.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.566+0000 7ff03ffff700 1 --2- 172.21.15.101:0/1270431299 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5615ea891ad0 0x5615ea88ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:48.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.566+0000 7ff03f7fe700 1 -- 172.21.15.101:0/1270431299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615ea8862d0 msgr2=0x5615ea891490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:48.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.566+0000 7ff03f7fe700 1 --2- 172.21.15.101:0/1270431299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615ea8862d0 0x5615ea891490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:48.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.566+0000 7ff03f7fe700 1 -- 172.21.15.101:0/1270431299 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5615ea891ad0 msgr2=0x5615ea88ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:48.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.566+0000 7ff03f7fe700 1 --2- 172.21.15.101:0/1270431299 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5615ea891ad0 0x5615ea88ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:48.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.566+0000 7ff03f7fe700 1 -- 172.21.15.101:0/1270431299 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff030009dd0 con 0x5615ea886be0 2023-07-18T03:32:48.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.566+0000 7ff03f7fe700 1 --2- 172.21.15.101:0/1270431299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ea886be0 0x5615ea88e710 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff030000c00 tx=0x7ff03000ef30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f45aa6dc059a9c60 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:48.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.566+0000 7ff01effd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:32:48.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.566+0000 7ff01effd700 1 -- 172.21.15.101:0/1270431299 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff030019630 con 0x5615ea886be0 2023-07-18T03:32:48.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.566+0000 7ff05980cb80 1 -- 172.21.15.101:0/1270431299 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5615ea7324f0 con 0x5615ea886be0 2023-07-18T03:32:48.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.566+0000 7ff01effd700 1 -- 172.21.15.101:0/1270431299 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 9461+0+0 (secure 0 0 0) 0x7ff030005ce0 con 0x5615ea886be0 2023-07-18T03:32:48.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.567+0000 7ff01effd700 1 -- 172.21.15.101:0/1270431299 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff030019790 con 0x5615ea886be0 2023-07-18T03:32:48.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.567+0000 7ff05980cb80 10 client.29210 fetch_fsmap learned FSMap version 49 2023-07-18T03:32:48.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.567+0000 7ff05980cb80 1 -- 172.21.15.101:0/1270431299 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5615ea5efc60 con 0x5615ea886be0 2023-07-18T03:32:48.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.567+0000 7ff01effd700 1 -- 172.21.15.101:0/1270431299 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5615ea7324f0 con 0x5615ea886be0 2023-07-18T03:32:48.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.567+0000 7ff01effd700 1 -- 172.21.15.101:0/1270431299 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 49) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff030004930 con 0x5615ea886be0 2023-07-18T03:32:48.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.567+0000 7ff05980cb80 10 client.29210 fetch_fsmap finished waiting for FSMap version 49 2023-07-18T03:32:48.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.567+0000 7ff05980cb80 10 client.29210 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:32:48.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.567+0000 7ff05980cb80 20 client.29210 populate_metadata read hostname 'smithi101' 2023-07-18T03:32:48.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.567+0000 7ff05980cb80 1 --2- 172.21.15.101:0/1270431299 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5615ea8a1310 0x5615ea8a3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:48.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.567+0000 7ff05980cb80 4 client.29210 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:32:48.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.567+0000 7ff05980cb80 1 -- 172.21.15.101:0/1270431299 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5615ea7d1eb0 con 0x5615ea8a1310 2023-07-18T03:32:48.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.567+0000 7ff03ffff700 1 --2- 172.21.15.101:0/1270431299 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5615ea8a1310 0x5615ea8a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:48.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.568+0000 7ff01effd700 10 client.29210 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:32:48.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.568+0000 7ff03ffff700 1 --2- 172.21.15.101:0/1270431299 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5615ea8a1310 0x5615ea8a3720 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92b2514ca72807a1 server_cookie=5b16105d9ec23934 in_seq=0 out_seq=0 2023-07-18T03:32:48.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.569+0000 7ff01effd700 1 -- 172.21.15.101:0/1270431299 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5615ea7d1eb0 con 0x5615ea8a1310 2023-07-18T03:32:48.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.569+0000 7ff01effd700 10 client.29210 handle_command_reply: tid=0 2023-07-18T03:32:48.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.569+0000 7ff05980cb80 1 client.29210 shutdown 2023-07-18T03:32:48.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.569+0000 7ff05980cb80 1 -- 172.21.15.101:0/1270431299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ea886be0 msgr2=0x5615ea88e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:48.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.569+0000 7ff05980cb80 1 --2- 172.21.15.101:0/1270431299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ea886be0 0x5615ea88e710 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff030000c00 tx=0x7ff03000ef30 comp rx=0 tx=0).stop 2023-07-18T03:32:48.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.569+0000 7ff05980cb80 1 -- 172.21.15.101:0/1270431299 shutdown_connections 2023-07-18T03:32:48.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.569+0000 7ff05980cb80 1 --2- 172.21.15.101:0/1270431299 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5615ea8a1310 0x5615ea8a3720 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:48.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.569+0000 7ff05980cb80 1 --2- 172.21.15.101:0/1270431299 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5615ea891ad0 0x5615ea88ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:48.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.569+0000 7ff05980cb80 1 --2- 172.21.15.101:0/1270431299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615ea8862d0 0x5615ea891490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:48.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.569+0000 7ff01effd700 0 client.29210 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:32:48.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.569+0000 7ff05980cb80 1 --2- 172.21.15.101:0/1270431299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ea886be0 0x5615ea88e710 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:48.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.569+0000 7ff05980cb80 1 -- 172.21.15.101:0/1270431299 >> 172.21.15.101:0/1270431299 conn(0x5615ea87cda0 msgr2=0x5615ea882230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:48.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.569+0000 7ff05980cb80 1 -- 172.21.15.101:0/1270431299 shutdown_connections 2023-07-18T03:32:48.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.569+0000 7ff05980cb80 1 -- 172.21.15.101:0/1270431299 wait complete. 2023-07-18T03:32:48.581 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.569+0000 7ff05980cb80 20 client.29210 trim_cache size 0 max 16384 2023-07-18T03:32:48.582 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.586+0000 7ff05980cb80 1 Processor -- start 2023-07-18T03:32:48.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.586+0000 7ff05980cb80 1 -- start start 2023-07-18T03:32:48.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.586+0000 7ff05980cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615ea8862d0 0x5615ea89d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:48.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.586+0000 7ff05980cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ea886be0 0x5615ea8910c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:48.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.587+0000 7ff05980cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5615ea891ad0 0x5615ea88df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:48.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.587+0000 7ff05980cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ea883d20 con 0x5615ea891ad0 2023-07-18T03:32:48.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.587+0000 7ff05980cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ea883330 con 0x5615ea886be0 2023-07-18T03:32:48.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.587+0000 7ff05980cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ea887540 con 0x5615ea8862d0 2023-07-18T03:32:48.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.587+0000 7ff03ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615ea8862d0 0x5615ea89d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:48.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.587+0000 7ff03f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ea886be0 0x5615ea8910c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:48.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.587+0000 7ff03f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ea886be0 0x5615ea8910c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:33910/0 (socket says 172.21.15.101:33910) 2023-07-18T03:32:48.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.587+0000 7ff03f7fe700 1 -- 172.21.15.101:0/1542564003 learned_addr learned my addr 172.21.15.101:0/1542564003 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:32:48.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.587+0000 7ff03ffff700 1 --2- 172.21.15.101:0/1542564003 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5615ea891ad0 0x5615ea88df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:48.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.587+0000 7ff03f7fe700 1 -- 172.21.15.101:0/1542564003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615ea8862d0 msgr2=0x5615ea89d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:48.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.587+0000 7ff03f7fe700 1 --2- 172.21.15.101:0/1542564003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615ea8862d0 0x5615ea89d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:48.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.587+0000 7ff03f7fe700 1 -- 172.21.15.101:0/1542564003 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5615ea891ad0 msgr2=0x5615ea88df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:48.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.587+0000 7ff03f7fe700 1 --2- 172.21.15.101:0/1542564003 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5615ea891ad0 0x5615ea88df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:48.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.587+0000 7ff03f7fe700 1 -- 172.21.15.101:0/1542564003 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615ea62d250 con 0x5615ea886be0 2023-07-18T03:32:48.587 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.587+0000 7ff03ffff700 1 --2- 172.21.15.101:0/1542564003 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5615ea891ad0 0x5615ea88df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:32:48.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.587+0000 7ff03f7fe700 1 --2- 172.21.15.101:0/1542564003 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ea886be0 0x5615ea8910c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff03000e8a0 tx=0x7ff03000efa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1069269f0d28d972 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:48.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.588+0000 7ff0367fc700 1 -- 172.21.15.101:0/1542564003 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff030019dc0 con 0x5615ea886be0 2023-07-18T03:32:48.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.588+0000 7ff0367fc700 1 -- 172.21.15.101:0/1542564003 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0300198f0 con 0x5615ea886be0 2023-07-18T03:32:48.588 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.588+0000 7ff0367fc700 1 -- 172.21.15.101:0/1542564003 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0300056d0 con 0x5615ea886be0 2023-07-18T03:32:48.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.588+0000 7ff05980cb80 1 -- 172.21.15.101:0/1542564003 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ea886be0 msgr2=0x5615ea8910c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:48.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.588+0000 7ff05980cb80 1 --2- 172.21.15.101:0/1542564003 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ea886be0 0x5615ea8910c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff03000e8a0 tx=0x7ff03000efa0 comp rx=0 tx=0).stop 2023-07-18T03:32:48.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.588+0000 7ff05980cb80 1 -- 172.21.15.101:0/1542564003 shutdown_connections 2023-07-18T03:32:48.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.588+0000 7ff05980cb80 1 --2- 172.21.15.101:0/1542564003 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5615ea891ad0 0x5615ea88df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:48.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.588+0000 7ff05980cb80 1 --2- 172.21.15.101:0/1542564003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615ea8862d0 0x5615ea89d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:48.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.588+0000 7ff05980cb80 1 --2- 172.21.15.101:0/1542564003 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ea886be0 0x5615ea8910c0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:48.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.588+0000 7ff05980cb80 1 -- 172.21.15.101:0/1542564003 >> 172.21.15.101:0/1542564003 conn(0x5615ea87cda0 msgr2=0x5615ea880e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:48.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.588+0000 7ff05980cb80 1 -- 172.21.15.101:0/1542564003 shutdown_connections 2023-07-18T03:32:48.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.588+0000 7ff05980cb80 1 -- 172.21.15.101:0/1542564003 wait complete. 2023-07-18T03:32:48.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.589+0000 7ff05980cb80 1 -- start start 2023-07-18T03:32:48.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.589+0000 7ff05980cb80 1 Processor -- start 2023-07-18T03:32:48.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.589+0000 7ff05980cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615ea8862d0 0x5615ea897eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:48.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.589+0000 7ff05980cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5615ea886be0 0x5615ea8983f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:48.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.589+0000 7ff05980cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ea891ad0 0x5615ea896500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:48.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.589+0000 7ff05980cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ea896ac0 con 0x5615ea886be0 2023-07-18T03:32:48.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.589+0000 7ff05980cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ea896c30 con 0x5615ea891ad0 2023-07-18T03:32:48.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.589+0000 7ff05980cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ea896df0 con 0x5615ea8862d0 2023-07-18T03:32:48.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.589+0000 7ff05980cb80 10 client.0 _ll_register_callbacks cb 0x7ff038009440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:32:48.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.590+0000 7ff03f7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5615ea886be0 0x5615ea8983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:48.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.590+0000 7ff03f7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5615ea886be0 0x5615ea8983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:51310/0 (socket says 172.21.15.101:51310) 2023-07-18T03:32:48.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.590+0000 7ff03f7fe700 1 -- 172.21.15.101:0/4256451239 learned_addr learned my addr 172.21.15.101:0/4256451239 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:32:48.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.590+0000 7ff03ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615ea8862d0 0x5615ea897eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:48.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.590+0000 7ff03ffff700 1 --2- 172.21.15.101:0/4256451239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ea891ad0 0x5615ea896500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:48.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.590+0000 7ff03f7fe700 1 -- 172.21.15.101:0/4256451239 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615ea8862d0 msgr2=0x5615ea897eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:48.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.590+0000 7ff03f7fe700 1 --2- 172.21.15.101:0/4256451239 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615ea8862d0 0x5615ea897eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:48.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.590+0000 7ff03f7fe700 1 -- 172.21.15.101:0/4256451239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ea891ad0 msgr2=0x5615ea896500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:48.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.590+0000 7ff03f7fe700 1 --2- 172.21.15.101:0/4256451239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ea891ad0 0x5615ea896500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:48.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.590+0000 7ff03f7fe700 1 -- 172.21.15.101:0/4256451239 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5615ea5efc60 con 0x5615ea886be0 2023-07-18T03:32:48.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.590+0000 7ff03ffff700 1 --2- 172.21.15.101:0/4256451239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ea891ad0 0x5615ea896500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:32:48.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.590+0000 7ff03f7fe700 1 --2- 172.21.15.101:0/4256451239 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5615ea886be0 0x5615ea8983f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7ff03000ef70 tx=0x7ff030005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a5ee107ec94e37c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:48.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.590+0000 7ff01effd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:32:48.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.590+0000 7ff01effd700 1 -- 172.21.15.101:0/4256451239 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff030019dc0 con 0x5615ea886be0 2023-07-18T03:32:48.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.590+0000 7ff05980cb80 1 -- 172.21.15.101:0/4256451239 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff030019c50 con 0x5615ea886be0 2023-07-18T03:32:48.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.590+0000 7ff01effd700 1 -- 172.21.15.101:0/4256451239 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 9461+0+0 (secure 0 0 0) 0x7ff030000e40 con 0x5615ea886be0 2023-07-18T03:32:48.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.591+0000 7ff01effd700 1 -- 172.21.15.101:0/4256451239 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff030019a60 con 0x5615ea886be0 2023-07-18T03:32:48.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.591+0000 7ff05980cb80 10 client.19110 fetch_fsmap learned FSMap version 49 2023-07-18T03:32:48.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.591+0000 7ff05980cb80 1 -- 172.21.15.101:0/4256451239 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5615ea6052b0 con 0x5615ea886be0 2023-07-18T03:32:48.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.591+0000 7ff01effd700 1 -- 172.21.15.101:0/4256451239 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff030019c50 con 0x5615ea886be0 2023-07-18T03:32:48.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.591+0000 7ff01effd700 1 -- 172.21.15.101:0/4256451239 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 49) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff030003950 con 0x5615ea886be0 2023-07-18T03:32:48.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.591+0000 7ff05980cb80 10 client.19110 fetch_fsmap finished waiting for FSMap version 49 2023-07-18T03:32:48.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.591+0000 7ff05980cb80 10 client.19110 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:32:48.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.591+0000 7ff05980cb80 20 client.19110 populate_metadata read hostname 'smithi101' 2023-07-18T03:32:48.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.591+0000 7ff05980cb80 1 --2- 172.21.15.101:0/4256451239 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5615ea8f5f60 0x5615ea8f8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:48.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.591+0000 7ff05980cb80 4 client.19110 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:32:48.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.591+0000 7ff05980cb80 1 -- 172.21.15.101:0/4256451239 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5615ea794050 con 0x5615ea8f5f60 2023-07-18T03:32:48.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.591+0000 7ff03ffff700 1 --2- 172.21.15.101:0/4256451239 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5615ea8f5f60 0x5615ea8f8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:48.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.592+0000 7ff03ffff700 1 --2- 172.21.15.101:0/4256451239 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5615ea8f5f60 0x5615ea8f8340 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=542f7b40a1ccff18 server_cookie=eddc0e3fdceaa49f in_seq=0 out_seq=0 2023-07-18T03:32:48.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.592+0000 7ff01effd700 10 client.19110 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:32:48.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.592+0000 7ff01effd700 1 -- 172.21.15.101:0/4256451239 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5615ea794050 con 0x5615ea8f5f60 2023-07-18T03:32:48.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.592+0000 7ff01effd700 10 client.19110 handle_command_reply: tid=0 2023-07-18T03:32:48.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.592+0000 7ff05980cb80 1 client.19110 shutdown 2023-07-18T03:32:48.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.592+0000 7ff05980cb80 1 -- 172.21.15.101:0/4256451239 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5615ea886be0 msgr2=0x5615ea8983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:48.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.592+0000 7ff05980cb80 1 --2- 172.21.15.101:0/4256451239 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5615ea886be0 0x5615ea8983f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7ff03000ef70 tx=0x7ff030005a00 comp rx=0 tx=0).stop 2023-07-18T03:32:48.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.592+0000 7ff05980cb80 1 -- 172.21.15.101:0/4256451239 shutdown_connections 2023-07-18T03:32:48.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.592+0000 7ff05980cb80 1 --2- 172.21.15.101:0/4256451239 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5615ea8f5f60 0x5615ea8f8340 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:48.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.592+0000 7ff05980cb80 1 --2- 172.21.15.101:0/4256451239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615ea891ad0 0x5615ea896500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:48.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.592+0000 7ff05980cb80 1 --2- 172.21.15.101:0/4256451239 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5615ea8862d0 0x5615ea897eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:48.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.592+0000 7ff01effd700 0 client.19110 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:32:48.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.592+0000 7ff05980cb80 1 --2- 172.21.15.101:0/4256451239 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5615ea886be0 0x5615ea8983f0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:48.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.592+0000 7ff05980cb80 1 -- 172.21.15.101:0/4256451239 >> 172.21.15.101:0/4256451239 conn(0x5615ea87cda0 msgr2=0x5615ea8910a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:48.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.592+0000 7ff05980cb80 1 -- 172.21.15.101:0/4256451239 shutdown_connections 2023-07-18T03:32:48.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.592+0000 7ff05980cb80 1 -- 172.21.15.101:0/4256451239 wait complete. 2023-07-18T03:32:48.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.593+0000 7ff05980cb80 20 client.19110 trim_cache size 0 max 16384 2023-07-18T03:32:48.604 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:32:48.604 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:32:48.604 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:32:48.605 INFO:teuthology.orchestra.run.smithi101.stdout: "0f2f19e0-0429-465c-935e-223eb09490c3": { 2023-07-18T03:32:48.605 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:32:48.605 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "0f2f19e0-0429-465c-935e-223eb09490c3", 2023-07-18T03:32:48.605 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:32:48.605 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:32:48.606 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:32:48.606 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:32:48.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.593+0000 7ff045dff700 1 -- 172.21.15.101:0/3333453383 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff02c0a1340 msgr2=0x7ff02c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:48.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.593+0000 7ff045dff700 1 --2- 172.21.15.101:0/3333453383 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff02c0a1340 0x7ff02c0a37f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff02809b200 tx=0x7ff0280d1f90 comp rx=0 tx=0).stop 2023-07-18T03:32:48.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.593+0000 7ff045dff700 1 -- 172.21.15.101:0/3333453383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff040073180 msgr2=0x7ff040198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:48.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.593+0000 7ff045dff700 1 --2- 172.21.15.101:0/3333453383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff040073180 0x7ff040198370 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ff028005910 tx=0x7ff028005740 comp rx=0 tx=0).stop 2023-07-18T03:32:48.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.593+0000 7ff045dff700 1 -- 172.21.15.101:0/3333453383 shutdown_connections 2023-07-18T03:32:48.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.593+0000 7ff045dff700 1 --2- 172.21.15.101:0/3333453383 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff04010c360 0x7ff04019c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:48.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.593+0000 7ff045dff700 1 --2- 172.21.15.101:0/3333453383 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff040072870 0x7ff040197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:48.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.593+0000 7ff045dff700 1 --2- 172.21.15.101:0/3333453383 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff02c0a1340 0x7ff02c0a37f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:48.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.593+0000 7ff045dff700 1 --2- 172.21.15.101:0/3333453383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff040073180 0x7ff040198370 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:48.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.593+0000 7ff045dff700 1 -- 172.21.15.101:0/3333453383 >> 172.21.15.101:0/3333453383 conn(0x7ff0400fb5e0 msgr2=0x7ff040103dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:48.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.593+0000 7ff045dff700 1 -- 172.21.15.101:0/3333453383 shutdown_connections 2023-07-18T03:32:48.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:32:48.593+0000 7ff045dff700 1 -- 172.21.15.101:0/3333453383 wait complete. 2023-07-18T03:32:48.610 INFO:tasks.cephfs.filesystem:scrub status for tag:0f2f19e0-0429-465c-935e-223eb09490c3 - {'path': '/', 'tag': '0f2f19e0-0429-465c-935e-223eb09490c3', 'options': 'recursive,force'} 2023-07-18T03:32:49.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:32:49 smithi179 ceph-mon[129229]: pgmap v798: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:32:49.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:32:49 smithi101 ceph-mon[126379]: pgmap v798: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:32:49.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:32:49 smithi155 ceph-mon[128885]: pgmap v798: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:32:50.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:32:50 smithi179 ceph-mon[129229]: pgmap v799: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:32:50.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:32:50 smithi101 ceph-mon[126379]: pgmap v799: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:32:50.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:32:50 smithi155 ceph-mon[128885]: pgmap v799: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:32:53.454 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:32:53 smithi179 ceph-mon[129229]: pgmap v800: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:32:53.491 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:32:53 smithi101 ceph-mon[126379]: pgmap v800: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:32:53.549 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:32:53 smithi155 ceph-mon[128885]: pgmap v800: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-18T03:32:55.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:32:55 smithi179 ceph-mon[129229]: pgmap v801: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:32:55.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:32:55 smithi101 ceph-mon[126379]: pgmap v801: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:32:55.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:32:55 smithi155 ceph-mon[128885]: pgmap v801: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:32:57.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:32:57 smithi179 ceph-mon[129229]: pgmap v802: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:32:57.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:32:57 smithi101 ceph-mon[126379]: pgmap v802: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:32:57.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:32:57 smithi155 ceph-mon[128885]: pgmap v802: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:32:59.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:32:59 smithi179 ceph-mon[129229]: pgmap v803: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:32:59.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:32:59 smithi101 ceph-mon[126379]: pgmap v803: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:32:59.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:32:59 smithi155 ceph-mon[128885]: pgmap v803: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:33:01.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:33:01 smithi179 ceph-mon[129229]: pgmap v804: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:33:01.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:33:01 smithi101 ceph-mon[126379]: pgmap v804: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:33:01.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:33:01 smithi155 ceph-mon[128885]: pgmap v804: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:33:03.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:33:03 smithi179 ceph-mon[129229]: pgmap v805: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-07-18T03:33:03.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:33:03 smithi101 ceph-mon[126379]: pgmap v805: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-07-18T03:33:03.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:33:03 smithi155 ceph-mon[128885]: pgmap v805: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-07-18T03:33:05.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:33:05 smithi179 ceph-mon[129229]: pgmap v806: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-07-18T03:33:05.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:33:05 smithi101 ceph-mon[126379]: pgmap v806: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-07-18T03:33:05.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:33:05 smithi155 ceph-mon[128885]: pgmap v806: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-07-18T03:33:07.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:33:07 smithi179 ceph-mon[129229]: pgmap v807: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-07-18T03:33:07.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:33:07 smithi101 ceph-mon[126379]: pgmap v807: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-07-18T03:33:07.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:33:07 smithi155 ceph-mon[128885]: pgmap v807: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-07-18T03:33:09.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:33:09 smithi179 ceph-mon[129229]: pgmap v808: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-07-18T03:33:09.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:33:09 smithi101 ceph-mon[126379]: pgmap v808: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-07-18T03:33:09.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:33:09 smithi155 ceph-mon[128885]: pgmap v808: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-07-18T03:33:11.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:33:11 smithi179 ceph-mon[129229]: pgmap v809: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-07-18T03:33:11.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:33:11 smithi179 ceph-mon[129229]: osdmap e131: 12 total, 12 up, 12 in 2023-07-18T03:33:11.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:33:11 smithi101 ceph-mon[126379]: pgmap v809: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-07-18T03:33:11.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:33:11 smithi101 ceph-mon[126379]: osdmap e131: 12 total, 12 up, 12 in 2023-07-18T03:33:11.800 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:33:11 smithi155 ceph-mon[128885]: pgmap v809: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-07-18T03:33:11.800 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:33:11 smithi155 ceph-mon[128885]: osdmap e131: 12 total, 12 up, 12 in 2023-07-18T03:33:13.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:33:13 smithi179 ceph-mon[129229]: pgmap v811: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 KiB/s wr, 21 op/s 2023-07-18T03:33:13.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:33:13 smithi101 ceph-mon[126379]: pgmap v811: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 KiB/s wr, 21 op/s 2023-07-18T03:33:13.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:33:13 smithi155 ceph-mon[128885]: pgmap v811: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 KiB/s wr, 21 op/s 2023-07-18T03:33:15.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:33:15 smithi179 ceph-mon[129229]: pgmap v812: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 KiB/s wr, 21 op/s 2023-07-18T03:33:15.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:33:15 smithi101 ceph-mon[126379]: pgmap v812: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 KiB/s wr, 21 op/s 2023-07-18T03:33:15.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:33:15 smithi155 ceph-mon[128885]: pgmap v812: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 KiB/s wr, 21 op/s 2023-07-18T03:33:17.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:33:17 smithi179 ceph-mon[129229]: pgmap v813: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:33:17.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:33:17 smithi101 ceph-mon[126379]: pgmap v813: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:33:17.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:33:17 smithi155 ceph-mon[128885]: pgmap v813: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:33:18.611 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:33:18.823 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.825+0000 7ffa12b9d700 1 -- 172.21.15.101:0/720327388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa0c1059e0 msgr2=0x7ffa0c105db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:18.824 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.825+0000 7ffa12b9d700 1 --2- 172.21.15.101:0/720327388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa0c1059e0 0x7ffa0c105db0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7ffa08009990 tx=0x7ffa08009ca0 comp rx=0 tx=0).stop 2023-07-18T03:33:18.824 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.825+0000 7ffa12b9d700 1 -- 172.21.15.101:0/720327388 shutdown_connections 2023-07-18T03:33:18.824 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.825+0000 7ffa12b9d700 1 --2- 172.21.15.101:0/720327388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa0c1059e0 0x7ffa0c105db0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:18.824 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.825+0000 7ffa12b9d700 1 --2- 172.21.15.101:0/720327388 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa0c100380 0x7ffa0c10c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:18.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.825+0000 7ffa12b9d700 1 --2- 172.21.15.101:0/720327388 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffa0c0ff9d0 0x7ffa0c0ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:18.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.825+0000 7ffa12b9d700 1 -- 172.21.15.101:0/720327388 >> 172.21.15.101:0/720327388 conn(0x7ffa0c0fb4b0 msgr2=0x7ffa0c0fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:18.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.825+0000 7ffa12b9d700 1 -- 172.21.15.101:0/720327388 shutdown_connections 2023-07-18T03:33:18.825 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.825+0000 7ffa12b9d700 1 -- 172.21.15.101:0/720327388 wait complete. 2023-07-18T03:33:18.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.826+0000 7ffa12b9d700 1 Processor -- start 2023-07-18T03:33:18.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.827+0000 7ffa12b9d700 1 -- start start 2023-07-18T03:33:18.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.827+0000 7ffa12b9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffa0c0ff9d0 0x7ffa0c19d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:18.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.827+0000 7ffa12b9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa0c100380 0x7ffa0c19de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:18.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.827+0000 7ffa12b9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa0c1059e0 0x7ffa0c1979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:18.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.827+0000 7ffa12b9d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0c198020 con 0x7ffa0c0ff9d0 2023-07-18T03:33:18.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.827+0000 7ffa12b9d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0c198190 con 0x7ffa0c1059e0 2023-07-18T03:33:18.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.827+0000 7ffa12b9d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0c198350 con 0x7ffa0c100380 2023-07-18T03:33:18.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.827+0000 7ffa1139a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa0c100380 0x7ffa0c19de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:18.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.827+0000 7ffa1139a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa0c100380 0x7ffa0c19de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:44068/0 (socket says 172.21.15.101:44068) 2023-07-18T03:33:18.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.827+0000 7ffa1139a700 1 -- 172.21.15.101:0/682083974 learned_addr learned my addr 172.21.15.101:0/682083974 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:33:18.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.827+0000 7ffa1239c700 1 --2- 172.21.15.101:0/682083974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa0c1059e0 0x7ffa0c1979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:18.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.827+0000 7ffa11b9b700 1 --2- 172.21.15.101:0/682083974 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffa0c0ff9d0 0x7ffa0c19d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:18.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.828+0000 7ffa1239c700 1 -- 172.21.15.101:0/682083974 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa0c100380 msgr2=0x7ffa0c19de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:18.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.828+0000 7ffa1239c700 1 --2- 172.21.15.101:0/682083974 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa0c100380 0x7ffa0c19de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:18.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.828+0000 7ffa1239c700 1 -- 172.21.15.101:0/682083974 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffa0c0ff9d0 msgr2=0x7ffa0c19d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:18.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.828+0000 7ffa1239c700 1 --2- 172.21.15.101:0/682083974 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffa0c0ff9d0 0x7ffa0c19d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:18.832 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.828+0000 7ffa1239c700 1 -- 172.21.15.101:0/682083974 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa080095f0 con 0x7ffa0c1059e0 2023-07-18T03:33:18.832 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.828+0000 7ffa1139a700 1 --2- 172.21.15.101:0/682083974 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa0c100380 0x7ffa0c19de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:33:18.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.828+0000 7ffa11b9b700 1 --2- 172.21.15.101:0/682083974 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffa0c0ff9d0 0x7ffa0c19d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:33:18.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.828+0000 7ffa1239c700 1 --2- 172.21.15.101:0/682083974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa0c1059e0 0x7ffa0c1979b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ffa0400efc0 tx=0x7ffa0400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:18.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.828+0000 7ffa02ffd700 1 -- 172.21.15.101:0/682083974 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa04018070 con 0x7ffa0c1059e0 2023-07-18T03:33:18.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.828+0000 7ffa02ffd700 1 -- 172.21.15.101:0/682083974 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa0400f040 con 0x7ffa0c1059e0 2023-07-18T03:33:18.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.828+0000 7ffa12b9d700 1 -- 172.21.15.101:0/682083974 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa0c1985d0 con 0x7ffa0c1059e0 2023-07-18T03:33:18.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.828+0000 7ffa02ffd700 1 -- 172.21.15.101:0/682083974 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0400cd40 con 0x7ffa0c1059e0 2023-07-18T03:33:18.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.828+0000 7ffa12b9d700 1 -- 172.21.15.101:0/682083974 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa0c1a4930 con 0x7ffa0c1059e0 2023-07-18T03:33:18.845 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.831+0000 7ffa02ffd700 1 -- 172.21.15.101:0/682083974 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7ffa04003a00 con 0x7ffa0c1059e0 2023-07-18T03:33:18.845 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.832+0000 7ffa27736b80 1 Processor -- start 2023-07-18T03:33:18.845 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.833+0000 7ffa27736b80 1 -- start start 2023-07-18T03:33:18.846 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.833+0000 7ffa27736b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560e540ea4f0 0x560e540ea8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:18.846 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.833+0000 7ffa27736b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e540eae00 0x560e540f47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:18.846 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.833+0000 7ffa27736b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e540f4d30 0x560e540f7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:18.846 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.833+0000 7ffa27736b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e53fefcb0 con 0x560e540ea4f0 2023-07-18T03:33:18.846 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.833+0000 7ffa27736b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e54042a40 con 0x560e540eae00 2023-07-18T03:33:18.847 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.833+0000 7ffa27736b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e5403bed0 con 0x560e540f4d30 2023-07-18T03:33:18.847 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.833+0000 7ffa02ffd700 1 --2- 172.21.15.101:0/682083974 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff9f80a1360 0x7ff9f80a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:18.848 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.833+0000 7ffa1139a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560e540ea4f0 0x560e540ea8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:18.848 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.833+0000 7ffa02ffd700 1 -- 172.21.15.101:0/682083974 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 9549+0+0 (secure 0 0 0) 0x7ffa040ccd40 con 0x7ffa0c1059e0 2023-07-18T03:33:18.848 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.833+0000 7ffa11b9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e540eae00 0x560e540f47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:18.848 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.833+0000 7ffa11b9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e540eae00 0x560e540f47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:44364/0 (socket says 172.21.15.101:44364) 2023-07-18T03:33:18.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.833+0000 7ffa11b9b700 1 -- 172.21.15.101:0/4266325273 learned_addr learned my addr 172.21.15.101:0/4266325273 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:33:18.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.833+0000 7ffa1139a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560e540ea4f0 0x560e540ea8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:40454/0 (socket says 172.21.15.101:40454) 2023-07-18T03:33:18.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.833+0000 7ffa1239c700 1 --2- 172.21.15.101:0/682083974 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff9f80a1360 0x7ff9f80a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:18.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.833+0000 7ffa1139a700 1 --2- 172.21.15.101:0/4266325273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e540f4d30 0x560e540f7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:18.849 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.834+0000 7ffa11b9b700 1 -- 172.21.15.101:0/4266325273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e540f4d30 msgr2=0x560e540f7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:18.850 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.834+0000 7ffa11b9b700 1 --2- 172.21.15.101:0/4266325273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e540f4d30 0x560e540f7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:18.850 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.834+0000 7ffa11b9b700 1 -- 172.21.15.101:0/4266325273 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560e540ea4f0 msgr2=0x560e540ea8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:18.850 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.834+0000 7ffa11b9b700 1 --2- 172.21.15.101:0/4266325273 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560e540ea4f0 0x560e540ea8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:18.850 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.834+0000 7ffa11b9b700 1 -- 172.21.15.101:0/4266325273 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e53e69de0 con 0x560e540eae00 2023-07-18T03:33:18.851 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.834+0000 7ffa1239c700 1 --2- 172.21.15.101:0/682083974 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff9f80a1360 0x7ff9f80a3810 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ffa0400c6f0 tx=0x7ffa04000f40 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:18.851 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.834+0000 7ffa11b9b700 1 --2- 172.21.15.101:0/4266325273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e540eae00 0x560e540f47f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ffa08015000 tx=0x7ffa08005ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba313f9005c5dbcd server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:18.851 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.834+0000 7ff9f7fff700 1 -- 172.21.15.101:0/4266325273 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0801d050 con 0x560e540eae00 2023-07-18T03:33:18.851 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.834+0000 7ff9f7fff700 1 -- 172.21.15.101:0/4266325273 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa0801e040 con 0x560e540eae00 2023-07-18T03:33:18.851 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.834+0000 7ff9f7fff700 1 -- 172.21.15.101:0/4266325273 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa08005430 con 0x560e540eae00 2023-07-18T03:33:18.852 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.835+0000 7ffa27736b80 1 -- 172.21.15.101:0/4266325273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e540eae00 msgr2=0x560e540f47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:18.852 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.835+0000 7ffa27736b80 1 --2- 172.21.15.101:0/4266325273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e540eae00 0x560e540f47f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ffa08015000 tx=0x7ffa08005ea0 comp rx=0 tx=0).stop 2023-07-18T03:33:18.852 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.835+0000 7ffa27736b80 1 -- 172.21.15.101:0/4266325273 shutdown_connections 2023-07-18T03:33:18.853 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.835+0000 7ffa27736b80 1 --2- 172.21.15.101:0/4266325273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e540f4d30 0x560e540f7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:18.853 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.835+0000 7ffa27736b80 1 --2- 172.21.15.101:0/4266325273 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560e540ea4f0 0x560e540ea8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:18.853 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.835+0000 7ffa27736b80 1 --2- 172.21.15.101:0/4266325273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e540eae00 0x560e540f47f0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:18.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.835+0000 7ffa27736b80 1 -- 172.21.15.101:0/4266325273 >> 172.21.15.101:0/4266325273 conn(0x560e540dfff0 msgr2=0x560e540e5930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:18.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.835+0000 7ffa27736b80 1 -- 172.21.15.101:0/4266325273 shutdown_connections 2023-07-18T03:33:18.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.835+0000 7ffa27736b80 1 -- 172.21.15.101:0/4266325273 wait complete. 2023-07-18T03:33:18.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.836+0000 7ffa27736b80 1 -- start start 2023-07-18T03:33:18.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.836+0000 7ffa27736b80 1 Processor -- start 2023-07-18T03:33:18.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.836+0000 7ffa27736b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560e540ea4f0 0x560e540f46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:18.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.836+0000 7ffa27736b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e540eae00 0x560e540f1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:18.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.836+0000 7ffa27736b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e540f4d30 0x560e540f1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:18.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.836+0000 7ffa27736b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e53fdbac0 con 0x560e540ea4f0 2023-07-18T03:33:18.856 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.836+0000 7ffa27736b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e53fdb7c0 con 0x560e540f4d30 2023-07-18T03:33:18.856 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.836+0000 7ffa27736b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e53ebf920 con 0x560e540eae00 2023-07-18T03:33:18.856 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.836+0000 7ffa27736b80 10 client.0 _ll_register_callbacks cb 0x560e53fd9760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:33:18.856 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.836+0000 7ffa1139a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560e540ea4f0 0x560e540f46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:18.857 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.836+0000 7ffa11b9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e540eae00 0x560e540f1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:18.857 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.836+0000 7ffa1139a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560e540ea4f0 0x560e540f46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:40458/0 (socket says 172.21.15.101:40458) 2023-07-18T03:33:18.857 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.837+0000 7ffa1139a700 1 -- 172.21.15.101:0/964511179 learned_addr learned my addr 172.21.15.101:0/964511179 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:33:18.858 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.836+0000 7ffa11b9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e540eae00 0x560e540f1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:44094/0 (socket says 172.21.15.101:44094) 2023-07-18T03:33:18.858 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.837+0000 7ffa1139a700 1 --2- 172.21.15.101:0/964511179 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e540f4d30 0x560e540f1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:18.858 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.837+0000 7ffa11b9b700 1 -- 172.21.15.101:0/964511179 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e540f4d30 msgr2=0x560e540f1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:18.859 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.837+0000 7ffa11b9b700 1 --2- 172.21.15.101:0/964511179 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e540f4d30 0x560e540f1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:18.859 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.837+0000 7ffa11b9b700 1 -- 172.21.15.101:0/964511179 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560e540ea4f0 msgr2=0x560e540f46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:18.859 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.837+0000 7ffa11b9b700 1 --2- 172.21.15.101:0/964511179 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560e540ea4f0 0x560e540f46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:18.859 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.837+0000 7ffa11b9b700 1 -- 172.21.15.101:0/964511179 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa0c10d110 con 0x560e540eae00 2023-07-18T03:33:18.860 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.837+0000 7ffa1139a700 1 --2- 172.21.15.101:0/964511179 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560e540ea4f0 0x560e540f46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:33:18.860 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.837+0000 7ffa11b9b700 1 --2- 172.21.15.101:0/964511179 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e540eae00 0x560e540f1940 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ffa08000c00 tx=0x7ffa08009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcda7210df867ad2 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:18.860 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.837+0000 7ff9f57fa700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:33:18.860 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.837+0000 7ff9f57fa700 1 -- 172.21.15.101:0/964511179 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa08031070 con 0x560e540eae00 2023-07-18T03:33:18.861 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.837+0000 7ffa27736b80 1 -- 172.21.15.101:0/964511179 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560e53f954f0 con 0x560e540eae00 2023-07-18T03:33:18.861 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.837+0000 7ff9f57fa700 1 -- 172.21.15.101:0/964511179 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 9549+0+0 (secure 0 0 0) 0x7ffa080058e0 con 0x560e540eae00 2023-07-18T03:33:18.861 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.838+0000 7ff9f57fa700 1 -- 172.21.15.101:0/964511179 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa0801d020 con 0x560e540eae00 2023-07-18T03:33:18.861 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.838+0000 7ffa27736b80 10 client.29218 fetch_fsmap learned FSMap version 49 2023-07-18T03:33:18.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.838+0000 7ffa27736b80 1 -- 172.21.15.101:0/964511179 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x560e53e52c60 con 0x560e540eae00 2023-07-18T03:33:18.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.838+0000 7ff9f57fa700 1 -- 172.21.15.101:0/964511179 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560e53f954f0 con 0x560e540eae00 2023-07-18T03:33:18.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.838+0000 7ff9f57fa700 1 -- 172.21.15.101:0/964511179 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 49) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ffa08034020 con 0x560e540eae00 2023-07-18T03:33:18.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.838+0000 7ffa27736b80 10 client.29218 fetch_fsmap finished waiting for FSMap version 49 2023-07-18T03:33:18.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.838+0000 7ffa27736b80 10 client.29218 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:33:18.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.838+0000 7ffa27736b80 20 client.29218 populate_metadata read hostname 'smithi101' 2023-07-18T03:33:18.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.838+0000 7ffa27736b80 1 --2- 172.21.15.101:0/964511179 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x560e54104150 0x560e54106530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:18.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.838+0000 7ffa27736b80 4 client.29218 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:33:18.864 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.838+0000 7ffa27736b80 1 -- 172.21.15.101:0/964511179 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560e54034eb0 con 0x560e54104150 2023-07-18T03:33:18.864 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.838+0000 7ffa1139a700 1 --2- 172.21.15.101:0/964511179 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x560e54104150 0x560e54106530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:18.864 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.839+0000 7ff9f57fa700 10 client.29218 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:33:18.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.839+0000 7ffa1139a700 1 --2- 172.21.15.101:0/964511179 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x560e54104150 0x560e54106530 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a9469e5a118b1ff server_cookie=f087a766f1e3e1a2 in_seq=0 out_seq=0 2023-07-18T03:33:18.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.839+0000 7ff9f57fa700 1 -- 172.21.15.101:0/964511179 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560e54034eb0 con 0x560e54104150 2023-07-18T03:33:18.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.839+0000 7ff9f57fa700 10 client.29218 handle_command_reply: tid=0 2023-07-18T03:33:18.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.839+0000 7ffa27736b80 1 client.29218 shutdown 2023-07-18T03:33:18.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.840+0000 7ffa27736b80 1 -- 172.21.15.101:0/964511179 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e540eae00 msgr2=0x560e540f1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:18.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.840+0000 7ffa27736b80 1 --2- 172.21.15.101:0/964511179 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e540eae00 0x560e540f1940 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ffa08000c00 tx=0x7ffa08009580 comp rx=0 tx=0).stop 2023-07-18T03:33:18.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.840+0000 7ffa27736b80 1 -- 172.21.15.101:0/964511179 shutdown_connections 2023-07-18T03:33:18.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.840+0000 7ffa27736b80 1 --2- 172.21.15.101:0/964511179 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x560e54104150 0x560e54106530 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:18.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.840+0000 7ffa27736b80 1 --2- 172.21.15.101:0/964511179 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e540f4d30 0x560e540f1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:18.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.840+0000 7ffa27736b80 1 --2- 172.21.15.101:0/964511179 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560e540ea4f0 0x560e540f46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:18.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.840+0000 7ffa27736b80 1 --2- 172.21.15.101:0/964511179 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e540eae00 0x560e540f1940 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:18.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.840+0000 7ff9f57fa700 0 client.29218 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:33:18.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.840+0000 7ffa27736b80 1 -- 172.21.15.101:0/964511179 >> 172.21.15.101:0/964511179 conn(0x560e540dfff0 msgr2=0x560e540e5520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:18.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.840+0000 7ffa27736b80 1 -- 172.21.15.101:0/964511179 shutdown_connections 2023-07-18T03:33:18.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.840+0000 7ffa27736b80 1 -- 172.21.15.101:0/964511179 wait complete. 2023-07-18T03:33:18.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.840+0000 7ffa27736b80 20 client.29218 trim_cache size 0 max 16384 2023-07-18T03:33:18.878 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.857+0000 7ffa27736b80 1 Processor -- start 2023-07-18T03:33:18.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.857+0000 7ffa27736b80 1 -- start start 2023-07-18T03:33:18.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.858+0000 7ffa27736b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e540ea4f0 0x560e540e4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:18.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.858+0000 7ffa27736b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e540eae00 0x560e540f4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:18.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.858+0000 7ffa27736b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560e540f4d30 0x560e540f1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:18.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.858+0000 7ffa27736b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e540eb980 con 0x560e540f4d30 2023-07-18T03:33:18.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.858+0000 7ffa27736b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e540f2f20 con 0x560e540eae00 2023-07-18T03:33:18.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.858+0000 7ffa27736b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e540dfb70 con 0x560e540ea4f0 2023-07-18T03:33:18.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.858+0000 7ffa11b9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e540eae00 0x560e540f4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:18.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.858+0000 7ffa1139a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e540ea4f0 0x560e540e4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:18.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.858+0000 7ffa11b9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e540eae00 0x560e540f4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:44384/0 (socket says 172.21.15.101:44384) 2023-07-18T03:33:18.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.858+0000 7ffa11b9b700 1 -- 172.21.15.101:0/1364456214 learned_addr learned my addr 172.21.15.101:0/1364456214 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:33:18.882 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.858+0000 7ffa1139a700 1 --2- 172.21.15.101:0/1364456214 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560e540f4d30 0x560e540f1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:18.882 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.858+0000 7ffa11b9b700 1 -- 172.21.15.101:0/1364456214 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e540ea4f0 msgr2=0x560e540e4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:18.882 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.858+0000 7ffa11b9b700 1 --2- 172.21.15.101:0/1364456214 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e540ea4f0 0x560e540e4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:18.882 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.858+0000 7ffa11b9b700 1 -- 172.21.15.101:0/1364456214 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560e540f4d30 msgr2=0x560e540f1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:18.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.858+0000 7ffa11b9b700 1 --2- 172.21.15.101:0/1364456214 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560e540f4d30 0x560e540f1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:18.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.858+0000 7ffa11b9b700 1 -- 172.21.15.101:0/1364456214 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e53e90250 con 0x560e540eae00 2023-07-18T03:33:18.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.858+0000 7ffa1139a700 1 --2- 172.21.15.101:0/1364456214 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e540ea4f0 0x560e540e4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:33:18.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.858+0000 7ffa11b9b700 1 --2- 172.21.15.101:0/1364456214 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e540eae00 0x560e540f4260 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ffa08009990 tx=0x7ffa08005ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1a6e85fc32b0f2b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:18.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.859+0000 7ffa00bff700 1 -- 172.21.15.101:0/1364456214 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0801d070 con 0x560e540eae00 2023-07-18T03:33:18.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.859+0000 7ffa00bff700 1 -- 172.21.15.101:0/1364456214 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa08023dd0 con 0x560e540eae00 2023-07-18T03:33:18.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.859+0000 7ffa00bff700 1 -- 172.21.15.101:0/1364456214 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa08031070 con 0x560e540eae00 2023-07-18T03:33:18.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.859+0000 7ffa27736b80 1 -- 172.21.15.101:0/1364456214 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e540eae00 msgr2=0x560e540f4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:18.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.859+0000 7ffa27736b80 1 --2- 172.21.15.101:0/1364456214 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e540eae00 0x560e540f4260 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ffa08009990 tx=0x7ffa08005ea0 comp rx=0 tx=0).stop 2023-07-18T03:33:18.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.859+0000 7ffa27736b80 1 -- 172.21.15.101:0/1364456214 shutdown_connections 2023-07-18T03:33:18.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.859+0000 7ffa27736b80 1 --2- 172.21.15.101:0/1364456214 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560e540f4d30 0x560e540f1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:18.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.859+0000 7ffa27736b80 1 --2- 172.21.15.101:0/1364456214 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e540ea4f0 0x560e540e4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:18.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.859+0000 7ffa27736b80 1 --2- 172.21.15.101:0/1364456214 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e540eae00 0x560e540f4260 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:18.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.859+0000 7ffa27736b80 1 -- 172.21.15.101:0/1364456214 >> 172.21.15.101:0/1364456214 conn(0x560e540dfff0 msgr2=0x560e541009e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:18.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.859+0000 7ffa27736b80 1 -- 172.21.15.101:0/1364456214 shutdown_connections 2023-07-18T03:33:18.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.859+0000 7ffa27736b80 1 -- 172.21.15.101:0/1364456214 wait complete. 2023-07-18T03:33:18.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.860+0000 7ffa27736b80 1 -- start start 2023-07-18T03:33:18.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.860+0000 7ffa27736b80 1 Processor -- start 2023-07-18T03:33:18.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.860+0000 7ffa27736b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e540ea4f0 0x560e540f9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:18.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.860+0000 7ffa27736b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e540eae00 0x560e540f9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:18.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.860+0000 7ffa27736b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560e540f4d30 0x560e54150e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:18.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.860+0000 7ffa27736b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e54151390 con 0x560e540f4d30 2023-07-18T03:33:18.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.860+0000 7ffa27736b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e54151500 con 0x560e540eae00 2023-07-18T03:33:18.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.860+0000 7ffa27736b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e541516c0 con 0x560e540ea4f0 2023-07-18T03:33:18.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.860+0000 7ffa27736b80 10 client.0 _ll_register_callbacks cb 0x7ff9fc013460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:33:18.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.861+0000 7ffa11b9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e540eae00 0x560e540f9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:18.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.861+0000 7ffa11b9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e540eae00 0x560e540f9ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:44400/0 (socket says 172.21.15.101:44400) 2023-07-18T03:33:18.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.861+0000 7ffa11b9b700 1 -- 172.21.15.101:0/1383907159 learned_addr learned my addr 172.21.15.101:0/1383907159 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:33:18.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.861+0000 7ffa1139a700 1 --2- 172.21.15.101:0/1383907159 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e540ea4f0 0x560e540f9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:18.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.861+0000 7ffa11b9b700 1 -- 172.21.15.101:0/1383907159 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e540ea4f0 msgr2=0x560e540f9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:18.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.861+0000 7ffa11b9b700 1 --2- 172.21.15.101:0/1383907159 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e540ea4f0 0x560e540f9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:18.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.861+0000 7ffa1139a700 1 --2- 172.21.15.101:0/1383907159 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560e540f4d30 0x560e54150e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:18.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.861+0000 7ffa11b9b700 1 -- 172.21.15.101:0/1383907159 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560e540f4d30 msgr2=0x560e54150e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:18.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.861+0000 7ffa11b9b700 1 --2- 172.21.15.101:0/1383907159 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560e540f4d30 0x560e54150e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:18.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.861+0000 7ffa11b9b700 1 -- 172.21.15.101:0/1383907159 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560e53e52c60 con 0x560e540eae00 2023-07-18T03:33:18.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.861+0000 7ffa1139a700 1 --2- 172.21.15.101:0/1383907159 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560e540f4d30 0x560e54150e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:33:18.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.861+0000 7ffa11b9b700 1 --2- 172.21.15.101:0/1383907159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e540eae00 0x560e540f9ec0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ffa08026040 tx=0x7ffa080049f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f588bc6771f33a69 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:18.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.861+0000 7ff9f57fa700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:33:18.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.861+0000 7ff9f57fa700 1 -- 172.21.15.101:0/1383907159 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa08031040 con 0x560e540eae00 2023-07-18T03:33:18.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.861+0000 7ffa27736b80 1 -- 172.21.15.101:0/1383907159 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffa0801e040 con 0x560e540eae00 2023-07-18T03:33:18.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.861+0000 7ff9f57fa700 1 -- 172.21.15.101:0/1383907159 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 9549+0+0 (secure 0 0 0) 0x7ffa08000e40 con 0x560e540eae00 2023-07-18T03:33:18.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.862+0000 7ff9f57fa700 1 -- 172.21.15.101:0/1383907159 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa0801d040 con 0x560e540eae00 2023-07-18T03:33:18.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.862+0000 7ffa27736b80 10 client.29234 fetch_fsmap learned FSMap version 49 2023-07-18T03:33:18.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.862+0000 7ffa27736b80 1 -- 172.21.15.101:0/1383907159 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x560e53e682b0 con 0x560e540eae00 2023-07-18T03:33:18.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.862+0000 7ff9f57fa700 1 -- 172.21.15.101:0/1383907159 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa0801e040 con 0x560e540eae00 2023-07-18T03:33:18.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.862+0000 7ff9f57fa700 1 -- 172.21.15.101:0/1383907159 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 49) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ffa08004500 con 0x560e540eae00 2023-07-18T03:33:18.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.862+0000 7ffa27736b80 10 client.29234 fetch_fsmap finished waiting for FSMap version 49 2023-07-18T03:33:18.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.862+0000 7ffa27736b80 10 client.29234 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:33:18.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.862+0000 7ffa27736b80 20 client.29234 populate_metadata read hostname 'smithi101' 2023-07-18T03:33:18.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.862+0000 7ffa27736b80 1 --2- 172.21.15.101:0/1383907159 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x560e54158b20 0x560e5415af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:18.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.862+0000 7ffa27736b80 4 client.29234 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:33:18.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.862+0000 7ffa27736b80 1 -- 172.21.15.101:0/1383907159 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560e53ff7050 con 0x560e54158b20 2023-07-18T03:33:18.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.862+0000 7ffa1139a700 1 --2- 172.21.15.101:0/1383907159 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x560e54158b20 0x560e5415af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:18.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.863+0000 7ff9f57fa700 10 client.29234 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:33:18.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.863+0000 7ffa1139a700 1 --2- 172.21.15.101:0/1383907159 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x560e54158b20 0x560e5415af30 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecbca6b8f7dca172 server_cookie=1bc83549fb24a38a in_seq=0 out_seq=0 2023-07-18T03:33:18.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.863+0000 7ff9f57fa700 1 -- 172.21.15.101:0/1383907159 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560e53ff7050 con 0x560e54158b20 2023-07-18T03:33:18.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.863+0000 7ff9f57fa700 10 client.29234 handle_command_reply: tid=0 2023-07-18T03:33:18.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.863+0000 7ffa27736b80 1 client.29234 shutdown 2023-07-18T03:33:18.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.863+0000 7ffa27736b80 1 -- 172.21.15.101:0/1383907159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e540eae00 msgr2=0x560e540f9ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:18.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.863+0000 7ffa27736b80 1 --2- 172.21.15.101:0/1383907159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e540eae00 0x560e540f9ec0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ffa08026040 tx=0x7ffa080049f0 comp rx=0 tx=0).stop 2023-07-18T03:33:18.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.863+0000 7ffa27736b80 1 -- 172.21.15.101:0/1383907159 shutdown_connections 2023-07-18T03:33:18.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.863+0000 7ffa27736b80 1 --2- 172.21.15.101:0/1383907159 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x560e54158b20 0x560e5415af30 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:18.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.863+0000 7ffa27736b80 1 --2- 172.21.15.101:0/1383907159 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560e540f4d30 0x560e54150e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:18.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.863+0000 7ff9f57fa700 0 client.29234 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:33:18.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.863+0000 7ffa27736b80 1 --2- 172.21.15.101:0/1383907159 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e540ea4f0 0x560e540f9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:18.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.863+0000 7ffa27736b80 1 --2- 172.21.15.101:0/1383907159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e540eae00 0x560e540f9ec0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:18.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.863+0000 7ffa27736b80 1 -- 172.21.15.101:0/1383907159 >> 172.21.15.101:0/1383907159 conn(0x560e540dfff0 msgr2=0x560e540eccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:18.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.863+0000 7ffa27736b80 1 -- 172.21.15.101:0/1383907159 shutdown_connections 2023-07-18T03:33:18.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.863+0000 7ffa27736b80 1 -- 172.21.15.101:0/1383907159 wait complete. 2023-07-18T03:33:18.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.864+0000 7ffa27736b80 20 client.29234 trim_cache size 0 max 16384 2023-07-18T03:33:18.899 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:33:18.899 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:33:18.899 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:33:18.900 INFO:teuthology.orchestra.run.smithi101.stdout: "0f2f19e0-0429-465c-935e-223eb09490c3": { 2023-07-18T03:33:18.900 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:33:18.900 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "0f2f19e0-0429-465c-935e-223eb09490c3", 2023-07-18T03:33:18.900 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:33:18.901 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:33:18.901 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:33:18.901 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:33:18.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.864+0000 7ffa12b9d700 1 -- 172.21.15.101:0/682083974 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff9f80a1360 msgr2=0x7ff9f80a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:18.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.864+0000 7ffa12b9d700 1 --2- 172.21.15.101:0/682083974 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff9f80a1360 0x7ff9f80a3810 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ffa0400c6f0 tx=0x7ffa04000f40 comp rx=0 tx=0).stop 2023-07-18T03:33:18.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.864+0000 7ffa12b9d700 1 -- 172.21.15.101:0/682083974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa0c1059e0 msgr2=0x7ffa0c1979b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:18.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.864+0000 7ffa12b9d700 1 --2- 172.21.15.101:0/682083974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa0c1059e0 0x7ffa0c1979b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ffa0400efc0 tx=0x7ffa0400c3a0 comp rx=0 tx=0).stop 2023-07-18T03:33:18.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.864+0000 7ffa12b9d700 1 -- 172.21.15.101:0/682083974 shutdown_connections 2023-07-18T03:33:18.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.864+0000 7ffa12b9d700 1 --2- 172.21.15.101:0/682083974 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7ff9f80a1360 0x7ff9f80a3810 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:18.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.864+0000 7ffa12b9d700 1 --2- 172.21.15.101:0/682083974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa0c1059e0 0x7ffa0c1979b0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:18.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.864+0000 7ffa12b9d700 1 --2- 172.21.15.101:0/682083974 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffa0c0ff9d0 0x7ffa0c19d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:18.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.864+0000 7ffa12b9d700 1 --2- 172.21.15.101:0/682083974 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa0c100380 0x7ffa0c19de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:18.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.864+0000 7ffa12b9d700 1 -- 172.21.15.101:0/682083974 >> 172.21.15.101:0/682083974 conn(0x7ffa0c0fb4b0 msgr2=0x7ffa0c072740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:18.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.864+0000 7ffa12b9d700 1 -- 172.21.15.101:0/682083974 shutdown_connections 2023-07-18T03:33:18.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:18.864+0000 7ffa12b9d700 1 -- 172.21.15.101:0/682083974 wait complete. 2023-07-18T03:33:18.906 INFO:tasks.cephfs.filesystem:scrub status for tag:0f2f19e0-0429-465c-935e-223eb09490c3 - {'path': '/', 'tag': '0f2f19e0-0429-465c-935e-223eb09490c3', 'options': 'recursive,force'} 2023-07-18T03:33:19.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:33:19 smithi179 ceph-mon[129229]: pgmap v814: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:33:19.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:33:19 smithi179 ceph-mon[129229]: osdmap e132: 12 total, 12 up, 12 in 2023-07-18T03:33:19.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:33:19 smithi101 ceph-mon[126379]: pgmap v814: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:33:19.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:33:19 smithi101 ceph-mon[126379]: osdmap e132: 12 total, 12 up, 12 in 2023-07-18T03:33:19.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:33:19 smithi155 ceph-mon[128885]: pgmap v814: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:33:19.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:33:19 smithi155 ceph-mon[128885]: osdmap e132: 12 total, 12 up, 12 in 2023-07-18T03:33:21.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:33:21 smithi179 ceph-mon[129229]: pgmap v816: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 210 B/s wr, 6 op/s 2023-07-18T03:33:21.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:33:21 smithi101 ceph-mon[126379]: pgmap v816: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 210 B/s wr, 6 op/s 2023-07-18T03:33:21.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:33:21 smithi155 ceph-mon[128885]: pgmap v816: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 210 B/s wr, 6 op/s 2023-07-18T03:33:23.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:33:23 smithi179 ceph-mon[129229]: pgmap v817: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:33:23.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:33:23 smithi101 ceph-mon[126379]: pgmap v817: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:33:23.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:33:23 smithi155 ceph-mon[128885]: pgmap v817: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:33:25.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:33:25 smithi179 ceph-mon[129229]: pgmap v818: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:33:25.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:33:25 smithi101 ceph-mon[126379]: pgmap v818: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:33:25.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:33:25 smithi155 ceph-mon[128885]: pgmap v818: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:33:27.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:33:27 smithi179 ceph-mon[129229]: pgmap v819: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:33:27.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:33:27 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:33:27.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:33:27 smithi101 ceph-mon[126379]: pgmap v819: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:33:27.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:33:27 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:33:27.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:33:27 smithi155 ceph-mon[128885]: pgmap v819: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:33:27.800 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:33:27 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:33:28.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:33:28 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:33:28.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:33:28 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:33:28.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:33:28 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:33:28.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:33:28 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:33:28.743 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:33:28 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:33:28.743 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:33:28 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:33:28.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:33:28 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:33:28.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:33:28 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:33:28.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:33:28 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:33:29.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:33:29 smithi179 ceph-mon[129229]: pgmap v820: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:33:29.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:33:29 smithi101 ceph-mon[126379]: pgmap v820: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:33:29.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:33:29 smithi155 ceph-mon[128885]: pgmap v820: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:33:31.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:33:31 smithi179 ceph-mon[129229]: pgmap v821: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 95 B/s wr, 8 op/s 2023-07-18T03:33:31.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:33:31 smithi101 ceph-mon[126379]: pgmap v821: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 95 B/s wr, 8 op/s 2023-07-18T03:33:31.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:33:31 smithi155 ceph-mon[128885]: pgmap v821: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 95 B/s wr, 8 op/s 2023-07-18T03:33:33.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:33:33 smithi179 ceph-mon[129229]: pgmap v822: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:33:33.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:33:33 smithi101 ceph-mon[126379]: pgmap v822: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:33:33.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:33:33 smithi155 ceph-mon[128885]: pgmap v822: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:33:35.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:33:35 smithi179 ceph-mon[129229]: pgmap v823: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:33:35.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:33:35 smithi101 ceph-mon[126379]: pgmap v823: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:33:35.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:33:35 smithi155 ceph-mon[128885]: pgmap v823: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:33:37.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:33:37 smithi179 ceph-mon[129229]: pgmap v824: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:33:37.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:33:37 smithi101 ceph-mon[126379]: pgmap v824: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:33:37.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:33:37 smithi155 ceph-mon[128885]: pgmap v824: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:33:39.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:33:39 smithi179 ceph-mon[129229]: pgmap v825: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:33:39.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:33:39 smithi101 ceph-mon[126379]: pgmap v825: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:33:39.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:33:39 smithi155 ceph-mon[128885]: pgmap v825: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:33:41.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:33:41 smithi179 ceph-mon[129229]: pgmap v826: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:33:41.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:33:41 smithi101 ceph-mon[126379]: pgmap v826: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:33:41.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:33:41 smithi155 ceph-mon[128885]: pgmap v826: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:33:43.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:33:43 smithi179 ceph-mon[129229]: pgmap v827: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:33:43.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:33:43 smithi101 ceph-mon[126379]: pgmap v827: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:33:43.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:33:43 smithi155 ceph-mon[128885]: pgmap v827: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:33:45.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:33:45 smithi179 ceph-mon[129229]: pgmap v828: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:33:45.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:33:45 smithi101 ceph-mon[126379]: pgmap v828: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:33:45.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:33:45 smithi155 ceph-mon[128885]: pgmap v828: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:33:47.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:33:47 smithi179 ceph-mon[129229]: pgmap v829: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:33:47.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:33:47 smithi101 ceph-mon[126379]: pgmap v829: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:33:47.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:33:47 smithi155 ceph-mon[128885]: pgmap v829: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:33:48.908 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:33:49.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.120+0000 7fc1d11ff700 1 -- 172.21.15.101:0/571622155 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1cc0728b0 msgr2=0x7fc1cc072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:49.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.120+0000 7fc1d11ff700 1 --2- 172.21.15.101:0/571622155 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1cc0728b0 0x7fc1cc072c80 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc009990 tx=0x7fc1bc009ca0 comp rx=0 tx=0).stop 2023-07-18T03:33:49.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.120+0000 7fc1d11ff700 1 -- 172.21.15.101:0/571622155 shutdown_connections 2023-07-18T03:33:49.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.120+0000 7fc1d11ff700 1 --2- 172.21.15.101:0/571622155 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc1cc10c320 0x7fc1cc10e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:49.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.120+0000 7fc1d11ff700 1 --2- 172.21.15.101:0/571622155 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1cc0728b0 0x7fc1cc072c80 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:49.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.120+0000 7fc1d11ff700 1 --2- 172.21.15.101:0/571622155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1cc0731c0 0x7fc1cc10bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:49.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.120+0000 7fc1d11ff700 1 -- 172.21.15.101:0/571622155 >> 172.21.15.101:0/571622155 conn(0x7fc1cc0fb5e0 msgr2=0x7fc1cc0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:49.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.120+0000 7fc1d11ff700 1 -- 172.21.15.101:0/571622155 shutdown_connections 2023-07-18T03:33:49.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.120+0000 7fc1d11ff700 1 -- 172.21.15.101:0/571622155 wait complete. 2023-07-18T03:33:49.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.121+0000 7fc1d11ff700 1 Processor -- start 2023-07-18T03:33:49.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.121+0000 7fc1d11ff700 1 -- start start 2023-07-18T03:33:49.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.122+0000 7fc1d11ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1cc0728b0 0x7fc1cc197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:49.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.122+0000 7fc1d11ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc1cc0731c0 0x7fc1cc198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:49.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.122+0000 7fc1d11ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1cc10c320 0x7fc1cc19c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:49.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.122+0000 7fc1d11ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc19cdf0 con 0x7fc1cc0731c0 2023-07-18T03:33:49.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.122+0000 7fc1d11ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc19cf60 con 0x7fc1cc0728b0 2023-07-18T03:33:49.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.122+0000 7fc1d11ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc19d120 con 0x7fc1cc10c320 2023-07-18T03:33:49.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.122+0000 7fc1cad9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1cc0728b0 0x7fc1cc197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:49.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.122+0000 7fc1cb59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1cc10c320 0x7fc1cc19c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:49.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.122+0000 7fc1cad9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1cc0728b0 0x7fc1cc197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:39724/0 (socket says 172.21.15.101:39724) 2023-07-18T03:33:49.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.122+0000 7fc1cad9d700 1 -- 172.21.15.101:0/196244747 learned_addr learned my addr 172.21.15.101:0/196244747 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:33:49.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.122+0000 7fc1ca59c700 1 --2- 172.21.15.101:0/196244747 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc1cc0731c0 0x7fc1cc198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:49.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.122+0000 7fc1cad9d700 1 -- 172.21.15.101:0/196244747 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1cc10c320 msgr2=0x7fc1cc19c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:49.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.122+0000 7fc1cad9d700 1 --2- 172.21.15.101:0/196244747 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1cc10c320 0x7fc1cc19c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:49.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.122+0000 7fc1cad9d700 1 -- 172.21.15.101:0/196244747 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc1cc0731c0 msgr2=0x7fc1cc198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:49.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.122+0000 7fc1cad9d700 1 --2- 172.21.15.101:0/196244747 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc1cc0731c0 0x7fc1cc198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:49.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.122+0000 7fc1cad9d700 1 -- 172.21.15.101:0/196244747 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1bc0095f0 con 0x7fc1cc0728b0 2023-07-18T03:33:49.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.122+0000 7fc1cb59e700 1 --2- 172.21.15.101:0/196244747 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1cc10c320 0x7fc1cc19c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:33:49.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.123+0000 7fc1cad9d700 1 --2- 172.21.15.101:0/196244747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1cc0728b0 0x7fc1cc197e10 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc005910 tx=0x7fc1bc005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:49.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.123+0000 7fc1b3fff700 1 -- 172.21.15.101:0/196244747 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc01d070 con 0x7fc1cc0728b0 2023-07-18T03:33:49.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.123+0000 7fc1b3fff700 1 -- 172.21.15.101:0/196244747 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1bc022470 con 0x7fc1cc0728b0 2023-07-18T03:33:49.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.123+0000 7fc1d11ff700 1 -- 172.21.15.101:0/196244747 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1cc19d3a0 con 0x7fc1cc0728b0 2023-07-18T03:33:49.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.123+0000 7fc1b3fff700 1 -- 172.21.15.101:0/196244747 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc017650 con 0x7fc1cc0728b0 2023-07-18T03:33:49.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.123+0000 7fc1d11ff700 1 -- 172.21.15.101:0/196244747 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1cc1a4c40 con 0x7fc1cc0728b0 2023-07-18T03:33:49.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.126+0000 7fc1b3fff700 1 -- 172.21.15.101:0/196244747 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fc1bc0225e0 con 0x7fc1cc0728b0 2023-07-18T03:33:49.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.128+0000 7fc1e4bb4b80 1 Processor -- start 2023-07-18T03:33:49.130 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.128+0000 7fc1e4bb4b80 1 -- start start 2023-07-18T03:33:49.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.128+0000 7fc1e4bb4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c627ed2d0 0x557c627ed6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:49.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.128+0000 7fc1e4bb4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c627edbe0 0x557c627f8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:49.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.128+0000 7fc1e4bb4b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c627f8ad0 0x557c627faeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:49.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.128+0000 7fc1e4bb4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c626f3cb0 con 0x557c627edbe0 2023-07-18T03:33:49.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.128+0000 7fc1e4bb4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c62746a40 con 0x557c627ed2d0 2023-07-18T03:33:49.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.128+0000 7fc1e4bb4b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c6273fed0 con 0x557c627f8ad0 2023-07-18T03:33:49.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.129+0000 7fc1cb59e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c627ed2d0 0x557c627ed6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:49.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.129+0000 7fc1cb59e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c627ed2d0 0x557c627ed6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:39730/0 (socket says 172.21.15.101:39730) 2023-07-18T03:33:49.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.129+0000 7fc1cb59e700 1 -- 172.21.15.101:0/2397931989 learned_addr learned my addr 172.21.15.101:0/2397931989 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:33:49.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.129+0000 7fc1ca59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c627f8ad0 0x557c627faeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:49.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.129+0000 7fc1cad9d700 1 --2- 172.21.15.101:0/2397931989 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c627edbe0 0x557c627f8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:49.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.129+0000 7fc1b3fff700 1 --2- 172.21.15.101:0/196244747 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc1b40a12c0 0x7fc1b40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:49.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.129+0000 7fc1ca59c700 1 -- 172.21.15.101:0/2397931989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c627ed2d0 msgr2=0x557c627ed6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:49.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.129+0000 7fc1ca59c700 1 --2- 172.21.15.101:0/2397931989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c627ed2d0 0x557c627ed6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:49.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.129+0000 7fc1ca59c700 1 -- 172.21.15.101:0/2397931989 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c627edbe0 msgr2=0x557c627f8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:49.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.129+0000 7fc1ca59c700 1 --2- 172.21.15.101:0/2397931989 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c627edbe0 0x557c627f8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:49.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.129+0000 7fc1b3fff700 1 -- 172.21.15.101:0/196244747 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fc1bc0ce430 con 0x7fc1cc0728b0 2023-07-18T03:33:49.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.129+0000 7fc1ca59c700 1 -- 172.21.15.101:0/2397931989 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c6256dde0 con 0x557c627f8ad0 2023-07-18T03:33:49.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.129+0000 7fc1cb59e700 1 --2- 172.21.15.101:0/196244747 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc1b40a12c0 0x7fc1b40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:49.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.129+0000 7fc1ca59c700 1 --2- 172.21.15.101:0/2397931989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c627f8ad0 0x557c627faeb0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0009c80 tx=0x7fc1c0009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69e1bf212010d764 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:49.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.130+0000 7fc1b17fa700 1 -- 172.21.15.101:0/2397931989 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c0011c50 con 0x557c627f8ad0 2023-07-18T03:33:49.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.130+0000 7fc1b17fa700 1 -- 172.21.15.101:0/2397931989 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1c0004510 con 0x557c627f8ad0 2023-07-18T03:33:49.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.130+0000 7fc1b17fa700 1 -- 172.21.15.101:0/2397931989 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c0005280 con 0x557c627f8ad0 2023-07-18T03:33:49.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.130+0000 7fc1e4bb4b80 1 -- 172.21.15.101:0/2397931989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c627f8ad0 msgr2=0x557c627faeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:49.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.130+0000 7fc1cb59e700 1 --2- 172.21.15.101:0/196244747 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc1b40a12c0 0x7fc1b40a3770 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4009fd0 tx=0x7fc1c4004ad0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:49.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.130+0000 7fc1e4bb4b80 1 --2- 172.21.15.101:0/2397931989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c627f8ad0 0x557c627faeb0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0009c80 tx=0x7fc1c0009f90 comp rx=0 tx=0).stop 2023-07-18T03:33:49.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.130+0000 7fc1e4bb4b80 1 -- 172.21.15.101:0/2397931989 shutdown_connections 2023-07-18T03:33:49.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.130+0000 7fc1e4bb4b80 1 --2- 172.21.15.101:0/2397931989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c627ed2d0 0x557c627ed6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:49.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.130+0000 7fc1e4bb4b80 1 --2- 172.21.15.101:0/2397931989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c627f8ad0 0x557c627faeb0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:49.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.130+0000 7fc1e4bb4b80 1 --2- 172.21.15.101:0/2397931989 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c627edbe0 0x557c627f8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:49.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.130+0000 7fc1e4bb4b80 1 -- 172.21.15.101:0/2397931989 >> 172.21.15.101:0/2397931989 conn(0x557c627e3da0 msgr2=0x557c627e96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:49.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.130+0000 7fc1e4bb4b80 1 -- 172.21.15.101:0/2397931989 shutdown_connections 2023-07-18T03:33:49.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.131+0000 7fc1e4bb4b80 1 -- 172.21.15.101:0/2397931989 wait complete. 2023-07-18T03:33:49.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.131+0000 7fc1e4bb4b80 1 -- start start 2023-07-18T03:33:49.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.131+0000 7fc1e4bb4b80 1 Processor -- start 2023-07-18T03:33:49.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.132+0000 7fc1e4bb4b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c627ed2d0 0x557c627f8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:49.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.132+0000 7fc1e4bb4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c627edbe0 0x557c627f5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:49.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.132+0000 7fc1e4bb4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c627f8ad0 0x557c627f5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:49.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.132+0000 7fc1e4bb4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c626dfac0 con 0x557c627f8ad0 2023-07-18T03:33:49.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.132+0000 7fc1e4bb4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c626df7c0 con 0x557c627edbe0 2023-07-18T03:33:49.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.132+0000 7fc1e4bb4b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c625c3920 con 0x557c627ed2d0 2023-07-18T03:33:49.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.132+0000 7fc1e4bb4b80 10 client.0 _ll_register_callbacks cb 0x557c626dd760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:33:49.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.132+0000 7fc1ca59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c627edbe0 0x557c627f5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:49.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.132+0000 7fc1cad9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c627ed2d0 0x557c627f8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:49.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.132+0000 7fc1cb59e700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c627f8ad0 0x557c627f5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:49.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.132+0000 7fc1ca59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c627edbe0 0x557c627f5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:39734/0 (socket says 172.21.15.101:39734) 2023-07-18T03:33:49.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.132+0000 7fc1ca59c700 1 -- 172.21.15.101:0/3471385721 learned_addr learned my addr 172.21.15.101:0/3471385721 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:33:49.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.132+0000 7fc1ca59c700 1 -- 172.21.15.101:0/3471385721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c627ed2d0 msgr2=0x557c627f8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:49.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.132+0000 7fc1ca59c700 1 --2- 172.21.15.101:0/3471385721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c627ed2d0 0x557c627f8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:49.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.132+0000 7fc1ca59c700 1 -- 172.21.15.101:0/3471385721 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c627f8ad0 msgr2=0x557c627f5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:49.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.132+0000 7fc1ca59c700 1 --2- 172.21.15.101:0/3471385721 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c627f8ad0 0x557c627f5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:49.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.132+0000 7fc1ca59c700 1 -- 172.21.15.101:0/3471385721 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1c00098e0 con 0x557c627edbe0 2023-07-18T03:33:49.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.132+0000 7fc1cad9d700 1 --2- 172.21.15.101:0/3471385721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c627ed2d0 0x557c627f8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:33:49.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.133+0000 7fc1ca59c700 1 --2- 172.21.15.101:0/3471385721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c627edbe0 0x557c627f5720 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0004b60 tx=0x7fc1c0004b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9259c15a3c0d968 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:49.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.133+0000 7fc1aeff5700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:33:49.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.133+0000 7fc1aeff5700 1 -- 172.21.15.101:0/3471385721 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c0011a50 con 0x557c627edbe0 2023-07-18T03:33:49.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.133+0000 7fc1e4bb4b80 1 -- 172.21.15.101:0/3471385721 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c626994f0 con 0x557c627edbe0 2023-07-18T03:33:49.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.133+0000 7fc1aeff5700 1 -- 172.21.15.101:0/3471385721 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fc1c0011bb0 con 0x557c627edbe0 2023-07-18T03:33:49.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.133+0000 7fc1aeff5700 1 -- 172.21.15.101:0/3471385721 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1c0011d70 con 0x557c627edbe0 2023-07-18T03:33:49.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.133+0000 7fc1e4bb4b80 10 client.29246 fetch_fsmap learned FSMap version 49 2023-07-18T03:33:49.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.133+0000 7fc1e4bb4b80 1 -- 172.21.15.101:0/3471385721 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x557c62556c60 con 0x557c627edbe0 2023-07-18T03:33:49.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.133+0000 7fc1aeff5700 1 -- 172.21.15.101:0/3471385721 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557c626994f0 con 0x557c627edbe0 2023-07-18T03:33:49.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.133+0000 7fc1aeff5700 1 -- 172.21.15.101:0/3471385721 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 49) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc1c0005040 con 0x557c627edbe0 2023-07-18T03:33:49.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.134+0000 7fc1e4bb4b80 10 client.29246 fetch_fsmap finished waiting for FSMap version 49 2023-07-18T03:33:49.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.134+0000 7fc1e4bb4b80 10 client.29246 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:33:49.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.134+0000 7fc1e4bb4b80 20 client.29246 populate_metadata read hostname 'smithi101' 2023-07-18T03:33:49.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.134+0000 7fc1e4bb4b80 1 --2- 172.21.15.101:0/3471385721 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557c62808260 0x557c6280a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:49.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.134+0000 7fc1e4bb4b80 4 client.29246 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:33:49.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.134+0000 7fc1e4bb4b80 1 -- 172.21.15.101:0/3471385721 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c62738eb0 con 0x557c62808260 2023-07-18T03:33:49.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.134+0000 7fc1cad9d700 1 --2- 172.21.15.101:0/3471385721 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557c62808260 0x557c6280a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:49.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.134+0000 7fc1aeff5700 10 client.29246 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:33:49.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.134+0000 7fc1cad9d700 1 --2- 172.21.15.101:0/3471385721 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557c62808260 0x557c6280a670 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d34b189e8151a28a server_cookie=5d2f81d23f2d702 in_seq=0 out_seq=0 2023-07-18T03:33:49.156 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.135+0000 7fc1aeff5700 1 -- 172.21.15.101:0/3471385721 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557c62738eb0 con 0x557c62808260 2023-07-18T03:33:49.156 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.135+0000 7fc1aeff5700 10 client.29246 handle_command_reply: tid=0 2023-07-18T03:33:49.156 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.135+0000 7fc1e4bb4b80 1 client.29246 shutdown 2023-07-18T03:33:49.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.135+0000 7fc1e4bb4b80 1 -- 172.21.15.101:0/3471385721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c627edbe0 msgr2=0x557c627f5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:49.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.135+0000 7fc1e4bb4b80 1 --2- 172.21.15.101:0/3471385721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c627edbe0 0x557c627f5720 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0004b60 tx=0x7fc1c0004b90 comp rx=0 tx=0).stop 2023-07-18T03:33:49.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.135+0000 7fc1e4bb4b80 1 -- 172.21.15.101:0/3471385721 shutdown_connections 2023-07-18T03:33:49.157 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.135+0000 7fc1e4bb4b80 1 --2- 172.21.15.101:0/3471385721 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557c62808260 0x557c6280a670 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:49.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.135+0000 7fc1e4bb4b80 1 --2- 172.21.15.101:0/3471385721 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c627f8ad0 0x557c627f5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:49.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.135+0000 7fc1e4bb4b80 1 --2- 172.21.15.101:0/3471385721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c627ed2d0 0x557c627f8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:49.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.135+0000 7fc1e4bb4b80 1 --2- 172.21.15.101:0/3471385721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c627edbe0 0x557c627f5720 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:49.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.135+0000 7fc1aeff5700 0 client.29246 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:33:49.158 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.135+0000 7fc1e4bb4b80 1 -- 172.21.15.101:0/3471385721 >> 172.21.15.101:0/3471385721 conn(0x557c627e3da0 msgr2=0x557c627e9230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:49.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.135+0000 7fc1e4bb4b80 1 -- 172.21.15.101:0/3471385721 shutdown_connections 2023-07-18T03:33:49.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.135+0000 7fc1e4bb4b80 1 -- 172.21.15.101:0/3471385721 wait complete. 2023-07-18T03:33:49.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.136+0000 7fc1e4bb4b80 20 client.29246 trim_cache size 0 max 16384 2023-07-18T03:33:49.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.153+0000 7fc1e4bb4b80 1 Processor -- start 2023-07-18T03:33:49.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.153+0000 7fc1e4bb4b80 1 -- start start 2023-07-18T03:33:49.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.153+0000 7fc1e4bb4b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c627ed2d0 0x557c627f8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:49.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.153+0000 7fc1e4bb4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c627edbe0 0x557c627f4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:49.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.153+0000 7fc1e4bb4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c627f8ad0 0x557c627f5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:49.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.153+0000 7fc1e4bb4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c627f6460 con 0x557c627edbe0 2023-07-18T03:33:49.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.153+0000 7fc1e4bb4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c627f5a40 con 0x557c627f8ad0 2023-07-18T03:33:49.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.153+0000 7fc1e4bb4b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c627e81d0 con 0x557c627ed2d0 2023-07-18T03:33:49.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.153+0000 7fc1cad9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c627ed2d0 0x557c627f8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:49.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.153+0000 7fc1ca59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c627edbe0 0x557c627f4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:49.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.153+0000 7fc1cad9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c627ed2d0 0x557c627f8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:46946/0 (socket says 172.21.15.101:46946) 2023-07-18T03:33:49.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.153+0000 7fc1cb59e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c627f8ad0 0x557c627f5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:49.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.153+0000 7fc1cad9d700 1 -- 172.21.15.101:0/1863552346 learned_addr learned my addr 172.21.15.101:0/1863552346 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:33:49.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.154+0000 7fc1cb59e700 1 -- 172.21.15.101:0/1863552346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c627ed2d0 msgr2=0x557c627f8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:49.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.154+0000 7fc1cb59e700 1 --2- 172.21.15.101:0/1863552346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c627ed2d0 0x557c627f8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:49.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.154+0000 7fc1cb59e700 1 -- 172.21.15.101:0/1863552346 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c627edbe0 msgr2=0x557c627f4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:49.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.154+0000 7fc1cb59e700 1 --2- 172.21.15.101:0/1863552346 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c627edbe0 0x557c627f4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:49.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.154+0000 7fc1cb59e700 1 -- 172.21.15.101:0/1863552346 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c62594250 con 0x557c627f8ad0 2023-07-18T03:33:49.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.154+0000 7fc1ca59c700 1 --2- 172.21.15.101:0/1863552346 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c627edbe0 0x557c627f4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:33:49.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.154+0000 7fc1cad9d700 1 --2- 172.21.15.101:0/1863552346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c627ed2d0 0x557c627f8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:33:49.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.154+0000 7fc1cb59e700 1 --2- 172.21.15.101:0/1863552346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c627f8ad0 0x557c627f5500 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc1c40043d0 tx=0x7fc1c4017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1287de8de22953b5 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:49.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.154+0000 7fc1b1ffb700 1 -- 172.21.15.101:0/1863552346 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c4007da0 con 0x557c627f8ad0 2023-07-18T03:33:49.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.154+0000 7fc1b1ffb700 1 -- 172.21.15.101:0/1863552346 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1c400f040 con 0x557c627f8ad0 2023-07-18T03:33:49.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.154+0000 7fc1b1ffb700 1 -- 172.21.15.101:0/1863552346 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c4017840 con 0x557c627f8ad0 2023-07-18T03:33:49.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.155+0000 7fc1e4bb4b80 1 -- 172.21.15.101:0/1863552346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c627f8ad0 msgr2=0x557c627f5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:49.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.155+0000 7fc1e4bb4b80 1 --2- 172.21.15.101:0/1863552346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c627f8ad0 0x557c627f5500 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc1c40043d0 tx=0x7fc1c4017000 comp rx=0 tx=0).stop 2023-07-18T03:33:49.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.155+0000 7fc1e4bb4b80 1 -- 172.21.15.101:0/1863552346 shutdown_connections 2023-07-18T03:33:49.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.155+0000 7fc1e4bb4b80 1 --2- 172.21.15.101:0/1863552346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c627f8ad0 0x557c627f5500 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:49.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.155+0000 7fc1e4bb4b80 1 --2- 172.21.15.101:0/1863552346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c627ed2d0 0x557c627f8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:49.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.155+0000 7fc1e4bb4b80 1 --2- 172.21.15.101:0/1863552346 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c627edbe0 0x557c627f4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:49.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.155+0000 7fc1e4bb4b80 1 -- 172.21.15.101:0/1863552346 >> 172.21.15.101:0/1863552346 conn(0x557c627e3da0 msgr2=0x557c62804700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:49.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.155+0000 7fc1e4bb4b80 1 -- 172.21.15.101:0/1863552346 shutdown_connections 2023-07-18T03:33:49.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.155+0000 7fc1e4bb4b80 1 -- 172.21.15.101:0/1863552346 wait complete. 2023-07-18T03:33:49.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.155+0000 7fc1e4bb4b80 1 -- start start 2023-07-18T03:33:49.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.155+0000 7fc1e4bb4b80 1 Processor -- start 2023-07-18T03:33:49.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.156+0000 7fc1e4bb4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c627ed2d0 0x557c627ff310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:49.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.156+0000 7fc1e4bb4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c627edbe0 0x557c627fd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:49.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.156+0000 7fc1e4bb4b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c627f8ad0 0x557c627fdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:49.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.156+0000 7fc1e4bb4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c627fe0c0 con 0x557c627ed2d0 2023-07-18T03:33:49.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.156+0000 7fc1e4bb4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c627fe230 con 0x557c627edbe0 2023-07-18T03:33:49.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.156+0000 7fc1e4bb4b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c627fe3a0 con 0x557c627f8ad0 2023-07-18T03:33:49.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.156+0000 7fc1e4bb4b80 10 client.0 _ll_register_callbacks cb 0x7fc1bc0972f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:33:49.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.156+0000 7fc1ca59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c627edbe0 0x557c627fd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:49.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.156+0000 7fc1ca59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c627edbe0 0x557c627fd640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:39754/0 (socket says 172.21.15.101:39754) 2023-07-18T03:33:49.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.156+0000 7fc1ca59c700 1 -- 172.21.15.101:0/2490819823 learned_addr learned my addr 172.21.15.101:0/2490819823 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:33:49.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.156+0000 7fc1cb59e700 1 --2- 172.21.15.101:0/2490819823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c627f8ad0 0x557c627fdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:49.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.156+0000 7fc1cad9d700 1 --2- 172.21.15.101:0/2490819823 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c627ed2d0 0x557c627ff310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:49.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.156+0000 7fc1ca59c700 1 -- 172.21.15.101:0/2490819823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c627f8ad0 msgr2=0x557c627fdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:49.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.156+0000 7fc1ca59c700 1 --2- 172.21.15.101:0/2490819823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c627f8ad0 0x557c627fdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:49.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.156+0000 7fc1ca59c700 1 -- 172.21.15.101:0/2490819823 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c627ed2d0 msgr2=0x557c627ff310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:49.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.156+0000 7fc1ca59c700 1 --2- 172.21.15.101:0/2490819823 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c627ed2d0 0x557c627ff310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:49.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.156+0000 7fc1ca59c700 1 -- 172.21.15.101:0/2490819823 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1cc1a4c40 con 0x557c627edbe0 2023-07-18T03:33:49.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.156+0000 7fc1cad9d700 1 --2- 172.21.15.101:0/2490819823 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c627ed2d0 0x557c627ff310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:33:49.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.157+0000 7fc1ca59c700 1 --2- 172.21.15.101:0/2490819823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c627edbe0 0x557c627fd640 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x557c627f6cd0 tx=0x7fc1c0022700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c9ba9e86cf175b4 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:49.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.157+0000 7fc1aeff5700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:33:49.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.157+0000 7fc1aeff5700 1 -- 172.21.15.101:0/2490819823 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c0011720 con 0x557c627edbe0 2023-07-18T03:33:49.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.157+0000 7fc1e4bb4b80 1 -- 172.21.15.101:0/2490819823 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc1c0004510 con 0x557c627edbe0 2023-07-18T03:33:49.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.157+0000 7fc1aeff5700 1 -- 172.21.15.101:0/2490819823 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fc1c0004960 con 0x557c627edbe0 2023-07-18T03:33:49.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.157+0000 7fc1aeff5700 1 -- 172.21.15.101:0/2490819823 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1c0011c60 con 0x557c627edbe0 2023-07-18T03:33:49.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.157+0000 7fc1e4bb4b80 10 client.29252 fetch_fsmap learned FSMap version 49 2023-07-18T03:33:49.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.157+0000 7fc1e4bb4b80 1 -- 172.21.15.101:0/2490819823 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x557c62556c60 con 0x557c627edbe0 2023-07-18T03:33:49.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.157+0000 7fc1aeff5700 1 -- 172.21.15.101:0/2490819823 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1c0004510 con 0x557c627edbe0 2023-07-18T03:33:49.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.158+0000 7fc1aeff5700 1 -- 172.21.15.101:0/2490819823 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 49) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc1c0022b00 con 0x557c627edbe0 2023-07-18T03:33:49.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.158+0000 7fc1e4bb4b80 10 client.29252 fetch_fsmap finished waiting for FSMap version 49 2023-07-18T03:33:49.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.158+0000 7fc1e4bb4b80 10 client.29252 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:33:49.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.158+0000 7fc1e4bb4b80 20 client.29252 populate_metadata read hostname 'smithi101' 2023-07-18T03:33:49.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.158+0000 7fc1e4bb4b80 1 --2- 172.21.15.101:0/2490819823 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557c6285d200 0x557c6285f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:49.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.158+0000 7fc1e4bb4b80 4 client.29252 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:33:49.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.158+0000 7fc1e4bb4b80 1 -- 172.21.15.101:0/2490819823 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557c626fb050 con 0x557c6285d200 2023-07-18T03:33:49.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.158+0000 7fc1cad9d700 1 --2- 172.21.15.101:0/2490819823 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557c6285d200 0x557c6285f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:49.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.158+0000 7fc1aeff5700 10 client.29252 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:33:49.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.158+0000 7fc1cad9d700 1 --2- 172.21.15.101:0/2490819823 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557c6285d200 0x557c6285f610 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69bf38c7eb567687 server_cookie=20053892ed75f5fd in_seq=0 out_seq=0 2023-07-18T03:33:49.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.159+0000 7fc1aeff5700 1 -- 172.21.15.101:0/2490819823 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x557c626fb050 con 0x557c6285d200 2023-07-18T03:33:49.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.159+0000 7fc1aeff5700 10 client.29252 handle_command_reply: tid=0 2023-07-18T03:33:49.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.159+0000 7fc1e4bb4b80 1 client.29252 shutdown 2023-07-18T03:33:49.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.159+0000 7fc1e4bb4b80 1 -- 172.21.15.101:0/2490819823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c627edbe0 msgr2=0x557c627fd640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:49.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.159+0000 7fc1e4bb4b80 1 --2- 172.21.15.101:0/2490819823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c627edbe0 0x557c627fd640 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x557c627f6cd0 tx=0x7fc1c0022700 comp rx=0 tx=0).stop 2023-07-18T03:33:49.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.159+0000 7fc1e4bb4b80 1 -- 172.21.15.101:0/2490819823 shutdown_connections 2023-07-18T03:33:49.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.159+0000 7fc1e4bb4b80 1 --2- 172.21.15.101:0/2490819823 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557c6285d200 0x557c6285f610 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:49.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.159+0000 7fc1e4bb4b80 1 --2- 172.21.15.101:0/2490819823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c627f8ad0 0x557c627fdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:49.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.159+0000 7fc1e4bb4b80 1 --2- 172.21.15.101:0/2490819823 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c627ed2d0 0x557c627ff310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:49.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.159+0000 7fc1e4bb4b80 1 --2- 172.21.15.101:0/2490819823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c627edbe0 0x557c627fd640 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:49.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.159+0000 7fc1aeff5700 0 client.29252 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:33:49.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.159+0000 7fc1e4bb4b80 1 -- 172.21.15.101:0/2490819823 >> 172.21.15.101:0/2490819823 conn(0x557c627e3da0 msgr2=0x557c627ef2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:49.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.159+0000 7fc1e4bb4b80 1 -- 172.21.15.101:0/2490819823 shutdown_connections 2023-07-18T03:33:49.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.159+0000 7fc1e4bb4b80 1 -- 172.21.15.101:0/2490819823 wait complete. 2023-07-18T03:33:49.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.159+0000 7fc1e4bb4b80 20 client.29252 trim_cache size 0 max 16384 2023-07-18T03:33:49.187 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:33:49.187 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:33:49.188 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:33:49.188 INFO:teuthology.orchestra.run.smithi101.stdout: "0f2f19e0-0429-465c-935e-223eb09490c3": { 2023-07-18T03:33:49.188 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:33:49.188 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "0f2f19e0-0429-465c-935e-223eb09490c3", 2023-07-18T03:33:49.189 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:33:49.189 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:33:49.189 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:33:49.190 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:33:49.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.160+0000 7fc1d11ff700 1 -- 172.21.15.101:0/196244747 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc1b40a12c0 msgr2=0x7fc1b40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:49.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.160+0000 7fc1d11ff700 1 --2- 172.21.15.101:0/196244747 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc1b40a12c0 0x7fc1b40a3770 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4009fd0 tx=0x7fc1c4004ad0 comp rx=0 tx=0).stop 2023-07-18T03:33:49.192 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.160+0000 7fc1d11ff700 1 -- 172.21.15.101:0/196244747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1cc0728b0 msgr2=0x7fc1cc197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:49.192 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.160+0000 7fc1d11ff700 1 --2- 172.21.15.101:0/196244747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1cc0728b0 0x7fc1cc197e10 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc005910 tx=0x7fc1bc005720 comp rx=0 tx=0).stop 2023-07-18T03:33:49.192 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.160+0000 7fc1d11ff700 1 -- 172.21.15.101:0/196244747 shutdown_connections 2023-07-18T03:33:49.192 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.160+0000 7fc1d11ff700 1 --2- 172.21.15.101:0/196244747 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc1b40a12c0 0x7fc1b40a3770 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:49.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.160+0000 7fc1d11ff700 1 --2- 172.21.15.101:0/196244747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1cc0728b0 0x7fc1cc197e10 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:49.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.160+0000 7fc1d11ff700 1 --2- 172.21.15.101:0/196244747 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1cc10c320 0x7fc1cc19c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:49.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.160+0000 7fc1d11ff700 1 --2- 172.21.15.101:0/196244747 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc1cc0731c0 0x7fc1cc198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:49.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.160+0000 7fc1d11ff700 1 -- 172.21.15.101:0/196244747 >> 172.21.15.101:0/196244747 conn(0x7fc1cc0fb5e0 msgr2=0x7fc1cc10e470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:49.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.160+0000 7fc1d11ff700 1 -- 172.21.15.101:0/196244747 shutdown_connections 2023-07-18T03:33:49.194 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:33:49.160+0000 7fc1d11ff700 1 -- 172.21.15.101:0/196244747 wait complete. 2023-07-18T03:33:49.194 INFO:tasks.cephfs.filesystem:scrub status for tag:0f2f19e0-0429-465c-935e-223eb09490c3 - {'path': '/', 'tag': '0f2f19e0-0429-465c-935e-223eb09490c3', 'options': 'recursive,force'} 2023-07-18T03:33:49.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:33:49 smithi179 ceph-mon[129229]: pgmap v830: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:33:49.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:33:49 smithi101 ceph-mon[126379]: pgmap v830: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:33:49.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:33:49 smithi155 ceph-mon[128885]: pgmap v830: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:33:51.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:33:51 smithi179 ceph-mon[129229]: pgmap v831: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:33:51.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:33:51 smithi101 ceph-mon[126379]: pgmap v831: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:33:51.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:33:51 smithi155 ceph-mon[128885]: pgmap v831: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:33:53.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:33:53 smithi179 ceph-mon[129229]: pgmap v832: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:33:53.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:33:53 smithi101 ceph-mon[126379]: pgmap v832: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:33:53.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:33:53 smithi155 ceph-mon[128885]: pgmap v832: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:33:55.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:33:55 smithi179 ceph-mon[129229]: pgmap v833: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:33:55.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:33:55 smithi101 ceph-mon[126379]: pgmap v833: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:33:55.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:33:55 smithi155 ceph-mon[128885]: pgmap v833: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:33:57.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:33:57 smithi179 ceph-mon[129229]: pgmap v834: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:33:57.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:33:57 smithi101 ceph-mon[126379]: pgmap v834: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:33:57.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:33:57 smithi155 ceph-mon[128885]: pgmap v834: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:33:59.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:33:59 smithi179 ceph-mon[129229]: pgmap v835: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:33:59.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:33:59 smithi101 ceph-mon[126379]: pgmap v835: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:33:59.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:33:59 smithi155 ceph-mon[128885]: pgmap v835: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:34:01.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:34:01 smithi179 ceph-mon[129229]: pgmap v836: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:34:01.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:34:01 smithi101 ceph-mon[126379]: pgmap v836: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:34:01.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:34:01 smithi155 ceph-mon[128885]: pgmap v836: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:34:02.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:34:02 smithi155 ceph-mon[128885]: pgmap v837: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-07-18T03:34:02.954 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:34:02 smithi179 ceph-mon[129229]: pgmap v837: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-07-18T03:34:02.991 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:34:02 smithi101 ceph-mon[126379]: pgmap v837: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-07-18T03:34:05.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:34:05 smithi179 ceph-mon[129229]: pgmap v838: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-07-18T03:34:05.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:34:05 smithi101 ceph-mon[126379]: pgmap v838: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-07-18T03:34:05.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:34:05 smithi155 ceph-mon[128885]: pgmap v838: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-07-18T03:34:07.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:34:07 smithi179 ceph-mon[129229]: pgmap v839: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-07-18T03:34:07.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:34:07 smithi101 ceph-mon[126379]: pgmap v839: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-07-18T03:34:07.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:34:07 smithi155 ceph-mon[128885]: pgmap v839: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-07-18T03:34:09.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:34:09 smithi179 ceph-mon[129229]: pgmap v840: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-07-18T03:34:09.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:34:09 smithi101 ceph-mon[126379]: pgmap v840: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-07-18T03:34:09.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:34:09 smithi155 ceph-mon[128885]: pgmap v840: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-07-18T03:34:11.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:34:11 smithi179 ceph-mon[129229]: pgmap v841: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-07-18T03:34:11.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:34:11 smithi179 ceph-mon[129229]: osdmap e133: 12 total, 12 up, 12 in 2023-07-18T03:34:11.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:34:11 smithi101 ceph-mon[126379]: pgmap v841: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-07-18T03:34:11.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:34:11 smithi101 ceph-mon[126379]: osdmap e133: 12 total, 12 up, 12 in 2023-07-18T03:34:11.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:34:11 smithi155 ceph-mon[128885]: pgmap v841: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-07-18T03:34:11.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:34:11 smithi155 ceph-mon[128885]: osdmap e133: 12 total, 12 up, 12 in 2023-07-18T03:34:13.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:34:13 smithi179 ceph-mon[129229]: pgmap v843: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 KiB/s wr, 21 op/s 2023-07-18T03:34:13.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:34:13 smithi101 ceph-mon[126379]: pgmap v843: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 KiB/s wr, 21 op/s 2023-07-18T03:34:13.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:34:13 smithi155 ceph-mon[128885]: pgmap v843: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 KiB/s wr, 21 op/s 2023-07-18T03:34:15.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:34:15 smithi179 ceph-mon[129229]: pgmap v844: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 KiB/s wr, 21 op/s 2023-07-18T03:34:15.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:34:15 smithi101 ceph-mon[126379]: pgmap v844: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 KiB/s wr, 21 op/s 2023-07-18T03:34:15.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:34:15 smithi155 ceph-mon[128885]: pgmap v844: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 KiB/s wr, 21 op/s 2023-07-18T03:34:17.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:34:17 smithi179 ceph-mon[129229]: pgmap v845: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:34:17.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:34:17 smithi101 ceph-mon[126379]: pgmap v845: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:34:17.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:34:17 smithi155 ceph-mon[128885]: pgmap v845: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:34:19.195 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:34:19.410 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.413+0000 7fda6cdff700 1 -- 172.21.15.101:0/4260610877 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda68105af0 msgr2=0x7fda68105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:19.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.413+0000 7fda6cdff700 1 --2- 172.21.15.101:0/4260610877 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda68105af0 0x7fda68105ec0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fda58009990 tx=0x7fda58009ca0 comp rx=0 tx=0).stop 2023-07-18T03:34:19.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.413+0000 7fda6cdff700 1 -- 172.21.15.101:0/4260610877 shutdown_connections 2023-07-18T03:34:19.411 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.413+0000 7fda6cdff700 1 --2- 172.21.15.101:0/4260610877 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda68100490 0x7fda6810c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:19.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.413+0000 7fda6cdff700 1 --2- 172.21.15.101:0/4260610877 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda68105af0 0x7fda68105ec0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:19.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.413+0000 7fda6cdff700 1 --2- 172.21.15.101:0/4260610877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda680ffae0 0x7fda680fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:19.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.413+0000 7fda6cdff700 1 -- 172.21.15.101:0/4260610877 >> 172.21.15.101:0/4260610877 conn(0x7fda680fb5e0 msgr2=0x7fda680fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:19.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.413+0000 7fda6cdff700 1 -- 172.21.15.101:0/4260610877 shutdown_connections 2023-07-18T03:34:19.412 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.413+0000 7fda6cdff700 1 -- 172.21.15.101:0/4260610877 wait complete. 2023-07-18T03:34:19.414 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.414+0000 7fda6cdff700 1 Processor -- start 2023-07-18T03:34:19.414 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.414+0000 7fda6cdff700 1 -- start start 2023-07-18T03:34:19.414 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.414+0000 7fda6cdff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda680ffae0 0x7fda68072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:19.415 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.414+0000 7fda6cdff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda68100490 0x7fda68072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:19.415 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.414+0000 7fda6cdff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda68105af0 0x7fda681a3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:19.415 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.414+0000 7fda6cdff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda68073580 con 0x7fda68105af0 2023-07-18T03:34:19.415 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.414+0000 7fda6cdff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda681a42f0 con 0x7fda680ffae0 2023-07-18T03:34:19.415 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.414+0000 7fda6cdff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda681a4430 con 0x7fda68100490 2023-07-18T03:34:19.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.415+0000 7fda6659c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda680ffae0 0x7fda68072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:19.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.415+0000 7fda6659c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda680ffae0 0x7fda68072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:59132/0 (socket says 172.21.15.101:59132) 2023-07-18T03:34:19.416 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.415+0000 7fda5ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda68100490 0x7fda68072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:19.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.415+0000 7fda6659c700 1 -- 172.21.15.101:0/2521169056 learned_addr learned my addr 172.21.15.101:0/2521169056 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:34:19.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.415+0000 7fda66d9d700 1 --2- 172.21.15.101:0/2521169056 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda68105af0 0x7fda681a3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:19.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.415+0000 7fda6659c700 1 -- 172.21.15.101:0/2521169056 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda68100490 msgr2=0x7fda68072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:19.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.415+0000 7fda6659c700 1 --2- 172.21.15.101:0/2521169056 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda68100490 0x7fda68072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:19.417 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.415+0000 7fda6659c700 1 -- 172.21.15.101:0/2521169056 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda68105af0 msgr2=0x7fda681a3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:19.418 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.415+0000 7fda6659c700 1 --2- 172.21.15.101:0/2521169056 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda68105af0 0x7fda681a3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:19.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.415+0000 7fda6659c700 1 -- 172.21.15.101:0/2521169056 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda580095f0 con 0x7fda680ffae0 2023-07-18T03:34:19.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.415+0000 7fda6659c700 1 --2- 172.21.15.101:0/2521169056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda680ffae0 0x7fda68072810 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fda58005910 tx=0x7fda58005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:19.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.415+0000 7fda5f7fe700 1 -- 172.21.15.101:0/2521169056 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5801d070 con 0x7fda680ffae0 2023-07-18T03:34:19.419 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.416+0000 7fda5f7fe700 1 -- 172.21.15.101:0/2521169056 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda58022470 con 0x7fda680ffae0 2023-07-18T03:34:19.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.416+0000 7fda5f7fe700 1 -- 172.21.15.101:0/2521169056 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda58017650 con 0x7fda680ffae0 2023-07-18T03:34:19.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.416+0000 7fda6cdff700 1 -- 172.21.15.101:0/2521169056 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda681a4630 con 0x7fda680ffae0 2023-07-18T03:34:19.420 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.416+0000 7fda6cdff700 1 -- 172.21.15.101:0/2521169056 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda681a4a00 con 0x7fda680ffae0 2023-07-18T03:34:19.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.420+0000 7fda5f7fe700 1 -- 172.21.15.101:0/2521169056 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fda580225e0 con 0x7fda680ffae0 2023-07-18T03:34:19.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.421+0000 7fda80811b80 1 Processor -- start 2023-07-18T03:34:19.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.421+0000 7fda80811b80 1 -- start start 2023-07-18T03:34:19.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.421+0000 7fda5f7fe700 1 --2- 172.21.15.101:0/2521169056 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fda540a1340 0x7fda540a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:19.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.422+0000 7fda80811b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561af566f2d0 0x561af566f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:19.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.422+0000 7fda80811b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af566fbe0 0x561af567a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:19.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.422+0000 7fda5f7fe700 1 -- 172.21.15.101:0/2521169056 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fda580ce380 con 0x7fda680ffae0 2023-07-18T03:34:19.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.422+0000 7fda80811b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561af567aad0 0x561af567ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:19.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.422+0000 7fda80811b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af5575cb0 con 0x561af566f2d0 2023-07-18T03:34:19.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.422+0000 7fda80811b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af55c8a40 con 0x561af566fbe0 2023-07-18T03:34:19.431 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.422+0000 7fda80811b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af55c1ed0 con 0x561af567aad0 2023-07-18T03:34:19.432 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.422+0000 7fda5ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af566fbe0 0x561af567a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:19.432 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.422+0000 7fda66d9d700 1 --2- 172.21.15.101:0/2521169056 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fda540a1340 0x7fda540a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:19.432 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.422+0000 7fda5ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af566fbe0 0x561af567a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:59144/0 (socket says 172.21.15.101:59144) 2023-07-18T03:34:19.432 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.422+0000 7fda5ffff700 1 -- 172.21.15.101:0/1881078027 learned_addr learned my addr 172.21.15.101:0/1881078027 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:34:19.432 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.422+0000 7fda66d9d700 1 --2- 172.21.15.101:0/1881078027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561af567aad0 0x561af567ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:19.433 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.422+0000 7fda6659c700 1 --2- 172.21.15.101:0/1881078027 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561af566f2d0 0x561af566f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:19.433 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.422+0000 7fda5ffff700 1 -- 172.21.15.101:0/1881078027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561af567aad0 msgr2=0x561af567ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:19.433 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.422+0000 7fda5ffff700 1 --2- 172.21.15.101:0/1881078027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561af567aad0 0x561af567ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:19.433 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.422+0000 7fda5ffff700 1 -- 172.21.15.101:0/1881078027 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561af566f2d0 msgr2=0x561af566f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:19.433 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.422+0000 7fda5ffff700 1 --2- 172.21.15.101:0/1881078027 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561af566f2d0 0x561af566f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:19.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.422+0000 7fda5ffff700 1 -- 172.21.15.101:0/1881078027 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561af53efde0 con 0x561af566fbe0 2023-07-18T03:34:19.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.423+0000 7fda5ffff700 1 --2- 172.21.15.101:0/1881078027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af566fbe0 0x561af567a590 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fda50009d20 tx=0x7fda50011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db24de3c3664a444 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:19.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.423+0000 7fda66d9d700 1 --2- 172.21.15.101:0/2521169056 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fda540a1340 0x7fda540a37f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fda60009ec0 tx=0x7fda60009920 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:19.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.423+0000 7fda5cff9700 1 -- 172.21.15.101:0/1881078027 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda50019630 con 0x561af566fbe0 2023-07-18T03:34:19.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.423+0000 7fda5cff9700 1 -- 172.21.15.101:0/1881078027 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda50019c50 con 0x561af566fbe0 2023-07-18T03:34:19.435 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.423+0000 7fda5cff9700 1 -- 172.21.15.101:0/1881078027 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5000ec40 con 0x561af566fbe0 2023-07-18T03:34:19.435 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.423+0000 7fda80811b80 1 -- 172.21.15.101:0/1881078027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af566fbe0 msgr2=0x561af567a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:19.435 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.423+0000 7fda80811b80 1 --2- 172.21.15.101:0/1881078027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af566fbe0 0x561af567a590 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fda50009d20 tx=0x7fda50011720 comp rx=0 tx=0).stop 2023-07-18T03:34:19.435 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.424+0000 7fda80811b80 1 -- 172.21.15.101:0/1881078027 shutdown_connections 2023-07-18T03:34:19.435 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.424+0000 7fda80811b80 1 --2- 172.21.15.101:0/1881078027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561af567aad0 0x561af567ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:19.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.424+0000 7fda80811b80 1 --2- 172.21.15.101:0/1881078027 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561af566f2d0 0x561af566f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:19.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.424+0000 7fda80811b80 1 --2- 172.21.15.101:0/1881078027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af566fbe0 0x561af567a590 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:19.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.424+0000 7fda80811b80 1 -- 172.21.15.101:0/1881078027 >> 172.21.15.101:0/1881078027 conn(0x561af5665da0 msgr2=0x561af566b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:19.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.424+0000 7fda80811b80 1 -- 172.21.15.101:0/1881078027 shutdown_connections 2023-07-18T03:34:19.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.424+0000 7fda80811b80 1 -- 172.21.15.101:0/1881078027 wait complete. 2023-07-18T03:34:19.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.424+0000 7fda80811b80 1 -- start start 2023-07-18T03:34:19.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.424+0000 7fda80811b80 1 Processor -- start 2023-07-18T03:34:19.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.425+0000 7fda80811b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561af566f2d0 0x561af567a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:19.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.425+0000 7fda80811b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af566fbe0 0x561af5677720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:19.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.425+0000 7fda80811b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561af567aad0 0x561af5677d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:19.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.425+0000 7fda80811b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af5561ac0 con 0x561af567aad0 2023-07-18T03:34:19.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.425+0000 7fda80811b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af55617c0 con 0x561af566fbe0 2023-07-18T03:34:19.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.425+0000 7fda80811b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af5445920 con 0x561af566f2d0 2023-07-18T03:34:19.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.425+0000 7fda80811b80 10 client.0 _ll_register_callbacks cb 0x561af555f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:34:19.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.425+0000 7fda6659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561af566f2d0 0x561af567a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:19.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.425+0000 7fda5ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af566fbe0 0x561af5677720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:19.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.425+0000 7fda6659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561af566f2d0 0x561af567a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:54676/0 (socket says 172.21.15.101:54676) 2023-07-18T03:34:19.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.425+0000 7fda5ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af566fbe0 0x561af5677720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:59152/0 (socket says 172.21.15.101:59152) 2023-07-18T03:34:19.440 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.425+0000 7fda6659c700 1 -- 172.21.15.101:0/2161697261 learned_addr learned my addr 172.21.15.101:0/2161697261 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:34:19.440 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.425+0000 7fda66d9d700 1 --2- 172.21.15.101:0/2161697261 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561af567aad0 0x561af5677d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:19.440 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.425+0000 7fda6659c700 1 -- 172.21.15.101:0/2161697261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af566fbe0 msgr2=0x561af5677720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:19.440 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.425+0000 7fda6659c700 1 --2- 172.21.15.101:0/2161697261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af566fbe0 0x561af5677720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:19.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.425+0000 7fda6659c700 1 -- 172.21.15.101:0/2161697261 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561af567aad0 msgr2=0x561af5677d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:19.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.425+0000 7fda6659c700 1 --2- 172.21.15.101:0/2161697261 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561af567aad0 0x561af5677d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:19.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.426+0000 7fda6659c700 1 -- 172.21.15.101:0/2161697261 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda50009980 con 0x561af566f2d0 2023-07-18T03:34:19.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.426+0000 7fda6659c700 1 --2- 172.21.15.101:0/2161697261 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561af566f2d0 0x561af567a4a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fda5809add0 tx=0x7fda5809ac90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b1f8f1e3d5dc0db server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:19.442 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.426+0000 7fda4dffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:34:19.442 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.426+0000 7fda4dffb700 1 -- 172.21.15.101:0/2161697261 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5809b0d0 con 0x561af566f2d0 2023-07-18T03:34:19.442 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.426+0000 7fda80811b80 1 -- 172.21.15.101:0/2161697261 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561af551b4f0 con 0x561af566f2d0 2023-07-18T03:34:19.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.426+0000 7fda4dffb700 1 -- 172.21.15.101:0/2161697261 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(134..134 src has 1..134) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fda5809f100 con 0x561af566f2d0 2023-07-18T03:34:19.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.426+0000 7fda4dffb700 1 -- 172.21.15.101:0/2161697261 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda580a3490 con 0x561af566f2d0 2023-07-18T03:34:19.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.426+0000 7fda80811b80 10 client.29254 fetch_fsmap learned FSMap version 49 2023-07-18T03:34:19.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.426+0000 7fda80811b80 1 -- 172.21.15.101:0/2161697261 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x561af53d8c60 con 0x561af566f2d0 2023-07-18T03:34:19.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.426+0000 7fda4dffb700 1 -- 172.21.15.101:0/2161697261 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561af551b4f0 con 0x561af566f2d0 2023-07-18T03:34:19.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.426+0000 7fda4dffb700 1 -- 172.21.15.101:0/2161697261 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 49) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fda580a3000 con 0x561af566f2d0 2023-07-18T03:34:19.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.427+0000 7fda80811b80 10 client.29254 fetch_fsmap finished waiting for FSMap version 49 2023-07-18T03:34:19.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.427+0000 7fda80811b80 10 client.29254 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:34:19.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.427+0000 7fda80811b80 20 client.29254 populate_metadata read hostname 'smithi101' 2023-07-18T03:34:19.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.427+0000 7fda80811b80 1 --2- 172.21.15.101:0/2161697261 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x561af5689fe0 0x561af568c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:19.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.427+0000 7fda80811b80 4 client.29254 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:34:19.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.427+0000 7fda80811b80 1 -- 172.21.15.101:0/2161697261 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561af55baeb0 con 0x561af5689fe0 2023-07-18T03:34:19.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.427+0000 7fda5ffff700 1 --2- 172.21.15.101:0/2161697261 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x561af5689fe0 0x561af568c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:19.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.427+0000 7fda5ffff700 1 --2- 172.21.15.101:0/2161697261 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x561af5689fe0 0x561af568c3f0 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29963714e39c5907 server_cookie=80126c2672ad03e5 in_seq=0 out_seq=0 2023-07-18T03:34:19.446 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.427+0000 7fda4dffb700 10 client.29254 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:34:19.446 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.428+0000 7fda4dffb700 1 -- 172.21.15.101:0/2161697261 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561af55baeb0 con 0x561af5689fe0 2023-07-18T03:34:19.446 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.428+0000 7fda4dffb700 10 client.29254 handle_command_reply: tid=0 2023-07-18T03:34:19.446 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.428+0000 7fda80811b80 1 client.29254 shutdown 2023-07-18T03:34:19.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.428+0000 7fda80811b80 1 -- 172.21.15.101:0/2161697261 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561af566f2d0 msgr2=0x561af567a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:19.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.428+0000 7fda80811b80 1 --2- 172.21.15.101:0/2161697261 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561af566f2d0 0x561af567a4a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fda5809add0 tx=0x7fda5809ac90 comp rx=0 tx=0).stop 2023-07-18T03:34:19.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.428+0000 7fda80811b80 1 -- 172.21.15.101:0/2161697261 shutdown_connections 2023-07-18T03:34:19.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.428+0000 7fda80811b80 1 --2- 172.21.15.101:0/2161697261 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x561af5689fe0 0x561af568c3f0 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:19.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.428+0000 7fda80811b80 1 --2- 172.21.15.101:0/2161697261 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561af567aad0 0x561af5677d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:19.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.428+0000 7fda4dffb700 0 client.29254 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:34:19.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.428+0000 7fda80811b80 1 --2- 172.21.15.101:0/2161697261 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561af566f2d0 0x561af567a4a0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:19.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.428+0000 7fda80811b80 1 --2- 172.21.15.101:0/2161697261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af566fbe0 0x561af5677720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:19.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.428+0000 7fda80811b80 1 -- 172.21.15.101:0/2161697261 >> 172.21.15.101:0/2161697261 conn(0x561af5665da0 msgr2=0x561af566b290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:19.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.428+0000 7fda80811b80 1 -- 172.21.15.101:0/2161697261 shutdown_connections 2023-07-18T03:34:19.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.428+0000 7fda80811b80 1 -- 172.21.15.101:0/2161697261 wait complete. 2023-07-18T03:34:19.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.429+0000 7fda80811b80 20 client.29254 trim_cache size 0 max 16384 2023-07-18T03:34:19.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.446+0000 7fda80811b80 1 Processor -- start 2023-07-18T03:34:19.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.446+0000 7fda80811b80 1 -- start start 2023-07-18T03:34:19.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.446+0000 7fda80811b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af566f2d0 0x561af56721b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:19.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.446+0000 7fda80811b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561af566fbe0 0x561af56814e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:19.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.446+0000 7fda80811b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561af567aad0 0x561af5679fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:19.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.446+0000 7fda80811b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af5675c80 con 0x561af567aad0 2023-07-18T03:34:19.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.446+0000 7fda80811b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af566a490 con 0x561af566f2d0 2023-07-18T03:34:19.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.446+0000 7fda80811b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af566c2e0 con 0x561af566fbe0 2023-07-18T03:34:19.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.446+0000 7fda6659c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af566f2d0 0x561af56721b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:19.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.446+0000 7fda5ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561af566fbe0 0x561af56814e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:19.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.446+0000 7fda6659c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af566f2d0 0x561af56721b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:59168/0 (socket says 172.21.15.101:59168) 2023-07-18T03:34:19.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.446+0000 7fda5ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561af566fbe0 0x561af56814e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:54688/0 (socket says 172.21.15.101:54688) 2023-07-18T03:34:19.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.446+0000 7fda6659c700 1 -- 172.21.15.101:0/207851902 learned_addr learned my addr 172.21.15.101:0/207851902 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:34:19.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.446+0000 7fda66d9d700 1 --2- 172.21.15.101:0/207851902 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561af567aad0 0x561af5679fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:19.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.447+0000 7fda6659c700 1 -- 172.21.15.101:0/207851902 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561af566fbe0 msgr2=0x561af56814e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:19.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.447+0000 7fda6659c700 1 --2- 172.21.15.101:0/207851902 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561af566fbe0 0x561af56814e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:19.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.447+0000 7fda6659c700 1 -- 172.21.15.101:0/207851902 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561af567aad0 msgr2=0x561af5679fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:19.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.447+0000 7fda6659c700 1 --2- 172.21.15.101:0/207851902 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561af567aad0 0x561af5679fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:19.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.447+0000 7fda6659c700 1 -- 172.21.15.101:0/207851902 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561af5416250 con 0x561af566f2d0 2023-07-18T03:34:19.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.447+0000 7fda66d9d700 1 --2- 172.21.15.101:0/207851902 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561af567aad0 0x561af5679fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:34:19.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.447+0000 7fda6659c700 1 --2- 172.21.15.101:0/207851902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af566f2d0 0x561af56721b0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fda580c7040 tx=0x7fda58022f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22456b036dd41e51 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:19.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.447+0000 7fda5d7fa700 1 -- 172.21.15.101:0/207851902 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5809f100 con 0x561af566f2d0 2023-07-18T03:34:19.462 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.447+0000 7fda5d7fa700 1 -- 172.21.15.101:0/207851902 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda58096ec0 con 0x561af566f2d0 2023-07-18T03:34:19.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.447+0000 7fda5d7fa700 1 -- 172.21.15.101:0/207851902 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda580a3000 con 0x561af566f2d0 2023-07-18T03:34:19.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.447+0000 7fda80811b80 1 -- 172.21.15.101:0/207851902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af566f2d0 msgr2=0x561af56721b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:19.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.448+0000 7fda80811b80 1 --2- 172.21.15.101:0/207851902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af566f2d0 0x561af56721b0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fda580c7040 tx=0x7fda58022f90 comp rx=0 tx=0).stop 2023-07-18T03:34:19.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.448+0000 7fda80811b80 1 -- 172.21.15.101:0/207851902 shutdown_connections 2023-07-18T03:34:19.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.448+0000 7fda80811b80 1 --2- 172.21.15.101:0/207851902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af566f2d0 0x561af56721b0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:19.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.448+0000 7fda80811b80 1 --2- 172.21.15.101:0/207851902 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561af567aad0 0x561af5679fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:19.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.448+0000 7fda80811b80 1 --2- 172.21.15.101:0/207851902 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561af566fbe0 0x561af56814e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:19.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.448+0000 7fda80811b80 1 -- 172.21.15.101:0/207851902 >> 172.21.15.101:0/207851902 conn(0x561af5665da0 msgr2=0x561af5672a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:19.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.448+0000 7fda80811b80 1 -- 172.21.15.101:0/207851902 shutdown_connections 2023-07-18T03:34:19.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.448+0000 7fda80811b80 1 -- 172.21.15.101:0/207851902 wait complete. 2023-07-18T03:34:19.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.448+0000 7fda80811b80 1 -- start start 2023-07-18T03:34:19.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.448+0000 7fda80811b80 1 Processor -- start 2023-07-18T03:34:19.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.449+0000 7fda80811b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561af566f2d0 0x561af56810d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:19.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.449+0000 7fda80811b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561af566fbe0 0x561af5681610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:19.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.449+0000 7fda80811b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af567aad0 0x561af567d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:19.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.449+0000 7fda80811b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af567dcc0 con 0x561af566f2d0 2023-07-18T03:34:19.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.449+0000 7fda80811b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af567de30 con 0x561af567aad0 2023-07-18T03:34:19.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.449+0000 7fda80811b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af567dff0 con 0x561af566fbe0 2023-07-18T03:34:19.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.449+0000 7fda80811b80 10 client.0 _ll_register_callbacks cb 0x7fda580974c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:34:19.471 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.449+0000 7fda5ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561af566fbe0 0x561af5681610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:19.471 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.449+0000 7fda5ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561af566fbe0 0x561af5681610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:54696/0 (socket says 172.21.15.101:54696) 2023-07-18T03:34:19.472 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.449+0000 7fda5ffff700 1 -- 172.21.15.101:0/626490907 learned_addr learned my addr 172.21.15.101:0/626490907 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:34:19.472 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.449+0000 7fda66d9d700 1 --2- 172.21.15.101:0/626490907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af567aad0 0x561af567d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:19.472 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.449+0000 7fda6659c700 1 --2- 172.21.15.101:0/626490907 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561af566f2d0 0x561af56810d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:19.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.449+0000 7fda5ffff700 1 -- 172.21.15.101:0/626490907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af567aad0 msgr2=0x561af567d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:19.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.449+0000 7fda5ffff700 1 --2- 172.21.15.101:0/626490907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af567aad0 0x561af567d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:19.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.449+0000 7fda5ffff700 1 -- 172.21.15.101:0/626490907 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561af566f2d0 msgr2=0x561af56810d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:19.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.449+0000 7fda5ffff700 1 --2- 172.21.15.101:0/626490907 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561af566f2d0 0x561af56810d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:19.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.449+0000 7fda5ffff700 1 -- 172.21.15.101:0/626490907 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561af53d8c60 con 0x561af566fbe0 2023-07-18T03:34:19.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.449+0000 7fda6659c700 1 --2- 172.21.15.101:0/626490907 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561af566f2d0 0x561af56810d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:34:19.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.449+0000 7fda66d9d700 1 --2- 172.21.15.101:0/626490907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af567aad0 0x561af567d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:34:19.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.450+0000 7fda5ffff700 1 --2- 172.21.15.101:0/626490907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561af566fbe0 0x561af5681610 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fda5001c040 tx=0x7fda5000e7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23bc9d8ad924117d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:19.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.450+0000 7fda4dffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:34:19.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.450+0000 7fda4dffb700 1 -- 172.21.15.101:0/626490907 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda50029050 con 0x561af566fbe0 2023-07-18T03:34:19.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.450+0000 7fda4dffb700 1 -- 172.21.15.101:0/626490907 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(134..134 src has 1..134) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fda5002a1b0 con 0x561af566fbe0 2023-07-18T03:34:19.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.450+0000 7fda80811b80 1 -- 172.21.15.101:0/626490907 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda50019c50 con 0x561af566fbe0 2023-07-18T03:34:19.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.450+0000 7fda4dffb700 1 -- 172.21.15.101:0/626490907 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda5002a3b0 con 0x561af566fbe0 2023-07-18T03:34:19.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.450+0000 7fda80811b80 10 client.29260 fetch_fsmap learned FSMap version 49 2023-07-18T03:34:19.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.450+0000 7fda80811b80 1 -- 172.21.15.101:0/626490907 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x561af53ee2b0 con 0x561af566fbe0 2023-07-18T03:34:19.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.450+0000 7fda4dffb700 1 -- 172.21.15.101:0/626490907 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda50019c50 con 0x561af566fbe0 2023-07-18T03:34:19.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.451+0000 7fda4dffb700 1 -- 172.21.15.101:0/626490907 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 49) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fda5002a880 con 0x561af566fbe0 2023-07-18T03:34:19.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.451+0000 7fda80811b80 10 client.29260 fetch_fsmap finished waiting for FSMap version 49 2023-07-18T03:34:19.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.451+0000 7fda80811b80 10 client.29260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:34:19.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.451+0000 7fda80811b80 20 client.29260 populate_metadata read hostname 'smithi101' 2023-07-18T03:34:19.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.451+0000 7fda80811b80 1 --2- 172.21.15.101:0/626490907 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x561af56df0f0 0x561af56e1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:19.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.451+0000 7fda80811b80 4 client.29260 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:34:19.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.451+0000 7fda80811b80 1 -- 172.21.15.101:0/626490907 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561af557d050 con 0x561af56df0f0 2023-07-18T03:34:19.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.451+0000 7fda6659c700 1 --2- 172.21.15.101:0/626490907 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x561af56df0f0 0x561af56e1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:19.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.451+0000 7fda4dffb700 10 client.29260 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:34:19.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.451+0000 7fda6659c700 1 --2- 172.21.15.101:0/626490907 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x561af56df0f0 0x561af56e1500 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b7b48b887dbd9dc server_cookie=72b9a992ab72b9c5 in_seq=0 out_seq=0 2023-07-18T03:34:19.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.452+0000 7fda4dffb700 1 -- 172.21.15.101:0/626490907 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x561af557d050 con 0x561af56df0f0 2023-07-18T03:34:19.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.452+0000 7fda4dffb700 10 client.29260 handle_command_reply: tid=0 2023-07-18T03:34:19.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.452+0000 7fda80811b80 1 client.29260 shutdown 2023-07-18T03:34:19.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.452+0000 7fda80811b80 1 -- 172.21.15.101:0/626490907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561af566fbe0 msgr2=0x561af5681610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:19.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.452+0000 7fda80811b80 1 --2- 172.21.15.101:0/626490907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561af566fbe0 0x561af5681610 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fda5001c040 tx=0x7fda5000e7e0 comp rx=0 tx=0).stop 2023-07-18T03:34:19.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.452+0000 7fda80811b80 1 -- 172.21.15.101:0/626490907 shutdown_connections 2023-07-18T03:34:19.480 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.452+0000 7fda80811b80 1 --2- 172.21.15.101:0/626490907 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x561af56df0f0 0x561af56e1500 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:19.480 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.452+0000 7fda80811b80 1 --2- 172.21.15.101:0/626490907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af567aad0 0x561af567d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:19.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.452+0000 7fda4dffb700 0 client.29260 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:34:19.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.452+0000 7fda80811b80 1 --2- 172.21.15.101:0/626490907 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561af566f2d0 0x561af56810d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:19.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.452+0000 7fda80811b80 1 --2- 172.21.15.101:0/626490907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561af566fbe0 0x561af5681610 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:19.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.452+0000 7fda80811b80 1 -- 172.21.15.101:0/626490907 >> 172.21.15.101:0/626490907 conn(0x561af5665da0 msgr2=0x561af566a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:19.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.452+0000 7fda80811b80 1 -- 172.21.15.101:0/626490907 shutdown_connections 2023-07-18T03:34:19.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.452+0000 7fda80811b80 1 -- 172.21.15.101:0/626490907 wait complete. 2023-07-18T03:34:19.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.452+0000 7fda80811b80 20 client.29260 trim_cache size 0 max 16384 2023-07-18T03:34:19.483 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:34:19.483 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:34:19.483 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:34:19.484 INFO:teuthology.orchestra.run.smithi101.stdout: "0f2f19e0-0429-465c-935e-223eb09490c3": { 2023-07-18T03:34:19.484 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:34:19.484 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "0f2f19e0-0429-465c-935e-223eb09490c3", 2023-07-18T03:34:19.484 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:34:19.484 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:34:19.485 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:34:19.485 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:34:19.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.452+0000 7fda6cdff700 1 -- 172.21.15.101:0/2521169056 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fda540a1340 msgr2=0x7fda540a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:19.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.452+0000 7fda6cdff700 1 --2- 172.21.15.101:0/2521169056 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fda540a1340 0x7fda540a37f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fda60009ec0 tx=0x7fda60009920 comp rx=0 tx=0).stop 2023-07-18T03:34:19.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.453+0000 7fda6cdff700 1 -- 172.21.15.101:0/2521169056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda680ffae0 msgr2=0x7fda68072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:19.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.453+0000 7fda6cdff700 1 --2- 172.21.15.101:0/2521169056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda680ffae0 0x7fda68072810 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fda58005910 tx=0x7fda58005720 comp rx=0 tx=0).stop 2023-07-18T03:34:19.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.453+0000 7fda6cdff700 1 -- 172.21.15.101:0/2521169056 shutdown_connections 2023-07-18T03:34:19.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.453+0000 7fda6cdff700 1 --2- 172.21.15.101:0/2521169056 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fda540a1340 0x7fda540a37f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:19.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.453+0000 7fda6cdff700 1 --2- 172.21.15.101:0/2521169056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda680ffae0 0x7fda68072810 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:19.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.453+0000 7fda6cdff700 1 --2- 172.21.15.101:0/2521169056 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fda68105af0 0x7fda681a3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:19.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.453+0000 7fda6cdff700 1 --2- 172.21.15.101:0/2521169056 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda68100490 0x7fda68072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:19.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.453+0000 7fda6cdff700 1 -- 172.21.15.101:0/2521169056 >> 172.21.15.101:0/2521169056 conn(0x7fda680fb5e0 msgr2=0x7fda680fbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:19.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.453+0000 7fda6cdff700 1 -- 172.21.15.101:0/2521169056 shutdown_connections 2023-07-18T03:34:19.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:19.453+0000 7fda6cdff700 1 -- 172.21.15.101:0/2521169056 wait complete. 2023-07-18T03:34:19.489 INFO:tasks.cephfs.filesystem:scrub status for tag:0f2f19e0-0429-465c-935e-223eb09490c3 - {'path': '/', 'tag': '0f2f19e0-0429-465c-935e-223eb09490c3', 'options': 'recursive,force'} 2023-07-18T03:34:19.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:34:19 smithi179 ceph-mon[129229]: pgmap v846: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:34:19.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:34:19 smithi179 ceph-mon[129229]: osdmap e134: 12 total, 12 up, 12 in 2023-07-18T03:34:19.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:34:19 smithi101 ceph-mon[126379]: pgmap v846: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:34:19.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:34:19 smithi101 ceph-mon[126379]: osdmap e134: 12 total, 12 up, 12 in 2023-07-18T03:34:19.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:34:19 smithi155 ceph-mon[128885]: pgmap v846: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:34:19.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:34:19 smithi155 ceph-mon[128885]: osdmap e134: 12 total, 12 up, 12 in 2023-07-18T03:34:21.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:34:21 smithi179 ceph-mon[129229]: pgmap v848: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 210 B/s wr, 6 op/s 2023-07-18T03:34:21.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:34:21 smithi101 ceph-mon[126379]: pgmap v848: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 210 B/s wr, 6 op/s 2023-07-18T03:34:21.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:34:21 smithi155 ceph-mon[128885]: pgmap v848: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 210 B/s wr, 6 op/s 2023-07-18T03:34:23.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:34:23 smithi179 ceph-mon[129229]: pgmap v849: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:34:23.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:34:23 smithi101 ceph-mon[126379]: pgmap v849: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:34:23.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:34:23 smithi155 ceph-mon[128885]: pgmap v849: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:34:25.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:34:25 smithi179 ceph-mon[129229]: pgmap v850: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:34:25.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:34:25 smithi101 ceph-mon[126379]: pgmap v850: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:34:25.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:34:25 smithi155 ceph-mon[128885]: pgmap v850: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:34:27.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:34:27 smithi179 ceph-mon[129229]: pgmap v851: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:34:27.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:34:27 smithi101 ceph-mon[126379]: pgmap v851: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:34:27.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:34:27 smithi155 ceph-mon[128885]: pgmap v851: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:34:28.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:34:28 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:34:28.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:34:28 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:34:28.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:34:28 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:34:29.453 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:34:29 smithi179 ceph-mon[129229]: pgmap v852: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:34:29.655 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:34:29 smithi155 ceph-mon[128885]: pgmap v852: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:34:29.662 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:34:29 smithi101 ceph-mon[126379]: pgmap v852: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:34:31.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:34:31 smithi179 ceph-mon[129229]: pgmap v853: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 95 B/s wr, 8 op/s 2023-07-18T03:34:31.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:34:31 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:34:31.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:34:31 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:34:31.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:34:31 smithi101 ceph-mon[126379]: pgmap v853: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 95 B/s wr, 8 op/s 2023-07-18T03:34:31.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:34:31 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:34:31.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:34:31 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:34:31.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:34:31 smithi155 ceph-mon[128885]: pgmap v853: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 95 B/s wr, 8 op/s 2023-07-18T03:34:31.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:34:31 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:34:31.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:34:31 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:34:32.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:34:32 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:34:32.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:34:32 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:34:32.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:34:32 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:34:32.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:34:32 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:34:32.705 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:34:32 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:34:32.705 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:34:32 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:34:32.705 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:34:32 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:34:32.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:34:32 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:34:32.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:34:32 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:34:32.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:34:32 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:34:32.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:34:32 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:34:32.743 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:34:32 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:34:32.743 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:34:32 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:34:32.743 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:34:32 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:34:32.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:34:32 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:34:32.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:34:32 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:34:32.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:34:32 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:34:32.800 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:34:32 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:34:32.800 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:34:32 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:34:32.800 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:34:32 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:34:32.800 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:34:32 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:34:33.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:34:33 smithi179 ceph-mon[129229]: pgmap v854: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:34:33.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:34:33 smithi101 ceph-mon[126379]: pgmap v854: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:34:33.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:34:33 smithi155 ceph-mon[128885]: pgmap v854: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:34:35.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:34:35 smithi179 ceph-mon[129229]: pgmap v855: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:34:35.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:34:35 smithi101 ceph-mon[126379]: pgmap v855: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:34:35.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:34:35 smithi155 ceph-mon[128885]: pgmap v855: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:34:37.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:34:37 smithi179 ceph-mon[129229]: pgmap v856: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:34:37.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:34:37 smithi101 ceph-mon[126379]: pgmap v856: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:34:37.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:34:37 smithi155 ceph-mon[128885]: pgmap v856: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:34:39.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:34:39 smithi179 ceph-mon[129229]: pgmap v857: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:34:39.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:34:39 smithi101 ceph-mon[126379]: pgmap v857: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:34:39.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:34:39 smithi155 ceph-mon[128885]: pgmap v857: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:34:41.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:34:41 smithi179 ceph-mon[129229]: pgmap v858: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:34:41.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:34:41 smithi101 ceph-mon[126379]: pgmap v858: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:34:41.800 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:34:41 smithi155 ceph-mon[128885]: pgmap v858: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:34:43.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:34:43 smithi179 ceph-mon[129229]: pgmap v859: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:34:43.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:34:43 smithi101 ceph-mon[126379]: pgmap v859: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:34:43.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:34:43 smithi155 ceph-mon[128885]: pgmap v859: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:34:45.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:34:45 smithi179 ceph-mon[129229]: pgmap v860: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:34:45.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:34:45 smithi101 ceph-mon[126379]: pgmap v860: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:34:45.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:34:45 smithi155 ceph-mon[128885]: pgmap v860: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:34:47.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:34:47 smithi179 ceph-mon[129229]: pgmap v861: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:34:47.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:34:47 smithi101 ceph-mon[126379]: pgmap v861: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:34:47.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:34:47 smithi155 ceph-mon[128885]: pgmap v861: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:34:49.490 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:34:49.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:34:49 smithi179 ceph-mon[129229]: pgmap v862: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:34:49.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.708+0000 7f02c63ff700 1 -- 172.21.15.101:0/189127278 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f02c0105ae0 msgr2=0x7f02c0105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:49.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.708+0000 7f02c63ff700 1 --2- 172.21.15.101:0/189127278 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f02c0105ae0 0x7f02c0105eb0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f02b00098f0 tx=0x7f02b0009c00 comp rx=0 tx=0).stop 2023-07-18T03:34:49.705 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.708+0000 7f02c63ff700 1 -- 172.21.15.101:0/189127278 shutdown_connections 2023-07-18T03:34:49.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.708+0000 7f02c63ff700 1 --2- 172.21.15.101:0/189127278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02c0100480 0x7f02c010c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:49.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.708+0000 7f02c63ff700 1 --2- 172.21.15.101:0/189127278 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f02c0105ae0 0x7f02c0105eb0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:49.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.708+0000 7f02c63ff700 1 --2- 172.21.15.101:0/189127278 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02c00ffad0 0x7f02c00fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:49.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.708+0000 7f02c63ff700 1 -- 172.21.15.101:0/189127278 >> 172.21.15.101:0/189127278 conn(0x7f02c00fb5f0 msgr2=0x7f02c00fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:49.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.709+0000 7f02c63ff700 1 -- 172.21.15.101:0/189127278 shutdown_connections 2023-07-18T03:34:49.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.709+0000 7f02c63ff700 1 -- 172.21.15.101:0/189127278 wait complete. 2023-07-18T03:34:49.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.710+0000 7f02c63ff700 1 Processor -- start 2023-07-18T03:34:49.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.710+0000 7f02c63ff700 1 -- start start 2023-07-18T03:34:49.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.710+0000 7f02c63ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02c00ffad0 0x7f02c0197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:49.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.710+0000 7f02c63ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02c0100480 0x7f02c0198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:49.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.710+0000 7f02c63ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f02c0105ae0 0x7f02c019c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:49.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.710+0000 7f02c63ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c019cdd0 con 0x7f02c0105ae0 2023-07-18T03:34:49.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.710+0000 7f02c63ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c019cf40 con 0x7f02c0100480 2023-07-18T03:34:49.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.710+0000 7f02c63ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c019d100 con 0x7f02c00ffad0 2023-07-18T03:34:49.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.710+0000 7f02bffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02c00ffad0 0x7f02c0197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:49.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.710+0000 7f02c499c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f02c0105ae0 0x7f02c019c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:49.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.710+0000 7f02bf7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02c0100480 0x7f02c0198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:49.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.710+0000 7f02bffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02c00ffad0 0x7f02c0197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:53844/0 (socket says 172.21.15.101:53844) 2023-07-18T03:34:49.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.710+0000 7f02bffff700 1 -- 172.21.15.101:0/61443213 learned_addr learned my addr 172.21.15.101:0/61443213 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:34:49.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.711+0000 7f02bffff700 1 -- 172.21.15.101:0/61443213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02c0100480 msgr2=0x7f02c0198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:49.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.711+0000 7f02bffff700 1 --2- 172.21.15.101:0/61443213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02c0100480 0x7f02c0198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:49.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.711+0000 7f02bffff700 1 -- 172.21.15.101:0/61443213 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f02c0105ae0 msgr2=0x7f02c019c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:49.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.711+0000 7f02bffff700 1 --2- 172.21.15.101:0/61443213 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f02c0105ae0 0x7f02c019c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:49.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.711+0000 7f02bffff700 1 -- 172.21.15.101:0/61443213 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b0009520 con 0x7f02c00ffad0 2023-07-18T03:34:49.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.711+0000 7f02bffff700 1 --2- 172.21.15.101:0/61443213 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02c00ffad0 0x7f02c0197df0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f02b0009230 tx=0x7f02b000fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:49.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.711+0000 7f02bd7fa700 1 -- 172.21.15.101:0/61443213 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b001c070 con 0x7f02c00ffad0 2023-07-18T03:34:49.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.711+0000 7f02bd7fa700 1 -- 172.21.15.101:0/61443213 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02b00058e0 con 0x7f02c00ffad0 2023-07-18T03:34:49.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.711+0000 7f02c63ff700 1 -- 172.21.15.101:0/61443213 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02c019d380 con 0x7f02c00ffad0 2023-07-18T03:34:49.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.711+0000 7f02bd7fa700 1 -- 172.21.15.101:0/61443213 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b00038e0 con 0x7f02c00ffad0 2023-07-18T03:34:49.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.711+0000 7f02c63ff700 1 -- 172.21.15.101:0/61443213 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02c01a4c20 con 0x7f02c00ffad0 2023-07-18T03:34:49.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.714+0000 7f02bd7fa700 1 -- 172.21.15.101:0/61443213 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f02b0020030 con 0x7f02c00ffad0 2023-07-18T03:34:49.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.716+0000 7f02bd7fa700 1 --2- 172.21.15.101:0/61443213 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f02a80a1340 0x7f02a80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:49.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.716+0000 7f02d9d18b80 1 Processor -- start 2023-07-18T03:34:49.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.716+0000 7f02d9d18b80 1 -- start start 2023-07-18T03:34:49.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.716+0000 7f02bd7fa700 1 -- 172.21.15.101:0/61443213 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f02b00cdeb0 con 0x7f02c00ffad0 2023-07-18T03:34:49.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.716+0000 7f02bf7fe700 1 --2- 172.21.15.101:0/61443213 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f02a80a1340 0x7f02a80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:49.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.716+0000 7f02d9d18b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c27f0d82d0 0x55c27f0d86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:49.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.716+0000 7f02d9d18b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c27f0d8be0 0x55c27f0e3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:49.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.716+0000 7f02d9d18b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c27f0e3ad0 0x55c27f0e5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:49.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.716+0000 7f02d9d18b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c27efdecb0 con 0x55c27f0d82d0 2023-07-18T03:34:49.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.716+0000 7f02d9d18b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c27f031a40 con 0x55c27f0e3ad0 2023-07-18T03:34:49.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.716+0000 7f02d9d18b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c27f02aed0 con 0x55c27f0d8be0 2023-07-18T03:34:49.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.717+0000 7f02bffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c27f0d82d0 0x55c27f0d86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:49.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.717+0000 7f02bffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c27f0d82d0 0x55c27f0d86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:34134/0 (socket says 172.21.15.101:34134) 2023-07-18T03:34:49.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.717+0000 7f02bffff700 1 -- 172.21.15.101:0/1709049553 learned_addr learned my addr 172.21.15.101:0/1709049553 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:34:49.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.717+0000 7f02bf7fe700 1 --2- 172.21.15.101:0/1709049553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c27f0d8be0 0x55c27f0e3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:49.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.717+0000 7f02c499c700 1 --2- 172.21.15.101:0/1709049553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c27f0e3ad0 0x55c27f0e5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:49.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.717+0000 7f02bf7fe700 1 --2- 172.21.15.101:0/61443213 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f02a80a1340 0x7f02a80a37f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f02c0199400 tx=0x7f02b4009260 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:49.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.717+0000 7f02bffff700 1 -- 172.21.15.101:0/1709049553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c27f0d8be0 msgr2=0x55c27f0e3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:49.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.717+0000 7f02bffff700 1 --2- 172.21.15.101:0/1709049553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c27f0d8be0 0x55c27f0e3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:49.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.717+0000 7f02bffff700 1 -- 172.21.15.101:0/1709049553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c27f0e3ad0 msgr2=0x55c27f0e5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:49.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.717+0000 7f02bffff700 1 --2- 172.21.15.101:0/1709049553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c27f0e3ad0 0x55c27f0e5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:49.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.717+0000 7f02bffff700 1 -- 172.21.15.101:0/1709049553 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c27ee58de0 con 0x55c27f0d82d0 2023-07-18T03:34:49.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.717+0000 7f02bffff700 1 --2- 172.21.15.101:0/1709049553 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c27f0d82d0 0x55c27f0d86a0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f02b0097670 tx=0x7f02b0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35fc4c92b61e162b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:49.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.717+0000 7f02a67fc700 1 -- 172.21.15.101:0/1709049553 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b00b2070 con 0x55c27f0d82d0 2023-07-18T03:34:49.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.717+0000 7f02a67fc700 1 -- 172.21.15.101:0/1709049553 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02b009ee90 con 0x55c27f0d82d0 2023-07-18T03:34:49.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.717+0000 7f02a67fc700 1 -- 172.21.15.101:0/1709049553 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b00cb470 con 0x55c27f0d82d0 2023-07-18T03:34:49.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.718+0000 7f02d9d18b80 1 -- 172.21.15.101:0/1709049553 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c27f0d82d0 msgr2=0x55c27f0d86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:49.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.718+0000 7f02d9d18b80 1 --2- 172.21.15.101:0/1709049553 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c27f0d82d0 0x55c27f0d86a0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f02b0097670 tx=0x7f02b0009260 comp rx=0 tx=0).stop 2023-07-18T03:34:49.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.718+0000 7f02d9d18b80 1 -- 172.21.15.101:0/1709049553 shutdown_connections 2023-07-18T03:34:49.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.718+0000 7f02d9d18b80 1 --2- 172.21.15.101:0/1709049553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c27f0e3ad0 0x55c27f0e5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:49.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.718+0000 7f02d9d18b80 1 --2- 172.21.15.101:0/1709049553 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c27f0d82d0 0x55c27f0d86a0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:49.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.718+0000 7f02d9d18b80 1 --2- 172.21.15.101:0/1709049553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c27f0d8be0 0x55c27f0e3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:49.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.718+0000 7f02d9d18b80 1 -- 172.21.15.101:0/1709049553 >> 172.21.15.101:0/1709049553 conn(0x55c27f0ceda0 msgr2=0x55c27f0d46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:49.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.718+0000 7f02d9d18b80 1 -- 172.21.15.101:0/1709049553 shutdown_connections 2023-07-18T03:34:49.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.718+0000 7f02d9d18b80 1 -- 172.21.15.101:0/1709049553 wait complete. 2023-07-18T03:34:49.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.719+0000 7f02d9d18b80 1 -- start start 2023-07-18T03:34:49.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.719+0000 7f02d9d18b80 1 Processor -- start 2023-07-18T03:34:49.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.719+0000 7f02d9d18b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c27f0d82d0 0x55c27f0e8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:49.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.719+0000 7f02d9d18b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c27f0d8be0 0x55c27f0e9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:49.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.719+0000 7f02d9d18b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c27f0e3ad0 0x55c27f0e2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:49.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.719+0000 7f02d9d18b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c27efcaac0 con 0x55c27f0d8be0 2023-07-18T03:34:49.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.719+0000 7f02d9d18b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c27efca7c0 con 0x55c27f0d82d0 2023-07-18T03:34:49.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.719+0000 7f02d9d18b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c27eeae920 con 0x55c27f0e3ad0 2023-07-18T03:34:49.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.719+0000 7f02d9d18b80 10 client.0 _ll_register_callbacks cb 0x55c27efc8760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:34:49.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.720+0000 7f02bf7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c27f0d8be0 0x55c27f0e9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:49.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.720+0000 7f02c499c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c27f0e3ad0 0x55c27f0e2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:49.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.720+0000 7f02bffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c27f0d82d0 0x55c27f0e8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:49.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.720+0000 7f02bffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c27f0d82d0 0x55c27f0e8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:33712/0 (socket says 172.21.15.101:33712) 2023-07-18T03:34:49.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.720+0000 7f02bf7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c27f0d8be0 0x55c27f0e9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:34144/0 (socket says 172.21.15.101:34144) 2023-07-18T03:34:49.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.720+0000 7f02c499c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c27f0e3ad0 0x55c27f0e2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:53866/0 (socket says 172.21.15.101:53866) 2023-07-18T03:34:49.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.720+0000 7f02bffff700 1 -- 172.21.15.101:0/1879105208 learned_addr learned my addr 172.21.15.101:0/1879105208 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:34:49.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.720+0000 7f02bffff700 1 -- 172.21.15.101:0/1879105208 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c27f0e3ad0 msgr2=0x55c27f0e2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:49.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.720+0000 7f02bffff700 1 --2- 172.21.15.101:0/1879105208 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c27f0e3ad0 0x55c27f0e2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:49.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.720+0000 7f02bffff700 1 -- 172.21.15.101:0/1879105208 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c27f0d8be0 msgr2=0x55c27f0e9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:49.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.720+0000 7f02bffff700 1 --2- 172.21.15.101:0/1879105208 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c27f0d8be0 0x55c27f0e9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:49.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.720+0000 7f02bffff700 1 -- 172.21.15.101:0/1879105208 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02c01a4c20 con 0x55c27f0d82d0 2023-07-18T03:34:49.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.720+0000 7f02bf7fe700 1 --2- 172.21.15.101:0/1879105208 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c27f0d8be0 0x55c27f0e9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:34:49.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.720+0000 7f02bffff700 1 --2- 172.21.15.101:0/1879105208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c27f0d82d0 0x55c27f0e8be0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f02b009adc0 tx=0x7f02b00a2fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d9bf075bb21ff43 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:49.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.720+0000 7f029f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:34:49.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.720+0000 7f029f7fe700 1 -- 172.21.15.101:0/1879105208 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b00b2030 con 0x55c27f0d82d0 2023-07-18T03:34:49.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.721+0000 7f02d9d18b80 1 -- 172.21.15.101:0/1879105208 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c27ef844f0 con 0x55c27f0d82d0 2023-07-18T03:34:49.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.721+0000 7f029f7fe700 1 -- 172.21.15.101:0/1879105208 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(134..134 src has 1..134) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f02b00b3070 con 0x55c27f0d82d0 2023-07-18T03:34:49.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.721+0000 7f029f7fe700 1 -- 172.21.15.101:0/1879105208 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02b0097310 con 0x55c27f0d82d0 2023-07-18T03:34:49.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.721+0000 7f02d9d18b80 10 client.29282 fetch_fsmap learned FSMap version 49 2023-07-18T03:34:49.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.721+0000 7f02d9d18b80 1 -- 172.21.15.101:0/1879105208 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55c27ee41c60 con 0x55c27f0d82d0 2023-07-18T03:34:49.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.721+0000 7f029f7fe700 1 -- 172.21.15.101:0/1879105208 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02b007f020 con 0x55c27f0d82d0 2023-07-18T03:34:49.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.721+0000 7f029f7fe700 1 -- 172.21.15.101:0/1879105208 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 49) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f02b009af60 con 0x55c27f0d82d0 2023-07-18T03:34:49.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.721+0000 7f02d9d18b80 10 client.29282 fetch_fsmap finished waiting for FSMap version 49 2023-07-18T03:34:49.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.721+0000 7f02d9d18b80 10 client.29282 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:34:49.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.721+0000 7f02d9d18b80 20 client.29282 populate_metadata read hostname 'smithi101' 2023-07-18T03:34:49.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.721+0000 7f02d9d18b80 1 --2- 172.21.15.101:0/1879105208 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c27f0f2c30 0x55c27f0f5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:49.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.721+0000 7f02d9d18b80 4 client.29282 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:34:49.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.721+0000 7f02d9d18b80 1 -- 172.21.15.101:0/1879105208 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c27f023eb0 con 0x55c27f0f2c30 2023-07-18T03:34:49.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.722+0000 7f02bf7fe700 1 --2- 172.21.15.101:0/1879105208 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c27f0f2c30 0x55c27f0f5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:49.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.722+0000 7f029f7fe700 10 client.29282 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:34:49.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.722+0000 7f02bf7fe700 1 --2- 172.21.15.101:0/1879105208 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c27f0f2c30 0x55c27f0f5040 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e4954fea7b32a8f server_cookie=ed83bad290b17e1d in_seq=0 out_seq=0 2023-07-18T03:34:49.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.723+0000 7f029f7fe700 1 -- 172.21.15.101:0/1879105208 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c27f023eb0 con 0x55c27f0f2c30 2023-07-18T03:34:49.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.723+0000 7f029f7fe700 10 client.29282 handle_command_reply: tid=0 2023-07-18T03:34:49.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.723+0000 7f02d9d18b80 1 client.29282 shutdown 2023-07-18T03:34:49.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.723+0000 7f02d9d18b80 1 -- 172.21.15.101:0/1879105208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c27f0d82d0 msgr2=0x55c27f0e8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:49.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.723+0000 7f02d9d18b80 1 --2- 172.21.15.101:0/1879105208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c27f0d82d0 0x55c27f0e8be0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f02b009adc0 tx=0x7f02b00a2fc0 comp rx=0 tx=0).stop 2023-07-18T03:34:49.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.723+0000 7f02d9d18b80 1 -- 172.21.15.101:0/1879105208 shutdown_connections 2023-07-18T03:34:49.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.723+0000 7f02d9d18b80 1 --2- 172.21.15.101:0/1879105208 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c27f0f2c30 0x55c27f0f5040 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:49.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.723+0000 7f02d9d18b80 1 --2- 172.21.15.101:0/1879105208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c27f0d82d0 0x55c27f0e8be0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:49.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.723+0000 7f02d9d18b80 1 --2- 172.21.15.101:0/1879105208 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c27f0e3ad0 0x55c27f0e2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:49.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.723+0000 7f029f7fe700 0 client.29282 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:34:49.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.723+0000 7f02d9d18b80 1 --2- 172.21.15.101:0/1879105208 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c27f0d8be0 0x55c27f0e9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:49.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.723+0000 7f02d9d18b80 1 -- 172.21.15.101:0/1879105208 >> 172.21.15.101:0/1879105208 conn(0x55c27f0ceda0 msgr2=0x55c27f0d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:49.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.723+0000 7f02d9d18b80 1 -- 172.21.15.101:0/1879105208 shutdown_connections 2023-07-18T03:34:49.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.723+0000 7f02d9d18b80 1 -- 172.21.15.101:0/1879105208 wait complete. 2023-07-18T03:34:49.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.723+0000 7f02d9d18b80 20 client.29282 trim_cache size 0 max 16384 2023-07-18T03:34:49.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.740+0000 7f02d9d18b80 1 Processor -- start 2023-07-18T03:34:49.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.740+0000 7f02d9d18b80 1 -- start start 2023-07-18T03:34:49.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.741+0000 7f02d9d18b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c27f0d82d0 0x55c27f0e2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:49.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.741+0000 7f02d9d18b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c27f0d8be0 0x55c27f0dd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:49.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.741+0000 7f02d9d18b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c27f0e3ad0 0x55c27f0e1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:49.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.741+0000 7f02d9d18b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c27f0d0ec0 con 0x55c27f0d82d0 2023-07-18T03:34:49.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.741+0000 7f02d9d18b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c27f0d3490 con 0x55c27f0e3ad0 2023-07-18T03:34:49.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.741+0000 7f02d9d18b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c27f0d5d20 con 0x55c27f0d8be0 2023-07-18T03:34:49.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.741+0000 7f02c499c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c27f0e3ad0 0x55c27f0e1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:49.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.741+0000 7f02bffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c27f0d82d0 0x55c27f0e2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:49.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.741+0000 7f02bf7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c27f0d8be0 0x55c27f0dd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:49.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.741+0000 7f02c499c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c27f0e3ad0 0x55c27f0e1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:33722/0 (socket says 172.21.15.101:33722) 2023-07-18T03:34:49.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.741+0000 7f02c499c700 1 -- 172.21.15.101:0/1851966201 learned_addr learned my addr 172.21.15.101:0/1851966201 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:34:49.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.741+0000 7f02c499c700 1 -- 172.21.15.101:0/1851966201 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c27f0d8be0 msgr2=0x55c27f0dd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:49.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.741+0000 7f02c499c700 1 --2- 172.21.15.101:0/1851966201 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c27f0d8be0 0x55c27f0dd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:49.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.741+0000 7f02c499c700 1 -- 172.21.15.101:0/1851966201 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c27f0d82d0 msgr2=0x55c27f0e2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:49.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.741+0000 7f02c499c700 1 --2- 172.21.15.101:0/1851966201 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c27f0d82d0 0x55c27f0e2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:49.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.741+0000 7f02c499c700 1 -- 172.21.15.101:0/1851966201 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c27ee7f250 con 0x55c27f0e3ad0 2023-07-18T03:34:49.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.741+0000 7f02bf7fe700 1 --2- 172.21.15.101:0/1851966201 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c27f0d8be0 0x55c27f0dd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:34:49.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.741+0000 7f02c499c700 1 --2- 172.21.15.101:0/1851966201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c27f0e3ad0 0x55c27f0e1980 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f02b8003f90 tx=0x7f02b80099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec19f3a807332ae1 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:49.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.742+0000 7f02a6ffd700 1 -- 172.21.15.101:0/1851966201 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b800e040 con 0x55c27f0e3ad0 2023-07-18T03:34:49.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.742+0000 7f02a6ffd700 1 -- 172.21.15.101:0/1851966201 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02b800f040 con 0x55c27f0e3ad0 2023-07-18T03:34:49.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.742+0000 7f02a6ffd700 1 -- 172.21.15.101:0/1851966201 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b8020680 con 0x55c27f0e3ad0 2023-07-18T03:34:49.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.742+0000 7f02d9d18b80 1 -- 172.21.15.101:0/1851966201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c27f0e3ad0 msgr2=0x55c27f0e1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:49.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.742+0000 7f02d9d18b80 1 --2- 172.21.15.101:0/1851966201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c27f0e3ad0 0x55c27f0e1980 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f02b8003f90 tx=0x7f02b80099e0 comp rx=0 tx=0).stop 2023-07-18T03:34:49.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.742+0000 7f02d9d18b80 1 -- 172.21.15.101:0/1851966201 shutdown_connections 2023-07-18T03:34:49.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.742+0000 7f02d9d18b80 1 --2- 172.21.15.101:0/1851966201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c27f0e3ad0 0x55c27f0e1980 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:49.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.742+0000 7f02d9d18b80 1 --2- 172.21.15.101:0/1851966201 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c27f0d82d0 0x55c27f0e2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:49.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.742+0000 7f02d9d18b80 1 --2- 172.21.15.101:0/1851966201 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c27f0d8be0 0x55c27f0dd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:49.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.742+0000 7f02d9d18b80 1 -- 172.21.15.101:0/1851966201 >> 172.21.15.101:0/1851966201 conn(0x55c27f0ceda0 msgr2=0x55c27f0e1270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:49.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.742+0000 7f02d9d18b80 1 -- 172.21.15.101:0/1851966201 shutdown_connections 2023-07-18T03:34:49.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.742+0000 7f02d9d18b80 1 -- 172.21.15.101:0/1851966201 wait complete. 2023-07-18T03:34:49.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.743+0000 7f02d9d18b80 1 -- start start 2023-07-18T03:34:49.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.743+0000 7f02d9d18b80 1 Processor -- start 2023-07-18T03:34:49.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.743+0000 7f02d9d18b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c27f0d82d0 0x55c27f0e90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:49.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.744+0000 7f02d9d18b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c27f0d8be0 0x55c27f0e7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:49.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.744+0000 7f02d9d18b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c27f0e3ad0 0x55c27f0e7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:49.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.744+0000 7f02d9d18b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c27f0e8210 con 0x55c27f0d8be0 2023-07-18T03:34:49.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.744+0000 7f02d9d18b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c27f146b00 con 0x55c27f0d82d0 2023-07-18T03:34:49.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.744+0000 7f02d9d18b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c27f146c40 con 0x55c27f0e3ad0 2023-07-18T03:34:49.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.744+0000 7f02d9d18b80 10 client.0 _ll_register_callbacks cb 0x7f02b8009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:34:49.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.744+0000 7f02bffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c27f0d82d0 0x55c27f0e90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:49.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.744+0000 7f02bffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c27f0d82d0 0x55c27f0e90e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:33730/0 (socket says 172.21.15.101:33730) 2023-07-18T03:34:49.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.744+0000 7f02bffff700 1 -- 172.21.15.101:0/3675803552 learned_addr learned my addr 172.21.15.101:0/3675803552 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:34:49.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.744+0000 7f02c499c700 1 --2- 172.21.15.101:0/3675803552 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c27f0e3ad0 0x55c27f0e7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:49.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.744+0000 7f02bf7fe700 1 --2- 172.21.15.101:0/3675803552 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c27f0d8be0 0x55c27f0e7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:49.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.744+0000 7f02bffff700 1 -- 172.21.15.101:0/3675803552 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c27f0e3ad0 msgr2=0x55c27f0e7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:49.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.744+0000 7f02bffff700 1 --2- 172.21.15.101:0/3675803552 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c27f0e3ad0 0x55c27f0e7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:49.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.744+0000 7f02bffff700 1 -- 172.21.15.101:0/3675803552 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c27f0d8be0 msgr2=0x55c27f0e7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:49.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.744+0000 7f02bffff700 1 --2- 172.21.15.101:0/3675803552 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c27f0d8be0 0x55c27f0e7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:49.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.744+0000 7f02bffff700 1 -- 172.21.15.101:0/3675803552 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02b4017050 con 0x55c27f0d82d0 2023-07-18T03:34:49.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.744+0000 7f02bffff700 1 --2- 172.21.15.101:0/3675803552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c27f0d82d0 0x55c27f0e90e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f02b009ec70 tx=0x7f02b009a7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba02f68b490772d1 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:49.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.744+0000 7f029f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:34:49.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.744+0000 7f029f7fe700 1 -- 172.21.15.101:0/3675803552 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b00b2060 con 0x55c27f0d82d0 2023-07-18T03:34:49.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.745+0000 7f02d9d18b80 1 -- 172.21.15.101:0/3675803552 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f02b009ee90 con 0x55c27f0d82d0 2023-07-18T03:34:49.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.745+0000 7f029f7fe700 1 -- 172.21.15.101:0/3675803552 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(134..134 src has 1..134) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f02b0080080 con 0x55c27f0d82d0 2023-07-18T03:34:49.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.745+0000 7f029f7fe700 1 -- 172.21.15.101:0/3675803552 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02b009a560 con 0x55c27f0d82d0 2023-07-18T03:34:49.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.745+0000 7f02d9d18b80 10 client.29288 fetch_fsmap learned FSMap version 49 2023-07-18T03:34:49.766 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:34:49 smithi101 ceph-mon[126379]: pgmap v862: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:34:49.769 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.745+0000 7f02d9d18b80 1 -- 172.21.15.101:0/3675803552 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f02b8003c10 con 0x55c27f0d82d0 2023-07-18T03:34:49.769 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.745+0000 7f029f7fe700 1 -- 172.21.15.101:0/3675803552 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02b009ee90 con 0x55c27f0d82d0 2023-07-18T03:34:49.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.745+0000 7f029f7fe700 1 -- 172.21.15.101:0/3675803552 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 49) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f02b00a2bc0 con 0x55c27f0d82d0 2023-07-18T03:34:49.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.745+0000 7f02d9d18b80 10 client.29288 fetch_fsmap finished waiting for FSMap version 49 2023-07-18T03:34:49.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.745+0000 7f02d9d18b80 10 client.29288 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:34:49.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.745+0000 7f02d9d18b80 20 client.29288 populate_metadata read hostname 'smithi101' 2023-07-18T03:34:49.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.745+0000 7f02d9d18b80 1 --2- 172.21.15.101:0/3675803552 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c27f147e90 0x55c27f14a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:49.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.745+0000 7f02d9d18b80 4 client.29288 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:34:49.771 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.745+0000 7f02d9d18b80 1 -- 172.21.15.101:0/3675803552 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c27efe6050 con 0x55c27f147e90 2023-07-18T03:34:49.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.745+0000 7f02bf7fe700 1 --2- 172.21.15.101:0/3675803552 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c27f147e90 0x55c27f14a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:49.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.746+0000 7f02bf7fe700 1 --2- 172.21.15.101:0/3675803552 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c27f147e90 0x55c27f14a2a0 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=715a36616de95933 server_cookie=1a12204b4fc454b8 in_seq=0 out_seq=0 2023-07-18T03:34:49.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.746+0000 7f029f7fe700 10 client.29288 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:34:49.772 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.746+0000 7f029f7fe700 1 -- 172.21.15.101:0/3675803552 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c27efe6050 con 0x55c27f147e90 2023-07-18T03:34:49.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.746+0000 7f029f7fe700 10 client.29288 handle_command_reply: tid=0 2023-07-18T03:34:49.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.746+0000 7f02d9d18b80 1 client.29288 shutdown 2023-07-18T03:34:49.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.746+0000 7f02d9d18b80 1 -- 172.21.15.101:0/3675803552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c27f0d82d0 msgr2=0x55c27f0e90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:49.773 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.746+0000 7f02d9d18b80 1 --2- 172.21.15.101:0/3675803552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c27f0d82d0 0x55c27f0e90e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f02b009ec70 tx=0x7f02b009a7f0 comp rx=0 tx=0).stop 2023-07-18T03:34:49.774 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.746+0000 7f02d9d18b80 1 -- 172.21.15.101:0/3675803552 shutdown_connections 2023-07-18T03:34:49.774 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.746+0000 7f02d9d18b80 1 --2- 172.21.15.101:0/3675803552 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55c27f147e90 0x55c27f14a2a0 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:49.774 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.746+0000 7f02d9d18b80 1 --2- 172.21.15.101:0/3675803552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c27f0d82d0 0x55c27f0e90e0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:49.774 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.746+0000 7f029f7fe700 0 client.29288 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:34:49.774 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.746+0000 7f02d9d18b80 1 --2- 172.21.15.101:0/3675803552 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c27f0e3ad0 0x55c27f0e7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:49.775 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.746+0000 7f02d9d18b80 1 --2- 172.21.15.101:0/3675803552 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c27f0d8be0 0x55c27f0e7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:49.775 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.746+0000 7f02d9d18b80 1 -- 172.21.15.101:0/3675803552 >> 172.21.15.101:0/3675803552 conn(0x55c27f0ceda0 msgr2=0x55c27f0d38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:49.775 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.747+0000 7f02d9d18b80 1 -- 172.21.15.101:0/3675803552 shutdown_connections 2023-07-18T03:34:49.775 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.747+0000 7f02d9d18b80 1 -- 172.21.15.101:0/3675803552 wait complete. 2023-07-18T03:34:49.776 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.747+0000 7f02d9d18b80 20 client.29288 trim_cache size 0 max 16384 2023-07-18T03:34:49.776 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:34:49.776 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:34:49.777 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:34:49.777 INFO:teuthology.orchestra.run.smithi101.stdout: "0f2f19e0-0429-465c-935e-223eb09490c3": { 2023-07-18T03:34:49.777 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:34:49.778 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "0f2f19e0-0429-465c-935e-223eb09490c3", 2023-07-18T03:34:49.778 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:34:49.778 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:34:49.778 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:34:49.778 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:34:49.779 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.747+0000 7f02c63ff700 1 -- 172.21.15.101:0/61443213 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f02a80a1340 msgr2=0x7f02a80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:49.780 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.747+0000 7f02c63ff700 1 --2- 172.21.15.101:0/61443213 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f02a80a1340 0x7f02a80a37f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f02c0199400 tx=0x7f02b4009260 comp rx=0 tx=0).stop 2023-07-18T03:34:49.780 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.747+0000 7f02c63ff700 1 -- 172.21.15.101:0/61443213 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02c00ffad0 msgr2=0x7f02c0197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:49.780 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.747+0000 7f02c63ff700 1 --2- 172.21.15.101:0/61443213 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02c00ffad0 0x7f02c0197df0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f02b0009230 tx=0x7f02b000fa00 comp rx=0 tx=0).stop 2023-07-18T03:34:49.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.747+0000 7f02c63ff700 1 -- 172.21.15.101:0/61443213 shutdown_connections 2023-07-18T03:34:49.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.747+0000 7f02c63ff700 1 --2- 172.21.15.101:0/61443213 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f02c0105ae0 0x7f02c019c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:49.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.747+0000 7f02c63ff700 1 --2- 172.21.15.101:0/61443213 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02c00ffad0 0x7f02c0197df0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:49.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.747+0000 7f02c63ff700 1 --2- 172.21.15.101:0/61443213 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f02a80a1340 0x7f02a80a37f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:49.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.747+0000 7f02c63ff700 1 --2- 172.21.15.101:0/61443213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02c0100480 0x7f02c0198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:49.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.747+0000 7f02c63ff700 1 -- 172.21.15.101:0/61443213 >> 172.21.15.101:0/61443213 conn(0x7f02c00fb5f0 msgr2=0x7f02c010ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:49.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.747+0000 7f02c63ff700 1 -- 172.21.15.101:0/61443213 shutdown_connections 2023-07-18T03:34:49.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:34:49.747+0000 7f02c63ff700 1 -- 172.21.15.101:0/61443213 wait complete. 2023-07-18T03:34:49.784 INFO:tasks.cephfs.filesystem:scrub status for tag:0f2f19e0-0429-465c-935e-223eb09490c3 - {'path': '/', 'tag': '0f2f19e0-0429-465c-935e-223eb09490c3', 'options': 'recursive,force'} 2023-07-18T03:34:49.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:34:49 smithi155 ceph-mon[128885]: pgmap v862: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:34:51.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:34:51 smithi179 ceph-mon[129229]: pgmap v863: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:34:51.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:34:51 smithi101 ceph-mon[126379]: pgmap v863: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:34:51.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:34:51 smithi155 ceph-mon[128885]: pgmap v863: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:34:53.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:34:53 smithi179 ceph-mon[129229]: pgmap v864: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:34:53.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:34:53 smithi101 ceph-mon[126379]: pgmap v864: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:34:53.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:34:53 smithi155 ceph-mon[128885]: pgmap v864: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:34:55.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:34:55 smithi179 ceph-mon[129229]: pgmap v865: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:34:55.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:34:55 smithi101 ceph-mon[126379]: pgmap v865: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:34:55.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:34:55 smithi155 ceph-mon[128885]: pgmap v865: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:34:57.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:34:57 smithi179 ceph-mon[129229]: pgmap v866: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:34:57.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:34:57 smithi101 ceph-mon[126379]: pgmap v866: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:34:57.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:34:57 smithi155 ceph-mon[128885]: pgmap v866: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:34:59.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:34:59 smithi179 ceph-mon[129229]: pgmap v867: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:34:59.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:34:59 smithi101 ceph-mon[126379]: pgmap v867: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:34:59.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:34:59 smithi155 ceph-mon[128885]: pgmap v867: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:35:01.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:35:01 smithi101 ceph-mon[126379]: pgmap v868: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:35:01.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:35:01 smithi155 ceph-mon[128885]: pgmap v868: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:35:01.953 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:35:01 smithi179 ceph-mon[129229]: pgmap v868: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:35:02.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:35:02 smithi101 ceph-mon[126379]: pgmap v869: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-07-18T03:35:02.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:35:02 smithi155 ceph-mon[128885]: pgmap v869: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-07-18T03:35:02.953 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:35:02 smithi179 ceph-mon[129229]: pgmap v869: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-07-18T03:35:05.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:35:05 smithi179 ceph-mon[129229]: pgmap v870: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-07-18T03:35:05.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:35:05 smithi101 ceph-mon[126379]: pgmap v870: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-07-18T03:35:05.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:35:05 smithi155 ceph-mon[128885]: pgmap v870: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-07-18T03:35:07.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:35:07 smithi179 ceph-mon[129229]: pgmap v871: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-07-18T03:35:07.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:35:07 smithi101 ceph-mon[126379]: pgmap v871: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-07-18T03:35:07.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:35:07 smithi155 ceph-mon[128885]: pgmap v871: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-07-18T03:35:09.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:35:09 smithi179 ceph-mon[129229]: pgmap v872: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-07-18T03:35:09.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:35:09 smithi101 ceph-mon[126379]: pgmap v872: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-07-18T03:35:09.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:35:09 smithi155 ceph-mon[128885]: pgmap v872: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-07-18T03:35:11.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:35:11 smithi179 ceph-mon[129229]: pgmap v873: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-07-18T03:35:11.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:35:11 smithi179 ceph-mon[129229]: osdmap e135: 12 total, 12 up, 12 in 2023-07-18T03:35:11.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:35:11 smithi101 ceph-mon[126379]: pgmap v873: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-07-18T03:35:11.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:35:11 smithi101 ceph-mon[126379]: osdmap e135: 12 total, 12 up, 12 in 2023-07-18T03:35:11.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:35:11 smithi155 ceph-mon[128885]: pgmap v873: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-07-18T03:35:11.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:35:11 smithi155 ceph-mon[128885]: osdmap e135: 12 total, 12 up, 12 in 2023-07-18T03:35:13.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:35:13 smithi179 ceph-mon[129229]: pgmap v875: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 KiB/s wr, 21 op/s 2023-07-18T03:35:13.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:35:13 smithi101 ceph-mon[126379]: pgmap v875: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 KiB/s wr, 21 op/s 2023-07-18T03:35:13.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:35:13 smithi155 ceph-mon[128885]: pgmap v875: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 KiB/s wr, 21 op/s 2023-07-18T03:35:15.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:35:15 smithi179 ceph-mon[129229]: pgmap v876: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 KiB/s wr, 21 op/s 2023-07-18T03:35:15.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:35:15 smithi101 ceph-mon[126379]: pgmap v876: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 KiB/s wr, 21 op/s 2023-07-18T03:35:15.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:35:15 smithi155 ceph-mon[128885]: pgmap v876: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 KiB/s wr, 21 op/s 2023-07-18T03:35:17.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:35:17 smithi179 ceph-mon[129229]: pgmap v877: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:35:17.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:35:17 smithi101 ceph-mon[126379]: pgmap v877: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:35:17.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:35:17 smithi155 ceph-mon[128885]: pgmap v877: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:35:19.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:35:19 smithi179 ceph-mon[129229]: pgmap v878: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:35:19.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:35:19 smithi179 ceph-mon[129229]: osdmap e136: 12 total, 12 up, 12 in 2023-07-18T03:35:19.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:35:19 smithi101 ceph-mon[126379]: pgmap v878: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:35:19.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:35:19 smithi101 ceph-mon[126379]: osdmap e136: 12 total, 12 up, 12 in 2023-07-18T03:35:19.786 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:35:19.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:35:19 smithi155 ceph-mon[128885]: pgmap v878: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:35:19.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:35:19 smithi155 ceph-mon[128885]: osdmap e136: 12 total, 12 up, 12 in 2023-07-18T03:35:19.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.001+0000 7f34ca7ff700 1 -- 172.21.15.101:0/624297920 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34c40728b0 msgr2=0x7f34c4072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:19.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.001+0000 7f34ca7ff700 1 --2- 172.21.15.101:0/624297920 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34c40728b0 0x7f34c4072c80 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f34b4009990 tx=0x7f34b4009ca0 comp rx=0 tx=0).stop 2023-07-18T03:35:19.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.001+0000 7f34ca7ff700 1 -- 172.21.15.101:0/624297920 shutdown_connections 2023-07-18T03:35:19.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.001+0000 7f34ca7ff700 1 --2- 172.21.15.101:0/624297920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f34c410c320 0x7f34c410e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:19.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.001+0000 7f34ca7ff700 1 --2- 172.21.15.101:0/624297920 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34c40728b0 0x7f34c4072c80 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:19.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.001+0000 7f34ca7ff700 1 --2- 172.21.15.101:0/624297920 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f34c40731c0 0x7f34c410bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:19.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.001+0000 7f34ca7ff700 1 -- 172.21.15.101:0/624297920 >> 172.21.15.101:0/624297920 conn(0x7f34c40fb5e0 msgr2=0x7f34c40fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:35:20.000 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.001+0000 7f34ca7ff700 1 -- 172.21.15.101:0/624297920 shutdown_connections 2023-07-18T03:35:20.000 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.001+0000 7f34ca7ff700 1 -- 172.21.15.101:0/624297920 wait complete. 2023-07-18T03:35:20.001 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.002+0000 7f34ca7ff700 1 Processor -- start 2023-07-18T03:35:20.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.002+0000 7f34ca7ff700 1 -- start start 2023-07-18T03:35:20.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.002+0000 7f34ca7ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f34c40728b0 0x7f34c4197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:20.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.002+0000 7f34ca7ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f34c40731c0 0x7f34c4198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:20.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.002+0000 7f34ca7ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34c410c320 0x7f34c419c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:20.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.002+0000 7f34ca7ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c419cda0 con 0x7f34c40728b0 2023-07-18T03:35:20.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.002+0000 7f34ca7ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c419cf10 con 0x7f34c40731c0 2023-07-18T03:35:20.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.002+0000 7f34ca7ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c419d0d0 con 0x7f34c410c320 2023-07-18T03:35:20.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.002+0000 7f34c8d9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34c410c320 0x7f34c419c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:20.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.002+0000 7f34c3fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f34c40728b0 0x7f34c4197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:20.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.003+0000 7f34c8d9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34c410c320 0x7f34c419c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:35350/0 (socket says 172.21.15.101:35350) 2023-07-18T03:35:20.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.003+0000 7f34c8d9c700 1 -- 172.21.15.101:0/1348182065 learned_addr learned my addr 172.21.15.101:0/1348182065 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:35:20.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.003+0000 7f34c37fe700 1 --2- 172.21.15.101:0/1348182065 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f34c40731c0 0x7f34c4198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:20.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.003+0000 7f34c8d9c700 1 -- 172.21.15.101:0/1348182065 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f34c40731c0 msgr2=0x7f34c4198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:20.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.003+0000 7f34c8d9c700 1 --2- 172.21.15.101:0/1348182065 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f34c40731c0 0x7f34c4198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:20.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.003+0000 7f34c8d9c700 1 -- 172.21.15.101:0/1348182065 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f34c40728b0 msgr2=0x7f34c4197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:20.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.003+0000 7f34c8d9c700 1 --2- 172.21.15.101:0/1348182065 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f34c40728b0 0x7f34c4197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:20.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.003+0000 7f34c8d9c700 1 -- 172.21.15.101:0/1348182065 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b8009520 con 0x7f34c410c320 2023-07-18T03:35:20.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.003+0000 7f34c8d9c700 1 --2- 172.21.15.101:0/1348182065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34c410c320 0x7f34c419c760 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f34bc00ef30 tx=0x7f34bc00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:20.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.003+0000 7f34c17fa700 1 -- 172.21.15.101:0/1348182065 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34bc018070 con 0x7f34c410c320 2023-07-18T03:35:20.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.003+0000 7f34ca7ff700 1 -- 172.21.15.101:0/1348182065 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34b40095f0 con 0x7f34c410c320 2023-07-18T03:35:20.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.003+0000 7f34c17fa700 1 -- 172.21.15.101:0/1348182065 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34bc00f040 con 0x7f34c410c320 2023-07-18T03:35:20.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.003+0000 7f34c17fa700 1 -- 172.21.15.101:0/1348182065 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34bc00cd00 con 0x7f34c410c320 2023-07-18T03:35:20.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.003+0000 7f34ca7ff700 1 -- 172.21.15.101:0/1348182065 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34c41a4a10 con 0x7f34c410c320 2023-07-18T03:35:20.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.007+0000 7f34c17fa700 1 -- 172.21.15.101:0/1348182065 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f34bc003a00 con 0x7f34c410c320 2023-07-18T03:35:20.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.008+0000 7f34de063b80 1 Processor -- start 2023-07-18T03:35:20.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.008+0000 7f34de063b80 1 -- start start 2023-07-18T03:35:20.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.009+0000 7f34de063b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636ca6d62d0 0x5636ca6d66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:20.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.009+0000 7f34de063b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ca6d6be0 0x5636ca6e1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:20.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.009+0000 7f34de063b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636ca6e1ad0 0x5636ca6e3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:20.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.009+0000 7f34de063b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ca5dccb0 con 0x5636ca6d62d0 2023-07-18T03:35:20.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.009+0000 7f34de063b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ca62fa40 con 0x5636ca6e1ad0 2023-07-18T03:35:20.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.009+0000 7f34de063b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ca628ed0 con 0x5636ca6d6be0 2023-07-18T03:35:20.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.009+0000 7f34c17fa700 1 --2- 172.21.15.101:0/1348182065 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f34ac0a12c0 0x7f34ac0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:20.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.009+0000 7f34c3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ca6d6be0 0x5636ca6e1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:20.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.009+0000 7f34c3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ca6d6be0 0x5636ca6e1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:35354/0 (socket says 172.21.15.101:35354) 2023-07-18T03:35:20.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.009+0000 7f34c3fff700 1 -- 172.21.15.101:0/4058122539 learned_addr learned my addr 172.21.15.101:0/4058122539 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:35:20.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.009+0000 7f34c37fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636ca6d62d0 0x5636ca6d66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:20.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.009+0000 7f34c17fa700 1 -- 172.21.15.101:0/1348182065 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f34bc0ccba0 con 0x7f34c410c320 2023-07-18T03:35:20.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.009+0000 7f34c37fe700 1 --2- 172.21.15.101:0/4058122539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636ca6e1ad0 0x5636ca6e3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:20.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.009+0000 7f34c8d9c700 1 --2- 172.21.15.101:0/1348182065 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f34ac0a12c0 0x7f34ac0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:20.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.009+0000 7f34c3fff700 1 -- 172.21.15.101:0/4058122539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636ca6e1ad0 msgr2=0x5636ca6e3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:20.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.009+0000 7f34c3fff700 1 --2- 172.21.15.101:0/4058122539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636ca6e1ad0 0x5636ca6e3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:20.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.009+0000 7f34c3fff700 1 -- 172.21.15.101:0/4058122539 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636ca6d62d0 msgr2=0x5636ca6d66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:20.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.009+0000 7f34c3fff700 1 --2- 172.21.15.101:0/4058122539 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636ca6d62d0 0x5636ca6d66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:20.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.009+0000 7f34c3fff700 1 -- 172.21.15.101:0/4058122539 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636ca456de0 con 0x5636ca6d6be0 2023-07-18T03:35:20.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.010+0000 7f34c37fe700 1 --2- 172.21.15.101:0/4058122539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636ca6e1ad0 0x5636ca6e3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:35:20.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.010+0000 7f34c3fff700 1 --2- 172.21.15.101:0/4058122539 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ca6d6be0 0x5636ca6e1590 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f34b4005ea0 tx=0x7f34b4005ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ffd1146feff1875 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:20.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.010+0000 7f34c8d9c700 1 --2- 172.21.15.101:0/1348182065 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f34ac0a12c0 0x7f34ac0a3770 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f34bc0c9f40 tx=0x7f34bc01c000 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:20.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.010+0000 7f34aa7fc700 1 -- 172.21.15.101:0/4058122539 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b401d070 con 0x5636ca6d6be0 2023-07-18T03:35:20.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.010+0000 7f34aa7fc700 1 -- 172.21.15.101:0/4058122539 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34b401e040 con 0x5636ca6d6be0 2023-07-18T03:35:20.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.010+0000 7f34aa7fc700 1 -- 172.21.15.101:0/4058122539 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b4004880 con 0x5636ca6d6be0 2023-07-18T03:35:20.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.011+0000 7f34de063b80 1 -- 172.21.15.101:0/4058122539 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ca6d6be0 msgr2=0x5636ca6e1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:20.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.011+0000 7f34de063b80 1 --2- 172.21.15.101:0/4058122539 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ca6d6be0 0x5636ca6e1590 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f34b4005ea0 tx=0x7f34b4005ed0 comp rx=0 tx=0).stop 2023-07-18T03:35:20.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.011+0000 7f34de063b80 1 -- 172.21.15.101:0/4058122539 shutdown_connections 2023-07-18T03:35:20.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.011+0000 7f34de063b80 1 --2- 172.21.15.101:0/4058122539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636ca6e1ad0 0x5636ca6e3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:20.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.011+0000 7f34de063b80 1 --2- 172.21.15.101:0/4058122539 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636ca6d62d0 0x5636ca6d66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:20.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.011+0000 7f34de063b80 1 --2- 172.21.15.101:0/4058122539 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ca6d6be0 0x5636ca6e1590 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:20.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.011+0000 7f34de063b80 1 -- 172.21.15.101:0/4058122539 >> 172.21.15.101:0/4058122539 conn(0x5636ca6ccda0 msgr2=0x5636ca6d26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:35:20.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.011+0000 7f34de063b80 1 -- 172.21.15.101:0/4058122539 shutdown_connections 2023-07-18T03:35:20.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.011+0000 7f34de063b80 1 -- 172.21.15.101:0/4058122539 wait complete. 2023-07-18T03:35:20.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.012+0000 7f34de063b80 1 -- start start 2023-07-18T03:35:20.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.012+0000 7f34de063b80 1 Processor -- start 2023-07-18T03:35:20.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.012+0000 7f34de063b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636ca6d62d0 0x5636ca6e1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:20.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.012+0000 7f34de063b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636ca6d6be0 0x5636ca6de760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:20.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.012+0000 7f34de063b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ca6e1ad0 0x5636ca6ded90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:20.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.012+0000 7f34de063b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ca5c8ac0 con 0x5636ca6d6be0 2023-07-18T03:35:20.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.012+0000 7f34de063b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ca5c87c0 con 0x5636ca6d62d0 2023-07-18T03:35:20.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.012+0000 7f34de063b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ca4ac920 con 0x5636ca6e1ad0 2023-07-18T03:35:20.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.012+0000 7f34de063b80 10 client.0 _ll_register_callbacks cb 0x5636ca5c6760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:35:20.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.012+0000 7f34c37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ca6e1ad0 0x5636ca6ded90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:20.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.012+0000 7f34c3fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636ca6d6be0 0x5636ca6de760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:20.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.013+0000 7f34c37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ca6e1ad0 0x5636ca6ded90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:35362/0 (socket says 172.21.15.101:35362) 2023-07-18T03:35:20.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.013+0000 7f34c37fe700 1 -- 172.21.15.101:0/974350452 learned_addr learned my addr 172.21.15.101:0/974350452 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:35:20.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.013+0000 7f34c37fe700 1 --2- 172.21.15.101:0/974350452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636ca6d62d0 0x5636ca6e1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:20.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.013+0000 7f34c37fe700 1 -- 172.21.15.101:0/974350452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636ca6d62d0 msgr2=0x5636ca6e1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:20.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.013+0000 7f34c37fe700 1 --2- 172.21.15.101:0/974350452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636ca6d62d0 0x5636ca6e1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:20.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.013+0000 7f34c37fe700 1 -- 172.21.15.101:0/974350452 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636ca6d6be0 msgr2=0x5636ca6de760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:20.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.013+0000 7f34c37fe700 1 --2- 172.21.15.101:0/974350452 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636ca6d6be0 0x5636ca6de760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:20.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.013+0000 7f34c37fe700 1 -- 172.21.15.101:0/974350452 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34c410f3a0 con 0x5636ca6e1ad0 2023-07-18T03:35:20.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.013+0000 7f34c37fe700 1 --2- 172.21.15.101:0/974350452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ca6e1ad0 0x5636ca6ded90 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f34b8013e60 tx=0x7f34b8015040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c058d40b0e3241f8 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:20.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.013+0000 7f34a37fe700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:35:20.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.013+0000 7f34a37fe700 1 -- 172.21.15.101:0/974350452 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b8010bd0 con 0x5636ca6e1ad0 2023-07-18T03:35:20.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.013+0000 7f34a37fe700 1 -- 172.21.15.101:0/974350452 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(136..136 src has 1..136) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f34b8010d30 con 0x5636ca6e1ad0 2023-07-18T03:35:20.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.013+0000 7f34de063b80 1 -- 172.21.15.101:0/974350452 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5636ca5824f0 con 0x5636ca6e1ad0 2023-07-18T03:35:20.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.014+0000 7f34a37fe700 1 -- 172.21.15.101:0/974350452 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34b8004050 con 0x5636ca6e1ad0 2023-07-18T03:35:20.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.014+0000 7f34de063b80 10 client.29290 fetch_fsmap learned FSMap version 49 2023-07-18T03:35:20.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.014+0000 7f34de063b80 1 -- 172.21.15.101:0/974350452 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5636ca43fc60 con 0x5636ca6e1ad0 2023-07-18T03:35:20.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.014+0000 7f34a37fe700 1 -- 172.21.15.101:0/974350452 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5636ca5824f0 con 0x5636ca6e1ad0 2023-07-18T03:35:20.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.014+0000 7f34a37fe700 1 -- 172.21.15.101:0/974350452 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 49) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f34b80213b0 con 0x5636ca6e1ad0 2023-07-18T03:35:20.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.014+0000 7f34de063b80 10 client.29290 fetch_fsmap finished waiting for FSMap version 49 2023-07-18T03:35:20.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.014+0000 7f34de063b80 10 client.29290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:35:20.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.014+0000 7f34de063b80 20 client.29290 populate_metadata read hostname 'smithi101' 2023-07-18T03:35:20.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.014+0000 7f34de063b80 1 --2- 172.21.15.101:0/974350452 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5636ca6f1310 0x5636ca6f3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:20.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.014+0000 7f34de063b80 4 client.29290 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:35:20.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.014+0000 7f34de063b80 1 -- 172.21.15.101:0/974350452 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5636ca621eb0 con 0x5636ca6f1310 2023-07-18T03:35:20.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.014+0000 7f34c3fff700 1 --2- 172.21.15.101:0/974350452 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5636ca6f1310 0x5636ca6f3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:20.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.015+0000 7f34c3fff700 1 --2- 172.21.15.101:0/974350452 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5636ca6f1310 0x5636ca6f3720 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f38b34c52241dbe5 server_cookie=f11771bcaffe1f6c in_seq=0 out_seq=0 2023-07-18T03:35:20.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.015+0000 7f34a37fe700 10 client.29290 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:35:20.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.015+0000 7f34a37fe700 1 -- 172.21.15.101:0/974350452 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5636ca621eb0 con 0x5636ca6f1310 2023-07-18T03:35:20.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.016+0000 7f34a37fe700 10 client.29290 handle_command_reply: tid=0 2023-07-18T03:35:20.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.016+0000 7f34de063b80 1 client.29290 shutdown 2023-07-18T03:35:20.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.016+0000 7f34de063b80 1 -- 172.21.15.101:0/974350452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ca6e1ad0 msgr2=0x5636ca6ded90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:20.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.016+0000 7f34de063b80 1 --2- 172.21.15.101:0/974350452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ca6e1ad0 0x5636ca6ded90 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f34b8013e60 tx=0x7f34b8015040 comp rx=0 tx=0).stop 2023-07-18T03:35:20.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.016+0000 7f34de063b80 1 -- 172.21.15.101:0/974350452 shutdown_connections 2023-07-18T03:35:20.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.016+0000 7f34de063b80 1 --2- 172.21.15.101:0/974350452 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5636ca6f1310 0x5636ca6f3720 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:20.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.016+0000 7f34de063b80 1 --2- 172.21.15.101:0/974350452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636ca6d62d0 0x5636ca6e1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:20.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.016+0000 7f34de063b80 1 --2- 172.21.15.101:0/974350452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ca6e1ad0 0x5636ca6ded90 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:20.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.016+0000 7f34de063b80 1 --2- 172.21.15.101:0/974350452 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636ca6d6be0 0x5636ca6de760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:20.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.016+0000 7f34a37fe700 0 client.29290 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:35:20.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.016+0000 7f34de063b80 1 -- 172.21.15.101:0/974350452 >> 172.21.15.101:0/974350452 conn(0x5636ca6ccda0 msgr2=0x5636ca6d2290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:35:20.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.016+0000 7f34de063b80 1 -- 172.21.15.101:0/974350452 shutdown_connections 2023-07-18T03:35:20.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.016+0000 7f34de063b80 1 -- 172.21.15.101:0/974350452 wait complete. 2023-07-18T03:35:20.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.016+0000 7f34de063b80 20 client.29290 trim_cache size 0 max 16384 2023-07-18T03:35:20.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.034+0000 7f34de063b80 1 Processor -- start 2023-07-18T03:35:20.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.034+0000 7f34de063b80 1 -- start start 2023-07-18T03:35:20.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.034+0000 7f34de063b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636ca6d62d0 0x5636ca6e1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:20.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.034+0000 7f34de063b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ca6d6be0 0x5636ca6db9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:20.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.034+0000 7f34de063b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636ca6e1ad0 0x5636ca6dbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:20.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.034+0000 7f34de063b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ca582520 con 0x5636ca6d62d0 2023-07-18T03:35:20.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.034+0000 7f34de063b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ca6d3360 con 0x5636ca6e1ad0 2023-07-18T03:35:20.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.034+0000 7f34de063b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ca6d7e30 con 0x5636ca6d6be0 2023-07-18T03:35:20.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.035+0000 7f34c3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ca6d6be0 0x5636ca6db9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:20.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.035+0000 7f34c3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ca6d6be0 0x5636ca6db9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:35378/0 (socket says 172.21.15.101:35378) 2023-07-18T03:35:20.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.035+0000 7f34c3fff700 1 -- 172.21.15.101:0/3657434409 learned_addr learned my addr 172.21.15.101:0/3657434409 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:35:20.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.035+0000 7f34c37fe700 1 --2- 172.21.15.101:0/3657434409 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636ca6d62d0 0x5636ca6e1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:20.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.035+0000 7f34c37fe700 1 --2- 172.21.15.101:0/3657434409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636ca6e1ad0 0x5636ca6dbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:20.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.035+0000 7f34c3fff700 1 -- 172.21.15.101:0/3657434409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636ca6e1ad0 msgr2=0x5636ca6dbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:20.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.035+0000 7f34c3fff700 1 --2- 172.21.15.101:0/3657434409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636ca6e1ad0 0x5636ca6dbf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:20.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.035+0000 7f34c3fff700 1 -- 172.21.15.101:0/3657434409 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636ca6d62d0 msgr2=0x5636ca6e1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:20.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.035+0000 7f34c3fff700 1 --2- 172.21.15.101:0/3657434409 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636ca6d62d0 0x5636ca6e1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:20.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.035+0000 7f34c3fff700 1 -- 172.21.15.101:0/3657434409 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636ca47d250 con 0x5636ca6d6be0 2023-07-18T03:35:20.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.035+0000 7f34c37fe700 1 --2- 172.21.15.101:0/3657434409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636ca6e1ad0 0x5636ca6dbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:35:20.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.035+0000 7f34c3fff700 1 --2- 172.21.15.101:0/3657434409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ca6d6be0 0x5636ca6db9f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f34b4005b00 tx=0x7f34b4009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e7350f26fc2308b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:20.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.035+0000 7f34aaffd700 1 -- 172.21.15.101:0/3657434409 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b401d040 con 0x5636ca6d6be0 2023-07-18T03:35:20.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.035+0000 7f34aaffd700 1 -- 172.21.15.101:0/3657434409 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34b4004570 con 0x5636ca6d6be0 2023-07-18T03:35:20.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.035+0000 7f34aaffd700 1 -- 172.21.15.101:0/3657434409 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b400b390 con 0x5636ca6d6be0 2023-07-18T03:35:20.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.036+0000 7f34de063b80 1 -- 172.21.15.101:0/3657434409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ca6d6be0 msgr2=0x5636ca6db9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:20.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.036+0000 7f34de063b80 1 --2- 172.21.15.101:0/3657434409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ca6d6be0 0x5636ca6db9f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f34b4005b00 tx=0x7f34b4009310 comp rx=0 tx=0).stop 2023-07-18T03:35:20.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.036+0000 7f34de063b80 1 -- 172.21.15.101:0/3657434409 shutdown_connections 2023-07-18T03:35:20.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.036+0000 7f34de063b80 1 --2- 172.21.15.101:0/3657434409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636ca6e1ad0 0x5636ca6dbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:20.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.036+0000 7f34de063b80 1 --2- 172.21.15.101:0/3657434409 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636ca6d62d0 0x5636ca6e1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:20.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.036+0000 7f34de063b80 1 --2- 172.21.15.101:0/3657434409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ca6d6be0 0x5636ca6db9f0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:20.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.036+0000 7f34de063b80 1 -- 172.21.15.101:0/3657434409 >> 172.21.15.101:0/3657434409 conn(0x5636ca6ccda0 msgr2=0x5636ca6d9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:35:20.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.036+0000 7f34de063b80 1 -- 172.21.15.101:0/3657434409 shutdown_connections 2023-07-18T03:35:20.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.036+0000 7f34de063b80 1 -- 172.21.15.101:0/3657434409 wait complete. 2023-07-18T03:35:20.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.037+0000 7f34de063b80 1 -- start start 2023-07-18T03:35:20.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.037+0000 7f34de063b80 1 Processor -- start 2023-07-18T03:35:20.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.038+0000 7f34de063b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636ca6d62d0 0x5636ca6e8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:20.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.038+0000 7f34de063b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ca6d6be0 0x5636ca6e8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:20.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.038+0000 7f34de063b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636ca6e1ad0 0x5636ca6e65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:20.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.038+0000 7f34de063b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ca6e6cc0 con 0x5636ca6d62d0 2023-07-18T03:35:20.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.038+0000 7f34de063b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ca6e6e30 con 0x5636ca6e1ad0 2023-07-18T03:35:20.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.038+0000 7f34de063b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ca6e6ff0 con 0x5636ca6d6be0 2023-07-18T03:35:20.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.038+0000 7f34de063b80 10 client.0 _ll_register_callbacks cb 0x7f34b8013980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:35:20.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.038+0000 7f34c3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ca6d6be0 0x5636ca6e8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:20.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.038+0000 7f34c37fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636ca6d62d0 0x5636ca6e8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:20.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.038+0000 7f34c3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ca6d6be0 0x5636ca6e8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:35392/0 (socket says 172.21.15.101:35392) 2023-07-18T03:35:20.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.038+0000 7f34c3fff700 1 -- 172.21.15.101:0/2941556887 learned_addr learned my addr 172.21.15.101:0/2941556887 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:35:20.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.038+0000 7f34c37fe700 1 --2- 172.21.15.101:0/2941556887 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636ca6e1ad0 0x5636ca6e65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:20.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.038+0000 7f34c3fff700 1 -- 172.21.15.101:0/2941556887 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636ca6e1ad0 msgr2=0x5636ca6e65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:20.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.038+0000 7f34c3fff700 1 --2- 172.21.15.101:0/2941556887 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636ca6e1ad0 0x5636ca6e65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:20.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.038+0000 7f34c3fff700 1 -- 172.21.15.101:0/2941556887 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636ca6d62d0 msgr2=0x5636ca6e8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:20.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.038+0000 7f34c3fff700 1 --2- 172.21.15.101:0/2941556887 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636ca6d62d0 0x5636ca6e8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:20.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.038+0000 7f34c3fff700 1 -- 172.21.15.101:0/2941556887 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5636ca456de0 con 0x5636ca6d6be0 2023-07-18T03:35:20.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.039+0000 7f34c3fff700 1 --2- 172.21.15.101:0/2941556887 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ca6d6be0 0x5636ca6e8640 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f34b400b6f0 tx=0x7f34b40058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee74fbdacab5637 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:20.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.039+0000 7f34a37fe700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:35:20.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.039+0000 7f34a37fe700 1 -- 172.21.15.101:0/2941556887 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b4004a30 con 0x5636ca6d6be0 2023-07-18T03:35:20.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.039+0000 7f34de063b80 1 -- 172.21.15.101:0/2941556887 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f34b401e040 con 0x5636ca6d6be0 2023-07-18T03:35:20.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.039+0000 7f34a37fe700 1 -- 172.21.15.101:0/2941556887 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(136..136 src has 1..136) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f34b4004b90 con 0x5636ca6d6be0 2023-07-18T03:35:20.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.039+0000 7f34a37fe700 1 -- 172.21.15.101:0/2941556887 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34b4004e60 con 0x5636ca6d6be0 2023-07-18T03:35:20.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.039+0000 7f34de063b80 10 client.29296 fetch_fsmap learned FSMap version 49 2023-07-18T03:35:20.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.039+0000 7f34de063b80 1 -- 172.21.15.101:0/2941556887 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5636ca4552b0 con 0x5636ca6d6be0 2023-07-18T03:35:20.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.039+0000 7f34a37fe700 1 -- 172.21.15.101:0/2941556887 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34b401e040 con 0x5636ca6d6be0 2023-07-18T03:35:20.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.040+0000 7f34a37fe700 1 -- 172.21.15.101:0/2941556887 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 49) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f34b400fa60 con 0x5636ca6d6be0 2023-07-18T03:35:20.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.040+0000 7f34de063b80 10 client.29296 fetch_fsmap finished waiting for FSMap version 49 2023-07-18T03:35:20.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.040+0000 7f34de063b80 10 client.29296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:35:20.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.040+0000 7f34de063b80 20 client.29296 populate_metadata read hostname 'smithi101' 2023-07-18T03:35:20.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.040+0000 7f34de063b80 1 --2- 172.21.15.101:0/2941556887 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5636ca745f20 0x5636ca748300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:20.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.040+0000 7f34de063b80 4 client.29296 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:35:20.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.040+0000 7f34de063b80 1 -- 172.21.15.101:0/2941556887 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5636ca5e4050 con 0x5636ca745f20 2023-07-18T03:35:20.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.040+0000 7f34c37fe700 1 --2- 172.21.15.101:0/2941556887 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5636ca745f20 0x5636ca748300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:20.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.040+0000 7f34a37fe700 10 client.29296 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:35:20.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.040+0000 7f34c37fe700 1 --2- 172.21.15.101:0/2941556887 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5636ca745f20 0x5636ca748300 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc7907cb09fc83e8 server_cookie=a5274fd4db398785 in_seq=0 out_seq=0 2023-07-18T03:35:20.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.040+0000 7f34a37fe700 1 -- 172.21.15.101:0/2941556887 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5636ca5e4050 con 0x5636ca745f20 2023-07-18T03:35:20.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.040+0000 7f34a37fe700 10 client.29296 handle_command_reply: tid=0 2023-07-18T03:35:20.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.040+0000 7f34de063b80 1 client.29296 shutdown 2023-07-18T03:35:20.049 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.041+0000 7f34de063b80 1 -- 172.21.15.101:0/2941556887 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ca6d6be0 msgr2=0x5636ca6e8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:20.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.041+0000 7f34de063b80 1 --2- 172.21.15.101:0/2941556887 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ca6d6be0 0x5636ca6e8640 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f34b400b6f0 tx=0x7f34b40058e0 comp rx=0 tx=0).stop 2023-07-18T03:35:20.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.041+0000 7f34de063b80 1 -- 172.21.15.101:0/2941556887 shutdown_connections 2023-07-18T03:35:20.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.041+0000 7f34de063b80 1 --2- 172.21.15.101:0/2941556887 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5636ca745f20 0x5636ca748300 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:20.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.041+0000 7f34de063b80 1 --2- 172.21.15.101:0/2941556887 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636ca6e1ad0 0x5636ca6e65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:20.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.041+0000 7f34de063b80 1 --2- 172.21.15.101:0/2941556887 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636ca6d62d0 0x5636ca6e8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:20.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.041+0000 7f34de063b80 1 --2- 172.21.15.101:0/2941556887 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ca6d6be0 0x5636ca6e8640 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:20.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.041+0000 7f34a37fe700 0 client.29296 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:35:20.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.041+0000 7f34de063b80 1 -- 172.21.15.101:0/2941556887 >> 172.21.15.101:0/2941556887 conn(0x5636ca6ccda0 msgr2=0x5636ca6d83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:35:20.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.041+0000 7f34de063b80 1 -- 172.21.15.101:0/2941556887 shutdown_connections 2023-07-18T03:35:20.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.041+0000 7f34de063b80 1 -- 172.21.15.101:0/2941556887 wait complete. 2023-07-18T03:35:20.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.041+0000 7f34de063b80 20 client.29296 trim_cache size 0 max 16384 2023-07-18T03:35:20.052 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:35:20.052 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:35:20.053 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:35:20.053 INFO:teuthology.orchestra.run.smithi101.stdout: "0f2f19e0-0429-465c-935e-223eb09490c3": { 2023-07-18T03:35:20.053 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:35:20.053 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "0f2f19e0-0429-465c-935e-223eb09490c3", 2023-07-18T03:35:20.053 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:35:20.054 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:35:20.054 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:35:20.054 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:35:20.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.041+0000 7f34ca7ff700 1 -- 172.21.15.101:0/1348182065 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f34ac0a12c0 msgr2=0x7f34ac0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:20.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.041+0000 7f34ca7ff700 1 --2- 172.21.15.101:0/1348182065 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f34ac0a12c0 0x7f34ac0a3770 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f34bc0c9f40 tx=0x7f34bc01c000 comp rx=0 tx=0).stop 2023-07-18T03:35:20.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.041+0000 7f34ca7ff700 1 -- 172.21.15.101:0/1348182065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34c410c320 msgr2=0x7f34c419c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:20.055 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.041+0000 7f34ca7ff700 1 --2- 172.21.15.101:0/1348182065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34c410c320 0x7f34c419c760 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f34bc00ef30 tx=0x7f34bc00c3a0 comp rx=0 tx=0).stop 2023-07-18T03:35:20.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.042+0000 7f34ca7ff700 1 -- 172.21.15.101:0/1348182065 shutdown_connections 2023-07-18T03:35:20.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.042+0000 7f34ca7ff700 1 --2- 172.21.15.101:0/1348182065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34c410c320 0x7f34c419c760 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:20.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.042+0000 7f34ca7ff700 1 --2- 172.21.15.101:0/1348182065 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f34c40728b0 0x7f34c4197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:20.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.042+0000 7f34ca7ff700 1 --2- 172.21.15.101:0/1348182065 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f34ac0a12c0 0x7f34ac0a3770 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:20.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.042+0000 7f34ca7ff700 1 --2- 172.21.15.101:0/1348182065 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f34c40731c0 0x7f34c4198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:20.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.042+0000 7f34ca7ff700 1 -- 172.21.15.101:0/1348182065 >> 172.21.15.101:0/1348182065 conn(0x7f34c40fb5e0 msgr2=0x7f34c410e400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:35:20.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.042+0000 7f34ca7ff700 1 -- 172.21.15.101:0/1348182065 shutdown_connections 2023-07-18T03:35:20.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:20.042+0000 7f34ca7ff700 1 -- 172.21.15.101:0/1348182065 wait complete. 2023-07-18T03:35:20.058 INFO:tasks.cephfs.filesystem:scrub status for tag:0f2f19e0-0429-465c-935e-223eb09490c3 - {'path': '/', 'tag': '0f2f19e0-0429-465c-935e-223eb09490c3', 'options': 'recursive,force'} 2023-07-18T03:35:21.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:35:21 smithi179 ceph-mon[129229]: pgmap v880: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 210 B/s wr, 6 op/s 2023-07-18T03:35:21.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:35:21 smithi101 ceph-mon[126379]: pgmap v880: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 210 B/s wr, 6 op/s 2023-07-18T03:35:21.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:35:21 smithi155 ceph-mon[128885]: pgmap v880: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 210 B/s wr, 6 op/s 2023-07-18T03:35:23.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:35:23 smithi179 ceph-mon[129229]: pgmap v881: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:35:23.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:35:23 smithi101 ceph-mon[126379]: pgmap v881: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:35:23.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:35:23 smithi155 ceph-mon[128885]: pgmap v881: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:35:25.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:35:25 smithi179 ceph-mon[129229]: pgmap v882: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:35:25.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:35:25 smithi101 ceph-mon[126379]: pgmap v882: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:35:25.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:35:25 smithi155 ceph-mon[128885]: pgmap v882: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:35:27.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:35:27 smithi179 ceph-mon[129229]: pgmap v883: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:35:27.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:35:27 smithi101 ceph-mon[126379]: pgmap v883: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:35:27.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:35:27 smithi155 ceph-mon[128885]: pgmap v883: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:35:29.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:35:29 smithi179 ceph-mon[129229]: pgmap v884: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:35:29.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:35:29 smithi101 ceph-mon[126379]: pgmap v884: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:35:29.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:35:29 smithi155 ceph-mon[128885]: pgmap v884: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:35:31.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:35:31 smithi179 ceph-mon[129229]: pgmap v885: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 95 B/s wr, 8 op/s 2023-07-18T03:35:31.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:35:31 smithi101 ceph-mon[126379]: pgmap v885: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 95 B/s wr, 8 op/s 2023-07-18T03:35:31.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:35:31 smithi155 ceph-mon[128885]: pgmap v885: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 95 B/s wr, 8 op/s 2023-07-18T03:35:32.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:35:32 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:35:32.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:35:32 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:35:32.809 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:35:32 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:35:33.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:35:33 smithi179 ceph-mon[129229]: pgmap v886: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:35:33.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:35:33 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:35:33.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:35:33 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:35:33.704 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:35:33 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:35:33.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:35:33 smithi101 ceph-mon[126379]: pgmap v886: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:35:33.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:35:33 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:35:33.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:35:33 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:35:33.742 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:35:33 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:35:33.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:35:33 smithi155 ceph-mon[128885]: pgmap v886: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:35:33.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:35:33 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:35:33.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:35:33 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:35:33.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:35:33 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:35:35.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:35:35 smithi179 ceph-mon[129229]: pgmap v887: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:35:35.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:35:35 smithi101 ceph-mon[126379]: pgmap v887: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:35:35.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:35:35 smithi155 ceph-mon[128885]: pgmap v887: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:35:37.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:35:37 smithi179 ceph-mon[129229]: pgmap v888: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:35:37.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:35:37 smithi101 ceph-mon[126379]: pgmap v888: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:35:37.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:35:37 smithi155 ceph-mon[128885]: pgmap v888: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:35:39.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:35:39 smithi179 ceph-mon[129229]: pgmap v889: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:35:39.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:35:39 smithi101 ceph-mon[126379]: pgmap v889: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:35:39.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:35:39 smithi155 ceph-mon[128885]: pgmap v889: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:35:41.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:35:41 smithi179 ceph-mon[129229]: pgmap v890: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:35:41.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:35:41 smithi101 ceph-mon[126379]: pgmap v890: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:35:41.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:35:41 smithi155 ceph-mon[128885]: pgmap v890: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:35:43.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:35:43 smithi179 ceph-mon[129229]: pgmap v891: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:35:43.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:35:43 smithi101 ceph-mon[126379]: pgmap v891: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:35:43.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:35:43 smithi155 ceph-mon[128885]: pgmap v891: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:35:45.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:35:45 smithi101 ceph-mon[126379]: pgmap v892: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:35:45.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:35:45 smithi155 ceph-mon[128885]: pgmap v892: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:35:45.953 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:35:45 smithi179 ceph-mon[129229]: pgmap v892: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:35:47.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:35:47 smithi101 ceph-mon[126379]: pgmap v893: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:35:47.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:35:47 smithi155 ceph-mon[128885]: pgmap v893: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:35:47.953 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:35:47 smithi179 ceph-mon[129229]: pgmap v893: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:35:49.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:35:49 smithi101 ceph-mon[126379]: pgmap v894: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:35:49.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:35:49 smithi155 ceph-mon[128885]: pgmap v894: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:35:49.953 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:35:49 smithi179 ceph-mon[129229]: pgmap v894: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:35:50.060 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:35:50.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.276+0000 7f0ee75ff700 1 -- 172.21.15.101:0/551088563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ee0100480 msgr2=0x7f0ee010c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:50.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.276+0000 7f0ee75ff700 1 --2- 172.21.15.101:0/551088563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ee0100480 0x7f0ee010c910 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f0ed800b0a0 tx=0x7f0ed800b3b0 comp rx=0 tx=0).stop 2023-07-18T03:35:50.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.276+0000 7f0ee75ff700 1 -- 172.21.15.101:0/551088563 shutdown_connections 2023-07-18T03:35:50.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.276+0000 7f0ee75ff700 1 --2- 172.21.15.101:0/551088563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ee0100480 0x7f0ee010c910 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:50.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.276+0000 7f0ee75ff700 1 --2- 172.21.15.101:0/551088563 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ee0105ae0 0x7f0ee0105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:50.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.276+0000 7f0ee75ff700 1 --2- 172.21.15.101:0/551088563 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0ee00ffad0 0x7f0ee00fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:50.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.276+0000 7f0ee75ff700 1 -- 172.21.15.101:0/551088563 >> 172.21.15.101:0/551088563 conn(0x7f0ee00fb5f0 msgr2=0x7f0ee00fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:35:50.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.277+0000 7f0ee75ff700 1 -- 172.21.15.101:0/551088563 shutdown_connections 2023-07-18T03:35:50.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.277+0000 7f0ee75ff700 1 -- 172.21.15.101:0/551088563 wait complete. 2023-07-18T03:35:50.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.278+0000 7f0ee75ff700 1 Processor -- start 2023-07-18T03:35:50.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.278+0000 7f0ee75ff700 1 -- start start 2023-07-18T03:35:50.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.279+0000 7f0ee75ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ee00ffad0 0x7f0ee0072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:50.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.279+0000 7f0ee75ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0ee0100480 0x7f0ee0072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:50.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.279+0000 7f0ee75ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ee0105ae0 0x7f0ee01a3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:50.279 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.279+0000 7f0ee75ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee0073580 con 0x7f0ee0100480 2023-07-18T03:35:50.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.279+0000 7f0ee75ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee01a43f0 con 0x7f0ee0105ae0 2023-07-18T03:35:50.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.279+0000 7f0ee75ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee01a4530 con 0x7f0ee00ffad0 2023-07-18T03:35:50.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.279+0000 7f0ee539b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ee00ffad0 0x7f0ee0072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:50.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.279+0000 7f0ee4b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0ee0100480 0x7f0ee0072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:50.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.279+0000 7f0ee539b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ee00ffad0 0x7f0ee0072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:60692/0 (socket says 172.21.15.101:60692) 2023-07-18T03:35:50.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.279+0000 7f0ee539b700 1 -- 172.21.15.101:0/1390623750 learned_addr learned my addr 172.21.15.101:0/1390623750 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:35:50.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.279+0000 7f0ee5b9c700 1 --2- 172.21.15.101:0/1390623750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ee0105ae0 0x7f0ee01a3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:50.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.279+0000 7f0ee539b700 1 -- 172.21.15.101:0/1390623750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ee0105ae0 msgr2=0x7f0ee01a3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:50.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.279+0000 7f0ee539b700 1 --2- 172.21.15.101:0/1390623750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ee0105ae0 0x7f0ee01a3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:50.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.279+0000 7f0ee539b700 1 -- 172.21.15.101:0/1390623750 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0ee0100480 msgr2=0x7f0ee0072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:50.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.280+0000 7f0ee539b700 1 --2- 172.21.15.101:0/1390623750 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0ee0100480 0x7f0ee0072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:50.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.280+0000 7f0ee539b700 1 -- 172.21.15.101:0/1390623750 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed8009d00 con 0x7f0ee00ffad0 2023-07-18T03:35:50.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.280+0000 7f0ee539b700 1 --2- 172.21.15.101:0/1390623750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ee00ffad0 0x7f0ee0072810 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f0edc00e9a0 tx=0x7f0edc00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:50.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.280+0000 7f0ed67fc700 1 -- 172.21.15.101:0/1390623750 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0edc009790 con 0x7f0ee00ffad0 2023-07-18T03:35:50.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.280+0000 7f0ed67fc700 1 -- 172.21.15.101:0/1390623750 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0edc010430 con 0x7f0ee00ffad0 2023-07-18T03:35:50.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.280+0000 7f0ee75ff700 1 -- 172.21.15.101:0/1390623750 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ee01a4790 con 0x7f0ee00ffad0 2023-07-18T03:35:50.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.280+0000 7f0ed67fc700 1 -- 172.21.15.101:0/1390623750 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0edc018680 con 0x7f0ee00ffad0 2023-07-18T03:35:50.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.280+0000 7f0ee75ff700 1 -- 172.21.15.101:0/1390623750 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ee01a4c70 con 0x7f0ee00ffad0 2023-07-18T03:35:50.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.285+0000 7f0ed67fc700 1 -- 172.21.15.101:0/1390623750 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f0edc0105a0 con 0x7f0ee00ffad0 2023-07-18T03:35:50.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.286+0000 7f0efaef9b80 1 Processor -- start 2023-07-18T03:35:50.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.286+0000 7f0efaef9b80 1 -- start start 2023-07-18T03:35:50.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.286+0000 7f0efaef9b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5648a2b5a2d0 0x5648a2b5a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:50.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.286+0000 7f0efaef9b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648a2b5abe0 0x5648a2b65590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:50.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.286+0000 7f0efaef9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5648a2b65ad0 0x5648a2b67eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:50.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.286+0000 7f0efaef9b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a2a60cb0 con 0x5648a2b5a2d0 2023-07-18T03:35:50.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.286+0000 7f0efaef9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a2ab3a40 con 0x5648a2b65ad0 2023-07-18T03:35:50.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.286+0000 7f0efaef9b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a2aaced0 con 0x5648a2b5abe0 2023-07-18T03:35:50.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.286+0000 7f0ed67fc700 1 --2- 172.21.15.101:0/1390623750 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0ecc0a12c0 0x7f0ecc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:50.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.286+0000 7f0ee539b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648a2b5abe0 0x5648a2b65590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:50.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.286+0000 7f0ee4b9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5648a2b65ad0 0x5648a2b67eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:50.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.286+0000 7f0ed67fc700 1 -- 172.21.15.101:0/1390623750 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f0edc014070 con 0x7f0ee00ffad0 2023-07-18T03:35:50.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.286+0000 7f0ee5b9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5648a2b5a2d0 0x5648a2b5a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:50.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.286+0000 7f0ee4b9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5648a2b65ad0 0x5648a2b67eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:41632/0 (socket says 172.21.15.101:41632) 2023-07-18T03:35:50.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.286+0000 7f0ee4b9a700 1 -- 172.21.15.101:0/1168239140 learned_addr learned my addr 172.21.15.101:0/1168239140 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:35:50.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.287+0000 7f0ee5b9c700 1 --2- 172.21.15.101:0/1390623750 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0ecc0a12c0 0x7f0ecc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:50.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.287+0000 7f0ee4b9a700 1 -- 172.21.15.101:0/1168239140 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648a2b5abe0 msgr2=0x5648a2b65590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:50.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.287+0000 7f0ee4b9a700 1 --2- 172.21.15.101:0/1168239140 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648a2b5abe0 0x5648a2b65590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:50.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.287+0000 7f0ee4b9a700 1 -- 172.21.15.101:0/1168239140 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5648a2b5a2d0 msgr2=0x5648a2b5a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:50.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.287+0000 7f0ee4b9a700 1 --2- 172.21.15.101:0/1168239140 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5648a2b5a2d0 0x5648a2b5a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:50.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.287+0000 7f0ee4b9a700 1 -- 172.21.15.101:0/1168239140 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648a28dade0 con 0x5648a2b65ad0 2023-07-18T03:35:50.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.287+0000 7f0ee539b700 1 --2- 172.21.15.101:0/1168239140 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648a2b5abe0 0x5648a2b65590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:35:50.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.287+0000 7f0ee4b9a700 1 --2- 172.21.15.101:0/1168239140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5648a2b65ad0 0x5648a2b67eb0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0007c90 tx=0x7f0ed00103d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=671c32b53c1af954 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:50.290 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.287+0000 7f0ee5b9c700 1 --2- 172.21.15.101:0/1390623750 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0ecc0a12c0 0x7f0ecc0a3770 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0074920 tx=0x7f0ed8006230 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:50.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.287+0000 7f0ecb7fe700 1 -- 172.21.15.101:0/1168239140 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed0010cb0 con 0x5648a2b65ad0 2023-07-18T03:35:50.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.287+0000 7f0ecb7fe700 1 -- 172.21.15.101:0/1168239140 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ed0004510 con 0x5648a2b65ad0 2023-07-18T03:35:50.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.287+0000 7f0ecb7fe700 1 -- 172.21.15.101:0/1168239140 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed0003ea0 con 0x5648a2b65ad0 2023-07-18T03:35:50.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.288+0000 7f0efaef9b80 1 -- 172.21.15.101:0/1168239140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5648a2b65ad0 msgr2=0x5648a2b67eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:50.291 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.288+0000 7f0efaef9b80 1 --2- 172.21.15.101:0/1168239140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5648a2b65ad0 0x5648a2b67eb0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0007c90 tx=0x7f0ed00103d0 comp rx=0 tx=0).stop 2023-07-18T03:35:50.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.288+0000 7f0efaef9b80 1 -- 172.21.15.101:0/1168239140 shutdown_connections 2023-07-18T03:35:50.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.288+0000 7f0efaef9b80 1 --2- 172.21.15.101:0/1168239140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5648a2b65ad0 0x5648a2b67eb0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:50.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.288+0000 7f0efaef9b80 1 --2- 172.21.15.101:0/1168239140 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5648a2b5a2d0 0x5648a2b5a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:50.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.288+0000 7f0efaef9b80 1 --2- 172.21.15.101:0/1168239140 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648a2b5abe0 0x5648a2b65590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:50.292 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.288+0000 7f0efaef9b80 1 -- 172.21.15.101:0/1168239140 >> 172.21.15.101:0/1168239140 conn(0x5648a2b50da0 msgr2=0x5648a2b566a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:35:50.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.288+0000 7f0efaef9b80 1 -- 172.21.15.101:0/1168239140 shutdown_connections 2023-07-18T03:35:50.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.288+0000 7f0efaef9b80 1 -- 172.21.15.101:0/1168239140 wait complete. 2023-07-18T03:35:50.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.289+0000 7f0efaef9b80 1 -- start start 2023-07-18T03:35:50.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.289+0000 7f0efaef9b80 1 Processor -- start 2023-07-18T03:35:50.295 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.290+0000 7f0efaef9b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648a2b5a2d0 0x5648a2b65510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:50.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.290+0000 7f0efaef9b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5648a2b5abe0 0x5648a2b62790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:50.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.290+0000 7f0efaef9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5648a2b65ad0 0x5648a2b62ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:50.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.290+0000 7f0efaef9b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a2a4cac0 con 0x5648a2b5abe0 2023-07-18T03:35:50.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.290+0000 7f0efaef9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a2a4c7c0 con 0x5648a2b65ad0 2023-07-18T03:35:50.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.290+0000 7f0efaef9b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a2930920 con 0x5648a2b5a2d0 2023-07-18T03:35:50.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.290+0000 7f0efaef9b80 10 client.0 _ll_register_callbacks cb 0x5648a2a4a760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:35:50.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.290+0000 7f0ee5b9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5648a2b65ad0 0x5648a2b62ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:50.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.290+0000 7f0ee539b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648a2b5a2d0 0x5648a2b65510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:50.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.290+0000 7f0ee4b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5648a2b5abe0 0x5648a2b62790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:50.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.290+0000 7f0ee5b9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5648a2b65ad0 0x5648a2b62ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:41638/0 (socket says 172.21.15.101:41638) 2023-07-18T03:35:50.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.290+0000 7f0ee5b9c700 1 -- 172.21.15.101:0/2553430905 learned_addr learned my addr 172.21.15.101:0/2553430905 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:35:50.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.290+0000 7f0ee539b700 1 -- 172.21.15.101:0/2553430905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5648a2b65ad0 msgr2=0x5648a2b62ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:50.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.290+0000 7f0ee539b700 1 --2- 172.21.15.101:0/2553430905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5648a2b65ad0 0x5648a2b62ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:50.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.290+0000 7f0ee539b700 1 -- 172.21.15.101:0/2553430905 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5648a2b5abe0 msgr2=0x5648a2b62790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:50.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.290+0000 7f0ee539b700 1 --2- 172.21.15.101:0/2553430905 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5648a2b5abe0 0x5648a2b62790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:50.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.290+0000 7f0ee539b700 1 -- 172.21.15.101:0/2553430905 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed00078c0 con 0x5648a2b5a2d0 2023-07-18T03:35:50.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.291+0000 7f0ee4b9a700 1 --2- 172.21.15.101:0/2553430905 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5648a2b5abe0 0x5648a2b62790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:35:50.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.291+0000 7f0ee539b700 1 --2- 172.21.15.101:0/2553430905 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648a2b5a2d0 0x5648a2b65510 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f0edc0ca5a0 tx=0x7f0edc01f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d54de09302718ec7 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:50.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.291+0000 7f0ec8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:35:50.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.291+0000 7f0ec8ff9700 1 -- 172.21.15.101:0/2553430905 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0edc099610 con 0x5648a2b5a2d0 2023-07-18T03:35:50.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.291+0000 7f0ec8ff9700 1 -- 172.21.15.101:0/2553430905 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(136..136 src has 1..136) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f0edc099f20 con 0x5648a2b5a2d0 2023-07-18T03:35:50.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.291+0000 7f0efaef9b80 1 -- 172.21.15.101:0/2553430905 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5648a2a064f0 con 0x5648a2b5a2d0 2023-07-18T03:35:50.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.291+0000 7f0ec8ff9700 1 -- 172.21.15.101:0/2553430905 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0edc09df50 con 0x5648a2b5a2d0 2023-07-18T03:35:50.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.291+0000 7f0efaef9b80 10 client.29308 fetch_fsmap learned FSMap version 49 2023-07-18T03:35:50.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.291+0000 7f0efaef9b80 1 -- 172.21.15.101:0/2553430905 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5648a28c3c60 con 0x5648a2b5a2d0 2023-07-18T03:35:50.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.291+0000 7f0ec8ff9700 1 -- 172.21.15.101:0/2553430905 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0edc093070 con 0x5648a2b5a2d0 2023-07-18T03:35:50.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.291+0000 7f0ec8ff9700 1 -- 172.21.15.101:0/2553430905 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 49) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0edc007ad0 con 0x5648a2b5a2d0 2023-07-18T03:35:50.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.291+0000 7f0efaef9b80 10 client.29308 fetch_fsmap finished waiting for FSMap version 49 2023-07-18T03:35:50.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.291+0000 7f0efaef9b80 10 client.29308 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:35:50.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.291+0000 7f0efaef9b80 20 client.29308 populate_metadata read hostname 'smithi101' 2023-07-18T03:35:50.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.292+0000 7f0efaef9b80 1 --2- 172.21.15.101:0/2553430905 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5648a2b75460 0x5648a2b77840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:50.302 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.292+0000 7f0efaef9b80 4 client.29308 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:35:50.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.292+0000 7f0efaef9b80 1 -- 172.21.15.101:0/2553430905 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5648a2aa5eb0 con 0x5648a2b75460 2023-07-18T03:35:50.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.292+0000 7f0ee4b9a700 1 --2- 172.21.15.101:0/2553430905 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5648a2b75460 0x5648a2b77840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:50.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.292+0000 7f0ec8ff9700 10 client.29308 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:35:50.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.292+0000 7f0ee4b9a700 1 --2- 172.21.15.101:0/2553430905 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5648a2b75460 0x5648a2b77840 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea1892fcbe5574bf server_cookie=d174f93ae75ca365 in_seq=0 out_seq=0 2023-07-18T03:35:50.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.293+0000 7f0ec8ff9700 1 -- 172.21.15.101:0/2553430905 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5648a2aa5eb0 con 0x5648a2b75460 2023-07-18T03:35:50.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.293+0000 7f0ec8ff9700 10 client.29308 handle_command_reply: tid=0 2023-07-18T03:35:50.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.293+0000 7f0efaef9b80 1 client.29308 shutdown 2023-07-18T03:35:50.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.293+0000 7f0efaef9b80 1 -- 172.21.15.101:0/2553430905 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648a2b5a2d0 msgr2=0x5648a2b65510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:50.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.293+0000 7f0efaef9b80 1 --2- 172.21.15.101:0/2553430905 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648a2b5a2d0 0x5648a2b65510 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f0edc0ca5a0 tx=0x7f0edc01f000 comp rx=0 tx=0).stop 2023-07-18T03:35:50.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.293+0000 7f0efaef9b80 1 -- 172.21.15.101:0/2553430905 shutdown_connections 2023-07-18T03:35:50.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.293+0000 7f0efaef9b80 1 --2- 172.21.15.101:0/2553430905 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5648a2b75460 0x5648a2b77840 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:50.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.293+0000 7f0efaef9b80 1 --2- 172.21.15.101:0/2553430905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5648a2b65ad0 0x5648a2b62ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:50.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.293+0000 7f0ec8ff9700 0 client.29308 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:35:50.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.293+0000 7f0efaef9b80 1 --2- 172.21.15.101:0/2553430905 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648a2b5a2d0 0x5648a2b65510 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:50.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.293+0000 7f0efaef9b80 1 --2- 172.21.15.101:0/2553430905 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5648a2b5abe0 0x5648a2b62790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:50.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.293+0000 7f0efaef9b80 1 -- 172.21.15.101:0/2553430905 >> 172.21.15.101:0/2553430905 conn(0x5648a2b50da0 msgr2=0x5648a2b56230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:35:50.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.293+0000 7f0efaef9b80 1 -- 172.21.15.101:0/2553430905 shutdown_connections 2023-07-18T03:35:50.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.293+0000 7f0efaef9b80 1 -- 172.21.15.101:0/2553430905 wait complete. 2023-07-18T03:35:50.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.294+0000 7f0efaef9b80 20 client.29308 trim_cache size 0 max 16384 2023-07-18T03:35:50.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.310+0000 7f0efaef9b80 1 Processor -- start 2023-07-18T03:35:50.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.311+0000 7f0efaef9b80 1 -- start start 2023-07-18T03:35:50.308 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.311+0000 7f0efaef9b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5648a2b5a2d0 0x5648a2b6c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:50.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.311+0000 7f0efaef9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5648a2b5abe0 0x5648a2b65040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:50.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.311+0000 7f0efaef9b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648a2b65ad0 0x5648a2b62030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:50.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.311+0000 7f0efaef9b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a2b5b1c0 con 0x5648a2b5a2d0 2023-07-18T03:35:50.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.311+0000 7f0efaef9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a2b6ca20 con 0x5648a2b5abe0 2023-07-18T03:35:50.309 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.311+0000 7f0efaef9b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a2b61d00 con 0x5648a2b65ad0 2023-07-18T03:35:50.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.311+0000 7f0ee4b9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5648a2b5abe0 0x5648a2b65040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:50.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.311+0000 7f0ee539b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5648a2b5a2d0 0x5648a2b6c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:50.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.311+0000 7f0ee4b9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5648a2b5abe0 0x5648a2b65040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:41652/0 (socket says 172.21.15.101:41652) 2023-07-18T03:35:50.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.311+0000 7f0ee4b9a700 1 -- 172.21.15.101:0/3549487656 learned_addr learned my addr 172.21.15.101:0/3549487656 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:35:50.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.311+0000 7f0ee5b9c700 1 --2- 172.21.15.101:0/3549487656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648a2b65ad0 0x5648a2b62030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:50.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.311+0000 7f0ee4b9a700 1 -- 172.21.15.101:0/3549487656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648a2b65ad0 msgr2=0x5648a2b62030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:50.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.311+0000 7f0ee4b9a700 1 --2- 172.21.15.101:0/3549487656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648a2b65ad0 0x5648a2b62030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:50.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.311+0000 7f0ee4b9a700 1 -- 172.21.15.101:0/3549487656 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5648a2b5a2d0 msgr2=0x5648a2b6c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:50.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.311+0000 7f0ee4b9a700 1 --2- 172.21.15.101:0/3549487656 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5648a2b5a2d0 0x5648a2b6c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:50.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.311+0000 7f0ee4b9a700 1 -- 172.21.15.101:0/3549487656 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648a2901250 con 0x5648a2b5abe0 2023-07-18T03:35:50.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.311+0000 7f0ee539b700 1 --2- 172.21.15.101:0/3549487656 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5648a2b5a2d0 0x5648a2b6c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:35:50.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.312+0000 7f0ee4b9a700 1 --2- 172.21.15.101:0/3549487656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5648a2b5abe0 0x5648a2b65040 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0000c00 tx=0x7f0ed00058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adb076fa01e7119e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:50.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.312+0000 7f0ecbfff700 1 -- 172.21.15.101:0/3549487656 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed001e040 con 0x5648a2b5abe0 2023-07-18T03:35:50.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.312+0000 7f0ecbfff700 1 -- 172.21.15.101:0/3549487656 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ed0004350 con 0x5648a2b5abe0 2023-07-18T03:35:50.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.312+0000 7f0ecbfff700 1 -- 172.21.15.101:0/3549487656 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed0003c40 con 0x5648a2b5abe0 2023-07-18T03:35:50.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.312+0000 7f0efaef9b80 1 -- 172.21.15.101:0/3549487656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5648a2b5abe0 msgr2=0x5648a2b65040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:50.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.312+0000 7f0efaef9b80 1 --2- 172.21.15.101:0/3549487656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5648a2b5abe0 0x5648a2b65040 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0000c00 tx=0x7f0ed00058e0 comp rx=0 tx=0).stop 2023-07-18T03:35:50.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.312+0000 7f0efaef9b80 1 -- 172.21.15.101:0/3549487656 shutdown_connections 2023-07-18T03:35:50.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.312+0000 7f0efaef9b80 1 --2- 172.21.15.101:0/3549487656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648a2b65ad0 0x5648a2b62030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:50.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.312+0000 7f0efaef9b80 1 --2- 172.21.15.101:0/3549487656 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5648a2b5a2d0 0x5648a2b6c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:50.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.312+0000 7f0efaef9b80 1 --2- 172.21.15.101:0/3549487656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5648a2b5abe0 0x5648a2b65040 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:50.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.312+0000 7f0efaef9b80 1 -- 172.21.15.101:0/3549487656 >> 172.21.15.101:0/3549487656 conn(0x5648a2b50da0 msgr2=0x5648a2b5da90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:35:50.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.313+0000 7f0efaef9b80 1 -- 172.21.15.101:0/3549487656 shutdown_connections 2023-07-18T03:35:50.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.313+0000 7f0efaef9b80 1 -- 172.21.15.101:0/3549487656 wait complete. 2023-07-18T03:35:50.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.313+0000 7f0efaef9b80 1 -- start start 2023-07-18T03:35:50.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.313+0000 7f0efaef9b80 1 Processor -- start 2023-07-18T03:35:50.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.314+0000 7f0efaef9b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648a2b5a2d0 0x5648a2b60100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:50.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.314+0000 7f0efaef9b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5648a2b5abe0 0x5648a2b5ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:50.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.314+0000 7f0efaef9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5648a2b65ad0 0x5648a2b5ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:50.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.314+0000 7f0efaef9b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a2b607f0 con 0x5648a2b5abe0 2023-07-18T03:35:50.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.314+0000 7f0efaef9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a2b5f490 con 0x5648a2b65ad0 2023-07-18T03:35:50.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.314+0000 7f0efaef9b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a2b5f650 con 0x5648a2b5a2d0 2023-07-18T03:35:50.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.314+0000 7f0efaef9b80 10 client.0 _ll_register_callbacks cb 0x7f0edc099a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:35:50.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.314+0000 7f0ee539b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648a2b5a2d0 0x5648a2b60100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:50.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.314+0000 7f0ee5b9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5648a2b65ad0 0x5648a2b5ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:50.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.314+0000 7f0ee539b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648a2b5a2d0 0x5648a2b60100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:60722/0 (socket says 172.21.15.101:60722) 2023-07-18T03:35:50.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.314+0000 7f0ee539b700 1 -- 172.21.15.101:0/3066526382 learned_addr learned my addr 172.21.15.101:0/3066526382 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:35:50.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.314+0000 7f0ee4b9a700 1 --2- 172.21.15.101:0/3066526382 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5648a2b5abe0 0x5648a2b5ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:50.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.315+0000 7f0ee539b700 1 -- 172.21.15.101:0/3066526382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5648a2b65ad0 msgr2=0x5648a2b5ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:50.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.315+0000 7f0ee539b700 1 --2- 172.21.15.101:0/3066526382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5648a2b65ad0 0x5648a2b5ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:50.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.315+0000 7f0ee539b700 1 -- 172.21.15.101:0/3066526382 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5648a2b5abe0 msgr2=0x5648a2b5ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:50.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.315+0000 7f0ee539b700 1 --2- 172.21.15.101:0/3066526382 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5648a2b5abe0 0x5648a2b5ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:50.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.315+0000 7f0ee539b700 1 -- 172.21.15.101:0/3066526382 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5648a28c3c60 con 0x5648a2b5a2d0 2023-07-18T03:35:50.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.315+0000 7f0ee539b700 1 --2- 172.21.15.101:0/3066526382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648a2b5a2d0 0x5648a2b60100 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f0edc096420 tx=0x7f0edc01cf00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0cb43429e3041da server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:50.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.315+0000 7f0ec8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:35:50.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.315+0000 7f0ec8ff9700 1 -- 172.21.15.101:0/3066526382 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0edc099f20 con 0x5648a2b5a2d0 2023-07-18T03:35:50.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.315+0000 7f0ec8ff9700 1 -- 172.21.15.101:0/3066526382 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(136..136 src has 1..136) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f0edc09df50 con 0x5648a2b5a2d0 2023-07-18T03:35:50.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.315+0000 7f0efaef9b80 1 -- 172.21.15.101:0/3066526382 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ed0004510 con 0x5648a2b5a2d0 2023-07-18T03:35:50.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.315+0000 7f0ec8ff9700 1 -- 172.21.15.101:0/3066526382 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0edc0a2370 con 0x5648a2b5a2d0 2023-07-18T03:35:50.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.315+0000 7f0efaef9b80 10 client.29314 fetch_fsmap learned FSMap version 49 2023-07-18T03:35:50.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.315+0000 7f0efaef9b80 1 -- 172.21.15.101:0/3066526382 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5648a28dade0 con 0x5648a2b5a2d0 2023-07-18T03:35:50.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.315+0000 7f0ec8ff9700 1 -- 172.21.15.101:0/3066526382 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ed0004510 con 0x5648a2b5a2d0 2023-07-18T03:35:50.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.315+0000 7f0ec8ff9700 1 -- 172.21.15.101:0/3066526382 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 49) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0edc095e30 con 0x5648a2b5a2d0 2023-07-18T03:35:50.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.315+0000 7f0efaef9b80 10 client.29314 fetch_fsmap finished waiting for FSMap version 49 2023-07-18T03:35:50.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.316+0000 7f0efaef9b80 10 client.29314 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:35:50.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.316+0000 7f0efaef9b80 20 client.29314 populate_metadata read hostname 'smithi101' 2023-07-18T03:35:50.323 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.316+0000 7f0efaef9b80 1 --2- 172.21.15.101:0/3066526382 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5648a2bca2d0 0x5648a2bcc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:50.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.316+0000 7f0efaef9b80 4 client.29314 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:35:50.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.316+0000 7f0efaef9b80 1 -- 172.21.15.101:0/3066526382 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5648a2a68050 con 0x5648a2bca2d0 2023-07-18T03:35:50.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.316+0000 7f0ee4b9a700 1 --2- 172.21.15.101:0/3066526382 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5648a2bca2d0 0x5648a2bcc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:50.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.316+0000 7f0ec8ff9700 10 client.29314 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:35:50.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.316+0000 7f0ee4b9a700 1 --2- 172.21.15.101:0/3066526382 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5648a2bca2d0 0x5648a2bcc6b0 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f0060f247b9d7b4 server_cookie=eea2923eabd8087b in_seq=0 out_seq=0 2023-07-18T03:35:50.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.316+0000 7f0ec8ff9700 1 -- 172.21.15.101:0/3066526382 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5648a2a68050 con 0x5648a2bca2d0 2023-07-18T03:35:50.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.316+0000 7f0ec8ff9700 10 client.29314 handle_command_reply: tid=0 2023-07-18T03:35:50.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.316+0000 7f0efaef9b80 1 client.29314 shutdown 2023-07-18T03:35:50.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.317+0000 7f0efaef9b80 1 -- 172.21.15.101:0/3066526382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648a2b5a2d0 msgr2=0x5648a2b60100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:50.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.317+0000 7f0efaef9b80 1 --2- 172.21.15.101:0/3066526382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648a2b5a2d0 0x5648a2b60100 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f0edc096420 tx=0x7f0edc01cf00 comp rx=0 tx=0).stop 2023-07-18T03:35:50.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.317+0000 7f0efaef9b80 1 -- 172.21.15.101:0/3066526382 shutdown_connections 2023-07-18T03:35:50.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.317+0000 7f0efaef9b80 1 --2- 172.21.15.101:0/3066526382 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x5648a2bca2d0 0x5648a2bcc6b0 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:50.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.317+0000 7f0efaef9b80 1 --2- 172.21.15.101:0/3066526382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5648a2b65ad0 0x5648a2b5ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:50.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.317+0000 7f0ec8ff9700 0 client.29314 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:35:50.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.317+0000 7f0efaef9b80 1 --2- 172.21.15.101:0/3066526382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648a2b5a2d0 0x5648a2b60100 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:50.327 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.317+0000 7f0efaef9b80 1 --2- 172.21.15.101:0/3066526382 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5648a2b5abe0 0x5648a2b5ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:50.327 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.317+0000 7f0efaef9b80 1 -- 172.21.15.101:0/3066526382 >> 172.21.15.101:0/3066526382 conn(0x5648a2b50da0 msgr2=0x5648a2b61fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:35:50.327 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.317+0000 7f0efaef9b80 1 -- 172.21.15.101:0/3066526382 shutdown_connections 2023-07-18T03:35:50.327 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.317+0000 7f0efaef9b80 1 -- 172.21.15.101:0/3066526382 wait complete. 2023-07-18T03:35:50.327 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.317+0000 7f0efaef9b80 20 client.29314 trim_cache size 0 max 16384 2023-07-18T03:35:50.328 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:35:50.328 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:35:50.328 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:35:50.329 INFO:teuthology.orchestra.run.smithi101.stdout: "0f2f19e0-0429-465c-935e-223eb09490c3": { 2023-07-18T03:35:50.329 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:35:50.329 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "0f2f19e0-0429-465c-935e-223eb09490c3", 2023-07-18T03:35:50.329 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:35:50.329 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:35:50.330 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:35:50.330 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:35:50.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.317+0000 7f0ee75ff700 1 -- 172.21.15.101:0/1390623750 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0ecc0a12c0 msgr2=0x7f0ecc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:50.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.317+0000 7f0ee75ff700 1 --2- 172.21.15.101:0/1390623750 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0ecc0a12c0 0x7f0ecc0a3770 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0074920 tx=0x7f0ed8006230 comp rx=0 tx=0).stop 2023-07-18T03:35:50.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.317+0000 7f0ee75ff700 1 -- 172.21.15.101:0/1390623750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ee00ffad0 msgr2=0x7f0ee0072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:50.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.317+0000 7f0ee75ff700 1 --2- 172.21.15.101:0/1390623750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ee00ffad0 0x7f0ee0072810 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f0edc00e9a0 tx=0x7f0edc00ecb0 comp rx=0 tx=0).stop 2023-07-18T03:35:50.331 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.318+0000 7f0ee75ff700 1 -- 172.21.15.101:0/1390623750 shutdown_connections 2023-07-18T03:35:50.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.318+0000 7f0ee75ff700 1 --2- 172.21.15.101:0/1390623750 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0ecc0a12c0 0x7f0ecc0a3770 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:50.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.318+0000 7f0ee75ff700 1 --2- 172.21.15.101:0/1390623750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ee0105ae0 0x7f0ee01a3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:50.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.318+0000 7f0ee75ff700 1 --2- 172.21.15.101:0/1390623750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ee00ffad0 0x7f0ee0072810 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:50.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.318+0000 7f0ee75ff700 1 --2- 172.21.15.101:0/1390623750 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0ee0100480 0x7f0ee0072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:50.332 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.318+0000 7f0ee75ff700 1 -- 172.21.15.101:0/1390623750 >> 172.21.15.101:0/1390623750 conn(0x7f0ee00fb5f0 msgr2=0x7f0ee01035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:35:50.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.318+0000 7f0ee75ff700 1 -- 172.21.15.101:0/1390623750 shutdown_connections 2023-07-18T03:35:50.333 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:35:50.318+0000 7f0ee75ff700 1 -- 172.21.15.101:0/1390623750 wait complete. 2023-07-18T03:35:50.334 INFO:tasks.cephfs.filesystem:scrub status for tag:0f2f19e0-0429-465c-935e-223eb09490c3 - {'path': '/', 'tag': '0f2f19e0-0429-465c-935e-223eb09490c3', 'options': 'recursive,force'} 2023-07-18T03:35:51.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:35:51 smithi101 ceph-mon[126379]: pgmap v895: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:35:51.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:35:51 smithi155 ceph-mon[128885]: pgmap v895: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:35:51.953 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:35:51 smithi179 ceph-mon[129229]: pgmap v895: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:35:52.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:35:52 smithi101 ceph-mon[126379]: pgmap v896: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:35:52.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:35:52 smithi155 ceph-mon[128885]: pgmap v896: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:35:52.953 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:35:52 smithi179 ceph-mon[129229]: pgmap v896: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:35:55.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:35:55 smithi179 ceph-mon[129229]: pgmap v897: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:35:55.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:35:55 smithi101 ceph-mon[126379]: pgmap v897: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:35:55.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:35:55 smithi155 ceph-mon[128885]: pgmap v897: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:35:57.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:35:57 smithi179 ceph-mon[129229]: pgmap v898: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:35:57.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:35:57 smithi101 ceph-mon[126379]: pgmap v898: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:35:57.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:35:57 smithi155 ceph-mon[128885]: pgmap v898: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:35:59.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:35:59 smithi179 ceph-mon[129229]: pgmap v899: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:35:59.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:35:59 smithi101 ceph-mon[126379]: pgmap v899: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:35:59.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:35:59 smithi155 ceph-mon[128885]: pgmap v899: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:36:01.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:36:01 smithi179 ceph-mon[129229]: pgmap v900: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:36:01.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:36:01 smithi101 ceph-mon[126379]: pgmap v900: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:36:01.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:36:01 smithi155 ceph-mon[128885]: pgmap v900: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:36:03.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:36:03 smithi179 ceph-mon[129229]: pgmap v901: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-07-18T03:36:03.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:36:03 smithi101 ceph-mon[126379]: pgmap v901: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-07-18T03:36:03.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:36:03 smithi155 ceph-mon[128885]: pgmap v901: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-07-18T03:36:05.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:36:05 smithi179 ceph-mon[129229]: pgmap v902: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-07-18T03:36:05.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:36:05 smithi101 ceph-mon[126379]: pgmap v902: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-07-18T03:36:05.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:36:05 smithi155 ceph-mon[128885]: pgmap v902: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-07-18T03:36:07.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:36:07 smithi179 ceph-mon[129229]: pgmap v903: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-07-18T03:36:07.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:36:07 smithi101 ceph-mon[126379]: pgmap v903: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-07-18T03:36:07.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:36:07 smithi155 ceph-mon[128885]: pgmap v903: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-07-18T03:36:09.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:36:09 smithi179 ceph-mon[129229]: pgmap v904: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-07-18T03:36:09.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:36:09 smithi101 ceph-mon[126379]: pgmap v904: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-07-18T03:36:09.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:36:09 smithi155 ceph-mon[128885]: pgmap v904: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-07-18T03:36:11.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:36:11 smithi179 ceph-mon[129229]: pgmap v905: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-07-18T03:36:11.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:36:11 smithi179 ceph-mon[129229]: osdmap e137: 12 total, 12 up, 12 in 2023-07-18T03:36:11.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:36:11 smithi101 ceph-mon[126379]: pgmap v905: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-07-18T03:36:11.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:36:11 smithi101 ceph-mon[126379]: osdmap e137: 12 total, 12 up, 12 in 2023-07-18T03:36:11.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:36:11 smithi155 ceph-mon[128885]: pgmap v905: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-07-18T03:36:11.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:36:11 smithi155 ceph-mon[128885]: osdmap e137: 12 total, 12 up, 12 in 2023-07-18T03:36:13.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:36:13 smithi179 ceph-mon[129229]: pgmap v907: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 KiB/s wr, 21 op/s 2023-07-18T03:36:13.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:36:13 smithi101 ceph-mon[126379]: pgmap v907: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 KiB/s wr, 21 op/s 2023-07-18T03:36:13.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:36:13 smithi155 ceph-mon[128885]: pgmap v907: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 KiB/s wr, 21 op/s 2023-07-18T03:36:15.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:36:15 smithi179 ceph-mon[129229]: pgmap v908: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 KiB/s wr, 21 op/s 2023-07-18T03:36:15.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:36:15 smithi101 ceph-mon[126379]: pgmap v908: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 KiB/s wr, 21 op/s 2023-07-18T03:36:15.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:36:15 smithi155 ceph-mon[128885]: pgmap v908: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 KiB/s wr, 21 op/s 2023-07-18T03:36:17.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:36:17 smithi179 ceph-mon[129229]: pgmap v909: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:36:17.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:36:17 smithi101 ceph-mon[126379]: pgmap v909: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:36:17.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:36:17 smithi155 ceph-mon[128885]: pgmap v909: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:36:19.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:36:19 smithi179 ceph-mon[129229]: pgmap v910: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:36:19.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:36:19 smithi179 ceph-mon[129229]: osdmap e138: 12 total, 12 up, 12 in 2023-07-18T03:36:19.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:36:19 smithi101 ceph-mon[126379]: pgmap v910: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:36:19.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:36:19 smithi101 ceph-mon[126379]: osdmap e138: 12 total, 12 up, 12 in 2023-07-18T03:36:19.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:36:19 smithi155 ceph-mon[128885]: pgmap v910: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:36:19.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:36:19 smithi155 ceph-mon[128885]: osdmap e138: 12 total, 12 up, 12 in 2023-07-18T03:36:20.335 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:36:20.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.552+0000 7f64291ff700 1 -- 172.21.15.101:0/1341516845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6424100490 msgr2=0x7f642410c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:20.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.552+0000 7f64291ff700 1 --2- 172.21.15.101:0/1341516845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6424100490 0x7f642410c920 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f641c009f90 tx=0x7f641c00b330 comp rx=0 tx=0).stop 2023-07-18T03:36:20.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.552+0000 7f64291ff700 1 -- 172.21.15.101:0/1341516845 shutdown_connections 2023-07-18T03:36:20.549 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.552+0000 7f64291ff700 1 --2- 172.21.15.101:0/1341516845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6424100490 0x7f642410c920 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.549 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.552+0000 7f64291ff700 1 --2- 172.21.15.101:0/1341516845 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6424105af0 0x7f6424105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.549 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.552+0000 7f64291ff700 1 --2- 172.21.15.101:0/1341516845 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f64240ffae0 0x7f64240fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.549 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.552+0000 7f64291ff700 1 -- 172.21.15.101:0/1341516845 >> 172.21.15.101:0/1341516845 conn(0x7f64240fb5e0 msgr2=0x7f64240fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:36:20.549 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.552+0000 7f64291ff700 1 -- 172.21.15.101:0/1341516845 shutdown_connections 2023-07-18T03:36:20.550 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.552+0000 7f64291ff700 1 -- 172.21.15.101:0/1341516845 wait complete. 2023-07-18T03:36:20.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.553+0000 7f64291ff700 1 Processor -- start 2023-07-18T03:36:20.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.553+0000 7f64291ff700 1 -- start start 2023-07-18T03:36:20.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.553+0000 7f64291ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64240ffae0 0x7f6424197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:20.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.553+0000 7f64291ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6424100490 0x7f6424198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:20.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.553+0000 7f64291ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6424105af0 0x7f642419c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:20.553 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.553+0000 7f64291ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642419cf00 con 0x7f6424105af0 2023-07-18T03:36:20.553 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.553+0000 7f64291ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642419d070 con 0x7f6424100490 2023-07-18T03:36:20.553 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.553+0000 7f64291ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642419d230 con 0x7f64240ffae0 2023-07-18T03:36:20.553 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.554+0000 7f6422d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64240ffae0 0x7f6424197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:20.553 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.554+0000 7f6422d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64240ffae0 0x7f6424197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:46348/0 (socket says 172.21.15.101:46348) 2023-07-18T03:36:20.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.554+0000 7f6422d9d700 1 -- 172.21.15.101:0/1303634663 learned_addr learned my addr 172.21.15.101:0/1303634663 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:36:20.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.554+0000 7f641bfff700 1 --2- 172.21.15.101:0/1303634663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6424100490 0x7f6424198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:20.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.554+0000 7f642359e700 1 --2- 172.21.15.101:0/1303634663 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6424105af0 0x7f642419c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:20.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.554+0000 7f641bfff700 1 -- 172.21.15.101:0/1303634663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64240ffae0 msgr2=0x7f6424197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:20.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.554+0000 7f641bfff700 1 --2- 172.21.15.101:0/1303634663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64240ffae0 0x7f6424197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.555 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.554+0000 7f641bfff700 1 -- 172.21.15.101:0/1303634663 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6424105af0 msgr2=0x7f642419c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:20.555 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.554+0000 7f641bfff700 1 --2- 172.21.15.101:0/1303634663 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6424105af0 0x7f642419c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.555 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.554+0000 7f641bfff700 1 -- 172.21.15.101:0/1303634663 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64140095f0 con 0x7f6424100490 2023-07-18T03:36:20.555 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.554+0000 7f6422d9d700 1 --2- 172.21.15.101:0/1303634663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64240ffae0 0x7f6424197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:36:20.555 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.554+0000 7f641bfff700 1 --2- 172.21.15.101:0/1303634663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6424100490 0x7f6424198460 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f640c00e9f0 tx=0x7f640c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:36:20.556 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.554+0000 7f6420d99700 1 -- 172.21.15.101:0/1303634663 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640c00c9c0 con 0x7f6424100490 2023-07-18T03:36:20.556 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.555+0000 7f64291ff700 1 -- 172.21.15.101:0/1303634663 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f641c009c40 con 0x7f6424100490 2023-07-18T03:36:20.556 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.555+0000 7f6420d99700 1 -- 172.21.15.101:0/1303634663 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f640c010430 con 0x7f6424100490 2023-07-18T03:36:20.556 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.555+0000 7f6420d99700 1 -- 172.21.15.101:0/1303634663 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640c018680 con 0x7f6424100490 2023-07-18T03:36:20.557 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.555+0000 7f64291ff700 1 -- 172.21.15.101:0/1303634663 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64241a4b70 con 0x7f6424100490 2023-07-18T03:36:20.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.558+0000 7f6420d99700 1 -- 172.21.15.101:0/1303634663 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f640c010860 con 0x7f6424100490 2023-07-18T03:36:20.564 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.559+0000 7f643cb7db80 1 Processor -- start 2023-07-18T03:36:20.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.559+0000 7f643cb7db80 1 -- start start 2023-07-18T03:36:20.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.559+0000 7f643cb7db80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c3186d2d0 0x557c3186d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:20.565 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.559+0000 7f643cb7db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c3186dbe0 0x557c31878590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:20.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.559+0000 7f643cb7db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c31878ad0 0x557c3187aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:20.566 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.559+0000 7f643cb7db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c31773cb0 con 0x557c3186dbe0 2023-07-18T03:36:20.567 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.559+0000 7f643cb7db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c317c6a40 con 0x557c31878ad0 2023-07-18T03:36:20.567 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.559+0000 7f643cb7db80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c317bfed0 con 0x557c3186d2d0 2023-07-18T03:36:20.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.559+0000 7f6420d99700 1 --2- 172.21.15.101:0/1303634663 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f64100a12c0 0x7f64100a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:20.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.560+0000 7f641bfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c31878ad0 0x557c3187aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:20.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.560+0000 7f641bfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c31878ad0 0x557c3187aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47478/0 (socket says 172.21.15.101:47478) 2023-07-18T03:36:20.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.560+0000 7f6420d99700 1 -- 172.21.15.101:0/1303634663 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f640c014070 con 0x7f6424100490 2023-07-18T03:36:20.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.560+0000 7f641bfff700 1 -- 172.21.15.101:0/2851813016 learned_addr learned my addr 172.21.15.101:0/2851813016 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:36:20.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.560+0000 7f642359e700 1 --2- 172.21.15.101:0/2851813016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c3186d2d0 0x557c3186d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:20.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.560+0000 7f642359e700 1 --2- 172.21.15.101:0/1303634663 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f64100a12c0 0x7f64100a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:20.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.560+0000 7f641bfff700 1 -- 172.21.15.101:0/2851813016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c3186d2d0 msgr2=0x557c3186d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:20.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.560+0000 7f641bfff700 1 --2- 172.21.15.101:0/2851813016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c3186d2d0 0x557c3186d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.560+0000 7f641bfff700 1 -- 172.21.15.101:0/2851813016 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c3186dbe0 msgr2=0x557c31878590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:20.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.560+0000 7f641bfff700 1 --2- 172.21.15.101:0/2851813016 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c3186dbe0 0x557c31878590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.560+0000 7f641bfff700 1 -- 172.21.15.101:0/2851813016 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c315edde0 con 0x557c31878ad0 2023-07-18T03:36:20.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.560+0000 7f641bfff700 1 --2- 172.21.15.101:0/2851813016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c31878ad0 0x557c3187aeb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f640c00ae70 tx=0x7f640c069370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee41049324265204 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:36:20.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.560+0000 7f64197fa700 1 -- 172.21.15.101:0/2851813016 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640c09df20 con 0x557c31878ad0 2023-07-18T03:36:20.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.560+0000 7f64197fa700 1 -- 172.21.15.101:0/2851813016 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f640c0a2340 con 0x557c31878ad0 2023-07-18T03:36:20.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.560+0000 7f642359e700 1 --2- 172.21.15.101:0/1303634663 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f64100a12c0 0x7f64100a3770 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f641c0060b0 tx=0x7f641c009700 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:36:20.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.560+0000 7f64197fa700 1 -- 172.21.15.101:0/2851813016 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640c09a150 con 0x557c31878ad0 2023-07-18T03:36:20.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.561+0000 7f643cb7db80 1 -- 172.21.15.101:0/2851813016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c31878ad0 msgr2=0x557c3187aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:20.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.561+0000 7f643cb7db80 1 --2- 172.21.15.101:0/2851813016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c31878ad0 0x557c3187aeb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f640c00ae70 tx=0x7f640c069370 comp rx=0 tx=0).stop 2023-07-18T03:36:20.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.561+0000 7f643cb7db80 1 -- 172.21.15.101:0/2851813016 shutdown_connections 2023-07-18T03:36:20.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.561+0000 7f643cb7db80 1 --2- 172.21.15.101:0/2851813016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c31878ad0 0x557c3187aeb0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.561+0000 7f643cb7db80 1 --2- 172.21.15.101:0/2851813016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c3186d2d0 0x557c3186d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.561+0000 7f643cb7db80 1 --2- 172.21.15.101:0/2851813016 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c3186dbe0 0x557c31878590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.561+0000 7f643cb7db80 1 -- 172.21.15.101:0/2851813016 >> 172.21.15.101:0/2851813016 conn(0x557c31863da0 msgr2=0x557c318696a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:36:20.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.561+0000 7f643cb7db80 1 -- 172.21.15.101:0/2851813016 shutdown_connections 2023-07-18T03:36:20.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.561+0000 7f643cb7db80 1 -- 172.21.15.101:0/2851813016 wait complete. 2023-07-18T03:36:20.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.562+0000 7f643cb7db80 1 -- start start 2023-07-18T03:36:20.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.562+0000 7f643cb7db80 1 Processor -- start 2023-07-18T03:36:20.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.563+0000 7f643cb7db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c3186d2d0 0x557c31878550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:20.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.563+0000 7f643cb7db80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c3186dbe0 0x557c318757d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:20.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.563+0000 7f643cb7db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c31878ad0 0x557c31875ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:20.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.563+0000 7f643cb7db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c3175fac0 con 0x557c31878ad0 2023-07-18T03:36:20.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.563+0000 7f643cb7db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c3175f7c0 con 0x557c3186d2d0 2023-07-18T03:36:20.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.563+0000 7f643cb7db80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c31643920 con 0x557c3186dbe0 2023-07-18T03:36:20.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.563+0000 7f643cb7db80 10 client.0 _ll_register_callbacks cb 0x557c3175d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:36:20.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.563+0000 7f6422d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c3186d2d0 0x557c31878550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:20.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.563+0000 7f6422d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c3186d2d0 0x557c31878550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47480/0 (socket says 172.21.15.101:47480) 2023-07-18T03:36:20.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.563+0000 7f641bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c3186dbe0 0x557c318757d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:20.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.563+0000 7f6422d9d700 1 -- 172.21.15.101:0/3232891026 learned_addr learned my addr 172.21.15.101:0/3232891026 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:36:20.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.563+0000 7f642359e700 1 --2- 172.21.15.101:0/3232891026 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c31878ad0 0x557c31875ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:20.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.563+0000 7f641bfff700 1 -- 172.21.15.101:0/3232891026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c3186d2d0 msgr2=0x557c31878550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:20.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.563+0000 7f641bfff700 1 --2- 172.21.15.101:0/3232891026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c3186d2d0 0x557c31878550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.563+0000 7f641bfff700 1 -- 172.21.15.101:0/3232891026 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c31878ad0 msgr2=0x557c31875ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:20.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.563+0000 7f641bfff700 1 --2- 172.21.15.101:0/3232891026 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c31878ad0 0x557c31875ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.563+0000 7f641bfff700 1 -- 172.21.15.101:0/3232891026 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64241a4b70 con 0x557c3186dbe0 2023-07-18T03:36:20.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.563+0000 7f6422d9d700 1 --2- 172.21.15.101:0/3232891026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c3186d2d0 0x557c31878550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:36:20.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.563+0000 7f641bfff700 1 --2- 172.21.15.101:0/3232891026 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c3186dbe0 0x557c318757d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f640c096960 tx=0x7f640c0107b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e736f691f62faf3 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:36:20.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.563+0000 7f640a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:36:20.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.564+0000 7f640a7fc700 1 -- 172.21.15.101:0/3232891026 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640c01c1c0 con 0x557c3186dbe0 2023-07-18T03:36:20.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.564+0000 7f640a7fc700 1 -- 172.21.15.101:0/3232891026 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f640c09df20 con 0x557c3186dbe0 2023-07-18T03:36:20.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.564+0000 7f643cb7db80 1 -- 172.21.15.101:0/3232891026 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c317194f0 con 0x557c3186dbe0 2023-07-18T03:36:20.580 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.564+0000 7f640a7fc700 1 -- 172.21.15.101:0/3232891026 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f640c0691a0 con 0x557c3186dbe0 2023-07-18T03:36:20.581 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.564+0000 7f643cb7db80 10 client.29326 fetch_fsmap learned FSMap version 49 2023-07-18T03:36:20.581 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.564+0000 7f643cb7db80 1 -- 172.21.15.101:0/3232891026 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x557c315d6c60 con 0x557c3186dbe0 2023-07-18T03:36:20.581 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.564+0000 7f640a7fc700 1 -- 172.21.15.101:0/3232891026 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557c317194f0 con 0x557c3186dbe0 2023-07-18T03:36:20.581 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.564+0000 7f640a7fc700 1 -- 172.21.15.101:0/3232891026 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 49) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f640c094b00 con 0x557c3186dbe0 2023-07-18T03:36:20.581 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.564+0000 7f643cb7db80 10 client.29326 fetch_fsmap finished waiting for FSMap version 49 2023-07-18T03:36:20.582 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.564+0000 7f643cb7db80 10 client.29326 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:36:20.582 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.564+0000 7f643cb7db80 20 client.29326 populate_metadata read hostname 'smithi101' 2023-07-18T03:36:20.582 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.564+0000 7f643cb7db80 1 --2- 172.21.15.101:0/3232891026 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557c31888030 0x557c3188a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:20.582 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.564+0000 7f643cb7db80 4 client.29326 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:36:20.582 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.564+0000 7f643cb7db80 1 -- 172.21.15.101:0/3232891026 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c317b8eb0 con 0x557c31888030 2023-07-18T03:36:20.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.565+0000 7f6422d9d700 1 --2- 172.21.15.101:0/3232891026 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557c31888030 0x557c3188a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:20.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.565+0000 7f6422d9d700 1 --2- 172.21.15.101:0/3232891026 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557c31888030 0x557c3188a440 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93e8b11353e28d66 server_cookie=850007fff61ca3f8 in_seq=0 out_seq=0 2023-07-18T03:36:20.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.565+0000 7f640a7fc700 10 client.29326 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:36:20.583 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.566+0000 7f640a7fc700 1 -- 172.21.15.101:0/3232891026 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557c317b8eb0 con 0x557c31888030 2023-07-18T03:36:20.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.566+0000 7f640a7fc700 10 client.29326 handle_command_reply: tid=0 2023-07-18T03:36:20.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.566+0000 7f643cb7db80 1 client.29326 shutdown 2023-07-18T03:36:20.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.566+0000 7f643cb7db80 1 -- 172.21.15.101:0/3232891026 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c3186dbe0 msgr2=0x557c318757d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:20.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.566+0000 7f643cb7db80 1 --2- 172.21.15.101:0/3232891026 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c3186dbe0 0x557c318757d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f640c096960 tx=0x7f640c0107b0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.584 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.566+0000 7f643cb7db80 1 -- 172.21.15.101:0/3232891026 shutdown_connections 2023-07-18T03:36:20.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.566+0000 7f643cb7db80 1 --2- 172.21.15.101:0/3232891026 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557c31888030 0x557c3188a440 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.566+0000 7f643cb7db80 1 --2- 172.21.15.101:0/3232891026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c3186d2d0 0x557c31878550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.566+0000 7f643cb7db80 1 --2- 172.21.15.101:0/3232891026 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c31878ad0 0x557c31875ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.566+0000 7f643cb7db80 1 --2- 172.21.15.101:0/3232891026 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c3186dbe0 0x557c318757d0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.585 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.566+0000 7f640a7fc700 0 client.29326 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:36:20.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.566+0000 7f643cb7db80 1 -- 172.21.15.101:0/3232891026 >> 172.21.15.101:0/3232891026 conn(0x557c31863da0 msgr2=0x557c31869230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:36:20.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.566+0000 7f643cb7db80 1 -- 172.21.15.101:0/3232891026 shutdown_connections 2023-07-18T03:36:20.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.566+0000 7f643cb7db80 1 -- 172.21.15.101:0/3232891026 wait complete. 2023-07-18T03:36:20.586 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.567+0000 7f643cb7db80 20 client.29326 trim_cache size 0 max 16384 2023-07-18T03:36:20.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.583+0000 7f643cb7db80 1 Processor -- start 2023-07-18T03:36:20.590 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.584+0000 7f643cb7db80 1 -- start start 2023-07-18T03:36:20.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.584+0000 7f643cb7db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c3186d2d0 0x557c318780f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:20.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.584+0000 7f643cb7db80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c3186dbe0 0x557c31870af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:20.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.584+0000 7f643cb7db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c31878ad0 0x557c31871030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:20.591 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.584+0000 7f643cb7db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c317194f0 con 0x557c3186d2d0 2023-07-18T03:36:20.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.584+0000 7f643cb7db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c3186a540 con 0x557c31878ad0 2023-07-18T03:36:20.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.584+0000 7f643cb7db80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c3186e350 con 0x557c3186dbe0 2023-07-18T03:36:20.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.584+0000 7f642359e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c31878ad0 0x557c31871030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:20.592 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.584+0000 7f6422d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c3186d2d0 0x557c318780f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:20.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.584+0000 7f642359e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c31878ad0 0x557c31871030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47486/0 (socket says 172.21.15.101:47486) 2023-07-18T03:36:20.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.584+0000 7f642359e700 1 -- 172.21.15.101:0/2170038678 learned_addr learned my addr 172.21.15.101:0/2170038678 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:36:20.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.584+0000 7f641bfff700 1 --2- 172.21.15.101:0/2170038678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c3186dbe0 0x557c31870af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:20.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.585+0000 7f642359e700 1 -- 172.21.15.101:0/2170038678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c3186dbe0 msgr2=0x557c31870af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:20.593 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.585+0000 7f642359e700 1 --2- 172.21.15.101:0/2170038678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c3186dbe0 0x557c31870af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.585+0000 7f642359e700 1 -- 172.21.15.101:0/2170038678 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c3186d2d0 msgr2=0x557c318780f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:20.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.585+0000 7f642359e700 1 --2- 172.21.15.101:0/2170038678 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c3186d2d0 0x557c318780f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.585+0000 7f642359e700 1 -- 172.21.15.101:0/2170038678 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c31614250 con 0x557c31878ad0 2023-07-18T03:36:20.594 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.585+0000 7f6422d9d700 1 --2- 172.21.15.101:0/2170038678 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c3186d2d0 0x557c318780f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:36:20.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.585+0000 7f642359e700 1 --2- 172.21.15.101:0/2170038678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c31878ad0 0x557c31871030 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f641c015000 tx=0x7f641c006020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=686403e95aeb0fac server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:36:20.595 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.585+0000 7f6419ffb700 1 -- 172.21.15.101:0/2170038678 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641c00e040 con 0x557c31878ad0 2023-07-18T03:36:20.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.585+0000 7f6419ffb700 1 -- 172.21.15.101:0/2170038678 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f641c028040 con 0x557c31878ad0 2023-07-18T03:36:20.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.585+0000 7f6419ffb700 1 -- 172.21.15.101:0/2170038678 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641c0057c0 con 0x557c31878ad0 2023-07-18T03:36:20.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.586+0000 7f643cb7db80 1 -- 172.21.15.101:0/2170038678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c31878ad0 msgr2=0x557c31871030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:20.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.586+0000 7f643cb7db80 1 --2- 172.21.15.101:0/2170038678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c31878ad0 0x557c31871030 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f641c015000 tx=0x7f641c006020 comp rx=0 tx=0).stop 2023-07-18T03:36:20.596 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.586+0000 7f643cb7db80 1 -- 172.21.15.101:0/2170038678 shutdown_connections 2023-07-18T03:36:20.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.586+0000 7f643cb7db80 1 --2- 172.21.15.101:0/2170038678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c31878ad0 0x557c31871030 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.586+0000 7f643cb7db80 1 --2- 172.21.15.101:0/2170038678 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c3186d2d0 0x557c318780f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.597 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.586+0000 7f643cb7db80 1 --2- 172.21.15.101:0/2170038678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c3186dbe0 0x557c31870af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.586+0000 7f643cb7db80 1 -- 172.21.15.101:0/2170038678 >> 172.21.15.101:0/2170038678 conn(0x557c31863da0 msgr2=0x557c31867e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:36:20.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.586+0000 7f643cb7db80 1 -- 172.21.15.101:0/2170038678 shutdown_connections 2023-07-18T03:36:20.598 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.586+0000 7f643cb7db80 1 -- 172.21.15.101:0/2170038678 wait complete. 2023-07-18T03:36:20.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.586+0000 7f643cb7db80 1 -- start start 2023-07-18T03:36:20.599 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.586+0000 7f643cb7db80 1 Processor -- start 2023-07-18T03:36:20.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.587+0000 7f643cb7db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c3186d2d0 0x557c3187f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:20.600 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.587+0000 7f643cb7db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c3186dbe0 0x557c3187d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:20.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.587+0000 7f643cb7db80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c31878ad0 0x557c3187db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:20.601 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.587+0000 7f643cb7db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c3187e0b0 con 0x557c3186dbe0 2023-07-18T03:36:20.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.587+0000 7f643cb7db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c3187e1f0 con 0x557c3186d2d0 2023-07-18T03:36:20.602 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.587+0000 7f643cb7db80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c3187e3b0 con 0x557c31878ad0 2023-07-18T03:36:20.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.587+0000 7f643cb7db80 10 client.0 _ll_register_callbacks cb 0x7f640c0998d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:36:20.603 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.587+0000 7f6422d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c3186d2d0 0x557c3187f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:20.604 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.587+0000 7f6422d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c3186d2d0 0x557c3187f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47502/0 (socket says 172.21.15.101:47502) 2023-07-18T03:36:20.604 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.587+0000 7f6422d9d700 1 -- 172.21.15.101:0/2430509396 learned_addr learned my addr 172.21.15.101:0/2430509396 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:36:20.604 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.587+0000 7f641bfff700 1 --2- 172.21.15.101:0/2430509396 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c3186dbe0 0x557c3187d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:20.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.587+0000 7f642359e700 1 --2- 172.21.15.101:0/2430509396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c31878ad0 0x557c3187db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:20.605 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.587+0000 7f6422d9d700 1 -- 172.21.15.101:0/2430509396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c31878ad0 msgr2=0x557c3187db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:20.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.587+0000 7f6422d9d700 1 --2- 172.21.15.101:0/2430509396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c31878ad0 0x557c3187db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.587+0000 7f6422d9d700 1 -- 172.21.15.101:0/2430509396 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c3186dbe0 msgr2=0x557c3187d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:20.606 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.588+0000 7f6422d9d700 1 --2- 172.21.15.101:0/2430509396 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c3186dbe0 0x557c3187d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.588+0000 7f6422d9d700 1 -- 172.21.15.101:0/2430509396 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64140043e0 con 0x557c3186d2d0 2023-07-18T03:36:20.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.588+0000 7f642359e700 1 --2- 172.21.15.101:0/2430509396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c31878ad0 0x557c3187db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:36:20.607 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.588+0000 7f641bfff700 1 --2- 172.21.15.101:0/2430509396 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c3186dbe0 0x557c3187d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:36:20.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.588+0000 7f6422d9d700 1 --2- 172.21.15.101:0/2430509396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c3186d2d0 0x557c3187f370 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6414005d10 tx=0x7f641401d960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5301fdc3036f7f1c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:36:20.608 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.588+0000 7f640a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:36:20.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.588+0000 7f640a7fc700 1 -- 172.21.15.101:0/2430509396 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6414016040 con 0x557c3186d2d0 2023-07-18T03:36:20.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.588+0000 7f640a7fc700 1 -- 172.21.15.101:0/2430509396 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f6414011ba0 con 0x557c3186d2d0 2023-07-18T03:36:20.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.588+0000 7f643cb7db80 1 -- 172.21.15.101:0/2430509396 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f640c0a2340 con 0x557c3186d2d0 2023-07-18T03:36:20.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.588+0000 7f640a7fc700 1 -- 172.21.15.101:0/2430509396 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6414011d60 con 0x557c3186d2d0 2023-07-18T03:36:20.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.588+0000 7f643cb7db80 10 client.29339 fetch_fsmap learned FSMap version 49 2023-07-18T03:36:20.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.588+0000 7f643cb7db80 1 -- 172.21.15.101:0/2430509396 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f642410d220 con 0x557c3186d2d0 2023-07-18T03:36:20.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.589+0000 7f640a7fc700 1 -- 172.21.15.101:0/2430509396 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f640c0a2340 con 0x557c3186d2d0 2023-07-18T03:36:20.610 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.589+0000 7f640a7fc700 1 -- 172.21.15.101:0/2430509396 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 49) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6414025520 con 0x557c3186d2d0 2023-07-18T03:36:20.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.589+0000 7f643cb7db80 10 client.29339 fetch_fsmap finished waiting for FSMap version 49 2023-07-18T03:36:20.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.589+0000 7f643cb7db80 10 client.29339 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:36:20.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.589+0000 7f643cb7db80 20 client.29339 populate_metadata read hostname 'smithi101' 2023-07-18T03:36:20.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.589+0000 7f643cb7db80 1 --2- 172.21.15.101:0/2430509396 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557c318dd3b0 0x557c318df7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:20.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.589+0000 7f643cb7db80 4 client.29339 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:36:20.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.589+0000 7f643cb7db80 1 -- 172.21.15.101:0/2430509396 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557c3177b050 con 0x557c318dd3b0 2023-07-18T03:36:20.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.589+0000 7f641bfff700 1 --2- 172.21.15.101:0/2430509396 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557c318dd3b0 0x557c318df7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:20.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.590+0000 7f640a7fc700 10 client.29339 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:36:20.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.590+0000 7f641bfff700 1 --2- 172.21.15.101:0/2430509396 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557c318dd3b0 0x557c318df7c0 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b14c9f4175bf29d server_cookie=d45bf390703c0be in_seq=0 out_seq=0 2023-07-18T03:36:20.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.590+0000 7f640a7fc700 1 -- 172.21.15.101:0/2430509396 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x557c3177b050 con 0x557c318dd3b0 2023-07-18T03:36:20.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.590+0000 7f640a7fc700 10 client.29339 handle_command_reply: tid=0 2023-07-18T03:36:20.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.590+0000 7f643cb7db80 1 client.29339 shutdown 2023-07-18T03:36:20.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.590+0000 7f643cb7db80 1 -- 172.21.15.101:0/2430509396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c3186d2d0 msgr2=0x557c3187f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:20.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.590+0000 7f643cb7db80 1 --2- 172.21.15.101:0/2430509396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c3186d2d0 0x557c3187f370 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6414005d10 tx=0x7f641401d960 comp rx=0 tx=0).stop 2023-07-18T03:36:20.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.590+0000 7f643cb7db80 1 -- 172.21.15.101:0/2430509396 shutdown_connections 2023-07-18T03:36:20.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.590+0000 7f643cb7db80 1 --2- 172.21.15.101:0/2430509396 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x557c318dd3b0 0x557c318df7c0 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.590+0000 7f643cb7db80 1 --2- 172.21.15.101:0/2430509396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c3186d2d0 0x557c3187f370 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.614 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.590+0000 7f643cb7db80 1 --2- 172.21.15.101:0/2430509396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557c31878ad0 0x557c3187db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.590+0000 7f643cb7db80 1 --2- 172.21.15.101:0/2430509396 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c3186dbe0 0x557c3187d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.590+0000 7f640a7fc700 0 client.29339 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:36:20.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.590+0000 7f643cb7db80 1 -- 172.21.15.101:0/2430509396 >> 172.21.15.101:0/2430509396 conn(0x557c31863da0 msgr2=0x557c31870af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:36:20.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.590+0000 7f643cb7db80 1 -- 172.21.15.101:0/2430509396 shutdown_connections 2023-07-18T03:36:20.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.590+0000 7f643cb7db80 1 -- 172.21.15.101:0/2430509396 wait complete. 2023-07-18T03:36:20.616 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.590+0000 7f643cb7db80 20 client.29339 trim_cache size 0 max 16384 2023-07-18T03:36:20.616 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:36:20.616 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:36:20.617 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:36:20.617 INFO:teuthology.orchestra.run.smithi101.stdout: "0f2f19e0-0429-465c-935e-223eb09490c3": { 2023-07-18T03:36:20.617 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:36:20.617 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "0f2f19e0-0429-465c-935e-223eb09490c3", 2023-07-18T03:36:20.617 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:36:20.618 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:36:20.618 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:36:20.618 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:36:20.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.591+0000 7f64291ff700 1 -- 172.21.15.101:0/1303634663 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f64100a12c0 msgr2=0x7f64100a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:20.619 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.591+0000 7f64291ff700 1 --2- 172.21.15.101:0/1303634663 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f64100a12c0 0x7f64100a3770 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f641c0060b0 tx=0x7f641c009700 comp rx=0 tx=0).stop 2023-07-18T03:36:20.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.591+0000 7f64291ff700 1 -- 172.21.15.101:0/1303634663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6424100490 msgr2=0x7f6424198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:20.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.591+0000 7f64291ff700 1 --2- 172.21.15.101:0/1303634663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6424100490 0x7f6424198460 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f640c00e9f0 tx=0x7f640c00ed00 comp rx=0 tx=0).stop 2023-07-18T03:36:20.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.591+0000 7f64291ff700 1 -- 172.21.15.101:0/1303634663 shutdown_connections 2023-07-18T03:36:20.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.591+0000 7f64291ff700 1 --2- 172.21.15.101:0/1303634663 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6424105af0 0x7f642419c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.591+0000 7f64291ff700 1 --2- 172.21.15.101:0/1303634663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64240ffae0 0x7f6424197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.591+0000 7f64291ff700 1 --2- 172.21.15.101:0/1303634663 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f64100a12c0 0x7f64100a3770 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.591+0000 7f64291ff700 1 --2- 172.21.15.101:0/1303634663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6424100490 0x7f6424198460 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.591+0000 7f64291ff700 1 -- 172.21.15.101:0/1303634663 >> 172.21.15.101:0/1303634663 conn(0x7f64240fb5e0 msgr2=0x7f642410b000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:36:20.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.591+0000 7f64291ff700 1 -- 172.21.15.101:0/1303634663 shutdown_connections 2023-07-18T03:36:20.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:20.591+0000 7f64291ff700 1 -- 172.21.15.101:0/1303634663 wait complete. 2023-07-18T03:36:20.622 INFO:tasks.cephfs.filesystem:scrub status for tag:0f2f19e0-0429-465c-935e-223eb09490c3 - {'path': '/', 'tag': '0f2f19e0-0429-465c-935e-223eb09490c3', 'options': 'recursive,force'} 2023-07-18T03:36:21.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:36:21 smithi179 ceph-mon[129229]: pgmap v912: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 210 B/s wr, 6 op/s 2023-07-18T03:36:21.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:36:21 smithi101 ceph-mon[126379]: pgmap v912: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 210 B/s wr, 6 op/s 2023-07-18T03:36:21.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:36:21 smithi155 ceph-mon[128885]: pgmap v912: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 210 B/s wr, 6 op/s 2023-07-18T03:36:23.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:36:23 smithi179 ceph-mon[129229]: pgmap v913: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:36:23.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:36:23 smithi101 ceph-mon[126379]: pgmap v913: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:36:23.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:36:23 smithi155 ceph-mon[128885]: pgmap v913: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:36:25.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:36:25 smithi179 ceph-mon[129229]: pgmap v914: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:36:25.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:36:25 smithi101 ceph-mon[126379]: pgmap v914: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:36:25.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:36:25 smithi155 ceph-mon[128885]: pgmap v914: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:36:27.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:36:27 smithi179 ceph-mon[129229]: pgmap v915: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:36:27.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:36:27 smithi101 ceph-mon[126379]: pgmap v915: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:36:27.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:36:27 smithi155 ceph-mon[128885]: pgmap v915: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:36:29.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:36:29 smithi179 ceph-mon[129229]: pgmap v916: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:36:29.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:36:29 smithi101 ceph-mon[126379]: pgmap v916: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:36:29.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:36:29 smithi155 ceph-mon[128885]: pgmap v916: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:36:31.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:36:31 smithi179 ceph-mon[129229]: pgmap v917: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 95 B/s wr, 8 op/s 2023-07-18T03:36:31.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:36:31 smithi101 ceph-mon[126379]: pgmap v917: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 95 B/s wr, 8 op/s 2023-07-18T03:36:31.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:36:31 smithi155 ceph-mon[128885]: pgmap v917: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 95 B/s wr, 8 op/s 2023-07-18T03:36:33.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:36:33 smithi179 ceph-mon[129229]: pgmap v918: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:36:33.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:36:33 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:36:33.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:36:33 smithi101 ceph-mon[126379]: pgmap v918: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:36:33.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:36:33 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:36:33.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:36:33 smithi155 ceph-mon[128885]: pgmap v918: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:36:33.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:36:33 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:36:34.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:36:34 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:36:34.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:36:34 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:36:34.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:36:34 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:36:34.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:36:34 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:36:34.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:36:34 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:36:34.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:36:34 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:36:34.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:36:34 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:36:34.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:36:34 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:36:34.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:36:34 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:36:35.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:36:35 smithi101 ceph-mon[126379]: pgmap v919: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:36:35.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:36:35 smithi155 ceph-mon[128885]: pgmap v919: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:36:35.953 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:36:35 smithi179 ceph-mon[129229]: pgmap v919: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:36:37.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:36:37 smithi101 ceph-mon[126379]: pgmap v920: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:36:37.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:36:37 smithi155 ceph-mon[128885]: pgmap v920: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:36:37.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:36:37 smithi179 ceph-mon[129229]: pgmap v920: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:36:39.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:36:39 smithi101 ceph-mon[126379]: pgmap v921: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:36:39.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:36:39 smithi155 ceph-mon[128885]: pgmap v921: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:36:39.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:36:39 smithi179 ceph-mon[129229]: pgmap v921: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:36:41.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:36:41 smithi101 ceph-mon[126379]: pgmap v922: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:36:41.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:36:41 smithi155 ceph-mon[128885]: pgmap v922: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:36:41.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:36:41 smithi179 ceph-mon[129229]: pgmap v922: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:36:43.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:36:43 smithi101 ceph-mon[126379]: pgmap v923: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:36:43.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:36:43 smithi155 ceph-mon[128885]: pgmap v923: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:36:43.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:36:43 smithi179 ceph-mon[129229]: pgmap v923: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:36:45.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:36:45 smithi101 ceph-mon[126379]: pgmap v924: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:36:45.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:36:45 smithi155 ceph-mon[128885]: pgmap v924: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:36:45.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:36:45 smithi179 ceph-mon[129229]: pgmap v924: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:36:47.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:36:47 smithi101 ceph-mon[126379]: pgmap v925: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:36:47.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:36:47 smithi155 ceph-mon[128885]: pgmap v925: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:36:47.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:36:47 smithi179 ceph-mon[129229]: pgmap v925: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:36:48.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:36:48 smithi155 ceph-mon[128885]: pgmap v926: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:36:48.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:36:48 smithi179 ceph-mon[129229]: pgmap v926: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:36:48.990 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:36:48 smithi101 ceph-mon[126379]: pgmap v926: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:36:50.623 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:36:50.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.840+0000 7f6db05ff700 1 -- 172.21.15.101:0/3365370177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6da8100490 msgr2=0x7f6da810c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:50.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.840+0000 7f6db05ff700 1 --2- 172.21.15.101:0/3365370177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6da8100490 0x7f6da810c920 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6da0009f90 tx=0x7f6da000b330 comp rx=0 tx=0).stop 2023-07-18T03:36:50.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.840+0000 7f6db05ff700 1 -- 172.21.15.101:0/3365370177 shutdown_connections 2023-07-18T03:36:50.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.840+0000 7f6db05ff700 1 --2- 172.21.15.101:0/3365370177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6da8100490 0x7f6da810c920 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:50.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.840+0000 7f6db05ff700 1 --2- 172.21.15.101:0/3365370177 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6da8105af0 0x7f6da8105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:50.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.840+0000 7f6db05ff700 1 --2- 172.21.15.101:0/3365370177 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6da80ffae0 0x7f6da80fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:50.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.840+0000 7f6db05ff700 1 -- 172.21.15.101:0/3365370177 >> 172.21.15.101:0/3365370177 conn(0x7f6da80fb5e0 msgr2=0x7f6da80fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:36:50.839 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.840+0000 7f6db05ff700 1 -- 172.21.15.101:0/3365370177 shutdown_connections 2023-07-18T03:36:50.839 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.840+0000 7f6db05ff700 1 -- 172.21.15.101:0/3365370177 wait complete. 2023-07-18T03:36:50.841 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.841+0000 7f6db05ff700 1 Processor -- start 2023-07-18T03:36:50.841 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.842+0000 7f6db05ff700 1 -- start start 2023-07-18T03:36:50.841 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.842+0000 7f6db05ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6da80ffae0 0x7f6da8195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:50.841 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.842+0000 7f6db05ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6da8100490 0x7f6da81962f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:50.842 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.842+0000 7f6db05ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6da8105af0 0x7f6da819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:50.842 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.842+0000 7f6db05ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da819ad90 con 0x7f6da8100490 2023-07-18T03:36:50.842 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.842+0000 7f6db05ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da819af00 con 0x7f6da8105af0 2023-07-18T03:36:50.842 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.842+0000 7f6db05ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da819b0c0 con 0x7f6da80ffae0 2023-07-18T03:36:50.843 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.842+0000 7f6daeb9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6da8105af0 0x7f6da819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:50.843 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.842+0000 7f6dae39b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6da80ffae0 0x7f6da8195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:50.843 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.842+0000 7f6dadb9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6da8100490 0x7f6da81962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:50.843 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.842+0000 7f6daeb9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6da8105af0 0x7f6da819a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:39318/0 (socket says 172.21.15.101:39318) 2023-07-18T03:36:50.843 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.843+0000 7f6daeb9c700 1 -- 172.21.15.101:0/2832888472 learned_addr learned my addr 172.21.15.101:0/2832888472 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:36:50.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.842+0000 7f6dae39b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6da80ffae0 0x7f6da8195db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:58016/0 (socket says 172.21.15.101:58016) 2023-07-18T03:36:50.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.843+0000 7f6daeb9c700 1 -- 172.21.15.101:0/2832888472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6da80ffae0 msgr2=0x7f6da8195db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:50.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.843+0000 7f6daeb9c700 1 --2- 172.21.15.101:0/2832888472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6da80ffae0 0x7f6da8195db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:50.844 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.843+0000 7f6daeb9c700 1 -- 172.21.15.101:0/2832888472 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6da8100490 msgr2=0x7f6da81962f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:50.845 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.843+0000 7f6daeb9c700 1 --2- 172.21.15.101:0/2832888472 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6da8100490 0x7f6da81962f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:50.845 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.843+0000 7f6daeb9c700 1 -- 172.21.15.101:0/2832888472 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da40095f0 con 0x7f6da8105af0 2023-07-18T03:36:50.845 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.843+0000 7f6daeb9c700 1 --2- 172.21.15.101:0/2832888472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6da8105af0 0x7f6da819a750 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6da0009010 tx=0x7f6da0004a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:36:50.845 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.843+0000 7f6d9f7fe700 1 -- 172.21.15.101:0/2832888472 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da000e040 con 0x7f6da8105af0 2023-07-18T03:36:50.845 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.843+0000 7f6d9f7fe700 1 -- 172.21.15.101:0/2832888472 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6da001d470 con 0x7f6da8105af0 2023-07-18T03:36:50.846 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.843+0000 7f6d9f7fe700 1 -- 172.21.15.101:0/2832888472 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da0005280 con 0x7f6da8105af0 2023-07-18T03:36:50.846 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.843+0000 7f6db05ff700 1 -- 172.21.15.101:0/2832888472 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6da0009c40 con 0x7f6da8105af0 2023-07-18T03:36:50.846 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.844+0000 7f6db05ff700 1 -- 172.21.15.101:0/2832888472 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6da81a2a00 con 0x7f6da8105af0 2023-07-18T03:36:50.851 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.847+0000 7f6d9f7fe700 1 -- 172.21.15.101:0/2832888472 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f6da001d5e0 con 0x7f6da8105af0 2023-07-18T03:36:50.851 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.848+0000 7f6dc3ee5b80 1 Processor -- start 2023-07-18T03:36:50.852 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.848+0000 7f6dc3ee5b80 1 -- start start 2023-07-18T03:36:50.852 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.849+0000 7f6dc3ee5b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7472fb2d0 0x55f7472fb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:50.852 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.849+0000 7f6dc3ee5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7472fbbe0 0x55f747306590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:50.852 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.849+0000 7f6dc3ee5b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f747306ad0 0x55f747308eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:50.853 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.849+0000 7f6dc3ee5b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f747201cb0 con 0x55f7472fb2d0 2023-07-18T03:36:50.853 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.849+0000 7f6dc3ee5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f747254a40 con 0x55f7472fbbe0 2023-07-18T03:36:50.853 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.849+0000 7f6dc3ee5b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f74724ded0 con 0x55f747306ad0 2023-07-18T03:36:50.853 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.849+0000 7f6d9f7fe700 1 --2- 172.21.15.101:0/2832888472 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6d940a1340 0x7f6d940a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:50.853 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.849+0000 7f6dae39b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7472fbbe0 0x55f747306590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:50.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.849+0000 7f6dadb9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7472fb2d0 0x55f7472fb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:50.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.849+0000 7f6dae39b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7472fbbe0 0x55f747306590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:39332/0 (socket says 172.21.15.101:39332) 2023-07-18T03:36:50.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.849+0000 7f6dae39b700 1 -- 172.21.15.101:0/3503585242 learned_addr learned my addr 172.21.15.101:0/3503585242 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:36:50.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.849+0000 7f6d9f7fe700 1 -- 172.21.15.101:0/2832888472 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f6da00ce540 con 0x7f6da8105af0 2023-07-18T03:36:50.854 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.849+0000 7f6dadb9a700 1 --2- 172.21.15.101:0/3503585242 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f747306ad0 0x55f747308eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:50.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.849+0000 7f6dae39b700 1 -- 172.21.15.101:0/3503585242 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f747306ad0 msgr2=0x55f747308eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:50.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.849+0000 7f6dae39b700 1 --2- 172.21.15.101:0/3503585242 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f747306ad0 0x55f747308eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:50.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.849+0000 7f6daeb9c700 1 --2- 172.21.15.101:0/2832888472 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6d940a1340 0x7f6d940a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:50.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.849+0000 7f6dae39b700 1 -- 172.21.15.101:0/3503585242 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7472fb2d0 msgr2=0x55f7472fb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:50.855 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.849+0000 7f6dae39b700 1 --2- 172.21.15.101:0/3503585242 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7472fb2d0 0x55f7472fb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:50.856 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.849+0000 7f6dae39b700 1 -- 172.21.15.101:0/3503585242 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f74707bde0 con 0x55f7472fbbe0 2023-07-18T03:36:50.856 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.849+0000 7f6dadb9a700 1 --2- 172.21.15.101:0/3503585242 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7472fb2d0 0x55f7472fb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:36:50.856 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.850+0000 7f6dae39b700 1 --2- 172.21.15.101:0/3503585242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7472fbbe0 0x55f747306590 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f6da4009f40 tx=0x7f6da4011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6d08544ec7b8d3f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:36:50.856 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.850+0000 7f6d9cbfe700 1 -- 172.21.15.101:0/3503585242 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da4004310 con 0x55f7472fbbe0 2023-07-18T03:36:50.857 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.850+0000 7f6d9cbfe700 1 -- 172.21.15.101:0/3503585242 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6da4004470 con 0x55f7472fbbe0 2023-07-18T03:36:50.857 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.850+0000 7f6d9cbfe700 1 -- 172.21.15.101:0/3503585242 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da4019910 con 0x55f7472fbbe0 2023-07-18T03:36:50.857 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.850+0000 7f6daeb9c700 1 --2- 172.21.15.101:0/2832888472 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6d940a1340 0x7f6d940a37f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6da00cb990 tx=0x7f6da0009920 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:36:50.857 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.850+0000 7f6dc3ee5b80 1 -- 172.21.15.101:0/3503585242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7472fbbe0 msgr2=0x55f747306590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:50.858 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.850+0000 7f6dc3ee5b80 1 --2- 172.21.15.101:0/3503585242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7472fbbe0 0x55f747306590 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f6da4009f40 tx=0x7f6da4011720 comp rx=0 tx=0).stop 2023-07-18T03:36:50.858 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.851+0000 7f6dc3ee5b80 1 -- 172.21.15.101:0/3503585242 shutdown_connections 2023-07-18T03:36:50.858 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.851+0000 7f6dc3ee5b80 1 --2- 172.21.15.101:0/3503585242 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f747306ad0 0x55f747308eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:50.859 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.851+0000 7f6dc3ee5b80 1 --2- 172.21.15.101:0/3503585242 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7472fb2d0 0x55f7472fb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:50.859 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.851+0000 7f6dc3ee5b80 1 --2- 172.21.15.101:0/3503585242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7472fbbe0 0x55f747306590 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:50.859 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.851+0000 7f6dc3ee5b80 1 -- 172.21.15.101:0/3503585242 >> 172.21.15.101:0/3503585242 conn(0x55f7472f1da0 msgr2=0x55f7472f76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:36:50.859 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.851+0000 7f6dc3ee5b80 1 -- 172.21.15.101:0/3503585242 shutdown_connections 2023-07-18T03:36:50.860 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.851+0000 7f6dc3ee5b80 1 -- 172.21.15.101:0/3503585242 wait complete. 2023-07-18T03:36:50.860 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.851+0000 7f6dc3ee5b80 1 -- start start 2023-07-18T03:36:50.861 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.851+0000 7f6dc3ee5b80 1 Processor -- start 2023-07-18T03:36:50.861 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.852+0000 7f6dc3ee5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7472fb2d0 0x55f7473064c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:50.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.852+0000 7f6dc3ee5b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7472fbbe0 0x55f747303740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:50.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.852+0000 7f6dc3ee5b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f747306ad0 0x55f747303dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:50.862 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.852+0000 7f6dc3ee5b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7471edac0 con 0x55f7472fbbe0 2023-07-18T03:36:50.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.852+0000 7f6dc3ee5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7471ed7c0 con 0x55f7472fb2d0 2023-07-18T03:36:50.863 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.852+0000 7f6dc3ee5b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7470d1920 con 0x55f747306ad0 2023-07-18T03:36:50.864 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.852+0000 7f6dc3ee5b80 10 client.0 _ll_register_callbacks cb 0x55f7471eb760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:36:50.864 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.852+0000 7f6dae39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7472fbbe0 0x55f747303740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:50.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.852+0000 7f6dadb9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7472fb2d0 0x55f7473064c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:50.865 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.852+0000 7f6dae39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7472fbbe0 0x55f747303740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:47770/0 (socket says 172.21.15.101:47770) 2023-07-18T03:36:50.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.852+0000 7f6dae39b700 1 -- 172.21.15.101:0/183699036 learned_addr learned my addr 172.21.15.101:0/183699036 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:36:50.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.852+0000 7f6dadb9a700 1 --2- 172.21.15.101:0/183699036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f747306ad0 0x55f747303dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:50.866 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.852+0000 7f6dae39b700 1 -- 172.21.15.101:0/183699036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f747306ad0 msgr2=0x55f747303dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:50.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.852+0000 7f6dae39b700 1 --2- 172.21.15.101:0/183699036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f747306ad0 0x55f747303dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:50.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.852+0000 7f6dae39b700 1 -- 172.21.15.101:0/183699036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7472fb2d0 msgr2=0x55f7473064c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:50.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.852+0000 7f6dae39b700 1 --2- 172.21.15.101:0/183699036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7472fb2d0 0x55f7473064c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:50.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.852+0000 7f6dae39b700 1 -- 172.21.15.101:0/183699036 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6da4009ba0 con 0x55f7472fbbe0 2023-07-18T03:36:50.867 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.852+0000 7f6dadb9a700 1 --2- 172.21.15.101:0/183699036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7472fb2d0 0x55f7473064c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:36:50.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.853+0000 7f6dae39b700 1 --2- 172.21.15.101:0/183699036 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7472fbbe0 0x55f747303740 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f6da4009a30 tx=0x7f6da4004670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a4d714f1b693f5 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:36:50.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.853+0000 7f6d91ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:36:50.868 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.853+0000 7f6d91ffb700 1 -- 172.21.15.101:0/183699036 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da4003740 con 0x55f7472fbbe0 2023-07-18T03:36:50.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.853+0000 7f6dc3ee5b80 1 -- 172.21.15.101:0/183699036 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f7471a74f0 con 0x55f7472fbbe0 2023-07-18T03:36:50.869 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.853+0000 7f6d91ffb700 1 -- 172.21.15.101:0/183699036 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f6da40038a0 con 0x55f7472fbbe0 2023-07-18T03:36:50.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.853+0000 7f6d91ffb700 1 -- 172.21.15.101:0/183699036 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6da4003a60 con 0x55f7472fbbe0 2023-07-18T03:36:50.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.853+0000 7f6dc3ee5b80 10 client.19242 fetch_fsmap learned FSMap version 49 2023-07-18T03:36:50.870 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.853+0000 7f6dc3ee5b80 1 -- 172.21.15.101:0/183699036 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55f747064c60 con 0x55f7472fbbe0 2023-07-18T03:36:50.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.853+0000 7f6d91ffb700 1 -- 172.21.15.101:0/183699036 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f7471a74f0 con 0x55f7472fbbe0 2023-07-18T03:36:50.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.854+0000 7f6d91ffb700 1 -- 172.21.15.101:0/183699036 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 49) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6da401a440 con 0x55f7472fbbe0 2023-07-18T03:36:50.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.854+0000 7f6dc3ee5b80 10 client.19242 fetch_fsmap finished waiting for FSMap version 49 2023-07-18T03:36:50.871 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.854+0000 7f6dc3ee5b80 10 client.19242 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:36:50.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.854+0000 7f6dc3ee5b80 20 client.19242 populate_metadata read hostname 'smithi101' 2023-07-18T03:36:50.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.854+0000 7f6dc3ee5b80 1 --2- 172.21.15.101:0/183699036 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f74731a750 0x55f74731cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:50.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.854+0000 7f6dc3ee5b80 4 client.19242 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:36:50.872 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.854+0000 7f6dc3ee5b80 1 -- 172.21.15.101:0/183699036 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f747246eb0 con 0x55f74731a750 2023-07-18T03:36:50.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.854+0000 7f6dadb9a700 1 --2- 172.21.15.101:0/183699036 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f74731a750 0x55f74731cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:50.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.854+0000 7f6d91ffb700 10 client.19242 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:36:50.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.854+0000 7f6dadb9a700 1 --2- 172.21.15.101:0/183699036 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f74731a750 0x55f74731cb60 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d41a7f3a5ec72077 server_cookie=c030eafd9666fb68 in_seq=0 out_seq=0 2023-07-18T03:36:50.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.855+0000 7f6d91ffb700 1 -- 172.21.15.101:0/183699036 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f747246eb0 con 0x55f74731a750 2023-07-18T03:36:50.873 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.855+0000 7f6d91ffb700 10 client.19242 handle_command_reply: tid=0 2023-07-18T03:36:50.874 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.855+0000 7f6dc3ee5b80 1 client.19242 shutdown 2023-07-18T03:36:50.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.855+0000 7f6dc3ee5b80 1 -- 172.21.15.101:0/183699036 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7472fbbe0 msgr2=0x55f747303740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:50.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.855+0000 7f6dc3ee5b80 1 --2- 172.21.15.101:0/183699036 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7472fbbe0 0x55f747303740 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f6da4009a30 tx=0x7f6da4004670 comp rx=0 tx=0).stop 2023-07-18T03:36:50.879 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.855+0000 7f6dc3ee5b80 1 -- 172.21.15.101:0/183699036 shutdown_connections 2023-07-18T03:36:50.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.855+0000 7f6dc3ee5b80 1 --2- 172.21.15.101:0/183699036 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f74731a750 0x55f74731cb60 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:50.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.855+0000 7f6dc3ee5b80 1 --2- 172.21.15.101:0/183699036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7472fb2d0 0x55f7473064c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:50.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.855+0000 7f6dc3ee5b80 1 --2- 172.21.15.101:0/183699036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f747306ad0 0x55f747303dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:50.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.855+0000 7f6dc3ee5b80 1 --2- 172.21.15.101:0/183699036 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7472fbbe0 0x55f747303740 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:50.880 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.855+0000 7f6d91ffb700 0 client.19242 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:36:50.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.855+0000 7f6dc3ee5b80 1 -- 172.21.15.101:0/183699036 >> 172.21.15.101:0/183699036 conn(0x55f7472f1da0 msgr2=0x55f7472f7290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:36:50.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.856+0000 7f6dc3ee5b80 1 -- 172.21.15.101:0/183699036 shutdown_connections 2023-07-18T03:36:50.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.856+0000 7f6dc3ee5b80 1 -- 172.21.15.101:0/183699036 wait complete. 2023-07-18T03:36:50.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.856+0000 7f6dc3ee5b80 20 client.19242 trim_cache size 0 max 16384 2023-07-18T03:36:50.882 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.873+0000 7f6dc3ee5b80 1 Processor -- start 2023-07-18T03:36:50.882 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.873+0000 7f6dc3ee5b80 1 -- start start 2023-07-18T03:36:50.882 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.873+0000 7f6dc3ee5b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7472fb2d0 0x55f7472fe1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:50.882 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.873+0000 7f6dc3ee5b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7472fbbe0 0x55f747302f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:50.882 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.873+0000 7f6dc3ee5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f747306ad0 0x55f7472fea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:50.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.873+0000 7f6dc3ee5b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7472fc540 con 0x55f7472fbbe0 2023-07-18T03:36:50.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.873+0000 7f6dc3ee5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f747304d20 con 0x55f747306ad0 2023-07-18T03:36:50.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.873+0000 7f6dc3ee5b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f747301ca0 con 0x55f7472fb2d0 2023-07-18T03:36:50.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.874+0000 7f6dadb9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7472fb2d0 0x55f7472fe1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:50.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.874+0000 7f6dadb9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7472fb2d0 0x55f7472fe1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:58042/0 (socket says 172.21.15.101:58042) 2023-07-18T03:36:50.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.874+0000 7f6dadb9a700 1 -- 172.21.15.101:0/560842066 learned_addr learned my addr 172.21.15.101:0/560842066 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:36:50.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.874+0000 7f6dadb9a700 1 --2- 172.21.15.101:0/560842066 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f747306ad0 0x55f7472fea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:50.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.874+0000 7f6dae39b700 1 --2- 172.21.15.101:0/560842066 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7472fbbe0 0x55f747302f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:50.884 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.874+0000 7f6dadb9a700 1 -- 172.21.15.101:0/560842066 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f747306ad0 msgr2=0x55f7472fea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:50.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.874+0000 7f6dadb9a700 1 --2- 172.21.15.101:0/560842066 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f747306ad0 0x55f7472fea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:50.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.874+0000 7f6dadb9a700 1 -- 172.21.15.101:0/560842066 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7472fbbe0 msgr2=0x55f747302f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:50.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.874+0000 7f6dadb9a700 1 --2- 172.21.15.101:0/560842066 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7472fbbe0 0x55f747302f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:50.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.874+0000 7f6dadb9a700 1 -- 172.21.15.101:0/560842066 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7470a2250 con 0x55f7472fb2d0 2023-07-18T03:36:50.885 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.874+0000 7f6dadb9a700 1 --2- 172.21.15.101:0/560842066 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7472fb2d0 0x55f7472fe1b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6d9800a010 tx=0x7f6d980058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ea73c39d02a6e0b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:36:50.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.874+0000 7f6d9d3ff700 1 -- 172.21.15.101:0/560842066 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d98004320 con 0x55f7472fb2d0 2023-07-18T03:36:50.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.875+0000 7f6d9d3ff700 1 -- 172.21.15.101:0/560842066 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d98019030 con 0x55f7472fb2d0 2023-07-18T03:36:50.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.875+0000 7f6d9d3ff700 1 -- 172.21.15.101:0/560842066 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d98005550 con 0x55f7472fb2d0 2023-07-18T03:36:50.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.875+0000 7f6dc3ee5b80 1 -- 172.21.15.101:0/560842066 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7472fb2d0 msgr2=0x55f7472fe1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:50.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.875+0000 7f6dc3ee5b80 1 --2- 172.21.15.101:0/560842066 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7472fb2d0 0x55f7472fe1b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6d9800a010 tx=0x7f6d980058e0 comp rx=0 tx=0).stop 2023-07-18T03:36:50.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.875+0000 7f6dc3ee5b80 1 -- 172.21.15.101:0/560842066 shutdown_connections 2023-07-18T03:36:50.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.875+0000 7f6dc3ee5b80 1 --2- 172.21.15.101:0/560842066 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f747306ad0 0x55f7472fea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:50.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.875+0000 7f6dc3ee5b80 1 --2- 172.21.15.101:0/560842066 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7472fb2d0 0x55f7472fe1b0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:50.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.875+0000 7f6dc3ee5b80 1 --2- 172.21.15.101:0/560842066 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7472fbbe0 0x55f747302f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:50.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.875+0000 7f6dc3ee5b80 1 -- 172.21.15.101:0/560842066 >> 172.21.15.101:0/560842066 conn(0x55f7472f1da0 msgr2=0x55f747316af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:36:50.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.875+0000 7f6dc3ee5b80 1 -- 172.21.15.101:0/560842066 shutdown_connections 2023-07-18T03:36:50.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.875+0000 7f6dc3ee5b80 1 -- 172.21.15.101:0/560842066 wait complete. 2023-07-18T03:36:50.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.876+0000 7f6dc3ee5b80 1 -- start start 2023-07-18T03:36:50.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.876+0000 7f6dc3ee5b80 1 Processor -- start 2023-07-18T03:36:50.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.877+0000 7f6dc3ee5b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7472fb2d0 0x55f747311a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:50.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.877+0000 7f6dc3ee5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7472fbbe0 0x55f74730df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:50.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.877+0000 7f6dc3ee5b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f747306ad0 0x55f74730e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:50.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.877+0000 7f6dc3ee5b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f74730ec80 con 0x55f747306ad0 2023-07-18T03:36:50.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.877+0000 7f6dc3ee5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f74730edf0 con 0x55f7472fbbe0 2023-07-18T03:36:50.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.877+0000 7f6dc3ee5b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f74736e620 con 0x55f7472fb2d0 2023-07-18T03:36:50.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.877+0000 7f6dc3ee5b80 10 client.0 _ll_register_callbacks cb 0x7f6da4009dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:36:50.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.877+0000 7f6dae39b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7472fbbe0 0x55f74730df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:50.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.877+0000 7f6dadb9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7472fb2d0 0x55f747311a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:50.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.877+0000 7f6dae39b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7472fbbe0 0x55f74730df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:39356/0 (socket says 172.21.15.101:39356) 2023-07-18T03:36:50.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.877+0000 7f6dae39b700 1 -- 172.21.15.101:0/1436236093 learned_addr learned my addr 172.21.15.101:0/1436236093 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:36:50.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.877+0000 7f6dadb9a700 1 --2- 172.21.15.101:0/1436236093 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f747306ad0 0x55f74730e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:50.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.877+0000 7f6dae39b700 1 -- 172.21.15.101:0/1436236093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7472fb2d0 msgr2=0x55f747311a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:50.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.877+0000 7f6dae39b700 1 --2- 172.21.15.101:0/1436236093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7472fb2d0 0x55f747311a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:50.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.877+0000 7f6dae39b700 1 -- 172.21.15.101:0/1436236093 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f747306ad0 msgr2=0x55f74730e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:50.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.877+0000 7f6dae39b700 1 --2- 172.21.15.101:0/1436236093 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f747306ad0 0x55f74730e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:50.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.877+0000 7f6dae39b700 1 -- 172.21.15.101:0/1436236093 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d9801b040 con 0x55f7472fbbe0 2023-07-18T03:36:50.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.877+0000 7f6dadb9a700 1 --2- 172.21.15.101:0/1436236093 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f747306ad0 0x55f74730e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:36:50.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.878+0000 7f6dae39b700 1 --2- 172.21.15.101:0/1436236093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7472fbbe0 0x55f74730df50 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f6da4004440 tx=0x7f6da4009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=167b3bcf8f3ad14 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:36:50.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.878+0000 7f6d91ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:36:50.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.878+0000 7f6d91ffb700 1 -- 172.21.15.101:0/1436236093 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da4003950 con 0x55f7472fbbe0 2023-07-18T03:36:50.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.878+0000 7f6dc3ee5b80 1 -- 172.21.15.101:0/1436236093 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6da4004470 con 0x55f7472fbbe0 2023-07-18T03:36:50.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.878+0000 7f6d91ffb700 1 -- 172.21.15.101:0/1436236093 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f6da4000e40 con 0x55f7472fbbe0 2023-07-18T03:36:50.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.878+0000 7f6d91ffb700 1 -- 172.21.15.101:0/1436236093 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6da4003ab0 con 0x55f7472fbbe0 2023-07-18T03:36:50.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.878+0000 7f6dc3ee5b80 10 client.29357 fetch_fsmap learned FSMap version 49 2023-07-18T03:36:50.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.878+0000 7f6dc3ee5b80 1 -- 172.21.15.101:0/1436236093 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55f74707a2b0 con 0x55f7472fbbe0 2023-07-18T03:36:50.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.878+0000 7f6d91ffb700 1 -- 172.21.15.101:0/1436236093 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6da4004470 con 0x55f7472fbbe0 2023-07-18T03:36:50.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.878+0000 7f6d91ffb700 1 -- 172.21.15.101:0/1436236093 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 49) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6da4005040 con 0x55f7472fbbe0 2023-07-18T03:36:50.894 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.878+0000 7f6dc3ee5b80 10 client.29357 fetch_fsmap finished waiting for FSMap version 49 2023-07-18T03:36:50.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.878+0000 7f6dc3ee5b80 10 client.29357 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:36:50.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.878+0000 7f6dc3ee5b80 20 client.29357 populate_metadata read hostname 'smithi101' 2023-07-18T03:36:50.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.878+0000 7f6dc3ee5b80 1 --2- 172.21.15.101:0/1436236093 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f74736fa20 0x55f747371e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:50.895 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.878+0000 7f6dc3ee5b80 4 client.29357 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:36:50.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.878+0000 7f6dc3ee5b80 1 -- 172.21.15.101:0/1436236093 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f747209050 con 0x55f74736fa20 2023-07-18T03:36:50.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.879+0000 7f6dadb9a700 1 --2- 172.21.15.101:0/1436236093 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f74736fa20 0x55f747371e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:50.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.879+0000 7f6dadb9a700 1 --2- 172.21.15.101:0/1436236093 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f74736fa20 0x55f747371e30 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a692f24ed87aee3e server_cookie=9cb543dd0e2262d1 in_seq=0 out_seq=0 2023-07-18T03:36:50.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.879+0000 7f6d91ffb700 10 client.29357 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:36:50.896 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.879+0000 7f6d91ffb700 1 -- 172.21.15.101:0/1436236093 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55f747209050 con 0x55f74736fa20 2023-07-18T03:36:50.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.879+0000 7f6d91ffb700 10 client.29357 handle_command_reply: tid=0 2023-07-18T03:36:50.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.879+0000 7f6dc3ee5b80 1 client.29357 shutdown 2023-07-18T03:36:50.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.879+0000 7f6dc3ee5b80 1 -- 172.21.15.101:0/1436236093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7472fbbe0 msgr2=0x55f74730df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:50.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.879+0000 7f6dc3ee5b80 1 --2- 172.21.15.101:0/1436236093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7472fbbe0 0x55f74730df50 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f6da4004440 tx=0x7f6da4009310 comp rx=0 tx=0).stop 2023-07-18T03:36:50.897 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.880+0000 7f6dc3ee5b80 1 -- 172.21.15.101:0/1436236093 shutdown_connections 2023-07-18T03:36:50.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.880+0000 7f6dc3ee5b80 1 --2- 172.21.15.101:0/1436236093 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f74736fa20 0x55f747371e30 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:50.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.880+0000 7f6dc3ee5b80 1 --2- 172.21.15.101:0/1436236093 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f747306ad0 0x55f74730e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:50.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.880+0000 7f6dc3ee5b80 1 --2- 172.21.15.101:0/1436236093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7472fb2d0 0x55f747311a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:50.898 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.880+0000 7f6dc3ee5b80 1 --2- 172.21.15.101:0/1436236093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7472fbbe0 0x55f74730df50 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:50.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.880+0000 7f6d91ffb700 0 client.29357 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:36:50.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.880+0000 7f6dc3ee5b80 1 -- 172.21.15.101:0/1436236093 >> 172.21.15.101:0/1436236093 conn(0x55f7472f1da0 msgr2=0x55f7472fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:36:50.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.880+0000 7f6dc3ee5b80 1 -- 172.21.15.101:0/1436236093 shutdown_connections 2023-07-18T03:36:50.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.880+0000 7f6dc3ee5b80 1 -- 172.21.15.101:0/1436236093 wait complete. 2023-07-18T03:36:50.899 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.880+0000 7f6dc3ee5b80 20 client.29357 trim_cache size 0 max 16384 2023-07-18T03:36:50.900 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:36:50.900 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:36:50.900 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:36:50.901 INFO:teuthology.orchestra.run.smithi101.stdout: "0f2f19e0-0429-465c-935e-223eb09490c3": { 2023-07-18T03:36:50.901 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:36:50.901 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "0f2f19e0-0429-465c-935e-223eb09490c3", 2023-07-18T03:36:50.901 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:36:50.902 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:36:50.902 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:36:50.902 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:36:50.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.880+0000 7f6db05ff700 1 -- 172.21.15.101:0/2832888472 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6d940a1340 msgr2=0x7f6d940a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:50.903 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.880+0000 7f6db05ff700 1 --2- 172.21.15.101:0/2832888472 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6d940a1340 0x7f6d940a37f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6da00cb990 tx=0x7f6da0009920 comp rx=0 tx=0).stop 2023-07-18T03:36:50.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.880+0000 7f6db05ff700 1 -- 172.21.15.101:0/2832888472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6da8105af0 msgr2=0x7f6da819a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:50.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.880+0000 7f6db05ff700 1 --2- 172.21.15.101:0/2832888472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6da8105af0 0x7f6da819a750 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6da0009010 tx=0x7f6da0004a50 comp rx=0 tx=0).stop 2023-07-18T03:36:50.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.880+0000 7f6db05ff700 1 -- 172.21.15.101:0/2832888472 shutdown_connections 2023-07-18T03:36:50.904 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.880+0000 7f6db05ff700 1 --2- 172.21.15.101:0/2832888472 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6d940a1340 0x7f6d940a37f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:50.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.880+0000 7f6db05ff700 1 --2- 172.21.15.101:0/2832888472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6da8105af0 0x7f6da819a750 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:50.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.880+0000 7f6db05ff700 1 --2- 172.21.15.101:0/2832888472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6da80ffae0 0x7f6da8195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:50.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.880+0000 7f6db05ff700 1 --2- 172.21.15.101:0/2832888472 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6da8100490 0x7f6da81962f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:50.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.880+0000 7f6db05ff700 1 -- 172.21.15.101:0/2832888472 >> 172.21.15.101:0/2832888472 conn(0x7f6da80fb5e0 msgr2=0x7f6da8106940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:36:50.905 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.881+0000 7f6db05ff700 1 -- 172.21.15.101:0/2832888472 shutdown_connections 2023-07-18T03:36:50.906 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:36:50.881+0000 7f6db05ff700 1 -- 172.21.15.101:0/2832888472 wait complete. 2023-07-18T03:36:50.906 INFO:tasks.cephfs.filesystem:scrub status for tag:0f2f19e0-0429-465c-935e-223eb09490c3 - {'path': '/', 'tag': '0f2f19e0-0429-465c-935e-223eb09490c3', 'options': 'recursive,force'} 2023-07-18T03:36:51.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:36:51 smithi179 ceph-mon[129229]: pgmap v927: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:36:51.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:36:51 smithi101 ceph-mon[126379]: pgmap v927: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:36:51.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:36:51 smithi155 ceph-mon[128885]: pgmap v927: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:36:53.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:36:53 smithi179 ceph-mon[129229]: pgmap v928: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:36:53.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:36:53 smithi101 ceph-mon[126379]: pgmap v928: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:36:53.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:36:53 smithi155 ceph-mon[128885]: pgmap v928: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:36:55.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:36:55 smithi179 ceph-mon[129229]: pgmap v929: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:36:55.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:36:55 smithi101 ceph-mon[126379]: pgmap v929: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:36:55.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:36:55 smithi155 ceph-mon[128885]: pgmap v929: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:36:57.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:36:57 smithi179 ceph-mon[129229]: pgmap v930: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:36:57.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:36:57 smithi101 ceph-mon[126379]: pgmap v930: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:36:57.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:36:57 smithi155 ceph-mon[128885]: pgmap v930: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:36:59.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:36:59 smithi179 ceph-mon[129229]: pgmap v931: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:36:59.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:36:59 smithi101 ceph-mon[126379]: pgmap v931: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:36:59.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:36:59 smithi155 ceph-mon[128885]: pgmap v931: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:37:01.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:37:01 smithi179 ceph-mon[129229]: pgmap v932: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:37:01.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:37:01 smithi101 ceph-mon[126379]: pgmap v932: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:37:01.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:37:01 smithi155 ceph-mon[128885]: pgmap v932: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:37:03.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:37:03 smithi179 ceph-mon[129229]: pgmap v933: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-07-18T03:37:03.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:37:03 smithi101 ceph-mon[126379]: pgmap v933: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-07-18T03:37:03.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:37:03 smithi155 ceph-mon[128885]: pgmap v933: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-07-18T03:37:05.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:37:05 smithi179 ceph-mon[129229]: pgmap v934: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-07-18T03:37:05.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:37:05 smithi101 ceph-mon[126379]: pgmap v934: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-07-18T03:37:05.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:37:05 smithi155 ceph-mon[128885]: pgmap v934: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-07-18T03:37:06.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:37:06 smithi179 ceph-mon[129229]: osdmap e139: 12 total, 12 up, 12 in 2023-07-18T03:37:06.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:37:06 smithi101 ceph-mon[126379]: osdmap e139: 12 total, 12 up, 12 in 2023-07-18T03:37:06.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:37:06 smithi155 ceph-mon[128885]: osdmap e139: 12 total, 12 up, 12 in 2023-07-18T03:37:07.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:37:07 smithi179 ceph-mon[129229]: pgmap v936: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-07-18T03:37:07.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:37:07 smithi101 ceph-mon[126379]: pgmap v936: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-07-18T03:37:07.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:37:07 smithi155 ceph-mon[128885]: pgmap v936: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-07-18T03:37:09.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:37:09 smithi179 ceph-mon[129229]: pgmap v937: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-07-18T03:37:09.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:37:09 smithi101 ceph-mon[126379]: pgmap v937: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-07-18T03:37:09.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:37:09 smithi155 ceph-mon[128885]: pgmap v937: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-07-18T03:37:11.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:37:11 smithi179 ceph-mon[129229]: pgmap v938: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-07-18T03:37:11.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:37:11 smithi101 ceph-mon[126379]: pgmap v938: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-07-18T03:37:11.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:37:11 smithi155 ceph-mon[128885]: pgmap v938: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-07-18T03:37:13.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:37:13 smithi179 ceph-mon[129229]: pgmap v939: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 921 B/s wr, 6 op/s 2023-07-18T03:37:13.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:37:13 smithi101 ceph-mon[126379]: pgmap v939: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 921 B/s wr, 6 op/s 2023-07-18T03:37:13.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:37:13 smithi155 ceph-mon[128885]: pgmap v939: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 921 B/s wr, 6 op/s 2023-07-18T03:37:15.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:37:15 smithi179 ceph-mon[129229]: pgmap v940: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 921 B/s wr, 6 op/s 2023-07-18T03:37:15.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:37:15 smithi179 ceph-mon[129229]: osdmap e140: 12 total, 12 up, 12 in 2023-07-18T03:37:15.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:37:15 smithi101 ceph-mon[126379]: pgmap v940: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 921 B/s wr, 6 op/s 2023-07-18T03:37:15.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:37:15 smithi101 ceph-mon[126379]: osdmap e140: 12 total, 12 up, 12 in 2023-07-18T03:37:15.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:37:15 smithi155 ceph-mon[128885]: pgmap v940: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 921 B/s wr, 6 op/s 2023-07-18T03:37:15.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:37:15 smithi155 ceph-mon[128885]: osdmap e140: 12 total, 12 up, 12 in 2023-07-18T03:37:17.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:37:17 smithi179 ceph-mon[129229]: pgmap v942: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:37:17.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:37:17 smithi101 ceph-mon[126379]: pgmap v942: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:37:17.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:37:17 smithi155 ceph-mon[128885]: pgmap v942: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:37:19.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:37:19 smithi179 ceph-mon[129229]: pgmap v943: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:37:19.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:37:19 smithi101 ceph-mon[126379]: pgmap v943: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:37:19.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:37:19 smithi155 ceph-mon[128885]: pgmap v943: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:37:20.907 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:37:21.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.122+0000 7f67e9dff700 1 -- 172.21.15.101:0/2448239760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67e4105af0 msgr2=0x7f67e4105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:21.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.122+0000 7f67e9dff700 1 --2- 172.21.15.101:0/2448239760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67e4105af0 0x7f67e4105ec0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f67d4009990 tx=0x7f67d4009ca0 comp rx=0 tx=0).stop 2023-07-18T03:37:21.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.122+0000 7f67e9dff700 1 -- 172.21.15.101:0/2448239760 shutdown_connections 2023-07-18T03:37:21.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.122+0000 7f67e9dff700 1 --2- 172.21.15.101:0/2448239760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67e4105af0 0x7f67e4105ec0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:21.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.122+0000 7f67e9dff700 1 --2- 172.21.15.101:0/2448239760 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f67e4100490 0x7f67e410c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:21.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.122+0000 7f67e9dff700 1 --2- 172.21.15.101:0/2448239760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67e40ffae0 0x7f67e40fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:21.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.122+0000 7f67e9dff700 1 -- 172.21.15.101:0/2448239760 >> 172.21.15.101:0/2448239760 conn(0x7f67e40fb5e0 msgr2=0x7f67e40fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:21.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.122+0000 7f67e9dff700 1 -- 172.21.15.101:0/2448239760 shutdown_connections 2023-07-18T03:37:21.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.122+0000 7f67e9dff700 1 -- 172.21.15.101:0/2448239760 wait complete. 2023-07-18T03:37:21.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.123+0000 7f67e9dff700 1 Processor -- start 2023-07-18T03:37:21.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.123+0000 7f67e9dff700 1 -- start start 2023-07-18T03:37:21.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.123+0000 7f67e9dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f67e40ffae0 0x7f67e4072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:21.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.123+0000 7f67e9dff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67e4100490 0x7f67e4072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:21.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.123+0000 7f67e9dff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67e4105af0 0x7f67e41a3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:21.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.123+0000 7f67e9dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e4073580 con 0x7f67e40ffae0 2023-07-18T03:37:21.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.123+0000 7f67e9dff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e41a42f0 con 0x7f67e4100490 2023-07-18T03:37:21.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.123+0000 7f67e9dff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e41a4430 con 0x7f67e4105af0 2023-07-18T03:37:21.123 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.124+0000 7f67e37fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f67e40ffae0 0x7f67e4072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:21.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.124+0000 7f67e37fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f67e40ffae0 0x7f67e4072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:33998/0 (socket says 172.21.15.101:33998) 2023-07-18T03:37:21.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.124+0000 7f67e37fe700 1 -- 172.21.15.101:0/2955482041 learned_addr learned my addr 172.21.15.101:0/2955482041 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:37:21.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.124+0000 7f67e2ffd700 1 --2- 172.21.15.101:0/2955482041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67e4100490 0x7f67e4072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:21.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.124+0000 7f67e3fff700 1 --2- 172.21.15.101:0/2955482041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67e4105af0 0x7f67e41a3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:21.124 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.124+0000 7f67e37fe700 1 -- 172.21.15.101:0/2955482041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67e4105af0 msgr2=0x7f67e41a3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:21.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.124+0000 7f67e37fe700 1 --2- 172.21.15.101:0/2955482041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67e4105af0 0x7f67e41a3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:21.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.124+0000 7f67e37fe700 1 -- 172.21.15.101:0/2955482041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67e4100490 msgr2=0x7f67e4072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:21.125 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.124+0000 7f67e37fe700 1 --2- 172.21.15.101:0/2955482041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67e4100490 0x7f67e4072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:21.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.124+0000 7f67e37fe700 1 -- 172.21.15.101:0/2955482041 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d40095f0 con 0x7f67e40ffae0 2023-07-18T03:37:21.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.124+0000 7f67e2ffd700 1 --2- 172.21.15.101:0/2955482041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67e4100490 0x7f67e4072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:37:21.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.124+0000 7f67e37fe700 1 --2- 172.21.15.101:0/2955482041 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f67e40ffae0 0x7f67e4072810 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f67d4005910 tx=0x7f67d40057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:21.126 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.124+0000 7f67e0ff9700 1 -- 172.21.15.101:0/2955482041 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d401d070 con 0x7f67e40ffae0 2023-07-18T03:37:21.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.125+0000 7f67e0ff9700 1 -- 172.21.15.101:0/2955482041 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67d4022470 con 0x7f67e40ffae0 2023-07-18T03:37:21.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.125+0000 7f67e9dff700 1 -- 172.21.15.101:0/2955482041 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67e41a4630 con 0x7f67e40ffae0 2023-07-18T03:37:21.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.125+0000 7f67e0ff9700 1 -- 172.21.15.101:0/2955482041 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d4017650 con 0x7f67e40ffae0 2023-07-18T03:37:21.127 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.125+0000 7f67e9dff700 1 -- 172.21.15.101:0/2955482041 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67e41a4a50 con 0x7f67e40ffae0 2023-07-18T03:37:21.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.127+0000 7f67e0ff9700 1 -- 172.21.15.101:0/2955482041 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f67d40225e0 con 0x7f67e40ffae0 2023-07-18T03:37:21.131 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.129+0000 7f67fd724b80 1 Processor -- start 2023-07-18T03:37:21.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.129+0000 7f67e0ff9700 1 --2- 172.21.15.101:0/2955482041 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f67cc0a12c0 0x7f67cc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:21.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.129+0000 7f67fd724b80 1 -- start start 2023-07-18T03:37:21.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.129+0000 7f67e0ff9700 1 -- 172.21.15.101:0/2955482041 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f67d40ce530 con 0x7f67e40ffae0 2023-07-18T03:37:21.132 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.129+0000 7f67e3fff700 1 --2- 172.21.15.101:0/2955482041 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f67cc0a12c0 0x7f67cc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:21.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.129+0000 7f67fd724b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b8f18652d0 0x55b8f18656a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:21.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.129+0000 7f67fd724b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b8f1865be0 0x55b8f1870590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:21.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.129+0000 7f67fd724b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b8f1870ad0 0x55b8f1872eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:21.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.129+0000 7f67fd724b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8f176bcb0 con 0x55b8f18652d0 2023-07-18T03:37:21.133 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.129+0000 7f67fd724b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8f17bea40 con 0x55b8f1870ad0 2023-07-18T03:37:21.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.129+0000 7f67fd724b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8f17b7ed0 con 0x55b8f1865be0 2023-07-18T03:37:21.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.130+0000 7f67e2ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b8f1865be0 0x55b8f1870590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:21.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.130+0000 7f67e3fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b8f1870ad0 0x55b8f1872eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:21.134 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.130+0000 7f67e2ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b8f1865be0 0x55b8f1870590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:35606/0 (socket says 172.21.15.101:35606) 2023-07-18T03:37:21.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.130+0000 7f67e2ffd700 1 -- 172.21.15.101:0/1691021631 learned_addr learned my addr 172.21.15.101:0/1691021631 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:37:21.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.130+0000 7f67e37fe700 1 --2- 172.21.15.101:0/1691021631 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b8f18652d0 0x55b8f18656a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:21.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.130+0000 7f67e3fff700 1 --2- 172.21.15.101:0/2955482041 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f67cc0a12c0 0x7f67cc0a3770 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f67e4074920 tx=0x7f67dc007a80 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:21.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.130+0000 7f67e2ffd700 1 -- 172.21.15.101:0/1691021631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b8f1870ad0 msgr2=0x55b8f1872eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:21.135 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.130+0000 7f67e2ffd700 1 --2- 172.21.15.101:0/1691021631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b8f1870ad0 0x55b8f1872eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:21.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.130+0000 7f67e2ffd700 1 -- 172.21.15.101:0/1691021631 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b8f18652d0 msgr2=0x55b8f18656a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:21.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.130+0000 7f67e2ffd700 1 --2- 172.21.15.101:0/1691021631 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b8f18652d0 0x55b8f18656a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:21.136 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.130+0000 7f67e2ffd700 1 -- 172.21.15.101:0/1691021631 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8f15e5de0 con 0x55b8f1865be0 2023-07-18T03:37:21.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.130+0000 7f67e2ffd700 1 --2- 172.21.15.101:0/1691021631 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b8f1865be0 0x55b8f1870590 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f67d8009d80 tx=0x7f67d80116c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f41f40148efbbd6 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:21.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.130+0000 7f67c9ffb700 1 -- 172.21.15.101:0/1691021631 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d8011d40 con 0x55b8f1865be0 2023-07-18T03:37:21.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.131+0000 7f67c9ffb700 1 -- 172.21.15.101:0/1691021631 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67d8011ea0 con 0x55b8f1865be0 2023-07-18T03:37:21.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.131+0000 7f67c9ffb700 1 -- 172.21.15.101:0/1691021631 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d800f7b0 con 0x55b8f1865be0 2023-07-18T03:37:21.137 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.131+0000 7f67fd724b80 1 -- 172.21.15.101:0/1691021631 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b8f1865be0 msgr2=0x55b8f1870590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:21.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.131+0000 7f67fd724b80 1 --2- 172.21.15.101:0/1691021631 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b8f1865be0 0x55b8f1870590 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f67d8009d80 tx=0x7f67d80116c0 comp rx=0 tx=0).stop 2023-07-18T03:37:21.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.131+0000 7f67fd724b80 1 -- 172.21.15.101:0/1691021631 shutdown_connections 2023-07-18T03:37:21.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.131+0000 7f67fd724b80 1 --2- 172.21.15.101:0/1691021631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b8f1870ad0 0x55b8f1872eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:21.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.131+0000 7f67fd724b80 1 --2- 172.21.15.101:0/1691021631 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b8f18652d0 0x55b8f18656a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:21.138 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.131+0000 7f67fd724b80 1 --2- 172.21.15.101:0/1691021631 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b8f1865be0 0x55b8f1870590 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:21.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.131+0000 7f67fd724b80 1 -- 172.21.15.101:0/1691021631 >> 172.21.15.101:0/1691021631 conn(0x55b8f185bda0 msgr2=0x55b8f18616a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:21.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.131+0000 7f67fd724b80 1 -- 172.21.15.101:0/1691021631 shutdown_connections 2023-07-18T03:37:21.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.131+0000 7f67fd724b80 1 -- 172.21.15.101:0/1691021631 wait complete. 2023-07-18T03:37:21.139 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.132+0000 7f67fd724b80 1 -- start start 2023-07-18T03:37:21.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.132+0000 7f67fd724b80 1 Processor -- start 2023-07-18T03:37:21.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.133+0000 7f67fd724b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b8f18652d0 0x55b8f1870470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:21.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.133+0000 7f67fd724b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b8f1865be0 0x55b8f186d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:21.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.133+0000 7f67fd724b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b8f1870ad0 0x55b8f186dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:21.140 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.133+0000 7f67fd724b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8f1757ac0 con 0x55b8f18652d0 2023-07-18T03:37:21.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.133+0000 7f67fd724b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8f17577c0 con 0x55b8f1870ad0 2023-07-18T03:37:21.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.133+0000 7f67fd724b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8f163b920 con 0x55b8f1865be0 2023-07-18T03:37:21.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.133+0000 7f67fd724b80 10 client.0 _ll_register_callbacks cb 0x55b8f1755760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:37:21.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.133+0000 7f67e2ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b8f1865be0 0x55b8f186d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:21.141 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.133+0000 7f67e3fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b8f1870ad0 0x55b8f186dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:21.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.133+0000 7f67e3fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b8f1870ad0 0x55b8f186dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:55732/0 (socket says 172.21.15.101:55732) 2023-07-18T03:37:21.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.133+0000 7f67e3fff700 1 -- 172.21.15.101:0/2139850971 learned_addr learned my addr 172.21.15.101:0/2139850971 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:37:21.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.133+0000 7f67e2ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b8f1865be0 0x55b8f186d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:35620/0 (socket says 172.21.15.101:35620) 2023-07-18T03:37:21.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.133+0000 7f67e37fe700 1 --2- 172.21.15.101:0/2139850971 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b8f18652d0 0x55b8f1870470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:21.142 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.133+0000 7f67e3fff700 1 -- 172.21.15.101:0/2139850971 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b8f1865be0 msgr2=0x55b8f186d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:21.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.133+0000 7f67e3fff700 1 --2- 172.21.15.101:0/2139850971 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b8f1865be0 0x55b8f186d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:21.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.133+0000 7f67e3fff700 1 -- 172.21.15.101:0/2139850971 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b8f18652d0 msgr2=0x55b8f1870470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:21.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.133+0000 7f67e3fff700 1 --2- 172.21.15.101:0/2139850971 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b8f18652d0 0x55b8f1870470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:21.143 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.133+0000 7f67e3fff700 1 -- 172.21.15.101:0/2139850971 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67d80099b0 con 0x55b8f1870ad0 2023-07-18T03:37:21.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.133+0000 7f67e2ffd700 1 --2- 172.21.15.101:0/2139850971 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b8f1865be0 0x55b8f186d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:37:21.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.134+0000 7f67e3fff700 1 --2- 172.21.15.101:0/2139850971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b8f1870ad0 0x55b8f186dd90 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f67dc01ac10 tx=0x7f67dc01a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1aaee9dd3b6ce server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:21.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.134+0000 7f67c2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:37:21.144 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.134+0000 7f67c2ffd700 1 -- 172.21.15.101:0/2139850971 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67dc00e050 con 0x55b8f1870ad0 2023-07-18T03:37:21.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.134+0000 7f67fd724b80 1 -- 172.21.15.101:0/2139850971 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b8f17114f0 con 0x55b8f1870ad0 2023-07-18T03:37:21.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.134+0000 7f67c2ffd700 1 -- 172.21.15.101:0/2139850971 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(140..140 src has 1..140) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f67dc019bf0 con 0x55b8f1870ad0 2023-07-18T03:37:21.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.134+0000 7f67c2ffd700 1 -- 172.21.15.101:0/2139850971 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67dc019db0 con 0x55b8f1870ad0 2023-07-18T03:37:21.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.134+0000 7f67fd724b80 10 client.29369 fetch_fsmap learned FSMap version 49 2023-07-18T03:37:21.145 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.134+0000 7f67fd724b80 1 -- 172.21.15.101:0/2139850971 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55b8f15cec60 con 0x55b8f1870ad0 2023-07-18T03:37:21.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.134+0000 7f67c2ffd700 1 -- 172.21.15.101:0/2139850971 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b8f17114f0 con 0x55b8f1870ad0 2023-07-18T03:37:21.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.135+0000 7f67c2ffd700 1 -- 172.21.15.101:0/2139850971 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 49) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f67dc0173d0 con 0x55b8f1870ad0 2023-07-18T03:37:21.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.135+0000 7f67fd724b80 10 client.29369 fetch_fsmap finished waiting for FSMap version 49 2023-07-18T03:37:21.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.135+0000 7f67fd724b80 10 client.29369 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:37:21.146 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.135+0000 7f67fd724b80 20 client.29369 populate_metadata read hostname 'smithi101' 2023-07-18T03:37:21.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.135+0000 7f67fd724b80 1 --2- 172.21.15.101:0/2139850971 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b8f1880470 0x55b8f1882880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:21.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.135+0000 7f67fd724b80 4 client.29369 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:37:21.147 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.135+0000 7f67fd724b80 1 -- 172.21.15.101:0/2139850971 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b8f17b0eb0 con 0x55b8f1880470 2023-07-18T03:37:21.148 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.135+0000 7f67e37fe700 1 --2- 172.21.15.101:0/2139850971 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b8f1880470 0x55b8f1882880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:21.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.137+0000 7f67c2ffd700 10 client.29369 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:37:21.149 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.137+0000 7f67e37fe700 1 --2- 172.21.15.101:0/2139850971 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b8f1880470 0x55b8f1882880 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c24a16c2131efdb server_cookie=a55c60f7925564da in_seq=0 out_seq=0 2023-07-18T03:37:21.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.138+0000 7f67c2ffd700 1 -- 172.21.15.101:0/2139850971 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b8f17b0eb0 con 0x55b8f1880470 2023-07-18T03:37:21.150 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.138+0000 7f67c2ffd700 10 client.29369 handle_command_reply: tid=0 2023-07-18T03:37:21.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.138+0000 7f67fd724b80 1 client.29369 shutdown 2023-07-18T03:37:21.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.138+0000 7f67fd724b80 1 -- 172.21.15.101:0/2139850971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b8f1870ad0 msgr2=0x55b8f186dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:21.151 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.138+0000 7f67fd724b80 1 --2- 172.21.15.101:0/2139850971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b8f1870ad0 0x55b8f186dd90 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f67dc01ac10 tx=0x7f67dc01a000 comp rx=0 tx=0).stop 2023-07-18T03:37:21.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.138+0000 7f67fd724b80 1 -- 172.21.15.101:0/2139850971 shutdown_connections 2023-07-18T03:37:21.152 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.138+0000 7f67fd724b80 1 --2- 172.21.15.101:0/2139850971 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b8f1880470 0x55b8f1882880 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:21.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.138+0000 7f67fd724b80 1 --2- 172.21.15.101:0/2139850971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b8f1870ad0 0x55b8f186dd90 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:21.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.138+0000 7f67fd724b80 1 --2- 172.21.15.101:0/2139850971 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b8f18652d0 0x55b8f1870470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:21.153 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.138+0000 7f67fd724b80 1 --2- 172.21.15.101:0/2139850971 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b8f1865be0 0x55b8f186d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:21.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.138+0000 7f67c2ffd700 0 client.29369 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:37:21.154 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.138+0000 7f67fd724b80 1 -- 172.21.15.101:0/2139850971 >> 172.21.15.101:0/2139850971 conn(0x55b8f185bda0 msgr2=0x55b8f1861230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:21.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.138+0000 7f67fd724b80 1 -- 172.21.15.101:0/2139850971 shutdown_connections 2023-07-18T03:37:21.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.138+0000 7f67fd724b80 1 -- 172.21.15.101:0/2139850971 wait complete. 2023-07-18T03:37:21.155 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.138+0000 7f67fd724b80 20 client.29369 trim_cache size 0 max 16384 2023-07-18T03:37:21.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.155+0000 7f67fd724b80 1 Processor -- start 2023-07-18T03:37:21.162 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.155+0000 7f67fd724b80 1 -- start start 2023-07-18T03:37:21.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.156+0000 7f67fd724b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b8f18652d0 0x55b8f1868a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:21.163 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.156+0000 7f67fd724b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b8f1865be0 0x55b8f1868fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:21.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.156+0000 7f67fd724b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b8f1870ad0 0x55b8f186a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:21.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.156+0000 7f67fd724b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8f18802e0 con 0x55b8f18652d0 2023-07-18T03:37:21.164 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.156+0000 7f67fd724b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8f1866e60 con 0x55b8f1870ad0 2023-07-18T03:37:21.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.156+0000 7f67fd724b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8f1866540 con 0x55b8f1865be0 2023-07-18T03:37:21.165 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.156+0000 7f67e2ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b8f1865be0 0x55b8f1868fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:21.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.156+0000 7f67e2ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b8f1865be0 0x55b8f1868fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:35632/0 (socket says 172.21.15.101:35632) 2023-07-18T03:37:21.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.156+0000 7f67e37fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b8f18652d0 0x55b8f1868a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:21.166 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.156+0000 7f67e2ffd700 1 -- 172.21.15.101:0/2367756314 learned_addr learned my addr 172.21.15.101:0/2367756314 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:37:21.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.156+0000 7f67e3fff700 1 --2- 172.21.15.101:0/2367756314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b8f1870ad0 0x55b8f186a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:21.167 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.156+0000 7f67e2ffd700 1 -- 172.21.15.101:0/2367756314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b8f1870ad0 msgr2=0x55b8f186a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:21.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.156+0000 7f67e2ffd700 1 --2- 172.21.15.101:0/2367756314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b8f1870ad0 0x55b8f186a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:21.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.156+0000 7f67e2ffd700 1 -- 172.21.15.101:0/2367756314 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b8f18652d0 msgr2=0x55b8f1868a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:21.168 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.156+0000 7f67e2ffd700 1 --2- 172.21.15.101:0/2367756314 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b8f18652d0 0x55b8f1868a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:21.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.156+0000 7f67e2ffd700 1 -- 172.21.15.101:0/2367756314 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8f160c250 con 0x55b8f1865be0 2023-07-18T03:37:21.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.157+0000 7f67e2ffd700 1 --2- 172.21.15.101:0/2367756314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b8f1865be0 0x55b8f1868fd0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f67d8011a10 tx=0x7f67d8005940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e28d378cc042aea2 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:21.169 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.157+0000 7f67ca7fc700 1 -- 172.21.15.101:0/2367756314 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d80052b0 con 0x55b8f1865be0 2023-07-18T03:37:21.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.157+0000 7f67ca7fc700 1 -- 172.21.15.101:0/2367756314 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67d8005410 con 0x55b8f1865be0 2023-07-18T03:37:21.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.157+0000 7f67ca7fc700 1 -- 172.21.15.101:0/2367756314 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d80126b0 con 0x55b8f1865be0 2023-07-18T03:37:21.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.157+0000 7f67fd724b80 1 -- 172.21.15.101:0/2367756314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b8f1865be0 msgr2=0x55b8f1868fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:21.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.157+0000 7f67fd724b80 1 --2- 172.21.15.101:0/2367756314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b8f1865be0 0x55b8f1868fd0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f67d8011a10 tx=0x7f67d8005940 comp rx=0 tx=0).stop 2023-07-18T03:37:21.171 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.157+0000 7f67fd724b80 1 -- 172.21.15.101:0/2367756314 shutdown_connections 2023-07-18T03:37:21.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.157+0000 7f67fd724b80 1 --2- 172.21.15.101:0/2367756314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b8f1870ad0 0x55b8f186a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:21.172 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.157+0000 7f67fd724b80 1 --2- 172.21.15.101:0/2367756314 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b8f18652d0 0x55b8f1868a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:21.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.157+0000 7f67fd724b80 1 --2- 172.21.15.101:0/2367756314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b8f1865be0 0x55b8f1868fd0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:21.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.157+0000 7f67fd724b80 1 -- 172.21.15.101:0/2367756314 >> 172.21.15.101:0/2367756314 conn(0x55b8f185bda0 msgr2=0x55b8f18700a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:21.173 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.158+0000 7f67fd724b80 1 -- 172.21.15.101:0/2367756314 shutdown_connections 2023-07-18T03:37:21.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.158+0000 7f67fd724b80 1 -- 172.21.15.101:0/2367756314 wait complete. 2023-07-18T03:37:21.174 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.158+0000 7f67fd724b80 1 -- start start 2023-07-18T03:37:21.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.158+0000 7f67fd724b80 1 Processor -- start 2023-07-18T03:37:21.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.158+0000 7f67fd724b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b8f18652d0 0x55b8f1877540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:21.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.158+0000 7f67fd724b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b8f1865be0 0x55b8f1875930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:21.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.158+0000 7f67fd724b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b8f1870ad0 0x55b8f1875e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:21.176 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.159+0000 7f67fd724b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8f1876570 con 0x55b8f18652d0 2023-07-18T03:37:21.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.159+0000 7f67fd724b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8f18766e0 con 0x55b8f1865be0 2023-07-18T03:37:21.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.159+0000 7f67fd724b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8f18d4340 con 0x55b8f1870ad0 2023-07-18T03:37:21.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.159+0000 7f67fd724b80 10 client.0 _ll_register_callbacks cb 0x7f67dc01a840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:37:21.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.159+0000 7f67e37fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b8f18652d0 0x55b8f1877540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:21.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.159+0000 7f67e2ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b8f1865be0 0x55b8f1875930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:21.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.159+0000 7f67e37fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b8f18652d0 0x55b8f1877540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:34038/0 (socket says 172.21.15.101:34038) 2023-07-18T03:37:21.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.159+0000 7f67e37fe700 1 -- 172.21.15.101:0/4167878684 learned_addr learned my addr 172.21.15.101:0/4167878684 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:37:21.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.159+0000 7f67e3fff700 1 --2- 172.21.15.101:0/4167878684 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b8f1870ad0 0x55b8f1875e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:21.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.159+0000 7f67e2ffd700 1 -- 172.21.15.101:0/4167878684 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b8f1870ad0 msgr2=0x55b8f1875e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:21.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.159+0000 7f67e2ffd700 1 --2- 172.21.15.101:0/4167878684 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b8f1870ad0 0x55b8f1875e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:21.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.159+0000 7f67e2ffd700 1 -- 172.21.15.101:0/4167878684 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b8f18652d0 msgr2=0x55b8f1877540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:21.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.159+0000 7f67e2ffd700 1 --2- 172.21.15.101:0/4167878684 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b8f18652d0 0x55b8f1877540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:21.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.159+0000 7f67e2ffd700 1 -- 172.21.15.101:0/4167878684 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b8f15e5de0 con 0x55b8f1865be0 2023-07-18T03:37:21.180 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.159+0000 7f67e37fe700 1 --2- 172.21.15.101:0/4167878684 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b8f18652d0 0x55b8f1877540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:37:21.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.159+0000 7f67e2ffd700 1 --2- 172.21.15.101:0/4167878684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b8f1865be0 0x55b8f1875930 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x55b8f185ba70 tx=0x7f67d8004740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a778887516b2a78 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:21.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.159+0000 7f67c2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2023-07-18T03:37:21.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.160+0000 7f67c2ffd700 1 -- 172.21.15.101:0/4167878684 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d8004c20 con 0x55b8f1865be0 2023-07-18T03:37:21.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.160+0000 7f67c2ffd700 1 -- 172.21.15.101:0/4167878684 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(140..140 src has 1..140) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f67d8004d80 con 0x55b8f1865be0 2023-07-18T03:37:21.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.160+0000 7f67fd724b80 1 -- 172.21.15.101:0/4167878684 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f67d8011ea0 con 0x55b8f1865be0 2023-07-18T03:37:21.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.160+0000 7f67c2ffd700 1 -- 172.21.15.101:0/4167878684 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67d8012450 con 0x55b8f1865be0 2023-07-18T03:37:21.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.160+0000 7f67fd724b80 10 client.29375 fetch_fsmap learned FSMap version 49 2023-07-18T03:37:21.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.160+0000 7f67fd724b80 1 -- 172.21.15.101:0/4167878684 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55b8f15e42b0 con 0x55b8f1865be0 2023-07-18T03:37:21.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.160+0000 7f67c2ffd700 1 -- 172.21.15.101:0/4167878684 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67d8011ea0 con 0x55b8f1865be0 2023-07-18T03:37:21.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.160+0000 7f67c2ffd700 1 -- 172.21.15.101:0/4167878684 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 49) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f67d803f070 con 0x55b8f1865be0 2023-07-18T03:37:21.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.160+0000 7f67fd724b80 10 client.29375 fetch_fsmap finished waiting for FSMap version 49 2023-07-18T03:37:21.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.160+0000 7f67fd724b80 10 client.29375 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:37:21.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.160+0000 7f67fd724b80 20 client.29375 populate_metadata read hostname 'smithi101' 2023-07-18T03:37:21.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.160+0000 7f67fd724b80 1 --2- 172.21.15.101:0/4167878684 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b8f18d55a0 0x55b8f18d7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:21.183 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.160+0000 7f67fd724b80 4 client.29375 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:37:21.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.160+0000 7f67fd724b80 1 -- 172.21.15.101:0/4167878684 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b8f1773050 con 0x55b8f18d55a0 2023-07-18T03:37:21.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.161+0000 7f67e37fe700 1 --2- 172.21.15.101:0/4167878684 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b8f18d55a0 0x55b8f18d7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:21.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.161+0000 7f67e37fe700 1 --2- 172.21.15.101:0/4167878684 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b8f18d55a0 0x55b8f18d7980 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e19280c7ac4309af server_cookie=7879d628abd0a6ad in_seq=0 out_seq=0 2023-07-18T03:37:21.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.161+0000 7f67c2ffd700 10 client.29375 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:37:21.184 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.161+0000 7f67c2ffd700 1 -- 172.21.15.101:0/4167878684 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55b8f1773050 con 0x55b8f18d55a0 2023-07-18T03:37:21.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.161+0000 7f67c2ffd700 10 client.29375 handle_command_reply: tid=0 2023-07-18T03:37:21.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.161+0000 7f67fd724b80 1 client.29375 shutdown 2023-07-18T03:37:21.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.161+0000 7f67fd724b80 1 -- 172.21.15.101:0/4167878684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b8f1865be0 msgr2=0x55b8f1875930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:21.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.161+0000 7f67fd724b80 1 --2- 172.21.15.101:0/4167878684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b8f1865be0 0x55b8f1875930 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x55b8f185ba70 tx=0x7f67d8004740 comp rx=0 tx=0).stop 2023-07-18T03:37:21.185 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.161+0000 7f67fd724b80 1 -- 172.21.15.101:0/4167878684 shutdown_connections 2023-07-18T03:37:21.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.161+0000 7f67fd724b80 1 --2- 172.21.15.101:0/4167878684 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55b8f18d55a0 0x55b8f18d7980 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:21.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.161+0000 7f67fd724b80 1 --2- 172.21.15.101:0/4167878684 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b8f1870ad0 0x55b8f1875e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:21.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.161+0000 7f67fd724b80 1 --2- 172.21.15.101:0/4167878684 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b8f18652d0 0x55b8f1877540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:21.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.161+0000 7f67fd724b80 1 --2- 172.21.15.101:0/4167878684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b8f1865be0 0x55b8f1875930 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:21.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.161+0000 7f67c2ffd700 0 client.29375 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:37:21.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.161+0000 7f67fd724b80 1 -- 172.21.15.101:0/4167878684 >> 172.21.15.101:0/4167878684 conn(0x55b8f185bda0 msgr2=0x55b8f18673e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:21.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.162+0000 7f67fd724b80 1 -- 172.21.15.101:0/4167878684 shutdown_connections 2023-07-18T03:37:21.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.162+0000 7f67fd724b80 1 -- 172.21.15.101:0/4167878684 wait complete. 2023-07-18T03:37:21.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.162+0000 7f67fd724b80 20 client.29375 trim_cache size 0 max 16384 2023-07-18T03:37:21.188 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:37:21.188 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:37:21.188 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:37:21.188 INFO:teuthology.orchestra.run.smithi101.stdout: "0f2f19e0-0429-465c-935e-223eb09490c3": { 2023-07-18T03:37:21.189 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:37:21.189 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "0f2f19e0-0429-465c-935e-223eb09490c3", 2023-07-18T03:37:21.189 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:37:21.189 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:37:21.189 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:37:21.190 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:37:21.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.162+0000 7f67e9dff700 1 -- 172.21.15.101:0/2955482041 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f67cc0a12c0 msgr2=0x7f67cc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:21.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.162+0000 7f67e9dff700 1 --2- 172.21.15.101:0/2955482041 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f67cc0a12c0 0x7f67cc0a3770 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f67e4074920 tx=0x7f67dc007a80 comp rx=0 tx=0).stop 2023-07-18T03:37:21.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.162+0000 7f67e9dff700 1 -- 172.21.15.101:0/2955482041 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f67e40ffae0 msgr2=0x7f67e4072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:21.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.162+0000 7f67e9dff700 1 --2- 172.21.15.101:0/2955482041 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f67e40ffae0 0x7f67e4072810 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f67d4005910 tx=0x7f67d40057d0 comp rx=0 tx=0).stop 2023-07-18T03:37:21.191 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.162+0000 7f67e9dff700 1 -- 172.21.15.101:0/2955482041 shutdown_connections 2023-07-18T03:37:21.192 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.162+0000 7f67e9dff700 1 --2- 172.21.15.101:0/2955482041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67e4105af0 0x7f67e41a3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:21.192 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.162+0000 7f67e9dff700 1 --2- 172.21.15.101:0/2955482041 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f67e40ffae0 0x7f67e4072810 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:21.192 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.162+0000 7f67e9dff700 1 --2- 172.21.15.101:0/2955482041 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f67cc0a12c0 0x7f67cc0a3770 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:21.192 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.162+0000 7f67e9dff700 1 --2- 172.21.15.101:0/2955482041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67e4100490 0x7f67e4072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:21.192 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.162+0000 7f67e9dff700 1 -- 172.21.15.101:0/2955482041 >> 172.21.15.101:0/2955482041 conn(0x7f67e40fb5e0 msgr2=0x7f67e40fbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:21.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.162+0000 7f67e9dff700 1 -- 172.21.15.101:0/2955482041 shutdown_connections 2023-07-18T03:37:21.193 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:21.162+0000 7f67e9dff700 1 -- 172.21.15.101:0/2955482041 wait complete. 2023-07-18T03:37:21.193 INFO:tasks.cephfs.filesystem:scrub status for tag:0f2f19e0-0429-465c-935e-223eb09490c3 - {'path': '/', 'tag': '0f2f19e0-0429-465c-935e-223eb09490c3', 'options': 'recursive,force'} 2023-07-18T03:37:21.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:37:21 smithi179 ceph-mon[129229]: pgmap v944: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:37:21.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:37:21 smithi101 ceph-mon[126379]: pgmap v944: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:37:21.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:37:21 smithi155 ceph-mon[128885]: pgmap v944: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:37:23.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:37:23 smithi179 ceph-mon[129229]: pgmap v945: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:37:23.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:37:23 smithi101 ceph-mon[126379]: pgmap v945: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:37:23.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:37:23 smithi155 ceph-mon[128885]: pgmap v945: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:37:25.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:37:25 smithi179 ceph-mon[129229]: pgmap v946: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:37:25.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:37:25 smithi101 ceph-mon[126379]: pgmap v946: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:37:25.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:37:25 smithi155 ceph-mon[128885]: pgmap v946: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:37:27.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:37:27 smithi179 ceph-mon[129229]: pgmap v947: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 87 B/s wr, 7 op/s 2023-07-18T03:37:27.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:37:27 smithi101 ceph-mon[126379]: pgmap v947: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 87 B/s wr, 7 op/s 2023-07-18T03:37:27.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:37:27 smithi155 ceph-mon[128885]: pgmap v947: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 87 B/s wr, 7 op/s 2023-07-18T03:37:29.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:37:29 smithi179 ceph-mon[129229]: pgmap v948: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:37:29.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:37:29 smithi101 ceph-mon[126379]: pgmap v948: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:37:29.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:37:29 smithi155 ceph-mon[128885]: pgmap v948: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:37:31.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:37:31 smithi179 ceph-mon[129229]: pgmap v949: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:37:31.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:37:31 smithi101 ceph-mon[126379]: pgmap v949: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:37:31.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:37:31 smithi155 ceph-mon[128885]: pgmap v949: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:37:33.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:37:33 smithi179 ceph-mon[129229]: pgmap v950: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:37:33.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:37:33 smithi101 ceph-mon[126379]: pgmap v950: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:37:33.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:37:33 smithi155 ceph-mon[128885]: pgmap v950: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-18T03:37:34.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:37:34 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:37:34.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:37:34 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:37:34.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:37:34 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:37:35.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:37:35 smithi179 ceph-mon[129229]: pgmap v951: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:37:35.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:37:35 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:37:35.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:37:35 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:37:35.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:37:35 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:37:35.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:37:35 smithi101 ceph-mon[126379]: pgmap v951: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:37:35.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:37:35 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:37:35.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:37:35 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:37:35.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:37:35 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:37:35.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:37:35 smithi155 ceph-mon[128885]: pgmap v951: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:37:35.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:37:35 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:37:35.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:37:35 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:37:35.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:37:35 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:37:37.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:37:37 smithi179 ceph-mon[129229]: pgmap v952: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:37:37.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:37:37 smithi101 ceph-mon[126379]: pgmap v952: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:37:37.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:37:37 smithi155 ceph-mon[128885]: pgmap v952: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:37:39.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:37:39 smithi179 ceph-mon[129229]: pgmap v953: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-07-18T03:37:39.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:37:39 smithi101 ceph-mon[126379]: pgmap v953: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-07-18T03:37:39.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:37:39 smithi155 ceph-mon[128885]: pgmap v953: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-07-18T03:37:41.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:37:41 smithi179 ceph-mon[129229]: pgmap v954: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:37:41.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:37:41 smithi101 ceph-mon[126379]: pgmap v954: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:37:41.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:37:41 smithi155 ceph-mon[128885]: pgmap v954: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:37:43.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:37:43 smithi101 ceph-mon[126379]: pgmap v955: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2023-07-18T03:37:43.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:37:43 smithi155 ceph-mon[128885]: pgmap v955: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2023-07-18T03:37:43.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:37:43 smithi179 ceph-mon[129229]: pgmap v955: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2023-07-18T03:37:45.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:37:45 smithi101 ceph-mon[126379]: pgmap v956: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-07-18T03:37:45.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:37:45 smithi155 ceph-mon[128885]: pgmap v956: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-07-18T03:37:45.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:37:45 smithi179 ceph-mon[129229]: pgmap v956: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-07-18T03:37:47.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:37:47 smithi101 ceph-mon[126379]: pgmap v957: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:37:47.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:37:47 smithi155 ceph-mon[128885]: pgmap v957: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:37:47.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:37:47 smithi179 ceph-mon[129229]: pgmap v957: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:37:49.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:37:49 smithi101 ceph-mon[126379]: pgmap v958: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:37:49.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:37:49 smithi155 ceph-mon[128885]: pgmap v958: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:37:49.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:37:49 smithi179 ceph-mon[129229]: pgmap v958: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:37:51.194 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:37:51.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.426+0000 7f490a7ff700 1 -- 172.21.15.101:0/2378016501 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4904105af0 msgr2=0x7f4904105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:51.422 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.426+0000 7f490a7ff700 1 --2- 172.21.15.101:0/2378016501 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4904105af0 0x7f4904105ec0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f48f4009990 tx=0x7f48f4009ca0 comp rx=0 tx=0).stop 2023-07-18T03:37:51.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.426+0000 7f490a7ff700 1 -- 172.21.15.101:0/2378016501 shutdown_connections 2023-07-18T03:37:51.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.426+0000 7f490a7ff700 1 --2- 172.21.15.101:0/2378016501 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4904100490 0x7f490410c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:51.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.426+0000 7f490a7ff700 1 --2- 172.21.15.101:0/2378016501 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4904105af0 0x7f4904105ec0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:51.423 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.426+0000 7f490a7ff700 1 --2- 172.21.15.101:0/2378016501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49040ffae0 0x7f49040fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:51.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.426+0000 7f490a7ff700 1 -- 172.21.15.101:0/2378016501 >> 172.21.15.101:0/2378016501 conn(0x7f49040fb5e0 msgr2=0x7f49040fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:51.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.426+0000 7f490a7ff700 1 -- 172.21.15.101:0/2378016501 shutdown_connections 2023-07-18T03:37:51.424 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.426+0000 7f490a7ff700 1 -- 172.21.15.101:0/2378016501 wait complete. 2023-07-18T03:37:51.425 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.427+0000 7f490a7ff700 1 Processor -- start 2023-07-18T03:37:51.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.427+0000 7f490a7ff700 1 -- start start 2023-07-18T03:37:51.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.428+0000 7f490a7ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49040ffae0 0x7f4904197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:51.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.428+0000 7f490a7ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4904100490 0x7f49041982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:51.426 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.428+0000 7f490a7ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4904105af0 0x7f490419c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:51.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.428+0000 7f490a7ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490419cd90 con 0x7f4904100490 2023-07-18T03:37:51.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.428+0000 7f490a7ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490419cf00 con 0x7f49040ffae0 2023-07-18T03:37:51.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.428+0000 7f490a7ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490419d0c0 con 0x7f4904105af0 2023-07-18T03:37:51.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.428+0000 7f4903fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49040ffae0 0x7f4904197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:51.427 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.428+0000 7f48fbfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4904100490 0x7f49041982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:51.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.428+0000 7f4903fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49040ffae0 0x7f4904197db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:56616/0 (socket says 172.21.15.101:56616) 2023-07-18T03:37:51.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.428+0000 7f4908d9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4904105af0 0x7f490419c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:51.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.428+0000 7f4903fff700 1 -- 172.21.15.101:0/1910355553 learned_addr learned my addr 172.21.15.101:0/1910355553 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:37:51.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.428+0000 7f4903fff700 1 -- 172.21.15.101:0/1910355553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4904105af0 msgr2=0x7f490419c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:51.428 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.428+0000 7f4903fff700 1 --2- 172.21.15.101:0/1910355553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4904105af0 0x7f490419c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:51.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.428+0000 7f4903fff700 1 -- 172.21.15.101:0/1910355553 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4904100490 msgr2=0x7f49041982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:51.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.428+0000 7f4903fff700 1 --2- 172.21.15.101:0/1910355553 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4904100490 0x7f49041982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:51.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.428+0000 7f4903fff700 1 -- 172.21.15.101:0/1910355553 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f40095f0 con 0x7f49040ffae0 2023-07-18T03:37:51.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.429+0000 7f4903fff700 1 --2- 172.21.15.101:0/1910355553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49040ffae0 0x7f4904197db0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f48f4005720 tx=0x7f48f4005750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:51.429 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.429+0000 7f4901ffb700 1 -- 172.21.15.101:0/1910355553 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f401d070 con 0x7f49040ffae0 2023-07-18T03:37:51.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.429+0000 7f490a7ff700 1 -- 172.21.15.101:0/1910355553 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f490419d340 con 0x7f49040ffae0 2023-07-18T03:37:51.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.429+0000 7f4901ffb700 1 -- 172.21.15.101:0/1910355553 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48f4022470 con 0x7f49040ffae0 2023-07-18T03:37:51.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.429+0000 7f4901ffb700 1 -- 172.21.15.101:0/1910355553 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f4017650 con 0x7f49040ffae0 2023-07-18T03:37:51.430 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.429+0000 7f490a7ff700 1 -- 172.21.15.101:0/1910355553 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49041a4be0 con 0x7f49040ffae0 2023-07-18T03:37:51.433 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.432+0000 7f4901ffb700 1 -- 172.21.15.101:0/1910355553 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f48f40225e0 con 0x7f49040ffae0 2023-07-18T03:37:51.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.433+0000 7f491e12bb80 1 Processor -- start 2023-07-18T03:37:51.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.433+0000 7f491e12bb80 1 -- start start 2023-07-18T03:37:51.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.433+0000 7f4901ffb700 1 --2- 172.21.15.101:0/1910355553 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f48f00a12c0 0x7f48f00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:51.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.434+0000 7f4901ffb700 1 -- 172.21.15.101:0/1910355553 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f48f40ce450 con 0x7f49040ffae0 2023-07-18T03:37:51.434 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.434+0000 7f491e12bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7790e12d0 0x55f7790e16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:51.435 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.434+0000 7f491e12bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7790e1be0 0x55f7790ec590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:51.435 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.434+0000 7f4908d9c700 1 --2- 172.21.15.101:0/1910355553 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f48f00a12c0 0x7f48f00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:51.435 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.434+0000 7f491e12bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7790ecad0 0x55f7790eeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:51.435 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.434+0000 7f491e12bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f778fe7cb0 con 0x55f7790e12d0 2023-07-18T03:37:51.435 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.434+0000 7f491e12bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f77903aa40 con 0x55f7790ecad0 2023-07-18T03:37:51.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.434+0000 7f491e12bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f779033ed0 con 0x55f7790e1be0 2023-07-18T03:37:51.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.434+0000 7f4908d9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7790ecad0 0x55f7790eeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:51.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.434+0000 7f48fbfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7790e1be0 0x55f7790ec590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:51.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.434+0000 7f4908d9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7790ecad0 0x55f7790eeeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:56622/0 (socket says 172.21.15.101:56622) 2023-07-18T03:37:51.436 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.434+0000 7f4903fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7790e12d0 0x55f7790e16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:51.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.434+0000 7f4908d9c700 1 -- 172.21.15.101:0/406821623 learned_addr learned my addr 172.21.15.101:0/406821623 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:37:51.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.434+0000 7f4903fff700 1 -- 172.21.15.101:0/406821623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7790e1be0 msgr2=0x55f7790ec590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:51.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.434+0000 7f4903fff700 1 --2- 172.21.15.101:0/406821623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7790e1be0 0x55f7790ec590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:51.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.434+0000 7f4903fff700 1 -- 172.21.15.101:0/406821623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7790ecad0 msgr2=0x55f7790eeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:51.437 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.434+0000 7f4903fff700 1 --2- 172.21.15.101:0/406821623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7790ecad0 0x55f7790eeeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:51.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.434+0000 7f4903fff700 1 -- 172.21.15.101:0/406821623 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f778e61de0 con 0x55f7790e12d0 2023-07-18T03:37:51.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.434+0000 7f48fbfff700 1 --2- 172.21.15.101:0/406821623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7790e1be0 0x55f7790ec590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:37:51.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.434+0000 7f4908d9c700 1 --2- 172.21.15.101:0/406821623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7790ecad0 0x55f7790eeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:37:51.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.435+0000 7f4908d9c700 1 --2- 172.21.15.101:0/1910355553 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f48f00a12c0 0x7f48f00a3770 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f48fc009ec0 tx=0x7f48fc013580 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:51.438 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.435+0000 7f4903fff700 1 --2- 172.21.15.101:0/406821623 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7790e12d0 0x55f7790e16a0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f48f40cb790 tx=0x7f48f4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b09027a0cf2cec server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:51.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.435+0000 7f48fa7fc700 1 -- 172.21.15.101:0/406821623 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f409ace0 con 0x55f7790e12d0 2023-07-18T03:37:51.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.435+0000 7f48fa7fc700 1 -- 172.21.15.101:0/406821623 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48f409ab00 con 0x55f7790e12d0 2023-07-18T03:37:51.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.435+0000 7f48fa7fc700 1 -- 172.21.15.101:0/406821623 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f40228e0 con 0x55f7790e12d0 2023-07-18T03:37:51.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.436+0000 7f491e12bb80 1 -- 172.21.15.101:0/406821623 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7790e12d0 msgr2=0x55f7790e16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:51.439 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.436+0000 7f491e12bb80 1 --2- 172.21.15.101:0/406821623 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7790e12d0 0x55f7790e16a0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f48f40cb790 tx=0x7f48f4009310 comp rx=0 tx=0).stop 2023-07-18T03:37:51.440 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.436+0000 7f491e12bb80 1 -- 172.21.15.101:0/406821623 shutdown_connections 2023-07-18T03:37:51.440 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.436+0000 7f491e12bb80 1 --2- 172.21.15.101:0/406821623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7790ecad0 0x55f7790eeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:51.440 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.436+0000 7f491e12bb80 1 --2- 172.21.15.101:0/406821623 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7790e12d0 0x55f7790e16a0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:51.440 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.436+0000 7f491e12bb80 1 --2- 172.21.15.101:0/406821623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7790e1be0 0x55f7790ec590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:51.440 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.436+0000 7f491e12bb80 1 -- 172.21.15.101:0/406821623 >> 172.21.15.101:0/406821623 conn(0x55f7790d7da0 msgr2=0x55f7790dd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:51.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.436+0000 7f491e12bb80 1 -- 172.21.15.101:0/406821623 shutdown_connections 2023-07-18T03:37:51.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.436+0000 7f491e12bb80 1 -- 172.21.15.101:0/406821623 wait complete. 2023-07-18T03:37:51.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.436+0000 7f491e12bb80 1 -- start start 2023-07-18T03:37:51.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.436+0000 7f491e12bb80 1 Processor -- start 2023-07-18T03:37:51.441 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.437+0000 7f491e12bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7790e12d0 0x55f7790ecea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:51.442 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.437+0000 7f491e12bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7790e1be0 0x55f7790ed3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:51.442 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.437+0000 7f491e12bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7790ecad0 0x55f7790ed920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:51.442 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.437+0000 7f491e12bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f778fd3ac0 con 0x55f7790ecad0 2023-07-18T03:37:51.442 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.437+0000 7f491e12bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f778fd37c0 con 0x55f7790e12d0 2023-07-18T03:37:51.442 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.437+0000 7f491e12bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f778eb7920 con 0x55f7790e1be0 2023-07-18T03:37:51.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.437+0000 7f491e12bb80 10 client.0 _ll_register_callbacks cb 0x55f778fd1760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:37:51.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.437+0000 7f48fbfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7790e1be0 0x55f7790ed3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:51.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.437+0000 7f4903fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7790e12d0 0x55f7790ecea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:51.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.437+0000 7f48fbfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7790e1be0 0x55f7790ed3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41164/0 (socket says 172.21.15.101:41164) 2023-07-18T03:37:51.443 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.437+0000 7f4903fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7790e12d0 0x55f7790ecea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:56630/0 (socket says 172.21.15.101:56630) 2023-07-18T03:37:51.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.437+0000 7f48fbfff700 1 -- 172.21.15.101:0/4175453036 learned_addr learned my addr 172.21.15.101:0/4175453036 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:37:51.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.437+0000 7f4908d9c700 1 --2- 172.21.15.101:0/4175453036 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7790ecad0 0x55f7790ed920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:51.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.437+0000 7f48fbfff700 1 -- 172.21.15.101:0/4175453036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7790e12d0 msgr2=0x55f7790ecea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:51.444 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.438+0000 7f48fbfff700 1 --2- 172.21.15.101:0/4175453036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7790e12d0 0x55f7790ecea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:51.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.438+0000 7f48fbfff700 1 -- 172.21.15.101:0/4175453036 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7790ecad0 msgr2=0x55f7790ed920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:51.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.438+0000 7f48fbfff700 1 --2- 172.21.15.101:0/4175453036 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7790ecad0 0x55f7790ed920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:51.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.438+0000 7f48fbfff700 1 -- 172.21.15.101:0/4175453036 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48ec0098e0 con 0x55f7790e1be0 2023-07-18T03:37:51.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.438+0000 7f48fbfff700 1 --2- 172.21.15.101:0/4175453036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7790e1be0 0x55f7790ed3e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f48ec011870 tx=0x7f48ec011b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94bd162fc1055dfd server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:51.445 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.438+0000 7f48e37fe700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:37:51.447 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.438+0000 7f48e37fe700 1 -- 172.21.15.101:0/4175453036 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec016070 con 0x55f7790e1be0 2023-07-18T03:37:51.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.438+0000 7f491e12bb80 1 -- 172.21.15.101:0/4175453036 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f778f8d4f0 con 0x55f7790e1be0 2023-07-18T03:37:51.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.438+0000 7f48e37fe700 1 -- 172.21.15.101:0/4175453036 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(140..140 src has 1..140) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f48ec004300 con 0x55f7790e1be0 2023-07-18T03:37:51.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.438+0000 7f48e37fe700 1 -- 172.21.15.101:0/4175453036 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48ec0044c0 con 0x55f7790e1be0 2023-07-18T03:37:51.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.438+0000 7f491e12bb80 10 client.29380 fetch_fsmap learned FSMap version 49 2023-07-18T03:37:51.448 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.438+0000 7f491e12bb80 1 -- 172.21.15.101:0/4175453036 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f48fc01ac10 con 0x55f7790e1be0 2023-07-18T03:37:51.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.439+0000 7f48e37fe700 1 -- 172.21.15.101:0/4175453036 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f778f8d4f0 con 0x55f7790e1be0 2023-07-18T03:37:51.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.439+0000 7f48e37fe700 1 -- 172.21.15.101:0/4175453036 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 49) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f48ec00f8c0 con 0x55f7790e1be0 2023-07-18T03:37:51.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.439+0000 7f491e12bb80 10 client.29380 fetch_fsmap finished waiting for FSMap version 49 2023-07-18T03:37:51.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.439+0000 7f491e12bb80 10 client.29380 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:37:51.449 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.439+0000 7f491e12bb80 20 client.29380 populate_metadata read hostname 'smithi101' 2023-07-18T03:37:51.450 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.439+0000 7f491e12bb80 1 --2- 172.21.15.101:0/4175453036 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f7790fc1f0 0x55f7790fe600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:51.450 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.439+0000 7f491e12bb80 4 client.29380 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:37:51.450 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.439+0000 7f491e12bb80 1 -- 172.21.15.101:0/4175453036 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f77902ceb0 con 0x55f7790fc1f0 2023-07-18T03:37:51.450 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.439+0000 7f4903fff700 1 --2- 172.21.15.101:0/4175453036 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f7790fc1f0 0x55f7790fe600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:51.450 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.440+0000 7f48e37fe700 10 client.29380 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:37:51.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.440+0000 7f4903fff700 1 --2- 172.21.15.101:0/4175453036 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f7790fc1f0 0x55f7790fe600 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e4729354aaa0354 server_cookie=5519784c21ed07db in_seq=0 out_seq=0 2023-07-18T03:37:51.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.440+0000 7f48e37fe700 1 -- 172.21.15.101:0/4175453036 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f77902ceb0 con 0x55f7790fc1f0 2023-07-18T03:37:51.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.440+0000 7f48e37fe700 10 client.29380 handle_command_reply: tid=0 2023-07-18T03:37:51.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.440+0000 7f491e12bb80 1 client.29380 shutdown 2023-07-18T03:37:51.451 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.441+0000 7f491e12bb80 1 -- 172.21.15.101:0/4175453036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7790e1be0 msgr2=0x55f7790ed3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:51.452 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.441+0000 7f491e12bb80 1 --2- 172.21.15.101:0/4175453036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7790e1be0 0x55f7790ed3e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f48ec011870 tx=0x7f48ec011b80 comp rx=0 tx=0).stop 2023-07-18T03:37:51.452 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.441+0000 7f491e12bb80 1 -- 172.21.15.101:0/4175453036 shutdown_connections 2023-07-18T03:37:51.452 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.441+0000 7f491e12bb80 1 --2- 172.21.15.101:0/4175453036 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f7790fc1f0 0x55f7790fe600 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:51.452 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.441+0000 7f491e12bb80 1 --2- 172.21.15.101:0/4175453036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7790e12d0 0x55f7790ecea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:51.452 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.441+0000 7f491e12bb80 1 --2- 172.21.15.101:0/4175453036 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7790ecad0 0x55f7790ed920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:51.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.441+0000 7f48e37fe700 0 client.29380 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:37:51.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.441+0000 7f491e12bb80 1 --2- 172.21.15.101:0/4175453036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7790e1be0 0x55f7790ed3e0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:51.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.441+0000 7f491e12bb80 1 -- 172.21.15.101:0/4175453036 >> 172.21.15.101:0/4175453036 conn(0x55f7790d7da0 msgr2=0x55f7790dd290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:51.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.441+0000 7f491e12bb80 1 -- 172.21.15.101:0/4175453036 shutdown_connections 2023-07-18T03:37:51.453 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.441+0000 7f491e12bb80 1 -- 172.21.15.101:0/4175453036 wait complete. 2023-07-18T03:37:51.454 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.441+0000 7f491e12bb80 20 client.29380 trim_cache size 0 max 16384 2023-07-18T03:37:51.455 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.458+0000 7f491e12bb80 1 Processor -- start 2023-07-18T03:37:51.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.458+0000 7f491e12bb80 1 -- start start 2023-07-18T03:37:51.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.459+0000 7f491e12bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7790e12d0 0x55f7790eb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:51.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.459+0000 7f491e12bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7790e1be0 0x55f7790ed5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:51.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.459+0000 7f491e12bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7790ecad0 0x55f7790f7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:51.456 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.459+0000 7f491e12bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7790e2540 con 0x55f7790ecad0 2023-07-18T03:37:51.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.459+0000 7f491e12bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7790e2d60 con 0x55f7790e12d0 2023-07-18T03:37:51.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.459+0000 7f491e12bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7790d7970 con 0x55f7790e1be0 2023-07-18T03:37:51.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.459+0000 7f4903fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7790e12d0 0x55f7790eb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:51.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.459+0000 7f4903fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7790e12d0 0x55f7790eb2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:56634/0 (socket says 172.21.15.101:56634) 2023-07-18T03:37:51.457 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.459+0000 7f4903fff700 1 -- 172.21.15.101:0/3194005086 learned_addr learned my addr 172.21.15.101:0/3194005086 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:37:51.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.459+0000 7f48fbfff700 1 --2- 172.21.15.101:0/3194005086 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7790e1be0 0x55f7790ed5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:51.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.459+0000 7f4908d9c700 1 --2- 172.21.15.101:0/3194005086 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7790ecad0 0x55f7790f7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:51.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.459+0000 7f4903fff700 1 -- 172.21.15.101:0/3194005086 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7790e1be0 msgr2=0x55f7790ed5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:51.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.459+0000 7f4903fff700 1 --2- 172.21.15.101:0/3194005086 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7790e1be0 0x55f7790ed5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:51.458 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.459+0000 7f4903fff700 1 -- 172.21.15.101:0/3194005086 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7790ecad0 msgr2=0x55f7790f7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:51.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.459+0000 7f4903fff700 1 --2- 172.21.15.101:0/3194005086 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7790ecad0 0x55f7790f7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:51.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.459+0000 7f4903fff700 1 -- 172.21.15.101:0/3194005086 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7790d95e0 con 0x55f7790e12d0 2023-07-18T03:37:51.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.459+0000 7f4908d9c700 1 --2- 172.21.15.101:0/3194005086 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7790ecad0 0x55f7790f7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:37:51.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.459+0000 7f4903fff700 1 --2- 172.21.15.101:0/3194005086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7790e12d0 0x55f7790eb2a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f48f4022890 tx=0x7f48f4022cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=deffbd0553a9e952 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:51.459 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.460+0000 7f48faffd700 1 -- 172.21.15.101:0/3194005086 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f4009290 con 0x55f7790e12d0 2023-07-18T03:37:51.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.460+0000 7f48faffd700 1 -- 172.21.15.101:0/3194005086 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48f409ace0 con 0x55f7790e12d0 2023-07-18T03:37:51.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.460+0000 7f48faffd700 1 -- 172.21.15.101:0/3194005086 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f40ce170 con 0x55f7790e12d0 2023-07-18T03:37:51.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.460+0000 7f491e12bb80 1 -- 172.21.15.101:0/3194005086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7790e12d0 msgr2=0x55f7790eb2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:51.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.460+0000 7f491e12bb80 1 --2- 172.21.15.101:0/3194005086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7790e12d0 0x55f7790eb2a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f48f4022890 tx=0x7f48f4022cb0 comp rx=0 tx=0).stop 2023-07-18T03:37:51.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.460+0000 7f491e12bb80 1 -- 172.21.15.101:0/3194005086 shutdown_connections 2023-07-18T03:37:51.463 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.460+0000 7f491e12bb80 1 --2- 172.21.15.101:0/3194005086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7790e12d0 0x55f7790eb2a0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:51.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.460+0000 7f491e12bb80 1 --2- 172.21.15.101:0/3194005086 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7790ecad0 0x55f7790f7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:51.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.460+0000 7f491e12bb80 1 --2- 172.21.15.101:0/3194005086 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7790e1be0 0x55f7790ed5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:51.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.460+0000 7f491e12bb80 1 -- 172.21.15.101:0/3194005086 >> 172.21.15.101:0/3194005086 conn(0x55f7790d7da0 msgr2=0x55f7790ecea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:51.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.460+0000 7f491e12bb80 1 -- 172.21.15.101:0/3194005086 shutdown_connections 2023-07-18T03:37:51.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.460+0000 7f491e12bb80 1 -- 172.21.15.101:0/3194005086 wait complete. 2023-07-18T03:37:51.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.461+0000 7f491e12bb80 1 -- start start 2023-07-18T03:37:51.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.461+0000 7f491e12bb80 1 Processor -- start 2023-07-18T03:37:51.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.461+0000 7f491e12bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7790e12d0 0x55f7790f00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:51.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.461+0000 7f491e12bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7790e1be0 0x55f7790f0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:51.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.461+0000 7f491e12bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7790ecad0 0x55f7790f1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:51.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.461+0000 7f491e12bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7790f1bf0 con 0x55f7790e12d0 2023-07-18T03:37:51.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.461+0000 7f491e12bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7790f1d60 con 0x55f7790ecad0 2023-07-18T03:37:51.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.461+0000 7f491e12bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7790f1f20 con 0x55f7790e1be0 2023-07-18T03:37:51.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.461+0000 7f491e12bb80 10 client.0 _ll_register_callbacks cb 0x7f48ec009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:37:51.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.462+0000 7f48fbfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7790e1be0 0x55f7790f0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:51.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.462+0000 7f4903fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7790e12d0 0x55f7790f00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:51.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.462+0000 7f48fbfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7790e1be0 0x55f7790f0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:41186/0 (socket says 172.21.15.101:41186) 2023-07-18T03:37:51.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.462+0000 7f48fbfff700 1 -- 172.21.15.101:0/1740251090 learned_addr learned my addr 172.21.15.101:0/1740251090 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:37:51.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.462+0000 7f4903fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7790e12d0 0x55f7790f00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:53500/0 (socket says 172.21.15.101:53500) 2023-07-18T03:37:51.467 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.462+0000 7f4908d9c700 1 --2- 172.21.15.101:0/1740251090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7790ecad0 0x55f7790f1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:51.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.462+0000 7f4903fff700 1 -- 172.21.15.101:0/1740251090 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7790e1be0 msgr2=0x55f7790f0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:51.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.462+0000 7f4903fff700 1 --2- 172.21.15.101:0/1740251090 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7790e1be0 0x55f7790f0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:51.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.462+0000 7f4903fff700 1 -- 172.21.15.101:0/1740251090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7790ecad0 msgr2=0x55f7790f1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:51.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.462+0000 7f4903fff700 1 --2- 172.21.15.101:0/1740251090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7790ecad0 0x55f7790f1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:51.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.462+0000 7f4903fff700 1 -- 172.21.15.101:0/1740251090 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48f40b2040 con 0x55f7790e12d0 2023-07-18T03:37:51.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.462+0000 7f48fbfff700 1 --2- 172.21.15.101:0/1740251090 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7790e1be0 0x55f7790f0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:37:51.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.462+0000 7f4903fff700 1 --2- 172.21.15.101:0/1740251090 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7790e12d0 0x55f7790f00e0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f48f4022c60 tx=0x7f48f409b230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e89870d33df6091 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:51.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.462+0000 7f48e37fe700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:37:51.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.462+0000 7f48e37fe700 1 -- 172.21.15.101:0/1740251090 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f4062020 con 0x55f7790e12d0 2023-07-18T03:37:51.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.462+0000 7f48e37fe700 1 -- 172.21.15.101:0/1740251090 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(140..140 src has 1..140) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f48f409f1d0 con 0x55f7790e12d0 2023-07-18T03:37:51.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.462+0000 7f491e12bb80 1 -- 172.21.15.101:0/1740251090 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f48f409ab00 con 0x55f7790e12d0 2023-07-18T03:37:51.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.462+0000 7f48e37fe700 1 -- 172.21.15.101:0/1740251090 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48f4067080 con 0x55f7790e12d0 2023-07-18T03:37:51.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.463+0000 7f491e12bb80 10 client.19284 fetch_fsmap learned FSMap version 49 2023-07-18T03:37:51.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.463+0000 7f491e12bb80 1 -- 172.21.15.101:0/1740251090 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f49041a4be0 con 0x55f7790e12d0 2023-07-18T03:37:51.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.463+0000 7f48e37fe700 1 -- 172.21.15.101:0/1740251090 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48f409ab00 con 0x55f7790e12d0 2023-07-18T03:37:51.471 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.463+0000 7f48e37fe700 1 -- 172.21.15.101:0/1740251090 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 49) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f48f407e080 con 0x55f7790e12d0 2023-07-18T03:37:51.471 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.463+0000 7f491e12bb80 10 client.19284 fetch_fsmap finished waiting for FSMap version 49 2023-07-18T03:37:51.471 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.463+0000 7f491e12bb80 10 client.19284 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:37:51.471 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.463+0000 7f491e12bb80 20 client.19284 populate_metadata read hostname 'smithi101' 2023-07-18T03:37:51.471 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.463+0000 7f491e12bb80 1 --2- 172.21.15.101:0/1740251090 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f779150e70 0x55f779153250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:51.472 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.463+0000 7f491e12bb80 4 client.19284 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:37:51.472 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.463+0000 7f491e12bb80 1 -- 172.21.15.101:0/1740251090 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f778fef050 con 0x55f779150e70 2023-07-18T03:37:51.472 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.463+0000 7f48fbfff700 1 --2- 172.21.15.101:0/1740251090 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f779150e70 0x55f779153250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:51.472 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.464+0000 7f48fbfff700 1 --2- 172.21.15.101:0/1740251090 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f779150e70 0x55f779153250 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=926b66e048b30021 server_cookie=bf7a99a79614342a in_seq=0 out_seq=0 2023-07-18T03:37:51.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.464+0000 7f48e37fe700 10 client.19284 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:37:51.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.464+0000 7f48e37fe700 1 -- 172.21.15.101:0/1740251090 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55f778fef050 con 0x55f779150e70 2023-07-18T03:37:51.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.464+0000 7f48e37fe700 10 client.19284 handle_command_reply: tid=0 2023-07-18T03:37:51.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.464+0000 7f491e12bb80 1 client.19284 shutdown 2023-07-18T03:37:51.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.464+0000 7f491e12bb80 1 -- 172.21.15.101:0/1740251090 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7790e12d0 msgr2=0x55f7790f00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:51.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.464+0000 7f491e12bb80 1 --2- 172.21.15.101:0/1740251090 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7790e12d0 0x55f7790f00e0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f48f4022c60 tx=0x7f48f409b230 comp rx=0 tx=0).stop 2023-07-18T03:37:51.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.464+0000 7f491e12bb80 1 -- 172.21.15.101:0/1740251090 shutdown_connections 2023-07-18T03:37:51.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.464+0000 7f491e12bb80 1 --2- 172.21.15.101:0/1740251090 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55f779150e70 0x55f779153250 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:51.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.464+0000 7f491e12bb80 1 --2- 172.21.15.101:0/1740251090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7790ecad0 0x55f7790f1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:51.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.464+0000 7f491e12bb80 1 --2- 172.21.15.101:0/1740251090 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7790e12d0 0x55f7790f00e0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:51.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.464+0000 7f491e12bb80 1 --2- 172.21.15.101:0/1740251090 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7790e1be0 0x55f7790f0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:51.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.464+0000 7f48e37fe700 0 client.19284 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:37:51.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.464+0000 7f491e12bb80 1 -- 172.21.15.101:0/1740251090 >> 172.21.15.101:0/1740251090 conn(0x55f7790d7da0 msgr2=0x55f7790e3c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:51.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.464+0000 7f491e12bb80 1 -- 172.21.15.101:0/1740251090 shutdown_connections 2023-07-18T03:37:51.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.464+0000 7f491e12bb80 1 -- 172.21.15.101:0/1740251090 wait complete. 2023-07-18T03:37:51.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.464+0000 7f491e12bb80 20 client.19284 trim_cache size 0 max 16384 2023-07-18T03:37:51.476 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:37:51.476 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:37:51.477 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:37:51.477 INFO:teuthology.orchestra.run.smithi101.stdout: "0f2f19e0-0429-465c-935e-223eb09490c3": { 2023-07-18T03:37:51.477 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:37:51.477 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "0f2f19e0-0429-465c-935e-223eb09490c3", 2023-07-18T03:37:51.477 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:37:51.478 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:37:51.478 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:37:51.478 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:37:51.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.465+0000 7f490a7ff700 1 -- 172.21.15.101:0/1910355553 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f48f00a12c0 msgr2=0x7f48f00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:51.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.465+0000 7f490a7ff700 1 --2- 172.21.15.101:0/1910355553 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f48f00a12c0 0x7f48f00a3770 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f48fc009ec0 tx=0x7f48fc013580 comp rx=0 tx=0).stop 2023-07-18T03:37:51.480 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.465+0000 7f490a7ff700 1 -- 172.21.15.101:0/1910355553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49040ffae0 msgr2=0x7f4904197db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:51.480 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.465+0000 7f490a7ff700 1 --2- 172.21.15.101:0/1910355553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49040ffae0 0x7f4904197db0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f48f4005720 tx=0x7f48f4005750 comp rx=0 tx=0).stop 2023-07-18T03:37:51.480 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.465+0000 7f490a7ff700 1 -- 172.21.15.101:0/1910355553 shutdown_connections 2023-07-18T03:37:51.480 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.465+0000 7f490a7ff700 1 --2- 172.21.15.101:0/1910355553 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f48f00a12c0 0x7f48f00a3770 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:51.480 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.465+0000 7f490a7ff700 1 --2- 172.21.15.101:0/1910355553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49040ffae0 0x7f4904197db0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:51.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.465+0000 7f490a7ff700 1 --2- 172.21.15.101:0/1910355553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4904105af0 0x7f490419c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:51.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.465+0000 7f490a7ff700 1 --2- 172.21.15.101:0/1910355553 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4904100490 0x7f49041982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:51.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.465+0000 7f490a7ff700 1 -- 172.21.15.101:0/1910355553 >> 172.21.15.101:0/1910355553 conn(0x7f49040fb5e0 msgr2=0x7f49040fbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:51.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.465+0000 7f490a7ff700 1 -- 172.21.15.101:0/1910355553 shutdown_connections 2023-07-18T03:37:51.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:37:51.465+0000 7f490a7ff700 1 -- 172.21.15.101:0/1910355553 wait complete. 2023-07-18T03:37:51.482 INFO:tasks.cephfs.filesystem:scrub status for tag:0f2f19e0-0429-465c-935e-223eb09490c3 - {'path': '/', 'tag': '0f2f19e0-0429-465c-935e-223eb09490c3', 'options': 'recursive,force'} 2023-07-18T03:37:51.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:37:51 smithi101 ceph-mon[126379]: pgmap v959: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:37:51.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:37:51 smithi155 ceph-mon[128885]: pgmap v959: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:37:51.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:37:51 smithi179 ceph-mon[129229]: pgmap v959: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:37:53.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:37:53 smithi101 ceph-mon[126379]: pgmap v960: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:37:53.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:37:53 smithi155 ceph-mon[128885]: pgmap v960: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:37:53.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:37:53 smithi179 ceph-mon[129229]: pgmap v960: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:37:55.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:37:55 smithi101 ceph-mon[126379]: pgmap v961: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:37:55.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:37:55 smithi155 ceph-mon[128885]: pgmap v961: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:37:55.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:37:55 smithi179 ceph-mon[129229]: pgmap v961: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:37:57.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:37:57 smithi101 ceph-mon[126379]: pgmap v962: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:37:57.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:37:57 smithi155 ceph-mon[128885]: pgmap v962: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:37:57.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:37:57 smithi179 ceph-mon[129229]: pgmap v962: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:37:59.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:37:59 smithi101 ceph-mon[126379]: pgmap v963: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:37:59.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:37:59 smithi155 ceph-mon[128885]: pgmap v963: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:37:59.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:37:59 smithi179 ceph-mon[129229]: pgmap v963: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:38:01.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:01 smithi101 ceph-mon[126379]: pgmap v964: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:38:01.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:01 smithi155 ceph-mon[128885]: pgmap v964: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:38:01.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:01 smithi179 ceph-mon[129229]: pgmap v964: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:38:03.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:03 smithi101 ceph-mon[126379]: pgmap v965: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:38:03.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:03 smithi155 ceph-mon[128885]: pgmap v965: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:38:03.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:03 smithi179 ceph-mon[129229]: pgmap v965: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:38:05.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:05 smithi101 ceph-mon[126379]: pgmap v966: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:38:05.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:05 smithi155 ceph-mon[128885]: pgmap v966: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:38:05.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:05 smithi179 ceph-mon[129229]: pgmap v966: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:38:06.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:06 smithi155 ceph-mon[128885]: osdmap e141: 12 total, 12 up, 12 in 2023-07-18T03:38:06.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:06 smithi155 ceph-mon[128885]: pgmap v968: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-07-18T03:38:06.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:06 smithi179 ceph-mon[129229]: osdmap e141: 12 total, 12 up, 12 in 2023-07-18T03:38:06.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:06 smithi179 ceph-mon[129229]: pgmap v968: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-07-18T03:38:06.989 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:06 smithi101 ceph-mon[126379]: osdmap e141: 12 total, 12 up, 12 in 2023-07-18T03:38:06.990 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:06 smithi101 ceph-mon[126379]: pgmap v968: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-07-18T03:38:09.490 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:09 smithi101 ceph-mon[126379]: pgmap v969: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-07-18T03:38:09.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:09 smithi155 ceph-mon[128885]: pgmap v969: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-07-18T03:38:09.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:09 smithi179 ceph-mon[129229]: pgmap v969: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-07-18T03:38:11.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:11 smithi179 ceph-mon[129229]: pgmap v970: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-07-18T03:38:11.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:11 smithi101 ceph-mon[126379]: pgmap v970: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-07-18T03:38:11.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:11 smithi155 ceph-mon[128885]: pgmap v970: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-07-18T03:38:13.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:13 smithi179 ceph-mon[129229]: pgmap v971: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-07-18T03:38:13.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:13 smithi101 ceph-mon[126379]: pgmap v971: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-07-18T03:38:13.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:13 smithi155 ceph-mon[128885]: pgmap v971: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-07-18T03:38:15.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:15 smithi179 ceph-mon[129229]: pgmap v972: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-07-18T03:38:15.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:15 smithi179 ceph-mon[129229]: osdmap e142: 12 total, 12 up, 12 in 2023-07-18T03:38:15.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:15 smithi101 ceph-mon[126379]: pgmap v972: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-07-18T03:38:15.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:15 smithi101 ceph-mon[126379]: osdmap e142: 12 total, 12 up, 12 in 2023-07-18T03:38:15.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:15 smithi155 ceph-mon[128885]: pgmap v972: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-07-18T03:38:15.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:15 smithi155 ceph-mon[128885]: osdmap e142: 12 total, 12 up, 12 in 2023-07-18T03:38:17.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:17 smithi179 ceph-mon[129229]: pgmap v974: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:38:17.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:17 smithi101 ceph-mon[126379]: pgmap v974: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:38:17.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:17 smithi155 ceph-mon[128885]: pgmap v974: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:38:19.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:19 smithi179 ceph-mon[129229]: pgmap v975: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:38:19.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:19 smithi101 ceph-mon[126379]: pgmap v975: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:38:19.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:19 smithi155 ceph-mon[128885]: pgmap v975: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:38:21.483 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:38:21.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.699+0000 7f016b1ff700 1 -- 172.21.15.101:0/3046856899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0164105b10 msgr2=0x7f0164105ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:21.694 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.699+0000 7f016b1ff700 1 --2- 172.21.15.101:0/3046856899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0164105b10 0x7f0164105ee0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f0158009990 tx=0x7f0158009ca0 comp rx=0 tx=0).stop 2023-07-18T03:38:21.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.699+0000 7f016b1ff700 1 -- 172.21.15.101:0/3046856899 shutdown_connections 2023-07-18T03:38:21.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.699+0000 7f016b1ff700 1 --2- 172.21.15.101:0/3046856899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0164105b10 0x7f0164105ee0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:21.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.699+0000 7f016b1ff700 1 --2- 172.21.15.101:0/3046856899 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f01641004b0 0x7f016410c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:21.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.699+0000 7f016b1ff700 1 --2- 172.21.15.101:0/3046856899 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01640ffb00 0x7f01640fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:21.695 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.699+0000 7f016b1ff700 1 -- 172.21.15.101:0/3046856899 >> 172.21.15.101:0/3046856899 conn(0x7f01640fb600 msgr2=0x7f01640fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:21.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.699+0000 7f016b1ff700 1 -- 172.21.15.101:0/3046856899 shutdown_connections 2023-07-18T03:38:21.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.699+0000 7f016b1ff700 1 -- 172.21.15.101:0/3046856899 wait complete. 2023-07-18T03:38:21.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.700+0000 7f016b1ff700 1 Processor -- start 2023-07-18T03:38:21.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.700+0000 7f016b1ff700 1 -- start start 2023-07-18T03:38:21.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.700+0000 7f016b1ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f01640ffb00 0x7f0164197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:21.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.700+0000 7f016b1ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01641004b0 0x7f0164198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:21.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.700+0000 7f016b1ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0164105b10 0x7f016419c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:21.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.700+0000 7f016b1ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016419ce10 con 0x7f01640ffb00 2023-07-18T03:38:21.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.700+0000 7f016b1ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016419cf80 con 0x7f0164105b10 2023-07-18T03:38:21.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.700+0000 7f016b1ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016419d140 con 0x7f01641004b0 2023-07-18T03:38:21.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.701+0000 7f0163fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01641004b0 0x7f0164198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:21.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.701+0000 7f0163fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01641004b0 0x7f0164198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:44502/0 (socket says 172.21.15.101:44502) 2023-07-18T03:38:21.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.701+0000 7f016979c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0164105b10 0x7f016419c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:21.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.701+0000 7f0163fff700 1 -- 172.21.15.101:0/1158344674 learned_addr learned my addr 172.21.15.101:0/1158344674 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:38:21.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.701+0000 7f0168f9b700 1 --2- 172.21.15.101:0/1158344674 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f01640ffb00 0x7f0164197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:21.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.701+0000 7f0163fff700 1 -- 172.21.15.101:0/1158344674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0164105b10 msgr2=0x7f016419c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:21.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.701+0000 7f0163fff700 1 --2- 172.21.15.101:0/1158344674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0164105b10 0x7f016419c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:21.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.701+0000 7f0163fff700 1 -- 172.21.15.101:0/1158344674 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f01640ffb00 msgr2=0x7f0164197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:21.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.701+0000 7f0163fff700 1 --2- 172.21.15.101:0/1158344674 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f01640ffb00 0x7f0164197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:21.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.701+0000 7f0163fff700 1 -- 172.21.15.101:0/1158344674 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01580095f0 con 0x7f01641004b0 2023-07-18T03:38:21.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.701+0000 7f0163fff700 1 --2- 172.21.15.101:0/1158344674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01641004b0 0x7f0164198370 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f015000e9a0 tx=0x7f015000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:21.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.701+0000 7f0161ffb700 1 -- 172.21.15.101:0/1158344674 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0150009790 con 0x7f01641004b0 2023-07-18T03:38:21.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.701+0000 7f016b1ff700 1 -- 172.21.15.101:0/1158344674 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f016419d420 con 0x7f01641004b0 2023-07-18T03:38:21.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.701+0000 7f0161ffb700 1 -- 172.21.15.101:0/1158344674 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0150010430 con 0x7f01641004b0 2023-07-18T03:38:21.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.701+0000 7f0161ffb700 1 -- 172.21.15.101:0/1158344674 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0150018680 con 0x7f01641004b0 2023-07-18T03:38:21.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.702+0000 7f016b1ff700 1 -- 172.21.15.101:0/1158344674 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01641a4c60 con 0x7f01641004b0 2023-07-18T03:38:21.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:21 smithi179 ceph-mon[129229]: pgmap v976: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:38:21.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.705+0000 7f0161ffb700 1 -- 172.21.15.101:0/1158344674 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f01500105a0 con 0x7f01641004b0 2023-07-18T03:38:21.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.706+0000 7f017eb9cb80 1 Processor -- start 2023-07-18T03:38:21.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.706+0000 7f017eb9cb80 1 -- start start 2023-07-18T03:38:21.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.706+0000 7f0161ffb700 1 --2- 172.21.15.101:0/1158344674 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f01540a12c0 0x7f01540a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:21.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.707+0000 7f0161ffb700 1 -- 172.21.15.101:0/1158344674 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f0150014070 con 0x7f01641004b0 2023-07-18T03:38:21.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.707+0000 7f016979c700 1 --2- 172.21.15.101:0/1158344674 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f01540a12c0 0x7f01540a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:21.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.707+0000 7f017eb9cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fceaf522d0 0x55fceaf526a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:21.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.707+0000 7f017eb9cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fceaf52be0 0x55fceaf5d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:21.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.707+0000 7f017eb9cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fceaf5dad0 0x55fceaf5feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:21.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.707+0000 7f017eb9cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fceae58cb0 con 0x55fceaf522d0 2023-07-18T03:38:21.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.707+0000 7f017eb9cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fceaeaba40 con 0x55fceaf5dad0 2023-07-18T03:38:21.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.707+0000 7f017eb9cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fceaea4ed0 con 0x55fceaf52be0 2023-07-18T03:38:21.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.707+0000 7f0163fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fceaf52be0 0x55fceaf5d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:21.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.707+0000 7f0163fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fceaf52be0 0x55fceaf5d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:44512/0 (socket says 172.21.15.101:44512) 2023-07-18T03:38:21.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.707+0000 7f0163fff700 1 -- 172.21.15.101:0/2335395332 learned_addr learned my addr 172.21.15.101:0/2335395332 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:38:21.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.707+0000 7f016979c700 1 --2- 172.21.15.101:0/2335395332 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fceaf5dad0 0x55fceaf5feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:21.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.707+0000 7f0168f9b700 1 --2- 172.21.15.101:0/2335395332 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fceaf522d0 0x55fceaf526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:21.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.707+0000 7f0163fff700 1 -- 172.21.15.101:0/2335395332 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fceaf5dad0 msgr2=0x55fceaf5feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:21.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.707+0000 7f0163fff700 1 --2- 172.21.15.101:0/2335395332 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fceaf5dad0 0x55fceaf5feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:21.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.707+0000 7f0163fff700 1 -- 172.21.15.101:0/2335395332 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fceaf522d0 msgr2=0x55fceaf526a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:21.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.707+0000 7f0163fff700 1 --2- 172.21.15.101:0/2335395332 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fceaf522d0 0x55fceaf526a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:21.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.707+0000 7f0163fff700 1 -- 172.21.15.101:0/2335395332 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fceacd2de0 con 0x55fceaf52be0 2023-07-18T03:38:21.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.708+0000 7f016979c700 1 --2- 172.21.15.101:0/1158344674 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f01540a12c0 0x7f01540a3770 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f015c00f880 tx=0x7f015c00fc40 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:21.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.708+0000 7f0163fff700 1 --2- 172.21.15.101:0/2335395332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fceaf52be0 0x55fceaf5d590 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0150096a10 tx=0x7f0150099c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=460d1aa6f46e77ba server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:21.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.708+0000 7f014effd700 1 -- 172.21.15.101:0/2335395332 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01500a23f0 con 0x55fceaf52be0 2023-07-18T03:38:21.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.708+0000 7f014effd700 1 -- 172.21.15.101:0/2335395332 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0150010880 con 0x55fceaf52be0 2023-07-18T03:38:21.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.708+0000 7f014effd700 1 -- 172.21.15.101:0/2335395332 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01500996d0 con 0x55fceaf52be0 2023-07-18T03:38:21.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.709+0000 7f017eb9cb80 1 -- 172.21.15.101:0/2335395332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fceaf52be0 msgr2=0x55fceaf5d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:21.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.709+0000 7f017eb9cb80 1 --2- 172.21.15.101:0/2335395332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fceaf52be0 0x55fceaf5d590 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0150096a10 tx=0x7f0150099c10 comp rx=0 tx=0).stop 2023-07-18T03:38:21.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.709+0000 7f017eb9cb80 1 -- 172.21.15.101:0/2335395332 shutdown_connections 2023-07-18T03:38:21.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.709+0000 7f017eb9cb80 1 --2- 172.21.15.101:0/2335395332 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fceaf5dad0 0x55fceaf5feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:21.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.709+0000 7f017eb9cb80 1 --2- 172.21.15.101:0/2335395332 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fceaf522d0 0x55fceaf526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:21.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.709+0000 7f017eb9cb80 1 --2- 172.21.15.101:0/2335395332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fceaf52be0 0x55fceaf5d590 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:21.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.709+0000 7f017eb9cb80 1 -- 172.21.15.101:0/2335395332 >> 172.21.15.101:0/2335395332 conn(0x55fceaf48da0 msgr2=0x55fceaf4e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:21.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.709+0000 7f017eb9cb80 1 -- 172.21.15.101:0/2335395332 shutdown_connections 2023-07-18T03:38:21.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.709+0000 7f017eb9cb80 1 -- 172.21.15.101:0/2335395332 wait complete. 2023-07-18T03:38:21.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.709+0000 7f017eb9cb80 1 -- start start 2023-07-18T03:38:21.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.709+0000 7f017eb9cb80 1 Processor -- start 2023-07-18T03:38:21.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.710+0000 7f017eb9cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fceaf522d0 0x55fceaf5d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:21.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.710+0000 7f017eb9cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fceaf52be0 0x55fceaf5a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:21.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.710+0000 7f017eb9cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fceaf5dad0 0x55fceaf5af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:21.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.710+0000 7f017eb9cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fceae44ac0 con 0x55fceaf52be0 2023-07-18T03:38:21.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.710+0000 7f017eb9cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fceae447c0 con 0x55fceaf5dad0 2023-07-18T03:38:21.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.710+0000 7f017eb9cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcead28920 con 0x55fceaf522d0 2023-07-18T03:38:21.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.710+0000 7f017eb9cb80 10 client.0 _ll_register_callbacks cb 0x55fceae42760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:38:21.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.710+0000 7f0168f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fceaf522d0 0x55fceaf5d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:21.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.710+0000 7f0168f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fceaf522d0 0x55fceaf5d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:44526/0 (socket says 172.21.15.101:44526) 2023-07-18T03:38:21.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.710+0000 7f0168f9b700 1 -- 172.21.15.101:0/2049895034 learned_addr learned my addr 172.21.15.101:0/2049895034 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:38:21.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.710+0000 7f0163fff700 1 --2- 172.21.15.101:0/2049895034 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fceaf52be0 0x55fceaf5a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:21.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.710+0000 7f016979c700 1 --2- 172.21.15.101:0/2049895034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fceaf5dad0 0x55fceaf5af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:21.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.711+0000 7f0168f9b700 1 -- 172.21.15.101:0/2049895034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fceaf5dad0 msgr2=0x55fceaf5af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:21.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.711+0000 7f0168f9b700 1 --2- 172.21.15.101:0/2049895034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fceaf5dad0 0x55fceaf5af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:21.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.711+0000 7f0168f9b700 1 -- 172.21.15.101:0/2049895034 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fceaf52be0 msgr2=0x55fceaf5a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:21.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.711+0000 7f0168f9b700 1 --2- 172.21.15.101:0/2049895034 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fceaf52be0 0x55fceaf5a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:21.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.711+0000 7f0168f9b700 1 -- 172.21.15.101:0/2049895034 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01641a4c60 con 0x55fceaf522d0 2023-07-18T03:38:21.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.711+0000 7f0168f9b700 1 --2- 172.21.15.101:0/2049895034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fceaf522d0 0x55fceaf5d580 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f015800c010 tx=0x7f0158009570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38a712bb5ee21a1c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:21.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.711+0000 7f0143fff700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:38:21.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.711+0000 7f0143fff700 1 -- 172.21.15.101:0/2049895034 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015801d030 con 0x55fceaf522d0 2023-07-18T03:38:21.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.711+0000 7f0143fff700 1 -- 172.21.15.101:0/2049895034 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(142..142 src has 1..142) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f0158004500 con 0x55fceaf522d0 2023-07-18T03:38:21.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.711+0000 7f017eb9cb80 1 -- 172.21.15.101:0/2049895034 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fceadfe4f0 con 0x55fceaf522d0 2023-07-18T03:38:21.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.711+0000 7f0143fff700 1 -- 172.21.15.101:0/2049895034 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01580046c0 con 0x55fceaf522d0 2023-07-18T03:38:21.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.711+0000 7f017eb9cb80 10 client.29398 fetch_fsmap learned FSMap version 49 2023-07-18T03:38:21.720 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.711+0000 7f017eb9cb80 1 -- 172.21.15.101:0/2049895034 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55fceacbbc60 con 0x55fceaf522d0 2023-07-18T03:38:21.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.712+0000 7f0143fff700 1 -- 172.21.15.101:0/2049895034 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fceadfe4f0 con 0x55fceaf522d0 2023-07-18T03:38:21.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.712+0000 7f0143fff700 1 -- 172.21.15.101:0/2049895034 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 49) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f015800f980 con 0x55fceaf522d0 2023-07-18T03:38:21.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.712+0000 7f017eb9cb80 10 client.29398 fetch_fsmap finished waiting for FSMap version 49 2023-07-18T03:38:21.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.712+0000 7f017eb9cb80 10 client.29398 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:38:21.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.712+0000 7f017eb9cb80 20 client.29398 populate_metadata read hostname 'smithi101' 2023-07-18T03:38:21.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.712+0000 7f017eb9cb80 1 --2- 172.21.15.101:0/2049895034 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55fceaf6d490 0x55fceaf6f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:21.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.712+0000 7f017eb9cb80 4 client.29398 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:38:21.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.712+0000 7f017eb9cb80 1 -- 172.21.15.101:0/2049895034 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fceae9deb0 con 0x55fceaf6d490 2023-07-18T03:38:21.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.712+0000 7f0163fff700 1 --2- 172.21.15.101:0/2049895034 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55fceaf6d490 0x55fceaf6f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:21.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.713+0000 7f0163fff700 1 --2- 172.21.15.101:0/2049895034 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55fceaf6d490 0x55fceaf6f8a0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e3d5c3383ec583d server_cookie=58ce89e45c452b3e in_seq=0 out_seq=0 2023-07-18T03:38:21.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.713+0000 7f0143fff700 10 client.29398 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:38:21.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.713+0000 7f0143fff700 1 -- 172.21.15.101:0/2049895034 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55fceae9deb0 con 0x55fceaf6d490 2023-07-18T03:38:21.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.713+0000 7f0143fff700 10 client.29398 handle_command_reply: tid=0 2023-07-18T03:38:21.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.713+0000 7f017eb9cb80 1 client.29398 shutdown 2023-07-18T03:38:21.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.713+0000 7f017eb9cb80 1 -- 172.21.15.101:0/2049895034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fceaf522d0 msgr2=0x55fceaf5d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:21.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.713+0000 7f017eb9cb80 1 --2- 172.21.15.101:0/2049895034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fceaf522d0 0x55fceaf5d580 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f015800c010 tx=0x7f0158009570 comp rx=0 tx=0).stop 2023-07-18T03:38:21.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.713+0000 7f017eb9cb80 1 -- 172.21.15.101:0/2049895034 shutdown_connections 2023-07-18T03:38:21.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.713+0000 7f017eb9cb80 1 --2- 172.21.15.101:0/2049895034 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55fceaf6d490 0x55fceaf6f8a0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:21.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.713+0000 7f017eb9cb80 1 --2- 172.21.15.101:0/2049895034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fceaf5dad0 0x55fceaf5af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:21.724 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.713+0000 7f017eb9cb80 1 --2- 172.21.15.101:0/2049895034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fceaf522d0 0x55fceaf5d580 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:21.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.713+0000 7f017eb9cb80 1 --2- 172.21.15.101:0/2049895034 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fceaf52be0 0x55fceaf5a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:21.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.713+0000 7f0143fff700 0 client.29398 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:38:21.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.713+0000 7f017eb9cb80 1 -- 172.21.15.101:0/2049895034 >> 172.21.15.101:0/2049895034 conn(0x55fceaf48da0 msgr2=0x55fceaf4e230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:21.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.714+0000 7f017eb9cb80 1 -- 172.21.15.101:0/2049895034 shutdown_connections 2023-07-18T03:38:21.725 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.714+0000 7f017eb9cb80 1 -- 172.21.15.101:0/2049895034 wait complete. 2023-07-18T03:38:21.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.714+0000 7f017eb9cb80 20 client.29398 trim_cache size 0 max 16384 2023-07-18T03:38:21.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.731+0000 7f017eb9cb80 1 Processor -- start 2023-07-18T03:38:21.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.731+0000 7f017eb9cb80 1 -- start start 2023-07-18T03:38:21.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.731+0000 7f017eb9cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fceaf522d0 0x55fceaf5d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:21.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.731+0000 7f017eb9cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fceaf52be0 0x55fceaf59fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:21.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.731+0000 7f017eb9cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fceaf5dad0 0x55fceaf5a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:21.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.731+0000 7f017eb9cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fceaf55680 con 0x55fceaf52be0 2023-07-18T03:38:21.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.731+0000 7f017eb9cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fceaf45fa0 con 0x55fceaf5dad0 2023-07-18T03:38:21.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.731+0000 7f017eb9cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fceaf528f0 con 0x55fceaf522d0 2023-07-18T03:38:21.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.731+0000 7f016979c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fceaf5dad0 0x55fceaf5a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:21.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.731+0000 7f0163fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fceaf52be0 0x55fceaf59fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:21.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.731+0000 7f0168f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fceaf522d0 0x55fceaf5d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:21.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.731+0000 7f016979c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fceaf5dad0 0x55fceaf5a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:37828/0 (socket says 172.21.15.101:37828) 2023-07-18T03:38:21.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.731+0000 7f0168f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fceaf522d0 0x55fceaf5d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:44540/0 (socket says 172.21.15.101:44540) 2023-07-18T03:38:21.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.731+0000 7f016979c700 1 -- 172.21.15.101:0/2930783948 learned_addr learned my addr 172.21.15.101:0/2930783948 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:38:21.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.732+0000 7f016979c700 1 -- 172.21.15.101:0/2930783948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fceaf522d0 msgr2=0x55fceaf5d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:21.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.732+0000 7f016979c700 1 --2- 172.21.15.101:0/2930783948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fceaf522d0 0x55fceaf5d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:21.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.732+0000 7f016979c700 1 -- 172.21.15.101:0/2930783948 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fceaf52be0 msgr2=0x55fceaf59fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:21.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.732+0000 7f016979c700 1 --2- 172.21.15.101:0/2930783948 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fceaf52be0 0x55fceaf59fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:21.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.732+0000 7f016979c700 1 -- 172.21.15.101:0/2930783948 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fceacf9250 con 0x55fceaf5dad0 2023-07-18T03:38:21.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.732+0000 7f0163fff700 1 --2- 172.21.15.101:0/2930783948 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fceaf52be0 0x55fceaf59fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:21.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.732+0000 7f016979c700 1 --2- 172.21.15.101:0/2930783948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fceaf5dad0 0x55fceaf5a500 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f015c009330 tx=0x7f015c01f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d25d0087662ae5f0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:21.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.732+0000 7f014f7fe700 1 -- 172.21.15.101:0/2930783948 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015c00e040 con 0x55fceaf5dad0 2023-07-18T03:38:21.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.732+0000 7f014f7fe700 1 -- 172.21.15.101:0/2930783948 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f015c00f040 con 0x55fceaf5dad0 2023-07-18T03:38:21.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.732+0000 7f014f7fe700 1 -- 172.21.15.101:0/2930783948 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015c004800 con 0x55fceaf5dad0 2023-07-18T03:38:21.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.733+0000 7f017eb9cb80 1 -- 172.21.15.101:0/2930783948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fceaf5dad0 msgr2=0x55fceaf5a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:21.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.733+0000 7f017eb9cb80 1 --2- 172.21.15.101:0/2930783948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fceaf5dad0 0x55fceaf5a500 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f015c009330 tx=0x7f015c01f000 comp rx=0 tx=0).stop 2023-07-18T03:38:21.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.733+0000 7f017eb9cb80 1 -- 172.21.15.101:0/2930783948 shutdown_connections 2023-07-18T03:38:21.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.733+0000 7f017eb9cb80 1 --2- 172.21.15.101:0/2930783948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fceaf5dad0 0x55fceaf5a500 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:21.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.733+0000 7f017eb9cb80 1 --2- 172.21.15.101:0/2930783948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fceaf522d0 0x55fceaf5d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:21.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.733+0000 7f017eb9cb80 1 --2- 172.21.15.101:0/2930783948 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fceaf52be0 0x55fceaf59fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:21.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.733+0000 7f017eb9cb80 1 -- 172.21.15.101:0/2930783948 >> 172.21.15.101:0/2930783948 conn(0x55fceaf48da0 msgr2=0x55fceaf55b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:21.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.733+0000 7f017eb9cb80 1 -- 172.21.15.101:0/2930783948 shutdown_connections 2023-07-18T03:38:21.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.733+0000 7f017eb9cb80 1 -- 172.21.15.101:0/2930783948 wait complete. 2023-07-18T03:38:21.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.733+0000 7f017eb9cb80 1 -- start start 2023-07-18T03:38:21.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.733+0000 7f017eb9cb80 1 Processor -- start 2023-07-18T03:38:21.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.734+0000 7f017eb9cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fceaf522d0 0x55fceaf646b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:21.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.734+0000 7f017eb9cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fceaf52be0 0x55fceaf62ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:21.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.734+0000 7f017eb9cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fceaf5dad0 0x55fceaf63210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:21.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.734+0000 7f017eb9cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fceaf638e0 con 0x55fceaf52be0 2023-07-18T03:38:21.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.734+0000 7f017eb9cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fceafc1360 con 0x55fceaf5dad0 2023-07-18T03:38:21.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.734+0000 7f017eb9cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fceafc14a0 con 0x55fceaf522d0 2023-07-18T03:38:21.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.734+0000 7f017eb9cb80 10 client.0 _ll_register_callbacks cb 0x7f015800bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:38:21.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.734+0000 7f0168f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fceaf522d0 0x55fceaf646b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:21.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.734+0000 7f0168f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fceaf522d0 0x55fceaf646b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:44546/0 (socket says 172.21.15.101:44546) 2023-07-18T03:38:21.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.734+0000 7f0168f9b700 1 -- 172.21.15.101:0/1932955647 learned_addr learned my addr 172.21.15.101:0/1932955647 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:38:21.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.734+0000 7f0163fff700 1 --2- 172.21.15.101:0/1932955647 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fceaf52be0 0x55fceaf62ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:21.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.734+0000 7f016979c700 1 --2- 172.21.15.101:0/1932955647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fceaf5dad0 0x55fceaf63210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:21.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.735+0000 7f0163fff700 1 -- 172.21.15.101:0/1932955647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fceaf522d0 msgr2=0x55fceaf646b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:21.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.735+0000 7f0163fff700 1 --2- 172.21.15.101:0/1932955647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fceaf522d0 0x55fceaf646b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:21.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.735+0000 7f0163fff700 1 -- 172.21.15.101:0/1932955647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fceaf5dad0 msgr2=0x55fceaf63210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:21.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.735+0000 7f0163fff700 1 --2- 172.21.15.101:0/1932955647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fceaf5dad0 0x55fceaf63210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:21.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.735+0000 7f0163fff700 1 -- 172.21.15.101:0/1932955647 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fceacd2de0 con 0x55fceaf52be0 2023-07-18T03:38:21.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.735+0000 7f0168f9b700 1 --2- 172.21.15.101:0/1932955647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fceaf522d0 0x55fceaf646b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:21.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.735+0000 7f016979c700 1 --2- 172.21.15.101:0/1932955647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fceaf5dad0 0x55fceaf63210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:21.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.735+0000 7f0163fff700 1 --2- 172.21.15.101:0/1932955647 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fceaf52be0 0x55fceaf62ad0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f0150099ca0 tx=0x7f01500a24f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2eb53ce23ec6edb server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:21.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.735+0000 7f0143fff700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:38:21.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.735+0000 7f0143fff700 1 -- 172.21.15.101:0/1932955647 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0150099690 con 0x55fceaf52be0 2023-07-18T03:38:21.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.735+0000 7f0143fff700 1 -- 172.21.15.101:0/1932955647 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(142..142 src has 1..142) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f015009a200 con 0x55fceaf52be0 2023-07-18T03:38:21.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.735+0000 7f017eb9cb80 1 -- 172.21.15.101:0/1932955647 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0150010880 con 0x55fceaf52be0 2023-07-18T03:38:21.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.735+0000 7f0143fff700 1 -- 172.21.15.101:0/1932955647 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f015009a3c0 con 0x55fceaf52be0 2023-07-18T03:38:21.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.735+0000 7f017eb9cb80 10 client.19302 fetch_fsmap learned FSMap version 49 2023-07-18T03:38:21.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.735+0000 7f017eb9cb80 1 -- 172.21.15.101:0/1932955647 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f015c009040 con 0x55fceaf52be0 2023-07-18T03:38:21.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.735+0000 7f0143fff700 1 -- 172.21.15.101:0/1932955647 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0150010880 con 0x55fceaf52be0 2023-07-18T03:38:21.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.736+0000 7f0143fff700 1 -- 172.21.15.101:0/1932955647 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 49) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0150094bb0 con 0x55fceaf52be0 2023-07-18T03:38:21.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.736+0000 7f017eb9cb80 10 client.19302 fetch_fsmap finished waiting for FSMap version 49 2023-07-18T03:38:21.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.736+0000 7f017eb9cb80 10 client.19302 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:38:21.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.736+0000 7f017eb9cb80 20 client.19302 populate_metadata read hostname 'smithi101' 2023-07-18T03:38:21.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.736+0000 7f017eb9cb80 1 --2- 172.21.15.101:0/1932955647 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55fceafc28d0 0x55fceafc4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:21.744 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.736+0000 7f017eb9cb80 4 client.19302 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:38:21.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.736+0000 7f017eb9cb80 1 -- 172.21.15.101:0/1932955647 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fceae60050 con 0x55fceafc28d0 2023-07-18T03:38:21.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.736+0000 7f0168f9b700 1 --2- 172.21.15.101:0/1932955647 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55fceafc28d0 0x55fceafc4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:21.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.736+0000 7f0143fff700 10 client.19302 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:38:21.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.736+0000 7f0168f9b700 1 --2- 172.21.15.101:0/1932955647 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55fceafc28d0 0x55fceafc4ce0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53d8d33030c30c5 server_cookie=9299990079b3dd90 in_seq=0 out_seq=0 2023-07-18T03:38:21.745 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.736+0000 7f0143fff700 1 -- 172.21.15.101:0/1932955647 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55fceae60050 con 0x55fceafc28d0 2023-07-18T03:38:21.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.737+0000 7f0143fff700 10 client.19302 handle_command_reply: tid=0 2023-07-18T03:38:21.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.737+0000 7f017eb9cb80 1 client.19302 shutdown 2023-07-18T03:38:21.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.737+0000 7f017eb9cb80 1 -- 172.21.15.101:0/1932955647 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fceaf52be0 msgr2=0x55fceaf62ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:21.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.737+0000 7f017eb9cb80 1 --2- 172.21.15.101:0/1932955647 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fceaf52be0 0x55fceaf62ad0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f0150099ca0 tx=0x7f01500a24f0 comp rx=0 tx=0).stop 2023-07-18T03:38:21.746 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.737+0000 7f017eb9cb80 1 -- 172.21.15.101:0/1932955647 shutdown_connections 2023-07-18T03:38:21.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.737+0000 7f017eb9cb80 1 --2- 172.21.15.101:0/1932955647 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55fceafc28d0 0x55fceafc4ce0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:21.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.737+0000 7f017eb9cb80 1 --2- 172.21.15.101:0/1932955647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fceaf5dad0 0x55fceaf63210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:21.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.737+0000 7f017eb9cb80 1 --2- 172.21.15.101:0/1932955647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fceaf522d0 0x55fceaf646b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:21.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.737+0000 7f0143fff700 0 client.19302 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:38:21.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.737+0000 7f017eb9cb80 1 --2- 172.21.15.101:0/1932955647 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fceaf52be0 0x55fceaf62ad0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:21.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.737+0000 7f017eb9cb80 1 -- 172.21.15.101:0/1932955647 >> 172.21.15.101:0/1932955647 conn(0x55fceaf48da0 msgr2=0x55fceaf543e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:21.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.737+0000 7f017eb9cb80 1 -- 172.21.15.101:0/1932955647 shutdown_connections 2023-07-18T03:38:21.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.737+0000 7f017eb9cb80 1 -- 172.21.15.101:0/1932955647 wait complete. 2023-07-18T03:38:21.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.737+0000 7f017eb9cb80 20 client.19302 trim_cache size 0 max 16384 2023-07-18T03:38:21.749 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:38:21.749 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:38:21.749 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:38:21.749 INFO:teuthology.orchestra.run.smithi101.stdout: "0f2f19e0-0429-465c-935e-223eb09490c3": { 2023-07-18T03:38:21.750 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:38:21.750 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "0f2f19e0-0429-465c-935e-223eb09490c3", 2023-07-18T03:38:21.750 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:38:21.750 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:38:21.750 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:38:21.751 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:38:21.751 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.738+0000 7f016b1ff700 1 -- 172.21.15.101:0/1158344674 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f01540a12c0 msgr2=0x7f01540a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:21.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.738+0000 7f016b1ff700 1 --2- 172.21.15.101:0/1158344674 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f01540a12c0 0x7f01540a3770 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f015c00f880 tx=0x7f015c00fc40 comp rx=0 tx=0).stop 2023-07-18T03:38:21.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.738+0000 7f016b1ff700 1 -- 172.21.15.101:0/1158344674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01641004b0 msgr2=0x7f0164198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:21.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.738+0000 7f016b1ff700 1 --2- 172.21.15.101:0/1158344674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01641004b0 0x7f0164198370 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f015000e9a0 tx=0x7f015000ecb0 comp rx=0 tx=0).stop 2023-07-18T03:38:21.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.738+0000 7f016b1ff700 1 -- 172.21.15.101:0/1158344674 shutdown_connections 2023-07-18T03:38:21.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.738+0000 7f016b1ff700 1 --2- 172.21.15.101:0/1158344674 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f01540a12c0 0x7f01540a3770 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:21.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.738+0000 7f016b1ff700 1 --2- 172.21.15.101:0/1158344674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0164105b10 0x7f016419c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:21.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.738+0000 7f016b1ff700 1 --2- 172.21.15.101:0/1158344674 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f01640ffb00 0x7f0164197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:21.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.738+0000 7f016b1ff700 1 --2- 172.21.15.101:0/1158344674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01641004b0 0x7f0164198370 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:21.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.738+0000 7f016b1ff700 1 -- 172.21.15.101:0/1158344674 >> 172.21.15.101:0/1158344674 conn(0x7f01640fb600 msgr2=0x7f016410ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:21.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.738+0000 7f016b1ff700 1 -- 172.21.15.101:0/1158344674 shutdown_connections 2023-07-18T03:38:21.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:21.738+0000 7f016b1ff700 1 -- 172.21.15.101:0/1158344674 wait complete. 2023-07-18T03:38:21.754 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:21 smithi101 ceph-mon[126379]: pgmap v976: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:38:21.755 INFO:tasks.cephfs.filesystem:scrub status for tag:0f2f19e0-0429-465c-935e-223eb09490c3 - {'path': '/', 'tag': '0f2f19e0-0429-465c-935e-223eb09490c3', 'options': 'recursive,force'} 2023-07-18T03:38:21.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:21 smithi155 ceph-mon[128885]: pgmap v976: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:38:23.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:23 smithi179 ceph-mon[129229]: pgmap v977: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:38:23.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:23 smithi101 ceph-mon[126379]: pgmap v977: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:38:23.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:23 smithi155 ceph-mon[128885]: pgmap v977: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:38:25.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:25 smithi179 ceph-mon[129229]: pgmap v978: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:38:25.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:25 smithi101 ceph-mon[126379]: pgmap v978: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:38:25.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:25 smithi155 ceph-mon[128885]: pgmap v978: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-07-18T03:38:27.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:27 smithi179 ceph-mon[129229]: pgmap v979: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 87 B/s wr, 7 op/s 2023-07-18T03:38:27.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:27 smithi101 ceph-mon[126379]: pgmap v979: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 87 B/s wr, 7 op/s 2023-07-18T03:38:27.796 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:27 smithi155 ceph-mon[128885]: pgmap v979: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 87 B/s wr, 7 op/s 2023-07-18T03:38:29.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:29 smithi101 ceph-mon[126379]: pgmap v980: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:38:29.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:29 smithi155 ceph-mon[128885]: pgmap v980: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:38:29.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:29 smithi179 ceph-mon[129229]: pgmap v980: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:38:31.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:31 smithi101 ceph-mon[126379]: pgmap v981: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:38:31.796 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:31 smithi155 ceph-mon[128885]: pgmap v981: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:38:31.951 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:31 smithi179 ceph-mon[129229]: pgmap v981: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:38:33.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:33 smithi101 ceph-mon[126379]: pgmap v982: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:38:33.796 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:33 smithi155 ceph-mon[128885]: pgmap v982: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:38:33.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:33 smithi179 ceph-mon[129229]: pgmap v982: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:38:35.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:35 smithi101 ceph-mon[126379]: pgmap v983: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:38:35.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:35 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:38:35.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:35 smithi155 ceph-mon[128885]: pgmap v983: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:38:35.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:35 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:38:35.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:35 smithi179 ceph-mon[129229]: pgmap v983: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:38:35.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:35 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:38:37.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:37 smithi101 ceph-mon[126379]: pgmap v984: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:38:37.951 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:37 smithi179 ceph-mon[129229]: pgmap v984: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:38:38.044 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:37 smithi155 ceph-mon[128885]: pgmap v984: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:38:39.701 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:39 smithi179 ceph-mon[129229]: pgmap v985: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:38:39.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:39 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:38:39.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:39 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:38:39.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:39 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:38:39.703 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:39 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:38:39.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:39 smithi101 ceph-mon[126379]: pgmap v985: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:38:39.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:39 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:38:39.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:39 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:38:39.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:39 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:38:39.741 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:39 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:38:39.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:39 smithi155 ceph-mon[128885]: pgmap v985: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:38:39.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:39 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:38:39.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:39 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:38:39.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:39 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:38:39.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:39 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:38:41.047 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:40 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:38:41.048 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:40 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:38:41.048 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:40 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:41.048 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:40 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:41.049 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:40 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:41.049 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:40 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:41.049 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:40 smithi155 ceph-mon[128885]: Adjusting osd_memory_target on smithi179 to 196.2M 2023-07-18T03:38:41.049 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:40 smithi155 ceph-mon[128885]: Unable to set osd_memory_target on smithi179 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-07-18T03:38:41.049 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:40 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:38:41.050 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:40 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:38:41.050 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:40 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:38:41.050 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:40 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:38:41.050 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:40 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:41.051 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:40 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:41.051 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:40 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:41.051 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:40 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:41.051 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:40 smithi155 ceph-mon[128885]: Adjusting osd_memory_target on smithi101 to 220.9M 2023-07-18T03:38:41.051 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:40 smithi155 ceph-mon[128885]: Unable to set osd_memory_target on smithi101 to 231653990: error parsing value: Value '231653990' is below minimum 939524096 2023-07-18T03:38:41.052 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:40 smithi155 ceph-mon[128885]: pgmap v986: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:38:41.204 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:40 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:38:41.205 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:40 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:38:41.205 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:40 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:41.205 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:40 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:41.205 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:40 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:41.206 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:40 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:41.206 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:40 smithi179 ceph-mon[129229]: Adjusting osd_memory_target on smithi179 to 196.2M 2023-07-18T03:38:41.206 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:40 smithi179 ceph-mon[129229]: Unable to set osd_memory_target on smithi179 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-07-18T03:38:41.206 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:40 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:38:41.206 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:40 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:38:41.207 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:40 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:38:41.207 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:40 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:38:41.207 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:40 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:41.207 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:40 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:41.207 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:40 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:41.208 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:40 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:41.208 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:40 smithi179 ceph-mon[129229]: Adjusting osd_memory_target on smithi101 to 220.9M 2023-07-18T03:38:41.208 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:40 smithi179 ceph-mon[129229]: Unable to set osd_memory_target on smithi101 to 231653990: error parsing value: Value '231653990' is below minimum 939524096 2023-07-18T03:38:41.208 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:40 smithi179 ceph-mon[129229]: pgmap v986: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:38:41.239 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:40 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:38:41.240 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:40 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:38:41.240 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:40 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:41.240 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:40 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:41.241 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:40 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:41.241 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:40 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:41.241 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:40 smithi101 ceph-mon[126379]: Adjusting osd_memory_target on smithi179 to 196.2M 2023-07-18T03:38:41.241 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:40 smithi101 ceph-mon[126379]: Unable to set osd_memory_target on smithi179 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-07-18T03:38:41.242 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:40 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:38:41.242 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:40 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:38:41.243 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:40 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:38:41.243 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:40 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:38:41.243 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:40 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:41.243 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:40 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:41.243 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:40 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:41.244 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:40 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:41.244 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:40 smithi101 ceph-mon[126379]: Adjusting osd_memory_target on smithi101 to 220.9M 2023-07-18T03:38:41.244 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:40 smithi101 ceph-mon[126379]: Unable to set osd_memory_target on smithi101 to 231653990: error parsing value: Value '231653990' is below minimum 939524096 2023-07-18T03:38:41.244 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:40 smithi101 ceph-mon[126379]: pgmap v986: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:38:42.202 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:41 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:38:42.202 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:41 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:38:42.203 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:41 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:42.203 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:41 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:42.203 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:41 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:42.203 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:41 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:42.203 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:41 smithi179 ceph-mon[129229]: Adjusting osd_memory_target on smithi155 to 196.2M 2023-07-18T03:38:42.204 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:41 smithi179 ceph-mon[129229]: Unable to set osd_memory_target on smithi155 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-07-18T03:38:42.204 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:41 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:38:42.204 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:41 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:38:42.204 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:41 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:38:42.239 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:41 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:38:42.240 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:41 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:38:42.240 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:41 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:42.240 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:41 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:42.240 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:41 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:42.241 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:41 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:42.241 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:41 smithi101 ceph-mon[126379]: Adjusting osd_memory_target on smithi155 to 196.2M 2023-07-18T03:38:42.241 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:41 smithi101 ceph-mon[126379]: Unable to set osd_memory_target on smithi155 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-07-18T03:38:42.241 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:41 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:38:42.241 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:41 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:38:42.242 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:41 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:38:42.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:41 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:38:42.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:41 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:38:42.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:41 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:42.297 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:41 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:42.298 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:41 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:42.298 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:41 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:42.298 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:41 smithi155 ceph-mon[128885]: Adjusting osd_memory_target on smithi155 to 196.2M 2023-07-18T03:38:42.298 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:41 smithi155 ceph-mon[128885]: Unable to set osd_memory_target on smithi155 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-07-18T03:38:42.298 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:41 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:38:42.299 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:41 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:38:42.299 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:41 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:38:43.201 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:42 smithi179 ceph-mon[129229]: pgmap v987: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:38:43.239 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:42 smithi101 ceph-mon[126379]: pgmap v987: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:38:43.296 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:42 smithi155 ceph-mon[128885]: pgmap v987: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:38:45.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:45 smithi179 ceph-mon[129229]: pgmap v988: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:38:45.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:45 smithi101 ceph-mon[126379]: pgmap v988: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:38:45.796 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:45 smithi155 ceph-mon[128885]: pgmap v988: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:38:47.701 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:47 smithi179 ceph-mon[129229]: pgmap v989: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:38:47.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:47 smithi101 ceph-mon[126379]: pgmap v989: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:38:47.796 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:47 smithi155 ceph-mon[128885]: pgmap v989: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:38:49.701 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:49 smithi179 ceph-mon[129229]: pgmap v990: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:38:49.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:49 smithi101 ceph-mon[126379]: pgmap v990: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:38:49.796 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:49 smithi155 ceph-mon[128885]: pgmap v990: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:38:51.701 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:51 smithi179 ceph-mon[129229]: pgmap v991: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:38:51.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:51 smithi101 ceph-mon[126379]: pgmap v991: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:38:51.756 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:38:51.796 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:51 smithi155 ceph-mon[128885]: pgmap v991: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:38:51.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.978+0000 7f1d08fff700 1 -- 172.21.15.101:0/1828835201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d04101470 msgr2=0x7f1d04109b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:51.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.978+0000 7f1d08fff700 1 --2- 172.21.15.101:0/1828835201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d04101470 0x7f1d04109b40 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f1cf80098f0 tx=0x7f1cf8009c00 comp rx=0 tx=0).stop 2023-07-18T03:38:51.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.978+0000 7f1d08fff700 1 -- 172.21.15.101:0/1828835201 shutdown_connections 2023-07-18T03:38:51.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.978+0000 7f1d08fff700 1 --2- 172.21.15.101:0/1828835201 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1d0410a080 0x7f1d0410c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:51.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.978+0000 7f1d08fff700 1 --2- 172.21.15.101:0/1828835201 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d04100b60 0x7f1d04100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:51.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.978+0000 7f1d08fff700 1 --2- 172.21.15.101:0/1828835201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d04101470 0x7f1d04109b40 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:51.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.978+0000 7f1d08fff700 1 -- 172.21.15.101:0/1828835201 >> 172.21.15.101:0/1828835201 conn(0x7f1d040fa4b0 msgr2=0x7f1d040fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:51.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.978+0000 7f1d08fff700 1 -- 172.21.15.101:0/1828835201 shutdown_connections 2023-07-18T03:38:51.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.978+0000 7f1d08fff700 1 -- 172.21.15.101:0/1828835201 wait complete. 2023-07-18T03:38:51.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.979+0000 7f1d08fff700 1 Processor -- start 2023-07-18T03:38:51.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.979+0000 7f1d08fff700 1 -- start start 2023-07-18T03:38:51.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.980+0000 7f1d08fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1d04100b60 0x7f1d04197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:51.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.980+0000 7f1d08fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d04101470 0x7f1d04198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:51.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.980+0000 7f1d08fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d0410a080 0x7f1d0419c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:51.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.980+0000 7f1d08fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0419cde0 con 0x7f1d04100b60 2023-07-18T03:38:51.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.980+0000 7f1d08fff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0419cf50 con 0x7f1d04101470 2023-07-18T03:38:51.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.980+0000 7f1d08fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0419d110 con 0x7f1d0410a080 2023-07-18T03:38:51.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.980+0000 7f1d0259c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1d04100b60 0x7f1d04197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:51.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.980+0000 7f1d01d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d04101470 0x7f1d04198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:51.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.980+0000 7f1d0259c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1d04100b60 0x7f1d04197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:51070/0 (socket says 172.21.15.101:51070) 2023-07-18T03:38:51.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.980+0000 7f1d02d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d0410a080 0x7f1d0419c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:51.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.980+0000 7f1d0259c700 1 -- 172.21.15.101:0/1631210163 learned_addr learned my addr 172.21.15.101:0/1631210163 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:38:51.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.980+0000 7f1d01d9b700 1 -- 172.21.15.101:0/1631210163 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d0410a080 msgr2=0x7f1d0419c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:51.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.980+0000 7f1d01d9b700 1 --2- 172.21.15.101:0/1631210163 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d0410a080 0x7f1d0419c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:51.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.980+0000 7f1d01d9b700 1 -- 172.21.15.101:0/1631210163 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1d04100b60 msgr2=0x7f1d04197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:51.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.980+0000 7f1d01d9b700 1 --2- 172.21.15.101:0/1631210163 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1d04100b60 0x7f1d04197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:51.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.980+0000 7f1d01d9b700 1 -- 172.21.15.101:0/1631210163 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf40095f0 con 0x7f1d04101470 2023-07-18T03:38:51.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.980+0000 7f1d0259c700 1 --2- 172.21.15.101:0/1631210163 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1d04100b60 0x7f1d04197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:51.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.981+0000 7f1d01d9b700 1 --2- 172.21.15.101:0/1631210163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d04101470 0x7f1d04198340 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8009810 tx=0x7f1cf800fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:51.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.981+0000 7f1cf37fe700 1 -- 172.21.15.101:0/1631210163 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf801c070 con 0x7f1d04101470 2023-07-18T03:38:51.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.981+0000 7f1d08fff700 1 -- 172.21.15.101:0/1631210163 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cf8009520 con 0x7f1d04101470 2023-07-18T03:38:51.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.981+0000 7f1cf37fe700 1 -- 172.21.15.101:0/1631210163 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1cf800bb20 con 0x7f1d04101470 2023-07-18T03:38:51.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.981+0000 7f1cf37fe700 1 -- 172.21.15.101:0/1631210163 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf80038e0 con 0x7f1d04101470 2023-07-18T03:38:51.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.981+0000 7f1d08fff700 1 -- 172.21.15.101:0/1631210163 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d041a4a50 con 0x7f1d04101470 2023-07-18T03:38:51.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.985+0000 7f1cf37fe700 1 -- 172.21.15.101:0/1631210163 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f1cf8020030 con 0x7f1d04101470 2023-07-18T03:38:51.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.986+0000 7f1d1c967b80 1 Processor -- start 2023-07-18T03:38:51.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.987+0000 7f1d1c967b80 1 -- start start 2023-07-18T03:38:51.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.987+0000 7f1d1c967b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55786f3fb2d0 0x55786f3fb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:51.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.987+0000 7f1d1c967b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55786f3fbbe0 0x55786f406590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:51.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.987+0000 7f1d1c967b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55786f406ad0 0x55786f408eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:51.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.987+0000 7f1d1c967b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55786f301cb0 con 0x55786f406ad0 2023-07-18T03:38:51.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.987+0000 7f1d1c967b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55786f354a40 con 0x55786f3fbbe0 2023-07-18T03:38:51.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.987+0000 7f1d1c967b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55786f34ded0 con 0x55786f3fb2d0 2023-07-18T03:38:51.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.987+0000 7f1d02d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55786f3fb2d0 0x55786f3fb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:51.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.987+0000 7f1cf37fe700 1 --2- 172.21.15.101:0/1631210163 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f1cec0a12c0 0x7f1cec0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:51.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.987+0000 7f1d0259c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55786f3fbbe0 0x55786f406590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:51.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.987+0000 7f1d02d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55786f3fb2d0 0x55786f3fb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:40102/0 (socket says 172.21.15.101:40102) 2023-07-18T03:38:51.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.987+0000 7f1d02d9d700 1 -- 172.21.15.101:0/2710389506 learned_addr learned my addr 172.21.15.101:0/2710389506 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:38:51.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.987+0000 7f1d01d9b700 1 --2- 172.21.15.101:0/2710389506 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55786f406ad0 0x55786f408eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:51.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.987+0000 7f1cf37fe700 1 -- 172.21.15.101:0/1631210163 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f1cf80cdfb0 con 0x7f1d04101470 2023-07-18T03:38:51.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.987+0000 7f1d0259c700 1 -- 172.21.15.101:0/2710389506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55786f3fb2d0 msgr2=0x55786f3fb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:51.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.987+0000 7f1d0259c700 1 --2- 172.21.15.101:0/2710389506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55786f3fb2d0 0x55786f3fb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:51.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.987+0000 7f1d02d9d700 1 --2- 172.21.15.101:0/1631210163 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f1cec0a12c0 0x7f1cec0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:51.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.987+0000 7f1d0259c700 1 -- 172.21.15.101:0/2710389506 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55786f406ad0 msgr2=0x55786f408eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:51.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.987+0000 7f1d0259c700 1 --2- 172.21.15.101:0/2710389506 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55786f406ad0 0x55786f408eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:51.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.987+0000 7f1d0259c700 1 -- 172.21.15.101:0/2710389506 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55786f17bde0 con 0x55786f3fbbe0 2023-07-18T03:38:51.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.988+0000 7f1d01d9b700 1 --2- 172.21.15.101:0/2710389506 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55786f406ad0 0x55786f408eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:38:51.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.988+0000 7f1d0259c700 1 --2- 172.21.15.101:0/2710389506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55786f3fbbe0 0x55786f406590 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4005d10 tx=0x7f1cf4011ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61f56b19e6c0ae98 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:51.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.988+0000 7f1cf0ff9700 1 -- 172.21.15.101:0/2710389506 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf4004830 con 0x55786f3fbbe0 2023-07-18T03:38:51.988 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.988+0000 7f1cf0ff9700 1 -- 172.21.15.101:0/2710389506 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1cf4004e50 con 0x55786f3fbbe0 2023-07-18T03:38:51.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.988+0000 7f1cf0ff9700 1 -- 172.21.15.101:0/2710389506 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf400f830 con 0x55786f3fbbe0 2023-07-18T03:38:51.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.989+0000 7f1d1c967b80 1 -- 172.21.15.101:0/2710389506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55786f3fbbe0 msgr2=0x55786f406590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:51.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.989+0000 7f1d02d9d700 1 --2- 172.21.15.101:0/1631210163 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f1cec0a12c0 0x7f1cec0a3770 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc00fb80 tx=0x7f1cfc004b40 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:51.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.989+0000 7f1d1c967b80 1 --2- 172.21.15.101:0/2710389506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55786f3fbbe0 0x55786f406590 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4005d10 tx=0x7f1cf4011ae0 comp rx=0 tx=0).stop 2023-07-18T03:38:51.992 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.989+0000 7f1d1c967b80 1 -- 172.21.15.101:0/2710389506 shutdown_connections 2023-07-18T03:38:51.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.989+0000 7f1d1c967b80 1 --2- 172.21.15.101:0/2710389506 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55786f406ad0 0x55786f408eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:51.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.989+0000 7f1d1c967b80 1 --2- 172.21.15.101:0/2710389506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55786f3fb2d0 0x55786f3fb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:51.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.989+0000 7f1d1c967b80 1 --2- 172.21.15.101:0/2710389506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55786f3fbbe0 0x55786f406590 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:51.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.989+0000 7f1d1c967b80 1 -- 172.21.15.101:0/2710389506 >> 172.21.15.101:0/2710389506 conn(0x55786f3f1da0 msgr2=0x55786f3f76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:51.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.989+0000 7f1d1c967b80 1 -- 172.21.15.101:0/2710389506 shutdown_connections 2023-07-18T03:38:51.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.989+0000 7f1d1c967b80 1 -- 172.21.15.101:0/2710389506 wait complete. 2023-07-18T03:38:51.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.989+0000 7f1d1c967b80 1 -- start start 2023-07-18T03:38:51.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.990+0000 7f1d1c967b80 1 Processor -- start 2023-07-18T03:38:51.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.990+0000 7f1d1c967b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55786f3fb2d0 0x55786f406550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:51.994 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.990+0000 7f1d1c967b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55786f3fbbe0 0x55786f403760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:51.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.990+0000 7f1d1c967b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55786f406ad0 0x55786f403e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:51.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.990+0000 7f1d1c967b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55786f2edac0 con 0x55786f406ad0 2023-07-18T03:38:51.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.990+0000 7f1d1c967b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55786f2ed7c0 con 0x55786f3fb2d0 2023-07-18T03:38:51.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.990+0000 7f1d1c967b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55786f1d1920 con 0x55786f3fbbe0 2023-07-18T03:38:51.995 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.990+0000 7f1d1c967b80 10 client.0 _ll_register_callbacks cb 0x55786f2eb760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:38:51.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.990+0000 7f1d0259c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55786f3fb2d0 0x55786f406550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:51.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.990+0000 7f1d01d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55786f3fbbe0 0x55786f403760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:51.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.990+0000 7f1d0259c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55786f3fb2d0 0x55786f406550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:52040/0 (socket says 172.21.15.101:52040) 2023-07-18T03:38:51.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.990+0000 7f1d0259c700 1 -- 172.21.15.101:0/905239546 learned_addr learned my addr 172.21.15.101:0/905239546 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:38:51.996 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.990+0000 7f1d02d9d700 1 --2- 172.21.15.101:0/905239546 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55786f406ad0 0x55786f403e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:51.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.991+0000 7f1d01d9b700 1 -- 172.21.15.101:0/905239546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55786f3fb2d0 msgr2=0x55786f406550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:51.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.991+0000 7f1d01d9b700 1 --2- 172.21.15.101:0/905239546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55786f3fb2d0 0x55786f406550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:51.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.991+0000 7f1d01d9b700 1 -- 172.21.15.101:0/905239546 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55786f406ad0 msgr2=0x55786f403e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:51.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.991+0000 7f1d01d9b700 1 --2- 172.21.15.101:0/905239546 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55786f406ad0 0x55786f403e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:51.997 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.991+0000 7f1d01d9b700 1 -- 172.21.15.101:0/905239546 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cf4011460 con 0x55786f3fbbe0 2023-07-18T03:38:51.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.991+0000 7f1d0259c700 1 --2- 172.21.15.101:0/905239546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55786f3fb2d0 0x55786f406550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:51.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.991+0000 7f1d01d9b700 1 --2- 172.21.15.101:0/905239546 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55786f3fbbe0 0x55786f403760 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f1cf80cb2f0 tx=0x7f1cf800efe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9ca13279c3ad3ec server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:51.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.991+0000 7f1ce9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:38:51.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.991+0000 7f1ce9ffb700 1 -- 172.21.15.101:0/905239546 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf8005340 con 0x55786f3fbbe0 2023-07-18T03:38:51.998 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.991+0000 7f1d1c967b80 1 -- 172.21.15.101:0/905239546 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55786f2a74f0 con 0x55786f3fbbe0 2023-07-18T03:38:51.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.991+0000 7f1ce9ffb700 1 -- 172.21.15.101:0/905239546 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(142..142 src has 1..142) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f1cf809ed30 con 0x55786f3fbbe0 2023-07-18T03:38:51.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.991+0000 7f1ce9ffb700 1 -- 172.21.15.101:0/905239546 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1cf80a30c0 con 0x55786f3fbbe0 2023-07-18T03:38:51.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.991+0000 7f1d1c967b80 10 client.29416 fetch_fsmap learned FSMap version 49 2023-07-18T03:38:51.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.991+0000 7f1d1c967b80 1 -- 172.21.15.101:0/905239546 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55786f164c60 con 0x55786f3fbbe0 2023-07-18T03:38:51.999 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.992+0000 7f1ce9ffb700 1 -- 172.21.15.101:0/905239546 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55786f2a74f0 con 0x55786f3fbbe0 2023-07-18T03:38:52.000 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.992+0000 7f1ce9ffb700 1 -- 172.21.15.101:0/905239546 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 49) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1cf809af60 con 0x55786f3fbbe0 2023-07-18T03:38:52.000 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.992+0000 7f1d1c967b80 10 client.29416 fetch_fsmap finished waiting for FSMap version 49 2023-07-18T03:38:52.000 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.992+0000 7f1d1c967b80 10 client.29416 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:38:52.000 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.992+0000 7f1d1c967b80 20 client.29416 populate_metadata read hostname 'smithi101' 2023-07-18T03:38:52.000 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.992+0000 7f1d1c967b80 1 --2- 172.21.15.101:0/905239546 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55786f415e40 0x55786f418220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:52.001 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.992+0000 7f1d1c967b80 4 client.29416 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:38:52.001 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.992+0000 7f1d1c967b80 1 -- 172.21.15.101:0/905239546 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55786f346eb0 con 0x55786f415e40 2023-07-18T03:38:52.001 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.992+0000 7f1d0259c700 1 --2- 172.21.15.101:0/905239546 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55786f415e40 0x55786f418220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:52.001 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.993+0000 7f1ce9ffb700 10 client.29416 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:38:52.001 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.993+0000 7f1d0259c700 1 --2- 172.21.15.101:0/905239546 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55786f415e40 0x55786f418220 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d0c743ae5158228 server_cookie=7a2f4b71854ab62 in_seq=0 out_seq=0 2023-07-18T03:38:52.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.993+0000 7f1ce9ffb700 1 -- 172.21.15.101:0/905239546 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55786f346eb0 con 0x55786f415e40 2023-07-18T03:38:52.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.993+0000 7f1ce9ffb700 10 client.29416 handle_command_reply: tid=0 2023-07-18T03:38:52.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.993+0000 7f1d1c967b80 1 client.29416 shutdown 2023-07-18T03:38:52.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.993+0000 7f1d1c967b80 1 -- 172.21.15.101:0/905239546 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55786f3fbbe0 msgr2=0x55786f403760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:52.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.993+0000 7f1d1c967b80 1 --2- 172.21.15.101:0/905239546 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55786f3fbbe0 0x55786f403760 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f1cf80cb2f0 tx=0x7f1cf800efe0 comp rx=0 tx=0).stop 2023-07-18T03:38:52.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.993+0000 7f1d1c967b80 1 -- 172.21.15.101:0/905239546 shutdown_connections 2023-07-18T03:38:52.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.993+0000 7f1d1c967b80 1 --2- 172.21.15.101:0/905239546 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55786f415e40 0x55786f418220 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:52.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.994+0000 7f1d1c967b80 1 --2- 172.21.15.101:0/905239546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55786f3fb2d0 0x55786f406550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:52.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.994+0000 7f1d1c967b80 1 --2- 172.21.15.101:0/905239546 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55786f406ad0 0x55786f403e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:52.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.994+0000 7f1d1c967b80 1 --2- 172.21.15.101:0/905239546 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55786f3fbbe0 0x55786f403760 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:52.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.994+0000 7f1ce9ffb700 0 client.29416 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:38:52.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.994+0000 7f1d1c967b80 1 -- 172.21.15.101:0/905239546 >> 172.21.15.101:0/905239546 conn(0x55786f3f1da0 msgr2=0x55786f3f7290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:52.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.994+0000 7f1d1c967b80 1 -- 172.21.15.101:0/905239546 shutdown_connections 2023-07-18T03:38:52.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.994+0000 7f1d1c967b80 1 -- 172.21.15.101:0/905239546 wait complete. 2023-07-18T03:38:52.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:51.994+0000 7f1d1c967b80 20 client.29416 trim_cache size 0 max 16384 2023-07-18T03:38:52.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.011+0000 7f1d1c967b80 1 Processor -- start 2023-07-18T03:38:52.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.011+0000 7f1d1c967b80 1 -- start start 2023-07-18T03:38:52.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.012+0000 7f1d1c967b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55786f3fb2d0 0x55786f403d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:52.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.012+0000 7f1d1c967b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55786f3fbbe0 0x55786f3f3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:52.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.012+0000 7f1d1c967b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55786f406ad0 0x55786f41cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:52.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.012+0000 7f1d1c967b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55786f3fce40 con 0x55786f406ad0 2023-07-18T03:38:52.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.012+0000 7f1d1c967b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55786f3f3850 con 0x55786f3fb2d0 2023-07-18T03:38:52.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.012+0000 7f1d1c967b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55786f41cff0 con 0x55786f3fbbe0 2023-07-18T03:38:52.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.012+0000 7f1d01d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55786f3fbbe0 0x55786f3f3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:52.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.012+0000 7f1d0259c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55786f3fb2d0 0x55786f403d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:52.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.012+0000 7f1d02d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55786f406ad0 0x55786f41cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:52.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.012+0000 7f1d01d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55786f3fbbe0 0x55786f3f3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:40116/0 (socket says 172.21.15.101:40116) 2023-07-18T03:38:52.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.012+0000 7f1d01d9b700 1 -- 172.21.15.101:0/3333323342 learned_addr learned my addr 172.21.15.101:0/3333323342 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:38:52.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.012+0000 7f1d0259c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55786f3fb2d0 0x55786f403d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:52056/0 (socket says 172.21.15.101:52056) 2023-07-18T03:38:52.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.012+0000 7f1d01d9b700 1 -- 172.21.15.101:0/3333323342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55786f3fb2d0 msgr2=0x55786f403d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:52.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.012+0000 7f1d01d9b700 1 --2- 172.21.15.101:0/3333323342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55786f3fb2d0 0x55786f403d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:52.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.012+0000 7f1d01d9b700 1 -- 172.21.15.101:0/3333323342 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55786f406ad0 msgr2=0x55786f41cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:52.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.012+0000 7f1d01d9b700 1 --2- 172.21.15.101:0/3333323342 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55786f406ad0 0x55786f41cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:52.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.012+0000 7f1d01d9b700 1 -- 172.21.15.101:0/3333323342 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55786f1a2250 con 0x55786f3fbbe0 2023-07-18T03:38:52.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.012+0000 7f1d0259c700 1 --2- 172.21.15.101:0/3333323342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55786f3fb2d0 0x55786f403d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:52.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.012+0000 7f1d01d9b700 1 --2- 172.21.15.101:0/3333323342 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55786f3fbbe0 0x55786f3f3310 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8097770 tx=0x7f1cf8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5df64f26b3b53d41 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:52.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.013+0000 7f1cf17fa700 1 -- 172.21.15.101:0/3333323342 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf809ed30 con 0x55786f3fbbe0 2023-07-18T03:38:52.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.013+0000 7f1cf17fa700 1 -- 172.21.15.101:0/3333323342 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1cf80a30c0 con 0x55786f3fbbe0 2023-07-18T03:38:52.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.013+0000 7f1cf17fa700 1 -- 172.21.15.101:0/3333323342 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf80ba310 con 0x55786f3fbbe0 2023-07-18T03:38:52.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.013+0000 7f1d1c967b80 1 -- 172.21.15.101:0/3333323342 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55786f3fbbe0 msgr2=0x55786f3f3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:52.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.013+0000 7f1d1c967b80 1 --2- 172.21.15.101:0/3333323342 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55786f3fbbe0 0x55786f3f3310 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8097770 tx=0x7f1cf8009c00 comp rx=0 tx=0).stop 2023-07-18T03:38:52.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.013+0000 7f1d1c967b80 1 -- 172.21.15.101:0/3333323342 shutdown_connections 2023-07-18T03:38:52.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.013+0000 7f1d1c967b80 1 --2- 172.21.15.101:0/3333323342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55786f3fb2d0 0x55786f403d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:52.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.013+0000 7f1d1c967b80 1 --2- 172.21.15.101:0/3333323342 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55786f406ad0 0x55786f41cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:52.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.013+0000 7f1d1c967b80 1 --2- 172.21.15.101:0/3333323342 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55786f3fbbe0 0x55786f3f3310 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:52.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.013+0000 7f1d1c967b80 1 -- 172.21.15.101:0/3333323342 >> 172.21.15.101:0/3333323342 conn(0x55786f3f1da0 msgr2=0x55786f3f5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:52.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.013+0000 7f1d1c967b80 1 -- 172.21.15.101:0/3333323342 shutdown_connections 2023-07-18T03:38:52.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.013+0000 7f1d1c967b80 1 -- 172.21.15.101:0/3333323342 wait complete. 2023-07-18T03:38:52.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.014+0000 7f1d1c967b80 1 -- start start 2023-07-18T03:38:52.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.014+0000 7f1d1c967b80 1 Processor -- start 2023-07-18T03:38:52.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.014+0000 7f1d1c967b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55786f3fb2d0 0x55786f400b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:52.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.014+0000 7f1d1c967b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55786f3fbbe0 0x55786f401080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:52.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.014+0000 7f1d1c967b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55786f406ad0 0x55786f3ff9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:52.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.014+0000 7f1d1c967b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55786f401780 con 0x55786f3fb2d0 2023-07-18T03:38:52.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.014+0000 7f1d1c967b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55786f3fff20 con 0x55786f3fbbe0 2023-07-18T03:38:52.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.014+0000 7f1d1c967b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55786f4000e0 con 0x55786f406ad0 2023-07-18T03:38:52.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.014+0000 7f1d1c967b80 10 client.0 _ll_register_callbacks cb 0x55786f2eb760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:38:52.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.015+0000 7f1d0259c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55786f3fb2d0 0x55786f400b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:52.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.015+0000 7f1d01d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55786f3fbbe0 0x55786f401080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:52.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.015+0000 7f1d02d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55786f406ad0 0x55786f3ff9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:52.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.015+0000 7f1d01d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55786f3fbbe0 0x55786f401080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:52070/0 (socket says 172.21.15.101:52070) 2023-07-18T03:38:52.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.015+0000 7f1d02d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55786f406ad0 0x55786f3ff9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:40130/0 (socket says 172.21.15.101:40130) 2023-07-18T03:38:52.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.015+0000 7f1d0259c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55786f3fb2d0 0x55786f400b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:51116/0 (socket says 172.21.15.101:51116) 2023-07-18T03:38:52.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.015+0000 7f1d01d9b700 1 -- 172.21.15.101:0/2269249073 learned_addr learned my addr 172.21.15.101:0/2269249073 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:38:52.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.015+0000 7f1d0259c700 1 -- 172.21.15.101:0/2269249073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55786f406ad0 msgr2=0x55786f3ff9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:52.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.015+0000 7f1d0259c700 1 --2- 172.21.15.101:0/2269249073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55786f406ad0 0x55786f3ff9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:52.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.015+0000 7f1d0259c700 1 -- 172.21.15.101:0/2269249073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55786f3fbbe0 msgr2=0x55786f401080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:52.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.015+0000 7f1d0259c700 1 --2- 172.21.15.101:0/2269249073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55786f3fbbe0 0x55786f401080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:52.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.015+0000 7f1d0259c700 1 -- 172.21.15.101:0/2269249073 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55786f17bde0 con 0x55786f3fb2d0 2023-07-18T03:38:52.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.015+0000 7f1d01d9b700 1 --2- 172.21.15.101:0/2269249073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55786f3fbbe0 0x55786f401080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:52.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.015+0000 7f1d0259c700 1 --2- 172.21.15.101:0/2269249073 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55786f3fb2d0 0x55786f400b40 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f1cf40056e0 tx=0x7f1cf40052d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d0dc297f08f6dfc server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:52.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.015+0000 7f1ce9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:38:52.021 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.015+0000 7f1ce9ffb700 1 -- 172.21.15.101:0/2269249073 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf4004ae0 con 0x55786f3fb2d0 2023-07-18T03:38:52.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.016+0000 7f1d1c967b80 1 -- 172.21.15.101:0/2269249073 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1cf4004e50 con 0x55786f3fb2d0 2023-07-18T03:38:52.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.016+0000 7f1ce9ffb700 1 -- 172.21.15.101:0/2269249073 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(142..142 src has 1..142) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f1cf4004c40 con 0x55786f3fb2d0 2023-07-18T03:38:52.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.016+0000 7f1ce9ffb700 1 -- 172.21.15.101:0/2269249073 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1cf40291b0 con 0x55786f3fb2d0 2023-07-18T03:38:52.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.016+0000 7f1d1c967b80 10 client.19320 fetch_fsmap learned FSMap version 49 2023-07-18T03:38:52.022 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.016+0000 7f1d1c967b80 1 -- 172.21.15.101:0/2269249073 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f1cf4011460 con 0x55786f3fb2d0 2023-07-18T03:38:52.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.016+0000 7f1ce9ffb700 1 -- 172.21.15.101:0/2269249073 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1cf4004e50 con 0x55786f3fb2d0 2023-07-18T03:38:52.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.016+0000 7f1ce9ffb700 1 -- 172.21.15.101:0/2269249073 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 49) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1cf4011800 con 0x55786f3fb2d0 2023-07-18T03:38:52.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.016+0000 7f1d1c967b80 10 client.19320 fetch_fsmap finished waiting for FSMap version 49 2023-07-18T03:38:52.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.016+0000 7f1d1c967b80 10 client.19320 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:38:52.023 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.016+0000 7f1d1c967b80 20 client.19320 populate_metadata read hostname 'smithi101' 2023-07-18T03:38:52.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.016+0000 7f1d1c967b80 1 --2- 172.21.15.101:0/2269249073 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55786f46a8a0 0x55786f46ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:52.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.016+0000 7f1d1c967b80 4 client.19320 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:38:52.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.016+0000 7f1d1c967b80 1 -- 172.21.15.101:0/2269249073 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55786f309050 con 0x55786f46a8a0 2023-07-18T03:38:52.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.017+0000 7f1d01d9b700 1 --2- 172.21.15.101:0/2269249073 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55786f46a8a0 0x55786f46ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:52.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.017+0000 7f1d01d9b700 1 --2- 172.21.15.101:0/2269249073 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55786f46a8a0 0x55786f46ccb0 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=135fab7e8e69a397 server_cookie=ba7e665916ec2edc in_seq=0 out_seq=0 2023-07-18T03:38:52.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.017+0000 7f1ce9ffb700 10 client.19320 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:38:52.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.017+0000 7f1ce9ffb700 1 -- 172.21.15.101:0/2269249073 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55786f309050 con 0x55786f46a8a0 2023-07-18T03:38:52.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.017+0000 7f1ce9ffb700 10 client.19320 handle_command_reply: tid=0 2023-07-18T03:38:52.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.017+0000 7f1d1c967b80 1 client.19320 shutdown 2023-07-18T03:38:52.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.017+0000 7f1d1c967b80 1 -- 172.21.15.101:0/2269249073 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55786f3fb2d0 msgr2=0x55786f400b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:52.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.017+0000 7f1d1c967b80 1 --2- 172.21.15.101:0/2269249073 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55786f3fb2d0 0x55786f400b40 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f1cf40056e0 tx=0x7f1cf40052d0 comp rx=0 tx=0).stop 2023-07-18T03:38:52.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.017+0000 7f1d1c967b80 1 -- 172.21.15.101:0/2269249073 shutdown_connections 2023-07-18T03:38:52.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.017+0000 7f1d1c967b80 1 --2- 172.21.15.101:0/2269249073 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55786f46a8a0 0x55786f46ccb0 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:52.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.017+0000 7f1d1c967b80 1 --2- 172.21.15.101:0/2269249073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55786f406ad0 0x55786f3ff9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:52.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.017+0000 7f1d1c967b80 1 --2- 172.21.15.101:0/2269249073 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55786f3fb2d0 0x55786f400b40 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:52.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.017+0000 7f1ce9ffb700 0 client.19320 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:38:52.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.017+0000 7f1d1c967b80 1 --2- 172.21.15.101:0/2269249073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55786f3fbbe0 0x55786f401080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:52.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.017+0000 7f1d1c967b80 1 -- 172.21.15.101:0/2269249073 >> 172.21.15.101:0/2269249073 conn(0x55786f3f1da0 msgr2=0x55786f3fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:52.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.017+0000 7f1d1c967b80 1 -- 172.21.15.101:0/2269249073 shutdown_connections 2023-07-18T03:38:52.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.017+0000 7f1d1c967b80 1 -- 172.21.15.101:0/2269249073 wait complete. 2023-07-18T03:38:52.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.018+0000 7f1d1c967b80 20 client.19320 trim_cache size 0 max 16384 2023-07-18T03:38:52.028 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:38:52.028 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:38:52.029 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:38:52.029 INFO:teuthology.orchestra.run.smithi101.stdout: "0f2f19e0-0429-465c-935e-223eb09490c3": { 2023-07-18T03:38:52.029 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:38:52.029 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "0f2f19e0-0429-465c-935e-223eb09490c3", 2023-07-18T03:38:52.029 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:38:52.030 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:38:52.030 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:38:52.030 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:38:52.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.018+0000 7f1d08fff700 1 -- 172.21.15.101:0/1631210163 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f1cec0a12c0 msgr2=0x7f1cec0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:52.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.018+0000 7f1d08fff700 1 --2- 172.21.15.101:0/1631210163 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f1cec0a12c0 0x7f1cec0a3770 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc00fb80 tx=0x7f1cfc004b40 comp rx=0 tx=0).stop 2023-07-18T03:38:52.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.018+0000 7f1d08fff700 1 -- 172.21.15.101:0/1631210163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d04101470 msgr2=0x7f1d04198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:52.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.018+0000 7f1d08fff700 1 --2- 172.21.15.101:0/1631210163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d04101470 0x7f1d04198340 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8009810 tx=0x7f1cf800fa00 comp rx=0 tx=0).stop 2023-07-18T03:38:52.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.018+0000 7f1d08fff700 1 -- 172.21.15.101:0/1631210163 shutdown_connections 2023-07-18T03:38:52.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.018+0000 7f1d08fff700 1 --2- 172.21.15.101:0/1631210163 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d0410a080 0x7f1d0419c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:52.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.018+0000 7f1d08fff700 1 --2- 172.21.15.101:0/1631210163 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1d04100b60 0x7f1d04197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:52.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.018+0000 7f1d08fff700 1 --2- 172.21.15.101:0/1631210163 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f1cec0a12c0 0x7f1cec0a3770 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:52.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.018+0000 7f1d08fff700 1 --2- 172.21.15.101:0/1631210163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d04101470 0x7f1d04198340 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:52.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.018+0000 7f1d08fff700 1 -- 172.21.15.101:0/1631210163 >> 172.21.15.101:0/1631210163 conn(0x7f1d040fa4b0 msgr2=0x7f1d0410c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:52.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.018+0000 7f1d08fff700 1 -- 172.21.15.101:0/1631210163 shutdown_connections 2023-07-18T03:38:52.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:38:52.018+0000 7f1d08fff700 1 -- 172.21.15.101:0/1631210163 wait complete. 2023-07-18T03:38:52.034 INFO:tasks.cephfs.filesystem:scrub status for tag:0f2f19e0-0429-465c-935e-223eb09490c3 - {'path': '/', 'tag': '0f2f19e0-0429-465c-935e-223eb09490c3', 'options': 'recursive,force'} 2023-07-18T03:38:53.701 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:53 smithi179 ceph-mon[129229]: pgmap v992: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:38:53.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:53 smithi101 ceph-mon[126379]: pgmap v992: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:38:53.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:53 smithi155 ceph-mon[128885]: pgmap v992: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:38:55.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:55 smithi101 ceph-mon[126379]: pgmap v993: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:38:55.796 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:55 smithi155 ceph-mon[128885]: pgmap v993: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:38:55.951 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:55 smithi179 ceph-mon[129229]: pgmap v993: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:38:57.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:57 smithi101 ceph-mon[126379]: pgmap v994: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:38:57.796 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:57 smithi155 ceph-mon[128885]: pgmap v994: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:38:57.951 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:57 smithi179 ceph-mon[129229]: pgmap v994: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:38:59.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:38:59 smithi101 ceph-mon[126379]: pgmap v995: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:38:59.796 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:38:59 smithi155 ceph-mon[128885]: pgmap v995: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:38:59.951 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:38:59 smithi179 ceph-mon[129229]: pgmap v995: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:39:01.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:39:01 smithi101 ceph-mon[126379]: pgmap v996: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:39:01.796 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:39:01 smithi155 ceph-mon[128885]: pgmap v996: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:39:01.951 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:39:01 smithi179 ceph-mon[129229]: pgmap v996: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:39:03.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:39:03 smithi101 ceph-mon[126379]: pgmap v997: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:39:03.796 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:39:03 smithi155 ceph-mon[128885]: pgmap v997: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:39:03.951 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:39:03 smithi179 ceph-mon[129229]: pgmap v997: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:39:05.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:39:05 smithi101 ceph-mon[126379]: pgmap v998: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:39:05.796 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:39:05 smithi155 ceph-mon[128885]: pgmap v998: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:39:05.951 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:39:05 smithi179 ceph-mon[129229]: pgmap v998: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:39:07.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:39:07 smithi101 ceph-mon[126379]: pgmap v999: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-07-18T03:39:07.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:39:07 smithi155 ceph-mon[128885]: pgmap v999: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-07-18T03:39:07.951 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:39:07 smithi179 ceph-mon[129229]: pgmap v999: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-07-18T03:39:09.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:39:09 smithi101 ceph-mon[126379]: pgmap v1000: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-07-18T03:39:09.796 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:39:09 smithi155 ceph-mon[128885]: pgmap v1000: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-07-18T03:39:09.951 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:39:09 smithi179 ceph-mon[129229]: pgmap v1000: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-07-18T03:39:11.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:39:11 smithi101 ceph-mon[126379]: pgmap v1001: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-07-18T03:39:11.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:39:11 smithi101 ceph-mon[126379]: osdmap e143: 12 total, 12 up, 12 in 2023-07-18T03:39:11.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:39:11 smithi155 ceph-mon[128885]: pgmap v1001: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-07-18T03:39:11.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:39:11 smithi155 ceph-mon[128885]: osdmap e143: 12 total, 12 up, 12 in 2023-07-18T03:39:11.951 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:39:11 smithi179 ceph-mon[129229]: pgmap v1001: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-07-18T03:39:11.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:39:11 smithi179 ceph-mon[129229]: osdmap e143: 12 total, 12 up, 12 in 2023-07-18T03:39:13.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:39:13 smithi101 ceph-mon[126379]: pgmap v1003: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-07-18T03:39:13.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:39:13 smithi155 ceph-mon[128885]: pgmap v1003: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-07-18T03:39:13.951 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:39:13 smithi179 ceph-mon[129229]: pgmap v1003: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-07-18T03:39:15.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:39:15 smithi101 ceph-mon[126379]: pgmap v1004: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-07-18T03:39:15.796 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:39:15 smithi155 ceph-mon[128885]: pgmap v1004: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-07-18T03:39:15.951 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:39:15 smithi179 ceph-mon[129229]: pgmap v1004: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-07-18T03:39:17.796 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:39:17 smithi155 ceph-mon[128885]: pgmap v1005: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:39:17.951 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:39:17 smithi179 ceph-mon[129229]: pgmap v1005: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:39:17.989 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:39:17 smithi101 ceph-mon[126379]: pgmap v1005: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:39:19.796 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:39:19 smithi155 ceph-mon[128885]: pgmap v1006: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:39:19.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:39:19 smithi155 ceph-mon[128885]: osdmap e144: 12 total, 12 up, 12 in 2023-07-18T03:39:19.951 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:39:19 smithi179 ceph-mon[129229]: pgmap v1006: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:39:19.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:39:19 smithi179 ceph-mon[129229]: osdmap e144: 12 total, 12 up, 12 in 2023-07-18T03:39:19.989 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:39:19 smithi101 ceph-mon[126379]: pgmap v1006: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:39:19.990 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:39:19 smithi101 ceph-mon[126379]: osdmap e144: 12 total, 12 up, 12 in 2023-07-18T03:39:21.796 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:39:21 smithi155 ceph-mon[128885]: pgmap v1008: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 316 B/s wr, 6 op/s 2023-07-18T03:39:21.951 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:39:21 smithi179 ceph-mon[129229]: pgmap v1008: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 316 B/s wr, 6 op/s 2023-07-18T03:39:21.989 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:39:21 smithi101 ceph-mon[126379]: pgmap v1008: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 316 B/s wr, 6 op/s 2023-07-18T03:39:22.035 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:39:22.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.250+0000 7fc8d0dff700 1 -- 172.21.15.101:0/1381551653 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8cc0728b0 msgr2=0x7fc8cc072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:22.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.250+0000 7fc8d0dff700 1 --2- 172.21.15.101:0/1381551653 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8cc0728b0 0x7fc8cc072c80 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc0098f0 tx=0x7fc8bc009c00 comp rx=0 tx=0).stop 2023-07-18T03:39:22.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.250+0000 7fc8d0dff700 1 -- 172.21.15.101:0/1381551653 shutdown_connections 2023-07-18T03:39:22.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.250+0000 7fc8d0dff700 1 --2- 172.21.15.101:0/1381551653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8cc10c310 0x7fc8cc10e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.250+0000 7fc8d0dff700 1 --2- 172.21.15.101:0/1381551653 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8cc0728b0 0x7fc8cc072c80 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.250+0000 7fc8d0dff700 1 --2- 172.21.15.101:0/1381551653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8cc0731c0 0x7fc8cc10bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.250+0000 7fc8d0dff700 1 -- 172.21.15.101:0/1381551653 >> 172.21.15.101:0/1381551653 conn(0x7fc8cc0fb5f0 msgr2=0x7fc8cc0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:22.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.250+0000 7fc8d0dff700 1 -- 172.21.15.101:0/1381551653 shutdown_connections 2023-07-18T03:39:22.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.250+0000 7fc8d0dff700 1 -- 172.21.15.101:0/1381551653 wait complete. 2023-07-18T03:39:22.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.252+0000 7fc8d0dff700 1 Processor -- start 2023-07-18T03:39:22.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.252+0000 7fc8d0dff700 1 -- start start 2023-07-18T03:39:22.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.253+0000 7fc8d0dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8cc0728b0 0x7fc8cc197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:22.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.253+0000 7fc8d0dff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8cc0731c0 0x7fc8cc1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:22.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.253+0000 7fc8d0dff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8cc10c310 0x7fc8cc19c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:22.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.253+0000 7fc8d0dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8cc19ce80 con 0x7fc8cc0728b0 2023-07-18T03:39:22.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.253+0000 7fc8d0dff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8cc19cff0 con 0x7fc8cc0731c0 2023-07-18T03:39:22.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.253+0000 7fc8d0dff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8cc19d1b0 con 0x7fc8cc10c310 2023-07-18T03:39:22.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.253+0000 7fc8c9d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8cc0731c0 0x7fc8cc1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:22.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.253+0000 7fc8ca59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8cc0728b0 0x7fc8cc197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:22.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.253+0000 7fc8c9d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8cc0731c0 0x7fc8cc1983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47298/0 (socket says 172.21.15.101:47298) 2023-07-18T03:39:22.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.253+0000 7fc8c9d9b700 1 -- 172.21.15.101:0/2921681173 learned_addr learned my addr 172.21.15.101:0/2921681173 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:39:22.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.253+0000 7fc8cad9d700 1 --2- 172.21.15.101:0/2921681173 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8cc10c310 0x7fc8cc19c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:22.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.254+0000 7fc8c9d9b700 1 -- 172.21.15.101:0/2921681173 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8cc10c310 msgr2=0x7fc8cc19c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:22.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.254+0000 7fc8c9d9b700 1 --2- 172.21.15.101:0/2921681173 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8cc10c310 0x7fc8cc19c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.254+0000 7fc8c9d9b700 1 -- 172.21.15.101:0/2921681173 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8cc0728b0 msgr2=0x7fc8cc197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:22.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.254+0000 7fc8c9d9b700 1 --2- 172.21.15.101:0/2921681173 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8cc0728b0 0x7fc8cc197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.254+0000 7fc8c9d9b700 1 -- 172.21.15.101:0/2921681173 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c4009d00 con 0x7fc8cc0731c0 2023-07-18T03:39:22.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.254+0000 7fc8c9d9b700 1 --2- 172.21.15.101:0/2921681173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8cc0731c0 0x7fc8cc1983e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc8c000e9a0 tx=0x7fc8c000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:22.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.254+0000 7fc8bb7fe700 1 -- 172.21.15.101:0/2921681173 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c0009790 con 0x7fc8cc0731c0 2023-07-18T03:39:22.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.254+0000 7fc8bb7fe700 1 -- 172.21.15.101:0/2921681173 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8c0010430 con 0x7fc8cc0731c0 2023-07-18T03:39:22.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.255+0000 7fc8bb7fe700 1 -- 172.21.15.101:0/2921681173 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c0018680 con 0x7fc8cc0731c0 2023-07-18T03:39:22.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.255+0000 7fc8d0dff700 1 -- 172.21.15.101:0/2921681173 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8bc009520 con 0x7fc8cc0731c0 2023-07-18T03:39:22.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.255+0000 7fc8d0dff700 1 -- 172.21.15.101:0/2921681173 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8cc1a4af0 con 0x7fc8cc0731c0 2023-07-18T03:39:22.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.259+0000 7fc8bb7fe700 1 -- 172.21.15.101:0/2921681173 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fc8c00105a0 con 0x7fc8cc0731c0 2023-07-18T03:39:22.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.260+0000 7fc8bb7fe700 1 --2- 172.21.15.101:0/2921681173 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc8b40a1340 0x7fc8b40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:22.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.260+0000 7fc8bb7fe700 1 -- 172.21.15.101:0/2921681173 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fc8c0014070 con 0x7fc8cc0731c0 2023-07-18T03:39:22.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.261+0000 7fc8ca59c700 1 --2- 172.21.15.101:0/2921681173 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc8b40a1340 0x7fc8b40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:22.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.261+0000 7fc8ca59c700 1 --2- 172.21.15.101:0/2921681173 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc8b40a1340 0x7fc8b40a37f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc005e90 tx=0x7fc8bc019040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:22.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.261+0000 7fc8e47d7b80 1 Processor -- start 2023-07-18T03:39:22.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.262+0000 7fc8e47d7b80 1 -- start start 2023-07-18T03:39:22.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.262+0000 7fc8e47d7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd1be232d0 0x55dd1be236a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:22.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.262+0000 7fc8e47d7b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd1be23be0 0x55dd1be2e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:22.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.262+0000 7fc8e47d7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd1be2ead0 0x55dd1be30eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:22.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.262+0000 7fc8e47d7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd1bd29cb0 con 0x55dd1be232d0 2023-07-18T03:39:22.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.262+0000 7fc8e47d7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd1bd7ca40 con 0x55dd1be2ead0 2023-07-18T03:39:22.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.262+0000 7fc8e47d7b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd1bd75ed0 con 0x55dd1be23be0 2023-07-18T03:39:22.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.262+0000 7fc8c9d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd1be23be0 0x55dd1be2e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:22.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.262+0000 7fc8c9d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd1be23be0 0x55dd1be2e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:59254/0 (socket says 172.21.15.101:59254) 2023-07-18T03:39:22.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.262+0000 7fc8c9d9b700 1 -- 172.21.15.101:0/174268053 learned_addr learned my addr 172.21.15.101:0/174268053 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:39:22.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.263+0000 7fc8ca59c700 1 --2- 172.21.15.101:0/174268053 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd1be232d0 0x55dd1be236a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:22.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.263+0000 7fc8cad9d700 1 --2- 172.21.15.101:0/174268053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd1be2ead0 0x55dd1be30eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:22.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.263+0000 7fc8c9d9b700 1 -- 172.21.15.101:0/174268053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd1be2ead0 msgr2=0x55dd1be30eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:22.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.263+0000 7fc8c9d9b700 1 --2- 172.21.15.101:0/174268053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd1be2ead0 0x55dd1be30eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.263+0000 7fc8c9d9b700 1 -- 172.21.15.101:0/174268053 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd1be232d0 msgr2=0x55dd1be236a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:22.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.263+0000 7fc8c9d9b700 1 --2- 172.21.15.101:0/174268053 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd1be232d0 0x55dd1be236a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.263+0000 7fc8c9d9b700 1 -- 172.21.15.101:0/174268053 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd1bba3de0 con 0x55dd1be23be0 2023-07-18T03:39:22.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.263+0000 7fc8ca59c700 1 --2- 172.21.15.101:0/174268053 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd1be232d0 0x55dd1be236a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:22.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.263+0000 7fc8c9d9b700 1 --2- 172.21.15.101:0/174268053 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd1be23be0 0x55dd1be2e590 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fc8c00a1d40 tx=0x7fc8c0099e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37f6fcb310b56172 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:22.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.263+0000 7fc8b8ff9700 1 -- 172.21.15.101:0/174268053 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c00b1070 con 0x55dd1be23be0 2023-07-18T03:39:22.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.263+0000 7fc8b8ff9700 1 -- 172.21.15.101:0/174268053 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8c00981c0 con 0x55dd1be23be0 2023-07-18T03:39:22.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.263+0000 7fc8b8ff9700 1 -- 172.21.15.101:0/174268053 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c0099570 con 0x55dd1be23be0 2023-07-18T03:39:22.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.264+0000 7fc8e47d7b80 1 -- 172.21.15.101:0/174268053 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd1be23be0 msgr2=0x55dd1be2e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:22.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.264+0000 7fc8e47d7b80 1 --2- 172.21.15.101:0/174268053 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd1be23be0 0x55dd1be2e590 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fc8c00a1d40 tx=0x7fc8c0099e20 comp rx=0 tx=0).stop 2023-07-18T03:39:22.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.264+0000 7fc8e47d7b80 1 -- 172.21.15.101:0/174268053 shutdown_connections 2023-07-18T03:39:22.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.264+0000 7fc8e47d7b80 1 --2- 172.21.15.101:0/174268053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd1be2ead0 0x55dd1be30eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.264+0000 7fc8e47d7b80 1 --2- 172.21.15.101:0/174268053 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd1be232d0 0x55dd1be236a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.264+0000 7fc8e47d7b80 1 --2- 172.21.15.101:0/174268053 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd1be23be0 0x55dd1be2e590 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.264+0000 7fc8e47d7b80 1 -- 172.21.15.101:0/174268053 >> 172.21.15.101:0/174268053 conn(0x55dd1be19da0 msgr2=0x55dd1be1f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:22.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.264+0000 7fc8e47d7b80 1 -- 172.21.15.101:0/174268053 shutdown_connections 2023-07-18T03:39:22.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.265+0000 7fc8e47d7b80 1 -- 172.21.15.101:0/174268053 wait complete. 2023-07-18T03:39:22.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.265+0000 7fc8e47d7b80 1 -- start start 2023-07-18T03:39:22.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.265+0000 7fc8e47d7b80 1 Processor -- start 2023-07-18T03:39:22.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.266+0000 7fc8e47d7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd1be232d0 0x55dd1be2b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:22.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.266+0000 7fc8e47d7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd1be23be0 0x55dd1be2bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:22.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.266+0000 7fc8e47d7b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd1be2ead0 0x55dd1be2c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:22.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.266+0000 7fc8e47d7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd1bd15ac0 con 0x55dd1be23be0 2023-07-18T03:39:22.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.266+0000 7fc8e47d7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd1bd157c0 con 0x55dd1be232d0 2023-07-18T03:39:22.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.266+0000 7fc8e47d7b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd1bbf9920 con 0x55dd1be2ead0 2023-07-18T03:39:22.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.266+0000 7fc8e47d7b80 10 client.0 _ll_register_callbacks cb 0x55dd1bd13760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:39:22.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.267+0000 7fc8ca59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd1be232d0 0x55dd1be2b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:22.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.267+0000 7fc8ca59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd1be232d0 0x55dd1be2b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47310/0 (socket says 172.21.15.101:47310) 2023-07-18T03:39:22.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.267+0000 7fc8ca59c700 1 -- 172.21.15.101:0/3869194721 learned_addr learned my addr 172.21.15.101:0/3869194721 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:39:22.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.267+0000 7fc8c9d9b700 1 --2- 172.21.15.101:0/3869194721 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd1be23be0 0x55dd1be2bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:22.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.267+0000 7fc8cad9d700 1 --2- 172.21.15.101:0/3869194721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd1be2ead0 0x55dd1be2c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:22.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.267+0000 7fc8c9d9b700 1 -- 172.21.15.101:0/3869194721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd1be2ead0 msgr2=0x55dd1be2c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:22.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.267+0000 7fc8c9d9b700 1 --2- 172.21.15.101:0/3869194721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd1be2ead0 0x55dd1be2c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.267+0000 7fc8c9d9b700 1 -- 172.21.15.101:0/3869194721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd1be232d0 msgr2=0x55dd1be2b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:22.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.267+0000 7fc8c9d9b700 1 --2- 172.21.15.101:0/3869194721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd1be232d0 0x55dd1be2b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.267+0000 7fc8c9d9b700 1 -- 172.21.15.101:0/3869194721 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8cc1a4af0 con 0x55dd1be23be0 2023-07-18T03:39:22.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.267+0000 7fc8cad9d700 1 --2- 172.21.15.101:0/3869194721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd1be2ead0 0x55dd1be2c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:22.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.267+0000 7fc8c9d9b700 1 --2- 172.21.15.101:0/3869194721 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd1be23be0 0x55dd1be2bd60 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0010b20 tx=0x7fc8c0010ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=349fbb21f2ef371b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:22.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.267+0000 7fc8b1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-18T03:39:22.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.268+0000 7fc8b1ffb700 1 -- 172.21.15.101:0/3869194721 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c00b1070 con 0x55dd1be23be0 2023-07-18T03:39:22.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.268+0000 7fc8b1ffb700 1 -- 172.21.15.101:0/3869194721 <== mon.0 v2:172.21.15.101:3300/0 2 ==== osd_map(144..144 src has 1..144) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fc8c00b8170 con 0x55dd1be23be0 2023-07-18T03:39:22.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.268+0000 7fc8e47d7b80 1 -- 172.21.15.101:0/3869194721 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dd1bccf4f0 con 0x55dd1be23be0 2023-07-18T03:39:22.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.268+0000 7fc8b1ffb700 1 -- 172.21.15.101:0/3869194721 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8c009a350 con 0x55dd1be23be0 2023-07-18T03:39:22.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.268+0000 7fc8e47d7b80 10 client.19332 fetch_fsmap learned FSMap version 49 2023-07-18T03:39:22.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.268+0000 7fc8e47d7b80 1 -- 172.21.15.101:0/3869194721 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55dd1bb8cc60 con 0x55dd1be23be0 2023-07-18T03:39:22.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.268+0000 7fc8b1ffb700 1 -- 172.21.15.101:0/3869194721 <== mon.0 v2:172.21.15.101:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dd1bccf4f0 con 0x55dd1be23be0 2023-07-18T03:39:22.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.268+0000 7fc8b1ffb700 1 -- 172.21.15.101:0/3869194721 <== mon.0 v2:172.21.15.101:3300/0 5 ==== fsmap(e 49) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc8c00a1f00 con 0x55dd1be23be0 2023-07-18T03:39:22.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.268+0000 7fc8e47d7b80 10 client.19332 fetch_fsmap finished waiting for FSMap version 49 2023-07-18T03:39:22.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.268+0000 7fc8e47d7b80 10 client.19332 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:39:22.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.269+0000 7fc8e47d7b80 20 client.19332 populate_metadata read hostname 'smithi101' 2023-07-18T03:39:22.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.269+0000 7fc8e47d7b80 1 --2- 172.21.15.101:0/3869194721 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55dd1be3e020 0x55dd1be40400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:22.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.269+0000 7fc8e47d7b80 4 client.19332 mds_command: new command op to 24562 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:39:22.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.269+0000 7fc8e47d7b80 1 -- 172.21.15.101:0/3869194721 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dd1bd6eeb0 con 0x55dd1be3e020 2023-07-18T03:39:22.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.269+0000 7fc8ca59c700 1 --2- 172.21.15.101:0/3869194721 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55dd1be3e020 0x55dd1be40400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:22.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.269+0000 7fc8b1ffb700 10 client.19332 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:39:22.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.269+0000 7fc8ca59c700 1 --2- 172.21.15.101:0/3869194721 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55dd1be3e020 0x55dd1be40400 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=221c3347ca650f74 server_cookie=e09de817bb6cd5d4 in_seq=0 out_seq=0 2023-07-18T03:39:22.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.270+0000 7fc8b1ffb700 1 -- 172.21.15.101:0/3869194721 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55dd1bd6eeb0 con 0x55dd1be3e020 2023-07-18T03:39:22.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.270+0000 7fc8b1ffb700 10 client.19332 handle_command_reply: tid=0 2023-07-18T03:39:22.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.270+0000 7fc8e47d7b80 1 client.19332 shutdown 2023-07-18T03:39:22.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.270+0000 7fc8e47d7b80 1 -- 172.21.15.101:0/3869194721 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd1be23be0 msgr2=0x55dd1be2bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:22.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.270+0000 7fc8e47d7b80 1 --2- 172.21.15.101:0/3869194721 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd1be23be0 0x55dd1be2bd60 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0010b20 tx=0x7fc8c0010ab0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.271+0000 7fc8e47d7b80 1 -- 172.21.15.101:0/3869194721 shutdown_connections 2023-07-18T03:39:22.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.271+0000 7fc8e47d7b80 1 --2- 172.21.15.101:0/3869194721 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55dd1be3e020 0x55dd1be40400 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.271+0000 7fc8e47d7b80 1 --2- 172.21.15.101:0/3869194721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd1be232d0 0x55dd1be2b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.271+0000 7fc8e47d7b80 1 --2- 172.21.15.101:0/3869194721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd1be2ead0 0x55dd1be2c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.271+0000 7fc8b1ffb700 0 client.19332 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:39:22.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.271+0000 7fc8e47d7b80 1 --2- 172.21.15.101:0/3869194721 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd1be23be0 0x55dd1be2bd60 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.271+0000 7fc8e47d7b80 1 -- 172.21.15.101:0/3869194721 >> 172.21.15.101:0/3869194721 conn(0x55dd1be19da0 msgr2=0x55dd1be1f230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:22.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.271+0000 7fc8e47d7b80 1 -- 172.21.15.101:0/3869194721 shutdown_connections 2023-07-18T03:39:22.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.271+0000 7fc8e47d7b80 1 -- 172.21.15.101:0/3869194721 wait complete. 2023-07-18T03:39:22.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.271+0000 7fc8e47d7b80 20 client.19332 trim_cache size 0 max 16384 2023-07-18T03:39:22.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.306+0000 7fc8e47d7b80 1 Processor -- start 2023-07-18T03:39:22.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.306+0000 7fc8e47d7b80 1 -- start start 2023-07-18T03:39:22.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.307+0000 7fc8e47d7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd1be3ab50 0x55dd1be354f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:22.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.307+0000 7fc8e47d7b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd1be19da0 0x55dd1be258b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:22.303 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.307+0000 7fc8e47d7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd1be232d0 0x55dd1be25df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:22.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.307+0000 7fc8e47d7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd1be238f0 con 0x55dd1be3ab50 2023-07-18T03:39:22.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.307+0000 7fc8e47d7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd1be1bec0 con 0x55dd1be232d0 2023-07-18T03:39:22.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.307+0000 7fc8e47d7b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd1be24540 con 0x55dd1be19da0 2023-07-18T03:39:22.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.307+0000 7fc8c9d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd1be19da0 0x55dd1be258b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:22.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.307+0000 7fc8c9d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd1be19da0 0x55dd1be258b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:59270/0 (socket says 172.21.15.101:59270) 2023-07-18T03:39:22.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.307+0000 7fc8c9d9b700 1 -- 172.21.15.101:0/2976090000 learned_addr learned my addr 172.21.15.101:0/2976090000 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:39:22.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.307+0000 7fc8cad9d700 1 --2- 172.21.15.101:0/2976090000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd1be232d0 0x55dd1be25df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:22.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.307+0000 7fc8ca59c700 1 --2- 172.21.15.101:0/2976090000 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd1be3ab50 0x55dd1be354f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:22.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.307+0000 7fc8c9d9b700 1 -- 172.21.15.101:0/2976090000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd1be232d0 msgr2=0x55dd1be25df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:22.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.307+0000 7fc8c9d9b700 1 --2- 172.21.15.101:0/2976090000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd1be232d0 0x55dd1be25df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.307+0000 7fc8c9d9b700 1 -- 172.21.15.101:0/2976090000 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd1be3ab50 msgr2=0x55dd1be354f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:22.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.307+0000 7fc8c9d9b700 1 --2- 172.21.15.101:0/2976090000 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd1be3ab50 0x55dd1be354f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.308+0000 7fc8c9d9b700 1 -- 172.21.15.101:0/2976090000 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd1be1b5e0 con 0x55dd1be19da0 2023-07-18T03:39:22.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.308+0000 7fc8c9d9b700 1 --2- 172.21.15.101:0/2976090000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd1be19da0 0x55dd1be258b0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fc8c00caa20 tx=0x7fc8c0099e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a47dc3b1e7811bb9 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:22.306 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.308+0000 7fc8b97fa700 1 -- 172.21.15.101:0/2976090000 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c00b1020 con 0x55dd1be19da0 2023-07-18T03:39:22.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.308+0000 7fc8b97fa700 1 -- 172.21.15.101:0/2976090000 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8c00b8170 con 0x55dd1be19da0 2023-07-18T03:39:22.307 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.308+0000 7fc8b97fa700 1 -- 172.21.15.101:0/2976090000 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c00ca140 con 0x55dd1be19da0 2023-07-18T03:39:22.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.309+0000 7fc8e47d7b80 1 -- 172.21.15.101:0/2976090000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd1be19da0 msgr2=0x55dd1be258b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:22.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.309+0000 7fc8e47d7b80 1 --2- 172.21.15.101:0/2976090000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd1be19da0 0x55dd1be258b0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fc8c00caa20 tx=0x7fc8c0099e20 comp rx=0 tx=0).stop 2023-07-18T03:39:22.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.309+0000 7fc8e47d7b80 1 -- 172.21.15.101:0/2976090000 shutdown_connections 2023-07-18T03:39:22.310 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.309+0000 7fc8e47d7b80 1 --2- 172.21.15.101:0/2976090000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd1be232d0 0x55dd1be25df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.309+0000 7fc8e47d7b80 1 --2- 172.21.15.101:0/2976090000 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd1be3ab50 0x55dd1be354f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.309+0000 7fc8e47d7b80 1 --2- 172.21.15.101:0/2976090000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd1be19da0 0x55dd1be258b0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.309+0000 7fc8e47d7b80 1 -- 172.21.15.101:0/2976090000 >> 172.21.15.101:0/2976090000 conn(0x55dd1be1e170 msgr2=0x55dd1be251a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:22.311 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.309+0000 7fc8e47d7b80 1 -- 172.21.15.101:0/2976090000 shutdown_connections 2023-07-18T03:39:22.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.309+0000 7fc8e47d7b80 1 -- 172.21.15.101:0/2976090000 wait complete. 2023-07-18T03:39:22.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.309+0000 7fc8e47d7b80 1 -- start start 2023-07-18T03:39:22.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.309+0000 7fc8e47d7b80 1 Processor -- start 2023-07-18T03:39:22.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.310+0000 7fc8e47d7b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd1be19da0 0x55dd1be8bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:22.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.310+0000 7fc8e47d7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd1be232d0 0x55dd1be8c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:22.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.310+0000 7fc8e47d7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd1be3ab50 0x55dd1be85e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:22.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.310+0000 7fc8e47d7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd1be86450 con 0x55dd1be232d0 2023-07-18T03:39:22.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.310+0000 7fc8e47d7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd1be865c0 con 0x55dd1be3ab50 2023-07-18T03:39:22.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.310+0000 7fc8e47d7b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd1be86780 con 0x55dd1be19da0 2023-07-18T03:39:22.313 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.310+0000 7fc8e47d7b80 10 client.0 _ll_register_callbacks cb 0x55dd1bd13760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:39:22.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.310+0000 7fc8ca59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd1be19da0 0x55dd1be8bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:22.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.311+0000 7fc8ca59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd1be19da0 0x55dd1be8bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:59278/0 (socket says 172.21.15.101:59278) 2023-07-18T03:39:22.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.311+0000 7fc8ca59c700 1 -- 172.21.15.101:0/4277102357 learned_addr learned my addr 172.21.15.101:0/4277102357 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:39:22.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.311+0000 7fc8c9d9b700 1 --2- 172.21.15.101:0/4277102357 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd1be232d0 0x55dd1be8c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:22.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.311+0000 7fc8cad9d700 1 --2- 172.21.15.101:0/4277102357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd1be3ab50 0x55dd1be85e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:22.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.311+0000 7fc8ca59c700 1 -- 172.21.15.101:0/4277102357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd1be3ab50 msgr2=0x55dd1be85e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:22.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.311+0000 7fc8ca59c700 1 --2- 172.21.15.101:0/4277102357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd1be3ab50 0x55dd1be85e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.311+0000 7fc8ca59c700 1 -- 172.21.15.101:0/4277102357 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd1be232d0 msgr2=0x55dd1be8c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:22.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.311+0000 7fc8ca59c700 1 --2- 172.21.15.101:0/4277102357 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd1be232d0 0x55dd1be8c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.311+0000 7fc8ca59c700 1 -- 172.21.15.101:0/4277102357 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dd1bb8cc60 con 0x55dd1be19da0 2023-07-18T03:39:22.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.311+0000 7fc8c9d9b700 1 --2- 172.21.15.101:0/4277102357 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd1be232d0 0x55dd1be8c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:22.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.311+0000 7fc8ca59c700 1 --2- 172.21.15.101:0/4277102357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd1be19da0 0x55dd1be8bb30 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc00fab0 tx=0x7fc8bc009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92decc76f70a6198 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:22.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.311+0000 7fc8b1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2023-07-18T03:39:22.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.311+0000 7fc8b1ffb700 1 -- 172.21.15.101:0/4277102357 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc02b050 con 0x55dd1be19da0 2023-07-18T03:39:22.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.311+0000 7fc8b1ffb700 1 -- 172.21.15.101:0/4277102357 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(144..144 src has 1..144) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fc8bc00b860 con 0x55dd1be19da0 2023-07-18T03:39:22.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.311+0000 7fc8e47d7b80 1 -- 172.21.15.101:0/4277102357 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc8c00981c0 con 0x55dd1be19da0 2023-07-18T03:39:22.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.312+0000 7fc8b1ffb700 1 -- 172.21.15.101:0/4277102357 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8bc032e50 con 0x55dd1be19da0 2023-07-18T03:39:22.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.312+0000 7fc8e47d7b80 10 client.29440 fetch_fsmap learned FSMap version 49 2023-07-18T03:39:22.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.312+0000 7fc8e47d7b80 1 -- 172.21.15.101:0/4277102357 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55dd1bbca250 con 0x55dd1be19da0 2023-07-18T03:39:22.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.312+0000 7fc8b1ffb700 1 -- 172.21.15.101:0/4277102357 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8c00981c0 con 0x55dd1be19da0 2023-07-18T03:39:22.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.312+0000 7fc8b1ffb700 1 -- 172.21.15.101:0/4277102357 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 49) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc8bc005370 con 0x55dd1be19da0 2023-07-18T03:39:22.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.312+0000 7fc8e47d7b80 10 client.29440 fetch_fsmap finished waiting for FSMap version 49 2023-07-18T03:39:22.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.312+0000 7fc8e47d7b80 10 client.29440 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-18T03:39:22.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.312+0000 7fc8e47d7b80 20 client.29440 populate_metadata read hostname 'smithi101' 2023-07-18T03:39:22.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.312+0000 7fc8e47d7b80 1 --2- 172.21.15.101:0/4277102357 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55dd1be23be0 0x55dd1be95970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:22.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.312+0000 7fc8e47d7b80 4 client.29440 mds_command: new command op to 24562 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:39:22.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.312+0000 7fc8e47d7b80 1 -- 172.21.15.101:0/4277102357 --> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55dd1bd31050 con 0x55dd1be23be0 2023-07-18T03:39:22.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.312+0000 7fc8c9d9b700 1 --2- 172.21.15.101:0/4277102357 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55dd1be23be0 0x55dd1be95970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:22.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.313+0000 7fc8c9d9b700 1 --2- 172.21.15.101:0/4277102357 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55dd1be23be0 0x55dd1be95970 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a121b38b52a0b9a6 server_cookie=f440fbb75b9b7639 in_seq=0 out_seq=0 2023-07-18T03:39:22.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.313+0000 7fc8b1ffb700 10 client.29440 ms_handle_connect on v2:172.21.15.179:6838/2061828474 2023-07-18T03:39:22.319 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.313+0000 7fc8b1ffb700 1 -- 172.21.15.101:0/4277102357 <== mds.0 v2:172.21.15.179:6838/2061828474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55dd1bd31050 con 0x55dd1be23be0 2023-07-18T03:39:22.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.313+0000 7fc8b1ffb700 10 client.29440 handle_command_reply: tid=0 2023-07-18T03:39:22.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.313+0000 7fc8e47d7b80 1 client.29440 shutdown 2023-07-18T03:39:22.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.313+0000 7fc8e47d7b80 1 -- 172.21.15.101:0/4277102357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd1be19da0 msgr2=0x55dd1be8bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:22.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.313+0000 7fc8e47d7b80 1 --2- 172.21.15.101:0/4277102357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd1be19da0 0x55dd1be8bb30 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc00fab0 tx=0x7fc8bc009260 comp rx=0 tx=0).stop 2023-07-18T03:39:22.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.313+0000 7fc8e47d7b80 1 -- 172.21.15.101:0/4277102357 shutdown_connections 2023-07-18T03:39:22.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.313+0000 7fc8e47d7b80 1 --2- 172.21.15.101:0/4277102357 >> [v2:172.21.15.179:6838/2061828474,v1:172.21.15.179:6839/2061828474] conn(0x55dd1be23be0 0x55dd1be95970 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.313+0000 7fc8e47d7b80 1 --2- 172.21.15.101:0/4277102357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd1be3ab50 0x55dd1be85e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.313+0000 7fc8e47d7b80 1 --2- 172.21.15.101:0/4277102357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd1be19da0 0x55dd1be8bb30 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.313+0000 7fc8b1ffb700 0 client.29440 ms_handle_reset on v2:172.21.15.179:6838/2061828474 2023-07-18T03:39:22.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.313+0000 7fc8e47d7b80 1 --2- 172.21.15.101:0/4277102357 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd1be232d0 0x55dd1be8c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.313+0000 7fc8e47d7b80 1 -- 172.21.15.101:0/4277102357 >> 172.21.15.101:0/4277102357 conn(0x55dd1be1e170 msgr2=0x55dd1be25a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:22.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.314+0000 7fc8e47d7b80 1 -- 172.21.15.101:0/4277102357 shutdown_connections 2023-07-18T03:39:22.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.314+0000 7fc8e47d7b80 1 -- 172.21.15.101:0/4277102357 wait complete. 2023-07-18T03:39:22.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.314+0000 7fc8e47d7b80 20 client.29440 trim_cache size 0 max 16384 2023-07-18T03:39:22.323 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:39:22.323 INFO:teuthology.orchestra.run.smithi101.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:39:22.323 INFO:teuthology.orchestra.run.smithi101.stdout: "scrubs": { 2023-07-18T03:39:22.323 INFO:teuthology.orchestra.run.smithi101.stdout: "0f2f19e0-0429-465c-935e-223eb09490c3": { 2023-07-18T03:39:22.324 INFO:teuthology.orchestra.run.smithi101.stdout: "path": "/", 2023-07-18T03:39:22.324 INFO:teuthology.orchestra.run.smithi101.stdout: "tag": "0f2f19e0-0429-465c-935e-223eb09490c3", 2023-07-18T03:39:22.324 INFO:teuthology.orchestra.run.smithi101.stdout: "options": "recursive,force" 2023-07-18T03:39:22.324 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:22.325 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:22.325 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:39:22.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.314+0000 7fc8d0dff700 1 -- 172.21.15.101:0/2921681173 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc8b40a1340 msgr2=0x7fc8b40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:22.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.314+0000 7fc8d0dff700 1 --2- 172.21.15.101:0/2921681173 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc8b40a1340 0x7fc8b40a37f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc005e90 tx=0x7fc8bc019040 comp rx=0 tx=0).stop 2023-07-18T03:39:22.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.314+0000 7fc8d0dff700 1 -- 172.21.15.101:0/2921681173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8cc0731c0 msgr2=0x7fc8cc1983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:22.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.314+0000 7fc8d0dff700 1 --2- 172.21.15.101:0/2921681173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8cc0731c0 0x7fc8cc1983e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc8c000e9a0 tx=0x7fc8c000ecb0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.314+0000 7fc8d0dff700 1 -- 172.21.15.101:0/2921681173 shutdown_connections 2023-07-18T03:39:22.327 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.315+0000 7fc8d0dff700 1 --2- 172.21.15.101:0/2921681173 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8cc10c310 0x7fc8cc19c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.327 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.315+0000 7fc8d0dff700 1 --2- 172.21.15.101:0/2921681173 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8cc0728b0 0x7fc8cc197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.327 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.315+0000 7fc8d0dff700 1 --2- 172.21.15.101:0/2921681173 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fc8b40a1340 0x7fc8b40a37f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.327 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.315+0000 7fc8d0dff700 1 --2- 172.21.15.101:0/2921681173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8cc0731c0 0x7fc8cc1983e0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.327 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.315+0000 7fc8d0dff700 1 -- 172.21.15.101:0/2921681173 >> 172.21.15.101:0/2921681173 conn(0x7fc8cc0fb5f0 msgr2=0x7fc8cc10e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:22.328 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.315+0000 7fc8d0dff700 1 -- 172.21.15.101:0/2921681173 shutdown_connections 2023-07-18T03:39:22.328 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.315+0000 7fc8d0dff700 1 -- 172.21.15.101:0/2921681173 wait complete. 2023-07-18T03:39:22.330 INFO:tasks.cephfs.filesystem:scrub status for tag:0f2f19e0-0429-465c-935e-223eb09490c3 - {'path': '/', 'tag': '0f2f19e0-0429-465c-935e-223eb09490c3', 'options': 'recursive,force'} 2023-07-18T03:39:22.331 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/fwd_scrub.py", line 76, in _scrub done = self.fs.wait_until_scrub_complete(tag=tag, sleep=30, timeout=self.scrub_timeout) File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/cephfs/filesystem.py", line 1684, in wait_until_scrub_complete while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (31) after waiting for 900 seconds 2023-07-18T03:39:22.340 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: reached maximum tries (31) after waiting for 900 seconds 2023-07-18T03:39:22.343 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-07-18T03:39:22.353 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:39:22.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.573+0000 7f308c7ff700 1 -- 172.21.15.101:0/3754496874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3084100ca0 msgr2=0x7f3084101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:22.568 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.573+0000 7f308c7ff700 1 --2- 172.21.15.101:0/3754496874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3084100ca0 0x7f3084101070 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f3080009990 tx=0x7f3080009ca0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.573+0000 7f308c7ff700 1 -- 172.21.15.101:0/3754496874 shutdown_connections 2023-07-18T03:39:22.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.573+0000 7f308c7ff700 1 --2- 172.21.15.101:0/3754496874 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f308410a070 0x7f308410c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.573+0000 7f308c7ff700 1 --2- 172.21.15.101:0/3754496874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3084100ca0 0x7f3084101070 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.573+0000 7f308c7ff700 1 --2- 172.21.15.101:0/3754496874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30841015b0 0x7f3084109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.569 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.573+0000 7f308c7ff700 1 -- 172.21.15.101:0/3754496874 >> 172.21.15.101:0/3754496874 conn(0x7f30840fa490 msgr2=0x7f30840fc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:22.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.573+0000 7f308c7ff700 1 -- 172.21.15.101:0/3754496874 shutdown_connections 2023-07-18T03:39:22.570 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.573+0000 7f308c7ff700 1 -- 172.21.15.101:0/3754496874 wait complete. 2023-07-18T03:39:22.571 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.574+0000 7f308c7ff700 1 Processor -- start 2023-07-18T03:39:22.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.575+0000 7f308c7ff700 1 -- start start 2023-07-18T03:39:22.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.575+0000 7f308c7ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3084100ca0 0x7f308419b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:22.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.575+0000 7f308c7ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f30841015b0 0x7f308419bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:22.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.575+0000 7f308c7ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f308410a070 0x7f3084195a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:22.572 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.575+0000 7f308c7ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30841960c0 con 0x7f30841015b0 2023-07-18T03:39:22.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.575+0000 7f308c7ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3084196230 con 0x7f308410a070 2023-07-18T03:39:22.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.575+0000 7f308c7ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30841963f0 con 0x7f3084100ca0 2023-07-18T03:39:22.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.575+0000 7f308a59b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3084100ca0 0x7f308419b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:22.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.575+0000 7f308a59b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3084100ca0 0x7f308419b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:59292/0 (socket says 172.21.15.101:59292) 2023-07-18T03:39:22.573 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.575+0000 7f308a59b700 1 -- 172.21.15.101:0/4240987869 learned_addr learned my addr 172.21.15.101:0/4240987869 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:39:22.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.576+0000 7f3089d9a700 1 --2- 172.21.15.101:0/4240987869 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f30841015b0 0x7f308419bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:22.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.576+0000 7f308ad9c700 1 --2- 172.21.15.101:0/4240987869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f308410a070 0x7f3084195a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:22.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.576+0000 7f308a59b700 1 -- 172.21.15.101:0/4240987869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f308410a070 msgr2=0x7f3084195a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:22.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.576+0000 7f308a59b700 1 --2- 172.21.15.101:0/4240987869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f308410a070 0x7f3084195a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.574 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.576+0000 7f308a59b700 1 -- 172.21.15.101:0/4240987869 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f30841015b0 msgr2=0x7f308419bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:22.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.576+0000 7f308a59b700 1 --2- 172.21.15.101:0/4240987869 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f30841015b0 0x7f308419bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.576+0000 7f308a59b700 1 -- 172.21.15.101:0/4240987869 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30800095f0 con 0x7f3084100ca0 2023-07-18T03:39:22.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.576+0000 7f3089d9a700 1 --2- 172.21.15.101:0/4240987869 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f30841015b0 0x7f308419bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:22.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.576+0000 7f308a59b700 1 --2- 172.21.15.101:0/4240987869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3084100ca0 0x7f308419b9e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f3080005910 tx=0x7f3080005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:22.575 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.576+0000 7f307b7fe700 1 -- 172.21.15.101:0/4240987869 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308001d070 con 0x7f3084100ca0 2023-07-18T03:39:22.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.576+0000 7f307b7fe700 1 -- 172.21.15.101:0/4240987869 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3080022470 con 0x7f3084100ca0 2023-07-18T03:39:22.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.576+0000 7f308c7ff700 1 -- 172.21.15.101:0/4240987869 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3084196670 con 0x7f3084100ca0 2023-07-18T03:39:22.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.576+0000 7f307b7fe700 1 -- 172.21.15.101:0/4240987869 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3080017650 con 0x7f3084100ca0 2023-07-18T03:39:22.576 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.577+0000 7f308c7ff700 1 -- 172.21.15.101:0/4240987869 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30841a29d0 con 0x7f3084100ca0 2023-07-18T03:39:22.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.578+0000 7f308c7ff700 1 -- 172.21.15.101:0/4240987869 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3084067300 con 0x7f3084100ca0 2023-07-18T03:39:22.577 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.579+0000 7f307b7fe700 1 -- 172.21.15.101:0/4240987869 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f30800225e0 con 0x7f3084100ca0 2023-07-18T03:39:22.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.581+0000 7f307b7fe700 1 --2- 172.21.15.101:0/4240987869 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f30700a1380 0x7f30700a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:22.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.581+0000 7f307b7fe700 1 -- 172.21.15.101:0/4240987869 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f30800ce580 con 0x7f3084100ca0 2023-07-18T03:39:22.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.581+0000 7f3089d9a700 1 --2- 172.21.15.101:0/4240987869 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f30700a1380 0x7f30700a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:22.578 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.582+0000 7f3089d9a700 1 --2- 172.21.15.101:0/4240987869 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f30700a1380 0x7f30700a3830 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f308410b580 tx=0x7f3074009260 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:22.579 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.584+0000 7f307b7fe700 1 -- 172.21.15.101:0/4240987869 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3080097670 con 0x7f3084100ca0 2023-07-18T03:39:22.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:39:22 smithi101 ceph-mon[126379]: pgmap v1009: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 307 B/s wr, 6 op/s 2023-07-18T03:39:22.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.796+0000 7f308c7ff700 1 -- 172.21.15.101:0/4240987869 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3084196f60 con 0x7f3084100ca0 2023-07-18T03:39:22.793 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.798+0000 7f307b7fe700 1 -- 172.21.15.101:0/4240987869 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) v1 ==== 74+0+25296 (secure 0 0 0) 0x7f308009ac90 con 0x7f3084100ca0 2023-07-18T03:39:22.793 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:39:22.794 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":144,"fsid":"b1782daa-2516-11ee-9b34-001a4aab830c","created":"2023-07-18T02:57:22.972483+0000","modified":"2023-07-18T03:39:19.449794+0000","last_up_change":"2023-07-18T03:04:11.429530+0000","last_in_change":"2023-07-18T03:03:54.158340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:00:43.353489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:04:57.771616+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-18T03:04:58.769590+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":59,"snap_epoch":143,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-18T03:05:02.358830+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":59,"snap_epoch":143,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc76b5d1-38a0-4f1a-9335-756bfc40a62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6803","nonce":3610719929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6805","nonce":3610719929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6809","nonce":3610719929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6807","nonce":3610719929}]},"public_addr":"172.21.15.101:6803/3610719929","cluster_addr":"172.21.15.101:6805/3610719929","heartbeat_back_addr":"172.21.15.101:6809/3610719929","heartbeat_front_addr":"172.21.15.101:6807/3610719929","state":["exists","up"]},{"osd":1,"uuid":"2a299573-0b34-4af9-aaae-b523e5058ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6801","nonce":1030229579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6803","nonce":1030229579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6807","nonce":1030229579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6805","nonce":1030229579}]},"public_addr":"172.21.15.155:6801/1030229579","cluster_addr":"172.21.15.155:6803/1030229579","heartbeat_back_addr":"172.21.15.155:6807/1030229579","heartbeat_front_addr":"172.21.15.155:6805/1030229579","state":["exists","up"]},{"osd":2,"uuid":"123b3749-c636-48fa-9e33-6229b5575dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6801","nonce":1349487507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6803","nonce":1349487507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6807","nonce":1349487507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6805","nonce":1349487507}]},"public_addr":"172.21.15.179:6801/1349487507","cluster_addr":"172.21.15.179:6803/1349487507","heartbeat_back_addr":"172.21.15.179:6807/1349487507","heartbeat_front_addr":"172.21.15.179:6805/1349487507","state":["exists","up"]},{"osd":3,"uuid":"4b0c409d-2954-4127-833a-b196bf226d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6811","nonce":4226577463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6813","nonce":4226577463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6817","nonce":4226577463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6815","nonce":4226577463}]},"public_addr":"172.21.15.101:6811/4226577463","cluster_addr":"172.21.15.101:6813/4226577463","heartbeat_back_addr":"172.21.15.101:6817/4226577463","heartbeat_front_addr":"172.21.15.101:6815/4226577463","state":["exists","up"]},{"osd":4,"uuid":"2137f2df-984e-45ce-baf6-b4ac12c7e3c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6809","nonce":1951135805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6811","nonce":1951135805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6815","nonce":1951135805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6813","nonce":1951135805}]},"public_addr":"172.21.15.155:6809/1951135805","cluster_addr":"172.21.15.155:6811/1951135805","heartbeat_back_addr":"172.21.15.155:6815/1951135805","heartbeat_front_addr":"172.21.15.155:6813/1951135805","state":["exists","up"]},{"osd":5,"uuid":"1020bfc5-8c2f-482e-9588-66c7f98e46e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6809","nonce":2746037021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6811","nonce":2746037021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6815","nonce":2746037021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6813","nonce":2746037021}]},"public_addr":"172.21.15.179:6809/2746037021","cluster_addr":"172.21.15.179:6811/2746037021","heartbeat_back_addr":"172.21.15.179:6815/2746037021","heartbeat_front_addr":"172.21.15.179:6813/2746037021","state":["exists","up"]},{"osd":6,"uuid":"2cd99260-5c36-4603-9338-cf697970ce58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6819","nonce":958044931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6821","nonce":958044931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6825","nonce":958044931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6823","nonce":958044931}]},"public_addr":"172.21.15.101:6819/958044931","cluster_addr":"172.21.15.101:6821/958044931","heartbeat_back_addr":"172.21.15.101:6825/958044931","heartbeat_front_addr":"172.21.15.101:6823/958044931","state":["exists","up"]},{"osd":7,"uuid":"ed23fd0a-678f-4e15-b42c-cfe873315714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6817","nonce":3256322291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6819","nonce":3256322291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6823","nonce":3256322291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6821","nonce":3256322291}]},"public_addr":"172.21.15.155:6817/3256322291","cluster_addr":"172.21.15.155:6819/3256322291","heartbeat_back_addr":"172.21.15.155:6823/3256322291","heartbeat_front_addr":"172.21.15.155:6821/3256322291","state":["exists","up"]},{"osd":8,"uuid":"eca3d8e6-7baf-47a5-b991-328001c7ebef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6817","nonce":3520612559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6819","nonce":3520612559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6823","nonce":3520612559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6821","nonce":3520612559}]},"public_addr":"172.21.15.179:6817/3520612559","cluster_addr":"172.21.15.179:6819/3520612559","heartbeat_back_addr":"172.21.15.179:6823/3520612559","heartbeat_front_addr":"172.21.15.179:6821/3520612559","state":["exists","up"]},{"osd":9,"uuid":"465a7ffb-67e4-49ef-812c-1b96feacfebf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6827","nonce":2166787667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6829","nonce":2166787667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6833","nonce":2166787667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6831","nonce":2166787667}]},"public_addr":"172.21.15.101:6827/2166787667","cluster_addr":"172.21.15.101:6829/2166787667","heartbeat_back_addr":"172.21.15.101:6833/2166787667","heartbeat_front_addr":"172.21.15.101:6831/2166787667","state":["exists","up"]},{"osd":10,"uuid":"1897a027-f25d-434f-8df5-5fc2bd866578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6825","nonce":3682446349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6827","nonce":3682446349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6831","nonce":3682446349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6829","nonce":3682446349}]},"public_addr":"172.21.15.155:6825/3682446349","cluster_addr":"172.21.15.155:6827/3682446349","heartbeat_back_addr":"172.21.15.155:6831/3682446349","heartbeat_front_addr":"172.21.15.155:6829/3682446349","state":["exists","up"]},{"osd":11,"uuid":"b62effaf-45e2-4585-b8d0-db448f3474ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6825","nonce":2602640858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6827","nonce":2602640858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6831","nonce":2602640858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6829","nonce":2602640858}]},"public_addr":"172.21.15.179:6825/2602640858","cluster_addr":"172.21.15.179:6827/2602640858","heartbeat_back_addr":"172.21.15.179:6831/2602640858","heartbeat_front_addr":"172.21.15.179:6829/2602640858","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T02:59:56.074519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:19.358142+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:41.128966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:08.227054+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:31.442592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:52.751685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:19.251054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:40.698241+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:01.339150+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:27.955910+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:49.136980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:04:10.084982+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.101:0/2766365637":"2023-07-19T03:07:33.498516+0000","172.21.15.101:0/523372970":"2023-07-19T03:07:33.498516+0000","172.21.15.101:6800/3963742842":"2023-07-19T03:07:33.498516+0000","172.21.15.101:6800/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/1806538035":"2023-07-19T03:07:33.498516+0000","172.21.15.101:6801/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/604228291":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/1004430030":"2023-07-19T03:07:33.498516+0000","172.21.15.101:6801/3963742842":"2023-07-19T03:07:33.498516+0000","172.21.15.101:0/4055187849":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/2844574324":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/1603248215":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6801/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6800/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/3307578182":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/373189792":"2023-07-19T02:57:56.243200+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":46,"length":1},{"begin":59,"length":1}]},{"pool":4,"snaps":[{"begin":46,"length":1},{"begin":59,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:39:22.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.800+0000 7f308c7ff700 1 -- 172.21.15.101:0/4240987869 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f30700a1380 msgr2=0x7f30700a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:22.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.800+0000 7f308c7ff700 1 --2- 172.21.15.101:0/4240987869 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f30700a1380 0x7f30700a3830 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f308410b580 tx=0x7f3074009260 comp rx=0 tx=0).stop 2023-07-18T03:39:22.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.800+0000 7f308c7ff700 1 -- 172.21.15.101:0/4240987869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3084100ca0 msgr2=0x7f308419b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:22.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.800+0000 7f308c7ff700 1 --2- 172.21.15.101:0/4240987869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3084100ca0 0x7f308419b9e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f3080005910 tx=0x7f3080005720 comp rx=0 tx=0).stop 2023-07-18T03:39:22.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.800+0000 7f308c7ff700 1 -- 172.21.15.101:0/4240987869 shutdown_connections 2023-07-18T03:39:22.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.800+0000 7f308c7ff700 1 --2- 172.21.15.101:0/4240987869 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f30700a1380 0x7f30700a3830 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.800+0000 7f308c7ff700 1 --2- 172.21.15.101:0/4240987869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f308410a070 0x7f3084195a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.800+0000 7f308c7ff700 1 --2- 172.21.15.101:0/4240987869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3084100ca0 0x7f308419b9e0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.800+0000 7f308c7ff700 1 --2- 172.21.15.101:0/4240987869 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f30841015b0 0x7f308419bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:22.797 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.800+0000 7f308c7ff700 1 -- 172.21.15.101:0/4240987869 >> 172.21.15.101:0/4240987869 conn(0x7f30840fa490 msgr2=0x7f308410c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:22.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.801+0000 7f308c7ff700 1 -- 172.21.15.101:0/4240987869 shutdown_connections 2023-07-18T03:39:22.798 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:22.801+0000 7f308c7ff700 1 -- 172.21.15.101:0/4240987869 wait complete. 2023-07-18T03:39:22.815 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:00:43.353489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-18T03:04:57.771616+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-18T03:04:58.769590+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '143', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 59, 'snap_epoch': 143, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-18T03:05:02.358830+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '143', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 59, 'snap_epoch': 143, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-18T03:39:22.816 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-18T03:39:22.951 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:39:22 smithi179 ceph-mon[129229]: pgmap v1009: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 307 B/s wr, 6 op/s 2023-07-18T03:39:23.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.038+0000 7f4ef17ff700 1 -- 172.21.15.101:0/3645830046 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4eec1015b0 msgr2=0x7f4eec109aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:23.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.038+0000 7f4ef17ff700 1 --2- 172.21.15.101:0/3645830046 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4eec1015b0 0x7f4eec109aa0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0009990 tx=0x7f4ee0009ca0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.039+0000 7f4ef17ff700 1 -- 172.21.15.101:0/3645830046 shutdown_connections 2023-07-18T03:39:23.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.039+0000 7f4ef17ff700 1 --2- 172.21.15.101:0/3645830046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4eec100ca0 0x7f4eec101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.039+0000 7f4ef17ff700 1 --2- 172.21.15.101:0/3645830046 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4eec10a070 0x7f4eec10c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.039+0000 7f4ef17ff700 1 --2- 172.21.15.101:0/3645830046 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4eec1015b0 0x7f4eec109aa0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.039+0000 7f4ef17ff700 1 -- 172.21.15.101:0/3645830046 >> 172.21.15.101:0/3645830046 conn(0x7f4eec0fa4b0 msgr2=0x7f4eec0fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:23.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.039+0000 7f4ef17ff700 1 -- 172.21.15.101:0/3645830046 shutdown_connections 2023-07-18T03:39:23.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.039+0000 7f4ef17ff700 1 -- 172.21.15.101:0/3645830046 wait complete. 2023-07-18T03:39:23.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.040+0000 7f4ef17ff700 1 Processor -- start 2023-07-18T03:39:23.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.040+0000 7f4ef17ff700 1 -- start start 2023-07-18T03:39:23.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.040+0000 7f4ef17ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4eec100ca0 0x7f4eec197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:23.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.040+0000 7f4ef17ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4eec1015b0 0x7f4eec1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:23.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.040+0000 7f4ef17ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4eec10a070 0x7f4eec19c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:23.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.040+0000 7f4ef17ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec19ce80 con 0x7f4eec10a070 2023-07-18T03:39:23.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.040+0000 7f4ef17ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec19cff0 con 0x7f4eec100ca0 2023-07-18T03:39:23.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.040+0000 7f4ef17ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec19d1b0 con 0x7f4eec1015b0 2023-07-18T03:39:23.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.040+0000 7f4eeaffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4eec100ca0 0x7f4eec197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:23.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.040+0000 7f4eeaffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4eec100ca0 0x7f4eec197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47364/0 (socket says 172.21.15.101:47364) 2023-07-18T03:39:23.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.040+0000 7f4eeaffd700 1 -- 172.21.15.101:0/568018602 learned_addr learned my addr 172.21.15.101:0/568018602 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:39:23.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.041+0000 7f4eea7fc700 1 --2- 172.21.15.101:0/568018602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4eec1015b0 0x7f4eec1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:23.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.041+0000 7f4eeb7fe700 1 --2- 172.21.15.101:0/568018602 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4eec10a070 0x7f4eec19c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:23.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.041+0000 7f4eeaffd700 1 -- 172.21.15.101:0/568018602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4eec1015b0 msgr2=0x7f4eec1983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:23.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.041+0000 7f4eeaffd700 1 --2- 172.21.15.101:0/568018602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4eec1015b0 0x7f4eec1983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.041+0000 7f4eeaffd700 1 -- 172.21.15.101:0/568018602 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4eec10a070 msgr2=0x7f4eec19c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:23.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.041+0000 7f4eeaffd700 1 --2- 172.21.15.101:0/568018602 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4eec10a070 0x7f4eec19c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.041+0000 7f4eeaffd700 1 -- 172.21.15.101:0/568018602 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee00095f0 con 0x7f4eec100ca0 2023-07-18T03:39:23.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.041+0000 7f4eea7fc700 1 --2- 172.21.15.101:0/568018602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4eec1015b0 0x7f4eec1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:23.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.041+0000 7f4eeb7fe700 1 --2- 172.21.15.101:0/568018602 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4eec10a070 0x7f4eec19c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:23.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.041+0000 7f4eeaffd700 1 --2- 172.21.15.101:0/568018602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4eec100ca0 0x7f4eec197ea0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f4edc00e9a0 tx=0x7f4edc00ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:23.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.041+0000 7f4ed3fff700 1 -- 172.21.15.101:0/568018602 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4edc009790 con 0x7f4eec100ca0 2023-07-18T03:39:23.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.041+0000 7f4ed3fff700 1 -- 172.21.15.101:0/568018602 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4edc00cbd0 con 0x7f4eec100ca0 2023-07-18T03:39:23.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.041+0000 7f4ef17ff700 1 -- 172.21.15.101:0/568018602 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eec19d490 con 0x7f4eec100ca0 2023-07-18T03:39:23.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.041+0000 7f4ed3fff700 1 -- 172.21.15.101:0/568018602 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4edc018680 con 0x7f4eec100ca0 2023-07-18T03:39:23.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.042+0000 7f4ef17ff700 1 -- 172.21.15.101:0/568018602 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eec1a4cd0 con 0x7f4eec100ca0 2023-07-18T03:39:23.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.043+0000 7f4ef17ff700 1 -- 172.21.15.101:0/568018602 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eec067290 con 0x7f4eec100ca0 2023-07-18T03:39:23.049 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:39:22 smithi155 ceph-mon[128885]: pgmap v1009: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 307 B/s wr, 6 op/s 2023-07-18T03:39:23.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.045+0000 7f4ed3fff700 1 -- 172.21.15.101:0/568018602 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f4edc00cd40 con 0x7f4eec100ca0 2023-07-18T03:39:23.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.046+0000 7f4ed3fff700 1 --2- 172.21.15.101:0/568018602 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4ed40a12c0 0x7f4ed40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:23.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.046+0000 7f4ed3fff700 1 -- 172.21.15.101:0/568018602 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f4edc014070 con 0x7f4eec100ca0 2023-07-18T03:39:23.050 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.046+0000 7f4eea7fc700 1 --2- 172.21.15.101:0/568018602 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4ed40a12c0 0x7f4ed40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:23.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.047+0000 7f4eea7fc700 1 --2- 172.21.15.101:0/568018602 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4ed40a12c0 0x7f4ed40a3770 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0006010 tx=0x7f4ee000b260 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:23.051 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.051+0000 7f4ed3fff700 1 -- 172.21.15.101:0/568018602 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4edc051790 con 0x7f4eec100ca0 2023-07-18T03:39:23.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.282+0000 7f4ef17ff700 1 -- 172.21.15.101:0/568018602 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4eec1a4e90 con 0x7f4eec100ca0 2023-07-18T03:39:23.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.282+0000 7f4ed3fff700 1 -- 172.21.15.101:0/568018602 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v144) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4edc099f40 con 0x7f4eec100ca0 2023-07-18T03:39:23.278 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 1 2023-07-18T03:39:23.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.285+0000 7f4ef17ff700 1 -- 172.21.15.101:0/568018602 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4ed40a12c0 msgr2=0x7f4ed40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:23.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.285+0000 7f4ef17ff700 1 --2- 172.21.15.101:0/568018602 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4ed40a12c0 0x7f4ed40a3770 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0006010 tx=0x7f4ee000b260 comp rx=0 tx=0).stop 2023-07-18T03:39:23.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.285+0000 7f4ef17ff700 1 -- 172.21.15.101:0/568018602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4eec100ca0 msgr2=0x7f4eec197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:23.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.285+0000 7f4ef17ff700 1 --2- 172.21.15.101:0/568018602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4eec100ca0 0x7f4eec197ea0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f4edc00e9a0 tx=0x7f4edc00ed60 comp rx=0 tx=0).stop 2023-07-18T03:39:23.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.285+0000 7f4ef17ff700 1 -- 172.21.15.101:0/568018602 shutdown_connections 2023-07-18T03:39:23.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.285+0000 7f4ef17ff700 1 --2- 172.21.15.101:0/568018602 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f4ed40a12c0 0x7f4ed40a3770 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.285+0000 7f4ef17ff700 1 --2- 172.21.15.101:0/568018602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4eec100ca0 0x7f4eec197ea0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.285+0000 7f4ef17ff700 1 --2- 172.21.15.101:0/568018602 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4eec10a070 0x7f4eec19c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.285+0000 7f4ef17ff700 1 --2- 172.21.15.101:0/568018602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4eec1015b0 0x7f4eec1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.285+0000 7f4ef17ff700 1 -- 172.21.15.101:0/568018602 >> 172.21.15.101:0/568018602 conn(0x7f4eec0fa4b0 msgr2=0x7f4eec10c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:23.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.286+0000 7f4ef17ff700 1 -- 172.21.15.101:0/568018602 shutdown_connections 2023-07-18T03:39:23.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.286+0000 7f4ef17ff700 1 -- 172.21.15.101:0/568018602 wait complete. 2023-07-18T03:39:23.300 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-18T03:39:23.522 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.527+0000 7fb20f9ff700 1 -- 172.21.15.101:0/4059089596 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb2080ffad0 msgr2=0x7fb2080fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:23.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.527+0000 7fb20f9ff700 1 --2- 172.21.15.101:0/4059089596 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb2080ffad0 0x7fb2080fff40 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fb1f80098f0 tx=0x7fb1f8009c00 comp rx=0 tx=0).stop 2023-07-18T03:39:23.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.527+0000 7fb20f9ff700 1 -- 172.21.15.101:0/4059089596 shutdown_connections 2023-07-18T03:39:23.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.527+0000 7fb20f9ff700 1 --2- 172.21.15.101:0/4059089596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb208100480 0x7fb20810c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.527+0000 7fb20f9ff700 1 --2- 172.21.15.101:0/4059089596 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb208105ae0 0x7fb208105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.523 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.527+0000 7fb20f9ff700 1 --2- 172.21.15.101:0/4059089596 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb2080ffad0 0x7fb2080fff40 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.524 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.527+0000 7fb20f9ff700 1 -- 172.21.15.101:0/4059089596 >> 172.21.15.101:0/4059089596 conn(0x7fb2080fb5f0 msgr2=0x7fb2080fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:23.524 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.527+0000 7fb20f9ff700 1 -- 172.21.15.101:0/4059089596 shutdown_connections 2023-07-18T03:39:23.524 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.527+0000 7fb20f9ff700 1 -- 172.21.15.101:0/4059089596 wait complete. 2023-07-18T03:39:23.526 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.528+0000 7fb20f9ff700 1 Processor -- start 2023-07-18T03:39:23.526 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.528+0000 7fb20f9ff700 1 -- start start 2023-07-18T03:39:23.526 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.529+0000 7fb20f9ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2080ffad0 0x7fb20819ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:23.526 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.529+0000 7fb20f9ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb208100480 0x7fb20819bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:23.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.529+0000 7fb20f9ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb208105ae0 0x7fb208195aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:23.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.529+0000 7fb20f9ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb208196110 con 0x7fb208100480 2023-07-18T03:39:23.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.529+0000 7fb20f9ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb208196280 con 0x7fb208105ae0 2023-07-18T03:39:23.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.529+0000 7fb20f9ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb208196440 con 0x7fb2080ffad0 2023-07-18T03:39:23.527 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.529+0000 7fb20d79b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2080ffad0 0x7fb20819ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:23.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.529+0000 7fb20d79b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2080ffad0 0x7fb20819ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:59334/0 (socket says 172.21.15.101:59334) 2023-07-18T03:39:23.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.529+0000 7fb20d79b700 1 -- 172.21.15.101:0/2592571432 learned_addr learned my addr 172.21.15.101:0/2592571432 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:39:23.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.529+0000 7fb20cf9a700 1 --2- 172.21.15.101:0/2592571432 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb208100480 0x7fb20819bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:23.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.529+0000 7fb20df9c700 1 --2- 172.21.15.101:0/2592571432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb208105ae0 0x7fb208195aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:23.528 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.530+0000 7fb20d79b700 1 -- 172.21.15.101:0/2592571432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb208105ae0 msgr2=0x7fb208195aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:23.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.530+0000 7fb20d79b700 1 --2- 172.21.15.101:0/2592571432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb208105ae0 0x7fb208195aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.530+0000 7fb20d79b700 1 -- 172.21.15.101:0/2592571432 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb208100480 msgr2=0x7fb20819bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:23.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.530+0000 7fb20d79b700 1 --2- 172.21.15.101:0/2592571432 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb208100480 0x7fb20819bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.530+0000 7fb20d79b700 1 -- 172.21.15.101:0/2592571432 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2040095f0 con 0x7fb2080ffad0 2023-07-18T03:39:23.529 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.530+0000 7fb20d79b700 1 --2- 172.21.15.101:0/2592571432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2080ffad0 0x7fb20819ba30 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fb20400ec10 tx=0x7fb20400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:23.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.530+0000 7fb1fe7fc700 1 -- 172.21.15.101:0/2592571432 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20400cc90 con 0x7fb2080ffad0 2023-07-18T03:39:23.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.530+0000 7fb20f9ff700 1 -- 172.21.15.101:0/2592571432 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1f8009520 con 0x7fb2080ffad0 2023-07-18T03:39:23.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.530+0000 7fb1fe7fc700 1 -- 172.21.15.101:0/2592571432 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2040044e0 con 0x7fb2080ffad0 2023-07-18T03:39:23.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.530+0000 7fb1fe7fc700 1 -- 172.21.15.101:0/2592571432 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb204010430 con 0x7fb2080ffad0 2023-07-18T03:39:23.530 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.530+0000 7fb20f9ff700 1 -- 172.21.15.101:0/2592571432 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2081a2840 con 0x7fb2080ffad0 2023-07-18T03:39:23.531 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.532+0000 7fb20f9ff700 1 -- 172.21.15.101:0/2592571432 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb208067290 con 0x7fb2080ffad0 2023-07-18T03:39:23.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.533+0000 7fb1fe7fc700 1 -- 172.21.15.101:0/2592571432 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fb204003fb0 con 0x7fb2080ffad0 2023-07-18T03:39:23.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.535+0000 7fb1fe7fc700 1 --2- 172.21.15.101:0/2592571432 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fb1f40a12c0 0x7fb1f40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:23.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.535+0000 7fb1fe7fc700 1 -- 172.21.15.101:0/2592571432 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fb204014070 con 0x7fb2080ffad0 2023-07-18T03:39:23.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.535+0000 7fb20cf9a700 1 --2- 172.21.15.101:0/2592571432 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fb1f40a12c0 0x7fb1f40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:23.532 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.536+0000 7fb20cf9a700 1 --2- 172.21.15.101:0/2592571432 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fb1f40a12c0 0x7fb1f40a3770 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8005e90 tx=0x7fb1f8019040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:23.535 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.540+0000 7fb1fe7fc700 1 -- 172.21.15.101:0/2592571432 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb2040965c0 con 0x7fb2080ffad0 2023-07-18T03:39:23.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.769+0000 7fb20f9ff700 1 -- 172.21.15.101:0/2592571432 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb2081a2bd0 con 0x7fb2080ffad0 2023-07-18T03:39:23.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.770+0000 7fb1fe7fc700 1 -- 172.21.15.101:0/2592571432 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v144) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb204099be0 con 0x7fb2080ffad0 2023-07-18T03:39:23.765 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 32 2023-07-18T03:39:23.767 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.772+0000 7fb20f9ff700 1 -- 172.21.15.101:0/2592571432 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fb1f40a12c0 msgr2=0x7fb1f40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:23.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.772+0000 7fb20f9ff700 1 --2- 172.21.15.101:0/2592571432 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fb1f40a12c0 0x7fb1f40a3770 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8005e90 tx=0x7fb1f8019040 comp rx=0 tx=0).stop 2023-07-18T03:39:23.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.772+0000 7fb20f9ff700 1 -- 172.21.15.101:0/2592571432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2080ffad0 msgr2=0x7fb20819ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:23.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.772+0000 7fb20f9ff700 1 --2- 172.21.15.101:0/2592571432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2080ffad0 0x7fb20819ba30 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fb20400ec10 tx=0x7fb20400ef20 comp rx=0 tx=0).stop 2023-07-18T03:39:23.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.772+0000 7fb20f9ff700 1 -- 172.21.15.101:0/2592571432 shutdown_connections 2023-07-18T03:39:23.768 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.772+0000 7fb20f9ff700 1 --2- 172.21.15.101:0/2592571432 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fb1f40a12c0 0x7fb1f40a3770 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.769 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.772+0000 7fb20f9ff700 1 --2- 172.21.15.101:0/2592571432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb208105ae0 0x7fb208195aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.769 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.772+0000 7fb20f9ff700 1 --2- 172.21.15.101:0/2592571432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2080ffad0 0x7fb20819ba30 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.769 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.772+0000 7fb20f9ff700 1 --2- 172.21.15.101:0/2592571432 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb208100480 0x7fb20819bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.769 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.772+0000 7fb20f9ff700 1 -- 172.21.15.101:0/2592571432 >> 172.21.15.101:0/2592571432 conn(0x7fb2080fb5f0 msgr2=0x7fb2081035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:23.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.773+0000 7fb20f9ff700 1 -- 172.21.15.101:0/2592571432 shutdown_connections 2023-07-18T03:39:23.770 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:23.773+0000 7fb20f9ff700 1 -- 172.21.15.101:0/2592571432 wait complete. 2023-07-18T03:39:23.788 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-18T03:39:23.847 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:39:23 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/4240987869' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:39:23.848 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:39:23 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/568018602' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:39:23.951 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:39:23 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/4240987869' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:39:23.951 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:39:23 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/568018602' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:39:24.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.013+0000 7f68cd9ff700 1 -- 172.21.15.101:0/494203194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68c80728b0 msgr2=0x7f68c8072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:24.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.013+0000 7f68cd9ff700 1 --2- 172.21.15.101:0/494203194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68c80728b0 0x7f68c8072c80 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f68b8009990 tx=0x7f68b8009ca0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.013+0000 7f68cd9ff700 1 -- 172.21.15.101:0/494203194 shutdown_connections 2023-07-18T03:39:24.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.013+0000 7f68cd9ff700 1 --2- 172.21.15.101:0/494203194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68c80728b0 0x7f68c8072c80 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.013+0000 7f68cd9ff700 1 --2- 172.21.15.101:0/494203194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68c810c320 0x7f68c810e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.013+0000 7f68cd9ff700 1 --2- 172.21.15.101:0/494203194 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f68c80731c0 0x7f68c810bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.013+0000 7f68cd9ff700 1 -- 172.21.15.101:0/494203194 >> 172.21.15.101:0/494203194 conn(0x7f68c80fb5e0 msgr2=0x7f68c80fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:24.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.013+0000 7f68cd9ff700 1 -- 172.21.15.101:0/494203194 shutdown_connections 2023-07-18T03:39:24.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.013+0000 7f68cd9ff700 1 -- 172.21.15.101:0/494203194 wait complete. 2023-07-18T03:39:24.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.014+0000 7f68cd9ff700 1 Processor -- start 2023-07-18T03:39:24.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.014+0000 7f68cd9ff700 1 -- start start 2023-07-18T03:39:24.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.014+0000 7f68cd9ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68c80728b0 0x7f68c8197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:24.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.015+0000 7f68cd9ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f68c80731c0 0x7f68c8198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:24.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.015+0000 7f68cd9ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68c810c320 0x7f68c819c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:24.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.015+0000 7f68cd9ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c819ce30 con 0x7f68c80731c0 2023-07-18T03:39:24.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.015+0000 7f68cd9ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c819cfa0 con 0x7f68c810c320 2023-07-18T03:39:24.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.015+0000 7f68cd9ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c819d160 con 0x7f68c80728b0 2023-07-18T03:39:24.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.015+0000 7f68c6ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68c80728b0 0x7f68c8197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:24.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.015+0000 7f68c77fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68c810c320 0x7f68c819c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:24.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.015+0000 7f68c67fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f68c80731c0 0x7f68c8198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:24.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.015+0000 7f68c6ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68c80728b0 0x7f68c8197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:59352/0 (socket says 172.21.15.101:59352) 2023-07-18T03:39:24.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.015+0000 7f68c6ffd700 1 -- 172.21.15.101:0/4029923137 learned_addr learned my addr 172.21.15.101:0/4029923137 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:39:24.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.015+0000 7f68c6ffd700 1 -- 172.21.15.101:0/4029923137 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68c810c320 msgr2=0x7f68c819c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:24.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.015+0000 7f68c6ffd700 1 --2- 172.21.15.101:0/4029923137 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68c810c320 0x7f68c819c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.015 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.015+0000 7f68c6ffd700 1 -- 172.21.15.101:0/4029923137 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f68c80731c0 msgr2=0x7f68c8198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:24.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.015+0000 7f68c6ffd700 1 --2- 172.21.15.101:0/4029923137 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f68c80731c0 0x7f68c8198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.015+0000 7f68c6ffd700 1 -- 172.21.15.101:0/4029923137 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b80095f0 con 0x7f68c80728b0 2023-07-18T03:39:24.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.015+0000 7f68c77fe700 1 --2- 172.21.15.101:0/4029923137 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68c810c320 0x7f68c819c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:39:24.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.016+0000 7f68c6ffd700 1 --2- 172.21.15.101:0/4029923137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68c80728b0 0x7f68c8197e50 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f68b8000c00 tx=0x7f68b8004790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:24.016 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.016+0000 7f68cc9fd700 1 -- 172.21.15.101:0/4029923137 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b801d070 con 0x7f68c80728b0 2023-07-18T03:39:24.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.016+0000 7f68cc9fd700 1 -- 172.21.15.101:0/4029923137 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68b8021410 con 0x7f68c80728b0 2023-07-18T03:39:24.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.016+0000 7f68cd9ff700 1 -- 172.21.15.101:0/4029923137 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68c819d3e0 con 0x7f68c80728b0 2023-07-18T03:39:24.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.016+0000 7f68cc9fd700 1 -- 172.21.15.101:0/4029923137 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b8017650 con 0x7f68c80728b0 2023-07-18T03:39:24.017 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.016+0000 7f68cd9ff700 1 -- 172.21.15.101:0/4029923137 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68c81a4c80 con 0x7f68c80728b0 2023-07-18T03:39:24.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.017+0000 7f68cd9ff700 1 -- 172.21.15.101:0/4029923137 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68c8107530 con 0x7f68c80728b0 2023-07-18T03:39:24.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.019+0000 7f68cc9fd700 1 -- 172.21.15.101:0/4029923137 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f68b8021580 con 0x7f68c80728b0 2023-07-18T03:39:24.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.020+0000 7f68cc9fd700 1 --2- 172.21.15.101:0/4029923137 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f68b00a12c0 0x7f68b00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:24.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.020+0000 7f68cc9fd700 1 -- 172.21.15.101:0/4029923137 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f68b80ce3f0 con 0x7f68c80728b0 2023-07-18T03:39:24.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.021+0000 7f68c67fc700 1 --2- 172.21.15.101:0/4029923137 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f68b00a12c0 0x7f68b00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:24.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.021+0000 7f68c67fc700 1 --2- 172.21.15.101:0/4029923137 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f68b00a12c0 0x7f68b00a3770 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f68c8199460 tx=0x7f68bc009260 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:24.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.023+0000 7f68cc9fd700 1 -- 172.21.15.101:0/4029923137 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f68b8097480 con 0x7f68c80728b0 2023-07-18T03:39:24.046 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:39:23 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/4240987869' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:39:24.047 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:39:23 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/568018602' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:39:24.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.243+0000 7f68cd9ff700 1 -- 172.21.15.101:0/4029923137 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f68c8199240 con 0x7f68c80728b0 2023-07-18T03:39:24.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.244+0000 7f68cc9fd700 1 -- 172.21.15.101:0/4029923137 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v144) v1 ==== 100+0+11 (secure 0 0 0) 0x7f68b809aaa0 con 0x7f68c80728b0 2023-07-18T03:39:24.240 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 64 2023-07-18T03:39:24.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.248+0000 7f68cd9ff700 1 -- 172.21.15.101:0/4029923137 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f68b00a12c0 msgr2=0x7f68b00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:24.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.248+0000 7f68cd9ff700 1 --2- 172.21.15.101:0/4029923137 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f68b00a12c0 0x7f68b00a3770 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f68c8199460 tx=0x7f68bc009260 comp rx=0 tx=0).stop 2023-07-18T03:39:24.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.248+0000 7f68cd9ff700 1 -- 172.21.15.101:0/4029923137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68c80728b0 msgr2=0x7f68c8197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:24.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.248+0000 7f68cd9ff700 1 --2- 172.21.15.101:0/4029923137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68c80728b0 0x7f68c8197e50 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f68b8000c00 tx=0x7f68b8004790 comp rx=0 tx=0).stop 2023-07-18T03:39:24.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.248+0000 7f68cd9ff700 1 -- 172.21.15.101:0/4029923137 shutdown_connections 2023-07-18T03:39:24.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.248+0000 7f68cd9ff700 1 --2- 172.21.15.101:0/4029923137 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f68b00a12c0 0x7f68b00a3770 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.248+0000 7f68cd9ff700 1 --2- 172.21.15.101:0/4029923137 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68c810c320 0x7f68c819c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.248+0000 7f68cd9ff700 1 --2- 172.21.15.101:0/4029923137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68c80728b0 0x7f68c8197e50 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.248+0000 7f68cd9ff700 1 --2- 172.21.15.101:0/4029923137 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f68c80731c0 0x7f68c8198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.248+0000 7f68cd9ff700 1 -- 172.21.15.101:0/4029923137 >> 172.21.15.101:0/4029923137 conn(0x7f68c80fb5e0 msgr2=0x7f68c810e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:24.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.248+0000 7f68cd9ff700 1 -- 172.21.15.101:0/4029923137 shutdown_connections 2023-07-18T03:39:24.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.248+0000 7f68cd9ff700 1 -- 172.21.15.101:0/4029923137 wait complete. 2023-07-18T03:39:24.271 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-18T03:39:24.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.488+0000 7fa2229ff700 1 -- 172.21.15.101:0/1405242496 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa21c0731c0 msgr2=0x7fa21c10bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:24.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.488+0000 7fa2229ff700 1 --2- 172.21.15.101:0/1405242496 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa21c0731c0 0x7fa21c10bdd0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa210009990 tx=0x7fa210009ca0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.488+0000 7fa2229ff700 1 -- 172.21.15.101:0/1405242496 shutdown_connections 2023-07-18T03:39:24.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.488+0000 7fa2229ff700 1 --2- 172.21.15.101:0/1405242496 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa21c10c310 0x7fa21c10e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.488+0000 7fa2229ff700 1 --2- 172.21.15.101:0/1405242496 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa21c0728b0 0x7fa21c072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.488+0000 7fa2229ff700 1 --2- 172.21.15.101:0/1405242496 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa21c0731c0 0x7fa21c10bdd0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.488+0000 7fa2229ff700 1 -- 172.21.15.101:0/1405242496 >> 172.21.15.101:0/1405242496 conn(0x7fa21c0fb5f0 msgr2=0x7fa21c0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:24.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.488+0000 7fa2229ff700 1 -- 172.21.15.101:0/1405242496 shutdown_connections 2023-07-18T03:39:24.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.488+0000 7fa2229ff700 1 -- 172.21.15.101:0/1405242496 wait complete. 2023-07-18T03:39:24.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.489+0000 7fa2229ff700 1 Processor -- start 2023-07-18T03:39:24.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.490+0000 7fa2229ff700 1 -- start start 2023-07-18T03:39:24.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.490+0000 7fa2229ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa21c0728b0 0x7fa21c197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:24.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.490+0000 7fa2229ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa21c0731c0 0x7fa21c198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:24.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.490+0000 7fa2229ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa21c10c310 0x7fa21c19c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:24.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.490+0000 7fa2229ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c19cdd0 con 0x7fa21c10c310 2023-07-18T03:39:24.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.490+0000 7fa2229ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c19cf40 con 0x7fa21c0731c0 2023-07-18T03:39:24.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.490+0000 7fa2229ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c19d100 con 0x7fa21c0728b0 2023-07-18T03:39:24.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.490+0000 7fa21bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa21c0728b0 0x7fa21c197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:24.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.490+0000 7fa21b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa21c0731c0 0x7fa21c198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:24.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.490+0000 7fa21bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa21c0728b0 0x7fa21c197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:59380/0 (socket says 172.21.15.101:59380) 2023-07-18T03:39:24.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.490+0000 7fa21bfff700 1 -- 172.21.15.101:0/2998272586 learned_addr learned my addr 172.21.15.101:0/2998272586 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:39:24.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.490+0000 7fa21b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa21c0731c0 0x7fa21c198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47396/0 (socket says 172.21.15.101:47396) 2023-07-18T03:39:24.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.490+0000 7fa220f9c700 1 --2- 172.21.15.101:0/2998272586 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa21c10c310 0x7fa21c19c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:24.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.491+0000 7fa21bfff700 1 -- 172.21.15.101:0/2998272586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa21c0731c0 msgr2=0x7fa21c198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:24.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.491+0000 7fa21bfff700 1 --2- 172.21.15.101:0/2998272586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa21c0731c0 0x7fa21c198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.491+0000 7fa21bfff700 1 -- 172.21.15.101:0/2998272586 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa21c10c310 msgr2=0x7fa21c19c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:24.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.491+0000 7fa21bfff700 1 --2- 172.21.15.101:0/2998272586 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa21c10c310 0x7fa21c19c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.491+0000 7fa21bfff700 1 -- 172.21.15.101:0/2998272586 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2100095f0 con 0x7fa21c0728b0 2023-07-18T03:39:24.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.491+0000 7fa21bfff700 1 --2- 172.21.15.101:0/2998272586 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa21c0728b0 0x7fa21c197df0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa20c00e9a0 tx=0x7fa20c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:24.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.491+0000 7fa2197fa700 1 -- 172.21.15.101:0/2998272586 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c009790 con 0x7fa21c0728b0 2023-07-18T03:39:24.491 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.491+0000 7fa2197fa700 1 -- 172.21.15.101:0/2998272586 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa20c010430 con 0x7fa21c0728b0 2023-07-18T03:39:24.492 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.491+0000 7fa2229ff700 1 -- 172.21.15.101:0/2998272586 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa21c19d3e0 con 0x7fa21c0728b0 2023-07-18T03:39:24.492 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.491+0000 7fa2197fa700 1 -- 172.21.15.101:0/2998272586 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c018680 con 0x7fa21c0728b0 2023-07-18T03:39:24.492 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.491+0000 7fa2229ff700 1 -- 172.21.15.101:0/2998272586 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa21c1a4c20 con 0x7fa21c0728b0 2023-07-18T03:39:24.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.493+0000 7fa2229ff700 1 -- 172.21.15.101:0/2998272586 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa21c106c30 con 0x7fa21c0728b0 2023-07-18T03:39:24.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.496+0000 7fa2197fa700 1 -- 172.21.15.101:0/2998272586 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fa20c0105a0 con 0x7fa21c0728b0 2023-07-18T03:39:24.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.497+0000 7fa2197fa700 1 --2- 172.21.15.101:0/2998272586 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa2040a1340 0x7fa2040a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:24.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.497+0000 7fa2197fa700 1 -- 172.21.15.101:0/2998272586 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fa20c014070 con 0x7fa21c0728b0 2023-07-18T03:39:24.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.497+0000 7fa21b7fe700 1 --2- 172.21.15.101:0/2998272586 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa2040a1340 0x7fa2040a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:24.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.498+0000 7fa21b7fe700 1 --2- 172.21.15.101:0/2998272586 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa2040a1340 0x7fa2040a37f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa21c199400 tx=0x7fa210005fb0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:24.496 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.501+0000 7fa2197fa700 1 -- 172.21.15.101:0/2998272586 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa20c051790 con 0x7fa21c0728b0 2023-07-18T03:39:24.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.713+0000 7fa2229ff700 1 -- 172.21.15.101:0/2998272586 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa21c106610 con 0x7fa21c0728b0 2023-07-18T03:39:24.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.713+0000 7fa2197fa700 1 -- 172.21.15.101:0/2998272586 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v144) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa20c096280 con 0x7fa21c0728b0 2023-07-18T03:39:24.709 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 32 2023-07-18T03:39:24.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.716+0000 7fa2229ff700 1 -- 172.21.15.101:0/2998272586 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa2040a1340 msgr2=0x7fa2040a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:24.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.716+0000 7fa2229ff700 1 --2- 172.21.15.101:0/2998272586 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa2040a1340 0x7fa2040a37f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa21c199400 tx=0x7fa210005fb0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.716+0000 7fa2229ff700 1 -- 172.21.15.101:0/2998272586 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa21c0728b0 msgr2=0x7fa21c197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:24.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.716+0000 7fa2229ff700 1 --2- 172.21.15.101:0/2998272586 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa21c0728b0 0x7fa21c197df0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa20c00e9a0 tx=0x7fa20c00ecb0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.716+0000 7fa2229ff700 1 -- 172.21.15.101:0/2998272586 shutdown_connections 2023-07-18T03:39:24.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.716+0000 7fa2229ff700 1 --2- 172.21.15.101:0/2998272586 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa21c10c310 0x7fa21c19c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.716+0000 7fa2229ff700 1 --2- 172.21.15.101:0/2998272586 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa21c0728b0 0x7fa21c197df0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.716+0000 7fa2229ff700 1 --2- 172.21.15.101:0/2998272586 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa2040a1340 0x7fa2040a37f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.716+0000 7fa2229ff700 1 --2- 172.21.15.101:0/2998272586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa21c0731c0 0x7fa21c198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.716+0000 7fa2229ff700 1 -- 172.21.15.101:0/2998272586 >> 172.21.15.101:0/2998272586 conn(0x7fa21c0fb5f0 msgr2=0x7fa21c103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:24.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.716+0000 7fa2229ff700 1 -- 172.21.15.101:0/2998272586 shutdown_connections 2023-07-18T03:39:24.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.716+0000 7fa2229ff700 1 -- 172.21.15.101:0/2998272586 wait complete. 2023-07-18T03:39:24.731 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-07-18T03:39:24.792 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:39:24 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2592571432' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:39:24.792 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:39:24 smithi101 ceph-mon[126379]: pgmap v1010: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 307 B/s wr, 6 op/s 2023-07-18T03:39:24.792 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:39:24 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/4029923137' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:39:24.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:39:24 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/2592571432' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:39:24.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:39:24 smithi179 ceph-mon[129229]: pgmap v1010: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 307 B/s wr, 6 op/s 2023-07-18T03:39:24.953 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:39:24 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/4029923137' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:39:24.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.960+0000 7fa148fff700 1 -- 172.21.15.101:0/1244014399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1440ff190 msgr2=0x7fa14410bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:24.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.960+0000 7fa148fff700 1 --2- 172.21.15.101:0/1244014399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1440ff190 0x7fa14410bef0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fa138009990 tx=0x7fa138009ca0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.960+0000 7fa148fff700 1 -- 172.21.15.101:0/1244014399 shutdown_connections 2023-07-18T03:39:24.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.960+0000 7fa148fff700 1 --2- 172.21.15.101:0/1244014399 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa14410c430 0x7fa14410e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.960+0000 7fa148fff700 1 --2- 172.21.15.101:0/1244014399 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1440fe880 0x7fa1440fec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.960+0000 7fa148fff700 1 --2- 172.21.15.101:0/1244014399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1440ff190 0x7fa14410bef0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.960+0000 7fa148fff700 1 -- 172.21.15.101:0/1244014399 >> 172.21.15.101:0/1244014399 conn(0x7fa1440fa490 msgr2=0x7fa1440fc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:24.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.960+0000 7fa148fff700 1 -- 172.21.15.101:0/1244014399 shutdown_connections 2023-07-18T03:39:24.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.960+0000 7fa148fff700 1 -- 172.21.15.101:0/1244014399 wait complete. 2023-07-18T03:39:24.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.961+0000 7fa148fff700 1 Processor -- start 2023-07-18T03:39:24.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.961+0000 7fa148fff700 1 -- start start 2023-07-18T03:39:24.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.962+0000 7fa148fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa1440fe880 0x7fa144197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:24.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.962+0000 7fa148fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1440ff190 0x7fa144198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:24.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.962+0000 7fa148fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa14410c430 0x7fa14419c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:24.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.962+0000 7fa148fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14419cde0 con 0x7fa1440fe880 2023-07-18T03:39:24.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.962+0000 7fa148fff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14419cf50 con 0x7fa1440ff190 2023-07-18T03:39:24.970 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.962+0000 7fa148fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14419d110 con 0x7fa14410c430 2023-07-18T03:39:24.970 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.962+0000 7fa141d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1440ff190 0x7fa144198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:24.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.962+0000 7fa141d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1440ff190 0x7fa144198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:47424/0 (socket says 172.21.15.101:47424) 2023-07-18T03:39:24.971 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.962+0000 7fa141d9b700 1 -- 172.21.15.101:0/511151978 learned_addr learned my addr 172.21.15.101:0/511151978 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:39:24.972 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.962+0000 7fa14259c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa1440fe880 0x7fa144197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:24.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.962+0000 7fa142d9d700 1 --2- 172.21.15.101:0/511151978 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa14410c430 0x7fa14419c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:24.973 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.962+0000 7fa141d9b700 1 -- 172.21.15.101:0/511151978 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa14410c430 msgr2=0x7fa14419c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:24.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.962+0000 7fa141d9b700 1 --2- 172.21.15.101:0/511151978 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa14410c430 0x7fa14419c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.974 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.962+0000 7fa141d9b700 1 -- 172.21.15.101:0/511151978 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa1440fe880 msgr2=0x7fa144197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:24.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.962+0000 7fa141d9b700 1 --2- 172.21.15.101:0/511151978 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa1440fe880 0x7fa144197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:24.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.962+0000 7fa141d9b700 1 -- 172.21.15.101:0/511151978 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1380095f0 con 0x7fa1440ff190 2023-07-18T03:39:24.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.962+0000 7fa141d9b700 1 --2- 172.21.15.101:0/511151978 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1440ff190 0x7fa144198340 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa138005910 tx=0x7fa138004710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:24.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.963+0000 7fa1337fe700 1 -- 172.21.15.101:0/511151978 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13801d070 con 0x7fa1440ff190 2023-07-18T03:39:24.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.963+0000 7fa1337fe700 1 -- 172.21.15.101:0/511151978 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa138022470 con 0x7fa1440ff190 2023-07-18T03:39:24.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.963+0000 7fa148fff700 1 -- 172.21.15.101:0/511151978 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa14419d390 con 0x7fa1440ff190 2023-07-18T03:39:24.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.963+0000 7fa1337fe700 1 -- 172.21.15.101:0/511151978 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa138017650 con 0x7fa1440ff190 2023-07-18T03:39:24.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.963+0000 7fa148fff700 1 -- 172.21.15.101:0/511151978 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1441a4c30 con 0x7fa1440ff190 2023-07-18T03:39:24.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.964+0000 7fa148fff700 1 -- 172.21.15.101:0/511151978 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa144067300 con 0x7fa1440ff190 2023-07-18T03:39:24.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.966+0000 7fa1337fe700 1 -- 172.21.15.101:0/511151978 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fa1380225e0 con 0x7fa1440ff190 2023-07-18T03:39:24.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.967+0000 7fa1337fe700 1 --2- 172.21.15.101:0/511151978 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa12c0a1380 0x7fa12c0a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:24.978 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.967+0000 7fa1337fe700 1 -- 172.21.15.101:0/511151978 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fa1380ce470 con 0x7fa1440ff190 2023-07-18T03:39:24.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.968+0000 7fa14259c700 1 --2- 172.21.15.101:0/511151978 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa12c0a1380 0x7fa12c0a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:24.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.968+0000 7fa14259c700 1 --2- 172.21.15.101:0/511151978 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa12c0a1380 0x7fa12c0a3830 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa134009c60 tx=0x7fa134009260 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:24.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:24.973+0000 7fa1337fe700 1 -- 172.21.15.101:0/511151978 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa138097500 con 0x7fa1440ff190 2023-07-18T03:39:25.046 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:39:24 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/2592571432' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:39:25.047 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:39:24 smithi155 ceph-mon[128885]: pgmap v1010: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 307 B/s wr, 6 op/s 2023-07-18T03:39:25.047 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:39:24 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/4029923137' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:39:25.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:25.234+0000 7fa148fff700 1 -- 172.21.15.101:0/511151978 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7fa1441a4dc0 con 0x7fa1440ff190 2023-07-18T03:39:25.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:25.243+0000 7fa1337fe700 1 -- 172.21.15.101:0/511151978 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) v1 ==== 81+0+733783 (secure 0 0 0) 0x7fa13809ab20 con 0x7fa1440ff190 2023-07-18T03:39:25.239 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:39:25.240 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:39:25.240 INFO:teuthology.orchestra.run.smithi101.stdout: "epoch": 25, 2023-07-18T03:39:25.240 INFO:teuthology.orchestra.run.smithi101.stdout: "active_gid": 15486, 2023-07-18T03:39:25.240 INFO:teuthology.orchestra.run.smithi101.stdout: "active_name": "x", 2023-07-18T03:39:25.241 INFO:teuthology.orchestra.run.smithi101.stdout: "active_addrs": { 2023-07-18T03:39:25.241 INFO:teuthology.orchestra.run.smithi101.stdout: "addrvec": [ 2023-07-18T03:39:25.241 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:25.241 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "v2", 2023-07-18T03:39:25.241 INFO:teuthology.orchestra.run.smithi101.stdout: "addr": "172.21.15.101:6800", 2023-07-18T03:39:25.242 INFO:teuthology.orchestra.run.smithi101.stdout: "nonce": 486424816 2023-07-18T03:39:25.242 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.242 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:25.242 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "v1", 2023-07-18T03:39:25.242 INFO:teuthology.orchestra.run.smithi101.stdout: "addr": "172.21.15.101:6801", 2023-07-18T03:39:25.243 INFO:teuthology.orchestra.run.smithi101.stdout: "nonce": 486424816 2023-07-18T03:39:25.243 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:25.243 INFO:teuthology.orchestra.run.smithi101.stdout: ] 2023-07-18T03:39:25.243 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.243 INFO:teuthology.orchestra.run.smithi101.stdout: "active_addr": "172.21.15.101:6801/486424816", 2023-07-18T03:39:25.244 INFO:teuthology.orchestra.run.smithi101.stdout: "active_change": "2023-07-18T03:07:33.498829+0000", 2023-07-18T03:39:25.244 INFO:teuthology.orchestra.run.smithi101.stdout: "active_mgr_features": 4540138322906710015, 2023-07-18T03:39:25.244 INFO:teuthology.orchestra.run.smithi101.stdout: "available": true, 2023-07-18T03:39:25.244 INFO:teuthology.orchestra.run.smithi101.stdout: "standbys": [ 2023-07-18T03:39:25.244 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:25.245 INFO:teuthology.orchestra.run.smithi101.stdout: "gid": 25505, 2023-07-18T03:39:25.245 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "y", 2023-07-18T03:39:25.245 INFO:teuthology.orchestra.run.smithi101.stdout: "mgr_features": 4540138322906710015, 2023-07-18T03:39:25.245 INFO:teuthology.orchestra.run.smithi101.stdout: "available_modules": [ 2023-07-18T03:39:25.246 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:25.246 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "alerts", 2023-07-18T03:39:25.246 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:25.246 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:25.247 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:25.247 INFO:teuthology.orchestra.run.smithi101.stdout: "interval": { 2023-07-18T03:39:25.247 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "interval", 2023-07-18T03:39:25.247 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:25.247 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.248 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.248 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "60", 2023-07-18T03:39:25.248 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.248 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.249 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.249 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "How frequently to reexamine health status", 2023-07-18T03:39:25.249 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.250 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.250 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.250 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.251 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:25.251 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:25.251 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.251 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.251 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.252 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.252 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.252 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.252 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:25.253 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:25.253 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:25.253 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:25.253 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:25.253 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:25.254 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:25.254 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:25.254 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.254 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.254 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.255 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.255 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.255 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:25.255 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:25.256 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.256 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.256 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.257 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.257 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.257 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.257 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.257 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.258 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.258 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.258 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.258 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.258 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:25.259 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:25.259 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.259 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.259 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.260 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:25.260 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.260 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.260 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:25.261 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:25.261 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:25.261 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:25.262 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:25.262 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:25.262 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:25.262 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:25.262 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.263 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.263 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.263 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.263 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.263 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:25.264 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:25.264 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.264 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.265 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.265 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.265 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.265 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.266 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.266 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.266 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.266 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.267 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.267 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.267 INFO:teuthology.orchestra.run.smithi101.stdout: "smtp_destination": { 2023-07-18T03:39:25.267 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "smtp_destination", 2023-07-18T03:39:25.268 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.268 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.268 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.268 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.269 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.269 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.269 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.269 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Email address to send alerts to", 2023-07-18T03:39:25.270 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.270 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.270 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.270 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.270 INFO:teuthology.orchestra.run.smithi101.stdout: "smtp_from_name": { 2023-07-18T03:39:25.271 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "smtp_from_name", 2023-07-18T03:39:25.271 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.271 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.271 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.271 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "Ceph", 2023-07-18T03:39:25.272 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.272 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.272 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.272 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Email From: name", 2023-07-18T03:39:25.273 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.273 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.274 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.274 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.274 INFO:teuthology.orchestra.run.smithi101.stdout: "smtp_host": { 2023-07-18T03:39:25.275 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "smtp_host", 2023-07-18T03:39:25.275 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.275 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.275 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.276 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.276 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.276 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.276 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.276 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "SMTP server", 2023-07-18T03:39:25.277 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.277 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.278 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.278 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.278 INFO:teuthology.orchestra.run.smithi101.stdout: "smtp_password": { 2023-07-18T03:39:25.279 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "smtp_password", 2023-07-18T03:39:25.279 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.279 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.279 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.279 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.280 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.280 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.280 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.280 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Password to authenticate with", 2023-07-18T03:39:25.281 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.281 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.281 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.281 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.281 INFO:teuthology.orchestra.run.smithi101.stdout: "smtp_port": { 2023-07-18T03:39:25.282 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "smtp_port", 2023-07-18T03:39:25.282 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:25.282 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.282 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.283 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "465", 2023-07-18T03:39:25.283 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.283 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.283 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.284 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "SMTP port", 2023-07-18T03:39:25.284 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.284 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.285 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.285 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.285 INFO:teuthology.orchestra.run.smithi101.stdout: "smtp_sender": { 2023-07-18T03:39:25.286 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "smtp_sender", 2023-07-18T03:39:25.286 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.286 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.286 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.286 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.287 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.287 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.287 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.287 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "SMTP envelope sender", 2023-07-18T03:39:25.288 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.288 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.288 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.289 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.289 INFO:teuthology.orchestra.run.smithi101.stdout: "smtp_ssl": { 2023-07-18T03:39:25.289 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "smtp_ssl", 2023-07-18T03:39:25.289 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.289 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.290 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.290 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:25.290 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.290 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.291 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.291 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Use SSL to connect to SMTP server", 2023-07-18T03:39:25.291 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.291 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.291 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.292 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.292 INFO:teuthology.orchestra.run.smithi101.stdout: "smtp_user": { 2023-07-18T03:39:25.292 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "smtp_user", 2023-07-18T03:39:25.292 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.293 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.293 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.293 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.293 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.294 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.294 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.294 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "User to authenticate as", 2023-07-18T03:39:25.294 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.294 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.295 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.295 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:25.295 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:25.295 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.295 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:25.296 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "balancer", 2023-07-18T03:39:25.296 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:25.296 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:25.296 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:25.297 INFO:teuthology.orchestra.run.smithi101.stdout: "active": { 2023-07-18T03:39:25.297 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "active", 2023-07-18T03:39:25.297 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.297 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.298 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.298 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:25.299 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.299 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.300 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.300 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "automatically balance PGs across cluster", 2023-07-18T03:39:25.301 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.301 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.302 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.302 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.302 INFO:teuthology.orchestra.run.smithi101.stdout: "begin_time": { 2023-07-18T03:39:25.303 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "begin_time", 2023-07-18T03:39:25.303 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.303 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.304 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.304 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "0000", 2023-07-18T03:39:25.304 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.305 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.305 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.306 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "beginning time of day to automatically balance", 2023-07-18T03:39:25.306 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-18T03:39:25.306 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.307 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.307 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.307 INFO:teuthology.orchestra.run.smithi101.stdout: "begin_weekday": { 2023-07-18T03:39:25.307 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "begin_weekday", 2023-07-18T03:39:25.308 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "uint", 2023-07-18T03:39:25.308 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.308 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.309 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "0", 2023-07-18T03:39:25.309 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "0", 2023-07-18T03:39:25.309 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "6", 2023-07-18T03:39:25.310 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.310 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-07-18T03:39:25.310 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-18T03:39:25.310 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.311 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.311 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.311 INFO:teuthology.orchestra.run.smithi101.stdout: "crush_compat_max_iterations": { 2023-07-18T03:39:25.311 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "crush_compat_max_iterations", 2023-07-18T03:39:25.311 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "uint", 2023-07-18T03:39:25.312 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.312 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.312 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "25", 2023-07-18T03:39:25.312 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "1", 2023-07-18T03:39:25.313 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "250", 2023-07-18T03:39:25.313 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.313 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-07-18T03:39:25.313 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.314 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.314 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.314 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.314 INFO:teuthology.orchestra.run.smithi101.stdout: "crush_compat_metrics": { 2023-07-18T03:39:25.314 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "crush_compat_metrics", 2023-07-18T03:39:25.315 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.315 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.315 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.315 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "pgs,objects,bytes", 2023-07-18T03:39:25.316 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.316 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.316 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.316 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-07-18T03:39:25.317 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-07-18T03:39:25.317 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.317 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.317 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.317 INFO:teuthology.orchestra.run.smithi101.stdout: "crush_compat_step": { 2023-07-18T03:39:25.318 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "crush_compat_step", 2023-07-18T03:39:25.318 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "float", 2023-07-18T03:39:25.318 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.318 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.319 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "0.5", 2023-07-18T03:39:25.319 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "0.001", 2023-07-18T03:39:25.319 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "0.999", 2023-07-18T03:39:25.320 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.320 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "aggressiveness of optimization", 2023-07-18T03:39:25.320 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-07-18T03:39:25.320 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.321 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.321 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.321 INFO:teuthology.orchestra.run.smithi101.stdout: "end_time": { 2023-07-18T03:39:25.321 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "end_time", 2023-07-18T03:39:25.321 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.322 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.322 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.322 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "2359", 2023-07-18T03:39:25.322 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.323 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.323 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.323 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "ending time of day to automatically balance", 2023-07-18T03:39:25.323 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-18T03:39:25.323 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.324 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.324 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.325 INFO:teuthology.orchestra.run.smithi101.stdout: "end_weekday": { 2023-07-18T03:39:25.325 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "end_weekday", 2023-07-18T03:39:25.325 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "uint", 2023-07-18T03:39:25.326 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.326 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.326 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "0", 2023-07-18T03:39:25.326 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "0", 2023-07-18T03:39:25.327 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "6", 2023-07-18T03:39:25.327 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.327 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-07-18T03:39:25.327 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-18T03:39:25.328 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.328 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.328 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.328 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:25.328 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:25.329 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.329 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.329 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.329 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.330 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.330 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.330 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:25.330 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:25.331 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:25.331 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:25.331 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:25.331 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:25.332 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:25.332 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:25.332 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.333 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.333 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.333 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.333 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.334 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:25.334 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:25.334 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.335 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.335 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.335 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.341 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.341 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.341 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.342 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.342 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.342 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.342 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.342 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.343 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:25.343 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:25.343 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.343 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.343 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.344 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:25.344 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.344 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.344 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:25.345 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:25.345 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:25.345 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:25.345 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:25.345 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:25.346 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:25.346 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:25.346 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.346 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.346 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.347 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.347 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.347 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:25.347 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:25.347 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.348 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.348 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.348 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.348 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.348 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.349 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.349 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.349 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.349 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.349 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.350 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.350 INFO:teuthology.orchestra.run.smithi101.stdout: "min_score": { 2023-07-18T03:39:25.350 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "min_score", 2023-07-18T03:39:25.350 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "float", 2023-07-18T03:39:25.350 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.351 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.351 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "0", 2023-07-18T03:39:25.351 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.351 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.351 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.352 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-07-18T03:39:25.352 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.352 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.352 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.352 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.353 INFO:teuthology.orchestra.run.smithi101.stdout: "mode": { 2023-07-18T03:39:25.353 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "mode", 2023-07-18T03:39:25.353 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.353 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.353 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.354 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "upmap", 2023-07-18T03:39:25.354 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.354 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.354 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:25.354 INFO:teuthology.orchestra.run.smithi101.stdout: "crush-compat", 2023-07-18T03:39:25.355 INFO:teuthology.orchestra.run.smithi101.stdout: "none", 2023-07-18T03:39:25.355 INFO:teuthology.orchestra.run.smithi101.stdout: "upmap" 2023-07-18T03:39:25.355 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:25.355 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Balancer mode", 2023-07-18T03:39:25.355 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.356 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.356 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.356 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.356 INFO:teuthology.orchestra.run.smithi101.stdout: "pool_ids": { 2023-07-18T03:39:25.356 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "pool_ids", 2023-07-18T03:39:25.357 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.357 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.357 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.357 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.357 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.358 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.358 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.358 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-07-18T03:39:25.358 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.358 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.359 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.359 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.359 INFO:teuthology.orchestra.run.smithi101.stdout: "sleep_interval": { 2023-07-18T03:39:25.359 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "sleep_interval", 2023-07-18T03:39:25.359 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:25.360 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.360 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.360 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "60", 2023-07-18T03:39:25.360 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.360 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.361 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.361 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-07-18T03:39:25.361 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.361 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.361 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.362 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.362 INFO:teuthology.orchestra.run.smithi101.stdout: "upmap_max_deviation": { 2023-07-18T03:39:25.362 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "upmap_max_deviation", 2023-07-18T03:39:25.362 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:25.362 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.363 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.363 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "5", 2023-07-18T03:39:25.363 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "1", 2023-07-18T03:39:25.363 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.363 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.364 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "deviation below which no optimization is attempted", 2023-07-18T03:39:25.364 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-07-18T03:39:25.364 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.364 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.364 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.365 INFO:teuthology.orchestra.run.smithi101.stdout: "upmap_max_optimizations": { 2023-07-18T03:39:25.365 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "upmap_max_optimizations", 2023-07-18T03:39:25.365 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "uint", 2023-07-18T03:39:25.365 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.365 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.365 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "10", 2023-07-18T03:39:25.366 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.366 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.366 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.366 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-07-18T03:39:25.366 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.367 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.367 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.367 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:25.367 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:25.367 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.368 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:25.368 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "cephadm", 2023-07-18T03:39:25.368 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:25.368 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:25.368 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:25.369 INFO:teuthology.orchestra.run.smithi101.stdout: "agent_down_multiplier": { 2023-07-18T03:39:25.369 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "agent_down_multiplier", 2023-07-18T03:39:25.369 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "float", 2023-07-18T03:39:25.369 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.369 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.370 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "3.0", 2023-07-18T03:39:25.370 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.370 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.370 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.370 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-07-18T03:39:25.371 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.371 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.371 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.371 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.371 INFO:teuthology.orchestra.run.smithi101.stdout: "agent_refresh_rate": { 2023-07-18T03:39:25.372 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "agent_refresh_rate", 2023-07-18T03:39:25.372 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:25.372 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.372 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.372 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "20", 2023-07-18T03:39:25.373 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.373 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.373 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.373 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-07-18T03:39:25.373 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.374 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.374 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.374 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.374 INFO:teuthology.orchestra.run.smithi101.stdout: "agent_starting_port": { 2023-07-18T03:39:25.374 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "agent_starting_port", 2023-07-18T03:39:25.375 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:25.375 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.375 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.375 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "4721", 2023-07-18T03:39:25.375 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.376 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.376 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.376 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-07-18T03:39:25.376 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.376 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.377 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.377 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.377 INFO:teuthology.orchestra.run.smithi101.stdout: "alertmanager_web_password": { 2023-07-18T03:39:25.377 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "alertmanager_web_password", 2023-07-18T03:39:25.378 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.378 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.378 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.378 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "admin", 2023-07-18T03:39:25.378 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.379 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.379 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.379 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Alertmanager web password", 2023-07-18T03:39:25.379 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.379 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.380 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.380 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.380 INFO:teuthology.orchestra.run.smithi101.stdout: "alertmanager_web_user": { 2023-07-18T03:39:25.380 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "alertmanager_web_user", 2023-07-18T03:39:25.380 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.381 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.381 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.381 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "admin", 2023-07-18T03:39:25.381 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.381 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.382 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.382 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Alertmanager web user", 2023-07-18T03:39:25.382 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.382 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.382 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.383 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.383 INFO:teuthology.orchestra.run.smithi101.stdout: "allow_ptrace": { 2023-07-18T03:39:25.383 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "allow_ptrace", 2023-07-18T03:39:25.383 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.383 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.384 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.384 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.384 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.384 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.384 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.385 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-07-18T03:39:25.385 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-07-18T03:39:25.385 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.385 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.385 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.386 INFO:teuthology.orchestra.run.smithi101.stdout: "autotune_interval": { 2023-07-18T03:39:25.386 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "autotune_interval", 2023-07-18T03:39:25.386 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:25.386 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.386 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.387 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "600", 2023-07-18T03:39:25.387 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.387 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.387 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.387 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "how frequently to autotune daemon memory", 2023-07-18T03:39:25.388 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.388 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.388 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.388 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.388 INFO:teuthology.orchestra.run.smithi101.stdout: "autotune_memory_target_ratio": { 2023-07-18T03:39:25.389 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "autotune_memory_target_ratio", 2023-07-18T03:39:25.389 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "float", 2023-07-18T03:39:25.389 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.389 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.389 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "0.7", 2023-07-18T03:39:25.389 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.390 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.390 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.390 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-07-18T03:39:25.390 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.390 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.391 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.391 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.391 INFO:teuthology.orchestra.run.smithi101.stdout: "cgroups_split": { 2023-07-18T03:39:25.391 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "cgroups_split", 2023-07-18T03:39:25.391 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.392 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.392 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.392 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:25.392 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.392 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.393 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.393 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-07-18T03:39:25.393 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.393 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.393 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.394 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.394 INFO:teuthology.orchestra.run.smithi101.stdout: "config_checks_enabled": { 2023-07-18T03:39:25.394 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "config_checks_enabled", 2023-07-18T03:39:25.394 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.394 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.395 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.395 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.395 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.395 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.395 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.396 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-07-18T03:39:25.396 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.396 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.396 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.396 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.397 INFO:teuthology.orchestra.run.smithi101.stdout: "config_dashboard": { 2023-07-18T03:39:25.397 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "config_dashboard", 2023-07-18T03:39:25.397 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.397 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.397 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.398 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:25.398 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.398 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.398 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.398 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-07-18T03:39:25.399 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.399 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.399 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.399 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.399 INFO:teuthology.orchestra.run.smithi101.stdout: "container_image_alertmanager": { 2023-07-18T03:39:25.400 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "container_image_alertmanager", 2023-07-18T03:39:25.400 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.400 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.400 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.400 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-07-18T03:39:25.401 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.401 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.401 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.401 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Prometheus container image", 2023-07-18T03:39:25.401 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.402 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.402 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.402 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.402 INFO:teuthology.orchestra.run.smithi101.stdout: "container_image_base": { 2023-07-18T03:39:25.402 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "container_image_base", 2023-07-18T03:39:25.403 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.403 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.403 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.403 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "quay.io/ceph/ceph", 2023-07-18T03:39:25.403 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.404 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.404 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.404 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Container image name, without the tag", 2023-07-18T03:39:25.404 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.404 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.405 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.405 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.405 INFO:teuthology.orchestra.run.smithi101.stdout: "container_image_elasticsearch": { 2023-07-18T03:39:25.405 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "container_image_elasticsearch", 2023-07-18T03:39:25.405 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.406 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.406 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.406 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-07-18T03:39:25.406 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.406 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.407 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.407 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "elasticsearch container image", 2023-07-18T03:39:25.407 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.407 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.407 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.408 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.408 INFO:teuthology.orchestra.run.smithi101.stdout: "container_image_grafana": { 2023-07-18T03:39:25.408 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "container_image_grafana", 2023-07-18T03:39:25.408 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.408 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.409 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.409 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-07-18T03:39:25.409 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.409 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.409 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.410 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Prometheus container image", 2023-07-18T03:39:25.410 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.410 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.410 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.410 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.411 INFO:teuthology.orchestra.run.smithi101.stdout: "container_image_haproxy": { 2023-07-18T03:39:25.411 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "container_image_haproxy", 2023-07-18T03:39:25.411 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.411 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.411 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.412 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-07-18T03:39:25.412 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.412 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.412 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.412 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "HAproxy container image", 2023-07-18T03:39:25.413 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.413 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.413 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.413 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.413 INFO:teuthology.orchestra.run.smithi101.stdout: "container_image_jaeger_agent": { 2023-07-18T03:39:25.414 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "container_image_jaeger_agent", 2023-07-18T03:39:25.414 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.414 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.414 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.414 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-07-18T03:39:25.415 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.415 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.415 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.415 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Jaeger agent container image", 2023-07-18T03:39:25.415 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.416 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.416 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.416 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.416 INFO:teuthology.orchestra.run.smithi101.stdout: "container_image_jaeger_collector": { 2023-07-18T03:39:25.417 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "container_image_jaeger_collector", 2023-07-18T03:39:25.417 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.417 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.417 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.417 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-07-18T03:39:25.418 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.418 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.418 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.418 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Jaeger collector container image", 2023-07-18T03:39:25.418 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.419 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.419 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.419 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.419 INFO:teuthology.orchestra.run.smithi101.stdout: "container_image_jaeger_query": { 2023-07-18T03:39:25.419 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "container_image_jaeger_query", 2023-07-18T03:39:25.420 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.420 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.420 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.420 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-07-18T03:39:25.420 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.421 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.421 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.421 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Jaeger query container image", 2023-07-18T03:39:25.421 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.421 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.422 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.422 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.422 INFO:teuthology.orchestra.run.smithi101.stdout: "container_image_keepalived": { 2023-07-18T03:39:25.422 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "container_image_keepalived", 2023-07-18T03:39:25.422 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.423 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.423 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.423 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-07-18T03:39:25.423 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.423 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.424 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.424 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Keepalived container image", 2023-07-18T03:39:25.424 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.424 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.424 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.425 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.425 INFO:teuthology.orchestra.run.smithi101.stdout: "container_image_loki": { 2023-07-18T03:39:25.425 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "container_image_loki", 2023-07-18T03:39:25.425 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.425 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.426 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.426 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-07-18T03:39:25.426 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.426 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.426 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.427 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Loki container image", 2023-07-18T03:39:25.427 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.427 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.427 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.427 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.428 INFO:teuthology.orchestra.run.smithi101.stdout: "container_image_node_exporter": { 2023-07-18T03:39:25.428 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "container_image_node_exporter", 2023-07-18T03:39:25.428 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.428 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.428 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.429 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-07-18T03:39:25.429 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.429 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.429 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.429 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Prometheus container image", 2023-07-18T03:39:25.430 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.430 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.430 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.430 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.430 INFO:teuthology.orchestra.run.smithi101.stdout: "container_image_prometheus": { 2023-07-18T03:39:25.431 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "container_image_prometheus", 2023-07-18T03:39:25.431 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.431 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.431 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.431 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-07-18T03:39:25.432 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.432 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.432 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.432 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Prometheus container image", 2023-07-18T03:39:25.432 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.433 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.433 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.433 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.433 INFO:teuthology.orchestra.run.smithi101.stdout: "container_image_promtail": { 2023-07-18T03:39:25.433 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "container_image_promtail", 2023-07-18T03:39:25.434 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.434 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.434 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.434 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-07-18T03:39:25.434 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.435 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.435 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.435 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Promtail container image", 2023-07-18T03:39:25.435 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.435 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.436 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.436 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.436 INFO:teuthology.orchestra.run.smithi101.stdout: "container_image_snmp_gateway": { 2023-07-18T03:39:25.436 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "container_image_snmp_gateway", 2023-07-18T03:39:25.436 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.437 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.437 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.437 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-07-18T03:39:25.437 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.437 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.438 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.438 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "SNMP Gateway container image", 2023-07-18T03:39:25.438 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.438 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.438 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.439 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.439 INFO:teuthology.orchestra.run.smithi101.stdout: "container_init": { 2023-07-18T03:39:25.439 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "container_init", 2023-07-18T03:39:25.439 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.439 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.440 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.440 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:25.440 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.440 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.440 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.441 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Run podman/docker with `--init`", 2023-07-18T03:39:25.441 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.441 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.441 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.441 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.442 INFO:teuthology.orchestra.run.smithi101.stdout: "daemon_cache_timeout": { 2023-07-18T03:39:25.442 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "daemon_cache_timeout", 2023-07-18T03:39:25.442 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:25.442 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.442 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.442 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "600", 2023-07-18T03:39:25.443 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.443 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.443 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.443 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-07-18T03:39:25.443 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.444 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.444 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.444 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.444 INFO:teuthology.orchestra.run.smithi101.stdout: "default_cephadm_command_timeout": { 2023-07-18T03:39:25.445 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "default_cephadm_command_timeout", 2023-07-18T03:39:25.445 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:25.445 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.445 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.445 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "900", 2023-07-18T03:39:25.446 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.446 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.446 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.446 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-07-18T03:39:25.446 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.447 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.447 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.447 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.447 INFO:teuthology.orchestra.run.smithi101.stdout: "default_registry": { 2023-07-18T03:39:25.447 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "default_registry", 2023-07-18T03:39:25.448 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.448 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.448 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.448 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "docker.io", 2023-07-18T03:39:25.448 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.449 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.449 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.449 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-07-18T03:39:25.449 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.449 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.450 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.450 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.450 INFO:teuthology.orchestra.run.smithi101.stdout: "device_cache_timeout": { 2023-07-18T03:39:25.450 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "device_cache_timeout", 2023-07-18T03:39:25.450 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:25.451 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.451 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.451 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "1800", 2023-07-18T03:39:25.451 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.451 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.451 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.452 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "seconds to cache device inventory", 2023-07-18T03:39:25.452 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.452 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.452 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.452 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.453 INFO:teuthology.orchestra.run.smithi101.stdout: "device_enhanced_scan": { 2023-07-18T03:39:25.453 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "device_enhanced_scan", 2023-07-18T03:39:25.453 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.453 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.453 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.454 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.454 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.454 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.454 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.454 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Use libstoragemgmt during device scans", 2023-07-18T03:39:25.455 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.455 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.455 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.455 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.455 INFO:teuthology.orchestra.run.smithi101.stdout: "facts_cache_timeout": { 2023-07-18T03:39:25.456 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "facts_cache_timeout", 2023-07-18T03:39:25.456 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:25.456 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.456 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.456 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "60", 2023-07-18T03:39:25.457 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.457 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.457 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.457 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "seconds to cache host facts data", 2023-07-18T03:39:25.457 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.458 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.458 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.458 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.458 INFO:teuthology.orchestra.run.smithi101.stdout: "host_check_interval": { 2023-07-18T03:39:25.458 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "host_check_interval", 2023-07-18T03:39:25.459 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:25.459 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.459 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.459 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "600", 2023-07-18T03:39:25.459 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.460 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.460 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.460 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "how frequently to perform a host check", 2023-07-18T03:39:25.460 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.460 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.461 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.461 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.461 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:25.461 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:25.461 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.462 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.462 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.462 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.462 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.462 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.463 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:25.463 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:25.463 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:25.463 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:25.463 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:25.464 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:25.464 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:25.464 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:25.464 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.464 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.465 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.465 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.465 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.465 INFO:teuthology.orchestra.run.smithi101.stdout: "log_refresh_metadata": { 2023-07-18T03:39:25.465 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_refresh_metadata", 2023-07-18T03:39:25.466 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.466 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.466 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.466 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.466 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.466 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.467 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.467 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-07-18T03:39:25.467 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.467 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.467 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.468 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.468 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:25.468 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:25.468 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.468 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.469 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.469 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:25.469 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.469 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.469 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.470 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-07-18T03:39:25.470 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.470 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.470 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.470 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.471 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:25.471 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:25.471 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.471 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.471 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.472 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:25.472 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.472 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.472 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:25.472 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:25.473 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:25.473 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:25.473 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:25.473 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:25.473 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:25.474 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:25.474 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.474 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.474 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.474 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.475 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.475 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:25.475 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:25.475 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.475 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.475 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.476 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.476 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.476 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.476 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.476 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.477 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.477 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.477 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.477 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.477 INFO:teuthology.orchestra.run.smithi101.stdout: "manage_etc_ceph_ceph_conf": { 2023-07-18T03:39:25.478 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-07-18T03:39:25.478 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.478 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.478 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.478 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.479 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.479 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.479 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.479 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-07-18T03:39:25.480 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.480 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.480 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.480 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.480 INFO:teuthology.orchestra.run.smithi101.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-07-18T03:39:25.481 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-07-18T03:39:25.481 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.481 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.481 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.481 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "*", 2023-07-18T03:39:25.482 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.482 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.482 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.482 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-07-18T03:39:25.482 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.483 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.483 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.483 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.483 INFO:teuthology.orchestra.run.smithi101.stdout: "max_count_per_host": { 2023-07-18T03:39:25.483 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "max_count_per_host", 2023-07-18T03:39:25.483 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:25.484 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.484 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.484 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "10", 2023-07-18T03:39:25.484 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.484 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.485 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.485 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "max number of daemons per service per host", 2023-07-18T03:39:25.485 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.485 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.485 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.486 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.486 INFO:teuthology.orchestra.run.smithi101.stdout: "max_osd_draining_count": { 2023-07-18T03:39:25.486 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "max_osd_draining_count", 2023-07-18T03:39:25.486 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:25.486 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.487 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.487 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "10", 2023-07-18T03:39:25.487 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.487 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.487 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.488 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-07-18T03:39:25.488 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.488 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.488 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.488 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.489 INFO:teuthology.orchestra.run.smithi101.stdout: "migration_current": { 2023-07-18T03:39:25.489 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "migration_current", 2023-07-18T03:39:25.489 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:25.489 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.489 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.490 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.490 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.490 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.490 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.490 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "internal - do not modify", 2023-07-18T03:39:25.491 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.491 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.491 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.491 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.491 INFO:teuthology.orchestra.run.smithi101.stdout: "mode": { 2023-07-18T03:39:25.492 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "mode", 2023-07-18T03:39:25.492 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.492 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.492 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.492 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "root", 2023-07-18T03:39:25.493 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.493 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.493 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:25.493 INFO:teuthology.orchestra.run.smithi101.stdout: "cephadm-package", 2023-07-18T03:39:25.493 INFO:teuthology.orchestra.run.smithi101.stdout: "root" 2023-07-18T03:39:25.494 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:25.494 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "mode for remote execution of cephadm", 2023-07-18T03:39:25.494 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.494 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.494 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.495 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.495 INFO:teuthology.orchestra.run.smithi101.stdout: "prometheus_alerts_path": { 2023-07-18T03:39:25.495 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "prometheus_alerts_path", 2023-07-18T03:39:25.495 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.495 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.495 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.496 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-07-18T03:39:25.496 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.496 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.496 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.496 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-07-18T03:39:25.497 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.497 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.497 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.497 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.497 INFO:teuthology.orchestra.run.smithi101.stdout: "prometheus_web_password": { 2023-07-18T03:39:25.498 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "prometheus_web_password", 2023-07-18T03:39:25.498 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.498 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.498 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.498 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "admin", 2023-07-18T03:39:25.499 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.499 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.499 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.499 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Prometheus web password", 2023-07-18T03:39:25.499 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.500 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.500 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.500 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.500 INFO:teuthology.orchestra.run.smithi101.stdout: "prometheus_web_user": { 2023-07-18T03:39:25.500 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "prometheus_web_user", 2023-07-18T03:39:25.501 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.501 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.501 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.501 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "admin", 2023-07-18T03:39:25.501 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.502 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.502 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.502 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Prometheus web user", 2023-07-18T03:39:25.502 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.502 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.503 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.503 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.503 INFO:teuthology.orchestra.run.smithi101.stdout: "registry_insecure": { 2023-07-18T03:39:25.503 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "registry_insecure", 2023-07-18T03:39:25.503 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.504 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.504 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.504 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.504 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.504 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.505 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.505 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-07-18T03:39:25.505 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.505 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.505 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.505 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.506 INFO:teuthology.orchestra.run.smithi101.stdout: "registry_password": { 2023-07-18T03:39:25.506 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "registry_password", 2023-07-18T03:39:25.506 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.506 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.506 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.507 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.507 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.507 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.507 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.507 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-07-18T03:39:25.508 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.508 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.508 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.508 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.508 INFO:teuthology.orchestra.run.smithi101.stdout: "registry_url": { 2023-07-18T03:39:25.509 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "registry_url", 2023-07-18T03:39:25.509 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.509 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.509 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.509 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.510 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.510 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.510 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.510 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-07-18T03:39:25.510 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.511 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.511 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.511 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.511 INFO:teuthology.orchestra.run.smithi101.stdout: "registry_username": { 2023-07-18T03:39:25.511 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "registry_username", 2023-07-18T03:39:25.512 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.512 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.512 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.512 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.512 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.513 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.513 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.513 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-07-18T03:39:25.513 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.513 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.514 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.514 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.514 INFO:teuthology.orchestra.run.smithi101.stdout: "secure_monitoring_stack": { 2023-07-18T03:39:25.514 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "secure_monitoring_stack", 2023-07-18T03:39:25.514 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.515 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.515 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.515 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.515 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.515 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.516 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.516 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-07-18T03:39:25.516 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.516 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.516 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.517 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.517 INFO:teuthology.orchestra.run.smithi101.stdout: "service_discovery_port": { 2023-07-18T03:39:25.517 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "service_discovery_port", 2023-07-18T03:39:25.517 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:25.517 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.518 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.518 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "8765", 2023-07-18T03:39:25.518 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.518 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.518 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.519 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "cephadm service discovery port", 2023-07-18T03:39:25.519 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.519 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.519 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.519 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.520 INFO:teuthology.orchestra.run.smithi101.stdout: "ssh_config_file": { 2023-07-18T03:39:25.520 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "ssh_config_file", 2023-07-18T03:39:25.520 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.520 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.520 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.521 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.521 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.521 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.521 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.521 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-07-18T03:39:25.522 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.522 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.522 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.522 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.522 INFO:teuthology.orchestra.run.smithi101.stdout: "use_agent": { 2023-07-18T03:39:25.523 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "use_agent", 2023-07-18T03:39:25.523 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.523 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.523 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.523 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.523 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.524 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.524 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.524 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-07-18T03:39:25.524 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.525 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.525 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.525 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.525 INFO:teuthology.orchestra.run.smithi101.stdout: "use_repo_digest": { 2023-07-18T03:39:25.525 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "use_repo_digest", 2023-07-18T03:39:25.525 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.526 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.526 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.526 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:25.526 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.526 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.527 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.527 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-07-18T03:39:25.527 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.527 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.527 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.528 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.528 INFO:teuthology.orchestra.run.smithi101.stdout: "warn_on_failed_host_check": { 2023-07-18T03:39:25.528 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "warn_on_failed_host_check", 2023-07-18T03:39:25.528 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.528 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.529 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.529 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:25.529 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.529 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.529 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.530 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "raise a health warning if the host check fails", 2023-07-18T03:39:25.530 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.530 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.530 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.530 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.531 INFO:teuthology.orchestra.run.smithi101.stdout: "warn_on_stray_daemons": { 2023-07-18T03:39:25.531 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "warn_on_stray_daemons", 2023-07-18T03:39:25.531 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.531 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.531 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.532 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:25.532 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.532 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.532 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.532 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-07-18T03:39:25.533 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.533 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.533 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.533 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.533 INFO:teuthology.orchestra.run.smithi101.stdout: "warn_on_stray_hosts": { 2023-07-18T03:39:25.534 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "warn_on_stray_hosts", 2023-07-18T03:39:25.534 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.534 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.534 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.534 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:25.535 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.535 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.535 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.535 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-07-18T03:39:25.535 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.536 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.536 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.536 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:25.536 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:25.536 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.537 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:25.537 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "crash", 2023-07-18T03:39:25.537 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:25.537 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:25.537 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:25.537 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:25.538 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:25.538 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.538 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.538 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.538 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.539 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.539 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.539 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:25.539 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:25.539 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:25.540 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:25.540 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:25.540 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:25.540 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:25.540 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:25.541 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.541 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.541 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.541 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.541 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.542 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:25.542 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:25.542 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.542 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.542 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.543 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.543 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.543 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.543 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.543 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.544 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.544 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.544 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.544 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.544 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:25.545 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:25.545 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.545 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.545 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.545 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:25.546 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.546 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.546 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:25.546 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:25.546 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:25.547 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:25.547 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:25.547 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:25.547 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:25.547 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:25.547 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.548 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.548 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.548 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.548 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.549 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:25.549 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:25.549 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.549 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.549 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.550 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.550 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.550 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.550 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.550 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.551 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.551 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.551 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.551 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.551 INFO:teuthology.orchestra.run.smithi101.stdout: "retain_interval": { 2023-07-18T03:39:25.552 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "retain_interval", 2023-07-18T03:39:25.552 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:25.552 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.552 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.552 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "31536000", 2023-07-18T03:39:25.553 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.553 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.553 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.553 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "how long to retain crashes before pruning them", 2023-07-18T03:39:25.553 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.554 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.554 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.554 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.554 INFO:teuthology.orchestra.run.smithi101.stdout: "warn_recent_interval": { 2023-07-18T03:39:25.554 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "warn_recent_interval", 2023-07-18T03:39:25.554 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:25.555 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.556 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.556 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "1209600", 2023-07-18T03:39:25.556 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.556 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.556 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.557 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "time interval in which to warn about recent crashes", 2023-07-18T03:39:25.557 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.557 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.557 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.557 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:25.558 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:25.558 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.558 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:25.558 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "dashboard", 2023-07-18T03:39:25.558 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:25.559 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:25.559 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:25.559 INFO:teuthology.orchestra.run.smithi101.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-07-18T03:39:25.559 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-07-18T03:39:25.559 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:25.560 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.560 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.560 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "10", 2023-07-18T03:39:25.560 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.560 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.561 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.561 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.561 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.561 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.561 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.562 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.562 INFO:teuthology.orchestra.run.smithi101.stdout: "ALERTMANAGER_API_HOST": { 2023-07-18T03:39:25.562 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "ALERTMANAGER_API_HOST", 2023-07-18T03:39:25.562 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.562 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.562 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.563 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.563 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.563 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.563 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.563 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.564 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.564 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.564 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.564 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.564 INFO:teuthology.orchestra.run.smithi101.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-07-18T03:39:25.565 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-07-18T03:39:25.565 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.565 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.565 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.565 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:25.566 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.566 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.566 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.566 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.566 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.567 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.567 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.567 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.567 INFO:teuthology.orchestra.run.smithi101.stdout: "AUDIT_API_ENABLED": { 2023-07-18T03:39:25.567 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "AUDIT_API_ENABLED", 2023-07-18T03:39:25.568 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.568 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.568 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.568 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.568 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.569 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.569 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.569 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.569 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.569 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.570 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.570 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.570 INFO:teuthology.orchestra.run.smithi101.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-07-18T03:39:25.570 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-07-18T03:39:25.570 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.571 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.571 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.571 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:25.571 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.571 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.571 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.572 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.572 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.572 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.572 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.572 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.573 INFO:teuthology.orchestra.run.smithi101.stdout: "ENABLE_BROWSABLE_API": { 2023-07-18T03:39:25.573 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "ENABLE_BROWSABLE_API", 2023-07-18T03:39:25.573 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.573 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.573 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.574 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:25.574 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.574 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.574 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.574 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.575 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.575 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.575 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.575 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.575 INFO:teuthology.orchestra.run.smithi101.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-07-18T03:39:25.576 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-07-18T03:39:25.576 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.576 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.576 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.576 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:25.577 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.577 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.577 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.577 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.577 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.578 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.578 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.578 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.578 INFO:teuthology.orchestra.run.smithi101.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-07-18T03:39:25.578 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-07-18T03:39:25.578 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.579 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.579 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.579 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:25.579 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.579 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.580 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.580 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.580 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.580 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.580 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.581 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.581 INFO:teuthology.orchestra.run.smithi101.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-07-18T03:39:25.581 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-07-18T03:39:25.581 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.581 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.582 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.582 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:25.582 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.582 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.582 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.583 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.583 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.583 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.583 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.583 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.584 INFO:teuthology.orchestra.run.smithi101.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-07-18T03:39:25.584 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-07-18T03:39:25.584 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.584 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.584 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.585 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:25.585 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.585 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.585 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.585 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.586 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.586 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.586 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.586 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.586 INFO:teuthology.orchestra.run.smithi101.stdout: "FEATURE_TOGGLE_NFS": { 2023-07-18T03:39:25.587 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-07-18T03:39:25.587 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.587 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.587 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.587 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:25.588 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.588 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.588 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.588 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.588 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.589 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.589 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.589 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.589 INFO:teuthology.orchestra.run.smithi101.stdout: "FEATURE_TOGGLE_RBD": { 2023-07-18T03:39:25.589 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-07-18T03:39:25.590 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.590 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.590 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.590 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:25.590 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.591 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.591 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.591 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.591 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.591 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.591 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.592 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.592 INFO:teuthology.orchestra.run.smithi101.stdout: "FEATURE_TOGGLE_RGW": { 2023-07-18T03:39:25.592 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-07-18T03:39:25.592 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.592 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.593 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.593 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:25.593 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.593 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.593 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.594 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.594 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.594 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.594 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.594 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.595 INFO:teuthology.orchestra.run.smithi101.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-07-18T03:39:25.595 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-07-18T03:39:25.595 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.595 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.595 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.596 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.596 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.596 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.596 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.596 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.597 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.597 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.597 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.597 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.597 INFO:teuthology.orchestra.run.smithi101.stdout: "GRAFANA_API_PASSWORD": { 2023-07-18T03:39:25.598 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "GRAFANA_API_PASSWORD", 2023-07-18T03:39:25.598 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.598 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.598 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.598 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "admin", 2023-07-18T03:39:25.599 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.599 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.599 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.599 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.599 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.600 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.600 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.600 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.600 INFO:teuthology.orchestra.run.smithi101.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-07-18T03:39:25.600 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-07-18T03:39:25.601 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.601 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.601 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.601 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:25.601 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.602 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.602 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.602 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.602 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.602 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.602 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.603 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.603 INFO:teuthology.orchestra.run.smithi101.stdout: "GRAFANA_API_URL": { 2023-07-18T03:39:25.603 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "GRAFANA_API_URL", 2023-07-18T03:39:25.603 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.603 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.604 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.604 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.604 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.604 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.604 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.605 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.605 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.605 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.605 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.605 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.606 INFO:teuthology.orchestra.run.smithi101.stdout: "GRAFANA_API_USERNAME": { 2023-07-18T03:39:25.606 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "GRAFANA_API_USERNAME", 2023-07-18T03:39:25.606 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.606 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.606 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.607 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "admin", 2023-07-18T03:39:25.607 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.607 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.607 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.607 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.608 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.608 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.608 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.608 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.608 INFO:teuthology.orchestra.run.smithi101.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-07-18T03:39:25.609 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-07-18T03:39:25.609 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.609 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.609 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.609 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.610 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.610 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.610 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.610 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.610 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.610 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.611 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.611 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.611 INFO:teuthology.orchestra.run.smithi101.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-07-18T03:39:25.611 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-07-18T03:39:25.611 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.612 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.612 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.612 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.612 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.612 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.613 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.613 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.613 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.613 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.613 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.614 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.614 INFO:teuthology.orchestra.run.smithi101.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-07-18T03:39:25.614 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-07-18T03:39:25.614 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.614 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.615 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.615 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:25.615 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.615 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.615 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.616 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.616 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.616 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.616 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.616 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.617 INFO:teuthology.orchestra.run.smithi101.stdout: "ISSUE_TRACKER_API_KEY": { 2023-07-18T03:39:25.617 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-07-18T03:39:25.617 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.617 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.617 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.618 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.618 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.618 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.618 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.618 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.618 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.619 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.619 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.619 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.619 INFO:teuthology.orchestra.run.smithi101.stdout: "PROMETHEUS_API_HOST": { 2023-07-18T03:39:25.619 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "PROMETHEUS_API_HOST", 2023-07-18T03:39:25.620 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.620 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.620 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.620 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.620 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.621 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.621 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.621 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.621 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.621 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.622 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.622 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.622 INFO:teuthology.orchestra.run.smithi101.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-07-18T03:39:25.622 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-07-18T03:39:25.623 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.623 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.623 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.623 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:25.623 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.623 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.624 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.624 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.624 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.624 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.624 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.625 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.625 INFO:teuthology.orchestra.run.smithi101.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-07-18T03:39:25.625 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-07-18T03:39:25.625 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.625 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.626 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.626 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.626 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.626 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.626 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.627 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.627 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.627 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.627 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.627 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.628 INFO:teuthology.orchestra.run.smithi101.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-07-18T03:39:25.628 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-07-18T03:39:25.628 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.628 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.628 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.629 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.629 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.629 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.629 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.629 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.630 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.630 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.630 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.630 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.630 INFO:teuthology.orchestra.run.smithi101.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-07-18T03:39:25.631 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-07-18T03:39:25.631 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.631 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.631 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.631 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:25.632 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.632 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.632 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.632 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.632 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.633 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.633 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.633 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.633 INFO:teuthology.orchestra.run.smithi101.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-07-18T03:39:25.633 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-07-18T03:39:25.634 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.634 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.634 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.634 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:25.634 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.635 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.635 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.635 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.635 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.635 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.635 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.636 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.636 INFO:teuthology.orchestra.run.smithi101.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-07-18T03:39:25.636 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-07-18T03:39:25.636 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.636 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.637 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.637 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.637 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.637 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.637 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.638 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.638 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.638 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.638 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.638 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.639 INFO:teuthology.orchestra.run.smithi101.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-07-18T03:39:25.639 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-07-18T03:39:25.639 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.639 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.639 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.640 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.640 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.640 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.640 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.640 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.641 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.641 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.641 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.641 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.641 INFO:teuthology.orchestra.run.smithi101.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-07-18T03:39:25.642 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-07-18T03:39:25.642 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.642 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.642 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.642 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.642 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.643 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.643 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.643 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.643 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.643 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.644 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.644 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.644 INFO:teuthology.orchestra.run.smithi101.stdout: "PWD_POLICY_ENABLED": { 2023-07-18T03:39:25.644 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "PWD_POLICY_ENABLED", 2023-07-18T03:39:25.644 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.645 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.645 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.645 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:25.645 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.645 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.646 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.646 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.646 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.646 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.646 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.647 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.647 INFO:teuthology.orchestra.run.smithi101.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-07-18T03:39:25.647 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-07-18T03:39:25.647 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.647 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.648 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.648 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-07-18T03:39:25.648 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.648 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.648 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.649 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.649 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.649 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.649 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.649 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.649 INFO:teuthology.orchestra.run.smithi101.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-07-18T03:39:25.650 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-07-18T03:39:25.650 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:25.650 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.650 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.650 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "10", 2023-07-18T03:39:25.651 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.651 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.651 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.651 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.651 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.652 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.652 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.652 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.652 INFO:teuthology.orchestra.run.smithi101.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-07-18T03:39:25.652 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-07-18T03:39:25.653 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:25.653 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.653 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.653 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "8", 2023-07-18T03:39:25.653 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.654 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.654 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.654 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.654 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.654 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.655 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.655 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.655 INFO:teuthology.orchestra.run.smithi101.stdout: "REST_REQUESTS_TIMEOUT": { 2023-07-18T03:39:25.655 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-07-18T03:39:25.655 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:25.656 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.656 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.656 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "45", 2023-07-18T03:39:25.656 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.656 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.657 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.657 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.657 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.657 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.657 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.658 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.658 INFO:teuthology.orchestra.run.smithi101.stdout: "RGW_API_ACCESS_KEY": { 2023-07-18T03:39:25.658 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "RGW_API_ACCESS_KEY", 2023-07-18T03:39:25.658 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.658 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.659 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.659 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.659 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.659 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.659 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.660 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.660 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.660 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.660 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.660 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.661 INFO:teuthology.orchestra.run.smithi101.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-07-18T03:39:25.661 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-07-18T03:39:25.661 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.661 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.661 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.662 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "admin", 2023-07-18T03:39:25.662 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.662 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.662 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.662 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.662 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.663 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.663 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.663 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.663 INFO:teuthology.orchestra.run.smithi101.stdout: "RGW_API_SECRET_KEY": { 2023-07-18T03:39:25.663 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "RGW_API_SECRET_KEY", 2023-07-18T03:39:25.664 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.664 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.664 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.664 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.664 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.665 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.665 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.665 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.665 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.665 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.666 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.666 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.666 INFO:teuthology.orchestra.run.smithi101.stdout: "RGW_API_SSL_VERIFY": { 2023-07-18T03:39:25.666 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "RGW_API_SSL_VERIFY", 2023-07-18T03:39:25.666 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.667 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.667 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.667 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:25.667 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.667 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.668 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.668 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.668 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.668 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.668 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.669 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.669 INFO:teuthology.orchestra.run.smithi101.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-07-18T03:39:25.669 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-07-18T03:39:25.669 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:25.669 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.670 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.670 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "0", 2023-07-18T03:39:25.670 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.670 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.670 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.671 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.671 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.671 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.671 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.671 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.672 INFO:teuthology.orchestra.run.smithi101.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-07-18T03:39:25.672 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-07-18T03:39:25.672 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:25.672 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.672 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.673 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "10", 2023-07-18T03:39:25.673 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.673 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.673 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.673 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.674 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.674 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.674 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.674 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.674 INFO:teuthology.orchestra.run.smithi101.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-07-18T03:39:25.674 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-07-18T03:39:25.675 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:25.675 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.675 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.675 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "5", 2023-07-18T03:39:25.675 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.676 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.676 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.676 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.676 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.676 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.677 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.677 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.677 INFO:teuthology.orchestra.run.smithi101.stdout: "cross_origin_url": { 2023-07-18T03:39:25.677 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "cross_origin_url", 2023-07-18T03:39:25.677 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.678 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.678 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.678 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.678 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.678 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.679 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.679 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.679 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.679 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.679 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.680 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.680 INFO:teuthology.orchestra.run.smithi101.stdout: "crt_file": { 2023-07-18T03:39:25.680 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "crt_file", 2023-07-18T03:39:25.680 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.680 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.681 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.681 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.681 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.681 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.681 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.682 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.682 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.682 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.682 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.682 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.683 INFO:teuthology.orchestra.run.smithi101.stdout: "debug": { 2023-07-18T03:39:25.683 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "debug", 2023-07-18T03:39:25.683 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.683 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.683 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.684 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.684 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.684 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.684 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.684 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Enable/disable debug options", 2023-07-18T03:39:25.685 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.685 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.685 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.685 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.685 INFO:teuthology.orchestra.run.smithi101.stdout: "jwt_token_ttl": { 2023-07-18T03:39:25.686 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "jwt_token_ttl", 2023-07-18T03:39:25.686 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:25.686 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.686 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.686 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "28800", 2023-07-18T03:39:25.687 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.687 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.687 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.687 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.687 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.687 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.688 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.688 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.688 INFO:teuthology.orchestra.run.smithi101.stdout: "key_file": { 2023-07-18T03:39:25.688 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "key_file", 2023-07-18T03:39:25.688 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.689 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.689 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.689 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.689 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.689 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.690 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.690 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.690 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.690 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.690 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.691 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.691 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:25.691 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:25.691 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.691 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.692 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.692 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.692 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.692 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.692 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:25.693 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:25.693 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:25.693 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:25.693 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:25.693 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:25.694 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:25.694 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:25.694 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.694 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.694 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.695 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.695 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.695 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:25.695 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:25.695 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.696 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.696 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.696 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.696 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.696 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.697 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.697 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.697 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.697 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.697 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.698 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.698 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:25.698 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:25.698 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.698 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.699 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.699 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:25.699 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.699 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.699 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:25.700 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:25.700 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:25.700 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:25.700 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:25.700 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:25.701 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:25.701 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:25.701 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.701 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.701 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.702 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.702 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.702 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:25.702 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:25.702 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.702 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.703 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.703 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.703 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.703 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.703 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.704 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.704 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.704 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.704 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.704 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.705 INFO:teuthology.orchestra.run.smithi101.stdout: "motd": { 2023-07-18T03:39:25.705 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "motd", 2023-07-18T03:39:25.705 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.705 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.705 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.706 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.706 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.706 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.706 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.706 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "The message of the day", 2023-07-18T03:39:25.707 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.707 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.707 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.707 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.707 INFO:teuthology.orchestra.run.smithi101.stdout: "redirect_resolve_ip_addr": { 2023-07-18T03:39:25.708 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "redirect_resolve_ip_addr", 2023-07-18T03:39:25.708 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.708 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.708 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.708 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.709 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.709 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.709 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.709 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.709 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.710 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.710 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.710 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.710 INFO:teuthology.orchestra.run.smithi101.stdout: "server_addr": { 2023-07-18T03:39:25.710 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "server_addr", 2023-07-18T03:39:25.711 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.711 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.711 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.711 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "::", 2023-07-18T03:39:25.711 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.712 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.712 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.712 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.712 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.712 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.713 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.713 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.713 INFO:teuthology.orchestra.run.smithi101.stdout: "server_port": { 2023-07-18T03:39:25.713 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "server_port", 2023-07-18T03:39:25.713 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:25.714 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.714 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.714 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "8080", 2023-07-18T03:39:25.714 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.714 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.714 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.715 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.715 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.715 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.715 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.715 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.716 INFO:teuthology.orchestra.run.smithi101.stdout: "ssl": { 2023-07-18T03:39:25.716 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "ssl", 2023-07-18T03:39:25.716 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.716 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.716 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.717 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:25.717 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.717 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.717 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.717 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.718 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.718 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.718 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.718 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.718 INFO:teuthology.orchestra.run.smithi101.stdout: "ssl_server_port": { 2023-07-18T03:39:25.719 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "ssl_server_port", 2023-07-18T03:39:25.719 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:25.719 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.719 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.719 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "8443", 2023-07-18T03:39:25.720 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.720 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.720 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.720 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.720 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.721 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.721 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.721 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.721 INFO:teuthology.orchestra.run.smithi101.stdout: "standby_behaviour": { 2023-07-18T03:39:25.721 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "standby_behaviour", 2023-07-18T03:39:25.721 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.722 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.722 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.722 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "redirect", 2023-07-18T03:39:25.722 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.722 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.723 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:25.723 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:25.723 INFO:teuthology.orchestra.run.smithi101.stdout: "redirect" 2023-07-18T03:39:25.723 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:25.723 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.724 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.724 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.724 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.724 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.724 INFO:teuthology.orchestra.run.smithi101.stdout: "standby_error_status_code": { 2023-07-18T03:39:25.725 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "standby_error_status_code", 2023-07-18T03:39:25.725 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:25.725 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.725 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.725 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "500", 2023-07-18T03:39:25.726 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "400", 2023-07-18T03:39:25.726 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "599", 2023-07-18T03:39:25.726 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.726 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.726 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.727 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.727 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.727 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.727 INFO:teuthology.orchestra.run.smithi101.stdout: "url_prefix": { 2023-07-18T03:39:25.727 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "url_prefix", 2023-07-18T03:39:25.728 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.728 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.728 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.728 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.728 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.729 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.729 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.729 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.729 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.729 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.730 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.730 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:25.730 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:25.730 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.730 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:25.730 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "devicehealth", 2023-07-18T03:39:25.731 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:25.731 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:25.731 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:25.731 INFO:teuthology.orchestra.run.smithi101.stdout: "enable_monitoring": { 2023-07-18T03:39:25.731 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "enable_monitoring", 2023-07-18T03:39:25.732 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.732 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.732 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.732 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:25.733 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.733 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.733 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.733 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "monitor device health metrics", 2023-07-18T03:39:25.733 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.734 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.734 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.734 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.734 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:25.734 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:25.735 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.735 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.735 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.735 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.735 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.735 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.736 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:25.736 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:25.736 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:25.736 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:25.736 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:25.737 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:25.737 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:25.737 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:25.737 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.737 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.738 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.738 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.738 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.738 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:25.738 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:25.739 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.739 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.739 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.739 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.739 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.740 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.740 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.740 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.740 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.740 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.741 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.741 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.741 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:25.741 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:25.741 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.742 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.742 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.742 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:25.742 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.742 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.742 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:25.743 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:25.743 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:25.743 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:25.743 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:25.743 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:25.744 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:25.744 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:25.744 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.744 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.744 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.745 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.745 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.745 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:25.745 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:25.745 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.746 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.746 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.746 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.746 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.746 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.747 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.747 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.747 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.747 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.747 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.748 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.748 INFO:teuthology.orchestra.run.smithi101.stdout: "mark_out_threshold": { 2023-07-18T03:39:25.748 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "mark_out_threshold", 2023-07-18T03:39:25.748 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:25.748 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.749 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.749 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "2419200", 2023-07-18T03:39:25.749 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.749 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.749 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.749 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-07-18T03:39:25.750 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.750 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.750 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.750 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.750 INFO:teuthology.orchestra.run.smithi101.stdout: "pool_name": { 2023-07-18T03:39:25.751 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "pool_name", 2023-07-18T03:39:25.751 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.751 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.751 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.751 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "device_health_metrics", 2023-07-18T03:39:25.752 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.752 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.752 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.752 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "name of pool in which to store device health metrics", 2023-07-18T03:39:25.752 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.753 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.753 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.753 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.753 INFO:teuthology.orchestra.run.smithi101.stdout: "retention_period": { 2023-07-18T03:39:25.753 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "retention_period", 2023-07-18T03:39:25.754 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:25.754 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.754 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.754 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "15552000", 2023-07-18T03:39:25.754 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.755 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.755 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.755 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "how long to retain device health metrics", 2023-07-18T03:39:25.755 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.755 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.756 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.756 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.756 INFO:teuthology.orchestra.run.smithi101.stdout: "scrape_frequency": { 2023-07-18T03:39:25.756 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "scrape_frequency", 2023-07-18T03:39:25.756 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:25.756 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.757 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.757 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "86400", 2023-07-18T03:39:25.757 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.757 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.757 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.758 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "how frequently to scrape device health metrics", 2023-07-18T03:39:25.758 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.758 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.758 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.758 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.759 INFO:teuthology.orchestra.run.smithi101.stdout: "self_heal": { 2023-07-18T03:39:25.759 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "self_heal", 2023-07-18T03:39:25.759 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.759 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.759 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.760 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:25.760 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.760 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.760 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.760 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-07-18T03:39:25.761 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.761 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.761 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.761 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.761 INFO:teuthology.orchestra.run.smithi101.stdout: "sleep_interval": { 2023-07-18T03:39:25.762 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "sleep_interval", 2023-07-18T03:39:25.762 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:25.762 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.762 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.762 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "600", 2023-07-18T03:39:25.762 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.763 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.763 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.763 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "how frequently to wake up and check device health", 2023-07-18T03:39:25.763 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.763 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.764 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.764 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.764 INFO:teuthology.orchestra.run.smithi101.stdout: "warn_threshold": { 2023-07-18T03:39:25.764 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "warn_threshold", 2023-07-18T03:39:25.764 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:25.765 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.765 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.765 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "7257600", 2023-07-18T03:39:25.765 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.765 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.766 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.766 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-07-18T03:39:25.766 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.766 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.766 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.767 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:25.767 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:25.767 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.767 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:25.767 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "diskprediction_local", 2023-07-18T03:39:25.768 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:25.768 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:25.768 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:25.768 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:25.768 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:25.769 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.769 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.769 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.769 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.769 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.770 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.770 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:25.770 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:25.770 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:25.770 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:25.771 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:25.771 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:25.771 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:25.771 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:25.771 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.772 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.772 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.772 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.772 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.772 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:25.773 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:25.773 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.773 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.773 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.773 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.774 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.774 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.774 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.774 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.774 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.774 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.775 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.775 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.775 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:25.775 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:25.775 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.776 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.776 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.776 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:25.776 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.776 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.777 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:25.777 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:25.777 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:25.777 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:25.778 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:25.778 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:25.778 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:25.778 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:25.778 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.779 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.779 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.779 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.779 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.779 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:25.780 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:25.780 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.780 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.780 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.780 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.780 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.781 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.781 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.781 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.781 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.781 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.782 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.782 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.782 INFO:teuthology.orchestra.run.smithi101.stdout: "predict_interval": { 2023-07-18T03:39:25.782 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "predict_interval", 2023-07-18T03:39:25.782 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.783 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.783 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.783 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "86400", 2023-07-18T03:39:25.783 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.783 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.784 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.784 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.784 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.784 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.784 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.785 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.785 INFO:teuthology.orchestra.run.smithi101.stdout: "predictor_model": { 2023-07-18T03:39:25.785 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "predictor_model", 2023-07-18T03:39:25.785 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.785 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.786 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.786 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "prophetstor", 2023-07-18T03:39:25.786 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.786 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.786 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.787 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.787 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.787 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.787 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.787 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.788 INFO:teuthology.orchestra.run.smithi101.stdout: "sleep_interval": { 2023-07-18T03:39:25.788 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "sleep_interval", 2023-07-18T03:39:25.788 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.788 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.788 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.789 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "600", 2023-07-18T03:39:25.789 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.789 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.789 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.789 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.790 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.790 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.790 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.790 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:25.790 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:25.791 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.791 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:25.791 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "influx", 2023-07-18T03:39:25.791 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": false, 2023-07-18T03:39:25.791 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "influxdb python module not found", 2023-07-18T03:39:25.791 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:25.792 INFO:teuthology.orchestra.run.smithi101.stdout: "batch_size": { 2023-07-18T03:39:25.792 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "batch_size", 2023-07-18T03:39:25.792 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:25.792 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.792 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.793 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "5000", 2023-07-18T03:39:25.793 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.793 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.793 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.793 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-07-18T03:39:25.794 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.794 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.794 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.794 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.794 INFO:teuthology.orchestra.run.smithi101.stdout: "database": { 2023-07-18T03:39:25.795 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "database", 2023-07-18T03:39:25.795 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.795 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.795 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.795 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "ceph", 2023-07-18T03:39:25.796 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.796 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.796 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.796 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-07-18T03:39:25.796 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.797 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.797 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.797 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.797 INFO:teuthology.orchestra.run.smithi101.stdout: "hostname": { 2023-07-18T03:39:25.797 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "hostname", 2023-07-18T03:39:25.798 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.798 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.798 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.798 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.798 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.799 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.799 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.799 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "InfluxDB server hostname", 2023-07-18T03:39:25.799 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.799 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.800 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.800 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.800 INFO:teuthology.orchestra.run.smithi101.stdout: "interval": { 2023-07-18T03:39:25.800 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "interval", 2023-07-18T03:39:25.800 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:25.800 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.801 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.801 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "30", 2023-07-18T03:39:25.801 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "5", 2023-07-18T03:39:25.801 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.801 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.802 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-07-18T03:39:25.802 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.802 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.802 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.802 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.803 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:25.803 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:25.803 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.803 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.803 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.804 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.804 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.804 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.804 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:25.804 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:25.805 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:25.805 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:25.805 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:25.805 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:25.805 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:25.806 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:25.806 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.806 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.806 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.806 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.807 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.807 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:25.807 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:25.807 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.807 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.808 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.808 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.808 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.808 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.808 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.809 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.809 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.809 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.809 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.809 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.810 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:25.810 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:25.810 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.810 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.810 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.811 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:25.811 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.811 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.811 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:25.811 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:25.811 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:25.812 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:25.812 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:25.812 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:25.812 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:25.812 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:25.813 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.813 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.813 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.813 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.813 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.814 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:25.814 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:25.814 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.814 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.814 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.815 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.815 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.815 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.815 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.815 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.816 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.816 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.816 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.816 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.816 INFO:teuthology.orchestra.run.smithi101.stdout: "password": { 2023-07-18T03:39:25.817 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "password", 2023-07-18T03:39:25.817 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.817 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.817 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.817 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.818 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.818 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.818 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.818 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "password of InfluxDB server user", 2023-07-18T03:39:25.818 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.819 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.819 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.819 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.819 INFO:teuthology.orchestra.run.smithi101.stdout: "port": { 2023-07-18T03:39:25.819 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "port", 2023-07-18T03:39:25.819 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:25.820 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.820 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.820 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "8086", 2023-07-18T03:39:25.820 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.820 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.821 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.821 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "InfluxDB server port", 2023-07-18T03:39:25.821 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.821 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.821 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.822 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.822 INFO:teuthology.orchestra.run.smithi101.stdout: "ssl": { 2023-07-18T03:39:25.822 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "ssl", 2023-07-18T03:39:25.822 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.822 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.823 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.823 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "false", 2023-07-18T03:39:25.823 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.823 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.823 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.824 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-07-18T03:39:25.824 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.824 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.824 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.824 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.825 INFO:teuthology.orchestra.run.smithi101.stdout: "threads": { 2023-07-18T03:39:25.825 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "threads", 2023-07-18T03:39:25.825 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:25.825 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.825 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.826 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "5", 2023-07-18T03:39:25.826 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "1", 2023-07-18T03:39:25.826 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "32", 2023-07-18T03:39:25.826 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.826 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-07-18T03:39:25.827 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.827 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.827 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.827 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.827 INFO:teuthology.orchestra.run.smithi101.stdout: "username": { 2023-07-18T03:39:25.827 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "username", 2023-07-18T03:39:25.828 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.828 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.828 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.828 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.828 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.829 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.829 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.829 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "username of InfluxDB server user", 2023-07-18T03:39:25.829 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.829 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.830 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.830 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.830 INFO:teuthology.orchestra.run.smithi101.stdout: "verify_ssl": { 2023-07-18T03:39:25.830 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "verify_ssl", 2023-07-18T03:39:25.830 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.831 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.831 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.831 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "true", 2023-07-18T03:39:25.831 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.831 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.832 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.832 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-07-18T03:39:25.832 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.832 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.832 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.833 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:25.833 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:25.833 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.833 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:25.833 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "insights", 2023-07-18T03:39:25.834 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:25.834 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:25.834 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:25.834 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:25.834 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:25.834 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.835 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.835 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.835 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.835 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.835 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.836 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:25.836 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:25.836 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:25.836 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:25.836 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:25.837 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:25.837 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:25.837 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:25.837 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.837 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.838 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.838 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.838 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.838 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:25.838 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:25.839 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.839 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.839 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.839 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.839 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.840 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.840 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.840 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.840 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.840 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.841 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.841 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.841 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:25.841 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:25.841 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.842 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.842 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.842 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:25.842 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.842 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.843 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:25.843 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:25.843 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:25.843 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:25.843 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:25.844 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:25.844 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:25.844 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:25.844 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.844 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.845 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.845 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.845 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.845 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:25.845 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:25.846 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.846 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.846 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.846 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.846 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.847 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.847 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.847 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.847 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.847 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.848 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.848 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:25.848 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:25.848 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.848 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:25.849 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "iostat", 2023-07-18T03:39:25.849 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:25.849 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:25.849 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:25.849 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:25.849 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:25.850 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.850 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.850 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.850 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.850 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.851 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.851 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:25.851 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:25.851 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:25.851 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:25.852 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:25.852 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:25.852 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:25.852 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:25.852 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.853 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.853 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.853 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.853 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.853 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:25.854 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:25.854 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.854 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.854 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.854 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.855 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.855 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.855 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.855 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.855 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.856 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.856 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.856 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.856 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:25.856 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:25.857 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.857 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.857 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.857 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:25.857 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.858 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.858 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:25.858 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:25.858 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:25.858 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:25.859 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:25.859 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:25.859 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:25.859 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:25.859 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.859 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.860 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.860 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.860 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.860 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:25.860 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:25.861 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.861 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.861 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.861 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.861 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.862 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.862 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.862 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.862 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.862 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.863 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.863 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:25.863 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:25.863 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.863 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:25.864 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "k8sevents", 2023-07-18T03:39:25.864 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:25.864 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:25.864 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:25.864 INFO:teuthology.orchestra.run.smithi101.stdout: "ceph_event_retention_days": { 2023-07-18T03:39:25.865 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "ceph_event_retention_days", 2023-07-18T03:39:25.865 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:25.865 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.865 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.865 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "7", 2023-07-18T03:39:25.866 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.866 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.866 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.866 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Days to hold ceph event information within local cache", 2023-07-18T03:39:25.866 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.867 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.867 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.867 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.867 INFO:teuthology.orchestra.run.smithi101.stdout: "config_check_secs": { 2023-07-18T03:39:25.867 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "config_check_secs", 2023-07-18T03:39:25.868 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:25.868 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.868 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.868 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "10", 2023-07-18T03:39:25.868 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "10", 2023-07-18T03:39:25.869 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.869 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.869 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-07-18T03:39:25.869 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.869 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.869 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.870 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.870 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:25.870 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:25.870 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.870 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.871 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.871 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.871 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.871 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.871 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:25.872 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:25.872 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:25.872 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:25.872 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:25.872 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:25.873 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:25.873 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:25.873 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.873 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.873 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.874 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.874 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.874 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:25.874 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:25.874 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.875 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.875 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.875 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.875 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.875 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.875 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.876 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.876 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.876 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.876 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.876 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.877 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:25.877 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:25.877 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.877 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.877 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.878 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:25.878 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.878 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.878 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:25.878 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:25.879 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:25.879 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:25.879 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:25.879 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:25.879 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:25.880 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:25.880 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.880 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.880 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.880 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.881 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.881 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:25.881 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:25.881 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.881 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.882 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.882 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.882 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.882 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.882 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.883 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.883 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.883 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.883 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.883 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:25.884 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:25.884 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.884 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:25.884 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "localpool", 2023-07-18T03:39:25.884 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:25.885 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:25.885 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:25.885 INFO:teuthology.orchestra.run.smithi101.stdout: "failure_domain": { 2023-07-18T03:39:25.885 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "failure_domain", 2023-07-18T03:39:25.885 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.886 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.886 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.886 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "host", 2023-07-18T03:39:25.886 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.886 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.887 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.887 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "failure domain for any created local pool", 2023-07-18T03:39:25.887 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-07-18T03:39:25.887 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.887 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.888 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.888 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:25.888 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:25.888 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.888 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.889 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.889 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.889 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.889 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.889 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:25.890 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:25.890 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:25.890 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:25.890 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:25.890 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:25.891 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:25.891 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:25.891 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.891 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.891 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.891 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.892 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.892 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:25.892 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:25.892 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.892 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.893 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.893 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.893 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.893 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.893 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.894 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.894 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.894 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.894 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.894 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.895 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:25.895 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:25.895 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.895 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.895 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.896 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:25.896 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.896 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.896 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:25.896 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:25.897 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:25.897 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:25.897 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:25.897 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:25.897 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:25.898 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:25.898 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.898 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.898 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.898 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.899 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.899 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:25.899 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:25.899 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.899 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.899 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.900 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.900 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.900 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.900 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.900 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.901 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.901 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.901 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.901 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.901 INFO:teuthology.orchestra.run.smithi101.stdout: "min_size": { 2023-07-18T03:39:25.902 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "min_size", 2023-07-18T03:39:25.902 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:25.902 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.902 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.902 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.903 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.903 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.903 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.903 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "default min_size for any created local pool", 2023-07-18T03:39:25.903 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-07-18T03:39:25.904 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.904 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.904 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.904 INFO:teuthology.orchestra.run.smithi101.stdout: "num_rep": { 2023-07-18T03:39:25.904 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "num_rep", 2023-07-18T03:39:25.905 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:25.905 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.905 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.905 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "3", 2023-07-18T03:39:25.905 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.906 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.906 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.906 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "default replica count for any created local pool", 2023-07-18T03:39:25.906 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.906 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.907 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.907 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.907 INFO:teuthology.orchestra.run.smithi101.stdout: "pg_num": { 2023-07-18T03:39:25.907 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "pg_num", 2023-07-18T03:39:25.907 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:25.907 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.908 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.908 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "128", 2023-07-18T03:39:25.908 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.908 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.908 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.909 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "default pg_num for any created local pool", 2023-07-18T03:39:25.909 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.909 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.909 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.909 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.910 INFO:teuthology.orchestra.run.smithi101.stdout: "prefix": { 2023-07-18T03:39:25.910 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "prefix", 2023-07-18T03:39:25.910 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.910 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.910 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.911 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.911 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.911 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.911 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.911 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "name prefix for any created local pool", 2023-07-18T03:39:25.912 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.912 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.912 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.912 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.912 INFO:teuthology.orchestra.run.smithi101.stdout: "subtree": { 2023-07-18T03:39:25.913 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "subtree", 2023-07-18T03:39:25.913 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.913 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.913 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.913 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "rack", 2023-07-18T03:39:25.914 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.914 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.914 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.914 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "CRUSH level for which to create a local pool", 2023-07-18T03:39:25.914 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-07-18T03:39:25.915 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.915 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.915 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:25.915 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:25.915 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.916 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:25.916 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "mds_autoscaler", 2023-07-18T03:39:25.916 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:25.916 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:25.916 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:25.917 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:25.917 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:25.917 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.917 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.917 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.918 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.918 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.918 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.918 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:25.918 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:25.919 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:25.919 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:25.919 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:25.919 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:25.919 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:25.920 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:25.920 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.920 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.920 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.920 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.921 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.921 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:25.921 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:25.921 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.921 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.922 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.922 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.922 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.922 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.922 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.923 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.923 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.923 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.923 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.923 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.923 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:25.924 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:25.924 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.924 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.924 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.924 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:25.925 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.925 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.925 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:25.925 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:25.925 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:25.926 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:25.926 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:25.926 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:25.926 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:25.926 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:25.927 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.927 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.927 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.927 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.927 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.928 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:25.928 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:25.928 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.928 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.928 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.929 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.929 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.929 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.929 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.929 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.930 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.930 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.930 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.930 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:25.930 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:25.931 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.931 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:25.931 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "mirroring", 2023-07-18T03:39:25.931 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:25.931 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:25.932 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:25.932 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:25.932 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:25.932 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.932 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.933 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.933 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.933 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.933 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.933 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:25.933 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:25.934 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:25.934 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:25.934 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:25.934 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:25.934 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:25.935 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:25.935 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.935 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.935 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.935 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.936 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.936 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:25.936 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:25.936 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.936 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.937 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.937 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.937 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.937 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.937 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.938 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.938 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.938 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.938 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.938 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.939 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:25.939 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:25.939 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.939 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.939 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.940 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:25.940 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.940 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.940 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:25.940 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:25.941 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:25.941 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:25.941 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:25.941 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:25.941 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:25.941 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:25.942 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.942 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.942 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.942 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.942 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.943 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:25.943 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:25.943 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.943 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.943 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.944 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.944 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.944 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.944 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.944 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.945 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.945 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.945 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.945 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:25.945 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:25.946 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.946 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:25.946 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "nfs", 2023-07-18T03:39:25.946 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:25.946 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:25.947 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:25.947 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:25.947 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:25.947 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.947 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.947 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.948 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.948 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.948 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.948 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:25.948 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:25.949 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:25.949 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:25.949 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:25.949 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:25.949 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:25.950 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:25.950 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.950 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.950 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.950 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.951 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.951 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:25.951 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:25.951 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.951 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.952 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.952 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.952 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.952 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.952 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.953 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.953 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.953 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.953 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.953 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.954 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:25.954 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:25.954 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.954 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.954 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.955 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:25.955 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.955 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.955 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:25.955 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:25.956 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:25.956 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:25.956 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:25.956 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:25.956 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:25.957 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:25.957 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.957 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.957 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.957 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.958 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.958 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:25.958 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:25.958 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.958 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.959 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.959 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.959 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.959 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.959 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.960 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.960 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.960 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.960 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.960 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:25.961 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:25.961 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.961 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:25.961 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "orchestrator", 2023-07-18T03:39:25.961 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:25.961 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:25.962 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:25.962 INFO:teuthology.orchestra.run.smithi101.stdout: "fail_fs": { 2023-07-18T03:39:25.962 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "fail_fs", 2023-07-18T03:39:25.962 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.962 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.963 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:25.963 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.963 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.963 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.963 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.964 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-07-18T03:39:25.964 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.964 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.964 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.964 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.965 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:25.965 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:25.965 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.965 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.965 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.966 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.966 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.966 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.966 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:25.966 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:25.967 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:25.967 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:25.967 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:25.967 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:25.967 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:25.968 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:25.968 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.968 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.968 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.968 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.969 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.969 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:25.969 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:25.969 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.969 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.969 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.970 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.970 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.970 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.970 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.970 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.971 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.971 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.971 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.971 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.971 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:25.972 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:25.972 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.972 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.972 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.972 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:25.973 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.973 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.973 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:25.973 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:25.973 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:25.974 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:25.974 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:25.974 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:25.974 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:25.974 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:25.975 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.975 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.975 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.975 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.975 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.976 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:25.976 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:25.976 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.976 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.976 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.976 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.977 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.977 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.977 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.977 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.977 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.978 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.978 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.978 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.978 INFO:teuthology.orchestra.run.smithi101.stdout: "orchestrator": { 2023-07-18T03:39:25.978 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "orchestrator", 2023-07-18T03:39:25.979 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.979 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.979 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.979 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.979 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.980 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.980 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:25.980 INFO:teuthology.orchestra.run.smithi101.stdout: "cephadm", 2023-07-18T03:39:25.980 INFO:teuthology.orchestra.run.smithi101.stdout: "rook", 2023-07-18T03:39:25.980 INFO:teuthology.orchestra.run.smithi101.stdout: "test_orchestrator" 2023-07-18T03:39:25.981 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:25.981 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Orchestrator backend", 2023-07-18T03:39:25.981 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.981 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.981 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.982 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:25.982 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:25.982 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.982 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:25.982 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "osd_perf_query", 2023-07-18T03:39:25.983 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:25.983 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:25.983 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:25.983 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:25.983 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:25.983 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.984 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.984 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.984 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.984 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.984 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.985 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:25.985 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:25.985 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:25.985 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:25.985 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:25.986 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:25.986 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:25.986 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:25.986 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.986 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.987 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.987 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.987 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.987 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:25.987 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:25.988 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.988 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.988 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.988 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.988 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.989 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.989 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.989 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.989 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.989 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.990 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.990 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.990 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:25.990 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:25.990 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.991 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.991 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.991 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:25.991 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.991 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.991 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:25.992 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:25.992 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:25.992 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:25.992 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:25.992 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:25.993 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:25.993 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:25.993 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.993 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.994 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.994 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.994 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.994 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:25.994 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:25.995 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:25.995 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.995 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.995 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:25.995 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:25.996 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:25.996 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:25.996 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:25.996 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:25.996 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:25.997 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:25.997 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:25.997 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:25.997 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:25.997 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:25.997 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "osd_support", 2023-07-18T03:39:25.998 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:25.998 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:25.998 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:25.998 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:25.998 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:25.999 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:25.999 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:25.999 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:25.999 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:25.999 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.000 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.000 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.000 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.000 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.000 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.001 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.001 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.001 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.001 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.001 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.002 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.002 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.002 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.002 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.002 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:26.003 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:26.003 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.003 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.003 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.003 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.004 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.004 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.004 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.004 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.004 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.005 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.005 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.005 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.005 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:26.005 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:26.006 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.006 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.006 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.006 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:26.006 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.007 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.007 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.007 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.007 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.007 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.007 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.008 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.008 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.008 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.008 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.008 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.009 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.009 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.009 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.009 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:26.009 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:26.010 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.010 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.010 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.010 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.010 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.011 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.011 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.011 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.011 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.011 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.012 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.012 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.012 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.012 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.012 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:26.013 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "pg_autoscaler", 2023-07-18T03:39:26.013 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:26.013 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:26.013 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:26.013 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:26.014 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:26.014 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.014 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.014 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.014 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.015 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.015 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.015 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.015 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.015 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.016 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.016 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.016 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.016 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.016 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.016 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.017 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.017 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.017 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.017 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.017 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:26.018 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:26.018 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.018 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.018 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.018 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.019 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.019 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.019 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.019 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.019 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.020 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.020 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.020 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.020 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:26.020 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:26.021 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.021 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.021 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.021 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:26.021 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.022 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.022 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.022 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.022 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.022 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.023 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.023 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.023 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.023 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.023 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.024 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.024 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.024 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.024 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.024 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:26.025 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:26.025 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.025 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.025 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.025 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.025 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.026 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.026 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.026 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.026 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.026 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.027 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.027 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.027 INFO:teuthology.orchestra.run.smithi101.stdout: "noautoscale": { 2023-07-18T03:39:26.027 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "noautoscale", 2023-07-18T03:39:26.027 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.028 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.028 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.028 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.028 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.028 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.029 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.029 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "global autoscale flag", 2023-07-18T03:39:26.029 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-07-18T03:39:26.029 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.029 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.030 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.030 INFO:teuthology.orchestra.run.smithi101.stdout: "sleep_interval": { 2023-07-18T03:39:26.030 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "sleep_interval", 2023-07-18T03:39:26.030 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:26.030 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.031 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.031 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "60", 2023-07-18T03:39:26.031 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.031 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.031 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.031 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.032 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.032 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.032 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.032 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.033 INFO:teuthology.orchestra.run.smithi101.stdout: "threshold": { 2023-07-18T03:39:26.033 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "threshold", 2023-07-18T03:39:26.033 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "float", 2023-07-18T03:39:26.033 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.033 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.034 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "3.0", 2023-07-18T03:39:26.034 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "1.0", 2023-07-18T03:39:26.034 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.034 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.034 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "scaling threshold", 2023-07-18T03:39:26.035 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-07-18T03:39:26.035 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.035 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.035 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.035 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.036 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.036 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:26.036 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "progress", 2023-07-18T03:39:26.036 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:26.036 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:26.037 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:26.037 INFO:teuthology.orchestra.run.smithi101.stdout: "allow_pg_recovery_event": { 2023-07-18T03:39:26.037 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "allow_pg_recovery_event", 2023-07-18T03:39:26.037 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.037 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.037 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.038 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.038 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.038 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.038 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.038 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "allow the module to show pg recovery progress", 2023-07-18T03:39:26.039 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.039 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.039 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.039 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.039 INFO:teuthology.orchestra.run.smithi101.stdout: "enabled": { 2023-07-18T03:39:26.040 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "enabled", 2023-07-18T03:39:26.040 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.040 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.040 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.040 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:26.041 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.041 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.041 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.041 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.041 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.042 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.042 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.042 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.042 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:26.042 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:26.043 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.043 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.043 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.043 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.043 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.043 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.044 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.044 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.044 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.044 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.044 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.045 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.045 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.045 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.045 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.045 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.046 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.046 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.046 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.046 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:26.046 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:26.047 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.047 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.047 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.047 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.047 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.048 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.048 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.048 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.048 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.048 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.049 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.049 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.049 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:26.049 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:26.049 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.050 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.050 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.050 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:26.050 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.050 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.051 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.051 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.051 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.051 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.051 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.052 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.052 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.052 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.052 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.052 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.053 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.053 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.053 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.053 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:26.053 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:26.053 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.054 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.054 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.054 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.054 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.054 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.055 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.055 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.055 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.055 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.055 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.056 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.056 INFO:teuthology.orchestra.run.smithi101.stdout: "max_completed_events": { 2023-07-18T03:39:26.056 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "max_completed_events", 2023-07-18T03:39:26.056 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:26.056 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.057 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.057 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "50", 2023-07-18T03:39:26.057 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.057 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.057 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.058 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "number of past completed events to remember", 2023-07-18T03:39:26.058 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.058 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.058 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.058 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.059 INFO:teuthology.orchestra.run.smithi101.stdout: "sleep_interval": { 2023-07-18T03:39:26.059 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "sleep_interval", 2023-07-18T03:39:26.059 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:26.059 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.059 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.059 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "5", 2023-07-18T03:39:26.060 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.060 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.060 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.060 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "how long the module is going to sleep", 2023-07-18T03:39:26.060 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.061 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.061 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.061 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.061 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.061 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.062 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:26.062 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "prometheus", 2023-07-18T03:39:26.062 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:26.062 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:26.062 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:26.063 INFO:teuthology.orchestra.run.smithi101.stdout: "cache": { 2023-07-18T03:39:26.063 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "cache", 2023-07-18T03:39:26.063 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.063 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.063 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.064 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:26.064 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.064 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.064 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.064 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.065 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.065 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.065 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.065 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.065 INFO:teuthology.orchestra.run.smithi101.stdout: "exclude_perf_counters": { 2023-07-18T03:39:26.066 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "exclude_perf_counters", 2023-07-18T03:39:26.066 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.066 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.066 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.066 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:26.067 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.067 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.067 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.067 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-07-18T03:39:26.067 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-07-18T03:39:26.068 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.068 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.068 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.068 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:26.068 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:26.069 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.069 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.069 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.069 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.069 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.070 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.070 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.070 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.070 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.070 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.071 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.071 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.071 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.071 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.071 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.072 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.072 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.072 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.072 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.072 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:26.073 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:26.073 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.073 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.073 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.073 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.073 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.074 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.074 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.074 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.074 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.074 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.075 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.075 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.075 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:26.075 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:26.075 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.076 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.076 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.076 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:26.076 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.076 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.077 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.077 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.077 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.077 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.077 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.078 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.078 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.078 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.078 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.078 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.079 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.079 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.079 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.079 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:26.079 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:26.080 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.080 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.080 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.080 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.080 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.081 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.081 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.081 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.081 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.081 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.082 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.082 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.082 INFO:teuthology.orchestra.run.smithi101.stdout: "rbd_stats_pools": { 2023-07-18T03:39:26.082 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "rbd_stats_pools", 2023-07-18T03:39:26.082 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.082 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.083 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.083 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.083 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.083 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.083 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.084 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.084 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.084 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.084 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.084 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.085 INFO:teuthology.orchestra.run.smithi101.stdout: "rbd_stats_pools_refresh_interval": { 2023-07-18T03:39:26.085 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-07-18T03:39:26.085 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:26.085 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.085 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.086 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "300", 2023-07-18T03:39:26.086 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.086 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.086 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.086 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.087 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.087 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.087 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.087 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.087 INFO:teuthology.orchestra.run.smithi101.stdout: "scrape_interval": { 2023-07-18T03:39:26.088 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "scrape_interval", 2023-07-18T03:39:26.088 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "float", 2023-07-18T03:39:26.088 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.088 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.088 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "15.0", 2023-07-18T03:39:26.089 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.089 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.089 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.089 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.089 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.090 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.090 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.090 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.090 INFO:teuthology.orchestra.run.smithi101.stdout: "server_addr": { 2023-07-18T03:39:26.090 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "server_addr", 2023-07-18T03:39:26.090 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.091 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.091 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.091 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "::", 2023-07-18T03:39:26.091 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.091 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.092 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.092 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-07-18T03:39:26.092 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.092 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.092 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.093 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.093 INFO:teuthology.orchestra.run.smithi101.stdout: "server_port": { 2023-07-18T03:39:26.093 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "server_port", 2023-07-18T03:39:26.093 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:26.093 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.094 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.094 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "9283", 2023-07-18T03:39:26.094 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.094 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.094 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.095 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-07-18T03:39:26.095 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.095 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.095 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.095 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.096 INFO:teuthology.orchestra.run.smithi101.stdout: "stale_cache_strategy": { 2023-07-18T03:39:26.096 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "stale_cache_strategy", 2023-07-18T03:39:26.096 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.096 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.096 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.097 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "log", 2023-07-18T03:39:26.097 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.097 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.097 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.097 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.098 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.098 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.098 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.098 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.098 INFO:teuthology.orchestra.run.smithi101.stdout: "standby_behaviour": { 2023-07-18T03:39:26.098 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "standby_behaviour", 2023-07-18T03:39:26.099 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.099 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.099 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.099 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "default", 2023-07-18T03:39:26.099 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.100 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.100 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.100 INFO:teuthology.orchestra.run.smithi101.stdout: "default", 2023-07-18T03:39:26.100 INFO:teuthology.orchestra.run.smithi101.stdout: "error" 2023-07-18T03:39:26.100 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.101 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.101 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.101 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.101 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.101 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.102 INFO:teuthology.orchestra.run.smithi101.stdout: "standby_error_status_code": { 2023-07-18T03:39:26.102 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "standby_error_status_code", 2023-07-18T03:39:26.102 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:26.102 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.102 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.103 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "500", 2023-07-18T03:39:26.103 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "400", 2023-07-18T03:39:26.103 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "599", 2023-07-18T03:39:26.103 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.103 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.104 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.104 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.104 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.104 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.104 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.105 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.105 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:26.105 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "rbd_support", 2023-07-18T03:39:26.105 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:26.105 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:26.106 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:26.106 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:26.106 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:26.106 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.106 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.107 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.107 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.107 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.107 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.107 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.108 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.108 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.108 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.108 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.108 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.109 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.109 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.109 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.109 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.109 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.110 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.110 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.110 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:26.110 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:26.110 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.111 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.111 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.111 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.111 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.111 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.111 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.112 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.112 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.112 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.112 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.112 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.113 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:26.113 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:26.113 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.113 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.113 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.114 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:26.114 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.114 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.114 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.114 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.115 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.115 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.115 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.115 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.115 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.116 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.116 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.116 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.116 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.116 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.117 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.117 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:26.117 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:26.117 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.117 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.118 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.118 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.118 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.118 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.118 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.119 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.119 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.119 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.119 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.119 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.119 INFO:teuthology.orchestra.run.smithi101.stdout: "max_concurrent_snap_create": { 2023-07-18T03:39:26.120 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "max_concurrent_snap_create", 2023-07-18T03:39:26.120 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:26.120 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.120 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.120 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "10", 2023-07-18T03:39:26.121 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.121 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.121 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.121 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.121 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.122 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.122 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.122 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.122 INFO:teuthology.orchestra.run.smithi101.stdout: "mirror_snapshot_schedule": { 2023-07-18T03:39:26.122 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "mirror_snapshot_schedule", 2023-07-18T03:39:26.123 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.123 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.123 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.123 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.123 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.124 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.124 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.124 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.124 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.124 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.125 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.125 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.125 INFO:teuthology.orchestra.run.smithi101.stdout: "trash_purge_schedule": { 2023-07-18T03:39:26.125 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "trash_purge_schedule", 2023-07-18T03:39:26.125 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.126 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.126 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.126 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.126 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.126 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.126 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.127 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.127 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.127 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.127 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.127 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.128 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.128 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.128 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:26.128 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "restful", 2023-07-18T03:39:26.128 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:26.129 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:26.129 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:26.129 INFO:teuthology.orchestra.run.smithi101.stdout: "enable_auth": { 2023-07-18T03:39:26.129 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "enable_auth", 2023-07-18T03:39:26.129 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.130 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.130 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.130 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:26.130 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.130 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.131 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.131 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.131 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.131 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.131 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.132 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.132 INFO:teuthology.orchestra.run.smithi101.stdout: "key_file": { 2023-07-18T03:39:26.132 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "key_file", 2023-07-18T03:39:26.132 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.132 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.133 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.133 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.133 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.133 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.133 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.134 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.134 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.134 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.134 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.134 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.134 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:26.135 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:26.135 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.135 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.135 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.135 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.136 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.136 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.136 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.136 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.136 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.137 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.137 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.137 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.137 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.137 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.138 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.138 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.138 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.138 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.138 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.139 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:26.139 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:26.139 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.139 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.139 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.140 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.140 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.140 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.140 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.140 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.141 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.141 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.141 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.141 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.141 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:26.142 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:26.142 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.142 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.142 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.143 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:26.143 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.143 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.143 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.143 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.144 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.144 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.144 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.144 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.144 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.145 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.145 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.145 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.145 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.145 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.146 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.146 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:26.146 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:26.146 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.146 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.147 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.147 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.147 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.147 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.147 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.148 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.148 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.148 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.148 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.148 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.149 INFO:teuthology.orchestra.run.smithi101.stdout: "server_addr": { 2023-07-18T03:39:26.149 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "server_addr", 2023-07-18T03:39:26.149 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.149 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.149 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.149 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.150 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.150 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.150 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.150 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.150 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.151 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.151 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.151 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.151 INFO:teuthology.orchestra.run.smithi101.stdout: "server_port": { 2023-07-18T03:39:26.151 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "server_port", 2023-07-18T03:39:26.152 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.152 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.152 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.152 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.152 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.153 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.153 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.153 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.153 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.153 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.154 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.154 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.154 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.154 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.154 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:26.155 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "rgw", 2023-07-18T03:39:26.155 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:26.155 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:26.155 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:26.155 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:26.156 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:26.156 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.156 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.156 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.156 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.156 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.157 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.157 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.157 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.157 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.157 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.158 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.158 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.158 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.158 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.158 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.159 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.159 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.159 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.159 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.159 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:26.160 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:26.160 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.160 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.160 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.160 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.161 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.161 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.161 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.161 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.161 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.162 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.162 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.162 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.162 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:26.162 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:26.163 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.163 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.163 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.163 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:26.163 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.163 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.164 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.164 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.164 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.164 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.164 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.165 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.165 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.165 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.165 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.165 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.166 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.166 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.166 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.166 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:26.166 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:26.167 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.167 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.167 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.167 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.167 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.168 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.168 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.168 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.168 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.168 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.169 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.169 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.169 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.169 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.169 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:26.170 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "rook", 2023-07-18T03:39:26.170 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:26.170 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:26.170 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:26.170 INFO:teuthology.orchestra.run.smithi101.stdout: "drive_group_interval": { 2023-07-18T03:39:26.171 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "drive_group_interval", 2023-07-18T03:39:26.171 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "float", 2023-07-18T03:39:26.171 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.171 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.171 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "300.0", 2023-07-18T03:39:26.172 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.172 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.172 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.172 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-07-18T03:39:26.172 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.173 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.173 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.173 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.173 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:26.173 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:26.174 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.174 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.174 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.174 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.174 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.175 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.175 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.175 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.175 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.175 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.175 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.176 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.176 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.176 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.176 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.176 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.177 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.177 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.177 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.177 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:26.177 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:26.178 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.178 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.178 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.178 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.178 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.179 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.179 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.179 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.179 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.179 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.180 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.180 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.180 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:26.180 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:26.180 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.181 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.181 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.181 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:26.181 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.181 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.182 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.182 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.182 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.182 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.182 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.183 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.183 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.183 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.183 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.183 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.184 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.184 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.184 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.184 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:26.184 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:26.185 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.185 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.185 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.185 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.185 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.186 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.186 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.186 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.186 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.186 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.187 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.187 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.187 INFO:teuthology.orchestra.run.smithi101.stdout: "storage_class": { 2023-07-18T03:39:26.187 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "storage_class", 2023-07-18T03:39:26.187 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.188 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.188 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.188 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "local", 2023-07-18T03:39:26.188 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.188 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.189 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.189 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-07-18T03:39:26.189 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.189 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.189 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.189 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.190 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.190 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.190 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:26.190 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "selftest", 2023-07-18T03:39:26.190 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:26.191 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:26.191 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:26.191 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:26.191 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:26.191 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.192 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.192 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.192 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.192 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.192 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.193 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.193 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.193 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.193 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.193 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.194 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.194 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.194 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.194 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.194 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.195 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.195 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.195 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.195 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:26.195 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:26.196 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.196 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.196 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.196 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.196 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.197 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.197 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.197 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.197 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.197 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.197 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.198 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.198 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:26.198 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:26.198 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.198 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.199 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.199 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:26.199 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.199 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.199 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.200 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.200 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.200 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.200 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.200 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.201 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.201 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.201 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.201 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.201 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.202 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.202 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.202 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:26.202 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:26.202 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.203 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.203 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.203 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.203 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.203 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.204 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.204 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.204 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.204 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.204 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.205 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.205 INFO:teuthology.orchestra.run.smithi101.stdout: "roption1": { 2023-07-18T03:39:26.205 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "roption1", 2023-07-18T03:39:26.205 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.205 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.206 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.206 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.206 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.206 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.206 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.207 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.207 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.207 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.207 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.207 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.207 INFO:teuthology.orchestra.run.smithi101.stdout: "roption2": { 2023-07-18T03:39:26.208 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "roption2", 2023-07-18T03:39:26.208 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.208 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.208 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.208 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "xyz", 2023-07-18T03:39:26.209 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.209 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.209 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.209 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.209 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.210 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.210 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.210 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.210 INFO:teuthology.orchestra.run.smithi101.stdout: "rwoption1": { 2023-07-18T03:39:26.210 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "rwoption1", 2023-07-18T03:39:26.211 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.211 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.211 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.211 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.211 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.212 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.212 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.212 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.212 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.212 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.213 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.213 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.213 INFO:teuthology.orchestra.run.smithi101.stdout: "rwoption2": { 2023-07-18T03:39:26.213 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "rwoption2", 2023-07-18T03:39:26.213 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:26.214 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.214 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.214 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.214 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.214 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.215 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.215 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.215 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.215 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.215 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.215 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.216 INFO:teuthology.orchestra.run.smithi101.stdout: "rwoption3": { 2023-07-18T03:39:26.216 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "rwoption3", 2023-07-18T03:39:26.216 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "float", 2023-07-18T03:39:26.216 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.216 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.217 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.217 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.217 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.217 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.217 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.218 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.218 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.218 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.218 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.218 INFO:teuthology.orchestra.run.smithi101.stdout: "rwoption4": { 2023-07-18T03:39:26.219 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "rwoption4", 2023-07-18T03:39:26.219 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.219 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.219 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.219 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.220 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.220 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.220 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.220 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.220 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.221 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.221 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.221 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.221 INFO:teuthology.orchestra.run.smithi101.stdout: "rwoption5": { 2023-07-18T03:39:26.221 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "rwoption5", 2023-07-18T03:39:26.222 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.222 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.222 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.222 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.222 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.223 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.223 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.223 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.223 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.223 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.224 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.224 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.224 INFO:teuthology.orchestra.run.smithi101.stdout: "rwoption6": { 2023-07-18T03:39:26.224 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "rwoption6", 2023-07-18T03:39:26.224 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.225 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.225 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.225 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:26.225 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.225 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.226 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.226 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.226 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.226 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.226 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.227 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.227 INFO:teuthology.orchestra.run.smithi101.stdout: "rwoption7": { 2023-07-18T03:39:26.227 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "rwoption7", 2023-07-18T03:39:26.227 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:26.227 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.228 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.228 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.228 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "1", 2023-07-18T03:39:26.228 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "42", 2023-07-18T03:39:26.228 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.229 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.229 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.229 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.229 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.229 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.229 INFO:teuthology.orchestra.run.smithi101.stdout: "testkey": { 2023-07-18T03:39:26.230 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "testkey", 2023-07-18T03:39:26.230 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.230 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.230 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.230 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.231 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.231 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.231 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.231 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.231 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.232 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.232 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.232 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.232 INFO:teuthology.orchestra.run.smithi101.stdout: "testlkey": { 2023-07-18T03:39:26.232 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "testlkey", 2023-07-18T03:39:26.233 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.233 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.233 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.233 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.233 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.234 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.234 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.234 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.234 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.234 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.235 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.235 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.235 INFO:teuthology.orchestra.run.smithi101.stdout: "testnewline": { 2023-07-18T03:39:26.235 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "testnewline", 2023-07-18T03:39:26.235 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.236 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.236 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.236 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.236 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.236 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.237 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.237 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.237 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.237 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.237 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.237 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.238 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.238 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.238 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:26.238 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "snap_schedule", 2023-07-18T03:39:26.238 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:26.239 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:26.239 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:26.239 INFO:teuthology.orchestra.run.smithi101.stdout: "allow_m_granularity": { 2023-07-18T03:39:26.239 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "allow_m_granularity", 2023-07-18T03:39:26.239 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.240 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.240 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.240 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.240 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.240 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.241 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.241 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "allow minute scheduled snapshots", 2023-07-18T03:39:26.241 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.241 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.241 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.242 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.242 INFO:teuthology.orchestra.run.smithi101.stdout: "dump_on_update": { 2023-07-18T03:39:26.242 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "dump_on_update", 2023-07-18T03:39:26.242 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.242 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.243 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.243 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.243 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.243 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.243 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.244 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "dump database to debug log on update", 2023-07-18T03:39:26.244 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.244 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.244 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.244 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.245 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:26.245 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:26.245 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.245 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.245 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.246 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.246 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.246 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.246 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.246 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.246 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.247 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.247 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.247 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.247 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.247 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.248 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.248 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.248 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.248 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.248 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.249 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:26.249 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:26.249 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.249 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.249 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.250 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.250 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.250 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.250 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.250 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.251 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.251 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.251 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.251 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.251 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:26.252 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:26.252 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.252 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.252 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.252 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:26.253 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.253 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.253 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.253 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.253 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.253 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.254 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.254 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.254 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.254 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.254 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.255 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.255 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.255 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.255 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.255 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:26.256 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:26.256 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.256 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.256 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.256 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.257 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.257 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.257 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.257 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.257 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.258 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.258 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.258 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.258 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.258 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.259 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:26.259 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "stats", 2023-07-18T03:39:26.259 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:26.259 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:26.259 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:26.260 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:26.260 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:26.260 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.260 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.260 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.261 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.261 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.261 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.261 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.261 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.262 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.262 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.262 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.262 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.262 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.263 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.263 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.263 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.263 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.263 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.264 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.264 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:26.264 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:26.264 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.264 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.265 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.265 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.265 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.265 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.265 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.266 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.266 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.266 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.266 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.266 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.266 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:26.267 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:26.267 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.267 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.267 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.267 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:26.268 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.268 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.268 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.268 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.268 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.269 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.269 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.269 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.269 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.269 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.270 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.270 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.270 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.270 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.270 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.271 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:26.271 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:26.271 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.271 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.271 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.272 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.272 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.272 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.272 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.272 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.273 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.273 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.273 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.273 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.273 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.274 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.274 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:26.274 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "status", 2023-07-18T03:39:26.274 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:26.274 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:26.274 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:26.275 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:26.275 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:26.275 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.275 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.275 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.276 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.276 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.276 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.276 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.276 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.277 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.277 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.277 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.277 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.277 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.278 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.278 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.278 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.278 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.278 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.279 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.279 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:26.279 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:26.279 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.279 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.280 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.280 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.280 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.280 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.280 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.281 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.281 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.281 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.281 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.281 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.282 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:26.282 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:26.282 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.282 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.282 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.283 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:26.283 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.283 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.283 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.283 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.283 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.284 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.284 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.284 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.284 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.284 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.285 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.285 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.285 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.285 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.285 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.286 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:26.286 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:26.286 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.286 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.286 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.287 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.287 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.287 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.287 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.287 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.288 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.288 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.288 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.288 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.288 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.289 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.289 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:26.289 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "telegraf", 2023-07-18T03:39:26.289 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:26.290 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:26.290 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:26.290 INFO:teuthology.orchestra.run.smithi101.stdout: "address": { 2023-07-18T03:39:26.290 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "address", 2023-07-18T03:39:26.290 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.291 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.291 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.291 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-07-18T03:39:26.291 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.291 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.291 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.292 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.292 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.292 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.292 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.293 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.293 INFO:teuthology.orchestra.run.smithi101.stdout: "interval": { 2023-07-18T03:39:26.293 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "interval", 2023-07-18T03:39:26.293 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:26.293 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.294 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.294 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "15", 2023-07-18T03:39:26.294 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.294 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.294 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.295 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.295 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.295 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.295 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.296 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.296 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:26.296 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:26.296 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.297 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.297 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.297 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.297 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.298 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.298 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.298 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.298 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.298 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.299 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.299 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.299 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.299 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.299 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.300 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.300 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.300 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.300 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.300 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:26.301 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:26.301 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.301 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.301 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.301 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.302 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.302 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.302 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.302 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.302 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.303 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.303 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.303 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.303 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:26.303 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:26.304 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.304 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.304 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.304 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:26.304 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.305 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.305 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.305 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.305 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.305 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.305 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.306 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.306 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.306 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.306 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.306 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.307 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.307 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.307 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.307 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:26.307 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:26.308 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.308 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.308 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.308 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.308 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.309 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.309 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.309 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.309 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.309 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.310 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.310 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.310 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.310 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.310 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:26.311 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "telemetry", 2023-07-18T03:39:26.311 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:26.311 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:26.311 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:26.311 INFO:teuthology.orchestra.run.smithi101.stdout: "channel_basic": { 2023-07-18T03:39:26.312 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "channel_basic", 2023-07-18T03:39:26.312 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.312 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.312 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.312 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:26.313 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.313 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.313 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.313 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Share basic cluster information (size, version)", 2023-07-18T03:39:26.313 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.313 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.314 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.314 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.314 INFO:teuthology.orchestra.run.smithi101.stdout: "channel_crash": { 2023-07-18T03:39:26.314 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "channel_crash", 2023-07-18T03:39:26.314 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.315 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.315 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.315 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:26.315 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.315 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.316 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.316 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-07-18T03:39:26.316 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.316 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.316 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.317 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.317 INFO:teuthology.orchestra.run.smithi101.stdout: "channel_device": { 2023-07-18T03:39:26.317 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "channel_device", 2023-07-18T03:39:26.317 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.317 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.318 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.318 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:26.318 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.318 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.318 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.319 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-07-18T03:39:26.319 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.319 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.319 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.319 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.319 INFO:teuthology.orchestra.run.smithi101.stdout: "channel_ident": { 2023-07-18T03:39:26.320 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "channel_ident", 2023-07-18T03:39:26.320 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.320 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.320 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.320 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.321 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.321 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.321 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.321 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-07-18T03:39:26.321 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.322 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.322 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.322 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.322 INFO:teuthology.orchestra.run.smithi101.stdout: "channel_perf": { 2023-07-18T03:39:26.322 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "channel_perf", 2023-07-18T03:39:26.323 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.323 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.323 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.323 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.323 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.324 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.324 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.324 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Share various performance metrics of a cluster", 2023-07-18T03:39:26.324 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.324 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.325 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.325 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.325 INFO:teuthology.orchestra.run.smithi101.stdout: "contact": { 2023-07-18T03:39:26.325 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "contact", 2023-07-18T03:39:26.325 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.326 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.326 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.326 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.326 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.326 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.327 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.327 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.327 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.327 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.327 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.327 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.328 INFO:teuthology.orchestra.run.smithi101.stdout: "description": { 2023-07-18T03:39:26.328 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "description", 2023-07-18T03:39:26.328 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.328 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.329 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.329 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.329 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.329 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.329 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.329 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.330 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.330 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.330 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.330 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.330 INFO:teuthology.orchestra.run.smithi101.stdout: "device_url": { 2023-07-18T03:39:26.331 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "device_url", 2023-07-18T03:39:26.331 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.331 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.331 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.331 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-07-18T03:39:26.332 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.332 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.332 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.332 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.332 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.333 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.333 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.333 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.333 INFO:teuthology.orchestra.run.smithi101.stdout: "enabled": { 2023-07-18T03:39:26.333 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "enabled", 2023-07-18T03:39:26.334 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.334 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.334 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.334 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.334 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.335 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.335 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.335 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.335 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.335 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.336 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.336 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.336 INFO:teuthology.orchestra.run.smithi101.stdout: "interval": { 2023-07-18T03:39:26.336 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "interval", 2023-07-18T03:39:26.336 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:26.337 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.337 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.337 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "24", 2023-07-18T03:39:26.337 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "8", 2023-07-18T03:39:26.337 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.337 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.338 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.338 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.338 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.338 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.338 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.339 INFO:teuthology.orchestra.run.smithi101.stdout: "last_opt_revision": { 2023-07-18T03:39:26.339 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "last_opt_revision", 2023-07-18T03:39:26.339 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:26.339 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.339 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.340 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "1", 2023-07-18T03:39:26.340 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.340 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.340 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.340 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.341 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.341 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.341 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.341 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.341 INFO:teuthology.orchestra.run.smithi101.stdout: "leaderboard": { 2023-07-18T03:39:26.342 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "leaderboard", 2023-07-18T03:39:26.342 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.342 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.342 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.342 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.343 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.343 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.343 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.343 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.343 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.343 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.344 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.344 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.344 INFO:teuthology.orchestra.run.smithi101.stdout: "leaderboard_description": { 2023-07-18T03:39:26.344 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "leaderboard_description", 2023-07-18T03:39:26.344 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.345 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.345 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.345 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.345 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.345 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.346 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.346 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.346 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.346 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.346 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.347 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.347 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:26.347 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:26.347 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.347 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.348 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.348 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.348 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.348 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.348 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.349 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.349 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.349 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.349 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.349 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.350 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.350 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.350 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.350 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.350 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.351 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.351 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.351 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:26.351 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:26.351 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.351 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.352 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.352 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.352 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.352 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.353 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.353 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.353 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.353 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.353 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.354 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.354 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:26.354 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:26.354 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.354 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.354 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.355 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:26.355 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.355 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.355 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.355 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.356 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.356 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.356 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.356 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.356 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.357 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.357 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.357 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.357 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.357 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.358 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.358 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:26.358 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:26.358 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.358 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.359 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.359 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.359 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.359 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.359 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.360 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.360 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.360 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.360 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.360 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.360 INFO:teuthology.orchestra.run.smithi101.stdout: "organization": { 2023-07-18T03:39:26.361 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "organization", 2023-07-18T03:39:26.361 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.361 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.361 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.361 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.362 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.362 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.362 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.362 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.362 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.363 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.363 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.363 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.363 INFO:teuthology.orchestra.run.smithi101.stdout: "proxy": { 2023-07-18T03:39:26.363 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "proxy", 2023-07-18T03:39:26.364 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.364 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.364 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.364 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.364 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.365 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.365 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.365 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.365 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.365 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.366 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.366 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.366 INFO:teuthology.orchestra.run.smithi101.stdout: "url": { 2023-07-18T03:39:26.366 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "url", 2023-07-18T03:39:26.366 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.366 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.367 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.367 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-07-18T03:39:26.367 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.367 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.367 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.368 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.368 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.368 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.368 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.368 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.369 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.369 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.369 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:26.369 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "test_orchestrator", 2023-07-18T03:39:26.369 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:26.370 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:26.370 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:26.370 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:26.370 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:26.370 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.371 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.371 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.371 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.371 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.371 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.372 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.372 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.372 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.372 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.372 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.373 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.373 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.373 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.373 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.373 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.374 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.374 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.374 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.374 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:26.374 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:26.375 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.375 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.375 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.375 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.375 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.376 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.376 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.376 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.376 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.376 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.377 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.377 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.377 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:26.377 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:26.377 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.378 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.378 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.378 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:26.378 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.378 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.379 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.379 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.379 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.379 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.379 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.380 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.380 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.380 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.380 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.380 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.380 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.381 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.381 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.381 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:26.381 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:26.381 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.382 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.382 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.382 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.382 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.382 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.383 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.383 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.383 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.383 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.383 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.384 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.384 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.384 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.384 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:26.384 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "volumes", 2023-07-18T03:39:26.385 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:26.385 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:26.385 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:26.385 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:26.385 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:26.386 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.386 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.386 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.386 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.386 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.387 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.387 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.387 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.387 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.387 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.388 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.388 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.388 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.388 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.388 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.389 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.389 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.389 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.389 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.389 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:26.390 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:26.390 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.390 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.390 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.390 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.390 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.391 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.391 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.391 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.391 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.391 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.392 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.392 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.392 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:26.392 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:26.392 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.393 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.393 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.393 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:26.393 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.393 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.394 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.394 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.394 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.394 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.394 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.395 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.395 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.395 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.395 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.395 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.396 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.396 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.396 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.396 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:26.396 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:26.397 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.397 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.397 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.397 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.397 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.398 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.398 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.398 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.398 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.398 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.398 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.399 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.399 INFO:teuthology.orchestra.run.smithi101.stdout: "max_concurrent_clones": { 2023-07-18T03:39:26.399 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "max_concurrent_clones", 2023-07-18T03:39:26.399 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:26.399 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.400 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.400 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "4", 2023-07-18T03:39:26.400 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.400 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.400 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.401 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Number of asynchronous cloner threads", 2023-07-18T03:39:26.401 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.401 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.401 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.401 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.402 INFO:teuthology.orchestra.run.smithi101.stdout: "snapshot_clone_delay": { 2023-07-18T03:39:26.402 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "snapshot_clone_delay", 2023-07-18T03:39:26.402 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:26.402 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.402 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.403 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "0", 2023-07-18T03:39:26.403 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.403 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.403 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.403 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-07-18T03:39:26.404 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.404 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.404 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.404 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.404 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.405 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.405 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:26.405 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "zabbix", 2023-07-18T03:39:26.405 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:26.405 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:26.406 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:26.406 INFO:teuthology.orchestra.run.smithi101.stdout: "discovery_interval": { 2023-07-18T03:39:26.406 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "discovery_interval", 2023-07-18T03:39:26.406 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "uint", 2023-07-18T03:39:26.406 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.407 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.407 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "100", 2023-07-18T03:39:26.407 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.407 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.407 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.408 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.408 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.408 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.408 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.408 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.408 INFO:teuthology.orchestra.run.smithi101.stdout: "identifier": { 2023-07-18T03:39:26.409 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "identifier", 2023-07-18T03:39:26.409 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.409 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.409 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.410 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.410 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.410 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.410 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.410 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.411 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.411 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.411 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.411 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.411 INFO:teuthology.orchestra.run.smithi101.stdout: "interval": { 2023-07-18T03:39:26.412 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "interval", 2023-07-18T03:39:26.412 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:26.412 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.412 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.412 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "60", 2023-07-18T03:39:26.413 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.413 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.413 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.413 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.413 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.413 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.414 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.414 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.414 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:26.414 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:26.414 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.415 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.415 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.415 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.415 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.415 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.416 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.416 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.416 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.416 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.416 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.417 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.417 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.417 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.417 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.417 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.418 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.418 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.418 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.418 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:26.418 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:26.419 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.419 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.419 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.419 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.419 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.420 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.420 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.420 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.420 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.420 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.421 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.421 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.421 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:26.421 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:26.421 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.422 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.422 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.422 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:26.422 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.422 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.422 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.423 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.423 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.423 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.423 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.423 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.424 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.424 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.424 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.424 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.424 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.425 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.425 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.425 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:26.425 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:26.425 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.426 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.426 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.426 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.426 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.426 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.427 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.427 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.427 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.427 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.427 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.428 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.428 INFO:teuthology.orchestra.run.smithi101.stdout: "zabbix_host": { 2023-07-18T03:39:26.428 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "zabbix_host", 2023-07-18T03:39:26.428 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.428 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.429 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.429 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.429 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.429 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.429 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.430 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.430 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.430 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.430 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.430 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.430 INFO:teuthology.orchestra.run.smithi101.stdout: "zabbix_port": { 2023-07-18T03:39:26.431 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "zabbix_port", 2023-07-18T03:39:26.431 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:26.431 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.431 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.431 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "10051", 2023-07-18T03:39:26.432 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.432 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.432 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.432 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.432 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.433 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.433 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.433 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.433 INFO:teuthology.orchestra.run.smithi101.stdout: "zabbix_sender": { 2023-07-18T03:39:26.433 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "zabbix_sender", 2023-07-18T03:39:26.434 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.434 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.434 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.434 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-07-18T03:39:26.434 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.435 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.435 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.435 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.435 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.435 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.436 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.436 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.436 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.436 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.436 INFO:teuthology.orchestra.run.smithi101.stdout: ] 2023-07-18T03:39:26.437 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.437 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:26.437 INFO:teuthology.orchestra.run.smithi101.stdout: "gid": 25540, 2023-07-18T03:39:26.437 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "z", 2023-07-18T03:39:26.437 INFO:teuthology.orchestra.run.smithi101.stdout: "mgr_features": 4540138322906710015, 2023-07-18T03:39:26.437 INFO:teuthology.orchestra.run.smithi101.stdout: "available_modules": [ 2023-07-18T03:39:26.438 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:26.438 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "alerts", 2023-07-18T03:39:26.438 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:26.438 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:26.438 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:26.439 INFO:teuthology.orchestra.run.smithi101.stdout: "interval": { 2023-07-18T03:39:26.439 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "interval", 2023-07-18T03:39:26.439 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:26.439 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.439 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.440 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "60", 2023-07-18T03:39:26.440 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.440 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.440 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.440 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "How frequently to reexamine health status", 2023-07-18T03:39:26.441 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.441 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.441 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.441 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.441 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:26.442 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:26.442 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.442 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.442 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.442 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.443 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.443 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.443 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.443 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.443 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.444 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.444 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.444 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.444 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.444 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.445 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.445 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.445 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.445 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.445 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.446 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:26.446 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:26.446 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.446 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.446 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.447 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.447 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.447 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.447 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.447 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.447 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.448 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.448 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.448 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.448 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:26.449 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:26.449 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.449 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.449 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.449 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:26.450 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.450 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.450 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.450 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.450 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.451 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.451 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.451 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.451 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.451 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.451 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.452 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.452 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.452 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.452 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.452 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:26.453 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:26.453 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.453 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.453 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.453 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.454 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.454 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.454 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.454 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.454 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.455 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.455 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.455 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.455 INFO:teuthology.orchestra.run.smithi101.stdout: "smtp_destination": { 2023-07-18T03:39:26.455 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "smtp_destination", 2023-07-18T03:39:26.456 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.456 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.456 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.456 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.456 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.457 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.457 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.457 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Email address to send alerts to", 2023-07-18T03:39:26.457 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.457 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.458 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.458 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.458 INFO:teuthology.orchestra.run.smithi101.stdout: "smtp_from_name": { 2023-07-18T03:39:26.458 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "smtp_from_name", 2023-07-18T03:39:26.458 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.459 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.459 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.459 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "Ceph", 2023-07-18T03:39:26.459 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.459 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.460 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.460 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Email From: name", 2023-07-18T03:39:26.460 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.460 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.460 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.461 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.461 INFO:teuthology.orchestra.run.smithi101.stdout: "smtp_host": { 2023-07-18T03:39:26.461 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "smtp_host", 2023-07-18T03:39:26.461 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.461 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.461 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.462 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.462 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.462 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.462 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.462 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "SMTP server", 2023-07-18T03:39:26.463 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.463 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.463 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.463 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.463 INFO:teuthology.orchestra.run.smithi101.stdout: "smtp_password": { 2023-07-18T03:39:26.464 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "smtp_password", 2023-07-18T03:39:26.464 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.464 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.464 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.464 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.465 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.465 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.465 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.465 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Password to authenticate with", 2023-07-18T03:39:26.465 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.466 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.466 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.466 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.466 INFO:teuthology.orchestra.run.smithi101.stdout: "smtp_port": { 2023-07-18T03:39:26.466 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "smtp_port", 2023-07-18T03:39:26.467 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:26.467 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.467 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.467 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "465", 2023-07-18T03:39:26.467 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.468 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.468 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.468 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "SMTP port", 2023-07-18T03:39:26.468 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.468 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.469 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.469 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.469 INFO:teuthology.orchestra.run.smithi101.stdout: "smtp_sender": { 2023-07-18T03:39:26.469 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "smtp_sender", 2023-07-18T03:39:26.469 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.469 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.470 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.470 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.470 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.470 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.470 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.471 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "SMTP envelope sender", 2023-07-18T03:39:26.471 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.471 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.471 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.471 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.472 INFO:teuthology.orchestra.run.smithi101.stdout: "smtp_ssl": { 2023-07-18T03:39:26.472 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "smtp_ssl", 2023-07-18T03:39:26.472 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.472 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.472 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.473 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:26.473 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.473 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.473 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.473 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Use SSL to connect to SMTP server", 2023-07-18T03:39:26.474 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.474 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.474 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.474 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.474 INFO:teuthology.orchestra.run.smithi101.stdout: "smtp_user": { 2023-07-18T03:39:26.475 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "smtp_user", 2023-07-18T03:39:26.475 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.475 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.475 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.475 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.476 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.476 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.476 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.476 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "User to authenticate as", 2023-07-18T03:39:26.476 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.477 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.477 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.477 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.477 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.477 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.477 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:26.478 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "balancer", 2023-07-18T03:39:26.478 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:26.478 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:26.478 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:26.478 INFO:teuthology.orchestra.run.smithi101.stdout: "active": { 2023-07-18T03:39:26.479 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "active", 2023-07-18T03:39:26.479 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.479 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.479 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.479 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:26.480 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.480 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.480 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.480 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "automatically balance PGs across cluster", 2023-07-18T03:39:26.480 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.481 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.481 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.481 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.481 INFO:teuthology.orchestra.run.smithi101.stdout: "begin_time": { 2023-07-18T03:39:26.481 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "begin_time", 2023-07-18T03:39:26.482 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.482 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.482 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.482 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "0000", 2023-07-18T03:39:26.482 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.483 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.483 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.483 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "beginning time of day to automatically balance", 2023-07-18T03:39:26.483 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-18T03:39:26.483 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.484 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.484 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.484 INFO:teuthology.orchestra.run.smithi101.stdout: "begin_weekday": { 2023-07-18T03:39:26.484 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "begin_weekday", 2023-07-18T03:39:26.484 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "uint", 2023-07-18T03:39:26.485 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.485 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.485 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "0", 2023-07-18T03:39:26.485 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "0", 2023-07-18T03:39:26.485 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "6", 2023-07-18T03:39:26.486 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.486 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-07-18T03:39:26.486 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-18T03:39:26.486 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.486 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.487 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.487 INFO:teuthology.orchestra.run.smithi101.stdout: "crush_compat_max_iterations": { 2023-07-18T03:39:26.487 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "crush_compat_max_iterations", 2023-07-18T03:39:26.487 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "uint", 2023-07-18T03:39:26.487 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.488 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.488 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "25", 2023-07-18T03:39:26.488 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "1", 2023-07-18T03:39:26.488 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "250", 2023-07-18T03:39:26.488 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.489 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-07-18T03:39:26.489 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.489 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.489 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.489 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.490 INFO:teuthology.orchestra.run.smithi101.stdout: "crush_compat_metrics": { 2023-07-18T03:39:26.490 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "crush_compat_metrics", 2023-07-18T03:39:26.490 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.490 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.490 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.491 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "pgs,objects,bytes", 2023-07-18T03:39:26.491 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.491 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.491 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.491 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-07-18T03:39:26.492 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-07-18T03:39:26.492 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.492 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.492 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.492 INFO:teuthology.orchestra.run.smithi101.stdout: "crush_compat_step": { 2023-07-18T03:39:26.493 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "crush_compat_step", 2023-07-18T03:39:26.493 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "float", 2023-07-18T03:39:26.493 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.493 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.493 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "0.5", 2023-07-18T03:39:26.494 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "0.001", 2023-07-18T03:39:26.494 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "0.999", 2023-07-18T03:39:26.494 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.494 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "aggressiveness of optimization", 2023-07-18T03:39:26.494 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-07-18T03:39:26.494 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.495 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.495 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.495 INFO:teuthology.orchestra.run.smithi101.stdout: "end_time": { 2023-07-18T03:39:26.495 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "end_time", 2023-07-18T03:39:26.495 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.496 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.496 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.496 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "2359", 2023-07-18T03:39:26.496 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.496 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.497 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.497 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "ending time of day to automatically balance", 2023-07-18T03:39:26.497 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-18T03:39:26.497 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.497 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.498 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.498 INFO:teuthology.orchestra.run.smithi101.stdout: "end_weekday": { 2023-07-18T03:39:26.498 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "end_weekday", 2023-07-18T03:39:26.498 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "uint", 2023-07-18T03:39:26.498 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.499 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.499 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "0", 2023-07-18T03:39:26.499 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "0", 2023-07-18T03:39:26.499 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "6", 2023-07-18T03:39:26.499 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.500 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-07-18T03:39:26.500 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-18T03:39:26.500 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.500 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.500 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.501 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:26.501 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:26.501 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.501 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.501 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.502 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.502 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.502 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.502 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.502 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.503 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.503 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.503 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.503 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.503 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.504 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.504 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.504 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.504 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.504 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.505 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.505 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:26.505 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:26.505 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.505 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.505 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.506 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.506 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.506 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.506 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.506 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.507 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.507 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.507 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.507 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.507 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:26.508 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:26.508 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.508 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.508 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.508 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:26.509 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.509 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.509 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.509 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.509 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.510 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.510 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.510 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.510 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.510 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.511 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.511 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.511 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.511 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.512 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.512 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:26.512 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:26.513 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.513 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.513 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.513 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.513 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.514 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.514 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.514 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.514 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.514 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.515 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.515 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.515 INFO:teuthology.orchestra.run.smithi101.stdout: "min_score": { 2023-07-18T03:39:26.515 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "min_score", 2023-07-18T03:39:26.515 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "float", 2023-07-18T03:39:26.516 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.516 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.516 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "0", 2023-07-18T03:39:26.516 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.516 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.517 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.517 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-07-18T03:39:26.517 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.517 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.517 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.518 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.518 INFO:teuthology.orchestra.run.smithi101.stdout: "mode": { 2023-07-18T03:39:26.518 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "mode", 2023-07-18T03:39:26.518 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.518 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.519 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.519 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "upmap", 2023-07-18T03:39:26.519 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.519 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.519 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.520 INFO:teuthology.orchestra.run.smithi101.stdout: "crush-compat", 2023-07-18T03:39:26.520 INFO:teuthology.orchestra.run.smithi101.stdout: "none", 2023-07-18T03:39:26.520 INFO:teuthology.orchestra.run.smithi101.stdout: "upmap" 2023-07-18T03:39:26.520 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.521 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Balancer mode", 2023-07-18T03:39:26.521 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.521 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.521 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.521 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.522 INFO:teuthology.orchestra.run.smithi101.stdout: "pool_ids": { 2023-07-18T03:39:26.522 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "pool_ids", 2023-07-18T03:39:26.522 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.522 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.522 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.523 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.523 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.523 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.523 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.523 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-07-18T03:39:26.524 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.524 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.524 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.524 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.525 INFO:teuthology.orchestra.run.smithi101.stdout: "sleep_interval": { 2023-07-18T03:39:26.525 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "sleep_interval", 2023-07-18T03:39:26.525 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:26.525 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.525 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.526 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "60", 2023-07-18T03:39:26.526 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.526 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.526 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.526 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-07-18T03:39:26.527 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.527 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.527 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.527 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.527 INFO:teuthology.orchestra.run.smithi101.stdout: "upmap_max_deviation": { 2023-07-18T03:39:26.528 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "upmap_max_deviation", 2023-07-18T03:39:26.528 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:26.528 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.528 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.528 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "5", 2023-07-18T03:39:26.529 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "1", 2023-07-18T03:39:26.529 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.529 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.529 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "deviation below which no optimization is attempted", 2023-07-18T03:39:26.529 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-07-18T03:39:26.530 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.530 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.530 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.530 INFO:teuthology.orchestra.run.smithi101.stdout: "upmap_max_optimizations": { 2023-07-18T03:39:26.530 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "upmap_max_optimizations", 2023-07-18T03:39:26.531 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "uint", 2023-07-18T03:39:26.531 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.531 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.531 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "10", 2023-07-18T03:39:26.531 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.532 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.532 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.532 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-07-18T03:39:26.532 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.532 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.533 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.533 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.533 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.533 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.533 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:26.534 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "cephadm", 2023-07-18T03:39:26.534 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:26.534 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:26.534 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:26.534 INFO:teuthology.orchestra.run.smithi101.stdout: "agent_down_multiplier": { 2023-07-18T03:39:26.535 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "agent_down_multiplier", 2023-07-18T03:39:26.535 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "float", 2023-07-18T03:39:26.535 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.535 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.535 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "3.0", 2023-07-18T03:39:26.536 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.536 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.536 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.536 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-07-18T03:39:26.536 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.537 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.537 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.537 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.537 INFO:teuthology.orchestra.run.smithi101.stdout: "agent_refresh_rate": { 2023-07-18T03:39:26.537 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "agent_refresh_rate", 2023-07-18T03:39:26.538 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:26.538 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.538 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.538 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "20", 2023-07-18T03:39:26.538 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.539 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.539 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.539 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-07-18T03:39:26.539 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.539 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.540 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.540 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.540 INFO:teuthology.orchestra.run.smithi101.stdout: "agent_starting_port": { 2023-07-18T03:39:26.540 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "agent_starting_port", 2023-07-18T03:39:26.540 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:26.541 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.541 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.541 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "4721", 2023-07-18T03:39:26.541 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.541 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.542 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.542 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-07-18T03:39:26.542 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.542 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.542 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.543 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.543 INFO:teuthology.orchestra.run.smithi101.stdout: "alertmanager_web_password": { 2023-07-18T03:39:26.543 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "alertmanager_web_password", 2023-07-18T03:39:26.543 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.543 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.544 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.544 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "admin", 2023-07-18T03:39:26.544 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.544 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.544 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.545 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Alertmanager web password", 2023-07-18T03:39:26.545 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.545 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.545 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.545 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.546 INFO:teuthology.orchestra.run.smithi101.stdout: "alertmanager_web_user": { 2023-07-18T03:39:26.546 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "alertmanager_web_user", 2023-07-18T03:39:26.546 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.546 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.546 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.547 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "admin", 2023-07-18T03:39:26.547 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.547 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.547 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.547 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Alertmanager web user", 2023-07-18T03:39:26.548 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.548 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.548 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.548 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.548 INFO:teuthology.orchestra.run.smithi101.stdout: "allow_ptrace": { 2023-07-18T03:39:26.549 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "allow_ptrace", 2023-07-18T03:39:26.549 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.549 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.549 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.549 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.550 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.550 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.550 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.550 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-07-18T03:39:26.550 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-07-18T03:39:26.551 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.551 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.551 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.551 INFO:teuthology.orchestra.run.smithi101.stdout: "autotune_interval": { 2023-07-18T03:39:26.551 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "autotune_interval", 2023-07-18T03:39:26.552 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:26.552 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.552 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.552 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "600", 2023-07-18T03:39:26.552 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.553 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.553 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.553 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "how frequently to autotune daemon memory", 2023-07-18T03:39:26.553 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.553 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.554 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.554 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.554 INFO:teuthology.orchestra.run.smithi101.stdout: "autotune_memory_target_ratio": { 2023-07-18T03:39:26.554 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "autotune_memory_target_ratio", 2023-07-18T03:39:26.554 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "float", 2023-07-18T03:39:26.555 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.555 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.555 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "0.7", 2023-07-18T03:39:26.555 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.556 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.556 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.556 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-07-18T03:39:26.556 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.556 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.557 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.557 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.557 INFO:teuthology.orchestra.run.smithi101.stdout: "cgroups_split": { 2023-07-18T03:39:26.557 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "cgroups_split", 2023-07-18T03:39:26.557 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.558 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.558 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.558 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:26.558 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.558 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.559 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.559 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-07-18T03:39:26.559 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.559 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.559 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.560 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.560 INFO:teuthology.orchestra.run.smithi101.stdout: "config_checks_enabled": { 2023-07-18T03:39:26.560 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "config_checks_enabled", 2023-07-18T03:39:26.560 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.560 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.561 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.561 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.561 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.561 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.561 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.562 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-07-18T03:39:26.562 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.562 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.562 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.562 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.563 INFO:teuthology.orchestra.run.smithi101.stdout: "config_dashboard": { 2023-07-18T03:39:26.563 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "config_dashboard", 2023-07-18T03:39:26.563 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.563 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.563 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.564 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:26.564 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.564 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.564 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.564 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-07-18T03:39:26.565 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.565 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.565 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.565 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.565 INFO:teuthology.orchestra.run.smithi101.stdout: "container_image_alertmanager": { 2023-07-18T03:39:26.566 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "container_image_alertmanager", 2023-07-18T03:39:26.566 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.566 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.566 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.566 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-07-18T03:39:26.567 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.567 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.567 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.567 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Prometheus container image", 2023-07-18T03:39:26.567 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.568 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.568 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.568 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.568 INFO:teuthology.orchestra.run.smithi101.stdout: "container_image_base": { 2023-07-18T03:39:26.568 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "container_image_base", 2023-07-18T03:39:26.569 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.569 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.569 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.569 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "quay.io/ceph/ceph", 2023-07-18T03:39:26.569 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.570 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.570 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.570 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Container image name, without the tag", 2023-07-18T03:39:26.570 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.570 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.571 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.571 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.571 INFO:teuthology.orchestra.run.smithi101.stdout: "container_image_elasticsearch": { 2023-07-18T03:39:26.571 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "container_image_elasticsearch", 2023-07-18T03:39:26.571 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.572 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.572 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.572 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-07-18T03:39:26.572 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.572 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.573 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.573 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "elasticsearch container image", 2023-07-18T03:39:26.573 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.573 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.573 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.574 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.574 INFO:teuthology.orchestra.run.smithi101.stdout: "container_image_grafana": { 2023-07-18T03:39:26.574 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "container_image_grafana", 2023-07-18T03:39:26.574 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.574 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.574 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.575 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-07-18T03:39:26.575 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.575 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.575 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.575 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Prometheus container image", 2023-07-18T03:39:26.576 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.576 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.576 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.576 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.576 INFO:teuthology.orchestra.run.smithi101.stdout: "container_image_haproxy": { 2023-07-18T03:39:26.577 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "container_image_haproxy", 2023-07-18T03:39:26.577 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.577 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.577 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.577 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-07-18T03:39:26.578 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.578 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.578 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.578 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "HAproxy container image", 2023-07-18T03:39:26.578 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.579 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.579 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.579 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.579 INFO:teuthology.orchestra.run.smithi101.stdout: "container_image_jaeger_agent": { 2023-07-18T03:39:26.579 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "container_image_jaeger_agent", 2023-07-18T03:39:26.580 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.580 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.580 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.580 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-07-18T03:39:26.580 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.581 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.581 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.581 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Jaeger agent container image", 2023-07-18T03:39:26.581 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.581 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.582 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.582 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.582 INFO:teuthology.orchestra.run.smithi101.stdout: "container_image_jaeger_collector": { 2023-07-18T03:39:26.582 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "container_image_jaeger_collector", 2023-07-18T03:39:26.582 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.583 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.583 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.583 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-07-18T03:39:26.583 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.583 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.584 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.584 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Jaeger collector container image", 2023-07-18T03:39:26.584 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.584 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.584 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.585 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.585 INFO:teuthology.orchestra.run.smithi101.stdout: "container_image_jaeger_query": { 2023-07-18T03:39:26.585 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "container_image_jaeger_query", 2023-07-18T03:39:26.585 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.586 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.586 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.586 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-07-18T03:39:26.586 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.586 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.587 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.587 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Jaeger query container image", 2023-07-18T03:39:26.587 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.587 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.587 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.588 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.588 INFO:teuthology.orchestra.run.smithi101.stdout: "container_image_keepalived": { 2023-07-18T03:39:26.588 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "container_image_keepalived", 2023-07-18T03:39:26.588 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.589 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.589 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.589 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-07-18T03:39:26.589 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.589 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.590 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.590 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Keepalived container image", 2023-07-18T03:39:26.590 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.590 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.590 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.591 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.591 INFO:teuthology.orchestra.run.smithi101.stdout: "container_image_loki": { 2023-07-18T03:39:26.591 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "container_image_loki", 2023-07-18T03:39:26.591 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.591 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.592 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.592 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-07-18T03:39:26.592 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.592 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.592 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.593 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Loki container image", 2023-07-18T03:39:26.593 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.593 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.593 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.593 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.594 INFO:teuthology.orchestra.run.smithi101.stdout: "container_image_node_exporter": { 2023-07-18T03:39:26.594 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "container_image_node_exporter", 2023-07-18T03:39:26.594 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.594 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.594 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.594 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-07-18T03:39:26.595 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.595 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.595 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.595 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Prometheus container image", 2023-07-18T03:39:26.595 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.596 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.596 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.596 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.596 INFO:teuthology.orchestra.run.smithi101.stdout: "container_image_prometheus": { 2023-07-18T03:39:26.597 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "container_image_prometheus", 2023-07-18T03:39:26.597 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.597 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.597 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.597 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-07-18T03:39:26.597 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.598 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.598 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.598 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Prometheus container image", 2023-07-18T03:39:26.598 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.598 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.599 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.599 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.599 INFO:teuthology.orchestra.run.smithi101.stdout: "container_image_promtail": { 2023-07-18T03:39:26.599 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "container_image_promtail", 2023-07-18T03:39:26.599 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.600 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.600 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.600 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-07-18T03:39:26.600 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.600 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.601 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.601 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Promtail container image", 2023-07-18T03:39:26.601 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.601 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.601 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.602 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.602 INFO:teuthology.orchestra.run.smithi101.stdout: "container_image_snmp_gateway": { 2023-07-18T03:39:26.602 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "container_image_snmp_gateway", 2023-07-18T03:39:26.602 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.602 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.603 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.603 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-07-18T03:39:26.603 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.603 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.603 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.604 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "SNMP Gateway container image", 2023-07-18T03:39:26.604 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.604 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.604 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.604 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.605 INFO:teuthology.orchestra.run.smithi101.stdout: "container_init": { 2023-07-18T03:39:26.605 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "container_init", 2023-07-18T03:39:26.605 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.605 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.605 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.606 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:26.606 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.606 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.606 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.606 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Run podman/docker with `--init`", 2023-07-18T03:39:26.607 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.607 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.607 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.607 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.607 INFO:teuthology.orchestra.run.smithi101.stdout: "daemon_cache_timeout": { 2023-07-18T03:39:26.608 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "daemon_cache_timeout", 2023-07-18T03:39:26.608 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:26.608 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.608 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.608 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "600", 2023-07-18T03:39:26.609 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.609 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.609 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.609 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-07-18T03:39:26.609 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.610 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.610 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.610 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.610 INFO:teuthology.orchestra.run.smithi101.stdout: "default_cephadm_command_timeout": { 2023-07-18T03:39:26.610 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "default_cephadm_command_timeout", 2023-07-18T03:39:26.611 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:26.611 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.611 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.611 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "900", 2023-07-18T03:39:26.611 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.612 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.612 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.612 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-07-18T03:39:26.612 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.612 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.613 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.613 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.613 INFO:teuthology.orchestra.run.smithi101.stdout: "default_registry": { 2023-07-18T03:39:26.613 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "default_registry", 2023-07-18T03:39:26.613 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.614 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.614 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.614 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "docker.io", 2023-07-18T03:39:26.614 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.614 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.615 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.615 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-07-18T03:39:26.615 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.615 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.615 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.616 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.616 INFO:teuthology.orchestra.run.smithi101.stdout: "device_cache_timeout": { 2023-07-18T03:39:26.616 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "device_cache_timeout", 2023-07-18T03:39:26.616 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:26.616 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.617 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.617 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "1800", 2023-07-18T03:39:26.617 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.617 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.617 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.618 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "seconds to cache device inventory", 2023-07-18T03:39:26.618 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.618 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.618 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.618 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.619 INFO:teuthology.orchestra.run.smithi101.stdout: "device_enhanced_scan": { 2023-07-18T03:39:26.619 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "device_enhanced_scan", 2023-07-18T03:39:26.619 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.619 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.619 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.620 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.620 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.620 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.620 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.620 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Use libstoragemgmt during device scans", 2023-07-18T03:39:26.621 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.621 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.621 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.621 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.621 INFO:teuthology.orchestra.run.smithi101.stdout: "facts_cache_timeout": { 2023-07-18T03:39:26.622 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "facts_cache_timeout", 2023-07-18T03:39:26.622 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:26.622 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.622 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.622 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "60", 2023-07-18T03:39:26.623 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.623 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.623 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.623 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "seconds to cache host facts data", 2023-07-18T03:39:26.623 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.624 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.624 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.624 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.624 INFO:teuthology.orchestra.run.smithi101.stdout: "host_check_interval": { 2023-07-18T03:39:26.624 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "host_check_interval", 2023-07-18T03:39:26.625 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:26.625 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.625 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.625 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "600", 2023-07-18T03:39:26.625 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.626 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.626 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.626 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "how frequently to perform a host check", 2023-07-18T03:39:26.626 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.626 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.627 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.627 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.627 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:26.627 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:26.627 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.628 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.628 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.628 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.628 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.629 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.629 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.629 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.629 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.629 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.630 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.630 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.630 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.630 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.630 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.631 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.631 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.631 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.631 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.631 INFO:teuthology.orchestra.run.smithi101.stdout: "log_refresh_metadata": { 2023-07-18T03:39:26.632 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_refresh_metadata", 2023-07-18T03:39:26.632 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.632 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.632 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.632 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.633 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.633 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.633 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.633 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-07-18T03:39:26.633 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.634 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.634 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.634 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.634 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:26.634 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:26.635 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.635 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.635 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.635 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:26.636 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.636 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.636 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.637 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-07-18T03:39:26.637 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.637 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.637 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.638 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.638 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:26.638 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:26.638 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.638 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.639 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.639 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:26.639 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.639 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.639 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.640 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.640 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.640 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.640 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.641 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.641 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.641 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.641 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.641 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.642 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.642 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.642 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.642 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:26.642 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:26.643 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.643 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.643 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.643 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.643 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.644 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.644 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.644 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.644 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.645 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.645 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.645 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.645 INFO:teuthology.orchestra.run.smithi101.stdout: "manage_etc_ceph_ceph_conf": { 2023-07-18T03:39:26.645 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-07-18T03:39:26.646 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.646 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.646 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.646 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.646 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.647 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.647 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.647 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-07-18T03:39:26.647 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.648 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.648 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.648 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.648 INFO:teuthology.orchestra.run.smithi101.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-07-18T03:39:26.648 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-07-18T03:39:26.649 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.649 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.649 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.649 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "*", 2023-07-18T03:39:26.649 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.650 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.650 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.650 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-07-18T03:39:26.650 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.650 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.651 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.651 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.651 INFO:teuthology.orchestra.run.smithi101.stdout: "max_count_per_host": { 2023-07-18T03:39:26.651 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "max_count_per_host", 2023-07-18T03:39:26.652 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:26.652 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.652 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.652 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "10", 2023-07-18T03:39:26.652 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.653 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.653 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.653 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "max number of daemons per service per host", 2023-07-18T03:39:26.653 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.653 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.654 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.654 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.654 INFO:teuthology.orchestra.run.smithi101.stdout: "max_osd_draining_count": { 2023-07-18T03:39:26.654 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "max_osd_draining_count", 2023-07-18T03:39:26.654 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:26.655 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.655 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.655 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "10", 2023-07-18T03:39:26.655 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.655 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.656 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.656 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-07-18T03:39:26.656 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.656 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.657 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.657 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.657 INFO:teuthology.orchestra.run.smithi101.stdout: "migration_current": { 2023-07-18T03:39:26.657 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "migration_current", 2023-07-18T03:39:26.658 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:26.658 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.658 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.658 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.659 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.659 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.659 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.659 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "internal - do not modify", 2023-07-18T03:39:26.660 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.660 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.660 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.660 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.660 INFO:teuthology.orchestra.run.smithi101.stdout: "mode": { 2023-07-18T03:39:26.661 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "mode", 2023-07-18T03:39:26.661 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.661 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.661 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.662 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "root", 2023-07-18T03:39:26.662 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.662 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.662 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.662 INFO:teuthology.orchestra.run.smithi101.stdout: "cephadm-package", 2023-07-18T03:39:26.663 INFO:teuthology.orchestra.run.smithi101.stdout: "root" 2023-07-18T03:39:26.663 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.663 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "mode for remote execution of cephadm", 2023-07-18T03:39:26.663 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.663 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.663 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.664 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.664 INFO:teuthology.orchestra.run.smithi101.stdout: "prometheus_alerts_path": { 2023-07-18T03:39:26.664 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "prometheus_alerts_path", 2023-07-18T03:39:26.664 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.665 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.665 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.665 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-07-18T03:39:26.665 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.665 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.666 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.666 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-07-18T03:39:26.666 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.666 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.666 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.667 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.667 INFO:teuthology.orchestra.run.smithi101.stdout: "prometheus_web_password": { 2023-07-18T03:39:26.667 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "prometheus_web_password", 2023-07-18T03:39:26.667 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.667 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.668 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.668 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "admin", 2023-07-18T03:39:26.668 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.668 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.668 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.669 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Prometheus web password", 2023-07-18T03:39:26.669 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.669 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.669 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.669 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.670 INFO:teuthology.orchestra.run.smithi101.stdout: "prometheus_web_user": { 2023-07-18T03:39:26.670 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "prometheus_web_user", 2023-07-18T03:39:26.670 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.670 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.670 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.671 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "admin", 2023-07-18T03:39:26.671 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.671 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.671 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.671 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Prometheus web user", 2023-07-18T03:39:26.672 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.672 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.672 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.672 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.672 INFO:teuthology.orchestra.run.smithi101.stdout: "registry_insecure": { 2023-07-18T03:39:26.673 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "registry_insecure", 2023-07-18T03:39:26.673 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.673 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.674 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.674 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.674 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.674 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.675 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.675 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-07-18T03:39:26.675 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.676 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.676 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.676 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.676 INFO:teuthology.orchestra.run.smithi101.stdout: "registry_password": { 2023-07-18T03:39:26.677 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "registry_password", 2023-07-18T03:39:26.677 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.677 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.677 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.678 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.678 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.678 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.678 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.678 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-07-18T03:39:26.679 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.679 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.679 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.679 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.679 INFO:teuthology.orchestra.run.smithi101.stdout: "registry_url": { 2023-07-18T03:39:26.680 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "registry_url", 2023-07-18T03:39:26.680 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.680 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.680 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.680 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.681 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.681 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.681 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.681 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-07-18T03:39:26.681 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.682 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.682 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.682 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.682 INFO:teuthology.orchestra.run.smithi101.stdout: "registry_username": { 2023-07-18T03:39:26.682 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "registry_username", 2023-07-18T03:39:26.683 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.683 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.683 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.683 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.683 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.683 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.684 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.684 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-07-18T03:39:26.684 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.684 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.685 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.685 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.685 INFO:teuthology.orchestra.run.smithi101.stdout: "secure_monitoring_stack": { 2023-07-18T03:39:26.685 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "secure_monitoring_stack", 2023-07-18T03:39:26.685 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.686 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.686 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.686 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.686 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.686 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.686 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.687 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-07-18T03:39:26.687 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.687 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.687 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.687 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.688 INFO:teuthology.orchestra.run.smithi101.stdout: "service_discovery_port": { 2023-07-18T03:39:26.688 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "service_discovery_port", 2023-07-18T03:39:26.688 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:26.688 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.689 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.689 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "8765", 2023-07-18T03:39:26.689 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.689 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.689 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.690 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "cephadm service discovery port", 2023-07-18T03:39:26.690 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.690 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.690 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.690 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.690 INFO:teuthology.orchestra.run.smithi101.stdout: "ssh_config_file": { 2023-07-18T03:39:26.691 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "ssh_config_file", 2023-07-18T03:39:26.691 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.691 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.691 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.691 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.692 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.692 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.692 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.692 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-07-18T03:39:26.693 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.693 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.693 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.693 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.694 INFO:teuthology.orchestra.run.smithi101.stdout: "use_agent": { 2023-07-18T03:39:26.694 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "use_agent", 2023-07-18T03:39:26.694 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.694 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.695 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.695 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.695 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.695 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.696 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.696 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-07-18T03:39:26.696 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.696 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.697 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.697 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.697 INFO:teuthology.orchestra.run.smithi101.stdout: "use_repo_digest": { 2023-07-18T03:39:26.697 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "use_repo_digest", 2023-07-18T03:39:26.698 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.698 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.698 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.698 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:26.699 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.699 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.699 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.699 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-07-18T03:39:26.700 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.700 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.700 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.700 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.701 INFO:teuthology.orchestra.run.smithi101.stdout: "warn_on_failed_host_check": { 2023-07-18T03:39:26.701 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "warn_on_failed_host_check", 2023-07-18T03:39:26.701 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.701 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.702 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.702 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:26.702 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.702 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.702 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.703 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "raise a health warning if the host check fails", 2023-07-18T03:39:26.703 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.703 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.703 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.704 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.704 INFO:teuthology.orchestra.run.smithi101.stdout: "warn_on_stray_daemons": { 2023-07-18T03:39:26.704 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "warn_on_stray_daemons", 2023-07-18T03:39:26.704 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.704 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.705 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.705 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:26.705 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.705 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.705 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.706 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-07-18T03:39:26.706 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.706 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.706 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.706 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.707 INFO:teuthology.orchestra.run.smithi101.stdout: "warn_on_stray_hosts": { 2023-07-18T03:39:26.707 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "warn_on_stray_hosts", 2023-07-18T03:39:26.707 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.707 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.707 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.708 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:26.708 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.708 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.708 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.709 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-07-18T03:39:26.709 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.709 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.709 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.709 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.710 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.710 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.710 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:26.710 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "crash", 2023-07-18T03:39:26.710 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:26.711 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:26.711 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:26.711 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:26.711 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:26.711 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.713 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.714 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.714 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.714 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.714 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.715 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.715 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.715 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.715 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.715 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.716 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.716 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.716 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.716 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.717 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.717 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.717 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.717 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.717 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:26.718 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:26.718 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.718 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.718 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.718 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.719 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.719 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.719 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.719 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.720 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.720 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.720 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.720 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.720 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:26.721 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:26.721 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.721 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.721 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.722 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:26.722 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.722 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.722 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.722 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.723 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.723 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.723 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.723 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.724 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.724 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.724 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.724 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.725 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.725 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.725 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.726 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:26.726 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:26.726 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.726 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.726 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.727 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.727 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.727 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.728 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.728 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.728 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.728 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.729 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.729 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.730 INFO:teuthology.orchestra.run.smithi101.stdout: "retain_interval": { 2023-07-18T03:39:26.730 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "retain_interval", 2023-07-18T03:39:26.730 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:26.731 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.731 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.732 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "31536000", 2023-07-18T03:39:26.732 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.733 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.733 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.734 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "how long to retain crashes before pruning them", 2023-07-18T03:39:26.734 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.735 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.735 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.736 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.736 INFO:teuthology.orchestra.run.smithi101.stdout: "warn_recent_interval": { 2023-07-18T03:39:26.736 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "warn_recent_interval", 2023-07-18T03:39:26.737 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:26.737 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.738 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.738 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "1209600", 2023-07-18T03:39:26.739 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.739 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.739 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.739 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "time interval in which to warn about recent crashes", 2023-07-18T03:39:26.740 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.740 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.741 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.741 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.741 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.742 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.742 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:26.742 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "dashboard", 2023-07-18T03:39:26.742 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:26.742 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:26.743 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:26.743 INFO:teuthology.orchestra.run.smithi101.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-07-18T03:39:26.743 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-07-18T03:39:26.743 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:26.743 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.744 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.744 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "10", 2023-07-18T03:39:26.744 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.744 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.745 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.745 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.745 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.745 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.745 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.746 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.746 INFO:teuthology.orchestra.run.smithi101.stdout: "ALERTMANAGER_API_HOST": { 2023-07-18T03:39:26.746 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "ALERTMANAGER_API_HOST", 2023-07-18T03:39:26.746 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.747 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.747 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.747 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.747 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.748 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.748 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.748 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.748 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.749 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.749 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.749 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.749 INFO:teuthology.orchestra.run.smithi101.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-07-18T03:39:26.749 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-07-18T03:39:26.750 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.750 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.750 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.750 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:26.750 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.751 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.751 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.751 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.751 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.752 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.752 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.752 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.752 INFO:teuthology.orchestra.run.smithi101.stdout: "AUDIT_API_ENABLED": { 2023-07-18T03:39:26.753 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "AUDIT_API_ENABLED", 2023-07-18T03:39:26.753 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.753 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.753 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.754 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.754 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.754 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.754 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.755 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.755 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.755 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.755 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.756 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.756 INFO:teuthology.orchestra.run.smithi101.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-07-18T03:39:26.756 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-07-18T03:39:26.757 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.757 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.757 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.757 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:26.758 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.758 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.758 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.759 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.759 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.759 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.760 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.760 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.761 INFO:teuthology.orchestra.run.smithi101.stdout: "ENABLE_BROWSABLE_API": { 2023-07-18T03:39:26.761 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "ENABLE_BROWSABLE_API", 2023-07-18T03:39:26.762 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.762 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.763 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.763 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:26.764 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.764 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.765 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.765 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.765 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.766 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.766 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.767 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.767 INFO:teuthology.orchestra.run.smithi101.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-07-18T03:39:26.768 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-07-18T03:39:26.768 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.768 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.769 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.769 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:26.769 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.770 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.770 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.771 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.771 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.771 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.771 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.772 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.772 INFO:teuthology.orchestra.run.smithi101.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-07-18T03:39:26.772 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-07-18T03:39:26.772 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.772 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.773 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.773 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:26.773 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.773 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.773 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.774 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.774 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.774 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.774 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.775 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.775 INFO:teuthology.orchestra.run.smithi101.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-07-18T03:39:26.775 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-07-18T03:39:26.775 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.775 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.776 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.776 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:26.776 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.776 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.777 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.777 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.777 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.777 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.777 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.778 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.778 INFO:teuthology.orchestra.run.smithi101.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-07-18T03:39:26.778 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-07-18T03:39:26.778 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.778 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.779 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.779 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:26.779 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.779 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.780 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.780 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.780 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.780 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.780 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.781 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.781 INFO:teuthology.orchestra.run.smithi101.stdout: "FEATURE_TOGGLE_NFS": { 2023-07-18T03:39:26.781 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-07-18T03:39:26.781 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.781 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.782 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.782 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:26.782 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.782 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.782 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.783 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.783 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.783 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.783 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.784 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.784 INFO:teuthology.orchestra.run.smithi101.stdout: "FEATURE_TOGGLE_RBD": { 2023-07-18T03:39:26.784 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-07-18T03:39:26.784 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.785 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.785 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.785 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:26.785 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.785 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.786 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.786 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.786 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.787 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.787 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.787 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.787 INFO:teuthology.orchestra.run.smithi101.stdout: "FEATURE_TOGGLE_RGW": { 2023-07-18T03:39:26.788 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-07-18T03:39:26.788 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.788 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.788 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.789 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:26.789 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.789 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.790 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.790 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.790 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.791 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.791 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.791 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.792 INFO:teuthology.orchestra.run.smithi101.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-07-18T03:39:26.792 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-07-18T03:39:26.792 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.793 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.793 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.793 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.794 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.794 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.794 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.795 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.795 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.795 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.795 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.795 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.796 INFO:teuthology.orchestra.run.smithi101.stdout: "GRAFANA_API_PASSWORD": { 2023-07-18T03:39:26.796 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "GRAFANA_API_PASSWORD", 2023-07-18T03:39:26.796 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.796 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.796 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.797 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "admin", 2023-07-18T03:39:26.797 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.797 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.797 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.797 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.798 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.798 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.798 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.798 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.798 INFO:teuthology.orchestra.run.smithi101.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-07-18T03:39:26.799 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-07-18T03:39:26.799 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.799 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.799 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.799 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:26.800 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.800 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.800 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.800 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.800 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.801 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.801 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.801 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.801 INFO:teuthology.orchestra.run.smithi101.stdout: "GRAFANA_API_URL": { 2023-07-18T03:39:26.801 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "GRAFANA_API_URL", 2023-07-18T03:39:26.802 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.802 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.802 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.802 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.802 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.803 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.803 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.803 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.803 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.803 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.804 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.804 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.804 INFO:teuthology.orchestra.run.smithi101.stdout: "GRAFANA_API_USERNAME": { 2023-07-18T03:39:26.804 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "GRAFANA_API_USERNAME", 2023-07-18T03:39:26.805 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.805 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.805 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.805 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "admin", 2023-07-18T03:39:26.805 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.805 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.806 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.806 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.806 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.806 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.807 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.807 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.808 INFO:teuthology.orchestra.run.smithi101.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-07-18T03:39:26.808 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-07-18T03:39:26.809 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.809 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.809 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.810 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.810 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.811 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.811 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.812 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.812 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.813 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.813 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.814 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.814 INFO:teuthology.orchestra.run.smithi101.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-07-18T03:39:26.814 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-07-18T03:39:26.815 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.815 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.816 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.816 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.816 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.817 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.817 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.817 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.818 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.818 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.819 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.819 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.820 INFO:teuthology.orchestra.run.smithi101.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-07-18T03:39:26.820 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-07-18T03:39:26.820 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.821 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.821 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.821 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:26.821 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.821 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.822 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.822 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.822 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.822 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.822 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.823 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.823 INFO:teuthology.orchestra.run.smithi101.stdout: "ISSUE_TRACKER_API_KEY": { 2023-07-18T03:39:26.823 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-07-18T03:39:26.823 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.823 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.824 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.824 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.824 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.824 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.824 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.825 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.825 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.825 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.825 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.825 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.826 INFO:teuthology.orchestra.run.smithi101.stdout: "PROMETHEUS_API_HOST": { 2023-07-18T03:39:26.826 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "PROMETHEUS_API_HOST", 2023-07-18T03:39:26.826 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.827 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.827 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.827 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.827 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.827 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.828 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.828 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.828 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.828 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.829 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.829 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.829 INFO:teuthology.orchestra.run.smithi101.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-07-18T03:39:26.829 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-07-18T03:39:26.829 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.830 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.830 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.830 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:26.830 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.830 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.831 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.831 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.831 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.831 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.831 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.832 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.832 INFO:teuthology.orchestra.run.smithi101.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-07-18T03:39:26.832 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-07-18T03:39:26.832 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.832 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.833 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.833 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.833 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.833 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.833 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.834 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.834 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.834 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.834 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.834 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.835 INFO:teuthology.orchestra.run.smithi101.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-07-18T03:39:26.835 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-07-18T03:39:26.835 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.835 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.835 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.836 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.836 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.836 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.836 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.836 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.837 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.837 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.837 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.838 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.838 INFO:teuthology.orchestra.run.smithi101.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-07-18T03:39:26.838 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-07-18T03:39:26.838 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.839 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.839 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.839 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:26.839 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.840 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.840 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.840 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.840 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.841 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.841 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.841 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.841 INFO:teuthology.orchestra.run.smithi101.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-07-18T03:39:26.841 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-07-18T03:39:26.842 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.842 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.842 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.842 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:26.843 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.843 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.843 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.843 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.843 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.844 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.844 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.844 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.845 INFO:teuthology.orchestra.run.smithi101.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-07-18T03:39:26.845 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-07-18T03:39:26.846 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.846 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.846 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.847 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.847 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.848 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.848 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.849 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.849 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.850 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.850 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.851 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.851 INFO:teuthology.orchestra.run.smithi101.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-07-18T03:39:26.852 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-07-18T03:39:26.852 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.852 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.853 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.853 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.854 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.854 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.854 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.855 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.855 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.855 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.856 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.856 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.857 INFO:teuthology.orchestra.run.smithi101.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-07-18T03:39:26.857 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-07-18T03:39:26.857 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.857 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.857 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.858 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.858 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.858 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.858 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.858 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.859 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.859 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.859 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.859 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.859 INFO:teuthology.orchestra.run.smithi101.stdout: "PWD_POLICY_ENABLED": { 2023-07-18T03:39:26.860 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "PWD_POLICY_ENABLED", 2023-07-18T03:39:26.860 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.860 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.860 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.860 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:26.861 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.861 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.861 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.861 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.862 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.862 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.862 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.862 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.862 INFO:teuthology.orchestra.run.smithi101.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-07-18T03:39:26.863 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-07-18T03:39:26.863 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.863 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.863 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.863 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-07-18T03:39:26.864 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.864 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.864 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.864 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.864 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.865 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.865 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.866 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.866 INFO:teuthology.orchestra.run.smithi101.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-07-18T03:39:26.866 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-07-18T03:39:26.867 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:26.867 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.868 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.868 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "10", 2023-07-18T03:39:26.869 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.869 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.869 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.870 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.870 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.871 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.871 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.872 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.872 INFO:teuthology.orchestra.run.smithi101.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-07-18T03:39:26.872 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-07-18T03:39:26.873 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:26.873 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.874 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.874 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "8", 2023-07-18T03:39:26.875 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.875 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.876 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.876 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.876 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.876 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.876 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.877 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.877 INFO:teuthology.orchestra.run.smithi101.stdout: "REST_REQUESTS_TIMEOUT": { 2023-07-18T03:39:26.877 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-07-18T03:39:26.877 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:26.877 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.878 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.878 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "45", 2023-07-18T03:39:26.878 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.878 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.878 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.879 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.879 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.879 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.879 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.879 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.880 INFO:teuthology.orchestra.run.smithi101.stdout: "RGW_API_ACCESS_KEY": { 2023-07-18T03:39:26.880 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "RGW_API_ACCESS_KEY", 2023-07-18T03:39:26.880 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.880 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.880 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.881 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.881 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.881 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.881 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.881 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.882 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.882 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.882 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.882 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.882 INFO:teuthology.orchestra.run.smithi101.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-07-18T03:39:26.883 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-07-18T03:39:26.883 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.883 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.883 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.883 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "admin", 2023-07-18T03:39:26.884 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.884 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.884 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.884 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.884 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.885 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.885 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.885 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.885 INFO:teuthology.orchestra.run.smithi101.stdout: "RGW_API_SECRET_KEY": { 2023-07-18T03:39:26.885 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "RGW_API_SECRET_KEY", 2023-07-18T03:39:26.886 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.886 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.886 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.886 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.886 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.887 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.887 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.887 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.887 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.887 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.888 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.888 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.888 INFO:teuthology.orchestra.run.smithi101.stdout: "RGW_API_SSL_VERIFY": { 2023-07-18T03:39:26.888 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "RGW_API_SSL_VERIFY", 2023-07-18T03:39:26.888 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.889 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.889 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.889 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:26.889 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.889 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.890 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.890 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.890 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.890 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.890 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.890 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.891 INFO:teuthology.orchestra.run.smithi101.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-07-18T03:39:26.891 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-07-18T03:39:26.891 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:26.891 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.891 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.892 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "0", 2023-07-18T03:39:26.892 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.892 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.892 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.893 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.893 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.893 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.893 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.893 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.894 INFO:teuthology.orchestra.run.smithi101.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-07-18T03:39:26.894 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-07-18T03:39:26.894 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:26.894 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.894 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.895 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "10", 2023-07-18T03:39:26.895 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.895 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.895 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.895 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.895 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.896 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.896 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.896 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.896 INFO:teuthology.orchestra.run.smithi101.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-07-18T03:39:26.897 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-07-18T03:39:26.897 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:26.897 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.897 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.897 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "5", 2023-07-18T03:39:26.897 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.898 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.898 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.899 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.899 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.899 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.899 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.899 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.900 INFO:teuthology.orchestra.run.smithi101.stdout: "cross_origin_url": { 2023-07-18T03:39:26.900 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "cross_origin_url", 2023-07-18T03:39:26.900 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.900 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.901 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.901 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.901 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.901 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.901 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.902 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.902 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.902 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.902 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.902 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.903 INFO:teuthology.orchestra.run.smithi101.stdout: "crt_file": { 2023-07-18T03:39:26.903 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "crt_file", 2023-07-18T03:39:26.903 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.903 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.903 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.904 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.904 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.904 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.904 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.904 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.905 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.905 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.905 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.905 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.905 INFO:teuthology.orchestra.run.smithi101.stdout: "debug": { 2023-07-18T03:39:26.906 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "debug", 2023-07-18T03:39:26.906 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.906 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.906 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.906 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.907 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.907 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.907 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.907 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Enable/disable debug options", 2023-07-18T03:39:26.907 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.908 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.908 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.908 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.908 INFO:teuthology.orchestra.run.smithi101.stdout: "jwt_token_ttl": { 2023-07-18T03:39:26.908 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "jwt_token_ttl", 2023-07-18T03:39:26.909 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:26.909 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.909 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.909 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "28800", 2023-07-18T03:39:26.909 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.910 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.910 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.910 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.910 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.910 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.911 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.911 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.911 INFO:teuthology.orchestra.run.smithi101.stdout: "key_file": { 2023-07-18T03:39:26.911 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "key_file", 2023-07-18T03:39:26.911 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.912 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.912 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.912 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.912 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.912 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.913 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.913 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.913 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.913 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.913 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.914 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.914 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:26.914 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:26.914 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.914 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.915 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.915 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.915 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.915 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.915 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.916 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.916 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.916 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.917 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.917 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.917 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.917 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.917 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.918 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.918 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.918 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.918 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.918 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:26.919 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:26.919 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.919 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.919 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.919 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.920 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.920 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.920 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.920 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.920 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.921 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.921 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.921 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.921 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:26.921 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:26.922 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.922 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.922 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.922 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:26.922 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.923 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.923 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.923 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.923 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.923 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.924 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.924 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.924 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.924 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.924 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.925 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.925 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.925 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.925 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.925 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:26.926 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:26.926 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.926 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.926 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.926 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.927 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.927 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.927 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.927 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.927 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.928 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.928 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.928 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.928 INFO:teuthology.orchestra.run.smithi101.stdout: "motd": { 2023-07-18T03:39:26.928 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "motd", 2023-07-18T03:39:26.929 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.929 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.929 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.929 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.929 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.930 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.930 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.930 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "The message of the day", 2023-07-18T03:39:26.930 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.930 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.931 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.931 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.931 INFO:teuthology.orchestra.run.smithi101.stdout: "redirect_resolve_ip_addr": { 2023-07-18T03:39:26.931 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "redirect_resolve_ip_addr", 2023-07-18T03:39:26.931 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.932 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.932 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.932 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.932 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.932 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.933 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.933 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.933 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.933 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.933 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.934 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.934 INFO:teuthology.orchestra.run.smithi101.stdout: "server_addr": { 2023-07-18T03:39:26.934 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "server_addr", 2023-07-18T03:39:26.934 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.934 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.935 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.935 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "::", 2023-07-18T03:39:26.935 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.935 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.935 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.936 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.936 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.936 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.936 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.936 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.937 INFO:teuthology.orchestra.run.smithi101.stdout: "server_port": { 2023-07-18T03:39:26.937 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "server_port", 2023-07-18T03:39:26.937 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:26.937 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.937 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.938 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "8080", 2023-07-18T03:39:26.938 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.938 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.938 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.938 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.939 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.939 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.939 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.939 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.939 INFO:teuthology.orchestra.run.smithi101.stdout: "ssl": { 2023-07-18T03:39:26.940 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "ssl", 2023-07-18T03:39:26.940 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.940 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.940 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.940 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:26.941 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.941 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.941 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.941 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.941 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.942 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.942 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.942 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.942 INFO:teuthology.orchestra.run.smithi101.stdout: "ssl_server_port": { 2023-07-18T03:39:26.942 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "ssl_server_port", 2023-07-18T03:39:26.943 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:26.943 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.943 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.943 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "8443", 2023-07-18T03:39:26.943 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.943 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.944 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.944 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.944 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.944 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.944 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.945 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.945 INFO:teuthology.orchestra.run.smithi101.stdout: "standby_behaviour": { 2023-07-18T03:39:26.945 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "standby_behaviour", 2023-07-18T03:39:26.945 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.946 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.946 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.946 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "redirect", 2023-07-18T03:39:26.946 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.946 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.947 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.947 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.947 INFO:teuthology.orchestra.run.smithi101.stdout: "redirect" 2023-07-18T03:39:26.947 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.947 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.948 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.948 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.948 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.948 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.948 INFO:teuthology.orchestra.run.smithi101.stdout: "standby_error_status_code": { 2023-07-18T03:39:26.949 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "standby_error_status_code", 2023-07-18T03:39:26.949 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:26.949 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.949 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.949 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "500", 2023-07-18T03:39:26.950 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "400", 2023-07-18T03:39:26.950 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "599", 2023-07-18T03:39:26.950 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.950 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.950 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.950 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.951 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.951 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.951 INFO:teuthology.orchestra.run.smithi101.stdout: "url_prefix": { 2023-07-18T03:39:26.951 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "url_prefix", 2023-07-18T03:39:26.951 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.952 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.952 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:26.952 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.952 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.952 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.953 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.953 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.953 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.953 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.953 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.954 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.954 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.954 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.954 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:26.954 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "devicehealth", 2023-07-18T03:39:26.955 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:26.955 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:26.955 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:26.955 INFO:teuthology.orchestra.run.smithi101.stdout: "enable_monitoring": { 2023-07-18T03:39:26.955 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "enable_monitoring", 2023-07-18T03:39:26.956 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.956 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.956 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.956 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:26.956 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.957 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.957 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.957 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "monitor device health metrics", 2023-07-18T03:39:26.957 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.957 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.958 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.958 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.958 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:26.958 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:26.958 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.959 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.959 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.959 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.959 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.959 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.960 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.960 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.960 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.960 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.960 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.961 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.961 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.961 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.961 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.961 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.962 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.962 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.962 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.962 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:26.962 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:26.963 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.963 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.963 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.963 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.963 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.963 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.964 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.964 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.964 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.964 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.964 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.965 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.965 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:26.965 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:26.965 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.965 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.966 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.966 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:26.966 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.966 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.966 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.967 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.967 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.967 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.967 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.967 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.968 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.968 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.968 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.968 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.968 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.969 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.969 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.969 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:26.969 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:26.969 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.970 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.970 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.970 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.970 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.970 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.971 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.971 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.971 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.971 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.971 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.972 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.972 INFO:teuthology.orchestra.run.smithi101.stdout: "mark_out_threshold": { 2023-07-18T03:39:26.972 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "mark_out_threshold", 2023-07-18T03:39:26.972 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:26.972 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.973 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.973 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "2419200", 2023-07-18T03:39:26.973 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.973 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.973 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.974 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-07-18T03:39:26.974 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.974 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.974 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.974 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.975 INFO:teuthology.orchestra.run.smithi101.stdout: "pool_name": { 2023-07-18T03:39:26.975 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "pool_name", 2023-07-18T03:39:26.975 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.975 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.975 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.976 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "device_health_metrics", 2023-07-18T03:39:26.976 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.976 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.976 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.976 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "name of pool in which to store device health metrics", 2023-07-18T03:39:26.977 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.977 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.977 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.977 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.977 INFO:teuthology.orchestra.run.smithi101.stdout: "retention_period": { 2023-07-18T03:39:26.978 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "retention_period", 2023-07-18T03:39:26.978 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:26.978 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.978 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.978 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "15552000", 2023-07-18T03:39:26.978 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.979 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.979 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.979 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "how long to retain device health metrics", 2023-07-18T03:39:26.979 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.979 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.980 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.980 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.980 INFO:teuthology.orchestra.run.smithi101.stdout: "scrape_frequency": { 2023-07-18T03:39:26.980 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "scrape_frequency", 2023-07-18T03:39:26.980 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:26.981 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.981 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.981 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "86400", 2023-07-18T03:39:26.981 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.981 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.982 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.982 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "how frequently to scrape device health metrics", 2023-07-18T03:39:26.982 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.982 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.982 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.983 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.983 INFO:teuthology.orchestra.run.smithi101.stdout: "self_heal": { 2023-07-18T03:39:26.983 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "self_heal", 2023-07-18T03:39:26.983 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.983 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.984 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.984 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:26.984 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.984 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.984 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.985 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-07-18T03:39:26.985 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.985 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.985 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.985 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.986 INFO:teuthology.orchestra.run.smithi101.stdout: "sleep_interval": { 2023-07-18T03:39:26.986 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "sleep_interval", 2023-07-18T03:39:26.986 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:26.986 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.986 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.987 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "600", 2023-07-18T03:39:26.987 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.987 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.987 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.987 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "how frequently to wake up and check device health", 2023-07-18T03:39:26.988 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.988 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.988 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.988 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.988 INFO:teuthology.orchestra.run.smithi101.stdout: "warn_threshold": { 2023-07-18T03:39:26.989 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "warn_threshold", 2023-07-18T03:39:26.989 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:26.989 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.989 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.989 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "7257600", 2023-07-18T03:39:26.990 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.990 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.990 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.990 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-07-18T03:39:26.990 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.990 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.991 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.991 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.991 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:26.991 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.991 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:26.992 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "diskprediction_local", 2023-07-18T03:39:26.992 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:26.992 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:26.992 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:26.992 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:26.993 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:26.993 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:26.993 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.993 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.993 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:26.994 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.994 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.994 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:26.994 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:26.994 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:26.995 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:26.995 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:26.995 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:26.995 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:26.995 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:26.996 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.996 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.996 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.996 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.996 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.997 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:26.997 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:26.997 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:26.997 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:26.997 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:26.998 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:26.998 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:26.998 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:26.998 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:26.998 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:26.999 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:26.999 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:26.999 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:26.999 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:26.999 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:26.999 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:27.000 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.000 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.000 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.000 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:27.000 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.001 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.001 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.001 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.001 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.001 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.002 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.002 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.002 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.002 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.002 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.003 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.003 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.003 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.003 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.003 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:27.004 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:27.004 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.004 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.004 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.004 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.005 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.005 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.005 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.005 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.005 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.006 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.006 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.006 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.006 INFO:teuthology.orchestra.run.smithi101.stdout: "predict_interval": { 2023-07-18T03:39:27.006 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "predict_interval", 2023-07-18T03:39:27.007 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.007 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.007 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.007 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "86400", 2023-07-18T03:39:27.007 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.008 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.008 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.008 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.008 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.008 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.009 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.009 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.009 INFO:teuthology.orchestra.run.smithi101.stdout: "predictor_model": { 2023-07-18T03:39:27.009 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "predictor_model", 2023-07-18T03:39:27.009 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.010 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.010 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.010 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "prophetstor", 2023-07-18T03:39:27.010 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.010 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.011 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.011 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.011 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.011 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.011 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.012 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.012 INFO:teuthology.orchestra.run.smithi101.stdout: "sleep_interval": { 2023-07-18T03:39:27.012 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "sleep_interval", 2023-07-18T03:39:27.012 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.012 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.013 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.013 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "600", 2023-07-18T03:39:27.013 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.013 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.013 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.014 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.014 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.014 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.014 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.014 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.015 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.015 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.015 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:27.015 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "influx", 2023-07-18T03:39:27.015 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": false, 2023-07-18T03:39:27.016 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "influxdb python module not found", 2023-07-18T03:39:27.016 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:27.016 INFO:teuthology.orchestra.run.smithi101.stdout: "batch_size": { 2023-07-18T03:39:27.016 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "batch_size", 2023-07-18T03:39:27.016 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:27.017 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.017 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.017 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "5000", 2023-07-18T03:39:27.017 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.017 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.018 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.018 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-07-18T03:39:27.018 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.018 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.018 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.019 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.019 INFO:teuthology.orchestra.run.smithi101.stdout: "database": { 2023-07-18T03:39:27.019 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "database", 2023-07-18T03:39:27.019 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.019 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.020 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.020 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "ceph", 2023-07-18T03:39:27.020 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.020 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.020 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.021 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-07-18T03:39:27.021 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.021 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.021 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.021 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.022 INFO:teuthology.orchestra.run.smithi101.stdout: "hostname": { 2023-07-18T03:39:27.022 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "hostname", 2023-07-18T03:39:27.022 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.022 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.022 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.023 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.023 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.023 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.023 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.023 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "InfluxDB server hostname", 2023-07-18T03:39:27.024 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.024 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.024 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.024 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.024 INFO:teuthology.orchestra.run.smithi101.stdout: "interval": { 2023-07-18T03:39:27.025 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "interval", 2023-07-18T03:39:27.025 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:27.025 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.025 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.025 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "30", 2023-07-18T03:39:27.026 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "5", 2023-07-18T03:39:27.026 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.026 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.026 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-07-18T03:39:27.026 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.026 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.027 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.027 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.027 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:27.027 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:27.027 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.028 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.028 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.028 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.028 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.028 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.029 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.029 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.029 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.029 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.029 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.030 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.030 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.030 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.030 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.030 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.031 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.031 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.031 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.031 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:27.031 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:27.032 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.032 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.032 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.032 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.032 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.033 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.033 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.033 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.033 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.033 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.034 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.034 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.034 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:27.034 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:27.034 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.035 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.035 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.035 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:27.035 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.035 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.036 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.036 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.036 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.036 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.036 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.037 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.037 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.037 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.037 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.037 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.038 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.038 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.038 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.038 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:27.038 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:27.039 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.039 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.039 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.039 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.039 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.039 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.040 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.040 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.040 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.040 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.040 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.041 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.041 INFO:teuthology.orchestra.run.smithi101.stdout: "password": { 2023-07-18T03:39:27.041 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "password", 2023-07-18T03:39:27.041 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.041 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.042 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.042 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.042 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.042 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.042 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.043 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "password of InfluxDB server user", 2023-07-18T03:39:27.043 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.043 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.043 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.044 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.044 INFO:teuthology.orchestra.run.smithi101.stdout: "port": { 2023-07-18T03:39:27.044 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "port", 2023-07-18T03:39:27.044 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:27.044 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.045 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.045 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "8086", 2023-07-18T03:39:27.045 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.045 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.045 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.045 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "InfluxDB server port", 2023-07-18T03:39:27.046 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.046 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.046 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.046 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.046 INFO:teuthology.orchestra.run.smithi101.stdout: "ssl": { 2023-07-18T03:39:27.047 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "ssl", 2023-07-18T03:39:27.047 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.047 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.047 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.047 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "false", 2023-07-18T03:39:27.048 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.048 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.048 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.048 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-07-18T03:39:27.048 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.049 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.049 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.049 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.049 INFO:teuthology.orchestra.run.smithi101.stdout: "threads": { 2023-07-18T03:39:27.049 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "threads", 2023-07-18T03:39:27.050 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:27.050 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.050 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.050 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "5", 2023-07-18T03:39:27.050 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "1", 2023-07-18T03:39:27.051 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "32", 2023-07-18T03:39:27.051 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.051 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-07-18T03:39:27.051 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.051 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.052 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.052 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.052 INFO:teuthology.orchestra.run.smithi101.stdout: "username": { 2023-07-18T03:39:27.052 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "username", 2023-07-18T03:39:27.052 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.053 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.053 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.053 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.053 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.053 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.054 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.054 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "username of InfluxDB server user", 2023-07-18T03:39:27.054 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.054 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.054 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.054 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.055 INFO:teuthology.orchestra.run.smithi101.stdout: "verify_ssl": { 2023-07-18T03:39:27.055 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "verify_ssl", 2023-07-18T03:39:27.055 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.055 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.055 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.056 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "true", 2023-07-18T03:39:27.056 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.056 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.056 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.056 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-07-18T03:39:27.057 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.057 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.057 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.057 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.057 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.058 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.058 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:27.058 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "insights", 2023-07-18T03:39:27.058 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:27.058 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:27.059 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:27.059 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:27.059 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:27.059 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.059 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.060 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.060 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.060 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.060 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.060 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.061 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.061 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.061 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.061 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.061 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.061 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.062 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.062 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.062 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.062 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.062 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.063 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.063 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:27.063 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:27.063 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.063 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.064 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.064 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.064 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.064 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.064 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.065 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.065 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.065 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.065 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.065 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.066 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:27.066 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:27.066 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.066 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.066 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.067 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:27.067 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.067 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.067 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.067 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.068 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.068 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.068 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.068 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.068 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.069 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.069 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.069 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.069 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.069 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.070 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.070 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:27.070 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:27.070 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.070 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.070 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.071 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.071 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.071 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.071 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.071 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.072 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.072 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.072 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.072 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.072 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.073 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.073 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:27.073 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "iostat", 2023-07-18T03:39:27.073 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:27.073 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:27.074 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:27.074 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:27.074 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:27.074 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.074 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.075 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.075 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.075 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.075 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.075 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.076 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.076 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.076 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.076 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.076 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.077 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.077 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.077 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.077 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.077 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.078 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.078 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.078 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:27.078 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:27.078 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.079 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.079 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.079 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.079 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.079 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.080 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.080 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.080 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.080 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.080 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.081 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.081 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:27.081 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:27.081 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.081 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.082 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.082 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:27.082 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.082 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.082 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.083 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.083 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.083 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.083 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.083 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.084 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.084 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.084 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.084 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.084 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.084 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.085 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.085 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:27.085 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:27.085 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.085 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.086 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.086 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.086 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.086 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.086 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.087 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.087 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.087 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.087 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.087 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.088 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.088 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.088 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:27.088 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "k8sevents", 2023-07-18T03:39:27.088 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:27.089 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:27.089 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:27.089 INFO:teuthology.orchestra.run.smithi101.stdout: "ceph_event_retention_days": { 2023-07-18T03:39:27.089 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "ceph_event_retention_days", 2023-07-18T03:39:27.089 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:27.090 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.090 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.090 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "7", 2023-07-18T03:39:27.090 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.090 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.091 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.091 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Days to hold ceph event information within local cache", 2023-07-18T03:39:27.091 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.091 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.091 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.092 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.092 INFO:teuthology.orchestra.run.smithi101.stdout: "config_check_secs": { 2023-07-18T03:39:27.092 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "config_check_secs", 2023-07-18T03:39:27.092 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:27.092 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.093 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.093 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "10", 2023-07-18T03:39:27.093 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "10", 2023-07-18T03:39:27.093 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.093 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.094 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-07-18T03:39:27.094 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.094 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.094 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.094 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.095 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:27.095 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:27.095 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.095 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.095 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.095 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.096 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.096 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.096 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.096 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.096 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.097 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.097 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.097 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.097 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.097 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.098 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.098 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.098 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.098 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.098 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.099 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:27.099 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:27.099 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.099 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.099 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.100 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.100 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.100 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.100 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.100 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.101 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.101 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.101 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.101 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.101 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:27.102 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:27.102 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.102 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.102 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.102 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:27.103 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.103 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.103 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.103 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.103 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.103 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.104 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.104 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.104 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.104 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.104 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.105 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.105 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.105 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.105 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.105 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:27.106 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:27.106 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.106 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.106 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.106 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.107 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.107 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.107 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.107 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.107 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.108 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.108 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.108 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.108 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.108 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.109 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:27.109 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "localpool", 2023-07-18T03:39:27.109 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:27.109 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:27.109 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:27.110 INFO:teuthology.orchestra.run.smithi101.stdout: "failure_domain": { 2023-07-18T03:39:27.110 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "failure_domain", 2023-07-18T03:39:27.110 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.110 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.110 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.111 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "host", 2023-07-18T03:39:27.111 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.111 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.111 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.111 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "failure domain for any created local pool", 2023-07-18T03:39:27.111 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-07-18T03:39:27.112 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.112 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.112 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.112 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:27.112 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:27.113 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.113 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.113 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.113 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.113 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.114 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.114 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.114 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.114 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.114 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.115 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.115 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.115 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.115 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.115 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.116 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.116 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.116 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.116 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.116 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:27.117 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:27.117 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.117 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.117 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.117 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.118 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.118 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.118 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.118 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.118 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.119 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.119 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.119 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.119 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:27.119 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:27.120 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.120 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.120 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.120 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:27.120 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.121 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.121 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.121 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.121 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.121 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.122 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.122 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.122 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.122 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.122 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.123 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.123 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.123 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.123 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.123 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:27.124 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:27.124 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.124 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.124 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.124 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.125 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.125 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.125 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.125 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.125 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.125 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.126 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.126 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.126 INFO:teuthology.orchestra.run.smithi101.stdout: "min_size": { 2023-07-18T03:39:27.126 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "min_size", 2023-07-18T03:39:27.126 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:27.127 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.127 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.127 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.127 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.127 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.128 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.128 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "default min_size for any created local pool", 2023-07-18T03:39:27.128 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-07-18T03:39:27.128 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.128 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.129 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.129 INFO:teuthology.orchestra.run.smithi101.stdout: "num_rep": { 2023-07-18T03:39:27.129 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "num_rep", 2023-07-18T03:39:27.129 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:27.129 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.130 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.130 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "3", 2023-07-18T03:39:27.130 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.130 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.130 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.131 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "default replica count for any created local pool", 2023-07-18T03:39:27.131 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.131 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.131 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.131 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.132 INFO:teuthology.orchestra.run.smithi101.stdout: "pg_num": { 2023-07-18T03:39:27.132 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "pg_num", 2023-07-18T03:39:27.132 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:27.132 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.132 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.133 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "128", 2023-07-18T03:39:27.133 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.133 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.133 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.133 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "default pg_num for any created local pool", 2023-07-18T03:39:27.134 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.134 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.134 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.134 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.134 INFO:teuthology.orchestra.run.smithi101.stdout: "prefix": { 2023-07-18T03:39:27.135 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "prefix", 2023-07-18T03:39:27.135 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.135 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.135 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.135 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.136 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.136 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.136 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.136 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "name prefix for any created local pool", 2023-07-18T03:39:27.136 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.136 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.137 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.137 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.137 INFO:teuthology.orchestra.run.smithi101.stdout: "subtree": { 2023-07-18T03:39:27.137 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "subtree", 2023-07-18T03:39:27.137 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.138 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.138 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.138 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "rack", 2023-07-18T03:39:27.138 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.138 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.139 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.139 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "CRUSH level for which to create a local pool", 2023-07-18T03:39:27.139 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-07-18T03:39:27.139 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.139 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.140 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.140 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.140 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.140 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:27.140 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "mds_autoscaler", 2023-07-18T03:39:27.141 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:27.141 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:27.141 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:27.141 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:27.141 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:27.142 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.142 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.142 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.142 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.142 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.143 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.143 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.143 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.143 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.143 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.144 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.144 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.144 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.144 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.144 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.145 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.145 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.145 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.145 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.145 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:27.145 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:27.146 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.146 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.146 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.146 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.146 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.147 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.147 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.147 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.147 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.147 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.148 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.148 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.148 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:27.148 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:27.148 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.149 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.149 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.149 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:27.149 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.149 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.150 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.150 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.150 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.150 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.150 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.151 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.151 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.151 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.151 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.151 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.152 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.152 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.152 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.152 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:27.152 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:27.153 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.153 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.153 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.153 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.153 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.154 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.154 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.154 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.154 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.154 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.155 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.155 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.155 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.155 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.156 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:27.156 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "mirroring", 2023-07-18T03:39:27.156 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:27.156 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:27.156 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:27.157 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:27.157 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:27.157 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.157 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.157 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.158 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.158 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.158 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.158 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.158 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.159 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.159 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.159 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.159 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.159 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.160 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.160 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.160 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.160 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.160 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.161 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.161 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:27.161 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:27.161 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.161 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.162 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.162 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.162 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.162 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.162 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.163 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.163 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.163 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.163 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.163 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.163 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:27.164 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:27.164 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.164 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.164 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.164 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:27.165 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.165 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.165 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.165 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.165 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.166 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.166 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.166 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.166 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.166 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.167 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.167 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.167 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.167 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.167 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.168 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:27.168 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:27.168 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.168 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.168 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.169 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.169 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.169 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.169 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.169 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.170 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.170 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.170 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.170 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.170 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.171 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.171 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:27.171 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "nfs", 2023-07-18T03:39:27.171 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:27.171 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:27.172 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:27.172 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:27.172 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:27.172 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.172 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.173 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.173 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.173 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.173 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.173 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.174 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.174 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.174 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.174 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.174 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.174 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.175 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.175 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.175 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.175 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.175 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.176 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.176 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:27.176 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:27.176 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.176 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.177 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.177 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.177 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.177 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.177 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.178 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.178 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.178 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.178 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.178 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.179 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:27.179 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:27.179 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.180 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.180 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.180 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:27.180 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.181 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.181 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.181 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.181 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.181 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.182 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.182 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.182 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.182 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.182 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.183 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.183 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.183 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.183 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.183 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:27.184 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:27.184 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.184 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.184 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.184 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.185 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.185 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.185 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.185 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.186 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.186 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.186 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.186 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.187 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.187 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.187 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:27.187 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "orchestrator", 2023-07-18T03:39:27.188 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:27.188 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:27.188 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:27.188 INFO:teuthology.orchestra.run.smithi101.stdout: "fail_fs": { 2023-07-18T03:39:27.189 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "fail_fs", 2023-07-18T03:39:27.189 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.189 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.189 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.190 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.190 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.190 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.190 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.191 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-07-18T03:39:27.191 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.191 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.192 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.192 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.192 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:27.192 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:27.193 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.193 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.193 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.194 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.194 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.194 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.194 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.195 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.195 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.195 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.195 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.196 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.196 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.196 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.196 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.197 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.197 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.197 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.197 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.197 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:27.198 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:27.198 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.198 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.198 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.198 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.199 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.199 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.199 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.199 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.199 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.200 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.200 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.200 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.200 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:27.200 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:27.201 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.201 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.201 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.201 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:27.202 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.202 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.202 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.202 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.202 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.202 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.203 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.203 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.203 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.203 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.203 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.204 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.204 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.204 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.204 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.205 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:27.205 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:27.205 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.205 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.205 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.206 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.206 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.206 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.206 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.206 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.207 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.207 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.207 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.207 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.207 INFO:teuthology.orchestra.run.smithi101.stdout: "orchestrator": { 2023-07-18T03:39:27.208 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "orchestrator", 2023-07-18T03:39:27.208 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.208 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.208 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.208 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.209 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.209 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.209 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.209 INFO:teuthology.orchestra.run.smithi101.stdout: "cephadm", 2023-07-18T03:39:27.209 INFO:teuthology.orchestra.run.smithi101.stdout: "rook", 2023-07-18T03:39:27.210 INFO:teuthology.orchestra.run.smithi101.stdout: "test_orchestrator" 2023-07-18T03:39:27.210 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.210 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Orchestrator backend", 2023-07-18T03:39:27.210 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.210 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.211 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.211 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.211 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.211 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.212 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:27.212 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "osd_perf_query", 2023-07-18T03:39:27.212 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:27.212 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:27.212 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:27.213 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:27.213 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:27.213 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.213 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.213 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.214 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.214 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.214 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.214 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.214 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.215 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.215 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.215 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.215 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.216 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.216 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.216 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.216 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.216 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.217 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.217 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.217 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:27.217 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:27.218 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.218 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.218 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.218 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.219 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.219 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.219 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.219 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.220 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.220 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.220 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.220 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.221 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:27.221 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:27.221 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.221 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.222 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.222 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:27.222 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.223 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.223 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.223 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.223 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.224 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.224 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.224 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.225 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.225 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.225 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.225 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.226 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.226 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.226 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.226 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:27.227 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:27.227 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.227 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.227 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.227 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.228 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.228 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.228 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.228 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.228 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.229 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.229 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.229 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.229 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.229 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.230 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:27.230 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "osd_support", 2023-07-18T03:39:27.230 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:27.230 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:27.230 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:27.231 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:27.231 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:27.231 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.231 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.231 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.232 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.232 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.232 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.232 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.232 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.233 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.233 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.233 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.233 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.233 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.234 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.234 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.234 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.234 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.234 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.235 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.235 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:27.235 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:27.235 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.235 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.236 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.236 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.236 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.237 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.237 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.237 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.237 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.238 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.238 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.238 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.238 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:27.238 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:27.239 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.239 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.239 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.239 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:27.239 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.240 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.240 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.240 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.240 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.240 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.241 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.241 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.241 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.241 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.241 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.242 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.242 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.242 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.242 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.242 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:27.243 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:27.243 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.243 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.243 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.243 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.244 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.244 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.244 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.244 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.244 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.245 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.245 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.245 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.245 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.245 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.246 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:27.246 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "pg_autoscaler", 2023-07-18T03:39:27.246 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:27.246 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:27.246 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:27.247 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:27.247 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:27.247 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.247 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.247 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.248 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.248 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.248 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.248 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.248 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.249 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.249 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.249 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.249 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.249 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.250 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.250 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.250 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.250 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.250 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.251 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.251 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:27.251 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:27.251 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.251 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.252 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.252 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.252 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.252 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.252 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.253 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.253 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.253 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.253 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.253 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.254 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:27.254 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:27.254 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.254 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.254 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.255 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:27.255 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.255 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.255 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.255 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.256 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.256 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.256 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.256 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.256 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.257 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.257 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.257 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.257 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.257 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.258 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.258 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:27.258 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:27.258 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.258 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.259 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.259 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.259 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.259 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.259 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.260 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.260 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.260 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.260 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.260 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.261 INFO:teuthology.orchestra.run.smithi101.stdout: "noautoscale": { 2023-07-18T03:39:27.261 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "noautoscale", 2023-07-18T03:39:27.261 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.261 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.261 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.262 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.262 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.262 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.262 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.262 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "global autoscale flag", 2023-07-18T03:39:27.263 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-07-18T03:39:27.263 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.263 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.263 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.263 INFO:teuthology.orchestra.run.smithi101.stdout: "sleep_interval": { 2023-07-18T03:39:27.264 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "sleep_interval", 2023-07-18T03:39:27.264 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:27.264 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.264 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.264 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "60", 2023-07-18T03:39:27.265 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.265 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.265 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.265 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.265 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.266 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.266 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.266 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.266 INFO:teuthology.orchestra.run.smithi101.stdout: "threshold": { 2023-07-18T03:39:27.266 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "threshold", 2023-07-18T03:39:27.267 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "float", 2023-07-18T03:39:27.267 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.267 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.267 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "3.0", 2023-07-18T03:39:27.267 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "1.0", 2023-07-18T03:39:27.268 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.268 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.268 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "scaling threshold", 2023-07-18T03:39:27.268 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-07-18T03:39:27.268 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.269 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.269 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.269 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.269 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.269 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:27.270 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "progress", 2023-07-18T03:39:27.270 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:27.270 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:27.270 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:27.270 INFO:teuthology.orchestra.run.smithi101.stdout: "allow_pg_recovery_event": { 2023-07-18T03:39:27.271 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "allow_pg_recovery_event", 2023-07-18T03:39:27.271 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.271 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.271 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.271 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.272 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.272 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.272 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.272 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "allow the module to show pg recovery progress", 2023-07-18T03:39:27.272 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.273 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.273 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.273 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.273 INFO:teuthology.orchestra.run.smithi101.stdout: "enabled": { 2023-07-18T03:39:27.273 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "enabled", 2023-07-18T03:39:27.274 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.274 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.274 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.274 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:27.274 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.275 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.275 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.275 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.275 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.275 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.276 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.276 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.276 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:27.276 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:27.276 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.277 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.277 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.277 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.277 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.277 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.278 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.278 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.278 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.278 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.278 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.279 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.279 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.279 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.279 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.279 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.280 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.280 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.280 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.280 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:27.281 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:27.281 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.281 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.281 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.281 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.281 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.282 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.282 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.282 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.282 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.282 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.283 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.283 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.283 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:27.283 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:27.283 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.284 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.284 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.284 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:27.284 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.284 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.285 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.285 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.285 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.285 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.285 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.286 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.286 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.286 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.286 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.286 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.287 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.287 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.287 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.287 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:27.287 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:27.288 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.288 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.288 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.288 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.288 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.289 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.289 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.289 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.289 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.289 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.290 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.290 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.290 INFO:teuthology.orchestra.run.smithi101.stdout: "max_completed_events": { 2023-07-18T03:39:27.290 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "max_completed_events", 2023-07-18T03:39:27.290 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:27.291 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.291 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.291 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "50", 2023-07-18T03:39:27.291 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.291 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.292 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.292 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "number of past completed events to remember", 2023-07-18T03:39:27.292 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.292 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.292 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.293 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.293 INFO:teuthology.orchestra.run.smithi101.stdout: "sleep_interval": { 2023-07-18T03:39:27.293 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "sleep_interval", 2023-07-18T03:39:27.293 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:27.293 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.294 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.294 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "5", 2023-07-18T03:39:27.294 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.294 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.294 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.295 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "how long the module is going to sleep", 2023-07-18T03:39:27.295 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.295 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.295 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.295 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.296 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.296 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.296 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:27.296 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "prometheus", 2023-07-18T03:39:27.296 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:27.297 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:27.297 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:27.297 INFO:teuthology.orchestra.run.smithi101.stdout: "cache": { 2023-07-18T03:39:27.297 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "cache", 2023-07-18T03:39:27.297 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.297 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.298 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.298 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:27.298 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.298 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.298 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.299 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.299 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.299 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.299 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.299 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.300 INFO:teuthology.orchestra.run.smithi101.stdout: "exclude_perf_counters": { 2023-07-18T03:39:27.300 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "exclude_perf_counters", 2023-07-18T03:39:27.300 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.300 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.300 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.301 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:27.301 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.301 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.301 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.301 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-07-18T03:39:27.302 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-07-18T03:39:27.302 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.302 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.302 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.302 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:27.303 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:27.303 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.303 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.303 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.303 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.304 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.304 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.304 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.304 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.304 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.305 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.305 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.305 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.305 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.305 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.306 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.306 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.306 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.306 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.306 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.307 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:27.307 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:27.307 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.307 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.307 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.308 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.308 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.308 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.308 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.308 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.309 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.309 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.309 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.309 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.309 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:27.310 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:27.310 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.310 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.310 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.310 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:27.310 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.311 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.311 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.311 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.311 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.311 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.312 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.312 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.312 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.312 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.312 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.313 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.313 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.313 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.313 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.313 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:27.314 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:27.314 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.314 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.314 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.315 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.315 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.315 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.315 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.315 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.316 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.316 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.316 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.316 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.316 INFO:teuthology.orchestra.run.smithi101.stdout: "rbd_stats_pools": { 2023-07-18T03:39:27.317 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "rbd_stats_pools", 2023-07-18T03:39:27.317 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.317 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.317 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.317 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.318 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.318 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.318 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.318 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.318 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.319 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.319 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.319 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.319 INFO:teuthology.orchestra.run.smithi101.stdout: "rbd_stats_pools_refresh_interval": { 2023-07-18T03:39:27.319 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-07-18T03:39:27.320 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:27.320 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.320 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.320 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "300", 2023-07-18T03:39:27.320 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.321 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.321 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.321 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.321 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.321 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.322 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.322 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.322 INFO:teuthology.orchestra.run.smithi101.stdout: "scrape_interval": { 2023-07-18T03:39:27.322 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "scrape_interval", 2023-07-18T03:39:27.322 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "float", 2023-07-18T03:39:27.323 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.323 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.323 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "15.0", 2023-07-18T03:39:27.323 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.323 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.324 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.324 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.324 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.324 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.324 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.325 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.325 INFO:teuthology.orchestra.run.smithi101.stdout: "server_addr": { 2023-07-18T03:39:27.325 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "server_addr", 2023-07-18T03:39:27.325 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.325 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.325 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.326 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "::", 2023-07-18T03:39:27.326 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.326 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.326 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.326 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-07-18T03:39:27.327 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.327 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.327 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.327 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.327 INFO:teuthology.orchestra.run.smithi101.stdout: "server_port": { 2023-07-18T03:39:27.328 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "server_port", 2023-07-18T03:39:27.328 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:27.328 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.328 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.328 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "9283", 2023-07-18T03:39:27.329 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.329 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.329 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.329 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-07-18T03:39:27.329 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.330 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.330 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.330 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.330 INFO:teuthology.orchestra.run.smithi101.stdout: "stale_cache_strategy": { 2023-07-18T03:39:27.330 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "stale_cache_strategy", 2023-07-18T03:39:27.331 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.331 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.331 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.331 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "log", 2023-07-18T03:39:27.331 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.332 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.332 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.332 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.332 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.332 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.333 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.333 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.333 INFO:teuthology.orchestra.run.smithi101.stdout: "standby_behaviour": { 2023-07-18T03:39:27.333 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "standby_behaviour", 2023-07-18T03:39:27.333 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.334 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.334 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.334 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "default", 2023-07-18T03:39:27.334 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.334 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.335 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.335 INFO:teuthology.orchestra.run.smithi101.stdout: "default", 2023-07-18T03:39:27.335 INFO:teuthology.orchestra.run.smithi101.stdout: "error" 2023-07-18T03:39:27.335 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.335 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.336 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.336 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.336 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.336 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.337 INFO:teuthology.orchestra.run.smithi101.stdout: "standby_error_status_code": { 2023-07-18T03:39:27.337 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "standby_error_status_code", 2023-07-18T03:39:27.337 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:27.337 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.337 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.338 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "500", 2023-07-18T03:39:27.338 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "400", 2023-07-18T03:39:27.338 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "599", 2023-07-18T03:39:27.338 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.338 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.339 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.339 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.339 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.339 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.339 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.339 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.340 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:27.340 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "rbd_support", 2023-07-18T03:39:27.340 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:27.340 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:27.340 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:27.341 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:27.341 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:27.341 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.341 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.341 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.342 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.342 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.342 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.342 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.342 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.343 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.343 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.343 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.343 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.343 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.344 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.344 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.344 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.344 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.344 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.345 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.345 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:27.345 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:27.345 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.345 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.346 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.346 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.346 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.346 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.346 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.347 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.347 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.347 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.347 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.347 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.348 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:27.348 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:27.348 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.348 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.348 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.349 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:27.349 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.349 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.349 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.349 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.350 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.350 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.350 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.350 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.350 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.351 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.351 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.351 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.351 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.351 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.352 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.352 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:27.352 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:27.352 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.352 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.353 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.353 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.353 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.353 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.353 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.354 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.354 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.354 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.354 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.354 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.355 INFO:teuthology.orchestra.run.smithi101.stdout: "max_concurrent_snap_create": { 2023-07-18T03:39:27.355 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "max_concurrent_snap_create", 2023-07-18T03:39:27.355 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:27.355 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.355 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.356 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "10", 2023-07-18T03:39:27.356 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.356 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.356 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.356 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.357 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.357 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.357 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.357 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.357 INFO:teuthology.orchestra.run.smithi101.stdout: "mirror_snapshot_schedule": { 2023-07-18T03:39:27.358 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "mirror_snapshot_schedule", 2023-07-18T03:39:27.358 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.358 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.358 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.358 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.359 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.359 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.359 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.359 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.359 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.360 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.360 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.360 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.360 INFO:teuthology.orchestra.run.smithi101.stdout: "trash_purge_schedule": { 2023-07-18T03:39:27.360 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "trash_purge_schedule", 2023-07-18T03:39:27.361 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.361 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.361 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.361 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.361 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.362 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.362 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.362 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.362 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.362 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.362 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.363 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.363 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.363 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.363 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:27.363 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "restful", 2023-07-18T03:39:27.364 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:27.364 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:27.364 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:27.364 INFO:teuthology.orchestra.run.smithi101.stdout: "enable_auth": { 2023-07-18T03:39:27.364 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "enable_auth", 2023-07-18T03:39:27.365 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.365 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.365 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.365 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:27.366 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.366 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.366 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.366 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.366 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.367 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.367 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.367 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.367 INFO:teuthology.orchestra.run.smithi101.stdout: "key_file": { 2023-07-18T03:39:27.367 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "key_file", 2023-07-18T03:39:27.367 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.368 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.368 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.368 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.368 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.369 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.369 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.369 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.369 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.369 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.369 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.370 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.370 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:27.370 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:27.370 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.370 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.371 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.371 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.371 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.371 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.371 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.372 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.372 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.372 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.372 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.372 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.373 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.373 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.373 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.373 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.373 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.374 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.374 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.374 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:27.374 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:27.374 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.375 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.375 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.375 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.375 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.375 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.376 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.376 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.376 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.376 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.377 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.377 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.377 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:27.377 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:27.377 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.378 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.378 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.378 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:27.378 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.378 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.379 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.379 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.379 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.379 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.379 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.380 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.380 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.380 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.380 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.380 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.381 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.381 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.381 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.381 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:27.381 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:27.382 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.382 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.382 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.382 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.382 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.383 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.383 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.383 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.383 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.383 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.384 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.384 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.384 INFO:teuthology.orchestra.run.smithi101.stdout: "server_addr": { 2023-07-18T03:39:27.384 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "server_addr", 2023-07-18T03:39:27.384 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.385 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.385 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.385 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.385 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.385 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.385 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.386 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.386 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.386 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.386 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.386 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.387 INFO:teuthology.orchestra.run.smithi101.stdout: "server_port": { 2023-07-18T03:39:27.387 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "server_port", 2023-07-18T03:39:27.387 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.387 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.387 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.388 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.388 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.388 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.388 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.388 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.389 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.389 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.389 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.389 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.389 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.390 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.390 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:27.390 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "rgw", 2023-07-18T03:39:27.390 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:27.390 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:27.391 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:27.391 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:27.391 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:27.391 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.391 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.392 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.392 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.392 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.392 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.392 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.393 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.393 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.393 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.393 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.393 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.394 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.394 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.394 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.394 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.395 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.395 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.395 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.395 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:27.395 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:27.396 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.396 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.396 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.396 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.396 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.397 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.397 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.397 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.397 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.397 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.398 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.398 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.398 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:27.398 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:27.398 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.399 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.399 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.399 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:27.399 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.399 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.400 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.400 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.400 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.400 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.400 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.401 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.401 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.401 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.401 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.401 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.402 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.402 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.402 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.402 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:27.402 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:27.403 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.403 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.403 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.403 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.403 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.404 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.404 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.404 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.404 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.404 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.405 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.405 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.405 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.405 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.405 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:27.406 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "rook", 2023-07-18T03:39:27.406 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:27.406 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:27.406 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:27.406 INFO:teuthology.orchestra.run.smithi101.stdout: "drive_group_interval": { 2023-07-18T03:39:27.407 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "drive_group_interval", 2023-07-18T03:39:27.407 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "float", 2023-07-18T03:39:27.407 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.407 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.407 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "300.0", 2023-07-18T03:39:27.408 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.408 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.408 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.408 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-07-18T03:39:27.408 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.409 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.409 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.409 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.409 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:27.409 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:27.410 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.410 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.410 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.410 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.410 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.411 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.411 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.411 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.411 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.411 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.412 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.412 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.412 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.412 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.412 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.413 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.413 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.413 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.413 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.413 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:27.414 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:27.414 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.414 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.414 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.414 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.415 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.415 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.415 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.415 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.415 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.416 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.416 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.416 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.416 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:27.416 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:27.417 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.417 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.417 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.417 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:27.417 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.418 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.418 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.418 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.418 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.418 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.419 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.419 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.419 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.419 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.419 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.420 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.420 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.420 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.420 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.420 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:27.421 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:27.421 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.421 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.421 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.421 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.422 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.422 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.422 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.422 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.422 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.423 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.423 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.423 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.423 INFO:teuthology.orchestra.run.smithi101.stdout: "storage_class": { 2023-07-18T03:39:27.423 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "storage_class", 2023-07-18T03:39:27.424 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.424 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.424 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.424 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "local", 2023-07-18T03:39:27.424 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.425 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.425 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.425 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-07-18T03:39:27.425 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.425 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.426 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.426 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.426 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.426 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.426 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:27.427 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "selftest", 2023-07-18T03:39:27.427 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:27.427 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:27.427 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:27.427 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:27.428 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:27.428 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.428 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.428 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.428 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.429 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.429 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.429 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.429 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.429 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.430 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.430 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.430 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.430 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.430 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.430 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.431 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.431 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.431 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.431 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.432 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:27.432 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:27.432 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.432 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.432 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.433 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.433 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.433 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.433 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.433 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.434 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.434 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.434 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.434 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.434 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:27.434 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:27.435 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.435 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.435 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.435 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:27.435 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.436 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.436 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.436 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.436 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.436 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.437 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.437 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.437 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.437 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.437 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.438 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.438 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.438 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.438 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.438 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:27.439 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:27.439 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.439 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.439 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.439 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.440 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.440 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.440 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.440 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.440 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.441 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.441 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.441 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.441 INFO:teuthology.orchestra.run.smithi101.stdout: "roption1": { 2023-07-18T03:39:27.441 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "roption1", 2023-07-18T03:39:27.442 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.442 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.442 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.442 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.442 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.443 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.443 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.443 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.443 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.443 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.444 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.444 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.444 INFO:teuthology.orchestra.run.smithi101.stdout: "roption2": { 2023-07-18T03:39:27.444 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "roption2", 2023-07-18T03:39:27.444 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.445 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.445 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.445 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "xyz", 2023-07-18T03:39:27.445 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.445 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.446 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.446 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.446 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.446 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.446 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.447 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.447 INFO:teuthology.orchestra.run.smithi101.stdout: "rwoption1": { 2023-07-18T03:39:27.447 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "rwoption1", 2023-07-18T03:39:27.447 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.447 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.447 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.448 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.448 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.448 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.448 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.448 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.449 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.449 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.449 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.449 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.449 INFO:teuthology.orchestra.run.smithi101.stdout: "rwoption2": { 2023-07-18T03:39:27.450 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "rwoption2", 2023-07-18T03:39:27.450 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:27.450 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.450 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.450 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.451 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.451 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.451 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.451 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.451 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.452 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.452 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.452 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.452 INFO:teuthology.orchestra.run.smithi101.stdout: "rwoption3": { 2023-07-18T03:39:27.452 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "rwoption3", 2023-07-18T03:39:27.453 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "float", 2023-07-18T03:39:27.453 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.453 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.453 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.453 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.454 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.454 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.454 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.454 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.454 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.455 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.455 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.455 INFO:teuthology.orchestra.run.smithi101.stdout: "rwoption4": { 2023-07-18T03:39:27.455 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "rwoption4", 2023-07-18T03:39:27.455 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.456 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.456 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.456 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.456 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.456 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.457 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.457 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.457 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.457 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.457 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.458 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.458 INFO:teuthology.orchestra.run.smithi101.stdout: "rwoption5": { 2023-07-18T03:39:27.458 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "rwoption5", 2023-07-18T03:39:27.458 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.458 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.459 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.459 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.459 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.459 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.459 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.460 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.460 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.460 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.460 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.460 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.461 INFO:teuthology.orchestra.run.smithi101.stdout: "rwoption6": { 2023-07-18T03:39:27.461 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "rwoption6", 2023-07-18T03:39:27.461 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.461 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.461 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.462 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:27.462 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.462 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.462 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.462 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.463 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.463 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.463 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.463 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.463 INFO:teuthology.orchestra.run.smithi101.stdout: "rwoption7": { 2023-07-18T03:39:27.463 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "rwoption7", 2023-07-18T03:39:27.464 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:27.464 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.464 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.464 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.465 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "1", 2023-07-18T03:39:27.465 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "42", 2023-07-18T03:39:27.465 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.465 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.465 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.465 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.466 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.466 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.466 INFO:teuthology.orchestra.run.smithi101.stdout: "testkey": { 2023-07-18T03:39:27.466 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "testkey", 2023-07-18T03:39:27.466 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.467 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.467 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.467 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.467 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.467 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.468 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.468 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.468 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.468 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.468 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.469 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.469 INFO:teuthology.orchestra.run.smithi101.stdout: "testlkey": { 2023-07-18T03:39:27.469 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "testlkey", 2023-07-18T03:39:27.469 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.469 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.470 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.470 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.470 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.470 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.470 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.471 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.471 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.471 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.471 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.471 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.472 INFO:teuthology.orchestra.run.smithi101.stdout: "testnewline": { 2023-07-18T03:39:27.472 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "testnewline", 2023-07-18T03:39:27.472 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.472 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.472 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.473 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.473 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.473 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.473 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.473 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.474 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.474 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.474 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.474 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.474 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.475 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.475 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:27.475 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "snap_schedule", 2023-07-18T03:39:27.475 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:27.475 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:27.476 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:27.476 INFO:teuthology.orchestra.run.smithi101.stdout: "allow_m_granularity": { 2023-07-18T03:39:27.476 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "allow_m_granularity", 2023-07-18T03:39:27.476 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.476 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.477 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.477 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.477 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.477 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.477 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.478 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "allow minute scheduled snapshots", 2023-07-18T03:39:27.478 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.478 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.478 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.478 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.479 INFO:teuthology.orchestra.run.smithi101.stdout: "dump_on_update": { 2023-07-18T03:39:27.479 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "dump_on_update", 2023-07-18T03:39:27.479 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.479 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.479 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.480 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.480 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.480 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.480 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.480 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "dump database to debug log on update", 2023-07-18T03:39:27.481 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.481 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.481 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.481 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.481 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:27.482 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:27.482 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.482 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.482 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.482 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.482 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.483 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.483 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.483 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.483 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.483 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.484 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.484 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.484 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.484 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.484 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.485 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.485 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.485 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.485 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.485 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:27.486 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:27.486 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.486 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.486 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.486 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.487 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.487 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.487 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.487 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.487 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.488 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.488 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.488 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.488 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:27.488 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:27.489 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.489 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.489 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.489 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:27.489 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.490 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.490 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.490 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.490 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.490 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.491 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.491 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.491 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.491 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.491 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.492 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.492 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.492 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.492 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.492 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:27.493 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:27.493 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.493 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.493 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.493 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.494 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.494 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.494 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.494 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.494 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.495 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.495 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.495 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.495 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.495 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.496 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:27.496 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "stats", 2023-07-18T03:39:27.496 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:27.496 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:27.496 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:27.497 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:27.497 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:27.497 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.497 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.497 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.498 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.498 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.498 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.498 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.498 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.499 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.499 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.499 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.499 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.499 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.500 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.500 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.500 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.500 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.500 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.501 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.501 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:27.501 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:27.501 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.501 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.502 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.502 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.502 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.502 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.502 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.503 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.503 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.503 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.503 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.503 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.503 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:27.504 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:27.504 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.504 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.504 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.505 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:27.505 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.505 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.505 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.505 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.506 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.506 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.506 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.506 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.506 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.506 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.507 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.507 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.507 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.507 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.507 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.508 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:27.508 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:27.508 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.508 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.509 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.509 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.509 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.509 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.509 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.509 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.510 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.510 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.510 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.510 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.510 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.511 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.511 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:27.511 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "status", 2023-07-18T03:39:27.511 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:27.511 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:27.512 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:27.512 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:27.512 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:27.512 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.512 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.513 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.513 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.513 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.513 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.513 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.514 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.514 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.514 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.514 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.514 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.515 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.515 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.515 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.515 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.515 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.516 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.516 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.516 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:27.516 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:27.517 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.517 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.517 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.517 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.517 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.518 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.518 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.518 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.518 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.518 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.518 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.519 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.519 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:27.519 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:27.519 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.519 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.520 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.520 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:27.520 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.520 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.520 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.521 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.521 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.521 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.521 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.521 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.522 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.522 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.522 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.522 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.522 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.523 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.523 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.523 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:27.523 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:27.523 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.524 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.524 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.524 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.524 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.524 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.525 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.525 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.525 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.525 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.525 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.526 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.526 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.526 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.526 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:27.526 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "telegraf", 2023-07-18T03:39:27.527 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:27.527 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:27.527 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:27.527 INFO:teuthology.orchestra.run.smithi101.stdout: "address": { 2023-07-18T03:39:27.527 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "address", 2023-07-18T03:39:27.528 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.528 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.528 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.528 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-07-18T03:39:27.528 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.529 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.529 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.529 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.529 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.529 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.530 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.530 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.530 INFO:teuthology.orchestra.run.smithi101.stdout: "interval": { 2023-07-18T03:39:27.530 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "interval", 2023-07-18T03:39:27.530 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:27.531 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.531 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.531 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "15", 2023-07-18T03:39:27.531 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.531 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.532 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.532 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.532 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.532 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.532 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.533 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.533 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:27.533 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:27.533 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.533 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.533 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.534 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.534 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.534 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.534 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.534 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.535 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.535 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.535 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.535 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.535 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.536 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.536 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.536 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.536 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.536 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.537 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.537 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:27.537 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:27.537 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.537 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.538 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.538 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.538 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.538 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.538 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.539 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.539 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.539 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.539 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.539 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.540 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:27.540 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:27.540 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.540 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.540 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.541 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:27.541 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.541 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.541 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.541 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.542 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.542 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.542 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.542 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.542 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.543 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.543 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.543 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.543 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.543 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.544 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.544 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:27.544 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:27.544 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.544 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.545 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.545 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.545 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.545 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.545 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.546 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.546 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.546 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.546 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.546 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.547 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.547 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.547 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:27.547 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "telemetry", 2023-07-18T03:39:27.547 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:27.548 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:27.548 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:27.548 INFO:teuthology.orchestra.run.smithi101.stdout: "channel_basic": { 2023-07-18T03:39:27.548 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "channel_basic", 2023-07-18T03:39:27.548 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.549 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.549 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.549 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:27.549 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.549 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.550 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.550 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Share basic cluster information (size, version)", 2023-07-18T03:39:27.550 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.550 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.550 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.551 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.551 INFO:teuthology.orchestra.run.smithi101.stdout: "channel_crash": { 2023-07-18T03:39:27.551 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "channel_crash", 2023-07-18T03:39:27.551 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.551 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.551 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.552 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:27.552 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.552 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.552 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.553 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-07-18T03:39:27.553 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.553 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.553 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.553 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.553 INFO:teuthology.orchestra.run.smithi101.stdout: "channel_device": { 2023-07-18T03:39:27.554 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "channel_device", 2023-07-18T03:39:27.554 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.554 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.554 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.554 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:27.555 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.555 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.555 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.555 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-07-18T03:39:27.555 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.556 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.556 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.556 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.556 INFO:teuthology.orchestra.run.smithi101.stdout: "channel_ident": { 2023-07-18T03:39:27.556 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "channel_ident", 2023-07-18T03:39:27.557 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.557 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.557 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.557 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.557 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.558 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.558 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.558 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-07-18T03:39:27.558 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.558 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.559 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.559 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.559 INFO:teuthology.orchestra.run.smithi101.stdout: "channel_perf": { 2023-07-18T03:39:27.559 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "channel_perf", 2023-07-18T03:39:27.559 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.560 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.560 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.560 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.560 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.560 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.561 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.561 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Share various performance metrics of a cluster", 2023-07-18T03:39:27.561 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.561 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.561 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.562 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.562 INFO:teuthology.orchestra.run.smithi101.stdout: "contact": { 2023-07-18T03:39:27.562 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "contact", 2023-07-18T03:39:27.562 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.562 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.563 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.563 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.563 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.563 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.563 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.564 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.564 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.564 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.564 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.564 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.565 INFO:teuthology.orchestra.run.smithi101.stdout: "description": { 2023-07-18T03:39:27.565 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "description", 2023-07-18T03:39:27.565 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.565 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.565 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.566 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.566 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.566 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.566 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.566 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.567 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.567 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.567 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.567 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.567 INFO:teuthology.orchestra.run.smithi101.stdout: "device_url": { 2023-07-18T03:39:27.568 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "device_url", 2023-07-18T03:39:27.568 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.568 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.568 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.568 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-07-18T03:39:27.569 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.569 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.569 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.569 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.569 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.570 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.570 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.570 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.570 INFO:teuthology.orchestra.run.smithi101.stdout: "enabled": { 2023-07-18T03:39:27.570 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "enabled", 2023-07-18T03:39:27.571 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.571 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.572 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.572 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.572 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.572 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.572 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.573 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.573 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.573 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.573 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.573 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.574 INFO:teuthology.orchestra.run.smithi101.stdout: "interval": { 2023-07-18T03:39:27.574 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "interval", 2023-07-18T03:39:27.574 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:27.574 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.574 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.575 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "24", 2023-07-18T03:39:27.575 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "8", 2023-07-18T03:39:27.575 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.575 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.575 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.576 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.576 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.576 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.576 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.576 INFO:teuthology.orchestra.run.smithi101.stdout: "last_opt_revision": { 2023-07-18T03:39:27.577 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "last_opt_revision", 2023-07-18T03:39:27.577 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:27.577 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.577 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.577 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "1", 2023-07-18T03:39:27.578 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.578 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.578 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.578 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.578 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.579 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.579 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.579 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.579 INFO:teuthology.orchestra.run.smithi101.stdout: "leaderboard": { 2023-07-18T03:39:27.579 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "leaderboard", 2023-07-18T03:39:27.580 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.580 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.580 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.580 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.580 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.581 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.581 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.581 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.581 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.581 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.581 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.582 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.582 INFO:teuthology.orchestra.run.smithi101.stdout: "leaderboard_description": { 2023-07-18T03:39:27.582 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "leaderboard_description", 2023-07-18T03:39:27.582 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.582 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.583 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.583 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.583 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.583 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.583 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.584 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.584 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.584 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.584 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.584 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.585 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:27.585 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:27.585 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.585 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.585 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.586 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.586 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.586 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.586 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.586 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.587 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.587 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.587 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.587 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.587 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.588 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.588 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.588 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.588 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.588 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.589 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.589 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:27.589 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:27.589 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.589 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.590 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.590 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.590 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.590 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.590 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.591 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.591 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.591 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.591 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.591 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.592 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:27.592 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:27.592 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.592 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.592 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.593 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:27.593 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.593 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.593 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.593 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.593 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.594 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.594 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.594 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.594 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.594 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.595 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.595 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.595 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.595 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.595 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.596 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:27.596 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:27.596 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.596 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.597 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.597 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.597 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.597 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.597 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.597 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.598 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.598 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.598 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.598 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.598 INFO:teuthology.orchestra.run.smithi101.stdout: "organization": { 2023-07-18T03:39:27.599 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "organization", 2023-07-18T03:39:27.599 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.599 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.599 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.599 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.600 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.600 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.600 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.600 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.600 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.601 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.601 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.601 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.601 INFO:teuthology.orchestra.run.smithi101.stdout: "proxy": { 2023-07-18T03:39:27.601 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "proxy", 2023-07-18T03:39:27.602 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.602 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.602 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.602 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.602 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.603 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.603 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.603 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.603 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.603 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.604 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.604 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.604 INFO:teuthology.orchestra.run.smithi101.stdout: "url": { 2023-07-18T03:39:27.604 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "url", 2023-07-18T03:39:27.604 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.605 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.605 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.605 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-07-18T03:39:27.605 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.605 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.606 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.606 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.606 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.606 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.606 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.607 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.607 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.607 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.607 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:27.607 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "test_orchestrator", 2023-07-18T03:39:27.608 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:27.608 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:27.608 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:27.608 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:27.608 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:27.609 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.609 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.609 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.609 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.609 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.610 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.610 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.610 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.610 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.610 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.611 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.611 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.611 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.611 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.611 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.612 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.612 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.612 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.612 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.612 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:27.613 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:27.613 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.613 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.613 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.613 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.613 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.614 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.614 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.614 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.614 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.614 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.615 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.615 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.615 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:27.615 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:27.615 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.616 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.616 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.616 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:27.616 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.616 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.617 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.617 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.617 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.617 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.617 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.618 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.618 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.618 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.618 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.618 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.619 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.619 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.619 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.619 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:27.619 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:27.620 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.620 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.620 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.620 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.620 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.621 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.621 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.621 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.621 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.621 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.622 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.622 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.622 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.622 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.622 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:27.623 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "volumes", 2023-07-18T03:39:27.623 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:27.623 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:27.623 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:27.624 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:27.624 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:27.624 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.624 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.624 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.625 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.625 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.625 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.625 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.625 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.626 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.626 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.626 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.626 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.626 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.627 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.627 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.627 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.627 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.627 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.628 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.628 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:27.628 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:27.628 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.628 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.629 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.629 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.629 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.629 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.629 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.630 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.630 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.630 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.630 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.630 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.631 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:27.631 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:27.631 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.631 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.631 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.632 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:27.632 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.632 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.632 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.632 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.633 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.633 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.633 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.633 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.633 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.634 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.634 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.634 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.634 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.634 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.635 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.635 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:27.635 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:27.635 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.635 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.636 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.636 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.636 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.636 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.636 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.637 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.637 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.637 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.637 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.637 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.638 INFO:teuthology.orchestra.run.smithi101.stdout: "max_concurrent_clones": { 2023-07-18T03:39:27.638 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "max_concurrent_clones", 2023-07-18T03:39:27.638 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:27.638 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.638 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.639 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "4", 2023-07-18T03:39:27.639 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.639 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.639 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.639 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Number of asynchronous cloner threads", 2023-07-18T03:39:27.640 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.640 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.640 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.640 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.640 INFO:teuthology.orchestra.run.smithi101.stdout: "snapshot_clone_delay": { 2023-07-18T03:39:27.641 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "snapshot_clone_delay", 2023-07-18T03:39:27.641 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:27.641 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.641 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.641 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "0", 2023-07-18T03:39:27.641 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.642 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.642 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.642 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-07-18T03:39:27.642 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.642 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.643 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.643 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.643 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.643 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.643 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:27.644 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "zabbix", 2023-07-18T03:39:27.644 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:27.644 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:27.645 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:27.645 INFO:teuthology.orchestra.run.smithi101.stdout: "discovery_interval": { 2023-07-18T03:39:27.646 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "discovery_interval", 2023-07-18T03:39:27.646 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "uint", 2023-07-18T03:39:27.647 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.647 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.648 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "100", 2023-07-18T03:39:27.648 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.648 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.649 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.649 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.650 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.650 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.651 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.651 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.651 INFO:teuthology.orchestra.run.smithi101.stdout: "identifier": { 2023-07-18T03:39:27.652 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "identifier", 2023-07-18T03:39:27.652 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.653 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.653 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.654 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.654 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.655 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.655 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.655 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.656 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.656 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.657 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.657 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.658 INFO:teuthology.orchestra.run.smithi101.stdout: "interval": { 2023-07-18T03:39:27.658 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "interval", 2023-07-18T03:39:27.658 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:27.659 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.659 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.660 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "60", 2023-07-18T03:39:27.660 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.661 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.661 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.661 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.662 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.662 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.662 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.663 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.663 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:27.663 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:27.664 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.664 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.664 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.665 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.665 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.665 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.666 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.666 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.666 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.667 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.667 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.667 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.668 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.668 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.669 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.669 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.669 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.670 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.670 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.670 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:27.671 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:27.671 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.671 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.672 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.672 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.672 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.673 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.673 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.673 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.673 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.674 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.674 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.674 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.675 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:27.675 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:27.675 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.676 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.676 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.676 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:27.677 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.677 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.677 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.678 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.678 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.678 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.679 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.679 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.679 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.680 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.680 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.680 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.681 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.681 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.681 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.682 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:27.682 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:27.682 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.683 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.683 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.683 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.684 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.684 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.684 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.685 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.685 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.685 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.686 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.686 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.686 INFO:teuthology.orchestra.run.smithi101.stdout: "zabbix_host": { 2023-07-18T03:39:27.686 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "zabbix_host", 2023-07-18T03:39:27.686 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.687 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.687 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.687 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.687 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.687 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.688 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.688 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.688 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.688 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.689 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.689 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.689 INFO:teuthology.orchestra.run.smithi101.stdout: "zabbix_port": { 2023-07-18T03:39:27.689 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "zabbix_port", 2023-07-18T03:39:27.689 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:27.690 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.690 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.690 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "10051", 2023-07-18T03:39:27.690 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.690 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.691 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.691 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.691 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.691 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.691 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.691 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.692 INFO:teuthology.orchestra.run.smithi101.stdout: "zabbix_sender": { 2023-07-18T03:39:27.692 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "zabbix_sender", 2023-07-18T03:39:27.692 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.692 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.692 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.693 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-07-18T03:39:27.693 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.693 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.693 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.693 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.694 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.694 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.694 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.694 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.694 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.695 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.695 INFO:teuthology.orchestra.run.smithi101.stdout: ] 2023-07-18T03:39:27.695 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.695 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.695 INFO:teuthology.orchestra.run.smithi101.stdout: "modules": [ 2023-07-18T03:39:27.696 INFO:teuthology.orchestra.run.smithi101.stdout: "cephadm", 2023-07-18T03:39:27.696 INFO:teuthology.orchestra.run.smithi101.stdout: "dashboard", 2023-07-18T03:39:27.696 INFO:teuthology.orchestra.run.smithi101.stdout: "iostat", 2023-07-18T03:39:27.696 INFO:teuthology.orchestra.run.smithi101.stdout: "nfs", 2023-07-18T03:39:27.696 INFO:teuthology.orchestra.run.smithi101.stdout: "restful", 2023-07-18T03:39:27.697 INFO:teuthology.orchestra.run.smithi101.stdout: "snap_schedule" 2023-07-18T03:39:27.697 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.697 INFO:teuthology.orchestra.run.smithi101.stdout: "available_modules": [ 2023-07-18T03:39:27.697 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:27.697 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "alerts", 2023-07-18T03:39:27.698 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:27.698 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:27.698 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:27.698 INFO:teuthology.orchestra.run.smithi101.stdout: "interval": { 2023-07-18T03:39:27.698 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "interval", 2023-07-18T03:39:27.699 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:27.699 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.699 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.699 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "60", 2023-07-18T03:39:27.699 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.700 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.700 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.700 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "How frequently to reexamine health status", 2023-07-18T03:39:27.700 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.700 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.701 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.701 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.701 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:27.701 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:27.701 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.702 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.702 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.702 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.702 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.702 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.703 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.703 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.703 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.703 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.703 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.704 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.704 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.704 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.704 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.704 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.705 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.705 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.705 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.705 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:27.705 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:27.706 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.706 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.706 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.706 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.706 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.707 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.707 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.707 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.707 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.707 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.708 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.708 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.708 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:27.708 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:27.708 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.709 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.709 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.709 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:27.709 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.709 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.710 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.710 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.710 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.710 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.710 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.711 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.711 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.711 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.711 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.711 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.712 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.712 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.712 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.712 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:27.712 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:27.713 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.713 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.713 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.713 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.713 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.714 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.714 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.714 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.714 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.714 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.715 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.715 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.715 INFO:teuthology.orchestra.run.smithi101.stdout: "smtp_destination": { 2023-07-18T03:39:27.715 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "smtp_destination", 2023-07-18T03:39:27.715 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.716 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.716 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.716 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.716 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.716 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.717 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.717 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Email address to send alerts to", 2023-07-18T03:39:27.717 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.717 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.717 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.718 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.718 INFO:teuthology.orchestra.run.smithi101.stdout: "smtp_from_name": { 2023-07-18T03:39:27.718 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "smtp_from_name", 2023-07-18T03:39:27.718 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.718 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.719 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.719 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "Ceph", 2023-07-18T03:39:27.719 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.719 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.719 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.720 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Email From: name", 2023-07-18T03:39:27.720 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.720 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.720 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.720 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.721 INFO:teuthology.orchestra.run.smithi101.stdout: "smtp_host": { 2023-07-18T03:39:27.721 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "smtp_host", 2023-07-18T03:39:27.721 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.721 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.721 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.722 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.722 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.722 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.722 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.722 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "SMTP server", 2023-07-18T03:39:27.722 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.723 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.723 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.723 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.723 INFO:teuthology.orchestra.run.smithi101.stdout: "smtp_password": { 2023-07-18T03:39:27.723 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "smtp_password", 2023-07-18T03:39:27.724 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.724 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.724 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.724 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.724 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.725 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.725 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.725 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Password to authenticate with", 2023-07-18T03:39:27.725 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.725 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.726 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.726 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.726 INFO:teuthology.orchestra.run.smithi101.stdout: "smtp_port": { 2023-07-18T03:39:27.726 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "smtp_port", 2023-07-18T03:39:27.726 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:27.727 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.727 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.727 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "465", 2023-07-18T03:39:27.727 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.727 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.728 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.728 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "SMTP port", 2023-07-18T03:39:27.728 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.728 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.728 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.729 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.729 INFO:teuthology.orchestra.run.smithi101.stdout: "smtp_sender": { 2023-07-18T03:39:27.729 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "smtp_sender", 2023-07-18T03:39:27.729 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.729 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.730 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.730 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.730 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.730 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.730 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.731 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "SMTP envelope sender", 2023-07-18T03:39:27.731 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.731 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.731 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.731 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.732 INFO:teuthology.orchestra.run.smithi101.stdout: "smtp_ssl": { 2023-07-18T03:39:27.732 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "smtp_ssl", 2023-07-18T03:39:27.732 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.732 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.732 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.733 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:27.733 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.733 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.733 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.733 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Use SSL to connect to SMTP server", 2023-07-18T03:39:27.734 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.734 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.734 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.734 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.734 INFO:teuthology.orchestra.run.smithi101.stdout: "smtp_user": { 2023-07-18T03:39:27.735 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "smtp_user", 2023-07-18T03:39:27.735 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.735 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.735 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.735 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.736 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.736 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.736 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.736 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "User to authenticate as", 2023-07-18T03:39:27.736 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.737 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.737 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.737 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.737 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.737 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.738 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:27.738 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "balancer", 2023-07-18T03:39:27.738 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:27.738 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:27.738 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:27.739 INFO:teuthology.orchestra.run.smithi101.stdout: "active": { 2023-07-18T03:39:27.739 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "active", 2023-07-18T03:39:27.739 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.739 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.739 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.740 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:27.740 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.740 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.740 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.740 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "automatically balance PGs across cluster", 2023-07-18T03:39:27.741 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.741 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.741 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.741 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.741 INFO:teuthology.orchestra.run.smithi101.stdout: "begin_time": { 2023-07-18T03:39:27.742 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "begin_time", 2023-07-18T03:39:27.742 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.742 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.742 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.742 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "0000", 2023-07-18T03:39:27.743 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.743 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.743 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.743 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "beginning time of day to automatically balance", 2023-07-18T03:39:27.743 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-18T03:39:27.744 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.744 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.744 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.744 INFO:teuthology.orchestra.run.smithi101.stdout: "begin_weekday": { 2023-07-18T03:39:27.744 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "begin_weekday", 2023-07-18T03:39:27.745 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "uint", 2023-07-18T03:39:27.745 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.745 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.745 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "0", 2023-07-18T03:39:27.745 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "0", 2023-07-18T03:39:27.746 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "6", 2023-07-18T03:39:27.746 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.746 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-07-18T03:39:27.746 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-18T03:39:27.746 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.747 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.747 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.747 INFO:teuthology.orchestra.run.smithi101.stdout: "crush_compat_max_iterations": { 2023-07-18T03:39:27.747 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "crush_compat_max_iterations", 2023-07-18T03:39:27.747 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "uint", 2023-07-18T03:39:27.748 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.748 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.748 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "25", 2023-07-18T03:39:27.748 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "1", 2023-07-18T03:39:27.748 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "250", 2023-07-18T03:39:27.749 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.749 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-07-18T03:39:27.749 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.749 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.749 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.750 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.750 INFO:teuthology.orchestra.run.smithi101.stdout: "crush_compat_metrics": { 2023-07-18T03:39:27.750 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "crush_compat_metrics", 2023-07-18T03:39:27.750 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.750 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.750 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.751 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "pgs,objects,bytes", 2023-07-18T03:39:27.751 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.751 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.751 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.751 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-07-18T03:39:27.752 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-07-18T03:39:27.752 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.752 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.752 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.752 INFO:teuthology.orchestra.run.smithi101.stdout: "crush_compat_step": { 2023-07-18T03:39:27.753 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "crush_compat_step", 2023-07-18T03:39:27.753 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "float", 2023-07-18T03:39:27.753 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.753 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.753 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "0.5", 2023-07-18T03:39:27.754 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "0.001", 2023-07-18T03:39:27.754 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "0.999", 2023-07-18T03:39:27.754 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.754 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "aggressiveness of optimization", 2023-07-18T03:39:27.754 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-07-18T03:39:27.755 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.755 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.755 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.755 INFO:teuthology.orchestra.run.smithi101.stdout: "end_time": { 2023-07-18T03:39:27.755 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "end_time", 2023-07-18T03:39:27.756 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.756 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.756 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.756 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "2359", 2023-07-18T03:39:27.756 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.757 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.757 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.757 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "ending time of day to automatically balance", 2023-07-18T03:39:27.757 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-18T03:39:27.757 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.758 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.758 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.758 INFO:teuthology.orchestra.run.smithi101.stdout: "end_weekday": { 2023-07-18T03:39:27.758 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "end_weekday", 2023-07-18T03:39:27.758 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "uint", 2023-07-18T03:39:27.759 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.759 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.759 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "0", 2023-07-18T03:39:27.759 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "0", 2023-07-18T03:39:27.759 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "6", 2023-07-18T03:39:27.760 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.760 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-07-18T03:39:27.760 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-18T03:39:27.760 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.760 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.761 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.761 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:27.761 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:27.761 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.761 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.762 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.762 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.762 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.762 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.762 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.763 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.763 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.763 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.763 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.763 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.763 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.764 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.764 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.764 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.764 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.764 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.765 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.765 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:27.765 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:27.765 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.765 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.766 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.766 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.766 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.766 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.766 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.767 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.767 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.767 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.767 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.767 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.768 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:27.768 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:27.768 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.768 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.768 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.769 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:27.769 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.769 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.769 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.769 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.770 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.770 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.770 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.770 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.770 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.771 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.771 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.771 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.771 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.771 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.772 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.772 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:27.772 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:27.772 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.772 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.773 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.773 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.773 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.773 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.773 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.774 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.774 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.774 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.774 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.774 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.775 INFO:teuthology.orchestra.run.smithi101.stdout: "min_score": { 2023-07-18T03:39:27.775 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "min_score", 2023-07-18T03:39:27.775 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "float", 2023-07-18T03:39:27.775 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.775 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.776 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "0", 2023-07-18T03:39:27.776 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.776 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.776 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.776 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-07-18T03:39:27.777 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.777 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.777 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.777 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.777 INFO:teuthology.orchestra.run.smithi101.stdout: "mode": { 2023-07-18T03:39:27.778 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "mode", 2023-07-18T03:39:27.778 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.778 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.778 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.778 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "upmap", 2023-07-18T03:39:27.779 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.779 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.779 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.779 INFO:teuthology.orchestra.run.smithi101.stdout: "crush-compat", 2023-07-18T03:39:27.779 INFO:teuthology.orchestra.run.smithi101.stdout: "none", 2023-07-18T03:39:27.779 INFO:teuthology.orchestra.run.smithi101.stdout: "upmap" 2023-07-18T03:39:27.780 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.780 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Balancer mode", 2023-07-18T03:39:27.780 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.780 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.780 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.781 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.781 INFO:teuthology.orchestra.run.smithi101.stdout: "pool_ids": { 2023-07-18T03:39:27.781 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "pool_ids", 2023-07-18T03:39:27.781 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.781 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.782 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.782 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.782 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.782 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.782 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.783 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-07-18T03:39:27.783 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.783 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.783 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.783 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.784 INFO:teuthology.orchestra.run.smithi101.stdout: "sleep_interval": { 2023-07-18T03:39:27.784 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "sleep_interval", 2023-07-18T03:39:27.784 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:27.784 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.784 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.785 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "60", 2023-07-18T03:39:27.785 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.785 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.785 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.785 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-07-18T03:39:27.786 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.786 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.786 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.786 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.786 INFO:teuthology.orchestra.run.smithi101.stdout: "upmap_max_deviation": { 2023-07-18T03:39:27.787 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "upmap_max_deviation", 2023-07-18T03:39:27.787 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:27.787 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.787 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.787 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "5", 2023-07-18T03:39:27.788 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "1", 2023-07-18T03:39:27.788 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.788 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.788 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "deviation below which no optimization is attempted", 2023-07-18T03:39:27.788 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-07-18T03:39:27.789 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.789 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.789 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.789 INFO:teuthology.orchestra.run.smithi101.stdout: "upmap_max_optimizations": { 2023-07-18T03:39:27.789 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "upmap_max_optimizations", 2023-07-18T03:39:27.790 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "uint", 2023-07-18T03:39:27.790 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.790 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.790 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "10", 2023-07-18T03:39:27.790 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.790 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.791 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.791 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-07-18T03:39:27.791 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.791 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.791 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.792 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.792 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.792 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.792 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:27.793 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "cephadm", 2023-07-18T03:39:27.793 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:27.793 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:27.793 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:27.793 INFO:teuthology.orchestra.run.smithi101.stdout: "agent_down_multiplier": { 2023-07-18T03:39:27.793 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "agent_down_multiplier", 2023-07-18T03:39:27.794 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "float", 2023-07-18T03:39:27.794 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.794 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.794 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "3.0", 2023-07-18T03:39:27.794 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.795 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.795 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.795 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-07-18T03:39:27.795 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.795 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.796 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.796 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.796 INFO:teuthology.orchestra.run.smithi101.stdout: "agent_refresh_rate": { 2023-07-18T03:39:27.796 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "agent_refresh_rate", 2023-07-18T03:39:27.796 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:27.797 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.797 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.797 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "20", 2023-07-18T03:39:27.797 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.797 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.798 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.798 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-07-18T03:39:27.798 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.798 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.798 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.799 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.799 INFO:teuthology.orchestra.run.smithi101.stdout: "agent_starting_port": { 2023-07-18T03:39:27.799 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "agent_starting_port", 2023-07-18T03:39:27.799 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:27.799 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.800 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.800 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "4721", 2023-07-18T03:39:27.800 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.800 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.800 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.801 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-07-18T03:39:27.801 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.801 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.801 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.801 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.802 INFO:teuthology.orchestra.run.smithi101.stdout: "alertmanager_web_password": { 2023-07-18T03:39:27.802 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "alertmanager_web_password", 2023-07-18T03:39:27.802 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.802 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.802 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.803 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "admin", 2023-07-18T03:39:27.803 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.803 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.803 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.803 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Alertmanager web password", 2023-07-18T03:39:27.803 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.804 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.804 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.804 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.804 INFO:teuthology.orchestra.run.smithi101.stdout: "alertmanager_web_user": { 2023-07-18T03:39:27.804 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "alertmanager_web_user", 2023-07-18T03:39:27.805 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.805 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.805 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.805 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "admin", 2023-07-18T03:39:27.805 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.806 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.806 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.806 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Alertmanager web user", 2023-07-18T03:39:27.806 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.806 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.807 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.807 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.807 INFO:teuthology.orchestra.run.smithi101.stdout: "allow_ptrace": { 2023-07-18T03:39:27.807 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "allow_ptrace", 2023-07-18T03:39:27.807 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.808 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.808 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.808 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.808 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.808 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.809 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.809 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-07-18T03:39:27.809 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-07-18T03:39:27.809 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.809 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.810 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.810 INFO:teuthology.orchestra.run.smithi101.stdout: "autotune_interval": { 2023-07-18T03:39:27.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:25.251+0000 7fa148fff700 1 -- 172.21.15.101:0/511151978 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa12c0a1380 msgr2=0x7fa12c0a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:27.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:25.251+0000 7fa148fff700 1 --2- 172.21.15.101:0/511151978 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa12c0a1380 0x7fa12c0a3830 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa134009c60 tx=0x7fa134009260 comp rx=0 tx=0).stop 2023-07-18T03:39:27.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:25.251+0000 7fa148fff700 1 -- 172.21.15.101:0/511151978 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1440ff190 msgr2=0x7fa144198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:27.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:25.251+0000 7fa148fff700 1 --2- 172.21.15.101:0/511151978 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1440ff190 0x7fa144198340 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa138005910 tx=0x7fa138004710 comp rx=0 tx=0).stop 2023-07-18T03:39:27.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:25.252+0000 7fa148fff700 1 -- 172.21.15.101:0/511151978 shutdown_connections 2023-07-18T03:39:27.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:25.252+0000 7fa148fff700 1 --2- 172.21.15.101:0/511151978 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa14410c430 0x7fa14419c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:27.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:25.252+0000 7fa148fff700 1 --2- 172.21.15.101:0/511151978 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa1440fe880 0x7fa144197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:27.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:25.252+0000 7fa148fff700 1 --2- 172.21.15.101:0/511151978 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fa12c0a1380 0x7fa12c0a3830 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:27.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:25.252+0000 7fa148fff700 1 --2- 172.21.15.101:0/511151978 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1440ff190 0x7fa144198340 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:27.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:25.252+0000 7fa148fff700 1 -- 172.21.15.101:0/511151978 >> 172.21.15.101:0/511151978 conn(0x7fa1440fa490 msgr2=0x7fa14410ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:27.813 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:25.252+0000 7fa148fff700 1 -- 172.21.15.101:0/511151978 shutdown_connections 2023-07-18T03:39:27.813 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:39:25.252+0000 7fa148fff700 1 -- 172.21.15.101:0/511151978 wait complete. 2023-07-18T03:39:27.815 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "autotune_interval", 2023-07-18T03:39:27.815 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:27.815 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.815 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.816 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "600", 2023-07-18T03:39:27.816 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.816 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.816 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.816 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "how frequently to autotune daemon memory", 2023-07-18T03:39:27.817 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.817 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.817 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.817 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.817 INFO:teuthology.orchestra.run.smithi101.stdout: "autotune_memory_target_ratio": { 2023-07-18T03:39:27.818 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "autotune_memory_target_ratio", 2023-07-18T03:39:27.818 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "float", 2023-07-18T03:39:27.818 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.818 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.818 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "0.7", 2023-07-18T03:39:27.819 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.819 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.819 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.819 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-07-18T03:39:27.819 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.820 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.820 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.820 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.820 INFO:teuthology.orchestra.run.smithi101.stdout: "cgroups_split": { 2023-07-18T03:39:27.820 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "cgroups_split", 2023-07-18T03:39:27.821 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.821 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.821 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.821 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:27.821 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.822 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.822 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.822 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-07-18T03:39:27.822 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.822 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.823 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.823 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.823 INFO:teuthology.orchestra.run.smithi101.stdout: "config_checks_enabled": { 2023-07-18T03:39:27.823 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "config_checks_enabled", 2023-07-18T03:39:27.823 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.824 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.824 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.824 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.824 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.824 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.825 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.825 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-07-18T03:39:27.825 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.825 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.825 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.826 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.826 INFO:teuthology.orchestra.run.smithi101.stdout: "config_dashboard": { 2023-07-18T03:39:27.826 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "config_dashboard", 2023-07-18T03:39:27.826 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.826 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.827 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.827 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:27.827 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.827 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.827 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.828 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-07-18T03:39:27.828 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.828 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.828 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.828 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.829 INFO:teuthology.orchestra.run.smithi101.stdout: "container_image_alertmanager": { 2023-07-18T03:39:27.829 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "container_image_alertmanager", 2023-07-18T03:39:27.829 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.829 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.829 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.829 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-07-18T03:39:27.830 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.830 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.830 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.830 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Prometheus container image", 2023-07-18T03:39:27.830 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.831 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.831 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.831 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.831 INFO:teuthology.orchestra.run.smithi101.stdout: "container_image_base": { 2023-07-18T03:39:27.831 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "container_image_base", 2023-07-18T03:39:27.832 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.832 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.832 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.832 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "quay.io/ceph/ceph", 2023-07-18T03:39:27.832 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.833 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.833 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.833 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Container image name, without the tag", 2023-07-18T03:39:27.833 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.833 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.834 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.834 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.834 INFO:teuthology.orchestra.run.smithi101.stdout: "container_image_elasticsearch": { 2023-07-18T03:39:27.834 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "container_image_elasticsearch", 2023-07-18T03:39:27.834 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.835 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.835 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.835 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-07-18T03:39:27.835 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.835 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.836 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.836 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "elasticsearch container image", 2023-07-18T03:39:27.836 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.836 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.836 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.837 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.837 INFO:teuthology.orchestra.run.smithi101.stdout: "container_image_grafana": { 2023-07-18T03:39:27.837 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "container_image_grafana", 2023-07-18T03:39:27.837 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.837 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.837 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.838 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-07-18T03:39:27.838 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.838 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.838 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.838 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Prometheus container image", 2023-07-18T03:39:27.839 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.839 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.839 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.839 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.839 INFO:teuthology.orchestra.run.smithi101.stdout: "container_image_haproxy": { 2023-07-18T03:39:27.840 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "container_image_haproxy", 2023-07-18T03:39:27.840 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.840 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.840 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.840 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-07-18T03:39:27.841 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.841 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.841 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.841 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "HAproxy container image", 2023-07-18T03:39:27.841 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.842 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.842 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.842 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.842 INFO:teuthology.orchestra.run.smithi101.stdout: "container_image_jaeger_agent": { 2023-07-18T03:39:27.842 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "container_image_jaeger_agent", 2023-07-18T03:39:27.843 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.843 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.843 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.843 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-07-18T03:39:27.843 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.844 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.844 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.844 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Jaeger agent container image", 2023-07-18T03:39:27.844 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.844 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.845 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.845 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.845 INFO:teuthology.orchestra.run.smithi101.stdout: "container_image_jaeger_collector": { 2023-07-18T03:39:27.845 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "container_image_jaeger_collector", 2023-07-18T03:39:27.845 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.846 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.846 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.846 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-07-18T03:39:27.846 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.846 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.847 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.847 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Jaeger collector container image", 2023-07-18T03:39:27.847 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.847 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.847 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.847 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.848 INFO:teuthology.orchestra.run.smithi101.stdout: "container_image_jaeger_query": { 2023-07-18T03:39:27.848 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "container_image_jaeger_query", 2023-07-18T03:39:27.848 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.848 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.849 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.849 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-07-18T03:39:27.849 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.849 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.849 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.849 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Jaeger query container image", 2023-07-18T03:39:27.850 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.850 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.850 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.850 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.850 INFO:teuthology.orchestra.run.smithi101.stdout: "container_image_keepalived": { 2023-07-18T03:39:27.851 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "container_image_keepalived", 2023-07-18T03:39:27.851 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.851 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.851 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.851 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-07-18T03:39:27.852 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.852 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.852 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.852 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Keepalived container image", 2023-07-18T03:39:27.852 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.853 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.853 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.853 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.853 INFO:teuthology.orchestra.run.smithi101.stdout: "container_image_loki": { 2023-07-18T03:39:27.853 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "container_image_loki", 2023-07-18T03:39:27.854 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.854 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.854 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.854 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-07-18T03:39:27.854 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.855 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.855 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.855 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Loki container image", 2023-07-18T03:39:27.855 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.855 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.856 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.856 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.856 INFO:teuthology.orchestra.run.smithi101.stdout: "container_image_node_exporter": { 2023-07-18T03:39:27.856 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "container_image_node_exporter", 2023-07-18T03:39:27.857 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.857 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.857 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.857 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-07-18T03:39:27.857 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.858 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.858 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.858 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Prometheus container image", 2023-07-18T03:39:27.858 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.858 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.859 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.859 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.859 INFO:teuthology.orchestra.run.smithi101.stdout: "container_image_prometheus": { 2023-07-18T03:39:27.859 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "container_image_prometheus", 2023-07-18T03:39:27.859 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.860 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.860 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.860 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-07-18T03:39:27.860 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.860 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.861 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.861 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Prometheus container image", 2023-07-18T03:39:27.861 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.861 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.861 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.861 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.862 INFO:teuthology.orchestra.run.smithi101.stdout: "container_image_promtail": { 2023-07-18T03:39:27.862 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "container_image_promtail", 2023-07-18T03:39:27.862 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.862 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.862 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.863 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-07-18T03:39:27.863 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.863 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.863 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.863 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Promtail container image", 2023-07-18T03:39:27.864 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.864 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.864 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.864 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.864 INFO:teuthology.orchestra.run.smithi101.stdout: "container_image_snmp_gateway": { 2023-07-18T03:39:27.865 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "container_image_snmp_gateway", 2023-07-18T03:39:27.865 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.865 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.865 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.865 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-07-18T03:39:27.866 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.866 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.866 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.866 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "SNMP Gateway container image", 2023-07-18T03:39:27.866 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.867 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.867 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.867 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.867 INFO:teuthology.orchestra.run.smithi101.stdout: "container_init": { 2023-07-18T03:39:27.867 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "container_init", 2023-07-18T03:39:27.868 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.868 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.868 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.868 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:27.868 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.869 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.869 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.869 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Run podman/docker with `--init`", 2023-07-18T03:39:27.869 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.869 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.870 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.870 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.870 INFO:teuthology.orchestra.run.smithi101.stdout: "daemon_cache_timeout": { 2023-07-18T03:39:27.870 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "daemon_cache_timeout", 2023-07-18T03:39:27.870 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:27.871 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.871 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.871 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "600", 2023-07-18T03:39:27.871 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.871 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.871 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.872 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-07-18T03:39:27.872 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.872 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.872 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.872 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.873 INFO:teuthology.orchestra.run.smithi101.stdout: "default_cephadm_command_timeout": { 2023-07-18T03:39:27.873 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "default_cephadm_command_timeout", 2023-07-18T03:39:27.873 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:27.873 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.873 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.874 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "900", 2023-07-18T03:39:27.874 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.874 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.874 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.874 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-07-18T03:39:27.875 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.875 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.875 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.875 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.875 INFO:teuthology.orchestra.run.smithi101.stdout: "default_registry": { 2023-07-18T03:39:27.876 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "default_registry", 2023-07-18T03:39:27.876 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.876 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.876 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.876 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "docker.io", 2023-07-18T03:39:27.877 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.877 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.877 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.877 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-07-18T03:39:27.877 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.878 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.878 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.878 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.878 INFO:teuthology.orchestra.run.smithi101.stdout: "device_cache_timeout": { 2023-07-18T03:39:27.878 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "device_cache_timeout", 2023-07-18T03:39:27.879 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:27.879 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.879 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.879 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "1800", 2023-07-18T03:39:27.879 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.880 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.880 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.880 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "seconds to cache device inventory", 2023-07-18T03:39:27.880 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.880 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.881 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.881 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.881 INFO:teuthology.orchestra.run.smithi101.stdout: "device_enhanced_scan": { 2023-07-18T03:39:27.881 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "device_enhanced_scan", 2023-07-18T03:39:27.881 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.882 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.882 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.882 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.882 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.882 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.883 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.883 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Use libstoragemgmt during device scans", 2023-07-18T03:39:27.883 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.883 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.883 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.883 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.884 INFO:teuthology.orchestra.run.smithi101.stdout: "facts_cache_timeout": { 2023-07-18T03:39:27.884 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "facts_cache_timeout", 2023-07-18T03:39:27.884 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:27.884 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.884 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.885 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "60", 2023-07-18T03:39:27.885 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.885 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.885 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.885 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "seconds to cache host facts data", 2023-07-18T03:39:27.886 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.886 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.886 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.886 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.886 INFO:teuthology.orchestra.run.smithi101.stdout: "host_check_interval": { 2023-07-18T03:39:27.887 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "host_check_interval", 2023-07-18T03:39:27.887 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:27.887 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.887 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.887 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "600", 2023-07-18T03:39:27.888 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.888 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.888 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.888 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "how frequently to perform a host check", 2023-07-18T03:39:27.888 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.889 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.889 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.889 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.889 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:27.889 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:27.890 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.890 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.890 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.890 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.890 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.890 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.891 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.891 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.891 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.891 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.891 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.892 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.892 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.892 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.892 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.892 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.893 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.893 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.893 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.893 INFO:teuthology.orchestra.run.smithi101.stdout: "log_refresh_metadata": { 2023-07-18T03:39:27.893 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_refresh_metadata", 2023-07-18T03:39:27.894 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.894 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.894 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.894 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.894 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.895 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.895 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.895 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-07-18T03:39:27.895 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.895 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.896 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.896 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.896 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:27.896 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:27.896 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.897 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.897 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.897 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:27.897 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.897 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.898 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.898 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-07-18T03:39:27.898 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.898 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.898 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.899 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.899 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:27.899 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:27.899 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.899 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.900 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.900 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:27.900 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.900 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.900 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.901 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.901 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.901 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.901 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.901 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.902 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.902 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.902 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.902 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.902 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.903 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.903 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.903 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:27.903 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:27.903 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.903 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.904 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.904 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.904 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.904 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.904 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.905 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.905 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.905 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.905 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.906 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.906 INFO:teuthology.orchestra.run.smithi101.stdout: "manage_etc_ceph_ceph_conf": { 2023-07-18T03:39:27.906 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-07-18T03:39:27.906 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.907 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.907 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.907 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.907 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.907 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.908 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.908 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-07-18T03:39:27.908 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.908 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.908 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.909 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.909 INFO:teuthology.orchestra.run.smithi101.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-07-18T03:39:27.909 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-07-18T03:39:27.909 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.909 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.910 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.910 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "*", 2023-07-18T03:39:27.910 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.910 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.910 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.911 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-07-18T03:39:27.911 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.911 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.911 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.911 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.912 INFO:teuthology.orchestra.run.smithi101.stdout: "max_count_per_host": { 2023-07-18T03:39:27.912 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "max_count_per_host", 2023-07-18T03:39:27.912 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:27.912 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.912 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.913 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "10", 2023-07-18T03:39:27.913 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.913 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.913 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.913 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "max number of daemons per service per host", 2023-07-18T03:39:27.914 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.914 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.914 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.914 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.914 INFO:teuthology.orchestra.run.smithi101.stdout: "max_osd_draining_count": { 2023-07-18T03:39:27.915 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "max_osd_draining_count", 2023-07-18T03:39:27.915 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:27.915 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.915 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.915 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "10", 2023-07-18T03:39:27.916 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.916 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.916 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.916 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-07-18T03:39:27.916 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.917 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.917 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.917 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.917 INFO:teuthology.orchestra.run.smithi101.stdout: "migration_current": { 2023-07-18T03:39:27.917 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "migration_current", 2023-07-18T03:39:27.918 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:27.918 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.918 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.918 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.918 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.919 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.919 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.919 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "internal - do not modify", 2023-07-18T03:39:27.919 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.919 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.920 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.920 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.920 INFO:teuthology.orchestra.run.smithi101.stdout: "mode": { 2023-07-18T03:39:27.920 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "mode", 2023-07-18T03:39:27.920 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.921 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.921 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.921 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "root", 2023-07-18T03:39:27.921 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.921 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.922 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.922 INFO:teuthology.orchestra.run.smithi101.stdout: "cephadm-package", 2023-07-18T03:39:27.922 INFO:teuthology.orchestra.run.smithi101.stdout: "root" 2023-07-18T03:39:27.922 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.922 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "mode for remote execution of cephadm", 2023-07-18T03:39:27.923 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.923 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.923 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.923 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.923 INFO:teuthology.orchestra.run.smithi101.stdout: "prometheus_alerts_path": { 2023-07-18T03:39:27.924 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "prometheus_alerts_path", 2023-07-18T03:39:27.924 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.924 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.924 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.924 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-07-18T03:39:27.925 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.925 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.925 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.925 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-07-18T03:39:27.925 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.925 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.926 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.926 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.926 INFO:teuthology.orchestra.run.smithi101.stdout: "prometheus_web_password": { 2023-07-18T03:39:27.926 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "prometheus_web_password", 2023-07-18T03:39:27.926 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.927 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.927 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.927 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "admin", 2023-07-18T03:39:27.927 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.927 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.928 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.928 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Prometheus web password", 2023-07-18T03:39:27.928 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.928 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.928 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.929 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.929 INFO:teuthology.orchestra.run.smithi101.stdout: "prometheus_web_user": { 2023-07-18T03:39:27.929 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "prometheus_web_user", 2023-07-18T03:39:27.929 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.929 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.930 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.930 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "admin", 2023-07-18T03:39:27.930 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.930 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.930 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.931 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Prometheus web user", 2023-07-18T03:39:27.931 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.931 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.931 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.931 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.932 INFO:teuthology.orchestra.run.smithi101.stdout: "registry_insecure": { 2023-07-18T03:39:27.932 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "registry_insecure", 2023-07-18T03:39:27.932 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.932 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.932 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.933 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.933 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.933 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.933 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.933 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-07-18T03:39:27.934 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.934 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.934 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.934 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.934 INFO:teuthology.orchestra.run.smithi101.stdout: "registry_password": { 2023-07-18T03:39:27.934 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "registry_password", 2023-07-18T03:39:27.935 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.935 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.935 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.935 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.935 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.936 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.936 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.936 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-07-18T03:39:27.936 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.936 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.937 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.937 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.937 INFO:teuthology.orchestra.run.smithi101.stdout: "registry_url": { 2023-07-18T03:39:27.937 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "registry_url", 2023-07-18T03:39:27.937 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.938 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.938 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.938 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.938 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.938 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.939 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.939 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-07-18T03:39:27.939 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.939 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.939 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.940 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.940 INFO:teuthology.orchestra.run.smithi101.stdout: "registry_username": { 2023-07-18T03:39:27.940 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "registry_username", 2023-07-18T03:39:27.940 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.940 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.941 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.941 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.941 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.941 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.941 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.942 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-07-18T03:39:27.942 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.942 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.942 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.942 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.943 INFO:teuthology.orchestra.run.smithi101.stdout: "secure_monitoring_stack": { 2023-07-18T03:39:27.943 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "secure_monitoring_stack", 2023-07-18T03:39:27.943 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.943 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.943 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.944 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.944 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.944 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.944 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.944 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-07-18T03:39:27.945 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.945 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.945 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.945 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.945 INFO:teuthology.orchestra.run.smithi101.stdout: "service_discovery_port": { 2023-07-18T03:39:27.946 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "service_discovery_port", 2023-07-18T03:39:27.946 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:27.946 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.946 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.946 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "8765", 2023-07-18T03:39:27.947 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.947 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.947 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.947 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "cephadm service discovery port", 2023-07-18T03:39:27.947 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.947 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.948 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.948 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.948 INFO:teuthology.orchestra.run.smithi101.stdout: "ssh_config_file": { 2023-07-18T03:39:27.948 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "ssh_config_file", 2023-07-18T03:39:27.948 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.949 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.949 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.949 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.949 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.949 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.950 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.950 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-07-18T03:39:27.950 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.950 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.950 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.951 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.951 INFO:teuthology.orchestra.run.smithi101.stdout: "use_agent": { 2023-07-18T03:39:27.951 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "use_agent", 2023-07-18T03:39:27.951 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.951 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.952 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.952 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.952 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.952 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.952 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.953 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-07-18T03:39:27.953 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.953 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.953 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.953 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.954 INFO:teuthology.orchestra.run.smithi101.stdout: "use_repo_digest": { 2023-07-18T03:39:27.954 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "use_repo_digest", 2023-07-18T03:39:27.954 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.954 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.954 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.955 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:27.955 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.955 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.955 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.955 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-07-18T03:39:27.956 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.956 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.956 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.956 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.956 INFO:teuthology.orchestra.run.smithi101.stdout: "warn_on_failed_host_check": { 2023-07-18T03:39:27.957 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "warn_on_failed_host_check", 2023-07-18T03:39:27.957 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.957 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.957 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.957 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:27.958 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.958 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.958 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.958 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "raise a health warning if the host check fails", 2023-07-18T03:39:27.958 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.959 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.959 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.959 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.959 INFO:teuthology.orchestra.run.smithi101.stdout: "warn_on_stray_daemons": { 2023-07-18T03:39:27.959 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "warn_on_stray_daemons", 2023-07-18T03:39:27.959 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.960 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.960 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.960 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:27.960 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.960 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.961 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.961 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-07-18T03:39:27.961 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.961 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.961 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.962 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.962 INFO:teuthology.orchestra.run.smithi101.stdout: "warn_on_stray_hosts": { 2023-07-18T03:39:27.962 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "warn_on_stray_hosts", 2023-07-18T03:39:27.962 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.962 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.963 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.963 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:27.963 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.963 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.963 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.964 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-07-18T03:39:27.964 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.964 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.964 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.964 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.965 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.965 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.965 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:27.965 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "crash", 2023-07-18T03:39:27.965 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:27.966 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:27.966 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:27.966 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:27.966 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:27.966 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.967 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.967 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.967 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.967 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.967 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.967 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.968 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.968 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.968 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.968 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.968 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.969 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.969 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.969 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.969 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.969 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.970 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.970 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.970 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:27.970 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:27.970 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.971 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.971 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.971 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.971 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.971 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.972 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.972 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.972 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.972 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.972 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.973 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.973 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:27.973 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:27.973 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.973 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.974 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.974 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:27.974 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.974 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.974 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:27.975 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:27.975 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:27.975 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:27.975 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:27.975 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:27.975 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:27.976 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:27.976 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.976 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.976 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.976 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.977 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.977 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:27.977 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:27.977 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.977 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.978 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.978 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.978 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.978 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.978 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.979 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.979 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.979 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.979 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.979 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.980 INFO:teuthology.orchestra.run.smithi101.stdout: "retain_interval": { 2023-07-18T03:39:27.980 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "retain_interval", 2023-07-18T03:39:27.980 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:27.980 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.980 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.981 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "31536000", 2023-07-18T03:39:27.981 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.981 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.981 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.981 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "how long to retain crashes before pruning them", 2023-07-18T03:39:27.982 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.982 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.982 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.982 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.982 INFO:teuthology.orchestra.run.smithi101.stdout: "warn_recent_interval": { 2023-07-18T03:39:27.983 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "warn_recent_interval", 2023-07-18T03:39:27.983 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:27.983 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.983 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:27.983 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "1209600", 2023-07-18T03:39:27.984 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.984 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.984 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.984 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "time interval in which to warn about recent crashes", 2023-07-18T03:39:27.984 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.985 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.985 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.985 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.985 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:27.985 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.986 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:27.986 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "dashboard", 2023-07-18T03:39:27.986 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:27.986 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:27.986 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:27.987 INFO:teuthology.orchestra.run.smithi101.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-07-18T03:39:27.987 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-07-18T03:39:27.987 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:27.987 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.987 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.988 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "10", 2023-07-18T03:39:27.988 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.988 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.988 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.988 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.989 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.989 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.989 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.989 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.989 INFO:teuthology.orchestra.run.smithi101.stdout: "ALERTMANAGER_API_HOST": { 2023-07-18T03:39:27.990 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "ALERTMANAGER_API_HOST", 2023-07-18T03:39:27.990 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:27.990 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.990 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.990 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:27.990 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.991 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.991 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.991 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.991 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.991 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.992 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.992 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.992 INFO:teuthology.orchestra.run.smithi101.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-07-18T03:39:27.992 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-07-18T03:39:27.992 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.993 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.993 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.993 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:27.993 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.993 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.994 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.994 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.994 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.994 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.994 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.995 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.995 INFO:teuthology.orchestra.run.smithi101.stdout: "AUDIT_API_ENABLED": { 2023-07-18T03:39:27.995 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "AUDIT_API_ENABLED", 2023-07-18T03:39:27.995 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.995 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.996 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.996 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:27.996 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.996 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.996 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.997 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.997 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:27.997 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:27.997 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:27.997 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:27.998 INFO:teuthology.orchestra.run.smithi101.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-07-18T03:39:27.998 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-07-18T03:39:27.998 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:27.998 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:27.998 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:27.999 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:27.999 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:27.999 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:27.999 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:27.999 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:27.999 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.000 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.000 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.000 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.000 INFO:teuthology.orchestra.run.smithi101.stdout: "ENABLE_BROWSABLE_API": { 2023-07-18T03:39:28.000 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "ENABLE_BROWSABLE_API", 2023-07-18T03:39:28.001 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.001 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.001 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.001 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:28.001 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.002 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.002 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.002 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.002 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.002 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.003 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.003 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.003 INFO:teuthology.orchestra.run.smithi101.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-07-18T03:39:28.003 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-07-18T03:39:28.003 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.004 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.004 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.004 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:28.004 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.004 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.005 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.005 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.005 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.005 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.005 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.006 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.006 INFO:teuthology.orchestra.run.smithi101.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-07-18T03:39:28.006 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-07-18T03:39:28.006 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.006 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.007 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.007 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:28.007 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.007 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.007 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.008 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.008 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.008 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.008 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.008 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.009 INFO:teuthology.orchestra.run.smithi101.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-07-18T03:39:28.009 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-07-18T03:39:28.009 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.009 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.009 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.010 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:28.010 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.010 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.010 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.010 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.010 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.011 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.011 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.011 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.011 INFO:teuthology.orchestra.run.smithi101.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-07-18T03:39:28.011 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-07-18T03:39:28.012 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.012 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.012 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.012 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:28.013 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.013 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.013 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.013 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.013 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.013 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.014 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.014 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.014 INFO:teuthology.orchestra.run.smithi101.stdout: "FEATURE_TOGGLE_NFS": { 2023-07-18T03:39:28.014 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-07-18T03:39:28.014 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.015 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.015 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.015 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:28.015 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.015 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.016 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.016 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.016 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.016 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.016 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.017 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.017 INFO:teuthology.orchestra.run.smithi101.stdout: "FEATURE_TOGGLE_RBD": { 2023-07-18T03:39:28.017 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-07-18T03:39:28.017 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.017 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.018 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.018 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:28.018 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.018 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.018 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.019 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.019 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.019 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.019 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.019 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.020 INFO:teuthology.orchestra.run.smithi101.stdout: "FEATURE_TOGGLE_RGW": { 2023-07-18T03:39:28.020 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-07-18T03:39:28.020 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.020 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.020 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.021 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:28.021 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.021 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.021 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.021 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.022 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.022 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.022 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.022 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.022 INFO:teuthology.orchestra.run.smithi101.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-07-18T03:39:28.022 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-07-18T03:39:28.023 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.023 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.023 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.023 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.023 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.024 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.024 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.024 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.024 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.024 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.025 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.025 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.025 INFO:teuthology.orchestra.run.smithi101.stdout: "GRAFANA_API_PASSWORD": { 2023-07-18T03:39:28.025 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "GRAFANA_API_PASSWORD", 2023-07-18T03:39:28.025 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.026 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.026 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.026 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "admin", 2023-07-18T03:39:28.026 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.026 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.027 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.027 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.027 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.027 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.027 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.028 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.028 INFO:teuthology.orchestra.run.smithi101.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-07-18T03:39:28.028 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-07-18T03:39:28.028 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.028 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.029 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.029 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:28.029 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.029 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.029 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.030 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.030 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.030 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.030 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.030 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.031 INFO:teuthology.orchestra.run.smithi101.stdout: "GRAFANA_API_URL": { 2023-07-18T03:39:28.031 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "GRAFANA_API_URL", 2023-07-18T03:39:28.031 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.031 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.031 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.032 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.032 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.032 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.032 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.032 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.033 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.033 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.033 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.033 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.033 INFO:teuthology.orchestra.run.smithi101.stdout: "GRAFANA_API_USERNAME": { 2023-07-18T03:39:28.034 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "GRAFANA_API_USERNAME", 2023-07-18T03:39:28.034 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.034 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.034 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.034 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "admin", 2023-07-18T03:39:28.035 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.035 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.035 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.035 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.035 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.035 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.036 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.036 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.036 INFO:teuthology.orchestra.run.smithi101.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-07-18T03:39:28.036 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-07-18T03:39:28.036 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.037 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.037 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.037 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.037 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.037 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.038 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.038 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.038 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.038 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.038 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.039 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.039 INFO:teuthology.orchestra.run.smithi101.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-07-18T03:39:28.039 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-07-18T03:39:28.039 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.039 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.040 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.040 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.040 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.040 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.040 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.041 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.041 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.041 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.041 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.041 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.042 INFO:teuthology.orchestra.run.smithi101.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-07-18T03:39:28.042 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-07-18T03:39:28.042 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.042 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.042 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.043 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:28.043 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.043 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.043 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.043 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.044 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.044 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.044 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.044 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.044 INFO:teuthology.orchestra.run.smithi101.stdout: "ISSUE_TRACKER_API_KEY": { 2023-07-18T03:39:28.044 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-07-18T03:39:28.045 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.045 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.045 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.045 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.045 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.046 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.046 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.046 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.046 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.046 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.047 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.047 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.047 INFO:teuthology.orchestra.run.smithi101.stdout: "PROMETHEUS_API_HOST": { 2023-07-18T03:39:28.047 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "PROMETHEUS_API_HOST", 2023-07-18T03:39:28.047 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.048 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.048 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.048 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.048 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.048 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.049 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.049 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.049 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.049 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.049 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.050 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.050 INFO:teuthology.orchestra.run.smithi101.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-07-18T03:39:28.050 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-07-18T03:39:28.050 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.050 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.051 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.051 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:28.051 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.051 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.051 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.051 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.052 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.052 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.052 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.052 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.052 INFO:teuthology.orchestra.run.smithi101.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-07-18T03:39:28.053 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-07-18T03:39:28.053 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.053 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.053 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.053 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.054 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.054 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.054 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.054 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.054 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.055 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.055 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.055 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.055 INFO:teuthology.orchestra.run.smithi101.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-07-18T03:39:28.055 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-07-18T03:39:28.056 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.056 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.056 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.056 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.056 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.057 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.057 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.057 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.057 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.057 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.058 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.058 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.058 INFO:teuthology.orchestra.run.smithi101.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-07-18T03:39:28.058 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-07-18T03:39:28.058 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.059 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.059 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.059 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:28.059 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.059 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.060 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.060 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.060 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.060 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.060 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.061 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.061 INFO:teuthology.orchestra.run.smithi101.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-07-18T03:39:28.061 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-07-18T03:39:28.061 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.061 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.061 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.062 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:28.062 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.062 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.062 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.062 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.063 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.063 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.063 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.063 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.063 INFO:teuthology.orchestra.run.smithi101.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-07-18T03:39:28.064 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-07-18T03:39:28.064 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.064 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.064 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.064 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.065 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.065 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.065 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.065 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.065 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.066 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.066 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.066 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.066 INFO:teuthology.orchestra.run.smithi101.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-07-18T03:39:28.066 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-07-18T03:39:28.067 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.067 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.067 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.067 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.067 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.068 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.068 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.068 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.068 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.068 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.069 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.069 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.069 INFO:teuthology.orchestra.run.smithi101.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-07-18T03:39:28.069 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-07-18T03:39:28.069 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.070 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.070 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.070 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.070 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.070 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.071 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.071 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.071 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.071 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.071 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.072 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.072 INFO:teuthology.orchestra.run.smithi101.stdout: "PWD_POLICY_ENABLED": { 2023-07-18T03:39:28.072 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "PWD_POLICY_ENABLED", 2023-07-18T03:39:28.072 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.072 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.073 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.073 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:28.073 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.073 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.073 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.074 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.074 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.074 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.074 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.074 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.075 INFO:teuthology.orchestra.run.smithi101.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-07-18T03:39:28.075 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-07-18T03:39:28.075 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.075 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.075 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.076 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-07-18T03:39:28.076 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.076 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.076 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.076 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.077 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.077 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.077 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.077 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.077 INFO:teuthology.orchestra.run.smithi101.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-07-18T03:39:28.078 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-07-18T03:39:28.078 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:28.078 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.078 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.078 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "10", 2023-07-18T03:39:28.079 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.079 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.079 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.079 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.079 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.080 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.080 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.080 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.080 INFO:teuthology.orchestra.run.smithi101.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-07-18T03:39:28.080 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-07-18T03:39:28.081 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:28.081 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.081 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.081 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "8", 2023-07-18T03:39:28.081 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.081 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.082 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.082 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.082 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.082 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.082 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.083 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.083 INFO:teuthology.orchestra.run.smithi101.stdout: "REST_REQUESTS_TIMEOUT": { 2023-07-18T03:39:28.083 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-07-18T03:39:28.083 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:28.083 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.084 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.084 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "45", 2023-07-18T03:39:28.084 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.084 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.084 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.085 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.085 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.085 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.085 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.085 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.086 INFO:teuthology.orchestra.run.smithi101.stdout: "RGW_API_ACCESS_KEY": { 2023-07-18T03:39:28.086 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "RGW_API_ACCESS_KEY", 2023-07-18T03:39:28.086 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.086 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.086 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.087 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.087 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.087 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.087 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.087 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.088 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.088 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.088 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.088 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.088 INFO:teuthology.orchestra.run.smithi101.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-07-18T03:39:28.089 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-07-18T03:39:28.089 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.089 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.089 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.089 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "admin", 2023-07-18T03:39:28.089 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.090 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.090 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.090 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.090 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.090 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.091 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.091 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.091 INFO:teuthology.orchestra.run.smithi101.stdout: "RGW_API_SECRET_KEY": { 2023-07-18T03:39:28.091 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "RGW_API_SECRET_KEY", 2023-07-18T03:39:28.091 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.092 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.092 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.092 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.092 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.092 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.093 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.093 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.093 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.093 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.093 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.094 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.094 INFO:teuthology.orchestra.run.smithi101.stdout: "RGW_API_SSL_VERIFY": { 2023-07-18T03:39:28.094 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "RGW_API_SSL_VERIFY", 2023-07-18T03:39:28.094 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.094 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.095 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.095 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:28.095 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.095 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.095 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.096 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.096 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.096 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.096 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.096 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.097 INFO:teuthology.orchestra.run.smithi101.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-07-18T03:39:28.097 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-07-18T03:39:28.097 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:28.097 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.097 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.098 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "0", 2023-07-18T03:39:28.098 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.098 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.098 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.098 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.099 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.099 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.099 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.099 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.099 INFO:teuthology.orchestra.run.smithi101.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-07-18T03:39:28.100 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-07-18T03:39:28.100 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:28.100 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.100 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.100 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "10", 2023-07-18T03:39:28.101 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.101 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.101 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.101 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.101 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.102 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.102 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.102 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.102 INFO:teuthology.orchestra.run.smithi101.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-07-18T03:39:28.102 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-07-18T03:39:28.103 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:28.103 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.103 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.103 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "5", 2023-07-18T03:39:28.103 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.103 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.104 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.104 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.104 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.104 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.104 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.105 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.105 INFO:teuthology.orchestra.run.smithi101.stdout: "cross_origin_url": { 2023-07-18T03:39:28.105 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "cross_origin_url", 2023-07-18T03:39:28.105 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.105 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.106 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.106 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.106 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.106 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.106 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.107 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.107 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.107 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.107 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.107 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.108 INFO:teuthology.orchestra.run.smithi101.stdout: "crt_file": { 2023-07-18T03:39:28.108 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "crt_file", 2023-07-18T03:39:28.108 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.108 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.108 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.109 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.109 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.109 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.109 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.109 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.110 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.110 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.110 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.110 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.110 INFO:teuthology.orchestra.run.smithi101.stdout: "debug": { 2023-07-18T03:39:28.110 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "debug", 2023-07-18T03:39:28.111 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.111 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.111 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.111 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.111 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.112 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.112 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.112 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Enable/disable debug options", 2023-07-18T03:39:28.112 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.113 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.113 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.113 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.113 INFO:teuthology.orchestra.run.smithi101.stdout: "jwt_token_ttl": { 2023-07-18T03:39:28.113 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "jwt_token_ttl", 2023-07-18T03:39:28.113 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:28.114 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.114 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.114 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "28800", 2023-07-18T03:39:28.114 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.114 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.115 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.115 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.115 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.115 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.115 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.116 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.116 INFO:teuthology.orchestra.run.smithi101.stdout: "key_file": { 2023-07-18T03:39:28.116 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "key_file", 2023-07-18T03:39:28.116 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.116 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.117 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.117 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.117 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.117 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.117 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.118 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.118 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.118 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.118 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.118 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.119 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:28.119 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:28.119 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.119 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.119 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.120 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.120 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.120 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.120 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.120 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.121 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.121 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.121 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.121 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.121 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.122 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.122 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.122 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.122 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.122 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.123 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.123 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:28.123 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:28.123 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.123 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.124 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.124 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.124 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.124 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.124 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.125 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.125 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.125 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.125 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.125 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.126 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:28.126 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:28.126 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.126 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.126 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.127 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:28.127 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.127 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.127 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.127 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.128 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.128 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.128 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.128 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.128 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.129 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.129 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.129 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.129 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.129 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.130 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.130 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:28.130 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:28.130 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.130 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.131 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.131 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.131 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.131 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.131 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.132 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.132 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.132 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.132 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.132 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.133 INFO:teuthology.orchestra.run.smithi101.stdout: "motd": { 2023-07-18T03:39:28.133 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "motd", 2023-07-18T03:39:28.133 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.133 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.133 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.133 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.134 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.134 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.134 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.134 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "The message of the day", 2023-07-18T03:39:28.134 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.135 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.135 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.135 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.135 INFO:teuthology.orchestra.run.smithi101.stdout: "redirect_resolve_ip_addr": { 2023-07-18T03:39:28.135 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "redirect_resolve_ip_addr", 2023-07-18T03:39:28.136 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.136 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.136 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.136 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.136 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.137 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.137 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.137 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.137 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.137 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.138 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.138 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.138 INFO:teuthology.orchestra.run.smithi101.stdout: "server_addr": { 2023-07-18T03:39:28.138 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "server_addr", 2023-07-18T03:39:28.138 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.139 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.139 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.139 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "::", 2023-07-18T03:39:28.139 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.139 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.140 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.140 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.140 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.140 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.140 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.141 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.141 INFO:teuthology.orchestra.run.smithi101.stdout: "server_port": { 2023-07-18T03:39:28.141 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "server_port", 2023-07-18T03:39:28.141 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:28.141 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.142 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.142 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "8080", 2023-07-18T03:39:28.142 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.142 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.142 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.143 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.143 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.143 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.143 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.143 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.143 INFO:teuthology.orchestra.run.smithi101.stdout: "ssl": { 2023-07-18T03:39:28.144 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "ssl", 2023-07-18T03:39:28.144 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.144 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.144 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.144 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:28.145 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.145 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.145 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.145 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.145 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.146 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.146 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.146 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.146 INFO:teuthology.orchestra.run.smithi101.stdout: "ssl_server_port": { 2023-07-18T03:39:28.146 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "ssl_server_port", 2023-07-18T03:39:28.147 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:28.147 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.147 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.147 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "8443", 2023-07-18T03:39:28.147 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.148 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.148 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.148 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.148 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.148 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.149 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.149 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.149 INFO:teuthology.orchestra.run.smithi101.stdout: "standby_behaviour": { 2023-07-18T03:39:28.149 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "standby_behaviour", 2023-07-18T03:39:28.149 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.150 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.150 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.150 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "redirect", 2023-07-18T03:39:28.150 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.150 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.151 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.151 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.151 INFO:teuthology.orchestra.run.smithi101.stdout: "redirect" 2023-07-18T03:39:28.151 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.151 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.152 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.152 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.152 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.152 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.152 INFO:teuthology.orchestra.run.smithi101.stdout: "standby_error_status_code": { 2023-07-18T03:39:28.153 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "standby_error_status_code", 2023-07-18T03:39:28.153 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:28.153 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.153 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.153 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "500", 2023-07-18T03:39:28.153 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "400", 2023-07-18T03:39:28.154 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "599", 2023-07-18T03:39:28.154 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.154 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.154 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.154 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.155 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.155 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.155 INFO:teuthology.orchestra.run.smithi101.stdout: "url_prefix": { 2023-07-18T03:39:28.155 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "url_prefix", 2023-07-18T03:39:28.155 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.156 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.156 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.156 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.156 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.156 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.157 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.157 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.157 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.157 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.157 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.158 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.158 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.158 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.158 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:28.158 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "devicehealth", 2023-07-18T03:39:28.159 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:28.159 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:28.159 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:28.159 INFO:teuthology.orchestra.run.smithi101.stdout: "enable_monitoring": { 2023-07-18T03:39:28.159 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "enable_monitoring", 2023-07-18T03:39:28.160 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.160 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.160 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.160 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:28.160 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.161 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.161 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.161 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "monitor device health metrics", 2023-07-18T03:39:28.161 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.161 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.162 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.162 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.162 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:28.162 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:28.162 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.163 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.163 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.163 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.163 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.163 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.164 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.164 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.164 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.164 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.164 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.165 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.165 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.165 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.165 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.165 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.166 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.166 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.166 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.166 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:28.166 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:28.167 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.167 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.167 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.167 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.167 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.167 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.168 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.168 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.168 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.168 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.168 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.169 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.169 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:28.169 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:28.169 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.169 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.170 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.170 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:28.170 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.170 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.170 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.171 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.171 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.171 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.171 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.171 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.172 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.172 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.172 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.172 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.172 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.173 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.173 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.173 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:28.173 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:28.173 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.174 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.174 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.174 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.174 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.174 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.175 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.175 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.175 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.175 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.175 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.176 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.176 INFO:teuthology.orchestra.run.smithi101.stdout: "mark_out_threshold": { 2023-07-18T03:39:28.176 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "mark_out_threshold", 2023-07-18T03:39:28.176 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:28.176 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.177 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.177 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "2419200", 2023-07-18T03:39:28.177 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.177 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.177 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.178 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-07-18T03:39:28.178 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.178 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.178 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.178 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.178 INFO:teuthology.orchestra.run.smithi101.stdout: "pool_name": { 2023-07-18T03:39:28.179 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "pool_name", 2023-07-18T03:39:28.179 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.179 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.179 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.179 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "device_health_metrics", 2023-07-18T03:39:28.180 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.180 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.180 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.180 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "name of pool in which to store device health metrics", 2023-07-18T03:39:28.180 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.181 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.181 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.181 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.181 INFO:teuthology.orchestra.run.smithi101.stdout: "retention_period": { 2023-07-18T03:39:28.181 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "retention_period", 2023-07-18T03:39:28.182 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:28.182 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.182 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.182 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "15552000", 2023-07-18T03:39:28.182 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.183 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.183 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.183 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "how long to retain device health metrics", 2023-07-18T03:39:28.183 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.183 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.184 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.184 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.184 INFO:teuthology.orchestra.run.smithi101.stdout: "scrape_frequency": { 2023-07-18T03:39:28.184 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "scrape_frequency", 2023-07-18T03:39:28.184 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:28.185 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.185 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.185 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "86400", 2023-07-18T03:39:28.185 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.185 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.186 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.186 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "how frequently to scrape device health metrics", 2023-07-18T03:39:28.186 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.186 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.186 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.186 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.187 INFO:teuthology.orchestra.run.smithi101.stdout: "self_heal": { 2023-07-18T03:39:28.187 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "self_heal", 2023-07-18T03:39:28.187 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.187 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.187 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.188 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:28.188 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.188 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.188 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.188 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-07-18T03:39:28.189 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.189 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.189 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.189 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.189 INFO:teuthology.orchestra.run.smithi101.stdout: "sleep_interval": { 2023-07-18T03:39:28.190 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "sleep_interval", 2023-07-18T03:39:28.190 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:28.190 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.190 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.190 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "600", 2023-07-18T03:39:28.191 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.191 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.191 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.191 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "how frequently to wake up and check device health", 2023-07-18T03:39:28.191 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.192 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.192 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.192 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.192 INFO:teuthology.orchestra.run.smithi101.stdout: "warn_threshold": { 2023-07-18T03:39:28.192 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "warn_threshold", 2023-07-18T03:39:28.193 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:28.193 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.193 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.193 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "7257600", 2023-07-18T03:39:28.193 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.194 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.194 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.194 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-07-18T03:39:28.194 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.194 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.194 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.195 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.195 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.195 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.195 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:28.195 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "diskprediction_local", 2023-07-18T03:39:28.196 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:28.196 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:28.196 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:28.196 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:28.196 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:28.197 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.197 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.197 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.197 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.197 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.198 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.198 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.198 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.198 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.198 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.199 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.199 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.199 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.199 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.199 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.200 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.200 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.200 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.200 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.200 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:28.201 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:28.201 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.201 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.201 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.201 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.201 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.202 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.202 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.202 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.202 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.202 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.203 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.203 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.203 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:28.203 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:28.203 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.204 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.204 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.204 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:28.204 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.204 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.205 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.205 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.205 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.205 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.205 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.206 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.206 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.206 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.206 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.206 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.207 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.207 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.207 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.207 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:28.207 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:28.208 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.208 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.208 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.208 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.208 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.209 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.209 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.209 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.209 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.209 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.210 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.210 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.210 INFO:teuthology.orchestra.run.smithi101.stdout: "predict_interval": { 2023-07-18T03:39:28.210 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "predict_interval", 2023-07-18T03:39:28.210 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.211 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.211 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.211 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "86400", 2023-07-18T03:39:28.211 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.211 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.212 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.212 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.212 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.212 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.212 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.213 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.213 INFO:teuthology.orchestra.run.smithi101.stdout: "predictor_model": { 2023-07-18T03:39:28.213 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "predictor_model", 2023-07-18T03:39:28.213 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.213 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.214 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.214 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "prophetstor", 2023-07-18T03:39:28.214 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.214 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.214 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.214 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.215 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.215 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.215 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.215 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.215 INFO:teuthology.orchestra.run.smithi101.stdout: "sleep_interval": { 2023-07-18T03:39:28.216 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "sleep_interval", 2023-07-18T03:39:28.216 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.216 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.216 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.216 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "600", 2023-07-18T03:39:28.217 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.217 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.217 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.217 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.217 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.218 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.218 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.218 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.218 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.218 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.219 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:28.219 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "influx", 2023-07-18T03:39:28.219 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": false, 2023-07-18T03:39:28.219 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "influxdb python module not found", 2023-07-18T03:39:28.219 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:28.220 INFO:teuthology.orchestra.run.smithi101.stdout: "batch_size": { 2023-07-18T03:39:28.220 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "batch_size", 2023-07-18T03:39:28.220 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:28.220 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.220 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.221 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "5000", 2023-07-18T03:39:28.221 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.221 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.221 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.221 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-07-18T03:39:28.222 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.222 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.222 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.222 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.222 INFO:teuthology.orchestra.run.smithi101.stdout: "database": { 2023-07-18T03:39:28.223 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "database", 2023-07-18T03:39:28.223 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.223 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.223 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.223 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "ceph", 2023-07-18T03:39:28.223 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.224 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.224 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.224 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-07-18T03:39:28.224 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.224 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.225 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.225 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.225 INFO:teuthology.orchestra.run.smithi101.stdout: "hostname": { 2023-07-18T03:39:28.225 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "hostname", 2023-07-18T03:39:28.225 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.226 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.226 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.226 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.226 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.226 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.227 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.227 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "InfluxDB server hostname", 2023-07-18T03:39:28.227 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.227 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.227 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.228 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.228 INFO:teuthology.orchestra.run.smithi101.stdout: "interval": { 2023-07-18T03:39:28.228 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "interval", 2023-07-18T03:39:28.228 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:28.228 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.229 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.229 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "30", 2023-07-18T03:39:28.229 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "5", 2023-07-18T03:39:28.229 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.229 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.230 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-07-18T03:39:28.230 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.230 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.230 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.230 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.230 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:28.231 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:28.231 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.231 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.231 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.231 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.232 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.232 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.232 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.232 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.232 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.233 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.233 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.233 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.233 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.233 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.234 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.234 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.234 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.234 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.234 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.235 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:28.235 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:28.235 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.235 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.235 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.236 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.236 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.236 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.236 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.236 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.237 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.237 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.237 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.237 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.237 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:28.238 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:28.238 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.238 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.238 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.238 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:28.238 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.239 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.239 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.239 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.239 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.239 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.240 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.240 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.240 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.240 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.240 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.241 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.241 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.241 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.241 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.241 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:28.242 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:28.242 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.242 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.242 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.242 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.243 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.243 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.243 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.243 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.243 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.244 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.244 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.244 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.244 INFO:teuthology.orchestra.run.smithi101.stdout: "password": { 2023-07-18T03:39:28.244 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "password", 2023-07-18T03:39:28.245 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.245 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.245 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.245 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.245 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.245 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.246 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.246 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "password of InfluxDB server user", 2023-07-18T03:39:28.246 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.246 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.246 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.247 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.247 INFO:teuthology.orchestra.run.smithi101.stdout: "port": { 2023-07-18T03:39:28.247 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "port", 2023-07-18T03:39:28.247 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:28.247 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.248 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.248 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "8086", 2023-07-18T03:39:28.248 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.248 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.248 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.249 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "InfluxDB server port", 2023-07-18T03:39:28.249 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.249 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.249 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.249 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.250 INFO:teuthology.orchestra.run.smithi101.stdout: "ssl": { 2023-07-18T03:39:28.250 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "ssl", 2023-07-18T03:39:28.250 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.250 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.250 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.251 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "false", 2023-07-18T03:39:28.251 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.251 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.251 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.251 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-07-18T03:39:28.252 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.252 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.252 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.252 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.252 INFO:teuthology.orchestra.run.smithi101.stdout: "threads": { 2023-07-18T03:39:28.253 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "threads", 2023-07-18T03:39:28.253 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:28.253 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.253 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.253 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "5", 2023-07-18T03:39:28.254 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "1", 2023-07-18T03:39:28.254 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "32", 2023-07-18T03:39:28.254 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.254 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-07-18T03:39:28.254 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.255 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.255 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.255 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.255 INFO:teuthology.orchestra.run.smithi101.stdout: "username": { 2023-07-18T03:39:28.255 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "username", 2023-07-18T03:39:28.256 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.256 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.256 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.256 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.256 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.257 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.257 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.257 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "username of InfluxDB server user", 2023-07-18T03:39:28.257 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.257 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.258 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.258 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.258 INFO:teuthology.orchestra.run.smithi101.stdout: "verify_ssl": { 2023-07-18T03:39:28.258 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "verify_ssl", 2023-07-18T03:39:28.258 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.258 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.259 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.259 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "true", 2023-07-18T03:39:28.259 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.259 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.259 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.260 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-07-18T03:39:28.260 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.260 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.260 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.260 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.261 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.261 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.261 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:28.261 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "insights", 2023-07-18T03:39:28.261 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:28.262 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:28.262 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:28.262 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:28.262 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:28.262 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.263 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.263 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.263 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.263 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.263 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.264 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.264 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.264 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.264 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.264 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.265 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.265 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.265 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.265 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.265 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.266 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.266 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.266 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.266 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:28.266 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:28.267 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.267 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.267 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.267 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.267 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.267 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.268 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.268 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.268 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.268 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.268 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.269 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.269 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:28.269 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:28.269 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.269 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.270 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.270 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:28.270 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.270 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.270 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.271 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.271 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.271 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.271 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.271 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.272 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.272 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.272 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.272 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.272 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.273 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.273 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.273 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:28.273 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:28.273 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.274 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.274 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.274 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.274 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.274 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.275 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.275 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.275 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.275 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.275 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.275 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.276 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.276 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.276 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:28.276 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "iostat", 2023-07-18T03:39:28.276 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:28.277 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:28.277 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:28.277 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:28.277 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:28.277 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.278 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.278 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.278 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.278 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.278 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.279 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.279 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.279 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.279 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.279 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.280 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.280 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.280 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.280 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.280 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.281 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.281 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.281 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.281 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:28.281 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:28.282 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.282 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.282 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.282 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.282 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.283 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.283 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.283 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.283 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.283 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.284 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.284 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.284 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:28.284 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:28.284 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.285 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.285 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.285 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:28.285 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.285 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.286 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.286 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.286 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.286 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.286 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.286 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.287 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.287 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.287 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.287 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.287 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.288 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.288 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.288 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:28.288 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:28.288 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.289 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.289 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.289 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.289 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.289 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.290 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.290 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.290 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.290 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.290 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.291 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.291 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.291 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.291 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:28.291 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "k8sevents", 2023-07-18T03:39:28.292 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:28.292 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:28.292 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:28.292 INFO:teuthology.orchestra.run.smithi101.stdout: "ceph_event_retention_days": { 2023-07-18T03:39:28.292 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "ceph_event_retention_days", 2023-07-18T03:39:28.293 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:28.293 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.293 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.293 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "7", 2023-07-18T03:39:28.293 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.294 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.294 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.294 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Days to hold ceph event information within local cache", 2023-07-18T03:39:28.294 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.294 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.295 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.295 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.295 INFO:teuthology.orchestra.run.smithi101.stdout: "config_check_secs": { 2023-07-18T03:39:28.295 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "config_check_secs", 2023-07-18T03:39:28.295 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:28.296 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.296 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.296 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "10", 2023-07-18T03:39:28.296 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "10", 2023-07-18T03:39:28.296 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.297 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.297 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-07-18T03:39:28.297 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.297 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.297 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.298 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.298 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:28.298 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:28.298 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.298 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.299 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.299 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.299 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.299 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.299 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.300 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.300 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.300 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.300 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.300 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.301 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.301 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.301 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.301 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.301 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.301 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.302 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.302 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:28.302 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:28.302 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.302 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.303 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.303 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.303 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.303 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.303 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.304 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.304 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.304 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.304 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.304 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.305 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:28.305 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:28.305 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.305 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.305 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.306 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:28.306 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.306 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.306 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.306 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.307 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.307 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.307 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.307 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.307 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.308 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.308 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.308 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.308 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.308 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.309 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.309 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:28.309 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:28.309 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.309 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.310 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.310 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.310 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.310 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.310 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.311 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.311 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.311 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.311 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.311 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.312 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.312 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.312 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:28.312 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "localpool", 2023-07-18T03:39:28.312 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:28.313 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:28.313 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:28.313 INFO:teuthology.orchestra.run.smithi101.stdout: "failure_domain": { 2023-07-18T03:39:28.313 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "failure_domain", 2023-07-18T03:39:28.313 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.313 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.314 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.314 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "host", 2023-07-18T03:39:28.314 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.314 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.314 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.315 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "failure domain for any created local pool", 2023-07-18T03:39:28.315 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-07-18T03:39:28.315 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.315 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.315 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.316 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:28.316 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:28.316 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.316 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.316 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.317 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.317 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.317 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.317 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.317 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.318 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.318 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.318 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.318 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.318 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.319 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.319 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.319 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.319 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.319 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.320 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.320 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:28.320 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:28.320 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.320 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.321 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.321 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.321 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.321 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.321 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.321 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.322 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.322 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.322 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.322 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.322 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:28.323 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:28.323 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.323 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.323 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.323 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:28.324 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.324 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.324 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.324 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.324 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.325 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.325 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.325 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.325 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.325 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.326 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.326 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.326 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.326 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.326 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.327 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:28.327 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:28.327 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.327 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.327 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.328 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.328 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.328 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.328 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.328 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.329 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.329 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.329 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.329 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.329 INFO:teuthology.orchestra.run.smithi101.stdout: "min_size": { 2023-07-18T03:39:28.330 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "min_size", 2023-07-18T03:39:28.330 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:28.330 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.330 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.330 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.331 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.331 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.331 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.331 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "default min_size for any created local pool", 2023-07-18T03:39:28.331 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-07-18T03:39:28.332 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.332 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.332 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.332 INFO:teuthology.orchestra.run.smithi101.stdout: "num_rep": { 2023-07-18T03:39:28.332 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "num_rep", 2023-07-18T03:39:28.333 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:28.333 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.333 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.333 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "3", 2023-07-18T03:39:28.333 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.333 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.334 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.334 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "default replica count for any created local pool", 2023-07-18T03:39:28.334 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.334 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.334 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.335 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.335 INFO:teuthology.orchestra.run.smithi101.stdout: "pg_num": { 2023-07-18T03:39:28.335 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "pg_num", 2023-07-18T03:39:28.335 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:28.335 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.336 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.336 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "128", 2023-07-18T03:39:28.336 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.336 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.336 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.337 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "default pg_num for any created local pool", 2023-07-18T03:39:28.337 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.337 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.337 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.337 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.338 INFO:teuthology.orchestra.run.smithi101.stdout: "prefix": { 2023-07-18T03:39:28.338 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "prefix", 2023-07-18T03:39:28.338 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.338 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.338 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.339 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.339 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.339 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.339 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.339 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "name prefix for any created local pool", 2023-07-18T03:39:28.340 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.340 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.340 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.340 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.340 INFO:teuthology.orchestra.run.smithi101.stdout: "subtree": { 2023-07-18T03:39:28.341 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "subtree", 2023-07-18T03:39:28.341 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.341 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.341 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.341 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "rack", 2023-07-18T03:39:28.342 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.342 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.342 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.342 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "CRUSH level for which to create a local pool", 2023-07-18T03:39:28.342 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-07-18T03:39:28.343 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.343 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.343 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.343 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.343 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.344 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:28.344 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "mds_autoscaler", 2023-07-18T03:39:28.344 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:28.344 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:28.344 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:28.345 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:28.345 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:28.345 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.345 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.345 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.346 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.346 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.346 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.346 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.346 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.346 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.347 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.347 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.347 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.347 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.347 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.348 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.348 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.348 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.348 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.348 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.349 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:28.349 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:28.349 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.349 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.349 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.350 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.350 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.350 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.350 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.350 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.351 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.351 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.351 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.351 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.351 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:28.352 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:28.352 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.352 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.352 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.352 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:28.353 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.353 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.353 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.353 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.353 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.354 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.354 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.354 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.354 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.354 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.354 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.355 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.355 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.355 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.355 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.355 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:28.356 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:28.356 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.356 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.356 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.356 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.357 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.357 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.357 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.357 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.357 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.358 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.358 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.358 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.358 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.358 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.359 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:28.359 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "mirroring", 2023-07-18T03:39:28.359 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:28.359 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:28.359 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:28.360 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:28.360 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:28.360 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.360 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.360 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.361 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.361 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.361 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.361 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.361 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.362 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.362 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.362 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.362 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.362 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.362 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.363 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.363 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.363 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.363 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.363 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.364 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:28.364 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:28.364 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.364 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.364 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.365 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.365 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.365 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.365 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.365 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.366 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.366 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.366 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.366 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.366 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:28.367 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:28.367 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.367 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.367 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.367 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:28.368 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.368 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.368 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.368 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.368 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.369 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.369 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.369 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.369 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.369 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.370 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.370 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.370 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.370 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.370 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.371 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:28.371 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:28.371 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.371 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.371 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.371 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.372 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.372 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.372 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.372 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.372 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.373 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.373 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.373 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.373 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.373 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.374 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:28.374 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "nfs", 2023-07-18T03:39:28.374 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:28.374 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:28.374 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:28.375 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:28.375 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:28.375 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.375 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.375 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.376 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.376 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.376 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.376 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.376 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.377 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.377 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.377 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.377 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.377 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.378 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.378 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.378 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.378 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.378 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.379 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.379 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:28.379 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:28.379 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.379 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.379 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.380 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.380 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.380 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.380 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.380 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.381 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.381 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.381 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.381 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.381 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:28.382 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:28.382 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.382 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.382 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.382 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:28.383 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.383 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.383 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.383 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.383 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.384 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.384 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.384 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.384 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.384 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.385 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.385 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.385 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.385 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.385 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.386 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:28.386 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:28.386 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.386 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.386 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.387 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.387 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.387 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.387 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.387 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.388 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.388 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.388 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.388 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.388 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.389 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.389 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:28.389 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "orchestrator", 2023-07-18T03:39:28.389 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:28.389 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:28.390 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:28.390 INFO:teuthology.orchestra.run.smithi101.stdout: "fail_fs": { 2023-07-18T03:39:28.390 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "fail_fs", 2023-07-18T03:39:28.390 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.390 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.391 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.391 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.391 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.391 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.391 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.391 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-07-18T03:39:28.392 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.392 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.392 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.392 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.392 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:28.393 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:28.393 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.393 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.393 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.393 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.394 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.394 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.394 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.394 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.394 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.395 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.395 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.395 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.395 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.395 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.396 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.396 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.396 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.396 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.396 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.397 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:28.397 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:28.397 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.397 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.397 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.398 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.398 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.398 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.398 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.398 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.399 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.399 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.399 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.399 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.399 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:28.400 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:28.400 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.400 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.400 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.400 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:28.400 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.401 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.401 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.401 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.401 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.401 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.402 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.402 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.402 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.402 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.402 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.403 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.403 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.403 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.403 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.403 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:28.404 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:28.404 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.404 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.404 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.404 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.405 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.405 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.405 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.405 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.405 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.406 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.406 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.406 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.406 INFO:teuthology.orchestra.run.smithi101.stdout: "orchestrator": { 2023-07-18T03:39:28.406 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "orchestrator", 2023-07-18T03:39:28.407 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.407 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.407 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.407 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.407 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.408 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.408 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.408 INFO:teuthology.orchestra.run.smithi101.stdout: "cephadm", 2023-07-18T03:39:28.408 INFO:teuthology.orchestra.run.smithi101.stdout: "rook", 2023-07-18T03:39:28.408 INFO:teuthology.orchestra.run.smithi101.stdout: "test_orchestrator" 2023-07-18T03:39:28.409 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.409 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Orchestrator backend", 2023-07-18T03:39:28.409 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.409 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.409 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.409 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.410 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.410 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.410 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:28.410 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "osd_perf_query", 2023-07-18T03:39:28.410 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:28.411 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:28.411 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:28.411 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:28.411 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:28.411 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.412 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.412 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.412 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.412 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.412 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.413 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.413 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.413 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.413 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.413 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.414 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.414 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.414 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.414 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.414 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.415 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.415 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.415 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.415 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:28.415 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:28.416 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.416 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.416 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.416 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.416 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.417 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.417 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.417 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.417 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.417 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.418 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.418 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.418 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:28.418 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:28.418 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.419 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.419 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.419 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:28.419 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.419 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.420 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.420 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.420 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.420 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.420 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.421 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.421 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.421 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.421 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.421 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.421 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.422 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.422 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.422 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:28.422 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:28.422 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.423 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.423 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.423 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.423 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.423 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.424 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.424 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.424 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.424 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.424 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.425 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.425 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.425 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.425 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:28.425 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "osd_support", 2023-07-18T03:39:28.426 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:28.426 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:28.426 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:28.426 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:28.426 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:28.427 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.427 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.427 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.427 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.427 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.428 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.428 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.428 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.428 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.428 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.429 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.429 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.429 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.429 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.429 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.429 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.430 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.430 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.430 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.430 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:28.430 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:28.431 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.431 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.431 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.431 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.431 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.432 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.432 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.432 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.432 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.432 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.433 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.433 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.433 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:28.433 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:28.433 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.434 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.434 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.434 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:28.434 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.434 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.435 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.435 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.435 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.435 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.435 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.436 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.436 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.436 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.436 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.436 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.437 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.437 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.437 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.437 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:28.437 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:28.438 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.438 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.438 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.438 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.438 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.439 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.439 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.439 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.439 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.439 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.440 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.440 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.440 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.440 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.440 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:28.441 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "pg_autoscaler", 2023-07-18T03:39:28.441 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:28.441 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:28.441 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:28.441 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:28.442 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:28.442 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.442 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.442 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.442 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.443 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.443 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.443 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.443 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.443 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.444 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.444 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.444 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.444 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.444 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.445 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.445 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.445 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.445 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.445 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.446 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:28.446 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:28.446 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.446 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.446 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.446 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.447 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.447 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.447 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.447 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.447 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.448 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.448 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.448 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.448 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:28.448 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:28.449 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.449 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.449 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.449 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:28.449 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.450 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.450 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.450 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.450 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.450 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.451 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.451 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.451 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.451 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.451 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.452 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.452 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.452 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.452 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.452 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:28.453 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:28.453 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.453 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.453 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.453 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.454 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.454 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.454 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.454 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.455 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.455 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.455 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.455 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.456 INFO:teuthology.orchestra.run.smithi101.stdout: "noautoscale": { 2023-07-18T03:39:28.456 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "noautoscale", 2023-07-18T03:39:28.456 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.456 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.456 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.457 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.457 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.457 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.457 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.458 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "global autoscale flag", 2023-07-18T03:39:28.458 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-07-18T03:39:28.458 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.458 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.459 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.459 INFO:teuthology.orchestra.run.smithi101.stdout: "sleep_interval": { 2023-07-18T03:39:28.459 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "sleep_interval", 2023-07-18T03:39:28.459 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:28.460 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.460 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.460 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "60", 2023-07-18T03:39:28.460 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.460 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.461 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.461 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.461 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.461 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.461 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.461 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.462 INFO:teuthology.orchestra.run.smithi101.stdout: "threshold": { 2023-07-18T03:39:28.462 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "threshold", 2023-07-18T03:39:28.462 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "float", 2023-07-18T03:39:28.462 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.462 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.463 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "3.0", 2023-07-18T03:39:28.463 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "1.0", 2023-07-18T03:39:28.463 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.463 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.463 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "scaling threshold", 2023-07-18T03:39:28.464 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-07-18T03:39:28.464 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.464 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.464 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.464 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.465 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.465 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:28.465 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "progress", 2023-07-18T03:39:28.465 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:28.465 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:28.466 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:28.466 INFO:teuthology.orchestra.run.smithi101.stdout: "allow_pg_recovery_event": { 2023-07-18T03:39:28.466 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "allow_pg_recovery_event", 2023-07-18T03:39:28.466 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.466 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.467 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.467 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.467 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.467 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.467 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.468 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "allow the module to show pg recovery progress", 2023-07-18T03:39:28.468 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.468 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.468 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.468 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.469 INFO:teuthology.orchestra.run.smithi101.stdout: "enabled": { 2023-07-18T03:39:28.469 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "enabled", 2023-07-18T03:39:28.469 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.469 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.469 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.470 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:28.470 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.470 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.470 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.470 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.471 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.471 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.471 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.471 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.471 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:28.471 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:28.472 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.472 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.472 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.472 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.472 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.473 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.473 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.473 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.473 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.473 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.474 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.474 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.474 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.474 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.474 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.475 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.475 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.475 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.475 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.475 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:28.476 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:28.476 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.476 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.476 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.476 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.477 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.477 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.477 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.477 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.477 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.478 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.478 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.478 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.478 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:28.478 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:28.479 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.479 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.479 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.479 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:28.479 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.479 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.480 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.480 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.480 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.480 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.480 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.481 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.481 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.481 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.481 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.481 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.482 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.482 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.482 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.482 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:28.482 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:28.483 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.483 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.483 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.483 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.483 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.484 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.484 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.484 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.484 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.484 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.485 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.485 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.485 INFO:teuthology.orchestra.run.smithi101.stdout: "max_completed_events": { 2023-07-18T03:39:28.485 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "max_completed_events", 2023-07-18T03:39:28.485 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:28.486 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.486 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.486 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "50", 2023-07-18T03:39:28.486 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.486 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.487 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.487 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "number of past completed events to remember", 2023-07-18T03:39:28.487 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.487 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.487 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.488 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.488 INFO:teuthology.orchestra.run.smithi101.stdout: "sleep_interval": { 2023-07-18T03:39:28.488 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "sleep_interval", 2023-07-18T03:39:28.488 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:28.488 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.489 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.489 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "5", 2023-07-18T03:39:28.489 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.489 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.489 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.490 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "how long the module is going to sleep", 2023-07-18T03:39:28.490 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.490 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.490 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.490 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.491 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.491 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.491 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:28.491 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "prometheus", 2023-07-18T03:39:28.491 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:28.492 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:28.492 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:28.492 INFO:teuthology.orchestra.run.smithi101.stdout: "cache": { 2023-07-18T03:39:28.492 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "cache", 2023-07-18T03:39:28.492 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.493 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.493 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.493 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:28.493 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.493 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.493 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.494 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.494 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.494 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.494 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.494 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.495 INFO:teuthology.orchestra.run.smithi101.stdout: "exclude_perf_counters": { 2023-07-18T03:39:28.495 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "exclude_perf_counters", 2023-07-18T03:39:28.495 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.495 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.495 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.496 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:28.496 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.496 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.496 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.496 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-07-18T03:39:28.497 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-07-18T03:39:28.497 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.497 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.497 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.497 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:28.498 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:28.498 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.498 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.498 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.498 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.499 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.499 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.499 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.499 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.499 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.500 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.500 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.500 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.500 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.500 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.501 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.501 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.501 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.501 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.501 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.502 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:28.502 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:28.502 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.502 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.502 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.503 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.503 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.503 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.503 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.503 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.504 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.504 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.504 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.504 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.504 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:28.505 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:28.505 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.505 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.505 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.505 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:28.506 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.506 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.506 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.506 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.506 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.507 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.507 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.507 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.507 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.507 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.508 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.508 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.508 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.508 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.508 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.509 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:28.509 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:28.509 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.509 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.509 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.509 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.510 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.510 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.510 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.510 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.510 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.511 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.511 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.511 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.511 INFO:teuthology.orchestra.run.smithi101.stdout: "rbd_stats_pools": { 2023-07-18T03:39:28.511 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "rbd_stats_pools", 2023-07-18T03:39:28.512 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.512 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.512 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.512 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.512 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.513 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.513 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.513 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.513 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.513 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.514 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.514 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.514 INFO:teuthology.orchestra.run.smithi101.stdout: "rbd_stats_pools_refresh_interval": { 2023-07-18T03:39:28.514 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-07-18T03:39:28.514 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:28.515 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.515 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.515 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "300", 2023-07-18T03:39:28.515 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.515 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.516 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.516 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.516 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.516 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.516 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.517 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.517 INFO:teuthology.orchestra.run.smithi101.stdout: "scrape_interval": { 2023-07-18T03:39:28.517 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "scrape_interval", 2023-07-18T03:39:28.517 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "float", 2023-07-18T03:39:28.517 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.518 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.518 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "15.0", 2023-07-18T03:39:28.518 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.518 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.518 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.519 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.519 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.519 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.519 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.519 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.520 INFO:teuthology.orchestra.run.smithi101.stdout: "server_addr": { 2023-07-18T03:39:28.520 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "server_addr", 2023-07-18T03:39:28.520 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.520 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.520 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.521 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "::", 2023-07-18T03:39:28.521 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.521 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.521 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.521 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-07-18T03:39:28.521 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.522 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.522 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.522 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.522 INFO:teuthology.orchestra.run.smithi101.stdout: "server_port": { 2023-07-18T03:39:28.522 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "server_port", 2023-07-18T03:39:28.523 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:28.523 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.523 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.523 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "9283", 2023-07-18T03:39:28.523 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.524 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.524 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.524 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-07-18T03:39:28.524 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.524 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.525 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.525 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.525 INFO:teuthology.orchestra.run.smithi101.stdout: "stale_cache_strategy": { 2023-07-18T03:39:28.525 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "stale_cache_strategy", 2023-07-18T03:39:28.525 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.526 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.526 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.526 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "log", 2023-07-18T03:39:28.526 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.526 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.527 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.527 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.527 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.527 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.527 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.528 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.528 INFO:teuthology.orchestra.run.smithi101.stdout: "standby_behaviour": { 2023-07-18T03:39:28.528 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "standby_behaviour", 2023-07-18T03:39:28.528 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.528 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.529 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.529 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "default", 2023-07-18T03:39:28.529 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.529 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.529 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.530 INFO:teuthology.orchestra.run.smithi101.stdout: "default", 2023-07-18T03:39:28.530 INFO:teuthology.orchestra.run.smithi101.stdout: "error" 2023-07-18T03:39:28.530 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.530 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.530 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.531 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.531 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.531 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.531 INFO:teuthology.orchestra.run.smithi101.stdout: "standby_error_status_code": { 2023-07-18T03:39:28.531 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "standby_error_status_code", 2023-07-18T03:39:28.532 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:28.532 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.532 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.532 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "500", 2023-07-18T03:39:28.532 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "400", 2023-07-18T03:39:28.533 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "599", 2023-07-18T03:39:28.533 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.533 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.533 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.533 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.534 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.534 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.534 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.534 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.534 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:28.535 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "rbd_support", 2023-07-18T03:39:28.535 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:28.535 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:28.535 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:28.535 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:28.535 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:28.536 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.536 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.536 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.536 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.536 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.537 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.537 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.537 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.537 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.537 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.538 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.538 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.538 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.538 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.538 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.539 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.539 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.539 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.539 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.539 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:28.540 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:28.540 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.540 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.541 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.541 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.541 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.541 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.541 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.542 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.542 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.542 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.542 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.542 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.543 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:28.543 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:28.543 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.543 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.543 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.544 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:28.544 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.544 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.544 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.545 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.545 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.545 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.545 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.546 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.546 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.546 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.546 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.546 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.547 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.547 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.547 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.547 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:28.547 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:28.548 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.548 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.548 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.548 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.548 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.549 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.549 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.549 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.549 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.549 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.550 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.550 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.550 INFO:teuthology.orchestra.run.smithi101.stdout: "max_concurrent_snap_create": { 2023-07-18T03:39:28.550 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "max_concurrent_snap_create", 2023-07-18T03:39:28.550 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:28.551 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.551 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.551 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "10", 2023-07-18T03:39:28.551 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.552 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.552 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.552 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.552 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.552 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.553 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.553 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.553 INFO:teuthology.orchestra.run.smithi101.stdout: "mirror_snapshot_schedule": { 2023-07-18T03:39:28.553 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "mirror_snapshot_schedule", 2023-07-18T03:39:28.553 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.554 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.554 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.554 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.554 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.554 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.555 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.555 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.555 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.555 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.555 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.556 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.556 INFO:teuthology.orchestra.run.smithi101.stdout: "trash_purge_schedule": { 2023-07-18T03:39:28.556 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "trash_purge_schedule", 2023-07-18T03:39:28.556 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.556 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.557 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.557 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.557 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.557 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.557 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.558 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.558 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.558 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.558 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.558 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.559 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.559 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.559 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:28.559 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "restful", 2023-07-18T03:39:28.559 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:28.560 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:28.560 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:28.560 INFO:teuthology.orchestra.run.smithi101.stdout: "enable_auth": { 2023-07-18T03:39:28.560 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "enable_auth", 2023-07-18T03:39:28.560 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.561 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.561 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.561 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:28.561 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.561 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.562 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.562 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.562 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.562 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.562 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.563 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.563 INFO:teuthology.orchestra.run.smithi101.stdout: "key_file": { 2023-07-18T03:39:28.563 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "key_file", 2023-07-18T03:39:28.563 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.563 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.564 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.564 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.564 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.564 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.564 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.565 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.565 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.565 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.565 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.566 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.566 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:28.566 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:28.566 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.566 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.567 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.567 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.567 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.567 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.567 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.568 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.568 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.568 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.568 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.568 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.569 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.569 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.569 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.569 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.569 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.570 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.570 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.570 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:28.570 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:28.570 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.570 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.571 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.571 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.571 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.571 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.571 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.572 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.572 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.572 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.572 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.572 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.573 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:28.573 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:28.573 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.573 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.573 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.574 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:28.574 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.574 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.574 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.574 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.575 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.575 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.575 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.575 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.576 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.576 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.576 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.576 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.576 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.577 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.577 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.577 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:28.577 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:28.577 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.578 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.578 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.578 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.578 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.578 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.579 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.579 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.579 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.579 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.579 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.580 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.580 INFO:teuthology.orchestra.run.smithi101.stdout: "server_addr": { 2023-07-18T03:39:28.580 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "server_addr", 2023-07-18T03:39:28.580 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.580 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.581 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.581 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.581 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.581 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.581 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.582 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.582 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.582 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.582 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.582 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.583 INFO:teuthology.orchestra.run.smithi101.stdout: "server_port": { 2023-07-18T03:39:28.583 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "server_port", 2023-07-18T03:39:28.583 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.583 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.583 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.584 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.584 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.584 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.584 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.584 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.585 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.585 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.585 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.585 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.585 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.586 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.586 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:28.586 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "rgw", 2023-07-18T03:39:28.586 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:28.586 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:28.587 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:28.587 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:28.587 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:28.587 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.587 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.588 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.588 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.588 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.588 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.588 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.589 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.589 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.589 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.589 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.589 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.590 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.590 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.590 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.590 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.590 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.591 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.591 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.591 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:28.591 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:28.591 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.592 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.592 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.592 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.592 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.592 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.593 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.593 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.593 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.593 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.593 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.594 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.594 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:28.594 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:28.594 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.594 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.595 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.595 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:28.595 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.595 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.595 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.596 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.596 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.596 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.596 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.596 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.597 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.597 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.597 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.597 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.598 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.598 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.598 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.598 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:28.598 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:28.599 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.599 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.599 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.599 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.599 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.599 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.600 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.600 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.600 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.600 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.600 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.601 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.601 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.601 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.601 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:28.601 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "rook", 2023-07-18T03:39:28.602 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:28.602 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:28.602 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:28.602 INFO:teuthology.orchestra.run.smithi101.stdout: "drive_group_interval": { 2023-07-18T03:39:28.602 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "drive_group_interval", 2023-07-18T03:39:28.603 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "float", 2023-07-18T03:39:28.603 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.603 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.603 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "300.0", 2023-07-18T03:39:28.603 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.604 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.604 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.604 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-07-18T03:39:28.604 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.604 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.605 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.605 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.605 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:28.605 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:28.605 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.606 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.606 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.606 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.606 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.606 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.607 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.607 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.607 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.607 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.607 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.608 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.608 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.608 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.608 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.608 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.609 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.609 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.609 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.609 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:28.609 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:28.610 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.610 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.610 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.610 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.610 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.611 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.611 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.611 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.611 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.611 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.612 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.612 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.612 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:28.612 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:28.612 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.613 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.613 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.613 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:28.613 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.613 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.614 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.614 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.614 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.614 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.614 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.615 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.615 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.615 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.615 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.615 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.616 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.616 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.616 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.616 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:28.616 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:28.617 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.617 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.617 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.617 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.617 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.618 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.618 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.618 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.618 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.618 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.619 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.619 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.619 INFO:teuthology.orchestra.run.smithi101.stdout: "storage_class": { 2023-07-18T03:39:28.619 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "storage_class", 2023-07-18T03:39:28.619 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.620 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.620 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.620 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "local", 2023-07-18T03:39:28.620 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.620 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.621 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.621 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-07-18T03:39:28.621 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.621 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.621 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.621 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.622 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.622 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.622 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:28.622 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "selftest", 2023-07-18T03:39:28.622 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:28.623 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:28.623 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:28.623 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:28.623 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:28.624 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.624 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.624 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.624 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.624 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.625 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.625 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.625 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.625 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.626 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.626 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.626 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.626 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.626 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.627 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.627 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.627 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.627 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.627 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.627 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:28.628 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:28.628 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.628 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.628 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.629 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.629 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.629 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.629 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.629 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.629 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.630 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.630 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.630 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.630 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:28.630 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:28.631 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.631 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.631 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.631 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:28.631 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.632 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.632 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.632 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.632 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.632 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.633 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.633 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.633 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.633 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.633 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.634 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.634 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.634 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.634 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.634 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:28.635 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:28.635 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.635 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.635 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.635 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.636 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.636 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.636 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.636 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.636 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.637 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.637 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.637 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.637 INFO:teuthology.orchestra.run.smithi101.stdout: "roption1": { 2023-07-18T03:39:28.637 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "roption1", 2023-07-18T03:39:28.638 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.638 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.638 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.638 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.638 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.639 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.639 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.639 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.639 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.639 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.640 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.640 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.640 INFO:teuthology.orchestra.run.smithi101.stdout: "roption2": { 2023-07-18T03:39:28.640 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "roption2", 2023-07-18T03:39:28.640 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.641 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.641 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.641 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "xyz", 2023-07-18T03:39:28.641 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.641 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.642 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.642 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.642 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.642 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.642 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.642 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.643 INFO:teuthology.orchestra.run.smithi101.stdout: "rwoption1": { 2023-07-18T03:39:28.643 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "rwoption1", 2023-07-18T03:39:28.643 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.643 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.643 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.644 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.644 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.644 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.644 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.645 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.645 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.645 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.645 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.645 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.645 INFO:teuthology.orchestra.run.smithi101.stdout: "rwoption2": { 2023-07-18T03:39:28.646 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "rwoption2", 2023-07-18T03:39:28.646 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:28.646 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.646 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.646 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.647 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.647 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.647 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.647 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.647 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.648 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.648 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.648 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.648 INFO:teuthology.orchestra.run.smithi101.stdout: "rwoption3": { 2023-07-18T03:39:28.648 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "rwoption3", 2023-07-18T03:39:28.649 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "float", 2023-07-18T03:39:28.649 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.649 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.649 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.649 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.650 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.650 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.650 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.650 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.650 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.651 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.651 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.651 INFO:teuthology.orchestra.run.smithi101.stdout: "rwoption4": { 2023-07-18T03:39:28.651 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "rwoption4", 2023-07-18T03:39:28.651 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.652 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.652 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.652 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.652 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.652 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.653 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.653 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.653 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.653 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.653 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.654 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.654 INFO:teuthology.orchestra.run.smithi101.stdout: "rwoption5": { 2023-07-18T03:39:28.654 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "rwoption5", 2023-07-18T03:39:28.654 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.654 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.655 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.655 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.655 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.655 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.656 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.656 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.656 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.656 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.656 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.657 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.657 INFO:teuthology.orchestra.run.smithi101.stdout: "rwoption6": { 2023-07-18T03:39:28.657 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "rwoption6", 2023-07-18T03:39:28.657 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.657 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.658 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.658 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:28.658 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.658 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.658 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.659 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.659 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.659 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.659 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.659 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.660 INFO:teuthology.orchestra.run.smithi101.stdout: "rwoption7": { 2023-07-18T03:39:28.660 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "rwoption7", 2023-07-18T03:39:28.660 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:28.660 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.661 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.661 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.661 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "1", 2023-07-18T03:39:28.661 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "42", 2023-07-18T03:39:28.661 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.662 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.662 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.662 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.662 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.662 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.663 INFO:teuthology.orchestra.run.smithi101.stdout: "testkey": { 2023-07-18T03:39:28.663 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "testkey", 2023-07-18T03:39:28.663 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.663 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.663 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.663 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.664 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.664 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.664 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.664 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.665 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.665 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.665 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.665 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.665 INFO:teuthology.orchestra.run.smithi101.stdout: "testlkey": { 2023-07-18T03:39:28.665 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "testlkey", 2023-07-18T03:39:28.666 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.666 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.666 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.666 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.666 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.667 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.667 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.667 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.667 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.667 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.668 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.668 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.668 INFO:teuthology.orchestra.run.smithi101.stdout: "testnewline": { 2023-07-18T03:39:28.668 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "testnewline", 2023-07-18T03:39:28.668 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.669 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.669 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.669 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.669 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.669 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.670 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.670 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.670 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.670 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.670 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.671 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.671 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.671 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.671 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:28.671 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "snap_schedule", 2023-07-18T03:39:28.672 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:28.672 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:28.672 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:28.672 INFO:teuthology.orchestra.run.smithi101.stdout: "allow_m_granularity": { 2023-07-18T03:39:28.672 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "allow_m_granularity", 2023-07-18T03:39:28.673 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.673 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.673 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.673 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.673 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.674 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.674 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.674 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "allow minute scheduled snapshots", 2023-07-18T03:39:28.674 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.674 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.675 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.675 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.675 INFO:teuthology.orchestra.run.smithi101.stdout: "dump_on_update": { 2023-07-18T03:39:28.675 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "dump_on_update", 2023-07-18T03:39:28.675 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.676 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.676 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.676 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.676 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.676 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.677 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.677 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "dump database to debug log on update", 2023-07-18T03:39:28.677 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.677 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.677 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.678 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.678 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:28.678 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:28.678 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.678 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.679 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.679 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.679 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.679 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.679 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.680 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.680 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.680 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.680 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.680 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.681 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.681 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.681 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.681 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.681 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.682 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.682 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.682 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:28.682 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:28.682 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.683 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.683 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.683 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.683 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.683 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.684 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.684 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.684 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.684 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.684 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.685 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.685 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:28.685 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:28.685 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.686 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.686 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.686 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:28.686 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.686 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.687 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.687 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.687 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.687 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.687 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.688 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.688 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.688 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.688 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.688 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.689 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.689 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.689 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.689 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:28.689 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:28.689 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.690 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.690 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.690 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.690 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.690 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.691 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.691 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.691 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.691 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.691 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.692 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.692 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.692 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.692 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:28.692 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "stats", 2023-07-18T03:39:28.693 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:28.693 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:28.693 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:28.693 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:28.693 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:28.694 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.694 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.694 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.694 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.694 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.695 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.695 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.695 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.695 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.695 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.696 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.696 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.696 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.696 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.696 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.697 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.697 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.697 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.697 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.697 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:28.698 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:28.698 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.698 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.698 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.698 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.699 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.699 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.699 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.699 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.699 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.700 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.700 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.700 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.700 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:28.700 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:28.701 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.701 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.701 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.701 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:28.701 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.702 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.702 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.702 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.702 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.702 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.703 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.703 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.703 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.703 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.703 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.704 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.704 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.704 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.704 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.704 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:28.705 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:28.705 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.705 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.705 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.705 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.706 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.706 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.706 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.706 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.706 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.707 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.707 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.707 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.707 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.707 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.708 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:28.708 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "status", 2023-07-18T03:39:28.708 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:28.708 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:28.708 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:28.709 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:28.709 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:28.709 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.709 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.709 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.710 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.710 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.710 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.710 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.710 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.711 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.711 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.711 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.711 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.711 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.712 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.712 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.712 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.712 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.712 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.713 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.713 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:28.713 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:28.713 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.713 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.714 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.714 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.714 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.714 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.714 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.715 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.715 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.715 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.715 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.715 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.716 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:28.716 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:28.716 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.716 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.717 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.717 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:28.717 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.717 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.717 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.717 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.718 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.718 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.718 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.718 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.718 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.719 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.719 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.719 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.719 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.719 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.720 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.720 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:28.720 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:28.720 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.720 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.721 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.721 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.721 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.721 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.721 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.722 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.722 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.722 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.722 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.722 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.723 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.723 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.723 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:28.723 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "telegraf", 2023-07-18T03:39:28.723 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:28.724 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:28.724 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:28.724 INFO:teuthology.orchestra.run.smithi101.stdout: "address": { 2023-07-18T03:39:28.724 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "address", 2023-07-18T03:39:28.724 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.725 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.725 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.725 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-07-18T03:39:28.725 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.725 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.726 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.726 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.726 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.726 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.726 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.727 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.727 INFO:teuthology.orchestra.run.smithi101.stdout: "interval": { 2023-07-18T03:39:28.727 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "interval", 2023-07-18T03:39:28.727 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:28.727 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.728 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.728 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "15", 2023-07-18T03:39:28.728 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.728 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.728 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.729 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.729 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.729 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.729 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.729 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.730 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:28.730 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:28.730 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.730 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.730 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.731 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.731 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.731 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.731 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.731 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.732 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.732 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.732 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.732 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.732 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.733 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.733 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.733 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.733 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.733 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.734 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.734 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:28.734 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:28.734 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.734 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.734 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.735 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.735 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.735 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.735 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.735 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.736 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.736 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.736 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.736 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.736 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:28.737 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:28.737 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.737 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.737 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.737 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:28.738 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.738 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.738 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.738 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.738 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.739 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.739 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.739 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.739 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.739 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.740 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.740 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.740 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.740 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.740 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.741 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:28.741 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:28.741 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.741 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.741 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.742 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.742 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.742 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.742 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.742 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.743 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.743 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.743 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.743 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.743 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.744 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.744 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:28.744 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "telemetry", 2023-07-18T03:39:28.744 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:28.744 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:28.745 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:28.745 INFO:teuthology.orchestra.run.smithi101.stdout: "channel_basic": { 2023-07-18T03:39:28.745 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "channel_basic", 2023-07-18T03:39:28.745 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.746 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.746 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.746 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:28.746 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.746 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.747 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.747 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Share basic cluster information (size, version)", 2023-07-18T03:39:28.747 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.747 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.747 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.747 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.748 INFO:teuthology.orchestra.run.smithi101.stdout: "channel_crash": { 2023-07-18T03:39:28.748 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "channel_crash", 2023-07-18T03:39:28.748 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.748 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.748 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.749 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:28.749 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.749 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.749 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.749 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-07-18T03:39:28.750 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.750 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.750 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.750 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.750 INFO:teuthology.orchestra.run.smithi101.stdout: "channel_device": { 2023-07-18T03:39:28.751 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "channel_device", 2023-07-18T03:39:28.751 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.751 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.751 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.751 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "True", 2023-07-18T03:39:28.752 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.752 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.752 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.752 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-07-18T03:39:28.752 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.753 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.753 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.753 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.753 INFO:teuthology.orchestra.run.smithi101.stdout: "channel_ident": { 2023-07-18T03:39:28.753 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "channel_ident", 2023-07-18T03:39:28.754 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.754 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.754 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.754 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.754 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.755 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.755 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.755 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-07-18T03:39:28.755 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.755 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.756 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.756 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.756 INFO:teuthology.orchestra.run.smithi101.stdout: "channel_perf": { 2023-07-18T03:39:28.756 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "channel_perf", 2023-07-18T03:39:28.756 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.757 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.757 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.757 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.757 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.757 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.758 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.758 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Share various performance metrics of a cluster", 2023-07-18T03:39:28.758 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.758 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.758 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.759 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.759 INFO:teuthology.orchestra.run.smithi101.stdout: "contact": { 2023-07-18T03:39:28.759 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "contact", 2023-07-18T03:39:28.759 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.759 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.760 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.760 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.760 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.760 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.761 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.761 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.761 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.761 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.761 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.761 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.762 INFO:teuthology.orchestra.run.smithi101.stdout: "description": { 2023-07-18T03:39:28.762 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "description", 2023-07-18T03:39:28.762 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.762 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.762 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.763 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.763 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.763 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.763 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.763 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.764 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.764 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.764 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.764 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.764 INFO:teuthology.orchestra.run.smithi101.stdout: "device_url": { 2023-07-18T03:39:28.765 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "device_url", 2023-07-18T03:39:28.765 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.765 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.765 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.765 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-07-18T03:39:28.766 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.766 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.766 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.766 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.766 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.767 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.767 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.767 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.767 INFO:teuthology.orchestra.run.smithi101.stdout: "enabled": { 2023-07-18T03:39:28.767 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "enabled", 2023-07-18T03:39:28.768 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.768 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.768 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.768 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.768 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.769 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.769 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.769 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.769 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.769 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.770 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.770 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.770 INFO:teuthology.orchestra.run.smithi101.stdout: "interval": { 2023-07-18T03:39:28.770 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "interval", 2023-07-18T03:39:28.770 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:28.771 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.771 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.771 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "24", 2023-07-18T03:39:28.771 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "8", 2023-07-18T03:39:28.771 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.772 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.772 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.772 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.772 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.772 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.773 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.773 INFO:teuthology.orchestra.run.smithi101.stdout: "last_opt_revision": { 2023-07-18T03:39:28.773 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "last_opt_revision", 2023-07-18T03:39:28.773 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:28.773 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.774 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.774 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "1", 2023-07-18T03:39:28.774 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.774 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.774 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.774 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.775 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.775 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.775 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.775 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.776 INFO:teuthology.orchestra.run.smithi101.stdout: "leaderboard": { 2023-07-18T03:39:28.776 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "leaderboard", 2023-07-18T03:39:28.776 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.776 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.776 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.777 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.777 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.777 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.777 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.777 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.778 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.778 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.778 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.778 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.778 INFO:teuthology.orchestra.run.smithi101.stdout: "leaderboard_description": { 2023-07-18T03:39:28.779 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "leaderboard_description", 2023-07-18T03:39:28.779 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.779 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.779 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.779 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.780 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.780 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.780 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.780 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.780 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.781 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.781 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.781 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.781 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:28.781 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:28.782 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.782 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.782 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.782 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.782 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.782 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.783 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.783 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.783 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.783 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.783 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.784 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.784 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.784 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.784 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.784 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.785 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.785 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.785 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.785 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:28.785 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:28.786 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.786 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.786 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.786 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.786 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.787 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.787 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.787 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.787 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.787 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.788 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.788 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.788 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:28.788 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:28.788 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.789 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.789 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.789 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:28.789 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.789 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.790 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.790 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.790 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.790 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.790 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.791 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.791 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.791 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.791 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.791 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.792 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.792 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.792 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.792 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:28.792 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:28.793 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.793 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.793 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.793 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.793 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.794 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.794 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.794 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.794 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.794 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.795 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.795 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.795 INFO:teuthology.orchestra.run.smithi101.stdout: "organization": { 2023-07-18T03:39:28.795 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "organization", 2023-07-18T03:39:28.795 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.795 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.796 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.796 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.796 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.796 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.797 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.797 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.797 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.797 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.797 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.797 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.798 INFO:teuthology.orchestra.run.smithi101.stdout: "proxy": { 2023-07-18T03:39:28.798 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "proxy", 2023-07-18T03:39:28.798 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.798 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.798 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.799 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.799 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.799 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.799 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.799 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.800 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.800 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.800 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.800 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.800 INFO:teuthology.orchestra.run.smithi101.stdout: "url": { 2023-07-18T03:39:28.801 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "url", 2023-07-18T03:39:28.801 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.801 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.801 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.801 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-07-18T03:39:28.802 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.802 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.802 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.802 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.802 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.803 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.803 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.803 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.803 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.803 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.804 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:28.804 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "test_orchestrator", 2023-07-18T03:39:28.804 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:28.804 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:28.804 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:28.805 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:28.805 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:28.805 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.805 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.805 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.806 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.806 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.806 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.806 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.806 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.807 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.807 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.807 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.807 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.807 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.808 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.808 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.808 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.808 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.809 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.809 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.809 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:28.809 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:28.809 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.810 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.810 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.810 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.810 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.810 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.811 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.811 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.811 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.811 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.811 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.811 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.812 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:28.812 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:28.812 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.812 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.813 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.813 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:28.813 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.813 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.813 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.814 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.814 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.814 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.814 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.814 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.815 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.815 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.815 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.815 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.815 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.816 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.816 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.816 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:28.816 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:28.816 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.817 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.817 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.817 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.817 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.817 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.818 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.818 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.818 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.818 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.818 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.819 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.819 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.819 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.819 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:28.819 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "volumes", 2023-07-18T03:39:28.820 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:28.820 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:28.820 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:28.820 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:28.820 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:28.821 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.821 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.821 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.821 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.821 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.821 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.822 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.822 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.822 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.822 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.822 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.823 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.823 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.823 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.823 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.823 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.824 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.824 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.824 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.824 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:28.825 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:28.825 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.825 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.825 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.825 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.825 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.826 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.826 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.826 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.826 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.826 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.827 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.827 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.827 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:28.827 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:28.827 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.828 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.828 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.828 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:28.828 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.828 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.829 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.829 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.829 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.829 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.829 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.830 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.830 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.830 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.830 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.830 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.831 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.831 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.831 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.831 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:28.831 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:28.832 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.832 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.832 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.832 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.832 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.833 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.833 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.833 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.833 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.833 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.834 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.834 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.834 INFO:teuthology.orchestra.run.smithi101.stdout: "max_concurrent_clones": { 2023-07-18T03:39:28.834 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "max_concurrent_clones", 2023-07-18T03:39:28.834 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:28.835 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.835 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.835 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "4", 2023-07-18T03:39:28.835 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.835 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.836 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.836 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Number of asynchronous cloner threads", 2023-07-18T03:39:28.836 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.836 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.836 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.837 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.837 INFO:teuthology.orchestra.run.smithi101.stdout: "snapshot_clone_delay": { 2023-07-18T03:39:28.837 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "snapshot_clone_delay", 2023-07-18T03:39:28.837 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:28.837 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.838 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.838 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "0", 2023-07-18T03:39:28.838 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.838 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.838 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.839 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-07-18T03:39:28.839 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.839 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.839 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.839 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.840 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.840 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.840 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:28.840 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "zabbix", 2023-07-18T03:39:28.840 INFO:teuthology.orchestra.run.smithi101.stdout: "can_run": true, 2023-07-18T03:39:28.841 INFO:teuthology.orchestra.run.smithi101.stdout: "error_string": "", 2023-07-18T03:39:28.841 INFO:teuthology.orchestra.run.smithi101.stdout: "module_options": { 2023-07-18T03:39:28.841 INFO:teuthology.orchestra.run.smithi101.stdout: "discovery_interval": { 2023-07-18T03:39:28.841 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "discovery_interval", 2023-07-18T03:39:28.841 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "uint", 2023-07-18T03:39:28.842 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.842 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.842 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "100", 2023-07-18T03:39:28.842 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.842 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.843 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.843 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.843 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.843 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.843 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.844 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.844 INFO:teuthology.orchestra.run.smithi101.stdout: "identifier": { 2023-07-18T03:39:28.844 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "identifier", 2023-07-18T03:39:28.844 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.844 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.845 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.845 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.845 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.845 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.845 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.846 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.846 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.846 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.846 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.846 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.847 INFO:teuthology.orchestra.run.smithi101.stdout: "interval": { 2023-07-18T03:39:28.847 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "interval", 2023-07-18T03:39:28.847 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "secs", 2023-07-18T03:39:28.847 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.847 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.848 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "60", 2023-07-18T03:39:28.848 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.848 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.848 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.848 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.849 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.849 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.849 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.849 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.849 INFO:teuthology.orchestra.run.smithi101.stdout: "log_level": { 2023-07-18T03:39:28.850 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_level", 2023-07-18T03:39:28.850 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.850 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.850 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.850 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.851 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.851 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.851 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.851 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.851 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.851 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.852 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.852 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.852 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.852 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.852 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.853 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.853 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.853 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.853 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.853 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster": { 2023-07-18T03:39:28.854 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster", 2023-07-18T03:39:28.854 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.854 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.854 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.854 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.855 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.855 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.855 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.855 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.855 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.856 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.856 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.856 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.856 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_cluster_level": { 2023-07-18T03:39:28.856 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_cluster_level", 2023-07-18T03:39:28.857 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.857 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.857 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.857 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "info", 2023-07-18T03:39:28.857 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.858 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.858 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [ 2023-07-18T03:39:28.858 INFO:teuthology.orchestra.run.smithi101.stdout: "", 2023-07-18T03:39:28.858 INFO:teuthology.orchestra.run.smithi101.stdout: "critical", 2023-07-18T03:39:28.858 INFO:teuthology.orchestra.run.smithi101.stdout: "debug", 2023-07-18T03:39:28.859 INFO:teuthology.orchestra.run.smithi101.stdout: "error", 2023-07-18T03:39:28.859 INFO:teuthology.orchestra.run.smithi101.stdout: "info", 2023-07-18T03:39:28.859 INFO:teuthology.orchestra.run.smithi101.stdout: "warning" 2023-07-18T03:39:28.859 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.859 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.860 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.860 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.860 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.860 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.860 INFO:teuthology.orchestra.run.smithi101.stdout: "log_to_file": { 2023-07-18T03:39:28.861 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "log_to_file", 2023-07-18T03:39:28.861 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "bool", 2023-07-18T03:39:28.861 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.861 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 1, 2023-07-18T03:39:28.861 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "False", 2023-07-18T03:39:28.862 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.862 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.862 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.862 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.862 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.863 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.863 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.863 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.863 INFO:teuthology.orchestra.run.smithi101.stdout: "zabbix_host": { 2023-07-18T03:39:28.863 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "zabbix_host", 2023-07-18T03:39:28.864 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.864 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.864 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.864 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "", 2023-07-18T03:39:28.864 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.865 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.865 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.865 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.865 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.866 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.866 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.866 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.866 INFO:teuthology.orchestra.run.smithi101.stdout: "zabbix_port": { 2023-07-18T03:39:28.866 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "zabbix_port", 2023-07-18T03:39:28.867 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "int", 2023-07-18T03:39:28.867 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.867 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.867 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "10051", 2023-07-18T03:39:28.867 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.867 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.868 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.868 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.868 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.868 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.869 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.869 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.869 INFO:teuthology.orchestra.run.smithi101.stdout: "zabbix_sender": { 2023-07-18T03:39:28.869 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "zabbix_sender", 2023-07-18T03:39:28.869 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "str", 2023-07-18T03:39:28.870 INFO:teuthology.orchestra.run.smithi101.stdout: "level": "advanced", 2023-07-18T03:39:28.870 INFO:teuthology.orchestra.run.smithi101.stdout: "flags": 0, 2023-07-18T03:39:28.870 INFO:teuthology.orchestra.run.smithi101.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-07-18T03:39:28.870 INFO:teuthology.orchestra.run.smithi101.stdout: "min": "", 2023-07-18T03:39:28.870 INFO:teuthology.orchestra.run.smithi101.stdout: "max": "", 2023-07-18T03:39:28.870 INFO:teuthology.orchestra.run.smithi101.stdout: "enum_allowed": [], 2023-07-18T03:39:28.871 INFO:teuthology.orchestra.run.smithi101.stdout: "desc": "", 2023-07-18T03:39:28.871 INFO:teuthology.orchestra.run.smithi101.stdout: "long_desc": "", 2023-07-18T03:39:28.871 INFO:teuthology.orchestra.run.smithi101.stdout: "tags": [], 2023-07-18T03:39:28.871 INFO:teuthology.orchestra.run.smithi101.stdout: "see_also": [] 2023-07-18T03:39:28.871 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.872 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.872 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.872 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.872 INFO:teuthology.orchestra.run.smithi101.stdout: "services": { 2023-07-18T03:39:28.872 INFO:teuthology.orchestra.run.smithi101.stdout: "dashboard": "https://172.21.15.101:8443/" 2023-07-18T03:39:28.873 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.873 INFO:teuthology.orchestra.run.smithi101.stdout: "always_on_modules": { 2023-07-18T03:39:28.873 INFO:teuthology.orchestra.run.smithi101.stdout: "octopus": [ 2023-07-18T03:39:28.873 INFO:teuthology.orchestra.run.smithi101.stdout: "balancer", 2023-07-18T03:39:28.873 INFO:teuthology.orchestra.run.smithi101.stdout: "crash", 2023-07-18T03:39:28.874 INFO:teuthology.orchestra.run.smithi101.stdout: "devicehealth", 2023-07-18T03:39:28.874 INFO:teuthology.orchestra.run.smithi101.stdout: "orchestrator", 2023-07-18T03:39:28.874 INFO:teuthology.orchestra.run.smithi101.stdout: "pg_autoscaler", 2023-07-18T03:39:28.874 INFO:teuthology.orchestra.run.smithi101.stdout: "progress", 2023-07-18T03:39:28.874 INFO:teuthology.orchestra.run.smithi101.stdout: "rbd_support", 2023-07-18T03:39:28.875 INFO:teuthology.orchestra.run.smithi101.stdout: "status", 2023-07-18T03:39:28.875 INFO:teuthology.orchestra.run.smithi101.stdout: "telemetry", 2023-07-18T03:39:28.875 INFO:teuthology.orchestra.run.smithi101.stdout: "volumes" 2023-07-18T03:39:28.875 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.875 INFO:teuthology.orchestra.run.smithi101.stdout: "pacific": [ 2023-07-18T03:39:28.876 INFO:teuthology.orchestra.run.smithi101.stdout: "balancer", 2023-07-18T03:39:28.876 INFO:teuthology.orchestra.run.smithi101.stdout: "crash", 2023-07-18T03:39:28.876 INFO:teuthology.orchestra.run.smithi101.stdout: "devicehealth", 2023-07-18T03:39:28.876 INFO:teuthology.orchestra.run.smithi101.stdout: "orchestrator", 2023-07-18T03:39:28.876 INFO:teuthology.orchestra.run.smithi101.stdout: "pg_autoscaler", 2023-07-18T03:39:28.877 INFO:teuthology.orchestra.run.smithi101.stdout: "progress", 2023-07-18T03:39:28.877 INFO:teuthology.orchestra.run.smithi101.stdout: "rbd_support", 2023-07-18T03:39:28.877 INFO:teuthology.orchestra.run.smithi101.stdout: "status", 2023-07-18T03:39:28.877 INFO:teuthology.orchestra.run.smithi101.stdout: "telemetry", 2023-07-18T03:39:28.877 INFO:teuthology.orchestra.run.smithi101.stdout: "volumes" 2023-07-18T03:39:28.878 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.878 INFO:teuthology.orchestra.run.smithi101.stdout: "quincy": [ 2023-07-18T03:39:28.878 INFO:teuthology.orchestra.run.smithi101.stdout: "balancer", 2023-07-18T03:39:28.878 INFO:teuthology.orchestra.run.smithi101.stdout: "crash", 2023-07-18T03:39:28.878 INFO:teuthology.orchestra.run.smithi101.stdout: "devicehealth", 2023-07-18T03:39:28.879 INFO:teuthology.orchestra.run.smithi101.stdout: "orchestrator", 2023-07-18T03:39:28.879 INFO:teuthology.orchestra.run.smithi101.stdout: "pg_autoscaler", 2023-07-18T03:39:28.879 INFO:teuthology.orchestra.run.smithi101.stdout: "progress", 2023-07-18T03:39:28.879 INFO:teuthology.orchestra.run.smithi101.stdout: "rbd_support", 2023-07-18T03:39:28.879 INFO:teuthology.orchestra.run.smithi101.stdout: "status", 2023-07-18T03:39:28.880 INFO:teuthology.orchestra.run.smithi101.stdout: "telemetry", 2023-07-18T03:39:28.880 INFO:teuthology.orchestra.run.smithi101.stdout: "volumes" 2023-07-18T03:39:28.880 INFO:teuthology.orchestra.run.smithi101.stdout: ], 2023-07-18T03:39:28.880 INFO:teuthology.orchestra.run.smithi101.stdout: "reef": [ 2023-07-18T03:39:28.880 INFO:teuthology.orchestra.run.smithi101.stdout: "balancer", 2023-07-18T03:39:28.881 INFO:teuthology.orchestra.run.smithi101.stdout: "crash", 2023-07-18T03:39:28.881 INFO:teuthology.orchestra.run.smithi101.stdout: "devicehealth", 2023-07-18T03:39:28.881 INFO:teuthology.orchestra.run.smithi101.stdout: "orchestrator", 2023-07-18T03:39:28.881 INFO:teuthology.orchestra.run.smithi101.stdout: "pg_autoscaler", 2023-07-18T03:39:28.881 INFO:teuthology.orchestra.run.smithi101.stdout: "progress", 2023-07-18T03:39:28.881 INFO:teuthology.orchestra.run.smithi101.stdout: "rbd_support", 2023-07-18T03:39:28.882 INFO:teuthology.orchestra.run.smithi101.stdout: "status", 2023-07-18T03:39:28.882 INFO:teuthology.orchestra.run.smithi101.stdout: "telemetry", 2023-07-18T03:39:28.882 INFO:teuthology.orchestra.run.smithi101.stdout: "volumes" 2023-07-18T03:39:28.882 INFO:teuthology.orchestra.run.smithi101.stdout: ] 2023-07-18T03:39:28.882 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.883 INFO:teuthology.orchestra.run.smithi101.stdout: "last_failure_osd_epoch": 92, 2023-07-18T03:39:28.883 INFO:teuthology.orchestra.run.smithi101.stdout: "active_clients": [ 2023-07-18T03:39:28.883 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:28.883 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "devicehealth", 2023-07-18T03:39:28.883 INFO:teuthology.orchestra.run.smithi101.stdout: "addrvec": [ 2023-07-18T03:39:28.884 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:28.884 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "v2", 2023-07-18T03:39:28.884 INFO:teuthology.orchestra.run.smithi101.stdout: "addr": "172.21.15.101:0", 2023-07-18T03:39:28.884 INFO:teuthology.orchestra.run.smithi101.stdout: "nonce": 3022708472 2023-07-18T03:39:28.884 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.885 INFO:teuthology.orchestra.run.smithi101.stdout: ] 2023-07-18T03:39:28.885 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.885 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:28.885 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "libcephsqlite", 2023-07-18T03:39:28.885 INFO:teuthology.orchestra.run.smithi101.stdout: "addrvec": [ 2023-07-18T03:39:28.886 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:28.886 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "v2", 2023-07-18T03:39:28.886 INFO:teuthology.orchestra.run.smithi101.stdout: "addr": "172.21.15.101:0", 2023-07-18T03:39:28.886 INFO:teuthology.orchestra.run.smithi101.stdout: "nonce": 3451856522 2023-07-18T03:39:28.886 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.887 INFO:teuthology.orchestra.run.smithi101.stdout: ] 2023-07-18T03:39:28.887 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.887 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:28.887 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "rbd_support", 2023-07-18T03:39:28.887 INFO:teuthology.orchestra.run.smithi101.stdout: "addrvec": [ 2023-07-18T03:39:28.888 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:28.888 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "v2", 2023-07-18T03:39:28.888 INFO:teuthology.orchestra.run.smithi101.stdout: "addr": "172.21.15.101:0", 2023-07-18T03:39:28.888 INFO:teuthology.orchestra.run.smithi101.stdout: "nonce": 2349441837 2023-07-18T03:39:28.888 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.889 INFO:teuthology.orchestra.run.smithi101.stdout: ] 2023-07-18T03:39:28.889 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.889 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:28.889 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "snap_schedule", 2023-07-18T03:39:28.889 INFO:teuthology.orchestra.run.smithi101.stdout: "addrvec": [ 2023-07-18T03:39:28.889 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:28.890 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "v2", 2023-07-18T03:39:28.890 INFO:teuthology.orchestra.run.smithi101.stdout: "addr": "172.21.15.101:0", 2023-07-18T03:39:28.890 INFO:teuthology.orchestra.run.smithi101.stdout: "nonce": 3790527713 2023-07-18T03:39:28.890 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.890 INFO:teuthology.orchestra.run.smithi101.stdout: ] 2023-07-18T03:39:28.891 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.891 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:28.891 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "snap_schedule", 2023-07-18T03:39:28.891 INFO:teuthology.orchestra.run.smithi101.stdout: "addrvec": [ 2023-07-18T03:39:28.891 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:28.892 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "v2", 2023-07-18T03:39:28.892 INFO:teuthology.orchestra.run.smithi101.stdout: "addr": "172.21.15.101:0", 2023-07-18T03:39:28.892 INFO:teuthology.orchestra.run.smithi101.stdout: "nonce": 4044618130 2023-07-18T03:39:28.892 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.892 INFO:teuthology.orchestra.run.smithi101.stdout: ] 2023-07-18T03:39:28.893 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:28.893 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:28.893 INFO:teuthology.orchestra.run.smithi101.stdout: "name": "volumes", 2023-07-18T03:39:28.893 INFO:teuthology.orchestra.run.smithi101.stdout: "addrvec": [ 2023-07-18T03:39:28.893 INFO:teuthology.orchestra.run.smithi101.stdout: { 2023-07-18T03:39:28.894 INFO:teuthology.orchestra.run.smithi101.stdout: "type": "v2", 2023-07-18T03:39:28.894 INFO:teuthology.orchestra.run.smithi101.stdout: "addr": "172.21.15.101:0", 2023-07-18T03:39:28.894 INFO:teuthology.orchestra.run.smithi101.stdout: "nonce": 2501919258 2023-07-18T03:39:28.894 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.894 INFO:teuthology.orchestra.run.smithi101.stdout: ] 2023-07-18T03:39:28.895 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:28.895 INFO:teuthology.orchestra.run.smithi101.stdout: ] 2023-07-18T03:39:28.895 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:39:28.901 DEBUG:tasks.check_counter:Getting stats from a 2023-07-18T03:39:28.902 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph daemon mds.a perf dump 2023-07-18T03:39:28.903 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:39:25 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2998272586' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-18T03:39:28.903 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:39:25 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/511151978' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-07-18T03:39:28.903 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:39:26 smithi101 ceph-mon[126379]: pgmap v1011: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:39:28.904 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:39:25 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/2998272586' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-18T03:39:28.904 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:39:25 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/511151978' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-07-18T03:39:28.905 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:39:26 smithi155 ceph-mon[128885]: pgmap v1011: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:39:28.905 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:39:25 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/2998272586' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-18T03:39:28.905 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:39:25 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/511151978' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-07-18T03:39:28.905 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:39:26 smithi179 ceph-mon[129229]: pgmap v1011: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:39:29.546 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:39:29 smithi155 ceph-mon[128885]: pgmap v1012: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:39:29.665 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:39:29.701 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:39:29 smithi179 ceph-mon[129229]: pgmap v1012: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:39:29.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:39:29 smithi101 ceph-mon[126379]: pgmap v1012: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 204 B/s wr, 6 op/s 2023-07-18T03:39:31.453 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:39:31.453 INFO:teuthology.orchestra.run.smithi101.stdout: "AsyncMessenger::Worker-0": { 2023-07-18T03:39:31.454 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_messages": 560, 2023-07-18T03:39:31.454 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages": 550, 2023-07-18T03:39:31.454 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_bytes": 1519376, 2023-07-18T03:39:31.454 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_bytes": 315792, 2023-07-18T03:39:31.455 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_created_connections": 4, 2023-07-18T03:39:31.455 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_active_connections": 0, 2023-07-18T03:39:31.455 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_total_time": 0.107423915, 2023-07-18T03:39:31.455 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_send_time": 0.046606801, 2023-07-18T03:39:31.455 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_recv_time": 0.044862450, 2023-07-18T03:39:31.456 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_fast_dispatch_time": 0.012487902, 2023-07-18T03:39:31.456 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:39:31.456 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 549, 2023-07-18T03:39:31.456 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.058126490, 2023-07-18T03:39:31.456 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000105877 2023-07-18T03:39:31.457 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:31.457 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:39:31.457 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-18T03:39:31.457 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-18T03:39:31.458 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-18T03:39:31.458 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:31.458 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_encrypted_bytes": 1519376, 2023-07-18T03:39:31.458 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_encrypted_bytes": 315792 2023-07-18T03:39:31.458 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:31.459 INFO:teuthology.orchestra.run.smithi101.stdout: "AsyncMessenger::Worker-1": { 2023-07-18T03:39:31.459 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_messages": 8, 2023-07-18T03:39:31.459 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages": 424, 2023-07-18T03:39:31.459 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_bytes": 3584, 2023-07-18T03:39:31.459 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_bytes": 377872, 2023-07-18T03:39:31.460 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_created_connections": 7, 2023-07-18T03:39:31.460 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_active_connections": 3, 2023-07-18T03:39:31.460 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_total_time": 0.042883517, 2023-07-18T03:39:31.460 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_send_time": 0.031467367, 2023-07-18T03:39:31.460 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_recv_time": 0.004293233, 2023-07-18T03:39:31.461 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-18T03:39:31.461 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:39:31.461 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 422, 2023-07-18T03:39:31.461 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.040520369, 2023-07-18T03:39:31.461 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000096019 2023-07-18T03:39:31.462 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:31.462 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:39:31.462 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-18T03:39:31.462 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-18T03:39:31.462 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-18T03:39:31.463 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:31.463 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_encrypted_bytes": 3584, 2023-07-18T03:39:31.463 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_encrypted_bytes": 377872 2023-07-18T03:39:31.463 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:31.463 INFO:teuthology.orchestra.run.smithi101.stdout: "AsyncMessenger::Worker-2": { 2023-07-18T03:39:31.464 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_messages": 0, 2023-07-18T03:39:31.464 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages": 2, 2023-07-18T03:39:31.464 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_bytes": 0, 2023-07-18T03:39:31.464 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_bytes": 0, 2023-07-18T03:39:31.465 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_created_connections": 2, 2023-07-18T03:39:31.465 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_active_connections": 1, 2023-07-18T03:39:31.465 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_total_time": 0.000823081, 2023-07-18T03:39:31.465 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_send_time": 0.000000000, 2023-07-18T03:39:31.465 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_recv_time": 0.000644572, 2023-07-18T03:39:31.466 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-18T03:39:31.466 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:39:31.466 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-18T03:39:31.466 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-18T03:39:31.466 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-18T03:39:31.467 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:31.467 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:39:31.467 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-18T03:39:31.467 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-18T03:39:31.467 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-18T03:39:31.468 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:31.468 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-18T03:39:31.468 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_encrypted_bytes": 0 2023-07-18T03:39:31.468 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:31.468 INFO:teuthology.orchestra.run.smithi101.stdout: "cct": { 2023-07-18T03:39:31.469 INFO:teuthology.orchestra.run.smithi101.stdout: "total_workers": 0, 2023-07-18T03:39:31.469 INFO:teuthology.orchestra.run.smithi101.stdout: "unhealthy_workers": 0 2023-07-18T03:39:31.469 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:31.469 INFO:teuthology.orchestra.run.smithi101.stdout: "mempool": { 2023-07-18T03:39:31.469 INFO:teuthology.orchestra.run.smithi101.stdout: "bloom_filter_bytes": 0, 2023-07-18T03:39:31.470 INFO:teuthology.orchestra.run.smithi101.stdout: "bloom_filter_items": 0, 2023-07-18T03:39:31.470 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_alloc_bytes": 0, 2023-07-18T03:39:31.470 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_alloc_items": 0, 2023-07-18T03:39:31.470 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_data_bytes": 0, 2023-07-18T03:39:31.470 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_data_items": 0, 2023-07-18T03:39:31.471 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-18T03:39:31.471 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_onode_items": 0, 2023-07-18T03:39:31.471 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-18T03:39:31.471 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_meta_items": 0, 2023-07-18T03:39:31.471 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_other_bytes": 0, 2023-07-18T03:39:31.472 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_other_items": 0, 2023-07-18T03:39:31.472 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-18T03:39:31.472 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_buffer_items": 0, 2023-07-18T03:39:31.472 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_extent_bytes": 0, 2023-07-18T03:39:31.473 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_extent_items": 0, 2023-07-18T03:39:31.473 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_blob_bytes": 0, 2023-07-18T03:39:31.473 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_blob_items": 0, 2023-07-18T03:39:31.473 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-18T03:39:31.473 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_shared_blob_items": 0, 2023-07-18T03:39:31.474 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-18T03:39:31.474 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_inline_bl_items": 0, 2023-07-18T03:39:31.474 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_fsck_bytes": 0, 2023-07-18T03:39:31.474 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_fsck_items": 0, 2023-07-18T03:39:31.474 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_txc_bytes": 0, 2023-07-18T03:39:31.475 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_txc_items": 0, 2023-07-18T03:39:31.475 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-18T03:39:31.475 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_writing_deferred_items": 0, 2023-07-18T03:39:31.475 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_writing_bytes": 0, 2023-07-18T03:39:31.475 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_writing_items": 0, 2023-07-18T03:39:31.476 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_bytes": 0, 2023-07-18T03:39:31.476 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_items": 0, 2023-07-18T03:39:31.476 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_file_reader_bytes": 0, 2023-07-18T03:39:31.476 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_file_reader_items": 0, 2023-07-18T03:39:31.476 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_file_writer_bytes": 0, 2023-07-18T03:39:31.477 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_file_writer_items": 0, 2023-07-18T03:39:31.477 INFO:teuthology.orchestra.run.smithi101.stdout: "buffer_anon_bytes": 148714, 2023-07-18T03:39:31.477 INFO:teuthology.orchestra.run.smithi101.stdout: "buffer_anon_items": 30, 2023-07-18T03:39:31.477 INFO:teuthology.orchestra.run.smithi101.stdout: "buffer_meta_bytes": 0, 2023-07-18T03:39:31.477 INFO:teuthology.orchestra.run.smithi101.stdout: "buffer_meta_items": 0, 2023-07-18T03:39:31.478 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_bytes": 0, 2023-07-18T03:39:31.478 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_items": 0, 2023-07-18T03:39:31.478 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_mapbl_bytes": 0, 2023-07-18T03:39:31.478 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_mapbl_items": 0, 2023-07-18T03:39:31.479 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_pglog_bytes": 0, 2023-07-18T03:39:31.479 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_pglog_items": 0, 2023-07-18T03:39:31.479 INFO:teuthology.orchestra.run.smithi101.stdout: "osdmap_bytes": 0, 2023-07-18T03:39:31.479 INFO:teuthology.orchestra.run.smithi101.stdout: "osdmap_items": 0, 2023-07-18T03:39:31.479 INFO:teuthology.orchestra.run.smithi101.stdout: "osdmap_mapping_bytes": 0, 2023-07-18T03:39:31.480 INFO:teuthology.orchestra.run.smithi101.stdout: "osdmap_mapping_items": 0, 2023-07-18T03:39:31.480 INFO:teuthology.orchestra.run.smithi101.stdout: "pgmap_bytes": 0, 2023-07-18T03:39:31.480 INFO:teuthology.orchestra.run.smithi101.stdout: "pgmap_items": 0, 2023-07-18T03:39:31.480 INFO:teuthology.orchestra.run.smithi101.stdout: "mds_co_bytes": 928, 2023-07-18T03:39:31.480 INFO:teuthology.orchestra.run.smithi101.stdout: "mds_co_items": 2, 2023-07-18T03:39:31.481 INFO:teuthology.orchestra.run.smithi101.stdout: "unittest_1_bytes": 0, 2023-07-18T03:39:31.481 INFO:teuthology.orchestra.run.smithi101.stdout: "unittest_1_items": 0, 2023-07-18T03:39:31.481 INFO:teuthology.orchestra.run.smithi101.stdout: "unittest_2_bytes": 0, 2023-07-18T03:39:31.481 INFO:teuthology.orchestra.run.smithi101.stdout: "unittest_2_items": 0 2023-07-18T03:39:31.481 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:31.482 INFO:teuthology.orchestra.run.smithi101.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-18T03:39:31.482 INFO:teuthology.orchestra.run.smithi101.stdout: "val": 0, 2023-07-18T03:39:31.482 INFO:teuthology.orchestra.run.smithi101.stdout: "max": 104857600, 2023-07-18T03:39:31.482 INFO:teuthology.orchestra.run.smithi101.stdout: "get_started": 0, 2023-07-18T03:39:31.482 INFO:teuthology.orchestra.run.smithi101.stdout: "get": 564, 2023-07-18T03:39:31.483 INFO:teuthology.orchestra.run.smithi101.stdout: "get_sum": 1440225, 2023-07-18T03:39:31.483 INFO:teuthology.orchestra.run.smithi101.stdout: "get_or_fail_fail": 0, 2023-07-18T03:39:31.483 INFO:teuthology.orchestra.run.smithi101.stdout: "get_or_fail_success": 564, 2023-07-18T03:39:31.483 INFO:teuthology.orchestra.run.smithi101.stdout: "take": 0, 2023-07-18T03:39:31.483 INFO:teuthology.orchestra.run.smithi101.stdout: "take_sum": 0, 2023-07-18T03:39:31.484 INFO:teuthology.orchestra.run.smithi101.stdout: "put": 564, 2023-07-18T03:39:31.484 INFO:teuthology.orchestra.run.smithi101.stdout: "put_sum": 1440225, 2023-07-18T03:39:31.484 INFO:teuthology.orchestra.run.smithi101.stdout: "wait": { 2023-07-18T03:39:31.484 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-18T03:39:31.485 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-18T03:39:31.485 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-18T03:39:31.485 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:31.485 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:31.485 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:39:31.725 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:39:31 smithi155 ceph-mon[128885]: pgmap v1013: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 190 B/s wr, 8 op/s 2023-07-18T03:39:31.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:39:31 smithi101 ceph-mon[126379]: pgmap v1013: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 190 B/s wr, 8 op/s 2023-07-18T03:39:31.951 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:39:31 smithi179 ceph-mon[129229]: pgmap v1013: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 190 B/s wr, 8 op/s 2023-07-18T03:39:31.961 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2023-07-18T03:39:31.961 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2023-07-18T03:39:31.962 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2023-07-18T03:39:31.962 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2023-07-18T03:39:31.962 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2023-07-18T03:39:31.962 DEBUG:tasks.check_counter:Getting stats from d 2023-07-18T03:39:31.963 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph daemon mds.d perf dump 2023-07-18T03:39:32.702 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:39:33.573 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:39:33 smithi101 ceph-mon[126379]: pgmap v1014: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:39:33.796 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:39:33 smithi155 ceph-mon[128885]: pgmap v1014: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:39:33.951 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:39:33 smithi179 ceph-mon[129229]: pgmap v1014: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:39:34.357 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:39:34.358 INFO:teuthology.orchestra.run.smithi101.stdout: "AsyncMessenger::Worker-0": { 2023-07-18T03:39:34.358 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_messages": 0, 2023-07-18T03:39:34.358 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages": 2, 2023-07-18T03:39:34.359 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_bytes": 0, 2023-07-18T03:39:34.359 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_bytes": 0, 2023-07-18T03:39:34.359 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_created_connections": 4, 2023-07-18T03:39:34.359 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_active_connections": 0, 2023-07-18T03:39:34.359 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_total_time": 0.003295338, 2023-07-18T03:39:34.360 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_send_time": 0.000000000, 2023-07-18T03:39:34.360 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_recv_time": 0.000658982, 2023-07-18T03:39:34.360 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-18T03:39:34.360 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:39:34.360 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-18T03:39:34.361 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-18T03:39:34.361 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-18T03:39:34.361 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:34.361 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:39:34.361 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-18T03:39:34.362 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-18T03:39:34.362 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-18T03:39:34.362 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:34.362 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-18T03:39:34.362 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_encrypted_bytes": 0 2023-07-18T03:39:34.363 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:34.363 INFO:teuthology.orchestra.run.smithi101.stdout: "AsyncMessenger::Worker-1": { 2023-07-18T03:39:34.363 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_messages": 4, 2023-07-18T03:39:34.363 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages": 426, 2023-07-18T03:39:34.363 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_bytes": 640, 2023-07-18T03:39:34.364 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_bytes": 378128, 2023-07-18T03:39:34.364 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_created_connections": 7, 2023-07-18T03:39:34.364 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_active_connections": 3, 2023-07-18T03:39:34.364 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_total_time": 0.043276235, 2023-07-18T03:39:34.364 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_send_time": 0.032791008, 2023-07-18T03:39:34.365 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_recv_time": 0.003589508, 2023-07-18T03:39:34.365 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-18T03:39:34.365 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:39:34.365 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 423, 2023-07-18T03:39:34.365 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.041772884, 2023-07-18T03:39:34.365 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000098753 2023-07-18T03:39:34.366 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:34.366 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:39:34.366 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-18T03:39:34.366 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-18T03:39:34.366 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-18T03:39:34.367 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:34.367 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_encrypted_bytes": 640, 2023-07-18T03:39:34.367 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_encrypted_bytes": 378128 2023-07-18T03:39:34.367 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:34.367 INFO:teuthology.orchestra.run.smithi101.stdout: "AsyncMessenger::Worker-2": { 2023-07-18T03:39:34.368 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_messages": 570, 2023-07-18T03:39:34.368 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages": 558, 2023-07-18T03:39:34.368 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_bytes": 1523952, 2023-07-18T03:39:34.368 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_bytes": 319792, 2023-07-18T03:39:34.368 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_created_connections": 2, 2023-07-18T03:39:34.369 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_active_connections": 1, 2023-07-18T03:39:34.369 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_total_time": 0.108288365, 2023-07-18T03:39:34.369 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_send_time": 0.048744558, 2023-07-18T03:39:34.369 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_recv_time": 0.046528325, 2023-07-18T03:39:34.369 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_fast_dispatch_time": 0.012904645, 2023-07-18T03:39:34.370 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:39:34.370 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 558, 2023-07-18T03:39:34.370 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.047292279, 2023-07-18T03:39:34.370 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000084753 2023-07-18T03:39:34.370 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:34.371 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:39:34.371 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-18T03:39:34.371 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-18T03:39:34.371 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-18T03:39:34.371 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:34.371 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_encrypted_bytes": 1523952, 2023-07-18T03:39:34.372 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_encrypted_bytes": 319792 2023-07-18T03:39:34.372 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:34.372 INFO:teuthology.orchestra.run.smithi101.stdout: "cct": { 2023-07-18T03:39:34.372 INFO:teuthology.orchestra.run.smithi101.stdout: "total_workers": 0, 2023-07-18T03:39:34.372 INFO:teuthology.orchestra.run.smithi101.stdout: "unhealthy_workers": 0 2023-07-18T03:39:34.373 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:34.373 INFO:teuthology.orchestra.run.smithi101.stdout: "mempool": { 2023-07-18T03:39:34.373 INFO:teuthology.orchestra.run.smithi101.stdout: "bloom_filter_bytes": 0, 2023-07-18T03:39:34.373 INFO:teuthology.orchestra.run.smithi101.stdout: "bloom_filter_items": 0, 2023-07-18T03:39:34.373 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_alloc_bytes": 0, 2023-07-18T03:39:34.374 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_alloc_items": 0, 2023-07-18T03:39:34.374 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_data_bytes": 0, 2023-07-18T03:39:34.374 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_data_items": 0, 2023-07-18T03:39:34.374 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-18T03:39:34.374 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_onode_items": 0, 2023-07-18T03:39:34.375 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-18T03:39:34.375 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_meta_items": 0, 2023-07-18T03:39:34.375 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_other_bytes": 0, 2023-07-18T03:39:34.375 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_other_items": 0, 2023-07-18T03:39:34.375 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-18T03:39:34.376 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_buffer_items": 0, 2023-07-18T03:39:34.376 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_extent_bytes": 0, 2023-07-18T03:39:34.376 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_extent_items": 0, 2023-07-18T03:39:34.376 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_blob_bytes": 0, 2023-07-18T03:39:34.376 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_blob_items": 0, 2023-07-18T03:39:34.377 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-18T03:39:34.377 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_shared_blob_items": 0, 2023-07-18T03:39:34.377 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-18T03:39:34.377 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_inline_bl_items": 0, 2023-07-18T03:39:34.377 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_fsck_bytes": 0, 2023-07-18T03:39:34.377 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_fsck_items": 0, 2023-07-18T03:39:34.378 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_txc_bytes": 0, 2023-07-18T03:39:34.378 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_txc_items": 0, 2023-07-18T03:39:34.378 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-18T03:39:34.378 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_writing_deferred_items": 0, 2023-07-18T03:39:34.378 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_writing_bytes": 0, 2023-07-18T03:39:34.379 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_writing_items": 0, 2023-07-18T03:39:34.379 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_bytes": 0, 2023-07-18T03:39:34.379 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_items": 0, 2023-07-18T03:39:34.379 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_file_reader_bytes": 0, 2023-07-18T03:39:34.379 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_file_reader_items": 0, 2023-07-18T03:39:34.380 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_file_writer_bytes": 0, 2023-07-18T03:39:34.380 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_file_writer_items": 0, 2023-07-18T03:39:34.380 INFO:teuthology.orchestra.run.smithi101.stdout: "buffer_anon_bytes": 147930, 2023-07-18T03:39:34.380 INFO:teuthology.orchestra.run.smithi101.stdout: "buffer_anon_items": 28, 2023-07-18T03:39:34.380 INFO:teuthology.orchestra.run.smithi101.stdout: "buffer_meta_bytes": 0, 2023-07-18T03:39:34.381 INFO:teuthology.orchestra.run.smithi101.stdout: "buffer_meta_items": 0, 2023-07-18T03:39:34.381 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_bytes": 0, 2023-07-18T03:39:34.381 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_items": 0, 2023-07-18T03:39:34.381 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_mapbl_bytes": 0, 2023-07-18T03:39:34.381 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_mapbl_items": 0, 2023-07-18T03:39:34.382 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_pglog_bytes": 0, 2023-07-18T03:39:34.382 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_pglog_items": 0, 2023-07-18T03:39:34.382 INFO:teuthology.orchestra.run.smithi101.stdout: "osdmap_bytes": 0, 2023-07-18T03:39:34.382 INFO:teuthology.orchestra.run.smithi101.stdout: "osdmap_items": 0, 2023-07-18T03:39:34.382 INFO:teuthology.orchestra.run.smithi101.stdout: "osdmap_mapping_bytes": 0, 2023-07-18T03:39:34.383 INFO:teuthology.orchestra.run.smithi101.stdout: "osdmap_mapping_items": 0, 2023-07-18T03:39:34.383 INFO:teuthology.orchestra.run.smithi101.stdout: "pgmap_bytes": 0, 2023-07-18T03:39:34.383 INFO:teuthology.orchestra.run.smithi101.stdout: "pgmap_items": 0, 2023-07-18T03:39:34.383 INFO:teuthology.orchestra.run.smithi101.stdout: "mds_co_bytes": 928, 2023-07-18T03:39:34.383 INFO:teuthology.orchestra.run.smithi101.stdout: "mds_co_items": 2, 2023-07-18T03:39:34.383 INFO:teuthology.orchestra.run.smithi101.stdout: "unittest_1_bytes": 0, 2023-07-18T03:39:34.384 INFO:teuthology.orchestra.run.smithi101.stdout: "unittest_1_items": 0, 2023-07-18T03:39:34.384 INFO:teuthology.orchestra.run.smithi101.stdout: "unittest_2_bytes": 0, 2023-07-18T03:39:34.384 INFO:teuthology.orchestra.run.smithi101.stdout: "unittest_2_items": 0 2023-07-18T03:39:34.384 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:34.384 INFO:teuthology.orchestra.run.smithi101.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-18T03:39:34.385 INFO:teuthology.orchestra.run.smithi101.stdout: "val": 0, 2023-07-18T03:39:34.385 INFO:teuthology.orchestra.run.smithi101.stdout: "max": 104857600, 2023-07-18T03:39:34.385 INFO:teuthology.orchestra.run.smithi101.stdout: "get_started": 0, 2023-07-18T03:39:34.385 INFO:teuthology.orchestra.run.smithi101.stdout: "get": 570, 2023-07-18T03:39:34.385 INFO:teuthology.orchestra.run.smithi101.stdout: "get_sum": 1441005, 2023-07-18T03:39:34.386 INFO:teuthology.orchestra.run.smithi101.stdout: "get_or_fail_fail": 0, 2023-07-18T03:39:34.386 INFO:teuthology.orchestra.run.smithi101.stdout: "get_or_fail_success": 570, 2023-07-18T03:39:34.386 INFO:teuthology.orchestra.run.smithi101.stdout: "take": 0, 2023-07-18T03:39:34.386 INFO:teuthology.orchestra.run.smithi101.stdout: "take_sum": 0, 2023-07-18T03:39:34.386 INFO:teuthology.orchestra.run.smithi101.stdout: "put": 570, 2023-07-18T03:39:34.387 INFO:teuthology.orchestra.run.smithi101.stdout: "put_sum": 1441005, 2023-07-18T03:39:34.387 INFO:teuthology.orchestra.run.smithi101.stdout: "wait": { 2023-07-18T03:39:34.387 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-18T03:39:34.387 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-18T03:39:34.387 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-18T03:39:34.388 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:34.388 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:34.388 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:39:34.793 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2023-07-18T03:39:34.793 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2023-07-18T03:39:34.794 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2023-07-18T03:39:34.794 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2023-07-18T03:39:34.794 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2023-07-18T03:39:34.794 DEBUG:tasks.check_counter:Getting stats from g 2023-07-18T03:39:34.795 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph daemon mds.g perf dump 2023-07-18T03:39:35.536 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:39:35.738 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:39:35 smithi101 ceph-mon[126379]: pgmap v1015: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:39:35.796 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:39:35 smithi155 ceph-mon[128885]: pgmap v1015: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:39:35.951 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:39:35 smithi179 ceph-mon[129229]: pgmap v1015: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:39:37.402 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:39:37.402 INFO:teuthology.orchestra.run.smithi101.stdout: "AsyncMessenger::Worker-0": { 2023-07-18T03:39:37.402 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_messages": 0, 2023-07-18T03:39:37.403 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages": 2, 2023-07-18T03:39:37.403 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_bytes": 0, 2023-07-18T03:39:37.403 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_bytes": 0, 2023-07-18T03:39:37.403 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_created_connections": 4, 2023-07-18T03:39:37.403 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_active_connections": 0, 2023-07-18T03:39:37.404 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_total_time": 0.003268128, 2023-07-18T03:39:37.404 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_send_time": 0.000000000, 2023-07-18T03:39:37.404 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_recv_time": 0.000522925, 2023-07-18T03:39:37.404 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-18T03:39:37.404 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:39:37.405 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-18T03:39:37.405 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-18T03:39:37.405 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-18T03:39:37.405 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:37.405 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:39:37.406 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-18T03:39:37.406 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-18T03:39:37.406 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-18T03:39:37.406 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:37.406 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-18T03:39:37.407 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_encrypted_bytes": 0 2023-07-18T03:39:37.407 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:37.407 INFO:teuthology.orchestra.run.smithi101.stdout: "AsyncMessenger::Worker-1": { 2023-07-18T03:39:37.407 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_messages": 8, 2023-07-18T03:39:37.407 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages": 426, 2023-07-18T03:39:37.408 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_bytes": 3584, 2023-07-18T03:39:37.408 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_bytes": 378224, 2023-07-18T03:39:37.408 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_created_connections": 7, 2023-07-18T03:39:37.408 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_active_connections": 3, 2023-07-18T03:39:37.408 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_total_time": 0.043311919, 2023-07-18T03:39:37.409 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_send_time": 0.032323546, 2023-07-18T03:39:37.409 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_recv_time": 0.004069491, 2023-07-18T03:39:37.409 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-18T03:39:37.409 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:39:37.409 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 424, 2023-07-18T03:39:37.409 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.042370153, 2023-07-18T03:39:37.410 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000099929 2023-07-18T03:39:37.410 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:37.410 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:39:37.410 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-18T03:39:37.410 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-18T03:39:37.411 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-18T03:39:37.411 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:37.411 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_encrypted_bytes": 3584, 2023-07-18T03:39:37.411 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_encrypted_bytes": 378224 2023-07-18T03:39:37.411 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:37.412 INFO:teuthology.orchestra.run.smithi101.stdout: "AsyncMessenger::Worker-2": { 2023-07-18T03:39:37.412 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_messages": 563, 2023-07-18T03:39:37.413 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages": 553, 2023-07-18T03:39:37.413 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_bytes": 1520192, 2023-07-18T03:39:37.413 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_bytes": 317568, 2023-07-18T03:39:37.414 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_created_connections": 2, 2023-07-18T03:39:37.414 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_active_connections": 1, 2023-07-18T03:39:37.414 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_total_time": 0.105482636, 2023-07-18T03:39:37.414 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_send_time": 0.047030088, 2023-07-18T03:39:37.415 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_recv_time": 0.045973993, 2023-07-18T03:39:37.415 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_fast_dispatch_time": 0.012605974, 2023-07-18T03:39:37.415 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:39:37.415 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 552, 2023-07-18T03:39:37.415 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.041505346, 2023-07-18T03:39:37.416 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000075190 2023-07-18T03:39:37.416 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:37.416 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:39:37.417 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-18T03:39:37.417 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-18T03:39:37.417 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-18T03:39:37.417 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:37.417 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_encrypted_bytes": 1520192, 2023-07-18T03:39:37.418 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_encrypted_bytes": 317568 2023-07-18T03:39:37.418 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:37.418 INFO:teuthology.orchestra.run.smithi101.stdout: "cct": { 2023-07-18T03:39:37.418 INFO:teuthology.orchestra.run.smithi101.stdout: "total_workers": 0, 2023-07-18T03:39:37.419 INFO:teuthology.orchestra.run.smithi101.stdout: "unhealthy_workers": 0 2023-07-18T03:39:37.419 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:37.419 INFO:teuthology.orchestra.run.smithi101.stdout: "mempool": { 2023-07-18T03:39:37.419 INFO:teuthology.orchestra.run.smithi101.stdout: "bloom_filter_bytes": 0, 2023-07-18T03:39:37.419 INFO:teuthology.orchestra.run.smithi101.stdout: "bloom_filter_items": 0, 2023-07-18T03:39:37.420 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_alloc_bytes": 0, 2023-07-18T03:39:37.420 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_alloc_items": 0, 2023-07-18T03:39:37.420 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_data_bytes": 0, 2023-07-18T03:39:37.420 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_data_items": 0, 2023-07-18T03:39:37.421 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-18T03:39:37.421 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_onode_items": 0, 2023-07-18T03:39:37.421 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-18T03:39:37.421 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_meta_items": 0, 2023-07-18T03:39:37.422 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_other_bytes": 0, 2023-07-18T03:39:37.422 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_other_items": 0, 2023-07-18T03:39:37.422 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-18T03:39:37.422 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_buffer_items": 0, 2023-07-18T03:39:37.423 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_extent_bytes": 0, 2023-07-18T03:39:37.423 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_extent_items": 0, 2023-07-18T03:39:37.424 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_blob_bytes": 0, 2023-07-18T03:39:37.424 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_blob_items": 0, 2023-07-18T03:39:37.424 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-18T03:39:37.425 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_shared_blob_items": 0, 2023-07-18T03:39:37.425 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-18T03:39:37.425 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_inline_bl_items": 0, 2023-07-18T03:39:37.425 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_fsck_bytes": 0, 2023-07-18T03:39:37.426 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_fsck_items": 0, 2023-07-18T03:39:37.426 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_txc_bytes": 0, 2023-07-18T03:39:37.426 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_txc_items": 0, 2023-07-18T03:39:37.426 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-18T03:39:37.427 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_writing_deferred_items": 0, 2023-07-18T03:39:37.427 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_writing_bytes": 0, 2023-07-18T03:39:37.427 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_writing_items": 0, 2023-07-18T03:39:37.427 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_bytes": 0, 2023-07-18T03:39:37.427 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_items": 0, 2023-07-18T03:39:37.428 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_file_reader_bytes": 0, 2023-07-18T03:39:37.428 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_file_reader_items": 0, 2023-07-18T03:39:37.428 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_file_writer_bytes": 0, 2023-07-18T03:39:37.428 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_file_writer_items": 0, 2023-07-18T03:39:37.428 INFO:teuthology.orchestra.run.smithi101.stdout: "buffer_anon_bytes": 148714, 2023-07-18T03:39:37.429 INFO:teuthology.orchestra.run.smithi101.stdout: "buffer_anon_items": 30, 2023-07-18T03:39:37.429 INFO:teuthology.orchestra.run.smithi101.stdout: "buffer_meta_bytes": 0, 2023-07-18T03:39:37.429 INFO:teuthology.orchestra.run.smithi101.stdout: "buffer_meta_items": 0, 2023-07-18T03:39:37.429 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_bytes": 0, 2023-07-18T03:39:37.429 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_items": 0, 2023-07-18T03:39:37.430 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_mapbl_bytes": 0, 2023-07-18T03:39:37.430 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_mapbl_items": 0, 2023-07-18T03:39:37.430 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_pglog_bytes": 0, 2023-07-18T03:39:37.430 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_pglog_items": 0, 2023-07-18T03:39:37.430 INFO:teuthology.orchestra.run.smithi101.stdout: "osdmap_bytes": 0, 2023-07-18T03:39:37.430 INFO:teuthology.orchestra.run.smithi101.stdout: "osdmap_items": 0, 2023-07-18T03:39:37.431 INFO:teuthology.orchestra.run.smithi101.stdout: "osdmap_mapping_bytes": 0, 2023-07-18T03:39:37.431 INFO:teuthology.orchestra.run.smithi101.stdout: "osdmap_mapping_items": 0, 2023-07-18T03:39:37.431 INFO:teuthology.orchestra.run.smithi101.stdout: "pgmap_bytes": 0, 2023-07-18T03:39:37.431 INFO:teuthology.orchestra.run.smithi101.stdout: "pgmap_items": 0, 2023-07-18T03:39:37.431 INFO:teuthology.orchestra.run.smithi101.stdout: "mds_co_bytes": 928, 2023-07-18T03:39:37.432 INFO:teuthology.orchestra.run.smithi101.stdout: "mds_co_items": 2, 2023-07-18T03:39:37.432 INFO:teuthology.orchestra.run.smithi101.stdout: "unittest_1_bytes": 0, 2023-07-18T03:39:37.432 INFO:teuthology.orchestra.run.smithi101.stdout: "unittest_1_items": 0, 2023-07-18T03:39:37.432 INFO:teuthology.orchestra.run.smithi101.stdout: "unittest_2_bytes": 0, 2023-07-18T03:39:37.432 INFO:teuthology.orchestra.run.smithi101.stdout: "unittest_2_items": 0 2023-07-18T03:39:37.433 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:37.433 INFO:teuthology.orchestra.run.smithi101.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-18T03:39:37.433 INFO:teuthology.orchestra.run.smithi101.stdout: "val": 0, 2023-07-18T03:39:37.433 INFO:teuthology.orchestra.run.smithi101.stdout: "max": 104857600, 2023-07-18T03:39:37.433 INFO:teuthology.orchestra.run.smithi101.stdout: "get_started": 0, 2023-07-18T03:39:37.434 INFO:teuthology.orchestra.run.smithi101.stdout: "get": 567, 2023-07-18T03:39:37.434 INFO:teuthology.orchestra.run.smithi101.stdout: "get_sum": 1440615, 2023-07-18T03:39:37.434 INFO:teuthology.orchestra.run.smithi101.stdout: "get_or_fail_fail": 0, 2023-07-18T03:39:37.434 INFO:teuthology.orchestra.run.smithi101.stdout: "get_or_fail_success": 567, 2023-07-18T03:39:37.434 INFO:teuthology.orchestra.run.smithi101.stdout: "take": 0, 2023-07-18T03:39:37.435 INFO:teuthology.orchestra.run.smithi101.stdout: "take_sum": 0, 2023-07-18T03:39:37.435 INFO:teuthology.orchestra.run.smithi101.stdout: "put": 567, 2023-07-18T03:39:37.435 INFO:teuthology.orchestra.run.smithi101.stdout: "put_sum": 1440615, 2023-07-18T03:39:37.435 INFO:teuthology.orchestra.run.smithi101.stdout: "wait": { 2023-07-18T03:39:37.435 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-18T03:39:37.436 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-18T03:39:37.436 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-18T03:39:37.436 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:37.436 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:37.437 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:39:37.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:39:37 smithi101 ceph-mon[126379]: pgmap v1016: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:39:37.796 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:39:37 smithi155 ceph-mon[128885]: pgmap v1016: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:39:37.931 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2023-07-18T03:39:37.932 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2023-07-18T03:39:37.932 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2023-07-18T03:39:37.932 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2023-07-18T03:39:37.933 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2023-07-18T03:39:37.933 DEBUG:tasks.check_counter:Getting stats from j 2023-07-18T03:39:37.934 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph daemon mds.j perf dump 2023-07-18T03:39:37.951 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:39:37 smithi179 ceph-mon[129229]: pgmap v1016: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:39:38.656 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:39:39.612 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:39:39 smithi101 ceph-mon[126379]: pgmap v1017: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:39:39.796 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:39:39 smithi155 ceph-mon[128885]: pgmap v1017: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:39:39.951 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:39:39 smithi179 ceph-mon[129229]: pgmap v1017: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:39:40.349 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-18T03:39:40.350 INFO:teuthology.orchestra.run.smithi101.stdout: "AsyncMessenger::Worker-0": { 2023-07-18T03:39:40.350 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_messages": 4, 2023-07-18T03:39:40.350 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages": 4, 2023-07-18T03:39:40.350 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_bytes": 2944, 2023-07-18T03:39:40.350 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_bytes": 448, 2023-07-18T03:39:40.351 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_created_connections": 4, 2023-07-18T03:39:40.351 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_active_connections": 0, 2023-07-18T03:39:40.351 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_total_time": 0.003473563, 2023-07-18T03:39:40.351 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_send_time": 0.000058821, 2023-07-18T03:39:40.351 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_recv_time": 0.000927687, 2023-07-18T03:39:40.352 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-18T03:39:40.352 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:39:40.352 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 3, 2023-07-18T03:39:40.352 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.001129361, 2023-07-18T03:39:40.352 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000376453 2023-07-18T03:39:40.353 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:40.353 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:39:40.353 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-18T03:39:40.353 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-18T03:39:40.353 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-18T03:39:40.354 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:40.354 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-07-18T03:39:40.354 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_encrypted_bytes": 448 2023-07-18T03:39:40.354 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:40.354 INFO:teuthology.orchestra.run.smithi101.stdout: "AsyncMessenger::Worker-1": { 2023-07-18T03:39:40.355 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_messages": 568, 2023-07-18T03:39:40.355 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages": 558, 2023-07-18T03:39:40.355 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_bytes": 1521552, 2023-07-18T03:39:40.355 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_bytes": 320528, 2023-07-18T03:39:40.355 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_created_connections": 2, 2023-07-18T03:39:40.356 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_active_connections": 1, 2023-07-18T03:39:40.356 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_total_time": 0.106349756, 2023-07-18T03:39:40.356 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_send_time": 0.046608246, 2023-07-18T03:39:40.356 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_recv_time": 0.047507400, 2023-07-18T03:39:40.356 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_fast_dispatch_time": 0.012953873, 2023-07-18T03:39:40.357 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:39:40.357 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 557, 2023-07-18T03:39:40.357 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.041825754, 2023-07-18T03:39:40.357 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000075091 2023-07-18T03:39:40.357 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:40.358 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:39:40.358 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-18T03:39:40.358 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-18T03:39:40.358 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-18T03:39:40.358 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:40.359 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_encrypted_bytes": 1521552, 2023-07-18T03:39:40.359 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_encrypted_bytes": 320528 2023-07-18T03:39:40.359 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:40.359 INFO:teuthology.orchestra.run.smithi101.stdout: "AsyncMessenger::Worker-2": { 2023-07-18T03:39:40.359 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_messages": 4, 2023-07-18T03:39:40.359 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages": 428, 2023-07-18T03:39:40.360 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_bytes": 640, 2023-07-18T03:39:40.360 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_bytes": 378480, 2023-07-18T03:39:40.360 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_created_connections": 7, 2023-07-18T03:39:40.360 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_active_connections": 3, 2023-07-18T03:39:40.360 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_total_time": 0.041376325, 2023-07-18T03:39:40.361 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_send_time": 0.030698742, 2023-07-18T03:39:40.361 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_recv_time": 0.003799184, 2023-07-18T03:39:40.361 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-18T03:39:40.361 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:39:40.361 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 425, 2023-07-18T03:39:40.362 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.047870817, 2023-07-18T03:39:40.362 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000112637 2023-07-18T03:39:40.362 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:40.362 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:39:40.362 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-18T03:39:40.363 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-18T03:39:40.363 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-18T03:39:40.363 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:40.363 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_encrypted_bytes": 640, 2023-07-18T03:39:40.363 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_encrypted_bytes": 378480 2023-07-18T03:39:40.364 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:40.364 INFO:teuthology.orchestra.run.smithi101.stdout: "cct": { 2023-07-18T03:39:40.364 INFO:teuthology.orchestra.run.smithi101.stdout: "total_workers": 0, 2023-07-18T03:39:40.364 INFO:teuthology.orchestra.run.smithi101.stdout: "unhealthy_workers": 0 2023-07-18T03:39:40.364 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:40.365 INFO:teuthology.orchestra.run.smithi101.stdout: "mempool": { 2023-07-18T03:39:40.365 INFO:teuthology.orchestra.run.smithi101.stdout: "bloom_filter_bytes": 0, 2023-07-18T03:39:40.365 INFO:teuthology.orchestra.run.smithi101.stdout: "bloom_filter_items": 0, 2023-07-18T03:39:40.365 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_alloc_bytes": 0, 2023-07-18T03:39:40.365 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_alloc_items": 0, 2023-07-18T03:39:40.366 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_data_bytes": 0, 2023-07-18T03:39:40.366 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_data_items": 0, 2023-07-18T03:39:40.366 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-18T03:39:40.366 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_onode_items": 0, 2023-07-18T03:39:40.366 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-18T03:39:40.367 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_meta_items": 0, 2023-07-18T03:39:40.367 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_other_bytes": 0, 2023-07-18T03:39:40.367 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_other_items": 0, 2023-07-18T03:39:40.367 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-18T03:39:40.367 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_buffer_items": 0, 2023-07-18T03:39:40.367 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_extent_bytes": 0, 2023-07-18T03:39:40.368 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_extent_items": 0, 2023-07-18T03:39:40.368 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_blob_bytes": 0, 2023-07-18T03:39:40.368 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_blob_items": 0, 2023-07-18T03:39:40.368 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-18T03:39:40.368 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_shared_blob_items": 0, 2023-07-18T03:39:40.369 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-18T03:39:40.369 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_inline_bl_items": 0, 2023-07-18T03:39:40.369 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_fsck_bytes": 0, 2023-07-18T03:39:40.369 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_fsck_items": 0, 2023-07-18T03:39:40.369 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_txc_bytes": 0, 2023-07-18T03:39:40.370 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_txc_items": 0, 2023-07-18T03:39:40.370 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-18T03:39:40.370 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_writing_deferred_items": 0, 2023-07-18T03:39:40.370 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_writing_bytes": 0, 2023-07-18T03:39:40.370 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_writing_items": 0, 2023-07-18T03:39:40.371 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_bytes": 0, 2023-07-18T03:39:40.371 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_items": 0, 2023-07-18T03:39:40.371 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_file_reader_bytes": 0, 2023-07-18T03:39:40.371 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_file_reader_items": 0, 2023-07-18T03:39:40.371 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_file_writer_bytes": 0, 2023-07-18T03:39:40.372 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_file_writer_items": 0, 2023-07-18T03:39:40.372 INFO:teuthology.orchestra.run.smithi101.stdout: "buffer_anon_bytes": 149001, 2023-07-18T03:39:40.372 INFO:teuthology.orchestra.run.smithi101.stdout: "buffer_anon_items": 30, 2023-07-18T03:39:40.372 INFO:teuthology.orchestra.run.smithi101.stdout: "buffer_meta_bytes": 0, 2023-07-18T03:39:40.372 INFO:teuthology.orchestra.run.smithi101.stdout: "buffer_meta_items": 0, 2023-07-18T03:39:40.373 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_bytes": 0, 2023-07-18T03:39:40.373 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_items": 0, 2023-07-18T03:39:40.373 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_mapbl_bytes": 0, 2023-07-18T03:39:40.373 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_mapbl_items": 0, 2023-07-18T03:39:40.373 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_pglog_bytes": 0, 2023-07-18T03:39:40.374 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_pglog_items": 0, 2023-07-18T03:39:40.374 INFO:teuthology.orchestra.run.smithi101.stdout: "osdmap_bytes": 0, 2023-07-18T03:39:40.374 INFO:teuthology.orchestra.run.smithi101.stdout: "osdmap_items": 0, 2023-07-18T03:39:40.374 INFO:teuthology.orchestra.run.smithi101.stdout: "osdmap_mapping_bytes": 0, 2023-07-18T03:39:40.374 INFO:teuthology.orchestra.run.smithi101.stdout: "osdmap_mapping_items": 0, 2023-07-18T03:39:40.375 INFO:teuthology.orchestra.run.smithi101.stdout: "pgmap_bytes": 0, 2023-07-18T03:39:40.375 INFO:teuthology.orchestra.run.smithi101.stdout: "pgmap_items": 0, 2023-07-18T03:39:40.375 INFO:teuthology.orchestra.run.smithi101.stdout: "mds_co_bytes": 928, 2023-07-18T03:39:40.375 INFO:teuthology.orchestra.run.smithi101.stdout: "mds_co_items": 2, 2023-07-18T03:39:40.375 INFO:teuthology.orchestra.run.smithi101.stdout: "unittest_1_bytes": 0, 2023-07-18T03:39:40.376 INFO:teuthology.orchestra.run.smithi101.stdout: "unittest_1_items": 0, 2023-07-18T03:39:40.376 INFO:teuthology.orchestra.run.smithi101.stdout: "unittest_2_bytes": 0, 2023-07-18T03:39:40.376 INFO:teuthology.orchestra.run.smithi101.stdout: "unittest_2_items": 0 2023-07-18T03:39:40.376 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-18T03:39:40.376 INFO:teuthology.orchestra.run.smithi101.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-18T03:39:40.377 INFO:teuthology.orchestra.run.smithi101.stdout: "val": 0, 2023-07-18T03:39:40.377 INFO:teuthology.orchestra.run.smithi101.stdout: "max": 104857600, 2023-07-18T03:39:40.377 INFO:teuthology.orchestra.run.smithi101.stdout: "get_started": 0, 2023-07-18T03:39:40.377 INFO:teuthology.orchestra.run.smithi101.stdout: "get": 572, 2023-07-18T03:39:40.377 INFO:teuthology.orchestra.run.smithi101.stdout: "get_sum": 1441265, 2023-07-18T03:39:40.377 INFO:teuthology.orchestra.run.smithi101.stdout: "get_or_fail_fail": 0, 2023-07-18T03:39:40.378 INFO:teuthology.orchestra.run.smithi101.stdout: "get_or_fail_success": 572, 2023-07-18T03:39:40.378 INFO:teuthology.orchestra.run.smithi101.stdout: "take": 0, 2023-07-18T03:39:40.378 INFO:teuthology.orchestra.run.smithi101.stdout: "take_sum": 0, 2023-07-18T03:39:40.378 INFO:teuthology.orchestra.run.smithi101.stdout: "put": 572, 2023-07-18T03:39:40.378 INFO:teuthology.orchestra.run.smithi101.stdout: "put_sum": 1441265, 2023-07-18T03:39:40.379 INFO:teuthology.orchestra.run.smithi101.stdout: "wait": { 2023-07-18T03:39:40.379 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-18T03:39:40.379 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-18T03:39:40.379 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-18T03:39:40.379 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:40.380 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-18T03:39:40.380 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-18T03:39:40.842 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2023-07-18T03:39:40.842 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2023-07-18T03:39:40.842 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2023-07-18T03:39:40.843 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2023-07-18T03:39:40.843 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2023-07-18T03:39:40.843 DEBUG:tasks.check_counter:Getting stats from b 2023-07-18T03:39:40.844 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph daemon mds.b perf dump 2023-07-18T03:39:41.639 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.b/config 2023-07-18T03:39:41.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:39:41 smithi101 ceph-mon[126379]: pgmap v1018: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:39:41.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:39:41 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:39:41.796 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:39:41 smithi155 ceph-mon[128885]: pgmap v1018: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:39:41.796 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:39:41 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:39:41.951 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:39:41 smithi179 ceph-mon[129229]: pgmap v1018: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:39:41.951 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:39:41 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:39:42.912 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2023-07-18T03:39:42.912 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-0": { 2023-07-18T03:39:42.912 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 427658, 2023-07-18T03:39:42.913 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 548669, 2023-07-18T03:39:42.913 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 150837840, 2023-07-18T03:39:42.913 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 502024258, 2023-07-18T03:39:42.913 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 68, 2023-07-18T03:39:42.913 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 3, 2023-07-18T03:39:42.914 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 24.485236054, 2023-07-18T03:39:42.914 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 10.632430106, 2023-07-18T03:39:42.914 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 16.972012920, 2023-07-18T03:39:42.914 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.883375256, 2023-07-18T03:39:42.914 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:39:42.915 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 548667, 2023-07-18T03:39:42.915 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 44.439611404, 2023-07-18T03:39:42.915 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000080995 2023-07-18T03:39:42.915 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.915 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:39:42.916 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 813992, 2023-07-18T03:39:42.916 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.120519359, 2023-07-18T03:39:42.916 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000148 2023-07-18T03:39:42.916 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.916 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 81854752, 2023-07-18T03:39:42.917 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 314032256 2023-07-18T03:39:42.917 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.917 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-1": { 2023-07-18T03:39:42.917 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 31905, 2023-07-18T03:39:42.917 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 32326, 2023-07-18T03:39:42.918 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 8037650, 2023-07-18T03:39:42.918 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 138049689, 2023-07-18T03:39:42.918 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 22, 2023-07-18T03:39:42.918 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 5, 2023-07-18T03:39:42.918 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 2.262703398, 2023-07-18T03:39:42.918 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.719424980, 2023-07-18T03:39:42.919 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.487215292, 2023-07-18T03:39:42.919 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.913378436, 2023-07-18T03:39:42.919 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:39:42.919 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 32323, 2023-07-18T03:39:42.919 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 4.172807763, 2023-07-18T03:39:42.920 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000129097 2023-07-18T03:39:42.920 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.920 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:39:42.920 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 30, 2023-07-18T03:39:42.920 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000004550, 2023-07-18T03:39:42.921 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000151 2023-07-18T03:39:42.921 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.921 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 3264, 2023-07-18T03:39:42.921 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 977184 2023-07-18T03:39:42.921 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.922 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-2": { 2023-07-18T03:39:42.922 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 7413405, 2023-07-18T03:39:42.922 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 7413520, 2023-07-18T03:39:42.922 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 1215196890, 2023-07-18T03:39:42.922 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 1434354844, 2023-07-18T03:39:42.923 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 7, 2023-07-18T03:39:42.923 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 6, 2023-07-18T03:39:42.923 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 182.487102846, 2023-07-18T03:39:42.923 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 70.925762981, 2023-07-18T03:39:42.923 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 122.466356476, 2023-07-18T03:39:42.924 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 1.220305453, 2023-07-18T03:39:42.924 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:39:42.924 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 7413520, 2023-07-18T03:39:42.924 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 65.699201310, 2023-07-18T03:39:42.924 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000008862 2023-07-18T03:39:42.925 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.925 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:39:42.925 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 14741269, 2023-07-18T03:39:42.925 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.996594038, 2023-07-18T03:39:42.925 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000067 2023-07-18T03:39:42.926 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.926 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 1727072, 2023-07-18T03:39:42.926 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 366944 2023-07-18T03:39:42.926 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.926 INFO:teuthology.orchestra.run.smithi155.stdout: "cct": { 2023-07-18T03:39:42.926 INFO:teuthology.orchestra.run.smithi155.stdout: "total_workers": 1, 2023-07-18T03:39:42.927 INFO:teuthology.orchestra.run.smithi155.stdout: "unhealthy_workers": 0 2023-07-18T03:39:42.927 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.927 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-MDSRank": { 2023-07-18T03:39:42.928 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2023-07-18T03:39:42.928 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2023-07-18T03:39:42.928 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 81121, 2023-07-18T03:39:42.928 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 82.634069846, 2023-07-18T03:39:42.929 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.001018652 2023-07-18T03:39:42.929 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-18T03:39:42.929 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.929 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-PurgeQueue": { 2023-07-18T03:39:42.929 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2023-07-18T03:39:42.930 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2023-07-18T03:39:42.930 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 44629, 2023-07-18T03:39:42.930 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 7.622764350, 2023-07-18T03:39:42.930 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000170802 2023-07-18T03:39:42.930 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-18T03:39:42.931 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.931 INFO:teuthology.orchestra.run.smithi155.stdout: "mds": { 2023-07-18T03:39:42.931 INFO:teuthology.orchestra.run.smithi155.stdout: "request": 160995, 2023-07-18T03:39:42.931 INFO:teuthology.orchestra.run.smithi155.stdout: "reply": 154149, 2023-07-18T03:39:42.931 INFO:teuthology.orchestra.run.smithi155.stdout: "reply_latency": { 2023-07-18T03:39:42.931 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 154149, 2023-07-18T03:39:42.932 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 530.545560832, 2023-07-18T03:39:42.932 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.003441771 2023-07-18T03:39:42.932 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.932 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_reply": 0, 2023-07-18T03:39:42.932 INFO:teuthology.orchestra.run.smithi155.stdout: "forward": 6846, 2023-07-18T03:39:42.933 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_complete": 7, 2023-07-18T03:39:42.933 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_keys": 0, 2023-07-18T03:39:42.933 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_commit": 113, 2023-07-18T03:39:42.933 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_split": 11, 2023-07-18T03:39:42.933 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_merge": 8, 2023-07-18T03:39:42.934 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes": 2095, 2023-07-18T03:39:42.934 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_top": 579, 2023-07-18T03:39:42.934 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_bottom": 1417, 2023-07-18T03:39:42.934 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pin_tail": 99, 2023-07-18T03:39:42.934 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pinned": 1150, 2023-07-18T03:39:42.935 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_expired": 798, 2023-07-18T03:39:42.935 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_with_caps": 203, 2023-07-18T03:39:42.935 INFO:teuthology.orchestra.run.smithi155.stdout: "caps": 203, 2023-07-18T03:39:42.935 INFO:teuthology.orchestra.run.smithi155.stdout: "subtrees": 26, 2023-07-18T03:39:42.935 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse": 274944, 2023-07-18T03:39:42.936 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_hit": 156314, 2023-07-18T03:39:42.936 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_forward": 6831, 2023-07-18T03:39:42.936 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_discover": 187, 2023-07-18T03:39:42.936 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_dir_fetch": 0, 2023-07-18T03:39:42.936 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_remote_ino": 0, 2023-07-18T03:39:42.937 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_lock": 3881, 2023-07-18T03:39:42.937 INFO:teuthology.orchestra.run.smithi155.stdout: "load_cent": 1000, 2023-07-18T03:39:42.937 INFO:teuthology.orchestra.run.smithi155.stdout: "q": 1, 2023-07-18T03:39:42.937 INFO:teuthology.orchestra.run.smithi155.stdout: "exported": 19, 2023-07-18T03:39:42.937 INFO:teuthology.orchestra.run.smithi155.stdout: "exported_inodes": 414, 2023-07-18T03:39:42.938 INFO:teuthology.orchestra.run.smithi155.stdout: "imported": 11, 2023-07-18T03:39:42.938 INFO:teuthology.orchestra.run.smithi155.stdout: "imported_inodes": 639, 2023-07-18T03:39:42.938 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_dir_fetch": 0, 2023-07-18T03:39:42.938 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_backtrace_fetch": 2322, 2023-07-18T03:39:42.938 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_peer_discover": 0, 2023-07-18T03:39:42.939 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rfiles": 679, 2023-07-18T03:39:42.939 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rbytes": 221129114, 2023-07-18T03:39:42.939 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rsnaps": 6, 2023-07-18T03:39:42.939 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_fetch": 2322, 2023-07-18T03:39:42.939 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_set_tag": 0, 2023-07-18T03:39:42.940 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_repaired": 0, 2023-07-18T03:39:42.940 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_inotable_repaired": 0, 2023-07-18T03:39:42.940 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_inodes": 210, 2023-07-18T03:39:42.940 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_base_inodes": 0, 2023-07-18T03:39:42.940 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dirfrag_rstats": 210, 2023-07-18T03:39:42.941 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_file_inodes": 2112, 2023-07-18T03:39:42.941 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_inode_file_caps": 8, 2023-07-18T03:39:42.941 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_revoke": 9476, 2023-07-18T03:39:42.941 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_grant": 114085, 2023-07-18T03:39:42.941 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_trunc": 0, 2023-07-18T03:39:42.941 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flushsnap_ack": 1352, 2023-07-18T03:39:42.942 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-18T03:39:42.942 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps": 48034, 2023-07-18T03:39:42.942 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps_dirty": 27277, 2023-07-18T03:39:42.942 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_cap_release": 5801, 2023-07-18T03:39:42.942 INFO:teuthology.orchestra.run.smithi155.stdout: "process_request_cap_release": 119597 2023-07-18T03:39:42.943 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.943 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_cache": { 2023-07-18T03:39:42.943 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update": 19, 2023-07-18T03:39:42.943 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update_receipt": 113, 2023-07-18T03:39:42.943 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_try_discover": 63, 2023-07-18T03:39:42.944 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_send_discover": 304, 2023-07-18T03:39:42.944 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_handle_discover": 158, 2023-07-18T03:39:42.944 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays": 0, 2023-07-18T03:39:42.944 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_delayed": 0, 2023-07-18T03:39:42.944 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_enqueuing": 0, 2023-07-18T03:39:42.945 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_created": 26100, 2023-07-18T03:39:42.945 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_enqueued": 26100, 2023-07-18T03:39:42.945 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_reintegrated": 0, 2023-07-18T03:39:42.945 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_migrated": 0, 2023-07-18T03:39:42.945 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_processing": 0, 2023-07-18T03:39:42.946 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_enqueued": 0, 2023-07-18T03:39:42.946 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_prioritized": 0, 2023-07-18T03:39:42.946 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_started": 0, 2023-07-18T03:39:42.946 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_completed": 0, 2023-07-18T03:39:42.946 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_enqueue_scrub": 0, 2023-07-18T03:39:42.947 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_exportdir": 23, 2023-07-18T03:39:42.947 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_flush": 0, 2023-07-18T03:39:42.947 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragmentdir": 19, 2023-07-18T03:39:42.947 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragstats": 0, 2023-07-18T03:39:42.947 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_inodestats": 0 2023-07-18T03:39:42.948 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.948 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_log": { 2023-07-18T03:39:42.948 INFO:teuthology.orchestra.run.smithi155.stdout: "evadd": 143900, 2023-07-18T03:39:42.948 INFO:teuthology.orchestra.run.smithi155.stdout: "evex": 45406, 2023-07-18T03:39:42.948 INFO:teuthology.orchestra.run.smithi155.stdout: "evtrm": 45406, 2023-07-18T03:39:42.949 INFO:teuthology.orchestra.run.smithi155.stdout: "ev": 98494, 2023-07-18T03:39:42.949 INFO:teuthology.orchestra.run.smithi155.stdout: "evexg": 0, 2023-07-18T03:39:42.949 INFO:teuthology.orchestra.run.smithi155.stdout: "evexd": 1893, 2023-07-18T03:39:42.949 INFO:teuthology.orchestra.run.smithi155.stdout: "segadd": 184, 2023-07-18T03:39:42.949 INFO:teuthology.orchestra.run.smithi155.stdout: "segex": 56, 2023-07-18T03:39:42.949 INFO:teuthology.orchestra.run.smithi155.stdout: "segtrm": 56, 2023-07-18T03:39:42.950 INFO:teuthology.orchestra.run.smithi155.stdout: "seg": 130, 2023-07-18T03:39:42.950 INFO:teuthology.orchestra.run.smithi155.stdout: "segexg": 0, 2023-07-18T03:39:42.950 INFO:teuthology.orchestra.run.smithi155.stdout: "segexd": 2, 2023-07-18T03:39:42.950 INFO:teuthology.orchestra.run.smithi155.stdout: "expos": 83886779, 2023-07-18T03:39:42.950 INFO:teuthology.orchestra.run.smithi155.stdout: "wrpos": 533832275, 2023-07-18T03:39:42.951 INFO:teuthology.orchestra.run.smithi155.stdout: "rdpos": 0, 2023-07-18T03:39:42.951 INFO:teuthology.orchestra.run.smithi155.stdout: "jlat": { 2023-07-18T03:39:42.951 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 42896, 2023-07-18T03:39:42.951 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 245.912942499, 2023-07-18T03:39:42.951 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.005732770 2023-07-18T03:39:42.952 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.952 INFO:teuthology.orchestra.run.smithi155.stdout: "replayed": 0 2023-07-18T03:39:42.952 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.952 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_mem": { 2023-07-18T03:39:42.952 INFO:teuthology.orchestra.run.smithi155.stdout: "ino": 1234, 2023-07-18T03:39:42.953 INFO:teuthology.orchestra.run.smithi155.stdout: "ino+": 27919, 2023-07-18T03:39:42.953 INFO:teuthology.orchestra.run.smithi155.stdout: "ino-": 26685, 2023-07-18T03:39:42.953 INFO:teuthology.orchestra.run.smithi155.stdout: "dir": 186, 2023-07-18T03:39:42.953 INFO:teuthology.orchestra.run.smithi155.stdout: "dir+": 442, 2023-07-18T03:39:42.953 INFO:teuthology.orchestra.run.smithi155.stdout: "dir-": 256, 2023-07-18T03:39:42.954 INFO:teuthology.orchestra.run.smithi155.stdout: "dn": 2095, 2023-07-18T03:39:42.954 INFO:teuthology.orchestra.run.smithi155.stdout: "dn+": 47444, 2023-07-18T03:39:42.954 INFO:teuthology.orchestra.run.smithi155.stdout: "dn-": 45349, 2023-07-18T03:39:42.954 INFO:teuthology.orchestra.run.smithi155.stdout: "cap": 203, 2023-07-18T03:39:42.954 INFO:teuthology.orchestra.run.smithi155.stdout: "cap+": 28244, 2023-07-18T03:39:42.955 INFO:teuthology.orchestra.run.smithi155.stdout: "cap-": 28041, 2023-07-18T03:39:42.955 INFO:teuthology.orchestra.run.smithi155.stdout: "rss": 77288, 2023-07-18T03:39:42.955 INFO:teuthology.orchestra.run.smithi155.stdout: "heap": 198964 2023-07-18T03:39:42.955 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.955 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_server": { 2023-07-18T03:39:42.956 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_client_request": 244686, 2023-07-18T03:39:42.956 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_server_request": 3271, 2023-07-18T03:39:42.956 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_request": 160995, 2023-07-18T03:39:42.956 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_session": 4825, 2023-07-18T03:39:42.956 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_peer_request": 19154, 2023-07-18T03:39:42.957 INFO:teuthology.orchestra.run.smithi155.stdout: "req_create_latency": { 2023-07-18T03:39:42.957 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 26157, 2023-07-18T03:39:42.957 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 81.509181737, 2023-07-18T03:39:42.957 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.003116151 2023-07-18T03:39:42.957 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.958 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getattr_latency": { 2023-07-18T03:39:42.958 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 2422, 2023-07-18T03:39:42.958 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 39.583984277, 2023-07-18T03:39:42.958 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.016343511 2023-07-18T03:39:42.958 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.959 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getfilelock_latency": { 2023-07-18T03:39:42.959 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:42.959 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:42.959 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:42.959 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.959 INFO:teuthology.orchestra.run.smithi155.stdout: "req_link_latency": { 2023-07-18T03:39:42.960 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:42.960 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:42.960 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:42.960 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.960 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookup_latency": { 2023-07-18T03:39:42.961 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 39137, 2023-07-18T03:39:42.961 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 76.759459748, 2023-07-18T03:39:42.961 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.001961301 2023-07-18T03:39:42.961 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.961 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookuphash_latency": { 2023-07-18T03:39:42.962 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:42.962 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:42.962 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:42.962 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.962 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupino_latency": { 2023-07-18T03:39:42.963 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 1, 2023-07-18T03:39:42.963 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000094446, 2023-07-18T03:39:42.963 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000094446 2023-07-18T03:39:42.963 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.963 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupname_latency": { 2023-07-18T03:39:42.964 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:42.964 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:42.964 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:42.964 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.965 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupparent_latency": { 2023-07-18T03:39:42.965 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:42.965 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:42.965 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:42.965 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.966 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupsnap_latency": { 2023-07-18T03:39:42.966 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:42.966 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:42.966 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:42.966 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.967 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lssnap_latency": { 2023-07-18T03:39:42.967 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:42.967 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:42.967 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:42.967 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.968 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mkdir_latency": { 2023-07-18T03:39:42.968 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:42.968 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:42.968 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:42.968 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.969 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mknod_latency": { 2023-07-18T03:39:42.969 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:42.969 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:42.969 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:42.970 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.970 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mksnap_latency": { 2023-07-18T03:39:42.970 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:42.970 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:42.970 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:42.971 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.971 INFO:teuthology.orchestra.run.smithi155.stdout: "req_open_latency": { 2023-07-18T03:39:42.971 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:42.971 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:42.971 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:42.972 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.972 INFO:teuthology.orchestra.run.smithi155.stdout: "req_readdir_latency": { 2023-07-18T03:39:42.972 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 52829, 2023-07-18T03:39:42.972 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 239.542142473, 2023-07-18T03:39:42.972 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.004534292 2023-07-18T03:39:42.973 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.973 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rename_latency": { 2023-07-18T03:39:42.973 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 6485, 2023-07-18T03:39:42.973 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 17.226453851, 2023-07-18T03:39:42.973 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.002656353 2023-07-18T03:39:42.974 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.974 INFO:teuthology.orchestra.run.smithi155.stdout: "req_renamesnap_latency": { 2023-07-18T03:39:42.974 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:42.974 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:42.974 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:42.974 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.975 INFO:teuthology.orchestra.run.smithi155.stdout: "req_snapdiff_latency": { 2023-07-18T03:39:42.975 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:42.975 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:42.975 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:42.975 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.976 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmdir_latency": { 2023-07-18T03:39:42.976 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:42.976 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:42.976 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:42.976 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.977 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmsnap_latency": { 2023-07-18T03:39:42.977 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:42.977 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:42.977 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:42.977 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.978 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmxattr_latency": { 2023-07-18T03:39:42.978 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:42.978 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:42.978 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:42.978 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.979 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setattr_latency": { 2023-07-18T03:39:42.979 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 38, 2023-07-18T03:39:42.979 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.492709859, 2023-07-18T03:39:42.979 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.012966048 2023-07-18T03:39:42.979 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.980 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setdirlayout_latency": { 2023-07-18T03:39:42.980 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:42.980 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:42.980 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:42.981 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.981 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setfilelock_latency": { 2023-07-18T03:39:42.981 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 980, 2023-07-18T03:39:42.981 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.868574895, 2023-07-18T03:39:42.982 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000886300 2023-07-18T03:39:42.982 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.983 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setlayout_latency": { 2023-07-18T03:39:42.983 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:42.984 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:42.984 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:42.984 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.984 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setxattr_latency": { 2023-07-18T03:39:42.984 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:42.985 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:42.985 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:42.985 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.985 INFO:teuthology.orchestra.run.smithi155.stdout: "req_symlink_latency": { 2023-07-18T03:39:42.985 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:42.986 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:42.986 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:42.986 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.986 INFO:teuthology.orchestra.run.smithi155.stdout: "req_unlink_latency": { 2023-07-18T03:39:42.987 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 26100, 2023-07-18T03:39:42.987 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 74.562959546, 2023-07-18T03:39:42.987 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.002856818 2023-07-18T03:39:42.987 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.987 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_revoke_eviction": 0, 2023-07-18T03:39:42.988 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_acquisition_throttle": 0, 2023-07-18T03:39:42.988 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getvxattr_latency": { 2023-07-18T03:39:42.988 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:42.988 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:42.989 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:42.989 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-18T03:39:42.989 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.989 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_sessions": { 2023-07-18T03:39:42.989 INFO:teuthology.orchestra.run.smithi155.stdout: "session_count": 1, 2023-07-18T03:39:42.990 INFO:teuthology.orchestra.run.smithi155.stdout: "session_add": 3, 2023-07-18T03:39:42.990 INFO:teuthology.orchestra.run.smithi155.stdout: "session_remove": 2, 2023-07-18T03:39:42.990 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_open": 1, 2023-07-18T03:39:42.990 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_stale": 0, 2023-07-18T03:39:42.990 INFO:teuthology.orchestra.run.smithi155.stdout: "total_load": 30580, 2023-07-18T03:39:42.991 INFO:teuthology.orchestra.run.smithi155.stdout: "average_load": 30580, 2023-07-18T03:39:42.991 INFO:teuthology.orchestra.run.smithi155.stdout: "avg_session_uptime": 1923 2023-07-18T03:39:42.991 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:42.991 INFO:teuthology.orchestra.run.smithi155.stdout: "mempool": { 2023-07-18T03:39:42.991 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_bytes": 0, 2023-07-18T03:39:42.992 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_items": 0, 2023-07-18T03:39:42.992 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_bytes": 0, 2023-07-18T03:39:42.992 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_items": 0, 2023-07-18T03:39:42.993 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_bytes": 0, 2023-07-18T03:39:42.993 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_items": 0, 2023-07-18T03:39:42.993 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-18T03:39:42.993 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_items": 0, 2023-07-18T03:39:42.993 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-18T03:39:42.994 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_items": 0, 2023-07-18T03:39:42.994 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_bytes": 0, 2023-07-18T03:39:42.994 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_items": 0, 2023-07-18T03:39:42.994 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-18T03:39:42.994 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_items": 0, 2023-07-18T03:39:42.995 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_bytes": 0, 2023-07-18T03:39:42.995 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_items": 0, 2023-07-18T03:39:42.996 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_bytes": 0, 2023-07-18T03:39:42.996 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_items": 0, 2023-07-18T03:39:42.996 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-18T03:39:42.996 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_items": 0, 2023-07-18T03:39:42.996 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-18T03:39:42.997 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_items": 0, 2023-07-18T03:39:42.997 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_bytes": 0, 2023-07-18T03:39:42.997 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_items": 0, 2023-07-18T03:39:42.997 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_bytes": 0, 2023-07-18T03:39:42.997 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_items": 0, 2023-07-18T03:39:42.998 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-18T03:39:42.998 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_items": 0, 2023-07-18T03:39:42.998 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_bytes": 0, 2023-07-18T03:39:42.998 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_items": 0, 2023-07-18T03:39:42.998 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_bytes": 0, 2023-07-18T03:39:42.999 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_items": 0, 2023-07-18T03:39:42.999 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_bytes": 0, 2023-07-18T03:39:42.999 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_items": 0, 2023-07-18T03:39:42.999 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_bytes": 0, 2023-07-18T03:39:42.999 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_items": 0, 2023-07-18T03:39:43.000 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_bytes": 335907, 2023-07-18T03:39:43.000 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_items": 742, 2023-07-18T03:39:43.000 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_bytes": 176, 2023-07-18T03:39:43.000 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_items": 2, 2023-07-18T03:39:43.001 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_bytes": 0, 2023-07-18T03:39:43.001 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_items": 0, 2023-07-18T03:39:43.001 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_bytes": 0, 2023-07-18T03:39:43.001 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_items": 0, 2023-07-18T03:39:43.001 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_bytes": 0, 2023-07-18T03:39:43.002 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_items": 0, 2023-07-18T03:39:43.002 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_bytes": 9616, 2023-07-18T03:39:43.002 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_items": 185, 2023-07-18T03:39:43.003 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_bytes": 0, 2023-07-18T03:39:43.003 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_items": 0, 2023-07-18T03:39:43.003 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_bytes": 0, 2023-07-18T03:39:43.004 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_items": 0, 2023-07-18T03:39:43.004 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_bytes": 5142071, 2023-07-18T03:39:43.004 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_items": 45220, 2023-07-18T03:39:43.004 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_bytes": 0, 2023-07-18T03:39:43.005 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_items": 0, 2023-07-18T03:39:43.005 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_bytes": 0, 2023-07-18T03:39:43.005 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_items": 0 2023-07-18T03:39:43.005 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:43.005 INFO:teuthology.orchestra.run.smithi155.stdout: "objecter": { 2023-07-18T03:39:43.006 INFO:teuthology.orchestra.run.smithi155.stdout: "op_active": 0, 2023-07-18T03:39:43.006 INFO:teuthology.orchestra.run.smithi155.stdout: "op_laggy": 0, 2023-07-18T03:39:43.006 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send": 100314, 2023-07-18T03:39:43.006 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send_bytes": 535443563, 2023-07-18T03:39:43.006 INFO:teuthology.orchestra.run.smithi155.stdout: "op_resend": 28, 2023-07-18T03:39:43.007 INFO:teuthology.orchestra.run.smithi155.stdout: "op_reply": 100314, 2023-07-18T03:39:43.007 INFO:teuthology.orchestra.run.smithi155.stdout: "op_latency": { 2023-07-18T03:39:43.007 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 100314, 2023-07-18T03:39:43.008 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 864.064268550, 2023-07-18T03:39:43.008 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.008613595 2023-07-18T03:39:43.009 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:43.009 INFO:teuthology.orchestra.run.smithi155.stdout: "op_inflight": 0, 2023-07-18T03:39:43.010 INFO:teuthology.orchestra.run.smithi155.stdout: "oplen_avg": { 2023-07-18T03:39:43.010 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 100314, 2023-07-18T03:39:43.010 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 100867 2023-07-18T03:39:43.011 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:43.011 INFO:teuthology.orchestra.run.smithi155.stdout: "op": 100314, 2023-07-18T03:39:43.012 INFO:teuthology.orchestra.run.smithi155.stdout: "op_r": 3248, 2023-07-18T03:39:43.012 INFO:teuthology.orchestra.run.smithi155.stdout: "op_w": 97066, 2023-07-18T03:39:43.012 INFO:teuthology.orchestra.run.smithi155.stdout: "op_rmw": 0, 2023-07-18T03:39:43.013 INFO:teuthology.orchestra.run.smithi155.stdout: "op_pg": 0, 2023-07-18T03:39:43.013 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_stat": 0, 2023-07-18T03:39:43.014 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_create": 21, 2023-07-18T03:39:43.014 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_read": 919, 2023-07-18T03:39:43.015 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_write": 43815, 2023-07-18T03:39:43.015 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writefull": 509, 2023-07-18T03:39:43.015 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writesame": 0, 2023-07-18T03:39:43.016 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_append": 0, 2023-07-18T03:39:43.016 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_zero": 0, 2023-07-18T03:39:43.017 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_truncate": 11, 2023-07-18T03:39:43.017 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_delete": 52393, 2023-07-18T03:39:43.017 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_mapext": 0, 2023-07-18T03:39:43.018 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_sparse_read": 0, 2023-07-18T03:39:43.018 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_clonerange": 0, 2023-07-18T03:39:43.019 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_getxattr": 2322, 2023-07-18T03:39:43.019 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_setxattr": 42, 2023-07-18T03:39:43.020 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_cmpxattr": 0, 2023-07-18T03:39:43.020 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_rmxattr": 0, 2023-07-18T03:39:43.020 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_resetxattrs": 0, 2023-07-18T03:39:43.021 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_call": 0, 2023-07-18T03:39:43.021 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_watch": 0, 2023-07-18T03:39:43.022 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_notify": 0, 2023-07-18T03:39:43.022 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_src_cmpxattr": 0, 2023-07-18T03:39:43.023 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls": 0, 2023-07-18T03:39:43.023 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls_filter": 0, 2023-07-18T03:39:43.023 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_other": 0, 2023-07-18T03:39:43.024 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_active": 0, 2023-07-18T03:39:43.024 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_send": 0, 2023-07-18T03:39:43.025 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_resend": 0, 2023-07-18T03:39:43.025 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_ping": 0, 2023-07-18T03:39:43.026 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_active": 0, 2023-07-18T03:39:43.026 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_send": 0, 2023-07-18T03:39:43.026 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_resend": 0, 2023-07-18T03:39:43.027 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_active": 0, 2023-07-18T03:39:43.027 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_send": 0, 2023-07-18T03:39:43.028 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_resend": 0, 2023-07-18T03:39:43.028 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_active": 0, 2023-07-18T03:39:43.028 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_send": 0, 2023-07-18T03:39:43.029 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_resend": 0, 2023-07-18T03:39:43.029 INFO:teuthology.orchestra.run.smithi155.stdout: "command_active": 0, 2023-07-18T03:39:43.030 INFO:teuthology.orchestra.run.smithi155.stdout: "command_send": 0, 2023-07-18T03:39:43.030 INFO:teuthology.orchestra.run.smithi155.stdout: "command_resend": 0, 2023-07-18T03:39:43.031 INFO:teuthology.orchestra.run.smithi155.stdout: "map_epoch": 144, 2023-07-18T03:39:43.031 INFO:teuthology.orchestra.run.smithi155.stdout: "map_full": 0, 2023-07-18T03:39:43.031 INFO:teuthology.orchestra.run.smithi155.stdout: "map_inc": 55, 2023-07-18T03:39:43.031 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_sessions": 12, 2023-07-18T03:39:43.032 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_open": 12, 2023-07-18T03:39:43.032 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_close": 0, 2023-07-18T03:39:43.032 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_laggy": 0, 2023-07-18T03:39:43.033 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_wr": 610, 2023-07-18T03:39:43.033 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_rd": 14, 2023-07-18T03:39:43.033 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_del": 211 2023-07-18T03:39:43.033 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:43.034 INFO:teuthology.orchestra.run.smithi155.stdout: "oft": { 2023-07-18T03:39:43.034 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_objs": 1, 2023-07-18T03:39:43.034 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_kv_pairs": 0, 2023-07-18T03:39:43.034 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_updates": 0, 2023-07-18T03:39:43.035 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_removes": 0 2023-07-18T03:39:43.035 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:43.035 INFO:teuthology.orchestra.run.smithi155.stdout: "osd-slow-ops": { 2023-07-18T03:39:43.035 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_ops_count": 162672 2023-07-18T03:39:43.036 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:43.036 INFO:teuthology.orchestra.run.smithi155.stdout: "purge_queue": { 2023-07-18T03:39:43.036 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops": 0, 2023-07-18T03:39:43.037 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops_high_water": 16, 2023-07-18T03:39:43.037 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing": 0, 2023-07-18T03:39:43.037 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_high_water": 8, 2023-07-18T03:39:43.037 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed": 26100, 2023-07-18T03:39:43.038 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_item_in_journal": 0 2023-07-18T03:39:43.038 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:43.038 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-18T03:39:43.038 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2023-07-18T03:39:43.039 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2023-07-18T03:39:43.039 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2023-07-18T03:39:43.039 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 7872966, 2023-07-18T03:39:43.039 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 655951539, 2023-07-18T03:39:43.040 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2023-07-18T03:39:43.040 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 7872966, 2023-07-18T03:39:43.040 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2023-07-18T03:39:43.041 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2023-07-18T03:39:43.041 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 7872966, 2023-07-18T03:39:43.041 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 655951539, 2023-07-18T03:39:43.042 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2023-07-18T03:39:43.042 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:43.043 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:43.043 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:43.043 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-18T03:39:43.043 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:43.043 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_bytes": { 2023-07-18T03:39:43.044 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2023-07-18T03:39:43.044 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2023-07-18T03:39:43.044 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2023-07-18T03:39:43.044 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2023-07-18T03:39:43.044 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2023-07-18T03:39:43.045 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2023-07-18T03:39:43.045 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2023-07-18T03:39:43.045 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 100314, 2023-07-18T03:39:43.045 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 538293247, 2023-07-18T03:39:43.045 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 100314, 2023-07-18T03:39:43.046 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 538293247, 2023-07-18T03:39:43.046 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2023-07-18T03:39:43.046 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:43.046 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:43.046 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:43.047 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-18T03:39:43.047 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:43.047 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_ops": { 2023-07-18T03:39:43.047 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2023-07-18T03:39:43.047 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 1024, 2023-07-18T03:39:43.048 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2023-07-18T03:39:43.048 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2023-07-18T03:39:43.048 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2023-07-18T03:39:43.048 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2023-07-18T03:39:43.048 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2023-07-18T03:39:43.049 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 100314, 2023-07-18T03:39:43.049 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 100314, 2023-07-18T03:39:43.049 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 100314, 2023-07-18T03:39:43.049 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 100314, 2023-07-18T03:39:43.049 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2023-07-18T03:39:43.050 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:43.050 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:43.050 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:43.050 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-18T03:39:43.050 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:43.051 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle": { 2023-07-18T03:39:43.051 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2023-07-18T03:39:43.051 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2023-07-18T03:39:43.051 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2023-07-18T03:39:43.051 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 26100, 2023-07-18T03:39:43.052 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 2849796, 2023-07-18T03:39:43.052 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2023-07-18T03:39:43.052 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 26100, 2023-07-18T03:39:43.052 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2023-07-18T03:39:43.052 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2023-07-18T03:39:43.053 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 919, 2023-07-18T03:39:43.053 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 2849796, 2023-07-18T03:39:43.053 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2023-07-18T03:39:43.053 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:43.053 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:43.054 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:43.054 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-18T03:39:43.054 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:43.054 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle-0x5598965e8000": { 2023-07-18T03:39:43.054 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2023-07-18T03:39:43.054 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2023-07-18T03:39:43.055 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2023-07-18T03:39:43.055 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 143900, 2023-07-18T03:39:43.055 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 529637971, 2023-07-18T03:39:43.055 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2023-07-18T03:39:43.055 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 143900, 2023-07-18T03:39:43.056 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2023-07-18T03:39:43.056 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2023-07-18T03:39:43.056 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 42896, 2023-07-18T03:39:43.056 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 529637971, 2023-07-18T03:39:43.056 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2023-07-18T03:39:43.057 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:43.057 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:43.057 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:43.057 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-18T03:39:43.057 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-18T03:39:43.058 INFO:teuthology.orchestra.run.smithi155.stdout:} 2023-07-18T03:39:43.058 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:39:42 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:39:43.059 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:39:42 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:39:43.059 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:39:42 smithi101 ceph-mon[126379]: pgmap v1019: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:39:43.059 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:39:42 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:39:43.059 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:39:42 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:39:43.060 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:39:42 smithi179 ceph-mon[129229]: pgmap v1019: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:39:43.060 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:39:42 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:39:43.060 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:39:42 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:39:43.060 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:39:42 smithi155 ceph-mon[128885]: pgmap v1019: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:39:43.220 INFO:tasks.check_counter:Daemon mds.b mds.exported=19 2023-07-18T03:39:43.221 INFO:tasks.check_counter:Daemon mds.b mds.imported=11 2023-07-18T03:39:43.221 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2023-07-18T03:39:43.222 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=0 2023-07-18T03:39:43.222 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=0 2023-07-18T03:39:43.222 DEBUG:tasks.check_counter:Getting stats from e 2023-07-18T03:39:43.223 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph daemon mds.e perf dump 2023-07-18T03:39:43.824 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.b/config 2023-07-18T03:39:45.046 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2023-07-18T03:39:45.046 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-0": { 2023-07-18T03:39:45.046 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 1372, 2023-07-18T03:39:45.047 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 1372, 2023-07-18T03:39:45.047 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 210075969, 2023-07-18T03:39:45.047 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 425960, 2023-07-18T03:39:45.047 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 8, 2023-07-18T03:39:45.047 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 4, 2023-07-18T03:39:45.048 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.210961400, 2023-07-18T03:39:45.048 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.028892367, 2023-07-18T03:39:45.048 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.135665982, 2023-07-18T03:39:45.048 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.036184812, 2023-07-18T03:39:45.048 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:39:45.049 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 1371, 2023-07-18T03:39:45.049 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.071942678, 2023-07-18T03:39:45.049 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000052474 2023-07-18T03:39:45.049 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.049 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:39:45.050 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:45.050 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:45.050 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:45.050 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.050 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-07-18T03:39:45.051 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 448 2023-07-18T03:39:45.051 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.051 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-1": { 2023-07-18T03:39:45.051 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 2336, 2023-07-18T03:39:45.051 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 2688, 2023-07-18T03:39:45.052 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 148924060, 2023-07-18T03:39:45.052 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 1692348, 2023-07-18T03:39:45.052 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 7, 2023-07-18T03:39:45.052 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 5, 2023-07-18T03:39:45.052 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.293316158, 2023-07-18T03:39:45.053 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.065069413, 2023-07-18T03:39:45.053 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.157421275, 2023-07-18T03:39:45.053 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.063709380, 2023-07-18T03:39:45.053 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:39:45.053 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 2687, 2023-07-18T03:39:45.054 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.108753089, 2023-07-18T03:39:45.054 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000040473 2023-07-18T03:39:45.054 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.054 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:39:45.054 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:45.055 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:45.055 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:45.055 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.055 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 1703216, 2023-07-18T03:39:45.055 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 1169552 2023-07-18T03:39:45.056 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.056 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-2": { 2023-07-18T03:39:45.056 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 4100, 2023-07-18T03:39:45.056 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 4137, 2023-07-18T03:39:45.056 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 146377073, 2023-07-18T03:39:45.057 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 1399813, 2023-07-18T03:39:45.057 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 8, 2023-07-18T03:39:45.057 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 6, 2023-07-18T03:39:45.057 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.373535646, 2023-07-18T03:39:45.057 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.079991847, 2023-07-18T03:39:45.057 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.155751698, 2023-07-18T03:39:45.058 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.109762055, 2023-07-18T03:39:45.058 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:39:45.058 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 4134, 2023-07-18T03:39:45.058 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.154530380, 2023-07-18T03:39:45.058 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000037380 2023-07-18T03:39:45.059 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.059 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:39:45.059 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:45.059 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:45.059 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:45.060 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.060 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 160, 2023-07-18T03:39:45.060 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 125024 2023-07-18T03:39:45.060 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.060 INFO:teuthology.orchestra.run.smithi155.stdout: "cct": { 2023-07-18T03:39:45.061 INFO:teuthology.orchestra.run.smithi155.stdout: "total_workers": 1, 2023-07-18T03:39:45.061 INFO:teuthology.orchestra.run.smithi155.stdout: "unhealthy_workers": 0 2023-07-18T03:39:45.061 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.061 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-MDSRank": { 2023-07-18T03:39:45.061 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2023-07-18T03:39:45.062 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2023-07-18T03:39:45.062 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 11853, 2023-07-18T03:39:45.062 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.378103019, 2023-07-18T03:39:45.062 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000031899 2023-07-18T03:39:45.062 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-18T03:39:45.063 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.063 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-PurgeQueue": { 2023-07-18T03:39:45.063 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2023-07-18T03:39:45.063 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2023-07-18T03:39:45.063 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:45.064 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:45.064 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:45.064 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-18T03:39:45.064 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.064 INFO:teuthology.orchestra.run.smithi155.stdout: "mds": { 2023-07-18T03:39:45.065 INFO:teuthology.orchestra.run.smithi155.stdout: "request": 0, 2023-07-18T03:39:45.065 INFO:teuthology.orchestra.run.smithi155.stdout: "reply": 0, 2023-07-18T03:39:45.065 INFO:teuthology.orchestra.run.smithi155.stdout: "reply_latency": { 2023-07-18T03:39:45.065 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:45.065 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:45.066 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:45.066 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.066 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_reply": 0, 2023-07-18T03:39:45.066 INFO:teuthology.orchestra.run.smithi155.stdout: "forward": 0, 2023-07-18T03:39:45.066 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_complete": 0, 2023-07-18T03:39:45.066 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_keys": 0, 2023-07-18T03:39:45.067 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_commit": 0, 2023-07-18T03:39:45.067 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_split": 0, 2023-07-18T03:39:45.067 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_merge": 0, 2023-07-18T03:39:45.067 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes": 21587, 2023-07-18T03:39:45.067 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_top": 140, 2023-07-18T03:39:45.068 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_bottom": 486, 2023-07-18T03:39:45.068 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pin_tail": 20961, 2023-07-18T03:39:45.068 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pinned": 21386, 2023-07-18T03:39:45.068 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_expired": 7199, 2023-07-18T03:39:45.069 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_with_caps": 0, 2023-07-18T03:39:45.069 INFO:teuthology.orchestra.run.smithi155.stdout: "caps": 0, 2023-07-18T03:39:45.069 INFO:teuthology.orchestra.run.smithi155.stdout: "subtrees": 15, 2023-07-18T03:39:45.069 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse": 0, 2023-07-18T03:39:45.069 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_hit": 0, 2023-07-18T03:39:45.070 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_forward": 0, 2023-07-18T03:39:45.070 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_discover": 0, 2023-07-18T03:39:45.070 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_dir_fetch": 0, 2023-07-18T03:39:45.070 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_remote_ino": 0, 2023-07-18T03:39:45.070 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_lock": 0, 2023-07-18T03:39:45.071 INFO:teuthology.orchestra.run.smithi155.stdout: "load_cent": 0, 2023-07-18T03:39:45.071 INFO:teuthology.orchestra.run.smithi155.stdout: "q": 0, 2023-07-18T03:39:45.071 INFO:teuthology.orchestra.run.smithi155.stdout: "exported": 0, 2023-07-18T03:39:45.071 INFO:teuthology.orchestra.run.smithi155.stdout: "exported_inodes": 0, 2023-07-18T03:39:45.071 INFO:teuthology.orchestra.run.smithi155.stdout: "imported": 0, 2023-07-18T03:39:45.072 INFO:teuthology.orchestra.run.smithi155.stdout: "imported_inodes": 0, 2023-07-18T03:39:45.072 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_dir_fetch": 0, 2023-07-18T03:39:45.072 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_backtrace_fetch": 0, 2023-07-18T03:39:45.072 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_peer_discover": 0, 2023-07-18T03:39:45.072 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rfiles": 0, 2023-07-18T03:39:45.073 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rbytes": 0, 2023-07-18T03:39:45.073 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rsnaps": 0, 2023-07-18T03:39:45.073 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_fetch": 0, 2023-07-18T03:39:45.073 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_set_tag": 0, 2023-07-18T03:39:45.073 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_repaired": 0, 2023-07-18T03:39:45.074 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_inotable_repaired": 0, 2023-07-18T03:39:45.074 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_inodes": 0, 2023-07-18T03:39:45.074 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_base_inodes": 0, 2023-07-18T03:39:45.074 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dirfrag_rstats": 0, 2023-07-18T03:39:45.074 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_file_inodes": 0, 2023-07-18T03:39:45.075 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_inode_file_caps": 0, 2023-07-18T03:39:45.075 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_revoke": 0, 2023-07-18T03:39:45.075 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_grant": 0, 2023-07-18T03:39:45.075 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_trunc": 0, 2023-07-18T03:39:45.075 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-18T03:39:45.076 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-18T03:39:45.076 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps": 0, 2023-07-18T03:39:45.076 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps_dirty": 0, 2023-07-18T03:39:45.076 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_cap_release": 0, 2023-07-18T03:39:45.077 INFO:teuthology.orchestra.run.smithi155.stdout: "process_request_cap_release": 0 2023-07-18T03:39:45.077 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.077 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_cache": { 2023-07-18T03:39:45.077 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update": 0, 2023-07-18T03:39:45.077 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update_receipt": 0, 2023-07-18T03:39:45.078 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_try_discover": 0, 2023-07-18T03:39:45.078 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_send_discover": 0, 2023-07-18T03:39:45.078 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_handle_discover": 0, 2023-07-18T03:39:45.078 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays": 0, 2023-07-18T03:39:45.078 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_delayed": 0, 2023-07-18T03:39:45.078 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_enqueuing": 0, 2023-07-18T03:39:45.079 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_created": 0, 2023-07-18T03:39:45.079 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_enqueued": 0, 2023-07-18T03:39:45.079 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_reintegrated": 0, 2023-07-18T03:39:45.079 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_migrated": 0, 2023-07-18T03:39:45.079 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_processing": 0, 2023-07-18T03:39:45.080 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_enqueued": 0, 2023-07-18T03:39:45.080 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_prioritized": 0, 2023-07-18T03:39:45.080 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_started": 0, 2023-07-18T03:39:45.080 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_completed": 0, 2023-07-18T03:39:45.080 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_enqueue_scrub": 0, 2023-07-18T03:39:45.081 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_exportdir": 0, 2023-07-18T03:39:45.081 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_flush": 0, 2023-07-18T03:39:45.081 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragmentdir": 0, 2023-07-18T03:39:45.081 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragstats": 0, 2023-07-18T03:39:45.081 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_inodestats": 0 2023-07-18T03:39:45.082 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.082 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_log": { 2023-07-18T03:39:45.082 INFO:teuthology.orchestra.run.smithi155.stdout: "evadd": 0, 2023-07-18T03:39:45.082 INFO:teuthology.orchestra.run.smithi155.stdout: "evex": 0, 2023-07-18T03:39:45.082 INFO:teuthology.orchestra.run.smithi155.stdout: "evtrm": 0, 2023-07-18T03:39:45.083 INFO:teuthology.orchestra.run.smithi155.stdout: "ev": 0, 2023-07-18T03:39:45.083 INFO:teuthology.orchestra.run.smithi155.stdout: "evexg": 0, 2023-07-18T03:39:45.083 INFO:teuthology.orchestra.run.smithi155.stdout: "evexd": 0, 2023-07-18T03:39:45.083 INFO:teuthology.orchestra.run.smithi155.stdout: "segadd": 0, 2023-07-18T03:39:45.083 INFO:teuthology.orchestra.run.smithi155.stdout: "segex": 0, 2023-07-18T03:39:45.084 INFO:teuthology.orchestra.run.smithi155.stdout: "segtrm": 0, 2023-07-18T03:39:45.084 INFO:teuthology.orchestra.run.smithi155.stdout: "seg": 128, 2023-07-18T03:39:45.084 INFO:teuthology.orchestra.run.smithi155.stdout: "segexg": 0, 2023-07-18T03:39:45.084 INFO:teuthology.orchestra.run.smithi155.stdout: "segexd": 0, 2023-07-18T03:39:45.084 INFO:teuthology.orchestra.run.smithi155.stdout: "expos": 65195813, 2023-07-18T03:39:45.085 INFO:teuthology.orchestra.run.smithi155.stdout: "wrpos": 0, 2023-07-18T03:39:45.085 INFO:teuthology.orchestra.run.smithi155.stdout: "rdpos": 505865047, 2023-07-18T03:39:45.085 INFO:teuthology.orchestra.run.smithi155.stdout: "jlat": { 2023-07-18T03:39:45.085 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:45.085 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:45.086 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:45.086 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.086 INFO:teuthology.orchestra.run.smithi155.stdout: "replayed": 127224 2023-07-18T03:39:45.086 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.086 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_mem": { 2023-07-18T03:39:45.087 INFO:teuthology.orchestra.run.smithi155.stdout: "ino": 1567, 2023-07-18T03:39:45.087 INFO:teuthology.orchestra.run.smithi155.stdout: "ino+": 28952, 2023-07-18T03:39:45.087 INFO:teuthology.orchestra.run.smithi155.stdout: "ino-": 27385, 2023-07-18T03:39:45.087 INFO:teuthology.orchestra.run.smithi155.stdout: "dir": 121, 2023-07-18T03:39:45.087 INFO:teuthology.orchestra.run.smithi155.stdout: "dir+": 217, 2023-07-18T03:39:45.088 INFO:teuthology.orchestra.run.smithi155.stdout: "dir-": 96, 2023-07-18T03:39:45.088 INFO:teuthology.orchestra.run.smithi155.stdout: "dn": 21587, 2023-07-18T03:39:45.088 INFO:teuthology.orchestra.run.smithi155.stdout: "dn+": 29564, 2023-07-18T03:39:45.088 INFO:teuthology.orchestra.run.smithi155.stdout: "dn-": 7977, 2023-07-18T03:39:45.088 INFO:teuthology.orchestra.run.smithi155.stdout: "cap": 0, 2023-07-18T03:39:45.089 INFO:teuthology.orchestra.run.smithi155.stdout: "cap+": 0, 2023-07-18T03:39:45.089 INFO:teuthology.orchestra.run.smithi155.stdout: "cap-": 0, 2023-07-18T03:39:45.089 INFO:teuthology.orchestra.run.smithi155.stdout: "rss": 75908, 2023-07-18T03:39:45.089 INFO:teuthology.orchestra.run.smithi155.stdout: "heap": 207156 2023-07-18T03:39:45.089 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.090 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_server": { 2023-07-18T03:39:45.090 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_client_request": 0, 2023-07-18T03:39:45.090 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_server_request": 0, 2023-07-18T03:39:45.090 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_request": 0, 2023-07-18T03:39:45.090 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_session": 0, 2023-07-18T03:39:45.090 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_peer_request": 0, 2023-07-18T03:39:45.091 INFO:teuthology.orchestra.run.smithi155.stdout: "req_create_latency": { 2023-07-18T03:39:45.091 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:45.091 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:45.091 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:45.091 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.092 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getattr_latency": { 2023-07-18T03:39:45.092 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:45.092 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:45.092 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:45.092 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.093 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getfilelock_latency": { 2023-07-18T03:39:45.093 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:45.093 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:45.093 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:45.093 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.094 INFO:teuthology.orchestra.run.smithi155.stdout: "req_link_latency": { 2023-07-18T03:39:45.094 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:45.094 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:45.094 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:45.094 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.095 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookup_latency": { 2023-07-18T03:39:45.095 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:45.095 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:45.095 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:45.095 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.096 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookuphash_latency": { 2023-07-18T03:39:45.096 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:45.096 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:45.096 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:45.096 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.097 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupino_latency": { 2023-07-18T03:39:45.097 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:45.097 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:45.097 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:45.097 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.098 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupname_latency": { 2023-07-18T03:39:45.098 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:45.098 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:45.098 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:45.098 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.099 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupparent_latency": { 2023-07-18T03:39:45.099 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:45.099 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:45.099 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:45.099 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.100 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupsnap_latency": { 2023-07-18T03:39:45.100 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:45.100 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:45.100 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:45.100 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.101 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lssnap_latency": { 2023-07-18T03:39:45.101 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:45.101 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:45.101 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:45.101 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.102 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mkdir_latency": { 2023-07-18T03:39:45.102 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:45.102 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:45.102 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:45.102 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.103 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mknod_latency": { 2023-07-18T03:39:45.103 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:45.103 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:45.103 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:45.103 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.104 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mksnap_latency": { 2023-07-18T03:39:45.104 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:45.104 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:45.104 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:45.104 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.105 INFO:teuthology.orchestra.run.smithi155.stdout: "req_open_latency": { 2023-07-18T03:39:45.105 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:45.105 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:45.105 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:45.105 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.106 INFO:teuthology.orchestra.run.smithi155.stdout: "req_readdir_latency": { 2023-07-18T03:39:45.106 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:45.106 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:45.106 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:45.106 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.107 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rename_latency": { 2023-07-18T03:39:45.107 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:45.107 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:45.107 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:45.107 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.107 INFO:teuthology.orchestra.run.smithi155.stdout: "req_renamesnap_latency": { 2023-07-18T03:39:45.108 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:45.108 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:45.108 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:45.108 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.108 INFO:teuthology.orchestra.run.smithi155.stdout: "req_snapdiff_latency": { 2023-07-18T03:39:45.109 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:45.109 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:45.109 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:45.109 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.109 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmdir_latency": { 2023-07-18T03:39:45.110 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:45.110 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:45.110 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:45.110 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.110 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmsnap_latency": { 2023-07-18T03:39:45.111 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:45.111 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:45.111 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:45.111 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.111 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmxattr_latency": { 2023-07-18T03:39:45.112 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:45.112 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:45.112 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:45.112 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.113 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setattr_latency": { 2023-07-18T03:39:45.113 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:45.113 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:45.113 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:45.113 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.113 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setdirlayout_latency": { 2023-07-18T03:39:45.114 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:45.114 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:45.114 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:45.114 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.114 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setfilelock_latency": { 2023-07-18T03:39:45.115 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:45.115 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:45.115 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:45.115 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.115 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setlayout_latency": { 2023-07-18T03:39:45.116 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:45.116 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:45.116 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:45.116 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.116 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setxattr_latency": { 2023-07-18T03:39:45.117 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:45.117 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:45.117 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:45.117 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.117 INFO:teuthology.orchestra.run.smithi155.stdout: "req_symlink_latency": { 2023-07-18T03:39:45.118 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:45.118 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:45.118 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:45.118 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.118 INFO:teuthology.orchestra.run.smithi155.stdout: "req_unlink_latency": { 2023-07-18T03:39:45.119 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:45.119 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:45.119 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:45.119 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.119 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_revoke_eviction": 0, 2023-07-18T03:39:45.120 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_acquisition_throttle": 0, 2023-07-18T03:39:45.120 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getvxattr_latency": { 2023-07-18T03:39:45.120 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:45.120 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:45.120 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:45.121 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-18T03:39:45.121 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.121 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_sessions": { 2023-07-18T03:39:45.121 INFO:teuthology.orchestra.run.smithi155.stdout: "session_count": 1, 2023-07-18T03:39:45.121 INFO:teuthology.orchestra.run.smithi155.stdout: "session_add": 3, 2023-07-18T03:39:45.122 INFO:teuthology.orchestra.run.smithi155.stdout: "session_remove": 2, 2023-07-18T03:39:45.122 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_open": 3, 2023-07-18T03:39:45.122 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_stale": 0, 2023-07-18T03:39:45.122 INFO:teuthology.orchestra.run.smithi155.stdout: "total_load": 0, 2023-07-18T03:39:45.122 INFO:teuthology.orchestra.run.smithi155.stdout: "average_load": 0, 2023-07-18T03:39:45.122 INFO:teuthology.orchestra.run.smithi155.stdout: "avg_session_uptime": 8772 2023-07-18T03:39:45.123 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.123 INFO:teuthology.orchestra.run.smithi155.stdout: "mempool": { 2023-07-18T03:39:45.123 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_bytes": 0, 2023-07-18T03:39:45.123 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_items": 0, 2023-07-18T03:39:45.123 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_bytes": 0, 2023-07-18T03:39:45.124 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_items": 0, 2023-07-18T03:39:45.124 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_bytes": 0, 2023-07-18T03:39:45.124 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_items": 0, 2023-07-18T03:39:45.124 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-18T03:39:45.124 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_items": 0, 2023-07-18T03:39:45.125 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-18T03:39:45.125 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_items": 0, 2023-07-18T03:39:45.125 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_bytes": 0, 2023-07-18T03:39:45.125 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_items": 0, 2023-07-18T03:39:45.125 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-18T03:39:45.126 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_items": 0, 2023-07-18T03:39:45.126 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_bytes": 0, 2023-07-18T03:39:45.126 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_items": 0, 2023-07-18T03:39:45.126 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_bytes": 0, 2023-07-18T03:39:45.126 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_items": 0, 2023-07-18T03:39:45.127 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-18T03:39:45.127 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_items": 0, 2023-07-18T03:39:45.127 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-18T03:39:45.127 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_items": 0, 2023-07-18T03:39:45.127 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_bytes": 0, 2023-07-18T03:39:45.128 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_items": 0, 2023-07-18T03:39:45.128 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_bytes": 0, 2023-07-18T03:39:45.128 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_items": 0, 2023-07-18T03:39:45.128 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-18T03:39:45.128 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_items": 0, 2023-07-18T03:39:45.129 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_bytes": 0, 2023-07-18T03:39:45.129 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_items": 0, 2023-07-18T03:39:45.129 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_bytes": 0, 2023-07-18T03:39:45.129 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_items": 0, 2023-07-18T03:39:45.129 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_bytes": 0, 2023-07-18T03:39:45.130 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_items": 0, 2023-07-18T03:39:45.130 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_bytes": 0, 2023-07-18T03:39:45.130 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_items": 0, 2023-07-18T03:39:45.130 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_bytes": 306571, 2023-07-18T03:39:45.130 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_items": 1991, 2023-07-18T03:39:45.130 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_bytes": 0, 2023-07-18T03:39:45.131 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_items": 0, 2023-07-18T03:39:45.131 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_bytes": 0, 2023-07-18T03:39:45.131 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_items": 0, 2023-07-18T03:39:45.131 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_bytes": 0, 2023-07-18T03:39:45.131 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_items": 0, 2023-07-18T03:39:45.132 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_bytes": 0, 2023-07-18T03:39:45.132 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_items": 0, 2023-07-18T03:39:45.132 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_bytes": 9616, 2023-07-18T03:39:45.132 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_items": 185, 2023-07-18T03:39:45.132 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_bytes": 0, 2023-07-18T03:39:45.133 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_items": 0, 2023-07-18T03:39:45.133 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_bytes": 0, 2023-07-18T03:39:45.133 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_items": 0, 2023-07-18T03:39:45.133 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_bytes": 19405144, 2023-07-18T03:39:45.133 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_items": 130663, 2023-07-18T03:39:45.134 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_bytes": 0, 2023-07-18T03:39:45.134 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_items": 0, 2023-07-18T03:39:45.134 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_bytes": 0, 2023-07-18T03:39:45.134 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_items": 0 2023-07-18T03:39:45.134 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.135 INFO:teuthology.orchestra.run.smithi155.stdout: "objecter": { 2023-07-18T03:39:45.135 INFO:teuthology.orchestra.run.smithi155.stdout: "op_active": 0, 2023-07-18T03:39:45.135 INFO:teuthology.orchestra.run.smithi155.stdout: "op_laggy": 0, 2023-07-18T03:39:45.135 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send": 7148, 2023-07-18T03:39:45.135 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send_bytes": 21, 2023-07-18T03:39:45.136 INFO:teuthology.orchestra.run.smithi155.stdout: "op_resend": 3, 2023-07-18T03:39:45.136 INFO:teuthology.orchestra.run.smithi155.stdout: "op_reply": 7148, 2023-07-18T03:39:45.136 INFO:teuthology.orchestra.run.smithi155.stdout: "op_latency": { 2023-07-18T03:39:45.136 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 7148, 2023-07-18T03:39:45.136 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 11.468765663, 2023-07-18T03:39:45.137 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.001604471 2023-07-18T03:39:45.137 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.137 INFO:teuthology.orchestra.run.smithi155.stdout: "op_inflight": 0, 2023-07-18T03:39:45.137 INFO:teuthology.orchestra.run.smithi155.stdout: "oplen_avg": { 2023-07-18T03:39:45.137 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 7148, 2023-07-18T03:39:45.138 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 7149 2023-07-18T03:39:45.138 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.138 INFO:teuthology.orchestra.run.smithi155.stdout: "op": 7148, 2023-07-18T03:39:45.138 INFO:teuthology.orchestra.run.smithi155.stdout: "op_r": 7148, 2023-07-18T03:39:45.138 INFO:teuthology.orchestra.run.smithi155.stdout: "op_w": 0, 2023-07-18T03:39:45.139 INFO:teuthology.orchestra.run.smithi155.stdout: "op_rmw": 0, 2023-07-18T03:39:45.139 INFO:teuthology.orchestra.run.smithi155.stdout: "op_pg": 0, 2023-07-18T03:39:45.139 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_stat": 4226, 2023-07-18T03:39:45.139 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_create": 0, 2023-07-18T03:39:45.139 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_read": 2920, 2023-07-18T03:39:45.140 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_write": 0, 2023-07-18T03:39:45.140 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writefull": 0, 2023-07-18T03:39:45.140 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writesame": 0, 2023-07-18T03:39:45.140 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_append": 0, 2023-07-18T03:39:45.140 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_zero": 0, 2023-07-18T03:39:45.140 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_truncate": 0, 2023-07-18T03:39:45.141 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_delete": 0, 2023-07-18T03:39:45.141 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_mapext": 0, 2023-07-18T03:39:45.141 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_sparse_read": 0, 2023-07-18T03:39:45.141 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_clonerange": 0, 2023-07-18T03:39:45.141 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_getxattr": 1, 2023-07-18T03:39:45.142 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_setxattr": 0, 2023-07-18T03:39:45.142 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_cmpxattr": 0, 2023-07-18T03:39:45.142 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_rmxattr": 0, 2023-07-18T03:39:45.142 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_resetxattrs": 0, 2023-07-18T03:39:45.142 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_call": 0, 2023-07-18T03:39:45.143 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_watch": 0, 2023-07-18T03:39:45.143 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_notify": 0, 2023-07-18T03:39:45.143 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_src_cmpxattr": 0, 2023-07-18T03:39:45.143 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls": 0, 2023-07-18T03:39:45.143 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls_filter": 0, 2023-07-18T03:39:45.144 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_other": 0, 2023-07-18T03:39:45.144 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_active": 0, 2023-07-18T03:39:45.144 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_send": 0, 2023-07-18T03:39:45.144 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_resend": 0, 2023-07-18T03:39:45.144 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_ping": 0, 2023-07-18T03:39:45.145 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_active": 0, 2023-07-18T03:39:45.145 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_send": 0, 2023-07-18T03:39:45.145 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_resend": 0, 2023-07-18T03:39:45.145 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_active": 0, 2023-07-18T03:39:45.145 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_send": 0, 2023-07-18T03:39:45.146 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_resend": 0, 2023-07-18T03:39:45.146 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_active": 0, 2023-07-18T03:39:45.146 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_send": 0, 2023-07-18T03:39:45.146 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_resend": 0, 2023-07-18T03:39:45.146 INFO:teuthology.orchestra.run.smithi155.stdout: "command_active": 0, 2023-07-18T03:39:45.147 INFO:teuthology.orchestra.run.smithi155.stdout: "command_send": 0, 2023-07-18T03:39:45.147 INFO:teuthology.orchestra.run.smithi155.stdout: "command_resend": 0, 2023-07-18T03:39:45.147 INFO:teuthology.orchestra.run.smithi155.stdout: "map_epoch": 144, 2023-07-18T03:39:45.147 INFO:teuthology.orchestra.run.smithi155.stdout: "map_full": 0, 2023-07-18T03:39:45.147 INFO:teuthology.orchestra.run.smithi155.stdout: "map_inc": 55, 2023-07-18T03:39:45.148 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_sessions": 11, 2023-07-18T03:39:45.148 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_open": 11, 2023-07-18T03:39:45.148 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_close": 0, 2023-07-18T03:39:45.148 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_laggy": 0, 2023-07-18T03:39:45.148 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_wr": 0, 2023-07-18T03:39:45.149 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_rd": 2, 2023-07-18T03:39:45.149 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_del": 0 2023-07-18T03:39:45.149 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.149 INFO:teuthology.orchestra.run.smithi155.stdout: "oft": { 2023-07-18T03:39:45.149 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_objs": 0, 2023-07-18T03:39:45.149 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_kv_pairs": 0, 2023-07-18T03:39:45.150 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_updates": 0, 2023-07-18T03:39:45.150 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_removes": 0 2023-07-18T03:39:45.150 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.150 INFO:teuthology.orchestra.run.smithi155.stdout: "osd-slow-ops": { 2023-07-18T03:39:45.150 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_ops_count": 0 2023-07-18T03:39:45.151 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.151 INFO:teuthology.orchestra.run.smithi155.stdout: "purge_queue": { 2023-07-18T03:39:45.151 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops": 0, 2023-07-18T03:39:45.151 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops_high_water": 0, 2023-07-18T03:39:45.151 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing": 0, 2023-07-18T03:39:45.152 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_high_water": 0, 2023-07-18T03:39:45.152 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed": 0, 2023-07-18T03:39:45.152 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_item_in_journal": 0 2023-07-18T03:39:45.152 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.152 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-18T03:39:45.153 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2023-07-18T03:39:45.153 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2023-07-18T03:39:45.153 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2023-07-18T03:39:45.153 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 7804, 2023-07-18T03:39:45.153 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 504638378, 2023-07-18T03:39:45.154 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2023-07-18T03:39:45.154 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 7804, 2023-07-18T03:39:45.154 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2023-07-18T03:39:45.154 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2023-07-18T03:39:45.154 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 7804, 2023-07-18T03:39:45.155 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 504638378, 2023-07-18T03:39:45.155 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2023-07-18T03:39:45.155 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:45.155 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:45.155 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:45.156 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-18T03:39:45.156 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.156 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_bytes": { 2023-07-18T03:39:45.156 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2023-07-18T03:39:45.156 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2023-07-18T03:39:45.157 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2023-07-18T03:39:45.157 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2023-07-18T03:39:45.157 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2023-07-18T03:39:45.157 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2023-07-18T03:39:45.157 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2023-07-18T03:39:45.158 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 7148, 2023-07-18T03:39:45.158 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 501686062, 2023-07-18T03:39:45.158 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 7148, 2023-07-18T03:39:45.158 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 501686062, 2023-07-18T03:39:45.158 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2023-07-18T03:39:45.159 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:45.159 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:45.159 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:45.159 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-18T03:39:45.159 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.160 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_ops": { 2023-07-18T03:39:45.160 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2023-07-18T03:39:45.160 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 1024, 2023-07-18T03:39:45.160 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2023-07-18T03:39:45.160 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2023-07-18T03:39:45.161 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2023-07-18T03:39:45.161 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2023-07-18T03:39:45.161 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2023-07-18T03:39:45.161 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 7148, 2023-07-18T03:39:45.161 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 7148, 2023-07-18T03:39:45.161 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 7148, 2023-07-18T03:39:45.162 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 7148, 2023-07-18T03:39:45.162 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2023-07-18T03:39:45.162 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:45.162 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:45.162 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:45.163 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-18T03:39:45.163 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.163 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle": { 2023-07-18T03:39:45.163 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2023-07-18T03:39:45.163 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2023-07-18T03:39:45.164 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2023-07-18T03:39:45.164 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2023-07-18T03:39:45.164 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2023-07-18T03:39:45.164 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2023-07-18T03:39:45.164 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2023-07-18T03:39:45.165 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2023-07-18T03:39:45.165 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2023-07-18T03:39:45.165 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 0, 2023-07-18T03:39:45.165 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 0, 2023-07-18T03:39:45.165 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2023-07-18T03:39:45.166 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:45.166 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:45.166 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:45.166 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-18T03:39:45.166 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:45.167 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle-0x55a32fb74460": { 2023-07-18T03:39:45.167 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2023-07-18T03:39:45.167 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2023-07-18T03:39:45.167 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2023-07-18T03:39:45.167 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2023-07-18T03:39:45.168 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2023-07-18T03:39:45.168 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2023-07-18T03:39:45.168 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2023-07-18T03:39:45.168 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2023-07-18T03:39:45.168 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2023-07-18T03:39:45.169 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 0, 2023-07-18T03:39:45.169 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 0, 2023-07-18T03:39:45.169 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2023-07-18T03:39:45.169 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:45.169 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:45.170 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:45.170 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-18T03:39:45.170 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-18T03:39:45.170 INFO:teuthology.orchestra.run.smithi155.stdout:} 2023-07-18T03:39:45.498 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2023-07-18T03:39:45.498 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2023-07-18T03:39:45.499 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2023-07-18T03:39:45.499 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2023-07-18T03:39:45.499 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2023-07-18T03:39:45.499 DEBUG:tasks.check_counter:Getting stats from h 2023-07-18T03:39:45.500 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph daemon mds.h perf dump 2023-07-18T03:39:45.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:39:45 smithi101 ceph-mon[126379]: pgmap v1020: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:39:45.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:39:45 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:39:45.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:39:45 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:39:45.740 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:39:45 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:39:45.796 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:39:45 smithi155 ceph-mon[128885]: pgmap v1020: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:39:45.796 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:39:45 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:39:45.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:39:45 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:39:45.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:39:45 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:39:45.951 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:39:45 smithi179 ceph-mon[129229]: pgmap v1020: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:39:45.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:39:45 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:39:45.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:39:45 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:39:45.953 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:39:45 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' 2023-07-18T03:39:46.086 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.b/config 2023-07-18T03:39:47.326 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2023-07-18T03:39:47.327 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-0": { 2023-07-18T03:39:47.327 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 4405, 2023-07-18T03:39:47.327 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 4407, 2023-07-18T03:39:47.327 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 195394778, 2023-07-18T03:39:47.328 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 1369962, 2023-07-18T03:39:47.328 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 8, 2023-07-18T03:39:47.328 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 4, 2023-07-18T03:39:47.329 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.407149065, 2023-07-18T03:39:47.329 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.096606728, 2023-07-18T03:39:47.329 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.173220368, 2023-07-18T03:39:47.329 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.109383289, 2023-07-18T03:39:47.330 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:39:47.330 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 4405, 2023-07-18T03:39:47.330 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.145268225, 2023-07-18T03:39:47.331 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000032978 2023-07-18T03:39:47.331 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.331 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:39:47.331 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:47.331 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:47.332 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:47.332 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.332 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-18T03:39:47.332 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 0 2023-07-18T03:39:47.332 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.333 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-1": { 2023-07-18T03:39:47.333 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 1512, 2023-07-18T03:39:47.333 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 1477, 2023-07-18T03:39:47.333 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 148732634, 2023-07-18T03:39:47.334 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 597954, 2023-07-18T03:39:47.334 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 6, 2023-07-18T03:39:47.334 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 5, 2023-07-18T03:39:47.334 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.208645021, 2023-07-18T03:39:47.335 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.043657896, 2023-07-18T03:39:47.335 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.132078561, 2023-07-18T03:39:47.335 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.033918748, 2023-07-18T03:39:47.335 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:39:47.335 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 1476, 2023-07-18T03:39:47.336 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.060093210, 2023-07-18T03:39:47.336 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000040713 2023-07-18T03:39:47.336 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.336 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:39:47.337 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:47.337 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:47.337 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:47.337 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.337 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 1702624, 2023-07-18T03:39:47.338 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 330432 2023-07-18T03:39:47.338 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.338 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-2": { 2023-07-18T03:39:47.338 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 1938, 2023-07-18T03:39:47.338 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 2359, 2023-07-18T03:39:47.339 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 189236825, 2023-07-18T03:39:47.339 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 1565917, 2023-07-18T03:39:47.339 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 9, 2023-07-18T03:39:47.339 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 6, 2023-07-18T03:39:47.339 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.252283388, 2023-07-18T03:39:47.340 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.048935807, 2023-07-18T03:39:47.340 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.137661654, 2023-07-18T03:39:47.340 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.050854199, 2023-07-18T03:39:47.340 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:39:47.340 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 2357, 2023-07-18T03:39:47.341 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.104221859, 2023-07-18T03:39:47.341 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000044218 2023-07-18T03:39:47.341 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.341 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:39:47.341 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:47.342 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:47.342 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:47.342 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.342 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 3424, 2023-07-18T03:39:47.343 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 965376 2023-07-18T03:39:47.343 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.343 INFO:teuthology.orchestra.run.smithi155.stdout: "cct": { 2023-07-18T03:39:47.344 INFO:teuthology.orchestra.run.smithi155.stdout: "total_workers": 1, 2023-07-18T03:39:47.344 INFO:teuthology.orchestra.run.smithi155.stdout: "unhealthy_workers": 0 2023-07-18T03:39:47.345 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.345 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-MDSRank": { 2023-07-18T03:39:47.346 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2023-07-18T03:39:47.346 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2023-07-18T03:39:47.347 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 12018, 2023-07-18T03:39:47.348 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.335233616, 2023-07-18T03:39:47.348 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000027894 2023-07-18T03:39:47.349 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-18T03:39:47.349 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.350 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-PurgeQueue": { 2023-07-18T03:39:47.350 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2023-07-18T03:39:47.351 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2023-07-18T03:39:47.351 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:47.352 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:47.352 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:47.353 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-18T03:39:47.353 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.354 INFO:teuthology.orchestra.run.smithi155.stdout: "mds": { 2023-07-18T03:39:47.355 INFO:teuthology.orchestra.run.smithi155.stdout: "request": 0, 2023-07-18T03:39:47.355 INFO:teuthology.orchestra.run.smithi155.stdout: "reply": 0, 2023-07-18T03:39:47.355 INFO:teuthology.orchestra.run.smithi155.stdout: "reply_latency": { 2023-07-18T03:39:47.356 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:47.356 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:47.357 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:47.357 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.358 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_reply": 0, 2023-07-18T03:39:47.358 INFO:teuthology.orchestra.run.smithi155.stdout: "forward": 0, 2023-07-18T03:39:47.359 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_complete": 0, 2023-07-18T03:39:47.359 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_keys": 0, 2023-07-18T03:39:47.360 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_commit": 0, 2023-07-18T03:39:47.360 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_split": 0, 2023-07-18T03:39:47.361 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_merge": 0, 2023-07-18T03:39:47.361 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes": 22376, 2023-07-18T03:39:47.362 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_top": 1250, 2023-07-18T03:39:47.362 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_bottom": 18108, 2023-07-18T03:39:47.363 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pin_tail": 3018, 2023-07-18T03:39:47.363 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pinned": 20589, 2023-07-18T03:39:47.363 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_expired": 5138, 2023-07-18T03:39:47.364 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_with_caps": 0, 2023-07-18T03:39:47.364 INFO:teuthology.orchestra.run.smithi155.stdout: "caps": 0, 2023-07-18T03:39:47.364 INFO:teuthology.orchestra.run.smithi155.stdout: "subtrees": 19, 2023-07-18T03:39:47.365 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse": 0, 2023-07-18T03:39:47.365 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_hit": 0, 2023-07-18T03:39:47.366 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_forward": 0, 2023-07-18T03:39:47.366 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_discover": 0, 2023-07-18T03:39:47.367 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_dir_fetch": 0, 2023-07-18T03:39:47.367 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_remote_ino": 0, 2023-07-18T03:39:47.367 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_lock": 0, 2023-07-18T03:39:47.368 INFO:teuthology.orchestra.run.smithi155.stdout: "load_cent": 0, 2023-07-18T03:39:47.368 INFO:teuthology.orchestra.run.smithi155.stdout: "q": 0, 2023-07-18T03:39:47.369 INFO:teuthology.orchestra.run.smithi155.stdout: "exported": 0, 2023-07-18T03:39:47.369 INFO:teuthology.orchestra.run.smithi155.stdout: "exported_inodes": 0, 2023-07-18T03:39:47.369 INFO:teuthology.orchestra.run.smithi155.stdout: "imported": 0, 2023-07-18T03:39:47.369 INFO:teuthology.orchestra.run.smithi155.stdout: "imported_inodes": 0, 2023-07-18T03:39:47.369 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_dir_fetch": 0, 2023-07-18T03:39:47.370 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_backtrace_fetch": 0, 2023-07-18T03:39:47.370 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_peer_discover": 0, 2023-07-18T03:39:47.370 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rfiles": 0, 2023-07-18T03:39:47.370 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rbytes": 0, 2023-07-18T03:39:47.370 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rsnaps": 0, 2023-07-18T03:39:47.371 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_fetch": 0, 2023-07-18T03:39:47.371 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_set_tag": 0, 2023-07-18T03:39:47.371 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_repaired": 0, 2023-07-18T03:39:47.371 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_inotable_repaired": 0, 2023-07-18T03:39:47.371 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_inodes": 0, 2023-07-18T03:39:47.372 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_base_inodes": 0, 2023-07-18T03:39:47.372 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dirfrag_rstats": 0, 2023-07-18T03:39:47.372 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_file_inodes": 0, 2023-07-18T03:39:47.372 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_inode_file_caps": 0, 2023-07-18T03:39:47.372 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_revoke": 0, 2023-07-18T03:39:47.373 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_grant": 0, 2023-07-18T03:39:47.373 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_trunc": 0, 2023-07-18T03:39:47.373 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-18T03:39:47.373 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-18T03:39:47.373 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps": 0, 2023-07-18T03:39:47.374 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps_dirty": 0, 2023-07-18T03:39:47.374 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_cap_release": 0, 2023-07-18T03:39:47.374 INFO:teuthology.orchestra.run.smithi155.stdout: "process_request_cap_release": 0 2023-07-18T03:39:47.374 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.374 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_cache": { 2023-07-18T03:39:47.375 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update": 0, 2023-07-18T03:39:47.375 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update_receipt": 0, 2023-07-18T03:39:47.375 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_try_discover": 0, 2023-07-18T03:39:47.375 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_send_discover": 0, 2023-07-18T03:39:47.375 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_handle_discover": 0, 2023-07-18T03:39:47.376 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays": 0, 2023-07-18T03:39:47.376 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_delayed": 0, 2023-07-18T03:39:47.376 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_enqueuing": 0, 2023-07-18T03:39:47.377 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_created": 0, 2023-07-18T03:39:47.377 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_enqueued": 0, 2023-07-18T03:39:47.377 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_reintegrated": 0, 2023-07-18T03:39:47.378 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_migrated": 0, 2023-07-18T03:39:47.379 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_processing": 0, 2023-07-18T03:39:47.379 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_enqueued": 0, 2023-07-18T03:39:47.379 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_prioritized": 0, 2023-07-18T03:39:47.380 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_started": 0, 2023-07-18T03:39:47.381 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_completed": 0, 2023-07-18T03:39:47.381 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_enqueue_scrub": 0, 2023-07-18T03:39:47.382 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_exportdir": 0, 2023-07-18T03:39:47.382 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_flush": 0, 2023-07-18T03:39:47.383 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragmentdir": 0, 2023-07-18T03:39:47.383 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragstats": 0, 2023-07-18T03:39:47.384 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_inodestats": 0 2023-07-18T03:39:47.385 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.385 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_log": { 2023-07-18T03:39:47.386 INFO:teuthology.orchestra.run.smithi155.stdout: "evadd": 0, 2023-07-18T03:39:47.386 INFO:teuthology.orchestra.run.smithi155.stdout: "evex": 0, 2023-07-18T03:39:47.387 INFO:teuthology.orchestra.run.smithi155.stdout: "evtrm": 0, 2023-07-18T03:39:47.387 INFO:teuthology.orchestra.run.smithi155.stdout: "ev": 0, 2023-07-18T03:39:47.387 INFO:teuthology.orchestra.run.smithi155.stdout: "evexg": 0, 2023-07-18T03:39:47.388 INFO:teuthology.orchestra.run.smithi155.stdout: "evexd": 0, 2023-07-18T03:39:47.388 INFO:teuthology.orchestra.run.smithi155.stdout: "segadd": 0, 2023-07-18T03:39:47.388 INFO:teuthology.orchestra.run.smithi155.stdout: "segex": 0, 2023-07-18T03:39:47.388 INFO:teuthology.orchestra.run.smithi155.stdout: "segtrm": 0, 2023-07-18T03:39:47.389 INFO:teuthology.orchestra.run.smithi155.stdout: "seg": 128, 2023-07-18T03:39:47.389 INFO:teuthology.orchestra.run.smithi155.stdout: "segexg": 0, 2023-07-18T03:39:47.389 INFO:teuthology.orchestra.run.smithi155.stdout: "segexd": 0, 2023-07-18T03:39:47.389 INFO:teuthology.orchestra.run.smithi155.stdout: "expos": 83886779, 2023-07-18T03:39:47.390 INFO:teuthology.orchestra.run.smithi155.stdout: "wrpos": 0, 2023-07-18T03:39:47.390 INFO:teuthology.orchestra.run.smithi155.stdout: "rdpos": 533831701, 2023-07-18T03:39:47.390 INFO:teuthology.orchestra.run.smithi155.stdout: "jlat": { 2023-07-18T03:39:47.390 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:47.390 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:47.390 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:47.391 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.391 INFO:teuthology.orchestra.run.smithi155.stdout: "replayed": 143900 2023-07-18T03:39:47.391 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.391 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_mem": { 2023-07-18T03:39:47.391 INFO:teuthology.orchestra.run.smithi155.stdout: "ino": 1202, 2023-07-18T03:39:47.392 INFO:teuthology.orchestra.run.smithi155.stdout: "ino+": 27762, 2023-07-18T03:39:47.392 INFO:teuthology.orchestra.run.smithi155.stdout: "ino-": 26560, 2023-07-18T03:39:47.392 INFO:teuthology.orchestra.run.smithi155.stdout: "dir": 96, 2023-07-18T03:39:47.392 INFO:teuthology.orchestra.run.smithi155.stdout: "dir+": 160, 2023-07-18T03:39:47.393 INFO:teuthology.orchestra.run.smithi155.stdout: "dir-": 64, 2023-07-18T03:39:47.393 INFO:teuthology.orchestra.run.smithi155.stdout: "dn": 22376, 2023-07-18T03:39:47.393 INFO:teuthology.orchestra.run.smithi155.stdout: "dn+": 27917, 2023-07-18T03:39:47.393 INFO:teuthology.orchestra.run.smithi155.stdout: "dn-": 5541, 2023-07-18T03:39:47.393 INFO:teuthology.orchestra.run.smithi155.stdout: "cap": 0, 2023-07-18T03:39:47.394 INFO:teuthology.orchestra.run.smithi155.stdout: "cap+": 0, 2023-07-18T03:39:47.394 INFO:teuthology.orchestra.run.smithi155.stdout: "cap-": 0, 2023-07-18T03:39:47.394 INFO:teuthology.orchestra.run.smithi155.stdout: "rss": 78168, 2023-07-18T03:39:47.394 INFO:teuthology.orchestra.run.smithi155.stdout: "heap": 207156 2023-07-18T03:39:47.394 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.394 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_server": { 2023-07-18T03:39:47.395 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_client_request": 0, 2023-07-18T03:39:47.395 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_server_request": 0, 2023-07-18T03:39:47.395 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_request": 0, 2023-07-18T03:39:47.395 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_session": 0, 2023-07-18T03:39:47.395 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_peer_request": 0, 2023-07-18T03:39:47.396 INFO:teuthology.orchestra.run.smithi155.stdout: "req_create_latency": { 2023-07-18T03:39:47.396 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:47.396 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:47.396 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:47.396 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.397 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getattr_latency": { 2023-07-18T03:39:47.397 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:47.397 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:47.397 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:47.398 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.398 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getfilelock_latency": { 2023-07-18T03:39:47.398 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:47.398 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:47.398 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:47.399 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.399 INFO:teuthology.orchestra.run.smithi155.stdout: "req_link_latency": { 2023-07-18T03:39:47.399 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:47.399 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:47.399 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:47.400 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.400 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookup_latency": { 2023-07-18T03:39:47.400 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:47.400 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:47.400 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:47.401 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.401 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookuphash_latency": { 2023-07-18T03:39:47.401 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:47.401 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:47.401 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:47.402 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.402 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupino_latency": { 2023-07-18T03:39:47.402 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:47.402 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:47.403 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:47.403 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.403 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupname_latency": { 2023-07-18T03:39:47.403 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:47.404 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:47.404 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:47.404 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.404 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupparent_latency": { 2023-07-18T03:39:47.405 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:47.405 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:47.405 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:47.405 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.405 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupsnap_latency": { 2023-07-18T03:39:47.406 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:47.406 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:47.406 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:47.406 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.407 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lssnap_latency": { 2023-07-18T03:39:47.407 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:47.407 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:47.407 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:47.408 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.408 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mkdir_latency": { 2023-07-18T03:39:47.408 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:47.409 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:47.409 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:47.409 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.409 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mknod_latency": { 2023-07-18T03:39:47.410 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:47.410 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:47.410 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:47.410 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.411 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mksnap_latency": { 2023-07-18T03:39:47.411 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:47.411 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:47.411 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:47.411 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.412 INFO:teuthology.orchestra.run.smithi155.stdout: "req_open_latency": { 2023-07-18T03:39:47.412 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:47.412 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:47.412 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:47.412 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.413 INFO:teuthology.orchestra.run.smithi155.stdout: "req_readdir_latency": { 2023-07-18T03:39:47.413 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:47.413 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:47.413 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:47.413 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.414 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rename_latency": { 2023-07-18T03:39:47.414 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:47.414 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:47.414 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:47.415 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.415 INFO:teuthology.orchestra.run.smithi155.stdout: "req_renamesnap_latency": { 2023-07-18T03:39:47.415 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:47.415 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:47.416 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:47.416 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.416 INFO:teuthology.orchestra.run.smithi155.stdout: "req_snapdiff_latency": { 2023-07-18T03:39:47.416 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:47.416 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:47.417 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:47.417 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.417 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmdir_latency": { 2023-07-18T03:39:47.417 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:47.417 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:47.418 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:47.418 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.418 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmsnap_latency": { 2023-07-18T03:39:47.418 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:47.418 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:47.419 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:47.419 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.419 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmxattr_latency": { 2023-07-18T03:39:47.419 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:47.419 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:47.420 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:47.420 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.420 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setattr_latency": { 2023-07-18T03:39:47.420 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:47.420 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:47.421 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:47.421 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.421 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setdirlayout_latency": { 2023-07-18T03:39:47.421 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:47.421 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:47.422 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:47.422 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.422 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setfilelock_latency": { 2023-07-18T03:39:47.422 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:47.423 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:47.423 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:47.423 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.423 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setlayout_latency": { 2023-07-18T03:39:47.423 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:47.424 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:47.424 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:47.424 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.424 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setxattr_latency": { 2023-07-18T03:39:47.425 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:47.425 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:47.425 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:47.425 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.425 INFO:teuthology.orchestra.run.smithi155.stdout: "req_symlink_latency": { 2023-07-18T03:39:47.426 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:47.426 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:47.426 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:47.426 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.426 INFO:teuthology.orchestra.run.smithi155.stdout: "req_unlink_latency": { 2023-07-18T03:39:47.426 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:47.427 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:47.427 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:47.427 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.427 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_revoke_eviction": 0, 2023-07-18T03:39:47.427 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_acquisition_throttle": 0, 2023-07-18T03:39:47.428 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getvxattr_latency": { 2023-07-18T03:39:47.428 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:47.428 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:47.428 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:47.429 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-18T03:39:47.429 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.429 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_sessions": { 2023-07-18T03:39:47.429 INFO:teuthology.orchestra.run.smithi155.stdout: "session_count": 1, 2023-07-18T03:39:47.429 INFO:teuthology.orchestra.run.smithi155.stdout: "session_add": 3, 2023-07-18T03:39:47.430 INFO:teuthology.orchestra.run.smithi155.stdout: "session_remove": 2, 2023-07-18T03:39:47.430 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_open": 2, 2023-07-18T03:39:47.430 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_stale": 0, 2023-07-18T03:39:47.430 INFO:teuthology.orchestra.run.smithi155.stdout: "total_load": 0, 2023-07-18T03:39:47.430 INFO:teuthology.orchestra.run.smithi155.stdout: "average_load": 0, 2023-07-18T03:39:47.431 INFO:teuthology.orchestra.run.smithi155.stdout: "avg_session_uptime": 6492 2023-07-18T03:39:47.431 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.431 INFO:teuthology.orchestra.run.smithi155.stdout: "mempool": { 2023-07-18T03:39:47.432 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_bytes": 0, 2023-07-18T03:39:47.432 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_items": 0, 2023-07-18T03:39:47.432 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_bytes": 0, 2023-07-18T03:39:47.432 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_items": 0, 2023-07-18T03:39:47.432 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_bytes": 0, 2023-07-18T03:39:47.433 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_items": 0, 2023-07-18T03:39:47.433 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-18T03:39:47.433 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_items": 0, 2023-07-18T03:39:47.433 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-18T03:39:47.434 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_items": 0, 2023-07-18T03:39:47.434 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_bytes": 0, 2023-07-18T03:39:47.434 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_items": 0, 2023-07-18T03:39:47.434 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-18T03:39:47.434 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_items": 0, 2023-07-18T03:39:47.435 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_bytes": 0, 2023-07-18T03:39:47.435 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_items": 0, 2023-07-18T03:39:47.435 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_bytes": 0, 2023-07-18T03:39:47.436 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_items": 0, 2023-07-18T03:39:47.436 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-18T03:39:47.436 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_items": 0, 2023-07-18T03:39:47.436 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-18T03:39:47.437 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_items": 0, 2023-07-18T03:39:47.437 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_bytes": 0, 2023-07-18T03:39:47.437 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_items": 0, 2023-07-18T03:39:47.437 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_bytes": 0, 2023-07-18T03:39:47.438 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_items": 0, 2023-07-18T03:39:47.438 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-18T03:39:47.438 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_items": 0, 2023-07-18T03:39:47.438 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_bytes": 0, 2023-07-18T03:39:47.438 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_items": 0, 2023-07-18T03:39:47.439 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_bytes": 0, 2023-07-18T03:39:47.439 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_items": 0, 2023-07-18T03:39:47.439 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_bytes": 0, 2023-07-18T03:39:47.440 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_items": 0, 2023-07-18T03:39:47.440 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_bytes": 0, 2023-07-18T03:39:47.440 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_items": 0, 2023-07-18T03:39:47.441 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_bytes": 289151, 2023-07-18T03:39:47.441 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_items": 1478, 2023-07-18T03:39:47.441 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_bytes": 0, 2023-07-18T03:39:47.441 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_items": 0, 2023-07-18T03:39:47.442 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_bytes": 0, 2023-07-18T03:39:47.442 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_items": 0, 2023-07-18T03:39:47.442 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_bytes": 0, 2023-07-18T03:39:47.442 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_items": 0, 2023-07-18T03:39:47.442 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_bytes": 0, 2023-07-18T03:39:47.443 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_items": 0, 2023-07-18T03:39:47.443 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_bytes": 9616, 2023-07-18T03:39:47.443 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_items": 185, 2023-07-18T03:39:47.443 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_bytes": 0, 2023-07-18T03:39:47.443 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_items": 0, 2023-07-18T03:39:47.443 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_bytes": 0, 2023-07-18T03:39:47.444 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_items": 0, 2023-07-18T03:39:47.444 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_bytes": 18962519, 2023-07-18T03:39:47.444 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_items": 122831, 2023-07-18T03:39:47.444 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_bytes": 0, 2023-07-18T03:39:47.444 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_items": 0, 2023-07-18T03:39:47.445 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_bytes": 0, 2023-07-18T03:39:47.445 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_items": 0 2023-07-18T03:39:47.445 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.445 INFO:teuthology.orchestra.run.smithi155.stdout: "objecter": { 2023-07-18T03:39:47.445 INFO:teuthology.orchestra.run.smithi155.stdout: "op_active": 0, 2023-07-18T03:39:47.446 INFO:teuthology.orchestra.run.smithi155.stdout: "op_laggy": 0, 2023-07-18T03:39:47.446 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send": 7196, 2023-07-18T03:39:47.446 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send_bytes": 21, 2023-07-18T03:39:47.446 INFO:teuthology.orchestra.run.smithi155.stdout: "op_resend": 3, 2023-07-18T03:39:47.446 INFO:teuthology.orchestra.run.smithi155.stdout: "op_reply": 7196, 2023-07-18T03:39:47.447 INFO:teuthology.orchestra.run.smithi155.stdout: "op_latency": { 2023-07-18T03:39:47.447 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 7196, 2023-07-18T03:39:47.447 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 10.142792468, 2023-07-18T03:39:47.447 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.001409504 2023-07-18T03:39:47.447 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.448 INFO:teuthology.orchestra.run.smithi155.stdout: "op_inflight": 0, 2023-07-18T03:39:47.448 INFO:teuthology.orchestra.run.smithi155.stdout: "oplen_avg": { 2023-07-18T03:39:47.448 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 7196, 2023-07-18T03:39:47.448 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 7197 2023-07-18T03:39:47.449 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.449 INFO:teuthology.orchestra.run.smithi155.stdout: "op": 7196, 2023-07-18T03:39:47.449 INFO:teuthology.orchestra.run.smithi155.stdout: "op_r": 7196, 2023-07-18T03:39:47.449 INFO:teuthology.orchestra.run.smithi155.stdout: "op_w": 0, 2023-07-18T03:39:47.449 INFO:teuthology.orchestra.run.smithi155.stdout: "op_rmw": 0, 2023-07-18T03:39:47.450 INFO:teuthology.orchestra.run.smithi155.stdout: "op_pg": 0, 2023-07-18T03:39:47.450 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_stat": 4235, 2023-07-18T03:39:47.450 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_create": 0, 2023-07-18T03:39:47.450 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_read": 2959, 2023-07-18T03:39:47.450 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_write": 0, 2023-07-18T03:39:47.451 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writefull": 0, 2023-07-18T03:39:47.451 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writesame": 0, 2023-07-18T03:39:47.451 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_append": 0, 2023-07-18T03:39:47.451 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_zero": 0, 2023-07-18T03:39:47.451 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_truncate": 0, 2023-07-18T03:39:47.452 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_delete": 0, 2023-07-18T03:39:47.452 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_mapext": 0, 2023-07-18T03:39:47.452 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_sparse_read": 0, 2023-07-18T03:39:47.452 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_clonerange": 0, 2023-07-18T03:39:47.452 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_getxattr": 1, 2023-07-18T03:39:47.453 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_setxattr": 0, 2023-07-18T03:39:47.453 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_cmpxattr": 0, 2023-07-18T03:39:47.453 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_rmxattr": 0, 2023-07-18T03:39:47.453 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_resetxattrs": 0, 2023-07-18T03:39:47.453 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_call": 0, 2023-07-18T03:39:47.454 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_watch": 0, 2023-07-18T03:39:47.454 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_notify": 0, 2023-07-18T03:39:47.454 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_src_cmpxattr": 0, 2023-07-18T03:39:47.454 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls": 0, 2023-07-18T03:39:47.454 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls_filter": 0, 2023-07-18T03:39:47.454 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_other": 0, 2023-07-18T03:39:47.455 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_active": 0, 2023-07-18T03:39:47.455 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_send": 0, 2023-07-18T03:39:47.455 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_resend": 0, 2023-07-18T03:39:47.455 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_ping": 0, 2023-07-18T03:39:47.455 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_active": 0, 2023-07-18T03:39:47.456 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_send": 0, 2023-07-18T03:39:47.456 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_resend": 0, 2023-07-18T03:39:47.456 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_active": 0, 2023-07-18T03:39:47.456 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_send": 0, 2023-07-18T03:39:47.457 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_resend": 0, 2023-07-18T03:39:47.457 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_active": 0, 2023-07-18T03:39:47.457 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_send": 0, 2023-07-18T03:39:47.457 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_resend": 0, 2023-07-18T03:39:47.457 INFO:teuthology.orchestra.run.smithi155.stdout: "command_active": 0, 2023-07-18T03:39:47.458 INFO:teuthology.orchestra.run.smithi155.stdout: "command_send": 0, 2023-07-18T03:39:47.458 INFO:teuthology.orchestra.run.smithi155.stdout: "command_resend": 0, 2023-07-18T03:39:47.458 INFO:teuthology.orchestra.run.smithi155.stdout: "map_epoch": 144, 2023-07-18T03:39:47.458 INFO:teuthology.orchestra.run.smithi155.stdout: "map_full": 0, 2023-07-18T03:39:47.459 INFO:teuthology.orchestra.run.smithi155.stdout: "map_inc": 55, 2023-07-18T03:39:47.459 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_sessions": 11, 2023-07-18T03:39:47.459 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_open": 11, 2023-07-18T03:39:47.459 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_close": 0, 2023-07-18T03:39:47.460 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_laggy": 0, 2023-07-18T03:39:47.460 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_wr": 0, 2023-07-18T03:39:47.460 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_rd": 2, 2023-07-18T03:39:47.460 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_del": 0 2023-07-18T03:39:47.461 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.461 INFO:teuthology.orchestra.run.smithi155.stdout: "oft": { 2023-07-18T03:39:47.461 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_objs": 0, 2023-07-18T03:39:47.461 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_kv_pairs": 0, 2023-07-18T03:39:47.461 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_updates": 0, 2023-07-18T03:39:47.462 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_removes": 0 2023-07-18T03:39:47.462 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.462 INFO:teuthology.orchestra.run.smithi155.stdout: "osd-slow-ops": { 2023-07-18T03:39:47.462 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_ops_count": 0 2023-07-18T03:39:47.463 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.463 INFO:teuthology.orchestra.run.smithi155.stdout: "purge_queue": { 2023-07-18T03:39:47.463 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops": 0, 2023-07-18T03:39:47.463 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops_high_water": 0, 2023-07-18T03:39:47.464 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing": 0, 2023-07-18T03:39:47.464 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_high_water": 0, 2023-07-18T03:39:47.464 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed": 0, 2023-07-18T03:39:47.465 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_item_in_journal": 0 2023-07-18T03:39:47.465 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.465 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-18T03:39:47.465 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2023-07-18T03:39:47.466 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2023-07-18T03:39:47.466 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2023-07-18T03:39:47.466 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 7851, 2023-07-18T03:39:47.466 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 532621335, 2023-07-18T03:39:47.467 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2023-07-18T03:39:47.467 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 7851, 2023-07-18T03:39:47.467 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2023-07-18T03:39:47.467 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2023-07-18T03:39:47.467 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 7851, 2023-07-18T03:39:47.467 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 532621335, 2023-07-18T03:39:47.468 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2023-07-18T03:39:47.468 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:47.468 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:47.468 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:47.469 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-18T03:39:47.469 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.469 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_bytes": { 2023-07-18T03:39:47.469 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2023-07-18T03:39:47.470 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2023-07-18T03:39:47.470 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2023-07-18T03:39:47.470 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2023-07-18T03:39:47.470 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2023-07-18T03:39:47.470 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2023-07-18T03:39:47.471 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2023-07-18T03:39:47.471 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 7196, 2023-07-18T03:39:47.471 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 529661545, 2023-07-18T03:39:47.471 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 7196, 2023-07-18T03:39:47.471 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 529661545, 2023-07-18T03:39:47.472 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2023-07-18T03:39:47.472 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:47.472 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:47.472 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:47.472 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-18T03:39:47.473 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.473 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_ops": { 2023-07-18T03:39:47.473 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2023-07-18T03:39:47.473 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 1024, 2023-07-18T03:39:47.473 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2023-07-18T03:39:47.474 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2023-07-18T03:39:47.474 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2023-07-18T03:39:47.474 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2023-07-18T03:39:47.474 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2023-07-18T03:39:47.474 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 7196, 2023-07-18T03:39:47.475 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 7196, 2023-07-18T03:39:47.475 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 7196, 2023-07-18T03:39:47.475 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 7196, 2023-07-18T03:39:47.475 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2023-07-18T03:39:47.475 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:47.476 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:47.476 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:47.476 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-18T03:39:47.476 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.476 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle": { 2023-07-18T03:39:47.477 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2023-07-18T03:39:47.477 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2023-07-18T03:39:47.477 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2023-07-18T03:39:47.477 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2023-07-18T03:39:47.477 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2023-07-18T03:39:47.477 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2023-07-18T03:39:47.478 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2023-07-18T03:39:47.478 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2023-07-18T03:39:47.478 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2023-07-18T03:39:47.478 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 0, 2023-07-18T03:39:47.478 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 0, 2023-07-18T03:39:47.479 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2023-07-18T03:39:47.479 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:47.479 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:47.479 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:47.479 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-18T03:39:47.480 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:47.480 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle-0x5596e51e2460": { 2023-07-18T03:39:47.480 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2023-07-18T03:39:47.480 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2023-07-18T03:39:47.480 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2023-07-18T03:39:47.481 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2023-07-18T03:39:47.481 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2023-07-18T03:39:47.481 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2023-07-18T03:39:47.481 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2023-07-18T03:39:47.481 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2023-07-18T03:39:47.482 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2023-07-18T03:39:47.482 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 0, 2023-07-18T03:39:47.482 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 0, 2023-07-18T03:39:47.482 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2023-07-18T03:39:47.482 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:47.483 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:47.483 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:47.483 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-18T03:39:47.483 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-18T03:39:47.483 INFO:teuthology.orchestra.run.smithi155.stdout:} 2023-07-18T03:39:47.674 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:39:47 smithi155 ceph-mon[128885]: pgmap v1021: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:39:47.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:39:47 smithi101 ceph-mon[126379]: pgmap v1021: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:39:47.742 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2023-07-18T03:39:47.742 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2023-07-18T03:39:47.743 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2023-07-18T03:39:47.743 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2023-07-18T03:39:47.743 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2023-07-18T03:39:47.744 DEBUG:tasks.check_counter:Getting stats from k 2023-07-18T03:39:47.744 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph daemon mds.k perf dump 2023-07-18T03:39:47.951 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:39:47 smithi179 ceph-mon[129229]: pgmap v1021: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:39:48.324 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.b/config 2023-07-18T03:39:49.557 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2023-07-18T03:39:49.558 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-0": { 2023-07-18T03:39:49.558 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 571, 2023-07-18T03:39:49.558 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 561, 2023-07-18T03:39:49.558 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 1522368, 2023-07-18T03:39:49.559 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 322288, 2023-07-18T03:39:49.559 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 4, 2023-07-18T03:39:49.559 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 0, 2023-07-18T03:39:49.559 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.073445197, 2023-07-18T03:39:49.559 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.024788833, 2023-07-18T03:39:49.560 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.039800910, 2023-07-18T03:39:49.560 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.012804416, 2023-07-18T03:39:49.560 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:39:49.560 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 560, 2023-07-18T03:39:49.560 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.022353941, 2023-07-18T03:39:49.561 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000039917 2023-07-18T03:39:49.561 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:49.561 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:39:49.561 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:49.561 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:49.562 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:49.562 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:49.562 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 1522368, 2023-07-18T03:39:49.562 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 322288 2023-07-18T03:39:49.562 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:49.563 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-1": { 2023-07-18T03:39:49.563 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 8, 2023-07-18T03:39:49.563 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 433, 2023-07-18T03:39:49.563 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 3584, 2023-07-18T03:39:49.563 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 379456, 2023-07-18T03:39:49.564 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 7, 2023-07-18T03:39:49.564 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 3, 2023-07-18T03:39:49.564 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.018363690, 2023-07-18T03:39:49.564 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.012593745, 2023-07-18T03:39:49.564 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.002228760, 2023-07-18T03:39:49.565 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-18T03:39:49.565 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:39:49.565 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 431, 2023-07-18T03:39:49.565 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.031349893, 2023-07-18T03:39:49.565 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000072737 2023-07-18T03:39:49.566 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:49.566 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:39:49.566 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:49.566 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:49.566 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:49.567 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:49.567 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 3584, 2023-07-18T03:39:49.567 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 379456 2023-07-18T03:39:49.567 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:49.567 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-2": { 2023-07-18T03:39:49.567 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 0, 2023-07-18T03:39:49.568 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 2, 2023-07-18T03:39:49.568 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 0, 2023-07-18T03:39:49.568 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 0, 2023-07-18T03:39:49.568 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 2, 2023-07-18T03:39:49.568 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 1, 2023-07-18T03:39:49.569 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.000355206, 2023-07-18T03:39:49.569 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.000000000, 2023-07-18T03:39:49.569 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.000274242, 2023-07-18T03:39:49.569 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-18T03:39:49.569 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:39:49.570 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:49.570 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:49.570 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:49.570 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:49.570 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:39:49.571 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:49.571 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:49.571 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:49.571 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:49.571 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-18T03:39:49.572 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 0 2023-07-18T03:39:49.572 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:49.572 INFO:teuthology.orchestra.run.smithi155.stdout: "cct": { 2023-07-18T03:39:49.572 INFO:teuthology.orchestra.run.smithi155.stdout: "total_workers": 0, 2023-07-18T03:39:49.572 INFO:teuthology.orchestra.run.smithi155.stdout: "unhealthy_workers": 0 2023-07-18T03:39:49.573 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:49.573 INFO:teuthology.orchestra.run.smithi155.stdout: "mempool": { 2023-07-18T03:39:49.573 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_bytes": 0, 2023-07-18T03:39:49.573 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_items": 0, 2023-07-18T03:39:49.573 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_bytes": 0, 2023-07-18T03:39:49.574 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_items": 0, 2023-07-18T03:39:49.574 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_bytes": 0, 2023-07-18T03:39:49.574 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_items": 0, 2023-07-18T03:39:49.574 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-18T03:39:49.574 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_items": 0, 2023-07-18T03:39:49.575 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-18T03:39:49.575 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_items": 0, 2023-07-18T03:39:49.575 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_bytes": 0, 2023-07-18T03:39:49.575 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_items": 0, 2023-07-18T03:39:49.575 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-18T03:39:49.575 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_items": 0, 2023-07-18T03:39:49.576 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_bytes": 0, 2023-07-18T03:39:49.576 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_items": 0, 2023-07-18T03:39:49.576 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_bytes": 0, 2023-07-18T03:39:49.576 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_items": 0, 2023-07-18T03:39:49.576 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-18T03:39:49.577 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_items": 0, 2023-07-18T03:39:49.577 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-18T03:39:49.577 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_items": 0, 2023-07-18T03:39:49.577 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_bytes": 0, 2023-07-18T03:39:49.577 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_items": 0, 2023-07-18T03:39:49.578 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_bytes": 0, 2023-07-18T03:39:49.578 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_items": 0, 2023-07-18T03:39:49.578 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-18T03:39:49.578 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_items": 0, 2023-07-18T03:39:49.578 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_bytes": 0, 2023-07-18T03:39:49.579 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_items": 0, 2023-07-18T03:39:49.579 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_bytes": 0, 2023-07-18T03:39:49.579 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_items": 0, 2023-07-18T03:39:49.579 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_bytes": 0, 2023-07-18T03:39:49.579 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_items": 0, 2023-07-18T03:39:49.580 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_bytes": 0, 2023-07-18T03:39:49.580 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_items": 0, 2023-07-18T03:39:49.580 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_bytes": 147947, 2023-07-18T03:39:49.580 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_items": 29, 2023-07-18T03:39:49.580 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_bytes": 0, 2023-07-18T03:39:49.581 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_items": 0, 2023-07-18T03:39:49.581 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_bytes": 0, 2023-07-18T03:39:49.581 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_items": 0, 2023-07-18T03:39:49.581 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_bytes": 0, 2023-07-18T03:39:49.581 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_items": 0, 2023-07-18T03:39:49.582 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_bytes": 0, 2023-07-18T03:39:49.582 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_items": 0, 2023-07-18T03:39:49.582 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_bytes": 0, 2023-07-18T03:39:49.582 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_items": 0, 2023-07-18T03:39:49.582 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_bytes": 0, 2023-07-18T03:39:49.583 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_items": 0, 2023-07-18T03:39:49.583 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_bytes": 0, 2023-07-18T03:39:49.583 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_items": 0, 2023-07-18T03:39:49.583 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_bytes": 928, 2023-07-18T03:39:49.583 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_items": 2, 2023-07-18T03:39:49.583 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_bytes": 0, 2023-07-18T03:39:49.584 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_items": 0, 2023-07-18T03:39:49.584 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_bytes": 0, 2023-07-18T03:39:49.584 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_items": 0 2023-07-18T03:39:49.584 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-07-18T03:39:49.584 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-18T03:39:49.585 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2023-07-18T03:39:49.585 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2023-07-18T03:39:49.585 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2023-07-18T03:39:49.585 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 575, 2023-07-18T03:39:49.585 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 1441655, 2023-07-18T03:39:49.586 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2023-07-18T03:39:49.586 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 575, 2023-07-18T03:39:49.586 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2023-07-18T03:39:49.586 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2023-07-18T03:39:49.586 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 575, 2023-07-18T03:39:49.587 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 1441655, 2023-07-18T03:39:49.587 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2023-07-18T03:39:49.587 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2023-07-18T03:39:49.587 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2023-07-18T03:39:49.587 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2023-07-18T03:39:49.588 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-18T03:39:49.588 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-07-18T03:39:49.588 INFO:teuthology.orchestra.run.smithi155.stdout:} 2023-07-18T03:39:49.689 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:39:49 smithi155 ceph-mon[128885]: pgmap v1022: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:39:49.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:39:49 smithi101 ceph-mon[126379]: pgmap v1022: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:39:49.951 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:39:49 smithi179 ceph-mon[129229]: pgmap v1022: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:39:49.990 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2023-07-18T03:39:49.991 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2023-07-18T03:39:49.992 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2023-07-18T03:39:49.992 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2023-07-18T03:39:49.992 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2023-07-18T03:39:49.992 DEBUG:tasks.check_counter:Getting stats from c 2023-07-18T03:39:49.993 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph daemon mds.c perf dump 2023-07-18T03:39:50.789 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.c/config 2023-07-18T03:39:51.653 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:39:51 smithi179 ceph-mon[129229]: pgmap v1023: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:39:51.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:39:51 smithi101 ceph-mon[126379]: pgmap v1023: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:39:51.796 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:39:51 smithi155 ceph-mon[128885]: pgmap v1023: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:39:52.273 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2023-07-18T03:39:52.273 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-0": { 2023-07-18T03:39:52.273 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 410639, 2023-07-18T03:39:52.273 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 546490, 2023-07-18T03:39:52.274 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 143639912, 2023-07-18T03:39:52.274 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 509895850, 2023-07-18T03:39:52.274 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 65, 2023-07-18T03:39:52.274 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 6, 2023-07-18T03:39:52.274 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 25.093983440, 2023-07-18T03:39:52.275 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 11.499074883, 2023-07-18T03:39:52.275 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 16.563464649, 2023-07-18T03:39:52.275 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.946758127, 2023-07-18T03:39:52.275 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:39:52.275 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 546490, 2023-07-18T03:39:52.276 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 74.947490918, 2023-07-18T03:39:52.276 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000137143 2023-07-18T03:39:52.276 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.276 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:39:52.277 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 785154, 2023-07-18T03:39:52.277 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.135241512, 2023-07-18T03:39:52.277 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000172 2023-07-18T03:39:52.277 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.277 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 77847456, 2023-07-18T03:39:52.278 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 299211728 2023-07-18T03:39:52.278 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.278 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-1": { 2023-07-18T03:39:52.278 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 7489347, 2023-07-18T03:39:52.278 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 7489234, 2023-07-18T03:39:52.279 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 1192772847, 2023-07-18T03:39:52.279 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 1469869104, 2023-07-18T03:39:52.279 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 16, 2023-07-18T03:39:52.279 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 5, 2023-07-18T03:39:52.279 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 173.708696158, 2023-07-18T03:39:52.279 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 67.699459859, 2023-07-18T03:39:52.280 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 115.274186903, 2023-07-18T03:39:52.280 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 1.300565334, 2023-07-18T03:39:52.280 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:39:52.280 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 7489234, 2023-07-18T03:39:52.281 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 72.793193153, 2023-07-18T03:39:52.281 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000009719 2023-07-18T03:39:52.281 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.281 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:39:52.281 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 14893039, 2023-07-18T03:39:52.282 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 1.093676853, 2023-07-18T03:39:52.282 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000073 2023-07-18T03:39:52.282 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.282 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 1808, 2023-07-18T03:39:52.282 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 129552 2023-07-18T03:39:52.283 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.283 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-2": { 2023-07-18T03:39:52.283 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 25579, 2023-07-18T03:39:52.283 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 25943, 2023-07-18T03:39:52.283 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 10879374, 2023-07-18T03:39:52.284 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 93839549, 2023-07-18T03:39:52.284 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 16, 2023-07-18T03:39:52.284 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 8, 2023-07-18T03:39:52.284 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 2.022842463, 2023-07-18T03:39:52.284 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.568959572, 2023-07-18T03:39:52.285 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.435898366, 2023-07-18T03:39:52.285 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.914261112, 2023-07-18T03:39:52.285 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:39:52.285 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 25938, 2023-07-18T03:39:52.285 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 5.681409247, 2023-07-18T03:39:52.286 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000219038 2023-07-18T03:39:52.286 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.286 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:39:52.286 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 23, 2023-07-18T03:39:52.286 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000002870, 2023-07-18T03:39:52.287 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000124 2023-07-18T03:39:52.287 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.287 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 1726976, 2023-07-18T03:39:52.287 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 1227792 2023-07-18T03:39:52.287 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.287 INFO:teuthology.orchestra.run.smithi179.stdout: "cct": { 2023-07-18T03:39:52.288 INFO:teuthology.orchestra.run.smithi179.stdout: "total_workers": 1, 2023-07-18T03:39:52.288 INFO:teuthology.orchestra.run.smithi179.stdout: "unhealthy_workers": 0 2023-07-18T03:39:52.288 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.288 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-MDSRank": { 2023-07-18T03:39:52.289 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2023-07-18T03:39:52.289 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2023-07-18T03:39:52.289 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 64027, 2023-07-18T03:39:52.289 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 96.393721099, 2023-07-18T03:39:52.290 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.001505516 2023-07-18T03:39:52.290 INFO:teuthology.orchestra.run.smithi179.stdout: } 2023-07-18T03:39:52.290 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.290 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-PurgeQueue": { 2023-07-18T03:39:52.291 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2023-07-18T03:39:52.291 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2023-07-18T03:39:52.292 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 44409, 2023-07-18T03:39:52.292 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 11.293264527, 2023-07-18T03:39:52.292 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000254301 2023-07-18T03:39:52.293 INFO:teuthology.orchestra.run.smithi179.stdout: } 2023-07-18T03:39:52.293 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.293 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": { 2023-07-18T03:39:52.294 INFO:teuthology.orchestra.run.smithi179.stdout: "request": 161541, 2023-07-18T03:39:52.294 INFO:teuthology.orchestra.run.smithi179.stdout: "reply": 155259, 2023-07-18T03:39:52.294 INFO:teuthology.orchestra.run.smithi179.stdout: "reply_latency": { 2023-07-18T03:39:52.295 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 155259, 2023-07-18T03:39:52.295 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 725.026617598, 2023-07-18T03:39:52.295 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.004669788 2023-07-18T03:39:52.295 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.296 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_reply": 0, 2023-07-18T03:39:52.296 INFO:teuthology.orchestra.run.smithi179.stdout: "forward": 6282, 2023-07-18T03:39:52.296 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_complete": 1, 2023-07-18T03:39:52.296 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_keys": 0, 2023-07-18T03:39:52.296 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_commit": 112, 2023-07-18T03:39:52.297 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_split": 11, 2023-07-18T03:39:52.297 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_merge": 11, 2023-07-18T03:39:52.297 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes": 3170, 2023-07-18T03:39:52.297 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_top": 933, 2023-07-18T03:39:52.297 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_bottom": 2078, 2023-07-18T03:39:52.298 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pin_tail": 159, 2023-07-18T03:39:52.298 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pinned": 1837, 2023-07-18T03:39:52.299 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_expired": 1107, 2023-07-18T03:39:52.299 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_with_caps": 191, 2023-07-18T03:39:52.299 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": 191, 2023-07-18T03:39:52.299 INFO:teuthology.orchestra.run.smithi179.stdout: "subtrees": 23, 2023-07-18T03:39:52.299 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse": 271312, 2023-07-18T03:39:52.300 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_hit": 147008, 2023-07-18T03:39:52.300 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_forward": 6255, 2023-07-18T03:39:52.300 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_discover": 66, 2023-07-18T03:39:52.300 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_dir_fetch": 0, 2023-07-18T03:39:52.300 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_remote_ino": 0, 2023-07-18T03:39:52.301 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_lock": 2734, 2023-07-18T03:39:52.301 INFO:teuthology.orchestra.run.smithi179.stdout: "load_cent": 0, 2023-07-18T03:39:52.301 INFO:teuthology.orchestra.run.smithi179.stdout: "q": 0, 2023-07-18T03:39:52.301 INFO:teuthology.orchestra.run.smithi179.stdout: "exported": 18, 2023-07-18T03:39:52.301 INFO:teuthology.orchestra.run.smithi179.stdout: "exported_inodes": 822, 2023-07-18T03:39:52.302 INFO:teuthology.orchestra.run.smithi179.stdout: "imported": 29, 2023-07-18T03:39:52.302 INFO:teuthology.orchestra.run.smithi179.stdout: "imported_inodes": 1813, 2023-07-18T03:39:52.302 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_dir_fetch": 0, 2023-07-18T03:39:52.302 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_backtrace_fetch": 2541, 2023-07-18T03:39:52.302 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_peer_discover": 1, 2023-07-18T03:39:52.303 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rfiles": 679, 2023-07-18T03:39:52.303 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rbytes": 221129114, 2023-07-18T03:39:52.303 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rsnaps": 6, 2023-07-18T03:39:52.304 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_fetch": 2541, 2023-07-18T03:39:52.304 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_set_tag": 0, 2023-07-18T03:39:52.304 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_repaired": 0, 2023-07-18T03:39:52.304 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_inotable_repaired": 0, 2023-07-18T03:39:52.304 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_inodes": 279, 2023-07-18T03:39:52.305 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_base_inodes": 0, 2023-07-18T03:39:52.305 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dirfrag_rstats": 279, 2023-07-18T03:39:52.305 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_file_inodes": 2262, 2023-07-18T03:39:52.305 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_inode_file_caps": 41, 2023-07-18T03:39:52.305 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_revoke": 11418, 2023-07-18T03:39:52.306 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_grant": 109060, 2023-07-18T03:39:52.306 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_trunc": 0, 2023-07-18T03:39:52.306 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flushsnap_ack": 1192, 2023-07-18T03:39:52.306 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-18T03:39:52.306 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps": 37792, 2023-07-18T03:39:52.307 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps_dirty": 17763, 2023-07-18T03:39:52.307 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_cap_release": 4121, 2023-07-18T03:39:52.307 INFO:teuthology.orchestra.run.smithi179.stdout: "process_request_cap_release": 120275 2023-07-18T03:39:52.307 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.307 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_cache": { 2023-07-18T03:39:52.308 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update": 47, 2023-07-18T03:39:52.308 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update_receipt": 47, 2023-07-18T03:39:52.308 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_try_discover": 31, 2023-07-18T03:39:52.308 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_send_discover": 96, 2023-07-18T03:39:52.309 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_handle_discover": 102, 2023-07-18T03:39:52.309 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays": 0, 2023-07-18T03:39:52.309 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_delayed": 0, 2023-07-18T03:39:52.309 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_enqueuing": 0, 2023-07-18T03:39:52.309 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_created": 26716, 2023-07-18T03:39:52.310 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_enqueued": 26716, 2023-07-18T03:39:52.310 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_reintegrated": 0, 2023-07-18T03:39:52.310 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_migrated": 0, 2023-07-18T03:39:52.310 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_processing": 0, 2023-07-18T03:39:52.310 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_enqueued": 0, 2023-07-18T03:39:52.310 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_prioritized": 0, 2023-07-18T03:39:52.311 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_started": 0, 2023-07-18T03:39:52.311 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_completed": 0, 2023-07-18T03:39:52.311 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_enqueue_scrub": 0, 2023-07-18T03:39:52.311 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_exportdir": 23, 2023-07-18T03:39:52.311 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_flush": 0, 2023-07-18T03:39:52.312 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragmentdir": 22, 2023-07-18T03:39:52.312 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragstats": 0, 2023-07-18T03:39:52.312 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_inodestats": 0 2023-07-18T03:39:52.312 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.313 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_log": { 2023-07-18T03:39:52.313 INFO:teuthology.orchestra.run.smithi179.stdout: "evadd": 127224, 2023-07-18T03:39:52.313 INFO:teuthology.orchestra.run.smithi179.stdout: "evex": 35457, 2023-07-18T03:39:52.313 INFO:teuthology.orchestra.run.smithi179.stdout: "evtrm": 35457, 2023-07-18T03:39:52.313 INFO:teuthology.orchestra.run.smithi179.stdout: "ev": 91767, 2023-07-18T03:39:52.314 INFO:teuthology.orchestra.run.smithi179.stdout: "evexg": 0, 2023-07-18T03:39:52.314 INFO:teuthology.orchestra.run.smithi179.stdout: "evexd": 1024, 2023-07-18T03:39:52.314 INFO:teuthology.orchestra.run.smithi179.stdout: "segadd": 171, 2023-07-18T03:39:52.314 INFO:teuthology.orchestra.run.smithi179.stdout: "segex": 43, 2023-07-18T03:39:52.314 INFO:teuthology.orchestra.run.smithi179.stdout: "segtrm": 43, 2023-07-18T03:39:52.315 INFO:teuthology.orchestra.run.smithi179.stdout: "seg": 129, 2023-07-18T03:39:52.315 INFO:teuthology.orchestra.run.smithi179.stdout: "segexg": 0, 2023-07-18T03:39:52.315 INFO:teuthology.orchestra.run.smithi179.stdout: "segexd": 1, 2023-07-18T03:39:52.315 INFO:teuthology.orchestra.run.smithi179.stdout: "expos": 65195813, 2023-07-18T03:39:52.316 INFO:teuthology.orchestra.run.smithi179.stdout: "wrpos": 505865621, 2023-07-18T03:39:52.316 INFO:teuthology.orchestra.run.smithi179.stdout: "rdpos": 0, 2023-07-18T03:39:52.316 INFO:teuthology.orchestra.run.smithi179.stdout: "jlat": { 2023-07-18T03:39:52.317 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 35010, 2023-07-18T03:39:52.317 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 257.683347728, 2023-07-18T03:39:52.317 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.007360278 2023-07-18T03:39:52.318 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.318 INFO:teuthology.orchestra.run.smithi179.stdout: "replayed": 0 2023-07-18T03:39:52.318 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.319 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_mem": { 2023-07-18T03:39:52.319 INFO:teuthology.orchestra.run.smithi179.stdout: "ino": 1625, 2023-07-18T03:39:52.319 INFO:teuthology.orchestra.run.smithi179.stdout: "ino+": 29112, 2023-07-18T03:39:52.319 INFO:teuthology.orchestra.run.smithi179.stdout: "ino-": 27487, 2023-07-18T03:39:52.320 INFO:teuthology.orchestra.run.smithi179.stdout: "dir": 198, 2023-07-18T03:39:52.320 INFO:teuthology.orchestra.run.smithi179.stdout: "dir+": 368, 2023-07-18T03:39:52.320 INFO:teuthology.orchestra.run.smithi179.stdout: "dir-": 170, 2023-07-18T03:39:52.321 INFO:teuthology.orchestra.run.smithi179.stdout: "dn": 3170, 2023-07-18T03:39:52.321 INFO:teuthology.orchestra.run.smithi179.stdout: "dn+": 49878, 2023-07-18T03:39:52.321 INFO:teuthology.orchestra.run.smithi179.stdout: "dn-": 46708, 2023-07-18T03:39:52.321 INFO:teuthology.orchestra.run.smithi179.stdout: "cap": 191, 2023-07-18T03:39:52.322 INFO:teuthology.orchestra.run.smithi179.stdout: "cap+": 28392, 2023-07-18T03:39:52.322 INFO:teuthology.orchestra.run.smithi179.stdout: "cap-": 28201, 2023-07-18T03:39:52.322 INFO:teuthology.orchestra.run.smithi179.stdout: "rss": 77888, 2023-07-18T03:39:52.322 INFO:teuthology.orchestra.run.smithi179.stdout: "heap": 198964 2023-07-18T03:39:52.323 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.323 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_server": { 2023-07-18T03:39:52.323 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_client_request": 246641, 2023-07-18T03:39:52.323 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_server_request": 1460, 2023-07-18T03:39:52.323 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_request": 161541, 2023-07-18T03:39:52.324 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_session": 2585, 2023-07-18T03:39:52.324 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_peer_request": 16857, 2023-07-18T03:39:52.324 INFO:teuthology.orchestra.run.smithi179.stdout: "req_create_latency": { 2023-07-18T03:39:52.324 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 26708, 2023-07-18T03:39:52.325 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 114.065109138, 2023-07-18T03:39:52.325 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.004270821 2023-07-18T03:39:52.325 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.325 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getattr_latency": { 2023-07-18T03:39:52.326 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 2733, 2023-07-18T03:39:52.326 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 48.236032116, 2023-07-18T03:39:52.326 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.017649481 2023-07-18T03:39:52.326 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.327 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getfilelock_latency": { 2023-07-18T03:39:52.327 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:52.327 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:52.327 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:52.327 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.328 INFO:teuthology.orchestra.run.smithi179.stdout: "req_link_latency": { 2023-07-18T03:39:52.328 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:52.328 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:52.328 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:52.328 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.329 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookup_latency": { 2023-07-18T03:39:52.329 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 40851, 2023-07-18T03:39:52.329 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 118.662961853, 2023-07-18T03:39:52.329 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.002904774 2023-07-18T03:39:52.330 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.330 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookuphash_latency": { 2023-07-18T03:39:52.330 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:52.330 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:52.330 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:52.331 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.331 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupino_latency": { 2023-07-18T03:39:52.331 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:52.331 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:52.332 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:52.332 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.332 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupname_latency": { 2023-07-18T03:39:52.332 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:52.332 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:52.333 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:52.333 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.333 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupparent_latency": { 2023-07-18T03:39:52.333 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:52.333 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:52.334 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:52.334 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.334 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupsnap_latency": { 2023-07-18T03:39:52.334 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:52.334 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:52.335 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:52.335 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.335 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lssnap_latency": { 2023-07-18T03:39:52.335 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:52.335 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:52.336 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:52.336 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.336 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mkdir_latency": { 2023-07-18T03:39:52.336 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:52.337 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:52.337 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:52.337 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.337 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mknod_latency": { 2023-07-18T03:39:52.337 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:52.338 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:52.338 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:52.338 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.338 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mksnap_latency": { 2023-07-18T03:39:52.338 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:52.339 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:52.339 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:52.339 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.339 INFO:teuthology.orchestra.run.smithi179.stdout: "req_open_latency": { 2023-07-18T03:39:52.339 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 4, 2023-07-18T03:39:52.340 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.003807158, 2023-07-18T03:39:52.340 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000951789 2023-07-18T03:39:52.340 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.340 INFO:teuthology.orchestra.run.smithi179.stdout: "req_readdir_latency": { 2023-07-18T03:39:52.340 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 52063, 2023-07-18T03:39:52.341 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 291.723775654, 2023-07-18T03:39:52.341 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.005603284 2023-07-18T03:39:52.341 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.341 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rename_latency": { 2023-07-18T03:39:52.341 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 5091, 2023-07-18T03:39:52.342 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 26.931246465, 2023-07-18T03:39:52.342 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.005289971 2023-07-18T03:39:52.342 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.342 INFO:teuthology.orchestra.run.smithi179.stdout: "req_renamesnap_latency": { 2023-07-18T03:39:52.342 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:52.343 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:52.343 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:52.343 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.343 INFO:teuthology.orchestra.run.smithi179.stdout: "req_snapdiff_latency": { 2023-07-18T03:39:52.343 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:52.343 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:52.344 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:52.344 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.344 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmdir_latency": { 2023-07-18T03:39:52.344 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:52.345 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:52.345 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:52.345 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.345 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmsnap_latency": { 2023-07-18T03:39:52.345 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:52.346 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:52.346 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:52.346 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.346 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmxattr_latency": { 2023-07-18T03:39:52.346 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:52.347 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:52.347 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:52.347 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.347 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setattr_latency": { 2023-07-18T03:39:52.347 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 201, 2023-07-18T03:39:52.347 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 13.486221275, 2023-07-18T03:39:52.348 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.067095628 2023-07-18T03:39:52.348 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.348 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setdirlayout_latency": { 2023-07-18T03:39:52.348 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:52.349 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:52.349 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:52.349 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.349 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setfilelock_latency": { 2023-07-18T03:39:52.349 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 892, 2023-07-18T03:39:52.350 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 1.952848528, 2023-07-18T03:39:52.350 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.002189292 2023-07-18T03:39:52.350 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.350 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setlayout_latency": { 2023-07-18T03:39:52.350 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:52.351 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:52.351 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:52.351 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.351 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setxattr_latency": { 2023-07-18T03:39:52.351 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:52.352 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:52.352 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:52.352 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.352 INFO:teuthology.orchestra.run.smithi179.stdout: "req_symlink_latency": { 2023-07-18T03:39:52.353 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:52.353 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:52.353 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:52.353 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.354 INFO:teuthology.orchestra.run.smithi179.stdout: "req_unlink_latency": { 2023-07-18T03:39:52.354 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 26716, 2023-07-18T03:39:52.354 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 109.964615411, 2023-07-18T03:39:52.354 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.004116058 2023-07-18T03:39:52.355 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.355 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_revoke_eviction": 0, 2023-07-18T03:39:52.356 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_acquisition_throttle": 0, 2023-07-18T03:39:52.356 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getvxattr_latency": { 2023-07-18T03:39:52.356 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:52.357 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:52.357 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:52.358 INFO:teuthology.orchestra.run.smithi179.stdout: } 2023-07-18T03:39:52.358 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.358 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_sessions": { 2023-07-18T03:39:52.358 INFO:teuthology.orchestra.run.smithi179.stdout: "session_count": 1, 2023-07-18T03:39:52.359 INFO:teuthology.orchestra.run.smithi179.stdout: "session_add": 3, 2023-07-18T03:39:52.359 INFO:teuthology.orchestra.run.smithi179.stdout: "session_remove": 2, 2023-07-18T03:39:52.359 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_open": 1, 2023-07-18T03:39:52.359 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_stale": 0, 2023-07-18T03:39:52.360 INFO:teuthology.orchestra.run.smithi179.stdout: "total_load": 31213, 2023-07-18T03:39:52.360 INFO:teuthology.orchestra.run.smithi179.stdout: "average_load": 31213, 2023-07-18T03:39:52.360 INFO:teuthology.orchestra.run.smithi179.stdout: "avg_session_uptime": 1934 2023-07-18T03:39:52.360 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.360 INFO:teuthology.orchestra.run.smithi179.stdout: "mempool": { 2023-07-18T03:39:52.361 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_bytes": 0, 2023-07-18T03:39:52.361 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_items": 0, 2023-07-18T03:39:52.361 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_bytes": 0, 2023-07-18T03:39:52.361 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_items": 0, 2023-07-18T03:39:52.361 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_bytes": 0, 2023-07-18T03:39:52.362 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_items": 0, 2023-07-18T03:39:52.362 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-18T03:39:52.362 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_items": 0, 2023-07-18T03:39:52.363 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-18T03:39:52.363 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_items": 0, 2023-07-18T03:39:52.363 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_bytes": 0, 2023-07-18T03:39:52.364 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_items": 0, 2023-07-18T03:39:52.364 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-18T03:39:52.365 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_items": 0, 2023-07-18T03:39:52.365 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_bytes": 0, 2023-07-18T03:39:52.365 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_items": 0, 2023-07-18T03:39:52.366 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_bytes": 0, 2023-07-18T03:39:52.366 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_items": 0, 2023-07-18T03:39:52.367 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-18T03:39:52.367 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_items": 0, 2023-07-18T03:39:52.367 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-18T03:39:52.368 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_items": 0, 2023-07-18T03:39:52.368 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_bytes": 0, 2023-07-18T03:39:52.369 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_items": 0, 2023-07-18T03:39:52.369 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_bytes": 0, 2023-07-18T03:39:52.369 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_items": 0, 2023-07-18T03:39:52.369 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-18T03:39:52.369 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_items": 0, 2023-07-18T03:39:52.370 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_bytes": 0, 2023-07-18T03:39:52.370 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_items": 0, 2023-07-18T03:39:52.370 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_bytes": 0, 2023-07-18T03:39:52.370 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_items": 0, 2023-07-18T03:39:52.370 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_bytes": 0, 2023-07-18T03:39:52.371 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_items": 0, 2023-07-18T03:39:52.371 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_bytes": 0, 2023-07-18T03:39:52.371 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_items": 0, 2023-07-18T03:39:52.371 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_bytes": 333531, 2023-07-18T03:39:52.371 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_items": 1338, 2023-07-18T03:39:52.372 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_bytes": 440, 2023-07-18T03:39:52.372 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_items": 5, 2023-07-18T03:39:52.372 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_bytes": 0, 2023-07-18T03:39:52.372 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_items": 0, 2023-07-18T03:39:52.372 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_bytes": 0, 2023-07-18T03:39:52.373 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_items": 0, 2023-07-18T03:39:52.373 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_bytes": 0, 2023-07-18T03:39:52.373 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_items": 0, 2023-07-18T03:39:52.373 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_bytes": 9616, 2023-07-18T03:39:52.373 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_items": 185, 2023-07-18T03:39:52.374 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_bytes": 0, 2023-07-18T03:39:52.374 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_items": 0, 2023-07-18T03:39:52.374 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_bytes": 0, 2023-07-18T03:39:52.374 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_items": 0, 2023-07-18T03:39:52.374 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_bytes": 6982384, 2023-07-18T03:39:52.375 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_items": 67614, 2023-07-18T03:39:52.375 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_bytes": 0, 2023-07-18T03:39:52.375 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_items": 0, 2023-07-18T03:39:52.375 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_bytes": 0, 2023-07-18T03:39:52.375 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_items": 0 2023-07-18T03:39:52.376 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.376 INFO:teuthology.orchestra.run.smithi179.stdout: "objecter": { 2023-07-18T03:39:52.376 INFO:teuthology.orchestra.run.smithi179.stdout: "op_active": 0, 2023-07-18T03:39:52.376 INFO:teuthology.orchestra.run.smithi179.stdout: "op_laggy": 0, 2023-07-18T03:39:52.376 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send": 94087, 2023-07-18T03:39:52.377 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send_bytes": 507402649, 2023-07-18T03:39:52.377 INFO:teuthology.orchestra.run.smithi179.stdout: "op_resend": 28, 2023-07-18T03:39:52.377 INFO:teuthology.orchestra.run.smithi179.stdout: "op_reply": 94087, 2023-07-18T03:39:52.377 INFO:teuthology.orchestra.run.smithi179.stdout: "op_latency": { 2023-07-18T03:39:52.378 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 94087, 2023-07-18T03:39:52.378 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 871.306397928, 2023-07-18T03:39:52.378 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.009260645 2023-07-18T03:39:52.378 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.378 INFO:teuthology.orchestra.run.smithi179.stdout: "op_inflight": 0, 2023-07-18T03:39:52.379 INFO:teuthology.orchestra.run.smithi179.stdout: "oplen_avg": { 2023-07-18T03:39:52.379 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 94087, 2023-07-18T03:39:52.379 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 94644 2023-07-18T03:39:52.379 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.379 INFO:teuthology.orchestra.run.smithi179.stdout: "op": 94087, 2023-07-18T03:39:52.380 INFO:teuthology.orchestra.run.smithi179.stdout: "op_r": 3536, 2023-07-18T03:39:52.380 INFO:teuthology.orchestra.run.smithi179.stdout: "op_w": 90551, 2023-07-18T03:39:52.380 INFO:teuthology.orchestra.run.smithi179.stdout: "op_rmw": 0, 2023-07-18T03:39:52.380 INFO:teuthology.orchestra.run.smithi179.stdout: "op_pg": 0, 2023-07-18T03:39:52.380 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_stat": 0, 2023-07-18T03:39:52.381 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_create": 21, 2023-07-18T03:39:52.381 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_read": 994, 2023-07-18T03:39:52.381 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_write": 36004, 2023-07-18T03:39:52.381 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writefull": 555, 2023-07-18T03:39:52.381 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writesame": 0, 2023-07-18T03:39:52.382 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_append": 0, 2023-07-18T03:39:52.382 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_zero": 0, 2023-07-18T03:39:52.382 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_truncate": 11, 2023-07-18T03:39:52.382 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_delete": 53639, 2023-07-18T03:39:52.382 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_mapext": 0, 2023-07-18T03:39:52.383 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_sparse_read": 0, 2023-07-18T03:39:52.383 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_clonerange": 0, 2023-07-18T03:39:52.383 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_getxattr": 2541, 2023-07-18T03:39:52.383 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_setxattr": 42, 2023-07-18T03:39:52.383 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_cmpxattr": 0, 2023-07-18T03:39:52.384 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_rmxattr": 0, 2023-07-18T03:39:52.384 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_resetxattrs": 0, 2023-07-18T03:39:52.384 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_call": 0, 2023-07-18T03:39:52.384 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_watch": 0, 2023-07-18T03:39:52.385 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_notify": 0, 2023-07-18T03:39:52.385 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_src_cmpxattr": 0, 2023-07-18T03:39:52.386 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls": 0, 2023-07-18T03:39:52.386 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls_filter": 0, 2023-07-18T03:39:52.386 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_other": 0, 2023-07-18T03:39:52.387 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_active": 0, 2023-07-18T03:39:52.387 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_send": 0, 2023-07-18T03:39:52.387 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_resend": 0, 2023-07-18T03:39:52.387 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_ping": 0, 2023-07-18T03:39:52.387 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_active": 0, 2023-07-18T03:39:52.388 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_send": 0, 2023-07-18T03:39:52.388 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_resend": 0, 2023-07-18T03:39:52.388 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_active": 0, 2023-07-18T03:39:52.388 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_send": 0, 2023-07-18T03:39:52.388 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_resend": 0, 2023-07-18T03:39:52.389 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_active": 0, 2023-07-18T03:39:52.389 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_send": 0, 2023-07-18T03:39:52.389 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_resend": 0, 2023-07-18T03:39:52.389 INFO:teuthology.orchestra.run.smithi179.stdout: "command_active": 0, 2023-07-18T03:39:52.390 INFO:teuthology.orchestra.run.smithi179.stdout: "command_send": 0, 2023-07-18T03:39:52.390 INFO:teuthology.orchestra.run.smithi179.stdout: "command_resend": 0, 2023-07-18T03:39:52.390 INFO:teuthology.orchestra.run.smithi179.stdout: "map_epoch": 144, 2023-07-18T03:39:52.390 INFO:teuthology.orchestra.run.smithi179.stdout: "map_full": 0, 2023-07-18T03:39:52.390 INFO:teuthology.orchestra.run.smithi179.stdout: "map_inc": 55, 2023-07-18T03:39:52.391 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_sessions": 12, 2023-07-18T03:39:52.391 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_open": 12, 2023-07-18T03:39:52.391 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_close": 0, 2023-07-18T03:39:52.391 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_laggy": 0, 2023-07-18T03:39:52.391 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_wr": 624, 2023-07-18T03:39:52.392 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_rd": 2, 2023-07-18T03:39:52.392 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_del": 211 2023-07-18T03:39:52.392 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.392 INFO:teuthology.orchestra.run.smithi179.stdout: "oft": { 2023-07-18T03:39:52.392 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_objs": 1, 2023-07-18T03:39:52.393 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_kv_pairs": 0, 2023-07-18T03:39:52.393 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_updates": 0, 2023-07-18T03:39:52.393 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_removes": 0 2023-07-18T03:39:52.393 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.393 INFO:teuthology.orchestra.run.smithi179.stdout: "osd-slow-ops": { 2023-07-18T03:39:52.394 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_ops_count": 162974 2023-07-18T03:39:52.394 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.394 INFO:teuthology.orchestra.run.smithi179.stdout: "purge_queue": { 2023-07-18T03:39:52.394 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops": 0, 2023-07-18T03:39:52.395 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops_high_water": 16, 2023-07-18T03:39:52.395 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing": 0, 2023-07-18T03:39:52.395 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_high_water": 8, 2023-07-18T03:39:52.395 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executed": 26716, 2023-07-18T03:39:52.395 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_item_in_journal": 0 2023-07-18T03:39:52.396 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.396 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-18T03:39:52.396 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2023-07-18T03:39:52.397 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2023-07-18T03:39:52.397 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2023-07-18T03:39:52.397 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 7925563, 2023-07-18T03:39:52.397 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 624938003, 2023-07-18T03:39:52.397 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2023-07-18T03:39:52.398 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 7925563, 2023-07-18T03:39:52.398 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2023-07-18T03:39:52.398 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2023-07-18T03:39:52.398 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 7925563, 2023-07-18T03:39:52.399 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 624938003, 2023-07-18T03:39:52.399 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2023-07-18T03:39:52.399 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:52.399 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:52.399 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:52.400 INFO:teuthology.orchestra.run.smithi179.stdout: } 2023-07-18T03:39:52.400 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.400 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_bytes": { 2023-07-18T03:39:52.400 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2023-07-18T03:39:52.401 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2023-07-18T03:39:52.401 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2023-07-18T03:39:52.401 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2023-07-18T03:39:52.402 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2023-07-18T03:39:52.402 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2023-07-18T03:39:52.402 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2023-07-18T03:39:52.402 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 94087, 2023-07-18T03:39:52.403 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 510317613, 2023-07-18T03:39:52.403 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 94087, 2023-07-18T03:39:52.403 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 510317613, 2023-07-18T03:39:52.403 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2023-07-18T03:39:52.403 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:52.404 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:52.404 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:52.404 INFO:teuthology.orchestra.run.smithi179.stdout: } 2023-07-18T03:39:52.404 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.405 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_ops": { 2023-07-18T03:39:52.405 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2023-07-18T03:39:52.405 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 1024, 2023-07-18T03:39:52.405 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2023-07-18T03:39:52.405 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2023-07-18T03:39:52.406 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2023-07-18T03:39:52.406 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2023-07-18T03:39:52.406 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2023-07-18T03:39:52.406 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 94087, 2023-07-18T03:39:52.406 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 94087, 2023-07-18T03:39:52.407 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 94087, 2023-07-18T03:39:52.407 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 94087, 2023-07-18T03:39:52.407 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2023-07-18T03:39:52.407 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:52.407 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:52.408 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:52.408 INFO:teuthology.orchestra.run.smithi179.stdout: } 2023-07-18T03:39:52.408 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.408 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle": { 2023-07-18T03:39:52.409 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2023-07-18T03:39:52.409 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2023-07-18T03:39:52.409 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2023-07-18T03:39:52.409 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 26716, 2023-07-18T03:39:52.410 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 2914980, 2023-07-18T03:39:52.410 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2023-07-18T03:39:52.410 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 26716, 2023-07-18T03:39:52.410 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2023-07-18T03:39:52.410 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2023-07-18T03:39:52.411 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 994, 2023-07-18T03:39:52.411 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 2914980, 2023-07-18T03:39:52.411 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2023-07-18T03:39:52.411 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:52.412 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:52.412 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:52.412 INFO:teuthology.orchestra.run.smithi179.stdout: } 2023-07-18T03:39:52.412 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:52.413 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle-0x5568296660a0": { 2023-07-18T03:39:52.413 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2023-07-18T03:39:52.413 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2023-07-18T03:39:52.413 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2023-07-18T03:39:52.413 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 127224, 2023-07-18T03:39:52.414 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 501671317, 2023-07-18T03:39:52.414 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2023-07-18T03:39:52.414 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 127224, 2023-07-18T03:39:52.414 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2023-07-18T03:39:52.414 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2023-07-18T03:39:52.415 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 35010, 2023-07-18T03:39:52.415 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 501671317, 2023-07-18T03:39:52.415 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2023-07-18T03:39:52.415 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:52.415 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:52.416 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:52.416 INFO:teuthology.orchestra.run.smithi179.stdout: } 2023-07-18T03:39:52.416 INFO:teuthology.orchestra.run.smithi179.stdout: } 2023-07-18T03:39:52.416 INFO:teuthology.orchestra.run.smithi179.stdout:} 2023-07-18T03:39:52.772 INFO:tasks.check_counter:Daemon mds.c mds.exported=18 2023-07-18T03:39:52.773 INFO:tasks.check_counter:Daemon mds.c mds.imported=29 2023-07-18T03:39:52.773 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2023-07-18T03:39:52.774 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2023-07-18T03:39:52.774 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2023-07-18T03:39:52.775 DEBUG:tasks.check_counter:Getting stats from f 2023-07-18T03:39:52.775 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph daemon mds.f perf dump 2023-07-18T03:39:53.357 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.c/config 2023-07-18T03:39:53.738 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:39:53 smithi101 ceph-mon[126379]: pgmap v1024: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:39:53.794 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:39:53 smithi179 ceph-mon[129229]: pgmap v1024: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:39:53.796 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:39:53 smithi155 ceph-mon[128885]: pgmap v1024: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:39:54.617 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2023-07-18T03:39:54.618 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-0": { 2023-07-18T03:39:54.618 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 3671, 2023-07-18T03:39:54.618 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 3673, 2023-07-18T03:39:54.618 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 332464227, 2023-07-18T03:39:54.618 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 1141691, 2023-07-18T03:39:54.619 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 8, 2023-07-18T03:39:54.619 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 4, 2023-07-18T03:39:54.619 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.447480034, 2023-07-18T03:39:54.619 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.077021250, 2023-07-18T03:39:54.619 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.238285064, 2023-07-18T03:39:54.620 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.105518331, 2023-07-18T03:39:54.620 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:39:54.620 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 3671, 2023-07-18T03:39:54.620 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.169978793, 2023-07-18T03:39:54.621 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000046303 2023-07-18T03:39:54.621 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.621 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:39:54.621 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:54.621 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:54.621 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:54.622 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.622 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-18T03:39:54.622 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 0 2023-07-18T03:39:54.622 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.622 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-1": { 2023-07-18T03:39:54.623 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 2549, 2023-07-18T03:39:54.623 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 2512, 2023-07-18T03:39:54.623 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 278381249, 2023-07-18T03:39:54.623 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 920361, 2023-07-18T03:39:54.623 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 6, 2023-07-18T03:39:54.624 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 5, 2023-07-18T03:39:54.624 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.358006375, 2023-07-18T03:39:54.624 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.061345019, 2023-07-18T03:39:54.624 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.208055609, 2023-07-18T03:39:54.624 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.083741400, 2023-07-18T03:39:54.625 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:39:54.625 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 2512, 2023-07-18T03:39:54.625 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.106833618, 2023-07-18T03:39:54.625 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000042529 2023-07-18T03:39:54.625 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.626 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:39:54.626 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:54.626 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:54.626 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:54.626 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.627 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 1706544, 2023-07-18T03:39:54.627 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 332816 2023-07-18T03:39:54.627 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.628 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-2": { 2023-07-18T03:39:54.628 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 2092, 2023-07-18T03:39:54.628 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 2518, 2023-07-18T03:39:54.628 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 285749848, 2023-07-18T03:39:54.629 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 1618240, 2023-07-18T03:39:54.629 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 10, 2023-07-18T03:39:54.629 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 6, 2023-07-18T03:39:54.629 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.303214687, 2023-07-18T03:39:54.629 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.053799648, 2023-07-18T03:39:54.630 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.174639115, 2023-07-18T03:39:54.630 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.057958245, 2023-07-18T03:39:54.630 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:39:54.630 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 2515, 2023-07-18T03:39:54.630 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.122122333, 2023-07-18T03:39:54.631 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000048557 2023-07-18T03:39:54.631 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.631 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:39:54.631 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:54.631 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:54.632 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:54.632 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.632 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 480, 2023-07-18T03:39:54.632 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 968560 2023-07-18T03:39:54.632 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.633 INFO:teuthology.orchestra.run.smithi179.stdout: "cct": { 2023-07-18T03:39:54.633 INFO:teuthology.orchestra.run.smithi179.stdout: "total_workers": 1, 2023-07-18T03:39:54.633 INFO:teuthology.orchestra.run.smithi179.stdout: "unhealthy_workers": 0 2023-07-18T03:39:54.633 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.633 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-MDSRank": { 2023-07-18T03:39:54.634 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2023-07-18T03:39:54.634 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2023-07-18T03:39:54.634 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 12584, 2023-07-18T03:39:54.634 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.360075149, 2023-07-18T03:39:54.634 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000028613 2023-07-18T03:39:54.635 INFO:teuthology.orchestra.run.smithi179.stdout: } 2023-07-18T03:39:54.635 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.635 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-PurgeQueue": { 2023-07-18T03:39:54.635 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2023-07-18T03:39:54.635 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2023-07-18T03:39:54.636 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:54.636 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:54.636 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:54.636 INFO:teuthology.orchestra.run.smithi179.stdout: } 2023-07-18T03:39:54.636 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.637 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": { 2023-07-18T03:39:54.637 INFO:teuthology.orchestra.run.smithi179.stdout: "request": 0, 2023-07-18T03:39:54.637 INFO:teuthology.orchestra.run.smithi179.stdout: "reply": 0, 2023-07-18T03:39:54.637 INFO:teuthology.orchestra.run.smithi179.stdout: "reply_latency": { 2023-07-18T03:39:54.638 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:54.638 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:54.638 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:54.639 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.639 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_reply": 0, 2023-07-18T03:39:54.639 INFO:teuthology.orchestra.run.smithi179.stdout: "forward": 0, 2023-07-18T03:39:54.639 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_complete": 0, 2023-07-18T03:39:54.639 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_keys": 0, 2023-07-18T03:39:54.640 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_commit": 0, 2023-07-18T03:39:54.640 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_split": 0, 2023-07-18T03:39:54.640 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_merge": 0, 2023-07-18T03:39:54.640 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes": 21509, 2023-07-18T03:39:54.640 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_top": 1674, 2023-07-18T03:39:54.641 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_bottom": 19615, 2023-07-18T03:39:54.641 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pin_tail": 220, 2023-07-18T03:39:54.641 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pinned": 19117, 2023-07-18T03:39:54.641 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_expired": 4004, 2023-07-18T03:39:54.641 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_with_caps": 0, 2023-07-18T03:39:54.642 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": 0, 2023-07-18T03:39:54.642 INFO:teuthology.orchestra.run.smithi179.stdout: "subtrees": 16, 2023-07-18T03:39:54.642 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse": 0, 2023-07-18T03:39:54.642 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_hit": 0, 2023-07-18T03:39:54.642 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_forward": 0, 2023-07-18T03:39:54.643 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_discover": 0, 2023-07-18T03:39:54.643 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_dir_fetch": 0, 2023-07-18T03:39:54.643 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_remote_ino": 0, 2023-07-18T03:39:54.643 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_lock": 0, 2023-07-18T03:39:54.643 INFO:teuthology.orchestra.run.smithi179.stdout: "load_cent": 0, 2023-07-18T03:39:54.644 INFO:teuthology.orchestra.run.smithi179.stdout: "q": 0, 2023-07-18T03:39:54.644 INFO:teuthology.orchestra.run.smithi179.stdout: "exported": 0, 2023-07-18T03:39:54.644 INFO:teuthology.orchestra.run.smithi179.stdout: "exported_inodes": 0, 2023-07-18T03:39:54.644 INFO:teuthology.orchestra.run.smithi179.stdout: "imported": 0, 2023-07-18T03:39:54.644 INFO:teuthology.orchestra.run.smithi179.stdout: "imported_inodes": 0, 2023-07-18T03:39:54.645 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_dir_fetch": 0, 2023-07-18T03:39:54.645 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_backtrace_fetch": 0, 2023-07-18T03:39:54.645 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_peer_discover": 0, 2023-07-18T03:39:54.645 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rfiles": 679, 2023-07-18T03:39:54.645 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rbytes": 221129114, 2023-07-18T03:39:54.646 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rsnaps": 6, 2023-07-18T03:39:54.646 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_fetch": 0, 2023-07-18T03:39:54.646 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_set_tag": 0, 2023-07-18T03:39:54.646 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_repaired": 0, 2023-07-18T03:39:54.646 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_inotable_repaired": 0, 2023-07-18T03:39:54.647 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_inodes": 0, 2023-07-18T03:39:54.647 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_base_inodes": 0, 2023-07-18T03:39:54.647 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dirfrag_rstats": 0, 2023-07-18T03:39:54.647 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_file_inodes": 0, 2023-07-18T03:39:54.647 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_inode_file_caps": 0, 2023-07-18T03:39:54.648 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_revoke": 0, 2023-07-18T03:39:54.648 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_grant": 0, 2023-07-18T03:39:54.648 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_trunc": 0, 2023-07-18T03:39:54.649 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-18T03:39:54.649 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-18T03:39:54.649 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps": 0, 2023-07-18T03:39:54.649 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps_dirty": 0, 2023-07-18T03:39:54.649 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_cap_release": 0, 2023-07-18T03:39:54.650 INFO:teuthology.orchestra.run.smithi179.stdout: "process_request_cap_release": 0 2023-07-18T03:39:54.650 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.650 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_cache": { 2023-07-18T03:39:54.650 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update": 0, 2023-07-18T03:39:54.650 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update_receipt": 0, 2023-07-18T03:39:54.651 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_try_discover": 0, 2023-07-18T03:39:54.651 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_send_discover": 0, 2023-07-18T03:39:54.651 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_handle_discover": 0, 2023-07-18T03:39:54.651 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays": 0, 2023-07-18T03:39:54.651 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_delayed": 0, 2023-07-18T03:39:54.652 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_enqueuing": 0, 2023-07-18T03:39:54.652 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_created": 0, 2023-07-18T03:39:54.652 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_enqueued": 0, 2023-07-18T03:39:54.652 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_reintegrated": 0, 2023-07-18T03:39:54.652 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_migrated": 0, 2023-07-18T03:39:54.653 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_processing": 0, 2023-07-18T03:39:54.653 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_enqueued": 0, 2023-07-18T03:39:54.653 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_prioritized": 0, 2023-07-18T03:39:54.653 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_started": 0, 2023-07-18T03:39:54.653 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_completed": 0, 2023-07-18T03:39:54.654 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_enqueue_scrub": 0, 2023-07-18T03:39:54.654 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_exportdir": 0, 2023-07-18T03:39:54.654 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_flush": 0, 2023-07-18T03:39:54.654 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragmentdir": 0, 2023-07-18T03:39:54.654 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragstats": 0, 2023-07-18T03:39:54.655 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_inodestats": 0 2023-07-18T03:39:54.655 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.655 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_log": { 2023-07-18T03:39:54.655 INFO:teuthology.orchestra.run.smithi179.stdout: "evadd": 0, 2023-07-18T03:39:54.655 INFO:teuthology.orchestra.run.smithi179.stdout: "evex": 0, 2023-07-18T03:39:54.656 INFO:teuthology.orchestra.run.smithi179.stdout: "evtrm": 0, 2023-07-18T03:39:54.656 INFO:teuthology.orchestra.run.smithi179.stdout: "ev": 0, 2023-07-18T03:39:54.656 INFO:teuthology.orchestra.run.smithi179.stdout: "evexg": 0, 2023-07-18T03:39:54.656 INFO:teuthology.orchestra.run.smithi179.stdout: "evexd": 0, 2023-07-18T03:39:54.656 INFO:teuthology.orchestra.run.smithi179.stdout: "segadd": 0, 2023-07-18T03:39:54.657 INFO:teuthology.orchestra.run.smithi179.stdout: "segex": 0, 2023-07-18T03:39:54.657 INFO:teuthology.orchestra.run.smithi179.stdout: "segtrm": 0, 2023-07-18T03:39:54.657 INFO:teuthology.orchestra.run.smithi179.stdout: "seg": 129, 2023-07-18T03:39:54.657 INFO:teuthology.orchestra.run.smithi179.stdout: "segexg": 0, 2023-07-18T03:39:54.657 INFO:teuthology.orchestra.run.smithi179.stdout: "segexd": 0, 2023-07-18T03:39:54.658 INFO:teuthology.orchestra.run.smithi179.stdout: "expos": 377505627, 2023-07-18T03:39:54.658 INFO:teuthology.orchestra.run.smithi179.stdout: "wrpos": 0, 2023-07-18T03:39:54.658 INFO:teuthology.orchestra.run.smithi179.stdout: "rdpos": 896930248, 2023-07-18T03:39:54.658 INFO:teuthology.orchestra.run.smithi179.stdout: "jlat": { 2023-07-18T03:39:54.659 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:54.659 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:54.659 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:54.660 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.660 INFO:teuthology.orchestra.run.smithi179.stdout: "replayed": 141949 2023-07-18T03:39:54.660 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.660 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_mem": { 2023-07-18T03:39:54.660 INFO:teuthology.orchestra.run.smithi179.stdout: "ino": 1441, 2023-07-18T03:39:54.661 INFO:teuthology.orchestra.run.smithi179.stdout: "ino+": 26729, 2023-07-18T03:39:54.661 INFO:teuthology.orchestra.run.smithi179.stdout: "ino-": 25288, 2023-07-18T03:39:54.661 INFO:teuthology.orchestra.run.smithi179.stdout: "dir": 89, 2023-07-18T03:39:54.661 INFO:teuthology.orchestra.run.smithi179.stdout: "dir+": 314, 2023-07-18T03:39:54.662 INFO:teuthology.orchestra.run.smithi179.stdout: "dir-": 225, 2023-07-18T03:39:54.662 INFO:teuthology.orchestra.run.smithi179.stdout: "dn": 21509, 2023-07-18T03:39:54.662 INFO:teuthology.orchestra.run.smithi179.stdout: "dn+": 27546, 2023-07-18T03:39:54.662 INFO:teuthology.orchestra.run.smithi179.stdout: "dn-": 6037, 2023-07-18T03:39:54.662 INFO:teuthology.orchestra.run.smithi179.stdout: "cap": 0, 2023-07-18T03:39:54.662 INFO:teuthology.orchestra.run.smithi179.stdout: "cap+": 0, 2023-07-18T03:39:54.663 INFO:teuthology.orchestra.run.smithi179.stdout: "cap-": 0, 2023-07-18T03:39:54.663 INFO:teuthology.orchestra.run.smithi179.stdout: "rss": 79080, 2023-07-18T03:39:54.663 INFO:teuthology.orchestra.run.smithi179.stdout: "heap": 207156 2023-07-18T03:39:54.663 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.664 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_server": { 2023-07-18T03:39:54.664 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_client_request": 0, 2023-07-18T03:39:54.664 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_server_request": 0, 2023-07-18T03:39:54.664 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_request": 0, 2023-07-18T03:39:54.664 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_session": 0, 2023-07-18T03:39:54.665 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_peer_request": 0, 2023-07-18T03:39:54.665 INFO:teuthology.orchestra.run.smithi179.stdout: "req_create_latency": { 2023-07-18T03:39:54.665 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:54.665 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:54.665 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:54.666 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.666 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getattr_latency": { 2023-07-18T03:39:54.666 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:54.666 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:54.666 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:54.666 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.667 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getfilelock_latency": { 2023-07-18T03:39:54.667 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:54.667 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:54.667 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:54.667 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.668 INFO:teuthology.orchestra.run.smithi179.stdout: "req_link_latency": { 2023-07-18T03:39:54.668 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:54.668 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:54.668 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:54.668 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.669 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookup_latency": { 2023-07-18T03:39:54.669 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:54.669 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:54.670 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:54.670 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.670 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookuphash_latency": { 2023-07-18T03:39:54.670 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:54.671 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:54.671 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:54.671 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.671 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupino_latency": { 2023-07-18T03:39:54.671 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:54.672 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:54.672 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:54.672 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.672 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupname_latency": { 2023-07-18T03:39:54.672 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:54.673 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:54.673 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:54.673 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.673 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupparent_latency": { 2023-07-18T03:39:54.673 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:54.674 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:54.674 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:54.674 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.674 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupsnap_latency": { 2023-07-18T03:39:54.674 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:54.675 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:54.675 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:54.675 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.675 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lssnap_latency": { 2023-07-18T03:39:54.675 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:54.676 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:54.676 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:54.676 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.676 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mkdir_latency": { 2023-07-18T03:39:54.676 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:54.677 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:54.677 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:54.677 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.677 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mknod_latency": { 2023-07-18T03:39:54.677 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:54.678 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:54.678 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:54.678 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.678 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mksnap_latency": { 2023-07-18T03:39:54.678 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:54.679 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:54.679 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:54.679 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.679 INFO:teuthology.orchestra.run.smithi179.stdout: "req_open_latency": { 2023-07-18T03:39:54.680 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:54.680 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:54.680 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:54.680 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.681 INFO:teuthology.orchestra.run.smithi179.stdout: "req_readdir_latency": { 2023-07-18T03:39:54.681 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:54.681 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:54.681 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:54.682 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.682 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rename_latency": { 2023-07-18T03:39:54.682 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:54.682 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:54.682 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:54.683 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.683 INFO:teuthology.orchestra.run.smithi179.stdout: "req_renamesnap_latency": { 2023-07-18T03:39:54.683 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:54.683 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:54.684 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:54.684 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.684 INFO:teuthology.orchestra.run.smithi179.stdout: "req_snapdiff_latency": { 2023-07-18T03:39:54.684 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:54.684 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:54.685 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:54.685 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.685 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmdir_latency": { 2023-07-18T03:39:54.685 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:54.685 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:54.686 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:54.686 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.686 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmsnap_latency": { 2023-07-18T03:39:54.686 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:54.686 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:54.687 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:54.687 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.687 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmxattr_latency": { 2023-07-18T03:39:54.687 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:54.687 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:54.688 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:54.688 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.688 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setattr_latency": { 2023-07-18T03:39:54.688 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:54.688 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:54.689 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:54.689 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.689 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setdirlayout_latency": { 2023-07-18T03:39:54.689 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:54.689 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:54.689 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:54.690 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.690 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setfilelock_latency": { 2023-07-18T03:39:54.690 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:54.691 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:54.691 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:54.691 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.691 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setlayout_latency": { 2023-07-18T03:39:54.692 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:54.692 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:54.692 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:54.692 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.692 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setxattr_latency": { 2023-07-18T03:39:54.693 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:54.693 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:54.693 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:54.693 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.693 INFO:teuthology.orchestra.run.smithi179.stdout: "req_symlink_latency": { 2023-07-18T03:39:54.694 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:54.694 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:54.694 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:54.694 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.694 INFO:teuthology.orchestra.run.smithi179.stdout: "req_unlink_latency": { 2023-07-18T03:39:54.695 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:54.695 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:54.695 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:54.695 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.695 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_revoke_eviction": 0, 2023-07-18T03:39:54.696 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_acquisition_throttle": 0, 2023-07-18T03:39:54.696 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getvxattr_latency": { 2023-07-18T03:39:54.696 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:54.696 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:54.697 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:54.697 INFO:teuthology.orchestra.run.smithi179.stdout: } 2023-07-18T03:39:54.697 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.697 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_sessions": { 2023-07-18T03:39:54.697 INFO:teuthology.orchestra.run.smithi179.stdout: "session_count": 2, 2023-07-18T03:39:54.698 INFO:teuthology.orchestra.run.smithi179.stdout: "session_add": 6, 2023-07-18T03:39:54.698 INFO:teuthology.orchestra.run.smithi179.stdout: "session_remove": 4, 2023-07-18T03:39:54.698 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_open": 4, 2023-07-18T03:39:54.698 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_stale": 0, 2023-07-18T03:39:54.698 INFO:teuthology.orchestra.run.smithi179.stdout: "total_load": 0, 2023-07-18T03:39:54.699 INFO:teuthology.orchestra.run.smithi179.stdout: "average_load": 0, 2023-07-18T03:39:54.699 INFO:teuthology.orchestra.run.smithi179.stdout: "avg_session_uptime": 6478 2023-07-18T03:39:54.699 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.699 INFO:teuthology.orchestra.run.smithi179.stdout: "mempool": { 2023-07-18T03:39:54.699 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_bytes": 0, 2023-07-18T03:39:54.700 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_items": 0, 2023-07-18T03:39:54.700 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_bytes": 0, 2023-07-18T03:39:54.700 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_items": 0, 2023-07-18T03:39:54.700 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_bytes": 0, 2023-07-18T03:39:54.700 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_items": 0, 2023-07-18T03:39:54.701 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-18T03:39:54.701 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_items": 0, 2023-07-18T03:39:54.701 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-18T03:39:54.701 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_items": 0, 2023-07-18T03:39:54.702 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_bytes": 0, 2023-07-18T03:39:54.702 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_items": 0, 2023-07-18T03:39:54.702 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-18T03:39:54.702 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_items": 0, 2023-07-18T03:39:54.702 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_bytes": 0, 2023-07-18T03:39:54.703 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_items": 0, 2023-07-18T03:39:54.703 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_bytes": 0, 2023-07-18T03:39:54.703 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_items": 0, 2023-07-18T03:39:54.703 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-18T03:39:54.703 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_items": 0, 2023-07-18T03:39:54.704 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-18T03:39:54.704 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_items": 0, 2023-07-18T03:39:54.704 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_bytes": 0, 2023-07-18T03:39:54.704 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_items": 0, 2023-07-18T03:39:54.704 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_bytes": 0, 2023-07-18T03:39:54.705 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_items": 0, 2023-07-18T03:39:54.705 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-18T03:39:54.705 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_items": 0, 2023-07-18T03:39:54.705 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_bytes": 0, 2023-07-18T03:39:54.705 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_items": 0, 2023-07-18T03:39:54.706 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_bytes": 0, 2023-07-18T03:39:54.706 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_items": 0, 2023-07-18T03:39:54.706 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_bytes": 0, 2023-07-18T03:39:54.706 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_items": 0, 2023-07-18T03:39:54.706 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_bytes": 0, 2023-07-18T03:39:54.707 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_items": 0, 2023-07-18T03:39:54.707 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_bytes": 298587, 2023-07-18T03:39:54.707 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_items": 1797, 2023-07-18T03:39:54.707 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_bytes": 0, 2023-07-18T03:39:54.707 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_items": 0, 2023-07-18T03:39:54.708 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_bytes": 0, 2023-07-18T03:39:54.708 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_items": 0, 2023-07-18T03:39:54.708 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_bytes": 0, 2023-07-18T03:39:54.708 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_items": 0, 2023-07-18T03:39:54.708 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_bytes": 0, 2023-07-18T03:39:54.709 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_items": 0, 2023-07-18T03:39:54.709 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_bytes": 9616, 2023-07-18T03:39:54.709 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_items": 185, 2023-07-18T03:39:54.709 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_bytes": 0, 2023-07-18T03:39:54.709 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_items": 0, 2023-07-18T03:39:54.710 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_bytes": 0, 2023-07-18T03:39:54.710 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_items": 0, 2023-07-18T03:39:54.710 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_bytes": 19020583, 2023-07-18T03:39:54.710 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_items": 126074, 2023-07-18T03:39:54.710 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_bytes": 0, 2023-07-18T03:39:54.710 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_items": 0, 2023-07-18T03:39:54.711 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_bytes": 0, 2023-07-18T03:39:54.711 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_items": 0 2023-07-18T03:39:54.711 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.712 INFO:teuthology.orchestra.run.smithi179.stdout: "objecter": { 2023-07-18T03:39:54.712 INFO:teuthology.orchestra.run.smithi179.stdout: "op_active": 0, 2023-07-18T03:39:54.712 INFO:teuthology.orchestra.run.smithi179.stdout: "op_laggy": 0, 2023-07-18T03:39:54.712 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send": 7649, 2023-07-18T03:39:54.712 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send_bytes": 26, 2023-07-18T03:39:54.713 INFO:teuthology.orchestra.run.smithi179.stdout: "op_resend": 4, 2023-07-18T03:39:54.713 INFO:teuthology.orchestra.run.smithi179.stdout: "op_reply": 7649, 2023-07-18T03:39:54.713 INFO:teuthology.orchestra.run.smithi179.stdout: "op_latency": { 2023-07-18T03:39:54.713 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 7649, 2023-07-18T03:39:54.713 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 11.572752452, 2023-07-18T03:39:54.714 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.001512975 2023-07-18T03:39:54.714 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.714 INFO:teuthology.orchestra.run.smithi179.stdout: "op_inflight": 0, 2023-07-18T03:39:54.714 INFO:teuthology.orchestra.run.smithi179.stdout: "oplen_avg": { 2023-07-18T03:39:54.714 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 7649, 2023-07-18T03:39:54.715 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 7650 2023-07-18T03:39:54.715 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.715 INFO:teuthology.orchestra.run.smithi179.stdout: "op": 7649, 2023-07-18T03:39:54.715 INFO:teuthology.orchestra.run.smithi179.stdout: "op_r": 7649, 2023-07-18T03:39:54.715 INFO:teuthology.orchestra.run.smithi179.stdout: "op_w": 0, 2023-07-18T03:39:54.716 INFO:teuthology.orchestra.run.smithi179.stdout: "op_rmw": 0, 2023-07-18T03:39:54.716 INFO:teuthology.orchestra.run.smithi179.stdout: "op_pg": 0, 2023-07-18T03:39:54.716 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_stat": 4466, 2023-07-18T03:39:54.716 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_create": 0, 2023-07-18T03:39:54.716 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_read": 3180, 2023-07-18T03:39:54.717 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_write": 0, 2023-07-18T03:39:54.717 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writefull": 0, 2023-07-18T03:39:54.717 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writesame": 0, 2023-07-18T03:39:54.717 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_append": 0, 2023-07-18T03:39:54.717 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_zero": 0, 2023-07-18T03:39:54.718 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_truncate": 0, 2023-07-18T03:39:54.718 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_delete": 0, 2023-07-18T03:39:54.718 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_mapext": 0, 2023-07-18T03:39:54.718 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_sparse_read": 0, 2023-07-18T03:39:54.718 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_clonerange": 0, 2023-07-18T03:39:54.719 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_getxattr": 2, 2023-07-18T03:39:54.719 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_setxattr": 0, 2023-07-18T03:39:54.719 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_cmpxattr": 0, 2023-07-18T03:39:54.719 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_rmxattr": 0, 2023-07-18T03:39:54.719 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_resetxattrs": 0, 2023-07-18T03:39:54.720 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_call": 0, 2023-07-18T03:39:54.720 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_watch": 0, 2023-07-18T03:39:54.720 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_notify": 0, 2023-07-18T03:39:54.720 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_src_cmpxattr": 0, 2023-07-18T03:39:54.720 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls": 0, 2023-07-18T03:39:54.721 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls_filter": 0, 2023-07-18T03:39:54.721 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_other": 0, 2023-07-18T03:39:54.721 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_active": 0, 2023-07-18T03:39:54.721 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_send": 0, 2023-07-18T03:39:54.721 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_resend": 0, 2023-07-18T03:39:54.722 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_ping": 0, 2023-07-18T03:39:54.722 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_active": 0, 2023-07-18T03:39:54.722 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_send": 0, 2023-07-18T03:39:54.722 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_resend": 0, 2023-07-18T03:39:54.723 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_active": 0, 2023-07-18T03:39:54.723 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_send": 0, 2023-07-18T03:39:54.723 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_resend": 0, 2023-07-18T03:39:54.723 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_active": 0, 2023-07-18T03:39:54.723 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_send": 0, 2023-07-18T03:39:54.724 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_resend": 0, 2023-07-18T03:39:54.724 INFO:teuthology.orchestra.run.smithi179.stdout: "command_active": 0, 2023-07-18T03:39:54.724 INFO:teuthology.orchestra.run.smithi179.stdout: "command_send": 0, 2023-07-18T03:39:54.724 INFO:teuthology.orchestra.run.smithi179.stdout: "command_resend": 0, 2023-07-18T03:39:54.724 INFO:teuthology.orchestra.run.smithi179.stdout: "map_epoch": 144, 2023-07-18T03:39:54.725 INFO:teuthology.orchestra.run.smithi179.stdout: "map_full": 0, 2023-07-18T03:39:54.725 INFO:teuthology.orchestra.run.smithi179.stdout: "map_inc": 55, 2023-07-18T03:39:54.725 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_sessions": 11, 2023-07-18T03:39:54.725 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_open": 12, 2023-07-18T03:39:54.725 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_close": 1, 2023-07-18T03:39:54.726 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_laggy": 0, 2023-07-18T03:39:54.726 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_wr": 0, 2023-07-18T03:39:54.726 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_rd": 2, 2023-07-18T03:39:54.726 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_del": 0 2023-07-18T03:39:54.726 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.727 INFO:teuthology.orchestra.run.smithi179.stdout: "oft": { 2023-07-18T03:39:54.727 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_objs": 0, 2023-07-18T03:39:54.727 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_kv_pairs": 0, 2023-07-18T03:39:54.727 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_updates": 0, 2023-07-18T03:39:54.727 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_removes": 0 2023-07-18T03:39:54.727 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.728 INFO:teuthology.orchestra.run.smithi179.stdout: "osd-slow-ops": { 2023-07-18T03:39:54.728 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_ops_count": 0 2023-07-18T03:39:54.728 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.728 INFO:teuthology.orchestra.run.smithi179.stdout: "purge_queue": { 2023-07-18T03:39:54.729 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops": 0, 2023-07-18T03:39:54.729 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops_high_water": 0, 2023-07-18T03:39:54.729 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing": 0, 2023-07-18T03:39:54.729 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_high_water": 0, 2023-07-18T03:39:54.729 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executed": 0, 2023-07-18T03:39:54.730 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_item_in_journal": 0 2023-07-18T03:39:54.730 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.730 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-18T03:39:54.730 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2023-07-18T03:39:54.730 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2023-07-18T03:39:54.731 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2023-07-18T03:39:54.731 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 8308, 2023-07-18T03:39:54.731 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 895811090, 2023-07-18T03:39:54.731 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2023-07-18T03:39:54.731 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 8308, 2023-07-18T03:39:54.732 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2023-07-18T03:39:54.732 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2023-07-18T03:39:54.732 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 8308, 2023-07-18T03:39:54.732 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 895811090, 2023-07-18T03:39:54.733 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2023-07-18T03:39:54.733 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:54.733 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:54.733 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:54.734 INFO:teuthology.orchestra.run.smithi179.stdout: } 2023-07-18T03:39:54.734 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.734 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_bytes": { 2023-07-18T03:39:54.734 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2023-07-18T03:39:54.734 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2023-07-18T03:39:54.735 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2023-07-18T03:39:54.735 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2023-07-18T03:39:54.735 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2023-07-18T03:39:54.735 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2023-07-18T03:39:54.735 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2023-07-18T03:39:54.736 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 7649, 2023-07-18T03:39:54.736 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 892771880, 2023-07-18T03:39:54.736 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 7649, 2023-07-18T03:39:54.736 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 892771880, 2023-07-18T03:39:54.736 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2023-07-18T03:39:54.737 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:54.737 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:54.737 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:54.737 INFO:teuthology.orchestra.run.smithi179.stdout: } 2023-07-18T03:39:54.737 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.738 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_ops": { 2023-07-18T03:39:54.738 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2023-07-18T03:39:54.738 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 1024, 2023-07-18T03:39:54.738 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2023-07-18T03:39:54.738 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2023-07-18T03:39:54.739 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2023-07-18T03:39:54.739 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2023-07-18T03:39:54.739 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2023-07-18T03:39:54.739 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 7649, 2023-07-18T03:39:54.739 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 7649, 2023-07-18T03:39:54.740 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 7649, 2023-07-18T03:39:54.740 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 7649, 2023-07-18T03:39:54.740 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2023-07-18T03:39:54.740 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:54.740 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:54.741 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:54.741 INFO:teuthology.orchestra.run.smithi179.stdout: } 2023-07-18T03:39:54.741 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.741 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle": { 2023-07-18T03:39:54.741 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2023-07-18T03:39:54.742 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2023-07-18T03:39:54.742 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2023-07-18T03:39:54.742 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2023-07-18T03:39:54.742 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2023-07-18T03:39:54.742 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2023-07-18T03:39:54.743 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2023-07-18T03:39:54.743 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2023-07-18T03:39:54.743 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2023-07-18T03:39:54.744 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 0, 2023-07-18T03:39:54.744 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 0, 2023-07-18T03:39:54.744 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2023-07-18T03:39:54.744 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:54.745 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:54.745 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:54.745 INFO:teuthology.orchestra.run.smithi179.stdout: } 2023-07-18T03:39:54.745 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:54.745 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle-0x55786b764460": { 2023-07-18T03:39:54.746 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2023-07-18T03:39:54.746 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2023-07-18T03:39:54.746 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2023-07-18T03:39:54.746 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2023-07-18T03:39:54.746 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2023-07-18T03:39:54.747 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2023-07-18T03:39:54.747 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2023-07-18T03:39:54.747 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2023-07-18T03:39:54.747 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2023-07-18T03:39:54.747 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 0, 2023-07-18T03:39:54.748 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 0, 2023-07-18T03:39:54.748 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2023-07-18T03:39:54.748 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:54.748 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:54.748 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:54.749 INFO:teuthology.orchestra.run.smithi179.stdout: } 2023-07-18T03:39:54.749 INFO:teuthology.orchestra.run.smithi179.stdout: } 2023-07-18T03:39:54.749 INFO:teuthology.orchestra.run.smithi179.stdout:} 2023-07-18T03:39:55.022 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2023-07-18T03:39:55.022 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2023-07-18T03:39:55.022 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2023-07-18T03:39:55.023 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2023-07-18T03:39:55.023 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2023-07-18T03:39:55.023 DEBUG:tasks.check_counter:Getting stats from i 2023-07-18T03:39:55.024 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph daemon mds.i perf dump 2023-07-18T03:39:55.643 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.c/config 2023-07-18T03:39:55.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:39:55 smithi101 ceph-mon[126379]: pgmap v1025: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:39:55.796 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:39:55 smithi155 ceph-mon[128885]: pgmap v1025: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:39:55.858 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:39:55 smithi179 ceph-mon[129229]: pgmap v1025: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:39:57.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:39:57 smithi101 ceph-mon[126379]: pgmap v1026: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:39:57.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:39:57 smithi155 ceph-mon[128885]: pgmap v1026: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:39:57.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:39:57 smithi179 ceph-mon[129229]: pgmap v1026: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:39:58.643 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2023-07-18T03:39:58.644 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-0": { 2023-07-18T03:39:58.644 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 195249, 2023-07-18T03:39:58.645 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 225915, 2023-07-18T03:39:58.645 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 49377555, 2023-07-18T03:39:58.646 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 538671357, 2023-07-18T03:39:58.646 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 305, 2023-07-18T03:39:58.646 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 4, 2023-07-18T03:39:58.647 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 12.240449916, 2023-07-18T03:39:58.647 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 4.595772019, 2023-07-18T03:39:58.647 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 6.311271694, 2023-07-18T03:39:58.648 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 1.896680683, 2023-07-18T03:39:58.648 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:39:58.648 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 225916, 2023-07-18T03:39:58.649 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 16.680282105, 2023-07-18T03:39:58.649 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000073834 2023-07-18T03:39:58.649 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.650 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:39:58.650 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 316426, 2023-07-18T03:39:58.650 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.051160194, 2023-07-18T03:39:58.650 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000161 2023-07-18T03:39:58.651 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.651 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 42368, 2023-07-18T03:39:58.651 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 138032 2023-07-18T03:39:58.652 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.652 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-1": { 2023-07-18T03:39:58.652 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 324565, 2023-07-18T03:39:58.653 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 458026, 2023-07-18T03:39:58.653 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 116892007, 2023-07-18T03:39:58.653 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 608503296, 2023-07-18T03:39:58.654 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 20, 2023-07-18T03:39:58.654 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 5, 2023-07-18T03:39:58.654 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 21.530549055, 2023-07-18T03:39:58.655 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 9.653670474, 2023-07-18T03:39:58.655 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 13.780861930, 2023-07-18T03:39:58.656 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 1.354565162, 2023-07-18T03:39:58.656 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:39:58.657 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 458025, 2023-07-18T03:39:58.657 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 61.888171419, 2023-07-18T03:39:58.658 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000135119 2023-07-18T03:39:58.658 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.658 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:39:58.659 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 576590, 2023-07-18T03:39:58.659 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.095850941, 2023-07-18T03:39:58.659 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000166 2023-07-18T03:39:58.660 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.660 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 104771840, 2023-07-18T03:39:58.660 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 349085744 2023-07-18T03:39:58.661 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.661 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-2": { 2023-07-18T03:39:58.661 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 184721, 2023-07-18T03:39:58.662 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 200212, 2023-07-18T03:39:58.662 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 47873746, 2023-07-18T03:39:58.662 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 245122532, 2023-07-18T03:39:58.663 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 25, 2023-07-18T03:39:58.663 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 7, 2023-07-18T03:39:58.664 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 11.851578085, 2023-07-18T03:39:58.664 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 5.075187937, 2023-07-18T03:39:58.664 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 6.581583585, 2023-07-18T03:39:58.664 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 1.266587747, 2023-07-18T03:39:58.665 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:39:58.665 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 200207, 2023-07-18T03:39:58.665 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 15.774922509, 2023-07-18T03:39:58.665 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000078793 2023-07-18T03:39:58.665 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.666 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:39:58.666 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 319357, 2023-07-18T03:39:58.667 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.052084686, 2023-07-18T03:39:58.667 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000163 2023-07-18T03:39:58.667 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.667 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 1066704, 2023-07-18T03:39:58.667 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 481440 2023-07-18T03:39:58.668 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.668 INFO:teuthology.orchestra.run.smithi179.stdout: "cct": { 2023-07-18T03:39:58.668 INFO:teuthology.orchestra.run.smithi179.stdout: "total_workers": 1, 2023-07-18T03:39:58.668 INFO:teuthology.orchestra.run.smithi179.stdout: "unhealthy_workers": 0 2023-07-18T03:39:58.669 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.669 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-MDSRank": { 2023-07-18T03:39:58.669 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2023-07-18T03:39:58.669 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2023-07-18T03:39:58.669 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 118738, 2023-07-18T03:39:58.670 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 116.783024307, 2023-07-18T03:39:58.670 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000983535 2023-07-18T03:39:58.670 INFO:teuthology.orchestra.run.smithi179.stdout: } 2023-07-18T03:39:58.670 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.670 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-PurgeQueue": { 2023-07-18T03:39:58.671 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2023-07-18T03:39:58.671 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2023-07-18T03:39:58.671 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 38934, 2023-07-18T03:39:58.671 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 11.287588734, 2023-07-18T03:39:58.671 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000289915 2023-07-18T03:39:58.672 INFO:teuthology.orchestra.run.smithi179.stdout: } 2023-07-18T03:39:58.672 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.672 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": { 2023-07-18T03:39:58.672 INFO:teuthology.orchestra.run.smithi179.stdout: "request": 202942, 2023-07-18T03:39:58.673 INFO:teuthology.orchestra.run.smithi179.stdout: "reply": 183743, 2023-07-18T03:39:58.673 INFO:teuthology.orchestra.run.smithi179.stdout: "reply_latency": { 2023-07-18T03:39:58.673 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 183743, 2023-07-18T03:39:58.673 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 1202.305381985, 2023-07-18T03:39:58.673 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.006543407 2023-07-18T03:39:58.674 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.674 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_reply": 0, 2023-07-18T03:39:58.674 INFO:teuthology.orchestra.run.smithi179.stdout: "forward": 19199, 2023-07-18T03:39:58.675 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_complete": 31, 2023-07-18T03:39:58.675 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_keys": 2, 2023-07-18T03:39:58.675 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_commit": 193, 2023-07-18T03:39:58.675 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_split": 18, 2023-07-18T03:39:58.675 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_merge": 15, 2023-07-18T03:39:58.676 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes": 3035, 2023-07-18T03:39:58.676 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_top": 1390, 2023-07-18T03:39:58.676 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_bottom": 1361, 2023-07-18T03:39:58.676 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pin_tail": 284, 2023-07-18T03:39:58.677 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pinned": 1048, 2023-07-18T03:39:58.677 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_expired": 3018, 2023-07-18T03:39:58.677 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_with_caps": 89, 2023-07-18T03:39:58.677 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": 89, 2023-07-18T03:39:58.677 INFO:teuthology.orchestra.run.smithi179.stdout: "subtrees": 18, 2023-07-18T03:39:58.678 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse": 291989, 2023-07-18T03:39:58.678 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_hit": 200465, 2023-07-18T03:39:58.678 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_forward": 19178, 2023-07-18T03:39:58.678 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_discover": 148, 2023-07-18T03:39:58.678 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_dir_fetch": 2, 2023-07-18T03:39:58.679 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_remote_ino": 0, 2023-07-18T03:39:58.679 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_lock": 1171, 2023-07-18T03:39:58.679 INFO:teuthology.orchestra.run.smithi179.stdout: "load_cent": 0, 2023-07-18T03:39:58.679 INFO:teuthology.orchestra.run.smithi179.stdout: "q": 0, 2023-07-18T03:39:58.680 INFO:teuthology.orchestra.run.smithi179.stdout: "exported": 31, 2023-07-18T03:39:58.680 INFO:teuthology.orchestra.run.smithi179.stdout: "exported_inodes": 2193, 2023-07-18T03:39:58.680 INFO:teuthology.orchestra.run.smithi179.stdout: "imported": 28, 2023-07-18T03:39:58.680 INFO:teuthology.orchestra.run.smithi179.stdout: "imported_inodes": 977, 2023-07-18T03:39:58.680 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_dir_fetch": 0, 2023-07-18T03:39:58.681 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_backtrace_fetch": 3830, 2023-07-18T03:39:58.681 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_peer_discover": 0, 2023-07-18T03:39:58.681 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rfiles": 679, 2023-07-18T03:39:58.681 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rbytes": 221129114, 2023-07-18T03:39:58.681 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rsnaps": 6, 2023-07-18T03:39:58.682 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_fetch": 3830, 2023-07-18T03:39:58.682 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_set_tag": 0, 2023-07-18T03:39:58.682 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_repaired": 0, 2023-07-18T03:39:58.682 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_inotable_repaired": 0, 2023-07-18T03:39:58.683 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_inodes": 934, 2023-07-18T03:39:58.683 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_base_inodes": 32, 2023-07-18T03:39:58.683 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dirfrag_rstats": 902, 2023-07-18T03:39:58.683 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_file_inodes": 2896, 2023-07-18T03:39:58.683 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_inode_file_caps": 18, 2023-07-18T03:39:58.684 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_revoke": 34536, 2023-07-18T03:39:58.684 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_grant": 100140, 2023-07-18T03:39:58.684 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_trunc": 0, 2023-07-18T03:39:58.684 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flushsnap_ack": 1658, 2023-07-18T03:39:58.684 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-18T03:39:58.685 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps": 68869, 2023-07-18T03:39:58.685 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps_dirty": 24180, 2023-07-18T03:39:58.685 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_cap_release": 6031, 2023-07-18T03:39:58.685 INFO:teuthology.orchestra.run.smithi179.stdout: "process_request_cap_release": 111128 2023-07-18T03:39:58.686 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.686 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_cache": { 2023-07-18T03:39:58.686 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update": 12, 2023-07-18T03:39:58.686 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update_receipt": 34, 2023-07-18T03:39:58.686 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_try_discover": 22, 2023-07-18T03:39:58.687 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_send_discover": 162, 2023-07-18T03:39:58.687 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_handle_discover": 302, 2023-07-18T03:39:58.687 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays": 0, 2023-07-18T03:39:58.687 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_delayed": 0, 2023-07-18T03:39:58.687 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_enqueuing": 0, 2023-07-18T03:39:58.688 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_created": 23869, 2023-07-18T03:39:58.688 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_enqueued": 23869, 2023-07-18T03:39:58.688 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_reintegrated": 0, 2023-07-18T03:39:58.689 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_migrated": 0, 2023-07-18T03:39:58.689 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_processing": 0, 2023-07-18T03:39:58.689 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_enqueued": 0, 2023-07-18T03:39:58.689 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_prioritized": 0, 2023-07-18T03:39:58.690 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_started": 0, 2023-07-18T03:39:58.690 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_completed": 0, 2023-07-18T03:39:58.690 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_enqueue_scrub": 32, 2023-07-18T03:39:58.690 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_exportdir": 32, 2023-07-18T03:39:58.690 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_flush": 0, 2023-07-18T03:39:58.691 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragmentdir": 33, 2023-07-18T03:39:58.691 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragstats": 0, 2023-07-18T03:39:58.691 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_inodestats": 0 2023-07-18T03:39:58.691 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.691 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_log": { 2023-07-18T03:39:58.692 INFO:teuthology.orchestra.run.smithi179.stdout: "evadd": 141949, 2023-07-18T03:39:58.692 INFO:teuthology.orchestra.run.smithi179.stdout: "evex": 57759, 2023-07-18T03:39:58.692 INFO:teuthology.orchestra.run.smithi179.stdout: "evtrm": 57759, 2023-07-18T03:39:58.692 INFO:teuthology.orchestra.run.smithi179.stdout: "ev": 84190, 2023-07-18T03:39:58.692 INFO:teuthology.orchestra.run.smithi179.stdout: "evexg": 0, 2023-07-18T03:39:58.693 INFO:teuthology.orchestra.run.smithi179.stdout: "evexd": 276, 2023-07-18T03:39:58.693 INFO:teuthology.orchestra.run.smithi179.stdout: "segadd": 231, 2023-07-18T03:39:58.693 INFO:teuthology.orchestra.run.smithi179.stdout: "segex": 103, 2023-07-18T03:39:58.693 INFO:teuthology.orchestra.run.smithi179.stdout: "segtrm": 103, 2023-07-18T03:39:58.694 INFO:teuthology.orchestra.run.smithi179.stdout: "seg": 129, 2023-07-18T03:39:58.694 INFO:teuthology.orchestra.run.smithi179.stdout: "segexg": 0, 2023-07-18T03:39:58.694 INFO:teuthology.orchestra.run.smithi179.stdout: "segexd": 1, 2023-07-18T03:39:58.694 INFO:teuthology.orchestra.run.smithi179.stdout: "expos": 377505627, 2023-07-18T03:39:58.695 INFO:teuthology.orchestra.run.smithi179.stdout: "wrpos": 896930384, 2023-07-18T03:39:58.695 INFO:teuthology.orchestra.run.smithi179.stdout: "rdpos": 0, 2023-07-18T03:39:58.695 INFO:teuthology.orchestra.run.smithi179.stdout: "jlat": { 2023-07-18T03:39:58.695 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 63238, 2023-07-18T03:39:58.695 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 350.047847265, 2023-07-18T03:39:58.696 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.005535403 2023-07-18T03:39:58.696 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.696 INFO:teuthology.orchestra.run.smithi179.stdout: "replayed": 0 2023-07-18T03:39:58.696 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.697 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_mem": { 2023-07-18T03:39:58.697 INFO:teuthology.orchestra.run.smithi179.stdout: "ino": 1436, 2023-07-18T03:39:58.697 INFO:teuthology.orchestra.run.smithi179.stdout: "ino+": 26948, 2023-07-18T03:39:58.697 INFO:teuthology.orchestra.run.smithi179.stdout: "ino-": 25512, 2023-07-18T03:39:58.697 INFO:teuthology.orchestra.run.smithi179.stdout: "dir": 183, 2023-07-18T03:39:58.698 INFO:teuthology.orchestra.run.smithi179.stdout: "dir+": 448, 2023-07-18T03:39:58.698 INFO:teuthology.orchestra.run.smithi179.stdout: "dir-": 265, 2023-07-18T03:39:58.698 INFO:teuthology.orchestra.run.smithi179.stdout: "dn": 3035, 2023-07-18T03:39:58.698 INFO:teuthology.orchestra.run.smithi179.stdout: "dn+": 49219, 2023-07-18T03:39:58.698 INFO:teuthology.orchestra.run.smithi179.stdout: "dn-": 46184, 2023-07-18T03:39:58.699 INFO:teuthology.orchestra.run.smithi179.stdout: "cap": 89, 2023-07-18T03:39:58.699 INFO:teuthology.orchestra.run.smithi179.stdout: "cap+": 26187, 2023-07-18T03:39:58.699 INFO:teuthology.orchestra.run.smithi179.stdout: "cap-": 26098, 2023-07-18T03:39:58.699 INFO:teuthology.orchestra.run.smithi179.stdout: "rss": 85016, 2023-07-18T03:39:58.699 INFO:teuthology.orchestra.run.smithi179.stdout: "heap": 207156 2023-07-18T03:39:58.700 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.700 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_server": { 2023-07-18T03:39:58.700 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_client_request": 296052, 2023-07-18T03:39:58.700 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_server_request": 27907, 2023-07-18T03:39:58.701 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_request": 202942, 2023-07-18T03:39:58.701 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_session": 2994, 2023-07-18T03:39:58.701 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_peer_request": 58003, 2023-07-18T03:39:58.701 INFO:teuthology.orchestra.run.smithi179.stdout: "req_create_latency": { 2023-07-18T03:39:58.701 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 24499, 2023-07-18T03:39:58.702 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 121.327424835, 2023-07-18T03:39:58.702 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.004952341 2023-07-18T03:39:58.702 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.702 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getattr_latency": { 2023-07-18T03:39:58.703 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 33124, 2023-07-18T03:39:58.703 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 217.067959493, 2023-07-18T03:39:58.703 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.006553192 2023-07-18T03:39:58.703 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.704 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getfilelock_latency": { 2023-07-18T03:39:58.704 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:58.704 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:58.704 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:58.705 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.705 INFO:teuthology.orchestra.run.smithi179.stdout: "req_link_latency": { 2023-07-18T03:39:58.705 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:58.705 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:58.705 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:58.706 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.706 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookup_latency": { 2023-07-18T03:39:58.706 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 34139, 2023-07-18T03:39:58.706 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 193.483146775, 2023-07-18T03:39:58.706 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.005667510 2023-07-18T03:39:58.707 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.707 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookuphash_latency": { 2023-07-18T03:39:58.707 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:58.707 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:58.707 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:58.708 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.708 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupino_latency": { 2023-07-18T03:39:58.708 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 1, 2023-07-18T03:39:58.708 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000155788, 2023-07-18T03:39:58.709 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000155788 2023-07-18T03:39:58.709 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.709 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupname_latency": { 2023-07-18T03:39:58.709 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:58.709 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:58.710 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:58.710 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.710 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupparent_latency": { 2023-07-18T03:39:58.710 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:58.710 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:58.711 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:58.711 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.711 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupsnap_latency": { 2023-07-18T03:39:58.711 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:58.711 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:58.712 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:58.713 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.713 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lssnap_latency": { 2023-07-18T03:39:58.713 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 32, 2023-07-18T03:39:58.713 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.010651791, 2023-07-18T03:39:58.713 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000332868 2023-07-18T03:39:58.714 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.714 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mkdir_latency": { 2023-07-18T03:39:58.714 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 119, 2023-07-18T03:39:58.715 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.782061685, 2023-07-18T03:39:58.715 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.006571946 2023-07-18T03:39:58.715 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.715 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mknod_latency": { 2023-07-18T03:39:58.715 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:58.716 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:58.716 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:58.716 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.716 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mksnap_latency": { 2023-07-18T03:39:58.717 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 32, 2023-07-18T03:39:58.717 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 6.734635790, 2023-07-18T03:39:58.717 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.210457368 2023-07-18T03:39:58.717 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.717 INFO:teuthology.orchestra.run.smithi179.stdout: "req_open_latency": { 2023-07-18T03:39:58.718 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 9, 2023-07-18T03:39:58.718 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.012994545, 2023-07-18T03:39:58.718 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.001443838 2023-07-18T03:39:58.718 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.719 INFO:teuthology.orchestra.run.smithi179.stdout: "req_readdir_latency": { 2023-07-18T03:39:58.719 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 62711, 2023-07-18T03:39:58.719 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 336.210834530, 2023-07-18T03:39:58.719 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.005361273 2023-07-18T03:39:58.719 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.720 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rename_latency": { 2023-07-18T03:39:58.720 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 4537, 2023-07-18T03:39:58.720 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 193.038492345, 2023-07-18T03:39:58.720 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.042547606 2023-07-18T03:39:58.720 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.721 INFO:teuthology.orchestra.run.smithi179.stdout: "req_renamesnap_latency": { 2023-07-18T03:39:58.721 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:58.721 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:58.721 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:58.722 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.722 INFO:teuthology.orchestra.run.smithi179.stdout: "req_snapdiff_latency": { 2023-07-18T03:39:58.722 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:58.722 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:58.722 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:58.723 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.723 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmdir_latency": { 2023-07-18T03:39:58.723 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 1, 2023-07-18T03:39:58.723 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000947059, 2023-07-18T03:39:58.723 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000947059 2023-07-18T03:39:58.724 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.724 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmsnap_latency": { 2023-07-18T03:39:58.724 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 26, 2023-07-18T03:39:58.724 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.366671276, 2023-07-18T03:39:58.724 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.014102741 2023-07-18T03:39:58.725 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.725 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmxattr_latency": { 2023-07-18T03:39:58.725 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:58.725 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:58.726 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:58.726 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.726 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setattr_latency": { 2023-07-18T03:39:58.726 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 33, 2023-07-18T03:39:58.726 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.528524084, 2023-07-18T03:39:58.727 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.016015881 2023-07-18T03:39:58.727 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.727 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setdirlayout_latency": { 2023-07-18T03:39:58.727 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:58.727 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:58.728 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:58.728 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.728 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setfilelock_latency": { 2023-07-18T03:39:58.728 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 604, 2023-07-18T03:39:58.729 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 1.840955580, 2023-07-18T03:39:58.729 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.003047939 2023-07-18T03:39:58.729 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.729 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setlayout_latency": { 2023-07-18T03:39:58.729 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:58.730 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:58.730 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:58.730 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.730 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setxattr_latency": { 2023-07-18T03:39:58.730 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 7, 2023-07-18T03:39:58.731 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.020285013, 2023-07-18T03:39:58.731 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.002897859 2023-07-18T03:39:58.731 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.731 INFO:teuthology.orchestra.run.smithi179.stdout: "req_symlink_latency": { 2023-07-18T03:39:58.731 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:58.732 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:58.732 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:58.732 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.732 INFO:teuthology.orchestra.run.smithi179.stdout: "req_unlink_latency": { 2023-07-18T03:39:58.733 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 23869, 2023-07-18T03:39:58.733 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 130.879641396, 2023-07-18T03:39:58.733 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.005483247 2023-07-18T03:39:58.733 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.733 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_revoke_eviction": 0, 2023-07-18T03:39:58.734 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_acquisition_throttle": 0, 2023-07-18T03:39:58.734 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getvxattr_latency": { 2023-07-18T03:39:58.734 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:58.734 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:58.734 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:58.735 INFO:teuthology.orchestra.run.smithi179.stdout: } 2023-07-18T03:39:58.735 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.735 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_sessions": { 2023-07-18T03:39:58.735 INFO:teuthology.orchestra.run.smithi179.stdout: "session_count": 2, 2023-07-18T03:39:58.736 INFO:teuthology.orchestra.run.smithi179.stdout: "session_add": 6, 2023-07-18T03:39:58.736 INFO:teuthology.orchestra.run.smithi179.stdout: "session_remove": 4, 2023-07-18T03:39:58.736 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_open": 2, 2023-07-18T03:39:58.736 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_stale": 0, 2023-07-18T03:39:58.736 INFO:teuthology.orchestra.run.smithi179.stdout: "total_load": 29, 2023-07-18T03:39:58.737 INFO:teuthology.orchestra.run.smithi179.stdout: "average_load": 14, 2023-07-18T03:39:58.737 INFO:teuthology.orchestra.run.smithi179.stdout: "avg_session_uptime": 1938 2023-07-18T03:39:58.737 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.737 INFO:teuthology.orchestra.run.smithi179.stdout: "mempool": { 2023-07-18T03:39:58.737 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_bytes": 0, 2023-07-18T03:39:58.738 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_items": 0, 2023-07-18T03:39:58.738 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_bytes": 0, 2023-07-18T03:39:58.738 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_items": 0, 2023-07-18T03:39:58.738 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_bytes": 0, 2023-07-18T03:39:58.739 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_items": 0, 2023-07-18T03:39:58.739 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-18T03:39:58.739 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_items": 0, 2023-07-18T03:39:58.739 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-18T03:39:58.739 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_items": 0, 2023-07-18T03:39:58.740 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_bytes": 0, 2023-07-18T03:39:58.740 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_items": 0, 2023-07-18T03:39:58.740 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-18T03:39:58.740 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_items": 0, 2023-07-18T03:39:58.740 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_bytes": 0, 2023-07-18T03:39:58.741 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_items": 0, 2023-07-18T03:39:58.741 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_bytes": 0, 2023-07-18T03:39:58.741 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_items": 0, 2023-07-18T03:39:58.741 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-18T03:39:58.742 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_items": 0, 2023-07-18T03:39:58.742 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-18T03:39:58.742 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_items": 0, 2023-07-18T03:39:58.742 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_bytes": 0, 2023-07-18T03:39:58.742 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_items": 0, 2023-07-18T03:39:58.743 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_bytes": 0, 2023-07-18T03:39:58.743 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_items": 0, 2023-07-18T03:39:58.743 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-18T03:39:58.743 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_items": 0, 2023-07-18T03:39:58.743 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_bytes": 0, 2023-07-18T03:39:58.744 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_items": 0, 2023-07-18T03:39:58.744 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_bytes": 0, 2023-07-18T03:39:58.744 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_items": 0, 2023-07-18T03:39:58.744 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_bytes": 0, 2023-07-18T03:39:58.745 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_items": 0, 2023-07-18T03:39:58.745 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_bytes": 0, 2023-07-18T03:39:58.745 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_items": 0, 2023-07-18T03:39:58.745 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_bytes": 421127, 2023-07-18T03:39:58.745 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_items": 1142, 2023-07-18T03:39:58.746 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_bytes": 792, 2023-07-18T03:39:58.746 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_items": 9, 2023-07-18T03:39:58.746 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_bytes": 0, 2023-07-18T03:39:58.746 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_items": 0, 2023-07-18T03:39:58.746 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_bytes": 0, 2023-07-18T03:39:58.747 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_items": 0, 2023-07-18T03:39:58.747 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_bytes": 0, 2023-07-18T03:39:58.747 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_items": 0, 2023-07-18T03:39:58.747 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_bytes": 9616, 2023-07-18T03:39:58.747 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_items": 185, 2023-07-18T03:39:58.748 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_bytes": 0, 2023-07-18T03:39:58.748 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_items": 0, 2023-07-18T03:39:58.748 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_bytes": 0, 2023-07-18T03:39:58.748 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_items": 0, 2023-07-18T03:39:58.748 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_bytes": 6436225, 2023-07-18T03:39:58.749 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_items": 64298, 2023-07-18T03:39:58.749 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_bytes": 0, 2023-07-18T03:39:58.749 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_items": 0, 2023-07-18T03:39:58.749 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_bytes": 0, 2023-07-18T03:39:58.750 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_items": 0 2023-07-18T03:39:58.750 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.750 INFO:teuthology.orchestra.run.smithi179.stdout: "objecter": { 2023-07-18T03:39:58.750 INFO:teuthology.orchestra.run.smithi179.stdout: "op_active": 0, 2023-07-18T03:39:58.750 INFO:teuthology.orchestra.run.smithi179.stdout: "op_laggy": 0, 2023-07-18T03:39:58.751 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send": 118407, 2023-07-18T03:39:58.751 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send_bytes": 899348803, 2023-07-18T03:39:58.751 INFO:teuthology.orchestra.run.smithi179.stdout: "op_resend": 31, 2023-07-18T03:39:58.751 INFO:teuthology.orchestra.run.smithi179.stdout: "op_reply": 118407, 2023-07-18T03:39:58.751 INFO:teuthology.orchestra.run.smithi179.stdout: "op_latency": { 2023-07-18T03:39:58.752 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 118407, 2023-07-18T03:39:58.752 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 913.286490640, 2023-07-18T03:39:58.752 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.007713112 2023-07-18T03:39:58.752 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.752 INFO:teuthology.orchestra.run.smithi179.stdout: "op_inflight": 0, 2023-07-18T03:39:58.753 INFO:teuthology.orchestra.run.smithi179.stdout: "oplen_avg": { 2023-07-18T03:39:58.753 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 118407, 2023-07-18T03:39:58.753 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 119160 2023-07-18T03:39:58.753 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.754 INFO:teuthology.orchestra.run.smithi179.stdout: "op": 118407, 2023-07-18T03:39:58.754 INFO:teuthology.orchestra.run.smithi179.stdout: "op_r": 4729, 2023-07-18T03:39:58.754 INFO:teuthology.orchestra.run.smithi179.stdout: "op_w": 113678, 2023-07-18T03:39:58.754 INFO:teuthology.orchestra.run.smithi179.stdout: "op_rmw": 0, 2023-07-18T03:39:58.754 INFO:teuthology.orchestra.run.smithi179.stdout: "op_pg": 0, 2023-07-18T03:39:58.755 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_stat": 4, 2023-07-18T03:39:58.755 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_create": 51, 2023-07-18T03:39:58.755 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_read": 834, 2023-07-18T03:39:58.755 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_write": 64040, 2023-07-18T03:39:58.755 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writefull": 500, 2023-07-18T03:39:58.756 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writesame": 0, 2023-07-18T03:39:58.756 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_append": 0, 2023-07-18T03:39:58.756 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_zero": 0, 2023-07-18T03:39:58.756 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_truncate": 18, 2023-07-18T03:39:58.756 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_delete": 48658, 2023-07-18T03:39:58.757 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_mapext": 0, 2023-07-18T03:39:58.757 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_sparse_read": 0, 2023-07-18T03:39:58.757 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_clonerange": 0, 2023-07-18T03:39:58.757 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_getxattr": 3874, 2023-07-18T03:39:58.758 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_setxattr": 98, 2023-07-18T03:39:58.758 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_cmpxattr": 0, 2023-07-18T03:39:58.758 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_rmxattr": 0, 2023-07-18T03:39:58.758 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_resetxattrs": 0, 2023-07-18T03:39:58.759 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_call": 0, 2023-07-18T03:39:58.759 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_watch": 0, 2023-07-18T03:39:58.759 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_notify": 0, 2023-07-18T03:39:58.759 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_src_cmpxattr": 0, 2023-07-18T03:39:58.759 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls": 0, 2023-07-18T03:39:58.760 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls_filter": 0, 2023-07-18T03:39:58.760 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_other": 0, 2023-07-18T03:39:58.760 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_active": 0, 2023-07-18T03:39:58.760 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_send": 0, 2023-07-18T03:39:58.761 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_resend": 0, 2023-07-18T03:39:58.761 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_ping": 0, 2023-07-18T03:39:58.761 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_active": 0, 2023-07-18T03:39:58.761 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_send": 0, 2023-07-18T03:39:58.761 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_resend": 0, 2023-07-18T03:39:58.762 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_active": 0, 2023-07-18T03:39:58.762 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_send": 0, 2023-07-18T03:39:58.762 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_resend": 0, 2023-07-18T03:39:58.762 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_active": 0, 2023-07-18T03:39:58.762 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_send": 0, 2023-07-18T03:39:58.763 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_resend": 0, 2023-07-18T03:39:58.763 INFO:teuthology.orchestra.run.smithi179.stdout: "command_active": 0, 2023-07-18T03:39:58.763 INFO:teuthology.orchestra.run.smithi179.stdout: "command_send": 0, 2023-07-18T03:39:58.763 INFO:teuthology.orchestra.run.smithi179.stdout: "command_resend": 0, 2023-07-18T03:39:58.763 INFO:teuthology.orchestra.run.smithi179.stdout: "map_epoch": 144, 2023-07-18T03:39:58.764 INFO:teuthology.orchestra.run.smithi179.stdout: "map_full": 0, 2023-07-18T03:39:58.764 INFO:teuthology.orchestra.run.smithi179.stdout: "map_inc": 62, 2023-07-18T03:39:58.764 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_sessions": 12, 2023-07-18T03:39:58.764 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_open": 12, 2023-07-18T03:39:58.765 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_close": 0, 2023-07-18T03:39:58.765 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_laggy": 0, 2023-07-18T03:39:58.765 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_wr": 790, 2023-07-18T03:39:58.765 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_rd": 66, 2023-07-18T03:39:58.765 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_del": 227 2023-07-18T03:39:58.766 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.766 INFO:teuthology.orchestra.run.smithi179.stdout: "oft": { 2023-07-18T03:39:58.766 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_objs": 1, 2023-07-18T03:39:58.766 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_kv_pairs": 0, 2023-07-18T03:39:58.766 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_updates": 0, 2023-07-18T03:39:58.767 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_removes": 0 2023-07-18T03:39:58.767 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.767 INFO:teuthology.orchestra.run.smithi179.stdout: "osd-slow-ops": { 2023-07-18T03:39:58.767 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_ops_count": 231050 2023-07-18T03:39:58.767 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.768 INFO:teuthology.orchestra.run.smithi179.stdout: "purge_queue": { 2023-07-18T03:39:58.768 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops": 0, 2023-07-18T03:39:58.768 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops_high_water": 16, 2023-07-18T03:39:58.768 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing": 0, 2023-07-18T03:39:58.769 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_high_water": 8, 2023-07-18T03:39:58.769 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executed": 23869, 2023-07-18T03:39:58.769 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_item_in_journal": 0 2023-07-18T03:39:58.769 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.769 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-18T03:39:58.770 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2023-07-18T03:39:58.770 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2023-07-18T03:39:58.770 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2023-07-18T03:39:58.770 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 704531, 2023-07-18T03:39:58.770 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 138298183, 2023-07-18T03:39:58.771 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2023-07-18T03:39:58.771 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 704531, 2023-07-18T03:39:58.771 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2023-07-18T03:39:58.771 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2023-07-18T03:39:58.771 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 704531, 2023-07-18T03:39:58.772 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 138298183, 2023-07-18T03:39:58.772 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2023-07-18T03:39:58.772 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:58.772 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:58.773 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:58.773 INFO:teuthology.orchestra.run.smithi179.stdout: } 2023-07-18T03:39:58.773 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.773 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_bytes": { 2023-07-18T03:39:58.773 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2023-07-18T03:39:58.774 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2023-07-18T03:39:58.774 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2023-07-18T03:39:58.774 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2023-07-18T03:39:58.775 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2023-07-18T03:39:58.775 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2023-07-18T03:39:58.775 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2023-07-18T03:39:58.775 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 118407, 2023-07-18T03:39:58.775 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 901955212, 2023-07-18T03:39:58.776 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 118407, 2023-07-18T03:39:58.776 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 901955212, 2023-07-18T03:39:58.776 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2023-07-18T03:39:58.776 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:58.777 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:58.777 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:58.777 INFO:teuthology.orchestra.run.smithi179.stdout: } 2023-07-18T03:39:58.777 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.777 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_ops": { 2023-07-18T03:39:58.778 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2023-07-18T03:39:58.778 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 1024, 2023-07-18T03:39:58.778 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2023-07-18T03:39:58.778 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2023-07-18T03:39:58.778 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2023-07-18T03:39:58.779 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2023-07-18T03:39:58.779 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2023-07-18T03:39:58.779 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 118407, 2023-07-18T03:39:58.779 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 118407, 2023-07-18T03:39:58.779 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 118407, 2023-07-18T03:39:58.780 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 118407, 2023-07-18T03:39:58.780 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2023-07-18T03:39:58.780 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:58.780 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:58.781 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:58.781 INFO:teuthology.orchestra.run.smithi179.stdout: } 2023-07-18T03:39:58.781 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.781 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle": { 2023-07-18T03:39:58.781 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2023-07-18T03:39:58.782 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2023-07-18T03:39:58.782 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2023-07-18T03:39:58.782 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 23869, 2023-07-18T03:39:58.782 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 2606913, 2023-07-18T03:39:58.782 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2023-07-18T03:39:58.783 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 23869, 2023-07-18T03:39:58.783 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2023-07-18T03:39:58.783 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2023-07-18T03:39:58.783 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 802, 2023-07-18T03:39:58.783 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 2606913, 2023-07-18T03:39:58.784 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2023-07-18T03:39:58.784 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:58.784 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:58.784 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:58.785 INFO:teuthology.orchestra.run.smithi179.stdout: } 2023-07-18T03:39:58.785 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:39:58.785 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle-0x562a6198e0a0": { 2023-07-18T03:39:58.785 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2023-07-18T03:39:58.785 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2023-07-18T03:39:58.786 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2023-07-18T03:39:58.786 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 141949, 2023-07-18T03:39:58.786 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 892736080, 2023-07-18T03:39:58.786 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2023-07-18T03:39:58.786 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 141949, 2023-07-18T03:39:58.787 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2023-07-18T03:39:58.787 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2023-07-18T03:39:58.787 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 63238, 2023-07-18T03:39:58.787 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 892736080, 2023-07-18T03:39:58.787 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2023-07-18T03:39:58.788 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:39:58.788 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:39:58.788 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:39:58.788 INFO:teuthology.orchestra.run.smithi179.stdout: } 2023-07-18T03:39:58.789 INFO:teuthology.orchestra.run.smithi179.stdout: } 2023-07-18T03:39:58.789 INFO:teuthology.orchestra.run.smithi179.stdout:} 2023-07-18T03:39:59.005 INFO:tasks.check_counter:Daemon mds.i mds.exported=31 2023-07-18T03:39:59.005 INFO:tasks.check_counter:Daemon mds.i mds.imported=28 2023-07-18T03:39:59.005 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2023-07-18T03:39:59.006 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=32 2023-07-18T03:39:59.006 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=26 2023-07-18T03:39:59.006 DEBUG:tasks.check_counter:Getting stats from l 2023-07-18T03:39:59.006 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid b1782daa-2516-11ee-9b34-001a4aab830c -- ceph daemon mds.l perf dump 2023-07-18T03:39:59.603 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/mon.c/config 2023-07-18T03:39:59.738 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:39:59 smithi101 ceph-mon[126379]: pgmap v1027: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:39:59.796 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:39:59 smithi155 ceph-mon[128885]: pgmap v1027: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:39:59.799 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:39:59 smithi179 ceph-mon[129229]: pgmap v1027: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:40:00.643 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:00 smithi179 ceph-mon[129229]: overall HEALTH_OK 2023-07-18T03:40:00.738 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:00 smithi101 ceph-mon[126379]: overall HEALTH_OK 2023-07-18T03:40:00.796 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:00 smithi155 ceph-mon[128885]: overall HEALTH_OK 2023-07-18T03:40:00.864 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2023-07-18T03:40:00.864 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-0": { 2023-07-18T03:40:00.865 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 0, 2023-07-18T03:40:00.865 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 2, 2023-07-18T03:40:00.865 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 0, 2023-07-18T03:40:00.865 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 0, 2023-07-18T03:40:00.866 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 4, 2023-07-18T03:40:00.866 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 0, 2023-07-18T03:40:00.866 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.001469556, 2023-07-18T03:40:00.866 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.000000000, 2023-07-18T03:40:00.866 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.000556741, 2023-07-18T03:40:00.867 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-18T03:40:00.867 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:40:00.867 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:40:00.867 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:40:00.867 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:40:00.868 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:40:00.868 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:40:00.868 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:40:00.868 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:40:00.868 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:40:00.869 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:40:00.869 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-18T03:40:00.869 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 0 2023-07-18T03:40:00.869 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:40:00.870 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-1": { 2023-07-18T03:40:00.870 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 578, 2023-07-18T03:40:00.870 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 566, 2023-07-18T03:40:00.870 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 1526128, 2023-07-18T03:40:00.871 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 324512, 2023-07-18T03:40:00.871 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 2, 2023-07-18T03:40:00.871 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 1, 2023-07-18T03:40:00.871 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.069082260, 2023-07-18T03:40:00.871 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.019256274, 2023-07-18T03:40:00.872 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.038398566, 2023-07-18T03:40:00.872 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.014038649, 2023-07-18T03:40:00.872 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:40:00.872 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 566, 2023-07-18T03:40:00.872 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.022973647, 2023-07-18T03:40:00.873 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000040589 2023-07-18T03:40:00.873 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:40:00.873 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:40:00.873 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:40:00.873 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:40:00.874 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:40:00.874 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:40:00.874 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 1526128, 2023-07-18T03:40:00.874 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 324512 2023-07-18T03:40:00.874 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:40:00.875 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-2": { 2023-07-18T03:40:00.875 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 4, 2023-07-18T03:40:00.876 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 433, 2023-07-18T03:40:00.876 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 640, 2023-07-18T03:40:00.876 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 379360, 2023-07-18T03:40:00.876 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 7, 2023-07-18T03:40:00.877 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 3, 2023-07-18T03:40:00.877 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.017585052, 2023-07-18T03:40:00.877 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.012180832, 2023-07-18T03:40:00.878 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.001844234, 2023-07-18T03:40:00.878 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-18T03:40:00.878 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:40:00.878 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 430, 2023-07-18T03:40:00.878 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.026722913, 2023-07-18T03:40:00.879 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000062146 2023-07-18T03:40:00.879 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:40:00.879 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:40:00.879 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:40:00.879 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:40:00.880 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:40:00.880 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:40:00.880 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 640, 2023-07-18T03:40:00.881 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 379360 2023-07-18T03:40:00.881 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:40:00.881 INFO:teuthology.orchestra.run.smithi179.stdout: "cct": { 2023-07-18T03:40:00.881 INFO:teuthology.orchestra.run.smithi179.stdout: "total_workers": 0, 2023-07-18T03:40:00.881 INFO:teuthology.orchestra.run.smithi179.stdout: "unhealthy_workers": 0 2023-07-18T03:40:00.882 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:40:00.882 INFO:teuthology.orchestra.run.smithi179.stdout: "mempool": { 2023-07-18T03:40:00.882 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_bytes": 0, 2023-07-18T03:40:00.882 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_items": 0, 2023-07-18T03:40:00.882 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_bytes": 0, 2023-07-18T03:40:00.883 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_items": 0, 2023-07-18T03:40:00.883 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_bytes": 0, 2023-07-18T03:40:00.883 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_items": 0, 2023-07-18T03:40:00.883 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-18T03:40:00.883 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_items": 0, 2023-07-18T03:40:00.884 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-18T03:40:00.884 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_items": 0, 2023-07-18T03:40:00.884 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_bytes": 0, 2023-07-18T03:40:00.884 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_items": 0, 2023-07-18T03:40:00.884 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-18T03:40:00.885 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_items": 0, 2023-07-18T03:40:00.885 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_bytes": 0, 2023-07-18T03:40:00.885 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_items": 0, 2023-07-18T03:40:00.885 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_bytes": 0, 2023-07-18T03:40:00.885 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_items": 0, 2023-07-18T03:40:00.886 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-18T03:40:00.886 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_items": 0, 2023-07-18T03:40:00.886 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-18T03:40:00.886 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_items": 0, 2023-07-18T03:40:00.886 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_bytes": 0, 2023-07-18T03:40:00.887 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_items": 0, 2023-07-18T03:40:00.887 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_bytes": 0, 2023-07-18T03:40:00.887 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_items": 0, 2023-07-18T03:40:00.887 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-18T03:40:00.887 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_items": 0, 2023-07-18T03:40:00.888 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_bytes": 0, 2023-07-18T03:40:00.888 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_items": 0, 2023-07-18T03:40:00.888 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_bytes": 0, 2023-07-18T03:40:00.888 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_items": 0, 2023-07-18T03:40:00.888 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_bytes": 0, 2023-07-18T03:40:00.889 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_items": 0, 2023-07-18T03:40:00.889 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_bytes": 0, 2023-07-18T03:40:00.889 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_items": 0, 2023-07-18T03:40:00.889 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_bytes": 148961, 2023-07-18T03:40:00.889 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_items": 29, 2023-07-18T03:40:00.890 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_bytes": 0, 2023-07-18T03:40:00.890 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_items": 0, 2023-07-18T03:40:00.890 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_bytes": 0, 2023-07-18T03:40:00.890 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_items": 0, 2023-07-18T03:40:00.891 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_bytes": 0, 2023-07-18T03:40:00.891 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_items": 0, 2023-07-18T03:40:00.891 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_bytes": 0, 2023-07-18T03:40:00.891 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_items": 0, 2023-07-18T03:40:00.891 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_bytes": 0, 2023-07-18T03:40:00.892 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_items": 0, 2023-07-18T03:40:00.892 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_bytes": 0, 2023-07-18T03:40:00.892 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_items": 0, 2023-07-18T03:40:00.892 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_bytes": 0, 2023-07-18T03:40:00.892 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_items": 0, 2023-07-18T03:40:00.893 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_bytes": 928, 2023-07-18T03:40:00.893 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_items": 2, 2023-07-18T03:40:00.893 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_bytes": 0, 2023-07-18T03:40:00.893 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_items": 0, 2023-07-18T03:40:00.893 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_bytes": 0, 2023-07-18T03:40:00.894 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_items": 0 2023-07-18T03:40:00.894 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2023-07-18T03:40:00.894 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-18T03:40:00.894 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2023-07-18T03:40:00.894 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2023-07-18T03:40:00.895 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2023-07-18T03:40:00.895 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 578, 2023-07-18T03:40:00.895 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 1442045, 2023-07-18T03:40:00.895 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2023-07-18T03:40:00.895 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 578, 2023-07-18T03:40:00.896 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2023-07-18T03:40:00.896 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2023-07-18T03:40:00.896 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 578, 2023-07-18T03:40:00.896 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 1442045, 2023-07-18T03:40:00.896 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2023-07-18T03:40:00.897 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2023-07-18T03:40:00.897 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2023-07-18T03:40:00.897 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2023-07-18T03:40:00.897 INFO:teuthology.orchestra.run.smithi179.stdout: } 2023-07-18T03:40:00.897 INFO:teuthology.orchestra.run.smithi179.stdout: } 2023-07-18T03:40:00.898 INFO:teuthology.orchestra.run.smithi179.stdout:} 2023-07-18T03:40:01.281 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2023-07-18T03:40:01.281 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2023-07-18T03:40:01.281 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2023-07-18T03:40:01.282 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2023-07-18T03:40:01.282 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2023-07-18T03:40:01.282 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-07-18T03:40:01.292 INFO:tasks.kclient:Unmounting kernel clients... 2023-07-18T03:40:01.293 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-07-18T03:40:01.293 DEBUG:teuthology.orchestra.run.smithi101:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:40:01.340 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-07-18T03:40:01.341 DEBUG:teuthology.orchestra.run.smithi101:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:40:01.425 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:40:01.670 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.674+0000 7f0bf8fff700 1 -- 172.21.15.101:0/3349235405 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bf40ffaf0 msgr2=0x7f0bf40fff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:01.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.674+0000 7f0bf8fff700 1 --2- 172.21.15.101:0/3349235405 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bf40ffaf0 0x7f0bf40fff60 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f0be80098f0 tx=0x7f0be8009c00 comp rx=0 tx=0).stop 2023-07-18T03:40:01.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.675+0000 7f0bf8fff700 1 -- 172.21.15.101:0/3349235405 shutdown_connections 2023-07-18T03:40:01.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.675+0000 7f0bf8fff700 1 --2- 172.21.15.101:0/3349235405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0bf4105b00 0x7f0bf4105ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.671 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.675+0000 7f0bf8fff700 1 --2- 172.21.15.101:0/3349235405 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0bf41004a0 0x7f0bf410c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.675+0000 7f0bf8fff700 1 --2- 172.21.15.101:0/3349235405 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bf40ffaf0 0x7f0bf40fff60 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.675+0000 7f0bf8fff700 1 -- 172.21.15.101:0/3349235405 >> 172.21.15.101:0/3349235405 conn(0x7f0bf40fb630 msgr2=0x7f0bf40fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:01.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.675+0000 7f0bf8fff700 1 -- 172.21.15.101:0/3349235405 shutdown_connections 2023-07-18T03:40:01.672 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.675+0000 7f0bf8fff700 1 -- 172.21.15.101:0/3349235405 wait complete. 2023-07-18T03:40:01.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.676+0000 7f0bf8fff700 1 Processor -- start 2023-07-18T03:40:01.674 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.676+0000 7f0bf8fff700 1 -- start start 2023-07-18T03:40:01.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.677+0000 7f0bf8fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bf40ffaf0 0x7f0bf419b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:01.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.677+0000 7f0bf8fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0bf41004a0 0x7f0bf419bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:01.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.677+0000 7f0bf8fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0bf4105b00 0x7f0bf4195a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:01.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.677+0000 7f0bf8fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf41960b0 con 0x7f0bf4105b00 2023-07-18T03:40:01.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.677+0000 7f0bf8fff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf4196220 con 0x7f0bf41004a0 2023-07-18T03:40:01.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.677+0000 7f0bf8fff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf41963e0 con 0x7f0bf40ffaf0 2023-07-18T03:40:01.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.677+0000 7f0bf259c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bf40ffaf0 0x7f0bf419b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:01.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.677+0000 7f0bf259c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bf40ffaf0 0x7f0bf419b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:33896/0 (socket says 172.21.15.101:33896) 2023-07-18T03:40:01.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.677+0000 7f0bf1d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0bf41004a0 0x7f0bf419bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:01.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.677+0000 7f0bf259c700 1 -- 172.21.15.101:0/3749472903 learned_addr learned my addr 172.21.15.101:0/3749472903 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:40:01.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.677+0000 7f0bf2d9d700 1 --2- 172.21.15.101:0/3749472903 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0bf4105b00 0x7f0bf4195a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:01.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.677+0000 7f0bf259c700 1 -- 172.21.15.101:0/3749472903 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0bf41004a0 msgr2=0x7f0bf419bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:01.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.677+0000 7f0bf259c700 1 --2- 172.21.15.101:0/3749472903 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0bf41004a0 0x7f0bf419bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.677 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.677+0000 7f0bf259c700 1 -- 172.21.15.101:0/3749472903 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0bf4105b00 msgr2=0x7f0bf4195a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:01.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.677+0000 7f0bf259c700 1 --2- 172.21.15.101:0/3749472903 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0bf4105b00 0x7f0bf4195a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.677+0000 7f0bf259c700 1 -- 172.21.15.101:0/3749472903 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be8009520 con 0x7f0bf40ffaf0 2023-07-18T03:40:01.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.677+0000 7f0bf1d9b700 1 --2- 172.21.15.101:0/3749472903 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0bf41004a0 0x7f0bf419bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:01.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.678+0000 7f0bf259c700 1 --2- 172.21.15.101:0/3749472903 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bf40ffaf0 0x7f0bf419b9d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0be400e9c0 tx=0x7f0be400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:01.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.678+0000 7f0be37fe700 1 -- 172.21.15.101:0/3749472903 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be4004a30 con 0x7f0bf40ffaf0 2023-07-18T03:40:01.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.678+0000 7f0be37fe700 1 -- 172.21.15.101:0/3749472903 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0be400cc80 con 0x7f0bf40ffaf0 2023-07-18T03:40:01.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.678+0000 7f0bf8fff700 1 -- 172.21.15.101:0/3749472903 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bf41966c0 con 0x7f0bf40ffaf0 2023-07-18T03:40:01.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.678+0000 7f0be37fe700 1 -- 172.21.15.101:0/3749472903 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be40054e0 con 0x7f0bf40ffaf0 2023-07-18T03:40:01.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.678+0000 7f0bf8fff700 1 -- 172.21.15.101:0/3749472903 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bf41a29c0 con 0x7f0bf40ffaf0 2023-07-18T03:40:01.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.679+0000 7f0bf8fff700 1 -- 172.21.15.101:0/3749472903 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bf4067300 con 0x7f0bf40ffaf0 2023-07-18T03:40:01.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.681+0000 7f0be37fe700 1 -- 172.21.15.101:0/3749472903 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f0be4004b90 con 0x7f0bf40ffaf0 2023-07-18T03:40:01.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.682+0000 7f0be37fe700 1 --2- 172.21.15.101:0/3749472903 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0bdc0a1300 0x7f0bdc0a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:01.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.682+0000 7f0be37fe700 1 -- 172.21.15.101:0/3749472903 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f0be4014070 con 0x7f0bf40ffaf0 2023-07-18T03:40:01.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.683+0000 7f0bf1d9b700 1 --2- 172.21.15.101:0/3749472903 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0bdc0a1300 0x7f0bdc0a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:01.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.683+0000 7f0bf1d9b700 1 --2- 172.21.15.101:0/3749472903 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0bdc0a1300 0x7f0bdc0a37b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0be8005e90 tx=0x7f0be8019040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:01.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.685+0000 7f0be37fe700 1 -- 172.21.15.101:0/3749472903 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0be40968d0 con 0x7f0bf40ffaf0 2023-07-18T03:40:01.701 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:01 smithi179 ceph-mon[129229]: pgmap v1028: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:40:01.738 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:01 smithi101 ceph-mon[126379]: pgmap v1028: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:40:01.796 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:01 smithi155 ceph-mon[128885]: pgmap v1028: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-18T03:40:01.881 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.886+0000 7f0bf8fff700 1 -- 172.21.15.101:0/3749472903 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0bf41a2cf0 con 0x7f0bf40ffaf0 2023-07-18T03:40:01.883 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.888+0000 7f0be37fe700 1 -- 172.21.15.101:0/3749472903 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) v1 ==== 74+0+25296 (secure 0 0 0) 0x7f0be4099ef0 con 0x7f0bf40ffaf0 2023-07-18T03:40:01.884 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:40:01.885 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":144,"fsid":"b1782daa-2516-11ee-9b34-001a4aab830c","created":"2023-07-18T02:57:22.972483+0000","modified":"2023-07-18T03:39:19.449794+0000","last_up_change":"2023-07-18T03:04:11.429530+0000","last_in_change":"2023-07-18T03:03:54.158340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:00:43.353489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:04:57.771616+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-18T03:04:58.769590+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":59,"snap_epoch":143,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-18T03:05:02.358830+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":59,"snap_epoch":143,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc76b5d1-38a0-4f1a-9335-756bfc40a62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6803","nonce":3610719929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6805","nonce":3610719929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6809","nonce":3610719929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6807","nonce":3610719929}]},"public_addr":"172.21.15.101:6803/3610719929","cluster_addr":"172.21.15.101:6805/3610719929","heartbeat_back_addr":"172.21.15.101:6809/3610719929","heartbeat_front_addr":"172.21.15.101:6807/3610719929","state":["exists","up"]},{"osd":1,"uuid":"2a299573-0b34-4af9-aaae-b523e5058ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6801","nonce":1030229579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6803","nonce":1030229579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6807","nonce":1030229579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6805","nonce":1030229579}]},"public_addr":"172.21.15.155:6801/1030229579","cluster_addr":"172.21.15.155:6803/1030229579","heartbeat_back_addr":"172.21.15.155:6807/1030229579","heartbeat_front_addr":"172.21.15.155:6805/1030229579","state":["exists","up"]},{"osd":2,"uuid":"123b3749-c636-48fa-9e33-6229b5575dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6801","nonce":1349487507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6803","nonce":1349487507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6807","nonce":1349487507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6805","nonce":1349487507}]},"public_addr":"172.21.15.179:6801/1349487507","cluster_addr":"172.21.15.179:6803/1349487507","heartbeat_back_addr":"172.21.15.179:6807/1349487507","heartbeat_front_addr":"172.21.15.179:6805/1349487507","state":["exists","up"]},{"osd":3,"uuid":"4b0c409d-2954-4127-833a-b196bf226d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6811","nonce":4226577463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6813","nonce":4226577463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6817","nonce":4226577463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6815","nonce":4226577463}]},"public_addr":"172.21.15.101:6811/4226577463","cluster_addr":"172.21.15.101:6813/4226577463","heartbeat_back_addr":"172.21.15.101:6817/4226577463","heartbeat_front_addr":"172.21.15.101:6815/4226577463","state":["exists","up"]},{"osd":4,"uuid":"2137f2df-984e-45ce-baf6-b4ac12c7e3c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6809","nonce":1951135805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6811","nonce":1951135805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6815","nonce":1951135805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6813","nonce":1951135805}]},"public_addr":"172.21.15.155:6809/1951135805","cluster_addr":"172.21.15.155:6811/1951135805","heartbeat_back_addr":"172.21.15.155:6815/1951135805","heartbeat_front_addr":"172.21.15.155:6813/1951135805","state":["exists","up"]},{"osd":5,"uuid":"1020bfc5-8c2f-482e-9588-66c7f98e46e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6809","nonce":2746037021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6811","nonce":2746037021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6815","nonce":2746037021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6813","nonce":2746037021}]},"public_addr":"172.21.15.179:6809/2746037021","cluster_addr":"172.21.15.179:6811/2746037021","heartbeat_back_addr":"172.21.15.179:6815/2746037021","heartbeat_front_addr":"172.21.15.179:6813/2746037021","state":["exists","up"]},{"osd":6,"uuid":"2cd99260-5c36-4603-9338-cf697970ce58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6819","nonce":958044931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6821","nonce":958044931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6825","nonce":958044931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6823","nonce":958044931}]},"public_addr":"172.21.15.101:6819/958044931","cluster_addr":"172.21.15.101:6821/958044931","heartbeat_back_addr":"172.21.15.101:6825/958044931","heartbeat_front_addr":"172.21.15.101:6823/958044931","state":["exists","up"]},{"osd":7,"uuid":"ed23fd0a-678f-4e15-b42c-cfe873315714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6817","nonce":3256322291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6819","nonce":3256322291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6823","nonce":3256322291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6821","nonce":3256322291}]},"public_addr":"172.21.15.155:6817/3256322291","cluster_addr":"172.21.15.155:6819/3256322291","heartbeat_back_addr":"172.21.15.155:6823/3256322291","heartbeat_front_addr":"172.21.15.155:6821/3256322291","state":["exists","up"]},{"osd":8,"uuid":"eca3d8e6-7baf-47a5-b991-328001c7ebef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6817","nonce":3520612559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6819","nonce":3520612559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6823","nonce":3520612559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6821","nonce":3520612559}]},"public_addr":"172.21.15.179:6817/3520612559","cluster_addr":"172.21.15.179:6819/3520612559","heartbeat_back_addr":"172.21.15.179:6823/3520612559","heartbeat_front_addr":"172.21.15.179:6821/3520612559","state":["exists","up"]},{"osd":9,"uuid":"465a7ffb-67e4-49ef-812c-1b96feacfebf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6827","nonce":2166787667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6829","nonce":2166787667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6833","nonce":2166787667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6831","nonce":2166787667}]},"public_addr":"172.21.15.101:6827/2166787667","cluster_addr":"172.21.15.101:6829/2166787667","heartbeat_back_addr":"172.21.15.101:6833/2166787667","heartbeat_front_addr":"172.21.15.101:6831/2166787667","state":["exists","up"]},{"osd":10,"uuid":"1897a027-f25d-434f-8df5-5fc2bd866578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6825","nonce":3682446349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6827","nonce":3682446349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6831","nonce":3682446349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6829","nonce":3682446349}]},"public_addr":"172.21.15.155:6825/3682446349","cluster_addr":"172.21.15.155:6827/3682446349","heartbeat_back_addr":"172.21.15.155:6831/3682446349","heartbeat_front_addr":"172.21.15.155:6829/3682446349","state":["exists","up"]},{"osd":11,"uuid":"b62effaf-45e2-4585-b8d0-db448f3474ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6825","nonce":2602640858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6827","nonce":2602640858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6831","nonce":2602640858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6829","nonce":2602640858}]},"public_addr":"172.21.15.179:6825/2602640858","cluster_addr":"172.21.15.179:6827/2602640858","heartbeat_back_addr":"172.21.15.179:6831/2602640858","heartbeat_front_addr":"172.21.15.179:6829/2602640858","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T02:59:56.074519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:19.358142+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:41.128966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:08.227054+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:31.442592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:52.751685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:19.251054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:40.698241+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:01.339150+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:27.955910+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:49.136980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:04:10.084982+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.101:0/2766365637":"2023-07-19T03:07:33.498516+0000","172.21.15.101:0/523372970":"2023-07-19T03:07:33.498516+0000","172.21.15.101:6800/3963742842":"2023-07-19T03:07:33.498516+0000","172.21.15.101:6800/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/1806538035":"2023-07-19T03:07:33.498516+0000","172.21.15.101:6801/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/604228291":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/1004430030":"2023-07-19T03:07:33.498516+0000","172.21.15.101:6801/3963742842":"2023-07-19T03:07:33.498516+0000","172.21.15.101:0/4055187849":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/2844574324":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/1603248215":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6801/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6800/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/3307578182":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/373189792":"2023-07-19T02:57:56.243200+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":46,"length":1},{"begin":59,"length":1}]},{"pool":4,"snaps":[{"begin":46,"length":1},{"begin":59,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:40:01.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.890+0000 7f0bf8fff700 1 -- 172.21.15.101:0/3749472903 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0bdc0a1300 msgr2=0x7f0bdc0a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:01.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.890+0000 7f0bf8fff700 1 --2- 172.21.15.101:0/3749472903 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0bdc0a1300 0x7f0bdc0a37b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0be8005e90 tx=0x7f0be8019040 comp rx=0 tx=0).stop 2023-07-18T03:40:01.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.890+0000 7f0bf8fff700 1 -- 172.21.15.101:0/3749472903 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bf40ffaf0 msgr2=0x7f0bf419b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:01.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.890+0000 7f0bf8fff700 1 --2- 172.21.15.101:0/3749472903 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bf40ffaf0 0x7f0bf419b9d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0be400e9c0 tx=0x7f0be400ecd0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.891+0000 7f0bf8fff700 1 -- 172.21.15.101:0/3749472903 shutdown_connections 2023-07-18T03:40:01.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.891+0000 7f0bf8fff700 1 --2- 172.21.15.101:0/3749472903 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0bf4105b00 0x7f0bf4195a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.891+0000 7f0bf8fff700 1 --2- 172.21.15.101:0/3749472903 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bf40ffaf0 0x7f0bf419b9d0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.891+0000 7f0bf8fff700 1 --2- 172.21.15.101:0/3749472903 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f0bdc0a1300 0x7f0bdc0a37b0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.891+0000 7f0bf8fff700 1 --2- 172.21.15.101:0/3749472903 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0bf41004a0 0x7f0bf419bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.891+0000 7f0bf8fff700 1 -- 172.21.15.101:0/3749472903 >> 172.21.15.101:0/3749472903 conn(0x7f0bf40fb630 msgr2=0x7f0bf40fbc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:01.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.891+0000 7f0bf8fff700 1 -- 172.21.15.101:0/3749472903 shutdown_connections 2023-07-18T03:40:01.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:01.891+0000 7f0bf8fff700 1 -- 172.21.15.101:0/3749472903 wait complete. 2023-07-18T03:40:01.905 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:00:43.353489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-18T03:04:57.771616+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-18T03:04:58.769590+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '143', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 59, 'snap_epoch': 143, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-18T03:05:02.358830+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '143', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 59, 'snap_epoch': 143, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-18T03:40:01.906 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-18T03:40:02.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.114+0000 7f04425ff700 1 -- 172.21.15.101:0/2071697428 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f043c105af0 msgr2=0x7f043c105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:02.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.114+0000 7f04425ff700 1 --2- 172.21.15.101:0/2071697428 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f043c105af0 0x7f043c105ec0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f042c009990 tx=0x7f042c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:40:02.109 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.114+0000 7f04425ff700 1 -- 172.21.15.101:0/2071697428 shutdown_connections 2023-07-18T03:40:02.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.114+0000 7f04425ff700 1 --2- 172.21.15.101:0/2071697428 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f043c100490 0x7f043c10c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:02.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.114+0000 7f04425ff700 1 --2- 172.21.15.101:0/2071697428 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f043c105af0 0x7f043c105ec0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:02.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.114+0000 7f04425ff700 1 --2- 172.21.15.101:0/2071697428 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f043c0ffae0 0x7f043c0fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:02.110 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.114+0000 7f04425ff700 1 -- 172.21.15.101:0/2071697428 >> 172.21.15.101:0/2071697428 conn(0x7f043c0fb5e0 msgr2=0x7f043c0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:02.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.114+0000 7f04425ff700 1 -- 172.21.15.101:0/2071697428 shutdown_connections 2023-07-18T03:40:02.111 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.114+0000 7f04425ff700 1 -- 172.21.15.101:0/2071697428 wait complete. 2023-07-18T03:40:02.112 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.115+0000 7f04425ff700 1 Processor -- start 2023-07-18T03:40:02.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.115+0000 7f04425ff700 1 -- start start 2023-07-18T03:40:02.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.115+0000 7f04425ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f043c0ffae0 0x7f043c197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:02.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.115+0000 7f04425ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f043c100490 0x7f043c1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:02.113 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.115+0000 7f04425ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f043c105af0 0x7f043c19c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:02.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.115+0000 7f04425ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043c19ce90 con 0x7f043c0ffae0 2023-07-18T03:40:02.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.115+0000 7f04425ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043c19d000 con 0x7f043c105af0 2023-07-18T03:40:02.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.115+0000 7f04425ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043c19d1c0 con 0x7f043c100490 2023-07-18T03:40:02.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.116+0000 7f043bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f043c0ffae0 0x7f043c197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:02.114 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.116+0000 7f043b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f043c100490 0x7f043c1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:02.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.116+0000 7f043bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f043c0ffae0 0x7f043c197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:50618/0 (socket says 172.21.15.101:50618) 2023-07-18T03:40:02.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.116+0000 7f043bfff700 1 -- 172.21.15.101:0/612729481 learned_addr learned my addr 172.21.15.101:0/612729481 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:40:02.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.116+0000 7f043b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f043c100490 0x7f043c1983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:33906/0 (socket says 172.21.15.101:33906) 2023-07-18T03:40:02.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.116+0000 7f0440b9c700 1 --2- 172.21.15.101:0/612729481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f043c105af0 0x7f043c19c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:02.115 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.116+0000 7f043bfff700 1 -- 172.21.15.101:0/612729481 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f043c100490 msgr2=0x7f043c1983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:02.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.116+0000 7f043bfff700 1 --2- 172.21.15.101:0/612729481 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f043c100490 0x7f043c1983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:02.116 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.116+0000 7f043bfff700 1 -- 172.21.15.101:0/612729481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f043c105af0 msgr2=0x7f043c19c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:02.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.116+0000 7f043bfff700 1 --2- 172.21.15.101:0/612729481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f043c105af0 0x7f043c19c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:02.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.116+0000 7f043bfff700 1 -- 172.21.15.101:0/612729481 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0434009c40 con 0x7f043c0ffae0 2023-07-18T03:40:02.117 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.116+0000 7f043bfff700 1 --2- 172.21.15.101:0/612729481 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f043c0ffae0 0x7f043c197eb0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f042c005910 tx=0x7f042c004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:02.118 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.116+0000 7f04397fa700 1 -- 172.21.15.101:0/612729481 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042c01d070 con 0x7f043c0ffae0 2023-07-18T03:40:02.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.117+0000 7f04425ff700 1 -- 172.21.15.101:0/612729481 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f042c0095f0 con 0x7f043c0ffae0 2023-07-18T03:40:02.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.117+0000 7f04397fa700 1 -- 172.21.15.101:0/612729481 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f042c022470 con 0x7f043c0ffae0 2023-07-18T03:40:02.119 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.117+0000 7f04397fa700 1 -- 172.21.15.101:0/612729481 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042c017650 con 0x7f043c0ffae0 2023-07-18T03:40:02.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.117+0000 7f04425ff700 1 -- 172.21.15.101:0/612729481 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f043c1a4b00 con 0x7f043c0ffae0 2023-07-18T03:40:02.120 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.118+0000 7f04425ff700 1 -- 172.21.15.101:0/612729481 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f043c067290 con 0x7f043c0ffae0 2023-07-18T03:40:02.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.119+0000 7f04397fa700 1 -- 172.21.15.101:0/612729481 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f042c0225e0 con 0x7f043c0ffae0 2023-07-18T03:40:02.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.121+0000 7f04397fa700 1 --2- 172.21.15.101:0/612729481 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f04240a12c0 0x7f04240a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:02.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.121+0000 7f04397fa700 1 -- 172.21.15.101:0/612729481 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f042c0ce320 con 0x7f043c0ffae0 2023-07-18T03:40:02.121 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.121+0000 7f043b7fe700 1 --2- 172.21.15.101:0/612729481 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f04240a12c0 0x7f04240a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:02.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.122+0000 7f043b7fe700 1 --2- 172.21.15.101:0/612729481 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f04240a12c0 0x7f04240a3770 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f043c1994c0 tx=0x7f0430009260 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:02.122 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.125+0000 7f04397fa700 1 -- 172.21.15.101:0/612729481 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f042c05c790 con 0x7f043c0ffae0 2023-07-18T03:40:02.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.322+0000 7f04425ff700 1 -- 172.21.15.101:0/612729481 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f043c1a4c90 con 0x7f043c0ffae0 2023-07-18T03:40:02.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.323+0000 7f04397fa700 1 -- 172.21.15.101:0/612729481 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v144) v1 ==== 93+0+10 (secure 0 0 0) 0x7f042c0973b0 con 0x7f043c0ffae0 2023-07-18T03:40:02.318 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 1 2023-07-18T03:40:02.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.325+0000 7f04425ff700 1 -- 172.21.15.101:0/612729481 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f04240a12c0 msgr2=0x7f04240a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:02.320 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.325+0000 7f04425ff700 1 --2- 172.21.15.101:0/612729481 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f04240a12c0 0x7f04240a3770 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f043c1994c0 tx=0x7f0430009260 comp rx=0 tx=0).stop 2023-07-18T03:40:02.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.325+0000 7f04425ff700 1 -- 172.21.15.101:0/612729481 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f043c0ffae0 msgr2=0x7f043c197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:02.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.325+0000 7f04425ff700 1 --2- 172.21.15.101:0/612729481 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f043c0ffae0 0x7f043c197eb0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f042c005910 tx=0x7f042c004710 comp rx=0 tx=0).stop 2023-07-18T03:40:02.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.325+0000 7f04425ff700 1 -- 172.21.15.101:0/612729481 shutdown_connections 2023-07-18T03:40:02.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.325+0000 7f04425ff700 1 --2- 172.21.15.101:0/612729481 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f04240a12c0 0x7f04240a3770 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:02.321 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.325+0000 7f04425ff700 1 --2- 172.21.15.101:0/612729481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f043c105af0 0x7f043c19c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:02.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.325+0000 7f04425ff700 1 --2- 172.21.15.101:0/612729481 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f043c0ffae0 0x7f043c197eb0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:02.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.325+0000 7f04425ff700 1 --2- 172.21.15.101:0/612729481 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f043c100490 0x7f043c1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:02.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.325+0000 7f04425ff700 1 -- 172.21.15.101:0/612729481 >> 172.21.15.101:0/612729481 conn(0x7f043c0fb5e0 msgr2=0x7f043c10ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:02.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.325+0000 7f04425ff700 1 -- 172.21.15.101:0/612729481 shutdown_connections 2023-07-18T03:40:02.322 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.325+0000 7f04425ff700 1 -- 172.21.15.101:0/612729481 wait complete. 2023-07-18T03:40:02.339 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-18T03:40:02.540 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.546+0000 7f55dd9ff700 1 -- 172.21.15.101:0/1499685116 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55d8105af0 msgr2=0x7f55d8105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:02.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.546+0000 7f55dd9ff700 1 --2- 172.21.15.101:0/1499685116 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55d8105af0 0x7f55d8105ec0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f55c80098f0 tx=0x7f55c8009c00 comp rx=0 tx=0).stop 2023-07-18T03:40:02.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.546+0000 7f55dd9ff700 1 -- 172.21.15.101:0/1499685116 shutdown_connections 2023-07-18T03:40:02.541 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.546+0000 7f55dd9ff700 1 --2- 172.21.15.101:0/1499685116 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55d8100490 0x7f55d810c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:02.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.546+0000 7f55dd9ff700 1 --2- 172.21.15.101:0/1499685116 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55d8105af0 0x7f55d8105ec0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:02.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.546+0000 7f55dd9ff700 1 --2- 172.21.15.101:0/1499685116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55d80ffae0 0x7f55d80fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:02.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.546+0000 7f55dd9ff700 1 -- 172.21.15.101:0/1499685116 >> 172.21.15.101:0/1499685116 conn(0x7f55d80fb5e0 msgr2=0x7f55d80fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:02.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.546+0000 7f55dd9ff700 1 -- 172.21.15.101:0/1499685116 shutdown_connections 2023-07-18T03:40:02.542 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.546+0000 7f55dd9ff700 1 -- 172.21.15.101:0/1499685116 wait complete. 2023-07-18T03:40:02.544 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.547+0000 7f55dd9ff700 1 Processor -- start 2023-07-18T03:40:02.544 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.547+0000 7f55dd9ff700 1 -- start start 2023-07-18T03:40:02.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.547+0000 7f55dd9ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55d80ffae0 0x7f55d8072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:02.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.547+0000 7f55dd9ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55d8100490 0x7f55d8072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:02.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.547+0000 7f55dd9ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55d8105af0 0x7f55d81a3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:02.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.547+0000 7f55dd9ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d8073580 con 0x7f55d8105af0 2023-07-18T03:40:02.545 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.547+0000 7f55dd9ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d81a4340 con 0x7f55d80ffae0 2023-07-18T03:40:02.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.547+0000 7f55dd9ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d81a4480 con 0x7f55d8100490 2023-07-18T03:40:02.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.547+0000 7f55d6ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55d80ffae0 0x7f55d8072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:02.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.547+0000 7f55cffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55d8100490 0x7f55d8072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:02.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.547+0000 7f55d77fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55d8105af0 0x7f55d81a3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:02.546 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.547+0000 7f55d6ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55d80ffae0 0x7f55d8072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:50742/0 (socket says 172.21.15.101:50742) 2023-07-18T03:40:02.547 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.547+0000 7f55cffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55d8100490 0x7f55d8072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:33932/0 (socket says 172.21.15.101:33932) 2023-07-18T03:40:02.547 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.547+0000 7f55d6ffd700 1 -- 172.21.15.101:0/1621490042 learned_addr learned my addr 172.21.15.101:0/1621490042 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:40:02.547 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.548+0000 7f55d6ffd700 1 -- 172.21.15.101:0/1621490042 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55d8100490 msgr2=0x7f55d8072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:02.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.548+0000 7f55d6ffd700 1 --2- 172.21.15.101:0/1621490042 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55d8100490 0x7f55d8072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:02.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.548+0000 7f55d6ffd700 1 -- 172.21.15.101:0/1621490042 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55d8105af0 msgr2=0x7f55d81a3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:02.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.548+0000 7f55d6ffd700 1 --2- 172.21.15.101:0/1621490042 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55d8105af0 0x7f55d81a3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:02.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.548+0000 7f55d6ffd700 1 -- 172.21.15.101:0/1621490042 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c8009520 con 0x7f55d80ffae0 2023-07-18T03:40:02.548 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.548+0000 7f55cffff700 1 --2- 172.21.15.101:0/1621490042 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55d8100490 0x7f55d8072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:02.549 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.548+0000 7f55d77fe700 1 --2- 172.21.15.101:0/1621490042 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55d8105af0 0x7f55d81a3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:02.549 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.548+0000 7f55d6ffd700 1 --2- 172.21.15.101:0/1621490042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55d80ffae0 0x7f55d8072810 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f55c8009810 tx=0x7f55c800fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:02.549 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.548+0000 7f55dc9fd700 1 -- 172.21.15.101:0/1621490042 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c801c070 con 0x7f55d80ffae0 2023-07-18T03:40:02.549 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.548+0000 7f55dc9fd700 1 -- 172.21.15.101:0/1621490042 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55c800bbd0 con 0x7f55d80ffae0 2023-07-18T03:40:02.549 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.548+0000 7f55dd9ff700 1 -- 172.21.15.101:0/1621490042 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55d81a4680 con 0x7f55d80ffae0 2023-07-18T03:40:02.550 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.548+0000 7f55dc9fd700 1 -- 172.21.15.101:0/1621490042 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c8004930 con 0x7f55d80ffae0 2023-07-18T03:40:02.550 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.549+0000 7f55dd9ff700 1 -- 172.21.15.101:0/1621490042 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55d81a4aa0 con 0x7f55d80ffae0 2023-07-18T03:40:02.550 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.550+0000 7f55dd9ff700 1 -- 172.21.15.101:0/1621490042 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55d8067290 con 0x7f55d80ffae0 2023-07-18T03:40:02.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.551+0000 7f55dc9fd700 1 -- 172.21.15.101:0/1621490042 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f55c8020070 con 0x7f55d80ffae0 2023-07-18T03:40:02.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.553+0000 7f55dc9fd700 1 --2- 172.21.15.101:0/1621490042 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f55c40a12c0 0x7f55c40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:02.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.553+0000 7f55dc9fd700 1 -- 172.21.15.101:0/1621490042 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f55c80cdcf0 con 0x7f55d80ffae0 2023-07-18T03:40:02.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.553+0000 7f55cffff700 1 --2- 172.21.15.101:0/1621490042 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f55c40a12c0 0x7f55c40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:02.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.554+0000 7f55cffff700 1 --2- 172.21.15.101:0/1621490042 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f55c40a12c0 0x7f55c40a3770 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f55d8074000 tx=0x7f55c0009310 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:02.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.556+0000 7f55dc9fd700 1 -- 172.21.15.101:0/1621490042 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f55c8096d80 con 0x7f55d80ffae0 2023-07-18T03:40:02.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:02 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3749472903' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:40:02.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:02 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/612729481' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:40:02.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.754+0000 7f55dd9ff700 1 -- 172.21.15.101:0/1621490042 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f55d81a4dc0 con 0x7f55d80ffae0 2023-07-18T03:40:02.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.755+0000 7f55dc9fd700 1 -- 172.21.15.101:0/1621490042 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v144) v1 ==== 104+0+11 (secure 0 0 0) 0x7f55c809a3a0 con 0x7f55d80ffae0 2023-07-18T03:40:02.751 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 32 2023-07-18T03:40:02.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.757+0000 7f55dd9ff700 1 -- 172.21.15.101:0/1621490042 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f55c40a12c0 msgr2=0x7f55c40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:02.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.757+0000 7f55dd9ff700 1 --2- 172.21.15.101:0/1621490042 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f55c40a12c0 0x7f55c40a3770 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f55d8074000 tx=0x7f55c0009310 comp rx=0 tx=0).stop 2023-07-18T03:40:02.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.757+0000 7f55dd9ff700 1 -- 172.21.15.101:0/1621490042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55d80ffae0 msgr2=0x7f55d8072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:02.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.757+0000 7f55dd9ff700 1 --2- 172.21.15.101:0/1621490042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55d80ffae0 0x7f55d8072810 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f55c8009810 tx=0x7f55c800fa00 comp rx=0 tx=0).stop 2023-07-18T03:40:02.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.757+0000 7f55dd9ff700 1 -- 172.21.15.101:0/1621490042 shutdown_connections 2023-07-18T03:40:02.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.757+0000 7f55dd9ff700 1 --2- 172.21.15.101:0/1621490042 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f55c40a12c0 0x7f55c40a3770 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:02.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.757+0000 7f55dd9ff700 1 --2- 172.21.15.101:0/1621490042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55d80ffae0 0x7f55d8072810 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:02.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.757+0000 7f55dd9ff700 1 --2- 172.21.15.101:0/1621490042 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55d8105af0 0x7f55d81a3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:02.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.757+0000 7f55dd9ff700 1 --2- 172.21.15.101:0/1621490042 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55d8100490 0x7f55d8072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:02.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.757+0000 7f55dd9ff700 1 -- 172.21.15.101:0/1621490042 >> 172.21.15.101:0/1621490042 conn(0x7f55d80fb5e0 msgr2=0x7f55d80fbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:02.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.757+0000 7f55dd9ff700 1 -- 172.21.15.101:0/1621490042 shutdown_connections 2023-07-18T03:40:02.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.757+0000 7f55dd9ff700 1 -- 172.21.15.101:0/1621490042 wait complete. 2023-07-18T03:40:02.771 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-18T03:40:02.796 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:02 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/3749472903' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:40:02.796 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:02 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/612729481' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:40:02.951 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:02 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/3749472903' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:40:02.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:02 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/612729481' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:40:02.975 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.980+0000 7fab3f1ff700 1 -- 172.21.15.101:0/2468098571 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab380ffae0 msgr2=0x7fab380fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:02.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.980+0000 7fab3f1ff700 1 --2- 172.21.15.101:0/2468098571 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab380ffae0 0x7fab380fff50 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fab24009990 tx=0x7fab24009ca0 comp rx=0 tx=0).stop 2023-07-18T03:40:02.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.980+0000 7fab3f1ff700 1 -- 172.21.15.101:0/2468098571 shutdown_connections 2023-07-18T03:40:02.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.980+0000 7fab3f1ff700 1 --2- 172.21.15.101:0/2468098571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab38105af0 0x7fab38105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:02.976 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.980+0000 7fab3f1ff700 1 --2- 172.21.15.101:0/2468098571 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fab38100490 0x7fab3810c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:02.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.980+0000 7fab3f1ff700 1 --2- 172.21.15.101:0/2468098571 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab380ffae0 0x7fab380fff50 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:02.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.980+0000 7fab3f1ff700 1 -- 172.21.15.101:0/2468098571 >> 172.21.15.101:0/2468098571 conn(0x7fab380fb5e0 msgr2=0x7fab380fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:02.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.980+0000 7fab3f1ff700 1 -- 172.21.15.101:0/2468098571 shutdown_connections 2023-07-18T03:40:02.977 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.980+0000 7fab3f1ff700 1 -- 172.21.15.101:0/2468098571 wait complete. 2023-07-18T03:40:02.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.981+0000 7fab3f1ff700 1 Processor -- start 2023-07-18T03:40:02.979 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.981+0000 7fab3f1ff700 1 -- start start 2023-07-18T03:40:02.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.981+0000 7fab3f1ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab380ffae0 0x7fab38072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:02.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.981+0000 7fab3f1ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab38100490 0x7fab38072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:02.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.981+0000 7fab3f1ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fab38105af0 0x7fab381a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:02.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.982+0000 7fab3f1ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab38073580 con 0x7fab38105af0 2023-07-18T03:40:02.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.982+0000 7fab3f1ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab381a42e0 con 0x7fab380ffae0 2023-07-18T03:40:02.981 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.982+0000 7fab3f1ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab381a4420 con 0x7fab38100490 2023-07-18T03:40:02.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.982+0000 7fab3cf9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab380ffae0 0x7fab38072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:02.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.982+0000 7fab37fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab38100490 0x7fab38072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:02.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.982+0000 7fab3cf9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab380ffae0 0x7fab38072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:50760/0 (socket says 172.21.15.101:50760) 2023-07-18T03:40:02.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.982+0000 7fab3cf9b700 1 -- 172.21.15.101:0/1418023908 learned_addr learned my addr 172.21.15.101:0/1418023908 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:40:02.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.982+0000 7fab3d79c700 1 --2- 172.21.15.101:0/1418023908 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fab38105af0 0x7fab381a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:02.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.982+0000 7fab37fff700 1 -- 172.21.15.101:0/1418023908 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab380ffae0 msgr2=0x7fab38072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:02.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.982+0000 7fab37fff700 1 --2- 172.21.15.101:0/1418023908 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab380ffae0 0x7fab38072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:02.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.982+0000 7fab37fff700 1 -- 172.21.15.101:0/1418023908 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fab38105af0 msgr2=0x7fab381a3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:02.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.982+0000 7fab37fff700 1 --2- 172.21.15.101:0/1418023908 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fab38105af0 0x7fab381a3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:02.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.982+0000 7fab37fff700 1 -- 172.21.15.101:0/1418023908 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab2c009520 con 0x7fab38100490 2023-07-18T03:40:02.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.982+0000 7fab3cf9b700 1 --2- 172.21.15.101:0/1418023908 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab380ffae0 0x7fab38072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:02.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.982+0000 7fab3d79c700 1 --2- 172.21.15.101:0/1418023908 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fab38105af0 0x7fab381a3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:02.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.982+0000 7fab37fff700 1 --2- 172.21.15.101:0/1418023908 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab38100490 0x7fab38072d50 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fab24000c00 tx=0x7fab24005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:02.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.983+0000 7fab35ffb700 1 -- 172.21.15.101:0/1418023908 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2401d070 con 0x7fab38100490 2023-07-18T03:40:02.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.983+0000 7fab35ffb700 1 -- 172.21.15.101:0/1418023908 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab24021410 con 0x7fab38100490 2023-07-18T03:40:02.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.983+0000 7fab3f1ff700 1 -- 172.21.15.101:0/1418023908 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab240095f0 con 0x7fab38100490 2023-07-18T03:40:02.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.983+0000 7fab35ffb700 1 -- 172.21.15.101:0/1418023908 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab24017650 con 0x7fab38100490 2023-07-18T03:40:02.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.983+0000 7fab3f1ff700 1 -- 172.21.15.101:0/1418023908 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab381a4910 con 0x7fab38100490 2023-07-18T03:40:02.985 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.984+0000 7fab3f1ff700 1 -- 172.21.15.101:0/1418023908 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab38067290 con 0x7fab38100490 2023-07-18T03:40:02.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.986+0000 7fab35ffb700 1 -- 172.21.15.101:0/1418023908 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fab240217f0 con 0x7fab38100490 2023-07-18T03:40:02.986 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.987+0000 7fab35ffb700 1 --2- 172.21.15.101:0/1418023908 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fab280a1340 0x7fab280a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:02.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.987+0000 7fab35ffb700 1 -- 172.21.15.101:0/1418023908 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fab240ce870 con 0x7fab38100490 2023-07-18T03:40:02.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.987+0000 7fab3cf9b700 1 --2- 172.21.15.101:0/1418023908 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fab280a1340 0x7fab280a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:02.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.988+0000 7fab3cf9b700 1 --2- 172.21.15.101:0/1418023908 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fab280a1340 0x7fab280a37f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fab2c0116b0 tx=0x7fab2c009260 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:02.987 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:02.989+0000 7fab35ffb700 1 -- 172.21.15.101:0/1418023908 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fab24097900 con 0x7fab38100490 2023-07-18T03:40:03.181 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:03.186+0000 7fab3f1ff700 1 -- 172.21.15.101:0/1418023908 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fab381a4b00 con 0x7fab38100490 2023-07-18T03:40:03.182 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:03.187+0000 7fab35ffb700 1 -- 172.21.15.101:0/1418023908 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v144) v1 ==== 100+0+11 (secure 0 0 0) 0x7fab24021e60 con 0x7fab38100490 2023-07-18T03:40:03.182 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 64 2023-07-18T03:40:03.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:03.190+0000 7fab3f1ff700 1 -- 172.21.15.101:0/1418023908 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fab280a1340 msgr2=0x7fab280a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:03.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:03.190+0000 7fab3f1ff700 1 --2- 172.21.15.101:0/1418023908 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fab280a1340 0x7fab280a37f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fab2c0116b0 tx=0x7fab2c009260 comp rx=0 tx=0).stop 2023-07-18T03:40:03.186 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:03.190+0000 7fab3f1ff700 1 -- 172.21.15.101:0/1418023908 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab38100490 msgr2=0x7fab38072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:03.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:03.190+0000 7fab3f1ff700 1 --2- 172.21.15.101:0/1418023908 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab38100490 0x7fab38072d50 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fab24000c00 tx=0x7fab24005740 comp rx=0 tx=0).stop 2023-07-18T03:40:03.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:03.190+0000 7fab3f1ff700 1 -- 172.21.15.101:0/1418023908 shutdown_connections 2023-07-18T03:40:03.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:03.190+0000 7fab3f1ff700 1 --2- 172.21.15.101:0/1418023908 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fab280a1340 0x7fab280a37f0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:03.187 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:03.190+0000 7fab3f1ff700 1 --2- 172.21.15.101:0/1418023908 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab380ffae0 0x7fab38072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:03.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:03.190+0000 7fab3f1ff700 1 --2- 172.21.15.101:0/1418023908 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fab38105af0 0x7fab381a3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:03.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:03.190+0000 7fab3f1ff700 1 --2- 172.21.15.101:0/1418023908 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab38100490 0x7fab38072d50 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:03.188 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:03.190+0000 7fab3f1ff700 1 -- 172.21.15.101:0/1418023908 >> 172.21.15.101:0/1418023908 conn(0x7fab380fb5e0 msgr2=0x7fab381035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:03.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:03.190+0000 7fab3f1ff700 1 -- 172.21.15.101:0/1418023908 shutdown_connections 2023-07-18T03:40:03.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:03.190+0000 7fab3f1ff700 1 -- 172.21.15.101:0/1418023908 wait complete. 2023-07-18T03:40:03.204 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-18T03:40:03.825 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:03 smithi155 ceph-mon[128885]: pgmap v1029: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:40:03.825 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:03 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1621490042' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:40:03.825 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:03 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1418023908' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:40:03.894 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:03 smithi101 ceph-mon[126379]: pgmap v1029: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:40:03.895 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:03 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1621490042' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:40:03.895 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:03 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1418023908' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:40:03.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:03 smithi179 ceph-mon[129229]: pgmap v1029: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:40:03.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:03 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1621490042' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:40:03.953 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:03 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1418023908' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:40:04.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.044+0000 7f55c61ff700 1 -- 172.21.15.101:0/1694792208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55c0105ae0 msgr2=0x7f55c0105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:04.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.044+0000 7f55c61ff700 1 --2- 172.21.15.101:0/1694792208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55c0105ae0 0x7f55c0105eb0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f55b0009990 tx=0x7f55b0009ca0 comp rx=0 tx=0).stop 2023-07-18T03:40:04.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.044+0000 7f55c61ff700 1 -- 172.21.15.101:0/1694792208 shutdown_connections 2023-07-18T03:40:04.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.044+0000 7f55c61ff700 1 --2- 172.21.15.101:0/1694792208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55c0105ae0 0x7f55c0105eb0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:04.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.044+0000 7f55c61ff700 1 --2- 172.21.15.101:0/1694792208 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55c0100480 0x7f55c010c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:04.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.044+0000 7f55c61ff700 1 --2- 172.21.15.101:0/1694792208 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55c00ffad0 0x7f55c00fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:04.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.044+0000 7f55c61ff700 1 -- 172.21.15.101:0/1694792208 >> 172.21.15.101:0/1694792208 conn(0x7f55c00fb5f0 msgr2=0x7f55c00fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:04.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.044+0000 7f55c61ff700 1 -- 172.21.15.101:0/1694792208 shutdown_connections 2023-07-18T03:40:04.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.044+0000 7f55c61ff700 1 -- 172.21.15.101:0/1694792208 wait complete. 2023-07-18T03:40:04.056 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.045+0000 7f55c61ff700 1 Processor -- start 2023-07-18T03:40:04.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.045+0000 7f55c61ff700 1 -- start start 2023-07-18T03:40:04.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.046+0000 7f55c61ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55c00ffad0 0x7f55c019b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:04.057 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.046+0000 7f55c61ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55c0100480 0x7f55c019bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:04.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.046+0000 7f55c61ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55c0105ae0 0x7f55c0195a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:04.058 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.046+0000 7f55c61ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c01960a0 con 0x7f55c0100480 2023-07-18T03:40:04.059 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.046+0000 7f55c61ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c0196210 con 0x7f55c0105ae0 2023-07-18T03:40:04.059 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.046+0000 7f55c61ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c01963d0 con 0x7f55c00ffad0 2023-07-18T03:40:04.059 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.046+0000 7f55bf7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55c00ffad0 0x7f55c019b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:04.059 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.046+0000 7f55bf7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55c00ffad0 0x7f55c019b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:33976/0 (socket says 172.21.15.101:33976) 2023-07-18T03:40:04.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.046+0000 7f55bf7fe700 1 -- 172.21.15.101:0/97478452 learned_addr learned my addr 172.21.15.101:0/97478452 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:40:04.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.046+0000 7f55beffd700 1 --2- 172.21.15.101:0/97478452 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55c0100480 0x7f55c019bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:04.060 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.046+0000 7f55bffff700 1 --2- 172.21.15.101:0/97478452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55c0105ae0 0x7f55c0195a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:04.061 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.046+0000 7f55bf7fe700 1 -- 172.21.15.101:0/97478452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55c0105ae0 msgr2=0x7f55c0195a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:04.061 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.046+0000 7f55bf7fe700 1 --2- 172.21.15.101:0/97478452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55c0105ae0 0x7f55c0195a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:04.061 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.046+0000 7f55bf7fe700 1 -- 172.21.15.101:0/97478452 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55c0100480 msgr2=0x7f55c019bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:04.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.046+0000 7f55bf7fe700 1 --2- 172.21.15.101:0/97478452 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55c0100480 0x7f55c019bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:04.062 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.046+0000 7f55bf7fe700 1 -- 172.21.15.101:0/97478452 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b00095f0 con 0x7f55c00ffad0 2023-07-18T03:40:04.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.046+0000 7f55bf7fe700 1 --2- 172.21.15.101:0/97478452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55c00ffad0 0x7f55c019b9c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f55b0005910 tx=0x7f55b0004710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:04.063 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.047+0000 7f55bcff9700 1 -- 172.21.15.101:0/97478452 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b001d070 con 0x7f55c00ffad0 2023-07-18T03:40:04.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.047+0000 7f55bcff9700 1 -- 172.21.15.101:0/97478452 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55b0022470 con 0x7f55c00ffad0 2023-07-18T03:40:04.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.047+0000 7f55c61ff700 1 -- 172.21.15.101:0/97478452 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55c0196650 con 0x7f55c00ffad0 2023-07-18T03:40:04.064 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.047+0000 7f55bcff9700 1 -- 172.21.15.101:0/97478452 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b0017650 con 0x7f55c00ffad0 2023-07-18T03:40:04.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.047+0000 7f55c61ff700 1 -- 172.21.15.101:0/97478452 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55c01a29b0 con 0x7f55c00ffad0 2023-07-18T03:40:04.065 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.048+0000 7f55c61ff700 1 -- 172.21.15.101:0/97478452 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55c0067290 con 0x7f55c00ffad0 2023-07-18T03:40:04.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.050+0000 7f55bcff9700 1 -- 172.21.15.101:0/97478452 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f55b00225e0 con 0x7f55c00ffad0 2023-07-18T03:40:04.066 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.051+0000 7f55bcff9700 1 --2- 172.21.15.101:0/97478452 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f55a80a12c0 0x7f55a80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:04.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.051+0000 7f55bcff9700 1 -- 172.21.15.101:0/97478452 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f55b00ce260 con 0x7f55c00ffad0 2023-07-18T03:40:04.067 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.051+0000 7f55beffd700 1 --2- 172.21.15.101:0/97478452 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f55a80a12c0 0x7f55a80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:04.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.052+0000 7f55beffd700 1 --2- 172.21.15.101:0/97478452 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f55a80a12c0 0x7f55a80a3770 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f55c01051b0 tx=0x7f55b4009260 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:04.068 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.053+0000 7f55bcff9700 1 -- 172.21.15.101:0/97478452 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f55b0097350 con 0x7f55c00ffad0 2023-07-18T03:40:04.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.252+0000 7f55c61ff700 1 -- 172.21.15.101:0/97478452 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f55c01a2b40 con 0x7f55c00ffad0 2023-07-18T03:40:04.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.253+0000 7f55bcff9700 1 -- 172.21.15.101:0/97478452 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v144) v1 ==== 103+0+11 (secure 0 0 0) 0x7f55b009a970 con 0x7f55c00ffad0 2023-07-18T03:40:04.250 INFO:teuthology.orchestra.run.smithi101.stdout:pg_num: 32 2023-07-18T03:40:04.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.255+0000 7f55c61ff700 1 -- 172.21.15.101:0/97478452 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f55a80a12c0 msgr2=0x7f55a80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:04.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.255+0000 7f55c61ff700 1 --2- 172.21.15.101:0/97478452 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f55a80a12c0 0x7f55a80a3770 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f55c01051b0 tx=0x7f55b4009260 comp rx=0 tx=0).stop 2023-07-18T03:40:04.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.255+0000 7f55c61ff700 1 -- 172.21.15.101:0/97478452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55c00ffad0 msgr2=0x7f55c019b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:04.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.255+0000 7f55c61ff700 1 --2- 172.21.15.101:0/97478452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55c00ffad0 0x7f55c019b9c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f55b0005910 tx=0x7f55b0004710 comp rx=0 tx=0).stop 2023-07-18T03:40:04.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.255+0000 7f55c61ff700 1 -- 172.21.15.101:0/97478452 shutdown_connections 2023-07-18T03:40:04.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.255+0000 7f55c61ff700 1 --2- 172.21.15.101:0/97478452 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f55a80a12c0 0x7f55a80a3770 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:04.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.255+0000 7f55c61ff700 1 --2- 172.21.15.101:0/97478452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55c0105ae0 0x7f55c0195a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:04.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.255+0000 7f55c61ff700 1 --2- 172.21.15.101:0/97478452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55c00ffad0 0x7f55c019b9c0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:04.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.255+0000 7f55c61ff700 1 --2- 172.21.15.101:0/97478452 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55c0100480 0x7f55c019bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:04.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.255+0000 7f55c61ff700 1 -- 172.21.15.101:0/97478452 >> 172.21.15.101:0/97478452 conn(0x7f55c00fb5f0 msgr2=0x7f55c00fbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:04.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.255+0000 7f55c61ff700 1 -- 172.21.15.101:0/97478452 shutdown_connections 2023-07-18T03:40:04.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.255+0000 7f55c61ff700 1 -- 172.21.15.101:0/97478452 wait complete. 2023-07-18T03:40:04.272 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-18T03:40:04.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.477+0000 7fb5661ff700 1 -- 172.21.15.101:0/1970873476 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb560105af0 msgr2=0x7fb560105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:04.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.477+0000 7fb5661ff700 1 --2- 172.21.15.101:0/1970873476 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb560105af0 0x7fb560105ec0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fb5500098f0 tx=0x7fb550009c00 comp rx=0 tx=0).stop 2023-07-18T03:40:04.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.477+0000 7fb5661ff700 1 -- 172.21.15.101:0/1970873476 shutdown_connections 2023-07-18T03:40:04.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.477+0000 7fb5661ff700 1 --2- 172.21.15.101:0/1970873476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb560100490 0x7fb56010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:04.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.477+0000 7fb5661ff700 1 --2- 172.21.15.101:0/1970873476 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb560105af0 0x7fb560105ec0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:04.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.477+0000 7fb5661ff700 1 --2- 172.21.15.101:0/1970873476 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5600ffae0 0x7fb5600fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:04.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.477+0000 7fb5661ff700 1 -- 172.21.15.101:0/1970873476 >> 172.21.15.101:0/1970873476 conn(0x7fb5600fb5e0 msgr2=0x7fb5600fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:04.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.477+0000 7fb5661ff700 1 -- 172.21.15.101:0/1970873476 shutdown_connections 2023-07-18T03:40:04.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.477+0000 7fb5661ff700 1 -- 172.21.15.101:0/1970873476 wait complete. 2023-07-18T03:40:04.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.478+0000 7fb5661ff700 1 Processor -- start 2023-07-18T03:40:04.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.478+0000 7fb5661ff700 1 -- start start 2023-07-18T03:40:04.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.479+0000 7fb5661ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb5600ffae0 0x7fb560072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:04.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.479+0000 7fb5661ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb560100490 0x7fb560072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:04.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.479+0000 7fb5661ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb560105af0 0x7fb5601a3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:04.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.479+0000 7fb5661ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb560073580 con 0x7fb5600ffae0 2023-07-18T03:40:04.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.479+0000 7fb5661ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5601a4320 con 0x7fb560100490 2023-07-18T03:40:04.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.479+0000 7fb5661ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5601a4460 con 0x7fb560105af0 2023-07-18T03:40:04.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.479+0000 7fb55effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb560100490 0x7fb560072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:04.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.479+0000 7fb55ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb560105af0 0x7fb5601a3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:04.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.479+0000 7fb55effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb560100490 0x7fb560072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.101:50808/0 (socket says 172.21.15.101:50808) 2023-07-18T03:40:04.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.479+0000 7fb55effd700 1 -- 172.21.15.101:0/3393155139 learned_addr learned my addr 172.21.15.101:0/3393155139 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:40:04.484 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.479+0000 7fb55f7fe700 1 --2- 172.21.15.101:0/3393155139 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb5600ffae0 0x7fb560072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:04.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.479+0000 7fb55effd700 1 -- 172.21.15.101:0/3393155139 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb560105af0 msgr2=0x7fb5601a3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:04.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.479+0000 7fb55effd700 1 --2- 172.21.15.101:0/3393155139 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb560105af0 0x7fb5601a3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:04.485 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.479+0000 7fb55effd700 1 -- 172.21.15.101:0/3393155139 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb5600ffae0 msgr2=0x7fb560072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:04.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.479+0000 7fb55effd700 1 --2- 172.21.15.101:0/3393155139 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb5600ffae0 0x7fb560072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:04.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.479+0000 7fb55effd700 1 -- 172.21.15.101:0/3393155139 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb550009520 con 0x7fb560100490 2023-07-18T03:40:04.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.479+0000 7fb55ffff700 1 --2- 172.21.15.101:0/3393155139 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb560105af0 0x7fb5601a3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:04.486 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.480+0000 7fb55effd700 1 --2- 172.21.15.101:0/3393155139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb560100490 0x7fb560072d50 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fb55400e9c0 tx=0x7fb55400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:04.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.480+0000 7fb55cff9700 1 -- 172.21.15.101:0/3393155139 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb554004a30 con 0x7fb560100490 2023-07-18T03:40:04.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.480+0000 7fb55cff9700 1 -- 172.21.15.101:0/3393155139 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb55400cc80 con 0x7fb560100490 2023-07-18T03:40:04.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.480+0000 7fb5661ff700 1 -- 172.21.15.101:0/3393155139 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5601a46c0 con 0x7fb560100490 2023-07-18T03:40:04.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.480+0000 7fb55cff9700 1 -- 172.21.15.101:0/3393155139 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5540054e0 con 0x7fb560100490 2023-07-18T03:40:04.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.480+0000 7fb5661ff700 1 -- 172.21.15.101:0/3393155139 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5601a4bf0 con 0x7fb560100490 2023-07-18T03:40:04.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.481+0000 7fb5661ff700 1 -- 172.21.15.101:0/3393155139 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb560067290 con 0x7fb560100490 2023-07-18T03:40:04.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.483+0000 7fb55cff9700 1 -- 172.21.15.101:0/3393155139 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fb554004b90 con 0x7fb560100490 2023-07-18T03:40:04.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.484+0000 7fb55cff9700 1 --2- 172.21.15.101:0/3393155139 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fb5480a1340 0x7fb5480a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:04.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.484+0000 7fb55cff9700 1 -- 172.21.15.101:0/3393155139 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fb554014070 con 0x7fb560100490 2023-07-18T03:40:04.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.484+0000 7fb55f7fe700 1 --2- 172.21.15.101:0/3393155139 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fb5480a1340 0x7fb5480a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:04.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.485+0000 7fb55f7fe700 1 --2- 172.21.15.101:0/3393155139 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fb5480a1340 0x7fb5480a37f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb550005e90 tx=0x7fb550019040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:04.490 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.488+0000 7fb55cff9700 1 -- 172.21.15.101:0/3393155139 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb554096150 con 0x7fb560100490 2023-07-18T03:40:04.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.704+0000 7fb5661ff700 1 -- 172.21.15.101:0/3393155139 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb5601a4f70 con 0x7fb560100490 2023-07-18T03:40:04.700 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.704+0000 7fb55cff9700 1 -- 172.21.15.101:0/3393155139 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v144) v1 ==== 81+0+959 (secure 0 0 0) 0x7fb554099770 con 0x7fb560100490 2023-07-18T03:40:04.701 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.15.101:0/2766365637 2023-07-19T03:07:33.498516+0000 2023-07-18T03:40:04.701 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.15.101:0/523372970 2023-07-19T03:07:33.498516+0000 2023-07-18T03:40:04.701 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.15.101:6800/3963742842 2023-07-19T03:07:33.498516+0000 2023-07-18T03:40:04.701 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.15.101:6800/2461037266 2023-07-19T02:58:22.762480+0000 2023-07-18T03:40:04.702 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.15.101:0/1806538035 2023-07-19T03:07:33.498516+0000 2023-07-18T03:40:04.702 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.15.101:6801/2461037266 2023-07-19T02:58:22.762480+0000 2023-07-18T03:40:04.702 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.15.101:0/604228291 2023-07-19T02:58:22.762480+0000 2023-07-18T03:40:04.703 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.15.101:0/1004430030 2023-07-19T03:07:33.498516+0000 2023-07-18T03:40:04.703 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.15.101:6801/3963742842 2023-07-19T03:07:33.498516+0000 2023-07-18T03:40:04.703 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.15.101:0/4055187849 2023-07-19T02:58:22.762480+0000 2023-07-18T03:40:04.704 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.15.101:0/2844574324 2023-07-19T02:57:56.243200+0000 2023-07-18T03:40:04.704 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.15.101:0/1603248215 2023-07-19T02:57:56.243200+0000 2023-07-18T03:40:04.704 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.15.101:6801/3497914514 2023-07-19T02:57:56.243200+0000 2023-07-18T03:40:04.704 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.15.101:6800/3497914514 2023-07-19T02:57:56.243200+0000 2023-07-18T03:40:04.705 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.15.101:0/3307578182 2023-07-19T02:58:22.762480+0000 2023-07-18T03:40:04.708 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.15.101:0/373189792 2023-07-19T02:57:56.243200+0000 2023-07-18T03:40:04.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.707+0000 7fb5661ff700 1 -- 172.21.15.101:0/3393155139 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fb5480a1340 msgr2=0x7fb5480a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:04.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.707+0000 7fb5661ff700 1 --2- 172.21.15.101:0/3393155139 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fb5480a1340 0x7fb5480a37f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb550005e90 tx=0x7fb550019040 comp rx=0 tx=0).stop 2023-07-18T03:40:04.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.707+0000 7fb5661ff700 1 -- 172.21.15.101:0/3393155139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb560100490 msgr2=0x7fb560072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:04.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.707+0000 7fb5661ff700 1 --2- 172.21.15.101:0/3393155139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb560100490 0x7fb560072d50 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fb55400e9c0 tx=0x7fb55400ecd0 comp rx=0 tx=0).stop 2023-07-18T03:40:04.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.707+0000 7fb5661ff700 1 -- 172.21.15.101:0/3393155139 shutdown_connections 2023-07-18T03:40:04.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.707+0000 7fb5661ff700 1 --2- 172.21.15.101:0/3393155139 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb560105af0 0x7fb5601a3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:04.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.707+0000 7fb5661ff700 1 --2- 172.21.15.101:0/3393155139 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb5600ffae0 0x7fb560072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:04.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.707+0000 7fb5661ff700 1 --2- 172.21.15.101:0/3393155139 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fb5480a1340 0x7fb5480a37f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:04.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.707+0000 7fb5661ff700 1 --2- 172.21.15.101:0/3393155139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb560100490 0x7fb560072d50 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:04.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.707+0000 7fb5661ff700 1 -- 172.21.15.101:0/3393155139 >> 172.21.15.101:0/3393155139 conn(0x7fb5600fb5e0 msgr2=0x7fb5600fbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:04.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.707+0000 7fb5661ff700 1 -- 172.21.15.101:0/3393155139 shutdown_connections 2023-07-18T03:40:04.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:04.707+0000 7fb5661ff700 1 -- 172.21.15.101:0/3393155139 wait complete. 2023-07-18T03:40:04.712 INFO:teuthology.orchestra.run.smithi101.stderr:listed 16 entries 2023-07-18T03:40:04.718 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:04 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/97478452' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-18T03:40:04.721 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-07-18T03:40:04.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:04.722 DEBUG:teuthology.orchestra.run.smithi101:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-07-18T03:40:04.796 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:04 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/97478452' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-18T03:40:04.935 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi101.front.sepia.ceph.com 2023-07-18T03:40:04.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:04.937 DEBUG:teuthology.orchestra.run.smithi101:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-18T03:40:04.959 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:04 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/97478452' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-18T03:40:05.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:05.059 DEBUG:teuthology.orchestra.run.smithi101:> ip netns list 2023-07-18T03:40:05.103 INFO:teuthology.orchestra.run.smithi101.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-18T03:40:05.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:05.105 DEBUG:teuthology.orchestra.run.smithi101:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-18T03:40:05.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:05.216 DEBUG:teuthology.orchestra.run.smithi101:> sudo ip link delete ceph-brx 2023-07-18T03:40:05.322 INFO:tasks.kclient:Unmounting kernel clients... 2023-07-18T03:40:05.323 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-07-18T03:40:05.323 DEBUG:teuthology.orchestra.run.smithi101:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:40:05.369 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345548k,nr_inodes=4086387,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3274996k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/55964af0baacd1ca4bfded6a334b30958f464a89dec9a967cd612dbd48024176/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/JTZH6T6FIH5PRT6OT24XQ2GGIG:/var/lib/containers/storage/overlay/l/X5KDLMGGSNOHDKVLRE3FVHBHSP,upperdir=/var/lib/containers/storage/overlay/55964af0baacd1ca4bfded6a334b30958f464a89dec9a967cd612dbd48024176/diff,workdir=/var/lib/containers/storage/overlay/55964af0baacd1ca4bfded6a334b30958f464a89dec9a967cd612dbd48024176/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/fe464aa7bbec76c12de754763ae800e2ac86a5389d4bd4e80bdc812f66711a1a/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/JTZH6T6FIH5PRT6OT24XQ2GGIG:/var/lib/containers/storage/overlay/l/X5KDLMGGSNOHDKVLRE3FVHBHSP,upperdir=/var/lib/containers/storage/overlay/fe464aa7bbec76c12de754763ae800e2ac86a5389d4bd4e80bdc812f66711a1a/diff,workdir=/var/lib/containers/storage/overlay/fe464aa7bbec76c12de754763ae800e2ac86a5389d4bd4e80bdc812f66711a1a/work,redirect_dir=on,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3274996k,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,relatime 0 0 overlay /var/lib/containers/storage/overlay/2d026e9a23ead9b6cd483262f9282947feb82d672d3c728ce5a3177300cc3c46/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/JTZH6T6FIH5PRT6OT24XQ2GGIG:/var/lib/containers/storage/overlay/l/X5KDLMGGSNOHDKVLRE3FVHBHSP,upperdir=/var/lib/containers/storage/overlay/2d026e9a23ead9b6cd483262f9282947feb82d672d3c728ce5a3177300cc3c46/diff,workdir=/var/lib/containers/storage/overlay/2d026e9a23ead9b6cd483262f9282947feb82d672d3c728ce5a3177300cc3c46/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/088e4159f81f225d599c69b96c39d1671e3580671c102dd22dc0bea3946ebb35/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/JTZH6T6FIH5PRT6OT24XQ2GGIG:/var/lib/containers/storage/overlay/l/X5KDLMGGSNOHDKVLRE3FVHBHSP,upperdir=/var/lib/containers/storage/overlay/088e4159f81f225d599c69b96c39d1671e3580671c102dd22dc0bea3946ebb35/diff,workdir=/var/lib/containers/storage/overlay/088e4159f81f225d599c69b96c39d1671e3580671c102dd22dc0bea3946ebb35/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/a7deda21e36e6f5c950778b7bb3cb0998286042566a586e21e0d248ab84d1b8b/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/JTZH6T6FIH5PRT6OT24XQ2GGIG:/var/lib/containers/storage/overlay/l/X5KDLMGGSNOHDKVLRE3FVHBHSP,upperdir=/var/lib/containers/storage/overlay/a7deda21e36e6f5c950778b7bb3cb0998286042566a586e21e0d248ab84d1b8b/diff,workdir=/var/lib/containers/storage/overlay/a7deda21e36e6f5c950778b7bb3cb0998286042566a586e21e0d248ab84d1b8b/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/a0d81a064832522f645c235dfc4928714f33ad809e37b46e70204bddbbdea99f/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/JTZH6T6FIH5PRT6OT24XQ2GGIG:/var/lib/containers/storage/overlay/l/X5KDLMGGSNOHDKVLRE3FVHBHSP,upperdir=/var/lib/containers/storage/overlay/a0d81a064832522f645c235dfc4928714f33ad809e37b46e70204bddbbdea99f/diff,workdir=/var/lib/containers/storage/overlay/a0d81a064832522f645c235dfc4928714f33ad809e37b46e70204bddbbdea99f/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/699a72d612be15366086fea63ddbd8aa9f85b5e4fb26a2fe9e75653a10ce7c39/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/JTZH6T6FIH5PRT6OT24XQ2GGIG:/var/lib/containers/storage/overlay/l/X5KDLMGGSNOHDKVLRE3FVHBHSP,upperdir=/var/lib/containers/storage/overlay/699a72d612be15366086fea63ddbd8aa9f85b5e4fb26a2fe9e75653a10ce7c39/diff,workdir=/var/lib/containers/storage/overlay/699a72d612be15366086fea63ddbd8aa9f85b5e4fb26a2fe9e75653a10ce7c39/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/4f45414c80af7ae6e9d3bbc0dfbe437ed7f82b49e301705d5aaaee2c271da365/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/JTZH6T6FIH5PRT6OT24XQ2GGIG:/var/lib/containers/storage/overlay/l/X5KDLMGGSNOHDKVLRE3FVHBHSP,upperdir=/var/lib/containers/storage/overlay/4f45414c80af7ae6e9d3bbc0dfbe437ed7f82b49e301705d5aaaee2c271da365/diff,workdir=/var/lib/containers/storage/overlay/4f45414c80af7ae6e9d3bbc0dfbe437ed7f82b49e301705d5aaaee2c271da365/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/82f4e5557914a25408c3ebac836370710283ecc2f861392f2e240ad8e0ed9aa2/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/JTZH6T6FIH5PRT6OT24XQ2GGIG:/var/lib/containers/storage/overlay/l/X5KDLMGGSNOHDKVLRE3FVHBHSP,upperdir=/var/lib/containers/storage/overlay/82f4e5557914a25408c3ebac836370710283ecc2f861392f2e240ad8e0ed9aa2/diff,workdir=/var/lib/containers/storage/overlay/82f4e5557914a25408c3ebac836370710283ecc2f861392f2e240ad8e0ed9aa2/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/988405ca88b92bd746cb9a0f0105c9414e4b83818ea749e0f4e6c2e04b2b4b65/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/JTZH6T6FIH5PRT6OT24XQ2GGIG:/var/lib/containers/storage/overlay/l/X5KDLMGGSNOHDKVLRE3FVHBHSP,upperdir=/var/lib/containers/storage/overlay/988405ca88b92bd746cb9a0f0105c9414e4b83818ea749e0f4e6c2e04b2b4b65/diff,workdir=/var/lib/containers/storage/overlay/988405ca88b92bd746cb9a0f0105c9414e4b83818ea749e0f4e6c2e04b2b4b65/work,redirect_dir=on,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 2023-07-18T03:40:05.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:05.370 DEBUG:teuthology.orchestra.run.smithi101:> ip netns list 2023-07-18T03:40:05.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:40:05.459 DEBUG:teuthology.orchestra.run.smithi101:> sudo ip link delete ceph-brx 2023-07-18T03:40:05.583 INFO:teuthology.orchestra.run.smithi101.stderr:Cannot find device "ceph-brx" 2023-07-18T03:40:05.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-18T03:40:05.587 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-07-18T03:40:05.869 INFO:tasks.cephadm:Teardown begin 2023-07-18T03:40:05.870 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-18T03:40:05.871 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:40:05.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:05 smithi155 ceph-mon[128885]: pgmap v1030: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:40:05.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:05 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/3393155139' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-18T03:40:05.934 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:05 smithi101 ceph-mon[126379]: pgmap v1030: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:40:05.935 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:05 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3393155139' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-18T03:40:05.951 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:05 smithi179 ceph-mon[129229]: pgmap v1030: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-18T03:40:05.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:05 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/3393155139' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-18T03:40:06.073 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.077+0000 7f924e5ff700 1 -- 172.21.15.101:0/1247695869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92480728b0 msgr2=0x7f9248072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:06.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.077+0000 7f924e5ff700 1 --2- 172.21.15.101:0/1247695869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92480728b0 0x7f9248072c80 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f9238009990 tx=0x7f9238009ca0 comp rx=0 tx=0).stop 2023-07-18T03:40:06.074 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.077+0000 7f924e5ff700 1 -- 172.21.15.101:0/1247695869 shutdown_connections 2023-07-18T03:40:06.075 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.077+0000 7f924e5ff700 1 --2- 172.21.15.101:0/1247695869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92480728b0 0x7f9248072c80 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:06.075 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.077+0000 7f924e5ff700 1 --2- 172.21.15.101:0/1247695869 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f924810c310 0x7f924810e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:06.076 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.077+0000 7f924e5ff700 1 --2- 172.21.15.101:0/1247695869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f92480731c0 0x7f924810bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:06.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.077+0000 7f924e5ff700 1 -- 172.21.15.101:0/1247695869 >> 172.21.15.101:0/1247695869 conn(0x7f92480fb590 msgr2=0x7f92480fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:06.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.077+0000 7f924e5ff700 1 -- 172.21.15.101:0/1247695869 shutdown_connections 2023-07-18T03:40:06.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.077+0000 7f924e5ff700 1 -- 172.21.15.101:0/1247695869 wait complete. 2023-07-18T03:40:06.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.078+0000 7f924e5ff700 1 Processor -- start 2023-07-18T03:40:06.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.079+0000 7f924e5ff700 1 -- start start 2023-07-18T03:40:06.082 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.079+0000 7f924e5ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f92480728b0 0x7f9248197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:06.083 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.079+0000 7f924e5ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92480731c0 0x7f9248198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:06.083 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.079+0000 7f924e5ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f924810c310 0x7f924819c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:06.084 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.079+0000 7f924e5ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f924819cde0 con 0x7f92480728b0 2023-07-18T03:40:06.084 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.079+0000 7f924e5ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f924819cf50 con 0x7f92480731c0 2023-07-18T03:40:06.085 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.079+0000 7f924e5ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f924819d110 con 0x7f924810c310 2023-07-18T03:40:06.085 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.079+0000 7f9247fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f92480728b0 0x7f9248197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:06.085 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.079+0000 7f92477fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92480731c0 0x7f9248198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:06.086 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.079+0000 7f9247fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f92480728b0 0x7f9248197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:59136/0 (socket says 172.21.15.101:59136) 2023-07-18T03:40:06.086 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.079+0000 7f9247fff700 1 -- 172.21.15.101:0/2832810889 learned_addr learned my addr 172.21.15.101:0/2832810889 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:40:06.087 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.080+0000 7f924cb9c700 1 --2- 172.21.15.101:0/2832810889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f924810c310 0x7f924819c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:06.087 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.080+0000 7f9247fff700 1 -- 172.21.15.101:0/2832810889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f924810c310 msgr2=0x7f924819c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:06.088 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.080+0000 7f9247fff700 1 --2- 172.21.15.101:0/2832810889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f924810c310 0x7f924819c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:06.088 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.080+0000 7f9247fff700 1 -- 172.21.15.101:0/2832810889 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92480731c0 msgr2=0x7f9248198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:06.088 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.080+0000 7f9247fff700 1 --2- 172.21.15.101:0/2832810889 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92480731c0 0x7f9248198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:06.089 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.080+0000 7f9247fff700 1 -- 172.21.15.101:0/2832810889 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9240009c40 con 0x7f92480728b0 2023-07-18T03:40:06.089 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.080+0000 7f9247fff700 1 --2- 172.21.15.101:0/2832810889 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f92480728b0 0x7f9248197e00 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f9238005910 tx=0x7f9238004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:06.089 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.080+0000 7f92457fa700 1 -- 172.21.15.101:0/2832810889 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923801d070 con 0x7f92480728b0 2023-07-18T03:40:06.089 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.080+0000 7f924e5ff700 1 -- 172.21.15.101:0/2832810889 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92380095f0 con 0x7f92480728b0 2023-07-18T03:40:06.090 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.080+0000 7f92457fa700 1 -- 172.21.15.101:0/2832810889 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9238022470 con 0x7f92480728b0 2023-07-18T03:40:06.090 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.080+0000 7f92457fa700 1 -- 172.21.15.101:0/2832810889 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9238017650 con 0x7f92480728b0 2023-07-18T03:40:06.090 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.080+0000 7f924e5ff700 1 -- 172.21.15.101:0/2832810889 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92481a4a50 con 0x7f92480728b0 2023-07-18T03:40:06.090 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.081+0000 7f924e5ff700 1 -- 172.21.15.101:0/2832810889 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9248067300 con 0x7f92480728b0 2023-07-18T03:40:06.090 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.083+0000 7f92457fa700 1 -- 172.21.15.101:0/2832810889 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f92380225e0 con 0x7f92480728b0 2023-07-18T03:40:06.091 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.085+0000 7f92457fa700 1 --2- 172.21.15.101:0/2832810889 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f92300a1300 0x7f92300a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:06.091 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.085+0000 7f92457fa700 1 -- 172.21.15.101:0/2832810889 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f92380ce810 con 0x7f92480728b0 2023-07-18T03:40:06.091 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.085+0000 7f92477fe700 1 --2- 172.21.15.101:0/2832810889 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f92300a1300 0x7f92300a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:06.091 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.085+0000 7f92477fe700 1 --2- 172.21.15.101:0/2832810889 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f92300a1300 0x7f92300a37b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9248199410 tx=0x7f923c009260 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:06.091 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.087+0000 7f92457fa700 1 -- 172.21.15.101:0/2832810889 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9238097850 con 0x7f92480728b0 2023-07-18T03:40:06.324 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.330+0000 7f924e5ff700 1 -- 172.21.15.101:0/2832810889 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f924819d2d0 con 0x7f92480728b0 2023-07-18T03:40:06.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.331+0000 7f92457fa700 1 -- 172.21.15.101:0/2832810889 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+10858 (secure 0 0 0) 0x7f923809ae70 con 0x7f92480728b0 2023-07-18T03:40:06.326 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:40:06.326 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/4192846769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":4192846769},{"type":"v1","addr":"172.21.15.101:6839","nonce":4192846769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6841/3824340175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6840","nonce":3824340175},{"type":"v1","addr":"172.21.15.101:6841","nonce":3824340175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":49,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:04:59.361185+0000","modified":"2023-07-18T03:28:46.287122+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.155:6839/2055462483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2055462483},{"type":"v1","addr":"172.21.15.155:6839","nonce":2055462483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:40:06.328 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.333+0000 7f924e5ff700 1 -- 172.21.15.101:0/2832810889 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f92300a1300 msgr2=0x7f92300a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:06.328 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.333+0000 7f924e5ff700 1 --2- 172.21.15.101:0/2832810889 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f92300a1300 0x7f92300a37b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9248199410 tx=0x7f923c009260 comp rx=0 tx=0).stop 2023-07-18T03:40:06.328 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.333+0000 7f924e5ff700 1 -- 172.21.15.101:0/2832810889 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f92480728b0 msgr2=0x7f9248197e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:06.328 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.333+0000 7f924e5ff700 1 --2- 172.21.15.101:0/2832810889 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f92480728b0 0x7f9248197e00 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f9238005910 tx=0x7f9238004710 comp rx=0 tx=0).stop 2023-07-18T03:40:06.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.333+0000 7f924e5ff700 1 -- 172.21.15.101:0/2832810889 shutdown_connections 2023-07-18T03:40:06.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.333+0000 7f924e5ff700 1 --2- 172.21.15.101:0/2832810889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f924810c310 0x7f924819c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:06.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.333+0000 7f924e5ff700 1 --2- 172.21.15.101:0/2832810889 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f92480728b0 0x7f9248197e00 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:06.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.333+0000 7f924e5ff700 1 --2- 172.21.15.101:0/2832810889 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f92300a1300 0x7f92300a37b0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:06.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.333+0000 7f924e5ff700 1 --2- 172.21.15.101:0/2832810889 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92480731c0 0x7f9248198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:06.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.333+0000 7f924e5ff700 1 -- 172.21.15.101:0/2832810889 >> 172.21.15.101:0/2832810889 conn(0x7f92480fb590 msgr2=0x7f924810e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:06.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.333+0000 7f924e5ff700 1 -- 172.21.15.101:0/2832810889 shutdown_connections 2023-07-18T03:40:06.330 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.333+0000 7f924e5ff700 1 -- 172.21.15.101:0/2832810889 wait complete. 2023-07-18T03:40:06.330 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 49 2023-07-18T03:40:06.347 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:40:06.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.556+0000 7f6c8cbff700 1 -- 172.21.15.101:0/2313900418 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c880728b0 msgr2=0x7f6c88072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:06.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.556+0000 7f6c8cbff700 1 --2- 172.21.15.101:0/2313900418 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c880728b0 0x7f6c88072c80 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f6c78009990 tx=0x7f6c78009ca0 comp rx=0 tx=0).stop 2023-07-18T03:40:06.553 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.557+0000 7f6c8cbff700 1 -- 172.21.15.101:0/2313900418 shutdown_connections 2023-07-18T03:40:06.553 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.557+0000 7f6c8cbff700 1 --2- 172.21.15.101:0/2313900418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c8810c330 0x7f6c8810e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:06.553 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.557+0000 7f6c8cbff700 1 --2- 172.21.15.101:0/2313900418 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c880728b0 0x7f6c88072c80 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:06.553 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.557+0000 7f6c8cbff700 1 --2- 172.21.15.101:0/2313900418 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c880731c0 0x7f6c8810bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:06.553 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.557+0000 7f6c8cbff700 1 -- 172.21.15.101:0/2313900418 >> 172.21.15.101:0/2313900418 conn(0x7f6c880fb630 msgr2=0x7f6c880fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:06.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.557+0000 7f6c8cbff700 1 -- 172.21.15.101:0/2313900418 shutdown_connections 2023-07-18T03:40:06.554 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.557+0000 7f6c8cbff700 1 -- 172.21.15.101:0/2313900418 wait complete. 2023-07-18T03:40:06.556 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.558+0000 7f6c8cbff700 1 Processor -- start 2023-07-18T03:40:06.556 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.558+0000 7f6c8cbff700 1 -- start start 2023-07-18T03:40:06.556 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.558+0000 7f6c8cbff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c880728b0 0x7f6c88197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:06.556 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.558+0000 7f6c8cbff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c880731c0 0x7f6c88198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:06.557 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.558+0000 7f6c8cbff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c8810c330 0x7f6c8819c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:06.557 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.558+0000 7f6c8cbff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8819cde0 con 0x7f6c8810c330 2023-07-18T03:40:06.557 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.558+0000 7f6c8cbff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8819cf50 con 0x7f6c880731c0 2023-07-18T03:40:06.557 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.558+0000 7f6c8cbff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8819d110 con 0x7f6c880728b0 2023-07-18T03:40:06.557 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.558+0000 7f6c8659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c880728b0 0x7f6c88197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:06.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.558+0000 7f6c8659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c880728b0 0x7f6c88197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:46154/0 (socket says 172.21.15.101:46154) 2023-07-18T03:40:06.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.558+0000 7f6c8659c700 1 -- 172.21.15.101:0/2918218583 learned_addr learned my addr 172.21.15.101:0/2918218583 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:40:06.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.558+0000 7f6c7ffff700 1 --2- 172.21.15.101:0/2918218583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c880731c0 0x7f6c88198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:06.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.558+0000 7f6c86d9d700 1 --2- 172.21.15.101:0/2918218583 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c8810c330 0x7f6c8819c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:06.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.559+0000 7f6c8659c700 1 -- 172.21.15.101:0/2918218583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c880731c0 msgr2=0x7f6c88198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:06.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.559+0000 7f6c8659c700 1 --2- 172.21.15.101:0/2918218583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c880731c0 0x7f6c88198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:06.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.559+0000 7f6c8659c700 1 -- 172.21.15.101:0/2918218583 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c8810c330 msgr2=0x7f6c8819c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:06.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.559+0000 7f6c8659c700 1 --2- 172.21.15.101:0/2918218583 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c8810c330 0x7f6c8819c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:06.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.559+0000 7f6c8659c700 1 -- 172.21.15.101:0/2918218583 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c70009520 con 0x7f6c880728b0 2023-07-18T03:40:06.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.559+0000 7f6c7ffff700 1 --2- 172.21.15.101:0/2918218583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c880731c0 0x7f6c88198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:06.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.559+0000 7f6c86d9d700 1 --2- 172.21.15.101:0/2918218583 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c8810c330 0x7f6c8819c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:06.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.559+0000 7f6c8659c700 1 --2- 172.21.15.101:0/2918218583 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c880728b0 0x7f6c88197e00 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f6c78005910 tx=0x7f6c78004710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:06.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.559+0000 7f6c7f7fe700 1 -- 172.21.15.101:0/2918218583 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7801d070 con 0x7f6c880728b0 2023-07-18T03:40:06.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.559+0000 7f6c8cbff700 1 -- 172.21.15.101:0/2918218583 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c780095f0 con 0x7f6c880728b0 2023-07-18T03:40:06.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.559+0000 7f6c7f7fe700 1 -- 172.21.15.101:0/2918218583 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c78022470 con 0x7f6c880728b0 2023-07-18T03:40:06.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.559+0000 7f6c7f7fe700 1 -- 172.21.15.101:0/2918218583 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c78017650 con 0x7f6c880728b0 2023-07-18T03:40:06.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.560+0000 7f6c8cbff700 1 -- 172.21.15.101:0/2918218583 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c881a4a50 con 0x7f6c880728b0 2023-07-18T03:40:06.561 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.560+0000 7f6c8cbff700 1 -- 172.21.15.101:0/2918218583 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c88067300 con 0x7f6c880728b0 2023-07-18T03:40:06.562 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.562+0000 7f6c7f7fe700 1 -- 172.21.15.101:0/2918218583 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f6c780225e0 con 0x7f6c880728b0 2023-07-18T03:40:06.562 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.564+0000 7f6c7f7fe700 1 --2- 172.21.15.101:0/2918218583 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6c740a1300 0x7f6c740a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:06.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.564+0000 7f6c7f7fe700 1 -- 172.21.15.101:0/2918218583 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f6c780ce7d0 con 0x7f6c880728b0 2023-07-18T03:40:06.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.564+0000 7f6c7ffff700 1 --2- 172.21.15.101:0/2918218583 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6c740a1300 0x7f6c740a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:06.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.565+0000 7f6c7ffff700 1 --2- 172.21.15.101:0/2918218583 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6c740a1300 0x7f6c740a37b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6c70009fd0 tx=0x7f6c70009260 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:06.563 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.565+0000 7f6c7f7fe700 1 -- 172.21.15.101:0/2918218583 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6c78097810 con 0x7f6c880728b0 2023-07-18T03:40:06.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:06 smithi101 ceph-mon[126379]: osdmap e145: 12 total, 12 up, 12 in 2023-07-18T03:40:06.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:06 smithi101 ceph-mon[126379]: pgmap v1032: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-07-18T03:40:06.739 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:06 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2832810889' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:40:06.796 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:06 smithi155 ceph-mon[128885]: osdmap e145: 12 total, 12 up, 12 in 2023-07-18T03:40:06.796 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:06 smithi155 ceph-mon[128885]: pgmap v1032: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-07-18T03:40:06.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:06 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/2832810889' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:40:06.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.810+0000 7f6c8cbff700 1 -- 172.21.15.101:0/2918218583 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6c88199150 con 0x7f6c880728b0 2023-07-18T03:40:06.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.811+0000 7f6c7f7fe700 1 -- 172.21.15.101:0/2918218583 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+10858 (secure 0 0 0) 0x7f6c7809ae30 con 0x7f6c880728b0 2023-07-18T03:40:06.806 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:40:06.807 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2688212803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2688212803},{"type":"v1","addr":"172.21.15.179:6833","nonce":2688212803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":14436,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/296128976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":296128976},{"type":"v1","addr":"172.21.15.101:6835","nonce":296128976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/2478893321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":2478893321},{"type":"v1","addr":"172.21.15.101:6837","nonce":2478893321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/4192846769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":4192846769},{"type":"v1","addr":"172.21.15.101:6839","nonce":4192846769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6841/3824340175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6840","nonce":3824340175},{"type":"v1","addr":"172.21.15.101:6841","nonce":3824340175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3388721497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3388721497},{"type":"v1","addr":"172.21.15.155:6833","nonce":3388721497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":49,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:04:59.361185+0000","modified":"2023-07-18T03:28:46.287122+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24562,"mds_1":24557,"mds_2":24541},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/943645270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":943645270},{"type":"v1","addr":"172.21.15.155:6835","nonce":943645270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/48029036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":48029036},{"type":"v1","addr":"172.21.15.155:6837","nonce":48029036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/1268903554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1268903554},{"type":"v1","addr":"172.21.15.179:6835","nonce":1268903554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.179:6837/3458593442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":3458593442},{"type":"v1","addr":"172.21.15.179:6837","nonce":3458593442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.155:6839/2055462483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2055462483},{"type":"v1","addr":"172.21.15.155:6839","nonce":2055462483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24562":{"gid":24562,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.179:6839/2061828474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2061828474},{"type":"v1","addr":"172.21.15.179:6839","nonce":2061828474}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:40:06.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.814+0000 7f6c8cbff700 1 -- 172.21.15.101:0/2918218583 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6c740a1300 msgr2=0x7f6c740a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:06.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.814+0000 7f6c8cbff700 1 --2- 172.21.15.101:0/2918218583 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6c740a1300 0x7f6c740a37b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6c70009fd0 tx=0x7f6c70009260 comp rx=0 tx=0).stop 2023-07-18T03:40:06.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.814+0000 7f6c8cbff700 1 -- 172.21.15.101:0/2918218583 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c880728b0 msgr2=0x7f6c88197e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:06.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.814+0000 7f6c8cbff700 1 --2- 172.21.15.101:0/2918218583 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c880728b0 0x7f6c88197e00 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f6c78005910 tx=0x7f6c78004710 comp rx=0 tx=0).stop 2023-07-18T03:40:06.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.814+0000 7f6c8cbff700 1 -- 172.21.15.101:0/2918218583 shutdown_connections 2023-07-18T03:40:06.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.814+0000 7f6c8cbff700 1 --2- 172.21.15.101:0/2918218583 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c8810c330 0x7f6c8819c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:06.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.814+0000 7f6c8cbff700 1 --2- 172.21.15.101:0/2918218583 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c880728b0 0x7f6c88197e00 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:06.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.814+0000 7f6c8cbff700 1 --2- 172.21.15.101:0/2918218583 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f6c740a1300 0x7f6c740a37b0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:06.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.814+0000 7f6c8cbff700 1 --2- 172.21.15.101:0/2918218583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c880731c0 0x7f6c88198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:06.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.814+0000 7f6c8cbff700 1 -- 172.21.15.101:0/2918218583 >> 172.21.15.101:0/2918218583 conn(0x7f6c880fb630 msgr2=0x7f6c8810e420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:06.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.814+0000 7f6c8cbff700 1 -- 172.21.15.101:0/2918218583 shutdown_connections 2023-07-18T03:40:06.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:06.814+0000 7f6c8cbff700 1 -- 172.21.15.101:0/2918218583 wait complete. 2023-07-18T03:40:06.812 INFO:teuthology.orchestra.run.smithi101.stderr:dumped fsmap epoch 49 2023-07-18T03:40:06.827 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:40:06.951 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:06 smithi179 ceph-mon[129229]: osdmap e145: 12 total, 12 up, 12 in 2023-07-18T03:40:06.951 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:06 smithi179 ceph-mon[129229]: pgmap v1032: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-07-18T03:40:06.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:06 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/2832810889' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:40:07.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.032+0000 7f02183ff700 1 -- 172.21.15.101:0/55168827 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0210105ae0 msgr2=0x7f0210105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:07.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.032+0000 7f02183ff700 1 --2- 172.21.15.101:0/55168827 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0210105ae0 0x7f0210105eb0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f020c009990 tx=0x7f020c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:40:07.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.032+0000 7f02183ff700 1 -- 172.21.15.101:0/55168827 shutdown_connections 2023-07-18T03:40:07.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.032+0000 7f02183ff700 1 --2- 172.21.15.101:0/55168827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0210100480 0x7f021010c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:07.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.032+0000 7f02183ff700 1 --2- 172.21.15.101:0/55168827 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0210105ae0 0x7f0210105eb0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:07.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.032+0000 7f02183ff700 1 --2- 172.21.15.101:0/55168827 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f02100ffad0 0x7f02100fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:07.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.032+0000 7f02183ff700 1 -- 172.21.15.101:0/55168827 >> 172.21.15.101:0/55168827 conn(0x7f02100fb590 msgr2=0x7f02100fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:07.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.032+0000 7f02183ff700 1 -- 172.21.15.101:0/55168827 shutdown_connections 2023-07-18T03:40:07.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.032+0000 7f02183ff700 1 -- 172.21.15.101:0/55168827 wait complete. 2023-07-18T03:40:07.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.033+0000 7f02183ff700 1 Processor -- start 2023-07-18T03:40:07.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.033+0000 7f02183ff700 1 -- start start 2023-07-18T03:40:07.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.033+0000 7f02183ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02100ffad0 0x7f0210197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:07.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.034+0000 7f02183ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0210100480 0x7f0210198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:07.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.034+0000 7f02183ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0210105ae0 0x7f021019c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:07.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.034+0000 7f02183ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021019cd10 con 0x7f0210105ae0 2023-07-18T03:40:07.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.034+0000 7f02183ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021019ce80 con 0x7f0210100480 2023-07-18T03:40:07.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.034+0000 7f02183ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021019d040 con 0x7f02100ffad0 2023-07-18T03:40:07.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.034+0000 7f021619b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02100ffad0 0x7f0210197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:07.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.034+0000 7f021699c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0210105ae0 0x7f021019c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:07.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.034+0000 7f021599a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0210100480 0x7f0210198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:07.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.034+0000 7f021619b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02100ffad0 0x7f0210197d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:46170/0 (socket says 172.21.15.101:46170) 2023-07-18T03:40:07.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.034+0000 7f021619b700 1 -- 172.21.15.101:0/3842377404 learned_addr learned my addr 172.21.15.101:0/3842377404 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:40:07.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.034+0000 7f021599a700 1 -- 172.21.15.101:0/3842377404 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02100ffad0 msgr2=0x7f0210197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:07.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.034+0000 7f021599a700 1 --2- 172.21.15.101:0/3842377404 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02100ffad0 0x7f0210197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:07.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.034+0000 7f021599a700 1 -- 172.21.15.101:0/3842377404 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0210105ae0 msgr2=0x7f021019c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:07.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.034+0000 7f021599a700 1 --2- 172.21.15.101:0/3842377404 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0210105ae0 0x7f021019c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:07.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.034+0000 7f021599a700 1 -- 172.21.15.101:0/3842377404 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020c0095f0 con 0x7f0210100480 2023-07-18T03:40:07.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.034+0000 7f021619b700 1 --2- 172.21.15.101:0/3842377404 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02100ffad0 0x7f0210197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:07.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.034+0000 7f021699c700 1 --2- 172.21.15.101:0/3842377404 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0210105ae0 0x7f021019c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:07.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.034+0000 7f021599a700 1 --2- 172.21.15.101:0/3842377404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0210100480 0x7f0210198270 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f020000e9f0 tx=0x7f020000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:07.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.035+0000 7f02077fe700 1 -- 172.21.15.101:0/3842377404 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020000c9c0 con 0x7f0210100480 2023-07-18T03:40:07.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.035+0000 7f02077fe700 1 -- 172.21.15.101:0/3842377404 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0200010430 con 0x7f0210100480 2023-07-18T03:40:07.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.035+0000 7f02183ff700 1 -- 172.21.15.101:0/3842377404 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f021019d320 con 0x7f0210100480 2023-07-18T03:40:07.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.035+0000 7f02077fe700 1 -- 172.21.15.101:0/3842377404 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0200018680 con 0x7f0210100480 2023-07-18T03:40:07.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.035+0000 7f02183ff700 1 -- 172.21.15.101:0/3842377404 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02101a4b60 con 0x7f0210100480 2023-07-18T03:40:07.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.036+0000 7f02183ff700 1 -- 172.21.15.101:0/3842377404 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0210067300 con 0x7f0210100480 2023-07-18T03:40:07.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.038+0000 7f02077fe700 1 -- 172.21.15.101:0/3842377404 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f0200010860 con 0x7f0210100480 2023-07-18T03:40:07.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.039+0000 7f02077fe700 1 --2- 172.21.15.101:0/3842377404 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f01fc0a1300 0x7f01fc0a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:07.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.039+0000 7f02077fe700 1 -- 172.21.15.101:0/3842377404 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f0200014070 con 0x7f0210100480 2023-07-18T03:40:07.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.039+0000 7f021619b700 1 --2- 172.21.15.101:0/3842377404 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f01fc0a1300 0x7f01fc0a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:07.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.040+0000 7f021619b700 1 --2- 172.21.15.101:0/3842377404 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f01fc0a1300 0x7f01fc0a37b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f020c006010 tx=0x7f020c005960 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:07.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.042+0000 7f02077fe700 1 -- 172.21.15.101:0/3842377404 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0200096200 con 0x7f0210100480 2023-07-18T03:40:07.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.244+0000 7f02183ff700 1 -- 172.21.15.101:0/3842377404 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0210199080 con 0x7f0210100480 2023-07-18T03:40:07.240 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.246+0000 7f02077fe700 1 -- 172.21.15.101:0/3842377404 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) v1 ==== 74+0+25433 (secure 0 0 0) 0x7f0200099820 con 0x7f0210100480 2023-07-18T03:40:07.241 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-18T03:40:07.241 INFO:teuthology.orchestra.run.smithi101.stdout:{"epoch":145,"fsid":"b1782daa-2516-11ee-9b34-001a4aab830c","created":"2023-07-18T02:57:22.972483+0000","modified":"2023-07-18T03:40:05.504549+0000","last_up_change":"2023-07-18T03:04:11.429530+0000","last_in_change":"2023-07-18T03:03:54.158340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:00:43.353489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:04:57.771616+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-18T03:04:58.769590+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":61,"snap_epoch":145,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-18T03:05:02.358830+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":61,"snap_epoch":145,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dc76b5d1-38a0-4f1a-9335-756bfc40a62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6803","nonce":3610719929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6805","nonce":3610719929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6809","nonce":3610719929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3610719929},{"type":"v1","addr":"172.21.15.101:6807","nonce":3610719929}]},"public_addr":"172.21.15.101:6803/3610719929","cluster_addr":"172.21.15.101:6805/3610719929","heartbeat_back_addr":"172.21.15.101:6809/3610719929","heartbeat_front_addr":"172.21.15.101:6807/3610719929","state":["exists","up"]},{"osd":1,"uuid":"2a299573-0b34-4af9-aaae-b523e5058ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6801","nonce":1030229579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6803","nonce":1030229579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6807","nonce":1030229579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1030229579},{"type":"v1","addr":"172.21.15.155:6805","nonce":1030229579}]},"public_addr":"172.21.15.155:6801/1030229579","cluster_addr":"172.21.15.155:6803/1030229579","heartbeat_back_addr":"172.21.15.155:6807/1030229579","heartbeat_front_addr":"172.21.15.155:6805/1030229579","state":["exists","up"]},{"osd":2,"uuid":"123b3749-c636-48fa-9e33-6229b5575dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6801","nonce":1349487507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6803","nonce":1349487507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6807","nonce":1349487507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":1349487507},{"type":"v1","addr":"172.21.15.179:6805","nonce":1349487507}]},"public_addr":"172.21.15.179:6801/1349487507","cluster_addr":"172.21.15.179:6803/1349487507","heartbeat_back_addr":"172.21.15.179:6807/1349487507","heartbeat_front_addr":"172.21.15.179:6805/1349487507","state":["exists","up"]},{"osd":3,"uuid":"4b0c409d-2954-4127-833a-b196bf226d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6811","nonce":4226577463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6813","nonce":4226577463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6817","nonce":4226577463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":4226577463},{"type":"v1","addr":"172.21.15.101:6815","nonce":4226577463}]},"public_addr":"172.21.15.101:6811/4226577463","cluster_addr":"172.21.15.101:6813/4226577463","heartbeat_back_addr":"172.21.15.101:6817/4226577463","heartbeat_front_addr":"172.21.15.101:6815/4226577463","state":["exists","up"]},{"osd":4,"uuid":"2137f2df-984e-45ce-baf6-b4ac12c7e3c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6809","nonce":1951135805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6811","nonce":1951135805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6815","nonce":1951135805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":1951135805},{"type":"v1","addr":"172.21.15.155:6813","nonce":1951135805}]},"public_addr":"172.21.15.155:6809/1951135805","cluster_addr":"172.21.15.155:6811/1951135805","heartbeat_back_addr":"172.21.15.155:6815/1951135805","heartbeat_front_addr":"172.21.15.155:6813/1951135805","state":["exists","up"]},{"osd":5,"uuid":"1020bfc5-8c2f-482e-9588-66c7f98e46e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6809","nonce":2746037021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6811","nonce":2746037021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6815","nonce":2746037021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":2746037021},{"type":"v1","addr":"172.21.15.179:6813","nonce":2746037021}]},"public_addr":"172.21.15.179:6809/2746037021","cluster_addr":"172.21.15.179:6811/2746037021","heartbeat_back_addr":"172.21.15.179:6815/2746037021","heartbeat_front_addr":"172.21.15.179:6813/2746037021","state":["exists","up"]},{"osd":6,"uuid":"2cd99260-5c36-4603-9338-cf697970ce58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6819","nonce":958044931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6821","nonce":958044931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6825","nonce":958044931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":958044931},{"type":"v1","addr":"172.21.15.101:6823","nonce":958044931}]},"public_addr":"172.21.15.101:6819/958044931","cluster_addr":"172.21.15.101:6821/958044931","heartbeat_back_addr":"172.21.15.101:6825/958044931","heartbeat_front_addr":"172.21.15.101:6823/958044931","state":["exists","up"]},{"osd":7,"uuid":"ed23fd0a-678f-4e15-b42c-cfe873315714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6817","nonce":3256322291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6819","nonce":3256322291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6823","nonce":3256322291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3256322291},{"type":"v1","addr":"172.21.15.155:6821","nonce":3256322291}]},"public_addr":"172.21.15.155:6817/3256322291","cluster_addr":"172.21.15.155:6819/3256322291","heartbeat_back_addr":"172.21.15.155:6823/3256322291","heartbeat_front_addr":"172.21.15.155:6821/3256322291","state":["exists","up"]},{"osd":8,"uuid":"eca3d8e6-7baf-47a5-b991-328001c7ebef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6817","nonce":3520612559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6819","nonce":3520612559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6823","nonce":3520612559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3520612559},{"type":"v1","addr":"172.21.15.179:6821","nonce":3520612559}]},"public_addr":"172.21.15.179:6817/3520612559","cluster_addr":"172.21.15.179:6819/3520612559","heartbeat_back_addr":"172.21.15.179:6823/3520612559","heartbeat_front_addr":"172.21.15.179:6821/3520612559","state":["exists","up"]},{"osd":9,"uuid":"465a7ffb-67e4-49ef-812c-1b96feacfebf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6827","nonce":2166787667}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6829","nonce":2166787667}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6833","nonce":2166787667}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":2166787667},{"type":"v1","addr":"172.21.15.101:6831","nonce":2166787667}]},"public_addr":"172.21.15.101:6827/2166787667","cluster_addr":"172.21.15.101:6829/2166787667","heartbeat_back_addr":"172.21.15.101:6833/2166787667","heartbeat_front_addr":"172.21.15.101:6831/2166787667","state":["exists","up"]},{"osd":10,"uuid":"1897a027-f25d-434f-8df5-5fc2bd866578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6825","nonce":3682446349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6827","nonce":3682446349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6831","nonce":3682446349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":3682446349},{"type":"v1","addr":"172.21.15.155:6829","nonce":3682446349}]},"public_addr":"172.21.15.155:6825/3682446349","cluster_addr":"172.21.15.155:6827/3682446349","heartbeat_back_addr":"172.21.15.155:6831/3682446349","heartbeat_front_addr":"172.21.15.155:6829/3682446349","state":["exists","up"]},{"osd":11,"uuid":"b62effaf-45e2-4585-b8d0-db448f3474ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6825","nonce":2602640858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6827","nonce":2602640858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6831","nonce":2602640858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":2602640858},{"type":"v1","addr":"172.21.15.179:6829","nonce":2602640858}]},"public_addr":"172.21.15.179:6825/2602640858","cluster_addr":"172.21.15.179:6827/2602640858","heartbeat_back_addr":"172.21.15.179:6831/2602640858","heartbeat_front_addr":"172.21.15.179:6829/2602640858","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T02:59:56.074519+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:19.358142+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:00:41.128966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:08.227054+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:31.442592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:01:52.751685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:19.251054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:02:40.698241+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:01.339150+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:27.955910+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:03:49.136980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:04:10.084982+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.101:0/2766365637":"2023-07-19T03:07:33.498516+0000","172.21.15.101:0/523372970":"2023-07-19T03:07:33.498516+0000","172.21.15.101:6800/3963742842":"2023-07-19T03:07:33.498516+0000","172.21.15.101:6800/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/1806538035":"2023-07-19T03:07:33.498516+0000","172.21.15.101:6801/2461037266":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/604228291":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/1004430030":"2023-07-19T03:07:33.498516+0000","172.21.15.101:6801/3963742842":"2023-07-19T03:07:33.498516+0000","172.21.15.101:0/4055187849":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/2844574324":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/1603248215":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6801/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:6800/3497914514":"2023-07-19T02:57:56.243200+0000","172.21.15.101:0/3307578182":"2023-07-19T02:58:22.762480+0000","172.21.15.101:0/373189792":"2023-07-19T02:57:56.243200+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":48,"length":1},{"begin":61,"length":1}]},{"pool":4,"snaps":[{"begin":48,"length":1},{"begin":61,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":48,"length":1},{"begin":61,"length":1}]},{"pool":4,"snaps":[{"begin":48,"length":1},{"begin":61,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:40:07.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.248+0000 7f02183ff700 1 -- 172.21.15.101:0/3842377404 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f01fc0a1300 msgr2=0x7f01fc0a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:07.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.248+0000 7f02183ff700 1 --2- 172.21.15.101:0/3842377404 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f01fc0a1300 0x7f01fc0a37b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f020c006010 tx=0x7f020c005960 comp rx=0 tx=0).stop 2023-07-18T03:40:07.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.248+0000 7f02183ff700 1 -- 172.21.15.101:0/3842377404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0210100480 msgr2=0x7f0210198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:07.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.248+0000 7f02183ff700 1 --2- 172.21.15.101:0/3842377404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0210100480 0x7f0210198270 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f020000e9f0 tx=0x7f020000ed00 comp rx=0 tx=0).stop 2023-07-18T03:40:07.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.248+0000 7f02183ff700 1 -- 172.21.15.101:0/3842377404 shutdown_connections 2023-07-18T03:40:07.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.248+0000 7f02183ff700 1 --2- 172.21.15.101:0/3842377404 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0210105ae0 0x7f021019c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:07.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.248+0000 7f02183ff700 1 --2- 172.21.15.101:0/3842377404 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02100ffad0 0x7f0210197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:07.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.248+0000 7f02183ff700 1 --2- 172.21.15.101:0/3842377404 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f01fc0a1300 0x7f01fc0a37b0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:07.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.248+0000 7f02183ff700 1 --2- 172.21.15.101:0/3842377404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0210100480 0x7f0210198270 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:07.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.248+0000 7f02183ff700 1 -- 172.21.15.101:0/3842377404 >> 172.21.15.101:0/3842377404 conn(0x7f02100fb590 msgr2=0x7f021010ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:07.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.249+0000 7f02183ff700 1 -- 172.21.15.101:0/3842377404 shutdown_connections 2023-07-18T03:40:07.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.249+0000 7f02183ff700 1 -- 172.21.15.101:0/3842377404 wait complete. 2023-07-18T03:40:07.263 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-18T03:40:07.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.468+0000 7f34069ff700 1 -- 172.21.15.101:0/1974411849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3400105ae0 msgr2=0x7f3400105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:07.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.468+0000 7f34069ff700 1 --2- 172.21.15.101:0/1974411849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3400105ae0 0x7f3400105eb0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f33f00098f0 tx=0x7f33f0009c00 comp rx=0 tx=0).stop 2023-07-18T03:40:07.464 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.468+0000 7f34069ff700 1 -- 172.21.15.101:0/1974411849 shutdown_connections 2023-07-18T03:40:07.465 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.468+0000 7f34069ff700 1 --2- 172.21.15.101:0/1974411849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3400105ae0 0x7f3400105eb0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:07.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.468+0000 7f34069ff700 1 --2- 172.21.15.101:0/1974411849 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3400100480 0x7f340010c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:07.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.468+0000 7f34069ff700 1 --2- 172.21.15.101:0/1974411849 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f34000ffad0 0x7f34000fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:07.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.468+0000 7f34069ff700 1 -- 172.21.15.101:0/1974411849 >> 172.21.15.101:0/1974411849 conn(0x7f34000fb5f0 msgr2=0x7f34000fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:07.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.469+0000 7f34069ff700 1 -- 172.21.15.101:0/1974411849 shutdown_connections 2023-07-18T03:40:07.466 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.469+0000 7f34069ff700 1 -- 172.21.15.101:0/1974411849 wait complete. 2023-07-18T03:40:07.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.470+0000 7f34069ff700 1 Processor -- start 2023-07-18T03:40:07.468 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.470+0000 7f34069ff700 1 -- start start 2023-07-18T03:40:07.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.470+0000 7f34069ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34000ffad0 0x7f3400197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:07.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.470+0000 7f34069ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3400100480 0x7f3400198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:07.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.470+0000 7f34069ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3400105ae0 0x7f340019c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:07.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.470+0000 7f34069ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340019ce30 con 0x7f3400100480 2023-07-18T03:40:07.469 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.470+0000 7f34069ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340019cfa0 con 0x7f3400105ae0 2023-07-18T03:40:07.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.470+0000 7f34069ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340019d160 con 0x7f34000ffad0 2023-07-18T03:40:07.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.470+0000 7f33fffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34000ffad0 0x7f3400197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:07.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.470+0000 7f33fffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34000ffad0 0x7f3400197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:46188/0 (socket says 172.21.15.101:46188) 2023-07-18T03:40:07.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.470+0000 7f33fffff700 1 -- 172.21.15.101:0/2094347302 learned_addr learned my addr 172.21.15.101:0/2094347302 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:40:07.470 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.470+0000 7f3404f9c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3400105ae0 0x7f340019c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:07.471 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.470+0000 7f33ff7fe700 1 --2- 172.21.15.101:0/2094347302 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3400100480 0x7f3400198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:07.471 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.471+0000 7f33fffff700 1 -- 172.21.15.101:0/2094347302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3400105ae0 msgr2=0x7f340019c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:07.471 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.471+0000 7f33fffff700 1 --2- 172.21.15.101:0/2094347302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3400105ae0 0x7f340019c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:07.471 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.471+0000 7f33fffff700 1 -- 172.21.15.101:0/2094347302 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3400100480 msgr2=0x7f3400198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:07.471 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.471+0000 7f33fffff700 1 --2- 172.21.15.101:0/2094347302 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3400100480 0x7f3400198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:07.472 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.471+0000 7f33fffff700 1 -- 172.21.15.101:0/2094347302 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f8009d00 con 0x7f34000ffad0 2023-07-18T03:40:07.472 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.471+0000 7f33fffff700 1 --2- 172.21.15.101:0/2094347302 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34000ffad0 0x7f3400197e50 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f33f0009810 tx=0x7f33f000fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:07.472 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.471+0000 7f33fd7fa700 1 -- 172.21.15.101:0/2094347302 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f001c070 con 0x7f34000ffad0 2023-07-18T03:40:07.472 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.471+0000 7f34069ff700 1 -- 172.21.15.101:0/2094347302 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33f0009520 con 0x7f34000ffad0 2023-07-18T03:40:07.472 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.471+0000 7f33fd7fa700 1 -- 172.21.15.101:0/2094347302 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f33f000bb20 con 0x7f34000ffad0 2023-07-18T03:40:07.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.471+0000 7f33fd7fa700 1 -- 172.21.15.101:0/2094347302 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f0004930 con 0x7f34000ffad0 2023-07-18T03:40:07.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.471+0000 7f34069ff700 1 -- 172.21.15.101:0/2094347302 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f340019d670 con 0x7f34000ffad0 2023-07-18T03:40:07.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.472+0000 7f34069ff700 1 -- 172.21.15.101:0/2094347302 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3400067290 con 0x7f34000ffad0 2023-07-18T03:40:07.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.474+0000 7f33fd7fa700 1 -- 172.21.15.101:0/2094347302 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f33f0020070 con 0x7f34000ffad0 2023-07-18T03:40:07.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.475+0000 7f33fd7fa700 1 --2- 172.21.15.101:0/2094347302 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f33e80a12c0 0x7f33e80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:07.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.476+0000 7f33fd7fa700 1 -- 172.21.15.101:0/2094347302 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f33f00ce220 con 0x7f34000ffad0 2023-07-18T03:40:07.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.476+0000 7f33ff7fe700 1 --2- 172.21.15.101:0/2094347302 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f33e80a12c0 0x7f33e80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:07.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.476+0000 7f33ff7fe700 1 --2- 172.21.15.101:0/2094347302 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f33e80a12c0 0x7f33e80a3770 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3400199460 tx=0x7f33f4009260 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:07.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.478+0000 7f33fd7fa700 1 -- 172.21.15.101:0/2094347302 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f33f0097260 con 0x7f34000ffad0 2023-07-18T03:40:07.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.725+0000 7f34069ff700 1 -- 172.21.15.101:0/2094347302 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f34001991a0 con 0x7f34000ffad0 2023-07-18T03:40:07.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.733+0000 7f33fd7fa700 1 -- 172.21.15.101:0/2094347302 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v50) v1 ==== 161+0+0 (secure 0 0 0) 0x7f33f009a880 con 0x7f34000ffad0 2023-07-18T03:40:07.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.735+0000 7f34069ff700 1 -- 172.21.15.101:0/2094347302 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f33e80a12c0 msgr2=0x7f33e80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:07.730 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.735+0000 7f34069ff700 1 --2- 172.21.15.101:0/2094347302 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f33e80a12c0 0x7f33e80a3770 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3400199460 tx=0x7f33f4009260 comp rx=0 tx=0).stop 2023-07-18T03:40:07.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.735+0000 7f34069ff700 1 -- 172.21.15.101:0/2094347302 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34000ffad0 msgr2=0x7f3400197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:07.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.735+0000 7f34069ff700 1 --2- 172.21.15.101:0/2094347302 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34000ffad0 0x7f3400197e50 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f33f0009810 tx=0x7f33f000fa00 comp rx=0 tx=0).stop 2023-07-18T03:40:07.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.735+0000 7f34069ff700 1 -- 172.21.15.101:0/2094347302 shutdown_connections 2023-07-18T03:40:07.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.735+0000 7f34069ff700 1 --2- 172.21.15.101:0/2094347302 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f33e80a12c0 0x7f33e80a3770 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:07.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.735+0000 7f34069ff700 1 --2- 172.21.15.101:0/2094347302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3400105ae0 0x7f340019c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:07.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.735+0000 7f34069ff700 1 --2- 172.21.15.101:0/2094347302 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34000ffad0 0x7f3400197e50 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:07.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.735+0000 7f34069ff700 1 --2- 172.21.15.101:0/2094347302 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3400100480 0x7f3400198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:07.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.735+0000 7f34069ff700 1 -- 172.21.15.101:0/2094347302 >> 172.21.15.101:0/2094347302 conn(0x7f34000fb5f0 msgr2=0x7f340010ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:07.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.736+0000 7f34069ff700 1 -- 172.21.15.101:0/2094347302 shutdown_connections 2023-07-18T03:40:07.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.736+0000 7f34069ff700 1 -- 172.21.15.101:0/2094347302 wait complete. 2023-07-18T03:40:07.733 INFO:teuthology.orchestra.run.smithi101.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-18T03:40:07.753 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-18T03:40:07.790 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:07 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/2918218583' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:40:07.791 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:07 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/3842377404' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:40:07.808 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:07 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/2918218583' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:40:07.808 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:07 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/3842377404' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:40:07.814 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:07 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2918218583' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:40:07.814 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:07 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3842377404' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:40:07.815 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:07 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mon-a[126354]: 2023-07-18T03:40:07.726+0000 7f32aab9e700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-18T03:40:07.954 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.959+0000 7f8fe65ff700 1 -- 172.21.15.101:0/3066528097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8fe00ffad0 msgr2=0x7f8fe00fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:07.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.959+0000 7f8fe65ff700 1 --2- 172.21.15.101:0/3066528097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8fe00ffad0 0x7f8fe00fff40 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4009990 tx=0x7f8fd4009ca0 comp rx=0 tx=0).stop 2023-07-18T03:40:07.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.959+0000 7f8fe65ff700 1 -- 172.21.15.101:0/3066528097 shutdown_connections 2023-07-18T03:40:07.955 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.959+0000 7f8fe65ff700 1 --2- 172.21.15.101:0/3066528097 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8fe0100480 0x7f8fe010c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:07.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.959+0000 7f8fe65ff700 1 --2- 172.21.15.101:0/3066528097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8fe0105ae0 0x7f8fe0105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:07.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.959+0000 7f8fe65ff700 1 --2- 172.21.15.101:0/3066528097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8fe00ffad0 0x7f8fe00fff40 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:07.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.959+0000 7f8fe65ff700 1 -- 172.21.15.101:0/3066528097 >> 172.21.15.101:0/3066528097 conn(0x7f8fe00fb5f0 msgr2=0x7f8fe00fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:07.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.959+0000 7f8fe65ff700 1 -- 172.21.15.101:0/3066528097 shutdown_connections 2023-07-18T03:40:07.956 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.959+0000 7f8fe65ff700 1 -- 172.21.15.101:0/3066528097 wait complete. 2023-07-18T03:40:07.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.960+0000 7f8fe65ff700 1 Processor -- start 2023-07-18T03:40:07.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.961+0000 7f8fe65ff700 1 -- start start 2023-07-18T03:40:07.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.961+0000 7f8fe65ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8fe00ffad0 0x7f8fe019ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:07.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.961+0000 7f8fe65ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8fe0100480 0x7f8fe019bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:07.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.961+0000 7f8fe65ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8fe0105ae0 0x7f8fe0195a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:07.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.961+0000 7f8fe65ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe01960e0 con 0x7f8fe00ffad0 2023-07-18T03:40:07.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.961+0000 7f8fe65ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe0196250 con 0x7f8fe0100480 2023-07-18T03:40:07.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.961+0000 7f8fe65ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe0196410 con 0x7f8fe0105ae0 2023-07-18T03:40:07.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.961+0000 7f8fdffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8fe00ffad0 0x7f8fe019ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:07.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.961+0000 7f8fe4b9c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8fe0105ae0 0x7f8fe0195a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:07.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.961+0000 7f8fdffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8fe00ffad0 0x7f8fe019ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:59202/0 (socket says 172.21.15.101:59202) 2023-07-18T03:40:07.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.961+0000 7f8fdffff700 1 -- 172.21.15.101:0/424673235 learned_addr learned my addr 172.21.15.101:0/424673235 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:40:07.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.961+0000 7f8fdf7fe700 1 --2- 172.21.15.101:0/424673235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8fe0100480 0x7f8fe019bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:07.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.962+0000 7f8fe4b9c700 1 -- 172.21.15.101:0/424673235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8fe0100480 msgr2=0x7f8fe019bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:07.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.962+0000 7f8fe4b9c700 1 --2- 172.21.15.101:0/424673235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8fe0100480 0x7f8fe019bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:07.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.962+0000 7f8fe4b9c700 1 -- 172.21.15.101:0/424673235 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8fe00ffad0 msgr2=0x7f8fe019ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:07.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.962+0000 7f8fe4b9c700 1 --2- 172.21.15.101:0/424673235 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8fe00ffad0 0x7f8fe019ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:07.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.962+0000 7f8fe4b9c700 1 -- 172.21.15.101:0/424673235 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd40095f0 con 0x7f8fe0105ae0 2023-07-18T03:40:07.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.962+0000 7f8fdffff700 1 --2- 172.21.15.101:0/424673235 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8fe00ffad0 0x7f8fe019ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:07.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.962+0000 7f8fe4b9c700 1 --2- 172.21.15.101:0/424673235 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8fe0105ae0 0x7f8fe0195a70 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f8fd800efc0 tx=0x7f8fd800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:07.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.962+0000 7f8fdd7fa700 1 -- 172.21.15.101:0/424673235 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8018070 con 0x7f8fe0105ae0 2023-07-18T03:40:07.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.962+0000 7f8fe65ff700 1 -- 172.21.15.101:0/424673235 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fe0196690 con 0x7f8fe0105ae0 2023-07-18T03:40:07.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.962+0000 7f8fdd7fa700 1 -- 172.21.15.101:0/424673235 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fd800f040 con 0x7f8fe0105ae0 2023-07-18T03:40:07.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.962+0000 7f8fdd7fa700 1 -- 172.21.15.101:0/424673235 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd800cd40 con 0x7f8fe0105ae0 2023-07-18T03:40:07.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.962+0000 7f8fe65ff700 1 -- 172.21.15.101:0/424673235 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fe01a29f0 con 0x7f8fe0105ae0 2023-07-18T03:40:07.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.963+0000 7f8fe65ff700 1 -- 172.21.15.101:0/424673235 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fe0067290 con 0x7f8fe0105ae0 2023-07-18T03:40:07.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.965+0000 7f8fdd7fa700 1 -- 172.21.15.101:0/424673235 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f8fd8003a00 con 0x7f8fe0105ae0 2023-07-18T03:40:07.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.967+0000 7f8fdd7fa700 1 --2- 172.21.15.101:0/424673235 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f8fc80a12c0 0x7f8fc80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:07.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.967+0000 7f8fdd7fa700 1 -- 172.21.15.101:0/424673235 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 9719+0+0 (secure 0 0 0) 0x7f8fd80ccdf0 con 0x7f8fe0105ae0 2023-07-18T03:40:07.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.967+0000 7f8fdffff700 1 --2- 172.21.15.101:0/424673235 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f8fc80a12c0 0x7f8fc80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:07.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.967+0000 7f8fdffff700 1 --2- 172.21.15.101:0/424673235 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f8fc80a12c0 0x7f8fc80a3770 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0011550 tx=0x7f8fd0009260 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:07.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:07.971+0000 7f8fdd7fa700 1 -- 172.21.15.101:0/424673235 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8fd8095d80 con 0x7f8fe0105ae0 2023-07-18T03:40:08.046 INFO:journalctl@ceph.mds.e.smithi155.stdout:Jul 18 03:40:07 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-e[153469]: ignoring --setuser ceph since I am not root 2023-07-18T03:40:08.047 INFO:journalctl@ceph.mds.e.smithi155.stdout:Jul 18 03:40:07 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-e[153469]: ignoring --setgroup ceph since I am not root 2023-07-18T03:40:08.047 INFO:journalctl@ceph.mds.e.smithi155.stdout:Jul 18 03:40:07 smithi155 ceph-mds[153493]: -- 172.21.15.155:0/3865251106 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561d49a231e0 con 0x561d499f8800 2023-07-18T03:40:08.047 INFO:journalctl@ceph.mds.e.smithi155.stdout:Jul 18 03:40:07 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-e[153469]: starting mds.e at 2023-07-18T03:40:08.048 INFO:journalctl@ceph.mds.h.smithi155.stdout:Jul 18 03:40:07 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-h[153809]: ignoring --setuser ceph since I am not root 2023-07-18T03:40:08.048 INFO:journalctl@ceph.mds.h.smithi155.stdout:Jul 18 03:40:07 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-h[153809]: ignoring --setgroup ceph since I am not root 2023-07-18T03:40:08.048 INFO:journalctl@ceph.mds.h.smithi155.stdout:Jul 18 03:40:07 smithi155 ceph-mds[153833]: -- 172.21.15.155:0/3646820623 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562a642971e0 con 0x562a6426d000 2023-07-18T03:40:08.049 INFO:journalctl@ceph.mds.h.smithi155.stdout:Jul 18 03:40:07 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-h[153809]: starting mds.h at 2023-07-18T03:40:08.049 INFO:journalctl@ceph.mds.b.smithi155.stdout:Jul 18 03:40:07 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-b[154157]: ignoring --setuser ceph since I am not root 2023-07-18T03:40:08.049 INFO:journalctl@ceph.mds.b.smithi155.stdout:Jul 18 03:40:07 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-b[154157]: ignoring --setgroup ceph since I am not root 2023-07-18T03:40:08.049 INFO:journalctl@ceph.mds.b.smithi155.stdout:Jul 18 03:40:07 smithi155 ceph-mds[154181]: -- 172.21.15.155:0/2485515891 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ad710bb1e0 con 0x55ad71091000 2023-07-18T03:40:08.050 INFO:journalctl@ceph.mds.b.smithi155.stdout:Jul 18 03:40:07 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-b[154157]: starting mds.b at 2023-07-18T03:40:08.201 INFO:journalctl@ceph.mds.f.smithi179.stdout:Jul 18 03:40:07 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-f[154631]: ignoring --setuser ceph since I am not root 2023-07-18T03:40:08.202 INFO:journalctl@ceph.mds.f.smithi179.stdout:Jul 18 03:40:07 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-f[154631]: ignoring --setgroup ceph since I am not root 2023-07-18T03:40:08.202 INFO:journalctl@ceph.mds.f.smithi179.stdout:Jul 18 03:40:07 smithi179 ceph-mds[154655]: -- 172.21.15.179:0/3971927728 <== mon.2 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5652c3d071e0 con 0x5652c3cdc800 2023-07-18T03:40:08.203 INFO:journalctl@ceph.mds.f.smithi179.stdout:Jul 18 03:40:07 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-f[154631]: starting mds.f at 2023-07-18T03:40:08.203 INFO:journalctl@ceph.mds.i.smithi179.stdout:Jul 18 03:40:07 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-i[155309]: ignoring --setuser ceph since I am not root 2023-07-18T03:40:08.203 INFO:journalctl@ceph.mds.i.smithi179.stdout:Jul 18 03:40:07 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-i[155309]: ignoring --setgroup ceph since I am not root 2023-07-18T03:40:08.204 INFO:journalctl@ceph.mds.i.smithi179.stdout:Jul 18 03:40:07 smithi179 ceph-mds[155333]: -- 172.21.15.179:0/3771417799 <== mon.2 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b1684351e0 con 0x55b16840ac00 2023-07-18T03:40:08.204 INFO:journalctl@ceph.mds.i.smithi179.stdout:Jul 18 03:40:07 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-i[155309]: starting mds.i at 2023-07-18T03:40:08.205 INFO:journalctl@ceph.mds.c.smithi179.stdout:Jul 18 03:40:07 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-c[154971]: ignoring --setuser ceph since I am not root 2023-07-18T03:40:08.205 INFO:journalctl@ceph.mds.c.smithi179.stdout:Jul 18 03:40:07 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-c[154971]: ignoring --setgroup ceph since I am not root 2023-07-18T03:40:08.205 INFO:journalctl@ceph.mds.c.smithi179.stdout:Jul 18 03:40:07 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-c[154971]: starting mds.c at 2023-07-18T03:40:08.212 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.217+0000 7f8fe65ff700 1 -- 172.21.15.101:0/424673235 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8fe01a2c20 con 0x7f8fe0105ae0 2023-07-18T03:40:08.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.733+0000 7f8fdd7fa700 1 -- 172.21.15.101:0/424673235 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v51) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8fd80993a0 con 0x7f8fe0105ae0 2023-07-18T03:40:08.731 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.736+0000 7f8fe65ff700 1 -- 172.21.15.101:0/424673235 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f8fc80a12c0 msgr2=0x7f8fc80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:08.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.736+0000 7f8fe65ff700 1 --2- 172.21.15.101:0/424673235 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f8fc80a12c0 0x7f8fc80a3770 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0011550 tx=0x7f8fd0009260 comp rx=0 tx=0).stop 2023-07-18T03:40:08.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.737+0000 7f8fe65ff700 1 -- 172.21.15.101:0/424673235 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8fe0105ae0 msgr2=0x7f8fe0195a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:08.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.737+0000 7f8fe65ff700 1 --2- 172.21.15.101:0/424673235 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8fe0105ae0 0x7f8fe0195a70 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f8fd800efc0 tx=0x7f8fd800c3a0 comp rx=0 tx=0).stop 2023-07-18T03:40:08.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.737+0000 7f8fe65ff700 1 -- 172.21.15.101:0/424673235 shutdown_connections 2023-07-18T03:40:08.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.737+0000 7f8fe65ff700 1 --2- 172.21.15.101:0/424673235 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8fe0105ae0 0x7f8fe0195a70 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:08.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.737+0000 7f8fe65ff700 1 --2- 172.21.15.101:0/424673235 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8fe00ffad0 0x7f8fe019ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:08.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.737+0000 7f8fe65ff700 1 --2- 172.21.15.101:0/424673235 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f8fc80a12c0 0x7f8fc80a3770 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:08.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.737+0000 7f8fe65ff700 1 --2- 172.21.15.101:0/424673235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8fe0100480 0x7f8fe019bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:08.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.737+0000 7f8fe65ff700 1 -- 172.21.15.101:0/424673235 >> 172.21.15.101:0/424673235 conn(0x7f8fe00fb5f0 msgr2=0x7f8fe00fbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:08.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.737+0000 7f8fe65ff700 1 -- 172.21.15.101:0/424673235 shutdown_connections 2023-07-18T03:40:08.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.737+0000 7f8fe65ff700 1 -- 172.21.15.101:0/424673235 wait complete. 2023-07-18T03:40:08.753 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-18T03:40:08.796 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:08 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/2094347302' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-18T03:40:08.796 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:08 smithi155 ceph-mon[128885]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-18T03:40:08.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:08 smithi155 ceph-mon[128885]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-07-18T03:40:08.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:08 smithi155 ceph-mon[128885]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-18T03:40:08.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:08 smithi155 ceph-mon[128885]: osdmap e146: 12 total, 12 up, 12 in 2023-07-18T03:40:08.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:08 smithi155 ceph-mon[128885]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-07-18T03:40:08.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:08 smithi155 ceph-mon[128885]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-07-18T03:40:08.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:08 smithi155 ceph-mon[128885]: pgmap v1034: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 66 KiB/s wr, 25 op/s 2023-07-18T03:40:08.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:08 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/424673235' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-18T03:40:08.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:08 smithi155 ceph-mon[128885]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-18T03:40:08.815 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:08 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2094347302' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-18T03:40:08.816 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:08 smithi101 ceph-mon[126379]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-18T03:40:08.816 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:08 smithi101 ceph-mon[126379]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-07-18T03:40:08.816 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:08 smithi101 ceph-mon[126379]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-18T03:40:08.816 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:08 smithi101 ceph-mon[126379]: osdmap e146: 12 total, 12 up, 12 in 2023-07-18T03:40:08.817 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:08 smithi101 ceph-mon[126379]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-07-18T03:40:08.817 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:08 smithi101 ceph-mon[126379]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-07-18T03:40:08.817 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:08 smithi101 ceph-mon[126379]: pgmap v1034: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 66 KiB/s wr, 25 op/s 2023-07-18T03:40:08.817 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:08 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/424673235' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-18T03:40:08.817 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:08 smithi101 ceph-mon[126379]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-18T03:40:08.951 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:08 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/2094347302' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-18T03:40:08.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:08 smithi179 ceph-mon[129229]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-18T03:40:08.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:08 smithi179 ceph-mon[129229]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-07-18T03:40:08.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:08 smithi179 ceph-mon[129229]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-18T03:40:08.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:08 smithi179 ceph-mon[129229]: osdmap e146: 12 total, 12 up, 12 in 2023-07-18T03:40:08.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:08 smithi179 ceph-mon[129229]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-07-18T03:40:08.953 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:08 smithi179 ceph-mon[129229]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-07-18T03:40:08.953 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:08 smithi179 ceph-mon[129229]: pgmap v1034: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 66 KiB/s wr, 25 op/s 2023-07-18T03:40:08.953 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:08 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/424673235' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-18T03:40:08.953 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:08 smithi179 ceph-mon[129229]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-18T03:40:08.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.962+0000 7f444d9ff700 1 -- 172.21.15.101:0/3685177226 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f444810c320 msgr2=0x7f444810e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:08.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.962+0000 7f444d9ff700 1 --2- 172.21.15.101:0/3685177226 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f444810c320 0x7f444810e700 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f444000b0a0 tx=0x7f444000b3b0 comp rx=0 tx=0).stop 2023-07-18T03:40:08.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.962+0000 7f444d9ff700 1 -- 172.21.15.101:0/3685177226 shutdown_connections 2023-07-18T03:40:08.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.962+0000 7f444d9ff700 1 --2- 172.21.15.101:0/3685177226 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44480728b0 0x7f4448072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:08.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.962+0000 7f444d9ff700 1 --2- 172.21.15.101:0/3685177226 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f444810c320 0x7f444810e700 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:08.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.962+0000 7f444d9ff700 1 --2- 172.21.15.101:0/3685177226 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44480731c0 0x7f444810bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:08.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.962+0000 7f444d9ff700 1 -- 172.21.15.101:0/3685177226 >> 172.21.15.101:0/3685177226 conn(0x7f44480fb5e0 msgr2=0x7f44480fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:08.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.962+0000 7f444d9ff700 1 -- 172.21.15.101:0/3685177226 shutdown_connections 2023-07-18T03:40:08.959 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.962+0000 7f444d9ff700 1 -- 172.21.15.101:0/3685177226 wait complete. 2023-07-18T03:40:08.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.963+0000 7f444d9ff700 1 Processor -- start 2023-07-18T03:40:08.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.963+0000 7f444d9ff700 1 -- start start 2023-07-18T03:40:08.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.964+0000 7f444d9ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44480728b0 0x7f4448197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:08.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.964+0000 7f444d9ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f44480731c0 0x7f4448198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:08.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.964+0000 7f444d9ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f444810c320 0x7f444819c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:08.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.964+0000 7f444d9ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444819cf30 con 0x7f44480731c0 2023-07-18T03:40:08.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.964+0000 7f444d9ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444819d0a0 con 0x7f44480728b0 2023-07-18T03:40:08.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.964+0000 7f444d9ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444819d260 con 0x7f444810c320 2023-07-18T03:40:08.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.964+0000 7f44467fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f44480731c0 0x7f4448198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:08.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.964+0000 7f44467fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f44480731c0 0x7f4448198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:59214/0 (socket says 172.21.15.101:59214) 2023-07-18T03:40:08.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.964+0000 7f44467fc700 1 -- 172.21.15.101:0/1998690551 learned_addr learned my addr 172.21.15.101:0/1998690551 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:40:08.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.964+0000 7f4446ffd700 1 --2- 172.21.15.101:0/1998690551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44480728b0 0x7f4448197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:08.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.964+0000 7f44477fe700 1 --2- 172.21.15.101:0/1998690551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f444810c320 0x7f444819c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:08.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.964+0000 7f44467fc700 1 -- 172.21.15.101:0/1998690551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f444810c320 msgr2=0x7f444819c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:08.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.964+0000 7f44467fc700 1 --2- 172.21.15.101:0/1998690551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f444810c320 0x7f444819c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:08.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.964+0000 7f44467fc700 1 -- 172.21.15.101:0/1998690551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44480728b0 msgr2=0x7f4448197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:08.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.964+0000 7f44467fc700 1 --2- 172.21.15.101:0/1998690551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44480728b0 0x7f4448197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:08.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.964+0000 7f44467fc700 1 -- 172.21.15.101:0/1998690551 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4440009d00 con 0x7f44480731c0 2023-07-18T03:40:08.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.965+0000 7f44467fc700 1 --2- 172.21.15.101:0/1998690551 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f44480731c0 0x7f4448198490 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f443c00e9a0 tx=0x7f443c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:08.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.965+0000 7f444c9fd700 1 -- 172.21.15.101:0/1998690551 <== mon.0 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443c009790 con 0x7f44480731c0 2023-07-18T03:40:08.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.965+0000 7f444d9ff700 1 -- 172.21.15.101:0/1998690551 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f444819d540 con 0x7f44480731c0 2023-07-18T03:40:08.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.965+0000 7f444c9fd700 1 -- 172.21.15.101:0/1998690551 <== mon.0 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f443c010430 con 0x7f44480731c0 2023-07-18T03:40:08.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.965+0000 7f444c9fd700 1 -- 172.21.15.101:0/1998690551 <== mon.0 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443c018680 con 0x7f44480731c0 2023-07-18T03:40:08.966 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.965+0000 7f444d9ff700 1 -- 172.21.15.101:0/1998690551 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44481a4d80 con 0x7f44480731c0 2023-07-18T03:40:08.967 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.966+0000 7f444d9ff700 1 -- 172.21.15.101:0/1998690551 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4448067290 con 0x7f44480731c0 2023-07-18T03:40:08.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.968+0000 7f444c9fd700 1 -- 172.21.15.101:0/1998690551 <== mon.0 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7f443c0105a0 con 0x7f44480731c0 2023-07-18T03:40:08.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.970+0000 7f444c9fd700 1 --2- 172.21.15.101:0/1998690551 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f44300a1340 0x7f44300a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:08.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.970+0000 7f444c9fd700 1 -- 172.21.15.101:0/1998690551 <== mon.0 v2:172.21.15.101:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 9719+0+0 (secure 0 0 0) 0x7f443c014070 con 0x7f44480731c0 2023-07-18T03:40:08.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.970+0000 7f4446ffd700 1 --2- 172.21.15.101:0/1998690551 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f44300a1340 0x7f44300a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:08.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.971+0000 7f4446ffd700 1 --2- 172.21.15.101:0/1998690551 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f44300a1340 0x7f44300a37f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4438009c60 tx=0x7f4438009260 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:08.969 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:08.975+0000 7f444c9fd700 1 -- 172.21.15.101:0/1998690551 <== mon.0 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f443c051790 con 0x7f44480731c0 2023-07-18T03:40:09.170 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:09.175+0000 7f444d9ff700 1 -- 172.21.15.101:0/1998690551 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f44481a5040 con 0x7f44480731c0 2023-07-18T03:40:09.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:09.741+0000 7f444c9fd700 1 -- 172.21.15.101:0/1998690551 <== mon.0 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v147) v1 ==== 181+0+0 (secure 0 0 0) 0x7f443c099ce0 con 0x7f44480731c0 2023-07-18T03:40:09.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:09.745+0000 7f444d9ff700 1 -- 172.21.15.101:0/1998690551 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f44300a1340 msgr2=0x7f44300a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:09.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:09.745+0000 7f444d9ff700 1 --2- 172.21.15.101:0/1998690551 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f44300a1340 0x7f44300a37f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4438009c60 tx=0x7f4438009260 comp rx=0 tx=0).stop 2023-07-18T03:40:09.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:09.745+0000 7f444d9ff700 1 -- 172.21.15.101:0/1998690551 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f44480731c0 msgr2=0x7f4448198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:09.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:09.745+0000 7f444d9ff700 1 --2- 172.21.15.101:0/1998690551 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f44480731c0 0x7f4448198490 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f443c00e9a0 tx=0x7f443c00ecb0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:09.745+0000 7f444d9ff700 1 -- 172.21.15.101:0/1998690551 shutdown_connections 2023-07-18T03:40:09.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:09.745+0000 7f444d9ff700 1 --2- 172.21.15.101:0/1998690551 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7f44300a1340 0x7f44300a37f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:09.745+0000 7f444d9ff700 1 --2- 172.21.15.101:0/1998690551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44480728b0 0x7f4448197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:09.745+0000 7f444d9ff700 1 --2- 172.21.15.101:0/1998690551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f444810c320 0x7f444819c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:09.745+0000 7f444d9ff700 1 --2- 172.21.15.101:0/1998690551 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f44480731c0 0x7f4448198490 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:09.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:09.745+0000 7f444d9ff700 1 -- 172.21.15.101:0/1998690551 >> 172.21.15.101:0/1998690551 conn(0x7f44480fb5e0 msgr2=0x7f444810e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:09.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:09.745+0000 7f444d9ff700 1 -- 172.21.15.101:0/1998690551 shutdown_connections 2023-07-18T03:40:09.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:09.745+0000 7f444d9ff700 1 -- 172.21.15.101:0/1998690551 wait complete. 2023-07-18T03:40:09.743 INFO:teuthology.orchestra.run.smithi101.stderr:pool 'cephfs_metadata' removed 2023-07-18T03:40:09.762 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-18T03:40:09.796 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:09 smithi155 ceph-mon[128885]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-07-18T03:40:09.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:09 smithi155 ceph-mon[128885]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-07-18T03:40:09.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:09 smithi155 ceph-mon[128885]: Cluster is now healthy 2023-07-18T03:40:09.797 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:09 smithi155 ceph-mon[128885]: mds.? [v2:172.21.15.155:6834/2248463834,v1:172.21.15.155:6838/2248463834] up:boot 2023-07-18T03:40:09.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:09 smithi155 ceph-mon[128885]: mds.? [v2:172.21.15.155:6835/801256482,v1:172.21.15.155:6839/801256482] up:boot 2023-07-18T03:40:09.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:09 smithi155 ceph-mon[128885]: mds.? [v2:172.21.15.155:6836/3259339572,v1:172.21.15.155:6837/3259339572] up:boot 2023-07-18T03:40:09.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:09 smithi155 ceph-mon[128885]: mds.? [v2:172.21.15.179:6836/1301340146,v1:172.21.15.179:6838/1301340146] up:boot 2023-07-18T03:40:09.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:09 smithi155 ceph-mon[128885]: mds.? [v2:172.21.15.179:6835/2955018898,v1:172.21.15.179:6839/2955018898] up:boot 2023-07-18T03:40:09.798 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:09 smithi155 ceph-mon[128885]: mds.? [v2:172.21.15.179:6834/981012902,v1:172.21.15.179:6837/981012902] up:boot 2023-07-18T03:40:09.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:09 smithi155 ceph-mon[128885]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-07-18T03:40:09.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:09 smithi155 ceph-mon[128885]: fsmap 12 up:standby 2023-07-18T03:40:09.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:09 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-18T03:40:09.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:09 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-18T03:40:09.799 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:09 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-18T03:40:09.800 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:09 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-18T03:40:09.800 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:09 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-18T03:40:09.800 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:09 smithi155 ceph-mon[128885]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-18T03:40:09.800 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:09 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1998690551' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-18T03:40:09.831 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:09 smithi101 ceph-mon[126379]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-07-18T03:40:09.832 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:09 smithi101 ceph-mon[126379]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-07-18T03:40:09.832 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:09 smithi101 ceph-mon[126379]: Cluster is now healthy 2023-07-18T03:40:09.832 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:09 smithi101 ceph-mon[126379]: mds.? [v2:172.21.15.155:6834/2248463834,v1:172.21.15.155:6838/2248463834] up:boot 2023-07-18T03:40:09.833 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:09 smithi101 ceph-mon[126379]: mds.? [v2:172.21.15.155:6835/801256482,v1:172.21.15.155:6839/801256482] up:boot 2023-07-18T03:40:09.833 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:09 smithi101 ceph-mon[126379]: mds.? [v2:172.21.15.155:6836/3259339572,v1:172.21.15.155:6837/3259339572] up:boot 2023-07-18T03:40:09.833 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:09 smithi101 ceph-mon[126379]: mds.? [v2:172.21.15.179:6836/1301340146,v1:172.21.15.179:6838/1301340146] up:boot 2023-07-18T03:40:09.833 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:09 smithi101 ceph-mon[126379]: mds.? [v2:172.21.15.179:6835/2955018898,v1:172.21.15.179:6839/2955018898] up:boot 2023-07-18T03:40:09.833 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:09 smithi101 ceph-mon[126379]: mds.? [v2:172.21.15.179:6834/981012902,v1:172.21.15.179:6837/981012902] up:boot 2023-07-18T03:40:09.834 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:09 smithi101 ceph-mon[126379]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-07-18T03:40:09.834 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:09 smithi101 ceph-mon[126379]: fsmap 12 up:standby 2023-07-18T03:40:09.834 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:09 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-18T03:40:09.834 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:09 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-18T03:40:09.834 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:09 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-18T03:40:09.835 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:09 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-18T03:40:09.835 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:09 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-18T03:40:09.835 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:09 smithi101 ceph-mon[126379]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-18T03:40:09.836 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:09 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1998690551' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-18T03:40:09.951 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:09 smithi179 ceph-mon[129229]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-07-18T03:40:09.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:09 smithi179 ceph-mon[129229]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-07-18T03:40:09.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:09 smithi179 ceph-mon[129229]: Cluster is now healthy 2023-07-18T03:40:09.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:09 smithi179 ceph-mon[129229]: mds.? [v2:172.21.15.155:6834/2248463834,v1:172.21.15.155:6838/2248463834] up:boot 2023-07-18T03:40:09.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:09 smithi179 ceph-mon[129229]: mds.? [v2:172.21.15.155:6835/801256482,v1:172.21.15.155:6839/801256482] up:boot 2023-07-18T03:40:09.952 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:09 smithi179 ceph-mon[129229]: mds.? [v2:172.21.15.155:6836/3259339572,v1:172.21.15.155:6837/3259339572] up:boot 2023-07-18T03:40:09.953 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:09 smithi179 ceph-mon[129229]: mds.? [v2:172.21.15.179:6836/1301340146,v1:172.21.15.179:6838/1301340146] up:boot 2023-07-18T03:40:09.953 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:09 smithi179 ceph-mon[129229]: mds.? [v2:172.21.15.179:6835/2955018898,v1:172.21.15.179:6839/2955018898] up:boot 2023-07-18T03:40:09.953 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:09 smithi179 ceph-mon[129229]: mds.? [v2:172.21.15.179:6834/981012902,v1:172.21.15.179:6837/981012902] up:boot 2023-07-18T03:40:09.953 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:09 smithi179 ceph-mon[129229]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-07-18T03:40:09.954 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:09 smithi179 ceph-mon[129229]: fsmap 12 up:standby 2023-07-18T03:40:09.954 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:09 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-18T03:40:09.954 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:09 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-18T03:40:09.954 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:09 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-18T03:40:09.954 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:09 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-18T03:40:09.955 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:09 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-18T03:40:09.955 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:09 smithi179 ceph-mon[129229]: from='mgr.15486 172.21.15.101:0/4185835670' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-18T03:40:09.955 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:09 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1998690551' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-18T03:40:10.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.036+0000 7fd4f99ff700 1 -- 172.21.15.101:0/1687853592 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4f4100480 msgr2=0x7fd4f410c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:10.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.036+0000 7fd4f99ff700 1 --2- 172.21.15.101:0/1687853592 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4f4100480 0x7fd4f410c910 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec009f90 tx=0x7fd4ec00b330 comp rx=0 tx=0).stop 2023-07-18T03:40:10.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.036+0000 7fd4f99ff700 1 -- 172.21.15.101:0/1687853592 shutdown_connections 2023-07-18T03:40:10.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.036+0000 7fd4f99ff700 1 --2- 172.21.15.101:0/1687853592 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4f4100480 0x7fd4f410c910 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:10.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.036+0000 7fd4f99ff700 1 --2- 172.21.15.101:0/1687853592 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4f4105ae0 0x7fd4f4105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:10.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.036+0000 7fd4f99ff700 1 --2- 172.21.15.101:0/1687853592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4f40ffad0 0x7fd4f40fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:10.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.036+0000 7fd4f99ff700 1 -- 172.21.15.101:0/1687853592 >> 172.21.15.101:0/1687853592 conn(0x7fd4f40fb5f0 msgr2=0x7fd4f40fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:10.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.037+0000 7fd4f99ff700 1 -- 172.21.15.101:0/1687853592 shutdown_connections 2023-07-18T03:40:10.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.037+0000 7fd4f99ff700 1 -- 172.21.15.101:0/1687853592 wait complete. 2023-07-18T03:40:10.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.038+0000 7fd4f99ff700 1 Processor -- start 2023-07-18T03:40:10.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.038+0000 7fd4f99ff700 1 -- start start 2023-07-18T03:40:10.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.038+0000 7fd4f99ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4f40ffad0 0x7fd4f410c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:10.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.038+0000 7fd4f99ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4f4100480 0x7fd4f4109650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:10.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.038+0000 7fd4f99ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4f4105ae0 0x7fd4f4109b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:10.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.038+0000 7fd4f99ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f410a270 con 0x7fd4f4105ae0 2023-07-18T03:40:10.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.039+0000 7fd4f99ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f410a3e0 con 0x7fd4f4100480 2023-07-18T03:40:10.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.039+0000 7fd4f99ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f4070f80 con 0x7fd4f40ffad0 2023-07-18T03:40:10.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.039+0000 7fd4f2ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4f40ffad0 0x7fd4f410c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:10.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.039+0000 7fd4f2ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4f40ffad0 0x7fd4f410c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:46240/0 (socket says 172.21.15.101:46240) 2023-07-18T03:40:10.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.039+0000 7fd4f2ffd700 1 -- 172.21.15.101:0/3484785390 learned_addr learned my addr 172.21.15.101:0/3484785390 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:40:10.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.039+0000 7fd4f27fc700 1 --2- 172.21.15.101:0/3484785390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4f4100480 0x7fd4f4109650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:10.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.039+0000 7fd4f37fe700 1 --2- 172.21.15.101:0/3484785390 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4f4105ae0 0x7fd4f4109b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:10.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.040+0000 7fd4f27fc700 1 -- 172.21.15.101:0/3484785390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4f40ffad0 msgr2=0x7fd4f410c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:10.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.040+0000 7fd4f27fc700 1 --2- 172.21.15.101:0/3484785390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4f40ffad0 0x7fd4f410c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:10.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.040+0000 7fd4f27fc700 1 -- 172.21.15.101:0/3484785390 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4f4105ae0 msgr2=0x7fd4f4109b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:10.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.040+0000 7fd4f27fc700 1 --2- 172.21.15.101:0/3484785390 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4f4105ae0 0x7fd4f4109b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:10.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.040+0000 7fd4f27fc700 1 -- 172.21.15.101:0/3484785390 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ec009c40 con 0x7fd4f4100480 2023-07-18T03:40:10.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.040+0000 7fd4f37fe700 1 --2- 172.21.15.101:0/3484785390 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4f4105ae0 0x7fd4f4109b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:40:10.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.040+0000 7fd4f27fc700 1 --2- 172.21.15.101:0/3484785390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4f4100480 0x7fd4f4109650 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400e9c0 tx=0x7fd4e400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:10.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.040+0000 7fd4f89fd700 1 -- 172.21.15.101:0/3484785390 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e400c9c0 con 0x7fd4f4100480 2023-07-18T03:40:10.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.040+0000 7fd4f99ff700 1 -- 172.21.15.101:0/3484785390 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4f4071260 con 0x7fd4f4100480 2023-07-18T03:40:10.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.040+0000 7fd4f99ff700 1 -- 172.21.15.101:0/3484785390 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4f40717c0 con 0x7fd4f4100480 2023-07-18T03:40:10.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.040+0000 7fd4f89fd700 1 -- 172.21.15.101:0/3484785390 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4e4010430 con 0x7fd4f4100480 2023-07-18T03:40:10.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.040+0000 7fd4f89fd700 1 -- 172.21.15.101:0/3484785390 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4018680 con 0x7fd4f4100480 2023-07-18T03:40:10.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.042+0000 7fd4f99ff700 1 -- 172.21.15.101:0/3484785390 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4d4005320 con 0x7fd4f4100480 2023-07-18T03:40:10.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.045+0000 7fd4f89fd700 1 -- 172.21.15.101:0/3484785390 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7fd4e400cb20 con 0x7fd4f4100480 2023-07-18T03:40:10.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.048+0000 7fd4f89fd700 1 --2- 172.21.15.101:0/3484785390 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd4e00a1340 0x7fd4e00a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:10.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.048+0000 7fd4f89fd700 1 -- 172.21.15.101:0/3484785390 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fd4e4014070 con 0x7fd4f4100480 2023-07-18T03:40:10.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.048+0000 7fd4f89fd700 1 -- 172.21.15.101:0/3484785390 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd4e40cd1b0 con 0x7fd4f4100480 2023-07-18T03:40:10.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.048+0000 7fd4f2ffd700 1 --2- 172.21.15.101:0/3484785390 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd4e00a1340 0x7fd4e00a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:10.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.049+0000 7fd4f2ffd700 1 --2- 172.21.15.101:0/3484785390 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd4e00a1340 0x7fd4e00a37f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc009ca0 tx=0x7fd4dc009260 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:10.280 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.286+0000 7fd4f99ff700 1 -- 172.21.15.101:0/3484785390 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd4d4005190 con 0x7fd4f4100480 2023-07-18T03:40:10.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.748+0000 7fd4f89fd700 1 -- 172.21.15.101:0/3484785390 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v148) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd4e4096060 con 0x7fd4f4100480 2023-07-18T03:40:10.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.752+0000 7fd4f99ff700 1 -- 172.21.15.101:0/3484785390 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd4e00a1340 msgr2=0x7fd4e00a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:10.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.752+0000 7fd4f99ff700 1 --2- 172.21.15.101:0/3484785390 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd4e00a1340 0x7fd4e00a37f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc009ca0 tx=0x7fd4dc009260 comp rx=0 tx=0).stop 2023-07-18T03:40:10.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.752+0000 7fd4f99ff700 1 -- 172.21.15.101:0/3484785390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4f4100480 msgr2=0x7fd4f4109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:10.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.752+0000 7fd4f99ff700 1 --2- 172.21.15.101:0/3484785390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4f4100480 0x7fd4f4109650 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400e9c0 tx=0x7fd4e400ecd0 comp rx=0 tx=0).stop 2023-07-18T03:40:10.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.752+0000 7fd4f99ff700 1 -- 172.21.15.101:0/3484785390 shutdown_connections 2023-07-18T03:40:10.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.752+0000 7fd4f99ff700 1 --2- 172.21.15.101:0/3484785390 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4f4105ae0 0x7fd4f4109b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:10.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.752+0000 7fd4f99ff700 1 --2- 172.21.15.101:0/3484785390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4f40ffad0 0x7fd4f410c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:10.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.752+0000 7fd4f99ff700 1 --2- 172.21.15.101:0/3484785390 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7fd4e00a1340 0x7fd4e00a37f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:10.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.752+0000 7fd4f99ff700 1 --2- 172.21.15.101:0/3484785390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4f4100480 0x7fd4f4109650 secure :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400e9c0 tx=0x7fd4e400ecd0 comp rx=0 tx=0).stop 2023-07-18T03:40:10.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.752+0000 7fd4f99ff700 1 -- 172.21.15.101:0/3484785390 >> 172.21.15.101:0/3484785390 conn(0x7fd4f40fb5f0 msgr2=0x7fd4f40fbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:10.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.753+0000 7fd4f99ff700 1 -- 172.21.15.101:0/3484785390 shutdown_connections 2023-07-18T03:40:10.750 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:10.753+0000 7fd4f99ff700 1 -- 172.21.15.101:0/3484785390 wait complete. 2023-07-18T03:40:10.750 INFO:teuthology.orchestra.run.smithi101.stderr:pool 'cephfs_data' removed 2023-07-18T03:40:10.776 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-07-18T03:40:10.988 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:10 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/1998690551' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-07-18T03:40:10.989 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:10 smithi101 ceph-mon[126379]: osdmap e147: 12 total, 12 up, 12 in 2023-07-18T03:40:10.989 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:10 smithi101 ceph-mon[126379]: pgmap v1036: 97 pgs: 97 active+clean; 211 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:40:10.989 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:10 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/3484785390' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-18T03:40:10.990 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:10 smithi101 ceph-mon[126379]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-18T03:40:11.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.037+0000 7faf071ff700 1 -- 172.21.15.101:0/2131845314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf000731c0 msgr2=0x7faf00110570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:11.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.037+0000 7faf071ff700 1 --2- 172.21.15.101:0/2131845314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf000731c0 0x7faf00110570 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7faef001a9a0 tx=0x7faef001acb0 comp rx=0 tx=0).stop 2023-07-18T03:40:11.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.037+0000 7faf071ff700 1 -- 172.21.15.101:0/2131845314 shutdown_connections 2023-07-18T03:40:11.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.037+0000 7faf071ff700 1 --2- 172.21.15.101:0/2131845314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf00110ab0 0x7faf00112e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:11.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.037+0000 7faf071ff700 1 --2- 172.21.15.101:0/2131845314 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf000728b0 0x7faf00072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:11.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.037+0000 7faf071ff700 1 --2- 172.21.15.101:0/2131845314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf000731c0 0x7faf00110570 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:11.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.037+0000 7faf071ff700 1 -- 172.21.15.101:0/2131845314 >> 172.21.15.101:0/2131845314 conn(0x7faf000fb5a0 msgr2=0x7faf000fd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:11.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.037+0000 7faf071ff700 1 -- 172.21.15.101:0/2131845314 shutdown_connections 2023-07-18T03:40:11.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.037+0000 7faf071ff700 1 -- 172.21.15.101:0/2131845314 wait complete. 2023-07-18T03:40:11.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.039+0000 7faf071ff700 1 Processor -- start 2023-07-18T03:40:11.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.039+0000 7faf071ff700 1 -- start start 2023-07-18T03:40:11.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.039+0000 7faf071ff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf000728b0 0x7faf0010be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:11.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.039+0000 7faf071ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf000731c0 0x7faf0010c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:11.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.039+0000 7faf071ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf00110ab0 0x7faf0019f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:11.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.039+0000 7faf071ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0019fcc0 con 0x7faf000731c0 2023-07-18T03:40:11.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.039+0000 7faf071ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0019fe00 con 0x7faf00110ab0 2023-07-18T03:40:11.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.039+0000 7faf071ff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0019ffc0 con 0x7faf000728b0 2023-07-18T03:40:11.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.040+0000 7faf04f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf000728b0 0x7faf0010be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:11.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.040+0000 7faf04f9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf000728b0 0x7faf0010be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.101:46244/0 (socket says 172.21.15.101:46244) 2023-07-18T03:40:11.042 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.040+0000 7faf04f9b700 1 -- 172.21.15.101:0/2192284351 learned_addr learned my addr 172.21.15.101:0/2192284351 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-18T03:40:11.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.040+0000 7faef7fff700 1 --2- 172.21.15.101:0/2192284351 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf000731c0 0x7faf0010c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:11.043 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.040+0000 7faf0579c700 1 --2- 172.21.15.101:0/2192284351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf00110ab0 0x7faf0019f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:11.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.040+0000 7faf04f9b700 1 -- 172.21.15.101:0/2192284351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf00110ab0 msgr2=0x7faf0019f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:11.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.040+0000 7faf04f9b700 1 --2- 172.21.15.101:0/2192284351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf00110ab0 0x7faf0019f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:11.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.040+0000 7faf04f9b700 1 -- 172.21.15.101:0/2192284351 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf000731c0 msgr2=0x7faf0010c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:11.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.040+0000 7faf04f9b700 1 --2- 172.21.15.101:0/2192284351 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf000731c0 0x7faf0010c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:11.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.040+0000 7faf04f9b700 1 -- 172.21.15.101:0/2192284351 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef001a600 con 0x7faf000728b0 2023-07-18T03:40:11.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.041+0000 7faf04f9b700 1 --2- 172.21.15.101:0/2192284351 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf000728b0 0x7faf0010be10 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7faefc00b560 tx=0x7faefc00b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:11.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.041+0000 7faef5ffb700 1 -- 172.21.15.101:0/2192284351 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc011830 con 0x7faf000728b0 2023-07-18T03:40:11.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.041+0000 7faf071ff700 1 -- 172.21.15.101:0/2192284351 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf001a02a0 con 0x7faf000728b0 2023-07-18T03:40:11.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.041+0000 7faf071ff700 1 -- 172.21.15.101:0/2192284351 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf001a0800 con 0x7faf000728b0 2023-07-18T03:40:11.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.041+0000 7faef5ffb700 1 -- 172.21.15.101:0/2192284351 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faefc011e50 con 0x7faf000728b0 2023-07-18T03:40:11.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.041+0000 7faef5ffb700 1 -- 172.21.15.101:0/2192284351 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc010e50 con 0x7faf000728b0 2023-07-18T03:40:11.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.042+0000 7faf071ff700 1 -- 172.21.15.101:0/2192284351 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeec005320 con 0x7faf000728b0 2023-07-18T03:40:11.049 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:10 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/1998690551' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-07-18T03:40:11.050 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:10 smithi155 ceph-mon[128885]: osdmap e147: 12 total, 12 up, 12 in 2023-07-18T03:40:11.050 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:10 smithi155 ceph-mon[128885]: pgmap v1036: 97 pgs: 97 active+clean; 211 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:40:11.050 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:10 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/3484785390' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-18T03:40:11.051 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:10 smithi155 ceph-mon[128885]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-18T03:40:11.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.045+0000 7faef5ffb700 1 -- 172.21.15.101:0/2192284351 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) v1 ==== 135847+0+0 (secure 0 0 0) 0x7faefc011990 con 0x7faf000728b0 2023-07-18T03:40:11.052 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.048+0000 7faef5ffb700 1 --2- 172.21.15.101:0/2192284351 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7faee80a1340 0x7faee80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:11.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.048+0000 7faef5ffb700 1 -- 172.21.15.101:0/2192284351 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 8598+0+0 (secure 0 0 0) 0x7faefc0cca90 con 0x7faf000728b0 2023-07-18T03:40:11.053 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.048+0000 7faef7fff700 1 --2- 172.21.15.101:0/2192284351 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7faee80a1340 0x7faee80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:11.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.049+0000 7faef7fff700 1 --2- 172.21.15.101:0/2192284351 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7faee80a1340 0x7faee80a37f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7faf00110a80 tx=0x7faef002a040 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:11.054 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.052+0000 7faef5ffb700 1 -- 172.21.15.101:0/2192284351 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7faefc095f00 con 0x7faf000728b0 2023-07-18T03:40:11.201 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:10 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/1998690551' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-07-18T03:40:11.201 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:10 smithi179 ceph-mon[129229]: osdmap e147: 12 total, 12 up, 12 in 2023-07-18T03:40:11.202 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:10 smithi179 ceph-mon[129229]: pgmap v1036: 97 pgs: 97 active+clean; 211 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:40:11.202 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:10 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/3484785390' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-18T03:40:11.202 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:10 smithi179 ceph-mon[129229]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-18T03:40:11.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.271+0000 7faf071ff700 1 -- 172.21.15.101:0/2192284351 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7faeec0059f0 con 0x7faf000728b0 2023-07-18T03:40:11.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.762+0000 7faef5ffb700 1 -- 172.21.15.101:0/2192284351 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v149) v1 ==== 178+0+0 (secure 0 0 0) 0x7faefc015020 con 0x7faf000728b0 2023-07-18T03:40:11.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.767+0000 7faf071ff700 1 -- 172.21.15.101:0/2192284351 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7faee80a1340 msgr2=0x7faee80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:11.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.767+0000 7faf071ff700 1 --2- 172.21.15.101:0/2192284351 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7faee80a1340 0x7faee80a37f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7faf00110a80 tx=0x7faef002a040 comp rx=0 tx=0).stop 2023-07-18T03:40:11.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.767+0000 7faf071ff700 1 -- 172.21.15.101:0/2192284351 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf000728b0 msgr2=0x7faf0010be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:11.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.767+0000 7faf071ff700 1 --2- 172.21.15.101:0/2192284351 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf000728b0 0x7faf0010be10 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7faefc00b560 tx=0x7faefc00b870 comp rx=0 tx=0).stop 2023-07-18T03:40:11.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.767+0000 7faf071ff700 1 -- 172.21.15.101:0/2192284351 shutdown_connections 2023-07-18T03:40:11.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.767+0000 7faf071ff700 1 --2- 172.21.15.101:0/2192284351 >> [v2:172.21.15.101:6800/486424816,v1:172.21.15.101:6801/486424816] conn(0x7faee80a1340 0x7faee80a37f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:11.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.767+0000 7faf071ff700 1 --2- 172.21.15.101:0/2192284351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf00110ab0 0x7faf0019f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:11.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.767+0000 7faf071ff700 1 --2- 172.21.15.101:0/2192284351 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf000728b0 0x7faf0010be10 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:11.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.767+0000 7faf071ff700 1 --2- 172.21.15.101:0/2192284351 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf000731c0 0x7faf0010c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:11.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.767+0000 7faf071ff700 1 -- 172.21.15.101:0/2192284351 >> 172.21.15.101:0/2192284351 conn(0x7faf000fb5a0 msgr2=0x7faf000fc200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:11.764 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.767+0000 7faf071ff700 1 -- 172.21.15.101:0/2192284351 shutdown_connections 2023-07-18T03:40:11.765 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-18T03:40:11.767+0000 7faf071ff700 1 -- 172.21.15.101:0/2192284351 wait complete. 2023-07-18T03:40:11.766 INFO:teuthology.orchestra.run.smithi101.stderr:pool 'cephfs_data_ec' removed 2023-07-18T03:40:11.787 DEBUG:teuthology.orchestra.run.smithi101:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-18T03:40:11.873 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-18T03:40:11.929 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-18T03:40:12.037 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:11 smithi179 ceph-mon[129229]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-07-18T03:40:12.038 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:11 smithi179 ceph-mon[129229]: osdmap e148: 12 total, 12 up, 12 in 2023-07-18T03:40:12.038 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:11 smithi179 ceph-mon[129229]: from='client.? 172.21.15.101:0/2192284351' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-18T03:40:12.038 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:11 smithi179 ceph-mon[129229]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-18T03:40:12.039 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-07-18T03:40:12.040 DEBUG:teuthology.orchestra.run.smithi101:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-07-18T03:40:12.046 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:11 smithi155 ceph-mon[128885]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-07-18T03:40:12.047 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:11 smithi155 ceph-mon[128885]: osdmap e148: 12 total, 12 up, 12 in 2023-07-18T03:40:12.047 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:11 smithi155 ceph-mon[128885]: from='client.? 172.21.15.101:0/2192284351' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-18T03:40:12.048 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:11 smithi155 ceph-mon[128885]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-18T03:40:12.084 DEBUG:teuthology.orchestra.run.smithi155:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-07-18T03:40:12.102 DEBUG:teuthology.orchestra.run.smithi179:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-07-18T03:40:12.119 INFO:tasks.cephadm:Stopping all daemons... 2023-07-18T03:40:12.120 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-07-18T03:40:12.120 DEBUG:teuthology.orchestra.run.smithi101:> sudo systemctl stop ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mon.a 2023-07-18T03:40:12.183 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:11 smithi101 ceph-mon[126379]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-07-18T03:40:12.183 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:11 smithi101 ceph-mon[126379]: osdmap e148: 12 total, 12 up, 12 in 2023-07-18T03:40:12.184 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:11 smithi101 ceph-mon[126379]: from='client.? 172.21.15.101:0/2192284351' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-18T03:40:12.184 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:11 smithi101 ceph-mon[126379]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-18T03:40:12.489 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:12 smithi101 systemd[1]: Stopping Ceph mon.a for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T03:40:12.977 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:12 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mon-a[126354]: 2023-07-18T03:40:12.649+0000 7f32b03a9700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:40:12.977 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:12 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mon-a[126354]: 2023-07-18T03:40:12.649+0000 7f32b03a9700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-07-18T03:40:13.239 INFO:journalctl@ceph.mon.a.smithi101.stdout:Jul 18 03:40:13 smithi101 bash[205353]: ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mon-a 2023-07-18T03:40:13.648 DEBUG:teuthology.orchestra.run.smithi101:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mon.a.service' 2023-07-18T03:40:13.745 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:40:13.746 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-07-18T03:40:13.747 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-07-18T03:40:13.747 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mon.b 2023-07-18T03:40:14.046 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:13 smithi155 systemd[1]: Stopping Ceph mon.b for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T03:40:14.546 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:14 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mon-b[128861]: 2023-07-18T03:40:14.208+0000 7ff025523700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:40:14.547 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:14 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mon-b[128861]: 2023-07-18T03:40:14.208+0000 7ff025523700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-07-18T03:40:14.840 INFO:journalctl@ceph.mon.b.smithi155.stdout:Jul 18 03:40:14 smithi155 bash[166583]: ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mon-b 2023-07-18T03:40:14.847 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mon.b.service' 2023-07-18T03:40:14.898 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:40:14.899 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-07-18T03:40:14.899 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-07-18T03:40:14.900 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mon.c 2023-07-18T03:40:15.202 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:14 smithi179 systemd[1]: Stopping Ceph mon.c for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T03:40:15.701 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:15 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mon-c[129201]: 2023-07-18T03:40:15.420+0000 7fd1b8778700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:40:15.702 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:15 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mon-c[129201]: 2023-07-18T03:40:15.420+0000 7fd1b8778700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-07-18T03:40:16.065 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jul 18 03:40:15 smithi179 bash[166874]: ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mon-c 2023-07-18T03:40:16.251 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mon.c.service' 2023-07-18T03:40:16.297 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:40:16.298 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-07-18T03:40:16.298 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-07-18T03:40:16.299 DEBUG:teuthology.orchestra.run.smithi101:> sudo systemctl stop ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mgr.x 2023-07-18T03:40:16.738 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:40:16 smithi101 systemd[1]: Stopping Ceph mgr.x for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T03:40:17.083 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:40:16 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:40:16.815+0000 7f023515c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:40:17.084 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:40:16 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-18T03:40:16.815+0000 7f023515c700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-07-18T03:40:17.488 INFO:journalctl@ceph.mgr.x.smithi101.stdout:Jul 18 03:40:17 smithi101 bash[205499]: ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-x 2023-07-18T03:40:17.611 DEBUG:teuthology.orchestra.run.smithi101:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mgr.x.service' 2023-07-18T03:40:17.697 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:40:17.698 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-07-18T03:40:17.699 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-07-18T03:40:17.699 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mgr.y 2023-07-18T03:40:18.046 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 03:40:17 smithi155 systemd[1]: Stopping Ceph mgr.y for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T03:40:18.405 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 03:40:18 smithi155 bash[166722]: ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-y 2023-07-18T03:40:18.405 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Jul 18 03:40:18 smithi155 systemd[1]: ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-07-18T03:40:18.588 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mgr.y.service' 2023-07-18T03:40:18.626 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:40:18.627 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-07-18T03:40:18.627 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-07-18T03:40:18.627 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mgr.z 2023-07-18T03:40:18.951 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 03:40:18 smithi179 systemd[1]: Stopping Ceph mgr.z for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T03:40:19.333 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 03:40:19 smithi179 bash[167024]: ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mgr-z 2023-07-18T03:40:19.516 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jul 18 03:40:19 smithi179 systemd[1]: ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-07-18T03:40:19.520 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mgr.z.service' 2023-07-18T03:40:19.558 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:40:19.559 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-07-18T03:40:19.559 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-07-18T03:40:19.560 DEBUG:teuthology.orchestra.run.smithi101:> sudo systemctl stop ceph-b1782daa-2516-11ee-9b34-001a4aab830c@osd.0 2023-07-18T03:40:19.990 INFO:journalctl@ceph.osd.0.smithi101.stdout:Jul 18 03:40:19 smithi101 systemd[1]: Stopping Ceph osd.0 for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T03:40:20.488 INFO:journalctl@ceph.osd.0.smithi101.stdout:Jul 18 03:40:20 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-0[139271]: 2023-07-18T03:40:20.073+0000 7f7b608ce700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:40:20.489 INFO:journalctl@ceph.osd.0.smithi101.stdout:Jul 18 03:40:20 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-0[139271]: 2023-07-18T03:40:20.073+0000 7f7b608ce700 -1 osd.0 149 *** Got signal Terminated *** 2023-07-18T03:40:20.489 INFO:journalctl@ceph.osd.0.smithi101.stdout:Jul 18 03:40:20 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-0[139271]: 2023-07-18T03:40:20.073+0000 7f7b608ce700 -1 osd.0 149 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-18T03:40:26.239 INFO:journalctl@ceph.osd.0.smithi101.stdout:Jul 18 03:40:25 smithi101 bash[205648]: ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-0 2023-07-18T03:40:27.950 DEBUG:teuthology.orchestra.run.smithi101:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@osd.0.service' 2023-07-18T03:40:28.039 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:40:28.040 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-07-18T03:40:28.040 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-07-18T03:40:28.040 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-b1782daa-2516-11ee-9b34-001a4aab830c@osd.1 2023-07-18T03:40:28.392 INFO:journalctl@ceph.osd.1.smithi155.stdout:Jul 18 03:40:28 smithi155 systemd[1]: Stopping Ceph osd.1 for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T03:40:28.796 INFO:journalctl@ceph.osd.1.smithi155.stdout:Jul 18 03:40:28 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-1[133984]: 2023-07-18T03:40:28.397+0000 7fc2d90d9700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:40:28.796 INFO:journalctl@ceph.osd.1.smithi155.stdout:Jul 18 03:40:28 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-1[133984]: 2023-07-18T03:40:28.397+0000 7fc2d90d9700 -1 osd.1 149 *** Got signal Terminated *** 2023-07-18T03:40:28.797 INFO:journalctl@ceph.osd.1.smithi155.stdout:Jul 18 03:40:28 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-1[133984]: 2023-07-18T03:40:28.397+0000 7fc2d90d9700 -1 osd.1 149 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-18T03:40:34.075 INFO:journalctl@ceph.osd.1.smithi155.stdout:Jul 18 03:40:34 smithi155 bash[166871]: ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-1 2023-07-18T03:40:35.540 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@osd.1.service' 2023-07-18T03:40:35.580 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:40:35.581 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-07-18T03:40:35.581 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-07-18T03:40:35.581 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-b1782daa-2516-11ee-9b34-001a4aab830c@osd.2 2023-07-18T03:40:35.945 INFO:journalctl@ceph.osd.2.smithi179.stdout:Jul 18 03:40:35 smithi179 systemd[1]: Stopping Ceph osd.2 for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T03:40:36.201 INFO:journalctl@ceph.osd.2.smithi179.stdout:Jul 18 03:40:35 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-2[135082]: 2023-07-18T03:40:35.951+0000 7f6b317b3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:40:36.201 INFO:journalctl@ceph.osd.2.smithi179.stdout:Jul 18 03:40:35 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-2[135082]: 2023-07-18T03:40:35.951+0000 7f6b317b3700 -1 osd.2 149 *** Got signal Terminated *** 2023-07-18T03:40:36.202 INFO:journalctl@ceph.osd.2.smithi179.stdout:Jul 18 03:40:35 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-2[135082]: 2023-07-18T03:40:35.951+0000 7f6b317b3700 -1 osd.2 149 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-18T03:40:41.701 INFO:journalctl@ceph.osd.2.smithi179.stdout:Jul 18 03:40:41 smithi179 bash[167170]: ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-2 2023-07-18T03:40:43.287 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@osd.2.service' 2023-07-18T03:40:43.327 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:40:43.328 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-07-18T03:40:43.329 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-07-18T03:40:43.329 DEBUG:teuthology.orchestra.run.smithi101:> sudo systemctl stop ceph-b1782daa-2516-11ee-9b34-001a4aab830c@osd.3 2023-07-18T03:40:43.739 INFO:journalctl@ceph.osd.3.smithi101.stdout:Jul 18 03:40:43 smithi101 systemd[1]: Stopping Ceph osd.3 for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T03:40:44.238 INFO:journalctl@ceph.osd.3.smithi101.stdout:Jul 18 03:40:43 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-3[144817]: 2023-07-18T03:40:43.844+0000 7fa39ba63700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:40:44.239 INFO:journalctl@ceph.osd.3.smithi101.stdout:Jul 18 03:40:43 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-3[144817]: 2023-07-18T03:40:43.844+0000 7fa39ba63700 -1 osd.3 149 *** Got signal Terminated *** 2023-07-18T03:40:44.239 INFO:journalctl@ceph.osd.3.smithi101.stdout:Jul 18 03:40:43 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-3[144817]: 2023-07-18T03:40:43.844+0000 7fa39ba63700 -1 osd.3 149 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-18T03:40:46.451 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jul 18 03:40:46 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-5[140498]: 2023-07-18T03:40:46.204+0000 7f6352bf5700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:21.251904+0000 front 2023-07-18T03:40:21.251757+0000 (oldest deadline 2023-07-18T03:40:45.951887+0000) 2023-07-18T03:40:47.239 INFO:journalctl@ceph.osd.3.smithi101.stdout:Jul 18 03:40:46 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-3[144817]: 2023-07-18T03:40:46.891+0000 7fa396055700 -1 osd.3 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:20.070321+0000 front 2023-07-18T03:40:20.070264+0000 (oldest deadline 2023-07-18T03:40:45.970541+0000) 2023-07-18T03:40:47.701 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jul 18 03:40:47 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-5[140498]: 2023-07-18T03:40:47.243+0000 7f6352bf5700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:21.251904+0000 front 2023-07-18T03:40:21.251757+0000 (oldest deadline 2023-07-18T03:40:45.951887+0000) 2023-07-18T03:40:47.796 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:40:47 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:40:47.375+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:40:48.240 INFO:journalctl@ceph.osd.3.smithi101.stdout:Jul 18 03:40:47 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-3[144817]: 2023-07-18T03:40:47.916+0000 7fa396055700 -1 osd.3 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:20.070321+0000 front 2023-07-18T03:40:20.070264+0000 (oldest deadline 2023-07-18T03:40:45.970541+0000) 2023-07-18T03:40:48.701 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jul 18 03:40:48 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-5[140498]: 2023-07-18T03:40:48.223+0000 7f6352bf5700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:21.251904+0000 front 2023-07-18T03:40:21.251757+0000 (oldest deadline 2023-07-18T03:40:45.951887+0000) 2023-07-18T03:40:48.796 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:40:48 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:40:48.350+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:40:48.988 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:40:48 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:40:48.649+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:40:49.451 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jul 18 03:40:49 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-5[140498]: 2023-07-18T03:40:49.183+0000 7f6352bf5700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:21.251904+0000 front 2023-07-18T03:40:21.251757+0000 (oldest deadline 2023-07-18T03:40:45.951887+0000) 2023-07-18T03:40:49.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:40:49 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:40:49.238+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:40:49.796 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:40:49 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:40:49.397+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:40:49.952 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:40:49 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:40:49.644+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.220319+0000 front 2023-07-18T03:40:24.220087+0000 (oldest deadline 2023-07-18T03:40:48.920267+0000) 2023-07-18T03:40:49.990 INFO:journalctl@ceph.osd.3.smithi101.stdout:Jul 18 03:40:49 smithi101 bash[205936]: ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-3 2023-07-18T03:40:49.991 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:40:49 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:40:49.620+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:40:50.452 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jul 18 03:40:50 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-5[140498]: 2023-07-18T03:40:50.200+0000 7f6352bf5700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:21.251904+0000 front 2023-07-18T03:40:21.251757+0000 (oldest deadline 2023-07-18T03:40:45.951887+0000) 2023-07-18T03:40:50.453 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:40:50 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:40:50.200+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:40:50.796 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:40:50 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:40:50.377+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:40:50.951 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:40:50 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:40:50.634+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.220319+0000 front 2023-07-18T03:40:24.220087+0000 (oldest deadline 2023-07-18T03:40:48.920267+0000) 2023-07-18T03:40:50.988 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:40:50 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:40:50.643+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:40:51.451 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jul 18 03:40:51 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-5[140498]: 2023-07-18T03:40:51.169+0000 7f6352bf5700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:21.251904+0000 front 2023-07-18T03:40:21.251757+0000 (oldest deadline 2023-07-18T03:40:45.951887+0000) 2023-07-18T03:40:51.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:40:51 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:40:51.223+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:40:51.570 DEBUG:teuthology.orchestra.run.smithi101:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@osd.3.service' 2023-07-18T03:40:51.660 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:40:51.660 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-07-18T03:40:51.661 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-07-18T03:40:51.661 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-b1782daa-2516-11ee-9b34-001a4aab830c@osd.4 2023-07-18T03:40:51.692 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:40:51 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:40:51.399+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:40:51.951 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:40:51 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:40:51.649+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.220319+0000 front 2023-07-18T03:40:24.220087+0000 (oldest deadline 2023-07-18T03:40:48.920267+0000) 2023-07-18T03:40:51.988 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:40:51 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:40:51.596+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:40:52.046 INFO:journalctl@ceph.osd.4.smithi155.stdout:Jul 18 03:40:51 smithi155 systemd[1]: Stopping Ceph osd.4 for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T03:40:52.346 INFO:journalctl@ceph.osd.4.smithi155.stdout:Jul 18 03:40:52 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-4[139452]: 2023-07-18T03:40:52.093+0000 7fbd9e124700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:40:52.347 INFO:journalctl@ceph.osd.4.smithi155.stdout:Jul 18 03:40:52 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-4[139452]: 2023-07-18T03:40:52.093+0000 7fbd9e124700 -1 osd.4 149 *** Got signal Terminated *** 2023-07-18T03:40:52.347 INFO:journalctl@ceph.osd.4.smithi155.stdout:Jul 18 03:40:52 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-4[139452]: 2023-07-18T03:40:52.093+0000 7fbd9e124700 -1 osd.4 149 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-18T03:40:52.451 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jul 18 03:40:52 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-5[140498]: 2023-07-18T03:40:52.152+0000 7f6352bf5700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:21.251904+0000 front 2023-07-18T03:40:21.251757+0000 (oldest deadline 2023-07-18T03:40:45.951887+0000) 2023-07-18T03:40:52.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:40:52 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:40:52.184+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:40:52.796 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:40:52 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:40:52.350+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:40:52.952 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:40:52 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:40:52.695+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.220319+0000 front 2023-07-18T03:40:24.220087+0000 (oldest deadline 2023-07-18T03:40:48.920267+0000) 2023-07-18T03:40:52.988 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:40:52 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:40:52.618+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:40:53.451 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jul 18 03:40:53 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-5[140498]: 2023-07-18T03:40:53.171+0000 7f6352bf5700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:21.251904+0000 front 2023-07-18T03:40:21.251757+0000 (oldest deadline 2023-07-18T03:40:45.951887+0000) 2023-07-18T03:40:53.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:40:53 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:40:53.203+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:40:53.796 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:40:53 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:40:53.366+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:40:53.988 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:40:53 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:40:53.569+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:40:54.175 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:40:53 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:40:53.717+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.220319+0000 front 2023-07-18T03:40:24.220087+0000 (oldest deadline 2023-07-18T03:40:48.920267+0000) 2023-07-18T03:40:54.451 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jul 18 03:40:54 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-5[140498]: 2023-07-18T03:40:54.181+0000 7f6352bf5700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:21.251904+0000 front 2023-07-18T03:40:21.251757+0000 (oldest deadline 2023-07-18T03:40:45.951887+0000) 2023-07-18T03:40:54.451 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jul 18 03:40:54 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-5[140498]: 2023-07-18T03:40:54.181+0000 7f6352bf5700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.253079+0000 front 2023-07-18T03:40:28.253102+0000 (oldest deadline 2023-07-18T03:40:54.153072+0000) 2023-07-18T03:40:54.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:40:54 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:40:54.226+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:40:54.797 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:40:54 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:40:54.318+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:40:54.798 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:40:54 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:40:54.318+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:40:54.951 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:40:54 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:40:54.682+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.220319+0000 front 2023-07-18T03:40:24.220087+0000 (oldest deadline 2023-07-18T03:40:48.920267+0000) 2023-07-18T03:40:54.988 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:40:54 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:40:54.526+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:40:55.701 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:40:55 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:40:55.266+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:40:55.701 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jul 18 03:40:55 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-5[140498]: 2023-07-18T03:40:55.222+0000 7f6352bf5700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:21.251904+0000 front 2023-07-18T03:40:21.251757+0000 (oldest deadline 2023-07-18T03:40:45.951887+0000) 2023-07-18T03:40:55.702 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jul 18 03:40:55 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-5[140498]: 2023-07-18T03:40:55.222+0000 7f6352bf5700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.253079+0000 front 2023-07-18T03:40:28.253102+0000 (oldest deadline 2023-07-18T03:40:54.153072+0000) 2023-07-18T03:40:55.738 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:40:55 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:40:55.486+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:40:55.796 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:40:55 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:40:55.386+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:30.409213+0000 front 2023-07-18T03:40:30.409292+0000 (oldest deadline 2023-07-18T03:40:55.109366+0000) 2023-07-18T03:40:55.796 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:40:55 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:40:55.311+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:40:55.797 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:40:55 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:40:55.311+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:40:56.201 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:40:55 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:40:55.719+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.220319+0000 front 2023-07-18T03:40:24.220087+0000 (oldest deadline 2023-07-18T03:40:48.920267+0000) 2023-07-18T03:40:56.546 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:40:56 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:40:56.367+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:30.409213+0000 front 2023-07-18T03:40:30.409292+0000 (oldest deadline 2023-07-18T03:40:55.109366+0000) 2023-07-18T03:40:56.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:40:56 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:40:56.262+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:40:56.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:40:56 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:40:56.262+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:40:56.701 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:40:56 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:40:56.258+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:40:56.702 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jul 18 03:40:56 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-5[140498]: 2023-07-18T03:40:56.266+0000 7f6352bf5700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:21.251904+0000 front 2023-07-18T03:40:21.251757+0000 (oldest deadline 2023-07-18T03:40:45.951887+0000) 2023-07-18T03:40:56.702 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jul 18 03:40:56 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-5[140498]: 2023-07-18T03:40:56.266+0000 7f6352bf5700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.253079+0000 front 2023-07-18T03:40:28.253102+0000 (oldest deadline 2023-07-18T03:40:54.153072+0000) 2023-07-18T03:40:56.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:40:56 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:40:56.464+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:40:57.201 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:40:56 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:40:56.765+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.220319+0000 front 2023-07-18T03:40:24.220087+0000 (oldest deadline 2023-07-18T03:40:48.920267+0000) 2023-07-18T03:40:57.546 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:40:57 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:40:57.370+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:30.409213+0000 front 2023-07-18T03:40:30.409292+0000 (oldest deadline 2023-07-18T03:40:55.109366+0000) 2023-07-18T03:40:57.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:40:57 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:40:57.232+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:40:57.548 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:40:57 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:40:57.232+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:40:57.701 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:40:57 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:40:57.221+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:40:57.702 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:40:57 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:40:57.221+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:40:57.702 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jul 18 03:40:57 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-5[140498]: 2023-07-18T03:40:57.262+0000 7f6352bf5700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:21.251904+0000 front 2023-07-18T03:40:21.251757+0000 (oldest deadline 2023-07-18T03:40:45.951887+0000) 2023-07-18T03:40:57.702 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jul 18 03:40:57 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-5[140498]: 2023-07-18T03:40:57.262+0000 7f6352bf5700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.253079+0000 front 2023-07-18T03:40:28.253102+0000 (oldest deadline 2023-07-18T03:40:54.153072+0000) 2023-07-18T03:40:57.738 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:40:57 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:40:57.418+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:40:57.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:40:57 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:40:57.418+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.543459+0000 front 2023-07-18T03:40:32.543608+0000 (oldest deadline 2023-07-18T03:40:56.643373+0000) 2023-07-18T03:40:57.904 INFO:journalctl@ceph.osd.4.smithi155.stdout:Jul 18 03:40:57 smithi155 bash[167152]: ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-4 2023-07-18T03:40:58.201 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:40:57 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:40:57.763+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.220319+0000 front 2023-07-18T03:40:24.220087+0000 (oldest deadline 2023-07-18T03:40:48.920267+0000) 2023-07-18T03:40:58.546 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:40:58 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:40:58.338+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:30.409213+0000 front 2023-07-18T03:40:30.409292+0000 (oldest deadline 2023-07-18T03:40:55.109366+0000) 2023-07-18T03:40:58.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:40:58 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:40:58.220+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:40:58.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:40:58 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:40:58.220+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:40:58.700 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jul 18 03:40:58 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-5[140498]: 2023-07-18T03:40:58.216+0000 7f6352bf5700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:21.251904+0000 front 2023-07-18T03:40:21.251757+0000 (oldest deadline 2023-07-18T03:40:45.951887+0000) 2023-07-18T03:40:58.701 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jul 18 03:40:58 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-5[140498]: 2023-07-18T03:40:58.217+0000 7f6352bf5700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.253079+0000 front 2023-07-18T03:40:28.253102+0000 (oldest deadline 2023-07-18T03:40:54.153072+0000) 2023-07-18T03:40:58.701 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:40:58 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:40:58.245+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:40:58.702 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:40:58 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:40:58.245+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:40:58.738 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:40:58 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:40:58.381+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:40:58.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:40:58 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:40:58.381+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.543459+0000 front 2023-07-18T03:40:32.543608+0000 (oldest deadline 2023-07-18T03:40:56.643373+0000) 2023-07-18T03:40:59.164 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@osd.4.service' 2023-07-18T03:40:59.201 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:40:58 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:40:58.724+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.220319+0000 front 2023-07-18T03:40:24.220087+0000 (oldest deadline 2023-07-18T03:40:48.920267+0000) 2023-07-18T03:40:59.201 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:40:58 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:40:58.724+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.322385+0000 front 2023-07-18T03:40:32.322477+0000 (oldest deadline 2023-07-18T03:40:58.222425+0000) 2023-07-18T03:40:59.238 INFO:journalctl@ceph.osd.6.smithi101.stdout:Jul 18 03:40:58 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-6[150288]: 2023-07-18T03:40:58.824+0000 7f47cab67700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.198755+0000 front 2023-07-18T03:40:32.198734+0000 (oldest deadline 2023-07-18T03:40:58.098626+0000) 2023-07-18T03:40:59.246 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:40:59.247 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-07-18T03:40:59.248 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-07-18T03:40:59.248 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-b1782daa-2516-11ee-9b34-001a4aab830c@osd.5 2023-07-18T03:40:59.546 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:40:59 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:40:59.355+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:30.409213+0000 front 2023-07-18T03:40:30.409292+0000 (oldest deadline 2023-07-18T03:40:55.109366+0000) 2023-07-18T03:40:59.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:40:59 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:40:59.185+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:40:59.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:40:59 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:40:59.185+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:40:59.613 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:40:59 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:40:59.271+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:40:59.613 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:40:59 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:40:59.271+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:40:59.613 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jul 18 03:40:59 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-5[140498]: 2023-07-18T03:40:59.236+0000 7f6352bf5700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:21.251904+0000 front 2023-07-18T03:40:21.251757+0000 (oldest deadline 2023-07-18T03:40:45.951887+0000) 2023-07-18T03:40:59.614 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jul 18 03:40:59 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-5[140498]: 2023-07-18T03:40:59.236+0000 7f6352bf5700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.253079+0000 front 2023-07-18T03:40:28.253102+0000 (oldest deadline 2023-07-18T03:40:54.153072+0000) 2023-07-18T03:40:59.614 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jul 18 03:40:59 smithi179 systemd[1]: Stopping Ceph osd.5 for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T03:40:59.615 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jul 18 03:40:59 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-5[140498]: 2023-07-18T03:40:59.613+0000 7f63565ff700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:40:59.738 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:40:59 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:40:59.346+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:40:59.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:40:59 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:40:59.346+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.543459+0000 front 2023-07-18T03:40:32.543608+0000 (oldest deadline 2023-07-18T03:40:56.643373+0000) 2023-07-18T03:40:59.950 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jul 18 03:40:59 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-5[140498]: 2023-07-18T03:40:59.613+0000 7f63565ff700 -1 osd.5 149 *** Got signal Terminated *** 2023-07-18T03:40:59.951 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jul 18 03:40:59 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-5[140498]: 2023-07-18T03:40:59.613+0000 7f63565ff700 -1 osd.5 149 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-18T03:40:59.951 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:40:59 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:40:59.717+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.220319+0000 front 2023-07-18T03:40:24.220087+0000 (oldest deadline 2023-07-18T03:40:48.920267+0000) 2023-07-18T03:40:59.952 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:40:59 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:40:59.717+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.322385+0000 front 2023-07-18T03:40:32.322477+0000 (oldest deadline 2023-07-18T03:40:58.222425+0000) 2023-07-18T03:41:00.238 INFO:journalctl@ceph.osd.6.smithi101.stdout:Jul 18 03:40:59 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-6[150288]: 2023-07-18T03:40:59.801+0000 7f47cab67700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.198755+0000 front 2023-07-18T03:40:32.198734+0000 (oldest deadline 2023-07-18T03:40:58.098626+0000) 2023-07-18T03:41:00.546 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:00 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:00.328+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:30.409213+0000 front 2023-07-18T03:40:30.409292+0000 (oldest deadline 2023-07-18T03:40:55.109366+0000) 2023-07-18T03:41:00.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:00 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:00.143+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:41:00.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:00 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:00.143+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:41:00.701 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jul 18 03:41:00 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-5[140498]: 2023-07-18T03:41:00.207+0000 7f6352bf5700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:21.251904+0000 front 2023-07-18T03:40:21.251757+0000 (oldest deadline 2023-07-18T03:40:45.951887+0000) 2023-07-18T03:41:00.702 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jul 18 03:41:00 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-5[140498]: 2023-07-18T03:41:00.207+0000 7f6352bf5700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.253079+0000 front 2023-07-18T03:40:28.253102+0000 (oldest deadline 2023-07-18T03:40:54.153072+0000) 2023-07-18T03:41:00.703 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:00 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:00.314+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:00.703 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:00 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:00.314+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:00.738 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:00 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:00.327+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:41:00.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:00 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:00.327+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.543459+0000 front 2023-07-18T03:40:32.543608+0000 (oldest deadline 2023-07-18T03:40:56.643373+0000) 2023-07-18T03:41:01.195 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:00 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:00.710+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.220319+0000 front 2023-07-18T03:40:24.220087+0000 (oldest deadline 2023-07-18T03:40:48.920267+0000) 2023-07-18T03:41:01.196 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:00 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:00.710+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.322385+0000 front 2023-07-18T03:40:32.322477+0000 (oldest deadline 2023-07-18T03:40:58.222425+0000) 2023-07-18T03:41:01.238 INFO:journalctl@ceph.osd.6.smithi101.stdout:Jul 18 03:41:00 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-6[150288]: 2023-07-18T03:41:00.809+0000 7f47cab67700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.198755+0000 front 2023-07-18T03:40:32.198734+0000 (oldest deadline 2023-07-18T03:40:58.098626+0000) 2023-07-18T03:41:01.451 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jul 18 03:41:01 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-5[140498]: 2023-07-18T03:41:01.201+0000 7f6352bf5700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:21.251904+0000 front 2023-07-18T03:40:21.251757+0000 (oldest deadline 2023-07-18T03:40:45.951887+0000) 2023-07-18T03:41:01.452 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jul 18 03:41:01 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-5[140498]: 2023-07-18T03:41:01.201+0000 7f6352bf5700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.253079+0000 front 2023-07-18T03:40:28.253102+0000 (oldest deadline 2023-07-18T03:40:54.153072+0000) 2023-07-18T03:41:01.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:01 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:01.330+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:01.453 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:01 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:01.330+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:01.546 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:01 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:01.327+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:30.409213+0000 front 2023-07-18T03:40:30.409292+0000 (oldest deadline 2023-07-18T03:40:55.109366+0000) 2023-07-18T03:41:01.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:01 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:01.166+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:41:01.548 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:01 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:01.166+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:41:01.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:01 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:01.370+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:41:01.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:01 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:01.370+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.543459+0000 front 2023-07-18T03:40:32.543608+0000 (oldest deadline 2023-07-18T03:40:56.643373+0000) 2023-07-18T03:41:02.201 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:01 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:01.733+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.220319+0000 front 2023-07-18T03:40:24.220087+0000 (oldest deadline 2023-07-18T03:40:48.920267+0000) 2023-07-18T03:41:02.201 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:01 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:01.733+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.322385+0000 front 2023-07-18T03:40:32.322477+0000 (oldest deadline 2023-07-18T03:40:58.222425+0000) 2023-07-18T03:41:02.238 INFO:journalctl@ceph.osd.6.smithi101.stdout:Jul 18 03:41:01 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-6[150288]: 2023-07-18T03:41:01.796+0000 7f47cab67700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.198755+0000 front 2023-07-18T03:40:32.198734+0000 (oldest deadline 2023-07-18T03:40:58.098626+0000) 2023-07-18T03:41:02.546 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:02 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:02.316+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:30.409213+0000 front 2023-07-18T03:40:30.409292+0000 (oldest deadline 2023-07-18T03:40:55.109366+0000) 2023-07-18T03:41:02.546 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:02 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:02.316+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.810741+0000 front 2023-07-18T03:40:36.810842+0000 (oldest deadline 2023-07-18T03:41:02.110531+0000) 2023-07-18T03:41:02.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:02 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:02.128+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:41:02.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:02 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:02.128+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:41:02.701 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jul 18 03:41:02 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-5[140498]: 2023-07-18T03:41:02.245+0000 7f6352bf5700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:21.251904+0000 front 2023-07-18T03:40:21.251757+0000 (oldest deadline 2023-07-18T03:40:45.951887+0000) 2023-07-18T03:41:02.701 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jul 18 03:41:02 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-5[140498]: 2023-07-18T03:41:02.245+0000 7f6352bf5700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.253079+0000 front 2023-07-18T03:40:28.253102+0000 (oldest deadline 2023-07-18T03:40:54.153072+0000) 2023-07-18T03:41:02.702 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:02 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:02.312+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:02.702 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:02 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:02.312+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:02.702 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:02 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:02.312+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:02.738 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:02 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:02.382+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:41:02.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:02 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:02.382+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.543459+0000 front 2023-07-18T03:40:32.543608+0000 (oldest deadline 2023-07-18T03:40:56.643373+0000) 2023-07-18T03:41:02.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:02 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:02.382+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:37.744902+0000 front 2023-07-18T03:40:37.744760+0000 (oldest deadline 2023-07-18T03:41:01.844955+0000) 2023-07-18T03:41:03.200 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:02 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:02.765+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.220319+0000 front 2023-07-18T03:40:24.220087+0000 (oldest deadline 2023-07-18T03:40:48.920267+0000) 2023-07-18T03:41:03.201 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:02 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:02.765+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.322385+0000 front 2023-07-18T03:40:32.322477+0000 (oldest deadline 2023-07-18T03:40:58.222425+0000) 2023-07-18T03:41:03.238 INFO:journalctl@ceph.osd.6.smithi101.stdout:Jul 18 03:41:02 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-6[150288]: 2023-07-18T03:41:02.767+0000 7f47cab67700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.198755+0000 front 2023-07-18T03:40:32.198734+0000 (oldest deadline 2023-07-18T03:40:58.098626+0000) 2023-07-18T03:41:03.546 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:03 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:03.366+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:30.409213+0000 front 2023-07-18T03:40:30.409292+0000 (oldest deadline 2023-07-18T03:40:55.109366+0000) 2023-07-18T03:41:03.546 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:03 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:03.366+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.810741+0000 front 2023-07-18T03:40:36.810842+0000 (oldest deadline 2023-07-18T03:41:02.110531+0000) 2023-07-18T03:41:03.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:03 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:03.169+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:41:03.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:03 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:03.169+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:41:03.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:03 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:03.169+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.718618+0000 front 2023-07-18T03:40:38.718740+0000 (oldest deadline 2023-07-18T03:41:02.818660+0000) 2023-07-18T03:41:03.701 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jul 18 03:41:03 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-5[140498]: 2023-07-18T03:41:03.262+0000 7f6352bf5700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:21.251904+0000 front 2023-07-18T03:40:21.251757+0000 (oldest deadline 2023-07-18T03:40:45.951887+0000) 2023-07-18T03:41:03.701 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jul 18 03:41:03 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-5[140498]: 2023-07-18T03:41:03.262+0000 7f6352bf5700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.253079+0000 front 2023-07-18T03:40:28.253102+0000 (oldest deadline 2023-07-18T03:40:54.153072+0000) 2023-07-18T03:41:03.701 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:03 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:03.310+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:03.702 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:03 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:03.310+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:03.702 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:03 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:03.310+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:03.738 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:03 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:03.364+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:41:03.738 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:03 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:03.364+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.543459+0000 front 2023-07-18T03:40:32.543608+0000 (oldest deadline 2023-07-18T03:40:56.643373+0000) 2023-07-18T03:41:03.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:03 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:03.364+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:37.744902+0000 front 2023-07-18T03:40:37.744760+0000 (oldest deadline 2023-07-18T03:41:01.844955+0000) 2023-07-18T03:41:04.201 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:03 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:03.799+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.220319+0000 front 2023-07-18T03:40:24.220087+0000 (oldest deadline 2023-07-18T03:40:48.920267+0000) 2023-07-18T03:41:04.201 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:03 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:03.799+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.322385+0000 front 2023-07-18T03:40:32.322477+0000 (oldest deadline 2023-07-18T03:40:58.222425+0000) 2023-07-18T03:41:04.238 INFO:journalctl@ceph.osd.6.smithi101.stdout:Jul 18 03:41:03 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-6[150288]: 2023-07-18T03:41:03.780+0000 7f47cab67700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.198755+0000 front 2023-07-18T03:40:32.198734+0000 (oldest deadline 2023-07-18T03:40:58.098626+0000) 2023-07-18T03:41:04.239 INFO:journalctl@ceph.osd.6.smithi101.stdout:Jul 18 03:41:03 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-6[150288]: 2023-07-18T03:41:03.780+0000 7f47cab67700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.099548+0000 front 2023-07-18T03:40:38.099278+0000 (oldest deadline 2023-07-18T03:41:02.799255+0000) 2023-07-18T03:41:04.546 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:04 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:04.395+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:30.409213+0000 front 2023-07-18T03:40:30.409292+0000 (oldest deadline 2023-07-18T03:40:55.109366+0000) 2023-07-18T03:41:04.546 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:04 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:04.395+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.810741+0000 front 2023-07-18T03:40:36.810842+0000 (oldest deadline 2023-07-18T03:41:02.110531+0000) 2023-07-18T03:41:04.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:04 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:04.168+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:41:04.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:04 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:04.168+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:41:04.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:04 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:04.168+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.718618+0000 front 2023-07-18T03:40:38.718740+0000 (oldest deadline 2023-07-18T03:41:02.818660+0000) 2023-07-18T03:41:04.701 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jul 18 03:41:04 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-5[140498]: 2023-07-18T03:41:04.256+0000 7f6352bf5700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:21.251904+0000 front 2023-07-18T03:40:21.251757+0000 (oldest deadline 2023-07-18T03:40:45.951887+0000) 2023-07-18T03:41:04.701 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jul 18 03:41:04 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-5[140498]: 2023-07-18T03:41:04.256+0000 7f6352bf5700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.253079+0000 front 2023-07-18T03:40:28.253102+0000 (oldest deadline 2023-07-18T03:40:54.153072+0000) 2023-07-18T03:41:04.702 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jul 18 03:41:04 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-5[140498]: 2023-07-18T03:41:04.256+0000 7f6352bf5700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:39.354675+0000 front 2023-07-18T03:40:39.354648+0000 (oldest deadline 2023-07-18T03:41:04.054872+0000) 2023-07-18T03:41:04.702 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:04 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:04.287+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:04.702 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:04 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:04.287+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:04.702 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:04 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:04.287+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:04.738 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:04 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:04.396+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:41:04.738 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:04 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:04.396+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.543459+0000 front 2023-07-18T03:40:32.543608+0000 (oldest deadline 2023-07-18T03:40:56.643373+0000) 2023-07-18T03:41:04.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:04 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:04.396+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:37.744902+0000 front 2023-07-18T03:40:37.744760+0000 (oldest deadline 2023-07-18T03:41:01.844955+0000) 2023-07-18T03:41:05.194 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:04 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:04.849+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.220319+0000 front 2023-07-18T03:40:24.220087+0000 (oldest deadline 2023-07-18T03:40:48.920267+0000) 2023-07-18T03:41:05.194 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:04 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:04.849+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.322385+0000 front 2023-07-18T03:40:32.322477+0000 (oldest deadline 2023-07-18T03:40:58.222425+0000) 2023-07-18T03:41:05.194 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:04 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:04.849+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:39.923305+0000 front 2023-07-18T03:40:39.923435+0000 (oldest deadline 2023-07-18T03:41:04.623543+0000) 2023-07-18T03:41:05.238 INFO:journalctl@ceph.osd.6.smithi101.stdout:Jul 18 03:41:04 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-6[150288]: 2023-07-18T03:41:04.766+0000 7f47cab67700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.198755+0000 front 2023-07-18T03:40:32.198734+0000 (oldest deadline 2023-07-18T03:40:58.098626+0000) 2023-07-18T03:41:05.239 INFO:journalctl@ceph.osd.6.smithi101.stdout:Jul 18 03:41:04 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-6[150288]: 2023-07-18T03:41:04.766+0000 7f47cab67700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.099548+0000 front 2023-07-18T03:40:38.099278+0000 (oldest deadline 2023-07-18T03:41:02.799255+0000) 2023-07-18T03:41:05.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:05 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:05.271+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:05.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:05 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:05.271+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:05.447 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:05 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:05.271+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:05.545 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:05 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:05.425+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:30.409213+0000 front 2023-07-18T03:40:30.409292+0000 (oldest deadline 2023-07-18T03:40:55.109366+0000) 2023-07-18T03:41:05.546 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:05 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:05.425+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.810741+0000 front 2023-07-18T03:40:36.810842+0000 (oldest deadline 2023-07-18T03:41:02.110531+0000) 2023-07-18T03:41:05.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:05 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:05.183+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:41:05.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:05 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:05.183+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:41:05.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:05 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:05.183+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.718618+0000 front 2023-07-18T03:40:38.718740+0000 (oldest deadline 2023-07-18T03:41:02.818660+0000) 2023-07-18T03:41:05.701 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jul 18 03:41:05 smithi179 bash[167447]: ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-5 2023-07-18T03:41:05.738 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:05 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:05.428+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:41:05.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:05 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:05.428+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.543459+0000 front 2023-07-18T03:40:32.543608+0000 (oldest deadline 2023-07-18T03:40:56.643373+0000) 2023-07-18T03:41:05.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:05 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:05.428+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:37.744902+0000 front 2023-07-18T03:40:37.744760+0000 (oldest deadline 2023-07-18T03:41:01.844955+0000) 2023-07-18T03:41:06.200 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:05 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:05.874+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.220319+0000 front 2023-07-18T03:40:24.220087+0000 (oldest deadline 2023-07-18T03:40:48.920267+0000) 2023-07-18T03:41:06.201 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:05 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:05.874+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.322385+0000 front 2023-07-18T03:40:32.322477+0000 (oldest deadline 2023-07-18T03:40:58.222425+0000) 2023-07-18T03:41:06.201 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:05 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:05.874+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:39.923305+0000 front 2023-07-18T03:40:39.923435+0000 (oldest deadline 2023-07-18T03:41:04.623543+0000) 2023-07-18T03:41:06.238 INFO:journalctl@ceph.osd.6.smithi101.stdout:Jul 18 03:41:05 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-6[150288]: 2023-07-18T03:41:05.761+0000 7f47cab67700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.198755+0000 front 2023-07-18T03:40:32.198734+0000 (oldest deadline 2023-07-18T03:40:58.098626+0000) 2023-07-18T03:41:06.239 INFO:journalctl@ceph.osd.6.smithi101.stdout:Jul 18 03:41:05 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-6[150288]: 2023-07-18T03:41:05.761+0000 7f47cab67700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.099548+0000 front 2023-07-18T03:40:38.099278+0000 (oldest deadline 2023-07-18T03:41:02.799255+0000) 2023-07-18T03:41:06.546 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:06 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:06.386+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:30.409213+0000 front 2023-07-18T03:40:30.409292+0000 (oldest deadline 2023-07-18T03:40:55.109366+0000) 2023-07-18T03:41:06.546 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:06 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:06.386+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.810741+0000 front 2023-07-18T03:40:36.810842+0000 (oldest deadline 2023-07-18T03:41:02.110531+0000) 2023-07-18T03:41:06.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:06 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:06.155+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:41:06.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:06 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:06.155+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:41:06.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:06 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:06.155+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.718618+0000 front 2023-07-18T03:40:38.718740+0000 (oldest deadline 2023-07-18T03:41:02.818660+0000) 2023-07-18T03:41:06.701 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:06 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:06.314+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:06.701 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:06 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:06.314+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:06.702 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:06 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:06.314+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:06.738 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:06 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:06.416+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:41:06.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:06 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:06.416+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.543459+0000 front 2023-07-18T03:40:32.543608+0000 (oldest deadline 2023-07-18T03:40:56.643373+0000) 2023-07-18T03:41:06.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:06 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:06.416+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:37.744902+0000 front 2023-07-18T03:40:37.744760+0000 (oldest deadline 2023-07-18T03:41:01.844955+0000) 2023-07-18T03:41:06.757 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@osd.5.service' 2023-07-18T03:41:06.798 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:41:06.799 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-07-18T03:41:06.799 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-07-18T03:41:06.799 DEBUG:teuthology.orchestra.run.smithi101:> sudo systemctl stop ceph-b1782daa-2516-11ee-9b34-001a4aab830c@osd.6 2023-07-18T03:41:07.201 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:06 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:06.892+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.220319+0000 front 2023-07-18T03:40:24.220087+0000 (oldest deadline 2023-07-18T03:40:48.920267+0000) 2023-07-18T03:41:07.201 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:06 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:06.892+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.322385+0000 front 2023-07-18T03:40:32.322477+0000 (oldest deadline 2023-07-18T03:40:58.222425+0000) 2023-07-18T03:41:07.202 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:06 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:06.892+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:39.923305+0000 front 2023-07-18T03:40:39.923435+0000 (oldest deadline 2023-07-18T03:41:04.623543+0000) 2023-07-18T03:41:07.238 INFO:journalctl@ceph.osd.6.smithi101.stdout:Jul 18 03:41:06 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-6[150288]: 2023-07-18T03:41:06.766+0000 7f47cab67700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.198755+0000 front 2023-07-18T03:40:32.198734+0000 (oldest deadline 2023-07-18T03:40:58.098626+0000) 2023-07-18T03:41:07.239 INFO:journalctl@ceph.osd.6.smithi101.stdout:Jul 18 03:41:06 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-6[150288]: 2023-07-18T03:41:06.766+0000 7f47cab67700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.099548+0000 front 2023-07-18T03:40:38.099278+0000 (oldest deadline 2023-07-18T03:41:02.799255+0000) 2023-07-18T03:41:07.239 INFO:journalctl@ceph.osd.6.smithi101.stdout:Jul 18 03:41:06 smithi101 systemd[1]: Stopping Ceph osd.6 for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T03:41:07.546 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:07 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:07.391+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:30.409213+0000 front 2023-07-18T03:40:30.409292+0000 (oldest deadline 2023-07-18T03:40:55.109366+0000) 2023-07-18T03:41:07.546 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:07 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:07.391+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.810741+0000 front 2023-07-18T03:40:36.810842+0000 (oldest deadline 2023-07-18T03:41:02.110531+0000) 2023-07-18T03:41:07.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:07 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:07.194+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:41:07.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:07 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:07.194+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:41:07.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:07 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:07.194+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.718618+0000 front 2023-07-18T03:40:38.718740+0000 (oldest deadline 2023-07-18T03:41:02.818660+0000) 2023-07-18T03:41:07.701 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:07 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:07.286+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:07.701 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:07 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:07.286+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:07.701 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:07 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:07.286+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:07.738 INFO:journalctl@ceph.osd.6.smithi101.stdout:Jul 18 03:41:07 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-6[150288]: 2023-07-18T03:41:07.317+0000 7f47d0575700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:41:07.739 INFO:journalctl@ceph.osd.6.smithi101.stdout:Jul 18 03:41:07 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-6[150288]: 2023-07-18T03:41:07.317+0000 7f47d0575700 -1 osd.6 149 *** Got signal Terminated *** 2023-07-18T03:41:07.739 INFO:journalctl@ceph.osd.6.smithi101.stdout:Jul 18 03:41:07 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-6[150288]: 2023-07-18T03:41:07.317+0000 7f47d0575700 -1 osd.6 149 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-18T03:41:07.740 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:07 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:07.410+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:41:07.740 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:07 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:07.410+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.543459+0000 front 2023-07-18T03:40:32.543608+0000 (oldest deadline 2023-07-18T03:40:56.643373+0000) 2023-07-18T03:41:07.740 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:07 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:07.410+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:37.744902+0000 front 2023-07-18T03:40:37.744760+0000 (oldest deadline 2023-07-18T03:41:01.844955+0000) 2023-07-18T03:41:08.201 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:07 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:07.842+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.220319+0000 front 2023-07-18T03:40:24.220087+0000 (oldest deadline 2023-07-18T03:40:48.920267+0000) 2023-07-18T03:41:08.201 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:07 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:07.842+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.322385+0000 front 2023-07-18T03:40:32.322477+0000 (oldest deadline 2023-07-18T03:40:58.222425+0000) 2023-07-18T03:41:08.201 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:07 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:07.842+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:39.923305+0000 front 2023-07-18T03:40:39.923435+0000 (oldest deadline 2023-07-18T03:41:04.623543+0000) 2023-07-18T03:41:08.238 INFO:journalctl@ceph.osd.6.smithi101.stdout:Jul 18 03:41:07 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-6[150288]: 2023-07-18T03:41:07.756+0000 7f47cab67700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.198755+0000 front 2023-07-18T03:40:32.198734+0000 (oldest deadline 2023-07-18T03:40:58.098626+0000) 2023-07-18T03:41:08.239 INFO:journalctl@ceph.osd.6.smithi101.stdout:Jul 18 03:41:07 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-6[150288]: 2023-07-18T03:41:07.756+0000 7f47cab67700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.099548+0000 front 2023-07-18T03:40:38.099278+0000 (oldest deadline 2023-07-18T03:41:02.799255+0000) 2023-07-18T03:41:08.546 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:08 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:08.390+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:30.409213+0000 front 2023-07-18T03:40:30.409292+0000 (oldest deadline 2023-07-18T03:40:55.109366+0000) 2023-07-18T03:41:08.546 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:08 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:08.390+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.810741+0000 front 2023-07-18T03:40:36.810842+0000 (oldest deadline 2023-07-18T03:41:02.110531+0000) 2023-07-18T03:41:08.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:08 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:08.190+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:41:08.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:08 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:08.190+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:41:08.548 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:08 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:08.190+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.718618+0000 front 2023-07-18T03:40:38.718740+0000 (oldest deadline 2023-07-18T03:41:02.818660+0000) 2023-07-18T03:41:08.700 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:08 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:08.314+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:08.701 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:08 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:08.314+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:08.701 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:08 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:08.314+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:08.738 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:08 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:08.366+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:41:08.738 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:08 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:08.366+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.543459+0000 front 2023-07-18T03:40:32.543608+0000 (oldest deadline 2023-07-18T03:40:56.643373+0000) 2023-07-18T03:41:08.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:08 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:08.366+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:37.744902+0000 front 2023-07-18T03:40:37.744760+0000 (oldest deadline 2023-07-18T03:41:01.844955+0000) 2023-07-18T03:41:09.200 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:08 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:08.806+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.220319+0000 front 2023-07-18T03:40:24.220087+0000 (oldest deadline 2023-07-18T03:40:48.920267+0000) 2023-07-18T03:41:09.201 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:08 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:08.806+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.322385+0000 front 2023-07-18T03:40:32.322477+0000 (oldest deadline 2023-07-18T03:40:58.222425+0000) 2023-07-18T03:41:09.201 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:08 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:08.806+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:39.923305+0000 front 2023-07-18T03:40:39.923435+0000 (oldest deadline 2023-07-18T03:41:04.623543+0000) 2023-07-18T03:41:09.238 INFO:journalctl@ceph.osd.6.smithi101.stdout:Jul 18 03:41:08 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-6[150288]: 2023-07-18T03:41:08.767+0000 7f47cab67700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.198755+0000 front 2023-07-18T03:40:32.198734+0000 (oldest deadline 2023-07-18T03:40:58.098626+0000) 2023-07-18T03:41:09.238 INFO:journalctl@ceph.osd.6.smithi101.stdout:Jul 18 03:41:08 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-6[150288]: 2023-07-18T03:41:08.767+0000 7f47cab67700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.099548+0000 front 2023-07-18T03:40:38.099278+0000 (oldest deadline 2023-07-18T03:41:02.799255+0000) 2023-07-18T03:41:09.545 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:09 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:09.407+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:30.409213+0000 front 2023-07-18T03:40:30.409292+0000 (oldest deadline 2023-07-18T03:40:55.109366+0000) 2023-07-18T03:41:09.546 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:09 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:09.407+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.810741+0000 front 2023-07-18T03:40:36.810842+0000 (oldest deadline 2023-07-18T03:41:02.110531+0000) 2023-07-18T03:41:09.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:09 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:09.163+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:41:09.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:09 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:09.163+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:41:09.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:09 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:09.163+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.718618+0000 front 2023-07-18T03:40:38.718740+0000 (oldest deadline 2023-07-18T03:41:02.818660+0000) 2023-07-18T03:41:09.701 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:09 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:09.317+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:09.701 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:09 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:09.317+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:09.701 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:09 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:09.317+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:09.732 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:09 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:09.352+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:41:09.733 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:09 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:09.353+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.543459+0000 front 2023-07-18T03:40:32.543608+0000 (oldest deadline 2023-07-18T03:40:56.643373+0000) 2023-07-18T03:41:09.734 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:09 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:09.353+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:37.744902+0000 front 2023-07-18T03:40:37.744760+0000 (oldest deadline 2023-07-18T03:41:01.844955+0000) 2023-07-18T03:41:09.988 INFO:journalctl@ceph.osd.6.smithi101.stdout:Jul 18 03:41:09 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-6[150288]: 2023-07-18T03:41:09.738+0000 7f47cab67700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.198755+0000 front 2023-07-18T03:40:32.198734+0000 (oldest deadline 2023-07-18T03:40:58.098626+0000) 2023-07-18T03:41:09.989 INFO:journalctl@ceph.osd.6.smithi101.stdout:Jul 18 03:41:09 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-6[150288]: 2023-07-18T03:41:09.738+0000 7f47cab67700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.099548+0000 front 2023-07-18T03:40:38.099278+0000 (oldest deadline 2023-07-18T03:41:02.799255+0000) 2023-07-18T03:41:10.200 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:09 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:09.762+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.220319+0000 front 2023-07-18T03:40:24.220087+0000 (oldest deadline 2023-07-18T03:40:48.920267+0000) 2023-07-18T03:41:10.201 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:09 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:09.762+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.322385+0000 front 2023-07-18T03:40:32.322477+0000 (oldest deadline 2023-07-18T03:40:58.222425+0000) 2023-07-18T03:41:10.201 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:09 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:09.762+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:39.923305+0000 front 2023-07-18T03:40:39.923435+0000 (oldest deadline 2023-07-18T03:41:04.623543+0000) 2023-07-18T03:41:10.546 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:10 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:10.362+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:30.409213+0000 front 2023-07-18T03:40:30.409292+0000 (oldest deadline 2023-07-18T03:40:55.109366+0000) 2023-07-18T03:41:10.546 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:10 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:10.362+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.810741+0000 front 2023-07-18T03:40:36.810842+0000 (oldest deadline 2023-07-18T03:41:02.110531+0000) 2023-07-18T03:41:10.546 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:10 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:10.362+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:46.712404+0000 front 2023-07-18T03:40:46.712242+0000 (oldest deadline 2023-07-18T03:41:09.612355+0000) 2023-07-18T03:41:10.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:10 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:10.157+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:41:10.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:10 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:10.157+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:41:10.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:10 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:10.157+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.718618+0000 front 2023-07-18T03:40:38.718740+0000 (oldest deadline 2023-07-18T03:41:02.818660+0000) 2023-07-18T03:41:10.700 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:10 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:10.313+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:10.701 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:10 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:10.313+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:10.701 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:10 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:10.313+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:10.734 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:10 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:10.319+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:41:10.734 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:10 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:10.319+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.543459+0000 front 2023-07-18T03:40:32.543608+0000 (oldest deadline 2023-07-18T03:40:56.643373+0000) 2023-07-18T03:41:10.735 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:10 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:10.319+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:37.744902+0000 front 2023-07-18T03:40:37.744760+0000 (oldest deadline 2023-07-18T03:41:01.844955+0000) 2023-07-18T03:41:10.988 INFO:journalctl@ceph.osd.6.smithi101.stdout:Jul 18 03:41:10 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-6[150288]: 2023-07-18T03:41:10.740+0000 7f47cab67700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.198755+0000 front 2023-07-18T03:40:32.198734+0000 (oldest deadline 2023-07-18T03:40:58.098626+0000) 2023-07-18T03:41:10.989 INFO:journalctl@ceph.osd.6.smithi101.stdout:Jul 18 03:41:10 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-6[150288]: 2023-07-18T03:41:10.740+0000 7f47cab67700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.099548+0000 front 2023-07-18T03:40:38.099278+0000 (oldest deadline 2023-07-18T03:41:02.799255+0000) 2023-07-18T03:41:10.989 INFO:journalctl@ceph.osd.6.smithi101.stdout:Jul 18 03:41:10 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-6[150288]: 2023-07-18T03:41:10.740+0000 7f47cab67700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:47.500476+0000 front 2023-07-18T03:40:47.500415+0000 (oldest deadline 2023-07-18T03:41:10.400528+0000) 2023-07-18T03:41:11.201 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:10 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:10.749+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.220319+0000 front 2023-07-18T03:40:24.220087+0000 (oldest deadline 2023-07-18T03:40:48.920267+0000) 2023-07-18T03:41:11.201 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:10 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:10.749+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.322385+0000 front 2023-07-18T03:40:32.322477+0000 (oldest deadline 2023-07-18T03:40:58.222425+0000) 2023-07-18T03:41:11.201 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:10 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:10.749+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:39.923305+0000 front 2023-07-18T03:40:39.923435+0000 (oldest deadline 2023-07-18T03:41:04.623543+0000) 2023-07-18T03:41:11.546 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:11 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:11.334+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:30.409213+0000 front 2023-07-18T03:40:30.409292+0000 (oldest deadline 2023-07-18T03:40:55.109366+0000) 2023-07-18T03:41:11.546 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:11 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:11.334+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.810741+0000 front 2023-07-18T03:40:36.810842+0000 (oldest deadline 2023-07-18T03:41:02.110531+0000) 2023-07-18T03:41:11.546 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:11 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:11.334+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:46.712404+0000 front 2023-07-18T03:40:46.712242+0000 (oldest deadline 2023-07-18T03:41:09.612355+0000) 2023-07-18T03:41:11.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:11 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:11.134+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:41:11.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:11 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:11.134+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:41:11.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:11 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:11.134+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.718618+0000 front 2023-07-18T03:40:38.718740+0000 (oldest deadline 2023-07-18T03:41:02.818660+0000) 2023-07-18T03:41:11.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:11 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:11.134+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:46.820033+0000 front 2023-07-18T03:40:46.820086+0000 (oldest deadline 2023-07-18T03:41:10.319945+0000) 2023-07-18T03:41:11.696 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:11 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:11.299+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:41:11.696 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:11 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:11.299+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.543459+0000 front 2023-07-18T03:40:32.543608+0000 (oldest deadline 2023-07-18T03:40:56.643373+0000) 2023-07-18T03:41:11.697 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:11 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:11.299+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:37.744902+0000 front 2023-07-18T03:40:37.744760+0000 (oldest deadline 2023-07-18T03:41:01.844955+0000) 2023-07-18T03:41:11.701 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:11 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:11.291+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:11.701 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:11 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:11.291+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:11.702 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:11 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:11.291+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:11.988 INFO:journalctl@ceph.osd.6.smithi101.stdout:Jul 18 03:41:11 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-6[150288]: 2023-07-18T03:41:11.702+0000 7f47cab67700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.198755+0000 front 2023-07-18T03:40:32.198734+0000 (oldest deadline 2023-07-18T03:40:58.098626+0000) 2023-07-18T03:41:11.989 INFO:journalctl@ceph.osd.6.smithi101.stdout:Jul 18 03:41:11 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-6[150288]: 2023-07-18T03:41:11.702+0000 7f47cab67700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.099548+0000 front 2023-07-18T03:40:38.099278+0000 (oldest deadline 2023-07-18T03:41:02.799255+0000) 2023-07-18T03:41:11.989 INFO:journalctl@ceph.osd.6.smithi101.stdout:Jul 18 03:41:11 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-6[150288]: 2023-07-18T03:41:11.702+0000 7f47cab67700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:47.500476+0000 front 2023-07-18T03:40:47.500415+0000 (oldest deadline 2023-07-18T03:41:10.400528+0000) 2023-07-18T03:41:12.201 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:11 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:11.786+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.220319+0000 front 2023-07-18T03:40:24.220087+0000 (oldest deadline 2023-07-18T03:40:48.920267+0000) 2023-07-18T03:41:12.201 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:11 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:11.786+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.322385+0000 front 2023-07-18T03:40:32.322477+0000 (oldest deadline 2023-07-18T03:40:58.222425+0000) 2023-07-18T03:41:12.201 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:11 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:11.786+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:39.923305+0000 front 2023-07-18T03:40:39.923435+0000 (oldest deadline 2023-07-18T03:41:04.623543+0000) 2023-07-18T03:41:12.546 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:12 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:12.375+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:30.409213+0000 front 2023-07-18T03:40:30.409292+0000 (oldest deadline 2023-07-18T03:40:55.109366+0000) 2023-07-18T03:41:12.546 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:12 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:12.375+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.810741+0000 front 2023-07-18T03:40:36.810842+0000 (oldest deadline 2023-07-18T03:41:02.110531+0000) 2023-07-18T03:41:12.546 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:12 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:12.375+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:46.712404+0000 front 2023-07-18T03:40:46.712242+0000 (oldest deadline 2023-07-18T03:41:09.612355+0000) 2023-07-18T03:41:12.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:12 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:12.169+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:41:12.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:12 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:12.169+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:41:12.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:12 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:12.169+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.718618+0000 front 2023-07-18T03:40:38.718740+0000 (oldest deadline 2023-07-18T03:41:02.818660+0000) 2023-07-18T03:41:12.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:12 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:12.169+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:46.820033+0000 front 2023-07-18T03:40:46.820086+0000 (oldest deadline 2023-07-18T03:41:10.319945+0000) 2023-07-18T03:41:12.701 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:12 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:12.282+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:12.703 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:12 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:12.282+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:12.703 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:12 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:12.282+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:12.738 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:12 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:12.270+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:41:12.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:12 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:12.270+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.543459+0000 front 2023-07-18T03:40:32.543608+0000 (oldest deadline 2023-07-18T03:40:56.643373+0000) 2023-07-18T03:41:12.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:12 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:12.270+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:37.744902+0000 front 2023-07-18T03:40:37.744760+0000 (oldest deadline 2023-07-18T03:41:01.844955+0000) 2023-07-18T03:41:13.203 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:12 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:12.759+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.220319+0000 front 2023-07-18T03:40:24.220087+0000 (oldest deadline 2023-07-18T03:40:48.920267+0000) 2023-07-18T03:41:13.203 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:12 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:12.759+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.322385+0000 front 2023-07-18T03:40:32.322477+0000 (oldest deadline 2023-07-18T03:40:58.222425+0000) 2023-07-18T03:41:13.204 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:12 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:12.759+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:39.923305+0000 front 2023-07-18T03:40:39.923435+0000 (oldest deadline 2023-07-18T03:41:04.623543+0000) 2023-07-18T03:41:13.504 INFO:journalctl@ceph.osd.6.smithi101.stdout:Jul 18 03:41:13 smithi101 bash[206202]: ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-6 2023-07-18T03:41:13.504 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:13 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:13.247+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:41:13.505 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:13 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:13.247+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.543459+0000 front 2023-07-18T03:40:32.543608+0000 (oldest deadline 2023-07-18T03:40:56.643373+0000) 2023-07-18T03:41:13.505 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:13 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:13.247+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:37.744902+0000 front 2023-07-18T03:40:37.744760+0000 (oldest deadline 2023-07-18T03:41:01.844955+0000) 2023-07-18T03:41:13.546 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:13 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:13.403+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:30.409213+0000 front 2023-07-18T03:40:30.409292+0000 (oldest deadline 2023-07-18T03:40:55.109366+0000) 2023-07-18T03:41:13.547 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:13 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:13.403+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.810741+0000 front 2023-07-18T03:40:36.810842+0000 (oldest deadline 2023-07-18T03:41:02.110531+0000) 2023-07-18T03:41:13.547 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:13 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:13.403+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:46.712404+0000 front 2023-07-18T03:40:46.712242+0000 (oldest deadline 2023-07-18T03:41:09.612355+0000) 2023-07-18T03:41:13.548 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:13 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:13.197+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:41:13.548 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:13 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:13.197+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:41:13.549 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:13 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:13.197+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.718618+0000 front 2023-07-18T03:40:38.718740+0000 (oldest deadline 2023-07-18T03:41:02.818660+0000) 2023-07-18T03:41:13.549 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:13 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:13.197+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:46.820033+0000 front 2023-07-18T03:40:46.820086+0000 (oldest deadline 2023-07-18T03:41:10.319945+0000) 2023-07-18T03:41:13.701 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:13 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:13.259+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:13.701 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:13 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:13.259+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:13.701 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:13 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:13.259+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:14.200 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:13 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:13.739+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.220319+0000 front 2023-07-18T03:40:24.220087+0000 (oldest deadline 2023-07-18T03:40:48.920267+0000) 2023-07-18T03:41:14.201 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:13 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:13.739+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.322385+0000 front 2023-07-18T03:40:32.322477+0000 (oldest deadline 2023-07-18T03:40:58.222425+0000) 2023-07-18T03:41:14.201 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:13 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:13.739+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:39.923305+0000 front 2023-07-18T03:40:39.923435+0000 (oldest deadline 2023-07-18T03:41:04.623543+0000) 2023-07-18T03:41:14.424 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:14 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:14.162+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:41:14.425 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:14 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:14.163+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:41:14.425 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:14 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:14.163+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.718618+0000 front 2023-07-18T03:40:38.718740+0000 (oldest deadline 2023-07-18T03:41:02.818660+0000) 2023-07-18T03:41:14.425 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:14 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:14.163+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:46.820033+0000 front 2023-07-18T03:40:46.820086+0000 (oldest deadline 2023-07-18T03:41:10.319945+0000) 2023-07-18T03:41:14.700 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:14 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:14.232+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:14.701 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:14 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:14.232+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:14.701 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:14 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:14.232+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:14.738 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:14 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:14.260+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:41:14.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:14 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:14.260+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.543459+0000 front 2023-07-18T03:40:32.543608+0000 (oldest deadline 2023-07-18T03:40:56.643373+0000) 2023-07-18T03:41:14.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:14 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:14.260+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:37.744902+0000 front 2023-07-18T03:40:37.744760+0000 (oldest deadline 2023-07-18T03:41:01.844955+0000) 2023-07-18T03:41:14.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:14 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:14.260+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:48.047789+0000 front 2023-07-18T03:40:48.047719+0000 (oldest deadline 2023-07-18T03:41:13.947754+0000) 2023-07-18T03:41:14.796 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:14 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:14.430+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:30.409213+0000 front 2023-07-18T03:40:30.409292+0000 (oldest deadline 2023-07-18T03:40:55.109366+0000) 2023-07-18T03:41:14.796 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:14 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:14.430+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.810741+0000 front 2023-07-18T03:40:36.810842+0000 (oldest deadline 2023-07-18T03:41:02.110531+0000) 2023-07-18T03:41:14.796 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:14 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:14.430+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:46.712404+0000 front 2023-07-18T03:40:46.712242+0000 (oldest deadline 2023-07-18T03:41:09.612355+0000) 2023-07-18T03:41:15.192 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:14 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:14.720+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.220319+0000 front 2023-07-18T03:40:24.220087+0000 (oldest deadline 2023-07-18T03:40:48.920267+0000) 2023-07-18T03:41:15.193 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:14 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:14.720+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.322385+0000 front 2023-07-18T03:40:32.322477+0000 (oldest deadline 2023-07-18T03:40:58.222425+0000) 2023-07-18T03:41:15.193 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:14 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:14.720+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:39.923305+0000 front 2023-07-18T03:40:39.923435+0000 (oldest deadline 2023-07-18T03:41:04.623543+0000) 2023-07-18T03:41:15.413 DEBUG:teuthology.orchestra.run.smithi101:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@osd.6.service' 2023-07-18T03:41:15.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:15 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:15.198+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:15.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:15 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:15.198+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:15.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:15 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:15.198+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:15.476 INFO:journalctl@ceph.osd.6.smithi101.stdout:Jul 18 03:41:15 smithi101 systemd[1]: ceph-b1782daa-2516-11ee-9b34-001a4aab830c@osd.6.service: Succeeded. 2023-07-18T03:41:15.477 INFO:journalctl@ceph.osd.6.smithi101.stdout:Jul 18 03:41:15 smithi101 systemd[1]: Stopped Ceph osd.6 for b1782daa-2516-11ee-9b34-001a4aab830c. 2023-07-18T03:41:15.477 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:15 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:15.229+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:41:15.477 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:15 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:15.229+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.543459+0000 front 2023-07-18T03:40:32.543608+0000 (oldest deadline 2023-07-18T03:40:56.643373+0000) 2023-07-18T03:41:15.478 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:15 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:15.229+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:37.744902+0000 front 2023-07-18T03:40:37.744760+0000 (oldest deadline 2023-07-18T03:41:01.844955+0000) 2023-07-18T03:41:15.478 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:15 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:15.229+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:48.047789+0000 front 2023-07-18T03:40:48.047719+0000 (oldest deadline 2023-07-18T03:41:13.947754+0000) 2023-07-18T03:41:15.502 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:41:15.503 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-07-18T03:41:15.503 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-07-18T03:41:15.503 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-b1782daa-2516-11ee-9b34-001a4aab830c@osd.7 2023-07-18T03:41:15.536 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:15 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:15.381+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:30.409213+0000 front 2023-07-18T03:40:30.409292+0000 (oldest deadline 2023-07-18T03:40:55.109366+0000) 2023-07-18T03:41:15.537 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:15 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:15.381+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.810741+0000 front 2023-07-18T03:40:36.810842+0000 (oldest deadline 2023-07-18T03:41:02.110531+0000) 2023-07-18T03:41:15.537 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:15 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:15.381+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:46.712404+0000 front 2023-07-18T03:40:46.712242+0000 (oldest deadline 2023-07-18T03:41:09.612355+0000) 2023-07-18T03:41:15.537 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:15 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:15.137+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:41:15.538 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:15 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:15.137+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:41:15.538 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:15 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:15.137+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.718618+0000 front 2023-07-18T03:40:38.718740+0000 (oldest deadline 2023-07-18T03:41:02.818660+0000) 2023-07-18T03:41:15.538 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:15 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:15.137+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:46.820033+0000 front 2023-07-18T03:40:46.820086+0000 (oldest deadline 2023-07-18T03:41:10.319945+0000) 2023-07-18T03:41:15.796 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:15 smithi155 systemd[1]: Stopping Ceph osd.7 for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T03:41:16.141 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:15 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:15.865+0000 7f2371488700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:41:16.141 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:15 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:15.865+0000 7f2371488700 -1 osd.7 149 *** Got signal Terminated *** 2023-07-18T03:41:16.141 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:15 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:15.865+0000 7f2371488700 -1 osd.7 149 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-18T03:41:16.147 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:15 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:15.740+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.220319+0000 front 2023-07-18T03:40:24.220087+0000 (oldest deadline 2023-07-18T03:40:48.920267+0000) 2023-07-18T03:41:16.147 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:15 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:15.740+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.322385+0000 front 2023-07-18T03:40:32.322477+0000 (oldest deadline 2023-07-18T03:40:58.222425+0000) 2023-07-18T03:41:16.148 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:15 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:15.740+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:39.923305+0000 front 2023-07-18T03:40:39.923435+0000 (oldest deadline 2023-07-18T03:41:04.623543+0000) 2023-07-18T03:41:16.450 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:16 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:16.154+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:16.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:16 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:16.154+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:16.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:16 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:16.154+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:16.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:16 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:16.147+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:41:16.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:16 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:16.147+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:41:16.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:16 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:16.147+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.718618+0000 front 2023-07-18T03:40:38.718740+0000 (oldest deadline 2023-07-18T03:41:02.818660+0000) 2023-07-18T03:41:16.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:16 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:16.147+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:46.820033+0000 front 2023-07-18T03:40:46.820086+0000 (oldest deadline 2023-07-18T03:41:10.319945+0000) 2023-07-18T03:41:16.547 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:16 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:16.385+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:30.409213+0000 front 2023-07-18T03:40:30.409292+0000 (oldest deadline 2023-07-18T03:40:55.109366+0000) 2023-07-18T03:41:16.547 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:16 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:16.385+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.810741+0000 front 2023-07-18T03:40:36.810842+0000 (oldest deadline 2023-07-18T03:41:02.110531+0000) 2023-07-18T03:41:16.548 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:16 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:16.385+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:46.712404+0000 front 2023-07-18T03:40:46.712242+0000 (oldest deadline 2023-07-18T03:41:09.612355+0000) 2023-07-18T03:41:16.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:16 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:16.261+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:41:16.740 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:16 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:16.261+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.543459+0000 front 2023-07-18T03:40:32.543608+0000 (oldest deadline 2023-07-18T03:40:56.643373+0000) 2023-07-18T03:41:16.740 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:16 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:16.261+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:37.744902+0000 front 2023-07-18T03:40:37.744760+0000 (oldest deadline 2023-07-18T03:41:01.844955+0000) 2023-07-18T03:41:16.740 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:16 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:16.261+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:48.047789+0000 front 2023-07-18T03:40:48.047719+0000 (oldest deadline 2023-07-18T03:41:13.947754+0000) 2023-07-18T03:41:17.138 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:16 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:16.719+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.220319+0000 front 2023-07-18T03:40:24.220087+0000 (oldest deadline 2023-07-18T03:40:48.920267+0000) 2023-07-18T03:41:17.138 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:16 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:16.719+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.322385+0000 front 2023-07-18T03:40:32.322477+0000 (oldest deadline 2023-07-18T03:40:58.222425+0000) 2023-07-18T03:41:17.138 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:16 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:16.719+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:39.923305+0000 front 2023-07-18T03:40:39.923435+0000 (oldest deadline 2023-07-18T03:41:04.623543+0000) 2023-07-18T03:41:17.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:17 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:17.144+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:17.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:17 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:17.144+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:17.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:17 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:17.144+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:17.488 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:17 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:17.215+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:41:17.488 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:17 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:17.215+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.543459+0000 front 2023-07-18T03:40:32.543608+0000 (oldest deadline 2023-07-18T03:40:56.643373+0000) 2023-07-18T03:41:17.489 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:17 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:17.215+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:37.744902+0000 front 2023-07-18T03:40:37.744760+0000 (oldest deadline 2023-07-18T03:41:01.844955+0000) 2023-07-18T03:41:17.489 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:17 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:17.215+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:48.047789+0000 front 2023-07-18T03:40:48.047719+0000 (oldest deadline 2023-07-18T03:41:13.947754+0000) 2023-07-18T03:41:17.545 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:17 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:17.418+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:30.409213+0000 front 2023-07-18T03:40:30.409292+0000 (oldest deadline 2023-07-18T03:40:55.109366+0000) 2023-07-18T03:41:17.546 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:17 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:17.418+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.810741+0000 front 2023-07-18T03:40:36.810842+0000 (oldest deadline 2023-07-18T03:41:02.110531+0000) 2023-07-18T03:41:17.546 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:17 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:17.418+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:46.712404+0000 front 2023-07-18T03:40:46.712242+0000 (oldest deadline 2023-07-18T03:41:09.612355+0000) 2023-07-18T03:41:17.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:17 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:17.196+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:41:17.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:17 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:17.196+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:41:17.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:17 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:17.196+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.718618+0000 front 2023-07-18T03:40:38.718740+0000 (oldest deadline 2023-07-18T03:41:02.818660+0000) 2023-07-18T03:41:17.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:17 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:17.196+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:46.820033+0000 front 2023-07-18T03:40:46.820086+0000 (oldest deadline 2023-07-18T03:41:10.319945+0000) 2023-07-18T03:41:18.100 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:17 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:17.738+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.220319+0000 front 2023-07-18T03:40:24.220087+0000 (oldest deadline 2023-07-18T03:40:48.920267+0000) 2023-07-18T03:41:18.101 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:17 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:17.738+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.322385+0000 front 2023-07-18T03:40:32.322477+0000 (oldest deadline 2023-07-18T03:40:58.222425+0000) 2023-07-18T03:41:18.101 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:17 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:17.738+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:39.923305+0000 front 2023-07-18T03:40:39.923435+0000 (oldest deadline 2023-07-18T03:41:04.623543+0000) 2023-07-18T03:41:18.454 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:18 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:18.107+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:18.454 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:18 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:18.107+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:18.455 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:18 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:18.107+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:18.488 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:18 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:18.243+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:41:18.488 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:18 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:18.243+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.543459+0000 front 2023-07-18T03:40:32.543608+0000 (oldest deadline 2023-07-18T03:40:56.643373+0000) 2023-07-18T03:41:18.489 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:18 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:18.243+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:37.744902+0000 front 2023-07-18T03:40:37.744760+0000 (oldest deadline 2023-07-18T03:41:01.844955+0000) 2023-07-18T03:41:18.489 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:18 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:18.243+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:48.047789+0000 front 2023-07-18T03:40:48.047719+0000 (oldest deadline 2023-07-18T03:41:13.947754+0000) 2023-07-18T03:41:18.546 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:18 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:18.416+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:30.409213+0000 front 2023-07-18T03:40:30.409292+0000 (oldest deadline 2023-07-18T03:40:55.109366+0000) 2023-07-18T03:41:18.546 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:18 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:18.416+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.810741+0000 front 2023-07-18T03:40:36.810842+0000 (oldest deadline 2023-07-18T03:41:02.110531+0000) 2023-07-18T03:41:18.546 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:18 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:18.416+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:46.712404+0000 front 2023-07-18T03:40:46.712242+0000 (oldest deadline 2023-07-18T03:41:09.612355+0000) 2023-07-18T03:41:18.547 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:18 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:18.416+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.714541+0000 front 2023-07-18T03:40:54.714570+0000 (oldest deadline 2023-07-18T03:41:17.614746+0000) 2023-07-18T03:41:18.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:18 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:18.202+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:41:18.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:18 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:18.202+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:41:18.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:18 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:18.202+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.718618+0000 front 2023-07-18T03:40:38.718740+0000 (oldest deadline 2023-07-18T03:41:02.818660+0000) 2023-07-18T03:41:18.548 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:18 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:18.202+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:46.820033+0000 front 2023-07-18T03:40:46.820086+0000 (oldest deadline 2023-07-18T03:41:10.319945+0000) 2023-07-18T03:41:19.101 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:18 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:18.746+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.220319+0000 front 2023-07-18T03:40:24.220087+0000 (oldest deadline 2023-07-18T03:40:48.920267+0000) 2023-07-18T03:41:19.102 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:18 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:18.746+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.322385+0000 front 2023-07-18T03:40:32.322477+0000 (oldest deadline 2023-07-18T03:40:58.222425+0000) 2023-07-18T03:41:19.102 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:18 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:18.746+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:39.923305+0000 front 2023-07-18T03:40:39.923435+0000 (oldest deadline 2023-07-18T03:41:04.623543+0000) 2023-07-18T03:41:19.450 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:19 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:19.108+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:19.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:19 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:19.108+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:19.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:19 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:19.108+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:19.545 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:19 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:19.373+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:30.409213+0000 front 2023-07-18T03:40:30.409292+0000 (oldest deadline 2023-07-18T03:40:55.109366+0000) 2023-07-18T03:41:19.546 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:19 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:19.373+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.810741+0000 front 2023-07-18T03:40:36.810842+0000 (oldest deadline 2023-07-18T03:41:02.110531+0000) 2023-07-18T03:41:19.546 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:19 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:19.373+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:46.712404+0000 front 2023-07-18T03:40:46.712242+0000 (oldest deadline 2023-07-18T03:41:09.612355+0000) 2023-07-18T03:41:19.546 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:19 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:19.373+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.714541+0000 front 2023-07-18T03:40:54.714570+0000 (oldest deadline 2023-07-18T03:41:17.614746+0000) 2023-07-18T03:41:19.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:19 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:19.182+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:41:19.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:19 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:19.182+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:41:19.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:19 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:19.182+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.718618+0000 front 2023-07-18T03:40:38.718740+0000 (oldest deadline 2023-07-18T03:41:02.818660+0000) 2023-07-18T03:41:19.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:19 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:19.182+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:46.820033+0000 front 2023-07-18T03:40:46.820086+0000 (oldest deadline 2023-07-18T03:41:10.319945+0000) 2023-07-18T03:41:19.548 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:19 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:19.182+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.921232+0000 front 2023-07-18T03:40:54.921268+0000 (oldest deadline 2023-07-18T03:41:18.921985+0000) 2023-07-18T03:41:19.738 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:19 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:19.245+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:41:19.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:19 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:19.245+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.543459+0000 front 2023-07-18T03:40:32.543608+0000 (oldest deadline 2023-07-18T03:40:56.643373+0000) 2023-07-18T03:41:19.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:19 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:19.245+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:37.744902+0000 front 2023-07-18T03:40:37.744760+0000 (oldest deadline 2023-07-18T03:41:01.844955+0000) 2023-07-18T03:41:19.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:19 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:19.245+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:48.047789+0000 front 2023-07-18T03:40:48.047719+0000 (oldest deadline 2023-07-18T03:41:13.947754+0000) 2023-07-18T03:41:20.080 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:19 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:19.734+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.220319+0000 front 2023-07-18T03:40:24.220087+0000 (oldest deadline 2023-07-18T03:40:48.920267+0000) 2023-07-18T03:41:20.081 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:19 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:19.734+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.322385+0000 front 2023-07-18T03:40:32.322477+0000 (oldest deadline 2023-07-18T03:40:58.222425+0000) 2023-07-18T03:41:20.081 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:19 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:19.734+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:39.923305+0000 front 2023-07-18T03:40:39.923435+0000 (oldest deadline 2023-07-18T03:41:04.623543+0000) 2023-07-18T03:41:20.081 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:19 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:19.734+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:53.226333+0000 front 2023-07-18T03:40:53.226216+0000 (oldest deadline 2023-07-18T03:41:19.126337+0000) 2023-07-18T03:41:20.450 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:20 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:20.087+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:20.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:20 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:20.087+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:20.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:20 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:20.087+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:20.488 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:20 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:20.211+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:41:20.489 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:20 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:20.211+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.543459+0000 front 2023-07-18T03:40:32.543608+0000 (oldest deadline 2023-07-18T03:40:56.643373+0000) 2023-07-18T03:41:20.489 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:20 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:20.211+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:37.744902+0000 front 2023-07-18T03:40:37.744760+0000 (oldest deadline 2023-07-18T03:41:01.844955+0000) 2023-07-18T03:41:20.489 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:20 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:20.211+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:48.047789+0000 front 2023-07-18T03:40:48.047719+0000 (oldest deadline 2023-07-18T03:41:13.947754+0000) 2023-07-18T03:41:20.546 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:20 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:20.402+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:30.409213+0000 front 2023-07-18T03:40:30.409292+0000 (oldest deadline 2023-07-18T03:40:55.109366+0000) 2023-07-18T03:41:20.546 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:20 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:20.402+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.810741+0000 front 2023-07-18T03:40:36.810842+0000 (oldest deadline 2023-07-18T03:41:02.110531+0000) 2023-07-18T03:41:20.547 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:20 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:20.402+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:46.712404+0000 front 2023-07-18T03:40:46.712242+0000 (oldest deadline 2023-07-18T03:41:09.612355+0000) 2023-07-18T03:41:20.547 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:20 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7[144915]: 2023-07-18T03:41:20.402+0000 7f236ca7c700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.714541+0000 front 2023-07-18T03:40:54.714570+0000 (oldest deadline 2023-07-18T03:41:17.614746+0000) 2023-07-18T03:41:20.548 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:20 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:20.175+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:41:20.548 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:20 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:20.175+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:41:20.549 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:20 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:20.175+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.718618+0000 front 2023-07-18T03:40:38.718740+0000 (oldest deadline 2023-07-18T03:41:02.818660+0000) 2023-07-18T03:41:20.549 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:20 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:20.175+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:46.820033+0000 front 2023-07-18T03:40:46.820086+0000 (oldest deadline 2023-07-18T03:41:10.319945+0000) 2023-07-18T03:41:20.550 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:20 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:20.175+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.921232+0000 front 2023-07-18T03:40:54.921268+0000 (oldest deadline 2023-07-18T03:41:18.921985+0000) 2023-07-18T03:41:20.950 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:20 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:20.701+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.220319+0000 front 2023-07-18T03:40:24.220087+0000 (oldest deadline 2023-07-18T03:40:48.920267+0000) 2023-07-18T03:41:20.951 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:20 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:20.701+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.322385+0000 front 2023-07-18T03:40:32.322477+0000 (oldest deadline 2023-07-18T03:40:58.222425+0000) 2023-07-18T03:41:20.951 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:20 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:20.701+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:39.923305+0000 front 2023-07-18T03:40:39.923435+0000 (oldest deadline 2023-07-18T03:41:04.623543+0000) 2023-07-18T03:41:20.951 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:20 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:20.701+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:53.226333+0000 front 2023-07-18T03:40:53.226216+0000 (oldest deadline 2023-07-18T03:41:19.126337+0000) 2023-07-18T03:41:21.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:21 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:21.042+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:21.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:21 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:21.042+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:21.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:21 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:21.042+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:21.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:21 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:21.042+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.239591+0000 front 2023-07-18T03:40:54.239502+0000 (oldest deadline 2023-07-18T03:41:20.139603+0000) 2023-07-18T03:41:21.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:21 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:21.198+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:41:21.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:21 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:21.198+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:41:21.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:21 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:21.198+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.718618+0000 front 2023-07-18T03:40:38.718740+0000 (oldest deadline 2023-07-18T03:41:02.818660+0000) 2023-07-18T03:41:21.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:21 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:21.198+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:46.820033+0000 front 2023-07-18T03:40:46.820086+0000 (oldest deadline 2023-07-18T03:41:10.319945+0000) 2023-07-18T03:41:21.548 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:21 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:21.198+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.921232+0000 front 2023-07-18T03:40:54.921268+0000 (oldest deadline 2023-07-18T03:41:18.921985+0000) 2023-07-18T03:41:21.738 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:21 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:21.255+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:41:21.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:21 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:21.256+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.543459+0000 front 2023-07-18T03:40:32.543608+0000 (oldest deadline 2023-07-18T03:40:56.643373+0000) 2023-07-18T03:41:21.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:21 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:21.256+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:37.744902+0000 front 2023-07-18T03:40:37.744760+0000 (oldest deadline 2023-07-18T03:41:01.844955+0000) 2023-07-18T03:41:21.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:21 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:21.256+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:48.047789+0000 front 2023-07-18T03:40:48.047719+0000 (oldest deadline 2023-07-18T03:41:13.947754+0000) 2023-07-18T03:41:21.950 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:21 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:21.698+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.220319+0000 front 2023-07-18T03:40:24.220087+0000 (oldest deadline 2023-07-18T03:40:48.920267+0000) 2023-07-18T03:41:21.951 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:21 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:21.698+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.322385+0000 front 2023-07-18T03:40:32.322477+0000 (oldest deadline 2023-07-18T03:40:58.222425+0000) 2023-07-18T03:41:21.951 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:21 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:21.698+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:39.923305+0000 front 2023-07-18T03:40:39.923435+0000 (oldest deadline 2023-07-18T03:41:04.623543+0000) 2023-07-18T03:41:21.951 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:21 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:21.698+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:53.226333+0000 front 2023-07-18T03:40:53.226216+0000 (oldest deadline 2023-07-18T03:41:19.126337+0000) 2023-07-18T03:41:22.046 INFO:journalctl@ceph.osd.7.smithi155.stdout:Jul 18 03:41:21 smithi155 bash[167415]: ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-7 2023-07-18T03:41:22.450 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:22 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:22.080+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:22.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:22 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:22.080+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:22.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:22 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:22.080+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:22.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:22 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:22.080+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.239591+0000 front 2023-07-18T03:40:54.239502+0000 (oldest deadline 2023-07-18T03:41:20.139603+0000) 2023-07-18T03:41:22.545 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:22 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:22.153+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:41:22.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:22 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:22.153+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:41:22.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:22 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:22.153+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.718618+0000 front 2023-07-18T03:40:38.718740+0000 (oldest deadline 2023-07-18T03:41:02.818660+0000) 2023-07-18T03:41:22.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:22 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:22.153+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:46.820033+0000 front 2023-07-18T03:40:46.820086+0000 (oldest deadline 2023-07-18T03:41:10.319945+0000) 2023-07-18T03:41:22.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:22 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:22.153+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.921232+0000 front 2023-07-18T03:40:54.921268+0000 (oldest deadline 2023-07-18T03:41:18.921985+0000) 2023-07-18T03:41:22.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:22 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:22.273+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:41:22.740 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:22 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:22.273+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.543459+0000 front 2023-07-18T03:40:32.543608+0000 (oldest deadline 2023-07-18T03:40:56.643373+0000) 2023-07-18T03:41:22.740 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:22 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:22.273+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:37.744902+0000 front 2023-07-18T03:40:37.744760+0000 (oldest deadline 2023-07-18T03:41:01.844955+0000) 2023-07-18T03:41:22.741 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:22 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:22.273+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:48.047789+0000 front 2023-07-18T03:40:48.047719+0000 (oldest deadline 2023-07-18T03:41:13.947754+0000) 2023-07-18T03:41:22.741 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:22 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:22.273+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:56.749796+0000 front 2023-07-18T03:40:56.749944+0000 (oldest deadline 2023-07-18T03:41:21.950347+0000) 2023-07-18T03:41:22.943 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@osd.7.service' 2023-07-18T03:41:22.950 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:22 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:22.707+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.220319+0000 front 2023-07-18T03:40:24.220087+0000 (oldest deadline 2023-07-18T03:40:48.920267+0000) 2023-07-18T03:41:22.951 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:22 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:22.707+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.322385+0000 front 2023-07-18T03:40:32.322477+0000 (oldest deadline 2023-07-18T03:40:58.222425+0000) 2023-07-18T03:41:22.951 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:22 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:22.707+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:39.923305+0000 front 2023-07-18T03:40:39.923435+0000 (oldest deadline 2023-07-18T03:41:04.623543+0000) 2023-07-18T03:41:22.951 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:22 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:22.707+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:53.226333+0000 front 2023-07-18T03:40:53.226216+0000 (oldest deadline 2023-07-18T03:41:19.126337+0000) 2023-07-18T03:41:23.022 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:41:23.023 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-07-18T03:41:23.023 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-07-18T03:41:23.024 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-b1782daa-2516-11ee-9b34-001a4aab830c@osd.8 2023-07-18T03:41:23.296 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:23 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:23.160+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:41:23.296 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:23 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:23.160+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:41:23.296 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:23 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:23.160+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.718618+0000 front 2023-07-18T03:40:38.718740+0000 (oldest deadline 2023-07-18T03:41:02.818660+0000) 2023-07-18T03:41:23.296 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:23 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:23.160+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:46.820033+0000 front 2023-07-18T03:40:46.820086+0000 (oldest deadline 2023-07-18T03:41:10.319945+0000) 2023-07-18T03:41:23.297 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:23 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:23.160+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.921232+0000 front 2023-07-18T03:40:54.921268+0000 (oldest deadline 2023-07-18T03:41:18.921985+0000) 2023-07-18T03:41:23.371 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:23 smithi179 systemd[1]: Stopping Ceph osd.8 for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T03:41:23.372 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:23 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:23.377+0000 7f046410c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:41:23.372 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:23 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:23.377+0000 7f046410c700 -1 osd.8 149 *** Got signal Terminated *** 2023-07-18T03:41:23.372 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:23 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:23.119+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:23.372 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:23 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:23.119+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:23.373 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:23 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:23.119+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:23.373 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:23 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:23.119+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.239591+0000 front 2023-07-18T03:40:54.239502+0000 (oldest deadline 2023-07-18T03:41:20.139603+0000) 2023-07-18T03:41:23.655 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:23 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:23.377+0000 7f046410c700 -1 osd.8 149 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-18T03:41:23.738 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:23 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:23.278+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:41:23.738 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:23 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:23.278+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.543459+0000 front 2023-07-18T03:40:32.543608+0000 (oldest deadline 2023-07-18T03:40:56.643373+0000) 2023-07-18T03:41:23.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:23 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:23.278+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:37.744902+0000 front 2023-07-18T03:40:37.744760+0000 (oldest deadline 2023-07-18T03:41:01.844955+0000) 2023-07-18T03:41:23.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:23 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:23.278+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:48.047789+0000 front 2023-07-18T03:40:48.047719+0000 (oldest deadline 2023-07-18T03:41:13.947754+0000) 2023-07-18T03:41:23.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:23 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:23.278+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:56.749796+0000 front 2023-07-18T03:40:56.749944+0000 (oldest deadline 2023-07-18T03:41:21.950347+0000) 2023-07-18T03:41:23.950 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:23 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:23.662+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.220319+0000 front 2023-07-18T03:40:24.220087+0000 (oldest deadline 2023-07-18T03:40:48.920267+0000) 2023-07-18T03:41:23.951 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:23 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:23.662+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.322385+0000 front 2023-07-18T03:40:32.322477+0000 (oldest deadline 2023-07-18T03:40:58.222425+0000) 2023-07-18T03:41:23.951 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:23 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:23.662+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:39.923305+0000 front 2023-07-18T03:40:39.923435+0000 (oldest deadline 2023-07-18T03:41:04.623543+0000) 2023-07-18T03:41:23.951 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:23 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:23.662+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:53.226333+0000 front 2023-07-18T03:40:53.226216+0000 (oldest deadline 2023-07-18T03:41:19.126337+0000) 2023-07-18T03:41:24.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:24 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:24.138+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:24.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:24 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:24.138+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:24.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:24 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:24.138+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:24.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:24 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:24.138+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.239591+0000 front 2023-07-18T03:40:54.239502+0000 (oldest deadline 2023-07-18T03:41:20.139603+0000) 2023-07-18T03:41:24.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:24 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:24.138+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:41:24.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:24 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:24.138+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:41:24.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:24 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:24.138+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.718618+0000 front 2023-07-18T03:40:38.718740+0000 (oldest deadline 2023-07-18T03:41:02.818660+0000) 2023-07-18T03:41:24.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:24 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:24.138+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:46.820033+0000 front 2023-07-18T03:40:46.820086+0000 (oldest deadline 2023-07-18T03:41:10.319945+0000) 2023-07-18T03:41:24.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:24 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:24.138+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.921232+0000 front 2023-07-18T03:40:54.921268+0000 (oldest deadline 2023-07-18T03:41:18.921985+0000) 2023-07-18T03:41:24.738 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:24 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:24.246+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:41:24.738 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:24 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:24.246+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.543459+0000 front 2023-07-18T03:40:32.543608+0000 (oldest deadline 2023-07-18T03:40:56.643373+0000) 2023-07-18T03:41:24.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:24 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:24.246+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:37.744902+0000 front 2023-07-18T03:40:37.744760+0000 (oldest deadline 2023-07-18T03:41:01.844955+0000) 2023-07-18T03:41:24.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:24 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:24.246+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:48.047789+0000 front 2023-07-18T03:40:48.047719+0000 (oldest deadline 2023-07-18T03:41:13.947754+0000) 2023-07-18T03:41:24.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:24 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:24.246+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:56.749796+0000 front 2023-07-18T03:40:56.749944+0000 (oldest deadline 2023-07-18T03:41:21.950347+0000) 2023-07-18T03:41:24.951 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:24 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:24.697+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.220319+0000 front 2023-07-18T03:40:24.220087+0000 (oldest deadline 2023-07-18T03:40:48.920267+0000) 2023-07-18T03:41:24.951 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:24 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:24.697+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.322385+0000 front 2023-07-18T03:40:32.322477+0000 (oldest deadline 2023-07-18T03:40:58.222425+0000) 2023-07-18T03:41:24.952 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:24 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:24.697+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:39.923305+0000 front 2023-07-18T03:40:39.923435+0000 (oldest deadline 2023-07-18T03:41:04.623543+0000) 2023-07-18T03:41:24.952 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:24 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:24.697+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:53.226333+0000 front 2023-07-18T03:40:53.226216+0000 (oldest deadline 2023-07-18T03:41:19.126337+0000) 2023-07-18T03:41:25.450 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:25 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:25.151+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:25.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:25 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:25.151+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:25.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:25 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:25.151+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:25.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:25 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:25.151+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.239591+0000 front 2023-07-18T03:40:54.239502+0000 (oldest deadline 2023-07-18T03:41:20.139603+0000) 2023-07-18T03:41:25.545 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:25 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:25.174+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:41:25.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:25 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:25.174+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:41:25.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:25 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:25.174+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.718618+0000 front 2023-07-18T03:40:38.718740+0000 (oldest deadline 2023-07-18T03:41:02.818660+0000) 2023-07-18T03:41:25.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:25 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:25.174+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:46.820033+0000 front 2023-07-18T03:40:46.820086+0000 (oldest deadline 2023-07-18T03:41:10.319945+0000) 2023-07-18T03:41:25.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:25 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:25.174+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.921232+0000 front 2023-07-18T03:40:54.921268+0000 (oldest deadline 2023-07-18T03:41:18.921985+0000) 2023-07-18T03:41:25.738 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:25 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:25.253+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:41:25.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:25 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:25.253+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.543459+0000 front 2023-07-18T03:40:32.543608+0000 (oldest deadline 2023-07-18T03:40:56.643373+0000) 2023-07-18T03:41:25.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:25 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:25.253+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:37.744902+0000 front 2023-07-18T03:40:37.744760+0000 (oldest deadline 2023-07-18T03:41:01.844955+0000) 2023-07-18T03:41:25.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:25 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:25.253+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:48.047789+0000 front 2023-07-18T03:40:48.047719+0000 (oldest deadline 2023-07-18T03:41:13.947754+0000) 2023-07-18T03:41:25.740 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:25 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:25.253+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:56.749796+0000 front 2023-07-18T03:40:56.749944+0000 (oldest deadline 2023-07-18T03:41:21.950347+0000) 2023-07-18T03:41:25.950 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:25 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:25.671+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.220319+0000 front 2023-07-18T03:40:24.220087+0000 (oldest deadline 2023-07-18T03:40:48.920267+0000) 2023-07-18T03:41:25.951 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:25 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:25.671+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.322385+0000 front 2023-07-18T03:40:32.322477+0000 (oldest deadline 2023-07-18T03:40:58.222425+0000) 2023-07-18T03:41:25.951 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:25 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:25.671+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:39.923305+0000 front 2023-07-18T03:40:39.923435+0000 (oldest deadline 2023-07-18T03:41:04.623543+0000) 2023-07-18T03:41:25.951 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:25 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:25.671+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:53.226333+0000 front 2023-07-18T03:40:53.226216+0000 (oldest deadline 2023-07-18T03:41:19.126337+0000) 2023-07-18T03:41:26.450 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:26 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:26.146+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:26.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:26 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:26.146+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:26.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:26 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:26.146+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:26.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:26 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:26.146+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.239591+0000 front 2023-07-18T03:40:54.239502+0000 (oldest deadline 2023-07-18T03:41:20.139603+0000) 2023-07-18T03:41:26.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:26 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:26.214+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:41:26.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:26 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:26.214+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:41:26.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:26 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:26.214+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.718618+0000 front 2023-07-18T03:40:38.718740+0000 (oldest deadline 2023-07-18T03:41:02.818660+0000) 2023-07-18T03:41:26.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:26 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:26.214+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:46.820033+0000 front 2023-07-18T03:40:46.820086+0000 (oldest deadline 2023-07-18T03:41:10.319945+0000) 2023-07-18T03:41:26.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:26 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:26.214+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.921232+0000 front 2023-07-18T03:40:54.921268+0000 (oldest deadline 2023-07-18T03:41:18.921985+0000) 2023-07-18T03:41:26.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:26 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:26.214+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:02.423041+0000 front 2023-07-18T03:41:02.422948+0000 (oldest deadline 2023-07-18T03:41:25.322779+0000) 2023-07-18T03:41:26.738 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:26 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:26.269+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:41:26.738 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:26 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:26.269+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.543459+0000 front 2023-07-18T03:40:32.543608+0000 (oldest deadline 2023-07-18T03:40:56.643373+0000) 2023-07-18T03:41:26.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:26 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:26.269+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:37.744902+0000 front 2023-07-18T03:40:37.744760+0000 (oldest deadline 2023-07-18T03:41:01.844955+0000) 2023-07-18T03:41:26.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:26 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:26.269+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:48.047789+0000 front 2023-07-18T03:40:48.047719+0000 (oldest deadline 2023-07-18T03:41:13.947754+0000) 2023-07-18T03:41:26.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:26 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:26.269+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:56.749796+0000 front 2023-07-18T03:40:56.749944+0000 (oldest deadline 2023-07-18T03:41:21.950347+0000) 2023-07-18T03:41:26.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:26 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:26.269+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:01.951247+0000 front 2023-07-18T03:41:01.951093+0000 (oldest deadline 2023-07-18T03:41:25.450805+0000) 2023-07-18T03:41:26.951 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:26 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:26.653+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.220319+0000 front 2023-07-18T03:40:24.220087+0000 (oldest deadline 2023-07-18T03:40:48.920267+0000) 2023-07-18T03:41:26.951 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:26 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:26.653+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.322385+0000 front 2023-07-18T03:40:32.322477+0000 (oldest deadline 2023-07-18T03:40:58.222425+0000) 2023-07-18T03:41:26.951 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:26 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:26.653+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:39.923305+0000 front 2023-07-18T03:40:39.923435+0000 (oldest deadline 2023-07-18T03:41:04.623543+0000) 2023-07-18T03:41:26.951 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:26 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:26.653+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:53.226333+0000 front 2023-07-18T03:40:53.226216+0000 (oldest deadline 2023-07-18T03:41:19.126337+0000) 2023-07-18T03:41:26.952 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:26 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:26.653+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:00.827112+0000 front 2023-07-18T03:41:00.827130+0000 (oldest deadline 2023-07-18T03:41:26.627894+0000) 2023-07-18T03:41:27.450 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:27 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:27.138+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:27.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:27 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:27.138+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:27.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:27 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:27.138+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:27.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:27 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:27.138+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.239591+0000 front 2023-07-18T03:40:54.239502+0000 (oldest deadline 2023-07-18T03:41:20.139603+0000) 2023-07-18T03:41:27.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:27 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:27.217+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:41:27.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:27 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:27.217+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:41:27.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:27 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:27.217+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.718618+0000 front 2023-07-18T03:40:38.718740+0000 (oldest deadline 2023-07-18T03:41:02.818660+0000) 2023-07-18T03:41:27.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:27 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:27.217+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:46.820033+0000 front 2023-07-18T03:40:46.820086+0000 (oldest deadline 2023-07-18T03:41:10.319945+0000) 2023-07-18T03:41:27.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:27 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:27.217+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.921232+0000 front 2023-07-18T03:40:54.921268+0000 (oldest deadline 2023-07-18T03:41:18.921985+0000) 2023-07-18T03:41:27.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:27 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:27.217+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:02.423041+0000 front 2023-07-18T03:41:02.422948+0000 (oldest deadline 2023-07-18T03:41:25.322779+0000) 2023-07-18T03:41:27.738 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:27 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:27.275+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:41:27.738 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:27 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:27.275+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.543459+0000 front 2023-07-18T03:40:32.543608+0000 (oldest deadline 2023-07-18T03:40:56.643373+0000) 2023-07-18T03:41:27.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:27 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:27.275+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:37.744902+0000 front 2023-07-18T03:40:37.744760+0000 (oldest deadline 2023-07-18T03:41:01.844955+0000) 2023-07-18T03:41:27.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:27 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:27.275+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:48.047789+0000 front 2023-07-18T03:40:48.047719+0000 (oldest deadline 2023-07-18T03:41:13.947754+0000) 2023-07-18T03:41:27.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:27 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:27.275+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:56.749796+0000 front 2023-07-18T03:40:56.749944+0000 (oldest deadline 2023-07-18T03:41:21.950347+0000) 2023-07-18T03:41:27.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:27 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:27.275+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:01.951247+0000 front 2023-07-18T03:41:01.951093+0000 (oldest deadline 2023-07-18T03:41:25.450805+0000) 2023-07-18T03:41:27.950 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:27 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:27.664+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.220319+0000 front 2023-07-18T03:40:24.220087+0000 (oldest deadline 2023-07-18T03:40:48.920267+0000) 2023-07-18T03:41:27.951 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:27 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:27.664+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.322385+0000 front 2023-07-18T03:40:32.322477+0000 (oldest deadline 2023-07-18T03:40:58.222425+0000) 2023-07-18T03:41:27.951 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:27 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:27.664+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:39.923305+0000 front 2023-07-18T03:40:39.923435+0000 (oldest deadline 2023-07-18T03:41:04.623543+0000) 2023-07-18T03:41:27.951 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:27 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:27.664+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:53.226333+0000 front 2023-07-18T03:40:53.226216+0000 (oldest deadline 2023-07-18T03:41:19.126337+0000) 2023-07-18T03:41:27.951 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:27 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8[146041]: 2023-07-18T03:41:27.664+0000 7f045defd700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:00.827112+0000 front 2023-07-18T03:41:00.827130+0000 (oldest deadline 2023-07-18T03:41:26.627894+0000) 2023-07-18T03:41:28.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:28 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:28.120+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:28.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:28 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:28.120+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:28.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:28 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:28.120+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:28.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:28 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:28.120+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.239591+0000 front 2023-07-18T03:40:54.239502+0000 (oldest deadline 2023-07-18T03:41:20.139603+0000) 2023-07-18T03:41:28.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:28 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:28.120+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:02.940985+0000 front 2023-07-18T03:41:02.941034+0000 (oldest deadline 2023-07-18T03:41:27.641205+0000) 2023-07-18T03:41:28.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:28 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:28.169+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:41:28.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:28 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:28.169+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:41:28.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:28 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:28.169+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.718618+0000 front 2023-07-18T03:40:38.718740+0000 (oldest deadline 2023-07-18T03:41:02.818660+0000) 2023-07-18T03:41:28.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:28 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:28.169+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:46.820033+0000 front 2023-07-18T03:40:46.820086+0000 (oldest deadline 2023-07-18T03:41:10.319945+0000) 2023-07-18T03:41:28.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:28 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:28.169+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.921232+0000 front 2023-07-18T03:40:54.921268+0000 (oldest deadline 2023-07-18T03:41:18.921985+0000) 2023-07-18T03:41:28.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:28 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:28.169+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:02.423041+0000 front 2023-07-18T03:41:02.422948+0000 (oldest deadline 2023-07-18T03:41:25.322779+0000) 2023-07-18T03:41:28.738 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:28 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:28.271+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:41:28.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:28 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:28.271+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.543459+0000 front 2023-07-18T03:40:32.543608+0000 (oldest deadline 2023-07-18T03:40:56.643373+0000) 2023-07-18T03:41:28.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:28 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:28.271+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:37.744902+0000 front 2023-07-18T03:40:37.744760+0000 (oldest deadline 2023-07-18T03:41:01.844955+0000) 2023-07-18T03:41:28.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:28 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:28.271+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:48.047789+0000 front 2023-07-18T03:40:48.047719+0000 (oldest deadline 2023-07-18T03:41:13.947754+0000) 2023-07-18T03:41:28.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:28 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:28.271+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:56.749796+0000 front 2023-07-18T03:40:56.749944+0000 (oldest deadline 2023-07-18T03:41:21.950347+0000) 2023-07-18T03:41:28.740 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:28 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:28.271+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:01.951247+0000 front 2023-07-18T03:41:01.951093+0000 (oldest deadline 2023-07-18T03:41:25.450805+0000) 2023-07-18T03:41:29.450 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jul 18 03:41:29 smithi179 bash[167707]: ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-8 2023-07-18T03:41:29.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:29 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:29.090+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:29.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:29 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:29.090+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:29.453 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:29 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:29.090+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:29.453 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:29 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:29.090+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.239591+0000 front 2023-07-18T03:40:54.239502+0000 (oldest deadline 2023-07-18T03:41:20.139603+0000) 2023-07-18T03:41:29.453 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:29 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:29.090+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:02.940985+0000 front 2023-07-18T03:41:02.941034+0000 (oldest deadline 2023-07-18T03:41:27.641205+0000) 2023-07-18T03:41:29.545 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:29 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:29.191+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:41:29.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:29 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:29.191+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:41:29.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:29 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:29.191+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.718618+0000 front 2023-07-18T03:40:38.718740+0000 (oldest deadline 2023-07-18T03:41:02.818660+0000) 2023-07-18T03:41:29.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:29 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:29.191+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:46.820033+0000 front 2023-07-18T03:40:46.820086+0000 (oldest deadline 2023-07-18T03:41:10.319945+0000) 2023-07-18T03:41:29.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:29 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:29.191+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.921232+0000 front 2023-07-18T03:40:54.921268+0000 (oldest deadline 2023-07-18T03:41:18.921985+0000) 2023-07-18T03:41:29.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:29 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:29.191+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:02.423041+0000 front 2023-07-18T03:41:02.422948+0000 (oldest deadline 2023-07-18T03:41:25.322779+0000) 2023-07-18T03:41:29.738 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:29 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:29.304+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:41:29.738 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:29 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:29.304+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.543459+0000 front 2023-07-18T03:40:32.543608+0000 (oldest deadline 2023-07-18T03:40:56.643373+0000) 2023-07-18T03:41:29.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:29 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:29.304+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:37.744902+0000 front 2023-07-18T03:40:37.744760+0000 (oldest deadline 2023-07-18T03:41:01.844955+0000) 2023-07-18T03:41:29.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:29 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:29.304+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:48.047789+0000 front 2023-07-18T03:40:48.047719+0000 (oldest deadline 2023-07-18T03:41:13.947754+0000) 2023-07-18T03:41:29.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:29 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:29.304+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:56.749796+0000 front 2023-07-18T03:40:56.749944+0000 (oldest deadline 2023-07-18T03:41:21.950347+0000) 2023-07-18T03:41:29.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:29 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:29.304+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:01.951247+0000 front 2023-07-18T03:41:01.951093+0000 (oldest deadline 2023-07-18T03:41:25.450805+0000) 2023-07-18T03:41:30.200 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:30 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:30.090+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:30.201 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:30 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:30.090+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:30.201 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:30 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:30.090+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:30.201 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:30 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:30.090+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.239591+0000 front 2023-07-18T03:40:54.239502+0000 (oldest deadline 2023-07-18T03:41:20.139603+0000) 2023-07-18T03:41:30.201 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:30 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:30.090+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:02.940985+0000 front 2023-07-18T03:41:02.941034+0000 (oldest deadline 2023-07-18T03:41:27.641205+0000) 2023-07-18T03:41:30.545 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:30 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:30.145+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:41:30.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:30 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:30.145+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:41:30.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:30 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:30.145+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.718618+0000 front 2023-07-18T03:40:38.718740+0000 (oldest deadline 2023-07-18T03:41:02.818660+0000) 2023-07-18T03:41:30.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:30 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:30.145+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:46.820033+0000 front 2023-07-18T03:40:46.820086+0000 (oldest deadline 2023-07-18T03:41:10.319945+0000) 2023-07-18T03:41:30.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:30 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:30.145+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.921232+0000 front 2023-07-18T03:40:54.921268+0000 (oldest deadline 2023-07-18T03:41:18.921985+0000) 2023-07-18T03:41:30.548 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:30 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:30.145+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:02.423041+0000 front 2023-07-18T03:41:02.422948+0000 (oldest deadline 2023-07-18T03:41:25.322779+0000) 2023-07-18T03:41:30.637 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@osd.8.service' 2023-07-18T03:41:30.677 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:41:30.678 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-07-18T03:41:30.678 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-07-18T03:41:30.679 DEBUG:teuthology.orchestra.run.smithi101:> sudo systemctl stop ceph-b1782daa-2516-11ee-9b34-001a4aab830c@osd.9 2023-07-18T03:41:30.738 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:30 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:30.339+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:41:30.738 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:30 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:30.339+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.543459+0000 front 2023-07-18T03:40:32.543608+0000 (oldest deadline 2023-07-18T03:40:56.643373+0000) 2023-07-18T03:41:30.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:30 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:30.339+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:37.744902+0000 front 2023-07-18T03:40:37.744760+0000 (oldest deadline 2023-07-18T03:41:01.844955+0000) 2023-07-18T03:41:30.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:30 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:30.339+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:48.047789+0000 front 2023-07-18T03:40:48.047719+0000 (oldest deadline 2023-07-18T03:41:13.947754+0000) 2023-07-18T03:41:30.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:30 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:30.339+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:56.749796+0000 front 2023-07-18T03:40:56.749944+0000 (oldest deadline 2023-07-18T03:41:21.950347+0000) 2023-07-18T03:41:30.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:30 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:30.339+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:01.951247+0000 front 2023-07-18T03:41:01.951093+0000 (oldest deadline 2023-07-18T03:41:25.450805+0000) 2023-07-18T03:41:31.192 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:30 smithi101 systemd[1]: Stopping Ceph osd.9 for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T03:41:31.193 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:31 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:31.197+0000 7f3fce91e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:41:31.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:31 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:31.097+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:31.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:31 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:31.097+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:31.453 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:31 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:31.097+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:31.453 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:31 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:31.097+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.239591+0000 front 2023-07-18T03:40:54.239502+0000 (oldest deadline 2023-07-18T03:41:20.139603+0000) 2023-07-18T03:41:31.453 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:31 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:31.097+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:02.940985+0000 front 2023-07-18T03:41:02.941034+0000 (oldest deadline 2023-07-18T03:41:27.641205+0000) 2023-07-18T03:41:31.488 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:31 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:31.197+0000 7f3fce91e700 -1 osd.9 149 *** Got signal Terminated *** 2023-07-18T03:41:31.489 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:31 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:31.197+0000 7f3fce91e700 -1 osd.9 149 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-18T03:41:31.489 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:31 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:31.381+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:41:31.489 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:31 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:31.381+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.543459+0000 front 2023-07-18T03:40:32.543608+0000 (oldest deadline 2023-07-18T03:40:56.643373+0000) 2023-07-18T03:41:31.490 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:31 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:31.381+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:37.744902+0000 front 2023-07-18T03:40:37.744760+0000 (oldest deadline 2023-07-18T03:41:01.844955+0000) 2023-07-18T03:41:31.490 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:31 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:31.381+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:48.047789+0000 front 2023-07-18T03:40:48.047719+0000 (oldest deadline 2023-07-18T03:41:13.947754+0000) 2023-07-18T03:41:31.490 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:31 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:31.381+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:56.749796+0000 front 2023-07-18T03:40:56.749944+0000 (oldest deadline 2023-07-18T03:41:21.950347+0000) 2023-07-18T03:41:31.491 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:31 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:31.381+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:01.951247+0000 front 2023-07-18T03:41:01.951093+0000 (oldest deadline 2023-07-18T03:41:25.450805+0000) 2023-07-18T03:41:31.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:31 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:31.110+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:41:31.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:31 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:31.110+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:41:31.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:31 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:31.110+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.718618+0000 front 2023-07-18T03:40:38.718740+0000 (oldest deadline 2023-07-18T03:41:02.818660+0000) 2023-07-18T03:41:31.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:31 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:31.110+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:46.820033+0000 front 2023-07-18T03:40:46.820086+0000 (oldest deadline 2023-07-18T03:41:10.319945+0000) 2023-07-18T03:41:31.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:31 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:31.110+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.921232+0000 front 2023-07-18T03:40:54.921268+0000 (oldest deadline 2023-07-18T03:41:18.921985+0000) 2023-07-18T03:41:31.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:31 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:31.110+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:02.423041+0000 front 2023-07-18T03:41:02.422948+0000 (oldest deadline 2023-07-18T03:41:25.322779+0000) 2023-07-18T03:41:32.450 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:32 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:32.121+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:32.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:32 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:32.121+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:32.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:32 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:32.121+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:32.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:32 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:32.121+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.239591+0000 front 2023-07-18T03:40:54.239502+0000 (oldest deadline 2023-07-18T03:41:20.139603+0000) 2023-07-18T03:41:32.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:32 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:32.121+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:02.940985+0000 front 2023-07-18T03:41:02.941034+0000 (oldest deadline 2023-07-18T03:41:27.641205+0000) 2023-07-18T03:41:32.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:32 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:32.114+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:41:32.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:32 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:32.114+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:41:32.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:32 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:32.114+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.718618+0000 front 2023-07-18T03:40:38.718740+0000 (oldest deadline 2023-07-18T03:41:02.818660+0000) 2023-07-18T03:41:32.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:32 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:32.114+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:46.820033+0000 front 2023-07-18T03:40:46.820086+0000 (oldest deadline 2023-07-18T03:41:10.319945+0000) 2023-07-18T03:41:32.548 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:32 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:32.114+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.921232+0000 front 2023-07-18T03:40:54.921268+0000 (oldest deadline 2023-07-18T03:41:18.921985+0000) 2023-07-18T03:41:32.548 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:32 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:32.114+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:02.423041+0000 front 2023-07-18T03:41:02.422948+0000 (oldest deadline 2023-07-18T03:41:25.322779+0000) 2023-07-18T03:41:32.738 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:32 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:32.372+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:41:32.738 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:32 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:32.372+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.543459+0000 front 2023-07-18T03:40:32.543608+0000 (oldest deadline 2023-07-18T03:40:56.643373+0000) 2023-07-18T03:41:32.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:32 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:32.372+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:37.744902+0000 front 2023-07-18T03:40:37.744760+0000 (oldest deadline 2023-07-18T03:41:01.844955+0000) 2023-07-18T03:41:32.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:32 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:32.372+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:48.047789+0000 front 2023-07-18T03:40:48.047719+0000 (oldest deadline 2023-07-18T03:41:13.947754+0000) 2023-07-18T03:41:32.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:32 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:32.372+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:56.749796+0000 front 2023-07-18T03:40:56.749944+0000 (oldest deadline 2023-07-18T03:41:21.950347+0000) 2023-07-18T03:41:32.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:32 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:32.372+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:01.951247+0000 front 2023-07-18T03:41:01.951093+0000 (oldest deadline 2023-07-18T03:41:25.450805+0000) 2023-07-18T03:41:33.450 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:33 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:33.099+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:33.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:33 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:33.099+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:33.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:33 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:33.099+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:33.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:33 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:33.100+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.239591+0000 front 2023-07-18T03:40:54.239502+0000 (oldest deadline 2023-07-18T03:41:20.139603+0000) 2023-07-18T03:41:33.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:33 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:33.100+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:02.940985+0000 front 2023-07-18T03:41:02.941034+0000 (oldest deadline 2023-07-18T03:41:27.641205+0000) 2023-07-18T03:41:33.545 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:33 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:33.108+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:41:33.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:33 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:33.108+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:41:33.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:33 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:33.108+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.718618+0000 front 2023-07-18T03:40:38.718740+0000 (oldest deadline 2023-07-18T03:41:02.818660+0000) 2023-07-18T03:41:33.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:33 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:33.108+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:46.820033+0000 front 2023-07-18T03:40:46.820086+0000 (oldest deadline 2023-07-18T03:41:10.319945+0000) 2023-07-18T03:41:33.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:33 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:33.108+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.921232+0000 front 2023-07-18T03:40:54.921268+0000 (oldest deadline 2023-07-18T03:41:18.921985+0000) 2023-07-18T03:41:33.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:33 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:33.108+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:02.423041+0000 front 2023-07-18T03:41:02.422948+0000 (oldest deadline 2023-07-18T03:41:25.322779+0000) 2023-07-18T03:41:33.738 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:33 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:33.410+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:41:33.738 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:33 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:33.410+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.543459+0000 front 2023-07-18T03:40:32.543608+0000 (oldest deadline 2023-07-18T03:40:56.643373+0000) 2023-07-18T03:41:33.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:33 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:33.410+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:37.744902+0000 front 2023-07-18T03:40:37.744760+0000 (oldest deadline 2023-07-18T03:41:01.844955+0000) 2023-07-18T03:41:33.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:33 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:33.410+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:48.047789+0000 front 2023-07-18T03:40:48.047719+0000 (oldest deadline 2023-07-18T03:41:13.947754+0000) 2023-07-18T03:41:33.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:33 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:33.410+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:56.749796+0000 front 2023-07-18T03:40:56.749944+0000 (oldest deadline 2023-07-18T03:41:21.950347+0000) 2023-07-18T03:41:33.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:33 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:33.410+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:01.951247+0000 front 2023-07-18T03:41:01.951093+0000 (oldest deadline 2023-07-18T03:41:25.450805+0000) 2023-07-18T03:41:34.450 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:34 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:34.142+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:34.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:34 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:34.142+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:34.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:34 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:34.142+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:34.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:34 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:34.142+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.239591+0000 front 2023-07-18T03:40:54.239502+0000 (oldest deadline 2023-07-18T03:41:20.139603+0000) 2023-07-18T03:41:34.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:34 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:34.142+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:02.940985+0000 front 2023-07-18T03:41:02.941034+0000 (oldest deadline 2023-07-18T03:41:27.641205+0000) 2023-07-18T03:41:34.545 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:34 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:34.123+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:41:34.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:34 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:34.123+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:41:34.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:34 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:34.123+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.718618+0000 front 2023-07-18T03:40:38.718740+0000 (oldest deadline 2023-07-18T03:41:02.818660+0000) 2023-07-18T03:41:34.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:34 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:34.123+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:46.820033+0000 front 2023-07-18T03:40:46.820086+0000 (oldest deadline 2023-07-18T03:41:10.319945+0000) 2023-07-18T03:41:34.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:34 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:34.123+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.921232+0000 front 2023-07-18T03:40:54.921268+0000 (oldest deadline 2023-07-18T03:41:18.921985+0000) 2023-07-18T03:41:34.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:34 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:34.123+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:02.423041+0000 front 2023-07-18T03:41:02.422948+0000 (oldest deadline 2023-07-18T03:41:25.322779+0000) 2023-07-18T03:41:34.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:34 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:34.123+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6822 osd.6 since back 2023-07-18T03:41:11.724412+0000 front 2023-07-18T03:41:11.724566+0000 (oldest deadline 2023-07-18T03:41:33.424161+0000) 2023-07-18T03:41:34.738 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:34 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:34.384+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:41:34.738 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:34 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:34.384+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.543459+0000 front 2023-07-18T03:40:32.543608+0000 (oldest deadline 2023-07-18T03:40:56.643373+0000) 2023-07-18T03:41:34.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:34 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:34.384+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:37.744902+0000 front 2023-07-18T03:40:37.744760+0000 (oldest deadline 2023-07-18T03:41:01.844955+0000) 2023-07-18T03:41:34.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:34 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:34.384+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:48.047789+0000 front 2023-07-18T03:40:48.047719+0000 (oldest deadline 2023-07-18T03:41:13.947754+0000) 2023-07-18T03:41:34.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:34 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:34.384+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:56.749796+0000 front 2023-07-18T03:40:56.749944+0000 (oldest deadline 2023-07-18T03:41:21.950347+0000) 2023-07-18T03:41:34.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:34 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:34.384+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:01.951247+0000 front 2023-07-18T03:41:01.951093+0000 (oldest deadline 2023-07-18T03:41:25.450805+0000) 2023-07-18T03:41:35.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:35 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:35.139+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:35.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:35 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:35.139+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:35.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:35 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:35.139+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:35.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:35 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:35.139+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.239591+0000 front 2023-07-18T03:40:54.239502+0000 (oldest deadline 2023-07-18T03:41:20.139603+0000) 2023-07-18T03:41:35.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:35 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:35.139+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:02.940985+0000 front 2023-07-18T03:41:02.941034+0000 (oldest deadline 2023-07-18T03:41:27.641205+0000) 2023-07-18T03:41:35.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:35 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:35.137+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:41:35.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:35 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:35.137+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:41:35.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:35 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:35.137+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.718618+0000 front 2023-07-18T03:40:38.718740+0000 (oldest deadline 2023-07-18T03:41:02.818660+0000) 2023-07-18T03:41:35.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:35 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:35.137+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:46.820033+0000 front 2023-07-18T03:40:46.820086+0000 (oldest deadline 2023-07-18T03:41:10.319945+0000) 2023-07-18T03:41:35.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:35 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:35.137+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.921232+0000 front 2023-07-18T03:40:54.921268+0000 (oldest deadline 2023-07-18T03:41:18.921985+0000) 2023-07-18T03:41:35.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:35 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:35.137+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:02.423041+0000 front 2023-07-18T03:41:02.422948+0000 (oldest deadline 2023-07-18T03:41:25.322779+0000) 2023-07-18T03:41:35.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:35 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:35.137+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6822 osd.6 since back 2023-07-18T03:41:11.724412+0000 front 2023-07-18T03:41:11.724566+0000 (oldest deadline 2023-07-18T03:41:33.424161+0000) 2023-07-18T03:41:35.738 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:35 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:35.414+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.741850+0000 front 2023-07-18T03:40:23.742020+0000 (oldest deadline 2023-07-18T03:40:48.442060+0000) 2023-07-18T03:41:35.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:35 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:35.414+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:32.543459+0000 front 2023-07-18T03:40:32.543608+0000 (oldest deadline 2023-07-18T03:40:56.643373+0000) 2023-07-18T03:41:35.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:35 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:35.414+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:37.744902+0000 front 2023-07-18T03:40:37.744760+0000 (oldest deadline 2023-07-18T03:41:01.844955+0000) 2023-07-18T03:41:35.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:35 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:35.414+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:48.047789+0000 front 2023-07-18T03:40:48.047719+0000 (oldest deadline 2023-07-18T03:41:13.947754+0000) 2023-07-18T03:41:35.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:35 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:35.414+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:56.749796+0000 front 2023-07-18T03:40:56.749944+0000 (oldest deadline 2023-07-18T03:41:21.950347+0000) 2023-07-18T03:41:35.739 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:35 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9[155787]: 2023-07-18T03:41:35.414+0000 7f3fc8f10700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:01.951247+0000 front 2023-07-18T03:41:01.951093+0000 (oldest deadline 2023-07-18T03:41:25.450805+0000) 2023-07-18T03:41:36.450 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:36 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:36.111+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:36.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:36 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:36.111+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:36.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:36 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:36.111+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:36.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:36 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:36.111+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.239591+0000 front 2023-07-18T03:40:54.239502+0000 (oldest deadline 2023-07-18T03:41:20.139603+0000) 2023-07-18T03:41:36.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:36 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:36.111+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:02.940985+0000 front 2023-07-18T03:41:02.941034+0000 (oldest deadline 2023-07-18T03:41:27.641205+0000) 2023-07-18T03:41:36.545 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:36 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:36.091+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:41:36.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:36 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:36.091+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:41:36.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:36 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:36.091+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.718618+0000 front 2023-07-18T03:40:38.718740+0000 (oldest deadline 2023-07-18T03:41:02.818660+0000) 2023-07-18T03:41:36.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:36 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:36.091+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:46.820033+0000 front 2023-07-18T03:40:46.820086+0000 (oldest deadline 2023-07-18T03:41:10.319945+0000) 2023-07-18T03:41:36.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:36 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:36.091+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.921232+0000 front 2023-07-18T03:40:54.921268+0000 (oldest deadline 2023-07-18T03:41:18.921985+0000) 2023-07-18T03:41:36.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:36 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:36.091+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:02.423041+0000 front 2023-07-18T03:41:02.422948+0000 (oldest deadline 2023-07-18T03:41:25.322779+0000) 2023-07-18T03:41:36.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:36 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:36.091+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6822 osd.6 since back 2023-07-18T03:41:11.724412+0000 front 2023-07-18T03:41:11.724566+0000 (oldest deadline 2023-07-18T03:41:33.424161+0000) 2023-07-18T03:41:37.296 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:37 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:37.051+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:41:37.297 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:37 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:37.051+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:41:37.298 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:37 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:37.051+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.718618+0000 front 2023-07-18T03:40:38.718740+0000 (oldest deadline 2023-07-18T03:41:02.818660+0000) 2023-07-18T03:41:37.298 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:37 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:37.051+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:46.820033+0000 front 2023-07-18T03:40:46.820086+0000 (oldest deadline 2023-07-18T03:41:10.319945+0000) 2023-07-18T03:41:37.298 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:37 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:37.051+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.921232+0000 front 2023-07-18T03:40:54.921268+0000 (oldest deadline 2023-07-18T03:41:18.921985+0000) 2023-07-18T03:41:37.299 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:37 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:37.051+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:02.423041+0000 front 2023-07-18T03:41:02.422948+0000 (oldest deadline 2023-07-18T03:41:25.322779+0000) 2023-07-18T03:41:37.299 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:37 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:37.051+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6822 osd.6 since back 2023-07-18T03:41:11.724412+0000 front 2023-07-18T03:41:11.724566+0000 (oldest deadline 2023-07-18T03:41:33.424161+0000) 2023-07-18T03:41:37.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:37 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:37.069+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:37.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:37 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:37.069+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:37.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:37 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:37.069+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:37.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:37 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:37.069+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.239591+0000 front 2023-07-18T03:40:54.239502+0000 (oldest deadline 2023-07-18T03:41:20.139603+0000) 2023-07-18T03:41:37.453 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:37 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:37.069+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:02.940985+0000 front 2023-07-18T03:41:02.941034+0000 (oldest deadline 2023-07-18T03:41:27.641205+0000) 2023-07-18T03:41:37.488 INFO:journalctl@ceph.osd.9.smithi101.stdout:Jul 18 03:41:37 smithi101 bash[206482]: ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-9 2023-07-18T03:41:38.450 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:38 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:38.028+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:38.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:38 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:38.028+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:38.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:38 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:38.028+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:38.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:38 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:38.028+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.239591+0000 front 2023-07-18T03:40:54.239502+0000 (oldest deadline 2023-07-18T03:41:20.139603+0000) 2023-07-18T03:41:38.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:38 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:38.028+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:02.940985+0000 front 2023-07-18T03:41:02.941034+0000 (oldest deadline 2023-07-18T03:41:27.641205+0000) 2023-07-18T03:41:38.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:38 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:38.064+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:41:38.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:38 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:38.064+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:41:38.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:38 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:38.064+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.718618+0000 front 2023-07-18T03:40:38.718740+0000 (oldest deadline 2023-07-18T03:41:02.818660+0000) 2023-07-18T03:41:38.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:38 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:38.064+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:46.820033+0000 front 2023-07-18T03:40:46.820086+0000 (oldest deadline 2023-07-18T03:41:10.319945+0000) 2023-07-18T03:41:38.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:38 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:38.064+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.921232+0000 front 2023-07-18T03:40:54.921268+0000 (oldest deadline 2023-07-18T03:41:18.921985+0000) 2023-07-18T03:41:38.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:38 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:38.064+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:02.423041+0000 front 2023-07-18T03:41:02.422948+0000 (oldest deadline 2023-07-18T03:41:25.322779+0000) 2023-07-18T03:41:38.548 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:38 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:38.064+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6822 osd.6 since back 2023-07-18T03:41:11.724412+0000 front 2023-07-18T03:41:11.724566+0000 (oldest deadline 2023-07-18T03:41:33.424161+0000) 2023-07-18T03:41:39.103 DEBUG:teuthology.orchestra.run.smithi101:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@osd.9.service' 2023-07-18T03:41:39.192 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:41:39.193 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-07-18T03:41:39.193 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-07-18T03:41:39.194 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-b1782daa-2516-11ee-9b34-001a4aab830c@osd.10 2023-07-18T03:41:39.450 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:39 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:39.069+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:39.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:39 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:39.069+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:39.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:39 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:39.069+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:39.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:39 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:39.069+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.239591+0000 front 2023-07-18T03:40:54.239502+0000 (oldest deadline 2023-07-18T03:41:20.139603+0000) 2023-07-18T03:41:39.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:39 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:39.069+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:02.940985+0000 front 2023-07-18T03:41:02.941034+0000 (oldest deadline 2023-07-18T03:41:27.641205+0000) 2023-07-18T03:41:39.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:39 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:39.069+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6822 osd.6 since back 2023-07-18T03:41:07.641906+0000 front 2023-07-18T03:41:07.641794+0000 (oldest deadline 2023-07-18T03:41:38.242139+0000) 2023-07-18T03:41:39.545 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:39 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:39.108+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:41:39.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:39 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:39.108+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:41:39.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:39 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:39.108+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.718618+0000 front 2023-07-18T03:40:38.718740+0000 (oldest deadline 2023-07-18T03:41:02.818660+0000) 2023-07-18T03:41:39.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:39 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:39.108+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:46.820033+0000 front 2023-07-18T03:40:46.820086+0000 (oldest deadline 2023-07-18T03:41:10.319945+0000) 2023-07-18T03:41:39.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:39 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:39.108+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.921232+0000 front 2023-07-18T03:40:54.921268+0000 (oldest deadline 2023-07-18T03:41:18.921985+0000) 2023-07-18T03:41:39.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:39 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:39.108+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:02.423041+0000 front 2023-07-18T03:41:02.422948+0000 (oldest deadline 2023-07-18T03:41:25.322779+0000) 2023-07-18T03:41:39.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:39 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:39.108+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6822 osd.6 since back 2023-07-18T03:41:11.724412+0000 front 2023-07-18T03:41:11.724566+0000 (oldest deadline 2023-07-18T03:41:33.424161+0000) 2023-07-18T03:41:39.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:39 smithi155 systemd[1]: Stopping Ceph osd.10 for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T03:41:40.045 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:39 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:39.552+0000 7fa5a2e68700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:41:40.046 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:39 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:39.552+0000 7fa5a2e68700 -1 osd.10 149 *** Got signal Terminated *** 2023-07-18T03:41:40.046 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:39 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:39.552+0000 7fa5a2e68700 -1 osd.10 149 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-18T03:41:40.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:40 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:40.029+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:40.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:40 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:40.029+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:40.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:40 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:40.029+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:40.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:40 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:40.029+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.239591+0000 front 2023-07-18T03:40:54.239502+0000 (oldest deadline 2023-07-18T03:41:20.139603+0000) 2023-07-18T03:41:40.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:40 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:40.029+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:02.940985+0000 front 2023-07-18T03:41:02.941034+0000 (oldest deadline 2023-07-18T03:41:27.641205+0000) 2023-07-18T03:41:40.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:40 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:40.029+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6822 osd.6 since back 2023-07-18T03:41:07.641906+0000 front 2023-07-18T03:41:07.641794+0000 (oldest deadline 2023-07-18T03:41:38.242139+0000) 2023-07-18T03:41:40.545 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:40 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:40.085+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:41:40.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:40 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:40.085+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:41:40.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:40 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:40.085+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.718618+0000 front 2023-07-18T03:40:38.718740+0000 (oldest deadline 2023-07-18T03:41:02.818660+0000) 2023-07-18T03:41:40.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:40 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:40.085+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:46.820033+0000 front 2023-07-18T03:40:46.820086+0000 (oldest deadline 2023-07-18T03:41:10.319945+0000) 2023-07-18T03:41:40.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:40 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:40.085+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.921232+0000 front 2023-07-18T03:40:54.921268+0000 (oldest deadline 2023-07-18T03:41:18.921985+0000) 2023-07-18T03:41:40.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:40 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:40.085+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:02.423041+0000 front 2023-07-18T03:41:02.422948+0000 (oldest deadline 2023-07-18T03:41:25.322779+0000) 2023-07-18T03:41:40.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:40 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:40.085+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6822 osd.6 since back 2023-07-18T03:41:11.724412+0000 front 2023-07-18T03:41:11.724566+0000 (oldest deadline 2023-07-18T03:41:33.424161+0000) 2023-07-18T03:41:41.450 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:41 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:41.043+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:41.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:41 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:41.043+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:41.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:41 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:41.043+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:41.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:41 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:41.043+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.239591+0000 front 2023-07-18T03:40:54.239502+0000 (oldest deadline 2023-07-18T03:41:20.139603+0000) 2023-07-18T03:41:41.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:41 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:41.043+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:02.940985+0000 front 2023-07-18T03:41:02.941034+0000 (oldest deadline 2023-07-18T03:41:27.641205+0000) 2023-07-18T03:41:41.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:41 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:41.043+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6822 osd.6 since back 2023-07-18T03:41:07.641906+0000 front 2023-07-18T03:41:07.641794+0000 (oldest deadline 2023-07-18T03:41:38.242139+0000) 2023-07-18T03:41:41.545 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:41 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:41.099+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:41:41.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:41 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:41.099+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:41:41.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:41 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:41.099+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.718618+0000 front 2023-07-18T03:40:38.718740+0000 (oldest deadline 2023-07-18T03:41:02.818660+0000) 2023-07-18T03:41:41.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:41 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:41.099+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:46.820033+0000 front 2023-07-18T03:40:46.820086+0000 (oldest deadline 2023-07-18T03:41:10.319945+0000) 2023-07-18T03:41:41.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:41 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:41.099+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.921232+0000 front 2023-07-18T03:40:54.921268+0000 (oldest deadline 2023-07-18T03:41:18.921985+0000) 2023-07-18T03:41:41.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:41 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:41.099+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:02.423041+0000 front 2023-07-18T03:41:02.422948+0000 (oldest deadline 2023-07-18T03:41:25.322779+0000) 2023-07-18T03:41:41.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:41 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:41.099+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6822 osd.6 since back 2023-07-18T03:41:11.724412+0000 front 2023-07-18T03:41:11.724566+0000 (oldest deadline 2023-07-18T03:41:33.424161+0000) 2023-07-18T03:41:42.450 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:42 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:42.086+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:42.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:42 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:42.086+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:42.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:42 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:42.086+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:42.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:42 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:42.086+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.239591+0000 front 2023-07-18T03:40:54.239502+0000 (oldest deadline 2023-07-18T03:41:20.139603+0000) 2023-07-18T03:41:42.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:42 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:42.086+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:02.940985+0000 front 2023-07-18T03:41:02.941034+0000 (oldest deadline 2023-07-18T03:41:27.641205+0000) 2023-07-18T03:41:42.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:42 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:42.086+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6822 osd.6 since back 2023-07-18T03:41:07.641906+0000 front 2023-07-18T03:41:07.641794+0000 (oldest deadline 2023-07-18T03:41:38.242139+0000) 2023-07-18T03:41:42.545 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:42 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:42.061+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:41:42.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:42 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:42.061+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:41:42.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:42 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:42.061+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.718618+0000 front 2023-07-18T03:40:38.718740+0000 (oldest deadline 2023-07-18T03:41:02.818660+0000) 2023-07-18T03:41:42.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:42 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:42.061+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:46.820033+0000 front 2023-07-18T03:40:46.820086+0000 (oldest deadline 2023-07-18T03:41:10.319945+0000) 2023-07-18T03:41:42.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:42 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:42.061+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.921232+0000 front 2023-07-18T03:40:54.921268+0000 (oldest deadline 2023-07-18T03:41:18.921985+0000) 2023-07-18T03:41:42.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:42 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:42.061+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:02.423041+0000 front 2023-07-18T03:41:02.422948+0000 (oldest deadline 2023-07-18T03:41:25.322779+0000) 2023-07-18T03:41:42.547 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:42 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:42.061+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6822 osd.6 since back 2023-07-18T03:41:11.724412+0000 front 2023-07-18T03:41:11.724566+0000 (oldest deadline 2023-07-18T03:41:33.424161+0000) 2023-07-18T03:41:43.295 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:43 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:43.014+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:41:43.296 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:43 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:43.014+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:41:43.296 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:43 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:43.014+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.718618+0000 front 2023-07-18T03:40:38.718740+0000 (oldest deadline 2023-07-18T03:41:02.818660+0000) 2023-07-18T03:41:43.296 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:43 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:43.014+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:46.820033+0000 front 2023-07-18T03:40:46.820086+0000 (oldest deadline 2023-07-18T03:41:10.319945+0000) 2023-07-18T03:41:43.297 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:43 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:43.014+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.921232+0000 front 2023-07-18T03:40:54.921268+0000 (oldest deadline 2023-07-18T03:41:18.921985+0000) 2023-07-18T03:41:43.297 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:43 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:43.014+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:02.423041+0000 front 2023-07-18T03:41:02.422948+0000 (oldest deadline 2023-07-18T03:41:25.322779+0000) 2023-07-18T03:41:43.297 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:43 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:43.014+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6822 osd.6 since back 2023-07-18T03:41:11.724412+0000 front 2023-07-18T03:41:11.724566+0000 (oldest deadline 2023-07-18T03:41:33.424161+0000) 2023-07-18T03:41:43.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:43 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:43.095+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:43.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:43 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:43.095+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:43.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:43 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:43.095+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:43.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:43 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:43.095+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.239591+0000 front 2023-07-18T03:40:54.239502+0000 (oldest deadline 2023-07-18T03:41:20.139603+0000) 2023-07-18T03:41:43.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:43 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:43.095+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:02.940985+0000 front 2023-07-18T03:41:02.941034+0000 (oldest deadline 2023-07-18T03:41:27.641205+0000) 2023-07-18T03:41:43.453 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:43 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:43.095+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6822 osd.6 since back 2023-07-18T03:41:07.641906+0000 front 2023-07-18T03:41:07.641794+0000 (oldest deadline 2023-07-18T03:41:38.242139+0000) 2023-07-18T03:41:43.453 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:43 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:43.095+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2023-07-18T03:41:18.743175+0000 front 2023-07-18T03:41:18.743079+0000 (oldest deadline 2023-07-18T03:41:42.842910+0000) 2023-07-18T03:41:44.295 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:44 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:44.007+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:23.615782+0000 front 2023-07-18T03:40:23.615877+0000 (oldest deadline 2023-07-18T03:40:46.416488+0000) 2023-07-18T03:41:44.296 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:44 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:44.008+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:28.117579+0000 front 2023-07-18T03:40:28.117596+0000 (oldest deadline 2023-07-18T03:40:54.017724+0000) 2023-07-18T03:41:44.296 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:44 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:44.008+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:38.718618+0000 front 2023-07-18T03:40:38.718740+0000 (oldest deadline 2023-07-18T03:41:02.818660+0000) 2023-07-18T03:41:44.296 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:44 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:44.008+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6814 osd.3 since back 2023-07-18T03:40:46.820033+0000 front 2023-07-18T03:40:46.820086+0000 (oldest deadline 2023-07-18T03:41:10.319945+0000) 2023-07-18T03:41:44.297 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:44 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:44.008+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.921232+0000 front 2023-07-18T03:40:54.921268+0000 (oldest deadline 2023-07-18T03:41:18.921985+0000) 2023-07-18T03:41:44.297 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:44 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:44.008+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:02.423041+0000 front 2023-07-18T03:41:02.422948+0000 (oldest deadline 2023-07-18T03:41:25.322779+0000) 2023-07-18T03:41:44.297 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:44 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10[150480]: 2023-07-18T03:41:44.008+0000 7fa59d45a700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6822 osd.6 since back 2023-07-18T03:41:11.724412+0000 front 2023-07-18T03:41:11.724566+0000 (oldest deadline 2023-07-18T03:41:33.424161+0000) 2023-07-18T03:41:44.450 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:44 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:44.142+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:44.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:44 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:44.142+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:44.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:44 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:44.142+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:44.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:44 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:44.142+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.239591+0000 front 2023-07-18T03:40:54.239502+0000 (oldest deadline 2023-07-18T03:41:20.139603+0000) 2023-07-18T03:41:44.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:44 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:44.142+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:02.940985+0000 front 2023-07-18T03:41:02.941034+0000 (oldest deadline 2023-07-18T03:41:27.641205+0000) 2023-07-18T03:41:44.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:44 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:44.142+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6822 osd.6 since back 2023-07-18T03:41:07.641906+0000 front 2023-07-18T03:41:07.641794+0000 (oldest deadline 2023-07-18T03:41:38.242139+0000) 2023-07-18T03:41:44.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:44 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:44.142+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2023-07-18T03:41:18.743175+0000 front 2023-07-18T03:41:18.743079+0000 (oldest deadline 2023-07-18T03:41:42.842910+0000) 2023-07-18T03:41:45.450 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:45 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:45.124+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:45.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:45 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:45.124+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:45.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:45 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:45.124+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:45.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:45 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:45.124+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.239591+0000 front 2023-07-18T03:40:54.239502+0000 (oldest deadline 2023-07-18T03:41:20.139603+0000) 2023-07-18T03:41:45.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:45 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:45.124+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:02.940985+0000 front 2023-07-18T03:41:02.941034+0000 (oldest deadline 2023-07-18T03:41:27.641205+0000) 2023-07-18T03:41:45.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:45 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:45.124+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6822 osd.6 since back 2023-07-18T03:41:07.641906+0000 front 2023-07-18T03:41:07.641794+0000 (oldest deadline 2023-07-18T03:41:38.242139+0000) 2023-07-18T03:41:45.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:45 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:45.124+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2023-07-18T03:41:18.743175+0000 front 2023-07-18T03:41:18.743079+0000 (oldest deadline 2023-07-18T03:41:42.842910+0000) 2023-07-18T03:41:45.545 INFO:journalctl@ceph.osd.10.smithi155.stdout:Jul 18 03:41:45 smithi155 bash[167679]: ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-10 2023-07-18T03:41:46.450 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:46 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:46.087+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:46.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:46 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:46.087+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:46.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:46 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:46.087+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:46.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:46 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:46.087+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.239591+0000 front 2023-07-18T03:40:54.239502+0000 (oldest deadline 2023-07-18T03:41:20.139603+0000) 2023-07-18T03:41:46.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:46 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:46.087+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:02.940985+0000 front 2023-07-18T03:41:02.941034+0000 (oldest deadline 2023-07-18T03:41:27.641205+0000) 2023-07-18T03:41:46.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:46 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:46.087+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6822 osd.6 since back 2023-07-18T03:41:07.641906+0000 front 2023-07-18T03:41:07.641794+0000 (oldest deadline 2023-07-18T03:41:38.242139+0000) 2023-07-18T03:41:46.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:46 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:46.087+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2023-07-18T03:41:18.743175+0000 front 2023-07-18T03:41:18.743079+0000 (oldest deadline 2023-07-18T03:41:42.842910+0000) 2023-07-18T03:41:46.666 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@osd.10.service' 2023-07-18T03:41:46.705 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:41:46.705 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-07-18T03:41:46.706 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-07-18T03:41:46.706 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-b1782daa-2516-11ee-9b34-001a4aab830c@osd.11 2023-07-18T03:41:47.067 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:46 smithi179 systemd[1]: Stopping Ceph osd.11 for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T03:41:47.067 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:47 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:47.073+0000 7f74900b5700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:41:47.450 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:47 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:47.073+0000 7f74900b5700 -1 osd.11 149 *** Got signal Terminated *** 2023-07-18T03:41:47.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:47 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:47.073+0000 7f74900b5700 -1 osd.11 149 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-18T03:41:47.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:47 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:47.131+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:47.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:47 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:47.131+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:47.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:47 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:47.131+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:47.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:47 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:47.131+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.239591+0000 front 2023-07-18T03:40:54.239502+0000 (oldest deadline 2023-07-18T03:41:20.139603+0000) 2023-07-18T03:41:47.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:47 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:47.131+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:02.940985+0000 front 2023-07-18T03:41:02.941034+0000 (oldest deadline 2023-07-18T03:41:27.641205+0000) 2023-07-18T03:41:47.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:47 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:47.131+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6822 osd.6 since back 2023-07-18T03:41:07.641906+0000 front 2023-07-18T03:41:07.641794+0000 (oldest deadline 2023-07-18T03:41:38.242139+0000) 2023-07-18T03:41:47.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:47 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:47.131+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2023-07-18T03:41:18.743175+0000 front 2023-07-18T03:41:18.743079+0000 (oldest deadline 2023-07-18T03:41:42.842910+0000) 2023-07-18T03:41:48.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:48 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:48.153+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:48.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:48 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:48.153+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:48.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:48 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:48.153+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:48.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:48 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:48.153+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.239591+0000 front 2023-07-18T03:40:54.239502+0000 (oldest deadline 2023-07-18T03:41:20.139603+0000) 2023-07-18T03:41:48.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:48 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:48.153+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:02.940985+0000 front 2023-07-18T03:41:02.941034+0000 (oldest deadline 2023-07-18T03:41:27.641205+0000) 2023-07-18T03:41:48.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:48 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:48.153+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6822 osd.6 since back 2023-07-18T03:41:07.641906+0000 front 2023-07-18T03:41:07.641794+0000 (oldest deadline 2023-07-18T03:41:38.242139+0000) 2023-07-18T03:41:48.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:48 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:48.153+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2023-07-18T03:41:18.743175+0000 front 2023-07-18T03:41:18.743079+0000 (oldest deadline 2023-07-18T03:41:42.842910+0000) 2023-07-18T03:41:49.450 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:49 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:49.181+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:49.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:49 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:49.181+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:49.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:49 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:49.181+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:49.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:49 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:49.181+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.239591+0000 front 2023-07-18T03:40:54.239502+0000 (oldest deadline 2023-07-18T03:41:20.139603+0000) 2023-07-18T03:41:49.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:49 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:49.181+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:02.940985+0000 front 2023-07-18T03:41:02.941034+0000 (oldest deadline 2023-07-18T03:41:27.641205+0000) 2023-07-18T03:41:49.453 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:49 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:49.181+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6822 osd.6 since back 2023-07-18T03:41:07.641906+0000 front 2023-07-18T03:41:07.641794+0000 (oldest deadline 2023-07-18T03:41:38.242139+0000) 2023-07-18T03:41:49.453 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:49 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:49.181+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2023-07-18T03:41:18.743175+0000 front 2023-07-18T03:41:18.743079+0000 (oldest deadline 2023-07-18T03:41:42.842910+0000) 2023-07-18T03:41:50.450 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:50 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:50.148+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:50.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:50 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:50.148+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:50.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:50 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:50.148+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:50.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:50 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:50.148+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.239591+0000 front 2023-07-18T03:40:54.239502+0000 (oldest deadline 2023-07-18T03:41:20.139603+0000) 2023-07-18T03:41:50.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:50 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:50.148+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:02.940985+0000 front 2023-07-18T03:41:02.941034+0000 (oldest deadline 2023-07-18T03:41:27.641205+0000) 2023-07-18T03:41:50.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:50 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:50.148+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6822 osd.6 since back 2023-07-18T03:41:07.641906+0000 front 2023-07-18T03:41:07.641794+0000 (oldest deadline 2023-07-18T03:41:38.242139+0000) 2023-07-18T03:41:50.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:50 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:50.148+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2023-07-18T03:41:18.743175+0000 front 2023-07-18T03:41:18.743079+0000 (oldest deadline 2023-07-18T03:41:42.842910+0000) 2023-07-18T03:41:51.450 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:51 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:51.125+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6806 osd.0 since back 2023-07-18T03:40:24.335408+0000 front 2023-07-18T03:40:24.335284+0000 (oldest deadline 2023-07-18T03:40:49.035384+0000) 2023-07-18T03:41:51.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:51 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:51.125+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2023-07-18T03:40:31.936626+0000 front 2023-07-18T03:40:31.936500+0000 (oldest deadline 2023-07-18T03:40:56.636638+0000) 2023-07-18T03:41:51.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:51 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:51.125+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2023-07-18T03:40:36.637149+0000 front 2023-07-18T03:40:36.637388+0000 (oldest deadline 2023-07-18T03:41:01.937366+0000) 2023-07-18T03:41:51.451 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:51 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:51.125+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2023-07-18T03:40:54.239591+0000 front 2023-07-18T03:40:54.239502+0000 (oldest deadline 2023-07-18T03:41:20.139603+0000) 2023-07-18T03:41:51.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:51 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:51.125+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2023-07-18T03:41:02.940985+0000 front 2023-07-18T03:41:02.941034+0000 (oldest deadline 2023-07-18T03:41:27.641205+0000) 2023-07-18T03:41:51.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:51 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:51.125+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6822 osd.6 since back 2023-07-18T03:41:07.641906+0000 front 2023-07-18T03:41:07.641794+0000 (oldest deadline 2023-07-18T03:41:38.242139+0000) 2023-07-18T03:41:51.452 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:51 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11[151476]: 2023-07-18T03:41:51.125+0000 7f748a6a7700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2023-07-18T03:41:18.743175+0000 front 2023-07-18T03:41:18.743079+0000 (oldest deadline 2023-07-18T03:41:42.842910+0000) 2023-07-18T03:41:52.842 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jul 18 03:41:52 smithi179 bash[167977]: ceph-b1782daa-2516-11ee-9b34-001a4aab830c-osd-11 2023-07-18T03:41:54.107 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@osd.11.service' 2023-07-18T03:41:54.147 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:41:54.148 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-07-18T03:41:54.148 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-07-18T03:41:54.149 DEBUG:teuthology.orchestra.run.smithi101:> sudo systemctl stop ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mds.a 2023-07-18T03:41:54.488 INFO:journalctl@ceph.mds.a.smithi101.stdout:Jul 18 03:41:54 smithi101 systemd[1]: Stopping Ceph mds.a for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T03:41:54.988 INFO:journalctl@ceph.mds.a.smithi101.stdout:Jul 18 03:41:54 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-a[161215]: 2023-07-18T03:41:54.728+0000 7f21210ff700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:41:54.988 INFO:journalctl@ceph.mds.a.smithi101.stdout:Jul 18 03:41:54 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-a[161215]: 2023-07-18T03:41:54.728+0000 7f21210ff700 -1 mds.a *** got signal Terminated *** 2023-07-18T03:41:56.738 INFO:journalctl@ceph.mds.a.smithi101.stdout:Jul 18 03:41:56 smithi101 bash[206753]: ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-a 2023-07-18T03:41:56.931 DEBUG:teuthology.orchestra.run.smithi101:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mds.a.service' 2023-07-18T03:41:57.017 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:41:57.018 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-07-18T03:41:57.018 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-07-18T03:41:57.019 DEBUG:teuthology.orchestra.run.smithi101:> sudo systemctl stop ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mds.d 2023-07-18T03:41:57.238 INFO:journalctl@ceph.mds.d.smithi101.stdout:Jul 18 03:41:57 smithi101 systemd[1]: Stopping Ceph mds.d for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T03:41:57.988 INFO:journalctl@ceph.mds.d.smithi101.stdout:Jul 18 03:41:57 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-d[160213]: 2023-07-18T03:41:57.510+0000 7fb536a39700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:41:57.988 INFO:journalctl@ceph.mds.d.smithi101.stdout:Jul 18 03:41:57 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-d[160213]: 2023-07-18T03:41:57.510+0000 7fb536a39700 -1 mds.d *** got signal Terminated *** 2023-07-18T03:42:02.238 INFO:journalctl@ceph.mds.d.smithi101.stdout:Jul 18 03:42:01 smithi101 bash[206898]: ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-d 2023-07-18T03:42:02.403 DEBUG:teuthology.orchestra.run.smithi101:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mds.d.service' 2023-07-18T03:42:02.486 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:42:02.488 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-07-18T03:42:02.489 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-07-18T03:42:02.489 DEBUG:teuthology.orchestra.run.smithi101:> sudo systemctl stop ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mds.g 2023-07-18T03:42:02.742 INFO:journalctl@ceph.mds.g.smithi101.stdout:Jul 18 03:42:02 smithi101 systemd[1]: Stopping Ceph mds.g for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T03:42:03.238 INFO:journalctl@ceph.mds.g.smithi101.stdout:Jul 18 03:42:02 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-g[160829]: 2023-07-18T03:42:02.993+0000 7f3ec8639700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:42:03.238 INFO:journalctl@ceph.mds.g.smithi101.stdout:Jul 18 03:42:02 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-g[160829]: 2023-07-18T03:42:02.993+0000 7f3ec8639700 -1 mds.g *** got signal Terminated *** 2023-07-18T03:42:05.738 INFO:journalctl@ceph.mds.g.smithi101.stdout:Jul 18 03:42:05 smithi101 bash[207045]: ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-g 2023-07-18T03:42:05.776 DEBUG:teuthology.orchestra.run.smithi101:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mds.g.service' 2023-07-18T03:42:05.857 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:42:05.857 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-07-18T03:42:05.858 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-07-18T03:42:05.858 DEBUG:teuthology.orchestra.run.smithi101:> sudo systemctl stop ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mds.j 2023-07-18T03:42:06.238 INFO:journalctl@ceph.mds.j.smithi101.stdout:Jul 18 03:42:05 smithi101 systemd[1]: Stopping Ceph mds.j for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T03:42:06.738 INFO:journalctl@ceph.mds.j.smithi101.stdout:Jul 18 03:42:06 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-j[159828]: 2023-07-18T03:42:06.358+0000 7f3498532700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:42:06.739 INFO:journalctl@ceph.mds.j.smithi101.stdout:Jul 18 03:42:06 smithi101 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-j[159828]: 2023-07-18T03:42:06.358+0000 7f3498532700 -1 mds.j *** got signal Terminated *** 2023-07-18T03:42:10.881 INFO:journalctl@ceph.mds.j.smithi101.stdout:Jul 18 03:42:10 smithi101 bash[207189]: ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-j 2023-07-18T03:42:11.102 DEBUG:teuthology.orchestra.run.smithi101:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mds.j.service' 2023-07-18T03:42:11.162 INFO:journalctl@ceph.mds.j.smithi101.stdout:Jul 18 03:42:11 smithi101 systemd[1]: ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mds.j.service: Succeeded. 2023-07-18T03:42:11.163 INFO:journalctl@ceph.mds.j.smithi101.stdout:Jul 18 03:42:11 smithi101 systemd[1]: Stopped Ceph mds.j for b1782daa-2516-11ee-9b34-001a4aab830c. 2023-07-18T03:42:11.185 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:42:11.185 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-07-18T03:42:11.186 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-07-18T03:42:11.186 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mds.b 2023-07-18T03:42:11.545 INFO:journalctl@ceph.mds.b.smithi155.stdout:Jul 18 03:42:11 smithi155 systemd[1]: Stopping Ceph mds.b for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T03:42:12.045 INFO:journalctl@ceph.mds.b.smithi155.stdout:Jul 18 03:42:11 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-b[154157]: 2023-07-18T03:42:11.556+0000 7f11a1890700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:42:12.046 INFO:journalctl@ceph.mds.b.smithi155.stdout:Jul 18 03:42:11 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-b[154157]: 2023-07-18T03:42:11.556+0000 7f11a1890700 -1 mds.b *** got signal Terminated *** 2023-07-18T03:42:16.045 INFO:journalctl@ceph.mds.b.smithi155.stdout:Jul 18 03:42:15 smithi155 bash[167952]: ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-b 2023-07-18T03:42:16.162 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mds.b.service' 2023-07-18T03:42:16.199 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:42:16.200 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-07-18T03:42:16.201 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-07-18T03:42:16.201 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mds.e 2023-07-18T03:42:16.545 INFO:journalctl@ceph.mds.e.smithi155.stdout:Jul 18 03:42:16 smithi155 systemd[1]: Stopping Ceph mds.e for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T03:42:17.045 INFO:journalctl@ceph.mds.e.smithi155.stdout:Jul 18 03:42:16 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-e[153469]: 2023-07-18T03:42:16.591+0000 7fc18be59700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:42:17.046 INFO:journalctl@ceph.mds.e.smithi155.stdout:Jul 18 03:42:16 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-e[153469]: 2023-07-18T03:42:16.591+0000 7fc18be59700 -1 mds.e *** got signal Terminated *** 2023-07-18T03:42:20.149 INFO:journalctl@ceph.mds.e.smithi155.stdout:Jul 18 03:42:20 smithi155 bash[168100]: ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-e 2023-07-18T03:42:20.338 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mds.e.service' 2023-07-18T03:42:20.373 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:42:20.373 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-07-18T03:42:20.373 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-07-18T03:42:20.374 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mds.h 2023-07-18T03:42:20.758 INFO:journalctl@ceph.mds.h.smithi155.stdout:Jul 18 03:42:20 smithi155 systemd[1]: Stopping Ceph mds.h for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T03:42:21.045 INFO:journalctl@ceph.mds.h.smithi155.stdout:Jul 18 03:42:20 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-h[153809]: 2023-07-18T03:42:20.762+0000 7ff43f2db700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:42:21.045 INFO:journalctl@ceph.mds.h.smithi155.stdout:Jul 18 03:42:20 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-h[153809]: 2023-07-18T03:42:20.762+0000 7ff43f2db700 -1 mds.h *** got signal Terminated *** 2023-07-18T03:42:24.119 INFO:journalctl@ceph.mds.h.smithi155.stdout:Jul 18 03:42:24 smithi155 bash[168243]: ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-h 2023-07-18T03:42:24.306 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mds.h.service' 2023-07-18T03:42:24.340 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:42:24.340 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-07-18T03:42:24.341 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-07-18T03:42:24.341 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mds.k 2023-07-18T03:42:24.730 INFO:journalctl@ceph.mds.k.smithi155.stdout:Jul 18 03:42:24 smithi155 systemd[1]: Stopping Ceph mds.k for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T03:42:25.045 INFO:journalctl@ceph.mds.k.smithi155.stdout:Jul 18 03:42:24 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-k[153128]: 2023-07-18T03:42:24.735+0000 7f9699804700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:42:25.045 INFO:journalctl@ceph.mds.k.smithi155.stdout:Jul 18 03:42:24 smithi155 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-k[153128]: 2023-07-18T03:42:24.735+0000 7f9699804700 -1 mds.k *** got signal Terminated *** 2023-07-18T03:42:27.895 INFO:journalctl@ceph.mds.k.smithi155.stdout:Jul 18 03:42:27 smithi155 bash[168380]: ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-k 2023-07-18T03:42:28.094 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mds.k.service' 2023-07-18T03:42:28.129 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:42:28.130 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-07-18T03:42:28.130 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-07-18T03:42:28.131 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mds.c 2023-07-18T03:42:28.450 INFO:journalctl@ceph.mds.c.smithi179.stdout:Jul 18 03:42:28 smithi179 systemd[1]: Stopping Ceph mds.c for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T03:42:28.950 INFO:journalctl@ceph.mds.c.smithi179.stdout:Jul 18 03:42:28 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-c[154971]: 2023-07-18T03:42:28.500+0000 7f6d068bc700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:42:28.951 INFO:journalctl@ceph.mds.c.smithi179.stdout:Jul 18 03:42:28 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-c[154971]: 2023-07-18T03:42:28.500+0000 7f6d068bc700 -1 mds.c *** got signal Terminated *** 2023-07-18T03:42:32.344 INFO:journalctl@ceph.mds.c.smithi179.stdout:Jul 18 03:42:32 smithi179 bash[168243]: ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-c 2023-07-18T03:42:32.351 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mds.c.service' 2023-07-18T03:42:32.394 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:42:32.395 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-07-18T03:42:32.395 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-07-18T03:42:32.396 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mds.f 2023-07-18T03:42:32.700 INFO:journalctl@ceph.mds.f.smithi179.stdout:Jul 18 03:42:32 smithi179 systemd[1]: Stopping Ceph mds.f for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T03:42:33.200 INFO:journalctl@ceph.mds.f.smithi179.stdout:Jul 18 03:42:32 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-f[154631]: 2023-07-18T03:42:32.775+0000 7f265361f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:42:33.201 INFO:journalctl@ceph.mds.f.smithi179.stdout:Jul 18 03:42:32 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-f[154631]: 2023-07-18T03:42:32.775+0000 7f265361f700 -1 mds.f *** got signal Terminated *** 2023-07-18T03:42:36.450 INFO:journalctl@ceph.mds.f.smithi179.stdout:Jul 18 03:42:36 smithi179 bash[168383]: ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-f 2023-07-18T03:42:36.481 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mds.f.service' 2023-07-18T03:42:36.515 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:42:36.515 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-07-18T03:42:36.516 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-07-18T03:42:36.516 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mds.i 2023-07-18T03:42:36.881 INFO:journalctl@ceph.mds.i.smithi179.stdout:Jul 18 03:42:36 smithi179 systemd[1]: Stopping Ceph mds.i for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T03:42:37.200 INFO:journalctl@ceph.mds.i.smithi179.stdout:Jul 18 03:42:36 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-i[155309]: 2023-07-18T03:42:36.887+0000 7f1531370700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:42:37.200 INFO:journalctl@ceph.mds.i.smithi179.stdout:Jul 18 03:42:36 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-i[155309]: 2023-07-18T03:42:36.887+0000 7f1531370700 -1 mds.i *** got signal Terminated *** 2023-07-18T03:42:40.158 INFO:journalctl@ceph.mds.i.smithi179.stdout:Jul 18 03:42:40 smithi179 bash[168528]: ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-i 2023-07-18T03:42:40.349 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mds.i.service' 2023-07-18T03:42:40.384 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:42:40.385 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-07-18T03:42:40.385 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-07-18T03:42:40.386 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mds.l 2023-07-18T03:42:40.700 INFO:journalctl@ceph.mds.l.smithi179.stdout:Jul 18 03:42:40 smithi179 systemd[1]: Stopping Ceph mds.l for b1782daa-2516-11ee-9b34-001a4aab830c... 2023-07-18T03:42:41.200 INFO:journalctl@ceph.mds.l.smithi179.stdout:Jul 18 03:42:40 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-l[154290]: 2023-07-18T03:42:40.764+0000 7f36bd09b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:42:41.200 INFO:journalctl@ceph.mds.l.smithi179.stdout:Jul 18 03:42:40 smithi179 ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-l[154290]: 2023-07-18T03:42:40.764+0000 7f36bd09b700 -1 mds.l *** got signal Terminated *** 2023-07-18T03:42:45.200 INFO:journalctl@ceph.mds.l.smithi179.stdout:Jul 18 03:42:44 smithi179 bash[168664]: ceph-b1782daa-2516-11ee-9b34-001a4aab830c-mds-l 2023-07-18T03:42:45.323 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1782daa-2516-11ee-9b34-001a4aab830c@mds.l.service' 2023-07-18T03:42:45.357 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:42:45.358 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-07-18T03:42:45.358 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b1782daa-2516-11ee-9b34-001a4aab830c --force --keep-logs 2023-07-18T03:42:48.970 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b1782daa-2516-11ee-9b34-001a4aab830c --force --keep-logs 2023-07-18T03:42:50.927 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b1782daa-2516-11ee-9b34-001a4aab830c --force --keep-logs 2023-07-18T03:42:52.935 DEBUG:teuthology.orchestra.run.smithi101:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-18T03:42:53.031 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-18T03:42:53.067 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-18T03:42:53.096 INFO:tasks.cephadm:Archiving crash dumps... 2023-07-18T03:42:53.104 DEBUG:teuthology.misc:Transferring archived files from smithi101:/var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342713/remote/smithi101/crash 2023-07-18T03:42:53.105 DEBUG:teuthology.orchestra.run.smithi101:> sudo tar c -f - -C /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/crash -- . 2023-07-18T03:42:53.249 INFO:teuthology.orchestra.run.smithi101.stderr:tar: /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/crash: Cannot open: No such file or directory 2023-07-18T03:42:53.249 INFO:teuthology.orchestra.run.smithi101.stderr:tar: Error is not recoverable: exiting now 2023-07-18T03:42:53.252 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342713/remote/smithi155/crash 2023-07-18T03:42:53.253 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/crash -- . 2023-07-18T03:42:53.373 INFO:teuthology.orchestra.run.smithi155.stderr:tar: /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/crash: Cannot open: No such file or directory 2023-07-18T03:42:53.373 INFO:teuthology.orchestra.run.smithi155.stderr:tar: Error is not recoverable: exiting now 2023-07-18T03:42:53.375 DEBUG:teuthology.misc:Transferring archived files from smithi179:/var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342713/remote/smithi179/crash 2023-07-18T03:42:53.377 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar c -f - -C /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/crash -- . 2023-07-18T03:42:53.452 INFO:teuthology.orchestra.run.smithi179.stderr:tar: /var/lib/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/crash: Cannot open: No such file or directory 2023-07-18T03:42:53.453 INFO:teuthology.orchestra.run.smithi179.stderr:tar: Error is not recoverable: exiting now 2023-07-18T03:42:53.454 INFO:tasks.cephadm:Checking cluster log for badness... 2023-07-18T03:42:53.455 DEBUG:teuthology.orchestra.run.smithi101:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-07-18T03:42:53.534 INFO:teuthology.orchestra.run.smithi101.stderr:grep: /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph.log: No such file or directory 2023-07-18T03:42:53.538 INFO:tasks.cephadm:Compressing logs... 2023-07-18T03:42:53.539 DEBUG:teuthology.orchestra.run.smithi101:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-18T03:42:53.544 DEBUG:teuthology.orchestra.run.smithi155:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-18T03:42:53.547 DEBUG:teuthology.orchestra.run.smithi179:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-18T03:42:53.592 INFO:teuthology.orchestra.run.smithi179.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-07-18T03:42:53.593 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-07-18T03:42:53.598 INFO:teuthology.orchestra.run.smithi155.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-07-18T03:42:53.599 INFO:teuthology.orchestra.run.smithi155.stderr:: No such file or directory 2023-07-18T03:42:53.600 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-osd.5.log 2023-07-18T03:42:53.601 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-osd.2.log 2023-07-18T03:42:53.602 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-osd.11.log 2023-07-18T03:42:53.602 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mgr.z.log 2023-07-18T03:42:53.602 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-osd.8.log 2023-07-18T03:42:53.603 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mds.f.log 2023-07-18T03:42:53.603 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mds.c.log 2023-07-18T03:42:53.603 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mds.i.log 2023-07-18T03:42:53.604 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mon.c.log 2023-07-18T03:42:53.605 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mds.c.log: /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-volume.log 2023-07-18T03:42:53.605 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mds.l.log 2023-07-18T03:42:53.606 INFO:teuthology.orchestra.run.smithi101.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-07-18T03:42:53.606 INFO:teuthology.orchestra.run.smithi101.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-07-18T03:42:53.607 INFO:teuthology.orchestra.run.smithi101.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2023-07-18T03:42:53.608 INFO:teuthology.orchestra.run.smithi101.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-osd.0.log 2023-07-18T03:42:53.609 INFO:teuthology.orchestra.run.smithi101.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mds.g.log 2023-07-18T03:42:53.610 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-volume.log: /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mds.l.log: 94.6% -- replaced with /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mds.l.log.gz 2023-07-18T03:42:53.611 INFO:teuthology.orchestra.run.smithi179.stderr: 92.9% -- replaced with /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mgr.z.log.gz 2023-07-18T03:42:53.611 INFO:teuthology.orchestra.run.smithi101.stderr:/var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mds.a.log 2023-07-18T03:42:53.611 INFO:teuthology.orchestra.run.smithi101.stderr:/var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mds.g.log: 93.9% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2023-07-18T03:42:53.612 INFO:teuthology.orchestra.run.smithi101.stderr:gzip -5 --verbose -- /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-osd.6.log 2023-07-18T03:42:53.613 INFO:teuthology.orchestra.run.smithi101.stderr:/var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-osd.3.log 2023-07-18T03:42:53.613 INFO:teuthology.orchestra.run.smithi101.stderr:/var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-osd.6.log: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2023-07-18T03:42:53.614 INFO:teuthology.orchestra.run.smithi101.stderr:gzip -5 --verbose -- /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mds.d.log 2023-07-18T03:42:53.616 INFO:teuthology.orchestra.run.smithi101.stderr:/var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mgr.x.log 2023-07-18T03:42:53.616 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-osd.10.log 2023-07-18T03:42:53.617 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mds.h.log 2023-07-18T03:42:53.618 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mds.e.log 2023-07-18T03:42:53.618 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-osd.10.log: /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-osd.4.log 2023-07-18T03:42:53.618 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-osd.1.log 2023-07-18T03:42:53.619 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mds.k.log 2023-07-18T03:42:53.619 INFO:teuthology.orchestra.run.smithi101.stderr:/var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-osd.9.log 2023-07-18T03:42:53.620 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-volume.log 2023-07-18T03:42:53.621 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-osd.7.log 2023-07-18T03:42:53.621 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mon.b.log 2023-07-18T03:42:53.621 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mds.b.log 2023-07-18T03:42:53.621 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mgr.y.log 2023-07-18T03:42:53.623 INFO:teuthology.orchestra.run.smithi101.stderr:/var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mgr.x.log: /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-volume.log 2023-07-18T03:42:53.623 INFO:teuthology.orchestra.run.smithi101.stderr: 94.5% -- replaced with /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mds.g.log.gz 2023-07-18T03:42:53.624 INFO:teuthology.orchestra.run.smithi101.stderr: 94.5% -- replaced with /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mds.a.log.gz 2023-07-18T03:42:53.624 INFO:teuthology.orchestra.run.smithi101.stderr:/var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mds.j.log 2023-07-18T03:42:53.625 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mds.b.log: /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mgr.y.log: 94.5% -- replaced with /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mds.k.log.gz 2023-07-18T03:42:53.629 INFO:teuthology.orchestra.run.smithi155.stderr: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2023-07-18T03:42:53.629 INFO:teuthology.orchestra.run.smithi155.stderr: 92.8% -- replaced with /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mgr.y.log.gz 2023-07-18T03:42:53.632 INFO:teuthology.orchestra.run.smithi101.stderr:/var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mds.j.log: 94.6% -- replaced with /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mds.d.log.gz 2023-07-18T03:42:53.633 INFO:teuthology.orchestra.run.smithi101.stderr:gzip -5 --verbose -- /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mon.a.log 2023-07-18T03:42:53.633 INFO:teuthology.orchestra.run.smithi101.stderr:/var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mon.a.log: 91.7% -- replaced with /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-volume.log.gz 2023-07-18T03:42:53.635 INFO:teuthology.orchestra.run.smithi101.stderr: 94.5% -- replaced with /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mds.j.log.gz 2023-07-18T03:42:53.750 INFO:teuthology.orchestra.run.smithi155.stderr: 91.6% -- replaced with /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-volume.log.gz 2023-07-18T03:42:53.896 INFO:teuthology.orchestra.run.smithi179.stderr: 91.6% -- replaced with /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-volume.log.gz 2023-07-18T03:42:54.210 INFO:teuthology.orchestra.run.smithi179.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2023-07-18T03:42:54.236 INFO:teuthology.orchestra.run.smithi101.stderr: 89.9% -- replaced with /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mgr.x.log.gz 2023-07-18T03:42:56.140 INFO:teuthology.orchestra.run.smithi101.stderr: 90.4% -- replaced with /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mon.a.log.gz 2023-07-18T03:42:58.154 INFO:teuthology.orchestra.run.smithi179.stderr: 92.0% -- replaced with /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mon.c.log.gz 2023-07-18T03:43:00.268 INFO:teuthology.orchestra.run.smithi155.stderr: 93.1% -- replaced with /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mon.b.log.gz 2023-07-18T03:43:15.718 INFO:teuthology.orchestra.run.smithi155.stderr: 95.0% -- replaced with /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mds.e.log.gz 2023-07-18T03:43:19.688 INFO:teuthology.orchestra.run.smithi155.stderr: 95.2% -- replaced with /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mds.h.log.gz 2023-07-18T03:43:25.352 INFO:teuthology.orchestra.run.smithi101.stderr: 91.6% -- replaced with /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-osd.0.log.gz 2023-07-18T03:43:33.865 INFO:teuthology.orchestra.run.smithi101.stderr: 91.3% -- replaced with /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-osd.3.log.gz 2023-07-18T03:43:35.533 INFO:teuthology.orchestra.run.smithi101.stderr: 91.2% -- replaced with /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-osd.6.log.gz 2023-07-18T03:43:36.019 INFO:teuthology.orchestra.run.smithi101.stderr: 91.1% -- replaced with /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-osd.9.log.gz 2023-07-18T03:43:36.021 INFO:teuthology.orchestra.run.smithi101.stderr: 2023-07-18T03:43:36.022 INFO:teuthology.orchestra.run.smithi101.stderr:real 0m42.442s 2023-07-18T03:43:36.022 INFO:teuthology.orchestra.run.smithi101.stderr:user 2m27.409s 2023-07-18T03:43:36.022 INFO:teuthology.orchestra.run.smithi101.stderr:sys 0m12.116s 2023-07-18T03:43:45.714 INFO:teuthology.orchestra.run.smithi179.stderr: 95.2% -- replaced with /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mds.f.log.gz 2023-07-18T03:44:04.698 INFO:teuthology.orchestra.run.smithi155.stderr: 91.2% -- replaced with /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-osd.10.log.gz 2023-07-18T03:44:10.260 INFO:teuthology.orchestra.run.smithi155.stderr: 91.5% -- replaced with /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-osd.1.log.gz 2023-07-18T03:44:26.145 INFO:teuthology.orchestra.run.smithi155.stderr: 91.2% -- replaced with /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-osd.7.log.gz 2023-07-18T03:44:26.770 INFO:teuthology.orchestra.run.smithi155.stderr: 91.3% -- replaced with /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-osd.4.log.gz 2023-07-18T03:44:49.459 INFO:teuthology.orchestra.run.smithi179.stderr: 91.5% -- replaced with /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-osd.11.log.gz 2023-07-18T03:45:00.819 INFO:teuthology.orchestra.run.smithi179.stderr: 91.1% -- replaced with /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-osd.8.log.gz 2023-07-18T03:45:22.082 INFO:teuthology.orchestra.run.smithi179.stderr: 91.3% -- replaced with /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-osd.2.log.gz 2023-07-18T03:45:42.329 INFO:teuthology.orchestra.run.smithi179.stderr: 91.0% -- replaced with /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-osd.5.log.gz 2023-07-18T03:46:49.871 INFO:teuthology.orchestra.run.smithi155.stderr: 95.1% -- replaced with /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mds.b.log.gz 2023-07-18T03:46:49.875 INFO:teuthology.orchestra.run.smithi155.stderr: 2023-07-18T03:46:49.875 INFO:teuthology.orchestra.run.smithi155.stderr:real 3m56.312s 2023-07-18T03:46:49.875 INFO:teuthology.orchestra.run.smithi155.stderr:user 5m20.345s 2023-07-18T03:46:49.876 INFO:teuthology.orchestra.run.smithi155.stderr:sys 0m12.870s 2023-07-18T03:50:23.095 INFO:teuthology.orchestra.run.smithi179.stderr: 93.3% -- replaced with /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mds.i.log.gz 2023-07-18T03:50:26.982 INFO:teuthology.orchestra.run.smithi179.stderr: 96.5% -- replaced with /var/log/ceph/b1782daa-2516-11ee-9b34-001a4aab830c/ceph-mds.c.log.gz 2023-07-18T03:50:26.985 INFO:teuthology.orchestra.run.smithi179.stderr: 2023-07-18T03:50:26.985 INFO:teuthology.orchestra.run.smithi179.stderr:real 7m33.423s 2023-07-18T03:50:26.985 INFO:teuthology.orchestra.run.smithi179.stderr:user 8m44.382s 2023-07-18T03:50:26.986 INFO:teuthology.orchestra.run.smithi179.stderr:sys 0m22.680s 2023-07-18T03:50:26.986 INFO:tasks.cephadm:Archiving logs... 2023-07-18T03:50:26.987 DEBUG:teuthology.misc:Transferring archived files from smithi101:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342713/remote/smithi101/log 2023-07-18T03:50:26.988 DEBUG:teuthology.orchestra.run.smithi101:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-18T03:50:47.993 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342713/remote/smithi155/log 2023-07-18T03:50:47.995 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-18T03:51:27.163 DEBUG:teuthology.misc:Transferring archived files from smithi179:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342713/remote/smithi179/log 2023-07-18T03:51:27.164 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-18T03:52:26.937 INFO:tasks.cephadm:Removing cluster... 2023-07-18T03:52:26.938 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b1782daa-2516-11ee-9b34-001a4aab830c --force 2023-07-18T03:52:28.652 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b1782daa-2516-11ee-9b34-001a4aab830c --force 2023-07-18T03:52:30.053 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b1782daa-2516-11ee-9b34-001a4aab830c --force 2023-07-18T03:52:31.268 INFO:tasks.cephadm:Removing cephadm ... 2023-07-18T03:52:31.269 DEBUG:teuthology.orchestra.run.smithi101:> rm -rf /home/ubuntu/cephtest/cephadm 2023-07-18T03:52:31.321 DEBUG:teuthology.orchestra.run.smithi155:> rm -rf /home/ubuntu/cephtest/cephadm 2023-07-18T03:52:31.416 DEBUG:teuthology.orchestra.run.smithi179:> rm -rf /home/ubuntu/cephtest/cephadm 2023-07-18T03:52:31.441 INFO:tasks.cephadm:Teardown complete 2023-07-18T03:52:31.442 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-07-18T03:52:31.458 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-07-18T03:52:31.460 DEBUG:teuthology.orchestra.run.smithi101:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-18T03:52:31.465 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-18T03:52:31.468 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-18T03:52:31.615 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-07-18T03:52:31.629 INFO:teuthology.task.clock:Checking final clock skew... 2023-07-18T03:52:31.630 DEBUG:teuthology.orchestra.run.smithi101:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-18T03:52:31.635 DEBUG:teuthology.orchestra.run.smithi155:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-18T03:52:31.638 DEBUG:teuthology.orchestra.run.smithi179:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-18T03:52:31.654 INFO:teuthology.orchestra.run.smithi179.stderr:bash: ntpq: command not found 2023-07-18T03:52:31.654 INFO:teuthology.orchestra.run.smithi155.stderr:bash: ntpq: command not found 2023-07-18T03:52:31.668 INFO:teuthology.orchestra.run.smithi179.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-18T03:52:31.669 INFO:teuthology.orchestra.run.smithi179.stdout:=============================================================================== 2023-07-18T03:52:31.669 INFO:teuthology.orchestra.run.smithi179.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 52 +3220us[+3180us] +/- 94ms 2023-07-18T03:52:31.669 INFO:teuthology.orchestra.run.smithi179.stdout:^* hv02.front.sepia.ceph.com 3 6 377 50 -3569us[-3609us] +/- 60ms 2023-07-18T03:52:31.670 INFO:teuthology.orchestra.run.smithi179.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 52 +2264us[+2225us] +/- 80ms 2023-07-18T03:52:31.670 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-07-18T03:52:31.674 INFO:teuthology.orchestra.run.smithi101.stderr:bash: ntpq: command not found 2023-07-18T03:52:31.676 INFO:teuthology.orchestra.run.smithi155.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-18T03:52:31.676 INFO:teuthology.orchestra.run.smithi155.stdout:=============================================================================== 2023-07-18T03:52:31.676 INFO:teuthology.orchestra.run.smithi155.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 51 +3236us[+3236us] +/- 94ms 2023-07-18T03:52:31.677 INFO:teuthology.orchestra.run.smithi155.stdout:^* hv02.front.sepia.ceph.com 3 6 377 53 -3610us[-3606us] +/- 60ms 2023-07-18T03:52:31.677 INFO:teuthology.orchestra.run.smithi155.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 53 +2288us[+2292us] +/- 80ms 2023-07-18T03:52:31.677 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-07-18T03:52:31.681 INFO:teuthology.orchestra.run.smithi101.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-18T03:52:31.682 INFO:teuthology.orchestra.run.smithi101.stdout:=============================================================================== 2023-07-18T03:52:31.682 INFO:teuthology.orchestra.run.smithi101.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 50 +3107us[+3107us] +/- 94ms 2023-07-18T03:52:31.682 INFO:teuthology.orchestra.run.smithi101.stdout:^* hv02.front.sepia.ceph.com 3 6 377 58 -3763us[-3719us] +/- 60ms 2023-07-18T03:52:31.682 INFO:teuthology.orchestra.run.smithi101.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 57 +2147us[+2147us] +/- 79ms 2023-07-18T03:52:31.683 INFO:teuthology.orchestra.run.smithi101.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-07-18T03:52:31.684 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-07-18T03:52:31.703 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-07-18T03:52:31.704 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-07-18T03:52:31.715 DEBUG:teuthology.orchestra.run.smithi101:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-18T03:52:31.907 DEBUG:teuthology.orchestra.run.smithi155:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-18T03:52:32.044 DEBUG:teuthology.orchestra.run.smithi179:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-18T03:52:32.228 DEBUG:teuthology.orchestra.run.smithi101:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-18T03:52:32.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-18T03:52:32.307 DEBUG:teuthology.orchestra.run.smithi155:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-18T03:52:32.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-18T03:52:32.343 DEBUG:teuthology.orchestra.run.smithi179:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-18T03:52:32.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-18T03:52:32.376 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-07-18T03:52:32.399 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-07-18T03:52:32.421 INFO:teuthology.task.internal:Duration was 4606.909635 seconds 2023-07-18T03:52:32.422 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-07-18T03:52:32.433 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-07-18T03:52:32.433 DEBUG:teuthology.orchestra.run.smithi101:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-18T03:52:32.438 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-18T03:52:32.441 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-18T03:52:32.519 INFO:teuthology.orchestra.run.smithi155.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-18T03:52:32.523 INFO:teuthology.orchestra.run.smithi179.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-18T03:52:32.551 INFO:teuthology.orchestra.run.smithi101.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-18T03:52:32.831 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-07-18T03:52:32.832 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi101.front.sepia.ceph.com 2023-07-18T03:52:32.832 DEBUG:teuthology.orchestra.run.smithi101:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-18T03:52:32.892 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi155.front.sepia.ceph.com 2023-07-18T03:52:32.893 DEBUG:teuthology.orchestra.run.smithi155:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-18T03:52:32.925 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi179.front.sepia.ceph.com 2023-07-18T03:52:32.926 DEBUG:teuthology.orchestra.run.smithi179:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-18T03:52:32.967 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-07-18T03:52:32.968 DEBUG:teuthology.orchestra.run.smithi101:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-18T03:52:32.972 DEBUG:teuthology.orchestra.run.smithi155:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-18T03:52:32.975 DEBUG:teuthology.orchestra.run.smithi179:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-18T03:52:33.089 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-07-18T03:52:33.090 DEBUG:teuthology.orchestra.run.smithi101:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-18T03:52:33.094 DEBUG:teuthology.orchestra.run.smithi155:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-18T03:52:33.098 DEBUG:teuthology.orchestra.run.smithi179:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-18T03:52:33.270 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-07-18T03:52:33.283 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-07-18T03:52:33.284 DEBUG:teuthology.orchestra.run.smithi101:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-18T03:52:33.353 DEBUG:teuthology.orchestra.run.smithi155:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-18T03:52:33.381 DEBUG:teuthology.orchestra.run.smithi179:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-18T03:52:33.459 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-07-18T03:52:33.471 DEBUG:teuthology.orchestra.run.smithi101:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-18T03:52:33.476 DEBUG:teuthology.orchestra.run.smithi155:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-18T03:52:33.478 DEBUG:teuthology.orchestra.run.smithi179:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-18T03:52:33.505 INFO:teuthology.orchestra.run.smithi179.stdout:kernel.core_pattern = core 2023-07-18T03:52:33.523 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern = core 2023-07-18T03:52:33.538 INFO:teuthology.orchestra.run.smithi101.stdout:kernel.core_pattern = core 2023-07-18T03:52:33.577 DEBUG:teuthology.orchestra.run.smithi101:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-18T03:52:33.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-18T03:52:33.619 DEBUG:teuthology.orchestra.run.smithi155:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-18T03:52:33.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-18T03:52:33.635 DEBUG:teuthology.orchestra.run.smithi179:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-18T03:52:33.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-18T03:52:33.652 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-07-18T03:52:33.667 INFO:teuthology.task.internal:Transferring archived files... 2023-07-18T03:52:33.669 DEBUG:teuthology.misc:Transferring archived files from smithi101:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342713/remote/smithi101 2023-07-18T03:52:33.670 DEBUG:teuthology.orchestra.run.smithi101:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-18T03:52:33.751 DEBUG:teuthology.misc:Transferring archived files from smithi155:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342713/remote/smithi155 2023-07-18T03:52:33.752 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-18T03:52:33.788 DEBUG:teuthology.misc:Transferring archived files from smithi179:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342713/remote/smithi179 2023-07-18T03:52:33.789 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-18T03:52:33.826 INFO:teuthology.task.internal:Removing archive directory... 2023-07-18T03:52:33.826 DEBUG:teuthology.orchestra.run.smithi101:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-18T03:52:33.831 DEBUG:teuthology.orchestra.run.smithi155:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-18T03:52:33.833 DEBUG:teuthology.orchestra.run.smithi179:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-18T03:52:33.878 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-07-18T03:52:33.896 INFO:teuthology.task.internal:Not uploading archives. 2023-07-18T03:52:33.896 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-07-18T03:52:33.941 INFO:teuthology.task.internal:Tidying up after the test... 2023-07-18T03:52:33.941 DEBUG:teuthology.orchestra.run.smithi101:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-18T03:52:33.947 DEBUG:teuthology.orchestra.run.smithi155:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-18T03:52:33.955 DEBUG:teuthology.orchestra.run.smithi179:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-18T03:52:33.968 INFO:teuthology.orchestra.run.smithi155.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 18 03:52 /home/ubuntu/cephtest 2023-07-18T03:52:33.972 INFO:teuthology.orchestra.run.smithi179.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 18 03:52 /home/ubuntu/cephtest 2023-07-18T03:52:33.986 INFO:teuthology.orchestra.run.smithi101.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 18 03:52 /home/ubuntu/cephtest 2023-07-18T03:52:33.989 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-07-18T03:52:34.003 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-07-18T03:52:34.033 INFO:teuthology.nuke:Checking targets against current locks 2023-07-18T03:52:34.064 DEBUG:teuthology.nuke:shortname: smithi101 2023-07-18T03:52:34.065 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-18T03:52:34.097 DEBUG:teuthology.nuke:shortname: smithi155 2023-07-18T03:52:34.098 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-18T03:52:34.102 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi101.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342713', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-18 02:24:47.806997', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-07-18T03:52:34.123 DEBUG:teuthology.nuke:shortname: smithi179 2023-07-18T03:52:34.124 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-18T03:52:34.132 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi155.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342713', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-18 02:24:47.809202', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-18T03:52:34.146 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi179.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342713', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-18 02:24:47.810498', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-07-18T03:52:34.163 INFO:teuthology.orchestra.console:Power off smithi101 2023-07-18T03:52:34.165 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-18T03:52:34.188 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-18T03:52:34.212 INFO:teuthology.orchestra.console:Power off smithi155 2023-07-18T03:52:34.213 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-18T03:52:34.233 INFO:teuthology.orchestra.console:Power off smithi179 2023-07-18T03:52:34.234 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-18T03:52:34.246 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-18T03:52:34.260 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-18T03:52:38.247 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-18T03:52:38.279 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-18T03:52:38.296 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-18T03:52:42.360 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-18T03:52:42.398 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-18T03:52:42.413 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-18T03:52:42.575 INFO:teuthology.orchestra.console:Power off for smithi101 completed 2023-07-18T03:52:42.611 INFO:teuthology.orchestra.console:Power off for smithi155 completed 2023-07-18T03:52:42.628 INFO:teuthology.orchestra.console:Power off for smithi179 completed 2023-07-18T03:52:42.735 INFO:teuthology.run:Summary data: client.0-kernel-sha1: a440c79eee0258ae07e4988d8690fe6187b8b7a3 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/dbench}} duration: 4606.909635066986 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi101 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=81cf785aa7781c4e08381c540f890cb9bc987ef9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=d487ea3f80bd4a6fbe3371da4256d74e status: fail success: false 2023-07-18T03:52:42.736 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-18T03:52:42.821 INFO:teuthology.run:FAIL